[==========] Running 8 tests from 2 test suites.
[----------] Global test environment set-up.
[----------] 6 tests from LeaderRebalancerTest
[ RUN      ] LeaderRebalancerTest.FunctionalTestForDivided
WARNING: Logging before InitGoogleLogging() is written to STDERR
I20260709 13:18:13.972867 15046 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.14.177.190:43613
I20260709 13:18:13.974434 15046 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260709 13:18:13.975452 15046 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:18:13.992157 15055 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:18:13.992231 15053 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:18:13.992183 15052 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:18:15.196916 15054 instance_detector.cc:116] could not retrieve GCE instance metadata: Timed out: curl timeout: Timeout was reached: Connection time-out
I20260709 13:18:15.197072 15046 server_base.cc:1056] Not found: could not retrieve instance metadata: unable to detect cloud type of this node, probably running in non-cloud environment
I20260709 13:18:15.200726 15046 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:18:15.200997 15046 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:18:15.201215 15046 hybrid_clock.cc:648] HybridClock initialized: now 1783603095201187 us; error 0 us; skew 500 ppm
I20260709 13:18:15.202049 15046 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:18:15.208601 15046 webserver.cc:533] Webserver started at http://127.14.177.190:35271/ using document root <none> and password file <none>
I20260709 13:18:15.209501 15046 fs_manager.cc:362] Metadata directory not provided
I20260709 13:18:15.209744 15046 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:18:15.210212 15046 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:18:15.216143 15046 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskdH_Ma7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783603093801613-15046-0/minicluster-data/master-0-root/instance:
uuid: "a1334aae5957436c903bbb9ed03c2525"
format_stamp: "Formatted at 2026-07-09 13:18:15 on dist-test-slave-v5z2"
I20260709 13:18:15.224223 15046 fs_manager.cc:696] Time spent creating directory manager: real 0.007s	user 0.006s	sys 0.001s
I20260709 13:18:15.229791 15061 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:18:15.230998 15046 fs_manager.cc:730] Time spent opening block manager: real 0.004s	user 0.005s	sys 0.001s
I20260709 13:18:15.231426 15046 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskdH_Ma7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783603093801613-15046-0/minicluster-data/master-0-root
uuid: "a1334aae5957436c903bbb9ed03c2525"
format_stamp: "Formatted at 2026-07-09 13:18:15 on dist-test-slave-v5z2"
I20260709 13:18:15.231797 15046 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdH_Ma7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783603093801613-15046-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskdH_Ma7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783603093801613-15046-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskdH_Ma7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783603093801613-15046-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:18:15.292400 15046 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:18:15.294167 15046 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260709 13:18:15.294693 15046 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:18:15.373906 15046 rpc_server.cc:307] RPC server started. Bound to: 127.14.177.190:43613
I20260709 13:18:15.373974 15112 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.14.177.190:43613 every 8 connection(s)
I20260709 13:18:15.382261 15113 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:15.404458 15113 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a1334aae5957436c903bbb9ed03c2525: Bootstrap starting.
I20260709 13:18:15.410422 15113 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a1334aae5957436c903bbb9ed03c2525: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:15.412356 15113 log.cc:826] T 00000000000000000000000000000000 P a1334aae5957436c903bbb9ed03c2525: Log is configured to *not* fsync() on all Append() calls
I20260709 13:18:15.416982 15113 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a1334aae5957436c903bbb9ed03c2525: No bootstrap required, opened a new log
I20260709 13:18:15.437371 15113 raft_consensus.cc:359] T 00000000000000000000000000000000 P a1334aae5957436c903bbb9ed03c2525 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a1334aae5957436c903bbb9ed03c2525" member_type: VOTER }
I20260709 13:18:15.437920 15113 raft_consensus.cc:385] T 00000000000000000000000000000000 P a1334aae5957436c903bbb9ed03c2525 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:15.438140 15113 raft_consensus.cc:740] T 00000000000000000000000000000000 P a1334aae5957436c903bbb9ed03c2525 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a1334aae5957436c903bbb9ed03c2525, State: Initialized, Role: FOLLOWER
I20260709 13:18:15.439059 15113 consensus_queue.cc:260] T 00000000000000000000000000000000 P a1334aae5957436c903bbb9ed03c2525 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a1334aae5957436c903bbb9ed03c2525" member_type: VOTER }
I20260709 13:18:15.439543 15113 raft_consensus.cc:399] T 00000000000000000000000000000000 P a1334aae5957436c903bbb9ed03c2525 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 13:18:15.439779 15113 raft_consensus.cc:493] T 00000000000000000000000000000000 P a1334aae5957436c903bbb9ed03c2525 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 13:18:15.440073 15113 raft_consensus.cc:3060] T 00000000000000000000000000000000 P a1334aae5957436c903bbb9ed03c2525 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:15.444226 15113 raft_consensus.cc:515] T 00000000000000000000000000000000 P a1334aae5957436c903bbb9ed03c2525 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a1334aae5957436c903bbb9ed03c2525" member_type: VOTER }
I20260709 13:18:15.444988 15113 leader_election.cc:304] T 00000000000000000000000000000000 P a1334aae5957436c903bbb9ed03c2525 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a1334aae5957436c903bbb9ed03c2525; no voters: 
I20260709 13:18:15.446789 15113 leader_election.cc:290] T 00000000000000000000000000000000 P a1334aae5957436c903bbb9ed03c2525 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 13:18:15.447276 15116 raft_consensus.cc:2804] T 00000000000000000000000000000000 P a1334aae5957436c903bbb9ed03c2525 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:18:15.449972 15116 raft_consensus.cc:697] T 00000000000000000000000000000000 P a1334aae5957436c903bbb9ed03c2525 [term 1 LEADER]: Becoming Leader. State: Replica: a1334aae5957436c903bbb9ed03c2525, State: Running, Role: LEADER
I20260709 13:18:15.450935 15116 consensus_queue.cc:237] T 00000000000000000000000000000000 P a1334aae5957436c903bbb9ed03c2525 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a1334aae5957436c903bbb9ed03c2525" member_type: VOTER }
I20260709 13:18:15.452103 15113 sys_catalog.cc:565] T 00000000000000000000000000000000 P a1334aae5957436c903bbb9ed03c2525 [sys.catalog]: configured and running, proceeding with master startup.
I20260709 13:18:15.462325 15117 sys_catalog.cc:455] T 00000000000000000000000000000000 P a1334aae5957436c903bbb9ed03c2525 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a1334aae5957436c903bbb9ed03c2525" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a1334aae5957436c903bbb9ed03c2525" member_type: VOTER } }
I20260709 13:18:15.462112 15118 sys_catalog.cc:455] T 00000000000000000000000000000000 P a1334aae5957436c903bbb9ed03c2525 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a1334aae5957436c903bbb9ed03c2525. Latest consensus state: current_term: 1 leader_uuid: "a1334aae5957436c903bbb9ed03c2525" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a1334aae5957436c903bbb9ed03c2525" member_type: VOTER } }
I20260709 13:18:15.463007 15118 sys_catalog.cc:458] T 00000000000000000000000000000000 P a1334aae5957436c903bbb9ed03c2525 [sys.catalog]: This master's current role is: LEADER
I20260709 13:18:15.462968 15117 sys_catalog.cc:458] T 00000000000000000000000000000000 P a1334aae5957436c903bbb9ed03c2525 [sys.catalog]: This master's current role is: LEADER
I20260709 13:18:15.466329 15125 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 13:18:15.478210 15125 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 13:18:15.482259 15046 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 13:18:15.495903 15125 catalog_manager.cc:1383] Generated new cluster ID: 63fe7b7657264f66b0b5dfe6354541bf
I20260709 13:18:15.496218 15125 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 13:18:15.509883 15125 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 13:18:15.511440 15125 catalog_manager.cc:1540] Loading token signing keys...
I20260709 13:18:15.532658 15125 catalog_manager.cc:6070] T 00000000000000000000000000000000 P a1334aae5957436c903bbb9ed03c2525: Generated new TSK 0
I20260709 13:18:15.533658 15125 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 13:18:15.548943 15046 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:18:15.557554 15135 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:18:15.559121 15136 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:18:15.562332 15138 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:18:15.562462 15046 server_base.cc:1061] running on GCE node
I20260709 13:18:15.563468 15046 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:18:15.563751 15046 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:18:15.563971 15046 hybrid_clock.cc:648] HybridClock initialized: now 1783603095563954 us; error 0 us; skew 500 ppm
I20260709 13:18:15.564814 15046 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:18:15.568929 15046 webserver.cc:533] Webserver started at http://127.14.177.129:43433/ using document root <none> and password file <none>
I20260709 13:18:15.569638 15046 fs_manager.cc:362] Metadata directory not provided
I20260709 13:18:15.569890 15046 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:18:15.570227 15046 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:18:15.571864 15046 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskdH_Ma7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783603093801613-15046-0/minicluster-data/ts-0-root/instance:
uuid: "fd44949e40b247f980ba14623ba0dd34"
format_stamp: "Formatted at 2026-07-09 13:18:15 on dist-test-slave-v5z2"
I20260709 13:18:15.578982 15046 fs_manager.cc:696] Time spent creating directory manager: real 0.006s	user 0.006s	sys 0.001s
I20260709 13:18:15.584227 15143 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:18:15.585129 15046 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.002s
I20260709 13:18:15.585515 15046 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskdH_Ma7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783603093801613-15046-0/minicluster-data/ts-0-root
uuid: "fd44949e40b247f980ba14623ba0dd34"
format_stamp: "Formatted at 2026-07-09 13:18:15 on dist-test-slave-v5z2"
I20260709 13:18:15.585901 15046 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdH_Ma7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783603093801613-15046-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskdH_Ma7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783603093801613-15046-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskdH_Ma7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783603093801613-15046-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:18:15.612185 15046 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:18:15.613983 15046 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:18:15.617067 15046 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:18:15.623025 15046 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:18:15.623427 15046 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.001s	sys 0.000s
I20260709 13:18:15.623759 15046 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:18:15.623984 15046 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:18:15.733724 15046 rpc_server.cc:307] RPC server started. Bound to: 127.14.177.129:37801
I20260709 13:18:15.733822 15205 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.14.177.129:37801 every 8 connection(s)
I20260709 13:18:15.741708 15046 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:18:15.756595 15210 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:18:15.757741 15211 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:18:15.765724 15213 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:18:15.766280 15046 server_base.cc:1061] running on GCE node
I20260709 13:18:15.767300 15046 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:18:15.767561 15046 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:18:15.767758 15046 hybrid_clock.cc:648] HybridClock initialized: now 1783603095767740 us; error 0 us; skew 500 ppm
I20260709 13:18:15.768544 15046 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:18:15.771705 15046 webserver.cc:533] Webserver started at http://127.14.177.130:37665/ using document root <none> and password file <none>
I20260709 13:18:15.772267 15206 heartbeater.cc:344] Connected to a master server at 127.14.177.190:43613
I20260709 13:18:15.772367 15046 fs_manager.cc:362] Metadata directory not provided
I20260709 13:18:15.772653 15046 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:18:15.772748 15206 heartbeater.cc:461] Registering TS with master...
I20260709 13:18:15.772996 15046 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:18:15.773833 15206 heartbeater.cc:507] Master 127.14.177.190:43613 requested a full tablet report, sending...
I20260709 13:18:15.774506 15046 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskdH_Ma7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783603093801613-15046-0/minicluster-data/ts-1-root/instance:
uuid: "12806407b57c4ffbbf5f5baf439af582"
format_stamp: "Formatted at 2026-07-09 13:18:15 on dist-test-slave-v5z2"
I20260709 13:18:15.776897 15078 ts_manager.cc:194] Registered new tserver with Master: fd44949e40b247f980ba14623ba0dd34 (127.14.177.129:37801)
I20260709 13:18:15.779268 15078 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:55444
I20260709 13:18:15.781544 15046 fs_manager.cc:696] Time spent creating directory manager: real 0.006s	user 0.007s	sys 0.002s
I20260709 13:18:15.786877 15218 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:18:15.787819 15046 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.000s
I20260709 13:18:15.788144 15046 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskdH_Ma7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783603093801613-15046-0/minicluster-data/ts-1-root
uuid: "12806407b57c4ffbbf5f5baf439af582"
format_stamp: "Formatted at 2026-07-09 13:18:15 on dist-test-slave-v5z2"
I20260709 13:18:15.788527 15046 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdH_Ma7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783603093801613-15046-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskdH_Ma7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783603093801613-15046-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskdH_Ma7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783603093801613-15046-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:18:15.810619 15046 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:18:15.812354 15046 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:18:15.815032 15046 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:18:15.818816 15046 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:18:15.819128 15046 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:18:15.819458 15046 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:18:15.819674 15046 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:18:15.909888 15046 rpc_server.cc:307] RPC server started. Bound to: 127.14.177.130:45327
I20260709 13:18:15.910058 15280 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.14.177.130:45327 every 8 connection(s)
I20260709 13:18:15.920264 15046 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:18:15.930150 15284 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:18:15.931869 15285 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:18:15.933912 15281 heartbeater.cc:344] Connected to a master server at 127.14.177.190:43613
I20260709 13:18:15.934402 15281 heartbeater.cc:461] Registering TS with master...
I20260709 13:18:15.935739 15281 heartbeater.cc:507] Master 127.14.177.190:43613 requested a full tablet report, sending...
I20260709 13:18:15.943552 15046 server_base.cc:1061] running on GCE node
I20260709 13:18:15.943761 15078 ts_manager.cc:194] Registered new tserver with Master: 12806407b57c4ffbbf5f5baf439af582 (127.14.177.130:45327)
W20260709 13:18:15.945191 15287 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:18:15.945622 15078 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:55446
I20260709 13:18:15.946357 15046 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:18:15.946666 15046 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:18:15.946887 15046 hybrid_clock.cc:648] HybridClock initialized: now 1783603095946866 us; error 0 us; skew 500 ppm
I20260709 13:18:15.947584 15046 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:18:15.950280 15046 webserver.cc:533] Webserver started at http://127.14.177.131:41645/ using document root <none> and password file <none>
I20260709 13:18:15.950824 15046 fs_manager.cc:362] Metadata directory not provided
I20260709 13:18:15.951085 15046 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:18:15.951454 15046 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:18:15.952672 15046 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskdH_Ma7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783603093801613-15046-0/minicluster-data/ts-2-root/instance:
uuid: "ce1dd5b4e0014ba8a3fa68f2ba226e49"
format_stamp: "Formatted at 2026-07-09 13:18:15 on dist-test-slave-v5z2"
I20260709 13:18:15.958073 15046 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.006s	sys 0.000s
I20260709 13:18:15.961879 15292 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:18:15.962903 15046 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.002s	sys 0.000s
I20260709 13:18:15.963312 15046 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskdH_Ma7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783603093801613-15046-0/minicluster-data/ts-2-root
uuid: "ce1dd5b4e0014ba8a3fa68f2ba226e49"
format_stamp: "Formatted at 2026-07-09 13:18:15 on dist-test-slave-v5z2"
I20260709 13:18:15.963693 15046 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdH_Ma7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783603093801613-15046-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskdH_Ma7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783603093801613-15046-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskdH_Ma7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783603093801613-15046-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:18:15.994518 15046 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:18:15.996244 15046 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:18:15.997968 15046 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:18:16.001749 15046 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:18:16.002036 15046 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:18:16.002322 15046 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:18:16.002535 15046 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:18:16.089079 15046 rpc_server.cc:307] RPC server started. Bound to: 127.14.177.131:34123
I20260709 13:18:16.089181 15354 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.14.177.131:34123 every 8 connection(s)
I20260709 13:18:16.106309 15355 heartbeater.cc:344] Connected to a master server at 127.14.177.190:43613
I20260709 13:18:16.106705 15355 heartbeater.cc:461] Registering TS with master...
I20260709 13:18:16.107589 15355 heartbeater.cc:507] Master 127.14.177.190:43613 requested a full tablet report, sending...
I20260709 13:18:16.109633 15078 ts_manager.cc:194] Registered new tserver with Master: ce1dd5b4e0014ba8a3fa68f2ba226e49 (127.14.177.131:34123)
I20260709 13:18:16.110579 15046 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.013045154s
I20260709 13:18:16.111229 15078 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:55450
I20260709 13:18:16.172821 15078 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:55454:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q""\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 13:18:16.177912 15078 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 13:18:16.313086 15171 tablet_service.cc:1511] Processing CreateTablet for tablet 54dc07b181bd495d8c8e2cae81240f16 (DEFAULT_TABLE table=test-workload [id=7cec0d216439484ca585665b2ec71341]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:18:16.313536 15170 tablet_service.cc:1511] Processing CreateTablet for tablet 28ac227079274b8eb6a3e35e4bf34e30 (DEFAULT_TABLE table=test-workload [id=7cec0d216439484ca585665b2ec71341]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:18:16.314468 15171 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 54dc07b181bd495d8c8e2cae81240f16. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:16.315790 15170 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 28ac227079274b8eb6a3e35e4bf34e30. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:16.315644 15169 tablet_service.cc:1511] Processing CreateTablet for tablet 58a8c0164db24ffd8e992493c228804c (DEFAULT_TABLE table=test-workload [id=7cec0d216439484ca585665b2ec71341]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:18:16.317351 15169 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 58a8c0164db24ffd8e992493c228804c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:16.334810 15165 tablet_service.cc:1511] Processing CreateTablet for tablet 81fce322177440a690ee6f46d2d67ad6 (DEFAULT_TABLE table=test-workload [id=7cec0d216439484ca585665b2ec71341]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:18:16.328435 15166 tablet_service.cc:1511] Processing CreateTablet for tablet 8391f4ec0b1349f4a1758a0ce2382b7c (DEFAULT_TABLE table=test-workload [id=7cec0d216439484ca585665b2ec71341]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:18:16.336319 15320 tablet_service.cc:1511] Processing CreateTablet for tablet 54dc07b181bd495d8c8e2cae81240f16 (DEFAULT_TABLE table=test-workload [id=7cec0d216439484ca585665b2ec71341]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:18:16.334889 15164 tablet_service.cc:1511] Processing CreateTablet for tablet 853d00d5717a4ba1a941a81884cbb7a2 (DEFAULT_TABLE table=test-workload [id=7cec0d216439484ca585665b2ec71341]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:18:16.331389 15167 tablet_service.cc:1511] Processing CreateTablet for tablet 6f3b305de2a04b54aef8674096fa6358 (DEFAULT_TABLE table=test-workload [id=7cec0d216439484ca585665b2ec71341]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:18:16.333124 15319 tablet_service.cc:1511] Processing CreateTablet for tablet 28ac227079274b8eb6a3e35e4bf34e30 (DEFAULT_TABLE table=test-workload [id=7cec0d216439484ca585665b2ec71341]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:18:16.343361 15320 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 54dc07b181bd495d8c8e2cae81240f16. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:16.339686 15168 tablet_service.cc:1511] Processing CreateTablet for tablet 98f6aef87e78472c9987c75cac6575af (DEFAULT_TABLE table=test-workload [id=7cec0d216439484ca585665b2ec71341]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:18:16.349004 15166 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8391f4ec0b1349f4a1758a0ce2382b7c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:16.356514 15319 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 28ac227079274b8eb6a3e35e4bf34e30. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:16.343384 15318 tablet_service.cc:1511] Processing CreateTablet for tablet 58a8c0164db24ffd8e992493c228804c (DEFAULT_TABLE table=test-workload [id=7cec0d216439484ca585665b2ec71341]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:18:16.346585 15163 tablet_service.cc:1511] Processing CreateTablet for tablet fbf069cd9abc4875a3d03f06ff06544e (DEFAULT_TABLE table=test-workload [id=7cec0d216439484ca585665b2ec71341]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:18:16.362558 15316 tablet_service.cc:1511] Processing CreateTablet for tablet 6f3b305de2a04b54aef8674096fa6358 (DEFAULT_TABLE table=test-workload [id=7cec0d216439484ca585665b2ec71341]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:18:16.364427 15315 tablet_service.cc:1511] Processing CreateTablet for tablet 8391f4ec0b1349f4a1758a0ce2382b7c (DEFAULT_TABLE table=test-workload [id=7cec0d216439484ca585665b2ec71341]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:18:16.367395 15318 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 58a8c0164db24ffd8e992493c228804c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:16.365689 15317 tablet_service.cc:1511] Processing CreateTablet for tablet 98f6aef87e78472c9987c75cac6575af (DEFAULT_TABLE table=test-workload [id=7cec0d216439484ca585665b2ec71341]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:18:16.367434 15313 tablet_service.cc:1511] Processing CreateTablet for tablet 853d00d5717a4ba1a941a81884cbb7a2 (DEFAULT_TABLE table=test-workload [id=7cec0d216439484ca585665b2ec71341]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:18:16.369069 15314 tablet_service.cc:1511] Processing CreateTablet for tablet 81fce322177440a690ee6f46d2d67ad6 (DEFAULT_TABLE table=test-workload [id=7cec0d216439484ca585665b2ec71341]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:18:16.369974 15312 tablet_service.cc:1511] Processing CreateTablet for tablet fbf069cd9abc4875a3d03f06ff06544e (DEFAULT_TABLE table=test-workload [id=7cec0d216439484ca585665b2ec71341]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:18:16.374475 15316 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6f3b305de2a04b54aef8674096fa6358. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:16.376997 15314 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 81fce322177440a690ee6f46d2d67ad6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:16.379674 15317 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 98f6aef87e78472c9987c75cac6575af. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:16.381537 15312 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fbf069cd9abc4875a3d03f06ff06544e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:16.386780 15313 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 853d00d5717a4ba1a941a81884cbb7a2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:16.390740 15165 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 81fce322177440a690ee6f46d2d67ad6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:16.392324 15315 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8391f4ec0b1349f4a1758a0ce2382b7c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:16.395030 15168 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 98f6aef87e78472c9987c75cac6575af. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:16.396049 15167 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6f3b305de2a04b54aef8674096fa6358. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:16.399509 15163 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fbf069cd9abc4875a3d03f06ff06544e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:16.415467 15164 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 853d00d5717a4ba1a941a81884cbb7a2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:16.449851 15246 tablet_service.cc:1511] Processing CreateTablet for tablet 54dc07b181bd495d8c8e2cae81240f16 (DEFAULT_TABLE table=test-workload [id=7cec0d216439484ca585665b2ec71341]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:18:16.453325 15243 tablet_service.cc:1511] Processing CreateTablet for tablet 98f6aef87e78472c9987c75cac6575af (DEFAULT_TABLE table=test-workload [id=7cec0d216439484ca585665b2ec71341]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:18:16.455857 15246 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 54dc07b181bd495d8c8e2cae81240f16. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:16.452361 15244 tablet_service.cc:1511] Processing CreateTablet for tablet 58a8c0164db24ffd8e992493c228804c (DEFAULT_TABLE table=test-workload [id=7cec0d216439484ca585665b2ec71341]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:18:16.445130 15245 tablet_service.cc:1511] Processing CreateTablet for tablet 28ac227079274b8eb6a3e35e4bf34e30 (DEFAULT_TABLE table=test-workload [id=7cec0d216439484ca585665b2ec71341]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:18:16.473536 15243 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 98f6aef87e78472c9987c75cac6575af. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:16.475348 15242 tablet_service.cc:1511] Processing CreateTablet for tablet 6f3b305de2a04b54aef8674096fa6358 (DEFAULT_TABLE table=test-workload [id=7cec0d216439484ca585665b2ec71341]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:18:16.474130 15240 tablet_service.cc:1511] Processing CreateTablet for tablet 81fce322177440a690ee6f46d2d67ad6 (DEFAULT_TABLE table=test-workload [id=7cec0d216439484ca585665b2ec71341]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:18:16.487550 15239 tablet_service.cc:1511] Processing CreateTablet for tablet 853d00d5717a4ba1a941a81884cbb7a2 (DEFAULT_TABLE table=test-workload [id=7cec0d216439484ca585665b2ec71341]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:18:16.482689 15238 tablet_service.cc:1511] Processing CreateTablet for tablet fbf069cd9abc4875a3d03f06ff06544e (DEFAULT_TABLE table=test-workload [id=7cec0d216439484ca585665b2ec71341]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:18:16.484658 15241 tablet_service.cc:1511] Processing CreateTablet for tablet 8391f4ec0b1349f4a1758a0ce2382b7c (DEFAULT_TABLE table=test-workload [id=7cec0d216439484ca585665b2ec71341]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:18:16.492612 15245 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 28ac227079274b8eb6a3e35e4bf34e30. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:18:16.494696 15126 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:18:16.498741 15238 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fbf069cd9abc4875a3d03f06ff06544e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:16.500595 15239 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 853d00d5717a4ba1a941a81884cbb7a2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:16.508409 15240 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 81fce322177440a690ee6f46d2d67ad6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:16.535121 15244 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 58a8c0164db24ffd8e992493c228804c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:16.561110 15241 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8391f4ec0b1349f4a1758a0ce2382b7c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:16.567921 15242 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6f3b305de2a04b54aef8674096fa6358. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:18:16.587898 15064 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.CreateTablet -> {remote=127.14.177.129:37801, user_credentials={real_user=slave}} blocked reactor thread for 103541us
I20260709 13:18:16.668623 15373 tablet_bootstrap.cc:492] T 28ac227079274b8eb6a3e35e4bf34e30 P fd44949e40b247f980ba14623ba0dd34: Bootstrap starting.
I20260709 13:18:16.683048 15374 tablet_bootstrap.cc:492] T 6f3b305de2a04b54aef8674096fa6358 P ce1dd5b4e0014ba8a3fa68f2ba226e49: Bootstrap starting.
I20260709 13:18:16.702698 15375 tablet_bootstrap.cc:492] T 54dc07b181bd495d8c8e2cae81240f16 P 12806407b57c4ffbbf5f5baf439af582: Bootstrap starting.
I20260709 13:18:16.706486 15373 tablet_bootstrap.cc:654] T 28ac227079274b8eb6a3e35e4bf34e30 P fd44949e40b247f980ba14623ba0dd34: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:16.731763 15374 tablet_bootstrap.cc:654] T 6f3b305de2a04b54aef8674096fa6358 P ce1dd5b4e0014ba8a3fa68f2ba226e49: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:16.733793 15375 tablet_bootstrap.cc:654] T 54dc07b181bd495d8c8e2cae81240f16 P 12806407b57c4ffbbf5f5baf439af582: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:16.734652 15373 tablet_bootstrap.cc:492] T 28ac227079274b8eb6a3e35e4bf34e30 P fd44949e40b247f980ba14623ba0dd34: No bootstrap required, opened a new log
I20260709 13:18:16.735193 15373 ts_tablet_manager.cc:1403] T 28ac227079274b8eb6a3e35e4bf34e30 P fd44949e40b247f980ba14623ba0dd34: Time spent bootstrapping tablet: real 0.120s	user 0.009s	sys 0.006s
I20260709 13:18:16.738760 15373 raft_consensus.cc:359] T 28ac227079274b8eb6a3e35e4bf34e30 P fd44949e40b247f980ba14623ba0dd34 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd44949e40b247f980ba14623ba0dd34" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 37801 } } peers { permanent_uuid: "12806407b57c4ffbbf5f5baf439af582" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 45327 } } peers { permanent_uuid: "ce1dd5b4e0014ba8a3fa68f2ba226e49" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34123 } }
I20260709 13:18:16.739965 15373 raft_consensus.cc:385] T 28ac227079274b8eb6a3e35e4bf34e30 P fd44949e40b247f980ba14623ba0dd34 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:16.740376 15373 raft_consensus.cc:740] T 28ac227079274b8eb6a3e35e4bf34e30 P fd44949e40b247f980ba14623ba0dd34 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fd44949e40b247f980ba14623ba0dd34, State: Initialized, Role: FOLLOWER
I20260709 13:18:16.741545 15373 consensus_queue.cc:260] T 28ac227079274b8eb6a3e35e4bf34e30 P fd44949e40b247f980ba14623ba0dd34 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd44949e40b247f980ba14623ba0dd34" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 37801 } } peers { permanent_uuid: "12806407b57c4ffbbf5f5baf439af582" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 45327 } } peers { permanent_uuid: "ce1dd5b4e0014ba8a3fa68f2ba226e49" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34123 } }
I20260709 13:18:16.746805 15375 tablet_bootstrap.cc:492] T 54dc07b181bd495d8c8e2cae81240f16 P 12806407b57c4ffbbf5f5baf439af582: No bootstrap required, opened a new log
I20260709 13:18:16.747392 15375 ts_tablet_manager.cc:1403] T 54dc07b181bd495d8c8e2cae81240f16 P 12806407b57c4ffbbf5f5baf439af582: Time spent bootstrapping tablet: real 0.055s	user 0.008s	sys 0.011s
I20260709 13:18:16.753038 15373 ts_tablet_manager.cc:1434] T 28ac227079274b8eb6a3e35e4bf34e30 P fd44949e40b247f980ba14623ba0dd34: Time spent starting tablet: real 0.018s	user 0.009s	sys 0.002s
I20260709 13:18:16.752725 15375 raft_consensus.cc:359] T 54dc07b181bd495d8c8e2cae81240f16 P 12806407b57c4ffbbf5f5baf439af582 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce1dd5b4e0014ba8a3fa68f2ba226e49" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34123 } } peers { permanent_uuid: "fd44949e40b247f980ba14623ba0dd34" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 37801 } } peers { permanent_uuid: "12806407b57c4ffbbf5f5baf439af582" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 45327 } }
I20260709 13:18:16.753605 15375 raft_consensus.cc:385] T 54dc07b181bd495d8c8e2cae81240f16 P 12806407b57c4ffbbf5f5baf439af582 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:16.753933 15206 heartbeater.cc:499] Master 127.14.177.190:43613 was elected leader, sending a full tablet report...
I20260709 13:18:16.753979 15375 raft_consensus.cc:740] T 54dc07b181bd495d8c8e2cae81240f16 P 12806407b57c4ffbbf5f5baf439af582 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 12806407b57c4ffbbf5f5baf439af582, State: Initialized, Role: FOLLOWER
I20260709 13:18:16.754295 15373 tablet_bootstrap.cc:492] T 8391f4ec0b1349f4a1758a0ce2382b7c P fd44949e40b247f980ba14623ba0dd34: Bootstrap starting.
I20260709 13:18:16.754859 15375 consensus_queue.cc:260] T 54dc07b181bd495d8c8e2cae81240f16 P 12806407b57c4ffbbf5f5baf439af582 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce1dd5b4e0014ba8a3fa68f2ba226e49" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34123 } } peers { permanent_uuid: "fd44949e40b247f980ba14623ba0dd34" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 37801 } } peers { permanent_uuid: "12806407b57c4ffbbf5f5baf439af582" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 45327 } }
I20260709 13:18:16.755442 15374 tablet_bootstrap.cc:492] T 6f3b305de2a04b54aef8674096fa6358 P ce1dd5b4e0014ba8a3fa68f2ba226e49: No bootstrap required, opened a new log
I20260709 13:18:16.760885 15374 ts_tablet_manager.cc:1403] T 6f3b305de2a04b54aef8674096fa6358 P ce1dd5b4e0014ba8a3fa68f2ba226e49: Time spent bootstrapping tablet: real 0.107s	user 0.015s	sys 0.009s
I20260709 13:18:16.765715 15374 raft_consensus.cc:359] T 6f3b305de2a04b54aef8674096fa6358 P ce1dd5b4e0014ba8a3fa68f2ba226e49 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd44949e40b247f980ba14623ba0dd34" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 37801 } } peers { permanent_uuid: "12806407b57c4ffbbf5f5baf439af582" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 45327 } } peers { permanent_uuid: "ce1dd5b4e0014ba8a3fa68f2ba226e49" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34123 } }
I20260709 13:18:16.766757 15374 raft_consensus.cc:385] T 6f3b305de2a04b54aef8674096fa6358 P ce1dd5b4e0014ba8a3fa68f2ba226e49 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:16.767190 15374 raft_consensus.cc:740] T 6f3b305de2a04b54aef8674096fa6358 P ce1dd5b4e0014ba8a3fa68f2ba226e49 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ce1dd5b4e0014ba8a3fa68f2ba226e49, State: Initialized, Role: FOLLOWER
I20260709 13:18:16.768321 15374 consensus_queue.cc:260] T 6f3b305de2a04b54aef8674096fa6358 P ce1dd5b4e0014ba8a3fa68f2ba226e49 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd44949e40b247f980ba14623ba0dd34" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 37801 } } peers { permanent_uuid: "12806407b57c4ffbbf5f5baf439af582" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 45327 } } peers { permanent_uuid: "ce1dd5b4e0014ba8a3fa68f2ba226e49" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34123 } }
I20260709 13:18:16.772305 15281 heartbeater.cc:499] Master 127.14.177.190:43613 was elected leader, sending a full tablet report...
I20260709 13:18:16.772904 15375 ts_tablet_manager.cc:1434] T 54dc07b181bd495d8c8e2cae81240f16 P 12806407b57c4ffbbf5f5baf439af582: Time spent starting tablet: real 0.025s	user 0.012s	sys 0.012s
I20260709 13:18:16.773813 15375 tablet_bootstrap.cc:492] T 853d00d5717a4ba1a941a81884cbb7a2 P 12806407b57c4ffbbf5f5baf439af582: Bootstrap starting.
I20260709 13:18:16.780654 15375 tablet_bootstrap.cc:654] T 853d00d5717a4ba1a941a81884cbb7a2 P 12806407b57c4ffbbf5f5baf439af582: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:16.781919 15355 heartbeater.cc:499] Master 127.14.177.190:43613 was elected leader, sending a full tablet report...
I20260709 13:18:16.784160 15373 tablet_bootstrap.cc:654] T 8391f4ec0b1349f4a1758a0ce2382b7c P fd44949e40b247f980ba14623ba0dd34: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:16.792589 15381 raft_consensus.cc:493] T 6f3b305de2a04b54aef8674096fa6358 P ce1dd5b4e0014ba8a3fa68f2ba226e49 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:16.793226 15373 tablet_bootstrap.cc:492] T 8391f4ec0b1349f4a1758a0ce2382b7c P fd44949e40b247f980ba14623ba0dd34: No bootstrap required, opened a new log
I20260709 13:18:16.793108 15381 raft_consensus.cc:515] T 6f3b305de2a04b54aef8674096fa6358 P ce1dd5b4e0014ba8a3fa68f2ba226e49 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd44949e40b247f980ba14623ba0dd34" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 37801 } } peers { permanent_uuid: "12806407b57c4ffbbf5f5baf439af582" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 45327 } } peers { permanent_uuid: "ce1dd5b4e0014ba8a3fa68f2ba226e49" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34123 } }
I20260709 13:18:16.793674 15373 ts_tablet_manager.cc:1403] T 8391f4ec0b1349f4a1758a0ce2382b7c P fd44949e40b247f980ba14623ba0dd34: Time spent bootstrapping tablet: real 0.040s	user 0.007s	sys 0.007s
I20260709 13:18:16.796645 15373 raft_consensus.cc:359] T 8391f4ec0b1349f4a1758a0ce2382b7c P fd44949e40b247f980ba14623ba0dd34 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd44949e40b247f980ba14623ba0dd34" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 37801 } } peers { permanent_uuid: "12806407b57c4ffbbf5f5baf439af582" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 45327 } } peers { permanent_uuid: "ce1dd5b4e0014ba8a3fa68f2ba226e49" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34123 } }
I20260709 13:18:16.797375 15373 raft_consensus.cc:385] T 8391f4ec0b1349f4a1758a0ce2382b7c P fd44949e40b247f980ba14623ba0dd34 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:16.797737 15373 raft_consensus.cc:740] T 8391f4ec0b1349f4a1758a0ce2382b7c P fd44949e40b247f980ba14623ba0dd34 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fd44949e40b247f980ba14623ba0dd34, State: Initialized, Role: FOLLOWER
I20260709 13:18:16.798538 15373 consensus_queue.cc:260] T 8391f4ec0b1349f4a1758a0ce2382b7c P fd44949e40b247f980ba14623ba0dd34 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd44949e40b247f980ba14623ba0dd34" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 37801 } } peers { permanent_uuid: "12806407b57c4ffbbf5f5baf439af582" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 45327 } } peers { permanent_uuid: "ce1dd5b4e0014ba8a3fa68f2ba226e49" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34123 } }
I20260709 13:18:16.799682 15381 leader_election.cc:290] T 6f3b305de2a04b54aef8674096fa6358 P ce1dd5b4e0014ba8a3fa68f2ba226e49 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fd44949e40b247f980ba14623ba0dd34 (127.14.177.129:37801), 12806407b57c4ffbbf5f5baf439af582 (127.14.177.130:45327)
I20260709 13:18:16.801080 15373 ts_tablet_manager.cc:1434] T 8391f4ec0b1349f4a1758a0ce2382b7c P fd44949e40b247f980ba14623ba0dd34: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 13:18:16.802285 15373 tablet_bootstrap.cc:492] T 54dc07b181bd495d8c8e2cae81240f16 P fd44949e40b247f980ba14623ba0dd34: Bootstrap starting.
I20260709 13:18:16.811679 15373 tablet_bootstrap.cc:654] T 54dc07b181bd495d8c8e2cae81240f16 P fd44949e40b247f980ba14623ba0dd34: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:16.825022 15374 ts_tablet_manager.cc:1434] T 6f3b305de2a04b54aef8674096fa6358 P ce1dd5b4e0014ba8a3fa68f2ba226e49: Time spent starting tablet: real 0.064s	user 0.008s	sys 0.001s
I20260709 13:18:16.826488 15374 tablet_bootstrap.cc:492] T 28ac227079274b8eb6a3e35e4bf34e30 P ce1dd5b4e0014ba8a3fa68f2ba226e49: Bootstrap starting.
I20260709 13:18:16.841148 15256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f3b305de2a04b54aef8674096fa6358" candidate_uuid: "ce1dd5b4e0014ba8a3fa68f2ba226e49" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12806407b57c4ffbbf5f5baf439af582" is_pre_election: true
W20260709 13:18:16.843150 15294 leader_election.cc:343] T 6f3b305de2a04b54aef8674096fa6358 P ce1dd5b4e0014ba8a3fa68f2ba226e49 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 12806407b57c4ffbbf5f5baf439af582 (127.14.177.130:45327): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:18:16.845345 15373 tablet_bootstrap.cc:492] T 54dc07b181bd495d8c8e2cae81240f16 P fd44949e40b247f980ba14623ba0dd34: No bootstrap required, opened a new log
I20260709 13:18:16.845844 15373 ts_tablet_manager.cc:1403] T 54dc07b181bd495d8c8e2cae81240f16 P fd44949e40b247f980ba14623ba0dd34: Time spent bootstrapping tablet: real 0.044s	user 0.021s	sys 0.013s
I20260709 13:18:16.845847 15181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f3b305de2a04b54aef8674096fa6358" candidate_uuid: "ce1dd5b4e0014ba8a3fa68f2ba226e49" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fd44949e40b247f980ba14623ba0dd34" is_pre_election: true
W20260709 13:18:16.848877 15295 leader_election.cc:343] T 6f3b305de2a04b54aef8674096fa6358 P ce1dd5b4e0014ba8a3fa68f2ba226e49 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fd44949e40b247f980ba14623ba0dd34 (127.14.177.129:37801): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:18:16.849464 15373 raft_consensus.cc:359] T 54dc07b181bd495d8c8e2cae81240f16 P fd44949e40b247f980ba14623ba0dd34 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce1dd5b4e0014ba8a3fa68f2ba226e49" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34123 } } peers { permanent_uuid: "fd44949e40b247f980ba14623ba0dd34" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 37801 } } peers { permanent_uuid: "12806407b57c4ffbbf5f5baf439af582" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 45327 } }
I20260709 13:18:16.849677 15295 leader_election.cc:304] T 6f3b305de2a04b54aef8674096fa6358 P ce1dd5b4e0014ba8a3fa68f2ba226e49 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce1dd5b4e0014ba8a3fa68f2ba226e49; no voters: 12806407b57c4ffbbf5f5baf439af582, fd44949e40b247f980ba14623ba0dd34
W20260709 13:18:16.850117 15356 tablet.cc:2406] T 6f3b305de2a04b54aef8674096fa6358 P ce1dd5b4e0014ba8a3fa68f2ba226e49: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260709 13:18:16.850237 15373 raft_consensus.cc:385] T 54dc07b181bd495d8c8e2cae81240f16 P fd44949e40b247f980ba14623ba0dd34 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:16.850656 15373 raft_consensus.cc:740] T 54dc07b181bd495d8c8e2cae81240f16 P fd44949e40b247f980ba14623ba0dd34 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fd44949e40b247f980ba14623ba0dd34, State: Initialized, Role: FOLLOWER
I20260709 13:18:16.850831 15381 raft_consensus.cc:2749] T 6f3b305de2a04b54aef8674096fa6358 P ce1dd5b4e0014ba8a3fa68f2ba226e49 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:18:16.851395 15373 consensus_queue.cc:260] T 54dc07b181bd495d8c8e2cae81240f16 P fd44949e40b247f980ba14623ba0dd34 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce1dd5b4e0014ba8a3fa68f2ba226e49" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34123 } } peers { permanent_uuid: "fd44949e40b247f980ba14623ba0dd34" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 37801 } } peers { permanent_uuid: "12806407b57c4ffbbf5f5baf439af582" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 45327 } }
I20260709 13:18:16.853806 15373 ts_tablet_manager.cc:1434] T 54dc07b181bd495d8c8e2cae81240f16 P fd44949e40b247f980ba14623ba0dd34: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:18:16.854784 15373 tablet_bootstrap.cc:492] T 58a8c0164db24ffd8e992493c228804c P fd44949e40b247f980ba14623ba0dd34: Bootstrap starting.
I20260709 13:18:16.861083 15373 tablet_bootstrap.cc:654] T 58a8c0164db24ffd8e992493c228804c P fd44949e40b247f980ba14623ba0dd34: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:16.873863 15373 tablet_bootstrap.cc:492] T 58a8c0164db24ffd8e992493c228804c P fd44949e40b247f980ba14623ba0dd34: No bootstrap required, opened a new log
I20260709 13:18:16.874344 15373 ts_tablet_manager.cc:1403] T 58a8c0164db24ffd8e992493c228804c P fd44949e40b247f980ba14623ba0dd34: Time spent bootstrapping tablet: real 0.020s	user 0.016s	sys 0.000s
I20260709 13:18:16.877256 15373 raft_consensus.cc:359] T 58a8c0164db24ffd8e992493c228804c P fd44949e40b247f980ba14623ba0dd34 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd44949e40b247f980ba14623ba0dd34" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 37801 } } peers { permanent_uuid: "12806407b57c4ffbbf5f5baf439af582" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 45327 } } peers { permanent_uuid: "ce1dd5b4e0014ba8a3fa68f2ba226e49" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34123 } }
I20260709 13:18:16.878003 15373 raft_consensus.cc:385] T 58a8c0164db24ffd8e992493c228804c P fd44949e40b247f980ba14623ba0dd34 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:16.878283 15373 raft_consensus.cc:740] T 58a8c0164db24ffd8e992493c228804c P fd44949e40b247f980ba14623ba0dd34 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fd44949e40b247f980ba14623ba0dd34, State: Initialized, Role: FOLLOWER
I20260709 13:18:16.880532 15373 consensus_queue.cc:260] T 58a8c0164db24ffd8e992493c228804c P fd44949e40b247f980ba14623ba0dd34 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd44949e40b247f980ba14623ba0dd34" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 37801 } } peers { permanent_uuid: "12806407b57c4ffbbf5f5baf439af582" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 45327 } } peers { permanent_uuid: "ce1dd5b4e0014ba8a3fa68f2ba226e49" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34123 } }
I20260709 13:18:16.883865 15373 ts_tablet_manager.cc:1434] T 58a8c0164db24ffd8e992493c228804c P fd44949e40b247f980ba14623ba0dd34: Time spent starting tablet: real 0.009s	user 0.008s	sys 0.000s
I20260709 13:18:16.884840 15373 tablet_bootstrap.cc:492] T 81fce322177440a690ee6f46d2d67ad6 P fd44949e40b247f980ba14623ba0dd34: Bootstrap starting.
I20260709 13:18:16.891692 15373 tablet_bootstrap.cc:654] T 81fce322177440a690ee6f46d2d67ad6 P fd44949e40b247f980ba14623ba0dd34: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:16.903024 15373 tablet_bootstrap.cc:492] T 81fce322177440a690ee6f46d2d67ad6 P fd44949e40b247f980ba14623ba0dd34: No bootstrap required, opened a new log
I20260709 13:18:16.903550 15373 ts_tablet_manager.cc:1403] T 81fce322177440a690ee6f46d2d67ad6 P fd44949e40b247f980ba14623ba0dd34: Time spent bootstrapping tablet: real 0.019s	user 0.013s	sys 0.005s
I20260709 13:18:16.906505 15379 raft_consensus.cc:493] T 58a8c0164db24ffd8e992493c228804c P fd44949e40b247f980ba14623ba0dd34 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:16.907006 15379 raft_consensus.cc:515] T 58a8c0164db24ffd8e992493c228804c P fd44949e40b247f980ba14623ba0dd34 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd44949e40b247f980ba14623ba0dd34" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 37801 } } peers { permanent_uuid: "12806407b57c4ffbbf5f5baf439af582" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 45327 } } peers { permanent_uuid: "ce1dd5b4e0014ba8a3fa68f2ba226e49" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34123 } }
I20260709 13:18:16.912499 15379 leader_election.cc:290] T 58a8c0164db24ffd8e992493c228804c P fd44949e40b247f980ba14623ba0dd34 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 12806407b57c4ffbbf5f5baf439af582 (127.14.177.130:45327), ce1dd5b4e0014ba8a3fa68f2ba226e49 (127.14.177.131:34123)
I20260709 13:18:16.921101 15373 raft_consensus.cc:359] T 81fce322177440a690ee6f46d2d67ad6 P fd44949e40b247f980ba14623ba0dd34 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd44949e40b247f980ba14623ba0dd34" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 37801 } } peers { permanent_uuid: "12806407b57c4ffbbf5f5baf439af582" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 45327 } } peers { permanent_uuid: "ce1dd5b4e0014ba8a3fa68f2ba226e49" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34123 } }
I20260709 13:18:16.927489 15373 raft_consensus.cc:385] T 81fce322177440a690ee6f46d2d67ad6 P fd44949e40b247f980ba14623ba0dd34 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:16.935940 15374 tablet_bootstrap.cc:654] T 28ac227079274b8eb6a3e35e4bf34e30 P ce1dd5b4e0014ba8a3fa68f2ba226e49: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:16.927826 15373 raft_consensus.cc:740] T 81fce322177440a690ee6f46d2d67ad6 P fd44949e40b247f980ba14623ba0dd34 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fd44949e40b247f980ba14623ba0dd34, State: Initialized, Role: FOLLOWER
I20260709 13:18:16.938382 15373 consensus_queue.cc:260] T 81fce322177440a690ee6f46d2d67ad6 P fd44949e40b247f980ba14623ba0dd34 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd44949e40b247f980ba14623ba0dd34" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 37801 } } peers { permanent_uuid: "12806407b57c4ffbbf5f5baf439af582" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 45327 } } peers { permanent_uuid: "ce1dd5b4e0014ba8a3fa68f2ba226e49" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34123 } }
I20260709 13:18:16.944612 15373 ts_tablet_manager.cc:1434] T 81fce322177440a690ee6f46d2d67ad6 P fd44949e40b247f980ba14623ba0dd34: Time spent starting tablet: real 0.041s	user 0.003s	sys 0.003s
I20260709 13:18:16.946830 15373 tablet_bootstrap.cc:492] T 98f6aef87e78472c9987c75cac6575af P fd44949e40b247f980ba14623ba0dd34: Bootstrap starting.
I20260709 13:18:16.947362 15375 tablet_bootstrap.cc:492] T 853d00d5717a4ba1a941a81884cbb7a2 P 12806407b57c4ffbbf5f5baf439af582: No bootstrap required, opened a new log
I20260709 13:18:16.951367 15375 ts_tablet_manager.cc:1403] T 853d00d5717a4ba1a941a81884cbb7a2 P 12806407b57c4ffbbf5f5baf439af582: Time spent bootstrapping tablet: real 0.178s	user 0.055s	sys 0.096s
I20260709 13:18:16.954686 15375 raft_consensus.cc:359] T 853d00d5717a4ba1a941a81884cbb7a2 P 12806407b57c4ffbbf5f5baf439af582 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd44949e40b247f980ba14623ba0dd34" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 37801 } } peers { permanent_uuid: "12806407b57c4ffbbf5f5baf439af582" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 45327 } } peers { permanent_uuid: "ce1dd5b4e0014ba8a3fa68f2ba226e49" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34123 } }
I20260709 13:18:16.955562 15375 raft_consensus.cc:385] T 853d00d5717a4ba1a941a81884cbb7a2 P 12806407b57c4ffbbf5f5baf439af582 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:16.955899 15375 raft_consensus.cc:740] T 853d00d5717a4ba1a941a81884cbb7a2 P 12806407b57c4ffbbf5f5baf439af582 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 12806407b57c4ffbbf5f5baf439af582, State: Initialized, Role: FOLLOWER
I20260709 13:18:16.956717 15375 consensus_queue.cc:260] T 853d00d5717a4ba1a941a81884cbb7a2 P 12806407b57c4ffbbf5f5baf439af582 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd44949e40b247f980ba14623ba0dd34" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 37801 } } peers { permanent_uuid: "12806407b57c4ffbbf5f5baf439af582" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 45327 } } peers { permanent_uuid: "ce1dd5b4e0014ba8a3fa68f2ba226e49" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34123 } }
I20260709 13:18:16.961498 15375 ts_tablet_manager.cc:1434] T 853d00d5717a4ba1a941a81884cbb7a2 P 12806407b57c4ffbbf5f5baf439af582: Time spent starting tablet: real 0.010s	user 0.006s	sys 0.000s
I20260709 13:18:16.964229 15379 raft_consensus.cc:493] T 28ac227079274b8eb6a3e35e4bf34e30 P fd44949e40b247f980ba14623ba0dd34 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:16.964781 15379 raft_consensus.cc:515] T 28ac227079274b8eb6a3e35e4bf34e30 P fd44949e40b247f980ba14623ba0dd34 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd44949e40b247f980ba14623ba0dd34" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 37801 } } peers { permanent_uuid: "12806407b57c4ffbbf5f5baf439af582" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 45327 } } peers { permanent_uuid: "ce1dd5b4e0014ba8a3fa68f2ba226e49" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34123 } }
I20260709 13:18:16.965550 15374 tablet_bootstrap.cc:492] T 28ac227079274b8eb6a3e35e4bf34e30 P ce1dd5b4e0014ba8a3fa68f2ba226e49: No bootstrap required, opened a new log
I20260709 13:18:16.966351 15374 ts_tablet_manager.cc:1403] T 28ac227079274b8eb6a3e35e4bf34e30 P ce1dd5b4e0014ba8a3fa68f2ba226e49: Time spent bootstrapping tablet: real 0.140s	user 0.014s	sys 0.006s
I20260709 13:18:16.967011 15379 leader_election.cc:290] T 28ac227079274b8eb6a3e35e4bf34e30 P fd44949e40b247f980ba14623ba0dd34 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 12806407b57c4ffbbf5f5baf439af582 (127.14.177.130:45327), ce1dd5b4e0014ba8a3fa68f2ba226e49 (127.14.177.131:34123)
I20260709 13:18:16.969683 15375 tablet_bootstrap.cc:492] T 81fce322177440a690ee6f46d2d67ad6 P 12806407b57c4ffbbf5f5baf439af582: Bootstrap starting.
I20260709 13:18:16.975709 15330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58a8c0164db24ffd8e992493c228804c" candidate_uuid: "fd44949e40b247f980ba14623ba0dd34" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce1dd5b4e0014ba8a3fa68f2ba226e49" is_pre_election: true
I20260709 13:18:16.976716 15329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28ac227079274b8eb6a3e35e4bf34e30" candidate_uuid: "fd44949e40b247f980ba14623ba0dd34" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce1dd5b4e0014ba8a3fa68f2ba226e49" is_pre_election: true
I20260709 13:18:16.978539 15373 tablet_bootstrap.cc:654] T 98f6aef87e78472c9987c75cac6575af P fd44949e40b247f980ba14623ba0dd34: Neither blocks nor log segments found. Creating new log.
W20260709 13:18:16.980352 15146 leader_election.cc:343] T 58a8c0164db24ffd8e992493c228804c P fd44949e40b247f980ba14623ba0dd34 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ce1dd5b4e0014ba8a3fa68f2ba226e49 (127.14.177.131:34123): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:18:16.981719 15146 leader_election.cc:343] T 28ac227079274b8eb6a3e35e4bf34e30 P fd44949e40b247f980ba14623ba0dd34 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ce1dd5b4e0014ba8a3fa68f2ba226e49 (127.14.177.131:34123): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:18:16.982523 15374 raft_consensus.cc:359] T 28ac227079274b8eb6a3e35e4bf34e30 P ce1dd5b4e0014ba8a3fa68f2ba226e49 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd44949e40b247f980ba14623ba0dd34" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 37801 } } peers { permanent_uuid: "12806407b57c4ffbbf5f5baf439af582" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 45327 } } peers { permanent_uuid: "ce1dd5b4e0014ba8a3fa68f2ba226e49" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34123 } }
I20260709 13:18:16.983245 15256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58a8c0164db24ffd8e992493c228804c" candidate_uuid: "fd44949e40b247f980ba14623ba0dd34" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12806407b57c4ffbbf5f5baf439af582" is_pre_election: true
I20260709 13:18:16.983894 15374 raft_consensus.cc:385] T 28ac227079274b8eb6a3e35e4bf34e30 P ce1dd5b4e0014ba8a3fa68f2ba226e49 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:16.983951 15255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28ac227079274b8eb6a3e35e4bf34e30" candidate_uuid: "fd44949e40b247f980ba14623ba0dd34" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12806407b57c4ffbbf5f5baf439af582" is_pre_election: true
I20260709 13:18:16.984356 15374 raft_consensus.cc:740] T 28ac227079274b8eb6a3e35e4bf34e30 P ce1dd5b4e0014ba8a3fa68f2ba226e49 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ce1dd5b4e0014ba8a3fa68f2ba226e49, State: Initialized, Role: FOLLOWER
W20260709 13:18:16.986322 15145 leader_election.cc:343] T 58a8c0164db24ffd8e992493c228804c P fd44949e40b247f980ba14623ba0dd34 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 12806407b57c4ffbbf5f5baf439af582 (127.14.177.130:45327): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:18:16.985977 15374 consensus_queue.cc:260] T 28ac227079274b8eb6a3e35e4bf34e30 P ce1dd5b4e0014ba8a3fa68f2ba226e49 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd44949e40b247f980ba14623ba0dd34" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 37801 } } peers { permanent_uuid: "12806407b57c4ffbbf5f5baf439af582" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 45327 } } peers { permanent_uuid: "ce1dd5b4e0014ba8a3fa68f2ba226e49" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34123 } }
I20260709 13:18:16.986822 15145 leader_election.cc:304] T 58a8c0164db24ffd8e992493c228804c P fd44949e40b247f980ba14623ba0dd34 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fd44949e40b247f980ba14623ba0dd34; no voters: 12806407b57c4ffbbf5f5baf439af582, ce1dd5b4e0014ba8a3fa68f2ba226e49
I20260709 13:18:16.988005 15379 raft_consensus.cc:2749] T 58a8c0164db24ffd8e992493c228804c P fd44949e40b247f980ba14623ba0dd34 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:18:16.991684 15374 ts_tablet_manager.cc:1434] T 28ac227079274b8eb6a3e35e4bf34e30 P ce1dd5b4e0014ba8a3fa68f2ba226e49: Time spent starting tablet: real 0.025s	user 0.008s	sys 0.004s
W20260709 13:18:16.988862 15145 leader_election.cc:343] T 28ac227079274b8eb6a3e35e4bf34e30 P fd44949e40b247f980ba14623ba0dd34 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 12806407b57c4ffbbf5f5baf439af582 (127.14.177.130:45327): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:18:16.994567 15374 tablet_bootstrap.cc:492] T fbf069cd9abc4875a3d03f06ff06544e P ce1dd5b4e0014ba8a3fa68f2ba226e49: Bootstrap starting.
I20260709 13:18:16.996068 15373 tablet_bootstrap.cc:492] T 98f6aef87e78472c9987c75cac6575af P fd44949e40b247f980ba14623ba0dd34: No bootstrap required, opened a new log
I20260709 13:18:16.996397 15373 ts_tablet_manager.cc:1403] T 98f6aef87e78472c9987c75cac6575af P fd44949e40b247f980ba14623ba0dd34: Time spent bootstrapping tablet: real 0.050s	user 0.010s	sys 0.006s
I20260709 13:18:16.997922 15145 leader_election.cc:304] T 28ac227079274b8eb6a3e35e4bf34e30 P fd44949e40b247f980ba14623ba0dd34 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fd44949e40b247f980ba14623ba0dd34; no voters: 12806407b57c4ffbbf5f5baf439af582, ce1dd5b4e0014ba8a3fa68f2ba226e49
I20260709 13:18:16.998785 15379 raft_consensus.cc:2749] T 28ac227079274b8eb6a3e35e4bf34e30 P fd44949e40b247f980ba14623ba0dd34 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:18:17.008625 15373 raft_consensus.cc:359] T 98f6aef87e78472c9987c75cac6575af P fd44949e40b247f980ba14623ba0dd34 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd44949e40b247f980ba14623ba0dd34" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 37801 } } peers { permanent_uuid: "12806407b57c4ffbbf5f5baf439af582" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 45327 } } peers { permanent_uuid: "ce1dd5b4e0014ba8a3fa68f2ba226e49" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34123 } }
I20260709 13:18:17.009516 15373 raft_consensus.cc:385] T 98f6aef87e78472c9987c75cac6575af P fd44949e40b247f980ba14623ba0dd34 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:17.009828 15373 raft_consensus.cc:740] T 98f6aef87e78472c9987c75cac6575af P fd44949e40b247f980ba14623ba0dd34 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fd44949e40b247f980ba14623ba0dd34, State: Initialized, Role: FOLLOWER
I20260709 13:18:17.010247 15375 tablet_bootstrap.cc:654] T 81fce322177440a690ee6f46d2d67ad6 P 12806407b57c4ffbbf5f5baf439af582: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:17.010650 15373 consensus_queue.cc:260] T 98f6aef87e78472c9987c75cac6575af P fd44949e40b247f980ba14623ba0dd34 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd44949e40b247f980ba14623ba0dd34" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 37801 } } peers { permanent_uuid: "12806407b57c4ffbbf5f5baf439af582" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 45327 } } peers { permanent_uuid: "ce1dd5b4e0014ba8a3fa68f2ba226e49" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34123 } }
I20260709 13:18:17.012668 15374 tablet_bootstrap.cc:654] T fbf069cd9abc4875a3d03f06ff06544e P ce1dd5b4e0014ba8a3fa68f2ba226e49: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:17.013176 15373 ts_tablet_manager.cc:1434] T 98f6aef87e78472c9987c75cac6575af P fd44949e40b247f980ba14623ba0dd34: Time spent starting tablet: real 0.017s	user 0.015s	sys 0.000s
I20260709 13:18:17.014120 15373 tablet_bootstrap.cc:492] T 853d00d5717a4ba1a941a81884cbb7a2 P fd44949e40b247f980ba14623ba0dd34: Bootstrap starting.
I20260709 13:18:17.020710 15373 tablet_bootstrap.cc:654] T 853d00d5717a4ba1a941a81884cbb7a2 P fd44949e40b247f980ba14623ba0dd34: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:17.023838 15375 tablet_bootstrap.cc:492] T 81fce322177440a690ee6f46d2d67ad6 P 12806407b57c4ffbbf5f5baf439af582: No bootstrap required, opened a new log
I20260709 13:18:17.024423 15375 ts_tablet_manager.cc:1403] T 81fce322177440a690ee6f46d2d67ad6 P 12806407b57c4ffbbf5f5baf439af582: Time spent bootstrapping tablet: real 0.055s	user 0.012s	sys 0.017s
I20260709 13:18:17.027695 15375 raft_consensus.cc:359] T 81fce322177440a690ee6f46d2d67ad6 P 12806407b57c4ffbbf5f5baf439af582 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd44949e40b247f980ba14623ba0dd34" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 37801 } } peers { permanent_uuid: "12806407b57c4ffbbf5f5baf439af582" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 45327 } } peers { permanent_uuid: "ce1dd5b4e0014ba8a3fa68f2ba226e49" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34123 } }
I20260709 13:18:17.028656 15375 raft_consensus.cc:385] T 81fce322177440a690ee6f46d2d67ad6 P 12806407b57c4ffbbf5f5baf439af582 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:17.029069 15375 raft_consensus.cc:740] T 81fce322177440a690ee6f46d2d67ad6 P 12806407b57c4ffbbf5f5baf439af582 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 12806407b57c4ffbbf5f5baf439af582, State: Initialized, Role: FOLLOWER
I20260709 13:18:17.029642 15374 tablet_bootstrap.cc:492] T fbf069cd9abc4875a3d03f06ff06544e P ce1dd5b4e0014ba8a3fa68f2ba226e49: No bootstrap required, opened a new log
I20260709 13:18:17.030139 15374 ts_tablet_manager.cc:1403] T fbf069cd9abc4875a3d03f06ff06544e P ce1dd5b4e0014ba8a3fa68f2ba226e49: Time spent bootstrapping tablet: real 0.036s	user 0.013s	sys 0.000s
I20260709 13:18:17.029995 15375 consensus_queue.cc:260] T 81fce322177440a690ee6f46d2d67ad6 P 12806407b57c4ffbbf5f5baf439af582 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd44949e40b247f980ba14623ba0dd34" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 37801 } } peers { permanent_uuid: "12806407b57c4ffbbf5f5baf439af582" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 45327 } } peers { permanent_uuid: "ce1dd5b4e0014ba8a3fa68f2ba226e49" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34123 } }
I20260709 13:18:17.032531 15375 ts_tablet_manager.cc:1434] T 81fce322177440a690ee6f46d2d67ad6 P 12806407b57c4ffbbf5f5baf439af582: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 13:18:17.033561 15374 raft_consensus.cc:359] T fbf069cd9abc4875a3d03f06ff06544e P ce1dd5b4e0014ba8a3fa68f2ba226e49 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd44949e40b247f980ba14623ba0dd34" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 37801 } } peers { permanent_uuid: "12806407b57c4ffbbf5f5baf439af582" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 45327 } } peers { permanent_uuid: "ce1dd5b4e0014ba8a3fa68f2ba226e49" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34123 } }
I20260709 13:18:17.034197 15375 tablet_bootstrap.cc:492] T 6f3b305de2a04b54aef8674096fa6358 P 12806407b57c4ffbbf5f5baf439af582: Bootstrap starting.
I20260709 13:18:17.034399 15374 raft_consensus.cc:385] T fbf069cd9abc4875a3d03f06ff06544e P ce1dd5b4e0014ba8a3fa68f2ba226e49 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:17.034777 15374 raft_consensus.cc:740] T fbf069cd9abc4875a3d03f06ff06544e P ce1dd5b4e0014ba8a3fa68f2ba226e49 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ce1dd5b4e0014ba8a3fa68f2ba226e49, State: Initialized, Role: FOLLOWER
I20260709 13:18:17.035688 15374 consensus_queue.cc:260] T fbf069cd9abc4875a3d03f06ff06544e P ce1dd5b4e0014ba8a3fa68f2ba226e49 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd44949e40b247f980ba14623ba0dd34" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 37801 } } peers { permanent_uuid: "12806407b57c4ffbbf5f5baf439af582" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 45327 } } peers { permanent_uuid: "ce1dd5b4e0014ba8a3fa68f2ba226e49" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34123 } }
I20260709 13:18:17.039628 15373 tablet_bootstrap.cc:492] T 853d00d5717a4ba1a941a81884cbb7a2 P fd44949e40b247f980ba14623ba0dd34: No bootstrap required, opened a new log
I20260709 13:18:17.040104 15373 ts_tablet_manager.cc:1403] T 853d00d5717a4ba1a941a81884cbb7a2 P fd44949e40b247f980ba14623ba0dd34: Time spent bootstrapping tablet: real 0.026s	user 0.010s	sys 0.012s
I20260709 13:18:17.043037 15373 raft_consensus.cc:359] T 853d00d5717a4ba1a941a81884cbb7a2 P fd44949e40b247f980ba14623ba0dd34 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd44949e40b247f980ba14623ba0dd34" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 37801 } } peers { permanent_uuid: "12806407b57c4ffbbf5f5baf439af582" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 45327 } } peers { permanent_uuid: "ce1dd5b4e0014ba8a3fa68f2ba226e49" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34123 } }
I20260709 13:18:17.043841 15373 raft_consensus.cc:385] T 853d00d5717a4ba1a941a81884cbb7a2 P fd44949e40b247f980ba14623ba0dd34 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:17.043946 15375 tablet_bootstrap.cc:654] T 6f3b305de2a04b54aef8674096fa6358 P 12806407b57c4ffbbf5f5baf439af582: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:17.044169 15373 raft_consensus.cc:740] T 853d00d5717a4ba1a941a81884cbb7a2 P fd44949e40b247f980ba14623ba0dd34 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fd44949e40b247f980ba14623ba0dd34, State: Initialized, Role: FOLLOWER
I20260709 13:18:17.044986 15373 consensus_queue.cc:260] T 853d00d5717a4ba1a941a81884cbb7a2 P fd44949e40b247f980ba14623ba0dd34 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd44949e40b247f980ba14623ba0dd34" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 37801 } } peers { permanent_uuid: "12806407b57c4ffbbf5f5baf439af582" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 45327 } } peers { permanent_uuid: "ce1dd5b4e0014ba8a3fa68f2ba226e49" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34123 } }
I20260709 13:18:17.048652 15374 ts_tablet_manager.cc:1434] T fbf069cd9abc4875a3d03f06ff06544e P ce1dd5b4e0014ba8a3fa68f2ba226e49: Time spent starting tablet: real 0.018s	user 0.009s	sys 0.000s
I20260709 13:18:17.048856 15373 ts_tablet_manager.cc:1434] T 853d00d5717a4ba1a941a81884cbb7a2 P fd44949e40b247f980ba14623ba0dd34: Time spent starting tablet: real 0.008s	user 0.003s	sys 0.004s
I20260709 13:18:17.049738 15374 tablet_bootstrap.cc:492] T 54dc07b181bd495d8c8e2cae81240f16 P ce1dd5b4e0014ba8a3fa68f2ba226e49: Bootstrap starting.
I20260709 13:18:17.050623 15379 raft_consensus.cc:493] T 8391f4ec0b1349f4a1758a0ce2382b7c P fd44949e40b247f980ba14623ba0dd34 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:17.051086 15379 raft_consensus.cc:515] T 8391f4ec0b1349f4a1758a0ce2382b7c P fd44949e40b247f980ba14623ba0dd34 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd44949e40b247f980ba14623ba0dd34" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 37801 } } peers { permanent_uuid: "12806407b57c4ffbbf5f5baf439af582" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 45327 } } peers { permanent_uuid: "ce1dd5b4e0014ba8a3fa68f2ba226e49" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34123 } }
I20260709 13:18:17.052670 15381 raft_consensus.cc:493] T fbf069cd9abc4875a3d03f06ff06544e P ce1dd5b4e0014ba8a3fa68f2ba226e49 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:17.053097 15379 leader_election.cc:290] T 8391f4ec0b1349f4a1758a0ce2382b7c P fd44949e40b247f980ba14623ba0dd34 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 12806407b57c4ffbbf5f5baf439af582 (127.14.177.130:45327), ce1dd5b4e0014ba8a3fa68f2ba226e49 (127.14.177.131:34123)
I20260709 13:18:17.053190 15381 raft_consensus.cc:515] T fbf069cd9abc4875a3d03f06ff06544e P ce1dd5b4e0014ba8a3fa68f2ba226e49 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd44949e40b247f980ba14623ba0dd34" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 37801 } } peers { permanent_uuid: "12806407b57c4ffbbf5f5baf439af582" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 45327 } } peers { permanent_uuid: "ce1dd5b4e0014ba8a3fa68f2ba226e49" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34123 } }
I20260709 13:18:17.054661 15329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8391f4ec0b1349f4a1758a0ce2382b7c" candidate_uuid: "fd44949e40b247f980ba14623ba0dd34" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce1dd5b4e0014ba8a3fa68f2ba226e49" is_pre_election: true
W20260709 13:18:17.056167 15146 leader_election.cc:343] T 8391f4ec0b1349f4a1758a0ce2382b7c P fd44949e40b247f980ba14623ba0dd34 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ce1dd5b4e0014ba8a3fa68f2ba226e49 (127.14.177.131:34123): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:18:17.055805 15381 leader_election.cc:290] T fbf069cd9abc4875a3d03f06ff06544e P ce1dd5b4e0014ba8a3fa68f2ba226e49 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fd44949e40b247f980ba14623ba0dd34 (127.14.177.129:37801), 12806407b57c4ffbbf5f5baf439af582 (127.14.177.130:45327)
I20260709 13:18:17.057720 15255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbf069cd9abc4875a3d03f06ff06544e" candidate_uuid: "ce1dd5b4e0014ba8a3fa68f2ba226e49" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12806407b57c4ffbbf5f5baf439af582" is_pre_election: true
I20260709 13:18:17.058821 15181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbf069cd9abc4875a3d03f06ff06544e" candidate_uuid: "ce1dd5b4e0014ba8a3fa68f2ba226e49" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fd44949e40b247f980ba14623ba0dd34" is_pre_election: true
I20260709 13:18:17.059103 15374 tablet_bootstrap.cc:654] T 54dc07b181bd495d8c8e2cae81240f16 P ce1dd5b4e0014ba8a3fa68f2ba226e49: Neither blocks nor log segments found. Creating new log.
W20260709 13:18:17.060679 15295 leader_election.cc:343] T fbf069cd9abc4875a3d03f06ff06544e P ce1dd5b4e0014ba8a3fa68f2ba226e49 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fd44949e40b247f980ba14623ba0dd34 (127.14.177.129:37801): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:18:17.061429 15294 leader_election.cc:343] T fbf069cd9abc4875a3d03f06ff06544e P ce1dd5b4e0014ba8a3fa68f2ba226e49 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 12806407b57c4ffbbf5f5baf439af582 (127.14.177.130:45327): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:18:17.061812 15294 leader_election.cc:304] T fbf069cd9abc4875a3d03f06ff06544e P ce1dd5b4e0014ba8a3fa68f2ba226e49 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce1dd5b4e0014ba8a3fa68f2ba226e49; no voters: 12806407b57c4ffbbf5f5baf439af582, fd44949e40b247f980ba14623ba0dd34
I20260709 13:18:17.062582 15381 raft_consensus.cc:2749] T fbf069cd9abc4875a3d03f06ff06544e P ce1dd5b4e0014ba8a3fa68f2ba226e49 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:18:17.063618 15255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8391f4ec0b1349f4a1758a0ce2382b7c" candidate_uuid: "fd44949e40b247f980ba14623ba0dd34" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12806407b57c4ffbbf5f5baf439af582" is_pre_election: true
W20260709 13:18:17.064812 15145 leader_election.cc:343] T 8391f4ec0b1349f4a1758a0ce2382b7c P fd44949e40b247f980ba14623ba0dd34 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 12806407b57c4ffbbf5f5baf439af582 (127.14.177.130:45327): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:18:17.065359 15145 leader_election.cc:304] T 8391f4ec0b1349f4a1758a0ce2382b7c P fd44949e40b247f980ba14623ba0dd34 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fd44949e40b247f980ba14623ba0dd34; no voters: 12806407b57c4ffbbf5f5baf439af582, ce1dd5b4e0014ba8a3fa68f2ba226e49
I20260709 13:18:17.066210 15379 raft_consensus.cc:2749] T 8391f4ec0b1349f4a1758a0ce2382b7c P fd44949e40b247f980ba14623ba0dd34 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:18:17.049775 15373 tablet_bootstrap.cc:492] T 6f3b305de2a04b54aef8674096fa6358 P fd44949e40b247f980ba14623ba0dd34: Bootstrap starting.
I20260709 13:18:17.081593 15373 tablet_bootstrap.cc:654] T 6f3b305de2a04b54aef8674096fa6358 P fd44949e40b247f980ba14623ba0dd34: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:17.082912 15375 tablet_bootstrap.cc:492] T 6f3b305de2a04b54aef8674096fa6358 P 12806407b57c4ffbbf5f5baf439af582: No bootstrap required, opened a new log
I20260709 13:18:17.083516 15375 ts_tablet_manager.cc:1403] T 6f3b305de2a04b54aef8674096fa6358 P 12806407b57c4ffbbf5f5baf439af582: Time spent bootstrapping tablet: real 0.050s	user 0.023s	sys 0.015s
I20260709 13:18:17.086570 15375 raft_consensus.cc:359] T 6f3b305de2a04b54aef8674096fa6358 P 12806407b57c4ffbbf5f5baf439af582 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd44949e40b247f980ba14623ba0dd34" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 37801 } } peers { permanent_uuid: "12806407b57c4ffbbf5f5baf439af582" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 45327 } } peers { permanent_uuid: "ce1dd5b4e0014ba8a3fa68f2ba226e49" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34123 } }
I20260709 13:18:17.087462 15373 tablet_bootstrap.cc:492] T 6f3b305de2a04b54aef8674096fa6358 P fd44949e40b247f980ba14623ba0dd34: No bootstrap required, opened a new log
I20260709 13:18:17.087344 15375 raft_consensus.cc:385] T 6f3b305de2a04b54aef8674096fa6358 P 12806407b57c4ffbbf5f5baf439af582 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:17.088459 15373 ts_tablet_manager.cc:1403] T 6f3b305de2a04b54aef8674096fa6358 P fd44949e40b247f980ba14623ba0dd34: Time spent bootstrapping tablet: real 0.039s	user 0.012s	sys 0.004s
I20260709 13:18:17.088380 15375 raft_consensus.cc:740] T 6f3b305de2a04b54aef8674096fa6358 P 12806407b57c4ffbbf5f5baf439af582 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 12806407b57c4ffbbf5f5baf439af582, State: Initialized, Role: FOLLOWER
I20260709 13:18:17.089401 15375 consensus_queue.cc:260] T 6f3b305de2a04b54aef8674096fa6358 P 12806407b57c4ffbbf5f5baf439af582 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd44949e40b247f980ba14623ba0dd34" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 37801 } } peers { permanent_uuid: "12806407b57c4ffbbf5f5baf439af582" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 45327 } } peers { permanent_uuid: "ce1dd5b4e0014ba8a3fa68f2ba226e49" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34123 } }
I20260709 13:18:17.090619 15373 raft_consensus.cc:359] T 6f3b305de2a04b54aef8674096fa6358 P fd44949e40b247f980ba14623ba0dd34 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd44949e40b247f980ba14623ba0dd34" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 37801 } } peers { permanent_uuid: "12806407b57c4ffbbf5f5baf439af582" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 45327 } } peers { permanent_uuid: "ce1dd5b4e0014ba8a3fa68f2ba226e49" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34123 } }
I20260709 13:18:17.091228 15373 raft_consensus.cc:385] T 6f3b305de2a04b54aef8674096fa6358 P fd44949e40b247f980ba14623ba0dd34 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:17.091560 15373 raft_consensus.cc:740] T 6f3b305de2a04b54aef8674096fa6358 P fd44949e40b247f980ba14623ba0dd34 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fd44949e40b247f980ba14623ba0dd34, State: Initialized, Role: FOLLOWER
I20260709 13:18:17.092308 15373 consensus_queue.cc:260] T 6f3b305de2a04b54aef8674096fa6358 P fd44949e40b247f980ba14623ba0dd34 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd44949e40b247f980ba14623ba0dd34" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 37801 } } peers { permanent_uuid: "12806407b57c4ffbbf5f5baf439af582" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 45327 } } peers { permanent_uuid: "ce1dd5b4e0014ba8a3fa68f2ba226e49" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34123 } }
I20260709 13:18:17.093549 15374 tablet_bootstrap.cc:492] T 54dc07b181bd495d8c8e2cae81240f16 P ce1dd5b4e0014ba8a3fa68f2ba226e49: No bootstrap required, opened a new log
I20260709 13:18:17.094059 15374 ts_tablet_manager.cc:1403] T 54dc07b181bd495d8c8e2cae81240f16 P ce1dd5b4e0014ba8a3fa68f2ba226e49: Time spent bootstrapping tablet: real 0.045s	user 0.011s	sys 0.004s
I20260709 13:18:17.094815 15373 ts_tablet_manager.cc:1434] T 6f3b305de2a04b54aef8674096fa6358 P fd44949e40b247f980ba14623ba0dd34: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.000s
I20260709 13:18:17.094941 15375 ts_tablet_manager.cc:1434] T 6f3b305de2a04b54aef8674096fa6358 P 12806407b57c4ffbbf5f5baf439af582: Time spent starting tablet: real 0.011s	user 0.005s	sys 0.002s
I20260709 13:18:17.095803 15373 tablet_bootstrap.cc:492] T fbf069cd9abc4875a3d03f06ff06544e P fd44949e40b247f980ba14623ba0dd34: Bootstrap starting.
I20260709 13:18:17.096109 15375 tablet_bootstrap.cc:492] T 98f6aef87e78472c9987c75cac6575af P 12806407b57c4ffbbf5f5baf439af582: Bootstrap starting.
I20260709 13:18:17.097061 15374 raft_consensus.cc:359] T 54dc07b181bd495d8c8e2cae81240f16 P ce1dd5b4e0014ba8a3fa68f2ba226e49 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce1dd5b4e0014ba8a3fa68f2ba226e49" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34123 } } peers { permanent_uuid: "fd44949e40b247f980ba14623ba0dd34" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 37801 } } peers { permanent_uuid: "12806407b57c4ffbbf5f5baf439af582" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 45327 } }
I20260709 13:18:17.097815 15374 raft_consensus.cc:385] T 54dc07b181bd495d8c8e2cae81240f16 P ce1dd5b4e0014ba8a3fa68f2ba226e49 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:17.098150 15374 raft_consensus.cc:740] T 54dc07b181bd495d8c8e2cae81240f16 P ce1dd5b4e0014ba8a3fa68f2ba226e49 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ce1dd5b4e0014ba8a3fa68f2ba226e49, State: Initialized, Role: FOLLOWER
I20260709 13:18:17.099010 15374 consensus_queue.cc:260] T 54dc07b181bd495d8c8e2cae81240f16 P ce1dd5b4e0014ba8a3fa68f2ba226e49 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce1dd5b4e0014ba8a3fa68f2ba226e49" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34123 } } peers { permanent_uuid: "fd44949e40b247f980ba14623ba0dd34" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 37801 } } peers { permanent_uuid: "12806407b57c4ffbbf5f5baf439af582" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 45327 } }
I20260709 13:18:17.102110 15374 ts_tablet_manager.cc:1434] T 54dc07b181bd495d8c8e2cae81240f16 P ce1dd5b4e0014ba8a3fa68f2ba226e49: Time spent starting tablet: real 0.008s	user 0.004s	sys 0.004s
I20260709 13:18:17.102622 15373 tablet_bootstrap.cc:654] T fbf069cd9abc4875a3d03f06ff06544e P fd44949e40b247f980ba14623ba0dd34: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:17.103132 15374 tablet_bootstrap.cc:492] T 8391f4ec0b1349f4a1758a0ce2382b7c P ce1dd5b4e0014ba8a3fa68f2ba226e49: Bootstrap starting.
I20260709 13:18:17.109834 15380 raft_consensus.cc:493] T 853d00d5717a4ba1a941a81884cbb7a2 P 12806407b57c4ffbbf5f5baf439af582 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:17.110517 15380 raft_consensus.cc:515] T 853d00d5717a4ba1a941a81884cbb7a2 P 12806407b57c4ffbbf5f5baf439af582 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd44949e40b247f980ba14623ba0dd34" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 37801 } } peers { permanent_uuid: "12806407b57c4ffbbf5f5baf439af582" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 45327 } } peers { permanent_uuid: "ce1dd5b4e0014ba8a3fa68f2ba226e49" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34123 } }
I20260709 13:18:17.125922 15374 tablet_bootstrap.cc:654] T 8391f4ec0b1349f4a1758a0ce2382b7c P ce1dd5b4e0014ba8a3fa68f2ba226e49: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:17.127489 15379 raft_consensus.cc:493] T 81fce322177440a690ee6f46d2d67ad6 P fd44949e40b247f980ba14623ba0dd34 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:17.127632 15380 leader_election.cc:290] T 853d00d5717a4ba1a941a81884cbb7a2 P 12806407b57c4ffbbf5f5baf439af582 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fd44949e40b247f980ba14623ba0dd34 (127.14.177.129:37801), ce1dd5b4e0014ba8a3fa68f2ba226e49 (127.14.177.131:34123)
I20260709 13:18:17.128037 15379 raft_consensus.cc:515] T 81fce322177440a690ee6f46d2d67ad6 P fd44949e40b247f980ba14623ba0dd34 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd44949e40b247f980ba14623ba0dd34" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 37801 } } peers { permanent_uuid: "12806407b57c4ffbbf5f5baf439af582" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 45327 } } peers { permanent_uuid: "ce1dd5b4e0014ba8a3fa68f2ba226e49" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34123 } }
I20260709 13:18:17.128937 15375 tablet_bootstrap.cc:654] T 98f6aef87e78472c9987c75cac6575af P 12806407b57c4ffbbf5f5baf439af582: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:17.145064 15373 tablet_bootstrap.cc:492] T fbf069cd9abc4875a3d03f06ff06544e P fd44949e40b247f980ba14623ba0dd34: No bootstrap required, opened a new log
I20260709 13:18:17.132501 15329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81fce322177440a690ee6f46d2d67ad6" candidate_uuid: "fd44949e40b247f980ba14623ba0dd34" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce1dd5b4e0014ba8a3fa68f2ba226e49" is_pre_election: true
I20260709 13:18:17.145581 15373 ts_tablet_manager.cc:1403] T fbf069cd9abc4875a3d03f06ff06544e P fd44949e40b247f980ba14623ba0dd34: Time spent bootstrapping tablet: real 0.050s	user 0.017s	sys 0.019s
I20260709 13:18:17.131716 15255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81fce322177440a690ee6f46d2d67ad6" candidate_uuid: "fd44949e40b247f980ba14623ba0dd34" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12806407b57c4ffbbf5f5baf439af582" is_pre_election: true
I20260709 13:18:17.147218 15255 raft_consensus.cc:2468] T 81fce322177440a690ee6f46d2d67ad6 P 12806407b57c4ffbbf5f5baf439af582 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fd44949e40b247f980ba14623ba0dd34 in term 0.
I20260709 13:18:17.148591 15145 leader_election.cc:304] T 81fce322177440a690ee6f46d2d67ad6 P fd44949e40b247f980ba14623ba0dd34 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12806407b57c4ffbbf5f5baf439af582, fd44949e40b247f980ba14623ba0dd34; no voters: 
I20260709 13:18:17.149093 15373 raft_consensus.cc:359] T fbf069cd9abc4875a3d03f06ff06544e P fd44949e40b247f980ba14623ba0dd34 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd44949e40b247f980ba14623ba0dd34" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 37801 } } peers { permanent_uuid: "12806407b57c4ffbbf5f5baf439af582" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 45327 } } peers { permanent_uuid: "ce1dd5b4e0014ba8a3fa68f2ba226e49" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34123 } }
I20260709 13:18:17.149930 15373 raft_consensus.cc:385] T fbf069cd9abc4875a3d03f06ff06544e P fd44949e40b247f980ba14623ba0dd34 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:17.150241 15373 raft_consensus.cc:740] T fbf069cd9abc4875a3d03f06ff06544e P fd44949e40b247f980ba14623ba0dd34 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fd44949e40b247f980ba14623ba0dd34, State: Initialized, Role: FOLLOWER
I20260709 13:18:17.130708 15379 leader_election.cc:290] T 81fce322177440a690ee6f46d2d67ad6 P fd44949e40b247f980ba14623ba0dd34 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 12806407b57c4ffbbf5f5baf439af582 (127.14.177.130:45327), ce1dd5b4e0014ba8a3fa68f2ba226e49 (127.14.177.131:34123)
I20260709 13:18:17.151758 15379 raft_consensus.cc:493] T 54dc07b181bd495d8c8e2cae81240f16 P fd44949e40b247f980ba14623ba0dd34 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:17.152251 15379 raft_consensus.cc:515] T 54dc07b181bd495d8c8e2cae81240f16 P fd44949e40b247f980ba14623ba0dd34 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce1dd5b4e0014ba8a3fa68f2ba226e49" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34123 } } peers { permanent_uuid: "fd44949e40b247f980ba14623ba0dd34" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 37801 } } peers { permanent_uuid: "12806407b57c4ffbbf5f5baf439af582" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 45327 } }
I20260709 13:18:17.154472 15379 leader_election.cc:290] T 54dc07b181bd495d8c8e2cae81240f16 P fd44949e40b247f980ba14623ba0dd34 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ce1dd5b4e0014ba8a3fa68f2ba226e49 (127.14.177.131:34123), 12806407b57c4ffbbf5f5baf439af582 (127.14.177.130:45327)
I20260709 13:18:17.155014 15379 raft_consensus.cc:493] T 98f6aef87e78472c9987c75cac6575af P fd44949e40b247f980ba14623ba0dd34 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:17.155481 15379 raft_consensus.cc:515] T 98f6aef87e78472c9987c75cac6575af P fd44949e40b247f980ba14623ba0dd34 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd44949e40b247f980ba14623ba0dd34" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 37801 } } peers { permanent_uuid: "12806407b57c4ffbbf5f5baf439af582" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 45327 } } peers { permanent_uuid: "ce1dd5b4e0014ba8a3fa68f2ba226e49" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34123 } }
I20260709 13:18:17.157424 15379 leader_election.cc:290] T 98f6aef87e78472c9987c75cac6575af P fd44949e40b247f980ba14623ba0dd34 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 12806407b57c4ffbbf5f5baf439af582 (127.14.177.130:45327), ce1dd5b4e0014ba8a3fa68f2ba226e49 (127.14.177.131:34123)
I20260709 13:18:17.157974 15379 raft_consensus.cc:2804] T 81fce322177440a690ee6f46d2d67ad6 P fd44949e40b247f980ba14623ba0dd34 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:18:17.158378 15379 raft_consensus.cc:493] T 81fce322177440a690ee6f46d2d67ad6 P fd44949e40b247f980ba14623ba0dd34 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:17.158780 15379 raft_consensus.cc:3060] T 81fce322177440a690ee6f46d2d67ad6 P fd44949e40b247f980ba14623ba0dd34 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:17.159860 15380 raft_consensus.cc:493] T 54dc07b181bd495d8c8e2cae81240f16 P 12806407b57c4ffbbf5f5baf439af582 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:17.160322 15380 raft_consensus.cc:515] T 54dc07b181bd495d8c8e2cae81240f16 P 12806407b57c4ffbbf5f5baf439af582 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce1dd5b4e0014ba8a3fa68f2ba226e49" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34123 } } peers { permanent_uuid: "fd44949e40b247f980ba14623ba0dd34" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 37801 } } peers { permanent_uuid: "12806407b57c4ffbbf5f5baf439af582" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 45327 } }
I20260709 13:18:17.162238 15380 leader_election.cc:290] T 54dc07b181bd495d8c8e2cae81240f16 P 12806407b57c4ffbbf5f5baf439af582 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ce1dd5b4e0014ba8a3fa68f2ba226e49 (127.14.177.131:34123), fd44949e40b247f980ba14623ba0dd34 (127.14.177.129:37801)
I20260709 13:18:17.151068 15373 consensus_queue.cc:260] T fbf069cd9abc4875a3d03f06ff06544e P fd44949e40b247f980ba14623ba0dd34 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd44949e40b247f980ba14623ba0dd34" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 37801 } } peers { permanent_uuid: "12806407b57c4ffbbf5f5baf439af582" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 45327 } } peers { permanent_uuid: "ce1dd5b4e0014ba8a3fa68f2ba226e49" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34123 } }
I20260709 13:18:17.169786 15375 tablet_bootstrap.cc:492] T 98f6aef87e78472c9987c75cac6575af P 12806407b57c4ffbbf5f5baf439af582: No bootstrap required, opened a new log
I20260709 13:18:17.170379 15375 ts_tablet_manager.cc:1403] T 98f6aef87e78472c9987c75cac6575af P 12806407b57c4ffbbf5f5baf439af582: Time spent bootstrapping tablet: real 0.075s	user 0.008s	sys 0.004s
I20260709 13:18:17.179811 15373 ts_tablet_manager.cc:1434] T fbf069cd9abc4875a3d03f06ff06544e P fd44949e40b247f980ba14623ba0dd34: Time spent starting tablet: real 0.034s	user 0.009s	sys 0.001s
I20260709 13:18:17.186401 15329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54dc07b181bd495d8c8e2cae81240f16" candidate_uuid: "fd44949e40b247f980ba14623ba0dd34" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce1dd5b4e0014ba8a3fa68f2ba226e49" is_pre_election: true
I20260709 13:18:17.187574 15374 tablet_bootstrap.cc:492] T 8391f4ec0b1349f4a1758a0ce2382b7c P ce1dd5b4e0014ba8a3fa68f2ba226e49: No bootstrap required, opened a new log
I20260709 13:18:17.187122 15329 raft_consensus.cc:2468] T 54dc07b181bd495d8c8e2cae81240f16 P ce1dd5b4e0014ba8a3fa68f2ba226e49 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fd44949e40b247f980ba14623ba0dd34 in term 0.
I20260709 13:18:17.188508 15374 ts_tablet_manager.cc:1403] T 8391f4ec0b1349f4a1758a0ce2382b7c P ce1dd5b4e0014ba8a3fa68f2ba226e49: Time spent bootstrapping tablet: real 0.086s	user 0.004s	sys 0.014s
I20260709 13:18:17.182088 15375 raft_consensus.cc:359] T 98f6aef87e78472c9987c75cac6575af P 12806407b57c4ffbbf5f5baf439af582 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd44949e40b247f980ba14623ba0dd34" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 37801 } } peers { permanent_uuid: "12806407b57c4ffbbf5f5baf439af582" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 45327 } } peers { permanent_uuid: "ce1dd5b4e0014ba8a3fa68f2ba226e49" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34123 } }
I20260709 13:18:17.190637 15255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54dc07b181bd495d8c8e2cae81240f16" candidate_uuid: "fd44949e40b247f980ba14623ba0dd34" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12806407b57c4ffbbf5f5baf439af582" is_pre_election: true
I20260709 13:18:17.190646 15329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98f6aef87e78472c9987c75cac6575af" candidate_uuid: "fd44949e40b247f980ba14623ba0dd34" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce1dd5b4e0014ba8a3fa68f2ba226e49" is_pre_election: true
W20260709 13:18:17.192404 15146 leader_election.cc:343] T 81fce322177440a690ee6f46d2d67ad6 P fd44949e40b247f980ba14623ba0dd34 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ce1dd5b4e0014ba8a3fa68f2ba226e49 (127.14.177.131:34123): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:18:17.192606 15255 raft_consensus.cc:2468] T 54dc07b181bd495d8c8e2cae81240f16 P 12806407b57c4ffbbf5f5baf439af582 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fd44949e40b247f980ba14623ba0dd34 in term 0.
I20260709 13:18:17.193243 15380 raft_consensus.cc:493] T 6f3b305de2a04b54aef8674096fa6358 P 12806407b57c4ffbbf5f5baf439af582 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:17.193703 15380 raft_consensus.cc:515] T 6f3b305de2a04b54aef8674096fa6358 P 12806407b57c4ffbbf5f5baf439af582 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd44949e40b247f980ba14623ba0dd34" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 37801 } } peers { permanent_uuid: "12806407b57c4ffbbf5f5baf439af582" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 45327 } } peers { permanent_uuid: "ce1dd5b4e0014ba8a3fa68f2ba226e49" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34123 } }
I20260709 13:18:17.194299 15181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "853d00d5717a4ba1a941a81884cbb7a2" candidate_uuid: "12806407b57c4ffbbf5f5baf439af582" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fd44949e40b247f980ba14623ba0dd34" is_pre_election: true
I20260709 13:18:17.194607 15330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54dc07b181bd495d8c8e2cae81240f16" candidate_uuid: "12806407b57c4ffbbf5f5baf439af582" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce1dd5b4e0014ba8a3fa68f2ba226e49" is_pre_election: true
I20260709 13:18:17.194968 15181 raft_consensus.cc:2468] T 853d00d5717a4ba1a941a81884cbb7a2 P fd44949e40b247f980ba14623ba0dd34 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12806407b57c4ffbbf5f5baf439af582 in term 0.
I20260709 13:18:17.195712 15380 leader_election.cc:290] T 6f3b305de2a04b54aef8674096fa6358 P 12806407b57c4ffbbf5f5baf439af582 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fd44949e40b247f980ba14623ba0dd34 (127.14.177.129:37801), ce1dd5b4e0014ba8a3fa68f2ba226e49 (127.14.177.131:34123)
I20260709 13:18:17.196040 15329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "853d00d5717a4ba1a941a81884cbb7a2" candidate_uuid: "12806407b57c4ffbbf5f5baf439af582" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce1dd5b4e0014ba8a3fa68f2ba226e49" is_pre_election: true
I20260709 13:18:17.195712 15330 raft_consensus.cc:2468] T 54dc07b181bd495d8c8e2cae81240f16 P ce1dd5b4e0014ba8a3fa68f2ba226e49 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12806407b57c4ffbbf5f5baf439af582 in term 0.
I20260709 13:18:17.197585 15180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54dc07b181bd495d8c8e2cae81240f16" candidate_uuid: "12806407b57c4ffbbf5f5baf439af582" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fd44949e40b247f980ba14623ba0dd34" is_pre_election: true
I20260709 13:18:17.199126 15180 raft_consensus.cc:2468] T 54dc07b181bd495d8c8e2cae81240f16 P fd44949e40b247f980ba14623ba0dd34 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12806407b57c4ffbbf5f5baf439af582 in term 0.
I20260709 13:18:17.191664 15375 raft_consensus.cc:385] T 98f6aef87e78472c9987c75cac6575af P 12806407b57c4ffbbf5f5baf439af582 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:17.200802 15375 raft_consensus.cc:740] T 98f6aef87e78472c9987c75cac6575af P 12806407b57c4ffbbf5f5baf439af582 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 12806407b57c4ffbbf5f5baf439af582, State: Initialized, Role: FOLLOWER
I20260709 13:18:17.196143 15379 raft_consensus.cc:515] T 81fce322177440a690ee6f46d2d67ad6 P fd44949e40b247f980ba14623ba0dd34 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd44949e40b247f980ba14623ba0dd34" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 37801 } } peers { permanent_uuid: "12806407b57c4ffbbf5f5baf439af582" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 45327 } } peers { permanent_uuid: "ce1dd5b4e0014ba8a3fa68f2ba226e49" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34123 } }
I20260709 13:18:17.202019 15255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98f6aef87e78472c9987c75cac6575af" candidate_uuid: "fd44949e40b247f980ba14623ba0dd34" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12806407b57c4ffbbf5f5baf439af582" is_pre_election: true
I20260709 13:18:17.205483 15375 consensus_queue.cc:260] T 98f6aef87e78472c9987c75cac6575af P 12806407b57c4ffbbf5f5baf439af582 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd44949e40b247f980ba14623ba0dd34" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 37801 } } peers { permanent_uuid: "12806407b57c4ffbbf5f5baf439af582" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 45327 } } peers { permanent_uuid: "ce1dd5b4e0014ba8a3fa68f2ba226e49" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34123 } }
I20260709 13:18:17.206470 15145 leader_election.cc:304] T 54dc07b181bd495d8c8e2cae81240f16 P fd44949e40b247f980ba14623ba0dd34 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12806407b57c4ffbbf5f5baf439af582, fd44949e40b247f980ba14623ba0dd34; no voters: 
I20260709 13:18:17.206316 15374 raft_consensus.cc:359] T 8391f4ec0b1349f4a1758a0ce2382b7c P ce1dd5b4e0014ba8a3fa68f2ba226e49 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd44949e40b247f980ba14623ba0dd34" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 37801 } } peers { permanent_uuid: "12806407b57c4ffbbf5f5baf439af582" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 45327 } } peers { permanent_uuid: "ce1dd5b4e0014ba8a3fa68f2ba226e49" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34123 } }
I20260709 13:18:17.207799 15374 raft_consensus.cc:385] T 8391f4ec0b1349f4a1758a0ce2382b7c P ce1dd5b4e0014ba8a3fa68f2ba226e49 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:17.208176 15374 raft_consensus.cc:740] T 8391f4ec0b1349f4a1758a0ce2382b7c P ce1dd5b4e0014ba8a3fa68f2ba226e49 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ce1dd5b4e0014ba8a3fa68f2ba226e49, State: Initialized, Role: FOLLOWER
I20260709 13:18:17.202323 15180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f3b305de2a04b54aef8674096fa6358" candidate_uuid: "12806407b57c4ffbbf5f5baf439af582" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fd44949e40b247f980ba14623ba0dd34" is_pre_election: true
I20260709 13:18:17.209885 15255 raft_consensus.cc:2468] T 98f6aef87e78472c9987c75cac6575af P 12806407b57c4ffbbf5f5baf439af582 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fd44949e40b247f980ba14623ba0dd34 in term 0.
I20260709 13:18:17.210762 15329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f3b305de2a04b54aef8674096fa6358" candidate_uuid: "12806407b57c4ffbbf5f5baf439af582" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce1dd5b4e0014ba8a3fa68f2ba226e49" is_pre_election: true
I20260709 13:18:17.211436 15329 raft_consensus.cc:2468] T 6f3b305de2a04b54aef8674096fa6358 P ce1dd5b4e0014ba8a3fa68f2ba226e49 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12806407b57c4ffbbf5f5baf439af582 in term 0.
I20260709 13:18:17.211771 15379 leader_election.cc:290] T 81fce322177440a690ee6f46d2d67ad6 P fd44949e40b247f980ba14623ba0dd34 [CANDIDATE]: Term 1 election: Requested vote from peers 12806407b57c4ffbbf5f5baf439af582 (127.14.177.130:45327), ce1dd5b4e0014ba8a3fa68f2ba226e49 (127.14.177.131:34123)
I20260709 13:18:17.212669 15180 raft_consensus.cc:2468] T 6f3b305de2a04b54aef8674096fa6358 P fd44949e40b247f980ba14623ba0dd34 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12806407b57c4ffbbf5f5baf439af582 in term 0.
I20260709 13:18:17.213172 15379 raft_consensus.cc:2804] T 54dc07b181bd495d8c8e2cae81240f16 P fd44949e40b247f980ba14623ba0dd34 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:18:17.213529 15379 raft_consensus.cc:493] T 54dc07b181bd495d8c8e2cae81240f16 P fd44949e40b247f980ba14623ba0dd34 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:17.213896 15379 raft_consensus.cc:3060] T 54dc07b181bd495d8c8e2cae81240f16 P fd44949e40b247f980ba14623ba0dd34 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:17.214288 15256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81fce322177440a690ee6f46d2d67ad6" candidate_uuid: "fd44949e40b247f980ba14623ba0dd34" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12806407b57c4ffbbf5f5baf439af582"
I20260709 13:18:17.214921 15256 raft_consensus.cc:3060] T 81fce322177440a690ee6f46d2d67ad6 P 12806407b57c4ffbbf5f5baf439af582 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:17.215976 15375 ts_tablet_manager.cc:1434] T 98f6aef87e78472c9987c75cac6575af P 12806407b57c4ffbbf5f5baf439af582: Time spent starting tablet: real 0.045s	user 0.009s	sys 0.004s
W20260709 13:18:17.217326 15146 leader_election.cc:343] T 98f6aef87e78472c9987c75cac6575af P fd44949e40b247f980ba14623ba0dd34 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ce1dd5b4e0014ba8a3fa68f2ba226e49 (127.14.177.131:34123): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:18:17.212554 15374 consensus_queue.cc:260] T 8391f4ec0b1349f4a1758a0ce2382b7c P ce1dd5b4e0014ba8a3fa68f2ba226e49 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd44949e40b247f980ba14623ba0dd34" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 37801 } } peers { permanent_uuid: "12806407b57c4ffbbf5f5baf439af582" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 45327 } } peers { permanent_uuid: "ce1dd5b4e0014ba8a3fa68f2ba226e49" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34123 } }
I20260709 13:18:17.223117 15374 ts_tablet_manager.cc:1434] T 8391f4ec0b1349f4a1758a0ce2382b7c P ce1dd5b4e0014ba8a3fa68f2ba226e49: Time spent starting tablet: real 0.034s	user 0.003s	sys 0.006s
I20260709 13:18:17.224089 15374 tablet_bootstrap.cc:492] T 58a8c0164db24ffd8e992493c228804c P ce1dd5b4e0014ba8a3fa68f2ba226e49: Bootstrap starting.
I20260709 13:18:17.225428 15256 raft_consensus.cc:2468] T 81fce322177440a690ee6f46d2d67ad6 P 12806407b57c4ffbbf5f5baf439af582 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fd44949e40b247f980ba14623ba0dd34 in term 1.
I20260709 13:18:17.226045 15329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81fce322177440a690ee6f46d2d67ad6" candidate_uuid: "fd44949e40b247f980ba14623ba0dd34" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce1dd5b4e0014ba8a3fa68f2ba226e49"
I20260709 13:18:17.228142 15375 tablet_bootstrap.cc:492] T 8391f4ec0b1349f4a1758a0ce2382b7c P 12806407b57c4ffbbf5f5baf439af582: Bootstrap starting.
W20260709 13:18:17.228778 15146 leader_election.cc:343] T 81fce322177440a690ee6f46d2d67ad6 P fd44949e40b247f980ba14623ba0dd34 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer ce1dd5b4e0014ba8a3fa68f2ba226e49 (127.14.177.131:34123): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:18:17.235119 15374 tablet_bootstrap.cc:654] T 58a8c0164db24ffd8e992493c228804c P ce1dd5b4e0014ba8a3fa68f2ba226e49: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:17.229408 15145 leader_election.cc:304] T 98f6aef87e78472c9987c75cac6575af P fd44949e40b247f980ba14623ba0dd34 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12806407b57c4ffbbf5f5baf439af582, fd44949e40b247f980ba14623ba0dd34; no voters: ce1dd5b4e0014ba8a3fa68f2ba226e49
I20260709 13:18:17.239180 15402 raft_consensus.cc:2804] T 98f6aef87e78472c9987c75cac6575af P fd44949e40b247f980ba14623ba0dd34 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:18:17.240280 15402 raft_consensus.cc:493] T 98f6aef87e78472c9987c75cac6575af P fd44949e40b247f980ba14623ba0dd34 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:17.240679 15402 raft_consensus.cc:3060] T 98f6aef87e78472c9987c75cac6575af P fd44949e40b247f980ba14623ba0dd34 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:17.241256 15221 leader_election.cc:304] T 853d00d5717a4ba1a941a81884cbb7a2 P 12806407b57c4ffbbf5f5baf439af582 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12806407b57c4ffbbf5f5baf439af582, fd44949e40b247f980ba14623ba0dd34; no voters: 
I20260709 13:18:17.241500 15145 leader_election.cc:304] T 81fce322177440a690ee6f46d2d67ad6 P fd44949e40b247f980ba14623ba0dd34 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12806407b57c4ffbbf5f5baf439af582, fd44949e40b247f980ba14623ba0dd34; no voters: ce1dd5b4e0014ba8a3fa68f2ba226e49
I20260709 13:18:17.245241 15404 raft_consensus.cc:2804] T 81fce322177440a690ee6f46d2d67ad6 P fd44949e40b247f980ba14623ba0dd34 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:18:17.244846 15380 raft_consensus.cc:2804] T 853d00d5717a4ba1a941a81884cbb7a2 P 12806407b57c4ffbbf5f5baf439af582 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:18:17.245159 15221 leader_election.cc:304] T 54dc07b181bd495d8c8e2cae81240f16 P 12806407b57c4ffbbf5f5baf439af582 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12806407b57c4ffbbf5f5baf439af582, fd44949e40b247f980ba14623ba0dd34; no voters: 
I20260709 13:18:17.246124 15404 raft_consensus.cc:697] T 81fce322177440a690ee6f46d2d67ad6 P fd44949e40b247f980ba14623ba0dd34 [term 1 LEADER]: Becoming Leader. State: Replica: fd44949e40b247f980ba14623ba0dd34, State: Running, Role: LEADER
I20260709 13:18:17.247437 15380 raft_consensus.cc:493] T 853d00d5717a4ba1a941a81884cbb7a2 P 12806407b57c4ffbbf5f5baf439af582 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:17.248003 15380 raft_consensus.cc:3060] T 853d00d5717a4ba1a941a81884cbb7a2 P 12806407b57c4ffbbf5f5baf439af582 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:17.248922 15404 consensus_queue.cc:237] T 81fce322177440a690ee6f46d2d67ad6 P fd44949e40b247f980ba14623ba0dd34 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd44949e40b247f980ba14623ba0dd34" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 37801 } } peers { permanent_uuid: "12806407b57c4ffbbf5f5baf439af582" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 45327 } } peers { permanent_uuid: "ce1dd5b4e0014ba8a3fa68f2ba226e49" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34123 } }
I20260709 13:18:17.250928 15379 raft_consensus.cc:515] T 54dc07b181bd495d8c8e2cae81240f16 P fd44949e40b247f980ba14623ba0dd34 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce1dd5b4e0014ba8a3fa68f2ba226e49" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34123 } } peers { permanent_uuid: "fd44949e40b247f980ba14623ba0dd34" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 37801 } } peers { permanent_uuid: "12806407b57c4ffbbf5f5baf439af582" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 45327 } }
I20260709 13:18:17.254065 15379 leader_election.cc:290] T 54dc07b181bd495d8c8e2cae81240f16 P fd44949e40b247f980ba14623ba0dd34 [CANDIDATE]: Term 1 election: Requested vote from peers ce1dd5b4e0014ba8a3fa68f2ba226e49 (127.14.177.131:34123), 12806407b57c4ffbbf5f5baf439af582 (127.14.177.130:45327)
I20260709 13:18:17.255117 15329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54dc07b181bd495d8c8e2cae81240f16" candidate_uuid: "fd44949e40b247f980ba14623ba0dd34" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce1dd5b4e0014ba8a3fa68f2ba226e49"
I20260709 13:18:17.255772 15329 raft_consensus.cc:3060] T 54dc07b181bd495d8c8e2cae81240f16 P ce1dd5b4e0014ba8a3fa68f2ba226e49 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:17.260663 15256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54dc07b181bd495d8c8e2cae81240f16" candidate_uuid: "fd44949e40b247f980ba14623ba0dd34" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12806407b57c4ffbbf5f5baf439af582"
I20260709 13:18:17.261673 15256 raft_consensus.cc:3060] T 54dc07b181bd495d8c8e2cae81240f16 P 12806407b57c4ffbbf5f5baf439af582 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:17.262115 15402 raft_consensus.cc:515] T 98f6aef87e78472c9987c75cac6575af P fd44949e40b247f980ba14623ba0dd34 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd44949e40b247f980ba14623ba0dd34" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 37801 } } peers { permanent_uuid: "12806407b57c4ffbbf5f5baf439af582" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 45327 } } peers { permanent_uuid: "ce1dd5b4e0014ba8a3fa68f2ba226e49" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34123 } }
I20260709 13:18:17.266989 15221 leader_election.cc:304] T 6f3b305de2a04b54aef8674096fa6358 P 12806407b57c4ffbbf5f5baf439af582 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12806407b57c4ffbbf5f5baf439af582, fd44949e40b247f980ba14623ba0dd34; no voters: 
I20260709 13:18:17.265458 15375 tablet_bootstrap.cc:654] T 8391f4ec0b1349f4a1758a0ce2382b7c P 12806407b57c4ffbbf5f5baf439af582: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:17.268797 15256 raft_consensus.cc:2468] T 54dc07b181bd495d8c8e2cae81240f16 P 12806407b57c4ffbbf5f5baf439af582 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fd44949e40b247f980ba14623ba0dd34 in term 1.
I20260709 13:18:17.269636 15402 leader_election.cc:290] T 98f6aef87e78472c9987c75cac6575af P fd44949e40b247f980ba14623ba0dd34 [CANDIDATE]: Term 1 election: Requested vote from peers 12806407b57c4ffbbf5f5baf439af582 (127.14.177.130:45327), ce1dd5b4e0014ba8a3fa68f2ba226e49 (127.14.177.131:34123)
I20260709 13:18:17.270396 15145 leader_election.cc:304] T 54dc07b181bd495d8c8e2cae81240f16 P fd44949e40b247f980ba14623ba0dd34 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12806407b57c4ffbbf5f5baf439af582, fd44949e40b247f980ba14623ba0dd34; no voters: 
I20260709 13:18:17.270763 15330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98f6aef87e78472c9987c75cac6575af" candidate_uuid: "fd44949e40b247f980ba14623ba0dd34" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce1dd5b4e0014ba8a3fa68f2ba226e49"
I20260709 13:18:17.272039 15409 raft_consensus.cc:2764] T 54dc07b181bd495d8c8e2cae81240f16 P 12806407b57c4ffbbf5f5baf439af582 [term 1 FOLLOWER]: Leader pre-election decision vote started in defunct term 0: won
I20260709 13:18:17.272746 15409 raft_consensus.cc:2804] T 6f3b305de2a04b54aef8674096fa6358 P 12806407b57c4ffbbf5f5baf439af582 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:18:17.273192 15409 raft_consensus.cc:493] T 6f3b305de2a04b54aef8674096fa6358 P 12806407b57c4ffbbf5f5baf439af582 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:17.273515 15409 raft_consensus.cc:3060] T 6f3b305de2a04b54aef8674096fa6358 P 12806407b57c4ffbbf5f5baf439af582 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:18:17.276069 15146 leader_election.cc:343] T 98f6aef87e78472c9987c75cac6575af P fd44949e40b247f980ba14623ba0dd34 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer ce1dd5b4e0014ba8a3fa68f2ba226e49 (127.14.177.131:34123): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:18:17.277109 15256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98f6aef87e78472c9987c75cac6575af" candidate_uuid: "fd44949e40b247f980ba14623ba0dd34" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12806407b57c4ffbbf5f5baf439af582"
I20260709 13:18:17.277773 15256 raft_consensus.cc:3060] T 98f6aef87e78472c9987c75cac6575af P 12806407b57c4ffbbf5f5baf439af582 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:17.283457 15402 raft_consensus.cc:2804] T 54dc07b181bd495d8c8e2cae81240f16 P fd44949e40b247f980ba14623ba0dd34 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:18:17.284191 15402 raft_consensus.cc:697] T 54dc07b181bd495d8c8e2cae81240f16 P fd44949e40b247f980ba14623ba0dd34 [term 1 LEADER]: Becoming Leader. State: Replica: fd44949e40b247f980ba14623ba0dd34, State: Running, Role: LEADER
I20260709 13:18:17.285387 15402 consensus_queue.cc:237] T 54dc07b181bd495d8c8e2cae81240f16 P fd44949e40b247f980ba14623ba0dd34 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce1dd5b4e0014ba8a3fa68f2ba226e49" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34123 } } peers { permanent_uuid: "fd44949e40b247f980ba14623ba0dd34" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 37801 } } peers { permanent_uuid: "12806407b57c4ffbbf5f5baf439af582" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 45327 } }
I20260709 13:18:17.266227 15329 raft_consensus.cc:2468] T 54dc07b181bd495d8c8e2cae81240f16 P ce1dd5b4e0014ba8a3fa68f2ba226e49 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fd44949e40b247f980ba14623ba0dd34 in term 1.
I20260709 13:18:17.293047 15256 raft_consensus.cc:2468] T 98f6aef87e78472c9987c75cac6575af P 12806407b57c4ffbbf5f5baf439af582 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fd44949e40b247f980ba14623ba0dd34 in term 1.
I20260709 13:18:17.265689 15374 tablet_bootstrap.cc:492] T 58a8c0164db24ffd8e992493c228804c P ce1dd5b4e0014ba8a3fa68f2ba226e49: No bootstrap required, opened a new log
I20260709 13:18:17.296361 15374 ts_tablet_manager.cc:1403] T 58a8c0164db24ffd8e992493c228804c P ce1dd5b4e0014ba8a3fa68f2ba226e49: Time spent bootstrapping tablet: real 0.072s	user 0.008s	sys 0.007s
W20260709 13:18:17.296774 15221 leader_election.cc:343] T 853d00d5717a4ba1a941a81884cbb7a2 P 12806407b57c4ffbbf5f5baf439af582 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ce1dd5b4e0014ba8a3fa68f2ba226e49 (127.14.177.131:34123): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:18:17.297034 15145 leader_election.cc:304] T 98f6aef87e78472c9987c75cac6575af P fd44949e40b247f980ba14623ba0dd34 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12806407b57c4ffbbf5f5baf439af582, fd44949e40b247f980ba14623ba0dd34; no voters: ce1dd5b4e0014ba8a3fa68f2ba226e49
I20260709 13:18:17.299381 15404 raft_consensus.cc:2804] T 98f6aef87e78472c9987c75cac6575af P fd44949e40b247f980ba14623ba0dd34 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:18:17.299554 15374 raft_consensus.cc:359] T 58a8c0164db24ffd8e992493c228804c P ce1dd5b4e0014ba8a3fa68f2ba226e49 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd44949e40b247f980ba14623ba0dd34" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 37801 } } peers { permanent_uuid: "12806407b57c4ffbbf5f5baf439af582" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 45327 } } peers { permanent_uuid: "ce1dd5b4e0014ba8a3fa68f2ba226e49" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34123 } }
I20260709 13:18:17.294656 15409 raft_consensus.cc:515] T 6f3b305de2a04b54aef8674096fa6358 P 12806407b57c4ffbbf5f5baf439af582 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd44949e40b247f980ba14623ba0dd34" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 37801 } } peers { permanent_uuid: "12806407b57c4ffbbf5f5baf439af582" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 45327 } } peers { permanent_uuid: "ce1dd5b4e0014ba8a3fa68f2ba226e49" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34123 } }
I20260709 13:18:17.293933 15075 catalog_manager.cc:5697] T 81fce322177440a690ee6f46d2d67ad6 P fd44949e40b247f980ba14623ba0dd34 reported cstate change: term changed from 0 to 1, leader changed from <none> to fd44949e40b247f980ba14623ba0dd34 (127.14.177.129). New cstate: current_term: 1 leader_uuid: "fd44949e40b247f980ba14623ba0dd34" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd44949e40b247f980ba14623ba0dd34" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 37801 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "12806407b57c4ffbbf5f5baf439af582" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 45327 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ce1dd5b4e0014ba8a3fa68f2ba226e49" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34123 } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:17.300303 15374 raft_consensus.cc:385] T 58a8c0164db24ffd8e992493c228804c P ce1dd5b4e0014ba8a3fa68f2ba226e49 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:17.300638 15374 raft_consensus.cc:740] T 58a8c0164db24ffd8e992493c228804c P ce1dd5b4e0014ba8a3fa68f2ba226e49 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ce1dd5b4e0014ba8a3fa68f2ba226e49, State: Initialized, Role: FOLLOWER
I20260709 13:18:17.301517 15374 consensus_queue.cc:260] T 58a8c0164db24ffd8e992493c228804c P ce1dd5b4e0014ba8a3fa68f2ba226e49 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd44949e40b247f980ba14623ba0dd34" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 37801 } } peers { permanent_uuid: "12806407b57c4ffbbf5f5baf439af582" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 45327 } } peers { permanent_uuid: "ce1dd5b4e0014ba8a3fa68f2ba226e49" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34123 } }
I20260709 13:18:17.299975 15404 raft_consensus.cc:697] T 98f6aef87e78472c9987c75cac6575af P fd44949e40b247f980ba14623ba0dd34 [term 1 LEADER]: Becoming Leader. State: Replica: fd44949e40b247f980ba14623ba0dd34, State: Running, Role: LEADER
I20260709 13:18:17.272780 15380 raft_consensus.cc:515] T 853d00d5717a4ba1a941a81884cbb7a2 P 12806407b57c4ffbbf5f5baf439af582 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd44949e40b247f980ba14623ba0dd34" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 37801 } } peers { permanent_uuid: "12806407b57c4ffbbf5f5baf439af582" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 45327 } } peers { permanent_uuid: "ce1dd5b4e0014ba8a3fa68f2ba226e49" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34123 } }
I20260709 13:18:17.302641 15180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f3b305de2a04b54aef8674096fa6358" candidate_uuid: "12806407b57c4ffbbf5f5baf439af582" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fd44949e40b247f980ba14623ba0dd34"
I20260709 13:18:17.307786 15180 raft_consensus.cc:3060] T 6f3b305de2a04b54aef8674096fa6358 P fd44949e40b247f980ba14623ba0dd34 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:17.304417 15374 ts_tablet_manager.cc:1434] T 58a8c0164db24ffd8e992493c228804c P ce1dd5b4e0014ba8a3fa68f2ba226e49: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.001s
I20260709 13:18:17.310454 15374 tablet_bootstrap.cc:492] T 81fce322177440a690ee6f46d2d67ad6 P ce1dd5b4e0014ba8a3fa68f2ba226e49: Bootstrap starting.
I20260709 13:18:17.305130 15329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f3b305de2a04b54aef8674096fa6358" candidate_uuid: "12806407b57c4ffbbf5f5baf439af582" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce1dd5b4e0014ba8a3fa68f2ba226e49"
I20260709 13:18:17.311442 15329 raft_consensus.cc:3060] T 6f3b305de2a04b54aef8674096fa6358 P ce1dd5b4e0014ba8a3fa68f2ba226e49 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:17.322031 15329 raft_consensus.cc:2468] T 6f3b305de2a04b54aef8674096fa6358 P ce1dd5b4e0014ba8a3fa68f2ba226e49 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12806407b57c4ffbbf5f5baf439af582 in term 1.
I20260709 13:18:17.323222 15221 leader_election.cc:304] T 6f3b305de2a04b54aef8674096fa6358 P 12806407b57c4ffbbf5f5baf439af582 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12806407b57c4ffbbf5f5baf439af582, ce1dd5b4e0014ba8a3fa68f2ba226e49; no voters: 
I20260709 13:18:17.323735 15374 tablet_bootstrap.cc:654] T 81fce322177440a690ee6f46d2d67ad6 P ce1dd5b4e0014ba8a3fa68f2ba226e49: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:17.324265 15411 raft_consensus.cc:2804] T 6f3b305de2a04b54aef8674096fa6358 P 12806407b57c4ffbbf5f5baf439af582 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:18:17.327363 15181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "853d00d5717a4ba1a941a81884cbb7a2" candidate_uuid: "12806407b57c4ffbbf5f5baf439af582" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fd44949e40b247f980ba14623ba0dd34"
I20260709 13:18:17.327956 15181 raft_consensus.cc:3060] T 853d00d5717a4ba1a941a81884cbb7a2 P fd44949e40b247f980ba14623ba0dd34 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:17.328635 15330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "853d00d5717a4ba1a941a81884cbb7a2" candidate_uuid: "12806407b57c4ffbbf5f5baf439af582" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce1dd5b4e0014ba8a3fa68f2ba226e49"
W20260709 13:18:17.329980 15221 leader_election.cc:343] T 853d00d5717a4ba1a941a81884cbb7a2 P 12806407b57c4ffbbf5f5baf439af582 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer ce1dd5b4e0014ba8a3fa68f2ba226e49 (127.14.177.131:34123): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:18:17.330495 15380 leader_election.cc:290] T 853d00d5717a4ba1a941a81884cbb7a2 P 12806407b57c4ffbbf5f5baf439af582 [CANDIDATE]: Term 1 election: Requested vote from peers fd44949e40b247f980ba14623ba0dd34 (127.14.177.129:37801), ce1dd5b4e0014ba8a3fa68f2ba226e49 (127.14.177.131:34123)
I20260709 13:18:17.301954 15409 leader_election.cc:290] T 6f3b305de2a04b54aef8674096fa6358 P 12806407b57c4ffbbf5f5baf439af582 [CANDIDATE]: Term 1 election: Requested vote from peers fd44949e40b247f980ba14623ba0dd34 (127.14.177.129:37801), ce1dd5b4e0014ba8a3fa68f2ba226e49 (127.14.177.131:34123)
I20260709 13:18:17.306033 15404 consensus_queue.cc:237] T 98f6aef87e78472c9987c75cac6575af P fd44949e40b247f980ba14623ba0dd34 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd44949e40b247f980ba14623ba0dd34" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 37801 } } peers { permanent_uuid: "12806407b57c4ffbbf5f5baf439af582" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 45327 } } peers { permanent_uuid: "ce1dd5b4e0014ba8a3fa68f2ba226e49" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34123 } }
I20260709 13:18:17.318871 15180 raft_consensus.cc:2468] T 6f3b305de2a04b54aef8674096fa6358 P fd44949e40b247f980ba14623ba0dd34 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12806407b57c4ffbbf5f5baf439af582 in term 1.
I20260709 13:18:17.333158 15411 raft_consensus.cc:697] T 6f3b305de2a04b54aef8674096fa6358 P 12806407b57c4ffbbf5f5baf439af582 [term 1 LEADER]: Becoming Leader. State: Replica: 12806407b57c4ffbbf5f5baf439af582, State: Running, Role: LEADER
I20260709 13:18:17.334242 15411 consensus_queue.cc:237] T 6f3b305de2a04b54aef8674096fa6358 P 12806407b57c4ffbbf5f5baf439af582 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd44949e40b247f980ba14623ba0dd34" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 37801 } } peers { permanent_uuid: "12806407b57c4ffbbf5f5baf439af582" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 45327 } } peers { permanent_uuid: "ce1dd5b4e0014ba8a3fa68f2ba226e49" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34123 } }
I20260709 13:18:17.335947 15181 raft_consensus.cc:2468] T 853d00d5717a4ba1a941a81884cbb7a2 P fd44949e40b247f980ba14623ba0dd34 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12806407b57c4ffbbf5f5baf439af582 in term 1.
I20260709 13:18:17.337215 15221 leader_election.cc:304] T 853d00d5717a4ba1a941a81884cbb7a2 P 12806407b57c4ffbbf5f5baf439af582 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 12806407b57c4ffbbf5f5baf439af582, fd44949e40b247f980ba14623ba0dd34; no voters: ce1dd5b4e0014ba8a3fa68f2ba226e49
I20260709 13:18:17.338526 15409 raft_consensus.cc:2804] T 853d00d5717a4ba1a941a81884cbb7a2 P 12806407b57c4ffbbf5f5baf439af582 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:18:17.339082 15409 raft_consensus.cc:697] T 853d00d5717a4ba1a941a81884cbb7a2 P 12806407b57c4ffbbf5f5baf439af582 [term 1 LEADER]: Becoming Leader. State: Replica: 12806407b57c4ffbbf5f5baf439af582, State: Running, Role: LEADER
I20260709 13:18:17.340087 15375 tablet_bootstrap.cc:492] T 8391f4ec0b1349f4a1758a0ce2382b7c P 12806407b57c4ffbbf5f5baf439af582: No bootstrap required, opened a new log
I20260709 13:18:17.345623 15375 ts_tablet_manager.cc:1403] T 8391f4ec0b1349f4a1758a0ce2382b7c P 12806407b57c4ffbbf5f5baf439af582: Time spent bootstrapping tablet: real 0.118s	user 0.008s	sys 0.010s
I20260709 13:18:17.340150 15409 consensus_queue.cc:237] T 853d00d5717a4ba1a941a81884cbb7a2 P 12806407b57c4ffbbf5f5baf439af582 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd44949e40b247f980ba14623ba0dd34" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 37801 } } peers { permanent_uuid: "12806407b57c4ffbbf5f5baf439af582" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 45327 } } peers { permanent_uuid: "ce1dd5b4e0014ba8a3fa68f2ba226e49" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34123 } }
I20260709 13:18:17.346105 15374 tablet_bootstrap.cc:492] T 81fce322177440a690ee6f46d2d67ad6 P ce1dd5b4e0014ba8a3fa68f2ba226e49: No bootstrap required, opened a new log
I20260709 13:18:17.346616 15374 ts_tablet_manager.cc:1403] T 81fce322177440a690ee6f46d2d67ad6 P ce1dd5b4e0014ba8a3fa68f2ba226e49: Time spent bootstrapping tablet: real 0.036s	user 0.007s	sys 0.010s
I20260709 13:18:17.349639 15374 raft_consensus.cc:359] T 81fce322177440a690ee6f46d2d67ad6 P ce1dd5b4e0014ba8a3fa68f2ba226e49 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd44949e40b247f980ba14623ba0dd34" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 37801 } } peers { permanent_uuid: "12806407b57c4ffbbf5f5baf439af582" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 45327 } } peers { permanent_uuid: "ce1dd5b4e0014ba8a3fa68f2ba226e49" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34123 } }
I20260709 13:18:17.350574 15374 raft_consensus.cc:385] T 81fce322177440a690ee6f46d2d67ad6 P ce1dd5b4e0014ba8a3fa68f2ba226e49 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:17.354722 15375 raft_consensus.cc:359] T 8391f4ec0b1349f4a1758a0ce2382b7c P 12806407b57c4ffbbf5f5baf439af582 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd44949e40b247f980ba14623ba0dd34" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 37801 } } peers { permanent_uuid: "12806407b57c4ffbbf5f5baf439af582" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 45327 } } peers { permanent_uuid: "ce1dd5b4e0014ba8a3fa68f2ba226e49" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34123 } }
I20260709 13:18:17.355528 15375 raft_consensus.cc:385] T 8391f4ec0b1349f4a1758a0ce2382b7c P 12806407b57c4ffbbf5f5baf439af582 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:17.355830 15375 raft_consensus.cc:740] T 8391f4ec0b1349f4a1758a0ce2382b7c P 12806407b57c4ffbbf5f5baf439af582 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 12806407b57c4ffbbf5f5baf439af582, State: Initialized, Role: FOLLOWER
I20260709 13:18:17.356685 15375 consensus_queue.cc:260] T 8391f4ec0b1349f4a1758a0ce2382b7c P 12806407b57c4ffbbf5f5baf439af582 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd44949e40b247f980ba14623ba0dd34" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 37801 } } peers { permanent_uuid: "12806407b57c4ffbbf5f5baf439af582" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 45327 } } peers { permanent_uuid: "ce1dd5b4e0014ba8a3fa68f2ba226e49" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34123 } }
I20260709 13:18:17.359047 15375 ts_tablet_manager.cc:1434] T 8391f4ec0b1349f4a1758a0ce2382b7c P 12806407b57c4ffbbf5f5baf439af582: Time spent starting tablet: real 0.013s	user 0.004s	sys 0.000s
I20260709 13:18:17.360091 15375 tablet_bootstrap.cc:492] T 28ac227079274b8eb6a3e35e4bf34e30 P 12806407b57c4ffbbf5f5baf439af582: Bootstrap starting.
I20260709 13:18:17.350929 15374 raft_consensus.cc:740] T 81fce322177440a690ee6f46d2d67ad6 P ce1dd5b4e0014ba8a3fa68f2ba226e49 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ce1dd5b4e0014ba8a3fa68f2ba226e49, State: Initialized, Role: FOLLOWER
I20260709 13:18:17.362623 15374 consensus_queue.cc:260] T 81fce322177440a690ee6f46d2d67ad6 P ce1dd5b4e0014ba8a3fa68f2ba226e49 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd44949e40b247f980ba14623ba0dd34" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 37801 } } peers { permanent_uuid: "12806407b57c4ffbbf5f5baf439af582" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 45327 } } peers { permanent_uuid: "ce1dd5b4e0014ba8a3fa68f2ba226e49" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34123 } }
I20260709 13:18:17.364938 15374 ts_tablet_manager.cc:1434] T 81fce322177440a690ee6f46d2d67ad6 P ce1dd5b4e0014ba8a3fa68f2ba226e49: Time spent starting tablet: real 0.018s	user 0.005s	sys 0.002s
I20260709 13:18:17.366016 15374 tablet_bootstrap.cc:492] T 98f6aef87e78472c9987c75cac6575af P ce1dd5b4e0014ba8a3fa68f2ba226e49: Bootstrap starting.
I20260709 13:18:17.368779 15375 tablet_bootstrap.cc:654] T 28ac227079274b8eb6a3e35e4bf34e30 P 12806407b57c4ffbbf5f5baf439af582: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:17.369007 15381 raft_consensus.cc:493] T 28ac227079274b8eb6a3e35e4bf34e30 P ce1dd5b4e0014ba8a3fa68f2ba226e49 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:17.369550 15381 raft_consensus.cc:515] T 28ac227079274b8eb6a3e35e4bf34e30 P ce1dd5b4e0014ba8a3fa68f2ba226e49 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd44949e40b247f980ba14623ba0dd34" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 37801 } } peers { permanent_uuid: "12806407b57c4ffbbf5f5baf439af582" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 45327 } } peers { permanent_uuid: "ce1dd5b4e0014ba8a3fa68f2ba226e49" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34123 } }
I20260709 13:18:17.372962 15381 leader_election.cc:290] T 28ac227079274b8eb6a3e35e4bf34e30 P ce1dd5b4e0014ba8a3fa68f2ba226e49 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fd44949e40b247f980ba14623ba0dd34 (127.14.177.129:37801), 12806407b57c4ffbbf5f5baf439af582 (127.14.177.130:45327)
I20260709 13:18:17.373039 15374 tablet_bootstrap.cc:654] T 98f6aef87e78472c9987c75cac6575af P ce1dd5b4e0014ba8a3fa68f2ba226e49: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:17.373147 15256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28ac227079274b8eb6a3e35e4bf34e30" candidate_uuid: "ce1dd5b4e0014ba8a3fa68f2ba226e49" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12806407b57c4ffbbf5f5baf439af582" is_pre_election: true
I20260709 13:18:17.375533 15181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28ac227079274b8eb6a3e35e4bf34e30" candidate_uuid: "ce1dd5b4e0014ba8a3fa68f2ba226e49" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fd44949e40b247f980ba14623ba0dd34" is_pre_election: true
I20260709 13:18:17.376506 15181 raft_consensus.cc:2468] T 28ac227079274b8eb6a3e35e4bf34e30 P fd44949e40b247f980ba14623ba0dd34 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce1dd5b4e0014ba8a3fa68f2ba226e49 in term 0.
W20260709 13:18:17.377468 15294 leader_election.cc:343] T 28ac227079274b8eb6a3e35e4bf34e30 P ce1dd5b4e0014ba8a3fa68f2ba226e49 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 12806407b57c4ffbbf5f5baf439af582 (127.14.177.130:45327): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:18:17.378821 15381 raft_consensus.cc:493] T 8391f4ec0b1349f4a1758a0ce2382b7c P ce1dd5b4e0014ba8a3fa68f2ba226e49 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:17.379284 15381 raft_consensus.cc:515] T 8391f4ec0b1349f4a1758a0ce2382b7c P ce1dd5b4e0014ba8a3fa68f2ba226e49 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd44949e40b247f980ba14623ba0dd34" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 37801 } } peers { permanent_uuid: "12806407b57c4ffbbf5f5baf439af582" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 45327 } } peers { permanent_uuid: "ce1dd5b4e0014ba8a3fa68f2ba226e49" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34123 } }
I20260709 13:18:17.381265 15381 leader_election.cc:290] T 8391f4ec0b1349f4a1758a0ce2382b7c P ce1dd5b4e0014ba8a3fa68f2ba226e49 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fd44949e40b247f980ba14623ba0dd34 (127.14.177.129:37801), 12806407b57c4ffbbf5f5baf439af582 (127.14.177.130:45327)
I20260709 13:18:17.378141 15295 leader_election.cc:304] T 28ac227079274b8eb6a3e35e4bf34e30 P ce1dd5b4e0014ba8a3fa68f2ba226e49 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ce1dd5b4e0014ba8a3fa68f2ba226e49, fd44949e40b247f980ba14623ba0dd34; no voters: 12806407b57c4ffbbf5f5baf439af582
I20260709 13:18:17.389056 15381 raft_consensus.cc:2804] T 28ac227079274b8eb6a3e35e4bf34e30 P ce1dd5b4e0014ba8a3fa68f2ba226e49 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:18:17.389430 15381 raft_consensus.cc:493] T 28ac227079274b8eb6a3e35e4bf34e30 P ce1dd5b4e0014ba8a3fa68f2ba226e49 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:17.389478 15375 tablet_bootstrap.cc:492] T 28ac227079274b8eb6a3e35e4bf34e30 P 12806407b57c4ffbbf5f5baf439af582: No bootstrap required, opened a new log
I20260709 13:18:17.389851 15381 raft_consensus.cc:3060] T 28ac227079274b8eb6a3e35e4bf34e30 P ce1dd5b4e0014ba8a3fa68f2ba226e49 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:17.390251 15375 ts_tablet_manager.cc:1403] T 28ac227079274b8eb6a3e35e4bf34e30 P 12806407b57c4ffbbf5f5baf439af582: Time spent bootstrapping tablet: real 0.030s	user 0.004s	sys 0.014s
I20260709 13:18:17.393430 15375 raft_consensus.cc:359] T 28ac227079274b8eb6a3e35e4bf34e30 P 12806407b57c4ffbbf5f5baf439af582 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd44949e40b247f980ba14623ba0dd34" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 37801 } } peers { permanent_uuid: "12806407b57c4ffbbf5f5baf439af582" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 45327 } } peers { permanent_uuid: "ce1dd5b4e0014ba8a3fa68f2ba226e49" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34123 } }
I20260709 13:18:17.394335 15375 raft_consensus.cc:385] T 28ac227079274b8eb6a3e35e4bf34e30 P 12806407b57c4ffbbf5f5baf439af582 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:17.394675 15375 raft_consensus.cc:740] T 28ac227079274b8eb6a3e35e4bf34e30 P 12806407b57c4ffbbf5f5baf439af582 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 12806407b57c4ffbbf5f5baf439af582, State: Initialized, Role: FOLLOWER
I20260709 13:18:17.395457 15375 consensus_queue.cc:260] T 28ac227079274b8eb6a3e35e4bf34e30 P 12806407b57c4ffbbf5f5baf439af582 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd44949e40b247f980ba14623ba0dd34" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 37801 } } peers { permanent_uuid: "12806407b57c4ffbbf5f5baf439af582" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 45327 } } peers { permanent_uuid: "ce1dd5b4e0014ba8a3fa68f2ba226e49" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34123 } }
I20260709 13:18:17.395972 15181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8391f4ec0b1349f4a1758a0ce2382b7c" candidate_uuid: "ce1dd5b4e0014ba8a3fa68f2ba226e49" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fd44949e40b247f980ba14623ba0dd34" is_pre_election: true
I20260709 13:18:17.396579 15181 raft_consensus.cc:2468] T 8391f4ec0b1349f4a1758a0ce2382b7c P fd44949e40b247f980ba14623ba0dd34 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce1dd5b4e0014ba8a3fa68f2ba226e49 in term 0.
I20260709 13:18:17.397951 15375 ts_tablet_manager.cc:1434] T 28ac227079274b8eb6a3e35e4bf34e30 P 12806407b57c4ffbbf5f5baf439af582: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:18:17.398800 15256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8391f4ec0b1349f4a1758a0ce2382b7c" candidate_uuid: "ce1dd5b4e0014ba8a3fa68f2ba226e49" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12806407b57c4ffbbf5f5baf439af582" is_pre_election: true
I20260709 13:18:17.399394 15256 raft_consensus.cc:2468] T 8391f4ec0b1349f4a1758a0ce2382b7c P 12806407b57c4ffbbf5f5baf439af582 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce1dd5b4e0014ba8a3fa68f2ba226e49 in term 0.
I20260709 13:18:17.400619 15294 leader_election.cc:304] T 8391f4ec0b1349f4a1758a0ce2382b7c P ce1dd5b4e0014ba8a3fa68f2ba226e49 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12806407b57c4ffbbf5f5baf439af582, ce1dd5b4e0014ba8a3fa68f2ba226e49; no voters: 
I20260709 13:18:17.405507 15375 tablet_bootstrap.cc:492] T fbf069cd9abc4875a3d03f06ff06544e P 12806407b57c4ffbbf5f5baf439af582: Bootstrap starting.
I20260709 13:18:17.409098 15076 catalog_manager.cc:5697] T 54dc07b181bd495d8c8e2cae81240f16 P fd44949e40b247f980ba14623ba0dd34 reported cstate change: term changed from 0 to 1, leader changed from <none> to fd44949e40b247f980ba14623ba0dd34 (127.14.177.129). New cstate: current_term: 1 leader_uuid: "fd44949e40b247f980ba14623ba0dd34" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce1dd5b4e0014ba8a3fa68f2ba226e49" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34123 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fd44949e40b247f980ba14623ba0dd34" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 37801 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "12806407b57c4ffbbf5f5baf439af582" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 45327 } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:17.405526 15077 catalog_manager.cc:5697] T 6f3b305de2a04b54aef8674096fa6358 P 12806407b57c4ffbbf5f5baf439af582 reported cstate change: term changed from 0 to 1, leader changed from <none> to 12806407b57c4ffbbf5f5baf439af582 (127.14.177.130). New cstate: current_term: 1 leader_uuid: "12806407b57c4ffbbf5f5baf439af582" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd44949e40b247f980ba14623ba0dd34" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 37801 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "12806407b57c4ffbbf5f5baf439af582" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 45327 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ce1dd5b4e0014ba8a3fa68f2ba226e49" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34123 } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:17.415248 15425 raft_consensus.cc:2804] T 8391f4ec0b1349f4a1758a0ce2382b7c P ce1dd5b4e0014ba8a3fa68f2ba226e49 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:18:17.424652 15425 raft_consensus.cc:493] T 8391f4ec0b1349f4a1758a0ce2382b7c P ce1dd5b4e0014ba8a3fa68f2ba226e49 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:17.425197 15425 raft_consensus.cc:3060] T 8391f4ec0b1349f4a1758a0ce2382b7c P ce1dd5b4e0014ba8a3fa68f2ba226e49 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:17.432335 15425 raft_consensus.cc:515] T 8391f4ec0b1349f4a1758a0ce2382b7c P ce1dd5b4e0014ba8a3fa68f2ba226e49 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd44949e40b247f980ba14623ba0dd34" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 37801 } } peers { permanent_uuid: "12806407b57c4ffbbf5f5baf439af582" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 45327 } } peers { permanent_uuid: "ce1dd5b4e0014ba8a3fa68f2ba226e49" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34123 } }
I20260709 13:18:17.424300 15375 tablet_bootstrap.cc:654] T fbf069cd9abc4875a3d03f06ff06544e P 12806407b57c4ffbbf5f5baf439af582: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:17.435240 15075 catalog_manager.cc:5697] T 853d00d5717a4ba1a941a81884cbb7a2 P 12806407b57c4ffbbf5f5baf439af582 reported cstate change: term changed from 0 to 1, leader changed from <none> to 12806407b57c4ffbbf5f5baf439af582 (127.14.177.130). New cstate: current_term: 1 leader_uuid: "12806407b57c4ffbbf5f5baf439af582" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd44949e40b247f980ba14623ba0dd34" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 37801 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "12806407b57c4ffbbf5f5baf439af582" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 45327 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ce1dd5b4e0014ba8a3fa68f2ba226e49" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34123 } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:17.438796 15181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8391f4ec0b1349f4a1758a0ce2382b7c" candidate_uuid: "ce1dd5b4e0014ba8a3fa68f2ba226e49" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fd44949e40b247f980ba14623ba0dd34"
I20260709 13:18:17.439404 15181 raft_consensus.cc:3060] T 8391f4ec0b1349f4a1758a0ce2382b7c P fd44949e40b247f980ba14623ba0dd34 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:17.446244 15181 raft_consensus.cc:2468] T 8391f4ec0b1349f4a1758a0ce2382b7c P fd44949e40b247f980ba14623ba0dd34 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce1dd5b4e0014ba8a3fa68f2ba226e49 in term 1.
I20260709 13:18:17.447701 15295 leader_election.cc:304] T 8391f4ec0b1349f4a1758a0ce2382b7c P ce1dd5b4e0014ba8a3fa68f2ba226e49 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce1dd5b4e0014ba8a3fa68f2ba226e49, fd44949e40b247f980ba14623ba0dd34; no voters: 
I20260709 13:18:17.448266 15404 raft_consensus.cc:493] T fbf069cd9abc4875a3d03f06ff06544e P fd44949e40b247f980ba14623ba0dd34 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:17.448819 15404 raft_consensus.cc:515] T fbf069cd9abc4875a3d03f06ff06544e P fd44949e40b247f980ba14623ba0dd34 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd44949e40b247f980ba14623ba0dd34" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 37801 } } peers { permanent_uuid: "12806407b57c4ffbbf5f5baf439af582" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 45327 } } peers { permanent_uuid: "ce1dd5b4e0014ba8a3fa68f2ba226e49" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34123 } }
I20260709 13:18:17.449708 15076 catalog_manager.cc:5697] T 98f6aef87e78472c9987c75cac6575af P fd44949e40b247f980ba14623ba0dd34 reported cstate change: term changed from 0 to 1, leader changed from <none> to fd44949e40b247f980ba14623ba0dd34 (127.14.177.129). New cstate: current_term: 1 leader_uuid: "fd44949e40b247f980ba14623ba0dd34" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd44949e40b247f980ba14623ba0dd34" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 37801 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "12806407b57c4ffbbf5f5baf439af582" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 45327 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ce1dd5b4e0014ba8a3fa68f2ba226e49" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34123 } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:17.452410 15329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbf069cd9abc4875a3d03f06ff06544e" candidate_uuid: "fd44949e40b247f980ba14623ba0dd34" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce1dd5b4e0014ba8a3fa68f2ba226e49" is_pre_election: true
I20260709 13:18:17.448879 15381 raft_consensus.cc:515] T 28ac227079274b8eb6a3e35e4bf34e30 P ce1dd5b4e0014ba8a3fa68f2ba226e49 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd44949e40b247f980ba14623ba0dd34" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 37801 } } peers { permanent_uuid: "12806407b57c4ffbbf5f5baf439af582" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 45327 } } peers { permanent_uuid: "ce1dd5b4e0014ba8a3fa68f2ba226e49" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34123 } }
I20260709 13:18:17.453019 15329 raft_consensus.cc:2468] T fbf069cd9abc4875a3d03f06ff06544e P ce1dd5b4e0014ba8a3fa68f2ba226e49 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fd44949e40b247f980ba14623ba0dd34 in term 0.
I20260709 13:18:17.454447 15146 leader_election.cc:304] T fbf069cd9abc4875a3d03f06ff06544e P fd44949e40b247f980ba14623ba0dd34 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce1dd5b4e0014ba8a3fa68f2ba226e49, fd44949e40b247f980ba14623ba0dd34; no voters: 
I20260709 13:18:17.454901 15381 leader_election.cc:290] T 28ac227079274b8eb6a3e35e4bf34e30 P ce1dd5b4e0014ba8a3fa68f2ba226e49 [CANDIDATE]: Term 1 election: Requested vote from peers fd44949e40b247f980ba14623ba0dd34 (127.14.177.129:37801), 12806407b57c4ffbbf5f5baf439af582 (127.14.177.130:45327)
I20260709 13:18:17.455344 15402 raft_consensus.cc:2804] T fbf069cd9abc4875a3d03f06ff06544e P fd44949e40b247f980ba14623ba0dd34 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:18:17.455935 15402 raft_consensus.cc:493] T fbf069cd9abc4875a3d03f06ff06544e P fd44949e40b247f980ba14623ba0dd34 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:17.456409 15256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28ac227079274b8eb6a3e35e4bf34e30" candidate_uuid: "ce1dd5b4e0014ba8a3fa68f2ba226e49" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12806407b57c4ffbbf5f5baf439af582"
I20260709 13:18:17.456421 15402 raft_consensus.cc:3060] T fbf069cd9abc4875a3d03f06ff06544e P fd44949e40b247f980ba14623ba0dd34 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:17.457177 15256 raft_consensus.cc:3060] T 28ac227079274b8eb6a3e35e4bf34e30 P 12806407b57c4ffbbf5f5baf439af582 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:17.460212 15404 leader_election.cc:290] T fbf069cd9abc4875a3d03f06ff06544e P fd44949e40b247f980ba14623ba0dd34 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 12806407b57c4ffbbf5f5baf439af582 (127.14.177.130:45327), ce1dd5b4e0014ba8a3fa68f2ba226e49 (127.14.177.131:34123)
I20260709 13:18:17.460628 15255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbf069cd9abc4875a3d03f06ff06544e" candidate_uuid: "fd44949e40b247f980ba14623ba0dd34" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12806407b57c4ffbbf5f5baf439af582" is_pre_election: true
W20260709 13:18:17.463760 15145 leader_election.cc:343] T fbf069cd9abc4875a3d03f06ff06544e P fd44949e40b247f980ba14623ba0dd34 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 12806407b57c4ffbbf5f5baf439af582 (127.14.177.130:45327): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:18:17.464717 15180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28ac227079274b8eb6a3e35e4bf34e30" candidate_uuid: "ce1dd5b4e0014ba8a3fa68f2ba226e49" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fd44949e40b247f980ba14623ba0dd34"
I20260709 13:18:17.465932 15180 raft_consensus.cc:3060] T 28ac227079274b8eb6a3e35e4bf34e30 P fd44949e40b247f980ba14623ba0dd34 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:17.468802 15256 raft_consensus.cc:2468] T 28ac227079274b8eb6a3e35e4bf34e30 P 12806407b57c4ffbbf5f5baf439af582 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce1dd5b4e0014ba8a3fa68f2ba226e49 in term 1.
I20260709 13:18:17.470613 15294 leader_election.cc:304] T 28ac227079274b8eb6a3e35e4bf34e30 P ce1dd5b4e0014ba8a3fa68f2ba226e49 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12806407b57c4ffbbf5f5baf439af582, ce1dd5b4e0014ba8a3fa68f2ba226e49; no voters: 
I20260709 13:18:17.474478 15180 raft_consensus.cc:2468] T 28ac227079274b8eb6a3e35e4bf34e30 P fd44949e40b247f980ba14623ba0dd34 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce1dd5b4e0014ba8a3fa68f2ba226e49 in term 1.
I20260709 13:18:17.469399 15381 raft_consensus.cc:2804] T 8391f4ec0b1349f4a1758a0ce2382b7c P ce1dd5b4e0014ba8a3fa68f2ba226e49 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:18:17.479216 15402 raft_consensus.cc:515] T fbf069cd9abc4875a3d03f06ff06544e P fd44949e40b247f980ba14623ba0dd34 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd44949e40b247f980ba14623ba0dd34" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 37801 } } peers { permanent_uuid: "12806407b57c4ffbbf5f5baf439af582" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 45327 } } peers { permanent_uuid: "ce1dd5b4e0014ba8a3fa68f2ba226e49" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34123 } }
I20260709 13:18:17.483609 15256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbf069cd9abc4875a3d03f06ff06544e" candidate_uuid: "fd44949e40b247f980ba14623ba0dd34" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12806407b57c4ffbbf5f5baf439af582"
I20260709 13:18:17.485069 15425 leader_election.cc:290] T 8391f4ec0b1349f4a1758a0ce2382b7c P ce1dd5b4e0014ba8a3fa68f2ba226e49 [CANDIDATE]: Term 1 election: Requested vote from peers fd44949e40b247f980ba14623ba0dd34 (127.14.177.129:37801), 12806407b57c4ffbbf5f5baf439af582 (127.14.177.130:45327)
I20260709 13:18:17.493000 15374 tablet_bootstrap.cc:492] T 98f6aef87e78472c9987c75cac6575af P ce1dd5b4e0014ba8a3fa68f2ba226e49: No bootstrap required, opened a new log
I20260709 13:18:17.485972 15402 leader_election.cc:290] T fbf069cd9abc4875a3d03f06ff06544e P fd44949e40b247f980ba14623ba0dd34 [CANDIDATE]: Term 1 election: Requested vote from peers 12806407b57c4ffbbf5f5baf439af582 (127.14.177.130:45327), ce1dd5b4e0014ba8a3fa68f2ba226e49 (127.14.177.131:34123)
W20260709 13:18:17.499110 15126 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
W20260709 13:18:17.485634 15145 leader_election.cc:343] T fbf069cd9abc4875a3d03f06ff06544e P fd44949e40b247f980ba14623ba0dd34 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 12806407b57c4ffbbf5f5baf439af582 (127.14.177.130:45327): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:18:17.492162 15429 raft_consensus.cc:493] T 81fce322177440a690ee6f46d2d67ad6 P ce1dd5b4e0014ba8a3fa68f2ba226e49 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:17.514622 15428 raft_consensus.cc:493] T 58a8c0164db24ffd8e992493c228804c P ce1dd5b4e0014ba8a3fa68f2ba226e49 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:17.513499 15381 raft_consensus.cc:697] T 8391f4ec0b1349f4a1758a0ce2382b7c P ce1dd5b4e0014ba8a3fa68f2ba226e49 [term 1 LEADER]: Becoming Leader. State: Replica: ce1dd5b4e0014ba8a3fa68f2ba226e49, State: Running, Role: LEADER
I20260709 13:18:17.516579 15374 ts_tablet_manager.cc:1403] T 98f6aef87e78472c9987c75cac6575af P ce1dd5b4e0014ba8a3fa68f2ba226e49: Time spent bootstrapping tablet: real 0.151s	user 0.045s	sys 0.082s
I20260709 13:18:17.516446 15429 raft_consensus.cc:515] T 81fce322177440a690ee6f46d2d67ad6 P ce1dd5b4e0014ba8a3fa68f2ba226e49 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd44949e40b247f980ba14623ba0dd34" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 37801 } } peers { permanent_uuid: "12806407b57c4ffbbf5f5baf439af582" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 45327 } } peers { permanent_uuid: "ce1dd5b4e0014ba8a3fa68f2ba226e49" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34123 } }
I20260709 13:18:17.517951 15381 consensus_queue.cc:237] T 8391f4ec0b1349f4a1758a0ce2382b7c P ce1dd5b4e0014ba8a3fa68f2ba226e49 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd44949e40b247f980ba14623ba0dd34" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 37801 } } peers { permanent_uuid: "12806407b57c4ffbbf5f5baf439af582" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 45327 } } peers { permanent_uuid: "ce1dd5b4e0014ba8a3fa68f2ba226e49" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34123 } }
I20260709 13:18:17.519738 15429 leader_election.cc:290] T 81fce322177440a690ee6f46d2d67ad6 P ce1dd5b4e0014ba8a3fa68f2ba226e49 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fd44949e40b247f980ba14623ba0dd34 (127.14.177.129:37801), 12806407b57c4ffbbf5f5baf439af582 (127.14.177.130:45327)
I20260709 13:18:17.487334 15329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbf069cd9abc4875a3d03f06ff06544e" candidate_uuid: "fd44949e40b247f980ba14623ba0dd34" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce1dd5b4e0014ba8a3fa68f2ba226e49"
I20260709 13:18:17.520435 15329 raft_consensus.cc:3060] T fbf069cd9abc4875a3d03f06ff06544e P ce1dd5b4e0014ba8a3fa68f2ba226e49 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:17.520273 15374 raft_consensus.cc:359] T 98f6aef87e78472c9987c75cac6575af P ce1dd5b4e0014ba8a3fa68f2ba226e49 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd44949e40b247f980ba14623ba0dd34" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 37801 } } peers { permanent_uuid: "12806407b57c4ffbbf5f5baf439af582" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 45327 } } peers { permanent_uuid: "ce1dd5b4e0014ba8a3fa68f2ba226e49" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34123 } }
I20260709 13:18:17.520565 15180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81fce322177440a690ee6f46d2d67ad6" candidate_uuid: "ce1dd5b4e0014ba8a3fa68f2ba226e49" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fd44949e40b247f980ba14623ba0dd34" is_pre_election: true
I20260709 13:18:17.521023 15374 raft_consensus.cc:385] T 98f6aef87e78472c9987c75cac6575af P ce1dd5b4e0014ba8a3fa68f2ba226e49 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:17.521334 15374 raft_consensus.cc:740] T 98f6aef87e78472c9987c75cac6575af P ce1dd5b4e0014ba8a3fa68f2ba226e49 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ce1dd5b4e0014ba8a3fa68f2ba226e49, State: Initialized, Role: FOLLOWER
I20260709 13:18:17.521942 15428 raft_consensus.cc:515] T 58a8c0164db24ffd8e992493c228804c P ce1dd5b4e0014ba8a3fa68f2ba226e49 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd44949e40b247f980ba14623ba0dd34" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 37801 } } peers { permanent_uuid: "12806407b57c4ffbbf5f5baf439af582" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 45327 } } peers { permanent_uuid: "ce1dd5b4e0014ba8a3fa68f2ba226e49" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34123 } }
I20260709 13:18:17.521978 15374 consensus_queue.cc:260] T 98f6aef87e78472c9987c75cac6575af P ce1dd5b4e0014ba8a3fa68f2ba226e49 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd44949e40b247f980ba14623ba0dd34" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 37801 } } peers { permanent_uuid: "12806407b57c4ffbbf5f5baf439af582" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 45327 } } peers { permanent_uuid: "ce1dd5b4e0014ba8a3fa68f2ba226e49" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34123 } }
I20260709 13:18:17.485733 15425 raft_consensus.cc:2804] T 28ac227079274b8eb6a3e35e4bf34e30 P ce1dd5b4e0014ba8a3fa68f2ba226e49 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:18:17.523608 15425 raft_consensus.cc:697] T 28ac227079274b8eb6a3e35e4bf34e30 P ce1dd5b4e0014ba8a3fa68f2ba226e49 [term 1 LEADER]: Becoming Leader. State: Replica: ce1dd5b4e0014ba8a3fa68f2ba226e49, State: Running, Role: LEADER
I20260709 13:18:17.524238 15374 ts_tablet_manager.cc:1434] T 98f6aef87e78472c9987c75cac6575af P ce1dd5b4e0014ba8a3fa68f2ba226e49: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.000s
I20260709 13:18:17.524402 15425 consensus_queue.cc:237] T 28ac227079274b8eb6a3e35e4bf34e30 P ce1dd5b4e0014ba8a3fa68f2ba226e49 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd44949e40b247f980ba14623ba0dd34" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 37801 } } peers { permanent_uuid: "12806407b57c4ffbbf5f5baf439af582" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 45327 } } peers { permanent_uuid: "ce1dd5b4e0014ba8a3fa68f2ba226e49" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34123 } }
I20260709 13:18:17.524940 15180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58a8c0164db24ffd8e992493c228804c" candidate_uuid: "ce1dd5b4e0014ba8a3fa68f2ba226e49" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fd44949e40b247f980ba14623ba0dd34" is_pre_election: true
I20260709 13:18:17.525377 15374 tablet_bootstrap.cc:492] T 853d00d5717a4ba1a941a81884cbb7a2 P ce1dd5b4e0014ba8a3fa68f2ba226e49: Bootstrap starting.
I20260709 13:18:17.525616 15180 raft_consensus.cc:2468] T 58a8c0164db24ffd8e992493c228804c P fd44949e40b247f980ba14623ba0dd34 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce1dd5b4e0014ba8a3fa68f2ba226e49 in term 0.
I20260709 13:18:17.524194 15428 leader_election.cc:290] T 58a8c0164db24ffd8e992493c228804c P ce1dd5b4e0014ba8a3fa68f2ba226e49 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fd44949e40b247f980ba14623ba0dd34 (127.14.177.129:37801), 12806407b57c4ffbbf5f5baf439af582 (127.14.177.130:45327)
I20260709 13:18:17.526953 15295 leader_election.cc:304] T 58a8c0164db24ffd8e992493c228804c P ce1dd5b4e0014ba8a3fa68f2ba226e49 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce1dd5b4e0014ba8a3fa68f2ba226e49, fd44949e40b247f980ba14623ba0dd34; no voters: 
W20260709 13:18:17.514052 15294 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:45327, user_credentials={real_user=slave}} blocked reactor thread for 43543.1us
I20260709 13:18:17.527756 15428 raft_consensus.cc:2804] T 58a8c0164db24ffd8e992493c228804c P ce1dd5b4e0014ba8a3fa68f2ba226e49 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:18:17.528167 15428 raft_consensus.cc:493] T 58a8c0164db24ffd8e992493c228804c P ce1dd5b4e0014ba8a3fa68f2ba226e49 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:17.528692 15428 raft_consensus.cc:3060] T 58a8c0164db24ffd8e992493c228804c P ce1dd5b4e0014ba8a3fa68f2ba226e49 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:17.529330 15256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8391f4ec0b1349f4a1758a0ce2382b7c" candidate_uuid: "ce1dd5b4e0014ba8a3fa68f2ba226e49" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12806407b57c4ffbbf5f5baf439af582"
I20260709 13:18:17.530064 15256 raft_consensus.cc:3060] T 8391f4ec0b1349f4a1758a0ce2382b7c P 12806407b57c4ffbbf5f5baf439af582 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:17.533010 15255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81fce322177440a690ee6f46d2d67ad6" candidate_uuid: "ce1dd5b4e0014ba8a3fa68f2ba226e49" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12806407b57c4ffbbf5f5baf439af582" is_pre_election: true
I20260709 13:18:17.534126 15255 raft_consensus.cc:2393] T 81fce322177440a690ee6f46d2d67ad6 P 12806407b57c4ffbbf5f5baf439af582 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ce1dd5b4e0014ba8a3fa68f2ba226e49 in current term 1: Already voted for candidate fd44949e40b247f980ba14623ba0dd34 in this term.
I20260709 13:18:17.535910 15294 leader_election.cc:304] T 81fce322177440a690ee6f46d2d67ad6 P ce1dd5b4e0014ba8a3fa68f2ba226e49 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ce1dd5b4e0014ba8a3fa68f2ba226e49; no voters: 12806407b57c4ffbbf5f5baf439af582, fd44949e40b247f980ba14623ba0dd34
I20260709 13:18:17.536144 15255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58a8c0164db24ffd8e992493c228804c" candidate_uuid: "ce1dd5b4e0014ba8a3fa68f2ba226e49" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12806407b57c4ffbbf5f5baf439af582" is_pre_election: true
I20260709 13:18:17.537029 15429 raft_consensus.cc:3060] T 81fce322177440a690ee6f46d2d67ad6 P ce1dd5b4e0014ba8a3fa68f2ba226e49 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:17.538103 15374 tablet_bootstrap.cc:654] T 853d00d5717a4ba1a941a81884cbb7a2 P ce1dd5b4e0014ba8a3fa68f2ba226e49: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:17.537755 15428 raft_consensus.cc:515] T 58a8c0164db24ffd8e992493c228804c P ce1dd5b4e0014ba8a3fa68f2ba226e49 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd44949e40b247f980ba14623ba0dd34" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 37801 } } peers { permanent_uuid: "12806407b57c4ffbbf5f5baf439af582" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 45327 } } peers { permanent_uuid: "ce1dd5b4e0014ba8a3fa68f2ba226e49" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34123 } }
I20260709 13:18:17.542686 15256 raft_consensus.cc:2468] T 8391f4ec0b1349f4a1758a0ce2382b7c P 12806407b57c4ffbbf5f5baf439af582 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce1dd5b4e0014ba8a3fa68f2ba226e49 in term 1.
I20260709 13:18:17.543375 15374 tablet_bootstrap.cc:492] T 853d00d5717a4ba1a941a81884cbb7a2 P ce1dd5b4e0014ba8a3fa68f2ba226e49: No bootstrap required, opened a new log
W20260709 13:18:17.543450 15294 leader_election.cc:343] T 58a8c0164db24ffd8e992493c228804c P ce1dd5b4e0014ba8a3fa68f2ba226e49 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 12806407b57c4ffbbf5f5baf439af582 (127.14.177.130:45327): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:18:17.543895 15374 ts_tablet_manager.cc:1403] T 853d00d5717a4ba1a941a81884cbb7a2 P ce1dd5b4e0014ba8a3fa68f2ba226e49: Time spent bootstrapping tablet: real 0.019s	user 0.011s	sys 0.000s
I20260709 13:18:17.546581 15375 tablet_bootstrap.cc:492] T fbf069cd9abc4875a3d03f06ff06544e P 12806407b57c4ffbbf5f5baf439af582: No bootstrap required, opened a new log
I20260709 13:18:17.546983 15375 ts_tablet_manager.cc:1403] T fbf069cd9abc4875a3d03f06ff06544e P 12806407b57c4ffbbf5f5baf439af582: Time spent bootstrapping tablet: real 0.142s	user 0.015s	sys 0.003s
I20260709 13:18:17.546684 15374 raft_consensus.cc:359] T 853d00d5717a4ba1a941a81884cbb7a2 P ce1dd5b4e0014ba8a3fa68f2ba226e49 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd44949e40b247f980ba14623ba0dd34" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 37801 } } peers { permanent_uuid: "12806407b57c4ffbbf5f5baf439af582" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 45327 } } peers { permanent_uuid: "ce1dd5b4e0014ba8a3fa68f2ba226e49" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34123 } }
I20260709 13:18:17.547468 15374 raft_consensus.cc:385] T 853d00d5717a4ba1a941a81884cbb7a2 P ce1dd5b4e0014ba8a3fa68f2ba226e49 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:17.547813 15374 raft_consensus.cc:740] T 853d00d5717a4ba1a941a81884cbb7a2 P ce1dd5b4e0014ba8a3fa68f2ba226e49 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ce1dd5b4e0014ba8a3fa68f2ba226e49, State: Initialized, Role: FOLLOWER
I20260709 13:18:17.548738 15374 consensus_queue.cc:260] T 853d00d5717a4ba1a941a81884cbb7a2 P ce1dd5b4e0014ba8a3fa68f2ba226e49 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd44949e40b247f980ba14623ba0dd34" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 37801 } } peers { permanent_uuid: "12806407b57c4ffbbf5f5baf439af582" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 45327 } } peers { permanent_uuid: "ce1dd5b4e0014ba8a3fa68f2ba226e49" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34123 } }
I20260709 13:18:17.549154 15375 raft_consensus.cc:359] T fbf069cd9abc4875a3d03f06ff06544e P 12806407b57c4ffbbf5f5baf439af582 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd44949e40b247f980ba14623ba0dd34" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 37801 } } peers { permanent_uuid: "12806407b57c4ffbbf5f5baf439af582" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 45327 } } peers { permanent_uuid: "ce1dd5b4e0014ba8a3fa68f2ba226e49" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34123 } }
I20260709 13:18:17.550848 15428 leader_election.cc:290] T 58a8c0164db24ffd8e992493c228804c P ce1dd5b4e0014ba8a3fa68f2ba226e49 [CANDIDATE]: Term 1 election: Requested vote from peers fd44949e40b247f980ba14623ba0dd34 (127.14.177.129:37801), 12806407b57c4ffbbf5f5baf439af582 (127.14.177.130:45327)
I20260709 13:18:17.551373 15374 ts_tablet_manager.cc:1434] T 853d00d5717a4ba1a941a81884cbb7a2 P ce1dd5b4e0014ba8a3fa68f2ba226e49: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 13:18:17.553320 15180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58a8c0164db24ffd8e992493c228804c" candidate_uuid: "ce1dd5b4e0014ba8a3fa68f2ba226e49" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fd44949e40b247f980ba14623ba0dd34"
I20260709 13:18:17.556077 15180 raft_consensus.cc:3060] T 58a8c0164db24ffd8e992493c228804c P fd44949e40b247f980ba14623ba0dd34 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:17.556895 15076 catalog_manager.cc:5697] T 8391f4ec0b1349f4a1758a0ce2382b7c P ce1dd5b4e0014ba8a3fa68f2ba226e49 reported cstate change: term changed from 0 to 1, leader changed from <none> to ce1dd5b4e0014ba8a3fa68f2ba226e49 (127.14.177.131). New cstate: current_term: 1 leader_uuid: "ce1dd5b4e0014ba8a3fa68f2ba226e49" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd44949e40b247f980ba14623ba0dd34" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 37801 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "12806407b57c4ffbbf5f5baf439af582" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 45327 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ce1dd5b4e0014ba8a3fa68f2ba226e49" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34123 } health_report { overall_health: HEALTHY } } }
I20260709 13:18:17.555663 15375 raft_consensus.cc:385] T fbf069cd9abc4875a3d03f06ff06544e P 12806407b57c4ffbbf5f5baf439af582 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:17.559558 15375 raft_consensus.cc:740] T fbf069cd9abc4875a3d03f06ff06544e P 12806407b57c4ffbbf5f5baf439af582 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 12806407b57c4ffbbf5f5baf439af582, State: Initialized, Role: FOLLOWER
I20260709 13:18:17.560477 15375 consensus_queue.cc:260] T fbf069cd9abc4875a3d03f06ff06544e P 12806407b57c4ffbbf5f5baf439af582 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd44949e40b247f980ba14623ba0dd34" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 37801 } } peers { permanent_uuid: "12806407b57c4ffbbf5f5baf439af582" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 45327 } } peers { permanent_uuid: "ce1dd5b4e0014ba8a3fa68f2ba226e49" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34123 } }
I20260709 13:18:17.554780 15256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58a8c0164db24ffd8e992493c228804c" candidate_uuid: "ce1dd5b4e0014ba8a3fa68f2ba226e49" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12806407b57c4ffbbf5f5baf439af582"
I20260709 13:18:17.564051 15180 raft_consensus.cc:2468] T 58a8c0164db24ffd8e992493c228804c P fd44949e40b247f980ba14623ba0dd34 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce1dd5b4e0014ba8a3fa68f2ba226e49 in term 1.
I20260709 13:18:17.565092 15295 leader_election.cc:304] T 58a8c0164db24ffd8e992493c228804c P ce1dd5b4e0014ba8a3fa68f2ba226e49 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce1dd5b4e0014ba8a3fa68f2ba226e49, fd44949e40b247f980ba14623ba0dd34; no voters: 
I20260709 13:18:17.565867 15375 ts_tablet_manager.cc:1434] T fbf069cd9abc4875a3d03f06ff06544e P 12806407b57c4ffbbf5f5baf439af582: Time spent starting tablet: real 0.019s	user 0.004s	sys 0.001s
I20260709 13:18:17.566001 15428 raft_consensus.cc:2804] T 58a8c0164db24ffd8e992493c228804c P ce1dd5b4e0014ba8a3fa68f2ba226e49 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:18:17.566479 15428 raft_consensus.cc:697] T 58a8c0164db24ffd8e992493c228804c P ce1dd5b4e0014ba8a3fa68f2ba226e49 [term 1 LEADER]: Becoming Leader. State: Replica: ce1dd5b4e0014ba8a3fa68f2ba226e49, State: Running, Role: LEADER
I20260709 13:18:17.566731 15375 tablet_bootstrap.cc:492] T 58a8c0164db24ffd8e992493c228804c P 12806407b57c4ffbbf5f5baf439af582: Bootstrap starting.
I20260709 13:18:17.567595 15329 raft_consensus.cc:2468] T fbf069cd9abc4875a3d03f06ff06544e P ce1dd5b4e0014ba8a3fa68f2ba226e49 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fd44949e40b247f980ba14623ba0dd34 in term 1.
I20260709 13:18:17.567405 15428 consensus_queue.cc:237] T 58a8c0164db24ffd8e992493c228804c P ce1dd5b4e0014ba8a3fa68f2ba226e49 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd44949e40b247f980ba14623ba0dd34" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 37801 } } peers { permanent_uuid: "12806407b57c4ffbbf5f5baf439af582" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 45327 } } peers { permanent_uuid: "ce1dd5b4e0014ba8a3fa68f2ba226e49" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34123 } }
I20260709 13:18:17.568760 15429 raft_consensus.cc:2749] T 81fce322177440a690ee6f46d2d67ad6 P ce1dd5b4e0014ba8a3fa68f2ba226e49 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:18:17.570284 15146 leader_election.cc:304] T fbf069cd9abc4875a3d03f06ff06544e P fd44949e40b247f980ba14623ba0dd34 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ce1dd5b4e0014ba8a3fa68f2ba226e49, fd44949e40b247f980ba14623ba0dd34; no voters: 12806407b57c4ffbbf5f5baf439af582
I20260709 13:18:17.574965 15375 tablet_bootstrap.cc:654] T 58a8c0164db24ffd8e992493c228804c P 12806407b57c4ffbbf5f5baf439af582: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:17.577706 15402 raft_consensus.cc:2804] T fbf069cd9abc4875a3d03f06ff06544e P fd44949e40b247f980ba14623ba0dd34 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:18:17.580948 15402 raft_consensus.cc:697] T fbf069cd9abc4875a3d03f06ff06544e P fd44949e40b247f980ba14623ba0dd34 [term 1 LEADER]: Becoming Leader. State: Replica: fd44949e40b247f980ba14623ba0dd34, State: Running, Role: LEADER
I20260709 13:18:17.582480 15402 consensus_queue.cc:237] T fbf069cd9abc4875a3d03f06ff06544e P fd44949e40b247f980ba14623ba0dd34 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd44949e40b247f980ba14623ba0dd34" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 37801 } } peers { permanent_uuid: "12806407b57c4ffbbf5f5baf439af582" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 45327 } } peers { permanent_uuid: "ce1dd5b4e0014ba8a3fa68f2ba226e49" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34123 } }
I20260709 13:18:17.587011 15375 tablet_bootstrap.cc:492] T 58a8c0164db24ffd8e992493c228804c P 12806407b57c4ffbbf5f5baf439af582: No bootstrap required, opened a new log
W20260709 13:18:17.583593 15294 leader_election.cc:343] T 58a8c0164db24ffd8e992493c228804c P ce1dd5b4e0014ba8a3fa68f2ba226e49 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 12806407b57c4ffbbf5f5baf439af582 (127.14.177.130:45327): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:18:17.587481 15375 ts_tablet_manager.cc:1403] T 58a8c0164db24ffd8e992493c228804c P 12806407b57c4ffbbf5f5baf439af582: Time spent bootstrapping tablet: real 0.021s	user 0.015s	sys 0.002s
I20260709 13:18:17.589784 15375 raft_consensus.cc:359] T 58a8c0164db24ffd8e992493c228804c P 12806407b57c4ffbbf5f5baf439af582 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd44949e40b247f980ba14623ba0dd34" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 37801 } } peers { permanent_uuid: "12806407b57c4ffbbf5f5baf439af582" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 45327 } } peers { permanent_uuid: "ce1dd5b4e0014ba8a3fa68f2ba226e49" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34123 } }
I20260709 13:18:17.590657 15375 raft_consensus.cc:385] T 58a8c0164db24ffd8e992493c228804c P 12806407b57c4ffbbf5f5baf439af582 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:17.590936 15375 raft_consensus.cc:740] T 58a8c0164db24ffd8e992493c228804c P 12806407b57c4ffbbf5f5baf439af582 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 12806407b57c4ffbbf5f5baf439af582, State: Initialized, Role: FOLLOWER
I20260709 13:18:17.591593 15375 consensus_queue.cc:260] T 58a8c0164db24ffd8e992493c228804c P 12806407b57c4ffbbf5f5baf439af582 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd44949e40b247f980ba14623ba0dd34" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 37801 } } peers { permanent_uuid: "12806407b57c4ffbbf5f5baf439af582" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 45327 } } peers { permanent_uuid: "ce1dd5b4e0014ba8a3fa68f2ba226e49" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34123 } }
I20260709 13:18:17.594020 15375 ts_tablet_manager.cc:1434] T 58a8c0164db24ffd8e992493c228804c P 12806407b57c4ffbbf5f5baf439af582: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 13:18:17.600221 15076 catalog_manager.cc:5697] T 58a8c0164db24ffd8e992493c228804c P ce1dd5b4e0014ba8a3fa68f2ba226e49 reported cstate change: term changed from 0 to 1, leader changed from <none> to ce1dd5b4e0014ba8a3fa68f2ba226e49 (127.14.177.131). New cstate: current_term: 1 leader_uuid: "ce1dd5b4e0014ba8a3fa68f2ba226e49" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd44949e40b247f980ba14623ba0dd34" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 37801 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "12806407b57c4ffbbf5f5baf439af582" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 45327 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ce1dd5b4e0014ba8a3fa68f2ba226e49" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34123 } health_report { overall_health: HEALTHY } } }
I20260709 13:18:17.605288 15076 catalog_manager.cc:5697] T 28ac227079274b8eb6a3e35e4bf34e30 P ce1dd5b4e0014ba8a3fa68f2ba226e49 reported cstate change: term changed from 0 to 1, leader changed from <none> to ce1dd5b4e0014ba8a3fa68f2ba226e49 (127.14.177.131). New cstate: current_term: 1 leader_uuid: "ce1dd5b4e0014ba8a3fa68f2ba226e49" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd44949e40b247f980ba14623ba0dd34" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 37801 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "12806407b57c4ffbbf5f5baf439af582" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 45327 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ce1dd5b4e0014ba8a3fa68f2ba226e49" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34123 } health_report { overall_health: HEALTHY } } }
I20260709 13:18:17.608116 15077 catalog_manager.cc:5697] T fbf069cd9abc4875a3d03f06ff06544e P fd44949e40b247f980ba14623ba0dd34 reported cstate change: term changed from 0 to 1, leader changed from <none> to fd44949e40b247f980ba14623ba0dd34 (127.14.177.129). New cstate: current_term: 1 leader_uuid: "fd44949e40b247f980ba14623ba0dd34" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd44949e40b247f980ba14623ba0dd34" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 37801 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "12806407b57c4ffbbf5f5baf439af582" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 45327 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ce1dd5b4e0014ba8a3fa68f2ba226e49" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34123 } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:17.653470 15411 raft_consensus.cc:493] T fbf069cd9abc4875a3d03f06ff06544e P 12806407b57c4ffbbf5f5baf439af582 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:17.654038 15411 raft_consensus.cc:515] T fbf069cd9abc4875a3d03f06ff06544e P 12806407b57c4ffbbf5f5baf439af582 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd44949e40b247f980ba14623ba0dd34" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 37801 } } peers { permanent_uuid: "12806407b57c4ffbbf5f5baf439af582" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 45327 } } peers { permanent_uuid: "ce1dd5b4e0014ba8a3fa68f2ba226e49" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34123 } }
I20260709 13:18:17.656157 15411 leader_election.cc:290] T fbf069cd9abc4875a3d03f06ff06544e P 12806407b57c4ffbbf5f5baf439af582 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fd44949e40b247f980ba14623ba0dd34 (127.14.177.129:37801), ce1dd5b4e0014ba8a3fa68f2ba226e49 (127.14.177.131:34123)
I20260709 13:18:17.657066 15180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbf069cd9abc4875a3d03f06ff06544e" candidate_uuid: "12806407b57c4ffbbf5f5baf439af582" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fd44949e40b247f980ba14623ba0dd34" is_pre_election: true
I20260709 13:18:17.657294 15329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbf069cd9abc4875a3d03f06ff06544e" candidate_uuid: "12806407b57c4ffbbf5f5baf439af582" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce1dd5b4e0014ba8a3fa68f2ba226e49" is_pre_election: true
I20260709 13:18:17.657984 15329 raft_consensus.cc:2393] T fbf069cd9abc4875a3d03f06ff06544e P ce1dd5b4e0014ba8a3fa68f2ba226e49 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12806407b57c4ffbbf5f5baf439af582 in current term 1: Already voted for candidate fd44949e40b247f980ba14623ba0dd34 in this term.
I20260709 13:18:17.658876 15221 leader_election.cc:304] T fbf069cd9abc4875a3d03f06ff06544e P 12806407b57c4ffbbf5f5baf439af582 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12806407b57c4ffbbf5f5baf439af582; no voters: ce1dd5b4e0014ba8a3fa68f2ba226e49, fd44949e40b247f980ba14623ba0dd34
I20260709 13:18:17.659552 15411 raft_consensus.cc:3060] T fbf069cd9abc4875a3d03f06ff06544e P 12806407b57c4ffbbf5f5baf439af582 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:17.666155 15411 raft_consensus.cc:2749] T fbf069cd9abc4875a3d03f06ff06544e P 12806407b57c4ffbbf5f5baf439af582 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:18:17.687973 15402 consensus_queue.cc:1048] T 54dc07b181bd495d8c8e2cae81240f16 P fd44949e40b247f980ba14623ba0dd34 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce1dd5b4e0014ba8a3fa68f2ba226e49" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:18:17.705543 15402 consensus_queue.cc:1048] T 54dc07b181bd495d8c8e2cae81240f16 P fd44949e40b247f980ba14623ba0dd34 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12806407b57c4ffbbf5f5baf439af582" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 45327 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:18:17.724226 15411 consensus_queue.cc:1048] T 6f3b305de2a04b54aef8674096fa6358 P 12806407b57c4ffbbf5f5baf439af582 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fd44949e40b247f980ba14623ba0dd34" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:18:17.750013 15411 consensus_queue.cc:1048] T 6f3b305de2a04b54aef8674096fa6358 P 12806407b57c4ffbbf5f5baf439af582 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce1dd5b4e0014ba8a3fa68f2ba226e49" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:18:17.757217 15330 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "28ac227079274b8eb6a3e35e4bf34e30"
dest_uuid: "ce1dd5b4e0014ba8a3fa68f2ba226e49"
mode: GRACEFUL
new_leader_uuid: "fd44949e40b247f980ba14623ba0dd34"
 from {username='slave'} at 127.0.0.1:49286
I20260709 13:18:17.757992 15330 raft_consensus.cc:606] T 28ac227079274b8eb6a3e35e4bf34e30 P ce1dd5b4e0014ba8a3fa68f2ba226e49 [term 1 LEADER]: Received request to transfer leadership to TS fd44949e40b247f980ba14623ba0dd34
I20260709 13:18:17.764521 15329 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "58a8c0164db24ffd8e992493c228804c"
dest_uuid: "ce1dd5b4e0014ba8a3fa68f2ba226e49"
mode: GRACEFUL
new_leader_uuid: "fd44949e40b247f980ba14623ba0dd34"
 from {username='slave'} at 127.0.0.1:49286
I20260709 13:18:17.765125 15329 raft_consensus.cc:606] T 58a8c0164db24ffd8e992493c228804c P ce1dd5b4e0014ba8a3fa68f2ba226e49 [term 1 LEADER]: Received request to transfer leadership to TS fd44949e40b247f980ba14623ba0dd34
I20260709 13:18:17.777065 15330 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8391f4ec0b1349f4a1758a0ce2382b7c"
dest_uuid: "ce1dd5b4e0014ba8a3fa68f2ba226e49"
mode: GRACEFUL
new_leader_uuid: "12806407b57c4ffbbf5f5baf439af582"
 from {username='slave'} at 127.0.0.1:49286
I20260709 13:18:17.777655 15330 raft_consensus.cc:606] T 8391f4ec0b1349f4a1758a0ce2382b7c P ce1dd5b4e0014ba8a3fa68f2ba226e49 [term 1 LEADER]: Received request to transfer leadership to TS 12806407b57c4ffbbf5f5baf439af582
I20260709 13:18:17.778692 15404 consensus_queue.cc:1048] T 81fce322177440a690ee6f46d2d67ad6 P fd44949e40b247f980ba14623ba0dd34 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce1dd5b4e0014ba8a3fa68f2ba226e49" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:18:17.793030 15402 consensus_queue.cc:1048] T 81fce322177440a690ee6f46d2d67ad6 P fd44949e40b247f980ba14623ba0dd34 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12806407b57c4ffbbf5f5baf439af582" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 45327 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:18:17.804873 15180 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "81fce322177440a690ee6f46d2d67ad6"
dest_uuid: "fd44949e40b247f980ba14623ba0dd34"
mode: GRACEFUL
new_leader_uuid: "12806407b57c4ffbbf5f5baf439af582"
 from {username='slave'} at 127.0.0.1:42424
I20260709 13:18:17.805461 15180 raft_consensus.cc:606] T 81fce322177440a690ee6f46d2d67ad6 P fd44949e40b247f980ba14623ba0dd34 [term 1 LEADER]: Received request to transfer leadership to TS 12806407b57c4ffbbf5f5baf439af582
I20260709 13:18:17.814433 15180 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "fbf069cd9abc4875a3d03f06ff06544e"
dest_uuid: "fd44949e40b247f980ba14623ba0dd34"
mode: GRACEFUL
new_leader_uuid: "ce1dd5b4e0014ba8a3fa68f2ba226e49"
 from {username='slave'} at 127.0.0.1:42424
I20260709 13:18:17.814968 15180 raft_consensus.cc:606] T fbf069cd9abc4875a3d03f06ff06544e P fd44949e40b247f980ba14623ba0dd34 [term 1 LEADER]: Received request to transfer leadership to TS ce1dd5b4e0014ba8a3fa68f2ba226e49
I20260709 13:18:17.855289 15379 consensus_queue.cc:1048] T 98f6aef87e78472c9987c75cac6575af P fd44949e40b247f980ba14623ba0dd34 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12806407b57c4ffbbf5f5baf439af582" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 45327 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:18:17.866593 15330 raft_consensus.cc:3060] T 98f6aef87e78472c9987c75cac6575af P ce1dd5b4e0014ba8a3fa68f2ba226e49 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:17.874375 15411 consensus_queue.cc:1048] T 853d00d5717a4ba1a941a81884cbb7a2 P 12806407b57c4ffbbf5f5baf439af582 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fd44949e40b247f980ba14623ba0dd34" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:18:17.876538 15404 consensus_queue.cc:1048] T 98f6aef87e78472c9987c75cac6575af P fd44949e40b247f980ba14623ba0dd34 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce1dd5b4e0014ba8a3fa68f2ba226e49" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:18:17.891610 15329 raft_consensus.cc:3060] T 853d00d5717a4ba1a941a81884cbb7a2 P ce1dd5b4e0014ba8a3fa68f2ba226e49 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:17.891918 15415 raft_consensus.cc:493] T 58a8c0164db24ffd8e992493c228804c P 12806407b57c4ffbbf5f5baf439af582 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:17.892387 15415 raft_consensus.cc:515] T 58a8c0164db24ffd8e992493c228804c P 12806407b57c4ffbbf5f5baf439af582 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd44949e40b247f980ba14623ba0dd34" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 37801 } } peers { permanent_uuid: "12806407b57c4ffbbf5f5baf439af582" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 45327 } } peers { permanent_uuid: "ce1dd5b4e0014ba8a3fa68f2ba226e49" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34123 } }
I20260709 13:18:17.894565 15415 leader_election.cc:290] T 58a8c0164db24ffd8e992493c228804c P 12806407b57c4ffbbf5f5baf439af582 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fd44949e40b247f980ba14623ba0dd34 (127.14.177.129:37801), ce1dd5b4e0014ba8a3fa68f2ba226e49 (127.14.177.131:34123)
I20260709 13:18:17.896281 15180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58a8c0164db24ffd8e992493c228804c" candidate_uuid: "12806407b57c4ffbbf5f5baf439af582" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fd44949e40b247f980ba14623ba0dd34" is_pre_election: true
I20260709 13:18:17.896426 15328 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58a8c0164db24ffd8e992493c228804c" candidate_uuid: "12806407b57c4ffbbf5f5baf439af582" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce1dd5b4e0014ba8a3fa68f2ba226e49" is_pre_election: true
I20260709 13:18:17.897065 15180 raft_consensus.cc:2393] T 58a8c0164db24ffd8e992493c228804c P fd44949e40b247f980ba14623ba0dd34 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 12806407b57c4ffbbf5f5baf439af582 in current term 1: Already voted for candidate ce1dd5b4e0014ba8a3fa68f2ba226e49 in this term.
I20260709 13:18:17.898487 15221 leader_election.cc:304] T 58a8c0164db24ffd8e992493c228804c P 12806407b57c4ffbbf5f5baf439af582 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 12806407b57c4ffbbf5f5baf439af582; no voters: ce1dd5b4e0014ba8a3fa68f2ba226e49, fd44949e40b247f980ba14623ba0dd34
I20260709 13:18:17.899812 15409 raft_consensus.cc:3060] T 58a8c0164db24ffd8e992493c228804c P 12806407b57c4ffbbf5f5baf439af582 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:17.905848 15409 raft_consensus.cc:2749] T 58a8c0164db24ffd8e992493c228804c P 12806407b57c4ffbbf5f5baf439af582 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:18:17.906016 15415 consensus_queue.cc:1048] T 853d00d5717a4ba1a941a81884cbb7a2 P 12806407b57c4ffbbf5f5baf439af582 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce1dd5b4e0014ba8a3fa68f2ba226e49" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:18:17.988572 15425 consensus_queue.cc:1048] T 28ac227079274b8eb6a3e35e4bf34e30 P ce1dd5b4e0014ba8a3fa68f2ba226e49 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12806407b57c4ffbbf5f5baf439af582" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 45327 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:18:17.990475 15425 consensus_queue.cc:1048] T 8391f4ec0b1349f4a1758a0ce2382b7c P ce1dd5b4e0014ba8a3fa68f2ba226e49 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fd44949e40b247f980ba14623ba0dd34" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:18:18.002553 15425 consensus_queue.cc:1048] T 28ac227079274b8eb6a3e35e4bf34e30 P ce1dd5b4e0014ba8a3fa68f2ba226e49 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fd44949e40b247f980ba14623ba0dd34" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:18:18.015699 15425 consensus_queue.cc:1048] T 8391f4ec0b1349f4a1758a0ce2382b7c P ce1dd5b4e0014ba8a3fa68f2ba226e49 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12806407b57c4ffbbf5f5baf439af582" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 45327 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:18:18.025504 15425 raft_consensus.cc:993] T 28ac227079274b8eb6a3e35e4bf34e30 P ce1dd5b4e0014ba8a3fa68f2ba226e49: : Instructing follower fd44949e40b247f980ba14623ba0dd34 to start an election
I20260709 13:18:18.025944 15425 raft_consensus.cc:1081] T 28ac227079274b8eb6a3e35e4bf34e30 P ce1dd5b4e0014ba8a3fa68f2ba226e49 [term 1 LEADER]: Signalling peer fd44949e40b247f980ba14623ba0dd34 to start an election
I20260709 13:18:18.028180 15179 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "28ac227079274b8eb6a3e35e4bf34e30"
dest_uuid: "fd44949e40b247f980ba14623ba0dd34"
 from {username='slave'} at 127.0.0.1:42406
I20260709 13:18:18.029001 15179 raft_consensus.cc:493] T 28ac227079274b8eb6a3e35e4bf34e30 P fd44949e40b247f980ba14623ba0dd34 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:18:18.029458 15179 raft_consensus.cc:3060] T 28ac227079274b8eb6a3e35e4bf34e30 P fd44949e40b247f980ba14623ba0dd34 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:18.038801 15179 raft_consensus.cc:515] T 28ac227079274b8eb6a3e35e4bf34e30 P fd44949e40b247f980ba14623ba0dd34 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd44949e40b247f980ba14623ba0dd34" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 37801 } } peers { permanent_uuid: "12806407b57c4ffbbf5f5baf439af582" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 45327 } } peers { permanent_uuid: "ce1dd5b4e0014ba8a3fa68f2ba226e49" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34123 } }
I20260709 13:18:18.045563 15425 raft_consensus.cc:993] T 8391f4ec0b1349f4a1758a0ce2382b7c P ce1dd5b4e0014ba8a3fa68f2ba226e49: : Instructing follower 12806407b57c4ffbbf5f5baf439af582 to start an election
I20260709 13:18:18.046006 15425 raft_consensus.cc:1081] T 8391f4ec0b1349f4a1758a0ce2382b7c P ce1dd5b4e0014ba8a3fa68f2ba226e49 [term 1 LEADER]: Signalling peer 12806407b57c4ffbbf5f5baf439af582 to start an election
I20260709 13:18:18.050966 15179 leader_election.cc:290] T 28ac227079274b8eb6a3e35e4bf34e30 P fd44949e40b247f980ba14623ba0dd34 [CANDIDATE]: Term 2 election: Requested vote from peers 12806407b57c4ffbbf5f5baf439af582 (127.14.177.130:45327), ce1dd5b4e0014ba8a3fa68f2ba226e49 (127.14.177.131:34123)
I20260709 13:18:18.051070 15255 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "8391f4ec0b1349f4a1758a0ce2382b7c"
dest_uuid: "12806407b57c4ffbbf5f5baf439af582"
 from {username='slave'} at 127.0.0.1:42046
I20260709 13:18:18.051774 15255 raft_consensus.cc:493] T 8391f4ec0b1349f4a1758a0ce2382b7c P 12806407b57c4ffbbf5f5baf439af582 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:18:18.052152 15255 raft_consensus.cc:3060] T 8391f4ec0b1349f4a1758a0ce2382b7c P 12806407b57c4ffbbf5f5baf439af582 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:18.052980 15254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28ac227079274b8eb6a3e35e4bf34e30" candidate_uuid: "fd44949e40b247f980ba14623ba0dd34" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "12806407b57c4ffbbf5f5baf439af582"
I20260709 13:18:18.053660 15254 raft_consensus.cc:3060] T 28ac227079274b8eb6a3e35e4bf34e30 P 12806407b57c4ffbbf5f5baf439af582 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:18.052037 15330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28ac227079274b8eb6a3e35e4bf34e30" candidate_uuid: "fd44949e40b247f980ba14623ba0dd34" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "ce1dd5b4e0014ba8a3fa68f2ba226e49"
I20260709 13:18:18.059563 15330 raft_consensus.cc:3055] T 28ac227079274b8eb6a3e35e4bf34e30 P ce1dd5b4e0014ba8a3fa68f2ba226e49 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:18:18.059970 15330 raft_consensus.cc:740] T 28ac227079274b8eb6a3e35e4bf34e30 P ce1dd5b4e0014ba8a3fa68f2ba226e49 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: ce1dd5b4e0014ba8a3fa68f2ba226e49, State: Running, Role: LEADER
I20260709 13:18:18.060865 15330 consensus_queue.cc:260] T 28ac227079274b8eb6a3e35e4bf34e30 P ce1dd5b4e0014ba8a3fa68f2ba226e49 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd44949e40b247f980ba14623ba0dd34" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 37801 } } peers { permanent_uuid: "12806407b57c4ffbbf5f5baf439af582" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 45327 } } peers { permanent_uuid: "ce1dd5b4e0014ba8a3fa68f2ba226e49" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34123 } }
I20260709 13:18:18.062377 15330 raft_consensus.cc:3060] T 28ac227079274b8eb6a3e35e4bf34e30 P ce1dd5b4e0014ba8a3fa68f2ba226e49 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:18.062319 15254 raft_consensus.cc:2468] T 28ac227079274b8eb6a3e35e4bf34e30 P 12806407b57c4ffbbf5f5baf439af582 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fd44949e40b247f980ba14623ba0dd34 in term 2.
I20260709 13:18:18.059638 15255 raft_consensus.cc:515] T 8391f4ec0b1349f4a1758a0ce2382b7c P 12806407b57c4ffbbf5f5baf439af582 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd44949e40b247f980ba14623ba0dd34" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 37801 } } peers { permanent_uuid: "12806407b57c4ffbbf5f5baf439af582" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 45327 } } peers { permanent_uuid: "ce1dd5b4e0014ba8a3fa68f2ba226e49" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34123 } }
I20260709 13:18:18.063648 15145 leader_election.cc:304] T 28ac227079274b8eb6a3e35e4bf34e30 P fd44949e40b247f980ba14623ba0dd34 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12806407b57c4ffbbf5f5baf439af582, fd44949e40b247f980ba14623ba0dd34; no voters: 
I20260709 13:18:18.064397 15256 raft_consensus.cc:1240] T 8391f4ec0b1349f4a1758a0ce2382b7c P 12806407b57c4ffbbf5f5baf439af582 [term 2 FOLLOWER]: Rejecting Update request from peer ce1dd5b4e0014ba8a3fa68f2ba226e49 for earlier term 1. Current term is 2. Ops: []
I20260709 13:18:18.064579 15404 raft_consensus.cc:2804] T 28ac227079274b8eb6a3e35e4bf34e30 P fd44949e40b247f980ba14623ba0dd34 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:18:18.065099 15404 raft_consensus.cc:697] T 28ac227079274b8eb6a3e35e4bf34e30 P fd44949e40b247f980ba14623ba0dd34 [term 2 LEADER]: Becoming Leader. State: Replica: fd44949e40b247f980ba14623ba0dd34, State: Running, Role: LEADER
I20260709 13:18:18.065888 15179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8391f4ec0b1349f4a1758a0ce2382b7c" candidate_uuid: "12806407b57c4ffbbf5f5baf439af582" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "fd44949e40b247f980ba14623ba0dd34"
I20260709 13:18:18.065953 15404 consensus_queue.cc:237] T 28ac227079274b8eb6a3e35e4bf34e30 P fd44949e40b247f980ba14623ba0dd34 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd44949e40b247f980ba14623ba0dd34" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 37801 } } peers { permanent_uuid: "12806407b57c4ffbbf5f5baf439af582" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 45327 } } peers { permanent_uuid: "ce1dd5b4e0014ba8a3fa68f2ba226e49" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34123 } }
I20260709 13:18:18.067405 15179 raft_consensus.cc:3060] T 8391f4ec0b1349f4a1758a0ce2382b7c P fd44949e40b247f980ba14623ba0dd34 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:18.067978 15329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8391f4ec0b1349f4a1758a0ce2382b7c" candidate_uuid: "12806407b57c4ffbbf5f5baf439af582" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "ce1dd5b4e0014ba8a3fa68f2ba226e49"
I20260709 13:18:18.068403 15255 leader_election.cc:290] T 8391f4ec0b1349f4a1758a0ce2382b7c P 12806407b57c4ffbbf5f5baf439af582 [CANDIDATE]: Term 2 election: Requested vote from peers fd44949e40b247f980ba14623ba0dd34 (127.14.177.129:37801), ce1dd5b4e0014ba8a3fa68f2ba226e49 (127.14.177.131:34123)
I20260709 13:18:18.069360 15330 raft_consensus.cc:2468] T 28ac227079274b8eb6a3e35e4bf34e30 P ce1dd5b4e0014ba8a3fa68f2ba226e49 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fd44949e40b247f980ba14623ba0dd34 in term 2.
I20260709 13:18:18.070375 15381 consensus_queue.cc:1059] T 8391f4ec0b1349f4a1758a0ce2382b7c P ce1dd5b4e0014ba8a3fa68f2ba226e49 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "12806407b57c4ffbbf5f5baf439af582" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 45327 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:18:18.075109 15329 raft_consensus.cc:3055] T 8391f4ec0b1349f4a1758a0ce2382b7c P ce1dd5b4e0014ba8a3fa68f2ba226e49 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:18:18.075500 15329 raft_consensus.cc:740] T 8391f4ec0b1349f4a1758a0ce2382b7c P ce1dd5b4e0014ba8a3fa68f2ba226e49 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: ce1dd5b4e0014ba8a3fa68f2ba226e49, State: Running, Role: LEADER
I20260709 13:18:18.076330 15329 consensus_queue.cc:260] T 8391f4ec0b1349f4a1758a0ce2382b7c P ce1dd5b4e0014ba8a3fa68f2ba226e49 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd44949e40b247f980ba14623ba0dd34" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 37801 } } peers { permanent_uuid: "12806407b57c4ffbbf5f5baf439af582" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 45327 } } peers { permanent_uuid: "ce1dd5b4e0014ba8a3fa68f2ba226e49" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34123 } }
I20260709 13:18:18.077544 15329 raft_consensus.cc:3060] T 8391f4ec0b1349f4a1758a0ce2382b7c P ce1dd5b4e0014ba8a3fa68f2ba226e49 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:18.078226 15179 raft_consensus.cc:2468] T 8391f4ec0b1349f4a1758a0ce2382b7c P fd44949e40b247f980ba14623ba0dd34 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12806407b57c4ffbbf5f5baf439af582 in term 2.
I20260709 13:18:18.079541 15221 leader_election.cc:304] T 8391f4ec0b1349f4a1758a0ce2382b7c P 12806407b57c4ffbbf5f5baf439af582 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12806407b57c4ffbbf5f5baf439af582, fd44949e40b247f980ba14623ba0dd34; no voters: 
I20260709 13:18:18.080418 15415 raft_consensus.cc:2804] T 8391f4ec0b1349f4a1758a0ce2382b7c P 12806407b57c4ffbbf5f5baf439af582 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:18:18.080904 15415 raft_consensus.cc:697] T 8391f4ec0b1349f4a1758a0ce2382b7c P 12806407b57c4ffbbf5f5baf439af582 [term 2 LEADER]: Becoming Leader. State: Replica: 12806407b57c4ffbbf5f5baf439af582, State: Running, Role: LEADER
I20260709 13:18:18.080964 15077 catalog_manager.cc:5697] T 28ac227079274b8eb6a3e35e4bf34e30 P fd44949e40b247f980ba14623ba0dd34 reported cstate change: term changed from 1 to 2, leader changed from ce1dd5b4e0014ba8a3fa68f2ba226e49 (127.14.177.131) to fd44949e40b247f980ba14623ba0dd34 (127.14.177.129). New cstate: current_term: 2 leader_uuid: "fd44949e40b247f980ba14623ba0dd34" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd44949e40b247f980ba14623ba0dd34" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 37801 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "12806407b57c4ffbbf5f5baf439af582" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 45327 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ce1dd5b4e0014ba8a3fa68f2ba226e49" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34123 } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:18.081744 15415 consensus_queue.cc:237] T 8391f4ec0b1349f4a1758a0ce2382b7c P 12806407b57c4ffbbf5f5baf439af582 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd44949e40b247f980ba14623ba0dd34" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 37801 } } peers { permanent_uuid: "12806407b57c4ffbbf5f5baf439af582" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 45327 } } peers { permanent_uuid: "ce1dd5b4e0014ba8a3fa68f2ba226e49" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34123 } }
I20260709 13:18:18.083891 15329 raft_consensus.cc:2468] T 8391f4ec0b1349f4a1758a0ce2382b7c P ce1dd5b4e0014ba8a3fa68f2ba226e49 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12806407b57c4ffbbf5f5baf439af582 in term 2.
W20260709 13:18:18.084381 15381 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 2 current term: 2 is higher.
I20260709 13:18:18.092164 15075 catalog_manager.cc:5697] T 8391f4ec0b1349f4a1758a0ce2382b7c P 12806407b57c4ffbbf5f5baf439af582 reported cstate change: term changed from 1 to 2, leader changed from ce1dd5b4e0014ba8a3fa68f2ba226e49 (127.14.177.131) to 12806407b57c4ffbbf5f5baf439af582 (127.14.177.130). New cstate: current_term: 2 leader_uuid: "12806407b57c4ffbbf5f5baf439af582" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd44949e40b247f980ba14623ba0dd34" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 37801 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "12806407b57c4ffbbf5f5baf439af582" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 45327 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ce1dd5b4e0014ba8a3fa68f2ba226e49" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34123 } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:18.129638 15404 consensus_queue.cc:1048] T fbf069cd9abc4875a3d03f06ff06544e P fd44949e40b247f980ba14623ba0dd34 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce1dd5b4e0014ba8a3fa68f2ba226e49" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:18:18.137084 15402 raft_consensus.cc:993] T fbf069cd9abc4875a3d03f06ff06544e P fd44949e40b247f980ba14623ba0dd34: : Instructing follower ce1dd5b4e0014ba8a3fa68f2ba226e49 to start an election
I20260709 13:18:18.137704 15402 raft_consensus.cc:1081] T fbf069cd9abc4875a3d03f06ff06544e P fd44949e40b247f980ba14623ba0dd34 [term 1 LEADER]: Signalling peer ce1dd5b4e0014ba8a3fa68f2ba226e49 to start an election
I20260709 13:18:18.140120 15330 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "fbf069cd9abc4875a3d03f06ff06544e"
dest_uuid: "ce1dd5b4e0014ba8a3fa68f2ba226e49"
 from {username='slave'} at 127.0.0.1:49256
I20260709 13:18:18.140751 15330 raft_consensus.cc:493] T fbf069cd9abc4875a3d03f06ff06544e P ce1dd5b4e0014ba8a3fa68f2ba226e49 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:18:18.141162 15330 raft_consensus.cc:3060] T fbf069cd9abc4875a3d03f06ff06544e P ce1dd5b4e0014ba8a3fa68f2ba226e49 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:18.143493 15402 consensus_queue.cc:1048] T fbf069cd9abc4875a3d03f06ff06544e P fd44949e40b247f980ba14623ba0dd34 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12806407b57c4ffbbf5f5baf439af582" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 45327 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:18:18.148488 15330 raft_consensus.cc:515] T fbf069cd9abc4875a3d03f06ff06544e P ce1dd5b4e0014ba8a3fa68f2ba226e49 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd44949e40b247f980ba14623ba0dd34" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 37801 } } peers { permanent_uuid: "12806407b57c4ffbbf5f5baf439af582" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 45327 } } peers { permanent_uuid: "ce1dd5b4e0014ba8a3fa68f2ba226e49" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34123 } }
I20260709 13:18:18.149905 15329 raft_consensus.cc:1240] T fbf069cd9abc4875a3d03f06ff06544e P ce1dd5b4e0014ba8a3fa68f2ba226e49 [term 2 FOLLOWER]: Rejecting Update request from peer fd44949e40b247f980ba14623ba0dd34 for earlier term 1. Current term is 2. Ops: []
I20260709 13:18:18.152191 15179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbf069cd9abc4875a3d03f06ff06544e" candidate_uuid: "ce1dd5b4e0014ba8a3fa68f2ba226e49" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "fd44949e40b247f980ba14623ba0dd34"
I20260709 13:18:18.152740 15179 raft_consensus.cc:3055] T fbf069cd9abc4875a3d03f06ff06544e P fd44949e40b247f980ba14623ba0dd34 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:18:18.153066 15179 raft_consensus.cc:740] T fbf069cd9abc4875a3d03f06ff06544e P fd44949e40b247f980ba14623ba0dd34 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: fd44949e40b247f980ba14623ba0dd34, State: Running, Role: LEADER
I20260709 13:18:18.152930 15254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbf069cd9abc4875a3d03f06ff06544e" candidate_uuid: "ce1dd5b4e0014ba8a3fa68f2ba226e49" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "12806407b57c4ffbbf5f5baf439af582"
I20260709 13:18:18.153973 15179 consensus_queue.cc:260] T fbf069cd9abc4875a3d03f06ff06544e P fd44949e40b247f980ba14623ba0dd34 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd44949e40b247f980ba14623ba0dd34" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 37801 } } peers { permanent_uuid: "12806407b57c4ffbbf5f5baf439af582" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 45327 } } peers { permanent_uuid: "ce1dd5b4e0014ba8a3fa68f2ba226e49" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34123 } }
I20260709 13:18:18.154881 15179 raft_consensus.cc:3060] T fbf069cd9abc4875a3d03f06ff06544e P fd44949e40b247f980ba14623ba0dd34 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:18.156766 15330 leader_election.cc:290] T fbf069cd9abc4875a3d03f06ff06544e P ce1dd5b4e0014ba8a3fa68f2ba226e49 [CANDIDATE]: Term 2 election: Requested vote from peers fd44949e40b247f980ba14623ba0dd34 (127.14.177.129:37801), 12806407b57c4ffbbf5f5baf439af582 (127.14.177.130:45327)
I20260709 13:18:18.159679 15381 consensus_queue.cc:1048] T 58a8c0164db24ffd8e992493c228804c P ce1dd5b4e0014ba8a3fa68f2ba226e49 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fd44949e40b247f980ba14623ba0dd34" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:18:18.163887 15179 raft_consensus.cc:2468] T fbf069cd9abc4875a3d03f06ff06544e P fd44949e40b247f980ba14623ba0dd34 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce1dd5b4e0014ba8a3fa68f2ba226e49 in term 2.
I20260709 13:18:18.165215 15295 leader_election.cc:304] T fbf069cd9abc4875a3d03f06ff06544e P ce1dd5b4e0014ba8a3fa68f2ba226e49 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ce1dd5b4e0014ba8a3fa68f2ba226e49, fd44949e40b247f980ba14623ba0dd34; no voters: 12806407b57c4ffbbf5f5baf439af582
I20260709 13:18:18.166797 15381 raft_consensus.cc:2804] T fbf069cd9abc4875a3d03f06ff06544e P ce1dd5b4e0014ba8a3fa68f2ba226e49 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:18:18.167295 15381 raft_consensus.cc:697] T fbf069cd9abc4875a3d03f06ff06544e P ce1dd5b4e0014ba8a3fa68f2ba226e49 [term 2 LEADER]: Becoming Leader. State: Replica: ce1dd5b4e0014ba8a3fa68f2ba226e49, State: Running, Role: LEADER
I20260709 13:18:18.168099 15381 consensus_queue.cc:237] T fbf069cd9abc4875a3d03f06ff06544e P ce1dd5b4e0014ba8a3fa68f2ba226e49 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd44949e40b247f980ba14623ba0dd34" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 37801 } } peers { permanent_uuid: "12806407b57c4ffbbf5f5baf439af582" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 45327 } } peers { permanent_uuid: "ce1dd5b4e0014ba8a3fa68f2ba226e49" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34123 } }
I20260709 13:18:18.172907 15381 consensus_queue.cc:1048] T 58a8c0164db24ffd8e992493c228804c P ce1dd5b4e0014ba8a3fa68f2ba226e49 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12806407b57c4ffbbf5f5baf439af582" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 45327 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:18:18.181326 15381 raft_consensus.cc:993] T 58a8c0164db24ffd8e992493c228804c P ce1dd5b4e0014ba8a3fa68f2ba226e49: : Instructing follower fd44949e40b247f980ba14623ba0dd34 to start an election
I20260709 13:18:18.181753 15381 raft_consensus.cc:1081] T 58a8c0164db24ffd8e992493c228804c P ce1dd5b4e0014ba8a3fa68f2ba226e49 [term 1 LEADER]: Signalling peer fd44949e40b247f980ba14623ba0dd34 to start an election
I20260709 13:18:18.183578 15179 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "58a8c0164db24ffd8e992493c228804c"
dest_uuid: "fd44949e40b247f980ba14623ba0dd34"
 from {username='slave'} at 127.0.0.1:42406
I20260709 13:18:18.184201 15179 raft_consensus.cc:493] T 58a8c0164db24ffd8e992493c228804c P fd44949e40b247f980ba14623ba0dd34 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:18:18.184609 15179 raft_consensus.cc:3060] T 58a8c0164db24ffd8e992493c228804c P fd44949e40b247f980ba14623ba0dd34 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:18.186400 15077 catalog_manager.cc:5697] T fbf069cd9abc4875a3d03f06ff06544e P ce1dd5b4e0014ba8a3fa68f2ba226e49 reported cstate change: term changed from 1 to 2, leader changed from fd44949e40b247f980ba14623ba0dd34 (127.14.177.129) to ce1dd5b4e0014ba8a3fa68f2ba226e49 (127.14.177.131). New cstate: current_term: 2 leader_uuid: "ce1dd5b4e0014ba8a3fa68f2ba226e49" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd44949e40b247f980ba14623ba0dd34" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 37801 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "12806407b57c4ffbbf5f5baf439af582" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 45327 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ce1dd5b4e0014ba8a3fa68f2ba226e49" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34123 } health_report { overall_health: HEALTHY } } }
I20260709 13:18:18.196663 15179 raft_consensus.cc:515] T 58a8c0164db24ffd8e992493c228804c P fd44949e40b247f980ba14623ba0dd34 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd44949e40b247f980ba14623ba0dd34" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 37801 } } peers { permanent_uuid: "12806407b57c4ffbbf5f5baf439af582" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 45327 } } peers { permanent_uuid: "ce1dd5b4e0014ba8a3fa68f2ba226e49" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34123 } }
I20260709 13:18:18.200827 15329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58a8c0164db24ffd8e992493c228804c" candidate_uuid: "fd44949e40b247f980ba14623ba0dd34" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "ce1dd5b4e0014ba8a3fa68f2ba226e49"
I20260709 13:18:18.201467 15329 raft_consensus.cc:3055] T 58a8c0164db24ffd8e992493c228804c P ce1dd5b4e0014ba8a3fa68f2ba226e49 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:18:18.201820 15329 raft_consensus.cc:740] T 58a8c0164db24ffd8e992493c228804c P ce1dd5b4e0014ba8a3fa68f2ba226e49 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: ce1dd5b4e0014ba8a3fa68f2ba226e49, State: Running, Role: LEADER
I20260709 13:18:18.202683 15329 consensus_queue.cc:260] T 58a8c0164db24ffd8e992493c228804c P ce1dd5b4e0014ba8a3fa68f2ba226e49 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd44949e40b247f980ba14623ba0dd34" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 37801 } } peers { permanent_uuid: "12806407b57c4ffbbf5f5baf439af582" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 45327 } } peers { permanent_uuid: "ce1dd5b4e0014ba8a3fa68f2ba226e49" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34123 } }
I20260709 13:18:18.203722 15329 raft_consensus.cc:3060] T 58a8c0164db24ffd8e992493c228804c P ce1dd5b4e0014ba8a3fa68f2ba226e49 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:18.204300 15254 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58a8c0164db24ffd8e992493c228804c" candidate_uuid: "fd44949e40b247f980ba14623ba0dd34" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "12806407b57c4ffbbf5f5baf439af582"
I20260709 13:18:18.206278 15379 raft_consensus.cc:993] T 81fce322177440a690ee6f46d2d67ad6 P fd44949e40b247f980ba14623ba0dd34: : Instructing follower 12806407b57c4ffbbf5f5baf439af582 to start an election
I20260709 13:18:18.206743 15379 raft_consensus.cc:1081] T 81fce322177440a690ee6f46d2d67ad6 P fd44949e40b247f980ba14623ba0dd34 [term 1 LEADER]: Signalling peer 12806407b57c4ffbbf5f5baf439af582 to start an election
I20260709 13:18:18.208640 15254 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "81fce322177440a690ee6f46d2d67ad6"
dest_uuid: "12806407b57c4ffbbf5f5baf439af582"
 from {username='slave'} at 127.0.0.1:42048
I20260709 13:18:18.209223 15254 raft_consensus.cc:493] T 81fce322177440a690ee6f46d2d67ad6 P 12806407b57c4ffbbf5f5baf439af582 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:18:18.209600 15254 raft_consensus.cc:3060] T 81fce322177440a690ee6f46d2d67ad6 P 12806407b57c4ffbbf5f5baf439af582 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:18.213387 15329 raft_consensus.cc:2468] T 58a8c0164db24ffd8e992493c228804c P ce1dd5b4e0014ba8a3fa68f2ba226e49 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fd44949e40b247f980ba14623ba0dd34 in term 2.
I20260709 13:18:18.214787 15146 leader_election.cc:304] T 58a8c0164db24ffd8e992493c228804c P fd44949e40b247f980ba14623ba0dd34 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ce1dd5b4e0014ba8a3fa68f2ba226e49, fd44949e40b247f980ba14623ba0dd34; no voters: 12806407b57c4ffbbf5f5baf439af582
I20260709 13:18:18.215781 15402 raft_consensus.cc:2804] T 58a8c0164db24ffd8e992493c228804c P fd44949e40b247f980ba14623ba0dd34 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:18:18.216286 15402 raft_consensus.cc:697] T 58a8c0164db24ffd8e992493c228804c P fd44949e40b247f980ba14623ba0dd34 [term 2 LEADER]: Becoming Leader. State: Replica: fd44949e40b247f980ba14623ba0dd34, State: Running, Role: LEADER
I20260709 13:18:18.217159 15402 consensus_queue.cc:237] T 58a8c0164db24ffd8e992493c228804c P fd44949e40b247f980ba14623ba0dd34 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd44949e40b247f980ba14623ba0dd34" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 37801 } } peers { permanent_uuid: "12806407b57c4ffbbf5f5baf439af582" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 45327 } } peers { permanent_uuid: "ce1dd5b4e0014ba8a3fa68f2ba226e49" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34123 } }
I20260709 13:18:18.220822 15179 leader_election.cc:290] T 58a8c0164db24ffd8e992493c228804c P fd44949e40b247f980ba14623ba0dd34 [CANDIDATE]: Term 2 election: Requested vote from peers 12806407b57c4ffbbf5f5baf439af582 (127.14.177.130:45327), ce1dd5b4e0014ba8a3fa68f2ba226e49 (127.14.177.131:34123)
I20260709 13:18:18.222273 15254 raft_consensus.cc:515] T 81fce322177440a690ee6f46d2d67ad6 P 12806407b57c4ffbbf5f5baf439af582 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd44949e40b247f980ba14623ba0dd34" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 37801 } } peers { permanent_uuid: "12806407b57c4ffbbf5f5baf439af582" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 45327 } } peers { permanent_uuid: "ce1dd5b4e0014ba8a3fa68f2ba226e49" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34123 } }
I20260709 13:18:18.224777 15180 raft_consensus.cc:1240] T 58a8c0164db24ffd8e992493c228804c P fd44949e40b247f980ba14623ba0dd34 [term 2 LEADER]: Rejecting Update request from peer ce1dd5b4e0014ba8a3fa68f2ba226e49 for earlier term 1. Current term is 2. Ops: []
I20260709 13:18:18.233237 15180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81fce322177440a690ee6f46d2d67ad6" candidate_uuid: "12806407b57c4ffbbf5f5baf439af582" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "fd44949e40b247f980ba14623ba0dd34"
I20260709 13:18:18.233906 15180 raft_consensus.cc:3055] T 81fce322177440a690ee6f46d2d67ad6 P fd44949e40b247f980ba14623ba0dd34 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:18:18.234229 15180 raft_consensus.cc:740] T 81fce322177440a690ee6f46d2d67ad6 P fd44949e40b247f980ba14623ba0dd34 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: fd44949e40b247f980ba14623ba0dd34, State: Running, Role: LEADER
I20260709 13:18:18.234962 15180 consensus_queue.cc:260] T 81fce322177440a690ee6f46d2d67ad6 P fd44949e40b247f980ba14623ba0dd34 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd44949e40b247f980ba14623ba0dd34" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 37801 } } peers { permanent_uuid: "12806407b57c4ffbbf5f5baf439af582" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 45327 } } peers { permanent_uuid: "ce1dd5b4e0014ba8a3fa68f2ba226e49" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34123 } }
I20260709 13:18:18.236096 15180 raft_consensus.cc:3060] T 81fce322177440a690ee6f46d2d67ad6 P fd44949e40b247f980ba14623ba0dd34 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:18.236016 15076 catalog_manager.cc:5697] T 58a8c0164db24ffd8e992493c228804c P fd44949e40b247f980ba14623ba0dd34 reported cstate change: term changed from 1 to 2, leader changed from ce1dd5b4e0014ba8a3fa68f2ba226e49 (127.14.177.131) to fd44949e40b247f980ba14623ba0dd34 (127.14.177.129). New cstate: current_term: 2 leader_uuid: "fd44949e40b247f980ba14623ba0dd34" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd44949e40b247f980ba14623ba0dd34" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 37801 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "12806407b57c4ffbbf5f5baf439af582" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 45327 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ce1dd5b4e0014ba8a3fa68f2ba226e49" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34123 } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:18.238616 15254 leader_election.cc:290] T 81fce322177440a690ee6f46d2d67ad6 P 12806407b57c4ffbbf5f5baf439af582 [CANDIDATE]: Term 2 election: Requested vote from peers fd44949e40b247f980ba14623ba0dd34 (127.14.177.129:37801), ce1dd5b4e0014ba8a3fa68f2ba226e49 (127.14.177.131:34123)
I20260709 13:18:18.239789 15329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81fce322177440a690ee6f46d2d67ad6" candidate_uuid: "12806407b57c4ffbbf5f5baf439af582" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "ce1dd5b4e0014ba8a3fa68f2ba226e49"
I20260709 13:18:18.240418 15329 raft_consensus.cc:3060] T 81fce322177440a690ee6f46d2d67ad6 P ce1dd5b4e0014ba8a3fa68f2ba226e49 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:18.245021 15180 raft_consensus.cc:2468] T 81fce322177440a690ee6f46d2d67ad6 P fd44949e40b247f980ba14623ba0dd34 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12806407b57c4ffbbf5f5baf439af582 in term 2.
I20260709 13:18:18.246467 15221 leader_election.cc:304] T 81fce322177440a690ee6f46d2d67ad6 P 12806407b57c4ffbbf5f5baf439af582 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12806407b57c4ffbbf5f5baf439af582, fd44949e40b247f980ba14623ba0dd34; no voters: 
I20260709 13:18:18.247356 15415 raft_consensus.cc:2804] T 81fce322177440a690ee6f46d2d67ad6 P 12806407b57c4ffbbf5f5baf439af582 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:18:18.247820 15415 raft_consensus.cc:697] T 81fce322177440a690ee6f46d2d67ad6 P 12806407b57c4ffbbf5f5baf439af582 [term 2 LEADER]: Becoming Leader. State: Replica: 12806407b57c4ffbbf5f5baf439af582, State: Running, Role: LEADER
I20260709 13:18:18.248605 15415 consensus_queue.cc:237] T 81fce322177440a690ee6f46d2d67ad6 P 12806407b57c4ffbbf5f5baf439af582 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd44949e40b247f980ba14623ba0dd34" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 37801 } } peers { permanent_uuid: "12806407b57c4ffbbf5f5baf439af582" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 45327 } } peers { permanent_uuid: "ce1dd5b4e0014ba8a3fa68f2ba226e49" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34123 } }
I20260709 13:18:18.249825 15329 raft_consensus.cc:2468] T 81fce322177440a690ee6f46d2d67ad6 P ce1dd5b4e0014ba8a3fa68f2ba226e49 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12806407b57c4ffbbf5f5baf439af582 in term 2.
I20260709 13:18:18.257115 15075 catalog_manager.cc:5697] T 81fce322177440a690ee6f46d2d67ad6 P 12806407b57c4ffbbf5f5baf439af582 reported cstate change: term changed from 1 to 2, leader changed from fd44949e40b247f980ba14623ba0dd34 (127.14.177.129) to 12806407b57c4ffbbf5f5baf439af582 (127.14.177.130). New cstate: current_term: 2 leader_uuid: "12806407b57c4ffbbf5f5baf439af582" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd44949e40b247f980ba14623ba0dd34" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 37801 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "12806407b57c4ffbbf5f5baf439af582" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 45327 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ce1dd5b4e0014ba8a3fa68f2ba226e49" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34123 } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:18.515283 15255 raft_consensus.cc:1275] T 28ac227079274b8eb6a3e35e4bf34e30 P 12806407b57c4ffbbf5f5baf439af582 [term 2 FOLLOWER]: Refusing update from remote peer fd44949e40b247f980ba14623ba0dd34: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:18:18.519744 15379 consensus_queue.cc:1048] T 28ac227079274b8eb6a3e35e4bf34e30 P fd44949e40b247f980ba14623ba0dd34 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12806407b57c4ffbbf5f5baf439af582" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 45327 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:18:18.531638 15329 raft_consensus.cc:1275] T 28ac227079274b8eb6a3e35e4bf34e30 P ce1dd5b4e0014ba8a3fa68f2ba226e49 [term 2 FOLLOWER]: Refusing update from remote peer fd44949e40b247f980ba14623ba0dd34: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:18:18.533391 15402 consensus_queue.cc:1048] T 28ac227079274b8eb6a3e35e4bf34e30 P fd44949e40b247f980ba14623ba0dd34 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce1dd5b4e0014ba8a3fa68f2ba226e49" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:18:18.532272 15180 raft_consensus.cc:1275] T 8391f4ec0b1349f4a1758a0ce2382b7c P fd44949e40b247f980ba14623ba0dd34 [term 2 FOLLOWER]: Refusing update from remote peer 12806407b57c4ffbbf5f5baf439af582: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:18:18.537170 15415 consensus_queue.cc:1048] T 8391f4ec0b1349f4a1758a0ce2382b7c P 12806407b57c4ffbbf5f5baf439af582 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fd44949e40b247f980ba14623ba0dd34" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:18:18.549180 15330 raft_consensus.cc:1275] T 8391f4ec0b1349f4a1758a0ce2382b7c P ce1dd5b4e0014ba8a3fa68f2ba226e49 [term 2 FOLLOWER]: Refusing update from remote peer 12806407b57c4ffbbf5f5baf439af582: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:18:18.550719 15411 consensus_queue.cc:1048] T 8391f4ec0b1349f4a1758a0ce2382b7c P 12806407b57c4ffbbf5f5baf439af582 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce1dd5b4e0014ba8a3fa68f2ba226e49" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:18:18.593811 15180 raft_consensus.cc:1275] T fbf069cd9abc4875a3d03f06ff06544e P fd44949e40b247f980ba14623ba0dd34 [term 2 FOLLOWER]: Refusing update from remote peer ce1dd5b4e0014ba8a3fa68f2ba226e49: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:18:18.595250 15429 consensus_queue.cc:1048] T fbf069cd9abc4875a3d03f06ff06544e P ce1dd5b4e0014ba8a3fa68f2ba226e49 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fd44949e40b247f980ba14623ba0dd34" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:18:18.604097 15255 raft_consensus.cc:3060] T fbf069cd9abc4875a3d03f06ff06544e P 12806407b57c4ffbbf5f5baf439af582 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:18.612072 15255 raft_consensus.cc:1275] T fbf069cd9abc4875a3d03f06ff06544e P 12806407b57c4ffbbf5f5baf439af582 [term 2 FOLLOWER]: Refusing update from remote peer ce1dd5b4e0014ba8a3fa68f2ba226e49: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:18:18.614369 15428 consensus_queue.cc:1048] T fbf069cd9abc4875a3d03f06ff06544e P ce1dd5b4e0014ba8a3fa68f2ba226e49 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12806407b57c4ffbbf5f5baf439af582" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 45327 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:18:18.614758 15329 raft_consensus.cc:1275] T 58a8c0164db24ffd8e992493c228804c P ce1dd5b4e0014ba8a3fa68f2ba226e49 [term 2 FOLLOWER]: Refusing update from remote peer fd44949e40b247f980ba14623ba0dd34: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:18:18.617393 15402 consensus_queue.cc:1048] T 58a8c0164db24ffd8e992493c228804c P fd44949e40b247f980ba14623ba0dd34 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce1dd5b4e0014ba8a3fa68f2ba226e49" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:18:18.631628 15254 raft_consensus.cc:3060] T 58a8c0164db24ffd8e992493c228804c P 12806407b57c4ffbbf5f5baf439af582 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:18.643208 15254 raft_consensus.cc:1275] T 58a8c0164db24ffd8e992493c228804c P 12806407b57c4ffbbf5f5baf439af582 [term 2 FOLLOWER]: Refusing update from remote peer fd44949e40b247f980ba14623ba0dd34: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:18:18.645615 15402 consensus_queue.cc:1048] T 58a8c0164db24ffd8e992493c228804c P fd44949e40b247f980ba14623ba0dd34 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12806407b57c4ffbbf5f5baf439af582" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 45327 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:18:18.659253 15180 raft_consensus.cc:1275] T 81fce322177440a690ee6f46d2d67ad6 P fd44949e40b247f980ba14623ba0dd34 [term 2 FOLLOWER]: Refusing update from remote peer 12806407b57c4ffbbf5f5baf439af582: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:18:18.660624 15411 consensus_queue.cc:1048] T 81fce322177440a690ee6f46d2d67ad6 P 12806407b57c4ffbbf5f5baf439af582 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fd44949e40b247f980ba14623ba0dd34" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:18:18.669373 15329 raft_consensus.cc:1275] T 81fce322177440a690ee6f46d2d67ad6 P ce1dd5b4e0014ba8a3fa68f2ba226e49 [term 2 FOLLOWER]: Refusing update from remote peer 12806407b57c4ffbbf5f5baf439af582: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:18:18.671437 15411 consensus_queue.cc:1048] T 81fce322177440a690ee6f46d2d67ad6 P 12806407b57c4ffbbf5f5baf439af582 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce1dd5b4e0014ba8a3fa68f2ba226e49" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:18:20.845546 15046 auto_leader_rebalancer-test.cc:316] The leader distribution is 
I20260709 13:18:20.846426 15046 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:18:20.868649 15256 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8391f4ec0b1349f4a1758a0ce2382b7c"
dest_uuid: "12806407b57c4ffbbf5f5baf439af582"
mode: GRACEFUL
new_leader_uuid: "ce1dd5b4e0014ba8a3fa68f2ba226e49"
 from {username='slave'} at 127.0.0.1:60774
I20260709 13:18:20.869236 15256 raft_consensus.cc:606] T 8391f4ec0b1349f4a1758a0ce2382b7c P 12806407b57c4ffbbf5f5baf439af582 [term 2 LEADER]: Received request to transfer leadership to TS ce1dd5b4e0014ba8a3fa68f2ba226e49
I20260709 13:18:20.879619 15179 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "98f6aef87e78472c9987c75cac6575af"
dest_uuid: "fd44949e40b247f980ba14623ba0dd34"
mode: GRACEFUL
new_leader_uuid: "ce1dd5b4e0014ba8a3fa68f2ba226e49"
 from {username='slave'} at 127.0.0.1:46746
I20260709 13:18:20.880133 15179 raft_consensus.cc:606] T 98f6aef87e78472c9987c75cac6575af P fd44949e40b247f980ba14623ba0dd34 [term 1 LEADER]: Received request to transfer leadership to TS ce1dd5b4e0014ba8a3fa68f2ba226e49
I20260709 13:18:20.881371 15046 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 13:18:20.881920 15046 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:18:21.076200 15467 raft_consensus.cc:993] T 8391f4ec0b1349f4a1758a0ce2382b7c P 12806407b57c4ffbbf5f5baf439af582: : Instructing follower ce1dd5b4e0014ba8a3fa68f2ba226e49 to start an election
I20260709 13:18:21.076645 15411 raft_consensus.cc:1081] T 8391f4ec0b1349f4a1758a0ce2382b7c P 12806407b57c4ffbbf5f5baf439af582 [term 2 LEADER]: Signalling peer ce1dd5b4e0014ba8a3fa68f2ba226e49 to start an election
I20260709 13:18:21.078007 15329 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "8391f4ec0b1349f4a1758a0ce2382b7c"
dest_uuid: "ce1dd5b4e0014ba8a3fa68f2ba226e49"
 from {username='slave'} at 127.0.0.1:49270
I20260709 13:18:21.078454 15329 raft_consensus.cc:493] T 8391f4ec0b1349f4a1758a0ce2382b7c P ce1dd5b4e0014ba8a3fa68f2ba226e49 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:18:21.078752 15329 raft_consensus.cc:3060] T 8391f4ec0b1349f4a1758a0ce2382b7c P ce1dd5b4e0014ba8a3fa68f2ba226e49 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:21.083245 15329 raft_consensus.cc:515] T 8391f4ec0b1349f4a1758a0ce2382b7c P ce1dd5b4e0014ba8a3fa68f2ba226e49 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd44949e40b247f980ba14623ba0dd34" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 37801 } } peers { permanent_uuid: "12806407b57c4ffbbf5f5baf439af582" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 45327 } } peers { permanent_uuid: "ce1dd5b4e0014ba8a3fa68f2ba226e49" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34123 } }
I20260709 13:18:21.084836 15329 leader_election.cc:290] T 8391f4ec0b1349f4a1758a0ce2382b7c P ce1dd5b4e0014ba8a3fa68f2ba226e49 [CANDIDATE]: Term 3 election: Requested vote from peers fd44949e40b247f980ba14623ba0dd34 (127.14.177.129:37801), 12806407b57c4ffbbf5f5baf439af582 (127.14.177.130:45327)
I20260709 13:18:21.085561 15179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8391f4ec0b1349f4a1758a0ce2382b7c" candidate_uuid: "ce1dd5b4e0014ba8a3fa68f2ba226e49" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "fd44949e40b247f980ba14623ba0dd34"
I20260709 13:18:21.085819 15256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8391f4ec0b1349f4a1758a0ce2382b7c" candidate_uuid: "ce1dd5b4e0014ba8a3fa68f2ba226e49" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "12806407b57c4ffbbf5f5baf439af582"
I20260709 13:18:21.086050 15179 raft_consensus.cc:3060] T 8391f4ec0b1349f4a1758a0ce2382b7c P fd44949e40b247f980ba14623ba0dd34 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:21.086370 15256 raft_consensus.cc:3055] T 8391f4ec0b1349f4a1758a0ce2382b7c P 12806407b57c4ffbbf5f5baf439af582 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:18:21.086640 15256 raft_consensus.cc:740] T 8391f4ec0b1349f4a1758a0ce2382b7c P 12806407b57c4ffbbf5f5baf439af582 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 12806407b57c4ffbbf5f5baf439af582, State: Running, Role: LEADER
I20260709 13:18:21.087349 15256 consensus_queue.cc:260] T 8391f4ec0b1349f4a1758a0ce2382b7c P 12806407b57c4ffbbf5f5baf439af582 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd44949e40b247f980ba14623ba0dd34" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 37801 } } peers { permanent_uuid: "12806407b57c4ffbbf5f5baf439af582" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 45327 } } peers { permanent_uuid: "ce1dd5b4e0014ba8a3fa68f2ba226e49" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34123 } }
I20260709 13:18:21.088250 15256 raft_consensus.cc:3060] T 8391f4ec0b1349f4a1758a0ce2382b7c P 12806407b57c4ffbbf5f5baf439af582 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:21.090888 15179 raft_consensus.cc:2468] T 8391f4ec0b1349f4a1758a0ce2382b7c P fd44949e40b247f980ba14623ba0dd34 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce1dd5b4e0014ba8a3fa68f2ba226e49 in term 3.
I20260709 13:18:21.092044 15295 leader_election.cc:304] T 8391f4ec0b1349f4a1758a0ce2382b7c P ce1dd5b4e0014ba8a3fa68f2ba226e49 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce1dd5b4e0014ba8a3fa68f2ba226e49, fd44949e40b247f980ba14623ba0dd34; no voters: 
I20260709 13:18:21.092782 15463 raft_consensus.cc:2804] T 8391f4ec0b1349f4a1758a0ce2382b7c P ce1dd5b4e0014ba8a3fa68f2ba226e49 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:18:21.093257 15256 raft_consensus.cc:2468] T 8391f4ec0b1349f4a1758a0ce2382b7c P 12806407b57c4ffbbf5f5baf439af582 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce1dd5b4e0014ba8a3fa68f2ba226e49 in term 3.
I20260709 13:18:21.094053 15463 raft_consensus.cc:697] T 8391f4ec0b1349f4a1758a0ce2382b7c P ce1dd5b4e0014ba8a3fa68f2ba226e49 [term 3 LEADER]: Becoming Leader. State: Replica: ce1dd5b4e0014ba8a3fa68f2ba226e49, State: Running, Role: LEADER
I20260709 13:18:21.094820 15463 consensus_queue.cc:237] T 8391f4ec0b1349f4a1758a0ce2382b7c P ce1dd5b4e0014ba8a3fa68f2ba226e49 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd44949e40b247f980ba14623ba0dd34" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 37801 } } peers { permanent_uuid: "12806407b57c4ffbbf5f5baf439af582" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 45327 } } peers { permanent_uuid: "ce1dd5b4e0014ba8a3fa68f2ba226e49" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34123 } }
I20260709 13:18:21.103487 15076 catalog_manager.cc:5697] T 8391f4ec0b1349f4a1758a0ce2382b7c P ce1dd5b4e0014ba8a3fa68f2ba226e49 reported cstate change: term changed from 2 to 3, leader changed from 12806407b57c4ffbbf5f5baf439af582 (127.14.177.130) to ce1dd5b4e0014ba8a3fa68f2ba226e49 (127.14.177.131). New cstate: current_term: 3 leader_uuid: "ce1dd5b4e0014ba8a3fa68f2ba226e49" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd44949e40b247f980ba14623ba0dd34" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 37801 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "12806407b57c4ffbbf5f5baf439af582" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 45327 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ce1dd5b4e0014ba8a3fa68f2ba226e49" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34123 } health_report { overall_health: HEALTHY } } }
I20260709 13:18:21.127085 15462 raft_consensus.cc:993] T 98f6aef87e78472c9987c75cac6575af P fd44949e40b247f980ba14623ba0dd34: : Instructing follower ce1dd5b4e0014ba8a3fa68f2ba226e49 to start an election
I20260709 13:18:21.127487 15462 raft_consensus.cc:1081] T 98f6aef87e78472c9987c75cac6575af P fd44949e40b247f980ba14623ba0dd34 [term 1 LEADER]: Signalling peer ce1dd5b4e0014ba8a3fa68f2ba226e49 to start an election
I20260709 13:18:21.128965 15329 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "98f6aef87e78472c9987c75cac6575af"
dest_uuid: "ce1dd5b4e0014ba8a3fa68f2ba226e49"
 from {username='slave'} at 127.0.0.1:49256
I20260709 13:18:21.129420 15329 raft_consensus.cc:493] T 98f6aef87e78472c9987c75cac6575af P ce1dd5b4e0014ba8a3fa68f2ba226e49 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:18:21.129684 15329 raft_consensus.cc:3060] T 98f6aef87e78472c9987c75cac6575af P ce1dd5b4e0014ba8a3fa68f2ba226e49 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:21.133597 15329 raft_consensus.cc:515] T 98f6aef87e78472c9987c75cac6575af P ce1dd5b4e0014ba8a3fa68f2ba226e49 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd44949e40b247f980ba14623ba0dd34" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 37801 } } peers { permanent_uuid: "12806407b57c4ffbbf5f5baf439af582" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 45327 } } peers { permanent_uuid: "ce1dd5b4e0014ba8a3fa68f2ba226e49" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34123 } }
I20260709 13:18:21.135293 15329 leader_election.cc:290] T 98f6aef87e78472c9987c75cac6575af P ce1dd5b4e0014ba8a3fa68f2ba226e49 [CANDIDATE]: Term 2 election: Requested vote from peers fd44949e40b247f980ba14623ba0dd34 (127.14.177.129:37801), 12806407b57c4ffbbf5f5baf439af582 (127.14.177.130:45327)
I20260709 13:18:21.135999 15179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98f6aef87e78472c9987c75cac6575af" candidate_uuid: "ce1dd5b4e0014ba8a3fa68f2ba226e49" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "fd44949e40b247f980ba14623ba0dd34"
I20260709 13:18:21.136590 15179 raft_consensus.cc:3055] T 98f6aef87e78472c9987c75cac6575af P fd44949e40b247f980ba14623ba0dd34 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:18:21.136472 15256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98f6aef87e78472c9987c75cac6575af" candidate_uuid: "ce1dd5b4e0014ba8a3fa68f2ba226e49" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "12806407b57c4ffbbf5f5baf439af582"
I20260709 13:18:21.136924 15179 raft_consensus.cc:740] T 98f6aef87e78472c9987c75cac6575af P fd44949e40b247f980ba14623ba0dd34 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: fd44949e40b247f980ba14623ba0dd34, State: Running, Role: LEADER
I20260709 13:18:21.137082 15256 raft_consensus.cc:3060] T 98f6aef87e78472c9987c75cac6575af P 12806407b57c4ffbbf5f5baf439af582 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:21.137727 15179 consensus_queue.cc:260] T 98f6aef87e78472c9987c75cac6575af P fd44949e40b247f980ba14623ba0dd34 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd44949e40b247f980ba14623ba0dd34" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 37801 } } peers { permanent_uuid: "12806407b57c4ffbbf5f5baf439af582" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 45327 } } peers { permanent_uuid: "ce1dd5b4e0014ba8a3fa68f2ba226e49" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34123 } }
I20260709 13:18:21.138934 15179 raft_consensus.cc:3060] T 98f6aef87e78472c9987c75cac6575af P fd44949e40b247f980ba14623ba0dd34 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:21.143934 15256 raft_consensus.cc:2468] T 98f6aef87e78472c9987c75cac6575af P 12806407b57c4ffbbf5f5baf439af582 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce1dd5b4e0014ba8a3fa68f2ba226e49 in term 2.
I20260709 13:18:21.145188 15294 leader_election.cc:304] T 98f6aef87e78472c9987c75cac6575af P ce1dd5b4e0014ba8a3fa68f2ba226e49 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12806407b57c4ffbbf5f5baf439af582, ce1dd5b4e0014ba8a3fa68f2ba226e49; no voters: 
I20260709 13:18:21.145550 15179 raft_consensus.cc:2468] T 98f6aef87e78472c9987c75cac6575af P fd44949e40b247f980ba14623ba0dd34 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce1dd5b4e0014ba8a3fa68f2ba226e49 in term 2.
I20260709 13:18:21.146086 15463 raft_consensus.cc:2804] T 98f6aef87e78472c9987c75cac6575af P ce1dd5b4e0014ba8a3fa68f2ba226e49 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:18:21.146643 15463 raft_consensus.cc:697] T 98f6aef87e78472c9987c75cac6575af P ce1dd5b4e0014ba8a3fa68f2ba226e49 [term 2 LEADER]: Becoming Leader. State: Replica: ce1dd5b4e0014ba8a3fa68f2ba226e49, State: Running, Role: LEADER
I20260709 13:18:21.147564 15463 consensus_queue.cc:237] T 98f6aef87e78472c9987c75cac6575af P ce1dd5b4e0014ba8a3fa68f2ba226e49 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd44949e40b247f980ba14623ba0dd34" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 37801 } } peers { permanent_uuid: "12806407b57c4ffbbf5f5baf439af582" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 45327 } } peers { permanent_uuid: "ce1dd5b4e0014ba8a3fa68f2ba226e49" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34123 } }
I20260709 13:18:21.156138 15075 catalog_manager.cc:5697] T 98f6aef87e78472c9987c75cac6575af P ce1dd5b4e0014ba8a3fa68f2ba226e49 reported cstate change: term changed from 1 to 2, leader changed from fd44949e40b247f980ba14623ba0dd34 (127.14.177.129) to ce1dd5b4e0014ba8a3fa68f2ba226e49 (127.14.177.131). New cstate: current_term: 2 leader_uuid: "ce1dd5b4e0014ba8a3fa68f2ba226e49" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd44949e40b247f980ba14623ba0dd34" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 37801 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "12806407b57c4ffbbf5f5baf439af582" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 45327 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ce1dd5b4e0014ba8a3fa68f2ba226e49" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34123 } health_report { overall_health: HEALTHY } } }
I20260709 13:18:21.532338 15256 raft_consensus.cc:1275] T 8391f4ec0b1349f4a1758a0ce2382b7c P 12806407b57c4ffbbf5f5baf439af582 [term 3 FOLLOWER]: Refusing update from remote peer ce1dd5b4e0014ba8a3fa68f2ba226e49: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:18:21.533581 15463 consensus_queue.cc:1048] T 8391f4ec0b1349f4a1758a0ce2382b7c P ce1dd5b4e0014ba8a3fa68f2ba226e49 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12806407b57c4ffbbf5f5baf439af582" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 45327 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:18:21.543941 15179 raft_consensus.cc:1275] T 8391f4ec0b1349f4a1758a0ce2382b7c P fd44949e40b247f980ba14623ba0dd34 [term 3 FOLLOWER]: Refusing update from remote peer ce1dd5b4e0014ba8a3fa68f2ba226e49: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:18:21.545543 15463 consensus_queue.cc:1048] T 8391f4ec0b1349f4a1758a0ce2382b7c P ce1dd5b4e0014ba8a3fa68f2ba226e49 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fd44949e40b247f980ba14623ba0dd34" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:18:21.689911 15179 raft_consensus.cc:1275] T 98f6aef87e78472c9987c75cac6575af P fd44949e40b247f980ba14623ba0dd34 [term 2 FOLLOWER]: Refusing update from remote peer ce1dd5b4e0014ba8a3fa68f2ba226e49: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:18:21.691267 15479 consensus_queue.cc:1048] T 98f6aef87e78472c9987c75cac6575af P ce1dd5b4e0014ba8a3fa68f2ba226e49 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fd44949e40b247f980ba14623ba0dd34" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:18:21.696059 15256 raft_consensus.cc:1275] T 98f6aef87e78472c9987c75cac6575af P 12806407b57c4ffbbf5f5baf439af582 [term 2 FOLLOWER]: Refusing update from remote peer ce1dd5b4e0014ba8a3fa68f2ba226e49: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:18:21.697297 15479 consensus_queue.cc:1048] T 98f6aef87e78472c9987c75cac6575af P ce1dd5b4e0014ba8a3fa68f2ba226e49 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12806407b57c4ffbbf5f5baf439af582" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 45327 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:18:21.882700 15046 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:18:21.893975 15179 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "54dc07b181bd495d8c8e2cae81240f16"
dest_uuid: "fd44949e40b247f980ba14623ba0dd34"
mode: GRACEFUL
new_leader_uuid: "12806407b57c4ffbbf5f5baf439af582"
 from {username='slave'} at 127.0.0.1:42424
I20260709 13:18:21.894487 15179 raft_consensus.cc:606] T 54dc07b181bd495d8c8e2cae81240f16 P fd44949e40b247f980ba14623ba0dd34 [term 1 LEADER]: Received request to transfer leadership to TS 12806407b57c4ffbbf5f5baf439af582
I20260709 13:18:21.899070 15179 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "28ac227079274b8eb6a3e35e4bf34e30"
dest_uuid: "fd44949e40b247f980ba14623ba0dd34"
mode: GRACEFUL
new_leader_uuid: "12806407b57c4ffbbf5f5baf439af582"
 from {username='slave'} at 127.0.0.1:42424
I20260709 13:18:21.899587 15179 raft_consensus.cc:606] T 28ac227079274b8eb6a3e35e4bf34e30 P fd44949e40b247f980ba14623ba0dd34 [term 2 LEADER]: Received request to transfer leadership to TS 12806407b57c4ffbbf5f5baf439af582
I20260709 13:18:21.904202 15179 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "58a8c0164db24ffd8e992493c228804c"
dest_uuid: "fd44949e40b247f980ba14623ba0dd34"
mode: GRACEFUL
new_leader_uuid: "12806407b57c4ffbbf5f5baf439af582"
 from {username='slave'} at 127.0.0.1:42424
I20260709 13:18:21.904708 15179 raft_consensus.cc:606] T 58a8c0164db24ffd8e992493c228804c P fd44949e40b247f980ba14623ba0dd34 [term 2 LEADER]: Received request to transfer leadership to TS 12806407b57c4ffbbf5f5baf439af582
I20260709 13:18:21.909531 15329 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "98f6aef87e78472c9987c75cac6575af"
dest_uuid: "ce1dd5b4e0014ba8a3fa68f2ba226e49"
mode: GRACEFUL
new_leader_uuid: "12806407b57c4ffbbf5f5baf439af582"
 from {username='slave'} at 127.0.0.1:49286
I20260709 13:18:21.910032 15329 raft_consensus.cc:606] T 98f6aef87e78472c9987c75cac6575af P ce1dd5b4e0014ba8a3fa68f2ba226e49 [term 2 LEADER]: Received request to transfer leadership to TS 12806407b57c4ffbbf5f5baf439af582
I20260709 13:18:21.916987 15329 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8391f4ec0b1349f4a1758a0ce2382b7c"
dest_uuid: "ce1dd5b4e0014ba8a3fa68f2ba226e49"
mode: GRACEFUL
new_leader_uuid: "12806407b57c4ffbbf5f5baf439af582"
 from {username='slave'} at 127.0.0.1:49286
I20260709 13:18:21.917446 15329 raft_consensus.cc:606] T 8391f4ec0b1349f4a1758a0ce2382b7c P ce1dd5b4e0014ba8a3fa68f2ba226e49 [term 3 LEADER]: Received request to transfer leadership to TS 12806407b57c4ffbbf5f5baf439af582
I20260709 13:18:21.940598 15474 raft_consensus.cc:993] T 28ac227079274b8eb6a3e35e4bf34e30 P fd44949e40b247f980ba14623ba0dd34: : Instructing follower 12806407b57c4ffbbf5f5baf439af582 to start an election
I20260709 13:18:21.941100 15462 raft_consensus.cc:1081] T 28ac227079274b8eb6a3e35e4bf34e30 P fd44949e40b247f980ba14623ba0dd34 [term 2 LEADER]: Signalling peer 12806407b57c4ffbbf5f5baf439af582 to start an election
I20260709 13:18:21.942399 15256 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "28ac227079274b8eb6a3e35e4bf34e30"
dest_uuid: "12806407b57c4ffbbf5f5baf439af582"
 from {username='slave'} at 127.0.0.1:42048
I20260709 13:18:21.942791 15256 raft_consensus.cc:493] T 28ac227079274b8eb6a3e35e4bf34e30 P 12806407b57c4ffbbf5f5baf439af582 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:18:21.943058 15256 raft_consensus.cc:3060] T 28ac227079274b8eb6a3e35e4bf34e30 P 12806407b57c4ffbbf5f5baf439af582 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:21.947355 15256 raft_consensus.cc:515] T 28ac227079274b8eb6a3e35e4bf34e30 P 12806407b57c4ffbbf5f5baf439af582 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd44949e40b247f980ba14623ba0dd34" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 37801 } } peers { permanent_uuid: "12806407b57c4ffbbf5f5baf439af582" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 45327 } } peers { permanent_uuid: "ce1dd5b4e0014ba8a3fa68f2ba226e49" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34123 } }
I20260709 13:18:21.948879 15256 leader_election.cc:290] T 28ac227079274b8eb6a3e35e4bf34e30 P 12806407b57c4ffbbf5f5baf439af582 [CANDIDATE]: Term 3 election: Requested vote from peers fd44949e40b247f980ba14623ba0dd34 (127.14.177.129:37801), ce1dd5b4e0014ba8a3fa68f2ba226e49 (127.14.177.131:34123)
I20260709 13:18:21.949554 15179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28ac227079274b8eb6a3e35e4bf34e30" candidate_uuid: "12806407b57c4ffbbf5f5baf439af582" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "fd44949e40b247f980ba14623ba0dd34"
I20260709 13:18:21.949815 15329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28ac227079274b8eb6a3e35e4bf34e30" candidate_uuid: "12806407b57c4ffbbf5f5baf439af582" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "ce1dd5b4e0014ba8a3fa68f2ba226e49"
I20260709 13:18:21.950093 15179 raft_consensus.cc:3055] T 28ac227079274b8eb6a3e35e4bf34e30 P fd44949e40b247f980ba14623ba0dd34 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:18:21.950307 15329 raft_consensus.cc:3060] T 28ac227079274b8eb6a3e35e4bf34e30 P ce1dd5b4e0014ba8a3fa68f2ba226e49 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:21.950443 15179 raft_consensus.cc:740] T 28ac227079274b8eb6a3e35e4bf34e30 P fd44949e40b247f980ba14623ba0dd34 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: fd44949e40b247f980ba14623ba0dd34, State: Running, Role: LEADER
I20260709 13:18:21.951303 15179 consensus_queue.cc:260] T 28ac227079274b8eb6a3e35e4bf34e30 P fd44949e40b247f980ba14623ba0dd34 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd44949e40b247f980ba14623ba0dd34" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 37801 } } peers { permanent_uuid: "12806407b57c4ffbbf5f5baf439af582" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 45327 } } peers { permanent_uuid: "ce1dd5b4e0014ba8a3fa68f2ba226e49" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34123 } }
I20260709 13:18:21.952162 15179 raft_consensus.cc:3060] T 28ac227079274b8eb6a3e35e4bf34e30 P fd44949e40b247f980ba14623ba0dd34 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:21.954291 15329 raft_consensus.cc:2468] T 28ac227079274b8eb6a3e35e4bf34e30 P ce1dd5b4e0014ba8a3fa68f2ba226e49 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12806407b57c4ffbbf5f5baf439af582 in term 3.
I20260709 13:18:21.955310 15221 leader_election.cc:304] T 28ac227079274b8eb6a3e35e4bf34e30 P 12806407b57c4ffbbf5f5baf439af582 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12806407b57c4ffbbf5f5baf439af582, ce1dd5b4e0014ba8a3fa68f2ba226e49; no voters: 
I20260709 13:18:21.956130 15486 raft_consensus.cc:2804] T 28ac227079274b8eb6a3e35e4bf34e30 P 12806407b57c4ffbbf5f5baf439af582 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:18:21.956573 15486 raft_consensus.cc:697] T 28ac227079274b8eb6a3e35e4bf34e30 P 12806407b57c4ffbbf5f5baf439af582 [term 3 LEADER]: Becoming Leader. State: Replica: 12806407b57c4ffbbf5f5baf439af582, State: Running, Role: LEADER
I20260709 13:18:21.957108 15179 raft_consensus.cc:2468] T 28ac227079274b8eb6a3e35e4bf34e30 P fd44949e40b247f980ba14623ba0dd34 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12806407b57c4ffbbf5f5baf439af582 in term 3.
I20260709 13:18:21.957437 15486 consensus_queue.cc:237] T 28ac227079274b8eb6a3e35e4bf34e30 P 12806407b57c4ffbbf5f5baf439af582 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd44949e40b247f980ba14623ba0dd34" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 37801 } } peers { permanent_uuid: "12806407b57c4ffbbf5f5baf439af582" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 45327 } } peers { permanent_uuid: "ce1dd5b4e0014ba8a3fa68f2ba226e49" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34123 } }
I20260709 13:18:21.965440 15077 catalog_manager.cc:5697] T 28ac227079274b8eb6a3e35e4bf34e30 P 12806407b57c4ffbbf5f5baf439af582 reported cstate change: term changed from 2 to 3, leader changed from fd44949e40b247f980ba14623ba0dd34 (127.14.177.129) to 12806407b57c4ffbbf5f5baf439af582 (127.14.177.130). New cstate: current_term: 3 leader_uuid: "12806407b57c4ffbbf5f5baf439af582" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd44949e40b247f980ba14623ba0dd34" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 37801 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "12806407b57c4ffbbf5f5baf439af582" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 45327 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ce1dd5b4e0014ba8a3fa68f2ba226e49" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34123 } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:22.110167 15468 raft_consensus.cc:993] T 8391f4ec0b1349f4a1758a0ce2382b7c P ce1dd5b4e0014ba8a3fa68f2ba226e49: : Instructing follower 12806407b57c4ffbbf5f5baf439af582 to start an election
I20260709 13:18:22.110611 15463 raft_consensus.cc:1081] T 8391f4ec0b1349f4a1758a0ce2382b7c P ce1dd5b4e0014ba8a3fa68f2ba226e49 [term 3 LEADER]: Signalling peer 12806407b57c4ffbbf5f5baf439af582 to start an election
I20260709 13:18:22.112030 15256 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "8391f4ec0b1349f4a1758a0ce2382b7c"
dest_uuid: "12806407b57c4ffbbf5f5baf439af582"
 from {username='slave'} at 127.0.0.1:42046
I20260709 13:18:22.112525 15256 raft_consensus.cc:493] T 8391f4ec0b1349f4a1758a0ce2382b7c P 12806407b57c4ffbbf5f5baf439af582 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:18:22.112841 15256 raft_consensus.cc:3060] T 8391f4ec0b1349f4a1758a0ce2382b7c P 12806407b57c4ffbbf5f5baf439af582 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:22.117172 15256 raft_consensus.cc:515] T 8391f4ec0b1349f4a1758a0ce2382b7c P 12806407b57c4ffbbf5f5baf439af582 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd44949e40b247f980ba14623ba0dd34" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 37801 } } peers { permanent_uuid: "12806407b57c4ffbbf5f5baf439af582" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 45327 } } peers { permanent_uuid: "ce1dd5b4e0014ba8a3fa68f2ba226e49" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34123 } }
I20260709 13:18:22.118516 15256 leader_election.cc:290] T 8391f4ec0b1349f4a1758a0ce2382b7c P 12806407b57c4ffbbf5f5baf439af582 [CANDIDATE]: Term 4 election: Requested vote from peers fd44949e40b247f980ba14623ba0dd34 (127.14.177.129:37801), ce1dd5b4e0014ba8a3fa68f2ba226e49 (127.14.177.131:34123)
I20260709 13:18:22.119390 15179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8391f4ec0b1349f4a1758a0ce2382b7c" candidate_uuid: "12806407b57c4ffbbf5f5baf439af582" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "fd44949e40b247f980ba14623ba0dd34"
I20260709 13:18:22.119633 15329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8391f4ec0b1349f4a1758a0ce2382b7c" candidate_uuid: "12806407b57c4ffbbf5f5baf439af582" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "ce1dd5b4e0014ba8a3fa68f2ba226e49"
I20260709 13:18:22.119904 15179 raft_consensus.cc:3060] T 8391f4ec0b1349f4a1758a0ce2382b7c P fd44949e40b247f980ba14623ba0dd34 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:22.120115 15329 raft_consensus.cc:3055] T 8391f4ec0b1349f4a1758a0ce2382b7c P ce1dd5b4e0014ba8a3fa68f2ba226e49 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:18:22.120422 15329 raft_consensus.cc:740] T 8391f4ec0b1349f4a1758a0ce2382b7c P ce1dd5b4e0014ba8a3fa68f2ba226e49 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: ce1dd5b4e0014ba8a3fa68f2ba226e49, State: Running, Role: LEADER
I20260709 13:18:22.121025 15329 consensus_queue.cc:260] T 8391f4ec0b1349f4a1758a0ce2382b7c P ce1dd5b4e0014ba8a3fa68f2ba226e49 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd44949e40b247f980ba14623ba0dd34" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 37801 } } peers { permanent_uuid: "12806407b57c4ffbbf5f5baf439af582" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 45327 } } peers { permanent_uuid: "ce1dd5b4e0014ba8a3fa68f2ba226e49" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34123 } }
I20260709 13:18:22.121790 15329 raft_consensus.cc:3060] T 8391f4ec0b1349f4a1758a0ce2382b7c P ce1dd5b4e0014ba8a3fa68f2ba226e49 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:22.124431 15179 raft_consensus.cc:2468] T 8391f4ec0b1349f4a1758a0ce2382b7c P fd44949e40b247f980ba14623ba0dd34 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12806407b57c4ffbbf5f5baf439af582 in term 4.
I20260709 13:18:22.125490 15221 leader_election.cc:304] T 8391f4ec0b1349f4a1758a0ce2382b7c P 12806407b57c4ffbbf5f5baf439af582 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12806407b57c4ffbbf5f5baf439af582, fd44949e40b247f980ba14623ba0dd34; no voters: 
I20260709 13:18:22.125759 15329 raft_consensus.cc:2468] T 8391f4ec0b1349f4a1758a0ce2382b7c P ce1dd5b4e0014ba8a3fa68f2ba226e49 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12806407b57c4ffbbf5f5baf439af582 in term 4.
I20260709 13:18:22.126150 15486 raft_consensus.cc:2804] T 8391f4ec0b1349f4a1758a0ce2382b7c P 12806407b57c4ffbbf5f5baf439af582 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:18:22.126662 15486 raft_consensus.cc:697] T 8391f4ec0b1349f4a1758a0ce2382b7c P 12806407b57c4ffbbf5f5baf439af582 [term 4 LEADER]: Becoming Leader. State: Replica: 12806407b57c4ffbbf5f5baf439af582, State: Running, Role: LEADER
I20260709 13:18:22.127533 15486 consensus_queue.cc:237] T 8391f4ec0b1349f4a1758a0ce2382b7c P 12806407b57c4ffbbf5f5baf439af582 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd44949e40b247f980ba14623ba0dd34" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 37801 } } peers { permanent_uuid: "12806407b57c4ffbbf5f5baf439af582" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 45327 } } peers { permanent_uuid: "ce1dd5b4e0014ba8a3fa68f2ba226e49" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34123 } }
I20260709 13:18:22.132731 15077 catalog_manager.cc:5697] T 8391f4ec0b1349f4a1758a0ce2382b7c P 12806407b57c4ffbbf5f5baf439af582 reported cstate change: term changed from 3 to 4, leader changed from ce1dd5b4e0014ba8a3fa68f2ba226e49 (127.14.177.131) to 12806407b57c4ffbbf5f5baf439af582 (127.14.177.130). New cstate: current_term: 4 leader_uuid: "12806407b57c4ffbbf5f5baf439af582" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd44949e40b247f980ba14623ba0dd34" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 37801 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "12806407b57c4ffbbf5f5baf439af582" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 45327 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ce1dd5b4e0014ba8a3fa68f2ba226e49" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34123 } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:22.208070 15468 raft_consensus.cc:993] T 98f6aef87e78472c9987c75cac6575af P ce1dd5b4e0014ba8a3fa68f2ba226e49: : Instructing follower 12806407b57c4ffbbf5f5baf439af582 to start an election
I20260709 13:18:22.208619 15463 raft_consensus.cc:1081] T 98f6aef87e78472c9987c75cac6575af P ce1dd5b4e0014ba8a3fa68f2ba226e49 [term 2 LEADER]: Signalling peer 12806407b57c4ffbbf5f5baf439af582 to start an election
I20260709 13:18:22.210471 15256 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "98f6aef87e78472c9987c75cac6575af"
dest_uuid: "12806407b57c4ffbbf5f5baf439af582"
 from {username='slave'} at 127.0.0.1:42046
I20260709 13:18:22.211208 15256 raft_consensus.cc:493] T 98f6aef87e78472c9987c75cac6575af P 12806407b57c4ffbbf5f5baf439af582 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:18:22.211542 15256 raft_consensus.cc:3060] T 98f6aef87e78472c9987c75cac6575af P 12806407b57c4ffbbf5f5baf439af582 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:22.215763 15256 raft_consensus.cc:515] T 98f6aef87e78472c9987c75cac6575af P 12806407b57c4ffbbf5f5baf439af582 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd44949e40b247f980ba14623ba0dd34" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 37801 } } peers { permanent_uuid: "12806407b57c4ffbbf5f5baf439af582" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 45327 } } peers { permanent_uuid: "ce1dd5b4e0014ba8a3fa68f2ba226e49" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34123 } }
I20260709 13:18:22.217397 15256 leader_election.cc:290] T 98f6aef87e78472c9987c75cac6575af P 12806407b57c4ffbbf5f5baf439af582 [CANDIDATE]: Term 3 election: Requested vote from peers fd44949e40b247f980ba14623ba0dd34 (127.14.177.129:37801), ce1dd5b4e0014ba8a3fa68f2ba226e49 (127.14.177.131:34123)
I20260709 13:18:22.218253 15179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98f6aef87e78472c9987c75cac6575af" candidate_uuid: "12806407b57c4ffbbf5f5baf439af582" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "fd44949e40b247f980ba14623ba0dd34"
I20260709 13:18:22.218477 15329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98f6aef87e78472c9987c75cac6575af" candidate_uuid: "12806407b57c4ffbbf5f5baf439af582" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "ce1dd5b4e0014ba8a3fa68f2ba226e49"
I20260709 13:18:22.218820 15179 raft_consensus.cc:3060] T 98f6aef87e78472c9987c75cac6575af P fd44949e40b247f980ba14623ba0dd34 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:22.219012 15329 raft_consensus.cc:3055] T 98f6aef87e78472c9987c75cac6575af P ce1dd5b4e0014ba8a3fa68f2ba226e49 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:18:22.219439 15329 raft_consensus.cc:740] T 98f6aef87e78472c9987c75cac6575af P ce1dd5b4e0014ba8a3fa68f2ba226e49 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: ce1dd5b4e0014ba8a3fa68f2ba226e49, State: Running, Role: LEADER
I20260709 13:18:22.220116 15329 consensus_queue.cc:260] T 98f6aef87e78472c9987c75cac6575af P ce1dd5b4e0014ba8a3fa68f2ba226e49 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd44949e40b247f980ba14623ba0dd34" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 37801 } } peers { permanent_uuid: "12806407b57c4ffbbf5f5baf439af582" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 45327 } } peers { permanent_uuid: "ce1dd5b4e0014ba8a3fa68f2ba226e49" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34123 } }
I20260709 13:18:22.220984 15329 raft_consensus.cc:3060] T 98f6aef87e78472c9987c75cac6575af P ce1dd5b4e0014ba8a3fa68f2ba226e49 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:22.223489 15179 raft_consensus.cc:2468] T 98f6aef87e78472c9987c75cac6575af P fd44949e40b247f980ba14623ba0dd34 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12806407b57c4ffbbf5f5baf439af582 in term 3.
I20260709 13:18:22.224448 15221 leader_election.cc:304] T 98f6aef87e78472c9987c75cac6575af P 12806407b57c4ffbbf5f5baf439af582 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12806407b57c4ffbbf5f5baf439af582, fd44949e40b247f980ba14623ba0dd34; no voters: 
I20260709 13:18:22.225119 15486 raft_consensus.cc:2804] T 98f6aef87e78472c9987c75cac6575af P 12806407b57c4ffbbf5f5baf439af582 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:18:22.225521 15486 raft_consensus.cc:697] T 98f6aef87e78472c9987c75cac6575af P 12806407b57c4ffbbf5f5baf439af582 [term 3 LEADER]: Becoming Leader. State: Replica: 12806407b57c4ffbbf5f5baf439af582, State: Running, Role: LEADER
I20260709 13:18:22.225526 15329 raft_consensus.cc:2468] T 98f6aef87e78472c9987c75cac6575af P ce1dd5b4e0014ba8a3fa68f2ba226e49 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12806407b57c4ffbbf5f5baf439af582 in term 3.
I20260709 13:18:22.226547 15486 consensus_queue.cc:237] T 98f6aef87e78472c9987c75cac6575af P 12806407b57c4ffbbf5f5baf439af582 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd44949e40b247f980ba14623ba0dd34" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 37801 } } peers { permanent_uuid: "12806407b57c4ffbbf5f5baf439af582" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 45327 } } peers { permanent_uuid: "ce1dd5b4e0014ba8a3fa68f2ba226e49" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34123 } }
I20260709 13:18:22.232762 15077 catalog_manager.cc:5697] T 98f6aef87e78472c9987c75cac6575af P 12806407b57c4ffbbf5f5baf439af582 reported cstate change: term changed from 2 to 3, leader changed from ce1dd5b4e0014ba8a3fa68f2ba226e49 (127.14.177.131) to 12806407b57c4ffbbf5f5baf439af582 (127.14.177.130). New cstate: current_term: 3 leader_uuid: "12806407b57c4ffbbf5f5baf439af582" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd44949e40b247f980ba14623ba0dd34" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 37801 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "12806407b57c4ffbbf5f5baf439af582" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 45327 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ce1dd5b4e0014ba8a3fa68f2ba226e49" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34123 } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:22.300239 15474 raft_consensus.cc:993] T 58a8c0164db24ffd8e992493c228804c P fd44949e40b247f980ba14623ba0dd34: : Instructing follower 12806407b57c4ffbbf5f5baf439af582 to start an election
I20260709 13:18:22.300659 15462 raft_consensus.cc:1081] T 58a8c0164db24ffd8e992493c228804c P fd44949e40b247f980ba14623ba0dd34 [term 2 LEADER]: Signalling peer 12806407b57c4ffbbf5f5baf439af582 to start an election
I20260709 13:18:22.301985 15256 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "58a8c0164db24ffd8e992493c228804c"
dest_uuid: "12806407b57c4ffbbf5f5baf439af582"
 from {username='slave'} at 127.0.0.1:42048
I20260709 13:18:22.302454 15256 raft_consensus.cc:493] T 58a8c0164db24ffd8e992493c228804c P 12806407b57c4ffbbf5f5baf439af582 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:18:22.302718 15256 raft_consensus.cc:3060] T 58a8c0164db24ffd8e992493c228804c P 12806407b57c4ffbbf5f5baf439af582 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:22.306488 15256 raft_consensus.cc:515] T 58a8c0164db24ffd8e992493c228804c P 12806407b57c4ffbbf5f5baf439af582 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd44949e40b247f980ba14623ba0dd34" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 37801 } } peers { permanent_uuid: "12806407b57c4ffbbf5f5baf439af582" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 45327 } } peers { permanent_uuid: "ce1dd5b4e0014ba8a3fa68f2ba226e49" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34123 } }
I20260709 13:18:22.307976 15256 leader_election.cc:290] T 58a8c0164db24ffd8e992493c228804c P 12806407b57c4ffbbf5f5baf439af582 [CANDIDATE]: Term 3 election: Requested vote from peers fd44949e40b247f980ba14623ba0dd34 (127.14.177.129:37801), ce1dd5b4e0014ba8a3fa68f2ba226e49 (127.14.177.131:34123)
I20260709 13:18:22.308693 15179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58a8c0164db24ffd8e992493c228804c" candidate_uuid: "12806407b57c4ffbbf5f5baf439af582" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "fd44949e40b247f980ba14623ba0dd34"
I20260709 13:18:22.308995 15329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58a8c0164db24ffd8e992493c228804c" candidate_uuid: "12806407b57c4ffbbf5f5baf439af582" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "ce1dd5b4e0014ba8a3fa68f2ba226e49"
I20260709 13:18:22.309312 15179 raft_consensus.cc:3055] T 58a8c0164db24ffd8e992493c228804c P fd44949e40b247f980ba14623ba0dd34 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:18:22.309490 15329 raft_consensus.cc:3060] T 58a8c0164db24ffd8e992493c228804c P ce1dd5b4e0014ba8a3fa68f2ba226e49 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:22.309648 15179 raft_consensus.cc:740] T 58a8c0164db24ffd8e992493c228804c P fd44949e40b247f980ba14623ba0dd34 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: fd44949e40b247f980ba14623ba0dd34, State: Running, Role: LEADER
I20260709 13:18:22.310436 15179 consensus_queue.cc:260] T 58a8c0164db24ffd8e992493c228804c P fd44949e40b247f980ba14623ba0dd34 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd44949e40b247f980ba14623ba0dd34" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 37801 } } peers { permanent_uuid: "12806407b57c4ffbbf5f5baf439af582" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 45327 } } peers { permanent_uuid: "ce1dd5b4e0014ba8a3fa68f2ba226e49" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34123 } }
I20260709 13:18:22.311602 15179 raft_consensus.cc:3060] T 58a8c0164db24ffd8e992493c228804c P fd44949e40b247f980ba14623ba0dd34 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:22.315564 15329 raft_consensus.cc:2468] T 58a8c0164db24ffd8e992493c228804c P ce1dd5b4e0014ba8a3fa68f2ba226e49 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12806407b57c4ffbbf5f5baf439af582 in term 3.
I20260709 13:18:22.316596 15221 leader_election.cc:304] T 58a8c0164db24ffd8e992493c228804c P 12806407b57c4ffbbf5f5baf439af582 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12806407b57c4ffbbf5f5baf439af582, ce1dd5b4e0014ba8a3fa68f2ba226e49; no voters: 
I20260709 13:18:22.317308 15486 raft_consensus.cc:2804] T 58a8c0164db24ffd8e992493c228804c P 12806407b57c4ffbbf5f5baf439af582 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:18:22.317673 15179 raft_consensus.cc:2468] T 58a8c0164db24ffd8e992493c228804c P fd44949e40b247f980ba14623ba0dd34 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12806407b57c4ffbbf5f5baf439af582 in term 3.
I20260709 13:18:22.317773 15486 raft_consensus.cc:697] T 58a8c0164db24ffd8e992493c228804c P 12806407b57c4ffbbf5f5baf439af582 [term 3 LEADER]: Becoming Leader. State: Replica: 12806407b57c4ffbbf5f5baf439af582, State: Running, Role: LEADER
I20260709 13:18:22.318670 15486 consensus_queue.cc:237] T 58a8c0164db24ffd8e992493c228804c P 12806407b57c4ffbbf5f5baf439af582 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd44949e40b247f980ba14623ba0dd34" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 37801 } } peers { permanent_uuid: "12806407b57c4ffbbf5f5baf439af582" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 45327 } } peers { permanent_uuid: "ce1dd5b4e0014ba8a3fa68f2ba226e49" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34123 } }
I20260709 13:18:22.325008 15077 catalog_manager.cc:5697] T 58a8c0164db24ffd8e992493c228804c P 12806407b57c4ffbbf5f5baf439af582 reported cstate change: term changed from 2 to 3, leader changed from fd44949e40b247f980ba14623ba0dd34 (127.14.177.129) to 12806407b57c4ffbbf5f5baf439af582 (127.14.177.130). New cstate: current_term: 3 leader_uuid: "12806407b57c4ffbbf5f5baf439af582" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd44949e40b247f980ba14623ba0dd34" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 37801 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "12806407b57c4ffbbf5f5baf439af582" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 45327 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ce1dd5b4e0014ba8a3fa68f2ba226e49" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34123 } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:22.331790 15474 raft_consensus.cc:993] T 54dc07b181bd495d8c8e2cae81240f16 P fd44949e40b247f980ba14623ba0dd34: : Instructing follower 12806407b57c4ffbbf5f5baf439af582 to start an election
I20260709 13:18:22.332238 15462 raft_consensus.cc:1081] T 54dc07b181bd495d8c8e2cae81240f16 P fd44949e40b247f980ba14623ba0dd34 [term 1 LEADER]: Signalling peer 12806407b57c4ffbbf5f5baf439af582 to start an election
I20260709 13:18:22.333550 15256 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "54dc07b181bd495d8c8e2cae81240f16"
dest_uuid: "12806407b57c4ffbbf5f5baf439af582"
 from {username='slave'} at 127.0.0.1:42048
I20260709 13:18:22.334005 15256 raft_consensus.cc:493] T 54dc07b181bd495d8c8e2cae81240f16 P 12806407b57c4ffbbf5f5baf439af582 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:18:22.334486 15256 raft_consensus.cc:3060] T 54dc07b181bd495d8c8e2cae81240f16 P 12806407b57c4ffbbf5f5baf439af582 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:22.339226 15256 raft_consensus.cc:515] T 54dc07b181bd495d8c8e2cae81240f16 P 12806407b57c4ffbbf5f5baf439af582 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce1dd5b4e0014ba8a3fa68f2ba226e49" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34123 } } peers { permanent_uuid: "fd44949e40b247f980ba14623ba0dd34" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 37801 } } peers { permanent_uuid: "12806407b57c4ffbbf5f5baf439af582" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 45327 } }
I20260709 13:18:22.341100 15256 leader_election.cc:290] T 54dc07b181bd495d8c8e2cae81240f16 P 12806407b57c4ffbbf5f5baf439af582 [CANDIDATE]: Term 2 election: Requested vote from peers ce1dd5b4e0014ba8a3fa68f2ba226e49 (127.14.177.131:34123), fd44949e40b247f980ba14623ba0dd34 (127.14.177.129:37801)
I20260709 13:18:22.341862 15329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54dc07b181bd495d8c8e2cae81240f16" candidate_uuid: "12806407b57c4ffbbf5f5baf439af582" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "ce1dd5b4e0014ba8a3fa68f2ba226e49"
I20260709 13:18:22.342432 15329 raft_consensus.cc:3060] T 54dc07b181bd495d8c8e2cae81240f16 P ce1dd5b4e0014ba8a3fa68f2ba226e49 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:22.342422 15179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54dc07b181bd495d8c8e2cae81240f16" candidate_uuid: "12806407b57c4ffbbf5f5baf439af582" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "fd44949e40b247f980ba14623ba0dd34"
I20260709 13:18:22.342948 15179 raft_consensus.cc:3055] T 54dc07b181bd495d8c8e2cae81240f16 P fd44949e40b247f980ba14623ba0dd34 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:18:22.343269 15179 raft_consensus.cc:740] T 54dc07b181bd495d8c8e2cae81240f16 P fd44949e40b247f980ba14623ba0dd34 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: fd44949e40b247f980ba14623ba0dd34, State: Running, Role: LEADER
I20260709 13:18:22.343853 15179 consensus_queue.cc:260] T 54dc07b181bd495d8c8e2cae81240f16 P fd44949e40b247f980ba14623ba0dd34 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce1dd5b4e0014ba8a3fa68f2ba226e49" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34123 } } peers { permanent_uuid: "fd44949e40b247f980ba14623ba0dd34" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 37801 } } peers { permanent_uuid: "12806407b57c4ffbbf5f5baf439af582" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 45327 } }
I20260709 13:18:22.344695 15179 raft_consensus.cc:3060] T 54dc07b181bd495d8c8e2cae81240f16 P fd44949e40b247f980ba14623ba0dd34 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:22.346925 15329 raft_consensus.cc:2468] T 54dc07b181bd495d8c8e2cae81240f16 P ce1dd5b4e0014ba8a3fa68f2ba226e49 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12806407b57c4ffbbf5f5baf439af582 in term 2.
I20260709 13:18:22.347911 15221 leader_election.cc:304] T 54dc07b181bd495d8c8e2cae81240f16 P 12806407b57c4ffbbf5f5baf439af582 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12806407b57c4ffbbf5f5baf439af582, ce1dd5b4e0014ba8a3fa68f2ba226e49; no voters: 
I20260709 13:18:22.348707 15411 raft_consensus.cc:2804] T 54dc07b181bd495d8c8e2cae81240f16 P 12806407b57c4ffbbf5f5baf439af582 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:18:22.349149 15411 raft_consensus.cc:697] T 54dc07b181bd495d8c8e2cae81240f16 P 12806407b57c4ffbbf5f5baf439af582 [term 2 LEADER]: Becoming Leader. State: Replica: 12806407b57c4ffbbf5f5baf439af582, State: Running, Role: LEADER
I20260709 13:18:22.349713 15179 raft_consensus.cc:2468] T 54dc07b181bd495d8c8e2cae81240f16 P fd44949e40b247f980ba14623ba0dd34 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12806407b57c4ffbbf5f5baf439af582 in term 2.
I20260709 13:18:22.349834 15411 consensus_queue.cc:237] T 54dc07b181bd495d8c8e2cae81240f16 P 12806407b57c4ffbbf5f5baf439af582 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce1dd5b4e0014ba8a3fa68f2ba226e49" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34123 } } peers { permanent_uuid: "fd44949e40b247f980ba14623ba0dd34" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 37801 } } peers { permanent_uuid: "12806407b57c4ffbbf5f5baf439af582" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 45327 } }
I20260709 13:18:22.355939 15077 catalog_manager.cc:5697] T 54dc07b181bd495d8c8e2cae81240f16 P 12806407b57c4ffbbf5f5baf439af582 reported cstate change: term changed from 1 to 2, leader changed from fd44949e40b247f980ba14623ba0dd34 (127.14.177.129) to 12806407b57c4ffbbf5f5baf439af582 (127.14.177.130). New cstate: current_term: 2 leader_uuid: "12806407b57c4ffbbf5f5baf439af582" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce1dd5b4e0014ba8a3fa68f2ba226e49" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34123 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fd44949e40b247f980ba14623ba0dd34" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 37801 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "12806407b57c4ffbbf5f5baf439af582" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 45327 } health_report { overall_health: HEALTHY } } }
I20260709 13:18:22.400393 15329 raft_consensus.cc:1275] T 28ac227079274b8eb6a3e35e4bf34e30 P ce1dd5b4e0014ba8a3fa68f2ba226e49 [term 3 FOLLOWER]: Refusing update from remote peer 12806407b57c4ffbbf5f5baf439af582: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:18:22.401661 15411 consensus_queue.cc:1048] T 28ac227079274b8eb6a3e35e4bf34e30 P 12806407b57c4ffbbf5f5baf439af582 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce1dd5b4e0014ba8a3fa68f2ba226e49" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:18:22.411561 15179 raft_consensus.cc:1275] T 28ac227079274b8eb6a3e35e4bf34e30 P fd44949e40b247f980ba14623ba0dd34 [term 3 FOLLOWER]: Refusing update from remote peer 12806407b57c4ffbbf5f5baf439af582: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:18:22.413367 15495 consensus_queue.cc:1048] T 28ac227079274b8eb6a3e35e4bf34e30 P 12806407b57c4ffbbf5f5baf439af582 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fd44949e40b247f980ba14623ba0dd34" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:18:22.554332 15179 raft_consensus.cc:1275] T 8391f4ec0b1349f4a1758a0ce2382b7c P fd44949e40b247f980ba14623ba0dd34 [term 4 FOLLOWER]: Refusing update from remote peer 12806407b57c4ffbbf5f5baf439af582: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:18:22.555673 15411 consensus_queue.cc:1048] T 8391f4ec0b1349f4a1758a0ce2382b7c P 12806407b57c4ffbbf5f5baf439af582 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fd44949e40b247f980ba14623ba0dd34" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:18:22.564110 15329 raft_consensus.cc:1275] T 8391f4ec0b1349f4a1758a0ce2382b7c P ce1dd5b4e0014ba8a3fa68f2ba226e49 [term 4 FOLLOWER]: Refusing update from remote peer 12806407b57c4ffbbf5f5baf439af582: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:18:22.565445 15500 consensus_queue.cc:1048] T 8391f4ec0b1349f4a1758a0ce2382b7c P 12806407b57c4ffbbf5f5baf439af582 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce1dd5b4e0014ba8a3fa68f2ba226e49" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:18:22.701552 15329 raft_consensus.cc:1275] T 98f6aef87e78472c9987c75cac6575af P ce1dd5b4e0014ba8a3fa68f2ba226e49 [term 3 FOLLOWER]: Refusing update from remote peer 12806407b57c4ffbbf5f5baf439af582: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:18:22.702862 15500 consensus_queue.cc:1048] T 98f6aef87e78472c9987c75cac6575af P 12806407b57c4ffbbf5f5baf439af582 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce1dd5b4e0014ba8a3fa68f2ba226e49" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:18:22.712497 15179 raft_consensus.cc:1275] T 98f6aef87e78472c9987c75cac6575af P fd44949e40b247f980ba14623ba0dd34 [term 3 FOLLOWER]: Refusing update from remote peer 12806407b57c4ffbbf5f5baf439af582: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:18:22.714151 15486 consensus_queue.cc:1048] T 98f6aef87e78472c9987c75cac6575af P 12806407b57c4ffbbf5f5baf439af582 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fd44949e40b247f980ba14623ba0dd34" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:18:22.783623 15180 raft_consensus.cc:1275] T 58a8c0164db24ffd8e992493c228804c P fd44949e40b247f980ba14623ba0dd34 [term 3 FOLLOWER]: Refusing update from remote peer 12806407b57c4ffbbf5f5baf439af582: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:18:22.784771 15411 consensus_queue.cc:1048] T 58a8c0164db24ffd8e992493c228804c P 12806407b57c4ffbbf5f5baf439af582 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fd44949e40b247f980ba14623ba0dd34" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:18:22.794265 15329 raft_consensus.cc:1275] T 58a8c0164db24ffd8e992493c228804c P ce1dd5b4e0014ba8a3fa68f2ba226e49 [term 3 FOLLOWER]: Refusing update from remote peer 12806407b57c4ffbbf5f5baf439af582: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:18:22.795660 15486 consensus_queue.cc:1048] T 58a8c0164db24ffd8e992493c228804c P 12806407b57c4ffbbf5f5baf439af582 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce1dd5b4e0014ba8a3fa68f2ba226e49" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:18:22.914553 15180 raft_consensus.cc:1275] T 54dc07b181bd495d8c8e2cae81240f16 P fd44949e40b247f980ba14623ba0dd34 [term 2 FOLLOWER]: Refusing update from remote peer 12806407b57c4ffbbf5f5baf439af582: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:18:22.915872 15486 consensus_queue.cc:1048] T 54dc07b181bd495d8c8e2cae81240f16 P 12806407b57c4ffbbf5f5baf439af582 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fd44949e40b247f980ba14623ba0dd34" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:18:22.921953 15329 raft_consensus.cc:1275] T 54dc07b181bd495d8c8e2cae81240f16 P ce1dd5b4e0014ba8a3fa68f2ba226e49 [term 2 FOLLOWER]: Refusing update from remote peer 12806407b57c4ffbbf5f5baf439af582: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:18:22.923507 15411 consensus_queue.cc:1048] T 54dc07b181bd495d8c8e2cae81240f16 P 12806407b57c4ffbbf5f5baf439af582 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce1dd5b4e0014ba8a3fa68f2ba226e49" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:18:24.927327 15046 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:18:24.934315 15256 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "54dc07b181bd495d8c8e2cae81240f16"
dest_uuid: "12806407b57c4ffbbf5f5baf439af582"
mode: GRACEFUL
new_leader_uuid: "fd44949e40b247f980ba14623ba0dd34"
 from {username='slave'} at 127.0.0.1:60774
I20260709 13:18:24.935087 15256 raft_consensus.cc:606] T 54dc07b181bd495d8c8e2cae81240f16 P 12806407b57c4ffbbf5f5baf439af582 [term 2 LEADER]: Received request to transfer leadership to TS fd44949e40b247f980ba14623ba0dd34
I20260709 13:18:24.938149 15256 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "58a8c0164db24ffd8e992493c228804c"
dest_uuid: "12806407b57c4ffbbf5f5baf439af582"
mode: GRACEFUL
new_leader_uuid: "fd44949e40b247f980ba14623ba0dd34"
 from {username='slave'} at 127.0.0.1:60774
I20260709 13:18:24.938578 15256 raft_consensus.cc:606] T 58a8c0164db24ffd8e992493c228804c P 12806407b57c4ffbbf5f5baf439af582 [term 3 LEADER]: Received request to transfer leadership to TS fd44949e40b247f980ba14623ba0dd34
I20260709 13:18:24.940812 15256 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6f3b305de2a04b54aef8674096fa6358"
dest_uuid: "12806407b57c4ffbbf5f5baf439af582"
mode: GRACEFUL
new_leader_uuid: "ce1dd5b4e0014ba8a3fa68f2ba226e49"
 from {username='slave'} at 127.0.0.1:60774
I20260709 13:18:24.941232 15256 raft_consensus.cc:606] T 6f3b305de2a04b54aef8674096fa6358 P 12806407b57c4ffbbf5f5baf439af582 [term 1 LEADER]: Received request to transfer leadership to TS ce1dd5b4e0014ba8a3fa68f2ba226e49
I20260709 13:18:24.943310 15256 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "81fce322177440a690ee6f46d2d67ad6"
dest_uuid: "12806407b57c4ffbbf5f5baf439af582"
mode: GRACEFUL
new_leader_uuid: "fd44949e40b247f980ba14623ba0dd34"
 from {username='slave'} at 127.0.0.1:60774
I20260709 13:18:24.943713 15256 raft_consensus.cc:606] T 81fce322177440a690ee6f46d2d67ad6 P 12806407b57c4ffbbf5f5baf439af582 [term 2 LEADER]: Received request to transfer leadership to TS fd44949e40b247f980ba14623ba0dd34
I20260709 13:18:24.945796 15256 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "853d00d5717a4ba1a941a81884cbb7a2"
dest_uuid: "12806407b57c4ffbbf5f5baf439af582"
mode: GRACEFUL
new_leader_uuid: "ce1dd5b4e0014ba8a3fa68f2ba226e49"
 from {username='slave'} at 127.0.0.1:60774
I20260709 13:18:24.946184 15256 raft_consensus.cc:606] T 853d00d5717a4ba1a941a81884cbb7a2 P 12806407b57c4ffbbf5f5baf439af582 [term 1 LEADER]: Received request to transfer leadership to TS ce1dd5b4e0014ba8a3fa68f2ba226e49
I20260709 13:18:24.947232 15046 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:18:24.947724 15046 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:18:24.987812 15411 raft_consensus.cc:993] T 54dc07b181bd495d8c8e2cae81240f16 P 12806407b57c4ffbbf5f5baf439af582: : Instructing follower fd44949e40b247f980ba14623ba0dd34 to start an election
I20260709 13:18:24.989215 15511 raft_consensus.cc:993] T 6f3b305de2a04b54aef8674096fa6358 P 12806407b57c4ffbbf5f5baf439af582: : Instructing follower ce1dd5b4e0014ba8a3fa68f2ba226e49 to start an election
I20260709 13:18:24.989629 15511 raft_consensus.cc:1081] T 6f3b305de2a04b54aef8674096fa6358 P 12806407b57c4ffbbf5f5baf439af582 [term 1 LEADER]: Signalling peer ce1dd5b4e0014ba8a3fa68f2ba226e49 to start an election
I20260709 13:18:24.989423 15510 raft_consensus.cc:1081] T 54dc07b181bd495d8c8e2cae81240f16 P 12806407b57c4ffbbf5f5baf439af582 [term 2 LEADER]: Signalling peer fd44949e40b247f980ba14623ba0dd34 to start an election
I20260709 13:18:24.991532 15329 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "6f3b305de2a04b54aef8674096fa6358"
dest_uuid: "ce1dd5b4e0014ba8a3fa68f2ba226e49"
 from {username='slave'} at 127.0.0.1:49270
I20260709 13:18:24.991933 15180 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "54dc07b181bd495d8c8e2cae81240f16"
dest_uuid: "fd44949e40b247f980ba14623ba0dd34"
 from {username='slave'} at 127.0.0.1:42418
I20260709 13:18:24.992026 15329 raft_consensus.cc:493] T 6f3b305de2a04b54aef8674096fa6358 P ce1dd5b4e0014ba8a3fa68f2ba226e49 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:18:24.992463 15329 raft_consensus.cc:3060] T 6f3b305de2a04b54aef8674096fa6358 P ce1dd5b4e0014ba8a3fa68f2ba226e49 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:24.992496 15180 raft_consensus.cc:493] T 54dc07b181bd495d8c8e2cae81240f16 P fd44949e40b247f980ba14623ba0dd34 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:18:24.992945 15180 raft_consensus.cc:3060] T 54dc07b181bd495d8c8e2cae81240f16 P fd44949e40b247f980ba14623ba0dd34 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:24.997543 15329 raft_consensus.cc:515] T 6f3b305de2a04b54aef8674096fa6358 P ce1dd5b4e0014ba8a3fa68f2ba226e49 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd44949e40b247f980ba14623ba0dd34" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 37801 } } peers { permanent_uuid: "12806407b57c4ffbbf5f5baf439af582" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 45327 } } peers { permanent_uuid: "ce1dd5b4e0014ba8a3fa68f2ba226e49" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34123 } }
I20260709 13:18:24.998020 15180 raft_consensus.cc:515] T 54dc07b181bd495d8c8e2cae81240f16 P fd44949e40b247f980ba14623ba0dd34 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce1dd5b4e0014ba8a3fa68f2ba226e49" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34123 } } peers { permanent_uuid: "fd44949e40b247f980ba14623ba0dd34" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 37801 } } peers { permanent_uuid: "12806407b57c4ffbbf5f5baf439af582" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 45327 } }
I20260709 13:18:24.999534 15329 leader_election.cc:290] T 6f3b305de2a04b54aef8674096fa6358 P ce1dd5b4e0014ba8a3fa68f2ba226e49 [CANDIDATE]: Term 2 election: Requested vote from peers fd44949e40b247f980ba14623ba0dd34 (127.14.177.129:37801), 12806407b57c4ffbbf5f5baf439af582 (127.14.177.130:45327)
I20260709 13:18:25.000906 15180 leader_election.cc:290] T 54dc07b181bd495d8c8e2cae81240f16 P fd44949e40b247f980ba14623ba0dd34 [CANDIDATE]: Term 3 election: Requested vote from peers ce1dd5b4e0014ba8a3fa68f2ba226e49 (127.14.177.131:34123), 12806407b57c4ffbbf5f5baf439af582 (127.14.177.130:45327)
I20260709 13:18:25.000921 15179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f3b305de2a04b54aef8674096fa6358" candidate_uuid: "ce1dd5b4e0014ba8a3fa68f2ba226e49" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "fd44949e40b247f980ba14623ba0dd34"
I20260709 13:18:25.001919 15179 raft_consensus.cc:3060] T 6f3b305de2a04b54aef8674096fa6358 P fd44949e40b247f980ba14623ba0dd34 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:25.001955 15255 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f3b305de2a04b54aef8674096fa6358" candidate_uuid: "ce1dd5b4e0014ba8a3fa68f2ba226e49" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "12806407b57c4ffbbf5f5baf439af582"
I20260709 13:18:25.002530 15255 raft_consensus.cc:3055] T 6f3b305de2a04b54aef8674096fa6358 P 12806407b57c4ffbbf5f5baf439af582 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:18:25.002734 15255 raft_consensus.cc:740] T 6f3b305de2a04b54aef8674096fa6358 P 12806407b57c4ffbbf5f5baf439af582 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 12806407b57c4ffbbf5f5baf439af582, State: Running, Role: LEADER
I20260709 13:18:25.003414 15255 consensus_queue.cc:260] T 6f3b305de2a04b54aef8674096fa6358 P 12806407b57c4ffbbf5f5baf439af582 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd44949e40b247f980ba14623ba0dd34" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 37801 } } peers { permanent_uuid: "12806407b57c4ffbbf5f5baf439af582" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 45327 } } peers { permanent_uuid: "ce1dd5b4e0014ba8a3fa68f2ba226e49" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34123 } }
I20260709 13:18:25.004227 15255 raft_consensus.cc:3060] T 6f3b305de2a04b54aef8674096fa6358 P 12806407b57c4ffbbf5f5baf439af582 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:25.006425 15179 raft_consensus.cc:2468] T 6f3b305de2a04b54aef8674096fa6358 P fd44949e40b247f980ba14623ba0dd34 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce1dd5b4e0014ba8a3fa68f2ba226e49 in term 2.
I20260709 13:18:25.007419 15256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54dc07b181bd495d8c8e2cae81240f16" candidate_uuid: "fd44949e40b247f980ba14623ba0dd34" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "12806407b57c4ffbbf5f5baf439af582"
I20260709 13:18:25.007630 15295 leader_election.cc:304] T 6f3b305de2a04b54aef8674096fa6358 P ce1dd5b4e0014ba8a3fa68f2ba226e49 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce1dd5b4e0014ba8a3fa68f2ba226e49, fd44949e40b247f980ba14623ba0dd34; no voters: 
I20260709 13:18:25.008127 15256 raft_consensus.cc:3055] T 54dc07b181bd495d8c8e2cae81240f16 P 12806407b57c4ffbbf5f5baf439af582 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:18:25.008459 15256 raft_consensus.cc:740] T 54dc07b181bd495d8c8e2cae81240f16 P 12806407b57c4ffbbf5f5baf439af582 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 12806407b57c4ffbbf5f5baf439af582, State: Running, Role: LEADER
I20260709 13:18:25.008157 15329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54dc07b181bd495d8c8e2cae81240f16" candidate_uuid: "fd44949e40b247f980ba14623ba0dd34" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "ce1dd5b4e0014ba8a3fa68f2ba226e49"
I20260709 13:18:25.009052 15329 raft_consensus.cc:3060] T 54dc07b181bd495d8c8e2cae81240f16 P ce1dd5b4e0014ba8a3fa68f2ba226e49 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:25.009191 15256 consensus_queue.cc:260] T 54dc07b181bd495d8c8e2cae81240f16 P 12806407b57c4ffbbf5f5baf439af582 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce1dd5b4e0014ba8a3fa68f2ba226e49" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34123 } } peers { permanent_uuid: "fd44949e40b247f980ba14623ba0dd34" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 37801 } } peers { permanent_uuid: "12806407b57c4ffbbf5f5baf439af582" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 45327 } }
I20260709 13:18:25.009878 15255 raft_consensus.cc:2468] T 6f3b305de2a04b54aef8674096fa6358 P 12806407b57c4ffbbf5f5baf439af582 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce1dd5b4e0014ba8a3fa68f2ba226e49 in term 2.
I20260709 13:18:25.010454 15256 raft_consensus.cc:3060] T 54dc07b181bd495d8c8e2cae81240f16 P 12806407b57c4ffbbf5f5baf439af582 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:25.010615 15463 raft_consensus.cc:2804] T 6f3b305de2a04b54aef8674096fa6358 P ce1dd5b4e0014ba8a3fa68f2ba226e49 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:18:25.011893 15463 raft_consensus.cc:697] T 6f3b305de2a04b54aef8674096fa6358 P ce1dd5b4e0014ba8a3fa68f2ba226e49 [term 2 LEADER]: Becoming Leader. State: Replica: ce1dd5b4e0014ba8a3fa68f2ba226e49, State: Running, Role: LEADER
I20260709 13:18:25.012899 15463 consensus_queue.cc:237] T 6f3b305de2a04b54aef8674096fa6358 P ce1dd5b4e0014ba8a3fa68f2ba226e49 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd44949e40b247f980ba14623ba0dd34" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 37801 } } peers { permanent_uuid: "12806407b57c4ffbbf5f5baf439af582" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 45327 } } peers { permanent_uuid: "ce1dd5b4e0014ba8a3fa68f2ba226e49" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34123 } }
I20260709 13:18:25.015220 15329 raft_consensus.cc:2468] T 54dc07b181bd495d8c8e2cae81240f16 P ce1dd5b4e0014ba8a3fa68f2ba226e49 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fd44949e40b247f980ba14623ba0dd34 in term 3.
I20260709 13:18:25.017028 15146 leader_election.cc:304] T 54dc07b181bd495d8c8e2cae81240f16 P fd44949e40b247f980ba14623ba0dd34 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce1dd5b4e0014ba8a3fa68f2ba226e49, fd44949e40b247f980ba14623ba0dd34; no voters: 
I20260709 13:18:25.018836 15515 raft_consensus.cc:2804] T 54dc07b181bd495d8c8e2cae81240f16 P fd44949e40b247f980ba14623ba0dd34 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:18:25.018945 15256 raft_consensus.cc:2468] T 54dc07b181bd495d8c8e2cae81240f16 P 12806407b57c4ffbbf5f5baf439af582 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fd44949e40b247f980ba14623ba0dd34 in term 3.
I20260709 13:18:25.021373 15515 raft_consensus.cc:697] T 54dc07b181bd495d8c8e2cae81240f16 P fd44949e40b247f980ba14623ba0dd34 [term 3 LEADER]: Becoming Leader. State: Replica: fd44949e40b247f980ba14623ba0dd34, State: Running, Role: LEADER
I20260709 13:18:25.022230 15515 consensus_queue.cc:237] T 54dc07b181bd495d8c8e2cae81240f16 P fd44949e40b247f980ba14623ba0dd34 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce1dd5b4e0014ba8a3fa68f2ba226e49" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34123 } } peers { permanent_uuid: "fd44949e40b247f980ba14623ba0dd34" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 37801 } } peers { permanent_uuid: "12806407b57c4ffbbf5f5baf439af582" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 45327 } }
I20260709 13:18:25.024104 15077 catalog_manager.cc:5697] T 6f3b305de2a04b54aef8674096fa6358 P ce1dd5b4e0014ba8a3fa68f2ba226e49 reported cstate change: term changed from 1 to 2, leader changed from 12806407b57c4ffbbf5f5baf439af582 (127.14.177.130) to ce1dd5b4e0014ba8a3fa68f2ba226e49 (127.14.177.131). New cstate: current_term: 2 leader_uuid: "ce1dd5b4e0014ba8a3fa68f2ba226e49" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd44949e40b247f980ba14623ba0dd34" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 37801 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "12806407b57c4ffbbf5f5baf439af582" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 45327 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ce1dd5b4e0014ba8a3fa68f2ba226e49" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34123 } health_report { overall_health: HEALTHY } } }
I20260709 13:18:25.029994 15076 catalog_manager.cc:5697] T 54dc07b181bd495d8c8e2cae81240f16 P fd44949e40b247f980ba14623ba0dd34 reported cstate change: term changed from 2 to 3, leader changed from 12806407b57c4ffbbf5f5baf439af582 (127.14.177.130) to fd44949e40b247f980ba14623ba0dd34 (127.14.177.129). New cstate: current_term: 3 leader_uuid: "fd44949e40b247f980ba14623ba0dd34" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce1dd5b4e0014ba8a3fa68f2ba226e49" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34123 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fd44949e40b247f980ba14623ba0dd34" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 37801 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "12806407b57c4ffbbf5f5baf439af582" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 45327 } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:25.118043 15512 raft_consensus.cc:993] T 81fce322177440a690ee6f46d2d67ad6 P 12806407b57c4ffbbf5f5baf439af582: : Instructing follower fd44949e40b247f980ba14623ba0dd34 to start an election
I20260709 13:18:25.118480 15510 raft_consensus.cc:1081] T 81fce322177440a690ee6f46d2d67ad6 P 12806407b57c4ffbbf5f5baf439af582 [term 2 LEADER]: Signalling peer fd44949e40b247f980ba14623ba0dd34 to start an election
I20260709 13:18:25.119784 15179 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "81fce322177440a690ee6f46d2d67ad6"
dest_uuid: "fd44949e40b247f980ba14623ba0dd34"
 from {username='slave'} at 127.0.0.1:42418
I20260709 13:18:25.120165 15179 raft_consensus.cc:493] T 81fce322177440a690ee6f46d2d67ad6 P fd44949e40b247f980ba14623ba0dd34 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:18:25.120381 15179 raft_consensus.cc:3060] T 81fce322177440a690ee6f46d2d67ad6 P fd44949e40b247f980ba14623ba0dd34 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:25.124148 15179 raft_consensus.cc:515] T 81fce322177440a690ee6f46d2d67ad6 P fd44949e40b247f980ba14623ba0dd34 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd44949e40b247f980ba14623ba0dd34" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 37801 } } peers { permanent_uuid: "12806407b57c4ffbbf5f5baf439af582" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 45327 } } peers { permanent_uuid: "ce1dd5b4e0014ba8a3fa68f2ba226e49" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34123 } }
I20260709 13:18:25.125453 15179 leader_election.cc:290] T 81fce322177440a690ee6f46d2d67ad6 P fd44949e40b247f980ba14623ba0dd34 [CANDIDATE]: Term 3 election: Requested vote from peers 12806407b57c4ffbbf5f5baf439af582 (127.14.177.130:45327), ce1dd5b4e0014ba8a3fa68f2ba226e49 (127.14.177.131:34123)
I20260709 13:18:25.126241 15256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81fce322177440a690ee6f46d2d67ad6" candidate_uuid: "fd44949e40b247f980ba14623ba0dd34" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "12806407b57c4ffbbf5f5baf439af582"
I20260709 13:18:25.126518 15329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81fce322177440a690ee6f46d2d67ad6" candidate_uuid: "fd44949e40b247f980ba14623ba0dd34" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "ce1dd5b4e0014ba8a3fa68f2ba226e49"
I20260709 13:18:25.126748 15256 raft_consensus.cc:3055] T 81fce322177440a690ee6f46d2d67ad6 P 12806407b57c4ffbbf5f5baf439af582 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:18:25.127003 15329 raft_consensus.cc:3060] T 81fce322177440a690ee6f46d2d67ad6 P ce1dd5b4e0014ba8a3fa68f2ba226e49 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:25.127094 15256 raft_consensus.cc:740] T 81fce322177440a690ee6f46d2d67ad6 P 12806407b57c4ffbbf5f5baf439af582 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 12806407b57c4ffbbf5f5baf439af582, State: Running, Role: LEADER
I20260709 13:18:25.127849 15256 consensus_queue.cc:260] T 81fce322177440a690ee6f46d2d67ad6 P 12806407b57c4ffbbf5f5baf439af582 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd44949e40b247f980ba14623ba0dd34" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 37801 } } peers { permanent_uuid: "12806407b57c4ffbbf5f5baf439af582" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 45327 } } peers { permanent_uuid: "ce1dd5b4e0014ba8a3fa68f2ba226e49" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34123 } }
I20260709 13:18:25.128616 15256 raft_consensus.cc:3060] T 81fce322177440a690ee6f46d2d67ad6 P 12806407b57c4ffbbf5f5baf439af582 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:25.131069 15329 raft_consensus.cc:2468] T 81fce322177440a690ee6f46d2d67ad6 P ce1dd5b4e0014ba8a3fa68f2ba226e49 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fd44949e40b247f980ba14623ba0dd34 in term 3.
I20260709 13:18:25.132156 15146 leader_election.cc:304] T 81fce322177440a690ee6f46d2d67ad6 P fd44949e40b247f980ba14623ba0dd34 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce1dd5b4e0014ba8a3fa68f2ba226e49, fd44949e40b247f980ba14623ba0dd34; no voters: 
I20260709 13:18:25.132639 15256 raft_consensus.cc:2468] T 81fce322177440a690ee6f46d2d67ad6 P 12806407b57c4ffbbf5f5baf439af582 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fd44949e40b247f980ba14623ba0dd34 in term 3.
I20260709 13:18:25.132831 15515 raft_consensus.cc:2804] T 81fce322177440a690ee6f46d2d67ad6 P fd44949e40b247f980ba14623ba0dd34 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:18:25.133347 15515 raft_consensus.cc:697] T 81fce322177440a690ee6f46d2d67ad6 P fd44949e40b247f980ba14623ba0dd34 [term 3 LEADER]: Becoming Leader. State: Replica: fd44949e40b247f980ba14623ba0dd34, State: Running, Role: LEADER
I20260709 13:18:25.134130 15515 consensus_queue.cc:237] T 81fce322177440a690ee6f46d2d67ad6 P fd44949e40b247f980ba14623ba0dd34 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd44949e40b247f980ba14623ba0dd34" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 37801 } } peers { permanent_uuid: "12806407b57c4ffbbf5f5baf439af582" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 45327 } } peers { permanent_uuid: "ce1dd5b4e0014ba8a3fa68f2ba226e49" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34123 } }
I20260709 13:18:25.140713 15076 catalog_manager.cc:5697] T 81fce322177440a690ee6f46d2d67ad6 P fd44949e40b247f980ba14623ba0dd34 reported cstate change: term changed from 2 to 3, leader changed from 12806407b57c4ffbbf5f5baf439af582 (127.14.177.130) to fd44949e40b247f980ba14623ba0dd34 (127.14.177.129). New cstate: current_term: 3 leader_uuid: "fd44949e40b247f980ba14623ba0dd34" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd44949e40b247f980ba14623ba0dd34" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 37801 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "12806407b57c4ffbbf5f5baf439af582" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 45327 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ce1dd5b4e0014ba8a3fa68f2ba226e49" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34123 } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:25.290709 15512 raft_consensus.cc:993] T 853d00d5717a4ba1a941a81884cbb7a2 P 12806407b57c4ffbbf5f5baf439af582: : Instructing follower ce1dd5b4e0014ba8a3fa68f2ba226e49 to start an election
I20260709 13:18:25.291103 15510 raft_consensus.cc:1081] T 853d00d5717a4ba1a941a81884cbb7a2 P 12806407b57c4ffbbf5f5baf439af582 [term 1 LEADER]: Signalling peer ce1dd5b4e0014ba8a3fa68f2ba226e49 to start an election
I20260709 13:18:25.292409 15329 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "853d00d5717a4ba1a941a81884cbb7a2"
dest_uuid: "ce1dd5b4e0014ba8a3fa68f2ba226e49"
 from {username='slave'} at 127.0.0.1:49270
I20260709 13:18:25.292793 15329 raft_consensus.cc:493] T 853d00d5717a4ba1a941a81884cbb7a2 P ce1dd5b4e0014ba8a3fa68f2ba226e49 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:18:25.293017 15329 raft_consensus.cc:3060] T 853d00d5717a4ba1a941a81884cbb7a2 P ce1dd5b4e0014ba8a3fa68f2ba226e49 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:25.296762 15329 raft_consensus.cc:515] T 853d00d5717a4ba1a941a81884cbb7a2 P ce1dd5b4e0014ba8a3fa68f2ba226e49 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd44949e40b247f980ba14623ba0dd34" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 37801 } } peers { permanent_uuid: "12806407b57c4ffbbf5f5baf439af582" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 45327 } } peers { permanent_uuid: "ce1dd5b4e0014ba8a3fa68f2ba226e49" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34123 } }
I20260709 13:18:25.298346 15329 leader_election.cc:290] T 853d00d5717a4ba1a941a81884cbb7a2 P ce1dd5b4e0014ba8a3fa68f2ba226e49 [CANDIDATE]: Term 2 election: Requested vote from peers fd44949e40b247f980ba14623ba0dd34 (127.14.177.129:37801), 12806407b57c4ffbbf5f5baf439af582 (127.14.177.130:45327)
I20260709 13:18:25.299203 15179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "853d00d5717a4ba1a941a81884cbb7a2" candidate_uuid: "ce1dd5b4e0014ba8a3fa68f2ba226e49" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "fd44949e40b247f980ba14623ba0dd34"
I20260709 13:18:25.299384 15256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "853d00d5717a4ba1a941a81884cbb7a2" candidate_uuid: "ce1dd5b4e0014ba8a3fa68f2ba226e49" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "12806407b57c4ffbbf5f5baf439af582"
I20260709 13:18:25.299656 15179 raft_consensus.cc:3060] T 853d00d5717a4ba1a941a81884cbb7a2 P fd44949e40b247f980ba14623ba0dd34 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:25.299867 15256 raft_consensus.cc:3055] T 853d00d5717a4ba1a941a81884cbb7a2 P 12806407b57c4ffbbf5f5baf439af582 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:18:25.300176 15256 raft_consensus.cc:740] T 853d00d5717a4ba1a941a81884cbb7a2 P 12806407b57c4ffbbf5f5baf439af582 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 12806407b57c4ffbbf5f5baf439af582, State: Running, Role: LEADER
I20260709 13:18:25.300828 15256 consensus_queue.cc:260] T 853d00d5717a4ba1a941a81884cbb7a2 P 12806407b57c4ffbbf5f5baf439af582 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd44949e40b247f980ba14623ba0dd34" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 37801 } } peers { permanent_uuid: "12806407b57c4ffbbf5f5baf439af582" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 45327 } } peers { permanent_uuid: "ce1dd5b4e0014ba8a3fa68f2ba226e49" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34123 } }
I20260709 13:18:25.301671 15256 raft_consensus.cc:3060] T 853d00d5717a4ba1a941a81884cbb7a2 P 12806407b57c4ffbbf5f5baf439af582 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:25.303921 15179 raft_consensus.cc:2468] T 853d00d5717a4ba1a941a81884cbb7a2 P fd44949e40b247f980ba14623ba0dd34 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce1dd5b4e0014ba8a3fa68f2ba226e49 in term 2.
I20260709 13:18:25.304775 15295 leader_election.cc:304] T 853d00d5717a4ba1a941a81884cbb7a2 P ce1dd5b4e0014ba8a3fa68f2ba226e49 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce1dd5b4e0014ba8a3fa68f2ba226e49, fd44949e40b247f980ba14623ba0dd34; no voters: 
I20260709 13:18:25.305444 15463 raft_consensus.cc:2804] T 853d00d5717a4ba1a941a81884cbb7a2 P ce1dd5b4e0014ba8a3fa68f2ba226e49 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:18:25.305609 15256 raft_consensus.cc:2468] T 853d00d5717a4ba1a941a81884cbb7a2 P 12806407b57c4ffbbf5f5baf439af582 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce1dd5b4e0014ba8a3fa68f2ba226e49 in term 2.
I20260709 13:18:25.305861 15463 raft_consensus.cc:697] T 853d00d5717a4ba1a941a81884cbb7a2 P ce1dd5b4e0014ba8a3fa68f2ba226e49 [term 2 LEADER]: Becoming Leader. State: Replica: ce1dd5b4e0014ba8a3fa68f2ba226e49, State: Running, Role: LEADER
I20260709 13:18:25.306670 15463 consensus_queue.cc:237] T 853d00d5717a4ba1a941a81884cbb7a2 P ce1dd5b4e0014ba8a3fa68f2ba226e49 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd44949e40b247f980ba14623ba0dd34" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 37801 } } peers { permanent_uuid: "12806407b57c4ffbbf5f5baf439af582" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 45327 } } peers { permanent_uuid: "ce1dd5b4e0014ba8a3fa68f2ba226e49" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34123 } }
I20260709 13:18:25.312624 15076 catalog_manager.cc:5697] T 853d00d5717a4ba1a941a81884cbb7a2 P ce1dd5b4e0014ba8a3fa68f2ba226e49 reported cstate change: term changed from 1 to 2, leader changed from 12806407b57c4ffbbf5f5baf439af582 (127.14.177.130) to ce1dd5b4e0014ba8a3fa68f2ba226e49 (127.14.177.131). New cstate: current_term: 2 leader_uuid: "ce1dd5b4e0014ba8a3fa68f2ba226e49" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd44949e40b247f980ba14623ba0dd34" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 37801 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "12806407b57c4ffbbf5f5baf439af582" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 45327 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ce1dd5b4e0014ba8a3fa68f2ba226e49" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34123 } health_report { overall_health: HEALTHY } } }
I20260709 13:18:25.358821 15512 raft_consensus.cc:993] T 58a8c0164db24ffd8e992493c228804c P 12806407b57c4ffbbf5f5baf439af582: : Instructing follower fd44949e40b247f980ba14623ba0dd34 to start an election
I20260709 13:18:25.359376 15510 raft_consensus.cc:1081] T 58a8c0164db24ffd8e992493c228804c P 12806407b57c4ffbbf5f5baf439af582 [term 3 LEADER]: Signalling peer fd44949e40b247f980ba14623ba0dd34 to start an election
I20260709 13:18:25.360900 15179 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "58a8c0164db24ffd8e992493c228804c"
dest_uuid: "fd44949e40b247f980ba14623ba0dd34"
 from {username='slave'} at 127.0.0.1:42418
I20260709 13:18:25.361370 15179 raft_consensus.cc:493] T 58a8c0164db24ffd8e992493c228804c P fd44949e40b247f980ba14623ba0dd34 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:18:25.361720 15179 raft_consensus.cc:3060] T 58a8c0164db24ffd8e992493c228804c P fd44949e40b247f980ba14623ba0dd34 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:25.366151 15179 raft_consensus.cc:515] T 58a8c0164db24ffd8e992493c228804c P fd44949e40b247f980ba14623ba0dd34 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd44949e40b247f980ba14623ba0dd34" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 37801 } } peers { permanent_uuid: "12806407b57c4ffbbf5f5baf439af582" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 45327 } } peers { permanent_uuid: "ce1dd5b4e0014ba8a3fa68f2ba226e49" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34123 } }
I20260709 13:18:25.367561 15179 leader_election.cc:290] T 58a8c0164db24ffd8e992493c228804c P fd44949e40b247f980ba14623ba0dd34 [CANDIDATE]: Term 4 election: Requested vote from peers 12806407b57c4ffbbf5f5baf439af582 (127.14.177.130:45327), ce1dd5b4e0014ba8a3fa68f2ba226e49 (127.14.177.131:34123)
I20260709 13:18:25.368306 15256 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58a8c0164db24ffd8e992493c228804c" candidate_uuid: "fd44949e40b247f980ba14623ba0dd34" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "12806407b57c4ffbbf5f5baf439af582"
I20260709 13:18:25.368643 15329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58a8c0164db24ffd8e992493c228804c" candidate_uuid: "fd44949e40b247f980ba14623ba0dd34" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "ce1dd5b4e0014ba8a3fa68f2ba226e49"
I20260709 13:18:25.368873 15256 raft_consensus.cc:3055] T 58a8c0164db24ffd8e992493c228804c P 12806407b57c4ffbbf5f5baf439af582 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:18:25.369151 15329 raft_consensus.cc:3060] T 58a8c0164db24ffd8e992493c228804c P ce1dd5b4e0014ba8a3fa68f2ba226e49 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:25.369163 15256 raft_consensus.cc:740] T 58a8c0164db24ffd8e992493c228804c P 12806407b57c4ffbbf5f5baf439af582 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 12806407b57c4ffbbf5f5baf439af582, State: Running, Role: LEADER
I20260709 13:18:25.370026 15256 consensus_queue.cc:260] T 58a8c0164db24ffd8e992493c228804c P 12806407b57c4ffbbf5f5baf439af582 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd44949e40b247f980ba14623ba0dd34" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 37801 } } peers { permanent_uuid: "12806407b57c4ffbbf5f5baf439af582" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 45327 } } peers { permanent_uuid: "ce1dd5b4e0014ba8a3fa68f2ba226e49" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34123 } }
I20260709 13:18:25.371074 15256 raft_consensus.cc:3060] T 58a8c0164db24ffd8e992493c228804c P 12806407b57c4ffbbf5f5baf439af582 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:25.375078 15329 raft_consensus.cc:2468] T 58a8c0164db24ffd8e992493c228804c P ce1dd5b4e0014ba8a3fa68f2ba226e49 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fd44949e40b247f980ba14623ba0dd34 in term 4.
I20260709 13:18:25.376119 15146 leader_election.cc:304] T 58a8c0164db24ffd8e992493c228804c P fd44949e40b247f980ba14623ba0dd34 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce1dd5b4e0014ba8a3fa68f2ba226e49, fd44949e40b247f980ba14623ba0dd34; no voters: 
I20260709 13:18:25.376816 15256 raft_consensus.cc:2468] T 58a8c0164db24ffd8e992493c228804c P 12806407b57c4ffbbf5f5baf439af582 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fd44949e40b247f980ba14623ba0dd34 in term 4.
I20260709 13:18:25.376850 15515 raft_consensus.cc:2804] T 58a8c0164db24ffd8e992493c228804c P fd44949e40b247f980ba14623ba0dd34 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:18:25.377442 15515 raft_consensus.cc:697] T 58a8c0164db24ffd8e992493c228804c P fd44949e40b247f980ba14623ba0dd34 [term 4 LEADER]: Becoming Leader. State: Replica: fd44949e40b247f980ba14623ba0dd34, State: Running, Role: LEADER
I20260709 13:18:25.378237 15515 consensus_queue.cc:237] T 58a8c0164db24ffd8e992493c228804c P fd44949e40b247f980ba14623ba0dd34 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd44949e40b247f980ba14623ba0dd34" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 37801 } } peers { permanent_uuid: "12806407b57c4ffbbf5f5baf439af582" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 45327 } } peers { permanent_uuid: "ce1dd5b4e0014ba8a3fa68f2ba226e49" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34123 } }
I20260709 13:18:25.384444 15076 catalog_manager.cc:5697] T 58a8c0164db24ffd8e992493c228804c P fd44949e40b247f980ba14623ba0dd34 reported cstate change: term changed from 3 to 4, leader changed from 12806407b57c4ffbbf5f5baf439af582 (127.14.177.130) to fd44949e40b247f980ba14623ba0dd34 (127.14.177.129). New cstate: current_term: 4 leader_uuid: "fd44949e40b247f980ba14623ba0dd34" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd44949e40b247f980ba14623ba0dd34" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 37801 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "12806407b57c4ffbbf5f5baf439af582" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 45327 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ce1dd5b4e0014ba8a3fa68f2ba226e49" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34123 } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:25.413477 15256 raft_consensus.cc:1275] T 54dc07b181bd495d8c8e2cae81240f16 P 12806407b57c4ffbbf5f5baf439af582 [term 3 FOLLOWER]: Refusing update from remote peer fd44949e40b247f980ba14623ba0dd34: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:18:25.414727 15515 consensus_queue.cc:1048] T 54dc07b181bd495d8c8e2cae81240f16 P fd44949e40b247f980ba14623ba0dd34 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12806407b57c4ffbbf5f5baf439af582" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 45327 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:18:25.425398 15329 raft_consensus.cc:1275] T 54dc07b181bd495d8c8e2cae81240f16 P ce1dd5b4e0014ba8a3fa68f2ba226e49 [term 3 FOLLOWER]: Refusing update from remote peer fd44949e40b247f980ba14623ba0dd34: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:18:25.426951 15529 consensus_queue.cc:1048] T 54dc07b181bd495d8c8e2cae81240f16 P fd44949e40b247f980ba14623ba0dd34 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce1dd5b4e0014ba8a3fa68f2ba226e49" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:18:25.524518 15329 raft_consensus.cc:1275] T 81fce322177440a690ee6f46d2d67ad6 P ce1dd5b4e0014ba8a3fa68f2ba226e49 [term 3 FOLLOWER]: Refusing update from remote peer fd44949e40b247f980ba14623ba0dd34: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:18:25.525686 15529 consensus_queue.cc:1048] T 81fce322177440a690ee6f46d2d67ad6 P fd44949e40b247f980ba14623ba0dd34 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce1dd5b4e0014ba8a3fa68f2ba226e49" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:18:25.535136 15256 raft_consensus.cc:1275] T 81fce322177440a690ee6f46d2d67ad6 P 12806407b57c4ffbbf5f5baf439af582 [term 3 FOLLOWER]: Refusing update from remote peer fd44949e40b247f980ba14623ba0dd34: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:18:25.536729 15529 consensus_queue.cc:1048] T 81fce322177440a690ee6f46d2d67ad6 P fd44949e40b247f980ba14623ba0dd34 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12806407b57c4ffbbf5f5baf439af582" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 45327 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:18:25.558919 15256 raft_consensus.cc:1275] T 6f3b305de2a04b54aef8674096fa6358 P 12806407b57c4ffbbf5f5baf439af582 [term 2 FOLLOWER]: Refusing update from remote peer ce1dd5b4e0014ba8a3fa68f2ba226e49: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:18:25.560096 15463 consensus_queue.cc:1048] T 6f3b305de2a04b54aef8674096fa6358 P ce1dd5b4e0014ba8a3fa68f2ba226e49 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12806407b57c4ffbbf5f5baf439af582" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 45327 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:18:25.568657 15179 raft_consensus.cc:1275] T 6f3b305de2a04b54aef8674096fa6358 P fd44949e40b247f980ba14623ba0dd34 [term 2 FOLLOWER]: Refusing update from remote peer ce1dd5b4e0014ba8a3fa68f2ba226e49: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:18:25.570060 15535 consensus_queue.cc:1048] T 6f3b305de2a04b54aef8674096fa6358 P ce1dd5b4e0014ba8a3fa68f2ba226e49 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fd44949e40b247f980ba14623ba0dd34" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:18:25.718987 15256 raft_consensus.cc:1275] T 853d00d5717a4ba1a941a81884cbb7a2 P 12806407b57c4ffbbf5f5baf439af582 [term 2 FOLLOWER]: Refusing update from remote peer ce1dd5b4e0014ba8a3fa68f2ba226e49: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:18:25.720288 15535 consensus_queue.cc:1048] T 853d00d5717a4ba1a941a81884cbb7a2 P ce1dd5b4e0014ba8a3fa68f2ba226e49 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12806407b57c4ffbbf5f5baf439af582" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 45327 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:18:25.728885 15179 raft_consensus.cc:1275] T 853d00d5717a4ba1a941a81884cbb7a2 P fd44949e40b247f980ba14623ba0dd34 [term 2 FOLLOWER]: Refusing update from remote peer ce1dd5b4e0014ba8a3fa68f2ba226e49: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:18:25.730422 15513 consensus_queue.cc:1048] T 853d00d5717a4ba1a941a81884cbb7a2 P ce1dd5b4e0014ba8a3fa68f2ba226e49 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fd44949e40b247f980ba14623ba0dd34" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 37801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:18:25.905640 15256 raft_consensus.cc:1275] T 58a8c0164db24ffd8e992493c228804c P 12806407b57c4ffbbf5f5baf439af582 [term 4 FOLLOWER]: Refusing update from remote peer fd44949e40b247f980ba14623ba0dd34: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:18:25.907011 15529 consensus_queue.cc:1048] T 58a8c0164db24ffd8e992493c228804c P fd44949e40b247f980ba14623ba0dd34 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12806407b57c4ffbbf5f5baf439af582" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 45327 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:18:25.915649 15329 raft_consensus.cc:1275] T 58a8c0164db24ffd8e992493c228804c P ce1dd5b4e0014ba8a3fa68f2ba226e49 [term 4 FOLLOWER]: Refusing update from remote peer fd44949e40b247f980ba14623ba0dd34: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:18:25.917127 15515 consensus_queue.cc:1048] T 58a8c0164db24ffd8e992493c228804c P fd44949e40b247f980ba14623ba0dd34 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce1dd5b4e0014ba8a3fa68f2ba226e49" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:18:25.948670 15046 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:18:25.955297 15046 tablet_server.cc:179] TabletServer@127.14.177.129:0 shutting down...
I20260709 13:18:25.985548 15046 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:18:25.986212 15046 tablet_replica.cc:333] T fbf069cd9abc4875a3d03f06ff06544e P fd44949e40b247f980ba14623ba0dd34: stopping tablet replica
I20260709 13:18:25.986933 15046 raft_consensus.cc:2243] T fbf069cd9abc4875a3d03f06ff06544e P fd44949e40b247f980ba14623ba0dd34 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:25.987622 15046 raft_consensus.cc:2272] T fbf069cd9abc4875a3d03f06ff06544e P fd44949e40b247f980ba14623ba0dd34 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:25.990128 15046 tablet_replica.cc:333] T 6f3b305de2a04b54aef8674096fa6358 P fd44949e40b247f980ba14623ba0dd34: stopping tablet replica
I20260709 13:18:25.990751 15046 raft_consensus.cc:2243] T 6f3b305de2a04b54aef8674096fa6358 P fd44949e40b247f980ba14623ba0dd34 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:25.991246 15046 raft_consensus.cc:2272] T 6f3b305de2a04b54aef8674096fa6358 P fd44949e40b247f980ba14623ba0dd34 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:25.994210 15046 tablet_replica.cc:333] T 853d00d5717a4ba1a941a81884cbb7a2 P fd44949e40b247f980ba14623ba0dd34: stopping tablet replica
I20260709 13:18:25.994998 15046 raft_consensus.cc:2243] T 853d00d5717a4ba1a941a81884cbb7a2 P fd44949e40b247f980ba14623ba0dd34 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:25.995579 15046 raft_consensus.cc:2272] T 853d00d5717a4ba1a941a81884cbb7a2 P fd44949e40b247f980ba14623ba0dd34 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:25.997891 15046 tablet_replica.cc:333] T 81fce322177440a690ee6f46d2d67ad6 P fd44949e40b247f980ba14623ba0dd34: stopping tablet replica
I20260709 13:18:25.998420 15046 raft_consensus.cc:2243] T 81fce322177440a690ee6f46d2d67ad6 P fd44949e40b247f980ba14623ba0dd34 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:18:25.999295 15046 raft_consensus.cc:2272] T 81fce322177440a690ee6f46d2d67ad6 P fd44949e40b247f980ba14623ba0dd34 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:26.001194 15046 tablet_replica.cc:333] T 54dc07b181bd495d8c8e2cae81240f16 P fd44949e40b247f980ba14623ba0dd34: stopping tablet replica
I20260709 13:18:26.001672 15046 raft_consensus.cc:2243] T 54dc07b181bd495d8c8e2cae81240f16 P fd44949e40b247f980ba14623ba0dd34 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:18:26.002465 15046 raft_consensus.cc:2272] T 54dc07b181bd495d8c8e2cae81240f16 P fd44949e40b247f980ba14623ba0dd34 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:26.004558 15046 tablet_replica.cc:333] T 58a8c0164db24ffd8e992493c228804c P fd44949e40b247f980ba14623ba0dd34: stopping tablet replica
I20260709 13:18:26.005038 15046 raft_consensus.cc:2243] T 58a8c0164db24ffd8e992493c228804c P fd44949e40b247f980ba14623ba0dd34 [term 4 LEADER]: Raft consensus shutting down.
I20260709 13:18:26.005748 15046 raft_consensus.cc:2272] T 58a8c0164db24ffd8e992493c228804c P fd44949e40b247f980ba14623ba0dd34 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:26.007702 15046 tablet_replica.cc:333] T 8391f4ec0b1349f4a1758a0ce2382b7c P fd44949e40b247f980ba14623ba0dd34: stopping tablet replica
I20260709 13:18:26.008242 15046 raft_consensus.cc:2243] T 8391f4ec0b1349f4a1758a0ce2382b7c P fd44949e40b247f980ba14623ba0dd34 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:26.008637 15046 raft_consensus.cc:2272] T 8391f4ec0b1349f4a1758a0ce2382b7c P fd44949e40b247f980ba14623ba0dd34 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:26.010550 15046 tablet_replica.cc:333] T 98f6aef87e78472c9987c75cac6575af P fd44949e40b247f980ba14623ba0dd34: stopping tablet replica
I20260709 13:18:26.011309 15046 raft_consensus.cc:2243] T 98f6aef87e78472c9987c75cac6575af P fd44949e40b247f980ba14623ba0dd34 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:26.011881 15046 raft_consensus.cc:2272] T 98f6aef87e78472c9987c75cac6575af P fd44949e40b247f980ba14623ba0dd34 [term 3 FOLLOWER]: Raft consensus is shut down!
W20260709 13:18:26.012442 15295 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:18:26.014990 15046 tablet_replica.cc:333] T 28ac227079274b8eb6a3e35e4bf34e30 P fd44949e40b247f980ba14623ba0dd34: stopping tablet replica
I20260709 13:18:26.015745 15046 raft_consensus.cc:2243] T 28ac227079274b8eb6a3e35e4bf34e30 P fd44949e40b247f980ba14623ba0dd34 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:26.016287 15046 raft_consensus.cc:2272] T 28ac227079274b8eb6a3e35e4bf34e30 P fd44949e40b247f980ba14623ba0dd34 [term 3 FOLLOWER]: Raft consensus is shut down!
W20260709 13:18:26.016706 15295 consensus_peers.cc:597] T 6f3b305de2a04b54aef8674096fa6358 P ce1dd5b4e0014ba8a3fa68f2ba226e49 -> Peer fd44949e40b247f980ba14623ba0dd34 (127.14.177.129:37801): Couldn't send request to peer fd44949e40b247f980ba14623ba0dd34. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:18:26.112159 15046 tablet_server.cc:196] TabletServer@127.14.177.129:0 shutdown complete.
I20260709 13:18:26.129302 15046 tablet_server.cc:179] TabletServer@127.14.177.130:0 shutting down...
I20260709 13:18:26.149838 15046 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:18:26.150477 15046 tablet_replica.cc:333] T 58a8c0164db24ffd8e992493c228804c P 12806407b57c4ffbbf5f5baf439af582: stopping tablet replica
I20260709 13:18:26.151031 15046 raft_consensus.cc:2243] T 58a8c0164db24ffd8e992493c228804c P 12806407b57c4ffbbf5f5baf439af582 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:26.151515 15046 raft_consensus.cc:2272] T 58a8c0164db24ffd8e992493c228804c P 12806407b57c4ffbbf5f5baf439af582 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:26.153594 15046 tablet_replica.cc:333] T fbf069cd9abc4875a3d03f06ff06544e P 12806407b57c4ffbbf5f5baf439af582: stopping tablet replica
I20260709 13:18:26.154102 15046 raft_consensus.cc:2243] T fbf069cd9abc4875a3d03f06ff06544e P 12806407b57c4ffbbf5f5baf439af582 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:26.154598 15046 raft_consensus.cc:2272] T fbf069cd9abc4875a3d03f06ff06544e P 12806407b57c4ffbbf5f5baf439af582 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:26.156713 15046 tablet_replica.cc:333] T 8391f4ec0b1349f4a1758a0ce2382b7c P 12806407b57c4ffbbf5f5baf439af582: stopping tablet replica
I20260709 13:18:26.157250 15046 raft_consensus.cc:2243] T 8391f4ec0b1349f4a1758a0ce2382b7c P 12806407b57c4ffbbf5f5baf439af582 [term 4 LEADER]: Raft consensus shutting down.
I20260709 13:18:26.158205 15046 raft_consensus.cc:2272] T 8391f4ec0b1349f4a1758a0ce2382b7c P 12806407b57c4ffbbf5f5baf439af582 [term 4 FOLLOWER]: Raft consensus is shut down!
W20260709 13:18:26.158332 15294 consensus_peers.cc:597] T 853d00d5717a4ba1a941a81884cbb7a2 P ce1dd5b4e0014ba8a3fa68f2ba226e49 -> Peer 12806407b57c4ffbbf5f5baf439af582 (127.14.177.130:45327): Couldn't send request to peer 12806407b57c4ffbbf5f5baf439af582. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:18:26.159925 15046 tablet_replica.cc:333] T 28ac227079274b8eb6a3e35e4bf34e30 P 12806407b57c4ffbbf5f5baf439af582: stopping tablet replica
I20260709 13:18:26.160446 15046 raft_consensus.cc:2243] T 28ac227079274b8eb6a3e35e4bf34e30 P 12806407b57c4ffbbf5f5baf439af582 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:18:26.161121 15046 raft_consensus.cc:2272] T 28ac227079274b8eb6a3e35e4bf34e30 P 12806407b57c4ffbbf5f5baf439af582 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:26.162693 15046 tablet_replica.cc:333] T 98f6aef87e78472c9987c75cac6575af P 12806407b57c4ffbbf5f5baf439af582: stopping tablet replica
I20260709 13:18:26.163203 15046 raft_consensus.cc:2243] T 98f6aef87e78472c9987c75cac6575af P 12806407b57c4ffbbf5f5baf439af582 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:18:26.163905 15046 raft_consensus.cc:2272] T 98f6aef87e78472c9987c75cac6575af P 12806407b57c4ffbbf5f5baf439af582 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:26.165366 15046 tablet_replica.cc:333] T 6f3b305de2a04b54aef8674096fa6358 P 12806407b57c4ffbbf5f5baf439af582: stopping tablet replica
I20260709 13:18:26.165882 15046 raft_consensus.cc:2243] T 6f3b305de2a04b54aef8674096fa6358 P 12806407b57c4ffbbf5f5baf439af582 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:26.166255 15046 raft_consensus.cc:2272] T 6f3b305de2a04b54aef8674096fa6358 P 12806407b57c4ffbbf5f5baf439af582 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:26.168264 15046 tablet_replica.cc:333] T 81fce322177440a690ee6f46d2d67ad6 P 12806407b57c4ffbbf5f5baf439af582: stopping tablet replica
I20260709 13:18:26.168722 15046 raft_consensus.cc:2243] T 81fce322177440a690ee6f46d2d67ad6 P 12806407b57c4ffbbf5f5baf439af582 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:26.169104 15046 raft_consensus.cc:2272] T 81fce322177440a690ee6f46d2d67ad6 P 12806407b57c4ffbbf5f5baf439af582 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:26.170980 15046 tablet_replica.cc:333] T 853d00d5717a4ba1a941a81884cbb7a2 P 12806407b57c4ffbbf5f5baf439af582: stopping tablet replica
I20260709 13:18:26.171458 15046 raft_consensus.cc:2243] T 853d00d5717a4ba1a941a81884cbb7a2 P 12806407b57c4ffbbf5f5baf439af582 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:26.171885 15046 raft_consensus.cc:2272] T 853d00d5717a4ba1a941a81884cbb7a2 P 12806407b57c4ffbbf5f5baf439af582 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:26.173843 15046 tablet_replica.cc:333] T 54dc07b181bd495d8c8e2cae81240f16 P 12806407b57c4ffbbf5f5baf439af582: stopping tablet replica
I20260709 13:18:26.174260 15046 raft_consensus.cc:2243] T 54dc07b181bd495d8c8e2cae81240f16 P 12806407b57c4ffbbf5f5baf439af582 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:26.174628 15046 raft_consensus.cc:2272] T 54dc07b181bd495d8c8e2cae81240f16 P 12806407b57c4ffbbf5f5baf439af582 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:26.201774 15046 tablet_server.cc:196] TabletServer@127.14.177.130:0 shutdown complete.
I20260709 13:18:26.217808 15046 tablet_server.cc:179] TabletServer@127.14.177.131:0 shutting down...
I20260709 13:18:26.240427 15046 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:18:26.241066 15046 tablet_replica.cc:333] T 853d00d5717a4ba1a941a81884cbb7a2 P ce1dd5b4e0014ba8a3fa68f2ba226e49: stopping tablet replica
I20260709 13:18:26.241642 15046 raft_consensus.cc:2243] T 853d00d5717a4ba1a941a81884cbb7a2 P ce1dd5b4e0014ba8a3fa68f2ba226e49 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:18:26.242427 15046 raft_consensus.cc:2272] T 853d00d5717a4ba1a941a81884cbb7a2 P ce1dd5b4e0014ba8a3fa68f2ba226e49 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:26.244810 15046 tablet_replica.cc:333] T 81fce322177440a690ee6f46d2d67ad6 P ce1dd5b4e0014ba8a3fa68f2ba226e49: stopping tablet replica
I20260709 13:18:26.245394 15046 raft_consensus.cc:2243] T 81fce322177440a690ee6f46d2d67ad6 P ce1dd5b4e0014ba8a3fa68f2ba226e49 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:26.245847 15046 raft_consensus.cc:2272] T 81fce322177440a690ee6f46d2d67ad6 P ce1dd5b4e0014ba8a3fa68f2ba226e49 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:26.248270 15046 tablet_replica.cc:333] T 58a8c0164db24ffd8e992493c228804c P ce1dd5b4e0014ba8a3fa68f2ba226e49: stopping tablet replica
I20260709 13:18:26.248780 15046 raft_consensus.cc:2243] T 58a8c0164db24ffd8e992493c228804c P ce1dd5b4e0014ba8a3fa68f2ba226e49 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:26.249203 15046 raft_consensus.cc:2272] T 58a8c0164db24ffd8e992493c228804c P ce1dd5b4e0014ba8a3fa68f2ba226e49 [term 4 FOLLOWER]: Raft consensus is shut down!
W20260709 13:18:26.249715 15295 consensus_peers.cc:597] T fbf069cd9abc4875a3d03f06ff06544e P ce1dd5b4e0014ba8a3fa68f2ba226e49 -> Peer fd44949e40b247f980ba14623ba0dd34 (127.14.177.129:37801): Couldn't send request to peer fd44949e40b247f980ba14623ba0dd34. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:37801: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:18:26.251389 15046 tablet_replica.cc:333] T 8391f4ec0b1349f4a1758a0ce2382b7c P ce1dd5b4e0014ba8a3fa68f2ba226e49: stopping tablet replica
I20260709 13:18:26.251921 15046 raft_consensus.cc:2243] T 8391f4ec0b1349f4a1758a0ce2382b7c P ce1dd5b4e0014ba8a3fa68f2ba226e49 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:26.252321 15046 raft_consensus.cc:2272] T 8391f4ec0b1349f4a1758a0ce2382b7c P ce1dd5b4e0014ba8a3fa68f2ba226e49 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:26.253901 15046 tablet_replica.cc:333] T 54dc07b181bd495d8c8e2cae81240f16 P ce1dd5b4e0014ba8a3fa68f2ba226e49: stopping tablet replica
I20260709 13:18:26.254374 15046 raft_consensus.cc:2243] T 54dc07b181bd495d8c8e2cae81240f16 P ce1dd5b4e0014ba8a3fa68f2ba226e49 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:26.254747 15046 raft_consensus.cc:2272] T 54dc07b181bd495d8c8e2cae81240f16 P ce1dd5b4e0014ba8a3fa68f2ba226e49 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:26.256969 15046 tablet_replica.cc:333] T fbf069cd9abc4875a3d03f06ff06544e P ce1dd5b4e0014ba8a3fa68f2ba226e49: stopping tablet replica
I20260709 13:18:26.257440 15046 raft_consensus.cc:2243] T fbf069cd9abc4875a3d03f06ff06544e P ce1dd5b4e0014ba8a3fa68f2ba226e49 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:18:26.258145 15046 raft_consensus.cc:2272] T fbf069cd9abc4875a3d03f06ff06544e P ce1dd5b4e0014ba8a3fa68f2ba226e49 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:26.259721 15046 tablet_replica.cc:333] T 98f6aef87e78472c9987c75cac6575af P ce1dd5b4e0014ba8a3fa68f2ba226e49: stopping tablet replica
I20260709 13:18:26.260202 15046 raft_consensus.cc:2243] T 98f6aef87e78472c9987c75cac6575af P ce1dd5b4e0014ba8a3fa68f2ba226e49 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:26.260576 15046 raft_consensus.cc:2272] T 98f6aef87e78472c9987c75cac6575af P ce1dd5b4e0014ba8a3fa68f2ba226e49 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:26.262102 15046 tablet_replica.cc:333] T 28ac227079274b8eb6a3e35e4bf34e30 P ce1dd5b4e0014ba8a3fa68f2ba226e49: stopping tablet replica
I20260709 13:18:26.262534 15046 raft_consensus.cc:2243] T 28ac227079274b8eb6a3e35e4bf34e30 P ce1dd5b4e0014ba8a3fa68f2ba226e49 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:26.262935 15046 raft_consensus.cc:2272] T 28ac227079274b8eb6a3e35e4bf34e30 P ce1dd5b4e0014ba8a3fa68f2ba226e49 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:26.264408 15046 tablet_replica.cc:333] T 6f3b305de2a04b54aef8674096fa6358 P ce1dd5b4e0014ba8a3fa68f2ba226e49: stopping tablet replica
I20260709 13:18:26.264851 15046 raft_consensus.cc:2243] T 6f3b305de2a04b54aef8674096fa6358 P ce1dd5b4e0014ba8a3fa68f2ba226e49 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:18:26.265528 15046 raft_consensus.cc:2272] T 6f3b305de2a04b54aef8674096fa6358 P ce1dd5b4e0014ba8a3fa68f2ba226e49 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:26.291700 15046 tablet_server.cc:196] TabletServer@127.14.177.131:0 shutdown complete.
I20260709 13:18:26.307463 15046 master.cc:562] Master@127.14.177.190:43613 shutting down...
I20260709 13:18:26.391592 15133 catalog_manager.cc:805] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Closing
I20260709 13:18:26.504599 15046 raft_consensus.cc:2243] T 00000000000000000000000000000000 P a1334aae5957436c903bbb9ed03c2525 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:18:26.505172 15046 raft_consensus.cc:2272] T 00000000000000000000000000000000 P a1334aae5957436c903bbb9ed03c2525 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:26.505535 15046 tablet_replica.cc:333] T 00000000000000000000000000000000 P a1334aae5957436c903bbb9ed03c2525: stopping tablet replica
I20260709 13:18:26.525290 15046 master.cc:584] Master@127.14.177.190:43613 shutdown complete.
12806407b57c4ffbbf5f5baf439af582  4
ce1dd5b4e0014ba8a3fa68f2ba226e49  1
fd44949e40b247f980ba14623ba0dd34  4
[       OK ] LeaderRebalancerTest.FunctionalTestForDivided (12595 ms)
[ RUN      ] LeaderRebalancerTest.FunctionalTestForNotDivided
I20260709 13:18:26.565322 15046 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.14.177.190:38145
I20260709 13:18:26.566403 15046 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:18:26.571723 15544 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:18:26.572057 15545 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:18:26.573010 15547 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:18:26.573539 15046 server_base.cc:1061] running on GCE node
I20260709 13:18:26.574395 15046 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:18:26.574615 15046 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:18:26.574754 15046 hybrid_clock.cc:648] HybridClock initialized: now 1783603106574740 us; error 0 us; skew 500 ppm
I20260709 13:18:26.575361 15046 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:18:26.577678 15046 webserver.cc:533] Webserver started at http://127.14.177.190:40375/ using document root <none> and password file <none>
I20260709 13:18:26.578155 15046 fs_manager.cc:362] Metadata directory not provided
I20260709 13:18:26.578358 15046 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:18:26.578629 15046 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:18:26.579821 15046 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskdH_Ma7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783603093801613-15046-0/minicluster-data/master-0-root/instance:
uuid: "63272e34c419449bbb90a9013aa32b21"
format_stamp: "Formatted at 2026-07-09 13:18:26 on dist-test-slave-v5z2"
I20260709 13:18:26.584596 15046 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.005s	sys 0.000s
I20260709 13:18:26.588230 15552 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:18:26.588990 15046 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.004s	sys 0.000s
I20260709 13:18:26.589296 15046 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskdH_Ma7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783603093801613-15046-0/minicluster-data/master-0-root
uuid: "63272e34c419449bbb90a9013aa32b21"
format_stamp: "Formatted at 2026-07-09 13:18:26 on dist-test-slave-v5z2"
I20260709 13:18:26.589593 15046 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdH_Ma7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783603093801613-15046-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskdH_Ma7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783603093801613-15046-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskdH_Ma7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783603093801613-15046-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:18:26.608863 15046 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:18:26.610114 15046 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:18:26.647478 15046 rpc_server.cc:307] RPC server started. Bound to: 127.14.177.190:38145
I20260709 13:18:26.647584 15603 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.14.177.190:38145 every 8 connection(s)
I20260709 13:18:26.653677 15604 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:26.663707 15604 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 63272e34c419449bbb90a9013aa32b21: Bootstrap starting.
I20260709 13:18:26.668267 15604 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 63272e34c419449bbb90a9013aa32b21: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:26.672381 15604 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 63272e34c419449bbb90a9013aa32b21: No bootstrap required, opened a new log
I20260709 13:18:26.674477 15604 raft_consensus.cc:359] T 00000000000000000000000000000000 P 63272e34c419449bbb90a9013aa32b21 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "63272e34c419449bbb90a9013aa32b21" member_type: VOTER }
I20260709 13:18:26.674892 15604 raft_consensus.cc:385] T 00000000000000000000000000000000 P 63272e34c419449bbb90a9013aa32b21 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:26.675123 15604 raft_consensus.cc:740] T 00000000000000000000000000000000 P 63272e34c419449bbb90a9013aa32b21 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 63272e34c419449bbb90a9013aa32b21, State: Initialized, Role: FOLLOWER
I20260709 13:18:26.675730 15604 consensus_queue.cc:260] T 00000000000000000000000000000000 P 63272e34c419449bbb90a9013aa32b21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "63272e34c419449bbb90a9013aa32b21" member_type: VOTER }
I20260709 13:18:26.676187 15604 raft_consensus.cc:399] T 00000000000000000000000000000000 P 63272e34c419449bbb90a9013aa32b21 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 13:18:26.676429 15604 raft_consensus.cc:493] T 00000000000000000000000000000000 P 63272e34c419449bbb90a9013aa32b21 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 13:18:26.676688 15604 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 63272e34c419449bbb90a9013aa32b21 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:26.680646 15604 raft_consensus.cc:515] T 00000000000000000000000000000000 P 63272e34c419449bbb90a9013aa32b21 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "63272e34c419449bbb90a9013aa32b21" member_type: VOTER }
I20260709 13:18:26.681142 15604 leader_election.cc:304] T 00000000000000000000000000000000 P 63272e34c419449bbb90a9013aa32b21 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 63272e34c419449bbb90a9013aa32b21; no voters: 
I20260709 13:18:26.682210 15604 leader_election.cc:290] T 00000000000000000000000000000000 P 63272e34c419449bbb90a9013aa32b21 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 13:18:26.682610 15607 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 63272e34c419449bbb90a9013aa32b21 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:18:26.683907 15607 raft_consensus.cc:697] T 00000000000000000000000000000000 P 63272e34c419449bbb90a9013aa32b21 [term 1 LEADER]: Becoming Leader. State: Replica: 63272e34c419449bbb90a9013aa32b21, State: Running, Role: LEADER
I20260709 13:18:26.684578 15607 consensus_queue.cc:237] T 00000000000000000000000000000000 P 63272e34c419449bbb90a9013aa32b21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "63272e34c419449bbb90a9013aa32b21" member_type: VOTER }
I20260709 13:18:26.685341 15604 sys_catalog.cc:565] T 00000000000000000000000000000000 P 63272e34c419449bbb90a9013aa32b21 [sys.catalog]: configured and running, proceeding with master startup.
I20260709 13:18:26.689762 15608 sys_catalog.cc:455] T 00000000000000000000000000000000 P 63272e34c419449bbb90a9013aa32b21 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "63272e34c419449bbb90a9013aa32b21" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "63272e34c419449bbb90a9013aa32b21" member_type: VOTER } }
I20260709 13:18:26.690505 15608 sys_catalog.cc:458] T 00000000000000000000000000000000 P 63272e34c419449bbb90a9013aa32b21 [sys.catalog]: This master's current role is: LEADER
I20260709 13:18:26.690577 15609 sys_catalog.cc:455] T 00000000000000000000000000000000 P 63272e34c419449bbb90a9013aa32b21 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 63272e34c419449bbb90a9013aa32b21. Latest consensus state: current_term: 1 leader_uuid: "63272e34c419449bbb90a9013aa32b21" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "63272e34c419449bbb90a9013aa32b21" member_type: VOTER } }
I20260709 13:18:26.691216 15609 sys_catalog.cc:458] T 00000000000000000000000000000000 P 63272e34c419449bbb90a9013aa32b21 [sys.catalog]: This master's current role is: LEADER
I20260709 13:18:26.693780 15615 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 13:18:26.701707 15615 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 13:18:26.702726 15046 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 13:18:26.710001 15615 catalog_manager.cc:1383] Generated new cluster ID: 5761c53f7891435ca5e118e862e80b4c
I20260709 13:18:26.710278 15615 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 13:18:26.724484 15615 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 13:18:26.725687 15615 catalog_manager.cc:1540] Loading token signing keys...
I20260709 13:18:26.738310 15615 catalog_manager.cc:6070] T 00000000000000000000000000000000 P 63272e34c419449bbb90a9013aa32b21: Generated new TSK 0
I20260709 13:18:26.738893 15615 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 13:18:26.769750 15046 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:18:26.775995 15625 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:18:26.777611 15626 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:18:26.778410 15628 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:18:26.779156 15046 server_base.cc:1061] running on GCE node
I20260709 13:18:26.780049 15046 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:18:26.780265 15046 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:18:26.780457 15046 hybrid_clock.cc:648] HybridClock initialized: now 1783603106780420 us; error 0 us; skew 500 ppm
I20260709 13:18:26.781018 15046 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:18:26.783496 15046 webserver.cc:533] Webserver started at http://127.14.177.129:43225/ using document root <none> and password file <none>
I20260709 13:18:26.783999 15046 fs_manager.cc:362] Metadata directory not provided
I20260709 13:18:26.784197 15046 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:18:26.784531 15046 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:18:26.785701 15046 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskdH_Ma7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783603093801613-15046-0/minicluster-data/ts-0-root/instance:
uuid: "8c1822e697d64b758a23b11c8fdeb59a"
format_stamp: "Formatted at 2026-07-09 13:18:26 on dist-test-slave-v5z2"
I20260709 13:18:26.790467 15046 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.006s	sys 0.000s
I20260709 13:18:26.793905 15633 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:18:26.794732 15046 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260709 13:18:26.795058 15046 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskdH_Ma7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783603093801613-15046-0/minicluster-data/ts-0-root
uuid: "8c1822e697d64b758a23b11c8fdeb59a"
format_stamp: "Formatted at 2026-07-09 13:18:26 on dist-test-slave-v5z2"
I20260709 13:18:26.795424 15046 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdH_Ma7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783603093801613-15046-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskdH_Ma7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783603093801613-15046-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskdH_Ma7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783603093801613-15046-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:18:26.812938 15046 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:18:26.814838 15046 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:18:26.816695 15046 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:18:26.818898 15046 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:18:26.819118 15046 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:18:26.819449 15046 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:18:26.819638 15046 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:18:26.859225 15046 rpc_server.cc:307] RPC server started. Bound to: 127.14.177.129:39067
I20260709 13:18:26.859340 15695 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.14.177.129:39067 every 8 connection(s)
I20260709 13:18:26.864075 15046 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:18:26.871696 15700 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:18:26.871948 15701 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:18:26.876820 15046 server_base.cc:1061] running on GCE node
W20260709 13:18:26.877142 15703 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:18:26.878059 15046 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:18:26.878343 15046 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:18:26.878535 15046 hybrid_clock.cc:648] HybridClock initialized: now 1783603106878512 us; error 0 us; skew 500 ppm
I20260709 13:18:26.879076 15046 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:18:26.882022 15046 webserver.cc:533] Webserver started at http://127.14.177.130:35613/ using document root <none> and password file <none>
I20260709 13:18:26.882133 15696 heartbeater.cc:344] Connected to a master server at 127.14.177.190:38145
I20260709 13:18:26.882543 15696 heartbeater.cc:461] Registering TS with master...
I20260709 13:18:26.882774 15046 fs_manager.cc:362] Metadata directory not provided
I20260709 13:18:26.883055 15046 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:18:26.883438 15046 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:18:26.883492 15696 heartbeater.cc:507] Master 127.14.177.190:38145 requested a full tablet report, sending...
I20260709 13:18:26.885238 15046 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskdH_Ma7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783603093801613-15046-0/minicluster-data/ts-1-root/instance:
uuid: "7cd7c680ef2e474c981ac81abadecfde"
format_stamp: "Formatted at 2026-07-09 13:18:26 on dist-test-slave-v5z2"
I20260709 13:18:26.885986 15569 ts_manager.cc:194] Registered new tserver with Master: 8c1822e697d64b758a23b11c8fdeb59a (127.14.177.129:39067)
I20260709 13:18:26.888185 15569 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:42342
I20260709 13:18:26.890497 15046 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.006s	sys 0.000s
I20260709 13:18:26.894217 15708 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:18:26.895092 15046 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.000s
I20260709 13:18:26.895457 15046 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskdH_Ma7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783603093801613-15046-0/minicluster-data/ts-1-root
uuid: "7cd7c680ef2e474c981ac81abadecfde"
format_stamp: "Formatted at 2026-07-09 13:18:26 on dist-test-slave-v5z2"
I20260709 13:18:26.895735 15046 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdH_Ma7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783603093801613-15046-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskdH_Ma7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783603093801613-15046-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskdH_Ma7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783603093801613-15046-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:18:26.918009 15046 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:18:26.919375 15046 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:18:26.920825 15046 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:18:26.922986 15046 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:18:26.923206 15046 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:18:26.923480 15046 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:18:26.923635 15046 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:18:26.965197 15046 rpc_server.cc:307] RPC server started. Bound to: 127.14.177.130:40117
I20260709 13:18:26.965266 15770 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.14.177.130:40117 every 8 connection(s)
I20260709 13:18:26.970155 15046 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:18:26.976428 15774 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:18:26.978111 15775 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:18:26.979661 15046 server_base.cc:1061] running on GCE node
W20260709 13:18:26.981233 15778 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:18:26.982115 15771 heartbeater.cc:344] Connected to a master server at 127.14.177.190:38145
I20260709 13:18:26.982465 15771 heartbeater.cc:461] Registering TS with master...
I20260709 13:18:26.982590 15046 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:18:26.982903 15046 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:18:26.983099 15046 hybrid_clock.cc:648] HybridClock initialized: now 1783603106983061 us; error 0 us; skew 500 ppm
I20260709 13:18:26.983320 15771 heartbeater.cc:507] Master 127.14.177.190:38145 requested a full tablet report, sending...
I20260709 13:18:26.983827 15046 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:18:26.985666 15569 ts_manager.cc:194] Registered new tserver with Master: 7cd7c680ef2e474c981ac81abadecfde (127.14.177.130:40117)
I20260709 13:18:26.987244 15046 webserver.cc:533] Webserver started at http://127.14.177.131:37463/ using document root <none> and password file <none>
I20260709 13:18:26.987708 15569 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:42350
I20260709 13:18:26.988042 15046 fs_manager.cc:362] Metadata directory not provided
I20260709 13:18:26.988338 15046 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:18:26.988677 15046 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:18:26.990046 15046 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskdH_Ma7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783603093801613-15046-0/minicluster-data/ts-2-root/instance:
uuid: "62bbd271280e4b0a9ddf9ab46151e455"
format_stamp: "Formatted at 2026-07-09 13:18:26 on dist-test-slave-v5z2"
I20260709 13:18:26.994488 15046 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.006s	sys 0.000s
I20260709 13:18:26.997977 15782 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:18:26.998849 15046 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260709 13:18:26.999116 15046 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskdH_Ma7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783603093801613-15046-0/minicluster-data/ts-2-root
uuid: "62bbd271280e4b0a9ddf9ab46151e455"
format_stamp: "Formatted at 2026-07-09 13:18:26 on dist-test-slave-v5z2"
I20260709 13:18:26.999430 15046 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdH_Ma7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783603093801613-15046-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskdH_Ma7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783603093801613-15046-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskdH_Ma7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783603093801613-15046-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:18:27.016106 15046 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:18:27.017278 15046 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:18:27.018666 15046 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:18:27.020870 15046 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:18:27.021234 15046 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:18:27.021549 15046 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:18:27.021742 15046 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:18:27.061973 15046 rpc_server.cc:307] RPC server started. Bound to: 127.14.177.131:33917
I20260709 13:18:27.062083 15844 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.14.177.131:33917 every 8 connection(s)
I20260709 13:18:27.076128 15845 heartbeater.cc:344] Connected to a master server at 127.14.177.190:38145
I20260709 13:18:27.076491 15845 heartbeater.cc:461] Registering TS with master...
I20260709 13:18:27.077256 15845 heartbeater.cc:507] Master 127.14.177.190:38145 requested a full tablet report, sending...
I20260709 13:18:27.079056 15569 ts_manager.cc:194] Registered new tserver with Master: 62bbd271280e4b0a9ddf9ab46151e455 (127.14.177.131:33917)
I20260709 13:18:27.079531 15046 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.014368978s
I20260709 13:18:27.080708 15569 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:42352
I20260709 13:18:27.104017 15569 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:42366:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\314\314\314\014\004\001\000\230\231\231\031\004\001\000dff&\004\001\0000333\004\001\000\374\377\377?\004\001\000\310\314\314L\004\001\000\224\231\231Y\004\001\000`fff\004\001\000,33s""\004\001\000\314\314\314\014\004\001\000\230\231\231\031\004\001\000dff&\004\001\0000333\004\001\000\374\377\377?\004\001\000\310\314\314L\004\001\000\224\231\231Y\004\001\000`fff\004\001\000,33s"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 13:18:27.106741 15569 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 13:18:27.175113 15735 tablet_service.cc:1511] Processing CreateTablet for tablet 8cff7911abd649b9a276ddae431311d1 (DEFAULT_TABLE table=test-workload [id=02a7e661bd66418bbe84dbb3a917bcca]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 13:18:27.176540 15734 tablet_service.cc:1511] Processing CreateTablet for tablet 0ce7d20d574f47649d3ad0f3c382198d (DEFAULT_TABLE table=test-workload [id=02a7e661bd66418bbe84dbb3a917bcca]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 13:18:27.177862 15735 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8cff7911abd649b9a276ddae431311d1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:27.178526 15734 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0ce7d20d574f47649d3ad0f3c382198d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:27.177474 15809 tablet_service.cc:1511] Processing CreateTablet for tablet 8cff7911abd649b9a276ddae431311d1 (DEFAULT_TABLE table=test-workload [id=02a7e661bd66418bbe84dbb3a917bcca]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 13:18:27.179375 15808 tablet_service.cc:1511] Processing CreateTablet for tablet 0ce7d20d574f47649d3ad0f3c382198d (DEFAULT_TABLE table=test-workload [id=02a7e661bd66418bbe84dbb3a917bcca]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 13:18:27.180240 15809 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8cff7911abd649b9a276ddae431311d1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:27.175632 15736 tablet_service.cc:1511] Processing CreateTablet for tablet 4d83a05b849340c186cc995649575d3b (DEFAULT_TABLE table=test-workload [id=02a7e661bd66418bbe84dbb3a917bcca]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 13:18:27.182050 15736 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4d83a05b849340c186cc995649575d3b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:27.176142 15810 tablet_service.cc:1511] Processing CreateTablet for tablet 4d83a05b849340c186cc995649575d3b (DEFAULT_TABLE table=test-workload [id=02a7e661bd66418bbe84dbb3a917bcca]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 13:18:27.188690 15810 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4d83a05b849340c186cc995649575d3b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:27.191572 15808 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0ce7d20d574f47649d3ad0f3c382198d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:27.191409 15733 tablet_service.cc:1511] Processing CreateTablet for tablet 566eae6d93af4ec492b9b1a270dc03b7 (DEFAULT_TABLE table=test-workload [id=02a7e661bd66418bbe84dbb3a917bcca]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 13:18:27.192946 15733 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 566eae6d93af4ec492b9b1a270dc03b7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:27.211292 15807 tablet_service.cc:1511] Processing CreateTablet for tablet 566eae6d93af4ec492b9b1a270dc03b7 (DEFAULT_TABLE table=test-workload [id=02a7e661bd66418bbe84dbb3a917bcca]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 13:18:27.212775 15807 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 566eae6d93af4ec492b9b1a270dc03b7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:27.216398 15661 tablet_service.cc:1511] Processing CreateTablet for tablet 4d83a05b849340c186cc995649575d3b (DEFAULT_TABLE table=test-workload [id=02a7e661bd66418bbe84dbb3a917bcca]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 13:18:27.218015 15661 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4d83a05b849340c186cc995649575d3b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:27.220731 15660 tablet_service.cc:1511] Processing CreateTablet for tablet 8cff7911abd649b9a276ddae431311d1 (DEFAULT_TABLE table=test-workload [id=02a7e661bd66418bbe84dbb3a917bcca]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 13:18:27.224010 15659 tablet_service.cc:1511] Processing CreateTablet for tablet 0ce7d20d574f47649d3ad0f3c382198d (DEFAULT_TABLE table=test-workload [id=02a7e661bd66418bbe84dbb3a917bcca]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 13:18:27.226984 15732 tablet_service.cc:1511] Processing CreateTablet for tablet 8fc26f7d12464423a212a1f595ca4b88 (DEFAULT_TABLE table=test-workload [id=02a7e661bd66418bbe84dbb3a917bcca]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 13:18:27.227365 15658 tablet_service.cc:1511] Processing CreateTablet for tablet 566eae6d93af4ec492b9b1a270dc03b7 (DEFAULT_TABLE table=test-workload [id=02a7e661bd66418bbe84dbb3a917bcca]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 13:18:27.228821 15732 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8fc26f7d12464423a212a1f595ca4b88. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:27.230808 15657 tablet_service.cc:1511] Processing CreateTablet for tablet 8fc26f7d12464423a212a1f595ca4b88 (DEFAULT_TABLE table=test-workload [id=02a7e661bd66418bbe84dbb3a917bcca]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 13:18:27.239675 15660 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8cff7911abd649b9a276ddae431311d1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:27.243671 15659 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0ce7d20d574f47649d3ad0f3c382198d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:27.247720 15658 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 566eae6d93af4ec492b9b1a270dc03b7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:27.248078 15806 tablet_service.cc:1511] Processing CreateTablet for tablet 8fc26f7d12464423a212a1f595ca4b88 (DEFAULT_TABLE table=test-workload [id=02a7e661bd66418bbe84dbb3a917bcca]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 13:18:27.249938 15806 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8fc26f7d12464423a212a1f595ca4b88. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:27.251677 15657 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8fc26f7d12464423a212a1f595ca4b88. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:27.269995 15656 tablet_service.cc:1511] Processing CreateTablet for tablet 408866bb47904e4189bc235ca5b0c7c7 (DEFAULT_TABLE table=test-workload [id=02a7e661bd66418bbe84dbb3a917bcca]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 13:18:27.272583 15656 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 408866bb47904e4189bc235ca5b0c7c7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:27.300670 15861 tablet_bootstrap.cc:492] T 8cff7911abd649b9a276ddae431311d1 P 62bbd271280e4b0a9ddf9ab46151e455: Bootstrap starting.
I20260709 13:18:27.303887 15860 tablet_bootstrap.cc:492] T 8cff7911abd649b9a276ddae431311d1 P 7cd7c680ef2e474c981ac81abadecfde: Bootstrap starting.
I20260709 13:18:27.308769 15735 tablet_service.cc:1511] Processing CreateTablet for tablet 17f97fe9a93f4a86b1e2bad621bfb561 (DEFAULT_TABLE table=test-workload [id=02a7e661bd66418bbe84dbb3a917bcca]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 13:18:27.310106 15735 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 17f97fe9a93f4a86b1e2bad621bfb561. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:27.313009 15808 tablet_service.cc:1511] Processing CreateTablet for tablet 408866bb47904e4189bc235ca5b0c7c7 (DEFAULT_TABLE table=test-workload [id=02a7e661bd66418bbe84dbb3a917bcca]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 13:18:27.314432 15808 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 408866bb47904e4189bc235ca5b0c7c7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:27.314162 15655 tablet_service.cc:1511] Processing CreateTablet for tablet 8309d9a5a4ef4e6d914751bc0f410015 (DEFAULT_TABLE table=test-workload [id=02a7e661bd66418bbe84dbb3a917bcca]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 13:18:27.315560 15655 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8309d9a5a4ef4e6d914751bc0f410015. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:27.316471 15656 tablet_service.cc:1511] Processing CreateTablet for tablet 17f97fe9a93f4a86b1e2bad621bfb561 (DEFAULT_TABLE table=test-workload [id=02a7e661bd66418bbe84dbb3a917bcca]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 13:18:27.315739 15809 tablet_service.cc:1511] Processing CreateTablet for tablet 17f97fe9a93f4a86b1e2bad621bfb561 (DEFAULT_TABLE table=test-workload [id=02a7e661bd66418bbe84dbb3a917bcca]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 13:18:27.317804 15656 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 17f97fe9a93f4a86b1e2bad621bfb561. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:27.318457 15809 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 17f97fe9a93f4a86b1e2bad621bfb561. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:27.320550 15733 tablet_service.cc:1511] Processing CreateTablet for tablet 8309d9a5a4ef4e6d914751bc0f410015 (DEFAULT_TABLE table=test-workload [id=02a7e661bd66418bbe84dbb3a917bcca]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 13:18:27.321893 15733 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8309d9a5a4ef4e6d914751bc0f410015. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:27.322852 15861 tablet_bootstrap.cc:654] T 8cff7911abd649b9a276ddae431311d1 P 62bbd271280e4b0a9ddf9ab46151e455: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:27.324117 15736 tablet_service.cc:1511] Processing CreateTablet for tablet 408866bb47904e4189bc235ca5b0c7c7 (DEFAULT_TABLE table=test-workload [id=02a7e661bd66418bbe84dbb3a917bcca]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 13:18:27.325348 15736 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 408866bb47904e4189bc235ca5b0c7c7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:27.340785 15862 tablet_bootstrap.cc:492] T 408866bb47904e4189bc235ca5b0c7c7 P 8c1822e697d64b758a23b11c8fdeb59a: Bootstrap starting.
I20260709 13:18:27.343892 15805 tablet_service.cc:1511] Processing CreateTablet for tablet 8309d9a5a4ef4e6d914751bc0f410015 (DEFAULT_TABLE table=test-workload [id=02a7e661bd66418bbe84dbb3a917bcca]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 13:18:27.344744 15660 tablet_service.cc:1511] Processing CreateTablet for tablet 48e8b6e82d7b42e6bdd4b0a8a25dfd9e (DEFAULT_TABLE table=test-workload [id=02a7e661bd66418bbe84dbb3a917bcca]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 13:18:27.345248 15805 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8309d9a5a4ef4e6d914751bc0f410015. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:27.346096 15660 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 48e8b6e82d7b42e6bdd4b0a8a25dfd9e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:27.350136 15860 tablet_bootstrap.cc:654] T 8cff7911abd649b9a276ddae431311d1 P 7cd7c680ef2e474c981ac81abadecfde: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:27.359005 15734 tablet_service.cc:1511] Processing CreateTablet for tablet 48e8b6e82d7b42e6bdd4b0a8a25dfd9e (DEFAULT_TABLE table=test-workload [id=02a7e661bd66418bbe84dbb3a917bcca]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 13:18:27.360992 15734 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 48e8b6e82d7b42e6bdd4b0a8a25dfd9e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:27.371570 15808 tablet_service.cc:1511] Processing CreateTablet for tablet 48e8b6e82d7b42e6bdd4b0a8a25dfd9e (DEFAULT_TABLE table=test-workload [id=02a7e661bd66418bbe84dbb3a917bcca]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 13:18:27.374368 15862 tablet_bootstrap.cc:654] T 408866bb47904e4189bc235ca5b0c7c7 P 8c1822e697d64b758a23b11c8fdeb59a: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:27.385054 15808 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 48e8b6e82d7b42e6bdd4b0a8a25dfd9e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:27.392230 15658 tablet_service.cc:1511] Processing CreateTablet for tablet 502d133e880b4a859d8674b9d0cb6931 (DEFAULT_TABLE table=test-workload [id=02a7e661bd66418bbe84dbb3a917bcca]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 13:18:27.394469 15658 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 502d133e880b4a859d8674b9d0cb6931. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:27.394984 15734 tablet_service.cc:1511] Processing CreateTablet for tablet 502d133e880b4a859d8674b9d0cb6931 (DEFAULT_TABLE table=test-workload [id=02a7e661bd66418bbe84dbb3a917bcca]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 13:18:27.396234 15734 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 502d133e880b4a859d8674b9d0cb6931. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:27.396000 15809 tablet_service.cc:1511] Processing CreateTablet for tablet 502d133e880b4a859d8674b9d0cb6931 (DEFAULT_TABLE table=test-workload [id=02a7e661bd66418bbe84dbb3a917bcca]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 13:18:27.397145 15809 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 502d133e880b4a859d8674b9d0cb6931. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:27.410320 15860 tablet_bootstrap.cc:492] T 8cff7911abd649b9a276ddae431311d1 P 7cd7c680ef2e474c981ac81abadecfde: No bootstrap required, opened a new log
I20260709 13:18:27.410854 15860 ts_tablet_manager.cc:1403] T 8cff7911abd649b9a276ddae431311d1 P 7cd7c680ef2e474c981ac81abadecfde: Time spent bootstrapping tablet: real 0.113s	user 0.012s	sys 0.009s
I20260709 13:18:27.417943 15861 tablet_bootstrap.cc:492] T 8cff7911abd649b9a276ddae431311d1 P 62bbd271280e4b0a9ddf9ab46151e455: No bootstrap required, opened a new log
I20260709 13:18:27.418516 15861 ts_tablet_manager.cc:1403] T 8cff7911abd649b9a276ddae431311d1 P 62bbd271280e4b0a9ddf9ab46151e455: Time spent bootstrapping tablet: real 0.118s	user 0.010s	sys 0.015s
I20260709 13:18:27.422525 15861 raft_consensus.cc:359] T 8cff7911abd649b9a276ddae431311d1 P 62bbd271280e4b0a9ddf9ab46151e455 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c1822e697d64b758a23b11c8fdeb59a" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 39067 } } peers { permanent_uuid: "7cd7c680ef2e474c981ac81abadecfde" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 40117 } } peers { permanent_uuid: "62bbd271280e4b0a9ddf9ab46151e455" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 33917 } }
I20260709 13:18:27.423605 15861 raft_consensus.cc:385] T 8cff7911abd649b9a276ddae431311d1 P 62bbd271280e4b0a9ddf9ab46151e455 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:27.424152 15861 raft_consensus.cc:740] T 8cff7911abd649b9a276ddae431311d1 P 62bbd271280e4b0a9ddf9ab46151e455 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 62bbd271280e4b0a9ddf9ab46151e455, State: Initialized, Role: FOLLOWER
I20260709 13:18:27.423599 15860 raft_consensus.cc:359] T 8cff7911abd649b9a276ddae431311d1 P 7cd7c680ef2e474c981ac81abadecfde [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c1822e697d64b758a23b11c8fdeb59a" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 39067 } } peers { permanent_uuid: "7cd7c680ef2e474c981ac81abadecfde" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 40117 } } peers { permanent_uuid: "62bbd271280e4b0a9ddf9ab46151e455" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 33917 } }
I20260709 13:18:27.424600 15860 raft_consensus.cc:385] T 8cff7911abd649b9a276ddae431311d1 P 7cd7c680ef2e474c981ac81abadecfde [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:27.424935 15860 raft_consensus.cc:740] T 8cff7911abd649b9a276ddae431311d1 P 7cd7c680ef2e474c981ac81abadecfde [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7cd7c680ef2e474c981ac81abadecfde, State: Initialized, Role: FOLLOWER
I20260709 13:18:27.425035 15861 consensus_queue.cc:260] T 8cff7911abd649b9a276ddae431311d1 P 62bbd271280e4b0a9ddf9ab46151e455 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c1822e697d64b758a23b11c8fdeb59a" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 39067 } } peers { permanent_uuid: "7cd7c680ef2e474c981ac81abadecfde" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 40117 } } peers { permanent_uuid: "62bbd271280e4b0a9ddf9ab46151e455" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 33917 } }
I20260709 13:18:27.425611 15860 consensus_queue.cc:260] T 8cff7911abd649b9a276ddae431311d1 P 7cd7c680ef2e474c981ac81abadecfde [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c1822e697d64b758a23b11c8fdeb59a" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 39067 } } peers { permanent_uuid: "7cd7c680ef2e474c981ac81abadecfde" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 40117 } } peers { permanent_uuid: "62bbd271280e4b0a9ddf9ab46151e455" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 33917 } }
I20260709 13:18:27.426414 15862 tablet_bootstrap.cc:492] T 408866bb47904e4189bc235ca5b0c7c7 P 8c1822e697d64b758a23b11c8fdeb59a: No bootstrap required, opened a new log
I20260709 13:18:27.427068 15862 ts_tablet_manager.cc:1403] T 408866bb47904e4189bc235ca5b0c7c7 P 8c1822e697d64b758a23b11c8fdeb59a: Time spent bootstrapping tablet: real 0.087s	user 0.004s	sys 0.021s
I20260709 13:18:27.431257 15862 raft_consensus.cc:359] T 408866bb47904e4189bc235ca5b0c7c7 P 8c1822e697d64b758a23b11c8fdeb59a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c1822e697d64b758a23b11c8fdeb59a" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 39067 } } peers { permanent_uuid: "7cd7c680ef2e474c981ac81abadecfde" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 40117 } } peers { permanent_uuid: "62bbd271280e4b0a9ddf9ab46151e455" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 33917 } }
I20260709 13:18:27.432358 15862 raft_consensus.cc:385] T 408866bb47904e4189bc235ca5b0c7c7 P 8c1822e697d64b758a23b11c8fdeb59a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:27.432775 15862 raft_consensus.cc:740] T 408866bb47904e4189bc235ca5b0c7c7 P 8c1822e697d64b758a23b11c8fdeb59a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c1822e697d64b758a23b11c8fdeb59a, State: Initialized, Role: FOLLOWER
I20260709 13:18:27.433750 15862 consensus_queue.cc:260] T 408866bb47904e4189bc235ca5b0c7c7 P 8c1822e697d64b758a23b11c8fdeb59a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c1822e697d64b758a23b11c8fdeb59a" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 39067 } } peers { permanent_uuid: "7cd7c680ef2e474c981ac81abadecfde" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 40117 } } peers { permanent_uuid: "62bbd271280e4b0a9ddf9ab46151e455" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 33917 } }
I20260709 13:18:27.434489 15861 ts_tablet_manager.cc:1434] T 8cff7911abd649b9a276ddae431311d1 P 62bbd271280e4b0a9ddf9ab46151e455: Time spent starting tablet: real 0.016s	user 0.007s	sys 0.007s
I20260709 13:18:27.435356 15845 heartbeater.cc:499] Master 127.14.177.190:38145 was elected leader, sending a full tablet report...
I20260709 13:18:27.438822 15771 heartbeater.cc:499] Master 127.14.177.190:38145 was elected leader, sending a full tablet report...
I20260709 13:18:27.439430 15860 ts_tablet_manager.cc:1434] T 8cff7911abd649b9a276ddae431311d1 P 7cd7c680ef2e474c981ac81abadecfde: Time spent starting tablet: real 0.028s	user 0.010s	sys 0.009s
I20260709 13:18:27.440354 15860 tablet_bootstrap.cc:492] T 0ce7d20d574f47649d3ad0f3c382198d P 7cd7c680ef2e474c981ac81abadecfde: Bootstrap starting.
I20260709 13:18:27.443462 15696 heartbeater.cc:499] Master 127.14.177.190:38145 was elected leader, sending a full tablet report...
I20260709 13:18:27.444113 15862 ts_tablet_manager.cc:1434] T 408866bb47904e4189bc235ca5b0c7c7 P 8c1822e697d64b758a23b11c8fdeb59a: Time spent starting tablet: real 0.017s	user 0.009s	sys 0.006s
I20260709 13:18:27.445178 15862 tablet_bootstrap.cc:492] T 8fc26f7d12464423a212a1f595ca4b88 P 8c1822e697d64b758a23b11c8fdeb59a: Bootstrap starting.
I20260709 13:18:27.446939 15860 tablet_bootstrap.cc:654] T 0ce7d20d574f47649d3ad0f3c382198d P 7cd7c680ef2e474c981ac81abadecfde: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:27.452517 15862 tablet_bootstrap.cc:654] T 8fc26f7d12464423a212a1f595ca4b88 P 8c1822e697d64b758a23b11c8fdeb59a: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:27.455924 15861 tablet_bootstrap.cc:492] T 0ce7d20d574f47649d3ad0f3c382198d P 62bbd271280e4b0a9ddf9ab46151e455: Bootstrap starting.
I20260709 13:18:27.457151 15860 tablet_bootstrap.cc:492] T 0ce7d20d574f47649d3ad0f3c382198d P 7cd7c680ef2e474c981ac81abadecfde: No bootstrap required, opened a new log
I20260709 13:18:27.457619 15860 ts_tablet_manager.cc:1403] T 0ce7d20d574f47649d3ad0f3c382198d P 7cd7c680ef2e474c981ac81abadecfde: Time spent bootstrapping tablet: real 0.017s	user 0.011s	sys 0.004s
I20260709 13:18:27.459256 15862 tablet_bootstrap.cc:492] T 8fc26f7d12464423a212a1f595ca4b88 P 8c1822e697d64b758a23b11c8fdeb59a: No bootstrap required, opened a new log
I20260709 13:18:27.459721 15862 ts_tablet_manager.cc:1403] T 8fc26f7d12464423a212a1f595ca4b88 P 8c1822e697d64b758a23b11c8fdeb59a: Time spent bootstrapping tablet: real 0.015s	user 0.011s	sys 0.002s
I20260709 13:18:27.460547 15860 raft_consensus.cc:359] T 0ce7d20d574f47649d3ad0f3c382198d P 7cd7c680ef2e474c981ac81abadecfde [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c1822e697d64b758a23b11c8fdeb59a" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 39067 } } peers { permanent_uuid: "7cd7c680ef2e474c981ac81abadecfde" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 40117 } } peers { permanent_uuid: "62bbd271280e4b0a9ddf9ab46151e455" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 33917 } }
I20260709 13:18:27.461246 15860 raft_consensus.cc:385] T 0ce7d20d574f47649d3ad0f3c382198d P 7cd7c680ef2e474c981ac81abadecfde [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:27.461525 15860 raft_consensus.cc:740] T 0ce7d20d574f47649d3ad0f3c382198d P 7cd7c680ef2e474c981ac81abadecfde [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7cd7c680ef2e474c981ac81abadecfde, State: Initialized, Role: FOLLOWER
I20260709 13:18:27.462414 15862 raft_consensus.cc:359] T 8fc26f7d12464423a212a1f595ca4b88 P 8c1822e697d64b758a23b11c8fdeb59a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c1822e697d64b758a23b11c8fdeb59a" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 39067 } } peers { permanent_uuid: "7cd7c680ef2e474c981ac81abadecfde" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 40117 } } peers { permanent_uuid: "62bbd271280e4b0a9ddf9ab46151e455" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 33917 } }
I20260709 13:18:27.462356 15860 consensus_queue.cc:260] T 0ce7d20d574f47649d3ad0f3c382198d P 7cd7c680ef2e474c981ac81abadecfde [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c1822e697d64b758a23b11c8fdeb59a" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 39067 } } peers { permanent_uuid: "7cd7c680ef2e474c981ac81abadecfde" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 40117 } } peers { permanent_uuid: "62bbd271280e4b0a9ddf9ab46151e455" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 33917 } }
I20260709 13:18:27.463127 15862 raft_consensus.cc:385] T 8fc26f7d12464423a212a1f595ca4b88 P 8c1822e697d64b758a23b11c8fdeb59a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:27.463487 15862 raft_consensus.cc:740] T 8fc26f7d12464423a212a1f595ca4b88 P 8c1822e697d64b758a23b11c8fdeb59a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c1822e697d64b758a23b11c8fdeb59a, State: Initialized, Role: FOLLOWER
I20260709 13:18:27.463784 15861 tablet_bootstrap.cc:654] T 0ce7d20d574f47649d3ad0f3c382198d P 62bbd271280e4b0a9ddf9ab46151e455: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:27.464867 15860 ts_tablet_manager.cc:1434] T 0ce7d20d574f47649d3ad0f3c382198d P 7cd7c680ef2e474c981ac81abadecfde: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 13:18:27.464359 15862 consensus_queue.cc:260] T 8fc26f7d12464423a212a1f595ca4b88 P 8c1822e697d64b758a23b11c8fdeb59a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c1822e697d64b758a23b11c8fdeb59a" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 39067 } } peers { permanent_uuid: "7cd7c680ef2e474c981ac81abadecfde" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 40117 } } peers { permanent_uuid: "62bbd271280e4b0a9ddf9ab46151e455" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 33917 } }
I20260709 13:18:27.465790 15860 tablet_bootstrap.cc:492] T 4d83a05b849340c186cc995649575d3b P 7cd7c680ef2e474c981ac81abadecfde: Bootstrap starting.
I20260709 13:18:27.466667 15862 ts_tablet_manager.cc:1434] T 8fc26f7d12464423a212a1f595ca4b88 P 8c1822e697d64b758a23b11c8fdeb59a: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 13:18:27.467748 15862 tablet_bootstrap.cc:492] T 0ce7d20d574f47649d3ad0f3c382198d P 8c1822e697d64b758a23b11c8fdeb59a: Bootstrap starting.
I20260709 13:18:27.473953 15860 tablet_bootstrap.cc:654] T 4d83a05b849340c186cc995649575d3b P 7cd7c680ef2e474c981ac81abadecfde: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:27.474107 15862 tablet_bootstrap.cc:654] T 0ce7d20d574f47649d3ad0f3c382198d P 8c1822e697d64b758a23b11c8fdeb59a: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:27.476054 15861 tablet_bootstrap.cc:492] T 0ce7d20d574f47649d3ad0f3c382198d P 62bbd271280e4b0a9ddf9ab46151e455: No bootstrap required, opened a new log
I20260709 13:18:27.476609 15861 ts_tablet_manager.cc:1403] T 0ce7d20d574f47649d3ad0f3c382198d P 62bbd271280e4b0a9ddf9ab46151e455: Time spent bootstrapping tablet: real 0.021s	user 0.017s	sys 0.001s
I20260709 13:18:27.481336 15862 tablet_bootstrap.cc:492] T 0ce7d20d574f47649d3ad0f3c382198d P 8c1822e697d64b758a23b11c8fdeb59a: No bootstrap required, opened a new log
I20260709 13:18:27.480762 15861 raft_consensus.cc:359] T 0ce7d20d574f47649d3ad0f3c382198d P 62bbd271280e4b0a9ddf9ab46151e455 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c1822e697d64b758a23b11c8fdeb59a" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 39067 } } peers { permanent_uuid: "7cd7c680ef2e474c981ac81abadecfde" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 40117 } } peers { permanent_uuid: "62bbd271280e4b0a9ddf9ab46151e455" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 33917 } }
I20260709 13:18:27.481839 15861 raft_consensus.cc:385] T 0ce7d20d574f47649d3ad0f3c382198d P 62bbd271280e4b0a9ddf9ab46151e455 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:27.481964 15862 ts_tablet_manager.cc:1403] T 0ce7d20d574f47649d3ad0f3c382198d P 8c1822e697d64b758a23b11c8fdeb59a: Time spent bootstrapping tablet: real 0.014s	user 0.011s	sys 0.000s
I20260709 13:18:27.482209 15861 raft_consensus.cc:740] T 0ce7d20d574f47649d3ad0f3c382198d P 62bbd271280e4b0a9ddf9ab46151e455 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 62bbd271280e4b0a9ddf9ab46151e455, State: Initialized, Role: FOLLOWER
I20260709 13:18:27.483335 15861 consensus_queue.cc:260] T 0ce7d20d574f47649d3ad0f3c382198d P 62bbd271280e4b0a9ddf9ab46151e455 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c1822e697d64b758a23b11c8fdeb59a" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 39067 } } peers { permanent_uuid: "7cd7c680ef2e474c981ac81abadecfde" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 40117 } } peers { permanent_uuid: "62bbd271280e4b0a9ddf9ab46151e455" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 33917 } }
I20260709 13:18:27.486001 15861 ts_tablet_manager.cc:1434] T 0ce7d20d574f47649d3ad0f3c382198d P 62bbd271280e4b0a9ddf9ab46151e455: Time spent starting tablet: real 0.009s	user 0.004s	sys 0.003s
I20260709 13:18:27.485864 15862 raft_consensus.cc:359] T 0ce7d20d574f47649d3ad0f3c382198d P 8c1822e697d64b758a23b11c8fdeb59a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c1822e697d64b758a23b11c8fdeb59a" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 39067 } } peers { permanent_uuid: "7cd7c680ef2e474c981ac81abadecfde" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 40117 } } peers { permanent_uuid: "62bbd271280e4b0a9ddf9ab46151e455" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 33917 } }
I20260709 13:18:27.486941 15862 raft_consensus.cc:385] T 0ce7d20d574f47649d3ad0f3c382198d P 8c1822e697d64b758a23b11c8fdeb59a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:27.487262 15861 tablet_bootstrap.cc:492] T 8fc26f7d12464423a212a1f595ca4b88 P 62bbd271280e4b0a9ddf9ab46151e455: Bootstrap starting.
I20260709 13:18:27.487360 15862 raft_consensus.cc:740] T 0ce7d20d574f47649d3ad0f3c382198d P 8c1822e697d64b758a23b11c8fdeb59a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c1822e697d64b758a23b11c8fdeb59a, State: Initialized, Role: FOLLOWER
I20260709 13:18:27.488443 15862 consensus_queue.cc:260] T 0ce7d20d574f47649d3ad0f3c382198d P 8c1822e697d64b758a23b11c8fdeb59a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c1822e697d64b758a23b11c8fdeb59a" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 39067 } } peers { permanent_uuid: "7cd7c680ef2e474c981ac81abadecfde" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 40117 } } peers { permanent_uuid: "62bbd271280e4b0a9ddf9ab46151e455" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 33917 } }
I20260709 13:18:27.491431 15862 ts_tablet_manager.cc:1434] T 0ce7d20d574f47649d3ad0f3c382198d P 8c1822e697d64b758a23b11c8fdeb59a: Time spent starting tablet: real 0.009s	user 0.003s	sys 0.008s
I20260709 13:18:27.492625 15862 tablet_bootstrap.cc:492] T 4d83a05b849340c186cc995649575d3b P 8c1822e697d64b758a23b11c8fdeb59a: Bootstrap starting.
I20260709 13:18:27.495676 15867 raft_consensus.cc:493] T 0ce7d20d574f47649d3ad0f3c382198d P 7cd7c680ef2e474c981ac81abadecfde [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:27.496280 15867 raft_consensus.cc:515] T 0ce7d20d574f47649d3ad0f3c382198d P 7cd7c680ef2e474c981ac81abadecfde [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c1822e697d64b758a23b11c8fdeb59a" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 39067 } } peers { permanent_uuid: "7cd7c680ef2e474c981ac81abadecfde" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 40117 } } peers { permanent_uuid: "62bbd271280e4b0a9ddf9ab46151e455" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 33917 } }
I20260709 13:18:27.499446 15861 tablet_bootstrap.cc:654] T 8fc26f7d12464423a212a1f595ca4b88 P 62bbd271280e4b0a9ddf9ab46151e455: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:27.499570 15867 leader_election.cc:290] T 0ce7d20d574f47649d3ad0f3c382198d P 7cd7c680ef2e474c981ac81abadecfde [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8c1822e697d64b758a23b11c8fdeb59a (127.14.177.129:39067), 62bbd271280e4b0a9ddf9ab46151e455 (127.14.177.131:33917)
I20260709 13:18:27.508729 15860 tablet_bootstrap.cc:492] T 4d83a05b849340c186cc995649575d3b P 7cd7c680ef2e474c981ac81abadecfde: No bootstrap required, opened a new log
I20260709 13:18:27.509728 15860 ts_tablet_manager.cc:1403] T 4d83a05b849340c186cc995649575d3b P 7cd7c680ef2e474c981ac81abadecfde: Time spent bootstrapping tablet: real 0.044s	user 0.009s	sys 0.022s
I20260709 13:18:27.510474 15862 tablet_bootstrap.cc:654] T 4d83a05b849340c186cc995649575d3b P 8c1822e697d64b758a23b11c8fdeb59a: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:27.512902 15860 raft_consensus.cc:359] T 4d83a05b849340c186cc995649575d3b P 7cd7c680ef2e474c981ac81abadecfde [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62bbd271280e4b0a9ddf9ab46151e455" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 33917 } } peers { permanent_uuid: "7cd7c680ef2e474c981ac81abadecfde" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 40117 } } peers { permanent_uuid: "8c1822e697d64b758a23b11c8fdeb59a" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 39067 } }
I20260709 13:18:27.518532 15860 raft_consensus.cc:385] T 4d83a05b849340c186cc995649575d3b P 7cd7c680ef2e474c981ac81abadecfde [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:27.519055 15860 raft_consensus.cc:740] T 4d83a05b849340c186cc995649575d3b P 7cd7c680ef2e474c981ac81abadecfde [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7cd7c680ef2e474c981ac81abadecfde, State: Initialized, Role: FOLLOWER
I20260709 13:18:27.520107 15861 tablet_bootstrap.cc:492] T 8fc26f7d12464423a212a1f595ca4b88 P 62bbd271280e4b0a9ddf9ab46151e455: No bootstrap required, opened a new log
I20260709 13:18:27.520635 15861 ts_tablet_manager.cc:1403] T 8fc26f7d12464423a212a1f595ca4b88 P 62bbd271280e4b0a9ddf9ab46151e455: Time spent bootstrapping tablet: real 0.034s	user 0.014s	sys 0.001s
I20260709 13:18:27.520303 15860 consensus_queue.cc:260] T 4d83a05b849340c186cc995649575d3b P 7cd7c680ef2e474c981ac81abadecfde [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62bbd271280e4b0a9ddf9ab46151e455" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 33917 } } peers { permanent_uuid: "7cd7c680ef2e474c981ac81abadecfde" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 40117 } } peers { permanent_uuid: "8c1822e697d64b758a23b11c8fdeb59a" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 39067 } }
I20260709 13:18:27.523882 15860 ts_tablet_manager.cc:1434] T 4d83a05b849340c186cc995649575d3b P 7cd7c680ef2e474c981ac81abadecfde: Time spent starting tablet: real 0.014s	user 0.007s	sys 0.000s
I20260709 13:18:27.523933 15861 raft_consensus.cc:359] T 8fc26f7d12464423a212a1f595ca4b88 P 62bbd271280e4b0a9ddf9ab46151e455 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c1822e697d64b758a23b11c8fdeb59a" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 39067 } } peers { permanent_uuid: "7cd7c680ef2e474c981ac81abadecfde" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 40117 } } peers { permanent_uuid: "62bbd271280e4b0a9ddf9ab46151e455" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 33917 } }
I20260709 13:18:27.524963 15861 raft_consensus.cc:385] T 8fc26f7d12464423a212a1f595ca4b88 P 62bbd271280e4b0a9ddf9ab46151e455 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:27.525308 15861 raft_consensus.cc:740] T 8fc26f7d12464423a212a1f595ca4b88 P 62bbd271280e4b0a9ddf9ab46151e455 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 62bbd271280e4b0a9ddf9ab46151e455, State: Initialized, Role: FOLLOWER
I20260709 13:18:27.525455 15860 tablet_bootstrap.cc:492] T 566eae6d93af4ec492b9b1a270dc03b7 P 7cd7c680ef2e474c981ac81abadecfde: Bootstrap starting.
I20260709 13:18:27.526356 15861 consensus_queue.cc:260] T 8fc26f7d12464423a212a1f595ca4b88 P 62bbd271280e4b0a9ddf9ab46151e455 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c1822e697d64b758a23b11c8fdeb59a" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 39067 } } peers { permanent_uuid: "7cd7c680ef2e474c981ac81abadecfde" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 40117 } } peers { permanent_uuid: "62bbd271280e4b0a9ddf9ab46151e455" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 33917 } }
I20260709 13:18:27.528228 15862 tablet_bootstrap.cc:492] T 4d83a05b849340c186cc995649575d3b P 8c1822e697d64b758a23b11c8fdeb59a: No bootstrap required, opened a new log
I20260709 13:18:27.528687 15862 ts_tablet_manager.cc:1403] T 4d83a05b849340c186cc995649575d3b P 8c1822e697d64b758a23b11c8fdeb59a: Time spent bootstrapping tablet: real 0.036s	user 0.012s	sys 0.011s
I20260709 13:18:27.528713 15861 ts_tablet_manager.cc:1434] T 8fc26f7d12464423a212a1f595ca4b88 P 62bbd271280e4b0a9ddf9ab46151e455: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 13:18:27.529857 15861 tablet_bootstrap.cc:492] T 566eae6d93af4ec492b9b1a270dc03b7 P 62bbd271280e4b0a9ddf9ab46151e455: Bootstrap starting.
I20260709 13:18:27.531832 15862 raft_consensus.cc:359] T 4d83a05b849340c186cc995649575d3b P 8c1822e697d64b758a23b11c8fdeb59a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62bbd271280e4b0a9ddf9ab46151e455" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 33917 } } peers { permanent_uuid: "7cd7c680ef2e474c981ac81abadecfde" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 40117 } } peers { permanent_uuid: "8c1822e697d64b758a23b11c8fdeb59a" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 39067 } }
I20260709 13:18:27.532538 15862 raft_consensus.cc:385] T 4d83a05b849340c186cc995649575d3b P 8c1822e697d64b758a23b11c8fdeb59a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:27.532845 15862 raft_consensus.cc:740] T 4d83a05b849340c186cc995649575d3b P 8c1822e697d64b758a23b11c8fdeb59a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c1822e697d64b758a23b11c8fdeb59a, State: Initialized, Role: FOLLOWER
I20260709 13:18:27.533790 15860 tablet_bootstrap.cc:654] T 566eae6d93af4ec492b9b1a270dc03b7 P 7cd7c680ef2e474c981ac81abadecfde: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:27.533520 15862 consensus_queue.cc:260] T 4d83a05b849340c186cc995649575d3b P 8c1822e697d64b758a23b11c8fdeb59a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62bbd271280e4b0a9ddf9ab46151e455" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 33917 } } peers { permanent_uuid: "7cd7c680ef2e474c981ac81abadecfde" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 40117 } } peers { permanent_uuid: "8c1822e697d64b758a23b11c8fdeb59a" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 39067 } }
I20260709 13:18:27.535856 15862 ts_tablet_manager.cc:1434] T 4d83a05b849340c186cc995649575d3b P 8c1822e697d64b758a23b11c8fdeb59a: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:18:27.540138 15862 tablet_bootstrap.cc:492] T 8cff7911abd649b9a276ddae431311d1 P 8c1822e697d64b758a23b11c8fdeb59a: Bootstrap starting.
I20260709 13:18:27.545451 15861 tablet_bootstrap.cc:654] T 566eae6d93af4ec492b9b1a270dc03b7 P 62bbd271280e4b0a9ddf9ab46151e455: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:27.545993 15860 tablet_bootstrap.cc:492] T 566eae6d93af4ec492b9b1a270dc03b7 P 7cd7c680ef2e474c981ac81abadecfde: No bootstrap required, opened a new log
I20260709 13:18:27.546547 15860 ts_tablet_manager.cc:1403] T 566eae6d93af4ec492b9b1a270dc03b7 P 7cd7c680ef2e474c981ac81abadecfde: Time spent bootstrapping tablet: real 0.021s	user 0.009s	sys 0.009s
I20260709 13:18:27.547516 15862 tablet_bootstrap.cc:654] T 8cff7911abd649b9a276ddae431311d1 P 8c1822e697d64b758a23b11c8fdeb59a: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:27.549664 15860 raft_consensus.cc:359] T 566eae6d93af4ec492b9b1a270dc03b7 P 7cd7c680ef2e474c981ac81abadecfde [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c1822e697d64b758a23b11c8fdeb59a" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 39067 } } peers { permanent_uuid: "7cd7c680ef2e474c981ac81abadecfde" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 40117 } } peers { permanent_uuid: "62bbd271280e4b0a9ddf9ab46151e455" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 33917 } }
I20260709 13:18:27.550657 15860 raft_consensus.cc:385] T 566eae6d93af4ec492b9b1a270dc03b7 P 7cd7c680ef2e474c981ac81abadecfde [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:27.551100 15860 raft_consensus.cc:740] T 566eae6d93af4ec492b9b1a270dc03b7 P 7cd7c680ef2e474c981ac81abadecfde [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7cd7c680ef2e474c981ac81abadecfde, State: Initialized, Role: FOLLOWER
I20260709 13:18:27.552228 15860 consensus_queue.cc:260] T 566eae6d93af4ec492b9b1a270dc03b7 P 7cd7c680ef2e474c981ac81abadecfde [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c1822e697d64b758a23b11c8fdeb59a" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 39067 } } peers { permanent_uuid: "7cd7c680ef2e474c981ac81abadecfde" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 40117 } } peers { permanent_uuid: "62bbd271280e4b0a9ddf9ab46151e455" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 33917 } }
I20260709 13:18:27.558758 15860 ts_tablet_manager.cc:1434] T 566eae6d93af4ec492b9b1a270dc03b7 P 7cd7c680ef2e474c981ac81abadecfde: Time spent starting tablet: real 0.012s	user 0.003s	sys 0.003s
I20260709 13:18:27.559767 15820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ce7d20d574f47649d3ad0f3c382198d" candidate_uuid: "7cd7c680ef2e474c981ac81abadecfde" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "62bbd271280e4b0a9ddf9ab46151e455" is_pre_election: true
I20260709 13:18:27.560134 15860 tablet_bootstrap.cc:492] T 8fc26f7d12464423a212a1f595ca4b88 P 7cd7c680ef2e474c981ac81abadecfde: Bootstrap starting.
I20260709 13:18:27.560527 15820 raft_consensus.cc:2468] T 0ce7d20d574f47649d3ad0f3c382198d P 62bbd271280e4b0a9ddf9ab46151e455 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7cd7c680ef2e474c981ac81abadecfde in term 0.
I20260709 13:18:27.562053 15712 leader_election.cc:304] T 0ce7d20d574f47649d3ad0f3c382198d P 7cd7c680ef2e474c981ac81abadecfde [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 62bbd271280e4b0a9ddf9ab46151e455, 7cd7c680ef2e474c981ac81abadecfde; no voters: 
I20260709 13:18:27.563004 15867 raft_consensus.cc:2804] T 0ce7d20d574f47649d3ad0f3c382198d P 7cd7c680ef2e474c981ac81abadecfde [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:18:27.563433 15867 raft_consensus.cc:493] T 0ce7d20d574f47649d3ad0f3c382198d P 7cd7c680ef2e474c981ac81abadecfde [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:27.563805 15867 raft_consensus.cc:3060] T 0ce7d20d574f47649d3ad0f3c382198d P 7cd7c680ef2e474c981ac81abadecfde [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:27.570216 15860 tablet_bootstrap.cc:654] T 8fc26f7d12464423a212a1f595ca4b88 P 7cd7c680ef2e474c981ac81abadecfde: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:27.572839 15862 tablet_bootstrap.cc:492] T 8cff7911abd649b9a276ddae431311d1 P 8c1822e697d64b758a23b11c8fdeb59a: No bootstrap required, opened a new log
I20260709 13:18:27.573285 15867 raft_consensus.cc:515] T 0ce7d20d574f47649d3ad0f3c382198d P 7cd7c680ef2e474c981ac81abadecfde [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c1822e697d64b758a23b11c8fdeb59a" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 39067 } } peers { permanent_uuid: "7cd7c680ef2e474c981ac81abadecfde" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 40117 } } peers { permanent_uuid: "62bbd271280e4b0a9ddf9ab46151e455" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 33917 } }
I20260709 13:18:27.574673 15862 ts_tablet_manager.cc:1403] T 8cff7911abd649b9a276ddae431311d1 P 8c1822e697d64b758a23b11c8fdeb59a: Time spent bootstrapping tablet: real 0.035s	user 0.013s	sys 0.013s
I20260709 13:18:27.575847 15861 tablet_bootstrap.cc:492] T 566eae6d93af4ec492b9b1a270dc03b7 P 62bbd271280e4b0a9ddf9ab46151e455: No bootstrap required, opened a new log
I20260709 13:18:27.576280 15861 ts_tablet_manager.cc:1403] T 566eae6d93af4ec492b9b1a270dc03b7 P 62bbd271280e4b0a9ddf9ab46151e455: Time spent bootstrapping tablet: real 0.047s	user 0.022s	sys 0.004s
I20260709 13:18:27.576246 15671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ce7d20d574f47649d3ad0f3c382198d" candidate_uuid: "7cd7c680ef2e474c981ac81abadecfde" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c1822e697d64b758a23b11c8fdeb59a" is_pre_election: true
I20260709 13:18:27.577174 15671 raft_consensus.cc:2468] T 0ce7d20d574f47649d3ad0f3c382198d P 8c1822e697d64b758a23b11c8fdeb59a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7cd7c680ef2e474c981ac81abadecfde in term 0.
I20260709 13:18:27.577702 15862 raft_consensus.cc:359] T 8cff7911abd649b9a276ddae431311d1 P 8c1822e697d64b758a23b11c8fdeb59a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c1822e697d64b758a23b11c8fdeb59a" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 39067 } } peers { permanent_uuid: "7cd7c680ef2e474c981ac81abadecfde" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 40117 } } peers { permanent_uuid: "62bbd271280e4b0a9ddf9ab46151e455" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 33917 } }
I20260709 13:18:27.578117 15670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ce7d20d574f47649d3ad0f3c382198d" candidate_uuid: "7cd7c680ef2e474c981ac81abadecfde" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c1822e697d64b758a23b11c8fdeb59a"
I20260709 13:18:27.578408 15862 raft_consensus.cc:385] T 8cff7911abd649b9a276ddae431311d1 P 8c1822e697d64b758a23b11c8fdeb59a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:27.578593 15670 raft_consensus.cc:3060] T 0ce7d20d574f47649d3ad0f3c382198d P 8c1822e697d64b758a23b11c8fdeb59a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:27.578743 15862 raft_consensus.cc:740] T 8cff7911abd649b9a276ddae431311d1 P 8c1822e697d64b758a23b11c8fdeb59a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c1822e697d64b758a23b11c8fdeb59a, State: Initialized, Role: FOLLOWER
I20260709 13:18:27.579350 15861 raft_consensus.cc:359] T 566eae6d93af4ec492b9b1a270dc03b7 P 62bbd271280e4b0a9ddf9ab46151e455 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c1822e697d64b758a23b11c8fdeb59a" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 39067 } } peers { permanent_uuid: "7cd7c680ef2e474c981ac81abadecfde" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 40117 } } peers { permanent_uuid: "62bbd271280e4b0a9ddf9ab46151e455" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 33917 } }
I20260709 13:18:27.579476 15862 consensus_queue.cc:260] T 8cff7911abd649b9a276ddae431311d1 P 8c1822e697d64b758a23b11c8fdeb59a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c1822e697d64b758a23b11c8fdeb59a" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 39067 } } peers { permanent_uuid: "7cd7c680ef2e474c981ac81abadecfde" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 40117 } } peers { permanent_uuid: "62bbd271280e4b0a9ddf9ab46151e455" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 33917 } }
I20260709 13:18:27.580070 15861 raft_consensus.cc:385] T 566eae6d93af4ec492b9b1a270dc03b7 P 62bbd271280e4b0a9ddf9ab46151e455 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:27.580499 15861 raft_consensus.cc:740] T 566eae6d93af4ec492b9b1a270dc03b7 P 62bbd271280e4b0a9ddf9ab46151e455 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 62bbd271280e4b0a9ddf9ab46151e455, State: Initialized, Role: FOLLOWER
I20260709 13:18:27.580113 15867 leader_election.cc:290] T 0ce7d20d574f47649d3ad0f3c382198d P 7cd7c680ef2e474c981ac81abadecfde [CANDIDATE]: Term 1 election: Requested vote from peers 8c1822e697d64b758a23b11c8fdeb59a (127.14.177.129:39067), 62bbd271280e4b0a9ddf9ab46151e455 (127.14.177.131:33917)
I20260709 13:18:27.581712 15862 ts_tablet_manager.cc:1434] T 8cff7911abd649b9a276ddae431311d1 P 8c1822e697d64b758a23b11c8fdeb59a: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 13:18:27.581313 15861 consensus_queue.cc:260] T 566eae6d93af4ec492b9b1a270dc03b7 P 62bbd271280e4b0a9ddf9ab46151e455 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c1822e697d64b758a23b11c8fdeb59a" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 39067 } } peers { permanent_uuid: "7cd7c680ef2e474c981ac81abadecfde" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 40117 } } peers { permanent_uuid: "62bbd271280e4b0a9ddf9ab46151e455" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 33917 } }
I20260709 13:18:27.582849 15862 tablet_bootstrap.cc:492] T 566eae6d93af4ec492b9b1a270dc03b7 P 8c1822e697d64b758a23b11c8fdeb59a: Bootstrap starting.
I20260709 13:18:27.583865 15861 ts_tablet_manager.cc:1434] T 566eae6d93af4ec492b9b1a270dc03b7 P 62bbd271280e4b0a9ddf9ab46151e455: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 13:18:27.586844 15820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ce7d20d574f47649d3ad0f3c382198d" candidate_uuid: "7cd7c680ef2e474c981ac81abadecfde" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "62bbd271280e4b0a9ddf9ab46151e455"
I20260709 13:18:27.587759 15820 raft_consensus.cc:3060] T 0ce7d20d574f47649d3ad0f3c382198d P 62bbd271280e4b0a9ddf9ab46151e455 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:27.590059 15862 tablet_bootstrap.cc:654] T 566eae6d93af4ec492b9b1a270dc03b7 P 8c1822e697d64b758a23b11c8fdeb59a: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:27.599237 15868 raft_consensus.cc:493] T 8cff7911abd649b9a276ddae431311d1 P 8c1822e697d64b758a23b11c8fdeb59a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:27.599575 15820 raft_consensus.cc:2468] T 0ce7d20d574f47649d3ad0f3c382198d P 62bbd271280e4b0a9ddf9ab46151e455 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7cd7c680ef2e474c981ac81abadecfde in term 1.
I20260709 13:18:27.599800 15868 raft_consensus.cc:515] T 8cff7911abd649b9a276ddae431311d1 P 8c1822e697d64b758a23b11c8fdeb59a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c1822e697d64b758a23b11c8fdeb59a" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 39067 } } peers { permanent_uuid: "7cd7c680ef2e474c981ac81abadecfde" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 40117 } } peers { permanent_uuid: "62bbd271280e4b0a9ddf9ab46151e455" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 33917 } }
I20260709 13:18:27.601102 15712 leader_election.cc:304] T 0ce7d20d574f47649d3ad0f3c382198d P 7cd7c680ef2e474c981ac81abadecfde [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 62bbd271280e4b0a9ddf9ab46151e455, 7cd7c680ef2e474c981ac81abadecfde; no voters: 
I20260709 13:18:27.602286 15867 raft_consensus.cc:2804] T 0ce7d20d574f47649d3ad0f3c382198d P 7cd7c680ef2e474c981ac81abadecfde [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:18:27.610730 15861 tablet_bootstrap.cc:492] T 4d83a05b849340c186cc995649575d3b P 62bbd271280e4b0a9ddf9ab46151e455: Bootstrap starting.
I20260709 13:18:27.623265 15862 tablet_bootstrap.cc:492] T 566eae6d93af4ec492b9b1a270dc03b7 P 8c1822e697d64b758a23b11c8fdeb59a: No bootstrap required, opened a new log
I20260709 13:18:27.623816 15862 ts_tablet_manager.cc:1403] T 566eae6d93af4ec492b9b1a270dc03b7 P 8c1822e697d64b758a23b11c8fdeb59a: Time spent bootstrapping tablet: real 0.041s	user 0.007s	sys 0.012s
I20260709 13:18:27.624385 15670 raft_consensus.cc:2468] T 0ce7d20d574f47649d3ad0f3c382198d P 8c1822e697d64b758a23b11c8fdeb59a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7cd7c680ef2e474c981ac81abadecfde in term 1.
I20260709 13:18:27.607545 15868 leader_election.cc:290] T 8cff7911abd649b9a276ddae431311d1 P 8c1822e697d64b758a23b11c8fdeb59a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7cd7c680ef2e474c981ac81abadecfde (127.14.177.130:40117), 62bbd271280e4b0a9ddf9ab46151e455 (127.14.177.131:33917)
I20260709 13:18:27.637281 15886 raft_consensus.cc:493] T 8cff7911abd649b9a276ddae431311d1 P 7cd7c680ef2e474c981ac81abadecfde [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:27.635382 15867 raft_consensus.cc:697] T 0ce7d20d574f47649d3ad0f3c382198d P 7cd7c680ef2e474c981ac81abadecfde [term 1 LEADER]: Becoming Leader. State: Replica: 7cd7c680ef2e474c981ac81abadecfde, State: Running, Role: LEADER
I20260709 13:18:27.639467 15867 consensus_queue.cc:237] T 0ce7d20d574f47649d3ad0f3c382198d P 7cd7c680ef2e474c981ac81abadecfde [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c1822e697d64b758a23b11c8fdeb59a" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 39067 } } peers { permanent_uuid: "7cd7c680ef2e474c981ac81abadecfde" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 40117 } } peers { permanent_uuid: "62bbd271280e4b0a9ddf9ab46151e455" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 33917 } }
I20260709 13:18:27.642105 15860 tablet_bootstrap.cc:492] T 8fc26f7d12464423a212a1f595ca4b88 P 7cd7c680ef2e474c981ac81abadecfde: No bootstrap required, opened a new log
I20260709 13:18:27.642608 15860 ts_tablet_manager.cc:1403] T 8fc26f7d12464423a212a1f595ca4b88 P 7cd7c680ef2e474c981ac81abadecfde: Time spent bootstrapping tablet: real 0.083s	user 0.025s	sys 0.035s
I20260709 13:18:27.643653 15861 tablet_bootstrap.cc:654] T 4d83a05b849340c186cc995649575d3b P 62bbd271280e4b0a9ddf9ab46151e455: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:27.645262 15862 raft_consensus.cc:359] T 566eae6d93af4ec492b9b1a270dc03b7 P 8c1822e697d64b758a23b11c8fdeb59a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c1822e697d64b758a23b11c8fdeb59a" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 39067 } } peers { permanent_uuid: "7cd7c680ef2e474c981ac81abadecfde" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 40117 } } peers { permanent_uuid: "62bbd271280e4b0a9ddf9ab46151e455" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 33917 } }
I20260709 13:18:27.646063 15862 raft_consensus.cc:385] T 566eae6d93af4ec492b9b1a270dc03b7 P 8c1822e697d64b758a23b11c8fdeb59a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:27.646078 15860 raft_consensus.cc:359] T 8fc26f7d12464423a212a1f595ca4b88 P 7cd7c680ef2e474c981ac81abadecfde [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c1822e697d64b758a23b11c8fdeb59a" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 39067 } } peers { permanent_uuid: "7cd7c680ef2e474c981ac81abadecfde" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 40117 } } peers { permanent_uuid: "62bbd271280e4b0a9ddf9ab46151e455" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 33917 } }
I20260709 13:18:27.646826 15862 raft_consensus.cc:740] T 566eae6d93af4ec492b9b1a270dc03b7 P 8c1822e697d64b758a23b11c8fdeb59a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c1822e697d64b758a23b11c8fdeb59a, State: Initialized, Role: FOLLOWER
I20260709 13:18:27.647215 15860 raft_consensus.cc:385] T 8fc26f7d12464423a212a1f595ca4b88 P 7cd7c680ef2e474c981ac81abadecfde [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:27.647670 15860 raft_consensus.cc:740] T 8fc26f7d12464423a212a1f595ca4b88 P 7cd7c680ef2e474c981ac81abadecfde [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7cd7c680ef2e474c981ac81abadecfde, State: Initialized, Role: FOLLOWER
I20260709 13:18:27.647969 15862 consensus_queue.cc:260] T 566eae6d93af4ec492b9b1a270dc03b7 P 8c1822e697d64b758a23b11c8fdeb59a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c1822e697d64b758a23b11c8fdeb59a" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 39067 } } peers { permanent_uuid: "7cd7c680ef2e474c981ac81abadecfde" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 40117 } } peers { permanent_uuid: "62bbd271280e4b0a9ddf9ab46151e455" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 33917 } }
I20260709 13:18:27.648522 15860 consensus_queue.cc:260] T 8fc26f7d12464423a212a1f595ca4b88 P 7cd7c680ef2e474c981ac81abadecfde [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c1822e697d64b758a23b11c8fdeb59a" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 39067 } } peers { permanent_uuid: "7cd7c680ef2e474c981ac81abadecfde" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 40117 } } peers { permanent_uuid: "62bbd271280e4b0a9ddf9ab46151e455" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 33917 } }
I20260709 13:18:27.637760 15886 raft_consensus.cc:515] T 8cff7911abd649b9a276ddae431311d1 P 7cd7c680ef2e474c981ac81abadecfde [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c1822e697d64b758a23b11c8fdeb59a" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 39067 } } peers { permanent_uuid: "7cd7c680ef2e474c981ac81abadecfde" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 40117 } } peers { permanent_uuid: "62bbd271280e4b0a9ddf9ab46151e455" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 33917 } }
I20260709 13:18:27.653592 15860 ts_tablet_manager.cc:1434] T 8fc26f7d12464423a212a1f595ca4b88 P 7cd7c680ef2e474c981ac81abadecfde: Time spent starting tablet: real 0.011s	user 0.009s	sys 0.001s
I20260709 13:18:27.654594 15860 tablet_bootstrap.cc:492] T 17f97fe9a93f4a86b1e2bad621bfb561 P 7cd7c680ef2e474c981ac81abadecfde: Bootstrap starting.
I20260709 13:18:27.656566 15820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cff7911abd649b9a276ddae431311d1" candidate_uuid: "8c1822e697d64b758a23b11c8fdeb59a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "62bbd271280e4b0a9ddf9ab46151e455" is_pre_election: true
I20260709 13:18:27.657260 15820 raft_consensus.cc:2468] T 8cff7911abd649b9a276ddae431311d1 P 62bbd271280e4b0a9ddf9ab46151e455 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c1822e697d64b758a23b11c8fdeb59a in term 0.
I20260709 13:18:27.658365 15637 leader_election.cc:304] T 8cff7911abd649b9a276ddae431311d1 P 8c1822e697d64b758a23b11c8fdeb59a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 62bbd271280e4b0a9ddf9ab46151e455, 8c1822e697d64b758a23b11c8fdeb59a; no voters: 
I20260709 13:18:27.659440 15861 tablet_bootstrap.cc:492] T 4d83a05b849340c186cc995649575d3b P 62bbd271280e4b0a9ddf9ab46151e455: No bootstrap required, opened a new log
I20260709 13:18:27.659502 15862 ts_tablet_manager.cc:1434] T 566eae6d93af4ec492b9b1a270dc03b7 P 8c1822e697d64b758a23b11c8fdeb59a: Time spent starting tablet: real 0.035s	user 0.009s	sys 0.001s
I20260709 13:18:27.659896 15861 ts_tablet_manager.cc:1403] T 4d83a05b849340c186cc995649575d3b P 62bbd271280e4b0a9ddf9ab46151e455: Time spent bootstrapping tablet: real 0.050s	user 0.011s	sys 0.004s
I20260709 13:18:27.660513 15862 tablet_bootstrap.cc:492] T 8309d9a5a4ef4e6d914751bc0f410015 P 8c1822e697d64b758a23b11c8fdeb59a: Bootstrap starting.
I20260709 13:18:27.661829 15861 raft_consensus.cc:359] T 4d83a05b849340c186cc995649575d3b P 62bbd271280e4b0a9ddf9ab46151e455 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62bbd271280e4b0a9ddf9ab46151e455" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 33917 } } peers { permanent_uuid: "7cd7c680ef2e474c981ac81abadecfde" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 40117 } } peers { permanent_uuid: "8c1822e697d64b758a23b11c8fdeb59a" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 39067 } }
I20260709 13:18:27.662292 15861 raft_consensus.cc:385] T 4d83a05b849340c186cc995649575d3b P 62bbd271280e4b0a9ddf9ab46151e455 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:27.662467 15861 raft_consensus.cc:740] T 4d83a05b849340c186cc995649575d3b P 62bbd271280e4b0a9ddf9ab46151e455 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 62bbd271280e4b0a9ddf9ab46151e455, State: Initialized, Role: FOLLOWER
I20260709 13:18:27.663021 15861 consensus_queue.cc:260] T 4d83a05b849340c186cc995649575d3b P 62bbd271280e4b0a9ddf9ab46151e455 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62bbd271280e4b0a9ddf9ab46151e455" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 33917 } } peers { permanent_uuid: "7cd7c680ef2e474c981ac81abadecfde" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 40117 } } peers { permanent_uuid: "8c1822e697d64b758a23b11c8fdeb59a" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 39067 } }
I20260709 13:18:27.665006 15670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cff7911abd649b9a276ddae431311d1" candidate_uuid: "7cd7c680ef2e474c981ac81abadecfde" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c1822e697d64b758a23b11c8fdeb59a" is_pre_election: true
I20260709 13:18:27.665987 15569 catalog_manager.cc:5697] T 0ce7d20d574f47649d3ad0f3c382198d P 7cd7c680ef2e474c981ac81abadecfde reported cstate change: term changed from 0 to 1, leader changed from <none> to 7cd7c680ef2e474c981ac81abadecfde (127.14.177.130). New cstate: current_term: 1 leader_uuid: "7cd7c680ef2e474c981ac81abadecfde" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c1822e697d64b758a23b11c8fdeb59a" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 39067 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7cd7c680ef2e474c981ac81abadecfde" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 40117 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "62bbd271280e4b0a9ddf9ab46151e455" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 33917 } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:27.667028 15862 tablet_bootstrap.cc:654] T 8309d9a5a4ef4e6d914751bc0f410015 P 8c1822e697d64b758a23b11c8fdeb59a: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:27.667663 15886 leader_election.cc:290] T 8cff7911abd649b9a276ddae431311d1 P 7cd7c680ef2e474c981ac81abadecfde [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8c1822e697d64b758a23b11c8fdeb59a (127.14.177.129:39067), 62bbd271280e4b0a9ddf9ab46151e455 (127.14.177.131:33917)
I20260709 13:18:27.668869 15820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cff7911abd649b9a276ddae431311d1" candidate_uuid: "7cd7c680ef2e474c981ac81abadecfde" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "62bbd271280e4b0a9ddf9ab46151e455" is_pre_election: true
I20260709 13:18:27.669499 15820 raft_consensus.cc:2468] T 8cff7911abd649b9a276ddae431311d1 P 62bbd271280e4b0a9ddf9ab46151e455 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7cd7c680ef2e474c981ac81abadecfde in term 0.
I20260709 13:18:27.659382 15868 raft_consensus.cc:2804] T 8cff7911abd649b9a276ddae431311d1 P 8c1822e697d64b758a23b11c8fdeb59a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:18:27.671180 15868 raft_consensus.cc:493] T 8cff7911abd649b9a276ddae431311d1 P 8c1822e697d64b758a23b11c8fdeb59a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:27.671077 15712 leader_election.cc:304] T 8cff7911abd649b9a276ddae431311d1 P 7cd7c680ef2e474c981ac81abadecfde [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 62bbd271280e4b0a9ddf9ab46151e455, 7cd7c680ef2e474c981ac81abadecfde; no voters: 
I20260709 13:18:27.671798 15868 raft_consensus.cc:3060] T 8cff7911abd649b9a276ddae431311d1 P 8c1822e697d64b758a23b11c8fdeb59a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:27.672127 15861 ts_tablet_manager.cc:1434] T 4d83a05b849340c186cc995649575d3b P 62bbd271280e4b0a9ddf9ab46151e455: Time spent starting tablet: real 0.012s	user 0.002s	sys 0.003s
I20260709 13:18:27.672396 15886 raft_consensus.cc:2804] T 8cff7911abd649b9a276ddae431311d1 P 7cd7c680ef2e474c981ac81abadecfde [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:18:27.672717 15886 raft_consensus.cc:493] T 8cff7911abd649b9a276ddae431311d1 P 7cd7c680ef2e474c981ac81abadecfde [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:27.673053 15886 raft_consensus.cc:3060] T 8cff7911abd649b9a276ddae431311d1 P 7cd7c680ef2e474c981ac81abadecfde [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:27.673134 15861 tablet_bootstrap.cc:492] T 408866bb47904e4189bc235ca5b0c7c7 P 62bbd271280e4b0a9ddf9ab46151e455: Bootstrap starting.
I20260709 13:18:27.682127 15886 raft_consensus.cc:515] T 8cff7911abd649b9a276ddae431311d1 P 7cd7c680ef2e474c981ac81abadecfde [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c1822e697d64b758a23b11c8fdeb59a" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 39067 } } peers { permanent_uuid: "7cd7c680ef2e474c981ac81abadecfde" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 40117 } } peers { permanent_uuid: "62bbd271280e4b0a9ddf9ab46151e455" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 33917 } }
I20260709 13:18:27.685413 15671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cff7911abd649b9a276ddae431311d1" candidate_uuid: "7cd7c680ef2e474c981ac81abadecfde" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c1822e697d64b758a23b11c8fdeb59a"
I20260709 13:18:27.686993 15746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cff7911abd649b9a276ddae431311d1" candidate_uuid: "8c1822e697d64b758a23b11c8fdeb59a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cd7c680ef2e474c981ac81abadecfde" is_pre_election: true
I20260709 13:18:27.687928 15746 raft_consensus.cc:2393] T 8cff7911abd649b9a276ddae431311d1 P 7cd7c680ef2e474c981ac81abadecfde [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c1822e697d64b758a23b11c8fdeb59a in current term 1: Already voted for candidate 7cd7c680ef2e474c981ac81abadecfde in this term.
I20260709 13:18:27.690874 15886 leader_election.cc:290] T 8cff7911abd649b9a276ddae431311d1 P 7cd7c680ef2e474c981ac81abadecfde [CANDIDATE]: Term 1 election: Requested vote from peers 8c1822e697d64b758a23b11c8fdeb59a (127.14.177.129:39067), 62bbd271280e4b0a9ddf9ab46151e455 (127.14.177.131:33917)
I20260709 13:18:27.691869 15820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cff7911abd649b9a276ddae431311d1" candidate_uuid: "7cd7c680ef2e474c981ac81abadecfde" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "62bbd271280e4b0a9ddf9ab46151e455"
I20260709 13:18:27.692559 15820 raft_consensus.cc:3060] T 8cff7911abd649b9a276ddae431311d1 P 62bbd271280e4b0a9ddf9ab46151e455 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:18:27.696561 15617 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:18:27.698470 15868 raft_consensus.cc:515] T 8cff7911abd649b9a276ddae431311d1 P 8c1822e697d64b758a23b11c8fdeb59a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c1822e697d64b758a23b11c8fdeb59a" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 39067 } } peers { permanent_uuid: "7cd7c680ef2e474c981ac81abadecfde" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 40117 } } peers { permanent_uuid: "62bbd271280e4b0a9ddf9ab46151e455" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 33917 } }
I20260709 13:18:27.699863 15670 raft_consensus.cc:2393] T 8cff7911abd649b9a276ddae431311d1 P 8c1822e697d64b758a23b11c8fdeb59a [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7cd7c680ef2e474c981ac81abadecfde in current term 1: Already voted for candidate 8c1822e697d64b758a23b11c8fdeb59a in this term.
I20260709 13:18:27.700179 15861 tablet_bootstrap.cc:654] T 408866bb47904e4189bc235ca5b0c7c7 P 62bbd271280e4b0a9ddf9ab46151e455: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:27.703996 15820 raft_consensus.cc:2468] T 8cff7911abd649b9a276ddae431311d1 P 62bbd271280e4b0a9ddf9ab46151e455 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7cd7c680ef2e474c981ac81abadecfde in term 1.
I20260709 13:18:27.704576 15860 tablet_bootstrap.cc:654] T 17f97fe9a93f4a86b1e2bad621bfb561 P 7cd7c680ef2e474c981ac81abadecfde: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:27.707906 15862 tablet_bootstrap.cc:492] T 8309d9a5a4ef4e6d914751bc0f410015 P 8c1822e697d64b758a23b11c8fdeb59a: No bootstrap required, opened a new log
I20260709 13:18:27.708076 15746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cff7911abd649b9a276ddae431311d1" candidate_uuid: "8c1822e697d64b758a23b11c8fdeb59a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cd7c680ef2e474c981ac81abadecfde"
I20260709 13:18:27.708375 15862 ts_tablet_manager.cc:1403] T 8309d9a5a4ef4e6d914751bc0f410015 P 8c1822e697d64b758a23b11c8fdeb59a: Time spent bootstrapping tablet: real 0.048s	user 0.016s	sys 0.029s
I20260709 13:18:27.708453 15712 leader_election.cc:304] T 8cff7911abd649b9a276ddae431311d1 P 7cd7c680ef2e474c981ac81abadecfde [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 62bbd271280e4b0a9ddf9ab46151e455, 7cd7c680ef2e474c981ac81abadecfde; no voters: 8c1822e697d64b758a23b11c8fdeb59a
I20260709 13:18:27.708974 15746 raft_consensus.cc:2393] T 8cff7911abd649b9a276ddae431311d1 P 7cd7c680ef2e474c981ac81abadecfde [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 8c1822e697d64b758a23b11c8fdeb59a in current term 1: Already voted for candidate 7cd7c680ef2e474c981ac81abadecfde in this term.
I20260709 13:18:27.709118 15861 tablet_bootstrap.cc:492] T 408866bb47904e4189bc235ca5b0c7c7 P 62bbd271280e4b0a9ddf9ab46151e455: No bootstrap required, opened a new log
I20260709 13:18:27.709561 15861 ts_tablet_manager.cc:1403] T 408866bb47904e4189bc235ca5b0c7c7 P 62bbd271280e4b0a9ddf9ab46151e455: Time spent bootstrapping tablet: real 0.037s	user 0.008s	sys 0.003s
I20260709 13:18:27.710561 15886 raft_consensus.cc:2804] T 8cff7911abd649b9a276ddae431311d1 P 7cd7c680ef2e474c981ac81abadecfde [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:18:27.711378 15886 raft_consensus.cc:697] T 8cff7911abd649b9a276ddae431311d1 P 7cd7c680ef2e474c981ac81abadecfde [term 1 LEADER]: Becoming Leader. State: Replica: 7cd7c680ef2e474c981ac81abadecfde, State: Running, Role: LEADER
I20260709 13:18:27.712028 15862 raft_consensus.cc:359] T 8309d9a5a4ef4e6d914751bc0f410015 P 8c1822e697d64b758a23b11c8fdeb59a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c1822e697d64b758a23b11c8fdeb59a" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 39067 } } peers { permanent_uuid: "7cd7c680ef2e474c981ac81abadecfde" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 40117 } } peers { permanent_uuid: "62bbd271280e4b0a9ddf9ab46151e455" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 33917 } }
I20260709 13:18:27.712306 15886 consensus_queue.cc:237] T 8cff7911abd649b9a276ddae431311d1 P 7cd7c680ef2e474c981ac81abadecfde [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c1822e697d64b758a23b11c8fdeb59a" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 39067 } } peers { permanent_uuid: "7cd7c680ef2e474c981ac81abadecfde" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 40117 } } peers { permanent_uuid: "62bbd271280e4b0a9ddf9ab46151e455" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 33917 } }
I20260709 13:18:27.713027 15862 raft_consensus.cc:385] T 8309d9a5a4ef4e6d914751bc0f410015 P 8c1822e697d64b758a23b11c8fdeb59a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:27.713460 15862 raft_consensus.cc:740] T 8309d9a5a4ef4e6d914751bc0f410015 P 8c1822e697d64b758a23b11c8fdeb59a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c1822e697d64b758a23b11c8fdeb59a, State: Initialized, Role: FOLLOWER
I20260709 13:18:27.712744 15861 raft_consensus.cc:359] T 408866bb47904e4189bc235ca5b0c7c7 P 62bbd271280e4b0a9ddf9ab46151e455 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c1822e697d64b758a23b11c8fdeb59a" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 39067 } } peers { permanent_uuid: "7cd7c680ef2e474c981ac81abadecfde" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 40117 } } peers { permanent_uuid: "62bbd271280e4b0a9ddf9ab46151e455" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 33917 } }
I20260709 13:18:27.714395 15861 raft_consensus.cc:385] T 408866bb47904e4189bc235ca5b0c7c7 P 62bbd271280e4b0a9ddf9ab46151e455 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:27.714320 15862 consensus_queue.cc:260] T 8309d9a5a4ef4e6d914751bc0f410015 P 8c1822e697d64b758a23b11c8fdeb59a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c1822e697d64b758a23b11c8fdeb59a" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 39067 } } peers { permanent_uuid: "7cd7c680ef2e474c981ac81abadecfde" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 40117 } } peers { permanent_uuid: "62bbd271280e4b0a9ddf9ab46151e455" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 33917 } }
I20260709 13:18:27.714798 15861 raft_consensus.cc:740] T 408866bb47904e4189bc235ca5b0c7c7 P 62bbd271280e4b0a9ddf9ab46151e455 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 62bbd271280e4b0a9ddf9ab46151e455, State: Initialized, Role: FOLLOWER
I20260709 13:18:27.716023 15861 consensus_queue.cc:260] T 408866bb47904e4189bc235ca5b0c7c7 P 62bbd271280e4b0a9ddf9ab46151e455 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c1822e697d64b758a23b11c8fdeb59a" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 39067 } } peers { permanent_uuid: "7cd7c680ef2e474c981ac81abadecfde" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 40117 } } peers { permanent_uuid: "62bbd271280e4b0a9ddf9ab46151e455" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 33917 } }
I20260709 13:18:27.716575 15868 leader_election.cc:290] T 8cff7911abd649b9a276ddae431311d1 P 8c1822e697d64b758a23b11c8fdeb59a [CANDIDATE]: Term 1 election: Requested vote from peers 7cd7c680ef2e474c981ac81abadecfde (127.14.177.130:40117), 62bbd271280e4b0a9ddf9ab46151e455 (127.14.177.131:33917)
I20260709 13:18:27.717280 15862 ts_tablet_manager.cc:1434] T 8309d9a5a4ef4e6d914751bc0f410015 P 8c1822e697d64b758a23b11c8fdeb59a: Time spent starting tablet: real 0.009s	user 0.009s	sys 0.000s
I20260709 13:18:27.718199 15862 tablet_bootstrap.cc:492] T 17f97fe9a93f4a86b1e2bad621bfb561 P 8c1822e697d64b758a23b11c8fdeb59a: Bootstrap starting.
I20260709 13:18:27.720078 15820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cff7911abd649b9a276ddae431311d1" candidate_uuid: "8c1822e697d64b758a23b11c8fdeb59a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "62bbd271280e4b0a9ddf9ab46151e455"
I20260709 13:18:27.720850 15820 raft_consensus.cc:2393] T 8cff7911abd649b9a276ddae431311d1 P 62bbd271280e4b0a9ddf9ab46151e455 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 8c1822e697d64b758a23b11c8fdeb59a in current term 1: Already voted for candidate 7cd7c680ef2e474c981ac81abadecfde in this term.
I20260709 13:18:27.722257 15637 leader_election.cc:304] T 8cff7911abd649b9a276ddae431311d1 P 8c1822e697d64b758a23b11c8fdeb59a [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c1822e697d64b758a23b11c8fdeb59a; no voters: 62bbd271280e4b0a9ddf9ab46151e455, 7cd7c680ef2e474c981ac81abadecfde
I20260709 13:18:27.723749 15893 raft_consensus.cc:2749] T 8cff7911abd649b9a276ddae431311d1 P 8c1822e697d64b758a23b11c8fdeb59a [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:18:27.724653 15862 tablet_bootstrap.cc:654] T 17f97fe9a93f4a86b1e2bad621bfb561 P 8c1822e697d64b758a23b11c8fdeb59a: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:27.729051 15862 tablet_bootstrap.cc:492] T 17f97fe9a93f4a86b1e2bad621bfb561 P 8c1822e697d64b758a23b11c8fdeb59a: No bootstrap required, opened a new log
I20260709 13:18:27.729451 15862 ts_tablet_manager.cc:1403] T 17f97fe9a93f4a86b1e2bad621bfb561 P 8c1822e697d64b758a23b11c8fdeb59a: Time spent bootstrapping tablet: real 0.011s	user 0.007s	sys 0.003s
I20260709 13:18:27.731876 15862 raft_consensus.cc:359] T 17f97fe9a93f4a86b1e2bad621bfb561 P 8c1822e697d64b758a23b11c8fdeb59a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c1822e697d64b758a23b11c8fdeb59a" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 39067 } } peers { permanent_uuid: "7cd7c680ef2e474c981ac81abadecfde" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 40117 } } peers { permanent_uuid: "62bbd271280e4b0a9ddf9ab46151e455" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 33917 } }
I20260709 13:18:27.732381 15862 raft_consensus.cc:385] T 17f97fe9a93f4a86b1e2bad621bfb561 P 8c1822e697d64b758a23b11c8fdeb59a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:27.732649 15862 raft_consensus.cc:740] T 17f97fe9a93f4a86b1e2bad621bfb561 P 8c1822e697d64b758a23b11c8fdeb59a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c1822e697d64b758a23b11c8fdeb59a, State: Initialized, Role: FOLLOWER
I20260709 13:18:27.733258 15862 consensus_queue.cc:260] T 17f97fe9a93f4a86b1e2bad621bfb561 P 8c1822e697d64b758a23b11c8fdeb59a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c1822e697d64b758a23b11c8fdeb59a" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 39067 } } peers { permanent_uuid: "7cd7c680ef2e474c981ac81abadecfde" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 40117 } } peers { permanent_uuid: "62bbd271280e4b0a9ddf9ab46151e455" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 33917 } }
I20260709 13:18:27.734678 15862 ts_tablet_manager.cc:1434] T 17f97fe9a93f4a86b1e2bad621bfb561 P 8c1822e697d64b758a23b11c8fdeb59a: Time spent starting tablet: real 0.005s	user 0.003s	sys 0.000s
I20260709 13:18:27.735517 15862 tablet_bootstrap.cc:492] T 48e8b6e82d7b42e6bdd4b0a8a25dfd9e P 8c1822e697d64b758a23b11c8fdeb59a: Bootstrap starting.
I20260709 13:18:27.735579 15861 ts_tablet_manager.cc:1434] T 408866bb47904e4189bc235ca5b0c7c7 P 62bbd271280e4b0a9ddf9ab46151e455: Time spent starting tablet: real 0.026s	user 0.010s	sys 0.000s
I20260709 13:18:27.736187 15860 tablet_bootstrap.cc:492] T 17f97fe9a93f4a86b1e2bad621bfb561 P 7cd7c680ef2e474c981ac81abadecfde: No bootstrap required, opened a new log
I20260709 13:18:27.736814 15860 ts_tablet_manager.cc:1403] T 17f97fe9a93f4a86b1e2bad621bfb561 P 7cd7c680ef2e474c981ac81abadecfde: Time spent bootstrapping tablet: real 0.082s	user 0.012s	sys 0.003s
I20260709 13:18:27.740072 15861 tablet_bootstrap.cc:492] T 8309d9a5a4ef4e6d914751bc0f410015 P 62bbd271280e4b0a9ddf9ab46151e455: Bootstrap starting.
I20260709 13:18:27.741492 15862 tablet_bootstrap.cc:654] T 48e8b6e82d7b42e6bdd4b0a8a25dfd9e P 8c1822e697d64b758a23b11c8fdeb59a: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:27.740995 15860 raft_consensus.cc:359] T 17f97fe9a93f4a86b1e2bad621bfb561 P 7cd7c680ef2e474c981ac81abadecfde [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c1822e697d64b758a23b11c8fdeb59a" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 39067 } } peers { permanent_uuid: "7cd7c680ef2e474c981ac81abadecfde" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 40117 } } peers { permanent_uuid: "62bbd271280e4b0a9ddf9ab46151e455" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 33917 } }
I20260709 13:18:27.741923 15860 raft_consensus.cc:385] T 17f97fe9a93f4a86b1e2bad621bfb561 P 7cd7c680ef2e474c981ac81abadecfde [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:27.742292 15860 raft_consensus.cc:740] T 17f97fe9a93f4a86b1e2bad621bfb561 P 7cd7c680ef2e474c981ac81abadecfde [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7cd7c680ef2e474c981ac81abadecfde, State: Initialized, Role: FOLLOWER
I20260709 13:18:27.743443 15860 consensus_queue.cc:260] T 17f97fe9a93f4a86b1e2bad621bfb561 P 7cd7c680ef2e474c981ac81abadecfde [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c1822e697d64b758a23b11c8fdeb59a" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 39067 } } peers { permanent_uuid: "7cd7c680ef2e474c981ac81abadecfde" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 40117 } } peers { permanent_uuid: "62bbd271280e4b0a9ddf9ab46151e455" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 33917 } }
I20260709 13:18:27.746299 15860 ts_tablet_manager.cc:1434] T 17f97fe9a93f4a86b1e2bad621bfb561 P 7cd7c680ef2e474c981ac81abadecfde: Time spent starting tablet: real 0.009s	user 0.009s	sys 0.000s
I20260709 13:18:27.747385 15860 tablet_bootstrap.cc:492] T 8309d9a5a4ef4e6d914751bc0f410015 P 7cd7c680ef2e474c981ac81abadecfde: Bootstrap starting.
I20260709 13:18:27.751685 15866 raft_consensus.cc:493] T 8fc26f7d12464423a212a1f595ca4b88 P 62bbd271280e4b0a9ddf9ab46151e455 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:27.751897 15861 tablet_bootstrap.cc:654] T 8309d9a5a4ef4e6d914751bc0f410015 P 62bbd271280e4b0a9ddf9ab46151e455: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:27.752251 15866 raft_consensus.cc:515] T 8fc26f7d12464423a212a1f595ca4b88 P 62bbd271280e4b0a9ddf9ab46151e455 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c1822e697d64b758a23b11c8fdeb59a" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 39067 } } peers { permanent_uuid: "7cd7c680ef2e474c981ac81abadecfde" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 40117 } } peers { permanent_uuid: "62bbd271280e4b0a9ddf9ab46151e455" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 33917 } }
I20260709 13:18:27.758765 15868 raft_consensus.cc:493] T 4d83a05b849340c186cc995649575d3b P 8c1822e697d64b758a23b11c8fdeb59a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:27.759289 15868 raft_consensus.cc:515] T 4d83a05b849340c186cc995649575d3b P 8c1822e697d64b758a23b11c8fdeb59a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62bbd271280e4b0a9ddf9ab46151e455" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 33917 } } peers { permanent_uuid: "7cd7c680ef2e474c981ac81abadecfde" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 40117 } } peers { permanent_uuid: "8c1822e697d64b758a23b11c8fdeb59a" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 39067 } }
I20260709 13:18:27.761334 15868 leader_election.cc:290] T 4d83a05b849340c186cc995649575d3b P 8c1822e697d64b758a23b11c8fdeb59a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 62bbd271280e4b0a9ddf9ab46151e455 (127.14.177.131:33917), 7cd7c680ef2e474c981ac81abadecfde (127.14.177.130:40117)
I20260709 13:18:27.763058 15820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d83a05b849340c186cc995649575d3b" candidate_uuid: "8c1822e697d64b758a23b11c8fdeb59a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "62bbd271280e4b0a9ddf9ab46151e455" is_pre_election: true
I20260709 13:18:27.763868 15820 raft_consensus.cc:2468] T 4d83a05b849340c186cc995649575d3b P 62bbd271280e4b0a9ddf9ab46151e455 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c1822e697d64b758a23b11c8fdeb59a in term 0.
I20260709 13:18:27.763942 15746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d83a05b849340c186cc995649575d3b" candidate_uuid: "8c1822e697d64b758a23b11c8fdeb59a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cd7c680ef2e474c981ac81abadecfde" is_pre_election: true
I20260709 13:18:27.764773 15746 raft_consensus.cc:2468] T 4d83a05b849340c186cc995649575d3b P 7cd7c680ef2e474c981ac81abadecfde [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c1822e697d64b758a23b11c8fdeb59a in term 0.
I20260709 13:18:27.766252 15867 raft_consensus.cc:493] T 4d83a05b849340c186cc995649575d3b P 7cd7c680ef2e474c981ac81abadecfde [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:27.767647 15636 leader_election.cc:304] T 4d83a05b849340c186cc995649575d3b P 8c1822e697d64b758a23b11c8fdeb59a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7cd7c680ef2e474c981ac81abadecfde, 8c1822e697d64b758a23b11c8fdeb59a; no voters: 
I20260709 13:18:27.769627 15866 leader_election.cc:290] T 8fc26f7d12464423a212a1f595ca4b88 P 62bbd271280e4b0a9ddf9ab46151e455 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8c1822e697d64b758a23b11c8fdeb59a (127.14.177.129:39067), 7cd7c680ef2e474c981ac81abadecfde (127.14.177.130:40117)
I20260709 13:18:27.769835 15861 tablet_bootstrap.cc:492] T 8309d9a5a4ef4e6d914751bc0f410015 P 62bbd271280e4b0a9ddf9ab46151e455: No bootstrap required, opened a new log
I20260709 13:18:27.766808 15867 raft_consensus.cc:515] T 4d83a05b849340c186cc995649575d3b P 7cd7c680ef2e474c981ac81abadecfde [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62bbd271280e4b0a9ddf9ab46151e455" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 33917 } } peers { permanent_uuid: "7cd7c680ef2e474c981ac81abadecfde" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 40117 } } peers { permanent_uuid: "8c1822e697d64b758a23b11c8fdeb59a" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 39067 } }
I20260709 13:18:27.771831 15861 ts_tablet_manager.cc:1403] T 8309d9a5a4ef4e6d914751bc0f410015 P 62bbd271280e4b0a9ddf9ab46151e455: Time spent bootstrapping tablet: real 0.032s	user 0.016s	sys 0.005s
I20260709 13:18:27.774189 15670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d83a05b849340c186cc995649575d3b" candidate_uuid: "7cd7c680ef2e474c981ac81abadecfde" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c1822e697d64b758a23b11c8fdeb59a" is_pre_election: true
I20260709 13:18:27.774889 15670 raft_consensus.cc:2468] T 4d83a05b849340c186cc995649575d3b P 8c1822e697d64b758a23b11c8fdeb59a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7cd7c680ef2e474c981ac81abadecfde in term 0.
I20260709 13:18:27.776269 15709 leader_election.cc:304] T 4d83a05b849340c186cc995649575d3b P 7cd7c680ef2e474c981ac81abadecfde [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7cd7c680ef2e474c981ac81abadecfde, 8c1822e697d64b758a23b11c8fdeb59a; no voters: 
I20260709 13:18:27.777153 15884 raft_consensus.cc:493] T 566eae6d93af4ec492b9b1a270dc03b7 P 7cd7c680ef2e474c981ac81abadecfde [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:27.777325 15868 raft_consensus.cc:2804] T 4d83a05b849340c186cc995649575d3b P 8c1822e697d64b758a23b11c8fdeb59a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:18:27.777904 15868 raft_consensus.cc:493] T 4d83a05b849340c186cc995649575d3b P 8c1822e697d64b758a23b11c8fdeb59a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:27.777630 15884 raft_consensus.cc:515] T 566eae6d93af4ec492b9b1a270dc03b7 P 7cd7c680ef2e474c981ac81abadecfde [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c1822e697d64b758a23b11c8fdeb59a" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 39067 } } peers { permanent_uuid: "7cd7c680ef2e474c981ac81abadecfde" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 40117 } } peers { permanent_uuid: "62bbd271280e4b0a9ddf9ab46151e455" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 33917 } }
I20260709 13:18:27.778367 15868 raft_consensus.cc:3060] T 4d83a05b849340c186cc995649575d3b P 8c1822e697d64b758a23b11c8fdeb59a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:27.780045 15884 leader_election.cc:290] T 566eae6d93af4ec492b9b1a270dc03b7 P 7cd7c680ef2e474c981ac81abadecfde [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8c1822e697d64b758a23b11c8fdeb59a (127.14.177.129:39067), 62bbd271280e4b0a9ddf9ab46151e455 (127.14.177.131:33917)
I20260709 13:18:27.780561 15884 raft_consensus.cc:2804] T 4d83a05b849340c186cc995649575d3b P 7cd7c680ef2e474c981ac81abadecfde [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:18:27.780818 15867 leader_election.cc:290] T 4d83a05b849340c186cc995649575d3b P 7cd7c680ef2e474c981ac81abadecfde [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 62bbd271280e4b0a9ddf9ab46151e455 (127.14.177.131:33917), 8c1822e697d64b758a23b11c8fdeb59a (127.14.177.129:39067)
I20260709 13:18:27.780893 15884 raft_consensus.cc:493] T 4d83a05b849340c186cc995649575d3b P 7cd7c680ef2e474c981ac81abadecfde [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:27.776682 15860 tablet_bootstrap.cc:654] T 8309d9a5a4ef4e6d914751bc0f410015 P 7cd7c680ef2e474c981ac81abadecfde: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:27.781461 15884 raft_consensus.cc:3060] T 4d83a05b849340c186cc995649575d3b P 7cd7c680ef2e474c981ac81abadecfde [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:27.782721 15670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "566eae6d93af4ec492b9b1a270dc03b7" candidate_uuid: "7cd7c680ef2e474c981ac81abadecfde" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c1822e697d64b758a23b11c8fdeb59a" is_pre_election: true
I20260709 13:18:27.783396 15670 raft_consensus.cc:2468] T 566eae6d93af4ec492b9b1a270dc03b7 P 8c1822e697d64b758a23b11c8fdeb59a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7cd7c680ef2e474c981ac81abadecfde in term 0.
I20260709 13:18:27.784595 15820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d83a05b849340c186cc995649575d3b" candidate_uuid: "7cd7c680ef2e474c981ac81abadecfde" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "62bbd271280e4b0a9ddf9ab46151e455" is_pre_election: true
I20260709 13:18:27.784672 15709 leader_election.cc:304] T 566eae6d93af4ec492b9b1a270dc03b7 P 7cd7c680ef2e474c981ac81abadecfde [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7cd7c680ef2e474c981ac81abadecfde, 8c1822e697d64b758a23b11c8fdeb59a; no voters: 
I20260709 13:18:27.785257 15820 raft_consensus.cc:2468] T 4d83a05b849340c186cc995649575d3b P 62bbd271280e4b0a9ddf9ab46151e455 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7cd7c680ef2e474c981ac81abadecfde in term 0.
I20260709 13:18:27.785549 15867 raft_consensus.cc:2804] T 566eae6d93af4ec492b9b1a270dc03b7 P 7cd7c680ef2e474c981ac81abadecfde [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:18:27.785903 15867 raft_consensus.cc:493] T 566eae6d93af4ec492b9b1a270dc03b7 P 7cd7c680ef2e474c981ac81abadecfde [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:27.786224 15867 raft_consensus.cc:3060] T 566eae6d93af4ec492b9b1a270dc03b7 P 7cd7c680ef2e474c981ac81abadecfde [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:27.786301 15819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "566eae6d93af4ec492b9b1a270dc03b7" candidate_uuid: "7cd7c680ef2e474c981ac81abadecfde" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "62bbd271280e4b0a9ddf9ab46151e455" is_pre_election: true
I20260709 13:18:27.787005 15819 raft_consensus.cc:2468] T 566eae6d93af4ec492b9b1a270dc03b7 P 62bbd271280e4b0a9ddf9ab46151e455 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7cd7c680ef2e474c981ac81abadecfde in term 0.
I20260709 13:18:27.783638 15861 raft_consensus.cc:359] T 8309d9a5a4ef4e6d914751bc0f410015 P 62bbd271280e4b0a9ddf9ab46151e455 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c1822e697d64b758a23b11c8fdeb59a" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 39067 } } peers { permanent_uuid: "7cd7c680ef2e474c981ac81abadecfde" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 40117 } } peers { permanent_uuid: "62bbd271280e4b0a9ddf9ab46151e455" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 33917 } }
I20260709 13:18:27.790978 15861 raft_consensus.cc:385] T 8309d9a5a4ef4e6d914751bc0f410015 P 62bbd271280e4b0a9ddf9ab46151e455 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:27.791404 15861 raft_consensus.cc:740] T 8309d9a5a4ef4e6d914751bc0f410015 P 62bbd271280e4b0a9ddf9ab46151e455 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 62bbd271280e4b0a9ddf9ab46151e455, State: Initialized, Role: FOLLOWER
I20260709 13:18:27.792241 15861 consensus_queue.cc:260] T 8309d9a5a4ef4e6d914751bc0f410015 P 62bbd271280e4b0a9ddf9ab46151e455 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c1822e697d64b758a23b11c8fdeb59a" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 39067 } } peers { permanent_uuid: "7cd7c680ef2e474c981ac81abadecfde" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 40117 } } peers { permanent_uuid: "62bbd271280e4b0a9ddf9ab46151e455" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 33917 } }
I20260709 13:18:27.793255 15867 raft_consensus.cc:515] T 566eae6d93af4ec492b9b1a270dc03b7 P 7cd7c680ef2e474c981ac81abadecfde [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c1822e697d64b758a23b11c8fdeb59a" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 39067 } } peers { permanent_uuid: "7cd7c680ef2e474c981ac81abadecfde" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 40117 } } peers { permanent_uuid: "62bbd271280e4b0a9ddf9ab46151e455" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 33917 } }
I20260709 13:18:27.796773 15670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc26f7d12464423a212a1f595ca4b88" candidate_uuid: "62bbd271280e4b0a9ddf9ab46151e455" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c1822e697d64b758a23b11c8fdeb59a" is_pre_election: true
I20260709 13:18:27.797274 15861 ts_tablet_manager.cc:1434] T 8309d9a5a4ef4e6d914751bc0f410015 P 62bbd271280e4b0a9ddf9ab46151e455: Time spent starting tablet: real 0.023s	user 0.006s	sys 0.000s
I20260709 13:18:27.797432 15670 raft_consensus.cc:2468] T 8fc26f7d12464423a212a1f595ca4b88 P 8c1822e697d64b758a23b11c8fdeb59a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 62bbd271280e4b0a9ddf9ab46151e455 in term 0.
I20260709 13:18:27.797708 15671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "566eae6d93af4ec492b9b1a270dc03b7" candidate_uuid: "7cd7c680ef2e474c981ac81abadecfde" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c1822e697d64b758a23b11c8fdeb59a"
I20260709 13:18:27.798280 15671 raft_consensus.cc:3060] T 566eae6d93af4ec492b9b1a270dc03b7 P 8c1822e697d64b758a23b11c8fdeb59a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:27.798666 15861 tablet_bootstrap.cc:492] T 17f97fe9a93f4a86b1e2bad621bfb561 P 62bbd271280e4b0a9ddf9ab46151e455: Bootstrap starting.
I20260709 13:18:27.799453 15783 leader_election.cc:304] T 8fc26f7d12464423a212a1f595ca4b88 P 62bbd271280e4b0a9ddf9ab46151e455 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 62bbd271280e4b0a9ddf9ab46151e455, 8c1822e697d64b758a23b11c8fdeb59a; no voters: 
I20260709 13:18:27.799510 15884 raft_consensus.cc:515] T 4d83a05b849340c186cc995649575d3b P 7cd7c680ef2e474c981ac81abadecfde [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62bbd271280e4b0a9ddf9ab46151e455" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 33917 } } peers { permanent_uuid: "7cd7c680ef2e474c981ac81abadecfde" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 40117 } } peers { permanent_uuid: "8c1822e697d64b758a23b11c8fdeb59a" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 39067 } }
I20260709 13:18:27.800328 15866 raft_consensus.cc:2804] T 8fc26f7d12464423a212a1f595ca4b88 P 62bbd271280e4b0a9ddf9ab46151e455 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:18:27.800671 15866 raft_consensus.cc:493] T 8fc26f7d12464423a212a1f595ca4b88 P 62bbd271280e4b0a9ddf9ab46151e455 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:27.801088 15866 raft_consensus.cc:3060] T 8fc26f7d12464423a212a1f595ca4b88 P 62bbd271280e4b0a9ddf9ab46151e455 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:27.802415 15819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d83a05b849340c186cc995649575d3b" candidate_uuid: "7cd7c680ef2e474c981ac81abadecfde" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "62bbd271280e4b0a9ddf9ab46151e455"
I20260709 13:18:27.803012 15819 raft_consensus.cc:3060] T 4d83a05b849340c186cc995649575d3b P 62bbd271280e4b0a9ddf9ab46151e455 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:27.807060 15862 tablet_bootstrap.cc:492] T 48e8b6e82d7b42e6bdd4b0a8a25dfd9e P 8c1822e697d64b758a23b11c8fdeb59a: No bootstrap required, opened a new log
I20260709 13:18:27.807560 15862 ts_tablet_manager.cc:1403] T 48e8b6e82d7b42e6bdd4b0a8a25dfd9e P 8c1822e697d64b758a23b11c8fdeb59a: Time spent bootstrapping tablet: real 0.072s	user 0.021s	sys 0.022s
I20260709 13:18:27.808076 15861 tablet_bootstrap.cc:654] T 17f97fe9a93f4a86b1e2bad621bfb561 P 62bbd271280e4b0a9ddf9ab46151e455: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:27.810021 15819 raft_consensus.cc:2468] T 4d83a05b849340c186cc995649575d3b P 62bbd271280e4b0a9ddf9ab46151e455 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7cd7c680ef2e474c981ac81abadecfde in term 1.
I20260709 13:18:27.808991 15866 raft_consensus.cc:515] T 8fc26f7d12464423a212a1f595ca4b88 P 62bbd271280e4b0a9ddf9ab46151e455 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c1822e697d64b758a23b11c8fdeb59a" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 39067 } } peers { permanent_uuid: "7cd7c680ef2e474c981ac81abadecfde" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 40117 } } peers { permanent_uuid: "62bbd271280e4b0a9ddf9ab46151e455" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 33917 } }
I20260709 13:18:27.809629 15893 raft_consensus.cc:493] T 17f97fe9a93f4a86b1e2bad621bfb561 P 8c1822e697d64b758a23b11c8fdeb59a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:27.795707 15868 raft_consensus.cc:515] T 4d83a05b849340c186cc995649575d3b P 8c1822e697d64b758a23b11c8fdeb59a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62bbd271280e4b0a9ddf9ab46151e455" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 33917 } } peers { permanent_uuid: "7cd7c680ef2e474c981ac81abadecfde" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 40117 } } peers { permanent_uuid: "8c1822e697d64b758a23b11c8fdeb59a" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 39067 } }
I20260709 13:18:27.810736 15893 raft_consensus.cc:515] T 17f97fe9a93f4a86b1e2bad621bfb561 P 8c1822e697d64b758a23b11c8fdeb59a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c1822e697d64b758a23b11c8fdeb59a" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 39067 } } peers { permanent_uuid: "7cd7c680ef2e474c981ac81abadecfde" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 40117 } } peers { permanent_uuid: "62bbd271280e4b0a9ddf9ab46151e455" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 33917 } }
I20260709 13:18:27.811355 15712 leader_election.cc:304] T 4d83a05b849340c186cc995649575d3b P 7cd7c680ef2e474c981ac81abadecfde [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 62bbd271280e4b0a9ddf9ab46151e455, 7cd7c680ef2e474c981ac81abadecfde; no voters: 
I20260709 13:18:27.809185 15671 raft_consensus.cc:2468] T 566eae6d93af4ec492b9b1a270dc03b7 P 8c1822e697d64b758a23b11c8fdeb59a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7cd7c680ef2e474c981ac81abadecfde in term 1.
I20260709 13:18:27.812415 15886 raft_consensus.cc:2804] T 4d83a05b849340c186cc995649575d3b P 7cd7c680ef2e474c981ac81abadecfde [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:18:27.812989 15709 leader_election.cc:304] T 566eae6d93af4ec492b9b1a270dc03b7 P 7cd7c680ef2e474c981ac81abadecfde [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7cd7c680ef2e474c981ac81abadecfde, 8c1822e697d64b758a23b11c8fdeb59a; no voters: 
I20260709 13:18:27.813172 15893 leader_election.cc:290] T 17f97fe9a93f4a86b1e2bad621bfb561 P 8c1822e697d64b758a23b11c8fdeb59a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7cd7c680ef2e474c981ac81abadecfde (127.14.177.130:40117), 62bbd271280e4b0a9ddf9ab46151e455 (127.14.177.131:33917)
I20260709 13:18:27.814487 15670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc26f7d12464423a212a1f595ca4b88" candidate_uuid: "62bbd271280e4b0a9ddf9ab46151e455" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c1822e697d64b758a23b11c8fdeb59a"
I20260709 13:18:27.815054 15670 raft_consensus.cc:3060] T 8fc26f7d12464423a212a1f595ca4b88 P 8c1822e697d64b758a23b11c8fdeb59a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:27.815671 15746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17f97fe9a93f4a86b1e2bad621bfb561" candidate_uuid: "8c1822e697d64b758a23b11c8fdeb59a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cd7c680ef2e474c981ac81abadecfde" is_pre_election: true
I20260709 13:18:27.816136 15866 leader_election.cc:290] T 8fc26f7d12464423a212a1f595ca4b88 P 62bbd271280e4b0a9ddf9ab46151e455 [CANDIDATE]: Term 1 election: Requested vote from peers 8c1822e697d64b758a23b11c8fdeb59a (127.14.177.129:39067), 7cd7c680ef2e474c981ac81abadecfde (127.14.177.130:40117)
I20260709 13:18:27.816277 15746 raft_consensus.cc:2468] T 17f97fe9a93f4a86b1e2bad621bfb561 P 7cd7c680ef2e474c981ac81abadecfde [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c1822e697d64b758a23b11c8fdeb59a in term 0.
I20260709 13:18:27.818372 15636 leader_election.cc:304] T 17f97fe9a93f4a86b1e2bad621bfb561 P 8c1822e697d64b758a23b11c8fdeb59a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7cd7c680ef2e474c981ac81abadecfde, 8c1822e697d64b758a23b11c8fdeb59a; no voters: 
I20260709 13:18:27.819197 15893 raft_consensus.cc:2804] T 17f97fe9a93f4a86b1e2bad621bfb561 P 8c1822e697d64b758a23b11c8fdeb59a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:18:27.819540 15893 raft_consensus.cc:493] T 17f97fe9a93f4a86b1e2bad621bfb561 P 8c1822e697d64b758a23b11c8fdeb59a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:27.819890 15893 raft_consensus.cc:3060] T 17f97fe9a93f4a86b1e2bad621bfb561 P 8c1822e697d64b758a23b11c8fdeb59a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:27.822131 15670 raft_consensus.cc:2468] T 8fc26f7d12464423a212a1f595ca4b88 P 8c1822e697d64b758a23b11c8fdeb59a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 62bbd271280e4b0a9ddf9ab46151e455 in term 1.
I20260709 13:18:27.823350 15783 leader_election.cc:304] T 8fc26f7d12464423a212a1f595ca4b88 P 62bbd271280e4b0a9ddf9ab46151e455 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 62bbd271280e4b0a9ddf9ab46151e455, 8c1822e697d64b758a23b11c8fdeb59a; no voters: 
I20260709 13:18:27.823566 15867 leader_election.cc:290] T 566eae6d93af4ec492b9b1a270dc03b7 P 7cd7c680ef2e474c981ac81abadecfde [CANDIDATE]: Term 1 election: Requested vote from peers 8c1822e697d64b758a23b11c8fdeb59a (127.14.177.129:39067), 62bbd271280e4b0a9ddf9ab46151e455 (127.14.177.131:33917)
I20260709 13:18:27.823936 15820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "566eae6d93af4ec492b9b1a270dc03b7" candidate_uuid: "7cd7c680ef2e474c981ac81abadecfde" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "62bbd271280e4b0a9ddf9ab46151e455"
I20260709 13:18:27.824186 15866 raft_consensus.cc:2804] T 8fc26f7d12464423a212a1f595ca4b88 P 62bbd271280e4b0a9ddf9ab46151e455 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:18:27.824503 15820 raft_consensus.cc:3060] T 566eae6d93af4ec492b9b1a270dc03b7 P 62bbd271280e4b0a9ddf9ab46151e455 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:27.811270 15862 raft_consensus.cc:359] T 48e8b6e82d7b42e6bdd4b0a8a25dfd9e P 8c1822e697d64b758a23b11c8fdeb59a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c1822e697d64b758a23b11c8fdeb59a" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 39067 } } peers { permanent_uuid: "7cd7c680ef2e474c981ac81abadecfde" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 40117 } } peers { permanent_uuid: "62bbd271280e4b0a9ddf9ab46151e455" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 33917 } }
I20260709 13:18:27.825598 15862 raft_consensus.cc:385] T 48e8b6e82d7b42e6bdd4b0a8a25dfd9e P 8c1822e697d64b758a23b11c8fdeb59a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:27.825886 15862 raft_consensus.cc:740] T 48e8b6e82d7b42e6bdd4b0a8a25dfd9e P 8c1822e697d64b758a23b11c8fdeb59a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c1822e697d64b758a23b11c8fdeb59a, State: Initialized, Role: FOLLOWER
I20260709 13:18:27.826524 15862 consensus_queue.cc:260] T 48e8b6e82d7b42e6bdd4b0a8a25dfd9e P 8c1822e697d64b758a23b11c8fdeb59a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c1822e697d64b758a23b11c8fdeb59a" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 39067 } } peers { permanent_uuid: "7cd7c680ef2e474c981ac81abadecfde" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 40117 } } peers { permanent_uuid: "62bbd271280e4b0a9ddf9ab46151e455" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 33917 } }
I20260709 13:18:27.824271 15867 raft_consensus.cc:2804] T 566eae6d93af4ec492b9b1a270dc03b7 P 7cd7c680ef2e474c981ac81abadecfde [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:18:27.827636 15867 raft_consensus.cc:697] T 566eae6d93af4ec492b9b1a270dc03b7 P 7cd7c680ef2e474c981ac81abadecfde [term 1 LEADER]: Becoming Leader. State: Replica: 7cd7c680ef2e474c981ac81abadecfde, State: Running, Role: LEADER
I20260709 13:18:27.828473 15867 consensus_queue.cc:237] T 566eae6d93af4ec492b9b1a270dc03b7 P 7cd7c680ef2e474c981ac81abadecfde [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c1822e697d64b758a23b11c8fdeb59a" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 39067 } } peers { permanent_uuid: "7cd7c680ef2e474c981ac81abadecfde" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 40117 } } peers { permanent_uuid: "62bbd271280e4b0a9ddf9ab46151e455" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 33917 } }
I20260709 13:18:27.817004 15819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17f97fe9a93f4a86b1e2bad621bfb561" candidate_uuid: "8c1822e697d64b758a23b11c8fdeb59a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "62bbd271280e4b0a9ddf9ab46151e455" is_pre_election: true
W20260709 13:18:27.833122 15637 leader_election.cc:343] T 17f97fe9a93f4a86b1e2bad621bfb561 P 8c1822e697d64b758a23b11c8fdeb59a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 62bbd271280e4b0a9ddf9ab46151e455 (127.14.177.131:33917): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:18:27.834631 15893 raft_consensus.cc:515] T 17f97fe9a93f4a86b1e2bad621bfb561 P 8c1822e697d64b758a23b11c8fdeb59a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c1822e697d64b758a23b11c8fdeb59a" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 39067 } } peers { permanent_uuid: "7cd7c680ef2e474c981ac81abadecfde" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 40117 } } peers { permanent_uuid: "62bbd271280e4b0a9ddf9ab46151e455" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 33917 } }
I20260709 13:18:27.838443 15746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17f97fe9a93f4a86b1e2bad621bfb561" candidate_uuid: "8c1822e697d64b758a23b11c8fdeb59a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cd7c680ef2e474c981ac81abadecfde"
I20260709 13:18:27.839187 15746 raft_consensus.cc:3060] T 17f97fe9a93f4a86b1e2bad621bfb561 P 7cd7c680ef2e474c981ac81abadecfde [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:27.842618 15820 raft_consensus.cc:2468] T 566eae6d93af4ec492b9b1a270dc03b7 P 62bbd271280e4b0a9ddf9ab46151e455 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7cd7c680ef2e474c981ac81abadecfde in term 1.
I20260709 13:18:27.846586 15861 tablet_bootstrap.cc:492] T 17f97fe9a93f4a86b1e2bad621bfb561 P 62bbd271280e4b0a9ddf9ab46151e455: No bootstrap required, opened a new log
I20260709 13:18:27.847355 15861 ts_tablet_manager.cc:1403] T 17f97fe9a93f4a86b1e2bad621bfb561 P 62bbd271280e4b0a9ddf9ab46151e455: Time spent bootstrapping tablet: real 0.049s	user 0.015s	sys 0.003s
I20260709 13:18:27.851097 15861 raft_consensus.cc:359] T 17f97fe9a93f4a86b1e2bad621bfb561 P 62bbd271280e4b0a9ddf9ab46151e455 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c1822e697d64b758a23b11c8fdeb59a" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 39067 } } peers { permanent_uuid: "7cd7c680ef2e474c981ac81abadecfde" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 40117 } } peers { permanent_uuid: "62bbd271280e4b0a9ddf9ab46151e455" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 33917 } }
I20260709 13:18:27.851970 15861 raft_consensus.cc:385] T 17f97fe9a93f4a86b1e2bad621bfb561 P 62bbd271280e4b0a9ddf9ab46151e455 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:27.852324 15861 raft_consensus.cc:740] T 17f97fe9a93f4a86b1e2bad621bfb561 P 62bbd271280e4b0a9ddf9ab46151e455 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 62bbd271280e4b0a9ddf9ab46151e455, State: Initialized, Role: FOLLOWER
I20260709 13:18:27.852780 15745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc26f7d12464423a212a1f595ca4b88" candidate_uuid: "62bbd271280e4b0a9ddf9ab46151e455" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cd7c680ef2e474c981ac81abadecfde" is_pre_election: true
I20260709 13:18:27.853470 15745 raft_consensus.cc:2468] T 8fc26f7d12464423a212a1f595ca4b88 P 7cd7c680ef2e474c981ac81abadecfde [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 62bbd271280e4b0a9ddf9ab46151e455 in term 0.
I20260709 13:18:27.853201 15861 consensus_queue.cc:260] T 17f97fe9a93f4a86b1e2bad621bfb561 P 62bbd271280e4b0a9ddf9ab46151e455 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c1822e697d64b758a23b11c8fdeb59a" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 39067 } } peers { permanent_uuid: "7cd7c680ef2e474c981ac81abadecfde" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 40117 } } peers { permanent_uuid: "62bbd271280e4b0a9ddf9ab46151e455" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 33917 } }
I20260709 13:18:27.854671 15744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc26f7d12464423a212a1f595ca4b88" candidate_uuid: "62bbd271280e4b0a9ddf9ab46151e455" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cd7c680ef2e474c981ac81abadecfde"
I20260709 13:18:27.855556 15744 raft_consensus.cc:3060] T 8fc26f7d12464423a212a1f595ca4b88 P 7cd7c680ef2e474c981ac81abadecfde [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:27.857949 15746 raft_consensus.cc:2468] T 17f97fe9a93f4a86b1e2bad621bfb561 P 7cd7c680ef2e474c981ac81abadecfde [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c1822e697d64b758a23b11c8fdeb59a in term 1.
I20260709 13:18:27.859498 15636 leader_election.cc:304] T 17f97fe9a93f4a86b1e2bad621bfb561 P 8c1822e697d64b758a23b11c8fdeb59a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7cd7c680ef2e474c981ac81abadecfde, 8c1822e697d64b758a23b11c8fdeb59a; no voters: 
I20260709 13:18:27.862352 15886 raft_consensus.cc:697] T 4d83a05b849340c186cc995649575d3b P 7cd7c680ef2e474c981ac81abadecfde [term 1 LEADER]: Becoming Leader. State: Replica: 7cd7c680ef2e474c981ac81abadecfde, State: Running, Role: LEADER
I20260709 13:18:27.863878 15861 ts_tablet_manager.cc:1434] T 17f97fe9a93f4a86b1e2bad621bfb561 P 62bbd271280e4b0a9ddf9ab46151e455: Time spent starting tablet: real 0.016s	user 0.006s	sys 0.008s
I20260709 13:18:27.863972 15866 raft_consensus.cc:697] T 8fc26f7d12464423a212a1f595ca4b88 P 62bbd271280e4b0a9ddf9ab46151e455 [term 1 LEADER]: Becoming Leader. State: Replica: 62bbd271280e4b0a9ddf9ab46151e455, State: Running, Role: LEADER
I20260709 13:18:27.865294 15744 raft_consensus.cc:2468] T 8fc26f7d12464423a212a1f595ca4b88 P 7cd7c680ef2e474c981ac81abadecfde [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 62bbd271280e4b0a9ddf9ab46151e455 in term 1.
I20260709 13:18:27.865820 15861 tablet_bootstrap.cc:492] T 502d133e880b4a859d8674b9d0cb6931 P 62bbd271280e4b0a9ddf9ab46151e455: Bootstrap starting.
I20260709 13:18:27.867851 15820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d83a05b849340c186cc995649575d3b" candidate_uuid: "8c1822e697d64b758a23b11c8fdeb59a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "62bbd271280e4b0a9ddf9ab46151e455"
I20260709 13:18:27.868880 15820 raft_consensus.cc:2393] T 4d83a05b849340c186cc995649575d3b P 62bbd271280e4b0a9ddf9ab46151e455 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 8c1822e697d64b758a23b11c8fdeb59a in current term 1: Already voted for candidate 7cd7c680ef2e474c981ac81abadecfde in this term.
I20260709 13:18:27.863590 15886 consensus_queue.cc:237] T 4d83a05b849340c186cc995649575d3b P 7cd7c680ef2e474c981ac81abadecfde [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62bbd271280e4b0a9ddf9ab46151e455" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 33917 } } peers { permanent_uuid: "7cd7c680ef2e474c981ac81abadecfde" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 40117 } } peers { permanent_uuid: "8c1822e697d64b758a23b11c8fdeb59a" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 39067 } }
I20260709 13:18:27.865245 15866 consensus_queue.cc:237] T 8fc26f7d12464423a212a1f595ca4b88 P 62bbd271280e4b0a9ddf9ab46151e455 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c1822e697d64b758a23b11c8fdeb59a" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 39067 } } peers { permanent_uuid: "7cd7c680ef2e474c981ac81abadecfde" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 40117 } } peers { permanent_uuid: "62bbd271280e4b0a9ddf9ab46151e455" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 33917 } }
I20260709 13:18:27.872243 15884 leader_election.cc:290] T 4d83a05b849340c186cc995649575d3b P 7cd7c680ef2e474c981ac81abadecfde [CANDIDATE]: Term 1 election: Requested vote from peers 62bbd271280e4b0a9ddf9ab46151e455 (127.14.177.131:33917), 8c1822e697d64b758a23b11c8fdeb59a (127.14.177.129:39067)
I20260709 13:18:27.873319 15868 leader_election.cc:290] T 4d83a05b849340c186cc995649575d3b P 8c1822e697d64b758a23b11c8fdeb59a [CANDIDATE]: Term 1 election: Requested vote from peers 62bbd271280e4b0a9ddf9ab46151e455 (127.14.177.131:33917), 7cd7c680ef2e474c981ac81abadecfde (127.14.177.130:40117)
I20260709 13:18:27.873410 15670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d83a05b849340c186cc995649575d3b" candidate_uuid: "7cd7c680ef2e474c981ac81abadecfde" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c1822e697d64b758a23b11c8fdeb59a"
I20260709 13:18:27.874099 15868 raft_consensus.cc:493] T 8309d9a5a4ef4e6d914751bc0f410015 P 8c1822e697d64b758a23b11c8fdeb59a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:27.874181 15670 raft_consensus.cc:2393] T 4d83a05b849340c186cc995649575d3b P 8c1822e697d64b758a23b11c8fdeb59a [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 7cd7c680ef2e474c981ac81abadecfde in current term 1: Already voted for candidate 8c1822e697d64b758a23b11c8fdeb59a in this term.
I20260709 13:18:27.876252 15907 raft_consensus.cc:2804] T 17f97fe9a93f4a86b1e2bad621bfb561 P 8c1822e697d64b758a23b11c8fdeb59a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:18:27.876482 15862 ts_tablet_manager.cc:1434] T 48e8b6e82d7b42e6bdd4b0a8a25dfd9e P 8c1822e697d64b758a23b11c8fdeb59a: Time spent starting tablet: real 0.069s	user 0.009s	sys 0.016s
I20260709 13:18:27.876883 15907 raft_consensus.cc:697] T 17f97fe9a93f4a86b1e2bad621bfb561 P 8c1822e697d64b758a23b11c8fdeb59a [term 1 LEADER]: Becoming Leader. State: Replica: 8c1822e697d64b758a23b11c8fdeb59a, State: Running, Role: LEADER
I20260709 13:18:27.877439 15862 tablet_bootstrap.cc:492] T 502d133e880b4a859d8674b9d0cb6931 P 8c1822e697d64b758a23b11c8fdeb59a: Bootstrap starting.
I20260709 13:18:27.877800 15907 consensus_queue.cc:237] T 17f97fe9a93f4a86b1e2bad621bfb561 P 8c1822e697d64b758a23b11c8fdeb59a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c1822e697d64b758a23b11c8fdeb59a" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 39067 } } peers { permanent_uuid: "7cd7c680ef2e474c981ac81abadecfde" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 40117 } } peers { permanent_uuid: "62bbd271280e4b0a9ddf9ab46151e455" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 33917 } }
I20260709 13:18:27.880995 15861 tablet_bootstrap.cc:654] T 502d133e880b4a859d8674b9d0cb6931 P 62bbd271280e4b0a9ddf9ab46151e455: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:27.883594 15820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17f97fe9a93f4a86b1e2bad621bfb561" candidate_uuid: "8c1822e697d64b758a23b11c8fdeb59a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "62bbd271280e4b0a9ddf9ab46151e455"
I20260709 13:18:27.884069 15862 tablet_bootstrap.cc:654] T 502d133e880b4a859d8674b9d0cb6931 P 8c1822e697d64b758a23b11c8fdeb59a: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:27.884361 15820 raft_consensus.cc:3060] T 17f97fe9a93f4a86b1e2bad621bfb561 P 62bbd271280e4b0a9ddf9ab46151e455 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:27.893970 15820 raft_consensus.cc:2468] T 17f97fe9a93f4a86b1e2bad621bfb561 P 62bbd271280e4b0a9ddf9ab46151e455 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c1822e697d64b758a23b11c8fdeb59a in term 1.
I20260709 13:18:27.896023 15893 leader_election.cc:290] T 17f97fe9a93f4a86b1e2bad621bfb561 P 8c1822e697d64b758a23b11c8fdeb59a [CANDIDATE]: Term 1 election: Requested vote from peers 7cd7c680ef2e474c981ac81abadecfde (127.14.177.130:40117), 62bbd271280e4b0a9ddf9ab46151e455 (127.14.177.131:33917)
I20260709 13:18:27.900475 15744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d83a05b849340c186cc995649575d3b" candidate_uuid: "8c1822e697d64b758a23b11c8fdeb59a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cd7c680ef2e474c981ac81abadecfde"
I20260709 13:18:27.901085 15860 tablet_bootstrap.cc:492] T 8309d9a5a4ef4e6d914751bc0f410015 P 7cd7c680ef2e474c981ac81abadecfde: No bootstrap required, opened a new log
I20260709 13:18:27.901639 15860 ts_tablet_manager.cc:1403] T 8309d9a5a4ef4e6d914751bc0f410015 P 7cd7c680ef2e474c981ac81abadecfde: Time spent bootstrapping tablet: real 0.154s	user 0.011s	sys 0.009s
I20260709 13:18:27.902406 15636 leader_election.cc:304] T 4d83a05b849340c186cc995649575d3b P 8c1822e697d64b758a23b11c8fdeb59a [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c1822e697d64b758a23b11c8fdeb59a; no voters: 62bbd271280e4b0a9ddf9ab46151e455, 7cd7c680ef2e474c981ac81abadecfde
I20260709 13:18:27.903707 15911 raft_consensus.cc:2749] T 4d83a05b849340c186cc995649575d3b P 8c1822e697d64b758a23b11c8fdeb59a [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:18:27.874619 15868 raft_consensus.cc:515] T 8309d9a5a4ef4e6d914751bc0f410015 P 8c1822e697d64b758a23b11c8fdeb59a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c1822e697d64b758a23b11c8fdeb59a" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 39067 } } peers { permanent_uuid: "7cd7c680ef2e474c981ac81abadecfde" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 40117 } } peers { permanent_uuid: "62bbd271280e4b0a9ddf9ab46151e455" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 33917 } }
I20260709 13:18:27.904633 15860 raft_consensus.cc:359] T 8309d9a5a4ef4e6d914751bc0f410015 P 7cd7c680ef2e474c981ac81abadecfde [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c1822e697d64b758a23b11c8fdeb59a" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 39067 } } peers { permanent_uuid: "7cd7c680ef2e474c981ac81abadecfde" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 40117 } } peers { permanent_uuid: "62bbd271280e4b0a9ddf9ab46151e455" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 33917 } }
I20260709 13:18:27.905385 15860 raft_consensus.cc:385] T 8309d9a5a4ef4e6d914751bc0f410015 P 7cd7c680ef2e474c981ac81abadecfde [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:27.905690 15860 raft_consensus.cc:740] T 8309d9a5a4ef4e6d914751bc0f410015 P 7cd7c680ef2e474c981ac81abadecfde [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7cd7c680ef2e474c981ac81abadecfde, State: Initialized, Role: FOLLOWER
I20260709 13:18:27.911006 15910 raft_consensus.cc:493] T 8309d9a5a4ef4e6d914751bc0f410015 P 62bbd271280e4b0a9ddf9ab46151e455 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:27.911223 15746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8309d9a5a4ef4e6d914751bc0f410015" candidate_uuid: "8c1822e697d64b758a23b11c8fdeb59a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cd7c680ef2e474c981ac81abadecfde" is_pre_election: true
I20260709 13:18:27.911587 15910 raft_consensus.cc:515] T 8309d9a5a4ef4e6d914751bc0f410015 P 62bbd271280e4b0a9ddf9ab46151e455 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c1822e697d64b758a23b11c8fdeb59a" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 39067 } } peers { permanent_uuid: "7cd7c680ef2e474c981ac81abadecfde" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 40117 } } peers { permanent_uuid: "62bbd271280e4b0a9ddf9ab46151e455" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 33917 } }
I20260709 13:18:27.912148 15820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8309d9a5a4ef4e6d914751bc0f410015" candidate_uuid: "8c1822e697d64b758a23b11c8fdeb59a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "62bbd271280e4b0a9ddf9ab46151e455" is_pre_election: true
I20260709 13:18:27.912727 15820 raft_consensus.cc:2468] T 8309d9a5a4ef4e6d914751bc0f410015 P 62bbd271280e4b0a9ddf9ab46151e455 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c1822e697d64b758a23b11c8fdeb59a in term 0.
I20260709 13:18:27.913936 15910 leader_election.cc:290] T 8309d9a5a4ef4e6d914751bc0f410015 P 62bbd271280e4b0a9ddf9ab46151e455 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8c1822e697d64b758a23b11c8fdeb59a (127.14.177.129:39067), 7cd7c680ef2e474c981ac81abadecfde (127.14.177.130:40117)
I20260709 13:18:27.913877 15637 leader_election.cc:304] T 8309d9a5a4ef4e6d914751bc0f410015 P 8c1822e697d64b758a23b11c8fdeb59a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 62bbd271280e4b0a9ddf9ab46151e455, 8c1822e697d64b758a23b11c8fdeb59a; no voters: 
I20260709 13:18:27.914763 15911 raft_consensus.cc:2804] T 8309d9a5a4ef4e6d914751bc0f410015 P 8c1822e697d64b758a23b11c8fdeb59a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:18:27.915127 15911 raft_consensus.cc:493] T 8309d9a5a4ef4e6d914751bc0f410015 P 8c1822e697d64b758a23b11c8fdeb59a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:27.915506 15911 raft_consensus.cc:3060] T 8309d9a5a4ef4e6d914751bc0f410015 P 8c1822e697d64b758a23b11c8fdeb59a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:27.915584 15744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8309d9a5a4ef4e6d914751bc0f410015" candidate_uuid: "62bbd271280e4b0a9ddf9ab46151e455" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cd7c680ef2e474c981ac81abadecfde" is_pre_election: true
I20260709 13:18:27.918068 15670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8309d9a5a4ef4e6d914751bc0f410015" candidate_uuid: "62bbd271280e4b0a9ddf9ab46151e455" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c1822e697d64b758a23b11c8fdeb59a" is_pre_election: true
I20260709 13:18:27.917040 15909 raft_consensus.cc:493] T 408866bb47904e4189bc235ca5b0c7c7 P 8c1822e697d64b758a23b11c8fdeb59a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:27.919231 15868 leader_election.cc:290] T 8309d9a5a4ef4e6d914751bc0f410015 P 8c1822e697d64b758a23b11c8fdeb59a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7cd7c680ef2e474c981ac81abadecfde (127.14.177.130:40117), 62bbd271280e4b0a9ddf9ab46151e455 (127.14.177.131:33917)
I20260709 13:18:27.919494 15909 raft_consensus.cc:515] T 408866bb47904e4189bc235ca5b0c7c7 P 8c1822e697d64b758a23b11c8fdeb59a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c1822e697d64b758a23b11c8fdeb59a" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 39067 } } peers { permanent_uuid: "7cd7c680ef2e474c981ac81abadecfde" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 40117 } } peers { permanent_uuid: "62bbd271280e4b0a9ddf9ab46151e455" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 33917 } }
I20260709 13:18:27.924786 15820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "408866bb47904e4189bc235ca5b0c7c7" candidate_uuid: "8c1822e697d64b758a23b11c8fdeb59a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "62bbd271280e4b0a9ddf9ab46151e455" is_pre_election: true
I20260709 13:18:27.926337 15820 raft_consensus.cc:2468] T 408866bb47904e4189bc235ca5b0c7c7 P 62bbd271280e4b0a9ddf9ab46151e455 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c1822e697d64b758a23b11c8fdeb59a in term 0.
I20260709 13:18:27.927335 15744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "408866bb47904e4189bc235ca5b0c7c7" candidate_uuid: "8c1822e697d64b758a23b11c8fdeb59a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cd7c680ef2e474c981ac81abadecfde" is_pre_election: true
I20260709 13:18:27.929304 15637 leader_election.cc:304] T 408866bb47904e4189bc235ca5b0c7c7 P 8c1822e697d64b758a23b11c8fdeb59a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 62bbd271280e4b0a9ddf9ab46151e455, 8c1822e697d64b758a23b11c8fdeb59a; no voters: 
I20260709 13:18:27.930220 15868 raft_consensus.cc:2804] T 408866bb47904e4189bc235ca5b0c7c7 P 8c1822e697d64b758a23b11c8fdeb59a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:18:27.930649 15868 raft_consensus.cc:493] T 408866bb47904e4189bc235ca5b0c7c7 P 8c1822e697d64b758a23b11c8fdeb59a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:27.930995 15868 raft_consensus.cc:3060] T 408866bb47904e4189bc235ca5b0c7c7 P 8c1822e697d64b758a23b11c8fdeb59a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:27.922938 15909 leader_election.cc:290] T 408866bb47904e4189bc235ca5b0c7c7 P 8c1822e697d64b758a23b11c8fdeb59a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7cd7c680ef2e474c981ac81abadecfde (127.14.177.130:40117), 62bbd271280e4b0a9ddf9ab46151e455 (127.14.177.131:33917)
W20260709 13:18:27.935695 15636 leader_election.cc:343] T 408866bb47904e4189bc235ca5b0c7c7 P 8c1822e697d64b758a23b11c8fdeb59a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7cd7c680ef2e474c981ac81abadecfde (127.14.177.130:40117): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:18:27.933938 15860 consensus_queue.cc:260] T 8309d9a5a4ef4e6d914751bc0f410015 P 7cd7c680ef2e474c981ac81abadecfde [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c1822e697d64b758a23b11c8fdeb59a" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 39067 } } peers { permanent_uuid: "7cd7c680ef2e474c981ac81abadecfde" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 40117 } } peers { permanent_uuid: "62bbd271280e4b0a9ddf9ab46151e455" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 33917 } }
I20260709 13:18:27.940361 15746 raft_consensus.cc:2468] T 8309d9a5a4ef4e6d914751bc0f410015 P 7cd7c680ef2e474c981ac81abadecfde [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c1822e697d64b758a23b11c8fdeb59a in term 0.
I20260709 13:18:27.941764 15860 ts_tablet_manager.cc:1434] T 8309d9a5a4ef4e6d914751bc0f410015 P 7cd7c680ef2e474c981ac81abadecfde: Time spent starting tablet: real 0.040s	user 0.005s	sys 0.004s
I20260709 13:18:27.942749 15860 tablet_bootstrap.cc:492] T 48e8b6e82d7b42e6bdd4b0a8a25dfd9e P 7cd7c680ef2e474c981ac81abadecfde: Bootstrap starting.
I20260709 13:18:27.945479 15861 tablet_bootstrap.cc:492] T 502d133e880b4a859d8674b9d0cb6931 P 62bbd271280e4b0a9ddf9ab46151e455: No bootstrap required, opened a new log
I20260709 13:18:27.918938 15569 catalog_manager.cc:5697] T 566eae6d93af4ec492b9b1a270dc03b7 P 7cd7c680ef2e474c981ac81abadecfde reported cstate change: term changed from 0 to 1, leader changed from <none> to 7cd7c680ef2e474c981ac81abadecfde (127.14.177.130). New cstate: current_term: 1 leader_uuid: "7cd7c680ef2e474c981ac81abadecfde" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c1822e697d64b758a23b11c8fdeb59a" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 39067 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7cd7c680ef2e474c981ac81abadecfde" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 40117 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "62bbd271280e4b0a9ddf9ab46151e455" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 33917 } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:27.946041 15861 ts_tablet_manager.cc:1403] T 502d133e880b4a859d8674b9d0cb6931 P 62bbd271280e4b0a9ddf9ab46151e455: Time spent bootstrapping tablet: real 0.080s	user 0.014s	sys 0.024s
I20260709 13:18:27.947005 15868 raft_consensus.cc:515] T 408866bb47904e4189bc235ca5b0c7c7 P 8c1822e697d64b758a23b11c8fdeb59a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c1822e697d64b758a23b11c8fdeb59a" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 39067 } } peers { permanent_uuid: "7cd7c680ef2e474c981ac81abadecfde" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 40117 } } peers { permanent_uuid: "62bbd271280e4b0a9ddf9ab46151e455" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 33917 } }
I20260709 13:18:27.949182 15861 raft_consensus.cc:359] T 502d133e880b4a859d8674b9d0cb6931 P 62bbd271280e4b0a9ddf9ab46151e455 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c1822e697d64b758a23b11c8fdeb59a" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 39067 } } peers { permanent_uuid: "7cd7c680ef2e474c981ac81abadecfde" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 40117 } } peers { permanent_uuid: "62bbd271280e4b0a9ddf9ab46151e455" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 33917 } }
I20260709 13:18:27.950039 15861 raft_consensus.cc:385] T 502d133e880b4a859d8674b9d0cb6931 P 62bbd271280e4b0a9ddf9ab46151e455 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:27.947032 15569 catalog_manager.cc:5697] T 8cff7911abd649b9a276ddae431311d1 P 7cd7c680ef2e474c981ac81abadecfde reported cstate change: term changed from 0 to 1, leader changed from <none> to 7cd7c680ef2e474c981ac81abadecfde (127.14.177.130). New cstate: current_term: 1 leader_uuid: "7cd7c680ef2e474c981ac81abadecfde" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c1822e697d64b758a23b11c8fdeb59a" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 39067 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7cd7c680ef2e474c981ac81abadecfde" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 40117 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "62bbd271280e4b0a9ddf9ab46151e455" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 33917 } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:27.950371 15746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "408866bb47904e4189bc235ca5b0c7c7" candidate_uuid: "8c1822e697d64b758a23b11c8fdeb59a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cd7c680ef2e474c981ac81abadecfde"
I20260709 13:18:27.950377 15861 raft_consensus.cc:740] T 502d133e880b4a859d8674b9d0cb6931 P 62bbd271280e4b0a9ddf9ab46151e455 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 62bbd271280e4b0a9ddf9ab46151e455, State: Initialized, Role: FOLLOWER
I20260709 13:18:27.950659 15820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "408866bb47904e4189bc235ca5b0c7c7" candidate_uuid: "8c1822e697d64b758a23b11c8fdeb59a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "62bbd271280e4b0a9ddf9ab46151e455"
I20260709 13:18:27.951264 15820 raft_consensus.cc:3060] T 408866bb47904e4189bc235ca5b0c7c7 P 62bbd271280e4b0a9ddf9ab46151e455 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:27.951447 15861 consensus_queue.cc:260] T 502d133e880b4a859d8674b9d0cb6931 P 62bbd271280e4b0a9ddf9ab46151e455 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c1822e697d64b758a23b11c8fdeb59a" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 39067 } } peers { permanent_uuid: "7cd7c680ef2e474c981ac81abadecfde" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 40117 } } peers { permanent_uuid: "62bbd271280e4b0a9ddf9ab46151e455" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 33917 } }
W20260709 13:18:27.951895 15636 leader_election.cc:343] T 408866bb47904e4189bc235ca5b0c7c7 P 8c1822e697d64b758a23b11c8fdeb59a [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 7cd7c680ef2e474c981ac81abadecfde (127.14.177.130:40117): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:18:27.951697 15569 catalog_manager.cc:5697] T 4d83a05b849340c186cc995649575d3b P 7cd7c680ef2e474c981ac81abadecfde reported cstate change: term changed from 0 to 1, leader changed from <none> to 7cd7c680ef2e474c981ac81abadecfde (127.14.177.130). New cstate: current_term: 1 leader_uuid: "7cd7c680ef2e474c981ac81abadecfde" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62bbd271280e4b0a9ddf9ab46151e455" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 33917 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7cd7c680ef2e474c981ac81abadecfde" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 40117 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8c1822e697d64b758a23b11c8fdeb59a" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 39067 } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:27.953691 15861 ts_tablet_manager.cc:1434] T 502d133e880b4a859d8674b9d0cb6931 P 62bbd271280e4b0a9ddf9ab46151e455: Time spent starting tablet: real 0.007s	user 0.002s	sys 0.003s
I20260709 13:18:27.954413 15862 tablet_bootstrap.cc:492] T 502d133e880b4a859d8674b9d0cb6931 P 8c1822e697d64b758a23b11c8fdeb59a: No bootstrap required, opened a new log
I20260709 13:18:27.954675 15861 tablet_bootstrap.cc:492] T 48e8b6e82d7b42e6bdd4b0a8a25dfd9e P 62bbd271280e4b0a9ddf9ab46151e455: Bootstrap starting.
I20260709 13:18:27.955022 15868 leader_election.cc:290] T 408866bb47904e4189bc235ca5b0c7c7 P 8c1822e697d64b758a23b11c8fdeb59a [CANDIDATE]: Term 1 election: Requested vote from peers 7cd7c680ef2e474c981ac81abadecfde (127.14.177.130:40117), 62bbd271280e4b0a9ddf9ab46151e455 (127.14.177.131:33917)
I20260709 13:18:27.954836 15862 ts_tablet_manager.cc:1403] T 502d133e880b4a859d8674b9d0cb6931 P 8c1822e697d64b758a23b11c8fdeb59a: Time spent bootstrapping tablet: real 0.078s	user 0.027s	sys 0.027s
I20260709 13:18:27.958611 15820 raft_consensus.cc:2468] T 408866bb47904e4189bc235ca5b0c7c7 P 62bbd271280e4b0a9ddf9ab46151e455 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c1822e697d64b758a23b11c8fdeb59a in term 1.
I20260709 13:18:27.958912 15911 raft_consensus.cc:515] T 8309d9a5a4ef4e6d914751bc0f410015 P 8c1822e697d64b758a23b11c8fdeb59a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c1822e697d64b758a23b11c8fdeb59a" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 39067 } } peers { permanent_uuid: "7cd7c680ef2e474c981ac81abadecfde" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 40117 } } peers { permanent_uuid: "62bbd271280e4b0a9ddf9ab46151e455" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 33917 } }
I20260709 13:18:27.959887 15637 leader_election.cc:304] T 408866bb47904e4189bc235ca5b0c7c7 P 8c1822e697d64b758a23b11c8fdeb59a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 62bbd271280e4b0a9ddf9ab46151e455, 8c1822e697d64b758a23b11c8fdeb59a; no voters: 7cd7c680ef2e474c981ac81abadecfde
I20260709 13:18:27.960232 15670 raft_consensus.cc:2393] T 8309d9a5a4ef4e6d914751bc0f410015 P 8c1822e697d64b758a23b11c8fdeb59a [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 62bbd271280e4b0a9ddf9ab46151e455 in current term 1: Already voted for candidate 8c1822e697d64b758a23b11c8fdeb59a in this term.
I20260709 13:18:27.960686 15868 raft_consensus.cc:2804] T 408866bb47904e4189bc235ca5b0c7c7 P 8c1822e697d64b758a23b11c8fdeb59a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:18:27.961158 15868 raft_consensus.cc:697] T 408866bb47904e4189bc235ca5b0c7c7 P 8c1822e697d64b758a23b11c8fdeb59a [term 1 LEADER]: Becoming Leader. State: Replica: 8c1822e697d64b758a23b11c8fdeb59a, State: Running, Role: LEADER
I20260709 13:18:27.961992 15746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8309d9a5a4ef4e6d914751bc0f410015" candidate_uuid: "8c1822e697d64b758a23b11c8fdeb59a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cd7c680ef2e474c981ac81abadecfde"
I20260709 13:18:27.961958 15868 consensus_queue.cc:237] T 408866bb47904e4189bc235ca5b0c7c7 P 8c1822e697d64b758a23b11c8fdeb59a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c1822e697d64b758a23b11c8fdeb59a" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 39067 } } peers { permanent_uuid: "7cd7c680ef2e474c981ac81abadecfde" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 40117 } } peers { permanent_uuid: "62bbd271280e4b0a9ddf9ab46151e455" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 33917 } }
I20260709 13:18:27.962646 15746 raft_consensus.cc:3060] T 8309d9a5a4ef4e6d914751bc0f410015 P 7cd7c680ef2e474c981ac81abadecfde [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:27.963316 15783 leader_election.cc:304] T 8309d9a5a4ef4e6d914751bc0f410015 P 62bbd271280e4b0a9ddf9ab46151e455 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 62bbd271280e4b0a9ddf9ab46151e455; no voters: 7cd7c680ef2e474c981ac81abadecfde, 8c1822e697d64b758a23b11c8fdeb59a
I20260709 13:18:27.964128 15866 raft_consensus.cc:3060] T 8309d9a5a4ef4e6d914751bc0f410015 P 62bbd271280e4b0a9ddf9ab46151e455 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:27.964532 15862 raft_consensus.cc:359] T 502d133e880b4a859d8674b9d0cb6931 P 8c1822e697d64b758a23b11c8fdeb59a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c1822e697d64b758a23b11c8fdeb59a" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 39067 } } peers { permanent_uuid: "7cd7c680ef2e474c981ac81abadecfde" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 40117 } } peers { permanent_uuid: "62bbd271280e4b0a9ddf9ab46151e455" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 33917 } }
I20260709 13:18:27.965296 15862 raft_consensus.cc:385] T 502d133e880b4a859d8674b9d0cb6931 P 8c1822e697d64b758a23b11c8fdeb59a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:27.965605 15862 raft_consensus.cc:740] T 502d133e880b4a859d8674b9d0cb6931 P 8c1822e697d64b758a23b11c8fdeb59a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c1822e697d64b758a23b11c8fdeb59a, State: Initialized, Role: FOLLOWER
I20260709 13:18:27.966280 15862 consensus_queue.cc:260] T 502d133e880b4a859d8674b9d0cb6931 P 8c1822e697d64b758a23b11c8fdeb59a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c1822e697d64b758a23b11c8fdeb59a" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 39067 } } peers { permanent_uuid: "7cd7c680ef2e474c981ac81abadecfde" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 40117 } } peers { permanent_uuid: "62bbd271280e4b0a9ddf9ab46151e455" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 33917 } }
I20260709 13:18:27.968367 15862 ts_tablet_manager.cc:1434] T 502d133e880b4a859d8674b9d0cb6931 P 8c1822e697d64b758a23b11c8fdeb59a: Time spent starting tablet: real 0.011s	user 0.005s	sys 0.000s
I20260709 13:18:27.968545 15568 catalog_manager.cc:5697] T 8fc26f7d12464423a212a1f595ca4b88 P 62bbd271280e4b0a9ddf9ab46151e455 reported cstate change: term changed from 0 to 1, leader changed from <none> to 62bbd271280e4b0a9ddf9ab46151e455 (127.14.177.131). New cstate: current_term: 1 leader_uuid: "62bbd271280e4b0a9ddf9ab46151e455" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c1822e697d64b758a23b11c8fdeb59a" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 39067 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7cd7c680ef2e474c981ac81abadecfde" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 40117 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "62bbd271280e4b0a9ddf9ab46151e455" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 33917 } health_report { overall_health: HEALTHY } } }
I20260709 13:18:27.969492 15746 raft_consensus.cc:2468] T 8309d9a5a4ef4e6d914751bc0f410015 P 7cd7c680ef2e474c981ac81abadecfde [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c1822e697d64b758a23b11c8fdeb59a in term 1.
I20260709 13:18:27.970759 15636 leader_election.cc:304] T 8309d9a5a4ef4e6d914751bc0f410015 P 8c1822e697d64b758a23b11c8fdeb59a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7cd7c680ef2e474c981ac81abadecfde, 8c1822e697d64b758a23b11c8fdeb59a; no voters: 
I20260709 13:18:27.971087 15866 raft_consensus.cc:2749] T 8309d9a5a4ef4e6d914751bc0f410015 P 62bbd271280e4b0a9ddf9ab46151e455 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:18:27.971926 15820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8309d9a5a4ef4e6d914751bc0f410015" candidate_uuid: "8c1822e697d64b758a23b11c8fdeb59a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "62bbd271280e4b0a9ddf9ab46151e455"
I20260709 13:18:27.974691 15861 tablet_bootstrap.cc:654] T 48e8b6e82d7b42e6bdd4b0a8a25dfd9e P 62bbd271280e4b0a9ddf9ab46151e455: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:27.974689 15860 tablet_bootstrap.cc:654] T 48e8b6e82d7b42e6bdd4b0a8a25dfd9e P 7cd7c680ef2e474c981ac81abadecfde: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:27.972488 15911 leader_election.cc:290] T 8309d9a5a4ef4e6d914751bc0f410015 P 8c1822e697d64b758a23b11c8fdeb59a [CANDIDATE]: Term 1 election: Requested vote from peers 7cd7c680ef2e474c981ac81abadecfde (127.14.177.130:40117), 62bbd271280e4b0a9ddf9ab46151e455 (127.14.177.131:33917)
I20260709 13:18:27.976055 15907 raft_consensus.cc:2804] T 8309d9a5a4ef4e6d914751bc0f410015 P 8c1822e697d64b758a23b11c8fdeb59a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:18:27.977141 15907 raft_consensus.cc:697] T 8309d9a5a4ef4e6d914751bc0f410015 P 8c1822e697d64b758a23b11c8fdeb59a [term 1 LEADER]: Becoming Leader. State: Replica: 8c1822e697d64b758a23b11c8fdeb59a, State: Running, Role: LEADER
I20260709 13:18:27.980698 15907 consensus_queue.cc:237] T 8309d9a5a4ef4e6d914751bc0f410015 P 8c1822e697d64b758a23b11c8fdeb59a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c1822e697d64b758a23b11c8fdeb59a" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 39067 } } peers { permanent_uuid: "7cd7c680ef2e474c981ac81abadecfde" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 40117 } } peers { permanent_uuid: "62bbd271280e4b0a9ddf9ab46151e455" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 33917 } }
I20260709 13:18:27.984675 15567 catalog_manager.cc:5697] T 17f97fe9a93f4a86b1e2bad621bfb561 P 8c1822e697d64b758a23b11c8fdeb59a reported cstate change: term changed from 0 to 1, leader changed from <none> to 8c1822e697d64b758a23b11c8fdeb59a (127.14.177.129). New cstate: current_term: 1 leader_uuid: "8c1822e697d64b758a23b11c8fdeb59a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c1822e697d64b758a23b11c8fdeb59a" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 39067 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7cd7c680ef2e474c981ac81abadecfde" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 40117 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "62bbd271280e4b0a9ddf9ab46151e455" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 33917 } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:27.986491 15861 tablet_bootstrap.cc:492] T 48e8b6e82d7b42e6bdd4b0a8a25dfd9e P 62bbd271280e4b0a9ddf9ab46151e455: No bootstrap required, opened a new log
I20260709 13:18:27.986967 15861 ts_tablet_manager.cc:1403] T 48e8b6e82d7b42e6bdd4b0a8a25dfd9e P 62bbd271280e4b0a9ddf9ab46151e455: Time spent bootstrapping tablet: real 0.033s	user 0.009s	sys 0.003s
I20260709 13:18:27.990809 15820 raft_consensus.cc:2468] T 8309d9a5a4ef4e6d914751bc0f410015 P 62bbd271280e4b0a9ddf9ab46151e455 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c1822e697d64b758a23b11c8fdeb59a in term 1.
I20260709 13:18:27.993379 15861 raft_consensus.cc:359] T 48e8b6e82d7b42e6bdd4b0a8a25dfd9e P 62bbd271280e4b0a9ddf9ab46151e455 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c1822e697d64b758a23b11c8fdeb59a" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 39067 } } peers { permanent_uuid: "7cd7c680ef2e474c981ac81abadecfde" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 40117 } } peers { permanent_uuid: "62bbd271280e4b0a9ddf9ab46151e455" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 33917 } }
I20260709 13:18:28.001008 15861 raft_consensus.cc:385] T 48e8b6e82d7b42e6bdd4b0a8a25dfd9e P 62bbd271280e4b0a9ddf9ab46151e455 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:27.995874 15860 tablet_bootstrap.cc:492] T 48e8b6e82d7b42e6bdd4b0a8a25dfd9e P 7cd7c680ef2e474c981ac81abadecfde: No bootstrap required, opened a new log
I20260709 13:18:28.001497 15861 raft_consensus.cc:740] T 48e8b6e82d7b42e6bdd4b0a8a25dfd9e P 62bbd271280e4b0a9ddf9ab46151e455 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 62bbd271280e4b0a9ddf9ab46151e455, State: Initialized, Role: FOLLOWER
I20260709 13:18:28.001917 15860 ts_tablet_manager.cc:1403] T 48e8b6e82d7b42e6bdd4b0a8a25dfd9e P 7cd7c680ef2e474c981ac81abadecfde: Time spent bootstrapping tablet: real 0.059s	user 0.005s	sys 0.011s
I20260709 13:18:28.002321 15861 consensus_queue.cc:260] T 48e8b6e82d7b42e6bdd4b0a8a25dfd9e P 62bbd271280e4b0a9ddf9ab46151e455 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c1822e697d64b758a23b11c8fdeb59a" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 39067 } } peers { permanent_uuid: "7cd7c680ef2e474c981ac81abadecfde" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 40117 } } peers { permanent_uuid: "62bbd271280e4b0a9ddf9ab46151e455" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 33917 } }
I20260709 13:18:28.004374 15861 ts_tablet_manager.cc:1434] T 48e8b6e82d7b42e6bdd4b0a8a25dfd9e P 62bbd271280e4b0a9ddf9ab46151e455: Time spent starting tablet: real 0.017s	user 0.000s	sys 0.005s
I20260709 13:18:28.004513 15860 raft_consensus.cc:359] T 48e8b6e82d7b42e6bdd4b0a8a25dfd9e P 7cd7c680ef2e474c981ac81abadecfde [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c1822e697d64b758a23b11c8fdeb59a" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 39067 } } peers { permanent_uuid: "7cd7c680ef2e474c981ac81abadecfde" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 40117 } } peers { permanent_uuid: "62bbd271280e4b0a9ddf9ab46151e455" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 33917 } }
I20260709 13:18:28.005185 15860 raft_consensus.cc:385] T 48e8b6e82d7b42e6bdd4b0a8a25dfd9e P 7cd7c680ef2e474c981ac81abadecfde [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:28.005543 15860 raft_consensus.cc:740] T 48e8b6e82d7b42e6bdd4b0a8a25dfd9e P 7cd7c680ef2e474c981ac81abadecfde [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7cd7c680ef2e474c981ac81abadecfde, State: Initialized, Role: FOLLOWER
I20260709 13:18:28.006374 15860 consensus_queue.cc:260] T 48e8b6e82d7b42e6bdd4b0a8a25dfd9e P 7cd7c680ef2e474c981ac81abadecfde [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c1822e697d64b758a23b11c8fdeb59a" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 39067 } } peers { permanent_uuid: "7cd7c680ef2e474c981ac81abadecfde" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 40117 } } peers { permanent_uuid: "62bbd271280e4b0a9ddf9ab46151e455" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 33917 } }
I20260709 13:18:28.007872 15567 catalog_manager.cc:5697] T 8309d9a5a4ef4e6d914751bc0f410015 P 8c1822e697d64b758a23b11c8fdeb59a reported cstate change: term changed from 0 to 1, leader changed from <none> to 8c1822e697d64b758a23b11c8fdeb59a (127.14.177.129). New cstate: current_term: 1 leader_uuid: "8c1822e697d64b758a23b11c8fdeb59a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c1822e697d64b758a23b11c8fdeb59a" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 39067 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7cd7c680ef2e474c981ac81abadecfde" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 40117 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "62bbd271280e4b0a9ddf9ab46151e455" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 33917 } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:28.009275 15567 catalog_manager.cc:5697] T 408866bb47904e4189bc235ca5b0c7c7 P 8c1822e697d64b758a23b11c8fdeb59a reported cstate change: term changed from 0 to 1, leader changed from <none> to 8c1822e697d64b758a23b11c8fdeb59a (127.14.177.129). New cstate: current_term: 1 leader_uuid: "8c1822e697d64b758a23b11c8fdeb59a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c1822e697d64b758a23b11c8fdeb59a" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 39067 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7cd7c680ef2e474c981ac81abadecfde" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 40117 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "62bbd271280e4b0a9ddf9ab46151e455" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 33917 } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:28.010668 15860 ts_tablet_manager.cc:1434] T 48e8b6e82d7b42e6bdd4b0a8a25dfd9e P 7cd7c680ef2e474c981ac81abadecfde: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.000s
I20260709 13:18:28.011691 15860 tablet_bootstrap.cc:492] T 408866bb47904e4189bc235ca5b0c7c7 P 7cd7c680ef2e474c981ac81abadecfde: Bootstrap starting.
I20260709 13:18:28.017518 15860 tablet_bootstrap.cc:654] T 408866bb47904e4189bc235ca5b0c7c7 P 7cd7c680ef2e474c981ac81abadecfde: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:28.023846 15860 tablet_bootstrap.cc:492] T 408866bb47904e4189bc235ca5b0c7c7 P 7cd7c680ef2e474c981ac81abadecfde: No bootstrap required, opened a new log
I20260709 13:18:28.024226 15860 ts_tablet_manager.cc:1403] T 408866bb47904e4189bc235ca5b0c7c7 P 7cd7c680ef2e474c981ac81abadecfde: Time spent bootstrapping tablet: real 0.013s	user 0.008s	sys 0.001s
I20260709 13:18:28.025981 15860 raft_consensus.cc:359] T 408866bb47904e4189bc235ca5b0c7c7 P 7cd7c680ef2e474c981ac81abadecfde [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c1822e697d64b758a23b11c8fdeb59a" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 39067 } } peers { permanent_uuid: "7cd7c680ef2e474c981ac81abadecfde" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 40117 } } peers { permanent_uuid: "62bbd271280e4b0a9ddf9ab46151e455" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 33917 } }
I20260709 13:18:28.026464 15860 raft_consensus.cc:385] T 408866bb47904e4189bc235ca5b0c7c7 P 7cd7c680ef2e474c981ac81abadecfde [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:28.026676 15860 raft_consensus.cc:740] T 408866bb47904e4189bc235ca5b0c7c7 P 7cd7c680ef2e474c981ac81abadecfde [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7cd7c680ef2e474c981ac81abadecfde, State: Initialized, Role: FOLLOWER
I20260709 13:18:28.027254 15860 consensus_queue.cc:260] T 408866bb47904e4189bc235ca5b0c7c7 P 7cd7c680ef2e474c981ac81abadecfde [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c1822e697d64b758a23b11c8fdeb59a" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 39067 } } peers { permanent_uuid: "7cd7c680ef2e474c981ac81abadecfde" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 40117 } } peers { permanent_uuid: "62bbd271280e4b0a9ddf9ab46151e455" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 33917 } }
I20260709 13:18:28.028908 15860 ts_tablet_manager.cc:1434] T 408866bb47904e4189bc235ca5b0c7c7 P 7cd7c680ef2e474c981ac81abadecfde: Time spent starting tablet: real 0.004s	user 0.004s	sys 0.000s
I20260709 13:18:28.029865 15860 tablet_bootstrap.cc:492] T 502d133e880b4a859d8674b9d0cb6931 P 7cd7c680ef2e474c981ac81abadecfde: Bootstrap starting.
I20260709 13:18:28.033568 15886 raft_consensus.cc:493] T 408866bb47904e4189bc235ca5b0c7c7 P 7cd7c680ef2e474c981ac81abadecfde [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:28.033996 15886 raft_consensus.cc:515] T 408866bb47904e4189bc235ca5b0c7c7 P 7cd7c680ef2e474c981ac81abadecfde [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c1822e697d64b758a23b11c8fdeb59a" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 39067 } } peers { permanent_uuid: "7cd7c680ef2e474c981ac81abadecfde" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 40117 } } peers { permanent_uuid: "62bbd271280e4b0a9ddf9ab46151e455" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 33917 } }
I20260709 13:18:28.035130 15860 tablet_bootstrap.cc:654] T 502d133e880b4a859d8674b9d0cb6931 P 7cd7c680ef2e474c981ac81abadecfde: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:28.036146 15886 leader_election.cc:290] T 408866bb47904e4189bc235ca5b0c7c7 P 7cd7c680ef2e474c981ac81abadecfde [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8c1822e697d64b758a23b11c8fdeb59a (127.14.177.129:39067), 62bbd271280e4b0a9ddf9ab46151e455 (127.14.177.131:33917)
I20260709 13:18:28.036953 15670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "408866bb47904e4189bc235ca5b0c7c7" candidate_uuid: "7cd7c680ef2e474c981ac81abadecfde" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c1822e697d64b758a23b11c8fdeb59a" is_pre_election: true
I20260709 13:18:28.037648 15820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "408866bb47904e4189bc235ca5b0c7c7" candidate_uuid: "7cd7c680ef2e474c981ac81abadecfde" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "62bbd271280e4b0a9ddf9ab46151e455" is_pre_election: true
I20260709 13:18:28.038407 15820 raft_consensus.cc:2393] T 408866bb47904e4189bc235ca5b0c7c7 P 62bbd271280e4b0a9ddf9ab46151e455 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7cd7c680ef2e474c981ac81abadecfde in current term 1: Already voted for candidate 8c1822e697d64b758a23b11c8fdeb59a in this term.
I20260709 13:18:28.039657 15712 leader_election.cc:304] T 408866bb47904e4189bc235ca5b0c7c7 P 7cd7c680ef2e474c981ac81abadecfde [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7cd7c680ef2e474c981ac81abadecfde; no voters: 62bbd271280e4b0a9ddf9ab46151e455, 8c1822e697d64b758a23b11c8fdeb59a
I20260709 13:18:28.040349 15886 raft_consensus.cc:3060] T 408866bb47904e4189bc235ca5b0c7c7 P 7cd7c680ef2e474c981ac81abadecfde [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:28.041435 15860 tablet_bootstrap.cc:492] T 502d133e880b4a859d8674b9d0cb6931 P 7cd7c680ef2e474c981ac81abadecfde: No bootstrap required, opened a new log
I20260709 13:18:28.041807 15860 ts_tablet_manager.cc:1403] T 502d133e880b4a859d8674b9d0cb6931 P 7cd7c680ef2e474c981ac81abadecfde: Time spent bootstrapping tablet: real 0.012s	user 0.010s	sys 0.000s
I20260709 13:18:28.043716 15860 raft_consensus.cc:359] T 502d133e880b4a859d8674b9d0cb6931 P 7cd7c680ef2e474c981ac81abadecfde [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c1822e697d64b758a23b11c8fdeb59a" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 39067 } } peers { permanent_uuid: "7cd7c680ef2e474c981ac81abadecfde" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 40117 } } peers { permanent_uuid: "62bbd271280e4b0a9ddf9ab46151e455" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 33917 } }
I20260709 13:18:28.044155 15860 raft_consensus.cc:385] T 502d133e880b4a859d8674b9d0cb6931 P 7cd7c680ef2e474c981ac81abadecfde [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:28.044376 15860 raft_consensus.cc:740] T 502d133e880b4a859d8674b9d0cb6931 P 7cd7c680ef2e474c981ac81abadecfde [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7cd7c680ef2e474c981ac81abadecfde, State: Initialized, Role: FOLLOWER
I20260709 13:18:28.045135 15886 raft_consensus.cc:2749] T 408866bb47904e4189bc235ca5b0c7c7 P 7cd7c680ef2e474c981ac81abadecfde [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:18:28.044955 15860 consensus_queue.cc:260] T 502d133e880b4a859d8674b9d0cb6931 P 7cd7c680ef2e474c981ac81abadecfde [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c1822e697d64b758a23b11c8fdeb59a" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 39067 } } peers { permanent_uuid: "7cd7c680ef2e474c981ac81abadecfde" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 40117 } } peers { permanent_uuid: "62bbd271280e4b0a9ddf9ab46151e455" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 33917 } }
I20260709 13:18:28.046810 15860 ts_tablet_manager.cc:1434] T 502d133e880b4a859d8674b9d0cb6931 P 7cd7c680ef2e474c981ac81abadecfde: Time spent starting tablet: real 0.005s	user 0.006s	sys 0.000s
I20260709 13:18:28.129985 15868 raft_consensus.cc:493] T 48e8b6e82d7b42e6bdd4b0a8a25dfd9e P 8c1822e697d64b758a23b11c8fdeb59a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:28.130578 15868 raft_consensus.cc:515] T 48e8b6e82d7b42e6bdd4b0a8a25dfd9e P 8c1822e697d64b758a23b11c8fdeb59a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c1822e697d64b758a23b11c8fdeb59a" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 39067 } } peers { permanent_uuid: "7cd7c680ef2e474c981ac81abadecfde" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 40117 } } peers { permanent_uuid: "62bbd271280e4b0a9ddf9ab46151e455" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 33917 } }
I20260709 13:18:28.132536 15868 leader_election.cc:290] T 48e8b6e82d7b42e6bdd4b0a8a25dfd9e P 8c1822e697d64b758a23b11c8fdeb59a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7cd7c680ef2e474c981ac81abadecfde (127.14.177.130:40117), 62bbd271280e4b0a9ddf9ab46151e455 (127.14.177.131:33917)
I20260709 13:18:28.133605 15746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48e8b6e82d7b42e6bdd4b0a8a25dfd9e" candidate_uuid: "8c1822e697d64b758a23b11c8fdeb59a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cd7c680ef2e474c981ac81abadecfde" is_pre_election: true
I20260709 13:18:28.133737 15820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48e8b6e82d7b42e6bdd4b0a8a25dfd9e" candidate_uuid: "8c1822e697d64b758a23b11c8fdeb59a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "62bbd271280e4b0a9ddf9ab46151e455" is_pre_election: true
I20260709 13:18:28.134243 15746 raft_consensus.cc:2468] T 48e8b6e82d7b42e6bdd4b0a8a25dfd9e P 7cd7c680ef2e474c981ac81abadecfde [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c1822e697d64b758a23b11c8fdeb59a in term 0.
I20260709 13:18:28.134373 15820 raft_consensus.cc:2468] T 48e8b6e82d7b42e6bdd4b0a8a25dfd9e P 62bbd271280e4b0a9ddf9ab46151e455 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c1822e697d64b758a23b11c8fdeb59a in term 0.
I20260709 13:18:28.135329 15636 leader_election.cc:304] T 48e8b6e82d7b42e6bdd4b0a8a25dfd9e P 8c1822e697d64b758a23b11c8fdeb59a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7cd7c680ef2e474c981ac81abadecfde, 8c1822e697d64b758a23b11c8fdeb59a; no voters: 
I20260709 13:18:28.136036 15868 raft_consensus.cc:2804] T 48e8b6e82d7b42e6bdd4b0a8a25dfd9e P 8c1822e697d64b758a23b11c8fdeb59a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:18:28.136368 15868 raft_consensus.cc:493] T 48e8b6e82d7b42e6bdd4b0a8a25dfd9e P 8c1822e697d64b758a23b11c8fdeb59a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:28.136600 15868 raft_consensus.cc:3060] T 48e8b6e82d7b42e6bdd4b0a8a25dfd9e P 8c1822e697d64b758a23b11c8fdeb59a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:28.138353 15866 raft_consensus.cc:493] T 502d133e880b4a859d8674b9d0cb6931 P 62bbd271280e4b0a9ddf9ab46151e455 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:28.138828 15866 raft_consensus.cc:515] T 502d133e880b4a859d8674b9d0cb6931 P 62bbd271280e4b0a9ddf9ab46151e455 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c1822e697d64b758a23b11c8fdeb59a" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 39067 } } peers { permanent_uuid: "7cd7c680ef2e474c981ac81abadecfde" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 40117 } } peers { permanent_uuid: "62bbd271280e4b0a9ddf9ab46151e455" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 33917 } }
I20260709 13:18:28.141017 15866 leader_election.cc:290] T 502d133e880b4a859d8674b9d0cb6931 P 62bbd271280e4b0a9ddf9ab46151e455 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8c1822e697d64b758a23b11c8fdeb59a (127.14.177.129:39067), 7cd7c680ef2e474c981ac81abadecfde (127.14.177.130:40117)
I20260709 13:18:28.141820 15670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "502d133e880b4a859d8674b9d0cb6931" candidate_uuid: "62bbd271280e4b0a9ddf9ab46151e455" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c1822e697d64b758a23b11c8fdeb59a" is_pre_election: true
I20260709 13:18:28.142185 15746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "502d133e880b4a859d8674b9d0cb6931" candidate_uuid: "62bbd271280e4b0a9ddf9ab46151e455" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cd7c680ef2e474c981ac81abadecfde" is_pre_election: true
I20260709 13:18:28.142483 15670 raft_consensus.cc:2468] T 502d133e880b4a859d8674b9d0cb6931 P 8c1822e697d64b758a23b11c8fdeb59a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 62bbd271280e4b0a9ddf9ab46151e455 in term 0.
I20260709 13:18:28.142835 15746 raft_consensus.cc:2468] T 502d133e880b4a859d8674b9d0cb6931 P 7cd7c680ef2e474c981ac81abadecfde [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 62bbd271280e4b0a9ddf9ab46151e455 in term 0.
I20260709 13:18:28.142875 15868 raft_consensus.cc:515] T 48e8b6e82d7b42e6bdd4b0a8a25dfd9e P 8c1822e697d64b758a23b11c8fdeb59a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c1822e697d64b758a23b11c8fdeb59a" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 39067 } } peers { permanent_uuid: "7cd7c680ef2e474c981ac81abadecfde" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 40117 } } peers { permanent_uuid: "62bbd271280e4b0a9ddf9ab46151e455" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 33917 } }
I20260709 13:18:28.143668 15783 leader_election.cc:304] T 502d133e880b4a859d8674b9d0cb6931 P 62bbd271280e4b0a9ddf9ab46151e455 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 62bbd271280e4b0a9ddf9ab46151e455, 8c1822e697d64b758a23b11c8fdeb59a; no voters: 
I20260709 13:18:28.144436 15866 raft_consensus.cc:2804] T 502d133e880b4a859d8674b9d0cb6931 P 62bbd271280e4b0a9ddf9ab46151e455 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:18:28.144783 15866 raft_consensus.cc:493] T 502d133e880b4a859d8674b9d0cb6931 P 62bbd271280e4b0a9ddf9ab46151e455 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:28.145071 15868 leader_election.cc:290] T 48e8b6e82d7b42e6bdd4b0a8a25dfd9e P 8c1822e697d64b758a23b11c8fdeb59a [CANDIDATE]: Term 1 election: Requested vote from peers 7cd7c680ef2e474c981ac81abadecfde (127.14.177.130:40117), 62bbd271280e4b0a9ddf9ab46151e455 (127.14.177.131:33917)
I20260709 13:18:28.145100 15866 raft_consensus.cc:3060] T 502d133e880b4a859d8674b9d0cb6931 P 62bbd271280e4b0a9ddf9ab46151e455 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:28.145874 15746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48e8b6e82d7b42e6bdd4b0a8a25dfd9e" candidate_uuid: "8c1822e697d64b758a23b11c8fdeb59a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cd7c680ef2e474c981ac81abadecfde"
I20260709 13:18:28.146119 15820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48e8b6e82d7b42e6bdd4b0a8a25dfd9e" candidate_uuid: "8c1822e697d64b758a23b11c8fdeb59a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "62bbd271280e4b0a9ddf9ab46151e455"
I20260709 13:18:28.146497 15746 raft_consensus.cc:3060] T 48e8b6e82d7b42e6bdd4b0a8a25dfd9e P 7cd7c680ef2e474c981ac81abadecfde [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:28.146682 15820 raft_consensus.cc:3060] T 48e8b6e82d7b42e6bdd4b0a8a25dfd9e P 62bbd271280e4b0a9ddf9ab46151e455 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:28.152210 15746 raft_consensus.cc:2468] T 48e8b6e82d7b42e6bdd4b0a8a25dfd9e P 7cd7c680ef2e474c981ac81abadecfde [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c1822e697d64b758a23b11c8fdeb59a in term 1.
I20260709 13:18:28.152858 15866 raft_consensus.cc:515] T 502d133e880b4a859d8674b9d0cb6931 P 62bbd271280e4b0a9ddf9ab46151e455 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c1822e697d64b758a23b11c8fdeb59a" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 39067 } } peers { permanent_uuid: "7cd7c680ef2e474c981ac81abadecfde" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 40117 } } peers { permanent_uuid: "62bbd271280e4b0a9ddf9ab46151e455" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 33917 } }
I20260709 13:18:28.153237 15820 raft_consensus.cc:2468] T 48e8b6e82d7b42e6bdd4b0a8a25dfd9e P 62bbd271280e4b0a9ddf9ab46151e455 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c1822e697d64b758a23b11c8fdeb59a in term 1.
I20260709 13:18:28.153194 15636 leader_election.cc:304] T 48e8b6e82d7b42e6bdd4b0a8a25dfd9e P 8c1822e697d64b758a23b11c8fdeb59a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7cd7c680ef2e474c981ac81abadecfde, 8c1822e697d64b758a23b11c8fdeb59a; no voters: 
I20260709 13:18:28.154640 15868 raft_consensus.cc:2804] T 48e8b6e82d7b42e6bdd4b0a8a25dfd9e P 8c1822e697d64b758a23b11c8fdeb59a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:18:28.155115 15868 raft_consensus.cc:697] T 48e8b6e82d7b42e6bdd4b0a8a25dfd9e P 8c1822e697d64b758a23b11c8fdeb59a [term 1 LEADER]: Becoming Leader. State: Replica: 8c1822e697d64b758a23b11c8fdeb59a, State: Running, Role: LEADER
I20260709 13:18:28.156170 15866 leader_election.cc:290] T 502d133e880b4a859d8674b9d0cb6931 P 62bbd271280e4b0a9ddf9ab46151e455 [CANDIDATE]: Term 1 election: Requested vote from peers 8c1822e697d64b758a23b11c8fdeb59a (127.14.177.129:39067), 7cd7c680ef2e474c981ac81abadecfde (127.14.177.130:40117)
I20260709 13:18:28.155975 15868 consensus_queue.cc:237] T 48e8b6e82d7b42e6bdd4b0a8a25dfd9e P 8c1822e697d64b758a23b11c8fdeb59a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c1822e697d64b758a23b11c8fdeb59a" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 39067 } } peers { permanent_uuid: "7cd7c680ef2e474c981ac81abadecfde" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 40117 } } peers { permanent_uuid: "62bbd271280e4b0a9ddf9ab46151e455" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 33917 } }
I20260709 13:18:28.156370 15670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "502d133e880b4a859d8674b9d0cb6931" candidate_uuid: "62bbd271280e4b0a9ddf9ab46151e455" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c1822e697d64b758a23b11c8fdeb59a"
I20260709 13:18:28.157042 15670 raft_consensus.cc:3060] T 502d133e880b4a859d8674b9d0cb6931 P 8c1822e697d64b758a23b11c8fdeb59a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:28.157266 15746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "502d133e880b4a859d8674b9d0cb6931" candidate_uuid: "62bbd271280e4b0a9ddf9ab46151e455" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cd7c680ef2e474c981ac81abadecfde"
I20260709 13:18:28.157800 15746 raft_consensus.cc:3060] T 502d133e880b4a859d8674b9d0cb6931 P 7cd7c680ef2e474c981ac81abadecfde [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:28.163614 15670 raft_consensus.cc:2468] T 502d133e880b4a859d8674b9d0cb6931 P 8c1822e697d64b758a23b11c8fdeb59a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 62bbd271280e4b0a9ddf9ab46151e455 in term 1.
I20260709 13:18:28.164309 15746 raft_consensus.cc:2468] T 502d133e880b4a859d8674b9d0cb6931 P 7cd7c680ef2e474c981ac81abadecfde [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 62bbd271280e4b0a9ddf9ab46151e455 in term 1.
I20260709 13:18:28.164819 15783 leader_election.cc:304] T 502d133e880b4a859d8674b9d0cb6931 P 62bbd271280e4b0a9ddf9ab46151e455 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 62bbd271280e4b0a9ddf9ab46151e455, 8c1822e697d64b758a23b11c8fdeb59a; no voters: 
I20260709 13:18:28.165678 15866 raft_consensus.cc:2804] T 502d133e880b4a859d8674b9d0cb6931 P 62bbd271280e4b0a9ddf9ab46151e455 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:18:28.166107 15866 raft_consensus.cc:697] T 502d133e880b4a859d8674b9d0cb6931 P 62bbd271280e4b0a9ddf9ab46151e455 [term 1 LEADER]: Becoming Leader. State: Replica: 62bbd271280e4b0a9ddf9ab46151e455, State: Running, Role: LEADER
I20260709 13:18:28.165992 15567 catalog_manager.cc:5697] T 48e8b6e82d7b42e6bdd4b0a8a25dfd9e P 8c1822e697d64b758a23b11c8fdeb59a reported cstate change: term changed from 0 to 1, leader changed from <none> to 8c1822e697d64b758a23b11c8fdeb59a (127.14.177.129). New cstate: current_term: 1 leader_uuid: "8c1822e697d64b758a23b11c8fdeb59a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c1822e697d64b758a23b11c8fdeb59a" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 39067 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7cd7c680ef2e474c981ac81abadecfde" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 40117 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "62bbd271280e4b0a9ddf9ab46151e455" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 33917 } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:28.166829 15866 consensus_queue.cc:237] T 502d133e880b4a859d8674b9d0cb6931 P 62bbd271280e4b0a9ddf9ab46151e455 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c1822e697d64b758a23b11c8fdeb59a" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 39067 } } peers { permanent_uuid: "7cd7c680ef2e474c981ac81abadecfde" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 40117 } } peers { permanent_uuid: "62bbd271280e4b0a9ddf9ab46151e455" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 33917 } }
I20260709 13:18:28.175338 15569 catalog_manager.cc:5697] T 502d133e880b4a859d8674b9d0cb6931 P 62bbd271280e4b0a9ddf9ab46151e455 reported cstate change: term changed from 0 to 1, leader changed from <none> to 62bbd271280e4b0a9ddf9ab46151e455 (127.14.177.131). New cstate: current_term: 1 leader_uuid: "62bbd271280e4b0a9ddf9ab46151e455" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c1822e697d64b758a23b11c8fdeb59a" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 39067 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7cd7c680ef2e474c981ac81abadecfde" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 40117 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "62bbd271280e4b0a9ddf9ab46151e455" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 33917 } health_report { overall_health: HEALTHY } } }
I20260709 13:18:28.188683 15886 consensus_queue.cc:1048] T 8cff7911abd649b9a276ddae431311d1 P 7cd7c680ef2e474c981ac81abadecfde [LEADER]: Connected to new peer: Peer: permanent_uuid: "62bbd271280e4b0a9ddf9ab46151e455" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 33917 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:18:28.200593 15903 consensus_queue.cc:1048] T 8cff7911abd649b9a276ddae431311d1 P 7cd7c680ef2e474c981ac81abadecfde [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c1822e697d64b758a23b11c8fdeb59a" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 39067 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:18:28.220530 15884 consensus_queue.cc:1048] T 0ce7d20d574f47649d3ad0f3c382198d P 7cd7c680ef2e474c981ac81abadecfde [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c1822e697d64b758a23b11c8fdeb59a" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 39067 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:18:28.226735 15884 consensus_queue.cc:1048] T 0ce7d20d574f47649d3ad0f3c382198d P 7cd7c680ef2e474c981ac81abadecfde [LEADER]: Connected to new peer: Peer: permanent_uuid: "62bbd271280e4b0a9ddf9ab46151e455" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 33917 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:18:28.275476 15884 consensus_queue.cc:1048] T 566eae6d93af4ec492b9b1a270dc03b7 P 7cd7c680ef2e474c981ac81abadecfde [LEADER]: Connected to new peer: Peer: permanent_uuid: "62bbd271280e4b0a9ddf9ab46151e455" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 33917 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:18:28.286087 15884 consensus_queue.cc:1048] T 566eae6d93af4ec492b9b1a270dc03b7 P 7cd7c680ef2e474c981ac81abadecfde [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c1822e697d64b758a23b11c8fdeb59a" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 39067 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:18:28.312448 15884 consensus_queue.cc:1048] T 4d83a05b849340c186cc995649575d3b P 7cd7c680ef2e474c981ac81abadecfde [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c1822e697d64b758a23b11c8fdeb59a" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 39067 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:18:28.323045 15886 consensus_queue.cc:1048] T 4d83a05b849340c186cc995649575d3b P 7cd7c680ef2e474c981ac81abadecfde [LEADER]: Connected to new peer: Peer: permanent_uuid: "62bbd271280e4b0a9ddf9ab46151e455" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 33917 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:18:28.345031 15868 consensus_queue.cc:1048] T 17f97fe9a93f4a86b1e2bad621bfb561 P 8c1822e697d64b758a23b11c8fdeb59a [LEADER]: Connected to new peer: Peer: permanent_uuid: "62bbd271280e4b0a9ddf9ab46151e455" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 33917 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:18:28.348062 15868 consensus_queue.cc:1048] T 408866bb47904e4189bc235ca5b0c7c7 P 8c1822e697d64b758a23b11c8fdeb59a [LEADER]: Connected to new peer: Peer: permanent_uuid: "62bbd271280e4b0a9ddf9ab46151e455" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 33917 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:18:28.358973 15911 consensus_queue.cc:1048] T 17f97fe9a93f4a86b1e2bad621bfb561 P 8c1822e697d64b758a23b11c8fdeb59a [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cd7c680ef2e474c981ac81abadecfde" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 40117 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:18:28.364806 15911 consensus_queue.cc:1048] T 408866bb47904e4189bc235ca5b0c7c7 P 8c1822e697d64b758a23b11c8fdeb59a [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cd7c680ef2e474c981ac81abadecfde" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 40117 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:18:28.396080 15744 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4d83a05b849340c186cc995649575d3b"
dest_uuid: "7cd7c680ef2e474c981ac81abadecfde"
mode: GRACEFUL
new_leader_uuid: "8c1822e697d64b758a23b11c8fdeb59a"
 from {username='slave'} at 127.0.0.1:49470
I20260709 13:18:28.396663 15744 raft_consensus.cc:606] T 4d83a05b849340c186cc995649575d3b P 7cd7c680ef2e474c981ac81abadecfde [term 1 LEADER]: Received request to transfer leadership to TS 8c1822e697d64b758a23b11c8fdeb59a
I20260709 13:18:28.402335 15744 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8cff7911abd649b9a276ddae431311d1"
dest_uuid: "7cd7c680ef2e474c981ac81abadecfde"
mode: GRACEFUL
new_leader_uuid: "8c1822e697d64b758a23b11c8fdeb59a"
 from {username='slave'} at 127.0.0.1:49470
I20260709 13:18:28.402904 15744 raft_consensus.cc:606] T 8cff7911abd649b9a276ddae431311d1 P 7cd7c680ef2e474c981ac81abadecfde [term 1 LEADER]: Received request to transfer leadership to TS 8c1822e697d64b758a23b11c8fdeb59a
I20260709 13:18:28.408048 15744 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0ce7d20d574f47649d3ad0f3c382198d"
dest_uuid: "7cd7c680ef2e474c981ac81abadecfde"
mode: GRACEFUL
new_leader_uuid: "8c1822e697d64b758a23b11c8fdeb59a"
 from {username='slave'} at 127.0.0.1:49470
I20260709 13:18:28.408591 15744 raft_consensus.cc:606] T 0ce7d20d574f47649d3ad0f3c382198d P 7cd7c680ef2e474c981ac81abadecfde [term 1 LEADER]: Received request to transfer leadership to TS 8c1822e697d64b758a23b11c8fdeb59a
I20260709 13:18:28.413808 15744 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "566eae6d93af4ec492b9b1a270dc03b7"
dest_uuid: "7cd7c680ef2e474c981ac81abadecfde"
mode: GRACEFUL
new_leader_uuid: "8c1822e697d64b758a23b11c8fdeb59a"
 from {username='slave'} at 127.0.0.1:49470
I20260709 13:18:28.414369 15744 raft_consensus.cc:606] T 566eae6d93af4ec492b9b1a270dc03b7 P 7cd7c680ef2e474c981ac81abadecfde [term 1 LEADER]: Received request to transfer leadership to TS 8c1822e697d64b758a23b11c8fdeb59a
I20260709 13:18:28.427934 15819 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8fc26f7d12464423a212a1f595ca4b88"
dest_uuid: "62bbd271280e4b0a9ddf9ab46151e455"
mode: GRACEFUL
new_leader_uuid: "8c1822e697d64b758a23b11c8fdeb59a"
 from {username='slave'} at 127.0.0.1:40902
I20260709 13:18:28.428471 15819 raft_consensus.cc:606] T 8fc26f7d12464423a212a1f595ca4b88 P 62bbd271280e4b0a9ddf9ab46151e455 [term 1 LEADER]: Received request to transfer leadership to TS 8c1822e697d64b758a23b11c8fdeb59a
I20260709 13:18:28.432813 15866 consensus_queue.cc:1048] T 8fc26f7d12464423a212a1f595ca4b88 P 62bbd271280e4b0a9ddf9ab46151e455 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c1822e697d64b758a23b11c8fdeb59a" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 39067 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:18:28.441745 15866 raft_consensus.cc:993] T 8fc26f7d12464423a212a1f595ca4b88 P 62bbd271280e4b0a9ddf9ab46151e455: : Instructing follower 8c1822e697d64b758a23b11c8fdeb59a to start an election
I20260709 13:18:28.442130 15866 raft_consensus.cc:1081] T 8fc26f7d12464423a212a1f595ca4b88 P 62bbd271280e4b0a9ddf9ab46151e455 [term 1 LEADER]: Signalling peer 8c1822e697d64b758a23b11c8fdeb59a to start an election
I20260709 13:18:28.444162 15671 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "8fc26f7d12464423a212a1f595ca4b88"
dest_uuid: "8c1822e697d64b758a23b11c8fdeb59a"
 from {username='slave'} at 127.0.0.1:42822
I20260709 13:18:28.444813 15671 raft_consensus.cc:493] T 8fc26f7d12464423a212a1f595ca4b88 P 8c1822e697d64b758a23b11c8fdeb59a [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:18:28.445238 15671 raft_consensus.cc:3060] T 8fc26f7d12464423a212a1f595ca4b88 P 8c1822e697d64b758a23b11c8fdeb59a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:28.447578 15669 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "408866bb47904e4189bc235ca5b0c7c7"
dest_uuid: "8c1822e697d64b758a23b11c8fdeb59a"
mode: GRACEFUL
new_leader_uuid: "7cd7c680ef2e474c981ac81abadecfde"
 from {username='slave'} at 127.0.0.1:42832
I20260709 13:18:28.448081 15669 raft_consensus.cc:606] T 408866bb47904e4189bc235ca5b0c7c7 P 8c1822e697d64b758a23b11c8fdeb59a [term 1 LEADER]: Received request to transfer leadership to TS 7cd7c680ef2e474c981ac81abadecfde
I20260709 13:18:28.455035 15669 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "17f97fe9a93f4a86b1e2bad621bfb561"
dest_uuid: "8c1822e697d64b758a23b11c8fdeb59a"
mode: GRACEFUL
new_leader_uuid: "7cd7c680ef2e474c981ac81abadecfde"
 from {username='slave'} at 127.0.0.1:42832
I20260709 13:18:28.455655 15669 raft_consensus.cc:606] T 17f97fe9a93f4a86b1e2bad621bfb561 P 8c1822e697d64b758a23b11c8fdeb59a [term 1 LEADER]: Received request to transfer leadership to TS 7cd7c680ef2e474c981ac81abadecfde
I20260709 13:18:28.456386 15671 raft_consensus.cc:515] T 8fc26f7d12464423a212a1f595ca4b88 P 8c1822e697d64b758a23b11c8fdeb59a [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c1822e697d64b758a23b11c8fdeb59a" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 39067 } } peers { permanent_uuid: "7cd7c680ef2e474c981ac81abadecfde" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 40117 } } peers { permanent_uuid: "62bbd271280e4b0a9ddf9ab46151e455" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 33917 } }
I20260709 13:18:28.457387 15670 raft_consensus.cc:1240] T 8fc26f7d12464423a212a1f595ca4b88 P 8c1822e697d64b758a23b11c8fdeb59a [term 2 FOLLOWER]: Rejecting Update request from peer 62bbd271280e4b0a9ddf9ab46151e455 for earlier term 1. Current term is 2. Ops: []
I20260709 13:18:28.459692 15744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc26f7d12464423a212a1f595ca4b88" candidate_uuid: "8c1822e697d64b758a23b11c8fdeb59a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "7cd7c680ef2e474c981ac81abadecfde"
I20260709 13:18:28.459688 15819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc26f7d12464423a212a1f595ca4b88" candidate_uuid: "8c1822e697d64b758a23b11c8fdeb59a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "62bbd271280e4b0a9ddf9ab46151e455"
I20260709 13:18:28.460529 15744 raft_consensus.cc:3060] T 8fc26f7d12464423a212a1f595ca4b88 P 7cd7c680ef2e474c981ac81abadecfde [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:28.460943 15819 raft_consensus.cc:3055] T 8fc26f7d12464423a212a1f595ca4b88 P 62bbd271280e4b0a9ddf9ab46151e455 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:18:28.461287 15819 raft_consensus.cc:740] T 8fc26f7d12464423a212a1f595ca4b88 P 62bbd271280e4b0a9ddf9ab46151e455 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 62bbd271280e4b0a9ddf9ab46151e455, State: Running, Role: LEADER
I20260709 13:18:28.461969 15819 consensus_queue.cc:260] T 8fc26f7d12464423a212a1f595ca4b88 P 62bbd271280e4b0a9ddf9ab46151e455 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c1822e697d64b758a23b11c8fdeb59a" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 39067 } } peers { permanent_uuid: "7cd7c680ef2e474c981ac81abadecfde" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 40117 } } peers { permanent_uuid: "62bbd271280e4b0a9ddf9ab46151e455" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 33917 } }
I20260709 13:18:28.463132 15670 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8309d9a5a4ef4e6d914751bc0f410015"
dest_uuid: "8c1822e697d64b758a23b11c8fdeb59a"
mode: GRACEFUL
new_leader_uuid: "7cd7c680ef2e474c981ac81abadecfde"
 from {username='slave'} at 127.0.0.1:42832
I20260709 13:18:28.463788 15670 raft_consensus.cc:606] T 8309d9a5a4ef4e6d914751bc0f410015 P 8c1822e697d64b758a23b11c8fdeb59a [term 1 LEADER]: Received request to transfer leadership to TS 7cd7c680ef2e474c981ac81abadecfde
I20260709 13:18:28.467573 15744 raft_consensus.cc:2468] T 8fc26f7d12464423a212a1f595ca4b88 P 7cd7c680ef2e474c981ac81abadecfde [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c1822e697d64b758a23b11c8fdeb59a in term 2.
I20260709 13:18:28.468052 15671 leader_election.cc:290] T 8fc26f7d12464423a212a1f595ca4b88 P 8c1822e697d64b758a23b11c8fdeb59a [CANDIDATE]: Term 2 election: Requested vote from peers 7cd7c680ef2e474c981ac81abadecfde (127.14.177.130:40117), 62bbd271280e4b0a9ddf9ab46151e455 (127.14.177.131:33917)
I20260709 13:18:28.469241 15636 leader_election.cc:304] T 8fc26f7d12464423a212a1f595ca4b88 P 8c1822e697d64b758a23b11c8fdeb59a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7cd7c680ef2e474c981ac81abadecfde, 8c1822e697d64b758a23b11c8fdeb59a; no voters: 
I20260709 13:18:28.470885 15911 raft_consensus.cc:2804] T 8fc26f7d12464423a212a1f595ca4b88 P 8c1822e697d64b758a23b11c8fdeb59a [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:18:28.471547 15911 raft_consensus.cc:697] T 8fc26f7d12464423a212a1f595ca4b88 P 8c1822e697d64b758a23b11c8fdeb59a [term 2 LEADER]: Becoming Leader. State: Replica: 8c1822e697d64b758a23b11c8fdeb59a, State: Running, Role: LEADER
I20260709 13:18:28.471364 15671 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "48e8b6e82d7b42e6bdd4b0a8a25dfd9e"
dest_uuid: "8c1822e697d64b758a23b11c8fdeb59a"
mode: GRACEFUL
new_leader_uuid: "7cd7c680ef2e474c981ac81abadecfde"
 from {username='slave'} at 127.0.0.1:42832
I20260709 13:18:28.472347 15671 raft_consensus.cc:606] T 48e8b6e82d7b42e6bdd4b0a8a25dfd9e P 8c1822e697d64b758a23b11c8fdeb59a [term 1 LEADER]: Received request to transfer leadership to TS 7cd7c680ef2e474c981ac81abadecfde
I20260709 13:18:28.472656 15911 consensus_queue.cc:237] T 8fc26f7d12464423a212a1f595ca4b88 P 8c1822e697d64b758a23b11c8fdeb59a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c1822e697d64b758a23b11c8fdeb59a" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 39067 } } peers { permanent_uuid: "7cd7c680ef2e474c981ac81abadecfde" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 40117 } } peers { permanent_uuid: "62bbd271280e4b0a9ddf9ab46151e455" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 33917 } }
W20260709 13:18:28.463459 15945 consensus_queue.cc:1175] T 8fc26f7d12464423a212a1f595ca4b88 P 62bbd271280e4b0a9ddf9ab46151e455 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "7cd7c680ef2e474c981ac81abadecfde" responder_term: 1 status { last_received { term: 0 index: 0 } last_committed_idx: 0 error { code: PRECEDING_ENTRY_DIDNT_MATCH status { code: ILLEGAL_STATE message: "Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 1. (index mismatch)" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:18:28.474547 15910 consensus_queue.cc:1059] T 8fc26f7d12464423a212a1f595ca4b88 P 62bbd271280e4b0a9ddf9ab46151e455 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8c1822e697d64b758a23b11c8fdeb59a" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 39067 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:18:28.478942 15819 raft_consensus.cc:3060] T 8fc26f7d12464423a212a1f595ca4b88 P 62bbd271280e4b0a9ddf9ab46151e455 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:28.482509 15567 catalog_manager.cc:5697] T 8fc26f7d12464423a212a1f595ca4b88 P 8c1822e697d64b758a23b11c8fdeb59a reported cstate change: term changed from 1 to 2, leader changed from 62bbd271280e4b0a9ddf9ab46151e455 (127.14.177.131) to 8c1822e697d64b758a23b11c8fdeb59a (127.14.177.129). New cstate: current_term: 2 leader_uuid: "8c1822e697d64b758a23b11c8fdeb59a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c1822e697d64b758a23b11c8fdeb59a" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 39067 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7cd7c680ef2e474c981ac81abadecfde" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 40117 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "62bbd271280e4b0a9ddf9ab46151e455" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 33917 } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:28.485894 15819 raft_consensus.cc:2468] T 8fc26f7d12464423a212a1f595ca4b88 P 62bbd271280e4b0a9ddf9ab46151e455 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c1822e697d64b758a23b11c8fdeb59a in term 2.
I20260709 13:18:28.495574 15911 consensus_queue.cc:1048] T 8309d9a5a4ef4e6d914751bc0f410015 P 8c1822e697d64b758a23b11c8fdeb59a [LEADER]: Connected to new peer: Peer: permanent_uuid: "62bbd271280e4b0a9ddf9ab46151e455" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 33917 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:18:28.506282 15868 consensus_queue.cc:1048] T 8309d9a5a4ef4e6d914751bc0f410015 P 8c1822e697d64b758a23b11c8fdeb59a [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cd7c680ef2e474c981ac81abadecfde" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 40117 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:18:28.514626 15911 raft_consensus.cc:993] T 8309d9a5a4ef4e6d914751bc0f410015 P 8c1822e697d64b758a23b11c8fdeb59a: : Instructing follower 7cd7c680ef2e474c981ac81abadecfde to start an election
I20260709 13:18:28.515093 15868 raft_consensus.cc:1081] T 8309d9a5a4ef4e6d914751bc0f410015 P 8c1822e697d64b758a23b11c8fdeb59a [term 1 LEADER]: Signalling peer 7cd7c680ef2e474c981ac81abadecfde to start an election
I20260709 13:18:28.516816 15744 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "8309d9a5a4ef4e6d914751bc0f410015"
dest_uuid: "7cd7c680ef2e474c981ac81abadecfde"
 from {username='slave'} at 127.0.0.1:49440
I20260709 13:18:28.517388 15744 raft_consensus.cc:493] T 8309d9a5a4ef4e6d914751bc0f410015 P 7cd7c680ef2e474c981ac81abadecfde [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:18:28.517768 15744 raft_consensus.cc:3060] T 8309d9a5a4ef4e6d914751bc0f410015 P 7cd7c680ef2e474c981ac81abadecfde [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:28.522415 15744 raft_consensus.cc:515] T 8309d9a5a4ef4e6d914751bc0f410015 P 7cd7c680ef2e474c981ac81abadecfde [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c1822e697d64b758a23b11c8fdeb59a" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 39067 } } peers { permanent_uuid: "7cd7c680ef2e474c981ac81abadecfde" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 40117 } } peers { permanent_uuid: "62bbd271280e4b0a9ddf9ab46151e455" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 33917 } }
I20260709 13:18:28.524412 15744 leader_election.cc:290] T 8309d9a5a4ef4e6d914751bc0f410015 P 7cd7c680ef2e474c981ac81abadecfde [CANDIDATE]: Term 2 election: Requested vote from peers 8c1822e697d64b758a23b11c8fdeb59a (127.14.177.129:39067), 62bbd271280e4b0a9ddf9ab46151e455 (127.14.177.131:33917)
I20260709 13:18:28.525115 15671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8309d9a5a4ef4e6d914751bc0f410015" candidate_uuid: "7cd7c680ef2e474c981ac81abadecfde" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8c1822e697d64b758a23b11c8fdeb59a"
I20260709 13:18:28.525450 15819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8309d9a5a4ef4e6d914751bc0f410015" candidate_uuid: "7cd7c680ef2e474c981ac81abadecfde" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "62bbd271280e4b0a9ddf9ab46151e455"
I20260709 13:18:28.525732 15671 raft_consensus.cc:3055] T 8309d9a5a4ef4e6d914751bc0f410015 P 8c1822e697d64b758a23b11c8fdeb59a [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:18:28.526129 15819 raft_consensus.cc:3060] T 8309d9a5a4ef4e6d914751bc0f410015 P 62bbd271280e4b0a9ddf9ab46151e455 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:28.526137 15671 raft_consensus.cc:740] T 8309d9a5a4ef4e6d914751bc0f410015 P 8c1822e697d64b758a23b11c8fdeb59a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8c1822e697d64b758a23b11c8fdeb59a, State: Running, Role: LEADER
I20260709 13:18:28.527360 15671 consensus_queue.cc:260] T 8309d9a5a4ef4e6d914751bc0f410015 P 8c1822e697d64b758a23b11c8fdeb59a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c1822e697d64b758a23b11c8fdeb59a" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 39067 } } peers { permanent_uuid: "7cd7c680ef2e474c981ac81abadecfde" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 40117 } } peers { permanent_uuid: "62bbd271280e4b0a9ddf9ab46151e455" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 33917 } }
I20260709 13:18:28.528738 15671 raft_consensus.cc:3060] T 8309d9a5a4ef4e6d914751bc0f410015 P 8c1822e697d64b758a23b11c8fdeb59a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:28.531733 15819 raft_consensus.cc:2468] T 8309d9a5a4ef4e6d914751bc0f410015 P 62bbd271280e4b0a9ddf9ab46151e455 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7cd7c680ef2e474c981ac81abadecfde in term 2.
I20260709 13:18:28.532766 15712 leader_election.cc:304] T 8309d9a5a4ef4e6d914751bc0f410015 P 7cd7c680ef2e474c981ac81abadecfde [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 62bbd271280e4b0a9ddf9ab46151e455, 7cd7c680ef2e474c981ac81abadecfde; no voters: 
I20260709 13:18:28.533471 15886 raft_consensus.cc:2804] T 8309d9a5a4ef4e6d914751bc0f410015 P 7cd7c680ef2e474c981ac81abadecfde [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:18:28.533931 15886 raft_consensus.cc:697] T 8309d9a5a4ef4e6d914751bc0f410015 P 7cd7c680ef2e474c981ac81abadecfde [term 2 LEADER]: Becoming Leader. State: Replica: 7cd7c680ef2e474c981ac81abadecfde, State: Running, Role: LEADER
I20260709 13:18:28.534538 15671 raft_consensus.cc:2468] T 8309d9a5a4ef4e6d914751bc0f410015 P 8c1822e697d64b758a23b11c8fdeb59a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7cd7c680ef2e474c981ac81abadecfde in term 2.
I20260709 13:18:28.534657 15886 consensus_queue.cc:237] T 8309d9a5a4ef4e6d914751bc0f410015 P 7cd7c680ef2e474c981ac81abadecfde [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c1822e697d64b758a23b11c8fdeb59a" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 39067 } } peers { permanent_uuid: "7cd7c680ef2e474c981ac81abadecfde" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 40117 } } peers { permanent_uuid: "62bbd271280e4b0a9ddf9ab46151e455" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 33917 } }
I20260709 13:18:28.541111 15568 catalog_manager.cc:5697] T 8309d9a5a4ef4e6d914751bc0f410015 P 7cd7c680ef2e474c981ac81abadecfde reported cstate change: term changed from 1 to 2, leader changed from 8c1822e697d64b758a23b11c8fdeb59a (127.14.177.129) to 7cd7c680ef2e474c981ac81abadecfde (127.14.177.130). New cstate: current_term: 2 leader_uuid: "7cd7c680ef2e474c981ac81abadecfde" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c1822e697d64b758a23b11c8fdeb59a" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 39067 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7cd7c680ef2e474c981ac81abadecfde" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 40117 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "62bbd271280e4b0a9ddf9ab46151e455" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 33917 } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:28.548300 15910 consensus_queue.cc:1048] T 502d133e880b4a859d8674b9d0cb6931 P 62bbd271280e4b0a9ddf9ab46151e455 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cd7c680ef2e474c981ac81abadecfde" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 40117 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:18:28.560792 15945 consensus_queue.cc:1048] T 502d133e880b4a859d8674b9d0cb6931 P 62bbd271280e4b0a9ddf9ab46151e455 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c1822e697d64b758a23b11c8fdeb59a" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 39067 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:18:28.584599 15868 consensus_queue.cc:1048] T 48e8b6e82d7b42e6bdd4b0a8a25dfd9e P 8c1822e697d64b758a23b11c8fdeb59a [LEADER]: Connected to new peer: Peer: permanent_uuid: "62bbd271280e4b0a9ddf9ab46151e455" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 33917 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:18:28.594754 15909 consensus_queue.cc:1048] T 48e8b6e82d7b42e6bdd4b0a8a25dfd9e P 8c1822e697d64b758a23b11c8fdeb59a [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cd7c680ef2e474c981ac81abadecfde" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 40117 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:18:28.602826 15868 raft_consensus.cc:993] T 48e8b6e82d7b42e6bdd4b0a8a25dfd9e P 8c1822e697d64b758a23b11c8fdeb59a: : Instructing follower 7cd7c680ef2e474c981ac81abadecfde to start an election
I20260709 13:18:28.603348 15909 raft_consensus.cc:1081] T 48e8b6e82d7b42e6bdd4b0a8a25dfd9e P 8c1822e697d64b758a23b11c8fdeb59a [term 1 LEADER]: Signalling peer 7cd7c680ef2e474c981ac81abadecfde to start an election
I20260709 13:18:28.605062 15744 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "48e8b6e82d7b42e6bdd4b0a8a25dfd9e"
dest_uuid: "7cd7c680ef2e474c981ac81abadecfde"
 from {username='slave'} at 127.0.0.1:49440
I20260709 13:18:28.605643 15744 raft_consensus.cc:493] T 48e8b6e82d7b42e6bdd4b0a8a25dfd9e P 7cd7c680ef2e474c981ac81abadecfde [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:18:28.606055 15744 raft_consensus.cc:3060] T 48e8b6e82d7b42e6bdd4b0a8a25dfd9e P 7cd7c680ef2e474c981ac81abadecfde [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:28.611032 15744 raft_consensus.cc:515] T 48e8b6e82d7b42e6bdd4b0a8a25dfd9e P 7cd7c680ef2e474c981ac81abadecfde [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c1822e697d64b758a23b11c8fdeb59a" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 39067 } } peers { permanent_uuid: "7cd7c680ef2e474c981ac81abadecfde" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 40117 } } peers { permanent_uuid: "62bbd271280e4b0a9ddf9ab46151e455" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 33917 } }
I20260709 13:18:28.613083 15744 leader_election.cc:290] T 48e8b6e82d7b42e6bdd4b0a8a25dfd9e P 7cd7c680ef2e474c981ac81abadecfde [CANDIDATE]: Term 2 election: Requested vote from peers 8c1822e697d64b758a23b11c8fdeb59a (127.14.177.129:39067), 62bbd271280e4b0a9ddf9ab46151e455 (127.14.177.131:33917)
I20260709 13:18:28.613938 15671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48e8b6e82d7b42e6bdd4b0a8a25dfd9e" candidate_uuid: "7cd7c680ef2e474c981ac81abadecfde" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8c1822e697d64b758a23b11c8fdeb59a"
I20260709 13:18:28.614534 15819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48e8b6e82d7b42e6bdd4b0a8a25dfd9e" candidate_uuid: "7cd7c680ef2e474c981ac81abadecfde" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "62bbd271280e4b0a9ddf9ab46151e455"
I20260709 13:18:28.614583 15671 raft_consensus.cc:3055] T 48e8b6e82d7b42e6bdd4b0a8a25dfd9e P 8c1822e697d64b758a23b11c8fdeb59a [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:18:28.615202 15819 raft_consensus.cc:3060] T 48e8b6e82d7b42e6bdd4b0a8a25dfd9e P 62bbd271280e4b0a9ddf9ab46151e455 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:28.615146 15671 raft_consensus.cc:740] T 48e8b6e82d7b42e6bdd4b0a8a25dfd9e P 8c1822e697d64b758a23b11c8fdeb59a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8c1822e697d64b758a23b11c8fdeb59a, State: Running, Role: LEADER
I20260709 13:18:28.616180 15671 consensus_queue.cc:260] T 48e8b6e82d7b42e6bdd4b0a8a25dfd9e P 8c1822e697d64b758a23b11c8fdeb59a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c1822e697d64b758a23b11c8fdeb59a" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 39067 } } peers { permanent_uuid: "7cd7c680ef2e474c981ac81abadecfde" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 40117 } } peers { permanent_uuid: "62bbd271280e4b0a9ddf9ab46151e455" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 33917 } }
I20260709 13:18:28.617449 15671 raft_consensus.cc:3060] T 48e8b6e82d7b42e6bdd4b0a8a25dfd9e P 8c1822e697d64b758a23b11c8fdeb59a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:28.620702 15819 raft_consensus.cc:2468] T 48e8b6e82d7b42e6bdd4b0a8a25dfd9e P 62bbd271280e4b0a9ddf9ab46151e455 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7cd7c680ef2e474c981ac81abadecfde in term 2.
I20260709 13:18:28.621812 15712 leader_election.cc:304] T 48e8b6e82d7b42e6bdd4b0a8a25dfd9e P 7cd7c680ef2e474c981ac81abadecfde [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 62bbd271280e4b0a9ddf9ab46151e455, 7cd7c680ef2e474c981ac81abadecfde; no voters: 
I20260709 13:18:28.622496 15886 raft_consensus.cc:2804] T 48e8b6e82d7b42e6bdd4b0a8a25dfd9e P 7cd7c680ef2e474c981ac81abadecfde [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:18:28.622682 15671 raft_consensus.cc:2468] T 48e8b6e82d7b42e6bdd4b0a8a25dfd9e P 8c1822e697d64b758a23b11c8fdeb59a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7cd7c680ef2e474c981ac81abadecfde in term 2.
I20260709 13:18:28.622927 15886 raft_consensus.cc:697] T 48e8b6e82d7b42e6bdd4b0a8a25dfd9e P 7cd7c680ef2e474c981ac81abadecfde [term 2 LEADER]: Becoming Leader. State: Replica: 7cd7c680ef2e474c981ac81abadecfde, State: Running, Role: LEADER
I20260709 13:18:28.623883 15886 consensus_queue.cc:237] T 48e8b6e82d7b42e6bdd4b0a8a25dfd9e P 7cd7c680ef2e474c981ac81abadecfde [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c1822e697d64b758a23b11c8fdeb59a" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 39067 } } peers { permanent_uuid: "7cd7c680ef2e474c981ac81abadecfde" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 40117 } } peers { permanent_uuid: "62bbd271280e4b0a9ddf9ab46151e455" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 33917 } }
I20260709 13:18:28.630075 15568 catalog_manager.cc:5697] T 48e8b6e82d7b42e6bdd4b0a8a25dfd9e P 7cd7c680ef2e474c981ac81abadecfde reported cstate change: term changed from 1 to 2, leader changed from 8c1822e697d64b758a23b11c8fdeb59a (127.14.177.129) to 7cd7c680ef2e474c981ac81abadecfde (127.14.177.130). New cstate: current_term: 2 leader_uuid: "7cd7c680ef2e474c981ac81abadecfde" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c1822e697d64b758a23b11c8fdeb59a" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 39067 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7cd7c680ef2e474c981ac81abadecfde" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 40117 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "62bbd271280e4b0a9ddf9ab46151e455" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 33917 } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:28.686780 15867 raft_consensus.cc:993] T 566eae6d93af4ec492b9b1a270dc03b7 P 7cd7c680ef2e474c981ac81abadecfde: : Instructing follower 8c1822e697d64b758a23b11c8fdeb59a to start an election
I20260709 13:18:28.687314 15886 raft_consensus.cc:1081] T 566eae6d93af4ec492b9b1a270dc03b7 P 7cd7c680ef2e474c981ac81abadecfde [term 1 LEADER]: Signalling peer 8c1822e697d64b758a23b11c8fdeb59a to start an election
I20260709 13:18:28.688782 15671 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "566eae6d93af4ec492b9b1a270dc03b7"
dest_uuid: "8c1822e697d64b758a23b11c8fdeb59a"
 from {username='slave'} at 127.0.0.1:42812
I20260709 13:18:28.689357 15671 raft_consensus.cc:493] T 566eae6d93af4ec492b9b1a270dc03b7 P 8c1822e697d64b758a23b11c8fdeb59a [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:18:28.689679 15671 raft_consensus.cc:3060] T 566eae6d93af4ec492b9b1a270dc03b7 P 8c1822e697d64b758a23b11c8fdeb59a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:28.694483 15671 raft_consensus.cc:515] T 566eae6d93af4ec492b9b1a270dc03b7 P 8c1822e697d64b758a23b11c8fdeb59a [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c1822e697d64b758a23b11c8fdeb59a" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 39067 } } peers { permanent_uuid: "7cd7c680ef2e474c981ac81abadecfde" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 40117 } } peers { permanent_uuid: "62bbd271280e4b0a9ddf9ab46151e455" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 33917 } }
I20260709 13:18:28.696321 15671 leader_election.cc:290] T 566eae6d93af4ec492b9b1a270dc03b7 P 8c1822e697d64b758a23b11c8fdeb59a [CANDIDATE]: Term 2 election: Requested vote from peers 7cd7c680ef2e474c981ac81abadecfde (127.14.177.130:40117), 62bbd271280e4b0a9ddf9ab46151e455 (127.14.177.131:33917)
I20260709 13:18:28.697249 15744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "566eae6d93af4ec492b9b1a270dc03b7" candidate_uuid: "8c1822e697d64b758a23b11c8fdeb59a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "7cd7c680ef2e474c981ac81abadecfde"
I20260709 13:18:28.697518 15819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "566eae6d93af4ec492b9b1a270dc03b7" candidate_uuid: "8c1822e697d64b758a23b11c8fdeb59a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "62bbd271280e4b0a9ddf9ab46151e455"
I20260709 13:18:28.698948 15744 raft_consensus.cc:3055] T 566eae6d93af4ec492b9b1a270dc03b7 P 7cd7c680ef2e474c981ac81abadecfde [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:18:28.699016 15819 raft_consensus.cc:3060] T 566eae6d93af4ec492b9b1a270dc03b7 P 62bbd271280e4b0a9ddf9ab46151e455 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:28.699391 15744 raft_consensus.cc:740] T 566eae6d93af4ec492b9b1a270dc03b7 P 7cd7c680ef2e474c981ac81abadecfde [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 7cd7c680ef2e474c981ac81abadecfde, State: Running, Role: LEADER
I20260709 13:18:28.700244 15744 consensus_queue.cc:260] T 566eae6d93af4ec492b9b1a270dc03b7 P 7cd7c680ef2e474c981ac81abadecfde [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c1822e697d64b758a23b11c8fdeb59a" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 39067 } } peers { permanent_uuid: "7cd7c680ef2e474c981ac81abadecfde" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 40117 } } peers { permanent_uuid: "62bbd271280e4b0a9ddf9ab46151e455" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 33917 } }
I20260709 13:18:28.701395 15744 raft_consensus.cc:3060] T 566eae6d93af4ec492b9b1a270dc03b7 P 7cd7c680ef2e474c981ac81abadecfde [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:28.706370 15819 raft_consensus.cc:2468] T 566eae6d93af4ec492b9b1a270dc03b7 P 62bbd271280e4b0a9ddf9ab46151e455 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c1822e697d64b758a23b11c8fdeb59a in term 2.
I20260709 13:18:28.706532 15744 raft_consensus.cc:2468] T 566eae6d93af4ec492b9b1a270dc03b7 P 7cd7c680ef2e474c981ac81abadecfde [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c1822e697d64b758a23b11c8fdeb59a in term 2.
I20260709 13:18:28.707753 15636 leader_election.cc:304] T 566eae6d93af4ec492b9b1a270dc03b7 P 8c1822e697d64b758a23b11c8fdeb59a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7cd7c680ef2e474c981ac81abadecfde, 8c1822e697d64b758a23b11c8fdeb59a; no voters: 
I20260709 13:18:28.708592 15909 raft_consensus.cc:2804] T 566eae6d93af4ec492b9b1a270dc03b7 P 8c1822e697d64b758a23b11c8fdeb59a [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:18:28.709146 15909 raft_consensus.cc:697] T 566eae6d93af4ec492b9b1a270dc03b7 P 8c1822e697d64b758a23b11c8fdeb59a [term 2 LEADER]: Becoming Leader. State: Replica: 8c1822e697d64b758a23b11c8fdeb59a, State: Running, Role: LEADER
I20260709 13:18:28.709982 15909 consensus_queue.cc:237] T 566eae6d93af4ec492b9b1a270dc03b7 P 8c1822e697d64b758a23b11c8fdeb59a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c1822e697d64b758a23b11c8fdeb59a" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 39067 } } peers { permanent_uuid: "7cd7c680ef2e474c981ac81abadecfde" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 40117 } } peers { permanent_uuid: "62bbd271280e4b0a9ddf9ab46151e455" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 33917 } }
I20260709 13:18:28.717391 15568 catalog_manager.cc:5697] T 566eae6d93af4ec492b9b1a270dc03b7 P 8c1822e697d64b758a23b11c8fdeb59a reported cstate change: term changed from 1 to 2, leader changed from 7cd7c680ef2e474c981ac81abadecfde (127.14.177.130) to 8c1822e697d64b758a23b11c8fdeb59a (127.14.177.129). New cstate: current_term: 2 leader_uuid: "8c1822e697d64b758a23b11c8fdeb59a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c1822e697d64b758a23b11c8fdeb59a" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 39067 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7cd7c680ef2e474c981ac81abadecfde" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 40117 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "62bbd271280e4b0a9ddf9ab46151e455" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 33917 } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:28.750936 15903 raft_consensus.cc:993] T 0ce7d20d574f47649d3ad0f3c382198d P 7cd7c680ef2e474c981ac81abadecfde: : Instructing follower 8c1822e697d64b758a23b11c8fdeb59a to start an election
I20260709 13:18:28.751426 15884 raft_consensus.cc:1081] T 0ce7d20d574f47649d3ad0f3c382198d P 7cd7c680ef2e474c981ac81abadecfde [term 1 LEADER]: Signalling peer 8c1822e697d64b758a23b11c8fdeb59a to start an election
I20260709 13:18:28.752933 15671 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "0ce7d20d574f47649d3ad0f3c382198d"
dest_uuid: "8c1822e697d64b758a23b11c8fdeb59a"
 from {username='slave'} at 127.0.0.1:42812
I20260709 13:18:28.753484 15671 raft_consensus.cc:493] T 0ce7d20d574f47649d3ad0f3c382198d P 8c1822e697d64b758a23b11c8fdeb59a [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:18:28.753892 15671 raft_consensus.cc:3060] T 0ce7d20d574f47649d3ad0f3c382198d P 8c1822e697d64b758a23b11c8fdeb59a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:28.758755 15671 raft_consensus.cc:515] T 0ce7d20d574f47649d3ad0f3c382198d P 8c1822e697d64b758a23b11c8fdeb59a [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c1822e697d64b758a23b11c8fdeb59a" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 39067 } } peers { permanent_uuid: "7cd7c680ef2e474c981ac81abadecfde" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 40117 } } peers { permanent_uuid: "62bbd271280e4b0a9ddf9ab46151e455" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 33917 } }
I20260709 13:18:28.760404 15671 leader_election.cc:290] T 0ce7d20d574f47649d3ad0f3c382198d P 8c1822e697d64b758a23b11c8fdeb59a [CANDIDATE]: Term 2 election: Requested vote from peers 7cd7c680ef2e474c981ac81abadecfde (127.14.177.130:40117), 62bbd271280e4b0a9ddf9ab46151e455 (127.14.177.131:33917)
I20260709 13:18:28.761186 15744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ce7d20d574f47649d3ad0f3c382198d" candidate_uuid: "8c1822e697d64b758a23b11c8fdeb59a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "7cd7c680ef2e474c981ac81abadecfde"
I20260709 13:18:28.761360 15819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ce7d20d574f47649d3ad0f3c382198d" candidate_uuid: "8c1822e697d64b758a23b11c8fdeb59a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "62bbd271280e4b0a9ddf9ab46151e455"
I20260709 13:18:28.761799 15744 raft_consensus.cc:3055] T 0ce7d20d574f47649d3ad0f3c382198d P 7cd7c680ef2e474c981ac81abadecfde [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:18:28.761852 15819 raft_consensus.cc:3060] T 0ce7d20d574f47649d3ad0f3c382198d P 62bbd271280e4b0a9ddf9ab46151e455 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:28.762276 15744 raft_consensus.cc:740] T 0ce7d20d574f47649d3ad0f3c382198d P 7cd7c680ef2e474c981ac81abadecfde [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 7cd7c680ef2e474c981ac81abadecfde, State: Running, Role: LEADER
I20260709 13:18:28.763075 15744 consensus_queue.cc:260] T 0ce7d20d574f47649d3ad0f3c382198d P 7cd7c680ef2e474c981ac81abadecfde [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c1822e697d64b758a23b11c8fdeb59a" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 39067 } } peers { permanent_uuid: "7cd7c680ef2e474c981ac81abadecfde" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 40117 } } peers { permanent_uuid: "62bbd271280e4b0a9ddf9ab46151e455" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 33917 } }
I20260709 13:18:28.764302 15744 raft_consensus.cc:3060] T 0ce7d20d574f47649d3ad0f3c382198d P 7cd7c680ef2e474c981ac81abadecfde [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:28.768991 15819 raft_consensus.cc:2468] T 0ce7d20d574f47649d3ad0f3c382198d P 62bbd271280e4b0a9ddf9ab46151e455 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c1822e697d64b758a23b11c8fdeb59a in term 2.
I20260709 13:18:28.770262 15637 leader_election.cc:304] T 0ce7d20d574f47649d3ad0f3c382198d P 8c1822e697d64b758a23b11c8fdeb59a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 62bbd271280e4b0a9ddf9ab46151e455, 8c1822e697d64b758a23b11c8fdeb59a; no voters: 
I20260709 13:18:28.770782 15744 raft_consensus.cc:2468] T 0ce7d20d574f47649d3ad0f3c382198d P 7cd7c680ef2e474c981ac81abadecfde [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c1822e697d64b758a23b11c8fdeb59a in term 2.
I20260709 13:18:28.771086 15909 raft_consensus.cc:2804] T 0ce7d20d574f47649d3ad0f3c382198d P 8c1822e697d64b758a23b11c8fdeb59a [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:18:28.771636 15909 raft_consensus.cc:697] T 0ce7d20d574f47649d3ad0f3c382198d P 8c1822e697d64b758a23b11c8fdeb59a [term 2 LEADER]: Becoming Leader. State: Replica: 8c1822e697d64b758a23b11c8fdeb59a, State: Running, Role: LEADER
I20260709 13:18:28.772462 15909 consensus_queue.cc:237] T 0ce7d20d574f47649d3ad0f3c382198d P 8c1822e697d64b758a23b11c8fdeb59a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c1822e697d64b758a23b11c8fdeb59a" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 39067 } } peers { permanent_uuid: "7cd7c680ef2e474c981ac81abadecfde" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 40117 } } peers { permanent_uuid: "62bbd271280e4b0a9ddf9ab46151e455" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 33917 } }
I20260709 13:18:28.778744 15568 catalog_manager.cc:5697] T 0ce7d20d574f47649d3ad0f3c382198d P 8c1822e697d64b758a23b11c8fdeb59a reported cstate change: term changed from 1 to 2, leader changed from 7cd7c680ef2e474c981ac81abadecfde (127.14.177.130) to 8c1822e697d64b758a23b11c8fdeb59a (127.14.177.129). New cstate: current_term: 2 leader_uuid: "8c1822e697d64b758a23b11c8fdeb59a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c1822e697d64b758a23b11c8fdeb59a" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 39067 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7cd7c680ef2e474c981ac81abadecfde" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 40117 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "62bbd271280e4b0a9ddf9ab46151e455" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 33917 } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:28.794441 15903 raft_consensus.cc:993] T 8cff7911abd649b9a276ddae431311d1 P 7cd7c680ef2e474c981ac81abadecfde: : Instructing follower 8c1822e697d64b758a23b11c8fdeb59a to start an election
I20260709 13:18:28.794910 15884 raft_consensus.cc:1081] T 8cff7911abd649b9a276ddae431311d1 P 7cd7c680ef2e474c981ac81abadecfde [term 1 LEADER]: Signalling peer 8c1822e697d64b758a23b11c8fdeb59a to start an election
I20260709 13:18:28.796372 15671 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "8cff7911abd649b9a276ddae431311d1"
dest_uuid: "8c1822e697d64b758a23b11c8fdeb59a"
 from {username='slave'} at 127.0.0.1:42812
I20260709 13:18:28.796856 15671 raft_consensus.cc:493] T 8cff7911abd649b9a276ddae431311d1 P 8c1822e697d64b758a23b11c8fdeb59a [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:18:28.797274 15671 raft_consensus.cc:3060] T 8cff7911abd649b9a276ddae431311d1 P 8c1822e697d64b758a23b11c8fdeb59a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:28.802275 15671 raft_consensus.cc:515] T 8cff7911abd649b9a276ddae431311d1 P 8c1822e697d64b758a23b11c8fdeb59a [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c1822e697d64b758a23b11c8fdeb59a" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 39067 } } peers { permanent_uuid: "7cd7c680ef2e474c981ac81abadecfde" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 40117 } } peers { permanent_uuid: "62bbd271280e4b0a9ddf9ab46151e455" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 33917 } }
I20260709 13:18:28.803824 15671 leader_election.cc:290] T 8cff7911abd649b9a276ddae431311d1 P 8c1822e697d64b758a23b11c8fdeb59a [CANDIDATE]: Term 2 election: Requested vote from peers 7cd7c680ef2e474c981ac81abadecfde (127.14.177.130:40117), 62bbd271280e4b0a9ddf9ab46151e455 (127.14.177.131:33917)
I20260709 13:18:28.804630 15744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cff7911abd649b9a276ddae431311d1" candidate_uuid: "8c1822e697d64b758a23b11c8fdeb59a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "7cd7c680ef2e474c981ac81abadecfde"
I20260709 13:18:28.804828 15819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cff7911abd649b9a276ddae431311d1" candidate_uuid: "8c1822e697d64b758a23b11c8fdeb59a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "62bbd271280e4b0a9ddf9ab46151e455"
I20260709 13:18:28.805233 15744 raft_consensus.cc:3055] T 8cff7911abd649b9a276ddae431311d1 P 7cd7c680ef2e474c981ac81abadecfde [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:18:28.805317 15819 raft_consensus.cc:3060] T 8cff7911abd649b9a276ddae431311d1 P 62bbd271280e4b0a9ddf9ab46151e455 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:28.805550 15744 raft_consensus.cc:740] T 8cff7911abd649b9a276ddae431311d1 P 7cd7c680ef2e474c981ac81abadecfde [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 7cd7c680ef2e474c981ac81abadecfde, State: Running, Role: LEADER
I20260709 13:18:28.806361 15744 consensus_queue.cc:260] T 8cff7911abd649b9a276ddae431311d1 P 7cd7c680ef2e474c981ac81abadecfde [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c1822e697d64b758a23b11c8fdeb59a" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 39067 } } peers { permanent_uuid: "7cd7c680ef2e474c981ac81abadecfde" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 40117 } } peers { permanent_uuid: "62bbd271280e4b0a9ddf9ab46151e455" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 33917 } }
I20260709 13:18:28.807322 15744 raft_consensus.cc:3060] T 8cff7911abd649b9a276ddae431311d1 P 7cd7c680ef2e474c981ac81abadecfde [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:28.812393 15819 raft_consensus.cc:2468] T 8cff7911abd649b9a276ddae431311d1 P 62bbd271280e4b0a9ddf9ab46151e455 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c1822e697d64b758a23b11c8fdeb59a in term 2.
I20260709 13:18:28.813787 15637 leader_election.cc:304] T 8cff7911abd649b9a276ddae431311d1 P 8c1822e697d64b758a23b11c8fdeb59a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 62bbd271280e4b0a9ddf9ab46151e455, 8c1822e697d64b758a23b11c8fdeb59a; no voters: 
I20260709 13:18:28.814944 15744 raft_consensus.cc:2468] T 8cff7911abd649b9a276ddae431311d1 P 7cd7c680ef2e474c981ac81abadecfde [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c1822e697d64b758a23b11c8fdeb59a in term 2.
I20260709 13:18:28.815066 15911 raft_consensus.cc:2804] T 8cff7911abd649b9a276ddae431311d1 P 8c1822e697d64b758a23b11c8fdeb59a [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:18:28.817504 15911 raft_consensus.cc:697] T 8cff7911abd649b9a276ddae431311d1 P 8c1822e697d64b758a23b11c8fdeb59a [term 2 LEADER]: Becoming Leader. State: Replica: 8c1822e697d64b758a23b11c8fdeb59a, State: Running, Role: LEADER
I20260709 13:18:28.818341 15911 consensus_queue.cc:237] T 8cff7911abd649b9a276ddae431311d1 P 8c1822e697d64b758a23b11c8fdeb59a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c1822e697d64b758a23b11c8fdeb59a" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 39067 } } peers { permanent_uuid: "7cd7c680ef2e474c981ac81abadecfde" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 40117 } } peers { permanent_uuid: "62bbd271280e4b0a9ddf9ab46151e455" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 33917 } }
I20260709 13:18:28.821569 15909 raft_consensus.cc:993] T 408866bb47904e4189bc235ca5b0c7c7 P 8c1822e697d64b758a23b11c8fdeb59a: : Instructing follower 7cd7c680ef2e474c981ac81abadecfde to start an election
I20260709 13:18:28.822000 15911 raft_consensus.cc:1081] T 408866bb47904e4189bc235ca5b0c7c7 P 8c1822e697d64b758a23b11c8fdeb59a [term 1 LEADER]: Signalling peer 7cd7c680ef2e474c981ac81abadecfde to start an election
I20260709 13:18:28.823436 15744 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "408866bb47904e4189bc235ca5b0c7c7"
dest_uuid: "7cd7c680ef2e474c981ac81abadecfde"
 from {username='slave'} at 127.0.0.1:49440
I20260709 13:18:28.823956 15744 raft_consensus.cc:493] T 408866bb47904e4189bc235ca5b0c7c7 P 7cd7c680ef2e474c981ac81abadecfde [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:18:28.824309 15744 raft_consensus.cc:3060] T 408866bb47904e4189bc235ca5b0c7c7 P 7cd7c680ef2e474c981ac81abadecfde [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:28.825404 15568 catalog_manager.cc:5697] T 8cff7911abd649b9a276ddae431311d1 P 8c1822e697d64b758a23b11c8fdeb59a reported cstate change: term changed from 1 to 2, leader changed from 7cd7c680ef2e474c981ac81abadecfde (127.14.177.130) to 8c1822e697d64b758a23b11c8fdeb59a (127.14.177.129). New cstate: current_term: 2 leader_uuid: "8c1822e697d64b758a23b11c8fdeb59a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c1822e697d64b758a23b11c8fdeb59a" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 39067 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7cd7c680ef2e474c981ac81abadecfde" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 40117 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "62bbd271280e4b0a9ddf9ab46151e455" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 33917 } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:28.830185 15744 raft_consensus.cc:515] T 408866bb47904e4189bc235ca5b0c7c7 P 7cd7c680ef2e474c981ac81abadecfde [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c1822e697d64b758a23b11c8fdeb59a" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 39067 } } peers { permanent_uuid: "7cd7c680ef2e474c981ac81abadecfde" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 40117 } } peers { permanent_uuid: "62bbd271280e4b0a9ddf9ab46151e455" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 33917 } }
I20260709 13:18:28.832298 15744 leader_election.cc:290] T 408866bb47904e4189bc235ca5b0c7c7 P 7cd7c680ef2e474c981ac81abadecfde [CANDIDATE]: Term 2 election: Requested vote from peers 8c1822e697d64b758a23b11c8fdeb59a (127.14.177.129:39067), 62bbd271280e4b0a9ddf9ab46151e455 (127.14.177.131:33917)
I20260709 13:18:28.833046 15671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "408866bb47904e4189bc235ca5b0c7c7" candidate_uuid: "7cd7c680ef2e474c981ac81abadecfde" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8c1822e697d64b758a23b11c8fdeb59a"
I20260709 13:18:28.833637 15671 raft_consensus.cc:3055] T 408866bb47904e4189bc235ca5b0c7c7 P 8c1822e697d64b758a23b11c8fdeb59a [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:18:28.833953 15671 raft_consensus.cc:740] T 408866bb47904e4189bc235ca5b0c7c7 P 8c1822e697d64b758a23b11c8fdeb59a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8c1822e697d64b758a23b11c8fdeb59a, State: Running, Role: LEADER
I20260709 13:18:28.834940 15819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "408866bb47904e4189bc235ca5b0c7c7" candidate_uuid: "7cd7c680ef2e474c981ac81abadecfde" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "62bbd271280e4b0a9ddf9ab46151e455"
I20260709 13:18:28.834728 15671 consensus_queue.cc:260] T 408866bb47904e4189bc235ca5b0c7c7 P 8c1822e697d64b758a23b11c8fdeb59a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c1822e697d64b758a23b11c8fdeb59a" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 39067 } } peers { permanent_uuid: "7cd7c680ef2e474c981ac81abadecfde" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 40117 } } peers { permanent_uuid: "62bbd271280e4b0a9ddf9ab46151e455" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 33917 } }
I20260709 13:18:28.835557 15819 raft_consensus.cc:3060] T 408866bb47904e4189bc235ca5b0c7c7 P 62bbd271280e4b0a9ddf9ab46151e455 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:28.835875 15671 raft_consensus.cc:3060] T 408866bb47904e4189bc235ca5b0c7c7 P 8c1822e697d64b758a23b11c8fdeb59a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:28.842428 15819 raft_consensus.cc:2468] T 408866bb47904e4189bc235ca5b0c7c7 P 62bbd271280e4b0a9ddf9ab46151e455 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7cd7c680ef2e474c981ac81abadecfde in term 2.
I20260709 13:18:28.842732 15671 raft_consensus.cc:2468] T 408866bb47904e4189bc235ca5b0c7c7 P 8c1822e697d64b758a23b11c8fdeb59a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7cd7c680ef2e474c981ac81abadecfde in term 2.
I20260709 13:18:28.843708 15712 leader_election.cc:304] T 408866bb47904e4189bc235ca5b0c7c7 P 7cd7c680ef2e474c981ac81abadecfde [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 62bbd271280e4b0a9ddf9ab46151e455, 7cd7c680ef2e474c981ac81abadecfde; no voters: 
I20260709 13:18:28.844529 15884 raft_consensus.cc:2804] T 408866bb47904e4189bc235ca5b0c7c7 P 7cd7c680ef2e474c981ac81abadecfde [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:18:28.844974 15884 raft_consensus.cc:697] T 408866bb47904e4189bc235ca5b0c7c7 P 7cd7c680ef2e474c981ac81abadecfde [term 2 LEADER]: Becoming Leader. State: Replica: 7cd7c680ef2e474c981ac81abadecfde, State: Running, Role: LEADER
I20260709 13:18:28.845719 15884 consensus_queue.cc:237] T 408866bb47904e4189bc235ca5b0c7c7 P 7cd7c680ef2e474c981ac81abadecfde [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c1822e697d64b758a23b11c8fdeb59a" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 39067 } } peers { permanent_uuid: "7cd7c680ef2e474c981ac81abadecfde" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 40117 } } peers { permanent_uuid: "62bbd271280e4b0a9ddf9ab46151e455" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 33917 } }
I20260709 13:18:28.852157 15568 catalog_manager.cc:5697] T 408866bb47904e4189bc235ca5b0c7c7 P 7cd7c680ef2e474c981ac81abadecfde reported cstate change: term changed from 1 to 2, leader changed from 8c1822e697d64b758a23b11c8fdeb59a (127.14.177.129) to 7cd7c680ef2e474c981ac81abadecfde (127.14.177.130). New cstate: current_term: 2 leader_uuid: "7cd7c680ef2e474c981ac81abadecfde" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c1822e697d64b758a23b11c8fdeb59a" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 39067 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7cd7c680ef2e474c981ac81abadecfde" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 40117 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "62bbd271280e4b0a9ddf9ab46151e455" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 33917 } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:28.853016 15903 raft_consensus.cc:993] T 4d83a05b849340c186cc995649575d3b P 7cd7c680ef2e474c981ac81abadecfde: : Instructing follower 8c1822e697d64b758a23b11c8fdeb59a to start an election
I20260709 13:18:28.853526 15884 raft_consensus.cc:1081] T 4d83a05b849340c186cc995649575d3b P 7cd7c680ef2e474c981ac81abadecfde [term 1 LEADER]: Signalling peer 8c1822e697d64b758a23b11c8fdeb59a to start an election
I20260709 13:18:28.855065 15671 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "4d83a05b849340c186cc995649575d3b"
dest_uuid: "8c1822e697d64b758a23b11c8fdeb59a"
 from {username='slave'} at 127.0.0.1:42812
I20260709 13:18:28.855608 15671 raft_consensus.cc:493] T 4d83a05b849340c186cc995649575d3b P 8c1822e697d64b758a23b11c8fdeb59a [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:18:28.855892 15671 raft_consensus.cc:3060] T 4d83a05b849340c186cc995649575d3b P 8c1822e697d64b758a23b11c8fdeb59a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:28.862280 15671 raft_consensus.cc:515] T 4d83a05b849340c186cc995649575d3b P 8c1822e697d64b758a23b11c8fdeb59a [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62bbd271280e4b0a9ddf9ab46151e455" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 33917 } } peers { permanent_uuid: "7cd7c680ef2e474c981ac81abadecfde" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 40117 } } peers { permanent_uuid: "8c1822e697d64b758a23b11c8fdeb59a" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 39067 } }
I20260709 13:18:28.864105 15671 leader_election.cc:290] T 4d83a05b849340c186cc995649575d3b P 8c1822e697d64b758a23b11c8fdeb59a [CANDIDATE]: Term 2 election: Requested vote from peers 62bbd271280e4b0a9ddf9ab46151e455 (127.14.177.131:33917), 7cd7c680ef2e474c981ac81abadecfde (127.14.177.130:40117)
I20260709 13:18:28.864925 15819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d83a05b849340c186cc995649575d3b" candidate_uuid: "8c1822e697d64b758a23b11c8fdeb59a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "62bbd271280e4b0a9ddf9ab46151e455"
I20260709 13:18:28.865183 15744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d83a05b849340c186cc995649575d3b" candidate_uuid: "8c1822e697d64b758a23b11c8fdeb59a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "7cd7c680ef2e474c981ac81abadecfde"
I20260709 13:18:28.865476 15819 raft_consensus.cc:3060] T 4d83a05b849340c186cc995649575d3b P 62bbd271280e4b0a9ddf9ab46151e455 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:28.865764 15744 raft_consensus.cc:3055] T 4d83a05b849340c186cc995649575d3b P 7cd7c680ef2e474c981ac81abadecfde [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:18:28.866062 15744 raft_consensus.cc:740] T 4d83a05b849340c186cc995649575d3b P 7cd7c680ef2e474c981ac81abadecfde [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 7cd7c680ef2e474c981ac81abadecfde, State: Running, Role: LEADER
I20260709 13:18:28.866775 15744 consensus_queue.cc:260] T 4d83a05b849340c186cc995649575d3b P 7cd7c680ef2e474c981ac81abadecfde [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62bbd271280e4b0a9ddf9ab46151e455" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 33917 } } peers { permanent_uuid: "7cd7c680ef2e474c981ac81abadecfde" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 40117 } } peers { permanent_uuid: "8c1822e697d64b758a23b11c8fdeb59a" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 39067 } }
I20260709 13:18:28.867743 15744 raft_consensus.cc:3060] T 4d83a05b849340c186cc995649575d3b P 7cd7c680ef2e474c981ac81abadecfde [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:28.870318 15819 raft_consensus.cc:2468] T 4d83a05b849340c186cc995649575d3b P 62bbd271280e4b0a9ddf9ab46151e455 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c1822e697d64b758a23b11c8fdeb59a in term 2.
I20260709 13:18:28.871310 15637 leader_election.cc:304] T 4d83a05b849340c186cc995649575d3b P 8c1822e697d64b758a23b11c8fdeb59a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 62bbd271280e4b0a9ddf9ab46151e455, 8c1822e697d64b758a23b11c8fdeb59a; no voters: 
I20260709 13:18:28.871970 15911 raft_consensus.cc:2804] T 4d83a05b849340c186cc995649575d3b P 8c1822e697d64b758a23b11c8fdeb59a [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:18:28.872458 15911 raft_consensus.cc:697] T 4d83a05b849340c186cc995649575d3b P 8c1822e697d64b758a23b11c8fdeb59a [term 2 LEADER]: Becoming Leader. State: Replica: 8c1822e697d64b758a23b11c8fdeb59a, State: Running, Role: LEADER
I20260709 13:18:28.872624 15744 raft_consensus.cc:2468] T 4d83a05b849340c186cc995649575d3b P 7cd7c680ef2e474c981ac81abadecfde [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c1822e697d64b758a23b11c8fdeb59a in term 2.
I20260709 13:18:28.873308 15911 consensus_queue.cc:237] T 4d83a05b849340c186cc995649575d3b P 8c1822e697d64b758a23b11c8fdeb59a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62bbd271280e4b0a9ddf9ab46151e455" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 33917 } } peers { permanent_uuid: "7cd7c680ef2e474c981ac81abadecfde" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 40117 } } peers { permanent_uuid: "8c1822e697d64b758a23b11c8fdeb59a" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 39067 } }
I20260709 13:18:28.879011 15568 catalog_manager.cc:5697] T 4d83a05b849340c186cc995649575d3b P 8c1822e697d64b758a23b11c8fdeb59a reported cstate change: term changed from 1 to 2, leader changed from 7cd7c680ef2e474c981ac81abadecfde (127.14.177.130) to 8c1822e697d64b758a23b11c8fdeb59a (127.14.177.129). New cstate: current_term: 2 leader_uuid: "8c1822e697d64b758a23b11c8fdeb59a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62bbd271280e4b0a9ddf9ab46151e455" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 33917 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7cd7c680ef2e474c981ac81abadecfde" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 40117 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8c1822e697d64b758a23b11c8fdeb59a" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 39067 } health_report { overall_health: HEALTHY } } }
I20260709 13:18:28.890799 15819 raft_consensus.cc:1275] T 8fc26f7d12464423a212a1f595ca4b88 P 62bbd271280e4b0a9ddf9ab46151e455 [term 2 FOLLOWER]: Refusing update from remote peer 8c1822e697d64b758a23b11c8fdeb59a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:18:28.892163 15911 consensus_queue.cc:1048] T 8fc26f7d12464423a212a1f595ca4b88 P 8c1822e697d64b758a23b11c8fdeb59a [LEADER]: Connected to new peer: Peer: permanent_uuid: "62bbd271280e4b0a9ddf9ab46151e455" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 33917 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:18:28.900745 15744 raft_consensus.cc:1275] T 8fc26f7d12464423a212a1f595ca4b88 P 7cd7c680ef2e474c981ac81abadecfde [term 2 FOLLOWER]: Refusing update from remote peer 8c1822e697d64b758a23b11c8fdeb59a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:18:28.902386 15911 consensus_queue.cc:1048] T 8fc26f7d12464423a212a1f595ca4b88 P 8c1822e697d64b758a23b11c8fdeb59a [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cd7c680ef2e474c981ac81abadecfde" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 40117 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:18:28.938120 15907 raft_consensus.cc:993] T 17f97fe9a93f4a86b1e2bad621bfb561 P 8c1822e697d64b758a23b11c8fdeb59a: : Instructing follower 7cd7c680ef2e474c981ac81abadecfde to start an election
I20260709 13:18:28.938651 15911 raft_consensus.cc:1081] T 17f97fe9a93f4a86b1e2bad621bfb561 P 8c1822e697d64b758a23b11c8fdeb59a [term 1 LEADER]: Signalling peer 7cd7c680ef2e474c981ac81abadecfde to start an election
I20260709 13:18:28.940327 15744 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "17f97fe9a93f4a86b1e2bad621bfb561"
dest_uuid: "7cd7c680ef2e474c981ac81abadecfde"
 from {username='slave'} at 127.0.0.1:49440
I20260709 13:18:28.940922 15744 raft_consensus.cc:493] T 17f97fe9a93f4a86b1e2bad621bfb561 P 7cd7c680ef2e474c981ac81abadecfde [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:18:28.941291 15744 raft_consensus.cc:3060] T 17f97fe9a93f4a86b1e2bad621bfb561 P 7cd7c680ef2e474c981ac81abadecfde [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:28.946062 15744 raft_consensus.cc:515] T 17f97fe9a93f4a86b1e2bad621bfb561 P 7cd7c680ef2e474c981ac81abadecfde [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c1822e697d64b758a23b11c8fdeb59a" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 39067 } } peers { permanent_uuid: "7cd7c680ef2e474c981ac81abadecfde" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 40117 } } peers { permanent_uuid: "62bbd271280e4b0a9ddf9ab46151e455" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 33917 } }
I20260709 13:18:28.948199 15744 leader_election.cc:290] T 17f97fe9a93f4a86b1e2bad621bfb561 P 7cd7c680ef2e474c981ac81abadecfde [CANDIDATE]: Term 2 election: Requested vote from peers 8c1822e697d64b758a23b11c8fdeb59a (127.14.177.129:39067), 62bbd271280e4b0a9ddf9ab46151e455 (127.14.177.131:33917)
I20260709 13:18:28.948988 15671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17f97fe9a93f4a86b1e2bad621bfb561" candidate_uuid: "7cd7c680ef2e474c981ac81abadecfde" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8c1822e697d64b758a23b11c8fdeb59a"
I20260709 13:18:28.949290 15819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17f97fe9a93f4a86b1e2bad621bfb561" candidate_uuid: "7cd7c680ef2e474c981ac81abadecfde" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "62bbd271280e4b0a9ddf9ab46151e455"
I20260709 13:18:28.949622 15671 raft_consensus.cc:3055] T 17f97fe9a93f4a86b1e2bad621bfb561 P 8c1822e697d64b758a23b11c8fdeb59a [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:18:28.949815 15819 raft_consensus.cc:3060] T 17f97fe9a93f4a86b1e2bad621bfb561 P 62bbd271280e4b0a9ddf9ab46151e455 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:28.949995 15671 raft_consensus.cc:740] T 17f97fe9a93f4a86b1e2bad621bfb561 P 8c1822e697d64b758a23b11c8fdeb59a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8c1822e697d64b758a23b11c8fdeb59a, State: Running, Role: LEADER
I20260709 13:18:28.950778 15671 consensus_queue.cc:260] T 17f97fe9a93f4a86b1e2bad621bfb561 P 8c1822e697d64b758a23b11c8fdeb59a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c1822e697d64b758a23b11c8fdeb59a" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 39067 } } peers { permanent_uuid: "7cd7c680ef2e474c981ac81abadecfde" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 40117 } } peers { permanent_uuid: "62bbd271280e4b0a9ddf9ab46151e455" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 33917 } }
I20260709 13:18:28.951787 15671 raft_consensus.cc:3060] T 17f97fe9a93f4a86b1e2bad621bfb561 P 8c1822e697d64b758a23b11c8fdeb59a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:28.954653 15819 raft_consensus.cc:2468] T 17f97fe9a93f4a86b1e2bad621bfb561 P 62bbd271280e4b0a9ddf9ab46151e455 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7cd7c680ef2e474c981ac81abadecfde in term 2.
I20260709 13:18:28.955690 15712 leader_election.cc:304] T 17f97fe9a93f4a86b1e2bad621bfb561 P 7cd7c680ef2e474c981ac81abadecfde [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 62bbd271280e4b0a9ddf9ab46151e455, 7cd7c680ef2e474c981ac81abadecfde; no voters: 
I20260709 13:18:28.956367 15884 raft_consensus.cc:2804] T 17f97fe9a93f4a86b1e2bad621bfb561 P 7cd7c680ef2e474c981ac81abadecfde [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:18:28.956641 15671 raft_consensus.cc:2468] T 17f97fe9a93f4a86b1e2bad621bfb561 P 8c1822e697d64b758a23b11c8fdeb59a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7cd7c680ef2e474c981ac81abadecfde in term 2.
I20260709 13:18:28.956774 15884 raft_consensus.cc:697] T 17f97fe9a93f4a86b1e2bad621bfb561 P 7cd7c680ef2e474c981ac81abadecfde [term 2 LEADER]: Becoming Leader. State: Replica: 7cd7c680ef2e474c981ac81abadecfde, State: Running, Role: LEADER
I20260709 13:18:28.957628 15884 consensus_queue.cc:237] T 17f97fe9a93f4a86b1e2bad621bfb561 P 7cd7c680ef2e474c981ac81abadecfde [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c1822e697d64b758a23b11c8fdeb59a" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 39067 } } peers { permanent_uuid: "7cd7c680ef2e474c981ac81abadecfde" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 40117 } } peers { permanent_uuid: "62bbd271280e4b0a9ddf9ab46151e455" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 33917 } }
I20260709 13:18:28.963986 15567 catalog_manager.cc:5697] T 17f97fe9a93f4a86b1e2bad621bfb561 P 7cd7c680ef2e474c981ac81abadecfde reported cstate change: term changed from 1 to 2, leader changed from 8c1822e697d64b758a23b11c8fdeb59a (127.14.177.129) to 7cd7c680ef2e474c981ac81abadecfde (127.14.177.130). New cstate: current_term: 2 leader_uuid: "7cd7c680ef2e474c981ac81abadecfde" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c1822e697d64b758a23b11c8fdeb59a" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 39067 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7cd7c680ef2e474c981ac81abadecfde" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 40117 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "62bbd271280e4b0a9ddf9ab46151e455" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 33917 } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:29.032994 15670 raft_consensus.cc:1275] T 8309d9a5a4ef4e6d914751bc0f410015 P 8c1822e697d64b758a23b11c8fdeb59a [term 2 FOLLOWER]: Refusing update from remote peer 7cd7c680ef2e474c981ac81abadecfde: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:18:29.034262 15884 consensus_queue.cc:1048] T 8309d9a5a4ef4e6d914751bc0f410015 P 7cd7c680ef2e474c981ac81abadecfde [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c1822e697d64b758a23b11c8fdeb59a" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 39067 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:18:29.041626 15819 raft_consensus.cc:1275] T 8309d9a5a4ef4e6d914751bc0f410015 P 62bbd271280e4b0a9ddf9ab46151e455 [term 2 FOLLOWER]: Refusing update from remote peer 7cd7c680ef2e474c981ac81abadecfde: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:18:29.043133 15884 consensus_queue.cc:1048] T 8309d9a5a4ef4e6d914751bc0f410015 P 7cd7c680ef2e474c981ac81abadecfde [LEADER]: Connected to new peer: Peer: permanent_uuid: "62bbd271280e4b0a9ddf9ab46151e455" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 33917 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:18:29.050418 15670 raft_consensus.cc:1275] T 48e8b6e82d7b42e6bdd4b0a8a25dfd9e P 8c1822e697d64b758a23b11c8fdeb59a [term 2 FOLLOWER]: Refusing update from remote peer 7cd7c680ef2e474c981ac81abadecfde: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:18:29.051761 15884 consensus_queue.cc:1048] T 48e8b6e82d7b42e6bdd4b0a8a25dfd9e P 7cd7c680ef2e474c981ac81abadecfde [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c1822e697d64b758a23b11c8fdeb59a" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 39067 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:18:29.059557 15819 raft_consensus.cc:1275] T 48e8b6e82d7b42e6bdd4b0a8a25dfd9e P 62bbd271280e4b0a9ddf9ab46151e455 [term 2 FOLLOWER]: Refusing update from remote peer 7cd7c680ef2e474c981ac81abadecfde: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:18:29.061132 15867 consensus_queue.cc:1048] T 48e8b6e82d7b42e6bdd4b0a8a25dfd9e P 7cd7c680ef2e474c981ac81abadecfde [LEADER]: Connected to new peer: Peer: permanent_uuid: "62bbd271280e4b0a9ddf9ab46151e455" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 33917 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:18:29.148138 15744 raft_consensus.cc:1275] T 566eae6d93af4ec492b9b1a270dc03b7 P 7cd7c680ef2e474c981ac81abadecfde [term 2 FOLLOWER]: Refusing update from remote peer 8c1822e697d64b758a23b11c8fdeb59a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:18:29.149487 15911 consensus_queue.cc:1048] T 566eae6d93af4ec492b9b1a270dc03b7 P 8c1822e697d64b758a23b11c8fdeb59a [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cd7c680ef2e474c981ac81abadecfde" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 40117 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:18:29.157370 15819 raft_consensus.cc:1275] T 566eae6d93af4ec492b9b1a270dc03b7 P 62bbd271280e4b0a9ddf9ab46151e455 [term 2 FOLLOWER]: Refusing update from remote peer 8c1822e697d64b758a23b11c8fdeb59a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:18:29.158866 15907 consensus_queue.cc:1048] T 566eae6d93af4ec492b9b1a270dc03b7 P 8c1822e697d64b758a23b11c8fdeb59a [LEADER]: Connected to new peer: Peer: permanent_uuid: "62bbd271280e4b0a9ddf9ab46151e455" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 33917 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:18:29.170851 15744 raft_consensus.cc:1275] T 0ce7d20d574f47649d3ad0f3c382198d P 7cd7c680ef2e474c981ac81abadecfde [term 2 FOLLOWER]: Refusing update from remote peer 8c1822e697d64b758a23b11c8fdeb59a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:18:29.172060 15907 consensus_queue.cc:1048] T 0ce7d20d574f47649d3ad0f3c382198d P 8c1822e697d64b758a23b11c8fdeb59a [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cd7c680ef2e474c981ac81abadecfde" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 40117 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:18:29.180089 15819 raft_consensus.cc:1275] T 0ce7d20d574f47649d3ad0f3c382198d P 62bbd271280e4b0a9ddf9ab46151e455 [term 2 FOLLOWER]: Refusing update from remote peer 8c1822e697d64b758a23b11c8fdeb59a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:18:29.181380 15907 consensus_queue.cc:1048] T 0ce7d20d574f47649d3ad0f3c382198d P 8c1822e697d64b758a23b11c8fdeb59a [LEADER]: Connected to new peer: Peer: permanent_uuid: "62bbd271280e4b0a9ddf9ab46151e455" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 33917 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:18:29.256233 15744 raft_consensus.cc:1275] T 8cff7911abd649b9a276ddae431311d1 P 7cd7c680ef2e474c981ac81abadecfde [term 2 FOLLOWER]: Refusing update from remote peer 8c1822e697d64b758a23b11c8fdeb59a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:18:29.257488 15907 consensus_queue.cc:1048] T 8cff7911abd649b9a276ddae431311d1 P 8c1822e697d64b758a23b11c8fdeb59a [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cd7c680ef2e474c981ac81abadecfde" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 40117 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:18:29.266321 15819 raft_consensus.cc:1275] T 8cff7911abd649b9a276ddae431311d1 P 62bbd271280e4b0a9ddf9ab46151e455 [term 2 FOLLOWER]: Refusing update from remote peer 8c1822e697d64b758a23b11c8fdeb59a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:18:29.268812 15907 consensus_queue.cc:1048] T 8cff7911abd649b9a276ddae431311d1 P 8c1822e697d64b758a23b11c8fdeb59a [LEADER]: Connected to new peer: Peer: permanent_uuid: "62bbd271280e4b0a9ddf9ab46151e455" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 33917 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:18:29.281435 15670 raft_consensus.cc:1275] T 408866bb47904e4189bc235ca5b0c7c7 P 8c1822e697d64b758a23b11c8fdeb59a [term 2 FOLLOWER]: Refusing update from remote peer 7cd7c680ef2e474c981ac81abadecfde: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:18:29.282725 15867 consensus_queue.cc:1048] T 408866bb47904e4189bc235ca5b0c7c7 P 7cd7c680ef2e474c981ac81abadecfde [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c1822e697d64b758a23b11c8fdeb59a" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 39067 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:18:29.287575 15744 raft_consensus.cc:1275] T 4d83a05b849340c186cc995649575d3b P 7cd7c680ef2e474c981ac81abadecfde [term 2 FOLLOWER]: Refusing update from remote peer 8c1822e697d64b758a23b11c8fdeb59a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:18:29.289563 15907 consensus_queue.cc:1048] T 4d83a05b849340c186cc995649575d3b P 8c1822e697d64b758a23b11c8fdeb59a [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cd7c680ef2e474c981ac81abadecfde" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 40117 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:18:29.291105 15819 raft_consensus.cc:1275] T 408866bb47904e4189bc235ca5b0c7c7 P 62bbd271280e4b0a9ddf9ab46151e455 [term 2 FOLLOWER]: Refusing update from remote peer 7cd7c680ef2e474c981ac81abadecfde: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:18:29.295356 15903 consensus_queue.cc:1048] T 408866bb47904e4189bc235ca5b0c7c7 P 7cd7c680ef2e474c981ac81abadecfde [LEADER]: Connected to new peer: Peer: permanent_uuid: "62bbd271280e4b0a9ddf9ab46151e455" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 33917 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:18:29.299793 15820 raft_consensus.cc:1275] T 4d83a05b849340c186cc995649575d3b P 62bbd271280e4b0a9ddf9ab46151e455 [term 2 FOLLOWER]: Refusing update from remote peer 8c1822e697d64b758a23b11c8fdeb59a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:18:29.303973 15868 consensus_queue.cc:1048] T 4d83a05b849340c186cc995649575d3b P 8c1822e697d64b758a23b11c8fdeb59a [LEADER]: Connected to new peer: Peer: permanent_uuid: "62bbd271280e4b0a9ddf9ab46151e455" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 33917 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:18:29.500626 15819 raft_consensus.cc:1275] T 17f97fe9a93f4a86b1e2bad621bfb561 P 62bbd271280e4b0a9ddf9ab46151e455 [term 2 FOLLOWER]: Refusing update from remote peer 7cd7c680ef2e474c981ac81abadecfde: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:18:29.501931 15903 consensus_queue.cc:1048] T 17f97fe9a93f4a86b1e2bad621bfb561 P 7cd7c680ef2e474c981ac81abadecfde [LEADER]: Connected to new peer: Peer: permanent_uuid: "62bbd271280e4b0a9ddf9ab46151e455" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 33917 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:18:29.510356 15670 raft_consensus.cc:1275] T 17f97fe9a93f4a86b1e2bad621bfb561 P 8c1822e697d64b758a23b11c8fdeb59a [term 2 FOLLOWER]: Refusing update from remote peer 7cd7c680ef2e474c981ac81abadecfde: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:18:29.511724 15884 consensus_queue.cc:1048] T 17f97fe9a93f4a86b1e2bad621bfb561 P 7cd7c680ef2e474c981ac81abadecfde [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c1822e697d64b758a23b11c8fdeb59a" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 39067 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:18:31.511466 15046 auto_leader_rebalancer-test.cc:348] The leader distribution is 
I20260709 13:18:31.512192 15046 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:18:31.532313 15670 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "566eae6d93af4ec492b9b1a270dc03b7"
dest_uuid: "8c1822e697d64b758a23b11c8fdeb59a"
mode: GRACEFUL
new_leader_uuid: "62bbd271280e4b0a9ddf9ab46151e455"
 from {username='slave'} at 127.0.0.1:56558
I20260709 13:18:31.532860 15670 raft_consensus.cc:606] T 566eae6d93af4ec492b9b1a270dc03b7 P 8c1822e697d64b758a23b11c8fdeb59a [term 2 LEADER]: Received request to transfer leadership to TS 62bbd271280e4b0a9ddf9ab46151e455
I20260709 13:18:31.535481 15670 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8cff7911abd649b9a276ddae431311d1"
dest_uuid: "8c1822e697d64b758a23b11c8fdeb59a"
mode: GRACEFUL
new_leader_uuid: "62bbd271280e4b0a9ddf9ab46151e455"
 from {username='slave'} at 127.0.0.1:56558
I20260709 13:18:31.535979 15670 raft_consensus.cc:606] T 8cff7911abd649b9a276ddae431311d1 P 8c1822e697d64b758a23b11c8fdeb59a [term 2 LEADER]: Received request to transfer leadership to TS 62bbd271280e4b0a9ddf9ab46151e455
I20260709 13:18:31.537027 15046 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 13:18:31.537449 15046 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:18:31.665259 15961 raft_consensus.cc:993] T 566eae6d93af4ec492b9b1a270dc03b7 P 8c1822e697d64b758a23b11c8fdeb59a: : Instructing follower 62bbd271280e4b0a9ddf9ab46151e455 to start an election
I20260709 13:18:31.665846 15911 raft_consensus.cc:1081] T 566eae6d93af4ec492b9b1a270dc03b7 P 8c1822e697d64b758a23b11c8fdeb59a [term 2 LEADER]: Signalling peer 62bbd271280e4b0a9ddf9ab46151e455 to start an election
I20260709 13:18:31.667593 15819 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "566eae6d93af4ec492b9b1a270dc03b7"
dest_uuid: "62bbd271280e4b0a9ddf9ab46151e455"
 from {username='slave'} at 127.0.0.1:40898
I20260709 13:18:31.668074 15819 raft_consensus.cc:493] T 566eae6d93af4ec492b9b1a270dc03b7 P 62bbd271280e4b0a9ddf9ab46151e455 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:18:31.668401 15819 raft_consensus.cc:3060] T 566eae6d93af4ec492b9b1a270dc03b7 P 62bbd271280e4b0a9ddf9ab46151e455 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:31.673061 15819 raft_consensus.cc:515] T 566eae6d93af4ec492b9b1a270dc03b7 P 62bbd271280e4b0a9ddf9ab46151e455 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c1822e697d64b758a23b11c8fdeb59a" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 39067 } } peers { permanent_uuid: "7cd7c680ef2e474c981ac81abadecfde" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 40117 } } peers { permanent_uuid: "62bbd271280e4b0a9ddf9ab46151e455" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 33917 } }
I20260709 13:18:31.675006 15819 leader_election.cc:290] T 566eae6d93af4ec492b9b1a270dc03b7 P 62bbd271280e4b0a9ddf9ab46151e455 [CANDIDATE]: Term 3 election: Requested vote from peers 8c1822e697d64b758a23b11c8fdeb59a (127.14.177.129:39067), 7cd7c680ef2e474c981ac81abadecfde (127.14.177.130:40117)
I20260709 13:18:31.675761 15670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "566eae6d93af4ec492b9b1a270dc03b7" candidate_uuid: "62bbd271280e4b0a9ddf9ab46151e455" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "8c1822e697d64b758a23b11c8fdeb59a"
I20260709 13:18:31.676152 15746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "566eae6d93af4ec492b9b1a270dc03b7" candidate_uuid: "62bbd271280e4b0a9ddf9ab46151e455" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "7cd7c680ef2e474c981ac81abadecfde"
I20260709 13:18:31.676362 15670 raft_consensus.cc:3055] T 566eae6d93af4ec492b9b1a270dc03b7 P 8c1822e697d64b758a23b11c8fdeb59a [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:18:31.676719 15670 raft_consensus.cc:740] T 566eae6d93af4ec492b9b1a270dc03b7 P 8c1822e697d64b758a23b11c8fdeb59a [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 8c1822e697d64b758a23b11c8fdeb59a, State: Running, Role: LEADER
I20260709 13:18:31.676755 15746 raft_consensus.cc:3060] T 566eae6d93af4ec492b9b1a270dc03b7 P 7cd7c680ef2e474c981ac81abadecfde [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:31.677623 15670 consensus_queue.cc:260] T 566eae6d93af4ec492b9b1a270dc03b7 P 8c1822e697d64b758a23b11c8fdeb59a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c1822e697d64b758a23b11c8fdeb59a" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 39067 } } peers { permanent_uuid: "7cd7c680ef2e474c981ac81abadecfde" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 40117 } } peers { permanent_uuid: "62bbd271280e4b0a9ddf9ab46151e455" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 33917 } }
I20260709 13:18:31.678563 15670 raft_consensus.cc:3060] T 566eae6d93af4ec492b9b1a270dc03b7 P 8c1822e697d64b758a23b11c8fdeb59a [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:31.683916 15746 raft_consensus.cc:2468] T 566eae6d93af4ec492b9b1a270dc03b7 P 7cd7c680ef2e474c981ac81abadecfde [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 62bbd271280e4b0a9ddf9ab46151e455 in term 3.
I20260709 13:18:31.684240 15670 raft_consensus.cc:2468] T 566eae6d93af4ec492b9b1a270dc03b7 P 8c1822e697d64b758a23b11c8fdeb59a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 62bbd271280e4b0a9ddf9ab46151e455 in term 3.
I20260709 13:18:31.685060 15785 leader_election.cc:304] T 566eae6d93af4ec492b9b1a270dc03b7 P 62bbd271280e4b0a9ddf9ab46151e455 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 62bbd271280e4b0a9ddf9ab46151e455, 7cd7c680ef2e474c981ac81abadecfde; no voters: 
I20260709 13:18:31.685855 15866 raft_consensus.cc:2804] T 566eae6d93af4ec492b9b1a270dc03b7 P 62bbd271280e4b0a9ddf9ab46151e455 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:18:31.686843 15866 raft_consensus.cc:697] T 566eae6d93af4ec492b9b1a270dc03b7 P 62bbd271280e4b0a9ddf9ab46151e455 [term 3 LEADER]: Becoming Leader. State: Replica: 62bbd271280e4b0a9ddf9ab46151e455, State: Running, Role: LEADER
I20260709 13:18:31.687664 15866 consensus_queue.cc:237] T 566eae6d93af4ec492b9b1a270dc03b7 P 62bbd271280e4b0a9ddf9ab46151e455 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c1822e697d64b758a23b11c8fdeb59a" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 39067 } } peers { permanent_uuid: "7cd7c680ef2e474c981ac81abadecfde" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 40117 } } peers { permanent_uuid: "62bbd271280e4b0a9ddf9ab46151e455" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 33917 } }
I20260709 13:18:31.694984 15566 catalog_manager.cc:5697] T 566eae6d93af4ec492b9b1a270dc03b7 P 62bbd271280e4b0a9ddf9ab46151e455 reported cstate change: term changed from 2 to 3, leader changed from 8c1822e697d64b758a23b11c8fdeb59a (127.14.177.129) to 62bbd271280e4b0a9ddf9ab46151e455 (127.14.177.131). New cstate: current_term: 3 leader_uuid: "62bbd271280e4b0a9ddf9ab46151e455" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c1822e697d64b758a23b11c8fdeb59a" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 39067 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7cd7c680ef2e474c981ac81abadecfde" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 40117 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "62bbd271280e4b0a9ddf9ab46151e455" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 33917 } health_report { overall_health: HEALTHY } } }
I20260709 13:18:31.868471 15961 raft_consensus.cc:993] T 8cff7911abd649b9a276ddae431311d1 P 8c1822e697d64b758a23b11c8fdeb59a: : Instructing follower 62bbd271280e4b0a9ddf9ab46151e455 to start an election
I20260709 13:18:31.868975 15911 raft_consensus.cc:1081] T 8cff7911abd649b9a276ddae431311d1 P 8c1822e697d64b758a23b11c8fdeb59a [term 2 LEADER]: Signalling peer 62bbd271280e4b0a9ddf9ab46151e455 to start an election
I20260709 13:18:31.870430 15819 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "8cff7911abd649b9a276ddae431311d1"
dest_uuid: "62bbd271280e4b0a9ddf9ab46151e455"
 from {username='slave'} at 127.0.0.1:40898
I20260709 13:18:31.870963 15819 raft_consensus.cc:493] T 8cff7911abd649b9a276ddae431311d1 P 62bbd271280e4b0a9ddf9ab46151e455 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:18:31.871294 15819 raft_consensus.cc:3060] T 8cff7911abd649b9a276ddae431311d1 P 62bbd271280e4b0a9ddf9ab46151e455 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:31.875361 15819 raft_consensus.cc:515] T 8cff7911abd649b9a276ddae431311d1 P 62bbd271280e4b0a9ddf9ab46151e455 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c1822e697d64b758a23b11c8fdeb59a" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 39067 } } peers { permanent_uuid: "7cd7c680ef2e474c981ac81abadecfde" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 40117 } } peers { permanent_uuid: "62bbd271280e4b0a9ddf9ab46151e455" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 33917 } }
I20260709 13:18:31.876753 15819 leader_election.cc:290] T 8cff7911abd649b9a276ddae431311d1 P 62bbd271280e4b0a9ddf9ab46151e455 [CANDIDATE]: Term 3 election: Requested vote from peers 8c1822e697d64b758a23b11c8fdeb59a (127.14.177.129:39067), 7cd7c680ef2e474c981ac81abadecfde (127.14.177.130:40117)
I20260709 13:18:31.877663 15670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cff7911abd649b9a276ddae431311d1" candidate_uuid: "62bbd271280e4b0a9ddf9ab46151e455" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "8c1822e697d64b758a23b11c8fdeb59a"
I20260709 13:18:31.877795 15746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cff7911abd649b9a276ddae431311d1" candidate_uuid: "62bbd271280e4b0a9ddf9ab46151e455" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "7cd7c680ef2e474c981ac81abadecfde"
I20260709 13:18:31.878194 15670 raft_consensus.cc:3055] T 8cff7911abd649b9a276ddae431311d1 P 8c1822e697d64b758a23b11c8fdeb59a [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:18:31.878328 15746 raft_consensus.cc:3060] T 8cff7911abd649b9a276ddae431311d1 P 7cd7c680ef2e474c981ac81abadecfde [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:31.878510 15670 raft_consensus.cc:740] T 8cff7911abd649b9a276ddae431311d1 P 8c1822e697d64b758a23b11c8fdeb59a [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 8c1822e697d64b758a23b11c8fdeb59a, State: Running, Role: LEADER
I20260709 13:18:31.879313 15670 consensus_queue.cc:260] T 8cff7911abd649b9a276ddae431311d1 P 8c1822e697d64b758a23b11c8fdeb59a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c1822e697d64b758a23b11c8fdeb59a" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 39067 } } peers { permanent_uuid: "7cd7c680ef2e474c981ac81abadecfde" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 40117 } } peers { permanent_uuid: "62bbd271280e4b0a9ddf9ab46151e455" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 33917 } }
I20260709 13:18:31.880185 15670 raft_consensus.cc:3060] T 8cff7911abd649b9a276ddae431311d1 P 8c1822e697d64b758a23b11c8fdeb59a [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:31.883096 15746 raft_consensus.cc:2468] T 8cff7911abd649b9a276ddae431311d1 P 7cd7c680ef2e474c981ac81abadecfde [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 62bbd271280e4b0a9ddf9ab46151e455 in term 3.
I20260709 13:18:31.884217 15785 leader_election.cc:304] T 8cff7911abd649b9a276ddae431311d1 P 62bbd271280e4b0a9ddf9ab46151e455 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 62bbd271280e4b0a9ddf9ab46151e455, 7cd7c680ef2e474c981ac81abadecfde; no voters: 
I20260709 13:18:31.884981 15866 raft_consensus.cc:2804] T 8cff7911abd649b9a276ddae431311d1 P 62bbd271280e4b0a9ddf9ab46151e455 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:18:31.885111 15670 raft_consensus.cc:2468] T 8cff7911abd649b9a276ddae431311d1 P 8c1822e697d64b758a23b11c8fdeb59a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 62bbd271280e4b0a9ddf9ab46151e455 in term 3.
I20260709 13:18:31.885452 15866 raft_consensus.cc:697] T 8cff7911abd649b9a276ddae431311d1 P 62bbd271280e4b0a9ddf9ab46151e455 [term 3 LEADER]: Becoming Leader. State: Replica: 62bbd271280e4b0a9ddf9ab46151e455, State: Running, Role: LEADER
I20260709 13:18:31.886324 15866 consensus_queue.cc:237] T 8cff7911abd649b9a276ddae431311d1 P 62bbd271280e4b0a9ddf9ab46151e455 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c1822e697d64b758a23b11c8fdeb59a" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 39067 } } peers { permanent_uuid: "7cd7c680ef2e474c981ac81abadecfde" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 40117 } } peers { permanent_uuid: "62bbd271280e4b0a9ddf9ab46151e455" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 33917 } }
I20260709 13:18:31.894052 15566 catalog_manager.cc:5697] T 8cff7911abd649b9a276ddae431311d1 P 62bbd271280e4b0a9ddf9ab46151e455 reported cstate change: term changed from 2 to 3, leader changed from 8c1822e697d64b758a23b11c8fdeb59a (127.14.177.129) to 62bbd271280e4b0a9ddf9ab46151e455 (127.14.177.131). New cstate: current_term: 3 leader_uuid: "62bbd271280e4b0a9ddf9ab46151e455" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c1822e697d64b758a23b11c8fdeb59a" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 39067 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7cd7c680ef2e474c981ac81abadecfde" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 40117 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "62bbd271280e4b0a9ddf9ab46151e455" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 33917 } health_report { overall_health: HEALTHY } } }
I20260709 13:18:32.168537 15746 raft_consensus.cc:1275] T 566eae6d93af4ec492b9b1a270dc03b7 P 7cd7c680ef2e474c981ac81abadecfde [term 3 FOLLOWER]: Refusing update from remote peer 62bbd271280e4b0a9ddf9ab46151e455: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:18:32.170231 15866 consensus_queue.cc:1048] T 566eae6d93af4ec492b9b1a270dc03b7 P 62bbd271280e4b0a9ddf9ab46151e455 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cd7c680ef2e474c981ac81abadecfde" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 40117 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:18:32.186404 15670 raft_consensus.cc:1275] T 566eae6d93af4ec492b9b1a270dc03b7 P 8c1822e697d64b758a23b11c8fdeb59a [term 3 FOLLOWER]: Refusing update from remote peer 62bbd271280e4b0a9ddf9ab46151e455: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:18:32.187793 15866 consensus_queue.cc:1048] T 566eae6d93af4ec492b9b1a270dc03b7 P 62bbd271280e4b0a9ddf9ab46151e455 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c1822e697d64b758a23b11c8fdeb59a" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 39067 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:18:32.295992 15670 raft_consensus.cc:1275] T 8cff7911abd649b9a276ddae431311d1 P 8c1822e697d64b758a23b11c8fdeb59a [term 3 FOLLOWER]: Refusing update from remote peer 62bbd271280e4b0a9ddf9ab46151e455: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:18:32.297375 15866 consensus_queue.cc:1048] T 8cff7911abd649b9a276ddae431311d1 P 62bbd271280e4b0a9ddf9ab46151e455 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c1822e697d64b758a23b11c8fdeb59a" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 39067 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:18:32.305423 15746 raft_consensus.cc:1275] T 8cff7911abd649b9a276ddae431311d1 P 7cd7c680ef2e474c981ac81abadecfde [term 3 FOLLOWER]: Refusing update from remote peer 62bbd271280e4b0a9ddf9ab46151e455: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:18:32.306810 15866 consensus_queue.cc:1048] T 8cff7911abd649b9a276ddae431311d1 P 62bbd271280e4b0a9ddf9ab46151e455 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cd7c680ef2e474c981ac81abadecfde" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 40117 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:18:32.538321 15046 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:18:32.551409 15820 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8cff7911abd649b9a276ddae431311d1"
dest_uuid: "62bbd271280e4b0a9ddf9ab46151e455"
mode: GRACEFUL
new_leader_uuid: "8c1822e697d64b758a23b11c8fdeb59a"
 from {username='slave'} at 127.0.0.1:40902
I20260709 13:18:32.551964 15820 raft_consensus.cc:606] T 8cff7911abd649b9a276ddae431311d1 P 62bbd271280e4b0a9ddf9ab46151e455 [term 3 LEADER]: Received request to transfer leadership to TS 8c1822e697d64b758a23b11c8fdeb59a
I20260709 13:18:32.559427 15820 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "566eae6d93af4ec492b9b1a270dc03b7"
dest_uuid: "62bbd271280e4b0a9ddf9ab46151e455"
mode: GRACEFUL
new_leader_uuid: "8c1822e697d64b758a23b11c8fdeb59a"
 from {username='slave'} at 127.0.0.1:40902
I20260709 13:18:32.559942 15820 raft_consensus.cc:606] T 566eae6d93af4ec492b9b1a270dc03b7 P 62bbd271280e4b0a9ddf9ab46151e455 [term 3 LEADER]: Received request to transfer leadership to TS 8c1822e697d64b758a23b11c8fdeb59a
I20260709 13:18:32.566738 15746 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "408866bb47904e4189bc235ca5b0c7c7"
dest_uuid: "7cd7c680ef2e474c981ac81abadecfde"
mode: GRACEFUL
new_leader_uuid: "8c1822e697d64b758a23b11c8fdeb59a"
 from {username='slave'} at 127.0.0.1:49470
I20260709 13:18:32.567219 15746 raft_consensus.cc:606] T 408866bb47904e4189bc235ca5b0c7c7 P 7cd7c680ef2e474c981ac81abadecfde [term 2 LEADER]: Received request to transfer leadership to TS 8c1822e697d64b758a23b11c8fdeb59a
I20260709 13:18:32.571923 15746 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "17f97fe9a93f4a86b1e2bad621bfb561"
dest_uuid: "7cd7c680ef2e474c981ac81abadecfde"
mode: GRACEFUL
new_leader_uuid: "8c1822e697d64b758a23b11c8fdeb59a"
 from {username='slave'} at 127.0.0.1:49470
I20260709 13:18:32.572373 15746 raft_consensus.cc:606] T 17f97fe9a93f4a86b1e2bad621bfb561 P 7cd7c680ef2e474c981ac81abadecfde [term 2 LEADER]: Received request to transfer leadership to TS 8c1822e697d64b758a23b11c8fdeb59a
I20260709 13:18:32.576892 15746 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8309d9a5a4ef4e6d914751bc0f410015"
dest_uuid: "7cd7c680ef2e474c981ac81abadecfde"
mode: GRACEFUL
new_leader_uuid: "8c1822e697d64b758a23b11c8fdeb59a"
 from {username='slave'} at 127.0.0.1:49470
I20260709 13:18:32.577342 15746 raft_consensus.cc:606] T 8309d9a5a4ef4e6d914751bc0f410015 P 7cd7c680ef2e474c981ac81abadecfde [term 2 LEADER]: Received request to transfer leadership to TS 8c1822e697d64b758a23b11c8fdeb59a
I20260709 13:18:32.589627 15977 raft_consensus.cc:993] T 17f97fe9a93f4a86b1e2bad621bfb561 P 7cd7c680ef2e474c981ac81abadecfde: : Instructing follower 8c1822e697d64b758a23b11c8fdeb59a to start an election
I20260709 13:18:32.590039 15903 raft_consensus.cc:1081] T 17f97fe9a93f4a86b1e2bad621bfb561 P 7cd7c680ef2e474c981ac81abadecfde [term 2 LEADER]: Signalling peer 8c1822e697d64b758a23b11c8fdeb59a to start an election
I20260709 13:18:32.591398 15670 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "17f97fe9a93f4a86b1e2bad621bfb561"
dest_uuid: "8c1822e697d64b758a23b11c8fdeb59a"
 from {username='slave'} at 127.0.0.1:42812
I20260709 13:18:32.591871 15670 raft_consensus.cc:493] T 17f97fe9a93f4a86b1e2bad621bfb561 P 8c1822e697d64b758a23b11c8fdeb59a [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:18:32.592218 15670 raft_consensus.cc:3060] T 17f97fe9a93f4a86b1e2bad621bfb561 P 8c1822e697d64b758a23b11c8fdeb59a [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:32.596920 15670 raft_consensus.cc:515] T 17f97fe9a93f4a86b1e2bad621bfb561 P 8c1822e697d64b758a23b11c8fdeb59a [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c1822e697d64b758a23b11c8fdeb59a" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 39067 } } peers { permanent_uuid: "7cd7c680ef2e474c981ac81abadecfde" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 40117 } } peers { permanent_uuid: "62bbd271280e4b0a9ddf9ab46151e455" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 33917 } }
I20260709 13:18:32.598596 15670 leader_election.cc:290] T 17f97fe9a93f4a86b1e2bad621bfb561 P 8c1822e697d64b758a23b11c8fdeb59a [CANDIDATE]: Term 3 election: Requested vote from peers 7cd7c680ef2e474c981ac81abadecfde (127.14.177.130:40117), 62bbd271280e4b0a9ddf9ab46151e455 (127.14.177.131:33917)
I20260709 13:18:32.599557 15746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17f97fe9a93f4a86b1e2bad621bfb561" candidate_uuid: "8c1822e697d64b758a23b11c8fdeb59a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "7cd7c680ef2e474c981ac81abadecfde"
I20260709 13:18:32.599695 15820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17f97fe9a93f4a86b1e2bad621bfb561" candidate_uuid: "8c1822e697d64b758a23b11c8fdeb59a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "62bbd271280e4b0a9ddf9ab46151e455"
I20260709 13:18:32.600108 15746 raft_consensus.cc:3055] T 17f97fe9a93f4a86b1e2bad621bfb561 P 7cd7c680ef2e474c981ac81abadecfde [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:18:32.600270 15820 raft_consensus.cc:3060] T 17f97fe9a93f4a86b1e2bad621bfb561 P 62bbd271280e4b0a9ddf9ab46151e455 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:32.600452 15746 raft_consensus.cc:740] T 17f97fe9a93f4a86b1e2bad621bfb561 P 7cd7c680ef2e474c981ac81abadecfde [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 7cd7c680ef2e474c981ac81abadecfde, State: Running, Role: LEADER
I20260709 13:18:32.601279 15746 consensus_queue.cc:260] T 17f97fe9a93f4a86b1e2bad621bfb561 P 7cd7c680ef2e474c981ac81abadecfde [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c1822e697d64b758a23b11c8fdeb59a" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 39067 } } peers { permanent_uuid: "7cd7c680ef2e474c981ac81abadecfde" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 40117 } } peers { permanent_uuid: "62bbd271280e4b0a9ddf9ab46151e455" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 33917 } }
I20260709 13:18:32.602176 15746 raft_consensus.cc:3060] T 17f97fe9a93f4a86b1e2bad621bfb561 P 7cd7c680ef2e474c981ac81abadecfde [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:32.605312 15820 raft_consensus.cc:2468] T 17f97fe9a93f4a86b1e2bad621bfb561 P 62bbd271280e4b0a9ddf9ab46151e455 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c1822e697d64b758a23b11c8fdeb59a in term 3.
I20260709 13:18:32.606470 15637 leader_election.cc:304] T 17f97fe9a93f4a86b1e2bad621bfb561 P 8c1822e697d64b758a23b11c8fdeb59a [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 62bbd271280e4b0a9ddf9ab46151e455, 8c1822e697d64b758a23b11c8fdeb59a; no voters: 
I20260709 13:18:32.606623 15746 raft_consensus.cc:2468] T 17f97fe9a93f4a86b1e2bad621bfb561 P 7cd7c680ef2e474c981ac81abadecfde [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c1822e697d64b758a23b11c8fdeb59a in term 3.
I20260709 13:18:32.607193 15961 raft_consensus.cc:2804] T 17f97fe9a93f4a86b1e2bad621bfb561 P 8c1822e697d64b758a23b11c8fdeb59a [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:18:32.607652 15961 raft_consensus.cc:697] T 17f97fe9a93f4a86b1e2bad621bfb561 P 8c1822e697d64b758a23b11c8fdeb59a [term 3 LEADER]: Becoming Leader. State: Replica: 8c1822e697d64b758a23b11c8fdeb59a, State: Running, Role: LEADER
I20260709 13:18:32.608374 15961 consensus_queue.cc:237] T 17f97fe9a93f4a86b1e2bad621bfb561 P 8c1822e697d64b758a23b11c8fdeb59a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c1822e697d64b758a23b11c8fdeb59a" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 39067 } } peers { permanent_uuid: "7cd7c680ef2e474c981ac81abadecfde" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 40117 } } peers { permanent_uuid: "62bbd271280e4b0a9ddf9ab46151e455" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 33917 } }
I20260709 13:18:32.614800 15566 catalog_manager.cc:5697] T 17f97fe9a93f4a86b1e2bad621bfb561 P 8c1822e697d64b758a23b11c8fdeb59a reported cstate change: term changed from 2 to 3, leader changed from 7cd7c680ef2e474c981ac81abadecfde (127.14.177.130) to 8c1822e697d64b758a23b11c8fdeb59a (127.14.177.129). New cstate: current_term: 3 leader_uuid: "8c1822e697d64b758a23b11c8fdeb59a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c1822e697d64b758a23b11c8fdeb59a" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 39067 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7cd7c680ef2e474c981ac81abadecfde" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 40117 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "62bbd271280e4b0a9ddf9ab46151e455" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 33917 } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:32.737320 15866 raft_consensus.cc:993] T 566eae6d93af4ec492b9b1a270dc03b7 P 62bbd271280e4b0a9ddf9ab46151e455: : Instructing follower 8c1822e697d64b758a23b11c8fdeb59a to start an election
I20260709 13:18:32.737823 15962 raft_consensus.cc:1081] T 566eae6d93af4ec492b9b1a270dc03b7 P 62bbd271280e4b0a9ddf9ab46151e455 [term 3 LEADER]: Signalling peer 8c1822e697d64b758a23b11c8fdeb59a to start an election
I20260709 13:18:32.739408 15670 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "566eae6d93af4ec492b9b1a270dc03b7"
dest_uuid: "8c1822e697d64b758a23b11c8fdeb59a"
 from {username='slave'} at 127.0.0.1:42822
I20260709 13:18:32.739864 15670 raft_consensus.cc:493] T 566eae6d93af4ec492b9b1a270dc03b7 P 8c1822e697d64b758a23b11c8fdeb59a [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:18:32.740139 15670 raft_consensus.cc:3060] T 566eae6d93af4ec492b9b1a270dc03b7 P 8c1822e697d64b758a23b11c8fdeb59a [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:32.744135 15670 raft_consensus.cc:515] T 566eae6d93af4ec492b9b1a270dc03b7 P 8c1822e697d64b758a23b11c8fdeb59a [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c1822e697d64b758a23b11c8fdeb59a" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 39067 } } peers { permanent_uuid: "7cd7c680ef2e474c981ac81abadecfde" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 40117 } } peers { permanent_uuid: "62bbd271280e4b0a9ddf9ab46151e455" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 33917 } }
I20260709 13:18:32.745535 15670 leader_election.cc:290] T 566eae6d93af4ec492b9b1a270dc03b7 P 8c1822e697d64b758a23b11c8fdeb59a [CANDIDATE]: Term 4 election: Requested vote from peers 7cd7c680ef2e474c981ac81abadecfde (127.14.177.130:40117), 62bbd271280e4b0a9ddf9ab46151e455 (127.14.177.131:33917)
I20260709 13:18:32.746487 15746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "566eae6d93af4ec492b9b1a270dc03b7" candidate_uuid: "8c1822e697d64b758a23b11c8fdeb59a" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "7cd7c680ef2e474c981ac81abadecfde"
I20260709 13:18:32.746609 15820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "566eae6d93af4ec492b9b1a270dc03b7" candidate_uuid: "8c1822e697d64b758a23b11c8fdeb59a" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "62bbd271280e4b0a9ddf9ab46151e455"
I20260709 13:18:32.747159 15746 raft_consensus.cc:3060] T 566eae6d93af4ec492b9b1a270dc03b7 P 7cd7c680ef2e474c981ac81abadecfde [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:32.747229 15820 raft_consensus.cc:3055] T 566eae6d93af4ec492b9b1a270dc03b7 P 62bbd271280e4b0a9ddf9ab46151e455 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:18:32.747686 15820 raft_consensus.cc:740] T 566eae6d93af4ec492b9b1a270dc03b7 P 62bbd271280e4b0a9ddf9ab46151e455 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 62bbd271280e4b0a9ddf9ab46151e455, State: Running, Role: LEADER
I20260709 13:18:32.748525 15820 consensus_queue.cc:260] T 566eae6d93af4ec492b9b1a270dc03b7 P 62bbd271280e4b0a9ddf9ab46151e455 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c1822e697d64b758a23b11c8fdeb59a" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 39067 } } peers { permanent_uuid: "7cd7c680ef2e474c981ac81abadecfde" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 40117 } } peers { permanent_uuid: "62bbd271280e4b0a9ddf9ab46151e455" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 33917 } }
I20260709 13:18:32.749769 15820 raft_consensus.cc:3060] T 566eae6d93af4ec492b9b1a270dc03b7 P 62bbd271280e4b0a9ddf9ab46151e455 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:32.754158 15746 raft_consensus.cc:2468] T 566eae6d93af4ec492b9b1a270dc03b7 P 7cd7c680ef2e474c981ac81abadecfde [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c1822e697d64b758a23b11c8fdeb59a in term 4.
I20260709 13:18:32.755528 15636 leader_election.cc:304] T 566eae6d93af4ec492b9b1a270dc03b7 P 8c1822e697d64b758a23b11c8fdeb59a [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7cd7c680ef2e474c981ac81abadecfde, 8c1822e697d64b758a23b11c8fdeb59a; no voters: 
I20260709 13:18:32.756347 15961 raft_consensus.cc:2804] T 566eae6d93af4ec492b9b1a270dc03b7 P 8c1822e697d64b758a23b11c8fdeb59a [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:18:32.756580 15820 raft_consensus.cc:2468] T 566eae6d93af4ec492b9b1a270dc03b7 P 62bbd271280e4b0a9ddf9ab46151e455 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c1822e697d64b758a23b11c8fdeb59a in term 4.
I20260709 13:18:32.756848 15961 raft_consensus.cc:697] T 566eae6d93af4ec492b9b1a270dc03b7 P 8c1822e697d64b758a23b11c8fdeb59a [term 4 LEADER]: Becoming Leader. State: Replica: 8c1822e697d64b758a23b11c8fdeb59a, State: Running, Role: LEADER
I20260709 13:18:32.758023 15961 consensus_queue.cc:237] T 566eae6d93af4ec492b9b1a270dc03b7 P 8c1822e697d64b758a23b11c8fdeb59a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c1822e697d64b758a23b11c8fdeb59a" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 39067 } } peers { permanent_uuid: "7cd7c680ef2e474c981ac81abadecfde" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 40117 } } peers { permanent_uuid: "62bbd271280e4b0a9ddf9ab46151e455" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 33917 } }
I20260709 13:18:32.764261 15566 catalog_manager.cc:5697] T 566eae6d93af4ec492b9b1a270dc03b7 P 8c1822e697d64b758a23b11c8fdeb59a reported cstate change: term changed from 3 to 4, leader changed from 62bbd271280e4b0a9ddf9ab46151e455 (127.14.177.131) to 8c1822e697d64b758a23b11c8fdeb59a (127.14.177.129). New cstate: current_term: 4 leader_uuid: "8c1822e697d64b758a23b11c8fdeb59a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c1822e697d64b758a23b11c8fdeb59a" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 39067 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7cd7c680ef2e474c981ac81abadecfde" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 40117 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "62bbd271280e4b0a9ddf9ab46151e455" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 33917 } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:32.896297 15977 raft_consensus.cc:993] T 408866bb47904e4189bc235ca5b0c7c7 P 7cd7c680ef2e474c981ac81abadecfde: : Instructing follower 8c1822e697d64b758a23b11c8fdeb59a to start an election
I20260709 13:18:32.896732 15977 raft_consensus.cc:1081] T 408866bb47904e4189bc235ca5b0c7c7 P 7cd7c680ef2e474c981ac81abadecfde [term 2 LEADER]: Signalling peer 8c1822e697d64b758a23b11c8fdeb59a to start an election
I20260709 13:18:32.898201 15670 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "408866bb47904e4189bc235ca5b0c7c7"
dest_uuid: "8c1822e697d64b758a23b11c8fdeb59a"
 from {username='slave'} at 127.0.0.1:42812
I20260709 13:18:32.898615 15670 raft_consensus.cc:493] T 408866bb47904e4189bc235ca5b0c7c7 P 8c1822e697d64b758a23b11c8fdeb59a [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:18:32.898885 15670 raft_consensus.cc:3060] T 408866bb47904e4189bc235ca5b0c7c7 P 8c1822e697d64b758a23b11c8fdeb59a [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:32.902987 15670 raft_consensus.cc:515] T 408866bb47904e4189bc235ca5b0c7c7 P 8c1822e697d64b758a23b11c8fdeb59a [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c1822e697d64b758a23b11c8fdeb59a" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 39067 } } peers { permanent_uuid: "7cd7c680ef2e474c981ac81abadecfde" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 40117 } } peers { permanent_uuid: "62bbd271280e4b0a9ddf9ab46151e455" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 33917 } }
I20260709 13:18:32.904340 15670 leader_election.cc:290] T 408866bb47904e4189bc235ca5b0c7c7 P 8c1822e697d64b758a23b11c8fdeb59a [CANDIDATE]: Term 3 election: Requested vote from peers 7cd7c680ef2e474c981ac81abadecfde (127.14.177.130:40117), 62bbd271280e4b0a9ddf9ab46151e455 (127.14.177.131:33917)
I20260709 13:18:32.905395 15746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "408866bb47904e4189bc235ca5b0c7c7" candidate_uuid: "8c1822e697d64b758a23b11c8fdeb59a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "7cd7c680ef2e474c981ac81abadecfde"
I20260709 13:18:32.905555 15820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "408866bb47904e4189bc235ca5b0c7c7" candidate_uuid: "8c1822e697d64b758a23b11c8fdeb59a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "62bbd271280e4b0a9ddf9ab46151e455"
I20260709 13:18:32.906002 15746 raft_consensus.cc:3055] T 408866bb47904e4189bc235ca5b0c7c7 P 7cd7c680ef2e474c981ac81abadecfde [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:18:32.906425 15746 raft_consensus.cc:740] T 408866bb47904e4189bc235ca5b0c7c7 P 7cd7c680ef2e474c981ac81abadecfde [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 7cd7c680ef2e474c981ac81abadecfde, State: Running, Role: LEADER
I20260709 13:18:32.906059 15820 raft_consensus.cc:3060] T 408866bb47904e4189bc235ca5b0c7c7 P 62bbd271280e4b0a9ddf9ab46151e455 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:32.907343 15746 consensus_queue.cc:260] T 408866bb47904e4189bc235ca5b0c7c7 P 7cd7c680ef2e474c981ac81abadecfde [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c1822e697d64b758a23b11c8fdeb59a" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 39067 } } peers { permanent_uuid: "7cd7c680ef2e474c981ac81abadecfde" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 40117 } } peers { permanent_uuid: "62bbd271280e4b0a9ddf9ab46151e455" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 33917 } }
I20260709 13:18:32.908690 15746 raft_consensus.cc:3060] T 408866bb47904e4189bc235ca5b0c7c7 P 7cd7c680ef2e474c981ac81abadecfde [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:32.910210 15866 raft_consensus.cc:993] T 8cff7911abd649b9a276ddae431311d1 P 62bbd271280e4b0a9ddf9ab46151e455: : Instructing follower 8c1822e697d64b758a23b11c8fdeb59a to start an election
I20260709 13:18:32.910693 15962 raft_consensus.cc:1081] T 8cff7911abd649b9a276ddae431311d1 P 62bbd271280e4b0a9ddf9ab46151e455 [term 3 LEADER]: Signalling peer 8c1822e697d64b758a23b11c8fdeb59a to start an election
I20260709 13:18:32.912351 15670 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "8cff7911abd649b9a276ddae431311d1"
dest_uuid: "8c1822e697d64b758a23b11c8fdeb59a"
 from {username='slave'} at 127.0.0.1:42822
I20260709 13:18:32.912591 15820 raft_consensus.cc:2468] T 408866bb47904e4189bc235ca5b0c7c7 P 62bbd271280e4b0a9ddf9ab46151e455 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c1822e697d64b758a23b11c8fdeb59a in term 3.
I20260709 13:18:32.912950 15670 raft_consensus.cc:493] T 8cff7911abd649b9a276ddae431311d1 P 8c1822e697d64b758a23b11c8fdeb59a [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:18:32.913427 15670 raft_consensus.cc:3060] T 8cff7911abd649b9a276ddae431311d1 P 8c1822e697d64b758a23b11c8fdeb59a [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:32.913715 15637 leader_election.cc:304] T 408866bb47904e4189bc235ca5b0c7c7 P 8c1822e697d64b758a23b11c8fdeb59a [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 62bbd271280e4b0a9ddf9ab46151e455, 8c1822e697d64b758a23b11c8fdeb59a; no voters: 
I20260709 13:18:32.914440 15961 raft_consensus.cc:2804] T 408866bb47904e4189bc235ca5b0c7c7 P 8c1822e697d64b758a23b11c8fdeb59a [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:18:32.914912 15961 raft_consensus.cc:697] T 408866bb47904e4189bc235ca5b0c7c7 P 8c1822e697d64b758a23b11c8fdeb59a [term 3 LEADER]: Becoming Leader. State: Replica: 8c1822e697d64b758a23b11c8fdeb59a, State: Running, Role: LEADER
I20260709 13:18:32.915732 15961 consensus_queue.cc:237] T 408866bb47904e4189bc235ca5b0c7c7 P 8c1822e697d64b758a23b11c8fdeb59a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c1822e697d64b758a23b11c8fdeb59a" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 39067 } } peers { permanent_uuid: "7cd7c680ef2e474c981ac81abadecfde" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 40117 } } peers { permanent_uuid: "62bbd271280e4b0a9ddf9ab46151e455" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 33917 } }
I20260709 13:18:32.916625 15746 raft_consensus.cc:2468] T 408866bb47904e4189bc235ca5b0c7c7 P 7cd7c680ef2e474c981ac81abadecfde [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c1822e697d64b758a23b11c8fdeb59a in term 3.
I20260709 13:18:32.919653 15670 raft_consensus.cc:515] T 8cff7911abd649b9a276ddae431311d1 P 8c1822e697d64b758a23b11c8fdeb59a [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c1822e697d64b758a23b11c8fdeb59a" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 39067 } } peers { permanent_uuid: "7cd7c680ef2e474c981ac81abadecfde" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 40117 } } peers { permanent_uuid: "62bbd271280e4b0a9ddf9ab46151e455" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 33917 } }
I20260709 13:18:32.921631 15670 leader_election.cc:290] T 8cff7911abd649b9a276ddae431311d1 P 8c1822e697d64b758a23b11c8fdeb59a [CANDIDATE]: Term 4 election: Requested vote from peers 7cd7c680ef2e474c981ac81abadecfde (127.14.177.130:40117), 62bbd271280e4b0a9ddf9ab46151e455 (127.14.177.131:33917)
I20260709 13:18:32.922597 15820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cff7911abd649b9a276ddae431311d1" candidate_uuid: "8c1822e697d64b758a23b11c8fdeb59a" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "62bbd271280e4b0a9ddf9ab46151e455"
I20260709 13:18:32.923089 15820 raft_consensus.cc:3055] T 8cff7911abd649b9a276ddae431311d1 P 62bbd271280e4b0a9ddf9ab46151e455 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:18:32.923388 15820 raft_consensus.cc:740] T 8cff7911abd649b9a276ddae431311d1 P 62bbd271280e4b0a9ddf9ab46151e455 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 62bbd271280e4b0a9ddf9ab46151e455, State: Running, Role: LEADER
I20260709 13:18:32.924046 15820 consensus_queue.cc:260] T 8cff7911abd649b9a276ddae431311d1 P 62bbd271280e4b0a9ddf9ab46151e455 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c1822e697d64b758a23b11c8fdeb59a" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 39067 } } peers { permanent_uuid: "7cd7c680ef2e474c981ac81abadecfde" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 40117 } } peers { permanent_uuid: "62bbd271280e4b0a9ddf9ab46151e455" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 33917 } }
I20260709 13:18:32.923913 15566 catalog_manager.cc:5697] T 408866bb47904e4189bc235ca5b0c7c7 P 8c1822e697d64b758a23b11c8fdeb59a reported cstate change: term changed from 2 to 3, leader changed from 7cd7c680ef2e474c981ac81abadecfde (127.14.177.130) to 8c1822e697d64b758a23b11c8fdeb59a (127.14.177.129). New cstate: current_term: 3 leader_uuid: "8c1822e697d64b758a23b11c8fdeb59a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c1822e697d64b758a23b11c8fdeb59a" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 39067 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7cd7c680ef2e474c981ac81abadecfde" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 40117 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "62bbd271280e4b0a9ddf9ab46151e455" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 33917 } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:32.924989 15820 raft_consensus.cc:3060] T 8cff7911abd649b9a276ddae431311d1 P 62bbd271280e4b0a9ddf9ab46151e455 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:32.926570 15746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cff7911abd649b9a276ddae431311d1" candidate_uuid: "8c1822e697d64b758a23b11c8fdeb59a" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "7cd7c680ef2e474c981ac81abadecfde"
I20260709 13:18:32.927285 15746 raft_consensus.cc:3060] T 8cff7911abd649b9a276ddae431311d1 P 7cd7c680ef2e474c981ac81abadecfde [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:32.930264 15820 raft_consensus.cc:2468] T 8cff7911abd649b9a276ddae431311d1 P 62bbd271280e4b0a9ddf9ab46151e455 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c1822e697d64b758a23b11c8fdeb59a in term 4.
I20260709 13:18:32.933182 15637 leader_election.cc:304] T 8cff7911abd649b9a276ddae431311d1 P 8c1822e697d64b758a23b11c8fdeb59a [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 62bbd271280e4b0a9ddf9ab46151e455, 8c1822e697d64b758a23b11c8fdeb59a; no voters: 
I20260709 13:18:32.933521 15746 raft_consensus.cc:2468] T 8cff7911abd649b9a276ddae431311d1 P 7cd7c680ef2e474c981ac81abadecfde [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c1822e697d64b758a23b11c8fdeb59a in term 4.
I20260709 13:18:32.933952 15961 raft_consensus.cc:2804] T 8cff7911abd649b9a276ddae431311d1 P 8c1822e697d64b758a23b11c8fdeb59a [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:18:32.934576 15961 raft_consensus.cc:697] T 8cff7911abd649b9a276ddae431311d1 P 8c1822e697d64b758a23b11c8fdeb59a [term 4 LEADER]: Becoming Leader. State: Replica: 8c1822e697d64b758a23b11c8fdeb59a, State: Running, Role: LEADER
I20260709 13:18:32.935318 15961 consensus_queue.cc:237] T 8cff7911abd649b9a276ddae431311d1 P 8c1822e697d64b758a23b11c8fdeb59a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c1822e697d64b758a23b11c8fdeb59a" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 39067 } } peers { permanent_uuid: "7cd7c680ef2e474c981ac81abadecfde" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 40117 } } peers { permanent_uuid: "62bbd271280e4b0a9ddf9ab46151e455" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 33917 } }
I20260709 13:18:32.941077 15566 catalog_manager.cc:5697] T 8cff7911abd649b9a276ddae431311d1 P 8c1822e697d64b758a23b11c8fdeb59a reported cstate change: term changed from 3 to 4, leader changed from 62bbd271280e4b0a9ddf9ab46151e455 (127.14.177.131) to 8c1822e697d64b758a23b11c8fdeb59a (127.14.177.129). New cstate: current_term: 4 leader_uuid: "8c1822e697d64b758a23b11c8fdeb59a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c1822e697d64b758a23b11c8fdeb59a" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 39067 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7cd7c680ef2e474c981ac81abadecfde" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 40117 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "62bbd271280e4b0a9ddf9ab46151e455" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 33917 } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:33.009572 15746 raft_consensus.cc:1275] T 17f97fe9a93f4a86b1e2bad621bfb561 P 7cd7c680ef2e474c981ac81abadecfde [term 3 FOLLOWER]: Refusing update from remote peer 8c1822e697d64b758a23b11c8fdeb59a: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:18:33.011059 15961 consensus_queue.cc:1048] T 17f97fe9a93f4a86b1e2bad621bfb561 P 8c1822e697d64b758a23b11c8fdeb59a [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cd7c680ef2e474c981ac81abadecfde" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 40117 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:18:33.020287 15820 raft_consensus.cc:1275] T 17f97fe9a93f4a86b1e2bad621bfb561 P 62bbd271280e4b0a9ddf9ab46151e455 [term 3 FOLLOWER]: Refusing update from remote peer 8c1822e697d64b758a23b11c8fdeb59a: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:18:33.021974 15961 consensus_queue.cc:1048] T 17f97fe9a93f4a86b1e2bad621bfb561 P 8c1822e697d64b758a23b11c8fdeb59a [LEADER]: Connected to new peer: Peer: permanent_uuid: "62bbd271280e4b0a9ddf9ab46151e455" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 33917 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:18:33.031512 15903 raft_consensus.cc:993] T 8309d9a5a4ef4e6d914751bc0f410015 P 7cd7c680ef2e474c981ac81abadecfde: : Instructing follower 8c1822e697d64b758a23b11c8fdeb59a to start an election
I20260709 13:18:33.032054 15977 raft_consensus.cc:1081] T 8309d9a5a4ef4e6d914751bc0f410015 P 7cd7c680ef2e474c981ac81abadecfde [term 2 LEADER]: Signalling peer 8c1822e697d64b758a23b11c8fdeb59a to start an election
I20260709 13:18:33.034147 15670 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "8309d9a5a4ef4e6d914751bc0f410015"
dest_uuid: "8c1822e697d64b758a23b11c8fdeb59a"
 from {username='slave'} at 127.0.0.1:42812
I20260709 13:18:33.034705 15670 raft_consensus.cc:493] T 8309d9a5a4ef4e6d914751bc0f410015 P 8c1822e697d64b758a23b11c8fdeb59a [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:18:33.035043 15670 raft_consensus.cc:3060] T 8309d9a5a4ef4e6d914751bc0f410015 P 8c1822e697d64b758a23b11c8fdeb59a [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:33.041729 15670 raft_consensus.cc:515] T 8309d9a5a4ef4e6d914751bc0f410015 P 8c1822e697d64b758a23b11c8fdeb59a [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c1822e697d64b758a23b11c8fdeb59a" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 39067 } } peers { permanent_uuid: "7cd7c680ef2e474c981ac81abadecfde" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 40117 } } peers { permanent_uuid: "62bbd271280e4b0a9ddf9ab46151e455" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 33917 } }
I20260709 13:18:33.043525 15670 leader_election.cc:290] T 8309d9a5a4ef4e6d914751bc0f410015 P 8c1822e697d64b758a23b11c8fdeb59a [CANDIDATE]: Term 3 election: Requested vote from peers 7cd7c680ef2e474c981ac81abadecfde (127.14.177.130:40117), 62bbd271280e4b0a9ddf9ab46151e455 (127.14.177.131:33917)
I20260709 13:18:33.044335 15746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8309d9a5a4ef4e6d914751bc0f410015" candidate_uuid: "8c1822e697d64b758a23b11c8fdeb59a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "7cd7c680ef2e474c981ac81abadecfde"
I20260709 13:18:33.044907 15746 raft_consensus.cc:3055] T 8309d9a5a4ef4e6d914751bc0f410015 P 7cd7c680ef2e474c981ac81abadecfde [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:18:33.044822 15820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8309d9a5a4ef4e6d914751bc0f410015" candidate_uuid: "8c1822e697d64b758a23b11c8fdeb59a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "62bbd271280e4b0a9ddf9ab46151e455"
I20260709 13:18:33.045303 15746 raft_consensus.cc:740] T 8309d9a5a4ef4e6d914751bc0f410015 P 7cd7c680ef2e474c981ac81abadecfde [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 7cd7c680ef2e474c981ac81abadecfde, State: Running, Role: LEADER
I20260709 13:18:33.045511 15820 raft_consensus.cc:3060] T 8309d9a5a4ef4e6d914751bc0f410015 P 62bbd271280e4b0a9ddf9ab46151e455 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:33.046141 15746 consensus_queue.cc:260] T 8309d9a5a4ef4e6d914751bc0f410015 P 7cd7c680ef2e474c981ac81abadecfde [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c1822e697d64b758a23b11c8fdeb59a" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 39067 } } peers { permanent_uuid: "7cd7c680ef2e474c981ac81abadecfde" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 40117 } } peers { permanent_uuid: "62bbd271280e4b0a9ddf9ab46151e455" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 33917 } }
I20260709 13:18:33.047338 15746 raft_consensus.cc:3060] T 8309d9a5a4ef4e6d914751bc0f410015 P 7cd7c680ef2e474c981ac81abadecfde [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:33.051812 15820 raft_consensus.cc:2468] T 8309d9a5a4ef4e6d914751bc0f410015 P 62bbd271280e4b0a9ddf9ab46151e455 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c1822e697d64b758a23b11c8fdeb59a in term 3.
I20260709 13:18:33.052933 15637 leader_election.cc:304] T 8309d9a5a4ef4e6d914751bc0f410015 P 8c1822e697d64b758a23b11c8fdeb59a [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 62bbd271280e4b0a9ddf9ab46151e455, 8c1822e697d64b758a23b11c8fdeb59a; no voters: 
I20260709 13:18:33.053722 15985 raft_consensus.cc:2804] T 8309d9a5a4ef4e6d914751bc0f410015 P 8c1822e697d64b758a23b11c8fdeb59a [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:18:33.053761 15746 raft_consensus.cc:2468] T 8309d9a5a4ef4e6d914751bc0f410015 P 7cd7c680ef2e474c981ac81abadecfde [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c1822e697d64b758a23b11c8fdeb59a in term 3.
I20260709 13:18:33.054322 15985 raft_consensus.cc:697] T 8309d9a5a4ef4e6d914751bc0f410015 P 8c1822e697d64b758a23b11c8fdeb59a [term 3 LEADER]: Becoming Leader. State: Replica: 8c1822e697d64b758a23b11c8fdeb59a, State: Running, Role: LEADER
I20260709 13:18:33.055127 15985 consensus_queue.cc:237] T 8309d9a5a4ef4e6d914751bc0f410015 P 8c1822e697d64b758a23b11c8fdeb59a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c1822e697d64b758a23b11c8fdeb59a" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 39067 } } peers { permanent_uuid: "7cd7c680ef2e474c981ac81abadecfde" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 40117 } } peers { permanent_uuid: "62bbd271280e4b0a9ddf9ab46151e455" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 33917 } }
I20260709 13:18:33.062110 15566 catalog_manager.cc:5697] T 8309d9a5a4ef4e6d914751bc0f410015 P 8c1822e697d64b758a23b11c8fdeb59a reported cstate change: term changed from 2 to 3, leader changed from 7cd7c680ef2e474c981ac81abadecfde (127.14.177.130) to 8c1822e697d64b758a23b11c8fdeb59a (127.14.177.129). New cstate: current_term: 3 leader_uuid: "8c1822e697d64b758a23b11c8fdeb59a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c1822e697d64b758a23b11c8fdeb59a" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 39067 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7cd7c680ef2e474c981ac81abadecfde" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 40117 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "62bbd271280e4b0a9ddf9ab46151e455" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 33917 } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:33.146621 15820 raft_consensus.cc:1275] T 566eae6d93af4ec492b9b1a270dc03b7 P 62bbd271280e4b0a9ddf9ab46151e455 [term 4 FOLLOWER]: Refusing update from remote peer 8c1822e697d64b758a23b11c8fdeb59a: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:18:33.147738 15985 consensus_queue.cc:1048] T 566eae6d93af4ec492b9b1a270dc03b7 P 8c1822e697d64b758a23b11c8fdeb59a [LEADER]: Connected to new peer: Peer: permanent_uuid: "62bbd271280e4b0a9ddf9ab46151e455" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 33917 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:18:33.155369 15746 raft_consensus.cc:1275] T 566eae6d93af4ec492b9b1a270dc03b7 P 7cd7c680ef2e474c981ac81abadecfde [term 4 FOLLOWER]: Refusing update from remote peer 8c1822e697d64b758a23b11c8fdeb59a: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:18:33.156883 15911 consensus_queue.cc:1048] T 566eae6d93af4ec492b9b1a270dc03b7 P 8c1822e697d64b758a23b11c8fdeb59a [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cd7c680ef2e474c981ac81abadecfde" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 40117 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:18:33.354305 15746 raft_consensus.cc:1275] T 408866bb47904e4189bc235ca5b0c7c7 P 7cd7c680ef2e474c981ac81abadecfde [term 3 FOLLOWER]: Refusing update from remote peer 8c1822e697d64b758a23b11c8fdeb59a: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:18:33.355599 15911 consensus_queue.cc:1048] T 408866bb47904e4189bc235ca5b0c7c7 P 8c1822e697d64b758a23b11c8fdeb59a [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cd7c680ef2e474c981ac81abadecfde" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 40117 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:18:33.363692 15820 raft_consensus.cc:1275] T 408866bb47904e4189bc235ca5b0c7c7 P 62bbd271280e4b0a9ddf9ab46151e455 [term 3 FOLLOWER]: Refusing update from remote peer 8c1822e697d64b758a23b11c8fdeb59a: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:18:33.365612 15911 consensus_queue.cc:1048] T 408866bb47904e4189bc235ca5b0c7c7 P 8c1822e697d64b758a23b11c8fdeb59a [LEADER]: Connected to new peer: Peer: permanent_uuid: "62bbd271280e4b0a9ddf9ab46151e455" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 33917 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:18:33.391220 15820 raft_consensus.cc:1275] T 8cff7911abd649b9a276ddae431311d1 P 62bbd271280e4b0a9ddf9ab46151e455 [term 4 FOLLOWER]: Refusing update from remote peer 8c1822e697d64b758a23b11c8fdeb59a: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:18:33.392756 15911 consensus_queue.cc:1048] T 8cff7911abd649b9a276ddae431311d1 P 8c1822e697d64b758a23b11c8fdeb59a [LEADER]: Connected to new peer: Peer: permanent_uuid: "62bbd271280e4b0a9ddf9ab46151e455" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 33917 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:18:33.401458 15746 raft_consensus.cc:1275] T 8cff7911abd649b9a276ddae431311d1 P 7cd7c680ef2e474c981ac81abadecfde [term 4 FOLLOWER]: Refusing update from remote peer 8c1822e697d64b758a23b11c8fdeb59a: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:18:33.403071 15961 consensus_queue.cc:1048] T 8cff7911abd649b9a276ddae431311d1 P 8c1822e697d64b758a23b11c8fdeb59a [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cd7c680ef2e474c981ac81abadecfde" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 40117 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:18:33.437373 15820 raft_consensus.cc:1275] T 8309d9a5a4ef4e6d914751bc0f410015 P 62bbd271280e4b0a9ddf9ab46151e455 [term 3 FOLLOWER]: Refusing update from remote peer 8c1822e697d64b758a23b11c8fdeb59a: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:18:33.438567 15961 consensus_queue.cc:1048] T 8309d9a5a4ef4e6d914751bc0f410015 P 8c1822e697d64b758a23b11c8fdeb59a [LEADER]: Connected to new peer: Peer: permanent_uuid: "62bbd271280e4b0a9ddf9ab46151e455" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 33917 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:18:33.447674 15746 raft_consensus.cc:1275] T 8309d9a5a4ef4e6d914751bc0f410015 P 7cd7c680ef2e474c981ac81abadecfde [term 3 FOLLOWER]: Refusing update from remote peer 8c1822e697d64b758a23b11c8fdeb59a: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:18:33.450031 15911 consensus_queue.cc:1048] T 8309d9a5a4ef4e6d914751bc0f410015 P 8c1822e697d64b758a23b11c8fdeb59a [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cd7c680ef2e474c981ac81abadecfde" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 40117 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:18:35.584523 15046 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:18:35.593351 15670 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0ce7d20d574f47649d3ad0f3c382198d"
dest_uuid: "8c1822e697d64b758a23b11c8fdeb59a"
mode: GRACEFUL
new_leader_uuid: "62bbd271280e4b0a9ddf9ab46151e455"
 from {username='slave'} at 127.0.0.1:56558
I20260709 13:18:35.593983 15670 raft_consensus.cc:606] T 0ce7d20d574f47649d3ad0f3c382198d P 8c1822e697d64b758a23b11c8fdeb59a [term 2 LEADER]: Received request to transfer leadership to TS 62bbd271280e4b0a9ddf9ab46151e455
I20260709 13:18:35.596455 15670 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4d83a05b849340c186cc995649575d3b"
dest_uuid: "8c1822e697d64b758a23b11c8fdeb59a"
mode: GRACEFUL
new_leader_uuid: "7cd7c680ef2e474c981ac81abadecfde"
 from {username='slave'} at 127.0.0.1:56558
I20260709 13:18:35.596927 15670 raft_consensus.cc:606] T 4d83a05b849340c186cc995649575d3b P 8c1822e697d64b758a23b11c8fdeb59a [term 2 LEADER]: Received request to transfer leadership to TS 7cd7c680ef2e474c981ac81abadecfde
I20260709 13:18:35.599144 15670 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8309d9a5a4ef4e6d914751bc0f410015"
dest_uuid: "8c1822e697d64b758a23b11c8fdeb59a"
mode: GRACEFUL
new_leader_uuid: "62bbd271280e4b0a9ddf9ab46151e455"
 from {username='slave'} at 127.0.0.1:56558
I20260709 13:18:35.599598 15670 raft_consensus.cc:606] T 8309d9a5a4ef4e6d914751bc0f410015 P 8c1822e697d64b758a23b11c8fdeb59a [term 3 LEADER]: Received request to transfer leadership to TS 62bbd271280e4b0a9ddf9ab46151e455
I20260709 13:18:35.601997 15670 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8fc26f7d12464423a212a1f595ca4b88"
dest_uuid: "8c1822e697d64b758a23b11c8fdeb59a"
mode: GRACEFUL
new_leader_uuid: "7cd7c680ef2e474c981ac81abadecfde"
 from {username='slave'} at 127.0.0.1:56558
I20260709 13:18:35.602408 15670 raft_consensus.cc:606] T 8fc26f7d12464423a212a1f595ca4b88 P 8c1822e697d64b758a23b11c8fdeb59a [term 2 LEADER]: Received request to transfer leadership to TS 7cd7c680ef2e474c981ac81abadecfde
I20260709 13:18:35.603399 15046 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 4
I20260709 13:18:35.603858 15046 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:18:35.838418 15961 raft_consensus.cc:993] T 0ce7d20d574f47649d3ad0f3c382198d P 8c1822e697d64b758a23b11c8fdeb59a: : Instructing follower 62bbd271280e4b0a9ddf9ab46151e455 to start an election
I20260709 13:18:35.839067 15998 raft_consensus.cc:1081] T 0ce7d20d574f47649d3ad0f3c382198d P 8c1822e697d64b758a23b11c8fdeb59a [term 2 LEADER]: Signalling peer 62bbd271280e4b0a9ddf9ab46151e455 to start an election
I20260709 13:18:35.840899 15819 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "0ce7d20d574f47649d3ad0f3c382198d"
dest_uuid: "62bbd271280e4b0a9ddf9ab46151e455"
 from {username='slave'} at 127.0.0.1:40898
I20260709 13:18:35.841465 15819 raft_consensus.cc:493] T 0ce7d20d574f47649d3ad0f3c382198d P 62bbd271280e4b0a9ddf9ab46151e455 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:18:35.841841 15819 raft_consensus.cc:3060] T 0ce7d20d574f47649d3ad0f3c382198d P 62bbd271280e4b0a9ddf9ab46151e455 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:35.847474 15819 raft_consensus.cc:515] T 0ce7d20d574f47649d3ad0f3c382198d P 62bbd271280e4b0a9ddf9ab46151e455 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c1822e697d64b758a23b11c8fdeb59a" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 39067 } } peers { permanent_uuid: "7cd7c680ef2e474c981ac81abadecfde" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 40117 } } peers { permanent_uuid: "62bbd271280e4b0a9ddf9ab46151e455" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 33917 } }
I20260709 13:18:35.849248 15819 leader_election.cc:290] T 0ce7d20d574f47649d3ad0f3c382198d P 62bbd271280e4b0a9ddf9ab46151e455 [CANDIDATE]: Term 3 election: Requested vote from peers 8c1822e697d64b758a23b11c8fdeb59a (127.14.177.129:39067), 7cd7c680ef2e474c981ac81abadecfde (127.14.177.130:40117)
I20260709 13:18:35.849957 15670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ce7d20d574f47649d3ad0f3c382198d" candidate_uuid: "62bbd271280e4b0a9ddf9ab46151e455" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "8c1822e697d64b758a23b11c8fdeb59a"
I20260709 13:18:35.850679 15670 raft_consensus.cc:3055] T 0ce7d20d574f47649d3ad0f3c382198d P 8c1822e697d64b758a23b11c8fdeb59a [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:18:35.850584 15746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ce7d20d574f47649d3ad0f3c382198d" candidate_uuid: "62bbd271280e4b0a9ddf9ab46151e455" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "7cd7c680ef2e474c981ac81abadecfde"
I20260709 13:18:35.850986 15670 raft_consensus.cc:740] T 0ce7d20d574f47649d3ad0f3c382198d P 8c1822e697d64b758a23b11c8fdeb59a [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 8c1822e697d64b758a23b11c8fdeb59a, State: Running, Role: LEADER
I20260709 13:18:35.851280 15746 raft_consensus.cc:3060] T 0ce7d20d574f47649d3ad0f3c382198d P 7cd7c680ef2e474c981ac81abadecfde [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:35.851890 15670 consensus_queue.cc:260] T 0ce7d20d574f47649d3ad0f3c382198d P 8c1822e697d64b758a23b11c8fdeb59a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c1822e697d64b758a23b11c8fdeb59a" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 39067 } } peers { permanent_uuid: "7cd7c680ef2e474c981ac81abadecfde" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 40117 } } peers { permanent_uuid: "62bbd271280e4b0a9ddf9ab46151e455" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 33917 } }
I20260709 13:18:35.852859 15670 raft_consensus.cc:3060] T 0ce7d20d574f47649d3ad0f3c382198d P 8c1822e697d64b758a23b11c8fdeb59a [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:35.856213 15746 raft_consensus.cc:2468] T 0ce7d20d574f47649d3ad0f3c382198d P 7cd7c680ef2e474c981ac81abadecfde [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 62bbd271280e4b0a9ddf9ab46151e455 in term 3.
I20260709 13:18:35.857420 15670 raft_consensus.cc:2468] T 0ce7d20d574f47649d3ad0f3c382198d P 8c1822e697d64b758a23b11c8fdeb59a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 62bbd271280e4b0a9ddf9ab46151e455 in term 3.
I20260709 13:18:35.857437 15785 leader_election.cc:304] T 0ce7d20d574f47649d3ad0f3c382198d P 62bbd271280e4b0a9ddf9ab46151e455 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 62bbd271280e4b0a9ddf9ab46151e455, 7cd7c680ef2e474c981ac81abadecfde; no voters: 
I20260709 13:18:35.858564 15866 raft_consensus.cc:2804] T 0ce7d20d574f47649d3ad0f3c382198d P 62bbd271280e4b0a9ddf9ab46151e455 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:18:35.859586 15866 raft_consensus.cc:697] T 0ce7d20d574f47649d3ad0f3c382198d P 62bbd271280e4b0a9ddf9ab46151e455 [term 3 LEADER]: Becoming Leader. State: Replica: 62bbd271280e4b0a9ddf9ab46151e455, State: Running, Role: LEADER
I20260709 13:18:35.860402 15866 consensus_queue.cc:237] T 0ce7d20d574f47649d3ad0f3c382198d P 62bbd271280e4b0a9ddf9ab46151e455 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c1822e697d64b758a23b11c8fdeb59a" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 39067 } } peers { permanent_uuid: "7cd7c680ef2e474c981ac81abadecfde" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 40117 } } peers { permanent_uuid: "62bbd271280e4b0a9ddf9ab46151e455" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 33917 } }
I20260709 13:18:35.867749 15567 catalog_manager.cc:5697] T 0ce7d20d574f47649d3ad0f3c382198d P 62bbd271280e4b0a9ddf9ab46151e455 reported cstate change: term changed from 2 to 3, leader changed from 8c1822e697d64b758a23b11c8fdeb59a (127.14.177.129) to 62bbd271280e4b0a9ddf9ab46151e455 (127.14.177.131). New cstate: current_term: 3 leader_uuid: "62bbd271280e4b0a9ddf9ab46151e455" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c1822e697d64b758a23b11c8fdeb59a" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 39067 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7cd7c680ef2e474c981ac81abadecfde" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 40117 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "62bbd271280e4b0a9ddf9ab46151e455" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 33917 } health_report { overall_health: HEALTHY } } }
I20260709 13:18:35.869257 15961 raft_consensus.cc:993] T 8fc26f7d12464423a212a1f595ca4b88 P 8c1822e697d64b758a23b11c8fdeb59a: : Instructing follower 7cd7c680ef2e474c981ac81abadecfde to start an election
I20260709 13:18:35.869750 15998 raft_consensus.cc:1081] T 8fc26f7d12464423a212a1f595ca4b88 P 8c1822e697d64b758a23b11c8fdeb59a [term 2 LEADER]: Signalling peer 7cd7c680ef2e474c981ac81abadecfde to start an election
I20260709 13:18:35.871584 15746 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "8fc26f7d12464423a212a1f595ca4b88"
dest_uuid: "7cd7c680ef2e474c981ac81abadecfde"
 from {username='slave'} at 127.0.0.1:49440
I20260709 13:18:35.872221 15746 raft_consensus.cc:493] T 8fc26f7d12464423a212a1f595ca4b88 P 7cd7c680ef2e474c981ac81abadecfde [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:18:35.872548 15746 raft_consensus.cc:3060] T 8fc26f7d12464423a212a1f595ca4b88 P 7cd7c680ef2e474c981ac81abadecfde [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:35.878873 15746 raft_consensus.cc:515] T 8fc26f7d12464423a212a1f595ca4b88 P 7cd7c680ef2e474c981ac81abadecfde [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c1822e697d64b758a23b11c8fdeb59a" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 39067 } } peers { permanent_uuid: "7cd7c680ef2e474c981ac81abadecfde" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 40117 } } peers { permanent_uuid: "62bbd271280e4b0a9ddf9ab46151e455" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 33917 } }
I20260709 13:18:35.881672 15670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc26f7d12464423a212a1f595ca4b88" candidate_uuid: "7cd7c680ef2e474c981ac81abadecfde" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "8c1822e697d64b758a23b11c8fdeb59a"
I20260709 13:18:35.882213 15670 raft_consensus.cc:3055] T 8fc26f7d12464423a212a1f595ca4b88 P 8c1822e697d64b758a23b11c8fdeb59a [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:18:35.882441 15670 raft_consensus.cc:740] T 8fc26f7d12464423a212a1f595ca4b88 P 8c1822e697d64b758a23b11c8fdeb59a [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 8c1822e697d64b758a23b11c8fdeb59a, State: Running, Role: LEADER
I20260709 13:18:35.882328 15819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fc26f7d12464423a212a1f595ca4b88" candidate_uuid: "7cd7c680ef2e474c981ac81abadecfde" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "62bbd271280e4b0a9ddf9ab46151e455"
I20260709 13:18:35.882982 15819 raft_consensus.cc:3060] T 8fc26f7d12464423a212a1f595ca4b88 P 62bbd271280e4b0a9ddf9ab46151e455 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:35.883116 15670 consensus_queue.cc:260] T 8fc26f7d12464423a212a1f595ca4b88 P 8c1822e697d64b758a23b11c8fdeb59a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c1822e697d64b758a23b11c8fdeb59a" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 39067 } } peers { permanent_uuid: "7cd7c680ef2e474c981ac81abadecfde" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 40117 } } peers { permanent_uuid: "62bbd271280e4b0a9ddf9ab46151e455" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 33917 } }
I20260709 13:18:35.884200 15670 raft_consensus.cc:3060] T 8fc26f7d12464423a212a1f595ca4b88 P 8c1822e697d64b758a23b11c8fdeb59a [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:35.887387 15746 leader_election.cc:290] T 8fc26f7d12464423a212a1f595ca4b88 P 7cd7c680ef2e474c981ac81abadecfde [CANDIDATE]: Term 3 election: Requested vote from peers 8c1822e697d64b758a23b11c8fdeb59a (127.14.177.129:39067), 62bbd271280e4b0a9ddf9ab46151e455 (127.14.177.131:33917)
I20260709 13:18:35.887920 15819 raft_consensus.cc:2468] T 8fc26f7d12464423a212a1f595ca4b88 P 62bbd271280e4b0a9ddf9ab46151e455 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7cd7c680ef2e474c981ac81abadecfde in term 3.
I20260709 13:18:35.888331 15670 raft_consensus.cc:2468] T 8fc26f7d12464423a212a1f595ca4b88 P 8c1822e697d64b758a23b11c8fdeb59a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7cd7c680ef2e474c981ac81abadecfde in term 3.
I20260709 13:18:35.889271 15712 leader_election.cc:304] T 8fc26f7d12464423a212a1f595ca4b88 P 7cd7c680ef2e474c981ac81abadecfde [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 62bbd271280e4b0a9ddf9ab46151e455, 7cd7c680ef2e474c981ac81abadecfde; no voters: 
I20260709 13:18:35.890134 15903 raft_consensus.cc:2804] T 8fc26f7d12464423a212a1f595ca4b88 P 7cd7c680ef2e474c981ac81abadecfde [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:18:35.891264 15903 raft_consensus.cc:697] T 8fc26f7d12464423a212a1f595ca4b88 P 7cd7c680ef2e474c981ac81abadecfde [term 3 LEADER]: Becoming Leader. State: Replica: 7cd7c680ef2e474c981ac81abadecfde, State: Running, Role: LEADER
I20260709 13:18:35.892124 15903 consensus_queue.cc:237] T 8fc26f7d12464423a212a1f595ca4b88 P 7cd7c680ef2e474c981ac81abadecfde [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c1822e697d64b758a23b11c8fdeb59a" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 39067 } } peers { permanent_uuid: "7cd7c680ef2e474c981ac81abadecfde" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 40117 } } peers { permanent_uuid: "62bbd271280e4b0a9ddf9ab46151e455" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 33917 } }
I20260709 13:18:35.898947 15567 catalog_manager.cc:5697] T 8fc26f7d12464423a212a1f595ca4b88 P 7cd7c680ef2e474c981ac81abadecfde reported cstate change: term changed from 2 to 3, leader changed from 8c1822e697d64b758a23b11c8fdeb59a (127.14.177.129) to 7cd7c680ef2e474c981ac81abadecfde (127.14.177.130). New cstate: current_term: 3 leader_uuid: "7cd7c680ef2e474c981ac81abadecfde" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c1822e697d64b758a23b11c8fdeb59a" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 39067 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7cd7c680ef2e474c981ac81abadecfde" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 40117 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "62bbd271280e4b0a9ddf9ab46151e455" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 33917 } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:36.029946 15961 raft_consensus.cc:993] T 4d83a05b849340c186cc995649575d3b P 8c1822e697d64b758a23b11c8fdeb59a: : Instructing follower 7cd7c680ef2e474c981ac81abadecfde to start an election
I20260709 13:18:36.030345 15998 raft_consensus.cc:1081] T 4d83a05b849340c186cc995649575d3b P 8c1822e697d64b758a23b11c8fdeb59a [term 2 LEADER]: Signalling peer 7cd7c680ef2e474c981ac81abadecfde to start an election
I20260709 13:18:36.031699 15746 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "4d83a05b849340c186cc995649575d3b"
dest_uuid: "7cd7c680ef2e474c981ac81abadecfde"
 from {username='slave'} at 127.0.0.1:49440
I20260709 13:18:36.032168 15746 raft_consensus.cc:493] T 4d83a05b849340c186cc995649575d3b P 7cd7c680ef2e474c981ac81abadecfde [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:18:36.032493 15746 raft_consensus.cc:3060] T 4d83a05b849340c186cc995649575d3b P 7cd7c680ef2e474c981ac81abadecfde [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:36.036769 15746 raft_consensus.cc:515] T 4d83a05b849340c186cc995649575d3b P 7cd7c680ef2e474c981ac81abadecfde [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62bbd271280e4b0a9ddf9ab46151e455" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 33917 } } peers { permanent_uuid: "7cd7c680ef2e474c981ac81abadecfde" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 40117 } } peers { permanent_uuid: "8c1822e697d64b758a23b11c8fdeb59a" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 39067 } }
I20260709 13:18:36.038352 15746 leader_election.cc:290] T 4d83a05b849340c186cc995649575d3b P 7cd7c680ef2e474c981ac81abadecfde [CANDIDATE]: Term 3 election: Requested vote from peers 62bbd271280e4b0a9ddf9ab46151e455 (127.14.177.131:33917), 8c1822e697d64b758a23b11c8fdeb59a (127.14.177.129:39067)
I20260709 13:18:36.039391 15819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d83a05b849340c186cc995649575d3b" candidate_uuid: "7cd7c680ef2e474c981ac81abadecfde" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "62bbd271280e4b0a9ddf9ab46151e455"
I20260709 13:18:36.039391 15670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d83a05b849340c186cc995649575d3b" candidate_uuid: "7cd7c680ef2e474c981ac81abadecfde" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "8c1822e697d64b758a23b11c8fdeb59a"
I20260709 13:18:36.040150 15670 raft_consensus.cc:3055] T 4d83a05b849340c186cc995649575d3b P 8c1822e697d64b758a23b11c8fdeb59a [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:18:36.040139 15819 raft_consensus.cc:3060] T 4d83a05b849340c186cc995649575d3b P 62bbd271280e4b0a9ddf9ab46151e455 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:36.040586 15670 raft_consensus.cc:740] T 4d83a05b849340c186cc995649575d3b P 8c1822e697d64b758a23b11c8fdeb59a [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 8c1822e697d64b758a23b11c8fdeb59a, State: Running, Role: LEADER
I20260709 13:18:36.041414 15670 consensus_queue.cc:260] T 4d83a05b849340c186cc995649575d3b P 8c1822e697d64b758a23b11c8fdeb59a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62bbd271280e4b0a9ddf9ab46151e455" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 33917 } } peers { permanent_uuid: "7cd7c680ef2e474c981ac81abadecfde" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 40117 } } peers { permanent_uuid: "8c1822e697d64b758a23b11c8fdeb59a" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 39067 } }
I20260709 13:18:36.042379 15670 raft_consensus.cc:3060] T 4d83a05b849340c186cc995649575d3b P 8c1822e697d64b758a23b11c8fdeb59a [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:18:36.042474 15636 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:36.046322 15819 raft_consensus.cc:2468] T 4d83a05b849340c186cc995649575d3b P 62bbd271280e4b0a9ddf9ab46151e455 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7cd7c680ef2e474c981ac81abadecfde in term 3.
I20260709 13:18:36.047395 15712 leader_election.cc:304] T 4d83a05b849340c186cc995649575d3b P 7cd7c680ef2e474c981ac81abadecfde [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 62bbd271280e4b0a9ddf9ab46151e455, 7cd7c680ef2e474c981ac81abadecfde; no voters: 
I20260709 13:18:36.048019 15670 raft_consensus.cc:2468] T 4d83a05b849340c186cc995649575d3b P 8c1822e697d64b758a23b11c8fdeb59a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7cd7c680ef2e474c981ac81abadecfde in term 3.
I20260709 13:18:36.048158 15903 raft_consensus.cc:2804] T 4d83a05b849340c186cc995649575d3b P 7cd7c680ef2e474c981ac81abadecfde [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:18:36.048620 15903 raft_consensus.cc:697] T 4d83a05b849340c186cc995649575d3b P 7cd7c680ef2e474c981ac81abadecfde [term 3 LEADER]: Becoming Leader. State: Replica: 7cd7c680ef2e474c981ac81abadecfde, State: Running, Role: LEADER
I20260709 13:18:36.049409 15903 consensus_queue.cc:237] T 4d83a05b849340c186cc995649575d3b P 7cd7c680ef2e474c981ac81abadecfde [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62bbd271280e4b0a9ddf9ab46151e455" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 33917 } } peers { permanent_uuid: "7cd7c680ef2e474c981ac81abadecfde" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 40117 } } peers { permanent_uuid: "8c1822e697d64b758a23b11c8fdeb59a" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 39067 } }
I20260709 13:18:36.055840 15567 catalog_manager.cc:5697] T 4d83a05b849340c186cc995649575d3b P 7cd7c680ef2e474c981ac81abadecfde reported cstate change: term changed from 2 to 3, leader changed from 8c1822e697d64b758a23b11c8fdeb59a (127.14.177.129) to 7cd7c680ef2e474c981ac81abadecfde (127.14.177.130). New cstate: current_term: 3 leader_uuid: "7cd7c680ef2e474c981ac81abadecfde" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62bbd271280e4b0a9ddf9ab46151e455" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 33917 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7cd7c680ef2e474c981ac81abadecfde" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 40117 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8c1822e697d64b758a23b11c8fdeb59a" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 39067 } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:36.165310 15961 raft_consensus.cc:993] T 8309d9a5a4ef4e6d914751bc0f410015 P 8c1822e697d64b758a23b11c8fdeb59a: : Instructing follower 62bbd271280e4b0a9ddf9ab46151e455 to start an election
I20260709 13:18:36.165736 15961 raft_consensus.cc:1081] T 8309d9a5a4ef4e6d914751bc0f410015 P 8c1822e697d64b758a23b11c8fdeb59a [term 3 LEADER]: Signalling peer 62bbd271280e4b0a9ddf9ab46151e455 to start an election
I20260709 13:18:36.166981 15819 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "8309d9a5a4ef4e6d914751bc0f410015"
dest_uuid: "62bbd271280e4b0a9ddf9ab46151e455"
 from {username='slave'} at 127.0.0.1:40898
I20260709 13:18:36.167431 15819 raft_consensus.cc:493] T 8309d9a5a4ef4e6d914751bc0f410015 P 62bbd271280e4b0a9ddf9ab46151e455 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:18:36.167701 15819 raft_consensus.cc:3060] T 8309d9a5a4ef4e6d914751bc0f410015 P 62bbd271280e4b0a9ddf9ab46151e455 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:36.171533 15819 raft_consensus.cc:515] T 8309d9a5a4ef4e6d914751bc0f410015 P 62bbd271280e4b0a9ddf9ab46151e455 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c1822e697d64b758a23b11c8fdeb59a" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 39067 } } peers { permanent_uuid: "7cd7c680ef2e474c981ac81abadecfde" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 40117 } } peers { permanent_uuid: "62bbd271280e4b0a9ddf9ab46151e455" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 33917 } }
I20260709 13:18:36.172859 15819 leader_election.cc:290] T 8309d9a5a4ef4e6d914751bc0f410015 P 62bbd271280e4b0a9ddf9ab46151e455 [CANDIDATE]: Term 4 election: Requested vote from peers 8c1822e697d64b758a23b11c8fdeb59a (127.14.177.129:39067), 7cd7c680ef2e474c981ac81abadecfde (127.14.177.130:40117)
I20260709 13:18:36.173609 15670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8309d9a5a4ef4e6d914751bc0f410015" candidate_uuid: "62bbd271280e4b0a9ddf9ab46151e455" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "8c1822e697d64b758a23b11c8fdeb59a"
I20260709 13:18:36.173913 15746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8309d9a5a4ef4e6d914751bc0f410015" candidate_uuid: "62bbd271280e4b0a9ddf9ab46151e455" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "7cd7c680ef2e474c981ac81abadecfde"
I20260709 13:18:36.174139 15670 raft_consensus.cc:3055] T 8309d9a5a4ef4e6d914751bc0f410015 P 8c1822e697d64b758a23b11c8fdeb59a [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:18:36.174427 15670 raft_consensus.cc:740] T 8309d9a5a4ef4e6d914751bc0f410015 P 8c1822e697d64b758a23b11c8fdeb59a [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 8c1822e697d64b758a23b11c8fdeb59a, State: Running, Role: LEADER
I20260709 13:18:36.174433 15746 raft_consensus.cc:3060] T 8309d9a5a4ef4e6d914751bc0f410015 P 7cd7c680ef2e474c981ac81abadecfde [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:36.175316 15670 consensus_queue.cc:260] T 8309d9a5a4ef4e6d914751bc0f410015 P 8c1822e697d64b758a23b11c8fdeb59a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c1822e697d64b758a23b11c8fdeb59a" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 39067 } } peers { permanent_uuid: "7cd7c680ef2e474c981ac81abadecfde" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 40117 } } peers { permanent_uuid: "62bbd271280e4b0a9ddf9ab46151e455" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 33917 } }
I20260709 13:18:36.176414 15670 raft_consensus.cc:3060] T 8309d9a5a4ef4e6d914751bc0f410015 P 8c1822e697d64b758a23b11c8fdeb59a [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:36.180516 15746 raft_consensus.cc:2468] T 8309d9a5a4ef4e6d914751bc0f410015 P 7cd7c680ef2e474c981ac81abadecfde [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 62bbd271280e4b0a9ddf9ab46151e455 in term 4.
I20260709 13:18:36.181481 15785 leader_election.cc:304] T 8309d9a5a4ef4e6d914751bc0f410015 P 62bbd271280e4b0a9ddf9ab46151e455 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 62bbd271280e4b0a9ddf9ab46151e455, 7cd7c680ef2e474c981ac81abadecfde; no voters: 
I20260709 13:18:36.182045 15670 raft_consensus.cc:2468] T 8309d9a5a4ef4e6d914751bc0f410015 P 8c1822e697d64b758a23b11c8fdeb59a [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 62bbd271280e4b0a9ddf9ab46151e455 in term 4.
I20260709 13:18:36.182237 15866 raft_consensus.cc:2804] T 8309d9a5a4ef4e6d914751bc0f410015 P 62bbd271280e4b0a9ddf9ab46151e455 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:18:36.182739 15866 raft_consensus.cc:697] T 8309d9a5a4ef4e6d914751bc0f410015 P 62bbd271280e4b0a9ddf9ab46151e455 [term 4 LEADER]: Becoming Leader. State: Replica: 62bbd271280e4b0a9ddf9ab46151e455, State: Running, Role: LEADER
I20260709 13:18:36.183677 15866 consensus_queue.cc:237] T 8309d9a5a4ef4e6d914751bc0f410015 P 62bbd271280e4b0a9ddf9ab46151e455 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c1822e697d64b758a23b11c8fdeb59a" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 39067 } } peers { permanent_uuid: "7cd7c680ef2e474c981ac81abadecfde" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 40117 } } peers { permanent_uuid: "62bbd271280e4b0a9ddf9ab46151e455" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 33917 } }
I20260709 13:18:36.189965 15567 catalog_manager.cc:5697] T 8309d9a5a4ef4e6d914751bc0f410015 P 62bbd271280e4b0a9ddf9ab46151e455 reported cstate change: term changed from 3 to 4, leader changed from 8c1822e697d64b758a23b11c8fdeb59a (127.14.177.129) to 62bbd271280e4b0a9ddf9ab46151e455 (127.14.177.131). New cstate: current_term: 4 leader_uuid: "62bbd271280e4b0a9ddf9ab46151e455" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c1822e697d64b758a23b11c8fdeb59a" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 39067 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7cd7c680ef2e474c981ac81abadecfde" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 40117 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "62bbd271280e4b0a9ddf9ab46151e455" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 33917 } health_report { overall_health: HEALTHY } } }
I20260709 13:18:36.253441 15746 raft_consensus.cc:1275] T 0ce7d20d574f47649d3ad0f3c382198d P 7cd7c680ef2e474c981ac81abadecfde [term 3 FOLLOWER]: Refusing update from remote peer 62bbd271280e4b0a9ddf9ab46151e455: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:18:36.254551 15866 consensus_queue.cc:1048] T 0ce7d20d574f47649d3ad0f3c382198d P 62bbd271280e4b0a9ddf9ab46151e455 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cd7c680ef2e474c981ac81abadecfde" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 40117 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:18:36.263926 15670 raft_consensus.cc:1275] T 0ce7d20d574f47649d3ad0f3c382198d P 8c1822e697d64b758a23b11c8fdeb59a [term 3 FOLLOWER]: Refusing update from remote peer 62bbd271280e4b0a9ddf9ab46151e455: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:18:36.266109 15999 consensus_queue.cc:1048] T 0ce7d20d574f47649d3ad0f3c382198d P 62bbd271280e4b0a9ddf9ab46151e455 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c1822e697d64b758a23b11c8fdeb59a" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 39067 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:18:36.363511 15819 raft_consensus.cc:1275] T 8fc26f7d12464423a212a1f595ca4b88 P 62bbd271280e4b0a9ddf9ab46151e455 [term 3 FOLLOWER]: Refusing update from remote peer 7cd7c680ef2e474c981ac81abadecfde: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:18:36.364835 15903 consensus_queue.cc:1048] T 8fc26f7d12464423a212a1f595ca4b88 P 7cd7c680ef2e474c981ac81abadecfde [LEADER]: Connected to new peer: Peer: permanent_uuid: "62bbd271280e4b0a9ddf9ab46151e455" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 33917 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:18:36.374089 15670 raft_consensus.cc:1275] T 8fc26f7d12464423a212a1f595ca4b88 P 8c1822e697d64b758a23b11c8fdeb59a [term 3 FOLLOWER]: Refusing update from remote peer 7cd7c680ef2e474c981ac81abadecfde: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:18:36.376598 15903 consensus_queue.cc:1048] T 8fc26f7d12464423a212a1f595ca4b88 P 7cd7c680ef2e474c981ac81abadecfde [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c1822e697d64b758a23b11c8fdeb59a" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 39067 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:18:36.499155 15819 raft_consensus.cc:1275] T 4d83a05b849340c186cc995649575d3b P 62bbd271280e4b0a9ddf9ab46151e455 [term 3 FOLLOWER]: Refusing update from remote peer 7cd7c680ef2e474c981ac81abadecfde: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:18:36.500339 15903 consensus_queue.cc:1048] T 4d83a05b849340c186cc995649575d3b P 7cd7c680ef2e474c981ac81abadecfde [LEADER]: Connected to new peer: Peer: permanent_uuid: "62bbd271280e4b0a9ddf9ab46151e455" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 33917 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:18:36.510505 15670 raft_consensus.cc:1275] T 4d83a05b849340c186cc995649575d3b P 8c1822e697d64b758a23b11c8fdeb59a [term 3 FOLLOWER]: Refusing update from remote peer 7cd7c680ef2e474c981ac81abadecfde: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:18:36.512039 16005 consensus_queue.cc:1048] T 4d83a05b849340c186cc995649575d3b P 7cd7c680ef2e474c981ac81abadecfde [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c1822e697d64b758a23b11c8fdeb59a" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 39067 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:18:36.589387 15744 raft_consensus.cc:1275] T 8309d9a5a4ef4e6d914751bc0f410015 P 7cd7c680ef2e474c981ac81abadecfde [term 4 FOLLOWER]: Refusing update from remote peer 62bbd271280e4b0a9ddf9ab46151e455: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:18:36.590776 15866 consensus_queue.cc:1048] T 8309d9a5a4ef4e6d914751bc0f410015 P 62bbd271280e4b0a9ddf9ab46151e455 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cd7c680ef2e474c981ac81abadecfde" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 40117 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:18:36.599728 15670 raft_consensus.cc:1275] T 8309d9a5a4ef4e6d914751bc0f410015 P 8c1822e697d64b758a23b11c8fdeb59a [term 4 FOLLOWER]: Refusing update from remote peer 62bbd271280e4b0a9ddf9ab46151e455: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:18:36.602537 16011 consensus_queue.cc:1048] T 8309d9a5a4ef4e6d914751bc0f410015 P 62bbd271280e4b0a9ddf9ab46151e455 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c1822e697d64b758a23b11c8fdeb59a" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 39067 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:18:36.604578 15046 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:18:36.612752 15046 tablet_server.cc:179] TabletServer@127.14.177.129:0 shutting down...
I20260709 13:18:36.639240 15046 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:18:36.639889 15046 tablet_replica.cc:333] T 48e8b6e82d7b42e6bdd4b0a8a25dfd9e P 8c1822e697d64b758a23b11c8fdeb59a: stopping tablet replica
I20260709 13:18:36.640429 15046 raft_consensus.cc:2243] T 48e8b6e82d7b42e6bdd4b0a8a25dfd9e P 8c1822e697d64b758a23b11c8fdeb59a [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:36.640857 15046 raft_consensus.cc:2272] T 48e8b6e82d7b42e6bdd4b0a8a25dfd9e P 8c1822e697d64b758a23b11c8fdeb59a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:36.642715 15046 tablet_replica.cc:333] T 17f97fe9a93f4a86b1e2bad621bfb561 P 8c1822e697d64b758a23b11c8fdeb59a: stopping tablet replica
I20260709 13:18:36.643343 15046 raft_consensus.cc:2243] T 17f97fe9a93f4a86b1e2bad621bfb561 P 8c1822e697d64b758a23b11c8fdeb59a [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:18:36.644095 15046 raft_consensus.cc:2272] T 17f97fe9a93f4a86b1e2bad621bfb561 P 8c1822e697d64b758a23b11c8fdeb59a [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:36.646461 15046 tablet_replica.cc:333] T 566eae6d93af4ec492b9b1a270dc03b7 P 8c1822e697d64b758a23b11c8fdeb59a: stopping tablet replica
I20260709 13:18:36.647046 15046 raft_consensus.cc:2243] T 566eae6d93af4ec492b9b1a270dc03b7 P 8c1822e697d64b758a23b11c8fdeb59a [term 4 LEADER]: Raft consensus shutting down.
I20260709 13:18:36.647786 15046 raft_consensus.cc:2272] T 566eae6d93af4ec492b9b1a270dc03b7 P 8c1822e697d64b758a23b11c8fdeb59a [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:36.649466 15046 tablet_replica.cc:333] T 502d133e880b4a859d8674b9d0cb6931 P 8c1822e697d64b758a23b11c8fdeb59a: stopping tablet replica
I20260709 13:18:36.649952 15046 raft_consensus.cc:2243] T 502d133e880b4a859d8674b9d0cb6931 P 8c1822e697d64b758a23b11c8fdeb59a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:36.650350 15046 raft_consensus.cc:2272] T 502d133e880b4a859d8674b9d0cb6931 P 8c1822e697d64b758a23b11c8fdeb59a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:36.652099 15046 tablet_replica.cc:333] T 8cff7911abd649b9a276ddae431311d1 P 8c1822e697d64b758a23b11c8fdeb59a: stopping tablet replica
I20260709 13:18:36.652570 15046 raft_consensus.cc:2243] T 8cff7911abd649b9a276ddae431311d1 P 8c1822e697d64b758a23b11c8fdeb59a [term 4 LEADER]: Raft consensus shutting down.
I20260709 13:18:36.653237 15046 raft_consensus.cc:2272] T 8cff7911abd649b9a276ddae431311d1 P 8c1822e697d64b758a23b11c8fdeb59a [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:36.654778 15046 tablet_replica.cc:333] T 8fc26f7d12464423a212a1f595ca4b88 P 8c1822e697d64b758a23b11c8fdeb59a: stopping tablet replica
I20260709 13:18:36.655309 15046 raft_consensus.cc:2243] T 8fc26f7d12464423a212a1f595ca4b88 P 8c1822e697d64b758a23b11c8fdeb59a [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:36.655690 15046 raft_consensus.cc:2272] T 8fc26f7d12464423a212a1f595ca4b88 P 8c1822e697d64b758a23b11c8fdeb59a [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:36.657696 15046 tablet_replica.cc:333] T 4d83a05b849340c186cc995649575d3b P 8c1822e697d64b758a23b11c8fdeb59a: stopping tablet replica
I20260709 13:18:36.658159 15046 raft_consensus.cc:2243] T 4d83a05b849340c186cc995649575d3b P 8c1822e697d64b758a23b11c8fdeb59a [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:36.658533 15046 raft_consensus.cc:2272] T 4d83a05b849340c186cc995649575d3b P 8c1822e697d64b758a23b11c8fdeb59a [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:36.660624 15046 tablet_replica.cc:333] T 8309d9a5a4ef4e6d914751bc0f410015 P 8c1822e697d64b758a23b11c8fdeb59a: stopping tablet replica
I20260709 13:18:36.661065 15046 raft_consensus.cc:2243] T 8309d9a5a4ef4e6d914751bc0f410015 P 8c1822e697d64b758a23b11c8fdeb59a [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:36.661437 15046 raft_consensus.cc:2272] T 8309d9a5a4ef4e6d914751bc0f410015 P 8c1822e697d64b758a23b11c8fdeb59a [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:36.663388 15046 tablet_replica.cc:333] T 0ce7d20d574f47649d3ad0f3c382198d P 8c1822e697d64b758a23b11c8fdeb59a: stopping tablet replica
I20260709 13:18:36.663825 15046 raft_consensus.cc:2243] T 0ce7d20d574f47649d3ad0f3c382198d P 8c1822e697d64b758a23b11c8fdeb59a [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:36.664215 15046 raft_consensus.cc:2272] T 0ce7d20d574f47649d3ad0f3c382198d P 8c1822e697d64b758a23b11c8fdeb59a [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:36.666042 15046 tablet_replica.cc:333] T 408866bb47904e4189bc235ca5b0c7c7 P 8c1822e697d64b758a23b11c8fdeb59a: stopping tablet replica
I20260709 13:18:36.666481 15046 raft_consensus.cc:2243] T 408866bb47904e4189bc235ca5b0c7c7 P 8c1822e697d64b758a23b11c8fdeb59a [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:18:36.667423 15046 raft_consensus.cc:2272] T 408866bb47904e4189bc235ca5b0c7c7 P 8c1822e697d64b758a23b11c8fdeb59a [term 3 FOLLOWER]: Raft consensus is shut down!
W20260709 13:18:36.693146 15783 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.14.177.129:39067: connect: Connection refused (error 111) [suppressed 2 similar messages]
W20260709 13:18:36.697160 15783 consensus_peers.cc:597] T 502d133e880b4a859d8674b9d0cb6931 P 62bbd271280e4b0a9ddf9ab46151e455 -> Peer 8c1822e697d64b758a23b11c8fdeb59a (127.14.177.129:39067): Couldn't send request to peer 8c1822e697d64b758a23b11c8fdeb59a. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:39067: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:18:36.699422 15046 tablet_server.cc:196] TabletServer@127.14.177.129:0 shutdown complete.
I20260709 13:18:36.717980 15046 tablet_server.cc:179] TabletServer@127.14.177.130:0 shutting down...
I20260709 13:18:36.747097 15046 ts_tablet_manager.cc:1507] Shutting down tablet manager...
W20260709 13:18:36.747087 15783 consensus_peers.cc:597] T 0ce7d20d574f47649d3ad0f3c382198d P 62bbd271280e4b0a9ddf9ab46151e455 -> Peer 8c1822e697d64b758a23b11c8fdeb59a (127.14.177.129:39067): Couldn't send request to peer 8c1822e697d64b758a23b11c8fdeb59a. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:39067: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:18:36.748070 15046 tablet_replica.cc:333] T 48e8b6e82d7b42e6bdd4b0a8a25dfd9e P 7cd7c680ef2e474c981ac81abadecfde: stopping tablet replica
I20260709 13:18:36.749105 15046 raft_consensus.cc:2243] T 48e8b6e82d7b42e6bdd4b0a8a25dfd9e P 7cd7c680ef2e474c981ac81abadecfde [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:18:36.750036 15046 raft_consensus.cc:2272] T 48e8b6e82d7b42e6bdd4b0a8a25dfd9e P 7cd7c680ef2e474c981ac81abadecfde [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:36.751919 15046 tablet_replica.cc:333] T 4d83a05b849340c186cc995649575d3b P 7cd7c680ef2e474c981ac81abadecfde: stopping tablet replica
I20260709 13:18:36.752395 15046 raft_consensus.cc:2243] T 4d83a05b849340c186cc995649575d3b P 7cd7c680ef2e474c981ac81abadecfde [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:18:36.753054 15046 raft_consensus.cc:2272] T 4d83a05b849340c186cc995649575d3b P 7cd7c680ef2e474c981ac81abadecfde [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:36.755064 15046 tablet_replica.cc:333] T 8fc26f7d12464423a212a1f595ca4b88 P 7cd7c680ef2e474c981ac81abadecfde: stopping tablet replica
I20260709 13:18:36.755642 15046 raft_consensus.cc:2243] T 8fc26f7d12464423a212a1f595ca4b88 P 7cd7c680ef2e474c981ac81abadecfde [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:18:36.756309 15046 raft_consensus.cc:2272] T 8fc26f7d12464423a212a1f595ca4b88 P 7cd7c680ef2e474c981ac81abadecfde [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:36.758311 15046 tablet_replica.cc:333] T 17f97fe9a93f4a86b1e2bad621bfb561 P 7cd7c680ef2e474c981ac81abadecfde: stopping tablet replica
I20260709 13:18:36.758864 15046 raft_consensus.cc:2243] T 17f97fe9a93f4a86b1e2bad621bfb561 P 7cd7c680ef2e474c981ac81abadecfde [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:36.759302 15046 raft_consensus.cc:2272] T 17f97fe9a93f4a86b1e2bad621bfb561 P 7cd7c680ef2e474c981ac81abadecfde [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:36.760965 15046 tablet_replica.cc:333] T 566eae6d93af4ec492b9b1a270dc03b7 P 7cd7c680ef2e474c981ac81abadecfde: stopping tablet replica
I20260709 13:18:36.761400 15046 raft_consensus.cc:2243] T 566eae6d93af4ec492b9b1a270dc03b7 P 7cd7c680ef2e474c981ac81abadecfde [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:36.761819 15046 raft_consensus.cc:2272] T 566eae6d93af4ec492b9b1a270dc03b7 P 7cd7c680ef2e474c981ac81abadecfde [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:36.763427 15046 tablet_replica.cc:333] T 408866bb47904e4189bc235ca5b0c7c7 P 7cd7c680ef2e474c981ac81abadecfde: stopping tablet replica
I20260709 13:18:36.763878 15046 raft_consensus.cc:2243] T 408866bb47904e4189bc235ca5b0c7c7 P 7cd7c680ef2e474c981ac81abadecfde [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:36.764256 15046 raft_consensus.cc:2272] T 408866bb47904e4189bc235ca5b0c7c7 P 7cd7c680ef2e474c981ac81abadecfde [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:36.765889 15046 tablet_replica.cc:333] T 8309d9a5a4ef4e6d914751bc0f410015 P 7cd7c680ef2e474c981ac81abadecfde: stopping tablet replica
I20260709 13:18:36.766315 15046 raft_consensus.cc:2243] T 8309d9a5a4ef4e6d914751bc0f410015 P 7cd7c680ef2e474c981ac81abadecfde [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:36.766692 15046 raft_consensus.cc:2272] T 8309d9a5a4ef4e6d914751bc0f410015 P 7cd7c680ef2e474c981ac81abadecfde [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:36.768841 15046 tablet_replica.cc:333] T 0ce7d20d574f47649d3ad0f3c382198d P 7cd7c680ef2e474c981ac81abadecfde: stopping tablet replica
I20260709 13:18:36.769279 15046 raft_consensus.cc:2243] T 0ce7d20d574f47649d3ad0f3c382198d P 7cd7c680ef2e474c981ac81abadecfde [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:36.769649 15046 raft_consensus.cc:2272] T 0ce7d20d574f47649d3ad0f3c382198d P 7cd7c680ef2e474c981ac81abadecfde [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:36.771965 15046 tablet_replica.cc:333] T 502d133e880b4a859d8674b9d0cb6931 P 7cd7c680ef2e474c981ac81abadecfde: stopping tablet replica
I20260709 13:18:36.772439 15046 raft_consensus.cc:2243] T 502d133e880b4a859d8674b9d0cb6931 P 7cd7c680ef2e474c981ac81abadecfde [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:36.772802 15046 raft_consensus.cc:2272] T 502d133e880b4a859d8674b9d0cb6931 P 7cd7c680ef2e474c981ac81abadecfde [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:36.774448 15046 tablet_replica.cc:333] T 8cff7911abd649b9a276ddae431311d1 P 7cd7c680ef2e474c981ac81abadecfde: stopping tablet replica
I20260709 13:18:36.774843 15046 raft_consensus.cc:2243] T 8cff7911abd649b9a276ddae431311d1 P 7cd7c680ef2e474c981ac81abadecfde [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:36.775271 15046 raft_consensus.cc:2272] T 8cff7911abd649b9a276ddae431311d1 P 7cd7c680ef2e474c981ac81abadecfde [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:36.805488 15046 tablet_server.cc:196] TabletServer@127.14.177.130:0 shutdown complete.
I20260709 13:18:36.825650 15046 tablet_server.cc:179] TabletServer@127.14.177.131:0 shutting down...
I20260709 13:18:36.850446 15046 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:18:36.851105 15046 tablet_replica.cc:333] T 48e8b6e82d7b42e6bdd4b0a8a25dfd9e P 62bbd271280e4b0a9ddf9ab46151e455: stopping tablet replica
I20260709 13:18:36.851744 15046 raft_consensus.cc:2243] T 48e8b6e82d7b42e6bdd4b0a8a25dfd9e P 62bbd271280e4b0a9ddf9ab46151e455 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:36.852175 15046 raft_consensus.cc:2272] T 48e8b6e82d7b42e6bdd4b0a8a25dfd9e P 62bbd271280e4b0a9ddf9ab46151e455 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:36.853955 15046 tablet_replica.cc:333] T 17f97fe9a93f4a86b1e2bad621bfb561 P 62bbd271280e4b0a9ddf9ab46151e455: stopping tablet replica
I20260709 13:18:36.854415 15046 raft_consensus.cc:2243] T 17f97fe9a93f4a86b1e2bad621bfb561 P 62bbd271280e4b0a9ddf9ab46151e455 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:36.854792 15046 raft_consensus.cc:2272] T 17f97fe9a93f4a86b1e2bad621bfb561 P 62bbd271280e4b0a9ddf9ab46151e455 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:36.856592 15046 tablet_replica.cc:333] T 566eae6d93af4ec492b9b1a270dc03b7 P 62bbd271280e4b0a9ddf9ab46151e455: stopping tablet replica
I20260709 13:18:36.857065 15046 raft_consensus.cc:2243] T 566eae6d93af4ec492b9b1a270dc03b7 P 62bbd271280e4b0a9ddf9ab46151e455 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:36.857465 15046 raft_consensus.cc:2272] T 566eae6d93af4ec492b9b1a270dc03b7 P 62bbd271280e4b0a9ddf9ab46151e455 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:36.859149 15046 tablet_replica.cc:333] T 4d83a05b849340c186cc995649575d3b P 62bbd271280e4b0a9ddf9ab46151e455: stopping tablet replica
I20260709 13:18:36.859664 15046 raft_consensus.cc:2243] T 4d83a05b849340c186cc995649575d3b P 62bbd271280e4b0a9ddf9ab46151e455 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:36.860036 15046 raft_consensus.cc:2272] T 4d83a05b849340c186cc995649575d3b P 62bbd271280e4b0a9ddf9ab46151e455 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:36.862514 15046 tablet_replica.cc:333] T 8fc26f7d12464423a212a1f595ca4b88 P 62bbd271280e4b0a9ddf9ab46151e455: stopping tablet replica
I20260709 13:18:36.862977 15046 raft_consensus.cc:2243] T 8fc26f7d12464423a212a1f595ca4b88 P 62bbd271280e4b0a9ddf9ab46151e455 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:36.863452 15046 raft_consensus.cc:2272] T 8fc26f7d12464423a212a1f595ca4b88 P 62bbd271280e4b0a9ddf9ab46151e455 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:36.865482 15046 tablet_replica.cc:333] T 8309d9a5a4ef4e6d914751bc0f410015 P 62bbd271280e4b0a9ddf9ab46151e455: stopping tablet replica
I20260709 13:18:36.865944 15046 raft_consensus.cc:2243] T 8309d9a5a4ef4e6d914751bc0f410015 P 62bbd271280e4b0a9ddf9ab46151e455 [term 4 LEADER]: Raft consensus shutting down.
I20260709 13:18:36.866603 15046 raft_consensus.cc:2272] T 8309d9a5a4ef4e6d914751bc0f410015 P 62bbd271280e4b0a9ddf9ab46151e455 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:36.868453 15046 tablet_replica.cc:333] T 408866bb47904e4189bc235ca5b0c7c7 P 62bbd271280e4b0a9ddf9ab46151e455: stopping tablet replica
I20260709 13:18:36.868978 15046 raft_consensus.cc:2243] T 408866bb47904e4189bc235ca5b0c7c7 P 62bbd271280e4b0a9ddf9ab46151e455 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:36.869336 15046 raft_consensus.cc:2272] T 408866bb47904e4189bc235ca5b0c7c7 P 62bbd271280e4b0a9ddf9ab46151e455 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:36.870991 15046 tablet_replica.cc:333] T 0ce7d20d574f47649d3ad0f3c382198d P 62bbd271280e4b0a9ddf9ab46151e455: stopping tablet replica
I20260709 13:18:36.871459 15046 raft_consensus.cc:2243] T 0ce7d20d574f47649d3ad0f3c382198d P 62bbd271280e4b0a9ddf9ab46151e455 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:18:36.872107 15046 raft_consensus.cc:2272] T 0ce7d20d574f47649d3ad0f3c382198d P 62bbd271280e4b0a9ddf9ab46151e455 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:36.873863 15046 tablet_replica.cc:333] T 502d133e880b4a859d8674b9d0cb6931 P 62bbd271280e4b0a9ddf9ab46151e455: stopping tablet replica
I20260709 13:18:36.874267 15046 raft_consensus.cc:2243] T 502d133e880b4a859d8674b9d0cb6931 P 62bbd271280e4b0a9ddf9ab46151e455 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:18:36.874951 15046 raft_consensus.cc:2272] T 502d133e880b4a859d8674b9d0cb6931 P 62bbd271280e4b0a9ddf9ab46151e455 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:36.876505 15046 tablet_replica.cc:333] T 8cff7911abd649b9a276ddae431311d1 P 62bbd271280e4b0a9ddf9ab46151e455: stopping tablet replica
I20260709 13:18:36.876897 15046 raft_consensus.cc:2243] T 8cff7911abd649b9a276ddae431311d1 P 62bbd271280e4b0a9ddf9ab46151e455 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:36.877238 15046 raft_consensus.cc:2272] T 8cff7911abd649b9a276ddae431311d1 P 62bbd271280e4b0a9ddf9ab46151e455 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:36.905558 15046 tablet_server.cc:196] TabletServer@127.14.177.131:0 shutdown complete.
I20260709 13:18:36.924337 15046 master.cc:562] Master@127.14.177.190:38145 shutting down...
I20260709 13:18:36.945933 15046 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 63272e34c419449bbb90a9013aa32b21 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:18:36.946432 15046 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 63272e34c419449bbb90a9013aa32b21 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:36.946739 15046 tablet_replica.cc:333] T 00000000000000000000000000000000 P 63272e34c419449bbb90a9013aa32b21: stopping tablet replica
I20260709 13:18:36.966571 15046 master.cc:584] Master@127.14.177.190:38145 shutdown complete.
62bbd271280e4b0a9ddf9ab46151e455  1
7cd7c680ef2e474c981ac81abadecfde  4
8c1822e697d64b758a23b11c8fdeb59a  5
[       OK ] LeaderRebalancerTest.FunctionalTestForNotDivided (10442 ms)
[ RUN      ] LeaderRebalancerTest.AddTserver
I20260709 13:18:37.007822 15046 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.14.177.190:44079
I20260709 13:18:37.008949 15046 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:18:37.015126 16025 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:18:37.015399 16024 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:18:37.015551 16027 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:18:37.016744 15046 server_base.cc:1061] running on GCE node
I20260709 13:18:37.017570 15046 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:18:37.017760 15046 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:18:37.017894 15046 hybrid_clock.cc:648] HybridClock initialized: now 1783603117017881 us; error 0 us; skew 500 ppm
I20260709 13:18:37.018394 15046 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:18:37.020745 15046 webserver.cc:533] Webserver started at http://127.14.177.190:35171/ using document root <none> and password file <none>
I20260709 13:18:37.021257 15046 fs_manager.cc:362] Metadata directory not provided
I20260709 13:18:37.021423 15046 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:18:37.021662 15046 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:18:37.022783 15046 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskdH_Ma7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783603093801613-15046-0/minicluster-data/master-0-root/instance:
uuid: "667a34b27a39474ebefc1af3cf5b4271"
format_stamp: "Formatted at 2026-07-09 13:18:37 on dist-test-slave-v5z2"
I20260709 13:18:37.028765 15046 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.002s	sys 0.003s
I20260709 13:18:37.032157 16032 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:18:37.032915 15046 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.000s
I20260709 13:18:37.033229 15046 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskdH_Ma7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783603093801613-15046-0/minicluster-data/master-0-root
uuid: "667a34b27a39474ebefc1af3cf5b4271"
format_stamp: "Formatted at 2026-07-09 13:18:37 on dist-test-slave-v5z2"
I20260709 13:18:37.033497 15046 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdH_Ma7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783603093801613-15046-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskdH_Ma7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783603093801613-15046-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskdH_Ma7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783603093801613-15046-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:18:37.053857 15046 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:18:37.055264 15046 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:18:37.095417 15046 rpc_server.cc:307] RPC server started. Bound to: 127.14.177.190:44079
I20260709 13:18:37.095525 16083 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.14.177.190:44079 every 8 connection(s)
I20260709 13:18:37.099911 16084 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:37.110687 16084 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 667a34b27a39474ebefc1af3cf5b4271: Bootstrap starting.
I20260709 13:18:37.115809 16084 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 667a34b27a39474ebefc1af3cf5b4271: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:37.120342 16084 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 667a34b27a39474ebefc1af3cf5b4271: No bootstrap required, opened a new log
I20260709 13:18:37.122555 16084 raft_consensus.cc:359] T 00000000000000000000000000000000 P 667a34b27a39474ebefc1af3cf5b4271 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "667a34b27a39474ebefc1af3cf5b4271" member_type: VOTER }
I20260709 13:18:37.122957 16084 raft_consensus.cc:385] T 00000000000000000000000000000000 P 667a34b27a39474ebefc1af3cf5b4271 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:37.123265 16084 raft_consensus.cc:740] T 00000000000000000000000000000000 P 667a34b27a39474ebefc1af3cf5b4271 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 667a34b27a39474ebefc1af3cf5b4271, State: Initialized, Role: FOLLOWER
I20260709 13:18:37.123957 16084 consensus_queue.cc:260] T 00000000000000000000000000000000 P 667a34b27a39474ebefc1af3cf5b4271 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "667a34b27a39474ebefc1af3cf5b4271" member_type: VOTER }
I20260709 13:18:37.124451 16084 raft_consensus.cc:399] T 00000000000000000000000000000000 P 667a34b27a39474ebefc1af3cf5b4271 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 13:18:37.124670 16084 raft_consensus.cc:493] T 00000000000000000000000000000000 P 667a34b27a39474ebefc1af3cf5b4271 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 13:18:37.124992 16084 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 667a34b27a39474ebefc1af3cf5b4271 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:37.129716 16084 raft_consensus.cc:515] T 00000000000000000000000000000000 P 667a34b27a39474ebefc1af3cf5b4271 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "667a34b27a39474ebefc1af3cf5b4271" member_type: VOTER }
I20260709 13:18:37.130254 16084 leader_election.cc:304] T 00000000000000000000000000000000 P 667a34b27a39474ebefc1af3cf5b4271 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 667a34b27a39474ebefc1af3cf5b4271; no voters: 
I20260709 13:18:37.131480 16084 leader_election.cc:290] T 00000000000000000000000000000000 P 667a34b27a39474ebefc1af3cf5b4271 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 13:18:37.131841 16087 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 667a34b27a39474ebefc1af3cf5b4271 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:18:37.133150 16087 raft_consensus.cc:697] T 00000000000000000000000000000000 P 667a34b27a39474ebefc1af3cf5b4271 [term 1 LEADER]: Becoming Leader. State: Replica: 667a34b27a39474ebefc1af3cf5b4271, State: Running, Role: LEADER
I20260709 13:18:37.133860 16087 consensus_queue.cc:237] T 00000000000000000000000000000000 P 667a34b27a39474ebefc1af3cf5b4271 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "667a34b27a39474ebefc1af3cf5b4271" member_type: VOTER }
I20260709 13:18:37.134686 16084 sys_catalog.cc:565] T 00000000000000000000000000000000 P 667a34b27a39474ebefc1af3cf5b4271 [sys.catalog]: configured and running, proceeding with master startup.
I20260709 13:18:37.139510 16088 sys_catalog.cc:455] T 00000000000000000000000000000000 P 667a34b27a39474ebefc1af3cf5b4271 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "667a34b27a39474ebefc1af3cf5b4271" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "667a34b27a39474ebefc1af3cf5b4271" member_type: VOTER } }
I20260709 13:18:37.140662 16088 sys_catalog.cc:458] T 00000000000000000000000000000000 P 667a34b27a39474ebefc1af3cf5b4271 [sys.catalog]: This master's current role is: LEADER
I20260709 13:18:37.139585 16089 sys_catalog.cc:455] T 00000000000000000000000000000000 P 667a34b27a39474ebefc1af3cf5b4271 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 667a34b27a39474ebefc1af3cf5b4271. Latest consensus state: current_term: 1 leader_uuid: "667a34b27a39474ebefc1af3cf5b4271" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "667a34b27a39474ebefc1af3cf5b4271" member_type: VOTER } }
I20260709 13:18:37.141978 16089 sys_catalog.cc:458] T 00000000000000000000000000000000 P 667a34b27a39474ebefc1af3cf5b4271 [sys.catalog]: This master's current role is: LEADER
I20260709 13:18:37.145658 16094 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 13:18:37.151880 16094 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 13:18:37.153424 15046 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 13:18:37.160522 16094 catalog_manager.cc:1383] Generated new cluster ID: 8650d72b97db4b69a5f4ff27b03123dd
I20260709 13:18:37.160804 16094 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 13:18:37.200824 16094 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 13:18:37.202152 16094 catalog_manager.cc:1540] Loading token signing keys...
I20260709 13:18:37.217660 16094 catalog_manager.cc:6070] T 00000000000000000000000000000000 P 667a34b27a39474ebefc1af3cf5b4271: Generated new TSK 0
I20260709 13:18:37.218286 16094 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 13:18:37.284224 15046 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:18:37.290081 16105 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:18:37.291647 16106 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:18:37.293327 15046 server_base.cc:1061] running on GCE node
W20260709 13:18:37.293407 16108 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:18:37.294430 15046 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:18:37.294646 15046 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:18:37.294802 15046 hybrid_clock.cc:648] HybridClock initialized: now 1783603117294786 us; error 0 us; skew 500 ppm
I20260709 13:18:37.295364 15046 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:18:37.297626 15046 webserver.cc:533] Webserver started at http://127.14.177.129:33313/ using document root <none> and password file <none>
I20260709 13:18:37.298096 15046 fs_manager.cc:362] Metadata directory not provided
I20260709 13:18:37.298296 15046 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:18:37.298556 15046 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:18:37.299656 15046 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskdH_Ma7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783603093801613-15046-0/minicluster-data/ts-0-root/instance:
uuid: "647a343a542d4a139fe106c19095129b"
format_stamp: "Formatted at 2026-07-09 13:18:37 on dist-test-slave-v5z2"
I20260709 13:18:37.304107 15046 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.004s	sys 0.000s
I20260709 13:18:37.307403 16113 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:18:37.308151 15046 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.000s
I20260709 13:18:37.308470 15046 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskdH_Ma7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783603093801613-15046-0/minicluster-data/ts-0-root
uuid: "647a343a542d4a139fe106c19095129b"
format_stamp: "Formatted at 2026-07-09 13:18:37 on dist-test-slave-v5z2"
I20260709 13:18:37.308838 15046 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdH_Ma7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783603093801613-15046-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskdH_Ma7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783603093801613-15046-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskdH_Ma7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783603093801613-15046-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:18:37.326195 15046 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:18:37.327376 15046 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:18:37.328717 15046 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:18:37.330976 15046 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:18:37.331173 15046 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:18:37.331453 15046 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:18:37.331619 15046 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:18:37.370941 15046 rpc_server.cc:307] RPC server started. Bound to: 127.14.177.129:36741
I20260709 13:18:37.371042 16175 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.14.177.129:36741 every 8 connection(s)
I20260709 13:18:37.375859 15046 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:18:37.382103 16180 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:18:37.383108 16181 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:18:37.385565 16183 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:18:37.386164 15046 server_base.cc:1061] running on GCE node
I20260709 13:18:37.387264 15046 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:18:37.387492 15046 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:18:37.387681 15046 hybrid_clock.cc:648] HybridClock initialized: now 1783603117387661 us; error 0 us; skew 500 ppm
I20260709 13:18:37.388237 15046 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:18:37.390525 15046 webserver.cc:533] Webserver started at http://127.14.177.130:42527/ using document root <none> and password file <none>
I20260709 13:18:37.391038 15046 fs_manager.cc:362] Metadata directory not provided
I20260709 13:18:37.391301 15046 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:18:37.391593 15046 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:18:37.392781 15046 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskdH_Ma7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783603093801613-15046-0/minicluster-data/ts-1-root/instance:
uuid: "1ca2725d290e47d68b1a4b609c9816ac"
format_stamp: "Formatted at 2026-07-09 13:18:37 on dist-test-slave-v5z2"
I20260709 13:18:37.398191 15046 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.006s	sys 0.000s
I20260709 13:18:37.398505 16176 heartbeater.cc:344] Connected to a master server at 127.14.177.190:44079
I20260709 13:18:37.398867 16176 heartbeater.cc:461] Registering TS with master...
I20260709 13:18:37.399612 16176 heartbeater.cc:507] Master 127.14.177.190:44079 requested a full tablet report, sending...
I20260709 13:18:37.401815 16049 ts_manager.cc:194] Registered new tserver with Master: 647a343a542d4a139fe106c19095129b (127.14.177.129:36741)
I20260709 13:18:37.402124 16188 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:18:37.402954 15046 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.002s	sys 0.001s
I20260709 13:18:37.403286 15046 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskdH_Ma7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783603093801613-15046-0/minicluster-data/ts-1-root
uuid: "1ca2725d290e47d68b1a4b609c9816ac"
format_stamp: "Formatted at 2026-07-09 13:18:37 on dist-test-slave-v5z2"
I20260709 13:18:37.403591 15046 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdH_Ma7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783603093801613-15046-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskdH_Ma7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783603093801613-15046-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskdH_Ma7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783603093801613-15046-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:18:37.403891 16049 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:49246
I20260709 13:18:37.416347 15046 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:18:37.417694 15046 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:18:37.419287 15046 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:18:37.421890 15046 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:18:37.422120 15046 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:18:37.422354 15046 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:18:37.422523 15046 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:18:37.461609 15046 rpc_server.cc:307] RPC server started. Bound to: 127.14.177.130:35943
I20260709 13:18:37.461707 16250 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.14.177.130:35943 every 8 connection(s)
I20260709 13:18:37.466209 15046 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:18:37.472201 16254 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:18:37.473122 16255 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:18:37.475793 16257 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:18:37.477130 15046 server_base.cc:1061] running on GCE node
I20260709 13:18:37.478044 15046 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:18:37.478264 15046 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:18:37.478438 15046 hybrid_clock.cc:648] HybridClock initialized: now 1783603117478420 us; error 0 us; skew 500 ppm
I20260709 13:18:37.478642 16251 heartbeater.cc:344] Connected to a master server at 127.14.177.190:44079
I20260709 13:18:37.479089 16251 heartbeater.cc:461] Registering TS with master...
I20260709 13:18:37.479089 15046 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:18:37.480067 16251 heartbeater.cc:507] Master 127.14.177.190:44079 requested a full tablet report, sending...
I20260709 13:18:37.481982 16049 ts_manager.cc:194] Registered new tserver with Master: 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:18:37.482373 15046 webserver.cc:533] Webserver started at http://127.14.177.131:40653/ using document root <none> and password file <none>
I20260709 13:18:37.482918 15046 fs_manager.cc:362] Metadata directory not provided
I20260709 13:18:37.483124 15046 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:18:37.483489 15046 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:18:37.483475 16049 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:49250
I20260709 13:18:37.484848 15046 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskdH_Ma7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783603093801613-15046-0/minicluster-data/ts-2-root/instance:
uuid: "90f3da57796b488c9087a2f523adf045"
format_stamp: "Formatted at 2026-07-09 13:18:37 on dist-test-slave-v5z2"
I20260709 13:18:37.489518 15046 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.005s	sys 0.000s
I20260709 13:18:37.493049 16262 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:18:37.493878 15046 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.001s	sys 0.000s
I20260709 13:18:37.494223 15046 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskdH_Ma7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783603093801613-15046-0/minicluster-data/ts-2-root
uuid: "90f3da57796b488c9087a2f523adf045"
format_stamp: "Formatted at 2026-07-09 13:18:37 on dist-test-slave-v5z2"
I20260709 13:18:37.494560 15046 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdH_Ma7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783603093801613-15046-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskdH_Ma7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783603093801613-15046-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskdH_Ma7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783603093801613-15046-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:18:37.532657 15046 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:18:37.534041 15046 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:18:37.535601 15046 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:18:37.538123 15046 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:18:37.538373 15046 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:18:37.538625 15046 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:18:37.538820 15046 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:18:37.581108 15046 rpc_server.cc:307] RPC server started. Bound to: 127.14.177.131:41389
I20260709 13:18:37.581211 16324 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.14.177.131:41389 every 8 connection(s)
I20260709 13:18:37.594628 16325 heartbeater.cc:344] Connected to a master server at 127.14.177.190:44079
I20260709 13:18:37.595009 16325 heartbeater.cc:461] Registering TS with master...
I20260709 13:18:37.595716 16325 heartbeater.cc:507] Master 127.14.177.190:44079 requested a full tablet report, sending...
I20260709 13:18:37.597527 16049 ts_manager.cc:194] Registered new tserver with Master: 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:18:37.597674 15046 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.013432098s
I20260709 13:18:37.598829 16049 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:49252
I20260709 13:18:37.624094 16049 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49260:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\313c+\002\004\001\000\226\307V\004\004\001\000a+\202\006\004\001\000,\217\255\010\004\001\000\367\362\330\n\004\001\000\302V\004\r\004\001\000\215\272/\017\004\001\000X\036[\021\004\001\000#\202\206\023\004\001\000\356\345\261\025\004\001\000\271I\335\027\004\001\000\204\255\010\032\004\001\000O\0214\034\004\001\000\032u_\036\004\001\000\345\330\212 \004\001\000\260<\266\"\004\001\000{\240\341$\004\001\000F\004\r\'\004\001\000\021h8)\004\001\000\334\313c+\004\001\000\247/\217-\004\001\000r\223\272/\004\001\000=\367\3451\004\001\000\010[\0214\004\001\000\323\276<6\004\001\000\236\"h8\004\001\000i\206\223:\004\001\0004\352\276<\004\001\000\377M\352>\004\001\000\312\261\025A\004\001\000\225\025AC\004\001\000`ylE\004\001\000+\335\227G\004\001\000\366@\303I\004\001\000\301\244\356K\004\001\000\214\010\032N\004\001\000WlEP\004\001\000\"\320pR\004\001\000\3553\234T\004\001\000\270\227\307V\004\001\000\203\373\362X\004\001\000N_\036[\004\001\000\031\303I]\004\001\000\344&u_\004\001\000\257\212\240a\004\001\000z\356\313c\004\001\000ER\367e\004\001\000\020\266\"h\004\001\000\333\031Nj\004\001\000\246}yl\004\001\000q\341\244n\004\001\000<E\320p\004\001\000\007\251\373r\004\001\000\322\014\'u\004\001\000\235pRw\004\001\000h\324}y\004\001\00038\251{\004\001\000\376\233\324}""\004\001\000\313c+\002\004\001\000\226\307V\004\004\001\000a+\202\006\004\001\000,\217\255\010\004\001\000\367\362\330\n\004\001\000\302V\004\r\004\001\000\215\272/\017\004\001\000X\036[\021\004\001\000#\202\206\023\004\001\000\356\345\261\025\004\001\000\271I\335\027\004\001\000\204\255\010\032\004\001\000O\0214\034\004\001\000\032u_\036\004\001\000\345\330\212 \004\001\000\260<\266\"\004\001\000{\240\341$\004\001\000F\004\r\'\004\001\000\021h8)\004\001\000\334\313c+\004\001\000\247/\217-\004\001\000r\223\272/\004\001\000=\367\3451\004\001\000\010[\0214\004\001\000\323\276<6\004\001\000\236\"h8\004\001\000i\206\223:\004\001\0004\352\276<\004\001\000\377M\352>\004\001\000\312\261\025A\004\001\000\225\025AC\004\001\000`ylE\004\001\000+\335\227G\004\001\000\366@\303I\004\001\000\301\244\356K\004\001\000\214\010\032N\004\001\000WlEP\004\001\000\"\320pR\004\001\000\3553\234T\004\001\000\270\227\307V\004\001\000\203\373\362X\004\001\000N_\036[\004\001\000\031\303I]\004\001\000\344&u_\004\001\000\257\212\240a\004\001\000z\356\313c\004\001\000ER\367e\004\001\000\020\266\"h\004\001\000\333\031Nj\004\001\000\246}yl\004\001\000q\341\244n\004\001\000<E\320p\004\001\000\007\251\373r\004\001\000\322\014\'u\004\001\000\235pRw\004\001\000h\324}y\004\001\00038\251{\004\001\000\376\233\324}"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 13:18:37.631016 16049 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 13:18:37.841575 16141 tablet_service.cc:1511] Processing CreateTablet for tablet 18ad7e3af5504571a69d10c00750a01c (DEFAULT_TABLE table=test-workload [id=ed3d15d8c9ad46bebc4fe3a402d2e54f]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 13:18:37.841867 16290 tablet_service.cc:1511] Processing CreateTablet for tablet 18ad7e3af5504571a69d10c00750a01c (DEFAULT_TABLE table=test-workload [id=ed3d15d8c9ad46bebc4fe3a402d2e54f]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 13:18:37.843081 16141 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 18ad7e3af5504571a69d10c00750a01c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:37.844381 16289 tablet_service.cc:1511] Processing CreateTablet for tablet c2f5f89186204242bf377f7c42b2e2da (DEFAULT_TABLE table=test-workload [id=ed3d15d8c9ad46bebc4fe3a402d2e54f]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 13:18:37.844635 16140 tablet_service.cc:1511] Processing CreateTablet for tablet c2f5f89186204242bf377f7c42b2e2da (DEFAULT_TABLE table=test-workload [id=ed3d15d8c9ad46bebc4fe3a402d2e54f]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 13:18:37.845937 16140 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c2f5f89186204242bf377f7c42b2e2da. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:37.846733 16288 tablet_service.cc:1511] Processing CreateTablet for tablet 8cc959fb7f704fba9009c0595467d074 (DEFAULT_TABLE table=test-workload [id=ed3d15d8c9ad46bebc4fe3a402d2e54f]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 13:18:37.848922 16287 tablet_service.cc:1511] Processing CreateTablet for tablet 9e65676b21b748728ad39668cfe722ec (DEFAULT_TABLE table=test-workload [id=ed3d15d8c9ad46bebc4fe3a402d2e54f]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 13:18:37.851038 16286 tablet_service.cc:1511] Processing CreateTablet for tablet 8a878999c0024d2abe517484180290fc (DEFAULT_TABLE table=test-workload [id=ed3d15d8c9ad46bebc4fe3a402d2e54f]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 13:18:37.853258 16285 tablet_service.cc:1511] Processing CreateTablet for tablet d2372981006d4b218215c28964dea623 (DEFAULT_TABLE table=test-workload [id=ed3d15d8c9ad46bebc4fe3a402d2e54f]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 13:18:37.855340 16284 tablet_service.cc:1511] Processing CreateTablet for tablet 434bff3082d141838068f323bdb0d1ed (DEFAULT_TABLE table=test-workload [id=ed3d15d8c9ad46bebc4fe3a402d2e54f]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 13:18:37.857506 16283 tablet_service.cc:1511] Processing CreateTablet for tablet fffab26427e24d398e2695aad40ca134 (DEFAULT_TABLE table=test-workload [id=ed3d15d8c9ad46bebc4fe3a402d2e54f]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 13:18:37.859558 16282 tablet_service.cc:1511] Processing CreateTablet for tablet 5a96373bb0e14dce8a266b2fc6c58507 (DEFAULT_TABLE table=test-workload [id=ed3d15d8c9ad46bebc4fe3a402d2e54f]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 13:18:37.861627 16281 tablet_service.cc:1511] Processing CreateTablet for tablet 02838c86531d4f98bf58d1c22b8baf77 (DEFAULT_TABLE table=test-workload [id=ed3d15d8c9ad46bebc4fe3a402d2e54f]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 13:18:37.864398 16290 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 18ad7e3af5504571a69d10c00750a01c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:37.865948 16340 tablet_bootstrap.cc:492] T 18ad7e3af5504571a69d10c00750a01c P 647a343a542d4a139fe106c19095129b: Bootstrap starting.
I20260709 13:18:37.866393 16281 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 02838c86531d4f98bf58d1c22b8baf77. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:37.871519 16285 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d2372981006d4b218215c28964dea623. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:37.872829 16340 tablet_bootstrap.cc:654] T 18ad7e3af5504571a69d10c00750a01c P 647a343a542d4a139fe106c19095129b: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:37.875502 16286 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8a878999c0024d2abe517484180290fc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:37.879592 16287 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9e65676b21b748728ad39668cfe722ec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:37.882872 16289 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c2f5f89186204242bf377f7c42b2e2da. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:37.884330 16342 tablet_bootstrap.cc:492] T 18ad7e3af5504571a69d10c00750a01c P 90f3da57796b488c9087a2f523adf045: Bootstrap starting.
I20260709 13:18:37.887633 16288 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8cc959fb7f704fba9009c0595467d074. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:37.889281 16290 tablet_service.cc:1511] Processing CreateTablet for tablet 4e9b2bf7011f4ae688a79d7bb5a447b4 (DEFAULT_TABLE table=test-workload [id=ed3d15d8c9ad46bebc4fe3a402d2e54f]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 13:18:37.890501 16290 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4e9b2bf7011f4ae688a79d7bb5a447b4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:37.891619 16282 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5a96373bb0e14dce8a266b2fc6c58507. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:37.892251 16284 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 434bff3082d141838068f323bdb0d1ed. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:37.895125 16283 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fffab26427e24d398e2695aad40ca134. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:37.895612 16340 tablet_bootstrap.cc:492] T 18ad7e3af5504571a69d10c00750a01c P 647a343a542d4a139fe106c19095129b: No bootstrap required, opened a new log
I20260709 13:18:37.896092 16340 ts_tablet_manager.cc:1403] T 18ad7e3af5504571a69d10c00750a01c P 647a343a542d4a139fe106c19095129b: Time spent bootstrapping tablet: real 0.030s	user 0.018s	sys 0.000s
I20260709 13:18:37.901230 16140 tablet_service.cc:1511] Processing CreateTablet for tablet 9e65676b21b748728ad39668cfe722ec (DEFAULT_TABLE table=test-workload [id=ed3d15d8c9ad46bebc4fe3a402d2e54f]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 13:18:37.903761 16140 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9e65676b21b748728ad39668cfe722ec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:37.900496 16141 tablet_service.cc:1511] Processing CreateTablet for tablet 8cc959fb7f704fba9009c0595467d074 (DEFAULT_TABLE table=test-workload [id=ed3d15d8c9ad46bebc4fe3a402d2e54f]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 13:18:37.908169 16141 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8cc959fb7f704fba9009c0595467d074. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:37.908700 16342 tablet_bootstrap.cc:654] T 18ad7e3af5504571a69d10c00750a01c P 90f3da57796b488c9087a2f523adf045: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:37.921950 16287 tablet_service.cc:1511] Processing CreateTablet for tablet 7c2095a5f4ad49408dbc7f69c606120d (DEFAULT_TABLE table=test-workload [id=ed3d15d8c9ad46bebc4fe3a402d2e54f]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 13:18:37.922420 16282 tablet_service.cc:1511] Processing CreateTablet for tablet 8c11074579b046d99e585c2818e18a2c (DEFAULT_TABLE table=test-workload [id=ed3d15d8c9ad46bebc4fe3a402d2e54f]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 13:18:37.923372 16287 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7c2095a5f4ad49408dbc7f69c606120d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:37.927666 16282 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8c11074579b046d99e585c2818e18a2c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:37.932087 16340 raft_consensus.cc:359] T 18ad7e3af5504571a69d10c00750a01c P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } }
I20260709 13:18:37.932858 16340 raft_consensus.cc:385] T 18ad7e3af5504571a69d10c00750a01c P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:37.933151 16340 raft_consensus.cc:740] T 18ad7e3af5504571a69d10c00750a01c P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Initialized, Role: FOLLOWER
I20260709 13:18:37.934006 16340 consensus_queue.cc:260] T 18ad7e3af5504571a69d10c00750a01c P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } }
I20260709 13:18:37.941576 16287 tablet_service.cc:1511] Processing CreateTablet for tablet a378d7f796f94615b3b347119a9880fd (DEFAULT_TABLE table=test-workload [id=ed3d15d8c9ad46bebc4fe3a402d2e54f]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 13:18:37.941592 16139 tablet_service.cc:1511] Processing CreateTablet for tablet 8a878999c0024d2abe517484180290fc (DEFAULT_TABLE table=test-workload [id=ed3d15d8c9ad46bebc4fe3a402d2e54f]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 13:18:37.943041 16287 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a378d7f796f94615b3b347119a9880fd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:37.945289 16139 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8a878999c0024d2abe517484180290fc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:37.947968 16136 tablet_service.cc:1511] Processing CreateTablet for tablet fffab26427e24d398e2695aad40ca134 (DEFAULT_TABLE table=test-workload [id=ed3d15d8c9ad46bebc4fe3a402d2e54f]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 13:18:37.948254 16135 tablet_service.cc:1511] Processing CreateTablet for tablet 5a96373bb0e14dce8a266b2fc6c58507 (DEFAULT_TABLE table=test-workload [id=ed3d15d8c9ad46bebc4fe3a402d2e54f]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 13:18:37.947191 16138 tablet_service.cc:1511] Processing CreateTablet for tablet d2372981006d4b218215c28964dea623 (DEFAULT_TABLE table=test-workload [id=ed3d15d8c9ad46bebc4fe3a402d2e54f]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 13:18:37.949784 16136 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fffab26427e24d398e2695aad40ca134. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:37.948273 16137 tablet_service.cc:1511] Processing CreateTablet for tablet 434bff3082d141838068f323bdb0d1ed (DEFAULT_TABLE table=test-workload [id=ed3d15d8c9ad46bebc4fe3a402d2e54f]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 13:18:37.946923 16340 ts_tablet_manager.cc:1434] T 18ad7e3af5504571a69d10c00750a01c P 647a343a542d4a139fe106c19095129b: Time spent starting tablet: real 0.051s	user 0.008s	sys 0.000s
I20260709 13:18:37.956046 16138 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d2372981006d4b218215c28964dea623. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:37.956713 16137 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 434bff3082d141838068f323bdb0d1ed. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:37.958143 16286 tablet_service.cc:1511] Processing CreateTablet for tablet ada5a004ba7c46819051e3ccec71ffdb (DEFAULT_TABLE table=test-workload [id=ed3d15d8c9ad46bebc4fe3a402d2e54f]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 13:18:37.959580 16135 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5a96373bb0e14dce8a266b2fc6c58507. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:37.960691 16285 tablet_service.cc:1511] Processing CreateTablet for tablet 68a6838d82ef4cac83507f906139d90f (DEFAULT_TABLE table=test-workload [id=ed3d15d8c9ad46bebc4fe3a402d2e54f]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 13:18:37.961381 16283 tablet_service.cc:1511] Processing CreateTablet for tablet b6f07842d3aa47858df5665874c86f3a (DEFAULT_TABLE table=test-workload [id=ed3d15d8c9ad46bebc4fe3a402d2e54f]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 13:18:37.961956 16285 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 68a6838d82ef4cac83507f906139d90f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:37.962631 16283 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b6f07842d3aa47858df5665874c86f3a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:37.965667 16284 tablet_service.cc:1511] Processing CreateTablet for tablet d2ab0ac8347641e6955e9bce7d90c1a2 (DEFAULT_TABLE table=test-workload [id=ed3d15d8c9ad46bebc4fe3a402d2e54f]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 13:18:37.965821 16134 tablet_service.cc:1511] Processing CreateTablet for tablet 02838c86531d4f98bf58d1c22b8baf77 (DEFAULT_TABLE table=test-workload [id=ed3d15d8c9ad46bebc4fe3a402d2e54f]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 13:18:37.967082 16284 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d2ab0ac8347641e6955e9bce7d90c1a2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:37.967844 16133 tablet_service.cc:1511] Processing CreateTablet for tablet 4e9b2bf7011f4ae688a79d7bb5a447b4 (DEFAULT_TABLE table=test-workload [id=ed3d15d8c9ad46bebc4fe3a402d2e54f]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 13:18:37.968449 16176 heartbeater.cc:499] Master 127.14.177.190:44079 was elected leader, sending a full tablet report...
I20260709 13:18:37.969621 16132 tablet_service.cc:1511] Processing CreateTablet for tablet 7c2095a5f4ad49408dbc7f69c606120d (DEFAULT_TABLE table=test-workload [id=ed3d15d8c9ad46bebc4fe3a402d2e54f]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 13:18:37.972337 16134 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 02838c86531d4f98bf58d1c22b8baf77. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:37.984565 16289 tablet_service.cc:1511] Processing CreateTablet for tablet 3a7ca61592e848ea9b0fb81a2ce33cdc (DEFAULT_TABLE table=test-workload [id=ed3d15d8c9ad46bebc4fe3a402d2e54f]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 13:18:37.975382 16340 tablet_bootstrap.cc:492] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b: Bootstrap starting.
I20260709 13:18:37.991816 16133 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4e9b2bf7011f4ae688a79d7bb5a447b4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:37.995288 16132 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7c2095a5f4ad49408dbc7f69c606120d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:37.997345 16286 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ada5a004ba7c46819051e3ccec71ffdb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:37.998032 16289 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3a7ca61592e848ea9b0fb81a2ce33cdc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:37.996975 16288 tablet_service.cc:1511] Processing CreateTablet for tablet 3884f864a96d428785895e5463e5728f (DEFAULT_TABLE table=test-workload [id=ed3d15d8c9ad46bebc4fe3a402d2e54f]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 13:18:38.008814 16288 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3884f864a96d428785895e5463e5728f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:38.008400 16139 tablet_service.cc:1511] Processing CreateTablet for tablet 8c11074579b046d99e585c2818e18a2c (DEFAULT_TABLE table=test-workload [id=ed3d15d8c9ad46bebc4fe3a402d2e54f]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 13:18:37.973677 16342 tablet_bootstrap.cc:492] T 18ad7e3af5504571a69d10c00750a01c P 90f3da57796b488c9087a2f523adf045: No bootstrap required, opened a new log
I20260709 13:18:38.032269 16342 ts_tablet_manager.cc:1403] T 18ad7e3af5504571a69d10c00750a01c P 90f3da57796b488c9087a2f523adf045: Time spent bootstrapping tablet: real 0.148s	user 0.012s	sys 0.007s
I20260709 13:18:38.037179 16141 tablet_service.cc:1511] Processing CreateTablet for tablet a378d7f796f94615b3b347119a9880fd (DEFAULT_TABLE table=test-workload [id=ed3d15d8c9ad46bebc4fe3a402d2e54f]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 13:18:38.038554 16141 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a378d7f796f94615b3b347119a9880fd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:38.039990 16285 tablet_service.cc:1511] Processing CreateTablet for tablet 5a959dbecc2648eb85d1cc550f5a3245 (DEFAULT_TABLE table=test-workload [id=ed3d15d8c9ad46bebc4fe3a402d2e54f]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 13:18:38.043551 16139 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8c11074579b046d99e585c2818e18a2c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:38.049304 16216 tablet_service.cc:1511] Processing CreateTablet for tablet 18ad7e3af5504571a69d10c00750a01c (DEFAULT_TABLE table=test-workload [id=ed3d15d8c9ad46bebc4fe3a402d2e54f]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 13:18:38.051118 16285 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5a959dbecc2648eb85d1cc550f5a3245. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:38.051353 16216 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 18ad7e3af5504571a69d10c00750a01c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:38.052732 16214 tablet_service.cc:1511] Processing CreateTablet for tablet 8cc959fb7f704fba9009c0595467d074 (DEFAULT_TABLE table=test-workload [id=ed3d15d8c9ad46bebc4fe3a402d2e54f]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 13:18:38.054065 16214 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8cc959fb7f704fba9009c0595467d074. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:38.056511 16213 tablet_service.cc:1511] Processing CreateTablet for tablet 9e65676b21b748728ad39668cfe722ec (DEFAULT_TABLE table=test-workload [id=ed3d15d8c9ad46bebc4fe3a402d2e54f]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 13:18:38.058718 16212 tablet_service.cc:1511] Processing CreateTablet for tablet 8a878999c0024d2abe517484180290fc (DEFAULT_TABLE table=test-workload [id=ed3d15d8c9ad46bebc4fe3a402d2e54f]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 13:18:38.060426 16289 tablet_service.cc:1511] Processing CreateTablet for tablet d16dfad0c3b540e5b4b86492ddf87208 (DEFAULT_TABLE table=test-workload [id=ed3d15d8c9ad46bebc4fe3a402d2e54f]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 13:18:38.060807 16211 tablet_service.cc:1511] Processing CreateTablet for tablet d2372981006d4b218215c28964dea623 (DEFAULT_TABLE table=test-workload [id=ed3d15d8c9ad46bebc4fe3a402d2e54f]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 13:18:38.074249 16340 tablet_bootstrap.cc:654] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:38.074790 16283 tablet_service.cc:1511] Processing CreateTablet for tablet 87ac0d1f7e8a419f9bcbc04aee0e067e (DEFAULT_TABLE table=test-workload [id=ed3d15d8c9ad46bebc4fe3a402d2e54f]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 13:18:38.076131 16137 tablet_service.cc:1511] Processing CreateTablet for tablet ada5a004ba7c46819051e3ccec71ffdb (DEFAULT_TABLE table=test-workload [id=ed3d15d8c9ad46bebc4fe3a402d2e54f]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 13:18:38.076565 16140 tablet_service.cc:1511] Processing CreateTablet for tablet 68a6838d82ef4cac83507f906139d90f (DEFAULT_TABLE table=test-workload [id=ed3d15d8c9ad46bebc4fe3a402d2e54f]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 13:18:38.076977 16283 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 87ac0d1f7e8a419f9bcbc04aee0e067e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:38.076131 16281 tablet_service.cc:1511] Processing CreateTablet for tablet 1d5d59407eb545b895d8d922e0dd69fd (DEFAULT_TABLE table=test-workload [id=ed3d15d8c9ad46bebc4fe3a402d2e54f]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 13:18:38.078094 16281 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1d5d59407eb545b895d8d922e0dd69fd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:38.080150 16342 raft_consensus.cc:359] T 18ad7e3af5504571a69d10c00750a01c P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } }
I20260709 13:18:38.080928 16342 raft_consensus.cc:385] T 18ad7e3af5504571a69d10c00750a01c P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:38.062332 16210 tablet_service.cc:1511] Processing CreateTablet for tablet 434bff3082d141838068f323bdb0d1ed (DEFAULT_TABLE table=test-workload [id=ed3d15d8c9ad46bebc4fe3a402d2e54f]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 13:18:38.064024 16209 tablet_service.cc:1511] Processing CreateTablet for tablet fffab26427e24d398e2695aad40ca134 (DEFAULT_TABLE table=test-workload [id=ed3d15d8c9ad46bebc4fe3a402d2e54f]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 13:18:38.051033 16215 tablet_service.cc:1511] Processing CreateTablet for tablet c2f5f89186204242bf377f7c42b2e2da (DEFAULT_TABLE table=test-workload [id=ed3d15d8c9ad46bebc4fe3a402d2e54f]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 13:18:38.066987 16208 tablet_service.cc:1511] Processing CreateTablet for tablet 5a96373bb0e14dce8a266b2fc6c58507 (DEFAULT_TABLE table=test-workload [id=ed3d15d8c9ad46bebc4fe3a402d2e54f]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 13:18:38.071501 16212 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8a878999c0024d2abe517484180290fc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:38.072806 16207 tablet_service.cc:1511] Processing CreateTablet for tablet 02838c86531d4f98bf58d1c22b8baf77 (DEFAULT_TABLE table=test-workload [id=ed3d15d8c9ad46bebc4fe3a402d2e54f]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 13:18:38.080618 16282 tablet_service.cc:1511] Processing CreateTablet for tablet 9c61a39fd4f84424b129c1a6f9ba404d (DEFAULT_TABLE table=test-workload [id=ed3d15d8c9ad46bebc4fe3a402d2e54f]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 13:18:38.085541 16211 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d2372981006d4b218215c28964dea623. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:38.086160 16210 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 434bff3082d141838068f323bdb0d1ed. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:38.087517 16213 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9e65676b21b748728ad39668cfe722ec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:38.088153 16215 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c2f5f89186204242bf377f7c42b2e2da. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:38.062107 16287 tablet_service.cc:1511] Processing CreateTablet for tablet 829b4ad8c5a54eababeea1181b960d4b (DEFAULT_TABLE table=test-workload [id=ed3d15d8c9ad46bebc4fe3a402d2e54f]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 13:18:38.085129 16286 tablet_service.cc:1511] Processing CreateTablet for tablet 66ae176961b74813a58bc15c9c4c6f2a (DEFAULT_TABLE table=test-workload [id=ed3d15d8c9ad46bebc4fe3a402d2e54f]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 13:18:38.082688 16289 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d16dfad0c3b540e5b4b86492ddf87208. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:38.088779 16209 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fffab26427e24d398e2695aad40ca134. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:38.081746 16342 raft_consensus.cc:740] T 18ad7e3af5504571a69d10c00750a01c P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Initialized, Role: FOLLOWER
I20260709 13:18:38.095762 16207 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 02838c86531d4f98bf58d1c22b8baf77. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:38.096257 16342 consensus_queue.cc:260] T 18ad7e3af5504571a69d10c00750a01c P 90f3da57796b488c9087a2f523adf045 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } }
I20260709 13:18:38.111400 16208 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5a96373bb0e14dce8a266b2fc6c58507. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:38.112905 16138 tablet_service.cc:1511] Processing CreateTablet for tablet d2ab0ac8347641e6955e9bce7d90c1a2 (DEFAULT_TABLE table=test-workload [id=ed3d15d8c9ad46bebc4fe3a402d2e54f]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 13:18:38.114720 16138 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d2ab0ac8347641e6955e9bce7d90c1a2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:38.122442 16135 tablet_service.cc:1511] Processing CreateTablet for tablet 5a959dbecc2648eb85d1cc550f5a3245 (DEFAULT_TABLE table=test-workload [id=ed3d15d8c9ad46bebc4fe3a402d2e54f]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 13:18:38.137465 16137 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ada5a004ba7c46819051e3ccec71ffdb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:38.144641 16282 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9c61a39fd4f84424b129c1a6f9ba404d. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:18:38.150198 16097 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:18:38.151599 16284 tablet_service.cc:1511] Processing CreateTablet for tablet c0eab25ce4c64e018a041beeda2867db (DEFAULT_TABLE table=test-workload [id=ed3d15d8c9ad46bebc4fe3a402d2e54f]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 13:18:38.154641 16136 tablet_service.cc:1511] Processing CreateTablet for tablet d16dfad0c3b540e5b4b86492ddf87208 (DEFAULT_TABLE table=test-workload [id=ed3d15d8c9ad46bebc4fe3a402d2e54f]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 13:18:38.155450 16287 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 829b4ad8c5a54eababeea1181b960d4b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:38.155951 16132 tablet_service.cc:1511] Processing CreateTablet for tablet 3884f864a96d428785895e5463e5728f (DEFAULT_TABLE table=test-workload [id=ed3d15d8c9ad46bebc4fe3a402d2e54f]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 13:18:38.157366 16133 tablet_service.cc:1511] Processing CreateTablet for tablet 3a7ca61592e848ea9b0fb81a2ce33cdc (DEFAULT_TABLE table=test-workload [id=ed3d15d8c9ad46bebc4fe3a402d2e54f]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 13:18:38.159456 16286 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 66ae176961b74813a58bc15c9c4c6f2a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:38.161932 16140 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 68a6838d82ef4cac83507f906139d90f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:38.167721 16134 tablet_service.cc:1511] Processing CreateTablet for tablet b6f07842d3aa47858df5665874c86f3a (DEFAULT_TABLE table=test-workload [id=ed3d15d8c9ad46bebc4fe3a402d2e54f]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 13:18:38.171031 16284 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c0eab25ce4c64e018a041beeda2867db. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:38.175283 16135 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5a959dbecc2648eb85d1cc550f5a3245. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:38.179469 16133 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3a7ca61592e848ea9b0fb81a2ce33cdc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:38.183478 16132 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3884f864a96d428785895e5463e5728f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:38.183344 16290 tablet_service.cc:1511] Processing CreateTablet for tablet 62fe3346cd3c42c4a55e14af9073bca4 (DEFAULT_TABLE table=test-workload [id=ed3d15d8c9ad46bebc4fe3a402d2e54f]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 13:18:38.184130 16134 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b6f07842d3aa47858df5665874c86f3a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:38.184250 16342 ts_tablet_manager.cc:1434] T 18ad7e3af5504571a69d10c00750a01c P 90f3da57796b488c9087a2f523adf045: Time spent starting tablet: real 0.152s	user 0.007s	sys 0.004s
I20260709 13:18:38.185019 16325 heartbeater.cc:499] Master 127.14.177.190:44079 was elected leader, sending a full tablet report...
I20260709 13:18:38.187453 16136 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d16dfad0c3b540e5b4b86492ddf87208. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:38.189198 16342 tablet_bootstrap.cc:492] T 9e65676b21b748728ad39668cfe722ec P 90f3da57796b488c9087a2f523adf045: Bootstrap starting.
I20260709 13:18:38.214170 16139 tablet_service.cc:1511] Processing CreateTablet for tablet 829b4ad8c5a54eababeea1181b960d4b (DEFAULT_TABLE table=test-workload [id=ed3d15d8c9ad46bebc4fe3a402d2e54f]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 13:18:38.216751 16139 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 829b4ad8c5a54eababeea1181b960d4b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:38.217072 16290 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 62fe3346cd3c42c4a55e14af9073bca4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:38.254469 16209 tablet_service.cc:1511] Processing CreateTablet for tablet ada5a004ba7c46819051e3ccec71ffdb (DEFAULT_TABLE table=test-workload [id=ed3d15d8c9ad46bebc4fe3a402d2e54f]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 13:18:38.255046 16286 tablet_service.cc:1511] Processing CreateTablet for tablet af57996a8be84d398f0f023e8d08ef77 (DEFAULT_TABLE table=test-workload [id=ed3d15d8c9ad46bebc4fe3a402d2e54f]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 13:18:38.221532 16141 tablet_service.cc:1511] Processing CreateTablet for tablet 87ac0d1f7e8a419f9bcbc04aee0e067e (DEFAULT_TABLE table=test-workload [id=ed3d15d8c9ad46bebc4fe3a402d2e54f]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 13:18:38.255774 16209 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ada5a004ba7c46819051e3ccec71ffdb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:38.228127 16285 tablet_service.cc:1511] Processing CreateTablet for tablet 74ca8140f35c4ae29e8a664f6bdfc829 (DEFAULT_TABLE table=test-workload [id=ed3d15d8c9ad46bebc4fe3a402d2e54f]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 13:18:38.229401 16288 tablet_service.cc:1511] Processing CreateTablet for tablet b87147731a68400e8cce9b879daa7c5d (DEFAULT_TABLE table=test-workload [id=ed3d15d8c9ad46bebc4fe3a402d2e54f]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 13:18:38.232617 16340 tablet_bootstrap.cc:492] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b: No bootstrap required, opened a new log
I20260709 13:18:38.259569 16340 ts_tablet_manager.cc:1403] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b: Time spent bootstrapping tablet: real 0.284s	user 0.016s	sys 0.000s
I20260709 13:18:38.260721 16141 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 87ac0d1f7e8a419f9bcbc04aee0e067e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:38.261570 16134 tablet_service.cc:1511] Processing CreateTablet for tablet 1d5d59407eb545b895d8d922e0dd69fd (DEFAULT_TABLE table=test-workload [id=ed3d15d8c9ad46bebc4fe3a402d2e54f]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 13:18:38.262847 16134 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1d5d59407eb545b895d8d922e0dd69fd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:38.262809 16340 raft_consensus.cc:359] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:38.263661 16340 raft_consensus.cc:385] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:38.263999 16340 raft_consensus.cc:740] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Initialized, Role: FOLLOWER
I20260709 13:18:38.264822 16345 raft_consensus.cc:493] T 18ad7e3af5504571a69d10c00750a01c P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:38.265280 16345 raft_consensus.cc:515] T 18ad7e3af5504571a69d10c00750a01c P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } }
I20260709 13:18:38.267414 16340 consensus_queue.cc:260] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:38.274485 16345 leader_election.cc:290] T 18ad7e3af5504571a69d10c00750a01c P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:18:38.237975 16342 tablet_bootstrap.cc:654] T 9e65676b21b748728ad39668cfe722ec P 90f3da57796b488c9087a2f523adf045: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:38.243322 16213 tablet_service.cc:1511] Processing CreateTablet for tablet 7c2095a5f4ad49408dbc7f69c606120d (DEFAULT_TABLE table=test-workload [id=ed3d15d8c9ad46bebc4fe3a402d2e54f]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 13:18:38.279940 16213 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7c2095a5f4ad49408dbc7f69c606120d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:38.245664 16216 tablet_service.cc:1511] Processing CreateTablet for tablet 8c11074579b046d99e585c2818e18a2c (DEFAULT_TABLE table=test-workload [id=ed3d15d8c9ad46bebc4fe3a402d2e54f]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 13:18:38.250407 16215 tablet_service.cc:1511] Processing CreateTablet for tablet a378d7f796f94615b3b347119a9880fd (DEFAULT_TABLE table=test-workload [id=ed3d15d8c9ad46bebc4fe3a402d2e54f]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 13:18:38.248229 16212 tablet_service.cc:1511] Processing CreateTablet for tablet 4e9b2bf7011f4ae688a79d7bb5a447b4 (DEFAULT_TABLE table=test-workload [id=ed3d15d8c9ad46bebc4fe3a402d2e54f]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 13:18:38.256253 16286 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet af57996a8be84d398f0f023e8d08ef77. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:38.292650 16344 raft_consensus.cc:493] T 18ad7e3af5504571a69d10c00750a01c P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:38.293090 16215 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a378d7f796f94615b3b347119a9880fd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:38.293166 16344 raft_consensus.cc:515] T 18ad7e3af5504571a69d10c00750a01c P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } }
I20260709 13:18:38.296620 16288 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b87147731a68400e8cce9b879daa7c5d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:38.238747 16283 tablet_service.cc:1511] Processing CreateTablet for tablet 5718c623ebee4d2bb770f0f9ddb2f164 (DEFAULT_TABLE table=test-workload [id=ed3d15d8c9ad46bebc4fe3a402d2e54f]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 13:18:38.267021 16208 tablet_service.cc:1511] Processing CreateTablet for tablet 68a6838d82ef4cac83507f906139d90f (DEFAULT_TABLE table=test-workload [id=ed3d15d8c9ad46bebc4fe3a402d2e54f]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 13:18:38.296008 16133 tablet_service.cc:1511] Processing CreateTablet for tablet 9c61a39fd4f84424b129c1a6f9ba404d (DEFAULT_TABLE table=test-workload [id=ed3d15d8c9ad46bebc4fe3a402d2e54f]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 13:18:38.312749 16285 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 74ca8140f35c4ae29e8a664f6bdfc829. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:38.322300 16214 tablet_service.cc:1511] Processing CreateTablet for tablet d2ab0ac8347641e6955e9bce7d90c1a2 (DEFAULT_TABLE table=test-workload [id=ed3d15d8c9ad46bebc4fe3a402d2e54f]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 13:18:38.328735 16347 tablet_bootstrap.cc:492] T 8cc959fb7f704fba9009c0595467d074 P 1ca2725d290e47d68b1a4b609c9816ac: Bootstrap starting.
I20260709 13:18:38.318883 16344 leader_election.cc:290] T 18ad7e3af5504571a69d10c00750a01c P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:18:38.332955 16344 raft_consensus.cc:493] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:38.333622 16344 raft_consensus.cc:515] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:38.316013 16216 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8c11074579b046d99e585c2818e18a2c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:38.356789 16287 tablet_service.cc:1511] Processing CreateTablet for tablet 806d7690c2b644959152074bb2cab0a7 (DEFAULT_TABLE table=test-workload [id=ed3d15d8c9ad46bebc4fe3a402d2e54f]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 13:18:38.362917 16212 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4e9b2bf7011f4ae688a79d7bb5a447b4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:38.364403 16211 tablet_service.cc:1511] Processing CreateTablet for tablet 3884f864a96d428785895e5463e5728f (DEFAULT_TABLE table=test-workload [id=ed3d15d8c9ad46bebc4fe3a402d2e54f]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 13:18:38.366032 16211 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3884f864a96d428785895e5463e5728f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:38.370275 16283 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5718c623ebee4d2bb770f0f9ddb2f164. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:38.375855 16344 leader_election.cc:290] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:18:38.382198 16342 tablet_bootstrap.cc:492] T 9e65676b21b748728ad39668cfe722ec P 90f3da57796b488c9087a2f523adf045: No bootstrap required, opened a new log
I20260709 13:18:38.382983 16342 ts_tablet_manager.cc:1403] T 9e65676b21b748728ad39668cfe722ec P 90f3da57796b488c9087a2f523adf045: Time spent bootstrapping tablet: real 0.194s	user 0.006s	sys 0.013s
I20260709 13:18:38.374400 16284 tablet_service.cc:1511] Processing CreateTablet for tablet 2794af5b99984efebdee564035841cb0 (DEFAULT_TABLE table=test-workload [id=ed3d15d8c9ad46bebc4fe3a402d2e54f]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260709 13:18:38.398525 16340 ts_tablet_manager.cc:1434] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b: Time spent starting tablet: real 0.137s	user 0.008s	sys 0.000s
I20260709 13:18:38.401861 16208 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 68a6838d82ef4cac83507f906139d90f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:38.407588 16133 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9c61a39fd4f84424b129c1a6f9ba404d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:38.412717 16287 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 806d7690c2b644959152074bb2cab0a7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:38.424808 16140 tablet_service.cc:1511] Processing CreateTablet for tablet 66ae176961b74813a58bc15c9c4c6f2a (DEFAULT_TABLE table=test-workload [id=ed3d15d8c9ad46bebc4fe3a402d2e54f]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 13:18:38.431810 16282 tablet_service.cc:1511] Processing CreateTablet for tablet 0a9de347c711477e87c582d003f36925 (DEFAULT_TABLE table=test-workload [id=ed3d15d8c9ad46bebc4fe3a402d2e54f]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 13:18:38.433264 16282 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0a9de347c711477e87c582d003f36925. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:38.438206 16340 tablet_bootstrap.cc:492] T 8a878999c0024d2abe517484180290fc P 647a343a542d4a139fe106c19095129b: Bootstrap starting.
I20260709 13:18:38.438728 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18ad7e3af5504571a69d10c00750a01c" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:18:38.439697 16300 raft_consensus.cc:2468] T 18ad7e3af5504571a69d10c00750a01c P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 0.
I20260709 13:18:38.441159 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f5f89186204242bf377f7c42b2e2da" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:18:38.443986 16347 tablet_bootstrap.cc:654] T 8cc959fb7f704fba9009c0595467d074 P 1ca2725d290e47d68b1a4b609c9816ac: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:38.433445 16132 tablet_service.cc:1511] Processing CreateTablet for tablet c0eab25ce4c64e018a041beeda2867db (DEFAULT_TABLE table=test-workload [id=ed3d15d8c9ad46bebc4fe3a402d2e54f]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 13:18:38.448169 16115 leader_election.cc:304] T 18ad7e3af5504571a69d10c00750a01c P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:18:38.450055 16344 raft_consensus.cc:2804] T 18ad7e3af5504571a69d10c00750a01c P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:18:38.450477 16344 raft_consensus.cc:493] T 18ad7e3af5504571a69d10c00750a01c P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:38.450840 16344 raft_consensus.cc:3060] T 18ad7e3af5504571a69d10c00750a01c P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:38.452121 16281 tablet_service.cc:1511] Processing CreateTablet for tablet 746aabf856ab401c9ef5e859371e7b67 (DEFAULT_TABLE table=test-workload [id=ed3d15d8c9ad46bebc4fe3a402d2e54f]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260709 13:18:38.453537 16281 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 746aabf856ab401c9ef5e859371e7b67. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:18:38.458374 16115 leader_election.cc:343] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:18:38.466913 16132 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c0eab25ce4c64e018a041beeda2867db. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:38.468506 16134 tablet_service.cc:1511] Processing CreateTablet for tablet 74ca8140f35c4ae29e8a664f6bdfc829 (DEFAULT_TABLE table=test-workload [id=ed3d15d8c9ad46bebc4fe3a402d2e54f]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 13:18:38.472584 16344 raft_consensus.cc:515] T 18ad7e3af5504571a69d10c00750a01c P 647a343a542d4a139fe106c19095129b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } }
I20260709 13:18:38.473076 16136 tablet_service.cc:1511] Processing CreateTablet for tablet b87147731a68400e8cce9b879daa7c5d (DEFAULT_TABLE table=test-workload [id=ed3d15d8c9ad46bebc4fe3a402d2e54f]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 13:18:38.474768 16344 leader_election.cc:290] T 18ad7e3af5504571a69d10c00750a01c P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 1 election: Requested vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:18:38.475926 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18ad7e3af5504571a69d10c00750a01c" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:18:38.475878 16287 tablet_service.cc:1511] Processing CreateTablet for tablet 8158bbd17b994d2eb96f560bd0f1b88d (DEFAULT_TABLE table=test-workload [id=ed3d15d8c9ad46bebc4fe3a402d2e54f]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260709 13:18:38.476750 16299 raft_consensus.cc:3060] T 18ad7e3af5504571a69d10c00750a01c P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:38.478766 16210 tablet_service.cc:1511] Processing CreateTablet for tablet b6f07842d3aa47858df5665874c86f3a (DEFAULT_TABLE table=test-workload [id=ed3d15d8c9ad46bebc4fe3a402d2e54f]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 13:18:38.479562 16151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18ad7e3af5504571a69d10c00750a01c" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:18:38.480487 16151 raft_consensus.cc:2393] T 18ad7e3af5504571a69d10c00750a01c P 647a343a542d4a139fe106c19095129b [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 1: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
I20260709 13:18:38.482553 16134 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 74ca8140f35c4ae29e8a664f6bdfc829. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:38.482633 16141 tablet_service.cc:1511] Processing CreateTablet for tablet 5718c623ebee4d2bb770f0f9ddb2f164 (DEFAULT_TABLE table=test-workload [id=ed3d15d8c9ad46bebc4fe3a402d2e54f]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 13:18:38.484028 16136 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b87147731a68400e8cce9b879daa7c5d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:38.485960 16251 heartbeater.cc:499] Master 127.14.177.190:44079 was elected leader, sending a full tablet report...
I20260709 13:18:38.476132 16137 tablet_service.cc:1511] Processing CreateTablet for tablet 62fe3346cd3c42c4a55e14af9073bca4 (DEFAULT_TABLE table=test-workload [id=ed3d15d8c9ad46bebc4fe3a402d2e54f]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 13:18:38.498031 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18ad7e3af5504571a69d10c00750a01c" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
W20260709 13:18:38.500057 16265 leader_election.cc:343] T 18ad7e3af5504571a69d10c00750a01c P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:18:38.500679 16265 leader_election.cc:304] T 18ad7e3af5504571a69d10c00750a01c P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:18:38.505121 16140 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 66ae176961b74813a58bc15c9c4c6f2a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:38.507858 16137 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 62fe3346cd3c42c4a55e14af9073bca4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:38.508935 16138 tablet_service.cc:1511] Processing CreateTablet for tablet af57996a8be84d398f0f023e8d08ef77 (DEFAULT_TABLE table=test-workload [id=ed3d15d8c9ad46bebc4fe3a402d2e54f]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 13:18:38.507488 16290 tablet_service.cc:1511] Processing CreateTablet for tablet ee9ed67006144797b1de6ea8c2013a68 (DEFAULT_TABLE table=test-workload [id=ed3d15d8c9ad46bebc4fe3a402d2e54f]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 13:18:38.510234 16138 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet af57996a8be84d398f0f023e8d08ef77. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:38.465530 16284 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2794af5b99984efebdee564035841cb0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:38.515120 16287 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8158bbd17b994d2eb96f560bd0f1b88d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:38.515918 16290 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ee9ed67006144797b1de6ea8c2013a68. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:38.517038 16289 tablet_service.cc:1511] Processing CreateTablet for tablet e4dc6b0541ca431ebc757beac00521f1 (DEFAULT_TABLE table=test-workload [id=ed3d15d8c9ad46bebc4fe3a402d2e54f]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260709 13:18:38.514312 16214 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d2ab0ac8347641e6955e9bce7d90c1a2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:38.521548 16141 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5718c623ebee4d2bb770f0f9ddb2f164. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:38.524430 16213 tablet_service.cc:1511] Processing CreateTablet for tablet 3a7ca61592e848ea9b0fb81a2ce33cdc (DEFAULT_TABLE table=test-workload [id=ed3d15d8c9ad46bebc4fe3a402d2e54f]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 13:18:38.525710 16289 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e4dc6b0541ca431ebc757beac00521f1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:38.528028 16135 tablet_service.cc:1511] Processing CreateTablet for tablet 806d7690c2b644959152074bb2cab0a7 (DEFAULT_TABLE table=test-workload [id=ed3d15d8c9ad46bebc4fe3a402d2e54f]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 13:18:38.534169 16135 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 806d7690c2b644959152074bb2cab0a7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:38.538553 16207 tablet_service.cc:1511] Processing CreateTablet for tablet 5a959dbecc2648eb85d1cc550f5a3245 (DEFAULT_TABLE table=test-workload [id=ed3d15d8c9ad46bebc4fe3a402d2e54f]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 13:18:38.551301 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18ad7e3af5504571a69d10c00750a01c" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:18:38.552305 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f5f89186204242bf377f7c42b2e2da" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:18:38.561051 16288 tablet_service.cc:1511] Processing CreateTablet for tablet 31969392ccdb43b491911a319e8bc7e1 (DEFAULT_TABLE table=test-workload [id=ed3d15d8c9ad46bebc4fe3a402d2e54f]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260709 13:18:38.553282 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18ad7e3af5504571a69d10c00750a01c" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:18:38.562474 16288 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 31969392ccdb43b491911a319e8bc7e1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:38.531915 16340 tablet_bootstrap.cc:654] T 8a878999c0024d2abe517484180290fc P 647a343a542d4a139fe106c19095129b: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:38.563838 16299 raft_consensus.cc:2468] T 18ad7e3af5504571a69d10c00750a01c P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 1.
W20260709 13:18:38.563886 16116 leader_election.cc:343] T 18ad7e3af5504571a69d10c00750a01c P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:18:38.565114 16116 leader_election.cc:343] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:18:38.565231 16115 leader_election.cc:304] T 18ad7e3af5504571a69d10c00750a01c P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:18:38.565619 16116 leader_election.cc:304] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:18:38.566138 16344 raft_consensus.cc:2804] T 18ad7e3af5504571a69d10c00750a01c P 647a343a542d4a139fe106c19095129b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:18:38.566515 16210 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b6f07842d3aa47858df5665874c86f3a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:38.566636 16350 raft_consensus.cc:2749] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:18:38.567399 16345 raft_consensus.cc:2749] T 18ad7e3af5504571a69d10c00750a01c P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:18:38.561517 16347 tablet_bootstrap.cc:492] T 8cc959fb7f704fba9009c0595467d074 P 1ca2725d290e47d68b1a4b609c9816ac: No bootstrap required, opened a new log
I20260709 13:18:38.571017 16347 ts_tablet_manager.cc:1403] T 8cc959fb7f704fba9009c0595467d074 P 1ca2725d290e47d68b1a4b609c9816ac: Time spent bootstrapping tablet: real 0.243s	user 0.013s	sys 0.010s
I20260709 13:18:38.572304 16342 raft_consensus.cc:359] T 9e65676b21b748728ad39668cfe722ec P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:38.573033 16342 raft_consensus.cc:385] T 9e65676b21b748728ad39668cfe722ec P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:38.573328 16342 raft_consensus.cc:740] T 9e65676b21b748728ad39668cfe722ec P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Initialized, Role: FOLLOWER
I20260709 13:18:38.571221 16213 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3a7ca61592e848ea9b0fb81a2ce33cdc. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:18:38.566921 16116 leader_election.cc:343] T 18ad7e3af5504571a69d10c00750a01c P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:18:38.574793 16347 raft_consensus.cc:359] T 8cc959fb7f704fba9009c0595467d074 P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:38.576670 16347 raft_consensus.cc:385] T 8cc959fb7f704fba9009c0595467d074 P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:38.577109 16347 raft_consensus.cc:740] T 8cc959fb7f704fba9009c0595467d074 P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Initialized, Role: FOLLOWER
I20260709 13:18:38.580744 16283 tablet_service.cc:1511] Processing CreateTablet for tablet c4f65cdf00b748bb963ee6e81ef43b79 (DEFAULT_TABLE table=test-workload [id=ed3d15d8c9ad46bebc4fe3a402d2e54f]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 13:18:38.582131 16283 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c4f65cdf00b748bb963ee6e81ef43b79. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:38.581847 16139 tablet_service.cc:1511] Processing CreateTablet for tablet 746aabf856ab401c9ef5e859371e7b67 (DEFAULT_TABLE table=test-workload [id=ed3d15d8c9ad46bebc4fe3a402d2e54f]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260709 13:18:38.582285 16207 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5a959dbecc2648eb85d1cc550f5a3245. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:38.584414 16342 consensus_queue.cc:260] T 9e65676b21b748728ad39668cfe722ec P 90f3da57796b488c9087a2f523adf045 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:38.589569 16132 tablet_service.cc:1511] Processing CreateTablet for tablet 0a9de347c711477e87c582d003f36925 (DEFAULT_TABLE table=test-workload [id=ed3d15d8c9ad46bebc4fe3a402d2e54f]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 13:18:38.590658 16347 consensus_queue.cc:260] T 8cc959fb7f704fba9009c0595467d074 P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:38.593099 16285 tablet_service.cc:1511] Processing CreateTablet for tablet 8638ee363273426db1393dd81e5c7b72 (DEFAULT_TABLE table=test-workload [id=ed3d15d8c9ad46bebc4fe3a402d2e54f]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260709 13:18:38.594568 16285 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8638ee363273426db1393dd81e5c7b72. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:38.597868 16215 tablet_service.cc:1511] Processing CreateTablet for tablet d16dfad0c3b540e5b4b86492ddf87208 (DEFAULT_TABLE table=test-workload [id=ed3d15d8c9ad46bebc4fe3a402d2e54f]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 13:18:38.583082 16139 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 746aabf856ab401c9ef5e859371e7b67. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:38.625011 16136 tablet_service.cc:1511] Processing CreateTablet for tablet 2794af5b99984efebdee564035841cb0 (DEFAULT_TABLE table=test-workload [id=ed3d15d8c9ad46bebc4fe3a402d2e54f]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260709 13:18:38.631022 16342 ts_tablet_manager.cc:1434] T 9e65676b21b748728ad39668cfe722ec P 90f3da57796b488c9087a2f523adf045: Time spent starting tablet: real 0.240s	user 0.008s	sys 0.000s
I20260709 13:18:38.605994 16286 tablet_service.cc:1511] Processing CreateTablet for tablet 08c74490c40243a88e62d1760b5cd814 (DEFAULT_TABLE table=test-workload [id=ed3d15d8c9ad46bebc4fe3a402d2e54f]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 13:18:38.631161 16347 ts_tablet_manager.cc:1434] T 8cc959fb7f704fba9009c0595467d074 P 1ca2725d290e47d68b1a4b609c9816ac: Time spent starting tablet: real 0.060s	user 0.010s	sys 0.005s
I20260709 13:18:38.632215 16342 tablet_bootstrap.cc:492] T 5a96373bb0e14dce8a266b2fc6c58507 P 90f3da57796b488c9087a2f523adf045: Bootstrap starting.
I20260709 13:18:38.645037 16286 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 08c74490c40243a88e62d1760b5cd814. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:38.640332 16344 raft_consensus.cc:697] T 18ad7e3af5504571a69d10c00750a01c P 647a343a542d4a139fe106c19095129b [term 1 LEADER]: Becoming Leader. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:18:38.648777 16344 consensus_queue.cc:237] T 18ad7e3af5504571a69d10c00750a01c P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } }
I20260709 13:18:38.650331 16284 tablet_service.cc:1511] Processing CreateTablet for tablet ecc1e363230043fe9fef390f0b979f20 (DEFAULT_TABLE table=test-workload [id=ed3d15d8c9ad46bebc4fe3a402d2e54f]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260709 13:18:38.652660 16132 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0a9de347c711477e87c582d003f36925. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:38.661197 16209 tablet_service.cc:1511] Processing CreateTablet for tablet 829b4ad8c5a54eababeea1181b960d4b (DEFAULT_TABLE table=test-workload [id=ed3d15d8c9ad46bebc4fe3a402d2e54f]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 13:18:38.662325 16347 tablet_bootstrap.cc:492] T c2f5f89186204242bf377f7c42b2e2da P 1ca2725d290e47d68b1a4b609c9816ac: Bootstrap starting.
I20260709 13:18:38.659104 16215 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d16dfad0c3b540e5b4b86492ddf87208. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:38.663484 16136 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2794af5b99984efebdee564035841cb0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:38.673244 16209 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 829b4ad8c5a54eababeea1181b960d4b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:38.656133 16282 tablet_service.cc:1511] Processing CreateTablet for tablet 08b7d1ab519044eda2d5e04a07242f25 (DEFAULT_TABLE table=test-workload [id=ed3d15d8c9ad46bebc4fe3a402d2e54f]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 13:18:38.678885 16290 tablet_service.cc:1511] Processing CreateTablet for tablet 1df93779209748b9860f6d893dd3de53 (DEFAULT_TABLE table=test-workload [id=ed3d15d8c9ad46bebc4fe3a402d2e54f]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 13:18:38.683522 16282 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 08b7d1ab519044eda2d5e04a07242f25. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:38.686702 16290 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1df93779209748b9860f6d893dd3de53. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:38.687574 16284 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ecc1e363230043fe9fef390f0b979f20. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:38.689348 16340 tablet_bootstrap.cc:492] T 8a878999c0024d2abe517484180290fc P 647a343a542d4a139fe106c19095129b: No bootstrap required, opened a new log
I20260709 13:18:38.689832 16340 ts_tablet_manager.cc:1403] T 8a878999c0024d2abe517484180290fc P 647a343a542d4a139fe106c19095129b: Time spent bootstrapping tablet: real 0.252s	user 0.015s	sys 0.011s
I20260709 13:18:38.692463 16213 tablet_service.cc:1511] Processing CreateTablet for tablet 1d5d59407eb545b895d8d922e0dd69fd (DEFAULT_TABLE table=test-workload [id=ed3d15d8c9ad46bebc4fe3a402d2e54f]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 13:18:38.685319 16134 tablet_service.cc:1511] Processing CreateTablet for tablet ee9ed67006144797b1de6ea8c2013a68 (DEFAULT_TABLE table=test-workload [id=ed3d15d8c9ad46bebc4fe3a402d2e54f]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 13:18:38.693852 16213 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1d5d59407eb545b895d8d922e0dd69fd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:38.665776 16211 tablet_service.cc:1511] Processing CreateTablet for tablet 87ac0d1f7e8a419f9bcbc04aee0e067e (DEFAULT_TABLE table=test-workload [id=ed3d15d8c9ad46bebc4fe3a402d2e54f]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 13:18:38.693994 16134 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ee9ed67006144797b1de6ea8c2013a68. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:38.696055 16211 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 87ac0d1f7e8a419f9bcbc04aee0e067e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:38.697842 16210 tablet_service.cc:1511] Processing CreateTablet for tablet 9c61a39fd4f84424b129c1a6f9ba404d (DEFAULT_TABLE table=test-workload [id=ed3d15d8c9ad46bebc4fe3a402d2e54f]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 13:18:38.702260 16210 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9c61a39fd4f84424b129c1a6f9ba404d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:38.708390 16347 tablet_bootstrap.cc:654] T c2f5f89186204242bf377f7c42b2e2da P 1ca2725d290e47d68b1a4b609c9816ac: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:38.713588 16286 tablet_service.cc:1511] Processing CreateTablet for tablet 8d4ab870a05347868956aa8b841b2981 (DEFAULT_TABLE table=test-workload [id=ed3d15d8c9ad46bebc4fe3a402d2e54f]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260709 13:18:38.715497 16340 raft_consensus.cc:359] T 8a878999c0024d2abe517484180290fc P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:38.718127 16283 tablet_service.cc:1511] Processing CreateTablet for tablet cdbd78acb4cf46298e0de8ba0d130e19 (DEFAULT_TABLE table=test-workload [id=ed3d15d8c9ad46bebc4fe3a402d2e54f]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 13:18:38.698976 16216 tablet_service.cc:1511] Processing CreateTablet for tablet 62fe3346cd3c42c4a55e14af9073bca4 (DEFAULT_TABLE table=test-workload [id=ed3d15d8c9ad46bebc4fe3a402d2e54f]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 13:18:38.727675 16345 raft_consensus.cc:493] T 9e65676b21b748728ad39668cfe722ec P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:38.728156 16345 raft_consensus.cc:515] T 9e65676b21b748728ad39668cfe722ec P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:38.730093 16345 leader_election.cc:290] T 9e65676b21b748728ad39668cfe722ec P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:18:38.733597 16151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e65676b21b748728ad39668cfe722ec" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
W20260709 13:18:38.735975 16265 leader_election.cc:343] T 9e65676b21b748728ad39668cfe722ec P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:18:38.737412 16342 tablet_bootstrap.cc:654] T 5a96373bb0e14dce8a266b2fc6c58507 P 90f3da57796b488c9087a2f523adf045: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:38.739526 16137 tablet_service.cc:1511] Processing CreateTablet for tablet 31969392ccdb43b491911a319e8bc7e1 (DEFAULT_TABLE table=test-workload [id=ed3d15d8c9ad46bebc4fe3a402d2e54f]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260709 13:18:38.724654 16340 raft_consensus.cc:385] T 8a878999c0024d2abe517484180290fc P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:38.743256 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e65676b21b748728ad39668cfe722ec" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
W20260709 13:18:38.744796 16265 leader_election.cc:343] T 9e65676b21b748728ad39668cfe722ec P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:18:38.745208 16265 leader_election.cc:304] T 9e65676b21b748728ad39668cfe722ec P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:18:38.747594 16138 tablet_service.cc:1511] Processing CreateTablet for tablet 8158bbd17b994d2eb96f560bd0f1b88d (DEFAULT_TABLE table=test-workload [id=ed3d15d8c9ad46bebc4fe3a402d2e54f]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260709 13:18:38.751734 16286 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8d4ab870a05347868956aa8b841b2981. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:38.753448 16345 raft_consensus.cc:2749] T 9e65676b21b748728ad39668cfe722ec P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:18:38.750128 16048 catalog_manager.cc:5697] T 18ad7e3af5504571a69d10c00750a01c P 647a343a542d4a139fe106c19095129b reported cstate change: term changed from 0 to 1, leader changed from <none> to 647a343a542d4a139fe106c19095129b (127.14.177.129). New cstate: current_term: 1 leader_uuid: "647a343a542d4a139fe106c19095129b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:38.742619 16340 raft_consensus.cc:740] T 8a878999c0024d2abe517484180290fc P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Initialized, Role: FOLLOWER
I20260709 13:18:38.755942 16340 consensus_queue.cc:260] T 8a878999c0024d2abe517484180290fc P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:38.757964 16340 ts_tablet_manager.cc:1434] T 8a878999c0024d2abe517484180290fc P 647a343a542d4a139fe106c19095129b: Time spent starting tablet: real 0.068s	user 0.005s	sys 0.000s
I20260709 13:18:38.758339 16212 tablet_service.cc:1511] Processing CreateTablet for tablet 66ae176961b74813a58bc15c9c4c6f2a (DEFAULT_TABLE table=test-workload [id=ed3d15d8c9ad46bebc4fe3a402d2e54f]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 13:18:38.759753 16212 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 66ae176961b74813a58bc15c9c4c6f2a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:38.760347 16208 tablet_service.cc:1511] Processing CreateTablet for tablet c0eab25ce4c64e018a041beeda2867db (DEFAULT_TABLE table=test-workload [id=ed3d15d8c9ad46bebc4fe3a402d2e54f]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 13:18:38.765786 16283 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cdbd78acb4cf46298e0de8ba0d130e19. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:38.765810 16281 tablet_service.cc:1511] Processing CreateTablet for tablet 57a744f5aaf944678a63469cbf514015 (DEFAULT_TABLE table=test-workload [id=ed3d15d8c9ad46bebc4fe3a402d2e54f]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260709 13:18:38.767273 16281 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 57a744f5aaf944678a63469cbf514015. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:38.768383 16207 tablet_service.cc:1511] Processing CreateTablet for tablet b87147731a68400e8cce9b879daa7c5d (DEFAULT_TABLE table=test-workload [id=ed3d15d8c9ad46bebc4fe3a402d2e54f]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 13:18:38.772576 16287 tablet_service.cc:1511] Processing CreateTablet for tablet 26354361e9b044cea1c2b86c4882fe92 (DEFAULT_TABLE table=test-workload [id=ed3d15d8c9ad46bebc4fe3a402d2e54f]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260709 13:18:38.773860 16287 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 26354361e9b044cea1c2b86c4882fe92. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:38.779785 16214 tablet_service.cc:1511] Processing CreateTablet for tablet 74ca8140f35c4ae29e8a664f6bdfc829 (DEFAULT_TABLE table=test-workload [id=ed3d15d8c9ad46bebc4fe3a402d2e54f]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 13:18:38.782253 16358 raft_consensus.cc:493] T 8cc959fb7f704fba9009c0595467d074 P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:38.782747 16358 raft_consensus.cc:515] T 8cc959fb7f704fba9009c0595467d074 P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:38.790068 16358 leader_election.cc:290] T 8cc959fb7f704fba9009c0595467d074 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:18:38.790133 16213 tablet_service.cc:1511] Processing CreateTablet for tablet 5718c623ebee4d2bb770f0f9ddb2f164 (DEFAULT_TABLE table=test-workload [id=ed3d15d8c9ad46bebc4fe3a402d2e54f]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 13:18:38.806823 16288 tablet_service.cc:1511] Processing CreateTablet for tablet e5ad4eb281564936a59f41e387a47156 (DEFAULT_TABLE table=test-workload [id=ed3d15d8c9ad46bebc4fe3a402d2e54f]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 13:18:38.799010 16216 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 62fe3346cd3c42c4a55e14af9073bca4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:38.808597 16208 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c0eab25ce4c64e018a041beeda2867db. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:38.813627 16207 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b87147731a68400e8cce9b879daa7c5d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:38.815789 16212 tablet_service.cc:1511] Processing CreateTablet for tablet af57996a8be84d398f0f023e8d08ef77 (DEFAULT_TABLE table=test-workload [id=ed3d15d8c9ad46bebc4fe3a402d2e54f]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 13:18:38.817292 16212 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet af57996a8be84d398f0f023e8d08ef77. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:38.834931 16288 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e5ad4eb281564936a59f41e387a47156. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:38.835703 16282 tablet_service.cc:1511] Processing CreateTablet for tablet 9ad5dbfa99ee4b8d9e28be92b3f36db5 (DEFAULT_TABLE table=test-workload [id=ed3d15d8c9ad46bebc4fe3a402d2e54f]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 13:18:38.835778 16214 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 74ca8140f35c4ae29e8a664f6bdfc829. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:38.862490 16282 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9ad5dbfa99ee4b8d9e28be92b3f36db5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:38.865224 16215 tablet_service.cc:1511] Processing CreateTablet for tablet 746aabf856ab401c9ef5e859371e7b67 (DEFAULT_TABLE table=test-workload [id=ed3d15d8c9ad46bebc4fe3a402d2e54f]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260709 13:18:38.867401 16138 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8158bbd17b994d2eb96f560bd0f1b88d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:38.869052 16134 tablet_service.cc:1511] Processing CreateTablet for tablet e4dc6b0541ca431ebc757beac00521f1 (DEFAULT_TABLE table=test-workload [id=ed3d15d8c9ad46bebc4fe3a402d2e54f]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260709 13:18:38.870880 16134 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e4dc6b0541ca431ebc757beac00521f1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:38.872817 16141 tablet_service.cc:1511] Processing CreateTablet for tablet 8638ee363273426db1393dd81e5c7b72 (DEFAULT_TABLE table=test-workload [id=ed3d15d8c9ad46bebc4fe3a402d2e54f]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260709 13:18:38.873171 16133 tablet_service.cc:1511] Processing CreateTablet for tablet c4f65cdf00b748bb963ee6e81ef43b79 (DEFAULT_TABLE table=test-workload [id=ed3d15d8c9ad46bebc4fe3a402d2e54f]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 13:18:38.875263 16340 tablet_bootstrap.cc:492] T 434bff3082d141838068f323bdb0d1ed P 647a343a542d4a139fe106c19095129b: Bootstrap starting.
I20260709 13:18:38.875065 16139 tablet_service.cc:1511] Processing CreateTablet for tablet 08c74490c40243a88e62d1760b5cd814 (DEFAULT_TABLE table=test-workload [id=ed3d15d8c9ad46bebc4fe3a402d2e54f]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 13:18:38.891690 16285 tablet_service.cc:1511] Processing CreateTablet for tablet c9944d5e4b8c4109af9955c4f08c9095 (DEFAULT_TABLE table=test-workload [id=ed3d15d8c9ad46bebc4fe3a402d2e54f]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260709 13:18:38.892961 16285 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c9944d5e4b8c4109af9955c4f08c9095. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:38.894487 16284 tablet_service.cc:1511] Processing CreateTablet for tablet 19baa7bc367844c288bca8f81439c413 (DEFAULT_TABLE table=test-workload [id=ed3d15d8c9ad46bebc4fe3a402d2e54f]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260709 13:18:38.907011 16215 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 746aabf856ab401c9ef5e859371e7b67. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:38.907984 16284 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 19baa7bc367844c288bca8f81439c413. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:38.881218 16140 tablet_service.cc:1511] Processing CreateTablet for tablet ecc1e363230043fe9fef390f0b979f20 (DEFAULT_TABLE table=test-workload [id=ed3d15d8c9ad46bebc4fe3a402d2e54f]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260709 13:18:38.881879 16289 tablet_service.cc:1511] Processing CreateTablet for tablet 9e4bfb9090c8460eb14fa7e0d818e7f1 (DEFAULT_TABLE table=test-workload [id=ed3d15d8c9ad46bebc4fe3a402d2e54f]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260709 13:18:38.883272 16135 tablet_service.cc:1511] Processing CreateTablet for tablet 1df93779209748b9860f6d893dd3de53 (DEFAULT_TABLE table=test-workload [id=ed3d15d8c9ad46bebc4fe3a402d2e54f]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 13:18:38.880513 16132 tablet_service.cc:1511] Processing CreateTablet for tablet 08b7d1ab519044eda2d5e04a07242f25 (DEFAULT_TABLE table=test-workload [id=ed3d15d8c9ad46bebc4fe3a402d2e54f]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 13:18:38.917784 16137 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 31969392ccdb43b491911a319e8bc7e1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:38.923564 16140 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ecc1e363230043fe9fef390f0b979f20. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:38.919324 16344 raft_consensus.cc:493] T 8a878999c0024d2abe517484180290fc P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:38.929925 16141 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8638ee363273426db1393dd81e5c7b72. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:38.930008 16344 raft_consensus.cc:515] T 8a878999c0024d2abe517484180290fc P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:38.930851 16342 tablet_bootstrap.cc:492] T 5a96373bb0e14dce8a266b2fc6c58507 P 90f3da57796b488c9087a2f523adf045: No bootstrap required, opened a new log
I20260709 13:18:38.931380 16342 ts_tablet_manager.cc:1403] T 5a96373bb0e14dce8a266b2fc6c58507 P 90f3da57796b488c9087a2f523adf045: Time spent bootstrapping tablet: real 0.299s	user 0.020s	sys 0.006s
I20260709 13:18:38.941310 16139 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 08c74490c40243a88e62d1760b5cd814. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:38.942094 16135 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1df93779209748b9860f6d893dd3de53. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:38.942934 16132 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 08b7d1ab519044eda2d5e04a07242f25. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:38.934242 16213 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5718c623ebee4d2bb770f0f9ddb2f164. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:38.951016 16289 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9e4bfb9090c8460eb14fa7e0d818e7f1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:38.951519 16151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cc959fb7f704fba9009c0595467d074" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
W20260709 13:18:38.953905 16191 leader_election.cc:343] T 8cc959fb7f704fba9009c0595467d074 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:18:38.955299 16344 leader_election.cc:290] T 8a878999c0024d2abe517484180290fc P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:18:38.954059 16133 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c4f65cdf00b748bb963ee6e81ef43b79. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:38.959771 16347 tablet_bootstrap.cc:492] T c2f5f89186204242bf377f7c42b2e2da P 1ca2725d290e47d68b1a4b609c9816ac: No bootstrap required, opened a new log
I20260709 13:18:38.965095 16290 tablet_service.cc:1511] Processing CreateTablet for tablet 84f2da98864b4425bf7c0055951fdd4b (DEFAULT_TABLE table=test-workload [id=ed3d15d8c9ad46bebc4fe3a402d2e54f]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260709 13:18:38.965339 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a878999c0024d2abe517484180290fc" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
W20260709 13:18:38.966872 16116 leader_election.cc:343] T 8a878999c0024d2abe517484180290fc P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:18:38.967232 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a878999c0024d2abe517484180290fc" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:18:38.968159 16290 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 84f2da98864b4425bf7c0055951fdd4b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:38.960887 16342 raft_consensus.cc:359] T 5a96373bb0e14dce8a266b2fc6c58507 P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:18:38.968436 16115 leader_election.cc:343] T 8a878999c0024d2abe517484180290fc P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:18:38.968663 16342 raft_consensus.cc:385] T 5a96373bb0e14dce8a266b2fc6c58507 P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:38.969787 16342 raft_consensus.cc:740] T 5a96373bb0e14dce8a266b2fc6c58507 P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Initialized, Role: FOLLOWER
I20260709 13:18:38.969676 16115 leader_election.cc:304] T 8a878999c0024d2abe517484180290fc P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:18:38.970932 16344 raft_consensus.cc:2749] T 8a878999c0024d2abe517484180290fc P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:18:38.972208 16347 ts_tablet_manager.cc:1403] T c2f5f89186204242bf377f7c42b2e2da P 1ca2725d290e47d68b1a4b609c9816ac: Time spent bootstrapping tablet: real 0.310s	user 0.013s	sys 0.015s
I20260709 13:18:38.972970 16342 consensus_queue.cc:260] T 5a96373bb0e14dce8a266b2fc6c58507 P 90f3da57796b488c9087a2f523adf045 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:38.966706 16136 tablet_service.cc:1511] Processing CreateTablet for tablet 8d4ab870a05347868956aa8b841b2981 (DEFAULT_TABLE table=test-workload [id=ed3d15d8c9ad46bebc4fe3a402d2e54f]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260709 13:18:38.968155 16340 tablet_bootstrap.cc:654] T 434bff3082d141838068f323bdb0d1ed P 647a343a542d4a139fe106c19095129b: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:38.977200 16136 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8d4ab870a05347868956aa8b841b2981. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:38.975121 16342 ts_tablet_manager.cc:1434] T 5a96373bb0e14dce8a266b2fc6c58507 P 90f3da57796b488c9087a2f523adf045: Time spent starting tablet: real 0.043s	user 0.004s	sys 0.001s
I20260709 13:18:38.974840 16347 raft_consensus.cc:359] T c2f5f89186204242bf377f7c42b2e2da P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:38.983814 16347 raft_consensus.cc:385] T c2f5f89186204242bf377f7c42b2e2da P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:38.986191 16347 raft_consensus.cc:740] T c2f5f89186204242bf377f7c42b2e2da P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Initialized, Role: FOLLOWER
I20260709 13:18:38.995330 16347 consensus_queue.cc:260] T c2f5f89186204242bf377f7c42b2e2da P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:38.999094 16342 tablet_bootstrap.cc:492] T 7c2095a5f4ad49408dbc7f69c606120d P 90f3da57796b488c9087a2f523adf045: Bootstrap starting.
I20260709 13:18:39.008188 16286 tablet_service.cc:1511] Processing CreateTablet for tablet 66f75124215f4ca6bef831e174a792d8 (DEFAULT_TABLE table=test-workload [id=ed3d15d8c9ad46bebc4fe3a402d2e54f]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260709 13:18:39.002135 16347 ts_tablet_manager.cc:1434] T c2f5f89186204242bf377f7c42b2e2da P 1ca2725d290e47d68b1a4b609c9816ac: Time spent starting tablet: real 0.030s	user 0.004s	sys 0.004s
I20260709 13:18:39.017205 16358 raft_consensus.cc:493] T c2f5f89186204242bf377f7c42b2e2da P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:39.024549 16347 tablet_bootstrap.cc:492] T 8a878999c0024d2abe517484180290fc P 1ca2725d290e47d68b1a4b609c9816ac: Bootstrap starting.
I20260709 13:18:39.025741 16358 raft_consensus.cc:515] T c2f5f89186204242bf377f7c42b2e2da P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:39.026814 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cc959fb7f704fba9009c0595467d074" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
W20260709 13:18:39.035211 16190 leader_election.cc:343] T 8cc959fb7f704fba9009c0595467d074 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:18:39.035699 16190 leader_election.cc:304] T 8cc959fb7f704fba9009c0595467d074 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:18:39.027514 16286 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 66f75124215f4ca6bef831e174a792d8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:39.036939 16151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f5f89186204242bf377f7c42b2e2da" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:18:39.037551 16151 raft_consensus.cc:2468] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 0.
I20260709 13:18:39.028867 16281 tablet_service.cc:1511] Processing CreateTablet for tablet 23f82224f4564fea93572e8206bad1fe (DEFAULT_TABLE table=test-workload [id=ed3d15d8c9ad46bebc4fe3a402d2e54f]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 13:18:39.038902 16191 leader_election.cc:304] T c2f5f89186204242bf377f7c42b2e2da P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
I20260709 13:18:39.035718 16358 leader_election.cc:290] T c2f5f89186204242bf377f7c42b2e2da P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:18:39.045819 16358 raft_consensus.cc:2749] T 8cc959fb7f704fba9009c0595467d074 P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:18:39.048547 16281 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 23f82224f4564fea93572e8206bad1fe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:39.048364 16358 raft_consensus.cc:2804] T c2f5f89186204242bf377f7c42b2e2da P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:18:39.056560 16358 raft_consensus.cc:493] T c2f5f89186204242bf377f7c42b2e2da P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:39.057046 16210 tablet_service.cc:1511] Processing CreateTablet for tablet 806d7690c2b644959152074bb2cab0a7 (DEFAULT_TABLE table=test-workload [id=ed3d15d8c9ad46bebc4fe3a402d2e54f]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 13:18:39.070442 16358 raft_consensus.cc:3060] T c2f5f89186204242bf377f7c42b2e2da P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:39.077472 16210 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 806d7690c2b644959152074bb2cab0a7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:39.081694 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f5f89186204242bf377f7c42b2e2da" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
W20260709 13:18:39.083212 16190 leader_election.cc:343] T c2f5f89186204242bf377f7c42b2e2da P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:18:39.085098 16211 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:57598 (request call id 23) took 1014 ms. Trace:
I20260709 13:18:39.075423 16212 tablet_service.cc:1511] Processing CreateTablet for tablet 0a9de347c711477e87c582d003f36925 (DEFAULT_TABLE table=test-workload [id=ed3d15d8c9ad46bebc4fe3a402d2e54f]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 13:18:39.085623 16211 rpcz_store.cc:276] 0709 13:18:38.070962 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:38.664855 (+593893us) service_pool.cc:224] Handling call
0709 13:18:38.695524 (+ 30669us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 13:18:38.695697 (+   173us) ts_tablet_manager.cc:682] Creating new metadata...
0709 13:18:38.883519 (+187822us) tablet_metadata.cc:665] Metadata flushed
0709 13:18:39.084607 (+201088us) tablet_replica.cc:206] Creating consensus instance
0709 13:18:39.085035 (+   428us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:39.105871 16211 tablet_service.cc:1511] Processing CreateTablet for tablet 2794af5b99984efebdee564035841cb0 (DEFAULT_TABLE table=test-workload [id=ed3d15d8c9ad46bebc4fe3a402d2e54f]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260709 13:18:39.108690 16135 tablet_service.cc:1511] Processing CreateTablet for tablet cdbd78acb4cf46298e0de8ba0d130e19 (DEFAULT_TABLE table=test-workload [id=ed3d15d8c9ad46bebc4fe3a402d2e54f]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 13:18:39.110383 16135 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cdbd78acb4cf46298e0de8ba0d130e19. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:39.120224 16345 raft_consensus.cc:493] T 5a96373bb0e14dce8a266b2fc6c58507 P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:39.120949 16345 raft_consensus.cc:515] T 5a96373bb0e14dce8a266b2fc6c58507 P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:39.122750 16342 tablet_bootstrap.cc:654] T 7c2095a5f4ad49408dbc7f69c606120d P 90f3da57796b488c9087a2f523adf045: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:39.124603 16151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a96373bb0e14dce8a266b2fc6c58507" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:18:39.127394 16345 leader_election.cc:290] T 5a96373bb0e14dce8a266b2fc6c58507 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:18:39.127772 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a96373bb0e14dce8a266b2fc6c58507" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
W20260709 13:18:39.128360 16265 leader_election.cc:343] T 5a96373bb0e14dce8a266b2fc6c58507 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:18:39.129228 16265 leader_election.cc:343] T 5a96373bb0e14dce8a266b2fc6c58507 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:18:39.129634 16265 leader_election.cc:304] T 5a96373bb0e14dce8a266b2fc6c58507 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:18:39.132582 16345 raft_consensus.cc:2749] T 5a96373bb0e14dce8a266b2fc6c58507 P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:18:39.120550 16212 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0a9de347c711477e87c582d003f36925. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:39.143672 16207 tablet_service.cc:1511] Processing CreateTablet for tablet ee9ed67006144797b1de6ea8c2013a68 (DEFAULT_TABLE table=test-workload [id=ed3d15d8c9ad46bebc4fe3a402d2e54f]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 13:18:39.145535 16209 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:57598 (request call id 22) took 1075 ms. Trace:
I20260709 13:18:39.146251 16134 tablet_service.cc:1511] Processing CreateTablet for tablet 57a744f5aaf944678a63469cbf514015 (DEFAULT_TABLE table=test-workload [id=ed3d15d8c9ad46bebc4fe3a402d2e54f]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260709 13:18:39.147089 16208 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:57598 (request call id 27) took 1043 ms. Trace:
I20260709 13:18:39.147598 16134 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 57a744f5aaf944678a63469cbf514015. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:39.147828 16207 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ee9ed67006144797b1de6ea8c2013a68. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:39.147548 16208 rpcz_store.cc:276] 0709 13:18:38.103498 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:38.757558 (+654060us) service_pool.cc:224] Handling call
0709 13:18:38.763371 (+  5813us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 13:18:38.764128 (+   757us) ts_tablet_manager.cc:682] Creating new metadata...
0709 13:18:38.842228 (+ 78100us) tablet_metadata.cc:665] Metadata flushed
0709 13:18:39.146472 (+304244us) tablet_replica.cc:206] Creating consensus instance
0709 13:18:39.147029 (+   557us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":35345}
I20260709 13:18:39.152158 16215 tablet_service.cc:1511] Processing CreateTablet for tablet 31969392ccdb43b491911a319e8bc7e1 (DEFAULT_TABLE table=test-workload [id=ed3d15d8c9ad46bebc4fe3a402d2e54f]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260709 13:18:39.153761 16215 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 31969392ccdb43b491911a319e8bc7e1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:39.155311 16208 tablet_service.cc:1511] Processing CreateTablet for tablet e4dc6b0541ca431ebc757beac00521f1 (DEFAULT_TABLE table=test-workload [id=ed3d15d8c9ad46bebc4fe3a402d2e54f]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260709 13:18:39.159261 16138 tablet_service.cc:1511] Processing CreateTablet for tablet e5ad4eb281564936a59f41e387a47156 (DEFAULT_TABLE table=test-workload [id=ed3d15d8c9ad46bebc4fe3a402d2e54f]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 13:18:39.145939 16209 rpcz_store.cc:276] 0709 13:18:38.070600 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:38.660255 (+589655us) service_pool.cc:224] Handling call
0709 13:18:38.662007 (+  1752us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 13:18:38.662152 (+   145us) ts_tablet_manager.cc:682] Creating new metadata...
0709 13:18:38.780529 (+118377us) tablet_metadata.cc:665] Metadata flushed
0709 13:18:39.145063 (+364534us) tablet_replica.cc:206] Creating consensus instance
0709 13:18:39.145478 (+   415us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":12447}
I20260709 13:18:39.160151 16213 tablet_service.cc:1511] Processing CreateTablet for tablet 8158bbd17b994d2eb96f560bd0f1b88d (DEFAULT_TABLE table=test-workload [id=ed3d15d8c9ad46bebc4fe3a402d2e54f]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260709 13:18:39.166352 16138 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e5ad4eb281564936a59f41e387a47156. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:39.167594 16340 tablet_bootstrap.cc:492] T 434bff3082d141838068f323bdb0d1ed P 647a343a542d4a139fe106c19095129b: No bootstrap required, opened a new log
I20260709 13:18:39.168489 16347 tablet_bootstrap.cc:654] T 8a878999c0024d2abe517484180290fc P 1ca2725d290e47d68b1a4b609c9816ac: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:39.170830 16137 tablet_service.cc:1511] Processing CreateTablet for tablet c9944d5e4b8c4109af9955c4f08c9095 (DEFAULT_TABLE table=test-workload [id=ed3d15d8c9ad46bebc4fe3a402d2e54f]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260709 13:18:39.171047 16132 tablet_service.cc:1511] Processing CreateTablet for tablet 9ad5dbfa99ee4b8d9e28be92b3f36db5 (DEFAULT_TABLE table=test-workload [id=ed3d15d8c9ad46bebc4fe3a402d2e54f]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 13:18:39.176462 16340 ts_tablet_manager.cc:1403] T 434bff3082d141838068f323bdb0d1ed P 647a343a542d4a139fe106c19095129b: Time spent bootstrapping tablet: real 0.301s	user 0.033s	sys 0.038s
W20260709 13:18:39.183017 16116 consensus_peers.cc:597] T 18ad7e3af5504571a69d10c00750a01c P 647a343a542d4a139fe106c19095129b -> Peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943): Couldn't send request to peer 1ca2725d290e47d68b1a4b609c9816ac. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:18:39.182430 16340 raft_consensus.cc:359] T 434bff3082d141838068f323bdb0d1ed P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:39.183722 16340 raft_consensus.cc:385] T 434bff3082d141838068f323bdb0d1ed P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:39.187811 16340 raft_consensus.cc:740] T 434bff3082d141838068f323bdb0d1ed P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Initialized, Role: FOLLOWER
I20260709 13:18:39.188558 16340 consensus_queue.cc:260] T 434bff3082d141838068f323bdb0d1ed P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:39.188937 16209 tablet_service.cc:1511] Processing CreateTablet for tablet 8638ee363273426db1393dd81e5c7b72 (DEFAULT_TABLE table=test-workload [id=ed3d15d8c9ad46bebc4fe3a402d2e54f]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260709 13:18:39.190802 16340 ts_tablet_manager.cc:1434] T 434bff3082d141838068f323bdb0d1ed P 647a343a542d4a139fe106c19095129b: Time spent starting tablet: real 0.014s	user 0.005s	sys 0.000s
I20260709 13:18:39.191623 16214 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:57598 (request call id 29) took 1034 ms. Trace:
I20260709 13:18:39.192010 16214 rpcz_store.cc:276] 0709 13:18:38.157364 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:38.778844 (+621480us) service_pool.cc:224] Handling call
0709 13:18:38.780598 (+  1754us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 13:18:38.780700 (+   102us) ts_tablet_manager.cc:682] Creating new metadata...
0709 13:18:39.085564 (+304864us) tablet_metadata.cc:665] Metadata flushed
0709 13:18:39.191104 (+105540us) tablet_replica.cc:206] Creating consensus instance
0709 13:18:39.191564 (+   460us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:39.194025 16214 tablet_service.cc:1511] Processing CreateTablet for tablet c4f65cdf00b748bb963ee6e81ef43b79 (DEFAULT_TABLE table=test-workload [id=ed3d15d8c9ad46bebc4fe3a402d2e54f]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 13:18:39.179039 16211 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2794af5b99984efebdee564035841cb0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:39.200937 16132 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9ad5dbfa99ee4b8d9e28be92b3f36db5. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:18:39.201902 16097 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:18:39.202912 16347 tablet_bootstrap.cc:492] T 8a878999c0024d2abe517484180290fc P 1ca2725d290e47d68b1a4b609c9816ac: No bootstrap required, opened a new log
I20260709 13:18:39.203451 16347 ts_tablet_manager.cc:1403] T 8a878999c0024d2abe517484180290fc P 1ca2725d290e47d68b1a4b609c9816ac: Time spent bootstrapping tablet: real 0.182s	user 0.009s	sys 0.011s
I20260709 13:18:39.238830 16213 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8158bbd17b994d2eb96f560bd0f1b88d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:39.240316 16208 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e4dc6b0541ca431ebc757beac00521f1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:39.243424 16209 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8638ee363273426db1393dd81e5c7b72. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:39.244213 16214 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c4f65cdf00b748bb963ee6e81ef43b79. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:39.152158 16140 tablet_service.cc:1511] Processing CreateTablet for tablet 26354361e9b044cea1c2b86c4882fe92 (DEFAULT_TABLE table=test-workload [id=ed3d15d8c9ad46bebc4fe3a402d2e54f]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260709 13:18:39.252923 16140 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 26354361e9b044cea1c2b86c4882fe92. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:39.256333 16342 tablet_bootstrap.cc:492] T 7c2095a5f4ad49408dbc7f69c606120d P 90f3da57796b488c9087a2f523adf045: No bootstrap required, opened a new log
I20260709 13:18:39.256102 16344 consensus_queue.cc:1048] T 18ad7e3af5504571a69d10c00750a01c P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:18:39.256425 16141 tablet_service.cc:1511] Processing CreateTablet for tablet 19baa7bc367844c288bca8f81439c413 (DEFAULT_TABLE table=test-workload [id=ed3d15d8c9ad46bebc4fe3a402d2e54f]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260709 13:18:39.257771 16342 ts_tablet_manager.cc:1403] T 7c2095a5f4ad49408dbc7f69c606120d P 90f3da57796b488c9087a2f523adf045: Time spent bootstrapping tablet: real 0.266s	user 0.008s	sys 0.027s
I20260709 13:18:39.260890 16342 raft_consensus.cc:359] T 7c2095a5f4ad49408dbc7f69c606120d P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:39.261605 16342 raft_consensus.cc:385] T 7c2095a5f4ad49408dbc7f69c606120d P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:39.261888 16342 raft_consensus.cc:740] T 7c2095a5f4ad49408dbc7f69c606120d P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Initialized, Role: FOLLOWER
I20260709 13:18:39.264019 16139 tablet_service.cc:1511] Processing CreateTablet for tablet 9e4bfb9090c8460eb14fa7e0d818e7f1 (DEFAULT_TABLE table=test-workload [id=ed3d15d8c9ad46bebc4fe3a402d2e54f]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260709 13:18:39.266878 16340 tablet_bootstrap.cc:492] T 8cc959fb7f704fba9009c0595467d074 P 647a343a542d4a139fe106c19095129b: Bootstrap starting.
I20260709 13:18:39.267277 16347 raft_consensus.cc:359] T 8a878999c0024d2abe517484180290fc P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:39.272169 16347 raft_consensus.cc:385] T 8a878999c0024d2abe517484180290fc P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:39.273059 16347 raft_consensus.cc:740] T 8a878999c0024d2abe517484180290fc P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Initialized, Role: FOLLOWER
I20260709 13:18:39.264318 16216 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:57598 (request call id 28) took 1160 ms. Trace:
I20260709 13:18:39.275295 16216 rpcz_store.cc:276] 0709 13:18:38.133807 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:38.698130 (+564323us) service_pool.cc:224] Handling call
0709 13:18:38.732509 (+ 34379us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 13:18:38.743030 (+ 10521us) ts_tablet_manager.cc:682] Creating new metadata...
0709 13:18:39.089664 (+346634us) tablet_metadata.cc:665] Metadata flushed
0709 13:18:39.263814 (+174150us) tablet_replica.cc:206] Creating consensus instance
0709 13:18:39.264259 (+   445us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:39.277894 16216 tablet_service.cc:1511] Processing CreateTablet for tablet 08c74490c40243a88e62d1760b5cd814 (DEFAULT_TABLE table=test-workload [id=ed3d15d8c9ad46bebc4fe3a402d2e54f]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 13:18:39.278255 16136 tablet_service.cc:1511] Processing CreateTablet for tablet 66f75124215f4ca6bef831e174a792d8 (DEFAULT_TABLE table=test-workload [id=ed3d15d8c9ad46bebc4fe3a402d2e54f]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260709 13:18:39.275848 16347 consensus_queue.cc:260] T 8a878999c0024d2abe517484180290fc P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:39.281589 16213 tablet_service.cc:1511] Processing CreateTablet for tablet 08b7d1ab519044eda2d5e04a07242f25 (DEFAULT_TABLE table=test-workload [id=ed3d15d8c9ad46bebc4fe3a402d2e54f]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 13:18:39.285739 16207 tablet_service.cc:1511] Processing CreateTablet for tablet ecc1e363230043fe9fef390f0b979f20 (DEFAULT_TABLE table=test-workload [id=ed3d15d8c9ad46bebc4fe3a402d2e54f]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260709 13:18:39.279546 16133 tablet_service.cc:1511] Processing CreateTablet for tablet 84f2da98864b4425bf7c0055951fdd4b (DEFAULT_TABLE table=test-workload [id=ed3d15d8c9ad46bebc4fe3a402d2e54f]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260709 13:18:39.289216 16347 ts_tablet_manager.cc:1434] T 8a878999c0024d2abe517484180290fc P 1ca2725d290e47d68b1a4b609c9816ac: Time spent starting tablet: real 0.085s	user 0.009s	sys 0.000s
I20260709 13:18:39.257822 16141 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 19baa7bc367844c288bca8f81439c413. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:39.296248 16137 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c9944d5e4b8c4109af9955c4f08c9095. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:39.268554 16358 raft_consensus.cc:515] T c2f5f89186204242bf377f7c42b2e2da P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:39.295207 16216 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 08c74490c40243a88e62d1760b5cd814. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:39.300166 16213 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 08b7d1ab519044eda2d5e04a07242f25. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:39.301509 16136 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 66f75124215f4ca6bef831e174a792d8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:39.304751 16207 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ecc1e363230043fe9fef390f0b979f20. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:39.304481 16139 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9e4bfb9090c8460eb14fa7e0d818e7f1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:39.262660 16342 consensus_queue.cc:260] T 7c2095a5f4ad49408dbc7f69c606120d P 90f3da57796b488c9087a2f523adf045 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:39.313097 16347 tablet_bootstrap.cc:492] T 9e65676b21b748728ad39668cfe722ec P 1ca2725d290e47d68b1a4b609c9816ac: Bootstrap starting.
I20260709 13:18:39.313230 16342 ts_tablet_manager.cc:1434] T 7c2095a5f4ad49408dbc7f69c606120d P 90f3da57796b488c9087a2f523adf045: Time spent starting tablet: real 0.055s	user 0.003s	sys 0.003s
I20260709 13:18:39.344225 16342 tablet_bootstrap.cc:492] T 8a878999c0024d2abe517484180290fc P 90f3da57796b488c9087a2f523adf045: Bootstrap starting.
I20260709 13:18:39.315066 16132 tablet_service.cc:1511] Processing CreateTablet for tablet 23f82224f4564fea93572e8206bad1fe (DEFAULT_TABLE table=test-workload [id=ed3d15d8c9ad46bebc4fe3a402d2e54f]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 13:18:39.319516 16340 tablet_bootstrap.cc:654] T 8cc959fb7f704fba9009c0595467d074 P 647a343a542d4a139fe106c19095129b: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:39.321142 16133 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 84f2da98864b4425bf7c0055951fdd4b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:39.348645 16132 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 23f82224f4564fea93572e8206bad1fe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:39.326364 16212 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:57598 (request call id 35) took 1093 ms. Trace:
I20260709 13:18:39.349133 16212 rpcz_store.cc:276] 0709 13:18:38.233490 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:39.023565 (+790075us) service_pool.cc:224] Handling call
0709 13:18:39.087672 (+ 64107us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 13:18:39.087773 (+   101us) ts_tablet_manager.cc:682] Creating new metadata...
0709 13:18:39.274854 (+187081us) tablet_metadata.cc:665] Metadata flushed
0709 13:18:39.325865 (+ 51011us) tablet_replica.cc:206] Creating consensus instance
0709 13:18:39.326310 (+   445us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:39.351097 16212 tablet_service.cc:1511] Processing CreateTablet for tablet 8d4ab870a05347868956aa8b841b2981 (DEFAULT_TABLE table=test-workload [id=ed3d15d8c9ad46bebc4fe3a402d2e54f]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260709 13:18:39.352344 16212 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8d4ab870a05347868956aa8b841b2981. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:39.358085 16209 tablet_service.cc:1511] Processing CreateTablet for tablet cdbd78acb4cf46298e0de8ba0d130e19 (DEFAULT_TABLE table=test-workload [id=ed3d15d8c9ad46bebc4fe3a402d2e54f]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 13:18:39.328624 16210 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:57598 (request call id 33) took 1114 ms. Trace:
I20260709 13:18:39.359324 16210 rpcz_store.cc:276] 0709 13:18:38.213834 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:38.977873 (+764039us) service_pool.cc:224] Handling call
0709 13:18:39.077017 (+ 99144us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 13:18:39.077113 (+    96us) ts_tablet_manager.cc:682] Creating new metadata...
0709 13:18:39.270359 (+193246us) tablet_metadata.cc:665] Metadata flushed
0709 13:18:39.327475 (+ 57116us) tablet_replica.cc:206] Creating consensus instance
0709 13:18:39.328568 (+  1093us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:39.362084 16210 tablet_service.cc:1511] Processing CreateTablet for tablet 57a744f5aaf944678a63469cbf514015 (DEFAULT_TABLE table=test-workload [id=ed3d15d8c9ad46bebc4fe3a402d2e54f]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260709 13:18:39.329782 16214 tablet_service.cc:1511] Processing CreateTablet for tablet 1df93779209748b9860f6d893dd3de53 (DEFAULT_TABLE table=test-workload [id=ed3d15d8c9ad46bebc4fe3a402d2e54f]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 13:18:39.359581 16209 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cdbd78acb4cf46298e0de8ba0d130e19. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:39.364359 16210 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 57a744f5aaf944678a63469cbf514015. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:39.367313 16342 tablet_bootstrap.cc:654] T 8a878999c0024d2abe517484180290fc P 90f3da57796b488c9087a2f523adf045: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:39.367444 16214 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1df93779209748b9860f6d893dd3de53. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:39.333582 16211 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:57598 (request call id 36) took 1075 ms. Trace:
I20260709 13:18:39.341396 16151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f5f89186204242bf377f7c42b2e2da" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:18:39.370833 16151 raft_consensus.cc:3060] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:39.370337 16211 rpcz_store.cc:276] 0709 13:18:38.257841 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:39.104703 (+846862us) service_pool.cc:224] Handling call
0709 13:18:39.108090 (+  3387us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 13:18:39.108245 (+   155us) ts_tablet_manager.cc:682] Creating new metadata...
0709 13:18:39.298560 (+190315us) tablet_metadata.cc:665] Metadata flushed
0709 13:18:39.332972 (+ 34412us) tablet_replica.cc:206] Creating consensus instance
0709 13:18:39.333507 (+   535us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:39.378039 16345 raft_consensus.cc:493] T 7c2095a5f4ad49408dbc7f69c606120d P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:39.378588 16345 raft_consensus.cc:515] T 7c2095a5f4ad49408dbc7f69c606120d P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:39.380019 16151 raft_consensus.cc:2468] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 1.
I20260709 13:18:39.382387 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c2095a5f4ad49408dbc7f69c606120d" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:18:39.380654 16345 leader_election.cc:290] T 7c2095a5f4ad49408dbc7f69c606120d P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:18:39.381278 16347 tablet_bootstrap.cc:654] T 9e65676b21b748728ad39668cfe722ec P 1ca2725d290e47d68b1a4b609c9816ac: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:39.382953 16191 leader_election.cc:304] T c2f5f89186204242bf377f7c42b2e2da P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
I20260709 13:18:39.388916 16366 raft_consensus.cc:2804] T c2f5f89186204242bf377f7c42b2e2da P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Leader election won for term 1
W20260709 13:18:39.387667 16265 leader_election.cc:343] T 7c2095a5f4ad49408dbc7f69c606120d P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:18:39.389225 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c2095a5f4ad49408dbc7f69c606120d" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:18:39.377709 16211 tablet_service.cc:1511] Processing CreateTablet for tablet e5ad4eb281564936a59f41e387a47156 (DEFAULT_TABLE table=test-workload [id=ed3d15d8c9ad46bebc4fe3a402d2e54f]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 13:18:39.396886 16211 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e5ad4eb281564936a59f41e387a47156. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:39.397392 16216 tablet_service.cc:1511] Processing CreateTablet for tablet c9944d5e4b8c4109af9955c4f08c9095 (DEFAULT_TABLE table=test-workload [id=ed3d15d8c9ad46bebc4fe3a402d2e54f]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
W20260709 13:18:39.397742 16265 leader_election.cc:343] T 7c2095a5f4ad49408dbc7f69c606120d P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:18:39.398607 16216 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c9944d5e4b8c4109af9955c4f08c9095. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:39.398371 16265 leader_election.cc:304] T 7c2095a5f4ad49408dbc7f69c606120d P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:18:39.399381 16345 raft_consensus.cc:2749] T 7c2095a5f4ad49408dbc7f69c606120d P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:18:39.406692 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f5f89186204242bf377f7c42b2e2da" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
W20260709 13:18:39.408237 16190 leader_election.cc:343] T c2f5f89186204242bf377f7c42b2e2da P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:18:39.425254 16342 tablet_bootstrap.cc:492] T 8a878999c0024d2abe517484180290fc P 90f3da57796b488c9087a2f523adf045: No bootstrap required, opened a new log
I20260709 13:18:39.425964 16342 ts_tablet_manager.cc:1403] T 8a878999c0024d2abe517484180290fc P 90f3da57796b488c9087a2f523adf045: Time spent bootstrapping tablet: real 0.082s	user 0.012s	sys 0.019s
I20260709 13:18:39.433837 16207 tablet_service.cc:1511] Processing CreateTablet for tablet 9ad5dbfa99ee4b8d9e28be92b3f36db5 (DEFAULT_TABLE table=test-workload [id=ed3d15d8c9ad46bebc4fe3a402d2e54f]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 13:18:39.377709 16213 tablet_service.cc:1511] Processing CreateTablet for tablet 26354361e9b044cea1c2b86c4882fe92 (DEFAULT_TABLE table=test-workload [id=ed3d15d8c9ad46bebc4fe3a402d2e54f]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260709 13:18:39.405308 16358 leader_election.cc:290] T c2f5f89186204242bf377f7c42b2e2da P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 1 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:18:39.440986 16207 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9ad5dbfa99ee4b8d9e28be92b3f36db5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:39.438447 16215 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:57598 (request call id 40) took 1066 ms. Trace:
I20260709 13:18:39.438594 16366 raft_consensus.cc:697] T c2f5f89186204242bf377f7c42b2e2da P 1ca2725d290e47d68b1a4b609c9816ac [term 1 LEADER]: Becoming Leader. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:18:39.463101 16213 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 26354361e9b044cea1c2b86c4882fe92. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:39.473093 16340 tablet_bootstrap.cc:492] T 8cc959fb7f704fba9009c0595467d074 P 647a343a542d4a139fe106c19095129b: No bootstrap required, opened a new log
I20260709 13:18:39.473639 16340 ts_tablet_manager.cc:1403] T 8cc959fb7f704fba9009c0595467d074 P 647a343a542d4a139fe106c19095129b: Time spent bootstrapping tablet: real 0.234s	user 0.021s	sys 0.008s
I20260709 13:18:39.452612 16215 rpcz_store.cc:276] 0709 13:18:38.371637 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:39.151364 (+779727us) service_pool.cc:224] Handling call
0709 13:18:39.153269 (+  1905us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 13:18:39.153356 (+    87us) ts_tablet_manager.cc:682] Creating new metadata...
0709 13:18:39.287528 (+134172us) tablet_metadata.cc:665] Metadata flushed
0709 13:18:39.410994 (+123466us) tablet_replica.cc:206] Creating consensus instance
0709 13:18:39.437503 (+ 26509us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:39.477029 16340 raft_consensus.cc:359] T 8cc959fb7f704fba9009c0595467d074 P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:39.477233 16215 tablet_service.cc:1511] Processing CreateTablet for tablet 9e4bfb9090c8460eb14fa7e0d818e7f1 (DEFAULT_TABLE table=test-workload [id=ed3d15d8c9ad46bebc4fe3a402d2e54f]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260709 13:18:39.477864 16340 raft_consensus.cc:385] T 8cc959fb7f704fba9009c0595467d074 P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:39.478210 16340 raft_consensus.cc:740] T 8cc959fb7f704fba9009c0595467d074 P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Initialized, Role: FOLLOWER
I20260709 13:18:39.478636 16215 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9e4bfb9090c8460eb14fa7e0d818e7f1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:39.480358 16214 tablet_service.cc:1511] Processing CreateTablet for tablet 19baa7bc367844c288bca8f81439c413 (DEFAULT_TABLE table=test-workload [id=ed3d15d8c9ad46bebc4fe3a402d2e54f]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260709 13:18:39.485459 16342 raft_consensus.cc:359] T 8a878999c0024d2abe517484180290fc P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:39.486255 16342 raft_consensus.cc:385] T 8a878999c0024d2abe517484180290fc P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:39.486574 16342 raft_consensus.cc:740] T 8a878999c0024d2abe517484180290fc P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Initialized, Role: FOLLOWER
I20260709 13:18:39.487425 16340 consensus_queue.cc:260] T 8cc959fb7f704fba9009c0595467d074 P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:39.487681 16342 consensus_queue.cc:260] T 8a878999c0024d2abe517484180290fc P 90f3da57796b488c9087a2f523adf045 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:39.488344 16214 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 19baa7bc367844c288bca8f81439c413. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:39.489967 16340 ts_tablet_manager.cc:1434] T 8cc959fb7f704fba9009c0595467d074 P 647a343a542d4a139fe106c19095129b: Time spent starting tablet: real 0.016s	user 0.006s	sys 0.001s
I20260709 13:18:39.492268 16208 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:57598 (request call id 39) took 1169 ms. Trace:
I20260709 13:18:39.492192 16209 tablet_service.cc:1511] Processing CreateTablet for tablet 66f75124215f4ca6bef831e174a792d8 (DEFAULT_TABLE table=test-workload [id=ed3d15d8c9ad46bebc4fe3a402d2e54f]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260709 13:18:39.492767 16208 rpcz_store.cc:276] 0709 13:18:38.323093 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:39.154243 (+831150us) service_pool.cc:224] Handling call
0709 13:18:39.163814 (+  9571us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 13:18:39.168989 (+  5175us) ts_tablet_manager.cc:682] Creating new metadata...
0709 13:18:39.290957 (+121968us) tablet_metadata.cc:665] Metadata flushed
0709 13:18:39.486322 (+195365us) tablet_replica.cc:206] Creating consensus instance
0709 13:18:39.492211 (+  5889us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:39.493584 16216 tablet_service.cc:1511] Processing CreateTablet for tablet 23f82224f4564fea93572e8206bad1fe (DEFAULT_TABLE table=test-workload [id=ed3d15d8c9ad46bebc4fe3a402d2e54f]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 13:18:39.496308 16340 tablet_bootstrap.cc:492] T 9e65676b21b748728ad39668cfe722ec P 647a343a542d4a139fe106c19095129b: Bootstrap starting.
I20260709 13:18:39.493666 16209 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 66f75124215f4ca6bef831e174a792d8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:39.494239 16212 tablet_service.cc:1511] Processing CreateTablet for tablet 84f2da98864b4425bf7c0055951fdd4b (DEFAULT_TABLE table=test-workload [id=ed3d15d8c9ad46bebc4fe3a402d2e54f]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260709 13:18:39.499207 16216 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 23f82224f4564fea93572e8206bad1fe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:39.493949 16366 consensus_queue.cc:237] T c2f5f89186204242bf377f7c42b2e2da P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:39.500420 16212 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 84f2da98864b4425bf7c0055951fdd4b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:39.500381 16347 tablet_bootstrap.cc:492] T 9e65676b21b748728ad39668cfe722ec P 1ca2725d290e47d68b1a4b609c9816ac: No bootstrap required, opened a new log
I20260709 13:18:39.507004 16347 ts_tablet_manager.cc:1403] T 9e65676b21b748728ad39668cfe722ec P 1ca2725d290e47d68b1a4b609c9816ac: Time spent bootstrapping tablet: real 0.194s	user 0.020s	sys 0.029s
I20260709 13:18:39.513947 16347 raft_consensus.cc:359] T 9e65676b21b748728ad39668cfe722ec P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:39.514719 16347 raft_consensus.cc:385] T 9e65676b21b748728ad39668cfe722ec P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:39.515035 16347 raft_consensus.cc:740] T 9e65676b21b748728ad39668cfe722ec P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Initialized, Role: FOLLOWER
I20260709 13:18:39.515897 16347 consensus_queue.cc:260] T 9e65676b21b748728ad39668cfe722ec P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:39.524259 16347 ts_tablet_manager.cc:1434] T 9e65676b21b748728ad39668cfe722ec P 1ca2725d290e47d68b1a4b609c9816ac: Time spent starting tablet: real 0.017s	user 0.006s	sys 0.000s
I20260709 13:18:39.524596 16342 ts_tablet_manager.cc:1434] T 8a878999c0024d2abe517484180290fc P 90f3da57796b488c9087a2f523adf045: Time spent starting tablet: real 0.098s	user 0.009s	sys 0.001s
I20260709 13:18:39.534705 16342 tablet_bootstrap.cc:492] T d2372981006d4b218215c28964dea623 P 90f3da57796b488c9087a2f523adf045: Bootstrap starting.
I20260709 13:18:39.525453 16340 tablet_bootstrap.cc:654] T 9e65676b21b748728ad39668cfe722ec P 647a343a542d4a139fe106c19095129b: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:39.541857 16342 tablet_bootstrap.cc:654] T d2372981006d4b218215c28964dea623 P 90f3da57796b488c9087a2f523adf045: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:39.525307 16347 tablet_bootstrap.cc:492] T 18ad7e3af5504571a69d10c00750a01c P 1ca2725d290e47d68b1a4b609c9816ac: Bootstrap starting.
I20260709 13:18:39.533504 16048 catalog_manager.cc:5697] T c2f5f89186204242bf377f7c42b2e2da P 1ca2725d290e47d68b1a4b609c9816ac reported cstate change: term changed from 0 to 1, leader changed from <none> to 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130). New cstate: current_term: 1 leader_uuid: "1ca2725d290e47d68b1a4b609c9816ac" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:39.548365 16347 tablet_bootstrap.cc:654] T 18ad7e3af5504571a69d10c00750a01c P 1ca2725d290e47d68b1a4b609c9816ac: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:39.548472 16340 tablet_bootstrap.cc:492] T 9e65676b21b748728ad39668cfe722ec P 647a343a542d4a139fe106c19095129b: No bootstrap required, opened a new log
I20260709 13:18:39.549170 16340 ts_tablet_manager.cc:1403] T 9e65676b21b748728ad39668cfe722ec P 647a343a542d4a139fe106c19095129b: Time spent bootstrapping tablet: real 0.053s	user 0.010s	sys 0.003s
I20260709 13:18:39.552840 16340 raft_consensus.cc:359] T 9e65676b21b748728ad39668cfe722ec P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:39.553587 16340 raft_consensus.cc:385] T 9e65676b21b748728ad39668cfe722ec P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:39.553722 16347 tablet_bootstrap.cc:492] T 18ad7e3af5504571a69d10c00750a01c P 1ca2725d290e47d68b1a4b609c9816ac: No bootstrap required, opened a new log
I20260709 13:18:39.553938 16340 raft_consensus.cc:740] T 9e65676b21b748728ad39668cfe722ec P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Initialized, Role: FOLLOWER
I20260709 13:18:39.554657 16347 ts_tablet_manager.cc:1403] T 18ad7e3af5504571a69d10c00750a01c P 1ca2725d290e47d68b1a4b609c9816ac: Time spent bootstrapping tablet: real 0.030s	user 0.011s	sys 0.000s
I20260709 13:18:39.558084 16340 consensus_queue.cc:260] T 9e65676b21b748728ad39668cfe722ec P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:39.558974 16347 raft_consensus.cc:359] T 18ad7e3af5504571a69d10c00750a01c P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } }
I20260709 13:18:39.560032 16347 raft_consensus.cc:385] T 18ad7e3af5504571a69d10c00750a01c P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:39.560423 16347 raft_consensus.cc:740] T 18ad7e3af5504571a69d10c00750a01c P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Initialized, Role: FOLLOWER
I20260709 13:18:39.560915 16340 ts_tablet_manager.cc:1434] T 9e65676b21b748728ad39668cfe722ec P 647a343a542d4a139fe106c19095129b: Time spent starting tablet: real 0.011s	user 0.003s	sys 0.005s
I20260709 13:18:39.561858 16340 tablet_bootstrap.cc:492] T 02838c86531d4f98bf58d1c22b8baf77 P 647a343a542d4a139fe106c19095129b: Bootstrap starting.
I20260709 13:18:39.561358 16347 consensus_queue.cc:260] T 18ad7e3af5504571a69d10c00750a01c P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } }
I20260709 13:18:39.562155 16342 tablet_bootstrap.cc:492] T d2372981006d4b218215c28964dea623 P 90f3da57796b488c9087a2f523adf045: No bootstrap required, opened a new log
I20260709 13:18:39.563261 16342 ts_tablet_manager.cc:1403] T d2372981006d4b218215c28964dea623 P 90f3da57796b488c9087a2f523adf045: Time spent bootstrapping tablet: real 0.029s	user 0.018s	sys 0.000s
I20260709 13:18:39.564194 16347 ts_tablet_manager.cc:1434] T 18ad7e3af5504571a69d10c00750a01c P 1ca2725d290e47d68b1a4b609c9816ac: Time spent starting tablet: real 0.009s	user 0.008s	sys 0.000s
I20260709 13:18:39.565305 16347 tablet_bootstrap.cc:492] T fffab26427e24d398e2695aad40ca134 P 1ca2725d290e47d68b1a4b609c9816ac: Bootstrap starting.
I20260709 13:18:39.566309 16342 raft_consensus.cc:359] T d2372981006d4b218215c28964dea623 P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:39.567392 16342 raft_consensus.cc:385] T d2372981006d4b218215c28964dea623 P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:39.567713 16342 raft_consensus.cc:740] T d2372981006d4b218215c28964dea623 P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Initialized, Role: FOLLOWER
I20260709 13:18:39.568287 16340 tablet_bootstrap.cc:654] T 02838c86531d4f98bf58d1c22b8baf77 P 647a343a542d4a139fe106c19095129b: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:39.572782 16347 tablet_bootstrap.cc:654] T fffab26427e24d398e2695aad40ca134 P 1ca2725d290e47d68b1a4b609c9816ac: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:39.580744 16342 consensus_queue.cc:260] T d2372981006d4b218215c28964dea623 P 90f3da57796b488c9087a2f523adf045 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:39.584203 16342 ts_tablet_manager.cc:1434] T d2372981006d4b218215c28964dea623 P 90f3da57796b488c9087a2f523adf045: Time spent starting tablet: real 0.021s	user 0.008s	sys 0.000s
I20260709 13:18:39.586184 16342 tablet_bootstrap.cc:492] T fffab26427e24d398e2695aad40ca134 P 90f3da57796b488c9087a2f523adf045: Bootstrap starting.
I20260709 13:18:39.598992 16342 tablet_bootstrap.cc:654] T fffab26427e24d398e2695aad40ca134 P 90f3da57796b488c9087a2f523adf045: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:39.605602 16347 tablet_bootstrap.cc:492] T fffab26427e24d398e2695aad40ca134 P 1ca2725d290e47d68b1a4b609c9816ac: No bootstrap required, opened a new log
I20260709 13:18:39.606117 16347 ts_tablet_manager.cc:1403] T fffab26427e24d398e2695aad40ca134 P 1ca2725d290e47d68b1a4b609c9816ac: Time spent bootstrapping tablet: real 0.041s	user 0.026s	sys 0.012s
I20260709 13:18:39.606384 16340 tablet_bootstrap.cc:492] T 02838c86531d4f98bf58d1c22b8baf77 P 647a343a542d4a139fe106c19095129b: No bootstrap required, opened a new log
I20260709 13:18:39.606868 16340 ts_tablet_manager.cc:1403] T 02838c86531d4f98bf58d1c22b8baf77 P 647a343a542d4a139fe106c19095129b: Time spent bootstrapping tablet: real 0.045s	user 0.022s	sys 0.021s
I20260709 13:18:39.606881 16342 tablet_bootstrap.cc:492] T fffab26427e24d398e2695aad40ca134 P 90f3da57796b488c9087a2f523adf045: No bootstrap required, opened a new log
I20260709 13:18:39.607514 16342 ts_tablet_manager.cc:1403] T fffab26427e24d398e2695aad40ca134 P 90f3da57796b488c9087a2f523adf045: Time spent bootstrapping tablet: real 0.022s	user 0.016s	sys 0.000s
I20260709 13:18:39.609166 16347 raft_consensus.cc:359] T fffab26427e24d398e2695aad40ca134 P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:39.609990 16347 raft_consensus.cc:385] T fffab26427e24d398e2695aad40ca134 P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:39.609850 16340 raft_consensus.cc:359] T 02838c86531d4f98bf58d1c22b8baf77 P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:39.610311 16347 raft_consensus.cc:740] T fffab26427e24d398e2695aad40ca134 P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Initialized, Role: FOLLOWER
I20260709 13:18:39.610119 16342 raft_consensus.cc:359] T fffab26427e24d398e2695aad40ca134 P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:39.610598 16340 raft_consensus.cc:385] T 02838c86531d4f98bf58d1c22b8baf77 P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:39.611074 16342 raft_consensus.cc:385] T fffab26427e24d398e2695aad40ca134 P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:39.611366 16340 raft_consensus.cc:740] T 02838c86531d4f98bf58d1c22b8baf77 P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Initialized, Role: FOLLOWER
I20260709 13:18:39.611296 16347 consensus_queue.cc:260] T fffab26427e24d398e2695aad40ca134 P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:39.611478 16342 raft_consensus.cc:740] T fffab26427e24d398e2695aad40ca134 P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Initialized, Role: FOLLOWER
I20260709 13:18:39.612440 16340 consensus_queue.cc:260] T 02838c86531d4f98bf58d1c22b8baf77 P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:39.612620 16342 consensus_queue.cc:260] T fffab26427e24d398e2695aad40ca134 P 90f3da57796b488c9087a2f523adf045 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:39.613492 16347 ts_tablet_manager.cc:1434] T fffab26427e24d398e2695aad40ca134 P 1ca2725d290e47d68b1a4b609c9816ac: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 13:18:39.614624 16347 tablet_bootstrap.cc:492] T 5a96373bb0e14dce8a266b2fc6c58507 P 1ca2725d290e47d68b1a4b609c9816ac: Bootstrap starting.
I20260709 13:18:39.614892 16340 ts_tablet_manager.cc:1434] T 02838c86531d4f98bf58d1c22b8baf77 P 647a343a542d4a139fe106c19095129b: Time spent starting tablet: real 0.008s	user 0.000s	sys 0.003s
I20260709 13:18:39.614905 16342 ts_tablet_manager.cc:1434] T fffab26427e24d398e2695aad40ca134 P 90f3da57796b488c9087a2f523adf045: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 13:18:39.616250 16342 tablet_bootstrap.cc:492] T 434bff3082d141838068f323bdb0d1ed P 90f3da57796b488c9087a2f523adf045: Bootstrap starting.
I20260709 13:18:39.616498 16340 tablet_bootstrap.cc:492] T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b: Bootstrap starting.
I20260709 13:18:39.621048 16347 tablet_bootstrap.cc:654] T 5a96373bb0e14dce8a266b2fc6c58507 P 1ca2725d290e47d68b1a4b609c9816ac: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:39.623415 16342 tablet_bootstrap.cc:654] T 434bff3082d141838068f323bdb0d1ed P 90f3da57796b488c9087a2f523adf045: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:39.623654 16340 tablet_bootstrap.cc:654] T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:39.632577 16224 raft_consensus.cc:3060] T 18ad7e3af5504571a69d10c00750a01c P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:39.638661 16342 tablet_bootstrap.cc:492] T 434bff3082d141838068f323bdb0d1ed P 90f3da57796b488c9087a2f523adf045: No bootstrap required, opened a new log
I20260709 13:18:39.639406 16342 ts_tablet_manager.cc:1403] T 434bff3082d141838068f323bdb0d1ed P 90f3da57796b488c9087a2f523adf045: Time spent bootstrapping tablet: real 0.023s	user 0.008s	sys 0.009s
I20260709 13:18:39.645143 16347 tablet_bootstrap.cc:492] T 5a96373bb0e14dce8a266b2fc6c58507 P 1ca2725d290e47d68b1a4b609c9816ac: No bootstrap required, opened a new log
I20260709 13:18:39.644914 16342 raft_consensus.cc:359] T 434bff3082d141838068f323bdb0d1ed P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:39.645668 16347 ts_tablet_manager.cc:1403] T 5a96373bb0e14dce8a266b2fc6c58507 P 1ca2725d290e47d68b1a4b609c9816ac: Time spent bootstrapping tablet: real 0.031s	user 0.010s	sys 0.007s
I20260709 13:18:39.645691 16342 raft_consensus.cc:385] T 434bff3082d141838068f323bdb0d1ed P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:39.646250 16340 tablet_bootstrap.cc:492] T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b: No bootstrap required, opened a new log
I20260709 13:18:39.646214 16342 raft_consensus.cc:740] T 434bff3082d141838068f323bdb0d1ed P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Initialized, Role: FOLLOWER
I20260709 13:18:39.646788 16340 ts_tablet_manager.cc:1403] T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b: Time spent bootstrapping tablet: real 0.031s	user 0.020s	sys 0.008s
I20260709 13:18:39.647212 16342 consensus_queue.cc:260] T 434bff3082d141838068f323bdb0d1ed P 90f3da57796b488c9087a2f523adf045 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:39.648909 16347 raft_consensus.cc:359] T 5a96373bb0e14dce8a266b2fc6c58507 P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:39.649608 16347 raft_consensus.cc:385] T 5a96373bb0e14dce8a266b2fc6c58507 P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:39.649951 16347 raft_consensus.cc:740] T 5a96373bb0e14dce8a266b2fc6c58507 P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Initialized, Role: FOLLOWER
I20260709 13:18:39.649724 16340 raft_consensus.cc:359] T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:39.650513 16340 raft_consensus.cc:385] T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:39.650830 16340 raft_consensus.cc:740] T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Initialized, Role: FOLLOWER
I20260709 13:18:39.650708 16347 consensus_queue.cc:260] T 5a96373bb0e14dce8a266b2fc6c58507 P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:39.651716 16340 consensus_queue.cc:260] T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:39.652485 16342 ts_tablet_manager.cc:1434] T 434bff3082d141838068f323bdb0d1ed P 90f3da57796b488c9087a2f523adf045: Time spent starting tablet: real 0.013s	user 0.007s	sys 0.000s
I20260709 13:18:39.652988 16347 ts_tablet_manager.cc:1434] T 5a96373bb0e14dce8a266b2fc6c58507 P 1ca2725d290e47d68b1a4b609c9816ac: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.004s
I20260709 13:18:39.653430 16342 tablet_bootstrap.cc:492] T 8cc959fb7f704fba9009c0595467d074 P 90f3da57796b488c9087a2f523adf045: Bootstrap starting.
I20260709 13:18:39.653839 16340 ts_tablet_manager.cc:1434] T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 13:18:39.653945 16347 tablet_bootstrap.cc:492] T 434bff3082d141838068f323bdb0d1ed P 1ca2725d290e47d68b1a4b609c9816ac: Bootstrap starting.
I20260709 13:18:39.654762 16340 tablet_bootstrap.cc:492] T 5a96373bb0e14dce8a266b2fc6c58507 P 647a343a542d4a139fe106c19095129b: Bootstrap starting.
I20260709 13:18:39.659909 16342 tablet_bootstrap.cc:654] T 8cc959fb7f704fba9009c0595467d074 P 90f3da57796b488c9087a2f523adf045: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:39.661176 16340 tablet_bootstrap.cc:654] T 5a96373bb0e14dce8a266b2fc6c58507 P 647a343a542d4a139fe106c19095129b: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:39.666152 16373 raft_consensus.cc:493] T 434bff3082d141838068f323bdb0d1ed P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:39.666899 16373 raft_consensus.cc:515] T 434bff3082d141838068f323bdb0d1ed P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:39.671295 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434bff3082d141838068f323bdb0d1ed" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:18:39.671653 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434bff3082d141838068f323bdb0d1ed" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:18:39.672232 16299 raft_consensus.cc:2468] T 434bff3082d141838068f323bdb0d1ed P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 0.
W20260709 13:18:39.673132 16116 leader_election.cc:343] T 434bff3082d141838068f323bdb0d1ed P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:18:39.673931 16115 leader_election.cc:304] T 434bff3082d141838068f323bdb0d1ed P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac
I20260709 13:18:39.674809 16344 raft_consensus.cc:2804] T 434bff3082d141838068f323bdb0d1ed P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:18:39.675185 16344 raft_consensus.cc:493] T 434bff3082d141838068f323bdb0d1ed P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:39.675540 16344 raft_consensus.cc:3060] T 434bff3082d141838068f323bdb0d1ed P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:39.677026 16340 tablet_bootstrap.cc:492] T 5a96373bb0e14dce8a266b2fc6c58507 P 647a343a542d4a139fe106c19095129b: No bootstrap required, opened a new log
I20260709 13:18:39.677697 16340 ts_tablet_manager.cc:1403] T 5a96373bb0e14dce8a266b2fc6c58507 P 647a343a542d4a139fe106c19095129b: Time spent bootstrapping tablet: real 0.023s	user 0.009s	sys 0.009s
I20260709 13:18:39.681617 16340 raft_consensus.cc:359] T 5a96373bb0e14dce8a266b2fc6c58507 P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:39.682595 16340 raft_consensus.cc:385] T 5a96373bb0e14dce8a266b2fc6c58507 P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:39.682966 16340 raft_consensus.cc:740] T 5a96373bb0e14dce8a266b2fc6c58507 P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Initialized, Role: FOLLOWER
I20260709 13:18:39.683935 16340 consensus_queue.cc:260] T 5a96373bb0e14dce8a266b2fc6c58507 P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:39.671682 16373 leader_election.cc:290] T 434bff3082d141838068f323bdb0d1ed P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:18:39.687902 16344 raft_consensus.cc:515] T 434bff3082d141838068f323bdb0d1ed P 647a343a542d4a139fe106c19095129b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:39.689577 16347 tablet_bootstrap.cc:654] T 434bff3082d141838068f323bdb0d1ed P 1ca2725d290e47d68b1a4b609c9816ac: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:39.690901 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434bff3082d141838068f323bdb0d1ed" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
W20260709 13:18:39.695263 16116 leader_election.cc:343] T 434bff3082d141838068f323bdb0d1ed P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:18:39.696441 16340 ts_tablet_manager.cc:1434] T 5a96373bb0e14dce8a266b2fc6c58507 P 647a343a542d4a139fe106c19095129b: Time spent starting tablet: real 0.018s	user 0.017s	sys 0.004s
I20260709 13:18:39.697809 16340 tablet_bootstrap.cc:492] T 7c2095a5f4ad49408dbc7f69c606120d P 647a343a542d4a139fe106c19095129b: Bootstrap starting.
I20260709 13:18:39.703085 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434bff3082d141838068f323bdb0d1ed" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:18:39.703802 16299 raft_consensus.cc:3060] T 434bff3082d141838068f323bdb0d1ed P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:39.704656 16340 tablet_bootstrap.cc:654] T 7c2095a5f4ad49408dbc7f69c606120d P 647a343a542d4a139fe106c19095129b: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:39.710216 16347 tablet_bootstrap.cc:492] T 434bff3082d141838068f323bdb0d1ed P 1ca2725d290e47d68b1a4b609c9816ac: No bootstrap required, opened a new log
I20260709 13:18:39.710461 16344 leader_election.cc:290] T 434bff3082d141838068f323bdb0d1ed P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 1 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:18:39.711110 16347 ts_tablet_manager.cc:1403] T 434bff3082d141838068f323bdb0d1ed P 1ca2725d290e47d68b1a4b609c9816ac: Time spent bootstrapping tablet: real 0.057s	user 0.016s	sys 0.004s
I20260709 13:18:39.711772 16340 tablet_bootstrap.cc:492] T 7c2095a5f4ad49408dbc7f69c606120d P 647a343a542d4a139fe106c19095129b: No bootstrap required, opened a new log
I20260709 13:18:39.712255 16340 ts_tablet_manager.cc:1403] T 7c2095a5f4ad49408dbc7f69c606120d P 647a343a542d4a139fe106c19095129b: Time spent bootstrapping tablet: real 0.015s	user 0.010s	sys 0.001s
I20260709 13:18:39.712759 16299 raft_consensus.cc:2468] T 434bff3082d141838068f323bdb0d1ed P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 1.
I20260709 13:18:39.714121 16115 leader_election.cc:304] T 434bff3082d141838068f323bdb0d1ed P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac
I20260709 13:18:39.714324 16347 raft_consensus.cc:359] T 434bff3082d141838068f323bdb0d1ed P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:39.715122 16347 raft_consensus.cc:385] T 434bff3082d141838068f323bdb0d1ed P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:39.715317 16344 raft_consensus.cc:2804] T 434bff3082d141838068f323bdb0d1ed P 647a343a542d4a139fe106c19095129b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:18:39.714995 16340 raft_consensus.cc:359] T 7c2095a5f4ad49408dbc7f69c606120d P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:39.715530 16347 raft_consensus.cc:740] T 434bff3082d141838068f323bdb0d1ed P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Initialized, Role: FOLLOWER
I20260709 13:18:39.715948 16344 raft_consensus.cc:697] T 434bff3082d141838068f323bdb0d1ed P 647a343a542d4a139fe106c19095129b [term 1 LEADER]: Becoming Leader. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:18:39.717608 16347 consensus_queue.cc:260] T 434bff3082d141838068f323bdb0d1ed P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:39.718320 16342 tablet_bootstrap.cc:492] T 8cc959fb7f704fba9009c0595467d074 P 90f3da57796b488c9087a2f523adf045: No bootstrap required, opened a new log
I20260709 13:18:39.717912 16344 consensus_queue.cc:237] T 434bff3082d141838068f323bdb0d1ed P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:39.718813 16342 ts_tablet_manager.cc:1403] T 8cc959fb7f704fba9009c0595467d074 P 90f3da57796b488c9087a2f523adf045: Time spent bootstrapping tablet: real 0.066s	user 0.022s	sys 0.017s
I20260709 13:18:39.720541 16347 ts_tablet_manager.cc:1434] T 434bff3082d141838068f323bdb0d1ed P 1ca2725d290e47d68b1a4b609c9816ac: Time spent starting tablet: real 0.009s	user 0.004s	sys 0.002s
I20260709 13:18:39.715921 16340 raft_consensus.cc:385] T 7c2095a5f4ad49408dbc7f69c606120d P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:39.721812 16342 raft_consensus.cc:359] T 8cc959fb7f704fba9009c0595467d074 P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:39.722829 16342 raft_consensus.cc:385] T 8cc959fb7f704fba9009c0595467d074 P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:39.723223 16342 raft_consensus.cc:740] T 8cc959fb7f704fba9009c0595467d074 P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Initialized, Role: FOLLOWER
I20260709 13:18:39.721844 16340 raft_consensus.cc:740] T 7c2095a5f4ad49408dbc7f69c606120d P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Initialized, Role: FOLLOWER
I20260709 13:18:39.725617 16347 tablet_bootstrap.cc:492] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac: Bootstrap starting.
I20260709 13:18:39.725178 16340 consensus_queue.cc:260] T 7c2095a5f4ad49408dbc7f69c606120d P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:39.728937 16342 consensus_queue.cc:260] T 8cc959fb7f704fba9009c0595467d074 P 90f3da57796b488c9087a2f523adf045 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:39.732220 16342 ts_tablet_manager.cc:1434] T 8cc959fb7f704fba9009c0595467d074 P 90f3da57796b488c9087a2f523adf045: Time spent starting tablet: real 0.013s	user 0.007s	sys 0.003s
I20260709 13:18:39.733582 16342 tablet_bootstrap.cc:492] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045: Bootstrap starting.
I20260709 13:18:39.728260 16340 ts_tablet_manager.cc:1434] T 7c2095a5f4ad49408dbc7f69c606120d P 647a343a542d4a139fe106c19095129b: Time spent starting tablet: real 0.016s	user 0.007s	sys 0.002s
I20260709 13:18:39.735914 16347 tablet_bootstrap.cc:654] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:39.736349 16340 tablet_bootstrap.cc:492] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 647a343a542d4a139fe106c19095129b: Bootstrap starting.
I20260709 13:18:39.745815 16340 tablet_bootstrap.cc:654] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 647a343a542d4a139fe106c19095129b: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:39.747505 16347 tablet_bootstrap.cc:492] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac: No bootstrap required, opened a new log
I20260709 13:18:39.748011 16347 ts_tablet_manager.cc:1403] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac: Time spent bootstrapping tablet: real 0.027s	user 0.013s	sys 0.004s
I20260709 13:18:39.750866 16347 raft_consensus.cc:359] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:39.751572 16347 raft_consensus.cc:385] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:39.753175 16347 raft_consensus.cc:740] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Initialized, Role: FOLLOWER
I20260709 13:18:39.756101 16046 catalog_manager.cc:5697] T 434bff3082d141838068f323bdb0d1ed P 647a343a542d4a139fe106c19095129b reported cstate change: term changed from 0 to 1, leader changed from <none> to 647a343a542d4a139fe106c19095129b (127.14.177.129). New cstate: current_term: 1 leader_uuid: "647a343a542d4a139fe106c19095129b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:39.753852 16347 consensus_queue.cc:260] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:39.759634 16347 ts_tablet_manager.cc:1434] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac: Time spent starting tablet: real 0.011s	user 0.007s	sys 0.000s
I20260709 13:18:39.761706 16342 tablet_bootstrap.cc:654] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:39.763331 16340 tablet_bootstrap.cc:492] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 647a343a542d4a139fe106c19095129b: No bootstrap required, opened a new log
I20260709 13:18:39.763764 16340 ts_tablet_manager.cc:1403] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 647a343a542d4a139fe106c19095129b: Time spent bootstrapping tablet: real 0.028s	user 0.018s	sys 0.008s
I20260709 13:18:39.765044 16366 raft_consensus.cc:493] T 8a878999c0024d2abe517484180290fc P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:39.765643 16366 raft_consensus.cc:515] T 8a878999c0024d2abe517484180290fc P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:39.766251 16340 raft_consensus.cc:359] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:39.766943 16340 raft_consensus.cc:385] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:39.767271 16340 raft_consensus.cc:740] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Initialized, Role: FOLLOWER
I20260709 13:18:39.767614 16366 leader_election.cc:290] T 8a878999c0024d2abe517484180290fc P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:18:39.767971 16342 tablet_bootstrap.cc:492] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045: No bootstrap required, opened a new log
I20260709 13:18:39.768844 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a878999c0024d2abe517484180290fc" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:18:39.769425 16299 raft_consensus.cc:2468] T 8a878999c0024d2abe517484180290fc P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 0.
I20260709 13:18:39.769790 16151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a878999c0024d2abe517484180290fc" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:18:39.770447 16151 raft_consensus.cc:2468] T 8a878999c0024d2abe517484180290fc P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 0.
I20260709 13:18:39.768393 16342 ts_tablet_manager.cc:1403] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045: Time spent bootstrapping tablet: real 0.035s	user 0.008s	sys 0.003s
I20260709 13:18:39.770819 16190 leader_election.cc:304] T 8a878999c0024d2abe517484180290fc P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:18:39.771596 16347 tablet_bootstrap.cc:492] T 7c2095a5f4ad49408dbc7f69c606120d P 1ca2725d290e47d68b1a4b609c9816ac: Bootstrap starting.
I20260709 13:18:39.771765 16366 raft_consensus.cc:2804] T 8a878999c0024d2abe517484180290fc P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:18:39.772179 16366 raft_consensus.cc:493] T 8a878999c0024d2abe517484180290fc P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:39.772535 16366 raft_consensus.cc:3060] T 8a878999c0024d2abe517484180290fc P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:39.773341 16342 raft_consensus.cc:359] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:39.773983 16342 raft_consensus.cc:385] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:39.774240 16342 raft_consensus.cc:740] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Initialized, Role: FOLLOWER
I20260709 13:18:39.775058 16358 raft_consensus.cc:493] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:39.775540 16358 raft_consensus.cc:515] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:39.779850 16347 tablet_bootstrap.cc:654] T 7c2095a5f4ad49408dbc7f69c606120d P 1ca2725d290e47d68b1a4b609c9816ac: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:39.780658 16358 leader_election.cc:290] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:18:39.780443 16366 raft_consensus.cc:515] T 8a878999c0024d2abe517484180290fc P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:39.782646 16366 leader_election.cc:290] T 8a878999c0024d2abe517484180290fc P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 1 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:18:39.768069 16340 consensus_queue.cc:260] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:39.783823 16151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2372981006d4b218215c28964dea623" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:18:39.784529 16151 raft_consensus.cc:2468] T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 0.
I20260709 13:18:39.784576 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a878999c0024d2abe517484180290fc" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:18:39.784994 16340 ts_tablet_manager.cc:1434] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 647a343a542d4a139fe106c19095129b: Time spent starting tablet: real 0.021s	user 0.005s	sys 0.000s
I20260709 13:18:39.785251 16150 raft_consensus.cc:3060] T 8a878999c0024d2abe517484180290fc P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:39.786540 16340 tablet_bootstrap.cc:492] T fffab26427e24d398e2695aad40ca134 P 647a343a542d4a139fe106c19095129b: Bootstrap starting.
I20260709 13:18:39.788084 16191 leader_election.cc:304] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
I20260709 13:18:39.788516 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a878999c0024d2abe517484180290fc" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:18:39.789171 16300 raft_consensus.cc:3060] T 8a878999c0024d2abe517484180290fc P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:39.793301 16344 raft_consensus.cc:493] T 9e65676b21b748728ad39668cfe722ec P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:39.793505 16366 raft_consensus.cc:2804] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:18:39.793999 16344 raft_consensus.cc:515] T 9e65676b21b748728ad39668cfe722ec P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:39.795074 16340 tablet_bootstrap.cc:654] T fffab26427e24d398e2695aad40ca134 P 647a343a542d4a139fe106c19095129b: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:39.794000 16366 raft_consensus.cc:493] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:39.796720 16366 raft_consensus.cc:3060] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:39.797467 16344 leader_election.cc:290] T 9e65676b21b748728ad39668cfe722ec P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:18:39.797741 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2372981006d4b218215c28964dea623" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:18:39.798447 16299 raft_consensus.cc:2468] T d2372981006d4b218215c28964dea623 P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 0.
I20260709 13:18:39.799856 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e65676b21b748728ad39668cfe722ec" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:18:39.800050 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e65676b21b748728ad39668cfe722ec" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:18:39.800784 16298 raft_consensus.cc:2468] T 9e65676b21b748728ad39668cfe722ec P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 0.
I20260709 13:18:39.801070 16224 raft_consensus.cc:2468] T 9e65676b21b748728ad39668cfe722ec P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 0.
I20260709 13:18:39.802443 16115 leader_election.cc:304] T 9e65676b21b748728ad39668cfe722ec P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:18:39.804176 16344 raft_consensus.cc:2804] T 9e65676b21b748728ad39668cfe722ec P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:18:39.804679 16344 raft_consensus.cc:493] T 9e65676b21b748728ad39668cfe722ec P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:39.805109 16344 raft_consensus.cc:3060] T 9e65676b21b748728ad39668cfe722ec P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:39.806804 16366 raft_consensus.cc:515] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:39.810231 16151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2372981006d4b218215c28964dea623" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:18:39.812207 16151 raft_consensus.cc:3060] T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:39.814458 16300 raft_consensus.cc:2468] T 8a878999c0024d2abe517484180290fc P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 1.
I20260709 13:18:39.815886 16190 leader_election.cc:304] T 8a878999c0024d2abe517484180290fc P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:18:39.816828 16358 raft_consensus.cc:2804] T 8a878999c0024d2abe517484180290fc P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:18:39.817667 16358 raft_consensus.cc:697] T 8a878999c0024d2abe517484180290fc P 1ca2725d290e47d68b1a4b609c9816ac [term 1 LEADER]: Becoming Leader. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:18:39.818512 16358 consensus_queue.cc:237] T 8a878999c0024d2abe517484180290fc P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:39.819499 16151 raft_consensus.cc:2468] T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 1.
I20260709 13:18:39.819746 16347 tablet_bootstrap.cc:492] T 7c2095a5f4ad49408dbc7f69c606120d P 1ca2725d290e47d68b1a4b609c9816ac: No bootstrap required, opened a new log
I20260709 13:18:39.820231 16347 ts_tablet_manager.cc:1403] T 7c2095a5f4ad49408dbc7f69c606120d P 1ca2725d290e47d68b1a4b609c9816ac: Time spent bootstrapping tablet: real 0.060s	user 0.022s	sys 0.017s
I20260709 13:18:39.820871 16191 leader_election.cc:304] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
I20260709 13:18:39.822086 16376 raft_consensus.cc:2804] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:18:39.822238 16366 leader_election.cc:290] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 1 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:18:39.822906 16347 raft_consensus.cc:359] T 7c2095a5f4ad49408dbc7f69c606120d P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:39.823491 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2372981006d4b218215c28964dea623" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:18:39.823815 16347 raft_consensus.cc:385] T 7c2095a5f4ad49408dbc7f69c606120d P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:39.824165 16300 raft_consensus.cc:3060] T d2372981006d4b218215c28964dea623 P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:39.824204 16347 raft_consensus.cc:740] T 7c2095a5f4ad49408dbc7f69c606120d P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Initialized, Role: FOLLOWER
I20260709 13:18:39.825817 16340 tablet_bootstrap.cc:492] T fffab26427e24d398e2695aad40ca134 P 647a343a542d4a139fe106c19095129b: No bootstrap required, opened a new log
I20260709 13:18:39.825464 16347 consensus_queue.cc:260] T 7c2095a5f4ad49408dbc7f69c606120d P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:39.826280 16340 ts_tablet_manager.cc:1403] T fffab26427e24d398e2695aad40ca134 P 647a343a542d4a139fe106c19095129b: Time spent bootstrapping tablet: real 0.040s	user 0.008s	sys 0.004s
I20260709 13:18:39.828106 16347 ts_tablet_manager.cc:1434] T 7c2095a5f4ad49408dbc7f69c606120d P 1ca2725d290e47d68b1a4b609c9816ac: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.003s
I20260709 13:18:39.829527 16347 tablet_bootstrap.cc:492] T 02838c86531d4f98bf58d1c22b8baf77 P 1ca2725d290e47d68b1a4b609c9816ac: Bootstrap starting.
I20260709 13:18:39.829154 16342 consensus_queue.cc:260] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:39.829458 16340 raft_consensus.cc:359] T fffab26427e24d398e2695aad40ca134 P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:39.829154 16366 raft_consensus.cc:493] T fffab26427e24d398e2695aad40ca134 P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:39.831655 16340 raft_consensus.cc:385] T fffab26427e24d398e2695aad40ca134 P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:39.832219 16340 raft_consensus.cc:740] T fffab26427e24d398e2695aad40ca134 P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Initialized, Role: FOLLOWER
I20260709 13:18:39.832381 16300 raft_consensus.cc:2468] T d2372981006d4b218215c28964dea623 P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 1.
I20260709 13:18:39.832226 16366 raft_consensus.cc:515] T fffab26427e24d398e2695aad40ca134 P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:39.836033 16151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fffab26427e24d398e2695aad40ca134" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:18:39.836982 16340 consensus_queue.cc:260] T fffab26427e24d398e2695aad40ca134 P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:39.839135 16347 tablet_bootstrap.cc:654] T 02838c86531d4f98bf58d1c22b8baf77 P 1ca2725d290e47d68b1a4b609c9816ac: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:39.839141 16151 raft_consensus.cc:2468] T fffab26427e24d398e2695aad40ca134 P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 0.
I20260709 13:18:39.840204 16342 ts_tablet_manager.cc:1434] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045: Time spent starting tablet: real 0.069s	user 0.005s	sys 0.001s
I20260709 13:18:39.841516 16366 leader_election.cc:290] T fffab26427e24d398e2695aad40ca134 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:18:39.843111 16340 ts_tablet_manager.cc:1434] T fffab26427e24d398e2695aad40ca134 P 647a343a542d4a139fe106c19095129b: Time spent starting tablet: real 0.017s	user 0.009s	sys 0.000s
I20260709 13:18:39.845518 16191 leader_election.cc:304] T fffab26427e24d398e2695aad40ca134 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
I20260709 13:18:39.841122 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fffab26427e24d398e2695aad40ca134" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:18:39.849713 16340 tablet_bootstrap.cc:492] T a378d7f796f94615b3b347119a9880fd P 647a343a542d4a139fe106c19095129b: Bootstrap starting.
I20260709 13:18:39.849880 16300 raft_consensus.cc:2468] T fffab26427e24d398e2695aad40ca134 P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 0.
I20260709 13:18:39.842980 16376 raft_consensus.cc:697] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [term 1 LEADER]: Becoming Leader. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:18:39.853119 16376 consensus_queue.cc:237] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:39.838512 16150 raft_consensus.cc:2468] T 8a878999c0024d2abe517484180290fc P 647a343a542d4a139fe106c19095129b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 1.
I20260709 13:18:39.856920 16340 tablet_bootstrap.cc:654] T a378d7f796f94615b3b347119a9880fd P 647a343a542d4a139fe106c19095129b: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:39.859397 16366 raft_consensus.cc:2804] T fffab26427e24d398e2695aad40ca134 P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:18:39.860004 16366 raft_consensus.cc:493] T fffab26427e24d398e2695aad40ca134 P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:39.860483 16366 raft_consensus.cc:3060] T fffab26427e24d398e2695aad40ca134 P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:39.861061 16344 raft_consensus.cc:515] T 9e65676b21b748728ad39668cfe722ec P 647a343a542d4a139fe106c19095129b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:39.863624 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e65676b21b748728ad39668cfe722ec" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:18:39.863835 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e65676b21b748728ad39668cfe722ec" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:18:39.864224 16224 raft_consensus.cc:3060] T 9e65676b21b748728ad39668cfe722ec P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:39.864352 16300 raft_consensus.cc:3060] T 9e65676b21b748728ad39668cfe722ec P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:39.871189 16300 raft_consensus.cc:2468] T 9e65676b21b748728ad39668cfe722ec P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 1.
I20260709 13:18:39.871189 16224 raft_consensus.cc:2468] T 9e65676b21b748728ad39668cfe722ec P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 1.
I20260709 13:18:39.872602 16116 leader_election.cc:304] T 9e65676b21b748728ad39668cfe722ec P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
I20260709 13:18:39.873582 16395 raft_consensus.cc:2804] T 9e65676b21b748728ad39668cfe722ec P 647a343a542d4a139fe106c19095129b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:18:39.873049 16366 raft_consensus.cc:515] T fffab26427e24d398e2695aad40ca134 P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:39.874104 16395 raft_consensus.cc:697] T 9e65676b21b748728ad39668cfe722ec P 647a343a542d4a139fe106c19095129b [term 1 LEADER]: Becoming Leader. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:18:39.875010 16395 consensus_queue.cc:237] T 9e65676b21b748728ad39668cfe722ec P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:39.841189 16342 tablet_bootstrap.cc:492] T 68a6838d82ef4cac83507f906139d90f P 90f3da57796b488c9087a2f523adf045: Bootstrap starting.
I20260709 13:18:39.879056 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fffab26427e24d398e2695aad40ca134" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:18:39.879504 16344 leader_election.cc:290] T 9e65676b21b748728ad39668cfe722ec P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 1 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:18:39.879892 16300 raft_consensus.cc:3060] T fffab26427e24d398e2695aad40ca134 P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:39.884652 16342 tablet_bootstrap.cc:654] T 68a6838d82ef4cac83507f906139d90f P 90f3da57796b488c9087a2f523adf045: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:39.893024 16366 leader_election.cc:290] T fffab26427e24d398e2695aad40ca134 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 1 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:18:39.892774 16049 catalog_manager.cc:5697] T 8a878999c0024d2abe517484180290fc P 1ca2725d290e47d68b1a4b609c9816ac reported cstate change: term changed from 0 to 1, leader changed from <none> to 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130). New cstate: current_term: 1 leader_uuid: "1ca2725d290e47d68b1a4b609c9816ac" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:39.897258 16347 tablet_bootstrap.cc:492] T 02838c86531d4f98bf58d1c22b8baf77 P 1ca2725d290e47d68b1a4b609c9816ac: No bootstrap required, opened a new log
I20260709 13:18:39.897698 16347 ts_tablet_manager.cc:1403] T 02838c86531d4f98bf58d1c22b8baf77 P 1ca2725d290e47d68b1a4b609c9816ac: Time spent bootstrapping tablet: real 0.068s	user 0.023s	sys 0.031s
I20260709 13:18:39.898854 16300 raft_consensus.cc:2468] T fffab26427e24d398e2695aad40ca134 P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 1.
I20260709 13:18:39.900524 16190 leader_election.cc:304] T fffab26427e24d398e2695aad40ca134 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:18:39.900992 16347 raft_consensus.cc:359] T 02838c86531d4f98bf58d1c22b8baf77 P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:39.901679 16347 raft_consensus.cc:385] T 02838c86531d4f98bf58d1c22b8baf77 P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:39.901724 16366 raft_consensus.cc:2804] T fffab26427e24d398e2695aad40ca134 P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:18:39.901980 16347 raft_consensus.cc:740] T 02838c86531d4f98bf58d1c22b8baf77 P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Initialized, Role: FOLLOWER
I20260709 13:18:39.902720 16366 raft_consensus.cc:697] T fffab26427e24d398e2695aad40ca134 P 1ca2725d290e47d68b1a4b609c9816ac [term 1 LEADER]: Becoming Leader. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:18:39.903733 16366 consensus_queue.cc:237] T fffab26427e24d398e2695aad40ca134 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:39.904688 16342 tablet_bootstrap.cc:492] T 68a6838d82ef4cac83507f906139d90f P 90f3da57796b488c9087a2f523adf045: No bootstrap required, opened a new log
I20260709 13:18:39.905158 16342 ts_tablet_manager.cc:1403] T 68a6838d82ef4cac83507f906139d90f P 90f3da57796b488c9087a2f523adf045: Time spent bootstrapping tablet: real 0.064s	user 0.005s	sys 0.005s
I20260709 13:18:39.905090 16347 consensus_queue.cc:260] T 02838c86531d4f98bf58d1c22b8baf77 P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:39.908011 16347 ts_tablet_manager.cc:1434] T 02838c86531d4f98bf58d1c22b8baf77 P 1ca2725d290e47d68b1a4b609c9816ac: Time spent starting tablet: real 0.010s	user 0.004s	sys 0.002s
I20260709 13:18:39.908891 16347 tablet_bootstrap.cc:492] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac: Bootstrap starting.
I20260709 13:18:39.908807 16344 raft_consensus.cc:493] T 8cc959fb7f704fba9009c0595467d074 P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:39.909668 16344 raft_consensus.cc:515] T 8cc959fb7f704fba9009c0595467d074 P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:39.911187 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fffab26427e24d398e2695aad40ca134" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:18:39.911840 16150 raft_consensus.cc:3060] T fffab26427e24d398e2695aad40ca134 P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:39.912601 16344 leader_election.cc:290] T 8cc959fb7f704fba9009c0595467d074 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:18:39.914733 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cc959fb7f704fba9009c0595467d074" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:18:39.915448 16300 raft_consensus.cc:2468] T 8cc959fb7f704fba9009c0595467d074 P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 0.
I20260709 13:18:39.915643 16342 raft_consensus.cc:359] T 68a6838d82ef4cac83507f906139d90f P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:39.916393 16342 raft_consensus.cc:385] T 68a6838d82ef4cac83507f906139d90f P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:39.916702 16342 raft_consensus.cc:740] T 68a6838d82ef4cac83507f906139d90f P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Initialized, Role: FOLLOWER
I20260709 13:18:39.916982 16115 leader_election.cc:304] T 8cc959fb7f704fba9009c0595467d074 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:18:39.918321 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cc959fb7f704fba9009c0595467d074" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:18:39.919308 16224 raft_consensus.cc:2468] T 8cc959fb7f704fba9009c0595467d074 P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 0.
I20260709 13:18:39.920186 16342 consensus_queue.cc:260] T 68a6838d82ef4cac83507f906139d90f P 90f3da57796b488c9087a2f523adf045 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:39.921097 16396 raft_consensus.cc:2804] T 8cc959fb7f704fba9009c0595467d074 P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:18:39.921643 16396 raft_consensus.cc:493] T 8cc959fb7f704fba9009c0595467d074 P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:39.922138 16396 raft_consensus.cc:3060] T 8cc959fb7f704fba9009c0595467d074 P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:39.924360 16347 tablet_bootstrap.cc:654] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:39.927671 16342 ts_tablet_manager.cc:1434] T 68a6838d82ef4cac83507f906139d90f P 90f3da57796b488c9087a2f523adf045: Time spent starting tablet: real 0.022s	user 0.004s	sys 0.004s
I20260709 13:18:39.933121 16396 raft_consensus.cc:515] T 8cc959fb7f704fba9009c0595467d074 P 647a343a542d4a139fe106c19095129b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:39.926062 16049 catalog_manager.cc:5697] T 9e65676b21b748728ad39668cfe722ec P 647a343a542d4a139fe106c19095129b reported cstate change: term changed from 0 to 1, leader changed from <none> to 647a343a542d4a139fe106c19095129b (127.14.177.129). New cstate: current_term: 1 leader_uuid: "647a343a542d4a139fe106c19095129b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:39.935153 16395 raft_consensus.cc:493] T 02838c86531d4f98bf58d1c22b8baf77 P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:39.936280 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cc959fb7f704fba9009c0595467d074" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:18:39.959960 16225 raft_consensus.cc:3060] T 8cc959fb7f704fba9009c0595467d074 P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:39.960464 16048 catalog_manager.cc:5697] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac reported cstate change: term changed from 0 to 1, leader changed from <none> to 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130). New cstate: current_term: 1 leader_uuid: "1ca2725d290e47d68b1a4b609c9816ac" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:39.963554 16395 raft_consensus.cc:515] T 02838c86531d4f98bf58d1c22b8baf77 P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:39.940267 16342 tablet_bootstrap.cc:492] T a378d7f796f94615b3b347119a9880fd P 90f3da57796b488c9087a2f523adf045: Bootstrap starting.
I20260709 13:18:39.968158 16366 raft_consensus.cc:493] T 7c2095a5f4ad49408dbc7f69c606120d P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:39.968693 16366 raft_consensus.cc:515] T 7c2095a5f4ad49408dbc7f69c606120d P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:39.971586 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02838c86531d4f98bf58d1c22b8baf77" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:18:39.971865 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c2095a5f4ad49408dbc7f69c606120d" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:18:39.972471 16224 raft_consensus.cc:2468] T 02838c86531d4f98bf58d1c22b8baf77 P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 0.
I20260709 13:18:39.974574 16342 tablet_bootstrap.cc:654] T a378d7f796f94615b3b347119a9880fd P 90f3da57796b488c9087a2f523adf045: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:39.974427 16116 leader_election.cc:304] T 02838c86531d4f98bf58d1c22b8baf77 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
I20260709 13:18:39.971583 16366 leader_election.cc:290] T 7c2095a5f4ad49408dbc7f69c606120d P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:18:39.976395 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cc959fb7f704fba9009c0595467d074" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:18:39.977051 16299 raft_consensus.cc:3060] T 8cc959fb7f704fba9009c0595467d074 P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:39.939720 16396 leader_election.cc:290] T 8cc959fb7f704fba9009c0595467d074 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 1 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:18:39.972573 16300 raft_consensus.cc:2468] T 7c2095a5f4ad49408dbc7f69c606120d P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 0.
I20260709 13:18:39.977967 16150 raft_consensus.cc:2468] T fffab26427e24d398e2695aad40ca134 P 647a343a542d4a139fe106c19095129b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 1.
I20260709 13:18:39.978566 16190 leader_election.cc:304] T 7c2095a5f4ad49408dbc7f69c606120d P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:18:39.963300 16048 catalog_manager.cc:5697] T fffab26427e24d398e2695aad40ca134 P 1ca2725d290e47d68b1a4b609c9816ac reported cstate change: term changed from 0 to 1, leader changed from <none> to 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130). New cstate: current_term: 1 leader_uuid: "1ca2725d290e47d68b1a4b609c9816ac" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:39.979583 16344 raft_consensus.cc:2804] T 02838c86531d4f98bf58d1c22b8baf77 P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:18:39.981544 16344 raft_consensus.cc:493] T 02838c86531d4f98bf58d1c22b8baf77 P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:39.982095 16344 raft_consensus.cc:3060] T 02838c86531d4f98bf58d1c22b8baf77 P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:39.982775 16151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c2095a5f4ad49408dbc7f69c606120d" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:18:39.983692 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02838c86531d4f98bf58d1c22b8baf77" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:18:39.983783 16151 raft_consensus.cc:2468] T 7c2095a5f4ad49408dbc7f69c606120d P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 0.
I20260709 13:18:39.980019 16366 raft_consensus.cc:2804] T 7c2095a5f4ad49408dbc7f69c606120d P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:18:39.985486 16366 raft_consensus.cc:493] T 7c2095a5f4ad49408dbc7f69c606120d P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:18:39.985776 16115 leader_election.cc:343] T 02838c86531d4f98bf58d1c22b8baf77 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:18:39.986030 16366 raft_consensus.cc:3060] T 7c2095a5f4ad49408dbc7f69c606120d P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:39.989773 16225 raft_consensus.cc:2468] T 8cc959fb7f704fba9009c0595467d074 P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 1.
I20260709 13:18:39.990198 16344 raft_consensus.cc:515] T 02838c86531d4f98bf58d1c22b8baf77 P 647a343a542d4a139fe106c19095129b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:39.991110 16116 leader_election.cc:304] T 8cc959fb7f704fba9009c0595467d074 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
I20260709 13:18:39.992456 16347 tablet_bootstrap.cc:492] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac: No bootstrap required, opened a new log
I20260709 13:18:39.992517 16344 leader_election.cc:290] T 02838c86531d4f98bf58d1c22b8baf77 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 1 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:18:39.993028 16396 raft_consensus.cc:2804] T 8cc959fb7f704fba9009c0595467d074 P 647a343a542d4a139fe106c19095129b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:18:39.993901 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02838c86531d4f98bf58d1c22b8baf77" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:18:39.995571 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02838c86531d4f98bf58d1c22b8baf77" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:18:39.998703 16225 raft_consensus.cc:3060] T 02838c86531d4f98bf58d1c22b8baf77 P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:18:39.999076 16115 leader_election.cc:343] T 02838c86531d4f98bf58d1c22b8baf77 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:18:39.999701 16396 raft_consensus.cc:697] T 8cc959fb7f704fba9009c0595467d074 P 647a343a542d4a139fe106c19095129b [term 1 LEADER]: Becoming Leader. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:18:40.001107 16396 consensus_queue.cc:237] T 8cc959fb7f704fba9009c0595467d074 P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:39.996214 16299 raft_consensus.cc:2468] T 8cc959fb7f704fba9009c0595467d074 P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 1.
I20260709 13:18:39.996742 16347 ts_tablet_manager.cc:1403] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac: Time spent bootstrapping tablet: real 0.088s	user 0.033s	sys 0.020s
I20260709 13:18:40.003798 16225 raft_consensus.cc:2468] T 02838c86531d4f98bf58d1c22b8baf77 P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 1.
I20260709 13:18:40.005066 16116 leader_election.cc:304] T 02838c86531d4f98bf58d1c22b8baf77 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 90f3da57796b488c9087a2f523adf045
I20260709 13:18:40.005975 16373 raft_consensus.cc:2804] T 02838c86531d4f98bf58d1c22b8baf77 P 647a343a542d4a139fe106c19095129b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:18:39.995875 16366 raft_consensus.cc:515] T 7c2095a5f4ad49408dbc7f69c606120d P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:40.010186 16347 raft_consensus.cc:359] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:40.011072 16347 raft_consensus.cc:385] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:40.011394 16347 raft_consensus.cc:740] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Initialized, Role: FOLLOWER
I20260709 13:18:40.012087 16347 consensus_queue.cc:260] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:40.013712 16347 ts_tablet_manager.cc:1434] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac: Time spent starting tablet: real 0.010s	user 0.003s	sys 0.001s
I20260709 13:18:40.014626 16347 tablet_bootstrap.cc:492] T 3884f864a96d428785895e5463e5728f P 1ca2725d290e47d68b1a4b609c9816ac: Bootstrap starting.
I20260709 13:18:40.016112 16340 tablet_bootstrap.cc:492] T a378d7f796f94615b3b347119a9880fd P 647a343a542d4a139fe106c19095129b: No bootstrap required, opened a new log
I20260709 13:18:40.016594 16340 ts_tablet_manager.cc:1403] T a378d7f796f94615b3b347119a9880fd P 647a343a542d4a139fe106c19095129b: Time spent bootstrapping tablet: real 0.167s	user 0.032s	sys 0.035s
I20260709 13:18:39.981143 16395 leader_election.cc:290] T 02838c86531d4f98bf58d1c22b8baf77 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:18:40.018441 16376 raft_consensus.cc:493] T 5a96373bb0e14dce8a266b2fc6c58507 P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:40.020592 16373 raft_consensus.cc:697] T 02838c86531d4f98bf58d1c22b8baf77 P 647a343a542d4a139fe106c19095129b [term 1 LEADER]: Becoming Leader. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:18:40.021780 16373 consensus_queue.cc:237] T 02838c86531d4f98bf58d1c22b8baf77 P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:40.022734 16342 tablet_bootstrap.cc:492] T a378d7f796f94615b3b347119a9880fd P 90f3da57796b488c9087a2f523adf045: No bootstrap required, opened a new log
I20260709 13:18:40.023234 16342 ts_tablet_manager.cc:1403] T a378d7f796f94615b3b347119a9880fd P 90f3da57796b488c9087a2f523adf045: Time spent bootstrapping tablet: real 0.083s	user 0.016s	sys 0.008s
I20260709 13:18:40.025957 16342 raft_consensus.cc:359] T a378d7f796f94615b3b347119a9880fd P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:40.026629 16342 raft_consensus.cc:385] T a378d7f796f94615b3b347119a9880fd P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:40.026908 16342 raft_consensus.cc:740] T a378d7f796f94615b3b347119a9880fd P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Initialized, Role: FOLLOWER
I20260709 13:18:40.027597 16342 consensus_queue.cc:260] T a378d7f796f94615b3b347119a9880fd P 90f3da57796b488c9087a2f523adf045 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:40.030031 16342 ts_tablet_manager.cc:1434] T a378d7f796f94615b3b347119a9880fd P 90f3da57796b488c9087a2f523adf045: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.000s
I20260709 13:18:40.029551 16376 raft_consensus.cc:515] T 5a96373bb0e14dce8a266b2fc6c58507 P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:40.033680 16342 tablet_bootstrap.cc:492] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 90f3da57796b488c9087a2f523adf045: Bootstrap starting.
I20260709 13:18:40.038905 16340 raft_consensus.cc:359] T a378d7f796f94615b3b347119a9880fd P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:40.039676 16340 raft_consensus.cc:385] T a378d7f796f94615b3b347119a9880fd P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:40.039953 16340 raft_consensus.cc:740] T a378d7f796f94615b3b347119a9880fd P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Initialized, Role: FOLLOWER
I20260709 13:18:40.048115 16340 consensus_queue.cc:260] T a378d7f796f94615b3b347119a9880fd P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:40.052119 16358 consensus_queue.cc:1048] T c2f5f89186204242bf377f7c42b2e2da P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:18:40.057705 16340 ts_tablet_manager.cc:1434] T a378d7f796f94615b3b347119a9880fd P 647a343a542d4a139fe106c19095129b: Time spent starting tablet: real 0.041s	user 0.008s	sys 0.000s
I20260709 13:18:40.058540 16376 leader_election.cc:290] T 5a96373bb0e14dce8a266b2fc6c58507 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:18:40.059721 16347 tablet_bootstrap.cc:654] T 3884f864a96d428785895e5463e5728f P 1ca2725d290e47d68b1a4b609c9816ac: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:40.060271 16151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a96373bb0e14dce8a266b2fc6c58507" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:18:40.058743 16340 tablet_bootstrap.cc:492] T 8c11074579b046d99e585c2818e18a2c P 647a343a542d4a139fe106c19095129b: Bootstrap starting.
I20260709 13:18:40.051223 16396 raft_consensus.cc:493] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:40.060765 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a96373bb0e14dce8a266b2fc6c58507" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:18:40.063781 16299 raft_consensus.cc:2468] T 5a96373bb0e14dce8a266b2fc6c58507 P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 0.
I20260709 13:18:40.068473 16366 leader_election.cc:290] T 7c2095a5f4ad49408dbc7f69c606120d P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 1 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:18:40.068002 16046 catalog_manager.cc:5697] T 8cc959fb7f704fba9009c0595467d074 P 647a343a542d4a139fe106c19095129b reported cstate change: term changed from 0 to 1, leader changed from <none> to 647a343a542d4a139fe106c19095129b (127.14.177.129). New cstate: current_term: 1 leader_uuid: "647a343a542d4a139fe106c19095129b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:40.069206 16190 leader_election.cc:304] T 5a96373bb0e14dce8a266b2fc6c58507 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:18:40.074313 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c2095a5f4ad49408dbc7f69c606120d" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:18:40.075136 16149 raft_consensus.cc:3060] T 7c2095a5f4ad49408dbc7f69c606120d P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:40.066514 16396 raft_consensus.cc:515] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:40.079257 16345 raft_consensus.cc:493] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:40.079749 16345 raft_consensus.cc:515] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:40.062498 16151 raft_consensus.cc:2468] T 5a96373bb0e14dce8a266b2fc6c58507 P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 0.
I20260709 13:18:40.081526 16396 leader_election.cc:290] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:18:40.081698 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c2095a5f4ad49408dbc7f69c606120d" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:18:40.082335 16299 raft_consensus.cc:3060] T 7c2095a5f4ad49408dbc7f69c606120d P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:40.082856 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e9b2bf7011f4ae688a79d7bb5a447b4" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:18:40.079782 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e9b2bf7011f4ae688a79d7bb5a447b4" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
W20260709 13:18:40.084717 16115 leader_election.cc:343] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:18:40.088234 16366 raft_consensus.cc:2804] T 5a96373bb0e14dce8a266b2fc6c58507 P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:18:40.088613 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f5f89186204242bf377f7c42b2e2da" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:18:40.088892 16366 raft_consensus.cc:493] T 5a96373bb0e14dce8a266b2fc6c58507 P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:40.089449 16366 raft_consensus.cc:3060] T 5a96373bb0e14dce8a266b2fc6c58507 P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:18:40.092056 16116 leader_election.cc:343] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:18:40.092811 16116 leader_election.cc:304] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:18:40.094561 16342 tablet_bootstrap.cc:654] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 90f3da57796b488c9087a2f523adf045: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:40.094494 16151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f5f89186204242bf377f7c42b2e2da" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:18:40.081853 16345 leader_election.cc:290] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:18:40.098472 16396 raft_consensus.cc:2749] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:18:40.104986 16265 leader_election.cc:304] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:18:40.108060 16345 raft_consensus.cc:2749] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:18:40.109736 16340 tablet_bootstrap.cc:654] T 8c11074579b046d99e585c2818e18a2c P 647a343a542d4a139fe106c19095129b: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:40.114012 16300 raft_consensus.cc:3060] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:40.119812 16299 raft_consensus.cc:2468] T 7c2095a5f4ad49408dbc7f69c606120d P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 1.
I20260709 13:18:40.121006 16190 leader_election.cc:304] T 7c2095a5f4ad49408dbc7f69c606120d P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:18:40.121821 16358 raft_consensus.cc:2804] T 7c2095a5f4ad49408dbc7f69c606120d P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:18:40.122287 16358 raft_consensus.cc:697] T 7c2095a5f4ad49408dbc7f69c606120d P 1ca2725d290e47d68b1a4b609c9816ac [term 1 LEADER]: Becoming Leader. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:18:40.123106 16358 consensus_queue.cc:237] T 7c2095a5f4ad49408dbc7f69c606120d P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:40.124933 16340 tablet_bootstrap.cc:492] T 8c11074579b046d99e585c2818e18a2c P 647a343a542d4a139fe106c19095129b: No bootstrap required, opened a new log
I20260709 13:18:40.125344 16340 ts_tablet_manager.cc:1403] T 8c11074579b046d99e585c2818e18a2c P 647a343a542d4a139fe106c19095129b: Time spent bootstrapping tablet: real 0.067s	user 0.011s	sys 0.001s
I20260709 13:18:40.128024 16340 raft_consensus.cc:359] T 8c11074579b046d99e585c2818e18a2c P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:40.128153 16358 consensus_queue.cc:1048] T c2f5f89186204242bf377f7c42b2e2da P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:18:40.128826 16340 raft_consensus.cc:385] T 8c11074579b046d99e585c2818e18a2c P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:40.129323 16340 raft_consensus.cc:740] T 8c11074579b046d99e585c2818e18a2c P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Initialized, Role: FOLLOWER
I20260709 13:18:40.130007 16340 consensus_queue.cc:260] T 8c11074579b046d99e585c2818e18a2c P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:40.116267 16149 raft_consensus.cc:2468] T 7c2095a5f4ad49408dbc7f69c606120d P 647a343a542d4a139fe106c19095129b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 1.
I20260709 13:18:40.113759 16048 catalog_manager.cc:5697] T 02838c86531d4f98bf58d1c22b8baf77 P 647a343a542d4a139fe106c19095129b reported cstate change: term changed from 0 to 1, leader changed from <none> to 647a343a542d4a139fe106c19095129b (127.14.177.129). New cstate: current_term: 1 leader_uuid: "647a343a542d4a139fe106c19095129b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:40.118917 16347 tablet_bootstrap.cc:492] T 3884f864a96d428785895e5463e5728f P 1ca2725d290e47d68b1a4b609c9816ac: No bootstrap required, opened a new log
I20260709 13:18:40.131831 16347 ts_tablet_manager.cc:1403] T 3884f864a96d428785895e5463e5728f P 1ca2725d290e47d68b1a4b609c9816ac: Time spent bootstrapping tablet: real 0.117s	user 0.014s	sys 0.026s
I20260709 13:18:40.114372 16366 raft_consensus.cc:515] T 5a96373bb0e14dce8a266b2fc6c58507 P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:40.137264 16347 raft_consensus.cc:359] T 3884f864a96d428785895e5463e5728f P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:40.138597 16347 raft_consensus.cc:385] T 3884f864a96d428785895e5463e5728f P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:40.138971 16347 raft_consensus.cc:740] T 3884f864a96d428785895e5463e5728f P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Initialized, Role: FOLLOWER
I20260709 13:18:40.139840 16347 consensus_queue.cc:260] T 3884f864a96d428785895e5463e5728f P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:40.141585 16340 ts_tablet_manager.cc:1434] T 8c11074579b046d99e585c2818e18a2c P 647a343a542d4a139fe106c19095129b: Time spent starting tablet: real 0.016s	user 0.008s	sys 0.000s
I20260709 13:18:40.142259 16347 ts_tablet_manager.cc:1434] T 3884f864a96d428785895e5463e5728f P 1ca2725d290e47d68b1a4b609c9816ac: Time spent starting tablet: real 0.010s	user 0.006s	sys 0.001s
I20260709 13:18:40.144138 16347 tablet_bootstrap.cc:492] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac: Bootstrap starting.
I20260709 13:18:40.144424 16342 tablet_bootstrap.cc:492] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 90f3da57796b488c9087a2f523adf045: No bootstrap required, opened a new log
I20260709 13:18:40.144842 16342 ts_tablet_manager.cc:1403] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 90f3da57796b488c9087a2f523adf045: Time spent bootstrapping tablet: real 0.114s	user 0.011s	sys 0.004s
I20260709 13:18:40.145459 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a96373bb0e14dce8a266b2fc6c58507" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:18:40.146064 16150 raft_consensus.cc:3060] T 5a96373bb0e14dce8a266b2fc6c58507 P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:40.151481 16342 raft_consensus.cc:359] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:40.152175 16342 raft_consensus.cc:385] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:40.152446 16342 raft_consensus.cc:740] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Initialized, Role: FOLLOWER
I20260709 13:18:40.152631 16340 tablet_bootstrap.cc:492] T b6f07842d3aa47858df5665874c86f3a P 647a343a542d4a139fe106c19095129b: Bootstrap starting.
I20260709 13:18:40.153093 16150 raft_consensus.cc:2468] T 5a96373bb0e14dce8a266b2fc6c58507 P 647a343a542d4a139fe106c19095129b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 1.
I20260709 13:18:40.155601 16406 raft_consensus.cc:493] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:40.156157 16406 raft_consensus.cc:515] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:40.157570 16191 leader_election.cc:304] T 5a96373bb0e14dce8a266b2fc6c58507 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
I20260709 13:18:40.158339 16406 leader_election.cc:290] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:18:40.158723 16358 raft_consensus.cc:2804] T 5a96373bb0e14dce8a266b2fc6c58507 P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:18:40.159245 16358 raft_consensus.cc:697] T 5a96373bb0e14dce8a266b2fc6c58507 P 1ca2725d290e47d68b1a4b609c9816ac [term 1 LEADER]: Becoming Leader. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:18:40.159397 16345 raft_consensus.cc:493] T a378d7f796f94615b3b347119a9880fd P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:40.160262 16340 tablet_bootstrap.cc:654] T b6f07842d3aa47858df5665874c86f3a P 647a343a542d4a139fe106c19095129b: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:40.160142 16358 consensus_queue.cc:237] T 5a96373bb0e14dce8a266b2fc6c58507 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:40.161651 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a378d7f796f94615b3b347119a9880fd" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:18:40.159900 16345 raft_consensus.cc:515] T a378d7f796f94615b3b347119a9880fd P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:40.162629 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a378d7f796f94615b3b347119a9880fd" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:18:40.163261 16149 raft_consensus.cc:2468] T a378d7f796f94615b3b347119a9880fd P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 0.
I20260709 13:18:40.163877 16345 leader_election.cc:290] T a378d7f796f94615b3b347119a9880fd P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:18:40.164600 16191 leader_election.cc:304] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
I20260709 13:18:40.165516 16299 raft_consensus.cc:2468] T a378d7f796f94615b3b347119a9880fd P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 0.
I20260709 13:18:40.165343 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a378d7f796f94615b3b347119a9880fd" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:18:40.166656 16149 raft_consensus.cc:2468] T a378d7f796f94615b3b347119a9880fd P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 0.
I20260709 13:18:40.167413 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a378d7f796f94615b3b347119a9880fd" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:18:40.168030 16225 raft_consensus.cc:2468] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 0.
I20260709 13:18:40.168249 16265 leader_election.cc:304] T a378d7f796f94615b3b347119a9880fd P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:18:40.169121 16345 raft_consensus.cc:2804] T a378d7f796f94615b3b347119a9880fd P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:18:40.169469 16345 raft_consensus.cc:493] T a378d7f796f94615b3b347119a9880fd P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:40.169632 16376 raft_consensus.cc:2804] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:18:40.169842 16345 raft_consensus.cc:3060] T a378d7f796f94615b3b347119a9880fd P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:40.170018 16376 raft_consensus.cc:493] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:40.170425 16376 raft_consensus.cc:3060] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:40.180248 16345 raft_consensus.cc:515] T a378d7f796f94615b3b347119a9880fd P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:40.183482 16342 consensus_queue.cc:260] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 90f3da57796b488c9087a2f523adf045 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:40.183952 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a378d7f796f94615b3b347119a9880fd" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:18:40.184504 16149 raft_consensus.cc:3060] T a378d7f796f94615b3b347119a9880fd P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:40.186087 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a378d7f796f94615b3b347119a9880fd" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:18:40.186846 16345 leader_election.cc:290] T a378d7f796f94615b3b347119a9880fd P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 1 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:18:40.182516 16049 catalog_manager.cc:5697] T 7c2095a5f4ad49408dbc7f69c606120d P 1ca2725d290e47d68b1a4b609c9816ac reported cstate change: term changed from 0 to 1, leader changed from <none> to 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130). New cstate: current_term: 1 leader_uuid: "1ca2725d290e47d68b1a4b609c9816ac" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:40.195757 16224 raft_consensus.cc:3060] T 434bff3082d141838068f323bdb0d1ed P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:40.203076 16373 consensus_queue.cc:1048] T 434bff3082d141838068f323bdb0d1ed P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:18:40.207608 16406 raft_consensus.cc:493] T 434bff3082d141838068f323bdb0d1ed P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:40.208156 16406 raft_consensus.cc:515] T 434bff3082d141838068f323bdb0d1ed P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:18:40.212906 16097 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:18:40.213130 16149 raft_consensus.cc:2468] T a378d7f796f94615b3b347119a9880fd P 647a343a542d4a139fe106c19095129b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 1.
I20260709 13:18:40.216848 16431 raft_consensus.cc:493] T 3884f864a96d428785895e5463e5728f P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:40.219451 16406 leader_election.cc:290] T 434bff3082d141838068f323bdb0d1ed P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:18:40.254262 16265 leader_election.cc:304] T a378d7f796f94615b3b347119a9880fd P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:18:40.221856 16347 tablet_bootstrap.cc:654] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:40.221227 16151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434bff3082d141838068f323bdb0d1ed" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:18:40.255968 16345 raft_consensus.cc:2804] T a378d7f796f94615b3b347119a9880fd P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:18:40.256505 16345 raft_consensus.cc:697] T a378d7f796f94615b3b347119a9880fd P 90f3da57796b488c9087a2f523adf045 [term 1 LEADER]: Becoming Leader. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:18:40.239356 16430 consensus_queue.cc:1048] T 8a878999c0024d2abe517484180290fc P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.014s
I20260709 13:18:40.218050 16376 raft_consensus.cc:515] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:40.239668 16366 leader_election.cc:290] T 5a96373bb0e14dce8a266b2fc6c58507 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 1 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:18:40.257567 16345 consensus_queue.cc:237] T a378d7f796f94615b3b347119a9880fd P 90f3da57796b488c9087a2f523adf045 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:40.228951 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a96373bb0e14dce8a266b2fc6c58507" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:18:40.220927 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434bff3082d141838068f323bdb0d1ed" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:18:40.264012 16048 catalog_manager.cc:5697] T 5a96373bb0e14dce8a266b2fc6c58507 P 1ca2725d290e47d68b1a4b609c9816ac reported cstate change: term changed from 0 to 1, leader changed from <none> to 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130). New cstate: current_term: 1 leader_uuid: "1ca2725d290e47d68b1a4b609c9816ac" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:40.265383 16225 raft_consensus.cc:2393] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 1: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
I20260709 13:18:40.226110 16373 consensus_queue.cc:1048] T 434bff3082d141838068f323bdb0d1ed P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.040s
I20260709 13:18:40.267467 16342 ts_tablet_manager.cc:1434] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 90f3da57796b488c9087a2f523adf045: Time spent starting tablet: real 0.122s	user 0.016s	sys 0.027s
I20260709 13:18:40.268791 16342 tablet_bootstrap.cc:492] T ada5a004ba7c46819051e3ccec71ffdb P 90f3da57796b488c9087a2f523adf045: Bootstrap starting.
I20260709 13:18:40.254215 16431 raft_consensus.cc:515] T 3884f864a96d428785895e5463e5728f P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:40.262331 16298 raft_consensus.cc:3060] T 5a96373bb0e14dce8a266b2fc6c58507 P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:40.274649 16190 leader_election.cc:304] T 434bff3082d141838068f323bdb0d1ed P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:18:40.271904 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a378d7f796f94615b3b347119a9880fd" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:18:40.270462 16376 leader_election.cc:290] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 1 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:18:40.297433 16150 raft_consensus.cc:2393] T a378d7f796f94615b3b347119a9880fd P 647a343a542d4a139fe106c19095129b [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 1: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
I20260709 13:18:40.298007 16406 raft_consensus.cc:2749] T 434bff3082d141838068f323bdb0d1ed P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:18:40.298909 16431 leader_election.cc:290] T 3884f864a96d428785895e5463e5728f P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:18:40.299311 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a378d7f796f94615b3b347119a9880fd" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:18:40.312577 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3884f864a96d428785895e5463e5728f" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
W20260709 13:18:40.314548 16190 leader_election.cc:343] T 3884f864a96d428785895e5463e5728f P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:18:40.317549 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3884f864a96d428785895e5463e5728f" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:18:40.317726 16191 leader_election.cc:304] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:18:40.321043 16406 raft_consensus.cc:2749] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:18:40.321646 16406 consensus_queue.cc:1048] T 8a878999c0024d2abe517484180290fc P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:18:40.324286 16415 consensus_queue.cc:1048] T 9e65676b21b748728ad39668cfe722ec P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 13:18:40.325029 16191 leader_election.cc:343] T 3884f864a96d428785895e5463e5728f P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:18:40.310454 16430 consensus_queue.cc:1048] T fffab26427e24d398e2695aad40ca134 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 13:18:40.325487 16191 leader_election.cc:304] T 3884f864a96d428785895e5463e5728f P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:18:40.326217 16431 raft_consensus.cc:2749] T 3884f864a96d428785895e5463e5728f P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:18:40.329931 16049 catalog_manager.cc:5697] T a378d7f796f94615b3b347119a9880fd P 90f3da57796b488c9087a2f523adf045 reported cstate change: term changed from 0 to 1, leader changed from <none> to 90f3da57796b488c9087a2f523adf045 (127.14.177.131). New cstate: current_term: 1 leader_uuid: "90f3da57796b488c9087a2f523adf045" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } health_report { overall_health: HEALTHY } } }
I20260709 13:18:40.331687 16347 tablet_bootstrap.cc:492] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac: No bootstrap required, opened a new log
I20260709 13:18:40.332357 16347 ts_tablet_manager.cc:1403] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac: Time spent bootstrapping tablet: real 0.189s	user 0.010s	sys 0.019s
I20260709 13:18:40.337285 16431 consensus_queue.cc:1048] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:18:40.352936 16427 raft_consensus.cc:493] T 68a6838d82ef4cac83507f906139d90f P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:40.355387 16427 raft_consensus.cc:515] T 68a6838d82ef4cac83507f906139d90f P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:40.355660 16396 consensus_queue.cc:1048] T 9e65676b21b748728ad39668cfe722ec P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:18:40.357861 16427 leader_election.cc:290] T 68a6838d82ef4cac83507f906139d90f P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:18:40.362011 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68a6838d82ef4cac83507f906139d90f" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:18:40.362231 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68a6838d82ef4cac83507f906139d90f" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:18:40.362227 16340 tablet_bootstrap.cc:492] T b6f07842d3aa47858df5665874c86f3a P 647a343a542d4a139fe106c19095129b: No bootstrap required, opened a new log
I20260709 13:18:40.366741 16342 tablet_bootstrap.cc:654] T ada5a004ba7c46819051e3ccec71ffdb P 90f3da57796b488c9087a2f523adf045: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:40.364224 16340 ts_tablet_manager.cc:1403] T b6f07842d3aa47858df5665874c86f3a P 647a343a542d4a139fe106c19095129b: Time spent bootstrapping tablet: real 0.212s	user 0.024s	sys 0.041s
W20260709 13:18:40.363631 16265 leader_election.cc:343] T 68a6838d82ef4cac83507f906139d90f P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:18:40.369764 16265 leader_election.cc:343] T 68a6838d82ef4cac83507f906139d90f P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:18:40.370383 16265 leader_election.cc:304] T 68a6838d82ef4cac83507f906139d90f P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:18:40.370714 16347 raft_consensus.cc:359] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:40.371393 16347 raft_consensus.cc:385] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:40.371706 16347 raft_consensus.cc:740] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Initialized, Role: FOLLOWER
I20260709 13:18:40.372414 16347 consensus_queue.cc:260] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:40.375607 16427 raft_consensus.cc:2749] T 68a6838d82ef4cac83507f906139d90f P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:18:40.381911 16298 raft_consensus.cc:2468] T 5a96373bb0e14dce8a266b2fc6c58507 P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 1.
I20260709 13:18:40.392840 16430 consensus_queue.cc:1048] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:18:40.416016 16347 ts_tablet_manager.cc:1434] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac: Time spent starting tablet: real 0.083s	user 0.007s	sys 0.001s
I20260709 13:18:40.420301 16340 raft_consensus.cc:359] T b6f07842d3aa47858df5665874c86f3a P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:40.422349 16340 raft_consensus.cc:385] T b6f07842d3aa47858df5665874c86f3a P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:40.422777 16340 raft_consensus.cc:740] T b6f07842d3aa47858df5665874c86f3a P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Initialized, Role: FOLLOWER
I20260709 13:18:40.423743 16340 consensus_queue.cc:260] T b6f07842d3aa47858df5665874c86f3a P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:40.421617 16396 consensus_queue.cc:1048] T 8cc959fb7f704fba9009c0595467d074 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:18:40.438571 16342 tablet_bootstrap.cc:492] T ada5a004ba7c46819051e3ccec71ffdb P 90f3da57796b488c9087a2f523adf045: No bootstrap required, opened a new log
I20260709 13:18:40.439352 16342 ts_tablet_manager.cc:1403] T ada5a004ba7c46819051e3ccec71ffdb P 90f3da57796b488c9087a2f523adf045: Time spent bootstrapping tablet: real 0.171s	user 0.042s	sys 0.016s
I20260709 13:18:40.445225 16347 tablet_bootstrap.cc:492] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 1ca2725d290e47d68b1a4b609c9816ac: Bootstrap starting.
I20260709 13:18:40.455289 16366 consensus_queue.cc:1048] T fffab26427e24d398e2695aad40ca134 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 13:18:40.457561 16115 consensus_peers.cc:597] T 02838c86531d4f98bf58d1c22b8baf77 P 647a343a542d4a139fe106c19095129b -> Peer 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389): Couldn't send request to peer 90f3da57796b488c9087a2f523adf045. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:18:40.460767 16340 ts_tablet_manager.cc:1434] T b6f07842d3aa47858df5665874c86f3a P 647a343a542d4a139fe106c19095129b: Time spent starting tablet: real 0.092s	user 0.003s	sys 0.004s
I20260709 13:18:40.461894 16340 tablet_bootstrap.cc:492] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 647a343a542d4a139fe106c19095129b: Bootstrap starting.
I20260709 13:18:40.467463 16344 consensus_queue.cc:1048] T 8cc959fb7f704fba9009c0595467d074 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:18:40.474828 16340 tablet_bootstrap.cc:654] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 647a343a542d4a139fe106c19095129b: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:40.477118 16347 tablet_bootstrap.cc:654] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 1ca2725d290e47d68b1a4b609c9816ac: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:40.479565 16427 raft_consensus.cc:493] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:40.479529 16342 raft_consensus.cc:359] T ada5a004ba7c46819051e3ccec71ffdb P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:40.480386 16427 raft_consensus.cc:515] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:40.480748 16342 raft_consensus.cc:385] T ada5a004ba7c46819051e3ccec71ffdb P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:40.481208 16342 raft_consensus.cc:740] T ada5a004ba7c46819051e3ccec71ffdb P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Initialized, Role: FOLLOWER
I20260709 13:18:40.482486 16427 leader_election.cc:290] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:18:40.482165 16342 consensus_queue.cc:260] T ada5a004ba7c46819051e3ccec71ffdb P 90f3da57796b488c9087a2f523adf045 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:40.485004 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7ca61592e848ea9b0fb81a2ce33cdc" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:18:40.485328 16342 ts_tablet_manager.cc:1434] T ada5a004ba7c46819051e3ccec71ffdb P 90f3da57796b488c9087a2f523adf045: Time spent starting tablet: real 0.046s	user 0.008s	sys 0.000s
I20260709 13:18:40.486606 16342 tablet_bootstrap.cc:492] T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045: Bootstrap starting.
I20260709 13:18:40.486702 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7ca61592e848ea9b0fb81a2ce33cdc" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
W20260709 13:18:40.488118 16265 leader_election.cc:343] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:18:40.488740 16265 leader_election.cc:343] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:18:40.489176 16265 leader_election.cc:304] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:18:40.490413 16345 raft_consensus.cc:2749] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:18:40.496731 16366 raft_consensus.cc:493] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:40.497228 16366 raft_consensus.cc:515] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:40.564807 16151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada5a004ba7c46819051e3ccec71ffdb" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:18:40.565625 16347 tablet_bootstrap.cc:492] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 1ca2725d290e47d68b1a4b609c9816ac: No bootstrap required, opened a new log
I20260709 13:18:40.566113 16347 ts_tablet_manager.cc:1403] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 1ca2725d290e47d68b1a4b609c9816ac: Time spent bootstrapping tablet: real 0.121s	user 0.021s	sys 0.042s
I20260709 13:18:40.566143 16366 leader_election.cc:290] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:18:40.566489 16342 tablet_bootstrap.cc:654] T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:40.567770 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada5a004ba7c46819051e3ccec71ffdb" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:18:40.568385 16299 raft_consensus.cc:2468] T ada5a004ba7c46819051e3ccec71ffdb P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 0.
I20260709 13:18:40.568868 16347 raft_consensus.cc:359] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:40.569511 16347 raft_consensus.cc:385] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:40.569769 16347 raft_consensus.cc:740] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Initialized, Role: FOLLOWER
I20260709 13:18:40.570446 16347 consensus_queue.cc:260] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:40.571959 16190 leader_election.cc:304] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:18:40.573365 16340 tablet_bootstrap.cc:492] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 647a343a542d4a139fe106c19095129b: No bootstrap required, opened a new log
I20260709 13:18:40.573809 16340 ts_tablet_manager.cc:1403] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 647a343a542d4a139fe106c19095129b: Time spent bootstrapping tablet: real 0.112s	user 0.012s	sys 0.032s
W20260709 13:18:40.574563 16191 leader_election.cc:343] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:18:40.576769 16340 raft_consensus.cc:359] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:40.577523 16340 raft_consensus.cc:385] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:40.577829 16431 raft_consensus.cc:2804] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:18:40.577883 16366 consensus_queue.cc:1048] T 5a96373bb0e14dce8a266b2fc6c58507 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:18:40.578366 16431 raft_consensus.cc:493] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:40.578732 16431 raft_consensus.cc:3060] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:40.577809 16340 raft_consensus.cc:740] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Initialized, Role: FOLLOWER
I20260709 13:18:40.589036 16340 consensus_queue.cc:260] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:40.591629 16340 ts_tablet_manager.cc:1434] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 647a343a542d4a139fe106c19095129b: Time spent starting tablet: real 0.018s	user 0.007s	sys 0.001s
I20260709 13:18:40.592590 16347 ts_tablet_manager.cc:1434] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 1ca2725d290e47d68b1a4b609c9816ac: Time spent starting tablet: real 0.026s	user 0.007s	sys 0.000s
I20260709 13:18:40.593616 16347 tablet_bootstrap.cc:492] T 8c11074579b046d99e585c2818e18a2c P 1ca2725d290e47d68b1a4b609c9816ac: Bootstrap starting.
I20260709 13:18:40.598937 16342 tablet_bootstrap.cc:492] T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045: No bootstrap required, opened a new log
I20260709 13:18:40.599488 16430 consensus_queue.cc:1048] T 5a96373bb0e14dce8a266b2fc6c58507 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:18:40.599507 16342 ts_tablet_manager.cc:1403] T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045: Time spent bootstrapping tablet: real 0.113s	user 0.007s	sys 0.015s
I20260709 13:18:40.602941 16342 raft_consensus.cc:359] T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:40.603754 16342 raft_consensus.cc:385] T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:40.604102 16342 raft_consensus.cc:740] T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Initialized, Role: FOLLOWER
I20260709 13:18:40.605011 16342 consensus_queue.cc:260] T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:40.605427 16344 consensus_queue.cc:1048] T 02838c86531d4f98bf58d1c22b8baf77 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:18:40.607899 16342 ts_tablet_manager.cc:1434] T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.001s
I20260709 13:18:40.609720 16431 raft_consensus.cc:515] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:40.609130 16342 tablet_bootstrap.cc:492] T 02838c86531d4f98bf58d1c22b8baf77 P 90f3da57796b488c9087a2f523adf045: Bootstrap starting.
I20260709 13:18:40.611511 16340 tablet_bootstrap.cc:492] T 3884f864a96d428785895e5463e5728f P 647a343a542d4a139fe106c19095129b: Bootstrap starting.
I20260709 13:18:40.618340 16340 tablet_bootstrap.cc:654] T 3884f864a96d428785895e5463e5728f P 647a343a542d4a139fe106c19095129b: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:40.620510 16342 tablet_bootstrap.cc:654] T 02838c86531d4f98bf58d1c22b8baf77 P 90f3da57796b488c9087a2f523adf045: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:40.621479 16151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada5a004ba7c46819051e3ccec71ffdb" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:18:40.621577 16431 leader_election.cc:290] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 1 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:18:40.622797 16296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada5a004ba7c46819051e3ccec71ffdb" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
W20260709 13:18:40.623108 16191 leader_election.cc:343] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:18:40.623497 16296 raft_consensus.cc:3060] T ada5a004ba7c46819051e3ccec71ffdb P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:40.632011 16344 raft_consensus.cc:493] T 8c11074579b046d99e585c2818e18a2c P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:40.632789 16344 raft_consensus.cc:515] T 8c11074579b046d99e585c2818e18a2c P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:40.633653 16296 raft_consensus.cc:2468] T ada5a004ba7c46819051e3ccec71ffdb P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 1.
I20260709 13:18:40.635869 16190 leader_election.cc:304] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 647a343a542d4a139fe106c19095129b
I20260709 13:18:40.637905 16430 raft_consensus.cc:2804] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:18:40.638499 16430 raft_consensus.cc:697] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac [term 1 LEADER]: Becoming Leader. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:18:40.639374 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c11074579b046d99e585c2818e18a2c" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:18:40.640393 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c11074579b046d99e585c2818e18a2c" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
W20260709 13:18:40.641964 16115 leader_election.cc:343] T 8c11074579b046d99e585c2818e18a2c P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:18:40.642158 16430 consensus_queue.cc:237] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:40.645936 16358 consensus_queue.cc:1048] T 7c2095a5f4ad49408dbc7f69c606120d P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 13:18:40.646555 16116 leader_election.cc:343] T 8c11074579b046d99e585c2818e18a2c P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:18:40.647033 16116 leader_election.cc:304] T 8c11074579b046d99e585c2818e18a2c P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:18:40.647892 16415 raft_consensus.cc:2749] T 8c11074579b046d99e585c2818e18a2c P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:18:40.649047 16344 leader_election.cc:290] T 8c11074579b046d99e585c2818e18a2c P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:18:40.651746 16347 tablet_bootstrap.cc:654] T 8c11074579b046d99e585c2818e18a2c P 1ca2725d290e47d68b1a4b609c9816ac: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:40.651669 16358 consensus_queue.cc:1048] T 7c2095a5f4ad49408dbc7f69c606120d P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:18:40.681528 16427 consensus_queue.cc:1048] T a378d7f796f94615b3b347119a9880fd P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:18:40.688608 16342 tablet_bootstrap.cc:492] T 02838c86531d4f98bf58d1c22b8baf77 P 90f3da57796b488c9087a2f523adf045: No bootstrap required, opened a new log
I20260709 13:18:40.689121 16342 ts_tablet_manager.cc:1403] T 02838c86531d4f98bf58d1c22b8baf77 P 90f3da57796b488c9087a2f523adf045: Time spent bootstrapping tablet: real 0.080s	user 0.027s	sys 0.009s
I20260709 13:18:40.708580 16342 raft_consensus.cc:359] T 02838c86531d4f98bf58d1c22b8baf77 P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:40.711503 16342 raft_consensus.cc:385] T 02838c86531d4f98bf58d1c22b8baf77 P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:40.711926 16342 raft_consensus.cc:740] T 02838c86531d4f98bf58d1c22b8baf77 P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Initialized, Role: FOLLOWER
I20260709 13:18:40.714803 16342 consensus_queue.cc:260] T 02838c86531d4f98bf58d1c22b8baf77 P 90f3da57796b488c9087a2f523adf045 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:40.716444 16347 tablet_bootstrap.cc:492] T 8c11074579b046d99e585c2818e18a2c P 1ca2725d290e47d68b1a4b609c9816ac: No bootstrap required, opened a new log
I20260709 13:18:40.716930 16347 ts_tablet_manager.cc:1403] T 8c11074579b046d99e585c2818e18a2c P 1ca2725d290e47d68b1a4b609c9816ac: Time spent bootstrapping tablet: real 0.124s	user 0.009s	sys 0.003s
I20260709 13:18:40.722568 16427 consensus_queue.cc:1048] T a378d7f796f94615b3b347119a9880fd P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:18:40.722518 16347 raft_consensus.cc:359] T 8c11074579b046d99e585c2818e18a2c P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:40.723459 16347 raft_consensus.cc:385] T 8c11074579b046d99e585c2818e18a2c P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:40.723754 16347 raft_consensus.cc:740] T 8c11074579b046d99e585c2818e18a2c P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Initialized, Role: FOLLOWER
I20260709 13:18:40.724404 16347 consensus_queue.cc:260] T 8c11074579b046d99e585c2818e18a2c P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:40.726266 16342 ts_tablet_manager.cc:1434] T 02838c86531d4f98bf58d1c22b8baf77 P 90f3da57796b488c9087a2f523adf045: Time spent starting tablet: real 0.037s	user 0.010s	sys 0.000s
I20260709 13:18:40.727286 16342 tablet_bootstrap.cc:492] T 8c11074579b046d99e585c2818e18a2c P 90f3da57796b488c9087a2f523adf045: Bootstrap starting.
I20260709 13:18:40.733224 16347 ts_tablet_manager.cc:1434] T 8c11074579b046d99e585c2818e18a2c P 1ca2725d290e47d68b1a4b609c9816ac: Time spent starting tablet: real 0.016s	user 0.007s	sys 0.000s
I20260709 13:18:40.738862 16347 tablet_bootstrap.cc:492] T b6f07842d3aa47858df5665874c86f3a P 1ca2725d290e47d68b1a4b609c9816ac: Bootstrap starting.
I20260709 13:18:40.746085 16342 tablet_bootstrap.cc:654] T 8c11074579b046d99e585c2818e18a2c P 90f3da57796b488c9087a2f523adf045: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:40.745963 16427 raft_consensus.cc:493] T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:40.747429 16427 raft_consensus.cc:515] T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:40.750154 16427 leader_election.cc:290] T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:18:40.737700 16046 catalog_manager.cc:5697] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac reported cstate change: term changed from 0 to 1, leader changed from <none> to 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130). New cstate: current_term: 1 leader_uuid: "1ca2725d290e47d68b1a4b609c9816ac" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:40.752859 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6f07842d3aa47858df5665874c86f3a" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:18:40.752787 16342 tablet_bootstrap.cc:492] T 8c11074579b046d99e585c2818e18a2c P 90f3da57796b488c9087a2f523adf045: No bootstrap required, opened a new log
I20260709 13:18:40.753717 16342 ts_tablet_manager.cc:1403] T 8c11074579b046d99e585c2818e18a2c P 90f3da57796b488c9087a2f523adf045: Time spent bootstrapping tablet: real 0.027s	user 0.011s	sys 0.003s
I20260709 13:18:40.753886 16340 tablet_bootstrap.cc:492] T 3884f864a96d428785895e5463e5728f P 647a343a542d4a139fe106c19095129b: No bootstrap required, opened a new log
I20260709 13:18:40.750819 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6f07842d3aa47858df5665874c86f3a" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:18:40.754297 16340 ts_tablet_manager.cc:1403] T 3884f864a96d428785895e5463e5728f P 647a343a542d4a139fe106c19095129b: Time spent bootstrapping tablet: real 0.143s	user 0.038s	sys 0.019s
I20260709 13:18:40.754678 16150 raft_consensus.cc:2468] T b6f07842d3aa47858df5665874c86f3a P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 0.
I20260709 13:18:40.756567 16265 leader_election.cc:304] T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
W20260709 13:18:40.757544 16265 leader_election.cc:343] T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:18:40.758522 16342 raft_consensus.cc:359] T 8c11074579b046d99e585c2818e18a2c P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:40.759411 16342 raft_consensus.cc:385] T 8c11074579b046d99e585c2818e18a2c P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:40.759720 16342 raft_consensus.cc:740] T 8c11074579b046d99e585c2818e18a2c P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Initialized, Role: FOLLOWER
I20260709 13:18:40.760386 16342 consensus_queue.cc:260] T 8c11074579b046d99e585c2818e18a2c P 90f3da57796b488c9087a2f523adf045 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:40.761390 16345 raft_consensus.cc:2804] T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:18:40.761806 16345 raft_consensus.cc:493] T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:40.762164 16345 raft_consensus.cc:3060] T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:40.762816 16342 ts_tablet_manager.cc:1434] T 8c11074579b046d99e585c2818e18a2c P 90f3da57796b488c9087a2f523adf045: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.000s
I20260709 13:18:40.763684 16342 tablet_bootstrap.cc:492] T d2ab0ac8347641e6955e9bce7d90c1a2 P 90f3da57796b488c9087a2f523adf045: Bootstrap starting.
I20260709 13:18:40.767469 16376 raft_consensus.cc:493] T 8c11074579b046d99e585c2818e18a2c P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:40.768002 16376 raft_consensus.cc:515] T 8c11074579b046d99e585c2818e18a2c P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:40.770277 16376 leader_election.cc:290] T 8c11074579b046d99e585c2818e18a2c P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:18:40.773170 16151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c11074579b046d99e585c2818e18a2c" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:18:40.773847 16151 raft_consensus.cc:2468] T 8c11074579b046d99e585c2818e18a2c P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 0.
I20260709 13:18:40.773190 16296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c11074579b046d99e585c2818e18a2c" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:18:40.746376 16347 tablet_bootstrap.cc:654] T b6f07842d3aa47858df5665874c86f3a P 1ca2725d290e47d68b1a4b609c9816ac: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:40.775223 16191 leader_election.cc:304] T 8c11074579b046d99e585c2818e18a2c P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
I20260709 13:18:40.775674 16342 tablet_bootstrap.cc:654] T d2ab0ac8347641e6955e9bce7d90c1a2 P 90f3da57796b488c9087a2f523adf045: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:40.775245 16296 raft_consensus.cc:2468] T 8c11074579b046d99e585c2818e18a2c P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 0.
I20260709 13:18:40.777175 16376 raft_consensus.cc:2804] T 8c11074579b046d99e585c2818e18a2c P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:18:40.757337 16340 raft_consensus.cc:359] T 3884f864a96d428785895e5463e5728f P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:40.777643 16376 raft_consensus.cc:493] T 8c11074579b046d99e585c2818e18a2c P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:40.777908 16340 raft_consensus.cc:385] T 3884f864a96d428785895e5463e5728f P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:40.778149 16376 raft_consensus.cc:3060] T 8c11074579b046d99e585c2818e18a2c P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:40.771787 16345 raft_consensus.cc:515] T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:40.783022 16151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6f07842d3aa47858df5665874c86f3a" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:18:40.783740 16151 raft_consensus.cc:3060] T b6f07842d3aa47858df5665874c86f3a P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:40.786233 16376 raft_consensus.cc:515] T 8c11074579b046d99e585c2818e18a2c P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:40.789094 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c11074579b046d99e585c2818e18a2c" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:18:40.789698 16150 raft_consensus.cc:3060] T 8c11074579b046d99e585c2818e18a2c P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:40.794271 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6f07842d3aa47858df5665874c86f3a" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:18:40.795322 16430 raft_consensus.cc:493] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:40.795800 16430 raft_consensus.cc:515] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:18:40.796180 16265 leader_election.cc:343] T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:18:40.796825 16345 leader_election.cc:290] T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 1 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:18:40.797849 16430 leader_election.cc:290] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:18:40.778241 16340 raft_consensus.cc:740] T 3884f864a96d428785895e5463e5728f P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Initialized, Role: FOLLOWER
I20260709 13:18:40.798708 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7ca61592e848ea9b0fb81a2ce33cdc" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:18:40.798969 16340 consensus_queue.cc:260] T 3884f864a96d428785895e5463e5728f P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:40.799398 16149 raft_consensus.cc:2468] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 0.
I20260709 13:18:40.800217 16151 raft_consensus.cc:2468] T b6f07842d3aa47858df5665874c86f3a P 647a343a542d4a139fe106c19095129b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 1.
I20260709 13:18:40.802732 16265 leader_election.cc:304] T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac
I20260709 13:18:40.804263 16345 raft_consensus.cc:2804] T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:18:40.800892 16296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7ca61592e848ea9b0fb81a2ce33cdc" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:18:40.805545 16296 raft_consensus.cc:2468] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 0.
I20260709 13:18:40.806425 16345 raft_consensus.cc:697] T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045 [term 1 LEADER]: Becoming Leader. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:18:40.806825 16190 leader_election.cc:304] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:18:40.807752 16430 raft_consensus.cc:2804] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:18:40.808177 16430 raft_consensus.cc:493] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:40.808547 16430 raft_consensus.cc:3060] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:40.807809 16345 consensus_queue.cc:237] T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:40.818547 16430 raft_consensus.cc:515] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:40.821774 16340 ts_tablet_manager.cc:1434] T 3884f864a96d428785895e5463e5728f P 647a343a542d4a139fe106c19095129b: Time spent starting tablet: real 0.067s	user 0.008s	sys 0.000s
I20260709 13:18:40.822049 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7ca61592e848ea9b0fb81a2ce33cdc" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:18:40.822636 16149 raft_consensus.cc:3060] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:40.822897 16340 tablet_bootstrap.cc:492] T 68a6838d82ef4cac83507f906139d90f P 647a343a542d4a139fe106c19095129b: Bootstrap starting.
I20260709 13:18:40.834491 16149 raft_consensus.cc:2468] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 647a343a542d4a139fe106c19095129b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 1.
I20260709 13:18:40.835222 16150 raft_consensus.cc:2468] T 8c11074579b046d99e585c2818e18a2c P 647a343a542d4a139fe106c19095129b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 1.
I20260709 13:18:40.836778 16191 leader_election.cc:304] T 8c11074579b046d99e585c2818e18a2c P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
I20260709 13:18:40.837685 16406 raft_consensus.cc:2804] T 8c11074579b046d99e585c2818e18a2c P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:18:40.838301 16406 raft_consensus.cc:697] T 8c11074579b046d99e585c2818e18a2c P 1ca2725d290e47d68b1a4b609c9816ac [term 1 LEADER]: Becoming Leader. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:18:40.840143 16296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c11074579b046d99e585c2818e18a2c" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:18:40.839408 16406 consensus_queue.cc:237] T 8c11074579b046d99e585c2818e18a2c P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:40.840874 16296 raft_consensus.cc:3060] T 8c11074579b046d99e585c2818e18a2c P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:40.849030 16376 leader_election.cc:290] T 8c11074579b046d99e585c2818e18a2c P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 1 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:18:40.851791 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7ca61592e848ea9b0fb81a2ce33cdc" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:18:40.852578 16299 raft_consensus.cc:3060] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:40.858453 16340 tablet_bootstrap.cc:654] T 68a6838d82ef4cac83507f906139d90f P 647a343a542d4a139fe106c19095129b: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:40.860402 16299 raft_consensus.cc:2468] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 1.
I20260709 13:18:40.861629 16190 leader_election.cc:304] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:18:40.860893 16045 catalog_manager.cc:5697] T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045 reported cstate change: term changed from 0 to 1, leader changed from <none> to 90f3da57796b488c9087a2f523adf045 (127.14.177.131). New cstate: current_term: 1 leader_uuid: "90f3da57796b488c9087a2f523adf045" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } health_report { overall_health: HEALTHY } } }
I20260709 13:18:40.862560 16406 raft_consensus.cc:2804] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:18:40.863183 16406 raft_consensus.cc:697] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 1ca2725d290e47d68b1a4b609c9816ac [term 1 LEADER]: Becoming Leader. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:18:40.862551 16048 catalog_manager.cc:5697] T 8c11074579b046d99e585c2818e18a2c P 1ca2725d290e47d68b1a4b609c9816ac reported cstate change: term changed from 0 to 1, leader changed from <none> to 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130). New cstate: current_term: 1 leader_uuid: "1ca2725d290e47d68b1a4b609c9816ac" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:40.864065 16406 consensus_queue.cc:237] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:40.867796 16342 tablet_bootstrap.cc:492] T d2ab0ac8347641e6955e9bce7d90c1a2 P 90f3da57796b488c9087a2f523adf045: No bootstrap required, opened a new log
I20260709 13:18:40.868449 16342 ts_tablet_manager.cc:1403] T d2ab0ac8347641e6955e9bce7d90c1a2 P 90f3da57796b488c9087a2f523adf045: Time spent bootstrapping tablet: real 0.105s	user 0.031s	sys 0.046s
I20260709 13:18:40.871981 16342 raft_consensus.cc:359] T d2ab0ac8347641e6955e9bce7d90c1a2 P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:40.872987 16342 raft_consensus.cc:385] T d2ab0ac8347641e6955e9bce7d90c1a2 P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:40.873404 16342 raft_consensus.cc:740] T d2ab0ac8347641e6955e9bce7d90c1a2 P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Initialized, Role: FOLLOWER
I20260709 13:18:40.873852 16296 raft_consensus.cc:2468] T 8c11074579b046d99e585c2818e18a2c P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 1.
I20260709 13:18:40.874415 16342 consensus_queue.cc:260] T d2ab0ac8347641e6955e9bce7d90c1a2 P 90f3da57796b488c9087a2f523adf045 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:40.876190 16430 leader_election.cc:290] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 1 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:18:40.878057 16342 ts_tablet_manager.cc:1434] T d2ab0ac8347641e6955e9bce7d90c1a2 P 90f3da57796b488c9087a2f523adf045: Time spent starting tablet: real 0.009s	user 0.008s	sys 0.000s
I20260709 13:18:40.879436 16347 tablet_bootstrap.cc:492] T b6f07842d3aa47858df5665874c86f3a P 1ca2725d290e47d68b1a4b609c9816ac: No bootstrap required, opened a new log
I20260709 13:18:40.879505 16342 tablet_bootstrap.cc:492] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 90f3da57796b488c9087a2f523adf045: Bootstrap starting.
I20260709 13:18:40.879896 16347 ts_tablet_manager.cc:1403] T b6f07842d3aa47858df5665874c86f3a P 1ca2725d290e47d68b1a4b609c9816ac: Time spent bootstrapping tablet: real 0.146s	user 0.018s	sys 0.009s
I20260709 13:18:40.882591 16347 raft_consensus.cc:359] T b6f07842d3aa47858df5665874c86f3a P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:40.883225 16347 raft_consensus.cc:385] T b6f07842d3aa47858df5665874c86f3a P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:40.883484 16347 raft_consensus.cc:740] T b6f07842d3aa47858df5665874c86f3a P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Initialized, Role: FOLLOWER
I20260709 13:18:40.884135 16347 consensus_queue.cc:260] T b6f07842d3aa47858df5665874c86f3a P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:40.886050 16347 ts_tablet_manager.cc:1434] T b6f07842d3aa47858df5665874c86f3a P 1ca2725d290e47d68b1a4b609c9816ac: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 13:18:40.886865 16347 tablet_bootstrap.cc:492] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 1ca2725d290e47d68b1a4b609c9816ac: Bootstrap starting.
I20260709 13:18:40.889204 16342 tablet_bootstrap.cc:654] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 90f3da57796b488c9087a2f523adf045: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:40.897132 16347 tablet_bootstrap.cc:654] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 1ca2725d290e47d68b1a4b609c9816ac: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:40.898407 16340 tablet_bootstrap.cc:492] T 68a6838d82ef4cac83507f906139d90f P 647a343a542d4a139fe106c19095129b: No bootstrap required, opened a new log
I20260709 13:18:40.898713 16342 tablet_bootstrap.cc:492] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 90f3da57796b488c9087a2f523adf045: No bootstrap required, opened a new log
I20260709 13:18:40.899055 16340 ts_tablet_manager.cc:1403] T 68a6838d82ef4cac83507f906139d90f P 647a343a542d4a139fe106c19095129b: Time spent bootstrapping tablet: real 0.076s	user 0.030s	sys 0.006s
I20260709 13:18:40.899714 16342 ts_tablet_manager.cc:1403] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 90f3da57796b488c9087a2f523adf045: Time spent bootstrapping tablet: real 0.021s	user 0.018s	sys 0.000s
I20260709 13:18:40.902045 16340 raft_consensus.cc:359] T 68a6838d82ef4cac83507f906139d90f P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:40.902870 16340 raft_consensus.cc:385] T 68a6838d82ef4cac83507f906139d90f P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:40.902504 16342 raft_consensus.cc:359] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:40.903394 16342 raft_consensus.cc:385] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:40.903801 16342 raft_consensus.cc:740] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Initialized, Role: FOLLOWER
I20260709 13:18:40.903630 16045 catalog_manager.cc:5697] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 1ca2725d290e47d68b1a4b609c9816ac reported cstate change: term changed from 0 to 1, leader changed from <none> to 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130). New cstate: current_term: 1 leader_uuid: "1ca2725d290e47d68b1a4b609c9816ac" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:40.904623 16340 raft_consensus.cc:740] T 68a6838d82ef4cac83507f906139d90f P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Initialized, Role: FOLLOWER
I20260709 13:18:40.904637 16342 consensus_queue.cc:260] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 90f3da57796b488c9087a2f523adf045 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:40.906256 16347 tablet_bootstrap.cc:492] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 1ca2725d290e47d68b1a4b609c9816ac: No bootstrap required, opened a new log
I20260709 13:18:40.906736 16347 ts_tablet_manager.cc:1403] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 1ca2725d290e47d68b1a4b609c9816ac: Time spent bootstrapping tablet: real 0.020s	user 0.009s	sys 0.006s
I20260709 13:18:40.906934 16342 ts_tablet_manager.cc:1434] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 90f3da57796b488c9087a2f523adf045: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.001s
I20260709 13:18:40.905637 16340 consensus_queue.cc:260] T 68a6838d82ef4cac83507f906139d90f P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:40.909448 16347 raft_consensus.cc:359] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:40.910327 16347 raft_consensus.cc:385] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:40.910635 16347 raft_consensus.cc:740] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Initialized, Role: FOLLOWER
I20260709 13:18:40.911008 16340 ts_tablet_manager.cc:1434] T 68a6838d82ef4cac83507f906139d90f P 647a343a542d4a139fe106c19095129b: Time spent starting tablet: real 0.011s	user 0.005s	sys 0.000s
I20260709 13:18:40.911527 16347 consensus_queue.cc:260] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:40.913841 16347 ts_tablet_manager.cc:1434] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 1ca2725d290e47d68b1a4b609c9816ac: Time spent starting tablet: real 0.007s	user 0.003s	sys 0.002s
I20260709 13:18:40.914794 16347 tablet_bootstrap.cc:492] T 68a6838d82ef4cac83507f906139d90f P 1ca2725d290e47d68b1a4b609c9816ac: Bootstrap starting.
I20260709 13:18:40.912015 16340 tablet_bootstrap.cc:492] T ada5a004ba7c46819051e3ccec71ffdb P 647a343a542d4a139fe106c19095129b: Bootstrap starting.
I20260709 13:18:40.908454 16342 tablet_bootstrap.cc:492] T 5a959dbecc2648eb85d1cc550f5a3245 P 90f3da57796b488c9087a2f523adf045: Bootstrap starting.
I20260709 13:18:40.930810 16340 tablet_bootstrap.cc:654] T ada5a004ba7c46819051e3ccec71ffdb P 647a343a542d4a139fe106c19095129b: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:40.931586 16347 tablet_bootstrap.cc:654] T 68a6838d82ef4cac83507f906139d90f P 1ca2725d290e47d68b1a4b609c9816ac: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:40.931764 16342 tablet_bootstrap.cc:654] T 5a959dbecc2648eb85d1cc550f5a3245 P 90f3da57796b488c9087a2f523adf045: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:40.938817 16340 tablet_bootstrap.cc:492] T ada5a004ba7c46819051e3ccec71ffdb P 647a343a542d4a139fe106c19095129b: No bootstrap required, opened a new log
I20260709 13:18:40.938866 16347 tablet_bootstrap.cc:492] T 68a6838d82ef4cac83507f906139d90f P 1ca2725d290e47d68b1a4b609c9816ac: No bootstrap required, opened a new log
I20260709 13:18:40.939471 16340 ts_tablet_manager.cc:1403] T ada5a004ba7c46819051e3ccec71ffdb P 647a343a542d4a139fe106c19095129b: Time spent bootstrapping tablet: real 0.028s	user 0.012s	sys 0.003s
I20260709 13:18:40.939543 16347 ts_tablet_manager.cc:1403] T 68a6838d82ef4cac83507f906139d90f P 1ca2725d290e47d68b1a4b609c9816ac: Time spent bootstrapping tablet: real 0.025s	user 0.014s	sys 0.002s
I20260709 13:18:40.940274 16342 tablet_bootstrap.cc:492] T 5a959dbecc2648eb85d1cc550f5a3245 P 90f3da57796b488c9087a2f523adf045: No bootstrap required, opened a new log
I20260709 13:18:40.940665 16342 ts_tablet_manager.cc:1403] T 5a959dbecc2648eb85d1cc550f5a3245 P 90f3da57796b488c9087a2f523adf045: Time spent bootstrapping tablet: real 0.032s	user 0.005s	sys 0.008s
I20260709 13:18:40.942209 16340 raft_consensus.cc:359] T ada5a004ba7c46819051e3ccec71ffdb P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:40.942168 16347 raft_consensus.cc:359] T 68a6838d82ef4cac83507f906139d90f P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:40.942971 16340 raft_consensus.cc:385] T ada5a004ba7c46819051e3ccec71ffdb P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:40.943017 16347 raft_consensus.cc:385] T 68a6838d82ef4cac83507f906139d90f P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:40.943369 16340 raft_consensus.cc:740] T ada5a004ba7c46819051e3ccec71ffdb P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Initialized, Role: FOLLOWER
I20260709 13:18:40.943218 16342 raft_consensus.cc:359] T 5a959dbecc2648eb85d1cc550f5a3245 P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:40.943416 16347 raft_consensus.cc:740] T 68a6838d82ef4cac83507f906139d90f P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Initialized, Role: FOLLOWER
I20260709 13:18:40.943962 16342 raft_consensus.cc:385] T 5a959dbecc2648eb85d1cc550f5a3245 P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:40.944245 16342 raft_consensus.cc:740] T 5a959dbecc2648eb85d1cc550f5a3245 P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Initialized, Role: FOLLOWER
I20260709 13:18:40.944284 16340 consensus_queue.cc:260] T ada5a004ba7c46819051e3ccec71ffdb P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:40.944473 16347 consensus_queue.cc:260] T 68a6838d82ef4cac83507f906139d90f P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:40.944950 16342 consensus_queue.cc:260] T 5a959dbecc2648eb85d1cc550f5a3245 P 90f3da57796b488c9087a2f523adf045 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:40.946496 16340 ts_tablet_manager.cc:1434] T ada5a004ba7c46819051e3ccec71ffdb P 647a343a542d4a139fe106c19095129b: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:18:40.946636 16347 ts_tablet_manager.cc:1434] T 68a6838d82ef4cac83507f906139d90f P 1ca2725d290e47d68b1a4b609c9816ac: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.001s
I20260709 13:18:40.946777 16342 ts_tablet_manager.cc:1434] T 5a959dbecc2648eb85d1cc550f5a3245 P 90f3da57796b488c9087a2f523adf045: Time spent starting tablet: real 0.006s	user 0.002s	sys 0.003s
I20260709 13:18:40.947615 16342 tablet_bootstrap.cc:492] T 3884f864a96d428785895e5463e5728f P 90f3da57796b488c9087a2f523adf045: Bootstrap starting.
I20260709 13:18:40.947645 16347 tablet_bootstrap.cc:492] T 5a959dbecc2648eb85d1cc550f5a3245 P 1ca2725d290e47d68b1a4b609c9816ac: Bootstrap starting.
I20260709 13:18:40.950457 16340 tablet_bootstrap.cc:492] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 647a343a542d4a139fe106c19095129b: Bootstrap starting.
I20260709 13:18:40.953136 16342 tablet_bootstrap.cc:654] T 3884f864a96d428785895e5463e5728f P 90f3da57796b488c9087a2f523adf045: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:40.953136 16347 tablet_bootstrap.cc:654] T 5a959dbecc2648eb85d1cc550f5a3245 P 1ca2725d290e47d68b1a4b609c9816ac: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:40.957065 16340 tablet_bootstrap.cc:654] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 647a343a542d4a139fe106c19095129b: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:40.962421 16347 tablet_bootstrap.cc:492] T 5a959dbecc2648eb85d1cc550f5a3245 P 1ca2725d290e47d68b1a4b609c9816ac: No bootstrap required, opened a new log
I20260709 13:18:40.962960 16347 ts_tablet_manager.cc:1403] T 5a959dbecc2648eb85d1cc550f5a3245 P 1ca2725d290e47d68b1a4b609c9816ac: Time spent bootstrapping tablet: real 0.015s	user 0.011s	sys 0.000s
I20260709 13:18:40.969282 16340 tablet_bootstrap.cc:492] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 647a343a542d4a139fe106c19095129b: No bootstrap required, opened a new log
I20260709 13:18:40.969790 16340 ts_tablet_manager.cc:1403] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 647a343a542d4a139fe106c19095129b: Time spent bootstrapping tablet: real 0.020s	user 0.012s	sys 0.004s
I20260709 13:18:40.966332 16347 raft_consensus.cc:359] T 5a959dbecc2648eb85d1cc550f5a3245 P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:40.971099 16347 raft_consensus.cc:385] T 5a959dbecc2648eb85d1cc550f5a3245 P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:40.971411 16347 raft_consensus.cc:740] T 5a959dbecc2648eb85d1cc550f5a3245 P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Initialized, Role: FOLLOWER
I20260709 13:18:40.972445 16345 raft_consensus.cc:493] T 02838c86531d4f98bf58d1c22b8baf77 P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:40.972911 16345 raft_consensus.cc:515] T 02838c86531d4f98bf58d1c22b8baf77 P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:40.973127 16340 raft_consensus.cc:359] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:40.973884 16340 raft_consensus.cc:385] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:40.974200 16340 raft_consensus.cc:740] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Initialized, Role: FOLLOWER
I20260709 13:18:40.974898 16345 leader_election.cc:290] T 02838c86531d4f98bf58d1c22b8baf77 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:18:40.975091 16340 consensus_queue.cc:260] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:40.977545 16415 raft_consensus.cc:493] T ada5a004ba7c46819051e3ccec71ffdb P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:40.977958 16340 ts_tablet_manager.cc:1434] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 647a343a542d4a139fe106c19095129b: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 13:18:40.978032 16415 raft_consensus.cc:515] T ada5a004ba7c46819051e3ccec71ffdb P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:40.979039 16340 tablet_bootstrap.cc:492] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b: Bootstrap starting.
I20260709 13:18:40.980569 16415 leader_election.cc:290] T ada5a004ba7c46819051e3ccec71ffdb P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:18:40.981276 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02838c86531d4f98bf58d1c22b8baf77" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:18:40.981640 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02838c86531d4f98bf58d1c22b8baf77" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:18:40.982272 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada5a004ba7c46819051e3ccec71ffdb" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:18:40.982506 16342 tablet_bootstrap.cc:492] T 3884f864a96d428785895e5463e5728f P 90f3da57796b488c9087a2f523adf045: No bootstrap required, opened a new log
I20260709 13:18:40.983044 16342 ts_tablet_manager.cc:1403] T 3884f864a96d428785895e5463e5728f P 90f3da57796b488c9087a2f523adf045: Time spent bootstrapping tablet: real 0.036s	user 0.011s	sys 0.013s
I20260709 13:18:40.983343 16265 leader_election.cc:304] T 02838c86531d4f98bf58d1c22b8baf77 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:18:40.984515 16345 raft_consensus.cc:2749] T 02838c86531d4f98bf58d1c22b8baf77 P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:18:40.984880 16347 consensus_queue.cc:260] T 5a959dbecc2648eb85d1cc550f5a3245 P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:40.985755 16342 raft_consensus.cc:359] T 3884f864a96d428785895e5463e5728f P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:40.986371 16342 raft_consensus.cc:385] T 3884f864a96d428785895e5463e5728f P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:40.986608 16342 raft_consensus.cc:740] T 3884f864a96d428785895e5463e5728f P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Initialized, Role: FOLLOWER
I20260709 13:18:40.987365 16342 consensus_queue.cc:260] T 3884f864a96d428785895e5463e5728f P 90f3da57796b488c9087a2f523adf045 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:40.989880 16342 ts_tablet_manager.cc:1434] T 3884f864a96d428785895e5463e5728f P 90f3da57796b488c9087a2f523adf045: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 13:18:40.990754 16340 tablet_bootstrap.cc:654] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:40.990778 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada5a004ba7c46819051e3ccec71ffdb" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:18:40.991765 16299 raft_consensus.cc:2393] T ada5a004ba7c46819051e3ccec71ffdb P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 1: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
I20260709 13:18:40.990767 16342 tablet_bootstrap.cc:492] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 90f3da57796b488c9087a2f523adf045: Bootstrap starting.
I20260709 13:18:40.993101 16115 leader_election.cc:304] T ada5a004ba7c46819051e3ccec71ffdb P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:18:40.986995 16347 ts_tablet_manager.cc:1434] T 5a959dbecc2648eb85d1cc550f5a3245 P 1ca2725d290e47d68b1a4b609c9816ac: Time spent starting tablet: real 0.024s	user 0.005s	sys 0.000s
I20260709 13:18:40.995054 16347 tablet_bootstrap.cc:492] T d2ab0ac8347641e6955e9bce7d90c1a2 P 1ca2725d290e47d68b1a4b609c9816ac: Bootstrap starting.
I20260709 13:18:40.995810 16344 raft_consensus.cc:3060] T ada5a004ba7c46819051e3ccec71ffdb P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:41.001979 16373 raft_consensus.cc:493] T 3884f864a96d428785895e5463e5728f P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:41.002537 16373 raft_consensus.cc:515] T 3884f864a96d428785895e5463e5728f P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:41.004866 16373 leader_election.cc:290] T 3884f864a96d428785895e5463e5728f P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:18:41.006121 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3884f864a96d428785895e5463e5728f" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:18:41.006786 16224 raft_consensus.cc:2468] T 3884f864a96d428785895e5463e5728f P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 0.
I20260709 13:18:41.007393 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3884f864a96d428785895e5463e5728f" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:18:41.007999 16299 raft_consensus.cc:2468] T 3884f864a96d428785895e5463e5728f P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 0.
I20260709 13:18:41.008682 16116 leader_election.cc:304] T 3884f864a96d428785895e5463e5728f P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
I20260709 13:18:41.009649 16396 raft_consensus.cc:2804] T 3884f864a96d428785895e5463e5728f P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:18:41.009989 16396 raft_consensus.cc:493] T 3884f864a96d428785895e5463e5728f P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:41.010342 16396 raft_consensus.cc:3060] T 3884f864a96d428785895e5463e5728f P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:41.018576 16344 raft_consensus.cc:2749] T ada5a004ba7c46819051e3ccec71ffdb P 647a343a542d4a139fe106c19095129b [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:18:41.019796 16373 raft_consensus.cc:493] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:41.020270 16373 raft_consensus.cc:515] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:41.021188 16340 tablet_bootstrap.cc:492] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b: No bootstrap required, opened a new log
I20260709 13:18:41.021705 16340 ts_tablet_manager.cc:1403] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b: Time spent bootstrapping tablet: real 0.043s	user 0.015s	sys 0.003s
I20260709 13:18:41.022301 16373 leader_election.cc:290] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:18:41.022873 16347 tablet_bootstrap.cc:654] T d2ab0ac8347641e6955e9bce7d90c1a2 P 1ca2725d290e47d68b1a4b609c9816ac: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:41.023795 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87ac0d1f7e8a419f9bcbc04aee0e067e" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:18:41.024721 16396 raft_consensus.cc:515] T 3884f864a96d428785895e5463e5728f P 647a343a542d4a139fe106c19095129b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:41.024387 16340 raft_consensus.cc:359] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:41.026139 16340 raft_consensus.cc:385] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:41.025974 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87ac0d1f7e8a419f9bcbc04aee0e067e" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:18:41.026438 16340 raft_consensus.cc:740] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Initialized, Role: FOLLOWER
I20260709 13:18:41.026918 16396 leader_election.cc:290] T 3884f864a96d428785895e5463e5728f P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 1 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:18:41.027704 16342 tablet_bootstrap.cc:654] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 90f3da57796b488c9087a2f523adf045: Neither blocks nor log segments found. Creating new log.
W20260709 13:18:41.028136 16116 leader_election.cc:343] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:18:41.027678 16340 consensus_queue.cc:260] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:18:41.030045 16115 leader_election.cc:343] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:18:41.030826 16340 ts_tablet_manager.cc:1434] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b: Time spent starting tablet: real 0.009s	user 0.003s	sys 0.004s
I20260709 13:18:41.031965 16299 raft_consensus.cc:3060] T 02838c86531d4f98bf58d1c22b8baf77 P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:41.033558 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3884f864a96d428785895e5463e5728f" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:18:41.034925 16115 leader_election.cc:304] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:18:41.035645 16340 tablet_bootstrap.cc:492] T d16dfad0c3b540e5b4b86492ddf87208 P 647a343a542d4a139fe106c19095129b: Bootstrap starting.
I20260709 13:18:41.035516 16300 raft_consensus.cc:3060] T 3884f864a96d428785895e5463e5728f P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:41.034482 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3884f864a96d428785895e5463e5728f" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:18:41.040066 16344 raft_consensus.cc:2749] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:18:41.042409 16347 tablet_bootstrap.cc:492] T d2ab0ac8347641e6955e9bce7d90c1a2 P 1ca2725d290e47d68b1a4b609c9816ac: No bootstrap required, opened a new log
I20260709 13:18:41.047941 16347 ts_tablet_manager.cc:1403] T d2ab0ac8347641e6955e9bce7d90c1a2 P 1ca2725d290e47d68b1a4b609c9816ac: Time spent bootstrapping tablet: real 0.053s	user 0.016s	sys 0.002s
I20260709 13:18:41.050863 16347 raft_consensus.cc:359] T d2ab0ac8347641e6955e9bce7d90c1a2 P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:41.051890 16347 raft_consensus.cc:385] T d2ab0ac8347641e6955e9bce7d90c1a2 P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:41.052204 16347 raft_consensus.cc:740] T d2ab0ac8347641e6955e9bce7d90c1a2 P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Initialized, Role: FOLLOWER
I20260709 13:18:41.052562 16224 raft_consensus.cc:3060] T 3884f864a96d428785895e5463e5728f P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:41.052663 16300 raft_consensus.cc:2468] T 3884f864a96d428785895e5463e5728f P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 1.
I20260709 13:18:41.052886 16347 consensus_queue.cc:260] T d2ab0ac8347641e6955e9bce7d90c1a2 P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:41.056128 16347 ts_tablet_manager.cc:1434] T d2ab0ac8347641e6955e9bce7d90c1a2 P 1ca2725d290e47d68b1a4b609c9816ac: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.002s
I20260709 13:18:41.059193 16347 tablet_bootstrap.cc:492] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac: Bootstrap starting.
I20260709 13:18:41.062203 16115 leader_election.cc:304] T 3884f864a96d428785895e5463e5728f P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:18:41.063182 16344 raft_consensus.cc:2804] T 3884f864a96d428785895e5463e5728f P 647a343a542d4a139fe106c19095129b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:18:41.063673 16344 raft_consensus.cc:697] T 3884f864a96d428785895e5463e5728f P 647a343a542d4a139fe106c19095129b [term 1 LEADER]: Becoming Leader. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:18:41.064822 16340 tablet_bootstrap.cc:654] T d16dfad0c3b540e5b4b86492ddf87208 P 647a343a542d4a139fe106c19095129b: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:41.064476 16344 consensus_queue.cc:237] T 3884f864a96d428785895e5463e5728f P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:41.072913 16347 tablet_bootstrap.cc:654] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:41.065575 16224 raft_consensus.cc:2468] T 3884f864a96d428785895e5463e5728f P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 1.
I20260709 13:18:41.075246 16342 tablet_bootstrap.cc:492] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 90f3da57796b488c9087a2f523adf045: No bootstrap required, opened a new log
I20260709 13:18:41.078598 16342 ts_tablet_manager.cc:1403] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 90f3da57796b488c9087a2f523adf045: Time spent bootstrapping tablet: real 0.088s	user 0.024s	sys 0.013s
I20260709 13:18:41.079522 16345 raft_consensus.cc:493] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:41.080044 16345 raft_consensus.cc:515] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:41.082238 16345 leader_election.cc:290] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:18:41.083855 16342 raft_consensus.cc:359] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:41.084823 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e9b2bf7011f4ae688a79d7bb5a447b4" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:18:41.085687 16150 raft_consensus.cc:2468] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 0.
I20260709 13:18:41.085593 16342 raft_consensus.cc:385] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:41.086757 16342 raft_consensus.cc:740] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Initialized, Role: FOLLOWER
I20260709 13:18:41.087136 16265 leader_election.cc:304] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:18:41.088691 16342 consensus_queue.cc:260] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 90f3da57796b488c9087a2f523adf045 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:41.089975 16376 raft_consensus.cc:493] T d2ab0ac8347641e6955e9bce7d90c1a2 P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:41.091369 16345 raft_consensus.cc:2804] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:18:41.091988 16345 raft_consensus.cc:493] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:41.091432 16376 raft_consensus.cc:515] T d2ab0ac8347641e6955e9bce7d90c1a2 P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:41.092396 16345 raft_consensus.cc:3060] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:41.096262 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ab0ac8347641e6955e9bce7d90c1a2" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:18:41.096602 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e9b2bf7011f4ae688a79d7bb5a447b4" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:18:41.097399 16224 raft_consensus.cc:2468] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 0.
W20260709 13:18:41.098798 16191 leader_election.cc:343] T d2ab0ac8347641e6955e9bce7d90c1a2 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:18:41.101594 16376 leader_election.cc:290] T d2ab0ac8347641e6955e9bce7d90c1a2 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:18:41.101850 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ab0ac8347641e6955e9bce7d90c1a2" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:18:41.102569 16300 raft_consensus.cc:2468] T d2ab0ac8347641e6955e9bce7d90c1a2 P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 0.
I20260709 13:18:41.104050 16190 leader_election.cc:304] T d2ab0ac8347641e6955e9bce7d90c1a2 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 647a343a542d4a139fe106c19095129b
I20260709 13:18:41.105320 16376 raft_consensus.cc:2804] T d2ab0ac8347641e6955e9bce7d90c1a2 P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:18:41.105804 16376 raft_consensus.cc:493] T d2ab0ac8347641e6955e9bce7d90c1a2 P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:41.106184 16376 raft_consensus.cc:3060] T d2ab0ac8347641e6955e9bce7d90c1a2 P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:41.107228 16342 ts_tablet_manager.cc:1434] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 90f3da57796b488c9087a2f523adf045: Time spent starting tablet: real 0.028s	user 0.008s	sys 0.002s
I20260709 13:18:41.109769 16342 tablet_bootstrap.cc:492] T 66ae176961b74813a58bc15c9c4c6f2a P 90f3da57796b488c9087a2f523adf045: Bootstrap starting.
I20260709 13:18:41.114020 16376 raft_consensus.cc:515] T d2ab0ac8347641e6955e9bce7d90c1a2 P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:41.117687 16345 raft_consensus.cc:515] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:41.120011 16345 leader_election.cc:290] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 1 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:18:41.121044 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e9b2bf7011f4ae688a79d7bb5a447b4" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:18:41.121690 16150 raft_consensus.cc:3060] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:41.122650 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ab0ac8347641e6955e9bce7d90c1a2" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:18:41.123294 16300 raft_consensus.cc:3060] T d2ab0ac8347641e6955e9bce7d90c1a2 P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:41.124434 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ab0ac8347641e6955e9bce7d90c1a2" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:18:41.125270 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e9b2bf7011f4ae688a79d7bb5a447b4" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:18:41.125906 16224 raft_consensus.cc:3060] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:18:41.126003 16191 leader_election.cc:343] T d2ab0ac8347641e6955e9bce7d90c1a2 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:18:41.130498 16150 raft_consensus.cc:2468] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 647a343a542d4a139fe106c19095129b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 1.
I20260709 13:18:41.131093 16430 raft_consensus.cc:493] T 68a6838d82ef4cac83507f906139d90f P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:41.131924 16265 leader_election.cc:304] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:18:41.133155 16345 raft_consensus.cc:2804] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:18:41.133802 16345 raft_consensus.cc:697] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 90f3da57796b488c9087a2f523adf045 [term 1 LEADER]: Becoming Leader. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:18:41.133283 16047 catalog_manager.cc:5697] T 3884f864a96d428785895e5463e5728f P 647a343a542d4a139fe106c19095129b reported cstate change: term changed from 0 to 1, leader changed from <none> to 647a343a542d4a139fe106c19095129b (127.14.177.129). New cstate: current_term: 1 leader_uuid: "647a343a542d4a139fe106c19095129b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:41.134697 16345 consensus_queue.cc:237] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:41.136277 16300 raft_consensus.cc:2468] T d2ab0ac8347641e6955e9bce7d90c1a2 P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 1.
I20260709 13:18:41.137668 16190 leader_election.cc:304] T d2ab0ac8347641e6955e9bce7d90c1a2 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 647a343a542d4a139fe106c19095129b
I20260709 13:18:41.138628 16406 raft_consensus.cc:2804] T d2ab0ac8347641e6955e9bce7d90c1a2 P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:18:41.139216 16406 raft_consensus.cc:697] T d2ab0ac8347641e6955e9bce7d90c1a2 P 1ca2725d290e47d68b1a4b609c9816ac [term 1 LEADER]: Becoming Leader. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:18:41.139267 16224 raft_consensus.cc:2468] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 1.
I20260709 13:18:41.131631 16430 raft_consensus.cc:515] T 68a6838d82ef4cac83507f906139d90f P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:41.140249 16406 consensus_queue.cc:237] T d2ab0ac8347641e6955e9bce7d90c1a2 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:41.141486 16430 leader_election.cc:290] T 68a6838d82ef4cac83507f906139d90f P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:18:41.143935 16376 leader_election.cc:290] T d2ab0ac8347641e6955e9bce7d90c1a2 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 1 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:18:41.147785 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68a6838d82ef4cac83507f906139d90f" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:18:41.148525 16150 raft_consensus.cc:2468] T 68a6838d82ef4cac83507f906139d90f P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 0.
I20260709 13:18:41.149065 16340 tablet_bootstrap.cc:492] T d16dfad0c3b540e5b4b86492ddf87208 P 647a343a542d4a139fe106c19095129b: No bootstrap required, opened a new log
I20260709 13:18:41.149541 16340 ts_tablet_manager.cc:1403] T d16dfad0c3b540e5b4b86492ddf87208 P 647a343a542d4a139fe106c19095129b: Time spent bootstrapping tablet: real 0.114s	user 0.015s	sys 0.008s
I20260709 13:18:41.150111 16191 leader_election.cc:304] T 68a6838d82ef4cac83507f906139d90f P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
I20260709 13:18:41.151343 16342 tablet_bootstrap.cc:654] T 66ae176961b74813a58bc15c9c4c6f2a P 90f3da57796b488c9087a2f523adf045: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:41.151512 16432 raft_consensus.cc:2804] T 68a6838d82ef4cac83507f906139d90f P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:18:41.151981 16432 raft_consensus.cc:493] T 68a6838d82ef4cac83507f906139d90f P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:41.152356 16432 raft_consensus.cc:3060] T 68a6838d82ef4cac83507f906139d90f P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:41.152578 16340 raft_consensus.cc:359] T d16dfad0c3b540e5b4b86492ddf87208 P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:41.153337 16340 raft_consensus.cc:385] T d16dfad0c3b540e5b4b86492ddf87208 P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:41.153632 16340 raft_consensus.cc:740] T d16dfad0c3b540e5b4b86492ddf87208 P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Initialized, Role: FOLLOWER
I20260709 13:18:41.154392 16340 consensus_queue.cc:260] T d16dfad0c3b540e5b4b86492ddf87208 P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:41.156582 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68a6838d82ef4cac83507f906139d90f" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:18:41.157274 16300 raft_consensus.cc:2468] T 68a6838d82ef4cac83507f906139d90f P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 0.
I20260709 13:18:41.165513 16340 ts_tablet_manager.cc:1434] T d16dfad0c3b540e5b4b86492ddf87208 P 647a343a542d4a139fe106c19095129b: Time spent starting tablet: real 0.016s	user 0.008s	sys 0.000s
I20260709 13:18:41.167089 16340 tablet_bootstrap.cc:492] T d2ab0ac8347641e6955e9bce7d90c1a2 P 647a343a542d4a139fe106c19095129b: Bootstrap starting.
I20260709 13:18:41.168512 16432 raft_consensus.cc:515] T 68a6838d82ef4cac83507f906139d90f P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:41.174716 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68a6838d82ef4cac83507f906139d90f" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:18:41.175513 16150 raft_consensus.cc:3060] T 68a6838d82ef4cac83507f906139d90f P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:41.180131 16340 tablet_bootstrap.cc:654] T d2ab0ac8347641e6955e9bce7d90c1a2 P 647a343a542d4a139fe106c19095129b: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:41.188099 16347 tablet_bootstrap.cc:492] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac: No bootstrap required, opened a new log
I20260709 13:18:41.196187 16347 ts_tablet_manager.cc:1403] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac: Time spent bootstrapping tablet: real 0.137s	user 0.020s	sys 0.042s
I20260709 13:18:41.195705 16432 leader_election.cc:290] T 68a6838d82ef4cac83507f906139d90f P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 1 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:18:41.199638 16340 tablet_bootstrap.cc:492] T d2ab0ac8347641e6955e9bce7d90c1a2 P 647a343a542d4a139fe106c19095129b: No bootstrap required, opened a new log
I20260709 13:18:41.199203 16347 raft_consensus.cc:359] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:41.200148 16347 raft_consensus.cc:385] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:41.200598 16347 raft_consensus.cc:740] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Initialized, Role: FOLLOWER
I20260709 13:18:41.201733 16342 tablet_bootstrap.cc:492] T 66ae176961b74813a58bc15c9c4c6f2a P 90f3da57796b488c9087a2f523adf045: No bootstrap required, opened a new log
I20260709 13:18:41.201357 16347 consensus_queue.cc:260] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:41.202201 16342 ts_tablet_manager.cc:1403] T 66ae176961b74813a58bc15c9c4c6f2a P 90f3da57796b488c9087a2f523adf045: Time spent bootstrapping tablet: real 0.093s	user 0.008s	sys 0.023s
I20260709 13:18:41.201436 16047 catalog_manager.cc:5697] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 90f3da57796b488c9087a2f523adf045 reported cstate change: term changed from 0 to 1, leader changed from <none> to 90f3da57796b488c9087a2f523adf045 (127.14.177.131). New cstate: current_term: 1 leader_uuid: "90f3da57796b488c9087a2f523adf045" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } health_report { overall_health: HEALTHY } } }
I20260709 13:18:41.203766 16347 ts_tablet_manager.cc:1434] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.003s
I20260709 13:18:41.195550 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68a6838d82ef4cac83507f906139d90f" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:18:41.200184 16340 ts_tablet_manager.cc:1403] T d2ab0ac8347641e6955e9bce7d90c1a2 P 647a343a542d4a139fe106c19095129b: Time spent bootstrapping tablet: real 0.033s	user 0.013s	sys 0.008s
I20260709 13:18:41.204722 16299 raft_consensus.cc:3060] T 68a6838d82ef4cac83507f906139d90f P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:41.206547 16342 raft_consensus.cc:359] T 66ae176961b74813a58bc15c9c4c6f2a P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:41.207298 16342 raft_consensus.cc:385] T 66ae176961b74813a58bc15c9c4c6f2a P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:41.204797 16347 tablet_bootstrap.cc:492] T d16dfad0c3b540e5b4b86492ddf87208 P 1ca2725d290e47d68b1a4b609c9816ac: Bootstrap starting.
I20260709 13:18:41.207589 16342 raft_consensus.cc:740] T 66ae176961b74813a58bc15c9c4c6f2a P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Initialized, Role: FOLLOWER
I20260709 13:18:41.208560 16342 consensus_queue.cc:260] T 66ae176961b74813a58bc15c9c4c6f2a P 90f3da57796b488c9087a2f523adf045 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:41.192260 16048 catalog_manager.cc:5697] T d2ab0ac8347641e6955e9bce7d90c1a2 P 1ca2725d290e47d68b1a4b609c9816ac reported cstate change: term changed from 0 to 1, leader changed from <none> to 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130). New cstate: current_term: 1 leader_uuid: "1ca2725d290e47d68b1a4b609c9816ac" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:41.187321 16150 raft_consensus.cc:2468] T 68a6838d82ef4cac83507f906139d90f P 647a343a542d4a139fe106c19095129b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 1.
I20260709 13:18:41.212539 16191 leader_election.cc:304] T 68a6838d82ef4cac83507f906139d90f P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
I20260709 13:18:41.213107 16342 ts_tablet_manager.cc:1434] T 66ae176961b74813a58bc15c9c4c6f2a P 90f3da57796b488c9087a2f523adf045: Time spent starting tablet: real 0.011s	user 0.007s	sys 0.000s
I20260709 13:18:41.213523 16432 raft_consensus.cc:2804] T 68a6838d82ef4cac83507f906139d90f P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:18:41.214465 16342 tablet_bootstrap.cc:492] T 1d5d59407eb545b895d8d922e0dd69fd P 90f3da57796b488c9087a2f523adf045: Bootstrap starting.
I20260709 13:18:41.214600 16347 tablet_bootstrap.cc:654] T d16dfad0c3b540e5b4b86492ddf87208 P 1ca2725d290e47d68b1a4b609c9816ac: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:41.216055 16432 raft_consensus.cc:697] T 68a6838d82ef4cac83507f906139d90f P 1ca2725d290e47d68b1a4b609c9816ac [term 1 LEADER]: Becoming Leader. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:18:41.207610 16340 raft_consensus.cc:359] T d2ab0ac8347641e6955e9bce7d90c1a2 P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:41.218571 16340 raft_consensus.cc:385] T d2ab0ac8347641e6955e9bce7d90c1a2 P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:41.218901 16340 raft_consensus.cc:740] T d2ab0ac8347641e6955e9bce7d90c1a2 P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Initialized, Role: FOLLOWER
I20260709 13:18:41.217024 16432 consensus_queue.cc:237] T 68a6838d82ef4cac83507f906139d90f P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:41.219762 16340 consensus_queue.cc:260] T d2ab0ac8347641e6955e9bce7d90c1a2 P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:41.221462 16342 tablet_bootstrap.cc:654] T 1d5d59407eb545b895d8d922e0dd69fd P 90f3da57796b488c9087a2f523adf045: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:41.224656 16299 raft_consensus.cc:2468] T 68a6838d82ef4cac83507f906139d90f P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 1.
I20260709 13:18:41.226987 16340 ts_tablet_manager.cc:1434] T d2ab0ac8347641e6955e9bce7d90c1a2 P 647a343a542d4a139fe106c19095129b: Time spent starting tablet: real 0.022s	user 0.006s	sys 0.000s
I20260709 13:18:41.228353 16340 tablet_bootstrap.cc:492] T 5a959dbecc2648eb85d1cc550f5a3245 P 647a343a542d4a139fe106c19095129b: Bootstrap starting.
I20260709 13:18:41.240075 16340 tablet_bootstrap.cc:654] T 5a959dbecc2648eb85d1cc550f5a3245 P 647a343a542d4a139fe106c19095129b: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:41.247677 16376 consensus_queue.cc:1048] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:18:41.249513 16345 consensus_queue.cc:1048] T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:18:41.256345 16432 consensus_queue.cc:1048] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:18:41.263250 16345 raft_consensus.cc:493] T 66ae176961b74813a58bc15c9c4c6f2a P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:41.263803 16345 raft_consensus.cc:515] T 66ae176961b74813a58bc15c9c4c6f2a P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:41.266103 16345 leader_election.cc:290] T 66ae176961b74813a58bc15c9c4c6f2a P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:18:41.268023 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66ae176961b74813a58bc15c9c4c6f2a" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:18:41.269763 16151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66ae176961b74813a58bc15c9c4c6f2a" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
W20260709 13:18:41.270354 16265 leader_election.cc:343] T 66ae176961b74813a58bc15c9c4c6f2a P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:18:41.275899 16432 raft_consensus.cc:493] T b6f07842d3aa47858df5665874c86f3a P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:41.280200 16265 leader_election.cc:343] T 66ae176961b74813a58bc15c9c4c6f2a P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:18:41.281471 16265 leader_election.cc:304] T 66ae176961b74813a58bc15c9c4c6f2a P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:18:41.282286 16427 raft_consensus.cc:2749] T 66ae176961b74813a58bc15c9c4c6f2a P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:18:41.278402 16046 catalog_manager.cc:5697] T 68a6838d82ef4cac83507f906139d90f P 1ca2725d290e47d68b1a4b609c9816ac reported cstate change: term changed from 0 to 1, leader changed from <none> to 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130). New cstate: current_term: 1 leader_uuid: "1ca2725d290e47d68b1a4b609c9816ac" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:41.278785 16432 raft_consensus.cc:515] T b6f07842d3aa47858df5665874c86f3a P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:18:41.286756 16097 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:18:41.287684 16224 raft_consensus.cc:3060] T b6f07842d3aa47858df5665874c86f3a P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:41.289393 16432 leader_election.cc:290] T b6f07842d3aa47858df5665874c86f3a P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:18:41.292762 16151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6f07842d3aa47858df5665874c86f3a" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:18:41.293485 16347 tablet_bootstrap.cc:492] T d16dfad0c3b540e5b4b86492ddf87208 P 1ca2725d290e47d68b1a4b609c9816ac: No bootstrap required, opened a new log
I20260709 13:18:41.294129 16347 ts_tablet_manager.cc:1403] T d16dfad0c3b540e5b4b86492ddf87208 P 1ca2725d290e47d68b1a4b609c9816ac: Time spent bootstrapping tablet: real 0.090s	user 0.009s	sys 0.013s
I20260709 13:18:41.296648 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6f07842d3aa47858df5665874c86f3a" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:18:41.298946 16190 leader_election.cc:304] T b6f07842d3aa47858df5665874c86f3a P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:18:41.299149 16406 consensus_queue.cc:1048] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:18:41.314435 16342 tablet_bootstrap.cc:492] T 1d5d59407eb545b895d8d922e0dd69fd P 90f3da57796b488c9087a2f523adf045: No bootstrap required, opened a new log
I20260709 13:18:41.315006 16342 ts_tablet_manager.cc:1403] T 1d5d59407eb545b895d8d922e0dd69fd P 90f3da57796b488c9087a2f523adf045: Time spent bootstrapping tablet: real 0.101s	user 0.036s	sys 0.039s
I20260709 13:18:41.317606 16430 raft_consensus.cc:2749] T b6f07842d3aa47858df5665874c86f3a P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:18:41.317889 16342 raft_consensus.cc:359] T 1d5d59407eb545b895d8d922e0dd69fd P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:41.318778 16342 raft_consensus.cc:385] T 1d5d59407eb545b895d8d922e0dd69fd P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:41.319093 16342 raft_consensus.cc:740] T 1d5d59407eb545b895d8d922e0dd69fd P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Initialized, Role: FOLLOWER
I20260709 13:18:41.320111 16406 raft_consensus.cc:493] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:41.319859 16342 consensus_queue.cc:260] T 1d5d59407eb545b895d8d922e0dd69fd P 90f3da57796b488c9087a2f523adf045 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:41.320662 16406 raft_consensus.cc:515] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:41.321612 16344 raft_consensus.cc:493] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:41.322181 16342 ts_tablet_manager.cc:1434] T 1d5d59407eb545b895d8d922e0dd69fd P 90f3da57796b488c9087a2f523adf045: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 13:18:41.322108 16344 raft_consensus.cc:515] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:41.322659 16406 leader_election.cc:290] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:18:41.323279 16342 tablet_bootstrap.cc:492] T 829b4ad8c5a54eababeea1181b960d4b P 90f3da57796b488c9087a2f523adf045: Bootstrap starting.
I20260709 13:18:41.323410 16148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5d59407eb545b895d8d922e0dd69fd" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:18:41.324074 16148 raft_consensus.cc:2468] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 0.
I20260709 13:18:41.325228 16344 leader_election.cc:290] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:18:41.325848 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5d59407eb545b895d8d922e0dd69fd" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:18:41.329519 16225 raft_consensus.cc:2468] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 0.
I20260709 13:18:41.329619 16191 leader_election.cc:304] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
I20260709 13:18:41.333236 16116 leader_election.cc:304] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
I20260709 13:18:41.334105 16344 raft_consensus.cc:2804] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:18:41.334316 16430 raft_consensus.cc:2804] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:18:41.334497 16344 raft_consensus.cc:493] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:41.334707 16430 raft_consensus.cc:493] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:41.334877 16344 raft_consensus.cc:3060] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:41.335071 16430 raft_consensus.cc:3060] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:41.345702 16376 raft_consensus.cc:493] T 5a959dbecc2648eb85d1cc550f5a3245 P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:41.346529 16376 raft_consensus.cc:515] T 5a959dbecc2648eb85d1cc550f5a3245 P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:41.346365 16347 raft_consensus.cc:359] T d16dfad0c3b540e5b4b86492ddf87208 P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:41.347990 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5d59407eb545b895d8d922e0dd69fd" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:18:41.348798 16376 leader_election.cc:290] T 5a959dbecc2648eb85d1cc550f5a3245 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:18:41.349336 16373 raft_consensus.cc:493] T d16dfad0c3b540e5b4b86492ddf87208 P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:41.348778 16300 raft_consensus.cc:2468] T 1d5d59407eb545b895d8d922e0dd69fd P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 0.
I20260709 13:18:41.349830 16373 raft_consensus.cc:515] T d16dfad0c3b540e5b4b86492ddf87208 P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:41.350132 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a959dbecc2648eb85d1cc550f5a3245" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:18:41.350842 16297 raft_consensus.cc:2468] T 5a959dbecc2648eb85d1cc550f5a3245 P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 0.
I20260709 13:18:41.351722 16463 consensus_queue.cc:1048] T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:18:41.352092 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5d59407eb545b895d8d922e0dd69fd" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:18:41.352414 16373 leader_election.cc:290] T d16dfad0c3b540e5b4b86492ddf87208 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:18:41.352722 16299 raft_consensus.cc:2468] T 1d5d59407eb545b895d8d922e0dd69fd P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 0.
I20260709 13:18:41.353773 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d16dfad0c3b540e5b4b86492ddf87208" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:18:41.355341 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d16dfad0c3b540e5b4b86492ddf87208" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
W20260709 13:18:41.357429 16115 leader_election.cc:343] T d16dfad0c3b540e5b4b86492ddf87208 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:18:41.357494 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a959dbecc2648eb85d1cc550f5a3245" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:18:41.358042 16190 leader_election.cc:304] T 5a959dbecc2648eb85d1cc550f5a3245 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:18:41.359484 16376 raft_consensus.cc:2804] T 5a959dbecc2648eb85d1cc550f5a3245 P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:18:41.359979 16376 raft_consensus.cc:493] T 5a959dbecc2648eb85d1cc550f5a3245 P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:41.360411 16376 raft_consensus.cc:3060] T 5a959dbecc2648eb85d1cc550f5a3245 P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:18:41.361464 16191 leader_election.cc:343] T 5a959dbecc2648eb85d1cc550f5a3245 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:18:41.364362 16342 tablet_bootstrap.cc:654] T 829b4ad8c5a54eababeea1181b960d4b P 90f3da57796b488c9087a2f523adf045: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:41.347486 16347 raft_consensus.cc:385] T d16dfad0c3b540e5b4b86492ddf87208 P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:41.370561 16344 raft_consensus.cc:515] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:41.367033 16347 raft_consensus.cc:740] T d16dfad0c3b540e5b4b86492ddf87208 P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Initialized, Role: FOLLOWER
I20260709 13:18:41.371556 16463 raft_consensus.cc:493] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:41.371551 16430 raft_consensus.cc:515] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:41.391208 16463 raft_consensus.cc:515] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:41.392976 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5d59407eb545b895d8d922e0dd69fd" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:18:41.393846 16226 raft_consensus.cc:2393] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 1: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
I20260709 13:18:41.394074 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5d59407eb545b895d8d922e0dd69fd" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:18:41.394712 16299 raft_consensus.cc:3060] T 1d5d59407eb545b895d8d922e0dd69fd P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:41.394886 16463 leader_election.cc:290] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:18:41.392102 16344 leader_election.cc:290] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 1 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:18:41.398037 16151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87ac0d1f7e8a419f9bcbc04aee0e067e" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:18:41.398773 16151 raft_consensus.cc:2468] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 0.
I20260709 13:18:41.401140 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87ac0d1f7e8a419f9bcbc04aee0e067e" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:18:41.401567 16265 leader_election.cc:304] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
W20260709 13:18:41.403066 16265 leader_election.cc:343] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:18:41.404026 16463 raft_consensus.cc:2804] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:18:41.404498 16463 raft_consensus.cc:493] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:41.407562 16463 raft_consensus.cc:3060] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:41.409274 16151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5d59407eb545b895d8d922e0dd69fd" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:18:41.410264 16151 raft_consensus.cc:2393] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 1: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
I20260709 13:18:41.411458 16342 tablet_bootstrap.cc:492] T 829b4ad8c5a54eababeea1181b960d4b P 90f3da57796b488c9087a2f523adf045: No bootstrap required, opened a new log
I20260709 13:18:41.411964 16342 ts_tablet_manager.cc:1403] T 829b4ad8c5a54eababeea1181b960d4b P 90f3da57796b488c9087a2f523adf045: Time spent bootstrapping tablet: real 0.089s	user 0.008s	sys 0.007s
I20260709 13:18:41.412813 16347 consensus_queue.cc:260] T d16dfad0c3b540e5b4b86492ddf87208 P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:41.414966 16342 raft_consensus.cc:359] T 829b4ad8c5a54eababeea1181b960d4b P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:41.415719 16342 raft_consensus.cc:385] T 829b4ad8c5a54eababeea1181b960d4b P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:41.415894 16224 raft_consensus.cc:2468] T d16dfad0c3b540e5b4b86492ddf87208 P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 0.
I20260709 13:18:41.416710 16430 leader_election.cc:290] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 1 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:18:41.416009 16342 raft_consensus.cc:740] T 829b4ad8c5a54eababeea1181b960d4b P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Initialized, Role: FOLLOWER
I20260709 13:18:41.419525 16342 consensus_queue.cc:260] T 829b4ad8c5a54eababeea1181b960d4b P 90f3da57796b488c9087a2f523adf045 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:41.421487 16340 tablet_bootstrap.cc:492] T 5a959dbecc2648eb85d1cc550f5a3245 P 647a343a542d4a139fe106c19095129b: No bootstrap required, opened a new log
I20260709 13:18:41.421955 16342 ts_tablet_manager.cc:1434] T 829b4ad8c5a54eababeea1181b960d4b P 90f3da57796b488c9087a2f523adf045: Time spent starting tablet: real 0.010s	user 0.007s	sys 0.000s
I20260709 13:18:41.421967 16340 ts_tablet_manager.cc:1403] T 5a959dbecc2648eb85d1cc550f5a3245 P 647a343a542d4a139fe106c19095129b: Time spent bootstrapping tablet: real 0.194s	user 0.014s	sys 0.004s
I20260709 13:18:41.423069 16342 tablet_bootstrap.cc:492] T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045: Bootstrap starting.
I20260709 13:18:41.419694 16116 leader_election.cc:304] T d16dfad0c3b540e5b4b86492ddf87208 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 90f3da57796b488c9087a2f523adf045
I20260709 13:18:41.423779 16373 raft_consensus.cc:2804] T d16dfad0c3b540e5b4b86492ddf87208 P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:18:41.424145 16373 raft_consensus.cc:493] T d16dfad0c3b540e5b4b86492ddf87208 P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:41.424470 16373 raft_consensus.cc:3060] T d16dfad0c3b540e5b4b86492ddf87208 P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:41.424345 16463 raft_consensus.cc:515] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:41.424346 16376 raft_consensus.cc:515] T 5a959dbecc2648eb85d1cc550f5a3245 P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:41.428416 16406 consensus_queue.cc:1048] T 8c11074579b046d99e585c2818e18a2c P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:18:41.428843 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a959dbecc2648eb85d1cc550f5a3245" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:18:41.422173 16500 consensus_queue.cc:1048] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.010s
I20260709 13:18:41.430784 16463 leader_election.cc:290] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 1 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:18:41.429667 16151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87ac0d1f7e8a419f9bcbc04aee0e067e" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:18:41.431629 16151 raft_consensus.cc:3060] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:41.431731 16376 leader_election.cc:290] T 5a959dbecc2648eb85d1cc550f5a3245 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 1 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:18:41.431877 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87ac0d1f7e8a419f9bcbc04aee0e067e" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:18:41.435279 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5d59407eb545b895d8d922e0dd69fd" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:18:41.437721 16347 ts_tablet_manager.cc:1434] T d16dfad0c3b540e5b4b86492ddf87208 P 1ca2725d290e47d68b1a4b609c9816ac: Time spent starting tablet: real 0.143s	user 0.009s	sys 0.001s
I20260709 13:18:41.440567 16347 tablet_bootstrap.cc:492] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac: Bootstrap starting.
W20260709 13:18:41.438753 16265 leader_election.cc:343] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:18:41.447212 16190 leader_election.cc:304] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:18:41.448326 16296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a959dbecc2648eb85d1cc550f5a3245" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:18:41.466280 16299 raft_consensus.cc:2468] T 1d5d59407eb545b895d8d922e0dd69fd P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 1.
I20260709 13:18:41.470451 16296 raft_consensus.cc:3060] T 5a959dbecc2648eb85d1cc550f5a3245 P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:41.478492 16151 raft_consensus.cc:2468] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 647a343a542d4a139fe106c19095129b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 1.
I20260709 13:18:41.480221 16376 raft_consensus.cc:2749] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:18:41.489626 16340 raft_consensus.cc:359] T 5a959dbecc2648eb85d1cc550f5a3245 P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:41.490156 16340 raft_consensus.cc:385] T 5a959dbecc2648eb85d1cc550f5a3245 P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:41.490358 16340 raft_consensus.cc:740] T 5a959dbecc2648eb85d1cc550f5a3245 P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Initialized, Role: FOLLOWER
I20260709 13:18:41.501257 16340 consensus_queue.cc:260] T 5a959dbecc2648eb85d1cc550f5a3245 P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:41.499044 16115 leader_election.cc:304] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac
I20260709 13:18:41.503244 16150 raft_consensus.cc:3060] T 5a959dbecc2648eb85d1cc550f5a3245 P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:41.504109 16344 raft_consensus.cc:2804] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:18:41.504622 16344 raft_consensus.cc:697] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [term 1 LEADER]: Becoming Leader. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:18:41.505442 16344 consensus_queue.cc:237] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:41.500568 16265 leader_election.cc:304] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac
I20260709 13:18:41.501230 16373 raft_consensus.cc:515] T d16dfad0c3b540e5b4b86492ddf87208 P 647a343a542d4a139fe106c19095129b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:41.509085 16463 raft_consensus.cc:2804] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:18:41.509505 16373 leader_election.cc:290] T d16dfad0c3b540e5b4b86492ddf87208 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 1 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:18:41.510063 16345 raft_consensus.cc:493] T 829b4ad8c5a54eababeea1181b960d4b P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:41.510601 16345 raft_consensus.cc:515] T 829b4ad8c5a54eababeea1181b960d4b P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:41.494588 16500 consensus_queue.cc:1048] T 8c11074579b046d99e585c2818e18a2c P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.007s
I20260709 13:18:41.515060 16345 leader_election.cc:290] T 829b4ad8c5a54eababeea1181b960d4b P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:18:41.510828 16150 raft_consensus.cc:2468] T 5a959dbecc2648eb85d1cc550f5a3245 P 647a343a542d4a139fe106c19095129b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 1.
I20260709 13:18:41.518829 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d16dfad0c3b540e5b4b86492ddf87208" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:18:41.519483 16224 raft_consensus.cc:3060] T d16dfad0c3b540e5b4b86492ddf87208 P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:41.519544 16295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d16dfad0c3b540e5b4b86492ddf87208" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:18:41.521456 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "829b4ad8c5a54eababeea1181b960d4b" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:18:41.524194 16347 tablet_bootstrap.cc:654] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:41.509658 16463 raft_consensus.cc:697] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 90f3da57796b488c9087a2f523adf045 [term 1 LEADER]: Becoming Leader. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:18:41.526057 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "829b4ad8c5a54eababeea1181b960d4b" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
W20260709 13:18:41.528203 16265 leader_election.cc:343] T 829b4ad8c5a54eababeea1181b960d4b P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:18:41.528955 16265 leader_election.cc:343] T 829b4ad8c5a54eababeea1181b960d4b P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:18:41.529418 16463 consensus_queue.cc:237] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 90f3da57796b488c9087a2f523adf045 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:41.530630 16265 leader_election.cc:304] T 829b4ad8c5a54eababeea1181b960d4b P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:18:41.533006 16427 raft_consensus.cc:2749] T 829b4ad8c5a54eababeea1181b960d4b P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:18:41.537387 16191 leader_election.cc:304] T 5a959dbecc2648eb85d1cc550f5a3245 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
I20260709 13:18:41.539047 16430 raft_consensus.cc:2804] T 5a959dbecc2648eb85d1cc550f5a3245 P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:18:41.539564 16430 raft_consensus.cc:697] T 5a959dbecc2648eb85d1cc550f5a3245 P 1ca2725d290e47d68b1a4b609c9816ac [term 1 LEADER]: Becoming Leader. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:18:41.539692 16224 raft_consensus.cc:2468] T d16dfad0c3b540e5b4b86492ddf87208 P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 1.
I20260709 13:18:41.542169 16296 raft_consensus.cc:2468] T 5a959dbecc2648eb85d1cc550f5a3245 P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 1.
I20260709 13:18:41.541816 16430 consensus_queue.cc:237] T 5a959dbecc2648eb85d1cc550f5a3245 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:41.552958 16342 tablet_bootstrap.cc:654] T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045: Neither blocks nor log segments found. Creating new log.
W20260709 13:18:41.539544 16115 leader_election.cc:343] T d16dfad0c3b540e5b4b86492ddf87208 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:18:41.560267 16116 leader_election.cc:304] T d16dfad0c3b540e5b4b86492ddf87208 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 90f3da57796b488c9087a2f523adf045
I20260709 13:18:41.566066 16048 catalog_manager.cc:5697] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b reported cstate change: term changed from 0 to 1, leader changed from <none> to 647a343a542d4a139fe106c19095129b (127.14.177.129). New cstate: current_term: 1 leader_uuid: "647a343a542d4a139fe106c19095129b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:41.569526 16396 raft_consensus.cc:2804] T d16dfad0c3b540e5b4b86492ddf87208 P 647a343a542d4a139fe106c19095129b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:18:41.570075 16396 raft_consensus.cc:697] T d16dfad0c3b540e5b4b86492ddf87208 P 647a343a542d4a139fe106c19095129b [term 1 LEADER]: Becoming Leader. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:18:41.571040 16344 raft_consensus.cc:493] T d2ab0ac8347641e6955e9bce7d90c1a2 P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:41.571563 16344 raft_consensus.cc:515] T d2ab0ac8347641e6955e9bce7d90c1a2 P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:41.573734 16344 leader_election.cc:290] T d2ab0ac8347641e6955e9bce7d90c1a2 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:18:41.574807 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ab0ac8347641e6955e9bce7d90c1a2" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:18:41.571074 16396 consensus_queue.cc:237] T d16dfad0c3b540e5b4b86492ddf87208 P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:41.576367 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ab0ac8347641e6955e9bce7d90c1a2" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:18:41.577184 16300 raft_consensus.cc:2393] T d2ab0ac8347641e6955e9bce7d90c1a2 P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 1: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
I20260709 13:18:41.569993 16340 ts_tablet_manager.cc:1434] T 5a959dbecc2648eb85d1cc550f5a3245 P 647a343a542d4a139fe106c19095129b: Time spent starting tablet: real 0.147s	user 0.010s	sys 0.000s
I20260709 13:18:41.581632 16340 tablet_bootstrap.cc:492] T 829b4ad8c5a54eababeea1181b960d4b P 647a343a542d4a139fe106c19095129b: Bootstrap starting.
I20260709 13:18:41.582695 16463 consensus_queue.cc:1048] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:18:41.583844 16347 tablet_bootstrap.cc:492] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac: No bootstrap required, opened a new log
I20260709 13:18:41.584393 16347 ts_tablet_manager.cc:1403] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac: Time spent bootstrapping tablet: real 0.146s	user 0.015s	sys 0.008s
I20260709 13:18:41.586373 16047 catalog_manager.cc:5697] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 90f3da57796b488c9087a2f523adf045 reported cstate change: term changed from 0 to 1, leader changed from <none> to 90f3da57796b488c9087a2f523adf045 (127.14.177.131). New cstate: current_term: 1 leader_uuid: "90f3da57796b488c9087a2f523adf045" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } health_report { overall_health: HEALTHY } } }
I20260709 13:18:41.588377 16347 raft_consensus.cc:359] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:41.590782 16347 raft_consensus.cc:385] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:41.591149 16347 raft_consensus.cc:740] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Initialized, Role: FOLLOWER
I20260709 13:18:41.592010 16347 consensus_queue.cc:260] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:41.599678 16347 ts_tablet_manager.cc:1434] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac: Time spent starting tablet: real 0.015s	user 0.009s	sys 0.000s
I20260709 13:18:41.600860 16347 tablet_bootstrap.cc:492] T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac: Bootstrap starting.
I20260709 13:18:41.607776 16048 catalog_manager.cc:5697] T 5a959dbecc2648eb85d1cc550f5a3245 P 1ca2725d290e47d68b1a4b609c9816ac reported cstate change: term changed from 0 to 1, leader changed from <none> to 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130). New cstate: current_term: 1 leader_uuid: "1ca2725d290e47d68b1a4b609c9816ac" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:41.614264 16340 tablet_bootstrap.cc:654] T 829b4ad8c5a54eababeea1181b960d4b P 647a343a542d4a139fe106c19095129b: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:41.616768 16342 tablet_bootstrap.cc:492] T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045: No bootstrap required, opened a new log
I20260709 13:18:41.617179 16342 ts_tablet_manager.cc:1403] T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045: Time spent bootstrapping tablet: real 0.194s	user 0.032s	sys 0.022s
I20260709 13:18:41.615178 16373 consensus_queue.cc:1048] T 3884f864a96d428785895e5463e5728f P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.004s
I20260709 13:18:41.631671 16345 consensus_queue.cc:1048] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:18:41.633661 16500 consensus_queue.cc:1048] T d2ab0ac8347641e6955e9bce7d90c1a2 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:18:41.636350 16116 leader_election.cc:304] T d2ab0ac8347641e6955e9bce7d90c1a2 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:18:41.645135 16344 raft_consensus.cc:3060] T d2ab0ac8347641e6955e9bce7d90c1a2 P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:41.645859 16347 tablet_bootstrap.cc:654] T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:41.671118 16516 consensus_queue.cc:1048] T 3884f864a96d428785895e5463e5728f P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:18:41.680846 16344 raft_consensus.cc:2749] T d2ab0ac8347641e6955e9bce7d90c1a2 P 647a343a542d4a139fe106c19095129b [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:18:41.695098 16342 raft_consensus.cc:359] T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:41.696234 16342 raft_consensus.cc:385] T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:41.696537 16342 raft_consensus.cc:740] T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Initialized, Role: FOLLOWER
I20260709 13:18:41.699851 16342 consensus_queue.cc:260] T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:41.703891 16342 ts_tablet_manager.cc:1434] T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045: Time spent starting tablet: real 0.087s	user 0.004s	sys 0.004s
I20260709 13:18:41.704957 16342 tablet_bootstrap.cc:492] T c0eab25ce4c64e018a041beeda2867db P 90f3da57796b488c9087a2f523adf045: Bootstrap starting.
I20260709 13:18:41.711445 16376 consensus_queue.cc:1048] T 68a6838d82ef4cac83507f906139d90f P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:18:41.719040 16347 tablet_bootstrap.cc:492] T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac: No bootstrap required, opened a new log
I20260709 13:18:41.719985 16046 catalog_manager.cc:5697] T d16dfad0c3b540e5b4b86492ddf87208 P 647a343a542d4a139fe106c19095129b reported cstate change: term changed from 0 to 1, leader changed from <none> to 647a343a542d4a139fe106c19095129b (127.14.177.129). New cstate: current_term: 1 leader_uuid: "647a343a542d4a139fe106c19095129b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:41.735005 16500 consensus_queue.cc:1048] T d2ab0ac8347641e6955e9bce7d90c1a2 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:18:41.737036 16347 ts_tablet_manager.cc:1403] T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac: Time spent bootstrapping tablet: real 0.136s	user 0.020s	sys 0.014s
I20260709 13:18:41.749495 16342 tablet_bootstrap.cc:654] T c0eab25ce4c64e018a041beeda2867db P 90f3da57796b488c9087a2f523adf045: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:41.749285 16347 raft_consensus.cc:359] T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:41.750969 16347 raft_consensus.cc:385] T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:41.751260 16347 raft_consensus.cc:740] T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Initialized, Role: FOLLOWER
I20260709 13:18:41.752485 16347 consensus_queue.cc:260] T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:41.753961 16340 tablet_bootstrap.cc:492] T 829b4ad8c5a54eababeea1181b960d4b P 647a343a542d4a139fe106c19095129b: No bootstrap required, opened a new log
I20260709 13:18:41.754429 16340 ts_tablet_manager.cc:1403] T 829b4ad8c5a54eababeea1181b960d4b P 647a343a542d4a139fe106c19095129b: Time spent bootstrapping tablet: real 0.173s	user 0.012s	sys 0.020s
I20260709 13:18:41.757068 16347 ts_tablet_manager.cc:1434] T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac: Time spent starting tablet: real 0.020s	user 0.003s	sys 0.002s
I20260709 13:18:41.763399 16347 tablet_bootstrap.cc:492] T af57996a8be84d398f0f023e8d08ef77 P 1ca2725d290e47d68b1a4b609c9816ac: Bootstrap starting.
I20260709 13:18:41.761564 16340 raft_consensus.cc:359] T 829b4ad8c5a54eababeea1181b960d4b P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:41.766017 16340 raft_consensus.cc:385] T 829b4ad8c5a54eababeea1181b960d4b P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:41.766407 16340 raft_consensus.cc:740] T 829b4ad8c5a54eababeea1181b960d4b P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Initialized, Role: FOLLOWER
I20260709 13:18:41.767760 16340 consensus_queue.cc:260] T 829b4ad8c5a54eababeea1181b960d4b P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:41.769526 16340 ts_tablet_manager.cc:1434] T 829b4ad8c5a54eababeea1181b960d4b P 647a343a542d4a139fe106c19095129b: Time spent starting tablet: real 0.015s	user 0.006s	sys 0.001s
I20260709 13:18:41.770442 16340 tablet_bootstrap.cc:492] T c0eab25ce4c64e018a041beeda2867db P 647a343a542d4a139fe106c19095129b: Bootstrap starting.
I20260709 13:18:41.772416 16342 tablet_bootstrap.cc:492] T c0eab25ce4c64e018a041beeda2867db P 90f3da57796b488c9087a2f523adf045: No bootstrap required, opened a new log
I20260709 13:18:41.773250 16342 ts_tablet_manager.cc:1403] T c0eab25ce4c64e018a041beeda2867db P 90f3da57796b488c9087a2f523adf045: Time spent bootstrapping tablet: real 0.069s	user 0.010s	sys 0.004s
I20260709 13:18:41.783375 16342 raft_consensus.cc:359] T c0eab25ce4c64e018a041beeda2867db P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:41.784061 16342 raft_consensus.cc:385] T c0eab25ce4c64e018a041beeda2867db P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:41.784569 16342 raft_consensus.cc:740] T c0eab25ce4c64e018a041beeda2867db P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Initialized, Role: FOLLOWER
I20260709 13:18:41.785171 16342 consensus_queue.cc:260] T c0eab25ce4c64e018a041beeda2867db P 90f3da57796b488c9087a2f523adf045 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:41.785903 16340 tablet_bootstrap.cc:654] T c0eab25ce4c64e018a041beeda2867db P 647a343a542d4a139fe106c19095129b: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:41.786585 16516 raft_consensus.cc:493] T 829b4ad8c5a54eababeea1181b960d4b P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:41.787030 16516 raft_consensus.cc:515] T 829b4ad8c5a54eababeea1181b960d4b P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:41.787448 16342 ts_tablet_manager.cc:1434] T c0eab25ce4c64e018a041beeda2867db P 90f3da57796b488c9087a2f523adf045: Time spent starting tablet: real 0.014s	user 0.006s	sys 0.000s
I20260709 13:18:41.788398 16342 tablet_bootstrap.cc:492] T 62fe3346cd3c42c4a55e14af9073bca4 P 90f3da57796b488c9087a2f523adf045: Bootstrap starting.
I20260709 13:18:41.789310 16516 leader_election.cc:290] T 829b4ad8c5a54eababeea1181b960d4b P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:18:41.790418 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "829b4ad8c5a54eababeea1181b960d4b" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:18:41.791931 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "829b4ad8c5a54eababeea1181b960d4b" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:18:41.792531 16300 raft_consensus.cc:2468] T 829b4ad8c5a54eababeea1181b960d4b P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 0.
I20260709 13:18:41.794282 16115 leader_election.cc:304] T 829b4ad8c5a54eababeea1181b960d4b P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:18:41.795059 16516 raft_consensus.cc:2804] T 829b4ad8c5a54eababeea1181b960d4b P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:18:41.795490 16516 raft_consensus.cc:493] T 829b4ad8c5a54eababeea1181b960d4b P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:41.795814 16516 raft_consensus.cc:3060] T 829b4ad8c5a54eababeea1181b960d4b P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:18:41.796489 16116 leader_election.cc:343] T 829b4ad8c5a54eababeea1181b960d4b P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:18:41.798224 16500 consensus_queue.cc:1048] T 68a6838d82ef4cac83507f906139d90f P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:18:41.804610 16516 raft_consensus.cc:515] T 829b4ad8c5a54eababeea1181b960d4b P 647a343a542d4a139fe106c19095129b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:41.819972 16516 leader_election.cc:290] T 829b4ad8c5a54eababeea1181b960d4b P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 1 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:18:41.821108 16296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "829b4ad8c5a54eababeea1181b960d4b" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:18:41.821727 16296 raft_consensus.cc:3060] T 829b4ad8c5a54eababeea1181b960d4b P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:41.827235 16500 raft_consensus.cc:493] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:41.821113 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "829b4ad8c5a54eababeea1181b960d4b" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:18:41.829988 16347 tablet_bootstrap.cc:654] T af57996a8be84d398f0f023e8d08ef77 P 1ca2725d290e47d68b1a4b609c9816ac: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:41.827908 16500 raft_consensus.cc:515] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:41.832500 16296 raft_consensus.cc:2468] T 829b4ad8c5a54eababeea1181b960d4b P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 1.
I20260709 13:18:41.833647 16115 leader_election.cc:304] T 829b4ad8c5a54eababeea1181b960d4b P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:18:41.834357 16500 leader_election.cc:290] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
W20260709 13:18:41.835310 16116 leader_election.cc:343] T 829b4ad8c5a54eababeea1181b960d4b P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:18:41.835642 16516 raft_consensus.cc:2804] T 829b4ad8c5a54eababeea1181b960d4b P 647a343a542d4a139fe106c19095129b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:18:41.836612 16516 raft_consensus.cc:697] T 829b4ad8c5a54eababeea1181b960d4b P 647a343a542d4a139fe106c19095129b [term 1 LEADER]: Becoming Leader. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:18:41.837924 16516 consensus_queue.cc:237] T 829b4ad8c5a54eababeea1181b960d4b P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:41.839084 16296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66ae176961b74813a58bc15c9c4c6f2a" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:18:41.839717 16296 raft_consensus.cc:2468] T 66ae176961b74813a58bc15c9c4c6f2a P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 0.
I20260709 13:18:41.839673 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66ae176961b74813a58bc15c9c4c6f2a" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:18:41.840896 16190 leader_election.cc:304] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:18:41.842535 16376 raft_consensus.cc:2804] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:18:41.843001 16376 raft_consensus.cc:493] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:41.843412 16376 raft_consensus.cc:3060] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:18:41.843986 16191 leader_election.cc:343] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:18:41.858115 16347 tablet_bootstrap.cc:492] T af57996a8be84d398f0f023e8d08ef77 P 1ca2725d290e47d68b1a4b609c9816ac: No bootstrap required, opened a new log
I20260709 13:18:41.858563 16347 ts_tablet_manager.cc:1403] T af57996a8be84d398f0f023e8d08ef77 P 1ca2725d290e47d68b1a4b609c9816ac: Time spent bootstrapping tablet: real 0.095s	user 0.016s	sys 0.002s
I20260709 13:18:41.858410 16376 raft_consensus.cc:515] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:41.861552 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66ae176961b74813a58bc15c9c4c6f2a" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:18:41.861909 16340 tablet_bootstrap.cc:492] T c0eab25ce4c64e018a041beeda2867db P 647a343a542d4a139fe106c19095129b: No bootstrap required, opened a new log
I20260709 13:18:41.861539 16347 raft_consensus.cc:359] T af57996a8be84d398f0f023e8d08ef77 P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:41.862582 16340 ts_tablet_manager.cc:1403] T c0eab25ce4c64e018a041beeda2867db P 647a343a542d4a139fe106c19095129b: Time spent bootstrapping tablet: real 0.092s	user 0.006s	sys 0.016s
I20260709 13:18:41.862650 16347 raft_consensus.cc:385] T af57996a8be84d398f0f023e8d08ef77 P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260709 13:18:41.862977 16191 leader_election.cc:343] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:18:41.863194 16347 raft_consensus.cc:740] T af57996a8be84d398f0f023e8d08ef77 P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Initialized, Role: FOLLOWER
I20260709 13:18:41.864363 16347 consensus_queue.cc:260] T af57996a8be84d398f0f023e8d08ef77 P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:41.865006 16296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66ae176961b74813a58bc15c9c4c6f2a" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:18:41.865633 16296 raft_consensus.cc:3060] T 66ae176961b74813a58bc15c9c4c6f2a P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:41.866276 16376 leader_election.cc:290] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 1 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:18:41.865821 16340 raft_consensus.cc:359] T c0eab25ce4c64e018a041beeda2867db P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:41.867362 16340 raft_consensus.cc:385] T c0eab25ce4c64e018a041beeda2867db P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:41.867602 16340 raft_consensus.cc:740] T c0eab25ce4c64e018a041beeda2867db P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Initialized, Role: FOLLOWER
I20260709 13:18:41.868376 16340 consensus_queue.cc:260] T c0eab25ce4c64e018a041beeda2867db P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:41.870348 16340 ts_tablet_manager.cc:1434] T c0eab25ce4c64e018a041beeda2867db P 647a343a542d4a139fe106c19095129b: Time spent starting tablet: real 0.007s	user 0.001s	sys 0.003s
I20260709 13:18:41.871263 16340 tablet_bootstrap.cc:492] T b87147731a68400e8cce9b879daa7c5d P 647a343a542d4a139fe106c19095129b: Bootstrap starting.
I20260709 13:18:41.872493 16342 tablet_bootstrap.cc:654] T 62fe3346cd3c42c4a55e14af9073bca4 P 90f3da57796b488c9087a2f523adf045: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:41.876757 16340 tablet_bootstrap.cc:654] T b87147731a68400e8cce9b879daa7c5d P 647a343a542d4a139fe106c19095129b: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:41.881577 16342 tablet_bootstrap.cc:492] T 62fe3346cd3c42c4a55e14af9073bca4 P 90f3da57796b488c9087a2f523adf045: No bootstrap required, opened a new log
I20260709 13:18:41.882800 16296 raft_consensus.cc:2468] T 66ae176961b74813a58bc15c9c4c6f2a P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 1.
I20260709 13:18:41.883402 16342 ts_tablet_manager.cc:1403] T 62fe3346cd3c42c4a55e14af9073bca4 P 90f3da57796b488c9087a2f523adf045: Time spent bootstrapping tablet: real 0.095s	user 0.011s	sys 0.001s
I20260709 13:18:41.880484 16347 ts_tablet_manager.cc:1434] T af57996a8be84d398f0f023e8d08ef77 P 1ca2725d290e47d68b1a4b609c9816ac: Time spent starting tablet: real 0.021s	user 0.004s	sys 0.004s
I20260709 13:18:41.884567 16190 leader_election.cc:304] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 647a343a542d4a139fe106c19095129b
I20260709 13:18:41.884979 16347 tablet_bootstrap.cc:492] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 1ca2725d290e47d68b1a4b609c9816ac: Bootstrap starting.
I20260709 13:18:41.885747 16342 raft_consensus.cc:359] T 62fe3346cd3c42c4a55e14af9073bca4 P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:41.886377 16342 raft_consensus.cc:385] T 62fe3346cd3c42c4a55e14af9073bca4 P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:41.886624 16342 raft_consensus.cc:740] T 62fe3346cd3c42c4a55e14af9073bca4 P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Initialized, Role: FOLLOWER
I20260709 13:18:41.887269 16342 consensus_queue.cc:260] T 62fe3346cd3c42c4a55e14af9073bca4 P 90f3da57796b488c9087a2f523adf045 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:41.889181 16342 ts_tablet_manager.cc:1434] T 62fe3346cd3c42c4a55e14af9073bca4 P 90f3da57796b488c9087a2f523adf045: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.000s
I20260709 13:18:41.890056 16345 raft_consensus.cc:493] T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:41.890252 16376 raft_consensus.cc:2804] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:18:41.889981 16047 catalog_manager.cc:5697] T 829b4ad8c5a54eababeea1181b960d4b P 647a343a542d4a139fe106c19095129b reported cstate change: term changed from 0 to 1, leader changed from <none> to 647a343a542d4a139fe106c19095129b (127.14.177.129). New cstate: current_term: 1 leader_uuid: "647a343a542d4a139fe106c19095129b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:41.890765 16376 raft_consensus.cc:697] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [term 1 LEADER]: Becoming Leader. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:18:41.890604 16345 raft_consensus.cc:515] T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:41.891721 16376 consensus_queue.cc:237] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:41.892791 16345 leader_election.cc:290] T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:18:41.894342 16342 tablet_bootstrap.cc:492] T d16dfad0c3b540e5b4b86492ddf87208 P 90f3da57796b488c9087a2f523adf045: Bootstrap starting.
I20260709 13:18:41.896334 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c61a39fd4f84424b129c1a6f9ba404d" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:18:41.897218 16225 raft_consensus.cc:2468] T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 0.
I20260709 13:18:41.898983 16265 leader_election.cc:304] T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:18:41.900295 16342 tablet_bootstrap.cc:654] T d16dfad0c3b540e5b4b86492ddf87208 P 90f3da57796b488c9087a2f523adf045: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:41.900410 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c61a39fd4f84424b129c1a6f9ba404d" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:18:41.902561 16345 raft_consensus.cc:2804] T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:18:41.902931 16345 raft_consensus.cc:493] T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:41.903338 16345 raft_consensus.cc:3060] T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:18:41.903820 16265 leader_election.cc:343] T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:18:41.910133 16048 catalog_manager.cc:5697] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac reported cstate change: term changed from 0 to 1, leader changed from <none> to 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130). New cstate: current_term: 1 leader_uuid: "1ca2725d290e47d68b1a4b609c9816ac" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:41.911265 16340 tablet_bootstrap.cc:492] T b87147731a68400e8cce9b879daa7c5d P 647a343a542d4a139fe106c19095129b: No bootstrap required, opened a new log
I20260709 13:18:41.911818 16340 ts_tablet_manager.cc:1403] T b87147731a68400e8cce9b879daa7c5d P 647a343a542d4a139fe106c19095129b: Time spent bootstrapping tablet: real 0.041s	user 0.009s	sys 0.004s
I20260709 13:18:41.914600 16340 raft_consensus.cc:359] T b87147731a68400e8cce9b879daa7c5d P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:41.915268 16340 raft_consensus.cc:385] T b87147731a68400e8cce9b879daa7c5d P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:41.915542 16340 raft_consensus.cc:740] T b87147731a68400e8cce9b879daa7c5d P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Initialized, Role: FOLLOWER
I20260709 13:18:41.916122 16340 consensus_queue.cc:260] T b87147731a68400e8cce9b879daa7c5d P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:41.921191 16340 ts_tablet_manager.cc:1434] T b87147731a68400e8cce9b879daa7c5d P 647a343a542d4a139fe106c19095129b: Time spent starting tablet: real 0.009s	user 0.005s	sys 0.000s
I20260709 13:18:41.921698 16342 tablet_bootstrap.cc:492] T d16dfad0c3b540e5b4b86492ddf87208 P 90f3da57796b488c9087a2f523adf045: No bootstrap required, opened a new log
I20260709 13:18:41.921801 16347 tablet_bootstrap.cc:654] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 1ca2725d290e47d68b1a4b609c9816ac: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:41.922240 16342 ts_tablet_manager.cc:1403] T d16dfad0c3b540e5b4b86492ddf87208 P 90f3da57796b488c9087a2f523adf045: Time spent bootstrapping tablet: real 0.028s	user 0.008s	sys 0.008s
I20260709 13:18:41.922490 16340 tablet_bootstrap.cc:492] T 74ca8140f35c4ae29e8a664f6bdfc829 P 647a343a542d4a139fe106c19095129b: Bootstrap starting.
I20260709 13:18:41.924481 16345 raft_consensus.cc:515] T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:41.925197 16342 raft_consensus.cc:359] T d16dfad0c3b540e5b4b86492ddf87208 P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:41.926026 16342 raft_consensus.cc:385] T d16dfad0c3b540e5b4b86492ddf87208 P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:41.926339 16342 raft_consensus.cc:740] T d16dfad0c3b540e5b4b86492ddf87208 P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Initialized, Role: FOLLOWER
I20260709 13:18:41.927350 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c61a39fd4f84424b129c1a6f9ba404d" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:18:41.927083 16342 consensus_queue.cc:260] T d16dfad0c3b540e5b4b86492ddf87208 P 90f3da57796b488c9087a2f523adf045 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:18:41.928634 16265 leader_election.cc:343] T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:18:41.930661 16340 tablet_bootstrap.cc:654] T 74ca8140f35c4ae29e8a664f6bdfc829 P 647a343a542d4a139fe106c19095129b: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:41.930542 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c61a39fd4f84424b129c1a6f9ba404d" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:18:41.931221 16225 raft_consensus.cc:3060] T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:41.934134 16342 ts_tablet_manager.cc:1434] T d16dfad0c3b540e5b4b86492ddf87208 P 90f3da57796b488c9087a2f523adf045: Time spent starting tablet: real 0.011s	user 0.007s	sys 0.000s
I20260709 13:18:41.935497 16342 tablet_bootstrap.cc:492] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045: Bootstrap starting.
I20260709 13:18:41.936697 16516 consensus_queue.cc:1048] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:18:41.942394 16376 consensus_queue.cc:1048] T 5a959dbecc2648eb85d1cc550f5a3245 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:18:41.947475 16342 tablet_bootstrap.cc:654] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:41.955947 16225 raft_consensus.cc:2468] T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 1.
I20260709 13:18:41.957095 16265 leader_election.cc:304] T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 647a343a542d4a139fe106c19095129b
I20260709 13:18:41.957929 16463 raft_consensus.cc:2804] T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:18:41.958376 16463 raft_consensus.cc:697] T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045 [term 1 LEADER]: Becoming Leader. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:18:41.959226 16463 consensus_queue.cc:237] T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:41.960707 16342 tablet_bootstrap.cc:492] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045: No bootstrap required, opened a new log
I20260709 13:18:41.961189 16342 ts_tablet_manager.cc:1403] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045: Time spent bootstrapping tablet: real 0.026s	user 0.010s	sys 0.009s
I20260709 13:18:41.930938 16345 leader_election.cc:290] T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 1 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:18:41.968643 16430 consensus_queue.cc:1048] T 5a959dbecc2648eb85d1cc550f5a3245 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:18:41.977734 16520 consensus_queue.cc:1048] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:18:41.985895 16340 tablet_bootstrap.cc:492] T 74ca8140f35c4ae29e8a664f6bdfc829 P 647a343a542d4a139fe106c19095129b: No bootstrap required, opened a new log
I20260709 13:18:41.987181 16342 raft_consensus.cc:359] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:41.988178 16342 raft_consensus.cc:385] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:41.988481 16342 raft_consensus.cc:740] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Initialized, Role: FOLLOWER
I20260709 13:18:41.989216 16342 consensus_queue.cc:260] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:41.991531 16342 ts_tablet_manager.cc:1434] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045: Time spent starting tablet: real 0.030s	user 0.009s	sys 0.000s
I20260709 13:18:41.991725 16347 tablet_bootstrap.cc:492] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 1ca2725d290e47d68b1a4b609c9816ac: No bootstrap required, opened a new log
I20260709 13:18:41.992376 16347 ts_tablet_manager.cc:1403] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 1ca2725d290e47d68b1a4b609c9816ac: Time spent bootstrapping tablet: real 0.108s	user 0.012s	sys 0.001s
I20260709 13:18:41.992486 16342 tablet_bootstrap.cc:492] T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045: Bootstrap starting.
I20260709 13:18:41.993245 16340 ts_tablet_manager.cc:1403] T 74ca8140f35c4ae29e8a664f6bdfc829 P 647a343a542d4a139fe106c19095129b: Time spent bootstrapping tablet: real 0.071s	user 0.022s	sys 0.010s
I20260709 13:18:41.998942 16340 raft_consensus.cc:359] T 74ca8140f35c4ae29e8a664f6bdfc829 P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:41.999789 16340 raft_consensus.cc:385] T 74ca8140f35c4ae29e8a664f6bdfc829 P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:42.000108 16340 raft_consensus.cc:740] T 74ca8140f35c4ae29e8a664f6bdfc829 P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Initialized, Role: FOLLOWER
I20260709 13:18:42.000792 16340 consensus_queue.cc:260] T 74ca8140f35c4ae29e8a664f6bdfc829 P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:42.003026 16340 ts_tablet_manager.cc:1434] T 74ca8140f35c4ae29e8a664f6bdfc829 P 647a343a542d4a139fe106c19095129b: Time spent starting tablet: real 0.009s	user 0.003s	sys 0.000s
I20260709 13:18:42.010070 16345 consensus_queue.cc:1048] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:18:42.010964 16342 tablet_bootstrap.cc:654] T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:42.014074 16347 raft_consensus.cc:359] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:42.014760 16347 raft_consensus.cc:385] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:42.015026 16347 raft_consensus.cc:740] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Initialized, Role: FOLLOWER
I20260709 13:18:42.017477 16347 consensus_queue.cc:260] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:42.024396 16347 ts_tablet_manager.cc:1434] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 1ca2725d290e47d68b1a4b609c9816ac: Time spent starting tablet: real 0.032s	user 0.007s	sys 0.000s
I20260709 13:18:42.025220 16347 tablet_bootstrap.cc:492] T b87147731a68400e8cce9b879daa7c5d P 1ca2725d290e47d68b1a4b609c9816ac: Bootstrap starting.
I20260709 13:18:42.030149 16347 tablet_bootstrap.cc:654] T b87147731a68400e8cce9b879daa7c5d P 1ca2725d290e47d68b1a4b609c9816ac: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:42.030984 16342 tablet_bootstrap.cc:492] T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045: No bootstrap required, opened a new log
I20260709 13:18:42.031958 16342 ts_tablet_manager.cc:1403] T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045: Time spent bootstrapping tablet: real 0.040s	user 0.017s	sys 0.000s
I20260709 13:18:42.004031 16340 tablet_bootstrap.cc:492] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b: Bootstrap starting.
I20260709 13:18:42.041235 16048 catalog_manager.cc:5697] T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045 reported cstate change: term changed from 0 to 1, leader changed from <none> to 90f3da57796b488c9087a2f523adf045 (127.14.177.131). New cstate: current_term: 1 leader_uuid: "90f3da57796b488c9087a2f523adf045" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } health_report { overall_health: HEALTHY } } }
I20260709 13:18:42.042057 16340 tablet_bootstrap.cc:654] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:42.053910 16516 consensus_queue.cc:1048] T d16dfad0c3b540e5b4b86492ddf87208 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 13:18:42.063040 16224 raft_consensus.cc:3060] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:42.066172 16342 raft_consensus.cc:359] T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:42.066957 16342 raft_consensus.cc:385] T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:42.067265 16342 raft_consensus.cc:740] T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Initialized, Role: FOLLOWER
I20260709 13:18:42.069278 16342 consensus_queue.cc:260] T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:42.071578 16342 ts_tablet_manager.cc:1434] T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045: Time spent starting tablet: real 0.039s	user 0.007s	sys 0.000s
I20260709 13:18:42.072574 16342 tablet_bootstrap.cc:492] T b87147731a68400e8cce9b879daa7c5d P 90f3da57796b488c9087a2f523adf045: Bootstrap starting.
I20260709 13:18:42.078713 16463 consensus_queue.cc:1048] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:18:42.097498 16347 tablet_bootstrap.cc:492] T b87147731a68400e8cce9b879daa7c5d P 1ca2725d290e47d68b1a4b609c9816ac: No bootstrap required, opened a new log
I20260709 13:18:42.097424 16296 raft_consensus.cc:3060] T d16dfad0c3b540e5b4b86492ddf87208 P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:42.098155 16347 ts_tablet_manager.cc:1403] T b87147731a68400e8cce9b879daa7c5d P 1ca2725d290e47d68b1a4b609c9816ac: Time spent bootstrapping tablet: real 0.073s	user 0.021s	sys 0.007s
I20260709 13:18:42.098922 16516 raft_consensus.cc:493] T c0eab25ce4c64e018a041beeda2867db P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:42.099390 16516 raft_consensus.cc:515] T c0eab25ce4c64e018a041beeda2867db P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:42.102267 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0eab25ce4c64e018a041beeda2867db" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:18:42.102545 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0eab25ce4c64e018a041beeda2867db" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:18:42.103226 16300 raft_consensus.cc:2468] T c0eab25ce4c64e018a041beeda2867db P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 0.
W20260709 13:18:42.103705 16116 leader_election.cc:343] T c0eab25ce4c64e018a041beeda2867db P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:18:42.104036 16347 raft_consensus.cc:359] T b87147731a68400e8cce9b879daa7c5d P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:42.104511 16115 leader_election.cc:304] T c0eab25ce4c64e018a041beeda2867db P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac
I20260709 13:18:42.105352 16516 leader_election.cc:290] T c0eab25ce4c64e018a041beeda2867db P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:18:42.105556 16521 raft_consensus.cc:2804] T c0eab25ce4c64e018a041beeda2867db P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:18:42.106061 16521 raft_consensus.cc:493] T c0eab25ce4c64e018a041beeda2867db P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:42.106434 16521 raft_consensus.cc:3060] T c0eab25ce4c64e018a041beeda2867db P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:42.104732 16347 raft_consensus.cc:385] T b87147731a68400e8cce9b879daa7c5d P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:42.107002 16347 raft_consensus.cc:740] T b87147731a68400e8cce9b879daa7c5d P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Initialized, Role: FOLLOWER
I20260709 13:18:42.107064 16516 consensus_queue.cc:1048] T d16dfad0c3b540e5b4b86492ddf87208 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:18:42.107833 16347 consensus_queue.cc:260] T b87147731a68400e8cce9b879daa7c5d P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:42.116897 16521 raft_consensus.cc:515] T c0eab25ce4c64e018a041beeda2867db P 647a343a542d4a139fe106c19095129b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:42.117894 16342 tablet_bootstrap.cc:654] T b87147731a68400e8cce9b879daa7c5d P 90f3da57796b488c9087a2f523adf045: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:42.120313 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0eab25ce4c64e018a041beeda2867db" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:18:42.121826 16347 ts_tablet_manager.cc:1434] T b87147731a68400e8cce9b879daa7c5d P 1ca2725d290e47d68b1a4b609c9816ac: Time spent starting tablet: real 0.023s	user 0.007s	sys 0.000s
I20260709 13:18:42.122810 16347 tablet_bootstrap.cc:492] T 829b4ad8c5a54eababeea1181b960d4b P 1ca2725d290e47d68b1a4b609c9816ac: Bootstrap starting.
I20260709 13:18:42.131465 16340 tablet_bootstrap.cc:492] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b: No bootstrap required, opened a new log
W20260709 13:18:42.131765 16116 leader_election.cc:343] T c0eab25ce4c64e018a041beeda2867db P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:18:42.132045 16340 ts_tablet_manager.cc:1403] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b: Time spent bootstrapping tablet: real 0.128s	user 0.021s	sys 0.036s
I20260709 13:18:42.130995 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0eab25ce4c64e018a041beeda2867db" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:18:42.130483 16521 leader_election.cc:290] T c0eab25ce4c64e018a041beeda2867db P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 1 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:18:42.134053 16300 raft_consensus.cc:3060] T c0eab25ce4c64e018a041beeda2867db P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:42.138353 16340 raft_consensus.cc:359] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:42.139086 16340 raft_consensus.cc:385] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:42.139400 16340 raft_consensus.cc:740] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Initialized, Role: FOLLOWER
I20260709 13:18:42.140106 16340 consensus_queue.cc:260] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:42.142262 16340 ts_tablet_manager.cc:1434] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b: Time spent starting tablet: real 0.010s	user 0.004s	sys 0.000s
I20260709 13:18:42.142762 16300 raft_consensus.cc:2468] T c0eab25ce4c64e018a041beeda2867db P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 1.
I20260709 13:18:42.145305 16347 tablet_bootstrap.cc:654] T 829b4ad8c5a54eababeea1181b960d4b P 1ca2725d290e47d68b1a4b609c9816ac: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:42.147313 16340 tablet_bootstrap.cc:492] T af57996a8be84d398f0f023e8d08ef77 P 647a343a542d4a139fe106c19095129b: Bootstrap starting.
I20260709 13:18:42.147411 16115 leader_election.cc:304] T c0eab25ce4c64e018a041beeda2867db P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac
I20260709 13:18:42.153833 16516 raft_consensus.cc:2804] T c0eab25ce4c64e018a041beeda2867db P 647a343a542d4a139fe106c19095129b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:18:42.160470 16516 raft_consensus.cc:697] T c0eab25ce4c64e018a041beeda2867db P 647a343a542d4a139fe106c19095129b [term 1 LEADER]: Becoming Leader. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:18:42.161760 16516 consensus_queue.cc:237] T c0eab25ce4c64e018a041beeda2867db P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:42.169414 16342 tablet_bootstrap.cc:492] T b87147731a68400e8cce9b879daa7c5d P 90f3da57796b488c9087a2f523adf045: No bootstrap required, opened a new log
I20260709 13:18:42.169896 16342 ts_tablet_manager.cc:1403] T b87147731a68400e8cce9b879daa7c5d P 90f3da57796b488c9087a2f523adf045: Time spent bootstrapping tablet: real 0.098s	user 0.015s	sys 0.004s
I20260709 13:18:42.173836 16342 raft_consensus.cc:359] T b87147731a68400e8cce9b879daa7c5d P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:42.174823 16342 raft_consensus.cc:385] T b87147731a68400e8cce9b879daa7c5d P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:42.175226 16342 raft_consensus.cc:740] T b87147731a68400e8cce9b879daa7c5d P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Initialized, Role: FOLLOWER
I20260709 13:18:42.182029 16342 consensus_queue.cc:260] T b87147731a68400e8cce9b879daa7c5d P 90f3da57796b488c9087a2f523adf045 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:42.185781 16340 tablet_bootstrap.cc:654] T af57996a8be84d398f0f023e8d08ef77 P 647a343a542d4a139fe106c19095129b: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:42.187039 16342 ts_tablet_manager.cc:1434] T b87147731a68400e8cce9b879daa7c5d P 90f3da57796b488c9087a2f523adf045: Time spent starting tablet: real 0.017s	user 0.006s	sys 0.000s
I20260709 13:18:42.191489 16340 tablet_bootstrap.cc:492] T af57996a8be84d398f0f023e8d08ef77 P 647a343a542d4a139fe106c19095129b: No bootstrap required, opened a new log
I20260709 13:18:42.191985 16340 ts_tablet_manager.cc:1403] T af57996a8be84d398f0f023e8d08ef77 P 647a343a542d4a139fe106c19095129b: Time spent bootstrapping tablet: real 0.049s	user 0.013s	sys 0.008s
I20260709 13:18:42.195195 16340 raft_consensus.cc:359] T af57996a8be84d398f0f023e8d08ef77 P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:42.195971 16340 raft_consensus.cc:385] T af57996a8be84d398f0f023e8d08ef77 P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:42.196280 16340 raft_consensus.cc:740] T af57996a8be84d398f0f023e8d08ef77 P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Initialized, Role: FOLLOWER
I20260709 13:18:42.197046 16347 tablet_bootstrap.cc:492] T 829b4ad8c5a54eababeea1181b960d4b P 1ca2725d290e47d68b1a4b609c9816ac: No bootstrap required, opened a new log
I20260709 13:18:42.197018 16340 consensus_queue.cc:260] T af57996a8be84d398f0f023e8d08ef77 P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:42.197676 16347 ts_tablet_manager.cc:1403] T 829b4ad8c5a54eababeea1181b960d4b P 1ca2725d290e47d68b1a4b609c9816ac: Time spent bootstrapping tablet: real 0.075s	user 0.013s	sys 0.008s
I20260709 13:18:42.196983 16048 catalog_manager.cc:5697] T c0eab25ce4c64e018a041beeda2867db P 647a343a542d4a139fe106c19095129b reported cstate change: term changed from 0 to 1, leader changed from <none> to 647a343a542d4a139fe106c19095129b (127.14.177.129). New cstate: current_term: 1 leader_uuid: "647a343a542d4a139fe106c19095129b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:42.198925 16340 ts_tablet_manager.cc:1434] T af57996a8be84d398f0f023e8d08ef77 P 647a343a542d4a139fe106c19095129b: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 13:18:42.199863 16340 tablet_bootstrap.cc:492] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b: Bootstrap starting.
I20260709 13:18:42.200114 16347 raft_consensus.cc:359] T 829b4ad8c5a54eababeea1181b960d4b P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:42.200764 16347 raft_consensus.cc:385] T 829b4ad8c5a54eababeea1181b960d4b P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:42.201011 16347 raft_consensus.cc:740] T 829b4ad8c5a54eababeea1181b960d4b P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Initialized, Role: FOLLOWER
I20260709 13:18:42.201639 16347 consensus_queue.cc:260] T 829b4ad8c5a54eababeea1181b960d4b P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:42.203629 16347 ts_tablet_manager.cc:1434] T 829b4ad8c5a54eababeea1181b960d4b P 1ca2725d290e47d68b1a4b609c9816ac: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.000s
I20260709 13:18:42.204453 16347 tablet_bootstrap.cc:492] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac: Bootstrap starting.
I20260709 13:18:42.210305 16347 tablet_bootstrap.cc:654] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:42.210546 16340 tablet_bootstrap.cc:654] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:42.215641 16342 tablet_bootstrap.cc:492] T 5718c623ebee4d2bb770f0f9ddb2f164 P 90f3da57796b488c9087a2f523adf045: Bootstrap starting.
I20260709 13:18:42.219448 16340 tablet_bootstrap.cc:492] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b: No bootstrap required, opened a new log
I20260709 13:18:42.219923 16340 ts_tablet_manager.cc:1403] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b: Time spent bootstrapping tablet: real 0.020s	user 0.011s	sys 0.004s
I20260709 13:18:42.221437 16342 tablet_bootstrap.cc:654] T 5718c623ebee4d2bb770f0f9ddb2f164 P 90f3da57796b488c9087a2f523adf045: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:42.222417 16340 raft_consensus.cc:359] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:42.222970 16340 raft_consensus.cc:385] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:42.223240 16340 raft_consensus.cc:740] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Initialized, Role: FOLLOWER
I20260709 13:18:42.225531 16340 consensus_queue.cc:260] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:42.226603 16347 tablet_bootstrap.cc:492] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac: No bootstrap required, opened a new log
I20260709 13:18:42.227027 16347 ts_tablet_manager.cc:1403] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac: Time spent bootstrapping tablet: real 0.023s	user 0.000s	sys 0.011s
I20260709 13:18:42.227109 16342 tablet_bootstrap.cc:492] T 5718c623ebee4d2bb770f0f9ddb2f164 P 90f3da57796b488c9087a2f523adf045: No bootstrap required, opened a new log
I20260709 13:18:42.227552 16342 ts_tablet_manager.cc:1403] T 5718c623ebee4d2bb770f0f9ddb2f164 P 90f3da57796b488c9087a2f523adf045: Time spent bootstrapping tablet: real 0.012s	user 0.010s	sys 0.000s
I20260709 13:18:42.227736 16340 ts_tablet_manager.cc:1434] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 13:18:42.229071 16516 raft_consensus.cc:493] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:42.229554 16347 raft_consensus.cc:359] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:42.229861 16342 raft_consensus.cc:359] T 5718c623ebee4d2bb770f0f9ddb2f164 P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:42.229571 16516 raft_consensus.cc:515] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:42.230342 16347 raft_consensus.cc:385] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:42.230618 16347 raft_consensus.cc:740] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Initialized, Role: FOLLOWER
I20260709 13:18:42.230928 16340 tablet_bootstrap.cc:492] T ee9ed67006144797b1de6ea8c2013a68 P 647a343a542d4a139fe106c19095129b: Bootstrap starting.
I20260709 13:18:42.231240 16347 consensus_queue.cc:260] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:42.231861 16516 leader_election.cc:290] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:18:42.233469 16347 ts_tablet_manager.cc:1434] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 13:18:42.233284 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62fe3346cd3c42c4a55e14af9073bca4" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:18:42.234486 16347 tablet_bootstrap.cc:492] T c0eab25ce4c64e018a041beeda2867db P 1ca2725d290e47d68b1a4b609c9816ac: Bootstrap starting.
I20260709 13:18:42.234715 16296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62fe3346cd3c42c4a55e14af9073bca4" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:18:42.235399 16296 raft_consensus.cc:2468] T 62fe3346cd3c42c4a55e14af9073bca4 P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 0.
I20260709 13:18:42.237111 16115 leader_election.cc:304] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
W20260709 13:18:42.238045 16116 leader_election.cc:343] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:18:42.238940 16516 raft_consensus.cc:2804] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:18:42.239315 16516 raft_consensus.cc:493] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:42.239638 16516 raft_consensus.cc:3060] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:42.246248 16516 raft_consensus.cc:515] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:42.247367 16342 raft_consensus.cc:385] T 5718c623ebee4d2bb770f0f9ddb2f164 P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:42.247699 16342 raft_consensus.cc:740] T 5718c623ebee4d2bb770f0f9ddb2f164 P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Initialized, Role: FOLLOWER
I20260709 13:18:42.248484 16342 consensus_queue.cc:260] T 5718c623ebee4d2bb770f0f9ddb2f164 P 90f3da57796b488c9087a2f523adf045 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:42.250588 16342 ts_tablet_manager.cc:1434] T 5718c623ebee4d2bb770f0f9ddb2f164 P 90f3da57796b488c9087a2f523adf045: Time spent starting tablet: real 0.023s	user 0.004s	sys 0.000s
I20260709 13:18:42.252933 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62fe3346cd3c42c4a55e14af9073bca4" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:18:42.253309 16347 tablet_bootstrap.cc:654] T c0eab25ce4c64e018a041beeda2867db P 1ca2725d290e47d68b1a4b609c9816ac: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:42.254598 16432 raft_consensus.cc:493] T 829b4ad8c5a54eababeea1181b960d4b P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:42.255097 16116 leader_election.cc:343] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:18:42.255105 16432 raft_consensus.cc:515] T 829b4ad8c5a54eababeea1181b960d4b P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:42.257230 16340 tablet_bootstrap.cc:654] T ee9ed67006144797b1de6ea8c2013a68 P 647a343a542d4a139fe106c19095129b: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:42.257925 16432 leader_election.cc:290] T 829b4ad8c5a54eababeea1181b960d4b P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:18:42.259454 16345 raft_consensus.cc:493] T b87147731a68400e8cce9b879daa7c5d P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:42.259982 16345 raft_consensus.cc:515] T b87147731a68400e8cce9b879daa7c5d P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:42.262050 16345 leader_election.cc:290] T b87147731a68400e8cce9b879daa7c5d P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:18:42.262828 16295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "829b4ad8c5a54eababeea1181b960d4b" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:18:42.263669 16295 raft_consensus.cc:2393] T 829b4ad8c5a54eababeea1181b960d4b P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 1: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
I20260709 13:18:42.265069 16342 tablet_bootstrap.cc:492] T af57996a8be84d398f0f023e8d08ef77 P 90f3da57796b488c9087a2f523adf045: Bootstrap starting.
I20260709 13:18:42.265076 16347 tablet_bootstrap.cc:492] T c0eab25ce4c64e018a041beeda2867db P 1ca2725d290e47d68b1a4b609c9816ac: No bootstrap required, opened a new log
I20260709 13:18:42.265614 16347 ts_tablet_manager.cc:1403] T c0eab25ce4c64e018a041beeda2867db P 1ca2725d290e47d68b1a4b609c9816ac: Time spent bootstrapping tablet: real 0.031s	user 0.010s	sys 0.005s
I20260709 13:18:42.267620 16151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87147731a68400e8cce9b879daa7c5d" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:18:42.268568 16151 raft_consensus.cc:2468] T b87147731a68400e8cce9b879daa7c5d P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 0.
I20260709 13:18:42.268642 16347 raft_consensus.cc:359] T c0eab25ce4c64e018a041beeda2867db P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:42.269450 16347 raft_consensus.cc:385] T c0eab25ce4c64e018a041beeda2867db P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:42.269709 16347 raft_consensus.cc:740] T c0eab25ce4c64e018a041beeda2867db P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Initialized, Role: FOLLOWER
I20260709 13:18:42.269833 16516 leader_election.cc:290] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 1 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:18:42.270365 16347 consensus_queue.cc:260] T c0eab25ce4c64e018a041beeda2867db P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:42.272332 16265 leader_election.cc:304] T b87147731a68400e8cce9b879daa7c5d P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:18:42.272920 16342 tablet_bootstrap.cc:654] T af57996a8be84d398f0f023e8d08ef77 P 90f3da57796b488c9087a2f523adf045: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:42.275713 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87147731a68400e8cce9b879daa7c5d" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:18:42.276479 16347 ts_tablet_manager.cc:1434] T c0eab25ce4c64e018a041beeda2867db P 1ca2725d290e47d68b1a4b609c9816ac: Time spent starting tablet: real 0.011s	user 0.006s	sys 0.000s
I20260709 13:18:42.276633 16226 raft_consensus.cc:2468] T b87147731a68400e8cce9b879daa7c5d P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 0.
I20260709 13:18:42.277709 16347 tablet_bootstrap.cc:492] T 746aabf856ab401c9ef5e859371e7b67 P 1ca2725d290e47d68b1a4b609c9816ac: Bootstrap starting.
I20260709 13:18:42.280184 16432 raft_consensus.cc:493] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:42.280864 16432 raft_consensus.cc:515] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:42.283591 16516 raft_consensus.cc:493] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:42.284035 16516 raft_consensus.cc:515] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:42.285255 16376 raft_consensus.cc:493] T af57996a8be84d398f0f023e8d08ef77 P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:42.285877 16516 leader_election.cc:290] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:18:42.285946 16376 raft_consensus.cc:515] T af57996a8be84d398f0f023e8d08ef77 P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:42.287496 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5718c623ebee4d2bb770f0f9ddb2f164" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:18:42.288069 16226 raft_consensus.cc:2468] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 0.
I20260709 13:18:42.288414 16347 tablet_bootstrap.cc:654] T 746aabf856ab401c9ef5e859371e7b67 P 1ca2725d290e47d68b1a4b609c9816ac: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:42.288601 16376 leader_election.cc:290] T af57996a8be84d398f0f023e8d08ef77 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:18:42.289252 16116 leader_election.cc:304] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
I20260709 13:18:42.290019 16342 tablet_bootstrap.cc:492] T af57996a8be84d398f0f023e8d08ef77 P 90f3da57796b488c9087a2f523adf045: No bootstrap required, opened a new log
I20260709 13:18:42.290506 16342 ts_tablet_manager.cc:1403] T af57996a8be84d398f0f023e8d08ef77 P 90f3da57796b488c9087a2f523adf045: Time spent bootstrapping tablet: real 0.026s	user 0.014s	sys 0.003s
I20260709 13:18:42.290035 16516 raft_consensus.cc:2804] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:18:42.291585 16516 raft_consensus.cc:493] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:42.291904 16516 raft_consensus.cc:3060] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:42.292037 16295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af57996a8be84d398f0f023e8d08ef77" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:18:42.293148 16345 raft_consensus.cc:2804] T b87147731a68400e8cce9b879daa7c5d P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:18:42.293563 16345 raft_consensus.cc:493] T b87147731a68400e8cce9b879daa7c5d P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:42.293886 16345 raft_consensus.cc:3060] T b87147731a68400e8cce9b879daa7c5d P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:42.297930 16342 raft_consensus.cc:359] T af57996a8be84d398f0f023e8d08ef77 P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:42.298834 16342 raft_consensus.cc:385] T af57996a8be84d398f0f023e8d08ef77 P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:42.299196 16342 raft_consensus.cc:740] T af57996a8be84d398f0f023e8d08ef77 P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Initialized, Role: FOLLOWER
I20260709 13:18:42.299799 16432 leader_election.cc:290] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:18:42.300057 16342 consensus_queue.cc:260] T af57996a8be84d398f0f023e8d08ef77 P 90f3da57796b488c9087a2f523adf045 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:42.302122 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5718c623ebee4d2bb770f0f9ddb2f164" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:18:42.302906 16297 raft_consensus.cc:2468] T 5718c623ebee4d2bb770f0f9ddb2f164 P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 0.
I20260709 13:18:42.303655 16295 raft_consensus.cc:2468] T af57996a8be84d398f0f023e8d08ef77 P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 0.
I20260709 13:18:42.306348 16190 leader_election.cc:304] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:18:42.307410 16190 leader_election.cc:304] T af57996a8be84d398f0f023e8d08ef77 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:18:42.309082 16430 raft_consensus.cc:2804] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:18:42.309459 16430 raft_consensus.cc:493] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:42.309811 16430 raft_consensus.cc:3060] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:42.310698 16516 raft_consensus.cc:515] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:42.311987 16376 raft_consensus.cc:2804] T af57996a8be84d398f0f023e8d08ef77 P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:18:42.312572 16376 raft_consensus.cc:493] T af57996a8be84d398f0f023e8d08ef77 P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:42.312959 16376 raft_consensus.cc:3060] T af57996a8be84d398f0f023e8d08ef77 P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:42.314370 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5718c623ebee4d2bb770f0f9ddb2f164" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:18:42.302882 16342 ts_tablet_manager.cc:1434] T af57996a8be84d398f0f023e8d08ef77 P 90f3da57796b488c9087a2f523adf045: Time spent starting tablet: real 0.012s	user 0.003s	sys 0.002s
I20260709 13:18:42.316938 16342 tablet_bootstrap.cc:492] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045: Bootstrap starting.
I20260709 13:18:42.318429 16516 leader_election.cc:290] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 1 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:18:42.323535 16347 tablet_bootstrap.cc:492] T 746aabf856ab401c9ef5e859371e7b67 P 1ca2725d290e47d68b1a4b609c9816ac: No bootstrap required, opened a new log
I20260709 13:18:42.324884 16347 ts_tablet_manager.cc:1403] T 746aabf856ab401c9ef5e859371e7b67 P 1ca2725d290e47d68b1a4b609c9816ac: Time spent bootstrapping tablet: real 0.047s	user 0.023s	sys 0.018s
I20260709 13:18:42.324808 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62fe3346cd3c42c4a55e14af9073bca4" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:18:42.325795 16297 raft_consensus.cc:3060] T 62fe3346cd3c42c4a55e14af9073bca4 P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:42.324411 16342 tablet_bootstrap.cc:654] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:42.327543 16347 raft_consensus.cc:359] T 746aabf856ab401c9ef5e859371e7b67 P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:42.328202 16347 raft_consensus.cc:385] T 746aabf856ab401c9ef5e859371e7b67 P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:42.328449 16347 raft_consensus.cc:740] T 746aabf856ab401c9ef5e859371e7b67 P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Initialized, Role: FOLLOWER
I20260709 13:18:42.328433 16296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5718c623ebee4d2bb770f0f9ddb2f164" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:18:42.329262 16296 raft_consensus.cc:2468] T 5718c623ebee4d2bb770f0f9ddb2f164 P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 0.
I20260709 13:18:42.329262 16347 consensus_queue.cc:260] T 746aabf856ab401c9ef5e859371e7b67 P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:42.331524 16347 ts_tablet_manager.cc:1434] T 746aabf856ab401c9ef5e859371e7b67 P 1ca2725d290e47d68b1a4b609c9816ac: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.000s
I20260709 13:18:42.332388 16347 tablet_bootstrap.cc:492] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac: Bootstrap starting.
I20260709 13:18:42.332461 16296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5718c623ebee4d2bb770f0f9ddb2f164" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:18:42.333076 16296 raft_consensus.cc:3060] T 5718c623ebee4d2bb770f0f9ddb2f164 P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:42.334549 16297 raft_consensus.cc:2468] T 62fe3346cd3c42c4a55e14af9073bca4 P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 1.
I20260709 13:18:42.346345 16432 consensus_queue.cc:1048] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:18:42.348053 16463 raft_consensus.cc:493] T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:42.348556 16463 raft_consensus.cc:515] T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:42.350477 16463 leader_election.cc:290] T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:18:42.352193 16151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74ca8140f35c4ae29e8a664f6bdfc829" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:18:42.352818 16151 raft_consensus.cc:2468] T 74ca8140f35c4ae29e8a664f6bdfc829 P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 0.
I20260709 13:18:42.354308 16265 leader_election.cc:304] T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:18:42.355080 16463 raft_consensus.cc:2804] T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:18:42.355698 16463 raft_consensus.cc:493] T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:42.356014 16463 raft_consensus.cc:3060] T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:42.356456 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74ca8140f35c4ae29e8a664f6bdfc829" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
W20260709 13:18:42.361523 16265 leader_election.cc:343] T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:18:42.364238 16463 raft_consensus.cc:515] T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:42.366832 16151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74ca8140f35c4ae29e8a664f6bdfc829" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:18:42.367436 16151 raft_consensus.cc:3060] T 74ca8140f35c4ae29e8a664f6bdfc829 P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:42.369307 16347 tablet_bootstrap.cc:654] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:42.378276 16427 raft_consensus.cc:493] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:42.378880 16427 raft_consensus.cc:515] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:42.382499 16427 leader_election.cc:290] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:18:42.385792 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806d7690c2b644959152074bb2cab0a7" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
W20260709 13:18:42.387614 16265 leader_election.cc:343] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:18:42.398682 16097 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:18:42.403648 16151 raft_consensus.cc:2468] T 74ca8140f35c4ae29e8a664f6bdfc829 P 647a343a542d4a139fe106c19095129b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 1.
I20260709 13:18:42.405802 16463 leader_election.cc:290] T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 1 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:18:42.410880 16265 leader_election.cc:304] T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:18:42.414436 16225 raft_consensus.cc:3060] T 829b4ad8c5a54eababeea1181b960d4b P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:18:42.415251 16115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 72307.1us
I20260709 13:18:42.426280 16347 tablet_bootstrap.cc:492] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac: No bootstrap required, opened a new log
I20260709 13:18:42.426699 16347 ts_tablet_manager.cc:1403] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac: Time spent bootstrapping tablet: real 0.094s	user 0.020s	sys 0.013s
I20260709 13:18:42.426975 16376 raft_consensus.cc:515] T af57996a8be84d398f0f023e8d08ef77 P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:42.421854 16427 raft_consensus.cc:2804] T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:18:42.437521 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74ca8140f35c4ae29e8a664f6bdfc829" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:18:42.438235 16115 leader_election.cc:304] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac
W20260709 13:18:42.438769 16265 leader_election.cc:343] T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:18:42.439802 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "829b4ad8c5a54eababeea1181b960d4b" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:18:42.440121 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af57996a8be84d398f0f023e8d08ef77" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:18:42.440676 16149 raft_consensus.cc:2468] T af57996a8be84d398f0f023e8d08ef77 P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 0.
I20260709 13:18:42.441582 16147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5718c623ebee4d2bb770f0f9ddb2f164" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:18:42.442391 16147 raft_consensus.cc:2393] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 1: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
I20260709 13:18:42.442622 16430 raft_consensus.cc:515] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:42.433938 16376 leader_election.cc:290] T af57996a8be84d398f0f023e8d08ef77 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 1 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:18:42.444290 16347 raft_consensus.cc:359] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:42.448632 16347 raft_consensus.cc:385] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:42.447072 16516 raft_consensus.cc:2804] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:18:42.448962 16347 raft_consensus.cc:740] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Initialized, Role: FOLLOWER
I20260709 13:18:42.448930 16427 raft_consensus.cc:697] T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045 [term 1 LEADER]: Becoming Leader. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:18:42.447801 16226 raft_consensus.cc:2393] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 1: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
I20260709 13:18:42.449915 16347 consensus_queue.cc:260] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:42.450362 16427 consensus_queue.cc:237] T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:42.447376 16340 tablet_bootstrap.cc:492] T ee9ed67006144797b1de6ea8c2013a68 P 647a343a542d4a139fe106c19095129b: No bootstrap required, opened a new log
I20260709 13:18:42.449222 16516 raft_consensus.cc:697] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [term 1 LEADER]: Becoming Leader. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:18:42.463351 16521 consensus_queue.cc:1048] T 829b4ad8c5a54eababeea1181b960d4b P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:18:42.465162 16191 leader_election.cc:304] T 829b4ad8c5a54eababeea1181b960d4b P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:18:42.466446 16376 raft_consensus.cc:2749] T 829b4ad8c5a54eababeea1181b960d4b P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:18:42.470065 16342 tablet_bootstrap.cc:492] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045: No bootstrap required, opened a new log
I20260709 13:18:42.470574 16342 ts_tablet_manager.cc:1403] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045: Time spent bootstrapping tablet: real 0.154s	user 0.009s	sys 0.036s
I20260709 13:18:42.452256 16340 ts_tablet_manager.cc:1403] T ee9ed67006144797b1de6ea8c2013a68 P 647a343a542d4a139fe106c19095129b: Time spent bootstrapping tablet: real 0.223s	user 0.047s	sys 0.097s
I20260709 13:18:42.447836 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af57996a8be84d398f0f023e8d08ef77" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:18:42.455065 16145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af57996a8be84d398f0f023e8d08ef77" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:18:42.456548 16345 raft_consensus.cc:515] T b87147731a68400e8cce9b879daa7c5d P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:42.457297 16432 raft_consensus.cc:493] T 746aabf856ab401c9ef5e859371e7b67 P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:42.457226 16347 ts_tablet_manager.cc:1434] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac: Time spent starting tablet: real 0.030s	user 0.004s	sys 0.004s
I20260709 13:18:42.476101 16347 tablet_bootstrap.cc:492] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac: Bootstrap starting.
I20260709 13:18:42.461826 16296 raft_consensus.cc:2468] T 5718c623ebee4d2bb770f0f9ddb2f164 P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 1.
I20260709 13:18:42.474123 16151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806d7690c2b644959152074bb2cab0a7" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:18:42.478704 16340 raft_consensus.cc:359] T ee9ed67006144797b1de6ea8c2013a68 P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:42.479688 16340 raft_consensus.cc:385] T ee9ed67006144797b1de6ea8c2013a68 P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:42.480046 16340 raft_consensus.cc:740] T ee9ed67006144797b1de6ea8c2013a68 P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Initialized, Role: FOLLOWER
W20260709 13:18:42.480687 16265 leader_election.cc:343] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:18:42.480814 16340 consensus_queue.cc:260] T ee9ed67006144797b1de6ea8c2013a68 P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:42.481417 16265 leader_election.cc:304] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:18:42.485965 16340 ts_tablet_manager.cc:1434] T ee9ed67006144797b1de6ea8c2013a68 P 647a343a542d4a139fe106c19095129b: Time spent starting tablet: real 0.014s	user 0.008s	sys 0.000s
I20260709 13:18:42.462796 16516 consensus_queue.cc:237] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:42.488008 16342 raft_consensus.cc:359] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:42.488899 16342 raft_consensus.cc:385] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:42.489251 16342 raft_consensus.cc:740] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Initialized, Role: FOLLOWER
I20260709 13:18:42.490130 16342 consensus_queue.cc:260] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:42.491096 16345 leader_election.cc:290] T b87147731a68400e8cce9b879daa7c5d P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 1 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:18:42.492340 16342 ts_tablet_manager.cc:1434] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045: Time spent starting tablet: real 0.021s	user 0.008s	sys 0.000s
I20260709 13:18:42.485659 16145 raft_consensus.cc:3060] T af57996a8be84d398f0f023e8d08ef77 P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:42.493268 16342 tablet_bootstrap.cc:492] T 2794af5b99984efebdee564035841cb0 P 90f3da57796b488c9087a2f523adf045: Bootstrap starting.
I20260709 13:18:42.495525 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87147731a68400e8cce9b879daa7c5d" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
W20260709 13:18:42.495848 16191 consensus_peers.cc:597] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:18:42.496191 16150 raft_consensus.cc:3060] T b87147731a68400e8cce9b879daa7c5d P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:42.483826 16300 raft_consensus.cc:3060] T af57996a8be84d398f0f023e8d08ef77 P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:42.499096 16340 tablet_bootstrap.cc:492] T 9c61a39fd4f84424b129c1a6f9ba404d P 647a343a542d4a139fe106c19095129b: Bootstrap starting.
I20260709 13:18:42.502079 16347 tablet_bootstrap.cc:654] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:42.500551 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87147731a68400e8cce9b879daa7c5d" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:18:42.486667 16432 raft_consensus.cc:515] T 746aabf856ab401c9ef5e859371e7b67 P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:42.486016 16565 raft_consensus.cc:2749] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:18:42.502822 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 52154.7us
I20260709 13:18:42.514489 16430 leader_election.cc:290] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 1 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:18:42.515403 16115 leader_election.cc:304] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac
I20260709 13:18:42.514962 16521 consensus_queue.cc:1048] T 829b4ad8c5a54eababeea1181b960d4b P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:18:42.518039 16512 raft_consensus.cc:2804] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:18:42.518633 16512 raft_consensus.cc:697] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b [term 1 LEADER]: Becoming Leader. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:18:42.519731 16432 leader_election.cc:290] T 746aabf856ab401c9ef5e859371e7b67 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:18:42.521934 16512 consensus_queue.cc:237] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:42.506258 16225 raft_consensus.cc:3060] T b87147731a68400e8cce9b879daa7c5d P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:42.575455 16151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5718c623ebee4d2bb770f0f9ddb2f164" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
W20260709 13:18:42.579073 16265 consensus_peers.cc:597] T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:18:42.582176 16150 raft_consensus.cc:2468] T b87147731a68400e8cce9b879daa7c5d P 647a343a542d4a139fe106c19095129b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 1.
I20260709 13:18:42.582715 16047 catalog_manager.cc:5697] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b reported cstate change: term changed from 0 to 1, leader changed from <none> to 647a343a542d4a139fe106c19095129b (127.14.177.129). New cstate: current_term: 1 leader_uuid: "647a343a542d4a139fe106c19095129b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:42.584920 16265 leader_election.cc:304] T b87147731a68400e8cce9b879daa7c5d P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:18:42.585812 16565 raft_consensus.cc:2804] T b87147731a68400e8cce9b879daa7c5d P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:18:42.587535 16565 raft_consensus.cc:697] T b87147731a68400e8cce9b879daa7c5d P 90f3da57796b488c9087a2f523adf045 [term 1 LEADER]: Becoming Leader. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:18:42.590054 16565 consensus_queue.cc:237] T b87147731a68400e8cce9b879daa7c5d P 90f3da57796b488c9087a2f523adf045 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:42.590718 16296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5718c623ebee4d2bb770f0f9ddb2f164" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:18:42.591126 16295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746aabf856ab401c9ef5e859371e7b67" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:18:42.591542 16296 raft_consensus.cc:2393] T 5718c623ebee4d2bb770f0f9ddb2f164 P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 1: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
I20260709 13:18:42.598102 16427 consensus_queue.cc:1048] T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:18:42.604031 16225 raft_consensus.cc:2468] T b87147731a68400e8cce9b879daa7c5d P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 1.
I20260709 13:18:42.611776 16342 tablet_bootstrap.cc:654] T 2794af5b99984efebdee564035841cb0 P 90f3da57796b488c9087a2f523adf045: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:42.617300 16340 tablet_bootstrap.cc:654] T 9c61a39fd4f84424b129c1a6f9ba404d P 647a343a542d4a139fe106c19095129b: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:42.620486 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746aabf856ab401c9ef5e859371e7b67" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:18:42.618808 16300 raft_consensus.cc:2468] T af57996a8be84d398f0f023e8d08ef77 P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 1.
I20260709 13:18:42.628834 16145 raft_consensus.cc:2468] T af57996a8be84d398f0f023e8d08ef77 P 647a343a542d4a139fe106c19095129b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 1.
I20260709 13:18:42.631608 16566 raft_consensus.cc:493] T c0eab25ce4c64e018a041beeda2867db P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:42.632220 16566 raft_consensus.cc:515] T c0eab25ce4c64e018a041beeda2867db P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:42.640261 16047 catalog_manager.cc:5697] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b reported cstate change: term changed from 0 to 1, leader changed from <none> to 647a343a542d4a139fe106c19095129b (127.14.177.129). New cstate: current_term: 1 leader_uuid: "647a343a542d4a139fe106c19095129b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:42.647532 16565 raft_consensus.cc:493] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:42.648062 16565 raft_consensus.cc:515] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:18:42.654932 16190 leader_election.cc:343] T 746aabf856ab401c9ef5e859371e7b67 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:18:42.657522 16190 leader_election.cc:304] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:18:42.658314 16566 leader_election.cc:290] T c0eab25ce4c64e018a041beeda2867db P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:18:42.682539 16376 raft_consensus.cc:2749] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:18:42.678355 16046 catalog_manager.cc:5697] T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045 reported cstate change: term changed from 0 to 1, leader changed from <none> to 90f3da57796b488c9087a2f523adf045 (127.14.177.131). New cstate: current_term: 1 leader_uuid: "90f3da57796b488c9087a2f523adf045" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } health_report { overall_health: HEALTHY } } }
I20260709 13:18:42.683732 16145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0eab25ce4c64e018a041beeda2867db" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:18:42.684864 16190 leader_election.cc:304] T af57996a8be84d398f0f023e8d08ef77 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
W20260709 13:18:42.693974 16191 leader_election.cc:343] T 746aabf856ab401c9ef5e859371e7b67 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:18:42.674036 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a9de347c711477e87c582d003f36925" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:18:42.670341 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a9de347c711477e87c582d003f36925" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:18:42.667914 16565 leader_election.cc:290] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:18:42.694612 16191 leader_election.cc:304] T 746aabf856ab401c9ef5e859371e7b67 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:18:42.696002 16566 raft_consensus.cc:2749] T 746aabf856ab401c9ef5e859371e7b67 P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:18:42.697772 16265 leader_election.cc:343] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:18:42.698803 16224 raft_consensus.cc:3060] T c0eab25ce4c64e018a041beeda2867db P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:18:42.700171 16265 leader_election.cc:343] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:18:42.700686 16265 leader_election.cc:304] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:18:42.701915 16565 raft_consensus.cc:2749] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:18:42.702994 16347 tablet_bootstrap.cc:492] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac: No bootstrap required, opened a new log
I20260709 13:18:42.703522 16347 ts_tablet_manager.cc:1403] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac: Time spent bootstrapping tablet: real 0.228s	user 0.016s	sys 0.002s
I20260709 13:18:42.705948 16340 tablet_bootstrap.cc:492] T 9c61a39fd4f84424b129c1a6f9ba404d P 647a343a542d4a139fe106c19095129b: No bootstrap required, opened a new log
I20260709 13:18:42.706399 16340 ts_tablet_manager.cc:1403] T 9c61a39fd4f84424b129c1a6f9ba404d P 647a343a542d4a139fe106c19095129b: Time spent bootstrapping tablet: real 0.208s	user 0.009s	sys 0.004s
I20260709 13:18:42.706601 16347 raft_consensus.cc:359] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:42.707414 16347 raft_consensus.cc:385] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:42.707710 16347 raft_consensus.cc:740] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Initialized, Role: FOLLOWER
I20260709 13:18:42.708467 16347 consensus_queue.cc:260] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:42.709829 16406 raft_consensus.cc:2804] T af57996a8be84d398f0f023e8d08ef77 P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:18:42.710307 16406 raft_consensus.cc:697] T af57996a8be84d398f0f023e8d08ef77 P 1ca2725d290e47d68b1a4b609c9816ac [term 1 LEADER]: Becoming Leader. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:18:42.711191 16406 consensus_queue.cc:237] T af57996a8be84d398f0f023e8d08ef77 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:42.712064 16347 ts_tablet_manager.cc:1434] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.002s
I20260709 13:18:42.713043 16347 tablet_bootstrap.cc:492] T ee9ed67006144797b1de6ea8c2013a68 P 1ca2725d290e47d68b1a4b609c9816ac: Bootstrap starting.
I20260709 13:18:42.716354 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0eab25ce4c64e018a041beeda2867db" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:18:42.720568 16516 consensus_queue.cc:1048] T c0eab25ce4c64e018a041beeda2867db P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:18:42.715714 16340 raft_consensus.cc:359] T 9c61a39fd4f84424b129c1a6f9ba404d P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:42.722936 16340 raft_consensus.cc:385] T 9c61a39fd4f84424b129c1a6f9ba404d P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:42.723198 16342 tablet_bootstrap.cc:492] T 2794af5b99984efebdee564035841cb0 P 90f3da57796b488c9087a2f523adf045: No bootstrap required, opened a new log
I20260709 13:18:42.723325 16340 raft_consensus.cc:740] T 9c61a39fd4f84424b129c1a6f9ba404d P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Initialized, Role: FOLLOWER
I20260709 13:18:42.723690 16300 raft_consensus.cc:2393] T c0eab25ce4c64e018a041beeda2867db P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 1: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
I20260709 13:18:42.724762 16347 tablet_bootstrap.cc:654] T ee9ed67006144797b1de6ea8c2013a68 P 1ca2725d290e47d68b1a4b609c9816ac: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:42.723683 16342 ts_tablet_manager.cc:1403] T 2794af5b99984efebdee564035841cb0 P 90f3da57796b488c9087a2f523adf045: Time spent bootstrapping tablet: real 0.231s	user 0.025s	sys 0.051s
I20260709 13:18:42.724929 16190 leader_election.cc:304] T c0eab25ce4c64e018a041beeda2867db P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:18:42.724115 16340 consensus_queue.cc:260] T 9c61a39fd4f84424b129c1a6f9ba404d P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:42.732856 16340 ts_tablet_manager.cc:1434] T 9c61a39fd4f84424b129c1a6f9ba404d P 647a343a542d4a139fe106c19095129b: Time spent starting tablet: real 0.026s	user 0.009s	sys 0.000s
I20260709 13:18:42.733387 16342 raft_consensus.cc:359] T 2794af5b99984efebdee564035841cb0 P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:42.735057 16340 tablet_bootstrap.cc:492] T 746aabf856ab401c9ef5e859371e7b67 P 647a343a542d4a139fe106c19095129b: Bootstrap starting.
I20260709 13:18:42.735710 16342 raft_consensus.cc:385] T 2794af5b99984efebdee564035841cb0 P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:42.736127 16342 raft_consensus.cc:740] T 2794af5b99984efebdee564035841cb0 P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Initialized, Role: FOLLOWER
I20260709 13:18:42.731546 16566 raft_consensus.cc:2749] T c0eab25ce4c64e018a041beeda2867db P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:18:42.737088 16342 consensus_queue.cc:260] T 2794af5b99984efebdee564035841cb0 P 90f3da57796b488c9087a2f523adf045 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:42.742419 16342 ts_tablet_manager.cc:1434] T 2794af5b99984efebdee564035841cb0 P 90f3da57796b488c9087a2f523adf045: Time spent starting tablet: real 0.016s	user 0.005s	sys 0.001s
I20260709 13:18:42.744982 16342 tablet_bootstrap.cc:492] T ee9ed67006144797b1de6ea8c2013a68 P 90f3da57796b488c9087a2f523adf045: Bootstrap starting.
I20260709 13:18:42.749843 16046 catalog_manager.cc:5697] T af57996a8be84d398f0f023e8d08ef77 P 1ca2725d290e47d68b1a4b609c9816ac reported cstate change: term changed from 0 to 1, leader changed from <none> to 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130). New cstate: current_term: 1 leader_uuid: "1ca2725d290e47d68b1a4b609c9816ac" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:42.764410 16512 consensus_queue.cc:1048] T c0eab25ce4c64e018a041beeda2867db P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:18:42.766983 16347 tablet_bootstrap.cc:492] T ee9ed67006144797b1de6ea8c2013a68 P 1ca2725d290e47d68b1a4b609c9816ac: No bootstrap required, opened a new log
I20260709 13:18:42.767477 16347 ts_tablet_manager.cc:1403] T ee9ed67006144797b1de6ea8c2013a68 P 1ca2725d290e47d68b1a4b609c9816ac: Time spent bootstrapping tablet: real 0.055s	user 0.008s	sys 0.004s
I20260709 13:18:42.767522 16340 tablet_bootstrap.cc:654] T 746aabf856ab401c9ef5e859371e7b67 P 647a343a542d4a139fe106c19095129b: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:42.771791 16342 tablet_bootstrap.cc:654] T ee9ed67006144797b1de6ea8c2013a68 P 90f3da57796b488c9087a2f523adf045: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:42.772511 16347 raft_consensus.cc:359] T ee9ed67006144797b1de6ea8c2013a68 P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:42.773238 16347 raft_consensus.cc:385] T ee9ed67006144797b1de6ea8c2013a68 P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:42.773504 16347 raft_consensus.cc:740] T ee9ed67006144797b1de6ea8c2013a68 P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Initialized, Role: FOLLOWER
I20260709 13:18:42.774170 16347 consensus_queue.cc:260] T ee9ed67006144797b1de6ea8c2013a68 P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:42.780604 16347 ts_tablet_manager.cc:1434] T ee9ed67006144797b1de6ea8c2013a68 P 1ca2725d290e47d68b1a4b609c9816ac: Time spent starting tablet: real 0.013s	user 0.006s	sys 0.000s
I20260709 13:18:42.782387 16347 tablet_bootstrap.cc:492] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac: Bootstrap starting.
I20260709 13:18:42.792611 16048 catalog_manager.cc:5697] T b87147731a68400e8cce9b879daa7c5d P 90f3da57796b488c9087a2f523adf045 reported cstate change: term changed from 0 to 1, leader changed from <none> to 90f3da57796b488c9087a2f523adf045 (127.14.177.131). New cstate: current_term: 1 leader_uuid: "90f3da57796b488c9087a2f523adf045" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } health_report { overall_health: HEALTHY } } }
I20260709 13:18:42.816009 16340 tablet_bootstrap.cc:492] T 746aabf856ab401c9ef5e859371e7b67 P 647a343a542d4a139fe106c19095129b: No bootstrap required, opened a new log
I20260709 13:18:42.816465 16340 ts_tablet_manager.cc:1403] T 746aabf856ab401c9ef5e859371e7b67 P 647a343a542d4a139fe106c19095129b: Time spent bootstrapping tablet: real 0.082s	user 0.028s	sys 0.011s
I20260709 13:18:42.826975 16340 raft_consensus.cc:359] T 746aabf856ab401c9ef5e859371e7b67 P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:42.827723 16340 raft_consensus.cc:385] T 746aabf856ab401c9ef5e859371e7b67 P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:42.828027 16340 raft_consensus.cc:740] T 746aabf856ab401c9ef5e859371e7b67 P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Initialized, Role: FOLLOWER
I20260709 13:18:42.828733 16340 consensus_queue.cc:260] T 746aabf856ab401c9ef5e859371e7b67 P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:42.830871 16340 ts_tablet_manager.cc:1434] T 746aabf856ab401c9ef5e859371e7b67 P 647a343a542d4a139fe106c19095129b: Time spent starting tablet: real 0.014s	user 0.004s	sys 0.000s
I20260709 13:18:42.831913 16340 tablet_bootstrap.cc:492] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b: Bootstrap starting.
I20260709 13:18:42.839591 16347 tablet_bootstrap.cc:654] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:42.840937 16340 tablet_bootstrap.cc:654] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:42.846519 16516 raft_consensus.cc:493] T 9c61a39fd4f84424b129c1a6f9ba404d P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:42.847272 16516 raft_consensus.cc:515] T 9c61a39fd4f84424b129c1a6f9ba404d P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:42.853089 16516 leader_election.cc:290] T 9c61a39fd4f84424b129c1a6f9ba404d P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:18:42.853739 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c61a39fd4f84424b129c1a6f9ba404d" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:18:42.854872 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c61a39fd4f84424b129c1a6f9ba404d" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:18:42.855618 16342 tablet_bootstrap.cc:492] T ee9ed67006144797b1de6ea8c2013a68 P 90f3da57796b488c9087a2f523adf045: No bootstrap required, opened a new log
I20260709 13:18:42.856015 16342 ts_tablet_manager.cc:1403] T ee9ed67006144797b1de6ea8c2013a68 P 90f3da57796b488c9087a2f523adf045: Time spent bootstrapping tablet: real 0.111s	user 0.030s	sys 0.035s
I20260709 13:18:42.857818 16342 raft_consensus.cc:359] T ee9ed67006144797b1de6ea8c2013a68 P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:42.858300 16342 raft_consensus.cc:385] T ee9ed67006144797b1de6ea8c2013a68 P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:42.857975 16115 leader_election.cc:304] T 9c61a39fd4f84424b129c1a6f9ba404d P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:18:42.858537 16342 raft_consensus.cc:740] T ee9ed67006144797b1de6ea8c2013a68 P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Initialized, Role: FOLLOWER
I20260709 13:18:42.859083 16342 consensus_queue.cc:260] T ee9ed67006144797b1de6ea8c2013a68 P 90f3da57796b488c9087a2f523adf045 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:42.860092 16565 consensus_queue.cc:1048] T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:18:42.861654 16512 raft_consensus.cc:2749] T 9c61a39fd4f84424b129c1a6f9ba404d P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:18:42.862891 16342 ts_tablet_manager.cc:1434] T ee9ed67006144797b1de6ea8c2013a68 P 90f3da57796b488c9087a2f523adf045: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:18:42.863981 16342 tablet_bootstrap.cc:492] T c4f65cdf00b748bb963ee6e81ef43b79 P 90f3da57796b488c9087a2f523adf045: Bootstrap starting.
I20260709 13:18:42.867373 16340 tablet_bootstrap.cc:492] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b: No bootstrap required, opened a new log
I20260709 13:18:42.867805 16340 ts_tablet_manager.cc:1403] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b: Time spent bootstrapping tablet: real 0.036s	user 0.012s	sys 0.014s
I20260709 13:18:42.870671 16340 raft_consensus.cc:359] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:42.871584 16342 tablet_bootstrap.cc:654] T c4f65cdf00b748bb963ee6e81ef43b79 P 90f3da57796b488c9087a2f523adf045: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:42.872625 16340 raft_consensus.cc:385] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:42.872964 16340 raft_consensus.cc:740] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Initialized, Role: FOLLOWER
I20260709 13:18:42.873571 16340 consensus_queue.cc:260] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:42.875612 16340 ts_tablet_manager.cc:1434] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 13:18:42.876459 16340 tablet_bootstrap.cc:492] T 66ae176961b74813a58bc15c9c4c6f2a P 647a343a542d4a139fe106c19095129b: Bootstrap starting.
I20260709 13:18:42.880493 16345 raft_consensus.cc:493] T 2794af5b99984efebdee564035841cb0 P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:42.880966 16345 raft_consensus.cc:515] T 2794af5b99984efebdee564035841cb0 P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:42.884243 16340 tablet_bootstrap.cc:654] T 66ae176961b74813a58bc15c9c4c6f2a P 647a343a542d4a139fe106c19095129b: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:42.883020 16345 leader_election.cc:290] T 2794af5b99984efebdee564035841cb0 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:18:42.888249 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2794af5b99984efebdee564035841cb0" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:18:42.888659 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2794af5b99984efebdee564035841cb0" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:18:42.892010 16376 raft_consensus.cc:493] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:42.892457 16376 raft_consensus.cc:515] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:42.894006 16342 tablet_bootstrap.cc:492] T c4f65cdf00b748bb963ee6e81ef43b79 P 90f3da57796b488c9087a2f523adf045: No bootstrap required, opened a new log
I20260709 13:18:42.894505 16342 ts_tablet_manager.cc:1403] T c4f65cdf00b748bb963ee6e81ef43b79 P 90f3da57796b488c9087a2f523adf045: Time spent bootstrapping tablet: real 0.031s	user 0.018s	sys 0.001s
W20260709 13:18:42.894709 16265 leader_election.cc:343] T 2794af5b99984efebdee564035841cb0 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:18:42.895283 16376 leader_election.cc:290] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:18:42.896704 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74ca8140f35c4ae29e8a664f6bdfc829" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:18:42.897212 16342 raft_consensus.cc:359] T c4f65cdf00b748bb963ee6e81ef43b79 P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:42.898051 16342 raft_consensus.cc:385] T c4f65cdf00b748bb963ee6e81ef43b79 P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:42.898368 16342 raft_consensus.cc:740] T c4f65cdf00b748bb963ee6e81ef43b79 P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Initialized, Role: FOLLOWER
I20260709 13:18:42.899801 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74ca8140f35c4ae29e8a664f6bdfc829" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:18:42.901759 16190 leader_election.cc:304] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:18:42.902467 16340 tablet_bootstrap.cc:492] T 66ae176961b74813a58bc15c9c4c6f2a P 647a343a542d4a139fe106c19095129b: No bootstrap required, opened a new log
I20260709 13:18:42.902580 16376 raft_consensus.cc:2749] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:18:42.902976 16340 ts_tablet_manager.cc:1403] T 66ae176961b74813a58bc15c9c4c6f2a P 647a343a542d4a139fe106c19095129b: Time spent bootstrapping tablet: real 0.027s	user 0.006s	sys 0.009s
I20260709 13:18:42.904781 16342 consensus_queue.cc:260] T c4f65cdf00b748bb963ee6e81ef43b79 P 90f3da57796b488c9087a2f523adf045 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:42.905735 16516 raft_consensus.cc:493] T ee9ed67006144797b1de6ea8c2013a68 P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:42.906402 16516 raft_consensus.cc:515] T ee9ed67006144797b1de6ea8c2013a68 P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:42.907503 16342 ts_tablet_manager.cc:1434] T c4f65cdf00b748bb963ee6e81ef43b79 P 90f3da57796b488c9087a2f523adf045: Time spent starting tablet: real 0.013s	user 0.002s	sys 0.004s
I20260709 13:18:42.907815 16340 raft_consensus.cc:359] T 66ae176961b74813a58bc15c9c4c6f2a P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:42.908658 16342 tablet_bootstrap.cc:492] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045: Bootstrap starting.
I20260709 13:18:42.909448 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee9ed67006144797b1de6ea8c2013a68" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:18:42.910120 16225 raft_consensus.cc:2468] T ee9ed67006144797b1de6ea8c2013a68 P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 0.
I20260709 13:18:42.912499 16116 leader_election.cc:304] T ee9ed67006144797b1de6ea8c2013a68 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
I20260709 13:18:42.913909 16511 raft_consensus.cc:2804] T ee9ed67006144797b1de6ea8c2013a68 P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:18:42.914304 16511 raft_consensus.cc:493] T ee9ed67006144797b1de6ea8c2013a68 P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:42.914685 16511 raft_consensus.cc:3060] T ee9ed67006144797b1de6ea8c2013a68 P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:18:42.910614 16265 leader_election.cc:343] T 2794af5b99984efebdee564035841cb0 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:18:42.915381 16265 leader_election.cc:304] T 2794af5b99984efebdee564035841cb0 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:18:42.921419 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee9ed67006144797b1de6ea8c2013a68" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:18:42.922113 16300 raft_consensus.cc:2468] T ee9ed67006144797b1de6ea8c2013a68 P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 0.
I20260709 13:18:42.923681 16463 raft_consensus.cc:2749] T 2794af5b99984efebdee564035841cb0 P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:18:42.928114 16516 leader_election.cc:290] T ee9ed67006144797b1de6ea8c2013a68 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:18:42.908573 16340 raft_consensus.cc:385] T 66ae176961b74813a58bc15c9c4c6f2a P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:42.931942 16340 raft_consensus.cc:740] T 66ae176961b74813a58bc15c9c4c6f2a P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Initialized, Role: FOLLOWER
I20260709 13:18:42.932570 16511 raft_consensus.cc:515] T ee9ed67006144797b1de6ea8c2013a68 P 647a343a542d4a139fe106c19095129b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:42.933233 16225 raft_consensus.cc:3060] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:42.934921 16340 consensus_queue.cc:260] T 66ae176961b74813a58bc15c9c4c6f2a P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:42.935813 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee9ed67006144797b1de6ea8c2013a68" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:18:42.936385 16224 raft_consensus.cc:3060] T ee9ed67006144797b1de6ea8c2013a68 P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:42.939414 16340 ts_tablet_manager.cc:1434] T 66ae176961b74813a58bc15c9c4c6f2a P 647a343a542d4a139fe106c19095129b: Time spent starting tablet: real 0.036s	user 0.008s	sys 0.000s
I20260709 13:18:42.941473 16347 tablet_bootstrap.cc:492] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac: No bootstrap required, opened a new log
I20260709 13:18:42.941934 16347 ts_tablet_manager.cc:1403] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac: Time spent bootstrapping tablet: real 0.160s	user 0.031s	sys 0.031s
I20260709 13:18:42.942730 16342 tablet_bootstrap.cc:654] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:42.944541 16347 raft_consensus.cc:359] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:42.945230 16347 raft_consensus.cc:385] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:42.945487 16347 raft_consensus.cc:740] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Initialized, Role: FOLLOWER
I20260709 13:18:42.946236 16347 consensus_queue.cc:260] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:42.948210 16347 ts_tablet_manager.cc:1434] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac: Time spent starting tablet: real 0.006s	user 0.007s	sys 0.000s
I20260709 13:18:42.948784 16511 leader_election.cc:290] T ee9ed67006144797b1de6ea8c2013a68 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 1 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:18:42.950212 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee9ed67006144797b1de6ea8c2013a68" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:18:42.950726 16300 raft_consensus.cc:3060] T ee9ed67006144797b1de6ea8c2013a68 P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:42.951431 16347 tablet_bootstrap.cc:492] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac: Bootstrap starting.
I20260709 13:18:42.956993 16342 tablet_bootstrap.cc:492] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045: No bootstrap required, opened a new log
I20260709 13:18:42.957558 16347 tablet_bootstrap.cc:654] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:42.956926 16300 raft_consensus.cc:2468] T ee9ed67006144797b1de6ea8c2013a68 P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 1.
I20260709 13:18:42.957641 16342 ts_tablet_manager.cc:1403] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045: Time spent bootstrapping tablet: real 0.049s	user 0.017s	sys 0.002s
I20260709 13:18:42.958937 16115 leader_election.cc:304] T ee9ed67006144797b1de6ea8c2013a68 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:18:42.959793 16511 raft_consensus.cc:2804] T ee9ed67006144797b1de6ea8c2013a68 P 647a343a542d4a139fe106c19095129b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:18:42.960265 16511 raft_consensus.cc:697] T ee9ed67006144797b1de6ea8c2013a68 P 647a343a542d4a139fe106c19095129b [term 1 LEADER]: Becoming Leader. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:18:42.961074 16511 consensus_queue.cc:237] T ee9ed67006144797b1de6ea8c2013a68 P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:42.961963 16224 raft_consensus.cc:2468] T ee9ed67006144797b1de6ea8c2013a68 P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 1.
I20260709 13:18:42.961494 16342 raft_consensus.cc:359] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:42.962675 16342 raft_consensus.cc:385] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:42.963109 16342 raft_consensus.cc:740] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Initialized, Role: FOLLOWER
I20260709 13:18:42.964188 16342 consensus_queue.cc:260] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:42.968027 16340 tablet_bootstrap.cc:492] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b: Bootstrap starting.
I20260709 13:18:42.972823 16342 ts_tablet_manager.cc:1434] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045: Time spent starting tablet: real 0.014s	user 0.007s	sys 0.000s
I20260709 13:18:42.973826 16342 tablet_bootstrap.cc:492] T 746aabf856ab401c9ef5e859371e7b67 P 90f3da57796b488c9087a2f523adf045: Bootstrap starting.
I20260709 13:18:42.980532 16342 tablet_bootstrap.cc:654] T 746aabf856ab401c9ef5e859371e7b67 P 90f3da57796b488c9087a2f523adf045: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:42.982369 16463 consensus_queue.cc:1048] T b87147731a68400e8cce9b879daa7c5d P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:18:42.990378 16463 consensus_queue.cc:1048] T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:18:42.975036 16340 tablet_bootstrap.cc:654] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:43.000885 16406 raft_consensus.cc:493] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:43.001341 16406 raft_consensus.cc:515] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:43.003520 16406 leader_election.cc:290] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:18:43.005430 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62fe3346cd3c42c4a55e14af9073bca4" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:18:43.006188 16300 raft_consensus.cc:2393] T 62fe3346cd3c42c4a55e14af9073bca4 P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 1: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
I20260709 13:18:43.009956 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62fe3346cd3c42c4a55e14af9073bca4" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:18:43.019068 16048 catalog_manager.cc:5697] T ee9ed67006144797b1de6ea8c2013a68 P 647a343a542d4a139fe106c19095129b reported cstate change: term changed from 0 to 1, leader changed from <none> to 647a343a542d4a139fe106c19095129b (127.14.177.129). New cstate: current_term: 1 leader_uuid: "647a343a542d4a139fe106c19095129b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:43.020332 16191 leader_election.cc:304] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:18:43.027943 16406 raft_consensus.cc:3060] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:43.026979 16565 consensus_queue.cc:1048] T b87147731a68400e8cce9b879daa7c5d P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.006s
I20260709 13:18:43.035573 16342 tablet_bootstrap.cc:492] T 746aabf856ab401c9ef5e859371e7b67 P 90f3da57796b488c9087a2f523adf045: No bootstrap required, opened a new log
I20260709 13:18:43.036619 16342 ts_tablet_manager.cc:1403] T 746aabf856ab401c9ef5e859371e7b67 P 90f3da57796b488c9087a2f523adf045: Time spent bootstrapping tablet: real 0.063s	user 0.019s	sys 0.004s
I20260709 13:18:43.037431 16347 tablet_bootstrap.cc:492] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac: No bootstrap required, opened a new log
I20260709 13:18:43.030511 16512 consensus_queue.cc:1048] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.006s
I20260709 13:18:43.035475 16516 raft_consensus.cc:493] T 746aabf856ab401c9ef5e859371e7b67 P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:43.039239 16516 raft_consensus.cc:515] T 746aabf856ab401c9ef5e859371e7b67 P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:43.041149 16516 leader_election.cc:290] T 746aabf856ab401c9ef5e859371e7b67 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:18:43.043744 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746aabf856ab401c9ef5e859371e7b67" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:18:43.045635 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746aabf856ab401c9ef5e859371e7b67" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:18:43.046214 16226 raft_consensus.cc:2468] T 746aabf856ab401c9ef5e859371e7b67 P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 0.
I20260709 13:18:43.048489 16116 leader_election.cc:304] T 746aabf856ab401c9ef5e859371e7b67 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
I20260709 13:18:43.049284 16520 consensus_queue.cc:1048] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:18:43.050747 16520 raft_consensus.cc:2804] T 746aabf856ab401c9ef5e859371e7b67 P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:18:43.051075 16520 raft_consensus.cc:493] T 746aabf856ab401c9ef5e859371e7b67 P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:43.051445 16520 raft_consensus.cc:3060] T 746aabf856ab401c9ef5e859371e7b67 P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:43.052006 16406 raft_consensus.cc:2749] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:18:43.055289 16342 raft_consensus.cc:359] T 746aabf856ab401c9ef5e859371e7b67 P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:43.055985 16342 raft_consensus.cc:385] T 746aabf856ab401c9ef5e859371e7b67 P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:43.056236 16342 raft_consensus.cc:740] T 746aabf856ab401c9ef5e859371e7b67 P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Initialized, Role: FOLLOWER
I20260709 13:18:43.058751 16340 tablet_bootstrap.cc:492] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b: No bootstrap required, opened a new log
I20260709 13:18:43.059214 16340 ts_tablet_manager.cc:1403] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b: Time spent bootstrapping tablet: real 0.091s	user 0.007s	sys 0.005s
I20260709 13:18:43.062351 16340 raft_consensus.cc:359] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:43.063076 16340 raft_consensus.cc:385] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:43.063514 16340 raft_consensus.cc:740] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Initialized, Role: FOLLOWER
I20260709 13:18:43.064198 16340 consensus_queue.cc:260] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:43.065538 16342 consensus_queue.cc:260] T 746aabf856ab401c9ef5e859371e7b67 P 90f3da57796b488c9087a2f523adf045 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:43.066516 16300 raft_consensus.cc:2468] T 746aabf856ab401c9ef5e859371e7b67 P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 0.
I20260709 13:18:43.069610 16342 ts_tablet_manager.cc:1434] T 746aabf856ab401c9ef5e859371e7b67 P 90f3da57796b488c9087a2f523adf045: Time spent starting tablet: real 0.033s	user 0.006s	sys 0.000s
I20260709 13:18:43.070495 16342 tablet_bootstrap.cc:492] T 8158bbd17b994d2eb96f560bd0f1b88d P 90f3da57796b488c9087a2f523adf045: Bootstrap starting.
I20260709 13:18:43.072875 16516 consensus_queue.cc:1048] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:18:43.076489 16340 ts_tablet_manager.cc:1434] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b: Time spent starting tablet: real 0.017s	user 0.006s	sys 0.000s
I20260709 13:18:43.077409 16340 tablet_bootstrap.cc:492] T 2794af5b99984efebdee564035841cb0 P 647a343a542d4a139fe106c19095129b: Bootstrap starting.
I20260709 13:18:43.088245 16342 tablet_bootstrap.cc:654] T 8158bbd17b994d2eb96f560bd0f1b88d P 90f3da57796b488c9087a2f523adf045: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:43.088158 16340 tablet_bootstrap.cc:654] T 2794af5b99984efebdee564035841cb0 P 647a343a542d4a139fe106c19095129b: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:43.095053 16520 raft_consensus.cc:515] T 746aabf856ab401c9ef5e859371e7b67 P 647a343a542d4a139fe106c19095129b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:43.096321 16347 ts_tablet_manager.cc:1403] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac: Time spent bootstrapping tablet: real 0.145s	user 0.030s	sys 0.009s
I20260709 13:18:43.104835 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746aabf856ab401c9ef5e859371e7b67" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:18:43.105474 16224 raft_consensus.cc:3060] T 746aabf856ab401c9ef5e859371e7b67 P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:43.105914 16296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746aabf856ab401c9ef5e859371e7b67" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:18:43.106511 16296 raft_consensus.cc:3060] T 746aabf856ab401c9ef5e859371e7b67 P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:43.112178 16224 raft_consensus.cc:2468] T 746aabf856ab401c9ef5e859371e7b67 P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 1.
I20260709 13:18:43.113652 16116 leader_election.cc:304] T 746aabf856ab401c9ef5e859371e7b67 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
I20260709 13:18:43.114471 16512 raft_consensus.cc:2804] T 746aabf856ab401c9ef5e859371e7b67 P 647a343a542d4a139fe106c19095129b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:18:43.114984 16512 raft_consensus.cc:697] T 746aabf856ab401c9ef5e859371e7b67 P 647a343a542d4a139fe106c19095129b [term 1 LEADER]: Becoming Leader. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:18:43.115898 16512 consensus_queue.cc:237] T 746aabf856ab401c9ef5e859371e7b67 P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:43.119464 16296 raft_consensus.cc:2468] T 746aabf856ab401c9ef5e859371e7b67 P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 1.
I20260709 13:18:43.121606 16520 leader_election.cc:290] T 746aabf856ab401c9ef5e859371e7b67 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 1 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:18:43.124392 16342 tablet_bootstrap.cc:492] T 8158bbd17b994d2eb96f560bd0f1b88d P 90f3da57796b488c9087a2f523adf045: No bootstrap required, opened a new log
I20260709 13:18:43.124831 16342 ts_tablet_manager.cc:1403] T 8158bbd17b994d2eb96f560bd0f1b88d P 90f3da57796b488c9087a2f523adf045: Time spent bootstrapping tablet: real 0.055s	user 0.007s	sys 0.003s
I20260709 13:18:43.098951 16347 raft_consensus.cc:359] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:43.125399 16347 raft_consensus.cc:385] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:43.127758 16150 raft_consensus.cc:3060] T 9c61a39fd4f84424b129c1a6f9ba404d P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:43.128803 16342 raft_consensus.cc:359] T 8158bbd17b994d2eb96f560bd0f1b88d P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:43.129520 16342 raft_consensus.cc:385] T 8158bbd17b994d2eb96f560bd0f1b88d P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:43.129776 16342 raft_consensus.cc:740] T 8158bbd17b994d2eb96f560bd0f1b88d P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Initialized, Role: FOLLOWER
I20260709 13:18:43.130694 16342 consensus_queue.cc:260] T 8158bbd17b994d2eb96f560bd0f1b88d P 90f3da57796b488c9087a2f523adf045 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:43.132673 16342 ts_tablet_manager.cc:1434] T 8158bbd17b994d2eb96f560bd0f1b88d P 90f3da57796b488c9087a2f523adf045: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 13:18:43.133527 16342 tablet_bootstrap.cc:492] T 31969392ccdb43b491911a319e8bc7e1 P 90f3da57796b488c9087a2f523adf045: Bootstrap starting.
I20260709 13:18:43.139426 16342 tablet_bootstrap.cc:654] T 31969392ccdb43b491911a319e8bc7e1 P 90f3da57796b488c9087a2f523adf045: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:43.127002 16347 raft_consensus.cc:740] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Initialized, Role: FOLLOWER
I20260709 13:18:43.140975 16347 consensus_queue.cc:260] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:43.143625 16347 ts_tablet_manager.cc:1434] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac: Time spent starting tablet: real 0.047s	user 0.007s	sys 0.000s
I20260709 13:18:43.149695 16347 tablet_bootstrap.cc:492] T 0a9de347c711477e87c582d003f36925 P 1ca2725d290e47d68b1a4b609c9816ac: Bootstrap starting.
I20260709 13:18:43.157956 16347 tablet_bootstrap.cc:654] T 0a9de347c711477e87c582d003f36925 P 1ca2725d290e47d68b1a4b609c9816ac: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:43.158335 16342 tablet_bootstrap.cc:492] T 31969392ccdb43b491911a319e8bc7e1 P 90f3da57796b488c9087a2f523adf045: No bootstrap required, opened a new log
I20260709 13:18:43.162967 16406 raft_consensus.cc:493] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:43.165725 16342 ts_tablet_manager.cc:1403] T 31969392ccdb43b491911a319e8bc7e1 P 90f3da57796b488c9087a2f523adf045: Time spent bootstrapping tablet: real 0.032s	user 0.004s	sys 0.012s
I20260709 13:18:43.165706 16406 raft_consensus.cc:515] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:43.167943 16406 leader_election.cc:290] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:18:43.169086 16342 raft_consensus.cc:359] T 31969392ccdb43b491911a319e8bc7e1 P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:43.169752 16342 raft_consensus.cc:385] T 31969392ccdb43b491911a319e8bc7e1 P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:43.169785 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4f65cdf00b748bb963ee6e81ef43b79" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:18:43.170083 16342 raft_consensus.cc:740] T 31969392ccdb43b491911a319e8bc7e1 P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Initialized, Role: FOLLOWER
I20260709 13:18:43.170470 16300 raft_consensus.cc:2468] T c4f65cdf00b748bb963ee6e81ef43b79 P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 0.
I20260709 13:18:43.170495 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4f65cdf00b748bb963ee6e81ef43b79" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:18:43.170876 16342 consensus_queue.cc:260] T 31969392ccdb43b491911a319e8bc7e1 P 90f3da57796b488c9087a2f523adf045 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:43.171712 16190 leader_election.cc:304] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
W20260709 13:18:43.172715 16191 leader_election.cc:343] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:18:43.173003 16342 ts_tablet_manager.cc:1434] T 31969392ccdb43b491911a319e8bc7e1 P 90f3da57796b488c9087a2f523adf045: Time spent starting tablet: real 0.007s	user 0.002s	sys 0.003s
I20260709 13:18:43.173519 16406 raft_consensus.cc:2804] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:18:43.173895 16342 tablet_bootstrap.cc:492] T 08b7d1ab519044eda2d5e04a07242f25 P 90f3da57796b488c9087a2f523adf045: Bootstrap starting.
I20260709 13:18:43.173902 16406 raft_consensus.cc:493] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:43.174475 16406 raft_consensus.cc:3060] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:43.179633 16511 consensus_queue.cc:1048] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:18:43.180208 16342 tablet_bootstrap.cc:654] T 08b7d1ab519044eda2d5e04a07242f25 P 90f3da57796b488c9087a2f523adf045: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:43.190707 16376 consensus_queue.cc:1048] T af57996a8be84d398f0f023e8d08ef77 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:18:43.192852 16376 raft_consensus.cc:493] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:43.193342 16376 raft_consensus.cc:515] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:43.197955 16048 catalog_manager.cc:5697] T 746aabf856ab401c9ef5e859371e7b67 P 647a343a542d4a139fe106c19095129b reported cstate change: term changed from 0 to 1, leader changed from <none> to 647a343a542d4a139fe106c19095129b (127.14.177.129). New cstate: current_term: 1 leader_uuid: "647a343a542d4a139fe106c19095129b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:43.198673 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8158bbd17b994d2eb96f560bd0f1b88d" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
W20260709 13:18:43.201067 16191 leader_election.cc:343] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:18:43.202754 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8158bbd17b994d2eb96f560bd0f1b88d" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:18:43.206108 16299 raft_consensus.cc:2468] T 8158bbd17b994d2eb96f560bd0f1b88d P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 0.
I20260709 13:18:43.206794 16342 tablet_bootstrap.cc:492] T 08b7d1ab519044eda2d5e04a07242f25 P 90f3da57796b488c9087a2f523adf045: No bootstrap required, opened a new log
I20260709 13:18:43.207342 16342 ts_tablet_manager.cc:1403] T 08b7d1ab519044eda2d5e04a07242f25 P 90f3da57796b488c9087a2f523adf045: Time spent bootstrapping tablet: real 0.034s	user 0.009s	sys 0.010s
I20260709 13:18:43.207758 16190 leader_election.cc:304] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 647a343a542d4a139fe106c19095129b
I20260709 13:18:43.208550 16376 leader_election.cc:290] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:18:43.208581 16566 raft_consensus.cc:2804] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:18:43.208964 16347 tablet_bootstrap.cc:492] T 0a9de347c711477e87c582d003f36925 P 1ca2725d290e47d68b1a4b609c9816ac: No bootstrap required, opened a new log
I20260709 13:18:43.209111 16566 raft_consensus.cc:493] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:43.209501 16566 raft_consensus.cc:3060] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:43.209407 16347 ts_tablet_manager.cc:1403] T 0a9de347c711477e87c582d003f36925 P 1ca2725d290e47d68b1a4b609c9816ac: Time spent bootstrapping tablet: real 0.060s	user 0.009s	sys 0.011s
I20260709 13:18:43.215817 16516 raft_consensus.cc:493] T 66ae176961b74813a58bc15c9c4c6f2a P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:43.216351 16516 raft_consensus.cc:515] T 66ae176961b74813a58bc15c9c4c6f2a P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:43.218331 16516 leader_election.cc:290] T 66ae176961b74813a58bc15c9c4c6f2a P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:18:43.218252 16347 raft_consensus.cc:359] T 0a9de347c711477e87c582d003f36925 P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:43.218991 16347 raft_consensus.cc:385] T 0a9de347c711477e87c582d003f36925 P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:43.219282 16347 raft_consensus.cc:740] T 0a9de347c711477e87c582d003f36925 P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Initialized, Role: FOLLOWER
I20260709 13:18:43.219937 16406 raft_consensus.cc:515] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:43.220017 16347 consensus_queue.cc:260] T 0a9de347c711477e87c582d003f36925 P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:43.221827 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66ae176961b74813a58bc15c9c4c6f2a" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:18:43.222163 16347 ts_tablet_manager.cc:1434] T 0a9de347c711477e87c582d003f36925 P 1ca2725d290e47d68b1a4b609c9816ac: Time spent starting tablet: real 0.012s	user 0.005s	sys 0.000s
I20260709 13:18:43.222982 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66ae176961b74813a58bc15c9c4c6f2a" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:18:43.224301 16115 leader_election.cc:304] T 66ae176961b74813a58bc15c9c4c6f2a P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:18:43.224826 16566 raft_consensus.cc:515] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:43.225826 16516 raft_consensus.cc:2749] T 66ae176961b74813a58bc15c9c4c6f2a P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:18:43.226159 16347 tablet_bootstrap.cc:492] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac: Bootstrap starting.
I20260709 13:18:43.227397 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8158bbd17b994d2eb96f560bd0f1b88d" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:18:43.229167 16296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8158bbd17b994d2eb96f560bd0f1b88d" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:18:43.229408 16463 raft_consensus.cc:493] T 31969392ccdb43b491911a319e8bc7e1 P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:43.229722 16296 raft_consensus.cc:3060] T 8158bbd17b994d2eb96f560bd0f1b88d P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:43.229918 16463 raft_consensus.cc:515] T 31969392ccdb43b491911a319e8bc7e1 P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:43.231981 16463 leader_election.cc:290] T 31969392ccdb43b491911a319e8bc7e1 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:18:43.233012 16340 tablet_bootstrap.cc:492] T 2794af5b99984efebdee564035841cb0 P 647a343a542d4a139fe106c19095129b: No bootstrap required, opened a new log
I20260709 13:18:43.233422 16340 ts_tablet_manager.cc:1403] T 2794af5b99984efebdee564035841cb0 P 647a343a542d4a139fe106c19095129b: Time spent bootstrapping tablet: real 0.156s	user 0.043s	sys 0.027s
W20260709 13:18:43.234808 16191 leader_election.cc:343] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:18:43.236486 16296 raft_consensus.cc:2468] T 8158bbd17b994d2eb96f560bd0f1b88d P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 1.
I20260709 13:18:43.237473 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31969392ccdb43b491911a319e8bc7e1" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:18:43.237577 16190 leader_election.cc:304] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 647a343a542d4a139fe106c19095129b
I20260709 13:18:43.238755 16566 leader_election.cc:290] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 1 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:18:43.239517 16566 raft_consensus.cc:2804] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:18:43.239460 16145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4f65cdf00b748bb963ee6e81ef43b79" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:18:43.240083 16566 raft_consensus.cc:697] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac [term 1 LEADER]: Becoming Leader. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:18:43.240943 16566 consensus_queue.cc:237] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:18:43.241984 16265 leader_election.cc:343] T 31969392ccdb43b491911a319e8bc7e1 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:18:43.242720 16191 leader_election.cc:343] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:18:43.244650 16340 raft_consensus.cc:359] T 2794af5b99984efebdee564035841cb0 P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:43.245343 16340 raft_consensus.cc:385] T 2794af5b99984efebdee564035841cb0 P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:43.245412 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4f65cdf00b748bb963ee6e81ef43b79" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:18:43.245671 16340 raft_consensus.cc:740] T 2794af5b99984efebdee564035841cb0 P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Initialized, Role: FOLLOWER
I20260709 13:18:43.245949 16300 raft_consensus.cc:3060] T c4f65cdf00b748bb963ee6e81ef43b79 P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:43.245590 16342 raft_consensus.cc:359] T 08b7d1ab519044eda2d5e04a07242f25 P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:43.246336 16342 raft_consensus.cc:385] T 08b7d1ab519044eda2d5e04a07242f25 P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:43.246642 16342 raft_consensus.cc:740] T 08b7d1ab519044eda2d5e04a07242f25 P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Initialized, Role: FOLLOWER
I20260709 13:18:43.246500 16340 consensus_queue.cc:260] T 2794af5b99984efebdee564035841cb0 P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:43.248592 16340 ts_tablet_manager.cc:1434] T 2794af5b99984efebdee564035841cb0 P 647a343a542d4a139fe106c19095129b: Time spent starting tablet: real 0.015s	user 0.007s	sys 0.000s
I20260709 13:18:43.249337 16342 consensus_queue.cc:260] T 08b7d1ab519044eda2d5e04a07242f25 P 90f3da57796b488c9087a2f523adf045 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:43.250389 16347 tablet_bootstrap.cc:654] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:43.251567 16342 ts_tablet_manager.cc:1434] T 08b7d1ab519044eda2d5e04a07242f25 P 90f3da57796b488c9087a2f523adf045: Time spent starting tablet: real 0.044s	user 0.006s	sys 0.000s
I20260709 13:18:43.252415 16342 tablet_bootstrap.cc:492] T 8638ee363273426db1393dd81e5c7b72 P 90f3da57796b488c9087a2f523adf045: Bootstrap starting.
I20260709 13:18:43.253973 16340 tablet_bootstrap.cc:492] T 1df93779209748b9860f6d893dd3de53 P 647a343a542d4a139fe106c19095129b: Bootstrap starting.
I20260709 13:18:43.257220 16300 raft_consensus.cc:2468] T c4f65cdf00b748bb963ee6e81ef43b79 P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 1.
I20260709 13:18:43.258384 16190 leader_election.cc:304] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 647a343a542d4a139fe106c19095129b
I20260709 13:18:43.250962 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31969392ccdb43b491911a319e8bc7e1" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:18:43.267911 16376 raft_consensus.cc:2804] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:18:43.253705 16406 leader_election.cc:290] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 1 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
W20260709 13:18:43.288115 16265 leader_election.cc:343] T 31969392ccdb43b491911a319e8bc7e1 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:18:43.288563 16265 leader_election.cc:304] T 31969392ccdb43b491911a319e8bc7e1 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:18:43.289222 16463 raft_consensus.cc:2749] T 31969392ccdb43b491911a319e8bc7e1 P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:18:43.296142 16045 catalog_manager.cc:5697] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac reported cstate change: term changed from 0 to 1, leader changed from <none> to 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130). New cstate: current_term: 1 leader_uuid: "1ca2725d290e47d68b1a4b609c9816ac" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:43.307250 16610 consensus_queue.cc:1048] T af57996a8be84d398f0f023e8d08ef77 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:18:43.312907 16347 tablet_bootstrap.cc:492] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac: No bootstrap required, opened a new log
I20260709 13:18:43.313333 16347 ts_tablet_manager.cc:1403] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac: Time spent bootstrapping tablet: real 0.087s	user 0.009s	sys 0.008s
I20260709 13:18:43.320052 16376 raft_consensus.cc:697] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac [term 1 LEADER]: Becoming Leader. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:18:43.320873 16376 consensus_queue.cc:237] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:43.328070 16347 raft_consensus.cc:359] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:43.328752 16347 raft_consensus.cc:385] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:43.329030 16347 raft_consensus.cc:740] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Initialized, Role: FOLLOWER
I20260709 13:18:43.336802 16342 tablet_bootstrap.cc:654] T 8638ee363273426db1393dd81e5c7b72 P 90f3da57796b488c9087a2f523adf045: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:43.338483 16516 raft_consensus.cc:493] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:43.339035 16516 raft_consensus.cc:515] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:43.342890 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a9de347c711477e87c582d003f36925" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:18:43.343009 16347 consensus_queue.cc:260] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:43.343555 16300 raft_consensus.cc:2468] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 0.
I20260709 13:18:43.344784 16516 leader_election.cc:290] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:18:43.347496 16347 ts_tablet_manager.cc:1434] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac: Time spent starting tablet: real 0.034s	user 0.010s	sys 0.008s
I20260709 13:18:43.348469 16347 tablet_bootstrap.cc:492] T 2794af5b99984efebdee564035841cb0 P 1ca2725d290e47d68b1a4b609c9816ac: Bootstrap starting.
I20260709 13:18:43.345042 16115 leader_election.cc:304] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:18:43.346575 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a9de347c711477e87c582d003f36925" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:18:43.351469 16516 raft_consensus.cc:2804] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:18:43.351598 16150 raft_consensus.cc:3060] T 66ae176961b74813a58bc15c9c4c6f2a P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:43.353437 16226 raft_consensus.cc:2468] T 0a9de347c711477e87c582d003f36925 P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 0.
I20260709 13:18:43.351888 16516 raft_consensus.cc:493] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:43.367359 16516 raft_consensus.cc:3060] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:43.388669 16340 tablet_bootstrap.cc:654] T 1df93779209748b9860f6d893dd3de53 P 647a343a542d4a139fe106c19095129b: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:43.399906 16512 consensus_queue.cc:1048] T ee9ed67006144797b1de6ea8c2013a68 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:18:43.402735 16342 tablet_bootstrap.cc:492] T 8638ee363273426db1393dd81e5c7b72 P 90f3da57796b488c9087a2f523adf045: No bootstrap required, opened a new log
I20260709 13:18:43.403343 16342 ts_tablet_manager.cc:1403] T 8638ee363273426db1393dd81e5c7b72 P 90f3da57796b488c9087a2f523adf045: Time spent bootstrapping tablet: real 0.151s	user 0.011s	sys 0.003s
I20260709 13:18:43.412360 16347 tablet_bootstrap.cc:654] T 2794af5b99984efebdee564035841cb0 P 1ca2725d290e47d68b1a4b609c9816ac: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:43.414644 16048 catalog_manager.cc:5697] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac reported cstate change: term changed from 0 to 1, leader changed from <none> to 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130). New cstate: current_term: 1 leader_uuid: "1ca2725d290e47d68b1a4b609c9816ac" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:43.430114 16516 raft_consensus.cc:515] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:43.431782 16342 raft_consensus.cc:359] T 8638ee363273426db1393dd81e5c7b72 P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:43.433071 16516 leader_election.cc:290] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 1 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:18:43.433420 16296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a9de347c711477e87c582d003f36925" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:18:43.433970 16296 raft_consensus.cc:3060] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:43.432521 16342 raft_consensus.cc:385] T 8638ee363273426db1393dd81e5c7b72 P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:43.435038 16342 raft_consensus.cc:740] T 8638ee363273426db1393dd81e5c7b72 P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Initialized, Role: FOLLOWER
I20260709 13:18:43.436020 16342 consensus_queue.cc:260] T 8638ee363273426db1393dd81e5c7b72 P 90f3da57796b488c9087a2f523adf045 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:43.438172 16342 ts_tablet_manager.cc:1434] T 8638ee363273426db1393dd81e5c7b72 P 90f3da57796b488c9087a2f523adf045: Time spent starting tablet: real 0.032s	user 0.006s	sys 0.000s
I20260709 13:18:43.439043 16342 tablet_bootstrap.cc:492] T 1df93779209748b9860f6d893dd3de53 P 90f3da57796b488c9087a2f523adf045: Bootstrap starting.
I20260709 13:18:43.439896 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a9de347c711477e87c582d003f36925" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:18:43.448068 16225 raft_consensus.cc:3060] T 0a9de347c711477e87c582d003f36925 P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:43.463367 16565 raft_consensus.cc:493] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:43.464139 16565 raft_consensus.cc:515] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:43.469782 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c74490c40243a88e62d1760b5cd814" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
W20260709 13:18:43.472452 16265 leader_election.cc:343] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:18:43.496447 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c74490c40243a88e62d1760b5cd814" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
W20260709 13:18:43.503397 16265 leader_election.cc:343] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:18:43.503906 16265 leader_election.cc:304] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:18:43.506297 16463 raft_consensus.cc:2749] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:18:43.507269 16463 raft_consensus.cc:493] T 08b7d1ab519044eda2d5e04a07242f25 P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:43.496500 16565 leader_election.cc:290] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:18:43.507731 16463 raft_consensus.cc:515] T 08b7d1ab519044eda2d5e04a07242f25 P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:43.493676 16296 raft_consensus.cc:2468] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 1.
I20260709 13:18:43.509939 16463 leader_election.cc:290] T 08b7d1ab519044eda2d5e04a07242f25 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:18:43.510888 16342 tablet_bootstrap.cc:654] T 1df93779209748b9860f6d893dd3de53 P 90f3da57796b488c9087a2f523adf045: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:43.511256 16115 leader_election.cc:304] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:18:43.515666 16516 raft_consensus.cc:2804] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:18:43.515946 16340 tablet_bootstrap.cc:492] T 1df93779209748b9860f6d893dd3de53 P 647a343a542d4a139fe106c19095129b: No bootstrap required, opened a new log
I20260709 13:18:43.516202 16516 raft_consensus.cc:697] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b [term 1 LEADER]: Becoming Leader. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:18:43.517421 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b7d1ab519044eda2d5e04a07242f25" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:18:43.516489 16340 ts_tablet_manager.cc:1403] T 1df93779209748b9860f6d893dd3de53 P 647a343a542d4a139fe106c19095129b: Time spent bootstrapping tablet: real 0.265s	user 0.031s	sys 0.048s
W20260709 13:18:43.519388 16265 leader_election.cc:343] T 08b7d1ab519044eda2d5e04a07242f25 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:18:43.518848 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b7d1ab519044eda2d5e04a07242f25" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:18:43.521548 16511 consensus_queue.cc:1048] T ee9ed67006144797b1de6ea8c2013a68 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 13:18:43.521749 16265 leader_election.cc:343] T 08b7d1ab519044eda2d5e04a07242f25 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:18:43.522300 16265 leader_election.cc:304] T 08b7d1ab519044eda2d5e04a07242f25 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:18:43.522480 16340 raft_consensus.cc:359] T 1df93779209748b9860f6d893dd3de53 P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:43.522943 16463 raft_consensus.cc:2749] T 08b7d1ab519044eda2d5e04a07242f25 P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:18:43.523212 16340 raft_consensus.cc:385] T 1df93779209748b9860f6d893dd3de53 P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:43.523526 16340 raft_consensus.cc:740] T 1df93779209748b9860f6d893dd3de53 P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Initialized, Role: FOLLOWER
I20260709 13:18:43.524194 16340 consensus_queue.cc:260] T 1df93779209748b9860f6d893dd3de53 P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:43.526594 16340 ts_tablet_manager.cc:1434] T 1df93779209748b9860f6d893dd3de53 P 647a343a542d4a139fe106c19095129b: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.000s
I20260709 13:18:43.517962 16516 consensus_queue.cc:237] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:43.528573 16347 tablet_bootstrap.cc:492] T 2794af5b99984efebdee564035841cb0 P 1ca2725d290e47d68b1a4b609c9816ac: No bootstrap required, opened a new log
I20260709 13:18:43.529052 16347 ts_tablet_manager.cc:1403] T 2794af5b99984efebdee564035841cb0 P 1ca2725d290e47d68b1a4b609c9816ac: Time spent bootstrapping tablet: real 0.181s	user 0.031s	sys 0.030s
I20260709 13:18:43.535871 16340 tablet_bootstrap.cc:492] T 8158bbd17b994d2eb96f560bd0f1b88d P 647a343a542d4a139fe106c19095129b: Bootstrap starting.
I20260709 13:18:43.537475 16347 raft_consensus.cc:359] T 2794af5b99984efebdee564035841cb0 P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:43.538023 16347 raft_consensus.cc:385] T 2794af5b99984efebdee564035841cb0 P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:43.538254 16347 raft_consensus.cc:740] T 2794af5b99984efebdee564035841cb0 P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Initialized, Role: FOLLOWER
I20260709 13:18:43.539120 16347 consensus_queue.cc:260] T 2794af5b99984efebdee564035841cb0 P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:43.540915 16347 ts_tablet_manager.cc:1434] T 2794af5b99984efebdee564035841cb0 P 1ca2725d290e47d68b1a4b609c9816ac: Time spent starting tablet: real 0.011s	user 0.004s	sys 0.000s
I20260709 13:18:43.547498 16347 tablet_bootstrap.cc:492] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac: Bootstrap starting.
I20260709 13:18:43.532744 16225 raft_consensus.cc:2468] T 0a9de347c711477e87c582d003f36925 P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 1.
I20260709 13:18:43.559294 16511 raft_consensus.cc:493] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:43.560825 16511 raft_consensus.cc:515] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:43.564786 16511 leader_election.cc:290] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:18:43.564653 16295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806d7690c2b644959152074bb2cab0a7" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:18:43.598591 16340 tablet_bootstrap.cc:654] T 8158bbd17b994d2eb96f560bd0f1b88d P 647a343a542d4a139fe106c19095129b: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:43.563640 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806d7690c2b644959152074bb2cab0a7" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:18:43.604306 16225 raft_consensus.cc:2468] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 0.
I20260709 13:18:43.606901 16342 tablet_bootstrap.cc:492] T 1df93779209748b9860f6d893dd3de53 P 90f3da57796b488c9087a2f523adf045: No bootstrap required, opened a new log
I20260709 13:18:43.607713 16342 ts_tablet_manager.cc:1403] T 1df93779209748b9860f6d893dd3de53 P 90f3da57796b488c9087a2f523adf045: Time spent bootstrapping tablet: real 0.169s	user 0.015s	sys 0.001s
I20260709 13:18:43.609424 16116 leader_election.cc:304] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
I20260709 13:18:43.613816 16521 raft_consensus.cc:2804] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:18:43.614212 16521 raft_consensus.cc:493] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:43.614550 16521 raft_consensus.cc:3060] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:43.615849 16342 raft_consensus.cc:359] T 1df93779209748b9860f6d893dd3de53 P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:43.616549 16342 raft_consensus.cc:385] T 1df93779209748b9860f6d893dd3de53 P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:43.616815 16342 raft_consensus.cc:740] T 1df93779209748b9860f6d893dd3de53 P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Initialized, Role: FOLLOWER
I20260709 13:18:43.617532 16342 consensus_queue.cc:260] T 1df93779209748b9860f6d893dd3de53 P 90f3da57796b488c9087a2f523adf045 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:18:43.619714 16097 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:18:43.621420 16521 raft_consensus.cc:515] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:43.623771 16342 ts_tablet_manager.cc:1434] T 1df93779209748b9860f6d893dd3de53 P 90f3da57796b488c9087a2f523adf045: Time spent starting tablet: real 0.016s	user 0.006s	sys 0.000s
I20260709 13:18:43.625241 16342 tablet_bootstrap.cc:492] T ecc1e363230043fe9fef390f0b979f20 P 90f3da57796b488c9087a2f523adf045: Bootstrap starting.
I20260709 13:18:43.609493 16295 raft_consensus.cc:2468] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 0.
I20260709 13:18:43.640396 16516 consensus_queue.cc:1048] T 746aabf856ab401c9ef5e859371e7b67 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:18:43.643402 16521 leader_election.cc:290] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 1 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:18:43.643971 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806d7690c2b644959152074bb2cab0a7" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:18:43.644629 16225 raft_consensus.cc:3060] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:43.644116 16048 catalog_manager.cc:5697] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b reported cstate change: term changed from 0 to 1, leader changed from <none> to 647a343a542d4a139fe106c19095129b (127.14.177.129). New cstate: current_term: 1 leader_uuid: "647a343a542d4a139fe106c19095129b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:43.646790 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806d7690c2b644959152074bb2cab0a7" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:18:43.647534 16299 raft_consensus.cc:3060] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:43.647542 16463 raft_consensus.cc:493] T 1df93779209748b9860f6d893dd3de53 P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:43.651898 16463 raft_consensus.cc:515] T 1df93779209748b9860f6d893dd3de53 P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:43.654744 16151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1df93779209748b9860f6d893dd3de53" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:18:43.655349 16151 raft_consensus.cc:2468] T 1df93779209748b9860f6d893dd3de53 P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 0.
I20260709 13:18:43.657003 16463 leader_election.cc:290] T 1df93779209748b9860f6d893dd3de53 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:18:43.658299 16347 tablet_bootstrap.cc:654] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:43.658190 16265 leader_election.cc:304] T 1df93779209748b9860f6d893dd3de53 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:18:43.665069 16520 consensus_queue.cc:1048] T 746aabf856ab401c9ef5e859371e7b67 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 13:18:43.664896 16565 raft_consensus.cc:2804] T 1df93779209748b9860f6d893dd3de53 P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:18:43.670262 16565 raft_consensus.cc:493] T 1df93779209748b9860f6d893dd3de53 P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:43.671758 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1df93779209748b9860f6d893dd3de53" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
W20260709 13:18:43.674283 16265 leader_election.cc:343] T 1df93779209748b9860f6d893dd3de53 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:18:43.678925 16225 raft_consensus.cc:2468] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 1.
I20260709 13:18:43.670749 16565 raft_consensus.cc:3060] T 1df93779209748b9860f6d893dd3de53 P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:43.681416 16116 leader_election.cc:304] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
I20260709 13:18:43.693812 16299 raft_consensus.cc:2468] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 1.
I20260709 13:18:43.701563 16516 raft_consensus.cc:493] T 2794af5b99984efebdee564035841cb0 P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:43.702118 16516 raft_consensus.cc:515] T 2794af5b99984efebdee564035841cb0 P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:43.687551 16520 raft_consensus.cc:2804] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:18:43.723356 16611 consensus_queue.cc:1048] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:18:43.775584 16520 raft_consensus.cc:697] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b [term 1 LEADER]: Becoming Leader. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
W20260709 13:18:43.777413 16115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 81814.1us
I20260709 13:18:43.791220 16516 leader_election.cc:290] T 2794af5b99984efebdee564035841cb0 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:18:43.794199 16340 tablet_bootstrap.cc:492] T 8158bbd17b994d2eb96f560bd0f1b88d P 647a343a542d4a139fe106c19095129b: No bootstrap required, opened a new log
I20260709 13:18:43.795004 16340 ts_tablet_manager.cc:1403] T 8158bbd17b994d2eb96f560bd0f1b88d P 647a343a542d4a139fe106c19095129b: Time spent bootstrapping tablet: real 0.259s	user 0.022s	sys 0.031s
I20260709 13:18:43.802772 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2794af5b99984efebdee564035841cb0" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:18:43.803476 16224 raft_consensus.cc:2468] T 2794af5b99984efebdee564035841cb0 P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 0.
I20260709 13:18:43.807492 16520 consensus_queue.cc:237] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:18:43.810067 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 85588.7us
I20260709 13:18:43.818677 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2794af5b99984efebdee564035841cb0" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:18:43.819379 16300 raft_consensus.cc:2468] T 2794af5b99984efebdee564035841cb0 P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 0.
I20260709 13:18:43.844588 16115 leader_election.cc:304] T 2794af5b99984efebdee564035841cb0 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:18:43.846011 16516 raft_consensus.cc:2804] T 2794af5b99984efebdee564035841cb0 P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:18:43.846426 16516 raft_consensus.cc:493] T 2794af5b99984efebdee564035841cb0 P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:43.846808 16516 raft_consensus.cc:3060] T 2794af5b99984efebdee564035841cb0 P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:43.861338 16340 raft_consensus.cc:359] T 8158bbd17b994d2eb96f560bd0f1b88d P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:43.862057 16340 raft_consensus.cc:385] T 8158bbd17b994d2eb96f560bd0f1b88d P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:43.862330 16340 raft_consensus.cc:740] T 8158bbd17b994d2eb96f560bd0f1b88d P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Initialized, Role: FOLLOWER
I20260709 13:18:43.863582 16516 raft_consensus.cc:515] T 2794af5b99984efebdee564035841cb0 P 647a343a542d4a139fe106c19095129b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:43.865609 16340 consensus_queue.cc:260] T 8158bbd17b994d2eb96f560bd0f1b88d P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:18:43.868744 16191 consensus_peers.cc:597] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:18:43.877928 16516 leader_election.cc:290] T 2794af5b99984efebdee564035841cb0 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 1 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:18:43.877526 16340 ts_tablet_manager.cc:1434] T 8158bbd17b994d2eb96f560bd0f1b88d P 647a343a542d4a139fe106c19095129b: Time spent starting tablet: real 0.082s	user 0.003s	sys 0.004s
I20260709 13:18:43.883138 16223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2794af5b99984efebdee564035841cb0" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:18:43.883855 16223 raft_consensus.cc:3060] T 2794af5b99984efebdee564035841cb0 P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:43.885986 16340 tablet_bootstrap.cc:492] T ecc1e363230043fe9fef390f0b979f20 P 647a343a542d4a139fe106c19095129b: Bootstrap starting.
W20260709 13:18:43.888399 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 58853.7us
I20260709 13:18:43.902789 16295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2794af5b99984efebdee564035841cb0" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:18:43.903470 16295 raft_consensus.cc:3060] T 2794af5b99984efebdee564035841cb0 P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:43.910602 16295 raft_consensus.cc:2468] T 2794af5b99984efebdee564035841cb0 P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 1.
I20260709 13:18:43.889645 16629 raft_consensus.cc:493] T 8638ee363273426db1393dd81e5c7b72 P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:43.915366 16191 consensus_peers.cc:597] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:18:43.922401 16115 leader_election.cc:304] T 2794af5b99984efebdee564035841cb0 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:18:43.923230 16516 raft_consensus.cc:2804] T 2794af5b99984efebdee564035841cb0 P 647a343a542d4a139fe106c19095129b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:18:43.923695 16516 raft_consensus.cc:697] T 2794af5b99984efebdee564035841cb0 P 647a343a542d4a139fe106c19095129b [term 1 LEADER]: Becoming Leader. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:18:43.924472 16516 consensus_queue.cc:237] T 2794af5b99984efebdee564035841cb0 P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:43.931306 16342 tablet_bootstrap.cc:654] T ecc1e363230043fe9fef390f0b979f20 P 90f3da57796b488c9087a2f523adf045: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:43.918561 16629 raft_consensus.cc:515] T 8638ee363273426db1393dd81e5c7b72 P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:43.927656 16565 raft_consensus.cc:515] T 1df93779209748b9860f6d893dd3de53 P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:43.952257 16629 leader_election.cc:290] T 8638ee363273426db1393dd81e5c7b72 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:18:43.955760 16608 consensus_queue.cc:1048] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.005s
I20260709 13:18:43.958135 16340 tablet_bootstrap.cc:654] T ecc1e363230043fe9fef390f0b979f20 P 647a343a542d4a139fe106c19095129b: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:43.951148 16223 raft_consensus.cc:2468] T 2794af5b99984efebdee564035841cb0 P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 1.
W20260709 13:18:43.956620 16633 log.cc:927] Time spent T 8158bbd17b994d2eb96f560bd0f1b88d P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.062s	user 0.001s	sys 0.000s
W20260709 13:18:43.962760 16628 log.cc:927] Time spent T 746aabf856ab401c9ef5e859371e7b67 P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 0.081s	user 0.001s	sys 0.000s
I20260709 13:18:43.956852 16565 leader_election.cc:290] T 1df93779209748b9860f6d893dd3de53 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 1 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:18:43.975179 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8638ee363273426db1393dd81e5c7b72" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:18:43.976261 16151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1df93779209748b9860f6d893dd3de53" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:18:43.976893 16151 raft_consensus.cc:3060] T 1df93779209748b9860f6d893dd3de53 P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:43.981395 16048 catalog_manager.cc:5697] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b reported cstate change: term changed from 0 to 1, leader changed from <none> to 647a343a542d4a139fe106c19095129b (127.14.177.129). New cstate: current_term: 1 leader_uuid: "647a343a542d4a139fe106c19095129b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } health_report { overall_health: UNKNOWN } } }
W20260709 13:18:43.991381 16265 leader_election.cc:343] T 8638ee363273426db1393dd81e5c7b72 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:18:43.993000 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8638ee363273426db1393dd81e5c7b72" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:18:44.002229 16223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1df93779209748b9860f6d893dd3de53" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
W20260709 13:18:44.004703 16265 leader_election.cc:343] T 8638ee363273426db1393dd81e5c7b72 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:18:44.005239 16265 leader_election.cc:304] T 8638ee363273426db1393dd81e5c7b72 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
W20260709 13:18:44.006302 16265 leader_election.cc:343] T 1df93779209748b9860f6d893dd3de53 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:18:44.006907 16632 raft_consensus.cc:2749] T 8638ee363273426db1393dd81e5c7b72 P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:18:44.008816 16347 tablet_bootstrap.cc:492] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac: No bootstrap required, opened a new log
I20260709 13:18:44.010265 16347 ts_tablet_manager.cc:1403] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac: Time spent bootstrapping tablet: real 0.463s	user 0.012s	sys 0.010s
I20260709 13:18:44.018214 16342 tablet_bootstrap.cc:492] T ecc1e363230043fe9fef390f0b979f20 P 90f3da57796b488c9087a2f523adf045: No bootstrap required, opened a new log
I20260709 13:18:44.018772 16347 raft_consensus.cc:359] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:44.019481 16347 raft_consensus.cc:385] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:44.019734 16347 raft_consensus.cc:740] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Initialized, Role: FOLLOWER
I20260709 13:18:44.020481 16151 raft_consensus.cc:2468] T 1df93779209748b9860f6d893dd3de53 P 647a343a542d4a139fe106c19095129b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 1.
I20260709 13:18:44.020341 16347 consensus_queue.cc:260] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:44.018693 16342 ts_tablet_manager.cc:1403] T ecc1e363230043fe9fef390f0b979f20 P 90f3da57796b488c9087a2f523adf045: Time spent bootstrapping tablet: real 0.394s	user 0.013s	sys 0.006s
I20260709 13:18:44.026147 16265 leader_election.cc:304] T 1df93779209748b9860f6d893dd3de53 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac
I20260709 13:18:44.027326 16347 ts_tablet_manager.cc:1434] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac: Time spent starting tablet: real 0.017s	user 0.007s	sys 0.000s
I20260709 13:18:44.028158 16347 tablet_bootstrap.cc:492] T ecc1e363230043fe9fef390f0b979f20 P 1ca2725d290e47d68b1a4b609c9816ac: Bootstrap starting.
I20260709 13:18:44.029268 16629 raft_consensus.cc:2804] T 1df93779209748b9860f6d893dd3de53 P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:18:44.029878 16629 raft_consensus.cc:697] T 1df93779209748b9860f6d893dd3de53 P 90f3da57796b488c9087a2f523adf045 [term 1 LEADER]: Becoming Leader. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:18:44.030550 16342 raft_consensus.cc:359] T ecc1e363230043fe9fef390f0b979f20 P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:44.030685 16629 consensus_queue.cc:237] T 1df93779209748b9860f6d893dd3de53 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:44.031296 16342 raft_consensus.cc:385] T ecc1e363230043fe9fef390f0b979f20 P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:44.031565 16342 raft_consensus.cc:740] T ecc1e363230043fe9fef390f0b979f20 P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Initialized, Role: FOLLOWER
I20260709 13:18:44.037295 16342 consensus_queue.cc:260] T ecc1e363230043fe9fef390f0b979f20 P 90f3da57796b488c9087a2f523adf045 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:44.040490 16347 tablet_bootstrap.cc:654] T ecc1e363230043fe9fef390f0b979f20 P 1ca2725d290e47d68b1a4b609c9816ac: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:44.036237 16045 catalog_manager.cc:5697] T 2794af5b99984efebdee564035841cb0 P 647a343a542d4a139fe106c19095129b reported cstate change: term changed from 0 to 1, leader changed from <none> to 647a343a542d4a139fe106c19095129b (127.14.177.129). New cstate: current_term: 1 leader_uuid: "647a343a542d4a139fe106c19095129b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:44.042021 16340 tablet_bootstrap.cc:492] T ecc1e363230043fe9fef390f0b979f20 P 647a343a542d4a139fe106c19095129b: No bootstrap required, opened a new log
I20260709 13:18:44.042462 16340 ts_tablet_manager.cc:1403] T ecc1e363230043fe9fef390f0b979f20 P 647a343a542d4a139fe106c19095129b: Time spent bootstrapping tablet: real 0.157s	user 0.014s	sys 0.004s
I20260709 13:18:44.047375 16340 raft_consensus.cc:359] T ecc1e363230043fe9fef390f0b979f20 P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:44.048075 16340 raft_consensus.cc:385] T ecc1e363230043fe9fef390f0b979f20 P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:44.048396 16340 raft_consensus.cc:740] T ecc1e363230043fe9fef390f0b979f20 P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Initialized, Role: FOLLOWER
I20260709 13:18:44.049067 16340 consensus_queue.cc:260] T ecc1e363230043fe9fef390f0b979f20 P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:44.051193 16340 ts_tablet_manager.cc:1434] T ecc1e363230043fe9fef390f0b979f20 P 647a343a542d4a139fe106c19095129b: Time spent starting tablet: real 0.008s	user 0.001s	sys 0.003s
I20260709 13:18:44.051364 16342 ts_tablet_manager.cc:1434] T ecc1e363230043fe9fef390f0b979f20 P 90f3da57796b488c9087a2f523adf045: Time spent starting tablet: real 0.030s	user 0.002s	sys 0.005s
I20260709 13:18:44.052060 16340 tablet_bootstrap.cc:492] T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b: Bootstrap starting.
I20260709 13:18:44.052239 16342 tablet_bootstrap.cc:492] T e4dc6b0541ca431ebc757beac00521f1 P 90f3da57796b488c9087a2f523adf045: Bootstrap starting.
I20260709 13:18:44.053454 16048 catalog_manager.cc:5697] T 1df93779209748b9860f6d893dd3de53 P 90f3da57796b488c9087a2f523adf045 reported cstate change: term changed from 0 to 1, leader changed from <none> to 90f3da57796b488c9087a2f523adf045 (127.14.177.131). New cstate: current_term: 1 leader_uuid: "90f3da57796b488c9087a2f523adf045" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } health_report { overall_health: HEALTHY } } }
I20260709 13:18:44.057490 16347 tablet_bootstrap.cc:492] T ecc1e363230043fe9fef390f0b979f20 P 1ca2725d290e47d68b1a4b609c9816ac: No bootstrap required, opened a new log
I20260709 13:18:44.057983 16347 ts_tablet_manager.cc:1403] T ecc1e363230043fe9fef390f0b979f20 P 1ca2725d290e47d68b1a4b609c9816ac: Time spent bootstrapping tablet: real 0.030s	user 0.006s	sys 0.005s
I20260709 13:18:44.061352 16342 tablet_bootstrap.cc:654] T e4dc6b0541ca431ebc757beac00521f1 P 90f3da57796b488c9087a2f523adf045: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:44.061107 16347 raft_consensus.cc:359] T ecc1e363230043fe9fef390f0b979f20 P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:44.061784 16347 raft_consensus.cc:385] T ecc1e363230043fe9fef390f0b979f20 P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:44.062047 16347 raft_consensus.cc:740] T ecc1e363230043fe9fef390f0b979f20 P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Initialized, Role: FOLLOWER
I20260709 13:18:44.066929 16340 tablet_bootstrap.cc:654] T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:44.068091 16631 consensus_queue.cc:1048] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:18:44.069612 16347 consensus_queue.cc:260] T ecc1e363230043fe9fef390f0b979f20 P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:44.071784 16347 ts_tablet_manager.cc:1434] T ecc1e363230043fe9fef390f0b979f20 P 1ca2725d290e47d68b1a4b609c9816ac: Time spent starting tablet: real 0.014s	user 0.003s	sys 0.005s
I20260709 13:18:44.072625 16347 tablet_bootstrap.cc:492] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac: Bootstrap starting.
I20260709 13:18:44.119642 16347 tablet_bootstrap.cc:654] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:44.125572 16342 tablet_bootstrap.cc:492] T e4dc6b0541ca431ebc757beac00521f1 P 90f3da57796b488c9087a2f523adf045: No bootstrap required, opened a new log
I20260709 13:18:44.126600 16342 ts_tablet_manager.cc:1403] T e4dc6b0541ca431ebc757beac00521f1 P 90f3da57796b488c9087a2f523adf045: Time spent bootstrapping tablet: real 0.075s	user 0.036s	sys 0.016s
I20260709 13:18:44.132110 16630 consensus_queue.cc:1048] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:18:44.133423 16340 tablet_bootstrap.cc:492] T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b: No bootstrap required, opened a new log
I20260709 13:18:44.134065 16340 ts_tablet_manager.cc:1403] T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b: Time spent bootstrapping tablet: real 0.082s	user 0.005s	sys 0.008s
I20260709 13:18:44.142114 16342 raft_consensus.cc:359] T e4dc6b0541ca431ebc757beac00521f1 P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:44.143013 16342 raft_consensus.cc:385] T e4dc6b0541ca431ebc757beac00521f1 P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:44.143426 16342 raft_consensus.cc:740] T e4dc6b0541ca431ebc757beac00521f1 P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Initialized, Role: FOLLOWER
I20260709 13:18:44.144444 16342 consensus_queue.cc:260] T e4dc6b0541ca431ebc757beac00521f1 P 90f3da57796b488c9087a2f523adf045 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:44.152273 16342 ts_tablet_manager.cc:1434] T e4dc6b0541ca431ebc757beac00521f1 P 90f3da57796b488c9087a2f523adf045: Time spent starting tablet: real 0.025s	user 0.005s	sys 0.001s
I20260709 13:18:44.162966 16340 raft_consensus.cc:359] T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:44.163722 16340 raft_consensus.cc:385] T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:44.163921 16342 tablet_bootstrap.cc:492] T e5ad4eb281564936a59f41e387a47156 P 90f3da57796b488c9087a2f523adf045: Bootstrap starting.
I20260709 13:18:44.163995 16340 raft_consensus.cc:740] T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Initialized, Role: FOLLOWER
I20260709 13:18:44.166666 16340 consensus_queue.cc:260] T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:44.171828 16340 ts_tablet_manager.cc:1434] T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b: Time spent starting tablet: real 0.037s	user 0.009s	sys 0.000s
I20260709 13:18:44.172739 16340 tablet_bootstrap.cc:492] T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b: Bootstrap starting.
I20260709 13:18:44.185246 16340 tablet_bootstrap.cc:654] T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:44.191049 16347 tablet_bootstrap.cc:492] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac: No bootstrap required, opened a new log
I20260709 13:18:44.191689 16347 ts_tablet_manager.cc:1403] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac: Time spent bootstrapping tablet: real 0.119s	user 0.007s	sys 0.008s
I20260709 13:18:44.192647 16342 tablet_bootstrap.cc:654] T e5ad4eb281564936a59f41e387a47156 P 90f3da57796b488c9087a2f523adf045: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:44.195333 16347 raft_consensus.cc:359] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:44.196120 16347 raft_consensus.cc:385] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:44.196398 16347 raft_consensus.cc:740] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Initialized, Role: FOLLOWER
I20260709 13:18:44.197332 16340 tablet_bootstrap.cc:492] T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b: No bootstrap required, opened a new log
I20260709 13:18:44.197019 16347 consensus_queue.cc:260] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:44.197337 16342 tablet_bootstrap.cc:492] T e5ad4eb281564936a59f41e387a47156 P 90f3da57796b488c9087a2f523adf045: No bootstrap required, opened a new log
I20260709 13:18:44.197906 16340 ts_tablet_manager.cc:1403] T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b: Time spent bootstrapping tablet: real 0.025s	user 0.005s	sys 0.011s
I20260709 13:18:44.198174 16342 ts_tablet_manager.cc:1403] T e5ad4eb281564936a59f41e387a47156 P 90f3da57796b488c9087a2f523adf045: Time spent bootstrapping tablet: real 0.034s	user 0.008s	sys 0.002s
I20260709 13:18:44.247282 16631 raft_consensus.cc:493] T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:44.248581 16347 ts_tablet_manager.cc:1434] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac: Time spent starting tablet: real 0.056s	user 0.006s	sys 0.004s
I20260709 13:18:44.249572 16630 raft_consensus.cc:493] T ecc1e363230043fe9fef390f0b979f20 P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:44.249017 16631 raft_consensus.cc:515] T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:44.251344 16347 tablet_bootstrap.cc:492] T 08c74490c40243a88e62d1760b5cd814 P 1ca2725d290e47d68b1a4b609c9816ac: Bootstrap starting.
I20260709 13:18:44.253121 16631 leader_election.cc:290] T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:18:44.261081 16632 raft_consensus.cc:493] T e4dc6b0541ca431ebc757beac00521f1 P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:44.263432 16632 raft_consensus.cc:515] T e4dc6b0541ca431ebc757beac00521f1 P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:44.250505 16630 raft_consensus.cc:515] T ecc1e363230043fe9fef390f0b979f20 P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:44.274227 16630 leader_election.cc:290] T ecc1e363230043fe9fef390f0b979f20 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:18:44.275985 16151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4dc6b0541ca431ebc757beac00521f1" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:18:44.277040 16151 raft_consensus.cc:2468] T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 0.
I20260709 13:18:44.279536 16295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4dc6b0541ca431ebc757beac00521f1" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:18:44.280359 16295 raft_consensus.cc:2468] T e4dc6b0541ca431ebc757beac00521f1 P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 0.
I20260709 13:18:44.281345 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4dc6b0541ca431ebc757beac00521f1" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:18:44.283874 16511 raft_consensus.cc:493] T 8158bbd17b994d2eb96f560bd0f1b88d P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:44.286195 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
W20260709 13:18:44.286639 16116 leader_election.cc:343] T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:18:44.287089 16225 raft_consensus.cc:2468] T ecc1e363230043fe9fef390f0b979f20 P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 0.
I20260709 13:18:44.284716 16511 raft_consensus.cc:515] T 8158bbd17b994d2eb96f560bd0f1b88d P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:44.294973 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8158bbd17b994d2eb96f560bd0f1b88d" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:18:44.292342 16116 leader_election.cc:304] T ecc1e363230043fe9fef390f0b979f20 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
I20260709 13:18:44.290360 16632 leader_election.cc:290] T e4dc6b0541ca431ebc757beac00521f1 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:18:44.302143 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4dc6b0541ca431ebc757beac00521f1" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:18:44.303130 16631 raft_consensus.cc:2804] T ecc1e363230043fe9fef390f0b979f20 P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:18:44.303732 16631 raft_consensus.cc:493] T ecc1e363230043fe9fef390f0b979f20 P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:44.304154 16631 raft_consensus.cc:3060] T ecc1e363230043fe9fef390f0b979f20 P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:44.300299 16342 raft_consensus.cc:359] T e5ad4eb281564936a59f41e387a47156 P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:44.309438 16342 raft_consensus.cc:385] T e5ad4eb281564936a59f41e387a47156 P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:44.309803 16342 raft_consensus.cc:740] T e5ad4eb281564936a59f41e387a47156 P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Initialized, Role: FOLLOWER
I20260709 13:18:44.310664 16342 consensus_queue.cc:260] T e5ad4eb281564936a59f41e387a47156 P 90f3da57796b488c9087a2f523adf045 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:44.311681 16265 leader_election.cc:304] T e4dc6b0541ca431ebc757beac00521f1 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:18:44.311846 16115 leader_election.cc:304] T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac
W20260709 13:18:44.313495 16265 leader_election.cc:343] T e4dc6b0541ca431ebc757beac00521f1 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:18:44.315707 16634 raft_consensus.cc:2804] T e4dc6b0541ca431ebc757beac00521f1 P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:18:44.315850 16511 leader_election.cc:290] T 8158bbd17b994d2eb96f560bd0f1b88d P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:18:44.316200 16634 raft_consensus.cc:493] T e4dc6b0541ca431ebc757beac00521f1 P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:44.316382 16342 ts_tablet_manager.cc:1434] T e5ad4eb281564936a59f41e387a47156 P 90f3da57796b488c9087a2f523adf045: Time spent starting tablet: real 0.118s	user 0.001s	sys 0.011s
I20260709 13:18:44.316627 16634 raft_consensus.cc:3060] T e4dc6b0541ca431ebc757beac00521f1 P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:44.319581 16627 raft_consensus.cc:2804] T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:18:44.320067 16627 raft_consensus.cc:493] T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:44.320427 16627 raft_consensus.cc:3060] T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:44.320336 16296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:18:44.321180 16296 raft_consensus.cc:2468] T ecc1e363230043fe9fef390f0b979f20 P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 0.
I20260709 13:18:44.329324 16145 raft_consensus.cc:3060] T 8158bbd17b994d2eb96f560bd0f1b88d P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:44.332690 16521 consensus_queue.cc:1048] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.014s
I20260709 13:18:44.346355 16340 raft_consensus.cc:359] T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:44.347234 16340 raft_consensus.cc:385] T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:44.347509 16340 raft_consensus.cc:740] T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Initialized, Role: FOLLOWER
I20260709 13:18:44.361734 16347 tablet_bootstrap.cc:654] T 08c74490c40243a88e62d1760b5cd814 P 1ca2725d290e47d68b1a4b609c9816ac: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:44.363049 16340 consensus_queue.cc:260] T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:44.366356 16342 tablet_bootstrap.cc:492] T 57a744f5aaf944678a63469cbf514015 P 90f3da57796b488c9087a2f523adf045: Bootstrap starting.
I20260709 13:18:44.368536 16512 raft_consensus.cc:493] T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:44.369134 16512 raft_consensus.cc:515] T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:44.371688 16512 leader_election.cc:290] T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:18:44.372756 16295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8158bbd17b994d2eb96f560bd0f1b88d" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:18:44.380626 16340 ts_tablet_manager.cc:1434] T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b: Time spent starting tablet: real 0.182s	user 0.009s	sys 0.002s
I20260709 13:18:44.382726 16340 tablet_bootstrap.cc:492] T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b: Bootstrap starting.
I20260709 13:18:44.384079 16296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b7d1ab519044eda2d5e04a07242f25" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:18:44.384826 16296 raft_consensus.cc:2468] T 08b7d1ab519044eda2d5e04a07242f25 P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 0.
I20260709 13:18:44.384857 16115 leader_election.cc:304] T 8158bbd17b994d2eb96f560bd0f1b88d P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:18:44.388217 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b7d1ab519044eda2d5e04a07242f25" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:18:44.390147 16226 raft_consensus.cc:2468] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 0.
I20260709 13:18:44.390071 16651 consensus_queue.cc:1048] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:18:44.389217 16115 leader_election.cc:304] T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:18:44.391851 16651 raft_consensus.cc:2804] T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:18:44.392257 16651 raft_consensus.cc:493] T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:44.392603 16651 raft_consensus.cc:3060] T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:44.392789 16631 raft_consensus.cc:515] T ecc1e363230043fe9fef390f0b979f20 P 647a343a542d4a139fe106c19095129b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:44.365564 16627 raft_consensus.cc:515] T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:44.408248 16627 leader_election.cc:290] T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 1 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:18:44.409417 16631 leader_election.cc:290] T ecc1e363230043fe9fef390f0b979f20 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 1 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:18:44.420490 16223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:18:44.421180 16223 raft_consensus.cc:3060] T ecc1e363230043fe9fef390f0b979f20 P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:44.421557 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4dc6b0541ca431ebc757beac00521f1" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
W20260709 13:18:44.424438 16116 leader_election.cc:343] T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:18:44.412099 16634 raft_consensus.cc:515] T e4dc6b0541ca431ebc757beac00521f1 P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:44.439893 16512 raft_consensus.cc:2749] T 8158bbd17b994d2eb96f560bd0f1b88d P 647a343a542d4a139fe106c19095129b [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:18:44.452165 16342 tablet_bootstrap.cc:654] T 57a744f5aaf944678a63469cbf514015 P 90f3da57796b488c9087a2f523adf045: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:44.436704 16651 raft_consensus.cc:515] T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:44.457501 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4dc6b0541ca431ebc757beac00521f1" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:18:44.458197 16340 tablet_bootstrap.cc:654] T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:44.457562 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4dc6b0541ca431ebc757beac00521f1" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:18:44.459659 16295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4dc6b0541ca431ebc757beac00521f1" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:18:44.459939 16150 raft_consensus.cc:2393] T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 1: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
I20260709 13:18:44.460451 16295 raft_consensus.cc:2393] T e4dc6b0541ca431ebc757beac00521f1 P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 1: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
I20260709 13:18:44.460709 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:18:44.461242 16299 raft_consensus.cc:3060] T ecc1e363230043fe9fef390f0b979f20 P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:44.463575 16608 raft_consensus.cc:493] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:44.464221 16608 raft_consensus.cc:515] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:18:44.461647 16265 leader_election.cc:343] T e4dc6b0541ca431ebc757beac00521f1 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:18:44.469007 16265 leader_election.cc:304] T e4dc6b0541ca431ebc757beac00521f1 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:18:44.470728 16632 raft_consensus.cc:2749] T e4dc6b0541ca431ebc757beac00521f1 P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:18:44.473842 16223 raft_consensus.cc:2468] T ecc1e363230043fe9fef390f0b979f20 P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 1.
I20260709 13:18:44.475116 16651 leader_election.cc:290] T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 1 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:18:44.476310 16295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8638ee363273426db1393dd81e5c7b72" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:18:44.483723 16295 raft_consensus.cc:2468] T 8638ee363273426db1393dd81e5c7b72 P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 0.
I20260709 13:18:44.485155 16190 leader_election.cc:304] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:18:44.486090 16611 raft_consensus.cc:2804] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:18:44.486469 16611 raft_consensus.cc:493] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:44.486799 16611 raft_consensus.cc:3060] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:44.477109 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8638ee363273426db1393dd81e5c7b72" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:18:44.489308 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b7d1ab519044eda2d5e04a07242f25" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:18:44.489913 16225 raft_consensus.cc:3060] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:44.456607 16634 leader_election.cc:290] T e4dc6b0541ca431ebc757beac00521f1 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 1 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:18:44.477617 16115 leader_election.cc:304] T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:18:44.478134 16299 raft_consensus.cc:2468] T ecc1e363230043fe9fef390f0b979f20 P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 1.
I20260709 13:18:44.477830 16630 consensus_queue.cc:1048] T 2794af5b99984efebdee564035841cb0 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.005s
I20260709 13:18:44.480034 16608 leader_election.cc:290] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:18:44.493978 16516 consensus_queue.cc:1048] T 2794af5b99984efebdee564035841cb0 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:18:44.497138 16295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b7d1ab519044eda2d5e04a07242f25" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:18:44.498425 16116 leader_election.cc:304] T ecc1e363230043fe9fef390f0b979f20 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
I20260709 13:18:44.497773 16295 raft_consensus.cc:3060] T 08b7d1ab519044eda2d5e04a07242f25 P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:44.501444 16520 raft_consensus.cc:2749] T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:18:44.505024 16347 tablet_bootstrap.cc:492] T 08c74490c40243a88e62d1760b5cd814 P 1ca2725d290e47d68b1a4b609c9816ac: No bootstrap required, opened a new log
I20260709 13:18:44.505607 16347 ts_tablet_manager.cc:1403] T 08c74490c40243a88e62d1760b5cd814 P 1ca2725d290e47d68b1a4b609c9816ac: Time spent bootstrapping tablet: real 0.255s	user 0.014s	sys 0.017s
I20260709 13:18:44.501997 16611 raft_consensus.cc:515] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:44.507892 16651 raft_consensus.cc:2804] T ecc1e363230043fe9fef390f0b979f20 P 647a343a542d4a139fe106c19095129b [term 1 FOLLOWER]: Leader election won for term 1
W20260709 13:18:44.518059 16191 leader_election.cc:343] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:18:44.520409 16651 raft_consensus.cc:697] T ecc1e363230043fe9fef390f0b979f20 P 647a343a542d4a139fe106c19095129b [term 1 LEADER]: Becoming Leader. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:18:44.521548 16651 consensus_queue.cc:237] T ecc1e363230043fe9fef390f0b979f20 P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:44.522060 16611 leader_election.cc:290] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 1 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:18:44.523073 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8638ee363273426db1393dd81e5c7b72" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:18:44.524011 16299 raft_consensus.cc:3060] T 8638ee363273426db1393dd81e5c7b72 P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:44.526682 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8638ee363273426db1393dd81e5c7b72" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
W20260709 13:18:44.532768 16191 leader_election.cc:343] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:18:44.539225 16225 raft_consensus.cc:2468] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 1.
I20260709 13:18:44.537339 16295 raft_consensus.cc:2468] T 08b7d1ab519044eda2d5e04a07242f25 P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 1.
I20260709 13:18:44.547430 16347 raft_consensus.cc:359] T 08c74490c40243a88e62d1760b5cd814 P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:44.551733 16347 raft_consensus.cc:385] T 08c74490c40243a88e62d1760b5cd814 P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:44.552130 16347 raft_consensus.cc:740] T 08c74490c40243a88e62d1760b5cd814 P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Initialized, Role: FOLLOWER
I20260709 13:18:44.549958 16116 leader_election.cc:304] T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
I20260709 13:18:44.561658 16347 consensus_queue.cc:260] T 08c74490c40243a88e62d1760b5cd814 P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:44.560957 16520 raft_consensus.cc:2804] T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:18:44.564924 16520 raft_consensus.cc:697] T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b [term 1 LEADER]: Becoming Leader. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:18:44.566763 16299 raft_consensus.cc:2468] T 8638ee363273426db1393dd81e5c7b72 P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 1.
I20260709 13:18:44.579614 16347 ts_tablet_manager.cc:1434] T 08c74490c40243a88e62d1760b5cd814 P 1ca2725d290e47d68b1a4b609c9816ac: Time spent starting tablet: real 0.074s	user 0.003s	sys 0.005s
I20260709 13:18:44.565987 16520 consensus_queue.cc:237] T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:44.590005 16347 tablet_bootstrap.cc:492] T 31969392ccdb43b491911a319e8bc7e1 P 1ca2725d290e47d68b1a4b609c9816ac: Bootstrap starting.
I20260709 13:18:44.597193 16342 tablet_bootstrap.cc:492] T 57a744f5aaf944678a63469cbf514015 P 90f3da57796b488c9087a2f523adf045: No bootstrap required, opened a new log
I20260709 13:18:44.597750 16342 ts_tablet_manager.cc:1403] T 57a744f5aaf944678a63469cbf514015 P 90f3da57796b488c9087a2f523adf045: Time spent bootstrapping tablet: real 0.254s	user 0.009s	sys 0.015s
I20260709 13:18:44.599156 16347 tablet_bootstrap.cc:654] T 31969392ccdb43b491911a319e8bc7e1 P 1ca2725d290e47d68b1a4b609c9816ac: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:44.604525 16342 raft_consensus.cc:359] T 57a744f5aaf944678a63469cbf514015 P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:44.605465 16342 raft_consensus.cc:385] T 57a744f5aaf944678a63469cbf514015 P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:44.605803 16342 raft_consensus.cc:740] T 57a744f5aaf944678a63469cbf514015 P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Initialized, Role: FOLLOWER
I20260709 13:18:44.606575 16342 consensus_queue.cc:260] T 57a744f5aaf944678a63469cbf514015 P 90f3da57796b488c9087a2f523adf045 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:44.609058 16342 ts_tablet_manager.cc:1434] T 57a744f5aaf944678a63469cbf514015 P 90f3da57796b488c9087a2f523adf045: Time spent starting tablet: real 0.011s	user 0.007s	sys 0.002s
I20260709 13:18:44.610062 16342 tablet_bootstrap.cc:492] T 8d4ab870a05347868956aa8b841b2981 P 90f3da57796b488c9087a2f523adf045: Bootstrap starting.
W20260709 13:18:44.611510 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 43612.8us
I20260709 13:18:44.614233 16190 leader_election.cc:304] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 647a343a542d4a139fe106c19095129b
I20260709 13:18:44.615000 16608 raft_consensus.cc:2804] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:18:44.615608 16608 raft_consensus.cc:697] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [term 1 LEADER]: Becoming Leader. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:18:44.616645 16608 consensus_queue.cc:237] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:44.623483 16340 tablet_bootstrap.cc:492] T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b: No bootstrap required, opened a new log
I20260709 13:18:44.624015 16340 ts_tablet_manager.cc:1403] T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b: Time spent bootstrapping tablet: real 0.242s	user 0.003s	sys 0.014s
I20260709 13:18:44.626989 16340 raft_consensus.cc:359] T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:44.627737 16340 raft_consensus.cc:385] T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:44.628052 16340 raft_consensus.cc:740] T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Initialized, Role: FOLLOWER
I20260709 13:18:44.628693 16340 consensus_queue.cc:260] T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:44.637290 16632 raft_consensus.cc:493] T e5ad4eb281564936a59f41e387a47156 P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:44.637815 16632 raft_consensus.cc:515] T e5ad4eb281564936a59f41e387a47156 P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:44.639921 16632 leader_election.cc:290] T e5ad4eb281564936a59f41e387a47156 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:18:44.640586 16145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5ad4eb281564936a59f41e387a47156" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
W20260709 13:18:44.641911 16265 leader_election.cc:343] T e5ad4eb281564936a59f41e387a47156 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:18:44.643246 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5ad4eb281564936a59f41e387a47156" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
W20260709 13:18:44.644610 16265 leader_election.cc:343] T e5ad4eb281564936a59f41e387a47156 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:18:44.644989 16265 leader_election.cc:304] T e5ad4eb281564936a59f41e387a47156 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:18:44.646013 16632 raft_consensus.cc:2749] T e5ad4eb281564936a59f41e387a47156 P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:18:44.650136 16340 ts_tablet_manager.cc:1434] T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b: Time spent starting tablet: real 0.026s	user 0.007s	sys 0.000s
I20260709 13:18:44.651242 16340 tablet_bootstrap.cc:492] T 08c74490c40243a88e62d1760b5cd814 P 647a343a542d4a139fe106c19095129b: Bootstrap starting.
I20260709 13:18:44.653220 16661 raft_consensus.cc:493] T 08c74490c40243a88e62d1760b5cd814 P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:44.653745 16661 raft_consensus.cc:515] T 08c74490c40243a88e62d1760b5cd814 P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:44.655982 16661 leader_election.cc:290] T 08c74490c40243a88e62d1760b5cd814 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:18:44.656852 16342 tablet_bootstrap.cc:654] T 8d4ab870a05347868956aa8b841b2981 P 90f3da57796b488c9087a2f523adf045: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:44.657835 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c74490c40243a88e62d1760b5cd814" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:18:44.658598 16297 raft_consensus.cc:2468] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 0.
I20260709 13:18:44.659740 16190 leader_election.cc:304] T 08c74490c40243a88e62d1760b5cd814 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:18:44.661355 16632 consensus_queue.cc:1048] T 1df93779209748b9860f6d893dd3de53 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 13:18:44.664523 16265 consensus_peers.cc:597] T 1df93779209748b9860f6d893dd3de53 P 90f3da57796b488c9087a2f523adf045 -> Peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943): Couldn't send request to peer 1ca2725d290e47d68b1a4b609c9816ac. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:18:44.665665 16145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c74490c40243a88e62d1760b5cd814" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
W20260709 13:18:44.667476 16191 leader_election.cc:343] T 08c74490c40243a88e62d1760b5cd814 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:18:44.667953 16661 raft_consensus.cc:2804] T 08c74490c40243a88e62d1760b5cd814 P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:18:44.668551 16342 tablet_bootstrap.cc:492] T 8d4ab870a05347868956aa8b841b2981 P 90f3da57796b488c9087a2f523adf045: No bootstrap required, opened a new log
I20260709 13:18:44.667806 16048 catalog_manager.cc:5697] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac reported cstate change: term changed from 0 to 1, leader changed from <none> to 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130). New cstate: current_term: 1 leader_uuid: "1ca2725d290e47d68b1a4b609c9816ac" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:44.668815 16661 raft_consensus.cc:493] T 08c74490c40243a88e62d1760b5cd814 P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:44.669080 16342 ts_tablet_manager.cc:1403] T 8d4ab870a05347868956aa8b841b2981 P 90f3da57796b488c9087a2f523adf045: Time spent bootstrapping tablet: real 0.059s	user 0.009s	sys 0.009s
I20260709 13:18:44.669377 16661 raft_consensus.cc:3060] T 08c74490c40243a88e62d1760b5cd814 P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:44.671314 16340 tablet_bootstrap.cc:654] T 08c74490c40243a88e62d1760b5cd814 P 647a343a542d4a139fe106c19095129b: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:44.672133 16342 raft_consensus.cc:359] T 8d4ab870a05347868956aa8b841b2981 P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:44.672789 16342 raft_consensus.cc:385] T 8d4ab870a05347868956aa8b841b2981 P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:44.673060 16342 raft_consensus.cc:740] T 8d4ab870a05347868956aa8b841b2981 P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Initialized, Role: FOLLOWER
I20260709 13:18:44.673815 16342 consensus_queue.cc:260] T 8d4ab870a05347868956aa8b841b2981 P 90f3da57796b488c9087a2f523adf045 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:44.676139 16342 ts_tablet_manager.cc:1434] T 8d4ab870a05347868956aa8b841b2981 P 90f3da57796b488c9087a2f523adf045: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.001s
I20260709 13:18:44.676981 16342 tablet_bootstrap.cc:492] T c9944d5e4b8c4109af9955c4f08c9095 P 90f3da57796b488c9087a2f523adf045: Bootstrap starting.
I20260709 13:18:44.682821 16342 tablet_bootstrap.cc:654] T c9944d5e4b8c4109af9955c4f08c9095 P 90f3da57796b488c9087a2f523adf045: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:44.684754 16661 raft_consensus.cc:515] T 08c74490c40243a88e62d1760b5cd814 P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:44.687930 16145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c74490c40243a88e62d1760b5cd814" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:18:44.689028 16342 tablet_bootstrap.cc:492] T c9944d5e4b8c4109af9955c4f08c9095 P 90f3da57796b488c9087a2f523adf045: No bootstrap required, opened a new log
I20260709 13:18:44.689456 16342 ts_tablet_manager.cc:1403] T c9944d5e4b8c4109af9955c4f08c9095 P 90f3da57796b488c9087a2f523adf045: Time spent bootstrapping tablet: real 0.013s	user 0.009s	sys 0.000s
I20260709 13:18:44.688081 16340 tablet_bootstrap.cc:492] T 08c74490c40243a88e62d1760b5cd814 P 647a343a542d4a139fe106c19095129b: No bootstrap required, opened a new log
W20260709 13:18:44.689363 16191 leader_election.cc:343] T 08c74490c40243a88e62d1760b5cd814 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:18:44.690347 16340 ts_tablet_manager.cc:1403] T 08c74490c40243a88e62d1760b5cd814 P 647a343a542d4a139fe106c19095129b: Time spent bootstrapping tablet: real 0.039s	user 0.010s	sys 0.004s
I20260709 13:18:44.691704 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c74490c40243a88e62d1760b5cd814" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:18:44.663540 16047 catalog_manager.cc:5697] T ecc1e363230043fe9fef390f0b979f20 P 647a343a542d4a139fe106c19095129b reported cstate change: term changed from 0 to 1, leader changed from <none> to 647a343a542d4a139fe106c19095129b (127.14.177.129). New cstate: current_term: 1 leader_uuid: "647a343a542d4a139fe106c19095129b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:44.692291 16297 raft_consensus.cc:3060] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:44.692430 16342 raft_consensus.cc:359] T c9944d5e4b8c4109af9955c4f08c9095 P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:44.693106 16342 raft_consensus.cc:385] T c9944d5e4b8c4109af9955c4f08c9095 P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:44.693360 16342 raft_consensus.cc:740] T c9944d5e4b8c4109af9955c4f08c9095 P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Initialized, Role: FOLLOWER
I20260709 13:18:44.694078 16342 consensus_queue.cc:260] T c9944d5e4b8c4109af9955c4f08c9095 P 90f3da57796b488c9087a2f523adf045 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:44.696907 16342 ts_tablet_manager.cc:1434] T c9944d5e4b8c4109af9955c4f08c9095 P 90f3da57796b488c9087a2f523adf045: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.001s
I20260709 13:18:44.699370 16342 tablet_bootstrap.cc:492] T 26354361e9b044cea1c2b86c4882fe92 P 90f3da57796b488c9087a2f523adf045: Bootstrap starting.
I20260709 13:18:44.699632 16520 raft_consensus.cc:493] T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:44.700093 16520 raft_consensus.cc:515] T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:44.700826 16347 tablet_bootstrap.cc:492] T 31969392ccdb43b491911a319e8bc7e1 P 1ca2725d290e47d68b1a4b609c9816ac: No bootstrap required, opened a new log
I20260709 13:18:44.701247 16347 ts_tablet_manager.cc:1403] T 31969392ccdb43b491911a319e8bc7e1 P 1ca2725d290e47d68b1a4b609c9816ac: Time spent bootstrapping tablet: real 0.111s	user 0.015s	sys 0.003s
I20260709 13:18:44.697814 16661 leader_election.cc:290] T 08c74490c40243a88e62d1760b5cd814 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 1 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:18:44.702183 16520 leader_election.cc:290] T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:18:44.704063 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31969392ccdb43b491911a319e8bc7e1" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
W20260709 13:18:44.705542 16116 leader_election.cc:343] T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:18:44.705624 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31969392ccdb43b491911a319e8bc7e1" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:18:44.706274 16300 raft_consensus.cc:2468] T 31969392ccdb43b491911a319e8bc7e1 P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 0.
I20260709 13:18:44.706305 16347 raft_consensus.cc:359] T 31969392ccdb43b491911a319e8bc7e1 P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:44.707036 16347 raft_consensus.cc:385] T 31969392ccdb43b491911a319e8bc7e1 P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:44.707353 16347 raft_consensus.cc:740] T 31969392ccdb43b491911a319e8bc7e1 P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Initialized, Role: FOLLOWER
I20260709 13:18:44.707631 16297 raft_consensus.cc:2468] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 1.
I20260709 13:18:44.709304 16190 leader_election.cc:304] T 08c74490c40243a88e62d1760b5cd814 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 647a343a542d4a139fe106c19095129b
I20260709 13:18:44.711921 16342 tablet_bootstrap.cc:654] T 26354361e9b044cea1c2b86c4882fe92 P 90f3da57796b488c9087a2f523adf045: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:44.713413 16115 leader_election.cc:304] T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac
I20260709 13:18:44.715641 16634 raft_consensus.cc:493] T 57a744f5aaf944678a63469cbf514015 P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:44.716276 16634 raft_consensus.cc:515] T 57a744f5aaf944678a63469cbf514015 P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:44.720896 16608 raft_consensus.cc:2804] T 08c74490c40243a88e62d1760b5cd814 P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:18:44.721453 16608 raft_consensus.cc:697] T 08c74490c40243a88e62d1760b5cd814 P 1ca2725d290e47d68b1a4b609c9816ac [term 1 LEADER]: Becoming Leader. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:18:44.704774 16340 raft_consensus.cc:359] T 08c74490c40243a88e62d1760b5cd814 P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:44.722432 16340 raft_consensus.cc:385] T 08c74490c40243a88e62d1760b5cd814 P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:44.722239 16608 consensus_queue.cc:237] T 08c74490c40243a88e62d1760b5cd814 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:44.723156 16347 consensus_queue.cc:260] T 31969392ccdb43b491911a319e8bc7e1 P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:44.722688 16340 raft_consensus.cc:740] T 08c74490c40243a88e62d1760b5cd814 P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Initialized, Role: FOLLOWER
I20260709 13:18:44.724861 16631 raft_consensus.cc:2804] T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:18:44.725360 16631 raft_consensus.cc:493] T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:44.725770 16631 raft_consensus.cc:3060] T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:44.728909 16340 consensus_queue.cc:260] T 08c74490c40243a88e62d1760b5cd814 P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:44.731560 16658 raft_consensus.cc:493] T 31969392ccdb43b491911a319e8bc7e1 P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:44.731984 16342 tablet_bootstrap.cc:492] T 26354361e9b044cea1c2b86c4882fe92 P 90f3da57796b488c9087a2f523adf045: No bootstrap required, opened a new log
I20260709 13:18:44.732056 16658 raft_consensus.cc:515] T 31969392ccdb43b491911a319e8bc7e1 P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:44.732998 16342 ts_tablet_manager.cc:1403] T 26354361e9b044cea1c2b86c4882fe92 P 90f3da57796b488c9087a2f523adf045: Time spent bootstrapping tablet: real 0.035s	user 0.017s	sys 0.009s
I20260709 13:18:44.735056 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31969392ccdb43b491911a319e8bc7e1" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:18:44.735464 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31969392ccdb43b491911a319e8bc7e1" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:18:44.735977 16300 raft_consensus.cc:2468] T 31969392ccdb43b491911a319e8bc7e1 P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 0.
I20260709 13:18:44.737108 16190 leader_election.cc:304] T 31969392ccdb43b491911a319e8bc7e1 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:18:44.731031 16340 ts_tablet_manager.cc:1434] T 08c74490c40243a88e62d1760b5cd814 P 647a343a542d4a139fe106c19095129b: Time spent starting tablet: real 0.040s	user 0.007s	sys 0.000s
I20260709 13:18:44.718884 16634 leader_election.cc:290] T 57a744f5aaf944678a63469cbf514015 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:18:44.737708 16342 raft_consensus.cc:359] T 26354361e9b044cea1c2b86c4882fe92 P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:44.740931 16342 raft_consensus.cc:385] T 26354361e9b044cea1c2b86c4882fe92 P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:44.741400 16342 raft_consensus.cc:740] T 26354361e9b044cea1c2b86c4882fe92 P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Initialized, Role: FOLLOWER
I20260709 13:18:44.742100 16661 raft_consensus.cc:2804] T 31969392ccdb43b491911a319e8bc7e1 P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:18:44.742515 16661 raft_consensus.cc:493] T 31969392ccdb43b491911a319e8bc7e1 P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:44.742901 16661 raft_consensus.cc:3060] T 31969392ccdb43b491911a319e8bc7e1 P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:44.742550 16342 consensus_queue.cc:260] T 26354361e9b044cea1c2b86c4882fe92 P 90f3da57796b488c9087a2f523adf045 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:44.723719 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57a744f5aaf944678a63469cbf514015" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:18:44.744388 16658 leader_election.cc:290] T 31969392ccdb43b491911a319e8bc7e1 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
W20260709 13:18:44.750588 16265 leader_election.cc:343] T 57a744f5aaf944678a63469cbf514015 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:18:44.752779 16342 ts_tablet_manager.cc:1434] T 26354361e9b044cea1c2b86c4882fe92 P 90f3da57796b488c9087a2f523adf045: Time spent starting tablet: real 0.019s	user 0.004s	sys 0.004s
I20260709 13:18:44.754441 16342 tablet_bootstrap.cc:492] T 84f2da98864b4425bf7c0055951fdd4b P 90f3da57796b488c9087a2f523adf045: Bootstrap starting.
I20260709 13:18:44.754302 16631 raft_consensus.cc:515] T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:44.759690 16150 raft_consensus.cc:2393] T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 1: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
I20260709 13:18:44.760958 16145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57a744f5aaf944678a63469cbf514015" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:18:44.762778 16340 tablet_bootstrap.cc:492] T 8638ee363273426db1393dd81e5c7b72 P 647a343a542d4a139fe106c19095129b: Bootstrap starting.
I20260709 13:18:44.764542 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31969392ccdb43b491911a319e8bc7e1" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:18:44.765103 16297 raft_consensus.cc:3060] T 31969392ccdb43b491911a319e8bc7e1 P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:18:44.766589 16265 leader_election.cc:343] T 57a744f5aaf944678a63469cbf514015 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:18:44.767782 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31969392ccdb43b491911a319e8bc7e1" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:18:44.768539 16347 ts_tablet_manager.cc:1434] T 31969392ccdb43b491911a319e8bc7e1 P 1ca2725d290e47d68b1a4b609c9816ac: Time spent starting tablet: real 0.067s	user 0.028s	sys 0.016s
I20260709 13:18:44.767151 16265 leader_election.cc:304] T 57a744f5aaf944678a63469cbf514015 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:18:44.769675 16347 tablet_bootstrap.cc:492] T 1df93779209748b9860f6d893dd3de53 P 1ca2725d290e47d68b1a4b609c9816ac: Bootstrap starting.
I20260709 13:18:44.769791 16629 raft_consensus.cc:2749] T 57a744f5aaf944678a63469cbf514015 P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:18:44.770496 16661 raft_consensus.cc:515] T 31969392ccdb43b491911a319e8bc7e1 P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:44.772444 16661 leader_election.cc:290] T 31969392ccdb43b491911a319e8bc7e1 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 1 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:18:44.774571 16145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31969392ccdb43b491911a319e8bc7e1" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:18:44.774662 16297 raft_consensus.cc:2468] T 31969392ccdb43b491911a319e8bc7e1 P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 1.
I20260709 13:18:44.775463 16145 raft_consensus.cc:2393] T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 1: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
I20260709 13:18:44.776468 16115 leader_election.cc:304] T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:18:44.777547 16520 raft_consensus.cc:2804] T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:18:44.777987 16520 raft_consensus.cc:697] T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b [term 1 LEADER]: Becoming Leader. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:18:44.777874 16045 catalog_manager.cc:5697] T 08c74490c40243a88e62d1760b5cd814 P 1ca2725d290e47d68b1a4b609c9816ac reported cstate change: term changed from 0 to 1, leader changed from <none> to 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130). New cstate: current_term: 1 leader_uuid: "1ca2725d290e47d68b1a4b609c9816ac" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:44.782776 16347 tablet_bootstrap.cc:654] T 1df93779209748b9860f6d893dd3de53 P 1ca2725d290e47d68b1a4b609c9816ac: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:44.783483 16226 raft_consensus.cc:2393] T 31969392ccdb43b491911a319e8bc7e1 P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 1: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
I20260709 13:18:44.778787 16520 consensus_queue.cc:237] T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:44.779317 16631 leader_election.cc:290] T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 1 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
W20260709 13:18:44.791973 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 54934.4us
I20260709 13:18:44.790647 16340 tablet_bootstrap.cc:654] T 8638ee363273426db1393dd81e5c7b72 P 647a343a542d4a139fe106c19095129b: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:44.824520 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31969392ccdb43b491911a319e8bc7e1" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:18:44.825274 16300 raft_consensus.cc:2393] T 31969392ccdb43b491911a319e8bc7e1 P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 1: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
I20260709 13:18:44.823412 16047 catalog_manager.cc:5697] T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b reported cstate change: term changed from 0 to 1, leader changed from <none> to 647a343a542d4a139fe106c19095129b (127.14.177.129). New cstate: current_term: 1 leader_uuid: "647a343a542d4a139fe106c19095129b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:44.828673 16347 tablet_bootstrap.cc:492] T 1df93779209748b9860f6d893dd3de53 P 1ca2725d290e47d68b1a4b609c9816ac: No bootstrap required, opened a new log
I20260709 13:18:44.829134 16347 ts_tablet_manager.cc:1403] T 1df93779209748b9860f6d893dd3de53 P 1ca2725d290e47d68b1a4b609c9816ac: Time spent bootstrapping tablet: real 0.060s	user 0.016s	sys 0.006s
W20260709 13:18:44.831339 16097 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:18:44.831555 16342 tablet_bootstrap.cc:654] T 84f2da98864b4425bf7c0055951fdd4b P 90f3da57796b488c9087a2f523adf045: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:44.841205 16347 raft_consensus.cc:359] T 1df93779209748b9860f6d893dd3de53 P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:44.842479 16347 raft_consensus.cc:385] T 1df93779209748b9860f6d893dd3de53 P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:44.842952 16347 raft_consensus.cc:740] T 1df93779209748b9860f6d893dd3de53 P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Initialized, Role: FOLLOWER
I20260709 13:18:44.848798 16190 leader_election.cc:304] T 31969392ccdb43b491911a319e8bc7e1 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:18:44.850764 16658 raft_consensus.cc:2749] T 31969392ccdb43b491911a319e8bc7e1 P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:18:44.853179 16347 consensus_queue.cc:260] T 1df93779209748b9860f6d893dd3de53 P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:44.885766 16634 raft_consensus.cc:493] T c9944d5e4b8c4109af9955c4f08c9095 P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:44.886247 16634 raft_consensus.cc:515] T c9944d5e4b8c4109af9955c4f08c9095 P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:44.888211 16634 leader_election.cc:290] T c9944d5e4b8c4109af9955c4f08c9095 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:18:44.890030 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9944d5e4b8c4109af9955c4f08c9095" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
W20260709 13:18:44.891598 16265 leader_election.cc:343] T c9944d5e4b8c4109af9955c4f08c9095 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:18:44.891618 16145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9944d5e4b8c4109af9955c4f08c9095" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
W20260709 13:18:44.893038 16265 leader_election.cc:343] T c9944d5e4b8c4109af9955c4f08c9095 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:18:44.893443 16265 leader_election.cc:304] T c9944d5e4b8c4109af9955c4f08c9095 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:18:44.893452 16340 tablet_bootstrap.cc:492] T 8638ee363273426db1393dd81e5c7b72 P 647a343a542d4a139fe106c19095129b: No bootstrap required, opened a new log
I20260709 13:18:44.894569 16347 ts_tablet_manager.cc:1434] T 1df93779209748b9860f6d893dd3de53 P 1ca2725d290e47d68b1a4b609c9816ac: Time spent starting tablet: real 0.065s	user 0.010s	sys 0.001s
I20260709 13:18:44.894888 16340 ts_tablet_manager.cc:1403] T 8638ee363273426db1393dd81e5c7b72 P 647a343a542d4a139fe106c19095129b: Time spent bootstrapping tablet: real 0.132s	user 0.013s	sys 0.014s
I20260709 13:18:44.895961 16629 raft_consensus.cc:2749] T c9944d5e4b8c4109af9955c4f08c9095 P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:18:44.896747 16629 raft_consensus.cc:493] T 26354361e9b044cea1c2b86c4882fe92 P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:44.897387 16629 raft_consensus.cc:515] T 26354361e9b044cea1c2b86c4882fe92 P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:44.899451 16347 tablet_bootstrap.cc:492] T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac: Bootstrap starting.
I20260709 13:18:44.900514 16629 leader_election.cc:290] T 26354361e9b044cea1c2b86c4882fe92 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:18:44.905972 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26354361e9b044cea1c2b86c4882fe92" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:18:44.907716 16145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26354361e9b044cea1c2b86c4882fe92" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
W20260709 13:18:44.909652 16265 leader_election.cc:343] T 26354361e9b044cea1c2b86c4882fe92 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:18:44.911094 16521 consensus_queue.cc:1048] T ecc1e363230043fe9fef390f0b979f20 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 13:18:44.911638 16265 leader_election.cc:343] T 26354361e9b044cea1c2b86c4882fe92 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:18:44.912249 16265 leader_election.cc:304] T 26354361e9b044cea1c2b86c4882fe92 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:18:44.913718 16629 raft_consensus.cc:2749] T 26354361e9b044cea1c2b86c4882fe92 P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:18:44.921169 16676 raft_consensus.cc:493] T 1df93779209748b9860f6d893dd3de53 P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:44.916886 16045 catalog_manager.cc:5697] T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b reported cstate change: term changed from 0 to 1, leader changed from <none> to 647a343a542d4a139fe106c19095129b (127.14.177.129). New cstate: current_term: 1 leader_uuid: "647a343a542d4a139fe106c19095129b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:44.923990 16342 tablet_bootstrap.cc:492] T 84f2da98864b4425bf7c0055951fdd4b P 90f3da57796b488c9087a2f523adf045: No bootstrap required, opened a new log
I20260709 13:18:44.923199 16676 raft_consensus.cc:515] T 1df93779209748b9860f6d893dd3de53 P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:44.924619 16342 ts_tablet_manager.cc:1403] T 84f2da98864b4425bf7c0055951fdd4b P 90f3da57796b488c9087a2f523adf045: Time spent bootstrapping tablet: real 0.171s	user 0.020s	sys 0.013s
I20260709 13:18:44.927650 16347 tablet_bootstrap.cc:654] T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:44.928629 16676 leader_election.cc:290] T 1df93779209748b9860f6d893dd3de53 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:18:44.929332 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1df93779209748b9860f6d893dd3de53" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:18:44.936875 16342 raft_consensus.cc:359] T 84f2da98864b4425bf7c0055951fdd4b P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:44.937259 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1df93779209748b9860f6d893dd3de53" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:18:44.937876 16342 raft_consensus.cc:385] T 84f2da98864b4425bf7c0055951fdd4b P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:44.938222 16342 raft_consensus.cc:740] T 84f2da98864b4425bf7c0055951fdd4b P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Initialized, Role: FOLLOWER
I20260709 13:18:44.939071 16342 consensus_queue.cc:260] T 84f2da98864b4425bf7c0055951fdd4b P 90f3da57796b488c9087a2f523adf045 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:44.939299 16340 raft_consensus.cc:359] T 8638ee363273426db1393dd81e5c7b72 P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:44.940038 16340 raft_consensus.cc:385] T 8638ee363273426db1393dd81e5c7b72 P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:44.940343 16340 raft_consensus.cc:740] T 8638ee363273426db1393dd81e5c7b72 P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Initialized, Role: FOLLOWER
I20260709 13:18:44.945194 16342 ts_tablet_manager.cc:1434] T 84f2da98864b4425bf7c0055951fdd4b P 90f3da57796b488c9087a2f523adf045: Time spent starting tablet: real 0.020s	user 0.006s	sys 0.000s
I20260709 13:18:44.943683 16190 leader_election.cc:304] T 1df93779209748b9860f6d893dd3de53 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:18:44.946892 16611 raft_consensus.cc:2749] T 1df93779209748b9860f6d893dd3de53 P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:18:44.952103 16340 consensus_queue.cc:260] T 8638ee363273426db1393dd81e5c7b72 P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:44.954787 16340 ts_tablet_manager.cc:1434] T 8638ee363273426db1393dd81e5c7b72 P 647a343a542d4a139fe106c19095129b: Time spent starting tablet: real 0.059s	user 0.002s	sys 0.003s
I20260709 13:18:44.955474 16342 tablet_bootstrap.cc:492] T cdbd78acb4cf46298e0de8ba0d130e19 P 90f3da57796b488c9087a2f523adf045: Bootstrap starting.
I20260709 13:18:44.961484 16340 tablet_bootstrap.cc:492] T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b: Bootstrap starting.
I20260709 13:18:44.966424 16631 consensus_queue.cc:1048] T ecc1e363230043fe9fef390f0b979f20 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:18:44.973404 16342 tablet_bootstrap.cc:654] T cdbd78acb4cf46298e0de8ba0d130e19 P 90f3da57796b488c9087a2f523adf045: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:44.974205 16340 tablet_bootstrap.cc:654] T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:44.974619 16520 raft_consensus.cc:493] T 08c74490c40243a88e62d1760b5cd814 P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:44.975157 16520 raft_consensus.cc:515] T 08c74490c40243a88e62d1760b5cd814 P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:44.977383 16520 leader_election.cc:290] T 08c74490c40243a88e62d1760b5cd814 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:18:44.978852 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c74490c40243a88e62d1760b5cd814" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:18:44.979803 16297 raft_consensus.cc:2393] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 1: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
I20260709 13:18:44.980903 16223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c74490c40243a88e62d1760b5cd814" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:18:44.987064 16342 tablet_bootstrap.cc:492] T cdbd78acb4cf46298e0de8ba0d130e19 P 90f3da57796b488c9087a2f523adf045: No bootstrap required, opened a new log
I20260709 13:18:44.986974 16115 leader_election.cc:304] T 08c74490c40243a88e62d1760b5cd814 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:18:44.987668 16342 ts_tablet_manager.cc:1403] T cdbd78acb4cf46298e0de8ba0d130e19 P 90f3da57796b488c9087a2f523adf045: Time spent bootstrapping tablet: real 0.032s	user 0.014s	sys 0.003s
I20260709 13:18:44.988135 16521 raft_consensus.cc:3060] T 08c74490c40243a88e62d1760b5cd814 P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:44.990586 16342 raft_consensus.cc:359] T cdbd78acb4cf46298e0de8ba0d130e19 P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:44.994261 16342 raft_consensus.cc:385] T cdbd78acb4cf46298e0de8ba0d130e19 P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:44.994551 16342 raft_consensus.cc:740] T cdbd78acb4cf46298e0de8ba0d130e19 P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Initialized, Role: FOLLOWER
I20260709 13:18:44.995340 16342 consensus_queue.cc:260] T cdbd78acb4cf46298e0de8ba0d130e19 P 90f3da57796b488c9087a2f523adf045 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:44.996493 16521 raft_consensus.cc:2749] T 08c74490c40243a88e62d1760b5cd814 P 647a343a542d4a139fe106c19095129b [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:18:44.997915 16651 raft_consensus.cc:493] T 8638ee363273426db1393dd81e5c7b72 P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:44.998364 16651 raft_consensus.cc:515] T 8638ee363273426db1393dd81e5c7b72 P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:44.999545 16347 tablet_bootstrap.cc:492] T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac: No bootstrap required, opened a new log
I20260709 13:18:45.000371 16347 ts_tablet_manager.cc:1403] T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac: Time spent bootstrapping tablet: real 0.101s	user 0.009s	sys 0.005s
I20260709 13:18:45.000515 16651 leader_election.cc:290] T 8638ee363273426db1393dd81e5c7b72 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:18:45.003742 16342 ts_tablet_manager.cc:1434] T cdbd78acb4cf46298e0de8ba0d130e19 P 90f3da57796b488c9087a2f523adf045: Time spent starting tablet: real 0.016s	user 0.007s	sys 0.000s
I20260709 13:18:45.004840 16342 tablet_bootstrap.cc:492] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045: Bootstrap starting.
I20260709 13:18:45.005530 16629 raft_consensus.cc:493] T cdbd78acb4cf46298e0de8ba0d130e19 P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:45.006018 16629 raft_consensus.cc:515] T cdbd78acb4cf46298e0de8ba0d130e19 P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:45.004998 16347 raft_consensus.cc:359] T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:45.007256 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8638ee363273426db1393dd81e5c7b72" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:18:45.007496 16347 raft_consensus.cc:385] T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:45.008024 16629 leader_election.cc:290] T cdbd78acb4cf46298e0de8ba0d130e19 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:18:45.009950 16145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdbd78acb4cf46298e0de8ba0d130e19" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:18:45.007956 16347 raft_consensus.cc:740] T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Initialized, Role: FOLLOWER
W20260709 13:18:45.011586 16265 leader_election.cc:343] T cdbd78acb4cf46298e0de8ba0d130e19 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:18:45.011533 16347 consensus_queue.cc:260] T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:45.012591 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdbd78acb4cf46298e0de8ba0d130e19" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
W20260709 13:18:45.014000 16265 leader_election.cc:343] T cdbd78acb4cf46298e0de8ba0d130e19 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:18:45.014397 16265 leader_election.cc:304] T cdbd78acb4cf46298e0de8ba0d130e19 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:18:45.015072 16629 raft_consensus.cc:2749] T cdbd78acb4cf46298e0de8ba0d130e19 P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:18:45.019562 16342 tablet_bootstrap.cc:654] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:45.022164 16340 tablet_bootstrap.cc:492] T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b: No bootstrap required, opened a new log
I20260709 13:18:45.022799 16340 ts_tablet_manager.cc:1403] T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b: Time spent bootstrapping tablet: real 0.062s	user 0.008s	sys 0.012s
I20260709 13:18:45.025924 16347 ts_tablet_manager.cc:1434] T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac: Time spent starting tablet: real 0.025s	user 0.004s	sys 0.008s
I20260709 13:18:45.026643 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8638ee363273426db1393dd81e5c7b72" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:18:45.027395 16347 tablet_bootstrap.cc:492] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac: Bootstrap starting.
I20260709 13:18:45.027751 16297 raft_consensus.cc:2393] T 8638ee363273426db1393dd81e5c7b72 P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 1: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
I20260709 13:18:45.030298 16115 leader_election.cc:304] T 8638ee363273426db1393dd81e5c7b72 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:18:45.033771 16651 raft_consensus.cc:3060] T 8638ee363273426db1393dd81e5c7b72 P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:45.037717 16340 raft_consensus.cc:359] T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:45.038666 16340 raft_consensus.cc:385] T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:45.039096 16340 raft_consensus.cc:740] T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Initialized, Role: FOLLOWER
I20260709 13:18:45.039916 16340 consensus_queue.cc:260] T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:45.042446 16340 ts_tablet_manager.cc:1434] T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b: Time spent starting tablet: real 0.019s	user 0.008s	sys 0.000s
I20260709 13:18:45.049425 16347 tablet_bootstrap.cc:654] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:45.050025 16651 raft_consensus.cc:2749] T 8638ee363273426db1393dd81e5c7b72 P 647a343a542d4a139fe106c19095129b [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:18:45.050634 16340 tablet_bootstrap.cc:492] T 8d4ab870a05347868956aa8b841b2981 P 647a343a542d4a139fe106c19095129b: Bootstrap starting.
I20260709 13:18:45.053117 16342 tablet_bootstrap.cc:492] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045: No bootstrap required, opened a new log
I20260709 13:18:45.053550 16342 ts_tablet_manager.cc:1403] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045: Time spent bootstrapping tablet: real 0.049s	user 0.012s	sys 0.004s
I20260709 13:18:45.056197 16342 raft_consensus.cc:359] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:45.056845 16342 raft_consensus.cc:385] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:45.057124 16342 raft_consensus.cc:740] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Initialized, Role: FOLLOWER
I20260709 13:18:45.057895 16342 consensus_queue.cc:260] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:45.063467 16340 tablet_bootstrap.cc:654] T 8d4ab870a05347868956aa8b841b2981 P 647a343a542d4a139fe106c19095129b: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:45.063920 16225 raft_consensus.cc:3060] T 1df93779209748b9860f6d893dd3de53 P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:45.078629 16342 ts_tablet_manager.cc:1434] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045: Time spent starting tablet: real 0.025s	user 0.006s	sys 0.001s
I20260709 13:18:45.079754 16342 tablet_bootstrap.cc:492] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045: Bootstrap starting.
I20260709 13:18:45.080156 16347 tablet_bootstrap.cc:492] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac: No bootstrap required, opened a new log
I20260709 13:18:45.082787 16632 raft_consensus.cc:493] T 8d4ab870a05347868956aa8b841b2981 P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:45.086501 16632 raft_consensus.cc:515] T 8d4ab870a05347868956aa8b841b2981 P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:45.088757 16632 leader_election.cc:290] T 8d4ab870a05347868956aa8b841b2981 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:18:45.084924 16340 tablet_bootstrap.cc:492] T 8d4ab870a05347868956aa8b841b2981 P 647a343a542d4a139fe106c19095129b: No bootstrap required, opened a new log
I20260709 13:18:45.090526 16340 ts_tablet_manager.cc:1403] T 8d4ab870a05347868956aa8b841b2981 P 647a343a542d4a139fe106c19095129b: Time spent bootstrapping tablet: real 0.040s	user 0.009s	sys 0.006s
I20260709 13:18:45.091207 16223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d4ab870a05347868956aa8b841b2981" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:18:45.091853 16223 raft_consensus.cc:2468] T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 0.
I20260709 13:18:45.093184 16342 tablet_bootstrap.cc:654] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:45.093907 16340 raft_consensus.cc:359] T 8d4ab870a05347868956aa8b841b2981 P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:45.094733 16340 raft_consensus.cc:385] T 8d4ab870a05347868956aa8b841b2981 P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:45.093930 16265 leader_election.cc:304] T 8d4ab870a05347868956aa8b841b2981 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:18:45.095072 16340 raft_consensus.cc:740] T 8d4ab870a05347868956aa8b841b2981 P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Initialized, Role: FOLLOWER
I20260709 13:18:45.095960 16629 raft_consensus.cc:2804] T 8d4ab870a05347868956aa8b841b2981 P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:18:45.096370 16629 raft_consensus.cc:493] T 8d4ab870a05347868956aa8b841b2981 P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:45.096784 16629 raft_consensus.cc:3060] T 8d4ab870a05347868956aa8b841b2981 P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:45.096829 16340 consensus_queue.cc:260] T 8d4ab870a05347868956aa8b841b2981 P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:45.100289 16340 ts_tablet_manager.cc:1434] T 8d4ab870a05347868956aa8b841b2981 P 647a343a542d4a139fe106c19095129b: Time spent starting tablet: real 0.009s	user 0.005s	sys 0.003s
I20260709 13:18:45.101022 16347 ts_tablet_manager.cc:1403] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac: Time spent bootstrapping tablet: real 0.074s	user 0.023s	sys 0.012s
I20260709 13:18:45.105410 16145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d4ab870a05347868956aa8b841b2981" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:18:45.106076 16676 consensus_queue.cc:1048] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:18:45.106734 16347 raft_consensus.cc:359] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:45.107040 16145 raft_consensus.cc:2468] T 8d4ab870a05347868956aa8b841b2981 P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 0.
I20260709 13:18:45.107797 16347 raft_consensus.cc:385] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:45.120461 16347 raft_consensus.cc:740] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Initialized, Role: FOLLOWER
I20260709 13:18:45.121222 16347 consensus_queue.cc:260] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:45.124269 16347 ts_tablet_manager.cc:1434] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac: Time spent starting tablet: real 0.023s	user 0.007s	sys 0.000s
I20260709 13:18:45.125344 16347 tablet_bootstrap.cc:492] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac: Bootstrap starting.
I20260709 13:18:45.105613 16340 tablet_bootstrap.cc:492] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b: Bootstrap starting.
I20260709 13:18:45.107820 16631 consensus_queue.cc:1048] T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:18:45.139243 16342 tablet_bootstrap.cc:492] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045: No bootstrap required, opened a new log
I20260709 13:18:45.143356 16347 tablet_bootstrap.cc:654] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:45.139787 16342 ts_tablet_manager.cc:1403] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045: Time spent bootstrapping tablet: real 0.060s	user 0.019s	sys 0.018s
I20260709 13:18:45.164449 16340 tablet_bootstrap.cc:654] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:45.172761 16630 consensus_queue.cc:1048] T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:18:45.173993 16342 raft_consensus.cc:359] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:45.174748 16342 raft_consensus.cc:385] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:45.175046 16342 raft_consensus.cc:740] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Initialized, Role: FOLLOWER
I20260709 13:18:45.176465 16611 consensus_queue.cc:1048] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:18:45.154958 16629 raft_consensus.cc:515] T 8d4ab870a05347868956aa8b841b2981 P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:45.218974 16629 leader_election.cc:290] T 8d4ab870a05347868956aa8b841b2981 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 1 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:18:45.220438 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d4ab870a05347868956aa8b841b2981" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:18:45.221058 16150 raft_consensus.cc:3060] T 8d4ab870a05347868956aa8b841b2981 P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:45.175839 16342 consensus_queue.cc:260] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:45.209283 16347 tablet_bootstrap.cc:492] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac: No bootstrap required, opened a new log
I20260709 13:18:45.225250 16347 ts_tablet_manager.cc:1403] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac: Time spent bootstrapping tablet: real 0.100s	user 0.009s	sys 0.013s
I20260709 13:18:45.226383 16658 consensus_queue.cc:1048] T 08c74490c40243a88e62d1760b5cd814 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:18:45.230825 16347 raft_consensus.cc:359] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:45.232906 16347 raft_consensus.cc:385] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:45.232733 16629 raft_consensus.cc:493] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:45.233408 16347 raft_consensus.cc:740] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Initialized, Role: FOLLOWER
I20260709 13:18:45.233582 16629 raft_consensus.cc:515] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:45.234772 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d4ab870a05347868956aa8b841b2981" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:18:45.234606 16347 consensus_queue.cc:260] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:45.235301 16225 raft_consensus.cc:3060] T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:45.236264 16629 leader_election.cc:290] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:18:45.236533 16347 ts_tablet_manager.cc:1434] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.000s
I20260709 13:18:45.232779 16340 tablet_bootstrap.cc:492] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b: No bootstrap required, opened a new log
I20260709 13:18:45.237427 16340 ts_tablet_manager.cc:1403] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b: Time spent bootstrapping tablet: real 0.132s	user 0.015s	sys 0.012s
I20260709 13:18:45.237764 16347 tablet_bootstrap.cc:492] T e4dc6b0541ca431ebc757beac00521f1 P 1ca2725d290e47d68b1a4b609c9816ac: Bootstrap starting.
I20260709 13:18:45.240213 16342 ts_tablet_manager.cc:1434] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045: Time spent starting tablet: real 0.094s	user 0.014s	sys 0.011s
I20260709 13:18:45.241452 16342 tablet_bootstrap.cc:492] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 90f3da57796b488c9087a2f523adf045: Bootstrap starting.
I20260709 13:18:45.245272 16347 tablet_bootstrap.cc:654] T e4dc6b0541ca431ebc757beac00521f1 P 1ca2725d290e47d68b1a4b609c9816ac: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:45.244895 16651 consensus_queue.cc:1048] T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:18:45.246551 16225 raft_consensus.cc:2468] T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 1.
I20260709 13:18:45.285418 16342 tablet_bootstrap.cc:654] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 90f3da57796b488c9087a2f523adf045: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:45.253167 16148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e4bfb9090c8460eb14fa7e0d818e7f1" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:18:45.256661 16223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e4bfb9090c8460eb14fa7e0d818e7f1" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:18:45.288196 16658 consensus_queue.cc:1048] T 08c74490c40243a88e62d1760b5cd814 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:18:45.267515 16265 leader_election.cc:304] T 8d4ab870a05347868956aa8b841b2981 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:18:45.294790 16632 raft_consensus.cc:2804] T 8d4ab870a05347868956aa8b841b2981 P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Leader election won for term 1
W20260709 13:18:45.290974 16265 leader_election.cc:343] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:18:45.299695 16265 leader_election.cc:343] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:18:45.300263 16265 leader_election.cc:304] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:18:45.301296 16634 raft_consensus.cc:2749] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:18:45.249792 16150 raft_consensus.cc:2468] T 8d4ab870a05347868956aa8b841b2981 P 647a343a542d4a139fe106c19095129b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 1.
I20260709 13:18:45.277545 16651 consensus_queue.cc:1048] T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.030s
I20260709 13:18:45.312093 16632 raft_consensus.cc:697] T 8d4ab870a05347868956aa8b841b2981 P 90f3da57796b488c9087a2f523adf045 [term 1 LEADER]: Becoming Leader. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:18:45.248811 16340 raft_consensus.cc:359] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:45.313076 16632 consensus_queue.cc:237] T 8d4ab870a05347868956aa8b841b2981 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:45.314196 16340 raft_consensus.cc:385] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:45.314579 16340 raft_consensus.cc:740] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Initialized, Role: FOLLOWER
I20260709 13:18:45.316407 16340 consensus_queue.cc:260] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:45.320170 16340 ts_tablet_manager.cc:1434] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b: Time spent starting tablet: real 0.082s	user 0.007s	sys 0.003s
I20260709 13:18:45.322535 16629 raft_consensus.cc:493] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:45.335492 16342 tablet_bootstrap.cc:492] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 90f3da57796b488c9087a2f523adf045: No bootstrap required, opened a new log
I20260709 13:18:45.336845 16634 raft_consensus.cc:493] T 84f2da98864b4425bf7c0055951fdd4b P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:45.340109 16629 raft_consensus.cc:515] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:45.340778 16340 tablet_bootstrap.cc:492] T cdbd78acb4cf46298e0de8ba0d130e19 P 647a343a542d4a139fe106c19095129b: Bootstrap starting.
I20260709 13:18:45.340742 16634 raft_consensus.cc:515] T 84f2da98864b4425bf7c0055951fdd4b P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:45.343148 16634 leader_election.cc:290] T 84f2da98864b4425bf7c0055951fdd4b P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:18:45.343461 16342 ts_tablet_manager.cc:1403] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 90f3da57796b488c9087a2f523adf045: Time spent bootstrapping tablet: real 0.102s	user 0.014s	sys 0.023s
I20260709 13:18:45.343113 16629 leader_election.cc:290] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:18:45.339076 16347 tablet_bootstrap.cc:492] T e4dc6b0541ca431ebc757beac00521f1 P 1ca2725d290e47d68b1a4b609c9816ac: No bootstrap required, opened a new log
I20260709 13:18:45.344450 16347 ts_tablet_manager.cc:1403] T e4dc6b0541ca431ebc757beac00521f1 P 1ca2725d290e47d68b1a4b609c9816ac: Time spent bootstrapping tablet: real 0.107s	user 0.021s	sys 0.033s
I20260709 13:18:45.346102 16148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:18:45.347144 16347 raft_consensus.cc:359] T e4dc6b0541ca431ebc757beac00521f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:45.347613 16151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f2da98864b4425bf7c0055951fdd4b" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:18:45.347908 16340 tablet_bootstrap.cc:654] T cdbd78acb4cf46298e0de8ba0d130e19 P 647a343a542d4a139fe106c19095129b: Neither blocks nor log segments found. Creating new log.
W20260709 13:18:45.348405 16265 leader_election.cc:343] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:18:45.347934 16347 raft_consensus.cc:385] T e4dc6b0541ca431ebc757beac00521f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:45.349347 16347 raft_consensus.cc:740] T e4dc6b0541ca431ebc757beac00521f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Initialized, Role: FOLLOWER
I20260709 13:18:45.350046 16347 consensus_queue.cc:260] T e4dc6b0541ca431ebc757beac00521f1 P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:45.350383 16223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:18:45.350570 16342 raft_consensus.cc:359] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:45.351375 16342 raft_consensus.cc:385] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:45.351662 16342 raft_consensus.cc:740] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Initialized, Role: FOLLOWER
W20260709 13:18:45.352161 16265 leader_election.cc:343] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:18:45.352496 16342 consensus_queue.cc:260] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 90f3da57796b488c9087a2f523adf045 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:45.352814 16265 leader_election.cc:304] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:18:45.353818 16629 raft_consensus.cc:2749] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:18:45.354590 16265 leader_election.cc:343] T 84f2da98864b4425bf7c0055951fdd4b P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:18:45.352154 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f2da98864b4425bf7c0055951fdd4b" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:18:45.355890 16342 ts_tablet_manager.cc:1434] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 90f3da57796b488c9087a2f523adf045: Time spent starting tablet: real 0.012s	user 0.006s	sys 0.001s
I20260709 13:18:45.356990 16342 tablet_bootstrap.cc:492] T 23f82224f4564fea93572e8206bad1fe P 90f3da57796b488c9087a2f523adf045: Bootstrap starting.
W20260709 13:18:45.356913 16265 leader_election.cc:343] T 84f2da98864b4425bf7c0055951fdd4b P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:18:45.358042 16265 leader_election.cc:304] T 84f2da98864b4425bf7c0055951fdd4b P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:18:45.358762 16634 raft_consensus.cc:2749] T 84f2da98864b4425bf7c0055951fdd4b P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:18:45.359200 16347 ts_tablet_manager.cc:1434] T e4dc6b0541ca431ebc757beac00521f1 P 1ca2725d290e47d68b1a4b609c9816ac: Time spent starting tablet: real 0.014s	user 0.008s	sys 0.000s
I20260709 13:18:45.360463 16347 tablet_bootstrap.cc:492] T 57a744f5aaf944678a63469cbf514015 P 1ca2725d290e47d68b1a4b609c9816ac: Bootstrap starting.
I20260709 13:18:45.367080 16342 tablet_bootstrap.cc:654] T 23f82224f4564fea93572e8206bad1fe P 90f3da57796b488c9087a2f523adf045: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:45.380071 16342 tablet_bootstrap.cc:492] T 23f82224f4564fea93572e8206bad1fe P 90f3da57796b488c9087a2f523adf045: No bootstrap required, opened a new log
I20260709 13:18:45.380553 16342 ts_tablet_manager.cc:1403] T 23f82224f4564fea93572e8206bad1fe P 90f3da57796b488c9087a2f523adf045: Time spent bootstrapping tablet: real 0.024s	user 0.015s	sys 0.000s
I20260709 13:18:45.383105 16342 raft_consensus.cc:359] T 23f82224f4564fea93572e8206bad1fe P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:45.383883 16342 raft_consensus.cc:385] T 23f82224f4564fea93572e8206bad1fe P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:45.384202 16342 raft_consensus.cc:740] T 23f82224f4564fea93572e8206bad1fe P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Initialized, Role: FOLLOWER
I20260709 13:18:45.384891 16342 consensus_queue.cc:260] T 23f82224f4564fea93572e8206bad1fe P 90f3da57796b488c9087a2f523adf045 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:45.385679 16340 tablet_bootstrap.cc:492] T cdbd78acb4cf46298e0de8ba0d130e19 P 647a343a542d4a139fe106c19095129b: No bootstrap required, opened a new log
I20260709 13:18:45.386197 16340 ts_tablet_manager.cc:1403] T cdbd78acb4cf46298e0de8ba0d130e19 P 647a343a542d4a139fe106c19095129b: Time spent bootstrapping tablet: real 0.046s	user 0.011s	sys 0.009s
I20260709 13:18:45.387400 16342 ts_tablet_manager.cc:1434] T 23f82224f4564fea93572e8206bad1fe P 90f3da57796b488c9087a2f523adf045: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 13:18:45.387916 16347 tablet_bootstrap.cc:654] T 57a744f5aaf944678a63469cbf514015 P 1ca2725d290e47d68b1a4b609c9816ac: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:45.388468 16342 tablet_bootstrap.cc:492] T 66f75124215f4ca6bef831e174a792d8 P 90f3da57796b488c9087a2f523adf045: Bootstrap starting.
I20260709 13:18:45.389113 16340 raft_consensus.cc:359] T cdbd78acb4cf46298e0de8ba0d130e19 P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:45.389900 16340 raft_consensus.cc:385] T cdbd78acb4cf46298e0de8ba0d130e19 P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:45.390229 16340 raft_consensus.cc:740] T cdbd78acb4cf46298e0de8ba0d130e19 P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Initialized, Role: FOLLOWER
I20260709 13:18:45.391235 16340 consensus_queue.cc:260] T cdbd78acb4cf46298e0de8ba0d130e19 P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:45.395149 16340 ts_tablet_manager.cc:1434] T cdbd78acb4cf46298e0de8ba0d130e19 P 647a343a542d4a139fe106c19095129b: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.000s
I20260709 13:18:45.397327 16340 tablet_bootstrap.cc:492] T e5ad4eb281564936a59f41e387a47156 P 647a343a542d4a139fe106c19095129b: Bootstrap starting.
I20260709 13:18:45.397872 16611 raft_consensus.cc:493] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:45.398398 16611 raft_consensus.cc:515] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:45.400580 16611 leader_election.cc:290] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:18:45.397426 16342 tablet_bootstrap.cc:654] T 66f75124215f4ca6bef831e174a792d8 P 90f3da57796b488c9087a2f523adf045: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:45.401144 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdbd78acb4cf46298e0de8ba0d130e19" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:18:45.401818 16150 raft_consensus.cc:2468] T cdbd78acb4cf46298e0de8ba0d130e19 P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 0.
I20260709 13:18:45.403615 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdbd78acb4cf46298e0de8ba0d130e19" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:18:45.403002 16191 leader_election.cc:304] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
I20260709 13:18:45.404920 16611 raft_consensus.cc:2804] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:18:45.405316 16611 raft_consensus.cc:493] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:45.405696 16611 raft_consensus.cc:3060] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:45.404243 16300 raft_consensus.cc:2468] T cdbd78acb4cf46298e0de8ba0d130e19 P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 0.
I20260709 13:18:45.406960 16340 tablet_bootstrap.cc:654] T e5ad4eb281564936a59f41e387a47156 P 647a343a542d4a139fe106c19095129b: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:45.413569 16611 raft_consensus.cc:515] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:45.420048 16611 leader_election.cc:290] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 1 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:18:45.421494 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdbd78acb4cf46298e0de8ba0d130e19" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:18:45.422387 16300 raft_consensus.cc:3060] T cdbd78acb4cf46298e0de8ba0d130e19 P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:45.427865 16630 raft_consensus.cc:493] T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:45.428903 16342 tablet_bootstrap.cc:492] T 66f75124215f4ca6bef831e174a792d8 P 90f3da57796b488c9087a2f523adf045: No bootstrap required, opened a new log
I20260709 13:18:45.428679 16630 raft_consensus.cc:515] T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:45.429821 16342 ts_tablet_manager.cc:1403] T 66f75124215f4ca6bef831e174a792d8 P 90f3da57796b488c9087a2f523adf045: Time spent bootstrapping tablet: real 0.042s	user 0.019s	sys 0.000s
I20260709 13:18:45.431772 16340 tablet_bootstrap.cc:492] T e5ad4eb281564936a59f41e387a47156 P 647a343a542d4a139fe106c19095129b: No bootstrap required, opened a new log
I20260709 13:18:45.435571 16340 ts_tablet_manager.cc:1403] T e5ad4eb281564936a59f41e387a47156 P 647a343a542d4a139fe106c19095129b: Time spent bootstrapping tablet: real 0.038s	user 0.004s	sys 0.014s
I20260709 13:18:45.432783 16342 raft_consensus.cc:359] T 66f75124215f4ca6bef831e174a792d8 P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:45.434491 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4f65cdf00b748bb963ee6e81ef43b79" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:18:45.436857 16342 raft_consensus.cc:385] T 66f75124215f4ca6bef831e174a792d8 P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:45.437173 16342 raft_consensus.cc:740] T 66f75124215f4ca6bef831e174a792d8 P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Initialized, Role: FOLLOWER
I20260709 13:18:45.437861 16342 consensus_queue.cc:260] T 66f75124215f4ca6bef831e174a792d8 P 90f3da57796b488c9087a2f523adf045 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:45.443431 16342 ts_tablet_manager.cc:1434] T 66f75124215f4ca6bef831e174a792d8 P 90f3da57796b488c9087a2f523adf045: Time spent starting tablet: real 0.013s	user 0.006s	sys 0.000s
I20260709 13:18:45.444284 16340 raft_consensus.cc:359] T e5ad4eb281564936a59f41e387a47156 P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:45.445055 16340 raft_consensus.cc:385] T e5ad4eb281564936a59f41e387a47156 P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:45.445372 16340 raft_consensus.cc:740] T e5ad4eb281564936a59f41e387a47156 P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Initialized, Role: FOLLOWER
I20260709 13:18:45.446067 16340 consensus_queue.cc:260] T e5ad4eb281564936a59f41e387a47156 P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:45.447252 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdbd78acb4cf46298e0de8ba0d130e19" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:18:45.447837 16150 raft_consensus.cc:3060] T cdbd78acb4cf46298e0de8ba0d130e19 P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:45.448704 16340 ts_tablet_manager.cc:1434] T e5ad4eb281564936a59f41e387a47156 P 647a343a542d4a139fe106c19095129b: Time spent starting tablet: real 0.013s	user 0.005s	sys 0.002s
I20260709 13:18:45.448772 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4f65cdf00b748bb963ee6e81ef43b79" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:18:45.449934 16340 tablet_bootstrap.cc:492] T 57a744f5aaf944678a63469cbf514015 P 647a343a542d4a139fe106c19095129b: Bootstrap starting.
I20260709 13:18:45.434713 16630 leader_election.cc:290] T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:18:45.451198 16115 leader_election.cc:304] T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:18:45.457365 16340 tablet_bootstrap.cc:654] T 57a744f5aaf944678a63469cbf514015 P 647a343a542d4a139fe106c19095129b: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:45.457919 16048 catalog_manager.cc:5697] T 8d4ab870a05347868956aa8b841b2981 P 90f3da57796b488c9087a2f523adf045 reported cstate change: term changed from 0 to 1, leader changed from <none> to 90f3da57796b488c9087a2f523adf045 (127.14.177.131). New cstate: current_term: 1 leader_uuid: "90f3da57796b488c9087a2f523adf045" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } health_report { overall_health: HEALTHY } } }
I20260709 13:18:45.462463 16630 raft_consensus.cc:2749] T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:18:45.481644 16150 raft_consensus.cc:2468] T cdbd78acb4cf46298e0de8ba0d130e19 P 647a343a542d4a139fe106c19095129b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 1.
I20260709 13:18:45.487089 16611 raft_consensus.cc:493] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:45.487800 16611 raft_consensus.cc:515] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:45.490129 16611 leader_election.cc:290] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:18:45.491808 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9944d5e4b8c4109af9955c4f08c9095" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:18:45.492594 16297 raft_consensus.cc:2468] T c9944d5e4b8c4109af9955c4f08c9095 P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 0.
I20260709 13:18:45.494030 16191 leader_election.cc:304] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
I20260709 13:18:45.495611 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9944d5e4b8c4109af9955c4f08c9095" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:18:45.497052 16613 raft_consensus.cc:2804] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Leader election won for term 1
W20260709 13:18:45.497349 16191 leader_election.cc:343] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:18:45.497212 16629 raft_consensus.cc:493] T 23f82224f4564fea93572e8206bad1fe P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:45.498154 16629 raft_consensus.cc:515] T 23f82224f4564fea93572e8206bad1fe P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:45.499090 16190 leader_election.cc:304] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 647a343a542d4a139fe106c19095129b
I20260709 13:18:45.500156 16658 raft_consensus.cc:2804] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:18:45.500661 16658 raft_consensus.cc:493] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:45.501078 16658 raft_consensus.cc:3060] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:45.502120 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f82224f4564fea93572e8206bad1fe" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:18:45.497622 16613 raft_consensus.cc:697] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [term 1 LEADER]: Becoming Leader. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
W20260709 13:18:45.503923 16265 leader_election.cc:343] T 23f82224f4564fea93572e8206bad1fe P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:18:45.504496 16613 consensus_queue.cc:237] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:45.506695 16300 raft_consensus.cc:2468] T cdbd78acb4cf46298e0de8ba0d130e19 P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 1.
I20260709 13:18:45.508598 16223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f82224f4564fea93572e8206bad1fe" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:18:45.505206 16629 leader_election.cc:290] T 23f82224f4564fea93572e8206bad1fe P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
W20260709 13:18:45.511663 16265 leader_election.cc:343] T 23f82224f4564fea93572e8206bad1fe P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:18:45.512229 16265 leader_election.cc:304] T 23f82224f4564fea93572e8206bad1fe P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:18:45.513613 16629 raft_consensus.cc:2749] T 23f82224f4564fea93572e8206bad1fe P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:18:45.516805 16347 tablet_bootstrap.cc:492] T 57a744f5aaf944678a63469cbf514015 P 1ca2725d290e47d68b1a4b609c9816ac: No bootstrap required, opened a new log
I20260709 13:18:45.517347 16347 ts_tablet_manager.cc:1403] T 57a744f5aaf944678a63469cbf514015 P 1ca2725d290e47d68b1a4b609c9816ac: Time spent bootstrapping tablet: real 0.157s	user 0.030s	sys 0.069s
I20260709 13:18:45.520255 16347 raft_consensus.cc:359] T 57a744f5aaf944678a63469cbf514015 P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:45.522200 16658 raft_consensus.cc:515] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:45.522806 16347 raft_consensus.cc:385] T 57a744f5aaf944678a63469cbf514015 P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:45.523262 16347 raft_consensus.cc:740] T 57a744f5aaf944678a63469cbf514015 P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Initialized, Role: FOLLOWER
I20260709 13:18:45.524211 16150 raft_consensus.cc:3060] T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:45.523968 16347 consensus_queue.cc:260] T 57a744f5aaf944678a63469cbf514015 P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:45.525708 16347 ts_tablet_manager.cc:1434] T 57a744f5aaf944678a63469cbf514015 P 1ca2725d290e47d68b1a4b609c9816ac: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.000s
I20260709 13:18:45.526680 16347 tablet_bootstrap.cc:492] T 26354361e9b044cea1c2b86c4882fe92 P 1ca2725d290e47d68b1a4b609c9816ac: Bootstrap starting.
I20260709 13:18:45.522368 16340 tablet_bootstrap.cc:492] T 57a744f5aaf944678a63469cbf514015 P 647a343a542d4a139fe106c19095129b: No bootstrap required, opened a new log
I20260709 13:18:45.529387 16340 ts_tablet_manager.cc:1403] T 57a744f5aaf944678a63469cbf514015 P 647a343a542d4a139fe106c19095129b: Time spent bootstrapping tablet: real 0.080s	user 0.024s	sys 0.035s
I20260709 13:18:45.532719 16347 tablet_bootstrap.cc:654] T 26354361e9b044cea1c2b86c4882fe92 P 1ca2725d290e47d68b1a4b609c9816ac: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:45.532989 16611 raft_consensus.cc:493] T e4dc6b0541ca431ebc757beac00521f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:45.533496 16611 raft_consensus.cc:515] T e4dc6b0541ca431ebc757beac00521f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:45.538571 16658 leader_election.cc:290] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 1 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:18:45.541599 16611 leader_election.cc:290] T e4dc6b0541ca431ebc757beac00521f1 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:18:45.544271 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4dc6b0541ca431ebc757beac00521f1" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:18:45.545188 16297 raft_consensus.cc:2393] T e4dc6b0541ca431ebc757beac00521f1 P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 1: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
I20260709 13:18:45.547698 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9944d5e4b8c4109af9955c4f08c9095" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:18:45.548503 16300 raft_consensus.cc:3060] T c9944d5e4b8c4109af9955c4f08c9095 P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:45.558744 16340 raft_consensus.cc:359] T 57a744f5aaf944678a63469cbf514015 P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:45.559583 16340 raft_consensus.cc:385] T 57a744f5aaf944678a63469cbf514015 P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:45.559881 16340 raft_consensus.cc:740] T 57a744f5aaf944678a63469cbf514015 P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Initialized, Role: FOLLOWER
I20260709 13:18:45.560542 16340 consensus_queue.cc:260] T 57a744f5aaf944678a63469cbf514015 P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:45.562844 16340 ts_tablet_manager.cc:1434] T 57a744f5aaf944678a63469cbf514015 P 647a343a542d4a139fe106c19095129b: Time spent starting tablet: real 0.033s	user 0.006s	sys 0.000s
I20260709 13:18:45.564021 16340 tablet_bootstrap.cc:492] T 26354361e9b044cea1c2b86c4882fe92 P 647a343a542d4a139fe106c19095129b: Bootstrap starting.
I20260709 13:18:45.570451 16340 tablet_bootstrap.cc:654] T 26354361e9b044cea1c2b86c4882fe92 P 647a343a542d4a139fe106c19095129b: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:45.571097 16520 raft_consensus.cc:493] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:45.571712 16520 raft_consensus.cc:515] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:45.574229 16520 leader_election.cc:290] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:18:45.577955 16631 raft_consensus.cc:493] T 57a744f5aaf944678a63469cbf514015 P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:45.578610 16631 raft_consensus.cc:515] T 57a744f5aaf944678a63469cbf514015 P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:45.581669 16631 leader_election.cc:290] T 57a744f5aaf944678a63469cbf514015 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:18:45.585059 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ad5dbfa99ee4b8d9e28be92b3f36db5" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:18:45.587901 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57a744f5aaf944678a63469cbf514015" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:18:45.588819 16225 raft_consensus.cc:2468] T 57a744f5aaf944678a63469cbf514015 P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 0.
I20260709 13:18:45.591017 16148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4dc6b0541ca431ebc757beac00521f1" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:18:45.592002 16148 raft_consensus.cc:2393] T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 1: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
I20260709 13:18:45.591220 16151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9944d5e4b8c4109af9955c4f08c9095" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:18:45.594671 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ad5dbfa99ee4b8d9e28be92b3f36db5" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:18:45.595472 16299 raft_consensus.cc:2468] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 0.
I20260709 13:18:45.597222 16046 catalog_manager.cc:5697] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac reported cstate change: term changed from 0 to 1, leader changed from <none> to 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130). New cstate: current_term: 1 leader_uuid: "1ca2725d290e47d68b1a4b609c9816ac" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:45.598011 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57a744f5aaf944678a63469cbf514015" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:18:45.598650 16299 raft_consensus.cc:2468] T 57a744f5aaf944678a63469cbf514015 P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 0.
W20260709 13:18:45.602269 16116 leader_election.cc:343] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:18:45.602989 16116 leader_election.cc:304] T 57a744f5aaf944678a63469cbf514015 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
I20260709 13:18:45.604372 16651 raft_consensus.cc:2804] T 57a744f5aaf944678a63469cbf514015 P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:18:45.604914 16651 raft_consensus.cc:493] T 57a744f5aaf944678a63469cbf514015 P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:45.605394 16651 raft_consensus.cc:3060] T 57a744f5aaf944678a63469cbf514015 P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:45.616466 16115 leader_election.cc:304] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac
I20260709 13:18:45.618410 16631 raft_consensus.cc:2804] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:18:45.618976 16631 raft_consensus.cc:493] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:45.619501 16631 raft_consensus.cc:3060] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:45.624125 16703 raft_consensus.cc:493] T 66f75124215f4ca6bef831e174a792d8 P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:45.625012 16703 raft_consensus.cc:515] T 66f75124215f4ca6bef831e174a792d8 P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:45.627485 16191 leader_election.cc:304] T e4dc6b0541ca431ebc757beac00521f1 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:18:45.629572 16613 raft_consensus.cc:3060] T e4dc6b0541ca431ebc757beac00521f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:45.635128 16703 leader_election.cc:290] T 66f75124215f4ca6bef831e174a792d8 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
W20260709 13:18:45.636570 16191 leader_election.cc:343] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:18:45.638368 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66f75124215f4ca6bef831e174a792d8" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:18:45.641098 16151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66f75124215f4ca6bef831e174a792d8" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
W20260709 13:18:45.642653 16265 leader_election.cc:343] T 66f75124215f4ca6bef831e174a792d8 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:18:45.643350 16613 raft_consensus.cc:2749] T e4dc6b0541ca431ebc757beac00521f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:18:45.644088 16265 leader_election.cc:343] T 66f75124215f4ca6bef831e174a792d8 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:18:45.644605 16265 leader_election.cc:304] T 66f75124215f4ca6bef831e174a792d8 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:18:45.646461 16703 raft_consensus.cc:2749] T 66f75124215f4ca6bef831e174a792d8 P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:18:45.649044 16651 raft_consensus.cc:515] T 57a744f5aaf944678a63469cbf514015 P 647a343a542d4a139fe106c19095129b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:45.649616 16347 tablet_bootstrap.cc:492] T 26354361e9b044cea1c2b86c4882fe92 P 1ca2725d290e47d68b1a4b609c9816ac: No bootstrap required, opened a new log
I20260709 13:18:45.650218 16347 ts_tablet_manager.cc:1403] T 26354361e9b044cea1c2b86c4882fe92 P 1ca2725d290e47d68b1a4b609c9816ac: Time spent bootstrapping tablet: real 0.124s	user 0.039s	sys 0.076s
I20260709 13:18:45.653162 16347 raft_consensus.cc:359] T 26354361e9b044cea1c2b86c4882fe92 P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:45.653860 16347 raft_consensus.cc:385] T 26354361e9b044cea1c2b86c4882fe92 P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:45.654147 16347 raft_consensus.cc:740] T 26354361e9b044cea1c2b86c4882fe92 P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Initialized, Role: FOLLOWER
I20260709 13:18:45.654830 16347 consensus_queue.cc:260] T 26354361e9b044cea1c2b86c4882fe92 P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:45.657138 16347 ts_tablet_manager.cc:1434] T 26354361e9b044cea1c2b86c4882fe92 P 1ca2725d290e47d68b1a4b609c9816ac: Time spent starting tablet: real 0.007s	user 0.000s	sys 0.005s
I20260709 13:18:45.658257 16347 tablet_bootstrap.cc:492] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 1ca2725d290e47d68b1a4b609c9816ac: Bootstrap starting.
I20260709 13:18:45.665627 16651 leader_election.cc:290] T 57a744f5aaf944678a63469cbf514015 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 1 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:18:45.666675 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57a744f5aaf944678a63469cbf514015" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:18:45.667308 16225 raft_consensus.cc:3060] T 57a744f5aaf944678a63469cbf514015 P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:45.668869 16347 tablet_bootstrap.cc:654] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 1ca2725d290e47d68b1a4b609c9816ac: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:45.674305 16225 raft_consensus.cc:2468] T 57a744f5aaf944678a63469cbf514015 P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 1.
I20260709 13:18:45.675511 16116 leader_election.cc:304] T 57a744f5aaf944678a63469cbf514015 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
I20260709 13:18:45.676716 16520 raft_consensus.cc:2804] T 57a744f5aaf944678a63469cbf514015 P 647a343a542d4a139fe106c19095129b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:18:45.677255 16520 raft_consensus.cc:697] T 57a744f5aaf944678a63469cbf514015 P 647a343a542d4a139fe106c19095129b [term 1 LEADER]: Becoming Leader. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:18:45.678455 16520 consensus_queue.cc:237] T 57a744f5aaf944678a63469cbf514015 P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:45.688176 16631 raft_consensus.cc:515] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:45.693324 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ad5dbfa99ee4b8d9e28be92b3f36db5" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
W20260709 13:18:45.694835 16116 leader_election.cc:343] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:18:45.695544 16631 leader_election.cc:290] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 1 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:18:45.656008 16300 raft_consensus.cc:2468] T c9944d5e4b8c4109af9955c4f08c9095 P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 1.
I20260709 13:18:45.698666 16190 leader_election.cc:304] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 647a343a542d4a139fe106c19095129b
I20260709 13:18:45.699901 16658 raft_consensus.cc:2804] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:18:45.700064 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57a744f5aaf944678a63469cbf514015" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:18:45.700537 16658 raft_consensus.cc:697] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac [term 1 LEADER]: Becoming Leader. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:18:45.700935 16297 raft_consensus.cc:3060] T 57a744f5aaf944678a63469cbf514015 P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:45.701701 16658 consensus_queue.cc:237] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:45.711344 16046 catalog_manager.cc:5697] T 57a744f5aaf944678a63469cbf514015 P 647a343a542d4a139fe106c19095129b reported cstate change: term changed from 0 to 1, leader changed from <none> to 647a343a542d4a139fe106c19095129b (127.14.177.129). New cstate: current_term: 1 leader_uuid: "647a343a542d4a139fe106c19095129b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:45.712536 16297 raft_consensus.cc:2468] T 57a744f5aaf944678a63469cbf514015 P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 1.
I20260709 13:18:45.713765 16347 tablet_bootstrap.cc:492] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 1ca2725d290e47d68b1a4b609c9816ac: No bootstrap required, opened a new log
I20260709 13:18:45.714289 16347 ts_tablet_manager.cc:1403] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 1ca2725d290e47d68b1a4b609c9816ac: Time spent bootstrapping tablet: real 0.056s	user 0.012s	sys 0.017s
I20260709 13:18:45.715416 16340 tablet_bootstrap.cc:492] T 26354361e9b044cea1c2b86c4882fe92 P 647a343a542d4a139fe106c19095129b: No bootstrap required, opened a new log
I20260709 13:18:45.716478 16340 ts_tablet_manager.cc:1403] T 26354361e9b044cea1c2b86c4882fe92 P 647a343a542d4a139fe106c19095129b: Time spent bootstrapping tablet: real 0.153s	user 0.022s	sys 0.037s
I20260709 13:18:45.717262 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ad5dbfa99ee4b8d9e28be92b3f36db5" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:18:45.717854 16297 raft_consensus.cc:3060] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:45.722827 16347 raft_consensus.cc:359] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:45.723574 16347 raft_consensus.cc:385] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:45.723848 16347 raft_consensus.cc:740] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Initialized, Role: FOLLOWER
I20260709 13:18:45.724537 16347 consensus_queue.cc:260] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:45.726800 16347 ts_tablet_manager.cc:1434] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 1ca2725d290e47d68b1a4b609c9816ac: Time spent starting tablet: real 0.012s	user 0.007s	sys 0.000s
I20260709 13:18:45.726869 16297 raft_consensus.cc:2468] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 1.
I20260709 13:18:45.727923 16347 tablet_bootstrap.cc:492] T e5ad4eb281564936a59f41e387a47156 P 1ca2725d290e47d68b1a4b609c9816ac: Bootstrap starting.
I20260709 13:18:45.728032 16115 leader_election.cc:304] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac
I20260709 13:18:45.729683 16631 raft_consensus.cc:2804] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:18:45.730142 16631 raft_consensus.cc:697] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b [term 1 LEADER]: Becoming Leader. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:18:45.730921 16631 consensus_queue.cc:237] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:45.741432 16520 raft_consensus.cc:493] T e5ad4eb281564936a59f41e387a47156 P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:45.742188 16520 raft_consensus.cc:515] T e5ad4eb281564936a59f41e387a47156 P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:45.743626 16340 raft_consensus.cc:359] T 26354361e9b044cea1c2b86c4882fe92 P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:45.744879 16340 raft_consensus.cc:385] T 26354361e9b044cea1c2b86c4882fe92 P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:45.745080 16520 leader_election.cc:290] T e5ad4eb281564936a59f41e387a47156 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:18:45.745298 16340 raft_consensus.cc:740] T 26354361e9b044cea1c2b86c4882fe92 P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Initialized, Role: FOLLOWER
I20260709 13:18:45.746503 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5ad4eb281564936a59f41e387a47156" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:18:45.746592 16340 consensus_queue.cc:260] T 26354361e9b044cea1c2b86c4882fe92 P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:45.747233 16297 raft_consensus.cc:2468] T e5ad4eb281564936a59f41e387a47156 P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 0.
I20260709 13:18:45.748587 16340 ts_tablet_manager.cc:1434] T 26354361e9b044cea1c2b86c4882fe92 P 647a343a542d4a139fe106c19095129b: Time spent starting tablet: real 0.032s	user 0.006s	sys 0.000s
I20260709 13:18:45.749697 16340 tablet_bootstrap.cc:492] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b: Bootstrap starting.
I20260709 13:18:45.750015 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5ad4eb281564936a59f41e387a47156" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:18:45.751736 16115 leader_election.cc:304] T e5ad4eb281564936a59f41e387a47156 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:18:45.752424 16631 raft_consensus.cc:2804] T e5ad4eb281564936a59f41e387a47156 P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Leader pre-election won for term 1
W20260709 13:18:45.752604 16116 leader_election.cc:343] T e5ad4eb281564936a59f41e387a47156 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:18:45.756801 16340 tablet_bootstrap.cc:654] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:45.752797 16631 raft_consensus.cc:493] T e5ad4eb281564936a59f41e387a47156 P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:45.757402 16631 raft_consensus.cc:3060] T e5ad4eb281564936a59f41e387a47156 P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:45.758795 16347 tablet_bootstrap.cc:654] T e5ad4eb281564936a59f41e387a47156 P 1ca2725d290e47d68b1a4b609c9816ac: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:45.758011 16046 catalog_manager.cc:5697] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac reported cstate change: term changed from 0 to 1, leader changed from <none> to 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130). New cstate: current_term: 1 leader_uuid: "1ca2725d290e47d68b1a4b609c9816ac" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:45.773023 16631 raft_consensus.cc:515] T e5ad4eb281564936a59f41e387a47156 P 647a343a542d4a139fe106c19095129b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:45.776762 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5ad4eb281564936a59f41e387a47156" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:18:45.777352 16297 raft_consensus.cc:3060] T e5ad4eb281564936a59f41e387a47156 P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:45.777397 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5ad4eb281564936a59f41e387a47156" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
W20260709 13:18:45.778898 16116 leader_election.cc:343] T e5ad4eb281564936a59f41e387a47156 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:18:45.784969 16297 raft_consensus.cc:2468] T e5ad4eb281564936a59f41e387a47156 P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 1.
I20260709 13:18:45.786551 16115 leader_election.cc:304] T e5ad4eb281564936a59f41e387a47156 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac
I20260709 13:18:45.787408 16631 leader_election.cc:290] T e5ad4eb281564936a59f41e387a47156 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 1 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:18:45.787417 16651 raft_consensus.cc:2804] T e5ad4eb281564936a59f41e387a47156 P 647a343a542d4a139fe106c19095129b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:18:45.788385 16703 consensus_queue.cc:1048] T 8d4ab870a05347868956aa8b841b2981 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:18:45.788812 16651 raft_consensus.cc:697] T e5ad4eb281564936a59f41e387a47156 P 647a343a542d4a139fe106c19095129b [term 1 LEADER]: Becoming Leader. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:18:45.789721 16651 consensus_queue.cc:237] T e5ad4eb281564936a59f41e387a47156 P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:45.792076 16347 tablet_bootstrap.cc:492] T e5ad4eb281564936a59f41e387a47156 P 1ca2725d290e47d68b1a4b609c9816ac: No bootstrap required, opened a new log
I20260709 13:18:45.792557 16347 ts_tablet_manager.cc:1403] T e5ad4eb281564936a59f41e387a47156 P 1ca2725d290e47d68b1a4b609c9816ac: Time spent bootstrapping tablet: real 0.065s	user 0.010s	sys 0.004s
I20260709 13:18:45.795351 16347 raft_consensus.cc:359] T e5ad4eb281564936a59f41e387a47156 P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:45.796010 16347 raft_consensus.cc:385] T e5ad4eb281564936a59f41e387a47156 P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:45.796265 16347 raft_consensus.cc:740] T e5ad4eb281564936a59f41e387a47156 P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Initialized, Role: FOLLOWER
I20260709 13:18:45.796968 16347 consensus_queue.cc:260] T e5ad4eb281564936a59f41e387a47156 P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:45.798441 16347 ts_tablet_manager.cc:1434] T e5ad4eb281564936a59f41e387a47156 P 1ca2725d290e47d68b1a4b609c9816ac: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 13:18:45.798995 16658 raft_consensus.cc:493] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:45.799400 16347 tablet_bootstrap.cc:492] T 84f2da98864b4425bf7c0055951fdd4b P 1ca2725d290e47d68b1a4b609c9816ac: Bootstrap starting.
I20260709 13:18:45.801698 16046 catalog_manager.cc:5697] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b reported cstate change: term changed from 0 to 1, leader changed from <none> to 647a343a542d4a139fe106c19095129b (127.14.177.129). New cstate: current_term: 1 leader_uuid: "647a343a542d4a139fe106c19095129b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:45.799830 16658 raft_consensus.cc:515] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:45.805833 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ad5dbfa99ee4b8d9e28be92b3f36db5" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:18:45.806629 16300 raft_consensus.cc:2393] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 1: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
I20260709 13:18:45.808292 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ad5dbfa99ee4b8d9e28be92b3f36db5" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:18:45.809309 16340 tablet_bootstrap.cc:492] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b: No bootstrap required, opened a new log
I20260709 13:18:45.810559 16658 leader_election.cc:290] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:18:45.810302 16340 ts_tablet_manager.cc:1403] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b: Time spent bootstrapping tablet: real 0.061s	user 0.022s	sys 0.009s
I20260709 13:18:45.810871 16191 leader_election.cc:304] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:18:45.813724 16611 raft_consensus.cc:3060] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:45.814049 16347 tablet_bootstrap.cc:654] T 84f2da98864b4425bf7c0055951fdd4b P 1ca2725d290e47d68b1a4b609c9816ac: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:45.819651 16340 raft_consensus.cc:359] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:45.820504 16340 raft_consensus.cc:385] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:45.826560 16611 raft_consensus.cc:2749] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:18:45.827708 16340 raft_consensus.cc:740] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Initialized, Role: FOLLOWER
I20260709 13:18:45.828488 16340 consensus_queue.cc:260] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:45.830417 16340 ts_tablet_manager.cc:1434] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b: Time spent starting tablet: real 0.019s	user 0.002s	sys 0.001s
I20260709 13:18:45.837040 16046 catalog_manager.cc:5697] T e5ad4eb281564936a59f41e387a47156 P 647a343a542d4a139fe106c19095129b reported cstate change: term changed from 0 to 1, leader changed from <none> to 647a343a542d4a139fe106c19095129b (127.14.177.129). New cstate: current_term: 1 leader_uuid: "647a343a542d4a139fe106c19095129b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:45.842123 16340 tablet_bootstrap.cc:492] T c9944d5e4b8c4109af9955c4f08c9095 P 647a343a542d4a139fe106c19095129b: Bootstrap starting.
I20260709 13:18:45.843122 16347 tablet_bootstrap.cc:492] T 84f2da98864b4425bf7c0055951fdd4b P 1ca2725d290e47d68b1a4b609c9816ac: No bootstrap required, opened a new log
I20260709 13:18:45.843655 16347 ts_tablet_manager.cc:1403] T 84f2da98864b4425bf7c0055951fdd4b P 1ca2725d290e47d68b1a4b609c9816ac: Time spent bootstrapping tablet: real 0.044s	user 0.014s	sys 0.003s
I20260709 13:18:45.846858 16347 raft_consensus.cc:359] T 84f2da98864b4425bf7c0055951fdd4b P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:45.847602 16347 raft_consensus.cc:385] T 84f2da98864b4425bf7c0055951fdd4b P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:45.850001 16347 raft_consensus.cc:740] T 84f2da98864b4425bf7c0055951fdd4b P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Initialized, Role: FOLLOWER
I20260709 13:18:45.848699 16340 tablet_bootstrap.cc:654] T c9944d5e4b8c4109af9955c4f08c9095 P 647a343a542d4a139fe106c19095129b: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:45.850790 16651 raft_consensus.cc:493] T 26354361e9b044cea1c2b86c4882fe92 P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:45.851222 16347 consensus_queue.cc:260] T 84f2da98864b4425bf7c0055951fdd4b P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:45.851485 16651 raft_consensus.cc:515] T 26354361e9b044cea1c2b86c4882fe92 P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:45.853758 16347 ts_tablet_manager.cc:1434] T 84f2da98864b4425bf7c0055951fdd4b P 1ca2725d290e47d68b1a4b609c9816ac: Time spent starting tablet: real 0.010s	user 0.007s	sys 0.000s
I20260709 13:18:45.855064 16347 tablet_bootstrap.cc:492] T 19baa7bc367844c288bca8f81439c413 P 1ca2725d290e47d68b1a4b609c9816ac: Bootstrap starting.
I20260709 13:18:45.855494 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26354361e9b044cea1c2b86c4882fe92" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:18:45.856326 16225 raft_consensus.cc:2468] T 26354361e9b044cea1c2b86c4882fe92 P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 0.
I20260709 13:18:45.861430 16116 leader_election.cc:304] T 26354361e9b044cea1c2b86c4882fe92 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
I20260709 13:18:45.863561 16347 tablet_bootstrap.cc:654] T 19baa7bc367844c288bca8f81439c413 P 1ca2725d290e47d68b1a4b609c9816ac: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:45.865607 16631 raft_consensus.cc:2804] T 26354361e9b044cea1c2b86c4882fe92 P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:18:45.866174 16631 raft_consensus.cc:493] T 26354361e9b044cea1c2b86c4882fe92 P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:45.866587 16631 raft_consensus.cc:3060] T 26354361e9b044cea1c2b86c4882fe92 P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:45.875828 16651 leader_election.cc:290] T 26354361e9b044cea1c2b86c4882fe92 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:18:45.877364 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26354361e9b044cea1c2b86c4882fe92" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:18:45.878261 16300 raft_consensus.cc:2468] T 26354361e9b044cea1c2b86c4882fe92 P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 0.
I20260709 13:18:45.877736 16631 raft_consensus.cc:515] T 26354361e9b044cea1c2b86c4882fe92 P 647a343a542d4a139fe106c19095129b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:45.881475 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26354361e9b044cea1c2b86c4882fe92" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:18:45.882190 16225 raft_consensus.cc:3060] T 26354361e9b044cea1c2b86c4882fe92 P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:45.891884 16340 tablet_bootstrap.cc:492] T c9944d5e4b8c4109af9955c4f08c9095 P 647a343a542d4a139fe106c19095129b: No bootstrap required, opened a new log
I20260709 13:18:45.892370 16340 ts_tablet_manager.cc:1403] T c9944d5e4b8c4109af9955c4f08c9095 P 647a343a542d4a139fe106c19095129b: Time spent bootstrapping tablet: real 0.061s	user 0.024s	sys 0.021s
I20260709 13:18:45.892840 16225 raft_consensus.cc:2468] T 26354361e9b044cea1c2b86c4882fe92 P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 1.
I20260709 13:18:45.893993 16116 leader_election.cc:304] T 26354361e9b044cea1c2b86c4882fe92 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
I20260709 13:18:45.894711 16651 raft_consensus.cc:2804] T 26354361e9b044cea1c2b86c4882fe92 P 647a343a542d4a139fe106c19095129b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:18:45.895196 16651 raft_consensus.cc:697] T 26354361e9b044cea1c2b86c4882fe92 P 647a343a542d4a139fe106c19095129b [term 1 LEADER]: Becoming Leader. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:18:45.894898 16340 raft_consensus.cc:359] T c9944d5e4b8c4109af9955c4f08c9095 P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:45.897123 16340 raft_consensus.cc:385] T c9944d5e4b8c4109af9955c4f08c9095 P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:45.897521 16340 raft_consensus.cc:740] T c9944d5e4b8c4109af9955c4f08c9095 P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Initialized, Role: FOLLOWER
I20260709 13:18:45.898350 16340 consensus_queue.cc:260] T c9944d5e4b8c4109af9955c4f08c9095 P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:45.898872 16703 consensus_queue.cc:1048] T 8d4ab870a05347868956aa8b841b2981 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:18:45.896018 16631 leader_election.cc:290] T 26354361e9b044cea1c2b86c4882fe92 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 1 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:18:45.896021 16651 consensus_queue.cc:237] T 26354361e9b044cea1c2b86c4882fe92 P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:45.906750 16340 ts_tablet_manager.cc:1434] T c9944d5e4b8c4109af9955c4f08c9095 P 647a343a542d4a139fe106c19095129b: Time spent starting tablet: real 0.014s	user 0.008s	sys 0.000s
I20260709 13:18:45.908252 16340 tablet_bootstrap.cc:492] T 23f82224f4564fea93572e8206bad1fe P 647a343a542d4a139fe106c19095129b: Bootstrap starting.
I20260709 13:18:45.918829 16611 raft_consensus.cc:493] T 84f2da98864b4425bf7c0055951fdd4b P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:45.922708 16611 raft_consensus.cc:515] T 84f2da98864b4425bf7c0055951fdd4b P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:45.925913 16611 leader_election.cc:290] T 84f2da98864b4425bf7c0055951fdd4b P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:18:45.932186 16340 tablet_bootstrap.cc:654] T 23f82224f4564fea93572e8206bad1fe P 647a343a542d4a139fe106c19095129b: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:45.935747 16145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f2da98864b4425bf7c0055951fdd4b" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:18:45.926764 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f2da98864b4425bf7c0055951fdd4b" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:18:45.939666 16300 raft_consensus.cc:2468] T 84f2da98864b4425bf7c0055951fdd4b P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 0.
I20260709 13:18:45.941013 16190 leader_election.cc:304] T 84f2da98864b4425bf7c0055951fdd4b P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:18:45.942088 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26354361e9b044cea1c2b86c4882fe92" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:18:45.942799 16297 raft_consensus.cc:3060] T 26354361e9b044cea1c2b86c4882fe92 P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:45.947783 16658 raft_consensus.cc:2804] T 84f2da98864b4425bf7c0055951fdd4b P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:18:45.948333 16658 raft_consensus.cc:493] T 84f2da98864b4425bf7c0055951fdd4b P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:45.948800 16658 raft_consensus.cc:3060] T 84f2da98864b4425bf7c0055951fdd4b P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:18:45.948956 16097 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:18:45.951673 16520 raft_consensus.cc:493] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:45.952270 16520 raft_consensus.cc:515] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:18:45.952823 16191 leader_election.cc:343] T 84f2da98864b4425bf7c0055951fdd4b P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:18:45.952942 16676 consensus_queue.cc:1048] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:18:45.955497 16613 raft_consensus.cc:493] T e5ad4eb281564936a59f41e387a47156 P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:45.956027 16613 raft_consensus.cc:515] T e5ad4eb281564936a59f41e387a47156 P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:45.958209 16613 leader_election.cc:290] T e5ad4eb281564936a59f41e387a47156 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:18:45.958316 16520 leader_election.cc:290] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:18:45.960362 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5ad4eb281564936a59f41e387a47156" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:18:45.960551 16223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:18:45.961151 16299 raft_consensus.cc:2393] T e5ad4eb281564936a59f41e387a47156 P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 1: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
W20260709 13:18:45.962306 16116 leader_election.cc:343] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:18:45.962860 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:18:45.966576 16300 raft_consensus.cc:2468] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 0.
I20260709 13:18:45.970181 16297 raft_consensus.cc:2468] T 26354361e9b044cea1c2b86c4882fe92 P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 1.
I20260709 13:18:45.976159 16340 tablet_bootstrap.cc:492] T 23f82224f4564fea93572e8206bad1fe P 647a343a542d4a139fe106c19095129b: No bootstrap required, opened a new log
I20260709 13:18:45.972589 16658 raft_consensus.cc:515] T 84f2da98864b4425bf7c0055951fdd4b P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:45.973016 16347 tablet_bootstrap.cc:492] T 19baa7bc367844c288bca8f81439c413 P 1ca2725d290e47d68b1a4b609c9816ac: No bootstrap required, opened a new log
I20260709 13:18:45.976900 16340 ts_tablet_manager.cc:1403] T 23f82224f4564fea93572e8206bad1fe P 647a343a542d4a139fe106c19095129b: Time spent bootstrapping tablet: real 0.069s	user 0.013s	sys 0.004s
I20260709 13:18:45.977092 16347 ts_tablet_manager.cc:1403] T 19baa7bc367844c288bca8f81439c413 P 1ca2725d290e47d68b1a4b609c9816ac: Time spent bootstrapping tablet: real 0.122s	user 0.025s	sys 0.062s
I20260709 13:18:45.979460 16340 raft_consensus.cc:359] T 23f82224f4564fea93572e8206bad1fe P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:45.980165 16145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5ad4eb281564936a59f41e387a47156" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:18:45.980629 16340 raft_consensus.cc:385] T 23f82224f4564fea93572e8206bad1fe P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:45.980950 16340 raft_consensus.cc:740] T 23f82224f4564fea93572e8206bad1fe P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Initialized, Role: FOLLOWER
I20260709 13:18:45.981076 16658 leader_election.cc:290] T 84f2da98864b4425bf7c0055951fdd4b P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 1 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:18:45.980204 16347 raft_consensus.cc:359] T 19baa7bc367844c288bca8f81439c413 P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:45.981695 16340 consensus_queue.cc:260] T 23f82224f4564fea93572e8206bad1fe P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:45.982226 16347 raft_consensus.cc:385] T 19baa7bc367844c288bca8f81439c413 P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:45.982514 16347 raft_consensus.cc:740] T 19baa7bc367844c288bca8f81439c413 P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Initialized, Role: FOLLOWER
I20260709 13:18:45.982676 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f2da98864b4425bf7c0055951fdd4b" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:18:45.984198 16297 raft_consensus.cc:3060] T 84f2da98864b4425bf7c0055951fdd4b P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:45.985121 16191 leader_election.cc:304] T e5ad4eb281564936a59f41e387a47156 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:18:45.983722 16340 ts_tablet_manager.cc:1434] T 23f82224f4564fea93572e8206bad1fe P 647a343a542d4a139fe106c19095129b: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.001s
I20260709 13:18:45.986466 16151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f2da98864b4425bf7c0055951fdd4b" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:18:45.983115 16347 consensus_queue.cc:260] T 19baa7bc367844c288bca8f81439c413 P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:45.988462 16347 ts_tablet_manager.cc:1434] T 19baa7bc367844c288bca8f81439c413 P 1ca2725d290e47d68b1a4b609c9816ac: Time spent starting tablet: real 0.011s	user 0.005s	sys 0.000s
I20260709 13:18:45.988536 16115 leader_election.cc:304] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac
I20260709 13:18:45.989810 16631 raft_consensus.cc:2804] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:18:45.990223 16631 raft_consensus.cc:493] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:45.990620 16631 raft_consensus.cc:3060] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:45.991184 16297 raft_consensus.cc:2468] T 84f2da98864b4425bf7c0055951fdd4b P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 1.
I20260709 13:18:45.989392 16347 tablet_bootstrap.cc:492] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 1ca2725d290e47d68b1a4b609c9816ac: Bootstrap starting.
I20260709 13:18:45.992462 16190 leader_election.cc:304] T 84f2da98864b4425bf7c0055951fdd4b P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:18:45.990234 16340 tablet_bootstrap.cc:492] T 84f2da98864b4425bf7c0055951fdd4b P 647a343a542d4a139fe106c19095129b: Bootstrap starting.
W20260709 13:18:45.996141 16191 leader_election.cc:343] T 84f2da98864b4425bf7c0055951fdd4b P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:18:45.997625 16681 raft_consensus.cc:2804] T 84f2da98864b4425bf7c0055951fdd4b P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:18:45.998111 16681 raft_consensus.cc:697] T 84f2da98864b4425bf7c0055951fdd4b P 1ca2725d290e47d68b1a4b609c9816ac [term 1 LEADER]: Becoming Leader. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:18:45.999698 16631 raft_consensus.cc:515] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:45.998984 16681 consensus_queue.cc:237] T 84f2da98864b4425bf7c0055951fdd4b P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:46.003340 16676 raft_consensus.cc:3060] T e5ad4eb281564936a59f41e387a47156 P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:46.003767 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:18:46.004288 16347 tablet_bootstrap.cc:654] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 1ca2725d290e47d68b1a4b609c9816ac: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:46.004354 16226 raft_consensus.cc:3060] T 19baa7bc367844c288bca8f81439c413 P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:46.009981 16048 catalog_manager.cc:5697] T 26354361e9b044cea1c2b86c4882fe92 P 647a343a542d4a139fe106c19095129b reported cstate change: term changed from 0 to 1, leader changed from <none> to 647a343a542d4a139fe106c19095129b (127.14.177.129). New cstate: current_term: 1 leader_uuid: "647a343a542d4a139fe106c19095129b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:46.012785 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:18:46.013550 16300 raft_consensus.cc:3060] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:46.018249 16226 raft_consensus.cc:2468] T 19baa7bc367844c288bca8f81439c413 P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 1.
I20260709 13:18:46.019371 16116 leader_election.cc:304] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
I20260709 13:18:46.020066 16651 raft_consensus.cc:2804] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:18:46.020592 16651 raft_consensus.cc:697] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [term 1 LEADER]: Becoming Leader. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:18:46.005668 16631 leader_election.cc:290] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 1 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:18:46.021497 16300 raft_consensus.cc:2468] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 1.
I20260709 13:18:46.021454 16651 consensus_queue.cc:237] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:46.033401 16613 consensus_queue.cc:1048] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:18:46.043567 16631 raft_consensus.cc:493] T c9944d5e4b8c4109af9955c4f08c9095 P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:46.044937 16631 raft_consensus.cc:515] T c9944d5e4b8c4109af9955c4f08c9095 P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:46.048322 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9944d5e4b8c4109af9955c4f08c9095" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:18:46.057365 16631 leader_election.cc:290] T c9944d5e4b8c4109af9955c4f08c9095 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:18:46.059603 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9944d5e4b8c4109af9955c4f08c9095" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:18:46.060348 16300 raft_consensus.cc:2393] T c9944d5e4b8c4109af9955c4f08c9095 P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 1: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
I20260709 13:18:46.061327 16115 leader_election.cc:304] T c9944d5e4b8c4109af9955c4f08c9095 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:18:46.062994 16651 raft_consensus.cc:3060] T c9944d5e4b8c4109af9955c4f08c9095 P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:46.065699 16676 raft_consensus.cc:2749] T e5ad4eb281564936a59f41e387a47156 P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:18:46.081020 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 75062.4us
I20260709 13:18:46.087276 16340 tablet_bootstrap.cc:654] T 84f2da98864b4425bf7c0055951fdd4b P 647a343a542d4a139fe106c19095129b: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:46.072299 16048 catalog_manager.cc:5697] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b reported cstate change: term changed from 0 to 1, leader changed from <none> to 647a343a542d4a139fe106c19095129b (127.14.177.129). New cstate: current_term: 1 leader_uuid: "647a343a542d4a139fe106c19095129b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:46.098893 16651 raft_consensus.cc:2749] T c9944d5e4b8c4109af9955c4f08c9095 P 647a343a542d4a139fe106c19095129b [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:18:46.100346 16340 tablet_bootstrap.cc:492] T 84f2da98864b4425bf7c0055951fdd4b P 647a343a542d4a139fe106c19095129b: No bootstrap required, opened a new log
I20260709 13:18:46.100844 16347 tablet_bootstrap.cc:492] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 1ca2725d290e47d68b1a4b609c9816ac: No bootstrap required, opened a new log
I20260709 13:18:46.100890 16340 ts_tablet_manager.cc:1403] T 84f2da98864b4425bf7c0055951fdd4b P 647a343a542d4a139fe106c19095129b: Time spent bootstrapping tablet: real 0.111s	user 0.008s	sys 0.011s
I20260709 13:18:46.101444 16347 ts_tablet_manager.cc:1403] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 1ca2725d290e47d68b1a4b609c9816ac: Time spent bootstrapping tablet: real 0.112s	user 0.025s	sys 0.039s
I20260709 13:18:46.104367 16347 raft_consensus.cc:359] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:46.105094 16347 raft_consensus.cc:385] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:46.104748 16340 raft_consensus.cc:359] T 84f2da98864b4425bf7c0055951fdd4b P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:46.105396 16347 raft_consensus.cc:740] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Initialized, Role: FOLLOWER
I20260709 13:18:46.105703 16340 raft_consensus.cc:385] T 84f2da98864b4425bf7c0055951fdd4b P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:46.106063 16340 raft_consensus.cc:740] T 84f2da98864b4425bf7c0055951fdd4b P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Initialized, Role: FOLLOWER
I20260709 13:18:46.106182 16347 consensus_queue.cc:260] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:46.106974 16340 consensus_queue.cc:260] T 84f2da98864b4425bf7c0055951fdd4b P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:46.107873 16347 ts_tablet_manager.cc:1434] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 1ca2725d290e47d68b1a4b609c9816ac: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.000s
I20260709 13:18:46.108928 16347 tablet_bootstrap.cc:492] T 23f82224f4564fea93572e8206bad1fe P 1ca2725d290e47d68b1a4b609c9816ac: Bootstrap starting.
I20260709 13:18:46.109746 16340 ts_tablet_manager.cc:1434] T 84f2da98864b4425bf7c0055951fdd4b P 647a343a542d4a139fe106c19095129b: Time spent starting tablet: real 0.008s	user 0.001s	sys 0.005s
I20260709 13:18:46.111048 16340 tablet_bootstrap.cc:492] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 647a343a542d4a139fe106c19095129b: Bootstrap starting.
I20260709 13:18:46.115433 16347 tablet_bootstrap.cc:654] T 23f82224f4564fea93572e8206bad1fe P 1ca2725d290e47d68b1a4b609c9816ac: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:46.117762 16340 tablet_bootstrap.cc:654] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 647a343a542d4a139fe106c19095129b: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:46.135474 16347 tablet_bootstrap.cc:492] T 23f82224f4564fea93572e8206bad1fe P 1ca2725d290e47d68b1a4b609c9816ac: No bootstrap required, opened a new log
I20260709 13:18:46.135967 16347 ts_tablet_manager.cc:1403] T 23f82224f4564fea93572e8206bad1fe P 1ca2725d290e47d68b1a4b609c9816ac: Time spent bootstrapping tablet: real 0.027s	user 0.010s	sys 0.007s
I20260709 13:18:46.139315 16347 raft_consensus.cc:359] T 23f82224f4564fea93572e8206bad1fe P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:46.140257 16347 raft_consensus.cc:385] T 23f82224f4564fea93572e8206bad1fe P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:46.140625 16347 raft_consensus.cc:740] T 23f82224f4564fea93572e8206bad1fe P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Initialized, Role: FOLLOWER
I20260709 13:18:46.141464 16347 consensus_queue.cc:260] T 23f82224f4564fea93572e8206bad1fe P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:46.146049 16340 tablet_bootstrap.cc:492] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 647a343a542d4a139fe106c19095129b: No bootstrap required, opened a new log
I20260709 13:18:46.146543 16340 ts_tablet_manager.cc:1403] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 647a343a542d4a139fe106c19095129b: Time spent bootstrapping tablet: real 0.036s	user 0.017s	sys 0.009s
I20260709 13:18:46.150663 16340 raft_consensus.cc:359] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:46.151443 16340 raft_consensus.cc:385] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:46.151759 16340 raft_consensus.cc:740] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Initialized, Role: FOLLOWER
I20260709 13:18:46.151952 16347 ts_tablet_manager.cc:1434] T 23f82224f4564fea93572e8206bad1fe P 1ca2725d290e47d68b1a4b609c9816ac: Time spent starting tablet: real 0.016s	user 0.005s	sys 0.003s
I20260709 13:18:46.152448 16340 consensus_queue.cc:260] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:46.154948 16340 ts_tablet_manager.cc:1434] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 647a343a542d4a139fe106c19095129b: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 13:18:46.152951 16347 tablet_bootstrap.cc:492] T 66f75124215f4ca6bef831e174a792d8 P 1ca2725d290e47d68b1a4b609c9816ac: Bootstrap starting.
I20260709 13:18:46.156863 16048 catalog_manager.cc:5697] T 84f2da98864b4425bf7c0055951fdd4b P 1ca2725d290e47d68b1a4b609c9816ac reported cstate change: term changed from 0 to 1, leader changed from <none> to 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130). New cstate: current_term: 1 leader_uuid: "1ca2725d290e47d68b1a4b609c9816ac" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:46.161810 16347 tablet_bootstrap.cc:654] T 66f75124215f4ca6bef831e174a792d8 P 1ca2725d290e47d68b1a4b609c9816ac: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:46.165549 16340 tablet_bootstrap.cc:492] T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b: Bootstrap starting.
I20260709 13:18:46.167867 16347 tablet_bootstrap.cc:492] T 66f75124215f4ca6bef831e174a792d8 P 1ca2725d290e47d68b1a4b609c9816ac: No bootstrap required, opened a new log
I20260709 13:18:46.168326 16347 ts_tablet_manager.cc:1403] T 66f75124215f4ca6bef831e174a792d8 P 1ca2725d290e47d68b1a4b609c9816ac: Time spent bootstrapping tablet: real 0.016s	user 0.006s	sys 0.005s
I20260709 13:18:46.170833 16340 tablet_bootstrap.cc:654] T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:46.171130 16347 raft_consensus.cc:359] T 66f75124215f4ca6bef831e174a792d8 P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:46.171940 16347 raft_consensus.cc:385] T 66f75124215f4ca6bef831e174a792d8 P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:46.172233 16347 raft_consensus.cc:740] T 66f75124215f4ca6bef831e174a792d8 P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Initialized, Role: FOLLOWER
I20260709 13:18:46.173390 16347 consensus_queue.cc:260] T 66f75124215f4ca6bef831e174a792d8 P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:46.179112 16347 ts_tablet_manager.cc:1434] T 66f75124215f4ca6bef831e174a792d8 P 1ca2725d290e47d68b1a4b609c9816ac: Time spent starting tablet: real 0.010s	user 0.007s	sys 0.000s
I20260709 13:18:46.186494 16521 consensus_queue.cc:1048] T 57a744f5aaf944678a63469cbf514015 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:18:46.188186 16340 tablet_bootstrap.cc:492] T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b: No bootstrap required, opened a new log
I20260709 13:18:46.188664 16340 ts_tablet_manager.cc:1403] T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b: Time spent bootstrapping tablet: real 0.023s	user 0.009s	sys 0.005s
I20260709 13:18:46.191270 16340 raft_consensus.cc:359] T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:46.191888 16340 raft_consensus.cc:385] T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:46.192145 16340 raft_consensus.cc:740] T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Initialized, Role: FOLLOWER
I20260709 13:18:46.192879 16340 consensus_queue.cc:260] T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:46.195031 16340 ts_tablet_manager.cc:1434] T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b: Time spent starting tablet: real 0.006s	user 0.003s	sys 0.000s
I20260709 13:18:46.197703 16520 consensus_queue.cc:1048] T 57a744f5aaf944678a63469cbf514015 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:18:46.233126 16521 raft_consensus.cc:493] T 84f2da98864b4425bf7c0055951fdd4b P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:46.233646 16521 raft_consensus.cc:515] T 84f2da98864b4425bf7c0055951fdd4b P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:46.235705 16521 leader_election.cc:290] T 84f2da98864b4425bf7c0055951fdd4b P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:18:46.236555 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f2da98864b4425bf7c0055951fdd4b" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:18:46.237017 16223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f2da98864b4425bf7c0055951fdd4b" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:18:46.237262 16300 raft_consensus.cc:2393] T 84f2da98864b4425bf7c0055951fdd4b P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 1: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
I20260709 13:18:46.238354 16115 leader_election.cc:304] T 84f2da98864b4425bf7c0055951fdd4b P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:18:46.239015 16521 raft_consensus.cc:3060] T 84f2da98864b4425bf7c0055951fdd4b P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:46.240048 16520 consensus_queue.cc:1048] T e5ad4eb281564936a59f41e387a47156 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:18:46.245946 16521 raft_consensus.cc:2749] T 84f2da98864b4425bf7c0055951fdd4b P 647a343a542d4a139fe106c19095129b [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:18:46.248126 16658 consensus_queue.cc:1048] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:18:46.257830 16651 consensus_queue.cc:1048] T e5ad4eb281564936a59f41e387a47156 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:18:46.270422 16681 consensus_queue.cc:1048] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:18:46.296168 16651 consensus_queue.cc:1048] T 26354361e9b044cea1c2b86c4882fe92 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.008s
I20260709 13:18:46.303494 16520 consensus_queue.cc:1048] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 13:18:46.312883 16521 consensus_queue.cc:1048] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:18:46.326457 16520 consensus_queue.cc:1048] T 26354361e9b044cea1c2b86c4882fe92 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:18:46.329936 16703 raft_consensus.cc:493] T e4dc6b0541ca431ebc757beac00521f1 P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:46.330498 16703 raft_consensus.cc:515] T e4dc6b0541ca431ebc757beac00521f1 P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:46.333120 16703 leader_election.cc:290] T e4dc6b0541ca431ebc757beac00521f1 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:18:46.334790 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4dc6b0541ca431ebc757beac00521f1" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:18:46.335775 16150 raft_consensus.cc:2468] T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 1.
I20260709 13:18:46.337532 16265 leader_election.cc:304] T e4dc6b0541ca431ebc757beac00521f1 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:18:46.338274 16703 raft_consensus.cc:2804] T e4dc6b0541ca431ebc757beac00521f1 P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:18:46.338878 16703 raft_consensus.cc:493] T e4dc6b0541ca431ebc757beac00521f1 P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:46.339365 16703 raft_consensus.cc:3060] T e4dc6b0541ca431ebc757beac00521f1 P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:46.339289 16521 raft_consensus.cc:493] T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:46.340790 16521 raft_consensus.cc:515] T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:46.338212 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4dc6b0541ca431ebc757beac00521f1" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:18:46.343205 16226 raft_consensus.cc:2468] T e4dc6b0541ca431ebc757beac00521f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 1.
I20260709 13:18:46.356359 16521 leader_election.cc:290] T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:18:46.357959 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66f75124215f4ca6bef831e174a792d8" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:18:46.361961 16300 raft_consensus.cc:2468] T 66f75124215f4ca6bef831e174a792d8 P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 0.
I20260709 13:18:46.360903 16703 raft_consensus.cc:515] T e4dc6b0541ca431ebc757beac00521f1 P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:46.363579 16115 leader_election.cc:304] T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:18:46.365069 16520 raft_consensus.cc:2804] T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:18:46.365543 16520 raft_consensus.cc:493] T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:46.365993 16520 raft_consensus.cc:3060] T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:46.366034 16145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4dc6b0541ca431ebc757beac00521f1" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:18:46.366747 16145 raft_consensus.cc:3060] T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:46.370836 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4dc6b0541ca431ebc757beac00521f1" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:18:46.371511 16225 raft_consensus.cc:3060] T e4dc6b0541ca431ebc757beac00521f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:46.371925 16681 raft_consensus.cc:493] T 66f75124215f4ca6bef831e174a792d8 P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:46.372539 16681 raft_consensus.cc:515] T 66f75124215f4ca6bef831e174a792d8 P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:46.370282 16703 leader_election.cc:290] T e4dc6b0541ca431ebc757beac00521f1 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 2 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:18:46.391728 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66f75124215f4ca6bef831e174a792d8" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:18:46.435631 16225 raft_consensus.cc:2468] T e4dc6b0541ca431ebc757beac00521f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 2.
I20260709 13:18:46.437243 16265 leader_election.cc:304] T e4dc6b0541ca431ebc757beac00521f1 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:18:46.438931 16676 raft_consensus.cc:493] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:46.439332 16703 raft_consensus.cc:2804] T e4dc6b0541ca431ebc757beac00521f1 P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:18:46.439491 16676 raft_consensus.cc:515] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:46.440049 16703 raft_consensus.cc:697] T e4dc6b0541ca431ebc757beac00521f1 P 90f3da57796b488c9087a2f523adf045 [term 2 LEADER]: Becoming Leader. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:18:46.440745 16520 raft_consensus.cc:515] T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:46.441663 16676 leader_election.cc:290] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:18:46.438990 16145 raft_consensus.cc:2468] T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 2.
I20260709 13:18:46.445837 16226 raft_consensus.cc:2468] T 66f75124215f4ca6bef831e174a792d8 P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 0.
I20260709 13:18:46.445604 16703 consensus_queue.cc:237] T e4dc6b0541ca431ebc757beac00521f1 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:46.446838 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e4bfb9090c8460eb14fa7e0d818e7f1" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:18:46.447399 16300 raft_consensus.cc:2468] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 0.
I20260709 13:18:46.449659 16681 leader_election.cc:290] T 66f75124215f4ca6bef831e174a792d8 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:18:46.453886 16520 leader_election.cc:290] T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 1 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:18:46.453979 16190 leader_election.cc:304] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:18:46.455629 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66f75124215f4ca6bef831e174a792d8" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:18:46.461930 16300 raft_consensus.cc:2468] T 66f75124215f4ca6bef831e174a792d8 P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 0.
I20260709 13:18:46.463351 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66f75124215f4ca6bef831e174a792d8" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:18:46.463877 16300 raft_consensus.cc:3060] T 66f75124215f4ca6bef831e174a792d8 P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:46.466866 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66f75124215f4ca6bef831e174a792d8" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:18:46.467358 16225 raft_consensus.cc:3060] T 66f75124215f4ca6bef831e174a792d8 P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:46.469549 16300 raft_consensus.cc:2468] T 66f75124215f4ca6bef831e174a792d8 P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 1.
I20260709 13:18:46.472074 16225 raft_consensus.cc:2468] T 66f75124215f4ca6bef831e174a792d8 P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 1.
I20260709 13:18:46.472923 16520 raft_consensus.cc:493] T 23f82224f4564fea93572e8206bad1fe P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:46.473325 16145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e4bfb9090c8460eb14fa7e0d818e7f1" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:18:46.473464 16520 raft_consensus.cc:515] T 23f82224f4564fea93572e8206bad1fe P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:46.473960 16145 raft_consensus.cc:2468] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 0.
I20260709 13:18:46.475317 16520 leader_election.cc:290] T 23f82224f4564fea93572e8206bad1fe P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:18:46.475958 16223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f82224f4564fea93572e8206bad1fe" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:18:46.476522 16223 raft_consensus.cc:2468] T 23f82224f4564fea93572e8206bad1fe P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 0.
I20260709 13:18:46.477435 16116 leader_election.cc:304] T 23f82224f4564fea93572e8206bad1fe P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
I20260709 13:18:46.478083 16520 raft_consensus.cc:2804] T 23f82224f4564fea93572e8206bad1fe P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:18:46.478394 16520 raft_consensus.cc:493] T 23f82224f4564fea93572e8206bad1fe P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:46.478680 16520 raft_consensus.cc:3060] T 23f82224f4564fea93572e8206bad1fe P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:46.480041 16145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66f75124215f4ca6bef831e174a792d8" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:18:46.480705 16145 raft_consensus.cc:2393] T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 1: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
I20260709 13:18:46.482245 16115 leader_election.cc:304] T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:18:46.482908 16630 raft_consensus.cc:2804] T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:18:46.483359 16630 raft_consensus.cc:697] T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b [term 1 LEADER]: Becoming Leader. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:18:46.484165 16630 consensus_queue.cc:237] T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:46.484726 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f82224f4564fea93572e8206bad1fe" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:18:46.485317 16300 raft_consensus.cc:2468] T 23f82224f4564fea93572e8206bad1fe P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 0.
I20260709 13:18:46.490798 16676 raft_consensus.cc:2804] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:18:46.491194 16676 raft_consensus.cc:493] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:46.491537 16676 raft_consensus.cc:3060] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:46.492466 16190 leader_election.cc:304] T 66f75124215f4ca6bef831e174a792d8 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:18:46.497114 16520 raft_consensus.cc:515] T 23f82224f4564fea93572e8206bad1fe P 647a343a542d4a139fe106c19095129b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:46.497923 16676 raft_consensus.cc:515] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:46.499776 16658 raft_consensus.cc:2764] T 66f75124215f4ca6bef831e174a792d8 P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Leader pre-election decision vote started in defunct term 0: won
I20260709 13:18:46.499995 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f82224f4564fea93572e8206bad1fe" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:18:46.500610 16225 raft_consensus.cc:3060] T 23f82224f4564fea93572e8206bad1fe P 1ca2725d290e47d68b1a4b609c9816ac [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:46.502947 16048 catalog_manager.cc:5697] T e4dc6b0541ca431ebc757beac00521f1 P 90f3da57796b488c9087a2f523adf045 reported cstate change: term changed from 0 to 2, leader changed from <none> to 90f3da57796b488c9087a2f523adf045 (127.14.177.131). New cstate: current_term: 2 leader_uuid: "90f3da57796b488c9087a2f523adf045" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } health_report { overall_health: HEALTHY } } }
I20260709 13:18:46.504866 16520 leader_election.cc:290] T 23f82224f4564fea93572e8206bad1fe P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 1 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:18:46.507362 16737 consensus_queue.cc:1048] T 84f2da98864b4425bf7c0055951fdd4b P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:18:46.509734 16658 consensus_queue.cc:1048] T 84f2da98864b4425bf7c0055951fdd4b P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:18:46.510015 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e4bfb9090c8460eb14fa7e0d818e7f1" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:18:46.510560 16299 raft_consensus.cc:3060] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:46.512950 16145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e4bfb9090c8460eb14fa7e0d818e7f1" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:18:46.513695 16145 raft_consensus.cc:3060] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 647a343a542d4a139fe106c19095129b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:46.517620 16045 catalog_manager.cc:5697] T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b reported cstate change: term changed from 0 to 1, leader changed from <none> to 647a343a542d4a139fe106c19095129b (127.14.177.129). New cstate: current_term: 1 leader_uuid: "647a343a542d4a139fe106c19095129b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:46.521605 16225 raft_consensus.cc:2468] T 23f82224f4564fea93572e8206bad1fe P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 1.
I20260709 13:18:46.499790 16676 leader_election.cc:290] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 1 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:18:46.523178 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f82224f4564fea93572e8206bad1fe" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:18:46.523674 16300 raft_consensus.cc:3060] T 23f82224f4564fea93572e8206bad1fe P 90f3da57796b488c9087a2f523adf045 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:46.521389 16299 raft_consensus.cc:2468] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 1.
I20260709 13:18:46.528772 16116 leader_election.cc:304] T 23f82224f4564fea93572e8206bad1fe P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
I20260709 13:18:46.529501 16651 raft_consensus.cc:2804] T 23f82224f4564fea93572e8206bad1fe P 647a343a542d4a139fe106c19095129b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:18:46.529441 16190 leader_election.cc:304] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:18:46.530092 16651 raft_consensus.cc:697] T 23f82224f4564fea93572e8206bad1fe P 647a343a542d4a139fe106c19095129b [term 1 LEADER]: Becoming Leader. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:18:46.530906 16651 consensus_queue.cc:237] T 23f82224f4564fea93572e8206bad1fe P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:46.535384 16145 raft_consensus.cc:2468] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 647a343a542d4a139fe106c19095129b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 1.
I20260709 13:18:46.538753 16676 raft_consensus.cc:2804] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:18:46.542852 16300 raft_consensus.cc:2468] T 23f82224f4564fea93572e8206bad1fe P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 1.
I20260709 13:18:46.544648 16676 raft_consensus.cc:697] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 1 LEADER]: Becoming Leader. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:18:46.554667 16520 consensus_queue.cc:1048] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:18:46.558674 16676 consensus_queue.cc:237] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:18:46.572043 16521 consensus_queue.cc:1048] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:18:46.576009 16046 catalog_manager.cc:5697] T 23f82224f4564fea93572e8206bad1fe P 647a343a542d4a139fe106c19095129b reported cstate change: term changed from 0 to 1, leader changed from <none> to 647a343a542d4a139fe106c19095129b (127.14.177.129). New cstate: current_term: 1 leader_uuid: "647a343a542d4a139fe106c19095129b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:46.595854 16045 catalog_manager.cc:5697] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 1ca2725d290e47d68b1a4b609c9816ac reported cstate change: term changed from 0 to 1, leader changed from <none> to 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130). New cstate: current_term: 1 leader_uuid: "1ca2725d290e47d68b1a4b609c9816ac" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:46.868036 16225 raft_consensus.cc:1275] T e4dc6b0541ca431ebc757beac00521f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 13:18:46.869262 16703 consensus_queue.cc:1048] T e4dc6b0541ca431ebc757beac00521f1 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:18:46.880939 16150 raft_consensus.cc:1275] T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 13:18:46.882474 16634 consensus_queue.cc:1048] T e4dc6b0541ca431ebc757beac00521f1 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:18:46.920671 16651 consensus_queue.cc:1048] T 23f82224f4564fea93572e8206bad1fe P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:18:46.931619 16521 consensus_queue.cc:1048] T 23f82224f4564fea93572e8206bad1fe P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:18:46.984975 16520 consensus_queue.cc:1048] T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:18:47.000584 16521 consensus_queue.cc:1048] T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:18:47.105603 16658 consensus_queue.cc:1048] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:18:47.115621 16737 consensus_queue.cc:1048] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:18:48.467449 15046 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:18:48.501560 16780 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:18:48.503701 16779 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:18:48.505622 15046 server_base.cc:1061] running on GCE node
W20260709 13:18:48.506022 16782 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:18:48.507335 15046 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:18:48.507552 15046 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:18:48.507706 15046 hybrid_clock.cc:648] HybridClock initialized: now 1783603128507693 us; error 0 us; skew 500 ppm
I20260709 13:18:48.508220 15046 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:18:48.511344 15046 webserver.cc:533] Webserver started at http://127.14.177.132:39121/ using document root <none> and password file <none>
I20260709 13:18:48.511845 15046 fs_manager.cc:362] Metadata directory not provided
I20260709 13:18:48.512022 15046 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:18:48.512316 15046 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:18:48.513537 15046 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskdH_Ma7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783603093801613-15046-0/minicluster-data/ts-3-root/instance:
uuid: "cb82cc7bd6db46d980c75237a1402860"
format_stamp: "Formatted at 2026-07-09 13:18:48 on dist-test-slave-v5z2"
I20260709 13:18:48.518654 15046 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.003s	sys 0.004s
I20260709 13:18:48.523690 16787 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:18:48.524679 15046 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.001s
I20260709 13:18:48.524976 15046 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskdH_Ma7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783603093801613-15046-0/minicluster-data/ts-3-root
uuid: "cb82cc7bd6db46d980c75237a1402860"
format_stamp: "Formatted at 2026-07-09 13:18:48 on dist-test-slave-v5z2"
I20260709 13:18:48.525249 15046 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdH_Ma7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783603093801613-15046-0/minicluster-data/ts-3-root
metadata directory: /tmp/dist-test-taskdH_Ma7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783603093801613-15046-0/minicluster-data/ts-3-root
1 data directories: /tmp/dist-test-taskdH_Ma7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783603093801613-15046-0/minicluster-data/ts-3-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:18:48.547377 15046 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:18:48.549335 15046 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:18:48.551482 15046 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:18:48.554855 15046 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:18:48.555120 15046 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:18:48.555457 15046 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:18:48.555650 15046 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:18:48.640237 15046 rpc_server.cc:307] RPC server started. Bound to: 127.14.177.132:38923
I20260709 13:18:48.640365 16849 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.14.177.132:38923 every 8 connection(s)
I20260709 13:18:48.645998 15046 auto_leader_rebalancer-test.cc:384] add a tserver: cb82cc7bd6db46d980c75237a1402860
I20260709 13:18:48.661835 16850 heartbeater.cc:344] Connected to a master server at 127.14.177.190:44079
I20260709 13:18:48.662329 16850 heartbeater.cc:461] Registering TS with master...
I20260709 13:18:48.663266 16850 heartbeater.cc:507] Master 127.14.177.190:44079 requested a full tablet report, sending...
I20260709 13:18:48.665349 16045 ts_manager.cc:194] Registered new tserver with Master: cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:18:48.667119 16045 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:58400
I20260709 13:18:49.175232 16296 consensus_queue.cc:237] T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } }
I20260709 13:18:49.186185 16223 raft_consensus.cc:1275] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:18:49.196390 16151 raft_consensus.cc:1275] T 74ca8140f35c4ae29e8a664f6bdfc829 P 647a343a542d4a139fe106c19095129b [term 1 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:18:49.197494 16859 consensus_queue.cc:1048] T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:18:49.199887 16629 consensus_queue.cc:1048] T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:18:49.219029 16266 consensus_peers.cc:597] T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045 -> Peer cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923): Couldn't send request to peer cb82cc7bd6db46d980c75237a1402860. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 74ca8140f35c4ae29e8a664f6bdfc829. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:18:49.221509 16858 raft_consensus.cc:2955] T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } } }
I20260709 13:18:49.223279 16151 raft_consensus.cc:2955] T 74ca8140f35c4ae29e8a664f6bdfc829 P 647a343a542d4a139fe106c19095129b [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } } }
I20260709 13:18:49.224288 16223 raft_consensus.cc:2955] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } } }
I20260709 13:18:49.247902 16045 catalog_manager.cc:5697] T 74ca8140f35c4ae29e8a664f6bdfc829 P 647a343a542d4a139fe106c19095129b reported cstate change: config changed from index -1 to 2, NON_VOTER cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) added. New cstate: current_term: 1 leader_uuid: "90f3da57796b488c9087a2f523adf045" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } } }
I20260709 13:18:49.276813 16151 consensus_queue.cc:237] T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } }
I20260709 13:18:49.287863 16296 raft_consensus.cc:1275] T 66f75124215f4ca6bef831e174a792d8 P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:18:49.289314 16775 consensus_queue.cc:1048] T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:18:49.289888 16223 raft_consensus.cc:1275] T 66f75124215f4ca6bef831e174a792d8 P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:18:49.295352 16775 consensus_queue.cc:1048] T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:18:49.307700 16854 raft_consensus.cc:2955] T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } } }
I20260709 13:18:49.312525 16296 raft_consensus.cc:2955] T 66f75124215f4ca6bef831e174a792d8 P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } } }
W20260709 13:18:49.313601 16117 consensus_peers.cc:597] T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b -> Peer cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923): Couldn't send request to peer cb82cc7bd6db46d980c75237a1402860. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 66f75124215f4ca6bef831e174a792d8. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:18:49.319137 16223 raft_consensus.cc:2955] T 66f75124215f4ca6bef831e174a792d8 P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } } }
I20260709 13:18:49.324426 16299 consensus_queue.cc:237] T b87147731a68400e8cce9b879daa7c5d P 90f3da57796b488c9087a2f523adf045 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } }
I20260709 13:18:49.329488 16046 catalog_manager.cc:5697] T 66f75124215f4ca6bef831e174a792d8 P 90f3da57796b488c9087a2f523adf045 reported cstate change: config changed from index -1 to 2, NON_VOTER cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) added. New cstate: current_term: 1 leader_uuid: "647a343a542d4a139fe106c19095129b" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } } }
I20260709 13:18:49.334097 16226 raft_consensus.cc:1275] T b87147731a68400e8cce9b879daa7c5d P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 13:18:49.334419 16266 consensus_peers.cc:597] T b87147731a68400e8cce9b879daa7c5d P 90f3da57796b488c9087a2f523adf045 -> Peer cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923): Couldn't send request to peer cb82cc7bd6db46d980c75237a1402860. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b87147731a68400e8cce9b879daa7c5d. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:18:49.335480 16151 raft_consensus.cc:1275] T b87147731a68400e8cce9b879daa7c5d P 647a343a542d4a139fe106c19095129b [term 1 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:18:49.337816 16629 consensus_queue.cc:1048] T b87147731a68400e8cce9b879daa7c5d P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:18:49.339991 16862 consensus_queue.cc:1048] T b87147731a68400e8cce9b879daa7c5d P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:18:49.360843 16866 raft_consensus.cc:2955] T b87147731a68400e8cce9b879daa7c5d P 90f3da57796b488c9087a2f523adf045 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } } }
I20260709 13:18:49.364538 16150 raft_consensus.cc:2955] T b87147731a68400e8cce9b879daa7c5d P 647a343a542d4a139fe106c19095129b [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } } }
I20260709 13:18:49.375495 16223 raft_consensus.cc:2955] T b87147731a68400e8cce9b879daa7c5d P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } } }
I20260709 13:18:49.381009 16045 catalog_manager.cc:5697] T b87147731a68400e8cce9b879daa7c5d P 90f3da57796b488c9087a2f523adf045 reported cstate change: config changed from index -1 to 2, NON_VOTER cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) added. New cstate: current_term: 1 leader_uuid: "90f3da57796b488c9087a2f523adf045" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:49.407212 16223 consensus_queue.cc:237] T d2ab0ac8347641e6955e9bce7d90c1a2 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } }
I20260709 13:18:49.420051 16296 raft_consensus.cc:1275] T d2ab0ac8347641e6955e9bce7d90c1a2 P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:18:49.421379 16150 raft_consensus.cc:1275] T d2ab0ac8347641e6955e9bce7d90c1a2 P 647a343a542d4a139fe106c19095129b [term 1 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:18:49.423123 16777 consensus_queue.cc:1048] T d2ab0ac8347641e6955e9bce7d90c1a2 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:18:49.424743 16778 consensus_queue.cc:1048] T d2ab0ac8347641e6955e9bce7d90c1a2 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:18:49.440426 16778 raft_consensus.cc:2955] T d2ab0ac8347641e6955e9bce7d90c1a2 P 1ca2725d290e47d68b1a4b609c9816ac [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } } }
W20260709 13:18:49.453226 16192 consensus_peers.cc:597] T d2ab0ac8347641e6955e9bce7d90c1a2 P 1ca2725d290e47d68b1a4b609c9816ac -> Peer cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923): Couldn't send request to peer cb82cc7bd6db46d980c75237a1402860. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d2ab0ac8347641e6955e9bce7d90c1a2. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:18:49.482333 16151 raft_consensus.cc:2955] T d2ab0ac8347641e6955e9bce7d90c1a2 P 647a343a542d4a139fe106c19095129b [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } } }
I20260709 13:18:49.512382 16296 raft_consensus.cc:2955] T d2ab0ac8347641e6955e9bce7d90c1a2 P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } } }
I20260709 13:18:49.509864 16046 catalog_manager.cc:5697] T d2ab0ac8347641e6955e9bce7d90c1a2 P 647a343a542d4a139fe106c19095129b reported cstate change: config changed from index -1 to 2, NON_VOTER cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) added. New cstate: current_term: 1 leader_uuid: "1ca2725d290e47d68b1a4b609c9816ac" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } } }
I20260709 13:18:49.669901 16850 heartbeater.cc:499] Master 127.14.177.190:44079 was elected leader, sending a full tablet report...
I20260709 13:18:49.832513 16887 ts_tablet_manager.cc:933] T 66f75124215f4ca6bef831e174a792d8 P cb82cc7bd6db46d980c75237a1402860: Initiating tablet copy from peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741)
I20260709 13:18:49.835110 16887 tablet_copy_client.cc:323] T 66f75124215f4ca6bef831e174a792d8 P cb82cc7bd6db46d980c75237a1402860: tablet copy: Beginning tablet copy session from remote peer at address 127.14.177.129:36741
I20260709 13:18:49.941013 16894 ts_tablet_manager.cc:933] T b87147731a68400e8cce9b879daa7c5d P cb82cc7bd6db46d980c75237a1402860: Initiating tablet copy from peer 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:18:50.021958 16894 tablet_copy_client.cc:323] T b87147731a68400e8cce9b879daa7c5d P cb82cc7bd6db46d980c75237a1402860: tablet copy: Beginning tablet copy session from remote peer at address 127.14.177.131:41389
I20260709 13:18:50.059670 16901 ts_tablet_manager.cc:933] T 74ca8140f35c4ae29e8a664f6bdfc829 P cb82cc7bd6db46d980c75237a1402860: Initiating tablet copy from peer 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:18:50.073583 16903 ts_tablet_manager.cc:933] T d2ab0ac8347641e6955e9bce7d90c1a2 P cb82cc7bd6db46d980c75237a1402860: Initiating tablet copy from peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:18:50.079504 16903 tablet_copy_client.cc:323] T d2ab0ac8347641e6955e9bce7d90c1a2 P cb82cc7bd6db46d980c75237a1402860: tablet copy: Beginning tablet copy session from remote peer at address 127.14.177.130:35943
I20260709 13:18:50.080358 16901 tablet_copy_client.cc:323] T 74ca8140f35c4ae29e8a664f6bdfc829 P cb82cc7bd6db46d980c75237a1402860: tablet copy: Beginning tablet copy session from remote peer at address 127.14.177.131:41389
I20260709 13:18:50.096652 16310 tablet_copy_service.cc:140] P 90f3da57796b488c9087a2f523adf045: Received BeginTabletCopySession request for tablet b87147731a68400e8cce9b879daa7c5d from peer cb82cc7bd6db46d980c75237a1402860 ({username='slave'} at 127.0.0.1:40474)
I20260709 13:18:50.097251 16310 tablet_copy_service.cc:161] P 90f3da57796b488c9087a2f523adf045: Beginning new tablet copy session on tablet b87147731a68400e8cce9b879daa7c5d from peer cb82cc7bd6db46d980c75237a1402860 at {username='slave'} at 127.0.0.1:40474: session id = cb82cc7bd6db46d980c75237a1402860-b87147731a68400e8cce9b879daa7c5d
I20260709 13:18:50.097604 16309 tablet_copy_service.cc:140] P 90f3da57796b488c9087a2f523adf045: Received BeginTabletCopySession request for tablet 74ca8140f35c4ae29e8a664f6bdfc829 from peer cb82cc7bd6db46d980c75237a1402860 ({username='slave'} at 127.0.0.1:40474)
I20260709 13:18:50.098131 16309 tablet_copy_service.cc:161] P 90f3da57796b488c9087a2f523adf045: Beginning new tablet copy session on tablet 74ca8140f35c4ae29e8a664f6bdfc829 from peer cb82cc7bd6db46d980c75237a1402860 at {username='slave'} at 127.0.0.1:40474: session id = cb82cc7bd6db46d980c75237a1402860-74ca8140f35c4ae29e8a664f6bdfc829
I20260709 13:18:50.105475 16309 tablet_copy_source_session.cc:215] T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:18:50.105475 16310 tablet_copy_source_session.cc:215] T b87147731a68400e8cce9b879daa7c5d P 90f3da57796b488c9087a2f523adf045: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:18:50.112242 16901 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 74ca8140f35c4ae29e8a664f6bdfc829. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:50.114593 16161 tablet_copy_service.cc:140] P 647a343a542d4a139fe106c19095129b: Received BeginTabletCopySession request for tablet 66f75124215f4ca6bef831e174a792d8 from peer cb82cc7bd6db46d980c75237a1402860 ({username='slave'} at 127.0.0.1:46224)
I20260709 13:18:50.115509 16161 tablet_copy_service.cc:161] P 647a343a542d4a139fe106c19095129b: Beginning new tablet copy session on tablet 66f75124215f4ca6bef831e174a792d8 from peer cb82cc7bd6db46d980c75237a1402860 at {username='slave'} at 127.0.0.1:46224: session id = cb82cc7bd6db46d980c75237a1402860-66f75124215f4ca6bef831e174a792d8
I20260709 13:18:50.116318 16894 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b87147731a68400e8cce9b879daa7c5d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:50.124897 16161 tablet_copy_source_session.cc:215] T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:18:50.128383 16887 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 66f75124215f4ca6bef831e174a792d8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:50.142993 16236 tablet_copy_service.cc:140] P 1ca2725d290e47d68b1a4b609c9816ac: Received BeginTabletCopySession request for tablet d2ab0ac8347641e6955e9bce7d90c1a2 from peer cb82cc7bd6db46d980c75237a1402860 ({username='slave'} at 127.0.0.1:33050)
I20260709 13:18:50.143683 16236 tablet_copy_service.cc:161] P 1ca2725d290e47d68b1a4b609c9816ac: Beginning new tablet copy session on tablet d2ab0ac8347641e6955e9bce7d90c1a2 from peer cb82cc7bd6db46d980c75237a1402860 at {username='slave'} at 127.0.0.1:33050: session id = cb82cc7bd6db46d980c75237a1402860-d2ab0ac8347641e6955e9bce7d90c1a2
I20260709 13:18:50.147053 16887 tablet_copy_client.cc:806] T 66f75124215f4ca6bef831e174a792d8 P cb82cc7bd6db46d980c75237a1402860: tablet copy: Starting download of 0 data blocks...
I20260709 13:18:50.147832 16887 tablet_copy_client.cc:670] T 66f75124215f4ca6bef831e174a792d8 P cb82cc7bd6db46d980c75237a1402860: tablet copy: Starting download of 1 WAL segments...
I20260709 13:18:50.151458 16236 tablet_copy_source_session.cc:215] T d2ab0ac8347641e6955e9bce7d90c1a2 P 1ca2725d290e47d68b1a4b609c9816ac: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:18:50.153640 16887 tablet_copy_client.cc:538] T 66f75124215f4ca6bef831e174a792d8 P cb82cc7bd6db46d980c75237a1402860: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:18:50.154891 16903 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d2ab0ac8347641e6955e9bce7d90c1a2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:50.163026 16887 tablet_bootstrap.cc:492] T 66f75124215f4ca6bef831e174a792d8 P cb82cc7bd6db46d980c75237a1402860: Bootstrap starting.
I20260709 13:18:50.163233 16894 tablet_copy_client.cc:806] T b87147731a68400e8cce9b879daa7c5d P cb82cc7bd6db46d980c75237a1402860: tablet copy: Starting download of 0 data blocks...
I20260709 13:18:50.172039 16894 tablet_copy_client.cc:670] T b87147731a68400e8cce9b879daa7c5d P cb82cc7bd6db46d980c75237a1402860: tablet copy: Starting download of 1 WAL segments...
I20260709 13:18:50.172634 16901 tablet_copy_client.cc:806] T 74ca8140f35c4ae29e8a664f6bdfc829 P cb82cc7bd6db46d980c75237a1402860: tablet copy: Starting download of 0 data blocks...
I20260709 13:18:50.173213 16901 tablet_copy_client.cc:670] T 74ca8140f35c4ae29e8a664f6bdfc829 P cb82cc7bd6db46d980c75237a1402860: tablet copy: Starting download of 1 WAL segments...
I20260709 13:18:50.175319 16903 tablet_copy_client.cc:806] T d2ab0ac8347641e6955e9bce7d90c1a2 P cb82cc7bd6db46d980c75237a1402860: tablet copy: Starting download of 0 data blocks...
I20260709 13:18:50.175930 16903 tablet_copy_client.cc:670] T d2ab0ac8347641e6955e9bce7d90c1a2 P cb82cc7bd6db46d980c75237a1402860: tablet copy: Starting download of 1 WAL segments...
I20260709 13:18:50.177089 16901 tablet_copy_client.cc:538] T 74ca8140f35c4ae29e8a664f6bdfc829 P cb82cc7bd6db46d980c75237a1402860: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:18:50.178448 16894 tablet_copy_client.cc:538] T b87147731a68400e8cce9b879daa7c5d P cb82cc7bd6db46d980c75237a1402860: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:18:50.188594 16903 tablet_copy_client.cc:538] T d2ab0ac8347641e6955e9bce7d90c1a2 P cb82cc7bd6db46d980c75237a1402860: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:18:50.207835 16887 tablet_bootstrap.cc:492] T 66f75124215f4ca6bef831e174a792d8 P cb82cc7bd6db46d980c75237a1402860: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:18:50.208945 16887 tablet_bootstrap.cc:492] T 66f75124215f4ca6bef831e174a792d8 P cb82cc7bd6db46d980c75237a1402860: Bootstrap complete.
I20260709 13:18:50.209497 16887 ts_tablet_manager.cc:1403] T 66f75124215f4ca6bef831e174a792d8 P cb82cc7bd6db46d980c75237a1402860: Time spent bootstrapping tablet: real 0.047s	user 0.015s	sys 0.017s
I20260709 13:18:50.209614 16894 tablet_bootstrap.cc:492] T b87147731a68400e8cce9b879daa7c5d P cb82cc7bd6db46d980c75237a1402860: Bootstrap starting.
I20260709 13:18:50.212203 16887 raft_consensus.cc:359] T 66f75124215f4ca6bef831e174a792d8 P cb82cc7bd6db46d980c75237a1402860 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } }
I20260709 13:18:50.213021 16903 tablet_bootstrap.cc:492] T d2ab0ac8347641e6955e9bce7d90c1a2 P cb82cc7bd6db46d980c75237a1402860: Bootstrap starting.
I20260709 13:18:50.213125 16887 raft_consensus.cc:740] T 66f75124215f4ca6bef831e174a792d8 P cb82cc7bd6db46d980c75237a1402860 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Initialized, Role: LEARNER
I20260709 13:18:50.213971 16887 consensus_queue.cc:260] T 66f75124215f4ca6bef831e174a792d8 P cb82cc7bd6db46d980c75237a1402860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } }
I20260709 13:18:50.217850 16887 ts_tablet_manager.cc:1434] T 66f75124215f4ca6bef831e174a792d8 P cb82cc7bd6db46d980c75237a1402860: Time spent starting tablet: real 0.008s	user 0.000s	sys 0.008s
I20260709 13:18:50.257553 16161 tablet_copy_service.cc:342] P 647a343a542d4a139fe106c19095129b: Request end of tablet copy session cb82cc7bd6db46d980c75237a1402860-66f75124215f4ca6bef831e174a792d8 received from {username='slave'} at 127.0.0.1:46224
I20260709 13:18:50.258069 16161 tablet_copy_service.cc:434] P 647a343a542d4a139fe106c19095129b: ending tablet copy session cb82cc7bd6db46d980c75237a1402860-66f75124215f4ca6bef831e174a792d8 on tablet 66f75124215f4ca6bef831e174a792d8 with peer cb82cc7bd6db46d980c75237a1402860
I20260709 13:18:50.271112 16901 tablet_bootstrap.cc:492] T 74ca8140f35c4ae29e8a664f6bdfc829 P cb82cc7bd6db46d980c75237a1402860: Bootstrap starting.
I20260709 13:18:50.287187 16894 tablet_bootstrap.cc:492] T b87147731a68400e8cce9b879daa7c5d P cb82cc7bd6db46d980c75237a1402860: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:18:50.288317 16894 tablet_bootstrap.cc:492] T b87147731a68400e8cce9b879daa7c5d P cb82cc7bd6db46d980c75237a1402860: Bootstrap complete.
I20260709 13:18:50.289196 16894 ts_tablet_manager.cc:1403] T b87147731a68400e8cce9b879daa7c5d P cb82cc7bd6db46d980c75237a1402860: Time spent bootstrapping tablet: real 0.080s	user 0.023s	sys 0.001s
I20260709 13:18:50.292363 16894 raft_consensus.cc:359] T b87147731a68400e8cce9b879daa7c5d P cb82cc7bd6db46d980c75237a1402860 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } }
I20260709 13:18:50.293478 16894 raft_consensus.cc:740] T b87147731a68400e8cce9b879daa7c5d P cb82cc7bd6db46d980c75237a1402860 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Initialized, Role: LEARNER
I20260709 13:18:50.294093 16894 consensus_queue.cc:260] T b87147731a68400e8cce9b879daa7c5d P cb82cc7bd6db46d980c75237a1402860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } }
I20260709 13:18:50.296495 16903 tablet_bootstrap.cc:492] T d2ab0ac8347641e6955e9bce7d90c1a2 P cb82cc7bd6db46d980c75237a1402860: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:18:50.297334 16903 tablet_bootstrap.cc:492] T d2ab0ac8347641e6955e9bce7d90c1a2 P cb82cc7bd6db46d980c75237a1402860: Bootstrap complete.
I20260709 13:18:50.297966 16903 ts_tablet_manager.cc:1403] T d2ab0ac8347641e6955e9bce7d90c1a2 P cb82cc7bd6db46d980c75237a1402860: Time spent bootstrapping tablet: real 0.085s	user 0.014s	sys 0.011s
I20260709 13:18:50.301067 16903 raft_consensus.cc:359] T d2ab0ac8347641e6955e9bce7d90c1a2 P cb82cc7bd6db46d980c75237a1402860 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } }
I20260709 13:18:50.301923 16903 raft_consensus.cc:740] T d2ab0ac8347641e6955e9bce7d90c1a2 P cb82cc7bd6db46d980c75237a1402860 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Initialized, Role: LEARNER
I20260709 13:18:50.302951 16901 tablet_bootstrap.cc:492] T 74ca8140f35c4ae29e8a664f6bdfc829 P cb82cc7bd6db46d980c75237a1402860: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:18:50.302563 16903 consensus_queue.cc:260] T d2ab0ac8347641e6955e9bce7d90c1a2 P cb82cc7bd6db46d980c75237a1402860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } }
I20260709 13:18:50.303949 16901 tablet_bootstrap.cc:492] T 74ca8140f35c4ae29e8a664f6bdfc829 P cb82cc7bd6db46d980c75237a1402860: Bootstrap complete.
I20260709 13:18:50.304652 16901 ts_tablet_manager.cc:1403] T 74ca8140f35c4ae29e8a664f6bdfc829 P cb82cc7bd6db46d980c75237a1402860: Time spent bootstrapping tablet: real 0.034s	user 0.015s	sys 0.016s
I20260709 13:18:50.305083 16903 ts_tablet_manager.cc:1434] T d2ab0ac8347641e6955e9bce7d90c1a2 P cb82cc7bd6db46d980c75237a1402860: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 13:18:50.301513 16894 ts_tablet_manager.cc:1434] T b87147731a68400e8cce9b879daa7c5d P cb82cc7bd6db46d980c75237a1402860: Time spent starting tablet: real 0.012s	user 0.007s	sys 0.001s
I20260709 13:18:50.310037 16236 tablet_copy_service.cc:342] P 1ca2725d290e47d68b1a4b609c9816ac: Request end of tablet copy session cb82cc7bd6db46d980c75237a1402860-d2ab0ac8347641e6955e9bce7d90c1a2 received from {username='slave'} at 127.0.0.1:33050
I20260709 13:18:50.310454 16236 tablet_copy_service.cc:434] P 1ca2725d290e47d68b1a4b609c9816ac: ending tablet copy session cb82cc7bd6db46d980c75237a1402860-d2ab0ac8347641e6955e9bce7d90c1a2 on tablet d2ab0ac8347641e6955e9bce7d90c1a2 with peer cb82cc7bd6db46d980c75237a1402860
I20260709 13:18:50.310806 16310 tablet_copy_service.cc:342] P 90f3da57796b488c9087a2f523adf045: Request end of tablet copy session cb82cc7bd6db46d980c75237a1402860-b87147731a68400e8cce9b879daa7c5d received from {username='slave'} at 127.0.0.1:40474
I20260709 13:18:50.310276 16901 raft_consensus.cc:359] T 74ca8140f35c4ae29e8a664f6bdfc829 P cb82cc7bd6db46d980c75237a1402860 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } }
I20260709 13:18:50.311220 16310 tablet_copy_service.cc:434] P 90f3da57796b488c9087a2f523adf045: ending tablet copy session cb82cc7bd6db46d980c75237a1402860-b87147731a68400e8cce9b879daa7c5d on tablet b87147731a68400e8cce9b879daa7c5d with peer cb82cc7bd6db46d980c75237a1402860
I20260709 13:18:50.311532 16901 raft_consensus.cc:740] T 74ca8140f35c4ae29e8a664f6bdfc829 P cb82cc7bd6db46d980c75237a1402860 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Initialized, Role: LEARNER
I20260709 13:18:50.312332 16901 consensus_queue.cc:260] T 74ca8140f35c4ae29e8a664f6bdfc829 P cb82cc7bd6db46d980c75237a1402860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } }
I20260709 13:18:50.315001 16901 ts_tablet_manager.cc:1434] T 74ca8140f35c4ae29e8a664f6bdfc829 P cb82cc7bd6db46d980c75237a1402860: Time spent starting tablet: real 0.010s	user 0.005s	sys 0.000s
I20260709 13:18:50.316558 16310 tablet_copy_service.cc:342] P 90f3da57796b488c9087a2f523adf045: Request end of tablet copy session cb82cc7bd6db46d980c75237a1402860-74ca8140f35c4ae29e8a664f6bdfc829 received from {username='slave'} at 127.0.0.1:40474
I20260709 13:18:50.316910 16310 tablet_copy_service.cc:434] P 90f3da57796b488c9087a2f523adf045: ending tablet copy session cb82cc7bd6db46d980c75237a1402860-74ca8140f35c4ae29e8a664f6bdfc829 on tablet 74ca8140f35c4ae29e8a664f6bdfc829 with peer cb82cc7bd6db46d980c75237a1402860
I20260709 13:18:50.343856 16825 raft_consensus.cc:1217] T 66f75124215f4ca6bef831e174a792d8 P cb82cc7bd6db46d980c75237a1402860 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:18:50.453519 16825 raft_consensus.cc:1217] T b87147731a68400e8cce9b879daa7c5d P cb82cc7bd6db46d980c75237a1402860 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:18:50.468680 16825 raft_consensus.cc:1217] T 74ca8140f35c4ae29e8a664f6bdfc829 P cb82cc7bd6db46d980c75237a1402860 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:18:50.608597 16825 raft_consensus.cc:1217] T d2ab0ac8347641e6955e9bce7d90c1a2 P cb82cc7bd6db46d980c75237a1402860 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
W20260709 13:18:50.890941 16321 debug-util.cc:398] Leaking SignalData structure 0x7b080000e680 after lost signal to thread 15049
W20260709 13:18:50.892108 16321 debug-util.cc:398] Leaking SignalData structure 0x7b0800133960 after lost signal to thread 16083
W20260709 13:18:50.896467 16321 debug-util.cc:398] Leaking SignalData structure 0x7b08002dec60 after lost signal to thread 16849
I20260709 13:18:50.904520 16854 raft_consensus.cc:1064] T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b: attempting to promote NON_VOTER cb82cc7bd6db46d980c75237a1402860 to VOTER
I20260709 13:18:50.905568 16629 raft_consensus.cc:1064] T b87147731a68400e8cce9b879daa7c5d P 90f3da57796b488c9087a2f523adf045: attempting to promote NON_VOTER cb82cc7bd6db46d980c75237a1402860 to VOTER
I20260709 13:18:50.910162 16629 consensus_queue.cc:237] T b87147731a68400e8cce9b879daa7c5d P 90f3da57796b488c9087a2f523adf045 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:18:50.910851 16854 consensus_queue.cc:237] T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:18:50.928612 16824 raft_consensus.cc:1275] T b87147731a68400e8cce9b879daa7c5d P cb82cc7bd6db46d980c75237a1402860 [term 1 LEARNER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:18:50.929648 16148 raft_consensus.cc:1275] T b87147731a68400e8cce9b879daa7c5d P 647a343a542d4a139fe106c19095129b [term 1 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:18:50.930315 16900 consensus_queue.cc:1048] T b87147731a68400e8cce9b879daa7c5d P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:18:50.931459 16866 consensus_queue.cc:1048] T b87147731a68400e8cce9b879daa7c5d P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:18:50.931670 16223 raft_consensus.cc:1275] T b87147731a68400e8cce9b879daa7c5d P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:18:50.934674 16906 consensus_queue.cc:1048] T b87147731a68400e8cce9b879daa7c5d P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:18:50.990353 16629 raft_consensus.cc:1064] T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045: attempting to promote NON_VOTER cb82cc7bd6db46d980c75237a1402860 to VOTER
I20260709 13:18:50.992772 16629 consensus_queue.cc:237] T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:18:50.993216 16906 raft_consensus.cc:2955] T b87147731a68400e8cce9b879daa7c5d P 90f3da57796b488c9087a2f523adf045 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:18:51.018325 16226 raft_consensus.cc:2955] T b87147731a68400e8cce9b879daa7c5d P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:18:51.011014 16824 raft_consensus.cc:2955] T b87147731a68400e8cce9b879daa7c5d P cb82cc7bd6db46d980c75237a1402860 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:18:51.046981 16150 raft_consensus.cc:2955] T b87147731a68400e8cce9b879daa7c5d P 647a343a542d4a139fe106c19095129b [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:18:51.052910 16884 raft_consensus.cc:1064] T d2ab0ac8347641e6955e9bce7d90c1a2 P 1ca2725d290e47d68b1a4b609c9816ac: attempting to promote NON_VOTER cb82cc7bd6db46d980c75237a1402860 to VOTER
I20260709 13:18:51.055675 16884 consensus_queue.cc:237] T d2ab0ac8347641e6955e9bce7d90c1a2 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:18:51.081301 16046 catalog_manager.cc:5697] T b87147731a68400e8cce9b879daa7c5d P 647a343a542d4a139fe106c19095129b reported cstate change: config changed from index 2 to 3, cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "90f3da57796b488c9087a2f523adf045" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:18:51.158938 16296 raft_consensus.cc:1275] T 66f75124215f4ca6bef831e174a792d8 P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:18:51.154392 16825 raft_consensus.cc:1275] T 74ca8140f35c4ae29e8a664f6bdfc829 P cb82cc7bd6db46d980c75237a1402860 [term 1 LEARNER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:18:51.171033 16145 raft_consensus.cc:1275] T 74ca8140f35c4ae29e8a664f6bdfc829 P 647a343a542d4a139fe106c19095129b [term 1 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:18:51.177599 16224 raft_consensus.cc:1275] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:18:51.180342 16823 raft_consensus.cc:1275] T 66f75124215f4ca6bef831e174a792d8 P cb82cc7bd6db46d980c75237a1402860 [term 1 LEARNER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:18:51.185534 16866 consensus_queue.cc:1048] T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
W20260709 13:18:51.196841 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 64099.8us
W20260709 13:18:51.200559 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 66034.5us
I20260709 13:18:51.204849 16920 consensus_queue.cc:1048] T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:18:51.212893 16927 consensus_queue.cc:1048] T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:18:51.215976 16927 consensus_queue.cc:1048] T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:18:51.223896 16930 consensus_queue.cc:1048] T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
W20260709 13:18:51.264153 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 136383us
I20260709 13:18:51.276778 16822 raft_consensus.cc:1275] T d2ab0ac8347641e6955e9bce7d90c1a2 P cb82cc7bd6db46d980c75237a1402860 [term 1 LEARNER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:18:51.278644 16296 raft_consensus.cc:1275] T d2ab0ac8347641e6955e9bce7d90c1a2 P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:18:51.279366 16226 raft_consensus.cc:1275] T 66f75124215f4ca6bef831e174a792d8 P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:18:51.280129 16884 consensus_queue.cc:1048] T d2ab0ac8347641e6955e9bce7d90c1a2 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:18:51.281275 16926 consensus_queue.cc:1048] T d2ab0ac8347641e6955e9bce7d90c1a2 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:18:51.290278 16930 consensus_queue.cc:1048] T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:18:51.302004 16150 raft_consensus.cc:1275] T d2ab0ac8347641e6955e9bce7d90c1a2 P 647a343a542d4a139fe106c19095129b [term 1 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:18:51.299763 16927 raft_consensus.cc:2955] T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:18:51.304718 16825 raft_consensus.cc:2955] T 74ca8140f35c4ae29e8a664f6bdfc829 P cb82cc7bd6db46d980c75237a1402860 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:18:51.324618 16924 consensus_queue.cc:1048] T d2ab0ac8347641e6955e9bce7d90c1a2 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.002s
I20260709 13:18:51.344637 16150 raft_consensus.cc:2955] T 74ca8140f35c4ae29e8a664f6bdfc829 P 647a343a542d4a139fe106c19095129b [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:18:51.354024 16223 raft_consensus.cc:2955] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
W20260709 13:18:51.410622 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 56665.6us
I20260709 13:18:51.429896 16898 raft_consensus.cc:2955] T d2ab0ac8347641e6955e9bce7d90c1a2 P 1ca2725d290e47d68b1a4b609c9816ac [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:18:51.445768 16822 raft_consensus.cc:2955] T d2ab0ac8347641e6955e9bce7d90c1a2 P cb82cc7bd6db46d980c75237a1402860 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:18:51.453526 16920 raft_consensus.cc:2955] T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:18:51.461621 16296 raft_consensus.cc:2955] T d2ab0ac8347641e6955e9bce7d90c1a2 P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:18:51.470016 16297 consensus_queue.cc:237] T b87147731a68400e8cce9b879daa7c5d P 90f3da57796b488c9087a2f523adf045 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:18:51.473636 16047 catalog_manager.cc:5697] T 74ca8140f35c4ae29e8a664f6bdfc829 P cb82cc7bd6db46d980c75237a1402860 reported cstate change: config changed from index 2 to 3, cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "90f3da57796b488c9087a2f523adf045" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:18:51.478259 16148 raft_consensus.cc:1275] T b87147731a68400e8cce9b879daa7c5d P 647a343a542d4a139fe106c19095129b [term 1 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:18:51.480182 16937 consensus_queue.cc:1048] T b87147731a68400e8cce9b879daa7c5d P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:18:51.483793 16226 raft_consensus.cc:2955] T 66f75124215f4ca6bef831e174a792d8 P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:18:51.489236 16906 raft_consensus.cc:2955] T b87147731a68400e8cce9b879daa7c5d P 90f3da57796b488c9087a2f523adf045 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:18:51.491204 16145 raft_consensus.cc:2955] T b87147731a68400e8cce9b879daa7c5d P 647a343a542d4a139fe106c19095129b [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:18:51.507802 16825 raft_consensus.cc:2955] T 66f75124215f4ca6bef831e174a792d8 P cb82cc7bd6db46d980c75237a1402860 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:18:51.512399 16823 raft_consensus.cc:1275] T b87147731a68400e8cce9b879daa7c5d P cb82cc7bd6db46d980c75237a1402860 [term 1 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:18:51.516103 16048 catalog_manager.cc:5697] T d2ab0ac8347641e6955e9bce7d90c1a2 P 1ca2725d290e47d68b1a4b609c9816ac reported cstate change: config changed from index 2 to 3, cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "1ca2725d290e47d68b1a4b609c9816ac" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:18:51.532737 16034 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet b87147731a68400e8cce9b879daa7c5d with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:18:51.531893 16148 raft_consensus.cc:2955] T d2ab0ac8347641e6955e9bce7d90c1a2 P 647a343a542d4a139fe106c19095129b [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:18:51.539088 16046 catalog_manager.cc:5697] T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b reported cstate change: config changed from index 2 to 3, cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "647a343a542d4a139fe106c19095129b" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:18:51.549067 16923 consensus_queue.cc:1048] T b87147731a68400e8cce9b879daa7c5d P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:18:51.542047 16297 consensus_queue.cc:237] T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:18:51.559335 16225 consensus_queue.cc:237] T d2ab0ac8347641e6955e9bce7d90c1a2 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:18:51.541070 16296 raft_consensus.cc:2955] T 66f75124215f4ca6bef831e174a792d8 P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:18:51.584592 16223 raft_consensus.cc:1275] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:18:51.586617 16906 consensus_queue.cc:1048] T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:18:51.595423 16629 raft_consensus.cc:2955] T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 647a343a542d4a139fe106c19095129b (127.14.177.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:18:51.597699 16224 raft_consensus.cc:2955] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 647a343a542d4a139fe106c19095129b (127.14.177.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:18:51.606145 16034 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 74ca8140f35c4ae29e8a664f6bdfc829 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
W20260709 13:18:51.607398 16266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 61271.5us
I20260709 13:18:51.609753 16825 raft_consensus.cc:1275] T 74ca8140f35c4ae29e8a664f6bdfc829 P cb82cc7bd6db46d980c75237a1402860 [term 1 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:18:51.615767 16825 raft_consensus.cc:2955] T b87147731a68400e8cce9b879daa7c5d P cb82cc7bd6db46d980c75237a1402860 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:18:51.618898 16937 consensus_queue.cc:1048] T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:18:51.619122 16822 raft_consensus.cc:1275] T d2ab0ac8347641e6955e9bce7d90c1a2 P cb82cc7bd6db46d980c75237a1402860 [term 1 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:18:51.626744 16939 consensus_queue.cc:1048] T d2ab0ac8347641e6955e9bce7d90c1a2 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:18:51.627815 16823 raft_consensus.cc:2955] T 74ca8140f35c4ae29e8a664f6bdfc829 P cb82cc7bd6db46d980c75237a1402860 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 647a343a542d4a139fe106c19095129b (127.14.177.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:18:51.636412 16939 raft_consensus.cc:2955] T d2ab0ac8347641e6955e9bce7d90c1a2 P 1ca2725d290e47d68b1a4b609c9816ac [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 647a343a542d4a139fe106c19095129b (127.14.177.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:18:51.639694 16296 raft_consensus.cc:1275] T d2ab0ac8347641e6955e9bce7d90c1a2 P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:18:51.643574 16822 raft_consensus.cc:2955] T d2ab0ac8347641e6955e9bce7d90c1a2 P cb82cc7bd6db46d980c75237a1402860 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 647a343a542d4a139fe106c19095129b (127.14.177.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:18:51.641196 16884 consensus_queue.cc:1048] T d2ab0ac8347641e6955e9bce7d90c1a2 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:18:51.658946 16145 consensus_queue.cc:237] T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:18:51.653949 16047 catalog_manager.cc:5697] T b87147731a68400e8cce9b879daa7c5d P 90f3da57796b488c9087a2f523adf045 reported cstate change: config changed from index 3 to 4, VOTER 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130) evicted. New cstate: current_term: 1 leader_uuid: "90f3da57796b488c9087a2f523adf045" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:51.669070 16824 raft_consensus.cc:1275] T 66f75124215f4ca6bef831e174a792d8 P cb82cc7bd6db46d980c75237a1402860 [term 1 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:18:51.672127 16854 consensus_queue.cc:1048] T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:18:51.678470 16224 raft_consensus.cc:1275] T 66f75124215f4ca6bef831e174a792d8 P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:18:51.686153 16931 consensus_queue.cc:1048] T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.002s
I20260709 13:18:51.696514 16296 raft_consensus.cc:2955] T d2ab0ac8347641e6955e9bce7d90c1a2 P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 647a343a542d4a139fe106c19095129b (127.14.177.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:18:51.726516 16226 raft_consensus.cc:2955] T 66f75124215f4ca6bef831e174a792d8 P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 90f3da57796b488c9087a2f523adf045 (127.14.177.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:18:51.724453 16209 tablet_service.cc:1558] Processing DeleteTablet for tablet b87147731a68400e8cce9b879daa7c5d with delete_type TABLET_DATA_TOMBSTONED (TS 1ca2725d290e47d68b1a4b609c9816ac not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:57598
I20260709 13:18:51.738801 16046 catalog_manager.cc:5697] T 74ca8140f35c4ae29e8a664f6bdfc829 P cb82cc7bd6db46d980c75237a1402860 reported cstate change: config changed from index 3 to 4, VOTER 647a343a542d4a139fe106c19095129b (127.14.177.129) evicted. New cstate: current_term: 1 leader_uuid: "90f3da57796b488c9087a2f523adf045" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:18:51.735769 16823 raft_consensus.cc:2955] T 66f75124215f4ca6bef831e174a792d8 P cb82cc7bd6db46d980c75237a1402860 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 90f3da57796b488c9087a2f523adf045 (127.14.177.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:18:51.719519 16931 raft_consensus.cc:2955] T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 90f3da57796b488c9087a2f523adf045 (127.14.177.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:18:51.763600 16035 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 66f75124215f4ca6bef831e174a792d8 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:18:51.769064 16952 tablet_replica.cc:333] T b87147731a68400e8cce9b879daa7c5d P 1ca2725d290e47d68b1a4b609c9816ac: stopping tablet replica
I20260709 13:18:51.770399 16952 raft_consensus.cc:2243] T b87147731a68400e8cce9b879daa7c5d P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:51.771556 16035 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet d2ab0ac8347641e6955e9bce7d90c1a2 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:18:51.772827 16952 raft_consensus.cc:2272] T b87147731a68400e8cce9b879daa7c5d P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:51.777128 16952 ts_tablet_manager.cc:1916] T b87147731a68400e8cce9b879daa7c5d P 1ca2725d290e47d68b1a4b609c9816ac: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:18:51.780695 16048 catalog_manager.cc:5697] T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b reported cstate change: config changed from index 3 to 4, VOTER 90f3da57796b488c9087a2f523adf045 (127.14.177.131) evicted. New cstate: current_term: 1 leader_uuid: "647a343a542d4a139fe106c19095129b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:18:51.794382 16045 catalog_manager.cc:5697] T d2ab0ac8347641e6955e9bce7d90c1a2 P 90f3da57796b488c9087a2f523adf045 reported cstate change: config changed from index 3 to 4, VOTER 647a343a542d4a139fe106c19095129b (127.14.177.129) evicted. New cstate: current_term: 1 leader_uuid: "1ca2725d290e47d68b1a4b609c9816ac" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:18:51.797490 16952 ts_tablet_manager.cc:1929] T b87147731a68400e8cce9b879daa7c5d P 1ca2725d290e47d68b1a4b609c9816ac: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:18:51.797966 16952 log.cc:1199] T b87147731a68400e8cce9b879daa7c5d P 1ca2725d290e47d68b1a4b609c9816ac: Deleting WAL directory at /tmp/dist-test-taskdH_Ma7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783603093801613-15046-0/minicluster-data/ts-1-root/wals/b87147731a68400e8cce9b879daa7c5d
I20260709 13:18:51.799338 16136 tablet_service.cc:1558] Processing DeleteTablet for tablet 74ca8140f35c4ae29e8a664f6bdfc829 with delete_type TABLET_DATA_TOMBSTONED (TS 647a343a542d4a139fe106c19095129b not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:40822
I20260709 13:18:51.800004 16035 catalog_manager.cc:5028] TS 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943): tablet b87147731a68400e8cce9b879daa7c5d (table test-workload [id=ed3d15d8c9ad46bebc4fe3a402d2e54f]) successfully deleted
I20260709 13:18:51.817490 16133 tablet_service.cc:1558] Processing DeleteTablet for tablet d2ab0ac8347641e6955e9bce7d90c1a2 with delete_type TABLET_DATA_TOMBSTONED (TS 647a343a542d4a139fe106c19095129b not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:40822
I20260709 13:18:51.820287 16953 tablet_replica.cc:333] T 74ca8140f35c4ae29e8a664f6bdfc829 P 647a343a542d4a139fe106c19095129b: stopping tablet replica
I20260709 13:18:51.821285 16953 raft_consensus.cc:2243] T 74ca8140f35c4ae29e8a664f6bdfc829 P 647a343a542d4a139fe106c19095129b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:51.821906 16286 tablet_service.cc:1558] Processing DeleteTablet for tablet 66f75124215f4ca6bef831e174a792d8 with delete_type TABLET_DATA_TOMBSTONED (TS 90f3da57796b488c9087a2f523adf045 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:48670
I20260709 13:18:51.822173 16953 raft_consensus.cc:2272] T 74ca8140f35c4ae29e8a664f6bdfc829 P 647a343a542d4a139fe106c19095129b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:51.824542 16954 tablet_replica.cc:333] T 66f75124215f4ca6bef831e174a792d8 P 90f3da57796b488c9087a2f523adf045: stopping tablet replica
I20260709 13:18:51.825268 16954 raft_consensus.cc:2243] T 66f75124215f4ca6bef831e174a792d8 P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:51.825814 16954 raft_consensus.cc:2272] T 66f75124215f4ca6bef831e174a792d8 P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:51.828485 16953 ts_tablet_manager.cc:1916] T 74ca8140f35c4ae29e8a664f6bdfc829 P 647a343a542d4a139fe106c19095129b: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:18:51.828999 16954 ts_tablet_manager.cc:1916] T 66f75124215f4ca6bef831e174a792d8 P 90f3da57796b488c9087a2f523adf045: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:18:51.846514 16953 ts_tablet_manager.cc:1929] T 74ca8140f35c4ae29e8a664f6bdfc829 P 647a343a542d4a139fe106c19095129b: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:18:51.847009 16953 log.cc:1199] T 74ca8140f35c4ae29e8a664f6bdfc829 P 647a343a542d4a139fe106c19095129b: Deleting WAL directory at /tmp/dist-test-taskdH_Ma7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783603093801613-15046-0/minicluster-data/ts-0-root/wals/74ca8140f35c4ae29e8a664f6bdfc829
I20260709 13:18:51.848646 16035 catalog_manager.cc:5028] TS 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): tablet 74ca8140f35c4ae29e8a664f6bdfc829 (table test-workload [id=ed3d15d8c9ad46bebc4fe3a402d2e54f]) successfully deleted
I20260709 13:18:51.859506 16954 ts_tablet_manager.cc:1929] T 66f75124215f4ca6bef831e174a792d8 P 90f3da57796b488c9087a2f523adf045: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:18:51.860031 16954 log.cc:1199] T 66f75124215f4ca6bef831e174a792d8 P 90f3da57796b488c9087a2f523adf045: Deleting WAL directory at /tmp/dist-test-taskdH_Ma7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783603093801613-15046-0/minicluster-data/ts-2-root/wals/66f75124215f4ca6bef831e174a792d8
I20260709 13:18:51.861950 16034 catalog_manager.cc:5028] TS 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389): tablet 66f75124215f4ca6bef831e174a792d8 (table test-workload [id=ed3d15d8c9ad46bebc4fe3a402d2e54f]) successfully deleted
I20260709 13:18:51.861928 16953 tablet_replica.cc:333] T d2ab0ac8347641e6955e9bce7d90c1a2 P 647a343a542d4a139fe106c19095129b: stopping tablet replica
I20260709 13:18:51.869978 16953 raft_consensus.cc:2243] T d2ab0ac8347641e6955e9bce7d90c1a2 P 647a343a542d4a139fe106c19095129b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:51.870755 16953 raft_consensus.cc:2272] T d2ab0ac8347641e6955e9bce7d90c1a2 P 647a343a542d4a139fe106c19095129b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:51.873523 16953 ts_tablet_manager.cc:1916] T d2ab0ac8347641e6955e9bce7d90c1a2 P 647a343a542d4a139fe106c19095129b: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:18:51.886528 16953 ts_tablet_manager.cc:1929] T d2ab0ac8347641e6955e9bce7d90c1a2 P 647a343a542d4a139fe106c19095129b: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:18:51.886871 16953 log.cc:1199] T d2ab0ac8347641e6955e9bce7d90c1a2 P 647a343a542d4a139fe106c19095129b: Deleting WAL directory at /tmp/dist-test-taskdH_Ma7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783603093801613-15046-0/minicluster-data/ts-0-root/wals/d2ab0ac8347641e6955e9bce7d90c1a2
I20260709 13:18:51.888197 16035 catalog_manager.cc:5028] TS 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): tablet d2ab0ac8347641e6955e9bce7d90c1a2 (table test-workload [id=ed3d15d8c9ad46bebc4fe3a402d2e54f]) successfully deleted
I20260709 13:18:52.903955 16224 consensus_queue.cc:237] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } }
W20260709 13:18:52.911602 16192 consensus_peers.cc:597] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac -> Peer cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923): Couldn't send request to peer cb82cc7bd6db46d980c75237a1402860. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d2372981006d4b218215c28964dea623. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:18:52.913645 16148 raft_consensus.cc:1275] T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b [term 1 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:18:52.914388 16296 raft_consensus.cc:1275] T d2372981006d4b218215c28964dea623 P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:18:52.915885 16922 consensus_queue.cc:1048] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:18:52.916682 16955 consensus_queue.cc:1048] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:18:52.947232 16939 raft_consensus.cc:2955] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } } }
I20260709 13:18:52.949162 16295 raft_consensus.cc:2955] T d2372981006d4b218215c28964dea623 P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } } }
I20260709 13:18:52.969108 16145 consensus_queue.cc:237] T 2794af5b99984efebdee564035841cb0 P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } }
I20260709 13:18:52.972388 16148 raft_consensus.cc:2955] T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } } }
I20260709 13:18:52.970872 16047 catalog_manager.cc:5697] T d2372981006d4b218215c28964dea623 P 90f3da57796b488c9087a2f523adf045 reported cstate change: config changed from index -1 to 2, NON_VOTER cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) added. New cstate: current_term: 1 leader_uuid: "1ca2725d290e47d68b1a4b609c9816ac" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } } }
W20260709 13:18:52.984822 16117 consensus_peers.cc:597] T 2794af5b99984efebdee564035841cb0 P 647a343a542d4a139fe106c19095129b -> Peer cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923): Couldn't send request to peer cb82cc7bd6db46d980c75237a1402860. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 2794af5b99984efebdee564035841cb0. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:18:52.984735 16296 raft_consensus.cc:1275] T 2794af5b99984efebdee564035841cb0 P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:18:52.986598 16956 consensus_queue.cc:1048] T 2794af5b99984efebdee564035841cb0 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:18:52.992389 16224 raft_consensus.cc:1275] T 2794af5b99984efebdee564035841cb0 P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:18:52.996130 16854 consensus_queue.cc:1048] T 2794af5b99984efebdee564035841cb0 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:18:53.068811 16854 raft_consensus.cc:2955] T 2794af5b99984efebdee564035841cb0 P 647a343a542d4a139fe106c19095129b [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } } }
I20260709 13:18:53.070294 16295 raft_consensus.cc:2955] T 2794af5b99984efebdee564035841cb0 P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } } }
I20260709 13:18:53.089504 16224 raft_consensus.cc:2955] T 2794af5b99984efebdee564035841cb0 P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } } }
I20260709 13:18:53.093369 16145 consensus_queue.cc:237] T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } }
I20260709 13:18:53.108093 16048 catalog_manager.cc:5697] T 2794af5b99984efebdee564035841cb0 P 90f3da57796b488c9087a2f523adf045 reported cstate change: config changed from index -1 to 2, NON_VOTER cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) added. New cstate: current_term: 1 leader_uuid: "647a343a542d4a139fe106c19095129b" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } } }
W20260709 13:18:53.132084 16117 consensus_peers.cc:597] T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b -> Peer cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923): Couldn't send request to peer cb82cc7bd6db46d980c75237a1402860. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 08b7d1ab519044eda2d5e04a07242f25. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:18:53.136466 16224 raft_consensus.cc:1275] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:18:53.137928 16956 consensus_queue.cc:1048] T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:18:53.147945 16295 raft_consensus.cc:1275] T 08b7d1ab519044eda2d5e04a07242f25 P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:18:53.151816 16969 consensus_queue.cc:1048] T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:18:53.154549 16969 raft_consensus.cc:2955] T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } } }
I20260709 13:18:53.156407 16224 raft_consensus.cc:2955] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } } }
I20260709 13:18:53.168198 16148 consensus_queue.cc:237] T 26354361e9b044cea1c2b86c4882fe92 P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } }
I20260709 13:18:53.185523 16047 catalog_manager.cc:5697] T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b reported cstate change: config changed from index -1 to 2, NON_VOTER cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) added. New cstate: current_term: 1 leader_uuid: "647a343a542d4a139fe106c19095129b" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
W20260709 13:18:53.187809 16117 consensus_peers.cc:597] T 26354361e9b044cea1c2b86c4882fe92 P 647a343a542d4a139fe106c19095129b -> Peer cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923): Couldn't send request to peer cb82cc7bd6db46d980c75237a1402860. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 26354361e9b044cea1c2b86c4882fe92. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:18:53.187844 16297 raft_consensus.cc:1275] T 26354361e9b044cea1c2b86c4882fe92 P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:18:53.189082 16296 raft_consensus.cc:2955] T 08b7d1ab519044eda2d5e04a07242f25 P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } } }
I20260709 13:18:53.190312 16931 consensus_queue.cc:1048] T 26354361e9b044cea1c2b86c4882fe92 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:18:53.192160 16226 raft_consensus.cc:1275] T 26354361e9b044cea1c2b86c4882fe92 P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:18:53.196868 16931 consensus_queue.cc:1048] T 26354361e9b044cea1c2b86c4882fe92 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:18:53.228542 16969 raft_consensus.cc:2955] T 26354361e9b044cea1c2b86c4882fe92 P 647a343a542d4a139fe106c19095129b [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } } }
I20260709 13:18:53.234393 16295 raft_consensus.cc:2955] T 26354361e9b044cea1c2b86c4882fe92 P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } } }
I20260709 13:18:53.244782 16224 raft_consensus.cc:2955] T 26354361e9b044cea1c2b86c4882fe92 P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } } }
I20260709 13:18:53.265985 16048 catalog_manager.cc:5697] T 26354361e9b044cea1c2b86c4882fe92 P 90f3da57796b488c9087a2f523adf045 reported cstate change: config changed from index -1 to 2, NON_VOTER cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) added. New cstate: current_term: 1 leader_uuid: "647a343a542d4a139fe106c19095129b" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } } }
I20260709 13:18:53.463040 16976 ts_tablet_manager.cc:933] T 2794af5b99984efebdee564035841cb0 P cb82cc7bd6db46d980c75237a1402860: Initiating tablet copy from peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741)
I20260709 13:18:53.466061 16976 tablet_copy_client.cc:323] T 2794af5b99984efebdee564035841cb0 P cb82cc7bd6db46d980c75237a1402860: tablet copy: Beginning tablet copy session from remote peer at address 127.14.177.129:36741
I20260709 13:18:53.468504 16161 tablet_copy_service.cc:140] P 647a343a542d4a139fe106c19095129b: Received BeginTabletCopySession request for tablet 2794af5b99984efebdee564035841cb0 from peer cb82cc7bd6db46d980c75237a1402860 ({username='slave'} at 127.0.0.1:46224)
I20260709 13:18:53.469162 16161 tablet_copy_service.cc:161] P 647a343a542d4a139fe106c19095129b: Beginning new tablet copy session on tablet 2794af5b99984efebdee564035841cb0 from peer cb82cc7bd6db46d980c75237a1402860 at {username='slave'} at 127.0.0.1:46224: session id = cb82cc7bd6db46d980c75237a1402860-2794af5b99984efebdee564035841cb0
I20260709 13:18:53.489877 16161 tablet_copy_source_session.cc:215] T 2794af5b99984efebdee564035841cb0 P 647a343a542d4a139fe106c19095129b: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:18:53.527777 16976 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2794af5b99984efebdee564035841cb0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:53.547381 16978 ts_tablet_manager.cc:933] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860: Initiating tablet copy from peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:18:53.579254 16976 tablet_copy_client.cc:806] T 2794af5b99984efebdee564035841cb0 P cb82cc7bd6db46d980c75237a1402860: tablet copy: Starting download of 0 data blocks...
I20260709 13:18:53.580032 16976 tablet_copy_client.cc:670] T 2794af5b99984efebdee564035841cb0 P cb82cc7bd6db46d980c75237a1402860: tablet copy: Starting download of 1 WAL segments...
I20260709 13:18:53.587320 16976 tablet_copy_client.cc:538] T 2794af5b99984efebdee564035841cb0 P cb82cc7bd6db46d980c75237a1402860: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:18:53.604393 16980 ts_tablet_manager.cc:933] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860: Initiating tablet copy from peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741)
I20260709 13:18:53.606985 16978 tablet_copy_client.cc:323] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860: tablet copy: Beginning tablet copy session from remote peer at address 127.14.177.130:35943
I20260709 13:18:53.608953 16236 tablet_copy_service.cc:140] P 1ca2725d290e47d68b1a4b609c9816ac: Received BeginTabletCopySession request for tablet d2372981006d4b218215c28964dea623 from peer cb82cc7bd6db46d980c75237a1402860 ({username='slave'} at 127.0.0.1:33050)
I20260709 13:18:53.609680 16236 tablet_copy_service.cc:161] P 1ca2725d290e47d68b1a4b609c9816ac: Beginning new tablet copy session on tablet d2372981006d4b218215c28964dea623 from peer cb82cc7bd6db46d980c75237a1402860 at {username='slave'} at 127.0.0.1:33050: session id = cb82cc7bd6db46d980c75237a1402860-d2372981006d4b218215c28964dea623
I20260709 13:18:53.633903 16976 tablet_bootstrap.cc:492] T 2794af5b99984efebdee564035841cb0 P cb82cc7bd6db46d980c75237a1402860: Bootstrap starting.
I20260709 13:18:53.634744 16236 tablet_copy_source_session.cc:215] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:18:53.644804 16980 tablet_copy_client.cc:323] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860: tablet copy: Beginning tablet copy session from remote peer at address 127.14.177.129:36741
I20260709 13:18:53.645581 16978 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d2372981006d4b218215c28964dea623. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:53.646849 16161 tablet_copy_service.cc:140] P 647a343a542d4a139fe106c19095129b: Received BeginTabletCopySession request for tablet 08b7d1ab519044eda2d5e04a07242f25 from peer cb82cc7bd6db46d980c75237a1402860 ({username='slave'} at 127.0.0.1:46224)
I20260709 13:18:53.647464 16161 tablet_copy_service.cc:161] P 647a343a542d4a139fe106c19095129b: Beginning new tablet copy session on tablet 08b7d1ab519044eda2d5e04a07242f25 from peer cb82cc7bd6db46d980c75237a1402860 at {username='slave'} at 127.0.0.1:46224: session id = cb82cc7bd6db46d980c75237a1402860-08b7d1ab519044eda2d5e04a07242f25
I20260709 13:18:53.661101 16161 tablet_copy_source_session.cc:215] T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:18:53.669773 16980 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 08b7d1ab519044eda2d5e04a07242f25. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:53.699805 16976 tablet_bootstrap.cc:492] T 2794af5b99984efebdee564035841cb0 P cb82cc7bd6db46d980c75237a1402860: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:18:53.706202 16976 tablet_bootstrap.cc:492] T 2794af5b99984efebdee564035841cb0 P cb82cc7bd6db46d980c75237a1402860: Bootstrap complete.
I20260709 13:18:53.707357 16976 ts_tablet_manager.cc:1403] T 2794af5b99984efebdee564035841cb0 P cb82cc7bd6db46d980c75237a1402860: Time spent bootstrapping tablet: real 0.074s	user 0.031s	sys 0.000s
I20260709 13:18:53.710268 16976 raft_consensus.cc:359] T 2794af5b99984efebdee564035841cb0 P cb82cc7bd6db46d980c75237a1402860 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } }
I20260709 13:18:53.703189 16982 ts_tablet_manager.cc:933] T 26354361e9b044cea1c2b86c4882fe92 P cb82cc7bd6db46d980c75237a1402860: Initiating tablet copy from peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741)
I20260709 13:18:53.706516 16978 tablet_copy_client.cc:806] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860: tablet copy: Starting download of 0 data blocks...
I20260709 13:18:53.712721 16978 tablet_copy_client.cc:670] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860: tablet copy: Starting download of 1 WAL segments...
I20260709 13:18:53.705230 16980 tablet_copy_client.cc:806] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860: tablet copy: Starting download of 0 data blocks...
I20260709 13:18:53.713546 16980 tablet_copy_client.cc:670] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860: tablet copy: Starting download of 1 WAL segments...
I20260709 13:18:53.715523 16976 raft_consensus.cc:740] T 2794af5b99984efebdee564035841cb0 P cb82cc7bd6db46d980c75237a1402860 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Initialized, Role: LEARNER
I20260709 13:18:53.716718 16976 consensus_queue.cc:260] T 2794af5b99984efebdee564035841cb0 P cb82cc7bd6db46d980c75237a1402860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } }
I20260709 13:18:53.718271 16980 tablet_copy_client.cc:538] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:18:53.723544 16978 tablet_copy_client.cc:538] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:18:53.738351 16980 tablet_bootstrap.cc:492] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860: Bootstrap starting.
I20260709 13:18:53.741264 16976 ts_tablet_manager.cc:1434] T 2794af5b99984efebdee564035841cb0 P cb82cc7bd6db46d980c75237a1402860: Time spent starting tablet: real 0.034s	user 0.006s	sys 0.000s
I20260709 13:18:53.748008 16161 tablet_copy_service.cc:342] P 647a343a542d4a139fe106c19095129b: Request end of tablet copy session cb82cc7bd6db46d980c75237a1402860-2794af5b99984efebdee564035841cb0 received from {username='slave'} at 127.0.0.1:46224
I20260709 13:18:53.748402 16161 tablet_copy_service.cc:434] P 647a343a542d4a139fe106c19095129b: ending tablet copy session cb82cc7bd6db46d980c75237a1402860-2794af5b99984efebdee564035841cb0 on tablet 2794af5b99984efebdee564035841cb0 with peer cb82cc7bd6db46d980c75237a1402860
I20260709 13:18:53.744113 16978 tablet_bootstrap.cc:492] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860: Bootstrap starting.
I20260709 13:18:53.745591 16982 tablet_copy_client.cc:323] T 26354361e9b044cea1c2b86c4882fe92 P cb82cc7bd6db46d980c75237a1402860: tablet copy: Beginning tablet copy session from remote peer at address 127.14.177.129:36741
I20260709 13:18:53.761561 16161 tablet_copy_service.cc:140] P 647a343a542d4a139fe106c19095129b: Received BeginTabletCopySession request for tablet 26354361e9b044cea1c2b86c4882fe92 from peer cb82cc7bd6db46d980c75237a1402860 ({username='slave'} at 127.0.0.1:46224)
I20260709 13:18:53.762084 16161 tablet_copy_service.cc:161] P 647a343a542d4a139fe106c19095129b: Beginning new tablet copy session on tablet 26354361e9b044cea1c2b86c4882fe92 from peer cb82cc7bd6db46d980c75237a1402860 at {username='slave'} at 127.0.0.1:46224: session id = cb82cc7bd6db46d980c75237a1402860-26354361e9b044cea1c2b86c4882fe92
I20260709 13:18:53.808327 16980 tablet_bootstrap.cc:492] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:18:53.815937 16980 tablet_bootstrap.cc:492] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860: Bootstrap complete.
I20260709 13:18:53.816742 16980 ts_tablet_manager.cc:1403] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860: Time spent bootstrapping tablet: real 0.079s	user 0.019s	sys 0.007s
I20260709 13:18:53.808429 16161 tablet_copy_source_session.cc:215] T 26354361e9b044cea1c2b86c4882fe92 P 647a343a542d4a139fe106c19095129b: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:18:53.814764 16978 tablet_bootstrap.cc:492] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:18:53.820627 16978 tablet_bootstrap.cc:492] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860: Bootstrap complete.
I20260709 13:18:53.821311 16978 ts_tablet_manager.cc:1403] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860: Time spent bootstrapping tablet: real 0.077s	user 0.018s	sys 0.006s
I20260709 13:18:53.824332 16978 raft_consensus.cc:359] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } }
I20260709 13:18:53.825162 16978 raft_consensus.cc:740] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Initialized, Role: LEARNER
I20260709 13:18:53.825834 16978 consensus_queue.cc:260] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } }
I20260709 13:18:53.828996 16978 ts_tablet_manager.cc:1434] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.001s
I20260709 13:18:53.819769 16980 raft_consensus.cc:359] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } }
I20260709 13:18:53.829914 16980 raft_consensus.cc:740] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Initialized, Role: LEARNER
I20260709 13:18:53.830655 16980 consensus_queue.cc:260] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } }
I20260709 13:18:53.833276 16980 ts_tablet_manager.cc:1434] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860: Time spent starting tablet: real 0.016s	user 0.004s	sys 0.001s
I20260709 13:18:53.835016 16160 tablet_copy_service.cc:342] P 647a343a542d4a139fe106c19095129b: Request end of tablet copy session cb82cc7bd6db46d980c75237a1402860-08b7d1ab519044eda2d5e04a07242f25 received from {username='slave'} at 127.0.0.1:46224
I20260709 13:18:53.835453 16160 tablet_copy_service.cc:434] P 647a343a542d4a139fe106c19095129b: ending tablet copy session cb82cc7bd6db46d980c75237a1402860-08b7d1ab519044eda2d5e04a07242f25 on tablet 08b7d1ab519044eda2d5e04a07242f25 with peer cb82cc7bd6db46d980c75237a1402860
I20260709 13:18:53.843940 16236 tablet_copy_service.cc:342] P 1ca2725d290e47d68b1a4b609c9816ac: Request end of tablet copy session cb82cc7bd6db46d980c75237a1402860-d2372981006d4b218215c28964dea623 received from {username='slave'} at 127.0.0.1:33050
I20260709 13:18:53.844466 16236 tablet_copy_service.cc:434] P 1ca2725d290e47d68b1a4b609c9816ac: ending tablet copy session cb82cc7bd6db46d980c75237a1402860-d2372981006d4b218215c28964dea623 on tablet d2372981006d4b218215c28964dea623 with peer cb82cc7bd6db46d980c75237a1402860
I20260709 13:18:53.855058 16982 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 26354361e9b044cea1c2b86c4882fe92. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:53.875872 16982 tablet_copy_client.cc:806] T 26354361e9b044cea1c2b86c4882fe92 P cb82cc7bd6db46d980c75237a1402860: tablet copy: Starting download of 0 data blocks...
I20260709 13:18:53.876482 16982 tablet_copy_client.cc:670] T 26354361e9b044cea1c2b86c4882fe92 P cb82cc7bd6db46d980c75237a1402860: tablet copy: Starting download of 1 WAL segments...
I20260709 13:18:53.880084 16982 tablet_copy_client.cc:538] T 26354361e9b044cea1c2b86c4882fe92 P cb82cc7bd6db46d980c75237a1402860: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:18:53.898792 16982 tablet_bootstrap.cc:492] T 26354361e9b044cea1c2b86c4882fe92 P cb82cc7bd6db46d980c75237a1402860: Bootstrap starting.
I20260709 13:18:53.919508 16982 tablet_bootstrap.cc:492] T 26354361e9b044cea1c2b86c4882fe92 P cb82cc7bd6db46d980c75237a1402860: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:18:53.920507 16982 tablet_bootstrap.cc:492] T 26354361e9b044cea1c2b86c4882fe92 P cb82cc7bd6db46d980c75237a1402860: Bootstrap complete.
I20260709 13:18:53.921464 16982 ts_tablet_manager.cc:1403] T 26354361e9b044cea1c2b86c4882fe92 P cb82cc7bd6db46d980c75237a1402860: Time spent bootstrapping tablet: real 0.024s	user 0.013s	sys 0.008s
I20260709 13:18:53.924793 16982 raft_consensus.cc:359] T 26354361e9b044cea1c2b86c4882fe92 P cb82cc7bd6db46d980c75237a1402860 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } }
I20260709 13:18:53.925734 16982 raft_consensus.cc:740] T 26354361e9b044cea1c2b86c4882fe92 P cb82cc7bd6db46d980c75237a1402860 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Initialized, Role: LEARNER
I20260709 13:18:53.926462 16982 consensus_queue.cc:260] T 26354361e9b044cea1c2b86c4882fe92 P cb82cc7bd6db46d980c75237a1402860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } }
I20260709 13:18:53.929540 16982 ts_tablet_manager.cc:1434] T 26354361e9b044cea1c2b86c4882fe92 P cb82cc7bd6db46d980c75237a1402860: Time spent starting tablet: real 0.008s	user 0.003s	sys 0.003s
I20260709 13:18:53.931337 16161 tablet_copy_service.cc:342] P 647a343a542d4a139fe106c19095129b: Request end of tablet copy session cb82cc7bd6db46d980c75237a1402860-26354361e9b044cea1c2b86c4882fe92 received from {username='slave'} at 127.0.0.1:46224
I20260709 13:18:53.931766 16161 tablet_copy_service.cc:434] P 647a343a542d4a139fe106c19095129b: ending tablet copy session cb82cc7bd6db46d980c75237a1402860-26354361e9b044cea1c2b86c4882fe92 on tablet 26354361e9b044cea1c2b86c4882fe92 with peer cb82cc7bd6db46d980c75237a1402860
I20260709 13:18:54.060823 16823 raft_consensus.cc:1217] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:18:54.075038 16822 raft_consensus.cc:1217] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:18:54.103776 16823 raft_consensus.cc:1217] T 2794af5b99984efebdee564035841cb0 P cb82cc7bd6db46d980c75237a1402860 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:18:54.137460 16823 raft_consensus.cc:1217] T 26354361e9b044cea1c2b86c4882fe92 P cb82cc7bd6db46d980c75237a1402860 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
W20260709 13:18:54.414155 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 46405.1us
W20260709 13:18:54.497547 16115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 66938.4us
I20260709 13:18:54.534323 16999 raft_consensus.cc:1064] T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b: attempting to promote NON_VOTER cb82cc7bd6db46d980c75237a1402860 to VOTER
I20260709 13:18:54.538878 16999 consensus_queue.cc:237] T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:18:54.552001 17000 raft_consensus.cc:1064] T 26354361e9b044cea1c2b86c4882fe92 P 647a343a542d4a139fe106c19095129b: attempting to promote NON_VOTER cb82cc7bd6db46d980c75237a1402860 to VOTER
I20260709 13:18:54.556038 17000 consensus_queue.cc:237] T 26354361e9b044cea1c2b86c4882fe92 P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:18:54.573308 16823 raft_consensus.cc:1275] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 [term 1 LEARNER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:18:54.575850 16999 consensus_queue.cc:1048] T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:18:54.579612 16224 raft_consensus.cc:1275] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:18:54.581200 16998 consensus_queue.cc:1048] T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:18:54.581840 16822 raft_consensus.cc:1275] T 26354361e9b044cea1c2b86c4882fe92 P cb82cc7bd6db46d980c75237a1402860 [term 1 LEARNER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:18:54.583906 16969 consensus_queue.cc:1048] T 26354361e9b044cea1c2b86c4882fe92 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:18:54.609879 16223 raft_consensus.cc:1275] T 26354361e9b044cea1c2b86c4882fe92 P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:18:54.611670 17002 consensus_queue.cc:1048] T 26354361e9b044cea1c2b86c4882fe92 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:18:54.618788 16999 raft_consensus.cc:2955] T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:18:54.630416 16295 raft_consensus.cc:1275] T 08b7d1ab519044eda2d5e04a07242f25 P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:18:54.646152 16955 raft_consensus.cc:1064] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac: attempting to promote NON_VOTER cb82cc7bd6db46d980c75237a1402860 to VOTER
I20260709 13:18:54.646529 16226 raft_consensus.cc:2955] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:18:54.653338 16955 consensus_queue.cc:237] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:18:54.685649 16295 raft_consensus.cc:1275] T 26354361e9b044cea1c2b86c4882fe92 P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:18:54.691871 16822 raft_consensus.cc:2955] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:18:54.746064 16931 raft_consensus.cc:2955] T 26354361e9b044cea1c2b86c4882fe92 P 647a343a542d4a139fe106c19095129b [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:18:54.756484 16046 catalog_manager.cc:5697] T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b reported cstate change: config changed from index 2 to 3, cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "647a343a542d4a139fe106c19095129b" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:18:54.761595 16823 raft_consensus.cc:2955] T 26354361e9b044cea1c2b86c4882fe92 P cb82cc7bd6db46d980c75237a1402860 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:18:54.778260 16226 raft_consensus.cc:2955] T 26354361e9b044cea1c2b86c4882fe92 P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:18:54.794185 16825 raft_consensus.cc:1275] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 [term 1 LEARNER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
W20260709 13:18:54.794116 16115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 60428.6us
I20260709 13:18:54.795928 16295 raft_consensus.cc:1275] T d2372981006d4b218215c28964dea623 P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:18:54.802729 16994 consensus_queue.cc:1048] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.007s
I20260709 13:18:54.809872 16931 consensus_queue.cc:1048] T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:18:54.822856 16999 consensus_queue.cc:1048] T 26354361e9b044cea1c2b86c4882fe92 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:18:54.827455 16955 consensus_queue.cc:1048] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.003s
I20260709 13:18:54.863924 16144 raft_consensus.cc:1275] T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b [term 1 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:18:54.864224 17000 raft_consensus.cc:1064] T 2794af5b99984efebdee564035841cb0 P 647a343a542d4a139fe106c19095129b: attempting to promote NON_VOTER cb82cc7bd6db46d980c75237a1402860 to VOTER
I20260709 13:18:54.877391 16300 raft_consensus.cc:2955] T 08b7d1ab519044eda2d5e04a07242f25 P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:18:54.873454 16296 raft_consensus.cc:2955] T 26354361e9b044cea1c2b86c4882fe92 P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:18:54.889562 16989 consensus_queue.cc:1048] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:18:54.884743 17000 consensus_queue.cc:237] T 2794af5b99984efebdee564035841cb0 P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:18:54.901544 16955 raft_consensus.cc:2955] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:18:54.913302 16295 raft_consensus.cc:2955] T d2372981006d4b218215c28964dea623 P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:18:54.986588 16822 raft_consensus.cc:2955] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:18:54.988930 16825 raft_consensus.cc:1275] T 2794af5b99984efebdee564035841cb0 P cb82cc7bd6db46d980c75237a1402860 [term 1 LEARNER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:18:54.990877 16990 consensus_queue.cc:1048] T 2794af5b99984efebdee564035841cb0 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:18:54.995026 16145 consensus_queue.cc:237] T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:18:54.999059 16047 catalog_manager.cc:5697] T 26354361e9b044cea1c2b86c4882fe92 P cb82cc7bd6db46d980c75237a1402860 reported cstate change: config changed from index 2 to 3, cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "647a343a542d4a139fe106c19095129b" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:18:55.010666 16226 raft_consensus.cc:1275] T 2794af5b99984efebdee564035841cb0 P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:18:55.012539 16990 consensus_queue.cc:1048] T 2794af5b99984efebdee564035841cb0 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:18:55.026355 16045 catalog_manager.cc:5697] T d2372981006d4b218215c28964dea623 P 90f3da57796b488c9087a2f523adf045 reported cstate change: config changed from index 2 to 3, cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "1ca2725d290e47d68b1a4b609c9816ac" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:18:55.031733 16226 raft_consensus.cc:1275] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:18:55.039598 16299 raft_consensus.cc:1275] T 2794af5b99984efebdee564035841cb0 P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:18:55.039486 16990 consensus_queue.cc:1048] T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:18:55.047813 16822 raft_consensus.cc:1275] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 [term 1 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:18:55.051105 16225 raft_consensus.cc:2955] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 90f3da57796b488c9087a2f523adf045 (127.14.177.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:18:55.094466 16931 consensus_queue.cc:1048] T 2794af5b99984efebdee564035841cb0 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:18:55.058447 17000 raft_consensus.cc:2955] T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 90f3da57796b488c9087a2f523adf045 (127.14.177.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:18:55.059689 16145 raft_consensus.cc:2955] T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:18:55.067507 17002 raft_consensus.cc:2955] T 2794af5b99984efebdee564035841cb0 P 647a343a542d4a139fe106c19095129b [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:18:55.071305 16823 raft_consensus.cc:2955] T 2794af5b99984efebdee564035841cb0 P cb82cc7bd6db46d980c75237a1402860 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:18:55.115319 16045 catalog_manager.cc:5697] T 2794af5b99984efebdee564035841cb0 P cb82cc7bd6db46d980c75237a1402860 reported cstate change: config changed from index 2 to 3, cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "647a343a542d4a139fe106c19095129b" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
W20260709 13:18:55.146292 16115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 51607us
W20260709 13:18:55.186270 16117 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 72480us
I20260709 13:18:55.190726 16998 consensus_queue.cc:1048] T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:18:55.201328 16822 raft_consensus.cc:2955] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 90f3da57796b488c9087a2f523adf045 (127.14.177.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:18:55.208575 16035 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 08b7d1ab519044eda2d5e04a07242f25 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:18:55.218667 16224 consensus_queue.cc:237] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:18:55.231279 16045 catalog_manager.cc:5697] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 reported cstate change: config changed from index 3 to 4, VOTER 90f3da57796b488c9087a2f523adf045 (127.14.177.131) evicted. New cstate: current_term: 1 leader_uuid: "647a343a542d4a139fe106c19095129b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:18:55.235834 16225 raft_consensus.cc:2955] T 2794af5b99984efebdee564035841cb0 P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:18:55.258999 16298 raft_consensus.cc:2955] T 2794af5b99984efebdee564035841cb0 P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:18:55.268618 16286 tablet_service.cc:1558] Processing DeleteTablet for tablet 08b7d1ab519044eda2d5e04a07242f25 with delete_type TABLET_DATA_TOMBSTONED (TS 90f3da57796b488c9087a2f523adf045 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:48670
I20260709 13:18:55.278116 16822 raft_consensus.cc:1275] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 [term 1 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:18:55.279512 16151 raft_consensus.cc:1275] T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b [term 1 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:18:55.280519 16995 consensus_queue.cc:1048] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:18:55.284647 16955 consensus_queue.cc:1048] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:18:55.289637 16955 raft_consensus.cc:2955] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 90f3da57796b488c9087a2f523adf045 (127.14.177.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:18:55.302196 16822 raft_consensus.cc:2955] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 90f3da57796b488c9087a2f523adf045 (127.14.177.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:18:55.321557 17030 tablet_replica.cc:333] T 08b7d1ab519044eda2d5e04a07242f25 P 90f3da57796b488c9087a2f523adf045: stopping tablet replica
I20260709 13:18:55.324874 17030 raft_consensus.cc:2243] T 08b7d1ab519044eda2d5e04a07242f25 P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:55.327375 16035 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet d2372981006d4b218215c28964dea623 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:18:55.337001 17030 raft_consensus.cc:2272] T 08b7d1ab519044eda2d5e04a07242f25 P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:55.347604 16151 raft_consensus.cc:2955] T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 90f3da57796b488c9087a2f523adf045 (127.14.177.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:18:55.349068 16047 catalog_manager.cc:5697] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac reported cstate change: config changed from index 3 to 4, VOTER 90f3da57796b488c9087a2f523adf045 (127.14.177.131) evicted. New cstate: current_term: 1 leader_uuid: "1ca2725d290e47d68b1a4b609c9816ac" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:18:55.370421 17030 ts_tablet_manager.cc:1916] T 08b7d1ab519044eda2d5e04a07242f25 P 90f3da57796b488c9087a2f523adf045: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:18:55.378518 16150 consensus_queue.cc:237] T 26354361e9b044cea1c2b86c4882fe92 P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:18:55.387679 16822 raft_consensus.cc:1275] T 26354361e9b044cea1c2b86c4882fe92 P cb82cc7bd6db46d980c75237a1402860 [term 1 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:18:55.388008 16295 raft_consensus.cc:1275] T 26354361e9b044cea1c2b86c4882fe92 P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:18:55.389745 17027 consensus_queue.cc:1048] T 26354361e9b044cea1c2b86c4882fe92 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:18:55.398532 17000 raft_consensus.cc:2955] T 26354361e9b044cea1c2b86c4882fe92 P 647a343a542d4a139fe106c19095129b [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:18:55.400589 16822 raft_consensus.cc:2955] T 26354361e9b044cea1c2b86c4882fe92 P cb82cc7bd6db46d980c75237a1402860 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:18:55.408488 16286 tablet_service.cc:1558] Processing DeleteTablet for tablet d2372981006d4b218215c28964dea623 with delete_type TABLET_DATA_TOMBSTONED (TS 90f3da57796b488c9087a2f523adf045 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:48670
I20260709 13:18:55.412395 16990 consensus_queue.cc:1048] T 26354361e9b044cea1c2b86c4882fe92 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:18:55.430943 16035 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 26354361e9b044cea1c2b86c4882fe92 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:18:55.439649 16047 catalog_manager.cc:5697] T 26354361e9b044cea1c2b86c4882fe92 P 647a343a542d4a139fe106c19095129b reported cstate change: config changed from index 3 to 4, VOTER 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130) evicted. New cstate: current_term: 1 leader_uuid: "647a343a542d4a139fe106c19095129b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:18:55.445672 16298 raft_consensus.cc:2955] T 26354361e9b044cea1c2b86c4882fe92 P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:18:55.471185 16209 tablet_service.cc:1558] Processing DeleteTablet for tablet 26354361e9b044cea1c2b86c4882fe92 with delete_type TABLET_DATA_TOMBSTONED (TS 1ca2725d290e47d68b1a4b609c9816ac not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:57598
I20260709 13:18:55.476336 17031 tablet_replica.cc:333] T 26354361e9b044cea1c2b86c4882fe92 P 1ca2725d290e47d68b1a4b609c9816ac: stopping tablet replica
I20260709 13:18:55.486189 17031 raft_consensus.cc:2243] T 26354361e9b044cea1c2b86c4882fe92 P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:55.487254 17031 raft_consensus.cc:2272] T 26354361e9b044cea1c2b86c4882fe92 P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:55.484508 17030 ts_tablet_manager.cc:1929] T 08b7d1ab519044eda2d5e04a07242f25 P 90f3da57796b488c9087a2f523adf045: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:18:55.491372 17031 ts_tablet_manager.cc:1916] T 26354361e9b044cea1c2b86c4882fe92 P 1ca2725d290e47d68b1a4b609c9816ac: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:18:55.495810 17030 log.cc:1199] T 08b7d1ab519044eda2d5e04a07242f25 P 90f3da57796b488c9087a2f523adf045: Deleting WAL directory at /tmp/dist-test-taskdH_Ma7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783603093801613-15046-0/minicluster-data/ts-2-root/wals/08b7d1ab519044eda2d5e04a07242f25
I20260709 13:18:55.498032 16034 catalog_manager.cc:5028] TS 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389): tablet 08b7d1ab519044eda2d5e04a07242f25 (table test-workload [id=ed3d15d8c9ad46bebc4fe3a402d2e54f]) successfully deleted
I20260709 13:18:55.500311 17030 tablet_replica.cc:333] T d2372981006d4b218215c28964dea623 P 90f3da57796b488c9087a2f523adf045: stopping tablet replica
I20260709 13:18:55.515216 17031 ts_tablet_manager.cc:1929] T 26354361e9b044cea1c2b86c4882fe92 P 1ca2725d290e47d68b1a4b609c9816ac: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:18:55.515893 17031 log.cc:1199] T 26354361e9b044cea1c2b86c4882fe92 P 1ca2725d290e47d68b1a4b609c9816ac: Deleting WAL directory at /tmp/dist-test-taskdH_Ma7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783603093801613-15046-0/minicluster-data/ts-1-root/wals/26354361e9b044cea1c2b86c4882fe92
I20260709 13:18:55.517899 16035 catalog_manager.cc:5028] TS 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943): tablet 26354361e9b044cea1c2b86c4882fe92 (table test-workload [id=ed3d15d8c9ad46bebc4fe3a402d2e54f]) successfully deleted
I20260709 13:18:55.519456 17030 raft_consensus.cc:2243] T d2372981006d4b218215c28964dea623 P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:55.520205 17030 raft_consensus.cc:2272] T d2372981006d4b218215c28964dea623 P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:55.523684 17030 ts_tablet_manager.cc:1916] T d2372981006d4b218215c28964dea623 P 90f3da57796b488c9087a2f523adf045: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:18:55.587553 17030 ts_tablet_manager.cc:1929] T d2372981006d4b218215c28964dea623 P 90f3da57796b488c9087a2f523adf045: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:18:55.588094 17030 log.cc:1199] T d2372981006d4b218215c28964dea623 P 90f3da57796b488c9087a2f523adf045: Deleting WAL directory at /tmp/dist-test-taskdH_Ma7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783603093801613-15046-0/minicluster-data/ts-2-root/wals/d2372981006d4b218215c28964dea623
I20260709 13:18:55.590090 16034 catalog_manager.cc:5028] TS 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389): tablet d2372981006d4b218215c28964dea623 (table test-workload [id=ed3d15d8c9ad46bebc4fe3a402d2e54f]) successfully deleted
W20260709 13:18:55.599617 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 55376.6us
I20260709 13:18:55.717965 16151 consensus_queue.cc:237] T 2794af5b99984efebdee564035841cb0 P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:18:55.725083 16822 raft_consensus.cc:1275] T 2794af5b99984efebdee564035841cb0 P cb82cc7bd6db46d980c75237a1402860 [term 1 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:18:55.725338 16296 raft_consensus.cc:1275] T 2794af5b99984efebdee564035841cb0 P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:18:55.727239 17002 consensus_queue.cc:1048] T 2794af5b99984efebdee564035841cb0 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:18:55.730896 17027 consensus_queue.cc:1048] T 2794af5b99984efebdee564035841cb0 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:18:55.740242 16998 raft_consensus.cc:2955] T 2794af5b99984efebdee564035841cb0 P 647a343a542d4a139fe106c19095129b [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:18:55.757092 16823 raft_consensus.cc:2955] T 2794af5b99984efebdee564035841cb0 P cb82cc7bd6db46d980c75237a1402860 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:18:55.767482 16296 raft_consensus.cc:2955] T 2794af5b99984efebdee564035841cb0 P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:18:55.778918 16035 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 2794af5b99984efebdee564035841cb0 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:18:55.791049 16047 catalog_manager.cc:5697] T 2794af5b99984efebdee564035841cb0 P cb82cc7bd6db46d980c75237a1402860 reported cstate change: config changed from index 3 to 4, VOTER 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130) evicted. New cstate: current_term: 1 leader_uuid: "647a343a542d4a139fe106c19095129b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:18:55.804010 16209 tablet_service.cc:1558] Processing DeleteTablet for tablet 2794af5b99984efebdee564035841cb0 with delete_type TABLET_DATA_TOMBSTONED (TS 1ca2725d290e47d68b1a4b609c9816ac not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:57598
I20260709 13:18:55.805415 17031 tablet_replica.cc:333] T 2794af5b99984efebdee564035841cb0 P 1ca2725d290e47d68b1a4b609c9816ac: stopping tablet replica
I20260709 13:18:55.806442 17031 raft_consensus.cc:2243] T 2794af5b99984efebdee564035841cb0 P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:55.807210 17031 raft_consensus.cc:2272] T 2794af5b99984efebdee564035841cb0 P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:55.811345 17031 ts_tablet_manager.cc:1916] T 2794af5b99984efebdee564035841cb0 P 1ca2725d290e47d68b1a4b609c9816ac: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:18:55.826810 17031 ts_tablet_manager.cc:1929] T 2794af5b99984efebdee564035841cb0 P 1ca2725d290e47d68b1a4b609c9816ac: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:18:55.827425 17031 log.cc:1199] T 2794af5b99984efebdee564035841cb0 P 1ca2725d290e47d68b1a4b609c9816ac: Deleting WAL directory at /tmp/dist-test-taskdH_Ma7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783603093801613-15046-0/minicluster-data/ts-1-root/wals/2794af5b99984efebdee564035841cb0
I20260709 13:18:55.829066 16035 catalog_manager.cc:5028] TS 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943): tablet 2794af5b99984efebdee564035841cb0 (table test-workload [id=ed3d15d8c9ad46bebc4fe3a402d2e54f]) successfully deleted
I20260709 13:18:56.887481 16226 consensus_queue.cc:237] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } }
I20260709 13:18:56.905551 16298 raft_consensus.cc:1275] T cdbd78acb4cf46298e0de8ba0d130e19 P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 13:18:56.906845 16192 consensus_peers.cc:597] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac -> Peer cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923): Couldn't send request to peer cb82cc7bd6db46d980c75237a1402860. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: cdbd78acb4cf46298e0de8ba0d130e19. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:18:56.909531 16151 raft_consensus.cc:1275] T cdbd78acb4cf46298e0de8ba0d130e19 P 647a343a542d4a139fe106c19095129b [term 1 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:18:56.910674 16995 consensus_queue.cc:1048] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:18:56.911530 17010 consensus_queue.cc:1048] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:18:56.920025 17010 raft_consensus.cc:2955] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } } }
I20260709 13:18:56.921466 16151 raft_consensus.cc:2955] T cdbd78acb4cf46298e0de8ba0d130e19 P 647a343a542d4a139fe106c19095129b [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } } }
I20260709 13:18:56.945822 16298 raft_consensus.cc:2955] T cdbd78acb4cf46298e0de8ba0d130e19 P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } } }
I20260709 13:18:56.952098 16045 catalog_manager.cc:5697] T cdbd78acb4cf46298e0de8ba0d130e19 P 647a343a542d4a139fe106c19095129b reported cstate change: config changed from index -1 to 2, NON_VOTER cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) added. New cstate: current_term: 1 leader_uuid: "1ca2725d290e47d68b1a4b609c9816ac" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } } }
I20260709 13:18:56.958390 16151 consensus_queue.cc:237] T ee9ed67006144797b1de6ea8c2013a68 P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } }
W20260709 13:18:56.977613 16117 consensus_peers.cc:597] T ee9ed67006144797b1de6ea8c2013a68 P 647a343a542d4a139fe106c19095129b -> Peer cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923): Couldn't send request to peer cb82cc7bd6db46d980c75237a1402860. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: ee9ed67006144797b1de6ea8c2013a68. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:18:56.978502 16298 raft_consensus.cc:1275] T ee9ed67006144797b1de6ea8c2013a68 P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:18:56.979990 16990 consensus_queue.cc:1048] T ee9ed67006144797b1de6ea8c2013a68 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:18:56.980805 16225 raft_consensus.cc:1275] T ee9ed67006144797b1de6ea8c2013a68 P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 13:18:56.988669 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 49094.3us
I20260709 13:18:57.000322 17045 raft_consensus.cc:2955] T ee9ed67006144797b1de6ea8c2013a68 P 647a343a542d4a139fe106c19095129b [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } } }
I20260709 13:18:57.015736 17027 consensus_queue.cc:1048] T ee9ed67006144797b1de6ea8c2013a68 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:18:57.004868 16298 raft_consensus.cc:2955] T ee9ed67006144797b1de6ea8c2013a68 P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } } }
I20260709 13:18:57.031268 16226 raft_consensus.cc:2955] T ee9ed67006144797b1de6ea8c2013a68 P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } } }
I20260709 13:18:57.037730 16145 consensus_queue.cc:237] T 9e65676b21b748728ad39668cfe722ec P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } }
W20260709 13:18:57.163465 17049 log.cc:927] Time spent T ee9ed67006144797b1de6ea8c2013a68 P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 0.132s	user 0.014s	sys 0.006s
I20260709 13:18:57.195315 16224 raft_consensus.cc:1275] T 9e65676b21b748728ad39668cfe722ec P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:18:57.199467 16047 catalog_manager.cc:5697] T ee9ed67006144797b1de6ea8c2013a68 P 647a343a542d4a139fe106c19095129b reported cstate change: config changed from index -1 to 2, NON_VOTER cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) added. New cstate: current_term: 1 leader_uuid: "647a343a542d4a139fe106c19095129b" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
W20260709 13:18:57.208974 16117 consensus_peers.cc:597] T 9e65676b21b748728ad39668cfe722ec P 647a343a542d4a139fe106c19095129b -> Peer cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923): Couldn't send request to peer cb82cc7bd6db46d980c75237a1402860. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 9e65676b21b748728ad39668cfe722ec. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:18:57.220964 16298 raft_consensus.cc:1275] T 9e65676b21b748728ad39668cfe722ec P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:18:57.230942 17047 consensus_queue.cc:1048] T 9e65676b21b748728ad39668cfe722ec P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:18:57.235147 16990 consensus_queue.cc:1048] T 9e65676b21b748728ad39668cfe722ec P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:18:57.278364 17043 raft_consensus.cc:2955] T 9e65676b21b748728ad39668cfe722ec P 647a343a542d4a139fe106c19095129b [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } } }
I20260709 13:18:57.280696 16224 raft_consensus.cc:2955] T 9e65676b21b748728ad39668cfe722ec P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } } }
I20260709 13:18:57.302209 16226 consensus_queue.cc:237] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } }
I20260709 13:18:57.317946 16045 catalog_manager.cc:5697] T 9e65676b21b748728ad39668cfe722ec P 647a343a542d4a139fe106c19095129b reported cstate change: config changed from index -1 to 2, NON_VOTER cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) added. New cstate: current_term: 1 leader_uuid: "647a343a542d4a139fe106c19095129b" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:57.321722 16298 raft_consensus.cc:1275] T ada5a004ba7c46819051e3ccec71ffdb P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:18:57.323211 16300 raft_consensus.cc:2955] T 9e65676b21b748728ad39668cfe722ec P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } } }
W20260709 13:18:57.324211 16192 consensus_peers.cc:597] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac -> Peer cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923): Couldn't send request to peer cb82cc7bd6db46d980c75237a1402860. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: ada5a004ba7c46819051e3ccec71ffdb. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:18:57.325399 16995 consensus_queue.cc:1048] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:18:57.330705 16144 raft_consensus.cc:1275] T ada5a004ba7c46819051e3ccec71ffdb P 647a343a542d4a139fe106c19095129b [term 1 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:18:57.338557 17032 consensus_queue.cc:1048] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:18:57.352903 17032 raft_consensus.cc:2955] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } } }
I20260709 13:18:57.360544 16298 raft_consensus.cc:2955] T ada5a004ba7c46819051e3ccec71ffdb P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } } }
I20260709 13:18:57.372699 16144 raft_consensus.cc:2955] T ada5a004ba7c46819051e3ccec71ffdb P 647a343a542d4a139fe106c19095129b [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } } }
I20260709 13:18:57.376807 16045 catalog_manager.cc:5697] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac reported cstate change: config changed from index -1 to 2, NON_VOTER cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) added. New cstate: current_term: 1 leader_uuid: "1ca2725d290e47d68b1a4b609c9816ac" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:57.414345 17062 ts_tablet_manager.cc:933] T ee9ed67006144797b1de6ea8c2013a68 P cb82cc7bd6db46d980c75237a1402860: Initiating tablet copy from peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741)
I20260709 13:18:57.417861 17062 tablet_copy_client.cc:323] T ee9ed67006144797b1de6ea8c2013a68 P cb82cc7bd6db46d980c75237a1402860: tablet copy: Beginning tablet copy session from remote peer at address 127.14.177.129:36741
I20260709 13:18:57.420430 16161 tablet_copy_service.cc:140] P 647a343a542d4a139fe106c19095129b: Received BeginTabletCopySession request for tablet ee9ed67006144797b1de6ea8c2013a68 from peer cb82cc7bd6db46d980c75237a1402860 ({username='slave'} at 127.0.0.1:46224)
I20260709 13:18:57.421028 16161 tablet_copy_service.cc:161] P 647a343a542d4a139fe106c19095129b: Beginning new tablet copy session on tablet ee9ed67006144797b1de6ea8c2013a68 from peer cb82cc7bd6db46d980c75237a1402860 at {username='slave'} at 127.0.0.1:46224: session id = cb82cc7bd6db46d980c75237a1402860-ee9ed67006144797b1de6ea8c2013a68
I20260709 13:18:57.436024 16161 tablet_copy_source_session.cc:215] T ee9ed67006144797b1de6ea8c2013a68 P 647a343a542d4a139fe106c19095129b: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:18:57.444809 17062 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ee9ed67006144797b1de6ea8c2013a68. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:57.480612 17062 tablet_copy_client.cc:806] T ee9ed67006144797b1de6ea8c2013a68 P cb82cc7bd6db46d980c75237a1402860: tablet copy: Starting download of 0 data blocks...
I20260709 13:18:57.481398 17062 tablet_copy_client.cc:670] T ee9ed67006144797b1de6ea8c2013a68 P cb82cc7bd6db46d980c75237a1402860: tablet copy: Starting download of 1 WAL segments...
I20260709 13:18:57.488085 17062 tablet_copy_client.cc:538] T ee9ed67006144797b1de6ea8c2013a68 P cb82cc7bd6db46d980c75237a1402860: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:18:57.496152 17064 ts_tablet_manager.cc:933] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860: Initiating tablet copy from peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:18:57.499662 17062 tablet_bootstrap.cc:492] T ee9ed67006144797b1de6ea8c2013a68 P cb82cc7bd6db46d980c75237a1402860: Bootstrap starting.
I20260709 13:18:57.501950 17064 tablet_copy_client.cc:323] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860: tablet copy: Beginning tablet copy session from remote peer at address 127.14.177.130:35943
I20260709 13:18:57.504983 16236 tablet_copy_service.cc:140] P 1ca2725d290e47d68b1a4b609c9816ac: Received BeginTabletCopySession request for tablet cdbd78acb4cf46298e0de8ba0d130e19 from peer cb82cc7bd6db46d980c75237a1402860 ({username='slave'} at 127.0.0.1:33050)
I20260709 13:18:57.505534 16236 tablet_copy_service.cc:161] P 1ca2725d290e47d68b1a4b609c9816ac: Beginning new tablet copy session on tablet cdbd78acb4cf46298e0de8ba0d130e19 from peer cb82cc7bd6db46d980c75237a1402860 at {username='slave'} at 127.0.0.1:33050: session id = cb82cc7bd6db46d980c75237a1402860-cdbd78acb4cf46298e0de8ba0d130e19
I20260709 13:18:57.534468 17062 tablet_bootstrap.cc:492] T ee9ed67006144797b1de6ea8c2013a68 P cb82cc7bd6db46d980c75237a1402860: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:18:57.534653 16236 tablet_copy_source_session.cc:215] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:18:57.545768 17062 tablet_bootstrap.cc:492] T ee9ed67006144797b1de6ea8c2013a68 P cb82cc7bd6db46d980c75237a1402860: Bootstrap complete.
I20260709 13:18:57.546653 17062 ts_tablet_manager.cc:1403] T ee9ed67006144797b1de6ea8c2013a68 P cb82cc7bd6db46d980c75237a1402860: Time spent bootstrapping tablet: real 0.047s	user 0.022s	sys 0.011s
I20260709 13:18:57.546849 17064 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cdbd78acb4cf46298e0de8ba0d130e19. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:57.549947 17062 raft_consensus.cc:359] T ee9ed67006144797b1de6ea8c2013a68 P cb82cc7bd6db46d980c75237a1402860 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } }
I20260709 13:18:57.550774 17062 raft_consensus.cc:740] T ee9ed67006144797b1de6ea8c2013a68 P cb82cc7bd6db46d980c75237a1402860 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Initialized, Role: LEARNER
I20260709 13:18:57.551486 17062 consensus_queue.cc:260] T ee9ed67006144797b1de6ea8c2013a68 P cb82cc7bd6db46d980c75237a1402860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } }
I20260709 13:18:57.587778 17064 tablet_copy_client.cc:806] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860: tablet copy: Starting download of 0 data blocks...
I20260709 13:18:57.588729 17064 tablet_copy_client.cc:670] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860: tablet copy: Starting download of 1 WAL segments...
I20260709 13:18:57.651494 17064 tablet_copy_client.cc:538] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:18:57.678030 17064 tablet_bootstrap.cc:492] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860: Bootstrap starting.
I20260709 13:18:57.736456 17069 ts_tablet_manager.cc:933] T 9e65676b21b748728ad39668cfe722ec P cb82cc7bd6db46d980c75237a1402860: Initiating tablet copy from peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741)
I20260709 13:18:57.736606 17064 tablet_bootstrap.cc:492] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:18:57.744542 17064 tablet_bootstrap.cc:492] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860: Bootstrap complete.
I20260709 13:18:57.745306 17064 ts_tablet_manager.cc:1403] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860: Time spent bootstrapping tablet: real 0.068s	user 0.023s	sys 0.011s
I20260709 13:18:57.748736 17064 raft_consensus.cc:359] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } }
I20260709 13:18:57.749686 17064 raft_consensus.cc:740] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Initialized, Role: LEARNER
I20260709 13:18:57.750435 17064 consensus_queue.cc:260] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } }
I20260709 13:18:57.759591 17069 tablet_copy_client.cc:323] T 9e65676b21b748728ad39668cfe722ec P cb82cc7bd6db46d980c75237a1402860: tablet copy: Beginning tablet copy session from remote peer at address 127.14.177.129:36741
I20260709 13:18:57.764714 16161 tablet_copy_service.cc:140] P 647a343a542d4a139fe106c19095129b: Received BeginTabletCopySession request for tablet 9e65676b21b748728ad39668cfe722ec from peer cb82cc7bd6db46d980c75237a1402860 ({username='slave'} at 127.0.0.1:46224)
I20260709 13:18:57.765257 16161 tablet_copy_service.cc:161] P 647a343a542d4a139fe106c19095129b: Beginning new tablet copy session on tablet 9e65676b21b748728ad39668cfe722ec from peer cb82cc7bd6db46d980c75237a1402860 at {username='slave'} at 127.0.0.1:46224: session id = cb82cc7bd6db46d980c75237a1402860-9e65676b21b748728ad39668cfe722ec
I20260709 13:18:57.773061 16161 tablet_copy_source_session.cc:215] T 9e65676b21b748728ad39668cfe722ec P 647a343a542d4a139fe106c19095129b: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:18:57.776662 17069 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9e65676b21b748728ad39668cfe722ec. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:18:57.809444 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 45667.8us
I20260709 13:18:57.810837 17069 tablet_copy_client.cc:806] T 9e65676b21b748728ad39668cfe722ec P cb82cc7bd6db46d980c75237a1402860: tablet copy: Starting download of 0 data blocks...
I20260709 13:18:57.816857 17069 tablet_copy_client.cc:670] T 9e65676b21b748728ad39668cfe722ec P cb82cc7bd6db46d980c75237a1402860: tablet copy: Starting download of 1 WAL segments...
I20260709 13:18:57.823149 17064 ts_tablet_manager.cc:1434] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860: Time spent starting tablet: real 0.078s	user 0.016s	sys 0.023s
I20260709 13:18:57.825506 16236 tablet_copy_service.cc:342] P 1ca2725d290e47d68b1a4b609c9816ac: Request end of tablet copy session cb82cc7bd6db46d980c75237a1402860-cdbd78acb4cf46298e0de8ba0d130e19 received from {username='slave'} at 127.0.0.1:33050
I20260709 13:18:57.826088 16236 tablet_copy_service.cc:434] P 1ca2725d290e47d68b1a4b609c9816ac: ending tablet copy session cb82cc7bd6db46d980c75237a1402860-cdbd78acb4cf46298e0de8ba0d130e19 on tablet cdbd78acb4cf46298e0de8ba0d130e19 with peer cb82cc7bd6db46d980c75237a1402860
I20260709 13:18:57.826946 17069 tablet_copy_client.cc:538] T 9e65676b21b748728ad39668cfe722ec P cb82cc7bd6db46d980c75237a1402860: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:18:57.840229 17062 ts_tablet_manager.cc:1434] T ee9ed67006144797b1de6ea8c2013a68 P cb82cc7bd6db46d980c75237a1402860: Time spent starting tablet: real 0.293s	user 0.066s	sys 0.158s
I20260709 13:18:57.842257 16161 tablet_copy_service.cc:342] P 647a343a542d4a139fe106c19095129b: Request end of tablet copy session cb82cc7bd6db46d980c75237a1402860-ee9ed67006144797b1de6ea8c2013a68 received from {username='slave'} at 127.0.0.1:46224
I20260709 13:18:57.842730 16161 tablet_copy_service.cc:434] P 647a343a542d4a139fe106c19095129b: ending tablet copy session cb82cc7bd6db46d980c75237a1402860-ee9ed67006144797b1de6ea8c2013a68 on tablet ee9ed67006144797b1de6ea8c2013a68 with peer cb82cc7bd6db46d980c75237a1402860
I20260709 13:18:57.845187 17069 tablet_bootstrap.cc:492] T 9e65676b21b748728ad39668cfe722ec P cb82cc7bd6db46d980c75237a1402860: Bootstrap starting.
I20260709 13:18:57.870867 16822 raft_consensus.cc:1217] T ee9ed67006144797b1de6ea8c2013a68 P cb82cc7bd6db46d980c75237a1402860 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:18:57.946574 17062 ts_tablet_manager.cc:933] T ada5a004ba7c46819051e3ccec71ffdb P cb82cc7bd6db46d980c75237a1402860: Initiating tablet copy from peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:18:57.972604 17062 tablet_copy_client.cc:323] T ada5a004ba7c46819051e3ccec71ffdb P cb82cc7bd6db46d980c75237a1402860: tablet copy: Beginning tablet copy session from remote peer at address 127.14.177.130:35943
I20260709 13:18:57.976190 16236 tablet_copy_service.cc:140] P 1ca2725d290e47d68b1a4b609c9816ac: Received BeginTabletCopySession request for tablet ada5a004ba7c46819051e3ccec71ffdb from peer cb82cc7bd6db46d980c75237a1402860 ({username='slave'} at 127.0.0.1:33050)
I20260709 13:18:57.976712 16236 tablet_copy_service.cc:161] P 1ca2725d290e47d68b1a4b609c9816ac: Beginning new tablet copy session on tablet ada5a004ba7c46819051e3ccec71ffdb from peer cb82cc7bd6db46d980c75237a1402860 at {username='slave'} at 127.0.0.1:33050: session id = cb82cc7bd6db46d980c75237a1402860-ada5a004ba7c46819051e3ccec71ffdb
I20260709 13:18:57.988324 16236 tablet_copy_source_session.cc:215] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:18:57.993023 17062 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ada5a004ba7c46819051e3ccec71ffdb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:58.038169 16822 raft_consensus.cc:1217] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:18:58.066874 17069 tablet_bootstrap.cc:492] T 9e65676b21b748728ad39668cfe722ec P cb82cc7bd6db46d980c75237a1402860: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:18:58.068132 17069 tablet_bootstrap.cc:492] T 9e65676b21b748728ad39668cfe722ec P cb82cc7bd6db46d980c75237a1402860: Bootstrap complete.
I20260709 13:18:58.069008 17069 ts_tablet_manager.cc:1403] T 9e65676b21b748728ad39668cfe722ec P cb82cc7bd6db46d980c75237a1402860: Time spent bootstrapping tablet: real 0.224s	user 0.071s	sys 0.085s
I20260709 13:18:58.089548 17069 raft_consensus.cc:359] T 9e65676b21b748728ad39668cfe722ec P cb82cc7bd6db46d980c75237a1402860 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } }
I20260709 13:18:58.090685 17069 raft_consensus.cc:740] T 9e65676b21b748728ad39668cfe722ec P cb82cc7bd6db46d980c75237a1402860 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Initialized, Role: LEARNER
I20260709 13:18:58.091507 17069 consensus_queue.cc:260] T 9e65676b21b748728ad39668cfe722ec P cb82cc7bd6db46d980c75237a1402860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } }
I20260709 13:18:58.093509 17062 tablet_copy_client.cc:806] T ada5a004ba7c46819051e3ccec71ffdb P cb82cc7bd6db46d980c75237a1402860: tablet copy: Starting download of 0 data blocks...
I20260709 13:18:58.094622 17062 tablet_copy_client.cc:670] T ada5a004ba7c46819051e3ccec71ffdb P cb82cc7bd6db46d980c75237a1402860: tablet copy: Starting download of 1 WAL segments...
W20260709 13:18:58.097024 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 60055.6us
I20260709 13:18:58.101173 17069 ts_tablet_manager.cc:1434] T 9e65676b21b748728ad39668cfe722ec P cb82cc7bd6db46d980c75237a1402860: Time spent starting tablet: real 0.032s	user 0.005s	sys 0.001s
I20260709 13:18:58.103295 16161 tablet_copy_service.cc:342] P 647a343a542d4a139fe106c19095129b: Request end of tablet copy session cb82cc7bd6db46d980c75237a1402860-9e65676b21b748728ad39668cfe722ec received from {username='slave'} at 127.0.0.1:46224
I20260709 13:18:58.103785 16161 tablet_copy_service.cc:434] P 647a343a542d4a139fe106c19095129b: ending tablet copy session cb82cc7bd6db46d980c75237a1402860-9e65676b21b748728ad39668cfe722ec on tablet 9e65676b21b748728ad39668cfe722ec with peer cb82cc7bd6db46d980c75237a1402860
W20260709 13:18:58.134119 16115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 83766.9us
I20260709 13:18:58.155534 17062 tablet_copy_client.cc:538] T ada5a004ba7c46819051e3ccec71ffdb P cb82cc7bd6db46d980c75237a1402860: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:18:58.168107 17062 tablet_bootstrap.cc:492] T ada5a004ba7c46819051e3ccec71ffdb P cb82cc7bd6db46d980c75237a1402860: Bootstrap starting.
I20260709 13:18:58.192941 16822 raft_consensus.cc:1217] T 9e65676b21b748728ad39668cfe722ec P cb82cc7bd6db46d980c75237a1402860 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:18:58.212402 17062 tablet_bootstrap.cc:492] T ada5a004ba7c46819051e3ccec71ffdb P cb82cc7bd6db46d980c75237a1402860: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:18:58.213626 17062 tablet_bootstrap.cc:492] T ada5a004ba7c46819051e3ccec71ffdb P cb82cc7bd6db46d980c75237a1402860: Bootstrap complete.
I20260709 13:18:58.215845 17062 ts_tablet_manager.cc:1403] T ada5a004ba7c46819051e3ccec71ffdb P cb82cc7bd6db46d980c75237a1402860: Time spent bootstrapping tablet: real 0.048s	user 0.017s	sys 0.015s
I20260709 13:18:58.219261 17062 raft_consensus.cc:359] T ada5a004ba7c46819051e3ccec71ffdb P cb82cc7bd6db46d980c75237a1402860 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } }
I20260709 13:18:58.220268 17062 raft_consensus.cc:740] T ada5a004ba7c46819051e3ccec71ffdb P cb82cc7bd6db46d980c75237a1402860 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Initialized, Role: LEARNER
I20260709 13:18:58.221032 17062 consensus_queue.cc:260] T ada5a004ba7c46819051e3ccec71ffdb P cb82cc7bd6db46d980c75237a1402860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } }
I20260709 13:18:58.223724 17062 ts_tablet_manager.cc:1434] T ada5a004ba7c46819051e3ccec71ffdb P cb82cc7bd6db46d980c75237a1402860: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 13:18:58.225817 16236 tablet_copy_service.cc:342] P 1ca2725d290e47d68b1a4b609c9816ac: Request end of tablet copy session cb82cc7bd6db46d980c75237a1402860-ada5a004ba7c46819051e3ccec71ffdb received from {username='slave'} at 127.0.0.1:33050
I20260709 13:18:58.226338 16236 tablet_copy_service.cc:434] P 1ca2725d290e47d68b1a4b609c9816ac: ending tablet copy session cb82cc7bd6db46d980c75237a1402860-ada5a004ba7c46819051e3ccec71ffdb on tablet ada5a004ba7c46819051e3ccec71ffdb with peer cb82cc7bd6db46d980c75237a1402860
I20260709 13:18:58.424715 17037 raft_consensus.cc:1064] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac: attempting to promote NON_VOTER cb82cc7bd6db46d980c75237a1402860 to VOTER
I20260709 13:18:58.428377 17037 consensus_queue.cc:237] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:18:58.435954 17055 raft_consensus.cc:1064] T ee9ed67006144797b1de6ea8c2013a68 P 647a343a542d4a139fe106c19095129b: attempting to promote NON_VOTER cb82cc7bd6db46d980c75237a1402860 to VOTER
I20260709 13:18:58.440364 16823 raft_consensus.cc:1275] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 [term 1 LEARNER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:18:58.440043 17055 consensus_queue.cc:237] T ee9ed67006144797b1de6ea8c2013a68 P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:18:58.442134 16995 consensus_queue.cc:1048] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:18:58.442890 16299 raft_consensus.cc:1275] T cdbd78acb4cf46298e0de8ba0d130e19 P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:18:58.444468 17010 consensus_queue.cc:1048] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:18:58.447353 16145 raft_consensus.cc:1275] T cdbd78acb4cf46298e0de8ba0d130e19 P 647a343a542d4a139fe106c19095129b [term 1 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:18:58.448686 17010 consensus_queue.cc:1048] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:18:58.457890 16822 raft_consensus.cc:1275] T ee9ed67006144797b1de6ea8c2013a68 P cb82cc7bd6db46d980c75237a1402860 [term 1 LEARNER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:18:58.459957 16990 consensus_queue.cc:1048] T ee9ed67006144797b1de6ea8c2013a68 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:18:58.470930 16226 raft_consensus.cc:1275] T ee9ed67006144797b1de6ea8c2013a68 P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:18:58.470957 16300 raft_consensus.cc:1275] T ee9ed67006144797b1de6ea8c2013a68 P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:18:58.473460 17074 consensus_queue.cc:1048] T ee9ed67006144797b1de6ea8c2013a68 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:18:58.476250 17043 consensus_queue.cc:1048] T ee9ed67006144797b1de6ea8c2013a68 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:18:58.488348 16995 raft_consensus.cc:2955] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:18:58.495916 16823 raft_consensus.cc:2955] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:18:58.508265 17074 raft_consensus.cc:2955] T ee9ed67006144797b1de6ea8c2013a68 P 647a343a542d4a139fe106c19095129b [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:18:58.510743 16047 catalog_manager.cc:5697] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac reported cstate change: config changed from index 2 to 3, cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "1ca2725d290e47d68b1a4b609c9816ac" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:18:58.516328 16150 raft_consensus.cc:2955] T cdbd78acb4cf46298e0de8ba0d130e19 P 647a343a542d4a139fe106c19095129b [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:18:58.520643 16300 raft_consensus.cc:2955] T ee9ed67006144797b1de6ea8c2013a68 P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:18:58.524436 16822 raft_consensus.cc:2955] T ee9ed67006144797b1de6ea8c2013a68 P cb82cc7bd6db46d980c75237a1402860 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:18:58.528523 16823 raft_consensus.cc:1217] T ada5a004ba7c46819051e3ccec71ffdb P cb82cc7bd6db46d980c75237a1402860 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:18:58.529109 16299 raft_consensus.cc:2955] T cdbd78acb4cf46298e0de8ba0d130e19 P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:18:58.546363 16048 catalog_manager.cc:5697] T ee9ed67006144797b1de6ea8c2013a68 P 90f3da57796b488c9087a2f523adf045 reported cstate change: config changed from index 2 to 3, cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "647a343a542d4a139fe106c19095129b" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:18:58.557375 16226 raft_consensus.cc:2955] T ee9ed67006144797b1de6ea8c2013a68 P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:18:58.605466 16224 consensus_queue.cc:237] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:18:58.615051 16822 raft_consensus.cc:1275] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 [term 1 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:18:58.618109 16995 consensus_queue.cc:1048] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:18:58.628116 16299 raft_consensus.cc:1275] T cdbd78acb4cf46298e0de8ba0d130e19 P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:18:58.630425 17037 consensus_queue.cc:1048] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:18:58.634527 16150 consensus_queue.cc:237] T ee9ed67006144797b1de6ea8c2013a68 P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:18:58.635671 16995 raft_consensus.cc:2955] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 647a343a542d4a139fe106c19095129b (127.14.177.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:18:58.657212 16822 raft_consensus.cc:2955] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 647a343a542d4a139fe106c19095129b (127.14.177.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:18:58.660354 16300 raft_consensus.cc:2955] T cdbd78acb4cf46298e0de8ba0d130e19 P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 647a343a542d4a139fe106c19095129b (127.14.177.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:18:58.668182 16823 raft_consensus.cc:1275] T ee9ed67006144797b1de6ea8c2013a68 P cb82cc7bd6db46d980c75237a1402860 [term 1 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:18:58.669930 16990 consensus_queue.cc:1048] T ee9ed67006144797b1de6ea8c2013a68 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:18:58.678646 17050 raft_consensus.cc:2955] T ee9ed67006144797b1de6ea8c2013a68 P 647a343a542d4a139fe106c19095129b [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 90f3da57796b488c9087a2f523adf045 (127.14.177.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:18:58.680202 16823 raft_consensus.cc:2955] T ee9ed67006144797b1de6ea8c2013a68 P cb82cc7bd6db46d980c75237a1402860 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 90f3da57796b488c9087a2f523adf045 (127.14.177.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:18:58.685437 16035 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet cdbd78acb4cf46298e0de8ba0d130e19 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:18:58.686269 16048 catalog_manager.cc:5697] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 reported cstate change: config changed from index 3 to 4, VOTER 647a343a542d4a139fe106c19095129b (127.14.177.129) evicted. New cstate: current_term: 1 leader_uuid: "1ca2725d290e47d68b1a4b609c9816ac" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:18:58.701200 16035 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet ee9ed67006144797b1de6ea8c2013a68 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:18:58.704978 16136 tablet_service.cc:1558] Processing DeleteTablet for tablet cdbd78acb4cf46298e0de8ba0d130e19 with delete_type TABLET_DATA_TOMBSTONED (TS 647a343a542d4a139fe106c19095129b not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:40822
I20260709 13:18:58.722514 16045 catalog_manager.cc:5697] T ee9ed67006144797b1de6ea8c2013a68 P cb82cc7bd6db46d980c75237a1402860 reported cstate change: config changed from index 3 to 4, VOTER 90f3da57796b488c9087a2f523adf045 (127.14.177.131) evicted. New cstate: current_term: 1 leader_uuid: "647a343a542d4a139fe106c19095129b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:18:58.727476 16224 raft_consensus.cc:1275] T ee9ed67006144797b1de6ea8c2013a68 P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:18:58.732326 16990 consensus_queue.cc:1048] T ee9ed67006144797b1de6ea8c2013a68 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:18:58.734834 16286 tablet_service.cc:1558] Processing DeleteTablet for tablet ee9ed67006144797b1de6ea8c2013a68 with delete_type TABLET_DATA_TOMBSTONED (TS 90f3da57796b488c9087a2f523adf045 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:48670
I20260709 13:18:58.741458 17092 tablet_replica.cc:333] T cdbd78acb4cf46298e0de8ba0d130e19 P 647a343a542d4a139fe106c19095129b: stopping tablet replica
I20260709 13:18:58.745769 17092 raft_consensus.cc:2243] T cdbd78acb4cf46298e0de8ba0d130e19 P 647a343a542d4a139fe106c19095129b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:58.746637 17092 raft_consensus.cc:2272] T cdbd78acb4cf46298e0de8ba0d130e19 P 647a343a542d4a139fe106c19095129b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:58.750273 17092 ts_tablet_manager.cc:1916] T cdbd78acb4cf46298e0de8ba0d130e19 P 647a343a542d4a139fe106c19095129b: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:18:58.761260 17093 tablet_replica.cc:333] T ee9ed67006144797b1de6ea8c2013a68 P 90f3da57796b488c9087a2f523adf045: stopping tablet replica
I20260709 13:18:58.763348 17093 raft_consensus.cc:2243] T ee9ed67006144797b1de6ea8c2013a68 P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:58.769683 17093 raft_consensus.cc:2272] T ee9ed67006144797b1de6ea8c2013a68 P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:58.768540 17092 ts_tablet_manager.cc:1929] T cdbd78acb4cf46298e0de8ba0d130e19 P 647a343a542d4a139fe106c19095129b: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:18:58.770682 17092 log.cc:1199] T cdbd78acb4cf46298e0de8ba0d130e19 P 647a343a542d4a139fe106c19095129b: Deleting WAL directory at /tmp/dist-test-taskdH_Ma7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783603093801613-15046-0/minicluster-data/ts-0-root/wals/cdbd78acb4cf46298e0de8ba0d130e19
I20260709 13:18:58.772564 16224 raft_consensus.cc:2955] T ee9ed67006144797b1de6ea8c2013a68 P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 90f3da57796b488c9087a2f523adf045 (127.14.177.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:18:58.773376 16035 catalog_manager.cc:5028] TS 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): tablet cdbd78acb4cf46298e0de8ba0d130e19 (table test-workload [id=ed3d15d8c9ad46bebc4fe3a402d2e54f]) successfully deleted
I20260709 13:18:58.777263 17093 ts_tablet_manager.cc:1916] T ee9ed67006144797b1de6ea8c2013a68 P 90f3da57796b488c9087a2f523adf045: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:18:58.781396 17056 raft_consensus.cc:1064] T 9e65676b21b748728ad39668cfe722ec P 647a343a542d4a139fe106c19095129b: attempting to promote NON_VOTER cb82cc7bd6db46d980c75237a1402860 to VOTER
I20260709 13:18:58.786698 17056 consensus_queue.cc:237] T 9e65676b21b748728ad39668cfe722ec P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:18:58.801425 17093 ts_tablet_manager.cc:1929] T ee9ed67006144797b1de6ea8c2013a68 P 90f3da57796b488c9087a2f523adf045: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:18:58.801992 17093 log.cc:1199] T ee9ed67006144797b1de6ea8c2013a68 P 90f3da57796b488c9087a2f523adf045: Deleting WAL directory at /tmp/dist-test-taskdH_Ma7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783603093801613-15046-0/minicluster-data/ts-2-root/wals/ee9ed67006144797b1de6ea8c2013a68
I20260709 13:18:58.803710 16034 catalog_manager.cc:5028] TS 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389): tablet ee9ed67006144797b1de6ea8c2013a68 (table test-workload [id=ed3d15d8c9ad46bebc4fe3a402d2e54f]) successfully deleted
I20260709 13:18:58.811700 16822 raft_consensus.cc:1275] T 9e65676b21b748728ad39668cfe722ec P cb82cc7bd6db46d980c75237a1402860 [term 1 LEARNER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:18:58.812682 16300 raft_consensus.cc:1275] T 9e65676b21b748728ad39668cfe722ec P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:18:58.821440 17055 consensus_queue.cc:1048] T 9e65676b21b748728ad39668cfe722ec P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:18:58.822587 17074 consensus_queue.cc:1048] T 9e65676b21b748728ad39668cfe722ec P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:18:58.829052 16224 raft_consensus.cc:1275] T 9e65676b21b748728ad39668cfe722ec P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:18:58.830721 17056 consensus_queue.cc:1048] T 9e65676b21b748728ad39668cfe722ec P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:18:58.854606 16822 raft_consensus.cc:2955] T 9e65676b21b748728ad39668cfe722ec P cb82cc7bd6db46d980c75237a1402860 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:18:58.858486 16222 raft_consensus.cc:2955] T 9e65676b21b748728ad39668cfe722ec P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:18:58.852708 17056 raft_consensus.cc:2955] T 9e65676b21b748728ad39668cfe722ec P 647a343a542d4a139fe106c19095129b [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:18:58.875134 16299 raft_consensus.cc:2955] T 9e65676b21b748728ad39668cfe722ec P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:18:58.903720 16047 catalog_manager.cc:5697] T 9e65676b21b748728ad39668cfe722ec P cb82cc7bd6db46d980c75237a1402860 reported cstate change: config changed from index 2 to 3, cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "647a343a542d4a139fe106c19095129b" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:18:58.924763 16150 consensus_queue.cc:237] T 9e65676b21b748728ad39668cfe722ec P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:18:58.937784 16300 raft_consensus.cc:1275] T 9e65676b21b748728ad39668cfe722ec P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:18:58.938380 16822 raft_consensus.cc:1275] T 9e65676b21b748728ad39668cfe722ec P cb82cc7bd6db46d980c75237a1402860 [term 1 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:18:58.939858 17055 consensus_queue.cc:1048] T 9e65676b21b748728ad39668cfe722ec P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:18:58.941700 17055 consensus_queue.cc:1048] T 9e65676b21b748728ad39668cfe722ec P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:18:58.950284 16823 raft_consensus.cc:2955] T 9e65676b21b748728ad39668cfe722ec P cb82cc7bd6db46d980c75237a1402860 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:18:58.948601 17055 raft_consensus.cc:2955] T 9e65676b21b748728ad39668cfe722ec P 647a343a542d4a139fe106c19095129b [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:18:58.954030 16300 raft_consensus.cc:2955] T 9e65676b21b748728ad39668cfe722ec P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:18:58.962981 16035 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 9e65676b21b748728ad39668cfe722ec with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:18:58.973654 16047 catalog_manager.cc:5697] T 9e65676b21b748728ad39668cfe722ec P 647a343a542d4a139fe106c19095129b reported cstate change: config changed from index 3 to 4, VOTER 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130) evicted. New cstate: current_term: 1 leader_uuid: "647a343a542d4a139fe106c19095129b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:18:58.987474 16209 tablet_service.cc:1558] Processing DeleteTablet for tablet 9e65676b21b748728ad39668cfe722ec with delete_type TABLET_DATA_TOMBSTONED (TS 1ca2725d290e47d68b1a4b609c9816ac not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:57598
I20260709 13:18:58.989626 17098 tablet_replica.cc:333] T 9e65676b21b748728ad39668cfe722ec P 1ca2725d290e47d68b1a4b609c9816ac: stopping tablet replica
I20260709 13:18:58.990532 17098 raft_consensus.cc:2243] T 9e65676b21b748728ad39668cfe722ec P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:58.991144 17098 raft_consensus.cc:2272] T 9e65676b21b748728ad39668cfe722ec P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:58.993973 17098 ts_tablet_manager.cc:1916] T 9e65676b21b748728ad39668cfe722ec P 1ca2725d290e47d68b1a4b609c9816ac: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:18:59.007191 17098 ts_tablet_manager.cc:1929] T 9e65676b21b748728ad39668cfe722ec P 1ca2725d290e47d68b1a4b609c9816ac: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:18:59.007632 17098 log.cc:1199] T 9e65676b21b748728ad39668cfe722ec P 1ca2725d290e47d68b1a4b609c9816ac: Deleting WAL directory at /tmp/dist-test-taskdH_Ma7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783603093801613-15046-0/minicluster-data/ts-1-root/wals/9e65676b21b748728ad39668cfe722ec
I20260709 13:18:59.009181 16035 catalog_manager.cc:5028] TS 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943): tablet 9e65676b21b748728ad39668cfe722ec (table test-workload [id=ed3d15d8c9ad46bebc4fe3a402d2e54f]) successfully deleted
I20260709 13:18:59.025607 17010 raft_consensus.cc:1064] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac: attempting to promote NON_VOTER cb82cc7bd6db46d980c75237a1402860 to VOTER
I20260709 13:18:59.028342 17010 consensus_queue.cc:237] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:18:59.037292 16299 raft_consensus.cc:1275] T ada5a004ba7c46819051e3ccec71ffdb P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:18:59.039227 16995 consensus_queue.cc:1048] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:18:59.041105 16823 raft_consensus.cc:1275] T ada5a004ba7c46819051e3ccec71ffdb P cb82cc7bd6db46d980c75237a1402860 [term 1 LEARNER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:18:59.042711 16995 consensus_queue.cc:1048] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:18:59.043076 16150 raft_consensus.cc:1275] T ada5a004ba7c46819051e3ccec71ffdb P 647a343a542d4a139fe106c19095129b [term 1 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:18:59.044575 17037 consensus_queue.cc:1048] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:18:59.061128 16995 raft_consensus.cc:2955] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:18:59.062711 16150 raft_consensus.cc:2955] T ada5a004ba7c46819051e3ccec71ffdb P 647a343a542d4a139fe106c19095129b [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:18:59.067090 16823 raft_consensus.cc:2955] T ada5a004ba7c46819051e3ccec71ffdb P cb82cc7bd6db46d980c75237a1402860 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:18:59.084995 16299 raft_consensus.cc:2955] T ada5a004ba7c46819051e3ccec71ffdb P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:18:59.090900 16048 catalog_manager.cc:5697] T ada5a004ba7c46819051e3ccec71ffdb P cb82cc7bd6db46d980c75237a1402860 reported cstate change: config changed from index 2 to 3, cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "1ca2725d290e47d68b1a4b609c9816ac" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:18:59.111588 16224 consensus_queue.cc:237] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:18:59.118479 16823 raft_consensus.cc:1275] T ada5a004ba7c46819051e3ccec71ffdb P cb82cc7bd6db46d980c75237a1402860 [term 1 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:18:59.119815 17010 consensus_queue.cc:1048] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:18:59.120566 16299 raft_consensus.cc:1275] T ada5a004ba7c46819051e3ccec71ffdb P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:18:59.124589 17010 consensus_queue.cc:1048] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:18:59.131047 17032 raft_consensus.cc:2955] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 647a343a542d4a139fe106c19095129b (127.14.177.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:18:59.133248 16823 raft_consensus.cc:2955] T ada5a004ba7c46819051e3ccec71ffdb P cb82cc7bd6db46d980c75237a1402860 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 647a343a542d4a139fe106c19095129b (127.14.177.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:18:59.136777 16299 raft_consensus.cc:2955] T ada5a004ba7c46819051e3ccec71ffdb P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 647a343a542d4a139fe106c19095129b (127.14.177.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:18:59.153805 16035 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet ada5a004ba7c46819051e3ccec71ffdb with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:18:59.156067 16045 catalog_manager.cc:5697] T ada5a004ba7c46819051e3ccec71ffdb P cb82cc7bd6db46d980c75237a1402860 reported cstate change: config changed from index 3 to 4, VOTER 647a343a542d4a139fe106c19095129b (127.14.177.129) evicted. New cstate: current_term: 1 leader_uuid: "1ca2725d290e47d68b1a4b609c9816ac" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:18:59.166435 16136 tablet_service.cc:1558] Processing DeleteTablet for tablet ada5a004ba7c46819051e3ccec71ffdb with delete_type TABLET_DATA_TOMBSTONED (TS 647a343a542d4a139fe106c19095129b not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:40822
I20260709 13:18:59.167415 17092 tablet_replica.cc:333] T ada5a004ba7c46819051e3ccec71ffdb P 647a343a542d4a139fe106c19095129b: stopping tablet replica
I20260709 13:18:59.168501 17092 raft_consensus.cc:2243] T ada5a004ba7c46819051e3ccec71ffdb P 647a343a542d4a139fe106c19095129b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:59.169214 17092 raft_consensus.cc:2272] T ada5a004ba7c46819051e3ccec71ffdb P 647a343a542d4a139fe106c19095129b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:59.179914 17092 ts_tablet_manager.cc:1916] T ada5a004ba7c46819051e3ccec71ffdb P 647a343a542d4a139fe106c19095129b: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:18:59.194191 17092 ts_tablet_manager.cc:1929] T ada5a004ba7c46819051e3ccec71ffdb P 647a343a542d4a139fe106c19095129b: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:18:59.194537 17092 log.cc:1199] T ada5a004ba7c46819051e3ccec71ffdb P 647a343a542d4a139fe106c19095129b: Deleting WAL directory at /tmp/dist-test-taskdH_Ma7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783603093801613-15046-0/minicluster-data/ts-0-root/wals/ada5a004ba7c46819051e3ccec71ffdb
I20260709 13:18:59.195894 16035 catalog_manager.cc:5028] TS 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): tablet ada5a004ba7c46819051e3ccec71ffdb (table test-workload [id=ed3d15d8c9ad46bebc4fe3a402d2e54f]) successfully deleted
I20260709 13:19:00.241654 16145 consensus_queue.cc:237] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } }
I20260709 13:19:00.258194 16298 raft_consensus.cc:1275] T 62fe3346cd3c42c4a55e14af9073bca4 P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 13:19:00.259410 16117 consensus_peers.cc:597] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b -> Peer cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923): Couldn't send request to peer cb82cc7bd6db46d980c75237a1402860. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 62fe3346cd3c42c4a55e14af9073bca4. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:19:00.260082 17043 consensus_queue.cc:1048] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:19:00.260756 16224 raft_consensus.cc:1275] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:19:00.261891 17043 consensus_queue.cc:1048] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:19:00.270203 17055 raft_consensus.cc:2955] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } } }
I20260709 13:19:00.275152 16222 raft_consensus.cc:2955] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } } }
I20260709 13:19:00.275849 16299 raft_consensus.cc:2955] T 62fe3346cd3c42c4a55e14af9073bca4 P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } } }
I20260709 13:19:00.292255 16046 catalog_manager.cc:5697] T 62fe3346cd3c42c4a55e14af9073bca4 P 90f3da57796b488c9087a2f523adf045 reported cstate change: config changed from index -1 to 2, NON_VOTER cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) added. New cstate: current_term: 1 leader_uuid: "647a343a542d4a139fe106c19095129b" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } } }
I20260709 13:19:00.305356 16300 consensus_queue.cc:237] T a378d7f796f94615b3b347119a9880fd P 90f3da57796b488c9087a2f523adf045 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } }
I20260709 13:19:00.319591 16222 raft_consensus.cc:1275] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:19:00.320255 16150 raft_consensus.cc:1275] T a378d7f796f94615b3b347119a9880fd P 647a343a542d4a139fe106c19095129b [term 1 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 13:19:00.320379 16266 consensus_peers.cc:597] T a378d7f796f94615b3b347119a9880fd P 90f3da57796b488c9087a2f523adf045 -> Peer cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923): Couldn't send request to peer cb82cc7bd6db46d980c75237a1402860. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: a378d7f796f94615b3b347119a9880fd. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:19:00.321115 17053 consensus_queue.cc:1048] T a378d7f796f94615b3b347119a9880fd P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:19:00.322095 17077 consensus_queue.cc:1048] T a378d7f796f94615b3b347119a9880fd P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:19:00.339484 17077 raft_consensus.cc:2955] T a378d7f796f94615b3b347119a9880fd P 90f3da57796b488c9087a2f523adf045 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } } }
I20260709 13:19:00.352771 16222 raft_consensus.cc:2955] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } } }
I20260709 13:19:00.365311 16150 consensus_queue.cc:237] T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } }
I20260709 13:19:00.368396 16145 raft_consensus.cc:2955] T a378d7f796f94615b3b347119a9880fd P 647a343a542d4a139fe106c19095129b [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } } }
I20260709 13:19:00.372855 16045 catalog_manager.cc:5697] T a378d7f796f94615b3b347119a9880fd P 90f3da57796b488c9087a2f523adf045 reported cstate change: config changed from index -1 to 2, NON_VOTER cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) added. New cstate: current_term: 1 leader_uuid: "90f3da57796b488c9087a2f523adf045" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:19:00.384456 16300 raft_consensus.cc:1275] T 31969392ccdb43b491911a319e8bc7e1 P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:19:00.384994 16222 raft_consensus.cc:1275] T 31969392ccdb43b491911a319e8bc7e1 P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 13:19:00.385403 16117 consensus_peers.cc:597] T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b -> Peer cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923): Couldn't send request to peer cb82cc7bd6db46d980c75237a1402860. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 31969392ccdb43b491911a319e8bc7e1. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:19:00.386969 17043 consensus_queue.cc:1048] T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:19:00.388667 17106 consensus_queue.cc:1048] T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:19:00.395344 17055 raft_consensus.cc:2955] T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } } }
I20260709 13:19:00.399621 16222 raft_consensus.cc:2955] T 31969392ccdb43b491911a319e8bc7e1 P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } } }
I20260709 13:19:00.401664 16300 raft_consensus.cc:2955] T 31969392ccdb43b491911a319e8bc7e1 P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } } }
I20260709 13:19:00.410077 16150 consensus_queue.cc:237] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } }
I20260709 13:19:00.418723 16046 catalog_manager.cc:5697] T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b reported cstate change: config changed from index -1 to 2, NON_VOTER cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) added. New cstate: current_term: 1 leader_uuid: "647a343a542d4a139fe106c19095129b" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:19:00.422181 16224 raft_consensus.cc:1275] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:19:00.424258 17106 consensus_queue.cc:1048] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:19:00.427805 16117 consensus_peers.cc:597] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b -> Peer cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923): Couldn't send request to peer cb82cc7bd6db46d980c75237a1402860. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 1d5d59407eb545b895d8d922e0dd69fd. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:19:00.430614 16300 raft_consensus.cc:1275] T 1d5d59407eb545b895d8d922e0dd69fd P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:19:00.432236 17106 consensus_queue.cc:1048] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:19:00.446604 17106 raft_consensus.cc:2955] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } } }
I20260709 13:19:00.449110 16299 raft_consensus.cc:2955] T 1d5d59407eb545b895d8d922e0dd69fd P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } } }
I20260709 13:19:00.457825 16222 raft_consensus.cc:2955] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } } }
I20260709 13:19:00.474102 16046 catalog_manager.cc:5697] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b reported cstate change: config changed from index -1 to 2, NON_VOTER cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) added. New cstate: current_term: 1 leader_uuid: "647a343a542d4a139fe106c19095129b" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:19:00.720290 17122 ts_tablet_manager.cc:933] T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860: Initiating tablet copy from peer 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:00.722819 17122 tablet_copy_client.cc:323] T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860: tablet copy: Beginning tablet copy session from remote peer at address 127.14.177.131:41389
I20260709 13:19:00.724980 16310 tablet_copy_service.cc:140] P 90f3da57796b488c9087a2f523adf045: Received BeginTabletCopySession request for tablet a378d7f796f94615b3b347119a9880fd from peer cb82cc7bd6db46d980c75237a1402860 ({username='slave'} at 127.0.0.1:40474)
I20260709 13:19:00.725611 16310 tablet_copy_service.cc:161] P 90f3da57796b488c9087a2f523adf045: Beginning new tablet copy session on tablet a378d7f796f94615b3b347119a9880fd from peer cb82cc7bd6db46d980c75237a1402860 at {username='slave'} at 127.0.0.1:40474: session id = cb82cc7bd6db46d980c75237a1402860-a378d7f796f94615b3b347119a9880fd
I20260709 13:19:00.734447 16310 tablet_copy_source_session.cc:215] T a378d7f796f94615b3b347119a9880fd P 90f3da57796b488c9087a2f523adf045: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:19:00.734714 17124 ts_tablet_manager.cc:933] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860: Initiating tablet copy from peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741)
I20260709 13:19:00.738183 17124 tablet_copy_client.cc:323] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860: tablet copy: Beginning tablet copy session from remote peer at address 127.14.177.129:36741
I20260709 13:19:00.740146 16161 tablet_copy_service.cc:140] P 647a343a542d4a139fe106c19095129b: Received BeginTabletCopySession request for tablet 62fe3346cd3c42c4a55e14af9073bca4 from peer cb82cc7bd6db46d980c75237a1402860 ({username='slave'} at 127.0.0.1:46224)
I20260709 13:19:00.740423 17122 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a378d7f796f94615b3b347119a9880fd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:19:00.740823 16161 tablet_copy_service.cc:161] P 647a343a542d4a139fe106c19095129b: Beginning new tablet copy session on tablet 62fe3346cd3c42c4a55e14af9073bca4 from peer cb82cc7bd6db46d980c75237a1402860 at {username='slave'} at 127.0.0.1:46224: session id = cb82cc7bd6db46d980c75237a1402860-62fe3346cd3c42c4a55e14af9073bca4
I20260709 13:19:00.776893 16161 tablet_copy_source_session.cc:215] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:19:00.780925 17124 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 62fe3346cd3c42c4a55e14af9073bca4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:19:00.817956 17124 tablet_copy_client.cc:806] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860: tablet copy: Starting download of 0 data blocks...
I20260709 13:19:00.818737 17124 tablet_copy_client.cc:670] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860: tablet copy: Starting download of 1 WAL segments...
I20260709 13:19:00.848765 17122 tablet_copy_client.cc:806] T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860: tablet copy: Starting download of 0 data blocks...
I20260709 13:19:00.849565 17122 tablet_copy_client.cc:670] T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860: tablet copy: Starting download of 1 WAL segments...
I20260709 13:19:00.860204 17124 tablet_copy_client.cc:538] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:19:00.863870 17122 tablet_copy_client.cc:538] T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:19:00.881173 17122 tablet_bootstrap.cc:492] T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860: Bootstrap starting.
I20260709 13:19:00.884310 17124 tablet_bootstrap.cc:492] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860: Bootstrap starting.
I20260709 13:19:00.909286 17124 tablet_bootstrap.cc:492] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:19:00.910322 17124 tablet_bootstrap.cc:492] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860: Bootstrap complete.
I20260709 13:19:00.911495 17124 ts_tablet_manager.cc:1403] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860: Time spent bootstrapping tablet: real 0.027s	user 0.017s	sys 0.009s
I20260709 13:19:00.914496 17124 raft_consensus.cc:359] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } }
I20260709 13:19:00.915397 17124 raft_consensus.cc:740] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Initialized, Role: LEARNER
I20260709 13:19:00.915567 17122 tablet_bootstrap.cc:492] T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:19:00.916628 17122 tablet_bootstrap.cc:492] T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860: Bootstrap complete.
I20260709 13:19:00.916154 17124 consensus_queue.cc:260] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } }
I20260709 13:19:00.917528 17122 ts_tablet_manager.cc:1403] T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860: Time spent bootstrapping tablet: real 0.037s	user 0.019s	sys 0.007s
I20260709 13:19:00.927239 17122 raft_consensus.cc:359] T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } }
I20260709 13:19:00.928635 17122 raft_consensus.cc:740] T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Initialized, Role: LEARNER
I20260709 13:19:00.934376 17122 consensus_queue.cc:260] T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } }
I20260709 13:19:00.943292 17124 ts_tablet_manager.cc:1434] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860: Time spent starting tablet: real 0.031s	user 0.005s	sys 0.021s
I20260709 13:19:00.946314 16161 tablet_copy_service.cc:342] P 647a343a542d4a139fe106c19095129b: Request end of tablet copy session cb82cc7bd6db46d980c75237a1402860-62fe3346cd3c42c4a55e14af9073bca4 received from {username='slave'} at 127.0.0.1:46224
I20260709 13:19:00.946470 17122 ts_tablet_manager.cc:1434] T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860: Time spent starting tablet: real 0.024s	user 0.005s	sys 0.010s
I20260709 13:19:00.946851 16161 tablet_copy_service.cc:434] P 647a343a542d4a139fe106c19095129b: ending tablet copy session cb82cc7bd6db46d980c75237a1402860-62fe3346cd3c42c4a55e14af9073bca4 on tablet 62fe3346cd3c42c4a55e14af9073bca4 with peer cb82cc7bd6db46d980c75237a1402860
I20260709 13:19:00.949689 16310 tablet_copy_service.cc:342] P 90f3da57796b488c9087a2f523adf045: Request end of tablet copy session cb82cc7bd6db46d980c75237a1402860-a378d7f796f94615b3b347119a9880fd received from {username='slave'} at 127.0.0.1:40474
I20260709 13:19:00.950165 16310 tablet_copy_service.cc:434] P 90f3da57796b488c9087a2f523adf045: ending tablet copy session cb82cc7bd6db46d980c75237a1402860-a378d7f796f94615b3b347119a9880fd on tablet a378d7f796f94615b3b347119a9880fd with peer cb82cc7bd6db46d980c75237a1402860
I20260709 13:19:00.952980 17124 ts_tablet_manager.cc:933] T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860: Initiating tablet copy from peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741)
I20260709 13:19:00.962260 17122 ts_tablet_manager.cc:933] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860: Initiating tablet copy from peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741)
I20260709 13:19:00.975971 17122 tablet_copy_client.cc:323] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860: tablet copy: Beginning tablet copy session from remote peer at address 127.14.177.129:36741
I20260709 13:19:00.977712 16161 tablet_copy_service.cc:140] P 647a343a542d4a139fe106c19095129b: Received BeginTabletCopySession request for tablet 31969392ccdb43b491911a319e8bc7e1 from peer cb82cc7bd6db46d980c75237a1402860 ({username='slave'} at 127.0.0.1:46224)
I20260709 13:19:00.978243 16161 tablet_copy_service.cc:161] P 647a343a542d4a139fe106c19095129b: Beginning new tablet copy session on tablet 31969392ccdb43b491911a319e8bc7e1 from peer cb82cc7bd6db46d980c75237a1402860 at {username='slave'} at 127.0.0.1:46224: session id = cb82cc7bd6db46d980c75237a1402860-31969392ccdb43b491911a319e8bc7e1
I20260709 13:19:00.986675 16161 tablet_copy_source_session.cc:215] T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:19:00.989897 17122 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 31969392ccdb43b491911a319e8bc7e1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:19:01.011548 17122 tablet_copy_client.cc:806] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860: tablet copy: Starting download of 0 data blocks...
I20260709 13:19:01.012208 17122 tablet_copy_client.cc:670] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860: tablet copy: Starting download of 1 WAL segments...
I20260709 13:19:01.016527 17122 tablet_copy_client.cc:538] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:19:01.025517 17122 tablet_bootstrap.cc:492] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860: Bootstrap starting.
I20260709 13:19:01.060565 17124 tablet_copy_client.cc:323] T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860: tablet copy: Beginning tablet copy session from remote peer at address 127.14.177.129:36741
I20260709 13:19:01.064810 16161 tablet_copy_service.cc:140] P 647a343a542d4a139fe106c19095129b: Received BeginTabletCopySession request for tablet 1d5d59407eb545b895d8d922e0dd69fd from peer cb82cc7bd6db46d980c75237a1402860 ({username='slave'} at 127.0.0.1:46224)
I20260709 13:19:01.065387 16161 tablet_copy_service.cc:161] P 647a343a542d4a139fe106c19095129b: Beginning new tablet copy session on tablet 1d5d59407eb545b895d8d922e0dd69fd from peer cb82cc7bd6db46d980c75237a1402860 at {username='slave'} at 127.0.0.1:46224: session id = cb82cc7bd6db46d980c75237a1402860-1d5d59407eb545b895d8d922e0dd69fd
I20260709 13:19:01.073922 16161 tablet_copy_source_session.cc:215] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:19:01.077482 17124 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1d5d59407eb545b895d8d922e0dd69fd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:19:01.094413 17124 tablet_copy_client.cc:806] T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860: tablet copy: Starting download of 0 data blocks...
I20260709 13:19:01.094983 17124 tablet_copy_client.cc:670] T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860: tablet copy: Starting download of 1 WAL segments...
I20260709 13:19:01.096586 17122 tablet_bootstrap.cc:492] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:19:01.097800 17122 tablet_bootstrap.cc:492] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860: Bootstrap complete.
I20260709 13:19:01.098569 17122 ts_tablet_manager.cc:1403] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860: Time spent bootstrapping tablet: real 0.073s	user 0.015s	sys 0.030s
I20260709 13:19:01.100631 17124 tablet_copy_client.cc:538] T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:19:01.102110 17122 raft_consensus.cc:359] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } }
I20260709 13:19:01.103226 17122 raft_consensus.cc:740] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Initialized, Role: LEARNER
I20260709 13:19:01.104485 17122 consensus_queue.cc:260] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } }
I20260709 13:19:01.107564 17122 ts_tablet_manager.cc:1434] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860: Time spent starting tablet: real 0.009s	user 0.008s	sys 0.000s
I20260709 13:19:01.109972 16161 tablet_copy_service.cc:342] P 647a343a542d4a139fe106c19095129b: Request end of tablet copy session cb82cc7bd6db46d980c75237a1402860-31969392ccdb43b491911a319e8bc7e1 received from {username='slave'} at 127.0.0.1:46224
I20260709 13:19:01.110448 16161 tablet_copy_service.cc:434] P 647a343a542d4a139fe106c19095129b: ending tablet copy session cb82cc7bd6db46d980c75237a1402860-31969392ccdb43b491911a319e8bc7e1 on tablet 31969392ccdb43b491911a319e8bc7e1 with peer cb82cc7bd6db46d980c75237a1402860
I20260709 13:19:01.110581 17124 tablet_bootstrap.cc:492] T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860: Bootstrap starting.
I20260709 13:19:01.144263 17124 tablet_bootstrap.cc:492] T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:19:01.145339 17124 tablet_bootstrap.cc:492] T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860: Bootstrap complete.
I20260709 13:19:01.146114 17124 ts_tablet_manager.cc:1403] T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860: Time spent bootstrapping tablet: real 0.036s	user 0.028s	sys 0.001s
I20260709 13:19:01.149505 17124 raft_consensus.cc:359] T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } }
I20260709 13:19:01.150480 17124 raft_consensus.cc:740] T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Initialized, Role: LEARNER
I20260709 13:19:01.151240 17124 consensus_queue.cc:260] T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } }
I20260709 13:19:01.154140 17124 ts_tablet_manager.cc:1434] T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860: Time spent starting tablet: real 0.008s	user 0.003s	sys 0.003s
I20260709 13:19:01.156003 16161 tablet_copy_service.cc:342] P 647a343a542d4a139fe106c19095129b: Request end of tablet copy session cb82cc7bd6db46d980c75237a1402860-1d5d59407eb545b895d8d922e0dd69fd received from {username='slave'} at 127.0.0.1:46224
I20260709 13:19:01.156476 16161 tablet_copy_service.cc:434] P 647a343a542d4a139fe106c19095129b: ending tablet copy session cb82cc7bd6db46d980c75237a1402860-1d5d59407eb545b895d8d922e0dd69fd on tablet 1d5d59407eb545b895d8d922e0dd69fd with peer cb82cc7bd6db46d980c75237a1402860
I20260709 13:19:01.210763 16822 raft_consensus.cc:1217] T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:19:01.239884 16822 raft_consensus.cc:1217] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:19:01.486075 16822 raft_consensus.cc:1217] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:19:01.501889 16822 raft_consensus.cc:1217] T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:19:01.819227 17114 raft_consensus.cc:1064] T a378d7f796f94615b3b347119a9880fd P 90f3da57796b488c9087a2f523adf045: attempting to promote NON_VOTER cb82cc7bd6db46d980c75237a1402860 to VOTER
I20260709 13:19:01.821434 17114 consensus_queue.cc:237] T a378d7f796f94615b3b347119a9880fd P 90f3da57796b488c9087a2f523adf045 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:01.832793 16822 raft_consensus.cc:1275] T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860 [term 1 LEARNER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:19:01.835069 16222 raft_consensus.cc:1275] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:19:01.843150 17138 consensus_queue.cc:1048] T a378d7f796f94615b3b347119a9880fd P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:19:01.847409 17106 raft_consensus.cc:1064] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b: attempting to promote NON_VOTER cb82cc7bd6db46d980c75237a1402860 to VOTER
I20260709 13:19:01.848078 17138 consensus_queue.cc:1048] T a378d7f796f94615b3b347119a9880fd P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:19:01.850885 17106 consensus_queue.cc:237] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:01.909660 17055 raft_consensus.cc:1064] T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b: attempting to promote NON_VOTER cb82cc7bd6db46d980c75237a1402860 to VOTER
I20260709 13:19:01.914692 17055 consensus_queue.cc:237] T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:01.941431 16298 raft_consensus.cc:1275] T 31969392ccdb43b491911a319e8bc7e1 P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:19:01.962975 16823 raft_consensus.cc:1275] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [term 1 LEARNER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:19:01.973095 17043 consensus_queue.cc:1048] T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:19:01.977072 17147 consensus_queue.cc:1048] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:19:01.978899 16224 raft_consensus.cc:1275] T 31969392ccdb43b491911a319e8bc7e1 P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:19:01.978515 17138 raft_consensus.cc:2955] T a378d7f796f94615b3b347119a9880fd P 90f3da57796b488c9087a2f523adf045 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:19:01.988344 16224 raft_consensus.cc:1275] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:19:02.000355 16822 raft_consensus.cc:1275] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860 [term 1 LEARNER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:19:02.000418 16298 raft_consensus.cc:1275] T 62fe3346cd3c42c4a55e14af9073bca4 P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:19:02.003679 17055 consensus_queue.cc:1048] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:19:02.004426 16823 raft_consensus.cc:2955] T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:19:02.013212 17147 consensus_queue.cc:1048] T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:19:02.043555 16144 raft_consensus.cc:1275] T a378d7f796f94615b3b347119a9880fd P 647a343a542d4a139fe106c19095129b [term 1 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:19:02.045373 17149 consensus_queue.cc:1048] T a378d7f796f94615b3b347119a9880fd P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:19:02.048753 16224 raft_consensus.cc:2955] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:19:02.062788 17043 raft_consensus.cc:1064] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b: attempting to promote NON_VOTER cb82cc7bd6db46d980c75237a1402860 to VOTER
I20260709 13:19:02.065155 17043 consensus_queue.cc:237] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:02.068496 16047 catalog_manager.cc:5697] T a378d7f796f94615b3b347119a9880fd P 90f3da57796b488c9087a2f523adf045 reported cstate change: config changed from index 2 to 3, cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "90f3da57796b488c9087a2f523adf045" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:19:02.072369 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 88982.1us
I20260709 13:19:02.076148 17056 consensus_queue.cc:1048] T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:19:02.076936 17055 raft_consensus.cc:2955] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:19:02.085750 16821 raft_consensus.cc:1275] T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860 [term 1 LEARNER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:19:02.086513 16825 raft_consensus.cc:2955] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:19:02.089075 17147 consensus_queue.cc:1048] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:19:02.092798 16296 raft_consensus.cc:1275] T 1d5d59407eb545b895d8d922e0dd69fd P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:19:02.093568 17106 consensus_queue.cc:1048] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:19:02.105765 16144 raft_consensus.cc:2955] T a378d7f796f94615b3b347119a9880fd P 647a343a542d4a139fe106c19095129b [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:19:02.120908 17043 raft_consensus.cc:2955] T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:19:02.122808 16824 raft_consensus.cc:2955] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:19:02.128288 17163 consensus_queue.cc:1048] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:19:02.130609 16298 raft_consensus.cc:2955] T 62fe3346cd3c42c4a55e14af9073bca4 P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:19:02.156167 16222 raft_consensus.cc:1275] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:19:02.154560 16300 raft_consensus.cc:2955] T 31969392ccdb43b491911a319e8bc7e1 P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:19:02.172508 16048 catalog_manager.cc:5697] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b reported cstate change: config changed from index 2 to 3, cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "647a343a542d4a139fe106c19095129b" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:19:02.175473 17043 consensus_queue.cc:1048] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:19:02.195832 16223 raft_consensus.cc:2955] T 31969392ccdb43b491911a319e8bc7e1 P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:19:02.218191 17152 raft_consensus.cc:2955] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:19:02.227016 16222 raft_consensus.cc:2955] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:19:02.236548 16824 raft_consensus.cc:2955] T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:19:02.266919 16226 raft_consensus.cc:2955] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:19:02.288802 16300 consensus_queue.cc:237] T a378d7f796f94615b3b347119a9880fd P 90f3da57796b488c9087a2f523adf045 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:02.299546 16824 raft_consensus.cc:1275] T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860 [term 1 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:19:02.300982 16222 raft_consensus.cc:1275] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:19:02.301549 17149 consensus_queue.cc:1048] T a378d7f796f94615b3b347119a9880fd P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:19:02.302459 17138 consensus_queue.cc:1048] T a378d7f796f94615b3b347119a9880fd P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:19:02.306988 16047 catalog_manager.cc:5697] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860 reported cstate change: config changed from index 2 to 3, cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "647a343a542d4a139fe106c19095129b" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:19:02.308581 17149 raft_consensus.cc:2955] T a378d7f796f94615b3b347119a9880fd P 90f3da57796b488c9087a2f523adf045 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 647a343a542d4a139fe106c19095129b (127.14.177.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:19:02.311442 16223 raft_consensus.cc:2955] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 647a343a542d4a139fe106c19095129b (127.14.177.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:19:02.312150 16821 raft_consensus.cc:2955] T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 647a343a542d4a139fe106c19095129b (127.14.177.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:19:02.317800 16047 catalog_manager.cc:5697] T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860 reported cstate change: config changed from index 2 to 3, cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "647a343a542d4a139fe106c19095129b" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:19:02.321441 16300 raft_consensus.cc:2955] T 1d5d59407eb545b895d8d922e0dd69fd P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:19:02.326637 16034 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet a378d7f796f94615b3b347119a9880fd with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
W20260709 13:19:02.355394 16034 catalog_manager.cc:5220] ChangeConfig:REMOVE_PEER RPC for tablet a378d7f796f94615b3b347119a9880fd with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER failed with leader 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389) due to CAS failure; no further retry: Illegal state: Request specified cas_config_opid_index of 3 but the committed config has opid_index of 4
I20260709 13:19:02.356016 16047 catalog_manager.cc:5697] T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860 reported cstate change: config changed from index 3 to 4, VOTER 647a343a542d4a139fe106c19095129b (127.14.177.129) evicted. New cstate: current_term: 1 leader_uuid: "90f3da57796b488c9087a2f523adf045" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:19:02.356388 16150 consensus_queue.cc:237] T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:02.362543 16821 raft_consensus.cc:1275] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860 [term 1 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:19:02.364058 17162 consensus_queue.cc:1048] T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:19:02.366132 16136 tablet_service.cc:1558] Processing DeleteTablet for tablet a378d7f796f94615b3b347119a9880fd with delete_type TABLET_DATA_TOMBSTONED (TS 647a343a542d4a139fe106c19095129b not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:40822
I20260709 13:19:02.374675 16300 raft_consensus.cc:1275] T 31969392ccdb43b491911a319e8bc7e1 P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:19:02.374372 17170 tablet_replica.cc:333] T a378d7f796f94615b3b347119a9880fd P 647a343a542d4a139fe106c19095129b: stopping tablet replica
I20260709 13:19:02.378720 17170 raft_consensus.cc:2243] T a378d7f796f94615b3b347119a9880fd P 647a343a542d4a139fe106c19095129b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:19:02.379519 17170 raft_consensus.cc:2272] T a378d7f796f94615b3b347119a9880fd P 647a343a542d4a139fe106c19095129b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:19:02.378780 17055 raft_consensus.cc:2955] T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:19:02.382441 16150 consensus_queue.cc:237] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:02.383316 17043 consensus_queue.cc:1048] T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:19:02.386663 17170 ts_tablet_manager.cc:1916] T a378d7f796f94615b3b347119a9880fd P 647a343a542d4a139fe106c19095129b: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:19:02.390089 16821 raft_consensus.cc:1275] T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860 [term 1 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:19:02.391420 16824 raft_consensus.cc:2955] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:19:02.393390 16300 raft_consensus.cc:2955] T 31969392ccdb43b491911a319e8bc7e1 P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:19:02.398343 16223 raft_consensus.cc:1275] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:19:02.399457 16035 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 31969392ccdb43b491911a319e8bc7e1 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:19:02.402892 17162 consensus_queue.cc:1048] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:19:02.408561 17163 consensus_queue.cc:1048] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:19:02.410102 17056 raft_consensus.cc:2955] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 90f3da57796b488c9087a2f523adf045 (127.14.177.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:19:02.408761 16048 catalog_manager.cc:5697] T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b reported cstate change: config changed from index 3 to 4, VOTER 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130) evicted. New cstate: current_term: 1 leader_uuid: "647a343a542d4a139fe106c19095129b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:19:02.416204 16223 raft_consensus.cc:2955] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 90f3da57796b488c9087a2f523adf045 (127.14.177.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:19:02.424407 16209 tablet_service.cc:1558] Processing DeleteTablet for tablet 31969392ccdb43b491911a319e8bc7e1 with delete_type TABLET_DATA_TOMBSTONED (TS 1ca2725d290e47d68b1a4b609c9816ac not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:57598
I20260709 13:19:02.429507 16035 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 1d5d59407eb545b895d8d922e0dd69fd with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:19:02.431054 17171 tablet_replica.cc:333] T 31969392ccdb43b491911a319e8bc7e1 P 1ca2725d290e47d68b1a4b609c9816ac: stopping tablet replica
I20260709 13:19:02.432235 17171 raft_consensus.cc:2243] T 31969392ccdb43b491911a319e8bc7e1 P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:19:02.436678 17171 raft_consensus.cc:2272] T 31969392ccdb43b491911a319e8bc7e1 P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:19:02.441092 17171 ts_tablet_manager.cc:1916] T 31969392ccdb43b491911a319e8bc7e1 P 1ca2725d290e47d68b1a4b609c9816ac: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:19:02.435876 16823 raft_consensus.cc:2955] T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 90f3da57796b488c9087a2f523adf045 (127.14.177.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:19:02.453061 16048 catalog_manager.cc:5697] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b reported cstate change: config changed from index 3 to 4, VOTER 90f3da57796b488c9087a2f523adf045 (127.14.177.131) evicted. New cstate: current_term: 1 leader_uuid: "647a343a542d4a139fe106c19095129b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:19:02.471658 17171 ts_tablet_manager.cc:1929] T 31969392ccdb43b491911a319e8bc7e1 P 1ca2725d290e47d68b1a4b609c9816ac: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:19:02.475997 17171 log.cc:1199] T 31969392ccdb43b491911a319e8bc7e1 P 1ca2725d290e47d68b1a4b609c9816ac: Deleting WAL directory at /tmp/dist-test-taskdH_Ma7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783603093801613-15046-0/minicluster-data/ts-1-root/wals/31969392ccdb43b491911a319e8bc7e1
I20260709 13:19:02.477130 16286 tablet_service.cc:1558] Processing DeleteTablet for tablet 1d5d59407eb545b895d8d922e0dd69fd with delete_type TABLET_DATA_TOMBSTONED (TS 90f3da57796b488c9087a2f523adf045 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:48670
I20260709 13:19:02.478058 16035 catalog_manager.cc:5028] TS 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943): tablet 31969392ccdb43b491911a319e8bc7e1 (table test-workload [id=ed3d15d8c9ad46bebc4fe3a402d2e54f]) successfully deleted
I20260709 13:19:02.482933 17170 ts_tablet_manager.cc:1929] T a378d7f796f94615b3b347119a9880fd P 647a343a542d4a139fe106c19095129b: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:19:02.483394 17170 log.cc:1199] T a378d7f796f94615b3b347119a9880fd P 647a343a542d4a139fe106c19095129b: Deleting WAL directory at /tmp/dist-test-taskdH_Ma7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783603093801613-15046-0/minicluster-data/ts-0-root/wals/a378d7f796f94615b3b347119a9880fd
I20260709 13:19:02.484428 17172 tablet_replica.cc:333] T 1d5d59407eb545b895d8d922e0dd69fd P 90f3da57796b488c9087a2f523adf045: stopping tablet replica
I20260709 13:19:02.485050 16035 catalog_manager.cc:5028] TS 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): tablet a378d7f796f94615b3b347119a9880fd (table test-workload [id=ed3d15d8c9ad46bebc4fe3a402d2e54f]) successfully deleted
I20260709 13:19:02.485546 17172 raft_consensus.cc:2243] T 1d5d59407eb545b895d8d922e0dd69fd P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:19:02.486302 17172 raft_consensus.cc:2272] T 1d5d59407eb545b895d8d922e0dd69fd P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:19:02.489837 17172 ts_tablet_manager.cc:1916] T 1d5d59407eb545b895d8d922e0dd69fd P 90f3da57796b488c9087a2f523adf045: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:19:02.505215 17172 ts_tablet_manager.cc:1929] T 1d5d59407eb545b895d8d922e0dd69fd P 90f3da57796b488c9087a2f523adf045: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:19:02.505623 17172 log.cc:1199] T 1d5d59407eb545b895d8d922e0dd69fd P 90f3da57796b488c9087a2f523adf045: Deleting WAL directory at /tmp/dist-test-taskdH_Ma7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783603093801613-15046-0/minicluster-data/ts-2-root/wals/1d5d59407eb545b895d8d922e0dd69fd
I20260709 13:19:02.507326 16034 catalog_manager.cc:5028] TS 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389): tablet 1d5d59407eb545b895d8d922e0dd69fd (table test-workload [id=ed3d15d8c9ad46bebc4fe3a402d2e54f]) successfully deleted
I20260709 13:19:02.677438 16150 consensus_queue.cc:237] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:02.681486 16823 raft_consensus.cc:1275] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [term 1 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:19:02.681734 16222 raft_consensus.cc:1275] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:19:02.683068 17162 consensus_queue.cc:1048] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:19:02.684512 17152 consensus_queue.cc:1048] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:19:02.689395 17152 raft_consensus.cc:2955] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 90f3da57796b488c9087a2f523adf045 (127.14.177.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:19:02.693249 16823 raft_consensus.cc:2955] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 90f3da57796b488c9087a2f523adf045 (127.14.177.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:19:02.694197 16222 raft_consensus.cc:2955] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 90f3da57796b488c9087a2f523adf045 (127.14.177.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:19:02.704548 16035 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 62fe3346cd3c42c4a55e14af9073bca4 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:19:02.707525 16048 catalog_manager.cc:5697] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 reported cstate change: config changed from index 3 to 4, VOTER 90f3da57796b488c9087a2f523adf045 (127.14.177.131) evicted. New cstate: current_term: 1 leader_uuid: "647a343a542d4a139fe106c19095129b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:19:02.722873 16286 tablet_service.cc:1558] Processing DeleteTablet for tablet 62fe3346cd3c42c4a55e14af9073bca4 with delete_type TABLET_DATA_TOMBSTONED (TS 90f3da57796b488c9087a2f523adf045 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:48670
I20260709 13:19:02.723841 17172 tablet_replica.cc:333] T 62fe3346cd3c42c4a55e14af9073bca4 P 90f3da57796b488c9087a2f523adf045: stopping tablet replica
I20260709 13:19:02.724635 17172 raft_consensus.cc:2243] T 62fe3346cd3c42c4a55e14af9073bca4 P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:19:02.725261 17172 raft_consensus.cc:2272] T 62fe3346cd3c42c4a55e14af9073bca4 P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:19:02.728377 17172 ts_tablet_manager.cc:1916] T 62fe3346cd3c42c4a55e14af9073bca4 P 90f3da57796b488c9087a2f523adf045: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:19:02.741178 17172 ts_tablet_manager.cc:1929] T 62fe3346cd3c42c4a55e14af9073bca4 P 90f3da57796b488c9087a2f523adf045: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:19:02.741528 17172 log.cc:1199] T 62fe3346cd3c42c4a55e14af9073bca4 P 90f3da57796b488c9087a2f523adf045: Deleting WAL directory at /tmp/dist-test-taskdH_Ma7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783603093801613-15046-0/minicluster-data/ts-2-root/wals/62fe3346cd3c42c4a55e14af9073bca4
I20260709 13:19:02.742743 16034 catalog_manager.cc:5028] TS 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389): tablet 62fe3346cd3c42c4a55e14af9073bca4 (table test-workload [id=ed3d15d8c9ad46bebc4fe3a402d2e54f]) successfully deleted
I20260709 13:19:03.787636 16222 consensus_queue.cc:237] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } }
W20260709 13:19:03.794905 16192 consensus_peers.cc:597] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 1ca2725d290e47d68b1a4b609c9816ac -> Peer cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923): Couldn't send request to peer cb82cc7bd6db46d980c75237a1402860. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 3a7ca61592e848ea9b0fb81a2ce33cdc. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:19:03.794999 16300 raft_consensus.cc:1275] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:19:03.795964 16144 raft_consensus.cc:1275] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 647a343a542d4a139fe106c19095129b [term 1 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:19:03.796447 17173 consensus_queue.cc:1048] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:19:03.798472 17144 consensus_queue.cc:1048] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:19:03.806372 17173 raft_consensus.cc:2955] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 1ca2725d290e47d68b1a4b609c9816ac [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } } }
I20260709 13:19:03.808976 16296 raft_consensus.cc:2955] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } } }
I20260709 13:19:03.810190 16150 raft_consensus.cc:2955] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 647a343a542d4a139fe106c19095129b [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } } }
I20260709 13:19:03.821755 16144 consensus_queue.cc:237] T 3884f864a96d428785895e5463e5728f P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } }
I20260709 13:19:03.826143 16048 catalog_manager.cc:5697] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 90f3da57796b488c9087a2f523adf045 reported cstate change: config changed from index -1 to 2, NON_VOTER cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) added. New cstate: current_term: 1 leader_uuid: "1ca2725d290e47d68b1a4b609c9816ac" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } } }
W20260709 13:19:03.831175 16117 consensus_peers.cc:597] T 3884f864a96d428785895e5463e5728f P 647a343a542d4a139fe106c19095129b -> Peer cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923): Couldn't send request to peer cb82cc7bd6db46d980c75237a1402860. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 3884f864a96d428785895e5463e5728f. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:19:03.832186 16300 raft_consensus.cc:1275] T 3884f864a96d428785895e5463e5728f P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:19:03.832674 16222 raft_consensus.cc:1275] T 3884f864a96d428785895e5463e5728f P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:19:03.836675 17152 consensus_queue.cc:1048] T 3884f864a96d428785895e5463e5728f P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:19:03.841135 17181 consensus_queue.cc:1048] T 3884f864a96d428785895e5463e5728f P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:19:03.851972 17174 raft_consensus.cc:2955] T 3884f864a96d428785895e5463e5728f P 647a343a542d4a139fe106c19095129b [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } } }
I20260709 13:19:03.855793 16222 raft_consensus.cc:2955] T 3884f864a96d428785895e5463e5728f P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } } }
I20260709 13:19:03.858947 16296 raft_consensus.cc:2955] T 3884f864a96d428785895e5463e5728f P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } } }
I20260709 13:19:03.887603 16222 consensus_queue.cc:237] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } }
I20260709 13:19:03.895005 16296 raft_consensus.cc:1275] T 8638ee363273426db1393dd81e5c7b72 P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 13:19:03.895416 16192 consensus_peers.cc:597] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac -> Peer cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923): Couldn't send request to peer cb82cc7bd6db46d980c75237a1402860. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 8638ee363273426db1393dd81e5c7b72. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:19:03.896456 17144 consensus_queue.cc:1048] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:19:03.898409 16150 raft_consensus.cc:1275] T 8638ee363273426db1393dd81e5c7b72 P 647a343a542d4a139fe106c19095129b [term 1 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:19:03.903509 17173 consensus_queue.cc:1048] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:19:03.907927 16048 catalog_manager.cc:5697] T 3884f864a96d428785895e5463e5728f P 90f3da57796b488c9087a2f523adf045 reported cstate change: config changed from index -1 to 2, NON_VOTER cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) added. New cstate: current_term: 1 leader_uuid: "647a343a542d4a139fe106c19095129b" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } } }
I20260709 13:19:03.919407 17144 raft_consensus.cc:2955] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } } }
I20260709 13:19:03.921407 16144 raft_consensus.cc:2955] T 8638ee363273426db1393dd81e5c7b72 P 647a343a542d4a139fe106c19095129b [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } } }
I20260709 13:19:03.929049 16300 raft_consensus.cc:2955] T 8638ee363273426db1393dd81e5c7b72 P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } } }
I20260709 13:19:03.940188 16296 consensus_queue.cc:237] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } }
W20260709 13:19:03.946955 16266 consensus_peers.cc:597] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 90f3da57796b488c9087a2f523adf045 -> Peer cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923): Couldn't send request to peer cb82cc7bd6db46d980c75237a1402860. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 4e9b2bf7011f4ae688a79d7bb5a447b4. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:19:03.948015 16223 raft_consensus.cc:1275] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:19:03.948654 16046 catalog_manager.cc:5697] T 8638ee363273426db1393dd81e5c7b72 P 647a343a542d4a139fe106c19095129b reported cstate change: config changed from index -1 to 2, NON_VOTER cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) added. New cstate: current_term: 1 leader_uuid: "1ca2725d290e47d68b1a4b609c9816ac" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } } }
I20260709 13:19:03.949450 17138 consensus_queue.cc:1048] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:19:03.950872 16144 raft_consensus.cc:1275] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 647a343a542d4a139fe106c19095129b [term 1 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:19:03.952837 17138 consensus_queue.cc:1048] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:19:03.964015 17191 raft_consensus.cc:2955] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 90f3da57796b488c9087a2f523adf045 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } } }
I20260709 13:19:03.965658 16144 raft_consensus.cc:2955] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 647a343a542d4a139fe106c19095129b [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } } }
I20260709 13:19:03.967367 16223 raft_consensus.cc:2955] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } } }
I20260709 13:19:03.979830 16048 catalog_manager.cc:5697] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 647a343a542d4a139fe106c19095129b reported cstate change: config changed from index -1 to 2, NON_VOTER cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) added. New cstate: current_term: 1 leader_uuid: "90f3da57796b488c9087a2f523adf045" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } } }
I20260709 13:19:04.254864 17194 ts_tablet_manager.cc:933] T 3884f864a96d428785895e5463e5728f P cb82cc7bd6db46d980c75237a1402860: Initiating tablet copy from peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741)
I20260709 13:19:04.256703 17194 tablet_copy_client.cc:323] T 3884f864a96d428785895e5463e5728f P cb82cc7bd6db46d980c75237a1402860: tablet copy: Beginning tablet copy session from remote peer at address 127.14.177.129:36741
I20260709 13:19:04.258183 16161 tablet_copy_service.cc:140] P 647a343a542d4a139fe106c19095129b: Received BeginTabletCopySession request for tablet 3884f864a96d428785895e5463e5728f from peer cb82cc7bd6db46d980c75237a1402860 ({username='slave'} at 127.0.0.1:46224)
I20260709 13:19:04.258793 16161 tablet_copy_service.cc:161] P 647a343a542d4a139fe106c19095129b: Beginning new tablet copy session on tablet 3884f864a96d428785895e5463e5728f from peer cb82cc7bd6db46d980c75237a1402860 at {username='slave'} at 127.0.0.1:46224: session id = cb82cc7bd6db46d980c75237a1402860-3884f864a96d428785895e5463e5728f
I20260709 13:19:04.265298 16161 tablet_copy_source_session.cc:215] T 3884f864a96d428785895e5463e5728f P 647a343a542d4a139fe106c19095129b: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:19:04.268222 17194 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3884f864a96d428785895e5463e5728f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:19:04.281998 17194 tablet_copy_client.cc:806] T 3884f864a96d428785895e5463e5728f P cb82cc7bd6db46d980c75237a1402860: tablet copy: Starting download of 0 data blocks...
I20260709 13:19:04.282486 17194 tablet_copy_client.cc:670] T 3884f864a96d428785895e5463e5728f P cb82cc7bd6db46d980c75237a1402860: tablet copy: Starting download of 1 WAL segments...
I20260709 13:19:04.286278 17194 tablet_copy_client.cc:538] T 3884f864a96d428785895e5463e5728f P cb82cc7bd6db46d980c75237a1402860: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:19:04.294211 17194 tablet_bootstrap.cc:492] T 3884f864a96d428785895e5463e5728f P cb82cc7bd6db46d980c75237a1402860: Bootstrap starting.
I20260709 13:19:04.310441 17194 tablet_bootstrap.cc:492] T 3884f864a96d428785895e5463e5728f P cb82cc7bd6db46d980c75237a1402860: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:19:04.311133 17194 tablet_bootstrap.cc:492] T 3884f864a96d428785895e5463e5728f P cb82cc7bd6db46d980c75237a1402860: Bootstrap complete.
I20260709 13:19:04.311719 17194 ts_tablet_manager.cc:1403] T 3884f864a96d428785895e5463e5728f P cb82cc7bd6db46d980c75237a1402860: Time spent bootstrapping tablet: real 0.018s	user 0.015s	sys 0.004s
I20260709 13:19:04.313835 17194 raft_consensus.cc:359] T 3884f864a96d428785895e5463e5728f P cb82cc7bd6db46d980c75237a1402860 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } }
I20260709 13:19:04.314429 17194 raft_consensus.cc:740] T 3884f864a96d428785895e5463e5728f P cb82cc7bd6db46d980c75237a1402860 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Initialized, Role: LEARNER
I20260709 13:19:04.314940 17194 consensus_queue.cc:260] T 3884f864a96d428785895e5463e5728f P cb82cc7bd6db46d980c75237a1402860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } }
I20260709 13:19:04.317896 17194 ts_tablet_manager.cc:1434] T 3884f864a96d428785895e5463e5728f P cb82cc7bd6db46d980c75237a1402860: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 13:19:04.320534 16161 tablet_copy_service.cc:342] P 647a343a542d4a139fe106c19095129b: Request end of tablet copy session cb82cc7bd6db46d980c75237a1402860-3884f864a96d428785895e5463e5728f received from {username='slave'} at 127.0.0.1:46224
I20260709 13:19:04.321024 16161 tablet_copy_service.cc:434] P 647a343a542d4a139fe106c19095129b: ending tablet copy session cb82cc7bd6db46d980c75237a1402860-3884f864a96d428785895e5463e5728f on tablet 3884f864a96d428785895e5463e5728f with peer cb82cc7bd6db46d980c75237a1402860
I20260709 13:19:04.341246 17194 ts_tablet_manager.cc:933] T 8638ee363273426db1393dd81e5c7b72 P cb82cc7bd6db46d980c75237a1402860: Initiating tablet copy from peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:19:04.343099 17194 tablet_copy_client.cc:323] T 8638ee363273426db1393dd81e5c7b72 P cb82cc7bd6db46d980c75237a1402860: tablet copy: Beginning tablet copy session from remote peer at address 127.14.177.130:35943
I20260709 13:19:04.344767 16236 tablet_copy_service.cc:140] P 1ca2725d290e47d68b1a4b609c9816ac: Received BeginTabletCopySession request for tablet 8638ee363273426db1393dd81e5c7b72 from peer cb82cc7bd6db46d980c75237a1402860 ({username='slave'} at 127.0.0.1:33050)
I20260709 13:19:04.345247 16236 tablet_copy_service.cc:161] P 1ca2725d290e47d68b1a4b609c9816ac: Beginning new tablet copy session on tablet 8638ee363273426db1393dd81e5c7b72 from peer cb82cc7bd6db46d980c75237a1402860 at {username='slave'} at 127.0.0.1:33050: session id = cb82cc7bd6db46d980c75237a1402860-8638ee363273426db1393dd81e5c7b72
I20260709 13:19:04.355882 16236 tablet_copy_source_session.cc:215] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:19:04.358953 17194 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8638ee363273426db1393dd81e5c7b72. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:19:04.386464 17199 ts_tablet_manager.cc:933] T 3a7ca61592e848ea9b0fb81a2ce33cdc P cb82cc7bd6db46d980c75237a1402860: Initiating tablet copy from peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:19:04.388547 17199 tablet_copy_client.cc:323] T 3a7ca61592e848ea9b0fb81a2ce33cdc P cb82cc7bd6db46d980c75237a1402860: tablet copy: Beginning tablet copy session from remote peer at address 127.14.177.130:35943
I20260709 13:19:04.390195 16236 tablet_copy_service.cc:140] P 1ca2725d290e47d68b1a4b609c9816ac: Received BeginTabletCopySession request for tablet 3a7ca61592e848ea9b0fb81a2ce33cdc from peer cb82cc7bd6db46d980c75237a1402860 ({username='slave'} at 127.0.0.1:33050)
I20260709 13:19:04.390715 16236 tablet_copy_service.cc:161] P 1ca2725d290e47d68b1a4b609c9816ac: Beginning new tablet copy session on tablet 3a7ca61592e848ea9b0fb81a2ce33cdc from peer cb82cc7bd6db46d980c75237a1402860 at {username='slave'} at 127.0.0.1:33050: session id = cb82cc7bd6db46d980c75237a1402860-3a7ca61592e848ea9b0fb81a2ce33cdc
I20260709 13:19:04.397269 17194 tablet_copy_client.cc:806] T 8638ee363273426db1393dd81e5c7b72 P cb82cc7bd6db46d980c75237a1402860: tablet copy: Starting download of 0 data blocks...
I20260709 13:19:04.397954 17194 tablet_copy_client.cc:670] T 8638ee363273426db1393dd81e5c7b72 P cb82cc7bd6db46d980c75237a1402860: tablet copy: Starting download of 1 WAL segments...
I20260709 13:19:04.412451 16236 tablet_copy_source_session.cc:215] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 1ca2725d290e47d68b1a4b609c9816ac: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:19:04.414937 17194 tablet_copy_client.cc:538] T 8638ee363273426db1393dd81e5c7b72 P cb82cc7bd6db46d980c75237a1402860: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:19:04.420419 17199 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3a7ca61592e848ea9b0fb81a2ce33cdc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:19:04.425786 17194 tablet_bootstrap.cc:492] T 8638ee363273426db1393dd81e5c7b72 P cb82cc7bd6db46d980c75237a1402860: Bootstrap starting.
I20260709 13:19:04.442638 17199 tablet_copy_client.cc:806] T 3a7ca61592e848ea9b0fb81a2ce33cdc P cb82cc7bd6db46d980c75237a1402860: tablet copy: Starting download of 0 data blocks...
I20260709 13:19:04.443243 17199 tablet_copy_client.cc:670] T 3a7ca61592e848ea9b0fb81a2ce33cdc P cb82cc7bd6db46d980c75237a1402860: tablet copy: Starting download of 1 WAL segments...
I20260709 13:19:04.446987 17199 tablet_copy_client.cc:538] T 3a7ca61592e848ea9b0fb81a2ce33cdc P cb82cc7bd6db46d980c75237a1402860: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:19:04.451488 17194 tablet_bootstrap.cc:492] T 8638ee363273426db1393dd81e5c7b72 P cb82cc7bd6db46d980c75237a1402860: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:19:04.452562 17194 tablet_bootstrap.cc:492] T 8638ee363273426db1393dd81e5c7b72 P cb82cc7bd6db46d980c75237a1402860: Bootstrap complete.
I20260709 13:19:04.453344 17194 ts_tablet_manager.cc:1403] T 8638ee363273426db1393dd81e5c7b72 P cb82cc7bd6db46d980c75237a1402860: Time spent bootstrapping tablet: real 0.028s	user 0.022s	sys 0.000s
I20260709 13:19:04.455904 17199 tablet_bootstrap.cc:492] T 3a7ca61592e848ea9b0fb81a2ce33cdc P cb82cc7bd6db46d980c75237a1402860: Bootstrap starting.
I20260709 13:19:04.457849 17194 raft_consensus.cc:359] T 8638ee363273426db1393dd81e5c7b72 P cb82cc7bd6db46d980c75237a1402860 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } }
I20260709 13:19:04.458809 17194 raft_consensus.cc:740] T 8638ee363273426db1393dd81e5c7b72 P cb82cc7bd6db46d980c75237a1402860 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Initialized, Role: LEARNER
I20260709 13:19:04.459558 17194 consensus_queue.cc:260] T 8638ee363273426db1393dd81e5c7b72 P cb82cc7bd6db46d980c75237a1402860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } }
I20260709 13:19:04.462286 17194 ts_tablet_manager.cc:1434] T 8638ee363273426db1393dd81e5c7b72 P cb82cc7bd6db46d980c75237a1402860: Time spent starting tablet: real 0.009s	user 0.008s	sys 0.000s
I20260709 13:19:04.464190 16236 tablet_copy_service.cc:342] P 1ca2725d290e47d68b1a4b609c9816ac: Request end of tablet copy session cb82cc7bd6db46d980c75237a1402860-8638ee363273426db1393dd81e5c7b72 received from {username='slave'} at 127.0.0.1:33050
I20260709 13:19:04.464699 16236 tablet_copy_service.cc:434] P 1ca2725d290e47d68b1a4b609c9816ac: ending tablet copy session cb82cc7bd6db46d980c75237a1402860-8638ee363273426db1393dd81e5c7b72 on tablet 8638ee363273426db1393dd81e5c7b72 with peer cb82cc7bd6db46d980c75237a1402860
I20260709 13:19:04.469992 17194 ts_tablet_manager.cc:933] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860: Initiating tablet copy from peer 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:04.481973 17194 tablet_copy_client.cc:323] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860: tablet copy: Beginning tablet copy session from remote peer at address 127.14.177.131:41389
I20260709 13:19:04.483974 16310 tablet_copy_service.cc:140] P 90f3da57796b488c9087a2f523adf045: Received BeginTabletCopySession request for tablet 4e9b2bf7011f4ae688a79d7bb5a447b4 from peer cb82cc7bd6db46d980c75237a1402860 ({username='slave'} at 127.0.0.1:40474)
I20260709 13:19:04.484529 16310 tablet_copy_service.cc:161] P 90f3da57796b488c9087a2f523adf045: Beginning new tablet copy session on tablet 4e9b2bf7011f4ae688a79d7bb5a447b4 from peer cb82cc7bd6db46d980c75237a1402860 at {username='slave'} at 127.0.0.1:40474: session id = cb82cc7bd6db46d980c75237a1402860-4e9b2bf7011f4ae688a79d7bb5a447b4
I20260709 13:19:04.486209 17199 tablet_bootstrap.cc:492] T 3a7ca61592e848ea9b0fb81a2ce33cdc P cb82cc7bd6db46d980c75237a1402860: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:19:04.487211 17199 tablet_bootstrap.cc:492] T 3a7ca61592e848ea9b0fb81a2ce33cdc P cb82cc7bd6db46d980c75237a1402860: Bootstrap complete.
I20260709 13:19:04.488385 17199 ts_tablet_manager.cc:1403] T 3a7ca61592e848ea9b0fb81a2ce33cdc P cb82cc7bd6db46d980c75237a1402860: Time spent bootstrapping tablet: real 0.032s	user 0.019s	sys 0.008s
I20260709 13:19:04.491501 17199 raft_consensus.cc:359] T 3a7ca61592e848ea9b0fb81a2ce33cdc P cb82cc7bd6db46d980c75237a1402860 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } }
I20260709 13:19:04.492260 17199 raft_consensus.cc:740] T 3a7ca61592e848ea9b0fb81a2ce33cdc P cb82cc7bd6db46d980c75237a1402860 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Initialized, Role: LEARNER
I20260709 13:19:04.492944 17199 consensus_queue.cc:260] T 3a7ca61592e848ea9b0fb81a2ce33cdc P cb82cc7bd6db46d980c75237a1402860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } }
I20260709 13:19:04.540125 17199 ts_tablet_manager.cc:1434] T 3a7ca61592e848ea9b0fb81a2ce33cdc P cb82cc7bd6db46d980c75237a1402860: Time spent starting tablet: real 0.051s	user 0.005s	sys 0.005s
I20260709 13:19:04.589205 16310 tablet_copy_source_session.cc:215] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 90f3da57796b488c9087a2f523adf045: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:19:04.605468 17194 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4e9b2bf7011f4ae688a79d7bb5a447b4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:19:04.628662 16236 tablet_copy_service.cc:342] P 1ca2725d290e47d68b1a4b609c9816ac: Request end of tablet copy session cb82cc7bd6db46d980c75237a1402860-3a7ca61592e848ea9b0fb81a2ce33cdc received from {username='slave'} at 127.0.0.1:33050
I20260709 13:19:04.629521 16236 tablet_copy_service.cc:434] P 1ca2725d290e47d68b1a4b609c9816ac: ending tablet copy session cb82cc7bd6db46d980c75237a1402860-3a7ca61592e848ea9b0fb81a2ce33cdc on tablet 3a7ca61592e848ea9b0fb81a2ce33cdc with peer cb82cc7bd6db46d980c75237a1402860
I20260709 13:19:04.650170 17194 tablet_copy_client.cc:806] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860: tablet copy: Starting download of 0 data blocks...
I20260709 13:19:04.651118 17194 tablet_copy_client.cc:670] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860: tablet copy: Starting download of 1 WAL segments...
I20260709 13:19:04.656961 17194 tablet_copy_client.cc:538] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:19:04.664754 17194 tablet_bootstrap.cc:492] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860: Bootstrap starting.
I20260709 13:19:04.706120 17194 tablet_bootstrap.cc:492] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:19:04.707273 17194 tablet_bootstrap.cc:492] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860: Bootstrap complete.
I20260709 13:19:04.716457 17194 ts_tablet_manager.cc:1403] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860: Time spent bootstrapping tablet: real 0.052s	user 0.027s	sys 0.000s
I20260709 13:19:04.720082 17194 raft_consensus.cc:359] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } }
I20260709 13:19:04.721587 17194 raft_consensus.cc:740] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Initialized, Role: LEARNER
I20260709 13:19:04.722208 17194 consensus_queue.cc:260] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } }
I20260709 13:19:04.734390 17194 ts_tablet_manager.cc:1434] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860: Time spent starting tablet: real 0.017s	user 0.008s	sys 0.000s
I20260709 13:19:04.739387 16310 tablet_copy_service.cc:342] P 90f3da57796b488c9087a2f523adf045: Request end of tablet copy session cb82cc7bd6db46d980c75237a1402860-4e9b2bf7011f4ae688a79d7bb5a447b4 received from {username='slave'} at 127.0.0.1:40474
I20260709 13:19:04.739878 16310 tablet_copy_service.cc:434] P 90f3da57796b488c9087a2f523adf045: ending tablet copy session cb82cc7bd6db46d980c75237a1402860-4e9b2bf7011f4ae688a79d7bb5a447b4 on tablet 4e9b2bf7011f4ae688a79d7bb5a447b4 with peer cb82cc7bd6db46d980c75237a1402860
I20260709 13:19:04.775529 16823 raft_consensus.cc:1217] T 3884f864a96d428785895e5463e5728f P cb82cc7bd6db46d980c75237a1402860 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:19:04.800055 16823 raft_consensus.cc:1217] T 3a7ca61592e848ea9b0fb81a2ce33cdc P cb82cc7bd6db46d980c75237a1402860 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:19:04.896430 16824 raft_consensus.cc:1217] T 8638ee363273426db1393dd81e5c7b72 P cb82cc7bd6db46d980c75237a1402860 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:19:04.927793 16824 raft_consensus.cc:1217] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:19:05.290737 17152 raft_consensus.cc:1064] T 3884f864a96d428785895e5463e5728f P 647a343a542d4a139fe106c19095129b: attempting to promote NON_VOTER cb82cc7bd6db46d980c75237a1402860 to VOTER
I20260709 13:19:05.293392 17152 consensus_queue.cc:237] T 3884f864a96d428785895e5463e5728f P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:05.301244 16824 raft_consensus.cc:1275] T 3884f864a96d428785895e5463e5728f P cb82cc7bd6db46d980c75237a1402860 [term 1 LEARNER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:19:05.303792 16296 raft_consensus.cc:1275] T 3884f864a96d428785895e5463e5728f P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:19:05.303860 16222 raft_consensus.cc:1275] T 3884f864a96d428785895e5463e5728f P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:19:05.304864 17152 consensus_queue.cc:1048] T 3884f864a96d428785895e5463e5728f P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:19:05.305850 17208 consensus_queue.cc:1048] T 3884f864a96d428785895e5463e5728f P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:19:05.316838 17208 consensus_queue.cc:1048] T 3884f864a96d428785895e5463e5728f P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:19:05.328609 17176 raft_consensus.cc:1064] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 1ca2725d290e47d68b1a4b609c9816ac: attempting to promote NON_VOTER cb82cc7bd6db46d980c75237a1402860 to VOTER
I20260709 13:19:05.331743 17176 consensus_queue.cc:237] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:05.334419 17152 raft_consensus.cc:2955] T 3884f864a96d428785895e5463e5728f P 647a343a542d4a139fe106c19095129b [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:19:05.342397 17210 raft_consensus.cc:1064] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac: attempting to promote NON_VOTER cb82cc7bd6db46d980c75237a1402860 to VOTER
I20260709 13:19:05.343583 16298 raft_consensus.cc:2955] T 3884f864a96d428785895e5463e5728f P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:19:05.345276 17210 consensus_queue.cc:237] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:05.347700 16824 raft_consensus.cc:2955] T 3884f864a96d428785895e5463e5728f P cb82cc7bd6db46d980c75237a1402860 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:19:05.349578 16222 raft_consensus.cc:2955] T 3884f864a96d428785895e5463e5728f P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:19:05.388834 16298 raft_consensus.cc:1275] T 8638ee363273426db1393dd81e5c7b72 P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:19:05.390550 17210 consensus_queue.cc:1048] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:19:05.391877 16823 raft_consensus.cc:1275] T 8638ee363273426db1393dd81e5c7b72 P cb82cc7bd6db46d980c75237a1402860 [term 1 LEARNER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:19:05.376652 16045 catalog_manager.cc:5697] T 3884f864a96d428785895e5463e5728f P cb82cc7bd6db46d980c75237a1402860 reported cstate change: config changed from index 2 to 3, cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "647a343a542d4a139fe106c19095129b" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:19:05.402212 16144 raft_consensus.cc:1275] T 8638ee363273426db1393dd81e5c7b72 P 647a343a542d4a139fe106c19095129b [term 1 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:19:05.404359 17173 consensus_queue.cc:1048] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:19:05.405160 16823 raft_consensus.cc:1275] T 3a7ca61592e848ea9b0fb81a2ce33cdc P cb82cc7bd6db46d980c75237a1402860 [term 1 LEARNER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:19:05.406172 16299 raft_consensus.cc:1275] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:19:05.406558 17173 consensus_queue.cc:1048] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:19:05.408366 17207 consensus_queue.cc:1048] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:19:05.411445 17207 consensus_queue.cc:1048] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:19:05.426709 16150 raft_consensus.cc:1275] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 647a343a542d4a139fe106c19095129b [term 1 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:19:05.428539 17210 consensus_queue.cc:1048] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:19:05.443593 17207 raft_consensus.cc:2955] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:19:05.445679 16298 raft_consensus.cc:2955] T 8638ee363273426db1393dd81e5c7b72 P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:19:05.456521 17176 raft_consensus.cc:2955] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 1ca2725d290e47d68b1a4b609c9816ac [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:19:05.462533 16821 raft_consensus.cc:2955] T 8638ee363273426db1393dd81e5c7b72 P cb82cc7bd6db46d980c75237a1402860 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:19:05.464152 16296 raft_consensus.cc:2955] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:19:05.471746 16144 raft_consensus.cc:2955] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 647a343a542d4a139fe106c19095129b [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:19:05.473377 16824 raft_consensus.cc:2955] T 3a7ca61592e848ea9b0fb81a2ce33cdc P cb82cc7bd6db46d980c75237a1402860 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:19:05.508639 16150 raft_consensus.cc:2955] T 8638ee363273426db1393dd81e5c7b72 P 647a343a542d4a139fe106c19095129b [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:19:05.518183 16144 consensus_queue.cc:237] T 3884f864a96d428785895e5463e5728f P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:05.523418 16821 raft_consensus.cc:1275] T 3884f864a96d428785895e5463e5728f P cb82cc7bd6db46d980c75237a1402860 [term 1 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:19:05.526155 17152 consensus_queue.cc:1048] T 3884f864a96d428785895e5463e5728f P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:19:05.545444 17206 raft_consensus.cc:2955] T 3884f864a96d428785895e5463e5728f P 647a343a542d4a139fe106c19095129b [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:19:05.548491 16296 raft_consensus.cc:1275] T 3884f864a96d428785895e5463e5728f P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:19:05.550146 17216 consensus_queue.cc:1048] T 3884f864a96d428785895e5463e5728f P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:19:05.551988 17149 raft_consensus.cc:1064] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 90f3da57796b488c9087a2f523adf045: attempting to promote NON_VOTER cb82cc7bd6db46d980c75237a1402860 to VOTER
I20260709 13:19:05.554457 17149 consensus_queue.cc:237] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:05.562201 16821 raft_consensus.cc:2955] T 3884f864a96d428785895e5463e5728f P cb82cc7bd6db46d980c75237a1402860 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:19:05.565037 16298 raft_consensus.cc:2955] T 3884f864a96d428785895e5463e5728f P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:19:05.567881 16035 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 3884f864a96d428785895e5463e5728f with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:19:05.562049 16045 catalog_manager.cc:5697] T 3a7ca61592e848ea9b0fb81a2ce33cdc P cb82cc7bd6db46d980c75237a1402860 reported cstate change: config changed from index 2 to 3, cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "1ca2725d290e47d68b1a4b609c9816ac" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:19:05.572965 16045 catalog_manager.cc:5697] T 8638ee363273426db1393dd81e5c7b72 P cb82cc7bd6db46d980c75237a1402860 reported cstate change: config changed from index 2 to 3, cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "1ca2725d290e47d68b1a4b609c9816ac" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:19:05.589790 16821 raft_consensus.cc:1275] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860 [term 1 LEARNER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:19:05.592232 16222 raft_consensus.cc:1275] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:19:05.592721 16144 raft_consensus.cc:1275] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 647a343a542d4a139fe106c19095129b [term 1 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:19:05.594806 17149 consensus_queue.cc:1048] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:19:05.600206 17191 consensus_queue.cc:1048] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:19:05.601953 17191 consensus_queue.cc:1048] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:19:05.611680 16223 consensus_queue.cc:237] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:05.606371 16047 catalog_manager.cc:5697] T 3884f864a96d428785895e5463e5728f P cb82cc7bd6db46d980c75237a1402860 reported cstate change: config changed from index 3 to 4, VOTER 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130) evicted. New cstate: current_term: 1 leader_uuid: "647a343a542d4a139fe106c19095129b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:19:05.616367 16222 consensus_queue.cc:237] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:05.624040 16823 raft_consensus.cc:1275] T 8638ee363273426db1393dd81e5c7b72 P cb82cc7bd6db46d980c75237a1402860 [term 1 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:19:05.626906 17210 consensus_queue.cc:1048] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:19:05.633350 16298 raft_consensus.cc:1275] T 8638ee363273426db1393dd81e5c7b72 P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:19:05.634723 16299 raft_consensus.cc:1275] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:19:05.635848 17176 consensus_queue.cc:1048] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:19:05.636420 17173 consensus_queue.cc:1048] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:19:05.646095 17191 raft_consensus.cc:2955] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 90f3da57796b488c9087a2f523adf045 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:19:05.649051 16823 raft_consensus.cc:1275] T 3a7ca61592e848ea9b0fb81a2ce33cdc P cb82cc7bd6db46d980c75237a1402860 [term 1 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:19:05.651122 17207 raft_consensus.cc:2955] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 647a343a542d4a139fe106c19095129b (127.14.177.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:19:05.656255 16823 raft_consensus.cc:2955] T 8638ee363273426db1393dd81e5c7b72 P cb82cc7bd6db46d980c75237a1402860 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 647a343a542d4a139fe106c19095129b (127.14.177.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:19:05.659340 17176 raft_consensus.cc:2955] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 1ca2725d290e47d68b1a4b609c9816ac [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 647a343a542d4a139fe106c19095129b (127.14.177.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:19:05.661278 16035 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 8638ee363273426db1393dd81e5c7b72 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:19:05.661459 17207 consensus_queue.cc:1048] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:19:05.664486 16299 raft_consensus.cc:2955] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 647a343a542d4a139fe106c19095129b (127.14.177.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:19:05.667426 16824 raft_consensus.cc:2955] T 3a7ca61592e848ea9b0fb81a2ce33cdc P cb82cc7bd6db46d980c75237a1402860 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 647a343a542d4a139fe106c19095129b (127.14.177.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:19:05.673874 16296 raft_consensus.cc:2955] T 8638ee363273426db1393dd81e5c7b72 P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 647a343a542d4a139fe106c19095129b (127.14.177.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:19:05.675909 16144 raft_consensus.cc:2955] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 647a343a542d4a139fe106c19095129b [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:19:05.682801 16048 catalog_manager.cc:5697] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac reported cstate change: config changed from index 3 to 4, VOTER 647a343a542d4a139fe106c19095129b (127.14.177.129) evicted. New cstate: current_term: 1 leader_uuid: "1ca2725d290e47d68b1a4b609c9816ac" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:19:05.692556 16223 raft_consensus.cc:2955] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:19:05.706444 16136 tablet_service.cc:1558] Processing DeleteTablet for tablet 8638ee363273426db1393dd81e5c7b72 with delete_type TABLET_DATA_TOMBSTONED (Replica with old config index 3 (current committed config index is 4)) from {username='slave'} at 127.0.0.1:40822
I20260709 13:19:05.706169 16824 raft_consensus.cc:2955] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:19:05.712469 16035 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 3a7ca61592e848ea9b0fb81a2ce33cdc with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:19:05.714607 17235 tablet_replica.cc:333] T 8638ee363273426db1393dd81e5c7b72 P 647a343a542d4a139fe106c19095129b: stopping tablet replica
I20260709 13:19:05.715868 17235 raft_consensus.cc:2243] T 8638ee363273426db1393dd81e5c7b72 P 647a343a542d4a139fe106c19095129b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:19:05.716316 16209 tablet_service.cc:1558] Processing DeleteTablet for tablet 3884f864a96d428785895e5463e5728f with delete_type TABLET_DATA_TOMBSTONED (TS 1ca2725d290e47d68b1a4b609c9816ac not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:57598
I20260709 13:19:05.716760 17235 raft_consensus.cc:2272] T 8638ee363273426db1393dd81e5c7b72 P 647a343a542d4a139fe106c19095129b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:19:05.729383 16047 catalog_manager.cc:5697] T 3a7ca61592e848ea9b0fb81a2ce33cdc P cb82cc7bd6db46d980c75237a1402860 reported cstate change: config changed from index 3 to 4, VOTER 647a343a542d4a139fe106c19095129b (127.14.177.129) evicted. New cstate: current_term: 1 leader_uuid: "1ca2725d290e47d68b1a4b609c9816ac" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:19:05.742758 17235 ts_tablet_manager.cc:1916] T 8638ee363273426db1393dd81e5c7b72 P 647a343a542d4a139fe106c19095129b: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:19:05.744588 17236 tablet_replica.cc:333] T 3884f864a96d428785895e5463e5728f P 1ca2725d290e47d68b1a4b609c9816ac: stopping tablet replica
I20260709 13:19:05.745609 16136 tablet_service.cc:1558] Processing DeleteTablet for tablet 3a7ca61592e848ea9b0fb81a2ce33cdc with delete_type TABLET_DATA_TOMBSTONED (TS 647a343a542d4a139fe106c19095129b not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:40822
I20260709 13:19:05.745905 17236 raft_consensus.cc:2243] T 3884f864a96d428785895e5463e5728f P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:19:05.746938 16048 catalog_manager.cc:5697] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 647a343a542d4a139fe106c19095129b reported cstate change: config changed from index 2 to 3, cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "90f3da57796b488c9087a2f523adf045" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:19:05.748478 17236 raft_consensus.cc:2272] T 3884f864a96d428785895e5463e5728f P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:19:05.756475 17236 ts_tablet_manager.cc:1916] T 3884f864a96d428785895e5463e5728f P 1ca2725d290e47d68b1a4b609c9816ac: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:19:05.767056 17235 ts_tablet_manager.cc:1929] T 8638ee363273426db1393dd81e5c7b72 P 647a343a542d4a139fe106c19095129b: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:19:05.767684 17235 log.cc:1199] T 8638ee363273426db1393dd81e5c7b72 P 647a343a542d4a139fe106c19095129b: Deleting WAL directory at /tmp/dist-test-taskdH_Ma7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783603093801613-15046-0/minicluster-data/ts-0-root/wals/8638ee363273426db1393dd81e5c7b72
I20260709 13:19:05.769475 17235 tablet_replica.cc:333] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 647a343a542d4a139fe106c19095129b: stopping tablet replica
I20260709 13:19:05.770155 16035 catalog_manager.cc:5028] TS 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): tablet 8638ee363273426db1393dd81e5c7b72 (table test-workload [id=ed3d15d8c9ad46bebc4fe3a402d2e54f]) successfully deleted
I20260709 13:19:05.770473 17235 raft_consensus.cc:2243] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 647a343a542d4a139fe106c19095129b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:19:05.771320 17235 raft_consensus.cc:2272] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 647a343a542d4a139fe106c19095129b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:19:05.772222 17236 ts_tablet_manager.cc:1929] T 3884f864a96d428785895e5463e5728f P 1ca2725d290e47d68b1a4b609c9816ac: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:19:05.772730 17236 log.cc:1199] T 3884f864a96d428785895e5463e5728f P 1ca2725d290e47d68b1a4b609c9816ac: Deleting WAL directory at /tmp/dist-test-taskdH_Ma7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783603093801613-15046-0/minicluster-data/ts-1-root/wals/3884f864a96d428785895e5463e5728f
I20260709 13:19:05.776257 17235 ts_tablet_manager.cc:1916] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 647a343a542d4a139fe106c19095129b: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:19:05.774696 16035 catalog_manager.cc:5028] TS 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943): tablet 3884f864a96d428785895e5463e5728f (table test-workload [id=ed3d15d8c9ad46bebc4fe3a402d2e54f]) successfully deleted
I20260709 13:19:05.785122 16299 consensus_queue.cc:237] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:05.792238 16824 raft_consensus.cc:1275] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860 [term 1 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:19:05.798077 17235 ts_tablet_manager.cc:1929] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 647a343a542d4a139fe106c19095129b: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:19:05.798590 17235 log.cc:1199] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 647a343a542d4a139fe106c19095129b: Deleting WAL directory at /tmp/dist-test-taskdH_Ma7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783603093801613-15046-0/minicluster-data/ts-0-root/wals/3a7ca61592e848ea9b0fb81a2ce33cdc
I20260709 13:19:05.798920 17191 consensus_queue.cc:1048] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:19:05.800105 16035 catalog_manager.cc:5028] TS 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): tablet 3a7ca61592e848ea9b0fb81a2ce33cdc (table test-workload [id=ed3d15d8c9ad46bebc4fe3a402d2e54f]) successfully deleted
I20260709 13:19:05.804425 16144 raft_consensus.cc:1275] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 647a343a542d4a139fe106c19095129b [term 1 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:19:05.805982 17191 consensus_queue.cc:1048] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:19:05.809610 17149 raft_consensus.cc:2955] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 90f3da57796b488c9087a2f523adf045 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:19:05.812482 16823 raft_consensus.cc:2955] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:19:05.824028 16150 raft_consensus.cc:2955] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 647a343a542d4a139fe106c19095129b [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:19:05.829362 16045 catalog_manager.cc:5697] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860 reported cstate change: config changed from index 3 to 4, VOTER 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130) evicted. New cstate: current_term: 1 leader_uuid: "90f3da57796b488c9087a2f523adf045" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:19:05.841385 16034 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 4e9b2bf7011f4ae688a79d7bb5a447b4 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:19:05.845038 16209 tablet_service.cc:1558] Processing DeleteTablet for tablet 4e9b2bf7011f4ae688a79d7bb5a447b4 with delete_type TABLET_DATA_TOMBSTONED (TS 1ca2725d290e47d68b1a4b609c9816ac not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:57598
I20260709 13:19:05.846134 17236 tablet_replica.cc:333] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 1ca2725d290e47d68b1a4b609c9816ac: stopping tablet replica
I20260709 13:19:05.852943 17236 raft_consensus.cc:2243] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:19:05.853632 17236 raft_consensus.cc:2272] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:19:05.857136 17236 ts_tablet_manager.cc:1916] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 1ca2725d290e47d68b1a4b609c9816ac: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:19:05.875386 17236 ts_tablet_manager.cc:1929] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 1ca2725d290e47d68b1a4b609c9816ac: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:19:05.875746 17236 log.cc:1199] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 1ca2725d290e47d68b1a4b609c9816ac: Deleting WAL directory at /tmp/dist-test-taskdH_Ma7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783603093801613-15046-0/minicluster-data/ts-1-root/wals/4e9b2bf7011f4ae688a79d7bb5a447b4
I20260709 13:19:05.876884 16035 catalog_manager.cc:5028] TS 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943): tablet 4e9b2bf7011f4ae688a79d7bb5a447b4 (table test-workload [id=ed3d15d8c9ad46bebc4fe3a402d2e54f]) successfully deleted
I20260709 13:19:06.925195 16150 consensus_queue.cc:237] T 746aabf856ab401c9ef5e859371e7b67 P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } }
I20260709 13:19:06.931661 16300 raft_consensus.cc:1275] T 746aabf856ab401c9ef5e859371e7b67 P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 13:19:06.932047 16117 consensus_peers.cc:597] T 746aabf856ab401c9ef5e859371e7b67 P 647a343a542d4a139fe106c19095129b -> Peer cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923): Couldn't send request to peer cb82cc7bd6db46d980c75237a1402860. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 746aabf856ab401c9ef5e859371e7b67. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:19:06.932266 16222 raft_consensus.cc:1275] T 746aabf856ab401c9ef5e859371e7b67 P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:19:06.933033 17237 consensus_queue.cc:1048] T 746aabf856ab401c9ef5e859371e7b67 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:19:06.933976 17239 consensus_queue.cc:1048] T 746aabf856ab401c9ef5e859371e7b67 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:19:06.944427 16222 raft_consensus.cc:2955] T 746aabf856ab401c9ef5e859371e7b67 P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } } }
I20260709 13:19:06.944461 17239 raft_consensus.cc:2955] T 746aabf856ab401c9ef5e859371e7b67 P 647a343a542d4a139fe106c19095129b [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } } }
I20260709 13:19:06.945936 16300 raft_consensus.cc:2955] T 746aabf856ab401c9ef5e859371e7b67 P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } } }
I20260709 13:19:06.959107 16046 catalog_manager.cc:5697] T 746aabf856ab401c9ef5e859371e7b67 P 1ca2725d290e47d68b1a4b609c9816ac reported cstate change: config changed from index -1 to 2, NON_VOTER cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) added. New cstate: current_term: 1 leader_uuid: "647a343a542d4a139fe106c19095129b" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } } }
I20260709 13:19:06.972826 16150 consensus_queue.cc:237] T 57a744f5aaf944678a63469cbf514015 P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } }
W20260709 13:19:06.981319 16117 consensus_peers.cc:597] T 57a744f5aaf944678a63469cbf514015 P 647a343a542d4a139fe106c19095129b -> Peer cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923): Couldn't send request to peer cb82cc7bd6db46d980c75237a1402860. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 57a744f5aaf944678a63469cbf514015. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:19:06.982486 16222 raft_consensus.cc:1275] T 57a744f5aaf944678a63469cbf514015 P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:19:06.983835 17237 consensus_queue.cc:1048] T 57a744f5aaf944678a63469cbf514015 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:19:06.985656 16300 raft_consensus.cc:1275] T 57a744f5aaf944678a63469cbf514015 P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:19:06.986889 17237 consensus_queue.cc:1048] T 57a744f5aaf944678a63469cbf514015 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:19:06.995378 17216 raft_consensus.cc:2955] T 57a744f5aaf944678a63469cbf514015 P 647a343a542d4a139fe106c19095129b [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } } }
I20260709 13:19:06.999567 16300 raft_consensus.cc:2955] T 57a744f5aaf944678a63469cbf514015 P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } } }
I20260709 13:19:07.000020 16223 raft_consensus.cc:2955] T 57a744f5aaf944678a63469cbf514015 P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } } }
I20260709 13:19:07.015453 16223 consensus_queue.cc:237] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } }
I20260709 13:19:07.016158 16048 catalog_manager.cc:5697] T 57a744f5aaf944678a63469cbf514015 P 90f3da57796b488c9087a2f523adf045 reported cstate change: config changed from index -1 to 2, NON_VOTER cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) added. New cstate: current_term: 1 leader_uuid: "647a343a542d4a139fe106c19095129b" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } } }
I20260709 13:19:07.022531 16300 raft_consensus.cc:1275] T 66ae176961b74813a58bc15c9c4c6f2a P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:19:07.023231 16150 raft_consensus.cc:1275] T 66ae176961b74813a58bc15c9c4c6f2a P 647a343a542d4a139fe106c19095129b [term 1 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:19:07.024151 17207 consensus_queue.cc:1048] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:19:07.024953 17176 consensus_queue.cc:1048] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:19:07.028586 16192 consensus_peers.cc:597] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac -> Peer cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923): Couldn't send request to peer cb82cc7bd6db46d980c75237a1402860. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 66ae176961b74813a58bc15c9c4c6f2a. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:19:07.035034 17238 raft_consensus.cc:2955] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } } }
I20260709 13:19:07.038928 16300 raft_consensus.cc:2955] T 66ae176961b74813a58bc15c9c4c6f2a P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } } }
I20260709 13:19:07.041241 16144 raft_consensus.cc:2955] T 66ae176961b74813a58bc15c9c4c6f2a P 647a343a542d4a139fe106c19095129b [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } } }
I20260709 13:19:07.050716 16150 consensus_queue.cc:237] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } }
I20260709 13:19:07.053215 16045 catalog_manager.cc:5697] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac reported cstate change: config changed from index -1 to 2, NON_VOTER cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) added. New cstate: current_term: 1 leader_uuid: "1ca2725d290e47d68b1a4b609c9816ac" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
W20260709 13:19:07.062983 16117 consensus_peers.cc:597] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b -> Peer cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923): Couldn't send request to peer cb82cc7bd6db46d980c75237a1402860. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 5718c623ebee4d2bb770f0f9ddb2f164. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:19:07.064747 16300 raft_consensus.cc:1275] T 5718c623ebee4d2bb770f0f9ddb2f164 P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:19:07.065505 16223 raft_consensus.cc:1275] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:19:07.066701 17239 consensus_queue.cc:1048] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:19:07.067620 17216 consensus_queue.cc:1048] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:19:07.085952 17216 raft_consensus.cc:2955] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } } }
I20260709 13:19:07.087736 16223 raft_consensus.cc:2955] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } } }
I20260709 13:19:07.101367 16300 raft_consensus.cc:2955] T 5718c623ebee4d2bb770f0f9ddb2f164 P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } } }
I20260709 13:19:07.115502 16048 catalog_manager.cc:5697] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac reported cstate change: config changed from index -1 to 2, NON_VOTER cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) added. New cstate: current_term: 1 leader_uuid: "647a343a542d4a139fe106c19095129b" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } } }
I20260709 13:19:07.401974 17259 ts_tablet_manager.cc:933] T 57a744f5aaf944678a63469cbf514015 P cb82cc7bd6db46d980c75237a1402860: Initiating tablet copy from peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741)
I20260709 13:19:07.405589 17259 tablet_copy_client.cc:323] T 57a744f5aaf944678a63469cbf514015 P cb82cc7bd6db46d980c75237a1402860: tablet copy: Beginning tablet copy session from remote peer at address 127.14.177.129:36741
I20260709 13:19:07.407783 16161 tablet_copy_service.cc:140] P 647a343a542d4a139fe106c19095129b: Received BeginTabletCopySession request for tablet 57a744f5aaf944678a63469cbf514015 from peer cb82cc7bd6db46d980c75237a1402860 ({username='slave'} at 127.0.0.1:46224)
I20260709 13:19:07.408402 16161 tablet_copy_service.cc:161] P 647a343a542d4a139fe106c19095129b: Beginning new tablet copy session on tablet 57a744f5aaf944678a63469cbf514015 from peer cb82cc7bd6db46d980c75237a1402860 at {username='slave'} at 127.0.0.1:46224: session id = cb82cc7bd6db46d980c75237a1402860-57a744f5aaf944678a63469cbf514015
I20260709 13:19:07.422395 16161 tablet_copy_source_session.cc:215] T 57a744f5aaf944678a63469cbf514015 P 647a343a542d4a139fe106c19095129b: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:19:07.424652 17261 ts_tablet_manager.cc:933] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860: Initiating tablet copy from peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:19:07.435912 17259 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 57a744f5aaf944678a63469cbf514015. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:19:07.466975 17261 tablet_copy_client.cc:323] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860: tablet copy: Beginning tablet copy session from remote peer at address 127.14.177.130:35943
I20260709 13:19:07.469000 17262 ts_tablet_manager.cc:933] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860: Initiating tablet copy from peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741)
I20260709 13:19:07.469437 16236 tablet_copy_service.cc:140] P 1ca2725d290e47d68b1a4b609c9816ac: Received BeginTabletCopySession request for tablet 66ae176961b74813a58bc15c9c4c6f2a from peer cb82cc7bd6db46d980c75237a1402860 ({username='slave'} at 127.0.0.1:33050)
I20260709 13:19:07.470095 16236 tablet_copy_service.cc:161] P 1ca2725d290e47d68b1a4b609c9816ac: Beginning new tablet copy session on tablet 66ae176961b74813a58bc15c9c4c6f2a from peer cb82cc7bd6db46d980c75237a1402860 at {username='slave'} at 127.0.0.1:33050: session id = cb82cc7bd6db46d980c75237a1402860-66ae176961b74813a58bc15c9c4c6f2a
I20260709 13:19:07.471395 17262 tablet_copy_client.cc:323] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860: tablet copy: Beginning tablet copy session from remote peer at address 127.14.177.129:36741
I20260709 13:19:07.477396 16161 tablet_copy_service.cc:140] P 647a343a542d4a139fe106c19095129b: Received BeginTabletCopySession request for tablet 746aabf856ab401c9ef5e859371e7b67 from peer cb82cc7bd6db46d980c75237a1402860 ({username='slave'} at 127.0.0.1:46224)
I20260709 13:19:07.478744 16161 tablet_copy_service.cc:161] P 647a343a542d4a139fe106c19095129b: Beginning new tablet copy session on tablet 746aabf856ab401c9ef5e859371e7b67 from peer cb82cc7bd6db46d980c75237a1402860 at {username='slave'} at 127.0.0.1:46224: session id = cb82cc7bd6db46d980c75237a1402860-746aabf856ab401c9ef5e859371e7b67
I20260709 13:19:07.488263 17259 tablet_copy_client.cc:806] T 57a744f5aaf944678a63469cbf514015 P cb82cc7bd6db46d980c75237a1402860: tablet copy: Starting download of 0 data blocks...
I20260709 13:19:07.488903 17259 tablet_copy_client.cc:670] T 57a744f5aaf944678a63469cbf514015 P cb82cc7bd6db46d980c75237a1402860: tablet copy: Starting download of 1 WAL segments...
I20260709 13:19:07.490391 16236 tablet_copy_source_session.cc:215] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:19:07.492309 16161 tablet_copy_source_session.cc:215] T 746aabf856ab401c9ef5e859371e7b67 P 647a343a542d4a139fe106c19095129b: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:19:07.495252 17261 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 66ae176961b74813a58bc15c9c4c6f2a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:19:07.496166 17262 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 746aabf856ab401c9ef5e859371e7b67. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:19:07.493468 17259 tablet_copy_client.cc:538] T 57a744f5aaf944678a63469cbf514015 P cb82cc7bd6db46d980c75237a1402860: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:19:07.505638 17259 tablet_bootstrap.cc:492] T 57a744f5aaf944678a63469cbf514015 P cb82cc7bd6db46d980c75237a1402860: Bootstrap starting.
I20260709 13:19:07.513612 17261 tablet_copy_client.cc:806] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860: tablet copy: Starting download of 0 data blocks...
I20260709 13:19:07.516203 17261 tablet_copy_client.cc:670] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860: tablet copy: Starting download of 1 WAL segments...
I20260709 13:19:07.522373 17261 tablet_copy_client.cc:538] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:19:07.521974 17262 tablet_copy_client.cc:806] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860: tablet copy: Starting download of 0 data blocks...
I20260709 13:19:07.528079 17262 tablet_copy_client.cc:670] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860: tablet copy: Starting download of 1 WAL segments...
I20260709 13:19:07.535809 17261 tablet_bootstrap.cc:492] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860: Bootstrap starting.
I20260709 13:19:07.541913 17262 tablet_copy_client.cc:538] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:19:07.558211 17262 tablet_bootstrap.cc:492] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860: Bootstrap starting.
I20260709 13:19:07.558209 17259 tablet_bootstrap.cc:492] T 57a744f5aaf944678a63469cbf514015 P cb82cc7bd6db46d980c75237a1402860: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:19:07.559499 17259 tablet_bootstrap.cc:492] T 57a744f5aaf944678a63469cbf514015 P cb82cc7bd6db46d980c75237a1402860: Bootstrap complete.
I20260709 13:19:07.560226 17259 ts_tablet_manager.cc:1403] T 57a744f5aaf944678a63469cbf514015 P cb82cc7bd6db46d980c75237a1402860: Time spent bootstrapping tablet: real 0.055s	user 0.029s	sys 0.012s
I20260709 13:19:07.563390 17259 raft_consensus.cc:359] T 57a744f5aaf944678a63469cbf514015 P cb82cc7bd6db46d980c75237a1402860 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } }
I20260709 13:19:07.564320 17259 raft_consensus.cc:740] T 57a744f5aaf944678a63469cbf514015 P cb82cc7bd6db46d980c75237a1402860 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Initialized, Role: LEARNER
I20260709 13:19:07.564976 17259 consensus_queue.cc:260] T 57a744f5aaf944678a63469cbf514015 P cb82cc7bd6db46d980c75237a1402860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } }
I20260709 13:19:07.576829 17261 tablet_bootstrap.cc:492] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:19:07.577870 17261 tablet_bootstrap.cc:492] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860: Bootstrap complete.
I20260709 13:19:07.578575 17261 ts_tablet_manager.cc:1403] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860: Time spent bootstrapping tablet: real 0.043s	user 0.015s	sys 0.012s
I20260709 13:19:07.581976 17261 raft_consensus.cc:359] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } }
I20260709 13:19:07.584908 17261 raft_consensus.cc:740] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Initialized, Role: LEARNER
I20260709 13:19:07.585562 17261 consensus_queue.cc:260] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } }
I20260709 13:19:07.587546 17259 ts_tablet_manager.cc:1434] T 57a744f5aaf944678a63469cbf514015 P cb82cc7bd6db46d980c75237a1402860: Time spent starting tablet: real 0.027s	user 0.011s	sys 0.009s
I20260709 13:19:07.590202 17261 ts_tablet_manager.cc:1434] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860: Time spent starting tablet: real 0.011s	user 0.007s	sys 0.003s
I20260709 13:19:07.599030 16161 tablet_copy_service.cc:342] P 647a343a542d4a139fe106c19095129b: Request end of tablet copy session cb82cc7bd6db46d980c75237a1402860-57a744f5aaf944678a63469cbf514015 received from {username='slave'} at 127.0.0.1:46224
I20260709 13:19:07.602407 16161 tablet_copy_service.cc:434] P 647a343a542d4a139fe106c19095129b: ending tablet copy session cb82cc7bd6db46d980c75237a1402860-57a744f5aaf944678a63469cbf514015 on tablet 57a744f5aaf944678a63469cbf514015 with peer cb82cc7bd6db46d980c75237a1402860
I20260709 13:19:07.604799 17262 tablet_bootstrap.cc:492] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:19:07.605830 17262 tablet_bootstrap.cc:492] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860: Bootstrap complete.
I20260709 13:19:07.607435 17262 ts_tablet_manager.cc:1403] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860: Time spent bootstrapping tablet: real 0.049s	user 0.022s	sys 0.007s
I20260709 13:19:07.682729 17262 raft_consensus.cc:359] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } }
I20260709 13:19:07.683727 17262 raft_consensus.cc:740] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Initialized, Role: LEARNER
I20260709 13:19:07.684706 17269 ts_tablet_manager.cc:933] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860: Initiating tablet copy from peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741)
I20260709 13:19:07.684384 17262 consensus_queue.cc:260] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } }
I20260709 13:19:07.687654 16236 tablet_copy_service.cc:342] P 1ca2725d290e47d68b1a4b609c9816ac: Request end of tablet copy session cb82cc7bd6db46d980c75237a1402860-66ae176961b74813a58bc15c9c4c6f2a received from {username='slave'} at 127.0.0.1:33050
I20260709 13:19:07.690476 16236 tablet_copy_service.cc:434] P 1ca2725d290e47d68b1a4b609c9816ac: ending tablet copy session cb82cc7bd6db46d980c75237a1402860-66ae176961b74813a58bc15c9c4c6f2a on tablet 66ae176961b74813a58bc15c9c4c6f2a with peer cb82cc7bd6db46d980c75237a1402860
I20260709 13:19:07.694798 17269 tablet_copy_client.cc:323] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860: tablet copy: Beginning tablet copy session from remote peer at address 127.14.177.129:36741
I20260709 13:19:07.693614 17262 ts_tablet_manager.cc:1434] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860: Time spent starting tablet: real 0.086s	user 0.006s	sys 0.011s
I20260709 13:19:07.720341 16161 tablet_copy_service.cc:140] P 647a343a542d4a139fe106c19095129b: Received BeginTabletCopySession request for tablet 5718c623ebee4d2bb770f0f9ddb2f164 from peer cb82cc7bd6db46d980c75237a1402860 ({username='slave'} at 127.0.0.1:46224)
I20260709 13:19:07.720968 16161 tablet_copy_service.cc:161] P 647a343a542d4a139fe106c19095129b: Beginning new tablet copy session on tablet 5718c623ebee4d2bb770f0f9ddb2f164 from peer cb82cc7bd6db46d980c75237a1402860 at {username='slave'} at 127.0.0.1:46224: session id = cb82cc7bd6db46d980c75237a1402860-5718c623ebee4d2bb770f0f9ddb2f164
I20260709 13:19:07.722497 16160 tablet_copy_service.cc:342] P 647a343a542d4a139fe106c19095129b: Request end of tablet copy session cb82cc7bd6db46d980c75237a1402860-746aabf856ab401c9ef5e859371e7b67 received from {username='slave'} at 127.0.0.1:46224
I20260709 13:19:07.722885 16160 tablet_copy_service.cc:434] P 647a343a542d4a139fe106c19095129b: ending tablet copy session cb82cc7bd6db46d980c75237a1402860-746aabf856ab401c9ef5e859371e7b67 on tablet 746aabf856ab401c9ef5e859371e7b67 with peer cb82cc7bd6db46d980c75237a1402860
I20260709 13:19:07.731217 16161 tablet_copy_source_session.cc:215] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:19:07.735967 17269 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5718c623ebee4d2bb770f0f9ddb2f164. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:19:07.761994 17269 tablet_copy_client.cc:806] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860: tablet copy: Starting download of 0 data blocks...
I20260709 13:19:07.762643 17269 tablet_copy_client.cc:670] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860: tablet copy: Starting download of 1 WAL segments...
I20260709 13:19:07.766928 17269 tablet_copy_client.cc:538] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:19:07.777376 17269 tablet_bootstrap.cc:492] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860: Bootstrap starting.
I20260709 13:19:07.801626 16824 raft_consensus.cc:1217] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:19:07.813903 16824 raft_consensus.cc:1217] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:19:07.828560 17269 tablet_bootstrap.cc:492] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:19:07.829468 17269 tablet_bootstrap.cc:492] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860: Bootstrap complete.
I20260709 13:19:07.831045 17269 ts_tablet_manager.cc:1403] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860: Time spent bootstrapping tablet: real 0.054s	user 0.020s	sys 0.012s
I20260709 13:19:07.834085 17269 raft_consensus.cc:359] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } }
I20260709 13:19:07.834810 17269 raft_consensus.cc:740] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Initialized, Role: LEARNER
I20260709 13:19:07.835529 17269 consensus_queue.cc:260] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } }
I20260709 13:19:07.838076 17269 ts_tablet_manager.cc:1434] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860: Time spent starting tablet: real 0.007s	user 0.002s	sys 0.003s
I20260709 13:19:07.839706 16161 tablet_copy_service.cc:342] P 647a343a542d4a139fe106c19095129b: Request end of tablet copy session cb82cc7bd6db46d980c75237a1402860-5718c623ebee4d2bb770f0f9ddb2f164 received from {username='slave'} at 127.0.0.1:46224
I20260709 13:19:07.840144 16161 tablet_copy_service.cc:434] P 647a343a542d4a139fe106c19095129b: ending tablet copy session cb82cc7bd6db46d980c75237a1402860-5718c623ebee4d2bb770f0f9ddb2f164 on tablet 5718c623ebee4d2bb770f0f9ddb2f164 with peer cb82cc7bd6db46d980c75237a1402860
I20260709 13:19:07.984622 16824 raft_consensus.cc:1217] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:19:08.007704 16824 raft_consensus.cc:1217] T 57a744f5aaf944678a63469cbf514015 P cb82cc7bd6db46d980c75237a1402860 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:19:08.268657 17176 raft_consensus.cc:1064] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac: attempting to promote NON_VOTER cb82cc7bd6db46d980c75237a1402860 to VOTER
I20260709 13:19:08.270925 17176 consensus_queue.cc:237] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:08.294793 16824 raft_consensus.cc:1275] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [term 1 LEARNER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:19:08.306933 17207 consensus_queue.cc:1048] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:19:08.309998 17216 raft_consensus.cc:1064] T 746aabf856ab401c9ef5e859371e7b67 P 647a343a542d4a139fe106c19095129b: attempting to promote NON_VOTER cb82cc7bd6db46d980c75237a1402860 to VOTER
I20260709 13:19:08.330391 16299 raft_consensus.cc:1275] T 66ae176961b74813a58bc15c9c4c6f2a P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:19:08.335772 17238 consensus_queue.cc:1048] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:19:08.320492 17216 consensus_queue.cc:237] T 746aabf856ab401c9ef5e859371e7b67 P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:08.356539 16824 raft_consensus.cc:1275] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860 [term 1 LEARNER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:19:08.358485 17258 consensus_queue.cc:1048] T 746aabf856ab401c9ef5e859371e7b67 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:19:08.359375 16150 raft_consensus.cc:1275] T 66ae176961b74813a58bc15c9c4c6f2a P 647a343a542d4a139fe106c19095129b [term 1 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:19:08.360246 16299 raft_consensus.cc:1275] T 746aabf856ab401c9ef5e859371e7b67 P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:19:08.361913 17239 consensus_queue.cc:1048] T 746aabf856ab401c9ef5e859371e7b67 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:19:08.362576 17176 consensus_queue.cc:1048] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.002s
I20260709 13:19:08.374998 16222 raft_consensus.cc:1275] T 746aabf856ab401c9ef5e859371e7b67 P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:19:08.380862 17271 consensus_queue.cc:1048] T 746aabf856ab401c9ef5e859371e7b67 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:19:08.386718 17176 raft_consensus.cc:2955] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:19:08.388280 16299 raft_consensus.cc:2955] T 66ae176961b74813a58bc15c9c4c6f2a P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:19:08.401607 16824 raft_consensus.cc:2955] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:19:08.407284 16045 catalog_manager.cc:5697] T 66ae176961b74813a58bc15c9c4c6f2a P 90f3da57796b488c9087a2f523adf045 reported cstate change: config changed from index 2 to 3, cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "1ca2725d290e47d68b1a4b609c9816ac" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:19:08.411669 16150 raft_consensus.cc:2955] T 66ae176961b74813a58bc15c9c4c6f2a P 647a343a542d4a139fe106c19095129b [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:19:08.416889 17258 raft_consensus.cc:2955] T 746aabf856ab401c9ef5e859371e7b67 P 647a343a542d4a139fe106c19095129b [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:19:08.425357 16821 raft_consensus.cc:2955] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:19:08.431149 16222 raft_consensus.cc:2955] T 746aabf856ab401c9ef5e859371e7b67 P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:19:08.444787 17271 raft_consensus.cc:1064] T 57a744f5aaf944678a63469cbf514015 P 647a343a542d4a139fe106c19095129b: attempting to promote NON_VOTER cb82cc7bd6db46d980c75237a1402860 to VOTER
I20260709 13:19:08.451140 16299 raft_consensus.cc:2955] T 746aabf856ab401c9ef5e859371e7b67 P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:19:08.454823 16224 consensus_queue.cc:237] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:08.456465 17271 consensus_queue.cc:237] T 57a744f5aaf944678a63469cbf514015 P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:08.471709 17216 raft_consensus.cc:1064] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b: attempting to promote NON_VOTER cb82cc7bd6db46d980c75237a1402860 to VOTER
I20260709 13:19:08.477880 17216 consensus_queue.cc:237] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:08.462579 16824 raft_consensus.cc:1275] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [term 1 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:19:08.461479 16048 catalog_manager.cc:5697] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860 reported cstate change: config changed from index 2 to 3, cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "647a343a542d4a139fe106c19095129b" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:19:08.525492 16299 raft_consensus.cc:1275] T 66ae176961b74813a58bc15c9c4c6f2a P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
W20260709 13:19:08.527387 16115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 49232.8us
I20260709 13:19:08.548125 16299 raft_consensus.cc:1275] T 5718c623ebee4d2bb770f0f9ddb2f164 P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:19:08.550016 17239 consensus_queue.cc:1048] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:19:08.551669 16150 consensus_queue.cc:237] T 746aabf856ab401c9ef5e859371e7b67 P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:08.555459 16824 raft_consensus.cc:1275] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 [term 1 LEARNER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:19:08.557462 16824 raft_consensus.cc:1275] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860 [term 1 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:19:08.557379 17290 consensus_queue.cc:1048] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.002s
I20260709 13:19:08.562719 17293 consensus_queue.cc:1048] T 746aabf856ab401c9ef5e859371e7b67 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:19:08.566936 17207 consensus_queue.cc:1048] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.004s
I20260709 13:19:08.602147 16300 raft_consensus.cc:1275] T 57a744f5aaf944678a63469cbf514015 P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:19:08.602820 16222 raft_consensus.cc:1275] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:19:08.603354 16823 raft_consensus.cc:1275] T 57a744f5aaf944678a63469cbf514015 P cb82cc7bd6db46d980c75237a1402860 [term 1 LEARNER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:19:08.604456 16222 raft_consensus.cc:1275] T 746aabf856ab401c9ef5e859371e7b67 P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:19:08.605768 17293 consensus_queue.cc:1048] T 57a744f5aaf944678a63469cbf514015 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:19:08.605693 17258 raft_consensus.cc:2955] T 746aabf856ab401c9ef5e859371e7b67 P 647a343a542d4a139fe106c19095129b [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 90f3da57796b488c9087a2f523adf045 (127.14.177.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:19:08.610329 17239 consensus_queue.cc:1048] T 57a744f5aaf944678a63469cbf514015 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:19:08.590070 17276 consensus_queue.cc:1048] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.038s
I20260709 13:19:08.618295 16823 raft_consensus.cc:2955] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 90f3da57796b488c9087a2f523adf045 (127.14.177.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:19:08.628430 16222 raft_consensus.cc:1275] T 57a744f5aaf944678a63469cbf514015 P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:19:08.639356 17293 consensus_queue.cc:1048] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.016s
I20260709 13:19:08.657898 17216 consensus_queue.cc:1048] T 57a744f5aaf944678a63469cbf514015 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:19:08.663878 16035 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 746aabf856ab401c9ef5e859371e7b67 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:19:08.644811 17239 consensus_queue.cc:1048] T 746aabf856ab401c9ef5e859371e7b67 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.021s
I20260709 13:19:08.668150 17276 raft_consensus.cc:2955] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 647a343a542d4a139fe106c19095129b (127.14.177.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:19:08.674777 16049 catalog_manager.cc:5697] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860 reported cstate change: config changed from index 3 to 4, VOTER 90f3da57796b488c9087a2f523adf045 (127.14.177.131) evicted. New cstate: current_term: 1 leader_uuid: "647a343a542d4a139fe106c19095129b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:19:08.675541 16824 raft_consensus.cc:2955] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 647a343a542d4a139fe106c19095129b (127.14.177.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:19:08.683869 16035 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 66ae176961b74813a58bc15c9c4c6f2a with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
W20260709 13:19:08.692122 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 114461us
I20260709 13:19:08.697355 16286 tablet_service.cc:1558] Processing DeleteTablet for tablet 746aabf856ab401c9ef5e859371e7b67 with delete_type TABLET_DATA_TOMBSTONED (TS 90f3da57796b488c9087a2f523adf045 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:48670
I20260709 13:19:08.697142 16045 catalog_manager.cc:5697] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac reported cstate change: config changed from index 3 to 4, VOTER 647a343a542d4a139fe106c19095129b (127.14.177.129) evicted. New cstate: current_term: 1 leader_uuid: "1ca2725d290e47d68b1a4b609c9816ac" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:19:08.701421 16298 raft_consensus.cc:2955] T 66ae176961b74813a58bc15c9c4c6f2a P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 647a343a542d4a139fe106c19095129b (127.14.177.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
W20260709 13:19:08.713398 16117 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 74746us
I20260709 13:19:08.722213 17293 raft_consensus.cc:2955] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:19:08.724792 17306 tablet_replica.cc:333] T 746aabf856ab401c9ef5e859371e7b67 P 90f3da57796b488c9087a2f523adf045: stopping tablet replica
I20260709 13:19:08.726172 17306 raft_consensus.cc:2243] T 746aabf856ab401c9ef5e859371e7b67 P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:19:08.728178 17306 raft_consensus.cc:2272] T 746aabf856ab401c9ef5e859371e7b67 P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:19:08.730659 16824 raft_consensus.cc:2955] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:19:08.731983 17306 ts_tablet_manager.cc:1916] T 746aabf856ab401c9ef5e859371e7b67 P 90f3da57796b488c9087a2f523adf045: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:19:08.742985 16298 raft_consensus.cc:2955] T 5718c623ebee4d2bb770f0f9ddb2f164 P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
W20260709 13:19:08.749353 17304 log.cc:927] Time spent T 57a744f5aaf944678a63469cbf514015 P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.056s	user 0.001s	sys 0.000s
I20260709 13:19:08.752122 16136 tablet_service.cc:1558] Processing DeleteTablet for tablet 66ae176961b74813a58bc15c9c4c6f2a with delete_type TABLET_DATA_TOMBSTONED (TS 647a343a542d4a139fe106c19095129b not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:40822
I20260709 13:19:08.769577 16045 catalog_manager.cc:5697] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 reported cstate change: config changed from index 2 to 3, cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "647a343a542d4a139fe106c19095129b" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:19:08.787861 17303 raft_consensus.cc:2955] T 57a744f5aaf944678a63469cbf514015 P 647a343a542d4a139fe106c19095129b [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:19:08.789317 17306 ts_tablet_manager.cc:1929] T 746aabf856ab401c9ef5e859371e7b67 P 90f3da57796b488c9087a2f523adf045: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:19:08.789929 17306 log.cc:1199] T 746aabf856ab401c9ef5e859371e7b67 P 90f3da57796b488c9087a2f523adf045: Deleting WAL directory at /tmp/dist-test-taskdH_Ma7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783603093801613-15046-0/minicluster-data/ts-2-root/wals/746aabf856ab401c9ef5e859371e7b67
I20260709 13:19:08.792862 16034 catalog_manager.cc:5028] TS 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389): tablet 746aabf856ab401c9ef5e859371e7b67 (table test-workload [id=ed3d15d8c9ad46bebc4fe3a402d2e54f]) successfully deleted
I20260709 13:19:08.801407 16298 raft_consensus.cc:2955] T 57a744f5aaf944678a63469cbf514015 P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:19:08.814985 17307 tablet_replica.cc:333] T 66ae176961b74813a58bc15c9c4c6f2a P 647a343a542d4a139fe106c19095129b: stopping tablet replica
I20260709 13:19:08.774804 15046 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:19:08.822365 17307 raft_consensus.cc:2243] T 66ae176961b74813a58bc15c9c4c6f2a P 647a343a542d4a139fe106c19095129b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:19:08.823252 17307 raft_consensus.cc:2272] T 66ae176961b74813a58bc15c9c4c6f2a P 647a343a542d4a139fe106c19095129b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:19:08.827435 17307 ts_tablet_manager.cc:1916] T 66ae176961b74813a58bc15c9c4c6f2a P 647a343a542d4a139fe106c19095129b: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:19:08.828711 16824 raft_consensus.cc:2955] T 57a744f5aaf944678a63469cbf514015 P cb82cc7bd6db46d980c75237a1402860 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:19:08.845152 16049 catalog_manager.cc:5697] T 57a744f5aaf944678a63469cbf514015 P 90f3da57796b488c9087a2f523adf045 reported cstate change: config changed from index 2 to 3, cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "647a343a542d4a139fe106c19095129b" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
W20260709 13:19:08.848364 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 192044us
I20260709 13:19:08.852206 16223 raft_consensus.cc:2955] T 746aabf856ab401c9ef5e859371e7b67 P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 90f3da57796b488c9087a2f523adf045 (127.14.177.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:19:08.864148 17307 ts_tablet_manager.cc:1929] T 66ae176961b74813a58bc15c9c4c6f2a P 647a343a542d4a139fe106c19095129b: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:19:08.864768 17307 log.cc:1199] T 66ae176961b74813a58bc15c9c4c6f2a P 647a343a542d4a139fe106c19095129b: Deleting WAL directory at /tmp/dist-test-taskdH_Ma7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783603093801613-15046-0/minicluster-data/ts-0-root/wals/66ae176961b74813a58bc15c9c4c6f2a
I20260709 13:19:08.866592 16035 catalog_manager.cc:5028] TS 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): tablet 66ae176961b74813a58bc15c9c4c6f2a (table test-workload [id=ed3d15d8c9ad46bebc4fe3a402d2e54f]) successfully deleted
I20260709 13:19:08.953953 16226 raft_consensus.cc:2955] T 57a744f5aaf944678a63469cbf514015 P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:19:08.964962 16223 raft_consensus.cc:2955] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:19:08.974884 16150 consensus_queue.cc:237] T 57a744f5aaf944678a63469cbf514015 P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:08.987545 16825 raft_consensus.cc:1275] T 57a744f5aaf944678a63469cbf514015 P cb82cc7bd6db46d980c75237a1402860 [term 1 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:19:08.989296 17216 consensus_queue.cc:1048] T 57a744f5aaf944678a63469cbf514015 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:19:08.995848 16147 consensus_queue.cc:237] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:08.998895 17303 raft_consensus.cc:2955] T 57a744f5aaf944678a63469cbf514015 P 647a343a542d4a139fe106c19095129b [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:19:09.007347 16825 raft_consensus.cc:2955] T 57a744f5aaf944678a63469cbf514015 P cb82cc7bd6db46d980c75237a1402860 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:19:09.013121 16824 raft_consensus.cc:1275] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 [term 1 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:19:09.013447 16223 raft_consensus.cc:1275] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:19:09.014780 17271 consensus_queue.cc:1048] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:19:09.017940 16226 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3a7ca61592e848ea9b0fb81a2ce33cdc"
dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
mode: GRACEFUL
new_leader_uuid: "cb82cc7bd6db46d980c75237a1402860"
 from {username='slave'} at 127.0.0.1:39144
I20260709 13:19:09.018648 16226 raft_consensus.cc:606] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 1ca2725d290e47d68b1a4b609c9816ac [term 1 LEADER]: Received request to transfer leadership to TS cb82cc7bd6db46d980c75237a1402860
I20260709 13:19:09.020272 16298 raft_consensus.cc:1275] T 57a744f5aaf944678a63469cbf514015 P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:19:09.020680 17293 consensus_queue.cc:1048] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:19:09.021903 17271 consensus_queue.cc:1048] T 57a744f5aaf944678a63469cbf514015 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:19:09.023290 17289 raft_consensus.cc:2955] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 90f3da57796b488c9087a2f523adf045 (127.14.177.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:19:09.029006 16298 raft_consensus.cc:2955] T 57a744f5aaf944678a63469cbf514015 P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:19:09.037590 16223 raft_consensus.cc:2955] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 90f3da57796b488c9087a2f523adf045 (127.14.177.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:19:09.045372 16824 raft_consensus.cc:2955] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 90f3da57796b488c9087a2f523adf045 (127.14.177.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:19:09.059505 16045 catalog_manager.cc:5697] T 57a744f5aaf944678a63469cbf514015 P 647a343a542d4a139fe106c19095129b reported cstate change: config changed from index 3 to 4, VOTER 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130) evicted. New cstate: current_term: 1 leader_uuid: "647a343a542d4a139fe106c19095129b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:19:09.071468 16209 tablet_service.cc:1558] Processing DeleteTablet for tablet 57a744f5aaf944678a63469cbf514015 with delete_type TABLET_DATA_TOMBSTONED (TS 1ca2725d290e47d68b1a4b609c9816ac not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:57598
I20260709 13:19:09.083057 17315 tablet_replica.cc:333] T 57a744f5aaf944678a63469cbf514015 P 1ca2725d290e47d68b1a4b609c9816ac: stopping tablet replica
I20260709 13:19:09.084234 17315 raft_consensus.cc:2243] T 57a744f5aaf944678a63469cbf514015 P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:19:09.084936 17315 raft_consensus.cc:2272] T 57a744f5aaf944678a63469cbf514015 P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:19:09.085302 16049 catalog_manager.cc:5697] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b reported cstate change: config changed from index 3 to 4, VOTER 90f3da57796b488c9087a2f523adf045 (127.14.177.131) evicted. New cstate: current_term: 1 leader_uuid: "647a343a542d4a139fe106c19095129b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:19:09.092031 16035 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 57a744f5aaf944678a63469cbf514015 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:19:09.093917 16035 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 5718c623ebee4d2bb770f0f9ddb2f164 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:19:09.115937 16286 tablet_service.cc:1558] Processing DeleteTablet for tablet 5718c623ebee4d2bb770f0f9ddb2f164 with delete_type TABLET_DATA_TOMBSTONED (TS 90f3da57796b488c9087a2f523adf045 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:48670
I20260709 13:19:09.116922 17306 tablet_replica.cc:333] T 5718c623ebee4d2bb770f0f9ddb2f164 P 90f3da57796b488c9087a2f523adf045: stopping tablet replica
I20260709 13:19:09.118016 17306 raft_consensus.cc:2243] T 5718c623ebee4d2bb770f0f9ddb2f164 P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:19:09.118777 17306 raft_consensus.cc:2272] T 5718c623ebee4d2bb770f0f9ddb2f164 P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:19:09.122748 17306 ts_tablet_manager.cc:1916] T 5718c623ebee4d2bb770f0f9ddb2f164 P 90f3da57796b488c9087a2f523adf045: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:19:09.125911 17315 ts_tablet_manager.cc:1916] T 57a744f5aaf944678a63469cbf514015 P 1ca2725d290e47d68b1a4b609c9816ac: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:19:09.148126 17306 ts_tablet_manager.cc:1929] T 5718c623ebee4d2bb770f0f9ddb2f164 P 90f3da57796b488c9087a2f523adf045: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:19:09.148787 17306 log.cc:1199] T 5718c623ebee4d2bb770f0f9ddb2f164 P 90f3da57796b488c9087a2f523adf045: Deleting WAL directory at /tmp/dist-test-taskdH_Ma7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783603093801613-15046-0/minicluster-data/ts-2-root/wals/5718c623ebee4d2bb770f0f9ddb2f164
I20260709 13:19:09.150250 16034 catalog_manager.cc:5028] TS 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389): tablet 5718c623ebee4d2bb770f0f9ddb2f164 (table test-workload [id=ed3d15d8c9ad46bebc4fe3a402d2e54f]) successfully deleted
I20260709 13:19:09.159835 17315 ts_tablet_manager.cc:1929] T 57a744f5aaf944678a63469cbf514015 P 1ca2725d290e47d68b1a4b609c9816ac: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:19:09.160441 17315 log.cc:1199] T 57a744f5aaf944678a63469cbf514015 P 1ca2725d290e47d68b1a4b609c9816ac: Deleting WAL directory at /tmp/dist-test-taskdH_Ma7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783603093801613-15046-0/minicluster-data/ts-1-root/wals/57a744f5aaf944678a63469cbf514015
I20260709 13:19:09.163475 16035 catalog_manager.cc:5028] TS 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943): tablet 57a744f5aaf944678a63469cbf514015 (table test-workload [id=ed3d15d8c9ad46bebc4fe3a402d2e54f]) successfully deleted
I20260709 13:19:09.181852 16147 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5718c623ebee4d2bb770f0f9ddb2f164"
dest_uuid: "647a343a542d4a139fe106c19095129b"
mode: GRACEFUL
new_leader_uuid: "cb82cc7bd6db46d980c75237a1402860"
 from {username='slave'} at 127.0.0.1:40290
I20260709 13:19:09.182596 16147 raft_consensus.cc:606] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b [term 1 LEADER]: Received request to transfer leadership to TS cb82cc7bd6db46d980c75237a1402860
I20260709 13:19:09.193765 16223 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8158bbd17b994d2eb96f560bd0f1b88d"
dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
mode: GRACEFUL
new_leader_uuid: "90f3da57796b488c9087a2f523adf045"
 from {username='slave'} at 127.0.0.1:39144
I20260709 13:19:09.194380 16223 raft_consensus.cc:606] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac [term 1 LEADER]: Received request to transfer leadership to TS 90f3da57796b488c9087a2f523adf045
I20260709 13:19:09.201428 16223 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ada5a004ba7c46819051e3ccec71ffdb"
dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
mode: GRACEFUL
new_leader_uuid: "cb82cc7bd6db46d980c75237a1402860"
 from {username='slave'} at 127.0.0.1:39144
I20260709 13:19:09.202026 16223 raft_consensus.cc:606] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac [term 1 LEADER]: Received request to transfer leadership to TS cb82cc7bd6db46d980c75237a1402860
I20260709 13:19:09.205719 16223 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c2f5f89186204242bf377f7c42b2e2da"
dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
mode: GRACEFUL
new_leader_uuid: "90f3da57796b488c9087a2f523adf045"
 from {username='slave'} at 127.0.0.1:39144
I20260709 13:19:09.206385 16223 raft_consensus.cc:606] T c2f5f89186204242bf377f7c42b2e2da P 1ca2725d290e47d68b1a4b609c9816ac [term 1 LEADER]: Received request to transfer leadership to TS 90f3da57796b488c9087a2f523adf045
I20260709 13:19:09.209808 16223 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c4f65cdf00b748bb963ee6e81ef43b79"
dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
mode: GRACEFUL
new_leader_uuid: "90f3da57796b488c9087a2f523adf045"
 from {username='slave'} at 127.0.0.1:39144
I20260709 13:19:09.210412 16223 raft_consensus.cc:606] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac [term 1 LEADER]: Received request to transfer leadership to TS 90f3da57796b488c9087a2f523adf045
I20260709 13:19:09.211642 15046 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 13:19:09.213318 15046 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:19:09.281483 17207 raft_consensus.cc:993] T c2f5f89186204242bf377f7c42b2e2da P 1ca2725d290e47d68b1a4b609c9816ac: : Instructing follower 90f3da57796b488c9087a2f523adf045 to start an election
I20260709 13:19:09.282080 17292 raft_consensus.cc:1081] T c2f5f89186204242bf377f7c42b2e2da P 1ca2725d290e47d68b1a4b609c9816ac [term 1 LEADER]: Signalling peer 90f3da57796b488c9087a2f523adf045 to start an election
I20260709 13:19:09.284116 16298 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c2f5f89186204242bf377f7c42b2e2da"
dest_uuid: "90f3da57796b488c9087a2f523adf045"
 from {username='slave'} at 127.0.0.1:48684
I20260709 13:19:09.284759 16298 raft_consensus.cc:493] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:19:09.285384 16298 raft_consensus.cc:3060] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:19:09.289641 16298 raft_consensus.cc:515] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:09.291273 16298 leader_election.cc:290] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 2 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:19:09.292043 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f5f89186204242bf377f7c42b2e2da" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:19:09.292495 16223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f5f89186204242bf377f7c42b2e2da" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:19:09.292806 16150 raft_consensus.cc:3060] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:19:09.293109 16223 raft_consensus.cc:3055] T c2f5f89186204242bf377f7c42b2e2da P 1ca2725d290e47d68b1a4b609c9816ac [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:19:09.293432 16223 raft_consensus.cc:740] T c2f5f89186204242bf377f7c42b2e2da P 1ca2725d290e47d68b1a4b609c9816ac [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:19:09.294106 16223 consensus_queue.cc:260] T c2f5f89186204242bf377f7c42b2e2da P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:09.295370 16223 raft_consensus.cc:3060] T c2f5f89186204242bf377f7c42b2e2da P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:19:09.304692 16223 raft_consensus.cc:2468] T c2f5f89186204242bf377f7c42b2e2da P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 2.
I20260709 13:19:09.305133 16150 raft_consensus.cc:2468] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 2.
I20260709 13:19:09.306914 16265 leader_election.cc:304] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:19:09.307853 17309 raft_consensus.cc:2804] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:19:09.308620 17309 raft_consensus.cc:697] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045 [term 2 LEADER]: Becoming Leader. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:19:09.309641 17309 consensus_queue.cc:237] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:09.321180 16047 catalog_manager.cc:5697] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045 reported cstate change: term changed from 1 to 2, leader changed from 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130) to 90f3da57796b488c9087a2f523adf045 (127.14.177.131). New cstate: current_term: 2 leader_uuid: "90f3da57796b488c9087a2f523adf045" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } health_report { overall_health: HEALTHY } } }
I20260709 13:19:09.434787 17292 raft_consensus.cc:993] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 1ca2725d290e47d68b1a4b609c9816ac: : Instructing follower cb82cc7bd6db46d980c75237a1402860 to start an election
I20260709 13:19:09.435251 17292 raft_consensus.cc:1081] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 1ca2725d290e47d68b1a4b609c9816ac [term 1 LEADER]: Signalling peer cb82cc7bd6db46d980c75237a1402860 to start an election
I20260709 13:19:09.437012 16821 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "3a7ca61592e848ea9b0fb81a2ce33cdc"
dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
 from {username='slave'} at 127.0.0.1:49906
I20260709 13:19:09.437569 16821 raft_consensus.cc:493] T 3a7ca61592e848ea9b0fb81a2ce33cdc P cb82cc7bd6db46d980c75237a1402860 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:19:09.437914 16821 raft_consensus.cc:3060] T 3a7ca61592e848ea9b0fb81a2ce33cdc P cb82cc7bd6db46d980c75237a1402860 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:19:09.448134 16821 raft_consensus.cc:515] T 3a7ca61592e848ea9b0fb81a2ce33cdc P cb82cc7bd6db46d980c75237a1402860 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:09.450429 16821 leader_election.cc:290] T 3a7ca61592e848ea9b0fb81a2ce33cdc P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 2 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:09.451013 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7ca61592e848ea9b0fb81a2ce33cdc" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:19:09.451695 16226 raft_consensus.cc:3055] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 1ca2725d290e47d68b1a4b609c9816ac [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:19:09.452061 16226 raft_consensus.cc:740] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 1ca2725d290e47d68b1a4b609c9816ac [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:19:09.452387 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7ca61592e848ea9b0fb81a2ce33cdc" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:19:09.452993 16298 raft_consensus.cc:3060] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:19:09.453220 16226 consensus_queue.cc:260] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:09.454408 16226 raft_consensus.cc:3060] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:19:09.460134 16298 raft_consensus.cc:2468] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 2.
I20260709 13:19:09.463073 16789 leader_election.cc:304] T 3a7ca61592e848ea9b0fb81a2ce33cdc P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:19:09.464010 17288 raft_consensus.cc:2804] T 3a7ca61592e848ea9b0fb81a2ce33cdc P cb82cc7bd6db46d980c75237a1402860 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:19:09.467707 16226 raft_consensus.cc:2468] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 2.
I20260709 13:19:09.468351 17288 raft_consensus.cc:697] T 3a7ca61592e848ea9b0fb81a2ce33cdc P cb82cc7bd6db46d980c75237a1402860 [term 2 LEADER]: Becoming Leader. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:19:09.469580 17288 consensus_queue.cc:237] T 3a7ca61592e848ea9b0fb81a2ce33cdc P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:09.482678 16047 catalog_manager.cc:5697] T 3a7ca61592e848ea9b0fb81a2ce33cdc P cb82cc7bd6db46d980c75237a1402860 reported cstate change: term changed from 1 to 2, leader changed from 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130) to cb82cc7bd6db46d980c75237a1402860 (127.14.177.132). New cstate: current_term: 2 leader_uuid: "cb82cc7bd6db46d980c75237a1402860" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:19:09.494444 17207 raft_consensus.cc:993] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac: : Instructing follower 90f3da57796b488c9087a2f523adf045 to start an election
I20260709 13:19:09.494967 17207 raft_consensus.cc:1081] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac [term 1 LEADER]: Signalling peer 90f3da57796b488c9087a2f523adf045 to start an election
I20260709 13:19:09.496498 16298 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "8158bbd17b994d2eb96f560bd0f1b88d"
dest_uuid: "90f3da57796b488c9087a2f523adf045"
 from {username='slave'} at 127.0.0.1:48684
I20260709 13:19:09.496969 16298 raft_consensus.cc:493] T 8158bbd17b994d2eb96f560bd0f1b88d P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:19:09.497256 16298 raft_consensus.cc:3060] T 8158bbd17b994d2eb96f560bd0f1b88d P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:19:09.501516 16298 raft_consensus.cc:515] T 8158bbd17b994d2eb96f560bd0f1b88d P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:09.503429 16298 leader_election.cc:290] T 8158bbd17b994d2eb96f560bd0f1b88d P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 2 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:19:09.504297 16147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8158bbd17b994d2eb96f560bd0f1b88d" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:19:09.504537 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8158bbd17b994d2eb96f560bd0f1b88d" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:19:09.505066 16147 raft_consensus.cc:3060] T 8158bbd17b994d2eb96f560bd0f1b88d P 647a343a542d4a139fe106c19095129b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:19:09.505154 16226 raft_consensus.cc:3055] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:19:09.505563 16226 raft_consensus.cc:740] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:19:09.506292 16226 consensus_queue.cc:260] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:09.507292 16226 raft_consensus.cc:3060] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:19:09.510975 16147 raft_consensus.cc:2468] T 8158bbd17b994d2eb96f560bd0f1b88d P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 2.
I20260709 13:19:09.512408 16265 leader_election.cc:304] T 8158bbd17b994d2eb96f560bd0f1b88d P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:19:09.512003 16226 raft_consensus.cc:2468] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 2.
I20260709 13:19:09.513448 17297 raft_consensus.cc:2804] T 8158bbd17b994d2eb96f560bd0f1b88d P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:19:09.513948 17297 raft_consensus.cc:697] T 8158bbd17b994d2eb96f560bd0f1b88d P 90f3da57796b488c9087a2f523adf045 [term 2 LEADER]: Becoming Leader. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:19:09.514822 17297 consensus_queue.cc:237] T 8158bbd17b994d2eb96f560bd0f1b88d P 90f3da57796b488c9087a2f523adf045 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:09.527699 16047 catalog_manager.cc:5697] T 8158bbd17b994d2eb96f560bd0f1b88d P 90f3da57796b488c9087a2f523adf045 reported cstate change: term changed from 1 to 2, leader changed from 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130) to 90f3da57796b488c9087a2f523adf045 (127.14.177.131). New cstate: current_term: 2 leader_uuid: "90f3da57796b488c9087a2f523adf045" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } health_report { overall_health: HEALTHY } } }
I20260709 13:19:09.606581 17293 raft_consensus.cc:993] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b: : Instructing follower cb82cc7bd6db46d980c75237a1402860 to start an election
I20260709 13:19:09.607036 17305 raft_consensus.cc:1081] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b [term 1 LEADER]: Signalling peer cb82cc7bd6db46d980c75237a1402860 to start an election
I20260709 13:19:09.608704 16821 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "5718c623ebee4d2bb770f0f9ddb2f164"
dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
 from {username='slave'} at 127.0.0.1:49896
I20260709 13:19:09.609269 16821 raft_consensus.cc:493] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:19:09.609685 16821 raft_consensus.cc:3060] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:19:09.614892 16821 raft_consensus.cc:515] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:09.616972 16821 leader_election.cc:290] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 2 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:19:09.617688 16147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5718c623ebee4d2bb770f0f9ddb2f164" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:19:09.617998 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5718c623ebee4d2bb770f0f9ddb2f164" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:19:09.618360 16147 raft_consensus.cc:3055] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:19:09.618579 16226 raft_consensus.cc:3060] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:19:09.618742 16147 raft_consensus.cc:740] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:19:09.619792 16147 consensus_queue.cc:260] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:09.620843 16147 raft_consensus.cc:3060] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:19:09.624593 16226 raft_consensus.cc:2468] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 2.
I20260709 13:19:09.625690 16790 leader_election.cc:304] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:19:09.626469 17288 raft_consensus.cc:2804] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:19:09.626614 16147 raft_consensus.cc:2468] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 2.
I20260709 13:19:09.626920 17288 raft_consensus.cc:697] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 [term 2 LEADER]: Becoming Leader. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:19:09.628028 17288 consensus_queue.cc:237] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:09.635617 16047 catalog_manager.cc:5697] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 reported cstate change: term changed from 1 to 2, leader changed from 647a343a542d4a139fe106c19095129b (127.14.177.129) to cb82cc7bd6db46d980c75237a1402860 (127.14.177.132). New cstate: current_term: 2 leader_uuid: "cb82cc7bd6db46d980c75237a1402860" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:19:09.727478 17207 raft_consensus.cc:993] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac: : Instructing follower 90f3da57796b488c9087a2f523adf045 to start an election
I20260709 13:19:09.728010 17292 raft_consensus.cc:1081] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac [term 1 LEADER]: Signalling peer 90f3da57796b488c9087a2f523adf045 to start an election
I20260709 13:19:09.729610 16298 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c4f65cdf00b748bb963ee6e81ef43b79"
dest_uuid: "90f3da57796b488c9087a2f523adf045"
 from {username='slave'} at 127.0.0.1:48684
I20260709 13:19:09.730187 16298 raft_consensus.cc:493] T c4f65cdf00b748bb963ee6e81ef43b79 P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:19:09.730553 16298 raft_consensus.cc:3060] T c4f65cdf00b748bb963ee6e81ef43b79 P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:19:09.737362 16298 raft_consensus.cc:515] T c4f65cdf00b748bb963ee6e81ef43b79 P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:09.739300 16298 leader_election.cc:290] T c4f65cdf00b748bb963ee6e81ef43b79 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 2 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:19:09.740439 16223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4f65cdf00b748bb963ee6e81ef43b79" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:19:09.741060 16223 raft_consensus.cc:3055] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:19:09.741350 16223 raft_consensus.cc:740] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:19:09.742039 16223 consensus_queue.cc:260] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:09.743395 16223 raft_consensus.cc:3060] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:19:09.740468 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4f65cdf00b748bb963ee6e81ef43b79" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:19:09.746171 16150 raft_consensus.cc:3060] T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:19:09.748775 16223 raft_consensus.cc:2468] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 2.
I20260709 13:19:09.749811 16265 leader_election.cc:304] T c4f65cdf00b748bb963ee6e81ef43b79 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:19:09.750592 17309 raft_consensus.cc:2804] T c4f65cdf00b748bb963ee6e81ef43b79 P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:19:09.751041 17309 raft_consensus.cc:697] T c4f65cdf00b748bb963ee6e81ef43b79 P 90f3da57796b488c9087a2f523adf045 [term 2 LEADER]: Becoming Leader. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:19:09.751798 17309 consensus_queue.cc:237] T c4f65cdf00b748bb963ee6e81ef43b79 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:09.752115 16150 raft_consensus.cc:2468] T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 2.
I20260709 13:19:09.761107 16047 catalog_manager.cc:5697] T c4f65cdf00b748bb963ee6e81ef43b79 P 90f3da57796b488c9087a2f523adf045 reported cstate change: term changed from 1 to 2, leader changed from 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130) to 90f3da57796b488c9087a2f523adf045 (127.14.177.131). New cstate: current_term: 2 leader_uuid: "90f3da57796b488c9087a2f523adf045" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } health_report { overall_health: HEALTHY } } }
I20260709 13:19:09.770630 16150 raft_consensus.cc:1275] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:19:09.772037 17309 consensus_queue.cc:1048] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:19:09.783815 17301 raft_consensus.cc:993] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac: : Instructing follower cb82cc7bd6db46d980c75237a1402860 to start an election
I20260709 13:19:09.784297 17301 raft_consensus.cc:1081] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac [term 1 LEADER]: Signalling peer cb82cc7bd6db46d980c75237a1402860 to start an election
I20260709 13:19:09.785609 16226 raft_consensus.cc:1275] T c2f5f89186204242bf377f7c42b2e2da P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:19:09.786118 16821 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ada5a004ba7c46819051e3ccec71ffdb"
dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
 from {username='slave'} at 127.0.0.1:49906
I20260709 13:19:09.786728 16821 raft_consensus.cc:493] T ada5a004ba7c46819051e3ccec71ffdb P cb82cc7bd6db46d980c75237a1402860 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:19:09.787552 16821 raft_consensus.cc:3060] T ada5a004ba7c46819051e3ccec71ffdb P cb82cc7bd6db46d980c75237a1402860 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:19:09.789317 17309 consensus_queue.cc:1048] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:19:09.798542 16821 raft_consensus.cc:515] T ada5a004ba7c46819051e3ccec71ffdb P cb82cc7bd6db46d980c75237a1402860 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:09.801585 16223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada5a004ba7c46819051e3ccec71ffdb" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:19:09.802171 16223 raft_consensus.cc:3055] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:19:09.802485 16223 raft_consensus.cc:740] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:19:09.803086 16821 leader_election.cc:290] T ada5a004ba7c46819051e3ccec71ffdb P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 2 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:09.804297 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada5a004ba7c46819051e3ccec71ffdb" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:19:09.804872 16298 raft_consensus.cc:3060] T ada5a004ba7c46819051e3ccec71ffdb P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:19:09.805872 16223 consensus_queue.cc:260] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:09.807665 16223 raft_consensus.cc:3060] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:19:09.812943 16223 raft_consensus.cc:2468] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 2.
I20260709 13:19:09.814008 16790 leader_election.cc:304] T ada5a004ba7c46819051e3ccec71ffdb P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:19:09.814450 16298 raft_consensus.cc:2468] T ada5a004ba7c46819051e3ccec71ffdb P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 2.
I20260709 13:19:09.815023 17288 raft_consensus.cc:2804] T ada5a004ba7c46819051e3ccec71ffdb P cb82cc7bd6db46d980c75237a1402860 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:19:09.815681 17288 raft_consensus.cc:697] T ada5a004ba7c46819051e3ccec71ffdb P cb82cc7bd6db46d980c75237a1402860 [term 2 LEADER]: Becoming Leader. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:19:09.816562 17288 consensus_queue.cc:237] T ada5a004ba7c46819051e3ccec71ffdb P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:09.841658 16047 catalog_manager.cc:5697] T ada5a004ba7c46819051e3ccec71ffdb P cb82cc7bd6db46d980c75237a1402860 reported cstate change: term changed from 1 to 2, leader changed from 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130) to cb82cc7bd6db46d980c75237a1402860 (127.14.177.132). New cstate: current_term: 2 leader_uuid: "cb82cc7bd6db46d980c75237a1402860" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:19:09.985909 16298 raft_consensus.cc:1275] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:19:09.986346 16226 raft_consensus.cc:1275] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:19:09.987246 17288 consensus_queue.cc:1048] T 3a7ca61592e848ea9b0fb81a2ce33cdc P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:19:09.987761 17297 consensus_queue.cc:1048] T 8158bbd17b994d2eb96f560bd0f1b88d P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:19:10.000839 16147 raft_consensus.cc:1275] T 8158bbd17b994d2eb96f560bd0f1b88d P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:19:10.002295 17322 consensus_queue.cc:1048] T 8158bbd17b994d2eb96f560bd0f1b88d P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:19:10.010583 16223 raft_consensus.cc:1275] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:19:10.020341 17288 consensus_queue.cc:1048] T 3a7ca61592e848ea9b0fb81a2ce33cdc P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.004s
I20260709 13:19:10.105599 16223 raft_consensus.cc:1275] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:19:10.106917 17288 consensus_queue.cc:1048] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:19:10.116602 16150 raft_consensus.cc:1275] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:19:10.119632 17335 consensus_queue.cc:1048] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:19:10.193921 16226 consensus_queue.cc:237] T af57996a8be84d398f0f023e8d08ef77 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } }
I20260709 13:19:10.200390 16298 raft_consensus.cc:1275] T af57996a8be84d398f0f023e8d08ef77 P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:19:10.200717 16150 raft_consensus.cc:1275] T af57996a8be84d398f0f023e8d08ef77 P 647a343a542d4a139fe106c19095129b [term 1 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 13:19:10.200789 16192 consensus_peers.cc:597] T af57996a8be84d398f0f023e8d08ef77 P 1ca2725d290e47d68b1a4b609c9816ac -> Peer cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923): Couldn't send request to peer cb82cc7bd6db46d980c75237a1402860. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: af57996a8be84d398f0f023e8d08ef77. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:19:10.201645 17207 consensus_queue.cc:1048] T af57996a8be84d398f0f023e8d08ef77 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:19:10.202446 17292 consensus_queue.cc:1048] T af57996a8be84d398f0f023e8d08ef77 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:19:10.210877 17207 raft_consensus.cc:2955] T af57996a8be84d398f0f023e8d08ef77 P 1ca2725d290e47d68b1a4b609c9816ac [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } } }
I20260709 13:19:10.212541 16298 raft_consensus.cc:2955] T af57996a8be84d398f0f023e8d08ef77 P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } } }
I20260709 13:19:10.214355 15046 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:19:10.214509 16150 raft_consensus.cc:2955] T af57996a8be84d398f0f023e8d08ef77 P 647a343a542d4a139fe106c19095129b [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } } }
I20260709 13:19:10.227033 16147 consensus_queue.cc:237] T 23f82224f4564fea93572e8206bad1fe P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } }
I20260709 13:19:10.232822 16049 catalog_manager.cc:5697] T af57996a8be84d398f0f023e8d08ef77 P 1ca2725d290e47d68b1a4b609c9816ac reported cstate change: config changed from index -1 to 2, NON_VOTER cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) added. New cstate: current_term: 1 leader_uuid: "1ca2725d290e47d68b1a4b609c9816ac" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:19:10.248703 16144 raft_consensus.cc:1275] T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 13:19:10.250566 16117 consensus_peers.cc:597] T 23f82224f4564fea93572e8206bad1fe P 647a343a542d4a139fe106c19095129b -> Peer cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923): Couldn't send request to peer cb82cc7bd6db46d980c75237a1402860. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 23f82224f4564fea93572e8206bad1fe. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:19:10.250820 16298 raft_consensus.cc:1275] T 23f82224f4564fea93572e8206bad1fe P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:19:10.252030 17322 consensus_queue.cc:1048] T c4f65cdf00b748bb963ee6e81ef43b79 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:19:10.252801 17290 consensus_queue.cc:1048] T 23f82224f4564fea93572e8206bad1fe P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:19:10.254541 16226 raft_consensus.cc:1275] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:19:10.256234 17288 consensus_queue.cc:1048] T ada5a004ba7c46819051e3ccec71ffdb P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:19:10.266815 16223 raft_consensus.cc:1275] T 23f82224f4564fea93572e8206bad1fe P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:19:10.279655 16298 raft_consensus.cc:1275] T ada5a004ba7c46819051e3ccec71ffdb P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:19:10.281335 17293 consensus_queue.cc:1048] T 23f82224f4564fea93572e8206bad1fe P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:19:10.286631 17335 consensus_queue.cc:1048] T ada5a004ba7c46819051e3ccec71ffdb P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:19:10.289520 16226 raft_consensus.cc:1275] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:19:10.292785 17326 consensus_queue.cc:1048] T c4f65cdf00b748bb963ee6e81ef43b79 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:19:10.339303 15046 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:19:10.368235 17293 raft_consensus.cc:2955] T 23f82224f4564fea93572e8206bad1fe P 647a343a542d4a139fe106c19095129b [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } } }
I20260709 13:19:10.370960 16298 raft_consensus.cc:2955] T 23f82224f4564fea93572e8206bad1fe P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } } }
I20260709 13:19:10.386528 16226 raft_consensus.cc:2955] T 23f82224f4564fea93572e8206bad1fe P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } } }
I20260709 13:19:10.410081 16222 consensus_queue.cc:237] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } }
I20260709 13:19:10.414039 16045 catalog_manager.cc:5697] T 23f82224f4564fea93572e8206bad1fe P 647a343a542d4a139fe106c19095129b reported cstate change: config changed from index -1 to 2, NON_VOTER cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) added. New cstate: current_term: 1 leader_uuid: "647a343a542d4a139fe106c19095129b" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:19:10.434394 16147 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "19baa7bc367844c288bca8f81439c413"
dest_uuid: "647a343a542d4a139fe106c19095129b"
mode: GRACEFUL
new_leader_uuid: "90f3da57796b488c9087a2f523adf045"
 from {username='slave'} at 127.0.0.1:40290
I20260709 13:19:10.435035 16147 raft_consensus.cc:606] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [term 1 LEADER]: Received request to transfer leadership to TS 90f3da57796b488c9087a2f523adf045
I20260709 13:19:10.449811 16147 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "434bff3082d141838068f323bdb0d1ed"
dest_uuid: "647a343a542d4a139fe106c19095129b"
mode: GRACEFUL
new_leader_uuid: "90f3da57796b488c9087a2f523adf045"
 from {username='slave'} at 127.0.0.1:40290
I20260709 13:19:10.450537 16147 raft_consensus.cc:606] T 434bff3082d141838068f323bdb0d1ed P 647a343a542d4a139fe106c19095129b [term 1 LEADER]: Received request to transfer leadership to TS 90f3da57796b488c9087a2f523adf045
I20260709 13:19:10.450951 16298 raft_consensus.cc:1275] T c9944d5e4b8c4109af9955c4f08c9095 P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 13:19:10.453303 16192 consensus_peers.cc:597] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac -> Peer cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923): Couldn't send request to peer cb82cc7bd6db46d980c75237a1402860. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: c9944d5e4b8c4109af9955c4f08c9095. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:19:10.453860 17340 consensus_queue.cc:1048] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:19:10.454170 16147 raft_consensus.cc:1275] T c9944d5e4b8c4109af9955c4f08c9095 P 647a343a542d4a139fe106c19095129b [term 1 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:19:10.455756 17340 consensus_queue.cc:1048] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:19:10.465631 16222 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8638ee363273426db1393dd81e5c7b72"
dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
mode: GRACEFUL
new_leader_uuid: "cb82cc7bd6db46d980c75237a1402860"
 from {username='slave'} at 127.0.0.1:39144
I20260709 13:19:10.466300 16222 raft_consensus.cc:606] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [term 1 LEADER]: Received request to transfer leadership to TS cb82cc7bd6db46d980c75237a1402860
I20260709 13:19:10.470696 16226 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d2ab0ac8347641e6955e9bce7d90c1a2"
dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
mode: GRACEFUL
new_leader_uuid: "cb82cc7bd6db46d980c75237a1402860"
 from {username='slave'} at 127.0.0.1:39144
I20260709 13:19:10.471253 16226 raft_consensus.cc:606] T d2ab0ac8347641e6955e9bce7d90c1a2 P 1ca2725d290e47d68b1a4b609c9816ac [term 1 LEADER]: Received request to transfer leadership to TS cb82cc7bd6db46d980c75237a1402860
I20260709 13:19:10.475940 16150 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e5ad4eb281564936a59f41e387a47156"
dest_uuid: "647a343a542d4a139fe106c19095129b"
mode: GRACEFUL
new_leader_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
 from {username='slave'} at 127.0.0.1:40290
I20260709 13:19:10.476506 16150 raft_consensus.cc:606] T e5ad4eb281564936a59f41e387a47156 P 647a343a542d4a139fe106c19095129b [term 1 LEADER]: Received request to transfer leadership to TS 1ca2725d290e47d68b1a4b609c9816ac
I20260709 13:19:10.477753 15046 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:19:10.482857 15046 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:19:10.486001 17340 raft_consensus.cc:2955] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } } }
I20260709 13:19:10.489120 16150 raft_consensus.cc:2955] T c9944d5e4b8c4109af9955c4f08c9095 P 647a343a542d4a139fe106c19095129b [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } } }
I20260709 13:19:10.499703 16298 raft_consensus.cc:2955] T c9944d5e4b8c4109af9955c4f08c9095 P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } } }
I20260709 13:19:10.502731 16147 consensus_queue.cc:237] T 829b4ad8c5a54eababeea1181b960d4b P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } }
I20260709 13:19:10.503805 17351 raft_consensus.cc:993] T e5ad4eb281564936a59f41e387a47156 P 647a343a542d4a139fe106c19095129b: : Instructing follower 1ca2725d290e47d68b1a4b609c9816ac to start an election
I20260709 13:19:10.504343 17293 raft_consensus.cc:1081] T e5ad4eb281564936a59f41e387a47156 P 647a343a542d4a139fe106c19095129b [term 1 LEADER]: Signalling peer 1ca2725d290e47d68b1a4b609c9816ac to start an election
I20260709 13:19:10.506286 16222 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e5ad4eb281564936a59f41e387a47156"
dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
 from {username='slave'} at 127.0.0.1:57604
I20260709 13:19:10.506767 16222 raft_consensus.cc:493] T e5ad4eb281564936a59f41e387a47156 P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:19:10.507120 16222 raft_consensus.cc:3060] T e5ad4eb281564936a59f41e387a47156 P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:19:10.503119 16045 catalog_manager.cc:5697] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac reported cstate change: config changed from index -1 to 2, NON_VOTER cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) added. New cstate: current_term: 1 leader_uuid: "1ca2725d290e47d68b1a4b609c9816ac" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:19:10.514165 16222 raft_consensus.cc:515] T e5ad4eb281564936a59f41e387a47156 P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:10.517045 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5ad4eb281564936a59f41e387a47156" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:19:10.517045 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5ad4eb281564936a59f41e387a47156" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:19:10.517794 16150 raft_consensus.cc:3055] T e5ad4eb281564936a59f41e387a47156 P 647a343a542d4a139fe106c19095129b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:19:10.517856 16299 raft_consensus.cc:3060] T e5ad4eb281564936a59f41e387a47156 P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:19:10.518141 16150 raft_consensus.cc:740] T e5ad4eb281564936a59f41e387a47156 P 647a343a542d4a139fe106c19095129b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:19:10.518870 16150 consensus_queue.cc:260] T e5ad4eb281564936a59f41e387a47156 P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:10.519815 16150 raft_consensus.cc:3060] T e5ad4eb281564936a59f41e387a47156 P 647a343a542d4a139fe106c19095129b [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:19:10.528475 16117 consensus_peers.cc:597] T 829b4ad8c5a54eababeea1181b960d4b P 647a343a542d4a139fe106c19095129b -> Peer cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923): Couldn't send request to peer cb82cc7bd6db46d980c75237a1402860. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 829b4ad8c5a54eababeea1181b960d4b. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:19:10.528936 16298 raft_consensus.cc:1275] T 829b4ad8c5a54eababeea1181b960d4b P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:19:10.528980 16226 raft_consensus.cc:1275] T 829b4ad8c5a54eababeea1181b960d4b P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:19:10.529026 17340 raft_consensus.cc:993] T d2ab0ac8347641e6955e9bce7d90c1a2 P 1ca2725d290e47d68b1a4b609c9816ac: : Instructing follower cb82cc7bd6db46d980c75237a1402860 to start an election
I20260709 13:19:10.530706 17340 raft_consensus.cc:1081] T d2ab0ac8347641e6955e9bce7d90c1a2 P 1ca2725d290e47d68b1a4b609c9816ac [term 1 LEADER]: Signalling peer cb82cc7bd6db46d980c75237a1402860 to start an election
I20260709 13:19:10.530586 17349 consensus_queue.cc:1048] T 829b4ad8c5a54eababeea1181b960d4b P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:19:10.532585 17352 consensus_queue.cc:1048] T 829b4ad8c5a54eababeea1181b960d4b P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:19:10.533062 16821 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d2ab0ac8347641e6955e9bce7d90c1a2"
dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
 from {username='slave'} at 127.0.0.1:49906
I20260709 13:19:10.533615 16821 raft_consensus.cc:493] T d2ab0ac8347641e6955e9bce7d90c1a2 P cb82cc7bd6db46d980c75237a1402860 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:19:10.533948 16821 raft_consensus.cc:3060] T d2ab0ac8347641e6955e9bce7d90c1a2 P cb82cc7bd6db46d980c75237a1402860 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:19:10.551820 16821 raft_consensus.cc:515] T d2ab0ac8347641e6955e9bce7d90c1a2 P cb82cc7bd6db46d980c75237a1402860 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:10.554515 16223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ab0ac8347641e6955e9bce7d90c1a2" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:19:10.555097 16223 raft_consensus.cc:3055] T d2ab0ac8347641e6955e9bce7d90c1a2 P 1ca2725d290e47d68b1a4b609c9816ac [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:19:10.555442 16223 raft_consensus.cc:740] T d2ab0ac8347641e6955e9bce7d90c1a2 P 1ca2725d290e47d68b1a4b609c9816ac [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:19:10.556201 16223 consensus_queue.cc:260] T d2ab0ac8347641e6955e9bce7d90c1a2 P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:10.557410 16223 raft_consensus.cc:3060] T d2ab0ac8347641e6955e9bce7d90c1a2 P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:19:10.559420 16150 raft_consensus.cc:2468] T e5ad4eb281564936a59f41e387a47156 P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 2.
I20260709 13:19:10.560920 16299 raft_consensus.cc:2468] T e5ad4eb281564936a59f41e387a47156 P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 2.
I20260709 13:19:10.560808 16191 leader_election.cc:304] T e5ad4eb281564936a59f41e387a47156 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
I20260709 13:19:10.562203 17292 raft_consensus.cc:2804] T e5ad4eb281564936a59f41e387a47156 P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:19:10.562315 16222 leader_election.cc:290] T e5ad4eb281564936a59f41e387a47156 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 2 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:10.562772 17292 raft_consensus.cc:697] T e5ad4eb281564936a59f41e387a47156 P 1ca2725d290e47d68b1a4b609c9816ac [term 2 LEADER]: Becoming Leader. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:19:10.563750 17292 consensus_queue.cc:237] T e5ad4eb281564936a59f41e387a47156 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:10.565160 16821 leader_election.cc:290] T d2ab0ac8347641e6955e9bce7d90c1a2 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 2 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:10.566538 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ab0ac8347641e6955e9bce7d90c1a2" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:19:10.567533 16299 raft_consensus.cc:3060] T d2ab0ac8347641e6955e9bce7d90c1a2 P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:19:10.574867 16296 raft_consensus.cc:2955] T 829b4ad8c5a54eababeea1181b960d4b P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } } }
I20260709 13:19:10.578063 16299 raft_consensus.cc:2468] T d2ab0ac8347641e6955e9bce7d90c1a2 P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 2.
I20260709 13:19:10.571365 17352 raft_consensus.cc:2955] T 829b4ad8c5a54eababeea1181b960d4b P 647a343a542d4a139fe106c19095129b [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } } }
I20260709 13:19:10.579098 16789 leader_election.cc:304] T d2ab0ac8347641e6955e9bce7d90c1a2 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:19:10.579852 17335 raft_consensus.cc:2804] T d2ab0ac8347641e6955e9bce7d90c1a2 P cb82cc7bd6db46d980c75237a1402860 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:19:10.580340 17335 raft_consensus.cc:697] T d2ab0ac8347641e6955e9bce7d90c1a2 P cb82cc7bd6db46d980c75237a1402860 [term 2 LEADER]: Becoming Leader. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:19:10.581146 17335 consensus_queue.cc:237] T d2ab0ac8347641e6955e9bce7d90c1a2 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:10.584208 16223 raft_consensus.cc:2468] T d2ab0ac8347641e6955e9bce7d90c1a2 P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 2.
I20260709 13:19:10.589273 16226 raft_consensus.cc:2955] T 829b4ad8c5a54eababeea1181b960d4b P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } } }
I20260709 13:19:10.594318 16047 catalog_manager.cc:5697] T e5ad4eb281564936a59f41e387a47156 P 1ca2725d290e47d68b1a4b609c9816ac reported cstate change: term changed from 1 to 2, leader changed from 647a343a542d4a139fe106c19095129b (127.14.177.129) to 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130). New cstate: current_term: 2 leader_uuid: "1ca2725d290e47d68b1a4b609c9816ac" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } health_report { overall_health: UNKNOWN } } }
I20260709 13:19:10.614722 16049 catalog_manager.cc:5697] T 829b4ad8c5a54eababeea1181b960d4b P 647a343a542d4a139fe106c19095129b reported cstate change: config changed from index -1 to 2, NON_VOTER cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) added. New cstate: current_term: 1 leader_uuid: "647a343a542d4a139fe106c19095129b" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:19:10.623615 16046 catalog_manager.cc:5697] T d2ab0ac8347641e6955e9bce7d90c1a2 P cb82cc7bd6db46d980c75237a1402860 reported cstate change: term changed from 1 to 2, leader changed from 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130) to cb82cc7bd6db46d980c75237a1402860 (127.14.177.132). New cstate: current_term: 2 leader_uuid: "cb82cc7bd6db46d980c75237a1402860" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:19:10.798290 17351 raft_consensus.cc:993] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b: : Instructing follower 90f3da57796b488c9087a2f523adf045 to start an election
I20260709 13:19:10.798822 17305 raft_consensus.cc:1081] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [term 1 LEADER]: Signalling peer 90f3da57796b488c9087a2f523adf045 to start an election
I20260709 13:19:10.800599 16296 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "19baa7bc367844c288bca8f81439c413"
dest_uuid: "90f3da57796b488c9087a2f523adf045"
 from {username='slave'} at 127.0.0.1:48674
I20260709 13:19:10.801213 16296 raft_consensus.cc:493] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:19:10.801574 16296 raft_consensus.cc:3060] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:19:10.808916 16296 raft_consensus.cc:515] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:10.811542 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:19:10.812114 16150 raft_consensus.cc:3055] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:19:10.812399 16150 raft_consensus.cc:740] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:19:10.813143 16150 consensus_queue.cc:260] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:10.814180 16150 raft_consensus.cc:3060] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:19:10.820304 16296 leader_election.cc:290] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 2 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:19:10.824288 16150 raft_consensus.cc:2468] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 2.
I20260709 13:19:10.825464 16265 leader_election.cc:304] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:19:10.827595 17360 ts_tablet_manager.cc:933] T af57996a8be84d398f0f023e8d08ef77 P cb82cc7bd6db46d980c75237a1402860: Initiating tablet copy from peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:19:10.827513 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:19:10.828336 16226 raft_consensus.cc:3060] T 19baa7bc367844c288bca8f81439c413 P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:19:10.828711 17309 raft_consensus.cc:2804] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:19:10.829252 17309 raft_consensus.cc:697] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045 [term 2 LEADER]: Becoming Leader. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:19:10.830255 17309 consensus_queue.cc:237] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:10.864710 16048 catalog_manager.cc:5697] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045 reported cstate change: term changed from 1 to 2, leader changed from 647a343a542d4a139fe106c19095129b (127.14.177.129) to 90f3da57796b488c9087a2f523adf045 (127.14.177.131). New cstate: current_term: 2 leader_uuid: "90f3da57796b488c9087a2f523adf045" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } health_report { overall_health: HEALTHY } } }
I20260709 13:19:10.866345 17363 ts_tablet_manager.cc:933] T 23f82224f4564fea93572e8206bad1fe P cb82cc7bd6db46d980c75237a1402860: Initiating tablet copy from peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741)
I20260709 13:19:10.866818 16226 raft_consensus.cc:2468] T 19baa7bc367844c288bca8f81439c413 P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 2.
I20260709 13:19:10.884768 17360 tablet_copy_client.cc:323] T af57996a8be84d398f0f023e8d08ef77 P cb82cc7bd6db46d980c75237a1402860: tablet copy: Beginning tablet copy session from remote peer at address 127.14.177.130:35943
I20260709 13:19:10.888283 16236 tablet_copy_service.cc:140] P 1ca2725d290e47d68b1a4b609c9816ac: Received BeginTabletCopySession request for tablet af57996a8be84d398f0f023e8d08ef77 from peer cb82cc7bd6db46d980c75237a1402860 ({username='slave'} at 127.0.0.1:33050)
I20260709 13:19:10.888963 16236 tablet_copy_service.cc:161] P 1ca2725d290e47d68b1a4b609c9816ac: Beginning new tablet copy session on tablet af57996a8be84d398f0f023e8d08ef77 from peer cb82cc7bd6db46d980c75237a1402860 at {username='slave'} at 127.0.0.1:33050: session id = cb82cc7bd6db46d980c75237a1402860-af57996a8be84d398f0f023e8d08ef77
I20260709 13:19:10.896972 16236 tablet_copy_source_session.cc:215] T af57996a8be84d398f0f023e8d08ef77 P 1ca2725d290e47d68b1a4b609c9816ac: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:19:10.904255 17360 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet af57996a8be84d398f0f023e8d08ef77. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:19:10.924832 17363 tablet_copy_client.cc:323] T 23f82224f4564fea93572e8206bad1fe P cb82cc7bd6db46d980c75237a1402860: tablet copy: Beginning tablet copy session from remote peer at address 127.14.177.129:36741
I20260709 13:19:10.927726 16161 tablet_copy_service.cc:140] P 647a343a542d4a139fe106c19095129b: Received BeginTabletCopySession request for tablet 23f82224f4564fea93572e8206bad1fe from peer cb82cc7bd6db46d980c75237a1402860 ({username='slave'} at 127.0.0.1:46224)
I20260709 13:19:10.927898 17360 tablet_copy_client.cc:806] T af57996a8be84d398f0f023e8d08ef77 P cb82cc7bd6db46d980c75237a1402860: tablet copy: Starting download of 0 data blocks...
I20260709 13:19:10.928345 16161 tablet_copy_service.cc:161] P 647a343a542d4a139fe106c19095129b: Beginning new tablet copy session on tablet 23f82224f4564fea93572e8206bad1fe from peer cb82cc7bd6db46d980c75237a1402860 at {username='slave'} at 127.0.0.1:46224: session id = cb82cc7bd6db46d980c75237a1402860-23f82224f4564fea93572e8206bad1fe
I20260709 13:19:10.929270 17360 tablet_copy_client.cc:670] T af57996a8be84d398f0f023e8d08ef77 P cb82cc7bd6db46d980c75237a1402860: tablet copy: Starting download of 1 WAL segments...
I20260709 13:19:10.934357 17360 tablet_copy_client.cc:538] T af57996a8be84d398f0f023e8d08ef77 P cb82cc7bd6db46d980c75237a1402860: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:19:10.946362 17365 ts_tablet_manager.cc:933] T c9944d5e4b8c4109af9955c4f08c9095 P cb82cc7bd6db46d980c75237a1402860: Initiating tablet copy from peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:19:10.947705 17360 tablet_bootstrap.cc:492] T af57996a8be84d398f0f023e8d08ef77 P cb82cc7bd6db46d980c75237a1402860: Bootstrap starting.
I20260709 13:19:10.989240 17360 tablet_bootstrap.cc:492] T af57996a8be84d398f0f023e8d08ef77 P cb82cc7bd6db46d980c75237a1402860: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:19:10.990195 17360 tablet_bootstrap.cc:492] T af57996a8be84d398f0f023e8d08ef77 P cb82cc7bd6db46d980c75237a1402860: Bootstrap complete.
I20260709 13:19:10.990860 17360 ts_tablet_manager.cc:1403] T af57996a8be84d398f0f023e8d08ef77 P cb82cc7bd6db46d980c75237a1402860: Time spent bootstrapping tablet: real 0.043s	user 0.026s	sys 0.008s
I20260709 13:19:10.993790 17360 raft_consensus.cc:359] T af57996a8be84d398f0f023e8d08ef77 P cb82cc7bd6db46d980c75237a1402860 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } }
I20260709 13:19:10.994578 17360 raft_consensus.cc:740] T af57996a8be84d398f0f023e8d08ef77 P cb82cc7bd6db46d980c75237a1402860 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Initialized, Role: LEARNER
W20260709 13:19:10.995118 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 72213.5us
I20260709 13:19:11.002031 17360 consensus_queue.cc:260] T af57996a8be84d398f0f023e8d08ef77 P cb82cc7bd6db46d980c75237a1402860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } }
I20260709 13:19:11.004623 17360 ts_tablet_manager.cc:1434] T af57996a8be84d398f0f023e8d08ef77 P cb82cc7bd6db46d980c75237a1402860: Time spent starting tablet: real 0.013s	user 0.008s	sys 0.000s
I20260709 13:19:11.010763 16236 tablet_copy_service.cc:342] P 1ca2725d290e47d68b1a4b609c9816ac: Request end of tablet copy session cb82cc7bd6db46d980c75237a1402860-af57996a8be84d398f0f023e8d08ef77 received from {username='slave'} at 127.0.0.1:33050
I20260709 13:19:11.011385 16236 tablet_copy_service.cc:434] P 1ca2725d290e47d68b1a4b609c9816ac: ending tablet copy session cb82cc7bd6db46d980c75237a1402860-af57996a8be84d398f0f023e8d08ef77 on tablet af57996a8be84d398f0f023e8d08ef77 with peer cb82cc7bd6db46d980c75237a1402860
I20260709 13:19:11.015113 16150 raft_consensus.cc:1275] T e5ad4eb281564936a59f41e387a47156 P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:19:11.017038 17292 consensus_queue.cc:1048] T e5ad4eb281564936a59f41e387a47156 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:19:11.018509 16296 raft_consensus.cc:1275] T e5ad4eb281564936a59f41e387a47156 P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:19:11.026381 17352 raft_consensus.cc:993] T 434bff3082d141838068f323bdb0d1ed P 647a343a542d4a139fe106c19095129b: : Instructing follower 90f3da57796b488c9087a2f523adf045 to start an election
I20260709 13:19:11.026857 17352 raft_consensus.cc:1081] T 434bff3082d141838068f323bdb0d1ed P 647a343a542d4a139fe106c19095129b [term 1 LEADER]: Signalling peer 90f3da57796b488c9087a2f523adf045 to start an election
I20260709 13:19:11.027630 17292 consensus_queue.cc:1048] T e5ad4eb281564936a59f41e387a47156 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:19:11.037247 16222 raft_consensus.cc:1275] T d2ab0ac8347641e6955e9bce7d90c1a2 P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:19:11.039230 17335 consensus_queue.cc:1048] T d2ab0ac8347641e6955e9bce7d90c1a2 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:19:11.031314 16296 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "434bff3082d141838068f323bdb0d1ed"
dest_uuid: "90f3da57796b488c9087a2f523adf045"
 from {username='slave'} at 127.0.0.1:48674
I20260709 13:19:11.067184 17292 raft_consensus.cc:993] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac: : Instructing follower cb82cc7bd6db46d980c75237a1402860 to start an election
I20260709 13:19:11.080720 17292 raft_consensus.cc:1081] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [term 1 LEADER]: Signalling peer cb82cc7bd6db46d980c75237a1402860 to start an election
I20260709 13:19:11.082728 16821 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "8638ee363273426db1393dd81e5c7b72"
dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
 from {username='slave'} at 127.0.0.1:49906
I20260709 13:19:11.083263 16821 raft_consensus.cc:493] T 8638ee363273426db1393dd81e5c7b72 P cb82cc7bd6db46d980c75237a1402860 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:19:11.083623 16821 raft_consensus.cc:3060] T 8638ee363273426db1393dd81e5c7b72 P cb82cc7bd6db46d980c75237a1402860 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:19:11.087503 17360 ts_tablet_manager.cc:933] T 829b4ad8c5a54eababeea1181b960d4b P cb82cc7bd6db46d980c75237a1402860: Initiating tablet copy from peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741)
I20260709 13:19:11.069357 16161 tablet_copy_source_session.cc:215] T 23f82224f4564fea93572e8206bad1fe P 647a343a542d4a139fe106c19095129b: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:19:11.095605 17365 tablet_copy_client.cc:323] T c9944d5e4b8c4109af9955c4f08c9095 P cb82cc7bd6db46d980c75237a1402860: tablet copy: Beginning tablet copy session from remote peer at address 127.14.177.130:35943
I20260709 13:19:11.069111 16296 raft_consensus.cc:493] T 434bff3082d141838068f323bdb0d1ed P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:19:11.097172 16296 raft_consensus.cc:3060] T 434bff3082d141838068f323bdb0d1ed P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:19:11.097411 16236 tablet_copy_service.cc:140] P 1ca2725d290e47d68b1a4b609c9816ac: Received BeginTabletCopySession request for tablet c9944d5e4b8c4109af9955c4f08c9095 from peer cb82cc7bd6db46d980c75237a1402860 ({username='slave'} at 127.0.0.1:33050)
I20260709 13:19:11.097991 16236 tablet_copy_service.cc:161] P 1ca2725d290e47d68b1a4b609c9816ac: Beginning new tablet copy session on tablet c9944d5e4b8c4109af9955c4f08c9095 from peer cb82cc7bd6db46d980c75237a1402860 at {username='slave'} at 127.0.0.1:33050: session id = cb82cc7bd6db46d980c75237a1402860-c9944d5e4b8c4109af9955c4f08c9095
I20260709 13:19:11.103049 17363 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 23f82224f4564fea93572e8206bad1fe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:19:11.104730 16296 raft_consensus.cc:515] T 434bff3082d141838068f323bdb0d1ed P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:11.104803 16821 raft_consensus.cc:515] T 8638ee363273426db1393dd81e5c7b72 P cb82cc7bd6db46d980c75237a1402860 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:11.107836 16222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8638ee363273426db1393dd81e5c7b72" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:19:11.108412 16222 raft_consensus.cc:3055] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:19:11.108719 16222 raft_consensus.cc:740] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:19:11.109490 16222 consensus_queue.cc:260] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:11.111122 16222 raft_consensus.cc:3060] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:19:11.112959 17360 tablet_copy_client.cc:323] T 829b4ad8c5a54eababeea1181b960d4b P cb82cc7bd6db46d980c75237a1402860: tablet copy: Beginning tablet copy session from remote peer at address 127.14.177.129:36741
I20260709 13:19:11.115669 16161 tablet_copy_service.cc:140] P 647a343a542d4a139fe106c19095129b: Received BeginTabletCopySession request for tablet 829b4ad8c5a54eababeea1181b960d4b from peer cb82cc7bd6db46d980c75237a1402860 ({username='slave'} at 127.0.0.1:46224)
I20260709 13:19:11.116278 16161 tablet_copy_service.cc:161] P 647a343a542d4a139fe106c19095129b: Beginning new tablet copy session on tablet 829b4ad8c5a54eababeea1181b960d4b from peer cb82cc7bd6db46d980c75237a1402860 at {username='slave'} at 127.0.0.1:46224: session id = cb82cc7bd6db46d980c75237a1402860-829b4ad8c5a54eababeea1181b960d4b
I20260709 13:19:11.130752 16222 raft_consensus.cc:2468] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 2.
I20260709 13:19:11.115934 16236 tablet_copy_source_session.cc:215] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:19:11.136878 16821 leader_election.cc:290] T 8638ee363273426db1393dd81e5c7b72 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 2 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:11.138125 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8638ee363273426db1393dd81e5c7b72" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:19:11.140764 16298 raft_consensus.cc:3060] T 8638ee363273426db1393dd81e5c7b72 P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:19:11.141290 16790 leader_election.cc:304] T 8638ee363273426db1393dd81e5c7b72 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:19:11.142170 17335 raft_consensus.cc:2804] T 8638ee363273426db1393dd81e5c7b72 P cb82cc7bd6db46d980c75237a1402860 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:19:11.142635 17335 raft_consensus.cc:697] T 8638ee363273426db1393dd81e5c7b72 P cb82cc7bd6db46d980c75237a1402860 [term 2 LEADER]: Becoming Leader. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:19:11.143477 17335 consensus_queue.cc:237] T 8638ee363273426db1393dd81e5c7b72 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:11.148121 16298 raft_consensus.cc:2468] T 8638ee363273426db1393dd81e5c7b72 P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 2.
I20260709 13:19:11.163087 17363 tablet_copy_client.cc:806] T 23f82224f4564fea93572e8206bad1fe P cb82cc7bd6db46d980c75237a1402860: tablet copy: Starting download of 0 data blocks...
I20260709 13:19:11.164126 17363 tablet_copy_client.cc:670] T 23f82224f4564fea93572e8206bad1fe P cb82cc7bd6db46d980c75237a1402860: tablet copy: Starting download of 1 WAL segments...
I20260709 13:19:11.168223 17363 tablet_copy_client.cc:538] T 23f82224f4564fea93572e8206bad1fe P cb82cc7bd6db46d980c75237a1402860: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:19:11.173059 16298 raft_consensus.cc:1275] T d2ab0ac8347641e6955e9bce7d90c1a2 P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:19:11.175130 17288 consensus_queue.cc:1048] T d2ab0ac8347641e6955e9bce7d90c1a2 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:19:11.184480 17363 tablet_bootstrap.cc:492] T 23f82224f4564fea93572e8206bad1fe P cb82cc7bd6db46d980c75237a1402860: Bootstrap starting.
I20260709 13:19:11.191140 16296 leader_election.cc:290] T 434bff3082d141838068f323bdb0d1ed P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 2 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:19:11.194008 17365 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c9944d5e4b8c4109af9955c4f08c9095. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:19:11.203553 16161 tablet_copy_source_session.cc:215] T 829b4ad8c5a54eababeea1181b960d4b P 647a343a542d4a139fe106c19095129b: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:19:11.202404 16048 catalog_manager.cc:5697] T 8638ee363273426db1393dd81e5c7b72 P cb82cc7bd6db46d980c75237a1402860 reported cstate change: term changed from 1 to 2, leader changed from 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130) to cb82cc7bd6db46d980c75237a1402860 (127.14.177.132). New cstate: current_term: 2 leader_uuid: "cb82cc7bd6db46d980c75237a1402860" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:19:11.205031 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434bff3082d141838068f323bdb0d1ed" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:19:11.205565 16226 raft_consensus.cc:3060] T 434bff3082d141838068f323bdb0d1ed P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:19:11.208284 17360 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 829b4ad8c5a54eababeea1181b960d4b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:19:11.195593 16147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434bff3082d141838068f323bdb0d1ed" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:19:11.211709 16147 raft_consensus.cc:3055] T 434bff3082d141838068f323bdb0d1ed P 647a343a542d4a139fe106c19095129b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:19:11.212019 16147 raft_consensus.cc:740] T 434bff3082d141838068f323bdb0d1ed P 647a343a542d4a139fe106c19095129b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:19:11.212898 16147 consensus_queue.cc:260] T 434bff3082d141838068f323bdb0d1ed P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:11.214177 16147 raft_consensus.cc:3060] T 434bff3082d141838068f323bdb0d1ed P 647a343a542d4a139fe106c19095129b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:19:11.262471 16226 raft_consensus.cc:2468] T 434bff3082d141838068f323bdb0d1ed P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 2.
I20260709 13:19:11.264247 16265 leader_election.cc:304] T 434bff3082d141838068f323bdb0d1ed P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:19:11.270040 17297 raft_consensus.cc:2804] T 434bff3082d141838068f323bdb0d1ed P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:19:11.272073 17297 raft_consensus.cc:697] T 434bff3082d141838068f323bdb0d1ed P 90f3da57796b488c9087a2f523adf045 [term 2 LEADER]: Becoming Leader. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:19:11.273300 17297 consensus_queue.cc:237] T 434bff3082d141838068f323bdb0d1ed P 90f3da57796b488c9087a2f523adf045 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:11.288039 16147 raft_consensus.cc:2468] T 434bff3082d141838068f323bdb0d1ed P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 2.
I20260709 13:19:11.308653 16821 raft_consensus.cc:1217] T af57996a8be84d398f0f023e8d08ef77 P cb82cc7bd6db46d980c75237a1402860 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:19:11.321412 17365 tablet_copy_client.cc:806] T c9944d5e4b8c4109af9955c4f08c9095 P cb82cc7bd6db46d980c75237a1402860: tablet copy: Starting download of 0 data blocks...
I20260709 13:19:11.322045 17365 tablet_copy_client.cc:670] T c9944d5e4b8c4109af9955c4f08c9095 P cb82cc7bd6db46d980c75237a1402860: tablet copy: Starting download of 1 WAL segments...
I20260709 13:19:11.335271 17365 tablet_copy_client.cc:538] T c9944d5e4b8c4109af9955c4f08c9095 P cb82cc7bd6db46d980c75237a1402860: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:19:11.342823 17360 tablet_copy_client.cc:806] T 829b4ad8c5a54eababeea1181b960d4b P cb82cc7bd6db46d980c75237a1402860: tablet copy: Starting download of 0 data blocks...
I20260709 13:19:11.345075 17360 tablet_copy_client.cc:670] T 829b4ad8c5a54eababeea1181b960d4b P cb82cc7bd6db46d980c75237a1402860: tablet copy: Starting download of 1 WAL segments...
I20260709 13:19:11.354449 17363 tablet_bootstrap.cc:492] T 23f82224f4564fea93572e8206bad1fe P cb82cc7bd6db46d980c75237a1402860: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:19:11.355566 17363 tablet_bootstrap.cc:492] T 23f82224f4564fea93572e8206bad1fe P cb82cc7bd6db46d980c75237a1402860: Bootstrap complete.
I20260709 13:19:11.355647 17360 tablet_copy_client.cc:538] T 829b4ad8c5a54eababeea1181b960d4b P cb82cc7bd6db46d980c75237a1402860: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:19:11.356318 17363 ts_tablet_manager.cc:1403] T 23f82224f4564fea93572e8206bad1fe P cb82cc7bd6db46d980c75237a1402860: Time spent bootstrapping tablet: real 0.172s	user 0.023s	sys 0.007s
I20260709 13:19:11.363097 17363 raft_consensus.cc:359] T 23f82224f4564fea93572e8206bad1fe P cb82cc7bd6db46d980c75237a1402860 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } }
I20260709 13:19:11.364051 17363 raft_consensus.cc:740] T 23f82224f4564fea93572e8206bad1fe P cb82cc7bd6db46d980c75237a1402860 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Initialized, Role: LEARNER
I20260709 13:19:11.365130 17360 tablet_bootstrap.cc:492] T 829b4ad8c5a54eababeea1181b960d4b P cb82cc7bd6db46d980c75237a1402860: Bootstrap starting.
I20260709 13:19:11.364768 17363 consensus_queue.cc:260] T 23f82224f4564fea93572e8206bad1fe P cb82cc7bd6db46d980c75237a1402860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } }
I20260709 13:19:11.359760 17365 tablet_bootstrap.cc:492] T c9944d5e4b8c4109af9955c4f08c9095 P cb82cc7bd6db46d980c75237a1402860: Bootstrap starting.
I20260709 13:19:11.358899 16044 catalog_manager.cc:5697] T 434bff3082d141838068f323bdb0d1ed P 90f3da57796b488c9087a2f523adf045 reported cstate change: term changed from 1 to 2, leader changed from 647a343a542d4a139fe106c19095129b (127.14.177.129) to 90f3da57796b488c9087a2f523adf045 (127.14.177.131). New cstate: current_term: 2 leader_uuid: "90f3da57796b488c9087a2f523adf045" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } health_report { overall_health: HEALTHY } } }
I20260709 13:19:11.379336 17363 ts_tablet_manager.cc:1434] T 23f82224f4564fea93572e8206bad1fe P cb82cc7bd6db46d980c75237a1402860: Time spent starting tablet: real 0.019s	user 0.007s	sys 0.000s
I20260709 13:19:11.381140 16161 tablet_copy_service.cc:342] P 647a343a542d4a139fe106c19095129b: Request end of tablet copy session cb82cc7bd6db46d980c75237a1402860-23f82224f4564fea93572e8206bad1fe received from {username='slave'} at 127.0.0.1:46224
I20260709 13:19:11.381582 16161 tablet_copy_service.cc:434] P 647a343a542d4a139fe106c19095129b: ending tablet copy session cb82cc7bd6db46d980c75237a1402860-23f82224f4564fea93572e8206bad1fe on tablet 23f82224f4564fea93572e8206bad1fe with peer cb82cc7bd6db46d980c75237a1402860
I20260709 13:19:11.401566 16145 raft_consensus.cc:1275] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:19:11.403054 17297 consensus_queue.cc:1048] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:19:11.410797 17365 tablet_bootstrap.cc:492] T c9944d5e4b8c4109af9955c4f08c9095 P cb82cc7bd6db46d980c75237a1402860: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:19:11.414179 17360 tablet_bootstrap.cc:492] T 829b4ad8c5a54eababeea1181b960d4b P cb82cc7bd6db46d980c75237a1402860: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:19:11.415292 17360 tablet_bootstrap.cc:492] T 829b4ad8c5a54eababeea1181b960d4b P cb82cc7bd6db46d980c75237a1402860: Bootstrap complete.
I20260709 13:19:11.415344 17365 tablet_bootstrap.cc:492] T c9944d5e4b8c4109af9955c4f08c9095 P cb82cc7bd6db46d980c75237a1402860: Bootstrap complete.
I20260709 13:19:11.416010 17360 ts_tablet_manager.cc:1403] T 829b4ad8c5a54eababeea1181b960d4b P cb82cc7bd6db46d980c75237a1402860: Time spent bootstrapping tablet: real 0.051s	user 0.020s	sys 0.002s
I20260709 13:19:11.416291 17365 ts_tablet_manager.cc:1403] T c9944d5e4b8c4109af9955c4f08c9095 P cb82cc7bd6db46d980c75237a1402860: Time spent bootstrapping tablet: real 0.057s	user 0.019s	sys 0.014s
I20260709 13:19:11.418906 16222 raft_consensus.cc:1275] T 19baa7bc367844c288bca8f81439c413 P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:19:11.419241 17360 raft_consensus.cc:359] T 829b4ad8c5a54eababeea1181b960d4b P cb82cc7bd6db46d980c75237a1402860 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } }
I20260709 13:19:11.420091 17360 raft_consensus.cc:740] T 829b4ad8c5a54eababeea1181b960d4b P cb82cc7bd6db46d980c75237a1402860 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Initialized, Role: LEARNER
I20260709 13:19:11.420776 17297 consensus_queue.cc:1048] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:19:11.420746 17360 consensus_queue.cc:260] T 829b4ad8c5a54eababeea1181b960d4b P cb82cc7bd6db46d980c75237a1402860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } }
I20260709 13:19:11.425004 17360 ts_tablet_manager.cc:1434] T 829b4ad8c5a54eababeea1181b960d4b P cb82cc7bd6db46d980c75237a1402860: Time spent starting tablet: real 0.009s	user 0.005s	sys 0.002s
I20260709 13:19:11.427423 17365 raft_consensus.cc:359] T c9944d5e4b8c4109af9955c4f08c9095 P cb82cc7bd6db46d980c75237a1402860 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } }
I20260709 13:19:11.430953 16161 tablet_copy_service.cc:342] P 647a343a542d4a139fe106c19095129b: Request end of tablet copy session cb82cc7bd6db46d980c75237a1402860-829b4ad8c5a54eababeea1181b960d4b received from {username='slave'} at 127.0.0.1:46224
I20260709 13:19:11.431474 16161 tablet_copy_service.cc:434] P 647a343a542d4a139fe106c19095129b: ending tablet copy session cb82cc7bd6db46d980c75237a1402860-829b4ad8c5a54eababeea1181b960d4b on tablet 829b4ad8c5a54eababeea1181b960d4b with peer cb82cc7bd6db46d980c75237a1402860
I20260709 13:19:11.430944 17365 raft_consensus.cc:740] T c9944d5e4b8c4109af9955c4f08c9095 P cb82cc7bd6db46d980c75237a1402860 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Initialized, Role: LEARNER
I20260709 13:19:11.435549 17365 consensus_queue.cc:260] T c9944d5e4b8c4109af9955c4f08c9095 P cb82cc7bd6db46d980c75237a1402860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } }
I20260709 13:19:11.438419 17365 ts_tablet_manager.cc:1434] T c9944d5e4b8c4109af9955c4f08c9095 P cb82cc7bd6db46d980c75237a1402860: Time spent starting tablet: real 0.021s	user 0.006s	sys 0.000s
I20260709 13:19:11.448565 16236 tablet_copy_service.cc:342] P 1ca2725d290e47d68b1a4b609c9816ac: Request end of tablet copy session cb82cc7bd6db46d980c75237a1402860-c9944d5e4b8c4109af9955c4f08c9095 received from {username='slave'} at 127.0.0.1:33050
I20260709 13:19:11.449086 16236 tablet_copy_service.cc:434] P 1ca2725d290e47d68b1a4b609c9816ac: ending tablet copy session cb82cc7bd6db46d980c75237a1402860-c9944d5e4b8c4109af9955c4f08c9095 on tablet c9944d5e4b8c4109af9955c4f08c9095 with peer cb82cc7bd6db46d980c75237a1402860
I20260709 13:19:11.474642 16821 raft_consensus.cc:1217] T c9944d5e4b8c4109af9955c4f08c9095 P cb82cc7bd6db46d980c75237a1402860 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:19:11.484498 15046 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:19:11.528604 16824 raft_consensus.cc:1217] T 829b4ad8c5a54eababeea1181b960d4b P cb82cc7bd6db46d980c75237a1402860 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:19:11.563643 15046 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:19:11.615579 16145 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "02838c86531d4f98bf58d1c22b8baf77"
dest_uuid: "647a343a542d4a139fe106c19095129b"
mode: GRACEFUL
new_leader_uuid: "90f3da57796b488c9087a2f523adf045"
 from {username='slave'} at 127.0.0.1:40290
I20260709 13:19:11.617166 16145 raft_consensus.cc:606] T 02838c86531d4f98bf58d1c22b8baf77 P 647a343a542d4a139fe106c19095129b [term 1 LEADER]: Received request to transfer leadership to TS 90f3da57796b488c9087a2f523adf045
I20260709 13:19:11.620128 16145 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "62fe3346cd3c42c4a55e14af9073bca4"
dest_uuid: "647a343a542d4a139fe106c19095129b"
mode: GRACEFUL
new_leader_uuid: "cb82cc7bd6db46d980c75237a1402860"
 from {username='slave'} at 127.0.0.1:40290
I20260709 13:19:11.620692 16145 raft_consensus.cc:606] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [term 1 LEADER]: Received request to transfer leadership to TS cb82cc7bd6db46d980c75237a1402860
I20260709 13:19:11.623893 16150 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "829b4ad8c5a54eababeea1181b960d4b"
dest_uuid: "647a343a542d4a139fe106c19095129b"
mode: GRACEFUL
new_leader_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
 from {username='slave'} at 127.0.0.1:40290
I20260709 13:19:11.624426 16150 raft_consensus.cc:606] T 829b4ad8c5a54eababeea1181b960d4b P 647a343a542d4a139fe106c19095129b [term 1 LEADER]: Received request to transfer leadership to TS 1ca2725d290e47d68b1a4b609c9816ac
I20260709 13:19:11.627487 16150 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8cc959fb7f704fba9009c0595467d074"
dest_uuid: "647a343a542d4a139fe106c19095129b"
mode: GRACEFUL
new_leader_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
 from {username='slave'} at 127.0.0.1:40290
I20260709 13:19:11.628068 16150 raft_consensus.cc:606] T 8cc959fb7f704fba9009c0595467d074 P 647a343a542d4a139fe106c19095129b [term 1 LEADER]: Received request to transfer leadership to TS 1ca2725d290e47d68b1a4b609c9816ac
I20260709 13:19:11.630475 16223 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "cdbd78acb4cf46298e0de8ba0d130e19"
dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
mode: GRACEFUL
new_leader_uuid: "cb82cc7bd6db46d980c75237a1402860"
 from {username='slave'} at 127.0.0.1:39144
I20260709 13:19:11.630959 16223 raft_consensus.cc:606] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [term 1 LEADER]: Received request to transfer leadership to TS cb82cc7bd6db46d980c75237a1402860
I20260709 13:19:11.632032 17305 raft_consensus.cc:993] T 829b4ad8c5a54eababeea1181b960d4b P 647a343a542d4a139fe106c19095129b: : Instructing follower 1ca2725d290e47d68b1a4b609c9816ac to start an election
I20260709 13:19:11.632045 15046 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:19:11.632532 17305 raft_consensus.cc:1081] T 829b4ad8c5a54eababeea1181b960d4b P 647a343a542d4a139fe106c19095129b [term 1 LEADER]: Signalling peer 1ca2725d290e47d68b1a4b609c9816ac to start an election
I20260709 13:19:11.633302 15046 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:19:11.634172 16223 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "829b4ad8c5a54eababeea1181b960d4b"
dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
 from {username='slave'} at 127.0.0.1:57604
I20260709 13:19:11.634553 16223 raft_consensus.cc:493] T 829b4ad8c5a54eababeea1181b960d4b P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:19:11.634905 16223 raft_consensus.cc:3060] T 829b4ad8c5a54eababeea1181b960d4b P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:19:11.639732 16223 raft_consensus.cc:515] T 829b4ad8c5a54eababeea1181b960d4b P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } }
I20260709 13:19:11.641561 16223 leader_election.cc:290] T 829b4ad8c5a54eababeea1181b960d4b P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 2 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:11.642635 16296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "829b4ad8c5a54eababeea1181b960d4b" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: true dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:19:11.642849 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "829b4ad8c5a54eababeea1181b960d4b" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: true dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:19:11.643234 16296 raft_consensus.cc:3060] T 829b4ad8c5a54eababeea1181b960d4b P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:19:11.643416 16150 raft_consensus.cc:3055] T 829b4ad8c5a54eababeea1181b960d4b P 647a343a542d4a139fe106c19095129b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:19:11.643778 16150 raft_consensus.cc:740] T 829b4ad8c5a54eababeea1181b960d4b P 647a343a542d4a139fe106c19095129b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:19:11.644476 16150 consensus_queue.cc:260] T 829b4ad8c5a54eababeea1181b960d4b P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } }
I20260709 13:19:11.645968 16150 raft_consensus.cc:3060] T 829b4ad8c5a54eababeea1181b960d4b P 647a343a542d4a139fe106c19095129b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:19:11.649981 16296 raft_consensus.cc:2468] T 829b4ad8c5a54eababeea1181b960d4b P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 2.
I20260709 13:19:11.651307 16190 leader_election.cc:304] T 829b4ad8c5a54eababeea1181b960d4b P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:19:11.652124 17340 raft_consensus.cc:2804] T 829b4ad8c5a54eababeea1181b960d4b P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:19:11.652189 16150 raft_consensus.cc:2468] T 829b4ad8c5a54eababeea1181b960d4b P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 2.
I20260709 13:19:11.653406 17340 raft_consensus.cc:697] T 829b4ad8c5a54eababeea1181b960d4b P 1ca2725d290e47d68b1a4b609c9816ac [term 2 LEADER]: Becoming Leader. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:19:11.654341 17340 consensus_queue.cc:237] T 829b4ad8c5a54eababeea1181b960d4b P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } }
I20260709 13:19:11.666954 16048 catalog_manager.cc:5697] T 829b4ad8c5a54eababeea1181b960d4b P 1ca2725d290e47d68b1a4b609c9816ac reported cstate change: term changed from 1 to 2, leader changed from 647a343a542d4a139fe106c19095129b (127.14.177.129) to 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130). New cstate: current_term: 2 leader_uuid: "1ca2725d290e47d68b1a4b609c9816ac" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:19:11.721186 16226 raft_consensus.cc:1275] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:19:11.722340 17317 consensus_queue.cc:1048] T 8638ee363273426db1393dd81e5c7b72 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:19:11.741498 16299 raft_consensus.cc:1275] T 8638ee363273426db1393dd81e5c7b72 P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:19:11.743134 17335 consensus_queue.cc:1048] T 8638ee363273426db1393dd81e5c7b72 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:19:11.788532 17349 raft_consensus.cc:993] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b: : Instructing follower cb82cc7bd6db46d980c75237a1402860 to start an election
I20260709 13:19:11.789067 17305 raft_consensus.cc:1081] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [term 1 LEADER]: Signalling peer cb82cc7bd6db46d980c75237a1402860 to start an election
I20260709 13:19:11.791242 16824 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "62fe3346cd3c42c4a55e14af9073bca4"
dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
 from {username='slave'} at 127.0.0.1:49896
I20260709 13:19:11.791745 16824 raft_consensus.cc:493] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:19:11.792053 16824 raft_consensus.cc:3060] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:19:11.798203 16824 raft_consensus.cc:515] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:11.804306 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62fe3346cd3c42c4a55e14af9073bca4" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:19:11.804867 16150 raft_consensus.cc:3055] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:19:11.805174 16150 raft_consensus.cc:740] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:19:11.805848 16150 consensus_queue.cc:260] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:11.806805 16150 raft_consensus.cc:3060] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:19:11.808991 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62fe3346cd3c42c4a55e14af9073bca4" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:19:11.809423 16824 leader_election.cc:290] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 2 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:19:11.809523 16226 raft_consensus.cc:3060] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:19:11.812163 16824 raft_consensus.cc:1217] T 23f82224f4564fea93572e8206bad1fe P cb82cc7bd6db46d980c75237a1402860 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:19:11.819854 16150 raft_consensus.cc:2468] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 2.
I20260709 13:19:11.822338 16790 leader_election.cc:304] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:19:11.823201 17317 raft_consensus.cc:2804] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:19:11.823694 17317 raft_consensus.cc:697] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [term 2 LEADER]: Becoming Leader. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:19:11.824393 17301 raft_consensus.cc:1064] T af57996a8be84d398f0f023e8d08ef77 P 1ca2725d290e47d68b1a4b609c9816ac: attempting to promote NON_VOTER cb82cc7bd6db46d980c75237a1402860 to VOTER
I20260709 13:19:11.824630 17317 consensus_queue.cc:237] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:11.826895 17301 consensus_queue.cc:237] T af57996a8be84d398f0f023e8d08ef77 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:11.836416 17292 raft_consensus.cc:1064] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac: attempting to promote NON_VOTER cb82cc7bd6db46d980c75237a1402860 to VOTER
I20260709 13:19:11.842844 16226 raft_consensus.cc:2468] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 2.
I20260709 13:19:11.840758 17292 consensus_queue.cc:237] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:11.851594 16821 raft_consensus.cc:1275] T af57996a8be84d398f0f023e8d08ef77 P cb82cc7bd6db46d980c75237a1402860 [term 1 LEARNER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:19:11.851702 16044 catalog_manager.cc:5697] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 reported cstate change: term changed from 1 to 2, leader changed from 647a343a542d4a139fe106c19095129b (127.14.177.129) to cb82cc7bd6db46d980c75237a1402860 (127.14.177.132). New cstate: current_term: 2 leader_uuid: "cb82cc7bd6db46d980c75237a1402860" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:19:11.855026 16296 raft_consensus.cc:1275] T af57996a8be84d398f0f023e8d08ef77 P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:19:11.856333 17207 consensus_queue.cc:1048] T af57996a8be84d398f0f023e8d08ef77 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:19:11.859906 17301 consensus_queue.cc:1048] T af57996a8be84d398f0f023e8d08ef77 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:19:11.865101 16145 raft_consensus.cc:1275] T af57996a8be84d398f0f023e8d08ef77 P 647a343a542d4a139fe106c19095129b [term 1 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:19:11.867822 17301 consensus_queue.cc:1048] T af57996a8be84d398f0f023e8d08ef77 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:19:11.886983 16226 raft_consensus.cc:1275] T 434bff3082d141838068f323bdb0d1ed P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:19:11.888437 16150 raft_consensus.cc:1275] T 434bff3082d141838068f323bdb0d1ed P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:19:11.888291 17207 raft_consensus.cc:2955] T af57996a8be84d398f0f023e8d08ef77 P 1ca2725d290e47d68b1a4b609c9816ac [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:19:11.890069 17322 consensus_queue.cc:1048] T 434bff3082d141838068f323bdb0d1ed P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:19:11.891671 17297 consensus_queue.cc:1048] T 434bff3082d141838068f323bdb0d1ed P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:19:11.894670 16821 raft_consensus.cc:2955] T af57996a8be84d398f0f023e8d08ef77 P cb82cc7bd6db46d980c75237a1402860 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
W20260709 13:19:11.965047 17395 log.cc:927] Time spent T af57996a8be84d398f0f023e8d08ef77 P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 0.079s	user 0.001s	sys 0.000s
I20260709 13:19:12.051898 17305 raft_consensus.cc:993] T 8cc959fb7f704fba9009c0595467d074 P 647a343a542d4a139fe106c19095129b: : Instructing follower 1ca2725d290e47d68b1a4b609c9816ac to start an election
I20260709 13:19:12.052690 17305 raft_consensus.cc:1081] T 8cc959fb7f704fba9009c0595467d074 P 647a343a542d4a139fe106c19095129b [term 1 LEADER]: Signalling peer 1ca2725d290e47d68b1a4b609c9816ac to start an election
I20260709 13:19:12.054812 16223 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "8cc959fb7f704fba9009c0595467d074"
dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
 from {username='slave'} at 127.0.0.1:57604
I20260709 13:19:12.055408 16223 raft_consensus.cc:493] T 8cc959fb7f704fba9009c0595467d074 P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:19:12.055781 16223 raft_consensus.cc:3060] T 8cc959fb7f704fba9009c0595467d074 P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:19:12.080441 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 151788us
I20260709 13:19:12.152544 16145 raft_consensus.cc:2955] T af57996a8be84d398f0f023e8d08ef77 P 647a343a542d4a139fe106c19095129b [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:19:12.155786 16296 raft_consensus.cc:2955] T af57996a8be84d398f0f023e8d08ef77 P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:19:12.162576 16223 raft_consensus.cc:515] T 8cc959fb7f704fba9009c0595467d074 P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:12.165524 16223 leader_election.cc:290] T 8cc959fb7f704fba9009c0595467d074 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 2 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:12.172130 16044 catalog_manager.cc:5697] T af57996a8be84d398f0f023e8d08ef77 P cb82cc7bd6db46d980c75237a1402860 reported cstate change: config changed from index 2 to 3, cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "1ca2725d290e47d68b1a4b609c9816ac" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:19:12.198263 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cc959fb7f704fba9009c0595467d074" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:19:12.199136 16299 raft_consensus.cc:3060] T 8cc959fb7f704fba9009c0595467d074 P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:19:12.209395 16299 raft_consensus.cc:2468] T 8cc959fb7f704fba9009c0595467d074 P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 2.
I20260709 13:19:12.237274 16190 leader_election.cc:304] T 8cc959fb7f704fba9009c0595467d074 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:19:12.238867 17398 raft_consensus.cc:2804] T 8cc959fb7f704fba9009c0595467d074 P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:19:12.239384 17398 raft_consensus.cc:697] T 8cc959fb7f704fba9009c0595467d074 P 1ca2725d290e47d68b1a4b609c9816ac [term 2 LEADER]: Becoming Leader. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:19:12.240223 17398 consensus_queue.cc:237] T 8cc959fb7f704fba9009c0595467d074 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:12.265184 16145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cc959fb7f704fba9009c0595467d074" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:19:12.266040 16145 raft_consensus.cc:3055] T 8cc959fb7f704fba9009c0595467d074 P 647a343a542d4a139fe106c19095129b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:19:12.266434 16145 raft_consensus.cc:740] T 8cc959fb7f704fba9009c0595467d074 P 647a343a542d4a139fe106c19095129b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:19:12.280134 16299 raft_consensus.cc:1240] T 8cc959fb7f704fba9009c0595467d074 P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 1. Current term is 2. Ops: []
I20260709 13:19:12.282078 16298 raft_consensus.cc:1275] T c9944d5e4b8c4109af9955c4f08c9095 P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:19:12.283435 17207 consensus_queue.cc:1048] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:19:12.285356 17352 raft_consensus.cc:993] T 02838c86531d4f98bf58d1c22b8baf77 P 647a343a542d4a139fe106c19095129b: : Instructing follower 90f3da57796b488c9087a2f523adf045 to start an election
I20260709 13:19:12.285835 17352 raft_consensus.cc:1081] T 02838c86531d4f98bf58d1c22b8baf77 P 647a343a542d4a139fe106c19095129b [term 1 LEADER]: Signalling peer 90f3da57796b488c9087a2f523adf045 to start an election
I20260709 13:19:12.294947 16144 raft_consensus.cc:1275] T c9944d5e4b8c4109af9955c4f08c9095 P 647a343a542d4a139fe106c19095129b [term 1 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
W20260709 13:19:12.312414 16192 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 382018us
I20260709 13:19:12.313968 16145 consensus_queue.cc:260] T 8cc959fb7f704fba9009c0595467d074 P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:12.314656 16824 raft_consensus.cc:1275] T c9944d5e4b8c4109af9955c4f08c9095 P cb82cc7bd6db46d980c75237a1402860 [term 1 LEARNER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:19:12.315094 16145 raft_consensus.cc:3060] T 8cc959fb7f704fba9009c0595467d074 P 647a343a542d4a139fe106c19095129b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:19:12.345839 16295 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "02838c86531d4f98bf58d1c22b8baf77"
dest_uuid: "90f3da57796b488c9087a2f523adf045"
 from {username='slave'} at 127.0.0.1:48674
I20260709 13:19:12.346452 16295 raft_consensus.cc:493] T 02838c86531d4f98bf58d1c22b8baf77 P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:19:12.346796 16295 raft_consensus.cc:3060] T 02838c86531d4f98bf58d1c22b8baf77 P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:19:12.348217 17207 consensus_queue.cc:1048] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:19:12.369144 16145 raft_consensus.cc:2468] T 8cc959fb7f704fba9009c0595467d074 P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 2.
I20260709 13:19:12.372944 16821 raft_consensus.cc:3060] T 829b4ad8c5a54eababeea1181b960d4b P cb82cc7bd6db46d980c75237a1402860 [term 1 LEARNER]: Advancing to term 2
I20260709 13:19:12.388376 16821 raft_consensus.cc:1275] T 829b4ad8c5a54eababeea1181b960d4b P cb82cc7bd6db46d980c75237a1402860 [term 2 LEARNER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 13:19:12.394678 16299 raft_consensus.cc:1275] T 829b4ad8c5a54eababeea1181b960d4b P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 13:19:12.385344 16295 raft_consensus.cc:515] T 02838c86531d4f98bf58d1c22b8baf77 P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:12.411073 17399 consensus_queue.cc:1048] T 829b4ad8c5a54eababeea1181b960d4b P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:19:12.416487 16226 raft_consensus.cc:1275] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:19:12.429296 16295 leader_election.cc:290] T 02838c86531d4f98bf58d1c22b8baf77 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 2 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:19:12.431252 17387 consensus_queue.cc:1048] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
W20260709 13:19:12.448089 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 69931.9us
W20260709 13:19:12.461741 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 131115us
W20260709 13:19:12.484267 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 81649.9us
W20260709 13:19:12.486397 16117 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 110658us
I20260709 13:19:12.492733 16151 raft_consensus.cc:1275] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:19:12.498082 17352 raft_consensus.cc:1064] T 23f82224f4564fea93572e8206bad1fe P 647a343a542d4a139fe106c19095129b: attempting to promote NON_VOTER cb82cc7bd6db46d980c75237a1402860 to VOTER
I20260709 13:19:12.500669 17352 consensus_queue.cc:237] T 23f82224f4564fea93572e8206bad1fe P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:12.517647 17335 consensus_queue.cc:1048] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
W20260709 13:19:12.525305 16115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 148851us
I20260709 13:19:12.529644 17301 raft_consensus.cc:2955] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:19:12.538455 16226 raft_consensus.cc:1275] T 23f82224f4564fea93572e8206bad1fe P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:19:12.540319 17401 consensus_queue.cc:1048] T 23f82224f4564fea93572e8206bad1fe P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:19:12.541162 16145 raft_consensus.cc:1275] T 829b4ad8c5a54eababeea1181b960d4b P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 13:19:12.548962 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02838c86531d4f98bf58d1c22b8baf77" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:19:12.549763 16224 raft_consensus.cc:3060] T 02838c86531d4f98bf58d1c22b8baf77 P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:19:12.565972 16147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02838c86531d4f98bf58d1c22b8baf77" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:19:12.566637 16147 raft_consensus.cc:3055] T 02838c86531d4f98bf58d1c22b8baf77 P 647a343a542d4a139fe106c19095129b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:19:12.566967 16147 raft_consensus.cc:740] T 02838c86531d4f98bf58d1c22b8baf77 P 647a343a542d4a139fe106c19095129b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:19:12.567823 16147 consensus_queue.cc:260] T 02838c86531d4f98bf58d1c22b8baf77 P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:12.569274 16147 raft_consensus.cc:3060] T 02838c86531d4f98bf58d1c22b8baf77 P 647a343a542d4a139fe106c19095129b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:19:12.570065 16145 raft_consensus.cc:2955] T c9944d5e4b8c4109af9955c4f08c9095 P 647a343a542d4a139fe106c19095129b [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:19:12.579226 16147 raft_consensus.cc:2468] T 02838c86531d4f98bf58d1c22b8baf77 P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 2.
I20260709 13:19:12.580832 17399 consensus_queue.cc:1048] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:19:12.590072 16224 raft_consensus.cc:2468] T 02838c86531d4f98bf58d1c22b8baf77 P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 2.
I20260709 13:19:12.591223 16300 raft_consensus.cc:2955] T c9944d5e4b8c4109af9955c4f08c9095 P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:19:12.592247 17399 consensus_queue.cc:1048] T 829b4ad8c5a54eababeea1181b960d4b P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:19:12.602164 16296 raft_consensus.cc:1240] T 02838c86531d4f98bf58d1c22b8baf77 P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 1. Current term is 2. Ops: []
I20260709 13:19:12.602504 16299 raft_consensus.cc:1275] T 23f82224f4564fea93572e8206bad1fe P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:19:12.604301 17207 consensus_queue.cc:1048] T 829b4ad8c5a54eababeea1181b960d4b P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:19:12.609681 17352 consensus_queue.cc:1048] T 23f82224f4564fea93572e8206bad1fe P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:19:12.632138 16045 catalog_manager.cc:5697] T c9944d5e4b8c4109af9955c4f08c9095 P 647a343a542d4a139fe106c19095129b reported cstate change: config changed from index 2 to 3, cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "1ca2725d290e47d68b1a4b609c9816ac" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:19:12.635494 16265 leader_election.cc:304] T 02838c86531d4f98bf58d1c22b8baf77 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:19:12.638240 15046 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:19:12.639760 16823 raft_consensus.cc:1275] T 23f82224f4564fea93572e8206bad1fe P cb82cc7bd6db46d980c75237a1402860 [term 1 LEARNER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:19:12.641687 17408 raft_consensus.cc:2804] T 02838c86531d4f98bf58d1c22b8baf77 P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:19:12.642198 17408 raft_consensus.cc:697] T 02838c86531d4f98bf58d1c22b8baf77 P 90f3da57796b488c9087a2f523adf045 [term 2 LEADER]: Becoming Leader. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:19:12.644623 17408 consensus_queue.cc:237] T 02838c86531d4f98bf58d1c22b8baf77 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:12.652654 17399 raft_consensus.cc:993] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac: : Instructing follower cb82cc7bd6db46d980c75237a1402860 to start an election
I20260709 13:19:12.653143 17399 raft_consensus.cc:1081] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [term 1 LEADER]: Signalling peer cb82cc7bd6db46d980c75237a1402860 to start an election
I20260709 13:19:12.666730 17399 raft_consensus.cc:1064] T 829b4ad8c5a54eababeea1181b960d4b P 1ca2725d290e47d68b1a4b609c9816ac: attempting to promote NON_VOTER cb82cc7bd6db46d980c75237a1402860 to VOTER
I20260709 13:19:12.672477 17399 consensus_queue.cc:237] T 829b4ad8c5a54eababeea1181b960d4b P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 2, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:19:12.682413 16115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 75422.5us
I20260709 13:19:12.683425 16821 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "cdbd78acb4cf46298e0de8ba0d130e19"
dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
 from {username='slave'} at 127.0.0.1:49906
I20260709 13:19:12.684012 16821 raft_consensus.cc:493] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:19:12.684406 16821 raft_consensus.cc:3060] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:19:12.688263 16823 raft_consensus.cc:2955] T c9944d5e4b8c4109af9955c4f08c9095 P cb82cc7bd6db46d980c75237a1402860 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:19:12.714267 16824 raft_consensus.cc:1275] T 829b4ad8c5a54eababeea1181b960d4b P cb82cc7bd6db46d980c75237a1402860 [term 2 LEARNER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 13:19:12.718797 17414 consensus_queue.cc:1048] T 23f82224f4564fea93572e8206bad1fe P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:19:12.719129 16299 raft_consensus.cc:1275] T 829b4ad8c5a54eababeea1181b960d4b P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 13:19:12.734472 17207 consensus_queue.cc:1048] T 829b4ad8c5a54eababeea1181b960d4b P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:19:12.740377 17207 consensus_queue.cc:1048] T 829b4ad8c5a54eababeea1181b960d4b P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:19:12.746454 16821 raft_consensus.cc:515] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:12.754961 16222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdbd78acb4cf46298e0de8ba0d130e19" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:19:12.755837 16222 raft_consensus.cc:3055] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:19:12.756224 16222 raft_consensus.cc:740] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:19:12.757162 16222 consensus_queue.cc:260] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:12.758574 16222 raft_consensus.cc:3060] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:19:12.769999 16821 leader_election.cc:290] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 2 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:12.800225 16299 raft_consensus.cc:1275] T 8cc959fb7f704fba9009c0595467d074 P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:19:12.804363 17207 consensus_queue.cc:1048] T 8cc959fb7f704fba9009c0595467d074 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:19:12.789000 16222 raft_consensus.cc:2468] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 2.
I20260709 13:19:12.812004 16790 leader_election.cc:304] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 
W20260709 13:19:12.811053 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 58327.5us
I20260709 13:19:12.824453 17387 raft_consensus.cc:2804] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:19:12.833704 17387 raft_consensus.cc:697] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 [term 2 LEADER]: Becoming Leader. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:19:12.841516 17387 consensus_queue.cc:237] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:12.842414 16048 catalog_manager.cc:5697] T 8cc959fb7f704fba9009c0595467d074 P 1ca2725d290e47d68b1a4b609c9816ac reported cstate change: term changed from 1 to 2, leader changed from 647a343a542d4a139fe106c19095129b (127.14.177.129) to 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130). New cstate: current_term: 2 leader_uuid: "1ca2725d290e47d68b1a4b609c9816ac" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } health_report { overall_health: UNKNOWN } } }
I20260709 13:19:12.848315 17412 raft_consensus.cc:2955] T 829b4ad8c5a54eababeea1181b960d4b P 1ca2725d290e47d68b1a4b609c9816ac [term 2 LEADER]: Committing config change with OpId 2.4: config changed from index 2 to 4, cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:19:12.853283 16150 raft_consensus.cc:1275] T 8cc959fb7f704fba9009c0595467d074 P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:19:12.854718 17207 consensus_queue.cc:1048] T 8cc959fb7f704fba9009c0595467d074 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:19:12.831238 17389 raft_consensus.cc:2955] T 23f82224f4564fea93572e8206bad1fe P 647a343a542d4a139fe106c19095129b [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:19:12.865602 16824 raft_consensus.cc:2955] T 829b4ad8c5a54eababeea1181b960d4b P cb82cc7bd6db46d980c75237a1402860 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 2 to 4, cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
W20260709 13:19:12.876451 16789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 126052us
I20260709 13:19:12.880456 16222 consensus_queue.cc:237] T af57996a8be84d398f0f023e8d08ef77 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:12.889407 16145 raft_consensus.cc:1275] T 829b4ad8c5a54eababeea1181b960d4b P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 13:19:12.897348 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdbd78acb4cf46298e0de8ba0d130e19" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:19:12.897995 16300 raft_consensus.cc:3060] T cdbd78acb4cf46298e0de8ba0d130e19 P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:19:12.902549 16825 raft_consensus.cc:1275] T af57996a8be84d398f0f023e8d08ef77 P cb82cc7bd6db46d980c75237a1402860 [term 1 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:19:12.904268 17411 consensus_queue.cc:1048] T af57996a8be84d398f0f023e8d08ef77 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:19:12.912830 17207 consensus_queue.cc:1048] T 829b4ad8c5a54eababeea1181b960d4b P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:19:12.914914 16298 raft_consensus.cc:2955] T 23f82224f4564fea93572e8206bad1fe P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:19:12.925310 17207 raft_consensus.cc:2955] T af57996a8be84d398f0f023e8d08ef77 P 1ca2725d290e47d68b1a4b609c9816ac [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 90f3da57796b488c9087a2f523adf045 (127.14.177.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:19:12.928790 16825 raft_consensus.cc:2955] T af57996a8be84d398f0f023e8d08ef77 P cb82cc7bd6db46d980c75237a1402860 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 90f3da57796b488c9087a2f523adf045 (127.14.177.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:19:12.930207 16223 raft_consensus.cc:2955] T 23f82224f4564fea93572e8206bad1fe P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:19:12.931469 16150 raft_consensus.cc:1275] T af57996a8be84d398f0f023e8d08ef77 P 647a343a542d4a139fe106c19095129b [term 1 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:19:12.938014 16150 raft_consensus.cc:2955] T 829b4ad8c5a54eababeea1181b960d4b P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 2 to 4, cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:19:12.947702 16035 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet af57996a8be84d398f0f023e8d08ef77 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:19:12.952056 16295 raft_consensus.cc:2955] T 829b4ad8c5a54eababeea1181b960d4b P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 2 to 4, cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:19:12.953457 16300 raft_consensus.cc:2468] T cdbd78acb4cf46298e0de8ba0d130e19 P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 2.
I20260709 13:19:12.958367 17399 consensus_queue.cc:1048] T af57996a8be84d398f0f023e8d08ef77 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:19:12.975360 16144 raft_consensus.cc:2955] T af57996a8be84d398f0f023e8d08ef77 P 647a343a542d4a139fe106c19095129b [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 90f3da57796b488c9087a2f523adf045 (127.14.177.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:19:12.978782 16824 raft_consensus.cc:2955] T 23f82224f4564fea93572e8206bad1fe P cb82cc7bd6db46d980c75237a1402860 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:19:13.000164 16044 catalog_manager.cc:5697] T 23f82224f4564fea93572e8206bad1fe P 647a343a542d4a139fe106c19095129b reported cstate change: config changed from index 2 to 3, cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "647a343a542d4a139fe106c19095129b" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:19:13.030501 16145 consensus_queue.cc:237] T 23f82224f4564fea93572e8206bad1fe P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:13.036643 16820 raft_consensus.cc:1275] T 23f82224f4564fea93572e8206bad1fe P cb82cc7bd6db46d980c75237a1402860 [term 1 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:19:13.037590 16299 raft_consensus.cc:1275] T 23f82224f4564fea93572e8206bad1fe P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:19:13.039156 17389 consensus_queue.cc:1048] T 23f82224f4564fea93572e8206bad1fe P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:19:13.042640 17389 consensus_queue.cc:1048] T 23f82224f4564fea93572e8206bad1fe P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:19:13.048393 17349 raft_consensus.cc:2955] T 23f82224f4564fea93572e8206bad1fe P 647a343a542d4a139fe106c19095129b [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:19:13.053797 16047 catalog_manager.cc:5697] T 829b4ad8c5a54eababeea1181b960d4b P 1ca2725d290e47d68b1a4b609c9816ac reported cstate change: config changed from index 2 to 4, cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) changed from NON_VOTER to VOTER. New cstate: current_term: 2 leader_uuid: "1ca2725d290e47d68b1a4b609c9816ac" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:19:13.061595 16298 raft_consensus.cc:2955] T 23f82224f4564fea93572e8206bad1fe P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:19:13.070087 16045 catalog_manager.cc:5697] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 reported cstate change: term changed from 1 to 2, leader changed from 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130) to cb82cc7bd6db46d980c75237a1402860 (127.14.177.132). New cstate: current_term: 2 leader_uuid: "cb82cc7bd6db46d980c75237a1402860" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:19:13.071733 16824 raft_consensus.cc:2955] T 23f82224f4564fea93572e8206bad1fe P cb82cc7bd6db46d980c75237a1402860 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:19:13.078559 16035 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 23f82224f4564fea93572e8206bad1fe with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:19:13.089620 16048 catalog_manager.cc:5697] T 02838c86531d4f98bf58d1c22b8baf77 P 90f3da57796b488c9087a2f523adf045 reported cstate change: term changed from 1 to 2, leader changed from 647a343a542d4a139fe106c19095129b (127.14.177.129) to 90f3da57796b488c9087a2f523adf045 (127.14.177.131). New cstate: current_term: 2 leader_uuid: "90f3da57796b488c9087a2f523adf045" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } health_report { overall_health: HEALTHY } } }
I20260709 13:19:13.101861 16144 raft_consensus.cc:1275] T 02838c86531d4f98bf58d1c22b8baf77 P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:19:13.108606 17433 consensus_queue.cc:1048] T 02838c86531d4f98bf58d1c22b8baf77 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:19:13.112414 16044 catalog_manager.cc:5697] T 23f82224f4564fea93572e8206bad1fe P 647a343a542d4a139fe106c19095129b reported cstate change: config changed from index 3 to 4, VOTER 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130) evicted. New cstate: current_term: 1 leader_uuid: "647a343a542d4a139fe106c19095129b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:19:13.132608 16044 catalog_manager.cc:5697] T af57996a8be84d398f0f023e8d08ef77 P 647a343a542d4a139fe106c19095129b reported cstate change: config changed from index 3 to 4, VOTER 90f3da57796b488c9087a2f523adf045 (127.14.177.131) evicted. New cstate: current_term: 1 leader_uuid: "1ca2725d290e47d68b1a4b609c9816ac" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:19:13.135298 16223 raft_consensus.cc:1275] T 02838c86531d4f98bf58d1c22b8baf77 P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:19:13.137357 17406 consensus_queue.cc:1048] T 02838c86531d4f98bf58d1c22b8baf77 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:19:13.156672 16209 tablet_service.cc:1558] Processing DeleteTablet for tablet 23f82224f4564fea93572e8206bad1fe with delete_type TABLET_DATA_TOMBSTONED (TS 1ca2725d290e47d68b1a4b609c9816ac not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:57598
I20260709 13:19:13.158326 16286 tablet_service.cc:1558] Processing DeleteTablet for tablet af57996a8be84d398f0f023e8d08ef77 with delete_type TABLET_DATA_TOMBSTONED (TS 90f3da57796b488c9087a2f523adf045 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:48670
I20260709 13:19:13.163297 15046 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:19:13.173743 17442 tablet_replica.cc:333] T 23f82224f4564fea93572e8206bad1fe P 1ca2725d290e47d68b1a4b609c9816ac: stopping tablet replica
I20260709 13:19:13.174911 17442 raft_consensus.cc:2243] T 23f82224f4564fea93572e8206bad1fe P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:19:13.175814 17442 raft_consensus.cc:2272] T 23f82224f4564fea93572e8206bad1fe P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:19:13.192298 17442 ts_tablet_manager.cc:1916] T 23f82224f4564fea93572e8206bad1fe P 1ca2725d290e47d68b1a4b609c9816ac: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:19:13.200289 16224 consensus_queue.cc:237] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:13.214774 16824 raft_consensus.cc:1275] T c9944d5e4b8c4109af9955c4f08c9095 P cb82cc7bd6db46d980c75237a1402860 [term 1 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:19:13.216773 17207 consensus_queue.cc:1048] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:19:13.217906 17443 tablet_replica.cc:333] T af57996a8be84d398f0f023e8d08ef77 P 90f3da57796b488c9087a2f523adf045: stopping tablet replica
I20260709 13:19:13.218273 16298 raft_consensus.cc:1275] T c9944d5e4b8c4109af9955c4f08c9095 P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:19:13.219076 17443 raft_consensus.cc:2243] T af57996a8be84d398f0f023e8d08ef77 P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:19:13.219663 17207 consensus_queue.cc:1048] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:19:13.219856 17443 raft_consensus.cc:2272] T af57996a8be84d398f0f023e8d08ef77 P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:19:13.224946 17443 ts_tablet_manager.cc:1916] T af57996a8be84d398f0f023e8d08ef77 P 90f3da57796b488c9087a2f523adf045: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:19:13.228586 17412 raft_consensus.cc:2955] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 647a343a542d4a139fe106c19095129b (127.14.177.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:19:13.239351 16298 raft_consensus.cc:2955] T c9944d5e4b8c4109af9955c4f08c9095 P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 647a343a542d4a139fe106c19095129b (127.14.177.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:19:13.235451 16824 raft_consensus.cc:2955] T c9944d5e4b8c4109af9955c4f08c9095 P cb82cc7bd6db46d980c75237a1402860 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 647a343a542d4a139fe106c19095129b (127.14.177.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:19:13.253019 17442 ts_tablet_manager.cc:1929] T 23f82224f4564fea93572e8206bad1fe P 1ca2725d290e47d68b1a4b609c9816ac: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:19:13.253489 17442 log.cc:1199] T 23f82224f4564fea93572e8206bad1fe P 1ca2725d290e47d68b1a4b609c9816ac: Deleting WAL directory at /tmp/dist-test-taskdH_Ma7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783603093801613-15046-0/minicluster-data/ts-1-root/wals/23f82224f4564fea93572e8206bad1fe
I20260709 13:19:13.254980 16035 catalog_manager.cc:5028] TS 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943): tablet 23f82224f4564fea93572e8206bad1fe (table test-workload [id=ed3d15d8c9ad46bebc4fe3a402d2e54f]) successfully deleted
I20260709 13:19:13.256840 17443 ts_tablet_manager.cc:1929] T af57996a8be84d398f0f023e8d08ef77 P 90f3da57796b488c9087a2f523adf045: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:19:13.257441 17443 log.cc:1199] T af57996a8be84d398f0f023e8d08ef77 P 90f3da57796b488c9087a2f523adf045: Deleting WAL directory at /tmp/dist-test-taskdH_Ma7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783603093801613-15046-0/minicluster-data/ts-2-root/wals/af57996a8be84d398f0f023e8d08ef77
I20260709 13:19:13.261170 16035 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet c9944d5e4b8c4109af9955c4f08c9095 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:19:13.263933 16034 catalog_manager.cc:5028] TS 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389): tablet af57996a8be84d398f0f023e8d08ef77 (table test-workload [id=ed3d15d8c9ad46bebc4fe3a402d2e54f]) successfully deleted
I20260709 13:19:13.270421 16046 catalog_manager.cc:5697] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac reported cstate change: config changed from index 3 to 4, VOTER 647a343a542d4a139fe106c19095129b (127.14.177.129) evicted. New cstate: current_term: 1 leader_uuid: "1ca2725d290e47d68b1a4b609c9816ac" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:19:13.298897 16136 tablet_service.cc:1558] Processing DeleteTablet for tablet c9944d5e4b8c4109af9955c4f08c9095 with delete_type TABLET_DATA_TOMBSTONED (TS 647a343a542d4a139fe106c19095129b not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:40822
I20260709 13:19:13.315963 16145 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "08b7d1ab519044eda2d5e04a07242f25"
dest_uuid: "647a343a542d4a139fe106c19095129b"
mode: GRACEFUL
new_leader_uuid: "cb82cc7bd6db46d980c75237a1402860"
 from {username='slave'} at 127.0.0.1:40290
I20260709 13:19:13.316593 16145 raft_consensus.cc:606] T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b [term 1 LEADER]: Received request to transfer leadership to TS cb82cc7bd6db46d980c75237a1402860
I20260709 13:19:13.319795 17444 tablet_replica.cc:333] T c9944d5e4b8c4109af9955c4f08c9095 P 647a343a542d4a139fe106c19095129b: stopping tablet replica
I20260709 13:19:13.326761 16223 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "08c74490c40243a88e62d1760b5cd814"
dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
mode: GRACEFUL
new_leader_uuid: "90f3da57796b488c9087a2f523adf045"
 from {username='slave'} at 127.0.0.1:39144
I20260709 13:19:13.327409 16223 raft_consensus.cc:606] T 08c74490c40243a88e62d1760b5cd814 P 1ca2725d290e47d68b1a4b609c9816ac [term 1 LEADER]: Received request to transfer leadership to TS 90f3da57796b488c9087a2f523adf045
I20260709 13:19:13.330036 16150 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "18ad7e3af5504571a69d10c00750a01c"
dest_uuid: "647a343a542d4a139fe106c19095129b"
mode: GRACEFUL
new_leader_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
 from {username='slave'} at 127.0.0.1:40290
I20260709 13:19:13.330627 16150 raft_consensus.cc:606] T 18ad7e3af5504571a69d10c00750a01c P 647a343a542d4a139fe106c19095129b [term 1 LEADER]: Received request to transfer leadership to TS 1ca2725d290e47d68b1a4b609c9816ac
I20260709 13:19:13.333174 16150 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c0eab25ce4c64e018a041beeda2867db"
dest_uuid: "647a343a542d4a139fe106c19095129b"
mode: GRACEFUL
new_leader_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
 from {username='slave'} at 127.0.0.1:40290
I20260709 13:19:13.333678 16150 raft_consensus.cc:606] T c0eab25ce4c64e018a041beeda2867db P 647a343a542d4a139fe106c19095129b [term 1 LEADER]: Received request to transfer leadership to TS 1ca2725d290e47d68b1a4b609c9816ac
I20260709 13:19:13.337363 16224 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d2372981006d4b218215c28964dea623"
dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
mode: GRACEFUL
new_leader_uuid: "cb82cc7bd6db46d980c75237a1402860"
 from {username='slave'} at 127.0.0.1:39144
I20260709 13:19:13.337945 16224 raft_consensus.cc:606] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [term 1 LEADER]: Received request to transfer leadership to TS cb82cc7bd6db46d980c75237a1402860
I20260709 13:19:13.346268 16224 raft_consensus.cc:1275] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:19:13.348042 17387 consensus_queue.cc:1048] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:19:13.349323 17444 raft_consensus.cc:2243] T c9944d5e4b8c4109af9955c4f08c9095 P 647a343a542d4a139fe106c19095129b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:19:13.350241 17444 raft_consensus.cc:2272] T c9944d5e4b8c4109af9955c4f08c9095 P 647a343a542d4a139fe106c19095129b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:19:13.352756 17207 raft_consensus.cc:993] T 08c74490c40243a88e62d1760b5cd814 P 1ca2725d290e47d68b1a4b609c9816ac: : Instructing follower 90f3da57796b488c9087a2f523adf045 to start an election
I20260709 13:19:13.353328 17207 raft_consensus.cc:1081] T 08c74490c40243a88e62d1760b5cd814 P 1ca2725d290e47d68b1a4b609c9816ac [term 1 LEADER]: Signalling peer 90f3da57796b488c9087a2f523adf045 to start an election
I20260709 13:19:13.357003 16298 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "08c74490c40243a88e62d1760b5cd814"
dest_uuid: "90f3da57796b488c9087a2f523adf045"
 from {username='slave'} at 127.0.0.1:48684
I20260709 13:19:13.357612 16298 raft_consensus.cc:493] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:19:13.357972 16298 raft_consensus.cc:3060] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:19:13.356236 17444 ts_tablet_manager.cc:1916] T c9944d5e4b8c4109af9955c4f08c9095 P 647a343a542d4a139fe106c19095129b: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
W20260709 13:19:13.364033 16846 debug-util.cc:398] Leaking SignalData structure 0x7b08000bce60 after lost signal to thread 15049
I20260709 13:19:13.374254 16298 raft_consensus.cc:515] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:19:13.375581 16846 debug-util.cc:398] Leaking SignalData structure 0x7b08002d7e40 after lost signal to thread 16083
W20260709 13:19:13.376763 16846 debug-util.cc:398] Leaking SignalData structure 0x7b0800213560 after lost signal to thread 16175
I20260709 13:19:13.377696 16298 leader_election.cc:290] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 2 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
W20260709 13:19:13.377837 16846 debug-util.cc:398] Leaking SignalData structure 0x7b08001c0340 after lost signal to thread 16250
I20260709 13:19:13.378710 16223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c74490c40243a88e62d1760b5cd814" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:19:13.379308 16223 raft_consensus.cc:3055] T 08c74490c40243a88e62d1760b5cd814 P 1ca2725d290e47d68b1a4b609c9816ac [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:19:13.379602 16223 raft_consensus.cc:740] T 08c74490c40243a88e62d1760b5cd814 P 1ca2725d290e47d68b1a4b609c9816ac [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:19:13.380417 16223 consensus_queue.cc:260] T 08c74490c40243a88e62d1760b5cd814 P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:13.381470 16223 raft_consensus.cc:3060] T 08c74490c40243a88e62d1760b5cd814 P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:19:13.384106 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c74490c40243a88e62d1760b5cd814" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:19:13.384708 16150 raft_consensus.cc:3060] T 08c74490c40243a88e62d1760b5cd814 P 647a343a542d4a139fe106c19095129b [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:19:13.386222 16846 debug-util.cc:398] Leaking SignalData structure 0x7b0800330900 after lost signal to thread 16849
I20260709 13:19:13.414165 16223 raft_consensus.cc:2468] T 08c74490c40243a88e62d1760b5cd814 P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 2.
I20260709 13:19:13.429522 16265 leader_election.cc:304] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:19:13.430482 17406 raft_consensus.cc:2804] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:19:13.431002 17406 raft_consensus.cc:697] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045 [term 2 LEADER]: Becoming Leader. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:19:13.431857 17406 consensus_queue.cc:237] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:13.420256 16150 raft_consensus.cc:2468] T 08c74490c40243a88e62d1760b5cd814 P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 2.
I20260709 13:19:13.421420 16300 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e4dc6b0541ca431ebc757beac00521f1"
dest_uuid: "90f3da57796b488c9087a2f523adf045"
mode: GRACEFUL
new_leader_uuid: "647a343a542d4a139fe106c19095129b"
 from {username='slave'} at 127.0.0.1:43334
I20260709 13:19:13.438530 16300 raft_consensus.cc:606] T e4dc6b0541ca431ebc757beac00521f1 P 90f3da57796b488c9087a2f523adf045 [term 2 LEADER]: Received request to transfer leadership to TS 647a343a542d4a139fe106c19095129b
I20260709 13:19:13.441365 15046 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 13:19:13.442975 15046 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:19:13.452598 17444 ts_tablet_manager.cc:1929] T c9944d5e4b8c4109af9955c4f08c9095 P 647a343a542d4a139fe106c19095129b: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:19:13.453608 17444 log.cc:1199] T c9944d5e4b8c4109af9955c4f08c9095 P 647a343a542d4a139fe106c19095129b: Deleting WAL directory at /tmp/dist-test-taskdH_Ma7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783603093801613-15046-0/minicluster-data/ts-0-root/wals/c9944d5e4b8c4109af9955c4f08c9095
I20260709 13:19:13.456758 16046 catalog_manager.cc:5697] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045 reported cstate change: term changed from 1 to 2, leader changed from 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130) to 90f3da57796b488c9087a2f523adf045 (127.14.177.131). New cstate: current_term: 2 leader_uuid: "90f3da57796b488c9087a2f523adf045" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } health_report { overall_health: HEALTHY } } }
I20260709 13:19:13.458935 16035 catalog_manager.cc:5028] TS 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): tablet c9944d5e4b8c4109af9955c4f08c9095 (table test-workload [id=ed3d15d8c9ad46bebc4fe3a402d2e54f]) successfully deleted
I20260709 13:19:13.473382 16300 raft_consensus.cc:1275] T cdbd78acb4cf46298e0de8ba0d130e19 P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:19:13.477311 16223 consensus_queue.cc:237] T 829b4ad8c5a54eababeea1181b960d4b P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.4, Last appended by leader: 2, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:13.482242 17400 raft_consensus.cc:993] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac: : Instructing follower cb82cc7bd6db46d980c75237a1402860 to start an election
I20260709 13:19:13.482779 17400 raft_consensus.cc:1081] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [term 1 LEADER]: Signalling peer cb82cc7bd6db46d980c75237a1402860 to start an election
I20260709 13:19:13.484615 16824 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d2372981006d4b218215c28964dea623"
dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
 from {username='slave'} at 127.0.0.1:49906
I20260709 13:19:13.485148 16824 raft_consensus.cc:493] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:19:13.485489 16824 raft_consensus.cc:3060] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:19:13.492097 16820 raft_consensus.cc:1275] T 829b4ad8c5a54eababeea1181b960d4b P cb82cc7bd6db46d980c75237a1402860 [term 2 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:19:13.492700 16150 raft_consensus.cc:1275] T 829b4ad8c5a54eababeea1181b960d4b P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:19:13.494484 17411 consensus_queue.cc:1048] T 829b4ad8c5a54eababeea1181b960d4b P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:19:13.498677 17317 consensus_queue.cc:1048] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:19:13.527724 17301 raft_consensus.cc:2955] T 829b4ad8c5a54eababeea1181b960d4b P 1ca2725d290e47d68b1a4b609c9816ac [term 2 LEADER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 90f3da57796b488c9087a2f523adf045 (127.14.177.131) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:19:13.535437 17349 raft_consensus.cc:993] T c0eab25ce4c64e018a041beeda2867db P 647a343a542d4a139fe106c19095129b: : Instructing follower 1ca2725d290e47d68b1a4b609c9816ac to start an election
I20260709 13:19:13.535974 17349 raft_consensus.cc:1081] T c0eab25ce4c64e018a041beeda2867db P 647a343a542d4a139fe106c19095129b [term 1 LEADER]: Signalling peer 1ca2725d290e47d68b1a4b609c9816ac to start an election
I20260709 13:19:13.542299 17414 raft_consensus.cc:993] T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b: : Instructing follower cb82cc7bd6db46d980c75237a1402860 to start an election
I20260709 13:19:13.542775 17414 raft_consensus.cc:1081] T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b [term 1 LEADER]: Signalling peer cb82cc7bd6db46d980c75237a1402860 to start an election
I20260709 13:19:13.544486 16825 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "08b7d1ab519044eda2d5e04a07242f25"
dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
 from {username='slave'} at 127.0.0.1:49896
I20260709 13:19:13.543594 16145 raft_consensus.cc:2955] T 829b4ad8c5a54eababeea1181b960d4b P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 90f3da57796b488c9087a2f523adf045 (127.14.177.131) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:19:13.545112 16825 raft_consensus.cc:493] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:19:13.545624 16825 raft_consensus.cc:3060] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:19:13.549153 16223 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c0eab25ce4c64e018a041beeda2867db"
dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
 from {username='slave'} at 127.0.0.1:57604
I20260709 13:19:13.549758 16223 raft_consensus.cc:493] T c0eab25ce4c64e018a041beeda2867db P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:19:13.550127 16223 raft_consensus.cc:3060] T c0eab25ce4c64e018a041beeda2867db P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:19:13.577591 17399 consensus_queue.cc:1048] T 829b4ad8c5a54eababeea1181b960d4b P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:19:13.581023 16035 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 829b4ad8c5a54eababeea1181b960d4b with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:19:13.553035 16825 raft_consensus.cc:515] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:13.602833 16144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b7d1ab519044eda2d5e04a07242f25" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:19:13.603466 16144 raft_consensus.cc:3055] T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:19:13.603755 16144 raft_consensus.cc:740] T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:19:13.604461 16144 consensus_queue.cc:260] T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:13.605464 16144 raft_consensus.cc:3060] T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:19:13.605093 16046 catalog_manager.cc:5697] T 829b4ad8c5a54eababeea1181b960d4b P 1ca2725d290e47d68b1a4b609c9816ac reported cstate change: config changed from index 4 to 5, VOTER 90f3da57796b488c9087a2f523adf045 (127.14.177.131) evicted. New cstate: current_term: 2 leader_uuid: "1ca2725d290e47d68b1a4b609c9816ac" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:19:13.612053 16223 raft_consensus.cc:515] T c0eab25ce4c64e018a041beeda2867db P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:13.614009 16824 raft_consensus.cc:515] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:13.617753 16146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2372981006d4b218215c28964dea623" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:19:13.618432 16146 raft_consensus.cc:3060] T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:19:13.614562 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0eab25ce4c64e018a041beeda2867db" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:19:13.620510 16150 raft_consensus.cc:3055] T c0eab25ce4c64e018a041beeda2867db P 647a343a542d4a139fe106c19095129b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:19:13.620826 16150 raft_consensus.cc:740] T c0eab25ce4c64e018a041beeda2867db P 647a343a542d4a139fe106c19095129b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:19:13.621937 16150 consensus_queue.cc:260] T c0eab25ce4c64e018a041beeda2867db P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:13.621805 16820 raft_consensus.cc:2955] T 829b4ad8c5a54eababeea1181b960d4b P cb82cc7bd6db46d980c75237a1402860 [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 90f3da57796b488c9087a2f523adf045 (127.14.177.131) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:19:13.625103 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2372981006d4b218215c28964dea623" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:19:13.625685 16224 raft_consensus.cc:3055] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:19:13.625989 16224 raft_consensus.cc:740] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:19:13.626722 16146 raft_consensus.cc:2468] T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 2.
I20260709 13:19:13.626658 16224 consensus_queue.cc:260] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:13.627465 16286 tablet_service.cc:1558] Processing DeleteTablet for tablet 829b4ad8c5a54eababeea1181b960d4b with delete_type TABLET_DATA_TOMBSTONED (TS 90f3da57796b488c9087a2f523adf045 not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:48670
I20260709 13:19:13.627919 16224 raft_consensus.cc:3060] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:19:13.628398 17443 tablet_replica.cc:333] T 829b4ad8c5a54eababeea1181b960d4b P 90f3da57796b488c9087a2f523adf045: stopping tablet replica
I20260709 13:19:13.629509 17443 raft_consensus.cc:2243] T 829b4ad8c5a54eababeea1181b960d4b P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:19:13.630221 17443 raft_consensus.cc:2272] T 829b4ad8c5a54eababeea1181b960d4b P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:19:13.635068 17443 ts_tablet_manager.cc:1916] T 829b4ad8c5a54eababeea1181b960d4b P 90f3da57796b488c9087a2f523adf045: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:19:13.642678 16144 raft_consensus.cc:2468] T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 2.
I20260709 13:19:13.643931 16824 leader_election.cc:290] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 2 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:19:13.637646 16790 leader_election.cc:304] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:19:13.652951 16224 raft_consensus.cc:2468] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 2.
I20260709 13:19:13.622805 16150 raft_consensus.cc:3060] T c0eab25ce4c64e018a041beeda2867db P 647a343a542d4a139fe106c19095129b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:19:13.662072 16790 leader_election.cc:304] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:19:13.668998 16223 leader_election.cc:290] T c0eab25ce4c64e018a041beeda2867db P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 2 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:13.669473 16150 raft_consensus.cc:2468] T c0eab25ce4c64e018a041beeda2867db P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 2.
I20260709 13:19:13.670002 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0eab25ce4c64e018a041beeda2867db" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:19:13.670634 16300 raft_consensus.cc:3060] T c0eab25ce4c64e018a041beeda2867db P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:19:13.667387 17450 raft_consensus.cc:2804] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:19:13.675370 17387 raft_consensus.cc:2804] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:19:13.679355 16191 leader_election.cc:304] T c0eab25ce4c64e018a041beeda2867db P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
I20260709 13:19:13.673098 17450 raft_consensus.cc:697] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 [term 2 LEADER]: Becoming Leader. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:19:13.682301 17301 raft_consensus.cc:2804] T c0eab25ce4c64e018a041beeda2867db P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:19:13.682986 16223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b7d1ab519044eda2d5e04a07242f25" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:19:13.683240 17301 raft_consensus.cc:697] T c0eab25ce4c64e018a041beeda2867db P 1ca2725d290e47d68b1a4b609c9816ac [term 2 LEADER]: Becoming Leader. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:19:13.683575 16223 raft_consensus.cc:3060] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:19:13.684649 17301 consensus_queue.cc:237] T c0eab25ce4c64e018a041beeda2867db P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:13.680576 16825 leader_election.cc:290] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 2 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:19:13.686563 17450 consensus_queue.cc:237] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:13.681857 17387 raft_consensus.cc:697] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 [term 2 LEADER]: Becoming Leader. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:19:13.693782 17387 consensus_queue.cc:237] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:13.711867 17322 raft_consensus.cc:993] T e4dc6b0541ca431ebc757beac00521f1 P 90f3da57796b488c9087a2f523adf045: : Instructing follower 647a343a542d4a139fe106c19095129b to start an election
I20260709 13:19:13.712410 17406 raft_consensus.cc:1081] T e4dc6b0541ca431ebc757beac00521f1 P 90f3da57796b488c9087a2f523adf045 [term 2 LEADER]: Signalling peer 647a343a542d4a139fe106c19095129b to start an election
I20260709 13:19:13.714226 16146 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e4dc6b0541ca431ebc757beac00521f1"
dest_uuid: "647a343a542d4a139fe106c19095129b"
 from {username='slave'} at 127.0.0.1:40830
I20260709 13:19:13.714768 16146 raft_consensus.cc:493] T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:19:13.715215 16146 raft_consensus.cc:3060] T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:19:13.716372 16300 raft_consensus.cc:2468] T c0eab25ce4c64e018a041beeda2867db P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 2.
I20260709 13:19:13.723695 16223 raft_consensus.cc:2468] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 2.
I20260709 13:19:13.727936 16146 raft_consensus.cc:515] T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:13.728967 17443 ts_tablet_manager.cc:1929] T 829b4ad8c5a54eababeea1181b960d4b P 90f3da57796b488c9087a2f523adf045: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.4
I20260709 13:19:13.729414 17443 log.cc:1199] T 829b4ad8c5a54eababeea1181b960d4b P 90f3da57796b488c9087a2f523adf045: Deleting WAL directory at /tmp/dist-test-taskdH_Ma7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783603093801613-15046-0/minicluster-data/ts-2-root/wals/829b4ad8c5a54eababeea1181b960d4b
I20260709 13:19:13.730432 16146 leader_election.cc:290] T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 3 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:13.731016 16223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4dc6b0541ca431ebc757beac00521f1" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: true dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:19:13.730582 16046 catalog_manager.cc:5697] T c0eab25ce4c64e018a041beeda2867db P 1ca2725d290e47d68b1a4b609c9816ac reported cstate change: term changed from 1 to 2, leader changed from 647a343a542d4a139fe106c19095129b (127.14.177.129) to 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130). New cstate: current_term: 2 leader_uuid: "1ca2725d290e47d68b1a4b609c9816ac" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } health_report { overall_health: UNKNOWN } } }
I20260709 13:19:13.733685 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4dc6b0541ca431ebc757beac00521f1" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: true dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:19:13.734345 16298 raft_consensus.cc:3055] T e4dc6b0541ca431ebc757beac00521f1 P 90f3da57796b488c9087a2f523adf045 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:19:13.734680 16298 raft_consensus.cc:740] T e4dc6b0541ca431ebc757beac00521f1 P 90f3da57796b488c9087a2f523adf045 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:19:13.748114 16298 consensus_queue.cc:260] T e4dc6b0541ca431ebc757beac00521f1 P 90f3da57796b488c9087a2f523adf045 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:13.749449 16298 raft_consensus.cc:3060] T e4dc6b0541ca431ebc757beac00521f1 P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:19:13.738863 16223 raft_consensus.cc:3060] T e4dc6b0541ca431ebc757beac00521f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:19:13.741971 16034 catalog_manager.cc:5028] TS 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389): tablet 829b4ad8c5a54eababeea1181b960d4b (table test-workload [id=ed3d15d8c9ad46bebc4fe3a402d2e54f]) successfully deleted
I20260709 13:19:13.746253 16047 catalog_manager.cc:5697] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 reported cstate change: term changed from 1 to 2, leader changed from 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130) to cb82cc7bd6db46d980c75237a1402860 (127.14.177.132). New cstate: current_term: 2 leader_uuid: "cb82cc7bd6db46d980c75237a1402860" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:19:13.770102 16298 raft_consensus.cc:2468] T e4dc6b0541ca431ebc757beac00521f1 P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 3.
I20260709 13:19:13.776121 16115 leader_election.cc:304] T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:19:13.776867 17452 raft_consensus.cc:2804] T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:19:13.777365 17452 raft_consensus.cc:697] T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b [term 3 LEADER]: Becoming Leader. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:19:13.778237 17452 consensus_queue.cc:237] T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:13.795182 16223 raft_consensus.cc:2468] T e4dc6b0541ca431ebc757beac00521f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 3.
I20260709 13:19:13.800539 16044 catalog_manager.cc:5697] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 reported cstate change: term changed from 1 to 2, leader changed from 647a343a542d4a139fe106c19095129b (127.14.177.129) to cb82cc7bd6db46d980c75237a1402860 (127.14.177.132). New cstate: current_term: 2 leader_uuid: "cb82cc7bd6db46d980c75237a1402860" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:19:13.830763 16046 catalog_manager.cc:5697] T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b reported cstate change: term changed from 2 to 3, leader changed from 90f3da57796b488c9087a2f523adf045 (127.14.177.131) to 647a343a542d4a139fe106c19095129b (127.14.177.129). New cstate: current_term: 3 leader_uuid: "647a343a542d4a139fe106c19095129b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } health_report { overall_health: UNKNOWN } } }
I20260709 13:19:13.831913 17453 raft_consensus.cc:993] T 18ad7e3af5504571a69d10c00750a01c P 647a343a542d4a139fe106c19095129b: : Instructing follower 1ca2725d290e47d68b1a4b609c9816ac to start an election
I20260709 13:19:13.832382 17414 raft_consensus.cc:1081] T 18ad7e3af5504571a69d10c00750a01c P 647a343a542d4a139fe106c19095129b [term 1 LEADER]: Signalling peer 1ca2725d290e47d68b1a4b609c9816ac to start an election
I20260709 13:19:13.834095 16224 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "18ad7e3af5504571a69d10c00750a01c"
dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
 from {username='slave'} at 127.0.0.1:57604
I20260709 13:19:13.834632 16224 raft_consensus.cc:493] T 18ad7e3af5504571a69d10c00750a01c P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:19:13.835008 16224 raft_consensus.cc:3060] T 18ad7e3af5504571a69d10c00750a01c P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:19:13.842577 16224 raft_consensus.cc:515] T 18ad7e3af5504571a69d10c00750a01c P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } }
I20260709 13:19:13.844452 16224 leader_election.cc:290] T 18ad7e3af5504571a69d10c00750a01c P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 2 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:13.851919 16146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18ad7e3af5504571a69d10c00750a01c" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:19:13.852736 16146 raft_consensus.cc:3055] T 18ad7e3af5504571a69d10c00750a01c P 647a343a542d4a139fe106c19095129b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:19:13.853109 16146 raft_consensus.cc:740] T 18ad7e3af5504571a69d10c00750a01c P 647a343a542d4a139fe106c19095129b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:19:13.854965 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18ad7e3af5504571a69d10c00750a01c" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:19:13.855593 16298 raft_consensus.cc:3060] T 18ad7e3af5504571a69d10c00750a01c P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:19:13.855607 16146 consensus_queue.cc:260] T 18ad7e3af5504571a69d10c00750a01c P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } }
I20260709 13:19:13.860561 16146 raft_consensus.cc:3060] T 18ad7e3af5504571a69d10c00750a01c P 647a343a542d4a139fe106c19095129b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:19:13.867262 16298 raft_consensus.cc:2468] T 18ad7e3af5504571a69d10c00750a01c P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 2.
I20260709 13:19:13.868404 16190 leader_election.cc:304] T 18ad7e3af5504571a69d10c00750a01c P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:19:13.869267 17301 raft_consensus.cc:2804] T 18ad7e3af5504571a69d10c00750a01c P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:19:13.869710 17301 raft_consensus.cc:697] T 18ad7e3af5504571a69d10c00750a01c P 1ca2725d290e47d68b1a4b609c9816ac [term 2 LEADER]: Becoming Leader. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:19:13.870441 17301 consensus_queue.cc:237] T 18ad7e3af5504571a69d10c00750a01c P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } }
I20260709 13:19:13.883138 16146 raft_consensus.cc:2468] T 18ad7e3af5504571a69d10c00750a01c P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 2.
I20260709 13:19:13.897538 16046 catalog_manager.cc:5697] T 18ad7e3af5504571a69d10c00750a01c P 1ca2725d290e47d68b1a4b609c9816ac reported cstate change: term changed from 1 to 2, leader changed from 647a343a542d4a139fe106c19095129b (127.14.177.129) to 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130). New cstate: current_term: 2 leader_uuid: "1ca2725d290e47d68b1a4b609c9816ac" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: HEALTHY } } }
I20260709 13:19:13.903123 16145 raft_consensus.cc:1275] T 08c74490c40243a88e62d1760b5cd814 P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:19:13.906142 17322 consensus_queue.cc:1048] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:19:13.923414 16224 raft_consensus.cc:1275] T 08c74490c40243a88e62d1760b5cd814 P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:19:13.927309 17434 consensus_queue.cc:1048] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:19:14.082516 16224 raft_consensus.cc:1275] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:19:14.086313 17387 consensus_queue.cc:1048] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:19:14.110034 16150 raft_consensus.cc:1275] T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:19:14.117908 17387 consensus_queue.cc:1048] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.003s
I20260709 13:19:14.156847 16145 raft_consensus.cc:1275] T c0eab25ce4c64e018a041beeda2867db P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:19:14.158730 17400 consensus_queue.cc:1048] T c0eab25ce4c64e018a041beeda2867db P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:19:14.176532 16298 raft_consensus.cc:1275] T c0eab25ce4c64e018a041beeda2867db P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:19:14.181828 17400 consensus_queue.cc:1048] T c0eab25ce4c64e018a041beeda2867db P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:19:14.274691 16224 raft_consensus.cc:1275] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:19:14.276274 17387 consensus_queue.cc:1048] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:19:14.288661 16150 raft_consensus.cc:1275] T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:19:14.298456 17387 consensus_queue.cc:1048] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:19:14.314515 16299 raft_consensus.cc:1275] T 18ad7e3af5504571a69d10c00750a01c P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:19:14.316344 17400 consensus_queue.cc:1048] T 18ad7e3af5504571a69d10c00750a01c P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:19:14.358721 16150 raft_consensus.cc:1275] T 18ad7e3af5504571a69d10c00750a01c P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:19:14.360122 17301 consensus_queue.cc:1048] T 18ad7e3af5504571a69d10c00750a01c P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:19:14.376895 16224 raft_consensus.cc:1275] T e4dc6b0541ca431ebc757beac00521f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:19:14.378247 17452 consensus_queue.cc:1048] T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:19:14.381148 16298 raft_consensus.cc:1275] T e4dc6b0541ca431ebc757beac00521f1 P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:19:14.383483 17453 consensus_queue.cc:1048] T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:19:14.443897 15046 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:19:14.521142 15046 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:19:14.565634 16298 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1df93779209748b9860f6d893dd3de53"
dest_uuid: "90f3da57796b488c9087a2f523adf045"
mode: GRACEFUL
new_leader_uuid: "647a343a542d4a139fe106c19095129b"
 from {username='slave'} at 127.0.0.1:43334
I20260709 13:19:14.567955 16298 raft_consensus.cc:606] T 1df93779209748b9860f6d893dd3de53 P 90f3da57796b488c9087a2f523adf045 [term 1 LEADER]: Received request to transfer leadership to TS 647a343a542d4a139fe106c19095129b
I20260709 13:19:14.570322 16224 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "68a6838d82ef4cac83507f906139d90f"
dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
mode: GRACEFUL
new_leader_uuid: "90f3da57796b488c9087a2f523adf045"
 from {username='slave'} at 127.0.0.1:39144
I20260709 13:19:14.570796 16224 raft_consensus.cc:606] T 68a6838d82ef4cac83507f906139d90f P 1ca2725d290e47d68b1a4b609c9816ac [term 1 LEADER]: Received request to transfer leadership to TS 90f3da57796b488c9087a2f523adf045
I20260709 13:19:14.574610 16150 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20"
dest_uuid: "647a343a542d4a139fe106c19095129b"
mode: GRACEFUL
new_leader_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
 from {username='slave'} at 127.0.0.1:40290
I20260709 13:19:14.575201 16150 raft_consensus.cc:606] T ecc1e363230043fe9fef390f0b979f20 P 647a343a542d4a139fe106c19095129b [term 1 LEADER]: Received request to transfer leadership to TS 1ca2725d290e47d68b1a4b609c9816ac
I20260709 13:19:14.576480 15046 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 13:19:14.577849 15046 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:19:14.735930 17453 raft_consensus.cc:993] T ecc1e363230043fe9fef390f0b979f20 P 647a343a542d4a139fe106c19095129b: : Instructing follower 1ca2725d290e47d68b1a4b609c9816ac to start an election
I20260709 13:19:14.737004 17453 raft_consensus.cc:1081] T ecc1e363230043fe9fef390f0b979f20 P 647a343a542d4a139fe106c19095129b [term 1 LEADER]: Signalling peer 1ca2725d290e47d68b1a4b609c9816ac to start an election
I20260709 13:19:14.739885 16224 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20"
dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
 from {username='slave'} at 127.0.0.1:57604
I20260709 13:19:14.740916 16224 raft_consensus.cc:493] T ecc1e363230043fe9fef390f0b979f20 P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:19:14.741469 16224 raft_consensus.cc:3060] T ecc1e363230043fe9fef390f0b979f20 P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:19:14.781914 16224 raft_consensus.cc:515] T ecc1e363230043fe9fef390f0b979f20 P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:14.786571 16224 leader_election.cc:290] T ecc1e363230043fe9fef390f0b979f20 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 2 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:14.787750 16295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:19:14.788983 16295 raft_consensus.cc:3060] T ecc1e363230043fe9fef390f0b979f20 P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:19:14.790205 16144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:19:14.790941 16144 raft_consensus.cc:3055] T ecc1e363230043fe9fef390f0b979f20 P 647a343a542d4a139fe106c19095129b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:19:14.791409 16144 raft_consensus.cc:740] T ecc1e363230043fe9fef390f0b979f20 P 647a343a542d4a139fe106c19095129b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:19:14.792575 16144 consensus_queue.cc:260] T ecc1e363230043fe9fef390f0b979f20 P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:14.798718 16144 raft_consensus.cc:3060] T ecc1e363230043fe9fef390f0b979f20 P 647a343a542d4a139fe106c19095129b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:19:14.809669 16144 raft_consensus.cc:2468] T ecc1e363230043fe9fef390f0b979f20 P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 2.
I20260709 13:19:14.811756 16191 leader_election.cc:304] T ecc1e363230043fe9fef390f0b979f20 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
I20260709 13:19:14.812938 17400 raft_consensus.cc:2804] T ecc1e363230043fe9fef390f0b979f20 P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:19:14.813472 17400 raft_consensus.cc:697] T ecc1e363230043fe9fef390f0b979f20 P 1ca2725d290e47d68b1a4b609c9816ac [term 2 LEADER]: Becoming Leader. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:19:14.814656 17400 consensus_queue.cc:237] T ecc1e363230043fe9fef390f0b979f20 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:14.822247 16295 raft_consensus.cc:2468] T ecc1e363230043fe9fef390f0b979f20 P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 2.
I20260709 13:19:14.861663 16047 catalog_manager.cc:5697] T ecc1e363230043fe9fef390f0b979f20 P 1ca2725d290e47d68b1a4b609c9816ac reported cstate change: term changed from 1 to 2, leader changed from 647a343a542d4a139fe106c19095129b (127.14.177.129) to 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130). New cstate: current_term: 2 leader_uuid: "1ca2725d290e47d68b1a4b609c9816ac" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } health_report { overall_health: UNKNOWN } } }
I20260709 13:19:14.907006 17406 raft_consensus.cc:993] T 1df93779209748b9860f6d893dd3de53 P 90f3da57796b488c9087a2f523adf045: : Instructing follower 647a343a542d4a139fe106c19095129b to start an election
I20260709 13:19:14.907631 17406 raft_consensus.cc:1081] T 1df93779209748b9860f6d893dd3de53 P 90f3da57796b488c9087a2f523adf045 [term 1 LEADER]: Signalling peer 647a343a542d4a139fe106c19095129b to start an election
I20260709 13:19:14.912262 16150 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "1df93779209748b9860f6d893dd3de53"
dest_uuid: "647a343a542d4a139fe106c19095129b"
 from {username='slave'} at 127.0.0.1:40830
I20260709 13:19:14.912833 16150 raft_consensus.cc:493] T 1df93779209748b9860f6d893dd3de53 P 647a343a542d4a139fe106c19095129b [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:19:14.913179 16150 raft_consensus.cc:3060] T 1df93779209748b9860f6d893dd3de53 P 647a343a542d4a139fe106c19095129b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:19:14.951107 17301 raft_consensus.cc:993] T 68a6838d82ef4cac83507f906139d90f P 1ca2725d290e47d68b1a4b609c9816ac: : Instructing follower 90f3da57796b488c9087a2f523adf045 to start an election
I20260709 13:19:14.951710 17301 raft_consensus.cc:1081] T 68a6838d82ef4cac83507f906139d90f P 1ca2725d290e47d68b1a4b609c9816ac [term 1 LEADER]: Signalling peer 90f3da57796b488c9087a2f523adf045 to start an election
I20260709 13:19:14.958652 16298 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "68a6838d82ef4cac83507f906139d90f"
dest_uuid: "90f3da57796b488c9087a2f523adf045"
 from {username='slave'} at 127.0.0.1:48684
I20260709 13:19:14.963362 16298 raft_consensus.cc:493] T 68a6838d82ef4cac83507f906139d90f P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:19:14.963835 16298 raft_consensus.cc:3060] T 68a6838d82ef4cac83507f906139d90f P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:19:14.960008 16223 consensus_queue.cc:237] T 8c11074579b046d99e585c2818e18a2c P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } }
I20260709 13:19:14.971199 16150 raft_consensus.cc:515] T 1df93779209748b9860f6d893dd3de53 P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:14.972033 16298 raft_consensus.cc:515] T 68a6838d82ef4cac83507f906139d90f P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:14.974282 16298 leader_election.cc:290] T 68a6838d82ef4cac83507f906139d90f P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 2 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:19:14.975737 16144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68a6838d82ef4cac83507f906139d90f" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:19:14.976315 16144 raft_consensus.cc:3060] T 68a6838d82ef4cac83507f906139d90f P 647a343a542d4a139fe106c19095129b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:19:14.980114 16299 raft_consensus.cc:1275] T 8c11074579b046d99e585c2818e18a2c P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:19:14.981086 16150 leader_election.cc:290] T 1df93779209748b9860f6d893dd3de53 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 2 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
W20260709 13:19:14.982252 16192 consensus_peers.cc:597] T 8c11074579b046d99e585c2818e18a2c P 1ca2725d290e47d68b1a4b609c9816ac -> Peer cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923): Couldn't send request to peer cb82cc7bd6db46d980c75237a1402860. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 8c11074579b046d99e585c2818e18a2c. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:19:14.983078 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1df93779209748b9860f6d893dd3de53" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:19:14.983741 16299 raft_consensus.cc:3055] T 1df93779209748b9860f6d893dd3de53 P 90f3da57796b488c9087a2f523adf045 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:19:14.984081 16299 raft_consensus.cc:740] T 1df93779209748b9860f6d893dd3de53 P 90f3da57796b488c9087a2f523adf045 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:19:14.985037 16299 consensus_queue.cc:260] T 1df93779209748b9860f6d893dd3de53 P 90f3da57796b488c9087a2f523adf045 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:14.986312 16299 raft_consensus.cc:3060] T 1df93779209748b9860f6d893dd3de53 P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:19:14.987533 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1df93779209748b9860f6d893dd3de53" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:19:14.988098 16224 raft_consensus.cc:3060] T 1df93779209748b9860f6d893dd3de53 P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:19:14.983258 16144 raft_consensus.cc:2468] T 68a6838d82ef4cac83507f906139d90f P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 2.
I20260709 13:19:14.989504 16265 leader_election.cc:304] T 68a6838d82ef4cac83507f906139d90f P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:19:14.990463 17477 raft_consensus.cc:2804] T 68a6838d82ef4cac83507f906139d90f P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:19:14.975739 16145 raft_consensus.cc:1275] T 8c11074579b046d99e585c2818e18a2c P 647a343a542d4a139fe106c19095129b [term 1 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:19:14.991096 17477 raft_consensus.cc:697] T 68a6838d82ef4cac83507f906139d90f P 90f3da57796b488c9087a2f523adf045 [term 2 LEADER]: Becoming Leader. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:19:14.991793 17472 consensus_queue.cc:1048] T 8c11074579b046d99e585c2818e18a2c P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:19:14.992267 17477 consensus_queue.cc:237] T 68a6838d82ef4cac83507f906139d90f P 90f3da57796b488c9087a2f523adf045 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:14.995748 17400 consensus_queue.cc:1048] T 8c11074579b046d99e585c2818e18a2c P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:19:15.010454 16224 raft_consensus.cc:2468] T 1df93779209748b9860f6d893dd3de53 P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 2.
I20260709 13:19:15.010874 16223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68a6838d82ef4cac83507f906139d90f" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:19:15.011590 16223 raft_consensus.cc:3055] T 68a6838d82ef4cac83507f906139d90f P 1ca2725d290e47d68b1a4b609c9816ac [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:19:15.011902 16223 raft_consensus.cc:740] T 68a6838d82ef4cac83507f906139d90f P 1ca2725d290e47d68b1a4b609c9816ac [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:19:15.012035 16116 leader_election.cc:304] T 1df93779209748b9860f6d893dd3de53 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
I20260709 13:19:15.012748 16223 consensus_queue.cc:260] T 68a6838d82ef4cac83507f906139d90f P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:15.014185 16223 raft_consensus.cc:3060] T 68a6838d82ef4cac83507f906139d90f P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:19:15.015676 17453 raft_consensus.cc:2804] T 1df93779209748b9860f6d893dd3de53 P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:19:15.016381 17453 raft_consensus.cc:697] T 1df93779209748b9860f6d893dd3de53 P 647a343a542d4a139fe106c19095129b [term 2 LEADER]: Becoming Leader. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:19:15.017599 17453 consensus_queue.cc:237] T 1df93779209748b9860f6d893dd3de53 P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:15.019690 16047 catalog_manager.cc:5697] T 68a6838d82ef4cac83507f906139d90f P 90f3da57796b488c9087a2f523adf045 reported cstate change: term changed from 1 to 2, leader changed from 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130) to 90f3da57796b488c9087a2f523adf045 (127.14.177.131). New cstate: current_term: 2 leader_uuid: "90f3da57796b488c9087a2f523adf045" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } health_report { overall_health: HEALTHY } } }
I20260709 13:19:15.034162 16223 raft_consensus.cc:2468] T 68a6838d82ef4cac83507f906139d90f P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 2.
I20260709 13:19:15.010516 16299 raft_consensus.cc:2468] T 1df93779209748b9860f6d893dd3de53 P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 2.
I20260709 13:19:15.040261 17400 raft_consensus.cc:2955] T 8c11074579b046d99e585c2818e18a2c P 1ca2725d290e47d68b1a4b609c9816ac [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } } }
I20260709 13:19:15.046083 16299 raft_consensus.cc:2955] T 8c11074579b046d99e585c2818e18a2c P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } } }
I20260709 13:19:15.061740 16047 catalog_manager.cc:5697] T 8c11074579b046d99e585c2818e18a2c P 1ca2725d290e47d68b1a4b609c9816ac reported cstate change: config changed from index -1 to 2, NON_VOTER cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) added. New cstate: current_term: 1 leader_uuid: "1ca2725d290e47d68b1a4b609c9816ac" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:19:15.065032 16046 catalog_manager.cc:5697] T 1df93779209748b9860f6d893dd3de53 P 647a343a542d4a139fe106c19095129b reported cstate change: term changed from 1 to 2, leader changed from 90f3da57796b488c9087a2f523adf045 (127.14.177.131) to 647a343a542d4a139fe106c19095129b (127.14.177.129). New cstate: current_term: 2 leader_uuid: "647a343a542d4a139fe106c19095129b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } health_report { overall_health: UNKNOWN } } }
I20260709 13:19:15.072741 16298 consensus_queue.cc:237] T 434bff3082d141838068f323bdb0d1ed P 90f3da57796b488c9087a2f523adf045 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } }
I20260709 13:19:15.090579 16144 raft_consensus.cc:2955] T 8c11074579b046d99e585c2818e18a2c P 647a343a542d4a139fe106c19095129b [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } } }
W20260709 13:19:15.126540 16266 consensus_peers.cc:597] T 434bff3082d141838068f323bdb0d1ed P 90f3da57796b488c9087a2f523adf045 -> Peer cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923): Couldn't send request to peer cb82cc7bd6db46d980c75237a1402860. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 434bff3082d141838068f323bdb0d1ed. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:19:15.128434 16223 raft_consensus.cc:1275] T 434bff3082d141838068f323bdb0d1ed P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 13:19:15.128852 16145 raft_consensus.cc:1275] T 434bff3082d141838068f323bdb0d1ed P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 13:19:15.134614 17322 consensus_queue.cc:1048] T 434bff3082d141838068f323bdb0d1ed P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:19:15.137297 17406 consensus_queue.cc:1048] T 434bff3082d141838068f323bdb0d1ed P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:19:15.156971 17322 raft_consensus.cc:2955] T 434bff3082d141838068f323bdb0d1ed P 90f3da57796b488c9087a2f523adf045 [term 2 LEADER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } } }
I20260709 13:19:15.164714 16223 raft_consensus.cc:2955] T 434bff3082d141838068f323bdb0d1ed P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } } }
I20260709 13:19:15.182999 16144 consensus_queue.cc:237] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } }
I20260709 13:19:15.188233 16044 catalog_manager.cc:5697] T 434bff3082d141838068f323bdb0d1ed P 1ca2725d290e47d68b1a4b609c9816ac reported cstate change: config changed from index -1 to 3, NON_VOTER cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) added. New cstate: current_term: 2 leader_uuid: "90f3da57796b488c9087a2f523adf045" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } } }
I20260709 13:19:15.195602 16145 raft_consensus.cc:2955] T 434bff3082d141838068f323bdb0d1ed P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } } }
W20260709 13:19:15.199635 16117 consensus_peers.cc:597] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b -> Peer cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923): Couldn't send request to peer cb82cc7bd6db46d980c75237a1402860. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0a9de347c711477e87c582d003f36925. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:19:15.200251 16298 raft_consensus.cc:1275] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:19:15.201980 17414 consensus_queue.cc:1048] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:19:15.206200 16223 raft_consensus.cc:1275] T 0a9de347c711477e87c582d003f36925 P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:19:15.208493 17452 consensus_queue.cc:1048] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:19:15.228473 17414 raft_consensus.cc:2955] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } } }
I20260709 13:19:15.235505 16145 raft_consensus.cc:1275] T ecc1e363230043fe9fef390f0b979f20 P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:19:15.237985 17411 consensus_queue.cc:1048] T ecc1e363230043fe9fef390f0b979f20 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:19:15.248205 16295 raft_consensus.cc:2955] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } } }
I20260709 13:19:15.251814 16224 raft_consensus.cc:2955] T 0a9de347c711477e87c582d003f36925 P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } } }
I20260709 13:19:15.265064 16222 consensus_queue.cc:237] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } }
I20260709 13:19:15.278626 16046 catalog_manager.cc:5697] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b reported cstate change: config changed from index -1 to 2, NON_VOTER cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) added. New cstate: current_term: 1 leader_uuid: "647a343a542d4a139fe106c19095129b" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:19:15.314087 16299 raft_consensus.cc:1275] T ecc1e363230043fe9fef390f0b979f20 P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:19:15.315619 17472 consensus_queue.cc:1048] T ecc1e363230043fe9fef390f0b979f20 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:19:15.329286 16117 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 50646us
W20260709 13:19:15.332665 16192 consensus_peers.cc:597] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 1ca2725d290e47d68b1a4b609c9816ac -> Peer cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923): Couldn't send request to peer cb82cc7bd6db46d980c75237a1402860. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 9e4bfb9090c8460eb14fa7e0d818e7f1. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:19:15.350266 16144 raft_consensus.cc:1275] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 647a343a542d4a139fe106c19095129b [term 1 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:19:15.360353 16295 raft_consensus.cc:1275] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:19:15.368177 17472 consensus_queue.cc:1048] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:19:15.371569 17301 consensus_queue.cc:1048] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:19:15.402912 16224 raft_consensus.cc:1275] T 68a6838d82ef4cac83507f906139d90f P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:19:15.409976 17322 consensus_queue.cc:1048] T 68a6838d82ef4cac83507f906139d90f P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:19:15.410720 17472 raft_consensus.cc:2955] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } } }
I20260709 13:19:15.425761 16298 raft_consensus.cc:2955] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } } }
I20260709 13:19:15.445066 16145 raft_consensus.cc:2955] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 647a343a542d4a139fe106c19095129b [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } } }
W20260709 13:19:15.486030 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 44277.2us
I20260709 13:19:15.489872 17503 ts_tablet_manager.cc:933] T 8c11074579b046d99e585c2818e18a2c P cb82cc7bd6db46d980c75237a1402860: Initiating tablet copy from peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:19:15.560658 17503 tablet_copy_client.cc:323] T 8c11074579b046d99e585c2818e18a2c P cb82cc7bd6db46d980c75237a1402860: tablet copy: Beginning tablet copy session from remote peer at address 127.14.177.130:35943
I20260709 13:19:15.585022 16236 tablet_copy_service.cc:140] P 1ca2725d290e47d68b1a4b609c9816ac: Received BeginTabletCopySession request for tablet 8c11074579b046d99e585c2818e18a2c from peer cb82cc7bd6db46d980c75237a1402860 ({username='slave'} at 127.0.0.1:33050)
I20260709 13:19:15.585672 16236 tablet_copy_service.cc:161] P 1ca2725d290e47d68b1a4b609c9816ac: Beginning new tablet copy session on tablet 8c11074579b046d99e585c2818e18a2c from peer cb82cc7bd6db46d980c75237a1402860 at {username='slave'} at 127.0.0.1:33050: session id = cb82cc7bd6db46d980c75237a1402860-8c11074579b046d99e585c2818e18a2c
I20260709 13:19:15.599722 17516 ts_tablet_manager.cc:933] T 434bff3082d141838068f323bdb0d1ed P cb82cc7bd6db46d980c75237a1402860: Initiating tablet copy from peer 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:15.578904 15046 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:19:15.610272 16224 raft_consensus.cc:1275] T 1df93779209748b9860f6d893dd3de53 P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:19:15.611649 17453 consensus_queue.cc:1048] T 1df93779209748b9860f6d893dd3de53 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:19:15.612972 16236 tablet_copy_source_session.cc:215] T 8c11074579b046d99e585c2818e18a2c P 1ca2725d290e47d68b1a4b609c9816ac: Tablet Copy: opened 0 blocks and 1 log segments
W20260709 13:19:15.628000 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 68656.3us
I20260709 13:19:15.629966 16145 raft_consensus.cc:1275] T 68a6838d82ef4cac83507f906139d90f P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:19:15.633780 16047 catalog_manager.cc:5697] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 647a343a542d4a139fe106c19095129b reported cstate change: config changed from index -1 to 2, NON_VOTER cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) added. New cstate: current_term: 1 leader_uuid: "1ca2725d290e47d68b1a4b609c9816ac" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } } }
I20260709 13:19:15.647706 17515 consensus_queue.cc:1048] T 68a6838d82ef4cac83507f906139d90f P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:19:15.667068 17503 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8c11074579b046d99e585c2818e18a2c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:19:15.672376 16298 raft_consensus.cc:1275] T 1df93779209748b9860f6d893dd3de53 P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:19:15.685999 17414 consensus_queue.cc:1048] T 1df93779209748b9860f6d893dd3de53 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:19:15.688026 17516 tablet_copy_client.cc:323] T 434bff3082d141838068f323bdb0d1ed P cb82cc7bd6db46d980c75237a1402860: tablet copy: Beginning tablet copy session from remote peer at address 127.14.177.131:41389
I20260709 13:19:15.697337 16310 tablet_copy_service.cc:140] P 90f3da57796b488c9087a2f523adf045: Received BeginTabletCopySession request for tablet 434bff3082d141838068f323bdb0d1ed from peer cb82cc7bd6db46d980c75237a1402860 ({username='slave'} at 127.0.0.1:40474)
I20260709 13:19:15.697908 16310 tablet_copy_service.cc:161] P 90f3da57796b488c9087a2f523adf045: Beginning new tablet copy session on tablet 434bff3082d141838068f323bdb0d1ed from peer cb82cc7bd6db46d980c75237a1402860 at {username='slave'} at 127.0.0.1:40474: session id = cb82cc7bd6db46d980c75237a1402860-434bff3082d141838068f323bdb0d1ed
I20260709 13:19:15.708148 17522 ts_tablet_manager.cc:933] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860: Initiating tablet copy from peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741)
I20260709 13:19:15.721910 17522 tablet_copy_client.cc:323] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860: tablet copy: Beginning tablet copy session from remote peer at address 127.14.177.129:36741
I20260709 13:19:15.723738 16161 tablet_copy_service.cc:140] P 647a343a542d4a139fe106c19095129b: Received BeginTabletCopySession request for tablet 0a9de347c711477e87c582d003f36925 from peer cb82cc7bd6db46d980c75237a1402860 ({username='slave'} at 127.0.0.1:46224)
I20260709 13:19:15.724238 16161 tablet_copy_service.cc:161] P 647a343a542d4a139fe106c19095129b: Beginning new tablet copy session on tablet 0a9de347c711477e87c582d003f36925 from peer cb82cc7bd6db46d980c75237a1402860 at {username='slave'} at 127.0.0.1:46224: session id = cb82cc7bd6db46d980c75237a1402860-0a9de347c711477e87c582d003f36925
I20260709 13:19:15.733237 16161 tablet_copy_source_session.cc:215] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:19:15.744426 17522 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0a9de347c711477e87c582d003f36925. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:19:15.764912 17503 tablet_copy_client.cc:806] T 8c11074579b046d99e585c2818e18a2c P cb82cc7bd6db46d980c75237a1402860: tablet copy: Starting download of 0 data blocks...
I20260709 13:19:15.746469 16310 tablet_copy_source_session.cc:215] T 434bff3082d141838068f323bdb0d1ed P 90f3da57796b488c9087a2f523adf045: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:19:15.766680 17503 tablet_copy_client.cc:670] T 8c11074579b046d99e585c2818e18a2c P cb82cc7bd6db46d980c75237a1402860: tablet copy: Starting download of 1 WAL segments...
I20260709 13:19:15.814179 17516 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 434bff3082d141838068f323bdb0d1ed. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:19:15.819258 17503 tablet_copy_client.cc:538] T 8c11074579b046d99e585c2818e18a2c P cb82cc7bd6db46d980c75237a1402860: tablet copy: Tablet Copy complete. Replacing tablet superblock.
W20260709 13:19:15.835892 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 161588us
I20260709 13:19:15.900545 15046 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:19:15.917424 17516 tablet_copy_client.cc:806] T 434bff3082d141838068f323bdb0d1ed P cb82cc7bd6db46d980c75237a1402860: tablet copy: Starting download of 0 data blocks...
I20260709 13:19:15.918176 17516 tablet_copy_client.cc:670] T 434bff3082d141838068f323bdb0d1ed P cb82cc7bd6db46d980c75237a1402860: tablet copy: Starting download of 1 WAL segments...
I20260709 13:19:15.940120 17503 tablet_bootstrap.cc:492] T 8c11074579b046d99e585c2818e18a2c P cb82cc7bd6db46d980c75237a1402860: Bootstrap starting.
I20260709 13:19:15.945690 17522 tablet_copy_client.cc:806] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860: tablet copy: Starting download of 0 data blocks...
I20260709 13:19:15.946328 17522 tablet_copy_client.cc:670] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860: tablet copy: Starting download of 1 WAL segments...
I20260709 13:19:15.949569 17516 tablet_copy_client.cc:538] T 434bff3082d141838068f323bdb0d1ed P cb82cc7bd6db46d980c75237a1402860: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:19:15.953619 17522 tablet_copy_client.cc:538] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:19:15.997982 17522 tablet_bootstrap.cc:492] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860: Bootstrap starting.
I20260709 13:19:16.006258 17516 tablet_bootstrap.cc:492] T 434bff3082d141838068f323bdb0d1ed P cb82cc7bd6db46d980c75237a1402860: Bootstrap starting.
I20260709 13:19:16.002915 17527 ts_tablet_manager.cc:933] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P cb82cc7bd6db46d980c75237a1402860: Initiating tablet copy from peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:19:16.043766 16295 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4e9b2bf7011f4ae688a79d7bb5a447b4"
dest_uuid: "90f3da57796b488c9087a2f523adf045"
mode: GRACEFUL
new_leader_uuid: "647a343a542d4a139fe106c19095129b"
 from {username='slave'} at 127.0.0.1:43334
I20260709 13:19:16.052273 16295 raft_consensus.cc:606] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 90f3da57796b488c9087a2f523adf045 [term 1 LEADER]: Received request to transfer leadership to TS 647a343a542d4a139fe106c19095129b
I20260709 13:19:16.059759 16222 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "66ae176961b74813a58bc15c9c4c6f2a"
dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
mode: GRACEFUL
new_leader_uuid: "cb82cc7bd6db46d980c75237a1402860"
 from {username='slave'} at 127.0.0.1:39144
I20260709 13:19:16.060376 16222 raft_consensus.cc:606] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [term 1 LEADER]: Received request to transfer leadership to TS cb82cc7bd6db46d980c75237a1402860
I20260709 13:19:16.063679 16145 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9ad5dbfa99ee4b8d9e28be92b3f36db5"
dest_uuid: "647a343a542d4a139fe106c19095129b"
mode: GRACEFUL
new_leader_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
 from {username='slave'} at 127.0.0.1:40290
I20260709 13:19:16.064244 16145 raft_consensus.cc:606] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b [term 1 LEADER]: Received request to transfer leadership to TS 1ca2725d290e47d68b1a4b609c9816ac
I20260709 13:19:16.065477 15046 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 13:19:16.068162 15046 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:19:16.096364 17527 tablet_copy_client.cc:323] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P cb82cc7bd6db46d980c75237a1402860: tablet copy: Beginning tablet copy session from remote peer at address 127.14.177.130:35943
I20260709 13:19:16.107656 17503 tablet_bootstrap.cc:492] T 8c11074579b046d99e585c2818e18a2c P cb82cc7bd6db46d980c75237a1402860: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:19:16.108603 17503 tablet_bootstrap.cc:492] T 8c11074579b046d99e585c2818e18a2c P cb82cc7bd6db46d980c75237a1402860: Bootstrap complete.
I20260709 13:19:16.109314 17503 ts_tablet_manager.cc:1403] T 8c11074579b046d99e585c2818e18a2c P cb82cc7bd6db46d980c75237a1402860: Time spent bootstrapping tablet: real 0.169s	user 0.020s	sys 0.009s
I20260709 13:19:16.112530 17503 raft_consensus.cc:359] T 8c11074579b046d99e585c2818e18a2c P cb82cc7bd6db46d980c75237a1402860 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } }
I20260709 13:19:16.114897 16236 tablet_copy_service.cc:140] P 1ca2725d290e47d68b1a4b609c9816ac: Received BeginTabletCopySession request for tablet 9e4bfb9090c8460eb14fa7e0d818e7f1 from peer cb82cc7bd6db46d980c75237a1402860 ({username='slave'} at 127.0.0.1:33050)
I20260709 13:19:16.135349 16236 tablet_copy_service.cc:161] P 1ca2725d290e47d68b1a4b609c9816ac: Beginning new tablet copy session on tablet 9e4bfb9090c8460eb14fa7e0d818e7f1 from peer cb82cc7bd6db46d980c75237a1402860 at {username='slave'} at 127.0.0.1:33050: session id = cb82cc7bd6db46d980c75237a1402860-9e4bfb9090c8460eb14fa7e0d818e7f1
I20260709 13:19:16.122238 17503 raft_consensus.cc:740] T 8c11074579b046d99e585c2818e18a2c P cb82cc7bd6db46d980c75237a1402860 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Initialized, Role: LEARNER
I20260709 13:19:16.158565 17503 consensus_queue.cc:260] T 8c11074579b046d99e585c2818e18a2c P cb82cc7bd6db46d980c75237a1402860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } }
I20260709 13:19:16.166182 17503 ts_tablet_manager.cc:1434] T 8c11074579b046d99e585c2818e18a2c P cb82cc7bd6db46d980c75237a1402860: Time spent starting tablet: real 0.057s	user 0.009s	sys 0.000s
I20260709 13:19:16.208772 16235 tablet_copy_service.cc:342] P 1ca2725d290e47d68b1a4b609c9816ac: Request end of tablet copy session cb82cc7bd6db46d980c75237a1402860-8c11074579b046d99e585c2818e18a2c received from {username='slave'} at 127.0.0.1:33050
I20260709 13:19:16.209345 16235 tablet_copy_service.cc:434] P 1ca2725d290e47d68b1a4b609c9816ac: ending tablet copy session cb82cc7bd6db46d980c75237a1402860-8c11074579b046d99e585c2818e18a2c on tablet 8c11074579b046d99e585c2818e18a2c with peer cb82cc7bd6db46d980c75237a1402860
I20260709 13:19:16.234189 17516 tablet_bootstrap.cc:492] T 434bff3082d141838068f323bdb0d1ed P cb82cc7bd6db46d980c75237a1402860: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:19:16.236788 16236 tablet_copy_source_session.cc:215] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 1ca2725d290e47d68b1a4b609c9816ac: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:19:16.240224 17527 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9e4bfb9090c8460eb14fa7e0d818e7f1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:19:16.244249 17516 tablet_bootstrap.cc:492] T 434bff3082d141838068f323bdb0d1ed P cb82cc7bd6db46d980c75237a1402860: Bootstrap complete.
I20260709 13:19:16.245147 17516 ts_tablet_manager.cc:1403] T 434bff3082d141838068f323bdb0d1ed P cb82cc7bd6db46d980c75237a1402860: Time spent bootstrapping tablet: real 0.239s	user 0.030s	sys 0.019s
I20260709 13:19:16.248732 17522 tablet_bootstrap.cc:492] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:19:16.252148 17522 tablet_bootstrap.cc:492] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860: Bootstrap complete.
I20260709 13:19:16.252900 17522 ts_tablet_manager.cc:1403] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860: Time spent bootstrapping tablet: real 0.255s	user 0.045s	sys 0.041s
I20260709 13:19:16.253046 17520 raft_consensus.cc:993] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 90f3da57796b488c9087a2f523adf045: : Instructing follower 647a343a542d4a139fe106c19095129b to start an election
I20260709 13:19:16.253633 17517 raft_consensus.cc:1081] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 90f3da57796b488c9087a2f523adf045 [term 1 LEADER]: Signalling peer 647a343a542d4a139fe106c19095129b to start an election
I20260709 13:19:16.256032 16145 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "4e9b2bf7011f4ae688a79d7bb5a447b4"
dest_uuid: "647a343a542d4a139fe106c19095129b"
 from {username='slave'} at 127.0.0.1:40830
I20260709 13:19:16.256536 16145 raft_consensus.cc:493] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 647a343a542d4a139fe106c19095129b [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:19:16.256873 16145 raft_consensus.cc:3060] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 647a343a542d4a139fe106c19095129b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:19:16.248449 17516 raft_consensus.cc:359] T 434bff3082d141838068f323bdb0d1ed P cb82cc7bd6db46d980c75237a1402860 [term 2 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } }
I20260709 13:19:16.259410 17516 raft_consensus.cc:740] T 434bff3082d141838068f323bdb0d1ed P cb82cc7bd6db46d980c75237a1402860 [term 2 LEARNER]: Becoming Follower/Learner. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Initialized, Role: LEARNER
I20260709 13:19:16.260057 17516 consensus_queue.cc:260] T 434bff3082d141838068f323bdb0d1ed P cb82cc7bd6db46d980c75237a1402860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } }
I20260709 13:19:16.262529 17516 ts_tablet_manager.cc:1434] T 434bff3082d141838068f323bdb0d1ed P cb82cc7bd6db46d980c75237a1402860: Time spent starting tablet: real 0.017s	user 0.005s	sys 0.000s
I20260709 13:19:16.265386 17522 raft_consensus.cc:359] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } }
I20260709 13:19:16.266255 17522 raft_consensus.cc:740] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Initialized, Role: LEARNER
I20260709 13:19:16.266929 17522 consensus_queue.cc:260] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } }
I20260709 13:19:16.269395 17522 ts_tablet_manager.cc:1434] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860: Time spent starting tablet: real 0.016s	user 0.006s	sys 0.000s
I20260709 13:19:16.271126 16161 tablet_copy_service.cc:342] P 647a343a542d4a139fe106c19095129b: Request end of tablet copy session cb82cc7bd6db46d980c75237a1402860-0a9de347c711477e87c582d003f36925 received from {username='slave'} at 127.0.0.1:46224
I20260709 13:19:16.271546 16161 tablet_copy_service.cc:434] P 647a343a542d4a139fe106c19095129b: ending tablet copy session cb82cc7bd6db46d980c75237a1402860-0a9de347c711477e87c582d003f36925 on tablet 0a9de347c711477e87c582d003f36925 with peer cb82cc7bd6db46d980c75237a1402860
I20260709 13:19:16.277005 17527 tablet_copy_client.cc:806] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P cb82cc7bd6db46d980c75237a1402860: tablet copy: Starting download of 0 data blocks...
I20260709 13:19:16.277652 17527 tablet_copy_client.cc:670] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P cb82cc7bd6db46d980c75237a1402860: tablet copy: Starting download of 1 WAL segments...
I20260709 13:19:16.281536 16145 raft_consensus.cc:515] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:16.284122 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e9b2bf7011f4ae688a79d7bb5a447b4" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:19:16.284655 16298 raft_consensus.cc:3055] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 90f3da57796b488c9087a2f523adf045 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:19:16.284936 16298 raft_consensus.cc:740] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 90f3da57796b488c9087a2f523adf045 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:19:16.285671 16298 consensus_queue.cc:260] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 90f3da57796b488c9087a2f523adf045 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:16.286717 16298 raft_consensus.cc:3060] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:19:16.286820 17527 tablet_copy_client.cc:538] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P cb82cc7bd6db46d980c75237a1402860: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:19:16.287847 16310 tablet_copy_service.cc:342] P 90f3da57796b488c9087a2f523adf045: Request end of tablet copy session cb82cc7bd6db46d980c75237a1402860-434bff3082d141838068f323bdb0d1ed received from {username='slave'} at 127.0.0.1:40474
I20260709 13:19:16.288331 16310 tablet_copy_service.cc:434] P 90f3da57796b488c9087a2f523adf045: ending tablet copy session cb82cc7bd6db46d980c75237a1402860-434bff3082d141838068f323bdb0d1ed on tablet 434bff3082d141838068f323bdb0d1ed with peer cb82cc7bd6db46d980c75237a1402860
I20260709 13:19:16.309526 17527 tablet_bootstrap.cc:492] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P cb82cc7bd6db46d980c75237a1402860: Bootstrap starting.
I20260709 13:19:16.314299 16298 raft_consensus.cc:2468] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 2.
I20260709 13:19:16.315935 16820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e9b2bf7011f4ae688a79d7bb5a447b4" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:19:16.319852 16820 raft_consensus.cc:3060] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:19:16.317380 16145 leader_election.cc:290] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 2 election: Requested vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:19:16.340415 16820 raft_consensus.cc:2468] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 2.
I20260709 13:19:16.346408 16117 leader_election.cc:304] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:19:16.346556 17527 tablet_bootstrap.cc:492] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P cb82cc7bd6db46d980c75237a1402860: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:19:16.349751 17453 raft_consensus.cc:2804] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:19:16.350302 17453 raft_consensus.cc:697] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 647a343a542d4a139fe106c19095129b [term 2 LEADER]: Becoming Leader. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:19:16.353395 17527 tablet_bootstrap.cc:492] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P cb82cc7bd6db46d980c75237a1402860: Bootstrap complete.
I20260709 13:19:16.354271 17527 ts_tablet_manager.cc:1403] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P cb82cc7bd6db46d980c75237a1402860: Time spent bootstrapping tablet: real 0.045s	user 0.019s	sys 0.009s
I20260709 13:19:16.358208 17453 consensus_queue.cc:237] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:16.363075 17527 raft_consensus.cc:359] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P cb82cc7bd6db46d980c75237a1402860 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } }
I20260709 13:19:16.364375 17527 raft_consensus.cc:740] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P cb82cc7bd6db46d980c75237a1402860 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Initialized, Role: LEARNER
I20260709 13:19:16.365064 17527 consensus_queue.cc:260] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P cb82cc7bd6db46d980c75237a1402860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } }
I20260709 13:19:16.373409 17527 ts_tablet_manager.cc:1434] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P cb82cc7bd6db46d980c75237a1402860: Time spent starting tablet: real 0.019s	user 0.005s	sys 0.000s
I20260709 13:19:16.381431 16236 tablet_copy_service.cc:342] P 1ca2725d290e47d68b1a4b609c9816ac: Request end of tablet copy session cb82cc7bd6db46d980c75237a1402860-9e4bfb9090c8460eb14fa7e0d818e7f1 received from {username='slave'} at 127.0.0.1:33050
I20260709 13:19:16.381884 16236 tablet_copy_service.cc:434] P 1ca2725d290e47d68b1a4b609c9816ac: ending tablet copy session cb82cc7bd6db46d980c75237a1402860-9e4bfb9090c8460eb14fa7e0d818e7f1 on tablet 9e4bfb9090c8460eb14fa7e0d818e7f1 with peer cb82cc7bd6db46d980c75237a1402860
I20260709 13:19:16.388239 16046 catalog_manager.cc:5697] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 647a343a542d4a139fe106c19095129b reported cstate change: term changed from 1 to 2, leader changed from 90f3da57796b488c9087a2f523adf045 (127.14.177.131) to 647a343a542d4a139fe106c19095129b (127.14.177.129). New cstate: current_term: 2 leader_uuid: "647a343a542d4a139fe106c19095129b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:19:16.389457 17507 raft_consensus.cc:993] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b: : Instructing follower 1ca2725d290e47d68b1a4b609c9816ac to start an election
I20260709 13:19:16.390023 17507 raft_consensus.cc:1081] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b [term 1 LEADER]: Signalling peer 1ca2725d290e47d68b1a4b609c9816ac to start an election
I20260709 13:19:16.393697 16223 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "9ad5dbfa99ee4b8d9e28be92b3f36db5"
dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
 from {username='slave'} at 127.0.0.1:57604
I20260709 13:19:16.394217 16223 raft_consensus.cc:493] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:19:16.394539 16223 raft_consensus.cc:3060] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:19:16.402809 16223 raft_consensus.cc:515] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:16.406064 16145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ad5dbfa99ee4b8d9e28be92b3f36db5" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:19:16.406608 16145 raft_consensus.cc:3055] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:19:16.406876 16145 raft_consensus.cc:740] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:19:16.407626 16145 consensus_queue.cc:260] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:16.408643 16145 raft_consensus.cc:3060] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:19:16.413662 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ad5dbfa99ee4b8d9e28be92b3f36db5" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:19:16.414306 16299 raft_consensus.cc:3060] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:19:16.421900 16299 raft_consensus.cc:2468] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 2.
I20260709 13:19:16.422637 16223 leader_election.cc:290] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 2 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:16.428108 16145 raft_consensus.cc:2468] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 2.
I20260709 13:19:16.429620 16191 leader_election.cc:304] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
I20260709 13:19:16.430588 17511 raft_consensus.cc:2804] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:19:16.431054 17511 raft_consensus.cc:697] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 1ca2725d290e47d68b1a4b609c9816ac [term 2 LEADER]: Becoming Leader. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:19:16.432246 17511 consensus_queue.cc:237] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:16.456264 16047 catalog_manager.cc:5697] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 1ca2725d290e47d68b1a4b609c9816ac reported cstate change: term changed from 1 to 2, leader changed from 647a343a542d4a139fe106c19095129b (127.14.177.129) to 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130). New cstate: current_term: 2 leader_uuid: "1ca2725d290e47d68b1a4b609c9816ac" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } health_report { overall_health: UNKNOWN } } }
I20260709 13:19:16.491753 17511 raft_consensus.cc:993] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac: : Instructing follower cb82cc7bd6db46d980c75237a1402860 to start an election
I20260709 13:19:16.492290 17511 raft_consensus.cc:1081] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [term 1 LEADER]: Signalling peer cb82cc7bd6db46d980c75237a1402860 to start an election
I20260709 13:19:16.505789 16825 raft_consensus.cc:1217] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P cb82cc7bd6db46d980c75237a1402860 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:19:16.508564 16820 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "66ae176961b74813a58bc15c9c4c6f2a"
dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
 from {username='slave'} at 127.0.0.1:49906
I20260709 13:19:16.509174 16820 raft_consensus.cc:493] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:19:16.509533 16820 raft_consensus.cc:3060] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:19:16.533156 16825 raft_consensus.cc:1217] T 8c11074579b046d99e585c2818e18a2c P cb82cc7bd6db46d980c75237a1402860 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:19:16.553735 16824 raft_consensus.cc:1217] T 434bff3082d141838068f323bdb0d1ed P cb82cc7bd6db46d980c75237a1402860 [term 2 LEARNER]: Deduplicated request from leader. Original: 2.2->[2.3-2.3]   Dedup: 2.3->[]
I20260709 13:19:16.601643 16820 raft_consensus.cc:515] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:16.618659 16820 leader_election.cc:290] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 2 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:16.621696 16223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66ae176961b74813a58bc15c9c4c6f2a" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:19:16.622417 16223 raft_consensus.cc:3055] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:19:16.622731 16223 raft_consensus.cc:740] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:19:16.623587 16223 consensus_queue.cc:260] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:16.625170 16223 raft_consensus.cc:3060] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:19:16.631949 16825 raft_consensus.cc:1217] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:19:16.707397 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66ae176961b74813a58bc15c9c4c6f2a" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:19:16.708446 16298 raft_consensus.cc:3060] T 66ae176961b74813a58bc15c9c4c6f2a P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:19:16.752243 16820 raft_consensus.cc:1275] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860 [term 2 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:19:16.755581 17453 consensus_queue.cc:1048] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:19:16.771917 16223 raft_consensus.cc:2468] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 2.
I20260709 13:19:16.773828 16790 leader_election.cc:304] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:19:16.775638 17317 raft_consensus.cc:2804] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:19:16.776173 17317 raft_consensus.cc:697] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [term 2 LEADER]: Becoming Leader. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:19:16.777084 17317 consensus_queue.cc:237] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:16.786718 16298 raft_consensus.cc:2468] T 66ae176961b74813a58bc15c9c4c6f2a P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 2.
I20260709 13:19:16.794610 16295 raft_consensus.cc:1275] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:19:16.796514 17453 consensus_queue.cc:1048] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:19:16.837728 16046 catalog_manager.cc:5697] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 reported cstate change: term changed from 1 to 2, leader changed from 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130) to cb82cc7bd6db46d980c75237a1402860 (127.14.177.132). New cstate: current_term: 2 leader_uuid: "cb82cc7bd6db46d980c75237a1402860" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:19:16.881891 16298 raft_consensus.cc:1275] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:19:16.884951 17411 consensus_queue.cc:1048] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:19:16.902280 17511 raft_consensus.cc:1064] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 1ca2725d290e47d68b1a4b609c9816ac: attempting to promote NON_VOTER cb82cc7bd6db46d980c75237a1402860 to VOTER
I20260709 13:19:16.905537 17511 consensus_queue.cc:237] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:16.922365 16145 raft_consensus.cc:1275] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:19:16.925129 16820 raft_consensus.cc:1275] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P cb82cc7bd6db46d980c75237a1402860 [term 1 LEARNER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:19:16.928056 17400 consensus_queue.cc:1048] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:19:16.929693 17400 consensus_queue.cc:1048] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:19:16.941352 16299 raft_consensus.cc:1275] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:19:16.942899 17411 consensus_queue.cc:1048] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:19:17.016433 16298 raft_consensus.cc:2955] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:19:17.020514 17548 raft_consensus.cc:1064] T 8c11074579b046d99e585c2818e18a2c P 1ca2725d290e47d68b1a4b609c9816ac: attempting to promote NON_VOTER cb82cc7bd6db46d980c75237a1402860 to VOTER
I20260709 13:19:17.013207 17411 raft_consensus.cc:2955] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:19:17.033833 17548 consensus_queue.cc:237] T 8c11074579b046d99e585c2818e18a2c P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:17.055332 16047 catalog_manager.cc:5697] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045 reported cstate change: config changed from index 2 to 3, cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "1ca2725d290e47d68b1a4b609c9816ac" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:19:17.070111 15046 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:19:17.105458 16145 raft_consensus.cc:1275] T 8c11074579b046d99e585c2818e18a2c P 647a343a542d4a139fe106c19095129b [term 1 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:19:17.103963 16820 raft_consensus.cc:2955] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P cb82cc7bd6db46d980c75237a1402860 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:19:17.108981 16825 raft_consensus.cc:1275] T 8c11074579b046d99e585c2818e18a2c P cb82cc7bd6db46d980c75237a1402860 [term 1 LEARNER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:19:17.111869 16296 raft_consensus.cc:1275] T 8c11074579b046d99e585c2818e18a2c P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:19:17.114176 17557 consensus_queue.cc:1048] T 8c11074579b046d99e585c2818e18a2c P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:19:17.115038 17550 consensus_queue.cc:1048] T 8c11074579b046d99e585c2818e18a2c P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.010s
I20260709 13:19:17.117168 16145 raft_consensus.cc:1275] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 647a343a542d4a139fe106c19095129b [term 1 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:19:17.127566 17548 consensus_queue.cc:1048] T 8c11074579b046d99e585c2818e18a2c P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:19:17.135614 17553 raft_consensus.cc:1064] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b: attempting to promote NON_VOTER cb82cc7bd6db46d980c75237a1402860 to VOTER
I20260709 13:19:17.139070 17553 consensus_queue.cc:237] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:17.144049 17518 raft_consensus.cc:1064] T 434bff3082d141838068f323bdb0d1ed P 90f3da57796b488c9087a2f523adf045: attempting to promote NON_VOTER cb82cc7bd6db46d980c75237a1402860 to VOTER
I20260709 13:19:17.148075 17555 consensus_queue.cc:1048] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:19:17.147346 17518 consensus_queue.cc:237] T 434bff3082d141838068f323bdb0d1ed P 90f3da57796b488c9087a2f523adf045 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 1, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:17.205116 16150 raft_consensus.cc:2955] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 647a343a542d4a139fe106c19095129b [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:19:17.248256 16222 raft_consensus.cc:1275] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:19:17.253129 17317 consensus_queue.cc:1048] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.002s
I20260709 13:19:17.271658 16223 consensus_queue.cc:237] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:17.294600 16824 raft_consensus.cc:1275] T 434bff3082d141838068f323bdb0d1ed P cb82cc7bd6db46d980c75237a1402860 [term 2 LEARNER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
W20260709 13:19:17.363402 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 62825.2us
W20260709 13:19:17.363613 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 184076us
I20260709 13:19:17.370894 16824 raft_consensus.cc:1275] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P cb82cc7bd6db46d980c75237a1402860 [term 1 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:19:17.377067 16224 raft_consensus.cc:1275] T 434bff3082d141838068f323bdb0d1ed P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
W20260709 13:19:17.387929 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 88932.9us
I20260709 13:19:17.390023 16295 raft_consensus.cc:1275] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
W20260709 13:19:17.392664 16117 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 95278us
W20260709 13:19:17.400676 17559 log.cc:927] Time spent T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 0.172s	user 0.001s	sys 0.000s
I20260709 13:19:17.358904 17567 consensus_queue.cc:1048] T 434bff3082d141838068f323bdb0d1ed P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.090s
I20260709 13:19:17.424925 16295 raft_consensus.cc:1275] T 66ae176961b74813a58bc15c9c4c6f2a P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:19:17.407613 17400 consensus_queue.cc:1048] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.010s
I20260709 13:19:17.446080 17317 consensus_queue.cc:1048] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
W20260709 13:19:17.522078 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 44118.1us
W20260709 13:19:17.526391 17576 log.cc:927] Time spent T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 0.068s	user 0.000s	sys 0.000s
I20260709 13:19:17.523558 17556 consensus_queue.cc:1048] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.120s
W20260709 13:19:17.675449 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 117114us
I20260709 13:19:17.713490 15046 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:19:17.747882 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 161608us
W20260709 13:19:17.788002 16789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 81917us
I20260709 13:19:17.791630 16825 raft_consensus.cc:1275] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860 [term 1 LEARNER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
W20260709 13:19:17.807061 17577 log.cc:927] Time spent T 8c11074579b046d99e585c2818e18a2c P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
I20260709 13:19:17.800205 17571 consensus_queue.cc:1048] T 434bff3082d141838068f323bdb0d1ed P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.261s
I20260709 13:19:17.853467 16825 raft_consensus.cc:2955] T 8c11074579b046d99e585c2818e18a2c P cb82cc7bd6db46d980c75237a1402860 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:19:17.853832 17583 raft_consensus.cc:2955] T 8c11074579b046d99e585c2818e18a2c P 1ca2725d290e47d68b1a4b609c9816ac [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
W20260709 13:19:17.886610 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 186350us
W20260709 13:19:17.969986 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 397206us
I20260709 13:19:17.976058 17414 consensus_queue.cc:1048] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
W20260709 13:19:17.971647 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 120016us
W20260709 13:19:18.079312 16789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 253369us
W20260709 13:19:18.083873 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 75536.8us
I20260709 13:19:18.120631 16295 raft_consensus.cc:2955] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 647a343a542d4a139fe106c19095129b (127.14.177.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:19:18.124586 16296 raft_consensus.cc:2955] T 8c11074579b046d99e585c2818e18a2c P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:19:18.133183 17511 raft_consensus.cc:2955] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 647a343a542d4a139fe106c19095129b (127.14.177.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:19:18.143653 16145 raft_consensus.cc:1275] T 434bff3082d141838068f323bdb0d1ed P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 13:19:18.192705 17606 consensus_queue.cc:1048] T 434bff3082d141838068f323bdb0d1ed P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.030s
I20260709 13:19:18.217068 16222 raft_consensus.cc:1275] T 0a9de347c711477e87c582d003f36925 P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:19:18.222728 17553 consensus_queue.cc:1048] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.003s
I20260709 13:19:18.231227 16145 raft_consensus.cc:2955] T 8c11074579b046d99e585c2818e18a2c P 647a343a542d4a139fe106c19095129b [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
W20260709 13:19:18.304718 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 125613us
I20260709 13:19:18.329931 16046 catalog_manager.cc:5697] T 8c11074579b046d99e585c2818e18a2c P 90f3da57796b488c9087a2f523adf045 reported cstate change: config changed from index 2 to 3, cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "1ca2725d290e47d68b1a4b609c9816ac" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:19:18.314767 17571 raft_consensus.cc:493] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:19:18.385037 17562 raft_consensus.cc:2955] T 434bff3082d141838068f323bdb0d1ed P 90f3da57796b488c9087a2f523adf045 [term 2 LEADER]: Committing config change with OpId 2.4: config changed from index 3 to 4, cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:19:18.357591 17571 raft_consensus.cc:515] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } }
I20260709 13:19:18.408850 17571 leader_election.cc:290] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:19:18.411545 16300 raft_consensus.cc:1275] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:19:18.412253 16224 raft_consensus.cc:2955] T 434bff3082d141838068f323bdb0d1ed P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
W20260709 13:19:18.422406 16789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 239112us
I20260709 13:19:18.496215 16225 raft_consensus.cc:2955] T 0a9de347c711477e87c582d003f36925 P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:19:18.481645 17563 raft_consensus.cc:2955] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:19:18.518064 16151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a9de347c711477e87c582d003f36925" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:19:18.521813 16145 raft_consensus.cc:2955] T 434bff3082d141838068f323bdb0d1ed P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:19:18.543774 16824 raft_consensus.cc:2955] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:19:18.557672 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a9de347c711477e87c582d003f36925" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:18.588086 16825 raft_consensus.cc:2955] T 434bff3082d141838068f323bdb0d1ed P cb82cc7bd6db46d980c75237a1402860 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:19:18.599884 17511 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.ChangeConfig from 127.0.0.1:57598 (request call id 78) took 1339 ms. Trace:
I20260709 13:19:18.573418 17573 consensus_queue.cc:1048] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.084s
I20260709 13:19:18.651794 16265 leader_election.cc:304] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:19:18.651073 16823 raft_consensus.cc:2955] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P cb82cc7bd6db46d980c75237a1402860 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 647a343a542d4a139fe106c19095129b (127.14.177.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:19:18.652570 17606 raft_consensus.cc:2749] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:19:18.666711 17573 raft_consensus.cc:493] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 647a343a542d4a139fe106c19095129b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:19:18.646661 17511 rpcz_store.cc:276] 0709 13:19:17.264886 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:17.268114 (+  3228us) service_pool.cc:224] Handling call
0709 13:19:17.277552 (+  9438us) log.cc:844] Serialized 508 byte log entry
0709 13:19:17.386184 (+108632us) spinlock_profiling.cc:243] Waited 2.2 ms on lock 0x7b18002143c0. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b693ade1 00007f69b69475b1 00007f69ad9ecc0a 00007f69b712c219 00007f69ada74a40 00007f69ada6f4e4 00007f69ada6f455 00007f69b0ab8b0b 00007f69b0b3db63 00007f69b0b65752 00007f69b0b5d9fa
0709 13:19:18.589167 (+1202983us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":76036,"raft.queue_time_us":109442,"raft.run_cpu_time_us":1407,"raft.run_wall_time_us":31732,"spinlock_wait_cycles":5051648}
I20260709 13:19:18.667493 17573 raft_consensus.cc:515] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 647a343a542d4a139fe106c19095129b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:18.668406 16035 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 9e4bfb9090c8460eb14fa7e0d818e7f1 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:19:18.672503 16147 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0a9de347c711477e87c582d003f36925"
dest_uuid: "647a343a542d4a139fe106c19095129b"
mode: GRACEFUL
new_leader_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
 from {username='slave'} at 127.0.0.1:40290
I20260709 13:19:18.674405 17573 leader_election.cc:290] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:19:18.675454 16823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e4bfb9090c8460eb14fa7e0d818e7f1" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:19:18.671797 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e4bfb9090c8460eb14fa7e0d818e7f1" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:18.691872 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e4bfb9090c8460eb14fa7e0d818e7f1" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:18.693775 16116 leader_election.cc:304] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860
I20260709 13:19:18.694999 17573 raft_consensus.cc:2749] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 647a343a542d4a139fe106c19095129b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:19:18.701246 16147 raft_consensus.cc:606] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b [term 1 LEADER]: Received request to transfer leadership to TS 1ca2725d290e47d68b1a4b609c9816ac
I20260709 13:19:18.702767 16297 raft_consensus.cc:2955] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:19:18.708293 16147 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9e65676b21b748728ad39668cfe722ec"
dest_uuid: "647a343a542d4a139fe106c19095129b"
mode: GRACEFUL
new_leader_uuid: "90f3da57796b488c9087a2f523adf045"
 from {username='slave'} at 127.0.0.1:40290
I20260709 13:19:18.709040 16147 raft_consensus.cc:606] T 9e65676b21b748728ad39668cfe722ec P 647a343a542d4a139fe106c19095129b [term 1 LEADER]: Received request to transfer leadership to TS 90f3da57796b488c9087a2f523adf045
I20260709 13:19:18.713425 16224 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c9944d5e4b8c4109af9955c4f08c9095"
dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
mode: GRACEFUL
new_leader_uuid: "90f3da57796b488c9087a2f523adf045"
 from {username='slave'} at 127.0.0.1:39144
I20260709 13:19:18.714221 16224 raft_consensus.cc:606] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac [term 1 LEADER]: Received request to transfer leadership to TS 90f3da57796b488c9087a2f523adf045
I20260709 13:19:18.716136 15046 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 13:19:18.717738 15046 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:19:18.735018 16224 consensus_queue.cc:237] T 8c11074579b046d99e585c2818e18a2c P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:18.745232 16823 raft_consensus.cc:1275] T 8c11074579b046d99e585c2818e18a2c P cb82cc7bd6db46d980c75237a1402860 [term 1 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:19:18.745232 16297 raft_consensus.cc:1275] T 8c11074579b046d99e585c2818e18a2c P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:19:18.750516 17588 consensus_queue.cc:1048] T 8c11074579b046d99e585c2818e18a2c P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:19:18.751686 17556 consensus_queue.cc:1048] T 8c11074579b046d99e585c2818e18a2c P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:19:18.746440 16045 catalog_manager.cc:5697] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b reported cstate change: config changed from index 2 to 3, cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "647a343a542d4a139fe106c19095129b" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:19:18.766726 17511 raft_consensus.cc:2955] T 8c11074579b046d99e585c2818e18a2c P 1ca2725d290e47d68b1a4b609c9816ac [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 647a343a542d4a139fe106c19095129b (127.14.177.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:19:18.768131 16297 raft_consensus.cc:2955] T 8c11074579b046d99e585c2818e18a2c P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 647a343a542d4a139fe106c19095129b (127.14.177.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:19:18.784269 17588 raft_consensus.cc:993] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac: : Instructing follower 90f3da57796b488c9087a2f523adf045 to start an election
I20260709 13:19:18.784797 17588 raft_consensus.cc:1081] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac [term 1 LEADER]: Signalling peer 90f3da57796b488c9087a2f523adf045 to start an election
I20260709 13:19:18.786580 16298 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c9944d5e4b8c4109af9955c4f08c9095"
dest_uuid: "90f3da57796b488c9087a2f523adf045"
 from {username='slave'} at 127.0.0.1:48684
I20260709 13:19:18.787112 16298 raft_consensus.cc:493] T c9944d5e4b8c4109af9955c4f08c9095 P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:19:18.787482 16298 raft_consensus.cc:3060] T c9944d5e4b8c4109af9955c4f08c9095 P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:19:18.774446 16824 raft_consensus.cc:2955] T 8c11074579b046d99e585c2818e18a2c P cb82cc7bd6db46d980c75237a1402860 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 647a343a542d4a139fe106c19095129b (127.14.177.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:19:18.793972 16035 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 8c11074579b046d99e585c2818e18a2c with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:19:18.804567 16298 raft_consensus.cc:515] T c9944d5e4b8c4109af9955c4f08c9095 P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:18.837033 16046 catalog_manager.cc:5697] T 434bff3082d141838068f323bdb0d1ed P cb82cc7bd6db46d980c75237a1402860 reported cstate change: config changed from index 3 to 4, cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) changed from NON_VOTER to VOTER. New cstate: current_term: 2 leader_uuid: "90f3da57796b488c9087a2f523adf045" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:19:18.840122 16046 catalog_manager.cc:5697] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P cb82cc7bd6db46d980c75237a1402860 reported cstate change: config changed from index 3 to 4, VOTER 647a343a542d4a139fe106c19095129b (127.14.177.129) evicted. New cstate: current_term: 1 leader_uuid: "1ca2725d290e47d68b1a4b609c9816ac" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:19:18.850621 16825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9944d5e4b8c4109af9955c4f08c9095" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:19:18.851282 16825 raft_consensus.cc:3060] T c9944d5e4b8c4109af9955c4f08c9095 P cb82cc7bd6db46d980c75237a1402860 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:19:18.851673 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9944d5e4b8c4109af9955c4f08c9095" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:19:18.852210 16224 raft_consensus.cc:3055] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:19:18.852473 16224 raft_consensus.cc:740] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:19:18.853209 16224 consensus_queue.cc:260] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:18.854192 16224 raft_consensus.cc:3060] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:19:18.855610 16298 leader_election.cc:290] T c9944d5e4b8c4109af9955c4f08c9095 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 2 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:19:18.882221 16224 raft_consensus.cc:2468] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 2.
I20260709 13:19:18.882833 16825 raft_consensus.cc:2468] T c9944d5e4b8c4109af9955c4f08c9095 P cb82cc7bd6db46d980c75237a1402860 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 2.
I20260709 13:19:18.883589 16265 leader_election.cc:304] T c9944d5e4b8c4109af9955c4f08c9095 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:19:18.884549 17586 raft_consensus.cc:2804] T c9944d5e4b8c4109af9955c4f08c9095 P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:19:18.885000 17586 raft_consensus.cc:697] T c9944d5e4b8c4109af9955c4f08c9095 P 90f3da57796b488c9087a2f523adf045 [term 2 LEADER]: Becoming Leader. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:19:18.885892 17586 consensus_queue.cc:237] T c9944d5e4b8c4109af9955c4f08c9095 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:18.890012 16136 tablet_service.cc:1558] Processing DeleteTablet for tablet 9e4bfb9090c8460eb14fa7e0d818e7f1 with delete_type TABLET_DATA_TOMBSTONED (TS 647a343a542d4a139fe106c19095129b not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:40822
I20260709 13:19:18.901222 17573 raft_consensus.cc:993] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b: : Instructing follower 1ca2725d290e47d68b1a4b609c9816ac to start an election
I20260709 13:19:18.901750 17563 raft_consensus.cc:1081] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b [term 1 LEADER]: Signalling peer 1ca2725d290e47d68b1a4b609c9816ac to start an election
I20260709 13:19:18.908782 16048 catalog_manager.cc:5697] T 8c11074579b046d99e585c2818e18a2c P cb82cc7bd6db46d980c75237a1402860 reported cstate change: config changed from index 3 to 4, VOTER 647a343a542d4a139fe106c19095129b (127.14.177.129) evicted. New cstate: current_term: 1 leader_uuid: "1ca2725d290e47d68b1a4b609c9816ac" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:19:18.911358 16222 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "0a9de347c711477e87c582d003f36925"
dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
 from {username='slave'} at 127.0.0.1:57604
I20260709 13:19:18.916198 16222 raft_consensus.cc:493] T 0a9de347c711477e87c582d003f36925 P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:19:18.916589 16222 raft_consensus.cc:3060] T 0a9de347c711477e87c582d003f36925 P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:19:18.930996 16133 tablet_service.cc:1558] Processing DeleteTablet for tablet 8c11074579b046d99e585c2818e18a2c with delete_type TABLET_DATA_TOMBSTONED (TS 647a343a542d4a139fe106c19095129b not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:40822
I20260709 13:19:18.936239 16222 raft_consensus.cc:515] T 0a9de347c711477e87c582d003f36925 P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:18.938201 17625 tablet_replica.cc:333] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 647a343a542d4a139fe106c19095129b: stopping tablet replica
I20260709 13:19:18.952306 17625 raft_consensus.cc:2243] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 647a343a542d4a139fe106c19095129b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:19:18.953256 17625 raft_consensus.cc:2272] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 647a343a542d4a139fe106c19095129b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:19:18.955235 16147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a9de347c711477e87c582d003f36925" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:19:18.955839 16147 raft_consensus.cc:3055] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:19:18.956123 16147 raft_consensus.cc:740] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:19:18.957095 17625 ts_tablet_manager.cc:1916] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 647a343a542d4a139fe106c19095129b: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:19:18.956868 16147 consensus_queue.cc:260] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:18.958246 16147 raft_consensus.cc:3060] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:19:18.966277 17565 raft_consensus.cc:993] T 9e65676b21b748728ad39668cfe722ec P 647a343a542d4a139fe106c19095129b: : Instructing follower 90f3da57796b488c9087a2f523adf045 to start an election
I20260709 13:19:18.966789 17565 raft_consensus.cc:1081] T 9e65676b21b748728ad39668cfe722ec P 647a343a542d4a139fe106c19095129b [term 1 LEADER]: Signalling peer 90f3da57796b488c9087a2f523adf045 to start an election
I20260709 13:19:18.969123 16147 raft_consensus.cc:2468] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 2.
I20260709 13:19:18.972200 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a9de347c711477e87c582d003f36925" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:19:18.972787 16297 raft_consensus.cc:3060] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:19:18.973598 16300 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "9e65676b21b748728ad39668cfe722ec"
dest_uuid: "90f3da57796b488c9087a2f523adf045"
 from {username='slave'} at 127.0.0.1:48674
I20260709 13:19:18.974213 16300 raft_consensus.cc:493] T 9e65676b21b748728ad39668cfe722ec P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:19:18.974581 16300 raft_consensus.cc:3060] T 9e65676b21b748728ad39668cfe722ec P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:19:18.982086 17625 ts_tablet_manager.cc:1929] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 647a343a542d4a139fe106c19095129b: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:19:18.982669 17625 log.cc:1199] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 647a343a542d4a139fe106c19095129b: Deleting WAL directory at /tmp/dist-test-taskdH_Ma7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783603093801613-15046-0/minicluster-data/ts-0-root/wals/9e4bfb9090c8460eb14fa7e0d818e7f1
I20260709 13:19:18.984598 16035 catalog_manager.cc:5028] TS 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): tablet 9e4bfb9090c8460eb14fa7e0d818e7f1 (table test-workload [id=ed3d15d8c9ad46bebc4fe3a402d2e54f]) successfully deleted
I20260709 13:19:18.985134 16297 raft_consensus.cc:2468] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 2.
I20260709 13:19:18.986012 17625 tablet_replica.cc:333] T 8c11074579b046d99e585c2818e18a2c P 647a343a542d4a139fe106c19095129b: stopping tablet replica
I20260709 13:19:18.997993 17625 raft_consensus.cc:2243] T 8c11074579b046d99e585c2818e18a2c P 647a343a542d4a139fe106c19095129b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:19:18.999553 16190 leader_election.cc:304] T 0a9de347c711477e87c582d003f36925 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:19:19.000355 17625 raft_consensus.cc:2272] T 8c11074579b046d99e585c2818e18a2c P 647a343a542d4a139fe106c19095129b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:19:18.949395 16298 consensus_queue.cc:237] T 434bff3082d141838068f323bdb0d1ed P 90f3da57796b488c9087a2f523adf045 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.4, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:18.987519 16222 leader_election.cc:290] T 0a9de347c711477e87c582d003f36925 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 2 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:19:18.989689 16825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a9de347c711477e87c582d003f36925" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:19:19.000481 17511 raft_consensus.cc:2804] T 0a9de347c711477e87c582d003f36925 P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:19:19.006493 17511 raft_consensus.cc:697] T 0a9de347c711477e87c582d003f36925 P 1ca2725d290e47d68b1a4b609c9816ac [term 2 LEADER]: Becoming Leader. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:19:19.007179 17625 ts_tablet_manager.cc:1916] T 8c11074579b046d99e585c2818e18a2c P 647a343a542d4a139fe106c19095129b: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:19:19.007441 17511 consensus_queue.cc:237] T 0a9de347c711477e87c582d003f36925 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:18.985574 16300 raft_consensus.cc:515] T 9e65676b21b748728ad39668cfe722ec P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:19.015301 16300 leader_election.cc:290] T 9e65676b21b748728ad39668cfe722ec P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 2 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:19:19.015997 16151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e65676b21b748728ad39668cfe722ec" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:19:19.017529 16151 raft_consensus.cc:3055] T 9e65676b21b748728ad39668cfe722ec P 647a343a542d4a139fe106c19095129b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:19:19.017829 16151 raft_consensus.cc:740] T 9e65676b21b748728ad39668cfe722ec P 647a343a542d4a139fe106c19095129b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:19:19.018535 16151 consensus_queue.cc:260] T 9e65676b21b748728ad39668cfe722ec P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:19.019593 16151 raft_consensus.cc:3060] T 9e65676b21b748728ad39668cfe722ec P 647a343a542d4a139fe106c19095129b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:19:19.016853 16820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e65676b21b748728ad39668cfe722ec" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:19:19.023497 16820 raft_consensus.cc:3060] T 9e65676b21b748728ad39668cfe722ec P cb82cc7bd6db46d980c75237a1402860 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:19:19.025182 16048 catalog_manager.cc:5697] T 0a9de347c711477e87c582d003f36925 P 1ca2725d290e47d68b1a4b609c9816ac reported cstate change: term changed from 1 to 2, leader changed from 647a343a542d4a139fe106c19095129b (127.14.177.129) to 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130). New cstate: current_term: 2 leader_uuid: "1ca2725d290e47d68b1a4b609c9816ac" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:19:19.031409 16820 raft_consensus.cc:2468] T 9e65676b21b748728ad39668cfe722ec P cb82cc7bd6db46d980c75237a1402860 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 2.
I20260709 13:19:19.032696 16266 leader_election.cc:304] T 9e65676b21b748728ad39668cfe722ec P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:19:19.034897 17580 raft_consensus.cc:2804] T 9e65676b21b748728ad39668cfe722ec P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:19:19.035756 17580 raft_consensus.cc:697] T 9e65676b21b748728ad39668cfe722ec P 90f3da57796b488c9087a2f523adf045 [term 2 LEADER]: Becoming Leader. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:19:19.037041 16151 raft_consensus.cc:2468] T 9e65676b21b748728ad39668cfe722ec P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 2.
I20260709 13:19:19.037943 16825 raft_consensus.cc:3060] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:19:19.036701 17580 consensus_queue.cc:237] T 9e65676b21b748728ad39668cfe722ec P 90f3da57796b488c9087a2f523adf045 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:19.047883 16823 raft_consensus.cc:1275] T 434bff3082d141838068f323bdb0d1ed P cb82cc7bd6db46d980c75237a1402860 [term 2 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:19:19.048018 16151 raft_consensus.cc:1275] T 434bff3082d141838068f323bdb0d1ed P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:19:19.050917 17518 consensus_queue.cc:1048] T 434bff3082d141838068f323bdb0d1ed P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:19:19.055613 17606 consensus_queue.cc:1048] T 434bff3082d141838068f323bdb0d1ed P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:19:19.060946 17625 ts_tablet_manager.cc:1929] T 8c11074579b046d99e585c2818e18a2c P 647a343a542d4a139fe106c19095129b: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:19:19.063756 17625 log.cc:1199] T 8c11074579b046d99e585c2818e18a2c P 647a343a542d4a139fe106c19095129b: Deleting WAL directory at /tmp/dist-test-taskdH_Ma7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783603093801613-15046-0/minicluster-data/ts-0-root/wals/8c11074579b046d99e585c2818e18a2c
I20260709 13:19:19.066066 16035 catalog_manager.cc:5028] TS 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): tablet 8c11074579b046d99e585c2818e18a2c (table test-workload [id=ed3d15d8c9ad46bebc4fe3a402d2e54f]) successfully deleted
I20260709 13:19:19.066768 16825 raft_consensus.cc:2468] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 2.
I20260709 13:19:19.072783 17586 raft_consensus.cc:2955] T 434bff3082d141838068f323bdb0d1ed P 90f3da57796b488c9087a2f523adf045 [term 2 LEADER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:19:19.075645 16044 catalog_manager.cc:5697] T c9944d5e4b8c4109af9955c4f08c9095 P 90f3da57796b488c9087a2f523adf045 reported cstate change: term changed from 1 to 2, leader changed from 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130) to 90f3da57796b488c9087a2f523adf045 (127.14.177.131). New cstate: current_term: 2 leader_uuid: "90f3da57796b488c9087a2f523adf045" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:19:19.091547 16151 raft_consensus.cc:2955] T 434bff3082d141838068f323bdb0d1ed P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:19:19.094309 16224 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0a9de347c711477e87c582d003f36925"
dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:57598
I20260709 13:19:19.094902 16224 raft_consensus.cc:606] T 0a9de347c711477e87c582d003f36925 P 1ca2725d290e47d68b1a4b609c9816ac [term 2 LEADER]: Received request to transfer leadership
I20260709 13:19:19.101310 16224 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0a9de347c711477e87c582d003f36925"
dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:57598
I20260709 13:19:19.102283 16224 raft_consensus.cc:606] T 0a9de347c711477e87c582d003f36925 P 1ca2725d290e47d68b1a4b609c9816ac [term 2 LEADER]: Received request to transfer leadership
W20260709 13:19:19.103782 16035 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: leadership transfer for 0a9de347c711477e87c582d003f36925 already in progress [suppressed 1 similar messages]
I20260709 13:19:19.105866 16034 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 434bff3082d141838068f323bdb0d1ed with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:19:19.111824 16825 raft_consensus.cc:2955] T 434bff3082d141838068f323bdb0d1ed P cb82cc7bd6db46d980c75237a1402860 [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:19:19.142266 16044 catalog_manager.cc:5697] T 434bff3082d141838068f323bdb0d1ed P cb82cc7bd6db46d980c75237a1402860 reported cstate change: config changed from index 4 to 5, VOTER 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130) evicted. New cstate: current_term: 2 leader_uuid: "90f3da57796b488c9087a2f523adf045" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:19:19.157305 16224 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0a9de347c711477e87c582d003f36925"
dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:57598
I20260709 13:19:19.157845 16224 raft_consensus.cc:606] T 0a9de347c711477e87c582d003f36925 P 1ca2725d290e47d68b1a4b609c9816ac [term 2 LEADER]: Received request to transfer leadership
W20260709 13:19:19.186666 16097 auto_rebalancer.cc:893] Removing replace marker failed: Service unavailable: leader transfer in progress
W20260709 13:19:19.187196 16097 auto_rebalancer.cc:901] Could not move replica: Remote error: Service unavailable: leadership transfer for 0a9de347c711477e87c582d003f36925 already in progress
W20260709 13:19:19.187487 16097 auto_rebalancer.cc:330] scheduled replica move failed to complete: Remote error: Service unavailable: leadership transfer for 0a9de347c711477e87c582d003f36925 already in progress
I20260709 13:19:19.201646 16209 tablet_service.cc:1558] Processing DeleteTablet for tablet 434bff3082d141838068f323bdb0d1ed with delete_type TABLET_DATA_TOMBSTONED (TS 1ca2725d290e47d68b1a4b609c9816ac not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:57598
I20260709 13:19:19.209726 16048 catalog_manager.cc:5697] T 9e65676b21b748728ad39668cfe722ec P 90f3da57796b488c9087a2f523adf045 reported cstate change: term changed from 1 to 2, leader changed from 647a343a542d4a139fe106c19095129b (127.14.177.129) to 90f3da57796b488c9087a2f523adf045 (127.14.177.131). New cstate: current_term: 2 leader_uuid: "90f3da57796b488c9087a2f523adf045" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:19:19.275226 17629 tablet_replica.cc:333] T 434bff3082d141838068f323bdb0d1ed P 1ca2725d290e47d68b1a4b609c9816ac: stopping tablet replica
I20260709 13:19:19.276567 17629 raft_consensus.cc:2243] T 434bff3082d141838068f323bdb0d1ed P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:19:19.277480 17629 raft_consensus.cc:2272] T 434bff3082d141838068f323bdb0d1ed P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:19:19.290338 17629 ts_tablet_manager.cc:1916] T 434bff3082d141838068f323bdb0d1ed P 1ca2725d290e47d68b1a4b609c9816ac: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:19:19.305049 17629 ts_tablet_manager.cc:1929] T 434bff3082d141838068f323bdb0d1ed P 1ca2725d290e47d68b1a4b609c9816ac: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.4
I20260709 13:19:19.305434 17629 log.cc:1199] T 434bff3082d141838068f323bdb0d1ed P 1ca2725d290e47d68b1a4b609c9816ac: Deleting WAL directory at /tmp/dist-test-taskdH_Ma7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783603093801613-15046-0/minicluster-data/ts-1-root/wals/434bff3082d141838068f323bdb0d1ed
I20260709 13:19:19.307338 16035 catalog_manager.cc:5028] TS 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943): tablet 434bff3082d141838068f323bdb0d1ed (table test-workload [id=ed3d15d8c9ad46bebc4fe3a402d2e54f]) successfully deleted
I20260709 13:19:19.345214 16225 raft_consensus.cc:1275] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:19:19.346736 17586 consensus_queue.cc:1048] T c9944d5e4b8c4109af9955c4f08c9095 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:19:19.379631 16823 raft_consensus.cc:1275] T c9944d5e4b8c4109af9955c4f08c9095 P cb82cc7bd6db46d980c75237a1402860 [term 2 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:19:19.381121 17518 consensus_queue.cc:1048] T c9944d5e4b8c4109af9955c4f08c9095 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:19:19.426410 16147 raft_consensus.cc:1275] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 13:19:19.428674 17593 consensus_queue.cc:1048] T 0a9de347c711477e87c582d003f36925 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:19:19.438443 17557 raft_consensus.cc:993] T 0a9de347c711477e87c582d003f36925 P 1ca2725d290e47d68b1a4b609c9816ac: : Instructing follower 647a343a542d4a139fe106c19095129b to start an election
I20260709 13:19:19.438951 17593 raft_consensus.cc:1081] T 0a9de347c711477e87c582d003f36925 P 1ca2725d290e47d68b1a4b609c9816ac [term 2 LEADER]: Signalling peer 647a343a542d4a139fe106c19095129b to start an election
I20260709 13:19:19.446585 16147 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "0a9de347c711477e87c582d003f36925"
dest_uuid: "647a343a542d4a139fe106c19095129b"
 from {username='slave'} at 127.0.0.1:40844
I20260709 13:19:19.451349 16147 raft_consensus.cc:493] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:19:19.451689 16147 raft_consensus.cc:3060] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:19:19.458134 16147 raft_consensus.cc:515] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:19.460736 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a9de347c711477e87c582d003f36925" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: true dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:19:19.460846 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a9de347c711477e87c582d003f36925" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: true dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:19:19.461079 16825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a9de347c711477e87c582d003f36925" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: true dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:19:19.461354 16225 raft_consensus.cc:3055] T 0a9de347c711477e87c582d003f36925 P 1ca2725d290e47d68b1a4b609c9816ac [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:19:19.461418 16297 raft_consensus.cc:3060] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:19:19.461707 16147 leader_election.cc:290] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 3 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:19:19.461910 16225 raft_consensus.cc:740] T 0a9de347c711477e87c582d003f36925 P 1ca2725d290e47d68b1a4b609c9816ac [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:19:19.461592 16825 raft_consensus.cc:3060] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:19:19.462786 16225 consensus_queue.cc:260] T 0a9de347c711477e87c582d003f36925 P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.4, Last appended by leader: 4, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:19.464088 16225 raft_consensus.cc:3060] T 0a9de347c711477e87c582d003f36925 P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:19:19.469146 16825 raft_consensus.cc:2468] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 3.
I20260709 13:19:19.469949 16297 raft_consensus.cc:2468] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 3.
I20260709 13:19:19.471405 16115 leader_election.cc:304] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:19:19.473513 17573 raft_consensus.cc:2804] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:19:19.474009 17573 raft_consensus.cc:697] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b [term 3 LEADER]: Becoming Leader. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:19:19.474565 16225 raft_consensus.cc:2468] T 0a9de347c711477e87c582d003f36925 P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 3.
I20260709 13:19:19.474869 17573 consensus_queue.cc:237] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.4, Last appended by leader: 4, Current term: 3, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:19.485934 16044 catalog_manager.cc:5697] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b reported cstate change: term changed from 2 to 3, leader changed from 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130) to 647a343a542d4a139fe106c19095129b (127.14.177.129). New cstate: current_term: 3 leader_uuid: "647a343a542d4a139fe106c19095129b" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:19:19.505760 16825 raft_consensus.cc:1275] T 9e65676b21b748728ad39668cfe722ec P cb82cc7bd6db46d980c75237a1402860 [term 2 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:19:19.507326 17523 consensus_queue.cc:1048] T 9e65676b21b748728ad39668cfe722ec P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:19:19.517448 16147 raft_consensus.cc:1275] T 9e65676b21b748728ad39668cfe722ec P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:19:19.519081 17586 consensus_queue.cc:1048] T 9e65676b21b748728ad39668cfe722ec P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:19:19.718645 15046 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:19:19.832327 15046 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:19:19.887526 16297 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "08c74490c40243a88e62d1760b5cd814"
dest_uuid: "90f3da57796b488c9087a2f523adf045"
mode: GRACEFUL
new_leader_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
 from {username='slave'} at 127.0.0.1:43334
I20260709 13:19:19.888198 16297 raft_consensus.cc:606] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045 [term 2 LEADER]: Received request to transfer leadership to TS 1ca2725d290e47d68b1a4b609c9816ac
I20260709 13:19:19.891032 16151 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "746aabf856ab401c9ef5e859371e7b67"
dest_uuid: "647a343a542d4a139fe106c19095129b"
mode: GRACEFUL
new_leader_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
 from {username='slave'} at 127.0.0.1:40290
I20260709 13:19:19.891659 16151 raft_consensus.cc:606] T 746aabf856ab401c9ef5e859371e7b67 P 647a343a542d4a139fe106c19095129b [term 1 LEADER]: Received request to transfer leadership to TS 1ca2725d290e47d68b1a4b609c9816ac
I20260709 13:19:19.894379 16297 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "87ac0d1f7e8a419f9bcbc04aee0e067e"
dest_uuid: "90f3da57796b488c9087a2f523adf045"
mode: GRACEFUL
new_leader_uuid: "647a343a542d4a139fe106c19095129b"
 from {username='slave'} at 127.0.0.1:43334
I20260709 13:19:19.894955 16297 raft_consensus.cc:606] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 90f3da57796b488c9087a2f523adf045 [term 1 LEADER]: Received request to transfer leadership to TS 647a343a542d4a139fe106c19095129b
I20260709 13:19:19.900081 16225 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8c11074579b046d99e585c2818e18a2c"
dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
mode: GRACEFUL
new_leader_uuid: "cb82cc7bd6db46d980c75237a1402860"
 from {username='slave'} at 127.0.0.1:39144
I20260709 13:19:19.900665 16225 raft_consensus.cc:606] T 8c11074579b046d99e585c2818e18a2c P 1ca2725d290e47d68b1a4b609c9816ac [term 1 LEADER]: Received request to transfer leadership to TS cb82cc7bd6db46d980c75237a1402860
I20260709 13:19:19.904637 16225 raft_consensus.cc:1275] T 0a9de347c711477e87c582d003f36925 P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:19:19.905198 15046 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 4
I20260709 13:19:19.905998 17573 consensus_queue.cc:1048] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:19:19.906790 15046 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:19:19.960567 16825 raft_consensus.cc:1275] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860 [term 3 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:19:19.962410 17452 consensus_queue.cc:1048] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:19:19.977918 16297 raft_consensus.cc:1275] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:19:19.980777 17573 consensus_queue.cc:1048] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:19:20.175932 17642 raft_consensus.cc:993] T 746aabf856ab401c9ef5e859371e7b67 P 647a343a542d4a139fe106c19095129b: : Instructing follower 1ca2725d290e47d68b1a4b609c9816ac to start an election
I20260709 13:19:20.176450 17644 raft_consensus.cc:1081] T 746aabf856ab401c9ef5e859371e7b67 P 647a343a542d4a139fe106c19095129b [term 1 LEADER]: Signalling peer 1ca2725d290e47d68b1a4b609c9816ac to start an election
I20260709 13:19:20.178027 16224 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "746aabf856ab401c9ef5e859371e7b67"
dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
 from {username='slave'} at 127.0.0.1:57604
I20260709 13:19:20.178493 16224 raft_consensus.cc:493] T 746aabf856ab401c9ef5e859371e7b67 P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:19:20.178788 16224 raft_consensus.cc:3060] T 746aabf856ab401c9ef5e859371e7b67 P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:19:20.183748 16224 raft_consensus.cc:515] T 746aabf856ab401c9ef5e859371e7b67 P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:20.186264 16825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746aabf856ab401c9ef5e859371e7b67" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:19:20.186810 16825 raft_consensus.cc:3060] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:19:20.193042 16825 raft_consensus.cc:2468] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 2.
I20260709 13:19:20.191671 16147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746aabf856ab401c9ef5e859371e7b67" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:19:20.194154 16224 leader_election.cc:290] T 746aabf856ab401c9ef5e859371e7b67 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 2 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:19:20.194356 16192 leader_election.cc:304] T 746aabf856ab401c9ef5e859371e7b67 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:19:20.194443 16147 raft_consensus.cc:3055] T 746aabf856ab401c9ef5e859371e7b67 P 647a343a542d4a139fe106c19095129b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:19:20.194864 16147 raft_consensus.cc:740] T 746aabf856ab401c9ef5e859371e7b67 P 647a343a542d4a139fe106c19095129b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:19:20.195753 17638 raft_consensus.cc:2804] T 746aabf856ab401c9ef5e859371e7b67 P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:19:20.195659 16147 consensus_queue.cc:260] T 746aabf856ab401c9ef5e859371e7b67 P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:20.196235 17638 raft_consensus.cc:697] T 746aabf856ab401c9ef5e859371e7b67 P 1ca2725d290e47d68b1a4b609c9816ac [term 2 LEADER]: Becoming Leader. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:19:20.196939 16147 raft_consensus.cc:3060] T 746aabf856ab401c9ef5e859371e7b67 P 647a343a542d4a139fe106c19095129b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:19:20.197152 17638 consensus_queue.cc:237] T 746aabf856ab401c9ef5e859371e7b67 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:20.219630 16147 raft_consensus.cc:2468] T 746aabf856ab401c9ef5e859371e7b67 P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 2.
I20260709 13:19:20.221151 16046 catalog_manager.cc:5697] T 746aabf856ab401c9ef5e859371e7b67 P 1ca2725d290e47d68b1a4b609c9816ac reported cstate change: term changed from 1 to 2, leader changed from 647a343a542d4a139fe106c19095129b (127.14.177.129) to 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130). New cstate: current_term: 2 leader_uuid: "1ca2725d290e47d68b1a4b609c9816ac" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:19:20.250412 17562 raft_consensus.cc:993] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045: : Instructing follower 1ca2725d290e47d68b1a4b609c9816ac to start an election
I20260709 13:19:20.250895 17562 raft_consensus.cc:1081] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045 [term 2 LEADER]: Signalling peer 1ca2725d290e47d68b1a4b609c9816ac to start an election
I20260709 13:19:20.254170 16224 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "08c74490c40243a88e62d1760b5cd814"
dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
 from {username='slave'} at 127.0.0.1:57600
I20260709 13:19:20.254726 16224 raft_consensus.cc:493] T 08c74490c40243a88e62d1760b5cd814 P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:19:20.255105 16224 raft_consensus.cc:3060] T 08c74490c40243a88e62d1760b5cd814 P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:19:20.264281 16224 raft_consensus.cc:515] T 08c74490c40243a88e62d1760b5cd814 P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:20.267657 16224 leader_election.cc:290] T 08c74490c40243a88e62d1760b5cd814 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 3 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:20.267344 16147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c74490c40243a88e62d1760b5cd814" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:19:20.268458 16147 raft_consensus.cc:3060] T 08c74490c40243a88e62d1760b5cd814 P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:19:20.269083 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c74490c40243a88e62d1760b5cd814" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:19:20.269676 16300 raft_consensus.cc:3055] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:19:20.269995 16300 raft_consensus.cc:740] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:19:20.270840 16300 consensus_queue.cc:260] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:20.274178 16300 raft_consensus.cc:3060] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:19:20.281426 16300 raft_consensus.cc:2468] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 3.
I20260709 13:19:20.282684 16190 leader_election.cc:304] T 08c74490c40243a88e62d1760b5cd814 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:19:20.283808 17638 raft_consensus.cc:2804] T 08c74490c40243a88e62d1760b5cd814 P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:19:20.284307 17638 raft_consensus.cc:697] T 08c74490c40243a88e62d1760b5cd814 P 1ca2725d290e47d68b1a4b609c9816ac [term 3 LEADER]: Becoming Leader. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:19:20.285130 17638 consensus_queue.cc:237] T 08c74490c40243a88e62d1760b5cd814 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:20.288339 16147 raft_consensus.cc:2468] T 08c74490c40243a88e62d1760b5cd814 P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 3.
I20260709 13:19:20.298866 16046 catalog_manager.cc:5697] T 08c74490c40243a88e62d1760b5cd814 P 1ca2725d290e47d68b1a4b609c9816ac reported cstate change: term changed from 2 to 3, leader changed from 90f3da57796b488c9087a2f523adf045 (127.14.177.131) to 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130). New cstate: current_term: 3 leader_uuid: "1ca2725d290e47d68b1a4b609c9816ac" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } health_report { overall_health: UNKNOWN } } }
I20260709 13:19:20.300527 16147 consensus_queue.cc:237] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } }
W20260709 13:19:20.307463 16117 consensus_peers.cc:597] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b -> Peer cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923): Couldn't send request to peer cb82cc7bd6db46d980c75237a1402860. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 806d7690c2b644959152074bb2cab0a7. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:19:20.308543 16300 raft_consensus.cc:1275] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:19:20.308972 16224 raft_consensus.cc:1275] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:19:20.310295 17644 consensus_queue.cc:1048] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:19:20.311204 17642 consensus_queue.cc:1048] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:19:20.332854 17638 raft_consensus.cc:993] T 8c11074579b046d99e585c2818e18a2c P 1ca2725d290e47d68b1a4b609c9816ac: : Instructing follower cb82cc7bd6db46d980c75237a1402860 to start an election
I20260709 13:19:20.333312 17557 raft_consensus.cc:1081] T 8c11074579b046d99e585c2818e18a2c P 1ca2725d290e47d68b1a4b609c9816ac [term 1 LEADER]: Signalling peer cb82cc7bd6db46d980c75237a1402860 to start an election
I20260709 13:19:20.334914 16825 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "8c11074579b046d99e585c2818e18a2c"
dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
 from {username='slave'} at 127.0.0.1:49906
I20260709 13:19:20.335507 16825 raft_consensus.cc:493] T 8c11074579b046d99e585c2818e18a2c P cb82cc7bd6db46d980c75237a1402860 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:19:20.334300 17642 raft_consensus.cc:2955] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } } }
I20260709 13:19:20.335873 16825 raft_consensus.cc:3060] T 8c11074579b046d99e585c2818e18a2c P cb82cc7bd6db46d980c75237a1402860 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:19:20.338989 16224 raft_consensus.cc:2955] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } } }
I20260709 13:19:20.341868 16300 raft_consensus.cc:2955] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } } }
I20260709 13:19:20.342779 16825 raft_consensus.cc:515] T 8c11074579b046d99e585c2818e18a2c P cb82cc7bd6db46d980c75237a1402860 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:20.346568 16222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c11074579b046d99e585c2818e18a2c" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:19:20.347133 16222 raft_consensus.cc:3055] T 8c11074579b046d99e585c2818e18a2c P 1ca2725d290e47d68b1a4b609c9816ac [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:19:20.347463 16222 raft_consensus.cc:740] T 8c11074579b046d99e585c2818e18a2c P 1ca2725d290e47d68b1a4b609c9816ac [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:19:20.348158 16222 consensus_queue.cc:260] T 8c11074579b046d99e585c2818e18a2c P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:20.349165 16222 raft_consensus.cc:3060] T 8c11074579b046d99e585c2818e18a2c P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:19:20.355607 16222 raft_consensus.cc:2468] T 8c11074579b046d99e585c2818e18a2c P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 2.
I20260709 13:19:20.358376 16046 catalog_manager.cc:5697] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac reported cstate change: config changed from index -1 to 2, NON_VOTER cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) added. New cstate: current_term: 1 leader_uuid: "647a343a542d4a139fe106c19095129b" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } } }
I20260709 13:19:20.362512 16790 leader_election.cc:304] T 8c11074579b046d99e585c2818e18a2c P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:19:20.363512 17645 raft_consensus.cc:2804] T 8c11074579b046d99e585c2818e18a2c P cb82cc7bd6db46d980c75237a1402860 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:19:20.364046 17645 raft_consensus.cc:697] T 8c11074579b046d99e585c2818e18a2c P cb82cc7bd6db46d980c75237a1402860 [term 2 LEADER]: Becoming Leader. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:19:20.364827 17645 consensus_queue.cc:237] T 8c11074579b046d99e585c2818e18a2c P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:20.378127 16825 leader_election.cc:290] T 8c11074579b046d99e585c2818e18a2c P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 2 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:20.384913 16224 consensus_queue.cc:237] T ecc1e363230043fe9fef390f0b979f20 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } }
I20260709 13:19:20.386754 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c11074579b046d99e585c2818e18a2c" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:19:20.387360 16300 raft_consensus.cc:3060] T 8c11074579b046d99e585c2818e18a2c P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:19:20.406284 16300 raft_consensus.cc:2468] T 8c11074579b046d99e585c2818e18a2c P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 2.
I20260709 13:19:20.419449 16146 consensus_queue.cc:237] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:20.435473 16825 raft_consensus.cc:1275] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860 [term 3 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:19:20.439669 16300 raft_consensus.cc:1275] T ecc1e363230043fe9fef390f0b979f20 P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 13:19:20.441257 17654 consensus_queue.cc:1048] T ecc1e363230043fe9fef390f0b979f20 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
W20260709 13:19:20.443542 16192 consensus_peers.cc:597] T ecc1e363230043fe9fef390f0b979f20 P 1ca2725d290e47d68b1a4b609c9816ac -> Peer cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923): Couldn't send request to peer cb82cc7bd6db46d980c75237a1402860. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: ecc1e363230043fe9fef390f0b979f20. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:19:20.447666 16297 raft_consensus.cc:1275] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:19:20.452262 16146 raft_consensus.cc:1275] T ecc1e363230043fe9fef390f0b979f20 P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 13:19:20.455641 17654 consensus_queue.cc:1048] T ecc1e363230043fe9fef390f0b979f20 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.002s
I20260709 13:19:20.458211 17644 consensus_queue.cc:1048] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:19:20.479048 17593 raft_consensus.cc:2955] T ecc1e363230043fe9fef390f0b979f20 P 1ca2725d290e47d68b1a4b609c9816ac [term 2 LEADER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } } }
I20260709 13:19:20.488718 16300 raft_consensus.cc:2955] T ecc1e363230043fe9fef390f0b979f20 P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } } }
I20260709 13:19:20.509744 17573 consensus_queue.cc:1048] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:19:20.505043 16146 raft_consensus.cc:2955] T ecc1e363230043fe9fef390f0b979f20 P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } } }
I20260709 13:19:20.533536 17644 raft_consensus.cc:2955] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b [term 3 LEADER]: Committing config change with OpId 3.6: config changed from index 3 to 6, VOTER 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
W20260709 13:19:20.533742 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 105139us
I20260709 13:19:20.546084 16046 catalog_manager.cc:5697] T 8c11074579b046d99e585c2818e18a2c P cb82cc7bd6db46d980c75237a1402860 reported cstate change: term changed from 1 to 2, leader changed from 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130) to cb82cc7bd6db46d980c75237a1402860 (127.14.177.132). New cstate: current_term: 2 leader_uuid: "cb82cc7bd6db46d980c75237a1402860" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:19:20.551244 16825 raft_consensus.cc:2955] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860 [term 3 FOLLOWER]: Committing config change with OpId 3.6: config changed from index 3 to 6, VOTER 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:19:20.553004 16298 raft_consensus.cc:2955] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Committing config change with OpId 3.6: config changed from index 3 to 6, VOTER 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:19:20.567139 16046 catalog_manager.cc:5697] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860 reported cstate change: config changed from index 3 to 6, VOTER 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130) evicted. New cstate: current_term: 3 leader_uuid: "647a343a542d4a139fe106c19095129b" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:19:20.580279 16035 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 0a9de347c711477e87c582d003f36925 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:19:20.592092 16825 raft_consensus.cc:1275] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860 [term 2 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:19:20.594930 17654 consensus_queue.cc:1048] T 746aabf856ab401c9ef5e859371e7b67 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:19:20.594484 16044 catalog_manager.cc:5697] T ecc1e363230043fe9fef390f0b979f20 P 1ca2725d290e47d68b1a4b609c9816ac reported cstate change: config changed from index -1 to 3, NON_VOTER cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) added. New cstate: current_term: 2 leader_uuid: "1ca2725d290e47d68b1a4b609c9816ac" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
W20260709 13:19:20.626204 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 79210.8us
I20260709 13:19:20.681699 17562 raft_consensus.cc:993] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 90f3da57796b488c9087a2f523adf045: : Instructing follower 647a343a542d4a139fe106c19095129b to start an election
W20260709 13:19:20.687702 16097 auto_rebalancer.cc:629] Failed to schedule move for tablet 87ac0d1f7e8a419f9bcbc04aee0e067e: Remote error: Service unavailable: leader transfer in progress
I20260709 13:19:20.695362 16295 consensus_queue.cc:237] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } }
I20260709 13:19:20.700719 17662 raft_consensus.cc:1081] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 90f3da57796b488c9087a2f523adf045 [term 1 LEADER]: Signalling peer 647a343a542d4a139fe106c19095129b to start an election
I20260709 13:19:20.703783 16146 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "87ac0d1f7e8a419f9bcbc04aee0e067e"
dest_uuid: "647a343a542d4a139fe106c19095129b"
 from {username='slave'} at 127.0.0.1:40830
I20260709 13:19:20.704602 16146 raft_consensus.cc:493] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 647a343a542d4a139fe106c19095129b [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:19:20.705313 16146 raft_consensus.cc:3060] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 647a343a542d4a139fe106c19095129b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:19:20.731019 16151 raft_consensus.cc:1275] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 13:19:20.742063 17657 consensus_queue.cc:1048] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:19:20.746600 16151 raft_consensus.cc:1275] T 08c74490c40243a88e62d1760b5cd814 P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:19:20.755095 17671 consensus_queue.cc:1048] T 08c74490c40243a88e62d1760b5cd814 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:19:20.757735 16147 raft_consensus.cc:1275] T 746aabf856ab401c9ef5e859371e7b67 P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:19:20.769685 16209 tablet_service.cc:1558] Processing DeleteTablet for tablet 0a9de347c711477e87c582d003f36925 with delete_type TABLET_DATA_TOMBSTONED (TS 1ca2725d290e47d68b1a4b609c9816ac not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:57598
W20260709 13:19:20.772025 16266 consensus_peers.cc:597] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045 -> Peer cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923): Couldn't send request to peer cb82cc7bd6db46d980c75237a1402860. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 19baa7bc367844c288bca8f81439c413. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:19:20.773602 16224 raft_consensus.cc:1275] T 19baa7bc367844c288bca8f81439c413 P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 13:19:20.767756 17593 consensus_queue.cc:1048] T 746aabf856ab401c9ef5e859371e7b67 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.022s
I20260709 13:19:20.793653 17657 consensus_queue.cc:1048] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:19:20.827682 16295 raft_consensus.cc:1275] T 8c11074579b046d99e585c2818e18a2c P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:19:20.829078 16146 raft_consensus.cc:515] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:20.838325 16300 raft_consensus.cc:1275] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:19:20.836452 17645 consensus_queue.cc:1048] T 8c11074579b046d99e585c2818e18a2c P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.012s
I20260709 13:19:20.851052 16222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87ac0d1f7e8a419f9bcbc04aee0e067e" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:19:20.851053 16146 leader_election.cc:290] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 2 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:20.851739 16222 raft_consensus.cc:3060] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:19:20.858973 16225 raft_consensus.cc:1275] T 8c11074579b046d99e585c2818e18a2c P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:19:20.861713 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87ac0d1f7e8a419f9bcbc04aee0e067e" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:19:20.862377 16298 raft_consensus.cc:3055] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 90f3da57796b488c9087a2f523adf045 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:19:20.862686 16298 raft_consensus.cc:740] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 90f3da57796b488c9087a2f523adf045 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:19:20.854954 17557 consensus_queue.cc:1048] T 08c74490c40243a88e62d1760b5cd814 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.004s
I20260709 13:19:20.860900 17645 consensus_queue.cc:1048] T 8c11074579b046d99e585c2818e18a2c P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.006s
I20260709 13:19:20.868949 16298 consensus_queue.cc:260] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 90f3da57796b488c9087a2f523adf045 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:20.870098 16222 raft_consensus.cc:2468] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 2.
I20260709 13:19:20.871325 16116 leader_election.cc:304] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
I20260709 13:19:20.873255 17573 raft_consensus.cc:2804] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:19:20.873471 17677 tablet_replica.cc:333] T 0a9de347c711477e87c582d003f36925 P 1ca2725d290e47d68b1a4b609c9816ac: stopping tablet replica
I20260709 13:19:20.876122 17677 raft_consensus.cc:2243] T 0a9de347c711477e87c582d003f36925 P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:19:20.874229 17518 raft_consensus.cc:2955] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045 [term 2 LEADER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } } }
I20260709 13:19:20.876941 17677 raft_consensus.cc:2272] T 0a9de347c711477e87c582d003f36925 P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:19:20.878741 16298 raft_consensus.cc:3060] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:19:20.876564 16145 raft_consensus.cc:2955] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } } }
I20260709 13:19:20.873867 17573 raft_consensus.cc:697] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 647a343a542d4a139fe106c19095129b [term 2 LEADER]: Becoming Leader. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:19:20.898685 17681 ts_tablet_manager.cc:933] T 806d7690c2b644959152074bb2cab0a7 P cb82cc7bd6db46d980c75237a1402860: Initiating tablet copy from peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741)
I20260709 13:19:20.880287 17573 consensus_queue.cc:237] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:20.981444 16298 raft_consensus.cc:2468] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 2.
I20260709 13:19:21.023247 15046 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:19:21.093297 17683 log.cc:927] Time spent T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.082s	user 0.000s	sys 0.000s
W20260709 13:19:21.132860 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 90089.7us
W20260709 13:19:21.149065 17678 log.cc:927] Time spent T 19baa7bc367844c288bca8f81439c413 P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 0.256s	user 0.008s	sys 0.000s
I20260709 13:19:21.153960 17677 ts_tablet_manager.cc:1916] T 0a9de347c711477e87c582d003f36925 P 1ca2725d290e47d68b1a4b609c9816ac: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:19:21.221081 16224 raft_consensus.cc:2955] T 19baa7bc367844c288bca8f81439c413 P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } } }
W20260709 13:19:21.228179 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 78933.9us
I20260709 13:19:21.275228 17681 tablet_copy_client.cc:323] T 806d7690c2b644959152074bb2cab0a7 P cb82cc7bd6db46d980c75237a1402860: tablet copy: Beginning tablet copy session from remote peer at address 127.14.177.129:36741
W20260709 13:19:21.337352 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 44409.9us
I20260709 13:19:21.383774 17677 ts_tablet_manager.cc:1929] T 0a9de347c711477e87c582d003f36925 P 1ca2725d290e47d68b1a4b609c9816ac: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 3.5
I20260709 13:19:21.384320 17677 log.cc:1199] T 0a9de347c711477e87c582d003f36925 P 1ca2725d290e47d68b1a4b609c9816ac: Deleting WAL directory at /tmp/dist-test-taskdH_Ma7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783603093801613-15046-0/minicluster-data/ts-1-root/wals/0a9de347c711477e87c582d003f36925
W20260709 13:19:21.474849 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 126074us
W20260709 13:19:21.545482 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 66442.6us
I20260709 13:19:21.549185 17703 ts_tablet_manager.cc:933] T 19baa7bc367844c288bca8f81439c413 P cb82cc7bd6db46d980c75237a1402860: Initiating tablet copy from peer 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:21.550539 17704 ts_tablet_manager.cc:933] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860: Initiating tablet copy from peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:19:21.559984 17704 tablet_copy_client.cc:323] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860: tablet copy: Beginning tablet copy session from remote peer at address 127.14.177.130:35943
I20260709 13:19:21.564647 17703 tablet_copy_client.cc:323] T 19baa7bc367844c288bca8f81439c413 P cb82cc7bd6db46d980c75237a1402860: tablet copy: Beginning tablet copy session from remote peer at address 127.14.177.131:41389
I20260709 13:19:21.586719 16161 tablet_copy_service.cc:140] P 647a343a542d4a139fe106c19095129b: Received BeginTabletCopySession request for tablet 806d7690c2b644959152074bb2cab0a7 from peer cb82cc7bd6db46d980c75237a1402860 ({username='slave'} at 127.0.0.1:46224)
I20260709 13:19:21.587333 16161 tablet_copy_service.cc:161] P 647a343a542d4a139fe106c19095129b: Beginning new tablet copy session on tablet 806d7690c2b644959152074bb2cab0a7 from peer cb82cc7bd6db46d980c75237a1402860 at {username='slave'} at 127.0.0.1:46224: session id = cb82cc7bd6db46d980c75237a1402860-806d7690c2b644959152074bb2cab0a7
I20260709 13:19:21.596513 16035 catalog_manager.cc:5028] TS 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943): tablet 0a9de347c711477e87c582d003f36925 (table test-workload [id=ed3d15d8c9ad46bebc4fe3a402d2e54f]) successfully deleted
I20260709 13:19:21.597141 16161 tablet_copy_source_session.cc:215] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:19:21.628206 16310 tablet_copy_service.cc:140] P 90f3da57796b488c9087a2f523adf045: Received BeginTabletCopySession request for tablet 19baa7bc367844c288bca8f81439c413 from peer cb82cc7bd6db46d980c75237a1402860 ({username='slave'} at 127.0.0.1:40474)
I20260709 13:19:21.629030 16310 tablet_copy_service.cc:161] P 90f3da57796b488c9087a2f523adf045: Beginning new tablet copy session on tablet 19baa7bc367844c288bca8f81439c413 from peer cb82cc7bd6db46d980c75237a1402860 at {username='slave'} at 127.0.0.1:40474: session id = cb82cc7bd6db46d980c75237a1402860-19baa7bc367844c288bca8f81439c413
I20260709 13:19:21.636391 17681 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 806d7690c2b644959152074bb2cab0a7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:19:22.706075 17672 raft_consensus.cc:493] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
W20260709 13:19:22.807931 15051 debug-util.cc:398] Leaking SignalData structure 0x7b080000fac0 after lost signal to thread 16116
W20260709 13:19:22.808596 15051 kernel_stack_watchdog.cc:198] Thread 16116 stuck at /apache/dev/git/kudu/src/kudu/rpc/outbound_call.cc:307 for 233ms:
Kernel stack:
(could not read kernel stack)
User stack:
<Timed out: thread did not respond: maybe it is blocking signals>
I20260709 13:19:22.706864 17672 raft_consensus.cc:515] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:19:22.831970 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 1.19846e+06us
I20260709 13:19:22.834292 17672 leader_election.cc:290] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
W20260709 13:19:22.839296 16080 debug-util.cc:398] Leaking SignalData structure 0x7b0800091be0 after lost signal to thread 15049
I20260709 13:19:22.843194 17672 raft_consensus.cc:493] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:19:22.843853 17672 raft_consensus.cc:515] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:19:22.845795 16117 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.265s	user 0.033s	sys 0.064s
W20260709 13:19:22.846163 16117 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.265s	user 0.033s	sys 0.064s
I20260709 13:19:22.846096 17672 leader_election.cc:290] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
W20260709 13:19:22.846606 16117 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 1.26592e+06us
I20260709 13:19:22.846757 17672 raft_consensus.cc:493] T d2ab0ac8347641e6955e9bce7d90c1a2 P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:19:22.847344 17672 raft_consensus.cc:515] T d2ab0ac8347641e6955e9bce7d90c1a2 P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:22.849502 17672 leader_election.cc:290] T d2ab0ac8347641e6955e9bce7d90c1a2 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:19:22.896593 17645 raft_consensus.cc:493] T 57a744f5aaf944678a63469cbf514015 P cb82cc7bd6db46d980c75237a1402860 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:19:22.897290 17645 raft_consensus.cc:515] T 57a744f5aaf944678a63469cbf514015 P cb82cc7bd6db46d980c75237a1402860 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:22.899601 17645 leader_election.cc:290] T 57a744f5aaf944678a63469cbf514015 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:22.900091 17645 raft_consensus.cc:493] T 26354361e9b044cea1c2b86c4882fe92 P cb82cc7bd6db46d980c75237a1402860 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:19:22.900557 17645 raft_consensus.cc:515] T 26354361e9b044cea1c2b86c4882fe92 P cb82cc7bd6db46d980c75237a1402860 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:22.901656 17672 raft_consensus.cc:493] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:22.902267 17672 raft_consensus.cc:515] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:22.902616 17645 leader_election.cc:290] T 26354361e9b044cea1c2b86c4882fe92 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:22.904016 17645 raft_consensus.cc:493] T 2794af5b99984efebdee564035841cb0 P cb82cc7bd6db46d980c75237a1402860 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:19:22.904840 17672 leader_election.cc:290] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:22.905208 17645 raft_consensus.cc:515] T 2794af5b99984efebdee564035841cb0 P cb82cc7bd6db46d980c75237a1402860 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:22.909020 17645 leader_election.cc:290] T 2794af5b99984efebdee564035841cb0 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:22.909518 17645 raft_consensus.cc:493] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:19:22.910008 17645 raft_consensus.cc:515] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:22.912164 17645 leader_election.cc:290] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:22.914321 17645 raft_consensus.cc:493] T 23f82224f4564fea93572e8206bad1fe P cb82cc7bd6db46d980c75237a1402860 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:19:22.914805 17645 raft_consensus.cc:515] T 23f82224f4564fea93572e8206bad1fe P cb82cc7bd6db46d980c75237a1402860 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:22.918381 17645 leader_election.cc:290] T 23f82224f4564fea93572e8206bad1fe P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:22.919121 17645 raft_consensus.cc:493] T 829b4ad8c5a54eababeea1181b960d4b P cb82cc7bd6db46d980c75237a1402860 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:19:22.919878 17645 raft_consensus.cc:515] T 829b4ad8c5a54eababeea1181b960d4b P cb82cc7bd6db46d980c75237a1402860 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:22.922312 17645 leader_election.cc:290] T 829b4ad8c5a54eababeea1181b960d4b P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:19:22.949469 17696 raft_consensus.cc:493] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:19:22.950075 17696 raft_consensus.cc:515] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:22.954255 17709 raft_consensus.cc:493] T ee9ed67006144797b1de6ea8c2013a68 P cb82cc7bd6db46d980c75237a1402860 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:19:22.954094 17645 raft_consensus.cc:493] T 9e65676b21b748728ad39668cfe722ec P cb82cc7bd6db46d980c75237a1402860 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:19:22.956844 17709 raft_consensus.cc:515] T ee9ed67006144797b1de6ea8c2013a68 P cb82cc7bd6db46d980c75237a1402860 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:22.955786 17708 raft_consensus.cc:493] T b87147731a68400e8cce9b879daa7c5d P cb82cc7bd6db46d980c75237a1402860 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:19:22.957932 17696 leader_election.cc:290] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:19:22.958421 17708 raft_consensus.cc:515] T b87147731a68400e8cce9b879daa7c5d P cb82cc7bd6db46d980c75237a1402860 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:22.959771 17709 leader_election.cc:290] T ee9ed67006144797b1de6ea8c2013a68 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:19:22.960391 17645 raft_consensus.cc:515] T 9e65676b21b748728ad39668cfe722ec P cb82cc7bd6db46d980c75237a1402860 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:22.961711 17708 leader_election.cc:290] T b87147731a68400e8cce9b879daa7c5d P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:22.962769 17645 leader_election.cc:290] T 9e65676b21b748728ad39668cfe722ec P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:22.964313 17645 raft_consensus.cc:493] T 66f75124215f4ca6bef831e174a792d8 P cb82cc7bd6db46d980c75237a1402860 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:19:22.964833 17645 raft_consensus.cc:515] T 66f75124215f4ca6bef831e174a792d8 P cb82cc7bd6db46d980c75237a1402860 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:22.965807 17708 raft_consensus.cc:493] T af57996a8be84d398f0f023e8d08ef77 P cb82cc7bd6db46d980c75237a1402860 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:19:22.966367 17708 raft_consensus.cc:515] T af57996a8be84d398f0f023e8d08ef77 P cb82cc7bd6db46d980c75237a1402860 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:22.966995 17645 leader_election.cc:290] T 66f75124215f4ca6bef831e174a792d8 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:19:22.968736 17708 leader_election.cc:290] T af57996a8be84d398f0f023e8d08ef77 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:19:22.969420 17696 raft_consensus.cc:493] T e4dc6b0541ca431ebc757beac00521f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:19:22.969900 17696 raft_consensus.cc:515] T e4dc6b0541ca431ebc757beac00521f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:22.971660 17696 leader_election.cc:290] T e4dc6b0541ca431ebc757beac00521f1 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:22.987226 17696 raft_consensus.cc:493] T 19baa7bc367844c288bca8f81439c413 P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:19:22.987813 17696 raft_consensus.cc:515] T 19baa7bc367844c288bca8f81439c413 P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } }
I20260709 13:19:22.990077 17696 leader_election.cc:290] T 19baa7bc367844c288bca8f81439c413 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:22.992180 17654 raft_consensus.cc:493] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:19:22.992722 17654 raft_consensus.cc:515] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:22.993781 17692 raft_consensus.cc:493] T 8c11074579b046d99e585c2818e18a2c P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:19:22.994525 17593 raft_consensus.cc:493] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:19:22.994335 17692 raft_consensus.cc:515] T 8c11074579b046d99e585c2818e18a2c P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:22.995008 17654 leader_election.cc:290] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:19:22.995345 17690 raft_consensus.cc:493] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:19:22.995906 17690 raft_consensus.cc:515] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } }
I20260709 13:19:22.996642 17692 leader_election.cc:290] T 8c11074579b046d99e585c2818e18a2c P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:19:22.998230 17690 leader_election.cc:290] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:22.998660 16820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada5a004ba7c46819051e3ccec71ffdb" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:19:22.999032 16824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ab0ac8347641e6955e9bce7d90c1a2" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:19:22.998845 16818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74ca8140f35c4ae29e8a664f6bdfc829" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:19:22.995067 17593 raft_consensus.cc:515] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:23.002166 17593 leader_election.cc:290] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:19:23.005574 16818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdbd78acb4cf46298e0de8ba0d130e19" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:19:23.006085 16824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c11074579b046d99e585c2818e18a2c" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:19:23.007309 16820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5d59407eb545b895d8d922e0dd69fd" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:19:23.007910 16820 raft_consensus.cc:2468] T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 1.
I20260709 13:19:23.008701 16236 tablet_copy_service.cc:140] P 1ca2725d290e47d68b1a4b609c9816ac: Received BeginTabletCopySession request for tablet ecc1e363230043fe9fef390f0b979f20 from peer cb82cc7bd6db46d980c75237a1402860 ({username='slave'} at 127.0.0.1:33050)
I20260709 13:19:23.009316 16236 tablet_copy_service.cc:161] P 1ca2725d290e47d68b1a4b609c9816ac: Beginning new tablet copy session on tablet ecc1e363230043fe9fef390f0b979f20 from peer cb82cc7bd6db46d980c75237a1402860 at {username='slave'} at 127.0.0.1:33050: session id = cb82cc7bd6db46d980c75237a1402860-ecc1e363230043fe9fef390f0b979f20
I20260709 13:19:23.011323 16192 leader_election.cc:304] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:19:23.012501 17696 raft_consensus.cc:2804] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:19:23.013126 17696 raft_consensus.cc:493] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Starting leader election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:19:23.013732 17696 raft_consensus.cc:3060] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:19:23.018764 16825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b7d1ab519044eda2d5e04a07242f25" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:19:23.020345 17695 raft_consensus.cc:493] T 74ca8140f35c4ae29e8a664f6bdfc829 P cb82cc7bd6db46d980c75237a1402860 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:19:23.020900 17695 raft_consensus.cc:515] T 74ca8140f35c4ae29e8a664f6bdfc829 P cb82cc7bd6db46d980c75237a1402860 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:23.023655 17695 leader_election.cc:290] T 74ca8140f35c4ae29e8a664f6bdfc829 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:23.028045 17674 raft_consensus.cc:493] T b6f07842d3aa47858df5665874c86f3a P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:19:23.028589 17674 raft_consensus.cc:515] T b6f07842d3aa47858df5665874c86f3a P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:23.030416 16046 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:49246 (request call id 321) took 2141 ms. Trace:
I20260709 13:19:23.030799 16046 rpcz_store.cc:276] 0709 13:19:20.888900 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:20.893287 (+  4387us) service_pool.cc:224] Handling call
0709 13:19:21.549838 (+656551us) spinlock_profiling.cc:243] Waited 13.2 ms on lock 0x7b5400002490. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69ad8fadb1 00007f69ada3b640 00007f69b6912b76 00007f69b6a1cc3e 00007f69b4edd2a4 00007f69b4edd232 00007f69b4edd161 00007f69b4edd0dc 00007f69b4edc392 00007f69aee9a48c 00007f69aee998c6
0709 13:19:23.030295 (+1480457us) inbound_call.cc:177] Queueing success response
Metrics: {"reported_tablets":1,"spinlock_wait_cycles":30258048}
I20260709 13:19:23.023676 17696 raft_consensus.cc:515] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:23.036926 17696 leader_election.cc:290] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 2 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:19:23.030931 17674 leader_election.cc:290] T b6f07842d3aa47858df5665874c86f3a P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:23.038993 17695 raft_consensus.cc:493] T 3884f864a96d428785895e5463e5728f P cb82cc7bd6db46d980c75237a1402860 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:19:23.047545 16819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5d59407eb545b895d8d922e0dd69fd" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:19:23.050505 16819 raft_consensus.cc:3060] T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:19:23.051287 16236 tablet_copy_source_session.cc:215] T ecc1e363230043fe9fef390f0b979f20 P 1ca2725d290e47d68b1a4b609c9816ac: Tablet Copy: opened 0 blocks and 1 log segments
W20260709 13:19:23.051438 16080 debug-util.cc:398] Leaking SignalData structure 0x7b080008b2c0 after lost signal to thread 16083
I20260709 13:19:23.053954 17557 raft_consensus.cc:493] T c2f5f89186204242bf377f7c42b2e2da P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:19:23.047778 17695 raft_consensus.cc:515] T 3884f864a96d428785895e5463e5728f P cb82cc7bd6db46d980c75237a1402860 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:23.054666 17557 raft_consensus.cc:515] T c2f5f89186204242bf377f7c42b2e2da P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:23.057835 16151 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40844 (request call id 1586) took 1452 ms. Trace:
I20260709 13:19:23.058255 16151 rpcz_store.cc:276] 0709 13:19:21.605684 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:21.606039 (+   355us) service_pool.cc:224] Handling call
0709 13:19:21.606330 (+   291us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:21.606501 (+   171us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:21.606524 (+    23us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:21.606588 (+    64us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:19:21.606637 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:23.057589 (+1450952us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:23.057751 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:23.058959 17557 leader_election.cc:290] T c2f5f89186204242bf377f7c42b2e2da P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:23.062374 16047 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:49252 (request call id 284) took 1903 ms. Trace:
W20260709 13:19:23.070346 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 1.49583e+06us
I20260709 13:19:23.069809 16047 rpcz_store.cc:276] 0709 13:19:21.158683 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:21.158860 (+   177us) service_pool.cc:224] Handling call
0709 13:19:23.062283 (+1903423us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":201891,"reported_tablets":1}
W20260709 13:19:23.072225 16266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 1.43827e+06us
I20260709 13:19:23.066526 16819 raft_consensus.cc:2468] T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 2.
I20260709 13:19:23.068472 16044 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:49250 (request call id 314) took 2075 ms. Trace:
I20260709 13:19:23.074756 16044 rpcz_store.cc:276] 0709 13:19:20.993282 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:20.994049 (+   767us) service_pool.cc:224] Handling call
0709 13:19:21.165044 (+170995us) spinlock_profiling.cc:243] Waited 13.9 ms on lock 0x7b5400002490. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69ad8fadb1 00007f69ada3ab71 00007f69ada3b0b2 00007f69b69b4a89 00007f69b694cda6 00007f69b691e088 00007f69b6a1c37e 00007f69b4edd2a4 00007f69b4edd232 00007f69b4edd161 00007f69b4edd0dc
0709 13:19:23.068409 (+1903365us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1794905,"reported_tablets":1,"spinlock_wait_cycles":31868672}
I20260709 13:19:23.072553 17674 raft_consensus.cc:493] T 02838c86531d4f98bf58d1c22b8baf77 P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:19:23.072063 17557 raft_consensus.cc:493] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:19:23.090142 17452 raft_consensus.cc:493] T b87147731a68400e8cce9b879daa7c5d P 647a343a542d4a139fe106c19095129b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:19:23.084221 17695 leader_election.cc:290] T 3884f864a96d428785895e5463e5728f P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:23.086424 17708 raft_consensus.cc:493] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P cb82cc7bd6db46d980c75237a1402860 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:19:23.087309 17710 raft_consensus.cc:493] T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:19:23.086930 17709 raft_consensus.cc:493] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:19:23.088753 17644 raft_consensus.cc:493] T 434bff3082d141838068f323bdb0d1ed P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
W20260709 13:19:23.089563 16117 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 242274us
I20260709 13:19:23.090588 17452 raft_consensus.cc:515] T b87147731a68400e8cce9b879daa7c5d P 647a343a542d4a139fe106c19095129b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:23.092165 16192 leader_election.cc:304] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 
W20260709 13:19:23.123201 16115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 43591.2us
W20260709 13:19:23.141669 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 63602.8us
I20260709 13:19:23.089864 17672 raft_consensus.cc:493] T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:19:23.103382 17645 raft_consensus.cc:493] T c9944d5e4b8c4109af9955c4f08c9095 P cb82cc7bd6db46d980c75237a1402860 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:19:23.090514 17694 raft_consensus.cc:493] T 746aabf856ab401c9ef5e859371e7b67 P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:19:23.117774 17642 raft_consensus.cc:493] T 08c74490c40243a88e62d1760b5cd814 P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
W20260709 13:19:23.183735 16117 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 52698us
I20260709 13:19:23.143709 17681 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.StartTabletCopy from 127.0.0.1:49896 (request call id 622) took 2308 ms. Trace:
I20260709 13:19:23.101902 17557 raft_consensus.cc:515] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:23.176446 17452 leader_election.cc:290] T b87147731a68400e8cce9b879daa7c5d P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
W20260709 13:19:23.178018 16192 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 85949.8us
I20260709 13:19:23.180114 17701 raft_consensus.cc:493] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:19:23.180624 17673 raft_consensus.cc:493] T d16dfad0c3b540e5b4b86492ddf87208 P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:19:23.181099 17670 raft_consensus.cc:493] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:19:23.182616 17685 raft_consensus.cc:493] T 68a6838d82ef4cac83507f906139d90f P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:19:23.183104 17702 raft_consensus.cc:493] T 829b4ad8c5a54eababeea1181b960d4b P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:19:23.123718 17695 raft_consensus.cc:493] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:19:23.184434 16224 raft_consensus.cc:1240] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 1. Current term is 2. Ops: []
I20260709 13:19:23.092633 17674 raft_consensus.cc:515] T 02838c86531d4f98bf58d1c22b8baf77 P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:23.143631 17690 raft_consensus.cc:493] T ee9ed67006144797b1de6ea8c2013a68 P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:19:23.187649 16151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57a744f5aaf944678a63469cbf514015" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:19:23.172842 17654 raft_consensus.cc:493] T 1df93779209748b9860f6d893dd3de53 P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:19:23.170990 17686 raft_consensus.cc:493] T 434bff3082d141838068f323bdb0d1ed P cb82cc7bd6db46d980c75237a1402860 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:19:23.124974 17708 raft_consensus.cc:515] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P cb82cc7bd6db46d980c75237a1402860 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:23.172957 17692 raft_consensus.cc:493] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:19:23.195289 16147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a9de347c711477e87c582d003f36925" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:19:23.134245 17709 raft_consensus.cc:515] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:23.130450 17710 raft_consensus.cc:515] T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:23.196878 16151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2794af5b99984efebdee564035841cb0" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:19:23.142719 17644 raft_consensus.cc:515] T 434bff3082d141838068f323bdb0d1ed P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:23.205560 16146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26354361e9b044cea1c2b86c4882fe92" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:19:23.216907 17693 raft_consensus.cc:493] T 8d4ab870a05347868956aa8b841b2981 P 647a343a542d4a139fe106c19095129b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:19:23.184722 17672 raft_consensus.cc:515] T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:23.186069 17645 raft_consensus.cc:515] T c9944d5e4b8c4109af9955c4f08c9095 P cb82cc7bd6db46d980c75237a1402860 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:23.239126 16824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87147731a68400e8cce9b879daa7c5d" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:19:23.239740 16824 raft_consensus.cc:2468] T b87147731a68400e8cce9b879daa7c5d P cb82cc7bd6db46d980c75237a1402860 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 1.
I20260709 13:19:23.242563 16147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "829b4ad8c5a54eababeea1181b960d4b" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:19:23.187826 17642 raft_consensus.cc:515] T 08c74490c40243a88e62d1760b5cd814 P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:23.244688 17709 leader_election.cc:290] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:23.245955 17642 leader_election.cc:290] T 08c74490c40243a88e62d1760b5cd814 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:21.714296 16310 tablet_copy_source_session.cc:215] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:19:23.247789 16310 rpcz_store.cc:275] Call kudu.tserver.TabletCopyService.BeginTabletCopySession from 127.0.0.1:40474 (request call id 140) took 1624 ms. Trace:
I20260709 13:19:23.234912 17710 leader_election.cc:290] T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:23.235037 17644 leader_election.cc:290] T 434bff3082d141838068f323bdb0d1ed P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:19:23.202215 17654 raft_consensus.cc:515] T 1df93779209748b9860f6d893dd3de53 P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:23.238365 16825 raft_consensus.cc:1240] T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860 [term 2 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 1. Current term is 2. Ops: []
I20260709 13:19:23.195012 17695 raft_consensus.cc:515] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:23.201929 17690 raft_consensus.cc:515] T ee9ed67006144797b1de6ea8c2013a68 P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:23.258028 16151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee9ed67006144797b1de6ea8c2013a68" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:19:23.254941 17714 raft_consensus.cc:493] T 8158bbd17b994d2eb96f560bd0f1b88d P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:19:23.255067 16145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87147731a68400e8cce9b879daa7c5d" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:19:23.259222 16145 raft_consensus.cc:2468] T b87147731a68400e8cce9b879daa7c5d P 647a343a542d4a139fe106c19095129b [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 1.
I20260709 13:19:23.189848 17694 raft_consensus.cc:515] T 746aabf856ab401c9ef5e859371e7b67 P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:23.255476 16144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e65676b21b748728ad39668cfe722ec" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:19:23.256454 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66f75124215f4ca6bef831e174a792d8" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:19:23.262156 16144 raft_consensus.cc:2468] T 9e65676b21b748728ad39668cfe722ec P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 2.
I20260709 13:19:23.194495 17685 raft_consensus.cc:515] T 68a6838d82ef4cac83507f906139d90f P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:23.207805 17692 raft_consensus.cc:515] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:23.203051 17701 raft_consensus.cc:515] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:23.270726 17685 leader_election.cc:290] T 68a6838d82ef4cac83507f906139d90f P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:23.271981 17701 leader_election.cc:290] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:19:23.202912 17670 raft_consensus.cc:515] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:23.274904 17670 leader_election.cc:290] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:19:23.277724 16823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7ca61592e848ea9b0fb81a2ce33cdc" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:19:23.277946 16822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8638ee363273426db1393dd81e5c7b72" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:19:23.217478 17693 raft_consensus.cc:515] T 8d4ab870a05347868956aa8b841b2981 P 647a343a542d4a139fe106c19095129b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:23.218449 17686 raft_consensus.cc:515] T 434bff3082d141838068f323bdb0d1ed P cb82cc7bd6db46d980c75237a1402860 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:23.285506 17686 leader_election.cc:290] T 434bff3082d141838068f323bdb0d1ed P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:23.214882 17702 raft_consensus.cc:515] T 829b4ad8c5a54eababeea1181b960d4b P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:23.288130 17702 leader_election.cc:290] T 829b4ad8c5a54eababeea1181b960d4b P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:19:23.291369 16147 raft_consensus.cc:2468] T 829b4ad8c5a54eababeea1181b960d4b P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 2.
I20260709 13:19:23.279338 16148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3884f864a96d428785895e5463e5728f" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:19:23.302027 17694 leader_election.cc:290] T 746aabf856ab401c9ef5e859371e7b67 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
W20260709 13:19:23.279839 16080 debug-util.cc:398] Leaking SignalData structure 0x7b0800332220 after lost signal to thread 16175
I20260709 13:19:23.266817 17638 raft_consensus.cc:2804] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Leader election won for term 2
W20260709 13:19:23.322021 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 121529us
I20260709 13:19:23.248278 16310 rpcz_store.cc:276] 0709 13:19:21.622969 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:21.628020 (+  5051us) service_pool.cc:224] Handling call
0709 13:19:23.247678 (+1619658us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:23.204993 17673 raft_consensus.cc:515] T d16dfad0c3b540e5b4b86492ddf87208 P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:19:23.308223 16080 debug-util.cc:398] Leaking SignalData structure 0x7b0800332260 after lost signal to thread 16176
I20260709 13:19:23.297891 17701 raft_consensus.cc:493] T 66f75124215f4ca6bef831e174a792d8 P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:19:23.331243 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434bff3082d141838068f323bdb0d1ed" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:19:23.304059 17670 raft_consensus.cc:493] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:19:23.298708 17685 raft_consensus.cc:493] T 18ad7e3af5504571a69d10c00750a01c P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:19:23.305478 17702 raft_consensus.cc:493] T 5a959dbecc2648eb85d1cc550f5a3245 P 647a343a542d4a139fe106c19095129b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
W20260709 13:19:23.332086 16115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 199518us
I20260709 13:19:23.335350 16150 raft_consensus.cc:2468] T 434bff3082d141838068f323bdb0d1ed P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 2.
I20260709 13:19:23.185242 17681 rpcz_store.cc:276] 0709 13:19:20.834021 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:20.834193 (+   172us) service_pool.cc:224] Handling call
0709 13:19:20.990974 (+156781us) ts_tablet_manager.cc:934] T 806d7690c2b644959152074bb2cab0a7 P cb82cc7bd6db46d980c75237a1402860: Initiating tablet copy from peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741)
0709 13:19:21.634778 (+643804us) tablet_copy_client.cc:368] Tablet copy session begun
0709 13:19:23.086455 (+1451677us) tablet_metadata.cc:665] Metadata flushed
0709 13:19:23.086608 (+   153us) tablet_copy_client.cc:497] Wrote new tablet metadata
0709 13:19:23.141291 (+ 54683us) tablet_copy_client.cc:418] Wrote new consensus metadata
0709 13:19:23.141679 (+   388us) tablet_replica.cc:206] Creating consensus instance
0709 13:19:23.142071 (+   392us) ts_tablet_manager.cc:989] Replica registered.
0709 13:19:23.142166 (+    95us) inbound_call.cc:177] Queueing success response
Metrics: {"dns_us":49981,"tablet-copy.queue_time_us":41993,"thread_start_us":146832,"threads_started":2}
I20260709 13:19:23.339340 16148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e9b2bf7011f4ae688a79d7bb5a447b4" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:19:23.346601 17693 leader_election.cc:290] T 8d4ab870a05347868956aa8b841b2981 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:23.348465 17673 leader_election.cc:290] T d16dfad0c3b540e5b4b86492ddf87208 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:23.349035 17715 raft_consensus.cc:493] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:19:23.349797 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af57996a8be84d398f0f023e8d08ef77" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:19:23.349577 17715 raft_consensus.cc:515] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:23.352094 16149 raft_consensus.cc:2468] T af57996a8be84d398f0f023e8d08ef77 P 647a343a542d4a139fe106c19095129b [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 1.
I20260709 13:19:23.339025 17701 raft_consensus.cc:515] T 66f75124215f4ca6bef831e174a792d8 P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:23.351552 17685 raft_consensus.cc:515] T 18ad7e3af5504571a69d10c00750a01c P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } }
I20260709 13:19:23.360211 17718 raft_consensus.cc:493] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:19:23.353787 17702 raft_consensus.cc:515] T 5a959dbecc2648eb85d1cc550f5a3245 P 647a343a542d4a139fe106c19095129b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:23.341491 17557 leader_election.cc:290] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:19:23.364233 17722 raft_consensus.cc:493] T 68a6838d82ef4cac83507f906139d90f P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:19:23.364645 17654 leader_election.cc:290] T 1df93779209748b9860f6d893dd3de53 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:23.353619 17681 tablet_copy_client.cc:806] T 806d7690c2b644959152074bb2cab0a7 P cb82cc7bd6db46d980c75237a1402860: tablet copy: Starting download of 0 data blocks...
I20260709 13:19:23.349050 17673 raft_consensus.cc:493] T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:19:23.340978 17670 raft_consensus.cc:515] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:23.307814 17714 raft_consensus.cc:515] T 8158bbd17b994d2eb96f560bd0f1b88d P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:23.369233 17708 leader_election.cc:290] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:23.373867 17672 leader_election.cc:290] T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:23.377619 17690 leader_election.cc:290] T ee9ed67006144797b1de6ea8c2013a68 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:19:23.370568 17724 raft_consensus.cc:493] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:19:23.380496 17670 leader_election.cc:290] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:19:23.378003 17681 tablet_copy_client.cc:670] T 806d7690c2b644959152074bb2cab0a7 P cb82cc7bd6db46d980c75237a1402860: tablet copy: Starting download of 1 WAL segments...
I20260709 13:19:23.362087 17674 leader_election.cc:290] T 02838c86531d4f98bf58d1c22b8baf77 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:23.371313 17645 leader_election.cc:290] T c9944d5e4b8c4109af9955c4f08c9095 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:23.383550 16822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62fe3346cd3c42c4a55e14af9073bca4" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
W20260709 13:19:23.397572 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 65620us
W20260709 13:19:23.377239 16080 debug-util.cc:398] Leaking SignalData structure 0x7b0800320c00 after lost signal to thread 16250
I20260709 13:19:23.360884 17718 raft_consensus.cc:515] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:23.370395 17719 raft_consensus.cc:493] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:19:23.374790 16819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9944d5e4b8c4109af9955c4f08c9095" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:19:23.416898 16819 raft_consensus.cc:2468] T c9944d5e4b8c4109af9955c4f08c9095 P cb82cc7bd6db46d980c75237a1402860 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 2.
I20260709 13:19:23.416692 16821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee9ed67006144797b1de6ea8c2013a68" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:19:23.375725 17695 leader_election.cc:290] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:23.380759 17724 raft_consensus.cc:515] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:23.377041 17692 leader_election.cc:290] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:23.391607 17642 raft_consensus.cc:493] T e5ad4eb281564936a59f41e387a47156 P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:19:23.405941 17719 raft_consensus.cc:515] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:23.373433 17723 raft_consensus.cc:493] T 5a96373bb0e14dce8a266b2fc6c58507 P 647a343a542d4a139fe106c19095129b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
W20260709 13:19:23.404109 16080 debug-util.cc:398] Leaking SignalData structure 0x7b080033f4c0 after lost signal to thread 16251
I20260709 13:19:23.421952 17642 raft_consensus.cc:515] T e5ad4eb281564936a59f41e387a47156 P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:23.392309 17673 raft_consensus.cc:515] T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:23.419826 17644 raft_consensus.cc:493] T 7c2095a5f4ad49408dbc7f69c606120d P 647a343a542d4a139fe106c19095129b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:19:23.364777 17722 raft_consensus.cc:515] T 68a6838d82ef4cac83507f906139d90f P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:23.448900 17642 leader_election.cc:290] T e5ad4eb281564936a59f41e387a47156 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
W20260709 13:19:23.447608 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 115551us
I20260709 13:19:23.450914 16790 leader_election.cc:304] T b87147731a68400e8cce9b879daa7c5d P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:19:23.451587 16192 leader_election.cc:304] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:19:23.456429 16790 leader_election.cc:304] T 9e65676b21b748728ad39668cfe722ec P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:19:23.457743 16790 leader_election.cc:304] T 434bff3082d141838068f323bdb0d1ed P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:19:23.459646 16790 leader_election.cc:304] T af57996a8be84d398f0f023e8d08ef77 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:19:23.448559 17715 leader_election.cc:290] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:19:23.445333 17694 raft_consensus.cc:493] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:19:23.455313 17710 raft_consensus.cc:2804] T b87147731a68400e8cce9b879daa7c5d P cb82cc7bd6db46d980c75237a1402860 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:19:23.464906 17709 raft_consensus.cc:2804] T 434bff3082d141838068f323bdb0d1ed P cb82cc7bd6db46d980c75237a1402860 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:19:23.460328 17638 raft_consensus.cc:697] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac [term 2 LEADER]: Becoming Leader. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:19:23.460879 17645 raft_consensus.cc:2804] T 9e65676b21b748728ad39668cfe722ec P cb82cc7bd6db46d980c75237a1402860 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:19:23.466059 17672 raft_consensus.cc:2804] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:19:23.458637 17693 raft_consensus.cc:493] T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:19:23.464599 17686 raft_consensus.cc:2804] T af57996a8be84d398f0f023e8d08ef77 P cb82cc7bd6db46d980c75237a1402860 [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260709 13:19:23.467489 16080 debug-util.cc:398] Leaking SignalData structure 0x7b080024f660 after lost signal to thread 16324
I20260709 13:19:23.458626 17718 leader_election.cc:290] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:23.485670 16222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "829b4ad8c5a54eababeea1181b960d4b" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:23.466665 17710 raft_consensus.cc:493] T b87147731a68400e8cce9b879daa7c5d P cb82cc7bd6db46d980c75237a1402860 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:19:23.472900 17715 raft_consensus.cc:493] T 84f2da98864b4425bf7c0055951fdd4b P 647a343a542d4a139fe106c19095129b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:19:23.474467 17709 raft_consensus.cc:493] T 434bff3082d141838068f323bdb0d1ed P cb82cc7bd6db46d980c75237a1402860 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:19:23.538255 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66f75124215f4ca6bef831e174a792d8" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:23.543226 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee9ed67006144797b1de6ea8c2013a68" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:23.481464 17645 raft_consensus.cc:493] T 9e65676b21b748728ad39668cfe722ec P cb82cc7bd6db46d980c75237a1402860 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:19:23.475646 17707 raft_consensus.cc:493] T c0eab25ce4c64e018a041beeda2867db P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
W20260709 13:19:23.490878 16080 debug-util.cc:398] Leaking SignalData structure 0x7b08002f0220 after lost signal to thread 16325
I20260709 13:19:23.483023 17672 raft_consensus.cc:493] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Starting leader election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:19:23.545625 16224 raft_consensus.cc:2468] T 66f75124215f4ca6bef831e174a792d8 P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 1.
I20260709 13:19:23.545879 16225 raft_consensus.cc:2468] T ee9ed67006144797b1de6ea8c2013a68 P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 1.
I20260709 13:19:23.544062 17701 leader_election.cc:290] T 66f75124215f4ca6bef831e174a792d8 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:19:23.547935 17729 raft_consensus.cc:493] T b6f07842d3aa47858df5665874c86f3a P 647a343a542d4a139fe106c19095129b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:19:23.465111 17644 raft_consensus.cc:515] T 7c2095a5f4ad49408dbc7f69c606120d P 647a343a542d4a139fe106c19095129b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:23.488587 17686 raft_consensus.cc:493] T af57996a8be84d398f0f023e8d08ef77 P cb82cc7bd6db46d980c75237a1402860 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:19:23.474763 17694 raft_consensus.cc:515] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:23.543283 17710 raft_consensus.cc:3060] T b87147731a68400e8cce9b879daa7c5d P cb82cc7bd6db46d980c75237a1402860 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:19:23.587005 17452 raft_consensus.cc:493] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:19:23.548094 16220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a378d7f796f94615b3b347119a9880fd" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:23.489564 17693 raft_consensus.cc:515] T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:23.544759 17717 raft_consensus.cc:493] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:19:23.547428 17709 raft_consensus.cc:3060] T 434bff3082d141838068f323bdb0d1ed P cb82cc7bd6db46d980c75237a1402860 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:19:23.733171 16151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87ac0d1f7e8a419f9bcbc04aee0e067e" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:19:23.548655 17645 raft_consensus.cc:3060] T 9e65676b21b748728ad39668cfe722ec P cb82cc7bd6db46d980c75237a1402860 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:19:23.733539 16145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4dc6b0541ca431ebc757beac00521f1" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:19:23.726351 16144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b7d1ab519044eda2d5e04a07242f25" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:19:23.580322 17642 raft_consensus.cc:493] T 8cc959fb7f704fba9009c0595467d074 P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:19:23.754329 16151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806d7690c2b644959152074bb2cab0a7" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:19:23.553610 17672 raft_consensus.cc:3060] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:19:23.740188 16080 debug-util.cc:398] Leaking SignalData structure 0x7b0800223f00 after lost signal to thread 16849
I20260709 13:19:23.546679 17715 raft_consensus.cc:515] T 84f2da98864b4425bf7c0055951fdd4b P 647a343a542d4a139fe106c19095129b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:23.580965 17686 raft_consensus.cc:3060] T af57996a8be84d398f0f023e8d08ef77 P cb82cc7bd6db46d980c75237a1402860 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:19:23.760710 17452 raft_consensus.cc:515] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:23.463313 17723 raft_consensus.cc:515] T 5a96373bb0e14dce8a266b2fc6c58507 P 647a343a542d4a139fe106c19095129b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:23.616986 17685 leader_election.cc:290] T 18ad7e3af5504571a69d10c00750a01c P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:19:23.627454 17644 leader_election.cc:290] T 7c2095a5f4ad49408dbc7f69c606120d P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:23.617941 16146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f82224f4564fea93572e8206bad1fe" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
W20260709 13:19:23.769505 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 290414us
I20260709 13:19:23.618685 17694 leader_election.cc:290] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:23.587952 17719 leader_election.cc:290] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
W20260709 13:19:23.770903 16080 debug-util.cc:398] Leaking SignalData structure 0x7b08001db120 after lost signal to thread 17562
I20260709 13:19:23.740432 17702 leader_election.cc:290] T 5a959dbecc2648eb85d1cc550f5a3245 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:23.765741 16220 raft_consensus.cc:2468] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 1.
I20260709 13:19:23.773429 16151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5d59407eb545b895d8d922e0dd69fd" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:19:23.786388 17452 leader_election.cc:290] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:23.786409 16825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434bff3082d141838068f323bdb0d1ed" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:19:23.627920 17725 raft_consensus.cc:493] T fffab26427e24d398e2695aad40ca134 P 647a343a542d4a139fe106c19095129b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:19:23.725968 17673 leader_election.cc:290] T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:23.780175 17733 raft_consensus.cc:493] T 8a878999c0024d2abe517484180290fc P 647a343a542d4a139fe106c19095129b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:19:23.709945 17720 raft_consensus.cc:493] T 9c61a39fd4f84424b129c1a6f9ba404d P 647a343a542d4a139fe106c19095129b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:19:23.787943 16790 leader_election.cc:304] T ee9ed67006144797b1de6ea8c2013a68 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 647a343a542d4a139fe106c19095129b
I20260709 13:19:23.788095 16821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "829b4ad8c5a54eababeea1181b960d4b" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:19:23.783670 16142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:19:23.793383 16790 leader_election.cc:304] T 66f75124215f4ca6bef831e174a792d8 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 647a343a542d4a139fe106c19095129b
I20260709 13:19:23.788589 16117 leader_election.cc:304] T b87147731a68400e8cce9b879daa7c5d P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:19:23.789222 16824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62fe3346cd3c42c4a55e14af9073bca4" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:19:23.625299 16143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31969392ccdb43b491911a319e8bc7e1" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:19:23.721274 17638 consensus_queue.cc:237] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:23.771953 17724 leader_election.cc:290] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:19:23.789968 16221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af57996a8be84d398f0f023e8d08ef77" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:23.766567 17642 raft_consensus.cc:515] T 8cc959fb7f704fba9009c0595467d074 P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:23.792119 16151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d16dfad0c3b540e5b4b86492ddf87208" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:19:23.778218 17722 leader_election.cc:290] T 68a6838d82ef4cac83507f906139d90f P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:23.801306 16151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee9ed67006144797b1de6ea8c2013a68" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:19:23.804121 16821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66ae176961b74813a58bc15c9c4c6f2a" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:19:23.804695 16824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66f75124215f4ca6bef831e174a792d8" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:19:23.548316 17707 raft_consensus.cc:515] T c0eab25ce4c64e018a041beeda2867db P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:23.714433 17729 raft_consensus.cc:515] T b6f07842d3aa47858df5665874c86f3a P 647a343a542d4a139fe106c19095129b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:23.806514 16820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a378d7f796f94615b3b347119a9880fd" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:19:23.794728 17452 raft_consensus.cc:2804] T b87147731a68400e8cce9b879daa7c5d P 647a343a542d4a139fe106c19095129b [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:19:23.792611 16819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746aabf856ab401c9ef5e859371e7b67" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:19:23.778524 17685 raft_consensus.cc:493] T ecc1e363230043fe9fef390f0b979f20 P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:19:23.807046 16820 raft_consensus.cc:2468] T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 1.
I20260709 13:19:23.807219 17452 raft_consensus.cc:493] T b87147731a68400e8cce9b879daa7c5d P 647a343a542d4a139fe106c19095129b [term 1 FOLLOWER]: Starting leader election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:19:23.809072 16145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6f07842d3aa47858df5665874c86f3a" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:19:23.809288 17452 raft_consensus.cc:3060] T b87147731a68400e8cce9b879daa7c5d P 647a343a542d4a139fe106c19095129b [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:19:23.809329 15051 debug-util.cc:398] Leaking SignalData structure 0x7b080000f720 after lost signal to thread 16116
I20260709 13:19:23.815639 16151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f5f89186204242bf377f7c42b2e2da" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:19:23.816847 16151 raft_consensus.cc:2468] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 2.
I20260709 13:19:23.818112 16147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8158bbd17b994d2eb96f560bd0f1b88d" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:19:23.818593 16147 raft_consensus.cc:2468] T 8158bbd17b994d2eb96f560bd0f1b88d P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 2.
I20260709 13:19:23.820343 17715 leader_election.cc:290] T 84f2da98864b4425bf7c0055951fdd4b P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:23.820427 16223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9944d5e4b8c4109af9955c4f08c9095" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:23.824224 16144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5d59407eb545b895d8d922e0dd69fd" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:19:23.826458 16144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1df93779209748b9860f6d893dd3de53" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:19:23.827131 16145 raft_consensus.cc:2468] T b6f07842d3aa47858df5665874c86f3a P 647a343a542d4a139fe106c19095129b [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 1.
I20260709 13:19:23.827654 16192 leader_election.cc:304] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:19:23.807982 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02838c86531d4f98bf58d1c22b8baf77" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:19:23.801019 17736 raft_consensus.cc:2804] T ee9ed67006144797b1de6ea8c2013a68 P cb82cc7bd6db46d980c75237a1402860 [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260709 13:19:23.817541 16080 debug-util.cc:398] Leaking SignalData structure 0x7b080034ec20 after lost signal to thread 17668
I20260709 13:19:23.829300 16147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d4ab870a05347868956aa8b841b2981" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:19:23.808056 17581 raft_consensus.cc:2804] T 66f75124215f4ca6bef831e174a792d8 P cb82cc7bd6db46d980c75237a1402860 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:19:23.832556 16147 raft_consensus.cc:2468] T 8d4ab870a05347868956aa8b841b2981 P 647a343a542d4a139fe106c19095129b [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 1.
I20260709 13:19:23.832590 16144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62fe3346cd3c42c4a55e14af9073bca4" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:19:23.833851 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e4bfb9090c8460eb14fa7e0d818e7f1" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:23.850248 17726 raft_consensus.cc:2804] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:19:23.830286 16149 raft_consensus.cc:2468] T 02838c86531d4f98bf58d1c22b8baf77 P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 2.
I20260709 13:19:23.850695 17726 raft_consensus.cc:493] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Starting leader election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:19:23.772925 17717 raft_consensus.cc:515] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } }
I20260709 13:19:23.798137 17733 raft_consensus.cc:515] T 8a878999c0024d2abe517484180290fc P 647a343a542d4a139fe106c19095129b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:23.868130 15046 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:19:23.836539 17736 raft_consensus.cc:493] T ee9ed67006144797b1de6ea8c2013a68 P cb82cc7bd6db46d980c75237a1402860 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:19:23.873903 16144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4f65cdf00b748bb963ee6e81ef43b79" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:19:23.874781 16144 raft_consensus.cc:2468] T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 2.
I20260709 13:19:23.851127 17726 raft_consensus.cc:3060] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:19:23.886600 17668 raft_consensus.cc:493] T 57a744f5aaf944678a63469cbf514015 P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:19:23.851472 17581 raft_consensus.cc:493] T 66f75124215f4ca6bef831e174a792d8 P cb82cc7bd6db46d980c75237a1402860 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:19:23.825453 17685 raft_consensus.cc:515] T ecc1e363230043fe9fef390f0b979f20 P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } }
I20260709 13:19:23.798400 17720 raft_consensus.cc:515] T 9c61a39fd4f84424b129c1a6f9ba404d P 647a343a542d4a139fe106c19095129b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:23.896945 17581 raft_consensus.cc:3060] T 66f75124215f4ca6bef831e174a792d8 P cb82cc7bd6db46d980c75237a1402860 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:19:23.874104 17723 leader_election.cc:290] T 5a96373bb0e14dce8a266b2fc6c58507 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:23.889041 17736 raft_consensus.cc:3060] T ee9ed67006144797b1de6ea8c2013a68 P cb82cc7bd6db46d980c75237a1402860 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:19:23.903045 17714 leader_election.cc:290] T 8158bbd17b994d2eb96f560bd0f1b88d P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:23.872893 17693 leader_election.cc:290] T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:23.887398 17668 raft_consensus.cc:515] T 57a744f5aaf944678a63469cbf514015 P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:23.908615 17642 leader_election.cc:290] T 8cc959fb7f704fba9009c0595467d074 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:23.888573 16142 raft_consensus.cc:2468] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 2.
I20260709 13:19:23.910815 16144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66f75124215f4ca6bef831e174a792d8" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:19:23.909889 16147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c61a39fd4f84424b129c1a6f9ba404d" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:19:23.914417 16147 raft_consensus.cc:2468] T 9c61a39fd4f84424b129c1a6f9ba404d P 647a343a542d4a139fe106c19095129b [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 1.
I20260709 13:19:23.909452 17704 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ecc1e363230043fe9fef390f0b979f20. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:19:23.795876 17725 raft_consensus.cc:515] T fffab26427e24d398e2695aad40ca134 P 647a343a542d4a139fe106c19095129b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:23.920782 17717 leader_election.cc:290] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:23.922008 16219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74ca8140f35c4ae29e8a664f6bdfc829" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:23.923892 16219 raft_consensus.cc:2468] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 1.
I20260709 13:19:23.928541 17735 consensus_queue.cc:1059] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.141s
I20260709 13:19:23.959034 17735 raft_consensus.cc:3055] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:19:23.959421 17735 raft_consensus.cc:740] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:19:23.960244 17735 consensus_queue.cc:260] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:23.961445 17735 raft_consensus.cc:3060] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:19:23.932427 17720 leader_election.cc:290] T 9c61a39fd4f84424b129c1a6f9ba404d P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:23.963510 16790 leader_election.cc:304] T 829b4ad8c5a54eababeea1181b960d4b P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac
I20260709 13:19:23.964335 17733 leader_election.cc:290] T 8a878999c0024d2abe517484180290fc P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:23.965143 17681 tablet_copy_client.cc:538] T 806d7690c2b644959152074bb2cab0a7 P cb82cc7bd6db46d980c75237a1402860: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:19:23.966641 17725 leader_election.cc:290] T fffab26427e24d398e2695aad40ca134 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:23.967846 17668 leader_election.cc:290] T 57a744f5aaf944678a63469cbf514015 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:19:23.984113 16222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "829b4ad8c5a54eababeea1181b960d4b" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:23.984395 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c74490c40243a88e62d1760b5cd814" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:23.984261 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68a6838d82ef4cac83507f906139d90f" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:19:24.007234 17707 leader_election.cc:290] T c0eab25ce4c64e018a041beeda2867db P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:24.337006 16191 leader_election.cc:304] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
I20260709 13:19:24.339646 16149 raft_consensus.cc:2468] T 68a6838d82ef4cac83507f906139d90f P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 2.
I20260709 13:19:24.340269 17638 raft_consensus.cc:2749] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:19:24.067839 17586 raft_consensus.cc:493] T c0eab25ce4c64e018a041beeda2867db P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:19:24.215195 17581 raft_consensus.cc:515] T 66f75124215f4ca6bef831e174a792d8 P cb82cc7bd6db46d980c75237a1402860 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:24.054293 17658 raft_consensus.cc:493] T d16dfad0c3b540e5b4b86492ddf87208 P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:19:24.342226 17658 raft_consensus.cc:515] T d16dfad0c3b540e5b4b86492ddf87208 P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:24.065312 17697 raft_consensus.cc:493] T 7c2095a5f4ad49408dbc7f69c606120d P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:19:24.341634 17586 raft_consensus.cc:515] T c0eab25ce4c64e018a041beeda2867db P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:24.343392 17581 leader_election.cc:290] T 66f75124215f4ca6bef831e174a792d8 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 2 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:19:24.343631 16191 leader_election.cc:304] T ee9ed67006144797b1de6ea8c2013a68 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
W20260709 13:19:24.195621 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 1.36275e+06us
I20260709 13:19:24.344326 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d4ab870a05347868956aa8b841b2981" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:24.344543 17638 raft_consensus.cc:2749] T ee9ed67006144797b1de6ea8c2013a68 P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:19:24.344950 16226 raft_consensus.cc:2468] T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 1.
I20260709 13:19:24.345636 16191 leader_election.cc:304] T c2f5f89186204242bf377f7c42b2e2da P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
I20260709 13:19:24.346315 17638 raft_consensus.cc:2804] T c2f5f89186204242bf377f7c42b2e2da P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:19:24.346663 17638 raft_consensus.cc:493] T c2f5f89186204242bf377f7c42b2e2da P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Starting leader election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:19:24.346998 17638 raft_consensus.cc:3060] T c2f5f89186204242bf377f7c42b2e2da P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:19:24.349391 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5ad4eb281564936a59f41e387a47156" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:24.350485 17658 leader_election.cc:290] T d16dfad0c3b540e5b4b86492ddf87208 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:19:24.034086 17691 raft_consensus.cc:493] T 2794af5b99984efebdee564035841cb0 P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:19:24.351363 17691 raft_consensus.cc:515] T 2794af5b99984efebdee564035841cb0 P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:24.352205 16222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f2da98864b4425bf7c0055951fdd4b" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:24.353645 16222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c2095a5f4ad49408dbc7f69c606120d" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:24.068013 16220 raft_consensus.cc:1275] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:19:24.179855 17726 raft_consensus.cc:515] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:24.280535 17685 leader_election.cc:290] T ecc1e363230043fe9fef390f0b979f20 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:23.983248 17738 raft_consensus.cc:2804] T 829b4ad8c5a54eababeea1181b960d4b P cb82cc7bd6db46d980c75237a1402860 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:19:24.190001 16790 leader_election.cc:304] T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:19:24.357064 17738 raft_consensus.cc:493] T 829b4ad8c5a54eababeea1181b960d4b P cb82cc7bd6db46d980c75237a1402860 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
W20260709 13:19:24.357491 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 167548us
I20260709 13:19:24.064114 17662 raft_consensus.cc:493] T 3884f864a96d428785895e5463e5728f P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:19:24.361541 16191 leader_election.cc:304] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
I20260709 13:19:24.362025 16790 leader_election.cc:304] T 74ca8140f35c4ae29e8a664f6bdfc829 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 
W20260709 13:19:24.051968 16117 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 249279us
I20260709 13:19:24.354642 16222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18ad7e3af5504571a69d10c00750a01c" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:24.360582 17518 raft_consensus.cc:493] T ada5a004ba7c46819051e3ccec71ffdb P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:19:24.370075 17518 raft_consensus.cc:515] T ada5a004ba7c46819051e3ccec71ffdb P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:24.372347 17518 leader_election.cc:290] T ada5a004ba7c46819051e3ccec71ffdb P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:19:24.356411 16220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62fe3346cd3c42c4a55e14af9073bca4" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
W20260709 13:19:24.319480 17686 consensus_meta.cc:287] T af57996a8be84d398f0f023e8d08ef77 P cb82cc7bd6db46d980c75237a1402860: Time spent flushing consensus metadata: real 0.540s	user 0.015s	sys 0.019s
I20260709 13:19:24.005520 17729 leader_election.cc:290] T b6f07842d3aa47858df5665874c86f3a P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:24.365581 17695 raft_consensus.cc:2804] T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:19:24.357616 17738 raft_consensus.cc:3060] T 829b4ad8c5a54eababeea1181b960d4b P cb82cc7bd6db46d980c75237a1402860 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:19:24.358022 17691 leader_election.cc:290] T 2794af5b99984efebdee564035841cb0 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:19:24.383782 16222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f5f89186204242bf377f7c42b2e2da" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:24.386631 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746aabf856ab401c9ef5e859371e7b67" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:24.388000 16191 leader_election.cc:304] T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
I20260709 13:19:24.374256 17722 raft_consensus.cc:2804] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:19:24.354579 17638 raft_consensus.cc:515] T c2f5f89186204242bf377f7c42b2e2da P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:24.376729 17730 raft_consensus.cc:2804] T 74ca8140f35c4ae29e8a664f6bdfc829 P cb82cc7bd6db46d980c75237a1402860 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:19:24.394141 16191 leader_election.cc:304] T 02838c86531d4f98bf58d1c22b8baf77 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
I20260709 13:19:24.409371 16191 leader_election.cc:304] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
I20260709 13:19:24.409442 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ad5dbfa99ee4b8d9e28be92b3f36db5" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:24.343683 17697 raft_consensus.cc:515] T 7c2095a5f4ad49408dbc7f69c606120d P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:24.399660 17657 raft_consensus.cc:493] T 66ae176961b74813a58bc15c9c4c6f2a P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:19:24.400983 17658 raft_consensus.cc:493] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:19:24.401429 17744 raft_consensus.cc:493] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:19:24.401818 17562 raft_consensus.cc:493] T 84f2da98864b4425bf7c0055951fdd4b P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:19:24.402174 17691 raft_consensus.cc:493] T e4dc6b0541ca431ebc757beac00521f1 P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:19:24.390113 17665 raft_consensus.cc:493] T d2ab0ac8347641e6955e9bce7d90c1a2 P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:19:24.383235 16220 raft_consensus.cc:2468] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 2.
I20260709 13:19:24.397754 17518 raft_consensus.cc:493] T 1df93779209748b9860f6d893dd3de53 P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:19:24.412853 17746 raft_consensus.cc:493] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:19:24.383383 17726 leader_election.cc:290] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 2 election: Requested vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:19:24.388397 17695 raft_consensus.cc:493] T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:19:24.408666 17730 raft_consensus.cc:493] T 74ca8140f35c4ae29e8a664f6bdfc829 P cb82cc7bd6db46d980c75237a1402860 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
W20260709 13:19:24.396418 17710 consensus_meta.cc:287] T b87147731a68400e8cce9b879daa7c5d P cb82cc7bd6db46d980c75237a1402860: Time spent flushing consensus metadata: real 0.632s	user 0.005s	sys 0.005s
I20260709 13:19:24.417886 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cc959fb7f704fba9009c0595467d074" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:24.397717 17724 raft_consensus.cc:2804] T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:19:24.374130 17662 raft_consensus.cc:515] T 3884f864a96d428785895e5463e5728f P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:24.421806 16222 raft_consensus.cc:2393] T c2f5f89186204242bf377f7c42b2e2da P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 3: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
I20260709 13:19:24.407723 17745 raft_consensus.cc:493] T 26354361e9b044cea1c2b86c4882fe92 P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:19:24.423985 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8158bbd17b994d2eb96f560bd0f1b88d" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:24.398134 17722 raft_consensus.cc:493] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Starting leader election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:19:24.381073 17686 raft_consensus.cc:515] T af57996a8be84d398f0f023e8d08ef77 P cb82cc7bd6db46d980c75237a1402860 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:24.416500 17673 raft_consensus.cc:2804] T 02838c86531d4f98bf58d1c22b8baf77 P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:19:24.480719 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c61a39fd4f84424b129c1a6f9ba404d" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
W20260709 13:19:24.498799 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 89470.7us
I20260709 13:19:24.473860 17695 raft_consensus.cc:3060] T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:19:24.514393 16191 leader_election.cc:304] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
I20260709 13:19:24.461555 17562 raft_consensus.cc:515] T 84f2da98864b4425bf7c0055951fdd4b P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:24.418521 17657 raft_consensus.cc:515] T 66ae176961b74813a58bc15c9c4c6f2a P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:24.513931 16225 raft_consensus.cc:2468] T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 1.
I20260709 13:19:24.479897 17722 raft_consensus.cc:3060] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:19:24.478638 17724 raft_consensus.cc:493] T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Starting leader election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:19:24.498831 17673 raft_consensus.cc:493] T 02838c86531d4f98bf58d1c22b8baf77 P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Starting leader election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:19:24.465555 17691 raft_consensus.cc:515] T e4dc6b0541ca431ebc757beac00521f1 P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:24.476258 17730 raft_consensus.cc:3060] T 74ca8140f35c4ae29e8a664f6bdfc829 P cb82cc7bd6db46d980c75237a1402860 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:19:24.467521 17518 raft_consensus.cc:515] T 1df93779209748b9860f6d893dd3de53 P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:24.467803 17665 raft_consensus.cc:515] T d2ab0ac8347641e6955e9bce7d90c1a2 P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:24.420665 17658 raft_consensus.cc:515] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:24.555531 16151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57a744f5aaf944678a63469cbf514015" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:19:24.425320 17744 raft_consensus.cc:515] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:24.564494 17665 leader_election.cc:290] T d2ab0ac8347641e6955e9bce7d90c1a2 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:19:24.564885 17586 leader_election.cc:290] T c0eab25ce4c64e018a041beeda2867db P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:19:24.565122 17665 raft_consensus.cc:493] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:19:24.555039 16819 raft_consensus.cc:1240] T af57996a8be84d398f0f023e8d08ef77 P cb82cc7bd6db46d980c75237a1402860 [term 2 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 1. Current term is 2. Ops: []
I20260709 13:19:24.565685 17665 raft_consensus.cc:515] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:24.481033 17726 raft_consensus.cc:2749] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:19:24.560451 16190 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.14.177.131:41389 timed out after 1.526s (SENT)
I20260709 13:19:24.561627 17724 raft_consensus.cc:3060] T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:19:24.475405 17746 raft_consensus.cc:515] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:24.562621 17719 raft_consensus.cc:2804] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:19:24.563762 17518 leader_election.cc:290] T 1df93779209748b9860f6d893dd3de53 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:19:24.568320 16116 leader_election.cc:304] T 829b4ad8c5a54eababeea1181b960d4b P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860
I20260709 13:19:24.567049 17662 leader_election.cc:290] T 3884f864a96d428785895e5463e5728f P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:19:24.532761 17704 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.StartTabletCopy from 127.0.0.1:49906 (request call id 341) took 2996 ms. Trace:
I20260709 13:19:24.478948 17710 raft_consensus.cc:515] T b87147731a68400e8cce9b879daa7c5d P cb82cc7bd6db46d980c75237a1402860 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:24.557250 16220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6f07842d3aa47858df5665874c86f3a" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:24.582630 16116 leader_election.cc:304] T 8d4ab870a05347868956aa8b841b2981 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
W20260709 13:19:24.582612 17743 log.cc:927] Time spent T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 0.120s	user 0.000s	sys 0.001s
I20260709 13:19:24.560228 17673 raft_consensus.cc:3060] T 02838c86531d4f98bf58d1c22b8baf77 P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:19:24.576800 17719 raft_consensus.cc:493] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Starting leader election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:19:24.565737 17744 leader_election.cc:290] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
W20260709 13:19:24.585130 17452 consensus_meta.cc:287] T b87147731a68400e8cce9b879daa7c5d P 647a343a542d4a139fe106c19095129b: Time spent flushing consensus metadata: real 0.763s	user 0.011s	sys 0.003s
I20260709 13:19:24.586503 17723 raft_consensus.cc:2749] T 829b4ad8c5a54eababeea1181b960d4b P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:19:24.620376 17452 raft_consensus.cc:515] T b87147731a68400e8cce9b879daa7c5d P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:24.637107 16147 raft_consensus.cc:1240] T b87147731a68400e8cce9b879daa7c5d P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 1. Current term is 2. Ops: []
I20260709 13:19:24.502058 17745 raft_consensus.cc:515] T 26354361e9b044cea1c2b86c4882fe92 P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:19:24.643328 16789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 82037.4us
I20260709 13:19:24.645331 17452 leader_election.cc:290] T b87147731a68400e8cce9b879daa7c5d P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 2 election: Requested vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:19:24.607685 17685 raft_consensus.cc:2804] T 8d4ab870a05347868956aa8b841b2981 P 647a343a542d4a139fe106c19095129b [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260709 13:19:24.645576 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 85157.4us
I20260709 13:19:24.646672 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada5a004ba7c46819051e3ccec71ffdb" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:24.607491 17662 raft_consensus.cc:493] T 23f82224f4564fea93572e8206bad1fe P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:19:24.621744 17719 raft_consensus.cc:3060] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:19:24.649307 16299 raft_consensus.cc:2468] T ada5a004ba7c46819051e3ccec71ffdb P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 2.
I20260709 13:19:24.608174 17750 raft_consensus.cc:493] T 18ad7e3af5504571a69d10c00750a01c P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
W20260709 13:19:24.658401 17672 consensus_meta.cc:287] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac: Time spent flushing consensus metadata: real 0.888s	user 0.005s	sys 0.021s
I20260709 13:19:24.659382 17665 leader_election.cc:290] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:19:24.659546 17657 leader_election.cc:290] T 66ae176961b74813a58bc15c9c4c6f2a P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:19:24.660605 17658 leader_election.cc:290] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:19:24.669473 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a878999c0024d2abe517484180290fc" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:24.669935 17754 raft_consensus.cc:493] T 8cc959fb7f704fba9009c0595467d074 P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:19:24.670423 17754 raft_consensus.cc:515] T 8cc959fb7f704fba9009c0595467d074 P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:24.670704 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:24.671382 17586 raft_consensus.cc:493] T 8c11074579b046d99e585c2818e18a2c P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:19:24.671381 16225 raft_consensus.cc:2468] T 19baa7bc367844c288bca8f81439c413 P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 2.
I20260709 13:19:24.671686 17752 raft_consensus.cc:493] T 31969392ccdb43b491911a319e8bc7e1 P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:19:24.672062 17586 raft_consensus.cc:515] T 8c11074579b046d99e585c2818e18a2c P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:24.672451 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a96373bb0e14dce8a266b2fc6c58507" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:24.672319 17752 raft_consensus.cc:515] T 31969392ccdb43b491911a319e8bc7e1 P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:24.673480 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4f65cdf00b748bb963ee6e81ef43b79" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:24.674284 17586 leader_election.cc:290] T 8c11074579b046d99e585c2818e18a2c P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:19:24.674654 17752 leader_election.cc:290] T 31969392ccdb43b491911a319e8bc7e1 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:19:24.670609 17744 raft_consensus.cc:493] T ecc1e363230043fe9fef390f0b979f20 P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:19:24.691916 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4dc6b0541ca431ebc757beac00521f1" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:24.692749 16298 raft_consensus.cc:2468] T e4dc6b0541ca431ebc757beac00521f1 P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 3.
I20260709 13:19:24.596269 17518 raft_consensus.cc:493] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:19:24.609151 16220 raft_consensus.cc:2468] T b6f07842d3aa47858df5665874c86f3a P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 1.
I20260709 13:19:24.677335 16116 leader_election.cc:304] T 746aabf856ab401c9ef5e859371e7b67 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860
I20260709 13:19:24.680222 17562 leader_election.cc:290] T 84f2da98864b4425bf7c0055951fdd4b P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:19:24.680343 16820 raft_consensus.cc:1240] T 66f75124215f4ca6bef831e174a792d8 P cb82cc7bd6db46d980c75237a1402860 [term 2 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 1. Current term is 2. Ops: []
I20260709 13:19:24.680642 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ab0ac8347641e6955e9bce7d90c1a2" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:24.697667 16300 raft_consensus.cc:2468] T d2ab0ac8347641e6955e9bce7d90c1a2 P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 2.
I20260709 13:19:24.680898 17697 leader_election.cc:290] T 7c2095a5f4ad49408dbc7f69c606120d P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:19:24.698084 16822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87147731a68400e8cce9b879daa7c5d" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:19:24.698907 16822 raft_consensus.cc:2393] T b87147731a68400e8cce9b879daa7c5d P cb82cc7bd6db46d980c75237a1402860 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 2: Already voted for candidate cb82cc7bd6db46d980c75237a1402860 in this term.
I20260709 13:19:24.699065 17746 leader_election.cc:290] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:19:24.658891 17672 raft_consensus.cc:515] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:24.682018 16295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87ac0d1f7e8a419f9bcbc04aee0e067e" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:24.699896 17662 raft_consensus.cc:515] T 23f82224f4564fea93572e8206bad1fe P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:24.700474 16223 raft_consensus.cc:2393] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 in current term 3: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
I20260709 13:19:24.682854 16218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
W20260709 13:19:24.685701 17709 consensus_meta.cc:287] T 434bff3082d141838068f323bdb0d1ed P cb82cc7bd6db46d980c75237a1402860: Time spent flushing consensus metadata: real 0.921s	user 0.008s	sys 0.042s
W20260709 13:19:24.690339 17736 consensus_meta.cc:287] T ee9ed67006144797b1de6ea8c2013a68 P cb82cc7bd6db46d980c75237a1402860: Time spent flushing consensus metadata: real 0.776s	user 0.007s	sys 0.003s
I20260709 13:19:24.690274 16296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74ca8140f35c4ae29e8a664f6bdfc829" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:24.700295 16295 raft_consensus.cc:2468] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 2.
I20260709 13:19:24.682598 17671 consensus_queue.cc:1059] T af57996a8be84d398f0f023e8d08ef77 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:19:24.716389 17717 raft_consensus.cc:2749] T 746aabf856ab401c9ef5e859371e7b67 P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:19:24.716574 17754 leader_election.cc:290] T 8cc959fb7f704fba9009c0595467d074 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:19:24.716873 17671 raft_consensus.cc:3055] T af57996a8be84d398f0f023e8d08ef77 P 1ca2725d290e47d68b1a4b609c9816ac [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:19:24.647535 17729 consensus_queue.cc:1048] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.048s
I20260709 13:19:24.650287 17685 raft_consensus.cc:493] T 8d4ab870a05347868956aa8b841b2981 P 647a343a542d4a139fe106c19095129b [term 1 FOLLOWER]: Starting leader election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:19:24.709622 16116 leader_election.cc:304] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: cb82cc7bd6db46d980c75237a1402860
I20260709 13:19:24.642608 17747 raft_consensus.cc:493] T 8638ee363273426db1393dd81e5c7b72 P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:19:24.707545 17750 raft_consensus.cc:515] T 18ad7e3af5504571a69d10c00750a01c P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } }
I20260709 13:19:24.717281 17671 raft_consensus.cc:740] T af57996a8be84d398f0f023e8d08ef77 P 1ca2725d290e47d68b1a4b609c9816ac [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
W20260709 13:19:24.670974 17645 consensus_meta.cc:287] T 9e65676b21b748728ad39668cfe722ec P cb82cc7bd6db46d980c75237a1402860: Time spent flushing consensus metadata: real 0.904s	user 0.011s	sys 0.011s
I20260709 13:19:24.694113 17638 leader_election.cc:290] T c2f5f89186204242bf377f7c42b2e2da P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 3 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:24.780272 17518 raft_consensus.cc:515] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:24.782912 17759 raft_consensus.cc:493] T e5ad4eb281564936a59f41e387a47156 P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
W20260709 13:19:24.785374 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 265795us
I20260709 13:19:24.788187 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87147731a68400e8cce9b879daa7c5d" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:24.789536 16296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434bff3082d141838068f323bdb0d1ed" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:24.798035 16191 leader_election.cc:304] T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
I20260709 13:19:24.790220 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c74490c40243a88e62d1760b5cd814" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:24.790333 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68a6838d82ef4cac83507f906139d90f" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:24.791224 16295 raft_consensus.cc:1275] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:19:24.778257 17685 raft_consensus.cc:3060] T 8d4ab870a05347868956aa8b841b2981 P 647a343a542d4a139fe106c19095129b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:19:24.802583 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af57996a8be84d398f0f023e8d08ef77" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:19:24.803261 16150 raft_consensus.cc:3060] T af57996a8be84d398f0f023e8d08ef77 P 647a343a542d4a139fe106c19095129b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:19:24.799218 16191 leader_election.cc:304] T 66f75124215f4ca6bef831e174a792d8 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
W20260709 13:19:24.801651 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 92088.2us
I20260709 13:19:24.805296 17726 raft_consensus.cc:2804] T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:19:24.805838 17745 leader_election.cc:290] T 26354361e9b044cea1c2b86c4882fe92 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:19:24.807032 16116 leader_election.cc:304] T 9c61a39fd4f84424b129c1a6f9ba404d P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
I20260709 13:19:24.808190 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d4ab870a05347868956aa8b841b2981" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:24.849614 17452 raft_consensus.cc:2804] T 9c61a39fd4f84424b129c1a6f9ba404d P 647a343a542d4a139fe106c19095129b [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:19:24.850045 17452 raft_consensus.cc:493] T 9c61a39fd4f84424b129c1a6f9ba404d P 647a343a542d4a139fe106c19095129b [term 1 FOLLOWER]: Starting leader election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:19:24.850376 17452 raft_consensus.cc:3060] T 9c61a39fd4f84424b129c1a6f9ba404d P 647a343a542d4a139fe106c19095129b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:19:24.701690 17744 raft_consensus.cc:515] T ecc1e363230043fe9fef390f0b979f20 P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } }
W20260709 13:19:24.859625 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 52666us
I20260709 13:19:24.783492 17747 raft_consensus.cc:515] T 8638ee363273426db1393dd81e5c7b72 P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:24.724970 17736 raft_consensus.cc:515] T ee9ed67006144797b1de6ea8c2013a68 P cb82cc7bd6db46d980c75237a1402860 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:24.720723 17709 raft_consensus.cc:515] T 434bff3082d141838068f323bdb0d1ed P cb82cc7bd6db46d980c75237a1402860 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:24.586364 17704 rpcz_store.cc:276] 0709 13:19:21.527747 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:21.528141 (+   394us) service_pool.cc:224] Handling call
0709 13:19:21.551058 (+ 22917us) ts_tablet_manager.cc:934] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860: Initiating tablet copy from peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
0709 13:19:23.778364 (+2227306us) tablet_copy_client.cc:368] Tablet copy session begun
0709 13:19:24.402627 (+624263us) tablet_metadata.cc:665] Metadata flushed
0709 13:19:24.402751 (+   124us) tablet_copy_client.cc:497] Wrote new tablet metadata
0709 13:19:24.408132 (+  5381us) tablet_copy_client.cc:418] Wrote new consensus metadata
0709 13:19:24.466967 (+ 58835us) tablet_replica.cc:206] Creating consensus instance
0709 13:19:24.518639 (+ 51672us) ts_tablet_manager.cc:989] Replica registered.
0709 13:19:24.523819 (+  5180us) inbound_call.cc:177] Queueing success response
Metrics: {"dns_us":249,"tablet-copy.queue_time_us":21938,"thread_start_us":30532,"threads_started":2}
I20260709 13:19:24.864667 16116 leader_election.cc:304] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
I20260709 13:19:24.792845 17645 raft_consensus.cc:515] T 9e65676b21b748728ad39668cfe722ec P cb82cc7bd6db46d980c75237a1402860 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:24.807574 17644 raft_consensus.cc:2804] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:19:24.870842 17452 raft_consensus.cc:515] T 9c61a39fd4f84424b129c1a6f9ba404d P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:24.808805 17686 leader_election.cc:290] T af57996a8be84d398f0f023e8d08ef77 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 2 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:19:24.820278 17751 raft_consensus.cc:2749] T 66f75124215f4ca6bef831e174a792d8 P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:19:24.846611 17726 raft_consensus.cc:493] T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Starting leader election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:19:24.881677 16824 raft_consensus.cc:1240] T ee9ed67006144797b1de6ea8c2013a68 P cb82cc7bd6db46d980c75237a1402860 [term 2 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 1. Current term is 2. Ops: []
I20260709 13:19:24.794397 17717 consensus_queue.cc:1059] T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.072s
I20260709 13:19:24.884478 16825 raft_consensus.cc:2393] T 434bff3082d141838068f323bdb0d1ed P cb82cc7bd6db46d980c75237a1402860 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 3: Already voted for candidate cb82cc7bd6db46d980c75237a1402860 in this term.
I20260709 13:19:24.860993 17691 leader_election.cc:290] T e4dc6b0541ca431ebc757beac00521f1 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:19:24.866086 17662 leader_election.cc:290] T 23f82224f4564fea93572e8206bad1fe P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:19:24.866547 16220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0eab25ce4c64e018a041beeda2867db" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:24.889395 17452 leader_election.cc:290] T 9c61a39fd4f84424b129c1a6f9ba404d P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 2 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:24.886044 17681 tablet_bootstrap.cc:492] T 806d7690c2b644959152074bb2cab0a7 P cb82cc7bd6db46d980c75237a1402860: Bootstrap starting.
I20260709 13:19:24.881790 17704 tablet_copy_client.cc:806] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860: tablet copy: Starting download of 0 data blocks...
I20260709 13:19:24.880781 17644 raft_consensus.cc:493] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Starting leader election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:19:24.886101 17726 raft_consensus.cc:3060] T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:19:24.904886 17644 raft_consensus.cc:3060] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:19:24.905061 17739 raft_consensus.cc:2804] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:19:24.906524 17744 leader_election.cc:290] T ecc1e363230043fe9fef390f0b979f20 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:19:24.909999 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5ad4eb281564936a59f41e387a47156" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:24.807009 17759 raft_consensus.cc:515] T e5ad4eb281564936a59f41e387a47156 P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:24.898360 17738 raft_consensus.cc:515] T 829b4ad8c5a54eababeea1181b960d4b P cb82cc7bd6db46d980c75237a1402860 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:24.883569 17671 consensus_queue.cc:260] T af57996a8be84d398f0f023e8d08ef77 P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:24.943003 17750 leader_election.cc:290] T 18ad7e3af5504571a69d10c00750a01c P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:19:24.912020 17704 tablet_copy_client.cc:670] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860: tablet copy: Starting download of 1 WAL segments...
I20260709 13:19:24.912402 17710 leader_election.cc:290] T b87147731a68400e8cce9b879daa7c5d P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 2 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:24.935328 17672 leader_election.cc:290] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 3 election: Requested vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:19:24.937397 17736 leader_election.cc:290] T ee9ed67006144797b1de6ea8c2013a68 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 2 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:19:24.954581 17747 leader_election.cc:290] T 8638ee363273426db1393dd81e5c7b72 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:19:24.946728 17739 raft_consensus.cc:493] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Starting leader election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:19:24.963577 17739 raft_consensus.cc:3060] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:19:24.964622 17695 raft_consensus.cc:515] T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:24.969723 17671 raft_consensus.cc:3060] T af57996a8be84d398f0f023e8d08ef77 P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:19:24.945945 17759 leader_election.cc:290] T e5ad4eb281564936a59f41e387a47156 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:19:24.968713 17722 raft_consensus.cc:515] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:24.981909 17722 leader_election.cc:290] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 3 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:24.982183 17726 raft_consensus.cc:515] T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:19:25.008280 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 162177us
I20260709 13:19:25.010792 16191 leader_election.cc:304] T b6f07842d3aa47858df5665874c86f3a P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
I20260709 13:19:24.869891 16116 leader_election.cc:304] T b6f07842d3aa47858df5665874c86f3a P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
I20260709 13:19:25.012768 16191 leader_election.cc:304] T 19baa7bc367844c288bca8f81439c413 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
W20260709 13:19:25.013244 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 143316us
I20260709 13:19:25.013724 16145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee9ed67006144797b1de6ea8c2013a68" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:19:25.016228 16147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87147731a68400e8cce9b879daa7c5d" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:19:24.935567 16294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:25.039494 16295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c2095a5f4ad49408dbc7f69c606120d" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:25.016724 16191 leader_election.cc:304] T 68a6838d82ef4cac83507f906139d90f P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
I20260709 13:19:24.863440 16146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66f75124215f4ca6bef831e174a792d8" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:19:25.019584 16218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a959dbecc2648eb85d1cc550f5a3245" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:24.884895 16219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fffab26427e24d398e2695aad40ca134" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:25.055953 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18ad7e3af5504571a69d10c00750a01c" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:25.060493 16226 raft_consensus.cc:2393] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 3: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
I20260709 13:19:25.063398 17672 raft_consensus.cc:2804] T 19baa7bc367844c288bca8f81439c413 P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:19:25.064138 17672 raft_consensus.cc:493] T 19baa7bc367844c288bca8f81439c413 P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Starting leader election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:19:25.064800 17672 raft_consensus.cc:3060] T 19baa7bc367844c288bca8f81439c413 P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:19:25.034277 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c61a39fd4f84424b129c1a6f9ba404d" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:19:25.071060 16292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8158bbd17b994d2eb96f560bd0f1b88d" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:25.071346 17717 raft_consensus.cc:2804] T b6f07842d3aa47858df5665874c86f3a P 647a343a542d4a139fe106c19095129b [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:19:25.080314 17722 raft_consensus.cc:2804] T b6f07842d3aa47858df5665874c86f3a P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:19:25.080966 17722 raft_consensus.cc:493] T b6f07842d3aa47858df5665874c86f3a P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Starting leader election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:19:25.081461 17722 raft_consensus.cc:3060] T b6f07842d3aa47858df5665874c86f3a P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:19:25.089459 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ad5dbfa99ee4b8d9e28be92b3f36db5" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:25.034814 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdbd78acb4cf46298e0de8ba0d130e19" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:25.093652 16293 raft_consensus.cc:2468] T cdbd78acb4cf46298e0de8ba0d130e19 P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 2.
I20260709 13:19:25.094089 16296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4f65cdf00b748bb963ee6e81ef43b79" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:25.094938 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806d7690c2b644959152074bb2cab0a7" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:25.095896 16293 raft_consensus.cc:2468] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 1.
I20260709 13:19:25.092199 16147 raft_consensus.cc:2393] T b87147731a68400e8cce9b879daa7c5d P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 in current term 2: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
I20260709 13:19:25.097014 16296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6f07842d3aa47858df5665874c86f3a" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:25.103271 16822 raft_consensus.cc:1240] T 66f75124215f4ca6bef831e174a792d8 P cb82cc7bd6db46d980c75237a1402860 [term 2 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 1. Current term is 2. Ops: []
I20260709 13:19:25.098438 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a96373bb0e14dce8a266b2fc6c58507" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:25.099496 16295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f2da98864b4425bf7c0055951fdd4b" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:25.092692 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cc959fb7f704fba9009c0595467d074" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:25.104512 16145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434bff3082d141838068f323bdb0d1ed" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:19:25.073961 17717 raft_consensus.cc:493] T b6f07842d3aa47858df5665874c86f3a P 647a343a542d4a139fe106c19095129b [term 1 FOLLOWER]: Starting leader election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:19:25.107463 17717 raft_consensus.cc:3060] T b6f07842d3aa47858df5665874c86f3a P 647a343a542d4a139fe106c19095129b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:19:25.107604 16145 raft_consensus.cc:3060] T 434bff3082d141838068f323bdb0d1ed P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:19:25.099015 16146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "829b4ad8c5a54eababeea1181b960d4b" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
W20260709 13:19:25.105049 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 88649.9us
I20260709 13:19:25.118405 17762 raft_consensus.cc:2804] T 68a6838d82ef4cac83507f906139d90f P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:19:25.118166 17673 raft_consensus.cc:515] T 02838c86531d4f98bf58d1c22b8baf77 P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:25.119095 17762 raft_consensus.cc:493] T 68a6838d82ef4cac83507f906139d90f P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Starting leader election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:19:25.119740 17762 raft_consensus.cc:3060] T 68a6838d82ef4cac83507f906139d90f P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:19:25.083479 16291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f5f89186204242bf377f7c42b2e2da" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:25.097036 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f5f89186204242bf377f7c42b2e2da" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:25.139261 16295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d16dfad0c3b540e5b4b86492ddf87208" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:25.133940 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7ca61592e848ea9b0fb81a2ce33cdc" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:25.109757 16292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02838c86531d4f98bf58d1c22b8baf77" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:25.141350 16297 raft_consensus.cc:2468] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 2.
I20260709 13:19:25.142977 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1df93779209748b9860f6d893dd3de53" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:25.144572 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d4ab870a05347868956aa8b841b2981" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
W20260709 13:19:25.147409 16115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 256602us
I20260709 13:19:25.149729 16150 raft_consensus.cc:2468] T af57996a8be84d398f0f023e8d08ef77 P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 2.
I20260709 13:19:25.143031 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8638ee363273426db1393dd81e5c7b72" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
W20260709 13:19:25.153175 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 99690.5us
I20260709 13:19:25.162792 16296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9944d5e4b8c4109af9955c4f08c9095" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:25.163841 16295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8158bbd17b994d2eb96f560bd0f1b88d" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:25.163949 16299 raft_consensus.cc:2468] T 8cc959fb7f704fba9009c0595467d074 P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 2.
I20260709 13:19:25.171931 16221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68a6838d82ef4cac83507f906139d90f" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:25.163357 17673 leader_election.cc:290] T 02838c86531d4f98bf58d1c22b8baf77 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 3 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:25.178617 16217 raft_consensus.cc:2393] T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 2: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
I20260709 13:19:25.182574 16291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:25.191551 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c61a39fd4f84424b129c1a6f9ba404d" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:25.197153 16295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4f65cdf00b748bb963ee6e81ef43b79" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:25.188897 16219 raft_consensus.cc:1240] T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 1. Current term is 2. Ops: []
I20260709 13:19:25.199749 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a878999c0024d2abe517484180290fc" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:25.207209 17693 consensus_queue.cc:1059] T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.031s
I20260709 13:19:25.210336 16145 raft_consensus.cc:2468] T 434bff3082d141838068f323bdb0d1ed P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 3.
I20260709 13:19:25.200268 16293 raft_consensus.cc:2468] T 8638ee363273426db1393dd81e5c7b72 P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 2.
I20260709 13:19:25.219756 16291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a378d7f796f94615b3b347119a9880fd" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:25.224671 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6f07842d3aa47858df5665874c86f3a" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:25.229962 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c61a39fd4f84424b129c1a6f9ba404d" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:25.232293 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68a6838d82ef4cac83507f906139d90f" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:25.233711 16295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a378d7f796f94615b3b347119a9880fd" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:19:25.221979 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fffab26427e24d398e2695aad40ca134" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:25.238657 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66ae176961b74813a58bc15c9c4c6f2a" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
W20260709 13:19:25.243978 17719 consensus_meta.cc:287] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac: Time spent flushing consensus metadata: real 0.548s	user 0.013s	sys 0.001s
I20260709 13:19:25.244622 17719 raft_consensus.cc:515] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:25.246232 16225 raft_consensus.cc:2393] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 3: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
I20260709 13:19:25.249138 16220 raft_consensus.cc:1240] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 2. Current term is 3. Ops: []
I20260709 13:19:25.252827 16146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8158bbd17b994d2eb96f560bd0f1b88d" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:19:25.253659 16146 raft_consensus.cc:3060] T 8158bbd17b994d2eb96f560bd0f1b88d P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:19:25.258194 16150 raft_consensus.cc:1240] T af57996a8be84d398f0f023e8d08ef77 P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 1. Current term is 2. Ops: []
I20260709 13:19:25.263938 16294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c11074579b046d99e585c2818e18a2c" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:25.264864 16294 raft_consensus.cc:2468] T 8c11074579b046d99e585c2818e18a2c P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 2.
I20260709 13:19:25.269465 16147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d16dfad0c3b540e5b4b86492ddf87208" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:19:25.276988 16825 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49896 (request call id 644) took 1490 ms. Trace:
I20260709 13:19:25.277514 16825 rpcz_store.cc:276] 0709 13:19:23.786031 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:23.786172 (+   141us) service_pool.cc:224] Handling call
0709 13:19:25.276446 (+1490274us) spinlock_profiling.cc:243] Waited 1.1 s on lock 0x7b54007553d4. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:19:25.276744 (+   298us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2523270912}
I20260709 13:19:25.280251 16145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f5f89186204242bf377f7c42b2e2da" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:19:25.278776 17738 leader_election.cc:290] T 829b4ad8c5a54eababeea1181b960d4b P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 3 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:19:25.255847 17695 leader_election.cc:290] T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 2 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:25.281313 16151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02838c86531d4f98bf58d1c22b8baf77" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:19:25.282122 16144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1df93779209748b9860f6d893dd3de53" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:19:25.282174 16220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d16dfad0c3b540e5b4b86492ddf87208" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:25.284209 16147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2794af5b99984efebdee564035841cb0" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:19:25.285605 16143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a9de347c711477e87c582d003f36925" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:19:25.288350 16148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0eab25ce4c64e018a041beeda2867db" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:19:25.288738 16144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3884f864a96d428785895e5463e5728f" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:19:25.289323 16143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ad5dbfa99ee4b8d9e28be92b3f36db5" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:19:25.293803 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c2095a5f4ad49408dbc7f69c606120d" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:19:25.295348 16220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1df93779209748b9860f6d893dd3de53" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:25.295925 17518 leader_election.cc:290] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:19:25.296125 16294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0eab25ce4c64e018a041beeda2867db" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:25.296949 16220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e4bfb9090c8460eb14fa7e0d818e7f1" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:25.297633 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a959dbecc2648eb85d1cc550f5a3245" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:25.298053 16222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0eab25ce4c64e018a041beeda2867db" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:25.298245 16220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ab0ac8347641e6955e9bce7d90c1a2" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:25.298635 17719 leader_election.cc:290] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 3 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:25.299214 17726 leader_election.cc:290] T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 2 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:25.299453 16222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66ae176961b74813a58bc15c9c4c6f2a" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:25.300684 17644 raft_consensus.cc:515] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:25.302215 16142 raft_consensus.cc:1240] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 2. Current term is 3. Ops: []
I20260709 13:19:25.304309 17645 leader_election.cc:290] T 9e65676b21b748728ad39668cfe722ec P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 3 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:25.297719 16143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c61a39fd4f84424b129c1a6f9ba404d" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:19:25.306034 16146 raft_consensus.cc:2468] T 8158bbd17b994d2eb96f560bd0f1b88d P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 3.
I20260709 13:19:25.306972 17645 raft_consensus.cc:493] T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:25.315490 17645 raft_consensus.cc:515] T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:25.318027 17645 leader_election.cc:290] T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:19:25.326954 16143 raft_consensus.cc:2393] T 9c61a39fd4f84424b129c1a6f9ba404d P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 2: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
W20260709 13:19:25.330432 16223 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33050 (request call id 252) took 1783 ms (client timeout 1581 ms). Trace:
W20260709 13:19:25.332175 16223 rpcz_store.cc:269] 0709 13:19:23.547157 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:23.547274 (+   117us) service_pool.cc:224] Handling call
0709 13:19:23.820299 (+273025us) spinlock_profiling.cc:243] Waited 93.6 ms on lock 0x7b5800010b98. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69ad8fadb1 00007f69ada3ab71 00007f69ada3aa22 00007f69b6953c09 00007f69b5aadee0 00007f69b5b770ab 00007f69b5b77131 00007f69b5b250a0 00007f69b5b24984 00007f69af09f6d4 00007f69af09f662
0709 13:19:25.329947 (+1509648us) spinlock_profiling.cc:243] Waited 879 ms on lock 0x7b5400368654. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:19:25.330195 (+   248us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2237310464}
W20260709 13:19:25.332767 17724 consensus_meta.cc:287] T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac: Time spent flushing consensus metadata: real 0.740s	user 0.007s	sys 0.009s
I20260709 13:19:25.332041 16222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ad5dbfa99ee4b8d9e28be92b3f36db5" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:25.333349 17724 raft_consensus.cc:515] T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:25.334337 17729 raft_consensus.cc:3055] T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:19:25.334786 17729 raft_consensus.cc:740] T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:19:25.331354 16299 raft_consensus.cc:2468] T 66ae176961b74813a58bc15c9c4c6f2a P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 2.
I20260709 13:19:25.337869 17718 raft_consensus.cc:493] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:19:25.341697 16824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a378d7f796f94615b3b347119a9880fd" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:19:25.340823 16220 raft_consensus.cc:1240] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 1. Current term is 2. Ops: []
I20260709 13:19:25.343784 17685 raft_consensus.cc:515] T 8d4ab870a05347868956aa8b841b2981 P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:25.348214 16223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee9ed67006144797b1de6ea8c2013a68" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:19:25.348678 16219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62fe3346cd3c42c4a55e14af9073bca4" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:19:25.366171 17709 leader_election.cc:290] T 434bff3082d141838068f323bdb0d1ed P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 3 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:25.349555 17729 consensus_queue.cc:260] T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:25.349931 16825 raft_consensus.cc:1240] T 829b4ad8c5a54eababeea1181b960d4b P cb82cc7bd6db46d980c75237a1402860 [term 3 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 2. Current term is 3. Ops: []
I20260709 13:19:25.350546 16142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4f65cdf00b748bb963ee6e81ef43b79" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:19:25.354518 17693 consensus_queue.cc:1059] T ee9ed67006144797b1de6ea8c2013a68 P 647a343a542d4a139fe106c19095129b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:19:25.351984 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada5a004ba7c46819051e3ccec71ffdb" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:25.374240 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af57996a8be84d398f0f023e8d08ef77" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:19:25.375805 16824 raft_consensus.cc:2393] T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 2: Already voted for candidate cb82cc7bd6db46d980c75237a1402860 in this term.
W20260709 13:19:25.376067 16192 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:25.305111 17644 leader_election.cc:290] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 3 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:19:25.357788 16145 raft_consensus.cc:3060] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:19:25.358016 16820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9944d5e4b8c4109af9955c4f08c9095" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:19:25.359647 16226 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57604 (request call id 1781) took 1010 ms. Trace:
I20260709 13:19:25.329891 17739 raft_consensus.cc:515] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } }
I20260709 13:19:25.365365 16147 raft_consensus.cc:1240] T 9c61a39fd4f84424b129c1a6f9ba404d P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 1. Current term is 2. Ops: []
I20260709 13:19:25.368760 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:25.371824 16142 raft_consensus.cc:3060] T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:19:25.372011 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31969392ccdb43b491911a319e8bc7e1" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:19:25.385725 16146 raft_consensus.cc:1240] T 434bff3082d141838068f323bdb0d1ed P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 2. Current term is 3. Ops: []
I20260709 13:19:25.377730 16222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66f75124215f4ca6bef831e174a792d8" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:19:25.380910 16226 rpcz_store.cc:276] 0709 13:19:24.349105 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:24.360627 (+ 11522us) service_pool.cc:224] Handling call
0709 13:19:24.423830 (+ 63203us) spinlock_profiling.cc:243] Waited 2.14 ms on lock 0x7b5800010b98. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69ad8fadb1 00007f69ada3ab71 00007f69ada3aa22 00007f69b6953c09 00007f69b5aadee0 00007f69b5b770ab 00007f69b5b77131 00007f69b5b250a0 00007f69b5b24984 00007f69af09f6d4 00007f69af09f662
0709 13:19:25.359377 (+935547us) spinlock_profiling.cc:243] Waited 558 ms on lock 0x7b54002c9554. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:19:25.359583 (+   206us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1287357568}
I20260709 13:19:25.385892 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f2da98864b4425bf7c0055951fdd4b" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:19:25.389335 16820 raft_consensus.cc:3060] T c9944d5e4b8c4109af9955c4f08c9095 P cb82cc7bd6db46d980c75237a1402860 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:19:25.390147 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f2da98864b4425bf7c0055951fdd4b" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:25.393287 16824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57a744f5aaf944678a63469cbf514015" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:19:25.398905 17724 leader_election.cc:290] T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 2 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:25.400688 17739 leader_election.cc:290] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 3 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:25.376786 17693 raft_consensus.cc:3055] T ee9ed67006144797b1de6ea8c2013a68 P 647a343a542d4a139fe106c19095129b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:19:25.401836 17693 raft_consensus.cc:740] T ee9ed67006144797b1de6ea8c2013a68 P 647a343a542d4a139fe106c19095129b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:19:25.403541 16223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c11074579b046d99e585c2818e18a2c" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:25.403878 17717 raft_consensus.cc:515] T b6f07842d3aa47858df5665874c86f3a P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:25.405058 16223 raft_consensus.cc:1240] T c2f5f89186204242bf377f7c42b2e2da P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 2. Current term is 3. Ops: []
I20260709 13:19:25.405129 16151 raft_consensus.cc:1240] T b6f07842d3aa47858df5665874c86f3a P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 1. Current term is 2. Ops: []
I20260709 13:19:25.406303 17717 leader_election.cc:290] T b6f07842d3aa47858df5665874c86f3a P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 2 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:25.379588 17762 raft_consensus.cc:515] T 68a6838d82ef4cac83507f906139d90f P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:25.412281 16221 raft_consensus.cc:2393] T 68a6838d82ef4cac83507f906139d90f P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 3: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
I20260709 13:19:25.352274 16044 catalog_manager.cc:5697] T 19baa7bc367844c288bca8f81439c413 P 1ca2725d290e47d68b1a4b609c9816ac reported cstate change: config changed from index -1 to 3, NON_VOTER cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) added. New cstate: current_term: 2 leader_uuid: "90f3da57796b488c9087a2f523adf045" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } } }
I20260709 13:19:25.406387 16223 raft_consensus.cc:1240] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 2. Current term is 3. Ops: []
I20260709 13:19:25.416838 16221 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57604 (request call id 1772) took 1433 ms. Trace:
I20260709 13:19:25.417145 16224 raft_consensus.cc:1240] T 68a6838d82ef4cac83507f906139d90f P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 2. Current term is 3. Ops: []
I20260709 13:19:25.417588 16825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2794af5b99984efebdee564035841cb0" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:19:25.423024 16820 raft_consensus.cc:2468] T c9944d5e4b8c4109af9955c4f08c9095 P cb82cc7bd6db46d980c75237a1402860 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 3.
I20260709 13:19:25.425832 16825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada5a004ba7c46819051e3ccec71ffdb" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:19:25.417320 16221 rpcz_store.cc:276] 0709 13:19:23.983402 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:23.984863 (+  1461us) service_pool.cc:224] Handling call
0709 13:19:24.197847 (+212984us) spinlock_profiling.cc:243] Waited 7.48 ms on lock 0x7b5800010b98. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69ad8fadb1 00007f69ada3ab71 00007f69ada3aa22 00007f69b6953c09 00007f69b5aadee0 00007f69b5b770ab 00007f69b5b77131 00007f69b5b250a0 00007f69b5b24984 00007f69af09f6d4 00007f69af09f662
0709 13:19:25.171577 (+973730us) spinlock_profiling.cc:243] Waited 19.6 ms on lock 0x7b5800010b98. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69ad8fadb1 00007f69ada3ae9c 00007f69ada3aa5b 00007f69b6953c09 00007f69b5aadee0 00007f69b5b770ab 00007f69b5b77131 00007f69b5b250a0 00007f69b5b24984 00007f69af09f6d4 00007f69af09f662
0709 13:19:25.416540 (+244963us) spinlock_profiling.cc:243] Waited 239 ms on lock 0x7b5400297cd4. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:19:25.416762 (+   222us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":612646272}
I20260709 13:19:25.426748 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:19:25.426465 16191 leader_election.cc:304] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
I20260709 13:19:25.435658 16223 raft_consensus.cc:1240] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 2. Current term is 3. Ops: []
I20260709 13:19:25.420143 17762 leader_election.cc:290] T 68a6838d82ef4cac83507f906139d90f P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 3 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:25.447466 16151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c74490c40243a88e62d1760b5cd814" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:19:25.447788 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87147731a68400e8cce9b879daa7c5d" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:19:25.448851 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cc959fb7f704fba9009c0595467d074" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:19:25.450053 16144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f82224f4564fea93572e8206bad1fe" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:19:25.450163 16147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4dc6b0541ca431ebc757beac00521f1" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:19:25.425199 16219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a378d7f796f94615b3b347119a9880fd" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:19:25.459585 16825 raft_consensus.cc:1240] T c9944d5e4b8c4109af9955c4f08c9095 P cb82cc7bd6db46d980c75237a1402860 [term 3 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 2. Current term is 3. Ops: []
I20260709 13:19:25.464237 16147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26354361e9b044cea1c2b86c4882fe92" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:19:25.444939 16044 catalog_manager.cc:5697] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac reported cstate change: term changed from 1 to 2, leader changed from 647a343a542d4a139fe106c19095129b (127.14.177.129) to 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130). New cstate: current_term: 2 leader_uuid: "1ca2725d290e47d68b1a4b609c9816ac" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:19:25.402997 17693 consensus_queue.cc:260] T ee9ed67006144797b1de6ea8c2013a68 P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:25.468405 16144 raft_consensus.cc:1240] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 2. Current term is 3. Ops: []
I20260709 13:19:25.423810 17671 consensus_queue.cc:1059] T 829b4ad8c5a54eababeea1181b960d4b P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.040s
I20260709 13:19:25.471465 16820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ab0ac8347641e6955e9bce7d90c1a2" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:19:25.433730 17722 raft_consensus.cc:515] T b6f07842d3aa47858df5665874c86f3a P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:25.424978 17718 raft_consensus.cc:515] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:25.463393 17762 raft_consensus.cc:2804] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:19:25.475354 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c61a39fd4f84424b129c1a6f9ba404d" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:19:25.478670 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e65676b21b748728ad39668cfe722ec" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:19:25.479306 16150 raft_consensus.cc:3060] T 9e65676b21b748728ad39668cfe722ec P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:19:25.480543 16818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66ae176961b74813a58bc15c9c4c6f2a" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:19:25.481161 16817 raft_consensus.cc:1240] T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860 [term 2 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 1. Current term is 2. Ops: []
I20260709 13:19:25.481962 16818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31969392ccdb43b491911a319e8bc7e1" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:19:25.481899 16825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3884f864a96d428785895e5463e5728f" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:19:25.482926 16823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e4bfb9090c8460eb14fa7e0d818e7f1" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:19:25.484604 16816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c11074579b046d99e585c2818e18a2c" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:19:25.486801 16150 raft_consensus.cc:2468] T 9e65676b21b748728ad39668cfe722ec P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 3.
I20260709 13:19:25.495488 16820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a9de347c711477e87c582d003f36925" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:19:25.495882 17762 raft_consensus.cc:697] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac [term 3 LEADER]: Becoming Leader. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:19:25.496904 17762 consensus_queue.cc:237] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:25.500788 16145 raft_consensus.cc:2468] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 3.
I20260709 13:19:25.503912 17673 raft_consensus.cc:3055] T 829b4ad8c5a54eababeea1181b960d4b P 1ca2725d290e47d68b1a4b609c9816ac [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:19:25.504020 16192 leader_election.cc:304] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:19:25.504423 17673 raft_consensus.cc:740] T 829b4ad8c5a54eababeea1181b960d4b P 1ca2725d290e47d68b1a4b609c9816ac [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:19:25.474886 17672 raft_consensus.cc:515] T 19baa7bc367844c288bca8f81439c413 P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } }
I20260709 13:19:25.505384 17673 consensus_queue.cc:260] T 829b4ad8c5a54eababeea1181b960d4b P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:25.506072 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c74490c40243a88e62d1760b5cd814" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:25.506407 17672 leader_election.cc:290] T 19baa7bc367844c288bca8f81439c413 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 3 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:25.506561 17673 raft_consensus.cc:3060] T 829b4ad8c5a54eababeea1181b960d4b P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:19:25.507368 16218 raft_consensus.cc:1240] T 19baa7bc367844c288bca8f81439c413 P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 2. Current term is 3. Ops: []
I20260709 13:19:25.507483 17671 raft_consensus.cc:2804] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:19:25.508683 16218 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57600 (request call id 896) took 1161 ms. Trace:
I20260709 13:19:25.509239 17671 raft_consensus.cc:697] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac [term 3 LEADER]: Becoming Leader. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:19:25.510342 17671 consensus_queue.cc:237] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:25.513953 16151 raft_consensus.cc:1240] T 8158bbd17b994d2eb96f560bd0f1b88d P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 2. Current term is 3. Ops: []
I20260709 13:19:25.511483 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4dc6b0541ca431ebc757beac00521f1" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:25.520110 16148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:19:25.474439 16220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6f07842d3aa47858df5665874c86f3a" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:19:25.490757 16219 raft_consensus.cc:2393] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 in current term 2: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
I20260709 13:19:25.526777 16146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:19:25.534168 16146 raft_consensus.cc:2393] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 3: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
I20260709 13:19:25.527904 16148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68a6838d82ef4cac83507f906139d90f" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:19:25.530041 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87147731a68400e8cce9b879daa7c5d" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:25.504122 16217 raft_consensus.cc:2468] T af57996a8be84d398f0f023e8d08ef77 P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 2.
I20260709 13:19:25.532531 16223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d4ab870a05347868956aa8b841b2981" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:19:25.532652 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c74490c40243a88e62d1760b5cd814" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:25.488530 17729 raft_consensus.cc:3060] T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:19:25.490141 16819 raft_consensus.cc:1275] T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860 [term 2 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:19:25.538554 16225 raft_consensus.cc:1240] T b6f07842d3aa47858df5665874c86f3a P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 1. Current term is 2. Ops: []
W20260709 13:19:25.540350 17730 consensus_meta.cc:287] T 74ca8140f35c4ae29e8a664f6bdfc829 P cb82cc7bd6db46d980c75237a1402860: Time spent flushing consensus metadata: real 0.961s	user 0.016s	sys 0.005s
I20260709 13:19:25.541128 16822 raft_consensus.cc:1240] T b87147731a68400e8cce9b879daa7c5d P cb82cc7bd6db46d980c75237a1402860 [term 2 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 1. Current term is 2. Ops: []
I20260709 13:19:25.542336 16223 raft_consensus.cc:2393] T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 2: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
I20260709 13:19:25.542955 16144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18ad7e3af5504571a69d10c00750a01c" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:19:25.543365 16191 leader_election.cc:304] T c2f5f89186204242bf377f7c42b2e2da P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
I20260709 13:19:25.549855 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d4ab870a05347868956aa8b841b2981" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:19:25.545171 17730 raft_consensus.cc:515] T 74ca8140f35c4ae29e8a664f6bdfc829 P cb82cc7bd6db46d980c75237a1402860 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:25.552896 17730 leader_election.cc:290] T 74ca8140f35c4ae29e8a664f6bdfc829 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 2 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:25.553699 16148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5d59407eb545b895d8d922e0dd69fd" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:19:25.554437 16148 raft_consensus.cc:2468] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 2.
I20260709 13:19:25.555289 16824 raft_consensus.cc:1240] T 74ca8140f35c4ae29e8a664f6bdfc829 P cb82cc7bd6db46d980c75237a1402860 [term 2 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 1. Current term is 2. Ops: []
I20260709 13:19:25.555985 16222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c2095a5f4ad49408dbc7f69c606120d" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:25.563393 16221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cc959fb7f704fba9009c0595467d074" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:25.509027 16218 rpcz_store.cc:276] 0709 13:19:24.346981 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:25.071444 (+724463us) service_pool.cc:224] Handling call
0709 13:19:25.072006 (+   562us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:25.507802 (+435796us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:25.508451 (+   649us) spinlock_profiling.cc:243] Waited 435 ms on lock 0x7b5400246b54. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:19:25.508607 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1000790400}
I20260709 13:19:25.584141 16144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5ad4eb281564936a59f41e387a47156" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
W20260709 13:19:25.584498 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 68187us
I20260709 13:19:25.588857 17638 raft_consensus.cc:2804] T c2f5f89186204242bf377f7c42b2e2da P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:19:25.589609 17638 raft_consensus.cc:697] T c2f5f89186204242bf377f7c42b2e2da P 1ca2725d290e47d68b1a4b609c9816ac [term 3 LEADER]: Becoming Leader. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:19:25.592051 17718 leader_election.cc:290] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:19:25.546613 16143 raft_consensus.cc:1240] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 2. Current term is 3. Ops: []
I20260709 13:19:25.590814 17638 consensus_queue.cc:237] T c2f5f89186204242bf377f7c42b2e2da P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:25.595820 16148 raft_consensus.cc:1275] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:19:25.598132 16824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62fe3346cd3c42c4a55e14af9073bca4" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:19:25.602907 17722 leader_election.cc:290] T b6f07842d3aa47858df5665874c86f3a P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 2 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:25.605044 17693 raft_consensus.cc:3060] T ee9ed67006144797b1de6ea8c2013a68 P 647a343a542d4a139fe106c19095129b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:19:25.605650 16822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26354361e9b044cea1c2b86c4882fe92" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:19:25.607354 16822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f82224f4564fea93572e8206bad1fe" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:19:25.612814 16824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8638ee363273426db1393dd81e5c7b72" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:19:25.618625 16819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2372981006d4b218215c28964dea623" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:19:25.620113 17722 consensus_queue.cc:1048] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:19:25.621986 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:19:25.585435 17685 leader_election.cc:290] T 8d4ab870a05347868956aa8b841b2981 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 2 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:25.629463 16222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "829b4ad8c5a54eababeea1181b960d4b" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:19:25.629482 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5d59407eb545b895d8d922e0dd69fd" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:25.628062 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5ad4eb281564936a59f41e387a47156" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:25.632313 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18ad7e3af5504571a69d10c00750a01c" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:25.631436 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6f07842d3aa47858df5665874c86f3a" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:19:25.634999 16822 raft_consensus.cc:1240] T 434bff3082d141838068f323bdb0d1ed P cb82cc7bd6db46d980c75237a1402860 [term 3 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 2. Current term is 3. Ops: []
I20260709 13:19:25.633178 16223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9944d5e4b8c4109af9955c4f08c9095" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:25.620666 16221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8638ee363273426db1393dd81e5c7b72" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:25.637329 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68a6838d82ef4cac83507f906139d90f" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:25.642274 16117 leader_election.cc:304] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860
I20260709 13:19:25.643397 17644 raft_consensus.cc:2749] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:19:25.648316 16220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:25.649433 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434bff3082d141838068f323bdb0d1ed" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:25.650080 16218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7ca61592e848ea9b0fb81a2ce33cdc" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
W20260709 13:19:25.650250 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 51350.2us
I20260709 13:19:25.651302 16790 leader_election.cc:304] T af57996a8be84d398f0f023e8d08ef77 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:19:25.636128 16217 raft_consensus.cc:1240] T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 1. Current term is 2. Ops: []
I20260709 13:19:25.652180 17695 raft_consensus.cc:2804] T af57996a8be84d398f0f023e8d08ef77 P cb82cc7bd6db46d980c75237a1402860 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:19:25.652689 17695 raft_consensus.cc:697] T af57996a8be84d398f0f023e8d08ef77 P cb82cc7bd6db46d980c75237a1402860 [term 2 LEADER]: Becoming Leader. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
W20260709 13:19:25.653915 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 44847.7us
I20260709 13:19:25.654548 16790 leader_election.cc:304] T 434bff3082d141838068f323bdb0d1ed P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:19:25.656306 16822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7ca61592e848ea9b0fb81a2ce33cdc" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:19:25.658003 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdbd78acb4cf46298e0de8ba0d130e19" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:25.658561 16299 raft_consensus.cc:2468] T cdbd78acb4cf46298e0de8ba0d130e19 P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 2.
I20260709 13:19:25.660013 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cc959fb7f704fba9009c0595467d074" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:25.661325 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9944d5e4b8c4109af9955c4f08c9095" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:19:25.662177 16144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2372981006d4b218215c28964dea623" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:19:25.663080 16294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5ad4eb281564936a59f41e387a47156" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:25.663406 16296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8158bbd17b994d2eb96f560bd0f1b88d" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:19:25.664330 16292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d4ab870a05347868956aa8b841b2981" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:19:25.663117 16143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6f07842d3aa47858df5665874c86f3a" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:19:25.666476 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f5f89186204242bf377f7c42b2e2da" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:19:25.666080 16143 raft_consensus.cc:2393] T b6f07842d3aa47858df5665874c86f3a P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 2: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
I20260709 13:19:25.668565 16296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74ca8140f35c4ae29e8a664f6bdfc829" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:25.669811 16292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02838c86531d4f98bf58d1c22b8baf77" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:19:25.671052 16294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c61a39fd4f84424b129c1a6f9ba404d" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:19:25.672767 17709 raft_consensus.cc:2804] T 434bff3082d141838068f323bdb0d1ed P cb82cc7bd6db46d980c75237a1402860 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:19:25.673327 17709 raft_consensus.cc:697] T 434bff3082d141838068f323bdb0d1ed P cb82cc7bd6db46d980c75237a1402860 [term 3 LEADER]: Becoming Leader. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:19:25.674376 17709 consensus_queue.cc:237] T 434bff3082d141838068f323bdb0d1ed P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:25.676456 17681 tablet_bootstrap.cc:492] T 806d7690c2b644959152074bb2cab0a7 P cb82cc7bd6db46d980c75237a1402860: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:19:25.676944 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c61a39fd4f84424b129c1a6f9ba404d" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:25.677479 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c2095a5f4ad49408dbc7f69c606120d" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:25.677696 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4f65cdf00b748bb963ee6e81ef43b79" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:19:25.679518 16296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6f07842d3aa47858df5665874c86f3a" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:25.680438 16292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8158bbd17b994d2eb96f560bd0f1b88d" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:25.680691 17703 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 19baa7bc367844c288bca8f81439c413. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:19:25.681558 16294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f2da98864b4425bf7c0055951fdd4b" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:25.692459 16295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4dc6b0541ca431ebc757beac00521f1" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:25.682839 16291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d4ab870a05347868956aa8b841b2981" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:19:25.653659 17695 consensus_queue.cc:237] T af57996a8be84d398f0f023e8d08ef77 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:25.682976 17721 consensus_queue.cc:1059] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:19:25.683826 16790 leader_election.cc:304] T 9e65676b21b748728ad39668cfe722ec P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:19:25.687975 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada5a004ba7c46819051e3ccec71ffdb" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:25.696694 17730 raft_consensus.cc:3055] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:19:25.697229 17730 raft_consensus.cc:740] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:19:25.699543 17721 raft_consensus.cc:2804] T 9e65676b21b748728ad39668cfe722ec P cb82cc7bd6db46d980c75237a1402860 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:19:25.700157 17721 raft_consensus.cc:697] T 9e65676b21b748728ad39668cfe722ec P cb82cc7bd6db46d980c75237a1402860 [term 3 LEADER]: Becoming Leader. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:19:25.690618 17638 consensus_queue.cc:1048] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.011s
I20260709 13:19:25.701119 17721 consensus_queue.cc:237] T 9e65676b21b748728ad39668cfe722ec P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:25.704434 16790 leader_election.cc:304] T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:19:25.698516 17730 consensus_queue.cc:260] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:25.714581 16790 leader_election.cc:304] T ee9ed67006144797b1de6ea8c2013a68 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:19:25.719950 16219 raft_consensus.cc:1240] T 02838c86531d4f98bf58d1c22b8baf77 P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 2. Current term is 3. Ops: []
I20260709 13:19:25.720099 17645 raft_consensus.cc:2749] T ee9ed67006144797b1de6ea8c2013a68 P cb82cc7bd6db46d980c75237a1402860 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:19:25.729430 16298 raft_consensus.cc:2468] T ada5a004ba7c46819051e3ccec71ffdb P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 2.
I20260709 13:19:25.719954 16790 leader_election.cc:304] T 66f75124215f4ca6bef831e174a792d8 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:19:25.731220 16191 leader_election.cc:304] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
I20260709 13:19:25.721539 17730 raft_consensus.cc:3060] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:19:25.733779 17738 raft_consensus.cc:2804] T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:19:25.746001 17710 raft_consensus.cc:2749] T 66f75124215f4ca6bef831e174a792d8 P cb82cc7bd6db46d980c75237a1402860 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:19:25.759836 16219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74ca8140f35c4ae29e8a664f6bdfc829" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:19:25.738534 17671 raft_consensus.cc:2749] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:19:25.747035 17738 raft_consensus.cc:493] T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:19:25.765340 17738 raft_consensus.cc:3060] T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:19:25.797808 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 77942.7us
I20260709 13:19:25.764341 16142 raft_consensus.cc:2468] T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 3.
I20260709 13:19:25.823342 17681 tablet_bootstrap.cc:492] T 806d7690c2b644959152074bb2cab0a7 P cb82cc7bd6db46d980c75237a1402860: Bootstrap complete.
I20260709 13:19:25.837807 17681 ts_tablet_manager.cc:1403] T 806d7690c2b644959152074bb2cab0a7 P cb82cc7bd6db46d980c75237a1402860: Time spent bootstrapping tablet: real 0.972s	user 0.022s	sys 0.018s
I20260709 13:19:25.836118 16149 raft_consensus.cc:1240] T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 2. Current term is 3. Ops: []
W20260709 13:19:25.838261 16117 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 62007.2us
I20260709 13:19:25.854504 16222 raft_consensus.cc:2468] T 829b4ad8c5a54eababeea1181b960d4b P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 3.
W20260709 13:19:25.860077 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 108077us
I20260709 13:19:25.874171 17704 tablet_copy_client.cc:538] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:19:25.875633 17518 consensus_queue.cc:1059] T a378d7f796f94615b3b347119a9880fd P 90f3da57796b488c9087a2f523adf045 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:19:25.874264 17781 consensus_queue.cc:1059] T c9944d5e4b8c4109af9955c4f08c9095 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.003s
I20260709 13:19:25.877651 17781 raft_consensus.cc:3055] T a378d7f796f94615b3b347119a9880fd P 90f3da57796b488c9087a2f523adf045 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:19:25.877758 17518 raft_consensus.cc:3055] T c9944d5e4b8c4109af9955c4f08c9095 P 90f3da57796b488c9087a2f523adf045 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:19:25.878026 17781 raft_consensus.cc:740] T a378d7f796f94615b3b347119a9880fd P 90f3da57796b488c9087a2f523adf045 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:19:25.878162 17518 raft_consensus.cc:740] T c9944d5e4b8c4109af9955c4f08c9095 P 90f3da57796b488c9087a2f523adf045 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:19:25.881458 16191 leader_election.cc:304] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
W20260709 13:19:25.883003 16266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 144087us
W20260709 13:19:25.891942 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 226153us
I20260709 13:19:25.864491 17738 raft_consensus.cc:515] T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:25.884465 16190 leader_election.cc:304] T d2ab0ac8347641e6955e9bce7d90c1a2 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: cb82cc7bd6db46d980c75237a1402860
I20260709 13:19:25.899281 17672 raft_consensus.cc:2804] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:19:25.899653 17718 raft_consensus.cc:2804] T d2ab0ac8347641e6955e9bce7d90c1a2 P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:19:25.899844 16190 leader_election.cc:304] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 647a343a542d4a139fe106c19095129b
I20260709 13:19:25.900131 17718 raft_consensus.cc:493] T d2ab0ac8347641e6955e9bce7d90c1a2 P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Starting leader election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:19:25.900535 17718 raft_consensus.cc:3060] T d2ab0ac8347641e6955e9bce7d90c1a2 P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:19:25.899864 17672 raft_consensus.cc:697] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac [term 3 LEADER]: Becoming Leader. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:19:25.908141 17672 consensus_queue.cc:237] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:25.909651 16819 raft_consensus.cc:1240] T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860 [term 3 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 2. Current term is 3. Ops: []
I20260709 13:19:25.904247 17722 raft_consensus.cc:2804] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:19:25.914884 17722 raft_consensus.cc:493] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Starting leader election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:19:25.915421 17722 raft_consensus.cc:3060] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:19:25.878969 17781 consensus_queue.cc:260] T a378d7f796f94615b3b347119a9880fd P 90f3da57796b488c9087a2f523adf045 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:25.921666 16265 leader_election.cc:304] T 57a744f5aaf944678a63469cbf514015 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
I20260709 13:19:25.924261 16790 leader_election.cc:304] T 829b4ad8c5a54eababeea1181b960d4b P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 647a343a542d4a139fe106c19095129b
I20260709 13:19:25.929697 17645 raft_consensus.cc:2804] T 829b4ad8c5a54eababeea1181b960d4b P cb82cc7bd6db46d980c75237a1402860 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:19:25.878996 17518 consensus_queue.cc:260] T c9944d5e4b8c4109af9955c4f08c9095 P 90f3da57796b488c9087a2f523adf045 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:25.949039 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26354361e9b044cea1c2b86c4882fe92" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:25.949185 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57a744f5aaf944678a63469cbf514015" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:25.952927 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2794af5b99984efebdee564035841cb0" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:25.953397 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3884f864a96d428785895e5463e5728f" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:25.954416 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e65676b21b748728ad39668cfe722ec" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:25.952986 16291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87147731a68400e8cce9b879daa7c5d" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:25.956000 17781 raft_consensus.cc:3060] T a378d7f796f94615b3b347119a9880fd P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:19:25.956874 16295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f82224f4564fea93572e8206bad1fe" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:25.957937 16294 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8158bbd17b994d2eb96f560bd0f1b88d"
dest_uuid: "90f3da57796b488c9087a2f523adf045"
mode: GRACEFUL
new_leader_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
 from {username='slave'} at 127.0.0.1:43334
I20260709 13:19:25.958674 16294 raft_consensus.cc:606] T 8158bbd17b994d2eb96f560bd0f1b88d P 90f3da57796b488c9087a2f523adf045 [term 2 LEADER]: Received request to transfer leadership to TS 1ca2725d290e47d68b1a4b609c9816ac
I20260709 13:19:25.936726 17644 consensus_queue.cc:1048] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.023s
I20260709 13:19:25.960044 16292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a9de347c711477e87c582d003f36925" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:25.966979 16190 leader_election.cc:304] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 647a343a542d4a139fe106c19095129b
I20260709 13:19:25.955600 17681 raft_consensus.cc:359] T 806d7690c2b644959152074bb2cab0a7 P cb82cc7bd6db46d980c75237a1402860 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } }
I20260709 13:19:25.968006 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74ca8140f35c4ae29e8a664f6bdfc829" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:25.969069 17681 raft_consensus.cc:740] T 806d7690c2b644959152074bb2cab0a7 P cb82cc7bd6db46d980c75237a1402860 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Initialized, Role: LEARNER
I20260709 13:19:25.948951 17779 raft_consensus.cc:2749] T 57a744f5aaf944678a63469cbf514015 P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:19:25.969786 17681 consensus_queue.cc:260] T 806d7690c2b644959152074bb2cab0a7 P cb82cc7bd6db46d980c75237a1402860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } }
I20260709 13:19:25.971026 17779 consensus_queue.cc:1059] T b87147731a68400e8cce9b879daa7c5d P 90f3da57796b488c9087a2f523adf045 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:19:25.972239 17681 ts_tablet_manager.cc:1434] T 806d7690c2b644959152074bb2cab0a7 P cb82cc7bd6db46d980c75237a1402860: Time spent starting tablet: real 0.134s	user 0.004s	sys 0.004s
I20260709 13:19:25.980625 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e9b2bf7011f4ae688a79d7bb5a447b4" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:25.982807 16296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a378d7f796f94615b3b347119a9880fd" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:25.984118 17760 raft_consensus.cc:2804] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:19:25.984551 17760 raft_consensus.cc:493] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Starting leader election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:19:25.984946 17760 raft_consensus.cc:3060] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:19:25.987563 17718 raft_consensus.cc:515] T d2ab0ac8347641e6955e9bce7d90c1a2 P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:25.990686 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434bff3082d141838068f323bdb0d1ed" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:25.948935 17645 raft_consensus.cc:697] T 829b4ad8c5a54eababeea1181b960d4b P cb82cc7bd6db46d980c75237a1402860 [term 3 LEADER]: Becoming Leader. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:19:25.995913 16265 leader_election.cc:304] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
I20260709 13:19:25.998010 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9944d5e4b8c4109af9955c4f08c9095" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:25.992177 17673 consensus_queue.cc:1059] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.013s
I20260709 13:19:26.005620 16161 tablet_copy_service.cc:342] P 647a343a542d4a139fe106c19095129b: Request end of tablet copy session cb82cc7bd6db46d980c75237a1402860-806d7690c2b644959152074bb2cab0a7 received from {username='slave'} at 127.0.0.1:46224
I20260709 13:19:26.006141 17673 raft_consensus.cc:3055] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:19:25.990984 17518 raft_consensus.cc:3060] T c9944d5e4b8c4109af9955c4f08c9095 P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:19:26.017063 17673 raft_consensus.cc:740] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:19:26.017401 16047 catalog_manager.cc:5697] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 647a343a542d4a139fe106c19095129b reported cstate change: term changed from 1 to 2, leader changed from 90f3da57796b488c9087a2f523adf045 (127.14.177.131) to 647a343a542d4a139fe106c19095129b (127.14.177.129). New cstate: current_term: 2 leader_uuid: "647a343a542d4a139fe106c19095129b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } health_report { overall_health: UNKNOWN } } }
I20260709 13:19:26.007963 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e4bfb9090c8460eb14fa7e0d818e7f1" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:26.008724 17738 leader_election.cc:290] T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 3 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:19:26.016230 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5d59407eb545b895d8d922e0dd69fd" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:19:26.018042 17673 consensus_queue.cc:260] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:26.018210 17645 consensus_queue.cc:237] T 829b4ad8c5a54eababeea1181b960d4b P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:26.015547 16292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31969392ccdb43b491911a319e8bc7e1" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:26.006253 16161 tablet_copy_service.cc:434] P 647a343a542d4a139fe106c19095129b: ending tablet copy session cb82cc7bd6db46d980c75237a1402860-806d7690c2b644959152074bb2cab0a7 on tablet 806d7690c2b644959152074bb2cab0a7 with peer cb82cc7bd6db46d980c75237a1402860
I20260709 13:19:26.026957 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5d59407eb545b895d8d922e0dd69fd" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:19:26.031124 16265 leader_election.cc:304] T 3884f864a96d428785895e5463e5728f P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
I20260709 13:19:26.023675 16115 leader_election.cc:304] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:19:26.032775 17673 raft_consensus.cc:3060] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:19:26.038632 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 75322.7us
I20260709 13:19:26.041308 17764 raft_consensus.cc:2749] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:19:26.044001 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87147731a68400e8cce9b879daa7c5d" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:26.029923 17703 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.StartTabletCopy from 127.0.0.1:49892 (request call id 242) took 4706 ms. Trace:
I20260709 13:19:26.047071 16294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e9b2bf7011f4ae688a79d7bb5a447b4" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:26.052665 16295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87147731a68400e8cce9b879daa7c5d" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:19:26.052826 16292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57a744f5aaf944678a63469cbf514015" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:26.048820 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a9de347c711477e87c582d003f36925" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:26.054906 16115 leader_election.cc:304] T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:19:26.050860 16190 leader_election.cc:304] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: cb82cc7bd6db46d980c75237a1402860
I20260709 13:19:26.051410 16291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9944d5e4b8c4109af9955c4f08c9095" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:26.052009 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26354361e9b044cea1c2b86c4882fe92" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:26.052425 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e65676b21b748728ad39668cfe722ec" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:26.059636 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a378d7f796f94615b3b347119a9880fd" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:19:26.060531 17764 raft_consensus.cc:2749] T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:19:26.062049 16299 raft_consensus.cc:2468] T c9944d5e4b8c4109af9955c4f08c9095 P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 3.
I20260709 13:19:26.054008 17703 rpcz_store.cc:276] 0709 13:19:21.323244 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:21.323382 (+   138us) service_pool.cc:224] Handling call
0709 13:19:21.549539 (+226157us) ts_tablet_manager.cc:934] T 19baa7bc367844c288bca8f81439c413 P cb82cc7bd6db46d980c75237a1402860: Initiating tablet copy from peer 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
0709 13:19:25.679209 (+4129670us) tablet_copy_client.cc:368] Tablet copy session begun
0709 13:19:25.944432 (+265223us) tablet_metadata.cc:665] Metadata flushed
0709 13:19:25.950879 (+  6447us) tablet_copy_client.cc:497] Wrote new tablet metadata
0709 13:19:26.029225 (+ 78346us) tablet_copy_client.cc:418] Wrote new consensus metadata
0709 13:19:26.029516 (+   291us) tablet_replica.cc:206] Creating consensus instance
0709 13:19:26.029779 (+   263us) ts_tablet_manager.cc:989] Replica registered.
0709 13:19:26.029853 (+    74us) inbound_call.cc:177] Queueing success response
Metrics: {"dns_us":252,"tablet-copy.queue_time_us":225326,"thread_start_us":289360,"threads_started":2}
I20260709 13:19:26.062454 17722 raft_consensus.cc:515] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:26.063323 17703 tablet_copy_client.cc:806] T 19baa7bc367844c288bca8f81439c413 P cb82cc7bd6db46d980c75237a1402860: tablet copy: Starting download of 0 data blocks...
I20260709 13:19:26.063894 17703 tablet_copy_client.cc:670] T 19baa7bc367844c288bca8f81439c413 P cb82cc7bd6db46d980c75237a1402860: tablet copy: Starting download of 1 WAL segments...
I20260709 13:19:26.064476 17518 consensus_queue.cc:1059] T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:19:26.069931 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f82224f4564fea93572e8206bad1fe" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:26.067593 16115 leader_election.cc:304] T 5a96373bb0e14dce8a266b2fc6c58507 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:19:26.067735 17718 leader_election.cc:290] T d2ab0ac8347641e6955e9bce7d90c1a2 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 3 election: Requested vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:19:26.068710 16819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ab0ac8347641e6955e9bce7d90c1a2" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:19:26.068809 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31969392ccdb43b491911a319e8bc7e1" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:26.027532 17782 consensus_queue.cc:1059] T b87147731a68400e8cce9b879daa7c5d P 90f3da57796b488c9087a2f523adf045 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.076s
I20260709 13:19:26.072624 17764 raft_consensus.cc:2749] T 5a96373bb0e14dce8a266b2fc6c58507 P 647a343a542d4a139fe106c19095129b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:19:26.074417 16115 leader_election.cc:304] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:19:26.075766 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2794af5b99984efebdee564035841cb0" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:26.077857 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74ca8140f35c4ae29e8a664f6bdfc829" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:26.078684 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e65676b21b748728ad39668cfe722ec" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:19:26.080854 17671 raft_consensus.cc:2804] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:19:26.081283 17671 raft_consensus.cc:493] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Starting leader election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:19:26.081692 17671 raft_consensus.cc:3060] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:19:26.085312 17764 raft_consensus.cc:2749] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:19:26.085851 16192 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:26.086328 16291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434bff3082d141838068f323bdb0d1ed" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:19:26.088143 17518 raft_consensus.cc:2749] T 3884f864a96d428785895e5463e5728f P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:19:26.088860 17518 consensus_queue.cc:1059] T 434bff3082d141838068f323bdb0d1ed P 90f3da57796b488c9087a2f523adf045 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:19:26.093626 17704 tablet_bootstrap.cc:492] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860: Bootstrap starting.
W20260709 13:19:26.104797 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 73760.2us
I20260709 13:19:26.107457 16190 leader_election.cc:304] T 1df93779209748b9860f6d893dd3de53 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:19:26.039935 17760 raft_consensus.cc:515] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } }
W20260709 13:19:26.109249 16266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 61185us
I20260709 13:19:26.110659 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a378d7f796f94615b3b347119a9880fd" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:26.112777 16291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3884f864a96d428785895e5463e5728f" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:26.113682 16819 raft_consensus.cc:1240] T 9e65676b21b748728ad39668cfe722ec P cb82cc7bd6db46d980c75237a1402860 [term 3 LEADER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 2. Current term is 3. Ops: []
I20260709 13:19:26.117751 16115 leader_election.cc:304] T fffab26427e24d398e2695aad40ca134 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:19:26.111709 16265 leader_election.cc:304] T 2794af5b99984efebdee564035841cb0 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
I20260709 13:19:26.097609 17788 raft_consensus.cc:2749] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:19:26.118916 17672 raft_consensus.cc:2749] T 1df93779209748b9860f6d893dd3de53 P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:19:26.120237 17779 raft_consensus.cc:3055] T b87147731a68400e8cce9b879daa7c5d P 90f3da57796b488c9087a2f523adf045 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:19:26.137452 17723 raft_consensus.cc:2749] T fffab26427e24d398e2695aad40ca134 P 647a343a542d4a139fe106c19095129b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:19:26.110271 17722 leader_election.cc:290] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 3 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:26.131865 17782 consensus_queue.cc:1059] T 9e65676b21b748728ad39668cfe722ec P 90f3da57796b488c9087a2f523adf045 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:19:26.137650 17779 raft_consensus.cc:740] T b87147731a68400e8cce9b879daa7c5d P 90f3da57796b488c9087a2f523adf045 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:19:26.102146 17789 raft_consensus.cc:3055] T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:19:26.138302 16224 raft_consensus.cc:2468] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 3.
I20260709 13:19:26.138710 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434bff3082d141838068f323bdb0d1ed" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:26.111709 17790 raft_consensus.cc:3055] T 434bff3082d141838068f323bdb0d1ed P 90f3da57796b488c9087a2f523adf045 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:19:26.137759 16115 leader_election.cc:304] T c0eab25ce4c64e018a041beeda2867db P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:19:26.143625 17782 consensus_queue.cc:1243] T 9e65676b21b748728ad39668cfe722ec P 90f3da57796b488c9087a2f523adf045 [LEADER]: Peer cb82cc7bd6db46d980c75237a1402860 log is divergent from this leader: its last log entry 3.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 13:19:26.144212 16115 leader_election.cc:304] T 5a959dbecc2648eb85d1cc550f5a3245 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:19:26.144436 17723 raft_consensus.cc:2749] T c0eab25ce4c64e018a041beeda2867db P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:19:26.141062 16190 leader_election.cc:304] T d16dfad0c3b540e5b4b86492ddf87208 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:19:26.140689 16296 raft_consensus.cc:2468] T a378d7f796f94615b3b347119a9880fd P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 2.
I20260709 13:19:26.145807 16790 leader_election.cc:304] T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 647a343a542d4a139fe106c19095129b
I20260709 13:19:26.146538 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e4bfb9090c8460eb14fa7e0d818e7f1" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:26.147552 16190 leader_election.cc:304] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: cb82cc7bd6db46d980c75237a1402860
I20260709 13:19:26.147979 17518 raft_consensus.cc:3055] T 9e65676b21b748728ad39668cfe722ec P 90f3da57796b488c9087a2f523adf045 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:19:26.148353 17518 raft_consensus.cc:740] T 9e65676b21b748728ad39668cfe722ec P 90f3da57796b488c9087a2f523adf045 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:19:26.149246 17518 consensus_queue.cc:260] T 9e65676b21b748728ad39668cfe722ec P 90f3da57796b488c9087a2f523adf045 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:26.150377 17518 raft_consensus.cc:3060] T 9e65676b21b748728ad39668cfe722ec P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:19:26.140475 17789 raft_consensus.cc:740] T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:19:26.152092 17789 consensus_queue.cc:260] T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:26.143220 17790 raft_consensus.cc:740] T 434bff3082d141838068f323bdb0d1ed P 90f3da57796b488c9087a2f523adf045 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:19:26.153309 17789 raft_consensus.cc:3060] T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:19:26.153522 17790 consensus_queue.cc:260] T 434bff3082d141838068f323bdb0d1ed P 90f3da57796b488c9087a2f523adf045 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:26.154836 17787 raft_consensus.cc:2804] T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:19:26.155017 17764 raft_consensus.cc:2749] T 5a959dbecc2648eb85d1cc550f5a3245 P 647a343a542d4a139fe106c19095129b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:19:26.155416 17787 raft_consensus.cc:697] T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860 [term 3 LEADER]: Becoming Leader. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:19:26.155831 17779 consensus_queue.cc:260] T b87147731a68400e8cce9b879daa7c5d P 90f3da57796b488c9087a2f523adf045 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:26.156332 17787 consensus_queue.cc:237] T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:26.157276 17779 raft_consensus.cc:3060] T b87147731a68400e8cce9b879daa7c5d P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:19:26.158283 17672 raft_consensus.cc:2749] T d16dfad0c3b540e5b4b86492ddf87208 P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:19:26.158936 17672 raft_consensus.cc:2804] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:19:26.159274 17672 raft_consensus.cc:493] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Starting leader election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:19:26.160018 16115 leader_election.cc:304] T ecc1e363230043fe9fef390f0b979f20 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:19:26.159649 17672 raft_consensus.cc:3060] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:19:26.154843 17790 raft_consensus.cc:3060] T 434bff3082d141838068f323bdb0d1ed P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:19:26.162297 17729 raft_consensus.cc:2749] T ecc1e363230043fe9fef390f0b979f20 P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:19:26.163920 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74ca8140f35c4ae29e8a664f6bdfc829" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
W20260709 13:19:26.165505 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 53846.1us
I20260709 13:19:26.166751 17729 raft_consensus.cc:493] T 02838c86531d4f98bf58d1c22b8baf77 P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:19:26.167390 17729 raft_consensus.cc:515] T 02838c86531d4f98bf58d1c22b8baf77 P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:26.178539 17729 leader_election.cc:290] T 02838c86531d4f98bf58d1c22b8baf77 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:26.179257 16222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02838c86531d4f98bf58d1c22b8baf77" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:26.179975 16222 raft_consensus.cc:2393] T 02838c86531d4f98bf58d1c22b8baf77 P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 3: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
I20260709 13:19:26.175386 16047 catalog_manager.cc:5697] T af57996a8be84d398f0f023e8d08ef77 P cb82cc7bd6db46d980c75237a1402860 reported cstate change: term changed from 1 to 2, leader changed from 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130) to cb82cc7bd6db46d980c75237a1402860 (127.14.177.132). New cstate: current_term: 2 leader_uuid: "cb82cc7bd6db46d980c75237a1402860" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:19:26.184839 16299 raft_consensus.cc:2468] T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 2.
I20260709 13:19:26.176759 16224 raft_consensus.cc:1240] T d2ab0ac8347641e6955e9bce7d90c1a2 P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 2. Current term is 3. Ops: []
I20260709 13:19:26.188524 17760 leader_election.cc:290] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 2 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:26.188930 17787 consensus_queue.cc:1059] T d2ab0ac8347641e6955e9bce7d90c1a2 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:19:26.190213 16144 raft_consensus.cc:1275] T 9e65676b21b748728ad39668cfe722ec P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:19:26.190542 17787 raft_consensus.cc:3055] T d2ab0ac8347641e6955e9bce7d90c1a2 P cb82cc7bd6db46d980c75237a1402860 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:19:26.190908 17787 raft_consensus.cc:740] T d2ab0ac8347641e6955e9bce7d90c1a2 P cb82cc7bd6db46d980c75237a1402860 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:19:26.187598 16047 catalog_manager.cc:5697] T 9e65676b21b748728ad39668cfe722ec P cb82cc7bd6db46d980c75237a1402860 reported cstate change: term changed from 2 to 3, leader changed from 90f3da57796b488c9087a2f523adf045 (127.14.177.131) to cb82cc7bd6db46d980c75237a1402860 (127.14.177.132). New cstate: current_term: 3 leader_uuid: "cb82cc7bd6db46d980c75237a1402860" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:19:26.193557 17781 raft_consensus.cc:2749] T 2794af5b99984efebdee564035841cb0 P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:19:26.167286 17671 raft_consensus.cc:515] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:26.194386 16047 catalog_manager.cc:5697] T 434bff3082d141838068f323bdb0d1ed P cb82cc7bd6db46d980c75237a1402860 reported cstate change: term changed from 2 to 3, leader changed from 90f3da57796b488c9087a2f523adf045 (127.14.177.131) to cb82cc7bd6db46d980c75237a1402860 (127.14.177.132). New cstate: current_term: 3 leader_uuid: "cb82cc7bd6db46d980c75237a1402860" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:19:26.192834 17645 consensus_queue.cc:1048] T 9e65676b21b748728ad39668cfe722ec P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:19:26.200443 16819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7ca61592e848ea9b0fb81a2ce33cdc" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:19:26.196307 17671 leader_election.cc:290] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 3 election: Requested vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:19:26.199378 16218 raft_consensus.cc:1240] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 2. Current term is 3. Ops: []
I20260709 13:19:26.203572 17645 consensus_queue.cc:1059] T 3a7ca61592e848ea9b0fb81a2ce33cdc P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:19:26.204655 17709 raft_consensus.cc:3055] T 3a7ca61592e848ea9b0fb81a2ce33cdc P cb82cc7bd6db46d980c75237a1402860 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:19:26.205013 17709 raft_consensus.cc:740] T 3a7ca61592e848ea9b0fb81a2ce33cdc P cb82cc7bd6db46d980c75237a1402860 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:19:26.205876 17709 consensus_queue.cc:260] T 3a7ca61592e848ea9b0fb81a2ce33cdc P cb82cc7bd6db46d980c75237a1402860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:26.206872 17709 raft_consensus.cc:3060] T 3a7ca61592e848ea9b0fb81a2ce33cdc P cb82cc7bd6db46d980c75237a1402860 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:19:26.207490 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 60058us
I20260709 13:19:26.209702 16819 raft_consensus.cc:1275] T c9944d5e4b8c4109af9955c4f08c9095 P cb82cc7bd6db46d980c75237a1402860 [term 3 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:19:26.191808 17787 consensus_queue.cc:260] T d2ab0ac8347641e6955e9bce7d90c1a2 P cb82cc7bd6db46d980c75237a1402860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:26.213032 17787 raft_consensus.cc:3060] T d2ab0ac8347641e6955e9bce7d90c1a2 P cb82cc7bd6db46d980c75237a1402860 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:19:26.215929 16296 raft_consensus.cc:1275] T 9e65676b21b748728ad39668cfe722ec P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:19:26.216490 16265 leader_election.cc:304] T 31969392ccdb43b491911a319e8bc7e1 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
I20260709 13:19:26.213289 17673 consensus_queue.cc:1048] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.005s
I20260709 13:19:26.218586 16265 leader_election.cc:304] T 23f82224f4564fea93572e8206bad1fe P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
I20260709 13:19:26.221319 17782 consensus_queue.cc:1059] T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:19:26.222405 16047 catalog_manager.cc:5697] T 829b4ad8c5a54eababeea1181b960d4b P cb82cc7bd6db46d980c75237a1402860 reported cstate change: term changed from 2 to 3, leader changed from 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130) to cb82cc7bd6db46d980c75237a1402860 (127.14.177.132). New cstate: current_term: 3 leader_uuid: "cb82cc7bd6db46d980c75237a1402860" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:19:26.224557 17781 raft_consensus.cc:2749] T 31969392ccdb43b491911a319e8bc7e1 P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:19:26.225406 17790 consensus_queue.cc:1059] T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:19:26.226094 17781 raft_consensus.cc:2749] T 23f82224f4564fea93572e8206bad1fe P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:19:26.229323 16190 leader_election.cc:304] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860
I20260709 13:19:26.227913 17790 raft_consensus.cc:3055] T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:19:26.234797 17790 raft_consensus.cc:740] T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:19:26.232749 16115 leader_election.cc:304] T b87147731a68400e8cce9b879daa7c5d P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860
I20260709 13:19:26.213981 17672 raft_consensus.cc:515] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:26.238708 17788 consensus_queue.cc:1059] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true }, Status: INVALID_TERM, Last received: 2.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:19:26.239287 17673 raft_consensus.cc:2749] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:19:26.246663 16190 leader_election.cc:304] T 8c11074579b046d99e585c2818e18a2c P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: cb82cc7bd6db46d980c75237a1402860
W20260709 13:19:26.246773 16789 leader_election.cc:336] T b87147731a68400e8cce9b879daa7c5d P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389): Timed out: RequestConsensusVote RPC to 127.14.177.131:41389 timed out after 1.594s (SENT)
I20260709 13:19:26.248116 16218 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "af57996a8be84d398f0f023e8d08ef77"
dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
mode: GRACEFUL
new_leader_uuid: "647a343a542d4a139fe106c19095129b"
 from {username='slave'} at 127.0.0.1:39144
I20260709 13:19:26.261233 16218 raft_consensus.cc:606] T af57996a8be84d398f0f023e8d08ef77 P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Received request to transfer leadership to TS 647a343a542d4a139fe106c19095129b
I20260709 13:19:26.261613 16218 raft_consensus.cc:614] T af57996a8be84d398f0f023e8d08ef77 P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:19:26.258656 16789 leader_election.cc:304] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:19:26.258729 17790 consensus_queue.cc:260] T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:26.255750 16265 leader_election.cc:304] T 26354361e9b044cea1c2b86c4882fe92 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
I20260709 13:19:26.262231 16115 leader_election.cc:304] T 9c61a39fd4f84424b129c1a6f9ba404d P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:19:26.282286 17673 raft_consensus.cc:2804] T 8c11074579b046d99e585c2818e18a2c P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:19:26.282740 17673 raft_consensus.cc:493] T 8c11074579b046d99e585c2818e18a2c P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Starting leader election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:19:26.283155 17764 raft_consensus.cc:2749] T b87147731a68400e8cce9b879daa7c5d P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:19:26.281391 17789 raft_consensus.cc:3055] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:19:26.283883 17789 raft_consensus.cc:740] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:19:26.285461 16789 leader_election.cc:304] T b87147731a68400e8cce9b879daa7c5d P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:19:26.285490 16190 leader_election.cc:304] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: cb82cc7bd6db46d980c75237a1402860
I20260709 13:19:26.284795 17789 consensus_queue.cc:260] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } }
I20260709 13:19:26.288275 16789 leader_election.cc:304] T 57a744f5aaf944678a63469cbf514015 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:19:26.288681 17721 raft_consensus.cc:2749] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:19:26.289258 17789 raft_consensus.cc:3060] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:19:26.289527 16789 leader_election.cc:304] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:19:26.291359 16224 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e5ad4eb281564936a59f41e387a47156"
dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
mode: GRACEFUL
new_leader_uuid: "90f3da57796b488c9087a2f523adf045"
 from {username='slave'} at 127.0.0.1:39144
I20260709 13:19:26.292155 16224 raft_consensus.cc:606] T e5ad4eb281564936a59f41e387a47156 P 1ca2725d290e47d68b1a4b609c9816ac [term 2 LEADER]: Received request to transfer leadership to TS 90f3da57796b488c9087a2f523adf045
I20260709 13:19:26.292918 16222 raft_consensus.cc:1240] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 2. Current term is 3. Ops: []
W20260709 13:19:26.294106 16263 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:26.289466 17721 raft_consensus.cc:2749] T b87147731a68400e8cce9b879daa7c5d P cb82cc7bd6db46d980c75237a1402860 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
W20260709 13:19:26.299526 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 43858.7us
I20260709 13:19:26.301019 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a878999c0024d2abe517484180290fc" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:26.283114 17673 raft_consensus.cc:3060] T 8c11074579b046d99e585c2818e18a2c P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:19:26.305982 16299 raft_consensus.cc:2468] T 8a878999c0024d2abe517484180290fc P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 1.
W20260709 13:19:26.303956 16115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 45817.2us
I20260709 13:19:26.301689 17671 raft_consensus.cc:2804] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:19:26.308543 16190 leader_election.cc:304] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: cb82cc7bd6db46d980c75237a1402860
I20260709 13:19:26.309626 16296 raft_consensus.cc:1275] T 434bff3082d141838068f323bdb0d1ed P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:19:26.294310 17738 raft_consensus.cc:2749] T 57a744f5aaf944678a63469cbf514015 P cb82cc7bd6db46d980c75237a1402860 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:19:26.301508 17710 raft_consensus.cc:2749] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:19:26.312808 17518 consensus_queue.cc:1059] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:19:26.302922 17764 raft_consensus.cc:2749] T 9c61a39fd4f84424b129c1a6f9ba404d P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:19:26.314760 16789 leader_election.cc:304] T 26354361e9b044cea1c2b86c4882fe92 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:19:26.304482 17779 raft_consensus.cc:2749] T 26354361e9b044cea1c2b86c4882fe92 P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:19:26.332079 16190 leader_election.cc:304] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860
I20260709 13:19:26.333295 16190 leader_election.cc:304] T 02838c86531d4f98bf58d1c22b8baf77 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:19:26.315951 17794 consensus_queue.cc:1059] T c4f65cdf00b748bb963ee6e81ef43b79 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:19:26.317885 17672 leader_election.cc:290] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 3 election: Requested vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:19:26.321990 16142 raft_consensus.cc:1275] T af57996a8be84d398f0f023e8d08ef77 P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:19:26.323725 16143 raft_consensus.cc:1240] T 8d4ab870a05347868956aa8b841b2981 P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 1. Current term is 2. Ops: []
I20260709 13:19:26.310992 17788 consensus_queue.cc:1059] T 8158bbd17b994d2eb96f560bd0f1b88d P 90f3da57796b488c9087a2f523adf045 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.016s
I20260709 13:19:26.313977 17762 raft_consensus.cc:2804] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:19:26.336678 17762 raft_consensus.cc:493] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Starting leader election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:19:26.337008 17719 raft_consensus.cc:2749] T 02838c86531d4f98bf58d1c22b8baf77 P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:19:26.327320 17790 raft_consensus.cc:3060] T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:19:26.328392 16264 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:26.334739 17781 raft_consensus.cc:493] T fffab26427e24d398e2695aad40ca134 P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:19:26.338519 17781 raft_consensus.cc:515] T fffab26427e24d398e2695aad40ca134 P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:26.339741 17738 consensus_queue.cc:1048] T af57996a8be84d398f0f023e8d08ef77 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:19:26.340695 17781 leader_election.cc:290] T fffab26427e24d398e2695aad40ca134 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:19:26.335206 17710 raft_consensus.cc:2749] T 26354361e9b044cea1c2b86c4882fe92 P cb82cc7bd6db46d980c75237a1402860 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:19:26.305068 17722 raft_consensus.cc:493] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:19:26.311333 17671 raft_consensus.cc:493] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Starting leader election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:19:26.341655 17722 raft_consensus.cc:515] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:26.342067 17671 raft_consensus.cc:3060] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:19:26.328912 16822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8638ee363273426db1393dd81e5c7b72" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:19:26.337071 17762 raft_consensus.cc:3060] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:19:26.305752 17721 consensus_queue.cc:1059] T 8638ee363273426db1393dd81e5c7b72 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.029s
I20260709 13:19:26.349841 16115 leader_election.cc:304] T 08c74490c40243a88e62d1760b5cd814 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:19:26.350063 16789 leader_election.cc:304] T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:19:26.344355 16190 leader_election.cc:304] T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:19:26.353135 16789 leader_election.cc:304] T c9944d5e4b8c4109af9955c4f08c9095 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:19:26.354257 17721 raft_consensus.cc:2749] T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:19:26.348812 17760 raft_consensus.cc:2749] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:19:26.356765 17672 raft_consensus.cc:2749] T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:19:26.380172 16190 leader_election.cc:304] T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:19:26.357131 17764 raft_consensus.cc:2749] T 08c74490c40243a88e62d1760b5cd814 P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:19:26.382861 16789 leader_election.cc:304] T 23f82224f4564fea93572e8206bad1fe P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:19:26.356698 17722 leader_election.cc:290] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:19:26.384824 16115 leader_election.cc:304] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:19:26.357951 16819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62fe3346cd3c42c4a55e14af9073bca4" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:19:26.361106 17695 raft_consensus.cc:3055] T 8638ee363273426db1393dd81e5c7b72 P cb82cc7bd6db46d980c75237a1402860 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:19:26.365722 17779 raft_consensus.cc:3055] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:19:26.392553 16190 leader_election.cc:304] T e4dc6b0541ca431ebc757beac00521f1 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:19:26.402621 16115 leader_election.cc:304] T 68a6838d82ef4cac83507f906139d90f P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:19:26.385389 16143 raft_consensus.cc:1275] T 434bff3082d141838068f323bdb0d1ed P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:19:26.404314 17721 raft_consensus.cc:2749] T 23f82224f4564fea93572e8206bad1fe P cb82cc7bd6db46d980c75237a1402860 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:19:26.405586 16819 raft_consensus.cc:2468] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 3.
I20260709 13:19:26.398249 17719 raft_consensus.cc:2749] T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:19:26.401350 17782 raft_consensus.cc:493] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:26.413019 17782 raft_consensus.cc:515] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:26.405161 17739 raft_consensus.cc:2749] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:19:26.400300 15046 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 13:19:26.416639 16046 catalog_manager.cc:5697] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac reported cstate change: term changed from 2 to 3, leader changed from 90f3da57796b488c9087a2f523adf045 (127.14.177.131) to 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130). New cstate: current_term: 3 leader_uuid: "1ca2725d290e47d68b1a4b609c9816ac" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } health_report { overall_health: UNKNOWN } } }
I20260709 13:19:26.417960 15046 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:19:26.400574 17695 raft_consensus.cc:740] T 8638ee363273426db1393dd81e5c7b72 P cb82cc7bd6db46d980c75237a1402860 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:19:26.419075 17644 raft_consensus.cc:3060] T 68a6838d82ef4cac83507f906139d90f P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:19:26.419873 17695 consensus_queue.cc:260] T 8638ee363273426db1393dd81e5c7b72 P cb82cc7bd6db46d980c75237a1402860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:26.421228 17695 raft_consensus.cc:3060] T 8638ee363273426db1393dd81e5c7b72 P cb82cc7bd6db46d980c75237a1402860 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:19:26.424045 17760 raft_consensus.cc:2749] T e4dc6b0541ca431ebc757beac00521f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:19:26.425140 17791 raft_consensus.cc:3055] T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:19:26.425704 17791 raft_consensus.cc:740] T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:19:26.427685 17781 raft_consensus.cc:3055] T c4f65cdf00b748bb963ee6e81ef43b79 P 90f3da57796b488c9087a2f523adf045 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:19:26.428061 17781 raft_consensus.cc:740] T c4f65cdf00b748bb963ee6e81ef43b79 P 90f3da57796b488c9087a2f523adf045 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:19:26.428318 16192 leader_election.cc:304] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:19:26.429258 17758 raft_consensus.cc:2804] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:19:26.428867 17781 consensus_queue.cc:260] T c4f65cdf00b748bb963ee6e81ef43b79 P 90f3da57796b488c9087a2f523adf045 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:26.429631 17758 raft_consensus.cc:493] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Starting leader election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:19:26.430037 17758 raft_consensus.cc:3060] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:19:26.430096 17781 raft_consensus.cc:3060] T c4f65cdf00b748bb963ee6e81ef43b79 P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:19:26.418987 16046 catalog_manager.cc:5697] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac reported cstate change: term changed from 2 to 3, leader changed from 90f3da57796b488c9087a2f523adf045 (127.14.177.131) to 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130). New cstate: current_term: 3 leader_uuid: "1ca2725d290e47d68b1a4b609c9816ac" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:19:26.434659 16222 raft_consensus.cc:1240] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 2. Current term is 3. Ops: []
I20260709 13:19:26.434216 16115 leader_election.cc:304] T b6f07842d3aa47858df5665874c86f3a P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:19:26.435441 16190 leader_election.cc:304] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: cb82cc7bd6db46d980c75237a1402860
W20260709 13:19:26.436468 16789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 53967.5us
I20260709 13:19:26.437286 16789 leader_election.cc:304] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:19:26.434617 16046 catalog_manager.cc:5697] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac reported cstate change: term changed from 2 to 3, leader changed from 90f3da57796b488c9087a2f523adf045 (127.14.177.131) to 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130). New cstate: current_term: 3 leader_uuid: "1ca2725d290e47d68b1a4b609c9816ac" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } health_report { overall_health: UNKNOWN } } }
I20260709 13:19:26.437371 16115 leader_election.cc:304] T 434bff3082d141838068f323bdb0d1ed P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860
I20260709 13:19:26.406129 17710 raft_consensus.cc:2749] T c9944d5e4b8c4109af9955c4f08c9095 P cb82cc7bd6db46d980c75237a1402860 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:19:26.403865 17779 raft_consensus.cc:740] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:19:26.439014 16046 catalog_manager.cc:5697] T c2f5f89186204242bf377f7c42b2e2da P 1ca2725d290e47d68b1a4b609c9816ac reported cstate change: term changed from 2 to 3, leader changed from 90f3da57796b488c9087a2f523adf045 (127.14.177.131) to 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130). New cstate: current_term: 3 leader_uuid: "1ca2725d290e47d68b1a4b609c9816ac" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } health_report { overall_health: UNKNOWN } } }
I20260709 13:19:26.447750 16789 leader_election.cc:304] T 2794af5b99984efebdee564035841cb0 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:19:26.444881 16148 raft_consensus.cc:3060] T 829b4ad8c5a54eababeea1181b960d4b P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:19:26.439288 17739 raft_consensus.cc:2749] T b6f07842d3aa47858df5665874c86f3a P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:19:26.450027 16115 leader_election.cc:304] T 8cc959fb7f704fba9009c0595467d074 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:19:26.440500 17782 leader_election.cc:290] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:19:26.452092 17452 consensus_queue.cc:1059] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 647a343a542d4a139fe106c19095129b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.014s
I20260709 13:19:26.446651 17721 raft_consensus.cc:2749] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:19:26.442870 17638 raft_consensus.cc:2804] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:19:26.459054 17779 consensus_queue.cc:260] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:26.464824 17787 raft_consensus.cc:2749] T 2794af5b99984efebdee564035841cb0 P cb82cc7bd6db46d980c75237a1402860 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:19:26.467645 17764 raft_consensus.cc:3055] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 647a343a542d4a139fe106c19095129b [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:19:26.486477 17764 raft_consensus.cc:740] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 647a343a542d4a139fe106c19095129b [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:19:26.486698 17779 raft_consensus.cc:3060] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:19:26.487358 17764 consensus_queue.cc:260] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:26.488484 17764 raft_consensus.cc:3060] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:19:26.465197 17645 consensus_queue.cc:1048] T 434bff3082d141838068f323bdb0d1ed P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.007s
I20260709 13:19:26.491780 16295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806d7690c2b644959152074bb2cab0a7" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:19:26.473803 16294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ab0ac8347641e6955e9bce7d90c1a2" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:19:26.492595 16218 raft_consensus.cc:1240] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 1. Current term is 2. Ops: []
W20260709 13:19:26.474073 17793 log.cc:927] Time spent T 9e65676b21b748728ad39668cfe722ec P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 0.119s	user 0.000s	sys 0.002s
I20260709 13:19:26.494669 16149 raft_consensus.cc:1240] T 9e65676b21b748728ad39668cfe722ec P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 2. Current term is 3. Ops: []
W20260709 13:19:26.495478 16789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 47819.4us
I20260709 13:19:26.495805 16295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8638ee363273426db1393dd81e5c7b72" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:19:26.474808 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:19:26.475446 16296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68a6838d82ef4cac83507f906139d90f" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:19:26.477069 16298 raft_consensus.cc:1275] T c9944d5e4b8c4109af9955c4f08c9095 P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:19:26.497617 16789 leader_election.cc:304] T 3884f864a96d428785895e5463e5728f P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:19:26.474635 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87ac0d1f7e8a419f9bcbc04aee0e067e" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:19:26.498348 17709 raft_consensus.cc:2749] T 3884f864a96d428785895e5463e5728f P cb82cc7bd6db46d980c75237a1402860 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:19:26.498793 16293 raft_consensus.cc:3060] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:19:26.500487 16789 leader_election.cc:304] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:19:26.450169 17729 raft_consensus.cc:2749] T 434bff3082d141838068f323bdb0d1ed P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:19:26.484939 17762 raft_consensus.cc:515] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:26.515435 17723 raft_consensus.cc:2749] T 8cc959fb7f704fba9009c0595467d074 P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:19:26.504024 16294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7ca61592e848ea9b0fb81a2ce33cdc" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:19:26.521711 16218 raft_consensus.cc:1275] T 829b4ad8c5a54eababeea1181b960d4b P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
W20260709 13:19:26.519270 16115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 69303.3us
I20260709 13:19:26.518833 16297 raft_consensus.cc:3055] T 8158bbd17b994d2eb96f560bd0f1b88d P 90f3da57796b488c9087a2f523adf045 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:19:26.515949 17452 consensus_queue.cc:1059] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true }, Status: INVALID_TERM, Last received: 1.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.008s
I20260709 13:19:26.523541 16297 raft_consensus.cc:740] T 8158bbd17b994d2eb96f560bd0f1b88d P 90f3da57796b488c9087a2f523adf045 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:19:26.523864 17452 raft_consensus.cc:3055] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:19:26.524199 17452 raft_consensus.cc:740] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:19:26.524672 17762 leader_election.cc:290] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 3 election: Requested vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:19:26.524392 16297 consensus_queue.cc:260] T 8158bbd17b994d2eb96f560bd0f1b88d P 90f3da57796b488c9087a2f523adf045 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:19:26.525012 16192 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:26.501034 16266 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:26.485072 17673 raft_consensus.cc:515] T 8c11074579b046d99e585c2818e18a2c P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:26.426829 17791 consensus_queue.cc:260] T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:26.485790 17638 raft_consensus.cc:493] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Starting leader election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:19:26.494585 16291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6f07842d3aa47858df5665874c86f3a" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:19:26.458675 16148 raft_consensus.cc:1275] T 829b4ad8c5a54eababeea1181b960d4b P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
W20260709 13:19:26.563431 16264 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:26.524910 17452 consensus_queue.cc:260] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } }
W20260709 13:19:26.567489 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 54751.9us
I20260709 13:19:26.503651 17644 raft_consensus.cc:2749] T 68a6838d82ef4cac83507f906139d90f P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:19:26.468223 17671 raft_consensus.cc:515] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:26.523015 16115 leader_election.cc:304] T e5ad4eb281564936a59f41e387a47156 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
W20260709 13:19:26.568636 16266 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:26.571597 16790 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:26.571841 16046 catalog_manager.cc:5697] T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860 reported cstate change: term changed from 2 to 3, leader changed from 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130) to cb82cc7bd6db46d980c75237a1402860 (127.14.177.132). New cstate: current_term: 3 leader_uuid: "cb82cc7bd6db46d980c75237a1402860" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:19:26.570987 17638 raft_consensus.cc:3060] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:19:26.578409 17791 raft_consensus.cc:3060] T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:19:26.566892 17788 raft_consensus.cc:493] T 5a96373bb0e14dce8a266b2fc6c58507 P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:19:26.573958 16297 raft_consensus.cc:3060] T 8158bbd17b994d2eb96f560bd0f1b88d P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:19:26.559638 17709 raft_consensus.cc:2749] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P cb82cc7bd6db46d980c75237a1402860 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:19:26.571008 16190 leader_election.cc:304] T d2ab0ac8347641e6955e9bce7d90c1a2 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860
W20260709 13:19:26.580814 16789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 80386us
I20260709 13:19:26.581909 16265 leader_election.cc:304] T d16dfad0c3b540e5b4b86492ddf87208 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
W20260709 13:19:26.582757 16117 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:26.583391 17452 raft_consensus.cc:3060] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:19:26.582991 16789 leader_election.cc:304] T 74ca8140f35c4ae29e8a664f6bdfc829 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac
I20260709 13:19:26.584756 16224 raft_consensus.cc:1240] T 8c11074579b046d99e585c2818e18a2c P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 2. Current term is 3. Ops: []
I20260709 13:19:26.591794 16190 leader_election.cc:304] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860
I20260709 13:19:26.597210 16265 leader_election.cc:304] T 1df93779209748b9860f6d893dd3de53 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
W20260709 13:19:26.599797 16115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 76848.5us
I20260709 13:19:26.600538 16115 leader_election.cc:304] T 8d4ab870a05347868956aa8b841b2981 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:19:26.593211 17719 raft_consensus.cc:2749] T d2ab0ac8347641e6955e9bce7d90c1a2 P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
W20260709 13:19:26.671437 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 74313.5us
W20260709 13:19:26.693496 16263 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:26.696617 16265 leader_election.cc:304] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860
I20260709 13:19:26.667865 16299 raft_consensus.cc:2468] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 3.
W20260709 13:19:26.696615 16115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 96150.5us
I20260709 13:19:26.694403 16819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdbd78acb4cf46298e0de8ba0d130e19" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:19:26.587954 17729 raft_consensus.cc:2749] T e5ad4eb281564936a59f41e387a47156 P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:19:26.671809 17718 raft_consensus.cc:2749] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:19:26.699077 16297 raft_consensus.cc:1275] T 8158bbd17b994d2eb96f560bd0f1b88d P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:19:26.598752 17789 raft_consensus.cc:2749] T d16dfad0c3b540e5b4b86492ddf87208 P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:19:26.584278 17787 consensus_queue.cc:1048] T 829b4ad8c5a54eababeea1181b960d4b P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.113s
I20260709 13:19:26.700521 17723 raft_consensus.cc:2749] T 8d4ab870a05347868956aa8b841b2981 P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
W20260709 13:19:26.699308 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 107637us
W20260709 13:19:26.705590 17518 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 3 current term: 3 is higher.
W20260709 13:19:26.699746 16789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 116813us
I20260709 13:19:26.586905 17788 raft_consensus.cc:515] T 5a96373bb0e14dce8a266b2fc6c58507 P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:26.708009 17709 consensus_queue.cc:1048] T 829b4ad8c5a54eababeea1181b960d4b P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:19:26.710744 16190 leader_election.cc:304] T 68a6838d82ef4cac83507f906139d90f P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
W20260709 13:19:26.712430 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 132268us
I20260709 13:19:26.712540 16265 leader_election.cc:304] T c0eab25ce4c64e018a041beeda2867db P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:19:26.700102 17739 raft_consensus.cc:493] T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:26.714777 16115 leader_election.cc:304] T 8158bbd17b994d2eb96f560bd0f1b88d P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:19:26.714754 16190 leader_election.cc:304] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860
I20260709 13:19:26.716023 17719 raft_consensus.cc:2749] T 68a6838d82ef4cac83507f906139d90f P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:19:26.718147 17722 consensus_queue.cc:1048] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:19:26.704454 17645 raft_consensus.cc:2804] T 74ca8140f35c4ae29e8a664f6bdfc829 P cb82cc7bd6db46d980c75237a1402860 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:19:26.698706 17799 raft_consensus.cc:2749] T 1df93779209748b9860f6d893dd3de53 P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:19:26.722183 16265 leader_election.cc:304] T d2ab0ac8347641e6955e9bce7d90c1a2 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860
I20260709 13:19:26.724701 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdbd78acb4cf46298e0de8ba0d130e19" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:19:26.726657 17722 raft_consensus.cc:493] T 66f75124215f4ca6bef831e174a792d8 P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:19:26.727265 17703 tablet_copy_client.cc:538] T 19baa7bc367844c288bca8f81439c413 P cb82cc7bd6db46d980c75237a1402860: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:19:26.719723 17789 raft_consensus.cc:2749] T c0eab25ce4c64e018a041beeda2867db P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:19:26.709805 17788 leader_election.cc:290] T 5a96373bb0e14dce8a266b2fc6c58507 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:19:26.710633 17779 raft_consensus.cc:2749] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:19:26.727698 17802 raft_consensus.cc:2749] T 8158bbd17b994d2eb96f560bd0f1b88d P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:19:26.742376 16265 leader_election.cc:304] T 66ae176961b74813a58bc15c9c4c6f2a P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860
I20260709 13:19:26.729723 17719 raft_consensus.cc:2749] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
W20260709 13:19:26.745204 16263 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:26.746941 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66ae176961b74813a58bc15c9c4c6f2a" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:19:26.722805 17671 leader_election.cc:290] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 3 election: Requested vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:19:26.726763 17710 consensus_queue.cc:1059] T 8c11074579b046d99e585c2818e18a2c P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.018s
I20260709 13:19:26.761050 16265 leader_election.cc:304] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:19:26.762212 16265 leader_election.cc:304] T ada5a004ba7c46819051e3ccec71ffdb P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860
I20260709 13:19:26.761991 17638 raft_consensus.cc:515] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:26.763692 16265 leader_election.cc:304] T 84f2da98864b4425bf7c0055951fdd4b P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:19:26.752796 17803 consensus_queue.cc:1048] T 9e65676b21b748728ad39668cfe722ec P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:19:26.749545 16819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66ae176961b74813a58bc15c9c4c6f2a" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:19:26.740047 17645 raft_consensus.cc:697] T 74ca8140f35c4ae29e8a664f6bdfc829 P cb82cc7bd6db46d980c75237a1402860 [term 2 LEADER]: Becoming Leader. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:19:26.775394 17791 raft_consensus.cc:2749] T 84f2da98864b4425bf7c0055951fdd4b P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:19:26.775794 17645 consensus_queue.cc:237] T 74ca8140f35c4ae29e8a664f6bdfc829 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:26.776767 16265 leader_election.cc:304] T 8c11074579b046d99e585c2818e18a2c P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860
I20260709 13:19:26.756148 16115 leader_election.cc:304] T 7c2095a5f4ad49408dbc7f69c606120d P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:19:26.763024 17789 raft_consensus.cc:2749] T 66ae176961b74813a58bc15c9c4c6f2a P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:19:26.784765 17638 leader_election.cc:290] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 3 election: Requested vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:19:26.784971 17719 raft_consensus.cc:993] T e5ad4eb281564936a59f41e387a47156 P 1ca2725d290e47d68b1a4b609c9816ac: : Instructing follower 90f3da57796b488c9087a2f523adf045 to start an election
I20260709 13:19:26.784703 17671 consensus_queue.cc:1048] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:19:26.785517 17719 raft_consensus.cc:1081] T e5ad4eb281564936a59f41e387a47156 P 1ca2725d290e47d68b1a4b609c9816ac [term 2 LEADER]: Signalling peer 90f3da57796b488c9087a2f523adf045 to start an election
I20260709 13:19:26.764521 17790 raft_consensus.cc:2749] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:19:26.765064 17779 raft_consensus.cc:2749] T ada5a004ba7c46819051e3ccec71ffdb P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:19:26.752192 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c11074579b046d99e585c2818e18a2c" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:19:26.744783 17782 raft_consensus.cc:2749] T d2ab0ac8347641e6955e9bce7d90c1a2 P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:19:26.721623 17739 raft_consensus.cc:515] T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:26.781303 16298 raft_consensus.cc:1275] T c4f65cdf00b748bb963ee6e81ef43b79 P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:19:26.783273 16190 leader_election.cc:304] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860
I20260709 13:19:26.791379 17790 consensus_queue.cc:1059] T 68a6838d82ef4cac83507f906139d90f P 90f3da57796b488c9087a2f523adf045 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:19:26.790876 17789 raft_consensus.cc:2749] T 8c11074579b046d99e585c2818e18a2c P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:19:26.792445 17718 raft_consensus.cc:2749] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:19:26.792709 16190 leader_election.cc:304] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860
I20260709 13:19:26.794025 17718 raft_consensus.cc:2749] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:19:26.794504 17704 tablet_bootstrap.cc:492] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:19:26.792480 17794 raft_consensus.cc:3055] T 68a6838d82ef4cac83507f906139d90f P 90f3da57796b488c9087a2f523adf045 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:19:26.795077 17794 raft_consensus.cc:740] T 68a6838d82ef4cac83507f906139d90f P 90f3da57796b488c9087a2f523adf045 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:19:26.795094 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada5a004ba7c46819051e3ccec71ffdb" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:19:26.796034 16115 leader_election.cc:304] T 84f2da98864b4425bf7c0055951fdd4b P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:19:26.796432 16297 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e5ad4eb281564936a59f41e387a47156"
dest_uuid: "90f3da57796b488c9087a2f523adf045"
 from {username='slave'} at 127.0.0.1:48684
I20260709 13:19:26.796252 17794 consensus_queue.cc:260] T 68a6838d82ef4cac83507f906139d90f P 90f3da57796b488c9087a2f523adf045 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:26.796980 16297 raft_consensus.cc:493] T e5ad4eb281564936a59f41e387a47156 P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:19:26.797173 16115 leader_election.cc:304] T 8a878999c0024d2abe517484180290fc P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac
I20260709 13:19:26.797360 16297 raft_consensus.cc:3060] T e5ad4eb281564936a59f41e387a47156 P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:19:26.797577 17794 raft_consensus.cc:3060] T 68a6838d82ef4cac83507f906139d90f P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:19:26.800308 16265 leader_election.cc:304] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:19:26.804070 17723 raft_consensus.cc:2749] T 7c2095a5f4ad49408dbc7f69c606120d P 647a343a542d4a139fe106c19095129b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:19:26.804961 16295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:26.810784 17518 raft_consensus.cc:2749] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:19:26.811432 16295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d4ab870a05347868956aa8b841b2981" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:26.818693 16293 raft_consensus.cc:2468] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 3.
I20260709 13:19:26.746045 17722 raft_consensus.cc:515] T 66f75124215f4ca6bef831e174a792d8 P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:26.818976 17739 leader_election.cc:290] T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:19:26.861675 16291 raft_consensus.cc:2468] T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 2.
I20260709 13:19:26.824241 17729 raft_consensus.cc:493] T ee9ed67006144797b1de6ea8c2013a68 P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:26.820586 17764 raft_consensus.cc:2804] T 8a878999c0024d2abe517484180290fc P 647a343a542d4a139fe106c19095129b [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:19:26.829798 16297 raft_consensus.cc:515] T e5ad4eb281564936a59f41e387a47156 P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:26.863763 17764 raft_consensus.cc:493] T 8a878999c0024d2abe517484180290fc P 647a343a542d4a139fe106c19095129b [term 1 FOLLOWER]: Starting leader election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:19:26.864277 17764 raft_consensus.cc:3060] T 8a878999c0024d2abe517484180290fc P 647a343a542d4a139fe106c19095129b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:19:26.865537 16299 raft_consensus.cc:1240] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 13:19:26.867108 16299 raft_consensus.cc:1275] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:19:26.819478 17673 leader_election.cc:290] T 8c11074579b046d99e585c2818e18a2c P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 3 election: Requested vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
W20260709 13:19:26.854943 17801 log.cc:927] Time spent T af57996a8be84d398f0f023e8d08ef77 P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 0.156s	user 0.000s	sys 0.001s
I20260709 13:19:26.819880 17802 raft_consensus.cc:2749] T 84f2da98864b4425bf7c0055951fdd4b P 647a343a542d4a139fe106c19095129b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:19:26.867923 17729 raft_consensus.cc:515] T ee9ed67006144797b1de6ea8c2013a68 P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:26.821884 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66f75124215f4ca6bef831e174a792d8" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:26.812525 16265 leader_election.cc:304] T e4dc6b0541ca431ebc757beac00521f1 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:19:26.870553 16224 raft_consensus.cc:2468] T 66f75124215f4ca6bef831e174a792d8 P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 1.
I20260709 13:19:26.871575 17781 raft_consensus.cc:2749] T e4dc6b0541ca431ebc757beac00521f1 P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:19:26.872279 16116 leader_election.cc:304] T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
I20260709 13:19:26.828469 17695 raft_consensus.cc:3055] T 8c11074579b046d99e585c2818e18a2c P cb82cc7bd6db46d980c75237a1402860 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:19:26.873171 17695 raft_consensus.cc:740] T 8c11074579b046d99e585c2818e18a2c P cb82cc7bd6db46d980c75237a1402860 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:19:26.873391 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee9ed67006144797b1de6ea8c2013a68" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:26.873692 16824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66f75124215f4ca6bef831e174a792d8" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
W20260709 13:19:26.870914 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 58481.6us
I20260709 13:19:26.874544 16824 raft_consensus.cc:2393] T 66f75124215f4ca6bef831e174a792d8 P cb82cc7bd6db46d980c75237a1402860 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 2: Already voted for candidate cb82cc7bd6db46d980c75237a1402860 in this term.
I20260709 13:19:26.874166 17695 consensus_queue.cc:260] T 8c11074579b046d99e585c2818e18a2c P cb82cc7bd6db46d980c75237a1402860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:19:26.875092 16263 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:26.859683 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18ad7e3af5504571a69d10c00750a01c" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:26.807674 17758 raft_consensus.cc:515] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:26.876233 17729 leader_election.cc:290] T ee9ed67006144797b1de6ea8c2013a68 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:19:26.876901 17729 raft_consensus.cc:2804] T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:19:26.877314 17729 raft_consensus.cc:493] T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:26.877698 17729 raft_consensus.cc:3060] T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:19:26.878250 17758 leader_election.cc:290] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 3 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:19:26.865512 16297 leader_election.cc:290] T e5ad4eb281564936a59f41e387a47156 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 3 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:19:26.885320 16145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87ac0d1f7e8a419f9bcbc04aee0e067e" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:19:26.888331 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02838c86531d4f98bf58d1c22b8baf77" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:26.888630 16143 raft_consensus.cc:1275] T 8158bbd17b994d2eb96f560bd0f1b88d P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:19:26.890607 16151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62fe3346cd3c42c4a55e14af9073bca4" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:19:26.891515 16151 raft_consensus.cc:2393] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 3: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
I20260709 13:19:26.891631 16142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806d7690c2b644959152074bb2cab0a7" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:19:26.843791 17704 tablet_bootstrap.cc:492] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860: Bootstrap complete.
I20260709 13:19:26.865619 16190 leader_election.cc:304] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:19:26.874743 16265 leader_election.cc:304] T 7c2095a5f4ad49408dbc7f69c606120d P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:19:26.895756 17760 raft_consensus.cc:2804] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:19:26.895897 16190 leader_election.cc:304] T b6f07842d3aa47858df5665874c86f3a P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 647a343a542d4a139fe106c19095129b
I20260709 13:19:26.896347 17760 raft_consensus.cc:697] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 1ca2725d290e47d68b1a4b609c9816ac [term 3 LEADER]: Becoming Leader. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:19:26.896488 16265 leader_election.cc:304] T 8cc959fb7f704fba9009c0595467d074 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:19:26.897197 17518 raft_consensus.cc:2749] T 7c2095a5f4ad49408dbc7f69c606120d P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:19:26.897874 17518 raft_consensus.cc:2749] T 8cc959fb7f704fba9009c0595467d074 P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:19:26.897334 17760 consensus_queue.cc:237] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:26.899140 17718 raft_consensus.cc:2804] T b6f07842d3aa47858df5665874c86f3a P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:19:26.899363 16265 leader_election.cc:304] T 18ad7e3af5504571a69d10c00750a01c P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:19:26.898044 17704 ts_tablet_manager.cc:1403] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860: Time spent bootstrapping tablet: real 0.805s	user 0.045s	sys 0.034s
I20260709 13:19:26.900532 16265 leader_election.cc:304] T e5ad4eb281564936a59f41e387a47156 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:19:26.900624 16145 raft_consensus.cc:2468] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 3.
I20260709 13:19:26.900593 17718 raft_consensus.cc:697] T b6f07842d3aa47858df5665874c86f3a P 1ca2725d290e47d68b1a4b609c9816ac [term 2 LEADER]: Becoming Leader. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:19:26.900761 17781 raft_consensus.cc:2749] T 18ad7e3af5504571a69d10c00750a01c P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:19:26.902752 17718 consensus_queue.cc:237] T b6f07842d3aa47858df5665874c86f3a P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:26.826581 16819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada5a004ba7c46819051e3ccec71ffdb" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:19:26.905901 16192 leader_election.cc:304] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860
I20260709 13:19:26.897262 16190 leader_election.cc:304] T 19baa7bc367844c288bca8f81439c413 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 647a343a542d4a139fe106c19095129b
I20260709 13:19:26.836333 16822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c11074579b046d99e585c2818e18a2c" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:19:26.910352 16265 leader_election.cc:304] T 8638ee363273426db1393dd81e5c7b72 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860
I20260709 13:19:26.872433 17722 leader_election.cc:290] T 66f75124215f4ca6bef831e174a792d8 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
W20260709 13:19:26.939256 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 217984us
I20260709 13:19:26.950254 17518 raft_consensus.cc:2749] T 8638ee363273426db1393dd81e5c7b72 P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:19:26.953655 16224 raft_consensus.cc:1240] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 2. Current term is 3. Ops: []
I20260709 13:19:26.918908 17781 raft_consensus.cc:2749] T e5ad4eb281564936a59f41e387a47156 P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:19:26.957188 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 46904us
I20260709 13:19:26.961952 16265 leader_election.cc:304] T ecc1e363230043fe9fef390f0b979f20 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:19:26.986658 16224 raft_consensus.cc:1240] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 2. Current term is 3. Ops: []
I20260709 13:19:26.987407 16265 leader_election.cc:304] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860
I20260709 13:19:26.991999 17518 raft_consensus.cc:2749] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:19:26.957563 16819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62fe3346cd3c42c4a55e14af9073bca4" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
W20260709 13:19:26.992475 16789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 246088us
I20260709 13:19:26.937856 17719 consensus_queue.cc:1048] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.048s
I20260709 13:19:26.996860 17805 raft_consensus.cc:2804] T 19baa7bc367844c288bca8f81439c413 P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:19:26.997818 17800 raft_consensus.cc:2749] T ecc1e363230043fe9fef390f0b979f20 P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:19:27.005278 17518 consensus_queue.cc:1059] T 8d4ab870a05347868956aa8b841b2981 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:19:26.950232 17672 consensus_queue.cc:1048] T c2f5f89186204242bf377f7c42b2e2da P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.040s
W20260709 13:19:27.005954 16116 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:26.970248 17673 raft_consensus.cc:2749] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
W20260709 13:19:27.031997 16265 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:27.030071 16144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fffab26427e24d398e2695aad40ca134" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:19:27.034121 17518 raft_consensus.cc:3055] T 8d4ab870a05347868956aa8b841b2981 P 90f3da57796b488c9087a2f523adf045 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:19:27.034605 17518 raft_consensus.cc:740] T 8d4ab870a05347868956aa8b841b2981 P 90f3da57796b488c9087a2f523adf045 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:19:27.035539 17518 consensus_queue.cc:260] T 8d4ab870a05347868956aa8b841b2981 P 90f3da57796b488c9087a2f523adf045 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:27.040051 16146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87ac0d1f7e8a419f9bcbc04aee0e067e" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:19:27.040993 16146 raft_consensus.cc:2393] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 3: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
I20260709 13:19:27.046458 16144 raft_consensus.cc:1275] T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:19:27.048439 16146 raft_consensus.cc:1240] T 68a6838d82ef4cac83507f906139d90f P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 2. Current term is 3. Ops: []
I20260709 13:19:27.050951 17518 raft_consensus.cc:3060] T 8d4ab870a05347868956aa8b841b2981 P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:19:27.028676 17805 raft_consensus.cc:697] T 19baa7bc367844c288bca8f81439c413 P 1ca2725d290e47d68b1a4b609c9816ac [term 3 LEADER]: Becoming Leader. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:19:27.025863 17738 consensus_queue.cc:1048] T 434bff3082d141838068f323bdb0d1ed P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.024s
I20260709 13:19:27.028157 17790 consensus_queue.cc:1059] T 02838c86531d4f98bf58d1c22b8baf77 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.030s
I20260709 13:19:27.070076 16144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a96373bb0e14dce8a266b2fc6c58507" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:19:27.006021 17704 raft_consensus.cc:359] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860 [term 2 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } }
W20260709 13:19:27.074481 16192 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 168674us
I20260709 13:19:27.075397 16145 raft_consensus.cc:1275] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:19:27.080332 16145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5ad4eb281564936a59f41e387a47156" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:19:27.080930 16145 raft_consensus.cc:3060] T e5ad4eb281564936a59f41e387a47156 P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:19:27.081092 16117 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:27.075673 16142 raft_consensus.cc:2468] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 2.
I20260709 13:19:27.077244 17673 raft_consensus.cc:493] T ee9ed67006144797b1de6ea8c2013a68 P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:19:27.084236 17809 consensus_queue.cc:1059] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:19:27.085870 17809 consensus_queue.cc:1059] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:19:27.066654 16219 raft_consensus.cc:1275] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:19:27.087452 17804 raft_consensus.cc:3055] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:19:27.087870 17804 raft_consensus.cc:740] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:19:27.087945 17673 raft_consensus.cc:515] T ee9ed67006144797b1de6ea8c2013a68 P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:27.088784 17804 consensus_queue.cc:260] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:27.090041 17804 raft_consensus.cc:3060] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:19:27.086927 17809 raft_consensus.cc:3055] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:19:27.090499 17673 leader_election.cc:290] T ee9ed67006144797b1de6ea8c2013a68 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:19:27.095561 16145 raft_consensus.cc:2468] T e5ad4eb281564936a59f41e387a47156 P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 3.
I20260709 13:19:27.107360 16142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66f75124215f4ca6bef831e174a792d8" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:19:27.095103 17721 consensus_queue.cc:1048] T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.015s
I20260709 13:19:27.110769 16819 raft_consensus.cc:2468] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 3.
I20260709 13:19:27.111235 17703 tablet_bootstrap.cc:492] T 19baa7bc367844c288bca8f81439c413 P cb82cc7bd6db46d980c75237a1402860: Bootstrap starting.
W20260709 13:19:27.114044 17811 log.cc:927] Time spent T c9944d5e4b8c4109af9955c4f08c9095 P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.179s	user 0.000s	sys 0.004s
I20260709 13:19:27.111016 17695 raft_consensus.cc:3060] T 8c11074579b046d99e585c2818e18a2c P cb82cc7bd6db46d980c75237a1402860 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:19:27.134483 16145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62fe3346cd3c42c4a55e14af9073bca4" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:19:27.075945 17704 raft_consensus.cc:740] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860 [term 2 LEARNER]: Becoming Follower/Learner. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Initialized, Role: LEARNER
I20260709 13:19:27.135519 16145 raft_consensus.cc:2393] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 3: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
I20260709 13:19:27.116137 17729 raft_consensus.cc:515] T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:27.135983 17704 consensus_queue.cc:260] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } }
I20260709 13:19:27.096906 17809 raft_consensus.cc:740] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:19:27.140908 17805 consensus_queue.cc:237] T 19baa7bc367844c288bca8f81439c413 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } }
W20260709 13:19:27.149701 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 52331.4us
I20260709 13:19:27.151592 16265 leader_election.cc:304] T e5ad4eb281564936a59f41e387a47156 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:19:27.164126 16145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee9ed67006144797b1de6ea8c2013a68" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:19:27.164889 16145 raft_consensus.cc:2468] T ee9ed67006144797b1de6ea8c2013a68 P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 2.
I20260709 13:19:27.152997 17794 raft_consensus.cc:2804] T e5ad4eb281564936a59f41e387a47156 P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:19:27.175573 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87ac0d1f7e8a419f9bcbc04aee0e067e" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:27.142424 17764 raft_consensus.cc:515] T 8a878999c0024d2abe517484180290fc P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:27.182417 16265 leader_election.cc:304] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:19:27.190860 16824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee9ed67006144797b1de6ea8c2013a68" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:19:27.196990 16824 raft_consensus.cc:2393] T ee9ed67006144797b1de6ea8c2013a68 P cb82cc7bd6db46d980c75237a1402860 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 2: Already voted for candidate cb82cc7bd6db46d980c75237a1402860 in this term.
I20260709 13:19:27.182839 16142 raft_consensus.cc:2368] T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac for earlier term 2. Current term is 3.
I20260709 13:19:27.199462 16151 raft_consensus.cc:1240] T 8a878999c0024d2abe517484180290fc P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 1. Current term is 2. Ops: []
I20260709 13:19:27.176234 16220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5ad4eb281564936a59f41e387a47156" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:19:27.193455 16817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66f75124215f4ca6bef831e174a792d8" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:19:27.180374 17794 raft_consensus.cc:697] T e5ad4eb281564936a59f41e387a47156 P 90f3da57796b488c9087a2f523adf045 [term 3 LEADER]: Becoming Leader. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:19:27.216233 16115 leader_election.cc:304] T 18ad7e3af5504571a69d10c00750a01c P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:19:27.213183 16817 raft_consensus.cc:2468] T 66f75124215f4ca6bef831e174a792d8 P cb82cc7bd6db46d980c75237a1402860 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 2.
W20260709 13:19:27.217818 17813 log.cc:927] Time spent T 8158bbd17b994d2eb96f560bd0f1b88d P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.081s	user 0.001s	sys 0.000s
I20260709 13:19:27.194494 17781 raft_consensus.cc:2749] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:19:27.221318 17452 raft_consensus.cc:2749] T 18ad7e3af5504571a69d10c00750a01c P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:19:27.184803 17722 consensus_queue.cc:1048] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.031s
I20260709 13:19:27.200187 16218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fffab26427e24d398e2695aad40ca134" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:27.230460 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66f75124215f4ca6bef831e174a792d8" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:19:27.231365 16224 raft_consensus.cc:3060] T 66f75124215f4ca6bef831e174a792d8 P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Advancing to term 3
I20260709 13:19:27.251209 16115 leader_election.cc:304] T 02838c86531d4f98bf58d1c22b8baf77 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:19:27.229480 16818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee9ed67006144797b1de6ea8c2013a68" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:19:27.275488 17704 ts_tablet_manager.cc:1434] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860: Time spent starting tablet: real 0.375s	user 0.005s	sys 0.004s
I20260709 13:19:27.192139 17809 consensus_queue.cc:260] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:27.225399 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a96373bb0e14dce8a266b2fc6c58507" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:27.225829 16219 raft_consensus.cc:1275] T af57996a8be84d398f0f023e8d08ef77 P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:19:27.251776 16819 raft_consensus.cc:1217] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860 [term 2 LEARNER]: Deduplicated request from leader. Original: 2.2->[2.3-2.3]   Dedup: 2.3->[]
I20260709 13:19:27.272886 17794 consensus_queue.cc:237] T e5ad4eb281564936a59f41e387a47156 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:27.256879 17452 raft_consensus.cc:3060] T 02838c86531d4f98bf58d1c22b8baf77 P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:19:27.298679 16192 leader_election.cc:304] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:19:27.300555 16224 raft_consensus.cc:2468] T 66f75124215f4ca6bef831e174a792d8 P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 3.
I20260709 13:19:27.319940 16149 raft_consensus.cc:3060] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:19:27.305876 16818 raft_consensus.cc:2468] T ee9ed67006144797b1de6ea8c2013a68 P cb82cc7bd6db46d980c75237a1402860 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 2.
I20260709 13:19:27.320256 17644 raft_consensus.cc:493] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:27.325547 17644 raft_consensus.cc:515] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:27.326510 16265 leader_election.cc:304] T fffab26427e24d398e2695aad40ca134 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:19:27.327641 17644 leader_election.cc:290] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:19:27.327755 17518 raft_consensus.cc:2749] T fffab26427e24d398e2695aad40ca134 P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:19:27.325927 17805 raft_consensus.cc:2804] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:19:27.329767 17805 raft_consensus.cc:697] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [term 3 LEADER]: Becoming Leader. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:19:27.303694 17800 raft_consensus.cc:3055] T 02838c86531d4f98bf58d1c22b8baf77 P 90f3da57796b488c9087a2f523adf045 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:19:27.335047 16117 leader_election.cc:304] T ee9ed67006144797b1de6ea8c2013a68 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac
I20260709 13:19:27.333323 17452 raft_consensus.cc:2749] T 02838c86531d4f98bf58d1c22b8baf77 P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:19:27.330646 17805 consensus_queue.cc:237] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:27.340837 17452 raft_consensus.cc:2804] T ee9ed67006144797b1de6ea8c2013a68 P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:19:27.341274 17452 raft_consensus.cc:493] T ee9ed67006144797b1de6ea8c2013a68 P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:19:27.341775 16791 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:27.341689 17452 raft_consensus.cc:3060] T ee9ed67006144797b1de6ea8c2013a68 P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:19:27.317891 17729 leader_election.cc:290] T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 3 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:19:27.334730 17800 raft_consensus.cc:740] T 02838c86531d4f98bf58d1c22b8baf77 P 90f3da57796b488c9087a2f523adf045 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:19:27.341411 16816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66f75124215f4ca6bef831e174a792d8" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:19:27.367414 16818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62fe3346cd3c42c4a55e14af9073bca4" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:19:27.368197 16818 raft_consensus.cc:2468] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 3.
I20260709 13:19:27.368121 17452 raft_consensus.cc:515] T ee9ed67006144797b1de6ea8c2013a68 P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:27.374413 17452 leader_election.cc:290] T ee9ed67006144797b1de6ea8c2013a68 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 3 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:19:27.380299 17764 leader_election.cc:290] T 8a878999c0024d2abe517484180290fc P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 2 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:27.385357 16149 raft_consensus.cc:1275] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:19:27.387749 16265 leader_election.cc:304] T 5a96373bb0e14dce8a266b2fc6c58507 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:19:27.398844 17518 raft_consensus.cc:2749] T 5a96373bb0e14dce8a266b2fc6c58507 P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:19:27.394690 16117 leader_election.cc:304] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:19:27.397130 16297 raft_consensus.cc:1275] T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:19:27.400197 17452 raft_consensus.cc:2804] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:19:27.400605 17452 raft_consensus.cc:493] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:27.400974 17452 raft_consensus.cc:3060] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:19:27.412179 17452 raft_consensus.cc:515] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:27.397447 16816 raft_consensus.cc:3060] T 66f75124215f4ca6bef831e174a792d8 P cb82cc7bd6db46d980c75237a1402860 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:19:27.419898 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a878999c0024d2abe517484180290fc" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:19:27.431502 16818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee9ed67006144797b1de6ea8c2013a68" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:19:27.431650 17807 consensus_queue.cc:1048] T 74ca8140f35c4ae29e8a664f6bdfc829 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:19:27.432289 16818 raft_consensus.cc:3060] T ee9ed67006144797b1de6ea8c2013a68 P cb82cc7bd6db46d980c75237a1402860 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:19:27.432900 15046 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:19:27.436488 17826 log.cc:927] Time spent T 19baa7bc367844c288bca8f81439c413 P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 0.133s	user 0.001s	sys 0.000s
I20260709 13:19:27.439560 16217 raft_consensus.cc:1240] T 02838c86531d4f98bf58d1c22b8baf77 P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 2. Current term is 3. Ops: []
I20260709 13:19:27.447397 17800 consensus_queue.cc:260] T 02838c86531d4f98bf58d1c22b8baf77 P 90f3da57796b488c9087a2f523adf045 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:27.448896 17800 raft_consensus.cc:3060] T 02838c86531d4f98bf58d1c22b8baf77 P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:19:27.449301 16220 raft_consensus.cc:3055] T e5ad4eb281564936a59f41e387a47156 P 1ca2725d290e47d68b1a4b609c9816ac [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:19:27.449736 16220 raft_consensus.cc:740] T e5ad4eb281564936a59f41e387a47156 P 1ca2725d290e47d68b1a4b609c9816ac [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:19:27.453866 17809 raft_consensus.cc:3060] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:19:27.461009 17452 leader_election.cc:290] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 4 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:19:27.465001 16818 raft_consensus.cc:2468] T ee9ed67006144797b1de6ea8c2013a68 P cb82cc7bd6db46d980c75237a1402860 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 3.
I20260709 13:19:27.471793 16117 leader_election.cc:304] T ee9ed67006144797b1de6ea8c2013a68 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:19:27.472937 17729 raft_consensus.cc:2804] T ee9ed67006144797b1de6ea8c2013a68 P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:19:27.473502 17729 raft_consensus.cc:697] T ee9ed67006144797b1de6ea8c2013a68 P 647a343a542d4a139fe106c19095129b [term 3 LEADER]: Becoming Leader. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:19:27.479923 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a878999c0024d2abe517484180290fc" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:19:27.483836 16218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee9ed67006144797b1de6ea8c2013a68" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:19:27.484607 16218 raft_consensus.cc:3060] T ee9ed67006144797b1de6ea8c2013a68 P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Advancing to term 3
I20260709 13:19:27.450627 16220 consensus_queue.cc:260] T e5ad4eb281564936a59f41e387a47156 P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:27.489223 16220 raft_consensus.cc:3060] T e5ad4eb281564936a59f41e387a47156 P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:19:27.489113 16818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62fe3346cd3c42c4a55e14af9073bca4" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:19:27.487350 16222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62fe3346cd3c42c4a55e14af9073bca4" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:19:27.491402 16116 leader_election.cc:304] T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
I20260709 13:19:27.521164 17830 raft_consensus.cc:2804] T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:19:27.521886 17830 raft_consensus.cc:697] T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b [term 3 LEADER]: Becoming Leader. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:19:27.523067 17830 consensus_queue.cc:237] T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:27.532006 16116 leader_election.cc:304] T 8a878999c0024d2abe517484180290fc P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:19:27.533692 17739 raft_consensus.cc:2749] T 8a878999c0024d2abe517484180290fc P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:19:27.548743 16816 raft_consensus.cc:2468] T 66f75124215f4ca6bef831e174a792d8 P cb82cc7bd6db46d980c75237a1402860 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 3.
I20260709 13:19:27.474471 17729 consensus_queue.cc:237] T ee9ed67006144797b1de6ea8c2013a68 P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:27.572274 17819 raft_consensus.cc:493] T af57996a8be84d398f0f023e8d08ef77 P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:27.573069 17819 raft_consensus.cc:515] T af57996a8be84d398f0f023e8d08ef77 P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:27.581912 16818 raft_consensus.cc:3060] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:19:27.588328 17739 raft_consensus.cc:493] T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:27.588033 17819 leader_election.cc:290] T af57996a8be84d398f0f023e8d08ef77 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:19:27.589679 16822 raft_consensus.cc:2468] T 8c11074579b046d99e585c2818e18a2c P cb82cc7bd6db46d980c75237a1402860 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 3.
I20260709 13:19:27.588887 17739 raft_consensus.cc:515] T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:27.593698 16192 leader_election.cc:304] T 8c11074579b046d99e585c2818e18a2c P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 90f3da57796b488c9087a2f523adf045
I20260709 13:19:27.594455 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4f65cdf00b748bb963ee6e81ef43b79" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:27.594995 17819 raft_consensus.cc:2804] T 8c11074579b046d99e585c2818e18a2c P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:19:27.595191 16293 raft_consensus.cc:2468] T c4f65cdf00b748bb963ee6e81ef43b79 P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 3.
I20260709 13:19:27.595824 17819 raft_consensus.cc:697] T 8c11074579b046d99e585c2818e18a2c P 1ca2725d290e47d68b1a4b609c9816ac [term 3 LEADER]: Becoming Leader. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:19:27.596688 16115 leader_election.cc:304] T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:19:27.596813 17819 consensus_queue.cc:237] T 8c11074579b046d99e585c2818e18a2c P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:27.597586 17452 raft_consensus.cc:2804] T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:19:27.597924 17739 leader_election.cc:290] T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:27.598196 17452 raft_consensus.cc:493] T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:27.492583 16219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62fe3346cd3c42c4a55e14af9073bca4" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
W20260709 13:19:27.425971 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 251767us
I20260709 13:19:27.598851 17452 raft_consensus.cc:3060] T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:19:27.600231 16222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4f65cdf00b748bb963ee6e81ef43b79" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:27.621143 17452 raft_consensus.cc:515] T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:27.632155 16191 leader_election.cc:304] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 90f3da57796b488c9087a2f523adf045
I20260709 13:19:27.634287 17452 leader_election.cc:290] T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 4 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:27.634805 16191 leader_election.cc:304] T ee9ed67006144797b1de6ea8c2013a68 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: cb82cc7bd6db46d980c75237a1402860
I20260709 13:19:27.640137 16222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4f65cdf00b748bb963ee6e81ef43b79" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:19:27.619676 17781 raft_consensus.cc:493] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:27.631388 16816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af57996a8be84d398f0f023e8d08ef77" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:19:27.640341 17815 raft_consensus.cc:2804] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:19:27.661279 17758 consensus_queue.cc:1059] T 8a878999c0024d2abe517484180290fc P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:19:27.663031 17718 raft_consensus.cc:3055] T 8a878999c0024d2abe517484180290fc P 1ca2725d290e47d68b1a4b609c9816ac [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:19:27.663600 17718 raft_consensus.cc:740] T 8a878999c0024d2abe517484180290fc P 1ca2725d290e47d68b1a4b609c9816ac [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:19:27.664929 17718 consensus_queue.cc:260] T 8a878999c0024d2abe517484180290fc P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:27.670324 16191 leader_election.cc:400] T 66f75124215f4ca6bef831e174a792d8 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 2 pre-election: Vote denied by peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741) with higher term. Message: Invalid argument: T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac for earlier term 2. Current term is 3.
I20260709 13:19:27.672775 16191 leader_election.cc:403] T 66f75124215f4ca6bef831e174a792d8 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 2 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:19:27.676055 17760 raft_consensus.cc:2749] T 66f75124215f4ca6bef831e174a792d8 P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Leader pre-election lost for term 2. Reason: Vote denied by peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741) with higher term. Message: Invalid argument: T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac for earlier term 2. Current term is 3.
I20260709 13:19:27.641003 17638 raft_consensus.cc:493] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:19:27.680725 17638 raft_consensus.cc:515] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:27.685602 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4f65cdf00b748bb963ee6e81ef43b79" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:19:27.686338 16293 raft_consensus.cc:3060] T c4f65cdf00b748bb963ee6e81ef43b79 P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:19:27.686407 16822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2372981006d4b218215c28964dea623" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:19:27.687311 17831 consensus_queue.cc:1048] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.047s
I20260709 13:19:27.689160 16818 raft_consensus.cc:2468] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 4.
I20260709 13:19:27.690428 16117 leader_election.cc:304] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac
I20260709 13:19:27.696785 17452 raft_consensus.cc:2804] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:19:27.697348 17452 raft_consensus.cc:697] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [term 4 LEADER]: Becoming Leader. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:19:27.683022 17638 leader_election.cc:290] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:19:27.698191 17452 consensus_queue.cc:237] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:19:27.703083 16192 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:27.702160 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 52704.1us
I20260709 13:19:27.651906 17695 raft_consensus.cc:493] T 3a7ca61592e848ea9b0fb81a2ce33cdc P cb82cc7bd6db46d980c75237a1402860 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:27.721064 17695 raft_consensus.cc:515] T 3a7ca61592e848ea9b0fb81a2ce33cdc P cb82cc7bd6db46d980c75237a1402860 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:27.640559 17673 consensus_queue.cc:1048] T c2f5f89186204242bf377f7c42b2e2da P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.062s
W20260709 13:19:27.732092 16190 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.552s	user 0.020s	sys 0.026s
W20260709 13:19:27.828183 16190 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.648s	user 0.020s	sys 0.026s
I20260709 13:19:27.743403 17833 consensus_queue.cc:1048] T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.013s
I20260709 13:19:27.728232 16046 catalog_manager.cc:5697] T 74ca8140f35c4ae29e8a664f6bdfc829 P cb82cc7bd6db46d980c75237a1402860 reported cstate change: term changed from 1 to 2, leader changed from 90f3da57796b488c9087a2f523adf045 (127.14.177.131) to cb82cc7bd6db46d980c75237a1402860 (127.14.177.132). New cstate: current_term: 2 leader_uuid: "cb82cc7bd6db46d980c75237a1402860" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:19:27.656966 17781 raft_consensus.cc:515] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:27.732688 16218 raft_consensus.cc:2468] T ee9ed67006144797b1de6ea8c2013a68 P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 3.
I20260709 13:19:27.728086 17815 raft_consensus.cc:697] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac [term 2 LEADER]: Becoming Leader. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
W20260709 13:19:27.836098 16190 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:27.838488 17645 raft_consensus.cc:493] T 8638ee363273426db1393dd81e5c7b72 P cb82cc7bd6db46d980c75237a1402860 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:27.852384 16144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af57996a8be84d398f0f023e8d08ef77" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:19:27.856469 16298 raft_consensus.cc:1275] T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:19:27.868317 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7ca61592e848ea9b0fb81a2ce33cdc" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
W20260709 13:19:27.875927 16189 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:27.883150 16298 raft_consensus.cc:2468] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 2.
I20260709 13:19:27.889986 16295 raft_consensus.cc:1240] T e5ad4eb281564936a59f41e387a47156 P 90f3da57796b488c9087a2f523adf045 [term 3 LEADER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 2. Current term is 3. Ops: []
I20260709 13:19:27.888881 16816 raft_consensus.cc:1240] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [term 4 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 3. Current term is 4. Ops: []
I20260709 13:19:27.892429 17815 consensus_queue.cc:237] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } }
I20260709 13:19:27.894074 17827 raft_consensus.cc:2764] T ee9ed67006144797b1de6ea8c2013a68 P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Leader pre-election decision vote started in defunct term 1: won
I20260709 13:19:27.890863 16789 leader_election.cc:304] T 3a7ca61592e848ea9b0fb81a2ce33cdc P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:19:27.877334 17721 consensus_queue.cc:1048] T af57996a8be84d398f0f023e8d08ef77 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.018s
I20260709 13:19:27.850929 17830 raft_consensus.cc:493] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:27.859496 17803 raft_consensus.cc:493] T d2ab0ac8347641e6955e9bce7d90c1a2 P cb82cc7bd6db46d980c75237a1402860 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:27.861196 17695 leader_election.cc:290] T 3a7ca61592e848ea9b0fb81a2ce33cdc P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
W20260709 13:19:27.908149 17832 log.cc:927] Time spent T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 0.068s	user 0.001s	sys 0.000s
I20260709 13:19:27.908622 17804 raft_consensus.cc:493] T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:27.909090 16142 raft_consensus.cc:1240] T e5ad4eb281564936a59f41e387a47156 P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 2. Current term is 3. Ops: []
I20260709 13:19:27.915874 16143 raft_consensus.cc:1275] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:19:27.919716 17710 raft_consensus.cc:2804] T 3a7ca61592e848ea9b0fb81a2ce33cdc P cb82cc7bd6db46d980c75237a1402860 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:19:27.924100 17718 raft_consensus.cc:3060] T 8a878999c0024d2abe517484180290fc P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:19:27.972813 17710 raft_consensus.cc:493] T 3a7ca61592e848ea9b0fb81a2ce33cdc P cb82cc7bd6db46d980c75237a1402860 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:27.976477 17710 raft_consensus.cc:3060] T 3a7ca61592e848ea9b0fb81a2ce33cdc P cb82cc7bd6db46d980c75237a1402860 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:19:27.847126 17645 raft_consensus.cc:515] T 8638ee363273426db1393dd81e5c7b72 P cb82cc7bd6db46d980c75237a1402860 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:19:27.986642 16189 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:27.964442 17695 raft_consensus.cc:493] T b87147731a68400e8cce9b879daa7c5d P cb82cc7bd6db46d980c75237a1402860 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:27.991359 16192 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 94556.6us
I20260709 13:19:27.927513 17830 raft_consensus.cc:515] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:27.929828 17803 raft_consensus.cc:515] T d2ab0ac8347641e6955e9bce7d90c1a2 P cb82cc7bd6db46d980c75237a1402860 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:27.969058 17760 consensus_queue.cc:1059] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.026s
I20260709 13:19:27.971572 17804 raft_consensus.cc:515] T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:27.997630 17819 raft_consensus.cc:493] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:27.996886 17695 raft_consensus.cc:515] T b87147731a68400e8cce9b879daa7c5d P cb82cc7bd6db46d980c75237a1402860 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:28.011152 17800 raft_consensus.cc:493] T b87147731a68400e8cce9b879daa7c5d P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:28.024808 17721 raft_consensus.cc:493] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:19:28.025390 17721 raft_consensus.cc:515] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:28.031437 16218 raft_consensus.cc:1275] T ee9ed67006144797b1de6ea8c2013a68 P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:19:28.032159 17645 leader_election.cc:290] T 8638ee363273426db1393dd81e5c7b72 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:28.033339 17729 consensus_queue.cc:1048] T ee9ed67006144797b1de6ea8c2013a68 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:19:28.025657 17781 leader_election.cc:290] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:19:28.040560 16222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f5f89186204242bf377f7c42b2e2da" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:28.041647 16145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746aabf856ab401c9ef5e859371e7b67" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:19:28.020061 16293 raft_consensus.cc:2468] T c4f65cdf00b748bb963ee6e81ef43b79 P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 4.
I20260709 13:19:28.053084 16300 raft_consensus.cc:1240] T c4f65cdf00b748bb963ee6e81ef43b79 P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 3. Current term is 4. Ops: [3.3-3.3]
I20260709 13:19:28.044329 17794 raft_consensus.cc:493] T 68a6838d82ef4cac83507f906139d90f P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:28.053787 17794 raft_consensus.cc:515] T 68a6838d82ef4cac83507f906139d90f P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:28.056084 17794 leader_election.cc:290] T 68a6838d82ef4cac83507f906139d90f P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:19:28.044305 17835 raft_consensus.cc:3055] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:19:28.057899 17835 raft_consensus.cc:740] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:19:28.058279 16222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68a6838d82ef4cac83507f906139d90f" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:28.059223 16222 raft_consensus.cc:2468] T 68a6838d82ef4cac83507f906139d90f P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 3.
I20260709 13:19:28.058987 17835 consensus_queue.cc:260] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:28.060616 17835 raft_consensus.cc:3060] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:19:28.060583 16265 leader_election.cc:304] T 68a6838d82ef4cac83507f906139d90f P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:19:28.061829 17794 raft_consensus.cc:2804] T 68a6838d82ef4cac83507f906139d90f P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:19:28.062247 17794 raft_consensus.cc:493] T 68a6838d82ef4cac83507f906139d90f P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:28.062623 17794 raft_consensus.cc:3060] T 68a6838d82ef4cac83507f906139d90f P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:19:28.064397 17804 leader_election.cc:290] T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:28.067075 17695 leader_election.cc:290] T b87147731a68400e8cce9b879daa7c5d P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:28.068482 17803 leader_election.cc:290] T d2ab0ac8347641e6955e9bce7d90c1a2 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:28.016197 17819 raft_consensus.cc:515] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:28.065927 16048 catalog_manager.cc:5697] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 1ca2725d290e47d68b1a4b609c9816ac reported cstate change: term changed from 2 to 3, leader changed from 647a343a542d4a139fe106c19095129b (127.14.177.129) to 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130). New cstate: current_term: 3 leader_uuid: "1ca2725d290e47d68b1a4b609c9816ac" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } health_report { overall_health: UNKNOWN } } }
W20260709 13:19:28.075969 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 68507.7us
I20260709 13:19:28.047190 17723 raft_consensus.cc:493] T b87147731a68400e8cce9b879daa7c5d P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:28.044100 17800 raft_consensus.cc:515] T b87147731a68400e8cce9b879daa7c5d P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:28.077118 16294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8638ee363273426db1393dd81e5c7b72" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:28.087637 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f5f89186204242bf377f7c42b2e2da" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:28.088338 16294 raft_consensus.cc:2468] T 8638ee363273426db1393dd81e5c7b72 P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 2.
W20260709 13:19:28.090178 16220 consensus_meta.cc:287] T e5ad4eb281564936a59f41e387a47156 P 1ca2725d290e47d68b1a4b609c9816ac: Time spent flushing consensus metadata: real 0.600s	user 0.008s	sys 0.004s
I20260709 13:19:28.090747 16220 raft_consensus.cc:2468] T e5ad4eb281564936a59f41e387a47156 P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 3.
I20260709 13:19:28.091439 16143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68a6838d82ef4cac83507f906139d90f" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:19:28.092118 16143 raft_consensus.cc:2468] T 68a6838d82ef4cac83507f906139d90f P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 3.
I20260709 13:19:28.092973 16144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87147731a68400e8cce9b879daa7c5d" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:19:28.093817 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f5f89186204242bf377f7c42b2e2da" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:19:28.094384 16150 raft_consensus.cc:2468] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 3.
I20260709 13:19:28.082060 17723 raft_consensus.cc:515] T b87147731a68400e8cce9b879daa7c5d P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:28.098404 17723 leader_election.cc:290] T b87147731a68400e8cce9b879daa7c5d P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:19:28.098970 16294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87147731a68400e8cce9b879daa7c5d" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:28.098569 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87147731a68400e8cce9b879daa7c5d" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:28.102201 16144 raft_consensus.cc:2468] T b87147731a68400e8cce9b879daa7c5d P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 2.
I20260709 13:19:28.108268 16145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87147731a68400e8cce9b879daa7c5d" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:19:28.102945 16789 leader_election.cc:304] T 8638ee363273426db1393dd81e5c7b72 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:19:28.104562 16265 leader_election.cc:304] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac
I20260709 13:19:28.099715 16294 raft_consensus.cc:2468] T b87147731a68400e8cce9b879daa7c5d P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 2.
I20260709 13:19:28.116076 17781 raft_consensus.cc:2804] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:19:28.116511 17781 raft_consensus.cc:493] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:19:28.116914 17781 raft_consensus.cc:3060] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:19:28.083190 17830 leader_election.cc:290] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:28.080371 17794 raft_consensus.cc:515] T 68a6838d82ef4cac83507f906139d90f P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:28.085851 17800 leader_election.cc:290] T b87147731a68400e8cce9b879daa7c5d P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:19:28.121642 17794 leader_election.cc:290] T 68a6838d82ef4cac83507f906139d90f P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 4 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:19:28.122298 17695 raft_consensus.cc:2804] T 8638ee363273426db1393dd81e5c7b72 P cb82cc7bd6db46d980c75237a1402860 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:19:28.122745 17695 raft_consensus.cc:493] T 8638ee363273426db1393dd81e5c7b72 P cb82cc7bd6db46d980c75237a1402860 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:28.123119 17695 raft_consensus.cc:3060] T 8638ee363273426db1393dd81e5c7b72 P cb82cc7bd6db46d980c75237a1402860 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:19:28.125308 16151 raft_consensus.cc:1275] T e5ad4eb281564936a59f41e387a47156 P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:19:28.125847 17703 tablet_bootstrap.cc:492] T 19baa7bc367844c288bca8f81439c413 P cb82cc7bd6db46d980c75237a1402860: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:19:28.126834 17703 tablet_bootstrap.cc:492] T 19baa7bc367844c288bca8f81439c413 P cb82cc7bd6db46d980c75237a1402860: Bootstrap complete.
I20260709 13:19:28.127038 16295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a378d7f796f94615b3b347119a9880fd" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:28.127785 16295 raft_consensus.cc:2468] T a378d7f796f94615b3b347119a9880fd P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 2.
I20260709 13:19:28.132836 16115 leader_election.cc:304] T b87147731a68400e8cce9b879daa7c5d P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:19:28.133307 16822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87147731a68400e8cce9b879daa7c5d" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:19:28.134092 16822 raft_consensus.cc:2468] T b87147731a68400e8cce9b879daa7c5d P cb82cc7bd6db46d980c75237a1402860 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 2.
I20260709 13:19:28.136175 17721 leader_election.cc:290] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:19:28.119069 16296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ab0ac8347641e6955e9bce7d90c1a2" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:28.100970 17819 leader_election.cc:290] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:19:28.136989 16296 raft_consensus.cc:2468] T d2ab0ac8347641e6955e9bce7d90c1a2 P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 2.
I20260709 13:19:28.136399 17723 raft_consensus.cc:2804] T b87147731a68400e8cce9b879daa7c5d P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:19:28.137946 17723 raft_consensus.cc:493] T b87147731a68400e8cce9b879daa7c5d P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:28.138399 17723 raft_consensus.cc:3060] T b87147731a68400e8cce9b879daa7c5d P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:19:28.141983 16115 leader_election.cc:304] T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac
I20260709 13:19:28.144186 16265 leader_election.cc:304] T b87147731a68400e8cce9b879daa7c5d P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:19:28.144548 16789 leader_election.cc:304] T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:19:28.135967 16144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68a6838d82ef4cac83507f906139d90f" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:19:28.146124 16144 raft_consensus.cc:3060] T 68a6838d82ef4cac83507f906139d90f P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:19:28.146720 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 67912.4us
I20260709 13:19:28.147622 16222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68a6838d82ef4cac83507f906139d90f" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:19:28.148123 16222 raft_consensus.cc:3060] T 68a6838d82ef4cac83507f906139d90f P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:19:28.151549 16822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87147731a68400e8cce9b879daa7c5d" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:19:28.152109 16822 raft_consensus.cc:2468] T b87147731a68400e8cce9b879daa7c5d P cb82cc7bd6db46d980c75237a1402860 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 2.
I20260709 13:19:28.153990 17518 raft_consensus.cc:2804] T b87147731a68400e8cce9b879daa7c5d P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:19:28.154505 17518 raft_consensus.cc:493] T b87147731a68400e8cce9b879daa7c5d P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:28.154896 17518 raft_consensus.cc:3060] T b87147731a68400e8cce9b879daa7c5d P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:19:28.158056 16220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f5f89186204242bf377f7c42b2e2da" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:28.159548 16816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a378d7f796f94615b3b347119a9880fd" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:19:28.160281 16816 raft_consensus.cc:2468] T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 2.
I20260709 13:19:28.162341 16116 leader_election.cc:304] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:19:28.163479 17802 raft_consensus.cc:2749] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:19:28.163818 17710 raft_consensus.cc:515] T 3a7ca61592e848ea9b0fb81a2ce33cdc P cb82cc7bd6db46d980c75237a1402860 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:28.164633 17829 raft_consensus.cc:2804] T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:19:28.164356 17800 consensus_queue.cc:1048] T e5ad4eb281564936a59f41e387a47156 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:19:28.165027 17829 raft_consensus.cc:493] T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:28.165406 17829 raft_consensus.cc:3060] T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:19:28.165949 17800 raft_consensus.cc:493] T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:28.166121 17710 leader_election.cc:290] T 3a7ca61592e848ea9b0fb81a2ce33cdc P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 4 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:28.167412 16192 leader_election.cc:304] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:19:28.169548 16789 leader_election.cc:304] T d2ab0ac8347641e6955e9bce7d90c1a2 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:19:28.171296 16790 leader_election.cc:304] T b87147731a68400e8cce9b879daa7c5d P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:19:28.174525 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7ca61592e848ea9b0fb81a2ce33cdc" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:19:28.175895 17452 raft_consensus.cc:2804] T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:19:28.176347 17452 raft_consensus.cc:697] T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b [term 4 LEADER]: Becoming Leader. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:19:28.175482 16297 raft_consensus.cc:3060] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Advancing to term 4
I20260709 13:19:28.180680 17518 raft_consensus.cc:515] T b87147731a68400e8cce9b879daa7c5d P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:28.178929 16822 raft_consensus.cc:1275] T 66f75124215f4ca6bef831e174a792d8 P cb82cc7bd6db46d980c75237a1402860 [term 3 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:19:28.178337 17738 raft_consensus.cc:2804] T d2ab0ac8347641e6955e9bce7d90c1a2 P cb82cc7bd6db46d980c75237a1402860 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:19:28.183722 17847 raft_consensus.cc:2804] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:19:28.181348 17787 raft_consensus.cc:2749] T b87147731a68400e8cce9b879daa7c5d P cb82cc7bd6db46d980c75237a1402860 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:19:28.186627 17827 consensus_queue.cc:1059] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:19:28.181552 17672 consensus_queue.cc:1048] T b6f07842d3aa47858df5665874c86f3a P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.006s
I20260709 13:19:28.191557 16295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a378d7f796f94615b3b347119a9880fd" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:28.180387 17452 consensus_queue.cc:237] T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:28.192683 16816 raft_consensus.cc:1275] T 8c11074579b046d99e585c2818e18a2c P cb82cc7bd6db46d980c75237a1402860 [term 3 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:19:28.166452 17800 raft_consensus.cc:515] T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:28.216423 16292 raft_consensus.cc:1275] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:19:28.214480 17644 consensus_queue.cc:1048] T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.003s
I20260709 13:19:28.216423 16819 raft_consensus.cc:1275] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [term 4 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:19:28.217979 17800 leader_election.cc:290] T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:19:28.191094 17695 raft_consensus.cc:515] T 8638ee363273426db1393dd81e5c7b72 P cb82cc7bd6db46d980c75237a1402860 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:28.177443 16222 raft_consensus.cc:2468] T 68a6838d82ef4cac83507f906139d90f P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 4.
I20260709 13:19:28.171522 17703 ts_tablet_manager.cc:1403] T 19baa7bc367844c288bca8f81439c413 P cb82cc7bd6db46d980c75237a1402860: Time spent bootstrapping tablet: real 1.084s	user 0.040s	sys 0.000s
I20260709 13:19:28.204449 17518 leader_election.cc:290] T b87147731a68400e8cce9b879daa7c5d P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 3 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:19:28.205848 16822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87147731a68400e8cce9b879daa7c5d" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:19:28.168061 17638 raft_consensus.cc:493] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:19:28.214936 16295 raft_consensus.cc:2468] T a378d7f796f94615b3b347119a9880fd P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 2.
I20260709 13:19:28.209990 17738 raft_consensus.cc:493] T d2ab0ac8347641e6955e9bce7d90c1a2 P cb82cc7bd6db46d980c75237a1402860 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:19:28.267037 16189 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:28.285336 17518 raft_consensus.cc:493] T c9944d5e4b8c4109af9955c4f08c9095 P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:19:28.288021 16822 raft_consensus.cc:3060] T b87147731a68400e8cce9b879daa7c5d P cb82cc7bd6db46d980c75237a1402860 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:19:28.289179 17518 raft_consensus.cc:515] T c9944d5e4b8c4109af9955c4f08c9095 P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:28.211423 17847 raft_consensus.cc:493] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:28.226326 17762 raft_consensus.cc:493] T 02838c86531d4f98bf58d1c22b8baf77 P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:28.306587 16822 raft_consensus.cc:2468] T b87147731a68400e8cce9b879daa7c5d P cb82cc7bd6db46d980c75237a1402860 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 3.
I20260709 13:19:28.314567 17518 leader_election.cc:290] T c9944d5e4b8c4109af9955c4f08c9095 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:19:28.286028 17738 raft_consensus.cc:3060] T d2ab0ac8347641e6955e9bce7d90c1a2 P cb82cc7bd6db46d980c75237a1402860 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:19:28.318231 16297 raft_consensus.cc:2468] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 4.
I20260709 13:19:28.318343 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87147731a68400e8cce9b879daa7c5d" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:19:28.318866 16151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c61a39fd4f84424b129c1a6f9ba404d" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:19:28.322664 16265 leader_election.cc:304] T 68a6838d82ef4cac83507f906139d90f P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:19:28.226552 17644 consensus_queue.cc:1048] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.091s
I20260709 13:19:28.292132 17758 raft_consensus.cc:3055] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:19:28.293807 17847 raft_consensus.cc:3060] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:19:28.332875 16222 raft_consensus.cc:1275] T e5ad4eb281564936a59f41e387a47156 P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:19:28.292127 17827 raft_consensus.cc:493] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:28.336640 16151 raft_consensus.cc:2468] T 9c61a39fd4f84424b129c1a6f9ba404d P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 2.
W20260709 13:19:28.353824 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 129842us
I20260709 13:19:28.283511 17638 raft_consensus.cc:515] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:19:28.376808 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 54261.7us
I20260709 13:19:28.377153 17518 raft_consensus.cc:2804] T 68a6838d82ef4cac83507f906139d90f P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:19:28.317679 17762 raft_consensus.cc:515] T 02838c86531d4f98bf58d1c22b8baf77 P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:28.374884 17758 raft_consensus.cc:740] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:19:28.419534 17518 raft_consensus.cc:697] T 68a6838d82ef4cac83507f906139d90f P 90f3da57796b488c9087a2f523adf045 [term 4 LEADER]: Becoming Leader. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:19:28.423563 17518 consensus_queue.cc:237] T 68a6838d82ef4cac83507f906139d90f P 90f3da57796b488c9087a2f523adf045 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:28.424630 16297 raft_consensus.cc:3060] T 8c11074579b046d99e585c2818e18a2c P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:19:28.391652 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9944d5e4b8c4109af9955c4f08c9095" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:28.354257 17723 raft_consensus.cc:515] T b87147731a68400e8cce9b879daa7c5d P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:28.428259 17864 raft_consensus.cc:493] T b6f07842d3aa47858df5665874c86f3a P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:28.432432 17850 log.cc:927] Time spent T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 0.217s	user 0.001s	sys 0.000s
I20260709 13:19:28.432845 17835 consensus_queue.cc:1048] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:19:28.408267 17830 raft_consensus.cc:493] T 8d4ab870a05347868956aa8b841b2981 P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:28.420333 16219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c61a39fd4f84424b129c1a6f9ba404d" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:28.458931 16219 raft_consensus.cc:2468] T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 2.
I20260709 13:19:28.391424 17827 raft_consensus.cc:515] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:28.465502 16150 raft_consensus.cc:2393] T b87147731a68400e8cce9b879daa7c5d P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 3: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
I20260709 13:19:28.421990 17794 raft_consensus.cc:493] T 5a959dbecc2648eb85d1cc550f5a3245 P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:19:28.422551 17800 raft_consensus.cc:493] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:19:28.457682 17830 raft_consensus.cc:515] T 8d4ab870a05347868956aa8b841b2981 P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:28.461203 16297 raft_consensus.cc:1275] T 8c11074579b046d99e585c2818e18a2c P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:19:28.432768 17729 raft_consensus.cc:493] T 5a96373bb0e14dce8a266b2fc6c58507 P 647a343a542d4a139fe106c19095129b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:19:28.391898 17781 raft_consensus.cc:515] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:28.470358 17729 raft_consensus.cc:515] T 5a96373bb0e14dce8a266b2fc6c58507 P 647a343a542d4a139fe106c19095129b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:28.487413 16296 raft_consensus.cc:1240] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 3. Current term is 4. Ops: []
I20260709 13:19:28.488006 16144 raft_consensus.cc:2468] T 68a6838d82ef4cac83507f906139d90f P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 4.
I20260709 13:19:28.468562 17794 raft_consensus.cc:515] T 5a959dbecc2648eb85d1cc550f5a3245 P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:28.503320 17762 leader_election.cc:290] T 02838c86531d4f98bf58d1c22b8baf77 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:28.505101 17794 leader_election.cc:290] T 5a959dbecc2648eb85d1cc550f5a3245 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
W20260709 13:19:28.492185 17853 log.cc:927] Time spent T 00000000000000000000000000000000 P 667a34b27a39474ebefc1af3cf5b4271: Append to log took a long time: real 0.065s	user 0.000s	sys 0.001s
I20260709 13:19:28.508298 17723 leader_election.cc:290] T b87147731a68400e8cce9b879daa7c5d P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 3 election: Requested vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:19:28.492712 17703 raft_consensus.cc:359] T 19baa7bc367844c288bca8f81439c413 P cb82cc7bd6db46d980c75237a1402860 [term 2 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } }
I20260709 13:19:28.509061 17703 raft_consensus.cc:740] T 19baa7bc367844c288bca8f81439c413 P cb82cc7bd6db46d980c75237a1402860 [term 2 LEARNER]: Becoming Follower/Learner. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Initialized, Role: LEARNER
I20260709 13:19:28.509093 16220 raft_consensus.cc:1275] T 66f75124215f4ca6bef831e174a792d8 P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:19:28.510140 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8638ee363273426db1393dd81e5c7b72" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:19:28.509840 17703 consensus_queue.cc:260] T 19baa7bc367844c288bca8f81439c413 P cb82cc7bd6db46d980c75237a1402860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } }
I20260709 13:19:28.510677 16297 raft_consensus.cc:3060] T 8638ee363273426db1393dd81e5c7b72 P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Advancing to term 4
I20260709 13:19:28.493785 17781 leader_election.cc:290] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 4 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:19:28.499949 17829 raft_consensus.cc:515] T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:28.501799 17695 leader_election.cc:290] T 8638ee363273426db1393dd81e5c7b72 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 4 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:28.511775 17695 raft_consensus.cc:493] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:28.509155 16789 leader_election.cc:304] T 3a7ca61592e848ea9b0fb81a2ce33cdc P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:19:28.512274 17695 raft_consensus.cc:515] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:28.519716 16816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9944d5e4b8c4109af9955c4f08c9095" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:19:28.520933 17787 raft_consensus.cc:2804] T 3a7ca61592e848ea9b0fb81a2ce33cdc P cb82cc7bd6db46d980c75237a1402860 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:19:28.521638 17695 leader_election.cc:290] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:28.470769 16291 raft_consensus.cc:1275] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 13:19:28.467396 17758 consensus_queue.cc:260] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:28.523020 17787 raft_consensus.cc:697] T 3a7ca61592e848ea9b0fb81a2ce33cdc P cb82cc7bd6db46d980c75237a1402860 [term 4 LEADER]: Becoming Leader. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:19:28.511868 17638 leader_election.cc:290] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:19:28.443192 17864 raft_consensus.cc:515] T b6f07842d3aa47858df5665874c86f3a P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:28.523943 17787 consensus_queue.cc:237] T 3a7ca61592e848ea9b0fb81a2ce33cdc P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:19:28.524814 17851 log.cc:927] Time spent T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 0.204s	user 0.002s	sys 0.000s
I20260709 13:19:28.525578 17864 leader_election.cc:290] T b6f07842d3aa47858df5665874c86f3a P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:28.527681 16297 raft_consensus.cc:2468] T 8638ee363273426db1393dd81e5c7b72 P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 4.
I20260709 13:19:28.530908 16296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66ae176961b74813a58bc15c9c4c6f2a" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:28.523464 17758 raft_consensus.cc:3060] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:19:28.531129 16266 leader_election.cc:304] T b87147731a68400e8cce9b879daa7c5d P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:19:28.537242 17781 raft_consensus.cc:2804] T b87147731a68400e8cce9b879daa7c5d P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:19:28.537810 17781 raft_consensus.cc:697] T b87147731a68400e8cce9b879daa7c5d P 90f3da57796b488c9087a2f523adf045 [term 3 LEADER]: Becoming Leader. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:19:28.535871 16789 leader_election.cc:304] T 8638ee363273426db1393dd81e5c7b72 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:19:28.473408 17800 raft_consensus.cc:515] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } }
I20260709 13:19:28.548486 17804 raft_consensus.cc:2804] T 8638ee363273426db1393dd81e5c7b72 P cb82cc7bd6db46d980c75237a1402860 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:19:28.538713 17781 consensus_queue.cc:237] T b87147731a68400e8cce9b879daa7c5d P 90f3da57796b488c9087a2f523adf045 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:19:28.550451 17854 log.cc:927] Time spent T ee9ed67006144797b1de6ea8c2013a68 P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 0.142s	user 0.001s	sys 0.000s
I20260709 13:19:28.538683 17847 raft_consensus.cc:515] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:28.574903 16047 catalog_manager.cc:5697] T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "647a343a542d4a139fe106c19095129b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:19:28.579473 16047 catalog_manager.cc:5697] T ee9ed67006144797b1de6ea8c2013a68 P 647a343a542d4a139fe106c19095129b reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "647a343a542d4a139fe106c19095129b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:19:28.585649 16822 raft_consensus.cc:1275] T ee9ed67006144797b1de6ea8c2013a68 P cb82cc7bd6db46d980c75237a1402860 [term 3 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:19:28.590687 16222 raft_consensus.cc:3060] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:19:28.590806 17819 raft_consensus.cc:493] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:19:28.591812 16217 raft_consensus.cc:1240] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 2. Current term is 3. Ops: []
I20260709 13:19:28.591542 17819 raft_consensus.cc:515] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:28.593570 17819 leader_election.cc:290] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:19:28.601316 16222 raft_consensus.cc:1275] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:19:28.610524 16265 leader_election.cc:304] T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
W20260709 13:19:28.620663 16189 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:28.602236 17864 raft_consensus.cc:493] T 5a959dbecc2648eb85d1cc550f5a3245 P 647a343a542d4a139fe106c19095129b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:19:28.633752 16222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7ca61592e848ea9b0fb81a2ce33cdc" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:28.638501 16222 raft_consensus.cc:2468] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 3.
I20260709 13:19:28.631790 17855 raft_consensus.cc:493] T 8c11074579b046d99e585c2818e18a2c P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:28.617635 17794 raft_consensus.cc:2804] T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:19:28.644366 16222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a96373bb0e14dce8a266b2fc6c58507" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:28.682287 16822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87147731a68400e8cce9b879daa7c5d" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:19:28.683231 16822 raft_consensus.cc:2393] T b87147731a68400e8cce9b879daa7c5d P cb82cc7bd6db46d980c75237a1402860 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 3: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
I20260709 13:19:28.685030 16218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6f07842d3aa47858df5665874c86f3a" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:28.685259 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d4ab870a05347868956aa8b841b2981" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:28.685883 16224 raft_consensus.cc:2468] T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 2.
I20260709 13:19:28.630399 17830 leader_election.cc:290] T 8d4ab870a05347868956aa8b841b2981 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:28.640795 17794 raft_consensus.cc:493] T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:28.695817 17863 raft_consensus.cc:493] T 7c2095a5f4ad49408dbc7f69c606120d P 647a343a542d4a139fe106c19095129b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:19:28.695922 17781 raft_consensus.cc:493] T 66ae176961b74813a58bc15c9c4c6f2a P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:19:28.696568 17863 raft_consensus.cc:515] T 7c2095a5f4ad49408dbc7f69c606120d P 647a343a542d4a139fe106c19095129b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:28.696736 17781 raft_consensus.cc:515] T 66ae176961b74813a58bc15c9c4c6f2a P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:28.708830 16822 raft_consensus.cc:3060] T 19baa7bc367844c288bca8f81439c413 P cb82cc7bd6db46d980c75237a1402860 [term 2 LEARNER]: Advancing to term 3
I20260709 13:19:28.698912 17863 leader_election.cc:290] T 7c2095a5f4ad49408dbc7f69c606120d P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:28.643873 17827 leader_election.cc:290] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:19:28.641027 17864 raft_consensus.cc:515] T 5a959dbecc2648eb85d1cc550f5a3245 P 647a343a542d4a139fe106c19095129b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:28.643438 17855 raft_consensus.cc:515] T 8c11074579b046d99e585c2818e18a2c P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:28.977624 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6f07842d3aa47858df5665874c86f3a" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:28.929677 16148 raft_consensus.cc:1275] T b6f07842d3aa47858df5665874c86f3a P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:19:28.779805 16825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5718c623ebee4d2bb770f0f9ddb2f164" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:19:28.803519 16223 raft_consensus.cc:1275] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 6. (term mismatch)
W20260709 13:19:28.851248 16190 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:28.803740 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c2095a5f4ad49408dbc7f69c606120d" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
W20260709 13:19:28.895550 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 292575us
I20260709 13:19:28.802423 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ab0ac8347641e6955e9bce7d90c1a2" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:28.884500 16145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2372981006d4b218215c28964dea623" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
W20260709 13:19:28.896618 16189 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:28.886353 16150 raft_consensus.cc:1240] T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b [term 4 LEADER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 3. Current term is 4. Ops: [3.3-3.3]
I20260709 13:19:28.748843 17452 raft_consensus.cc:493] T ecc1e363230043fe9fef390f0b979f20 P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:19:28.707393 16236 tablet_copy_service.cc:342] P 1ca2725d290e47d68b1a4b609c9816ac: Request end of tablet copy session cb82cc7bd6db46d980c75237a1402860-ecc1e363230043fe9fef390f0b979f20 received from {username='slave'} at 127.0.0.1:33050
I20260709 13:19:28.977144 16295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87147731a68400e8cce9b879daa7c5d" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:19:28.797624 16822 raft_consensus.cc:1275] T 19baa7bc367844c288bca8f81439c413 P cb82cc7bd6db46d980c75237a1402860 [term 3 LEARNER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 13:19:28.729803 17703 ts_tablet_manager.cc:1434] T 19baa7bc367844c288bca8f81439c413 P cb82cc7bd6db46d980c75237a1402860: Time spent starting tablet: real 0.502s	user 0.005s	sys 0.022s
W20260709 13:19:28.853282 17853 log.cc:927] Time spent T 00000000000000000000000000000000 P 667a34b27a39474ebefc1af3cf5b4271: Append to log took a long time: real 0.086s	user 0.000s	sys 0.001s
I20260709 13:19:29.007587 16223 pending_rounds.cc:85] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac: Aborting all ops after (but not including) 5
I20260709 13:19:28.729566 17762 raft_consensus.cc:493] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:29.015424 17452 raft_consensus.cc:515] T ecc1e363230043fe9fef390f0b979f20 P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } }
I20260709 13:19:28.747943 17858 raft_consensus.cc:493] T 84f2da98864b4425bf7c0055951fdd4b P 647a343a542d4a139fe106c19095129b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:19:28.881263 17830 raft_consensus.cc:493] T 746aabf856ab401c9ef5e859371e7b67 P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
W20260709 13:19:28.834713 17867 log.cc:927] Time spent T c2f5f89186204242bf377f7c42b2e2da P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 0.135s	user 0.000s	sys 0.000s
W20260709 13:19:28.994568 17518 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.553s	user 0.003s	sys 0.056s
I20260709 13:19:29.016223 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5718c623ebee4d2bb770f0f9ddb2f164" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:19:28.784040 16821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5d59407eb545b895d8d922e0dd69fd" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
W20260709 13:19:28.955036 17738 consensus_meta.cc:287] T d2ab0ac8347641e6955e9bce7d90c1a2 P cb82cc7bd6db46d980c75237a1402860: Time spent flushing consensus metadata: real 0.583s	user 0.011s	sys 0.000s
I20260709 13:19:29.027438 16223 pending_rounds.cc:107] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac: Aborting uncommitted NO_OP operation due to leader change: 3.6
I20260709 13:19:29.022215 17882 consensus_queue.cc:1048] T e5ad4eb281564936a59f41e387a47156 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.015s
W20260709 13:19:28.791567 16192 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 50743.9us
W20260709 13:19:29.043944 17518 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.603s	user 0.003s	sys 0.056s
I20260709 13:19:29.026383 17758 consensus_queue.cc:1059] T c2f5f89186204242bf377f7c42b2e2da P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.012s
I20260709 13:19:28.954447 16143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5d59407eb545b895d8d922e0dd69fd" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:19:29.026804 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66ae176961b74813a58bc15c9c4c6f2a" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:28.858860 17861 raft_consensus.cc:493] T d2ab0ac8347641e6955e9bce7d90c1a2 P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:28.732884 17721 raft_consensus.cc:493] T c9944d5e4b8c4109af9955c4f08c9095 P cb82cc7bd6db46d980c75237a1402860 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:19:28.747540 16218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f5f89186204242bf377f7c42b2e2da" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:19:28.763626 17739 raft_consensus.cc:493] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:28.766834 16816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74ca8140f35c4ae29e8a664f6bdfc829" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:19:28.784906 17800 leader_election.cc:290] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:19:28.882023 17860 raft_consensus.cc:493] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:28.826462 16221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a378d7f796f94615b3b347119a9880fd" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:28.950779 16191 leader_election.cc:304] T af57996a8be84d398f0f023e8d08ef77 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
I20260709 13:19:28.825067 17847 leader_election.cc:290] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 3 election: Requested vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:19:28.732252 17794 raft_consensus.cc:3060] T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:19:28.928141 17829 leader_election.cc:290] T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 3 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:28.733445 17804 raft_consensus.cc:697] T 8638ee363273426db1393dd81e5c7b72 P cb82cc7bd6db46d980c75237a1402860 [term 4 LEADER]: Becoming Leader. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:19:28.917536 16819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66ae176961b74813a58bc15c9c4c6f2a" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:19:28.777647 17866 raft_consensus.cc:493] T 8cc959fb7f704fba9009c0595467d074 P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:19:28.838506 17781 leader_election.cc:290] T 66ae176961b74813a58bc15c9c4c6f2a P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:19:29.016633 16145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02838c86531d4f98bf58d1c22b8baf77" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:19:29.035737 17877 raft_consensus.cc:493] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:28.763607 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8638ee363273426db1393dd81e5c7b72" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:28.784690 16219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a959dbecc2648eb85d1cc550f5a3245" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:28.968461 17890 raft_consensus.cc:493] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:28.953907 17729 leader_election.cc:290] T 5a96373bb0e14dce8a266b2fc6c58507 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:29.039415 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c2095a5f4ad49408dbc7f69c606120d" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:29.044304 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a378d7f796f94615b3b347119a9880fd" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:19:29.026929 16236 tablet_copy_service.cc:434] P 1ca2725d290e47d68b1a4b609c9816ac: ending tablet copy session cb82cc7bd6db46d980c75237a1402860-ecc1e363230043fe9fef390f0b979f20 on tablet ecc1e363230043fe9fef390f0b979f20 with peer cb82cc7bd6db46d980c75237a1402860
I20260709 13:19:29.045274 16223 raft_consensus.cc:2889] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:19:28.887570 17644 consensus_queue.cc:1048] T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.153s
I20260709 13:19:28.939494 16149 raft_consensus.cc:1240] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [term 4 LEADER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 3. Current term is 4. Ops: []
I20260709 13:19:29.046833 17452 leader_election.cc:290] T ecc1e363230043fe9fef390f0b979f20 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:28.744724 17672 consensus_queue.cc:1048] T 8c11074579b046d99e585c2818e18a2c P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.296s
I20260709 13:19:29.051292 16822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a378d7f796f94615b3b347119a9880fd" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:19:29.094784 17645 consensus_queue.cc:1059] T ada5a004ba7c46819051e3ccec71ffdb P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.033s
I20260709 13:19:29.056208 17518 raft_consensus.cc:493] T 8d4ab870a05347868956aa8b841b2981 P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:29.042588 16294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a96373bb0e14dce8a266b2fc6c58507" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:29.043654 17888 raft_consensus.cc:493] T 02838c86531d4f98bf58d1c22b8baf77 P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:29.062254 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 248528us
W20260709 13:19:29.062522 16189 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:29.035008 17827 consensus_queue.cc:1048] T 19baa7bc367844c288bca8f81439c413 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.025s
I20260709 13:19:29.047372 17868 raft_consensus.cc:493] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:29.083928 16821 raft_consensus.cc:3060] T 806d7690c2b644959152074bb2cab0a7 P cb82cc7bd6db46d980c75237a1402860 [term 1 LEARNER]: Advancing to term 2
I20260709 13:19:29.050382 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d4ab870a05347868956aa8b841b2981" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
W20260709 13:19:29.087287 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 136634us
I20260709 13:19:29.089883 16222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746aabf856ab401c9ef5e859371e7b67" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:29.092043 16295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:29.032127 17762 raft_consensus.cc:515] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:29.170857 17890 raft_consensus.cc:515] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } }
I20260709 13:19:29.101279 16296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02838c86531d4f98bf58d1c22b8baf77" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:29.105367 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74ca8140f35c4ae29e8a664f6bdfc829" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:29.215435 16225 raft_consensus.cc:2468] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 3.
I20260709 13:19:29.216020 17890 leader_election.cc:290] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:19:29.046197 17830 raft_consensus.cc:515] T 746aabf856ab401c9ef5e859371e7b67 P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:29.050536 17738 raft_consensus.cc:515] T d2ab0ac8347641e6955e9bce7d90c1a2 P cb82cc7bd6db46d980c75237a1402860 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:29.115109 16145 raft_consensus.cc:2468] T 02838c86531d4f98bf58d1c22b8baf77 P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 3.
I20260709 13:19:29.056116 17858 raft_consensus.cc:515] T 84f2da98864b4425bf7c0055951fdd4b P 647a343a542d4a139fe106c19095129b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:19:29.123528 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 423902us
I20260709 13:19:29.096678 17800 raft_consensus.cc:493] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:29.093566 17760 raft_consensus.cc:493] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:29.127743 16297 raft_consensus.cc:3060] T a378d7f796f94615b3b347119a9880fd P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:19:29.064329 17721 raft_consensus.cc:515] T c9944d5e4b8c4109af9955c4f08c9095 P cb82cc7bd6db46d980c75237a1402860 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:29.175314 16822 raft_consensus.cc:2393] T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 3: Already voted for candidate cb82cc7bd6db46d980c75237a1402860 in this term.
W20260709 13:19:29.021862 17703 ts_tablet_manager.cc:1467] T 19baa7bc367844c288bca8f81439c413 P cb82cc7bd6db46d980c75237a1402860: Tablet startup took 791ms
I20260709 13:19:29.180593 17518 raft_consensus.cc:515] T 8d4ab870a05347868956aa8b841b2981 P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:29.160280 16221 raft_consensus.cc:2393] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 in current term 3: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
I20260709 13:19:29.117362 16819 raft_consensus.cc:2468] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 3.
I20260709 13:19:29.086776 17861 raft_consensus.cc:515] T d2ab0ac8347641e6955e9bce7d90c1a2 P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:29.093919 17739 raft_consensus.cc:515] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } }
W20260709 13:19:29.193126 16117 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 104425us
I20260709 13:19:29.193009 16223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7ca61592e848ea9b0fb81a2ce33cdc" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
W20260709 13:19:29.198096 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 161193us
I20260709 13:19:29.176656 16291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a378d7f796f94615b3b347119a9880fd" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:19:29.213395 17868 raft_consensus.cc:515] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:29.107939 17860 raft_consensus.cc:515] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } }
I20260709 13:19:29.166639 17877 raft_consensus.cc:515] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:29.127393 17866 raft_consensus.cc:515] T 8cc959fb7f704fba9009c0595467d074 P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:19:29.216822 16789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 127106us
W20260709 13:19:29.831588 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 613133us
I20260709 13:19:29.446874 17452 raft_consensus.cc:493] T 8158bbd17b994d2eb96f560bd0f1b88d P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
W20260709 13:19:29.309926 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 92242.2us
I20260709 13:19:29.836357 16223 raft_consensus.cc:3060] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:19:29.319361 16115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 223096us
I20260709 13:19:29.834090 17452 raft_consensus.cc:515] T 8158bbd17b994d2eb96f560bd0f1b88d P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:29.834547 16116 leader_election.cc:304] T b6f07842d3aa47858df5665874c86f3a P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:19:29.190999 17819 consensus_queue.cc:1048] T 8c11074579b046d99e585c2818e18a2c P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.703s
W20260709 13:19:29.796337 16189 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:29.994066 16265 leader_election.cc:304] T c9944d5e4b8c4109af9955c4f08c9095 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860
I20260709 13:19:29.619566 16821 raft_consensus.cc:1275] T 806d7690c2b644959152074bb2cab0a7 P cb82cc7bd6db46d980c75237a1402860 [term 2 LEARNER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
W20260709 13:19:29.836934 16791 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.738s	user 0.000s	sys 0.013s
I20260709 13:19:30.006572 16297 raft_consensus.cc:2468] T a378d7f796f94615b3b347119a9880fd P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 3.
I20260709 13:19:29.860193 16117 leader_election.cc:304] T b87147731a68400e8cce9b879daa7c5d P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860
W20260709 13:19:30.011350 16791 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.912s	user 0.000s	sys 0.021s
I20260709 13:19:29.995340 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66ae176961b74813a58bc15c9c4c6f2a" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:30.018741 16298 raft_consensus.cc:2468] T 66ae176961b74813a58bc15c9c4c6f2a P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 2.
I20260709 13:19:29.924350 16150 raft_consensus.cc:1275] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 13:19:30.025259 16145 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40844 (request call id 1699) took 1045 ms. Trace:
W20260709 13:19:30.027035 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 192566us
W20260709 13:19:30.027529 16117 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 167484us
I20260709 13:19:30.025853 16115 leader_election.cc:304] T 5a96373bb0e14dce8a266b2fc6c58507 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:19:29.191846 17893 raft_consensus.cc:493] T cdbd78acb4cf46298e0de8ba0d130e19 P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:19:30.027074 16145 rpcz_store.cc:276] 0709 13:19:28.949788 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:29.016408 (+ 66620us) service_pool.cc:224] Handling call
0709 13:19:29.994798 (+978390us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:30.027868 16116 leader_election.cc:304] T 8d4ab870a05347868956aa8b841b2981 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
I20260709 13:19:30.032925 17452 leader_election.cc:290] T 8158bbd17b994d2eb96f560bd0f1b88d P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:30.045879 17452 raft_consensus.cc:2749] T b6f07842d3aa47858df5665874c86f3a P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:19:30.046589 17452 raft_consensus.cc:2749] T b87147731a68400e8cce9b879daa7c5d P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:19:29.194094 17805 raft_consensus.cc:493] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:29.777958 16143 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.561s	user 0.020s	sys 0.010s
W20260709 13:19:29.324267 17787 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.635s	user 0.018s	sys 0.033s
I20260709 13:19:29.207679 16218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8638ee363273426db1393dd81e5c7b72" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
W20260709 13:19:29.800513 17891 log.cc:927] Time spent T af57996a8be84d398f0f023e8d08ef77 P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 0.183s	user 0.000s	sys 0.000s
W20260709 13:19:29.520134 17883 log.cc:927] Time spent T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860: Append to log took a long time: real 0.109s	user 0.001s	sys 0.000s
W20260709 13:19:29.798281 17794 consensus_meta.cc:287] T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045: Time spent flushing consensus metadata: real 0.683s	user 0.008s	sys 0.004s
I20260709 13:19:30.034628 16820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c11074579b046d99e585c2818e18a2c" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:19:29.985277 17865 raft_consensus.cc:493] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:29.194258 17888 raft_consensus.cc:515] T 02838c86531d4f98bf58d1c22b8baf77 P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:29.855701 17695 raft_consensus.cc:493] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:19:29.472822 17855 leader_election.cc:290] T 8c11074579b046d99e585c2818e18a2c P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:19:29.476934 17758 raft_consensus.cc:493] T 66f75124215f4ca6bef831e174a792d8 P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:29.325323 17804 consensus_queue.cc:237] T 8638ee363273426db1393dd81e5c7b72 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:29.355734 17864 leader_election.cc:290] T 5a959dbecc2648eb85d1cc550f5a3245 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
W20260709 13:19:30.268761 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 421867us
I20260709 13:19:29.878686 17800 raft_consensus.cc:515] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:29.684163 17878 raft_consensus.cc:493] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:19:29.834205 17833 raft_consensus.cc:493] T 8c11074579b046d99e585c2818e18a2c P cb82cc7bd6db46d980c75237a1402860 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:30.024741 17760 raft_consensus.cc:515] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:19:30.200101 17885 log.cc:927] Time spent T 829b4ad8c5a54eababeea1181b960d4b P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 0.785s	user 0.009s	sys 0.022s
I20260709 13:19:30.024742 16293 raft_consensus.cc:2468] T 8d4ab870a05347868956aa8b841b2981 P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 2.
W20260709 13:19:30.328177 16192 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 301721us
I20260709 13:19:29.325690 17847 consensus_queue.cc:1048] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.880s
I20260709 13:19:29.218250 17857 consensus_queue.cc:1048] T 19baa7bc367844c288bca8f81439c413 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 1.056s
I20260709 13:19:30.124264 17518 leader_election.cc:290] T 8d4ab870a05347868956aa8b841b2981 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
W20260709 13:19:30.286131 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 258313us
W20260709 13:19:30.197459 16143 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.981s	user 0.024s	sys 0.019s
I20260709 13:19:30.339646 16294 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48684 (request call id 1816) took 1206 ms. Trace:
I20260709 13:19:30.430111 16220 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33050 (request call id 301) took 1841 ms. Trace:
I20260709 13:19:30.382149 16191 leader_election.cc:304] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
W20260709 13:19:30.302418 16115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 276606us
I20260709 13:19:30.409611 17901 raft_consensus.cc:2749] T 5a96373bb0e14dce8a266b2fc6c58507 P 647a343a542d4a139fe106c19095129b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:19:30.410200 17899 raft_consensus.cc:493] T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:19:30.433897 17877 leader_election.cc:290] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:30.428254 17860 leader_election.cc:290] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:30.486636 17899 raft_consensus.cc:515] T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:30.203423 17858 leader_election.cc:290] T 84f2da98864b4425bf7c0055951fdd4b P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:29.864965 17870 consensus_queue.cc:1048] T ee9ed67006144797b1de6ea8c2013a68 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.545s
I20260709 13:19:29.542354 17873 raft_consensus.cc:493] T 8a878999c0024d2abe517484180290fc P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:29.929749 16217 raft_consensus.cc:2468] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 3.
W20260709 13:19:30.414283 16117 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 386138us
I20260709 13:19:30.085623 16226 raft_consensus.cc:2468] T d2ab0ac8347641e6955e9bce7d90c1a2 P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 3.
W20260709 13:19:30.340217 16190 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:30.286855 17894 raft_consensus.cc:493] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:19:30.450526 16220 rpcz_store.cc:276] 0709 13:19:28.589168 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:28.594919 (+  5751us) service_pool.cc:224] Handling call
0709 13:19:28.598620 (+  3701us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:28.631123 (+ 32503us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:28.631791 (+   668us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:28.729665 (+ 97874us) log.cc:844] Serialized 26 byte log entry
0709 13:19:29.267624 (+537959us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:19:29.308118 (+ 40494us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:29.323931 (+ 15813us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:30.429836 (+1105905us) raft_consensus.cc:1685] finished
0709 13:19:30.429863 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:30.430024 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":41719,"raft.queue_time_us":7235,"raft.run_cpu_time_us":200,"raft.run_wall_time_us":10292,"thread_start_us":451794,"threads_started":1,"wal-append.queue_time_us":1358971}
W20260709 13:19:30.381392 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 351060us
I20260709 13:19:30.450892 17861 leader_election.cc:290] T d2ab0ac8347641e6955e9bce7d90c1a2 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:19:29.211956 17869 raft_consensus.cc:2749] T af57996a8be84d398f0f023e8d08ef77 P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:19:29.472316 17729 raft_consensus.cc:493] T 8a878999c0024d2abe517484180290fc P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:30.381721 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 387702us
I20260709 13:19:30.290637 16789 leader_election.cc:304] T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 
W20260709 13:19:30.379499 17787 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.732s	user 0.035s	sys 0.055s
I20260709 13:19:30.480551 17830 leader_election.cc:290] T 746aabf856ab401c9ef5e859371e7b67 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:19:30.380533 17895 raft_consensus.cc:493] T 8cc959fb7f704fba9009c0595467d074 P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:19:30.030507 16819 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49892 (request call id 270) took 1108 ms. Trace:
I20260709 13:19:30.205382 16148 raft_consensus.cc:1275] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 13:19:29.707777 17806 raft_consensus.cc:493] T fffab26427e24d398e2695aad40ca134 P 647a343a542d4a139fe106c19095129b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:19:30.372380 17721 leader_election.cc:290] T c9944d5e4b8c4109af9955c4f08c9095 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:29.961696 17898 raft_consensus.cc:493] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:19:30.531313 17898 raft_consensus.cc:515] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:30.449179 16116 leader_election.cc:304] T 7c2095a5f4ad49408dbc7f69c606120d P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:19:30.422701 17884 raft_consensus.cc:493] T 434bff3082d141838068f323bdb0d1ed P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
W20260709 13:19:30.479682 16218 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33050 (request call id 316) took 1623 ms (client timeout 1963 ms). Trace:
I20260709 13:19:30.480338 17882 raft_consensus.cc:493] T e4dc6b0541ca431ebc757beac00521f1 P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:19:30.376118 17644 raft_consensus.cc:2804] T 8d4ab870a05347868956aa8b841b2981 P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 13:19:30.451272 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 69242.2us
I20260709 13:19:30.381067 17890 raft_consensus.cc:2749] T c9944d5e4b8c4109af9955c4f08c9095 P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:19:30.473943 17518 raft_consensus.cc:493] T 1df93779209748b9860f6d893dd3de53 P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:19:30.376945 17738 leader_election.cc:290] T d2ab0ac8347641e6955e9bce7d90c1a2 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 4 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:30.492904 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8158bbd17b994d2eb96f560bd0f1b88d" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:30.493173 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f2da98864b4425bf7c0055951fdd4b" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:30.466032 16296 raft_consensus.cc:2468] T 02838c86531d4f98bf58d1c22b8baf77 P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 3.
I20260709 13:19:30.085381 16221 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33050 (request call id 312) took 1289 ms. Trace:
I20260709 13:19:30.085201 17805 raft_consensus.cc:515] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:30.339387 16291 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48684 (request call id 1812) took 1182 ms. Trace:
I20260709 13:19:30.267419 17762 leader_election.cc:290] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:19:30.505560 16295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a959dbecc2648eb85d1cc550f5a3245" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:30.201028 16821 raft_consensus.cc:1275] T b87147731a68400e8cce9b879daa7c5d P cb82cc7bd6db46d980c75237a1402860 [term 3 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:19:30.431283 17803 raft_consensus.cc:3055] T ada5a004ba7c46819051e3ccec71ffdb P cb82cc7bd6db46d980c75237a1402860 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:19:30.418623 16192 leader_election.cc:304] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860
I20260709 13:19:30.516109 16145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f5f89186204242bf377f7c42b2e2da" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:19:30.407593 17758 raft_consensus.cc:515] T 66f75124215f4ca6bef831e174a792d8 P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:30.519541 17880 raft_consensus.cc:493] T 2794af5b99984efebdee564035841cb0 P cb82cc7bd6db46d980c75237a1402860 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:19:30.508555 16142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806d7690c2b644959152074bb2cab0a7" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:19:30.508291 17915 consensus_queue.cc:1048] T 74ca8140f35c4ae29e8a664f6bdfc829 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.025s
I20260709 13:19:30.428622 17739 leader_election.cc:290] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:30.488996 17781 raft_consensus.cc:493] T d16dfad0c3b540e5b4b86492ddf87208 P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
W20260709 13:19:30.513833 16790 connection.cc:256] RPC call timeout handler was delayed by -1.0418179119999422s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1587s for a response
I20260709 13:19:30.496515 17855 raft_consensus.cc:493] T d2ab0ac8347641e6955e9bce7d90c1a2 P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:19:30.557255 17802 raft_consensus.cc:493] T e5ad4eb281564936a59f41e387a47156 P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
W20260709 13:19:30.536253 16789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 245725us
W20260709 13:19:30.608866 16192 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 190400us
I20260709 13:19:30.418336 17794 raft_consensus.cc:515] T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:30.530122 17729 raft_consensus.cc:515] T 8a878999c0024d2abe517484180290fc P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:30.556713 17518 raft_consensus.cc:515] T 1df93779209748b9860f6d893dd3de53 P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:30.424229 17695 raft_consensus.cc:515] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:30.423548 17833 raft_consensus.cc:515] T 8c11074579b046d99e585c2818e18a2c P cb82cc7bd6db46d980c75237a1402860 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:30.516083 17894 raft_consensus.cc:515] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:30.655129 17518 leader_election.cc:290] T 1df93779209748b9860f6d893dd3de53 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:19:30.655822 17518 raft_consensus.cc:493] T 31969392ccdb43b491911a319e8bc7e1 P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
W20260709 13:19:29.993283 17703 ts_tablet_manager.cc:1469] T 19baa7bc367844c288bca8f81439c413 P cb82cc7bd6db46d980c75237a1402860: Trace:
0709 13:19:21.323244 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:21.323382 (+   138us) service_pool.cc:224] Handling call
0709 13:19:21.549539 (+226157us) ts_tablet_manager.cc:934] T 19baa7bc367844c288bca8f81439c413 P cb82cc7bd6db46d980c75237a1402860: Initiating tablet copy from peer 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
0709 13:19:25.679209 (+4129670us) tablet_copy_client.cc:368] Tablet copy session begun
0709 13:19:25.944432 (+265223us) tablet_metadata.cc:665] Metadata flushed
0709 13:19:25.950879 (+  6447us) tablet_copy_client.cc:497] Wrote new tablet metadata
0709 13:19:26.029225 (+ 78346us) tablet_copy_client.cc:418] Wrote new consensus metadata
0709 13:19:26.029516 (+   291us) tablet_replica.cc:206] Creating consensus instance
0709 13:19:26.029779 (+   263us) ts_tablet_manager.cc:989] Replica registered.
0709 13:19:26.029853 (+    74us) inbound_call.cc:177] Queueing success response
0709 13:19:27.086971 (+1057118us) ts_tablet_manager.cc:1379] Bootstrapping tablet
0709 13:19:28.231684 (+1144713us) ts_tablet_manager.cc:1435] Starting tablet replica
0709 13:19:28.287629 (+ 55945us) tablet_replica.cc:261] Starting instrumentation
0709 13:19:28.288271 (+   642us) tablet_replica.cc:280] Starting consensus
Metrics: {"dns_us":252,"mutex_wait_us":25095,"raft.queue_time_us":2448,"raft.run_cpu_time_us":139,"raft.run_wall_time_us":13606,"tablet-copy.queue_time_us":225326,"tablet-download-pool.queue_time_us":447,"tablet-download-pool.run_cpu_time_us":1173,"tablet-download-pool.run_wall_time_us":655531,"thread_start_us":289360,"threads_started":2}
I20260709 13:19:30.522538 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a959dbecc2648eb85d1cc550f5a3245" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:19:30.528354 16819 rpcz_store.cc:276] 0709 13:19:28.916958 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:28.917186 (+   228us) service_pool.cc:224] Handling call
0709 13:19:30.024847 (+1107661us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:30.425104 17865 raft_consensus.cc:515] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:30.593190 17847 raft_consensus.cc:493] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:19:30.545735 17644 raft_consensus.cc:493] T 8d4ab870a05347868956aa8b841b2981 P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:30.597222 16145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 4 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:19:30.417848 17878 raft_consensus.cc:515] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:30.554793 17800 leader_election.cc:290] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:19:30.549101 17858 raft_consensus.cc:493] T 08c74490c40243a88e62d1760b5cd814 P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:19:30.553061 17787 raft_consensus.cc:493] T 57a744f5aaf944678a63469cbf514015 P cb82cc7bd6db46d980c75237a1402860 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:19:30.604900 16224 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33050 (request call id 310) took 1912 ms. Trace:
I20260709 13:19:30.703725 16820 raft_consensus.cc:2468] T 8c11074579b046d99e585c2818e18a2c P cb82cc7bd6db46d980c75237a1402860 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 3.
I20260709 13:19:30.267745 17893 raft_consensus.cc:515] T cdbd78acb4cf46298e0de8ba0d130e19 P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:30.605752 16147 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40844 (request call id 1696) took 1650 ms. Trace:
I20260709 13:19:30.712750 16147 rpcz_store.cc:276] 0709 13:19:28.948336 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:28.949061 (+   725us) service_pool.cc:224] Handling call
0709 13:19:28.953677 (+  4616us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:30.085656 (+1131979us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:30.085691 (+    35us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:30.085731 (+    40us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:19:30.085756 (+    25us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:30.598749 (+512993us) spinlock_profiling.cc:243] Waited 1.13 s on lock 0x7b540006dbd4. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:19:30.598816 (+    67us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:30.599004 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2602909056}
I20260709 13:19:30.567654 17830 raft_consensus.cc:493] T 68a6838d82ef4cac83507f906139d90f P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:30.557499 17860 raft_consensus.cc:493] T 18ad7e3af5504571a69d10c00750a01c P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
W20260709 13:19:30.611308 16189 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:30.616021 17452 raft_consensus.cc:2749] T 7c2095a5f4ad49408dbc7f69c606120d P 647a343a542d4a139fe106c19095129b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:19:30.617360 16144 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40844 (request call id 1698) took 1668 ms. Trace:
I20260709 13:19:30.624312 16146 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40844 (request call id 1695) took 1701 ms. Trace:
I20260709 13:19:30.567689 17909 raft_consensus.cc:1064] T ecc1e363230043fe9fef390f0b979f20 P 1ca2725d290e47d68b1a4b609c9816ac: attempting to promote NON_VOTER cb82cc7bd6db46d980c75237a1402860 to VOTER
I20260709 13:19:30.450445 16294 rpcz_store.cc:276] 0709 13:19:29.105861 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:29.472033 (+366172us) service_pool.cc:224] Handling call
0709 13:19:29.472442 (+   409us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:29.998621 (+526179us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:30.030302 (+ 31681us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:30.035887 (+  5585us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:19:30.125631 (+ 89744us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:30.268497 (+142866us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:30.312278 (+ 43781us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:19:30.537042 16218 rpcz_store.cc:269] 0709 13:19:28.856101 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:29.178216 (+322115us) service_pool.cc:224] Handling call
0709 13:19:30.479563 (+1301347us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:30.642397 16296 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48684 (request call id 1809) took 1542 ms. Trace:
I20260709 13:19:30.690990 17866 leader_election.cc:290] T 8cc959fb7f704fba9009c0595467d074 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:30.577775 17803 raft_consensus.cc:740] T ada5a004ba7c46819051e3ccec71ffdb P cb82cc7bd6db46d980c75237a1402860 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
W20260709 13:19:30.561221 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 112127us
I20260709 13:19:30.553099 16221 rpcz_store.cc:276] 0709 13:19:28.747115 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:28.748244 (+  1129us) service_pool.cc:224] Handling call
0709 13:19:30.036481 (+1288237us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:30.588191 17877 raft_consensus.cc:493] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P cb82cc7bd6db46d980c75237a1402860 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
W20260709 13:19:30.683892 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 79905.7us
I20260709 13:19:30.688867 17518 raft_consensus.cc:515] T 31969392ccdb43b491911a319e8bc7e1 P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:30.716089 17905 raft_consensus.cc:2749] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:19:30.522907 17873 raft_consensus.cc:515] T 8a878999c0024d2abe517484180290fc P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:30.565631 17884 raft_consensus.cc:515] T 434bff3082d141838068f323bdb0d1ed P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:30.782541 17884 leader_election.cc:290] T 434bff3082d141838068f323bdb0d1ed P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:19:30.650889 17802 raft_consensus.cc:515] T e5ad4eb281564936a59f41e387a47156 P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:30.785470 17802 leader_election.cc:290] T e5ad4eb281564936a59f41e387a47156 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:30.725571 17870 raft_consensus.cc:493] T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:19:30.535346 17895 raft_consensus.cc:515] T 8cc959fb7f704fba9009c0595467d074 P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:30.553890 17806 raft_consensus.cc:515] T fffab26427e24d398e2695aad40ca134 P 647a343a542d4a139fe106c19095129b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:30.691847 17644 raft_consensus.cc:3060] T 8d4ab870a05347868956aa8b841b2981 P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:19:30.710079 17858 raft_consensus.cc:515] T 08c74490c40243a88e62d1760b5cd814 P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:30.723186 16146 rpcz_store.cc:276] 0709 13:19:28.923347 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:28.945109 (+ 21762us) service_pool.cc:224] Handling call
0709 13:19:28.945419 (+   310us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:29.923508 (+978089us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:29.923554 (+    46us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:29.923611 (+    57us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:19:29.923652 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:30.623930 (+700278us) spinlock_profiling.cc:243] Waited 972 ms on lock 0x7b54000d1954. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:19:30.624000 (+    70us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:30.624219 (+   219us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2235737472}
I20260709 13:19:30.553427 17882 raft_consensus.cc:515] T e4dc6b0541ca431ebc757beac00521f1 P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:30.567961 17901 raft_consensus.cc:493] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:19:30.639735 17781 raft_consensus.cc:515] T d16dfad0c3b540e5b4b86492ddf87208 P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:30.651119 17855 raft_consensus.cc:515] T d2ab0ac8347641e6955e9bce7d90c1a2 P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:30.736009 16296 rpcz_store.cc:276] 0709 13:19:29.099962 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:29.100964 (+  1002us) service_pool.cc:224] Handling call
0709 13:19:30.642131 (+1541167us) spinlock_profiling.cc:243] Waited 1.25 s on lock 0x7b540035e8d4. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:19:30.642305 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2876985600}
I20260709 13:19:30.694167 16149 raft_consensus.cc:2468] T 5a959dbecc2648eb85d1cc550f5a3245 P 647a343a542d4a139fe106c19095129b [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 1.
I20260709 13:19:30.936743 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31969392ccdb43b491911a319e8bc7e1" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:19:30.937101 17925 raft_consensus.cc:493] T 2794af5b99984efebdee564035841cb0 P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:19:30.769100 17898 leader_election.cc:290] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:30.707451 17800 raft_consensus.cc:493] T 57a744f5aaf944678a63469cbf514015 P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:19:30.739462 17861 raft_consensus.cc:2749] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:19:30.773959 17893 leader_election.cc:290] T cdbd78acb4cf46298e0de8ba0d130e19 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:19:30.954954 17893 raft_consensus.cc:493] T 9e65676b21b748728ad39668cfe722ec P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:19:30.955638 17893 raft_consensus.cc:515] T 9e65676b21b748728ad39668cfe722ec P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:30.774420 17919 raft_consensus.cc:493] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:30.564704 16291 rpcz_store.cc:276] 0709 13:19:29.100648 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:29.120401 (+ 19753us) service_pool.cc:224] Handling call
0709 13:19:30.283359 (+1162958us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:30.703567 16224 rpcz_store.cc:276] 0709 13:19:28.693031 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:28.693179 (+   148us) service_pool.cc:224] Handling call
0709 13:19:28.693483 (+   304us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:28.695185 (+  1702us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:28.695218 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:28.730200 (+ 34982us) log.cc:844] Serialized 26 byte log entry
0709 13:19:29.122741 (+392541us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:19:29.169870 (+ 47129us) log.cc:844] Serialized 14 byte log entry
0709 13:19:29.208238 (+ 38368us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:29.208368 (+   130us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:30.604592 (+1396224us) raft_consensus.cc:1685] finished
0709 13:19:30.604624 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:30.604813 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":46349,"prepare.queue_time_us":48309,"prepare.run_cpu_time_us":43,"prepare.run_wall_time_us":45,"raft.queue_time_us":84481,"raft.run_cpu_time_us":3287,"raft.run_wall_time_us":109213,"thread_start_us":355635,"threads_started":1,"wal-append.queue_time_us":686909}
I20260709 13:19:30.729969 17865 leader_election.cc:290] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:19:30.739385 17892 raft_consensus.cc:493] T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:19:30.797276 16147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d4ab870a05347868956aa8b841b2981" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:19:30.813513 17739 consensus_queue.cc:1048] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.127s
I20260709 13:19:30.724342 17762 consensus_queue.cc:1048] T b6f07842d3aa47858df5665874c86f3a P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.064s
W20260709 13:19:30.775113 16223 consensus_meta.cc:287] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 1ca2725d290e47d68b1a4b609c9816ac: Time spent flushing consensus metadata: real 0.916s	user 0.008s	sys 0.001s
I20260709 13:19:30.765831 17760 leader_election.cc:290] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:19:30.707938 17847 raft_consensus.cc:515] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:30.814507 17518 leader_election.cc:290] T 31969392ccdb43b491911a319e8bc7e1 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:19:31.011742 17518 raft_consensus.cc:493] T 3884f864a96d428785895e5463e5728f P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:19:30.721998 16144 rpcz_store.cc:276] 0709 13:19:28.948882 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:29.043097 (+ 94215us) service_pool.cc:224] Handling call
0709 13:19:29.043448 (+   351us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:29.949391 (+905943us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:29.949437 (+    46us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:29.949495 (+    58us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:19:29.949540 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:30.617009 (+667469us) spinlock_profiling.cc:243] Waited 906 ms on lock 0x7b540028da54. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:19:30.617070 (+    61us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:30.617278 (+   208us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2082812800}
I20260709 13:19:30.716930 17787 raft_consensus.cc:515] T 57a744f5aaf944678a63469cbf514015 P cb82cc7bd6db46d980c75237a1402860 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:19:30.800522 16790 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.14.177.130:35943 timed out after 1.527s (SENT) [suppressed 34 similar messages]
I20260709 13:19:30.766096 17890 raft_consensus.cc:493] T 8a878999c0024d2abe517484180290fc P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:19:30.802110 17884 raft_consensus.cc:493] T 26354361e9b044cea1c2b86c4882fe92 P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:19:30.732128 17830 raft_consensus.cc:515] T 68a6838d82ef4cac83507f906139d90f P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:30.777631 17899 leader_election.cc:290] T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:19:30.778364 17888 leader_election.cc:290] T 02838c86531d4f98bf58d1c22b8baf77 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:19:30.651588 17880 raft_consensus.cc:515] T 2794af5b99984efebdee564035841cb0 P cb82cc7bd6db46d980c75237a1402860 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:30.764053 17860 raft_consensus.cc:515] T 18ad7e3af5504571a69d10c00750a01c P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } }
I20260709 13:19:30.775271 17914 raft_consensus.cc:2804] T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:19:30.952368 17800 raft_consensus.cc:515] T 57a744f5aaf944678a63469cbf514015 P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:30.953822 17895 leader_election.cc:290] T 8cc959fb7f704fba9009c0595467d074 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:19:30.869251 16294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ab0ac8347641e6955e9bce7d90c1a2" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:19:31.046099 16294 raft_consensus.cc:3060] T d2ab0ac8347641e6955e9bce7d90c1a2 P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Advancing to term 4
I20260709 13:19:30.963207 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8158bbd17b994d2eb96f560bd0f1b88d" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:30.798116 17918 consensus_queue.cc:1048] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.009s
I20260709 13:19:30.937876 17806 leader_election.cc:290] T fffab26427e24d398e2695aad40ca134 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:31.001367 16223 raft_consensus.cc:2468] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 4.
I20260709 13:19:30.787134 17877 raft_consensus.cc:515] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P cb82cc7bd6db46d980c75237a1402860 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:31.015756 17926 raft_consensus.cc:493] T 8158bbd17b994d2eb96f560bd0f1b88d P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:19:30.959955 16220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:31.012388 17518 raft_consensus.cc:515] T 3884f864a96d428785895e5463e5728f P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:30.968209 17917 raft_consensus.cc:493] T 434bff3082d141838068f323bdb0d1ed P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:19:30.969272 17805 leader_election.cc:290] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:19:30.936777 17916 raft_consensus.cc:493] T d16dfad0c3b540e5b4b86492ddf87208 P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:19:31.015331 17833 leader_election.cc:290] T 8c11074579b046d99e585c2818e18a2c P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:30.828249 17870 raft_consensus.cc:515] T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:30.937691 17925 raft_consensus.cc:515] T 2794af5b99984efebdee564035841cb0 P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:30.958096 17919 raft_consensus.cc:515] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:31.047911 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f2da98864b4425bf7c0055951fdd4b" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
W20260709 13:19:31.251350 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 159172us
W20260709 13:19:31.280230 16225 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33050 (request call id 317) took 2389 ms (client timeout 1877 ms). Trace:
I20260709 13:19:31.018872 17758 leader_election.cc:290] T 66f75124215f4ca6bef831e174a792d8 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:19:30.963896 17901 raft_consensus.cc:515] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:19:31.287420 16223 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33050 (request call id 315) took 2431 ms (client timeout 2160 ms). Trace:
I20260709 13:19:31.283499 16296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806d7690c2b644959152074bb2cab0a7" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:31.362839 16296 raft_consensus.cc:2468] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 1.
W20260709 13:19:31.365584 16223 rpcz_store.cc:269] 0709 13:19:28.840734 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:29.174172 (+333438us) service_pool.cc:224] Handling call
0709 13:19:31.271892 (+2097720us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:31.366631 16223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746aabf856ab401c9ef5e859371e7b67" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:31.368824 16265 leader_election.cc:304] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:19:31.282699 16146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87ac0d1f7e8a419f9bcbc04aee0e067e" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:19:31.390092 16296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdbd78acb4cf46298e0de8ba0d130e19" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:31.394246 16296 raft_consensus.cc:2468] T cdbd78acb4cf46298e0de8ba0d130e19 P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 2.
W20260709 13:19:31.361593 16217 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33050 (request call id 311) took 2628 ms (client timeout 1587 ms). Trace:
W20260709 13:19:31.405318 16225 rpcz_store.cc:269] 0709 13:19:28.883137 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:29.026533 (+143396us) service_pool.cc:224] Handling call
0709 13:19:31.271827 (+2245294us) spinlock_profiling.cc:243] Waited 159 ms on lock 0x7b5400395154. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:19:31.271941 (+   114us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":364755840}
W20260709 13:19:31.360306 17920 log.cc:927] Time spent T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 0.378s	user 0.000s	sys 0.001s
I20260709 13:19:31.027206 17794 leader_election.cc:290] T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 3 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:19:31.475858 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8638ee363273426db1393dd81e5c7b72" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:31.424083 16296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 4 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:31.041041 16143 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46224 (request call id 203) took 2030 ms. Trace:
I20260709 13:19:31.418062 16223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 4 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
W20260709 13:19:31.336810 17907 log.cc:927] Time spent T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.590s	user 0.000s	sys 0.001s
I20260709 13:19:31.413281 16789 leader_election.cc:304] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 
W20260709 13:19:31.091689 17887 log.cc:927] Time spent T 3a7ca61592e848ea9b0fb81a2ce33cdc P cb82cc7bd6db46d980c75237a1402860: Append to log took a long time: real 0.492s	user 0.001s	sys 0.001s
I20260709 13:19:31.473994 17831 raft_consensus.cc:3055] T c2f5f89186204242bf377f7c42b2e2da P 1ca2725d290e47d68b1a4b609c9816ac [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:19:31.485455 17781 leader_election.cc:290] T d16dfad0c3b540e5b4b86492ddf87208 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:19:31.486436 16298 raft_consensus.cc:2393] T 8638ee363273426db1393dd81e5c7b72 P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 4: Already voted for candidate cb82cc7bd6db46d980c75237a1402860 in this term.
W20260709 13:19:31.424858 17924 log.cc:927] Time spent T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 0.059s	user 0.000s	sys 0.001s
I20260709 13:19:31.062829 17729 leader_election.cc:290] T 8a878999c0024d2abe517484180290fc P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:31.338889 17800 leader_election.cc:290] T 57a744f5aaf944678a63469cbf514015 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:19:31.090699 17895 raft_consensus.cc:493] T c4f65cdf00b748bb963ee6e81ef43b79 P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:31.257995 17888 raft_consensus.cc:493] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:31.258343 17899 raft_consensus.cc:493] T c0eab25ce4c64e018a041beeda2867db P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:19:31.094483 17695 leader_election.cc:290] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:31.030815 17890 raft_consensus.cc:515] T 8a878999c0024d2abe517484180290fc P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:19:31.494129 16789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 80961.4us
I20260709 13:19:31.483704 16296 raft_consensus.cc:2468] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 3.
W20260709 13:19:31.500258 17923 log.cc:927] Time spent T 87ac0d1f7e8a419f9bcbc04aee0e067e P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.341s	user 0.000s	sys 0.000s
I20260709 13:19:31.498423 17831 raft_consensus.cc:740] T c2f5f89186204242bf377f7c42b2e2da P 1ca2725d290e47d68b1a4b609c9816ac [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:19:31.501654 16292 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48684 (request call id 1803) took 3091 ms. Trace:
I20260709 13:19:31.498660 17833 raft_consensus.cc:2804] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:19:31.336740 17914 raft_consensus.cc:697] T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860 [term 3 LEADER]: Becoming Leader. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:19:31.435014 17855 leader_election.cc:290] T d2ab0ac8347641e6955e9bce7d90c1a2 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:19:31.289803 17866 raft_consensus.cc:493] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:31.509948 17794 raft_consensus.cc:493] T 23f82224f4564fea93572e8206bad1fe P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:19:31.092190 16291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9944d5e4b8c4109af9955c4f08c9095" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:31.435976 17858 leader_election.cc:290] T 08c74490c40243a88e62d1760b5cd814 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:31.423820 17847 leader_election.cc:290] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:19:31.508847 16225 raft_consensus.cc:1275] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 3. (term mismatch)
I20260709 13:19:31.640777 16225 pending_rounds.cc:85] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac: Aborting all ops after (but not including) 2
I20260709 13:19:31.641078 16225 pending_rounds.cc:107] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac: Aborting uncommitted NO_OP operation due to leader change: 3.3
I20260709 13:19:31.641323 16225 raft_consensus.cc:2889] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:19:31.338611 17926 raft_consensus.cc:515] T 8158bbd17b994d2eb96f560bd0f1b88d P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:19:31.509291 16264 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.971s	user 0.050s	sys 0.106s
W20260709 13:19:31.642756 16264 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.104s	user 0.050s	sys 0.106s
I20260709 13:19:31.473367 17787 leader_election.cc:290] T 57a744f5aaf944678a63469cbf514015 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:31.511915 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ab0ac8347641e6955e9bce7d90c1a2" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:31.496829 17518 leader_election.cc:290] T 3884f864a96d428785895e5463e5728f P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:19:31.513294 16223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5ad4eb281564936a59f41e387a47156" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:31.643935 16298 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48684 (request call id 1827) took 1136 ms. Trace:
I20260709 13:19:31.644451 16223 raft_consensus.cc:2468] T e5ad4eb281564936a59f41e387a47156 P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 3.
I20260709 13:19:31.518949 17878 leader_election.cc:290] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
W20260709 13:19:31.476773 16217 rpcz_store.cc:269] 0709 13:19:28.733674 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:28.736916 (+  3242us) service_pool.cc:224] Handling call
0709 13:19:31.361366 (+2624450us) spinlock_profiling.cc:243] Waited 802 ms on lock 0x7b5400141ed4. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:19:31.361518 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1845054208}
I20260709 13:19:31.430531 17894 leader_election.cc:290] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:19:31.520723 16296 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48674 (request call id 1926) took 1026 ms. Trace:
I20260709 13:19:31.646677 16296 rpcz_store.cc:276] 0709 13:19:30.494428 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:31.423784 (+929356us) service_pool.cc:224] Handling call
0709 13:19:31.520636 (+ 96852us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:31.648597 16296 raft_consensus.cc:1240] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 3. Current term is 4. Ops: []
I20260709 13:19:31.649317 16296 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48684 (request call id 1823) took 1143 ms. Trace:
I20260709 13:19:31.649715 16296 rpcz_store.cc:276] 0709 13:19:30.505929 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:31.647976 (+1142047us) service_pool.cc:224] Handling call
0709 13:19:31.648359 (+   383us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:31.649042 (+   683us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:31.649253 (+   211us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:31.614480 17833 raft_consensus.cc:493] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:31.436370 17882 leader_election.cc:290] T e4dc6b0541ca431ebc757beac00521f1 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:19:31.505584 17830 leader_election.cc:290] T 68a6838d82ef4cac83507f906139d90f P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:31.653497 17518 raft_consensus.cc:493] T 18ad7e3af5504571a69d10c00750a01c P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:19:31.309536 17884 raft_consensus.cc:515] T 26354361e9b044cea1c2b86c4882fe92 P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:31.618664 16220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cc959fb7f704fba9009c0595467d074" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:31.495541 17880 leader_election.cc:290] T 2794af5b99984efebdee564035841cb0 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:31.514046 17893 leader_election.cc:290] T 9e65676b21b748728ad39668cfe722ec P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:19:31.408481 17917 raft_consensus.cc:515] T 434bff3082d141838068f323bdb0d1ed P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:31.644490 16298 rpcz_store.cc:276] 0709 13:19:30.507242 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:31.511633 (+1004391us) service_pool.cc:224] Handling call
0709 13:19:31.643876 (+132243us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:31.424538 17873 leader_election.cc:290] T 8a878999c0024d2abe517484180290fc P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:31.655325 16296 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48684 (request call id 1825) took 1148 ms. Trace:
I20260709 13:19:31.674762 16296 rpcz_store.cc:276] 0709 13:19:30.506638 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:31.650597 (+1143959us) service_pool.cc:224] Handling call
0709 13:19:31.652876 (+  2279us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:31.654867 (+  1991us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:31.654909 (+    42us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:31.654966 (+    57us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:19:31.655006 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:31.655081 (+    75us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:31.655262 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:31.670461 17518 raft_consensus.cc:515] T 18ad7e3af5504571a69d10c00750a01c P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } }
I20260709 13:19:31.520717 17800 raft_consensus.cc:493] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
W20260709 13:19:31.676638 16191 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.118s	user 0.026s	sys 0.070s
I20260709 13:19:31.671736 16298 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48684 (request call id 1826) took 1164 ms. Trace:
I20260709 13:19:31.677204 16298 rpcz_store.cc:276] 0709 13:19:30.506926 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:31.670656 (+1163730us) service_pool.cc:224] Handling call
0709 13:19:31.671057 (+   401us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:31.671295 (+   238us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:31.671331 (+    36us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:31.671386 (+    55us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:19:31.671429 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:31.671507 (+    78us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:31.671666 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:31.673870 17870 leader_election.cc:290] T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
W20260709 13:19:31.657910 16192 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.048s	user 0.052s	sys 0.101s
I20260709 13:19:31.020129 17892 raft_consensus.cc:515] T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:19:31.683115 16192 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.073s	user 0.054s	sys 0.102s
W20260709 13:19:31.683696 16192 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 1.07368e+06us
I20260709 13:19:31.609328 17899 raft_consensus.cc:515] T c0eab25ce4c64e018a041beeda2867db P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:19:31.680697 16191 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.122s	user 0.026s	sys 0.070s
I20260709 13:19:31.685022 17933 raft_consensus.cc:493] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:19:31.733099 17518 leader_election.cc:290] T 18ad7e3af5504571a69d10c00750a01c P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:19:31.652843 17833 raft_consensus.cc:3060] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:19:31.656435 17855 raft_consensus.cc:2749] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
W20260709 13:19:31.736840 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 1.17848e+06us
I20260709 13:19:31.747331 16191 leader_election.cc:304] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
I20260709 13:19:31.609027 17888 raft_consensus.cc:515] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:31.641510 17866 raft_consensus.cc:515] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:31.651788 16291 raft_consensus.cc:2468] T c9944d5e4b8c4109af9955c4f08c9095 P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 3.
I20260709 13:19:31.670765 17882 raft_consensus.cc:493] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:19:31.643864 17794 raft_consensus.cc:515] T 23f82224f4564fea93572e8206bad1fe P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:31.760852 17890 leader_election.cc:290] T 8a878999c0024d2abe517484180290fc P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:19:31.762867 17899 leader_election.cc:290] T c0eab25ce4c64e018a041beeda2867db P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:19:31.707918 17800 raft_consensus.cc:515] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:31.616755 17895 raft_consensus.cc:515] T c4f65cdf00b748bb963ee6e81ef43b79 P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:31.760921 17926 leader_election.cc:290] T 8158bbd17b994d2eb96f560bd0f1b88d P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:19:31.682093 17831 consensus_queue.cc:260] T c2f5f89186204242bf377f7c42b2e2da P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:31.796351 16146 raft_consensus.cc:2410] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 for term 4 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:19:31.769336 16191 leader_election.cc:304] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
I20260709 13:19:31.798281 16191 leader_election.cc:304] T 02838c86531d4f98bf58d1c22b8baf77 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
I20260709 13:19:31.769523 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a878999c0024d2abe517484180290fc" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
W20260709 13:19:31.772171 16189 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:31.430217 17916 raft_consensus.cc:515] T d16dfad0c3b540e5b4b86492ddf87208 P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:31.777454 16223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806d7690c2b644959152074bb2cab0a7" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:31.786094 17925 leader_election.cc:290] T 2794af5b99984efebdee564035841cb0 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:19:31.766913 17873 raft_consensus.cc:2749] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:19:31.777014 17847 raft_consensus.cc:493] T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:31.784404 16291 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40474 (request call id 217) took 1088 ms. Trace:
I20260709 13:19:31.584723 16292 rpcz_store.cc:276] 0709 13:19:28.409996 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:28.416235 (+  6239us) service_pool.cc:224] Handling call
0709 13:19:28.858944 (+442709us) spinlock_profiling.cc:243] Waited 1.77 ms on lock 0x7b5800090098. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69ad8fadb1 00007f69ada3ab71 00007f69ada3aeee 0000000000375939 00007f69b5aae1e0 00007f69b5b770c4 00007f69b5b77131 00007f69b5b38fc0 00007f69b5b23929 00007f69b5b233d2 00007f69af09abd4
0709 13:19:28.859703 (+   759us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:29.124574 (+264871us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:29.135255 (+ 10681us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:29.472901 (+337646us) spinlock_profiling.cc:243] Waited 5.13 ms on lock 0x7b2c00132978. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69af458a11 00007f69af468372 00007f69b0b9aa01 00007f69b0b5f43a 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2
0709 13:19:29.473458 (+   557us) log.cc:844] Serialized 26 byte log entry
0709 13:19:29.777584 (+304126us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:19:29.777723 (+   139us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:29.777848 (+   125us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:31.501235 (+1723387us) raft_consensus.cc:1685] finished
0709 13:19:31.501267 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:31.501534 (+   267us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":195986,"raft.run_cpu_time_us":80,"raft.run_wall_time_us":79,"spinlock_wait_cycles":15883392,"thread_start_us":504056,"threads_started":2,"wal-append.queue_time_us":1095719}
I20260709 13:19:31.773587 17901 leader_election.cc:290] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
W20260709 13:19:31.809566 16222 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.938s	user 0.015s	sys 0.026s
W20260709 13:19:31.841406 16222 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.970s	user 0.016s	sys 0.026s
I20260709 13:19:31.813266 17936 raft_consensus.cc:2804] T 02838c86531d4f98bf58d1c22b8baf77 P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 13:19:31.813860 16791 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.321s	user 0.042s	sys 0.120s
W20260709 13:19:31.819231 16115 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.216s	user 0.011s	sys 0.032s
I20260709 13:19:31.789821 17877 leader_election.cc:290] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
W20260709 13:19:31.812247 16190 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.280s	user 0.009s	sys 0.036s
I20260709 13:19:31.852864 16223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 4 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
W20260709 13:19:31.853157 16190 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.321s	user 0.009s	sys 0.036s
W20260709 13:19:31.853418 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 1.32132e+06us
W20260709 13:19:31.819475 17804 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.802s	user 0.023s	sys 0.020s
I20260709 13:19:31.491912 16143 rpcz_store.cc:276] 0709 13:19:29.001265 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:29.127435 (+126170us) service_pool.cc:224] Handling call
0709 13:19:29.216040 (+ 88605us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:29.216263 (+   223us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:19:30.450283 (+1234020us) log.cc:844] Serialized 14 byte log entry
0709 13:19:30.910708 (+460425us) log.cc:844] Serialized 14 byte log entry
0709 13:19:30.959273 (+ 48565us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:30.971628 (+ 12355us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:19:30.978216 (+  6588us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:31.015307 (+ 37091us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:31.031535 (+ 16228us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":80356,"prepare.queue_time_us":1117655,"prepare.run_cpu_time_us":109,"prepare.run_wall_time_us":114,"raft.queue_time_us":166088,"raft.run_cpu_time_us":319,"raft.run_wall_time_us":26693,"thread_start_us":1531016,"threads_started":3,"wal-append.queue_time_us":278353}
W20260709 13:19:31.855141 17804 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.842s	user 0.023s	sys 0.028s
I20260709 13:19:31.856630 16143 raft_consensus.cc:1275] T 68a6838d82ef4cac83507f906139d90f P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:19:31.857417 16143 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40830 (request call id 946) took 1312 ms. Trace:
I20260709 13:19:31.857831 16143 rpcz_store.cc:276] 0709 13:19:30.544700 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:31.855780 (+1311080us) service_pool.cc:224] Handling call
0709 13:19:31.856194 (+   414us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:31.857090 (+   896us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:31.857349 (+   259us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:31.788902 16220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fffab26427e24d398e2695aad40ca134" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
W20260709 13:19:31.852594 16115 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.249s	user 0.011s	sys 0.032s
I20260709 13:19:31.806901 17758 raft_consensus.cc:2749] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:19:31.838707 17892 leader_election.cc:290] T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:19:31.768911 17933 raft_consensus.cc:515] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:31.787359 17882 raft_consensus.cc:515] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:19:31.852511 16791 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.359s	user 0.042s	sys 0.120s
I20260709 13:19:31.851626 17936 raft_consensus.cc:493] T 02838c86531d4f98bf58d1c22b8baf77 P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:31.834884 16217 raft_consensus.cc:2468] T 8a878999c0024d2abe517484180290fc P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 2.
W20260709 13:19:31.861212 16790 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.837s	user 0.017s	sys 0.018s
I20260709 13:19:31.863823 17884 leader_election.cc:290] T 26354361e9b044cea1c2b86c4882fe92 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:19:31.838723 17831 raft_consensus.cc:3060] T c2f5f89186204242bf377f7c42b2e2da P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:19:31.884370 16790 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.860s	user 0.017s	sys 0.018s
W20260709 13:19:31.887758 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 1.0872e+06us
I20260709 13:19:31.839638 17847 raft_consensus.cc:515] T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:31.838564 16291 rpcz_store.cc:276] 0709 13:19:30.696225 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:31.021034 (+324809us) service_pool.cc:224] Handling call
0709 13:19:31.781875 (+760841us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:31.788470 17914 consensus_queue.cc:237] T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:31.874990 17863 raft_consensus.cc:493] T af57996a8be84d398f0f023e8d08ef77 P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:19:31.883585 17804 raft_consensus.cc:493] T ee9ed67006144797b1de6ea8c2013a68 P cb82cc7bd6db46d980c75237a1402860 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:31.880195 17901 raft_consensus.cc:493] T 829b4ad8c5a54eababeea1181b960d4b P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
W20260709 13:19:31.904778 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 539964us
I20260709 13:19:31.885783 17936 raft_consensus.cc:3060] T 02838c86531d4f98bf58d1c22b8baf77 P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:19:31.884155 16115 leader_election.cc:304] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:19:31.885550 17877 raft_consensus.cc:493] T 66f75124215f4ca6bef831e174a792d8 P cb82cc7bd6db46d980c75237a1402860 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:19:31.909358 16115 leader_election.cc:304] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:19:31.906355 16222 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57604 (request call id 1853) took 1091 ms. Trace:
I20260709 13:19:31.904681 16143 raft_consensus.cc:1275] T b87147731a68400e8cce9b879daa7c5d P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
W20260709 13:19:31.914309 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 59863.4us
I20260709 13:19:31.922523 16190 leader_election.cc:304] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:19:31.920746 17794 leader_election.cc:290] T 23f82224f4564fea93572e8206bad1fe P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
W20260709 13:19:31.925504 16265 connection.cc:256] RPC call timeout handler was delayed by -1.3657983339999191s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2056s for a response
I20260709 13:19:31.926147 17873 raft_consensus.cc:2804] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:19:31.926528 17873 raft_consensus.cc:493] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:31.925524 16299 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48684 (request call id 1805) took 2827 ms. Trace:
I20260709 13:19:31.908878 17863 raft_consensus.cc:515] T af57996a8be84d398f0f023e8d08ef77 P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:31.928159 17882 leader_election.cc:290] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:19:31.927022 16299 rpcz_store.cc:276] 0709 13:19:29.097809 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:29.097966 (+   157us) service_pool.cc:224] Handling call
0709 13:19:29.098379 (+   413us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:19:30.431622 (+1333243us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:30.431656 (+    34us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:19:30.432302 (+   646us) log.cc:844] Serialized 52 byte log entry
0709 13:19:30.549103 (+116801us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:19:30.814293 (+265190us) log.cc:844] Serialized 14 byte log entry
0709 13:19:30.814477 (+   184us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:31.917527 (+1103050us) spinlock_profiling.cc:243] Waited 1.33 s on lock 0x7b54002e0c54. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:19:31.917576 (+    49us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:31.917625 (+    49us) raft_consensus.cc:1685] finished
0709 13:19:31.925312 (+  7687us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:31.925455 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":1195994,"prepare.run_cpu_time_us":46,"prepare.run_wall_time_us":47,"raft.queue_time_us":230013,"raft.run_cpu_time_us":196,"raft.run_wall_time_us":197,"spinlock_wait_cycles":3065004800,"thread_start_us":353911,"threads_started":2,"wal-append.queue_time_us":427576}
I20260709 13:19:31.929075 17863 leader_election.cc:290] T af57996a8be84d398f0f023e8d08ef77 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:19:31.929805 17917 leader_election.cc:290] T 434bff3082d141838068f323bdb0d1ed P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:19:31.907307 17919 leader_election.cc:290] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:31.917665 17870 raft_consensus.cc:2804] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:19:31.931859 17870 raft_consensus.cc:493] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:31.932353 17870 raft_consensus.cc:3060] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:19:31.933914 17833 raft_consensus.cc:515] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:19:31.935478 16266 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 1.117s	user 0.022s	sys 0.085s
W20260709 13:19:31.935964 16266 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 1.118s	user 0.022s	sys 0.085s
W20260709 13:19:31.936371 16266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 1.13212e+06us
I20260709 13:19:31.940337 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d4ab870a05347868956aa8b841b2981" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:31.942021 16224 raft_consensus.cc:2468] T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 2.
I20260709 13:19:31.943095 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdbd78acb4cf46298e0de8ba0d130e19" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:31.943933 16224 raft_consensus.cc:2393] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 3: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
I20260709 13:19:31.912586 17938 raft_consensus.cc:2804] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:19:31.926994 17873 raft_consensus.cc:3060] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:19:31.916155 17901 raft_consensus.cc:515] T 829b4ad8c5a54eababeea1181b960d4b P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:31.923574 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66ae176961b74813a58bc15c9c4c6f2a" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:31.944527 17787 raft_consensus.cc:493] T 26354361e9b044cea1c2b86c4882fe92 P cb82cc7bd6db46d980c75237a1402860 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:19:31.910890 17804 raft_consensus.cc:515] T ee9ed67006144797b1de6ea8c2013a68 P cb82cc7bd6db46d980c75237a1402860 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:31.919212 17877 raft_consensus.cc:515] T 66f75124215f4ca6bef831e174a792d8 P cb82cc7bd6db46d980c75237a1402860 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:31.944154 17794 raft_consensus.cc:493] T 84f2da98864b4425bf7c0055951fdd4b P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:19:31.937213 17934 raft_consensus.cc:493] T 23f82224f4564fea93572e8206bad1fe P cb82cc7bd6db46d980c75237a1402860 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:19:31.944829 17888 leader_election.cc:290] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
W20260709 13:19:31.979677 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 57472.7us
I20260709 13:19:31.950392 16143 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40830 (request call id 948) took 1399 ms. Trace:
I20260709 13:19:31.948866 17938 raft_consensus.cc:493] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:31.940547 17695 raft_consensus.cc:493] T 3884f864a96d428785895e5463e5728f P cb82cc7bd6db46d980c75237a1402860 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
W20260709 13:19:31.961261 16293 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48674 (request call id 1912) took 2967 ms (client timeout 2700 ms). Trace:
I20260709 13:19:31.949858 17942 raft_consensus.cc:493] T 7c2095a5f4ad49408dbc7f69c606120d P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:19:31.966423 17866 leader_election.cc:290] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:31.970348 16149 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46224 (request call id 205) took 1459 ms. Trace:
I20260709 13:19:31.994670 16190 leader_election.cc:304] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860
I20260709 13:19:31.943980 17882 consensus_queue.cc:1048] T b87147731a68400e8cce9b879daa7c5d P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.042s
W20260709 13:19:31.999713 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 60168.6us
I20260709 13:19:31.980230 17895 leader_election.cc:290] T c4f65cdf00b748bb963ee6e81ef43b79 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
W20260709 13:19:31.990409 17644 consensus_meta.cc:287] T 8d4ab870a05347868956aa8b841b2981 P 647a343a542d4a139fe106c19095129b: Time spent flushing consensus metadata: real 1.116s	user 0.007s	sys 0.007s
I20260709 13:19:32.001170 17942 raft_consensus.cc:515] T 7c2095a5f4ad49408dbc7f69c606120d P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:32.011490 17644 raft_consensus.cc:515] T 8d4ab870a05347868956aa8b841b2981 P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:32.013432 17942 leader_election.cc:290] T 7c2095a5f4ad49408dbc7f69c606120d P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:19:31.975237 16220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1df93779209748b9860f6d893dd3de53" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:32.006991 17847 leader_election.cc:290] T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:32.008582 17933 leader_election.cc:290] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:19:32.008898 17877 leader_election.cc:290] T 66f75124215f4ca6bef831e174a792d8 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:19:32.010177 17833 leader_election.cc:290] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 4 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
W20260709 13:19:32.010739 16294 consensus_meta.cc:287] T d2ab0ac8347641e6955e9bce7d90c1a2 P 90f3da57796b488c9087a2f523adf045: Time spent flushing consensus metadata: real 0.964s	user 0.009s	sys 0.000s
I20260709 13:19:32.015715 16147 raft_consensus.cc:2393] T 8d4ab870a05347868956aa8b841b2981 P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 3: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
I20260709 13:19:31.994508 17916 leader_election.cc:290] T d16dfad0c3b540e5b4b86492ddf87208 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:32.042474 16220 raft_consensus.cc:2468] T 1df93779209748b9860f6d893dd3de53 P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 2.
I20260709 13:19:32.043869 16220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7ca61592e848ea9b0fb81a2ce33cdc" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:32.044673 16220 raft_consensus.cc:2468] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 4.
I20260709 13:19:31.970785 17787 raft_consensus.cc:515] T 26354361e9b044cea1c2b86c4882fe92 P cb82cc7bd6db46d980c75237a1402860 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:32.045804 16220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8158bbd17b994d2eb96f560bd0f1b88d" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:32.047143 17787 leader_election.cc:290] T 26354361e9b044cea1c2b86c4882fe92 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:31.997004 16149 rpcz_store.cc:276] 0709 13:19:30.510838 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:30.937918 (+427080us) service_pool.cc:224] Handling call
0709 13:19:31.337066 (+399148us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:31.406968 (+ 69902us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:31.407013 (+    45us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:31.407057 (+    44us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:19:31.407122 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:31.966325 (+559203us) spinlock_profiling.cc:243] Waited 54.5 ms on lock 0x7b54000e9054. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:19:31.970133 (+  3808us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:31.970287 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":125444992}
I20260709 13:19:31.987717 17934 raft_consensus.cc:515] T 23f82224f4564fea93572e8206bad1fe P cb82cc7bd6db46d980c75237a1402860 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:32.048265 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e9b2bf7011f4ae688a79d7bb5a447b4" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:19:32.016904 17946 raft_consensus.cc:2749] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:19:32.040057 17804 leader_election.cc:290] T ee9ed67006144797b1de6ea8c2013a68 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:19:32.044013 17901 leader_election.cc:290] T 829b4ad8c5a54eababeea1181b960d4b P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:19:31.985953 17794 raft_consensus.cc:515] T 84f2da98864b4425bf7c0055951fdd4b P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:31.989108 17938 raft_consensus.cc:3060] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:19:32.049556 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62fe3346cd3c42c4a55e14af9073bca4" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:19:32.036417 17873 raft_consensus.cc:515] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:31.995615 17695 raft_consensus.cc:515] T 3884f864a96d428785895e5463e5728f P cb82cc7bd6db46d980c75237a1402860 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:32.001753 17936 raft_consensus.cc:515] T 02838c86531d4f98bf58d1c22b8baf77 P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:31.985226 16143 rpcz_store.cc:276] 0709 13:19:30.548308 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:31.858833 (+1310525us) service_pool.cc:224] Handling call
0709 13:19:31.873789 (+ 14956us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:31.919997 (+ 46208us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:31.948052 (+ 28055us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:32.017874 17866 raft_consensus.cc:493] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
W20260709 13:19:32.057673 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 63117.9us
I20260709 13:19:32.060271 17870 raft_consensus.cc:515] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } }
I20260709 13:19:32.100549 16048 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:49250 (request call id 318) took 4217 ms. Trace:
I20260709 13:19:32.101490 17868 leader_election.cc:290] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:19:32.102043 17870 leader_election.cc:290] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 4 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:32.051699 17644 leader_election.cc:290] T 8d4ab870a05347868956aa8b841b2981 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 3 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:32.051664 16220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c61a39fd4f84424b129c1a6f9ba404d" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:19:32.102896 16220 raft_consensus.cc:3060] T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:19:32.103091 16825 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49896 (request call id 724) took 1420 ms. Trace:
I20260709 13:19:32.105002 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada5a004ba7c46819051e3ccec71ffdb" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:32.050057 17934 leader_election.cc:290] T 23f82224f4564fea93572e8206bad1fe P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:32.070850 17866 raft_consensus.cc:515] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:32.072398 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66f75124215f4ca6bef831e174a792d8" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
W20260709 13:19:31.990705 16293 rpcz_store.cc:269] 0709 13:19:28.977591 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:29.015642 (+ 38051us) service_pool.cc:224] Handling call
0709 13:19:31.935905 (+2920263us) spinlock_profiling.cc:243] Waited 785 ms on lock 0x7b5400210054. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:19:31.944779 (+  8874us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1806244224}
I20260709 13:19:32.073282 16219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a959dbecc2648eb85d1cc550f5a3245" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:32.052520 16294 raft_consensus.cc:2468] T d2ab0ac8347641e6955e9bce7d90c1a2 P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 4.
I20260709 13:19:32.072154 17803 consensus_queue.cc:260] T ada5a004ba7c46819051e3ccec71ffdb P cb82cc7bd6db46d980c75237a1402860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:32.075484 17794 leader_election.cc:290] T 84f2da98864b4425bf7c0055951fdd4b P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:19:32.077008 17936 leader_election.cc:290] T 02838c86531d4f98bf58d1c22b8baf77 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 4 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:32.078047 17938 raft_consensus.cc:515] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } }
I20260709 13:19:32.081615 16816 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49896 (request call id 723) took 1399 ms. Trace:
W20260709 13:19:32.082677 16226 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33050 (request call id 313) took 3303 ms (client timeout 2116 ms). Trace:
I20260709 13:19:32.083559 16217 raft_consensus.cc:2368] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 for earlier term 3. Current term is 4.
I20260709 13:19:32.084404 17873 leader_election.cc:290] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 4 election: Requested vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:19:32.085811 17695 leader_election.cc:290] T 3884f864a96d428785895e5463e5728f P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:32.086539 16265 leader_election.cc:304] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:19:32.087142 16822 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49896 (request call id 722) took 1405 ms. Trace:
I20260709 13:19:32.105291 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdbd78acb4cf46298e0de8ba0d130e19" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:32.070334 16143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f82224f4564fea93572e8206bad1fe" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:19:32.104718 16825 rpcz_store.cc:276] 0709 13:19:30.682526 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:30.683708 (+  1182us) service_pool.cc:224] Handling call
0709 13:19:32.102399 (+1418691us) spinlock_profiling.cc:243] Waited 893 us on lock 0x7b5800004298. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69ad8fadb1 00007f69ada3ab71 00007f69ada3aa22 00007f69b6953c09 00007f69b5aadee0 00007f69b5b770ab 00007f69b5b77131 00007f69b5b38fc0 00007f69b5b23929 00007f69b5b233d2 00007f69af09abd4
0709 13:19:32.102592 (+   193us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:32.102763 (+   171us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:32.102787 (+    24us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:32.102843 (+    56us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:19:32.102875 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:32.102929 (+    54us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:32.103037 (+   108us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2052864}
I20260709 13:19:32.287755 16821 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49896 (request call id 721) took 1606 ms. Trace:
I20260709 13:19:32.105942 16299 raft_consensus.cc:2468] T ada5a004ba7c46819051e3ccec71ffdb P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 2.
W20260709 13:19:32.298642 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 212190us
W20260709 13:19:32.114537 16190 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:32.130482 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4f65cdf00b748bb963ee6e81ef43b79" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:32.299090 16300 raft_consensus.cc:2468] T cdbd78acb4cf46298e0de8ba0d130e19 P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 2.
I20260709 13:19:32.147256 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d16dfad0c3b540e5b4b86492ddf87208" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:32.301534 16298 raft_consensus.cc:2468] T d16dfad0c3b540e5b4b86492ddf87208 P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 1.
I20260709 13:19:32.171578 16295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a878999c0024d2abe517484180290fc" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:32.303406 16295 raft_consensus.cc:2468] T 8a878999c0024d2abe517484180290fc P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 1.
I20260709 13:19:32.299840 16225 raft_consensus.cc:2468] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 4.
W20260709 13:19:32.296365 16226 rpcz_store.cc:269] 0709 13:19:28.779794 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:28.779921 (+   127us) service_pool.cc:224] Handling call
0709 13:19:32.082471 (+3302550us) spinlock_profiling.cc:243] Waited 1.08 s on lock 0x7b5400245754. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:19:32.082597 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2495482240}
W20260709 13:19:32.295755 17896 log.cc:927] Time spent T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 2.824s	user 0.025s	sys 0.053s
I20260709 13:19:32.300159 16821 rpcz_store.cc:276] 0709 13:19:30.681502 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:30.693821 (+ 12319us) service_pool.cc:224] Handling call
0709 13:19:30.694799 (+   978us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:30.695190 (+   391us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:30.695243 (+    53us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:30.702891 (+  7648us) log.cc:844] Serialized 26 byte log entry
0709 13:19:32.103766 (+1400875us) spinlock_profiling.cc:243] Waited 10.9 ms on lock 0x7b180033e500. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b693ade1 00007f69b69475b1 00007f69ada54e07 00007f69b68afab9 00007f69ada7169b 00007f69ada6f93f 00007f69ada71c02 00007f69b06f920b 00007f69b06fed23 00007f69b06ff0d1 00007f69b0b1aa0f
0709 13:19:32.259689 (+155923us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:19:32.259748 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:32.265314 (+  5566us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:32.287449 (+ 22135us) raft_consensus.cc:1685] finished
0709 13:19:32.287469 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:32.287688 (+   219us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":15154,"raft.queue_time_us":15290,"raft.run_cpu_time_us":290,"raft.run_wall_time_us":35694,"spinlock_wait_cycles":24968960,"thread_start_us":155692,"threads_started":1,"wal-append.queue_time_us":1566609}
I20260709 13:19:32.307262 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d4ab870a05347868956aa8b841b2981" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:32.310622 16300 raft_consensus.cc:2468] T 8d4ab870a05347868956aa8b841b2981 P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 2.
I20260709 13:19:32.311553 16825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434bff3082d141838068f323bdb0d1ed" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:19:32.299594 16822 rpcz_store.cc:276] 0709 13:19:30.681845 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:30.683364 (+  1519us) service_pool.cc:224] Handling call
0709 13:19:32.075703 (+1392339us) spinlock_profiling.cc:243] Waited 4.96 ms on lock 0x7b5800004298. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69ad8fadb1 00007f69ada3ab71 00007f69ada3aa22 00007f69b6953c09 00007f69b5aadee0 00007f69b5b770ab 00007f69b5b77131 00007f69b5b38fc0 00007f69b5b23929 00007f69b5b233d2 00007f69af09abd4
0709 13:19:32.086638 (+ 10935us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:32.086808 (+   170us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:32.086834 (+    26us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:32.086870 (+    36us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:19:32.086899 (+    29us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:32.086950 (+    51us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:32.087077 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":11414272}
I20260709 13:19:32.312484 16821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdbd78acb4cf46298e0de8ba0d130e19" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:19:32.313390 16821 raft_consensus.cc:2368] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 for earlier term 3. Current term is 4.
I20260709 13:19:32.308991 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02838c86531d4f98bf58d1c22b8baf77" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:19:32.314275 16299 raft_consensus.cc:3060] T 02838c86531d4f98bf58d1c22b8baf77 P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:19:32.315456 17933 raft_consensus.cc:493] T 5a96373bb0e14dce8a266b2fc6c58507 P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:19:32.306720 17882 raft_consensus.cc:2749] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:19:32.316004 17933 raft_consensus.cc:515] T 5a96373bb0e14dce8a266b2fc6c58507 P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:32.316928 17803 raft_consensus.cc:3060] T ada5a004ba7c46819051e3ccec71ffdb P cb82cc7bd6db46d980c75237a1402860 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:19:32.318172 17933 leader_election.cc:290] T 5a96373bb0e14dce8a266b2fc6c58507 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:19:32.308583 17926 consensus_queue.cc:1048] T 68a6838d82ef4cac83507f906139d90f P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.007s
I20260709 13:19:32.321130 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a9de347c711477e87c582d003f36925" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:19:32.316453 17866 leader_election.cc:290] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
W20260709 13:19:32.308327 16294 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40474 (request call id 218) took 1602 ms (client timeout 1548 ms). Trace:
I20260709 13:19:32.319132 16821 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49892 (request call id 274) took 1499 ms. Trace:
W20260709 13:19:32.321961 16294 rpcz_store.cc:269] 0709 13:19:30.702385 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:30.790925 (+ 88540us) service_pool.cc:224] Handling call
0709 13:19:32.304452 (+1513527us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:32.322443 16821 rpcz_store.cc:276] 0709 13:19:30.814863 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:32.312183 (+1497320us) service_pool.cc:224] Handling call
0709 13:19:32.313832 (+  1649us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:32.323108 16143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b7d1ab519044eda2d5e04a07242f25" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:19:32.323390 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a378d7f796f94615b3b347119a9880fd" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:19:32.323742 16150 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40844 (request call id 1702) took 3367 ms. Trace:
I20260709 13:19:32.324126 16225 raft_consensus.cc:2393] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 in current term 3: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
I20260709 13:19:32.115222 17644 raft_consensus.cc:493] T 9e65676b21b748728ad39668cfe722ec P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:19:32.325114 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a378d7f796f94615b3b347119a9880fd" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:19:32.259915 17914 raft_consensus.cc:493] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:19:32.318674 17942 raft_consensus.cc:493] T ecc1e363230043fe9fef390f0b979f20 P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:19:32.325794 17914 raft_consensus.cc:515] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:32.326056 17942 raft_consensus.cc:515] T ecc1e363230043fe9fef390f0b979f20 P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } }
I20260709 13:19:32.146505 17863 raft_consensus.cc:493] T c0eab25ce4c64e018a041beeda2867db P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:19:32.327234 17863 raft_consensus.cc:515] T c0eab25ce4c64e018a041beeda2867db P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:32.328200 17942 leader_election.cc:290] T ecc1e363230043fe9fef390f0b979f20 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:19:32.325795 16225 raft_consensus.cc:2393] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 in current term 3: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
I20260709 13:19:32.329351 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a878999c0024d2abe517484180290fc" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:32.329502 17863 leader_election.cc:290] T c0eab25ce4c64e018a041beeda2867db P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:32.172299 17870 raft_consensus.cc:493] T 9c61a39fd4f84424b129c1a6f9ba404d P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:32.329875 16225 raft_consensus.cc:2468] T 8a878999c0024d2abe517484180290fc P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 2.
I20260709 13:19:32.330323 17870 raft_consensus.cc:515] T 9c61a39fd4f84424b129c1a6f9ba404d P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:32.330727 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18ad7e3af5504571a69d10c00750a01c" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:32.331780 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4dc6b0541ca431ebc757beac00521f1" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:32.332397 17870 leader_election.cc:290] T 9c61a39fd4f84424b129c1a6f9ba404d P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:32.235898 16292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4f65cdf00b748bb963ee6e81ef43b79" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:32.333261 16292 raft_consensus.cc:2468] T c4f65cdf00b748bb963ee6e81ef43b79 P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 4.
I20260709 13:19:32.323354 16821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7ca61592e848ea9b0fb81a2ce33cdc" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:19:32.325749 16817 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49896 (request call id 725) took 1643 ms. Trace:
I20260709 13:19:32.335011 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ab0ac8347641e6955e9bce7d90c1a2" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:19:32.324913 17644 raft_consensus.cc:515] T 9e65676b21b748728ad39668cfe722ec P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:32.324105 16150 rpcz_store.cc:276] 0709 13:19:28.956655 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:29.046041 (+ 89386us) service_pool.cc:224] Handling call
0709 13:19:29.046285 (+   244us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:30.020769 (+974484us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:32.323507 (+2302738us) spinlock_profiling.cc:243] Waited 878 ms on lock 0x7b540011add4. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:19:32.323654 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2018290048}
I20260709 13:19:32.313172 16822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74ca8140f35c4ae29e8a664f6bdfc829" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:19:32.191653 17892 raft_consensus.cc:493] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:19:32.307762 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66ae176961b74813a58bc15c9c4c6f2a" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
W20260709 13:19:32.335564 16825 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49892 (request call id 275) took 1518 ms (client timeout 1900 ms). Trace:
I20260709 13:19:32.337459 16821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f82224f4564fea93572e8206bad1fe" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:19:32.680701 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66f75124215f4ca6bef831e174a792d8" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:32.681155 17800 leader_election.cc:290] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:19:32.322293 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4f65cdf00b748bb963ee6e81ef43b79" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:19:32.682076 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ab0ac8347641e6955e9bce7d90c1a2" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:32.680676 16226 raft_consensus.cc:3060] T d2ab0ac8347641e6955e9bce7d90c1a2 P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:19:32.683224 16822 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49892 (request call id 273) took 1883 ms (client timeout 2162 ms). Trace:
I20260709 13:19:32.675259 16265 leader_election.cc:304] T 8d4ab870a05347868956aa8b841b2981 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:19:32.226594 16818 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49906 (request call id 408) took 3459 ms. Trace:
I20260709 13:19:32.345920 16151 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40844 (request call id 1688) took 3460 ms. Trace:
I20260709 13:19:32.693775 16145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cc959fb7f704fba9009c0595467d074" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:19:32.694667 16145 raft_consensus.cc:2468] T 8cc959fb7f704fba9009c0595467d074 P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 2.
I20260709 13:19:32.685214 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f82224f4564fea93572e8206bad1fe" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
W20260709 13:19:32.685731 16822 rpcz_store.cc:269] 0709 13:19:30.800252 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:32.312898 (+1512646us) service_pool.cc:224] Handling call
0709 13:19:32.683151 (+370253us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:19:32.685818 16825 rpcz_store.cc:269] 0709 13:19:30.817532 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:32.307417 (+1489885us) service_pool.cc:224] Handling call
0709 13:19:32.335506 (+ 28089us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:32.695995 16145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8158bbd17b994d2eb96f560bd0f1b88d" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:19:32.697399 16145 raft_consensus.cc:2468] T 8158bbd17b994d2eb96f560bd0f1b88d P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 3.
I20260709 13:19:32.324295 16143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3884f864a96d428785895e5463e5728f" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
W20260709 13:19:32.352175 16219 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57604 (request call id 1852) took 1549 ms (client timeout 1695 ms). Trace:
I20260709 13:19:32.346254 17938 leader_election.cc:290] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 3 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:32.688512 16218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806d7690c2b644959152074bb2cab0a7" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
W20260709 13:19:32.693827 17876 thread.cc:641] apply [worker] (thread pool) Time spent creating pthread: real 0.550s	user 0.011s	sys 0.019s
I20260709 13:19:32.721524 17800 raft_consensus.cc:2804] T 8d4ab870a05347868956aa8b841b2981 P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 13:19:32.721799 17876 thread.cc:608] apply [worker] (thread pool) Time spent starting thread: real 0.578s	user 0.011s	sys 0.019s
I20260709 13:19:32.369364 17803 raft_consensus.cc:493] T b87147731a68400e8cce9b879daa7c5d P cb82cc7bd6db46d980c75237a1402860 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:32.336491 16817 rpcz_store.cc:276] 0709 13:19:30.682840 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:30.687221 (+  4381us) service_pool.cc:224] Handling call
0709 13:19:32.325050 (+1637829us) spinlock_profiling.cc:243] Waited 4.73 ms on lock 0x7b5800004298. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69ad8fadb1 00007f69ada3ab71 00007f69ada3aa22 00007f69b6953c09 00007f69b5aadee0 00007f69b5b770ab 00007f69b5b77131 00007f69b5b38fc0 00007f69b5b23929 00007f69b5b233d2 00007f69af09abd4
0709 13:19:32.325233 (+   183us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:32.325413 (+   180us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:32.325442 (+    29us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:32.325492 (+    50us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:19:32.325528 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:32.325587 (+    59us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:32.325694 (+   107us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":10868096}
I20260709 13:19:32.682538 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c61a39fd4f84424b129c1a6f9ba404d" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:19:32.690171 17892 raft_consensus.cc:515] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:19:32.721225 16219 rpcz_store.cc:269] 0709 13:19:30.802645 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:32.073000 (+1270355us) service_pool.cc:224] Handling call
0709 13:19:32.340311 (+267311us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:19:32.722059 16218 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57604 (request call id 1857) took 1746 ms (client timeout 1778 ms). Trace:
I20260709 13:19:32.784107 17803 raft_consensus.cc:515] T b87147731a68400e8cce9b879daa7c5d P cb82cc7bd6db46d980c75237a1402860 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:32.784736 16143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a878999c0024d2abe517484180290fc" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
W20260709 13:19:32.784823 16218 rpcz_store.cc:269] 0709 13:19:30.975808 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:32.688274 (+1712466us) service_pool.cc:224] Handling call
0709 13:19:32.721996 (+ 33722us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:32.785710 16143 raft_consensus.cc:2393] T 8a878999c0024d2abe517484180290fc P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 2: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
I20260709 13:19:32.785941 17892 leader_election.cc:290] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:19:32.786171 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 4 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:32.786515 16143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18ad7e3af5504571a69d10c00750a01c" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:19:32.100947 16048 rpcz_store.cc:276] 0709 13:19:27.882831 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:27.883627 (+   796us) service_pool.cc:224] Handling call
0709 13:19:32.100457 (+4216830us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:19:28.216926 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:19:28.217445 (+   519us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:19:28.217474 (+    29us) write_op.cc:435] Acquired schema lock
0709 13:19:28.217492 (+    18us) tablet.cc:662] Decoding operations
0709 13:19:28.217805 (+   313us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:19:28.217849 (+    44us) write_op.cc:641] Partition lock acquired for write op
0709 13:19:28.217876 (+    27us) tablet.cc:685] Acquiring locks for 1 operations
0709 13:19:28.218054 (+   178us) tablet.cc:701] Row locks acquired
0709 13:19:28.218071 (+    17us) write_op.cc:260] PREPARE: finished
0709 13:19:28.218211 (+   140us) write_op.cc:270] Start()
0709 13:19:28.218314 (+   103us) write_op.cc:276] Timestamp: P: 1783603168218153 usec, L: 0
0709 13:19:28.218331 (+    17us) op_driver.cc:348] REPLICATION: starting
0709 13:19:28.218907 (+   576us) log.cc:844] Serialized 544 byte log entry
0709 13:19:29.102080 (+883173us) op_driver.cc:464] REPLICATION: finished
0709 13:19:29.696894 (+594814us) write_op.cc:301] APPLY: starting
0709 13:19:29.696984 (+    90us) tablet.cc:1367] starting BulkCheckPresence
0709 13:19:29.697124 (+   140us) tablet.cc:1370] finished BulkCheckPresence
0709 13:19:29.697247 (+   123us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:19:29.697477 (+   230us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:19:29.697659 (+   182us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0709 13:19:29.697723 (+    64us) write_op.cc:312] APPLY: finished
0709 13:19:29.697956 (+   233us) log.cc:844] Serialized 23 byte log entry
0709 13:19:29.703998 (+  6042us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:19:29.773349 (+ 69351us) write_op.cc:454] Released schema lock
0709 13:19:29.777918 (+  4569us) write_op.cc:341] FINISH: Updating metrics
0709 13:19:29.797102 (+ 19184us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:19:29.798402 (+  1300us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":816743,"reported_tablets":1,"child_traces":[["op",{"apply.queue_time_us":583182,"apply.run_cpu_time_us":2084,"apply.run_wall_time_us":89231,"num_ops":1,"prepare.queue_time_us":58351,"prepare.run_cpu_time_us":2215,"prepare.run_wall_time_us":2215,"replication_time_us":874488,"thread_start_us":310167,"threads_started":1}]]}
I20260709 13:19:32.787463 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c2095a5f4ad49408dbc7f69c606120d" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:32.722100 17800 raft_consensus.cc:493] T 8d4ab870a05347868956aa8b841b2981 P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:32.788410 17800 raft_consensus.cc:3060] T 8d4ab870a05347868956aa8b841b2981 P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:19:32.788800 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a96373bb0e14dce8a266b2fc6c58507" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:32.790311 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0eab25ce4c64e018a041beeda2867db" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
W20260709 13:19:32.545156 16115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 400878us
W20260709 13:19:32.677408 16147 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40830 (request call id 949) took 2124 ms (client timeout 2200 ms). Trace:
W20260709 13:19:32.800097 16147 rpcz_store.cc:269] 0709 13:19:30.553354 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:30.768060 (+214706us) service_pool.cc:224] Handling call
0709 13:19:32.677094 (+1909034us) spinlock_profiling.cc:243] Waited 1.05 s on lock 0x7b540041e6d4. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:19:32.677304 (+   210us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2407896320}
I20260709 13:19:32.801582 16147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a96373bb0e14dce8a266b2fc6c58507" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:19:32.802297 16147 raft_consensus.cc:2468] T 5a96373bb0e14dce8a266b2fc6c58507 P 647a343a542d4a139fe106c19095129b [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 1.
I20260709 13:19:32.804504 17901 raft_consensus.cc:493] T b87147731a68400e8cce9b879daa7c5d P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:32.805076 17901 raft_consensus.cc:515] T b87147731a68400e8cce9b879daa7c5d P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:32.805497 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e4bfb9090c8460eb14fa7e0d818e7f1" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:32.806968 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ad5dbfa99ee4b8d9e28be92b3f36db5" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:32.808005 17901 leader_election.cc:290] T b87147731a68400e8cce9b879daa7c5d P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:19:32.702072 16265 leader_election.cc:304] T 1df93779209748b9860f6d893dd3de53 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
W20260709 13:19:32.808910 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 106945us
I20260709 13:19:32.809060 17942 raft_consensus.cc:2804] T 1df93779209748b9860f6d893dd3de53 P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:19:32.809473 17942 raft_consensus.cc:493] T 1df93779209748b9860f6d893dd3de53 P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:19:32.809828 17942 raft_consensus.cc:3060] T 1df93779209748b9860f6d893dd3de53 P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:19:32.810021 16265 leader_election.cc:304] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:19:32.811446 17933 raft_consensus.cc:2804] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:19:32.811514 16265 leader_election.cc:304] T c4f65cdf00b748bb963ee6e81ef43b79 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:19:32.812724 16265 leader_election.cc:304] T 8a878999c0024d2abe517484180290fc P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:19:32.813441 16226 raft_consensus.cc:2468] T d2ab0ac8347641e6955e9bce7d90c1a2 P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 4.
I20260709 13:19:32.813814 17882 raft_consensus.cc:2804] T c4f65cdf00b748bb963ee6e81ef43b79 P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:19:32.814211 17882 raft_consensus.cc:493] T c4f65cdf00b748bb963ee6e81ef43b79 P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:32.814438 17918 raft_consensus.cc:493] T af57996a8be84d398f0f023e8d08ef77 P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:19:32.815339 17926 raft_consensus.cc:2804] T 8a878999c0024d2abe517484180290fc P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:19:32.707523 16266 leader_election.cc:400] T cdbd78acb4cf46298e0de8ba0d130e19 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 3 pre-election: Vote denied by peer cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923) with higher term. Message: Invalid argument: T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 for earlier term 3. Current term is 4.
I20260709 13:19:32.815780 16266 leader_election.cc:403] T cdbd78acb4cf46298e0de8ba0d130e19 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 3 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:19:31.495908 15046 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:19:32.717295 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26354361e9b044cea1c2b86c4882fe92" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:19:32.692813 16818 rpcz_store.cc:276] 0709 13:19:28.731890 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:28.733413 (+  1523us) service_pool.cc:224] Handling call
0709 13:19:28.763765 (+ 30352us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:30.084911 (+1321146us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:30.084941 (+    30us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:30.084984 (+    43us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:19:30.085011 (+    27us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:32.115353 (+2030342us) spinlock_profiling.cc:243] Waited 1.32 s on lock 0x7b54004b8cd4. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:19:32.121808 (+  6455us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:32.191126 (+ 69318us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3028620672}
I20260709 13:19:32.319079 17794 consensus_queue.cc:1048] T b87147731a68400e8cce9b879daa7c5d P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.412s
I20260709 13:19:32.817185 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4dc6b0541ca431ebc757beac00521f1" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:19:32.818235 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434bff3082d141838068f323bdb0d1ed" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:19:32.818398 17794 raft_consensus.cc:3060] T cdbd78acb4cf46298e0de8ba0d130e19 P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Advancing to term 4
I20260709 13:19:32.818742 16150 raft_consensus.cc:2468] T 434bff3082d141838068f323bdb0d1ed P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 3.
I20260709 13:19:32.819105 17857 raft_consensus.cc:493] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:19:32.819866 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57a744f5aaf944678a63469cbf514015" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:19:32.819819 17857 raft_consensus.cc:515] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:32.821368 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a878999c0024d2abe517484180290fc" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:19:32.822096 16150 raft_consensus.cc:2468] T 8a878999c0024d2abe517484180290fc P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 2.
I20260709 13:19:32.752009 16145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26354361e9b044cea1c2b86c4882fe92" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:19:32.790437 16143 raft_consensus.cc:2468] T 18ad7e3af5504571a69d10c00750a01c P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 2.
I20260709 13:19:32.791852 16149 raft_consensus.cc:3060] T 9c61a39fd4f84424b129c1a6f9ba404d P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:19:32.692401 17644 leader_election.cc:290] T 9e65676b21b748728ad39668cfe722ec P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:19:32.791894 17860 leader_election.cc:290] T 18ad7e3af5504571a69d10c00750a01c P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:19:32.798313 17800 raft_consensus.cc:515] T 8d4ab870a05347868956aa8b841b2981 P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:19:32.780634 16117 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 67892us
I20260709 13:19:32.398067 16819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62fe3346cd3c42c4a55e14af9073bca4" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:19:32.824033 16265 leader_election.cc:304] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:19:32.848345 16145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66f75124215f4ca6bef831e174a792d8" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
W20260709 13:19:32.848887 16789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 131906us
I20260709 13:19:32.828189 16147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a9de347c711477e87c582d003f36925" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:19:32.849817 16145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1df93779209748b9860f6d893dd3de53" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:19:32.304730 16816 rpcz_store.cc:276] 0709 13:19:30.682175 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:30.687300 (+  5125us) service_pool.cc:224] Handling call
0709 13:19:30.735679 (+ 48379us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:31.235504 (+499825us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:31.239984 (+  4480us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:31.272142 (+ 32158us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:19:31.274602 (+  2460us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:32.081267 (+806665us) spinlock_profiling.cc:243] Waited 303 ms on lock 0x7b54006dbfd4. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:19:32.081333 (+    66us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:32.081510 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":695903616}
I20260709 13:19:32.414901 16291 raft_consensus.cc:3060] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:19:32.498768 16221 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57604 (request call id 1849) took 1702 ms. Trace:
I20260709 13:19:32.673120 16223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d16dfad0c3b540e5b4b86492ddf87208" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:32.811980 17933 raft_consensus.cc:493] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:32.814535 17882 raft_consensus.cc:3060] T c4f65cdf00b748bb963ee6e81ef43b79 P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:19:32.815037 17918 raft_consensus.cc:515] T af57996a8be84d398f0f023e8d08ef77 P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:32.815735 17926 raft_consensus.cc:493] T 8a878999c0024d2abe517484180290fc P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:19:32.694350 16151 rpcz_store.cc:276] 0709 13:19:28.885485 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:28.885608 (+   123us) service_pool.cc:224] Handling call
0709 13:19:28.885915 (+   307us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:28.886254 (+   339us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:28.886282 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:28.923082 (+ 36800us) log.cc:844] Serialized 26 byte log entry
0709 13:19:29.056917 (+133835us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:19:29.056966 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:29.057061 (+    95us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:32.122084 (+3065023us) spinlock_profiling.cc:243] Waited 20.3 ms on lock 0x7b4400022b4c. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69ad8fadb1 00007f69ad998c85 00007f69ad9996a2 00007f69b070b1db 00007f69b0706aaf 00007f69b06fa86e 00007f69b06f9bb6 00007f69b06f97d8 00007f69b0703221 00007f69b07031d9 00007f69b0703169
0709 13:19:32.345623 (+223539us) raft_consensus.cc:1685] finished
0709 13:19:32.345644 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:32.345861 (+   217us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1479,"raft.run_cpu_time_us":167,"raft.run_wall_time_us":2231,"spinlock_wait_cycles":46592256,"thread_start_us":106585,"threads_started":1,"wal-append.queue_time_us":226248}
I20260709 13:19:32.817185 16818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3884f864a96d428785895e5463e5728f" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:19:32.421921 16046 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:58400 (request call id 187) took 4779 ms. Trace:
I20260709 13:19:32.420135 16148 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40844 (request call id 1697) took 3471 ms. Trace:
I20260709 13:19:32.644245 16299 raft_consensus.cc:2468] T 02838c86531d4f98bf58d1c22b8baf77 P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 4.
I20260709 13:19:32.502568 16220 raft_consensus.cc:2468] T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 3.
I20260709 13:19:31.913810 16222 rpcz_store.cc:276] 0709 13:19:30.815302 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:30.815450 (+   148us) service_pool.cc:224] Handling call
0709 13:19:30.815838 (+   388us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:30.816089 (+   251us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:19:31.842476 (+1026387us) log.cc:844] Serialized 14 byte log entry
0709 13:19:31.904952 (+ 62476us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:31.905010 (+    58us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:19:31.905051 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:31.905128 (+    77us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:31.906279 (+  1151us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":993568,"prepare.run_cpu_time_us":47,"prepare.run_wall_time_us":49,"raft.queue_time_us":4159,"raft.run_cpu_time_us":123,"raft.run_wall_time_us":1788,"thread_start_us":1028149,"threads_started":2}
I20260709 13:19:32.687359 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8638ee363273426db1393dd81e5c7b72" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:32.409570 16190 leader_election.cc:304] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:19:32.883141 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cc959fb7f704fba9009c0595467d074" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
W20260709 13:19:32.849865 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 166820us
I20260709 13:19:32.954435 16225 raft_consensus.cc:2468] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 3.
I20260709 13:19:32.917297 16147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2794af5b99984efebdee564035841cb0" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:19:32.349249 16297 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48674 (request call id 1922) took 1857 ms. Trace:
W20260709 13:19:32.956120 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 546773us
I20260709 13:19:32.956717 16298 raft_consensus.cc:3060] T 66ae176961b74813a58bc15c9c4c6f2a P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Advancing to term 4
I20260709 13:19:32.957129 16190 leader_election.cc:304] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:19:32.881687 16144 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46224 (request call id 206) took 2352 ms. Trace:
I20260709 13:19:32.823073 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d16dfad0c3b540e5b4b86492ddf87208" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:19:32.959324 16144 rpcz_store.cc:276] 0709 13:19:30.529425 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:31.020352 (+490927us) service_pool.cc:224] Handling call
0709 13:19:31.020716 (+   364us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:31.730638 (+709922us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:31.730665 (+    27us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:31.730704 (+    39us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:19:31.730735 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:32.881380 (+1150645us) spinlock_profiling.cc:243] Waited 710 ms on lock 0x7b5400073fd4. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:19:32.881435 (+    55us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:32.881614 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1632247680}
I20260709 13:19:32.880746 16824 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49896 (request call id 727) took 2195 ms. Trace:
I20260709 13:19:32.968668 16145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3884f864a96d428785895e5463e5728f" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
W20260709 13:19:32.903513 16820 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49892 (request call id 271) took 2889 ms (client timeout 3128 ms). Trace:
I20260709 13:19:32.956090 16297 rpcz_store.cc:276] 0709 13:19:30.491475 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:30.491600 (+   125us) service_pool.cc:224] Handling call
0709 13:19:30.491925 (+   325us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:30.942918 (+450993us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:30.942949 (+    31us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:30.943006 (+    57us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:19:30.943042 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:32.348934 (+1405892us) spinlock_profiling.cc:243] Waited 451 ms on lock 0x7b540030bbd4. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:19:32.348997 (+    63us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:32.349149 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1036746112}
I20260709 13:19:32.970522 16222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87ac0d1f7e8a419f9bcbc04aee0e067e" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:32.970860 16190 leader_election.cc:304] T d16dfad0c3b540e5b4b86492ddf87208 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:19:32.971307 16144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:19:32.957913 16148 rpcz_store.cc:276] 0709 13:19:28.948627 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:29.035388 (+ 86761us) service_pool.cc:224] Handling call
0709 13:19:29.052104 (+ 16716us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:30.524647 (+1472543us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:32.419846 (+1895199us) spinlock_profiling.cc:243] Waited 1.07 s on lock 0x7b540028f354. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:19:32.420062 (+   216us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2470559488}
I20260709 13:19:32.971815 16190 leader_election.cc:304] T 8a878999c0024d2abe517484180290fc P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:19:32.963519 16224 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57600 (request call id 950) took 1019 ms. Trace:
I20260709 13:19:32.973646 16224 rpcz_store.cc:276] 0709 13:19:31.943952 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:31.973690 (+ 29738us) service_pool.cc:224] Handling call
0709 13:19:32.882941 (+909251us) spinlock_profiling.cc:243] Waited 1.69 ms on lock 0x7b5800010b98. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69ad8fadb1 00007f69ada3ab71 00007f69ada3aa22 00007f69b6953c09 00007f69b5aadee0 00007f69b5b770ab 00007f69b5b77131 00007f69b5b250a0 00007f69b5b24984 00007f69af09f6d4 00007f69af09f662
0709 13:19:32.963451 (+ 80510us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3877632}
I20260709 13:19:32.966338 16147 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46224 (request call id 213) took 1047 ms. Trace:
I20260709 13:19:32.974623 16147 rpcz_store.cc:276] 0709 13:19:31.918705 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:32.850249 (+931544us) service_pool.cc:224] Handling call
0709 13:19:32.966284 (+116035us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:32.971946 16144 raft_consensus.cc:2468] T ecc1e363230043fe9fef390f0b979f20 P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 2.
I20260709 13:19:32.896561 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74ca8140f35c4ae29e8a664f6bdfc829" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:32.975920 16226 raft_consensus.cc:2468] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 2.
I20260709 13:19:32.976446 16226 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57600 (request call id 945) took 1063 ms. Trace:
I20260709 13:19:32.822986 17857 leader_election.cc:290] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:19:32.977454 16222 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57600 (request call id 944) took 1067 ms. Trace:
W20260709 13:19:32.969239 16820 rpcz_store.cc:269] 0709 13:19:29.992295 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:29.992423 (+   128us) service_pool.cc:224] Handling call
0709 13:19:32.881703 (+2889280us) spinlock_profiling.cc:243] Waited 291 ms on lock 0x7b540051b154. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:19:32.881834 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":668676992}
I20260709 13:19:32.982021 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02838c86531d4f98bf58d1c22b8baf77" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:32.982674 16225 raft_consensus.cc:2393] T 02838c86531d4f98bf58d1c22b8baf77 P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 4: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
W20260709 13:19:32.336784 16788 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:32.979280 16223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66ae176961b74813a58bc15c9c4c6f2a" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:32.979616 16190 leader_election.cc:304] T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:19:32.986445 16820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a9de347c711477e87c582d003f36925" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:19:32.987310 16825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66ae176961b74813a58bc15c9c4c6f2a" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:19:32.987543 16820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ab0ac8347641e6955e9bce7d90c1a2" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:19:32.987888 16825 raft_consensus.cc:2468] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 3.
I20260709 13:19:32.988135 16820 raft_consensus.cc:2368] T d2ab0ac8347641e6955e9bce7d90c1a2 P cb82cc7bd6db46d980c75237a1402860 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 for earlier term 3. Current term is 4.
I20260709 13:19:32.988525 16820 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49892 (request call id 278) took 1026 ms. Trace:
I20260709 13:19:32.988710 16825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b7d1ab519044eda2d5e04a07242f25" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:19:32.988821 16820 rpcz_store.cc:276] 0709 13:19:31.962067 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:32.987312 (+1025245us) service_pool.cc:224] Handling call
0709 13:19:32.988465 (+  1153us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:32.989502 16825 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49906 (request call id 424) took 1072 ms. Trace:
I20260709 13:19:32.989740 16820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2372981006d4b218215c28964dea623" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:19:32.989831 16825 rpcz_store.cc:276] 0709 13:19:31.917549 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:32.988481 (+1070932us) service_pool.cc:224] Handling call
0709 13:19:32.989447 (+   966us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:32.990813 16820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b7d1ab519044eda2d5e04a07242f25" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:19:32.991474 16825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66ae176961b74813a58bc15c9c4c6f2a" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:19:32.992043 16825 raft_consensus.cc:2468] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 3.
I20260709 13:19:32.992466 16825 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49906 (request call id 416) took 1110 ms. Trace:
I20260709 13:19:32.992733 16825 rpcz_store.cc:276] 0709 13:19:31.882325 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:32.991272 (+1108947us) service_pool.cc:224] Handling call
0709 13:19:32.992421 (+  1149us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:32.962896 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e4bfb9090c8460eb14fa7e0d818e7f1" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:32.995481 16044 catalog_manager.cc:5697] T 68a6838d82ef4cac83507f906139d90f P 90f3da57796b488c9087a2f523adf045 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "90f3da57796b488c9087a2f523adf045" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } health_report { overall_health: HEALTHY } } }
I20260709 13:19:32.996204 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:32.997519 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66ae176961b74813a58bc15c9c4c6f2a" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:32.997663 16044 catalog_manager.cc:5697] T e5ad4eb281564936a59f41e387a47156 P 90f3da57796b488c9087a2f523adf045 reported cstate change: term changed from 2 to 3, leader changed from 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130) to 90f3da57796b488c9087a2f523adf045 (127.14.177.131). New cstate: current_term: 3 leader_uuid: "90f3da57796b488c9087a2f523adf045" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } health_report { overall_health: HEALTHY } } }
I20260709 13:19:32.998708 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f2da98864b4425bf7c0055951fdd4b" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:32.617216 17909 consensus_queue.cc:237] T ecc1e363230043fe9fef390f0b979f20 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 1, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:32.962670 17933 raft_consensus.cc:3060] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:19:33.001258 16820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5d59407eb545b895d8d922e0dd69fd" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:19:33.002480 16823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746aabf856ab401c9ef5e859371e7b67" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:19:33.002701 16820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31969392ccdb43b491911a319e8bc7e1" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:19:32.977005 16819 raft_consensus.cc:2410] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac for term 5 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:19:33.003072 16823 raft_consensus.cc:2468] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 2.
I20260709 13:19:33.003324 16820 raft_consensus.cc:2468] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 1.
I20260709 13:19:33.003484 16819 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49906 (request call id 419) took 1116 ms. Trace:
W20260709 13:19:33.003610 16823 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49896 (request call id 728) took 2318 ms (client timeout 2130 ms). Trace:
W20260709 13:19:33.003962 16823 rpcz_store.cc:269] 0709 13:19:30.685215 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:30.704076 (+ 18861us) service_pool.cc:224] Handling call
0709 13:19:33.002335 (+2298259us) spinlock_profiling.cc:243] Waited 2.56 ms on lock 0x7b5800004298. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69ad8fadb1 00007f69ada3ab71 00007f69ada3aa22 00007f69b6953c09 00007f69b5aadee0 00007f69b5b770ab 00007f69b5b77131 00007f69b5b250a0 00007f69b5b24984 00007f69af09f6d4 00007f69af09f662
0709 13:19:33.003554 (+  1219us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5892352}
I20260709 13:19:33.003883 16819 rpcz_store.cc:276] 0709 13:19:31.887312 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:32.397731 (+510419us) service_pool.cc:224] Handling call
0709 13:19:33.003426 (+605695us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:33.005255 16823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57a744f5aaf944678a63469cbf514015" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:19:33.005524 16819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e4bfb9090c8460eb14fa7e0d818e7f1" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:19:33.005831 16823 raft_consensus.cc:2468] T 57a744f5aaf944678a63469cbf514015 P cb82cc7bd6db46d980c75237a1402860 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 1.
I20260709 13:19:33.006157 16819 raft_consensus.cc:2468] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P cb82cc7bd6db46d980c75237a1402860 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 1.
I20260709 13:19:33.006299 16823 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49892 (request call id 277) took 1055 ms. Trace:
I20260709 13:19:33.006669 16823 rpcz_store.cc:276] 0709 13:19:31.951204 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:33.005030 (+1053826us) service_pool.cc:224] Handling call
0709 13:19:33.006249 (+  1219us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:32.825984 17942 raft_consensus.cc:515] T 1df93779209748b9860f6d893dd3de53 P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:33.007038 16819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434bff3082d141838068f323bdb0d1ed" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:19:33.007741 16823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "829b4ad8c5a54eababeea1181b960d4b" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:19:33.007923 16821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66f75124215f4ca6bef831e174a792d8" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:19:33.008754 16821 raft_consensus.cc:2410] T 66f75124215f4ca6bef831e174a792d8 P cb82cc7bd6db46d980c75237a1402860 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac for term 4 because replica has last-logged OpId of term: 3 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 13:19:32.962644 17926 raft_consensus.cc:3060] T 8a878999c0024d2abe517484180290fc P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:19:33.009203 17942 leader_election.cc:290] T 1df93779209748b9860f6d893dd3de53 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 3 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:19:33.009359 16821 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49906 (request call id 423) took 1097 ms. Trace:
I20260709 13:19:33.009827 17942 raft_consensus.cc:2749] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:19:33.009874 16821 rpcz_store.cc:276] 0709 13:19:31.912223 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:33.007701 (+1095478us) service_pool.cc:224] Handling call
0709 13:19:33.009288 (+  1587us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:33.010644 17942 raft_consensus.cc:493] T 8638ee363273426db1393dd81e5c7b72 P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:32.979295 17758 raft_consensus.cc:2804] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:19:33.011464 17758 raft_consensus.cc:493] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:33.011826 17758 raft_consensus.cc:3060] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:19:33.012270 16822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e65676b21b748728ad39668cfe722ec" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:19:33.013708 16822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada5a004ba7c46819051e3ccec71ffdb" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:19:33.014401 16822 raft_consensus.cc:2468] T ada5a004ba7c46819051e3ccec71ffdb P cb82cc7bd6db46d980c75237a1402860 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 3.
I20260709 13:19:32.978909 17847 raft_consensus.cc:2804] T d16dfad0c3b540e5b4b86492ddf87208 P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:19:33.015698 17847 raft_consensus.cc:493] T d16dfad0c3b540e5b4b86492ddf87208 P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Starting leader election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:19:33.016069 17847 raft_consensus.cc:3060] T d16dfad0c3b540e5b4b86492ddf87208 P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:19:33.017225 16146 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40830 (request call id 950) took 2448 ms (client timeout 2242 ms). Trace:
W20260709 13:19:33.017622 16146 rpcz_store.cc:269] 0709 13:19:30.568473 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:30.868385 (+299912us) service_pool.cc:224] Handling call
0709 13:19:33.017017 (+2148632us) spinlock_profiling.cc:243] Waited 409 ms on lock 0x7b540028e454. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:19:33.017145 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":940696320}
I20260709 13:19:33.018303 16291 raft_consensus.cc:1275] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 13:19:33.018525 16219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9944d5e4b8c4109af9955c4f08c9095" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:32.982043 16222 rpcz_store.cc:276] 0709 13:19:31.910056 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:32.970281 (+1060225us) service_pool.cc:224] Handling call
0709 13:19:32.977375 (+  7094us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:32.979504 17794 raft_consensus.cc:2749] T cdbd78acb4cf46298e0de8ba0d130e19 P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Leader pre-election lost for term 3. Reason: Vote denied by peer cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923) with higher term. Message: Invalid argument: T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 for earlier term 3. Current term is 4.
I20260709 13:19:33.019580 16222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdbd78acb4cf46298e0de8ba0d130e19" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:32.971643 16824 rpcz_store.cc:276] 0709 13:19:30.684891 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:30.685002 (+   111us) service_pool.cc:224] Handling call
0709 13:19:32.879707 (+2194705us) spinlock_profiling.cc:243] Waited 3.17 ms on lock 0x7b5800004298. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69ad8fadb1 00007f69ada3ab71 00007f69ada3aa22 00007f69b6953c09 00007f69b5aadee0 00007f69b5b770ab 00007f69b5b77131 00007f69b5b38fc0 00007f69b5b23929 00007f69b5b233d2 00007f69af09abd4
0709 13:19:32.880017 (+   310us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:32.880294 (+   277us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:32.880332 (+    38us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:32.880394 (+    62us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:19:32.880438 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:32.880509 (+    71us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:32.880664 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":7299328}
I20260709 13:19:32.983048 16225 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57600 (request call id 949) took 1042 ms. Trace:
I20260709 13:19:32.985083 17911 raft_consensus.cc:2804] T 8a878999c0024d2abe517484180290fc P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:19:33.020620 16824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8638ee363273426db1393dd81e5c7b72" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:19:33.020870 17911 raft_consensus.cc:493] T 8a878999c0024d2abe517484180290fc P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:33.021263 17911 raft_consensus.cc:3060] T 8a878999c0024d2abe517484180290fc P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:19:33.021371 16824 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49906 (request call id 417) took 1138 ms. Trace:
I20260709 13:19:33.021487 16143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02838c86531d4f98bf58d1c22b8baf77" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:19:33.021970 16817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ab0ac8347641e6955e9bce7d90c1a2" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:19:33.022186 16143 raft_consensus.cc:2468] T 02838c86531d4f98bf58d1c22b8baf77 P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 3.
I20260709 13:19:33.022370 17803 leader_election.cc:290] T b87147731a68400e8cce9b879daa7c5d P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:33.022753 16817 raft_consensus.cc:2393] T d2ab0ac8347641e6955e9bce7d90c1a2 P cb82cc7bd6db46d980c75237a1402860 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 4: Already voted for candidate cb82cc7bd6db46d980c75237a1402860 in this term.
I20260709 13:19:33.003881 16820 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49892 (request call id 276) took 1056 ms. Trace:
I20260709 13:19:32.982378 16151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31969392ccdb43b491911a319e8bc7e1" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:19:32.951078 17844 raft_consensus.cc:493] T c2f5f89186204242bf377f7c42b2e2da P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:32.999310 16142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0eab25ce4c64e018a041beeda2867db" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:19:33.018658 16146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57a744f5aaf944678a63469cbf514015" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:19:33.019731 16219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdbd78acb4cf46298e0de8ba0d130e19" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:19:33.023142 16143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ad5dbfa99ee4b8d9e28be92b3f36db5" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:19:33.023211 16820 rpcz_store.cc:276] 0709 13:19:31.947867 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:33.002447 (+1054580us) service_pool.cc:224] Handling call
0709 13:19:33.003829 (+  1382us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:33.026993 16143 raft_consensus.cc:2468] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 2.
I20260709 13:19:33.014964 16822 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49906 (request call id 422) took 1105 ms. Trace:
I20260709 13:19:33.027411 16820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2372981006d4b218215c28964dea623" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:19:33.027567 16822 rpcz_store.cc:276] 0709 13:19:31.909648 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:33.013448 (+1103800us) service_pool.cc:224] Handling call
0709 13:19:33.014905 (+  1457us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:33.027964 16143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c74490c40243a88e62d1760b5cd814" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:19:33.028443 16820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdbd78acb4cf46298e0de8ba0d130e19" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:19:33.028566 16143 raft_consensus.cc:2468] T 08c74490c40243a88e62d1760b5cd814 P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 3.
I20260709 13:19:33.029110 16820 raft_consensus.cc:2393] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 4: Already voted for candidate cb82cc7bd6db46d980c75237a1402860 in this term.
I20260709 13:19:33.029500 16820 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49906 (request call id 420) took 1127 ms. Trace:
I20260709 13:19:33.029974 16820 rpcz_store.cc:276] 0709 13:19:31.902074 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:33.028218 (+1126144us) service_pool.cc:224] Handling call
0709 13:19:33.029434 (+  1216us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:33.020327 16225 rpcz_store.cc:276] 0709 13:19:31.940169 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:32.971218 (+1031049us) service_pool.cc:224] Handling call
0709 13:19:32.983004 (+ 11786us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:33.021838 16824 rpcz_store.cc:276] 0709 13:19:31.882533 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:33.020385 (+1137852us) service_pool.cc:224] Handling call
0709 13:19:33.021322 (+   937us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:33.024675 16146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a959dbecc2648eb85d1cc550f5a3245" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:19:33.029846 16822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66ae176961b74813a58bc15c9c4c6f2a" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:19:32.980039 16226 rpcz_store.cc:276] 0709 13:19:31.913149 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:32.896231 (+983082us) service_pool.cc:224] Handling call
0709 13:19:32.976393 (+ 80162us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:33.046717 16822 raft_consensus.cc:3060] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:19:33.029433 16143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e65676b21b748728ad39668cfe722ec" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:19:33.046751 16146 raft_consensus.cc:2468] T 5a959dbecc2648eb85d1cc550f5a3245 P 647a343a542d4a139fe106c19095129b [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 1.
I20260709 13:19:33.027448 16821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2794af5b99984efebdee564035841cb0" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
W20260709 13:19:33.047735 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 68194.2us
I20260709 13:19:33.047960 16821 raft_consensus.cc:2468] T 2794af5b99984efebdee564035841cb0 P cb82cc7bd6db46d980c75237a1402860 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 1.
I20260709 13:19:33.048095 16146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f2da98864b4425bf7c0055951fdd4b" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:19:33.048569 16821 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49892 (request call id 281) took 1076 ms. Trace:
I20260709 13:19:33.048678 16146 raft_consensus.cc:2468] T 84f2da98864b4425bf7c0055951fdd4b P 647a343a542d4a139fe106c19095129b [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 1.
I20260709 13:19:33.026209 16151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee9ed67006144797b1de6ea8c2013a68" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:19:33.049841 16146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62fe3346cd3c42c4a55e14af9073bca4" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:19:33.008920 16823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af57996a8be84d398f0f023e8d08ef77" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:19:33.051357 16146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87147731a68400e8cce9b879daa7c5d" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:19:33.052203 16146 raft_consensus.cc:2468] T b87147731a68400e8cce9b879daa7c5d P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 3.
I20260709 13:19:33.053229 16821 rpcz_store.cc:276] 0709 13:19:31.972108 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:33.010944 (+1038836us) service_pool.cc:224] Handling call
0709 13:19:33.048425 (+ 37481us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:33.054286 16821 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49906 (request call id 421) took 1149 ms. Trace:
I20260709 13:19:33.054579 16821 rpcz_store.cc:276] 0709 13:19:31.904586 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:33.053766 (+1149180us) service_pool.cc:224] Handling call
0709 13:19:33.053970 (+   204us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:33.054096 (+   126us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:33.054111 (+    15us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:33.054138 (+    27us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:19:33.054156 (+    18us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:33.054187 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:33.054251 (+    64us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:32.969822 17931 raft_consensus.cc:2804] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:19:32.971560 17644 raft_consensus.cc:493] T b6f07842d3aa47858df5665874c86f3a P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:33.057554 16790 leader_election.cc:304] T b87147731a68400e8cce9b879daa7c5d P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:19:33.063143 16223 raft_consensus.cc:2393] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 in current term 4: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
I20260709 13:19:33.064798 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c2095a5f4ad49408dbc7f69c606120d" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:19:32.977145 16220 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57600 (request call id 951) took 1007 ms. Trace:
I20260709 13:19:32.981151 16116 leader_election.cc:304] T 8158bbd17b994d2eb96f560bd0f1b88d P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:19:33.056255 16151 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46224 (request call id 218) took 1003 ms. Trace:
I20260709 13:19:33.076534 16151 rpcz_store.cc:276] 0709 13:19:32.052612 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:33.026018 (+973406us) service_pool.cc:224] Handling call
0709 13:19:33.056198 (+ 30180us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:19:33.076942 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 95842.2us
I20260709 13:19:33.076195 16150 raft_consensus.cc:2468] T 7c2095a5f4ad49408dbc7f69c606120d P 647a343a542d4a139fe106c19095129b [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 1.
I20260709 13:19:33.078119 16151 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46224 (request call id 208) took 2525 ms. Trace:
I20260709 13:19:33.078482 16151 rpcz_store.cc:276] 0709 13:19:30.553152 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:33.077364 (+2524212us) service_pool.cc:224] Handling call
0709 13:19:33.077656 (+   292us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:33.077810 (+   154us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:33.077832 (+    22us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:33.077871 (+    39us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:19:33.077898 (+    27us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:33.077979 (+    81us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:33.078070 (+    91us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:33.079514 16816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26354361e9b044cea1c2b86c4882fe92" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:19:33.079732 17800 leader_election.cc:290] T 8d4ab870a05347868956aa8b841b2981 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 3 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:19:33.080147 16816 raft_consensus.cc:2468] T 26354361e9b044cea1c2b86c4882fe92 P cb82cc7bd6db46d980c75237a1402860 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 1.
I20260709 13:19:33.080519 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ab0ac8347641e6955e9bce7d90c1a2" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:19:33.080662 16816 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49892 (request call id 282) took 1105 ms. Trace:
I20260709 13:19:33.081074 16226 raft_consensus.cc:2376] T d2ab0ac8347641e6955e9bce7d90c1a2 P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 4. Re-sending same reply.
I20260709 13:19:33.081148 17926 raft_consensus.cc:515] T 8a878999c0024d2abe517484180290fc P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:33.082154 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c61a39fd4f84424b129c1a6f9ba404d" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:33.082825 16226 raft_consensus.cc:2393] T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 3: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
I20260709 13:19:33.083452 17926 leader_election.cc:290] T 8a878999c0024d2abe517484180290fc P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 2 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:19:33.030570 17945 raft_consensus.cc:2804] T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:19:33.086822 17945 raft_consensus.cc:493] T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:33.023211 16817 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49906 (request call id 418) took 1139 ms. Trace:
I20260709 13:19:33.023942 16142 raft_consensus.cc:2468] T c0eab25ce4c64e018a041beeda2867db P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 2.
I20260709 13:19:33.011150 17942 raft_consensus.cc:515] T 8638ee363273426db1393dd81e5c7b72 P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:33.025524 16219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8638ee363273426db1393dd81e5c7b72" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:19:33.087896 16219 raft_consensus.cc:3060] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:19:33.084546 17644 raft_consensus.cc:515] T b6f07842d3aa47858df5665874c86f3a P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:33.088666 16142 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40844 (request call id 1703) took 1210 ms. Trace:
I20260709 13:19:33.023684 17844 raft_consensus.cc:515] T c2f5f89186204242bf377f7c42b2e2da P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:33.085839 17918 leader_election.cc:290] T af57996a8be84d398f0f023e8d08ef77 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:19:33.030112 16818 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49892 (request call id 279) took 1066 ms. Trace:
I20260709 13:19:33.003435 16218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c11074579b046d99e585c2818e18a2c" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:33.086376 16151 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46224 (request call id 210) took 2431 ms. Trace:
I20260709 13:19:33.048733 16190 leader_election.cc:304] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:19:33.085088 16823 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49906 (request call id 415) took 1203 ms. Trace:
I20260709 13:19:33.089885 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746aabf856ab401c9ef5e859371e7b67" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:33.019788 17794 raft_consensus.cc:493] T fffab26427e24d398e2695aad40ca134 P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:19:33.087270 17945 raft_consensus.cc:3060] T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:19:33.062428 16146 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46224 (request call id 207) took 2513 ms. Trace:
I20260709 13:19:33.008769 16819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e65676b21b748728ad39668cfe722ec" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:19:33.023684 17914 leader_election.cc:290] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:19:33.414986 17951 raft_consensus.cc:493] T 829b4ad8c5a54eababeea1181b960d4b P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:19:33.415745 16819 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49892 (request call id 280) took 1447 ms. Trace:
I20260709 13:19:33.175323 16217 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57600 (request call id 964) took 1084 ms. Trace:
I20260709 13:19:33.415880 17951 raft_consensus.cc:515] T 829b4ad8c5a54eababeea1181b960d4b P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:33.109808 17933 raft_consensus.cc:515] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:33.416652 16217 rpcz_store.cc:276] 0709 13:19:32.086938 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:32.998460 (+911522us) service_pool.cc:224] Handling call
0709 13:19:33.170825 (+172365us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:33.417389 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5d59407eb545b895d8d922e0dd69fd" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:33.076165 16220 rpcz_store.cc:276] 0709 13:19:31.962916 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:32.046913 (+ 83997us) service_pool.cc:224] Handling call
0709 13:19:32.970150 (+923237us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:33.418279 17951 leader_election.cc:290] T 829b4ad8c5a54eababeea1181b960d4b P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:19:33.176853 16150 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46224 (request call id 209) took 2600 ms. Trace:
I20260709 13:19:33.418278 17933 leader_election.cc:290] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 5 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:19:32.958782 16046 rpcz_store.cc:276] 0709 13:19:27.648500 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:27.655771 (+  7271us) service_pool.cc:224] Handling call
0709 13:19:27.694145 (+ 38374us) spinlock_profiling.cc:243] Waited 7.7 ms on lock 0x7b5400002490. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69ad8fadb1 00007f69ada3ab71 00007f69ada3b0b2 00007f69b69b4a89 00007f69b694cda6 00007f69b691e088 00007f69b6a1c37e 00007f69b4edd2a4 00007f69b4edd232 00007f69b4edd161 00007f69b4edd0dc
0709 13:19:32.421846 (+4727701us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:19:28.013910 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:19:28.037106 (+ 23196us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:19:28.037144 (+    38us) write_op.cc:435] Acquired schema lock
0709 13:19:28.037163 (+    19us) tablet.cc:662] Decoding operations
0709 13:19:28.037516 (+   353us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:19:28.037571 (+    55us) write_op.cc:641] Partition lock acquired for write op
0709 13:19:28.037598 (+    27us) tablet.cc:685] Acquiring locks for 1 operations
0709 13:19:28.037785 (+   187us) tablet.cc:701] Row locks acquired
0709 13:19:28.037808 (+    23us) write_op.cc:260] PREPARE: finished
0709 13:19:28.037950 (+   142us) write_op.cc:270] Start()
0709 13:19:28.038057 (+   107us) write_op.cc:276] Timestamp: P: 1783603168037911 usec, L: 0
0709 13:19:28.038086 (+    29us) op_driver.cc:348] REPLICATION: starting
0709 13:19:28.038813 (+   727us) log.cc:844] Serialized 539 byte log entry
0709 13:19:28.834644 (+795831us) op_driver.cc:464] REPLICATION: finished
0709 13:19:29.615989 (+781345us) write_op.cc:301] APPLY: starting
0709 13:19:29.616984 (+   995us) tablet.cc:1367] starting BulkCheckPresence
0709 13:19:29.620202 (+  3218us) tablet.cc:1370] finished BulkCheckPresence
0709 13:19:29.689145 (+ 68943us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:19:29.689493 (+   348us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:19:29.689709 (+   216us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0709 13:19:29.689787 (+    78us) write_op.cc:312] APPLY: finished
0709 13:19:29.690045 (+   258us) log.cc:844] Serialized 23 byte log entry
0709 13:19:29.691091 (+  1046us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:19:29.691226 (+   135us) write_op.cc:454] Released schema lock
0709 13:19:29.691280 (+    54us) write_op.cc:341] FINISH: Updating metrics
0709 13:19:29.696439 (+  5159us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:19:29.696484 (+    45us) write_op.cc:454] Released schema lock
0709 13:19:32.121583 (+2425099us) spinlock_profiling.cc:243] Waited 56.5 ms on lock 0x7b18002d2440. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b693ade1 00007f69b69475b1 00007f69ada723f3 00007f69ada747c1 00007f69ada74779 00007f69ada74709 00007f69ada746d1 00007f69ada739cd 000000000038f354 000000000038f199 00007f69ada5583a
Metrics: {"mutex_wait_us":1506527,"reported_tablets":2,"spinlock_wait_cycles":17720192,"child_traces":[["op",{"apply.queue_time_us":677217,"apply.run_cpu_time_us":2677,"apply.run_wall_time_us":81366,"num_ops":1,"prepare.queue_time_us":159421,"prepare.run_cpu_time_us":12233,"prepare.run_wall_time_us":212738,"raft.queue_time_us":1120666,"raft.run_cpu_time_us":72160,"raft.run_wall_time_us":1236795,"replication_time_us":787971,"spinlock_wait_cycles":130145024,"thread_start_us":639002,"threads_started":4,"wal-append.queue_time_us":316768,"wal-append.run_cpu_time_us":30059,"wal-append.run_wall_time_us":2327711}]]}
W20260709 13:19:33.333521 16266 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.517s	user 0.009s	sys 0.020s
W20260709 13:19:33.266242 17894 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.576s	user 0.005s	sys 0.020s
W20260709 13:19:33.420164 16266 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.604s	user 0.013s	sys 0.021s
W20260709 13:19:33.420995 16266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 713633us
I20260709 13:19:33.111636 16147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2794af5b99984efebdee564035841cb0" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:19:33.160718 16144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d4ab870a05347868956aa8b841b2981" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:19:33.186403 16822 raft_consensus.cc:2468] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 4.
I20260709 13:19:33.063565 17931 raft_consensus.cc:493] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:33.255293 17900 raft_consensus.cc:2804] T b87147731a68400e8cce9b879daa7c5d P cb82cc7bd6db46d980c75237a1402860 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:19:33.166860 16298 raft_consensus.cc:2468] T 66ae176961b74813a58bc15c9c4c6f2a P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 4.
I20260709 13:19:33.175184 16148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02838c86531d4f98bf58d1c22b8baf77" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:19:33.104897 17860 raft_consensus.cc:2749] T 8158bbd17b994d2eb96f560bd0f1b88d P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:19:33.421873 17794 raft_consensus.cc:515] T fffab26427e24d398e2695aad40ca134 P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:33.110402 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0eab25ce4c64e018a041beeda2867db" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:33.422972 16144 raft_consensus.cc:2393] T 8d4ab870a05347868956aa8b841b2981 P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 3: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
I20260709 13:19:33.166738 16145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4f65cdf00b748bb963ee6e81ef43b79" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:19:33.106348 16223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c74490c40243a88e62d1760b5cd814" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
W20260709 13:19:33.144656 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 87179.5us
I20260709 13:19:33.422719 16147 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40830 (request call id 969) took 1137 ms. Trace:
I20260709 13:19:33.423884 16298 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48684 (request call id 1838) took 1277 ms. Trace:
I20260709 13:19:33.428413 16147 rpcz_store.cc:276] 0709 13:19:32.285015 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:33.046034 (+761019us) service_pool.cc:224] Handling call
0709 13:19:33.422664 (+376630us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:33.429014 16298 rpcz_store.cc:276] 0709 13:19:32.146189 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:32.307547 (+161358us) service_pool.cc:224] Handling call
0709 13:19:33.423830 (+1116283us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:19:33.433212 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 384524us
I20260709 13:19:33.434058 16151 rpcz_store.cc:276] 0709 13:19:30.654843 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:33.079338 (+2424495us) service_pool.cc:224] Handling call
0709 13:19:33.085927 (+  6589us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:33.086089 (+   162us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:33.086115 (+    26us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:33.086157 (+    42us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:19:33.086186 (+    29us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:33.086240 (+    54us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:33.086326 (+    86us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:19:33.169440 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 91842.1us
I20260709 13:19:33.436404 16116 leader_election.cc:304] T 84f2da98864b4425bf7c0055951fdd4b P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:19:33.436964 16823 rpcz_store.cc:276] 0709 13:19:31.882014 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:33.051540 (+1169526us) service_pool.cc:224] Handling call
0709 13:19:33.084612 (+ 33072us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:33.084789 (+   177us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:33.084815 (+    26us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:33.084854 (+    39us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:19:33.084886 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:33.084946 (+    60us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:33.085035 (+    89us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:33.424206 16266 leader_election.cc:304] T 8c11074579b046d99e585c2818e18a2c P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:19:33.437636 16146 rpcz_store.cc:276] 0709 13:19:30.548826 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:33.061627 (+2512801us) service_pool.cc:224] Handling call
0709 13:19:33.061907 (+   280us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:33.062091 (+   184us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:33.062114 (+    23us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:33.062155 (+    41us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:19:33.062186 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:33.062245 (+    59us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:33.062375 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:33.120245 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18ad7e3af5504571a69d10c00750a01c" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
W20260709 13:19:33.443118 16789 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:32.959941 16221 rpcz_store.cc:276] 0709 13:19:30.795972 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:30.799469 (+  3497us) service_pool.cc:224] Handling call
0709 13:19:32.497732 (+1698263us) spinlock_profiling.cc:243] Waited 3.21 ms on lock 0x7b5800010b98. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69ad8fadb1 00007f69ada3ab71 00007f69ada3aa22 00007f69b6953c09 00007f69b5aadee0 00007f69b5b770ab 00007f69b5b77131 00007f69b5b38fc0 00007f69b5b23929 00007f69b5b233d2 00007f69af09abd4
0709 13:19:32.498001 (+   269us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:32.498255 (+   254us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:32.498303 (+    48us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:32.498372 (+    69us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:19:32.498423 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:32.498509 (+    86us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:32.498680 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":7392000}
I20260709 13:19:33.419067 16150 rpcz_store.cc:276] 0709 13:19:30.576384 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:33.078249 (+2501865us) service_pool.cc:224] Handling call
0709 13:19:33.176216 (+ 97967us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:33.176426 (+   210us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:33.176475 (+    49us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:33.176531 (+    56us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:19:33.176574 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:33.176643 (+    69us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:33.176786 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:33.424518 16148 raft_consensus.cc:3060] T 02838c86531d4f98bf58d1c22b8baf77 P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:19:33.418074 16225 raft_consensus.cc:2468] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 3.
I20260709 13:19:33.447986 16790 leader_election.cc:304] T 66f75124215f4ca6bef831e174a792d8 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:19:33.449199 16145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d16dfad0c3b540e5b4b86492ddf87208" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:19:33.449450 16824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434bff3082d141838068f323bdb0d1ed" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:19:33.444262 16266 leader_election.cc:304] T 66ae176961b74813a58bc15c9c4c6f2a P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:19:33.063755 16143 raft_consensus.cc:2468] T 9e65676b21b748728ad39668cfe722ec P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 3.
W20260709 13:19:33.420553 17894 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.731s	user 0.005s	sys 0.024s
I20260709 13:19:33.448611 16116 leader_election.cc:304] T ecc1e363230043fe9fef390f0b979f20 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:19:33.455238 16266 leader_election.cc:400] T d2ab0ac8347641e6955e9bce7d90c1a2 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 3 pre-election: Vote denied by peer cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923) with higher term. Message: Invalid argument: T d2ab0ac8347641e6955e9bce7d90c1a2 P cb82cc7bd6db46d980c75237a1402860 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 for earlier term 3. Current term is 4.
I20260709 13:19:33.473026 16266 leader_election.cc:403] T d2ab0ac8347641e6955e9bce7d90c1a2 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 3 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:19:33.423353 17931 raft_consensus.cc:3060] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:19:33.448731 17857 raft_consensus.cc:493] T 68a6838d82ef4cac83507f906139d90f P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:33.474809 17800 raft_consensus.cc:2749] T d2ab0ac8347641e6955e9bce7d90c1a2 P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Leader pre-election lost for term 3. Reason: Vote denied by peer cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923) with higher term. Message: Invalid argument: T d2ab0ac8347641e6955e9bce7d90c1a2 P cb82cc7bd6db46d980c75237a1402860 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 for earlier term 3. Current term is 4.
I20260709 13:19:33.475150 17857 raft_consensus.cc:515] T 68a6838d82ef4cac83507f906139d90f P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:33.432320 16818 rpcz_store.cc:276] 0709 13:19:31.963929 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:32.816947 (+853018us) service_pool.cc:224] Handling call
0709 13:19:32.986263 (+169316us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:33.476666 16818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5718c623ebee4d2bb770f0f9ddb2f164" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:19:33.477275 17857 leader_election.cc:290] T 68a6838d82ef4cac83507f906139d90f P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:33.477931 16818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746aabf856ab401c9ef5e859371e7b67" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:19:33.478585 16818 raft_consensus.cc:2468] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 2.
I20260709 13:19:33.087332 16817 rpcz_store.cc:276] 0709 13:19:31.884008 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:33.021725 (+1137717us) service_pool.cc:224] Handling call
0709 13:19:33.023140 (+  1415us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:33.479709 16818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87147731a68400e8cce9b879daa7c5d" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:19:33.480615 16817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7ca61592e848ea9b0fb81a2ce33cdc" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:19:33.480841 17945 raft_consensus.cc:515] T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:33.481573 16818 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49896 (request call id 732) took 1396 ms. Trace:
I20260709 13:19:33.426689 17844 leader_election.cc:290] T c2f5f89186204242bf377f7c42b2e2da P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:33.482028 16818 rpcz_store.cc:276] 0709 13:19:32.085151 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:33.480699 (+1395548us) service_pool.cc:224] Handling call
0709 13:19:33.481000 (+   301us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:33.481201 (+   201us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:33.481233 (+    32us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:33.481283 (+    50us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:19:33.481327 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:33.481392 (+    65us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:33.481515 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:33.483135 17945 leader_election.cc:290] T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 3 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:33.487680 16147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2372981006d4b218215c28964dea623" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:19:33.496268 16266 leader_election.cc:304] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:19:33.459770 16824 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49896 (request call id 731) took 1375 ms. Trace:
I20260709 13:19:33.456580 16146 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40844 (request call id 1705) took 1578 ms. Trace:
I20260709 13:19:33.457638 16151 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40844 (request call id 1704) took 1579 ms. Trace:
I20260709 13:19:33.501549 16151 rpcz_store.cc:276] 0709 13:19:31.878400 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:33.456638 (+1578238us) service_pool.cc:224] Handling call
0709 13:19:33.457005 (+   367us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:33.457218 (+   213us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:33.457256 (+    38us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:33.457312 (+    56us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:19:33.457359 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:33.457434 (+    75us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:33.457567 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:33.488534 16147 raft_consensus.cc:2468] T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 2.
I20260709 13:19:33.503698 16148 raft_consensus.cc:2468] T 02838c86531d4f98bf58d1c22b8baf77 P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 4.
I20260709 13:19:33.504454 16147 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46224 (request call id 216) took 1564 ms. Trace:
I20260709 13:19:33.504798 16147 rpcz_store.cc:276] 0709 13:19:31.940335 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:33.503737 (+1563402us) service_pool.cc:224] Handling call
0709 13:19:33.504036 (+   299us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:33.504184 (+   148us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:33.504206 (+    22us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:33.504246 (+    40us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:19:33.504273 (+    27us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:33.504320 (+    47us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:33.504409 (+    89us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:33.505712 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2372981006d4b218215c28964dea623" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:33.506275 16226 raft_consensus.cc:2468] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 2.
I20260709 13:19:33.089039 16142 rpcz_store.cc:276] 0709 13:19:31.878163 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:33.087884 (+1209721us) service_pool.cc:224] Handling call
0709 13:19:33.088157 (+   273us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:33.088322 (+   165us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:33.088349 (+    27us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:33.088396 (+    47us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:19:33.088427 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:33.088501 (+    74us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:33.088613 (+   112us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:33.499239 16824 rpcz_store.cc:276] 0709 13:19:32.084831 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:33.458851 (+1374020us) service_pool.cc:224] Handling call
0709 13:19:33.459132 (+   281us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:33.459352 (+   220us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:33.459388 (+    36us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:33.459438 (+    50us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:19:33.459492 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:33.459560 (+    68us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:33.459706 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:33.513379 16150 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40844 (request call id 1708) took 1634 ms. Trace:
I20260709 13:19:33.449702 17951 raft_consensus.cc:2804] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:19:33.519701 16150 rpcz_store.cc:276] 0709 13:19:31.879278 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:33.497039 (+1617761us) service_pool.cc:224] Handling call
0709 13:19:33.512689 (+ 15650us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:33.512932 (+   243us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:33.512971 (+    39us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:33.513026 (+    55us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:19:33.513072 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:33.513150 (+    78us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:33.513301 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:33.081095 16816 rpcz_store.cc:276] 0709 13:19:31.975387 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:32.985327 (+1009940us) service_pool.cc:224] Handling call
0709 13:19:33.080603 (+ 95276us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:33.451388 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee9ed67006144797b1de6ea8c2013a68" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
W20260709 13:19:33.521229 17958 log.cc:927] Time spent T 8c11074579b046d99e585c2818e18a2c P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.129s	user 0.000s	sys 0.000s
I20260709 13:19:33.524434 17931 raft_consensus.cc:515] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:33.538647 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62fe3346cd3c42c4a55e14af9073bca4" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:33.539660 16225 raft_consensus.cc:2468] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 4.
I20260709 13:19:33.463156 17926 raft_consensus.cc:2764] T 66ae176961b74813a58bc15c9c4c6f2a P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Leader pre-election decision vote started in defunct term 2: won
I20260709 13:19:33.450111 17933 raft_consensus.cc:2804] T 8c11074579b046d99e585c2818e18a2c P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 13:19:33.499357 16788 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:33.529155 16223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b7d1ab519044eda2d5e04a07242f25" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:33.558547 16223 raft_consensus.cc:2468] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 2.
I20260709 13:19:33.535118 17951 raft_consensus.cc:493] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:33.536190 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c74490c40243a88e62d1760b5cd814" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:33.449585 17957 raft_consensus.cc:2749] T 84f2da98864b4425bf7c0055951fdd4b P 647a343a542d4a139fe106c19095129b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:19:33.534967 16824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "829b4ad8c5a54eababeea1181b960d4b" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:19:33.498494 16143 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40830 (request call id 960) took 1260 ms. Trace:
I20260709 13:19:33.446671 16819 rpcz_store.cc:276] 0709 13:19:31.968087 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:33.008543 (+1040456us) service_pool.cc:224] Handling call
0709 13:19:33.415569 (+407026us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:19:33.539335 17758 consensus_meta.cc:287] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac: Time spent flushing consensus metadata: real 0.526s	user 0.009s	sys 0.001s
I20260709 13:19:33.567931 16296 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48684 (request call id 1833) took 1423 ms. Trace:
I20260709 13:19:33.533536 17942 leader_election.cc:290] T 8638ee363273426db1393dd81e5c7b72 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:19:33.527724 16218 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33050 (request call id 327) took 1393 ms. Trace:
I20260709 13:19:33.589094 16223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9944d5e4b8c4109af9955c4f08c9095" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:33.562973 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c11074579b046d99e585c2818e18a2c" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:33.565187 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "829b4ad8c5a54eababeea1181b960d4b" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:33.568480 16296 rpcz_store.cc:276] 0709 13:19:32.144760 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:32.146838 (+  2078us) service_pool.cc:224] Handling call
0709 13:19:32.173415 (+ 26577us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:32.173824 (+   409us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:32.173862 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:32.228041 (+ 54179us) log.cc:844] Serialized 26 byte log entry
0709 13:19:32.389178 (+161137us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:19:32.389215 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:32.389281 (+    66us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:33.567620 (+1178339us) raft_consensus.cc:1685] finished
0709 13:19:33.567647 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:33.567857 (+   210us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":17405,"raft.run_cpu_time_us":213,"raft.run_wall_time_us":23417,"thread_start_us":148555,"threads_started":1,"wal-append.queue_time_us":1066768}
I20260709 13:19:33.590631 16226 raft_consensus.cc:2468] T 829b4ad8c5a54eababeea1181b960d4b P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 3.
I20260709 13:19:33.591885 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af57996a8be84d398f0f023e8d08ef77" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:33.592608 16226 raft_consensus.cc:2468] T af57996a8be84d398f0f023e8d08ef77 P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 2.
I20260709 13:19:33.594050 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ab0ac8347641e6955e9bce7d90c1a2" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:33.545619 16818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af57996a8be84d398f0f023e8d08ef77" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:19:33.547278 17933 raft_consensus.cc:493] T 8c11074579b046d99e585c2818e18a2c P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:19:33.596414 16223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5718c623ebee4d2bb770f0f9ddb2f164" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
W20260709 13:19:33.602684 16266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 106552us
I20260709 13:19:33.594879 16226 raft_consensus.cc:2376] T d2ab0ac8347641e6955e9bce7d90c1a2 P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 4. Re-sending same reply.
I20260709 13:19:33.613094 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7ca61592e848ea9b0fb81a2ce33cdc" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
W20260709 13:19:33.613375 16044 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.612s	user 0.013s	sys 0.020s
W20260709 13:19:33.613895 16044 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.613s	user 0.013s	sys 0.020s
W20260709 13:19:33.602052 16265 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.637s	user 0.001s	sys 0.021s
I20260709 13:19:33.532684 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87ac0d1f7e8a419f9bcbc04aee0e067e" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
W20260709 13:19:33.627810 16265 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.662s	user 0.001s	sys 0.026s
I20260709 13:19:33.501500 16146 rpcz_store.cc:276] 0709 13:19:31.878626 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:33.455428 (+1576802us) service_pool.cc:224] Handling call
0709 13:19:33.455848 (+   420us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:33.456109 (+   261us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:33.456153 (+    44us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:33.456217 (+    64us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:19:33.456283 (+    66us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:33.456362 (+    79us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:33.456507 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:33.604076 16223 raft_consensus.cc:2468] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 2.
I20260709 13:19:33.570113 17951 raft_consensus.cc:3060] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:19:33.566488 16143 rpcz_store.cc:276] 0709 13:19:32.238234 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:33.029206 (+790972us) service_pool.cc:224] Handling call
0709 13:19:33.495361 (+466155us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:19:33.607898 16263 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.643s	user 0.005s	sys 0.037s
W20260709 13:19:33.631453 16263 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.666s	user 0.006s	sys 0.037s
I20260709 13:19:33.632395 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdbd78acb4cf46298e0de8ba0d130e19" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
W20260709 13:19:33.608376 16825 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.525s	user 0.009s	sys 0.019s
I20260709 13:19:33.627112 16266 leader_election.cc:304] T 57a744f5aaf944678a63469cbf514015 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:19:33.613821 16226 raft_consensus.cc:2376] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 4. Re-sending same reply.
I20260709 13:19:33.604044 16217 raft_consensus.cc:2410] T ee9ed67006144797b1de6ea8c2013a68 P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 for term 4 because replica has last-logged OpId of term: 3 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 13:19:33.635497 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c11074579b046d99e585c2818e18a2c" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:33.636050 16266 leader_election.cc:304] T 31969392ccdb43b491911a319e8bc7e1 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:19:33.636835 16266 leader_election.cc:304] T 2794af5b99984efebdee564035841cb0 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:19:33.636984 16297 raft_consensus.cc:1240] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 4. Current term is 5. Ops: []
I20260709 13:19:33.638041 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57a744f5aaf944678a63469cbf514015" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:33.638622 16299 raft_consensus.cc:2468] T 57a744f5aaf944678a63469cbf514015 P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 1.
I20260709 13:19:33.635078 16296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31969392ccdb43b491911a319e8bc7e1" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:33.639633 16296 raft_consensus.cc:2468] T 31969392ccdb43b491911a319e8bc7e1 P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 1.
I20260709 13:19:33.640259 16223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806d7690c2b644959152074bb2cab0a7" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:19:33.641196 16223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 4 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:19:33.642098 16223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68a6838d82ef4cac83507f906139d90f" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:33.642554 16223 raft_consensus.cc:2468] T 68a6838d82ef4cac83507f906139d90f P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 4.
I20260709 13:19:33.643729 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a9de347c711477e87c582d003f36925" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:33.645241 16295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2794af5b99984efebdee564035841cb0" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:33.605157 17933 raft_consensus.cc:3060] T 8c11074579b046d99e585c2818e18a2c P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:19:33.634523 16825 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.551s	user 0.009s	sys 0.019s
I20260709 13:19:33.635347 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a378d7f796f94615b3b347119a9880fd" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:33.650755 16226 raft_consensus.cc:2393] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 in current term 3: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
I20260709 13:19:33.588083 17644 leader_election.cc:290] T b6f07842d3aa47858df5665874c86f3a P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:33.645802 16295 raft_consensus.cc:2468] T 2794af5b99984efebdee564035841cb0 P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 1.
I20260709 13:19:33.587884 16218 rpcz_store.cc:276] 0709 13:19:32.134059 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:33.003186 (+869127us) service_pool.cc:224] Handling call
0709 13:19:33.512793 (+509607us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:33.653689 17800 raft_consensus.cc:2804] T 31969392ccdb43b491911a319e8bc7e1 P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:19:33.647184 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdbd78acb4cf46298e0de8ba0d130e19" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:19:33.663331 16224 raft_consensus.cc:1275] T 68a6838d82ef4cac83507f906139d90f P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:19:33.647792 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26354361e9b044cea1c2b86c4882fe92" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:33.663966 16224 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57600 (request call id 947) took 1739 ms. Trace:
I20260709 13:19:33.664081 16298 raft_consensus.cc:2468] T 26354361e9b044cea1c2b86c4882fe92 P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 1.
I20260709 13:19:33.664350 16224 rpcz_store.cc:276] 0709 13:19:31.924283 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:33.660183 (+1735900us) service_pool.cc:224] Handling call
0709 13:19:33.660422 (+   239us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:33.663712 (+  3290us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:33.663903 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:33.665002 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3884f864a96d428785895e5463e5728f" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:33.665881 16298 raft_consensus.cc:2468] T 3884f864a96d428785895e5463e5728f P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 1.
I20260709 13:19:33.667544 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87147731a68400e8cce9b879daa7c5d" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:33.648265 16223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d4ab870a05347868956aa8b841b2981" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:19:33.629689 16225 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57600 (request call id 940) took 1856 ms. Trace:
I20260709 13:19:33.648378 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e4bfb9090c8460eb14fa7e0d818e7f1" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:33.669837 16223 raft_consensus.cc:2393] T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 3: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
I20260709 13:19:33.670503 16225 rpcz_store.cc:276] 0709 13:19:31.773163 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:33.540623 (+1767460us) service_pool.cc:224] Handling call
0709 13:19:33.629627 (+ 89004us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:33.638162 16266 leader_election.cc:304] T 26354361e9b044cea1c2b86c4882fe92 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:19:33.606617 17942 raft_consensus.cc:2804] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:19:33.678364 16224 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33050 (request call id 319) took 1546 ms. Trace:
I20260709 13:19:33.577067 17758 raft_consensus.cc:515] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:33.654613 16295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9944d5e4b8c4109af9955c4f08c9095" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:33.679646 16295 raft_consensus.cc:2468] T c9944d5e4b8c4109af9955c4f08c9095 P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 3.
I20260709 13:19:33.644294 16300 raft_consensus.cc:2468] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 3.
I20260709 13:19:33.655285 16296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ab0ac8347641e6955e9bce7d90c1a2" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:19:33.680485 16295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cc959fb7f704fba9009c0595467d074" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:33.683308 16295 raft_consensus.cc:2468] T 8cc959fb7f704fba9009c0595467d074 P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 2.
I20260709 13:19:33.683339 16297 raft_consensus.cc:2468] T cdbd78acb4cf46298e0de8ba0d130e19 P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 4.
I20260709 13:19:33.669811 16298 raft_consensus.cc:1275] T a378d7f796f94615b3b347119a9880fd P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:19:33.652619 17926 raft_consensus.cc:2804] T 57a744f5aaf944678a63469cbf514015 P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:19:33.682318 16296 raft_consensus.cc:2376] T d2ab0ac8347641e6955e9bce7d90c1a2 P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 4. Re-sending same reply.
I20260709 13:19:33.655285 16217 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33050 (request call id 335) took 1357 ms. Trace:
I20260709 13:19:33.659493 17964 raft_consensus.cc:2804] T 2794af5b99984efebdee564035841cb0 P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:19:33.662760 16291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e9b2bf7011f4ae688a79d7bb5a447b4" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:33.661587 17644 raft_consensus.cc:2749] T ecc1e363230043fe9fef390f0b979f20 P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:19:33.701350 16310 tablet_copy_service.cc:342] P 90f3da57796b488c9087a2f523adf045: Request end of tablet copy session cb82cc7bd6db46d980c75237a1402860-19baa7bc367844c288bca8f81439c413 received from {username='slave'} at 127.0.0.1:40474
I20260709 13:19:33.661597 17800 raft_consensus.cc:493] T 31969392ccdb43b491911a319e8bc7e1 P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:19:33.710106 16222 raft_consensus.cc:2393] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 in current term 4: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
I20260709 13:19:33.712677 16225 raft_consensus.cc:1240] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 2. Current term is 4. Ops: []
I20260709 13:19:33.675208 17942 raft_consensus.cc:493] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
W20260709 13:19:33.658917 17909 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.639s	user 0.007s	sys 0.012s
I20260709 13:19:33.846527 16295 raft_consensus.cc:1240] T 1df93779209748b9860f6d893dd3de53 P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 2. Current term is 3. Ops: []
I20260709 13:19:33.779793 16225 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33050 (request call id 320) took 1647 ms. Trace:
I20260709 13:19:33.796679 16296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5ad4eb281564936a59f41e387a47156" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:33.699534 17926 raft_consensus.cc:493] T 57a744f5aaf944678a63469cbf514015 P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:19:33.850147 16225 rpcz_store.cc:276] 0709 13:19:32.132087 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:33.679724 (+1547637us) service_pool.cc:224] Handling call
0709 13:19:33.712445 (+ 32721us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:33.779411 (+ 66966us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:33.779714 (+   303us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:19:33.710067 16292 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.729s	user 0.006s	sys 0.019s
I20260709 13:19:33.710105 17900 raft_consensus.cc:493] T b87147731a68400e8cce9b879daa7c5d P cb82cc7bd6db46d980c75237a1402860 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:19:33.707373 16818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8638ee363273426db1393dd81e5c7b72" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
W20260709 13:19:33.853632 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 412187us
I20260709 13:19:33.678996 16224 rpcz_store.cc:276] 0709 13:19:32.131800 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:33.677100 (+1545300us) service_pool.cc:224] Handling call
0709 13:19:33.677475 (+   375us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:33.677888 (+   413us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:33.677931 (+    43us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:33.678026 (+    95us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:19:33.678074 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:33.678155 (+    81us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:33.678296 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":5777}
I20260709 13:19:33.735329 17800 raft_consensus.cc:3060] T 31969392ccdb43b491911a319e8bc7e1 P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:19:33.713451 17794 leader_election.cc:290] T fffab26427e24d398e2695aad40ca134 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
W20260709 13:19:33.869468 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 421529us
I20260709 13:19:33.813625 17942 raft_consensus.cc:3060] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:19:33.810256 16310 tablet_copy_service.cc:434] P 90f3da57796b488c9087a2f523adf045: ending tablet copy session cb82cc7bd6db46d980c75237a1402860-19baa7bc367844c288bca8f81439c413 on tablet 19baa7bc367844c288bca8f81439c413 with peer cb82cc7bd6db46d980c75237a1402860
I20260709 13:19:33.847532 17965 raft_consensus.cc:493] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:33.885736 16296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a878999c0024d2abe517484180290fc" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:33.886533 16296 raft_consensus.cc:2468] T 8a878999c0024d2abe517484180290fc P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 2.
I20260709 13:19:33.876184 16225 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33050 (request call id 326) took 1742 ms. Trace:
I20260709 13:19:33.887543 16225 rpcz_store.cc:276] 0709 13:19:32.133756 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:33.851302 (+1717546us) service_pool.cc:224] Handling call
0709 13:19:33.875576 (+ 24274us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:33.875757 (+   181us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:33.875797 (+    40us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:33.875853 (+    56us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:19:33.875898 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:33.875973 (+    75us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:33.876116 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:19:33.815739 17909 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.796s	user 0.009s	sys 0.013s
I20260709 13:19:33.704414 16217 rpcz_store.cc:276] 0709 13:19:32.298273 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:33.417617 (+1119344us) service_pool.cc:224] Handling call
0709 13:19:33.653375 (+235758us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:19:33.883808 17882 consensus_meta.cc:287] T c4f65cdf00b748bb963ee6e81ef43b79 P 90f3da57796b488c9087a2f523adf045: Time spent flushing consensus metadata: real 0.899s	user 0.009s	sys 0.001s
I20260709 13:19:33.727778 17967 raft_consensus.cc:2804] T 26354361e9b044cea1c2b86c4882fe92 P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260709 13:19:33.856442 16292 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.883s	user 0.006s	sys 0.019s
I20260709 13:19:33.875257 16191 leader_election.cc:304] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
I20260709 13:19:33.878970 17931 leader_election.cc:290] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 4 election: Requested vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
W20260709 13:19:33.872334 16149 consensus_meta.cc:287] T 9c61a39fd4f84424b129c1a6f9ba404d P 647a343a542d4a139fe106c19095129b: Time spent flushing consensus metadata: real 0.946s	user 0.001s	sys 0.009s
I20260709 13:19:33.853708 17926 raft_consensus.cc:3060] T 57a744f5aaf944678a63469cbf514015 P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:19:33.892108 16296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18ad7e3af5504571a69d10c00750a01c" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
W20260709 13:19:33.888538 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 260325us
I20260709 13:19:33.921842 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ad5dbfa99ee4b8d9e28be92b3f36db5" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:33.922614 16297 raft_consensus.cc:2468] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 2.
I20260709 13:19:33.877492 17900 raft_consensus.cc:3060] T b87147731a68400e8cce9b879daa7c5d P cb82cc7bd6db46d980c75237a1402860 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:19:33.919775 16296 raft_consensus.cc:2468] T 18ad7e3af5504571a69d10c00750a01c P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 2.
I20260709 13:19:33.932658 17900 raft_consensus.cc:515] T b87147731a68400e8cce9b879daa7c5d P cb82cc7bd6db46d980c75237a1402860 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:33.902987 16192 leader_election.cc:304] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860
W20260709 13:19:33.904222 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 463354us
I20260709 13:19:33.905612 16191 leader_election.cc:304] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
I20260709 13:19:33.847849 16291 raft_consensus.cc:2468] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 2.
W20260709 13:19:33.914011 16789 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:33.916940 16295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68a6838d82ef4cac83507f906139d90f" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:33.881770 16790 leader_election.cc:304] T d2ab0ac8347641e6955e9bce7d90c1a2 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 
W20260709 13:19:33.942077 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 60370.2us
W20260709 13:19:33.939258 16117 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 527366us
I20260709 13:19:33.943559 17803 raft_consensus.cc:2804] T d2ab0ac8347641e6955e9bce7d90c1a2 P cb82cc7bd6db46d980c75237a1402860 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:19:33.944033 17803 raft_consensus.cc:697] T d2ab0ac8347641e6955e9bce7d90c1a2 P cb82cc7bd6db46d980c75237a1402860 [term 4 LEADER]: Becoming Leader. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:19:33.921185 16265 leader_election.cc:304] T 8cc959fb7f704fba9009c0595467d074 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:19:33.944916 17803 consensus_queue.cc:237] T d2ab0ac8347641e6955e9bce7d90c1a2 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:33.945294 16265 leader_election.cc:304] T 8158bbd17b994d2eb96f560bd0f1b88d P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac
I20260709 13:19:33.946691 16825 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49896 (request call id 729) took 3261 ms. Trace:
I20260709 13:19:33.937712 16296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434bff3082d141838068f323bdb0d1ed" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:33.947785 16296 raft_consensus.cc:2468] T 434bff3082d141838068f323bdb0d1ed P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 3.
I20260709 13:19:33.938367 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87ac0d1f7e8a419f9bcbc04aee0e067e" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:33.949581 16297 raft_consensus.cc:2410] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b for term 4 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:19:33.904369 17967 raft_consensus.cc:493] T 26354361e9b044cea1c2b86c4882fe92 P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 647a343a542d4a139fe106c19095129b)
W20260709 13:19:33.954039 16192 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 51176us
W20260709 13:19:33.954536 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 508500us
I20260709 13:19:33.905205 16294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fffab26427e24d398e2695aad40ca134" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:33.954773 16192 leader_election.cc:304] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:19:33.905510 16218 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33050 (request call id 325) took 1768 ms. Trace:
W20260709 13:19:33.955767 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 50249.4us
I20260709 13:19:33.958850 16117 leader_election.cc:304] T 746aabf856ab401c9ef5e859371e7b67 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac
I20260709 13:19:33.947232 16825 rpcz_store.cc:276] 0709 13:19:30.685543 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:33.060948 (+2375405us) service_pool.cc:224] Handling call
0709 13:19:33.064036 (+  3088us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:33.064279 (+   243us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:19:33.652945 (+588666us) log.cc:844] Serialized 14 byte log entry
0709 13:19:33.946219 (+293274us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:33.946289 (+    70us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:19:33.946349 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:33.946453 (+   104us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:33.946619 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":747286,"prepare.run_cpu_time_us":73,"prepare.run_wall_time_us":16701,"raft.queue_time_us":3221,"raft.run_cpu_time_us":82,"raft.run_wall_time_us":83,"thread_start_us":857089,"threads_started":2}
I20260709 13:19:33.919816 16816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada5a004ba7c46819051e3ccec71ffdb" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:19:33.918393 16149 raft_consensus.cc:2468] T 9c61a39fd4f84424b129c1a6f9ba404d P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 3.
I20260709 13:19:33.964385 16116 leader_election.cc:304] T e5ad4eb281564936a59f41e387a47156 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
I20260709 13:19:33.962873 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 4 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:19:33.967180 16297 raft_consensus.cc:3060] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:19:33.904273 17882 raft_consensus.cc:515] T c4f65cdf00b748bb963ee6e81ef43b79 P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:33.951468 17914 raft_consensus.cc:2749] T 66f75124215f4ca6bef831e174a792d8 P cb82cc7bd6db46d980c75237a1402860 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:19:33.897276 17965 raft_consensus.cc:515] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:33.994752 16116 leader_election.cc:304] T 8a878999c0024d2abe517484180290fc P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
I20260709 13:19:34.021546 16297 raft_consensus.cc:2468] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 4.
I20260709 13:19:34.025480 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0eab25ce4c64e018a041beeda2867db" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:34.026042 16297 raft_consensus.cc:2468] T c0eab25ce4c64e018a041beeda2867db P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 2.
I20260709 13:19:33.961897 17794 raft_consensus.cc:2804] T 8cc959fb7f704fba9009c0595467d074 P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:19:34.034668 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c61a39fd4f84424b129c1a6f9ba404d" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:34.035969 16297 raft_consensus.cc:2393] T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 3: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
W20260709 13:19:34.037463 16117 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 71179.6us
I20260709 13:19:33.968427 17894 raft_consensus.cc:2804] T 8158bbd17b994d2eb96f560bd0f1b88d P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:19:33.966799 17644 raft_consensus.cc:2804] T 746aabf856ab401c9ef5e859371e7b67 P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:19:33.957442 17967 raft_consensus.cc:3060] T 26354361e9b044cea1c2b86c4882fe92 P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:19:34.033634 16116 leader_election.cc:304] T 5a959dbecc2648eb85d1cc550f5a3245 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:19:34.057541 16295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806d7690c2b644959152074bb2cab0a7" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:19:33.969254 16816 raft_consensus.cc:3060] T ada5a004ba7c46819051e3ccec71ffdb P cb82cc7bd6db46d980c75237a1402860 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:19:33.986969 16788 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:33.968847 16292 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48684 (request call id 1839) took 1630 ms. Trace:
I20260709 13:19:34.069402 16298 raft_consensus.cc:1240] T c4f65cdf00b748bb963ee6e81ef43b79 P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 4. Current term is 5. Ops: []
I20260709 13:19:33.965553 16294 raft_consensus.cc:2468] T fffab26427e24d398e2695aad40ca134 P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 1.
I20260709 13:19:34.070719 16295 raft_consensus.cc:3060] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:19:33.986352 16149 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40830 (request call id 955) took 1784 ms (client timeout 1689 ms). Trace:
W20260709 13:19:34.060326 17847 consensus_meta.cc:287] T d16dfad0c3b540e5b4b86492ddf87208 P 1ca2725d290e47d68b1a4b609c9816ac: Time spent flushing consensus metadata: real 1.044s	user 0.011s	sys 0.001s
I20260709 13:19:34.021445 16291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d4ab870a05347868956aa8b841b2981" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:19:34.088738 17982 raft_consensus.cc:2804] T 8a878999c0024d2abe517484180290fc P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:19:34.069820 17758 leader_election.cc:290] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 4 election: Requested vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:19:34.070207 17644 raft_consensus.cc:493] T 746aabf856ab401c9ef5e859371e7b67 P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Starting leader election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
W20260709 13:19:34.105990 16192 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 151295us
I20260709 13:19:34.084126 17945 raft_consensus.cc:2804] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:19:34.074373 17919 raft_consensus.cc:2749] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:19:34.060936 16221 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33050 (request call id 330) took 1845 ms. Trace:
I20260709 13:19:34.084438 17918 consensus_queue.cc:1048] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.005s
I20260709 13:19:34.109135 17982 raft_consensus.cc:493] T 8a878999c0024d2abe517484180290fc P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:33.986023 17974 consensus_queue.cc:1048] T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.133s
W20260709 13:19:34.127606 16789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 186170us
I20260709 13:19:34.109392 17847 raft_consensus.cc:515] T d16dfad0c3b540e5b4b86492ddf87208 P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:34.056262 17794 raft_consensus.cc:493] T 8cc959fb7f704fba9009c0595467d074 P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
W20260709 13:19:34.138506 16222 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33050 (request call id 323) took 2005 ms (client timeout 2170 ms). Trace:
I20260709 13:19:34.092243 16217 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33050 (request call id 343) took 1099 ms. Trace:
I20260709 13:19:34.131491 16192 leader_election.cc:304] T d2ab0ac8347641e6955e9bce7d90c1a2 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860
W20260709 13:19:34.175520 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 231211us
I20260709 13:19:34.070725 17894 raft_consensus.cc:493] T 8158bbd17b994d2eb96f560bd0f1b88d P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:19:34.132030 17644 raft_consensus.cc:3060] T 746aabf856ab401c9ef5e859371e7b67 P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:19:34.193704 16291 raft_consensus.cc:2393] T 8d4ab870a05347868956aa8b841b2981 P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 3: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
I20260709 13:19:34.195309 16294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e65676b21b748728ad39668cfe722ec" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
W20260709 13:19:34.203047 16222 rpcz_store.cc:269] 0709 13:19:32.132943 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:33.019356 (+886413us) service_pool.cc:224] Handling call
0709 13:19:34.138203 (+1118847us) spinlock_profiling.cc:243] Waited 688 ms on lock 0x7b54001423d4. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:19:34.138410 (+   207us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1583048064}
W20260709 13:19:34.204066 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 247302us
I20260709 13:19:34.205334 16224 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33050 (request call id 331) took 1970 ms. Trace:
W20260709 13:19:34.275861 17911 consensus_meta.cc:287] T 8a878999c0024d2abe517484180290fc P 1ca2725d290e47d68b1a4b609c9816ac: Time spent flushing consensus metadata: real 1.254s	user 0.011s	sys 0.002s
W20260709 13:19:34.208730 16117 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 140804us
I20260709 13:19:34.189764 17931 raft_consensus.cc:2749] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:19:34.191188 16220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ad5dbfa99ee4b8d9e28be92b3f36db5" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
W20260709 13:19:34.092027 16149 rpcz_store.cc:269] 0709 13:19:32.191629 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:32.682271 (+490642us) service_pool.cc:224] Handling call
0709 13:19:33.975897 (+1293626us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:19:34.364123 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 418882us
I20260709 13:19:34.271138 17794 raft_consensus.cc:3060] T 8cc959fb7f704fba9009c0595467d074 P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:19:34.418107 16265 leader_election.cc:304] T 23f82224f4564fea93572e8206bad1fe P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
I20260709 13:19:34.419474 16265 leader_election.cc:304] T 3884f864a96d428785895e5463e5728f P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
I20260709 13:19:34.323549 16226 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57600 (request call id 942) took 2550 ms. Trace:
I20260709 13:19:34.194041 17982 raft_consensus.cc:3060] T 8a878999c0024d2abe517484180290fc P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:19:34.320815 17983 log.cc:927] Time spent T e5ad4eb281564936a59f41e387a47156 P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.246s	user 0.000s	sys 0.000s
I20260709 13:19:34.390725 16224 rpcz_store.cc:276] 0709 13:19:32.235900 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:33.874582 (+1638682us) service_pool.cc:224] Handling call
0709 13:19:33.874886 (+   304us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:33.875071 (+   185us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:19:33.984503 (+109432us) log.cc:844] Serialized 14 byte log entry
0709 13:19:34.204881 (+220378us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:34.204961 (+    80us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:19:34.205025 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:34.205114 (+    89us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:34.205267 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":457331,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":41,"raft.queue_time_us":85077,"spinlock_wait_cycles":24711936,"thread_start_us":328604,"threads_started":2}
W20260709 13:19:34.392694 16789 leader_election.cc:336] T d2ab0ac8347641e6955e9bce7d90c1a2 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389): Timed out: RequestConsensusVote RPC to 127.14.177.131:41389 timed out after 1.548s (SENT)
I20260709 13:19:34.393194 16295 raft_consensus.cc:2468] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 3.
W20260709 13:19:34.395479 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 425284us
I20260709 13:19:34.393250 16191 leader_election.cc:304] T 66f75124215f4ca6bef831e174a792d8 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
I20260709 13:19:34.435276 16190 leader_election.cc:304] T 02838c86531d4f98bf58d1c22b8baf77 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:19:34.397217 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87147731a68400e8cce9b879daa7c5d" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:33.959467 16218 rpcz_store.cc:276] 0709 13:19:32.133495 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:33.695138 (+1561643us) service_pool.cc:224] Handling call
0709 13:19:33.732360 (+ 37222us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:33.851007 (+118647us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:33.871311 (+ 20304us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:33.876923 (+  5612us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:19:33.882000 (+  5077us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:33.888353 (+  6353us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:33.901488 (+ 13135us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:19:34.398708 16150 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.860s	user 0.012s	sys 0.008s
W20260709 13:19:34.399446 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 365905us
I20260709 13:19:34.400342 16223 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33050 (request call id 324) took 2267 ms. Trace:
I20260709 13:19:34.396487 16148 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40830 (request call id 963) took 2128 ms. Trace:
I20260709 13:19:34.339869 17974 raft_consensus.cc:2749] T 5a959dbecc2648eb85d1cc550f5a3245 P 647a343a542d4a139fe106c19095129b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:19:34.201020 17945 raft_consensus.cc:697] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [term 4 LEADER]: Becoming Leader. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
W20260709 13:19:34.483155 16147 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.594s	user 0.014s	sys 0.015s
I20260709 13:19:34.420464 16226 rpcz_store.cc:276] 0709 13:19:31.773442 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:33.656510 (+1883068us) service_pool.cc:224] Handling call
0709 13:19:33.656798 (+   288us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:33.664064 (+  7266us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:33.664098 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:33.684285 (+ 20187us) log.cc:844] Serialized 26 byte log entry
0709 13:19:33.958277 (+273992us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:19:33.958341 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:33.958434 (+    93us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:34.323255 (+364821us) raft_consensus.cc:1685] finished
0709 13:19:34.323290 (+    35us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:34.323494 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":12244,"raft.run_cpu_time_us":191,"raft.run_wall_time_us":13611,"thread_start_us":246680,"threads_started":1,"wal-append.queue_time_us":451024}
W20260709 13:19:34.293447 17969 log.cc:927] Time spent T ecc1e363230043fe9fef390f0b979f20 P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 0.373s	user 0.000s	sys 0.001s
I20260709 13:19:34.177121 16221 rpcz_store.cc:276] 0709 13:19:32.215835 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:33.875259 (+1659424us) service_pool.cc:224] Handling call
0709 13:19:33.935779 (+ 60520us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:33.959029 (+ 23250us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:33.961432 (+  2403us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:33.964573 (+  3141us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:19:33.969064 (+  4491us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:33.986263 (+ 17199us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:34.056809 (+ 70546us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:19:34.338871 17971 log.cc:927] Time spent T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 0.234s	user 0.000s	sys 0.000s
W20260709 13:19:34.363646 17993 log.cc:927] Time spent T 8158bbd17b994d2eb96f560bd0f1b88d P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 0.235s	user 0.001s	sys 0.000s
I20260709 13:19:34.411906 17911 raft_consensus.cc:515] T 8a878999c0024d2abe517484180290fc P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:19:34.357496 16219 consensus_meta.cc:287] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac: Time spent flushing consensus metadata: real 1.269s	user 0.010s	sys 0.002s
W20260709 13:19:34.290115 16192 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 158728us
W20260709 13:19:34.338012 17973 log.cc:927] Time spent T b87147731a68400e8cce9b879daa7c5d P cb82cc7bd6db46d980c75237a1402860: Append to log took a long time: real 0.369s	user 0.000s	sys 0.001s
W20260709 13:19:34.439855 16150 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.901s	user 0.012s	sys 0.008s
I20260709 13:19:34.340530 17976 raft_consensus.cc:2749] T d2ab0ac8347641e6955e9bce7d90c1a2 P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:19:34.358362 17900 leader_election.cc:290] T b87147731a68400e8cce9b879daa7c5d P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 4 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:34.348225 17965 leader_election.cc:290] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
W20260709 13:19:34.495880 17995 log.cc:927] Time spent T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 0.066s	user 0.000s	sys 0.001s
W20260709 13:19:34.482725 16220 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57604 (request call id 1869) took 1514 ms (client timeout 1718 ms). Trace:
I20260709 13:19:34.355662 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f82224f4564fea93572e8206bad1fe" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:34.345469 16143 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40844 (request call id 1721) took 1181 ms. Trace:
I20260709 13:19:34.339948 17882 leader_election.cc:290] T c4f65cdf00b748bb963ee6e81ef43b79 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 5 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:19:34.469015 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5ad4eb281564936a59f41e387a47156" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:34.445312 16148 rpcz_store.cc:276] 0709 13:19:32.268229 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:33.504313 (+1236084us) service_pool.cc:224] Handling call
0709 13:19:33.504572 (+   259us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:33.504805 (+   233us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:19:33.934466 (+429661us) log.cc:844] Serialized 14 byte log entry
0709 13:19:34.396102 (+461636us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:34.396154 (+    52us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:19:34.396196 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:34.396264 (+    68us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:34.396399 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":161785,"prepare.run_cpu_time_us":65,"prepare.run_wall_time_us":4270,"thread_start_us":854931,"threads_started":2}
W20260709 13:19:34.469996 16790 leader_election.cc:336] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943): Timed out: RequestConsensusVote RPC to 127.14.177.130:35943 timed out after 1.877s (SENT)
I20260709 13:19:34.356184 17979 raft_consensus.cc:493] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:19:34.549198 17979 raft_consensus.cc:515] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:19:34.550935 17926 consensus_meta.cc:287] T 57a744f5aaf944678a63469cbf514015 P 90f3da57796b488c9087a2f523adf045: Time spent flushing consensus metadata: real 0.595s	user 0.007s	sys 0.000s
W20260709 13:19:34.476981 16190 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:34.481542 16789 leader_election.cc:304] T 57a744f5aaf944678a63469cbf514015 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 647a343a542d4a139fe106c19095129b
W20260709 13:19:34.484586 16147 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.596s	user 0.015s	sys 0.015s
W20260709 13:19:34.486157 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 44479.2us
W20260709 13:19:34.561512 16789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 80083.8us
I20260709 13:19:34.561888 16116 leader_election.cc:304] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: cb82cc7bd6db46d980c75237a1402860
I20260709 13:19:34.562198 16789 leader_election.cc:304] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 647a343a542d4a139fe106c19095129b
I20260709 13:19:34.530059 17965 raft_consensus.cc:2749] T 3884f864a96d428785895e5463e5728f P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:19:34.319113 18002 log.cc:927] Time spent T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860: Append to log took a long time: real 0.117s	user 0.000s	sys 0.000s
I20260709 13:19:34.270803 16217 rpcz_store.cc:276] 0709 13:19:33.032492 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:33.906899 (+874407us) service_pool.cc:224] Handling call
0709 13:19:33.958803 (+ 51904us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:33.969808 (+ 11005us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:33.977179 (+  7371us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:34.023505 (+ 46326us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:19:34.035375 (+ 11870us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:34.069415 (+ 34040us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:34.086353 (+ 16938us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:34.468892 16291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d4ab870a05347868956aa8b841b2981" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:34.505035 16824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdbd78acb4cf46298e0de8ba0d130e19" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:19:34.577327 16291 raft_consensus.cc:2393] T 8d4ab870a05347868956aa8b841b2981 P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 3: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
I20260709 13:19:34.510268 16825 raft_consensus.cc:1275] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860 [term 2 LEARNER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
W20260709 13:19:34.473415 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 80197.1us
I20260709 13:19:34.479691 17978 raft_consensus.cc:2804] T 02838c86531d4f98bf58d1c22b8baf77 P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Leader election won for term 4
W20260709 13:19:34.526046 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 106645us
I20260709 13:19:34.579492 16145 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40844 (request call id 1707) took 2700 ms. Trace:
I20260709 13:19:34.473017 18006 consensus_queue.cc:1048] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.035s
I20260709 13:19:34.585613 16265 leader_election.cc:304] T 5a96373bb0e14dce8a266b2fc6c58507 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac
W20260709 13:19:34.523772 17951 consensus_meta.cc:287] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac: Time spent flushing consensus metadata: real 0.874s	user 0.011s	sys 0.001s
W20260709 13:19:34.532641 16117 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 129272us
I20260709 13:19:34.502707 17847 leader_election.cc:290] T d16dfad0c3b540e5b4b86492ddf87208 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 2 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:34.590260 17965 raft_consensus.cc:2804] T 5a96373bb0e14dce8a266b2fc6c58507 P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:19:34.536569 16143 rpcz_store.cc:276] 0709 13:19:33.164293 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:33.651868 (+487575us) service_pool.cc:224] Handling call
0709 13:19:33.691285 (+ 39417us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:33.850931 (+159646us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:33.853959 (+  3028us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:33.876762 (+ 22803us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:19:33.879280 (+  2518us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:34.345113 (+465833us) spinlock_profiling.cc:243] Waited 105 ms on lock 0x7b54003d9fd4. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:19:34.345177 (+    64us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:34.345385 (+   208us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":240422272}
I20260709 13:19:34.545760 17945 consensus_queue.cc:237] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:34.590704 17951 raft_consensus.cc:515] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:19:34.548182 17942 consensus_meta.cc:287] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045: Time spent flushing consensus metadata: real 0.657s	user 0.008s	sys 0.002s
I20260709 13:19:34.548058 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ad5dbfa99ee4b8d9e28be92b3f36db5" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
W20260709 13:19:34.529418 16220 rpcz_store.cc:269] 0709 13:19:32.967928 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:33.418784 (+450856us) service_pool.cc:224] Handling call
0709 13:19:34.173682 (+754898us) spinlock_profiling.cc:243] Waited 21.4 ms on lock 0x7b5800010b98. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69ad8fadb1 00007f69ada3ab71 00007f69ada3aa22 00007f69b6953c09 00007f69b5aadee0 00007f69b5b770ab 00007f69b5b77131 00007f69b5b250a0 00007f69b5b24984 00007f69af09f6d4 00007f69af09f662
0709 13:19:34.480298 (+306616us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":49257856}
I20260709 13:19:34.541668 16790 leader_election.cc:304] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:19:34.411710 17870 raft_consensus.cc:2804] T e5ad4eb281564936a59f41e387a47156 P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:19:34.551414 17979 leader_election.cc:290] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:34.505503 16219 raft_consensus.cc:2468] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 4.
I20260709 13:19:34.621884 16224 raft_consensus.cc:1275] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:19:34.554136 16820 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49892 (request call id 288) took 2470 ms. Trace:
I20260709 13:19:34.556294 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87ac0d1f7e8a419f9bcbc04aee0e067e" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:34.630499 17998 raft_consensus.cc:2749] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:19:34.631138 16293 raft_consensus.cc:2410] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b for term 4 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:19:34.632170 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87147731a68400e8cce9b879daa7c5d" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:19:34.632328 16191 leader_election.cc:304] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: cb82cc7bd6db46d980c75237a1402860
I20260709 13:19:34.633356 17847 raft_consensus.cc:2804] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:19:34.633838 17847 raft_consensus.cc:493] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Starting leader election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:19:34.634207 17847 raft_consensus.cc:3060] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:19:34.079248 16292 rpcz_store.cc:276] 0709 13:19:32.336525 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:32.391334 (+ 54809us) service_pool.cc:224] Handling call
0709 13:19:32.523727 (+132393us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:32.524035 (+   308us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 13:19:32.719147 (+195112us) log.cc:844] Serialized 14 byte log entry
0709 13:19:33.920567 (+1201420us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:33.936156 (+ 15589us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:19:33.939554 (+  3398us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:33.958666 (+ 19112us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:33.966610 (+  7944us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":755388,"prepare.run_cpu_time_us":53,"prepare.run_wall_time_us":55,"raft.queue_time_us":1200,"raft.run_cpu_time_us":130,"raft.run_wall_time_us":4233,"spinlock_wait_cycles":62392576,"thread_start_us":1043131,"threads_started":2,"wal-append.queue_time_us":1084394}
I20260709 13:19:34.524314 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87147731a68400e8cce9b879daa7c5d" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:19:34.280046 17894 raft_consensus.cc:3060] T 8158bbd17b994d2eb96f560bd0f1b88d P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:19:34.562907 16116 leader_election.cc:304] T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: cb82cc7bd6db46d980c75237a1402860
I20260709 13:19:34.635929 16292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f5f89186204242bf377f7c42b2e2da" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
W20260709 13:19:34.564028 17800 consensus_meta.cc:287] T 31969392ccdb43b491911a319e8bc7e1 P 90f3da57796b488c9087a2f523adf045: Time spent flushing consensus metadata: real 0.679s	user 0.009s	sys 0.003s
I20260709 13:19:34.568995 16822 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49906 (request call id 427) took 1876 ms. Trace:
I20260709 13:19:34.636690 16292 raft_consensus.cc:2468] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 4.
I20260709 13:19:34.636736 16822 rpcz_store.cc:276] 0709 13:19:32.692259 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:33.484735 (+792476us) service_pool.cc:224] Handling call
0709 13:19:33.485145 (+   410us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:33.919402 (+434257us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:33.919441 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:33.921747 (+  2306us) log.cc:844] Serialized 26 byte log entry
0709 13:19:34.398972 (+477225us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:19:34.399031 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:34.399100 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:34.568742 (+169642us) raft_consensus.cc:1685] finished
0709 13:19:34.568767 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:34.568930 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":121796,"raft.run_cpu_time_us":124,"raft.run_wall_time_us":6446,"thread_start_us":802024,"threads_started":2,"wal-append.queue_time_us":645651}
I20260709 13:19:34.637957 16292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d4ab870a05347868956aa8b841b2981" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:19:34.571750 18020 raft_consensus.cc:2804] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:19:34.574283 17992 raft_consensus.cc:2804] T 57a744f5aaf944678a63469cbf514015 P cb82cc7bd6db46d980c75237a1402860 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:19:34.638757 16292 raft_consensus.cc:2393] T 8d4ab870a05347868956aa8b841b2981 P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 3: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
I20260709 13:19:34.638938 17992 raft_consensus.cc:493] T 57a744f5aaf944678a63469cbf514015 P cb82cc7bd6db46d980c75237a1402860 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:19:34.639415 17992 raft_consensus.cc:3060] T 57a744f5aaf944678a63469cbf514015 P cb82cc7bd6db46d980c75237a1402860 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:19:34.639674 18024 raft_consensus.cc:2804] T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:19:34.612138 16291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434bff3082d141838068f323bdb0d1ed" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:34.574434 16823 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49896 (request call id 730) took 2490 ms. Trace:
I20260709 13:19:34.640468 16291 raft_consensus.cc:2468] T 434bff3082d141838068f323bdb0d1ed P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 3.
I20260709 13:19:34.533083 16294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c74490c40243a88e62d1760b5cd814" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:34.536973 17976 raft_consensus.cc:2749] T 66f75124215f4ca6bef831e174a792d8 P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:19:34.575788 16821 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49906 (request call id 428) took 1883 ms. Trace:
I20260709 13:19:34.575953 16789 leader_election.cc:304] T 2794af5b99984efebdee564035841cb0 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 647a343a542d4a139fe106c19095129b
I20260709 13:19:34.575999 17898 raft_consensus.cc:2804] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:19:34.577420 16824 raft_consensus.cc:2393] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 4: Already voted for candidate cb82cc7bd6db46d980c75237a1402860 in this term.
I20260709 13:19:34.442830 16223 rpcz_store.cc:276] 0709 13:19:32.133208 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:33.681020 (+1547812us) service_pool.cc:224] Handling call
0709 13:19:33.681263 (+   243us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:33.686323 (+  5060us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:33.686370 (+    47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:33.686944 (+   574us) log.cc:844] Serialized 26 byte log entry
0709 13:19:33.950084 (+263140us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:19:34.399221 (+449137us) log.cc:844] Serialized 14 byte log entry
0709 13:19:34.400009 (+   788us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:34.400091 (+    82us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:34.400136 (+    45us) raft_consensus.cc:1685] finished
0709 13:19:34.400153 (+    17us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:34.400289 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4692,"prepare.queue_time_us":411263,"prepare.run_cpu_time_us":29,"prepare.run_wall_time_us":30,"raft.queue_time_us":17276,"raft.run_cpu_time_us":338,"raft.run_wall_time_us":16068,"thread_start_us":534846,"threads_started":2,"wal-append.queue_time_us":581491}
I20260709 13:19:34.585294 16145 rpcz_store.cc:276] 0709 13:19:31.879046 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:33.488885 (+1609839us) service_pool.cc:224] Handling call
0709 13:19:33.489257 (+   372us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:33.907746 (+418489us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:33.915995 (+  8249us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:33.922943 (+  6948us) log.cc:844] Serialized 26 byte log entry
0709 13:19:34.071733 (+148790us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:19:34.071777 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:34.071852 (+    75us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:34.579288 (+507436us) raft_consensus.cc:1685] finished
0709 13:19:34.579307 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:34.579427 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":212945,"raft.run_cpu_time_us":110,"raft.run_wall_time_us":110,"thread_start_us":566197,"threads_started":2,"wal-append.queue_time_us":652282}
I20260709 13:19:34.586838 16147 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40830 (request call id 978) took 2253 ms. Trace:
I20260709 13:19:34.640682 16823 rpcz_store.cc:276] 0709 13:19:32.084526 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:33.458048 (+1373522us) service_pool.cc:224] Handling call
0709 13:19:33.497715 (+ 39667us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:33.890919 (+393204us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:33.890957 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:33.919382 (+ 28425us) log.cc:844] Serialized 26 byte log entry
0709 13:19:34.094882 (+175500us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:19:34.148155 (+ 53273us) log.cc:844] Serialized 14 byte log entry
0709 13:19:34.148425 (+   270us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:34.148534 (+   109us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:34.574230 (+425696us) raft_consensus.cc:1685] finished
0709 13:19:34.574251 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:34.574378 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":25165,"prepare.run_cpu_time_us":69,"prepare.run_wall_time_us":69,"raft.queue_time_us":198056,"raft.run_cpu_time_us":256,"raft.run_wall_time_us":11307,"thread_start_us":525959,"threads_started":2,"wal-append.queue_time_us":632747}
I20260709 13:19:34.501875 17990 consensus_queue.cc:1059] T 3a7ca61592e848ea9b0fb81a2ce33cdc P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.050s
I20260709 13:19:34.591905 16298 raft_consensus.cc:2468] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 2.
I20260709 13:19:34.542532 16816 raft_consensus.cc:2468] T ada5a004ba7c46819051e3ccec71ffdb P cb82cc7bd6db46d980c75237a1402860 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 4.
W20260709 13:19:34.578470 18005 log.cc:927] Time spent T b6f07842d3aa47858df5665874c86f3a P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 0.051s	user 0.000s	sys 0.001s
I20260709 13:19:34.618590 16295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18ad7e3af5504571a69d10c00750a01c" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
W20260709 13:19:34.621948 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 151979us
I20260709 13:19:34.530611 17999 raft_consensus.cc:2749] T 23f82224f4564fea93572e8206bad1fe P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:19:34.622856 16820 rpcz_store.cc:276] 0709 13:19:32.084193 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:33.104496 (+1020303us) service_pool.cc:224] Handling call
0709 13:19:33.171902 (+ 67406us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:33.423849 (+251947us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:33.423880 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:33.424378 (+   498us) log.cc:844] Serialized 26 byte log entry
0709 13:19:33.709941 (+285563us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:19:33.709984 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:34.102831 (+392847us) spinlock_profiling.cc:243] Waited 252 ms on lock 0x7b54004e0cd4. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:19:34.102899 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:34.551520 (+448621us) raft_consensus.cc:1685] finished
0709 13:19:34.551542 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:34.554066 (+  2524us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1178,"raft.run_cpu_time_us":105,"raft.run_wall_time_us":106,"spinlock_wait_cycles":578644864,"thread_start_us":285200,"threads_started":1,"wal-append.queue_time_us":482614}
I20260709 13:19:34.619270 17844 consensus_queue.cc:1048] T ecc1e363230043fe9fef390f0b979f20 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.011s
W20260709 13:19:34.627612 16219 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33050 (request call id 337) took 2155 ms (client timeout 1963 ms). Trace:
I20260709 13:19:34.628373 17982 raft_consensus.cc:515] T 8a878999c0024d2abe517484180290fc P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:34.629329 16146 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46224 (request call id 224) took 1191 ms. Trace:
I20260709 13:19:34.590723 17965 raft_consensus.cc:493] T 5a96373bb0e14dce8a266b2fc6c58507 P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:19:34.629345 17909 raft_consensus.cc:2749] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:19:34.611003 17942 raft_consensus.cc:515] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:19:34.642241 16190 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:34.647763 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 84922.2us
I20260709 13:19:34.692718 16116 leader_election.cc:304] T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: cb82cc7bd6db46d980c75237a1402860
W20260709 13:19:34.651309 16789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 75458.3us
I20260709 13:19:34.693884 16789 leader_election.cc:304] T 3884f864a96d428785895e5463e5728f P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 647a343a542d4a139fe106c19095129b
I20260709 13:19:34.694283 16192 leader_election.cc:304] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:19:34.650779 16821 rpcz_store.cc:276] 0709 13:19:32.692539 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:33.486215 (+793676us) service_pool.cc:224] Handling call
0709 13:19:33.486623 (+   408us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:33.917855 (+431232us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:33.917882 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:33.918888 (+  1006us) log.cc:844] Serialized 26 byte log entry
0709 13:19:34.138773 (+219885us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:19:34.138810 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:34.138881 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:34.575573 (+436692us) raft_consensus.cc:1685] finished
0709 13:19:34.575592 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:34.575720 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":142010,"raft.run_cpu_time_us":149,"raft.run_wall_time_us":15620,"spinlock_wait_cycles":56192,"thread_start_us":650277,"threads_started":2,"wal-append.queue_time_us":653349}
I20260709 13:19:34.660758 17794 raft_consensus.cc:515] T 8cc959fb7f704fba9009c0595467d074 P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:34.662498 16265 leader_election.cc:304] T e4dc6b0541ca431ebc757beac00521f1 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
W20260709 13:19:34.657886 16115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 171030us
I20260709 13:19:34.669004 16294 raft_consensus.cc:2468] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 3.
I20260709 13:19:34.698956 16789 leader_election.cc:304] T c9944d5e4b8c4109af9955c4f08c9095 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:19:34.700126 16265 leader_election.cc:304] T 434bff3082d141838068f323bdb0d1ed P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: cb82cc7bd6db46d980c75237a1402860
I20260709 13:19:34.671120 16150 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40830 (request call id 980) took 1704 ms. Trace:
I20260709 13:19:34.701017 16789 leader_election.cc:304] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac
I20260709 13:19:34.701117 16150 rpcz_store.cc:276] 0709 13:19:32.967056 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:33.528505 (+561449us) service_pool.cc:224] Handling call
0709 13:19:33.538359 (+  9854us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:34.509792 (+971433us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:34.509825 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:34.510353 (+   528us) log.cc:844] Serialized 26 byte log entry
0709 13:19:34.597656 (+ 87303us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:19:34.597716 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:34.597825 (+   109us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:34.670823 (+ 72998us) raft_consensus.cc:1685] finished
0709 13:19:34.670851 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:34.671048 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":164535,"raft.run_cpu_time_us":177,"raft.run_wall_time_us":15744,"thread_start_us":1018325,"threads_started":2,"wal-append.queue_time_us":125559}
I20260709 13:19:34.635456 17964 raft_consensus.cc:493] T 2794af5b99984efebdee564035841cb0 P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 647a343a542d4a139fe106c19095129b)
W20260709 13:19:34.648496 16788 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:34.656078 16147 rpcz_store.cc:276] 0709 13:19:32.333435 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:33.520127 (+1186692us) service_pool.cc:224] Handling call
0709 13:19:33.520490 (+   363us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:33.713551 (+193061us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:33.713607 (+    56us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:33.795510 (+ 81903us) log.cc:844] Serialized 26 byte log entry
0709 13:19:34.560546 (+765036us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:19:34.560586 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:34.560646 (+    60us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:34.586559 (+ 25913us) raft_consensus.cc:1685] finished
0709 13:19:34.586590 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:34.586761 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":182178,"raft.run_cpu_time_us":82,"raft.run_wall_time_us":83,"thread_start_us":864747,"threads_started":2,"wal-append.queue_time_us":703116}
I20260709 13:19:34.664644 17998 raft_consensus.cc:2804] T 2794af5b99984efebdee564035841cb0 P cb82cc7bd6db46d980c75237a1402860 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:19:34.640105 18024 raft_consensus.cc:493] T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Starting leader election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:19:34.633405 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6f07842d3aa47858df5665874c86f3a" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:34.587023 17926 raft_consensus.cc:515] T 57a744f5aaf944678a63469cbf514015 P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:34.675662 16295 raft_consensus.cc:2468] T 18ad7e3af5504571a69d10c00750a01c P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 2.
I20260709 13:19:34.692003 17965 raft_consensus.cc:3060] T 5a96373bb0e14dce8a266b2fc6c58507 P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:19:34.683957 16146 rpcz_store.cc:276] 0709 13:19:33.437714 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:33.674310 (+236596us) service_pool.cc:224] Handling call
0709 13:19:34.628543 (+954233us) spinlock_profiling.cc:243] Waited 5.83 ms on lock 0x7b5800011198. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69ad8fadb1 00007f69ada3ab71 00007f69ada3aa22 00007f69b6953c09 00007f69b5aadee0 00007f69b5b770ab 00007f69b5b77131 00007f69b5b38fc0 00007f69b5b23929 00007f69b5b233d2 00007f69af09abd4
0709 13:19:34.628798 (+   255us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:34.628989 (+   191us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:34.629018 (+    29us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:34.629064 (+    46us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:19:34.629096 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:34.629155 (+    59us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:34.629273 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":13404416}
I20260709 13:19:34.655486 17898 raft_consensus.cc:493] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:19:34.638784 18020 raft_consensus.cc:493] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Starting leader election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:19:34.619515 18011 consensus_queue.cc:1048] T 19baa7bc367844c288bca8f81439c413 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.092s
I20260709 13:19:34.667681 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada5a004ba7c46819051e3ccec71ffdb" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
W20260709 13:19:34.683923 17967 consensus_meta.cc:287] T 26354361e9b044cea1c2b86c4882fe92 P 90f3da57796b488c9087a2f523adf045: Time spent flushing consensus metadata: real 0.593s	user 0.004s	sys 0.006s
I20260709 13:19:34.681061 16291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68a6838d82ef4cac83507f906139d90f" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:34.636718 17800 raft_consensus.cc:515] T 31969392ccdb43b491911a319e8bc7e1 P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:34.723095 16151 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40844 (request call id 1711) took 2843 ms. Trace:
I20260709 13:19:34.723776 16151 rpcz_store.cc:276] 0709 13:19:31.879924 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:33.503315 (+1623391us) service_pool.cc:224] Handling call
0709 13:19:33.503814 (+   499us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:33.887880 (+384066us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:33.887932 (+    52us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:33.906501 (+ 18569us) log.cc:844] Serialized 26 byte log entry
0709 13:19:34.396269 (+489768us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:19:34.396322 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:34.396420 (+    98us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:34.722813 (+326393us) raft_consensus.cc:1685] finished
0709 13:19:34.722844 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:34.723037 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":198572,"raft.run_cpu_time_us":79,"raft.run_wall_time_us":80,"thread_start_us":819992,"threads_started":2,"wal-append.queue_time_us":515774}
I20260709 13:19:34.705355 16265 leader_election.cc:304] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
I20260709 13:19:34.705621 16116 leader_election.cc:304] T 829b4ad8c5a54eababeea1181b960d4b P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: cb82cc7bd6db46d980c75237a1402860
I20260709 13:19:34.705417 16115 leader_election.cc:304] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
W20260709 13:19:34.706622 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 51540.9us
I20260709 13:19:34.710884 16299 raft_consensus.cc:2393] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 in current term 2: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
W20260709 13:19:34.681087 16790 leader_election.cc:336] T 8638ee363273426db1393dd81e5c7b72 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943): Timed out: RequestConsensusVote RPC to 127.14.177.130:35943 timed out after 1.963s (SENT)
W20260709 13:19:34.727070 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 46041us
I20260709 13:19:34.726969 16116 leader_election.cc:304] T af57996a8be84d398f0f023e8d08ef77 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: cb82cc7bd6db46d980c75237a1402860
I20260709 13:19:34.706997 17978 raft_consensus.cc:697] T 02838c86531d4f98bf58d1c22b8baf77 P 1ca2725d290e47d68b1a4b609c9816ac [term 4 LEADER]: Becoming Leader. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
W20260709 13:19:34.727821 16790 leader_election.cc:336] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943): Timed out: RequestConsensusVote RPC to 127.14.177.130:35943 timed out after 1.977s (SENT)
I20260709 13:19:34.728279 16790 leader_election.cc:304] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
W20260709 13:19:34.680516 16219 rpcz_store.cc:269] 0709 13:19:32.472158 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:33.025264 (+553106us) service_pool.cc:224] Handling call
0709 13:19:34.627534 (+1602270us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:34.732324 17957 raft_consensus.cc:2804] T 829b4ad8c5a54eababeea1181b960d4b P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:19:34.732677 16115 leader_election.cc:304] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:19:34.733768 16115 leader_election.cc:304] T b87147731a68400e8cce9b879daa7c5d P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860
I20260709 13:19:34.734575 17857 raft_consensus.cc:2804] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:19:34.735424 18012 raft_consensus.cc:2804] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:19:34.707360 17999 raft_consensus.cc:2749] T e4dc6b0541ca431ebc757beac00521f1 P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:19:34.706085 17964 raft_consensus.cc:3060] T 2794af5b99984efebdee564035841cb0 P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:19:34.738117 16115 leader_election.cc:304] T 434bff3082d141838068f323bdb0d1ed P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: cb82cc7bd6db46d980c75237a1402860
I20260709 13:19:34.741535 16296 raft_consensus.cc:1240] T 31969392ccdb43b491911a319e8bc7e1 P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 1. Current term is 2. Ops: []
I20260709 13:19:34.710201 17900 raft_consensus.cc:2804] T c9944d5e4b8c4109af9955c4f08c9095 P cb82cc7bd6db46d980c75237a1402860 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:19:34.710791 18024 raft_consensus.cc:3060] T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:19:34.732849 18000 log.cc:927] Time spent T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 0.077s	user 0.000s	sys 0.001s
I20260709 13:19:34.721216 16298 raft_consensus.cc:1240] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 3. Current term is 4. Ops: [3.4-3.4]
I20260709 13:19:34.707907 17998 raft_consensus.cc:493] T 2794af5b99984efebdee564035841cb0 P cb82cc7bd6db46d980c75237a1402860 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:19:34.706444 17990 raft_consensus.cc:2804] T 3884f864a96d428785895e5463e5728f P cb82cc7bd6db46d980c75237a1402860 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:19:34.710005 17991 raft_consensus.cc:2804] T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:19:34.820135 16298 raft_consensus.cc:1240] T 8cc959fb7f704fba9009c0595467d074 P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 2. Current term is 3. Ops: []
I20260709 13:19:34.717383 17898 raft_consensus.cc:3060] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:19:34.737804 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c74490c40243a88e62d1760b5cd814" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:34.741235 16265 leader_election.cc:304] T 18ad7e3af5504571a69d10c00750a01c P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac
I20260709 13:19:34.745854 16144 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40844 (request call id 1706) took 2867 ms. Trace:
I20260709 13:19:34.830185 16300 raft_consensus.cc:2468] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 3.
W20260709 13:19:34.827409 16192 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 133193us
W20260709 13:19:34.828541 16789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 127613us
I20260709 13:19:34.718817 18023 raft_consensus.cc:2804] T 434bff3082d141838068f323bdb0d1ed P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 13:19:34.833580 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 105785us
W20260709 13:19:34.834126 16790 leader_election.cc:336] T d2ab0ac8347641e6955e9bce7d90c1a2 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943): Timed out: RequestConsensusVote RPC to 127.14.177.130:35943 timed out after 2.116s (SENT)
W20260709 13:19:34.834791 16790 leader_election.cc:336] T 3a7ca61592e848ea9b0fb81a2ce33cdc P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943): Timed out: RequestConsensusVote RPC to 127.14.177.130:35943 timed out after 2.160s (SENT)
I20260709 13:19:34.720430 18020 raft_consensus.cc:3060] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:19:34.836335 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 95197.5us
I20260709 13:19:34.830350 16144 rpcz_store.cc:276] 0709 13:19:31.878835 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:33.450051 (+1571216us) service_pool.cc:224] Handling call
0709 13:19:33.450307 (+   256us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:33.703411 (+253104us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:19:33.721552 (+ 18141us) log.cc:844] Serialized 14 byte log entry
0709 13:19:33.896595 (+175043us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:33.896647 (+    52us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:19:33.896689 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:34.745485 (+848796us) spinlock_profiling.cc:243] Waited 253 ms on lock 0x7b54000e9f54. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:19:34.745568 (+    83us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:34.745765 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":19286,"prepare.queue_time_us":24295,"prepare.run_cpu_time_us":47,"prepare.run_wall_time_us":48,"raft.queue_time_us":89,"raft.run_cpu_time_us":121,"raft.run_wall_time_us":6701,"spinlock_wait_cycles":581530368,"thread_start_us":174624,"threads_started":1,"wal-append.queue_time_us":353487}
W20260709 13:19:34.838165 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 111475us
W20260709 13:19:34.838532 16115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 100477us
I20260709 13:19:34.735121 17857 raft_consensus.cc:697] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac [term 4 LEADER]: Becoming Leader. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:19:34.807636 17974 raft_consensus.cc:2804] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Leader election won for term 3
W20260709 13:19:34.816258 17933 consensus_meta.cc:287] T 8c11074579b046d99e585c2818e18a2c P 90f3da57796b488c9087a2f523adf045: Time spent flushing consensus metadata: real 1.151s	user 0.012s	sys 0.000s
I20260709 13:19:34.819443 17991 raft_consensus.cc:493] T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Starting leader election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:19:34.807291 17901 raft_consensus.cc:2749] T b87147731a68400e8cce9b879daa7c5d P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:19:34.820407 17990 raft_consensus.cc:493] T 3884f864a96d428785895e5463e5728f P cb82cc7bd6db46d980c75237a1402860 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:19:34.831691 17979 raft_consensus.cc:2804] T af57996a8be84d398f0f023e8d08ef77 P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:19:34.845026 16265 leader_election.cc:304] T 02838c86531d4f98bf58d1c22b8baf77 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:19:34.857158 16225 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57604 (request call id 1868) took 1889 ms. Trace:
W20260709 13:19:34.844249 16789 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:34.821043 17644 consensus_meta.cc:287] T 746aabf856ab401c9ef5e859371e7b67 P 647a343a542d4a139fe106c19095129b: Time spent flushing consensus metadata: real 0.520s	user 0.007s	sys 0.003s
I20260709 13:19:34.731110 17967 raft_consensus.cc:515] T 26354361e9b044cea1c2b86c4882fe92 P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:34.837239 18023 raft_consensus.cc:493] T 434bff3082d141838068f323bdb0d1ed P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Starting leader election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:19:34.861603 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a9de347c711477e87c582d003f36925" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:34.862577 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26354361e9b044cea1c2b86c4882fe92" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:34.841450 16115 leader_election.cc:304] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:19:34.814057 17800 leader_election.cc:290] T 31969392ccdb43b491911a319e8bc7e1 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 2 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:19:34.816134 17900 raft_consensus.cc:493] T c9944d5e4b8c4109af9955c4f08c9095 P cb82cc7bd6db46d980c75237a1402860 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:19:34.816560 17882 raft_consensus.cc:2749] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:19:34.835659 16293 raft_consensus.cc:2410] T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b for term 3 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:19:34.821596 17914 raft_consensus.cc:2749] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:19:34.819136 17911 leader_election.cc:290] T 8a878999c0024d2abe517484180290fc P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 3 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:34.857542 17979 raft_consensus.cc:493] T af57996a8be84d398f0f023e8d08ef77 P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Starting leader election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:19:34.732782 17957 raft_consensus.cc:493] T 829b4ad8c5a54eababeea1181b960d4b P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Starting leader election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:19:34.820281 17998 raft_consensus.cc:3060] T 2794af5b99984efebdee564035841cb0 P cb82cc7bd6db46d980c75237a1402860 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:19:34.846489 17999 raft_consensus.cc:2804] T 18ad7e3af5504571a69d10c00750a01c P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:19:34.858112 16225 rpcz_store.cc:276] 0709 13:19:32.967786 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:33.893618 (+925832us) service_pool.cc:224] Handling call
0709 13:19:33.918099 (+ 24481us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:33.918439 (+   340us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:33.918469 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:33.918888 (+   419us) log.cc:844] Serialized 26 byte log entry
0709 13:19:34.185917 (+267029us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:19:34.185955 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:34.186020 (+    65us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:34.856951 (+670931us) raft_consensus.cc:1685] finished
0709 13:19:34.856976 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:34.857099 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":20223,"raft.run_cpu_time_us":135,"raft.run_wall_time_us":5700,"thread_start_us":266751,"threads_started":1,"wal-append.queue_time_us":666860}
I20260709 13:19:34.849900 18014 raft_consensus.cc:2804] T 434bff3082d141838068f323bdb0d1ed P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:19:34.855357 17991 raft_consensus.cc:3060] T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:19:34.719566 18025 consensus_queue.cc:1059] T 1df93779209748b9860f6d893dd3de53 P 647a343a542d4a139fe106c19095129b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.152s
I20260709 13:19:34.855329 17951 leader_election.cc:290] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 5 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:34.857112 17990 raft_consensus.cc:3060] T 3884f864a96d428785895e5463e5728f P cb82cc7bd6db46d980c75237a1402860 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:19:34.863511 17966 log.cc:927] Time spent T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.157s	user 0.001s	sys 0.000s
W20260709 13:19:34.895203 18019 log.cc:927] Time spent T 8c11074579b046d99e585c2818e18a2c P cb82cc7bd6db46d980c75237a1402860: Append to log took a long time: real 0.189s	user 0.000s	sys 0.001s
I20260709 13:19:34.896677 16817 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49906 (request call id 425) took 2205 ms. Trace:
I20260709 13:19:34.897070 16817 rpcz_store.cc:276] 0709 13:19:32.691327 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:33.481688 (+790361us) service_pool.cc:224] Handling call
0709 13:19:33.482037 (+   349us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:33.884957 (+402920us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:33.887651 (+  2694us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:33.966530 (+ 78879us) log.cc:844] Serialized 26 byte log entry
0709 13:19:34.448453 (+481923us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:19:34.469312 (+ 20859us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:34.483220 (+ 13908us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:34.896445 (+413225us) raft_consensus.cc:1685] finished
0709 13:19:34.896477 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:34.896617 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":95636,"raft.run_cpu_time_us":212,"raft.run_wall_time_us":14219,"thread_start_us":715093,"threads_started":2,"wal-append.queue_time_us":593969}
I20260709 13:19:34.898702 16047 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:49246 (request call id 328) took 6844 ms. Trace:
I20260709 13:19:34.878039 16224 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33050 (request call id 344) took 1795 ms. Trace:
I20260709 13:19:34.875821 17974 raft_consensus.cc:697] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b [term 3 LEADER]: Becoming Leader. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:19:34.820395 17972 raft_consensus.cc:2804] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:19:34.879756 16115 leader_election.cc:304] T 18ad7e3af5504571a69d10c00750a01c P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac
I20260709 13:19:34.881161 17977 raft_consensus.cc:2804] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:19:34.881628 16789 leader_election.cc:304] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
W20260709 13:19:34.881999 18027 log.cc:927] Time spent T 8c11074579b046d99e585c2818e18a2c P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 0.063s	user 0.001s	sys 0.000s
I20260709 13:19:34.865950 17644 raft_consensus.cc:515] T 746aabf856ab401c9ef5e859371e7b67 P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:34.838757 18034 consensus_queue.cc:1048] T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.033s
I20260709 13:19:34.888554 17982 leader_election.cc:290] T 8a878999c0024d2abe517484180290fc P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 3 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:34.864696 18012 raft_consensus.cc:697] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [term 4 LEADER]: Becoming Leader. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:19:34.869036 16291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f82224f4564fea93572e8206bad1fe" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:34.872625 17900 raft_consensus.cc:3060] T c9944d5e4b8c4109af9955c4f08c9095 P cb82cc7bd6db46d980c75237a1402860 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:19:34.853765 17933 raft_consensus.cc:515] T 8c11074579b046d99e585c2818e18a2c P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:34.864256 18023 raft_consensus.cc:3060] T 434bff3082d141838068f323bdb0d1ed P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:19:34.873483 17800 raft_consensus.cc:2764] T 02838c86531d4f98bf58d1c22b8baf77 P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Leader pre-election decision vote started in defunct term 3: won
I20260709 13:19:34.888278 18014 raft_consensus.cc:493] T 434bff3082d141838068f323bdb0d1ed P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Starting leader election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:19:34.951823 18014 raft_consensus.cc:3060] T 434bff3082d141838068f323bdb0d1ed P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:19:34.879590 17978 consensus_queue.cc:237] T 02838c86531d4f98bf58d1c22b8baf77 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:34.879282 16297 raft_consensus.cc:1240] T 26354361e9b044cea1c2b86c4882fe92 P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 1. Current term is 2. Ops: []
I20260709 13:19:34.886195 17979 raft_consensus.cc:3060] T af57996a8be84d398f0f023e8d08ef77 P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:19:34.901566 16224 rpcz_store.cc:276] 0709 13:19:33.082760 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:34.453010 (+1370250us) service_pool.cc:224] Handling call
0709 13:19:34.621578 (+168568us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:34.622475 (+   897us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:34.877793 (+255318us) spinlock_profiling.cc:243] Waited 168 ms on lock 0x7b5400141ed0. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5e8ad 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:19:34.877961 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":386724608}
I20260709 13:19:34.878715 17857 consensus_queue.cc:237] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:34.905452 17972 raft_consensus.cc:697] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 [term 4 LEADER]: Becoming Leader. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:19:34.886736 17957 raft_consensus.cc:3060] T 829b4ad8c5a54eababeea1181b960d4b P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:19:34.885506 17999 raft_consensus.cc:493] T 18ad7e3af5504571a69d10c00750a01c P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:19:34.840130 18021 consensus_queue.cc:1059] T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.067s
I20260709 13:19:34.899087 16047 rpcz_store.cc:276] 0709 13:19:28.054513 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:28.054644 (+   131us) service_pool.cc:224] Handling call
0709 13:19:28.405868 (+351224us) spinlock_profiling.cc:243] Waited 164 ms on lock 0x7b54003cdf54. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac9090 00007f69b0b67f3a 00007f69b691e1cc 00007f69b6a1c37e 00007f69b4edd2a4 00007f69b4edd232 00007f69b4edd161 00007f69b4edd0dc 00007f69b4edc392 00007f69aee9a48c 00007f69aee998c6
0709 13:19:34.898640 (+6492772us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:19:28.586239 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:19:28.617788 (+ 31549us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:19:28.620197 (+  2409us) write_op.cc:435] Acquired schema lock
0709 13:19:28.620243 (+    46us) tablet.cc:662] Decoding operations
0709 13:19:28.635415 (+ 15172us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:19:28.635457 (+    42us) write_op.cc:641] Partition lock acquired for write op
0709 13:19:28.635486 (+    29us) tablet.cc:685] Acquiring locks for 2 operations
0709 13:19:28.635778 (+   292us) tablet.cc:701] Row locks acquired
0709 13:19:28.635792 (+    14us) write_op.cc:260] PREPARE: finished
0709 13:19:28.635924 (+   132us) write_op.cc:270] Start()
0709 13:19:28.636047 (+   123us) write_op.cc:276] Timestamp: P: 1783603168635882 usec, L: 0
0709 13:19:28.636068 (+    21us) op_driver.cc:348] REPLICATION: starting
0709 13:19:28.636697 (+   629us) log.cc:844] Serialized 925 byte log entry
0709 13:19:32.143185 (+3506488us) op_driver.cc:464] REPLICATION: finished
0709 13:19:33.288297 (+1145112us) write_op.cc:301] APPLY: starting
0709 13:19:33.288387 (+    90us) tablet.cc:1367] starting BulkCheckPresence
0709 13:19:33.288568 (+   181us) tablet.cc:1370] finished BulkCheckPresence
0709 13:19:33.288709 (+   141us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:19:33.334029 (+ 45320us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:19:33.392518 (+ 58489us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0709 13:19:33.406390 (+ 13872us) write_op.cc:312] APPLY: finished
0709 13:19:33.496125 (+ 89735us) log.cc:844] Serialized 29 byte log entry
0709 13:19:33.903754 (+407629us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:19:33.918166 (+ 14412us) write_op.cc:454] Released schema lock
0709 13:19:33.919938 (+  1772us) write_op.cc:341] FINISH: Updating metrics
0709 13:19:33.969303 (+ 49365us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:19:33.976084 (+  6781us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":3,"spinlock_wait_cycles":377112320,"child_traces":[["op",{"apply.queue_time_us":1128403,"apply.run_cpu_time_us":41060,"apply.run_wall_time_us":671762,"num_ops":2,"prepare.queue_time_us":149,"prepare.run_cpu_time_us":3161,"prepare.run_wall_time_us":50713,"raft.queue_time_us":246017,"raft.run_cpu_time_us":9271,"raft.run_wall_time_us":77056,"replication_time_us":3506941,"thread_start_us":1127504,"threads_started":3,"wal-append.queue_time_us":697071}]]}
I20260709 13:19:34.924506 16115 leader_election.cc:304] T 08c74490c40243a88e62d1760b5cd814 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac
I20260709 13:19:34.902037 18031 raft_consensus.cc:493] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:34.901325 17794 leader_election.cc:290] T 8cc959fb7f704fba9009c0595467d074 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 3 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:19:34.932921 18041 raft_consensus.cc:493] T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:19:34.928622 17892 raft_consensus.cc:2804] T 18ad7e3af5504571a69d10c00750a01c P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:19:34.936501 16291 raft_consensus.cc:1275] T d2ab0ac8347641e6955e9bce7d90c1a2 P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:19:34.940683 18043 raft_consensus.cc:2749] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:19:34.943389 16790 leader_election.cc:304] T b87147731a68400e8cce9b879daa7c5d P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:19:34.906354 17977 raft_consensus.cc:493] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Starting leader election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:19:34.951113 17644 leader_election.cc:290] T 746aabf856ab401c9ef5e859371e7b67 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 3 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
W20260709 13:19:34.953944 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 112271us
W20260709 13:19:34.957273 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 112339us
I20260709 13:19:34.913825 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e4bfb9090c8460eb14fa7e0d818e7f1" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:34.929072 17942 leader_election.cc:290] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 2 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
W20260709 13:19:34.965032 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 238137us
I20260709 13:19:34.970003 18041 raft_consensus.cc:515] T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:34.966506 17892 raft_consensus.cc:493] T 18ad7e3af5504571a69d10c00750a01c P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Starting leader election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:19:34.972262 17972 consensus_queue.cc:237] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:34.963373 17999 raft_consensus.cc:3060] T 18ad7e3af5504571a69d10c00750a01c P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:19:34.992929 18023 raft_consensus.cc:515] T 434bff3082d141838068f323bdb0d1ed P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:35.014408 16295 raft_consensus.cc:1240] T 434bff3082d141838068f323bdb0d1ed P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 3. Current term is 4. Ops: []
I20260709 13:19:34.997925 17979 raft_consensus.cc:515] T af57996a8be84d398f0f023e8d08ef77 P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:35.002645 16116 leader_election.cc:304] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: cb82cc7bd6db46d980c75237a1402860
I20260709 13:19:35.017359 16116 leader_election.cc:304] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
W20260709 13:19:35.004139 16115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 79714.1us
I20260709 13:19:34.974658 17977 raft_consensus.cc:3060] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:19:35.011289 18025 raft_consensus.cc:2804] T 08c74490c40243a88e62d1760b5cd814 P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:19:34.974161 18031 raft_consensus.cc:515] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:35.003598 18021 raft_consensus.cc:3055] T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:19:34.987331 16265 leader_election.cc:304] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac
I20260709 13:19:35.023294 17926 leader_election.cc:290] T 57a744f5aaf944678a63469cbf514015 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 2 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:19:35.032194 16291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f5f89186204242bf377f7c42b2e2da" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:35.023864 17870 raft_consensus.cc:493] T e5ad4eb281564936a59f41e387a47156 P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Starting leader election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:19:35.013234 18024 raft_consensus.cc:515] T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:35.023859 17982 raft_consensus.cc:2749] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:19:35.024214 17901 raft_consensus.cc:2804] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:19:35.039009 17979 leader_election.cc:290] T af57996a8be84d398f0f023e8d08ef77 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 3 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:19:35.004531 17892 raft_consensus.cc:3060] T 18ad7e3af5504571a69d10c00750a01c P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:19:35.035630 18036 consensus_queue.cc:1059] T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.002s
I20260709 13:19:34.968732 17990 raft_consensus.cc:515] T 3884f864a96d428785895e5463e5728f P cb82cc7bd6db46d980c75237a1402860 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:35.046876 17870 raft_consensus.cc:3060] T e5ad4eb281564936a59f41e387a47156 P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:19:35.049655 16291 raft_consensus.cc:2393] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 4: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
I20260709 13:19:35.028303 18025 raft_consensus.cc:493] T 08c74490c40243a88e62d1760b5cd814 P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Starting leader election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:19:35.041314 18021 raft_consensus.cc:740] T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:19:35.047634 17967 leader_election.cc:290] T 26354361e9b044cea1c2b86c4882fe92 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 2 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:19:35.052704 17901 raft_consensus.cc:493] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Starting leader election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:19:35.040025 18012 consensus_queue.cc:237] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } }
I20260709 13:19:35.086329 18031 leader_election.cc:290] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:19:35.086058 17992 raft_consensus.cc:515] T 57a744f5aaf944678a63469cbf514015 P cb82cc7bd6db46d980c75237a1402860 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:35.087226 16820 raft_consensus.cc:1240] T 57a744f5aaf944678a63469cbf514015 P cb82cc7bd6db46d980c75237a1402860 [term 2 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 1. Current term is 2. Ops: []
W20260709 13:19:35.088914 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 103385us
W20260709 13:19:35.096352 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 109105us
W20260709 13:19:35.096247 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 152977us
I20260709 13:19:35.063069 18025 raft_consensus.cc:3060] T 08c74490c40243a88e62d1760b5cd814 P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:19:35.100445 16297 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48674 (request call id 1950) took 1037 ms. Trace:
I20260709 13:19:35.101584 16265 leader_election.cc:304] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:19:35.103736 16297 rpcz_store.cc:276] 0709 13:19:34.062826 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:34.062990 (+   164us) service_pool.cc:224] Handling call
0709 13:19:34.063377 (+   387us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:34.955688 (+892311us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:35.100138 (+144450us) spinlock_profiling.cc:243] Waited 816 ms on lock 0x7b54000cf654. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:19:35.100374 (+   236us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1875944832}
I20260709 13:19:35.104401 16265 leader_election.cc:304] T ecc1e363230043fe9fef390f0b979f20 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:19:35.105247 16265 leader_election.cc:304] T 5a959dbecc2648eb85d1cc550f5a3245 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac
I20260709 13:19:35.105296 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a878999c0024d2abe517484180290fc" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:19:35.106061 16297 raft_consensus.cc:3060] T 8a878999c0024d2abe517484180290fc P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:19:35.106313 16265 leader_election.cc:304] T 84f2da98864b4425bf7c0055951fdd4b P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
W20260709 13:19:35.109817 16790 leader_election.cc:336] T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943): Timed out: RequestConsensusVote RPC to 127.14.177.130:35943 timed out after 2.634s (SENT)
I20260709 13:19:35.111312 16151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57a744f5aaf944678a63469cbf514015" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:19:35.083721 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c61a39fd4f84424b129c1a6f9ba404d" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:35.089823 17901 raft_consensus.cc:3060] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:19:35.114078 16192 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 128009us
I20260709 13:19:34.973657 17974 consensus_queue.cc:237] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } }
I20260709 13:19:35.046628 17847 raft_consensus.cc:515] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:35.088284 17992 leader_election.cc:290] T 57a744f5aaf944678a63469cbf514015 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 2 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:35.111756 16816 raft_consensus.cc:1240] T 3884f864a96d428785895e5463e5728f P cb82cc7bd6db46d980c75237a1402860 [term 2 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 1. Current term is 2. Ops: []
W20260709 13:19:35.119879 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 100544us
I20260709 13:19:35.099539 17926 raft_consensus.cc:2804] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:19:35.162314 16265 leader_election.cc:304] T 7c2095a5f4ad49408dbc7f69c606120d P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac
I20260709 13:19:35.167876 16265 leader_election.cc:304] T c0eab25ce4c64e018a041beeda2867db P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac
I20260709 13:19:35.134439 18046 raft_consensus.cc:2804] T 84f2da98864b4425bf7c0055951fdd4b P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:19:35.113847 17942 raft_consensus.cc:2804] T ecc1e363230043fe9fef390f0b979f20 P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:19:35.162151 16116 leader_election.cc:304] T 68a6838d82ef4cac83507f906139d90f P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 90f3da57796b488c9087a2f523adf045
W20260709 13:19:35.114995 16790 leader_election.cc:336] T c9944d5e4b8c4109af9955c4f08c9095 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943): Timed out: RequestConsensusVote RPC to 127.14.177.130:35943 timed out after 1.908s (SENT)
I20260709 13:19:35.135195 18036 raft_consensus.cc:493] T 5a96373bb0e14dce8a266b2fc6c58507 P 647a343a542d4a139fe106c19095129b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:19:35.170542 17800 raft_consensus.cc:2804] T 7c2095a5f4ad49408dbc7f69c606120d P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:19:35.172333 17800 raft_consensus.cc:493] T 7c2095a5f4ad49408dbc7f69c606120d P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:19:35.172914 17800 raft_consensus.cc:3060] T 7c2095a5f4ad49408dbc7f69c606120d P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:19:35.172268 18036 raft_consensus.cc:515] T 5a96373bb0e14dce8a266b2fc6c58507 P 647a343a542d4a139fe106c19095129b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:35.114133 18033 raft_consensus.cc:2749] T b87147731a68400e8cce9b879daa7c5d P cb82cc7bd6db46d980c75237a1402860 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:19:35.176193 16116 leader_election.cc:304] T 8d4ab870a05347868956aa8b841b2981 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:19:35.124747 17794 raft_consensus.cc:2804] T 5a959dbecc2648eb85d1cc550f5a3245 P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260709 13:19:35.176952 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 61982.2us
I20260709 13:19:35.136008 16291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0eab25ce4c64e018a041beeda2867db" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:35.123646 17933 leader_election.cc:290] T 8c11074579b046d99e585c2818e18a2c P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 4 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:19:35.166630 17926 raft_consensus.cc:493] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:19:35.151695 16824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a378d7f796f94615b3b347119a9880fd" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:19:35.113729 17967 raft_consensus.cc:2804] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:19:35.154125 18023 leader_election.cc:290] T 434bff3082d141838068f323bdb0d1ed P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 4 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:19:35.171414 17942 raft_consensus.cc:493] T ecc1e363230043fe9fef390f0b979f20 P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:19:35.175014 18036 leader_election.cc:290] T 5a96373bb0e14dce8a266b2fc6c58507 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:35.171211 16293 raft_consensus.cc:2393] T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 3: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
I20260709 13:19:35.190398 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6f07842d3aa47858df5665874c86f3a" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:35.181756 17794 raft_consensus.cc:493] T 5a959dbecc2648eb85d1cc550f5a3245 P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:19:35.151971 18041 leader_election.cc:290] T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:19:35.189101 18037 raft_consensus.cc:2749] T 8d4ab870a05347868956aa8b841b2981 P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:19:35.198536 17926 raft_consensus.cc:3060] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:19:35.118969 17991 raft_consensus.cc:515] T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:35.198083 17967 raft_consensus.cc:493] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:19:35.212911 17965 raft_consensus.cc:515] T 5a96373bb0e14dce8a266b2fc6c58507 P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:35.179386 17979 raft_consensus.cc:2804] T 68a6838d82ef4cac83507f906139d90f P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:19:35.118969 17998 raft_consensus.cc:515] T 2794af5b99984efebdee564035841cb0 P cb82cc7bd6db46d980c75237a1402860 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:35.168748 18046 raft_consensus.cc:493] T 84f2da98864b4425bf7c0055951fdd4b P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:19:35.202497 16222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a878999c0024d2abe517484180290fc" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:19:35.236792 16222 raft_consensus.cc:2393] T 8a878999c0024d2abe517484180290fc P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 3: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
I20260709 13:19:35.199120 16297 raft_consensus.cc:2468] T 8a878999c0024d2abe517484180290fc P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 3.
I20260709 13:19:35.175762 17898 raft_consensus.cc:515] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:35.243124 17898 leader_election.cc:290] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 2 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:35.227356 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f5f89186204242bf377f7c42b2e2da" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:35.243803 16225 raft_consensus.cc:2468] T c2f5f89186204242bf377f7c42b2e2da P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 4.
I20260709 13:19:35.196544 16219 raft_consensus.cc:1240] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 13:19:35.150458 18020 raft_consensus.cc:515] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:35.245206 18014 raft_consensus.cc:515] T 434bff3082d141838068f323bdb0d1ed P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:35.246296 16144 raft_consensus.cc:1240] T 434bff3082d141838068f323bdb0d1ed P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 3. Current term is 4. Ops: []
I20260709 13:19:35.247114 18020 leader_election.cc:290] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 4 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:19:35.249068 17998 leader_election.cc:290] T 2794af5b99984efebdee564035841cb0 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 2 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:35.249645 16821 raft_consensus.cc:1240] T 2794af5b99984efebdee564035841cb0 P cb82cc7bd6db46d980c75237a1402860 [term 2 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 1. Current term is 2. Ops: []
I20260709 13:19:35.250591 16292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e9b2bf7011f4ae688a79d7bb5a447b4" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:35.198132 16291 raft_consensus.cc:2468] T c0eab25ce4c64e018a041beeda2867db P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 2.
I20260709 13:19:35.235137 16146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3884f864a96d428785895e5463e5728f" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:19:35.219354 17914 consensus_queue.cc:1048] T 8638ee363273426db1393dd81e5c7b72 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.018s
I20260709 13:19:35.226563 18023 raft_consensus.cc:493] T ada5a004ba7c46819051e3ccec71ffdb P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:19:35.240724 17991 leader_election.cc:290] T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 3 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:19:35.241192 16143 raft_consensus.cc:1240] T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 2. Current term is 3. Ops: []
I20260709 13:19:35.224812 17794 raft_consensus.cc:3060] T 5a959dbecc2648eb85d1cc550f5a3245 P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:19:35.216379 17942 raft_consensus.cc:3060] T ecc1e363230043fe9fef390f0b979f20 P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:19:35.238960 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e65676b21b748728ad39668cfe722ec" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:35.237489 18046 raft_consensus.cc:3060] T 84f2da98864b4425bf7c0055951fdd4b P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:19:35.293453 16297 raft_consensus.cc:2468] T 9e65676b21b748728ad39668cfe722ec P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 3.
I20260709 13:19:35.255472 16145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2794af5b99984efebdee564035841cb0" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:19:35.176532 18021 consensus_queue.cc:260] T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:35.261206 18024 leader_election.cc:290] T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 3 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:19:35.265275 17933 raft_consensus.cc:2804] T c0eab25ce4c64e018a041beeda2867db P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:19:35.260668 16221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a96373bb0e14dce8a266b2fc6c58507" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:35.259878 18014 leader_election.cc:290] T 434bff3082d141838068f323bdb0d1ed P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 4 election: Requested vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:19:35.264082 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31969392ccdb43b491911a319e8bc7e1" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:19:35.233928 16220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746aabf856ab401c9ef5e859371e7b67" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:19:35.238023 17967 raft_consensus.cc:3060] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:19:35.237763 17979 raft_consensus.cc:493] T 68a6838d82ef4cac83507f906139d90f P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Starting leader election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:19:35.203405 17957 raft_consensus.cc:515] T 829b4ad8c5a54eababeea1181b960d4b P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:35.235389 17964 raft_consensus.cc:515] T 2794af5b99984efebdee564035841cb0 P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:35.282159 18023 raft_consensus.cc:515] T ada5a004ba7c46819051e3ccec71ffdb P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:35.295137 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4f65cdf00b748bb963ee6e81ef43b79" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:19:35.298286 17990 leader_election.cc:290] T 3884f864a96d428785895e5463e5728f P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 2 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:35.277693 17900 raft_consensus.cc:515] T c9944d5e4b8c4109af9955c4f08c9095 P cb82cc7bd6db46d980c75237a1402860 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:35.289906 18025 raft_consensus.cc:515] T 08c74490c40243a88e62d1760b5cd814 P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:35.318125 17979 raft_consensus.cc:3060] T 68a6838d82ef4cac83507f906139d90f P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:19:35.312801 17965 leader_election.cc:290] T 5a96373bb0e14dce8a266b2fc6c58507 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 2 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:19:35.321158 16291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d16dfad0c3b540e5b4b86492ddf87208" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
W20260709 13:19:35.331379 16789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 449829us
W20260709 13:19:35.326238 17894 consensus_meta.cc:287] T 8158bbd17b994d2eb96f560bd0f1b88d P 90f3da57796b488c9087a2f523adf045: Time spent flushing consensus metadata: real 0.691s	user 0.012s	sys 0.004s
I20260709 13:19:35.323398 18021 raft_consensus.cc:3060] T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:19:35.347767 18045 consensus_queue.cc:1059] T 2794af5b99984efebdee564035841cb0 P 647a343a542d4a139fe106c19095129b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:19:35.348910 18045 raft_consensus.cc:3055] T 2794af5b99984efebdee564035841cb0 P 647a343a542d4a139fe106c19095129b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:19:35.349231 18045 raft_consensus.cc:740] T 2794af5b99984efebdee564035841cb0 P 647a343a542d4a139fe106c19095129b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:19:35.351608 16821 raft_consensus.cc:1240] T c9944d5e4b8c4109af9955c4f08c9095 P cb82cc7bd6db46d980c75237a1402860 [term 4 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 3. Current term is 4. Ops: []
I20260709 13:19:35.360646 16825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57a744f5aaf944678a63469cbf514015" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:19:35.360826 16824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31969392ccdb43b491911a319e8bc7e1" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:19:35.361738 16789 leader_election.cc:304] T 26354361e9b044cea1c2b86c4882fe92 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:19:35.361974 16824 raft_consensus.cc:2393] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 2: Already voted for candidate cb82cc7bd6db46d980c75237a1402860 in this term.
I20260709 13:19:35.362000 16825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434bff3082d141838068f323bdb0d1ed" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:19:35.370755 16824 raft_consensus.cc:1240] T b87147731a68400e8cce9b879daa7c5d P cb82cc7bd6db46d980c75237a1402860 [term 4 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 3. Current term is 4. Ops: []
I20260709 13:19:35.362993 16789 leader_election.cc:304] T 23f82224f4564fea93572e8206bad1fe P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:19:35.351109 16293 raft_consensus.cc:2393] T c4f65cdf00b748bb963ee6e81ef43b79 P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 5: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
I20260709 13:19:35.406101 18025 leader_election.cc:290] T 08c74490c40243a88e62d1760b5cd814 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 4 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:35.395309 17870 raft_consensus.cc:515] T e5ad4eb281564936a59f41e387a47156 P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:35.406450 16789 leader_election.cc:304] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:19:35.400898 18043 raft_consensus.cc:2749] T 26354361e9b044cea1c2b86c4882fe92 P cb82cc7bd6db46d980c75237a1402860 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:19:35.407384 16819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c11074579b046d99e585c2818e18a2c" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:19:35.349933 17894 raft_consensus.cc:515] T 8158bbd17b994d2eb96f560bd0f1b88d P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:35.423560 17981 raft_consensus.cc:3055] T 3a7ca61592e848ea9b0fb81a2ce33cdc P cb82cc7bd6db46d980c75237a1402860 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:19:35.406620 17964 leader_election.cc:290] T 2794af5b99984efebdee564035841cb0 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 2 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:19:35.401728 18033 raft_consensus.cc:2749] T 23f82224f4564fea93572e8206bad1fe P cb82cc7bd6db46d980c75237a1402860 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:19:35.402058 17847 leader_election.cc:290] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 3 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:19:35.407748 17900 leader_election.cc:290] T c9944d5e4b8c4109af9955c4f08c9095 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 4 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:35.420439 17933 raft_consensus.cc:493] T c0eab25ce4c64e018a041beeda2867db P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:19:35.435317 17981 raft_consensus.cc:740] T 3a7ca61592e848ea9b0fb81a2ce33cdc P cb82cc7bd6db46d980c75237a1402860 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:19:35.437858 17933 raft_consensus.cc:3060] T c0eab25ce4c64e018a041beeda2867db P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:19:35.407940 17800 raft_consensus.cc:515] T 7c2095a5f4ad49408dbc7f69c606120d P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:35.416045 17992 raft_consensus.cc:2749] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P cb82cc7bd6db46d980c75237a1402860 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:19:35.440706 17967 raft_consensus.cc:515] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:35.441720 17800 leader_election.cc:290] T 7c2095a5f4ad49408dbc7f69c606120d P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 2 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:19:35.438977 17870 leader_election.cc:290] T e5ad4eb281564936a59f41e387a47156 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 4 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:35.442873 17967 leader_election.cc:290] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 4 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:19:35.449126 17957 leader_election.cc:290] T 829b4ad8c5a54eababeea1181b960d4b P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 4 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:19:35.452836 18040 raft_consensus.cc:1064] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac: attempting to promote NON_VOTER cb82cc7bd6db46d980c75237a1402860 to VOTER
I20260709 13:19:35.406127 17998 consensus_queue.cc:1059] T 434bff3082d141838068f323bdb0d1ed P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.034s
W20260709 13:19:35.453318 18040 raft_consensus.cc:1068] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac: Unable to promote non-voter cb82cc7bd6db46d980c75237a1402860: Illegal state: Leader has not yet committed an operation in its own term
I20260709 13:19:35.427585 18045 consensus_queue.cc:260] T 2794af5b99984efebdee564035841cb0 P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:35.485191 18045 raft_consensus.cc:3060] T 2794af5b99984efebdee564035841cb0 P 647a343a542d4a139fe106c19095129b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:19:35.485630 18023 leader_election.cc:290] T ada5a004ba7c46819051e3ccec71ffdb P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:19:35.412886 17914 consensus_queue.cc:1048] T d2ab0ac8347641e6955e9bce7d90c1a2 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.035s
I20260709 13:19:35.491663 17999 raft_consensus.cc:515] T 18ad7e3af5504571a69d10c00750a01c P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } }
I20260709 13:19:35.534034 17999 leader_election.cc:290] T 18ad7e3af5504571a69d10c00750a01c P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 3 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:19:35.536530 17901 raft_consensus.cc:515] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:35.552753 17942 raft_consensus.cc:515] T ecc1e363230043fe9fef390f0b979f20 P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } }
I20260709 13:19:35.554442 17926 raft_consensus.cc:515] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:35.557730 17933 raft_consensus.cc:515] T c0eab25ce4c64e018a041beeda2867db P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:19:35.557771 17977 consensus_meta.cc:287] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b: Time spent flushing consensus metadata: real 0.516s	user 0.008s	sys 0.002s
I20260709 13:19:35.658560 18046 raft_consensus.cc:515] T 84f2da98864b4425bf7c0055951fdd4b P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:35.673537 18046 leader_election.cc:290] T 84f2da98864b4425bf7c0055951fdd4b P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 2 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
W20260709 13:19:35.716167 16115 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.676s	user 0.031s	sys 0.022s
W20260709 13:19:35.717193 16115 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.677s	user 0.031s	sys 0.023s
I20260709 13:19:35.425896 16294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3884f864a96d428785895e5463e5728f" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:19:35.722709 16294 raft_consensus.cc:3060] T 3884f864a96d428785895e5463e5728f P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:19:35.723671 16265 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.547s	user 0.000s	sys 0.008s
W20260709 13:19:35.738687 16265 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.562s	user 0.000s	sys 0.008s
W20260709 13:19:35.741147 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 573276us
I20260709 13:19:35.745241 16265 leader_election.cc:304] T 9e65676b21b748728ad39668cfe722ec P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: cb82cc7bd6db46d980c75237a1402860
I20260709 13:19:35.726084 16291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fffab26427e24d398e2695aad40ca134" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:35.752746 16291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a878999c0024d2abe517484180290fc" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:19:35.754012 16291 raft_consensus.cc:2393] T 8a878999c0024d2abe517484180290fc P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 3: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
I20260709 13:19:35.727653 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdbd78acb4cf46298e0de8ba0d130e19" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
W20260709 13:19:35.731781 16295 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.708s	user 0.000s	sys 0.012s
W20260709 13:19:35.756361 16295 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.733s	user 0.000s	sys 0.013s
I20260709 13:19:35.756474 16265 leader_election.cc:304] T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:19:35.758313 16295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2794af5b99984efebdee564035841cb0" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:19:35.760242 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af57996a8be84d398f0f023e8d08ef77" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:19:35.732216 16296 raft_consensus.cc:1240] T 18ad7e3af5504571a69d10c00750a01c P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 2. Current term is 3. Ops: []
W20260709 13:19:35.721642 16223 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.504s	user 0.004s	sys 0.005s
I20260709 13:19:35.762039 16146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d4ab870a05347868956aa8b841b2981" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
W20260709 13:19:35.762589 16223 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.546s	user 0.004s	sys 0.005s
I20260709 13:19:35.763293 16146 raft_consensus.cc:2393] T 8d4ab870a05347868956aa8b841b2981 P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 3: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
I20260709 13:19:35.765234 16824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2794af5b99984efebdee564035841cb0" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:19:35.766239 16824 raft_consensus.cc:2393] T 2794af5b99984efebdee564035841cb0 P cb82cc7bd6db46d980c75237a1402860 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 2: Already voted for candidate cb82cc7bd6db46d980c75237a1402860 in this term.
W20260709 13:19:35.767043 16823 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.538s	user 0.012s	sys 0.020s
W20260709 13:19:35.767458 16823 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.542s	user 0.013s	sys 0.020s
I20260709 13:19:35.950145 16145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f5f89186204242bf377f7c42b2e2da" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:19:35.951288 17758 raft_consensus.cc:493] T 1df93779209748b9860f6d893dd3de53 P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:19:35.951408 16145 raft_consensus.cc:2468] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 3.
I20260709 13:19:35.952270 17758 raft_consensus.cc:515] T 1df93779209748b9860f6d893dd3de53 P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:35.953713 16222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c11074579b046d99e585c2818e18a2c" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:35.955878 16222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9944d5e4b8c4109af9955c4f08c9095" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:19:35.956043 18031 raft_consensus.cc:493] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:19:35.957094 18031 raft_consensus.cc:515] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:35.937227 16291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31969392ccdb43b491911a319e8bc7e1" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
W20260709 13:19:35.973315 17978 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.519s	user 0.007s	sys 0.016s
I20260709 13:19:35.938076 16293 raft_consensus.cc:2393] T cdbd78acb4cf46298e0de8ba0d130e19 P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 4: Already voted for candidate cb82cc7bd6db46d980c75237a1402860 in this term.
W20260709 13:19:35.972772 17974 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.501s	user 0.005s	sys 0.006s
W20260709 13:19:35.971352 17972 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.645s	user 0.007s	sys 0.013s
I20260709 13:19:35.577437 17977 raft_consensus.cc:515] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:35.961864 16295 raft_consensus.cc:2393] T 2794af5b99984efebdee564035841cb0 P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 in current term 2: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
I20260709 13:19:36.000922 16824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2372981006d4b218215c28964dea623" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:19:36.023589 16219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee9ed67006144797b1de6ea8c2013a68" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:36.029234 16219 raft_consensus.cc:1240] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 2. Current term is 3. Ops: []
W20260709 13:19:36.031388 18038 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.992s	user 0.067s	sys 0.095s
W20260709 13:19:36.032006 18038 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.992s	user 0.067s	sys 0.095s
W20260709 13:19:36.033092 16817 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.725s	user 0.003s	sys 0.019s
W20260709 13:19:36.033684 16817 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.731s	user 0.004s	sys 0.019s
W20260709 13:19:36.016209 17978 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.588s	user 0.007s	sys 0.017s
I20260709 13:19:36.037864 16145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d4ab870a05347868956aa8b841b2981" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:19:35.993757 17758 leader_election.cc:290] T 1df93779209748b9860f6d893dd3de53 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
W20260709 13:19:36.016043 17972 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.720s	user 0.008s	sys 0.013s
W20260709 13:19:36.018450 17974 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.572s	user 0.005s	sys 0.007s
I20260709 13:19:36.079135 16145 raft_consensus.cc:2393] T 8d4ab870a05347868956aa8b841b2981 P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 3: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
I20260709 13:19:36.041627 16291 raft_consensus.cc:2393] T 31969392ccdb43b491911a319e8bc7e1 P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 in current term 2: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
W20260709 13:19:36.113224 16266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 85848.6us
I20260709 13:19:36.031497 17931 raft_consensus.cc:493] T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:36.113596 16146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "829b4ad8c5a54eababeea1181b960d4b" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:19:36.114203 16295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a878999c0024d2abe517484180290fc" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:36.120253 16295 raft_consensus.cc:2376] T 8a878999c0024d2abe517484180290fc P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 647a343a542d4a139fe106c19095129b in term 3. Re-sending same reply.
I20260709 13:19:36.136262 16148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1df93779209748b9860f6d893dd3de53" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:19:36.137771 16148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d4ab870a05347868956aa8b841b2981" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:19:36.138563 16148 raft_consensus.cc:2393] T 8d4ab870a05347868956aa8b841b2981 P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 3: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
I20260709 13:19:36.139207 18046 raft_consensus.cc:2804] T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:19:36.139889 16142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fffab26427e24d398e2695aad40ca134" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:19:36.140928 16142 raft_consensus.cc:2468] T fffab26427e24d398e2695aad40ca134 P 647a343a542d4a139fe106c19095129b [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 1.
I20260709 13:19:36.141688 17999 raft_consensus.cc:493] T d2ab0ac8347641e6955e9bce7d90c1a2 P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:36.142347 17999 raft_consensus.cc:515] T d2ab0ac8347641e6955e9bce7d90c1a2 P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:36.136240 17967 consensus_queue.cc:1059] T b87147731a68400e8cce9b879daa7c5d P 90f3da57796b488c9087a2f523adf045 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.074s
I20260709 13:19:36.144944 17999 leader_election.cc:290] T d2ab0ac8347641e6955e9bce7d90c1a2 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:19:36.145244 16291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9944d5e4b8c4109af9955c4f08c9095" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:19:36.145982 18021 raft_consensus.cc:493] T 84f2da98864b4425bf7c0055951fdd4b P 647a343a542d4a139fe106c19095129b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:19:36.146819 18021 raft_consensus.cc:515] T 84f2da98864b4425bf7c0055951fdd4b P 647a343a542d4a139fe106c19095129b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:36.147184 16291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a96373bb0e14dce8a266b2fc6c58507" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:36.148492 16291 raft_consensus.cc:2393] T 5a96373bb0e14dce8a266b2fc6c58507 P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 2: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
I20260709 13:19:36.149480 18021 leader_election.cc:290] T 84f2da98864b4425bf7c0055951fdd4b P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:36.150230 17926 leader_election.cc:290] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 3 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:19:36.136112 17931 raft_consensus.cc:515] T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:36.151835 17942 leader_election.cc:290] T ecc1e363230043fe9fef390f0b979f20 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 3 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:19:36.153869 17931 leader_election.cc:290] T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:36.149912 16291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c74490c40243a88e62d1760b5cd814" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:19:36.156661 18041 raft_consensus.cc:2804] T 9e65676b21b748728ad39668cfe722ec P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:19:36.157197 18041 raft_consensus.cc:493] T 9e65676b21b748728ad39668cfe722ec P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Starting leader election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:19:36.157625 18041 raft_consensus.cc:3060] T 9e65676b21b748728ad39668cfe722ec P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:19:36.161242 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62fe3346cd3c42c4a55e14af9073bca4" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
W20260709 13:19:36.163123 18019 log.cc:927] Time spent T 8c11074579b046d99e585c2818e18a2c P cb82cc7bd6db46d980c75237a1402860: Append to log took a long time: real 0.093s	user 0.000s	sys 0.000s
I20260709 13:19:36.164068 17933 leader_election.cc:290] T c0eab25ce4c64e018a041beeda2867db P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 3 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:19:36.165175 17894 leader_election.cc:290] T 8158bbd17b994d2eb96f560bd0f1b88d P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 4 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:19:36.168109 16220 raft_consensus.cc:1275] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:19:36.175336 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57a744f5aaf944678a63469cbf514015" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:19:36.176673 16293 raft_consensus.cc:2393] T 57a744f5aaf944678a63469cbf514015 P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 in current term 2: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
I20260709 13:19:36.177784 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434bff3082d141838068f323bdb0d1ed" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:19:36.178779 16293 raft_consensus.cc:2393] T 434bff3082d141838068f323bdb0d1ed P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 4: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
I20260709 13:19:36.170675 16143 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46224 (request call id 230) took 1488 ms. Trace:
I20260709 13:19:36.190276 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a878999c0024d2abe517484180290fc" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:19:36.122396 17901 leader_election.cc:290] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 3 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:19:36.130048 16295 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48674 (request call id 1966) took 1098 ms. Trace:
I20260709 13:19:36.188120 16145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68a6838d82ef4cac83507f906139d90f" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:19:36.244814 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5ad4eb281564936a59f41e387a47156" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:19:36.246089 16220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1df93779209748b9860f6d893dd3de53" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:19:36.246759 16220 raft_consensus.cc:3060] T 1df93779209748b9860f6d893dd3de53 P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:19:36.247025 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a878999c0024d2abe517484180290fc" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:19:36.247365 16219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8638ee363273426db1393dd81e5c7b72" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:36.248132 16226 raft_consensus.cc:2368] T 8a878999c0024d2abe517484180290fc P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 for earlier term 2. Current term is 3.
I20260709 13:19:36.249752 17758 raft_consensus.cc:493] T e5ad4eb281564936a59f41e387a47156 P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:19:36.250242 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d4ab870a05347868956aa8b841b2981" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:19:36.250572 17758 raft_consensus.cc:515] T e5ad4eb281564936a59f41e387a47156 P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:36.251302 16224 raft_consensus.cc:2393] T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 3: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
I20260709 13:19:36.251686 16296 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48684 (request call id 1857) took 1616 ms. Trace:
I20260709 13:19:36.250263 18013 raft_consensus.cc:3055] T 1df93779209748b9860f6d893dd3de53 P 647a343a542d4a139fe106c19095129b [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:19:36.252441 18013 raft_consensus.cc:740] T 1df93779209748b9860f6d893dd3de53 P 647a343a542d4a139fe106c19095129b [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:19:36.252637 16146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4f65cdf00b748bb963ee6e81ef43b79" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:19:36.253028 17758 leader_election.cc:290] T e5ad4eb281564936a59f41e387a47156 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:36.253171 16223 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57604 (request call id 1886) took 1059 ms. Trace:
I20260709 13:19:36.209690 18031 leader_election.cc:290] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:19:36.253839 16146 raft_consensus.cc:2410] T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 for term 5 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:19:36.200177 18023 raft_consensus.cc:493] T 66ae176961b74813a58bc15c9c4c6f2a P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:36.282433 18023 raft_consensus.cc:515] T 66ae176961b74813a58bc15c9c4c6f2a P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:36.253801 18013 consensus_queue.cc:260] T 1df93779209748b9860f6d893dd3de53 P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:36.268951 16822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e4bfb9090c8460eb14fa7e0d818e7f1" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:19:36.253705 16223 rpcz_store.cc:276] 0709 13:19:35.193342 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:35.193507 (+   165us) service_pool.cc:224] Handling call
0709 13:19:35.193868 (+   361us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:35.194112 (+   244us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:19:35.763260 (+569148us) log.cc:844] Serialized 14 byte log entry
0709 13:19:36.252630 (+489370us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:36.252726 (+    96us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:19:36.252797 (+    71us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:36.252891 (+    94us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:36.253076 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":228800,"prepare.queue_time_us":445581,"prepare.run_cpu_time_us":70,"prepare.run_wall_time_us":72,"raft.queue_time_us":229955,"raft.run_cpu_time_us":107,"raft.run_wall_time_us":106,"thread_start_us":795019,"threads_started":2,"wal-append.queue_time_us":362707}
I20260709 13:19:36.288652 16150 raft_consensus.cc:2368] T 8a878999c0024d2abe517484180290fc P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 for earlier term 2. Current term is 3.
I20260709 13:19:36.290848 17977 leader_election.cc:290] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 3 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
W20260709 13:19:36.291873 17857 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.954s	user 0.015s	sys 0.034s
W20260709 13:19:36.292315 17857 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.117s	user 0.015s	sys 0.035s
I20260709 13:19:36.191545 16295 rpcz_store.cc:276] 0709 13:19:35.028542 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:36.060720 (+1032178us) service_pool.cc:224] Handling call
0709 13:19:36.126528 (+ 65808us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:36.303704 16148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d16dfad0c3b540e5b4b86492ddf87208" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:19:36.305562 16148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4f65cdf00b748bb963ee6e81ef43b79" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:19:36.252027 16296 rpcz_store.cc:276] 0709 13:19:34.634780 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:34.895584 (+260804us) service_pool.cc:224] Handling call
0709 13:19:34.895897 (+   313us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:35.762018 (+866121us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:36.251390 (+489372us) spinlock_profiling.cc:243] Waited 836 ms on lock 0x7b540024c554. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:19:36.251588 (+   198us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1921755520}
I20260709 13:19:36.320086 18031 raft_consensus.cc:493] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:19:36.320839 16149 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46224 (request call id 229) took 1638 ms. Trace:
I20260709 13:19:36.320966 18031 raft_consensus.cc:515] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:36.321506 16149 rpcz_store.cc:276] 0709 13:19:34.681982 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:34.684812 (+  2830us) service_pool.cc:224] Handling call
0709 13:19:36.180825 (+1496013us) spinlock_profiling.cc:243] Waited 5.52 ms on lock 0x7b5800011198. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69ad8fadb1 00007f69ada3ab71 00007f69ada3aa22 00007f69b6953c09 00007f69b5aadee0 00007f69b5b770ab 00007f69b5b77131 00007f69b5b38fc0 00007f69b5b23929 00007f69b5b233d2 00007f69af09abd4
0709 13:19:36.184259 (+  3434us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:36.263456 (+ 79197us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:36.320088 (+ 56632us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:36.320206 (+   118us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:19:36.320312 (+   106us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:36.320395 (+    83us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:36.320599 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":12702464}
W20260709 13:19:36.276182 18067 log.cc:927] Time spent T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860: Append to log took a long time: real 0.242s	user 0.000s	sys 0.003s
I20260709 13:19:36.337247 18013 raft_consensus.cc:3060] T 1df93779209748b9860f6d893dd3de53 P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:19:36.340646 16150 raft_consensus.cc:1240] T 746aabf856ab401c9ef5e859371e7b67 P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 2. Current term is 3. Ops: []
I20260709 13:19:36.343513 16142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cc959fb7f704fba9009c0595467d074" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:19:36.306957 16148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a878999c0024d2abe517484180290fc" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:19:36.380525 16148 raft_consensus.cc:2393] T 8a878999c0024d2abe517484180290fc P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 3: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
I20260709 13:19:36.381537 16223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7ca61592e848ea9b0fb81a2ce33cdc" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
W20260709 13:19:36.345225 16294 consensus_meta.cc:287] T 3884f864a96d428785895e5463e5728f P 90f3da57796b488c9087a2f523adf045: Time spent flushing consensus metadata: real 0.598s	user 0.001s	sys 0.015s
I20260709 13:19:36.192263 16143 rpcz_store.cc:276] 0709 13:19:34.682341 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:34.684716 (+  2375us) service_pool.cc:224] Handling call
0709 13:19:34.710404 (+ 25688us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:35.292245 (+581841us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:36.169883 (+877638us) spinlock_profiling.cc:243] Waited 528 ms on lock 0x7b5400073fd4. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:19:36.170383 (+   500us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1214395264}
I20260709 13:19:36.421643 16142 raft_consensus.cc:3060] T 8cc959fb7f704fba9009c0595467d074 P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:19:36.422689 17892 consensus_meta.cc:287] T 18ad7e3af5504571a69d10c00750a01c P 647a343a542d4a139fe106c19095129b: Time spent flushing consensus metadata: real 1.366s	user 0.017s	sys 0.000s
I20260709 13:19:36.423236 17892 raft_consensus.cc:515] T 18ad7e3af5504571a69d10c00750a01c P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } }
W20260709 13:19:36.426587 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 313426us
I20260709 13:19:36.429225 18044 raft_consensus.cc:493] T 66f75124215f4ca6bef831e174a792d8 P cb82cc7bd6db46d980c75237a1402860 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:19:36.429890 18044 raft_consensus.cc:515] T 66f75124215f4ca6bef831e174a792d8 P cb82cc7bd6db46d980c75237a1402860 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:36.425326 16296 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48674 (request call id 1973) took 1393 ms. Trace:
I20260709 13:19:36.436681 16296 rpcz_store.cc:276] 0709 13:19:35.031692 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:36.424299 (+1392607us) service_pool.cc:224] Handling call
0709 13:19:36.424710 (+   411us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:36.425001 (+   291us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:36.425024 (+    23us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:36.425063 (+    39us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:19:36.425090 (+    27us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:36.425144 (+    54us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:36.425251 (+   107us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:36.361346 18031 leader_election.cc:290] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:19:36.446136 16148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26354361e9b044cea1c2b86c4882fe92" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:19:36.431005 16790 leader_election.cc:304] T ee9ed67006144797b1de6ea8c2013a68 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
W20260709 13:19:36.434139 17979 consensus_meta.cc:287] T 68a6838d82ef4cac83507f906139d90f P 647a343a542d4a139fe106c19095129b: Time spent flushing consensus metadata: real 1.092s	user 0.013s	sys 0.001s
I20260709 13:19:36.393054 16294 raft_consensus.cc:2468] T 3884f864a96d428785895e5463e5728f P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 2.
W20260709 13:19:36.416627 16265 connection.cc:256] RPC call timeout handler was delayed by -2.4292881670000952s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.3128s for a response
W20260709 13:19:36.457039 16265 connection.cc:256] RPC call timeout handler was delayed by -2.3387863670002389s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.168s for a response
W20260709 13:19:36.458961 16265 connection.cc:256] RPC call timeout handler was delayed by -2.3306863670002258s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16890000000000002s for a response
W20260709 13:19:36.459517 16265 connection.cc:256] RPC call timeout handler was delayed by -2.1830863670002145s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18530000000000002s for a response
W20260709 13:19:36.459908 16265 connection.cc:256] RPC call timeout handler was delayed by -1.922291813000129s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20310000000000003s for a response
W20260709 13:19:36.460299 16265 connection.cc:256] RPC call timeout handler was delayed by -1.9049863670002196s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2162s for a response
W20260709 13:19:36.460672 16265 connection.cc:256] RPC call timeout handler was delayed by -1.8932863670002007s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2175s for a response
W20260709 13:19:36.461047 16265 connection.cc:256] RPC call timeout handler was delayed by -1.8329863670001032s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2242s for a response
W20260709 13:19:36.461411 16265 connection.cc:256] RPC call timeout handler was delayed by -1.7199629379999806s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1896s for a response
W20260709 13:19:36.461774 16265 connection.cc:256] RPC call timeout handler was delayed by -1.6891682790001141s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22160000000000002s for a response
W20260709 13:19:36.462157 16265 connection.cc:256] RPC call timeout handler was delayed by -1.6717226089999713s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1764s for a response
W20260709 13:19:36.462523 16265 connection.cc:256] RPC call timeout handler was delayed by -1.1777920300000915s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21070000000000003s for a response
I20260709 13:19:36.463814 16819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ab0ac8347641e6955e9bce7d90c1a2" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:19:36.464421 16222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fffab26427e24d398e2695aad40ca134" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:36.466094 18058 raft_consensus.cc:493] T a378d7f796f94615b3b347119a9880fd P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:36.466306 17979 raft_consensus.cc:515] T 68a6838d82ef4cac83507f906139d90f P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:36.467005 18058 raft_consensus.cc:515] T a378d7f796f94615b3b347119a9880fd P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:36.467783 16145 raft_consensus.cc:2393] T 68a6838d82ef4cac83507f906139d90f P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 5: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
I20260709 13:19:36.469368 17979 leader_election.cc:290] T 68a6838d82ef4cac83507f906139d90f P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 5 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:36.443068 17892 leader_election.cc:290] T 18ad7e3af5504571a69d10c00750a01c P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 3 election: Requested vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:19:36.469911 18058 leader_election.cc:290] T a378d7f796f94615b3b347119a9880fd P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:19:36.471093 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31969392ccdb43b491911a319e8bc7e1" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:19:36.472587 16145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434bff3082d141838068f323bdb0d1ed" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:19:36.472846 16292 raft_consensus.cc:1240] T 3884f864a96d428785895e5463e5728f P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 1. Current term is 2. Ops: []
I20260709 13:19:36.473162 16816 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49896 (request call id 747) took 1592 ms. Trace:
I20260709 13:19:36.474328 16816 rpcz_store.cc:276] 0709 13:19:34.880685 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:34.882104 (+  1419us) service_pool.cc:224] Handling call
0709 13:19:34.882412 (+   308us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:35.164941 (+282529us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:36.472663 (+1307722us) spinlock_profiling.cc:243] Waited 206 ms on lock 0x7b54006bf654. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:19:36.472971 (+   308us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":474038656}
I20260709 13:19:36.475359 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cc959fb7f704fba9009c0595467d074" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:19:36.473287 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6f07842d3aa47858df5665874c86f3a" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:19:36.476758 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6f07842d3aa47858df5665874c86f3a" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:36.432785 18044 leader_election.cc:290] T 66f75124215f4ca6bef831e174a792d8 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:19:36.437110 18023 leader_election.cc:290] T 66ae176961b74813a58bc15c9c4c6f2a P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:19:36.534832 17644 raft_consensus.cc:3055] T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:19:36.535477 17644 raft_consensus.cc:740] T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
W20260709 13:19:36.535787 16789 leader_election.cc:336] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389): Timed out: RequestConsensusVote RPC to 127.14.177.131:41389 timed out after 1.644s (SENT)
I20260709 13:19:36.536306 18043 raft_consensus.cc:493] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:19:36.536542 16789 leader_election.cc:304] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:19:36.536682 17644 consensus_queue.cc:260] T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:36.537092 18043 raft_consensus.cc:515] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:36.538458 17644 raft_consensus.cc:3060] T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:19:36.538393 17998 raft_consensus.cc:2749] T ee9ed67006144797b1de6ea8c2013a68 P cb82cc7bd6db46d980c75237a1402860 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:19:36.539471 17998 raft_consensus.cc:2749] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:19:36.540362 17979 consensus_queue.cc:1059] T 3884f864a96d428785895e5463e5728f P 647a343a542d4a139fe106c19095129b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:19:36.478087 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c2095a5f4ad49408dbc7f69c606120d" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:19:36.478001 16145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c2095a5f4ad49408dbc7f69c606120d" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:19:36.541832 17892 raft_consensus.cc:3055] T 3884f864a96d428785895e5463e5728f P 647a343a542d4a139fe106c19095129b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:19:36.526934 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e4bfb9090c8460eb14fa7e0d818e7f1" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
W20260709 13:19:36.535682 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 104869us
I20260709 13:19:36.479488 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f5f89186204242bf377f7c42b2e2da" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:19:36.548909 16150 raft_consensus.cc:2468] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 3.
I20260709 13:19:36.534852 16219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c11074579b046d99e585c2818e18a2c" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:19:36.549533 16223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f5f89186204242bf377f7c42b2e2da" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:36.556160 16146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57a744f5aaf944678a63469cbf514015" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:19:36.558905 16222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4f65cdf00b748bb963ee6e81ef43b79" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:19:36.561288 16148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18ad7e3af5504571a69d10c00750a01c" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:19:36.562284 16148 raft_consensus.cc:2393] T 18ad7e3af5504571a69d10c00750a01c P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 3: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
I20260709 13:19:36.563205 16148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c74490c40243a88e62d1760b5cd814" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:19:36.564159 16148 raft_consensus.cc:2393] T 08c74490c40243a88e62d1760b5cd814 P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 4: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
I20260709 13:19:36.565050 16148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a96373bb0e14dce8a266b2fc6c58507" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:19:36.565809 16148 raft_consensus.cc:3060] T 5a96373bb0e14dce8a266b2fc6c58507 P 647a343a542d4a139fe106c19095129b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:19:36.544291 17892 raft_consensus.cc:740] T 3884f864a96d428785895e5463e5728f P 647a343a542d4a139fe106c19095129b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
W20260709 13:19:36.544546 16294 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40474 (request call id 259) took 1160 ms (client timeout 1524 ms). Trace:
I20260709 13:19:36.568727 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada5a004ba7c46819051e3ccec71ffdb" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:36.476785 16149 raft_consensus.cc:2468] T b6f07842d3aa47858df5665874c86f3a P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 2.
I20260709 13:19:36.570487 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18ad7e3af5504571a69d10c00750a01c" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:19:36.571183 16265 leader_election.cc:304] T d16dfad0c3b540e5b4b86492ddf87208 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:19:36.575227 16145 raft_consensus.cc:3060] T 7c2095a5f4ad49408dbc7f69c606120d P 647a343a542d4a139fe106c19095129b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:19:36.575407 16219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c74490c40243a88e62d1760b5cd814" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:19:36.576725 16219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f2da98864b4425bf7c0055951fdd4b" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:19:36.577867 16219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a96373bb0e14dce8a266b2fc6c58507" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:19:36.579488 16817 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49906 (request call id 437) took 1617 ms. Trace:
I20260709 13:19:36.579927 16817 rpcz_store.cc:276] 0709 13:19:34.961790 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:34.962679 (+   889us) service_pool.cc:224] Handling call
0709 13:19:35.044331 (+ 81652us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:35.088945 (+ 44614us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:35.091676 (+  2731us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:35.278433 (+186757us) log.cc:844] Serialized 572 byte log entry
0709 13:19:36.059263 (+780830us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:19:36.059409 (+   146us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:36.059547 (+   138us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:36.579225 (+519678us) raft_consensus.cc:1685] finished
0709 13:19:36.579245 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:36.579419 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":734426,"threads_started":1,"wal-append.queue_time_us":1281678}
I20260709 13:19:36.546435 18046 raft_consensus.cc:697] T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045 [term 3 LEADER]: Becoming Leader. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:19:36.587213 18063 raft_consensus.cc:1064] T 19baa7bc367844c288bca8f81439c413 P 1ca2725d290e47d68b1a4b609c9816ac: attempting to promote NON_VOTER cb82cc7bd6db46d980c75237a1402860 to VOTER
I20260709 13:19:36.587694 16222 raft_consensus.cc:2393] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 5: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
I20260709 13:19:36.588173 16825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada5a004ba7c46819051e3ccec71ffdb" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:19:36.589217 16825 raft_consensus.cc:2368] T ada5a004ba7c46819051e3ccec71ffdb P cb82cc7bd6db46d980c75237a1402860 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 for earlier term 3. Current term is 4.
W20260709 13:19:36.587667 18063 raft_consensus.cc:1068] T 19baa7bc367844c288bca8f81439c413 P 1ca2725d290e47d68b1a4b609c9816ac: Unable to promote non-voter cb82cc7bd6db46d980c75237a1402860: Illegal state: Leader has not yet committed an operation in its own term
I20260709 13:19:36.584461 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2794af5b99984efebdee564035841cb0" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:19:36.583528 16223 raft_consensus.cc:2468] T c2f5f89186204242bf377f7c42b2e2da P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 4.
I20260709 13:19:36.594992 16146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f2da98864b4425bf7c0055951fdd4b" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:19:36.596407 16220 raft_consensus.cc:2468] T 1df93779209748b9860f6d893dd3de53 P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 3.
I20260709 13:19:36.584126 16142 raft_consensus.cc:2468] T 8cc959fb7f704fba9009c0595467d074 P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 3.
I20260709 13:19:36.599726 16146 raft_consensus.cc:1240] T 8cc959fb7f704fba9009c0595467d074 P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 2. Current term is 3. Ops: []
W20260709 13:19:36.602414 17794 consensus_meta.cc:287] T 5a959dbecc2648eb85d1cc550f5a3245 P 90f3da57796b488c9087a2f523adf045: Time spent flushing consensus metadata: real 1.299s	user 0.003s	sys 0.015s
I20260709 13:19:36.603027 17794 raft_consensus.cc:515] T 5a959dbecc2648eb85d1cc550f5a3245 P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:36.604373 16296 raft_consensus.cc:1240] T 5a959dbecc2648eb85d1cc550f5a3245 P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 1. Current term is 2. Ops: []
I20260709 13:19:36.606168 16296 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48684 (request call id 1861) took 1387 ms. Trace:
I20260709 13:19:36.608347 18057 raft_consensus.cc:493] T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:36.608853 18057 raft_consensus.cc:515] T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:36.609364 16293 raft_consensus.cc:1240] T 8158bbd17b994d2eb96f560bd0f1b88d P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 3. Current term is 4. Ops: []
I20260709 13:19:36.610086 16293 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48684 (request call id 1862) took 1385 ms. Trace:
I20260709 13:19:36.610394 16293 rpcz_store.cc:276] 0709 13:19:35.228240 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:36.608673 (+1380433us) service_pool.cc:224] Handling call
0709 13:19:36.609063 (+   390us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:36.609812 (+   749us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:36.610012 (+   200us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:36.613487 16143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87ac0d1f7e8a419f9bcbc04aee0e067e" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:19:36.614482 16143 raft_consensus.cc:2410] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 for term 4 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:19:36.606101 17794 leader_election.cc:290] T 5a959dbecc2648eb85d1cc550f5a3245 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 2 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:19:36.606691 16296 rpcz_store.cc:276] 0709 13:19:35.218518 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:36.452869 (+1234351us) service_pool.cc:224] Handling call
0709 13:19:36.549650 (+ 96781us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:36.604792 (+ 55142us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:36.605919 (+  1127us) spinlock_profiling.cc:243] Waited 54.5 ms on lock 0x7b540010e354. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:19:36.606091 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":125291904}
I20260709 13:19:36.620115 16291 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48674 (request call id 1971) took 1589 ms. Trace:
I20260709 13:19:36.620926 16790 leader_election.cc:304] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:19:36.654021 16291 rpcz_store.cc:276] 0709 13:19:35.031013 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:36.415802 (+1384789us) service_pool.cc:224] Handling call
0709 13:19:36.570949 (+155147us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:36.619643 (+ 48694us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:36.619675 (+    32us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:36.619753 (+    78us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:19:36.619798 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:36.619873 (+    75us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:36.620033 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:36.655017 16296 raft_consensus.cc:1240] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 13:19:36.655448 17998 raft_consensus.cc:2749] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:19:36.656347 18052 consensus_queue.cc:1048] T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:19:36.621688 18071 consensus_queue.cc:1059] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:19:36.625087 16821 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49906 (request call id 441) took 1298 ms. Trace:
W20260709 13:19:36.568536 16294 rpcz_store.cc:269] 0709 13:19:35.367218 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:35.408806 (+ 41588us) service_pool.cc:224] Handling call
0709 13:19:36.504945 (+1096139us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:36.628002 18041 raft_consensus.cc:515] T 9e65676b21b748728ad39668cfe722ec P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:36.630707 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cc959fb7f704fba9009c0595467d074" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:36.630368 18046 consensus_queue.cc:237] T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:36.631902 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c11074579b046d99e585c2818e18a2c" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:36.634843 17989 raft_consensus.cc:3055] T 434bff3082d141838068f323bdb0d1ed P cb82cc7bd6db46d980c75237a1402860 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:19:36.594379 17892 consensus_queue.cc:260] T 3884f864a96d428785895e5463e5728f P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:36.585438 18043 leader_election.cc:290] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:19:36.593284 18023 raft_consensus.cc:2749] T d16dfad0c3b540e5b4b86492ddf87208 P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:19:36.642704 16825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5718c623ebee4d2bb770f0f9ddb2f164" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:19:36.644105 16817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62fe3346cd3c42c4a55e14af9073bca4" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:19:36.645110 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af57996a8be84d398f0f023e8d08ef77" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:19:36.647492 16293 raft_consensus.cc:1240] T 8c11074579b046d99e585c2818e18a2c P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 3. Current term is 4. Ops: []
I20260709 13:19:36.676316 16217 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57604 (request call id 1891) took 1472 ms. Trace:
I20260709 13:19:36.676610 16293 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48684 (request call id 1864) took 1424 ms. Trace:
I20260709 13:19:36.676769 16217 rpcz_store.cc:276] 0709 13:19:35.203491 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:35.214489 (+ 10998us) service_pool.cc:224] Handling call
0709 13:19:36.644888 (+1430399us) spinlock_profiling.cc:243] Waited 5.57 ms on lock 0x7b5800010b98. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69ad8fadb1 00007f69ada3ab71 00007f69ada3aeee 0000000000375939 00007f69b5aae1e0 00007f69b5b770c4 00007f69b5b77131 00007f69b5b250a0 00007f69b5b24984 00007f69af09f6d4 00007f69af09f662
0709 13:19:36.676226 (+ 31338us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":12817792}
I20260709 13:19:36.677328 16293 rpcz_store.cc:276] 0709 13:19:35.251817 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:36.646557 (+1394740us) service_pool.cc:224] Handling call
0709 13:19:36.646992 (+   435us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:36.676170 (+ 29178us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:36.676510 (+   340us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:36.611027 18057 leader_election.cc:290] T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
W20260709 13:19:36.648721 18053 log.cc:927] Time spent T 87ac0d1f7e8a419f9bcbc04aee0e067e P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 1.028s	user 0.008s	sys 0.019s
I20260709 13:19:36.649595 16818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26354361e9b044cea1c2b86c4882fe92" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:19:36.679373 16818 raft_consensus.cc:3060] T 26354361e9b044cea1c2b86c4882fe92 P cb82cc7bd6db46d980c75237a1402860 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:19:36.680917 16297 raft_consensus.cc:2393] T 8cc959fb7f704fba9009c0595467d074 P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 3: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
W20260709 13:19:36.681422 16297 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48674 (request call id 1964) took 1653 ms (client timeout 2056 ms). Trace:
W20260709 13:19:36.681751 16297 rpcz_store.cc:269] 0709 13:19:35.027910 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:36.630222 (+1602312us) service_pool.cc:224] Handling call
0709 13:19:36.681345 (+ 51123us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:36.650079 17972 raft_consensus.cc:493] T ada5a004ba7c46819051e3ccec71ffdb P cb82cc7bd6db46d980c75237a1402860 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:36.655820 16296 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40474 (request call id 253) took 1319 ms. Trace:
I20260709 13:19:36.663206 16145 raft_consensus.cc:2468] T 7c2095a5f4ad49408dbc7f69c606120d P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 2.
I20260709 13:19:36.659083 16821 rpcz_store.cc:276] 0709 13:19:35.326825 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:35.327175 (+   350us) service_pool.cc:224] Handling call
0709 13:19:35.327546 (+   371us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:35.356596 (+ 29050us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:36.624621 (+1268025us) spinlock_profiling.cc:243] Waited 23.9 ms on lock 0x7b54004b8cd4. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:19:36.624894 (+   273us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":54903040}
I20260709 13:19:36.668591 16150 raft_consensus.cc:2468] T 2794af5b99984efebdee564035841cb0 P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 2.
W20260709 13:19:36.667650 16298 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40474 (request call id 251) took 1823 ms (client timeout 1906 ms). Trace:
I20260709 13:19:36.666126 17989 raft_consensus.cc:740] T 434bff3082d141838068f323bdb0d1ed P cb82cc7bd6db46d980c75237a1402860 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:19:36.685568 16296 rpcz_store.cc:276] 0709 13:19:35.336170 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:36.654096 (+1317926us) service_pool.cc:224] Handling call
0709 13:19:36.654550 (+   454us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:36.655544 (+   994us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:36.655765 (+   221us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:19:36.684788 16298 rpcz_store.cc:269] 0709 13:19:34.844708 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:34.867128 (+ 22420us) service_pool.cc:224] Handling call
0709 13:19:36.631577 (+1764449us) spinlock_profiling.cc:243] Waited 1.66 ms on lock 0x7b5800090098. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69ad8fadb1 00007f69ada3ab71 00007f69ada3aa22 00007f69b6953c09 00007f69b5aadee0 00007f69b5b770ab 00007f69b5b77131 00007f69b5b250a0 00007f69b5b24984 00007f69af09f6d4 00007f69af09f662
0709 13:19:36.667589 (+ 36012us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3812224}
I20260709 13:19:36.687930 16297 raft_consensus.cc:1240] T 9e65676b21b748728ad39668cfe722ec P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 3. Current term is 4. Ops: []
I20260709 13:19:36.699698 16298 raft_consensus.cc:1240] T 2794af5b99984efebdee564035841cb0 P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 1. Current term is 2. Ops: []
I20260709 13:19:36.701224 16298 raft_consensus.cc:1240] T 57a744f5aaf944678a63469cbf514015 P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 1. Current term is 2. Ops: []
I20260709 13:19:36.702306 16298 raft_consensus.cc:1240] T 434bff3082d141838068f323bdb0d1ed P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 3. Current term is 4. Ops: []
I20260709 13:19:36.703958 17989 consensus_queue.cc:260] T 434bff3082d141838068f323bdb0d1ed P cb82cc7bd6db46d980c75237a1402860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:36.705120 17989 raft_consensus.cc:3060] T 434bff3082d141838068f323bdb0d1ed P cb82cc7bd6db46d980c75237a1402860 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:19:36.706727 16818 raft_consensus.cc:2468] T 26354361e9b044cea1c2b86c4882fe92 P cb82cc7bd6db46d980c75237a1402860 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 2.
I20260709 13:19:36.693377 18043 raft_consensus.cc:493] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:36.716935 16191 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.14.177.129:36741 timed out after 1.501s (SENT) [suppressed 42 similar messages]
I20260709 13:19:36.739264 16145 raft_consensus.cc:1240] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 2. Current term is 3. Ops: []
I20260709 13:19:36.740301 16148 raft_consensus.cc:2468] T 5a96373bb0e14dce8a266b2fc6c58507 P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 2.
W20260709 13:19:36.747238 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 243697us
I20260709 13:19:36.751463 16150 raft_consensus.cc:1240] T af57996a8be84d398f0f023e8d08ef77 P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 2. Current term is 3. Ops: []
I20260709 13:19:36.752473 16820 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49896 (request call id 748) took 1871 ms. Trace:
I20260709 13:19:36.753337 16150 raft_consensus.cc:1240] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 2. Current term is 3. Ops: []
W20260709 13:19:36.754946 16789 leader_election.cc:336] T 8c11074579b046d99e585c2818e18a2c P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389): Timed out: RequestConsensusVote RPC to 127.14.177.131:41389 timed out after 1.906s (SENT)
I20260709 13:19:36.755487 16789 leader_election.cc:304] T 8c11074579b046d99e585c2818e18a2c P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:19:36.752838 16820 rpcz_store.cc:276] 0709 13:19:34.881000 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:34.881566 (+   566us) service_pool.cc:224] Handling call
0709 13:19:34.881900 (+   334us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:35.087589 (+205689us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:36.752022 (+1664433us) spinlock_profiling.cc:243] Waited 205 ms on lock 0x7b54006dbfd4. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:19:36.752315 (+   293us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":471733504}
W20260709 13:19:36.757694 16192 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 44065.9us
I20260709 13:19:36.751628 16144 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46224 (request call id 231) took 1641 ms. Trace:
I20260709 13:19:36.734634 16817 raft_consensus.cc:2468] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 4.
W20260709 13:19:36.761494 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 105308us
I20260709 13:19:36.743930 16142 raft_consensus.cc:1240] T 7c2095a5f4ad49408dbc7f69c606120d P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 1. Current term is 2. Ops: []
I20260709 13:19:36.760815 16144 rpcz_store.cc:276] 0709 13:19:35.109845 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:35.110442 (+   597us) service_pool.cc:224] Handling call
0709 13:19:35.110816 (+   374us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:35.246689 (+135873us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:36.751152 (+1504463us) spinlock_profiling.cc:243] Waited 135 ms on lock 0x7b540020ba54. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:19:36.751452 (+   300us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":311118720}
I20260709 13:19:36.766394 16150 raft_consensus.cc:1240] T 829b4ad8c5a54eababeea1181b960d4b P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 3. Current term is 4. Ops: []
I20260709 13:19:36.694103 17972 raft_consensus.cc:515] T ada5a004ba7c46819051e3ccec71ffdb P cb82cc7bd6db46d980c75237a1402860 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:19:36.743355 16818 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49892 (request call id 295) took 1376 ms (client timeout 1827 ms). Trace:
I20260709 13:19:36.741962 18043 raft_consensus.cc:515] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:36.777900 16144 raft_consensus.cc:1240] T 5a96373bb0e14dce8a266b2fc6c58507 P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 1. Current term is 2. Ops: []
I20260709 13:19:36.779836 16148 raft_consensus.cc:1240] T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 2. Current term is 3. Ops: []
I20260709 13:19:36.776393 16291 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40474 (request call id 254) took 1436 ms. Trace:
I20260709 13:19:36.792729 16192 leader_election.cc:304] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:19:36.781841 18034 raft_consensus.cc:2749] T 8c11074579b046d99e585c2818e18a2c P cb82cc7bd6db46d980c75237a1402860 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:19:36.804133 16144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66f75124215f4ca6bef831e174a792d8" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:19:36.805446 17981 consensus_queue.cc:260] T 3a7ca61592e848ea9b0fb81a2ce33cdc P cb82cc7bd6db46d980c75237a1402860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:36.807103 17981 raft_consensus.cc:3060] T 3a7ca61592e848ea9b0fb81a2ce33cdc P cb82cc7bd6db46d980c75237a1402860 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:19:36.807238 18057 consensus_queue.cc:1059] T 57a744f5aaf944678a63469cbf514015 P 647a343a542d4a139fe106c19095129b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
W20260709 13:19:36.772276 16818 rpcz_store.cc:269] 0709 13:19:35.360550 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:35.394187 (+ 33637us) service_pool.cc:224] Handling call
0709 13:19:36.649334 (+1255147us) spinlock_profiling.cc:243] Waited 2.44 ms on lock 0x7b5800004298. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69ad8fadb1 00007f69ada3ab71 00007f69ada3aeee 0000000000375939 00007f69b5aae1e0 00007f69b5b770c4 00007f69b5b77131 00007f69b5b250a0 00007f69b5b24984 00007f69af09f6d4 00007f69af09f662
0709 13:19:36.737209 (+ 87875us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5617024}
I20260709 13:19:36.831698 16145 raft_consensus.cc:1240] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 3. Current term is 4. Ops: []
I20260709 13:19:36.829577 17644 raft_consensus.cc:3055] T 57a744f5aaf944678a63469cbf514015 P 647a343a542d4a139fe106c19095129b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:19:36.836787 17644 raft_consensus.cc:740] T 57a744f5aaf944678a63469cbf514015 P 647a343a542d4a139fe106c19095129b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:19:36.831281 18041 leader_election.cc:290] T 9e65676b21b748728ad39668cfe722ec P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 4 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
W20260709 13:19:36.841003 16114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:36.807885 16142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746aabf856ab401c9ef5e859371e7b67" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:19:36.795404 18007 consensus_queue.cc:1059] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.043s
I20260709 13:19:36.844280 16818 raft_consensus.cc:1275] T ada5a004ba7c46819051e3ccec71ffdb P cb82cc7bd6db46d980c75237a1402860 [term 4 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:19:36.851982 18006 raft_consensus.cc:2804] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:19:36.852434 18006 raft_consensus.cc:493] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Starting leader election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:19:36.793164 16291 rpcz_store.cc:276] 0709 13:19:35.336617 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:36.683589 (+1346972us) service_pool.cc:224] Handling call
0709 13:19:36.763700 (+ 80111us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:36.765996 (+  2296us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:36.766029 (+    33us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:36.766093 (+    64us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:19:36.766124 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:36.766184 (+    60us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:36.773334 (+  7150us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:36.874105 18007 raft_consensus.cc:3055] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:19:36.874558 18007 raft_consensus.cc:740] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:19:36.875123 16142 raft_consensus.cc:2393] T 746aabf856ab401c9ef5e859371e7b67 P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 in current term 3: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
W20260709 13:19:36.880208 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 44242.5us
I20260709 13:19:36.852757 17972 leader_election.cc:290] T ada5a004ba7c46819051e3ccec71ffdb P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:36.852804 18006 raft_consensus.cc:3060] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:19:36.877693 16225 raft_consensus.cc:1275] T d2ab0ac8347641e6955e9bce7d90c1a2 P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:19:36.879518 16190 leader_election.cc:304] T c2f5f89186204242bf377f7c42b2e2da P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
W20260709 13:19:36.898623 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 44500.1us
I20260709 13:19:36.875643 18039 consensus_queue.cc:1048] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.016s
W20260709 13:19:36.915123 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 198353us
I20260709 13:19:36.915509 17644 consensus_queue.cc:260] T 57a744f5aaf944678a63469cbf514015 P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:36.913576 18046 raft_consensus.cc:493] T 23f82224f4564fea93572e8206bad1fe P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
W20260709 13:19:36.919323 16299 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40474 (request call id 232) took 3268 ms (client timeout 1912 ms). Trace:
I20260709 13:19:36.919385 16291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada5a004ba7c46819051e3ccec71ffdb" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:36.920387 16291 raft_consensus.cc:2468] T ada5a004ba7c46819051e3ccec71ffdb P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 2.
I20260709 13:19:36.921531 16294 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40474 (request call id 255) took 1578 ms. Trace:
I20260709 13:19:36.921989 16294 rpcz_store.cc:276] 0709 13:19:35.342787 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:36.678222 (+1335435us) service_pool.cc:224] Handling call
0709 13:19:36.678611 (+   389us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:36.679020 (+   409us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:36.679051 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:36.679599 (+   548us) log.cc:844] Serialized 26 byte log entry
0709 13:19:36.769261 (+ 89662us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:19:36.772492 (+  3231us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:36.774842 (+  2350us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:36.921295 (+146453us) raft_consensus.cc:1685] finished
0709 13:19:36.921315 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:36.921478 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":465,"raft.run_cpu_time_us":157,"raft.run_wall_time_us":162,"thread_start_us":83635,"threads_started":1,"wal-append.queue_time_us":120494}
I20260709 13:19:36.923290 16300 raft_consensus.cc:1240] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 1. Current term is 2. Ops: []
W20260709 13:19:36.919915 16299 rpcz_store.cc:269] 0709 13:19:33.644004 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:33.648182 (+  4178us) service_pool.cc:224] Handling call
0709 13:19:36.886720 (+3238538us) spinlock_profiling.cc:243] Waited 1.04 s on lock 0x7b540030c0d4. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:19:36.912326 (+ 25606us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2392619904}
I20260709 13:19:36.959414 17644 raft_consensus.cc:3060] T 57a744f5aaf944678a63469cbf514015 P 647a343a542d4a139fe106c19095129b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:19:36.950312 18042 raft_consensus.cc:2804] T c2f5f89186204242bf377f7c42b2e2da P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:19:36.919869 18046 raft_consensus.cc:515] T 23f82224f4564fea93572e8206bad1fe P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:19:36.962742 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 45657us
I20260709 13:19:36.967864 17844 raft_consensus.cc:493] T 66f75124215f4ca6bef831e174a792d8 P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:19:36.968536 17844 raft_consensus.cc:515] T 66f75124215f4ca6bef831e174a792d8 P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:36.958966 18043 leader_election.cc:290] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:36.962005 18042 raft_consensus.cc:493] T c2f5f89186204242bf377f7c42b2e2da P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:36.977157 16296 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40474 (request call id 262) took 1232 ms. Trace:
W20260709 13:19:36.977861 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 72147.3us
I20260709 13:19:36.940819 18007 consensus_queue.cc:260] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:19:36.978339 16789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 65015.3us
I20260709 13:19:36.981567 16820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66f75124215f4ca6bef831e174a792d8" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:19:36.977604 16296 rpcz_store.cc:276] 0709 13:19:35.739406 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:36.686968 (+947562us) service_pool.cc:224] Handling call
0709 13:19:36.737292 (+ 50324us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:36.744646 (+  7354us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:36.744690 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:36.771438 (+ 26748us) log.cc:844] Serialized 26 byte log entry
0709 13:19:36.941524 (+170086us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:19:36.941580 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:36.941691 (+   111us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:36.971691 (+ 30000us) raft_consensus.cc:1685] finished
0709 13:19:36.971709 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:36.971841 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1435,"raft.run_cpu_time_us":133,"raft.run_wall_time_us":2101,"thread_start_us":143745,"threads_started":1,"wal-append.queue_time_us":159058}
I20260709 13:19:36.985371 16820 raft_consensus.cc:2468] T 66f75124215f4ca6bef831e174a792d8 P cb82cc7bd6db46d980c75237a1402860 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 3.
I20260709 13:19:36.978145 18042 raft_consensus.cc:3060] T c2f5f89186204242bf377f7c42b2e2da P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:19:36.993461 16265 leader_election.cc:304] T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: cb82cc7bd6db46d980c75237a1402860
I20260709 13:19:36.984486 18046 leader_election.cc:290] T 23f82224f4564fea93572e8206bad1fe P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:19:36.999504 17900 raft_consensus.cc:493] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:19:37.000175 17900 raft_consensus.cc:515] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:37.000656 18078 consensus_queue.cc:1059] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:19:37.001679 18078 consensus_queue.cc:1059] T af57996a8be84d398f0f023e8d08ef77 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:19:37.002493 17900 leader_election.cc:290] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:37.002624 18078 raft_consensus.cc:3055] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:19:37.003014 18078 raft_consensus.cc:740] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:19:37.003492 16818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746aabf856ab401c9ef5e859371e7b67" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:19:37.003934 18078 consensus_queue.cc:260] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:37.004395 16816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af57996a8be84d398f0f023e8d08ef77" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:19:37.005345 18078 raft_consensus.cc:3060] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:19:37.006217 18046 raft_consensus.cc:493] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:37.007371 18041 raft_consensus.cc:493] T cdbd78acb4cf46298e0de8ba0d130e19 P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:37.008044 18041 raft_consensus.cc:515] T cdbd78acb4cf46298e0de8ba0d130e19 P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:37.010206 18041 leader_election.cc:290] T cdbd78acb4cf46298e0de8ba0d130e19 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:19:37.010605 17918 raft_consensus.cc:1064] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac: attempting to promote NON_VOTER cb82cc7bd6db46d980c75237a1402860 to VOTER
W20260709 13:19:37.011142 17918 raft_consensus.cc:1068] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac: Unable to promote non-voter cb82cc7bd6db46d980c75237a1402860: Illegal state: Leader has not yet committed an operation in its own term
I20260709 13:19:37.015137 17972 raft_consensus.cc:493] T 3a7ca61592e848ea9b0fb81a2ce33cdc P cb82cc7bd6db46d980c75237a1402860 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:37.015357 17900 raft_consensus.cc:3055] T af57996a8be84d398f0f023e8d08ef77 P cb82cc7bd6db46d980c75237a1402860 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:19:37.015739 17972 raft_consensus.cc:515] T 3a7ca61592e848ea9b0fb81a2ce33cdc P cb82cc7bd6db46d980c75237a1402860 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:37.019690 16192 leader_election.cc:304] T 66f75124215f4ca6bef831e174a792d8 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:19:37.020917 18007 raft_consensus.cc:3060] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:19:37.012632 16299 raft_consensus.cc:1240] T 7c2095a5f4ad49408dbc7f69c606120d P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 1. Current term is 2. Ops: []
I20260709 13:19:37.023661 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a9de347c711477e87c582d003f36925" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:19:37.026698 18089 raft_consensus.cc:493] T 8158bbd17b994d2eb96f560bd0f1b88d P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:19:37.027418 18089 raft_consensus.cc:515] T 8158bbd17b994d2eb96f560bd0f1b88d P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:37.021035 18011 raft_consensus.cc:2804] T 66f75124215f4ca6bef831e174a792d8 P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:19:37.032219 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a378d7f796f94615b3b347119a9880fd" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:37.015799 17900 raft_consensus.cc:740] T af57996a8be84d398f0f023e8d08ef77 P cb82cc7bd6db46d980c75237a1402860 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:19:37.032220 18050 consensus_queue.cc:1048] T d2ab0ac8347641e6955e9bce7d90c1a2 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.009s
I20260709 13:19:37.041580 16820 raft_consensus.cc:1275] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [term 4 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:19:37.042150 16294 raft_consensus.cc:1275] T 66ae176961b74813a58bc15c9c4c6f2a P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
W20260709 13:19:37.042982 16791 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:37.032851 18044 consensus_queue.cc:1059] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.010s
I20260709 13:19:37.033646 18011 raft_consensus.cc:493] T 66f75124215f4ca6bef831e174a792d8 P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Starting leader election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:19:37.043699 18011 raft_consensus.cc:3060] T 66f75124215f4ca6bef831e174a792d8 P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:19:37.044832 18063 consensus_queue.cc:1048] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:19:36.997908 18023 raft_consensus.cc:2804] T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:19:37.046754 18023 raft_consensus.cc:493] T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Starting leader election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:19:37.047000 18086 raft_consensus.cc:493] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P cb82cc7bd6db46d980c75237a1402860 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:19:37.047123 18023 raft_consensus.cc:3060] T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:19:37.047605 18086 raft_consensus.cc:515] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P cb82cc7bd6db46d980c75237a1402860 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:37.048543 17844 leader_election.cc:290] T 66f75124215f4ca6bef831e174a792d8 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:19:37.050805 16292 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48674 (request call id 1970) took 2020 ms. Trace:
I20260709 13:19:37.051216 16292 rpcz_store.cc:276] 0709 13:19:35.030632 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:36.278069 (+1247437us) service_pool.cc:224] Handling call
0709 13:19:36.472510 (+194441us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:36.473963 (+  1453us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:37.050573 (+576610us) spinlock_profiling.cc:243] Waited 179 ms on lock 0x7b5400178c50. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5e8ad 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:19:37.050745 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":412339456}
W20260709 13:19:37.051889 18087 log.cc:927] Time spent T 8638ee363273426db1393dd81e5c7b72 P cb82cc7bd6db46d980c75237a1402860: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 13:19:37.043716 16293 raft_consensus.cc:1240] T c0eab25ce4c64e018a041beeda2867db P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 2. Current term is 3. Ops: []
I20260709 13:19:37.053350 16220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2372981006d4b218215c28964dea623" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:19:37.060750 16220 raft_consensus.cc:2393] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 3: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
I20260709 13:19:37.060781 17844 consensus_queue.cc:1059] T 19baa7bc367844c288bca8f81439c413 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: INVALID_TERM, Last received: 2.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:19:37.062029 17844 consensus_queue.cc:1059] T 8cc959fb7f704fba9009c0595467d074 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:19:37.006913 18046 raft_consensus.cc:515] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } }
I20260709 13:19:37.064468 18086 leader_election.cc:290] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:37.064600 18044 raft_consensus.cc:3055] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:19:37.065078 18044 raft_consensus.cc:740] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:19:37.066087 18044 consensus_queue.cc:260] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:37.067287 17919 raft_consensus.cc:1064] T 19baa7bc367844c288bca8f81439c413 P 1ca2725d290e47d68b1a4b609c9816ac: attempting to promote NON_VOTER cb82cc7bd6db46d980c75237a1402860 to VOTER
W20260709 13:19:37.067718 17919 raft_consensus.cc:1068] T 19baa7bc367844c288bca8f81439c413 P 1ca2725d290e47d68b1a4b609c9816ac: Unable to promote non-voter cb82cc7bd6db46d980c75237a1402860: Illegal state: Leader has not yet committed an operation in its own term
I20260709 13:19:37.071669 18041 raft_consensus.cc:493] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:19:37.071931 17919 raft_consensus.cc:3055] T 19baa7bc367844c288bca8f81439c413 P 1ca2725d290e47d68b1a4b609c9816ac [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:19:37.072350 17919 raft_consensus.cc:740] T 19baa7bc367844c288bca8f81439c413 P 1ca2725d290e47d68b1a4b609c9816ac [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:19:37.072469 18041 raft_consensus.cc:515] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:37.073269 17919 consensus_queue.cc:260] T 19baa7bc367844c288bca8f81439c413 P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.4, Last appended by leader: 4, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } }
I20260709 13:19:37.070811 17794 raft_consensus.cc:493] T c9944d5e4b8c4109af9955c4f08c9095 P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:19:37.074658 17794 raft_consensus.cc:515] T c9944d5e4b8c4109af9955c4f08c9095 P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:37.075326 16300 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48684 (request call id 1851) took 2442 ms. Trace:
I20260709 13:19:37.075698 16300 rpcz_store.cc:276] 0709 13:19:34.633234 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:34.839119 (+205885us) service_pool.cc:224] Handling call
0709 13:19:36.897047 (+2057928us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:36.955487 (+ 58440us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:37.075074 (+119587us) spinlock_profiling.cc:243] Waited 2.05 s on lock 0x7b540030c0d0. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5e8ad 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:19:37.075249 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4709215488}
I20260709 13:19:37.076568 18044 raft_consensus.cc:3060] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:19:37.076820 17794 leader_election.cc:290] T c9944d5e4b8c4109af9955c4f08c9095 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
W20260709 13:19:37.068229 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 74889.1us
I20260709 13:19:37.073931 16190 leader_election.cc:304] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860
I20260709 13:19:37.079631 17918 raft_consensus.cc:2749] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:19:37.080562 17918 consensus_queue.cc:1059] T 18ad7e3af5504571a69d10c00750a01c P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:19:37.081207 18039 consensus_queue.cc:1059] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:19:37.081712 17918 consensus_queue.cc:1059] T 5a959dbecc2648eb85d1cc550f5a3245 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:19:37.082468 18039 raft_consensus.cc:3055] T 18ad7e3af5504571a69d10c00750a01c P 1ca2725d290e47d68b1a4b609c9816ac [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:19:37.082890 18039 raft_consensus.cc:740] T 18ad7e3af5504571a69d10c00750a01c P 1ca2725d290e47d68b1a4b609c9816ac [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:19:37.083736 17972 leader_election.cc:290] T 3a7ca61592e848ea9b0fb81a2ce33cdc P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:37.088809 16265 leader_election.cc:304] T 8638ee363273426db1393dd81e5c7b72 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860
W20260709 13:19:37.085350 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 53616.6us
I20260709 13:19:37.075129 18041 leader_election.cc:290] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:19:37.097481 16300 raft_consensus.cc:1240] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 2. Current term is 3. Ops: [2.3-2.3]
I20260709 13:19:37.087651 18063 raft_consensus.cc:3055] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:19:37.098152 18073 raft_consensus.cc:2749] T 8638ee363273426db1393dd81e5c7b72 P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:19:37.100515 16044 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:49252 (request call id 286) took 6496 ms. Trace:
I20260709 13:19:37.100569 16151 raft_consensus.cc:1240] T af57996a8be84d398f0f023e8d08ef77 P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 2. Current term is 3. Ops: []
I20260709 13:19:37.075133 17919 raft_consensus.cc:3060] T 19baa7bc367844c288bca8f81439c413 P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:19:37.103688 16146 raft_consensus.cc:1240] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [term 4 LEADER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 3. Current term is 4. Ops: [3.4-3.4]
I20260709 13:19:37.105736 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada5a004ba7c46819051e3ccec71ffdb" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:37.104354 18046 leader_election.cc:290] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:19:37.107404 16296 raft_consensus.cc:1240] T 5a96373bb0e14dce8a266b2fc6c58507 P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 1. Current term is 2. Ops: []
I20260709 13:19:37.108278 18063 raft_consensus.cc:740] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:19:37.110350 16292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5ad4eb281564936a59f41e387a47156" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:37.110661 16266 leader_election.cc:400] T ada5a004ba7c46819051e3ccec71ffdb P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 3 pre-election: Vote denied by peer cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923) with higher term. Message: Invalid argument: T ada5a004ba7c46819051e3ccec71ffdb P cb82cc7bd6db46d980c75237a1402860 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 for earlier term 3. Current term is 4.
I20260709 13:19:37.110721 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c61a39fd4f84424b129c1a6f9ba404d" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:37.112848 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746aabf856ab401c9ef5e859371e7b67" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:37.112986 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2372981006d4b218215c28964dea623" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:19:37.123421 17999 raft_consensus.cc:3055] T b87147731a68400e8cce9b879daa7c5d P 90f3da57796b488c9087a2f523adf045 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:19:37.124792 16266 leader_election.cc:403] T ada5a004ba7c46819051e3ccec71ffdb P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 3 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:19:37.128715 16821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b7d1ab519044eda2d5e04a07242f25" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:19:37.131820 16146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1df93779209748b9860f6d893dd3de53" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:19:37.132345 16146 raft_consensus.cc:2468] T 1df93779209748b9860f6d893dd3de53 P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 3.
I20260709 13:19:37.105054 17892 raft_consensus.cc:3060] T 3884f864a96d428785895e5463e5728f P 647a343a542d4a139fe106c19095129b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:19:37.133340 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1df93779209748b9860f6d893dd3de53" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:37.134179 16300 raft_consensus.cc:2393] T 1df93779209748b9860f6d893dd3de53 P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 3: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
I20260709 13:19:37.135010 18063 consensus_queue.cc:260] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:37.136354 18063 raft_consensus.cc:3060] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:19:37.137151 16818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5d59407eb545b895d8d922e0dd69fd" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:19:37.138537 16825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434bff3082d141838068f323bdb0d1ed" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:19:37.130535 17999 raft_consensus.cc:740] T b87147731a68400e8cce9b879daa7c5d P 90f3da57796b488c9087a2f523adf045 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:19:37.140285 16819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2372981006d4b218215c28964dea623" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:19:37.140635 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66ae176961b74813a58bc15c9c4c6f2a" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:37.141002 16820 raft_consensus.cc:1240] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860 [term 2 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 1. Current term is 2. Ops: []
I20260709 13:19:37.142704 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c74490c40243a88e62d1760b5cd814" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:19:37.105720 16044 rpcz_store.cc:276] 0709 13:19:30.531605 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:30.531738 (+   133us) service_pool.cc:224] Handling call
0709 13:19:31.731304 (+1199566us) spinlock_profiling.cc:243] Waited 5.44 ms on lock 0x7b54003cdda8. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b6d51331 00007f69b6d4c3fc 00007f69b691e1b4 00007f69b6a1c37e 00007f69b4edd2a4 00007f69b4edd232 00007f69b4edd161 00007f69b4edd0dc 00007f69b4edc392 00007f69aee9a48c 00007f69aee998c6
0709 13:19:37.027754 (+5296450us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:19:33.702980 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:19:33.955071 (+252091us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:19:33.959344 (+  4273us) write_op.cc:435] Acquired schema lock
0709 13:19:33.961625 (+  2281us) tablet.cc:662] Decoding operations
0709 13:19:34.092153 (+130528us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:19:34.109984 (+ 17831us) write_op.cc:641] Partition lock acquired for write op
0709 13:19:34.125840 (+ 15856us) tablet.cc:685] Acquiring locks for 2 operations
0709 13:19:34.144876 (+ 19036us) tablet.cc:701] Row locks acquired
0709 13:19:34.144905 (+    29us) write_op.cc:260] PREPARE: finished
0709 13:19:34.145117 (+   212us) write_op.cc:270] Start()
0709 13:19:34.145264 (+   147us) write_op.cc:276] Timestamp: P: 1783603174145065 usec, L: 0
0709 13:19:34.145294 (+    30us) op_driver.cc:348] REPLICATION: starting
0709 13:19:34.146146 (+   852us) log.cc:844] Serialized 935 byte log entry
0709 13:19:34.571620 (+425474us) op_driver.cc:464] REPLICATION: finished
0709 13:19:34.675837 (+104217us) write_op.cc:301] APPLY: starting
0709 13:19:34.678936 (+  3099us) tablet.cc:1367] starting BulkCheckPresence
0709 13:19:34.690492 (+ 11556us) tablet.cc:1370] finished BulkCheckPresence
0709 13:19:34.707182 (+ 16690us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:19:34.733266 (+ 26084us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:19:34.733466 (+   200us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0709 13:19:34.733515 (+    49us) write_op.cc:312] APPLY: finished
0709 13:19:34.733732 (+   217us) log.cc:844] Serialized 29 byte log entry
0709 13:19:34.733899 (+   167us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:19:34.734003 (+   104us) write_op.cc:454] Released schema lock
0709 13:19:34.734047 (+    44us) write_op.cc:341] FINISH: Updating metrics
0709 13:19:34.734415 (+   368us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:19:34.734445 (+    30us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":835330,"reported_tablets":8,"spinlock_wait_cycles":12511488,"child_traces":[["op",{"apply.queue_time_us":60004,"apply.run_cpu_time_us":2651,"apply.run_wall_time_us":76097,"num_ops":2,"prepare.queue_time_us":701597,"prepare.run_cpu_time_us":4854,"prepare.run_wall_time_us":443482,"replication_time_us":426143,"thread_start_us":662478,"threads_started":2}]]}
I20260709 13:19:37.143245 16219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b7d1ab519044eda2d5e04a07242f25" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:19:37.144272 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5d59407eb545b895d8d922e0dd69fd" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:19:37.145275 18089 leader_election.cc:290] T 8158bbd17b994d2eb96f560bd0f1b88d P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:37.145471 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5ad4eb281564936a59f41e387a47156" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:19:37.145640 16820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "829b4ad8c5a54eababeea1181b960d4b" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:19:37.146054 16217 raft_consensus.cc:3060] T e5ad4eb281564936a59f41e387a47156 P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:19:37.146375 16825 raft_consensus.cc:2468] T 434bff3082d141838068f323bdb0d1ed P cb82cc7bd6db46d980c75237a1402860 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 4.
I20260709 13:19:37.147121 16820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5718c623ebee4d2bb770f0f9ddb2f164" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:19:37.148149 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "829b4ad8c5a54eababeea1181b960d4b" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:19:37.136389 18023 raft_consensus.cc:515] T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:37.150187 18006 raft_consensus.cc:515] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:19:37.151361 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 62622.5us
I20260709 13:19:37.151718 16819 raft_consensus.cc:3060] T 19baa7bc367844c288bca8f81439c413 P cb82cc7bd6db46d980c75237a1402860 [term 3 LEARNER]: Advancing to term 4
I20260709 13:19:37.152171 16265 leader_election.cc:400] T 8a878999c0024d2abe517484180290fc P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 2 election: Vote denied by peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943) with higher term. Message: Invalid argument: T 8a878999c0024d2abe517484180290fc P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 for earlier term 2. Current term is 3.
I20260709 13:19:37.152470 16265 leader_election.cc:403] T 8a878999c0024d2abe517484180290fc P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 2 election: Cancelling election due to peer responding with higher term
I20260709 13:19:37.153407 16265 leader_election.cc:304] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860
I20260709 13:19:37.153540 18046 raft_consensus.cc:2749] T 8a878999c0024d2abe517484180290fc P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Leader election lost for term 2. Reason: Vote denied by peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943) with higher term. Message: Invalid argument: T 8a878999c0024d2abe517484180290fc P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 for earlier term 2. Current term is 3.
I20260709 13:19:37.154259 18046 raft_consensus.cc:2749] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:19:37.156002 17898 consensus_queue.cc:1059] T 829b4ad8c5a54eababeea1181b960d4b P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:19:37.157048 17898 consensus_queue.cc:1059] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:19:37.157253 17989 consensus_queue.cc:1059] T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:19:37.158248 17989 raft_consensus.cc:3055] T 829b4ad8c5a54eababeea1181b960d4b P cb82cc7bd6db46d980c75237a1402860 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:19:37.158298 16146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5718c623ebee4d2bb770f0f9ddb2f164" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:19:37.158654 17989 raft_consensus.cc:740] T 829b4ad8c5a54eababeea1181b960d4b P cb82cc7bd6db46d980c75237a1402860 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:19:37.159502 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c61a39fd4f84424b129c1a6f9ba404d" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:19:37.159770 16219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5718c623ebee4d2bb770f0f9ddb2f164" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:19:37.160140 16150 raft_consensus.cc:2468] T 9c61a39fd4f84424b129c1a6f9ba404d P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 3.
I20260709 13:19:37.160383 18050 raft_consensus.cc:3055] T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:19:37.160758 18050 raft_consensus.cc:740] T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:19:37.159103 16146 raft_consensus.cc:2393] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 3: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
I20260709 13:19:37.096274 18061 consensus_queue.cc:1048] T 68a6838d82ef4cac83507f906139d90f P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.066s
I20260709 13:19:37.161779 16219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f2da98864b4425bf7c0055951fdd4b" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:37.161779 18050 consensus_queue.cc:260] T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:37.083850 18039 consensus_queue.cc:260] T 18ad7e3af5504571a69d10c00750a01c P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } }
I20260709 13:19:37.159678 17989 consensus_queue.cc:260] T 829b4ad8c5a54eababeea1181b960d4b P cb82cc7bd6db46d980c75237a1402860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:37.163995 16219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ad5dbfa99ee4b8d9e28be92b3f36db5" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:19:37.164296 18039 raft_consensus.cc:3060] T 18ad7e3af5504571a69d10c00750a01c P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:19:37.164760 17989 raft_consensus.cc:3060] T 829b4ad8c5a54eababeea1181b960d4b P cb82cc7bd6db46d980c75237a1402860 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:19:37.166118 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18ad7e3af5504571a69d10c00750a01c" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:19:37.167229 16219 raft_consensus.cc:1240] T 1df93779209748b9860f6d893dd3de53 P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 2. Current term is 3. Ops: []
I20260709 13:19:37.167534 16149 raft_consensus.cc:2393] T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 3: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
I20260709 13:19:37.167883 18050 raft_consensus.cc:3060] T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:19:37.169535 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68a6838d82ef4cac83507f906139d90f" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:19:37.170130 16225 raft_consensus.cc:3060] T 68a6838d82ef4cac83507f906139d90f P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:19:37.171768 16221 raft_consensus.cc:1240] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 4. Current term is 5. Ops: []
I20260709 13:19:37.171864 16818 raft_consensus.cc:1240] T 26354361e9b044cea1c2b86c4882fe92 P cb82cc7bd6db46d980c75237a1402860 [term 2 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 1. Current term is 2. Ops: []
I20260709 13:19:37.160086 18086 consensus_queue.cc:1059] T af57996a8be84d398f0f023e8d08ef77 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.013s
I20260709 13:19:37.174980 16143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62fe3346cd3c42c4a55e14af9073bca4" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:19:37.175745 16142 raft_consensus.cc:1275] T 02838c86531d4f98bf58d1c22b8baf77 P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:19:37.176295 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5ad4eb281564936a59f41e387a47156" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:19:37.177053 16150 raft_consensus.cc:2393] T e5ad4eb281564936a59f41e387a47156 P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 4: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
I20260709 13:19:37.177927 16115 leader_election.cc:304] T 8a878999c0024d2abe517484180290fc P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:19:37.178593 16225 raft_consensus.cc:2468] T 68a6838d82ef4cac83507f906139d90f P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 5.
I20260709 13:19:37.179252 16145 raft_consensus.cc:1240] T 18ad7e3af5504571a69d10c00750a01c P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 2. Current term is 3. Ops: []
I20260709 13:19:37.178434 16151 raft_consensus.cc:1240] T 08c74490c40243a88e62d1760b5cd814 P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 3. Current term is 4. Ops: []
I20260709 13:19:37.179512 18055 consensus_queue.cc:1059] T 26354361e9b044cea1c2b86c4882fe92 P 647a343a542d4a139fe106c19095129b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:19:37.180186 16115 leader_election.cc:304] T 9c61a39fd4f84424b129c1a6f9ba404d P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:19:37.180658 16221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4f65cdf00b748bb963ee6e81ef43b79" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
W20260709 13:19:37.180850 16266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 70372us
I20260709 13:19:37.181030 17979 raft_consensus.cc:3055] T 26354361e9b044cea1c2b86c4882fe92 P 647a343a542d4a139fe106c19095129b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:19:37.181285 16221 raft_consensus.cc:2468] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 5.
I20260709 13:19:37.181492 17979 raft_consensus.cc:740] T 26354361e9b044cea1c2b86c4882fe92 P 647a343a542d4a139fe106c19095129b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:19:37.181604 16266 leader_election.cc:304] T 26354361e9b044cea1c2b86c4882fe92 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:19:37.182479 17979 consensus_queue.cc:260] T 26354361e9b044cea1c2b86c4882fe92 P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:37.183351 18073 raft_consensus.cc:3060] T ada5a004ba7c46819051e3ccec71ffdb P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Advancing to term 4
I20260709 13:19:37.182981 16115 leader_election.cc:304] T c0eab25ce4c64e018a041beeda2867db P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac
I20260709 13:19:37.183907 17979 raft_consensus.cc:3060] T 26354361e9b044cea1c2b86c4882fe92 P 647a343a542d4a139fe106c19095129b [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:19:37.184106 16117 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:37.184448 16221 raft_consensus.cc:1240] T d16dfad0c3b540e5b4b86492ddf87208 P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 1. Current term is 2. Ops: []
I20260709 13:19:37.186033 18057 raft_consensus.cc:2804] T c0eab25ce4c64e018a041beeda2867db P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:19:37.186430 18057 raft_consensus.cc:493] T c0eab25ce4c64e018a041beeda2867db P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Starting leader election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:19:37.186784 18057 raft_consensus.cc:3060] T c0eab25ce4c64e018a041beeda2867db P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:19:37.180817 18055 raft_consensus.cc:2804] T 8a878999c0024d2abe517484180290fc P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:19:37.187810 18055 raft_consensus.cc:697] T 8a878999c0024d2abe517484180290fc P 647a343a542d4a139fe106c19095129b [term 3 LEADER]: Becoming Leader. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:19:37.188813 18055 consensus_queue.cc:237] T 8a878999c0024d2abe517484180290fc P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:19:37.192895 16790 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:37.196244 16265 leader_election.cc:304] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860
I20260709 13:19:37.172601 18023 leader_election.cc:290] T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 3 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:19:37.192091 16149 raft_consensus.cc:1240] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b [term 3 LEADER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 2. Current term is 3. Ops: [2.3-2.3]
I20260709 13:19:37.177910 16819 raft_consensus.cc:1275] T 19baa7bc367844c288bca8f81439c413 P cb82cc7bd6db46d980c75237a1402860 [term 4 LEARNER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 4 index: 4. (term mismatch)
I20260709 13:19:37.181634 18013 raft_consensus.cc:2749] T 9c61a39fd4f84424b129c1a6f9ba404d P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:19:37.211622 16819 pending_rounds.cc:85] T 19baa7bc367844c288bca8f81439c413 P cb82cc7bd6db46d980c75237a1402860: Aborting all ops after (but not including) 3
I20260709 13:19:37.213145 16789 leader_election.cc:304] T 2794af5b99984efebdee564035841cb0 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
W20260709 13:19:37.218169 16114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:37.161834 18007 consensus_queue.cc:1059] T 8c11074579b046d99e585c2818e18a2c P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.050s
I20260709 13:19:37.198443 16217 raft_consensus.cc:2468] T e5ad4eb281564936a59f41e387a47156 P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 4.
W20260709 13:19:37.223279 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 68122.3us
I20260709 13:19:37.226526 16115 leader_election.cc:304] T 9e65676b21b748728ad39668cfe722ec P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: cb82cc7bd6db46d980c75237a1402860
I20260709 13:19:37.227602 18052 raft_consensus.cc:3055] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:19:37.228000 16115 leader_election.cc:304] T fffab26427e24d398e2695aad40ca134 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:19:37.228195 18013 raft_consensus.cc:2804] T 9e65676b21b748728ad39668cfe722ec P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:19:37.212127 16220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66f75124215f4ca6bef831e174a792d8" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:37.214071 18041 raft_consensus.cc:2749] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:19:37.242350 16821 raft_consensus.cc:2468] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 3.
W20260709 13:19:37.242260 16791 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:37.244504 16265 leader_election.cc:304] T 8c11074579b046d99e585c2818e18a2c P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860
I20260709 13:19:37.215654 16819 pending_rounds.cc:107] T 19baa7bc367844c288bca8f81439c413 P cb82cc7bd6db46d980c75237a1402860: Aborting uncommitted NO_OP operation due to leader change: 3.4
I20260709 13:19:37.239094 18042 raft_consensus.cc:515] T c2f5f89186204242bf377f7c42b2e2da P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:37.247318 16819 raft_consensus.cc:2889] T 19baa7bc367844c288bca8f81439c413 P cb82cc7bd6db46d980c75237a1402860 [term 4 LEARNER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:19:37.249279 18042 leader_election.cc:290] T c2f5f89186204242bf377f7c42b2e2da P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 5 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:37.249996 17900 consensus_queue.cc:260] T af57996a8be84d398f0f023e8d08ef77 P cb82cc7bd6db46d980c75237a1402860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:37.251593 18035 raft_consensus.cc:3055] T 8cc959fb7f704fba9009c0595467d074 P 1ca2725d290e47d68b1a4b609c9816ac [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:19:37.251595 17900 raft_consensus.cc:3060] T af57996a8be84d398f0f023e8d08ef77 P cb82cc7bd6db46d980c75237a1402860 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:19:37.218505 18061 raft_consensus.cc:2804] T 26354361e9b044cea1c2b86c4882fe92 P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:19:37.252002 18035 raft_consensus.cc:740] T 8cc959fb7f704fba9009c0595467d074 P 1ca2725d290e47d68b1a4b609c9816ac [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:19:37.254482 18061 raft_consensus.cc:697] T 26354361e9b044cea1c2b86c4882fe92 P 90f3da57796b488c9087a2f523adf045 [term 2 LEADER]: Becoming Leader. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:19:37.254977 18035 consensus_queue.cc:260] T 8cc959fb7f704fba9009c0595467d074 P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:37.255491 18061 consensus_queue.cc:237] T 26354361e9b044cea1c2b86c4882fe92 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:37.256326 18035 raft_consensus.cc:3060] T 8cc959fb7f704fba9009c0595467d074 P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:19:37.239490 18052 raft_consensus.cc:740] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
W20260709 13:19:37.259150 16789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 51485.5us
I20260709 13:19:37.259503 18052 consensus_queue.cc:260] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:37.260995 18052 raft_consensus.cc:3060] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:19:37.262506 18055 raft_consensus.cc:2749] T fffab26427e24d398e2695aad40ca134 P 647a343a542d4a139fe106c19095129b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:19:37.239652 18013 raft_consensus.cc:493] T 9e65676b21b748728ad39668cfe722ec P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Starting leader election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:19:37.265956 18013 raft_consensus.cc:3060] T 9e65676b21b748728ad39668cfe722ec P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:19:37.266069 18050 raft_consensus.cc:2749] T 2794af5b99984efebdee564035841cb0 P cb82cc7bd6db46d980c75237a1402860 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:19:37.267479 18041 raft_consensus.cc:2749] T 8c11074579b046d99e585c2818e18a2c P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:19:37.267843 17844 raft_consensus.cc:3055] T 5a959dbecc2648eb85d1cc550f5a3245 P 1ca2725d290e47d68b1a4b609c9816ac [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:19:37.268247 17844 raft_consensus.cc:740] T 5a959dbecc2648eb85d1cc550f5a3245 P 1ca2725d290e47d68b1a4b609c9816ac [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:19:37.269034 17844 consensus_queue.cc:260] T 5a959dbecc2648eb85d1cc550f5a3245 P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:37.270220 17844 raft_consensus.cc:3060] T 5a959dbecc2648eb85d1cc550f5a3245 P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:19:37.270344 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e4bfb9090c8460eb14fa7e0d818e7f1" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:37.140328 17999 consensus_queue.cc:260] T b87147731a68400e8cce9b879daa7c5d P 90f3da57796b488c9087a2f523adf045 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:19:37.271627 16192 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:37.271569 17999 raft_consensus.cc:3060] T b87147731a68400e8cce9b879daa7c5d P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:19:37.276310 18013 raft_consensus.cc:515] T 9e65676b21b748728ad39668cfe722ec P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:37.274993 16265 leader_election.cc:304] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:19:37.277875 18023 raft_consensus.cc:2804] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:19:37.278419 18023 raft_consensus.cc:493] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:37.259984 16789 leader_election.cc:304] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:19:37.278812 18023 raft_consensus.cc:3060] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:19:37.279778 16789 leader_election.cc:304] T c9944d5e4b8c4109af9955c4f08c9095 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:19:37.280529 17972 raft_consensus.cc:2749] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:19:37.281028 16789 leader_election.cc:304] T 57a744f5aaf944678a63469cbf514015 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:19:37.281260 17972 raft_consensus.cc:2749] T c9944d5e4b8c4109af9955c4f08c9095 P cb82cc7bd6db46d980c75237a1402860 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:19:37.281845 18050 raft_consensus.cc:2749] T 57a744f5aaf944678a63469cbf514015 P cb82cc7bd6db46d980c75237a1402860 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:19:37.283372 17989 consensus_queue.cc:1059] T 9e65676b21b748728ad39668cfe722ec P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:19:37.284329 18071 raft_consensus.cc:3055] T 9e65676b21b748728ad39668cfe722ec P cb82cc7bd6db46d980c75237a1402860 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:19:37.265182 16115 leader_election.cc:304] T 8cc959fb7f704fba9009c0595467d074 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:19:37.284708 18071 raft_consensus.cc:740] T 9e65676b21b748728ad39668cfe722ec P cb82cc7bd6db46d980c75237a1402860 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:19:37.285652 18071 consensus_queue.cc:260] T 9e65676b21b748728ad39668cfe722ec P cb82cc7bd6db46d980c75237a1402860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:37.243898 18057 raft_consensus.cc:515] T c0eab25ce4c64e018a041beeda2867db P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:37.287259 18071 raft_consensus.cc:3060] T 9e65676b21b748728ad39668cfe722ec P cb82cc7bd6db46d980c75237a1402860 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:19:37.287719 18023 raft_consensus.cc:515] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:37.288805 18057 leader_election.cc:290] T c0eab25ce4c64e018a041beeda2867db P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 3 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:37.290064 18023 leader_election.cc:290] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 5 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:19:37.290771 16265 leader_election.cc:304] T 1df93779209748b9860f6d893dd3de53 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:19:37.287285 18055 raft_consensus.cc:2749] T 8cc959fb7f704fba9009c0595467d074 P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:19:37.292876 16789 leader_election.cc:304] T ada5a004ba7c46819051e3ccec71ffdb P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:19:37.293030 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f2da98864b4425bf7c0055951fdd4b" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:37.293900 16293 raft_consensus.cc:2393] T 84f2da98864b4425bf7c0055951fdd4b P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 2: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
I20260709 13:19:37.295940 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ad5dbfa99ee4b8d9e28be92b3f36db5" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:19:37.296792 16293 raft_consensus.cc:2393] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 3: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
I20260709 13:19:37.296471 18011 raft_consensus.cc:515] T 66f75124215f4ca6bef831e174a792d8 P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:37.278530 18013 leader_election.cc:290] T 9e65676b21b748728ad39668cfe722ec P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 4 election: Requested vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:19:37.297674 16220 raft_consensus.cc:2393] T 66f75124215f4ca6bef831e174a792d8 P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 in current term 4: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
I20260709 13:19:37.297750 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18ad7e3af5504571a69d10c00750a01c" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:19:37.298563 16300 raft_consensus.cc:2393] T 18ad7e3af5504571a69d10c00750a01c P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 3: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
I20260709 13:19:37.300388 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68a6838d82ef4cac83507f906139d90f" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:19:37.301621 17989 raft_consensus.cc:2804] T ada5a004ba7c46819051e3ccec71ffdb P cb82cc7bd6db46d980c75237a1402860 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:19:37.301966 18011 leader_election.cc:290] T 66f75124215f4ca6bef831e174a792d8 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 4 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:19:37.302243 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4f65cdf00b748bb963ee6e81ef43b79" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:37.302043 17989 raft_consensus.cc:493] T ada5a004ba7c46819051e3ccec71ffdb P cb82cc7bd6db46d980c75237a1402860 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
W20260709 13:19:37.305684 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 172511us
I20260709 13:19:37.311055 18006 leader_election.cc:290] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 5 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:19:37.308552 16292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66ae176961b74813a58bc15c9c4c6f2a" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:37.311133 16298 raft_consensus.cc:1240] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 2. Current term is 3. Ops: []
W20260709 13:19:37.307029 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 43989.2us
I20260709 13:19:37.316603 16117 leader_election.cc:304] T 434bff3082d141838068f323bdb0d1ed P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860; no voters: 90f3da57796b488c9087a2f523adf045
I20260709 13:19:37.311430 16295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e4bfb9090c8460eb14fa7e0d818e7f1" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:37.319737 16295 raft_consensus.cc:2393] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 in current term 2: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
I20260709 13:19:37.314630 16300 raft_consensus.cc:2468] T c4f65cdf00b748bb963ee6e81ef43b79 P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 5.
I20260709 13:19:37.311832 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7ca61592e848ea9b0fb81a2ce33cdc" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:37.321612 16821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66f75124215f4ca6bef831e174a792d8" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:19:37.321275 16790 leader_election.cc:304] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
W20260709 13:19:37.323936 16191 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:37.324812 16191 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:37.335371 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e65676b21b748728ad39668cfe722ec" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:19:37.336377 16298 raft_consensus.cc:2393] T 9e65676b21b748728ad39668cfe722ec P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 4: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
I20260709 13:19:37.337423 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0eab25ce4c64e018a041beeda2867db" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:19:37.332243 16297 raft_consensus.cc:2468] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 5.
I20260709 13:19:37.338292 16298 raft_consensus.cc:2393] T c0eab25ce4c64e018a041beeda2867db P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 3: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
I20260709 13:19:37.338927 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8158bbd17b994d2eb96f560bd0f1b88d" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:37.339308 16116 leader_election.cc:304] T b6f07842d3aa47858df5665874c86f3a P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:19:37.339660 16297 raft_consensus.cc:2393] T 8158bbd17b994d2eb96f560bd0f1b88d P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 4: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
I20260709 13:19:37.340132 17979 raft_consensus.cc:2749] T b6f07842d3aa47858df5665874c86f3a P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:19:37.340683 16116 leader_election.cc:304] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 90f3da57796b488c9087a2f523adf045
I20260709 13:19:37.340905 16825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a378d7f796f94615b3b347119a9880fd" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:19:37.341459 17979 raft_consensus.cc:2804] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:19:37.341845 17979 raft_consensus.cc:493] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Starting leader election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:19:37.342245 17979 raft_consensus.cc:3060] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:19:37.343839 18091 log.cc:927] Time spent T ada5a004ba7c46819051e3ccec71ffdb P cb82cc7bd6db46d980c75237a1402860: Append to log took a long time: real 0.107s	user 0.000s	sys 0.000s
I20260709 13:19:37.312083 16291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3884f864a96d428785895e5463e5728f" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:19:37.344695 16291 raft_consensus.cc:2376] T 3884f864a96d428785895e5463e5728f P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 2. Re-sending same reply.
I20260709 13:19:37.308679 17989 raft_consensus.cc:3060] T ada5a004ba7c46819051e3ccec71ffdb P cb82cc7bd6db46d980c75237a1402860 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:19:37.311544 16294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a9de347c711477e87c582d003f36925" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:37.347515 16818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e65676b21b748728ad39668cfe722ec" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:19:37.347669 16294 raft_consensus.cc:2468] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 3.
I20260709 13:19:37.343647 16819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62fe3346cd3c42c4a55e14af9073bca4" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:19:37.351433 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f5f89186204242bf377f7c42b2e2da" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:19:37.342819 16116 leader_election.cc:304] T 746aabf856ab401c9ef5e859371e7b67 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860
I20260709 13:19:37.361198 16298 raft_consensus.cc:2393] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 5: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
I20260709 13:19:37.363795 16044 catalog_manager.cc:5697] T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860 reported cstate change: term changed from 1 to 3, leader changed from 90f3da57796b488c9087a2f523adf045 (127.14.177.131) to cb82cc7bd6db46d980c75237a1402860 (127.14.177.132). New cstate: current_term: 3 leader_uuid: "cb82cc7bd6db46d980c75237a1402860" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:19:37.366866 16044 catalog_manager.cc:5697] T 8c11074579b046d99e585c2818e18a2c P cb82cc7bd6db46d980c75237a1402860 reported cstate change: term changed from 2 to 3, leader changed from cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) to 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130). New cstate: current_term: 3 leader_uuid: "1ca2725d290e47d68b1a4b609c9816ac" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:19:37.372114 16816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66ae176961b74813a58bc15c9c4c6f2a" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:19:37.332262 18041 raft_consensus.cc:2804] T 1df93779209748b9860f6d893dd3de53 P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:19:37.373809 18041 raft_consensus.cc:697] T 1df93779209748b9860f6d893dd3de53 P 90f3da57796b488c9087a2f523adf045 [term 3 LEADER]: Becoming Leader. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:19:37.377456 16299 raft_consensus.cc:1275] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:19:37.377734 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af57996a8be84d398f0f023e8d08ef77" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:19:37.350222 16294 raft_consensus.cc:1275] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:19:37.381587 16146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d16dfad0c3b540e5b4b86492ddf87208" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:19:37.313937 16292 raft_consensus.cc:2468] T 66ae176961b74813a58bc15c9c4c6f2a P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 4.
I20260709 13:19:37.383531 16790 leader_election.cc:304] T 66f75124215f4ca6bef831e174a792d8 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
W20260709 13:19:37.379186 16117 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 62670.5us
I20260709 13:19:37.391294 16146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8158bbd17b994d2eb96f560bd0f1b88d" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:19:37.392383 16822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f82224f4564fea93572e8206bad1fe" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:19:37.387976 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66ae176961b74813a58bc15c9c4c6f2a" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:37.355993 17900 raft_consensus.cc:3060] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:19:37.378690 16149 raft_consensus.cc:2393] T af57996a8be84d398f0f023e8d08ef77 P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 3: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
I20260709 13:19:37.406498 16144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66f75124215f4ca6bef831e174a792d8" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:19:37.394721 16145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4f65cdf00b748bb963ee6e81ef43b79" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:19:37.407758 16145 raft_consensus.cc:2410] T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac for term 5 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:19:37.401666 16821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdbd78acb4cf46298e0de8ba0d130e19" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
W20260709 13:19:37.402333 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 59575us
I20260709 13:19:37.403808 16146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ad5dbfa99ee4b8d9e28be92b3f36db5" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:19:37.409709 16223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ab0ac8347641e6955e9bce7d90c1a2" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:37.404634 16816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a9de347c711477e87c582d003f36925" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:19:37.404881 16219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8158bbd17b994d2eb96f560bd0f1b88d" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:19:37.416661 16821 raft_consensus.cc:3060] T 806d7690c2b644959152074bb2cab0a7 P cb82cc7bd6db46d980c75237a1402860 [term 2 LEARNER]: Advancing to term 3
I20260709 13:19:37.405064 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "829b4ad8c5a54eababeea1181b960d4b" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:19:37.405400 16222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a378d7f796f94615b3b347119a9880fd" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:37.405927 16147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62fe3346cd3c42c4a55e14af9073bca4" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:19:37.417285 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:19:37.411554 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a959dbecc2648eb85d1cc550f5a3245" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:19:37.413046 16146 raft_consensus.cc:2393] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 3: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
I20260709 13:19:37.375517 18073 raft_consensus.cc:2749] T ada5a004ba7c46819051e3ccec71ffdb P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Leader pre-election lost for term 3. Reason: Vote denied by peer cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923) with higher term. Message: Invalid argument: T ada5a004ba7c46819051e3ccec71ffdb P cb82cc7bd6db46d980c75237a1402860 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 for earlier term 3. Current term is 4.
I20260709 13:19:37.409147 16148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0eab25ce4c64e018a041beeda2867db" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:19:37.415055 16822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9944d5e4b8c4109af9955c4f08c9095" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:19:37.415650 16116 leader_election.cc:304] T 5a96373bb0e14dce8a266b2fc6c58507 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:19:37.415941 16825 raft_consensus.cc:1240] T b87147731a68400e8cce9b879daa7c5d P cb82cc7bd6db46d980c75237a1402860 [term 4 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 3. Current term is 4. Ops: []
I20260709 13:19:37.421962 17644 consensus_queue.cc:1048] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:19:37.425093 16789 leader_election.cc:304] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:19:37.381129 16220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7ca61592e848ea9b0fb81a2ce33cdc" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:37.413126 16151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f5f89186204242bf377f7c42b2e2da" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:19:37.412186 16044 catalog_manager.cc:5697] T 8638ee363273426db1393dd81e5c7b72 P cb82cc7bd6db46d980c75237a1402860 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "cb82cc7bd6db46d980c75237a1402860" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:19:37.433610 16144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:19:37.403218 17892 raft_consensus.cc:2749] T 746aabf856ab401c9ef5e859371e7b67 P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:19:37.398572 18057 raft_consensus.cc:2804] T 434bff3082d141838068f323bdb0d1ed P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:19:37.441704 16824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74ca8140f35c4ae29e8a664f6bdfc829" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:19:37.444095 16150 raft_consensus.cc:2393] T 829b4ad8c5a54eababeea1181b960d4b P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 4: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
I20260709 13:19:37.444404 16789 leader_election.cc:304] T 3a7ca61592e848ea9b0fb81a2ce33cdc P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 
W20260709 13:19:37.448289 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 55804.2us
I20260709 13:19:37.445849 16823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e65676b21b748728ad39668cfe722ec" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:19:37.449630 16789 leader_election.cc:304] T 3884f864a96d428785895e5463e5728f P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 647a343a542d4a139fe106c19095129b
I20260709 13:19:37.416847 18007 raft_consensus.cc:493] T ee9ed67006144797b1de6ea8c2013a68 P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:19:37.454383 16822 raft_consensus.cc:2393] T c9944d5e4b8c4109af9955c4f08c9095 P cb82cc7bd6db46d980c75237a1402860 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 4: Already voted for candidate cb82cc7bd6db46d980c75237a1402860 in this term.
I20260709 13:19:37.435943 16044 catalog_manager.cc:5697] T 19baa7bc367844c288bca8f81439c413 P cb82cc7bd6db46d980c75237a1402860 reported cstate change: term changed from 2 to 3, leader changed from 90f3da57796b488c9087a2f523adf045 (127.14.177.131) to 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130). New cstate: current_term: 3 leader_uuid: "1ca2725d290e47d68b1a4b609c9816ac" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } } }
I20260709 13:19:37.393630 17979 raft_consensus.cc:515] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:37.454497 17914 raft_consensus.cc:3060] T 66f75124215f4ca6bef831e174a792d8 P cb82cc7bd6db46d980c75237a1402860 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:19:37.454388 17918 consensus_queue.cc:1059] T 7c2095a5f4ad49408dbc7f69c606120d P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.003s
I20260709 13:19:37.458184 16191 leader_election.cc:304] T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:19:37.447638 18034 raft_consensus.cc:3060] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P cb82cc7bd6db46d980c75237a1402860 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:19:37.456130 16190 leader_election.cc:304] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860
W20260709 13:19:37.467319 16115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 81503.7us
W20260709 13:19:37.469415 16117 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 49277.4us
I20260709 13:19:37.457305 16218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ad5dbfa99ee4b8d9e28be92b3f36db5" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:19:37.470553 18096 raft_consensus.cc:493] T e4dc6b0541ca431ebc757beac00521f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:19:37.470733 18093 raft_consensus.cc:3055] T 8c11074579b046d99e585c2818e18a2c P 1ca2725d290e47d68b1a4b609c9816ac [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:19:37.472566 17931 consensus_queue.cc:1048] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:19:37.473183 18044 raft_consensus.cc:2804] T 3a7ca61592e848ea9b0fb81a2ce33cdc P cb82cc7bd6db46d980c75237a1402860 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:19:37.473245 17979 leader_election.cc:290] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 4 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:37.457365 17982 raft_consensus.cc:2749] T 5a96373bb0e14dce8a266b2fc6c58507 P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:19:37.441705 16143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f5f89186204242bf377f7c42b2e2da" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:19:37.459435 16297 raft_consensus.cc:1275] T ada5a004ba7c46819051e3ccec71ffdb P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:19:37.475708 16151 raft_consensus.cc:2468] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 4.
I20260709 13:19:37.455315 18061 raft_consensus.cc:493] T 02838c86531d4f98bf58d1c22b8baf77 P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:37.456480 18057 raft_consensus.cc:697] T 434bff3082d141838068f323bdb0d1ed P 647a343a542d4a139fe106c19095129b [term 4 LEADER]: Becoming Leader. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
W20260709 13:19:37.497713 16789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 48158.4us
I20260709 13:19:37.481419 16115 leader_election.cc:304] T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:19:37.502027 16789 leader_election.cc:304] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 647a343a542d4a139fe106c19095129b
I20260709 13:19:37.499334 16116 leader_election.cc:304] T af57996a8be84d398f0f023e8d08ef77 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860
I20260709 13:19:37.484737 16117 leader_election.cc:304] T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860; no voters: 
W20260709 13:19:37.506589 16114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:37.507746 16821 raft_consensus.cc:1275] T 806d7690c2b644959152074bb2cab0a7 P cb82cc7bd6db46d980c75237a1402860 [term 3 LEARNER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 3 index: 3. (term mismatch)
I20260709 13:19:37.484043 16044 catalog_manager.cc:5697] T d2ab0ac8347641e6955e9bce7d90c1a2 P cb82cc7bd6db46d980c75237a1402860 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "cb82cc7bd6db46d980c75237a1402860" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:19:37.526697 16117 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 47201.1us
I20260709 13:19:37.523677 17982 raft_consensus.cc:2804] T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:19:37.517149 17644 consensus_queue.cc:1048] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.014s
I20260709 13:19:37.488756 18052 raft_consensus.cc:2804] T 3884f864a96d428785895e5463e5728f P cb82cc7bd6db46d980c75237a1402860 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:19:37.529808 17981 raft_consensus.cc:2804] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:19:37.483376 16221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdbd78acb4cf46298e0de8ba0d130e19" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:37.495523 18011 raft_consensus.cc:2749] T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
W20260709 13:19:37.546002 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 63231.8us
I20260709 13:19:37.489545 18044 raft_consensus.cc:493] T 3a7ca61592e848ea9b0fb81a2ce33cdc P cb82cc7bd6db46d980c75237a1402860 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:37.474642 18007 raft_consensus.cc:515] T ee9ed67006144797b1de6ea8c2013a68 P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:37.488512 18093 raft_consensus.cc:740] T 8c11074579b046d99e585c2818e18a2c P 1ca2725d290e47d68b1a4b609c9816ac [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:19:37.436787 18041 consensus_queue.cc:237] T 1df93779209748b9860f6d893dd3de53 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:37.483732 16291 raft_consensus.cc:1240] T ecc1e363230043fe9fef390f0b979f20 P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 2. Current term is 3. Ops: [2.4-2.4]
I20260709 13:19:37.509085 18035 raft_consensus.cc:2749] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:19:37.485816 17918 raft_consensus.cc:3055] T 7c2095a5f4ad49408dbc7f69c606120d P 1ca2725d290e47d68b1a4b609c9816ac [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:19:37.565251 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9944d5e4b8c4109af9955c4f08c9095" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:37.552446 16821 pending_rounds.cc:85] T 806d7690c2b644959152074bb2cab0a7 P cb82cc7bd6db46d980c75237a1402860: Aborting all ops after (but not including) 2
I20260709 13:19:37.579737 16116 leader_election.cc:304] T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860
I20260709 13:19:37.579949 16220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f5f89186204242bf377f7c42b2e2da" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:19:37.608659 16225 raft_consensus.cc:2468] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 4.
I20260709 13:19:37.609622 17644 raft_consensus.cc:2749] T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:19:37.525800 18039 consensus_queue.cc:1059] T c0eab25ce4c64e018a041beeda2867db P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.029s
I20260709 13:19:37.609866 17918 raft_consensus.cc:740] T 7c2095a5f4ad49408dbc7f69c606120d P 1ca2725d290e47d68b1a4b609c9816ac [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:19:37.614295 18104 raft_consensus.cc:2804] T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:19:37.532645 17919 consensus_queue.cc:1059] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.046s
I20260709 13:19:37.553398 17982 raft_consensus.cc:493] T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:37.614506 17918 consensus_queue.cc:260] T 7c2095a5f4ad49408dbc7f69c606120d P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:37.615322 17982 raft_consensus.cc:3060] T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:19:37.615703 18057 consensus_queue.cc:237] T 434bff3082d141838068f323bdb0d1ed P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:37.553857 17892 raft_consensus.cc:2749] T af57996a8be84d398f0f023e8d08ef77 P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:19:37.618202 18093 consensus_queue.cc:260] T 8c11074579b046d99e585c2818e18a2c P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:37.525398 18061 raft_consensus.cc:515] T 02838c86531d4f98bf58d1c22b8baf77 P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:37.592095 16222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 5 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:37.595682 16223 raft_consensus.cc:1275] T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:19:37.573207 17981 raft_consensus.cc:493] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:19:37.581457 18044 raft_consensus.cc:3060] T 3a7ca61592e848ea9b0fb81a2ce33cdc P cb82cc7bd6db46d980c75237a1402860 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:19:37.526448 18096 raft_consensus.cc:515] T e4dc6b0541ca431ebc757beac00521f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:37.614933 18104 raft_consensus.cc:697] T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b [term 3 LEADER]: Becoming Leader. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:19:37.613814 18011 consensus_queue.cc:1059] T 746aabf856ab401c9ef5e859371e7b67 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
W20260709 13:19:37.580463 16115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 55258.6us
W20260709 13:19:37.629351 16264 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:37.634766 16222 raft_consensus.cc:2410] T 19baa7bc367844c288bca8f81439c413 P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 for term 5 because replica has last-logged OpId of term: 3 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 3.
I20260709 13:19:37.642482 16223 raft_consensus.cc:1240] T e5ad4eb281564936a59f41e387a47156 P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 3. Current term is 4. Ops: []
I20260709 13:19:37.615723 17918 raft_consensus.cc:3060] T 7c2095a5f4ad49408dbc7f69c606120d P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:19:37.608995 16821 pending_rounds.cc:107] T 806d7690c2b644959152074bb2cab0a7 P cb82cc7bd6db46d980c75237a1402860: Aborting uncommitted NO_OP operation due to leader change: 2.3
I20260709 13:19:37.654618 16821 raft_consensus.cc:2889] T 806d7690c2b644959152074bb2cab0a7 P cb82cc7bd6db46d980c75237a1402860 [term 3 LEARNER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:19:37.661043 16218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74ca8140f35c4ae29e8a664f6bdfc829" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:19:37.665951 16142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66f75124215f4ca6bef831e174a792d8" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:19:37.670207 16789 leader_election.cc:304] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac
I20260709 13:19:37.624550 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0eab25ce4c64e018a041beeda2867db" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:19:37.554765 16044 catalog_manager.cc:5697] T 806d7690c2b644959152074bb2cab0a7 P cb82cc7bd6db46d980c75237a1402860 reported cstate change: term changed from 1 to 2, leader changed from 647a343a542d4a139fe106c19095129b (127.14.177.129) to 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130). New cstate: current_term: 2 leader_uuid: "1ca2725d290e47d68b1a4b609c9816ac" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } } }
W20260709 13:19:37.689369 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 109760us
I20260709 13:19:37.635293 17981 raft_consensus.cc:3060] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:19:37.691687 16116 leader_election.cc:304] T 08c74490c40243a88e62d1760b5cd814 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:19:37.688165 16265 leader_election.cc:304] T 8d4ab870a05347868956aa8b841b2981 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:19:37.698246 16044 catalog_manager.cc:5697] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "cb82cc7bd6db46d980c75237a1402860" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:19:37.708127 16144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 5 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:19:37.709668 16151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e65676b21b748728ad39668cfe722ec" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:19:37.711263 16148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f5f89186204242bf377f7c42b2e2da" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:19:37.712136 16148 raft_consensus.cc:3060] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:19:37.672144 16220 raft_consensus.cc:2393] T c2f5f89186204242bf377f7c42b2e2da P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 5: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
I20260709 13:19:37.696239 16117 leader_election.cc:304] T 9e65676b21b748728ad39668cfe722ec P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860
I20260709 13:19:37.719027 17644 raft_consensus.cc:2749] T 9e65676b21b748728ad39668cfe722ec P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:19:37.720639 16147 raft_consensus.cc:1240] T e5ad4eb281564936a59f41e387a47156 P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 3. Current term is 4. Ops: []
W20260709 13:19:37.716539 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 188433us
I20260709 13:19:37.725044 16222 raft_consensus.cc:1240] T 68a6838d82ef4cac83507f906139d90f P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 13:19:37.701349 18052 raft_consensus.cc:697] T 3884f864a96d428785895e5463e5728f P cb82cc7bd6db46d980c75237a1402860 [term 2 LEADER]: Becoming Leader. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:19:37.731667 18052 consensus_queue.cc:237] T 3884f864a96d428785895e5463e5728f P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:37.723913 16044 catalog_manager.cc:5697] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 reported cstate change: term changed from 2 to 4, leader changed from cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) to 647a343a542d4a139fe106c19095129b (127.14.177.129). New cstate: current_term: 4 leader_uuid: "647a343a542d4a139fe106c19095129b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:19:37.702091 18039 raft_consensus.cc:3055] T c0eab25ce4c64e018a041beeda2867db P 1ca2725d290e47d68b1a4b609c9816ac [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:19:37.738583 16219 raft_consensus.cc:2468] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 4.
I20260709 13:19:37.739293 18073 raft_consensus.cc:2749] T 8d4ab870a05347868956aa8b841b2981 P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
W20260709 13:19:37.742404 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 243575us
I20260709 13:19:37.742519 18096 leader_election.cc:290] T e4dc6b0541ca431ebc757beac00521f1 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:37.751453 18117 raft_consensus.cc:493] T b87147731a68400e8cce9b879daa7c5d P cb82cc7bd6db46d980c75237a1402860 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:37.755246 18104 consensus_queue.cc:237] T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:37.739995 18097 raft_consensus.cc:3055] T 746aabf856ab401c9ef5e859371e7b67 P 1ca2725d290e47d68b1a4b609c9816ac [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:19:37.750312 16142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a959dbecc2648eb85d1cc550f5a3245" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:19:37.725423 17892 raft_consensus.cc:2749] T 08c74490c40243a88e62d1760b5cd814 P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:19:37.697278 18013 consensus_queue.cc:1048] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.044s
I20260709 13:19:37.764580 16821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee9ed67006144797b1de6ea8c2013a68" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:19:37.762961 16224 raft_consensus.cc:2468] T 18ad7e3af5504571a69d10c00750a01c P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 3.
I20260709 13:19:37.768283 16116 leader_election.cc:304] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860
I20260709 13:19:37.741106 18039 raft_consensus.cc:740] T c0eab25ce4c64e018a041beeda2867db P 1ca2725d290e47d68b1a4b609c9816ac [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
W20260709 13:19:37.761710 18102 log.cc:927] Time spent T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860: Append to log took a long time: real 0.207s	user 0.000s	sys 0.001s
I20260709 13:19:37.747871 18089 raft_consensus.cc:493] T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:37.780081 16148 raft_consensus.cc:2468] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 5.
I20260709 13:19:37.771683 16191 leader_election.cc:304] T 68a6838d82ef4cac83507f906139d90f P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:19:37.762750 18007 leader_election.cc:290] T ee9ed67006144797b1de6ea8c2013a68 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
W20260709 13:19:37.817548 16192 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:37.818924 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 47312.1us
I20260709 13:19:37.766413 18097 raft_consensus.cc:740] T 746aabf856ab401c9ef5e859371e7b67 P 1ca2725d290e47d68b1a4b609c9816ac [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
W20260709 13:19:37.827989 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 59796.1us
I20260709 13:19:37.828663 16116 leader_election.cc:304] T 829b4ad8c5a54eababeea1181b960d4b P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860
W20260709 13:19:37.840147 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 152413us
I20260709 13:19:37.840950 16298 raft_consensus.cc:1275] T 8a878999c0024d2abe517484180290fc P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:19:37.816092 17979 raft_consensus.cc:2749] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:19:37.826351 17844 raft_consensus.cc:2749] T 68a6838d82ef4cac83507f906139d90f P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:19:37.846822 18061 leader_election.cc:290] T 02838c86531d4f98bf58d1c22b8baf77 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:19:37.839984 17931 consensus_queue.cc:1059] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.009s
I20260709 13:19:37.812137 18096 consensus_queue.cc:1059] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: INVALID_TERM, Last received: 1.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.055s
I20260709 13:19:37.855993 17644 raft_consensus.cc:2749] T 829b4ad8c5a54eababeea1181b960d4b P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:19:37.858319 17992 raft_consensus.cc:2804] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:19:37.767181 18117 raft_consensus.cc:515] T b87147731a68400e8cce9b879daa7c5d P cb82cc7bd6db46d980c75237a1402860 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:37.870067 17844 consensus_queue.cc:1059] T ecc1e363230043fe9fef390f0b979f20 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true }, Status: INVALID_TERM, Last received: 2.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:19:37.870551 18117 leader_election.cc:290] T b87147731a68400e8cce9b879daa7c5d P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:37.871323 17844 raft_consensus.cc:3055] T ecc1e363230043fe9fef390f0b979f20 P 1ca2725d290e47d68b1a4b609c9816ac [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:19:37.871748 17844 raft_consensus.cc:740] T ecc1e363230043fe9fef390f0b979f20 P 1ca2725d290e47d68b1a4b609c9816ac [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:19:37.871474 18039 consensus_queue.cc:260] T c0eab25ce4c64e018a041beeda2867db P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:37.867098 18057 consensus_queue.cc:1048] T 8a878999c0024d2abe517484180290fc P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.005s
I20260709 13:19:37.874307 16148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87147731a68400e8cce9b879daa7c5d" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
W20260709 13:19:37.874511 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 45916.7us
I20260709 13:19:37.875514 16116 leader_election.cc:304] T 84f2da98864b4425bf7c0055951fdd4b P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:19:37.815769 18089 raft_consensus.cc:515] T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:37.880187 16299 raft_consensus.cc:1240] T 8c11074579b046d99e585c2818e18a2c P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 3. Current term is 4. Ops: []
I20260709 13:19:37.881827 16116 leader_election.cc:304] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:19:37.882870 17644 raft_consensus.cc:3060] T 84f2da98864b4425bf7c0055951fdd4b P 647a343a542d4a139fe106c19095129b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:19:37.884109 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87147731a68400e8cce9b879daa7c5d" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:37.872717 17844 consensus_queue.cc:260] T ecc1e363230043fe9fef390f0b979f20 P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.4, Last appended by leader: 4, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:37.885597 17844 raft_consensus.cc:3060] T ecc1e363230043fe9fef390f0b979f20 P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:19:37.891505 16191 leader_election.cc:304] T 1df93779209748b9860f6d893dd3de53 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 90f3da57796b488c9087a2f523adf045
I20260709 13:19:37.893443 17992 raft_consensus.cc:493] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:19:37.895957 16191 leader_election.cc:304] T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 90f3da57796b488c9087a2f523adf045
I20260709 13:19:37.896461 16265 leader_election.cc:304] T fffab26427e24d398e2695aad40ca134 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac
I20260709 13:19:37.895331 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4dc6b0541ca431ebc757beac00521f1" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:37.872844 18039 raft_consensus.cc:3060] T c0eab25ce4c64e018a041beeda2867db P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:19:37.881335 17999 consensus_meta.cc:287] T b87147731a68400e8cce9b879daa7c5d P 90f3da57796b488c9087a2f523adf045: Time spent flushing consensus metadata: real 0.606s	user 0.013s	sys 0.008s
I20260709 13:19:37.824298 18108 consensus_queue.cc:1059] T 5a96373bb0e14dce8a266b2fc6c58507 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.050s
I20260709 13:19:37.907617 17919 raft_consensus.cc:2764] T 1df93779209748b9860f6d893dd3de53 P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Leader pre-election decision vote started in defunct term 2: won
I20260709 13:19:37.907990 16818 raft_consensus.cc:2468] T 9e65676b21b748728ad39668cfe722ec P cb82cc7bd6db46d980c75237a1402860 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 4.
I20260709 13:19:37.893967 17992 raft_consensus.cc:3060] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:19:37.891378 17979 raft_consensus.cc:493] T 3884f864a96d428785895e5463e5728f P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:37.932231 17979 raft_consensus.cc:515] T 3884f864a96d428785895e5463e5728f P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:37.932981 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a9de347c711477e87c582d003f36925" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:19:37.934190 17999 raft_consensus.cc:2804] T fffab26427e24d398e2695aad40ca134 P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:19:37.934609 17999 raft_consensus.cc:493] T fffab26427e24d398e2695aad40ca134 P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:19:37.864213 18035 consensus_queue.cc:1048] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.037s
I20260709 13:19:37.917376 16300 raft_consensus.cc:2468] T b87147731a68400e8cce9b879daa7c5d P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 4.
I20260709 13:19:37.939390 16293 raft_consensus.cc:1275] T 02838c86531d4f98bf58d1c22b8baf77 P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:19:37.929519 18007 raft_consensus.cc:2804] T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:19:37.942735 18007 raft_consensus.cc:493] T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:37.943142 18007 raft_consensus.cc:3060] T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:19:37.897670 18006 consensus_queue.cc:1059] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.054s
I20260709 13:19:37.934952 17999 raft_consensus.cc:3060] T fffab26427e24d398e2695aad40ca134 P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:19:37.952576 18089 leader_election.cc:290] T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
W20260709 13:19:37.914824 17989 consensus_meta.cc:287] T ada5a004ba7c46819051e3ccec71ffdb P cb82cc7bd6db46d980c75237a1402860: Time spent flushing consensus metadata: real 0.567s	user 0.007s	sys 0.003s
I20260709 13:19:37.953591 17989 raft_consensus.cc:515] T ada5a004ba7c46819051e3ccec71ffdb P cb82cc7bd6db46d980c75237a1402860 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:37.879944 17981 raft_consensus.cc:515] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:19:37.937023 18112 log.cc:927] Time spent T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 0.230s	user 0.001s	sys 0.000s
I20260709 13:19:37.902522 18057 raft_consensus.cc:2749] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:19:37.956023 17989 leader_election.cc:290] T ada5a004ba7c46819051e3ccec71ffdb P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 5 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:37.957016 17981 leader_election.cc:290] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 4 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:37.917382 18097 consensus_queue.cc:260] T 746aabf856ab401c9ef5e859371e7b67 P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:37.959120 18034 raft_consensus.cc:2749] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P cb82cc7bd6db46d980c75237a1402860 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:19:37.959357 17999 raft_consensus.cc:515] T fffab26427e24d398e2695aad40ca134 P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:37.960279 16822 raft_consensus.cc:1240] T ada5a004ba7c46819051e3ccec71ffdb P cb82cc7bd6db46d980c75237a1402860 [term 5 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 4. Current term is 5. Ops: []
I20260709 13:19:37.961401 17999 leader_election.cc:290] T fffab26427e24d398e2695aad40ca134 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 2 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:19:37.962380 17914 raft_consensus.cc:2749] T 66f75124215f4ca6bef831e174a792d8 P cb82cc7bd6db46d980c75237a1402860 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:19:37.971444 16299 raft_consensus.cc:1240] T fffab26427e24d398e2695aad40ca134 P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 1. Current term is 2. Ops: []
I20260709 13:19:37.977790 17982 raft_consensus.cc:515] T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:37.990566 17982 leader_election.cc:290] T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 6 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
W20260709 13:19:37.980034 16820 consensus_meta.cc:287] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860: Time spent flushing consensus metadata: real 0.557s	user 0.013s	sys 0.004s
I20260709 13:19:38.000172 16820 raft_consensus.cc:2468] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 3.
I20260709 13:19:37.981534 18044 raft_consensus.cc:515] T 3a7ca61592e848ea9b0fb81a2ce33cdc P cb82cc7bd6db46d980c75237a1402860 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:37.931567 16265 leader_election.cc:400] T 8a878999c0024d2abe517484180290fc P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 2 election: Vote denied by peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741) with higher term. Message: Invalid argument: T 8a878999c0024d2abe517484180290fc P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 for earlier term 2. Current term is 3.
I20260709 13:19:37.983289 18042 consensus_queue.cc:1059] T 5a96373bb0e14dce8a266b2fc6c58507 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:19:37.988238 16226 raft_consensus.cc:2468] T 5a959dbecc2648eb85d1cc550f5a3245 P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 2.
I20260709 13:19:37.986881 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada5a004ba7c46819051e3ccec71ffdb" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:19:37.988358 18057 consensus_queue.cc:1059] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:19:38.012775 18057 raft_consensus.cc:3055] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:19:38.012969 16816 raft_consensus.cc:1275] T 26354361e9b044cea1c2b86c4882fe92 P cb82cc7bd6db46d980c75237a1402860 [term 2 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:19:38.013242 18057 raft_consensus.cc:740] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:19:38.014164 18057 consensus_queue.cc:260] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:37.988490 16116 leader_election.cc:304] T 68a6838d82ef4cac83507f906139d90f P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 90f3da57796b488c9087a2f523adf045
I20260709 13:19:37.991302 18013 consensus_queue.cc:1059] T d16dfad0c3b540e5b4b86492ddf87208 P 647a343a542d4a139fe106c19095129b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:19:37.979310 18108 raft_consensus.cc:3055] T 5a96373bb0e14dce8a266b2fc6c58507 P 1ca2725d290e47d68b1a4b609c9816ac [term 1 LEADER]: Stepping down as leader of term 1
W20260709 13:19:38.002972 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 71519.2us
I20260709 13:19:38.003919 17644 raft_consensus.cc:2749] T 84f2da98864b4425bf7c0055951fdd4b P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:19:38.005746 16298 raft_consensus.cc:3060] T ada5a004ba7c46819051e3ccec71ffdb P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:19:38.015792 16190 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:38.022415 18061 consensus_queue.cc:1048] T 26354361e9b044cea1c2b86c4882fe92 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:19:38.025089 16116 leader_election.cc:304] T e5ad4eb281564936a59f41e387a47156 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 90f3da57796b488c9087a2f523adf045
I20260709 13:19:38.023021 18108 raft_consensus.cc:740] T 5a96373bb0e14dce8a266b2fc6c58507 P 1ca2725d290e47d68b1a4b609c9816ac [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:19:38.022675 18013 raft_consensus.cc:3055] T d16dfad0c3b540e5b4b86492ddf87208 P 647a343a542d4a139fe106c19095129b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:19:38.023890 18055 raft_consensus.cc:2804] T 68a6838d82ef4cac83507f906139d90f P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:19:38.027644 18013 raft_consensus.cc:740] T d16dfad0c3b540e5b4b86492ddf87208 P 647a343a542d4a139fe106c19095129b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:19:38.024307 16265 leader_election.cc:304] T c4f65cdf00b748bb963ee6e81ef43b79 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
W20260709 13:19:38.015980 17900 consensus_meta.cc:287] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860: Time spent flushing consensus metadata: real 0.590s	user 0.015s	sys 0.007s
I20260709 13:19:38.030524 16265 leader_election.cc:304] T 31969392ccdb43b491911a319e8bc7e1 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
W20260709 13:19:38.034013 16190 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:38.043890 17644 raft_consensus.cc:2804] T e5ad4eb281564936a59f41e387a47156 P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Leader election won for term 4
W20260709 13:19:38.044023 16190 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:38.031359 17999 raft_consensus.cc:2749] T c4f65cdf00b748bb963ee6e81ef43b79 P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
W20260709 13:19:38.081866 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 51414.4us
I20260709 13:19:38.082998 16265 leader_election.cc:304] T 434bff3082d141838068f323bdb0d1ed P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
I20260709 13:19:37.997188 18007 raft_consensus.cc:515] T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:38.078826 17644 raft_consensus.cc:697] T e5ad4eb281564936a59f41e387a47156 P 647a343a542d4a139fe106c19095129b [term 4 LEADER]: Becoming Leader. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
W20260709 13:19:38.077353 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 52339.6us
I20260709 13:19:38.028882 18055 raft_consensus.cc:697] T 68a6838d82ef4cac83507f906139d90f P 647a343a542d4a139fe106c19095129b [term 5 LEADER]: Becoming Leader. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:19:38.031031 17900 raft_consensus.cc:2749] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:19:38.100391 17644 consensus_queue.cc:237] T e5ad4eb281564936a59f41e387a47156 P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:38.100945 16225 raft_consensus.cc:1275] T 19baa7bc367844c288bca8f81439c413 P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 4 index: 4. (term mismatch)
I20260709 13:19:38.101475 16225 pending_rounds.cc:85] T 19baa7bc367844c288bca8f81439c413 P 1ca2725d290e47d68b1a4b609c9816ac: Aborting all ops after (but not including) 3
I20260709 13:19:38.101752 16225 pending_rounds.cc:107] T 19baa7bc367844c288bca8f81439c413 P 1ca2725d290e47d68b1a4b609c9816ac: Aborting uncommitted NO_OP operation due to leader change: 3.4
I20260709 13:19:38.102044 16225 raft_consensus.cc:2889] T 19baa7bc367844c288bca8f81439c413 P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:19:38.103705 16221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f5f89186204242bf377f7c42b2e2da" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:19:38.104597 16221 raft_consensus.cc:2368] T c2f5f89186204242bf377f7c42b2e2da P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b for earlier term 4. Current term is 5.
I20260709 13:19:38.105190 18044 leader_election.cc:290] T 3a7ca61592e848ea9b0fb81a2ce33cdc P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 6 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:38.107550 16222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0eab25ce4c64e018a041beeda2867db" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:19:38.110417 17844 consensus_queue.cc:1048] T 02838c86531d4f98bf58d1c22b8baf77 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:19:38.084249 18057 raft_consensus.cc:3060] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:19:38.114038 16116 leader_election.cc:400] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 4 election: Vote denied by peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943) with higher term. Message: Invalid argument: T c2f5f89186204242bf377f7c42b2e2da P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b for earlier term 4. Current term is 5.
I20260709 13:19:38.115506 16116 leader_election.cc:403] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 4 election: Cancelling election due to peer responding with higher term
W20260709 13:19:38.115622 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 219757us
I20260709 13:19:38.086874 18041 raft_consensus.cc:2749] T 31969392ccdb43b491911a319e8bc7e1 P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:19:38.116714 16191 leader_election.cc:304] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
I20260709 13:19:38.121488 16191 leader_election.cc:304] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
I20260709 13:19:38.030943 18108 consensus_queue.cc:260] T 5a96373bb0e14dce8a266b2fc6c58507 P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:38.130502 16265 leader_election.cc:304] T 18ad7e3af5504571a69d10c00750a01c P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:19:38.096624 18123 raft_consensus.cc:2749] T 434bff3082d141838068f323bdb0d1ed P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:19:38.111392 16294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a9de347c711477e87c582d003f36925" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:19:38.159001 16821 raft_consensus.cc:1240] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860 [term 4 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 3. Current term is 4. Ops: []
I20260709 13:19:38.097056 18063 consensus_queue.cc:1059] T 8c11074579b046d99e585c2818e18a2c P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.036s
I20260709 13:19:38.128595 18035 raft_consensus.cc:3060] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:19:38.173717 16265 leader_election.cc:304] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
W20260709 13:19:38.174329 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 127485us
I20260709 13:19:38.174437 18061 raft_consensus.cc:2749] T 18ad7e3af5504571a69d10c00750a01c P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:19:38.127447 18013 consensus_queue.cc:260] T d16dfad0c3b540e5b4b86492ddf87208 P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:38.179663 16191 leader_election.cc:304] T e5ad4eb281564936a59f41e387a47156 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:19:38.148442 18104 raft_consensus.cc:2749] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Leader election lost for term 4. Reason: Vote denied by peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943) with higher term. Message: Invalid argument: T c2f5f89186204242bf377f7c42b2e2da P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b for earlier term 4. Current term is 5.
I20260709 13:19:38.116905 18089 consensus_queue.cc:1048] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.037s
I20260709 13:19:38.183142 16825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31969392ccdb43b491911a319e8bc7e1" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:19:38.187069 16825 raft_consensus.cc:2468] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 2.
I20260709 13:19:38.182631 16265 leader_election.cc:304] T 57a744f5aaf944678a63469cbf514015 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
W20260709 13:19:38.189203 16189 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:38.191531 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 77655.7us
I20260709 13:19:38.192584 16265 leader_election.cc:304] T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac
W20260709 13:19:38.183241 18131 log.cc:927] Time spent T b87147731a68400e8cce9b879daa7c5d P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 0.098s	user 0.001s	sys 0.000s
I20260709 13:19:38.183301 16816 raft_consensus.cc:1240] T 806d7690c2b644959152074bb2cab0a7 P cb82cc7bd6db46d980c75237a1402860 [term 3 LEARNER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 2. Current term is 3. Ops: []
I20260709 13:19:38.200323 16191 leader_election.cc:304] T d16dfad0c3b540e5b4b86492ddf87208 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:19:38.185973 18096 raft_consensus.cc:2749] T e5ad4eb281564936a59f41e387a47156 P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:19:38.186120 18123 raft_consensus.cc:2749] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:19:38.180457 18133 raft_consensus.cc:2749] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:19:38.192723 18108 raft_consensus.cc:3060] T 5a96373bb0e14dce8a266b2fc6c58507 P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:19:38.225028 18129 log.cc:927] Time spent T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.052s	user 0.001s	sys 0.000s
W20260709 13:19:38.225970 16264 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:38.208235 16143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f82224f4564fea93572e8206bad1fe" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:19:38.190284 16221 raft_consensus.cc:1275] T 8a878999c0024d2abe517484180290fc P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:19:38.181556 17992 raft_consensus.cc:515] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:38.228863 16819 raft_consensus.cc:1240] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [term 5 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 4. Current term is 5. Ops: []
I20260709 13:19:38.229485 18007 leader_election.cc:290] T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 4 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:38.229882 17992 leader_election.cc:290] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 5 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:38.185664 18039 consensus_queue.cc:1048] T 02838c86531d4f98bf58d1c22b8baf77 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.022s
I20260709 13:19:38.231752 18109 raft_consensus.cc:3055] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:19:38.232229 18109 raft_consensus.cc:740] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:19:38.208607 18041 raft_consensus.cc:2804] T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:19:38.232882 18041 raft_consensus.cc:493] T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:19:38.233284 18041 raft_consensus.cc:3060] T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:19:38.233084 18109 consensus_queue.cc:260] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:38.234202 18109 raft_consensus.cc:3060] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:19:38.239336 18134 consensus_queue.cc:1059] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: INVALID_TERM, Last received: 1.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.033s
I20260709 13:19:38.240105 18134 consensus_queue.cc:1243] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Peer 647a343a542d4a139fe106c19095129b log is divergent from this leader: its last log entry 3.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260709 13:19:38.210750 18138 raft_consensus.cc:2749] T d16dfad0c3b540e5b4b86492ddf87208 P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:19:38.218397 16825 raft_consensus.cc:1240] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P cb82cc7bd6db46d980c75237a1402860 [term 2 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 1. Current term is 2. Ops: []
I20260709 13:19:38.220067 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4f65cdf00b748bb963ee6e81ef43b79" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:19:38.218883 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7ca61592e848ea9b0fb81a2ce33cdc" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:19:38.244892 16225 raft_consensus.cc:3060] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:19:38.183755 18055 consensus_queue.cc:237] T 68a6838d82ef4cac83507f906139d90f P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:38.197125 18061 raft_consensus.cc:2749] T 57a744f5aaf944678a63469cbf514015 P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:19:38.244990 16299 raft_consensus.cc:3060] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:19:38.231670 16294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66ae176961b74813a58bc15c9c4c6f2a" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
W20260709 13:19:38.249691 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 57247.3us
I20260709 13:19:38.212311 16292 raft_consensus.cc:1240] T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 2. Current term is 3. Ops: []
I20260709 13:19:38.247089 16222 raft_consensus.cc:2468] T c0eab25ce4c64e018a041beeda2867db P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 3.
W20260709 13:19:38.247702 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 47468.2us
I20260709 13:19:38.250324 16265 leader_election.cc:304] T 84f2da98864b4425bf7c0055951fdd4b P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:19:38.252349 16191 leader_election.cc:304] T 66f75124215f4ca6bef831e174a792d8 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
W20260709 13:19:38.254140 16189 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:38.254796 16221 raft_consensus.cc:1240] T 66f75124215f4ca6bef831e174a792d8 P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 3. Current term is 4. Ops: []
I20260709 13:19:38.256470 16292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c61a39fd4f84424b129c1a6f9ba404d" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:19:38.256801 16291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f5f89186204242bf377f7c42b2e2da" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:19:38.257718 16291 raft_consensus.cc:2368] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b for earlier term 4. Current term is 5.
I20260709 13:19:38.259928 16224 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57604 (request call id 1904) took 1094 ms. Trace:
I20260709 13:19:38.248890 17931 consensus_queue.cc:1059] T fffab26427e24d398e2695aad40ca134 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.013s
I20260709 13:19:38.262684 17931 raft_consensus.cc:3055] T fffab26427e24d398e2695aad40ca134 P 1ca2725d290e47d68b1a4b609c9816ac [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:19:38.263139 17931 raft_consensus.cc:740] T fffab26427e24d398e2695aad40ca134 P 1ca2725d290e47d68b1a4b609c9816ac [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:19:38.264515 16219 raft_consensus.cc:3055] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:19:38.264917 16219 raft_consensus.cc:740] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:19:38.260329 16224 rpcz_store.cc:276] 0709 13:19:37.165494 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:37.165912 (+   418us) service_pool.cc:224] Handling call
0709 13:19:38.259763 (+1093851us) spinlock_profiling.cc:243] Waited 331 ms on lock 0x7b5400367254. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:19:38.259863 (+   100us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":761796864}
I20260709 13:19:38.200881 18063 consensus_queue.cc:1059] T 08c74490c40243a88e62d1760b5cd814 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.052s
I20260709 13:19:38.272707 17644 raft_consensus.cc:493] T ecc1e363230043fe9fef390f0b979f20 P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:19:38.273412 16265 leader_election.cc:304] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:19:38.273591 16116 leader_election.cc:304] T 18ad7e3af5504571a69d10c00750a01c P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 90f3da57796b488c9087a2f523adf045
I20260709 13:19:38.278924 16225 raft_consensus.cc:2468] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 6.
I20260709 13:19:38.281417 16191 leader_election.cc:304] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:19:38.282856 16296 raft_consensus.cc:1240] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 1. Current term is 2. Ops: []
I20260709 13:19:38.264612 17931 consensus_queue.cc:260] T fffab26427e24d398e2695aad40ca134 P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:38.287719 17644 raft_consensus.cc:515] T ecc1e363230043fe9fef390f0b979f20 P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } }
I20260709 13:19:38.271462 17999 raft_consensus.cc:2749] T 84f2da98864b4425bf7c0055951fdd4b P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:19:38.252599 18089 consensus_queue.cc:1048] T 8a878999c0024d2abe517484180290fc P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.030s
I20260709 13:19:38.298738 16265 leader_election.cc:304] T 8cc959fb7f704fba9009c0595467d074 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac
I20260709 13:19:38.265765 16219 consensus_queue.cc:260] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.3, Last appended by leader: 3, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } }
I20260709 13:19:38.280524 18138 raft_consensus.cc:2749] T 66f75124215f4ca6bef831e174a792d8 P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:19:38.305410 16191 leader_election.cc:304] T af57996a8be84d398f0f023e8d08ef77 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
I20260709 13:19:38.290827 18061 raft_consensus.cc:2749] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:19:38.279295 18104 consensus_queue.cc:1059] T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.026s
I20260709 13:19:38.319429 16266 leader_election.cc:304] T 9e65676b21b748728ad39668cfe722ec P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:19:38.323127 16298 raft_consensus.cc:2468] T ada5a004ba7c46819051e3ccec71ffdb P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 5.
I20260709 13:19:38.327821 16191 leader_election.cc:304] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
I20260709 13:19:38.326086 16265 leader_election.cc:304] T 7c2095a5f4ad49408dbc7f69c606120d P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac
W20260709 13:19:38.328197 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 54731.4us
I20260709 13:19:38.329053 16191 leader_election.cc:304] T 829b4ad8c5a54eababeea1181b960d4b P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
I20260709 13:19:38.326130 16299 raft_consensus.cc:2468] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 6.
I20260709 13:19:38.329741 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:38.329309 16116 leader_election.cc:304] T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
I20260709 13:19:38.330461 17644 leader_election.cc:290] T ecc1e363230043fe9fef390f0b979f20 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:38.330660 16226 raft_consensus.cc:2410] T ecc1e363230043fe9fef390f0b979f20 P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b for term 3 because replica has last-logged OpId of term: 2 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 3.
I20260709 13:19:38.333217 17644 raft_consensus.cc:493] T 02838c86531d4f98bf58d1c22b8baf77 P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:38.334136 16299 raft_consensus.cc:1275] T 3884f864a96d428785895e5463e5728f P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:19:38.313597 18116 raft_consensus.cc:2749] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac [term 6 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
W20260709 13:19:38.372167 16189 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:38.320098 18046 raft_consensus.cc:2804] T 8cc959fb7f704fba9009c0595467d074 P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Leader election won for term 3
W20260709 13:19:38.416980 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 87977.5us
I20260709 13:19:38.417784 16191 leader_election.cc:304] T c2f5f89186204242bf377f7c42b2e2da P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:19:38.327135 18006 raft_consensus.cc:3060] T af57996a8be84d398f0f023e8d08ef77 P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:19:38.420724 17644 raft_consensus.cc:515] T 02838c86531d4f98bf58d1c22b8baf77 P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:38.324841 18143 raft_consensus.cc:493] T 8d4ab870a05347868956aa8b841b2981 P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:38.330094 17931 raft_consensus.cc:3060] T fffab26427e24d398e2695aad40ca134 P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:19:38.427448 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 101448us
I20260709 13:19:38.428184 16265 leader_election.cc:304] T 2794af5b99984efebdee564035841cb0 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: cb82cc7bd6db46d980c75237a1402860
I20260709 13:19:38.332428 17999 raft_consensus.cc:2804] T 9e65676b21b748728ad39668cfe722ec P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Leader election won for term 4
W20260709 13:19:38.429754 16189 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:38.333576 18011 raft_consensus.cc:2749] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:19:38.449319 16298 raft_consensus.cc:1240] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 3. Current term is 4. Ops: []
I20260709 13:19:38.438531 18145 raft_consensus.cc:2804] T 18ad7e3af5504571a69d10c00750a01c P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:19:38.416092 18093 raft_consensus.cc:3060] T 8c11074579b046d99e585c2818e18a2c P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:19:38.416540 18138 raft_consensus.cc:3060] T 829b4ad8c5a54eababeea1181b960d4b P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:19:38.526872 17644 leader_election.cc:290] T 02838c86531d4f98bf58d1c22b8baf77 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:38.420166 18133 raft_consensus.cc:2749] T c2f5f89186204242bf377f7c42b2e2da P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:19:38.421648 18123 raft_consensus.cc:2804] T 7c2095a5f4ad49408dbc7f69c606120d P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Leader election won for term 2
W20260709 13:19:38.531615 16190 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:38.427083 16294 raft_consensus.cc:1240] T ada5a004ba7c46819051e3ccec71ffdb P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 13:19:38.428418 18046 raft_consensus.cc:697] T 8cc959fb7f704fba9009c0595467d074 P 90f3da57796b488c9087a2f523adf045 [term 3 LEADER]: Becoming Leader. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
W20260709 13:19:38.541215 16190 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:38.574647 16191 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:38.528682 18089 raft_consensus.cc:2804] T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:19:38.528662 18116 raft_consensus.cc:493] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:19:38.536777 18061 raft_consensus.cc:493] T 3884f864a96d428785895e5463e5728f P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:38.540185 17999 raft_consensus.cc:697] T 9e65676b21b748728ad39668cfe722ec P 90f3da57796b488c9087a2f523adf045 [term 4 LEADER]: Becoming Leader. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:19:38.539736 18145 raft_consensus.cc:697] T 18ad7e3af5504571a69d10c00750a01c P 647a343a542d4a139fe106c19095129b [term 3 LEADER]: Becoming Leader. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:19:38.454725 18143 raft_consensus.cc:515] T 8d4ab870a05347868956aa8b841b2981 P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:38.546473 16226 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "08c74490c40243a88e62d1760b5cd814"
dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
mode: GRACEFUL
new_leader_uuid: "90f3da57796b488c9087a2f523adf045"
 from {username='slave'} at 127.0.0.1:39144
I20260709 13:19:38.612527 18124 raft_consensus.cc:493] T 829b4ad8c5a54eababeea1181b960d4b P cb82cc7bd6db46d980c75237a1402860 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:38.613219 18124 raft_consensus.cc:515] T 829b4ad8c5a54eababeea1181b960d4b P cb82cc7bd6db46d980c75237a1402860 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:19:38.615705 18142 log.cc:927] Time spent T e5ad4eb281564936a59f41e387a47156 P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 0.291s	user 0.001s	sys 0.000s
I20260709 13:19:38.615718 18124 leader_election.cc:290] T 829b4ad8c5a54eababeea1181b960d4b P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:19:38.617785 18140 consensus_queue.cc:1048] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.155s
I20260709 13:19:38.619262 18133 raft_consensus.cc:493] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:19:38.619740 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4dc6b0541ca431ebc757beac00521f1" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:19:38.620093 18035 raft_consensus.cc:2749] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:19:38.619901 18133 raft_consensus.cc:515] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:38.615559 16143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a878999c0024d2abe517484180290fc" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:19:38.619634 16147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee9ed67006144797b1de6ea8c2013a68" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:19:38.588470 18089 raft_consensus.cc:697] T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b [term 6 LEADER]: Becoming Leader. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:19:38.629359 16150 raft_consensus.cc:1240] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 2. Current term is 3. Ops: []
I20260709 13:19:38.626595 16148 raft_consensus.cc:1240] T 84f2da98864b4425bf7c0055951fdd4b P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 1. Current term is 2. Ops: []
I20260709 13:19:38.632453 16150 raft_consensus.cc:1240] T 746aabf856ab401c9ef5e859371e7b67 P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 2. Current term is 3. Ops: []
I20260709 13:19:38.584748 18063 raft_consensus.cc:493] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:19:38.636526 16147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c61a39fd4f84424b129c1a6f9ba404d" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:19:38.637424 16147 raft_consensus.cc:3060] T 9c61a39fd4f84424b129c1a6f9ba404d P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:19:38.642565 17979 leader_election.cc:290] T 3884f864a96d428785895e5463e5728f P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:19:38.644218 17892 raft_consensus.cc:3055] T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:19:38.665743 17892 raft_consensus.cc:740] T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:19:38.665624 18145 consensus_queue.cc:237] T 18ad7e3af5504571a69d10c00750a01c P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } }
I20260709 13:19:38.645332 16822 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49906 (request call id 458) took 1150 ms. Trace:
I20260709 13:19:38.667395 16822 rpcz_store.cc:276] 0709 13:19:37.495272 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:37.495431 (+   159us) service_pool.cc:224] Handling call
0709 13:19:37.495781 (+   350us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:37.960737 (+464956us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:38.645055 (+684318us) spinlock_profiling.cc:243] Waited 464 ms on lock 0x7b54005519d4. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:19:38.645251 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1067786496}
I20260709 13:19:38.667418 17918 raft_consensus.cc:3055] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 1ca2725d290e47d68b1a4b609c9816ac [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:19:38.669147 17918 raft_consensus.cc:740] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 1ca2725d290e47d68b1a4b609c9816ac [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:19:38.670064 17918 consensus_queue.cc:260] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:19:38.646337 16820 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49896 (request call id 758) took 1499 ms (client timeout 1941 ms). Trace:
W20260709 13:19:38.671365 16820 rpcz_store.cc:269] 0709 13:19:37.146795 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:37.146897 (+   102us) service_pool.cc:224] Handling call
0709 13:19:38.646100 (+1499203us) spinlock_profiling.cc:243] Waited 270 ms on lock 0x7b54005c35d4. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:19:38.646257 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":620159360}
I20260709 13:19:38.647440 18042 raft_consensus.cc:3055] T 08c74490c40243a88e62d1760b5cd814 P 1ca2725d290e47d68b1a4b609c9816ac [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:19:38.671123 17918 raft_consensus.cc:3060] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:19:38.675524 17981 raft_consensus.cc:493] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:19:38.676158 17981 raft_consensus.cc:515] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:38.678529 17981 leader_election.cc:290] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:19:38.681057 18096 consensus_queue.cc:1059] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:19:38.681996 18096 raft_consensus.cc:3055] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:19:38.682149 18143 leader_election.cc:290] T 8d4ab870a05347868956aa8b841b2981 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:38.659991 16298 raft_consensus.cc:1240] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 2. Current term is 3. Ops: []
I20260709 13:19:38.632332 16218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada5a004ba7c46819051e3ccec71ffdb" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:19:38.661088 18007 consensus_queue.cc:1059] T 08c74490c40243a88e62d1760b5cd814 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:19:38.662966 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8638ee363273426db1393dd81e5c7b72" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:38.684938 16218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66ae176961b74813a58bc15c9c4c6f2a" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:19:38.685292 18097 raft_consensus.cc:3060] T 746aabf856ab401c9ef5e859371e7b67 P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:19:38.666976 17892 consensus_queue.cc:260] T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:38.687870 17892 raft_consensus.cc:3060] T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:19:38.663863 17999 consensus_queue.cc:237] T 9e65676b21b748728ad39668cfe722ec P 90f3da57796b488c9087a2f523adf045 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:38.687502 18063 raft_consensus.cc:515] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:38.690207 17844 consensus_queue.cc:1059] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:19:38.691031 18110 consensus_queue.cc:1059] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:19:38.691481 16220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8158bbd17b994d2eb96f560bd0f1b88d" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:38.672781 18042 raft_consensus.cc:740] T 08c74490c40243a88e62d1760b5cd814 P 1ca2725d290e47d68b1a4b609c9816ac [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:19:38.588107 18116 raft_consensus.cc:515] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:38.693841 16819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8638ee363273426db1393dd81e5c7b72" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:19:38.694597 16219 raft_consensus.cc:3060] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:19:38.672757 18124 raft_consensus.cc:493] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:19:38.696365 18013 raft_consensus.cc:3060] T d16dfad0c3b540e5b4b86492ddf87208 P 647a343a542d4a139fe106c19095129b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:19:38.756000 16192 leader_election.cc:304] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860
I20260709 13:19:38.757196 18035 raft_consensus.cc:2749] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:19:38.590965 18061 raft_consensus.cc:515] T 3884f864a96d428785895e5463e5728f P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:38.623842 18104 raft_consensus.cc:493] T 57a744f5aaf944678a63469cbf514015 P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:38.759660 18061 leader_election.cc:290] T 3884f864a96d428785895e5463e5728f P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:19:38.699556 16297 raft_consensus.cc:1240] T 84f2da98864b4425bf7c0055951fdd4b P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 1. Current term is 2. Ops: []
I20260709 13:19:38.700932 18089 consensus_queue.cc:237] T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:38.682389 18096 raft_consensus.cc:740] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:19:38.761577 18096 consensus_queue.cc:260] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:38.762915 18096 raft_consensus.cc:3060] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:19:38.702389 16117 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 395241us
I20260709 13:19:38.647596 18046 consensus_queue.cc:237] T 8cc959fb7f704fba9009c0595467d074 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:19:38.704756 16115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 70954.1us
W20260709 13:19:38.715446 16189 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:38.686671 16218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7ca61592e848ea9b0fb81a2ce33cdc" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:19:38.737778 16147 raft_consensus.cc:2468] T 9c61a39fd4f84424b129c1a6f9ba404d P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 4.
I20260709 13:19:38.774662 16191 leader_election.cc:304] T e4dc6b0541ca431ebc757beac00521f1 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
W20260709 13:19:38.702243 18144 log.cc:927] Time spent T 26354361e9b044cea1c2b86c4882fe92 P cb82cc7bd6db46d980c75237a1402860: Append to log took a long time: real 0.163s	user 0.001s	sys 0.000s
I20260709 13:19:38.745036 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "829b4ad8c5a54eababeea1181b960d4b" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:38.745085 17919 raft_consensus.cc:493] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:19:38.764664 17992 raft_consensus.cc:493] T af57996a8be84d398f0f023e8d08ef77 P cb82cc7bd6db46d980c75237a1402860 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:38.745743 16222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62fe3346cd3c42c4a55e14af9073bca4" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:38.629577 18133 leader_election.cc:290] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:19:38.747311 18042 consensus_queue.cc:260] T 08c74490c40243a88e62d1760b5cd814 P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:38.751376 18124 raft_consensus.cc:515] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:38.759488 18104 raft_consensus.cc:515] T 57a744f5aaf944678a63469cbf514015 P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:38.767717 16789 leader_election.cc:304] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:19:38.802139 16115 leader_election.cc:400] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 4 election: Vote denied by peer 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389) with higher term. Message: Invalid argument: T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b for earlier term 4. Current term is 5.
I20260709 13:19:38.803396 16789 leader_election.cc:304] T ada5a004ba7c46819051e3ccec71ffdb P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac
I20260709 13:19:38.746320 16220 raft_consensus.cc:2393] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 4: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
I20260709 13:19:38.776937 16222 raft_consensus.cc:2393] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 in current term 5: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
I20260709 13:19:38.777956 17992 raft_consensus.cc:515] T af57996a8be84d398f0f023e8d08ef77 P cb82cc7bd6db46d980c75237a1402860 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:38.809218 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d4ab870a05347868956aa8b841b2981" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:38.778312 16218 raft_consensus.cc:3060] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Advancing to term 6
I20260709 13:19:38.778438 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4f65cdf00b748bb963ee6e81ef43b79" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:19:38.779304 16292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31969392ccdb43b491911a319e8bc7e1" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:38.796185 17844 raft_consensus.cc:2749] T e4dc6b0541ca431ebc757beac00521f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:19:38.776379 17919 raft_consensus.cc:515] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:38.783469 16819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3884f864a96d428785895e5463e5728f" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:19:38.776467 18061 raft_consensus.cc:2804] T 2794af5b99984efebdee564035841cb0 P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:19:38.785425 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:38.785638 16291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02838c86531d4f98bf58d1c22b8baf77" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:38.769099 16294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3884f864a96d428785895e5463e5728f" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:38.853168 16294 raft_consensus.cc:2468] T 3884f864a96d428785895e5463e5728f P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 2.
I20260709 13:19:38.791580 16150 raft_consensus.cc:1275] T 3884f864a96d428785895e5463e5728f P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:19:38.791733 16147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a9de347c711477e87c582d003f36925" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:19:38.830556 18138 raft_consensus.cc:2749] T 829b4ad8c5a54eababeea1181b960d4b P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:19:38.805893 18034 raft_consensus.cc:2749] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:19:38.857110 18034 raft_consensus.cc:493] T c9944d5e4b8c4109af9955c4f08c9095 P cb82cc7bd6db46d980c75237a1402860 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:38.857590 18034 raft_consensus.cc:515] T c9944d5e4b8c4109af9955c4f08c9095 P cb82cc7bd6db46d980c75237a1402860 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:38.858841 17919 leader_election.cc:290] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:19:38.860158 16790 leader_election.cc:304] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:19:38.794776 16191 leader_election.cc:304] T 8a878999c0024d2abe517484180290fc P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
W20260709 13:19:38.861691 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 67013.8us
I20260709 13:19:38.861730 17919 raft_consensus.cc:2749] T 8a878999c0024d2abe517484180290fc P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:19:38.862537 16191 leader_election.cc:304] T ee9ed67006144797b1de6ea8c2013a68 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
I20260709 13:19:38.792588 16148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "829b4ad8c5a54eababeea1181b960d4b" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:19:38.865589 16148 raft_consensus.cc:2468] T 829b4ad8c5a54eababeea1181b960d4b P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 4.
I20260709 13:19:38.865768 18093 raft_consensus.cc:2749] T ee9ed67006144797b1de6ea8c2013a68 P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:19:38.866451 16217 raft_consensus.cc:2468] T 829b4ad8c5a54eababeea1181b960d4b P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 4.
I20260709 13:19:38.868098 17992 leader_election.cc:290] T af57996a8be84d398f0f023e8d08ef77 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:19:38.868657 17992 consensus_queue.cc:1048] T 3884f864a96d428785895e5463e5728f P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:19:38.869019 17982 raft_consensus.cc:493] T 26354361e9b044cea1c2b86c4882fe92 P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:38.869956 17992 raft_consensus.cc:2749] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:19:38.870334 18104 leader_election.cc:290] T 57a744f5aaf944678a63469cbf514015 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:19:38.869772 17982 raft_consensus.cc:515] T 26354361e9b044cea1c2b86c4882fe92 P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:38.872875 17982 leader_election.cc:290] T 26354361e9b044cea1c2b86c4882fe92 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:19:38.874109 16218 raft_consensus.cc:2468] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 1ca2725d290e47d68b1a4b609c9816ac [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 6.
I20260709 13:19:38.876362 18116 leader_election.cc:290] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:19:38.777503 18052 consensus_queue.cc:1059] T 74ca8140f35c4ae29e8a664f6bdfc829 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.051s
I20260709 13:19:38.800716 16117 leader_election.cc:304] T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:19:38.820390 16300 raft_consensus.cc:3060] T c4f65cdf00b748bb963ee6e81ef43b79 P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:19:38.821590 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 393456us
I20260709 13:19:38.816501 16297 raft_consensus.cc:2468] T 8d4ab870a05347868956aa8b841b2981 P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 3.
I20260709 13:19:38.780731 18134 raft_consensus.cc:3055] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:19:38.846416 16219 raft_consensus.cc:1275] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 3 index: 3. (term mismatch)
I20260709 13:19:38.799455 18110 consensus_queue.cc:1059] T 84f2da98864b4425bf7c0055951fdd4b P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.038s
I20260709 13:19:38.850526 16299 raft_consensus.cc:2393] T ecc1e363230043fe9fef390f0b979f20 P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 3: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
I20260709 13:19:38.893275 16265 leader_election.cc:304] T 5a96373bb0e14dce8a266b2fc6c58507 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac
W20260709 13:19:38.893839 16116 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.561s	user 0.005s	sys 0.010s
W20260709 13:19:38.894397 16264 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:38.917287 16116 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.584s	user 0.005s	sys 0.011s
I20260709 13:19:38.881968 16819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b7d1ab519044eda2d5e04a07242f25" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:19:38.882462 18151 raft_consensus.cc:493] T d16dfad0c3b540e5b4b86492ddf87208 P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:19:38.800550 17644 consensus_queue.cc:1059] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:19:38.878770 18124 leader_election.cc:290] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:38.821051 17981 raft_consensus.cc:2804] T ada5a004ba7c46819051e3ccec71ffdb P cb82cc7bd6db46d980c75237a1402860 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:19:38.881659 18123 raft_consensus.cc:697] T 7c2095a5f4ad49408dbc7f69c606120d P 90f3da57796b488c9087a2f523adf045 [term 2 LEADER]: Becoming Leader. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
W20260709 13:19:38.917937 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 588649us
I20260709 13:19:38.947067 17644 raft_consensus.cc:3055] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:19:38.879791 16142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62fe3346cd3c42c4a55e14af9073bca4" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:19:38.834074 16292 raft_consensus.cc:2468] T 31969392ccdb43b491911a319e8bc7e1 P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 2.
W20260709 13:19:38.934100 16117 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 133491us
I20260709 13:19:38.949733 16222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d4ab870a05347868956aa8b841b2981" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:38.950407 16222 raft_consensus.cc:2468] T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 3.
I20260709 13:19:38.950508 16790 leader_election.cc:304] T 829b4ad8c5a54eababeea1181b960d4b P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:19:38.932706 16823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5d59407eb545b895d8d922e0dd69fd" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:19:38.899840 16219 pending_rounds.cc:85] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac: Aborting all ops after (but not including) 2
I20260709 13:19:38.972374 16219 pending_rounds.cc:107] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac: Aborting uncommitted NO_OP operation due to leader change: 2.3
I20260709 13:19:38.972803 16219 raft_consensus.cc:2889] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:19:38.898824 18134 raft_consensus.cc:740] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
W20260709 13:19:38.975288 18114 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 3 current term: 3 is higher.
I20260709 13:19:38.977131 16790 leader_election.cc:304] T 3a7ca61592e848ea9b0fb81a2ce33cdc P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 
W20260709 13:19:38.891237 18137 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.716s	user 0.005s	sys 0.017s
W20260709 13:19:38.892935 18055 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.592s	user 0.002s	sys 0.014s
W20260709 13:19:38.917081 18108 consensus_meta.cc:287] T 5a96373bb0e14dce8a266b2fc6c58507 P 1ca2725d290e47d68b1a4b609c9816ac: Time spent flushing consensus metadata: real 0.692s	user 0.011s	sys 0.000s
I20260709 13:19:38.947479 17644 raft_consensus.cc:740] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:19:38.898247 18042 raft_consensus.cc:3060] T 08c74490c40243a88e62d1760b5cd814 P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:19:38.933096 18041 consensus_meta.cc:287] T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045: Time spent flushing consensus metadata: real 0.699s	user 0.009s	sys 0.002s
I20260709 13:19:38.959661 17992 raft_consensus.cc:3055] T 74ca8140f35c4ae29e8a664f6bdfc829 P cb82cc7bd6db46d980c75237a1402860 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:19:38.959779 18093 consensus_queue.cc:1059] T 84f2da98864b4425bf7c0055951fdd4b P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:19:38.960844 18123 consensus_queue.cc:237] T 7c2095a5f4ad49408dbc7f69c606120d P 90f3da57796b488c9087a2f523adf045 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:19:38.982846 18137 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.808s	user 0.014s	sys 0.020s
I20260709 13:19:38.951953 16115 leader_election.cc:304] T 3884f864a96d428785895e5463e5728f P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
W20260709 13:19:39.005995 16115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 54108.6us
I20260709 13:19:39.006897 16115 leader_election.cc:304] T 8d4ab870a05347868956aa8b841b2981 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:19:39.008087 16115 leader_election.cc:304] T ecc1e363230043fe9fef390f0b979f20 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
W20260709 13:19:38.965205 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 72028.6us
I20260709 13:19:38.966316 18034 leader_election.cc:290] T c9944d5e4b8c4109af9955c4f08c9095 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:38.967640 18156 raft_consensus.cc:2804] T 5a96373bb0e14dce8a266b2fc6c58507 P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:19:38.968603 18143 raft_consensus.cc:493] T 1df93779209748b9860f6d893dd3de53 P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:38.968772 17982 raft_consensus.cc:493] T 2794af5b99984efebdee564035841cb0 P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:39.024261 18143 raft_consensus.cc:515] T 1df93779209748b9860f6d893dd3de53 P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:39.026731 16299 raft_consensus.cc:1275] T 434bff3082d141838068f323bdb0d1ed P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260709 13:19:39.027026 16293 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48674 (request call id 1999) took 1110 ms. Trace:
I20260709 13:19:39.027554 16294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26354361e9b044cea1c2b86c4882fe92" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:38.966320 16116 leader_election.cc:304] T c0eab25ce4c64e018a041beeda2867db P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 90f3da57796b488c9087a2f523adf045
I20260709 13:19:39.027482 16293 rpcz_store.cc:276] 0709 13:19:37.916290 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:37.940418 (+ 24128us) service_pool.cc:224] Handling call
0709 13:19:37.940830 (+   412us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:37.941271 (+   441us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:37.941308 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:38.688047 (+746739us) spinlock_profiling.cc:243] Waited 37.4 ms on lock 0x7b2c00132978. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69af458a11 00007f69af468372 00007f69b0b9aa01 00007f69b0b5f43a 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2
0709 13:19:38.688515 (+   468us) log.cc:844] Serialized 26 byte log entry
0709 13:19:38.939575 (+251060us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:19:38.944691 (+  5116us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:38.949610 (+  4919us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:39.026741 (+ 77131us) raft_consensus.cc:1685] finished
0709 13:19:39.026766 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:39.026969 (+   203us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":487,"raft.run_cpu_time_us":140,"raft.run_wall_time_us":139,"spinlock_wait_cycles":86115712,"thread_start_us":218968,"threads_started":1,"wal-append.queue_time_us":336548}
W20260709 13:19:39.029246 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 62992.9us
I20260709 13:19:39.026981 18143 leader_election.cc:290] T 1df93779209748b9860f6d893dd3de53 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:38.987399 18124 raft_consensus.cc:2804] T 3a7ca61592e848ea9b0fb81a2ce33cdc P cb82cc7bd6db46d980c75237a1402860 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:19:38.944765 18151 raft_consensus.cc:515] T d16dfad0c3b540e5b4b86492ddf87208 P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:38.971306 16819 raft_consensus.cc:2393] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 3: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
I20260709 13:19:38.952037 16823 raft_consensus.cc:2468] T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 4.
I20260709 13:19:38.971762 18013 raft_consensus.cc:1064] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b: attempting to promote NON_VOTER cb82cc7bd6db46d980c75237a1402860 to VOTER
I20260709 13:19:38.979416 16148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af57996a8be84d398f0f023e8d08ef77" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:19:38.979851 16300 raft_consensus.cc:2468] T c4f65cdf00b748bb963ee6e81ef43b79 P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 6.
I20260709 13:19:38.981235 16222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9944d5e4b8c4109af9955c4f08c9095" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
W20260709 13:19:38.950125 18159 log.cc:927] Time spent T 00000000000000000000000000000000 P 667a34b27a39474ebefc1af3cf5b4271: Append to log took a long time: real 0.062s	user 0.001s	sys 0.001s
I20260709 13:19:38.946856 18096 raft_consensus.cc:493] T 8cc959fb7f704fba9009c0595467d074 P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:38.989598 16295 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48684 (request call id 1884) took 1113 ms. Trace:
W20260709 13:19:38.983871 16789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 178946us
I20260709 13:19:38.945353 18149 raft_consensus.cc:2804] T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:19:39.000679 16147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:19:38.948539 18039 raft_consensus.cc:493] T d2ab0ac8347641e6955e9bce7d90c1a2 P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:19:39.033859 16191 leader_election.cc:304] T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 90f3da57796b488c9087a2f523adf045
W20260709 13:19:38.984781 18055 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.683s	user 0.003s	sys 0.014s
I20260709 13:19:39.039109 16789 leader_election.cc:304] T b87147731a68400e8cce9b879daa7c5d P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 647a343a542d4a139fe106c19095129b
I20260709 13:19:39.015848 16265 leader_election.cc:304] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:19:39.040077 16148 raft_consensus.cc:2468] T af57996a8be84d398f0f023e8d08ef77 P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 3.
I20260709 13:19:38.985272 18052 raft_consensus.cc:2804] T 829b4ad8c5a54eababeea1181b960d4b P cb82cc7bd6db46d980c75237a1402860 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:19:39.021828 17644 consensus_queue.cc:260] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:39.050428 17644 raft_consensus.cc:3060] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:19:39.024758 16292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7ca61592e848ea9b0fb81a2ce33cdc" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:39.024355 17982 raft_consensus.cc:515] T 2794af5b99984efebdee564035841cb0 P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:38.970402 16824 raft_consensus.cc:1240] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [term 5 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 4. Current term is 5. Ops: []
I20260709 13:19:39.028681 18134 consensus_queue.cc:260] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:38.971282 16142 raft_consensus.cc:2468] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 5.
I20260709 13:19:39.030511 16116 leader_election.cc:304] T 8158bbd17b994d2eb96f560bd0f1b88d P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:19:39.032183 16222 raft_consensus.cc:2468] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 4.
W20260709 13:19:39.031281 18006 consensus_meta.cc:287] T af57996a8be84d398f0f023e8d08ef77 P 1ca2725d290e47d68b1a4b609c9816ac: Time spent flushing consensus metadata: real 0.604s	user 0.011s	sys 0.000s
I20260709 13:19:38.949793 17981 raft_consensus.cc:697] T ada5a004ba7c46819051e3ccec71ffdb P cb82cc7bd6db46d980c75237a1402860 [term 5 LEADER]: Becoming Leader. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:19:39.061797 16222 raft_consensus.cc:3060] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:19:39.029452 16294 raft_consensus.cc:3055] T 68a6838d82ef4cac83507f906139d90f P 90f3da57796b488c9087a2f523adf045 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:19:38.962041 16218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02838c86531d4f98bf58d1c22b8baf77" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:39.039708 18105 raft_consensus.cc:2804] T c0eab25ce4c64e018a041beeda2867db P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Leader election won for term 3
W20260709 13:19:39.022641 18109 consensus_meta.cc:287] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 1ca2725d290e47d68b1a4b609c9816ac: Time spent flushing consensus metadata: real 0.788s	user 0.015s	sys 0.000s
I20260709 13:19:39.049616 18052 raft_consensus.cc:493] T 829b4ad8c5a54eababeea1181b960d4b P cb82cc7bd6db46d980c75237a1402860 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:39.032145 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57a744f5aaf944678a63469cbf514015" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:39.104749 16297 raft_consensus.cc:2468] T 57a744f5aaf944678a63469cbf514015 P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 2.
I20260709 13:19:39.006409 16221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:39.036031 16295 rpcz_store.cc:276] 0709 13:19:37.875976 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:37.885045 (+  9069us) service_pool.cc:224] Handling call
0709 13:19:37.885357 (+   312us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:37.901625 (+ 16268us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:37.901656 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:37.915441 (+ 13785us) log.cc:844] Serialized 26 byte log entry
0709 13:19:38.434993 (+519552us) spinlock_profiling.cc:243] Waited 51.6 ms on lock 0x7b18000ba940. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b693ade1 00007f69b69475b1 00007f69ada54e07 00007f69b68afab9 00007f69ada7169b 00007f69ada6f93f 00007f69ada71c02 00007f69b06f920b 00007f69b06fed23 00007f69b06ff0d1 00007f69b0b1aa0f
0709 13:19:38.704096 (+269103us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:19:38.740652 (+ 36556us) log.cc:844] Serialized 14 byte log entry
0709 13:19:38.834459 (+ 93807us) spinlock_profiling.cc:243] Waited 1.13 ms on lock 0x7b18000ba940. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b693ade1 00007f69b69475b1 00007f69ada54e07 00007f69b68afab9 00007f69ada7169b 00007f69ada6f93f 00007f69ada6f455 00007f69b0b57060 00007f69b0b69716 00007f69b0b7a287 00007f69b0b7a208
0709 13:19:38.986589 (+152130us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:38.986702 (+   113us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:38.986762 (+    60us) raft_consensus.cc:1685] finished
0709 13:19:38.986783 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:38.989533 (+  2750us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":15849,"prepare.queue_time_us":38489,"prepare.run_cpu_time_us":44,"prepare.run_wall_time_us":45,"raft.queue_time_us":171600,"raft.run_cpu_time_us":187,"raft.run_wall_time_us":18092,"spinlock_wait_cycles":121349632,"thread_start_us":406029,"threads_started":2,"wal-append.queue_time_us":919029}
I20260709 13:19:39.030505 18104 raft_consensus.cc:2804] T 8d4ab870a05347868956aa8b841b2981 P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:19:39.030855 17979 raft_consensus.cc:3060] T ecc1e363230043fe9fef390f0b979f20 P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:19:39.063125 16148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b7d1ab519044eda2d5e04a07242f25" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:19:39.030786 16219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af57996a8be84d398f0f023e8d08ef77" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:39.037410 18108 raft_consensus.cc:493] T 8c11074579b046d99e585c2818e18a2c P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:39.070312 18006 raft_consensus.cc:2749] T af57996a8be84d398f0f023e8d08ef77 P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:19:39.041206 18149 raft_consensus.cc:493] T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:39.073951 16147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5d59407eb545b895d8d922e0dd69fd" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:19:39.074334 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1df93779209748b9860f6d893dd3de53" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
W20260709 13:19:39.102573 16189 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:39.048691 16265 leader_election.cc:304] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
I20260709 13:19:39.053313 18089 raft_consensus.cc:493] T 5a959dbecc2648eb85d1cc550f5a3245 P 647a343a542d4a139fe106c19095129b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:19:39.110000 16292 raft_consensus.cc:2368] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac for earlier term 5. Current term is 6.
I20260709 13:19:39.004024 18041 raft_consensus.cc:515] T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:39.057309 17919 raft_consensus.cc:2804] T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:19:39.127928 16296 raft_consensus.cc:1240] T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 2. Current term is 3. Ops: []
I20260709 13:19:39.107520 16223 raft_consensus.cc:1275] T 68a6838d82ef4cac83507f906139d90f P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:19:39.058012 18154 raft_consensus.cc:2804] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:19:39.072947 18145 raft_consensus.cc:3060] T 8158bbd17b994d2eb96f560bd0f1b88d P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:19:39.113989 16222 raft_consensus.cc:1275] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:19:39.000716 17992 raft_consensus.cc:740] T 74ca8140f35c4ae29e8a664f6bdfc829 P cb82cc7bd6db46d980c75237a1402860 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:19:39.039058 18046 consensus_queue.cc:1059] T e5ad4eb281564936a59f41e387a47156 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.060s
I20260709 13:19:39.074496 18134 raft_consensus.cc:3060] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:19:39.049218 18034 consensus_queue.cc:1048] T 3884f864a96d428785895e5463e5728f P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.054s
I20260709 13:19:39.053972 18156 raft_consensus.cc:697] T 5a96373bb0e14dce8a266b2fc6c58507 P 90f3da57796b488c9087a2f523adf045 [term 2 LEADER]: Becoming Leader. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
W20260709 13:19:39.114730 16117 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 164335us
I20260709 13:19:39.071702 16294 raft_consensus.cc:740] T 68a6838d82ef4cac83507f906139d90f P 90f3da57796b488c9087a2f523adf045 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:19:39.153174 16117 leader_election.cc:304] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac
I20260709 13:19:39.100062 18061 raft_consensus.cc:697] T 2794af5b99984efebdee564035841cb0 P 90f3da57796b488c9087a2f523adf045 [term 2 LEADER]: Becoming Leader. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:19:39.127322 18108 raft_consensus.cc:515] T 8c11074579b046d99e585c2818e18a2c P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:39.158067 18166 raft_consensus.cc:2804] T b87147731a68400e8cce9b879daa7c5d P cb82cc7bd6db46d980c75237a1402860 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:19:39.158617 18166 raft_consensus.cc:493] T b87147731a68400e8cce9b879daa7c5d P cb82cc7bd6db46d980c75237a1402860 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:39.159297 18108 leader_election.cc:290] T 8c11074579b046d99e585c2818e18a2c P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:19:39.159430 18148 raft_consensus.cc:2804] T 3884f864a96d428785895e5463e5728f P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:19:39.144079 18167 raft_consensus.cc:2749] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:19:39.154587 18154 raft_consensus.cc:697] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045 [term 5 LEADER]: Becoming Leader. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:19:39.127671 18149 raft_consensus.cc:3060] T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:19:39.162254 18154 consensus_queue.cc:237] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:39.131019 16147 raft_consensus.cc:2393] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 4: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
I20260709 13:19:39.055583 18096 raft_consensus.cc:515] T 8cc959fb7f704fba9009c0595467d074 P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:39.159439 16223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1df93779209748b9860f6d893dd3de53" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:39.166620 16223 raft_consensus.cc:2468] T 1df93779209748b9860f6d893dd3de53 P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 3.
W20260709 13:19:39.142426 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 93830.9us
I20260709 13:19:39.068225 18039 raft_consensus.cc:515] T d2ab0ac8347641e6955e9bce7d90c1a2 P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:39.109160 17918 raft_consensus.cc:493] T 7c2095a5f4ad49408dbc7f69c606120d P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:39.108742 18063 leader_election.cc:290] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:19:39.109354 18052 raft_consensus.cc:3060] T 829b4ad8c5a54eababeea1181b960d4b P cb82cc7bd6db46d980c75237a1402860 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:19:39.111765 18104 raft_consensus.cc:493] T 8d4ab870a05347868956aa8b841b2981 P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:39.074879 18143 consensus_queue.cc:1048] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.057s
I20260709 13:19:39.116361 18161 raft_consensus.cc:493] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:39.120187 16221 raft_consensus.cc:2468] T ecc1e363230043fe9fef390f0b979f20 P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 3.
I20260709 13:19:39.156275 18041 leader_election.cc:290] T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 3 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:19:39.156849 18061 consensus_queue.cc:237] T 2794af5b99984efebdee564035841cb0 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:19:39.132726 16789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 93702.1us
W20260709 13:19:39.169034 16117 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:39.185075 16115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 113647us
I20260709 13:19:39.265851 16148 raft_consensus.cc:1275] T 26354361e9b044cea1c2b86c4882fe92 P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:19:39.159017 18166 raft_consensus.cc:3060] T b87147731a68400e8cce9b879daa7c5d P cb82cc7bd6db46d980c75237a1402860 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:19:39.232739 16265 leader_election.cc:304] T 66ae176961b74813a58bc15c9c4c6f2a P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860
I20260709 13:19:39.153172 16219 raft_consensus.cc:2468] T af57996a8be84d398f0f023e8d08ef77 P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 3.
I20260709 13:19:39.235742 16223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02838c86531d4f98bf58d1c22b8baf77" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:39.237316 17992 consensus_queue.cc:260] T 74ca8140f35c4ae29e8a664f6bdfc829 P cb82cc7bd6db46d980c75237a1402860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:39.238588 18034 raft_consensus.cc:493] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:39.238936 18055 consensus_queue.cc:1048] T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:19:39.161948 16294 consensus_queue.cc:260] T 68a6838d82ef4cac83507f906139d90f P 90f3da57796b488c9087a2f523adf045 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:39.154229 18105 raft_consensus.cc:697] T c0eab25ce4c64e018a041beeda2867db P 647a343a542d4a139fe106c19095129b [term 3 LEADER]: Becoming Leader. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:19:39.234613 17918 raft_consensus.cc:515] T 7c2095a5f4ad49408dbc7f69c606120d P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:39.132674 18089 raft_consensus.cc:515] T 5a959dbecc2648eb85d1cc550f5a3245 P 647a343a542d4a139fe106c19095129b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:19:39.244136 18165 log.cc:927] Time spent T 8cc959fb7f704fba9009c0595467d074 P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.093s	user 0.000s	sys 0.000s
W20260709 13:19:39.245409 18164 log.cc:927] Time spent T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 0.090s	user 0.000s	sys 0.000s
I20260709 13:19:39.246483 16226 raft_consensus.cc:606] T 08c74490c40243a88e62d1760b5cd814 P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Received request to transfer leadership to TS 90f3da57796b488c9087a2f523adf045
I20260709 13:19:39.256013 18124 raft_consensus.cc:697] T 3a7ca61592e848ea9b0fb81a2ce33cdc P cb82cc7bd6db46d980c75237a1402860 [term 6 LEADER]: Becoming Leader. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:19:39.247844 18167 raft_consensus.cc:3055] T e5ad4eb281564936a59f41e387a47156 P 90f3da57796b488c9087a2f523adf045 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:19:39.309348 16190 leader_election.cc:400] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 5 pre-election: Vote denied by peer 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389) with higher term. Message: Invalid argument: T 3a7ca61592e848ea9b0fb81a2ce33cdc P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac for earlier term 5. Current term is 6.
I20260709 13:19:39.309779 16190 leader_election.cc:403] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 5 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:19:39.270093 16265 leader_election.cc:304] T ecc1e363230043fe9fef390f0b979f20 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:19:39.271894 16148 raft_consensus.cc:1275] T 1df93779209748b9860f6d893dd3de53 P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:19:39.310570 18124 consensus_queue.cc:237] T 3a7ca61592e848ea9b0fb81a2ce33cdc P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:39.258422 18104 raft_consensus.cc:3060] T 8d4ab870a05347868956aa8b841b2981 P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:19:39.281853 16144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6f07842d3aa47858df5665874c86f3a" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:19:39.278816 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:39.282299 16151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3884f864a96d428785895e5463e5728f" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:19:39.315119 16299 raft_consensus.cc:2393] T ecc1e363230043fe9fef390f0b979f20 P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 in current term 3: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
I20260709 13:19:39.169894 17981 consensus_queue.cc:237] T ada5a004ba7c46819051e3ccec71ffdb P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:39.287307 18177 raft_consensus.cc:493] T 2794af5b99984efebdee564035841cb0 P cb82cc7bd6db46d980c75237a1402860 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:39.316337 18177 raft_consensus.cc:515] T 2794af5b99984efebdee564035841cb0 P cb82cc7bd6db46d980c75237a1402860 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:39.317921 16217 raft_consensus.cc:1275] T e5ad4eb281564936a59f41e387a47156 P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:19:39.318722 18177 leader_election.cc:290] T 2794af5b99984efebdee564035841cb0 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:39.318861 18114 raft_consensus.cc:2749] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 1ca2725d290e47d68b1a4b609c9816ac [term 6 FOLLOWER]: Leader pre-election lost for term 5. Reason: Vote denied by peer 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389) with higher term. Message: Invalid argument: T 3a7ca61592e848ea9b0fb81a2ce33cdc P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac for earlier term 5. Current term is 6.
I20260709 13:19:39.321566 17644 raft_consensus.cc:493] T 5a96373bb0e14dce8a266b2fc6c58507 P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:39.322067 17644 raft_consensus.cc:515] T 5a96373bb0e14dce8a266b2fc6c58507 P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:39.323788 18143 consensus_queue.cc:1048] T 434bff3082d141838068f323bdb0d1ed P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:19:39.324839 18104 raft_consensus.cc:515] T 8d4ab870a05347868956aa8b841b2981 P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:39.300892 17982 leader_election.cc:290] T 2794af5b99984efebdee564035841cb0 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:19:39.306528 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c11074579b046d99e585c2818e18a2c" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:39.309262 18167 raft_consensus.cc:740] T e5ad4eb281564936a59f41e387a47156 P 90f3da57796b488c9087a2f523adf045 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:19:39.146377 17919 raft_consensus.cc:697] T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac [term 4 LEADER]: Becoming Leader. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:19:39.335312 17919 consensus_queue.cc:237] T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:39.336256 18104 leader_election.cc:290] T 8d4ab870a05347868956aa8b841b2981 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 4 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
W20260709 13:19:39.336787 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 66825.1us
I20260709 13:19:39.337693 16265 leader_election.cc:304] T a378d7f796f94615b3b347119a9880fd P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860
W20260709 13:19:39.311570 18163 log.cc:927] Time spent T 18ad7e3af5504571a69d10c00750a01c P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 0.126s	user 0.000s	sys 0.001s
W20260709 13:19:39.326243 17892 consensus_meta.cc:287] T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b: Time spent flushing consensus metadata: real 0.609s	user 0.011s	sys 0.000s
I20260709 13:19:39.264820 18161 raft_consensus.cc:515] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:39.327504 16300 raft_consensus.cc:2468] T 8c11074579b046d99e585c2818e18a2c P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 4.
I20260709 13:19:39.301445 16226 raft_consensus.cc:614] T 08c74490c40243a88e62d1760b5cd814 P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:19:39.330946 17644 leader_election.cc:290] T 5a96373bb0e14dce8a266b2fc6c58507 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:39.276328 18154 raft_consensus.cc:2749] T 66ae176961b74813a58bc15c9c4c6f2a P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:19:39.327986 16292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9944d5e4b8c4109af9955c4f08c9095" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:39.346737 16223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6f07842d3aa47858df5665874c86f3a" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:19:39.302146 16294 raft_consensus.cc:3060] T 68a6838d82ef4cac83507f906139d90f P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:19:39.291152 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fffab26427e24d398e2695aad40ca134" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
W20260709 13:19:39.349855 16115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 83542.3us
I20260709 13:19:39.289727 16221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fffab26427e24d398e2695aad40ca134" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:19:39.357264 16190 leader_election.cc:304] T 8c11074579b046d99e585c2818e18a2c P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:19:39.300755 18155 consensus_queue.cc:1048] T 68a6838d82ef4cac83507f906139d90f P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.183s
I20260709 13:19:39.359453 16115 leader_election.cc:304] T 57a744f5aaf944678a63469cbf514015 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:19:39.307768 16142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02838c86531d4f98bf58d1c22b8baf77" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:19:39.296404 18156 consensus_queue.cc:237] T 5a96373bb0e14dce8a266b2fc6c58507 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:39.366848 16292 raft_consensus.cc:2468] T c9944d5e4b8c4109af9955c4f08c9095 P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 3.
I20260709 13:19:39.370065 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a96373bb0e14dce8a266b2fc6c58507" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:39.362654 16192 leader_election.cc:304] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:19:39.372061 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a96373bb0e14dce8a266b2fc6c58507" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:39.373134 16192 leader_election.cc:304] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
I20260709 13:19:39.373301 18096 leader_election.cc:290] T 8cc959fb7f704fba9009c0595467d074 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:39.374177 18096 raft_consensus.cc:2804] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:19:39.374606 18096 raft_consensus.cc:493] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Starting leader election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:19:39.287974 18034 raft_consensus.cc:515] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:39.360780 17992 raft_consensus.cc:3060] T 74ca8140f35c4ae29e8a664f6bdfc829 P cb82cc7bd6db46d980c75237a1402860 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:19:39.339336 18151 leader_election.cc:290] T d16dfad0c3b540e5b4b86492ddf87208 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:19:39.383790 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2794af5b99984efebdee564035841cb0" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:39.345752 18041 raft_consensus.cc:2749] T ecc1e363230043fe9fef390f0b979f20 P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:19:39.348717 18063 raft_consensus.cc:493] T 5a96373bb0e14dce8a266b2fc6c58507 P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:39.339525 18181 raft_consensus.cc:2804] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:19:39.356459 18042 raft_consensus.cc:493] T 18ad7e3af5504571a69d10c00750a01c P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:39.350119 18046 raft_consensus.cc:2749] T a378d7f796f94615b3b347119a9880fd P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:19:39.410375 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 45414.3us
I20260709 13:19:39.387776 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d4ab870a05347868956aa8b841b2981" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:19:39.412343 16297 raft_consensus.cc:3060] T 8d4ab870a05347868956aa8b841b2981 P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:19:39.406826 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 86796us
I20260709 13:19:39.414095 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cc959fb7f704fba9009c0595467d074" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:39.375072 18096 raft_consensus.cc:3060] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:19:39.367856 18116 raft_consensus.cc:3055] T 84f2da98864b4425bf7c0055951fdd4b P 1ca2725d290e47d68b1a4b609c9816ac [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:19:39.414327 16790 leader_election.cc:304] T c9944d5e4b8c4109af9955c4f08c9095 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 
W20260709 13:19:39.518303 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 104519us
I20260709 13:19:39.376283 17892 raft_consensus.cc:2804] T 57a744f5aaf944678a63469cbf514015 P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:19:39.511185 16823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c11074579b046d99e585c2818e18a2c" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
W20260709 13:19:39.414417 18172 log.cc:927] Time spent T 8a878999c0024d2abe517484180290fc P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 0.146s	user 0.001s	sys 0.001s
I20260709 13:19:39.368841 18108 raft_consensus.cc:2804] T 8c11074579b046d99e585c2818e18a2c P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:19:39.408643 18170 raft_consensus.cc:493] T 8c11074579b046d99e585c2818e18a2c P cb82cc7bd6db46d980c75237a1402860 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:19:39.523801 17918 leader_election.cc:290] T 7c2095a5f4ad49408dbc7f69c606120d P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:39.421375 17844 raft_consensus.cc:2749] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:19:39.506042 18178 raft_consensus.cc:493] T 8cc959fb7f704fba9009c0595467d074 P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:39.527690 16220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d4ab870a05347868956aa8b841b2981" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:19:39.528498 18034 leader_election.cc:290] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:19:39.529165 17979 raft_consensus.cc:2749] T ecc1e363230043fe9fef390f0b979f20 P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:19:39.415105 18105 consensus_queue.cc:237] T c0eab25ce4c64e018a041beeda2867db P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:39.415814 18042 raft_consensus.cc:515] T 18ad7e3af5504571a69d10c00750a01c P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } }
I20260709 13:19:39.512956 18041 raft_consensus.cc:493] T 8a878999c0024d2abe517484180290fc P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:19:39.521425 17892 raft_consensus.cc:493] T 57a744f5aaf944678a63469cbf514015 P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:19:39.536999 18175 log.cc:927] Time spent T 02838c86531d4f98bf58d1c22b8baf77 P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.259s	user 0.000s	sys 0.001s
I20260709 13:19:39.543673 18108 raft_consensus.cc:493] T 8c11074579b046d99e585c2818e18a2c P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:39.526996 17981 raft_consensus.cc:2804] T c9944d5e4b8c4109af9955c4f08c9095 P cb82cc7bd6db46d980c75237a1402860 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:19:39.549808 18185 raft_consensus.cc:493] T 08c74490c40243a88e62d1760b5cd814 P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:39.552412 17982 raft_consensus.cc:493] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:39.554076 18180 log.cc:927] Time spent T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 0.178s	user 0.000s	sys 0.000s
I20260709 13:19:39.554802 18178 raft_consensus.cc:515] T 8cc959fb7f704fba9009c0595467d074 P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:39.514408 18124 raft_consensus.cc:493] T 434bff3082d141838068f323bdb0d1ed P cb82cc7bd6db46d980c75237a1402860 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:39.415298 18063 raft_consensus.cc:515] T 5a96373bb0e14dce8a266b2fc6c58507 P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:19:39.555523 18184 log.cc:927] Time spent T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.256s	user 0.000s	sys 0.001s
I20260709 13:19:39.554316 18149 raft_consensus.cc:515] T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:39.557082 16296 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48684 (request call id 1889) took 1380 ms. Trace:
I20260709 13:19:39.567414 16148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c2095a5f4ad49408dbc7f69c606120d" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:19:39.518026 18116 raft_consensus.cc:740] T 84f2da98864b4425bf7c0055951fdd4b P 1ca2725d290e47d68b1a4b609c9816ac [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:19:39.589668 16820 raft_consensus.cc:1240] T 66f75124215f4ca6bef831e174a792d8 P cb82cc7bd6db46d980c75237a1402860 [term 4 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 3. Current term is 4. Ops: []
I20260709 13:19:39.611894 16820 raft_consensus.cc:1275] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 [term 3 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:19:39.586908 16144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cc959fb7f704fba9009c0595467d074" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:19:39.543926 16297 raft_consensus.cc:2468] T 8d4ab870a05347868956aa8b841b2981 P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 4.
I20260709 13:19:39.580610 16819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7ca61592e848ea9b0fb81a2ce33cdc" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:19:39.614239 16825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57a744f5aaf944678a63469cbf514015" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:19:39.581622 16220 raft_consensus.cc:3060] T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:19:39.583920 18034 raft_consensus.cc:493] T 57a744f5aaf944678a63469cbf514015 P cb82cc7bd6db46d980c75237a1402860 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:39.619869 18174 log.cc:927] Time spent T 02838c86531d4f98bf58d1c22b8baf77 P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 0.102s	user 0.000s	sys 0.000s
I20260709 13:19:39.620007 18034 raft_consensus.cc:515] T 57a744f5aaf944678a63469cbf514015 P cb82cc7bd6db46d980c75237a1402860 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:39.620820 18149 leader_election.cc:290] T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 3 election: Requested vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:19:39.616199 18116 consensus_queue.cc:260] T 84f2da98864b4425bf7c0055951fdd4b P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:39.622646 18034 leader_election.cc:290] T 57a744f5aaf944678a63469cbf514015 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:39.626627 18147 raft_consensus.cc:493] T 746aabf856ab401c9ef5e859371e7b67 P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:39.627568 16224 raft_consensus.cc:2468] T 5a96373bb0e14dce8a266b2fc6c58507 P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 2.
I20260709 13:19:39.627372 18147 raft_consensus.cc:515] T 746aabf856ab401c9ef5e859371e7b67 P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:39.629015 18063 leader_election.cc:290] T 5a96373bb0e14dce8a266b2fc6c58507 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:39.594084 18041 raft_consensus.cc:515] T 8a878999c0024d2abe517484180290fc P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:39.630465 16144 raft_consensus.cc:2468] T 8cc959fb7f704fba9009c0595467d074 P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 3.
I20260709 13:19:39.630739 16825 raft_consensus.cc:2468] T 57a744f5aaf944678a63469cbf514015 P cb82cc7bd6db46d980c75237a1402860 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 2.
W20260709 13:19:39.603087 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 265478us
I20260709 13:19:39.591990 17892 raft_consensus.cc:3060] T 57a744f5aaf944678a63469cbf514015 P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:19:39.594959 17981 raft_consensus.cc:493] T c9944d5e4b8c4109af9955c4f08c9095 P cb82cc7bd6db46d980c75237a1402860 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:39.595845 18108 raft_consensus.cc:3060] T 8c11074579b046d99e585c2818e18a2c P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:19:39.607702 16148 raft_consensus.cc:2468] T 7c2095a5f4ad49408dbc7f69c606120d P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 2.
I20260709 13:19:39.607450 16296 rpcz_store.cc:276] 0709 13:19:38.177027 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:38.323997 (+146970us) service_pool.cc:224] Handling call
0709 13:19:38.324244 (+   247us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:39.132469 (+808225us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:39.555087 (+422618us) spinlock_profiling.cc:243] Waited 803 ms on lock 0x7b54002e0754. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:19:39.557021 (+  1934us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1847836416}
I20260709 13:19:39.609077 18042 leader_election.cc:290] T 18ad7e3af5504571a69d10c00750a01c P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:39.609558 16817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26354361e9b044cea1c2b86c4882fe92" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:19:39.557646 18170 raft_consensus.cc:515] T 8c11074579b046d99e585c2818e18a2c P cb82cc7bd6db46d980c75237a1402860 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:39.632984 16822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3884f864a96d428785895e5463e5728f" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:19:39.641098 16265 leader_election.cc:304] T d2ab0ac8347641e6955e9bce7d90c1a2 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860
I20260709 13:19:39.629765 18147 leader_election.cc:290] T 746aabf856ab401c9ef5e859371e7b67 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:19:39.599005 18181 raft_consensus.cc:697] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b [term 3 LEADER]: Becoming Leader. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:19:39.648540 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a959dbecc2648eb85d1cc550f5a3245" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:39.659987 16225 raft_consensus.cc:2393] T 5a959dbecc2648eb85d1cc550f5a3245 P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 2: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
W20260709 13:19:39.660095 18191 log.cc:927] Time spent T 5a96373bb0e14dce8a266b2fc6c58507 P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
I20260709 13:19:39.599697 18185 raft_consensus.cc:515] T 08c74490c40243a88e62d1760b5cd814 P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:39.661135 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2372981006d4b218215c28964dea623" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:39.661994 16225 raft_consensus.cc:2468] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 3.
I20260709 13:19:39.650985 18089 leader_election.cc:290] T 5a959dbecc2648eb85d1cc550f5a3245 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:39.663173 18185 leader_election.cc:290] T 08c74490c40243a88e62d1760b5cd814 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:39.663672 18177 raft_consensus.cc:493] T 23f82224f4564fea93572e8206bad1fe P cb82cc7bd6db46d980c75237a1402860 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:19:39.664410 18177 raft_consensus.cc:515] T 23f82224f4564fea93572e8206bad1fe P cb82cc7bd6db46d980c75237a1402860 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:39.668569 18178 leader_election.cc:290] T 8cc959fb7f704fba9009c0595467d074 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:39.669162 18179 raft_consensus.cc:493] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P cb82cc7bd6db46d980c75237a1402860 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:39.669690 18179 raft_consensus.cc:515] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P cb82cc7bd6db46d980c75237a1402860 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:39.674180 16146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a96373bb0e14dce8a266b2fc6c58507" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:19:39.606089 18124 raft_consensus.cc:515] T 434bff3082d141838068f323bdb0d1ed P cb82cc7bd6db46d980c75237a1402860 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:39.653471 18161 leader_election.cc:290] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:19:39.653990 18189 raft_consensus.cc:493] T 66f75124215f4ca6bef831e174a792d8 P cb82cc7bd6db46d980c75237a1402860 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:39.656430 16820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31969392ccdb43b491911a319e8bc7e1" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:19:39.674312 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31969392ccdb43b491911a319e8bc7e1" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:19:39.674852 16146 raft_consensus.cc:2468] T 5a96373bb0e14dce8a266b2fc6c58507 P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 2.
I20260709 13:19:39.651209 16265 leader_election.cc:304] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:19:39.644706 17981 raft_consensus.cc:3060] T c9944d5e4b8c4109af9955c4f08c9095 P cb82cc7bd6db46d980c75237a1402860 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:19:39.646239 16819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2794af5b99984efebdee564035841cb0" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:19:39.675799 16191 leader_election.cc:304] T 7c2095a5f4ad49408dbc7f69c606120d P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
I20260709 13:19:39.690094 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c74490c40243a88e62d1760b5cd814" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:39.690886 16191 leader_election.cc:304] T 5a96373bb0e14dce8a266b2fc6c58507 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
I20260709 13:19:39.605294 17982 raft_consensus.cc:515] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:39.682255 16820 raft_consensus.cc:3060] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:19:39.682480 16297 raft_consensus.cc:3060] T 31969392ccdb43b491911a319e8bc7e1 P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:19:39.690946 16298 raft_consensus.cc:2468] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 4.
I20260709 13:19:39.698983 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2794af5b99984efebdee564035841cb0" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:39.682101 16823 raft_consensus.cc:2468] T 8c11074579b046d99e585c2818e18a2c P cb82cc7bd6db46d980c75237a1402860 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 3.
I20260709 13:19:39.682098 16299 raft_consensus.cc:1275] T c4f65cdf00b748bb963ee6e81ef43b79 P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:19:39.693327 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746aabf856ab401c9ef5e859371e7b67" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:39.696144 16790 leader_election.cc:304] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:19:39.667094 18177 leader_election.cc:290] T 23f82224f4564fea93572e8206bad1fe P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:39.689823 16292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a959dbecc2648eb85d1cc550f5a3245" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:39.717963 16225 raft_consensus.cc:2468] T 746aabf856ab401c9ef5e859371e7b67 P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 3.
W20260709 13:19:39.720780 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 69688.5us
I20260709 13:19:39.721843 16265 leader_election.cc:304] T c9944d5e4b8c4109af9955c4f08c9095 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: cb82cc7bd6db46d980c75237a1402860
I20260709 13:19:39.723393 16265 leader_election.cc:304] T cdbd78acb4cf46298e0de8ba0d130e19 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860
I20260709 13:19:39.725976 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57a744f5aaf944678a63469cbf514015" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:39.726548 16298 raft_consensus.cc:2468] T 57a744f5aaf944678a63469cbf514015 P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 2.
I20260709 13:19:39.733346 16265 leader_election.cc:304] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:19:39.736279 16820 raft_consensus.cc:2468] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 3.
I20260709 13:19:39.700052 18034 raft_consensus.cc:493] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:39.747006 16265 leader_election.cc:304] T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860
I20260709 13:19:39.736393 16297 raft_consensus.cc:2468] T 31969392ccdb43b491911a319e8bc7e1 P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 3.
I20260709 13:19:39.699903 18063 raft_consensus.cc:2804] T 7c2095a5f4ad49408dbc7f69c606120d P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:19:39.715543 18161 raft_consensus.cc:2749] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:19:39.737604 16822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746aabf856ab401c9ef5e859371e7b67" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:19:39.745206 18151 raft_consensus.cc:2749] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:19:39.750275 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f82224f4564fea93572e8206bad1fe" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:39.719234 17918 raft_consensus.cc:493] T 746aabf856ab401c9ef5e859371e7b67 P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:39.756098 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cc959fb7f704fba9009c0595467d074" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:39.760208 16225 raft_consensus.cc:2468] T 8cc959fb7f704fba9009c0595467d074 P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 3.
I20260709 13:19:39.689823 18147 consensus_queue.cc:1048] T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.055s
I20260709 13:19:39.724756 18177 raft_consensus.cc:493] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:19:39.716535 16819 raft_consensus.cc:2468] T 2794af5b99984efebdee564035841cb0 P cb82cc7bd6db46d980c75237a1402860 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 2.
I20260709 13:19:39.745611 16292 raft_consensus.cc:2393] T 5a959dbecc2648eb85d1cc550f5a3245 P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 2: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
I20260709 13:19:39.751307 16816 raft_consensus.cc:1275] T 434bff3082d141838068f323bdb0d1ed P cb82cc7bd6db46d980c75237a1402860 [term 4 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260709 13:19:39.735347 18046 raft_consensus.cc:2804] T c9944d5e4b8c4109af9955c4f08c9095 P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:19:39.732501 16143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18ad7e3af5504571a69d10c00750a01c" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:19:39.715294 18134 raft_consensus.cc:2804] T 5a96373bb0e14dce8a266b2fc6c58507 P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:19:39.756707 18063 raft_consensus.cc:493] T 7c2095a5f4ad49408dbc7f69c606120d P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:39.738900 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cc959fb7f704fba9009c0595467d074" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:39.734949 18123 raft_consensus.cc:2749] T cdbd78acb4cf46298e0de8ba0d130e19 P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:19:39.747150 18156 consensus_queue.cc:1048] T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.036s
I20260709 13:19:39.750164 18154 raft_consensus.cc:2749] T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:19:39.771703 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c74490c40243a88e62d1760b5cd814" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
W20260709 13:19:39.830535 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 83684.2us
I20260709 13:19:39.831099 16225 raft_consensus.cc:2468] T 08c74490c40243a88e62d1760b5cd814 P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 4.
I20260709 13:19:39.768034 17918 raft_consensus.cc:515] T 746aabf856ab401c9ef5e859371e7b67 P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:39.771407 18193 raft_consensus.cc:493] T d16dfad0c3b540e5b4b86492ddf87208 P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:39.781108 18179 leader_election.cc:290] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
W20260709 13:19:39.781602 18192 log.cc:927] Time spent T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 0.127s	user 0.000s	sys 0.001s
I20260709 13:19:39.781966 18198 raft_consensus.cc:2804] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:19:39.825404 18063 raft_consensus.cc:3060] T 7c2095a5f4ad49408dbc7f69c606120d P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:19:39.759069 16822 raft_consensus.cc:2468] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 3.
I20260709 13:19:39.690443 18189 raft_consensus.cc:515] T 66f75124215f4ca6bef831e174a792d8 P cb82cc7bd6db46d980c75237a1402860 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:39.764108 16221 raft_consensus.cc:2468] T fffab26427e24d398e2695aad40ca134 P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 2.
I20260709 13:19:39.854874 18109 raft_consensus.cc:493] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:39.855475 18109 raft_consensus.cc:515] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:39.857530 18109 leader_election.cc:290] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:19:39.843801 16265 leader_election.cc:304] T c0eab25ce4c64e018a041beeda2867db P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
W20260709 13:19:39.858479 18166 consensus_meta.cc:287] T b87147731a68400e8cce9b879daa7c5d P cb82cc7bd6db46d980c75237a1402860: Time spent flushing consensus metadata: real 0.571s	user 0.006s	sys 0.003s
I20260709 13:19:39.767557 18199 raft_consensus.cc:2749] T d2ab0ac8347641e6955e9bce7d90c1a2 P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:19:39.784917 18114 raft_consensus.cc:493] T c0eab25ce4c64e018a041beeda2867db P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:39.783285 18046 raft_consensus.cc:493] T c9944d5e4b8c4109af9955c4f08c9095 P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:19:39.818475 18134 raft_consensus.cc:493] T 5a96373bb0e14dce8a266b2fc6c58507 P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:39.849462 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5d59407eb545b895d8d922e0dd69fd" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:39.782334 17982 leader_election.cc:290] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:19:39.769938 18177 raft_consensus.cc:515] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:39.855784 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e4bfb9090c8460eb14fa7e0d818e7f1" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:39.756508 18034 raft_consensus.cc:515] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:39.858935 18166 raft_consensus.cc:515] T b87147731a68400e8cce9b879daa7c5d P cb82cc7bd6db46d980c75237a1402860 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:19:39.862543 18052 consensus_meta.cc:287] T 829b4ad8c5a54eababeea1181b960d4b P cb82cc7bd6db46d980c75237a1402860: Time spent flushing consensus metadata: real 0.601s	user 0.007s	sys 0.000s
I20260709 13:19:39.851538 18198 raft_consensus.cc:493] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:39.863833 18190 raft_consensus.cc:493] T 9e65676b21b748728ad39668cfe722ec P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:39.863839 18187 log.cc:927] Time spent T 3a7ca61592e848ea9b0fb81a2ce33cdc P cb82cc7bd6db46d980c75237a1402860: Append to log took a long time: real 0.083s	user 0.000s	sys 0.000s
I20260709 13:19:39.828044 18096 raft_consensus.cc:515] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:39.879868 16265 leader_election.cc:304] T 8158bbd17b994d2eb96f560bd0f1b88d P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 647a343a542d4a139fe106c19095129b
I20260709 13:19:39.850224 18194 raft_consensus.cc:493] T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:39.873178 18186 log.cc:927] Time spent T 8a878999c0024d2abe517484180290fc P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 0.225s	user 0.000s	sys 0.001s
I20260709 13:19:39.847446 18151 consensus_queue.cc:1059] T e5ad4eb281564936a59f41e387a47156 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.035s
I20260709 13:19:39.875248 18046 raft_consensus.cc:3060] T c9944d5e4b8c4109af9955c4f08c9095 P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:19:39.893707 16265 leader_election.cc:304] T 5a959dbecc2648eb85d1cc550f5a3245 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 647a343a542d4a139fe106c19095129b
I20260709 13:19:39.874466 18134 raft_consensus.cc:3060] T 5a96373bb0e14dce8a266b2fc6c58507 P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:19:39.874931 18041 leader_election.cc:290] T 8a878999c0024d2abe517484180290fc P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:19:39.885180 16217 raft_consensus.cc:2468] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 2.
I20260709 13:19:39.880582 18161 raft_consensus.cc:2749] T c0eab25ce4c64e018a041beeda2867db P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:19:39.901126 16225 raft_consensus.cc:2468] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 4.
I20260709 13:19:39.818261 18181 consensus_queue.cc:237] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:39.901916 16265 leader_election.cc:304] T fffab26427e24d398e2695aad40ca134 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:19:39.904103 16265 leader_election.cc:304] T 23f82224f4564fea93572e8206bad1fe P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
I20260709 13:19:39.889488 18198 raft_consensus.cc:3060] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:19:39.912256 16220 raft_consensus.cc:2468] T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 4.
I20260709 13:19:39.917951 16220 raft_consensus.cc:1275] T 18ad7e3af5504571a69d10c00750a01c P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:19:39.906540 18123 raft_consensus.cc:2804] T fffab26427e24d398e2695aad40ca134 P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:19:39.925783 16217 raft_consensus.cc:1275] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac [term 6 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:19:39.852342 18193 raft_consensus.cc:515] T d16dfad0c3b540e5b4b86492ddf87208 P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:39.889299 18052 raft_consensus.cc:515] T 829b4ad8c5a54eababeea1181b960d4b P cb82cc7bd6db46d980c75237a1402860 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:19:39.926016 18145 consensus_meta.cc:287] T 8158bbd17b994d2eb96f560bd0f1b88d P 647a343a542d4a139fe106c19095129b: Time spent flushing consensus metadata: real 0.779s	user 0.011s	sys 0.000s
I20260709 13:19:39.906145 18204 raft_consensus.cc:493] T 84f2da98864b4425bf7c0055951fdd4b P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:39.941675 18204 raft_consensus.cc:515] T 84f2da98864b4425bf7c0055951fdd4b P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:39.944062 18204 leader_election.cc:290] T 84f2da98864b4425bf7c0055951fdd4b P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:39.907227 18154 raft_consensus.cc:2804] T 5a959dbecc2648eb85d1cc550f5a3245 P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:19:39.945127 18154 raft_consensus.cc:697] T 5a959dbecc2648eb85d1cc550f5a3245 P 90f3da57796b488c9087a2f523adf045 [term 2 LEADER]: Becoming Leader. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:19:39.946034 18154 consensus_queue.cc:237] T 5a959dbecc2648eb85d1cc550f5a3245 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:39.881676 18114 raft_consensus.cc:515] T c0eab25ce4c64e018a041beeda2867db P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:39.935535 18046 raft_consensus.cc:515] T c9944d5e4b8c4109af9955c4f08c9095 P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:39.936455 18096 leader_election.cc:290] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 4 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:19:39.936635 18052 leader_election.cc:290] T 829b4ad8c5a54eababeea1181b960d4b P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 5 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:19:39.929342 18123 raft_consensus.cc:697] T fffab26427e24d398e2695aad40ca134 P 90f3da57796b488c9087a2f523adf045 [term 2 LEADER]: Becoming Leader. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:19:39.937472 18042 raft_consensus.cc:493] T d16dfad0c3b540e5b4b86492ddf87208 P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:39.938341 18145 raft_consensus.cc:2749] T 8158bbd17b994d2eb96f560bd0f1b88d P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:19:39.939874 18124 leader_election.cc:290] T 434bff3082d141838068f323bdb0d1ed P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:39.892843 18190 raft_consensus.cc:515] T 9e65676b21b748728ad39668cfe722ec P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:39.959892 18200 raft_consensus.cc:493] T 9e65676b21b748728ad39668cfe722ec P cb82cc7bd6db46d980c75237a1402860 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:39.940814 18170 leader_election.cc:290] T 8c11074579b046d99e585c2818e18a2c P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:39.940609 17892 raft_consensus.cc:515] T 57a744f5aaf944678a63469cbf514015 P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:39.894807 18194 raft_consensus.cc:515] T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:39.897339 18199 raft_consensus.cc:2804] T 8158bbd17b994d2eb96f560bd0f1b88d P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:19:39.949775 18161 raft_consensus.cc:2749] T 23f82224f4564fea93572e8206bad1fe P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:19:39.962298 16294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f2da98864b4425bf7c0055951fdd4b" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:39.962833 18042 raft_consensus.cc:515] T d16dfad0c3b540e5b4b86492ddf87208 P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:19:39.951184 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 47134.8us
I20260709 13:19:39.949069 18114 leader_election.cc:290] T c0eab25ce4c64e018a041beeda2867db P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:39.952280 16821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e4bfb9090c8460eb14fa7e0d818e7f1" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:19:39.949179 18046 leader_election.cc:290] T c9944d5e4b8c4109af9955c4f08c9095 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 4 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:19:39.971123 16822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5d59407eb545b895d8d922e0dd69fd" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:19:39.964076 18200 raft_consensus.cc:515] T 9e65676b21b748728ad39668cfe722ec P cb82cc7bd6db46d980c75237a1402860 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:39.986909 18042 leader_election.cc:290] T d16dfad0c3b540e5b4b86492ddf87208 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:39.977912 16821 raft_consensus.cc:2468] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P cb82cc7bd6db46d980c75237a1402860 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 2.
W20260709 13:19:39.979545 18203 log.cc:927] Time spent T 434bff3082d141838068f323bdb0d1ed P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.128s	user 0.000s	sys 0.000s
I20260709 13:19:39.980239 18190 leader_election.cc:290] T 9e65676b21b748728ad39668cfe722ec P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:19:39.981196 18194 leader_election.cc:290] T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:19:39.994956 18041 consensus_queue.cc:1048] T 1df93779209748b9860f6d893dd3de53 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:19:39.983227 18151 consensus_queue.cc:1048] T 26354361e9b044cea1c2b86c4882fe92 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.006s
I20260709 13:19:39.984192 18052 raft_consensus.cc:493] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:39.985170 17892 leader_election.cc:290] T 57a744f5aaf944678a63469cbf514015 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 3 election: Requested vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:19:39.985008 16296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e65676b21b748728ad39668cfe722ec" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:40.002615 18052 raft_consensus.cc:515] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:40.003108 18200 leader_election.cc:290] T 9e65676b21b748728ad39668cfe722ec P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:39.989302 16265 leader_election.cc:304] T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:19:39.990612 16192 leader_election.cc:304] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:19:40.005295 16265 leader_election.cc:304] T 3884f864a96d428785895e5463e5728f P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
I20260709 13:19:39.978664 16294 raft_consensus.cc:2468] T 84f2da98864b4425bf7c0055951fdd4b P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 2.
I20260709 13:19:39.992784 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57a744f5aaf944678a63469cbf514015" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:19:39.991453 16822 raft_consensus.cc:2468] T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 4.
I20260709 13:19:40.006919 16265 leader_election.cc:304] T 02838c86531d4f98bf58d1c22b8baf77 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:19:40.012562 16192 leader_election.cc:304] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:19:40.018118 16146 raft_consensus.cc:1240] T 9c61a39fd4f84424b129c1a6f9ba404d P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 3. Current term is 4. Ops: []
I20260709 13:19:40.018801 16145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a878999c0024d2abe517484180290fc" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:19:39.978892 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f2da98864b4425bf7c0055951fdd4b" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:39.995296 16292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d16dfad0c3b540e5b4b86492ddf87208" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:39.978091 18199 raft_consensus.cc:697] T 8158bbd17b994d2eb96f560bd0f1b88d P 90f3da57796b488c9087a2f523adf045 [term 4 LEADER]: Becoming Leader. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:19:40.001811 18166 leader_election.cc:290] T b87147731a68400e8cce9b879daa7c5d P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 5 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:39.988371 18108 raft_consensus.cc:515] T 8c11074579b046d99e585c2818e18a2c P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:40.010591 18193 leader_election.cc:290] T d16dfad0c3b540e5b4b86492ddf87208 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:40.040937 18123 consensus_queue.cc:237] T fffab26427e24d398e2695aad40ca134 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:40.036264 18114 raft_consensus.cc:2804] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:19:40.044191 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2372981006d4b218215c28964dea623" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:40.046888 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9944d5e4b8c4109af9955c4f08c9095" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:19:40.057750 16224 raft_consensus.cc:2468] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 3.
I20260709 13:19:40.058424 18114 raft_consensus.cc:697] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac [term 4 LEADER]: Becoming Leader. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:19:40.058718 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a878999c0024d2abe517484180290fc" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:40.047454 16143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d16dfad0c3b540e5b4b86492ddf87208" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:19:40.024580 16293 raft_consensus.cc:3060] T 57a744f5aaf944678a63469cbf514015 P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:19:39.990931 18063 raft_consensus.cc:515] T 7c2095a5f4ad49408dbc7f69c606120d P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:40.024089 18161 raft_consensus.cc:2749] T 02838c86531d4f98bf58d1c22b8baf77 P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:19:40.010020 18041 raft_consensus.cc:2749] T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:19:40.001304 17981 raft_consensus.cc:515] T c9944d5e4b8c4109af9955c4f08c9095 P cb82cc7bd6db46d980c75237a1402860 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:40.011473 18046 raft_consensus.cc:2749] T 3884f864a96d428785895e5463e5728f P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:19:40.011699 18042 raft_consensus.cc:2804] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:19:40.061619 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d16dfad0c3b540e5b4b86492ddf87208" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:40.067981 16224 raft_consensus.cc:2468] T d16dfad0c3b540e5b4b86492ddf87208 P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 2.
I20260709 13:19:40.053632 18052 leader_election.cc:290] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:19:40.058948 18148 raft_consensus.cc:493] T 3884f864a96d428785895e5463e5728f P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Starting leader election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:19:40.016149 18134 raft_consensus.cc:515] T 5a96373bb0e14dce8a266b2fc6c58507 P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:40.055011 16292 raft_consensus.cc:2468] T d16dfad0c3b540e5b4b86492ddf87208 P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 1.
I20260709 13:19:40.069020 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e4bfb9090c8460eb14fa7e0d818e7f1" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:40.086175 18134 leader_election.cc:290] T 5a96373bb0e14dce8a266b2fc6c58507 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 3 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:40.076272 18161 consensus_queue.cc:1059] T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.012s
I20260709 13:19:40.076418 18148 raft_consensus.cc:3060] T 3884f864a96d428785895e5463e5728f P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:19:40.067924 18042 raft_consensus.cc:493] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:40.103047 16225 raft_consensus.cc:2468] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 4.
I20260709 13:19:40.065389 16143 raft_consensus.cc:2468] T d16dfad0c3b540e5b4b86492ddf87208 P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 2.
I20260709 13:19:40.071607 18114 consensus_queue.cc:237] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:40.159448 16825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ab0ac8347641e6955e9bce7d90c1a2" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
W20260709 13:19:40.213215 16190 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:40.195518 16822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c11074579b046d99e585c2818e18a2c" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
W20260709 13:19:40.167147 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 99452us
I20260709 13:19:40.276685 18145 raft_consensus.cc:493] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:19:40.088455 17999 raft_consensus.cc:3055] T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:19:40.293612 18145 raft_consensus.cc:515] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:40.294314 16265 leader_election.cc:304] T 8a878999c0024d2abe517484180290fc P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:19:40.126123 18042 raft_consensus.cc:3060] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:19:40.297050 16790 leader_election.cc:304] T af57996a8be84d398f0f023e8d08ef77 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:19:40.087152 18198 raft_consensus.cc:515] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:40.283062 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d16dfad0c3b540e5b4b86492ddf87208" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:40.251044 18190 raft_consensus.cc:493] T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:40.093432 17981 leader_election.cc:290] T c9944d5e4b8c4109af9955c4f08c9095 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 5 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:40.317474 17992 raft_consensus.cc:2804] T af57996a8be84d398f0f023e8d08ef77 P cb82cc7bd6db46d980c75237a1402860 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:19:40.092816 18039 leader_election.cc:290] T d2ab0ac8347641e6955e9bce7d90c1a2 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:19:40.167275 18194 raft_consensus.cc:493] T fffab26427e24d398e2695aad40ca134 P 647a343a542d4a139fe106c19095129b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:19:40.318014 17992 raft_consensus.cc:493] T af57996a8be84d398f0f023e8d08ef77 P cb82cc7bd6db46d980c75237a1402860 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:40.318459 17992 raft_consensus.cc:3060] T af57996a8be84d398f0f023e8d08ef77 P cb82cc7bd6db46d980c75237a1402860 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:19:40.319674 16225 raft_consensus.cc:2393] T d16dfad0c3b540e5b4b86492ddf87208 P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 2: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
I20260709 13:19:40.283452 17918 leader_election.cc:290] T 746aabf856ab401c9ef5e859371e7b67 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:19:40.094597 18063 leader_election.cc:290] T 7c2095a5f4ad49408dbc7f69c606120d P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 3 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:40.204942 18116 raft_consensus.cc:3060] T 84f2da98864b4425bf7c0055951fdd4b P 1ca2725d290e47d68b1a4b609c9816ac [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:19:40.237774 17844 raft_consensus.cc:493] T c2f5f89186204242bf377f7c42b2e2da P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:40.318486 18194 raft_consensus.cc:515] T fffab26427e24d398e2695aad40ca134 P 647a343a542d4a139fe106c19095129b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:40.321676 16293 raft_consensus.cc:2468] T 57a744f5aaf944678a63469cbf514015 P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 3.
I20260709 13:19:40.322113 16151 raft_consensus.cc:1275] T 9e65676b21b748728ad39668cfe722ec P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260709 13:19:40.167460 18149 raft_consensus.cc:493] T b87147731a68400e8cce9b879daa7c5d P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:19:40.213568 18108 leader_election.cc:290] T 8c11074579b046d99e585c2818e18a2c P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 5 election: Requested vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:19:40.298079 17999 raft_consensus.cc:740] T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:19:40.237494 18096 raft_consensus.cc:493] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:19:40.297909 18110 consensus_queue.cc:1059] T b6f07842d3aa47858df5665874c86f3a P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.066s
I20260709 13:19:40.179607 18199 consensus_queue.cc:237] T 8158bbd17b994d2eb96f560bd0f1b88d P 90f3da57796b488c9087a2f523adf045 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:40.420466 16225 raft_consensus.cc:1275] T c0eab25ce4c64e018a041beeda2867db P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:19:40.371762 16790 leader_election.cc:304] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac
W20260709 13:19:40.692625 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 320986us
I20260709 13:19:40.692291 16265 leader_election.cc:304] T c9944d5e4b8c4109af9955c4f08c9095 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:19:40.304672 18218 raft_consensus.cc:2749] T 8a878999c0024d2abe517484180290fc P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:19:40.700529 16296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e4bfb9090c8460eb14fa7e0d818e7f1" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:40.687122 18123 raft_consensus.cc:493] T 18ad7e3af5504571a69d10c00750a01c P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:40.702024 16296 raft_consensus.cc:2468] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 2.
I20260709 13:19:40.673090 18109 raft_consensus.cc:493] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:40.703510 16297 raft_consensus.cc:1240] T 84f2da98864b4425bf7c0055951fdd4b P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 1. Current term is 2. Ops: []
W20260709 13:19:40.692152 18171 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.336s	user 0.015s	sys 0.024s
I20260709 13:19:40.704865 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18ad7e3af5504571a69d10c00750a01c" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:40.421936 18217 raft_consensus.cc:493] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:19:40.692322 17981 raft_consensus.cc:2804] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:19:40.712397 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87ac0d1f7e8a419f9bcbc04aee0e067e" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:40.328246 18190 raft_consensus.cc:515] T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:40.649435 18198 leader_election.cc:290] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 4 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:19:40.695305 18096 raft_consensus.cc:515] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:40.330245 17844 raft_consensus.cc:515] T c2f5f89186204242bf377f7c42b2e2da P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:40.698429 18182 raft_consensus.cc:493] T 74ca8140f35c4ae29e8a664f6bdfc829 P cb82cc7bd6db46d980c75237a1402860 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:40.329087 18148 raft_consensus.cc:515] T 3884f864a96d428785895e5463e5728f P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:40.724018 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c11074579b046d99e585c2818e18a2c" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
W20260709 13:19:40.712145 18171 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.375s	user 0.015s	sys 0.025s
I20260709 13:19:40.726142 16296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a96373bb0e14dce8a266b2fc6c58507" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:19:40.694284 18149 raft_consensus.cc:515] T b87147731a68400e8cce9b879daa7c5d P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:40.711928 16294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a96373bb0e14dce8a266b2fc6c58507" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
W20260709 13:19:40.740073 18176 log.cc:927] Time spent T 3884f864a96d428785895e5463e5728f P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 1.401s	user 0.000s	sys 0.004s
I20260709 13:19:40.742808 18034 leader_election.cc:290] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:19:40.725162 18217 raft_consensus.cc:515] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:19:40.744156 16823 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49906 (request call id 470) took 1355 ms (client timeout 1701 ms). Trace:
I20260709 13:19:40.745358 16147 raft_consensus.cc:1240] T 68a6838d82ef4cac83507f906139d90f P 647a343a542d4a139fe106c19095129b [term 5 LEADER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 4. Current term is 5. Ops: []
I20260709 13:19:40.740932 18194 leader_election.cc:290] T fffab26427e24d398e2695aad40ca134 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
W20260709 13:19:40.755861 16823 rpcz_store.cc:269] 0709 13:19:39.388866 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:39.388983 (+   117us) service_pool.cc:224] Handling call
0709 13:19:40.743976 (+1354993us) spinlock_profiling.cc:243] Waited 161 ms on lock 0x7b540051b154. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:19:40.744103 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":370941824}
I20260709 13:19:40.756238 16147 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40830 (request call id 1013) took 1492 ms. Trace:
I20260709 13:19:40.756619 16147 rpcz_store.cc:276] 0709 13:19:39.264214 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:39.264383 (+   169us) service_pool.cc:224] Handling call
0709 13:19:39.277701 (+ 13318us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:40.745223 (+1467522us) spinlock_profiling.cc:243] Waited 90.7 ms on lock 0x7b5400312a20. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0ad6dc0 00007f69b0b611c0 00007f69b0b62231 00007f69b0b5ed8f 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c
0709 13:19:40.755984 (+ 10761us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:40.756188 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":208680832}
I20260709 13:19:40.737571 16297 raft_consensus.cc:3060] T 8c11074579b046d99e585c2818e18a2c P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:19:40.723690 18220 log.cc:927] Time spent T 02838c86531d4f98bf58d1c22b8baf77 P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 0.310s	user 0.000s	sys 0.001s
I20260709 13:19:40.713259 17981 raft_consensus.cc:493] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:19:40.712090 16220 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33050 (request call id 393) took 1017 ms. Trace:
I20260709 13:19:40.760564 16220 rpcz_store.cc:276] 0709 13:19:39.695066 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:39.924913 (+229847us) service_pool.cc:224] Handling call
0709 13:19:39.925200 (+   287us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:39.925395 (+   195us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:19:40.036853 (+111458us) log.cc:844] Serialized 14 byte log entry
0709 13:19:40.692911 (+656058us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:40.695004 (+  2093us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:19:40.698579 (+  3575us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:40.702575 (+  3996us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:40.710890 (+  8315us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":98029,"prepare.run_cpu_time_us":32,"prepare.run_wall_time_us":276,"raft.queue_time_us":315206,"raft.run_cpu_time_us":82,"raft.run_wall_time_us":82,"thread_start_us":388048,"threads_started":2,"wal-append.queue_time_us":683688}
I20260709 13:19:40.753856 16296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ab0ac8347641e6955e9bce7d90c1a2" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:40.762673 16296 raft_consensus.cc:2468] T d2ab0ac8347641e6955e9bce7d90c1a2 P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 4.
W20260709 13:19:40.762959 16219 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33050 (request call id 387) took 1783 ms (client timeout 2066 ms). Trace:
I20260709 13:19:40.723634 17992 raft_consensus.cc:515] T af57996a8be84d398f0f023e8d08ef77 P cb82cc7bd6db46d980c75237a1402860 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:40.708844 18123 raft_consensus.cc:515] T 18ad7e3af5504571a69d10c00750a01c P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } }
I20260709 13:19:40.731519 18225 raft_consensus.cc:2804] T c9944d5e4b8c4109af9955c4f08c9095 P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:19:40.709815 18109 raft_consensus.cc:515] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:40.780815 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c11074579b046d99e585c2818e18a2c" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:40.749276 18182 raft_consensus.cc:515] T 74ca8140f35c4ae29e8a664f6bdfc829 P cb82cc7bd6db46d980c75237a1402860 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:40.746495 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fffab26427e24d398e2695aad40ca134" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:40.726413 16217 raft_consensus.cc:2468] T 84f2da98864b4425bf7c0055951fdd4b P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 2.
I20260709 13:19:40.762034 16223 raft_consensus.cc:1240] T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac [term 4 LEADER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 13:19:40.763540 16116 leader_election.cc:304] T 02838c86531d4f98bf58d1c22b8baf77 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
W20260709 13:19:40.763458 16219 rpcz_store.cc:269] 0709 13:19:38.979796 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:38.982880 (+  3084us) service_pool.cc:224] Handling call
0709 13:19:40.762770 (+1779890us) spinlock_profiling.cc:243] Waited 31.2 ms on lock 0x7b5400395654. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:19:40.762882 (+   112us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":71774976}
I20260709 13:19:40.783979 16822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9944d5e4b8c4109af9955c4f08c9095" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:19:40.783782 17900 raft_consensus.cc:493] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:40.784789 16822 raft_consensus.cc:2368] T c9944d5e4b8c4109af9955c4f08c9095 P cb82cc7bd6db46d980c75237a1402860 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 for earlier term 4. Current term is 5.
I20260709 13:19:40.770383 16296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c2095a5f4ad49408dbc7f69c606120d" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:19:40.784960 17900 raft_consensus.cc:515] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:40.785789 16291 raft_consensus.cc:1275] T ada5a004ba7c46819051e3ccec71ffdb P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:19:40.786026 16296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87147731a68400e8cce9b879daa7c5d" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:19:40.786573 16296 raft_consensus.cc:3060] T b87147731a68400e8cce9b879daa7c5d P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:19:40.786826 16291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9944d5e4b8c4109af9955c4f08c9095" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:19:40.787518 17900 leader_election.cc:290] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:40.724961 18145 leader_election.cc:290] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
W20260709 13:19:40.787917 18224 log.cc:927] Time spent T fffab26427e24d398e2695aad40ca134 P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.063s	user 0.000s	sys 0.000s
I20260709 13:19:40.788472 16225 raft_consensus.cc:2393] T fffab26427e24d398e2695aad40ca134 P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 2: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
I20260709 13:19:40.788839 17844 leader_election.cc:290] T c2f5f89186204242bf377f7c42b2e2da P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:40.777755 18225 raft_consensus.cc:697] T c9944d5e4b8c4109af9955c4f08c9095 P 90f3da57796b488c9087a2f523adf045 [term 4 LEADER]: Becoming Leader. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:19:40.801682 18225 consensus_queue.cc:237] T c9944d5e4b8c4109af9955c4f08c9095 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:40.760711 18148 leader_election.cc:290] T 3884f864a96d428785895e5463e5728f P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 3 election: Requested vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:19:40.782552 16789 leader_election.cc:304] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:19:40.782639 16300 raft_consensus.cc:2468] T 18ad7e3af5504571a69d10c00750a01c P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 3.
I20260709 13:19:40.805126 18190 leader_election.cc:290] T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:19:40.760025 16150 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46224 (request call id 251) took 1778 ms. Trace:
I20260709 13:19:40.806917 18149 leader_election.cc:290] T b87147731a68400e8cce9b879daa7c5d P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:19:40.805872 16150 rpcz_store.cc:276] 0709 13:19:38.982049 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:38.982171 (+   122us) service_pool.cc:224] Handling call
0709 13:19:38.982502 (+   331us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:38.982845 (+   343us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:38.982877 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:38.987872 (+  4995us) log.cc:844] Serialized 26 byte log entry
0709 13:19:39.143709 (+155837us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:19:39.143750 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:39.143818 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:40.739429 (+1595611us) spinlock_profiling.cc:243] Waited 10.4 ms on lock 0x7b4400022b4c. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69ad8fadb1 00007f69ad998c85 00007f69ad9996a2 00007f69b070b1db 00007f69b0706aaf 00007f69b06fa86e 00007f69b06f9bb6 00007f69b06f97d8 00007f69b0703221 00007f69b07031d9 00007f69b0703169
0709 13:19:40.759755 (+ 20326us) raft_consensus.cc:1685] finished
0709 13:19:40.759777 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:40.759970 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2415,"raft.run_cpu_time_us":157,"raft.run_wall_time_us":14168,"spinlock_wait_cycles":23867136,"thread_start_us":119224,"threads_started":1,"wal-append.queue_time_us":268268}
I20260709 13:19:40.782802 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e65676b21b748728ad39668cfe722ec" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:40.809573 18198 raft_consensus.cc:3060] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:19:40.784027 16295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434bff3082d141838068f323bdb0d1ed" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:40.789849 16116 leader_election.cc:304] T 1df93779209748b9860f6d893dd3de53 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 90f3da57796b488c9087a2f523adf045
I20260709 13:19:40.790179 18217 leader_election.cc:290] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:19:40.789774 18167 consensus_queue.cc:260] T e5ad4eb281564936a59f41e387a47156 P 90f3da57796b488c9087a2f523adf045 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:40.815970 18167 raft_consensus.cc:3060] T e5ad4eb281564936a59f41e387a47156 P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:19:40.816794 18181 raft_consensus.cc:2804] T 1df93779209748b9860f6d893dd3de53 P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:19:40.817233 18181 raft_consensus.cc:493] T 1df93779209748b9860f6d893dd3de53 P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Starting leader election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:19:40.817600 18181 raft_consensus.cc:3060] T 1df93779209748b9860f6d893dd3de53 P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:19:40.765450 17981 raft_consensus.cc:3060] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:19:40.785048 16294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e4bfb9090c8460eb14fa7e0d818e7f1" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:40.790537 18140 raft_consensus.cc:2749] T 02838c86531d4f98bf58d1c22b8baf77 P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:19:40.770989 16825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e4bfb9090c8460eb14fa7e0d818e7f1" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:19:40.826061 16825 raft_consensus.cc:2468] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P cb82cc7bd6db46d980c75237a1402860 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 2.
I20260709 13:19:40.793175 16296 raft_consensus.cc:2468] T b87147731a68400e8cce9b879daa7c5d P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 5.
I20260709 13:19:40.721240 17999 consensus_queue.cc:260] T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:40.776525 16297 raft_consensus.cc:2468] T 8c11074579b046d99e585c2818e18a2c P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 5.
I20260709 13:19:40.795320 18109 leader_election.cc:290] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:19:40.800436 18177 leader_election.cc:290] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:40.783555 18042 raft_consensus.cc:515] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:40.827540 16116 leader_election.cc:304] T 746aabf856ab401c9ef5e859371e7b67 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
I20260709 13:19:40.829674 16116 leader_election.cc:304] T 8cc959fb7f704fba9009c0595467d074 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
I20260709 13:19:40.818295 18182 leader_election.cc:290] T 74ca8140f35c4ae29e8a664f6bdfc829 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:40.835525 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0eab25ce4c64e018a041beeda2867db" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:40.835726 18198 raft_consensus.cc:2749] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:19:40.836164 16299 raft_consensus.cc:2468] T c0eab25ce4c64e018a041beeda2867db P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 3.
I20260709 13:19:40.833887 17999 raft_consensus.cc:3060] T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:19:40.838390 18096 leader_election.cc:290] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
W20260709 13:19:40.839075 18195 log.cc:927] Time spent T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 1.076s	user 0.001s	sys 0.001s
I20260709 13:19:40.832788 16116 leader_election.cc:304] T 08c74490c40243a88e62d1760b5cd814 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
I20260709 13:19:40.843016 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d16dfad0c3b540e5b4b86492ddf87208" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:40.848220 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f5f89186204242bf377f7c42b2e2da" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:40.848847 16190 leader_election.cc:304] T d2ab0ac8347641e6955e9bce7d90c1a2 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:19:40.852274 16190 leader_election.cc:304] T 8c11074579b046d99e585c2818e18a2c P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:19:40.853449 16190 leader_election.cc:304] T c0eab25ce4c64e018a041beeda2867db P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:19:40.862571 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74ca8140f35c4ae29e8a664f6bdfc829" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:40.862962 16116 leader_election.cc:304] T 8d4ab870a05347868956aa8b841b2981 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
I20260709 13:19:40.863415 16297 raft_consensus.cc:2393] T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 3: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
I20260709 13:19:40.866022 16224 raft_consensus.cc:2393] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 3: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
W20260709 13:19:40.873193 16114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:40.829167 16294 raft_consensus.cc:2468] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 2.
I20260709 13:19:40.865186 18109 raft_consensus.cc:2804] T 8c11074579b046d99e585c2818e18a2c P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Leader election won for term 5
W20260709 13:19:40.892724 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 200504us
I20260709 13:19:40.826387 18148 consensus_queue.cc:1048] T e5ad4eb281564936a59f41e387a47156 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.040s
I20260709 13:19:40.886261 18061 raft_consensus.cc:493] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:40.860092 18096 raft_consensus.cc:2804] T d2ab0ac8347641e6955e9bce7d90c1a2 P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:19:40.860517 18145 raft_consensus.cc:2804] T 8cc959fb7f704fba9009c0595467d074 P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:19:40.859364 18149 raft_consensus.cc:2804] T 746aabf856ab401c9ef5e859371e7b67 P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:19:40.867446 18194 raft_consensus.cc:2804] T 8d4ab870a05347868956aa8b841b2981 P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:19:40.866281 18134 raft_consensus.cc:2804] T c0eab25ce4c64e018a041beeda2867db P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:19:40.866082 18140 raft_consensus.cc:2804] T 08c74490c40243a88e62d1760b5cd814 P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:19:40.866756 18041 raft_consensus.cc:493] T e4dc6b0541ca431ebc757beac00521f1 P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:19:40.926992 16296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87ac0d1f7e8a419f9bcbc04aee0e067e" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:40.868376 16293 raft_consensus.cc:2468] T d16dfad0c3b540e5b4b86492ddf87208 P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 1.
I20260709 13:19:40.899773 17844 raft_consensus.cc:493] T 68a6838d82ef4cac83507f906139d90f P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:40.935182 16296 raft_consensus.cc:2468] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 3.
I20260709 13:19:40.941457 16296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3884f864a96d428785895e5463e5728f" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:19:40.942145 16296 raft_consensus.cc:3060] T 3884f864a96d428785895e5463e5728f P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:19:40.929574 18145 raft_consensus.cc:493] T 8cc959fb7f704fba9009c0595467d074 P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:40.920029 18109 raft_consensus.cc:697] T 8c11074579b046d99e585c2818e18a2c P 1ca2725d290e47d68b1a4b609c9816ac [term 5 LEADER]: Becoming Leader. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:19:40.927884 18149 raft_consensus.cc:493] T 746aabf856ab401c9ef5e859371e7b67 P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:40.921294 18096 raft_consensus.cc:493] T d2ab0ac8347641e6955e9bce7d90c1a2 P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Starting leader election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:19:40.921715 18190 consensus_queue.cc:1048] T 18ad7e3af5504571a69d10c00750a01c P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.024s
I20260709 13:19:40.970813 18149 raft_consensus.cc:3060] T 746aabf856ab401c9ef5e859371e7b67 P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:19:40.940399 16789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 120962us
I20260709 13:19:40.922992 18061 raft_consensus.cc:515] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:40.955482 16297 raft_consensus.cc:1275] T c0eab25ce4c64e018a041beeda2867db P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:19:40.935318 18134 raft_consensus.cc:493] T c0eab25ce4c64e018a041beeda2867db P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:40.961756 16296 raft_consensus.cc:2468] T 3884f864a96d428785895e5463e5728f P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 3.
I20260709 13:19:40.935835 18140 raft_consensus.cc:493] T 08c74490c40243a88e62d1760b5cd814 P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:40.920202 18057 consensus_queue.cc:1048] T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.025s
I20260709 13:19:40.991294 16789 leader_election.cc:304] T 57a744f5aaf944678a63469cbf514015 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:19:40.960700 18145 raft_consensus.cc:3060] T 8cc959fb7f704fba9009c0595467d074 P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:19:40.959543 16294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fffab26427e24d398e2695aad40ca134" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:40.958813 18034 raft_consensus.cc:493] T 26354361e9b044cea1c2b86c4882fe92 P cb82cc7bd6db46d980c75237a1402860 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:19:40.955317 17844 raft_consensus.cc:515] T 68a6838d82ef4cac83507f906139d90f P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:40.975720 18096 raft_consensus.cc:3060] T d2ab0ac8347641e6955e9bce7d90c1a2 P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:19:40.939827 18041 raft_consensus.cc:515] T e4dc6b0541ca431ebc757beac00521f1 P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:40.990244 18116 raft_consensus.cc:493] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:41.001138 18093 raft_consensus.cc:493] T 19baa7bc367844c288bca8f81439c413 P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:19:41.023093 18034 raft_consensus.cc:515] T 26354361e9b044cea1c2b86c4882fe92 P cb82cc7bd6db46d980c75237a1402860 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:41.003892 18134 raft_consensus.cc:3060] T c0eab25ce4c64e018a041beeda2867db P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:19:40.993518 18194 raft_consensus.cc:697] T 8d4ab870a05347868956aa8b841b2981 P 647a343a542d4a139fe106c19095129b [term 4 LEADER]: Becoming Leader. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:19:41.004066 18140 raft_consensus.cc:3060] T 08c74490c40243a88e62d1760b5cd814 P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:19:41.035105 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 171020us
I20260709 13:19:41.009323 17981 raft_consensus.cc:515] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:41.009229 17900 raft_consensus.cc:2804] T 57a744f5aaf944678a63469cbf514015 P cb82cc7bd6db46d980c75237a1402860 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:19:41.038282 16265 leader_election.cc:304] T d16dfad0c3b540e5b4b86492ddf87208 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
W20260709 13:19:41.039113 16115 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:41.179495 18198 consensus_queue.cc:1048] T ada5a004ba7c46819051e3ccec71ffdb P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
W20260709 13:19:41.180775 16115 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:41.180348 18233 log.cc:927] Time spent T c9944d5e4b8c4109af9955c4f08c9095 P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.220s	user 0.000s	sys 0.000s
W20260709 13:19:41.181102 18229 log.cc:927] Time spent T 26354361e9b044cea1c2b86c4882fe92 P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.169s	user 0.000s	sys 0.000s
W20260709 13:19:41.181942 18226 log.cc:927] Time spent T 2794af5b99984efebdee564035841cb0 P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.240s	user 0.000s	sys 0.001s
I20260709 13:19:41.033188 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87147731a68400e8cce9b879daa7c5d" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:41.184070 16293 raft_consensus.cc:2368] T b87147731a68400e8cce9b879daa7c5d P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b for earlier term 4. Current term is 5.
I20260709 13:19:41.184438 18236 raft_consensus.cc:493] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:41.185017 18236 raft_consensus.cc:515] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:19:41.164966 16263 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:41.190595 16294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e4bfb9090c8460eb14fa7e0d818e7f1" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
W20260709 13:19:41.191598 16263 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:41.191558 16294 raft_consensus.cc:3060] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:19:41.192783 18181 raft_consensus.cc:515] T 1df93779209748b9860f6d893dd3de53 P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:41.228727 16298 raft_consensus.cc:1275] T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 2. (term mismatch)
I20260709 13:19:41.040417 17644 raft_consensus.cc:493] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:41.237835 17644 raft_consensus.cc:515] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:19:41.238714 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 200523us
I20260709 13:19:41.238301 16116 leader_election.cc:304] T 5a96373bb0e14dce8a266b2fc6c58507 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 90f3da57796b488c9087a2f523adf045
W20260709 13:19:41.259202 16263 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:41.179592 18168 log.cc:927] Time spent T d2ab0ac8347641e6955e9bce7d90c1a2 P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 2.064s	user 0.004s	sys 0.001s
I20260709 13:19:41.033706 17992 leader_election.cc:290] T af57996a8be84d398f0f023e8d08ef77 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 4 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:19:41.202926 17900 raft_consensus.cc:493] T 57a744f5aaf944678a63469cbf514015 P cb82cc7bd6db46d980c75237a1402860 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:41.330215 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e9b2bf7011f4ae688a79d7bb5a447b4" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:19:41.331490 16144 raft_consensus.cc:3060] T fffab26427e24d398e2695aad40ca134 P 647a343a542d4a139fe106c19095129b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:19:41.333434 16148 raft_consensus.cc:1275] T 2794af5b99984efebdee564035841cb0 P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:19:41.184479 17844 leader_election.cc:290] T 68a6838d82ef4cac83507f906139d90f P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:41.229800 16115 leader_election.cc:304] T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
W20260709 13:19:41.164340 18232 log.cc:927] Time spent T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 0.160s	user 0.000s	sys 0.000s
I20260709 13:19:41.229559 16298 pending_rounds.cc:85] T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045: Aborting all ops after (but not including) 1
I20260709 13:19:41.365015 16298 pending_rounds.cc:107] T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045: Aborting uncommitted NO_OP operation due to leader change: 3.2
I20260709 13:19:41.365825 16298 raft_consensus.cc:2889] T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:19:41.329064 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 90820.4us
I20260709 13:19:41.345155 16142 raft_consensus.cc:1240] T e5ad4eb281564936a59f41e387a47156 P 647a343a542d4a139fe106c19095129b [term 4 LEADER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 3. Current term is 4. Ops: []
I20260709 13:19:41.345311 18061 leader_election.cc:290] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:19:41.345947 16147 raft_consensus.cc:1240] T 1df93779209748b9860f6d893dd3de53 P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 3. Current term is 4. Ops: []
I20260709 13:19:41.346921 18240 raft_consensus.cc:2804] T 5a96373bb0e14dce8a266b2fc6c58507 P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:19:41.348882 16226 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:39144 (request call id 16) took 2921 ms. Trace:
I20260709 13:19:41.038645 18116 raft_consensus.cc:515] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:41.333326 16117 leader_election.cc:304] T 26354361e9b044cea1c2b86c4882fe92 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860
I20260709 13:19:41.356652 18236 leader_election.cc:290] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:41.357656 18189 leader_election.cc:290] T 66f75124215f4ca6bef831e174a792d8 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:19:41.326520 18238 raft_consensus.cc:493] T b6f07842d3aa47858df5665874c86f3a P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:19:41.327632 17999 raft_consensus.cc:2804] T d16dfad0c3b540e5b4b86492ddf87208 P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:19:41.323119 18114 raft_consensus.cc:493] T e4dc6b0541ca431ebc757beac00521f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:19:41.229225 18239 raft_consensus.cc:493] T ecc1e363230043fe9fef390f0b979f20 P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:41.363850 17844 raft_consensus.cc:493] T ee9ed67006144797b1de6ea8c2013a68 P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
W20260709 13:19:41.362366 16115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 132669us
I20260709 13:19:41.373085 18240 raft_consensus.cc:493] T 5a96373bb0e14dce8a266b2fc6c58507 P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:41.374127 18227 raft_consensus.cc:493] T ecc1e363230043fe9fef390f0b979f20 P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:41.370127 16226 rpcz_store.cc:276] 0709 13:19:38.462558 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:38.531475 (+ 68917us) service_pool.cc:224] Handling call
0709 13:19:38.675263 (+143788us) spinlock_profiling.cc:243] Waited 12.2 ms on lock 0x7b5800010b98. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69ad8fadb1 00007f69ada3ab71 00007f69ada3aa22 00007f69b6953c09 00007f69b5aadee0 00007f69b5b770ab 00007f69b5b77131 00007f69b5b3bc40 00007f69b5b27bb9 00007f69b5b2747c 00007f69af0ad7d4
0709 13:19:41.348599 (+2673336us) spinlock_profiling.cc:243] Waited 571 ms on lock 0x7b5400368154. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b59853 00007f69b5b2760a 00007f69af0ad7d4 00007f69af0ad762 00007f69af0ad691 00007f69af0ad60c 00007f69af0ac8c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:19:41.348763 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1341252864}
I20260709 13:19:41.228888 18123 leader_election.cc:290] T 18ad7e3af5504571a69d10c00750a01c P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:19:41.340893 18181 leader_election.cc:290] T 1df93779209748b9860f6d893dd3de53 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 4 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
W20260709 13:19:41.373297 16789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 367495us
I20260709 13:19:41.375430 16117 leader_election.cc:304] T 2794af5b99984efebdee564035841cb0 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860; no voters: 90f3da57796b488c9087a2f523adf045
I20260709 13:19:41.439095 16115 leader_election.cc:304] T 5a959dbecc2648eb85d1cc550f5a3245 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
W20260709 13:19:41.441112 16117 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 65791us
I20260709 13:19:41.175731 18149 raft_consensus.cc:515] T 746aabf856ab401c9ef5e859371e7b67 P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:41.355058 17900 raft_consensus.cc:3060] T 57a744f5aaf944678a63469cbf514015 P cb82cc7bd6db46d980c75237a1402860 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:19:41.323566 18190 consensus_queue.cc:1048] T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.039s
I20260709 13:19:41.379218 17644 leader_election.cc:290] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:41.448441 17644 raft_consensus.cc:2804] T 2794af5b99984efebdee564035841cb0 P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:19:41.338714 18199 consensus_queue.cc:1048] T 9e65676b21b748728ad39668cfe722ec P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.037s
I20260709 13:19:41.371057 18155 raft_consensus.cc:2804] T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:19:41.199491 18179 raft_consensus.cc:493] T ee9ed67006144797b1de6ea8c2013a68 P cb82cc7bd6db46d980c75237a1402860 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:19:41.111630 18093 raft_consensus.cc:515] T 19baa7bc367844c288bca8f81439c413 P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } }
I20260709 13:19:41.368991 18154 raft_consensus.cc:493] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:19:41.369616 18042 leader_election.cc:290] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 3 election: Requested vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:19:41.371068 16816 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49896 (request call id 777) took 1769 ms. Trace:
I20260709 13:19:41.109692 18109 consensus_queue.cc:237] T 8c11074579b046d99e585c2818e18a2c P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:19:41.450160 18237 log.cc:927] Time spent T 26354361e9b044cea1c2b86c4882fe92 P cb82cc7bd6db46d980c75237a1402860: Append to log took a long time: real 0.096s	user 0.000s	sys 0.000s
I20260709 13:19:41.438781 18105 raft_consensus.cc:493] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:41.444546 16789 leader_election.cc:304] T b87147731a68400e8cce9b879daa7c5d P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:19:41.374410 18061 raft_consensus.cc:493] T 434bff3082d141838068f323bdb0d1ed P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:19:41.489507 16823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746aabf856ab401c9ef5e859371e7b67" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:19:41.383715 18234 raft_consensus.cc:2749] T 26354361e9b044cea1c2b86c4882fe92 P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:19:41.382148 18046 raft_consensus.cc:493] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:19:41.385982 16294 raft_consensus.cc:2468] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 3.
I20260709 13:19:41.448979 17644 raft_consensus.cc:493] T 2794af5b99984efebdee564035841cb0 P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:41.503273 17644 raft_consensus.cc:3060] T 2794af5b99984efebdee564035841cb0 P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:19:41.367525 16116 leader_election.cc:304] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
I20260709 13:19:41.387674 17999 raft_consensus.cc:493] T d16dfad0c3b540e5b4b86492ddf87208 P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:19:41.392046 16143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18ad7e3af5504571a69d10c00750a01c" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:19:41.392194 18228 raft_consensus.cc:493] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:41.382256 18236 raft_consensus.cc:493] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:41.436970 18240 raft_consensus.cc:3060] T 5a96373bb0e14dce8a266b2fc6c58507 P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:19:41.485389 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 319552us
I20260709 13:19:41.485575 16825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5718c623ebee4d2bb770f0f9ddb2f164" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:19:41.441136 18063 raft_consensus.cc:493] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:19:41.496169 18042 raft_consensus.cc:493] T 66f75124215f4ca6bef831e174a792d8 P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:41.449596 18244 raft_consensus.cc:493] T 5a959dbecc2648eb85d1cc550f5a3245 P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:41.496860 16821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e4bfb9090c8460eb14fa7e0d818e7f1" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:19:41.396039 18114 raft_consensus.cc:515] T e4dc6b0541ca431ebc757beac00521f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:41.497423 16115 leader_election.cc:304] T 84f2da98864b4425bf7c0055951fdd4b P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:19:41.476230 18181 raft_consensus.cc:3060] T 5a959dbecc2648eb85d1cc550f5a3245 P 647a343a542d4a139fe106c19095129b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:19:41.500264 16822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74ca8140f35c4ae29e8a664f6bdfc829" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:19:41.387894 18238 raft_consensus.cc:515] T b6f07842d3aa47858df5665874c86f3a P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:19:41.489154 18167 consensus_meta.cc:287] T e5ad4eb281564936a59f41e387a47156 P 90f3da57796b488c9087a2f523adf045: Time spent flushing consensus metadata: real 0.673s	user 0.001s	sys 0.010s
I20260709 13:19:41.481650 18155 raft_consensus.cc:697] T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b [term 3 LEADER]: Becoming Leader. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:19:41.446412 17844 raft_consensus.cc:515] T ee9ed67006144797b1de6ea8c2013a68 P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:41.488422 17992 raft_consensus.cc:2804] T b87147731a68400e8cce9b879daa7c5d P cb82cc7bd6db46d980c75237a1402860 [term 5 FOLLOWER]: Leader election won for term 5
W20260709 13:19:41.508286 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 140861us
I20260709 13:19:41.550617 16116 leader_election.cc:304] T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
W20260709 13:19:41.509521 16117 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 66697us
I20260709 13:19:41.487884 16789 leader_election.cc:304] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 
W20260709 13:19:41.553134 16789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 65340.4us
I20260709 13:19:41.554733 18246 raft_consensus.cc:493] T e5ad4eb281564936a59f41e387a47156 P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:41.510423 16825 raft_consensus.cc:2468] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 3.
I20260709 13:19:41.555421 18246 raft_consensus.cc:515] T e5ad4eb281564936a59f41e387a47156 P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:41.556250 16825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2372981006d4b218215c28964dea623" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:19:41.556543 17844 leader_election.cc:290] T ee9ed67006144797b1de6ea8c2013a68 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:19:41.556951 16825 raft_consensus.cc:2393] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 4: Already voted for candidate cb82cc7bd6db46d980c75237a1402860 in this term.
I20260709 13:19:41.557049 17844 raft_consensus.cc:493] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:19:41.557541 17844 raft_consensus.cc:515] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:41.557821 18246 leader_election.cc:290] T e5ad4eb281564936a59f41e387a47156 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:41.558451 18246 raft_consensus.cc:493] T 8a878999c0024d2abe517484180290fc P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:19:41.560739 16820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e65676b21b748728ad39668cfe722ec" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:19:41.561317 16820 raft_consensus.cc:2468] T 9e65676b21b748728ad39668cfe722ec P cb82cc7bd6db46d980c75237a1402860 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 4.
I20260709 13:19:41.548894 18244 raft_consensus.cc:515] T 5a959dbecc2648eb85d1cc550f5a3245 P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:41.562557 17981 leader_election.cc:290] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 5 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:19:41.486449 18154 raft_consensus.cc:515] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } }
I20260709 13:19:41.507931 17999 raft_consensus.cc:3060] T d16dfad0c3b540e5b4b86492ddf87208 P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:19:41.482877 18179 raft_consensus.cc:515] T ee9ed67006144797b1de6ea8c2013a68 P cb82cc7bd6db46d980c75237a1402860 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:41.540623 16822 raft_consensus.cc:2468] T 74ca8140f35c4ae29e8a664f6bdfc829 P cb82cc7bd6db46d980c75237a1402860 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 3.
W20260709 13:19:41.543959 16115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 46610.7us
I20260709 13:19:41.568244 18154 leader_election.cc:290] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:19:41.568148 17900 raft_consensus.cc:515] T 57a744f5aaf944678a63469cbf514015 P cb82cc7bd6db46d980c75237a1402860 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:41.569269 16115 leader_election.cc:304] T d16dfad0c3b540e5b4b86492ddf87208 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:19:41.570104 16115 leader_election.cc:304] T 57a744f5aaf944678a63469cbf514015 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:19:41.499385 18041 leader_election.cc:290] T e4dc6b0541ca431ebc757beac00521f1 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:19:41.545069 16297 raft_consensus.cc:1275] T e5ad4eb281564936a59f41e387a47156 P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:19:41.434564 18239 raft_consensus.cc:515] T ecc1e363230043fe9fef390f0b979f20 P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:41.574478 16144 raft_consensus.cc:1275] T fffab26427e24d398e2695aad40ca134 P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:19:41.436103 18194 consensus_queue.cc:237] T 8d4ab870a05347868956aa8b841b2981 P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:41.554435 18155 consensus_queue.cc:237] T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:41.554625 16818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5d59407eb545b895d8d922e0dd69fd" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:19:41.625841 16818 raft_consensus.cc:2468] T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 4.
I20260709 13:19:41.446898 18227 raft_consensus.cc:515] T ecc1e363230043fe9fef390f0b979f20 P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } }
I20260709 13:19:41.628615 18227 leader_election.cc:290] T ecc1e363230043fe9fef390f0b979f20 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:19:41.540210 16821 raft_consensus.cc:3060] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P cb82cc7bd6db46d980c75237a1402860 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:19:41.565315 17644 raft_consensus.cc:515] T 2794af5b99984efebdee564035841cb0 P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:41.631232 17644 leader_election.cc:290] T 2794af5b99984efebdee564035841cb0 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 3 election: Requested vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:19:41.567852 18116 leader_election.cc:290] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:41.558990 18246 raft_consensus.cc:515] T 8a878999c0024d2abe517484180290fc P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:19:41.632387 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 81866.6us
I20260709 13:19:41.540850 18143 raft_consensus.cc:2804] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:19:41.570602 18179 leader_election.cc:290] T ee9ed67006144797b1de6ea8c2013a68 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:19:41.519536 18228 raft_consensus.cc:515] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:41.634652 17644 raft_consensus.cc:2804] T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:19:41.643654 17644 raft_consensus.cc:493] T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:41.568048 17844 leader_election.cc:290] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:19:41.497937 18061 raft_consensus.cc:515] T 434bff3082d141838068f323bdb0d1ed P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:41.564450 18234 raft_consensus.cc:2804] T 84f2da98864b4425bf7c0055951fdd4b P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:19:41.502657 18105 raft_consensus.cc:515] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:41.505504 18046 raft_consensus.cc:515] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:41.503106 18140 raft_consensus.cc:515] T 08c74490c40243a88e62d1760b5cd814 P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:41.644163 18116 raft_consensus.cc:493] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:41.570592 17900 leader_election.cc:290] T 57a744f5aaf944678a63469cbf514015 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 3 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:41.554636 18093 leader_election.cc:290] T 19baa7bc367844c288bca8f81439c413 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:41.643513 18161 raft_consensus.cc:493] T e5ad4eb281564936a59f41e387a47156 P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:41.644197 17644 raft_consensus.cc:3060] T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:19:41.482161 16816 rpcz_store.cc:276] 0709 13:19:39.601470 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:39.602184 (+   714us) service_pool.cc:224] Handling call
0709 13:19:39.609097 (+  6913us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:39.783187 (+174090us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:41.368751 (+1585564us) spinlock_profiling.cc:243] Waited 106 ms on lock 0x7b54007553d4. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:19:41.370993 (+  2242us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":243945728}
W20260709 13:19:41.658977 16115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 88952.3us
I20260709 13:19:41.638372 18173 raft_consensus.cc:2804] T d16dfad0c3b540e5b4b86492ddf87208 P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:19:41.522235 18236 raft_consensus.cc:515] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:41.760180 16822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3884f864a96d428785895e5463e5728f" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:19:41.640379 18143 raft_consensus.cc:493] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:41.635701 17992 raft_consensus.cc:697] T b87147731a68400e8cce9b879daa7c5d P cb82cc7bd6db46d980c75237a1402860 [term 5 LEADER]: Becoming Leader. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:19:41.545258 18063 raft_consensus.cc:515] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } }
I20260709 13:19:41.549926 18042 raft_consensus.cc:515] T 66f75124215f4ca6bef831e174a792d8 P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:41.636188 18253 raft_consensus.cc:2804] T 57a744f5aaf944678a63469cbf514015 P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:19:41.648201 18134 raft_consensus.cc:515] T c0eab25ce4c64e018a041beeda2867db P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:41.703233 16821 raft_consensus.cc:2468] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P cb82cc7bd6db46d980c75237a1402860 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 3.
I20260709 13:19:41.720853 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c11074579b046d99e585c2818e18a2c" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:41.770376 16192 leader_election.cc:304] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:19:41.657979 17844 raft_consensus.cc:493] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:41.725483 16222 raft_consensus.cc:1275] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 1ca2725d290e47d68b1a4b609c9816ac [term 6 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:19:41.655244 18250 raft_consensus.cc:493] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:41.655848 18234 raft_consensus.cc:493] T 84f2da98864b4425bf7c0055951fdd4b P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:41.759989 18093 raft_consensus.cc:493] T 08c74490c40243a88e62d1760b5cd814 P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:41.785982 16192 leader_election.cc:304] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 90f3da57796b488c9087a2f523adf045
I20260709 13:19:41.761396 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "829b4ad8c5a54eababeea1181b960d4b" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:19:41.790872 16266 leader_election.cc:400] T c9944d5e4b8c4109af9955c4f08c9095 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 4 election: Vote denied by peer cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923) with higher term. Message: Invalid argument: T c9944d5e4b8c4109af9955c4f08c9095 P cb82cc7bd6db46d980c75237a1402860 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 for earlier term 4. Current term is 5.
I20260709 13:19:41.791070 16225 raft_consensus.cc:3060] T 829b4ad8c5a54eababeea1181b960d4b P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:19:41.785249 16223 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "806d7690c2b644959152074bb2cab0a7"
dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:57598
I20260709 13:19:41.791486 16222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9944d5e4b8c4109af9955c4f08c9095" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:19:41.791970 16223 raft_consensus.cc:606] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Received request to transfer leadership
I20260709 13:19:41.792088 16222 raft_consensus.cc:3060] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:19:41.735445 18256 raft_consensus.cc:493] T 1df93779209748b9860f6d893dd3de53 P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:41.725116 18116 raft_consensus.cc:515] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:41.724262 18249 raft_consensus.cc:493] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:19:41.771067 16226 raft_consensus.cc:1240] T 7c2095a5f4ad49408dbc7f69c606120d P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 2. Current term is 3. Ops: []
I20260709 13:19:41.764520 18254 raft_consensus.cc:493] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:41.732462 16818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57a744f5aaf944678a63469cbf514015" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:19:41.736178 18173 raft_consensus.cc:493] T d16dfad0c3b540e5b4b86492ddf87208 P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:41.786275 17644 raft_consensus.cc:515] T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:19:41.788883 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 147645us
I20260709 13:19:41.733685 16115 leader_election.cc:304] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
W20260709 13:19:41.814289 16115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 80725.4us
I20260709 13:19:41.815368 16115 leader_election.cc:304] T 3884f864a96d428785895e5463e5728f P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:19:41.812930 16225 raft_consensus.cc:2468] T 829b4ad8c5a54eababeea1181b960d4b P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 5.
I20260709 13:19:41.816273 16115 leader_election.cc:304] T fffab26427e24d398e2695aad40ca134 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:19:41.763516 18143 raft_consensus.cc:3060] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:19:41.816924 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74ca8140f35c4ae29e8a664f6bdfc829" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:41.817747 16225 raft_consensus.cc:2468] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 2.
I20260709 13:19:41.819186 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66f75124215f4ca6bef831e174a792d8" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:41.819919 16225 raft_consensus.cc:2468] T 66f75124215f4ca6bef831e174a792d8 P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 4.
I20260709 13:19:41.821033 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1df93779209748b9860f6d893dd3de53" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:19:41.792258 16223 raft_consensus.cc:614] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 13:19:41.762862 16263 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:41.795804 16821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87147731a68400e8cce9b879daa7c5d" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:19:41.760389 18227 raft_consensus.cc:493] T ada5a004ba7c46819051e3ccec71ffdb P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:41.826228 17644 leader_election.cc:290] T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 4 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:19:41.784395 18253 raft_consensus.cc:697] T 57a744f5aaf944678a63469cbf514015 P 647a343a542d4a139fe106c19095129b [term 3 LEADER]: Becoming Leader. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:19:41.789630 18255 raft_consensus.cc:493] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:19:41.774257 16823 raft_consensus.cc:2468] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 3.
I20260709 13:19:41.733619 18161 raft_consensus.cc:515] T e5ad4eb281564936a59f41e387a47156 P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:41.801929 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b7d1ab519044eda2d5e04a07242f25" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:41.771070 18149 leader_election.cc:290] T 746aabf856ab401c9ef5e859371e7b67 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 4 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
W20260709 13:19:41.806151 18242 log.cc:927] Time spent T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 0.309s	user 0.001s	sys 0.000s
I20260709 13:19:41.783919 16221 raft_consensus.cc:1275] T c2f5f89186204242bf377f7c42b2e2da P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:19:41.786031 18234 raft_consensus.cc:3060] T 84f2da98864b4425bf7c0055951fdd4b P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:19:41.805226 18145 consensus_meta.cc:287] T 8cc959fb7f704fba9009c0595467d074 P 647a343a542d4a139fe106c19095129b: Time spent flushing consensus metadata: real 0.791s	user 0.009s	sys 0.001s
I20260709 13:19:41.825309 16116 leader_election.cc:304] T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
I20260709 13:19:41.811555 16222 raft_consensus.cc:2468] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 5.
I20260709 13:19:41.828436 16191 leader_election.cc:304] T 8cc959fb7f704fba9009c0595467d074 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 90f3da57796b488c9087a2f523adf045
I20260709 13:19:41.892980 16223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af57996a8be84d398f0f023e8d08ef77" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:19:41.895946 16223 raft_consensus.cc:3060] T af57996a8be84d398f0f023e8d08ef77 P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:19:41.800107 16266 leader_election.cc:304] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:19:41.896126 16222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2372981006d4b218215c28964dea623" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:19:41.896778 16222 raft_consensus.cc:3060] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:19:41.829466 18245 raft_consensus.cc:2764] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P cb82cc7bd6db46d980c75237a1402860 [term 3 FOLLOWER]: Leader pre-election decision vote started in defunct term 2: won
I20260709 13:19:41.830140 16295 raft_consensus.cc:1275] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:19:41.900552 16295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57a744f5aaf944678a63469cbf514015" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:19:41.900979 18241 raft_consensus.cc:493] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:41.901299 16295 raft_consensus.cc:2393] T 57a744f5aaf944678a63469cbf514015 P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 in current term 3: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
I20260709 13:19:41.901597 18241 raft_consensus.cc:515] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:41.902292 16295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74ca8140f35c4ae29e8a664f6bdfc829" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:41.903904 16222 raft_consensus.cc:2468] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 4.
I20260709 13:19:41.773200 17844 raft_consensus.cc:515] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:41.834609 16822 raft_consensus.cc:1275] T 9e65676b21b748728ad39668cfe722ec P cb82cc7bd6db46d980c75237a1402860 [term 4 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260709 13:19:41.837046 17644 raft_consensus.cc:2804] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:19:41.773274 18243 consensus_queue.cc:1048] T c0eab25ce4c64e018a041beeda2867db P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.045s
I20260709 13:19:41.822106 16817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66f75124215f4ca6bef831e174a792d8" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:19:41.840217 16818 raft_consensus.cc:2393] T 57a744f5aaf944678a63469cbf514015 P cb82cc7bd6db46d980c75237a1402860 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 3: Already voted for candidate cb82cc7bd6db46d980c75237a1402860 in this term.
I20260709 13:19:41.807075 16219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5718c623ebee4d2bb770f0f9ddb2f164" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
W20260709 13:19:41.929695 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 104499us
W20260709 13:19:41.846990 16192 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 61091.8us
I20260709 13:19:41.806798 16816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e9b2bf7011f4ae688a79d7bb5a447b4" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:19:41.822638 18173 raft_consensus.cc:3060] T d16dfad0c3b540e5b4b86492ddf87208 P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:19:41.808465 18257 raft_consensus.cc:2804] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:19:41.829530 16294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66ae176961b74813a58bc15c9c4c6f2a" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
W20260709 13:19:41.904991 16266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 104969us
I20260709 13:19:41.802917 18093 raft_consensus.cc:515] T 08c74490c40243a88e62d1760b5cd814 P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:19:41.904662 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 76342.1us
I20260709 13:19:41.838778 18227 raft_consensus.cc:515] T ada5a004ba7c46819051e3ccec71ffdb P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:41.907835 17644 raft_consensus.cc:493] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Starting leader election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:19:41.903905 18241 leader_election.cc:290] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:41.947266 17644 raft_consensus.cc:3060] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:19:41.823531 18254 raft_consensus.cc:515] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:19:41.907907 16115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 91709.8us
I20260709 13:19:41.902935 16295 raft_consensus.cc:2468] T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 3.
I20260709 13:19:41.805056 18256 raft_consensus.cc:515] T 1df93779209748b9860f6d893dd3de53 P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:41.791340 18250 raft_consensus.cc:515] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:41.925171 18143 raft_consensus.cc:515] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:41.929495 18217 raft_consensus.cc:493] T 66ae176961b74813a58bc15c9c4c6f2a P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:19:41.933058 16817 raft_consensus.cc:2468] T 66f75124215f4ca6bef831e174a792d8 P cb82cc7bd6db46d980c75237a1402860 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 4.
I20260709 13:19:41.929186 18154 raft_consensus.cc:493] T c4f65cdf00b748bb963ee6e81ef43b79 P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:41.936116 18199 raft_consensus.cc:2764] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Leader pre-election decision vote started in defunct term 2: won
I20260709 13:19:41.939711 18262 raft_consensus.cc:2804] T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:19:41.929112 18189 consensus_queue.cc:1048] T 3a7ca61592e848ea9b0fb81a2ce33cdc P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.011s
I20260709 13:19:41.907593 18167 raft_consensus.cc:493] T 31969392ccdb43b491911a319e8bc7e1 P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:41.905089 18149 raft_consensus.cc:2749] T fffab26427e24d398e2695aad40ca134 P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:19:41.906605 16222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66ae176961b74813a58bc15c9c4c6f2a" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:41.946852 16223 raft_consensus.cc:2468] T af57996a8be84d398f0f023e8d08ef77 P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 4.
I20260709 13:19:41.985077 16222 raft_consensus.cc:2468] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 5.
I20260709 13:19:41.948753 18271 raft_consensus.cc:2804] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 13:19:41.949150 16300 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48684 (request call id 1901) took 1251 ms (client timeout 1627 ms). Trace:
I20260709 13:19:41.928503 16220 raft_consensus.cc:1275] T 8cc959fb7f704fba9009c0595467d074 P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:19:41.954684 16115 leader_election.cc:400] T b87147731a68400e8cce9b879daa7c5d P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 4 pre-election: Vote denied by peer 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389) with higher term. Message: Invalid argument: T b87147731a68400e8cce9b879daa7c5d P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b for earlier term 4. Current term is 5.
I20260709 13:19:41.990078 16115 leader_election.cc:403] T b87147731a68400e8cce9b879daa7c5d P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 4 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:19:41.958657 18061 leader_election.cc:290] T 434bff3082d141838068f323bdb0d1ed P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:19:41.991082 16222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2372981006d4b218215c28964dea623" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:19:41.991832 16222 raft_consensus.cc:2393] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 4: Already voted for candidate cb82cc7bd6db46d980c75237a1402860 in this term.
I20260709 13:19:41.992614 16222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c74490c40243a88e62d1760b5cd814" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:19:41.992983 16222 raft_consensus.cc:3060] T 08c74490c40243a88e62d1760b5cd814 P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:19:41.938071 18123 raft_consensus.cc:493] T 23f82224f4564fea93572e8206bad1fe P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:19:41.907996 18145 raft_consensus.cc:515] T 8cc959fb7f704fba9009c0595467d074 P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:41.941315 18257 raft_consensus.cc:493] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:41.937567 16291 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40474 (request call id 303) took 1151 ms. Trace:
I20260709 13:19:41.934211 18225 raft_consensus.cc:493] T 8638ee363273426db1393dd81e5c7b72 P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:19:41.855369 17992 consensus_queue.cc:237] T b87147731a68400e8cce9b879daa7c5d P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:41.965588 16225 raft_consensus.cc:3060] T 1df93779209748b9860f6d893dd3de53 P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:19:41.971086 18264 raft_consensus.cc:2804] T 3884f864a96d428785895e5463e5728f P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Leader election won for term 3
W20260709 13:19:41.979241 18258 log.cc:927] Time spent T 8d4ab870a05347868956aa8b841b2981 P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 0.062s	user 0.000s	sys 0.000s
I20260709 13:19:41.953501 18041 raft_consensus.cc:493] T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:41.970878 17644 raft_consensus.cc:515] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:41.982911 16295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a9de347c711477e87c582d003f36925" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:41.807086 18249 raft_consensus.cc:515] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:41.957954 18238 leader_election.cc:290] T b6f07842d3aa47858df5665874c86f3a P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:41.957497 16217 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57604 (request call id 1939) took 2006 ms. Trace:
I20260709 13:19:41.981973 18262 raft_consensus.cc:493] T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:41.957201 18246 leader_election.cc:290] T 8a878999c0024d2abe517484180290fc P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
W20260709 13:19:41.988525 16300 rpcz_store.cc:269] 0709 13:19:40.697490 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:40.697631 (+   141us) service_pool.cc:224] Handling call
0709 13:19:41.948923 (+1251292us) spinlock_profiling.cc:243] Waited 71.3 ms on lock 0x7b540024c554. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:19:41.949070 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":163890048}
I20260709 13:19:41.889855 18255 raft_consensus.cc:515] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:41.931734 16221 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57600 (request call id 1012) took 1861 ms. Trace:
I20260709 13:19:41.991645 16223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f5f89186204242bf377f7c42b2e2da" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:41.931087 16192 leader_election.cc:304] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:19:41.995446 18265 raft_consensus.cc:493] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 1ca2725d290e47d68b1a4b609c9816ac [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:41.963434 18046 leader_election.cc:290] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:19:41.965498 18140 leader_election.cc:290] T 08c74490c40243a88e62d1760b5cd814 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 5 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:42.000051 16148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d16dfad0c3b540e5b4b86492ddf87208" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:19:42.000540 16145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a96373bb0e14dce8a266b2fc6c58507" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:19:42.003567 18272 raft_consensus.cc:2804] T 8cc959fb7f704fba9009c0595467d074 P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:19:42.002254 18271 raft_consensus.cc:493] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Starting leader election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:19:42.004142 18275 raft_consensus.cc:493] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:42.008529 16296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e9b2bf7011f4ae688a79d7bb5a447b4" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:42.013370 16225 raft_consensus.cc:2468] T 1df93779209748b9860f6d893dd3de53 P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 4.
I20260709 13:19:42.014820 16146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746aabf856ab401c9ef5e859371e7b67" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:19:41.984140 16293 raft_consensus.cc:1275] T 18ad7e3af5504571a69d10c00750a01c P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:19:42.020289 16295 raft_consensus.cc:2468] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 3.
W20260709 13:19:42.021118 16266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 79389.6us
I20260709 13:19:41.979806 18260 consensus_queue.cc:1048] T c0eab25ce4c64e018a041beeda2867db P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.057s
I20260709 13:19:41.980232 18261 raft_consensus.cc:493] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:19:42.031482 16151 raft_consensus.cc:1240] T 8cc959fb7f704fba9009c0595467d074 P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 3. Current term is 4. Ops: []
I20260709 13:19:41.964874 18217 raft_consensus.cc:515] T 66ae176961b74813a58bc15c9c4c6f2a P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:41.987299 18270 raft_consensus.cc:493] T ecc1e363230043fe9fef390f0b979f20 P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:41.972697 18154 raft_consensus.cc:515] T c4f65cdf00b748bb963ee6e81ef43b79 P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:41.960500 18253 consensus_queue.cc:237] T 57a744f5aaf944678a63469cbf514015 P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:42.016258 18225 raft_consensus.cc:515] T 8638ee363273426db1393dd81e5c7b72 P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:42.060279 18063 leader_election.cc:290] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:42.018890 18236 leader_election.cc:290] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:19:41.987843 18167 raft_consensus.cc:515] T 31969392ccdb43b491911a319e8bc7e1 P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:42.002806 18105 leader_election.cc:290] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:19:42.214365 17644 leader_election.cc:290] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 4 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:42.032459 16147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c2095a5f4ad49408dbc7f69c606120d" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:19:42.066237 16222 raft_consensus.cc:2468] T 08c74490c40243a88e62d1760b5cd814 P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 5.
I20260709 13:19:42.074987 16226 raft_consensus.cc:2410] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 for term 4 because replica has last-logged OpId of term: 3 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:19:42.069424 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31969392ccdb43b491911a319e8bc7e1" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:42.218117 16146 raft_consensus.cc:2393] T 746aabf856ab401c9ef5e859371e7b67 P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 4: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
I20260709 13:19:42.071080 16223 raft_consensus.cc:2468] T c2f5f89186204242bf377f7c42b2e2da P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 5.
W20260709 13:19:42.071476 16192 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 140497us
I20260709 13:19:41.997884 18257 raft_consensus.cc:3060] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:19:42.001986 18123 raft_consensus.cc:515] T 23f82224f4564fea93572e8206bad1fe P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:42.060281 18262 raft_consensus.cc:3060] T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:19:42.284333 16116 leader_election.cc:304] T 08c74490c40243a88e62d1760b5cd814 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
I20260709 13:19:42.284474 16147 raft_consensus.cc:3060] T 7c2095a5f4ad49408dbc7f69c606120d P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:19:42.066398 18239 leader_election.cc:290] T ecc1e363230043fe9fef390f0b979f20 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:19:42.285327 16116 leader_election.cc:304] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
I20260709 13:19:42.286352 16823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee9ed67006144797b1de6ea8c2013a68" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:19:42.031925 16217 rpcz_store.cc:276] 0709 13:19:39.950629 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:39.965016 (+ 14387us) service_pool.cc:224] Handling call
0709 13:19:41.951153 (+1986137us) spinlock_profiling.cc:243] Waited 687 ms on lock 0x7b5400396554. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:19:41.955339 (+  4186us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1579985152}
I20260709 13:19:42.287303 16823 raft_consensus.cc:2468] T ee9ed67006144797b1de6ea8c2013a68 P cb82cc7bd6db46d980c75237a1402860 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 3.
I20260709 13:19:42.287837 16222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746aabf856ab401c9ef5e859371e7b67" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:19:42.287998 16817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8638ee363273426db1393dd81e5c7b72" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:19:42.288546 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66f75124215f4ca6bef831e174a792d8" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:42.288547 16222 raft_consensus.cc:3060] T 746aabf856ab401c9ef5e859371e7b67 P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:19:42.059958 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0eab25ce4c64e018a041beeda2867db" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:19:42.032998 18181 raft_consensus.cc:2749] T 5a959dbecc2648eb85d1cc550f5a3245 P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:19:42.021130 18273 raft_consensus.cc:493] T fffab26427e24d398e2695aad40ca134 P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:42.286692 16223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6f07842d3aa47858df5665874c86f3a" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
W20260709 13:19:42.305444 18269 log.cc:927] Time spent T 434bff3082d141838068f323bdb0d1ed P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 0.234s	user 0.000s	sys 0.000s
I20260709 13:19:42.285867 16146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f5f89186204242bf377f7c42b2e2da" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
W20260709 13:19:42.143438 18263 log.cc:927] Time spent T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860: Append to log took a long time: real 0.083s	user 0.000s	sys 0.000s
I20260709 13:19:42.066551 18267 raft_consensus.cc:3060] T b87147731a68400e8cce9b879daa7c5d P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Advancing to term 5
I20260709 13:19:42.287190 18239 raft_consensus.cc:2804] T 08c74490c40243a88e62d1760b5cd814 P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:19:42.296710 16192 leader_election.cc:304] T 746aabf856ab401c9ef5e859371e7b67 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:19:42.219054 18046 raft_consensus.cc:493] T a378d7f796f94615b3b347119a9880fd P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:19:42.316388 16300 raft_consensus.cc:2468] T 31969392ccdb43b491911a319e8bc7e1 P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 3.
W20260709 13:19:42.213697 18251 log.cc:927] Time spent T 02838c86531d4f98bf58d1c22b8baf77 P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.237s	user 0.000s	sys 0.001s
I20260709 13:19:42.290279 16295 raft_consensus.cc:1240] T 3884f864a96d428785895e5463e5728f P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 2. Current term is 3. Ops: []
I20260709 13:19:42.212809 18249 leader_election.cc:290] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:19:42.144866 18246 raft_consensus.cc:2804] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:19:42.314005 16823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b7d1ab519044eda2d5e04a07242f25" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:19:42.031553 18041 raft_consensus.cc:515] T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:42.123420 17844 leader_election.cc:290] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:19:42.066982 16220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62fe3346cd3c42c4a55e14af9073bca4" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:42.220225 18116 leader_election.cc:290] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:41.997991 16291 rpcz_store.cc:276] 0709 13:19:40.781557 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:40.786611 (+  5054us) service_pool.cc:224] Handling call
0709 13:19:41.909010 (+1122399us) spinlock_profiling.cc:243] Waited 128 ms on lock 0x7b540010f254. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:19:41.933250 (+ 24240us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":294723328}
I20260709 13:19:42.220161 18265 raft_consensus.cc:515] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 1ca2725d290e47d68b1a4b609c9816ac [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:42.071638 18264 raft_consensus.cc:697] T 3884f864a96d428785895e5463e5728f P 647a343a542d4a139fe106c19095129b [term 3 LEADER]: Becoming Leader. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:19:42.297817 18140 raft_consensus.cc:2804] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:19:42.221267 18271 raft_consensus.cc:3060] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:19:42.317561 16192 leader_election.cc:304] T ee9ed67006144797b1de6ea8c2013a68 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:19:42.297472 18261 raft_consensus.cc:515] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:42.333868 18254 leader_election.cc:290] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:19:42.335716 16147 raft_consensus.cc:2468] T 7c2095a5f4ad49408dbc7f69c606120d P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 3.
W20260709 13:19:42.344601 18231 log.cc:927] Time spent T 8158bbd17b994d2eb96f560bd0f1b88d P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.970s	user 0.000s	sys 0.004s
I20260709 13:19:42.311381 16294 raft_consensus.cc:2468] T 66ae176961b74813a58bc15c9c4c6f2a P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 4.
I20260709 13:19:42.347853 16144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4dc6b0541ca431ebc757beac00521f1" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:19:42.337077 18239 raft_consensus.cc:697] T 08c74490c40243a88e62d1760b5cd814 P 647a343a542d4a139fe106c19095129b [term 5 LEADER]: Becoming Leader. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:19:42.333266 18046 raft_consensus.cc:515] T a378d7f796f94615b3b347119a9880fd P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:42.346060 16222 raft_consensus.cc:2468] T 746aabf856ab401c9ef5e859371e7b67 P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 4.
I20260709 13:19:42.353462 16823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62fe3346cd3c42c4a55e14af9073bca4" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:19:42.306763 16217 raft_consensus.cc:2468] T 66f75124215f4ca6bef831e174a792d8 P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 4.
I20260709 13:19:42.070755 16221 rpcz_store.cc:276] 0709 13:19:40.067373 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:41.648132 (+1580759us) service_pool.cc:224] Handling call
0709 13:19:41.730769 (+ 82637us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:41.853697 (+122928us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:41.928288 (+ 74591us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:42.303408 18270 raft_consensus.cc:515] T ecc1e363230043fe9fef390f0b979f20 P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:42.310398 18273 raft_consensus.cc:515] T fffab26427e24d398e2695aad40ca134 P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:42.310508 18275 raft_consensus.cc:515] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:42.346830 16817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66ae176961b74813a58bc15c9c4c6f2a" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:19:42.346827 18246 raft_consensus.cc:697] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 3 LEADER]: Becoming Leader. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:19:42.349502 18143 leader_election.cc:290] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 5 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:19:42.349309 18227 leader_election.cc:290] T ada5a004ba7c46819051e3ccec71ffdb P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
W20260709 13:19:42.489169 16115 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:42.352270 17844 raft_consensus.cc:2804] T ee9ed67006144797b1de6ea8c2013a68 P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:19:42.488907 16823 raft_consensus.cc:2468] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 5.
I20260709 13:19:42.382450 16144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0eab25ce4c64e018a041beeda2867db" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:19:42.349118 18140 raft_consensus.cc:493] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:42.493311 16192 leader_election.cc:304] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:19:42.490350 16116 leader_election.cc:304] T 746aabf856ab401c9ef5e859371e7b67 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
W20260709 13:19:42.492004 16115 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:42.349193 16220 raft_consensus.cc:2468] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 5.
I20260709 13:19:42.351935 18093 leader_election.cc:290] T 08c74490c40243a88e62d1760b5cd814 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:42.494889 16222 raft_consensus.cc:1275] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:19:42.496222 18105 raft_consensus.cc:2804] T 746aabf856ab401c9ef5e859371e7b67 P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:19:42.496738 18105 raft_consensus.cc:697] T 746aabf856ab401c9ef5e859371e7b67 P 647a343a542d4a139fe106c19095129b [term 4 LEADER]: Becoming Leader. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:19:42.496990 16144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8158bbd17b994d2eb96f560bd0f1b88d" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:19:42.497900 16144 raft_consensus.cc:2468] T 8158bbd17b994d2eb96f560bd0f1b88d P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 4.
I20260709 13:19:42.497653 18105 consensus_queue.cc:237] T 746aabf856ab401c9ef5e859371e7b67 P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:42.498883 16144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a959dbecc2648eb85d1cc550f5a3245" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:19:42.499536 16144 raft_consensus.cc:2468] T 5a959dbecc2648eb85d1cc550f5a3245 P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 2.
W20260709 13:19:42.469769 18096 consensus_meta.cc:287] T d2ab0ac8347641e6955e9bce7d90c1a2 P 1ca2725d290e47d68b1a4b609c9816ac: Time spent flushing consensus metadata: real 1.434s	user 0.012s	sys 0.000s
I20260709 13:19:42.500425 16144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee9ed67006144797b1de6ea8c2013a68" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
W20260709 13:19:42.413295 17999 consensus_meta.cc:287] T d16dfad0c3b540e5b4b86492ddf87208 P 90f3da57796b488c9087a2f523adf045: Time spent flushing consensus metadata: real 0.772s	user 0.008s	sys 0.004s
I20260709 13:19:42.503199 18265 leader_election.cc:290] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:19:42.491112 18109 raft_consensus.cc:2764] T 746aabf856ab401c9ef5e859371e7b67 P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Leader pre-election decision vote started in defunct term 3: won
W20260709 13:19:42.504724 18274 log.cc:927] Time spent T 8c11074579b046d99e585c2818e18a2c P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 0.102s	user 0.000s	sys 0.000s
W20260709 13:19:42.505244 18240 consensus_meta.cc:287] T 5a96373bb0e14dce8a266b2fc6c58507 P 647a343a542d4a139fe106c19095129b: Time spent flushing consensus metadata: real 0.965s	user 0.007s	sys 0.004s
I20260709 13:19:42.509089 18240 raft_consensus.cc:515] T 5a96373bb0e14dce8a266b2fc6c58507 P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:42.509550 18254 raft_consensus.cc:2804] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:19:42.489917 18145 leader_election.cc:290] T 8cc959fb7f704fba9009c0595467d074 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 4 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:42.511680 16145 raft_consensus.cc:2393] T 5a96373bb0e14dce8a266b2fc6c58507 P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 3: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
I20260709 13:19:42.413334 16818 raft_consensus.cc:3060] T 8c11074579b046d99e585c2818e18a2c P cb82cc7bd6db46d980c75237a1402860 [term 3 FOLLOWER]: Advancing to term 5
I20260709 13:19:42.481460 18181 raft_consensus.cc:493] T af57996a8be84d398f0f023e8d08ef77 P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:42.421819 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a878999c0024d2abe517484180290fc" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:19:42.493607 18167 leader_election.cc:290] T 31969392ccdb43b491911a319e8bc7e1 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:19:42.416159 18225 leader_election.cc:290] T 8638ee363273426db1393dd81e5c7b72 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:19:42.417369 18199 raft_consensus.cc:493] T 8a878999c0024d2abe517484180290fc P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:19:42.515317 16144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5718c623ebee4d2bb770f0f9ddb2f164" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:19:42.494342 17844 raft_consensus.cc:493] T ee9ed67006144797b1de6ea8c2013a68 P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Starting leader election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:19:42.514021 16823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7ca61592e848ea9b0fb81a2ce33cdc" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:19:42.501678 18140 raft_consensus.cc:3060] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:19:42.516884 18154 leader_election.cc:290] T c4f65cdf00b748bb963ee6e81ef43b79 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:19:42.494889 16817 raft_consensus.cc:2468] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 5.
I20260709 13:19:42.483242 18276 consensus_queue.cc:1048] T 9e65676b21b748728ad39668cfe722ec P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.294s
I20260709 13:19:42.418196 18173 raft_consensus.cc:515] T d16dfad0c3b540e5b4b86492ddf87208 P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:42.524698 16116 leader_election.cc:304] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
I20260709 13:19:42.525377 16144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806d7690c2b644959152074bb2cab0a7" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:19:42.521286 16292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c2095a5f4ad49408dbc7f69c606120d" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:42.528741 16824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada5a004ba7c46819051e3ccec71ffdb" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:19:42.528954 18261 leader_election.cc:290] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
W20260709 13:19:42.529147 16292 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48684 (request call id 1900) took 1834 ms (client timeout 2246 ms). Trace:
I20260709 13:19:42.530483 16144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68a6838d82ef4cac83507f906139d90f" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
W20260709 13:19:42.529561 16292 rpcz_store.cc:269] 0709 13:19:40.694618 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:40.700246 (+  5628us) service_pool.cc:224] Handling call
0709 13:19:42.519712 (+1819466us) spinlock_profiling.cc:243] Waited 1.16 ms on lock 0x7b5800090098. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69ad8fadb1 00007f69ada3af8c 00007f69ada3aef6 0000000000375939 00007f69b5aae1e0 00007f69b5b770c4 00007f69b5b77131 00007f69b5b250a0 00007f69b5b24984 00007f69af09f6d4 00007f69af09f662
0709 13:19:42.529085 (+  9373us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2677632}
I20260709 13:19:42.533404 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5ad4eb281564936a59f41e387a47156" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:19:42.529755 16816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a378d7f796f94615b3b347119a9880fd" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:19:42.534521 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4f65cdf00b748bb963ee6e81ef43b79" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 7 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:19:42.536409 16149 raft_consensus.cc:1240] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 3. Current term is 4. Ops: []
I20260709 13:19:42.522948 18255 leader_election.cc:290] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:19:42.530169 18145 raft_consensus.cc:2804] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:19:42.541626 18145 raft_consensus.cc:493] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:42.511339 18096 raft_consensus.cc:515] T d2ab0ac8347641e6955e9bce7d90c1a2 P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:42.517306 18239 consensus_queue.cc:237] T 08c74490c40243a88e62d1760b5cd814 P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:42.544610 16149 raft_consensus.cc:1275] T 9c61a39fd4f84424b129c1a6f9ba404d P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 2. (index mismatch)
I20260709 13:19:42.540231 16192 leader_election.cc:304] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:19:42.522837 18034 leader_election.cc:290] T 26354361e9b044cea1c2b86c4882fe92 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:42.538813 18271 raft_consensus.cc:515] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:42.523936 17844 raft_consensus.cc:3060] T ee9ed67006144797b1de6ea8c2013a68 P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:19:42.513759 17999 raft_consensus.cc:515] T d16dfad0c3b540e5b4b86492ddf87208 P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:42.543121 16148 raft_consensus.cc:2393] T d16dfad0c3b540e5b4b86492ddf87208 P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 3: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
I20260709 13:19:42.519770 18250 leader_election.cc:290] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:19:42.547529 16824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdbd78acb4cf46298e0de8ba0d130e19" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:19:42.548552 18267 raft_consensus.cc:2749] T b87147731a68400e8cce9b879daa7c5d P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Leader pre-election lost for term 4. Reason: Vote denied by peer 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389) with higher term. Message: Invalid argument: T b87147731a68400e8cce9b879daa7c5d P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b for earlier term 4. Current term is 5.
I20260709 13:19:42.551920 18272 raft_consensus.cc:493] T 8cc959fb7f704fba9009c0595467d074 P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:42.559428 18161 leader_election.cc:290] T e5ad4eb281564936a59f41e387a47156 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:19:42.560050 18096 leader_election.cc:290] T d2ab0ac8347641e6955e9bce7d90c1a2 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 5 election: Requested vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:19:42.561573 18173 leader_election.cc:290] T d16dfad0c3b540e5b4b86492ddf87208 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 3 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:42.541991 18145 raft_consensus.cc:3060] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:19:42.563738 18234 consensus_meta.cc:287] T 84f2da98864b4425bf7c0055951fdd4b P 647a343a542d4a139fe106c19095129b: Time spent flushing consensus metadata: real 0.656s	user 0.009s	sys 0.000s
I20260709 13:19:42.564644 18262 raft_consensus.cc:515] T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:42.418548 18264 consensus_queue.cc:237] T 3884f864a96d428785895e5463e5728f P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:42.524701 18199 raft_consensus.cc:515] T 8a878999c0024d2abe517484180290fc P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:42.518889 18181 raft_consensus.cc:515] T af57996a8be84d398f0f023e8d08ef77 P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:42.532501 16144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b7d1ab519044eda2d5e04a07242f25" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:19:42.773062 16821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2372981006d4b218215c28964dea623" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
W20260709 13:19:42.678782 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 1.16648e+06us
I20260709 13:19:42.717056 16145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f82224f4564fea93572e8206bad1fe" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:19:42.681598 16265 leader_election.cc:304] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
W20260709 13:19:42.785161 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 103646us
W20260709 13:19:42.804768 16263 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:42.791211 16790 leader_election.cc:304] T 8c11074579b046d99e585c2818e18a2c P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
W20260709 13:19:42.804949 16192 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 264772us
I20260709 13:19:42.790958 16821 raft_consensus.cc:2393] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 4: Already voted for candidate cb82cc7bd6db46d980c75237a1402860 in this term.
I20260709 13:19:42.808444 16145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b7d1ab519044eda2d5e04a07242f25" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:19:42.789245 16789 leader_election.cc:304] T 74ca8140f35c4ae29e8a664f6bdfc829 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:19:42.810290 16145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e65676b21b748728ad39668cfe722ec" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:19:42.811036 16145 raft_consensus.cc:2468] T 9e65676b21b748728ad39668cfe722ec P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 4.
W20260709 13:19:42.814286 16117 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.14.177.132:38923 timed out after 1.505s (SENT) [suppressed 7 similar messages]
I20260709 13:19:42.815034 16822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746aabf856ab401c9ef5e859371e7b67" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:19:42.816543 16822 raft_consensus.cc:3060] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:19:42.781443 18272 raft_consensus.cc:3060] T 8cc959fb7f704fba9009c0595467d074 P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:19:42.714298 16818 raft_consensus.cc:1275] T 8c11074579b046d99e585c2818e18a2c P cb82cc7bd6db46d980c75237a1402860 [term 5 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:19:42.714463 18225 raft_consensus.cc:493] T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:42.599555 18156 raft_consensus.cc:493] T 8d4ab870a05347868956aa8b841b2981 P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:42.715365 18254 raft_consensus.cc:493] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:42.568312 18217 leader_election.cc:290] T 66ae176961b74813a58bc15c9c4c6f2a P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:19:42.572289 18273 leader_election.cc:290] T fffab26427e24d398e2695aad40ca134 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:42.836158 18161 raft_consensus.cc:493] T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:42.683368 16142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2794af5b99984efebdee564035841cb0" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:19:42.572911 18276 raft_consensus.cc:493] T b87147731a68400e8cce9b879daa7c5d P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:42.531306 18246 consensus_queue.cc:237] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:42.568183 18270 leader_election.cc:290] T ecc1e363230043fe9fef390f0b979f20 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:19:42.572647 18046 leader_election.cc:290] T a378d7f796f94615b3b347119a9880fd P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:19:42.786423 18275 leader_election.cc:290] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:42.864789 16816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2794af5b99984efebdee564035841cb0" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:19:42.865816 18199 leader_election.cc:290] T 8a878999c0024d2abe517484180290fc P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:19:42.805459 18240 leader_election.cc:290] T 5a96373bb0e14dce8a266b2fc6c58507 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 3 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:42.818576 18250 raft_consensus.cc:2804] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:19:42.893592 16822 raft_consensus.cc:2468] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 4.
I20260709 13:19:42.678232 18140 raft_consensus.cc:515] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:42.836589 18151 raft_consensus.cc:493] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:42.723716 18154 raft_consensus.cc:493] T 68a6838d82ef4cac83507f906139d90f P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:19:42.714718 18167 consensus_queue.cc:1048] T 2794af5b99984efebdee564035841cb0 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.149s
I20260709 13:19:42.804380 18218 raft_consensus.cc:2749] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:19:42.643491 18257 raft_consensus.cc:515] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:42.789891 18234 raft_consensus.cc:515] T 84f2da98864b4425bf7c0055951fdd4b P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:42.886636 18225 raft_consensus.cc:515] T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:42.886371 18134 leader_election.cc:290] T c0eab25ce4c64e018a041beeda2867db P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 4 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:42.928889 16816 raft_consensus.cc:3060] T 2794af5b99984efebdee564035841cb0 P cb82cc7bd6db46d980c75237a1402860 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:19:42.886879 18241 raft_consensus.cc:2749] T 8c11074579b046d99e585c2818e18a2c P cb82cc7bd6db46d980c75237a1402860 [term 5 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:19:42.894116 18254 raft_consensus.cc:3060] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:19:42.921698 18046 raft_consensus.cc:493] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:42.896934 18217 raft_consensus.cc:493] T d2ab0ac8347641e6955e9bce7d90c1a2 P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:19:42.920812 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5718c623ebee4d2bb770f0f9ddb2f164" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:19:42.888782 18156 raft_consensus.cc:515] T 8d4ab870a05347868956aa8b841b2981 P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:42.976709 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1df93779209748b9860f6d893dd3de53" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:19:42.979507 16146 raft_consensus.cc:2393] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 6: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
W20260709 13:19:42.979923 16224 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57600 (request call id 1008) took 2952 ms (client timeout 2133 ms). Trace:
I20260709 13:19:42.950345 18250 raft_consensus.cc:493] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:19:42.983752 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 177485us
I20260709 13:19:43.004165 16790 leader_election.cc:304] T 829b4ad8c5a54eababeea1181b960d4b P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:19:43.005456 16790 leader_election.cc:304] T c9944d5e4b8c4109af9955c4f08c9095 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:19:42.965895 16142 raft_consensus.cc:2393] T 2794af5b99984efebdee564035841cb0 P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 in current term 3: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
I20260709 13:19:42.954515 16817 raft_consensus.cc:1275] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 [term 3 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:19:42.899433 18161 raft_consensus.cc:515] T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:42.986184 17992 raft_consensus.cc:2804] T 74ca8140f35c4ae29e8a664f6bdfc829 P cb82cc7bd6db46d980c75237a1402860 [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 13:19:42.995543 16224 rpcz_store.cc:269] 0709 13:19:40.018192 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:40.068715 (+ 50523us) service_pool.cc:224] Handling call
0709 13:19:42.970792 (+2902077us) spinlock_profiling.cc:243] Waited 782 ms on lock 0x7b5400139f54. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:19:42.970943 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1798142336}
I20260709 13:19:42.997988 16219 raft_consensus.cc:2393] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 in current term 4: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
I20260709 13:19:42.966869 18154 raft_consensus.cc:515] T 68a6838d82ef4cac83507f906139d90f P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:42.999635 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af57996a8be84d398f0f023e8d08ef77" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
W20260709 13:19:43.038514 16117 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 224246us
I20260709 13:19:43.002565 18156 leader_election.cc:290] T 8d4ab870a05347868956aa8b841b2981 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:19:43.009059 18034 raft_consensus.cc:2804] T 829b4ad8c5a54eababeea1181b960d4b P cb82cc7bd6db46d980c75237a1402860 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:19:42.993350 18217 raft_consensus.cc:515] T d2ab0ac8347641e6955e9bce7d90c1a2 P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:43.009902 18228 leader_election.cc:290] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:43.010502 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c74490c40243a88e62d1760b5cd814" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:19:43.014176 18198 raft_consensus.cc:2804] T c9944d5e4b8c4109af9955c4f08c9095 P cb82cc7bd6db46d980c75237a1402860 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:19:43.006433 16790 leader_election.cc:304] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:19:42.948982 18199 consensus_queue.cc:1059] T 1df93779209748b9860f6d893dd3de53 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.035s
W20260709 13:19:43.048309 16789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 259154us
I20260709 13:19:42.967065 18151 raft_consensus.cc:515] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:19:42.999956 18282 log.cc:927] Time spent T 57a744f5aaf944678a63469cbf514015 P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 0.112s	user 0.000s	sys 0.000s
I20260709 13:19:43.035915 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:43.051461 16224 raft_consensus.cc:2468] T ecc1e363230043fe9fef390f0b979f20 P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 3.
W20260709 13:19:43.035207 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 249452us
I20260709 13:19:42.917239 18276 raft_consensus.cc:515] T b87147731a68400e8cce9b879daa7c5d P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:43.049888 17999 leader_election.cc:290] T d16dfad0c3b540e5b4b86492ddf87208 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 2 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:19:43.056375 18276 leader_election.cc:290] T b87147731a68400e8cce9b879daa7c5d P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:19:42.979270 18189 consensus_queue.cc:1048] T 3a7ca61592e848ea9b0fb81a2ce33cdc P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.040s
I20260709 13:19:43.015920 18250 raft_consensus.cc:3060] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:19:43.058271 18189 raft_consensus.cc:2804] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:19:43.055022 18161 leader_election.cc:290] T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:19:43.058882 18189 raft_consensus.cc:697] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 [term 4 LEADER]: Becoming Leader. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:19:43.058463 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d16dfad0c3b540e5b4b86492ddf87208" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:19:43.060012 16224 raft_consensus.cc:3060] T d16dfad0c3b540e5b4b86492ddf87208 P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:19:43.060184 18189 consensus_queue.cc:237] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:43.063887 18240 raft_consensus.cc:493] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:43.064409 16220 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57604 (request call id 1952) took 1245 ms. Trace:
I20260709 13:19:43.064512 18240 raft_consensus.cc:515] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:43.064942 16220 rpcz_store.cc:276] 0709 13:19:41.819515 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:42.565326 (+745811us) service_pool.cc:224] Handling call
0709 13:19:42.565642 (+   316us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:42.565996 (+   354us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:42.566027 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:42.566517 (+   490us) log.cc:844] Serialized 26 byte log entry
0709 13:19:42.949762 (+383245us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:19:42.952447 (+  2685us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:42.969981 (+ 17534us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:42.997060 (+ 27079us) spinlock_profiling.cc:243] Waited 9.06 ms on lock 0x7b48000cfe58. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b5a88aa1 00007f69b5a7fb37 00007f69b5a7f9ca 00007f69b5b7726f 00007f69b5b76f37 00007f69b5b8b55c 00007f69b5b8b508 00007f69b5b8b488 00007f69b5b8b443 00007f69b5b8a6ec 00007f69b4ab4a96
0709 13:19:43.064092 (+ 67032us) raft_consensus.cc:1685] finished
0709 13:19:43.064125 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:43.064343 (+   218us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":10221,"raft.run_cpu_time_us":1506,"raft.run_wall_time_us":208517,"spinlock_wait_cycles":20848512,"thread_start_us":146068,"threads_started":1,"wal-append.queue_time_us":423317}
I20260709 13:19:43.033823 17992 raft_consensus.cc:493] T 74ca8140f35c4ae29e8a664f6bdfc829 P cb82cc7bd6db46d980c75237a1402860 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:43.067824 18198 raft_consensus.cc:697] T c9944d5e4b8c4109af9955c4f08c9095 P cb82cc7bd6db46d980c75237a1402860 [term 5 LEADER]: Becoming Leader. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:19:43.069283 18198 consensus_queue.cc:237] T c9944d5e4b8c4109af9955c4f08c9095 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:43.049181 16789 leader_election.cc:304] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:19:43.070979 16224 raft_consensus.cc:2468] T d16dfad0c3b540e5b4b86492ddf87208 P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 3.
I20260709 13:19:43.073598 18177 raft_consensus.cc:2804] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:19:43.074077 18177 raft_consensus.cc:493] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:43.074420 18177 raft_consensus.cc:3060] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:19:43.075202 18151 leader_election.cc:290] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:19:43.075452 16222 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57604 (request call id 1950) took 1256 ms. Trace:
I20260709 13:19:43.075863 16222 rpcz_store.cc:276] 0709 13:19:41.818781 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:42.552719 (+733938us) service_pool.cc:224] Handling call
0709 13:19:42.759792 (+207073us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:19:42.760316 (+   524us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:42.760354 (+    38us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:19:42.809952 (+ 49598us) log.cc:844] Serialized 52 byte log entry
0709 13:19:42.982234 (+172282us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:19:42.982296 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:42.982405 (+   109us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:43.075234 (+ 92829us) raft_consensus.cc:1685] finished
0709 13:19:43.075254 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:43.075395 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":142659,"raft.queue_time_us":26208,"raft.run_cpu_time_us":149,"raft.run_wall_time_us":10244,"thread_start_us":94781,"threads_started":1,"wal-append.queue_time_us":178591}
I20260709 13:19:43.079035 16225 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57604 (request call id 1954) took 1259 ms. Trace:
I20260709 13:19:43.079478 16225 rpcz_store.cc:276] 0709 13:19:41.820150 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:41.820769 (+   619us) service_pool.cc:224] Handling call
0709 13:19:43.078848 (+1258079us) spinlock_profiling.cc:243] Waited 144 ms on lock 0x7b5400246654. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:19:43.078980 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":330766208}
I20260709 13:19:43.054090 18104 raft_consensus.cc:493] T 829b4ad8c5a54eababeea1181b960d4b P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:43.067956 17992 raft_consensus.cc:3060] T 74ca8140f35c4ae29e8a664f6bdfc829 P cb82cc7bd6db46d980c75237a1402860 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:19:43.018271 18046 raft_consensus.cc:515] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:43.041720 18013 consensus_queue.cc:237] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 3, Current term: 4, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:19:43.040066 16142 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46224 (request call id 253) took 1647 ms (client timeout 2173 ms). Trace:
W20260709 13:19:43.246024 16117 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 193733us
W20260709 13:19:43.244277 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 109370us
I20260709 13:19:43.291332 16117 leader_election.cc:304] T 9e65676b21b748728ad39668cfe722ec P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860; no voters: 90f3da57796b488c9087a2f523adf045
I20260709 13:19:43.084878 16222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a96373bb0e14dce8a266b2fc6c58507" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:19:43.305923 16222 raft_consensus.cc:2393] T 5a96373bb0e14dce8a266b2fc6c58507 P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 3: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
I20260709 13:19:43.307763 16222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4dc6b0541ca431ebc757beac00521f1" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:43.045382 16149 raft_consensus.cc:3060] T af57996a8be84d398f0f023e8d08ef77 P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:19:43.132555 18281 log.cc:927] Time spent T ada5a004ba7c46819051e3ccec71ffdb P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.088s	user 0.000s	sys 0.001s
I20260709 13:19:43.330411 16222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806d7690c2b644959152074bb2cab0a7" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
W20260709 13:19:43.127671 18284 log.cc:927] Time spent T 8a878999c0024d2abe517484180290fc P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 0.178s	user 0.001s	sys 0.000s
I20260709 13:19:43.331387 16222 raft_consensus.cc:2468] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 3.
I20260709 13:19:43.333325 16222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5d59407eb545b895d8d922e0dd69fd" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:19:43.335997 16300 raft_consensus.cc:3060] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:19:43.336357 16222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66ae176961b74813a58bc15c9c4c6f2a" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:43.141721 18271 leader_election.cc:290] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 3 election: Requested vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:19:43.178038 16220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cc959fb7f704fba9009c0595467d074" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:19:43.384282 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d16dfad0c3b540e5b4b86492ddf87208" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:19:43.385174 16299 raft_consensus.cc:3060] T d16dfad0c3b540e5b4b86492ddf87208 P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:19:43.140859 18257 leader_election.cc:290] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 4 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
W20260709 13:19:43.337005 16789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 287871us
I20260709 13:19:43.130110 18154 leader_election.cc:290] T 68a6838d82ef4cac83507f906139d90f P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:19:43.117390 16218 raft_consensus.cc:1240] T 1df93779209748b9860f6d893dd3de53 P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 3. Current term is 4. Ops: []
I20260709 13:19:43.117977 18276 consensus_queue.cc:1048] T fffab26427e24d398e2695aad40ca134 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.282s
I20260709 13:19:43.128466 16292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:43.255039 18240 leader_election.cc:290] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:19:43.134613 18156 consensus_queue.cc:1059] T 7c2095a5f4ad49408dbc7f69c606120d P 90f3da57796b488c9087a2f523adf045 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.204s
I20260709 13:19:43.073612 18217 leader_election.cc:290] T d2ab0ac8347641e6955e9bce7d90c1a2 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:19:43.130512 18262 leader_election.cc:290] T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 5 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:19:43.133709 18225 leader_election.cc:290] T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
W20260709 13:19:43.345263 18145 consensus_meta.cc:287] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b: Time spent flushing consensus metadata: real 0.536s	user 0.008s	sys 0.001s
I20260709 13:19:43.346772 18046 leader_election.cc:290] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:19:43.325320 16816 raft_consensus.cc:2468] T 2794af5b99984efebdee564035841cb0 P cb82cc7bd6db46d980c75237a1402860 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 3.
I20260709 13:19:43.135535 18199 consensus_queue.cc:1048] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.207s
I20260709 13:19:43.135308 18167 consensus_queue.cc:1048] T 8cc959fb7f704fba9009c0595467d074 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.208s
I20260709 13:19:43.332405 18173 consensus_queue.cc:1048] T 434bff3082d141838068f323bdb0d1ed P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.026s
I20260709 13:19:43.363659 18254 raft_consensus.cc:515] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:43.323099 18204 raft_consensus.cc:2804] T 9e65676b21b748728ad39668cfe722ec P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:19:43.362556 17992 raft_consensus.cc:515] T 74ca8140f35c4ae29e8a664f6bdfc829 P cb82cc7bd6db46d980c75237a1402860 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:19:43.360620 17844 consensus_meta.cc:287] T ee9ed67006144797b1de6ea8c2013a68 P 1ca2725d290e47d68b1a4b609c9816ac: Time spent flushing consensus metadata: real 0.546s	user 0.010s	sys 0.000s
I20260709 13:19:43.081233 16294 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40474 (request call id 304) took 1258 ms. Trace:
I20260709 13:19:43.332366 18104 raft_consensus.cc:515] T 829b4ad8c5a54eababeea1181b960d4b P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:43.366315 18250 raft_consensus.cc:515] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:43.387068 18123 leader_election.cc:290] T 23f82224f4564fea93572e8206bad1fe P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:19:43.389923 16147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:19:43.459425 18244 leader_election.cc:290] T 5a959dbecc2648eb85d1cc550f5a3245 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
W20260709 13:19:43.290709 16142 rpcz_store.cc:269] 0709 13:19:41.389700 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:42.682992 (+1293292us) service_pool.cc:224] Handling call
0709 13:19:43.036893 (+353901us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:43.400511 18182 consensus_queue.cc:1059] T 3884f864a96d428785895e5463e5728f P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.060s
I20260709 13:19:43.461176 18179 raft_consensus.cc:3055] T 3884f864a96d428785895e5463e5728f P cb82cc7bd6db46d980c75237a1402860 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:19:43.461284 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5d59407eb545b895d8d922e0dd69fd" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:19:43.461604 18179 raft_consensus.cc:740] T 3884f864a96d428785895e5463e5728f P cb82cc7bd6db46d980c75237a1402860 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:19:43.461398 16142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62fe3346cd3c42c4a55e14af9073bca4" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:19:43.462069 16150 raft_consensus.cc:2368] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac for earlier term 4. Current term is 5.
W20260709 13:19:43.462519 16150 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40844 (request call id 1786) took 1463 ms (client timeout 1725 ms). Trace:
I20260709 13:19:43.462617 18179 consensus_queue.cc:260] T 3884f864a96d428785895e5463e5728f P cb82cc7bd6db46d980c75237a1402860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:43.463992 18179 raft_consensus.cc:3060] T 3884f864a96d428785895e5463e5728f P cb82cc7bd6db46d980c75237a1402860 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:19:43.401185 16218 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57600 (request call id 1014) took 1636 ms. Trace:
I20260709 13:19:43.465696 16218 rpcz_store.cc:276] 0709 13:19:41.764892 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:41.769940 (+  5048us) service_pool.cc:224] Handling call
0709 13:19:42.544303 (+774363us) spinlock_profiling.cc:243] Waited 3.47 ms on lock 0x7b5800010b98. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69ad8fadb1 00007f69ada3ab71 00007f69ada3aa22 00007f69b6953c09 00007f69b5aadee0 00007f69b5b770ab 00007f69b5b77131 00007f69b5b38fc0 00007f69b5b23929 00007f69b5b233d2 00007f69af09abd4
0709 13:19:43.091039 (+546736us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:43.400017 (+308978us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:43.400968 (+   951us) spinlock_profiling.cc:243] Waited 537 ms on lock 0x7b5400246650. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5e8ad 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:19:43.401111 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1244217984}
I20260709 13:19:43.401573 16292 raft_consensus.cc:2468] T ecc1e363230043fe9fef390f0b979f20 P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 3.
W20260709 13:19:43.462927 16150 rpcz_store.cc:269] 0709 13:19:41.998731 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:42.014532 (+ 15801us) service_pool.cc:224] Handling call
0709 13:19:43.461170 (+1446638us) spinlock_profiling.cc:243] Waited 3.89 ms on lock 0x7b5800011198. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69ad8fadb1 00007f69ada3ab71 00007f69ada3aeee 0000000000375939 00007f69b5aae1e0 00007f69b5b770c4 00007f69b5b77131 00007f69b5b250a0 00007f69b5b24984 00007f69af09f6d4 00007f69af09f662
0709 13:19:43.462459 (+  1289us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":8982144}
I20260709 13:19:43.477905 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e9b2bf7011f4ae688a79d7bb5a447b4" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:19:43.481427 17999 raft_consensus.cc:3055] T 1df93779209748b9860f6d893dd3de53 P 90f3da57796b488c9087a2f523adf045 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:19:43.481508 16292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8638ee363273426db1393dd81e5c7b72" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:43.405687 18204 raft_consensus.cc:493] T 9e65676b21b748728ad39668cfe722ec P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:43.482854 18204 raft_consensus.cc:3060] T 9e65676b21b748728ad39668cfe722ec P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:19:43.478163 18272 consensus_meta.cc:287] T 8cc959fb7f704fba9009c0595467d074 P 1ca2725d290e47d68b1a4b609c9816ac: Time spent flushing consensus metadata: real 0.591s	user 0.009s	sys 0.000s
I20260709 13:19:43.483268 16292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2794af5b99984efebdee564035841cb0" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:19:43.426422 18114 leader_election.cc:290] T e4dc6b0541ca431ebc757beac00521f1 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:43.415483 17844 raft_consensus.cc:515] T ee9ed67006144797b1de6ea8c2013a68 P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:43.418457 17918 raft_consensus.cc:3055] T b6f07842d3aa47858df5665874c86f3a P 1ca2725d290e47d68b1a4b609c9816ac [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:19:43.766433 16292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4dc6b0541ca431ebc757beac00521f1" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:43.766793 17918 raft_consensus.cc:740] T b6f07842d3aa47858df5665874c86f3a P 1ca2725d290e47d68b1a4b609c9816ac [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:19:43.767234 16292 raft_consensus.cc:2468] T e4dc6b0541ca431ebc757beac00521f1 P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 3.
W20260709 13:19:43.449884 16219 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33050 (request call id 398) took 1686 ms (client timeout 1507 ms). Trace:
I20260709 13:19:43.767822 17918 consensus_queue.cc:260] T b6f07842d3aa47858df5665874c86f3a P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:19:43.768535 16219 rpcz_store.cc:269] 0709 13:19:41.763531 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:41.764465 (+   934us) service_pool.cc:224] Handling call
0709 13:19:43.449556 (+1685091us) spinlock_profiling.cc:243] Waited 1.05 s on lock 0x7b54002c9054. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:19:43.449779 (+   223us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2406548352}
I20260709 13:19:43.441309 18234 leader_election.cc:290] T 84f2da98864b4425bf7c0055951fdd4b P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 3 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:43.427465 18250 leader_election.cc:290] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 6 election: Requested vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:19:43.770071 16219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66f75124215f4ca6bef831e174a792d8" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:19:43.442329 18177 raft_consensus.cc:515] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:43.770738 16219 raft_consensus.cc:3060] T 66f75124215f4ca6bef831e174a792d8 P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:19:43.697618 16790 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.651s	user 0.000s	sys 0.040s
W20260709 13:19:43.771497 16790 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.725s	user 0.001s	sys 0.041s
W20260709 13:19:43.771808 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 765405us
I20260709 13:19:43.772418 18177 leader_election.cc:290] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 5 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:43.485385 16116 leader_election.cc:304] T d16dfad0c3b540e5b4b86492ddf87208 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
I20260709 13:19:43.445166 18140 leader_election.cc:290] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 6 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:43.761186 18272 raft_consensus.cc:515] T 8cc959fb7f704fba9009c0595467d074 P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:19:43.754899 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 460204us
I20260709 13:19:43.424516 16222 raft_consensus.cc:2368] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 for earlier term 5. Current term is 6.
W20260709 13:19:43.774545 16190 connection.cc:256] RPC call timeout handler was delayed by -1.6125205949999781s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22460000000000002s for a response
W20260709 13:19:43.774861 16190 connection.cc:256] RPC call timeout handler was delayed by -1.3048257229997944s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2089s for a response
I20260709 13:19:43.411625 18256 leader_election.cc:290] T 1df93779209748b9860f6d893dd3de53 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:43.776041 18256 raft_consensus.cc:493] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:43.776525 18256 raft_consensus.cc:515] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:43.761648 18105 raft_consensus.cc:493] T 26354361e9b044cea1c2b86c4882fe92 P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:19:43.448052 16143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2372981006d4b218215c28964dea623" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:19:43.755595 18298 raft_consensus.cc:493] T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:43.448755 16151 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40830 (request call id 1029) took 2118 ms. Trace:
I20260709 13:19:43.426828 16297 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48674 (request call id 2036) took 2500 ms. Trace:
I20260709 13:19:43.420521 16299 raft_consensus.cc:2468] T d16dfad0c3b540e5b4b86492ddf87208 P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 3.
I20260709 13:19:43.433576 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87ac0d1f7e8a419f9bcbc04aee0e067e" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:19:43.446468 16296 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40474 (request call id 310) took 1622 ms. Trace:
I20260709 13:19:43.444783 16145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434bff3082d141838068f323bdb0d1ed" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:19:43.426594 16298 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48684 (request call id 1911) took 2615 ms. Trace:
I20260709 13:19:43.431588 18041 leader_election.cc:290] T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:19:43.425936 16148 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40844 (request call id 1788) took 1426 ms. Trace:
I20260709 13:19:43.424523 18248 raft_consensus.cc:1064] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b: attempting to promote NON_VOTER cb82cc7bd6db46d980c75237a1402860 to VOTER
I20260709 13:19:43.780387 17992 leader_election.cc:290] T 74ca8140f35c4ae29e8a664f6bdfc829 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 4 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:43.756366 16816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ab0ac8347641e6955e9bce7d90c1a2" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:19:43.401811 18156 raft_consensus.cc:3055] T 7c2095a5f4ad49408dbc7f69c606120d P 90f3da57796b488c9087a2f523adf045 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:19:43.780586 16145 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46224 (request call id 257) took 1096 ms. Trace:
I20260709 13:19:43.413064 18257 raft_consensus.cc:493] T 84f2da98864b4425bf7c0055951fdd4b P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:43.781776 16145 rpcz_store.cc:276] 0709 13:19:42.684223 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:42.811646 (+127423us) service_pool.cc:224] Handling call
0709 13:19:43.444602 (+632956us) spinlock_profiling.cc:243] Waited 1.34 ms on lock 0x7b5800011198. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69ad8fadb1 00007f69ada3ab71 00007f69ada3aa22 00007f69b6953c09 00007f69b5aadee0 00007f69b5b770ab 00007f69b5b77131 00007f69b5b250a0 00007f69b5b24984 00007f69af09f6d4 00007f69af09f662
0709 13:19:43.780531 (+335929us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3072768}
I20260709 13:19:43.782582 18257 raft_consensus.cc:515] T 84f2da98864b4425bf7c0055951fdd4b P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:43.420337 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ad5dbfa99ee4b8d9e28be92b3f36db5" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
W20260709 13:19:43.727986 18034 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.636s	user 0.009s	sys 0.007s
I20260709 13:19:43.790398 16143 raft_consensus.cc:2393] T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 in current term 4: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
W20260709 13:19:43.790534 18034 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.709s	user 0.009s	sys 0.008s
I20260709 13:19:43.790907 18034 raft_consensus.cc:697] T 829b4ad8c5a54eababeea1181b960d4b P cb82cc7bd6db46d980c75237a1402860 [term 5 LEADER]: Becoming Leader. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:19:43.791148 16143 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46224 (request call id 254) took 1108 ms. Trace:
I20260709 13:19:43.791450 16817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74ca8140f35c4ae29e8a664f6bdfc829" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:19:43.791718 16143 rpcz_store.cc:276] 0709 13:19:42.683286 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:43.447625 (+764339us) service_pool.cc:224] Handling call
0709 13:19:43.791079 (+343454us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:43.781486 16220 raft_consensus.cc:2393] T 8cc959fb7f704fba9009c0595467d074 P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 4: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
I20260709 13:19:43.792277 16817 raft_consensus.cc:2368] T 74ca8140f35c4ae29e8a664f6bdfc829 P cb82cc7bd6db46d980c75237a1402860 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac for earlier term 3. Current term is 4.
I20260709 13:19:43.459443 16147 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40844 (request call id 1799) took 1443 ms. Trace:
I20260709 13:19:43.418707 16294 rpcz_store.cc:276] 0709 13:19:41.822662 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:41.824741 (+  2079us) service_pool.cc:224] Handling call
0709 13:19:43.081060 (+1256319us) spinlock_profiling.cc:243] Waited 356 ms on lock 0x7b54002ce2d4. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:19:43.081172 (+   112us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":818668800}
I20260709 13:19:43.793193 16147 rpcz_store.cc:276] 0709 13:19:42.016328 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:43.389531 (+1373203us) service_pool.cc:224] Handling call
0709 13:19:43.459371 (+ 69840us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:43.780882 18105 raft_consensus.cc:515] T 26354361e9b044cea1c2b86c4882fe92 P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:43.794104 16147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87147731a68400e8cce9b879daa7c5d" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:19:43.794209 16294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6f07842d3aa47858df5665874c86f3a" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:43.794713 16220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a378d7f796f94615b3b347119a9880fd" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:43.795326 18105 leader_election.cc:290] T 26354361e9b044cea1c2b86c4882fe92 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:19:43.790215 16816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:19:43.795463 16220 raft_consensus.cc:2468] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 3.
I20260709 13:19:43.482290 17999 raft_consensus.cc:740] T 1df93779209748b9860f6d893dd3de53 P 90f3da57796b488c9087a2f523adf045 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:19:43.796037 16816 raft_consensus.cc:2468] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 3.
I20260709 13:19:43.778512 18298 raft_consensus.cc:515] T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:43.796451 16220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8638ee363273426db1393dd81e5c7b72" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:43.797590 16220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d16dfad0c3b540e5b4b86492ddf87208" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:19:43.796769 17999 consensus_queue.cc:260] T 1df93779209748b9860f6d893dd3de53 P 90f3da57796b488c9087a2f523adf045 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:43.798317 16220 raft_consensus.cc:2368] T d16dfad0c3b540e5b4b86492ddf87208 P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 for earlier term 2. Current term is 3.
I20260709 13:19:43.798237 18298 leader_election.cc:290] T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:19:43.798411 17999 raft_consensus.cc:3060] T 1df93779209748b9860f6d893dd3de53 P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:19:43.784186 16788 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:43.783593 18272 leader_election.cc:290] T 8cc959fb7f704fba9009c0595467d074 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 4 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:43.513934 16300 raft_consensus.cc:2468] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 5.
I20260709 13:19:43.800822 16817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5718c623ebee4d2bb770f0f9ddb2f164" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:19:43.801100 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cc959fb7f704fba9009c0595467d074" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:19:43.801415 16817 raft_consensus.cc:3060] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:19:43.455952 16293 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48674 (request call id 2039) took 2511 ms. Trace:
I20260709 13:19:43.801914 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66ae176961b74813a58bc15c9c4c6f2a" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:43.801985 16293 rpcz_store.cc:276] 0709 13:19:40.944654 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:41.185974 (+241320us) service_pool.cc:224] Handling call
0709 13:19:41.953967 (+767993us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:42.216482 (+262515us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:43.455724 (+1239242us) spinlock_profiling.cc:243] Waited 765 ms on lock 0x7b540024c550. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5e8ad 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:19:43.455887 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1760474496}
I20260709 13:19:43.802641 16300 raft_consensus.cc:2468] T 66ae176961b74813a58bc15c9c4c6f2a P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 4.
W20260709 13:19:43.451855 16146 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40844 (request call id 1793) took 1450 ms (client timeout 1673 ms). Trace:
I20260709 13:19:43.789106 16218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f5f89186204242bf377f7c42b2e2da" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:19:43.803447 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87ac0d1f7e8a419f9bcbc04aee0e067e" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:19:43.410115 16044 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:58400 (request call id 188) took 6115 ms. Trace:
W20260709 13:19:43.803574 16146 rpcz_store.cc:269] 0709 13:19:42.001157 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:42.285630 (+284473us) service_pool.cc:224] Handling call
0709 13:19:43.451656 (+1166026us) spinlock_profiling.cc:243] Waited 669 ms on lock 0x7b540018f954. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:19:43.451798 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1539323648}
I20260709 13:19:43.780055 16298 rpcz_store.cc:276] 0709 13:19:40.812352 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:40.815380 (+  3028us) service_pool.cc:224] Handling call
0709 13:19:40.815714 (+   334us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:41.368042 (+552328us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:43.426341 (+2058299us) spinlock_profiling.cc:243] Waited 385 ms on lock 0x7b5400347e54. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:19:43.426533 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":886013184}
I20260709 13:19:43.804843 16146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a9de347c711477e87c582d003f36925" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:19:43.423070 18104 leader_election.cc:290] T 829b4ad8c5a54eababeea1181b960d4b P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:19:43.805569 16146 raft_consensus.cc:2468] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 4.
I20260709 13:19:43.799962 16225 raft_consensus.cc:2468] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 3.
I20260709 13:19:43.805831 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74ca8140f35c4ae29e8a664f6bdfc829" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:19:43.806600 16298 raft_consensus.cc:2393] T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 3: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
I20260709 13:19:43.806924 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "829b4ad8c5a54eababeea1181b960d4b" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:43.807330 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806d7690c2b644959152074bb2cab0a7" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:43.783093 16145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5ad4eb281564936a59f41e387a47156" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:19:43.441908 18181 leader_election.cc:290] T af57996a8be84d398f0f023e8d08ef77 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:19:43.450855 16144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "829b4ad8c5a54eababeea1181b960d4b" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:19:43.803849 16218 raft_consensus.cc:3060] T c2f5f89186204242bf377f7c42b2e2da P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:19:43.807606 16225 raft_consensus.cc:2393] T 829b4ad8c5a54eababeea1181b960d4b P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 5: Already voted for candidate cb82cc7bd6db46d980c75237a1402860 in this term.
I20260709 13:19:43.812435 16823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66f75124215f4ca6bef831e174a792d8" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:19:43.772452 16790 leader_election.cc:304] T af57996a8be84d398f0f023e8d08ef77 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:19:43.823613 16822 raft_consensus.cc:1275] T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860 [term 4 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260709 13:19:43.823737 16823 raft_consensus.cc:2468] T 66f75124215f4ca6bef831e174a792d8 P cb82cc7bd6db46d980c75237a1402860 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 4.
I20260709 13:19:43.808873 16817 raft_consensus.cc:2468] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 4.
I20260709 13:19:43.783493 16151 rpcz_store.cc:276] 0709 13:19:41.330490 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:41.331263 (+   773us) service_pool.cc:224] Handling call
0709 13:19:41.345311 (+ 14048us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:42.214838 (+869527us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:43.448420 (+1233582us) spinlock_profiling.cc:243] Waited 681 ms on lock 0x7b540028da54. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:19:43.448647 (+   227us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1565816704}
I20260709 13:19:43.809228 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af57996a8be84d398f0f023e8d08ef77" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:43.836810 16224 raft_consensus.cc:2393] T af57996a8be84d398f0f023e8d08ef77 P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 4: Already voted for candidate cb82cc7bd6db46d980c75237a1402860 in this term.
I20260709 13:19:43.804342 16044 rpcz_store.cc:276] 0709 13:19:37.294656 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:37.294835 (+   179us) service_pool.cc:224] Handling call
0709 13:19:43.410023 (+6115188us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:19:38.103554 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:19:38.104281 (+   727us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:19:38.104333 (+    52us) write_op.cc:435] Acquired schema lock
0709 13:19:38.104355 (+    22us) tablet.cc:662] Decoding operations
0709 13:19:38.105393 (+  1038us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:19:38.105522 (+   129us) write_op.cc:641] Partition lock acquired for write op
0709 13:19:38.105560 (+    38us) tablet.cc:685] Acquiring locks for 8 operations
0709 13:19:38.106453 (+   893us) tablet.cc:701] Row locks acquired
0709 13:19:38.106510 (+    57us) write_op.cc:260] PREPARE: finished
0709 13:19:38.106714 (+   204us) write_op.cc:270] Start()
0709 13:19:38.106865 (+   151us) write_op.cc:276] Timestamp: P: 1783603178106646 usec, L: 0
0709 13:19:38.106906 (+    41us) op_driver.cc:348] REPLICATION: starting
0709 13:19:38.174527 (+ 67621us) log.cc:844] Serialized 3370 byte log entry
0709 13:19:38.663721 (+489194us) spinlock_profiling.cc:243] Waited 3.39 ms on lock 0x7b4400022b4c. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69ad8fadb1 00007f69ad998c85 00007f69ad9996a2 00007f69b070b1db 00007f69b0706aaf 00007f69ada550ac 00007f69b68afab9 00007f69ada7169b 00007f69ada6f93f 00007f69ada71c02 00007f69b06f920b
0709 13:19:39.578518 (+914797us) op_driver.cc:464] REPLICATION: finished
0709 13:19:39.885632 (+307114us) write_op.cc:301] APPLY: starting
0709 13:19:39.893247 (+  7615us) tablet.cc:1367] starting BulkCheckPresence
0709 13:19:39.919358 (+ 26111us) tablet.cc:1370] finished BulkCheckPresence
0709 13:19:39.919400 (+    42us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:19:39.920515 (+  1115us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:19:39.920821 (+   306us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=8
0709 13:19:39.920868 (+    47us) write_op.cc:312] APPLY: finished
0709 13:19:39.921235 (+   367us) log.cc:844] Serialized 65 byte log entry
0709 13:19:39.921434 (+   199us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:19:39.921552 (+   118us) write_op.cc:454] Released schema lock
0709 13:19:39.921608 (+    56us) write_op.cc:341] FINISH: Updating metrics
0709 13:19:39.922038 (+   430us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:19:39.922071 (+    33us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":79,"reported_tablets":19,"child_traces":[["op",{"apply.queue_time_us":241401,"apply.run_cpu_time_us":3255,"apply.run_wall_time_us":49501,"num_ops":8,"prepare.queue_time_us":108451,"prepare.run_cpu_time_us":40587,"prepare.run_wall_time_us":965960,"raft.queue_time_us":252175,"raft.run_cpu_time_us":93454,"raft.run_wall_time_us":333829,"replication_time_us":1411693,"spinlock_wait_cycles":7806848,"thread_start_us":1188320,"threads_started":4,"wal-append.queue_time_us":712521,"wal-append.run_cpu_time_us":42040,"wal-append.run_wall_time_us":2040957}]]}
I20260709 13:19:43.759649 16291 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40474 (request call id 313) took 1933 ms. Trace:
W20260709 13:19:43.763881 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 256965us
I20260709 13:19:43.783870 16296 rpcz_store.cc:276] 0709 13:19:41.824178 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:42.217045 (+392867us) service_pool.cc:224] Handling call
0709 13:19:42.217376 (+   331us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:42.307886 (+ 90510us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:42.307920 (+    34us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:42.307964 (+    44us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:19:42.307998 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:43.446176 (+1138178us) spinlock_profiling.cc:243] Waited 90.3 ms on lock 0x7b540010ed54. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:19:43.446232 (+    56us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:43.446396 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":207666944}
I20260709 13:19:43.838575 16296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdbd78acb4cf46298e0de8ba0d130e19" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:43.838749 16191 leader_election.cc:304] T 7c2095a5f4ad49408dbc7f69c606120d P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 90f3da57796b488c9087a2f523adf045
I20260709 13:19:43.781358 18156 raft_consensus.cc:740] T 7c2095a5f4ad49408dbc7f69c606120d P 90f3da57796b488c9087a2f523adf045 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:19:43.840709 16190 leader_election.cc:304] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860
I20260709 13:19:43.842401 16143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434bff3082d141838068f323bdb0d1ed" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:19:43.843077 16190 leader_election.cc:304] T e4dc6b0541ca431ebc757beac00521f1 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 647a343a542d4a139fe106c19095129b
I20260709 13:19:43.842705 18156 consensus_queue.cc:260] T 7c2095a5f4ad49408dbc7f69c606120d P 90f3da57796b488c9087a2f523adf045 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:43.843999 18156 raft_consensus.cc:3060] T 7c2095a5f4ad49408dbc7f69c606120d P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:19:43.401135 16789 leader_election.cc:304] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 
W20260709 13:19:43.845234 16789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 444145us
I20260709 13:19:43.436611 16223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5ad4eb281564936a59f41e387a47156" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:43.753899 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4dc6b0541ca431ebc757beac00521f1" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:19:43.785485 18114 consensus_queue.cc:1048] T 8c11074579b046d99e585c2818e18a2c P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.015s
I20260709 13:19:43.780076 16148 rpcz_store.cc:276] 0709 13:19:41.999469 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:41.999814 (+   345us) service_pool.cc:224] Handling call
0709 13:19:43.425700 (+1425886us) spinlock_profiling.cc:243] Waited 328 ms on lock 0x7b540013f1d4. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:19:43.425835 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":754817280}
I20260709 13:19:43.852057 16824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada5a004ba7c46819051e3ccec71ffdb" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:19:43.810308 16816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66ae176961b74813a58bc15c9c4c6f2a" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:19:43.768195 16292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a878999c0024d2abe517484180290fc" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:43.862809 16292 raft_consensus.cc:2468] T 8a878999c0024d2abe517484180290fc P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 3.
I20260709 13:19:43.863312 16816 raft_consensus.cc:3060] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:19:43.802810 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:43.864105 16299 raft_consensus.cc:2468] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 4.
I20260709 13:19:43.773648 16217 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57604 (request call id 1947) took 1955 ms. Trace:
I20260709 13:19:43.769063 17918 raft_consensus.cc:3060] T b6f07842d3aa47858df5665874c86f3a P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:19:43.823249 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8158bbd17b994d2eb96f560bd0f1b88d" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:43.391930 16295 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40474 (request call id 312) took 1567 ms. Trace:
W20260709 13:19:43.824138 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 51761.5us
I20260709 13:19:43.794940 16294 raft_consensus.cc:2393] T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 3: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
I20260709 13:19:43.841284 18298 raft_consensus.cc:2804] T af57996a8be84d398f0f023e8d08ef77 P cb82cc7bd6db46d980c75237a1402860 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:19:43.812403 16144 raft_consensus.cc:3060] T 829b4ad8c5a54eababeea1181b960d4b P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:19:43.825613 16146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806d7690c2b644959152074bb2cab0a7" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
W20260709 13:19:43.825964 16115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 61543.2us
I20260709 13:19:43.611578 16149 raft_consensus.cc:2468] T af57996a8be84d398f0f023e8d08ef77 P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 4.
I20260709 13:19:43.841110 18123 raft_consensus.cc:493] T 84f2da98864b4425bf7c0055951fdd4b P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:43.840734 18041 raft_consensus.cc:493] T 57a744f5aaf944678a63469cbf514015 P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:43.886709 16115 leader_election.cc:304] T ecc1e363230043fe9fef390f0b979f20 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
W20260709 13:19:43.825985 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 340682us
I20260709 13:19:43.739404 16047 catalog_manager.cc:5697] T b6f07842d3aa47858df5665874c86f3a P 1ca2725d290e47d68b1a4b609c9816ac reported cstate change: term changed from 1 to 2, leader changed from 90f3da57796b488c9087a2f523adf045 (127.14.177.131) to 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130). New cstate: current_term: 2 leader_uuid: "1ca2725d290e47d68b1a4b609c9816ac" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } health_report { overall_health: HEALTHY } } }
I20260709 13:19:43.804024 16300 raft_consensus.cc:3060] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:19:43.892690 16264 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:43.392905 18271 raft_consensus.cc:493] T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:43.853317 18255 raft_consensus.cc:2804] T 7c2095a5f4ad49408dbc7f69c606120d P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Leader election won for term 3
W20260709 13:19:43.454017 16226 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33050 (request call id 403) took 1663 ms (client timeout 1686 ms). Trace:
W20260709 13:19:43.898492 16226 rpcz_store.cc:269] 0709 13:19:41.790143 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:41.801708 (+ 11565us) service_pool.cc:224] Handling call
0709 13:19:43.453772 (+1652064us) spinlock_profiling.cc:243] Waited 238 ms on lock 0x7b54002c9a54. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:19:43.453948 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":548027136}
I20260709 13:19:43.899533 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ad5dbfa99ee4b8d9e28be92b3f36db5" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:43.840339 16296 raft_consensus.cc:2468] T cdbd78acb4cf46298e0de8ba0d130e19 P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 4.
I20260709 13:19:43.840307 16191 leader_election.cc:304] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
W20260709 13:19:43.900938 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 60677.7us
I20260709 13:19:43.850446 16147 raft_consensus.cc:2468] T b87147731a68400e8cce9b879daa7c5d P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 5.
I20260709 13:19:43.850597 16823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7ca61592e848ea9b0fb81a2ce33cdc" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 7 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:19:43.852804 16148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fffab26427e24d398e2695aad40ca134" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:19:43.902011 16147 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46224 (request call id 262) took 1072 ms. Trace:
I20260709 13:19:43.901904 16191 leader_election.cc:304] T 5a959dbecc2648eb85d1cc550f5a3245 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
I20260709 13:19:43.784021 16297 rpcz_store.cc:276] 0709 13:19:40.926487 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:41.023205 (+ 96718us) service_pool.cc:224] Handling call
0709 13:19:41.036780 (+ 13575us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:41.639999 (+603219us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:43.426625 (+1786626us) spinlock_profiling.cc:243] Waited 508 ms on lock 0x7b54001ef354. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:19:43.426776 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1168185216}
I20260709 13:19:43.854059 16820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31969392ccdb43b491911a319e8bc7e1" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:19:43.785372 18042 leader_election.cc:290] T 66f75124215f4ca6bef831e174a792d8 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:19:43.854324 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada5a004ba7c46819051e3ccec71ffdb" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:43.904781 16225 raft_consensus.cc:2410] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 for term 6 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:19:43.902403 16148 raft_consensus.cc:2468] T fffab26427e24d398e2695aad40ca134 P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 2.
I20260709 13:19:43.855868 16223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c61a39fd4f84424b129c1a6f9ba404d" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:43.856055 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1df93779209748b9860f6d893dd3de53" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:19:43.858078 16145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2372981006d4b218215c28964dea623" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:19:43.906435 16223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d4ab870a05347868956aa8b841b2981" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:43.906739 16145 raft_consensus.cc:2393] T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 in current term 4: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
I20260709 13:19:43.858409 17992 raft_consensus.cc:2804] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:19:43.855279 18272 raft_consensus.cc:2749] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:19:43.859059 16821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8638ee363273426db1393dd81e5c7b72" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:19:43.852815 18104 raft_consensus.cc:2804] T d16dfad0c3b540e5b4b86492ddf87208 P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:19:43.860450 16817 raft_consensus.cc:1240] T 2794af5b99984efebdee564035841cb0 P cb82cc7bd6db46d980c75237a1402860 [term 3 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 2. Current term is 3. Ops: []
I20260709 13:19:43.860783 18244 consensus_queue.cc:1048] T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:19:43.861472 16151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57a744f5aaf944678a63469cbf514015" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:19:43.865265 16190 leader_election.cc:304] T 8a878999c0024d2abe517484180290fc P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:19:43.866480 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f5f89186204242bf377f7c42b2e2da" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:43.922765 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cc959fb7f704fba9009c0595467d074" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:19:43.869995 16822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434bff3082d141838068f323bdb0d1ed" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:19:43.925858 16822 raft_consensus.cc:2468] T 434bff3082d141838068f323bdb0d1ed P cb82cc7bd6db46d980c75237a1402860 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 4.
I20260709 13:19:43.872406 16825 raft_consensus.cc:1275] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P cb82cc7bd6db46d980c75237a1402860 [term 3 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
W20260709 13:19:43.926553 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 61401.6us
I20260709 13:19:43.870923 16192 leader_election.cc:400] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 3 election: Vote denied by peer cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923) with higher term. Message: Invalid argument: T 74ca8140f35c4ae29e8a664f6bdfc829 P cb82cc7bd6db46d980c75237a1402860 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac for earlier term 3. Current term is 4.
I20260709 13:19:43.927264 16190 leader_election.cc:304] T 19baa7bc367844c288bca8f81439c413 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:19:43.927552 16192 leader_election.cc:403] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 3 election: Cancelling election due to peer responding with higher term
I20260709 13:19:43.881832 16790 leader_election.cc:304] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 
W20260709 13:19:43.928308 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 46615.8us
I20260709 13:19:43.864830 16222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ab0ac8347641e6955e9bce7d90c1a2" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:43.929270 16222 raft_consensus.cc:2393] T d2ab0ac8347641e6955e9bce7d90c1a2 P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 5: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
I20260709 13:19:43.883744 16295 rpcz_store.cc:276] 0709 13:19:41.824626 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:42.324869 (+500243us) service_pool.cc:224] Handling call
0709 13:19:42.344717 (+ 19848us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:42.511700 (+166983us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:42.511729 (+    29us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:42.511780 (+    51us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:19:42.511818 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:43.391579 (+879761us) spinlock_profiling.cc:243] Waited 167 ms on lock 0x7b54000ce254. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:19:43.391653 (+    74us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:43.391838 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":383561600}
I20260709 13:19:43.402539 18145 raft_consensus.cc:515] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:43.930184 16151 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46224 (request call id 255) took 1246 ms. Trace:
I20260709 13:19:43.913439 17992 raft_consensus.cc:493] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:43.887923 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18ad7e3af5504571a69d10c00750a01c" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:43.850535 16220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7ca61592e848ea9b0fb81a2ce33cdc" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 7 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:43.903753 18041 raft_consensus.cc:515] T 57a744f5aaf944678a63469cbf514015 P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:43.920200 18042 raft_consensus.cc:2804] T 5a959dbecc2648eb85d1cc550f5a3245 P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:19:43.896373 16146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ad5dbfa99ee4b8d9e28be92b3f36db5" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:19:43.927275 18204 raft_consensus.cc:515] T 9e65676b21b748728ad39668cfe722ec P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:43.913610 18298 raft_consensus.cc:697] T af57996a8be84d398f0f023e8d08ef77 P cb82cc7bd6db46d980c75237a1402860 [term 4 LEADER]: Becoming Leader. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:19:43.920347 16143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c74490c40243a88e62d1760b5cd814" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
W20260709 13:19:43.939734 16192 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 69002.2us
I20260709 13:19:43.941745 16790 leader_election.cc:304] T 66f75124215f4ca6bef831e174a792d8 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:19:43.942998 16295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a9de347c711477e87c582d003f36925" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:43.904803 16823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a378d7f796f94615b3b347119a9880fd" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:19:43.944324 16295 raft_consensus.cc:2468] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 3.
I20260709 13:19:43.908913 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68a6838d82ef4cac83507f906139d90f" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:43.909221 16145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66f75124215f4ca6bef831e174a792d8" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:19:43.944669 16192 leader_election.cc:304] T 66f75124215f4ca6bef831e174a792d8 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:19:43.945067 16224 raft_consensus.cc:2468] T 68a6838d82ef4cac83507f906139d90f P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 5.
I20260709 13:19:43.945281 16145 raft_consensus.cc:2393] T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 in current term 5: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
I20260709 13:19:43.867619 18109 raft_consensus.cc:2804] T e4dc6b0541ca431ebc757beac00521f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:19:43.903311 18154 raft_consensus.cc:493] T 8c11074579b046d99e585c2818e18a2c P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:43.915584 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18ad7e3af5504571a69d10c00750a01c" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:43.915935 16150 raft_consensus.cc:2393] T 1df93779209748b9860f6d893dd3de53 P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 4: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
I20260709 13:19:43.868340 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a959dbecc2648eb85d1cc550f5a3245" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:43.909781 16047 catalog_manager.cc:5697] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 1ca2725d290e47d68b1a4b609c9816ac reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "cb82cc7bd6db46d980c75237a1402860" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:19:43.932812 18272 raft_consensus.cc:2804] T 8a878999c0024d2abe517484180290fc P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:19:43.922765 16115 leader_election.cc:304] T b6f07842d3aa47858df5665874c86f3a P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:19:43.923630 16300 raft_consensus.cc:2468] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 4.
I20260709 13:19:43.904850 18243 raft_consensus.cc:2804] T ecc1e363230043fe9fef390f0b979f20 P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:19:43.965066 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6f07842d3aa47858df5665874c86f3a" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:43.928581 16190 leader_election.cc:304] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: cb82cc7bd6db46d980c75237a1402860
I20260709 13:19:43.791893 18034 consensus_queue.cc:237] T 829b4ad8c5a54eababeea1181b960d4b P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:43.930460 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a378d7f796f94615b3b347119a9880fd" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:43.909847 18114 raft_consensus.cc:2804] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 13:19:43.836885 16265 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:43.931109 16151 rpcz_store.cc:276] 0709 13:19:42.683599 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:43.848012 (+1164413us) service_pool.cc:224] Handling call
0709 13:19:43.930113 (+ 82101us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:19:43.921856 18189 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.553s	user 0.029s	sys 0.021s
I20260709 13:19:43.931819 16825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ab0ac8347641e6955e9bce7d90c1a2" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:19:43.932689 16296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0eab25ce4c64e018a041beeda2867db" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:19:43.981441 16296 raft_consensus.cc:3060] T c0eab25ce4c64e018a041beeda2867db P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:19:43.944268 18244 raft_consensus.cc:3060] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:19:43.933691 16294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f5f89186204242bf377f7c42b2e2da" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:43.932092 16222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 4 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:43.934482 18042 raft_consensus.cc:493] T 5a959dbecc2648eb85d1cc550f5a3245 P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:43.930102 18250 raft_consensus.cc:2804] T 19baa7bc367844c288bca8f81439c413 P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:19:43.896364 16116 leader_election.cc:304] T 1df93779209748b9860f6d893dd3de53 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 90f3da57796b488c9087a2f523adf045
W20260709 13:19:44.009087 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 112764us
I20260709 13:19:43.915740 18255 raft_consensus.cc:697] T 7c2095a5f4ad49408dbc7f69c606120d P 1ca2725d290e47d68b1a4b609c9816ac [term 3 LEADER]: Becoming Leader. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:19:43.900108 16226 raft_consensus.cc:2468] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 3.
I20260709 13:19:44.068295 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f2da98864b4425bf7c0055951fdd4b" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:19:44.068867 16226 raft_consensus.cc:3060] T 84f2da98864b4425bf7c0055951fdd4b P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:19:43.935454 16146 raft_consensus.cc:2468] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 3.
I20260709 13:19:43.940157 16144 raft_consensus.cc:2468] T 829b4ad8c5a54eababeea1181b960d4b P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 5.
I20260709 13:19:43.902423 16147 rpcz_store.cc:276] 0709 13:19:42.829956 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:43.793869 (+963913us) service_pool.cc:224] Handling call
0709 13:19:43.901960 (+108091us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:44.070660 16147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746aabf856ab401c9ef5e859371e7b67" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:19:43.942875 18177 consensus_queue.cc:1048] T ada5a004ba7c46819051e3ccec71ffdb P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.043s
I20260709 13:19:43.946131 16145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57a744f5aaf944678a63469cbf514015" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:19:43.955386 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a878999c0024d2abe517484180290fc" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:43.939908 16298 raft_consensus.cc:2468] T 18ad7e3af5504571a69d10c00750a01c P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 3.
I20260709 13:19:44.072451 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e4bfb9090c8460eb14fa7e0d818e7f1" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:19:43.953864 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fffab26427e24d398e2695aad40ca134" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:43.905737 18123 raft_consensus.cc:515] T 84f2da98864b4425bf7c0055951fdd4b P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:43.926822 16819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66ae176961b74813a58bc15c9c4c6f2a" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:19:43.971009 18041 leader_election.cc:290] T 57a744f5aaf944678a63469cbf514015 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:19:43.960215 18154 raft_consensus.cc:515] T 8c11074579b046d99e585c2818e18a2c P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:43.956362 16142 raft_consensus.cc:2368] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 for earlier term 5. Current term is 6.
I20260709 13:19:43.962416 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68a6838d82ef4cac83507f906139d90f" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:44.112249 16300 raft_consensus.cc:2410] T 68a6838d82ef4cac83507f906139d90f P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac for term 6 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:19:43.962576 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a9de347c711477e87c582d003f36925" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:19:44.112776 16150 raft_consensus.cc:2468] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 4.
I20260709 13:19:44.113646 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87147731a68400e8cce9b879daa7c5d" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:19:43.934242 16220 raft_consensus.cc:2410] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 1ca2725d290e47d68b1a4b609c9816ac [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 for term 7 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:19:43.963902 16295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f5f89186204242bf377f7c42b2e2da" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:19:43.837854 16291 rpcz_store.cc:276] 0709 13:19:41.825977 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:42.355098 (+529121us) service_pool.cc:224] Handling call
0709 13:19:42.446622 (+ 91524us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:19:42.459786 (+ 13164us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:42.459840 (+    54us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:19:42.531910 (+ 72070us) log.cc:844] Serialized 52 byte log entry
0709 13:19:42.837418 (+305508us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:19:42.857544 (+ 20126us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:42.866562 (+  9018us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:43.395250 (+528688us) raft_consensus.cc:1685] finished
0709 13:19:43.395291 (+    41us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:43.759588 (+364297us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":9051,"raft.run_cpu_time_us":174,"raft.run_wall_time_us":3964,"thread_start_us":266189,"threads_started":1,"wal-append.queue_time_us":451202}
W20260709 13:19:43.967357 16115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 44662.3us
I20260709 13:19:44.120682 16115 leader_election.cc:304] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:19:43.939494 17992 raft_consensus.cc:3060] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:19:43.971551 16265 leader_election.cc:304] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
W20260709 13:19:44.123560 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 152108us
I20260709 13:19:44.124733 16265 leader_election.cc:304] T a378d7f796f94615b3b347119a9880fd P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:19:44.126192 16265 leader_election.cc:400] T d16dfad0c3b540e5b4b86492ddf87208 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 2 election: Vote denied by peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943) with higher term. Message: Invalid argument: T d16dfad0c3b540e5b4b86492ddf87208 P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 for earlier term 2. Current term is 3.
I20260709 13:19:44.126488 16265 leader_election.cc:403] T d16dfad0c3b540e5b4b86492ddf87208 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 2 election: Cancelling election due to peer responding with higher term
I20260709 13:19:43.982892 16222 raft_consensus.cc:2410] T ecc1e363230043fe9fef390f0b979f20 P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 for term 4 because replica has last-logged OpId of term: 2 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 3.
I20260709 13:19:43.904286 18262 consensus_queue.cc:1048] T 18ad7e3af5504571a69d10c00750a01c P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.026s
I20260709 13:19:43.985217 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada5a004ba7c46819051e3ccec71ffdb" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:43.941161 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:19:43.957031 18179 raft_consensus.cc:2804] T 66f75124215f4ca6bef831e174a792d8 P cb82cc7bd6db46d980c75237a1402860 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:19:43.954962 18109 raft_consensus.cc:493] T e4dc6b0541ca431ebc757beac00521f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Starting leader election (detected failure of leader 647a343a542d4a139fe106c19095129b)
W20260709 13:19:43.980943 18189 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.620s	user 0.029s	sys 0.022s
I20260709 13:19:44.069908 16144 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46224 (request call id 260) took 1384 ms. Trace:
I20260709 13:19:44.145485 16144 rpcz_store.cc:276] 0709 13:19:42.685146 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:43.450595 (+765449us) service_pool.cc:224] Handling call
0709 13:19:44.069838 (+619243us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:44.146560 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ab0ac8347641e6955e9bce7d90c1a2" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:19:44.146720 16144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68a6838d82ef4cac83507f906139d90f" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:19:44.147513 16299 raft_consensus.cc:3060] T d2ab0ac8347641e6955e9bce7d90c1a2 P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:19:44.148396 16144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d4ab870a05347868956aa8b841b2981" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:19:43.906044 18271 raft_consensus.cc:515] T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:43.964915 18243 raft_consensus.cc:493] T ecc1e363230043fe9fef390f0b979f20 P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:43.956666 16822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f82224f4564fea93572e8206bad1fe" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:19:44.161867 16822 raft_consensus.cc:2468] T 23f82224f4564fea93572e8206bad1fe P cb82cc7bd6db46d980c75237a1402860 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 1.
I20260709 13:19:44.081213 16151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62fe3346cd3c42c4a55e14af9073bca4" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:19:44.083842 16294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1df93779209748b9860f6d893dd3de53" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:44.082255 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ad5dbfa99ee4b8d9e28be92b3f36db5" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:44.081373 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c74490c40243a88e62d1760b5cd814" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:44.168402 16297 raft_consensus.cc:2393] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 5: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
I20260709 13:19:43.954974 18057 consensus_queue.cc:1048] T e5ad4eb281564936a59f41e387a47156 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.056s
I20260709 13:19:44.099009 16821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87147731a68400e8cce9b879daa7c5d" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:19:44.103250 18257 leader_election.cc:290] T 84f2da98864b4425bf7c0055951fdd4b P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:44.090548 18298 consensus_queue.cc:237] T af57996a8be84d398f0f023e8d08ef77 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:44.088279 16825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62fe3346cd3c42c4a55e14af9073bca4" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:19:44.182811 16825 raft_consensus.cc:2468] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 5.
I20260709 13:19:44.102751 18254 leader_election.cc:290] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 6 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:19:44.100660 18123 leader_election.cc:290] T 84f2da98864b4425bf7c0055951fdd4b P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:19:44.100900 16148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31969392ccdb43b491911a319e8bc7e1" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:19:44.194548 16148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a878999c0024d2abe517484180290fc" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:19:44.009755 18250 raft_consensus.cc:493] T 19baa7bc367844c288bca8f81439c413 P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Starting leader election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:19:44.090142 16296 raft_consensus.cc:2468] T c0eab25ce4c64e018a041beeda2867db P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 4.
I20260709 13:19:44.104527 16224 raft_consensus.cc:2468] T b6f07842d3aa47858df5665874c86f3a P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 3.
I20260709 13:19:43.957489 16047 catalog_manager.cc:5697] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac reported cstate change: term changed from 2 to 4, leader changed from cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) to 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130). New cstate: current_term: 4 leader_uuid: "1ca2725d290e47d68b1a4b609c9816ac" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:19:43.961489 18272 raft_consensus.cc:493] T 8a878999c0024d2abe517484180290fc P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Starting leader election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:19:44.100379 17918 consensus_queue.cc:1048] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:19:44.067286 16817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74ca8140f35c4ae29e8a664f6bdfc829" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:19:44.106208 18256 leader_election.cc:290] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:19:44.008311 18042 raft_consensus.cc:3060] T 5a959dbecc2648eb85d1cc550f5a3245 P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:19:44.106781 16145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31969392ccdb43b491911a319e8bc7e1" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
W20260709 13:19:44.107234 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 205427us
I20260709 13:19:43.982402 18240 raft_consensus.cc:3060] T b6f07842d3aa47858df5665874c86f3a P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:19:43.895334 16818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a9de347c711477e87c582d003f36925" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:19:43.981729 18114 raft_consensus.cc:493] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:44.116652 16146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d16dfad0c3b540e5b4b86492ddf87208" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
W20260709 13:19:44.088747 18305 log.cc:927] Time spent T 3884f864a96d428785895e5463e5728f P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 0.158s	user 0.000s	sys 0.000s
W20260709 13:19:44.093848 18304 log.cc:927] Time spent T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 0.170s	user 0.000s	sys 0.003s
I20260709 13:19:44.128211 16147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee9ed67006144797b1de6ea8c2013a68" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:19:44.130061 16115 leader_election.cc:304] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:19:44.129940 16295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:44.114179 16150 raft_consensus.cc:2468] T b87147731a68400e8cce9b879daa7c5d P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 5.
I20260709 13:19:44.067469 18276 raft_consensus.cc:493] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:44.074544 18173 raft_consensus.cc:2804] T 1df93779209748b9860f6d893dd3de53 P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:19:44.133208 16222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c74490c40243a88e62d1760b5cd814" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:44.107491 18145 leader_election.cc:290] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 6 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:19:44.139102 16226 raft_consensus.cc:2468] T 84f2da98864b4425bf7c0055951fdd4b P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 3.
I20260709 13:19:44.320896 16265 leader_election.cc:304] T e4dc6b0541ca431ebc757beac00521f1 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:19:44.321439 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74ca8140f35c4ae29e8a664f6bdfc829" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:44.010740 16116 leader_election.cc:304] T 8cc959fb7f704fba9009c0595467d074 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:19:43.866288 16217 rpcz_store.cc:276] 0709 13:19:41.817971 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:42.514524 (+696553us) service_pool.cc:224] Handling call
0709 13:19:42.999546 (+485022us) spinlock_profiling.cc:243] Waited 2.1 ms on lock 0x7b5800010b98. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69ad8fadb1 00007f69ada3ab71 00007f69ada3aeee 0000000000375939 00007f69b5aae1e0 00007f69b5b770c4 00007f69b5b77131 00007f69b5b38fc0 00007f69b5b23929 00007f69b5b233d2 00007f69af09abd4
0709 13:19:42.999722 (+   176us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:43.000140 (+   418us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:19:43.019239 (+ 19099us) log.cc:844] Serialized 14 byte log entry
0709 13:19:43.322073 (+302834us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:43.329919 (+  7846us) log.cc:844] Serialized 26 byte log entry
0709 13:19:43.340577 (+ 10658us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:19:43.436638 (+ 96061us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:43.436761 (+   123us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:43.612941 (+176180us) raft_consensus.cc:1685] finished
0709 13:19:43.613284 (+   343us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:43.773589 (+160305us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":179,"prepare.run_cpu_time_us":37,"prepare.run_wall_time_us":38,"raft.queue_time_us":3065,"raft.run_cpu_time_us":237,"raft.run_wall_time_us":238,"spinlock_wait_cycles":4838656,"thread_start_us":258801,"threads_started":1,"wal-append.queue_time_us":285780}
W20260709 13:19:44.322556 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 311882us
I20260709 13:19:44.082334 16219 raft_consensus.cc:2468] T 66f75124215f4ca6bef831e174a792d8 P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 5.
I20260709 13:19:44.323717 16217 raft_consensus.cc:1240] T ee9ed67006144797b1de6ea8c2013a68 P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 3. Current term is 4. Ops: []
I20260709 13:19:44.324086 18154 leader_election.cc:290] T 8c11074579b046d99e585c2818e18a2c P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:19:44.324465 16217 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57604 (request call id 1953) took 2504 ms. Trace:
I20260709 13:19:44.324767 18154 consensus_queue.cc:1059] T 8cc959fb7f704fba9009c0595467d074 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:19:44.324891 16217 rpcz_store.cc:276] 0709 13:19:41.819829 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:44.323187 (+2503358us) service_pool.cc:224] Handling call
0709 13:19:44.323520 (+   333us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:44.324182 (+   662us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:44.324404 (+   222us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:44.325825 18154 raft_consensus.cc:2749] T e4dc6b0541ca431ebc757beac00521f1 P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:19:44.326485 18264 raft_consensus.cc:2749] T 8cc959fb7f704fba9009c0595467d074 P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
W20260709 13:19:44.140563 16192 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 195942us
I20260709 13:19:44.327489 16192 leader_election.cc:304] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:19:44.204113 16151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 4 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:19:44.117856 17844 leader_election.cc:290] T ee9ed67006144797b1de6ea8c2013a68 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 4 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:19:44.165805 16144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c61a39fd4f84424b129c1a6f9ba404d" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:19:44.129709 18104 raft_consensus.cc:697] T d16dfad0c3b540e5b4b86492ddf87208 P 647a343a542d4a139fe106c19095129b [term 3 LEADER]: Becoming Leader. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:19:44.253130 16294 raft_consensus.cc:2410] T 1df93779209748b9860f6d893dd3de53 P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac for term 4 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:19:44.133296 18041 raft_consensus.cc:2804] T a378d7f796f94615b3b347119a9880fd P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:19:44.133564 18140 raft_consensus.cc:2804] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:19:44.133687 17999 raft_consensus.cc:2804] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:19:44.133076 18179 raft_consensus.cc:493] T 66f75124215f4ca6bef831e174a792d8 P cb82cc7bd6db46d980c75237a1402860 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:44.141052 18156 raft_consensus.cc:2749] T d16dfad0c3b540e5b4b86492ddf87208 P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Leader election lost for term 2. Reason: Vote denied by peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943) with higher term. Message: Invalid argument: T d16dfad0c3b540e5b4b86492ddf87208 P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 for earlier term 2. Current term is 3.
I20260709 13:19:44.144428 18109 raft_consensus.cc:3060] T e4dc6b0541ca431ebc757beac00521f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:19:44.191797 16821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af57996a8be84d398f0f023e8d08ef77" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:19:44.350869 16147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a9de347c711477e87c582d003f36925" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:19:44.338359 18114 raft_consensus.cc:3060] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:19:44.194890 16299 raft_consensus.cc:2468] T d2ab0ac8347641e6955e9bce7d90c1a2 P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 5.
I20260709 13:19:44.169790 18243 raft_consensus.cc:3060] T ecc1e363230043fe9fef390f0b979f20 P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:19:44.203063 16790 connection.cc:256] RPC call timeout handler was delayed by -2.3751214630001414s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18660000000000002s for a response
I20260709 13:19:44.182863 16291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a96373bb0e14dce8a266b2fc6c58507" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
W20260709 13:19:44.208827 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 280294us
I20260709 13:19:44.341570 18319 raft_consensus.cc:2804] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:19:44.347012 18140 raft_consensus.cc:493] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:44.354633 16265 leader_election.cc:304] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:19:44.370831 16265 leader_election.cc:400] T 66ae176961b74813a58bc15c9c4c6f2a P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 5 pre-election: Vote denied by peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943) with higher term. Message: Invalid argument: T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 for earlier term 5. Current term is 6.
I20260709 13:19:44.203891 16047 catalog_manager.cc:5697] T 02838c86531d4f98bf58d1c22b8baf77 P 1ca2725d290e47d68b1a4b609c9816ac reported cstate change: term changed from 2 to 4, leader changed from 90f3da57796b488c9087a2f523adf045 (127.14.177.131) to 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130). New cstate: current_term: 4 leader_uuid: "1ca2725d290e47d68b1a4b609c9816ac" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } health_report { overall_health: UNKNOWN } } }
I20260709 13:19:44.373067 16265 leader_election.cc:403] T 66ae176961b74813a58bc15c9c4c6f2a P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 5 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:19:44.208951 16148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4f65cdf00b748bb963ee6e81ef43b79" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 7 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
W20260709 13:19:44.316888 16189 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:44.349839 18179 raft_consensus.cc:3060] T 66f75124215f4ca6bef831e174a792d8 P cb82cc7bd6db46d980c75237a1402860 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:19:44.201498 16819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:19:44.209925 16220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4f65cdf00b748bb963ee6e81ef43b79" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 7 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:44.087711 16218 raft_consensus.cc:2468] T c2f5f89186204242bf377f7c42b2e2da P 1ca2725d290e47d68b1a4b609c9816ac [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 6.
I20260709 13:19:44.159260 16823 raft_consensus.cc:1240] T c9944d5e4b8c4109af9955c4f08c9095 P cb82cc7bd6db46d980c75237a1402860 [term 5 LEADER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 4. Current term is 5. Ops: []
I20260709 13:19:44.317706 16145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5d59407eb545b895d8d922e0dd69fd" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:19:44.183578 16293 raft_consensus.cc:2468] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 3.
I20260709 13:19:44.253712 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a9de347c711477e87c582d003f36925" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:19:44.328675 16222 raft_consensus.cc:2393] T 08c74490c40243a88e62d1760b5cd814 P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 5: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
I20260709 13:19:44.344645 18276 raft_consensus.cc:515] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:44.407752 16298 raft_consensus.cc:2376] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 3. Re-sending same reply.
I20260709 13:19:44.332973 16295 raft_consensus.cc:2468] T ecc1e363230043fe9fef390f0b979f20 P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 3.
I20260709 13:19:44.333426 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57a744f5aaf944678a63469cbf514015" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:19:44.257264 16191 leader_election.cc:304] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
W20260709 13:19:44.458803 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 201629us
I20260709 13:19:44.234666 16296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d16dfad0c3b540e5b4b86492ddf87208" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:44.334658 16825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62fe3346cd3c42c4a55e14af9073bca4" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:19:44.177913 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9944d5e4b8c4109af9955c4f08c9095" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:44.460451 16825 raft_consensus.cc:3060] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:19:44.460577 16296 raft_consensus.cc:2393] T d16dfad0c3b540e5b4b86492ddf87208 P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 3: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
I20260709 13:19:44.203250 18250 raft_consensus.cc:3060] T 19baa7bc367844c288bca8f81439c413 P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:19:44.210490 18105 raft_consensus.cc:493] T 9c61a39fd4f84424b129c1a6f9ba404d P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:44.341552 16151 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40830 (request call id 1040) took 1004 ms. Trace:
I20260709 13:19:44.466549 16151 rpcz_store.cc:276] 0709 13:19:43.337454 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:44.175974 (+838520us) service_pool.cc:224] Handling call
0709 13:19:44.341486 (+165512us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:44.343346 16117 leader_election.cc:304] T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860
I20260709 13:19:44.467201 16191 leader_election.cc:304] T 5a96373bb0e14dce8a266b2fc6c58507 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
W20260709 13:19:44.467876 16117 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 124623us
I20260709 13:19:44.468364 16191 leader_election.cc:304] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
I20260709 13:19:44.210655 18272 raft_consensus.cc:3060] T 8a878999c0024d2abe517484180290fc P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:19:44.343348 18041 raft_consensus.cc:493] T a378d7f796f94615b3b347119a9880fd P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Starting leader election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:19:44.326540 18154 raft_consensus.cc:3055] T 8cc959fb7f704fba9009c0595467d074 P 90f3da57796b488c9087a2f523adf045 [term 3 LEADER]: Stepping down as leader of term 3
W20260709 13:19:44.346089 16115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 216130us
I20260709 13:19:44.197021 18318 raft_consensus.cc:2804] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:19:44.477027 16115 leader_election.cc:304] T 5a96373bb0e14dce8a266b2fc6c58507 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:19:44.343436 17844 raft_consensus.cc:493] T 746aabf856ab401c9ef5e859371e7b67 P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:44.201059 18255 consensus_queue.cc:237] T 7c2095a5f4ad49408dbc7f69c606120d P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:44.191799 16223 raft_consensus.cc:2468] T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 4.
I20260709 13:19:44.352280 16821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee9ed67006144797b1de6ea8c2013a68" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:19:44.492221 16821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66f75124215f4ca6bef831e174a792d8" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:19:44.352625 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26354361e9b044cea1c2b86c4882fe92" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:44.493647 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f2da98864b4425bf7c0055951fdd4b" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:19:44.329435 18310 raft_consensus.cc:2764] T 66f75124215f4ca6bef831e174a792d8 P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Leader pre-election decision vote started in defunct term 4: won
I20260709 13:19:44.494169 16299 raft_consensus.cc:3060] T 84f2da98864b4425bf7c0055951fdd4b P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:19:44.360363 16190 leader_election.cc:304] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860
I20260709 13:19:44.209131 16822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5d59407eb545b895d8d922e0dd69fd" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:19:44.495918 16822 raft_consensus.cc:3060] T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:19:44.496186 18309 log.cc:927] Time spent T 18ad7e3af5504571a69d10c00750a01c P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 0.214s	user 0.000s	sys 0.005s
W20260709 13:19:44.497480 18316 log.cc:927] Time spent T 829b4ad8c5a54eababeea1181b960d4b P cb82cc7bd6db46d980c75237a1402860: Append to log took a long time: real 0.216s	user 0.003s	sys 0.004s
I20260709 13:19:44.363083 16292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66ae176961b74813a58bc15c9c4c6f2a" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:19:44.364210 18271 leader_election.cc:290] T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:44.499907 17992 raft_consensus.cc:515] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:44.353338 16818 raft_consensus.cc:2368] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 for earlier term 4. Current term is 5.
I20260709 13:19:44.498880 18250 raft_consensus.cc:515] T 19baa7bc367844c288bca8f81439c413 P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } }
I20260709 13:19:44.501475 16818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a9de347c711477e87c582d003f36925" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:19:44.502256 16818 raft_consensus.cc:2393] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 5: Already voted for candidate cb82cc7bd6db46d980c75237a1402860 in this term.
I20260709 13:19:44.502671 18250 leader_election.cc:290] T 19baa7bc367844c288bca8f81439c413 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 5 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:44.318768 16817 raft_consensus.cc:2393] T 74ca8140f35c4ae29e8a664f6bdfc829 P cb82cc7bd6db46d980c75237a1402860 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 4: Already voted for candidate cb82cc7bd6db46d980c75237a1402860 in this term.
I20260709 13:19:44.503304 18250 raft_consensus.cc:2749] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:19:44.349030 18264 raft_consensus.cc:2804] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:19:44.503942 18264 raft_consensus.cc:697] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 647a343a542d4a139fe106c19095129b [term 4 LEADER]: Becoming Leader. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:19:44.504014 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e65676b21b748728ad39668cfe722ec" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:19:44.345993 18173 raft_consensus.cc:697] T 1df93779209748b9860f6d893dd3de53 P 647a343a542d4a139fe106c19095129b [term 4 LEADER]: Becoming Leader. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:19:44.375725 16265 leader_election.cc:304] T e5ad4eb281564936a59f41e387a47156 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:19:44.377388 16047 catalog_manager.cc:5697] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac reported cstate change: term changed from 2 to 4, leader changed from cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) to 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130). New cstate: current_term: 4 leader_uuid: "1ca2725d290e47d68b1a4b609c9816ac" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:19:44.359280 16816 raft_consensus.cc:2468] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 6.
I20260709 13:19:44.466427 18257 raft_consensus.cc:2749] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:19:44.382023 16148 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40830 (request call id 1045) took 1024 ms. Trace:
I20260709 13:19:44.383502 16145 raft_consensus.cc:2393] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 in current term 5: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
I20260709 13:19:44.328792 16116 leader_election.cc:304] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
W20260709 13:19:44.514343 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 191143us
I20260709 13:19:44.516342 16116 leader_election.cc:304] T 84f2da98864b4425bf7c0055951fdd4b P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
I20260709 13:19:44.385106 16222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c11074579b046d99e585c2818e18a2c" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
W20260709 13:19:44.517254 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 163121us
I20260709 13:19:44.375834 18140 raft_consensus.cc:3060] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:19:44.475764 18041 raft_consensus.cc:3060] T a378d7f796f94615b3b347119a9880fd P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:19:44.364229 16143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26354361e9b044cea1c2b86c4882fe92" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:19:44.377015 18156 raft_consensus.cc:2804] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:19:44.386672 18244 raft_consensus.cc:2749] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Leader election lost for term 3. Reason: Vote denied by peer cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923) with higher term. Message: Invalid argument: T 74ca8140f35c4ae29e8a664f6bdfc829 P cb82cc7bd6db46d980c75237a1402860 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac for earlier term 3. Current term is 4.
I20260709 13:19:44.387534 18320 raft_consensus.cc:3060] T 66ae176961b74813a58bc15c9c4c6f2a P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Advancing to term 6
I20260709 13:19:44.521899 18181 raft_consensus.cc:2804] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:19:44.530408 18181 raft_consensus.cc:493] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:44.530758 18181 raft_consensus.cc:3060] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:19:44.388600 16291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c2095a5f4ad49408dbc7f69c606120d" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:44.531478 18173 consensus_queue.cc:237] T 1df93779209748b9860f6d893dd3de53 P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:44.532284 16291 raft_consensus.cc:2410] T 7c2095a5f4ad49408dbc7f69c606120d P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac for term 3 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:19:44.516245 18167 raft_consensus.cc:2749] T e5ad4eb281564936a59f41e387a47156 P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:19:44.389906 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ad5dbfa99ee4b8d9e28be92b3f36db5" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:44.534297 16293 raft_consensus.cc:2468] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 3.
I20260709 13:19:44.526456 18260 raft_consensus.cc:2804] T 84f2da98864b4425bf7c0055951fdd4b P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:19:44.384069 18104 consensus_queue.cc:237] T d16dfad0c3b540e5b4b86492ddf87208 P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:44.535763 18260 raft_consensus.cc:697] T 84f2da98864b4425bf7c0055951fdd4b P 647a343a542d4a139fe106c19095129b [term 3 LEADER]: Becoming Leader. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:19:44.395426 16294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74ca8140f35c4ae29e8a664f6bdfc829" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:19:44.536862 16294 raft_consensus.cc:3060] T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:19:44.536600 18260 consensus_queue.cc:237] T 84f2da98864b4425bf7c0055951fdd4b P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:44.396317 16220 raft_consensus.cc:2410] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 for term 7 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:19:44.538166 16220 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57600 (request call id 1027) took 1202 ms. Trace:
I20260709 13:19:44.529582 16143 raft_consensus.cc:2468] T 26354361e9b044cea1c2b86c4882fe92 P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 2.
I20260709 13:19:44.538532 16220 rpcz_store.cc:276] 0709 13:19:43.336252 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:44.174800 (+838548us) service_pool.cc:224] Handling call
0709 13:19:44.538105 (+363305us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:44.346977 18313 consensus_queue.cc:1048] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.038s
I20260709 13:19:44.539453 16143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f2da98864b4425bf7c0055951fdd4b" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:19:44.397398 16221 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57604 (request call id 1948) took 2579 ms. Trace:
I20260709 13:19:44.540671 16143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5718c623ebee4d2bb770f0f9ddb2f164" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:19:44.528671 18156 raft_consensus.cc:493] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:19:44.384387 16819 raft_consensus.cc:2468] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 3.
W20260709 13:19:44.431565 16263 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:44.467033 16264 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:44.473033 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7ca61592e848ea9b0fb81a2ce33cdc" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:44.475735 16296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5ad4eb281564936a59f41e387a47156" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
W20260709 13:19:44.462397 16788 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:44.477978 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ad5dbfa99ee4b8d9e28be92b3f36db5" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:19:44.380116 18319 raft_consensus.cc:697] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac [term 4 LEADER]: Becoming Leader. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:19:44.472919 18105 raft_consensus.cc:515] T 9c61a39fd4f84424b129c1a6f9ba404d P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:44.481947 18318 raft_consensus.cc:493] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Starting leader election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:19:44.473106 18254 consensus_queue.cc:1059] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.009s
I20260709 13:19:44.475232 18256 raft_consensus.cc:2749] T 5a96373bb0e14dce8a266b2fc6c58507 P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:19:44.505932 18145 consensus_queue.cc:1048] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.068s
I20260709 13:19:44.487401 16151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6f07842d3aa47858df5665874c86f3a" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:19:44.478560 16295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4f65cdf00b748bb963ee6e81ef43b79" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 7 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:44.500224 18271 consensus_queue.cc:1048] T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:19:44.235865 16114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:44.503438 16820 raft_consensus.cc:2393] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 4: Already voted for candidate cb82cc7bd6db46d980c75237a1402860 in this term.
I20260709 13:19:44.347137 17999 raft_consensus.cc:493] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:19:44.490413 18276 leader_election.cc:290] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:19:44.507802 16822 raft_consensus.cc:2468] T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 5.
I20260709 13:19:44.522476 16116 leader_election.cc:304] T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
I20260709 13:19:44.505116 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26354361e9b044cea1c2b86c4882fe92" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:44.509708 16825 raft_consensus.cc:2468] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 6.
W20260709 13:19:44.511602 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 135998us
I20260709 13:19:44.386219 16218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f2da98864b4425bf7c0055951fdd4b" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:44.513657 16148 rpcz_store.cc:276] 0709 13:19:43.357274 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:44.195666 (+838392us) service_pool.cc:224] Handling call
0709 13:19:44.381936 (+186270us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:44.521205 16816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57a744f5aaf944678a63469cbf514015" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:19:44.503046 16818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9944d5e4b8c4109af9955c4f08c9095" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:19:44.504840 18264 consensus_queue.cc:237] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:44.476416 18234 raft_consensus.cc:2749] T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:19:44.521850 16145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c74490c40243a88e62d1760b5cd814" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:19:44.477573 18057 raft_consensus.cc:493] T 7c2095a5f4ad49408dbc7f69c606120d P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:44.476289 18154 raft_consensus.cc:740] T 8cc959fb7f704fba9009c0595467d074 P 90f3da57796b488c9087a2f523adf045 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:19:44.477221 17918 raft_consensus.cc:2749] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:19:44.522754 16192 leader_election.cc:304] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:19:44.513406 18262 raft_consensus.cc:2749] T 5a96373bb0e14dce8a266b2fc6c58507 P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:19:44.532009 16224 raft_consensus.cc:1275] T fffab26427e24d398e2695aad40ca134 P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:19:44.803449 18156 raft_consensus.cc:3060] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:19:44.803625 16192 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 280938us
I20260709 13:19:44.804211 18313 raft_consensus.cc:2804] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:19:44.513556 17844 raft_consensus.cc:515] T 746aabf856ab401c9ef5e859371e7b67 P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:44.804716 18313 raft_consensus.cc:697] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [term 6 LEADER]: Becoming Leader. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:19:44.805337 17999 raft_consensus.cc:3060] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:19:44.805511 18257 raft_consensus.cc:3055] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 1ca2725d290e47d68b1a4b609c9816ac [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:19:44.806012 18257 raft_consensus.cc:740] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 1ca2725d290e47d68b1a4b609c9816ac [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:19:44.805748 18313 consensus_queue.cc:237] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:44.806708 16145 raft_consensus.cc:1240] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 4. Current term is 5. Ops: []
I20260709 13:19:44.806855 18257 consensus_queue.cc:260] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:44.807340 16145 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40844 (request call id 1797) took 2791 ms. Trace:
I20260709 13:19:44.807384 16296 raft_consensus.cc:2468] T e5ad4eb281564936a59f41e387a47156 P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 4.
I20260709 13:19:44.807855 18257 raft_consensus.cc:3060] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:19:44.807781 16145 rpcz_store.cc:276] 0709 13:19:42.015643 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:44.806248 (+2790605us) service_pool.cc:224] Handling call
0709 13:19:44.806548 (+   300us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:44.807104 (+   556us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:44.807287 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:19:44.804884 16114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:44.799860 16790 connection.cc:256] RPC call timeout handler was delayed by -2.067234863000067s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21389999999999998s for a response
W20260709 13:19:44.811326 16790 connection.cc:256] RPC call timeout handler was delayed by -1.9930374610000854s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22480000000000003s for a response
I20260709 13:19:44.516153 18245 raft_consensus.cc:2764] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [term 6 FOLLOWER]: Leader pre-election decision vote started in defunct term 5: won
I20260709 13:19:44.809660 16822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2372981006d4b218215c28964dea623" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:19:44.814126 16218 raft_consensus.cc:2393] T 84f2da98864b4425bf7c0055951fdd4b P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 3: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
I20260709 13:19:44.814839 16822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66f75124215f4ca6bef831e174a792d8" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:19:44.817234 18311 raft_consensus.cc:2804] T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:19:44.817402 16295 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48684 (request call id 1928) took 1485 ms. Trace:
I20260709 13:19:44.817903 16295 rpcz_store.cc:276] 0709 13:19:43.332429 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:44.469890 (+1137461us) service_pool.cc:224] Handling call
0709 13:19:44.817344 (+347454us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:44.819073 16295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f2da98864b4425bf7c0055951fdd4b" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:44.820924 16300 raft_consensus.cc:2468] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 6.
I20260709 13:19:44.821532 16300 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48684 (request call id 1932) took 1061 ms. Trace:
I20260709 13:19:44.821987 16300 rpcz_store.cc:276] 0709 13:19:43.760537 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:44.461122 (+700585us) service_pool.cc:224] Handling call
0709 13:19:44.821470 (+360348us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:44.822705 17844 leader_election.cc:290] T 746aabf856ab401c9ef5e859371e7b67 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:19:44.823350 16300 raft_consensus.cc:1275] T 8d4ab870a05347868956aa8b841b2981 P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 2. (index mismatch)
I20260709 13:19:44.823434 16144 raft_consensus.cc:2468] T 9c61a39fd4f84424b129c1a6f9ba404d P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 4.
I20260709 13:19:44.824103 16300 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48674 (request call id 2053) took 1842 ms. Trace:
I20260709 13:19:44.853677 16300 rpcz_store.cc:276] 0709 13:19:42.981914 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:44.822663 (+1840749us) service_pool.cc:224] Handling call
0709 13:19:44.822937 (+   274us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:44.823901 (+   964us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:44.824058 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:44.816684 16818 raft_consensus.cc:1240] T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860 [term 5 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 4. Current term is 5. Ops: [4.7-4.7]
I20260709 13:19:44.857770 16818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "829b4ad8c5a54eababeea1181b960d4b" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:19:44.858021 16300 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48674 (request call id 2054) took 1872 ms. Trace:
I20260709 13:19:44.858366 16300 rpcz_store.cc:276] 0709 13:19:42.985321 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:44.857268 (+1871947us) service_pool.cc:224] Handling call
0709 13:19:44.857561 (+   293us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:44.857735 (+   174us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:44.857757 (+    22us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:44.857796 (+    39us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:19:44.857833 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:44.857883 (+    50us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:44.857973 (+    90us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:44.816597 16224 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57600 (request call id 1016) took 1524 ms. Trace:
W20260709 13:19:44.813370 16790 connection.cc:256] RPC call timeout handler was delayed by -1.968692775000136s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2202s for a response
W20260709 13:19:44.861052 16790 connection.cc:256] RPC call timeout handler was delayed by -1.6792475710001327s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1686s for a response
W20260709 13:19:44.861378 16790 connection.cc:256] RPC call timeout handler was delayed by -1.5966400290001275s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2948s for a response
W20260709 13:19:44.861686 16790 connection.cc:256] RPC call timeout handler was delayed by -1.5256373180002356s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1645s for a response
W20260709 13:19:44.862015 16790 connection.cc:256] RPC call timeout handler was delayed by -1.2287631239998973s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2247s for a response
I20260709 13:19:44.866216 16790 leader_election.cc:304] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:19:44.824043 18319 consensus_queue.cc:237] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:44.823423 16145 raft_consensus.cc:1240] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 5. Current term is 6. Ops: []
W20260709 13:19:44.679008 16142 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46224 (request call id 268) took 1623 ms (client timeout 1812 ms). Trace:
I20260709 13:19:44.825711 18262 raft_consensus.cc:493] T 8158bbd17b994d2eb96f560bd0f1b88d P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:44.826122 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 303704us
I20260709 13:19:44.884732 16116 leader_election.cc:304] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 90f3da57796b488c9087a2f523adf045
I20260709 13:19:44.548628 18042 raft_consensus.cc:515] T 5a959dbecc2648eb85d1cc550f5a3245 P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:44.543466 16226 raft_consensus.cc:2468] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 4.
I20260709 13:19:44.744534 18179 raft_consensus.cc:515] T 66f75124215f4ca6bef831e174a792d8 P cb82cc7bd6db46d980c75237a1402860 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:44.828377 18318 raft_consensus.cc:3060] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:19:44.509488 16299 raft_consensus.cc:2468] T 84f2da98864b4425bf7c0055951fdd4b P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 3.
I20260709 13:19:44.862689 18034 raft_consensus.cc:493] T 3884f864a96d428785895e5463e5728f P cb82cc7bd6db46d980c75237a1402860 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:44.833421 16192 leader_election.cc:304] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:19:44.540788 16221 rpcz_store.cc:276] 0709 13:19:41.818248 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:42.515098 (+696850us) service_pool.cc:224] Handling call
0709 13:19:43.456037 (+940939us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:43.456287 (+   250us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:19:43.456666 (+   379us) log.cc:844] Serialized 14 byte log entry
0709 13:19:43.680642 (+223976us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:43.680734 (+    92us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:19:43.680804 (+    70us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:43.680910 (+   106us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:44.093340 (+412430us) spinlock_profiling.cc:243] Waited 3.55 ms on lock 0x7b4400022b4c. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69ad8fadb1 00007f69ad998c85 00007f69ad9996a2 00007f69b070b1db 00007f69b0706aaf 00007f69b06fa86e 00007f69b06f9bb6 00007f69b06f97d8 00007f69b0703221 00007f69b07031d9 00007f69b0703169
0709 13:19:44.101722 (+  8382us) spinlock_profiling.cc:243] Waited 2.65 ms on lock 0x7b180042eb00. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b693ade1 00007f69b69475b1 00007f69ada402a6 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5e8ad 00007f69b5b23419 00007f69af09abd4
0709 13:19:44.397152 (+295430us) spinlock_profiling.cc:243] Waited 932 ms on lock 0x7b54002c9a50. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5e8ad 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:19:44.397318 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":199,"prepare.run_cpu_time_us":44,"prepare.run_wall_time_us":45,"raft.queue_time_us":133,"raft.run_cpu_time_us":83,"raft.run_wall_time_us":85,"spinlock_wait_cycles":2156927872,"thread_start_us":216656,"threads_started":1,"wal-append.queue_time_us":430371}
I20260709 13:19:44.849095 16223 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57600 (request call id 1032) took 1065 ms. Trace:
W20260709 13:19:44.896541 16192 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 63196us
I20260709 13:19:44.896654 16223 rpcz_store.cc:276] 0709 13:19:43.784075 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:43.906131 (+122056us) service_pool.cc:224] Handling call
0709 13:19:44.848865 (+942734us) spinlock_profiling.cc:243] Waited 284 ms on lock 0x7b5400396054. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:19:44.849025 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":654311680}
W20260709 13:19:44.849452 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 380085us
I20260709 13:19:44.744331 17992 leader_election.cc:290] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 4 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:44.874056 16145 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40830 (request call id 1033) took 1554 ms. Trace:
I20260709 13:19:44.874846 16300 raft_consensus.cc:1240] T c0eab25ce4c64e018a041beeda2867db P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 3. Current term is 4. Ops: [3.3-3.3]
I20260709 13:19:44.876170 16224 rpcz_store.cc:276] 0709 13:19:43.292200 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:44.513727 (+1221527us) service_pool.cc:224] Handling call
0709 13:19:44.531689 (+ 17962us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:44.816358 (+284669us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:44.816542 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:19:44.879825 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 519523us
I20260709 13:19:44.541829 16143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f82224f4564fea93572e8206bad1fe" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:19:44.556710 18240 raft_consensus.cc:2749] T b6f07842d3aa47858df5665874c86f3a P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:19:44.826058 16265 leader_election.cc:304] T 68a6838d82ef4cac83507f906139d90f P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 647a343a542d4a139fe106c19095129b
I20260709 13:19:44.677358 18204 leader_election.cc:290] T 9e65676b21b748728ad39668cfe722ec P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 5 election: Requested vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:19:44.893201 16299 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48674 (request call id 2062) took 1136 ms. Trace:
I20260709 13:19:44.832157 16296 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48684 (request call id 1920) took 1609 ms. Trace:
W20260709 13:19:44.913169 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 94031.7us
I20260709 13:19:44.913385 16296 rpcz_store.cc:276] 0709 13:19:43.223374 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:44.461148 (+1237774us) service_pool.cc:224] Handling call
0709 13:19:44.832083 (+370935us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:44.862540 18323 raft_consensus.cc:493] T 02838c86531d4f98bf58d1c22b8baf77 P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:19:44.899854 16191 leader_election.cc:400] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 4 election: Vote denied by peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741) with higher term. Message: Invalid argument: T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac for earlier term 4. Current term is 5.
I20260709 13:19:44.695017 18109 raft_consensus.cc:515] T e4dc6b0541ca431ebc757beac00521f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:44.918375 16191 leader_election.cc:304] T fffab26427e24d398e2695aad40ca134 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
I20260709 13:19:44.848338 18057 raft_consensus.cc:515] T 7c2095a5f4ad49408dbc7f69c606120d P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:44.919620 16191 leader_election.cc:304] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
I20260709 13:19:44.901839 16145 rpcz_store.cc:276] 0709 13:19:43.324946 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:44.808747 (+1483801us) service_pool.cc:224] Handling call
0709 13:19:44.823272 (+ 14525us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:44.871356 (+ 48084us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:44.873997 (+  2641us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:19:44.901077 18114 consensus_meta.cc:287] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac: Time spent flushing consensus metadata: real 0.523s	user 0.005s	sys 0.005s
I20260709 13:19:44.902838 16190 leader_election.cc:304] T 18ad7e3af5504571a69d10c00750a01c P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 647a343a542d4a139fe106c19095129b
I20260709 13:19:44.907312 16222 raft_consensus.cc:1240] T 5a959dbecc2648eb85d1cc550f5a3245 P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 2. Current term is 3. Ops: []
I20260709 13:19:44.908906 16821 raft_consensus.cc:2393] T 66f75124215f4ca6bef831e174a792d8 P cb82cc7bd6db46d980c75237a1402860 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 5: Already voted for candidate cb82cc7bd6db46d980c75237a1402860 in this term.
I20260709 13:19:44.893241 16816 raft_consensus.cc:2393] T 57a744f5aaf944678a63469cbf514015 P cb82cc7bd6db46d980c75237a1402860 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 3: Already voted for candidate cb82cc7bd6db46d980c75237a1402860 in this term.
I20260709 13:19:44.886857 16218 raft_consensus.cc:1240] T 5a96373bb0e14dce8a266b2fc6c58507 P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 2. Current term is 3. Ops: []
I20260709 13:19:44.743091 18140 raft_consensus.cc:515] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:44.887533 16220 raft_consensus.cc:1240] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 4. Current term is 5. Ops: []
I20260709 13:19:44.912870 16299 rpcz_store.cc:276] 0709 13:19:43.756617 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:44.493403 (+736786us) service_pool.cc:224] Handling call
0709 13:19:44.893145 (+399742us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:44.914265 16265 leader_election.cc:304] T 18ad7e3af5504571a69d10c00750a01c P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:19:44.948859 16265 leader_election.cc:304] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:19:44.884358 18298 raft_consensus.cc:2749] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:19:44.902326 16300 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48674 (request call id 2055) took 1914 ms. Trace:
I20260709 13:19:44.954250 16300 rpcz_store.cc:276] 0709 13:19:42.988396 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:44.874284 (+1885888us) service_pool.cc:224] Handling call
0709 13:19:44.874574 (+   290us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:44.902059 (+ 27485us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:44.902265 (+   206us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:44.955665 16300 raft_consensus.cc:1275] T 31969392ccdb43b491911a319e8bc7e1 P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:19:44.956068 16300 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48674 (request call id 2061) took 1200 ms. Trace:
I20260709 13:19:44.956249 16300 rpcz_store.cc:276] 0709 13:19:43.756016 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:44.955068 (+1199052us) service_pool.cc:224] Handling call
0709 13:19:44.955372 (+   304us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:44.955906 (+   534us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:44.956032 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:44.957319 16300 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48674 (request call id 2063) took 1200 ms. Trace:
I20260709 13:19:44.957624 16300 rpcz_store.cc:276] 0709 13:19:43.757237 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:44.956768 (+1199531us) service_pool.cc:224] Handling call
0709 13:19:44.957018 (+   250us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:44.957142 (+   124us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:44.957156 (+    14us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:44.957182 (+    26us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:19:44.957198 (+    16us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:44.957226 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:44.957286 (+    60us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:44.958665 16300 raft_consensus.cc:1275] T 8c11074579b046d99e585c2818e18a2c P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:19:44.959008 16300 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48684 (request call id 1940) took 1196 ms. Trace:
I20260709 13:19:44.872485 18234 consensus_queue.cc:1048] T 8d4ab870a05347868956aa8b841b2981 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.040s
I20260709 13:19:44.959275 16300 rpcz_store.cc:276] 0709 13:19:43.762746 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:44.958215 (+1195469us) service_pool.cc:224] Handling call
0709 13:19:44.958459 (+   244us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:44.958858 (+   399us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:44.958975 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:44.926265 16224 raft_consensus.cc:1240] T c0eab25ce4c64e018a041beeda2867db P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 3. Current term is 4. Ops: [3.3-3.3]
I20260709 13:19:44.960655 16224 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57604 (request call id 1976) took 1483 ms. Trace:
I20260709 13:19:44.522640 18243 raft_consensus.cc:515] T ecc1e363230043fe9fef390f0b979f20 P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:44.919703 16151 raft_consensus.cc:2468] T b6f07842d3aa47858df5665874c86f3a P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 3.
I20260709 13:19:44.904996 17844 raft_consensus.cc:493] T af57996a8be84d398f0f023e8d08ef77 P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:44.960952 16224 rpcz_store.cc:276] 0709 13:19:43.477259 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:44.902205 (+1424946us) service_pool.cc:224] Handling call
0709 13:19:44.926010 (+ 23805us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:44.960431 (+ 34421us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:44.960607 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:44.961555 17844 raft_consensus.cc:515] T af57996a8be84d398f0f023e8d08ef77 P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:44.962386 16224 raft_consensus.cc:1240] T 8cc959fb7f704fba9009c0595467d074 P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 3. Current term is 4. Ops: [3.3-3.3]
W20260709 13:19:44.940435 16117 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 365508us
I20260709 13:19:44.963009 16224 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57600 (request call id 1042) took 1128 ms. Trace:
I20260709 13:19:44.915299 18255 raft_consensus.cc:2804] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:19:44.892359 18262 raft_consensus.cc:515] T 8158bbd17b994d2eb96f560bd0f1b88d P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:44.964643 16144 raft_consensus.cc:1275] T 8158bbd17b994d2eb96f560bd0f1b88d P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:19:44.924732 17918 raft_consensus.cc:2804] T fffab26427e24d398e2695aad40ca134 P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:19:44.946475 16145 raft_consensus.cc:1275] T 5a959dbecc2648eb85d1cc550f5a3245 P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:19:44.910288 16819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746aabf856ab401c9ef5e859371e7b67" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
W20260709 13:19:44.950941 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 84831.2us
I20260709 13:19:44.951066 16148 raft_consensus.cc:1240] T 7c2095a5f4ad49408dbc7f69c606120d P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 2. Current term is 3. Ops: []
I20260709 13:19:44.968603 16145 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40830 (request call id 1037) took 1642 ms. Trace:
I20260709 13:19:44.968854 16790 leader_election.cc:304] T 434bff3082d141838068f323bdb0d1ed P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:19:44.969023 16145 rpcz_store.cc:276] 0709 13:19:43.325955 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:44.924852 (+1598897us) service_pool.cc:224] Handling call
0709 13:19:44.946235 (+ 21383us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:44.968347 (+ 22112us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:44.968547 (+   200us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:44.963651 16149 raft_consensus.cc:2393] T ecc1e363230043fe9fef390f0b979f20 P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 4: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
I20260709 13:19:44.970219 16147 raft_consensus.cc:2393] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 in current term 5: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
I20260709 13:19:44.969221 16296 raft_consensus.cc:1240] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 3. Current term is 4. Ops: []
I20260709 13:19:44.971211 16296 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48684 (request call id 1918) took 1794 ms. Trace:
I20260709 13:19:44.971763 16296 rpcz_store.cc:276] 0709 13:19:43.176455 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:44.944673 (+1768218us) service_pool.cc:224] Handling call
0709 13:19:44.969064 (+ 24391us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:44.970896 (+  1832us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:44.971131 (+   235us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:44.963358 16224 rpcz_store.cc:276] 0709 13:19:43.834513 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:44.961854 (+1127341us) service_pool.cc:224] Handling call
0709 13:19:44.962174 (+   320us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:44.962783 (+   609us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:44.962961 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:19:44.965639 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 80984.2us
I20260709 13:19:44.887637 18181 raft_consensus.cc:515] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:44.976310 16150 raft_consensus.cc:2393] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 4: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
I20260709 13:19:44.924594 18254 raft_consensus.cc:2804] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:19:44.977013 18254 raft_consensus.cc:493] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:44.977232 16790 leader_election.cc:304] T 57a744f5aaf944678a63469cbf514015 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:19:44.977567 18254 raft_consensus.cc:3060] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:19:44.874506 16142 rpcz_store.cc:269] 0709 13:19:43.055993 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:43.461037 (+405044us) service_pool.cc:224] Handling call
0709 13:19:44.678736 (+1217699us) spinlock_profiling.cc:243] Waited 485 ms on lock 0x7b540020c454. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:19:44.678928 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1116580992}
I20260709 13:19:44.980003 16142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cc959fb7f704fba9009c0595467d074" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:19:44.944031 18145 raft_consensus.cc:2804] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:19:44.906337 18034 raft_consensus.cc:515] T 3884f864a96d428785895e5463e5728f P cb82cc7bd6db46d980c75237a1402860 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:44.981889 16299 raft_consensus.cc:1275] T 57a744f5aaf944678a63469cbf514015 P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:19:44.960907 16300 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40474 (request call id 315) took 1113 ms. Trace:
I20260709 13:19:44.989928 16299 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48674 (request call id 2060) took 1234 ms. Trace:
W20260709 13:19:44.982491 16189 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:44.989841 16300 rpcz_store.cc:276] 0709 13:19:43.847838 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:44.960260 (+1112422us) service_pool.cc:224] Handling call
0709 13:19:44.960489 (+   229us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:44.960648 (+   159us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:44.960670 (+    22us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:44.960714 (+    44us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:19:44.960747 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:44.960793 (+    46us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:44.960866 (+    73us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:44.990374 16299 rpcz_store.cc:276] 0709 13:19:43.755391 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:44.947747 (+1192356us) service_pool.cc:224] Handling call
0709 13:19:44.981601 (+ 33854us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:44.989688 (+  8087us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:44.989875 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:44.992457 16299 raft_consensus.cc:1240] T d2ab0ac8347641e6955e9bce7d90c1a2 P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 4. Current term is 5. Ops: []
I20260709 13:19:44.990721 16265 leader_election.cc:304] T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:19:44.993824 16299 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40474 (request call id 316) took 1145 ms. Trace:
I20260709 13:19:44.994131 16299 rpcz_store.cc:276] 0709 13:19:43.848182 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:44.991903 (+1143721us) service_pool.cc:224] Handling call
0709 13:19:44.992180 (+   277us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:44.993595 (+  1415us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:44.993777 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:44.964126 17844 leader_election.cc:290] T af57996a8be84d398f0f023e8d08ef77 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:19:44.991861 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:19:44.996151 16299 raft_consensus.cc:1275] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:19:44.996333 16300 raft_consensus.cc:3060] T ecc1e363230043fe9fef390f0b979f20 P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:19:44.996788 16299 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48684 (request call id 1945) took 1140 ms. Trace:
I20260709 13:19:44.997259 16299 rpcz_store.cc:276] 0709 13:19:43.856685 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:44.995574 (+1138889us) service_pool.cc:224] Handling call
0709 13:19:44.995856 (+   282us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:44.996578 (+   722us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:44.996745 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:44.997603 18317 consensus_queue.cc:1048] T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:19:44.998742 16295 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48674 (request call id 2047) took 2038 ms. Trace:
I20260709 13:19:44.999280 16295 rpcz_store.cc:276] 0709 13:19:42.960776 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:44.820006 (+1859230us) service_pool.cc:224] Handling call
0709 13:19:44.820298 (+   292us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:44.853051 (+ 32753us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:19:44.947662 (+ 94611us) log.cc:844] Serialized 14 byte log entry
0709 13:19:44.998326 (+ 50664us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:44.998390 (+    64us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:19:44.998451 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:44.998534 (+    83us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:44.998673 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":124293,"threads_started":3}
I20260709 13:19:44.928725 18240 consensus_queue.cc:1059] T ee9ed67006144797b1de6ea8c2013a68 P 647a343a542d4a139fe106c19095129b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.053s
I20260709 13:19:44.939450 18311 raft_consensus.cc:697] T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b [term 5 LEADER]: Becoming Leader. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:19:44.964682 18167 raft_consensus.cc:2749] T 18ad7e3af5504571a69d10c00750a01c P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:19:45.005650 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 102904us
I20260709 13:19:44.970386 17918 raft_consensus.cc:493] T fffab26427e24d398e2695aad40ca134 P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:44.925323 18272 raft_consensus.cc:515] T 8a878999c0024d2abe517484180290fc P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:44.927973 18114 raft_consensus.cc:515] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:44.916925 18256 consensus_queue.cc:1059] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.049s
I20260709 13:19:45.018746 16145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:19:44.968300 18243 leader_election.cc:290] T ecc1e363230043fe9fef390f0b979f20 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 4 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:19:44.971218 18320 raft_consensus.cc:2749] T 66ae176961b74813a58bc15c9c4c6f2a P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Leader pre-election lost for term 5. Reason: Vote denied by peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943) with higher term. Message: Invalid argument: T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 for earlier term 5. Current term is 6.
I20260709 13:19:44.984516 18234 consensus_queue.cc:1059] T c0eab25ce4c64e018a041beeda2867db P 647a343a542d4a139fe106c19095129b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.040s
I20260709 13:19:44.995852 17992 raft_consensus.cc:2749] T 434bff3082d141838068f323bdb0d1ed P cb82cc7bd6db46d980c75237a1402860 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:19:44.996014 16220 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57600 (request call id 1020) took 1695 ms. Trace:
I20260709 13:19:44.984058 16218 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57600 (request call id 1028) took 1625 ms. Trace:
I20260709 13:19:45.025887 18255 raft_consensus.cc:697] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [term 6 LEADER]: Becoming Leader. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:19:45.027678 18313 raft_consensus.cc:2804] T 18ad7e3af5504571a69d10c00750a01c P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:19:45.030478 17918 raft_consensus.cc:3060] T fffab26427e24d398e2695aad40ca134 P 1ca2725d290e47d68b1a4b609c9816ac [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:19:45.027472 18337 raft_consensus.cc:2749] T 57a744f5aaf944678a63469cbf514015 P cb82cc7bd6db46d980c75237a1402860 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:19:44.930127 18248 consensus_queue.cc:237] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 2, Current term: 3, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:44.922428 18323 raft_consensus.cc:515] T 02838c86531d4f98bf58d1c22b8baf77 P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:45.039130 16296 raft_consensus.cc:1275] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:19:45.130764 16296 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48674 (request call id 2065) took 1219 ms. Trace:
I20260709 13:19:45.131202 16296 rpcz_store.cc:276] 0709 13:19:43.911151 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:45.008696 (+1097545us) service_pool.cc:224] Handling call
0709 13:19:45.034188 (+ 25492us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:45.130457 (+ 96269us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:45.130696 (+   239us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:19:45.043426 18041 consensus_meta.cc:287] T a378d7f796f94615b3b347119a9880fd P 90f3da57796b488c9087a2f523adf045: Time spent flushing consensus metadata: real 0.501s	user 0.006s	sys 0.003s
I20260709 13:19:45.043761 18256 raft_consensus.cc:3055] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:19:45.037642 18167 raft_consensus.cc:493] T 7c2095a5f4ad49408dbc7f69c606120d P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:45.044170 18324 consensus_queue.cc:1059] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.030s
I20260709 13:19:45.022192 17844 raft_consensus.cc:493] T 829b4ad8c5a54eababeea1181b960d4b P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:45.021801 16221 raft_consensus.cc:1240] T c2f5f89186204242bf377f7c42b2e2da P 1ca2725d290e47d68b1a4b609c9816ac [term 6 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 5. Current term is 6. Ops: [5.4-5.4]
I20260709 13:19:45.025913 18240 raft_consensus.cc:3055] T ee9ed67006144797b1de6ea8c2013a68 P 647a343a542d4a139fe106c19095129b [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:19:45.090171 16300 raft_consensus.cc:2468] T ecc1e363230043fe9fef390f0b979f20 P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 4.
I20260709 13:19:45.026340 16142 raft_consensus.cc:2393] T 8cc959fb7f704fba9009c0595467d074 P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 4: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
I20260709 13:19:45.109390 16225 raft_consensus.cc:2393] T 8a878999c0024d2abe517484180290fc P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 4: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
I20260709 13:19:45.110249 16224 raft_consensus.cc:1240] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 4. Current term is 5. Ops: []
I20260709 13:19:45.008970 18145 raft_consensus.cc:697] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b [term 6 LEADER]: Becoming Leader. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:19:44.905515 18154 consensus_queue.cc:260] T 8cc959fb7f704fba9009c0595467d074 P 90f3da57796b488c9087a2f523adf045 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:45.027621 16190 leader_election.cc:304] T c2f5f89186204242bf377f7c42b2e2da P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
W20260709 13:19:45.045367 18321 log.cc:927] Time spent T 68a6838d82ef4cac83507f906139d90f P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 0.218s	user 0.000s	sys 0.001s
W20260709 13:19:45.125365 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 148191us
I20260709 13:19:45.030465 16816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:19:45.079872 18317 raft_consensus.cc:3055] T c0eab25ce4c64e018a041beeda2867db P 647a343a542d4a139fe106c19095129b [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:19:45.134598 16145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af57996a8be84d398f0f023e8d08ef77" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:19:45.164076 16145 raft_consensus.cc:2468] T af57996a8be84d398f0f023e8d08ef77 P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 4.
I20260709 13:19:45.133558 18181 leader_election.cc:290] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 4 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:45.115809 16818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af57996a8be84d398f0f023e8d08ef77" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:19:45.111126 18313 raft_consensus.cc:493] T 18ad7e3af5504571a69d10c00750a01c P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Starting leader election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:19:45.143049 16300 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48674 (request call id 2068) took 1221 ms. Trace:
I20260709 13:19:45.203683 16300 rpcz_store.cc:276] 0709 13:19:43.921715 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:45.142275 (+1220560us) service_pool.cc:224] Handling call
0709 13:19:45.142614 (+   339us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:45.142767 (+   153us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:45.142792 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:45.142830 (+    38us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:19:45.142858 (+    28us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:45.142906 (+    48us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:45.142993 (+    87us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":141440}
I20260709 13:19:45.205994 16300 raft_consensus.cc:1275] T 3884f864a96d428785895e5463e5728f P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:19:45.206755 16300 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48674 (request call id 2069) took 1109 ms. Trace:
I20260709 13:19:45.141065 18240 raft_consensus.cc:740] T ee9ed67006144797b1de6ea8c2013a68 P 647a343a542d4a139fe106c19095129b [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:19:45.079478 16218 rpcz_store.cc:276] 0709 13:19:43.355466 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:44.814828 (+1459362us) service_pool.cc:224] Handling call
0709 13:19:44.860686 (+ 45858us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:44.942214 (+ 81528us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:44.969882 (+ 27668us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:45.214150 18240 consensus_queue.cc:260] T ee9ed67006144797b1de6ea8c2013a68 P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:45.216890 16218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62fe3346cd3c42c4a55e14af9073bca4" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:19:45.217417 16145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f2da98864b4425bf7c0055951fdd4b" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:19:45.218372 16218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5d59407eb545b895d8d922e0dd69fd" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:45.218582 16145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746aabf856ab401c9ef5e859371e7b67" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
W20260709 13:19:45.218863 18312 log.cc:927] Time spent T 9e65676b21b748728ad39668cfe722ec P cb82cc7bd6db46d980c75237a1402860: Append to log took a long time: real 0.899s	user 0.005s	sys 0.007s
I20260709 13:19:45.143049 16296 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48674 (request call id 2067) took 1223 ms. Trace:
I20260709 13:19:45.219679 16145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66f75124215f4ca6bef831e174a792d8" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:19:45.219714 16296 rpcz_store.cc:276] 0709 13:19:43.919876 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:45.142268 (+1222392us) service_pool.cc:224] Handling call
0709 13:19:45.142603 (+   335us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:45.142767 (+   164us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:45.142791 (+    24us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:45.142830 (+    39us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:19:45.142858 (+    28us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:45.142906 (+    48us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:45.142993 (+    87us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":7936}
I20260709 13:19:45.143947 16221 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57600 (request call id 1040) took 1310 ms. Trace:
I20260709 13:19:45.038628 18341 raft_consensus.cc:2804] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:19:45.230224 16824 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49892 (request call id 327) took 1363 ms. Trace:
W20260709 13:19:45.144711 18336 log.cc:927] Time spent T 1df93779209748b9860f6d893dd3de53 P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 0.111s	user 0.000s	sys 0.000s
I20260709 13:19:45.151628 16825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2794af5b99984efebdee564035841cb0" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
W20260709 13:19:45.155751 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 130735us
I20260709 13:19:45.140105 18167 raft_consensus.cc:515] T 7c2095a5f4ad49408dbc7f69c606120d P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:45.167181 18327 raft_consensus.cc:2749] T c2f5f89186204242bf377f7c42b2e2da P 1ca2725d290e47d68b1a4b609c9816ac [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:19:45.167861 16294 consensus_meta.cc:287] T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045: Time spent flushing consensus metadata: real 0.631s	user 0.010s	sys 0.000s
I20260709 13:19:45.145323 18347 raft_consensus.cc:2804] T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:19:45.153292 18255 consensus_queue.cc:237] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:45.168565 16219 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57604 (request call id 1955) took 3348 ms. Trace:
I20260709 13:19:45.264096 16219 rpcz_store.cc:276] 0709 13:19:41.820572 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:44.323607 (+2503035us) service_pool.cc:224] Handling call
0709 13:19:44.323933 (+   326us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:19:44.324329 (+   396us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:44.324363 (+    34us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:19:44.324951 (+   588us) log.cc:844] Serialized 52 byte log entry
0709 13:19:44.462026 (+137075us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:19:44.466535 (+  4509us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:44.475017 (+  8482us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:45.168262 (+693245us) raft_consensus.cc:1685] finished
0709 13:19:45.168294 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:45.168493 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":6692,"raft.run_cpu_time_us":177,"raft.run_wall_time_us":3920,"thread_start_us":55733,"threads_started":1,"wal-append.queue_time_us":410197}
I20260709 13:19:45.266394 16219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af57996a8be84d398f0f023e8d08ef77" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:45.079742 16220 rpcz_store.cc:276] 0709 13:19:43.295528 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:44.539289 (+1243761us) service_pool.cc:224] Handling call
0709 13:19:44.860121 (+320832us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:44.965629 (+105508us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:44.991331 (+ 25702us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:45.169274 18034 leader_election.cc:290] T 3884f864a96d428785895e5463e5728f P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
W20260709 13:19:45.170028 18325 log.cc:927] Time spent T af57996a8be84d398f0f023e8d08ef77 P cb82cc7bd6db46d980c75237a1402860: Append to log took a long time: real 0.223s	user 0.000s	sys 0.001s
I20260709 13:19:45.267828 16220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74ca8140f35c4ae29e8a664f6bdfc829" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:19:45.151563 16142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee9ed67006144797b1de6ea8c2013a68" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:19:45.152264 18179 leader_election.cc:290] T 66f75124215f4ca6bef831e174a792d8 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 5 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:19:45.163733 18317 raft_consensus.cc:740] T c0eab25ce4c64e018a041beeda2867db P 647a343a542d4a139fe106c19095129b [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:19:45.269155 16220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746aabf856ab401c9ef5e859371e7b67" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:45.268994 18156 raft_consensus.cc:515] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:45.270107 16220 raft_consensus.cc:2393] T 746aabf856ab401c9ef5e859371e7b67 P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 in current term 4: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
I20260709 13:19:45.269892 18317 consensus_queue.cc:260] T c0eab25ce4c64e018a041beeda2867db P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:45.271281 18317 raft_consensus.cc:3060] T c0eab25ce4c64e018a041beeda2867db P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:19:45.272910 16149 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40844 (request call id 1813) took 1509 ms. Trace:
I20260709 13:19:45.273307 16149 rpcz_store.cc:276] 0709 13:19:43.763488 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:43.924119 (+160631us) service_pool.cc:224] Handling call
0709 13:19:45.272727 (+1348608us) spinlock_profiling.cc:243] Waited 832 ms on lock 0x7b54003d9fd4. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:19:45.272853 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1912725248}
I20260709 13:19:45.148948 18339 raft_consensus.cc:493] T 5a959dbecc2648eb85d1cc550f5a3245 P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:45.175414 18140 leader_election.cc:290] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 5 election: Requested vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:19:45.175844 18057 leader_election.cc:290] T 7c2095a5f4ad49408dbc7f69c606120d P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:45.163712 18042 leader_election.cc:290] T 5a959dbecc2648eb85d1cc550f5a3245 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 3 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:45.176733 18272 leader_election.cc:290] T 8a878999c0024d2abe517484180290fc P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 4 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:45.177739 18109 leader_election.cc:290] T e4dc6b0541ca431ebc757beac00521f1 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 4 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:45.164255 18346 raft_consensus.cc:493] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:19:45.179083 18262 leader_election.cc:290] T 8158bbd17b994d2eb96f560bd0f1b88d P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:45.145468 18311 consensus_queue.cc:237] T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:45.180680 18323 leader_election.cc:290] T 02838c86531d4f98bf58d1c22b8baf77 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:19:45.141813 17844 raft_consensus.cc:515] T 829b4ad8c5a54eababeea1181b960d4b P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:45.164309 16790 leader_election.cc:304] T 26354361e9b044cea1c2b86c4882fe92 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860; no voters: 
W20260709 13:19:45.200812 18326 log.cc:927] Time spent T 7c2095a5f4ad49408dbc7f69c606120d P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 0.219s	user 0.001s	sys 0.000s
I20260709 13:19:45.136399 18256 raft_consensus.cc:740] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:19:45.169333 16816 raft_consensus.cc:2468] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 3.
I20260709 13:19:45.311187 18179 raft_consensus.cc:2804] T 26354361e9b044cea1c2b86c4882fe92 P cb82cc7bd6db46d980c75237a1402860 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:19:45.311650 18179 raft_consensus.cc:493] T 26354361e9b044cea1c2b86c4882fe92 P cb82cc7bd6db46d980c75237a1402860 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:19:45.306751 17918 raft_consensus.cc:515] T fffab26427e24d398e2695aad40ca134 P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:45.314877 17918 leader_election.cc:290] T fffab26427e24d398e2695aad40ca134 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 3 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
W20260709 13:19:45.316378 18349 log.cc:927] Time spent T 84f2da98864b4425bf7c0055951fdd4b P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 0.081s	user 0.001s	sys 0.000s
I20260709 13:19:45.148415 16819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62fe3346cd3c42c4a55e14af9073bca4" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:19:45.318136 16819 raft_consensus.cc:2393] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 6: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
I20260709 13:19:45.149448 18154 raft_consensus.cc:3060] T 8cc959fb7f704fba9009c0595467d074 P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:19:45.320010 18167 leader_election.cc:290] T 7c2095a5f4ad49408dbc7f69c606120d P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:19:45.315141 18179 raft_consensus.cc:3060] T 26354361e9b044cea1c2b86c4882fe92 P cb82cc7bd6db46d980c75237a1402860 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:19:45.138957 18041 raft_consensus.cc:515] T a378d7f796f94615b3b347119a9880fd P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:45.202888 16295 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40474 (request call id 317) took 1338 ms. Trace:
I20260709 13:19:45.323385 16295 rpcz_store.cc:276] 0709 13:19:43.864331 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:45.000843 (+1136512us) service_pool.cc:224] Handling call
0709 13:19:45.001325 (+   482us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:19:45.003055 (+  1730us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:45.003102 (+    47us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:19:45.003888 (+   786us) log.cc:844] Serialized 52 byte log entry
0709 13:19:45.024350 (+ 20462us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:19:45.024404 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:45.024494 (+    90us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:45.202627 (+178133us) raft_consensus.cc:1685] finished
0709 13:19:45.202651 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:45.202815 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":138112,"raft.run_cpu_time_us":142,"raft.run_wall_time_us":11726,"thread_start_us":20986,"threads_started":2,"wal-append.queue_time_us":196672}
I20260709 13:19:45.324827 18041 leader_election.cc:290] T a378d7f796f94615b3b347119a9880fd P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 4 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:19:45.208092 16300 rpcz_store.cc:276] 0709 13:19:44.096877 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:45.205143 (+1108266us) service_pool.cc:224] Handling call
0709 13:19:45.205531 (+   388us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:45.206451 (+   920us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:45.206689 (+   238us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:19:45.327049 16191 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:45.212419 16818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26354361e9b044cea1c2b86c4882fe92" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:19:45.213022 18313 raft_consensus.cc:3060] T 18ad7e3af5504571a69d10c00750a01c P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:19:45.177690 18254 raft_consensus.cc:515] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:45.234244 16825 raft_consensus.cc:2376] T 2794af5b99984efebdee564035841cb0 P cb82cc7bd6db46d980c75237a1402860 [term 3 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 647a343a542d4a139fe106c19095129b in term 3. Re-sending same reply.
I20260709 13:19:45.234530 16190 leader_election.cc:304] T 68a6838d82ef4cac83507f906139d90f P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:19:45.230037 18341 raft_consensus.cc:493] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:45.240635 16145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d4ab870a05347868956aa8b841b2981" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:19:45.230787 16824 rpcz_store.cc:276] 0709 13:19:43.866453 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:43.866838 (+   385us) service_pool.cc:224] Handling call
0709 13:19:43.867192 (+   354us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:43.885950 (+ 18758us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:43.885978 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:43.907420 (+ 21442us) log.cc:844] Serialized 26 byte log entry
0709 13:19:44.123574 (+216154us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:19:44.123624 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:44.123717 (+    93us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:45.218302 (+1094585us) spinlock_profiling.cc:243] Waited 10.8 ms on lock 0x7b4400022b4c. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69ad8fadb1 00007f69ad998c85 00007f69ad9996a2 00007f69b070b1db 00007f69b0706aaf 00007f69b06fa86e 00007f69b06f9bb6 00007f69b06f97d8 00007f69b0703221 00007f69b07031d9 00007f69b0703169
0709 13:19:45.220130 (+  1828us) raft_consensus.cc:1685] finished
0709 13:19:45.220148 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:45.230135 (+  9987us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":18311,"raft.queue_time_us":18795,"raft.run_cpu_time_us":136,"raft.run_wall_time_us":2584,"spinlock_wait_cycles":24798592,"thread_start_us":189517,"threads_started":1,"wal-append.queue_time_us":316952}
W20260709 13:19:45.261554 16114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:45.240300 16294 raft_consensus.cc:2468] T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 4.
I20260709 13:19:45.274304 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62fe3346cd3c42c4a55e14af9073bca4" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:19:45.274782 16297 raft_consensus.cc:3060] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:19:45.277040 16117 leader_election.cc:336] T 57a744f5aaf944678a63469cbf514015 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923): Timed out: RequestConsensusVote RPC to 127.14.177.132:38923 timed out after 1.505s (SENT)
I20260709 13:19:45.217458 18240 raft_consensus.cc:3060] T ee9ed67006144797b1de6ea8c2013a68 P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:19:45.283325 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 292630us
I20260709 13:19:45.267055 16219 raft_consensus.cc:2376] T af57996a8be84d398f0f023e8d08ef77 P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 4. Re-sending same reply.
I20260709 13:19:45.271023 16220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee9ed67006144797b1de6ea8c2013a68" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:45.154491 17999 raft_consensus.cc:515] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } }
I20260709 13:19:45.247224 18347 raft_consensus.cc:493] T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:45.152738 18145 consensus_queue.cc:237] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:19:45.297844 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 133582us
I20260709 13:19:45.344753 16790 leader_election.cc:400] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 5 election: Vote denied by peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741) with higher term. Message: Invalid argument: T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 for earlier term 5. Current term is 6.
I20260709 13:19:45.345060 16790 leader_election.cc:403] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 5 election: Cancelling election due to peer responding with higher term
I20260709 13:19:45.228062 16221 rpcz_store.cc:276] 0709 13:19:43.833924 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:44.924737 (+1090813us) service_pool.cc:224] Handling call
0709 13:19:44.970110 (+ 45373us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:45.141250 (+171140us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:45.143890 (+  2640us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:45.294899 18318 raft_consensus.cc:515] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:45.305320 17844 leader_election.cc:290] T 829b4ad8c5a54eababeea1181b960d4b P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:19:45.305423 16143 raft_consensus.cc:1240] T 9e65676b21b748728ad39668cfe722ec P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 4. Current term is 5. Ops: [4.7-4.7]
I20260709 13:19:45.306417 16225 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57600 (request call id 1031) took 1522 ms. Trace:
I20260709 13:19:45.347515 16225 rpcz_store.cc:276] 0709 13:19:43.783809 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:43.955199 (+171390us) service_pool.cc:224] Handling call
0709 13:19:45.306228 (+1351029us) spinlock_profiling.cc:243] Waited 1.04 s on lock 0x7b5400394754. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:19:45.306362 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2385350528}
I20260709 13:19:45.348702 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62fe3346cd3c42c4a55e14af9073bca4" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:19:45.307549 16223 raft_consensus.cc:1240] T 8a878999c0024d2abe517484180290fc P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 3. Current term is 4. Ops: []
I20260709 13:19:45.310766 18156 leader_election.cc:290] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 4 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
W20260709 13:19:45.331710 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 97290.6us
I20260709 13:19:45.331947 16825 raft_consensus.cc:1275] T 3884f864a96d428785895e5463e5728f P cb82cc7bd6db46d980c75237a1402860 [term 3 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:19:45.336458 16149 raft_consensus.cc:2393] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 6: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
W20260709 13:19:45.337045 16117 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 60060.4us
I20260709 13:19:45.337095 16295 raft_consensus.cc:1275] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
I20260709 13:19:45.337888 16265 leader_election.cc:304] T 8d4ab870a05347868956aa8b841b2981 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 647a343a542d4a139fe106c19095129b
I20260709 13:19:45.298700 18262 raft_consensus.cc:493] T 02838c86531d4f98bf58d1c22b8baf77 P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:19:45.281924 18339 raft_consensus.cc:515] T 5a959dbecc2648eb85d1cc550f5a3245 P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:45.295272 18346 raft_consensus.cc:515] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:45.356523 16145 raft_consensus.cc:1240] T 5a96373bb0e14dce8a266b2fc6c58507 P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 2. Current term is 3. Ops: []
I20260709 13:19:45.506014 16297 raft_consensus.cc:2468] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 5.
I20260709 13:19:45.739882 16145 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40830 (request call id 1043) took 2383 ms. Trace:
W20260709 13:19:45.740101 16117 leader_election.cc:336] T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923): Timed out: RequestConsensusVote RPC to 127.14.177.132:38923 timed out after 1.532s (SENT)
W20260709 13:19:45.679281 16292 consensus_meta.cc:287] T 66ae176961b74813a58bc15c9c4c6f2a P 90f3da57796b488c9087a2f523adf045: Time spent flushing consensus metadata: real 0.545s	user 0.011s	sys 0.000s
I20260709 13:19:45.333065 18272 raft_consensus.cc:2749] T 68a6838d82ef4cac83507f906139d90f P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:19:45.361696 16222 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57600 (request call id 1018) took 2067 ms. Trace:
I20260709 13:19:45.741605 16292 raft_consensus.cc:2468] T 66ae176961b74813a58bc15c9c4c6f2a P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 6.
I20260709 13:19:45.454917 16143 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40830 (request call id 1038) took 2120 ms. Trace:
I20260709 13:19:45.741892 16222 rpcz_store.cc:276] 0709 13:19:43.293877 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:44.517704 (+1223827us) service_pool.cc:224] Handling call
0709 13:19:44.517987 (+   283us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:44.924145 (+406158us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:45.361356 (+437211us) spinlock_profiling.cc:243] Waited 389 ms on lock 0x7b54002b3c54. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:19:45.361611 (+   255us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":894791424}
I20260709 13:19:45.344035 16294 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40474 (request call id 320) took 1445 ms. Trace:
I20260709 13:19:45.743305 16294 rpcz_store.cc:276] 0709 13:19:43.899293 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:44.367783 (+468490us) service_pool.cc:224] Handling call
0709 13:19:45.341028 (+973245us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:19:45.745070 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 407199us
I20260709 13:19:45.746183 18041 consensus_queue.cc:1048] T fffab26427e24d398e2695aad40ca134 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:19:45.746922 18352 consensus_queue.cc:1059] T 5a96373bb0e14dce8a266b2fc6c58507 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:19:45.747867 18041 raft_consensus.cc:3055] T 5a96373bb0e14dce8a266b2fc6c58507 P 90f3da57796b488c9087a2f523adf045 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:19:45.748250 18041 raft_consensus.cc:740] T 5a96373bb0e14dce8a266b2fc6c58507 P 90f3da57796b488c9087a2f523adf045 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:19:45.748485 18352 consensus_queue.cc:1059] T c9944d5e4b8c4109af9955c4f08c9095 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:19:45.749502 18167 raft_consensus.cc:3055] T c9944d5e4b8c4109af9955c4f08c9095 P 90f3da57796b488c9087a2f523adf045 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:19:45.749121 18041 consensus_queue.cc:260] T 5a96373bb0e14dce8a266b2fc6c58507 P 90f3da57796b488c9087a2f523adf045 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:45.749902 18167 raft_consensus.cc:740] T c9944d5e4b8c4109af9955c4f08c9095 P 90f3da57796b488c9087a2f523adf045 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:19:45.750389 18041 raft_consensus.cc:3060] T 5a96373bb0e14dce8a266b2fc6c58507 P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:19:45.345899 18347 raft_consensus.cc:3060] T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:19:45.459928 17999 leader_election.cc:290] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 4 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
W20260709 13:19:45.379676 16264 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:45.752285 16265 leader_election.cc:304] T 8a878999c0024d2abe517484180290fc P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:19:45.454627 18256 consensus_queue.cc:260] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.7, Last appended by leader: 7, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:45.753154 17999 raft_consensus.cc:3060] T 8a878999c0024d2abe517484180290fc P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:19:45.744465 18156 raft_consensus.cc:2804] T 8d4ab870a05347868956aa8b841b2981 P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:19:45.445925 16823 raft_consensus.cc:1275] T 57a744f5aaf944678a63469cbf514015 P cb82cc7bd6db46d980c75237a1402860 [term 3 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:19:45.753747 18156 raft_consensus.cc:493] T 8d4ab870a05347868956aa8b841b2981 P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:45.753875 18256 raft_consensus.cc:3060] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:19:45.754196 18156 raft_consensus.cc:3060] T 8d4ab870a05347868956aa8b841b2981 P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:19:45.517135 16189 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:45.398681 16820 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49892 (request call id 325) took 1545 ms. Trace:
I20260709 13:19:45.454528 16220 raft_consensus.cc:2393] T ee9ed67006144797b1de6ea8c2013a68 P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 in current term 4: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
I20260709 13:19:45.682225 18254 leader_election.cc:290] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 4 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:45.750808 18167 consensus_queue.cc:260] T c9944d5e4b8c4109af9955c4f08c9095 P 90f3da57796b488c9087a2f523adf045 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.7, Last appended by leader: 7, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:45.756254 18167 raft_consensus.cc:3060] T c9944d5e4b8c4109af9955c4f08c9095 P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:19:45.356153 18034 raft_consensus.cc:2749] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [term 6 FOLLOWER]: Leader election lost for term 5. Reason: Vote denied by peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741) with higher term. Message: Invalid argument: T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 for earlier term 5. Current term is 6.
I20260709 13:19:45.761920 18156 raft_consensus.cc:515] T 8d4ab870a05347868956aa8b841b2981 P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:45.396768 16144 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40830 (request call id 1036) took 2071 ms. Trace:
W20260709 13:19:45.631444 18350 log.cc:927] Time spent T c4f65cdf00b748bb963ee6e81ef43b79 P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.104s	user 0.001s	sys 0.000s
I20260709 13:19:45.391505 16226 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57600 (request call id 1041) took 1557 ms. Trace:
I20260709 13:19:45.744685 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31969392ccdb43b491911a319e8bc7e1" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
W20260709 13:19:45.592904 18354 log.cc:927] Time spent T 00000000000000000000000000000000 P 667a34b27a39474ebefc1af3cf5b4271: Append to log took a long time: real 0.094s	user 0.001s	sys 0.001s
W20260709 13:19:45.436708 16263 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:45.440624 16150 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40830 (request call id 1053) took 1311 ms. Trace:
I20260709 13:19:45.392387 16147 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46224 (request call id 274) took 1190 ms. Trace:
I20260709 13:19:45.336102 18341 raft_consensus.cc:3060] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:19:45.356391 16190 leader_election.cc:304] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860
W20260709 13:19:45.697567 18248 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.502s	user 0.013s	sys 0.012s
I20260709 13:19:45.397557 16148 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40830 (request call id 1034) took 2072 ms. Trace:
I20260709 13:19:45.742244 16143 rpcz_store.cc:276] 0709 13:19:43.332898 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:45.304831 (+1971933us) service_pool.cc:224] Handling call
0709 13:19:45.305179 (+   348us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:45.355290 (+ 50111us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:45.453560 (+ 98270us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:45.466689 16819 raft_consensus.cc:1240] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860 [term 4 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 3. Current term is 4. Ops: []
W20260709 13:19:45.612501 18342 log.cc:927] Time spent T 434bff3082d141838068f323bdb0d1ed P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.314s	user 0.001s	sys 0.001s
W20260709 13:19:45.666208 18351 log.cc:927] Time spent T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 0.126s	user 0.001s	sys 0.000s
I20260709 13:19:45.767545 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5718c623ebee4d2bb770f0f9ddb2f164" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:19:45.773329 16148 rpcz_store.cc:276] 0709 13:19:43.325238 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:44.839671 (+1514433us) service_pool.cc:224] Handling call
0709 13:19:44.840008 (+   337us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:44.968664 (+128656us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:45.397290 (+428626us) spinlock_profiling.cc:243] Waited 106 ms on lock 0x7b5400072bd4. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:19:45.397447 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":243769216}
I20260709 13:19:45.769623 16150 rpcz_store.cc:276] 0709 13:19:44.129327 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:44.466638 (+337311us) service_pool.cc:224] Handling call
0709 13:19:45.440377 (+973739us) spinlock_profiling.cc:243] Waited 155 ms on lock 0x7b54000744d4. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:19:45.440553 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":357187968}
I20260709 13:19:45.787299 16148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66f75124215f4ca6bef831e174a792d8" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:19:45.771754 16820 rpcz_store.cc:276] 0709 13:19:43.853730 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:43.853849 (+   119us) service_pool.cc:224] Handling call
0709 13:19:45.398495 (+1544646us) spinlock_profiling.cc:243] Waited 600 ms on lock 0x7b5400625554. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:19:45.398624 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1380340608}
I20260709 13:19:45.740492 16145 rpcz_store.cc:276] 0709 13:19:43.356784 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:45.336239 (+1979455us) service_pool.cc:224] Handling call
0709 13:19:45.356344 (+ 20105us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:45.739598 (+383254us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:45.739819 (+   221us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:45.740521 18318 leader_election.cc:290] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 5 election: Requested vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:19:45.772229 16226 rpcz_store.cc:276] 0709 13:19:43.834239 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:44.321209 (+486970us) service_pool.cc:224] Handling call
0709 13:19:45.391293 (+1070084us) spinlock_profiling.cc:243] Waited 221 ms on lock 0x7b5400245c54. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:19:45.391425 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":509138688}
I20260709 13:19:45.790688 16295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a9de347c711477e87c582d003f36925" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
W20260709 13:19:45.777781 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 421451us
I20260709 13:19:45.763926 16144 rpcz_store.cc:276] 0709 13:19:43.325718 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:44.873402 (+1547684us) service_pool.cc:224] Handling call
0709 13:19:44.873750 (+   348us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:44.965012 (+ 91262us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:45.396549 (+431537us) spinlock_profiling.cc:243] Waited 90.5 ms on lock 0x7b54000e9f54. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:19:45.396711 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":208249728}
I20260709 13:19:45.444844 16217 raft_consensus.cc:1240] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac [term 4 LEADER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 3. Current term is 4. Ops: []
I20260709 13:19:45.779088 18255 raft_consensus.cc:2749] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:19:45.770049 16147 rpcz_store.cc:276] 0709 13:19:44.201749 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:44.341199 (+139450us) service_pool.cc:224] Handling call
0709 13:19:45.392198 (+1050999us) spinlock_profiling.cc:243] Waited 618 ms on lock 0x7b5400073ad4. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:19:45.392326 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1422204800}
I20260709 13:19:45.795611 16145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e65676b21b748728ad39668cfe722ec" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:19:45.796581 16145 raft_consensus.cc:2393] T 9e65676b21b748728ad39668cfe722ec P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 in current term 5: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
I20260709 13:19:45.787083 18123 consensus_queue.cc:1059] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.013s
I20260709 13:19:45.797515 18347 raft_consensus.cc:515] T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:45.784061 18167 consensus_queue.cc:1059] T 5a959dbecc2648eb85d1cc550f5a3245 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.014s
I20260709 13:19:45.798571 18123 raft_consensus.cc:3055] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:19:45.798796 16298 raft_consensus.cc:1240] T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 2. Current term is 4. Ops: []
I20260709 13:19:45.804286 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3884f864a96d428785895e5463e5728f" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:19:45.792670 16144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2372981006d4b218215c28964dea623" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:19:45.791271 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5718c623ebee4d2bb770f0f9ddb2f164" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:45.766280 16291 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48674 (request call id 2044) took 2806 ms. Trace:
I20260709 13:19:45.797623 16143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f82224f4564fea93572e8206bad1fe" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:19:45.813511 16144 raft_consensus.cc:2393] T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 in current term 4: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
I20260709 13:19:45.792459 16190 leader_election.cc:304] T 08c74490c40243a88e62d1760b5cd814 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:19:45.814456 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e4bfb9090c8460eb14fa7e0d818e7f1" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:45.815133 16190 leader_election.cc:304] T c0eab25ce4c64e018a041beeda2867db P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 647a343a542d4a139fe106c19095129b
I20260709 13:19:45.813205 16295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ad5dbfa99ee4b8d9e28be92b3f36db5" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
W20260709 13:19:45.773137 18248 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.588s	user 0.015s	sys 0.013s
I20260709 13:19:45.821182 16190 leader_election.cc:304] T d2ab0ac8347641e6955e9bce7d90c1a2 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: cb82cc7bd6db46d980c75237a1402860
I20260709 13:19:45.816159 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02838c86531d4f98bf58d1c22b8baf77" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:45.822621 16190 leader_election.cc:304] T 1df93779209748b9860f6d893dd3de53 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
W20260709 13:19:45.826339 16265 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:45.826787 16265 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:45.841408 16144 raft_consensus.cc:1240] T 2794af5b99984efebdee564035841cb0 P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 2. Current term is 3. Ops: [2.5-2.5]
I20260709 13:19:45.842347 16144 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40830 (request call id 1050) took 1984 ms. Trace:
I20260709 13:19:45.842998 16144 rpcz_store.cc:276] 0709 13:19:43.858130 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:45.840651 (+1982521us) service_pool.cc:224] Handling call
0709 13:19:45.841101 (+   450us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:45.842007 (+   906us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:45.842277 (+   270us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:45.844748 16144 raft_consensus.cc:1240] T 1df93779209748b9860f6d893dd3de53 P 647a343a542d4a139fe106c19095129b [term 4 LEADER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 3. Current term is 4. Ops: []
I20260709 13:19:45.783908 18339 leader_election.cc:290] T 5a959dbecc2648eb85d1cc550f5a3245 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:45.846901 16265 leader_election.cc:304] T b87147731a68400e8cce9b879daa7c5d P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:19:45.839694 18318 raft_consensus.cc:2804] T c0eab25ce4c64e018a041beeda2867db P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:19:45.852248 18154 raft_consensus.cc:2804] T b87147731a68400e8cce9b879daa7c5d P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:19:45.817685 18256 raft_consensus.cc:2749] T 08c74490c40243a88e62d1760b5cd814 P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:19:45.852376 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a959dbecc2648eb85d1cc550f5a3245" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:45.852742 18154 raft_consensus.cc:493] T b87147731a68400e8cce9b879daa7c5d P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:45.846400 18255 raft_consensus.cc:2804] T d2ab0ac8347641e6955e9bce7d90c1a2 P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:19:45.853143 18154 raft_consensus.cc:3060] T b87147731a68400e8cce9b879daa7c5d P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:19:45.853358 16226 raft_consensus.cc:2393] T 5a959dbecc2648eb85d1cc550f5a3245 P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 3: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
I20260709 13:19:45.853475 18255 raft_consensus.cc:697] T d2ab0ac8347641e6955e9bce7d90c1a2 P 1ca2725d290e47d68b1a4b609c9816ac [term 5 LEADER]: Becoming Leader. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:19:45.854458 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d4ab870a05347868956aa8b841b2981" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:19:45.798987 18123 raft_consensus.cc:740] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:19:45.848767 16190 leader_election.cc:304] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860
I20260709 13:19:45.847299 16144 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40830 (request call id 1057) took 1637 ms. Trace:
I20260709 13:19:45.858574 16144 rpcz_store.cc:276] 0709 13:19:44.210143 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:45.844130 (+1633987us) service_pool.cc:224] Handling call
0709 13:19:45.844505 (+   375us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:45.845699 (+  1194us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:45.847246 (+  1547us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:45.859107 16190 leader_election.cc:304] T d16dfad0c3b540e5b4b86492ddf87208 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:19:45.764267 18156 leader_election.cc:290] T 8d4ab870a05347868956aa8b841b2981 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 5 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
W20260709 13:19:45.801563 18317 consensus_meta.cc:287] T c0eab25ce4c64e018a041beeda2867db P 647a343a542d4a139fe106c19095129b: Time spent flushing consensus metadata: real 0.530s	user 0.007s	sys 0.000s
I20260709 13:19:45.799603 18323 raft_consensus.cc:3055] T 5a959dbecc2648eb85d1cc550f5a3245 P 90f3da57796b488c9087a2f523adf045 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:19:45.863883 18323 raft_consensus.cc:740] T 5a959dbecc2648eb85d1cc550f5a3245 P 90f3da57796b488c9087a2f523adf045 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:19:45.864655 18323 consensus_queue.cc:260] T 5a959dbecc2648eb85d1cc550f5a3245 P 90f3da57796b488c9087a2f523adf045 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:19:45.865433 18179 consensus_meta.cc:287] T 26354361e9b044cea1c2b86c4882fe92 P cb82cc7bd6db46d980c75237a1402860: Time spent flushing consensus metadata: real 0.544s	user 0.010s	sys 0.000s
I20260709 13:19:45.865794 18323 raft_consensus.cc:3060] T 5a959dbecc2648eb85d1cc550f5a3245 P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:19:45.867820 18123 consensus_queue.cc:260] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:45.868770 18123 raft_consensus.cc:3060] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:19:45.869017 18240 consensus_meta.cc:287] T ee9ed67006144797b1de6ea8c2013a68 P 647a343a542d4a139fe106c19095129b: Time spent flushing consensus metadata: real 0.527s	user 0.009s	sys 0.000s
I20260709 13:19:45.769101 18262 raft_consensus.cc:515] T 02838c86531d4f98bf58d1c22b8baf77 P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:45.870992 18262 leader_election.cc:290] T 02838c86531d4f98bf58d1c22b8baf77 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:45.850868 18109 raft_consensus.cc:2749] T 1df93779209748b9860f6d893dd3de53 P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:19:45.768437 18346 leader_election.cc:290] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:19:45.873507 18105 leader_election.cc:290] T 9c61a39fd4f84424b129c1a6f9ba404d P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:45.875982 18114 leader_election.cc:290] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 5 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:45.876922 16224 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57600 (request call id 1043) took 1535 ms. Trace:
I20260709 13:19:45.877275 16224 rpcz_store.cc:276] 0709 13:19:44.342072 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:44.974542 (+632470us) service_pool.cc:224] Handling call
0709 13:19:44.974905 (+   363us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:45.141528 (+166623us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:45.876707 (+735179us) spinlock_profiling.cc:243] Waited 135 ms on lock 0x7b54002c9554. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:19:45.876869 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":310654848}
I20260709 13:19:45.878435 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806d7690c2b644959152074bb2cab0a7" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:19:45.880106 16820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62fe3346cd3c42c4a55e14af9073bca4" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:19:45.880868 16820 raft_consensus.cc:2393] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 6: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
I20260709 13:19:45.857370 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2794af5b99984efebdee564035841cb0" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:19:45.857879 16265 leader_election.cc:304] T ecc1e363230043fe9fef390f0b979f20 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:19:45.883096 16149 raft_consensus.cc:2393] T 2794af5b99984efebdee564035841cb0 P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 in current term 3: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
I20260709 13:19:45.857012 16226 raft_consensus.cc:3060] T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:19:45.860857 17844 raft_consensus.cc:2749] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:19:45.817579 16291 rpcz_store.cc:276] 0709 13:19:42.959866 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:44.532969 (+1573103us) service_pool.cc:224] Handling call
0709 13:19:44.533327 (+   358us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:19:44.533723 (+   396us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:44.533761 (+    38us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:19:44.901879 (+368118us) log.cc:844] Serialized 52 byte log entry
0709 13:19:45.287680 (+385801us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:19:45.295018 (+  7338us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:45.303892 (+  8874us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:45.766027 (+462135us) raft_consensus.cc:1685] finished
0709 13:19:45.766056 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:45.766223 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":578,"raft.run_cpu_time_us":98,"raft.run_wall_time_us":98,"thread_start_us":312628,"threads_started":1,"wal-append.queue_time_us":416979}
I20260709 13:19:45.863036 17999 raft_consensus.cc:2749] T 8a878999c0024d2abe517484180290fc P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:19:45.845981 18041 raft_consensus.cc:493] T 3884f864a96d428785895e5463e5728f P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:45.799832 18347 leader_election.cc:290] T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 4 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:19:45.883713 16265 leader_election.cc:304] T 84f2da98864b4425bf7c0055951fdd4b P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:19:45.898886 16265 leader_election.cc:304] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:19:45.851118 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b7d1ab519044eda2d5e04a07242f25" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:19:45.879009 16224 raft_consensus.cc:3060] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:19:45.890861 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 122418us
I20260709 13:19:45.892931 16190 leader_election.cc:304] T ecc1e363230043fe9fef390f0b979f20 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:19:45.895447 16820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e65676b21b748728ad39668cfe722ec" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:19:45.865988 18318 raft_consensus.cc:697] T c0eab25ce4c64e018a041beeda2867db P 1ca2725d290e47d68b1a4b609c9816ac [term 4 LEADER]: Becoming Leader. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:19:45.901954 16265 leader_election.cc:304] T c4f65cdf00b748bb963ee6e81ef43b79 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:19:45.915302 16265 leader_election.cc:304] T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac
I20260709 13:19:45.902372 16149 raft_consensus.cc:1275] T af57996a8be84d398f0f023e8d08ef77 P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:19:45.902143 16190 leader_election.cc:304] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:19:45.917558 16190 leader_election.cc:304] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: cb82cc7bd6db46d980c75237a1402860
I20260709 13:19:45.918936 16190 leader_election.cc:304] T e5ad4eb281564936a59f41e387a47156 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 647a343a542d4a139fe106c19095129b
I20260709 13:19:45.854501 18255 consensus_queue.cc:237] T d2ab0ac8347641e6955e9bce7d90c1a2 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:45.914327 16790 leader_election.cc:304] T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
W20260709 13:19:45.960700 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 46444.6us
I20260709 13:19:45.961648 16790 leader_election.cc:304] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:19:45.894510 18256 raft_consensus.cc:2749] T d16dfad0c3b540e5b4b86492ddf87208 P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:19:45.891814 18346 raft_consensus.cc:2749] T ecc1e363230043fe9fef390f0b979f20 P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:19:45.908803 18167 raft_consensus.cc:2749] T 84f2da98864b4425bf7c0055951fdd4b P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:19:45.901803 18041 raft_consensus.cc:515] T 3884f864a96d428785895e5463e5728f P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:45.913193 18271 raft_consensus.cc:2804] T ecc1e363230043fe9fef390f0b979f20 P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:19:45.908859 18156 raft_consensus.cc:2749] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:19:45.950160 16151 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40830 (request call id 1051) took 1841 ms. Trace:
I20260709 13:19:46.006705 16151 rpcz_store.cc:276] 0709 13:19:44.109118 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:44.467353 (+358235us) service_pool.cc:224] Handling call
0709 13:19:45.949977 (+1482624us) spinlock_profiling.cc:243] Waited 116 ms on lock 0x7b540012fa54. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:19:45.950094 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":267648384}
I20260709 13:19:45.941799 18109 raft_consensus.cc:2804] T e5ad4eb281564936a59f41e387a47156 P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 13:19:45.951200 18313 consensus_meta.cc:287] T 18ad7e3af5504571a69d10c00750a01c P 1ca2725d290e47d68b1a4b609c9816ac: Time spent flushing consensus metadata: real 0.612s	user 0.009s	sys 0.000s
I20260709 13:19:45.940201 18254 raft_consensus.cc:2749] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:19:45.942500 18347 raft_consensus.cc:2749] T c4f65cdf00b748bb963ee6e81ef43b79 P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:19:45.952229 16219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:19:45.955505 17844 consensus_queue.cc:1048] T 8c11074579b046d99e585c2818e18a2c P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:19:45.955936 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8158bbd17b994d2eb96f560bd0f1b88d" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:46.030128 16225 raft_consensus.cc:2393] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 5: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
I20260709 13:19:45.965935 16265 leader_election.cc:304] T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:19:45.865813 18179 raft_consensus.cc:515] T 26354361e9b044cea1c2b86c4882fe92 P cb82cc7bd6db46d980c75237a1402860 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:45.992080 16224 raft_consensus.cc:2468] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 4.
I20260709 13:19:46.039711 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a378d7f796f94615b3b347119a9880fd" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:19:46.040356 16224 raft_consensus.cc:3060] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:19:45.995390 16823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "829b4ad8c5a54eababeea1181b960d4b" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:19:46.040557 18179 leader_election.cc:290] T 26354361e9b044cea1c2b86c4882fe92 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 3 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:46.002064 16821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdbd78acb4cf46298e0de8ba0d130e19" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:19:45.941927 18114 raft_consensus.cc:2804] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 1ca2725d290e47d68b1a4b609c9816ac [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:19:46.042449 18114 raft_consensus.cc:493] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 1ca2725d290e47d68b1a4b609c9816ac [term 6 FOLLOWER]: Starting leader election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:19:46.042840 18114 raft_consensus.cc:3060] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 1ca2725d290e47d68b1a4b609c9816ac [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:19:46.009124 16825 raft_consensus.cc:1275] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 [term 4 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
W20260709 13:19:46.045377 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 79585.1us
I20260709 13:19:46.048530 16821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5d59407eb545b895d8d922e0dd69fd" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:19:46.051293 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02838c86531d4f98bf58d1c22b8baf77" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:46.052666 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c2095a5f4ad49408dbc7f69c606120d" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:46.053849 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c61a39fd4f84424b129c1a6f9ba404d" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:46.010829 18271 raft_consensus.cc:493] T ecc1e363230043fe9fef390f0b979f20 P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:46.018766 16222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66f75124215f4ca6bef831e174a792d8" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
W20260709 13:19:46.020135 16117 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 279038us
I20260709 13:19:46.082351 16222 raft_consensus.cc:2393] T 66f75124215f4ca6bef831e174a792d8 P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 in current term 5: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
W20260709 13:19:46.082412 16117 leader_election.cc:336] T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923): Timed out: RequestConsensusVote RPC to 127.14.177.132:38923 timed out after 1.714s (SENT)
I20260709 13:19:46.010417 18109 raft_consensus.cc:493] T e5ad4eb281564936a59f41e387a47156 P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Starting leader election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:19:45.962109 16291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8158bbd17b994d2eb96f560bd0f1b88d" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:45.977936 18034 raft_consensus.cc:2749] T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:19:46.022792 18313 raft_consensus.cc:515] T 18ad7e3af5504571a69d10c00750a01c P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } }
I20260709 13:19:46.033746 18245 raft_consensus.cc:2749] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:19:46.035504 16218 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33050 (request call id 412) took 1028 ms. Trace:
I20260709 13:19:45.993678 16226 raft_consensus.cc:2468] T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 5.
I20260709 13:19:46.081269 18156 raft_consensus.cc:493] T d16dfad0c3b540e5b4b86492ddf87208 P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:46.089581 16222 raft_consensus.cc:1240] T e4dc6b0541ca431ebc757beac00521f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 3. Current term is 4. Ops: []
I20260709 13:19:46.116415 16821 raft_consensus.cc:2376] T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860 [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 647a343a542d4a139fe106c19095129b in term 5. Re-sending same reply.
I20260709 13:19:46.118533 16222 raft_consensus.cc:1240] T 19baa7bc367844c288bca8f81439c413 P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 4. Current term is 5. Ops: []
I20260709 13:19:46.120867 16222 raft_consensus.cc:1275] T 08c74490c40243a88e62d1760b5cd814 P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:19:46.091889 16816 raft_consensus.cc:1275] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [term 6 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
W20260709 13:19:46.083236 18140 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.554s	user 0.004s	sys 0.017s
I20260709 13:19:46.096091 16817 raft_consensus.cc:1240] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 [term 4 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 3. Current term is 4. Ops: [3.6-3.6]
I20260709 13:19:46.081287 18346 consensus_queue.cc:1059] T 8158bbd17b994d2eb96f560bd0f1b88d P 90f3da57796b488c9087a2f523adf045 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.043s
I20260709 13:19:46.096364 16224 raft_consensus.cc:2468] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 4.
W20260709 13:19:46.092818 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 301058us
W20260709 13:19:46.017100 16297 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40474 (request call id 319) took 2121 ms (client timeout 2037 ms). Trace:
I20260709 13:19:46.098174 16226 raft_consensus.cc:1240] T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 4. Current term is 5. Ops: []
I20260709 13:19:46.083539 18271 raft_consensus.cc:3060] T ecc1e363230043fe9fef390f0b979f20 P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:19:46.088084 18109 raft_consensus.cc:3060] T e5ad4eb281564936a59f41e387a47156 P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:19:46.129561 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a9de347c711477e87c582d003f36925" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:19:46.137091 16150 raft_consensus.cc:2368] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 for earlier term 4. Current term is 5.
I20260709 13:19:46.062532 18347 raft_consensus.cc:2764] T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Leader pre-election decision vote started in defunct term 3: won
I20260709 13:19:46.094487 16220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ad5dbfa99ee4b8d9e28be92b3f36db5" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:19:46.131089 16225 raft_consensus.cc:1240] T 18ad7e3af5504571a69d10c00750a01c P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 3. Current term is 4. Ops: []
I20260709 13:19:46.133523 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02838c86531d4f98bf58d1c22b8baf77" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:19:46.134755 16224 raft_consensus.cc:1240] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 3. Current term is 4. Ops: []
W20260709 13:19:46.135174 16297 rpcz_store.cc:269] 0709 13:19:43.895623 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:44.196123 (+300500us) service_pool.cc:224] Handling call
0709 13:19:46.016858 (+1820735us) spinlock_profiling.cc:243] Waited 890 ms on lock 0x7b5400353ed4. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:19:46.017012 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2047737216}
W20260709 13:19:46.116129 16266 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.523s	user 0.019s	sys 0.009s
W20260709 13:19:46.149621 16266 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.556s	user 0.019s	sys 0.010s
W20260709 13:19:46.149896 16266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 592074us
I20260709 13:19:46.150053 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c61a39fd4f84424b129c1a6f9ba404d" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
W20260709 13:19:46.131268 18140 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.603s	user 0.004s	sys 0.017s
I20260709 13:19:46.148366 16149 raft_consensus.cc:2468] T 02838c86531d4f98bf58d1c22b8baf77 P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 4.
I20260709 13:19:46.125975 18034 raft_consensus.cc:493] T 8c11074579b046d99e585c2818e18a2c P cb82cc7bd6db46d980c75237a1402860 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:19:46.159586 18276 raft_consensus.cc:2804] T 68a6838d82ef4cac83507f906139d90f P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 13:19:46.161931 16191 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:46.168658 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 227770us
I20260709 13:19:46.085371 18318 consensus_queue.cc:237] T c0eab25ce4c64e018a041beeda2867db P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:19:46.166066 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 45706.9us
W20260709 13:19:46.164031 16192 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 199570us
I20260709 13:19:46.170162 16297 raft_consensus.cc:1240] T 8d4ab870a05347868956aa8b841b2981 P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 4. Current term is 5. Ops: [4.2-4.2]
I20260709 13:19:46.169104 16191 leader_election.cc:304] T 8cc959fb7f704fba9009c0595467d074 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:19:46.174531 16297 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48674 (request call id 2074) took 1207 ms. Trace:
W20260709 13:19:46.170576 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 208996us
I20260709 13:19:46.174954 16297 rpcz_store.cc:276] 0709 13:19:44.967325 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:46.150913 (+1183588us) service_pool.cc:224] Handling call
0709 13:19:46.169940 (+ 19027us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:46.174210 (+  4270us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:46.174467 (+   257us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:19:46.170615 16789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 109068us
I20260709 13:19:46.177780 16790 leader_election.cc:304] T ee9ed67006144797b1de6ea8c2013a68 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:19:46.126890 18167 consensus_queue.cc:1059] T 9e65676b21b748728ad39668cfe722ec P 90f3da57796b488c9087a2f523adf045 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.032s
I20260709 13:19:46.125986 18156 raft_consensus.cc:515] T d16dfad0c3b540e5b4b86492ddf87208 P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:46.136991 18145 consensus_queue.cc:1059] T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.042s
I20260709 13:19:46.157758 17999 raft_consensus.cc:2804] T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:19:46.144948 18240 consensus_queue.cc:1048] T 08c74490c40243a88e62d1760b5cd814 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.035s
W20260709 13:19:46.240579 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 62864us
I20260709 13:19:46.136221 18262 consensus_queue.cc:1059] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.043s
I20260709 13:19:46.262192 18145 raft_consensus.cc:3055] T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:19:46.262722 18145 raft_consensus.cc:740] T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:19:46.263585 18145 consensus_queue.cc:260] T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:46.262382 16816 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5718c623ebee4d2bb770f0f9ddb2f164"
dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
mode: GRACEFUL
new_leader_uuid: "647a343a542d4a139fe106c19095129b"
 from {username='slave'} at 127.0.0.1:51986
I20260709 13:19:46.264946 16816 raft_consensus.cc:606] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 [term 4 FOLLOWER]: Received request to transfer leadership to TS 647a343a542d4a139fe106c19095129b
I20260709 13:19:46.265628 16816 raft_consensus.cc:614] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 [term 4 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:19:46.264990 18262 raft_consensus.cc:3055] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:19:46.173555 18034 raft_consensus.cc:515] T 8c11074579b046d99e585c2818e18a2c P cb82cc7bd6db46d980c75237a1402860 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:46.289655 16822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a9de347c711477e87c582d003f36925" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
W20260709 13:19:46.307350 18360 log.cc:927] Time spent T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 0.180s	user 0.001s	sys 0.000s
I20260709 13:19:46.313977 18041 leader_election.cc:290] T 3884f864a96d428785895e5463e5728f P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:19:46.314287 18156 leader_election.cc:290] T d16dfad0c3b540e5b4b86492ddf87208 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:19:46.314913 18154 raft_consensus.cc:515] T b87147731a68400e8cce9b879daa7c5d P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:46.317972 18123 raft_consensus.cc:493] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:46.304752 18245 consensus_queue.cc:1059] T d2ab0ac8347641e6955e9bce7d90c1a2 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:19:46.249444 18347 consensus_queue.cc:1048] T 8158bbd17b994d2eb96f560bd0f1b88d P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.116s
W20260709 13:19:46.384773 18371 log.cc:927] Time spent T 68a6838d82ef4cac83507f906139d90f P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.118s	user 0.000s	sys 0.001s
I20260709 13:19:46.427449 18374 raft_consensus.cc:3060] T ee9ed67006144797b1de6ea8c2013a68 P cb82cc7bd6db46d980c75237a1402860 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:19:46.321754 18341 consensus_meta.cc:287] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 90f3da57796b488c9087a2f523adf045: Time spent flushing consensus metadata: real 0.551s	user 0.004s	sys 0.004s
I20260709 13:19:46.463444 18341 raft_consensus.cc:515] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:46.466320 18341 leader_election.cc:290] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 4 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:19:46.292660 18262 raft_consensus.cc:740] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:19:46.468809 18271 raft_consensus.cc:515] T ecc1e363230043fe9fef390f0b979f20 P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:46.320741 18313 leader_election.cc:290] T 18ad7e3af5504571a69d10c00750a01c P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 4 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:46.321487 18323 consensus_queue.cc:1059] T 2794af5b99984efebdee564035841cb0 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.155s
I20260709 13:19:46.500993 18262 consensus_queue.cc:260] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.5, Last appended by leader: 5, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:46.503283 18262 raft_consensus.cc:3060] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:19:46.509799 18271 leader_election.cc:290] T ecc1e363230043fe9fef390f0b979f20 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 4 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:19:46.528337 18390 raft_consensus.cc:3055] T 2794af5b99984efebdee564035841cb0 P 90f3da57796b488c9087a2f523adf045 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:19:46.528966 18390 raft_consensus.cc:740] T 2794af5b99984efebdee564035841cb0 P 90f3da57796b488c9087a2f523adf045 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:19:46.530022 18390 consensus_queue.cc:260] T 2794af5b99984efebdee564035841cb0 P 90f3da57796b488c9087a2f523adf045 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:46.531431 18390 raft_consensus.cc:3060] T 2794af5b99984efebdee564035841cb0 P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:19:46.514925 18374 raft_consensus.cc:2749] T ee9ed67006144797b1de6ea8c2013a68 P cb82cc7bd6db46d980c75237a1402860 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:19:46.117650 16142 raft_consensus.cc:2468] T ee9ed67006144797b1de6ea8c2013a68 P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 4.
I20260709 13:19:46.603993 16151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d4ab870a05347868956aa8b841b2981" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
W20260709 13:19:46.608006 16117 leader_election.cc:336] T ecc1e363230043fe9fef390f0b979f20 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923): Timed out: RequestConsensusVote RPC to 127.14.177.132:38923 timed out after 1.590s (SENT)
W20260709 13:19:46.606437 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 452538us
I20260709 13:19:46.616442 16219 raft_consensus.cc:2393] T ecc1e363230043fe9fef390f0b979f20 P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 4: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
W20260709 13:19:46.616786 16189 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:46.620048 16144 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40830 (request call id 1059) took 2306 ms. Trace:
I20260709 13:19:46.623759 16220 raft_consensus.cc:2393] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 4: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
I20260709 13:19:46.624609 16220 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57604 (request call id 1989) took 1323 ms. Trace:
I20260709 13:19:46.625052 16220 rpcz_store.cc:276] 0709 13:19:45.300698 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:46.049657 (+748959us) service_pool.cc:224] Handling call
0709 13:19:46.624456 (+574799us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:46.626873 16151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e9b2bf7011f4ae688a79d7bb5a447b4" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:19:46.629216 16151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6f07842d3aa47858df5665874c86f3a" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:19:46.629493 16822 raft_consensus.cc:2393] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 5: Already voted for candidate cb82cc7bd6db46d980c75237a1402860 in this term.
I20260709 13:19:46.097018 16218 rpcz_store.cc:276] 0709 13:19:45.006859 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:45.281023 (+274164us) service_pool.cc:224] Handling call
0709 13:19:45.309001 (+ 27978us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:45.322496 (+ 13495us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:19:45.322533 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:45.323058 (+   525us) log.cc:844] Serialized 26 byte log entry
0709 13:19:45.453364 (+130306us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:19:45.456522 (+  3158us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:45.456645 (+   123us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:46.035260 (+578615us) raft_consensus.cc:1685] finished
0709 13:19:46.035282 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:46.035441 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":115,"raft.run_cpu_time_us":85,"raft.run_wall_time_us":85,"thread_start_us":77914,"threads_started":1,"wal-append.queue_time_us":585089}
W20260709 13:19:46.635730 16789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 458018us
I20260709 13:19:46.637248 16296 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48684 (request call id 1951) took 2276 ms. Trace:
I20260709 13:19:46.635789 16151 raft_consensus.cc:3060] T b6f07842d3aa47858df5665874c86f3a P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:19:46.640080 16293 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48674 (request call id 2046) took 3679 ms. Trace:
I20260709 13:19:46.641160 16295 raft_consensus.cc:2393] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 4: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
I20260709 13:19:46.642280 18034 leader_election.cc:290] T 8c11074579b046d99e585c2818e18a2c P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
W20260709 13:19:46.643187 16265 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:46.643985 16265 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:46.640661 16293 rpcz_store.cc:276] 0709 13:19:42.960475 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:44.534976 (+1574501us) service_pool.cc:224] Handling call
0709 13:19:44.535342 (+   366us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:44.535572 (+   230us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:44.535599 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:45.949204 (+1413605us) spinlock_profiling.cc:243] Waited 44.1 ms on lock 0x7b2c00132978. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69af458a11 00007f69af468372 00007f69b0b9aa01 00007f69b0b5f43a 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2
0709 13:19:45.949663 (+   459us) log.cc:844] Serialized 26 byte log entry
0709 13:19:46.159282 (+209619us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:19:46.159346 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:46.159456 (+   110us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:46.639368 (+479912us) raft_consensus.cc:1685] finished
0709 13:19:46.639438 (+    70us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:46.639842 (+   404us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":101318400,"thread_start_us":195311,"threads_started":1,"wal-append.queue_time_us":277720}
I20260709 13:19:46.181079 16291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c2095a5f4ad49408dbc7f69c606120d" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:46.649096 16822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:19:46.650033 16822 raft_consensus.cc:3060] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:19:46.649251 16291 raft_consensus.cc:2410] T 7c2095a5f4ad49408dbc7f69c606120d P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b for term 4 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
W20260709 13:19:46.621322 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 452145us
W20260709 13:19:46.614719 16117 leader_election.cc:336] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923): Timed out: RequestConsensusVote RPC to 127.14.177.132:38923 timed out after 1.726s (SENT)
I20260709 13:19:46.610901 16143 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40844 (request call id 1825) took 1602 ms. Trace:
I20260709 13:19:46.415254 18145 raft_consensus.cc:3060] T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:19:46.637848 16296 rpcz_store.cc:276] 0709 13:19:44.360957 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:45.221328 (+860371us) service_pool.cc:224] Handling call
0709 13:19:45.221748 (+   420us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:45.222235 (+   487us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:45.222278 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:45.222945 (+   667us) log.cc:844] Serialized 26 byte log entry
0709 13:19:46.061069 (+838124us) spinlock_profiling.cc:243] Waited 6.33 ms on lock 0x7b18002ad300. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b693ade1 00007f69b69475b1 00007f69ada54e07 00007f69b68afab9 00007f69ada7169b 00007f69ada6f93f 00007f69ada71c02 00007f69b06f920b 00007f69b06fed23 00007f69b06ff0d1 00007f69b0b1aa0f
0709 13:19:46.156587 (+ 95518us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:19:46.156633 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:46.156812 (+   179us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:46.636362 (+479550us) raft_consensus.cc:1685] finished
0709 13:19:46.636411 (+    49us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:46.636924 (+   513us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":125,"raft.run_cpu_time_us":119,"raft.run_wall_time_us":5553,"spinlock_wait_cycles":14550272,"thread_start_us":95316,"threads_started":1,"wal-append.queue_time_us":1222498}
I20260709 13:19:46.698307 16220 raft_consensus.cc:1275] T 1df93779209748b9860f6d893dd3de53 P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:19:46.426647 18123 raft_consensus.cc:515] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:46.705859 18271 raft_consensus.cc:2749] T 8cc959fb7f704fba9009c0595467d074 P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:19:46.707386 18255 consensus_queue.cc:1048] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:19:46.709118 16220 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57604 (request call id 1992) took 1401 ms. Trace:
I20260709 13:19:46.476228 18245 raft_consensus.cc:3055] T d2ab0ac8347641e6955e9bce7d90c1a2 P cb82cc7bd6db46d980c75237a1402860 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:19:46.710171 16220 rpcz_store.cc:276] 0709 13:19:45.301409 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:46.626867 (+1325458us) service_pool.cc:224] Handling call
0709 13:19:46.691467 (+ 64600us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:46.700176 (+  8709us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:46.702587 (+  2411us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:46.695570 16291 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48674 (request call id 2080) took 1293 ms. Trace:
W20260709 13:19:46.708218 16117 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 93627.6us
I20260709 13:19:46.717033 16220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6f07842d3aa47858df5665874c86f3a" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:19:46.718164 16220 raft_consensus.cc:3060] T b6f07842d3aa47858df5665874c86f3a P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:19:46.719597 18384 raft_consensus.cc:493] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:46.720502 18384 raft_consensus.cc:515] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:46.721601 18154 leader_election.cc:290] T b87147731a68400e8cce9b879daa7c5d P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 6 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:19:46.723977 16191 leader_election.cc:304] T af57996a8be84d398f0f023e8d08ef77 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: cb82cc7bd6db46d980c75237a1402860
I20260709 13:19:46.726413 16191 leader_election.cc:304] T 84f2da98864b4425bf7c0055951fdd4b P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
W20260709 13:19:46.719952 16117 leader_election.cc:336] T 2794af5b99984efebdee564035841cb0 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923): Timed out: RequestConsensusVote RPC to 127.14.177.132:38923 timed out after 1.903s (SENT)
I20260709 13:19:46.733196 16117 leader_election.cc:304] T 2794af5b99984efebdee564035841cb0 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860
I20260709 13:19:46.726490 17844 raft_consensus.cc:2804] T af57996a8be84d398f0f023e8d08ef77 P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:19:46.733911 16265 leader_election.cc:304] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:19:46.734504 17844 raft_consensus.cc:493] T af57996a8be84d398f0f023e8d08ef77 P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:46.732378 18245 raft_consensus.cc:740] T d2ab0ac8347641e6955e9bce7d90c1a2 P cb82cc7bd6db46d980c75237a1402860 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:19:46.620746 16144 rpcz_store.cc:276] 0709 13:19:44.313381 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:45.859822 (+1546441us) service_pool.cc:224] Handling call
0709 13:19:45.860215 (+   393us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:45.896409 (+ 36194us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:45.896491 (+    82us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:45.902840 (+  6349us) log.cc:844] Serialized 26 byte log entry
0709 13:19:46.136782 (+233942us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:19:46.136822 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:46.136881 (+    59us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:46.608878 (+471997us) raft_consensus.cc:1685] finished
0709 13:19:46.608949 (+    71us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:46.619719 (+ 10770us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":26195,"raft.queue_time_us":15465,"raft.run_cpu_time_us":163,"raft.run_wall_time_us":16044,"thread_start_us":188551,"threads_started":1,"wal-append.queue_time_us":491278}
I20260709 13:19:46.735215 18318 raft_consensus.cc:493] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:46.736202 18318 raft_consensus.cc:515] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:46.736722 16144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806d7690c2b644959152074bb2cab0a7" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:19:46.735098 17844 raft_consensus.cc:3060] T af57996a8be84d398f0f023e8d08ef77 P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:19:46.736579 18245 consensus_queue.cc:260] T d2ab0ac8347641e6955e9bce7d90c1a2 P cb82cc7bd6db46d980c75237a1402860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:46.738145 16144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c2095a5f4ad49408dbc7f69c606120d" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:19:46.738345 18245 raft_consensus.cc:3060] T d2ab0ac8347641e6955e9bce7d90c1a2 P cb82cc7bd6db46d980c75237a1402860 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:19:46.739538 18057 raft_consensus.cc:2749] T 2794af5b99984efebdee564035841cb0 P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:19:46.714722 18240 consensus_queue.cc:1059] T 8d4ab870a05347868956aa8b841b2981 P 647a343a542d4a139fe106c19095129b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.028s
I20260709 13:19:46.741685 18140 raft_consensus.cc:3055] T 8d4ab870a05347868956aa8b841b2981 P 647a343a542d4a139fe106c19095129b [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:19:46.742097 18140 raft_consensus.cc:740] T 8d4ab870a05347868956aa8b841b2981 P 647a343a542d4a139fe106c19095129b [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:19:46.738996 16144 raft_consensus.cc:2468] T 7c2095a5f4ad49408dbc7f69c606120d P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 3.
I20260709 13:19:46.715087 18323 raft_consensus.cc:493] T c0eab25ce4c64e018a041beeda2867db P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:46.743541 18140 consensus_queue.cc:260] T 8d4ab870a05347868956aa8b841b2981 P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 4.2, Last appended by leader: 2, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:46.748910 18323 raft_consensus.cc:515] T c0eab25ce4c64e018a041beeda2867db P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:46.754693 16223 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57604 (request call id 1972) took 3274 ms. Trace:
I20260709 13:19:46.756026 16223 rpcz_store.cc:276] 0709 13:19:43.476162 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:44.897902 (+1421740us) service_pool.cc:224] Handling call
0709 13:19:45.307373 (+409471us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:45.350043 (+ 42670us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:46.750388 (+1400345us) spinlock_profiling.cc:243] Waited 409 ms on lock 0x7b5400394750. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5e8ad 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:19:46.750614 (+   226us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":940872576}
I20260709 13:19:46.758584 16223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d16dfad0c3b540e5b4b86492ddf87208" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:46.745994 18384 leader_election.cc:290] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:46.712036 18374 raft_consensus.cc:493] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:46.761641 18271 consensus_queue.cc:1048] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.002s
I20260709 13:19:46.761943 16223 raft_consensus.cc:2468] T d16dfad0c3b540e5b4b86492ddf87208 P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 3.
I20260709 13:19:46.712421 16291 rpcz_store.cc:276] 0709 13:19:45.400608 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:46.151724 (+751116us) service_pool.cc:224] Handling call
0709 13:19:46.693039 (+541315us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:46.759019 18323 leader_election.cc:290] T c0eab25ce4c64e018a041beeda2867db P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:19:46.731300 18390 raft_consensus.cc:493] T cdbd78acb4cf46298e0de8ba0d130e19 P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:19:46.769776 18374 raft_consensus.cc:515] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:46.770761 18154 raft_consensus.cc:2804] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:19:46.771462 18140 raft_consensus.cc:3060] T 8d4ab870a05347868956aa8b841b2981 P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:19:46.694407 16143 rpcz_store.cc:276] 0709 13:19:45.008587 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:45.863623 (+855036us) service_pool.cc:224] Handling call
0709 13:19:45.863971 (+   348us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:45.901660 (+ 37689us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:45.915298 (+ 13638us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:46.057070 (+141772us) log.cc:844] Serialized 26 byte log entry
0709 13:19:46.609778 (+552708us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:19:46.609977 (+   199us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:46.610184 (+   207us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:46.610298 (+   114us) raft_consensus.cc:1685] finished
0709 13:19:46.610316 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:46.610616 (+   300us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":20839,"raft.queue_time_us":28753,"raft.run_cpu_time_us":143,"raft.run_wall_time_us":6950,"thread_start_us":480055,"threads_started":1,"wal-append.queue_time_us":427666}
I20260709 13:19:46.772760 16225 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57604 (request call id 1978) took 1479 ms. Trace:
I20260709 13:19:46.779367 16225 rpcz_store.cc:276] 0709 13:19:45.292894 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:46.071396 (+778502us) service_pool.cc:224] Handling call
0709 13:19:46.071706 (+   310us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:46.147915 (+ 76209us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:46.764972 (+617057us) spinlock_profiling.cc:243] Waited 59.2 ms on lock 0x7b5400367254. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:19:46.772543 (+  7571us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":136083840}
I20260709 13:19:46.782043 16225 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33050 (request call id 415) took 1433 ms. Trace:
I20260709 13:19:46.778064 18154 raft_consensus.cc:697] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045 [term 4 LEADER]: Becoming Leader. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:19:46.777725 18390 raft_consensus.cc:515] T cdbd78acb4cf46298e0de8ba0d130e19 P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:19:46.782809 16292 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48684 (request call id 1942) took 2928 ms (client timeout 1519 ms). Trace:
I20260709 13:19:46.784945 18374 leader_election.cc:290] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:46.785516 16818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:19:46.787838 18390 leader_election.cc:290] T cdbd78acb4cf46298e0de8ba0d130e19 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:19:46.790524 16295 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48674 (request call id 2078) took 1395 ms. Trace:
W20260709 13:19:46.783516 16292 rpcz_store.cc:269] 0709 13:19:43.854663 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:43.863545 (+  8882us) service_pool.cc:224] Handling call
0709 13:19:44.362918 (+499373us) spinlock_profiling.cc:243] Waited 2.61 ms on lock 0x7b5800090098. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69ad8fadb1 00007f69ada3ab71 00007f69ada3aa22 00007f69b6953c09 00007f69b5aadee0 00007f69b5b770ab 00007f69b5b77131 00007f69b5b250a0 00007f69b5b24984 00007f69af09f6d4 00007f69af09f662
0709 13:19:46.782315 (+2419397us) spinlock_profiling.cc:243] Waited 612 ms on lock 0x7b54002ce2d4. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:19:46.782555 (+   240us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1413024512}
I20260709 13:19:46.791994 16265 leader_election.cc:304] T 8d4ab870a05347868956aa8b841b2981 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:19:46.793591 16265 leader_election.cc:304] T a378d7f796f94615b3b347119a9880fd P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:19:46.794869 18388 raft_consensus.cc:2804] T 8d4ab870a05347868956aa8b841b2981 P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Leader election won for term 5
W20260709 13:19:46.795574 16190 connection.cc:256] RPC call timeout handler was delayed by -1.475697301000082s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1519s for a response
I20260709 13:19:46.795584 18388 raft_consensus.cc:697] T 8d4ab870a05347868956aa8b841b2981 P 90f3da57796b488c9087a2f523adf045 [term 5 LEADER]: Becoming Leader. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:19:46.797150 18388 consensus_queue.cc:237] T 8d4ab870a05347868956aa8b841b2981 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:46.799580 18041 raft_consensus.cc:2804] T a378d7f796f94615b3b347119a9880fd P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:19:46.800155 18041 raft_consensus.cc:697] T a378d7f796f94615b3b347119a9880fd P 90f3da57796b488c9087a2f523adf045 [term 4 LEADER]: Becoming Leader. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:19:46.800401 16222 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57604 (request call id 1984) took 1500 ms. Trace:
I20260709 13:19:46.801301 18041 consensus_queue.cc:237] T a378d7f796f94615b3b347119a9880fd P 90f3da57796b488c9087a2f523adf045 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:46.801631 18384 consensus_queue.cc:1048] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:19:46.790957 16295 rpcz_store.cc:276] 0709 13:19:45.395132 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:45.812921 (+417789us) service_pool.cc:224] Handling call
0709 13:19:46.790288 (+977367us) spinlock_profiling.cc:243] Waited 824 ms on lock 0x7b54000be5d4. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:19:46.790453 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1894830336}
I20260709 13:19:46.785562 18313 raft_consensus.cc:2749] T 84f2da98864b4425bf7c0055951fdd4b P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:19:46.806118 18154 consensus_queue.cc:237] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } }
I20260709 13:19:46.807483 18123 leader_election.cc:290] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:19:46.831046 16151 raft_consensus.cc:2468] T b6f07842d3aa47858df5665874c86f3a P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 4.
I20260709 13:19:46.782428 16225 rpcz_store.cc:276] 0709 13:19:45.348653 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:46.780572 (+1431919us) service_pool.cc:224] Handling call
0709 13:19:46.781062 (+   490us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:46.781671 (+   609us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:46.781704 (+    33us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:46.781762 (+    58us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:19:46.781796 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:46.781850 (+    54us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:46.781960 (+   110us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":5034,"raft.run_cpu_time_us":153,"raft.run_wall_time_us":153}
I20260709 13:19:46.838388 16822 raft_consensus.cc:2468] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 4.
W20260709 13:19:46.811110 18114 consensus_meta.cc:287] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 1ca2725d290e47d68b1a4b609c9816ac: Time spent flushing consensus metadata: real 0.768s	user 0.007s	sys 0.007s
I20260709 13:19:46.844383 16191 leader_election.cc:304] T 746aabf856ab401c9ef5e859371e7b67 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
I20260709 13:19:46.800858 16222 rpcz_store.cc:276] 0709 13:19:45.299426 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:46.121676 (+822250us) service_pool.cc:224] Handling call
0709 13:19:46.121943 (+   267us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:46.122198 (+   255us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:19:46.180538 (+ 58340us) log.cc:844] Serialized 14 byte log entry
0709 13:19:46.798815 (+618277us) log.cc:844] Serialized 14 byte log entry
0709 13:19:46.799778 (+   963us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:46.799845 (+    67us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:19:46.799931 (+    86us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:46.800010 (+    79us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:46.800214 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":24400,"prepare.queue_time_us":302463,"prepare.run_cpu_time_us":100,"prepare.run_wall_time_us":101,"raft.queue_time_us":37217,"raft.run_cpu_time_us":354,"raft.run_wall_time_us":21478,"thread_start_us":544248,"threads_started":2,"wal-append.queue_time_us":496375}
I20260709 13:19:46.876874 16225 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33050 (request call id 417) took 1104 ms. Trace:
I20260709 13:19:46.877478 16225 rpcz_store.cc:276] 0709 13:19:45.772618 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:46.837715 (+1065097us) service_pool.cc:224] Handling call
0709 13:19:46.875938 (+ 38223us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:46.876389 (+   451us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:46.876426 (+    37us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:46.876501 (+    75us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:19:46.876542 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:46.876626 (+    84us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:46.876775 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:46.883718 16295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d4ab870a05347868956aa8b841b2981" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:46.885246 16295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdbd78acb4cf46298e0de8ba0d130e19" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:19:46.885830 16295 raft_consensus.cc:3060] T cdbd78acb4cf46298e0de8ba0d130e19 P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:19:46.887682 16292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:19:46.888509 16292 raft_consensus.cc:3060] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:19:46.890626 16192 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 83412.1us
I20260709 13:19:46.891657 16291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a959dbecc2648eb85d1cc550f5a3245" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:19:46.892109 18254 raft_consensus.cc:2749] T 746aabf856ab401c9ef5e859371e7b67 P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:19:46.892225 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 66721us
I20260709 13:19:46.892969 16291 raft_consensus.cc:2410] T 5a959dbecc2648eb85d1cc550f5a3245 P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac for term 3 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:19:46.894310 16291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4dc6b0541ca431ebc757beac00521f1" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:19:46.844489 18114 raft_consensus.cc:515] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:46.895670 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a878999c0024d2abe517484180290fc" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:19:46.897962 18114 leader_election.cc:290] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 7 election: Requested vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
W20260709 13:19:46.899466 16790 leader_election.cc:336] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943): Timed out: RequestConsensusVote RPC to 127.14.177.130:35943 timed out after 1.507s (SENT)
I20260709 13:19:46.899670 16296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8158bbd17b994d2eb96f560bd0f1b88d" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
W20260709 13:19:46.900455 16790 leader_election.cc:336] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Timed out: RequestConsensusVote RPC to 127.14.177.129:36741 timed out after 1.507s (SENT)
I20260709 13:19:46.901002 16296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ad5dbfa99ee4b8d9e28be92b3f36db5" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:19:46.901741 16296 raft_consensus.cc:2393] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 4: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
I20260709 13:19:46.902611 16296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fffab26427e24d398e2695aad40ca134" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:19:46.865053 18323 raft_consensus.cc:493] T e4dc6b0541ca431ebc757beac00521f1 P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:19:46.907994 18323 raft_consensus.cc:515] T e4dc6b0541ca431ebc757beac00521f1 P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:46.901034 16790 leader_election.cc:304] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:19:46.920964 16291 raft_consensus.cc:3060] T e4dc6b0541ca431ebc757beac00521f1 P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:19:46.926280 17844 raft_consensus.cc:515] T af57996a8be84d398f0f023e8d08ef77 P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:46.927462 17992 raft_consensus.cc:493] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:19:46.927762 16225 raft_consensus.cc:1275] T 829b4ad8c5a54eababeea1181b960d4b P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:19:46.928498 16225 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33050 (request call id 418) took 1155 ms. Trace:
I20260709 13:19:46.928160 17992 raft_consensus.cc:515] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:46.928881 17844 leader_election.cc:290] T af57996a8be84d398f0f023e8d08ef77 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 5 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:19:46.921571 16220 raft_consensus.cc:2468] T b6f07842d3aa47858df5665874c86f3a P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 4.
I20260709 13:19:46.930599 17992 leader_election.cc:290] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:46.931078 16220 raft_consensus.cc:1240] T fffab26427e24d398e2695aad40ca134 P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 13:19:46.932055 16220 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57600 (request call id 1051) took 1106 ms. Trace:
I20260709 13:19:46.932456 16220 rpcz_store.cc:276] 0709 13:19:45.825452 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:46.930227 (+1104775us) service_pool.cc:224] Handling call
0709 13:19:46.930710 (+   483us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:46.931722 (+  1012us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:46.931990 (+   268us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:46.933027 16266 leader_election.cc:304] T ada5a004ba7c46819051e3ccec71ffdb P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860
I20260709 13:19:46.935878 16220 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57604 (request call id 1996) took 1079 ms. Trace:
I20260709 13:19:46.937059 16819 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49896 (request call id 821) took 1949 ms. Trace:
I20260709 13:19:46.936436 16220 rpcz_store.cc:276] 0709 13:19:45.856766 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:46.933489 (+1076723us) service_pool.cc:224] Handling call
0709 13:19:46.934054 (+   565us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:46.935099 (+  1045us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:46.935140 (+    41us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:46.935500 (+   360us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:19:46.935564 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:46.935642 (+    78us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:46.935787 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":176,"raft.run_cpu_time_us":130,"raft.run_wall_time_us":131}
I20260709 13:19:46.937493 16819 rpcz_store.cc:276] 0709 13:19:44.987614 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:45.318875 (+331261us) service_pool.cc:224] Handling call
0709 13:19:45.445797 (+126922us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:45.774982 (+329185us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:46.936678 (+1161696us) spinlock_profiling.cc:243] Waited 124 ms on lock 0x7b5400625550. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5e8ad 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:19:46.936871 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":285087488}
I20260709 13:19:46.938473 18390 raft_consensus.cc:2749] T ada5a004ba7c46819051e3ccec71ffdb P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:19:46.940651 18156 raft_consensus.cc:3055] T 9e65676b21b748728ad39668cfe722ec P 90f3da57796b488c9087a2f523adf045 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:19:46.941171 18156 raft_consensus.cc:740] T 9e65676b21b748728ad39668cfe722ec P 90f3da57796b488c9087a2f523adf045 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
W20260709 13:19:46.941951 18109 consensus_meta.cc:287] T e5ad4eb281564936a59f41e387a47156 P 1ca2725d290e47d68b1a4b609c9816ac: Time spent flushing consensus metadata: real 0.784s	user 0.004s	sys 0.009s
I20260709 13:19:46.942416 18109 raft_consensus.cc:515] T e5ad4eb281564936a59f41e387a47156 P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:46.945369 16220 raft_consensus.cc:1240] T c2f5f89186204242bf377f7c42b2e2da P 1ca2725d290e47d68b1a4b609c9816ac [term 6 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 5. Current term is 6. Ops: [5.4-5.4]
I20260709 13:19:46.947738 16296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87ac0d1f7e8a419f9bcbc04aee0e067e" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:46.951042 16265 leader_election.cc:400] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 4 election: Vote denied by peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741) with higher term. Message: Invalid argument: T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 for earlier term 4. Current term is 5.
I20260709 13:19:46.951449 16265 leader_election.cc:403] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 4 election: Cancelling election due to peer responding with higher term
I20260709 13:19:46.952064 16296 raft_consensus.cc:2468] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 4.
I20260709 13:19:46.953159 16296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:19:46.953980 16296 raft_consensus.cc:2393] T ecc1e363230043fe9fef390f0b979f20 P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 4: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
I20260709 13:19:46.954732 16296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18ad7e3af5504571a69d10c00750a01c" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:19:46.910497 18323 leader_election.cc:290] T e4dc6b0541ca431ebc757beac00521f1 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:19:46.955796 16265 leader_election.cc:304] T 02838c86531d4f98bf58d1c22b8baf77 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac
I20260709 13:19:46.956321 18347 raft_consensus.cc:2749] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Leader election lost for term 4. Reason: Vote denied by peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741) with higher term. Message: Invalid argument: T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 for earlier term 4. Current term is 5.
I20260709 13:19:46.956516 16819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7ca61592e848ea9b0fb81a2ce33cdc" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:19:46.956746 16822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af57996a8be84d398f0f023e8d08ef77" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
W20260709 13:19:46.958573 18366 log.cc:927] Time spent T e5ad4eb281564936a59f41e387a47156 P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.711s	user 0.012s	sys 0.020s
I20260709 13:19:46.959075 16117 leader_election.cc:304] T af57996a8be84d398f0f023e8d08ef77 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860
I20260709 13:19:46.928915 16225 rpcz_store.cc:276] 0709 13:19:45.772879 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:46.878739 (+1105860us) service_pool.cc:224] Handling call
0709 13:19:46.927297 (+ 48558us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:46.928225 (+   928us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:46.928440 (+   215us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:46.931013 16221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c2095a5f4ad49408dbc7f69c606120d" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
W20260709 13:19:46.960173 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 59768.5us
I20260709 13:19:46.962013 16217 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57604 (request call id 1969) took 3947 ms. Trace:
I20260709 13:19:46.963924 16292 raft_consensus.cc:2468] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 5.
I20260709 13:19:46.942487 18156 consensus_queue.cc:260] T 9e65676b21b748728ad39668cfe722ec P 90f3da57796b488c9087a2f523adf045 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.7, Last appended by leader: 7, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:46.964154 16222 raft_consensus.cc:1275] T 84f2da98864b4425bf7c0055951fdd4b P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:19:46.934844 16266 leader_election.cc:304] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860
I20260709 13:19:46.972128 16295 raft_consensus.cc:2468] T cdbd78acb4cf46298e0de8ba0d130e19 P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 5.
W20260709 13:19:46.999281 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 201534us
I20260709 13:19:47.003548 16222 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57604 (request call id 1994) took 1219 ms. Trace:
I20260709 13:19:47.005527 16222 rpcz_store.cc:276] 0709 13:19:45.783796 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:46.921326 (+1137530us) service_pool.cc:224] Handling call
0709 13:19:46.963692 (+ 42366us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:47.003248 (+ 39556us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:47.003463 (+   215us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:47.012415 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdbd78acb4cf46298e0de8ba0d130e19" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:47.011180 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ad5dbfa99ee4b8d9e28be92b3f36db5" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:19:47.022639 16296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74ca8140f35c4ae29e8a664f6bdfc829" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:47.027298 16222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0eab25ce4c64e018a041beeda2867db" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
W20260709 13:19:47.032620 16790 leader_election.cc:336] T 23f82224f4564fea93572e8206bad1fe P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Timed out: RequestConsensusVote RPC to 127.14.177.129:36741 timed out after 1.630s (SENT)
I20260709 13:19:47.035137 16226 raft_consensus.cc:1240] T e5ad4eb281564936a59f41e387a47156 P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 4. Current term is 5. Ops: []
W20260709 13:19:47.035378 16117 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 76490us
I20260709 13:19:47.036280 16790 leader_election.cc:304] T 23f82224f4564fea93572e8206bad1fe P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:19:47.037582 16224 raft_consensus.cc:2393] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 4: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
I20260709 13:19:47.037958 18372 raft_consensus.cc:493] T 5a96373bb0e14dce8a266b2fc6c58507 P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:47.038403 16224 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8158bbd17b994d2eb96f560bd0f1b88d"
dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
mode: GRACEFUL
new_leader_uuid: "90f3da57796b488c9087a2f523adf045"
 from {username='slave'} at 127.0.0.1:39144
I20260709 13:19:47.038774 16143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ad5dbfa99ee4b8d9e28be92b3f36db5" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:19:47.039214 16224 raft_consensus.cc:606] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Received request to transfer leadership to TS 90f3da57796b488c9087a2f523adf045
I20260709 13:19:47.039390 18347 raft_consensus.cc:2804] T 02838c86531d4f98bf58d1c22b8baf77 P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:19:47.039717 16224 raft_consensus.cc:614] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:19:47.039812 16143 raft_consensus.cc:2393] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 4: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
I20260709 13:19:47.041170 16143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87147731a68400e8cce9b879daa7c5d" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:19:47.041095 18104 consensus_queue.cc:1048] T 1df93779209748b9860f6d893dd3de53 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:19:47.025225 18140 raft_consensus.cc:2749] T af57996a8be84d398f0f023e8d08ef77 P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:19:47.044360 16266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 109665us
I20260709 13:19:47.046223 16144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6f07842d3aa47858df5665874c86f3a" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:19:46.998315 16217 rpcz_store.cc:276] 0709 13:19:43.014513 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:44.325945 (+1311432us) service_pool.cc:224] Handling call
0709 13:19:44.345676 (+ 19731us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:45.794029 (+1448353us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:46.960325 (+1166296us) spinlock_profiling.cc:243] Waited 22.6 ms on lock 0x7b5400375b20. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0ad6dc0 00007f69b0b62a02 00007f69b0b5edd6 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2
0709 13:19:46.961504 (+  1179us) spinlock_profiling.cc:243] Waited 1 s on lock 0x7b54002c9054. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:19:46.961801 (+   297us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2357780096}
I20260709 13:19:47.046828 16144 raft_consensus.cc:2376] T b6f07842d3aa47858df5665874c86f3a P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 4. Re-sending same reply.
I20260709 13:19:47.049383 16225 raft_consensus.cc:2393] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 5: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
I20260709 13:19:47.050844 16151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d16dfad0c3b540e5b4b86492ddf87208" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
W20260709 13:19:47.034204 16221 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57604 (request call id 1990) took 1728 ms (client timeout 2248 ms). Trace:
I20260709 13:19:47.038671 18372 raft_consensus.cc:515] T 5a96373bb0e14dce8a266b2fc6c58507 P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:47.039929 18347 raft_consensus.cc:493] T 02838c86531d4f98bf58d1c22b8baf77 P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:19:47.057608 18374 raft_consensus.cc:2749] T 23f82224f4564fea93572e8206bad1fe P cb82cc7bd6db46d980c75237a1402860 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:19:47.057659 16293 raft_consensus.cc:2468] T 8a878999c0024d2abe517484180290fc P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 4.
I20260709 13:19:47.061312 16291 raft_consensus.cc:2468] T e4dc6b0541ca431ebc757beac00521f1 P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 4.
I20260709 13:19:47.062175 16821 raft_consensus.cc:1275] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [term 6 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:19:47.044055 18388 raft_consensus.cc:2749] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:19:47.068677 16144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ad5dbfa99ee4b8d9e28be92b3f36db5" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:19:47.062633 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3884f864a96d428785895e5463e5728f" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:19:47.062891 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b7d1ab519044eda2d5e04a07242f25" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:47.063292 18145 consensus_queue.cc:1059] T 8a878999c0024d2abe517484180290fc P 647a343a542d4a139fe106c19095129b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.009s
I20260709 13:19:47.056728 16296 raft_consensus.cc:2468] T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 4.
I20260709 13:19:47.061357 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87ac0d1f7e8a419f9bcbc04aee0e067e" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:19:47.071044 16144 raft_consensus.cc:2393] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 4: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
I20260709 13:19:47.055462 17992 raft_consensus.cc:2749] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:19:47.073616 18156 raft_consensus.cc:3060] T 9e65676b21b748728ad39668cfe722ec P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:19:47.073777 18145 raft_consensus.cc:3055] T 8a878999c0024d2abe517484180290fc P 647a343a542d4a139fe106c19095129b [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:19:47.074030 16299 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48674 (request call id 2073) took 2243 ms. Trace:
I20260709 13:19:47.074308 18145 raft_consensus.cc:740] T 8a878999c0024d2abe517484180290fc P 647a343a542d4a139fe106c19095129b [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:19:47.075327 16292 raft_consensus.cc:1275] T 7c2095a5f4ad49408dbc7f69c606120d P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 2. (term mismatch)
I20260709 13:19:47.075722 16292 pending_rounds.cc:85] T 7c2095a5f4ad49408dbc7f69c606120d P 90f3da57796b488c9087a2f523adf045: Aborting all ops after (but not including) 1
I20260709 13:19:47.074469 16299 rpcz_store.cc:276] 0709 13:19:44.830999 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:45.361560 (+530561us) service_pool.cc:224] Handling call
0709 13:19:45.361966 (+   406us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:45.391857 (+ 29891us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:45.391902 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:45.740182 (+348280us) log.cc:844] Serialized 26 byte log entry
0709 13:19:46.083391 (+343209us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:19:46.083442 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:46.083507 (+    65us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:46.957650 (+874143us) spinlock_profiling.cc:243] Waited 26 ms on lock 0x7b240024bac8. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69ad8fadb1 00007f69ad8f89c4 00007f69ad8f6acb 00007f69adab1dc9 00007f69adab1a55 00007f69adab109d 00007f69b06e310e 00007f69b06fa88c 00007f69b06f9bb6 00007f69b06f97d8 00007f69b0703221
0709 13:19:47.067845 (+110195us) raft_consensus.cc:1685] finished
0709 13:19:47.067867 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:47.073873 (+  6006us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":29405,"raft.queue_time_us":29797,"raft.run_cpu_time_us":144,"raft.run_wall_time_us":9440,"spinlock_wait_cycles":59741952,"thread_start_us":291929,"threads_started":1,"wal-append.queue_time_us":378536}
I20260709 13:19:47.075994 16292 pending_rounds.cc:107] T 7c2095a5f4ad49408dbc7f69c606120d P 90f3da57796b488c9087a2f523adf045: Aborting uncommitted NO_OP operation due to leader change: 2.2
I20260709 13:19:47.076377 16292 raft_consensus.cc:2889] T 7c2095a5f4ad49408dbc7f69c606120d P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:19:47.078020 16224 raft_consensus.cc:1240] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 3. Current term is 4. Ops: []
I20260709 13:19:47.079511 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66ae176961b74813a58bc15c9c4c6f2a" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:47.081313 16224 raft_consensus.cc:1275] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
W20260709 13:19:47.081957 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 126306us
I20260709 13:19:47.085187 16265 leader_election.cc:304] T 7c2095a5f4ad49408dbc7f69c606120d P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac
I20260709 13:19:47.085676 17999 raft_consensus.cc:493] T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:19:47.086228 17999 raft_consensus.cc:3060] T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:19:47.092217 16266 leader_election.cc:304] T 23f82224f4564fea93572e8206bad1fe P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 647a343a542d4a139fe106c19095129b
I20260709 13:19:47.092981 18109 leader_election.cc:290] T e5ad4eb281564936a59f41e387a47156 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 5 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:47.094594 16296 raft_consensus.cc:1275] T d2ab0ac8347641e6955e9bce7d90c1a2 P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
W20260709 13:19:47.095054 16115 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:47.095667 16151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d16dfad0c3b540e5b4b86492ddf87208" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:19:47.096166 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0eab25ce4c64e018a041beeda2867db" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:19:47.095326 16224 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8a878999c0024d2abe517484180290fc"
dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
mode: GRACEFUL
new_leader_uuid: "90f3da57796b488c9087a2f523adf045"
 from {username='slave'} at 127.0.0.1:39144
W20260709 13:19:47.054412 16221 rpcz_store.cc:269] 0709 13:19:45.300932 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:46.930431 (+1629499us) service_pool.cc:224] Handling call
0709 13:19:47.029762 (+ 99331us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:47.098906 16149 raft_consensus.cc:2410] T c0eab25ce4c64e018a041beeda2867db P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 for term 5 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:19:47.065622 18347 raft_consensus.cc:3060] T 02838c86531d4f98bf58d1c22b8baf77 P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:19:47.099066 18323 raft_consensus.cc:2804] T 7c2095a5f4ad49408dbc7f69c606120d P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 13:19:47.096983 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 64434.6us
I20260709 13:19:47.103379 18140 consensus_queue.cc:1059] T 18ad7e3af5504571a69d10c00750a01c P 647a343a542d4a139fe106c19095129b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.003s
I20260709 13:19:47.110498 18041 raft_consensus.cc:2804] T 23f82224f4564fea93572e8206bad1fe P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:19:47.111050 18041 raft_consensus.cc:493] T 23f82224f4564fea93572e8206bad1fe P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:19:47.111790 18041 raft_consensus.cc:3060] T 23f82224f4564fea93572e8206bad1fe P 90f3da57796b488c9087a2f523adf045 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:19:47.115231 16222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c11074579b046d99e585c2818e18a2c" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:47.115976 16266 leader_election.cc:304] T 31969392ccdb43b491911a319e8bc7e1 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
I20260709 13:19:47.105443 16224 raft_consensus.cc:606] T 8a878999c0024d2abe517484180290fc P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Received request to transfer leadership to TS 90f3da57796b488c9087a2f523adf045
I20260709 13:19:47.104044 16143 raft_consensus.cc:3060] T b87147731a68400e8cce9b879daa7c5d P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:19:47.127259 16224 raft_consensus.cc:614] T 8a878999c0024d2abe517484180290fc P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:19:47.107272 18323 raft_consensus.cc:493] T 7c2095a5f4ad49408dbc7f69c606120d P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:47.137712 16821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57a744f5aaf944678a63469cbf514015" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:19:47.126144 18154 raft_consensus.cc:3060] T 31969392ccdb43b491911a319e8bc7e1 P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:19:47.143070 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 101315us
W20260709 13:19:47.147480 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 62414.6us
W20260709 13:19:47.152405 16117 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 79420.1us
I20260709 13:19:47.141960 18323 raft_consensus.cc:3060] T 7c2095a5f4ad49408dbc7f69c606120d P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:19:47.148243 16821 raft_consensus.cc:2468] T 57a744f5aaf944678a63469cbf514015 P cb82cc7bd6db46d980c75237a1402860 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 3.
W20260709 13:19:47.154439 16114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:47.160799 16822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e4bfb9090c8460eb14fa7e0d818e7f1" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:19:47.162329 16265 leader_election.cc:304] T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:19:47.166431 16298 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48684 (request call id 1916) took 3990 ms. Trace:
I20260709 13:19:47.166904 16298 rpcz_store.cc:276] 0709 13:19:43.175751 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:45.282959 (+2107208us) service_pool.cc:224] Handling call
0709 13:19:45.283271 (+   312us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:45.812016 (+528745us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:47.165968 (+1353952us) spinlock_profiling.cc:243] Waited 515 ms on lock 0x7b54002e0754. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:19:47.166258 (+   290us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1185258880}
I20260709 13:19:47.167454 16224 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20"
dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
mode: GRACEFUL
new_leader_uuid: "90f3da57796b488c9087a2f523adf045"
 from {username='slave'} at 127.0.0.1:39144
I20260709 13:19:47.168177 16224 raft_consensus.cc:606] T ecc1e363230043fe9fef390f0b979f20 P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Received request to transfer leadership to TS 90f3da57796b488c9087a2f523adf045
I20260709 13:19:47.168573 16224 raft_consensus.cc:614] T ecc1e363230043fe9fef390f0b979f20 P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:19:47.169324 16223 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33050 (request call id 416) took 1397 ms. Trace:
I20260709 13:19:47.169687 16223 rpcz_store.cc:276] 0709 13:19:45.772330 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:46.824157 (+1051827us) service_pool.cc:224] Handling call
0709 13:19:46.824778 (+   621us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:46.832670 (+  7892us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:46.832700 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:46.855763 (+ 23063us) log.cc:844] Serialized 26 byte log entry
0709 13:19:47.013049 (+157286us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:19:47.013287 (+   238us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:47.013538 (+   251us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:47.169126 (+155588us) raft_consensus.cc:1685] finished
0709 13:19:47.169143 (+    17us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:47.169255 (+   112us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":105,"raft.run_cpu_time_us":162,"raft.run_wall_time_us":5860,"thread_start_us":136272,"threads_started":1,"wal-append.queue_time_us":250110}
I20260709 13:19:47.170553 16221 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.BulkChangeConfig from 127.0.0.1:33034 (request call id 13) took 3395 ms. Trace:
I20260709 13:19:47.178325 16221 rpcz_store.cc:276] 0709 13:19:43.775593 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:47.106046 (+3330453us) service_pool.cc:224] Handling call
0709 13:19:47.170467 (+ 64421us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:47.178565 16819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c11074579b046d99e585c2818e18a2c" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:19:47.179011 18346 raft_consensus.cc:3055] T 8158bbd17b994d2eb96f560bd0f1b88d P 90f3da57796b488c9087a2f523adf045 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:19:47.179461 18346 raft_consensus.cc:740] T 8158bbd17b994d2eb96f560bd0f1b88d P 90f3da57796b488c9087a2f523adf045 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:19:47.179800 16819 raft_consensus.cc:2410] T 8c11074579b046d99e585c2818e18a2c P cb82cc7bd6db46d980c75237a1402860 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 for term 6 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 6.
I20260709 13:19:47.180580 15046 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:19:47.180307 18346 consensus_queue.cc:260] T 8158bbd17b994d2eb96f560bd0f1b88d P 90f3da57796b488c9087a2f523adf045 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:47.182477 18276 raft_consensus.cc:493] T 68a6838d82ef4cac83507f906139d90f P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:19:47.183089 18276 raft_consensus.cc:3060] T 68a6838d82ef4cac83507f906139d90f P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:19:47.184163 16816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434bff3082d141838068f323bdb0d1ed" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:19:47.185068 16818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a9de347c711477e87c582d003f36925" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
W20260709 13:19:47.188639 16146 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40830 (request call id 1048) took 3351 ms (client timeout 2189 ms). Trace:
W20260709 13:19:47.189193 16142 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40844 (request call id 1818) took 2236 ms (client timeout 1718 ms). Trace:
I20260709 13:19:47.190086 18123 raft_consensus.cc:2804] T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:19:47.192507 16821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e9b2bf7011f4ae688a79d7bb5a447b4" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:19:47.203267 18372 leader_election.cc:290] T 5a96373bb0e14dce8a266b2fc6c58507 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:47.209795 16824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87147731a68400e8cce9b879daa7c5d" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:19:47.210064 16218 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57604 (request call id 1993) took 1880 ms. Trace:
I20260709 13:19:47.211663 16817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c11074579b046d99e585c2818e18a2c" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
W20260709 13:19:47.213409 16219 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57604 (request call id 1987) took 1913 ms (client timeout 1881 ms). Trace:
I20260709 13:19:47.214592 18387 consensus_queue.cc:1048] T 829b4ad8c5a54eababeea1181b960d4b P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:19:47.170243 18041 raft_consensus.cc:515] T 23f82224f4564fea93572e8206bad1fe P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:19:47.215891 18413 log.cc:927] Time spent T 31969392ccdb43b491911a319e8bc7e1 P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.158s	user 0.004s	sys 0.000s
W20260709 13:19:47.193003 16146 rpcz_store.cc:269] 0709 13:19:43.837226 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:44.103899 (+266673us) service_pool.cc:224] Handling call
0709 13:19:47.188192 (+3084293us) spinlock_profiling.cc:243] Waited 2.74 s on lock 0x7b540013f1d4. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:19:47.188382 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6307877632}
I20260709 13:19:47.217352 16821 raft_consensus.cc:2468] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 2.
W20260709 13:19:47.232806 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 78184.1us
I20260709 13:19:47.234563 16822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87147731a68400e8cce9b879daa7c5d" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:19:47.236408 16223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87ac0d1f7e8a419f9bcbc04aee0e067e" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:47.236799 16222 raft_consensus.cc:1240] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 4. Current term is 5. Ops: [4.4-4.4]
I20260709 13:19:47.237094 16223 raft_consensus.cc:2468] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 4.
I20260709 13:19:47.237090 16217 raft_consensus.cc:1240] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 6. Current term is 7. Ops: []
I20260709 13:19:47.239095 16821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66f75124215f4ca6bef831e174a792d8" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
W20260709 13:19:47.240132 16117 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 63941us
I20260709 13:19:47.248068 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4dc6b0541ca431ebc757beac00521f1" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:47.248768 16824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ab0ac8347641e6955e9bce7d90c1a2" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
W20260709 13:19:47.250227 16265 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:47.252168 16822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a9de347c711477e87c582d003f36925" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
W20260709 13:19:47.253633 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 231898us
W20260709 13:19:47.253979 16189 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:47.255904 16220 raft_consensus.cc:1240] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 3. Current term is 4. Ops: []
I20260709 13:19:47.231083 18123 raft_consensus.cc:697] T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045 [term 4 LEADER]: Becoming Leader. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:19:47.245110 16818 raft_consensus.cc:2368] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 for earlier term 4. Current term is 5.
I20260709 13:19:47.249440 16824 raft_consensus.cc:2468] T d2ab0ac8347641e6955e9bce7d90c1a2 P cb82cc7bd6db46d980c75237a1402860 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 5.
I20260709 13:19:47.215042 18401 consensus_queue.cc:1048] T c9944d5e4b8c4109af9955c4f08c9095 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.030s
I20260709 13:19:47.234104 16816 raft_consensus.cc:2468] T 434bff3082d141838068f323bdb0d1ed P cb82cc7bd6db46d980c75237a1402860 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 4.
I20260709 13:19:47.263564 16817 raft_consensus.cc:2410] T 8c11074579b046d99e585c2818e18a2c P cb82cc7bd6db46d980c75237a1402860 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 for term 6 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 6.
W20260709 13:19:47.216389 16142 rpcz_store.cc:269] 0709 13:19:44.952711 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:45.151284 (+198573us) service_pool.cc:224] Handling call
0709 13:19:47.188808 (+2037524us) spinlock_profiling.cc:243] Waited 602 ms on lock 0x7b54002d0d54. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:19:47.188998 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1385273728}
I20260709 13:19:47.275130 16824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62fe3346cd3c42c4a55e14af9073bca4" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:19:47.279944 16265 leader_election.cc:304] T d16dfad0c3b540e5b4b86492ddf87208 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:19:47.280270 16821 raft_consensus.cc:2393] T 66f75124215f4ca6bef831e174a792d8 P cb82cc7bd6db46d980c75237a1402860 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 5: Already voted for candidate cb82cc7bd6db46d980c75237a1402860 in this term.
I20260709 13:19:47.279975 16224 raft_consensus.cc:2393] T e4dc6b0541ca431ebc757beac00521f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 4: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
W20260709 13:19:47.281016 16114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:47.282224 16296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a96373bb0e14dce8a266b2fc6c58507" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:47.284482 16824 raft_consensus.cc:2393] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 6: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
I20260709 13:19:47.271603 16221 raft_consensus.cc:1240] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 4. Current term is 5. Ops: []
W20260709 13:19:47.228987 16219 rpcz_store.cc:269] 0709 13:19:45.300204 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:45.951915 (+651711us) service_pool.cc:224] Handling call
0709 13:19:47.213107 (+1261192us) spinlock_profiling.cc:243] Waited 586 ms on lock 0x7b54002b4b54. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:19:47.213296 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1347087616}
I20260709 13:19:47.266114 16819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3884f864a96d428785895e5463e5728f" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:19:47.287690 16296 raft_consensus.cc:2410] T 5a96373bb0e14dce8a266b2fc6c58507 P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac for term 4 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:19:47.271226 18156 raft_consensus.cc:493] T 5a959dbecc2648eb85d1cc550f5a3245 P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:47.295774 16822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66ae176961b74813a58bc15c9c4c6f2a" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:19:47.296528 16822 raft_consensus.cc:2368] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 for earlier term 5. Current term is 6.
I20260709 13:19:47.292593 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7ca61592e848ea9b0fb81a2ce33cdc" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
W20260709 13:19:47.298071 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 62037.4us
I20260709 13:19:47.274323 18254 raft_consensus.cc:493] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:47.279397 16816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8638ee363273426db1393dd81e5c7b72" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:19:47.299906 16117 leader_election.cc:304] T 26354361e9b044cea1c2b86c4882fe92 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860
I20260709 13:19:47.300915 18181 raft_consensus.cc:3055] T 18ad7e3af5504571a69d10c00750a01c P 647a343a542d4a139fe106c19095129b [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:19:47.300993 18105 consensus_queue.cc:1048] T 57a744f5aaf944678a63469cbf514015 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.006s
I20260709 13:19:47.301203 18402 raft_consensus.cc:493] T c9944d5e4b8c4109af9955c4f08c9095 P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:47.301458 18109 raft_consensus.cc:493] T c2f5f89186204242bf377f7c42b2e2da P 1ca2725d290e47d68b1a4b609c9816ac [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:47.300923 18352 raft_consensus.cc:493] T 8cc959fb7f704fba9009c0595467d074 P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:47.302296 18123 consensus_queue.cc:237] T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:47.303252 18418 consensus_queue.cc:1048] T 3884f864a96d428785895e5463e5728f P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.004s
I20260709 13:19:47.301805 16298 raft_consensus.cc:3060] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:19:47.306353 16790 leader_election.cc:336] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Timed out: RequestConsensusVote RPC to 127.14.177.129:36741 timed out after 1.686s (SENT)
I20260709 13:19:47.305792 16821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a9de347c711477e87c582d003f36925" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:19:47.318038 16790 leader_election.cc:304] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:19:47.314170 16824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "829b4ad8c5a54eababeea1181b960d4b" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:19:47.307508 16823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a378d7f796f94615b3b347119a9880fd" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:19:47.320923 16821 raft_consensus.cc:2368] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 for earlier term 4. Current term is 5.
I20260709 13:19:47.308135 18181 raft_consensus.cc:740] T 18ad7e3af5504571a69d10c00750a01c P 647a343a542d4a139fe106c19095129b [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:19:47.327493 18386 consensus_queue.cc:1059] T fffab26427e24d398e2695aad40ca134 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:19:47.327524 16819 raft_consensus.cc:2468] T 3884f864a96d428785895e5463e5728f P cb82cc7bd6db46d980c75237a1402860 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 3.
I20260709 13:19:47.331669 16297 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48674 (request call id 2076) took 2202 ms. Trace:
I20260709 13:19:47.322696 18400 raft_consensus.cc:2749] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:19:47.225229 16218 rpcz_store.cc:276] 0709 13:19:45.329990 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:46.633822 (+1303832us) service_pool.cc:224] Handling call
0709 13:19:46.634506 (+   684us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:46.635046 (+   540us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:19:46.709888 (+ 74842us) log.cc:844] Serialized 14 byte log entry
0709 13:19:46.910470 (+200582us) spinlock_profiling.cc:243] Waited 29.9 ms on lock 0x7b1800445a80. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b693ade1 00007f69b69475b1 00007f69ada723f3 00007f69ada747c1 00007f69ada74779 00007f69ada74709 00007f69ada746d1 00007f69ada739cd 000000000038f354 000000000038f199 00007f69ada5583a
0709 13:19:47.016282 (+105812us) log.cc:844] Serialized 14 byte log entry
0709 13:19:47.209448 (+193166us) spinlock_profiling.cc:243] Waited 3.32 ms on lock 0x7b1800445a80. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b693ade1 00007f69b69475b1 00007f69ad9ecc0a 00007f69b712c219 00007f69ada74a40 00007f69ada6f4e4 00007f69ada6f455 00007f69b0b57060 00007f69b0b69716 00007f69b0b7a287 00007f69b0b7a208
0709 13:19:47.209610 (+   162us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:47.209665 (+    55us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:19:47.209722 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:47.209806 (+    84us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:47.209921 (+   115us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":209159,"prepare.queue_time_us":19723,"prepare.run_cpu_time_us":115,"prepare.run_wall_time_us":3325,"raft.queue_time_us":227342,"raft.run_cpu_time_us":390,"raft.run_wall_time_us":27920,"spinlock_wait_cycles":76436096,"thread_start_us":229248,"threads_started":1,"wal-append.queue_time_us":339010}
I20260709 13:19:47.311879 18240 consensus_queue.cc:1059] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.026s
I20260709 13:19:47.300158 18156 raft_consensus.cc:515] T 5a959dbecc2648eb85d1cc550f5a3245 P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:47.336745 16266 leader_election.cc:304] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 647a343a542d4a139fe106c19095129b
I20260709 13:19:47.338346 16143 raft_consensus.cc:2468] T b87147731a68400e8cce9b879daa7c5d P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 6.
W20260709 13:19:47.340590 16791 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:47.321527 18390 raft_consensus.cc:2804] T d16dfad0c3b540e5b4b86492ddf87208 P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:19:47.343287 16294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5ad4eb281564936a59f41e387a47156" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:19:47.380224 15046 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:19:47.313618 16818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdbd78acb4cf46298e0de8ba0d130e19" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:19:47.352290 18140 consensus_queue.cc:1059] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:19:47.354465 18154 raft_consensus.cc:2749] T 31969392ccdb43b491911a319e8bc7e1 P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:19:47.354612 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 48276.6us
I20260709 13:19:47.317358 18347 raft_consensus.cc:515] T 02838c86531d4f98bf58d1c22b8baf77 P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:47.323375 18109 raft_consensus.cc:515] T c2f5f89186204242bf377f7c42b2e2da P 1ca2725d290e47d68b1a4b609c9816ac [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:47.400743 18424 raft_consensus.cc:2749] T 26354361e9b044cea1c2b86c4882fe92 P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:19:47.360020 16097 auto_rebalancer.cc:893] Removing replace marker failed: Replica 1ca2725d290e47d68b1a4b609c9816ac is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 4 committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } } }
I20260709 13:19:47.376523 18389 raft_consensus.cc:493] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:47.293567 18421 consensus_queue.cc:1048] T 84f2da98864b4425bf7c0055951fdd4b P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.076s
I20260709 13:19:47.361388 18255 consensus_queue.cc:1048] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:19:47.356244 16821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e4bfb9090c8460eb14fa7e0d818e7f1" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:19:47.354894 16226 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57604 (request call id 1986) took 2055 ms. Trace:
I20260709 13:19:47.336342 18145 consensus_queue.cc:260] T 8a878999c0024d2abe517484180290fc P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:47.315773 18254 raft_consensus.cc:515] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:47.371774 17999 raft_consensus.cc:515] T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:47.374817 16265 leader_election.cc:304] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
W20260709 13:19:47.375687 16117 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 79222.7us
I20260709 13:19:47.378638 16190 leader_election.cc:304] T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:19:47.388797 16298 raft_consensus.cc:2468] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 7.
I20260709 13:19:47.356357 16293 raft_consensus.cc:1275] T c0eab25ce4c64e018a041beeda2867db P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:19:47.364777 16297 rpcz_store.cc:276] 0709 13:19:45.129589 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:46.175908 (+1046319us) service_pool.cc:224] Handling call
0709 13:19:46.221364 (+ 45456us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:46.824114 (+602750us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:46.824177 (+    63us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:46.824828 (+   651us) log.cc:844] Serialized 26 byte log entry
0709 13:19:47.015964 (+191136us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:19:47.016072 (+   108us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:47.023905 (+  7833us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:47.198802 (+174897us) spinlock_profiling.cc:243] Waited 7.21 ms on lock 0x7b4400022b4c. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69ad8fadb1 00007f69ad998c85 00007f69ad9996a2 00007f69b070b1db 00007f69b0706aaf 00007f69b06fa86e 00007f69b06f9bb6 00007f69b06f97d8 00007f69b0703221 00007f69b07031d9 00007f69b0703169
0709 13:19:47.318209 (+119407us) raft_consensus.cc:1685] finished
0709 13:19:47.318228 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:47.331488 (+ 13260us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6073,"raft.queue_time_us":171917,"raft.run_cpu_time_us":63,"raft.run_wall_time_us":64,"spinlock_wait_cycles":16583936,"thread_start_us":672896,"threads_started":2,"wal-append.queue_time_us":231925}
W20260709 13:19:47.405133 16266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 68474.4us
I20260709 13:19:47.372287 18041 leader_election.cc:290] T 23f82224f4564fea93572e8206bad1fe P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 2 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:19:47.320035 18352 raft_consensus.cc:515] T 8cc959fb7f704fba9009c0595467d074 P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:47.419605 18352 leader_election.cc:290] T 8cc959fb7f704fba9009c0595467d074 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:19:47.316344 18402 raft_consensus.cc:515] T c9944d5e4b8c4109af9955c4f08c9095 P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:47.407222 16226 rpcz_store.cc:276] 0709 13:19:45.299957 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:46.167564 (+867607us) service_pool.cc:224] Handling call
0709 13:19:46.180817 (+ 13253us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:47.043635 (+862818us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:47.354622 (+310987us) spinlock_profiling.cc:243] Waited 824 ms on lock 0x7b54003d2a54. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:19:47.354824 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1894832000}
I20260709 13:19:47.377516 18318 leader_election.cc:290] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
W20260709 13:19:47.409890 16115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 83419.6us
I20260709 13:19:47.421761 18105 raft_consensus.cc:3055] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:19:47.422276 18105 raft_consensus.cc:740] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:19:47.424584 18406 raft_consensus.cc:493] T c0eab25ce4c64e018a041beeda2867db P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:47.425180 18406 raft_consensus.cc:515] T c0eab25ce4c64e018a041beeda2867db P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:47.425568 16115 leader_election.cc:304] T 8158bbd17b994d2eb96f560bd0f1b88d P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:19:47.425475 18422 consensus_queue.cc:1048] T 3884f864a96d428785895e5463e5728f P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:19:47.426069 16147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a9de347c711477e87c582d003f36925" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:19:47.412269 18347 leader_election.cc:290] T 02838c86531d4f98bf58d1c22b8baf77 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 5 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:19:47.427042 16147 raft_consensus.cc:2393] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 in current term 5: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
I20260709 13:19:47.427304 18140 raft_consensus.cc:3060] T 8158bbd17b994d2eb96f560bd0f1b88d P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:19:47.428125 16144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e9b2bf7011f4ae688a79d7bb5a447b4" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:19:47.429488 16146 raft_consensus.cc:1275] T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:19:47.429744 16142 raft_consensus.cc:1240] T b87147731a68400e8cce9b879daa7c5d P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 5. Current term is 6. Ops: []
W20260709 13:19:47.412487 16097 auto_rebalancer.cc:901] Could not move replica: Illegal state: not currently leader
W20260709 13:19:47.413019 16790 leader_election.cc:336] T 9e65676b21b748728ad39668cfe722ec P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Timed out: RequestConsensusVote RPC to 127.14.177.129:36741 timed out after 1.691s (SENT)
W20260709 13:19:47.430596 16097 auto_rebalancer.cc:330] scheduled replica move failed to complete: Illegal state: not currently leader
I20260709 13:19:47.431042 16143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26354361e9b044cea1c2b86c4882fe92" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
W20260709 13:19:47.433418 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 58660us
I20260709 13:19:47.412267 18418 consensus_queue.cc:1048] T 08c74490c40243a88e62d1760b5cd814 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.002s
I20260709 13:19:47.413962 18156 leader_election.cc:290] T 5a959dbecc2648eb85d1cc550f5a3245 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:19:47.414285 18240 consensus_queue.cc:1243] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b [LEADER]: Peer 1ca2725d290e47d68b1a4b609c9816ac log is divergent from this leader: its last log entry 4.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 13:19:47.413058 18109 leader_election.cc:290] T c2f5f89186204242bf377f7c42b2e2da P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:47.388453 18390 raft_consensus.cc:493] T d16dfad0c3b540e5b4b86492ddf87208 P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:19:47.413012 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 50910.5us
I20260709 13:19:47.430768 16790 leader_election.cc:304] T 9e65676b21b748728ad39668cfe722ec P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:19:47.447381 18402 leader_election.cc:290] T c9944d5e4b8c4109af9955c4f08c9095 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:19:47.448077 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02838c86531d4f98bf58d1c22b8baf77" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:19:47.448535 16115 leader_election.cc:304] T 9c61a39fd4f84424b129c1a6f9ba404d P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:19:47.453071 16266 leader_election.cc:304] T d2ab0ac8347641e6955e9bce7d90c1a2 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac
I20260709 13:19:47.388247 18323 raft_consensus.cc:515] T 7c2095a5f4ad49408dbc7f69c606120d P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:47.459713 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a959dbecc2648eb85d1cc550f5a3245" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:47.452350 18145 raft_consensus.cc:3060] T 8a878999c0024d2abe517484180290fc P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:19:47.453375 18390 raft_consensus.cc:3060] T d16dfad0c3b540e5b4b86492ddf87208 P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:19:47.375422 18181 consensus_queue.cc:260] T 18ad7e3af5504571a69d10c00750a01c P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } }
W20260709 13:19:47.470425 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 91926.4us
I20260709 13:19:47.473403 18109 raft_consensus.cc:2749] T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:19:47.475237 16218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9944d5e4b8c4109af9955c4f08c9095" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:47.475548 17999 leader_election.cc:290] T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 5 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:19:47.475943 16218 raft_consensus.cc:2468] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 5.
I20260709 13:19:47.476830 18181 raft_consensus.cc:3060] T 18ad7e3af5504571a69d10c00750a01c P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:19:47.478406 18402 raft_consensus.cc:2764] T d2ab0ac8347641e6955e9bce7d90c1a2 P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Leader pre-election decision vote started in defunct term 4: won
I20260709 13:19:47.479146 18402 raft_consensus.cc:493] T 1df93779209748b9860f6d893dd3de53 P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:47.479671 18402 raft_consensus.cc:515] T 1df93779209748b9860f6d893dd3de53 P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:47.480765 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e9b2bf7011f4ae688a79d7bb5a447b4" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:47.481665 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31969392ccdb43b491911a319e8bc7e1" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:19:47.482018 18402 leader_election.cc:290] T 1df93779209748b9860f6d893dd3de53 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:19:47.483045 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66ae176961b74813a58bc15c9c4c6f2a" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:47.483942 16818 raft_consensus.cc:1275] T d2ab0ac8347641e6955e9bce7d90c1a2 P cb82cc7bd6db46d980c75237a1402860 [term 5 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:19:47.484035 16293 raft_consensus.cc:2468] T 66ae176961b74813a58bc15c9c4c6f2a P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 6.
I20260709 13:19:47.475548 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3884f864a96d428785895e5463e5728f" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:47.485380 16293 raft_consensus.cc:1240] T cdbd78acb4cf46298e0de8ba0d130e19 P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 4. Current term is 5. Ops: []
I20260709 13:19:47.485805 16298 raft_consensus.cc:2468] T 3884f864a96d428785895e5463e5728f P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 3.
I20260709 13:19:47.483078 16789 leader_election.cc:304] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:19:47.461261 18352 raft_consensus.cc:2804] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:19:47.487046 16298 raft_consensus.cc:1240] T a378d7f796f94615b3b347119a9880fd P 90f3da57796b488c9087a2f523adf045 [term 4 LEADER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 3. Current term is 4. Ops: []
I20260709 13:19:47.486966 16293 raft_consensus.cc:1240] T d2ab0ac8347641e6955e9bce7d90c1a2 P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 4. Current term is 5. Ops: []
W20260709 13:19:47.502907 16115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 54477.3us
W20260709 13:19:47.485456 18363 log.cc:927] Time spent T 18ad7e3af5504571a69d10c00750a01c P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 1.172s	user 0.010s	sys 0.010s
I20260709 13:19:47.471694 18240 raft_consensus.cc:2749] T 9c61a39fd4f84424b129c1a6f9ba404d P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:19:47.472003 17992 raft_consensus.cc:2749] T 9e65676b21b748728ad39668cfe722ec P cb82cc7bd6db46d980c75237a1402860 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:19:47.509011 16266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 59426.7us
I20260709 13:19:47.475961 16292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c11074579b046d99e585c2818e18a2c" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:47.493883 18352 raft_consensus.cc:493] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:47.515435 16225 raft_consensus.cc:2468] T 5a959dbecc2648eb85d1cc550f5a3245 P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 3.
I20260709 13:19:47.448990 18389 raft_consensus.cc:515] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:47.519714 18387 raft_consensus.cc:2749] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:19:47.525327 18417 consensus_queue.cc:1048] T 57a744f5aaf944678a63469cbf514015 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.007s
I20260709 13:19:47.535053 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66ae176961b74813a58bc15c9c4c6f2a" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:19:47.528043 16224 raft_consensus.cc:1240] T 8a878999c0024d2abe517484180290fc P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 3. Current term is 4. Ops: []
I20260709 13:19:47.499665 16294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26354361e9b044cea1c2b86c4882fe92" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:19:47.537559 16190 leader_election.cc:304] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:19:47.544689 16266 leader_election.cc:304] T 434bff3082d141838068f323bdb0d1ed P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 647a343a542d4a139fe106c19095129b
I20260709 13:19:47.526180 16219 raft_consensus.cc:1275] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:19:47.535804 16299 raft_consensus.cc:2376] T 66ae176961b74813a58bc15c9c4c6f2a P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 6. Re-sending same reply.
I20260709 13:19:47.530776 16222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cc959fb7f704fba9009c0595467d074" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:47.453156 18105 consensus_queue.cc:260] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.4, Last appended by leader: 4, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:19:47.564419 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 151466us
W20260709 13:19:47.570638 16789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 87643.6us
I20260709 13:19:47.571312 16117 leader_election.cc:304] T 9e65676b21b748728ad39668cfe722ec P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860
W20260709 13:19:47.571848 18435 log.cc:927] Time spent T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.066s	user 0.000s	sys 0.000s
I20260709 13:19:47.573082 16789 leader_election.cc:304] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac
I20260709 13:19:47.573233 16117 leader_election.cc:304] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac
I20260709 13:19:47.575394 18418 raft_consensus.cc:2749] T 9e65676b21b748728ad39668cfe722ec P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:19:47.576241 18418 raft_consensus.cc:2804] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:19:47.576845 18013 consensus_queue.cc:1059] T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:19:47.576992 18422 consensus_queue.cc:1059] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:19:47.578043 18422 raft_consensus.cc:3055] T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:19:47.578399 18422 raft_consensus.cc:740] T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:19:47.503288 18424 consensus_queue.cc:1059] T e5ad4eb281564936a59f41e387a47156 P 647a343a542d4a139fe106c19095129b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: INVALID_TERM, Last received: 4.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.062s
I20260709 13:19:47.524856 16292 raft_consensus.cc:1240] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 6. Current term is 7. Ops: []
I20260709 13:19:47.538533 18140 raft_consensus.cc:2749] T 8158bbd17b994d2eb96f560bd0f1b88d P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:19:47.543738 18352 raft_consensus.cc:3060] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:19:47.582803 16266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 65827.3us
I20260709 13:19:47.585031 16299 raft_consensus.cc:1240] T b87147731a68400e8cce9b879daa7c5d P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 5. Current term is 6. Ops: []
I20260709 13:19:47.587174 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a9de347c711477e87c582d003f36925" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:19:47.588025 16293 raft_consensus.cc:2376] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 5. Re-sending same reply.
I20260709 13:19:47.589164 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74ca8140f35c4ae29e8a664f6bdfc829" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:19:47.589864 16293 raft_consensus.cc:2376] T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 4. Re-sending same reply.
I20260709 13:19:47.580951 16818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74ca8140f35c4ae29e8a664f6bdfc829" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:19:47.595364 16818 raft_consensus.cc:2468] T 74ca8140f35c4ae29e8a664f6bdfc829 P cb82cc7bd6db46d980c75237a1402860 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 4.
I20260709 13:19:47.596283 18424 raft_consensus.cc:3055] T e5ad4eb281564936a59f41e387a47156 P 647a343a542d4a139fe106c19095129b [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:19:47.596728 18424 raft_consensus.cc:740] T e5ad4eb281564936a59f41e387a47156 P 647a343a542d4a139fe106c19095129b [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:19:47.598603 16292 pending_rounds.cc:85] T c9944d5e4b8c4109af9955c4f08c9095 P 90f3da57796b488c9087a2f523adf045: Aborting all ops after (but not including) 6
I20260709 13:19:47.599052 16292 pending_rounds.cc:107] T c9944d5e4b8c4109af9955c4f08c9095 P 90f3da57796b488c9087a2f523adf045: Aborting uncommitted NO_OP operation due to leader change: 4.7
I20260709 13:19:47.599356 16292 raft_consensus.cc:2889] T c9944d5e4b8c4109af9955c4f08c9095 P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:19:47.602852 16115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 84780.1us
I20260709 13:19:47.607630 16222 raft_consensus.cc:2468] T 8cc959fb7f704fba9009c0595467d074 P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 4.
I20260709 13:19:47.609969 18388 raft_consensus.cc:2804] T 434bff3082d141838068f323bdb0d1ed P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:19:47.610440 18388 raft_consensus.cc:493] T 434bff3082d141838068f323bdb0d1ed P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:19:47.610836 18388 raft_consensus.cc:3060] T 434bff3082d141838068f323bdb0d1ed P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:19:47.616482 16790 leader_election.cc:304] T 2794af5b99984efebdee564035841cb0 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:19:47.617045 18346 raft_consensus.cc:3060] T 8158bbd17b994d2eb96f560bd0f1b88d P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:19:47.618908 18374 raft_consensus.cc:2749] T 2794af5b99984efebdee564035841cb0 P cb82cc7bd6db46d980c75237a1402860 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:19:47.619427 18401 consensus_queue.cc:1048] T af57996a8be84d398f0f023e8d08ef77 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:19:47.619788 18245 consensus_queue.cc:1048] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:19:47.621551 18401 consensus_queue.cc:1059] T b87147731a68400e8cce9b879daa7c5d P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:19:47.622486 18245 raft_consensus.cc:3055] T b87147731a68400e8cce9b879daa7c5d P cb82cc7bd6db46d980c75237a1402860 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:19:47.622936 18245 raft_consensus.cc:740] T b87147731a68400e8cce9b879daa7c5d P cb82cc7bd6db46d980c75237a1402860 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:19:47.624140 18245 consensus_queue.cc:260] T b87147731a68400e8cce9b879daa7c5d P cb82cc7bd6db46d980c75237a1402860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:47.625051 16190 leader_election.cc:304] T 19baa7bc367844c288bca8f81439c413 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 647a343a542d4a139fe106c19095129b
I20260709 13:19:47.625514 18245 raft_consensus.cc:3060] T b87147731a68400e8cce9b879daa7c5d P cb82cc7bd6db46d980c75237a1402860 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:19:47.628600 16266 leader_election.cc:400] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 4 election: Vote denied by peer cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923) with higher term. Message: Invalid argument: T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 for earlier term 4. Current term is 5.
I20260709 13:19:47.629110 16820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a378d7f796f94615b3b347119a9880fd" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:19:47.629619 16266 leader_election.cc:400] T 66ae176961b74813a58bc15c9c4c6f2a P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 5 pre-election: Vote denied by peer cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923) with higher term. Message: Invalid argument: T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 for earlier term 5. Current term is 6.
I20260709 13:19:47.630591 16266 leader_election.cc:304] T 8c11074579b046d99e585c2818e18a2c P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860
I20260709 13:19:47.632665 16266 leader_election.cc:400] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 4 pre-election: Vote denied by peer cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923) with higher term. Message: Invalid argument: T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 for earlier term 4. Current term is 5.
I20260709 13:19:47.632913 18105 raft_consensus.cc:3060] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:19:47.626478 16190 leader_election.cc:304] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: cb82cc7bd6db46d980c75237a1402860
I20260709 13:19:47.634848 16190 leader_election.cc:304] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:19:47.635769 16825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57a744f5aaf944678a63469cbf514015" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:19:47.636003 18318 raft_consensus.cc:2804] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:19:47.636440 18318 raft_consensus.cc:493] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:47.636523 16825 raft_consensus.cc:2468] T 57a744f5aaf944678a63469cbf514015 P cb82cc7bd6db46d980c75237a1402860 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 3.
I20260709 13:19:47.636891 18318 raft_consensus.cc:3060] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:19:47.635716 18404 raft_consensus.cc:2804] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:19:47.639051 18404 raft_consensus.cc:697] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [term 5 LEADER]: Becoming Leader. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:19:47.640034 18406 leader_election.cc:290] T c0eab25ce4c64e018a041beeda2867db P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:47.640161 18404 consensus_queue.cc:237] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:47.569428 18255 raft_consensus.cc:2804] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:19:47.647011 16222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0eab25ce4c64e018a041beeda2867db" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:47.628091 18272 raft_consensus.cc:2804] T 19baa7bc367844c288bca8f81439c413 P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:19:47.648979 16190 leader_election.cc:304] T 8a878999c0024d2abe517484180290fc P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:19:47.642320 16266 leader_election.cc:304] T 8638ee363273426db1393dd81e5c7b72 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860
I20260709 13:19:47.650838 18372 raft_consensus.cc:2804] T 8a878999c0024d2abe517484180290fc P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:19:47.660352 18372 raft_consensus.cc:697] T 8a878999c0024d2abe517484180290fc P 1ca2725d290e47d68b1a4b609c9816ac [term 4 LEADER]: Becoming Leader. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:19:47.576841 18418 raft_consensus.cc:697] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [term 5 LEADER]: Becoming Leader. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:19:47.642069 18276 raft_consensus.cc:515] T 68a6838d82ef4cac83507f906139d90f P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:47.586267 18422 consensus_queue.cc:260] T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:47.655828 18433 raft_consensus.cc:2804] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:19:47.656400 16190 leader_election.cc:304] T e4dc6b0541ca431ebc757beac00521f1 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:19:47.657074 17844 consensus_queue.cc:1048] T d2ab0ac8347641e6955e9bce7d90c1a2 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:19:47.657413 18389 leader_election.cc:290] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
W20260709 13:19:47.658145 16117 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 83070.5us
I20260709 13:19:47.599769 17992 consensus_queue.cc:1059] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.075s
I20260709 13:19:47.659654 16300 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48674 (request call id 2072) took 2828 ms. Trace:
W20260709 13:19:47.672289 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 53097.5us
I20260709 13:19:47.659399 18318 raft_consensus.cc:515] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:47.636571 18323 leader_election.cc:290] T 7c2095a5f4ad49408dbc7f69c606120d P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 4 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:19:47.679214 16222 raft_consensus.cc:1240] T af57996a8be84d398f0f023e8d08ef77 P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 4. Current term is 5. Ops: []
I20260709 13:19:47.685614 16297 raft_consensus.cc:2468] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 2.
I20260709 13:19:47.656605 18123 raft_consensus.cc:3055] T fffab26427e24d398e2695aad40ca134 P 90f3da57796b488c9087a2f523adf045 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:19:47.702860 16115 leader_election.cc:304] T 7c2095a5f4ad49408dbc7f69c606120d P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:19:47.704892 16222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c61a39fd4f84424b129c1a6f9ba404d" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:19:47.664655 16293 raft_consensus.cc:1240] T 23f82224f4564fea93572e8206bad1fe P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 1. Current term is 2. Ops: []
I20260709 13:19:47.656558 18384 consensus_queue.cc:1048] T 7c2095a5f4ad49408dbc7f69c606120d P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.054s
I20260709 13:19:47.692302 18387 raft_consensus.cc:2804] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:19:47.674304 16300 rpcz_store.cc:276] 0709 13:19:44.830717 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:45.326574 (+495857us) service_pool.cc:224] Handling call
0709 13:19:45.326886 (+   312us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:45.327259 (+   373us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:45.327292 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:45.332350 (+  5058us) log.cc:844] Serialized 26 byte log entry
0709 13:19:46.086481 (+754131us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:19:46.212180 (+125699us) log.cc:844] Serialized 14 byte log entry
0709 13:19:46.832974 (+620794us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:46.846001 (+ 13027us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:47.484522 (+638521us) spinlock_profiling.cc:243] Waited 49 ms on lock 0x7b4400022b4c. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69ad8fadb1 00007f69ad998c85 00007f69ad9996a2 00007f69b070b1db 00007f69b0706aaf 00007f69b06fa86e 00007f69b06f9bb6 00007f69b06f97d8 00007f69b0703221 00007f69b07031d9 00007f69b0703169
0709 13:19:47.659238 (+174716us) raft_consensus.cc:1685] finished
0709 13:19:47.659339 (+   101us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:47.659499 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":18214,"prepare.queue_time_us":355477,"prepare.run_cpu_time_us":45,"prepare.run_wall_time_us":47,"raft.queue_time_us":183133,"raft.run_cpu_time_us":247,"raft.run_wall_time_us":4104,"spinlock_wait_cycles":112773504,"thread_start_us":953154,"threads_started":3,"wal-append.queue_time_us":905387}
W20260709 13:19:47.734781 16117 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 62130.7us
I20260709 13:19:47.665086 18181 consensus_queue.cc:1048] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.056s
I20260709 13:19:47.700773 18433 raft_consensus.cc:493] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:19:47.710130 18123 raft_consensus.cc:740] T fffab26427e24d398e2695aad40ca134 P 90f3da57796b488c9087a2f523adf045 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:19:47.700536 18272 raft_consensus.cc:697] T 19baa7bc367844c288bca8f81439c413 P 1ca2725d290e47d68b1a4b609c9816ac [term 5 LEADER]: Becoming Leader. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:19:47.739044 16147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62fe3346cd3c42c4a55e14af9073bca4" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:19:47.744174 16115 leader_election.cc:304] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:19:47.745209 16117 leader_election.cc:304] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860
W20260709 13:19:47.749030 16266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 106849us
I20260709 13:19:47.726594 18145 raft_consensus.cc:2749] T 7c2095a5f4ad49408dbc7f69c606120d P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:19:47.761696 16225 raft_consensus.cc:1275] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:19:47.768592 18313 consensus_queue.cc:1048] T d2ab0ac8347641e6955e9bce7d90c1a2 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:19:47.768806 18434 raft_consensus.cc:2749] T 8638ee363273426db1393dd81e5c7b72 P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:19:47.725484 18400 consensus_queue.cc:1059] T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.039s
I20260709 13:19:47.769515 16147 raft_consensus.cc:2468] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 6.
I20260709 13:19:47.770463 18013 raft_consensus.cc:2749] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:19:47.700475 18372 consensus_queue.cc:237] T 8a878999c0024d2abe517484180290fc P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:47.751698 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1df93779209748b9860f6d893dd3de53" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:47.775842 16217 raft_consensus.cc:2468] T 1df93779209748b9860f6d893dd3de53 P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 4.
I20260709 13:19:47.716274 18374 consensus_queue.cc:1059] T 3a7ca61592e848ea9b0fb81a2ce33cdc P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 5, Time since last communication: 0.036s
I20260709 13:19:47.662087 18418 consensus_queue.cc:237] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:47.757925 18422 raft_consensus.cc:3060] T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:19:47.741572 18433 raft_consensus.cc:3060] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:19:47.784549 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c2095a5f4ad49408dbc7f69c606120d" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
W20260709 13:19:47.789059 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 92758.8us
W20260709 13:19:47.789561 16117 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 53924.9us
I20260709 13:19:47.791482 16117 leader_election.cc:304] T 829b4ad8c5a54eababeea1181b960d4b P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860
I20260709 13:19:47.753610 16142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66f75124215f4ca6bef831e174a792d8" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:19:47.739324 18401 consensus_queue.cc:1059] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.039s
I20260709 13:19:47.799345 18450 raft_consensus.cc:493] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:19:47.800092 18450 raft_consensus.cc:515] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:47.804663 18450 leader_election.cc:290] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:47.757025 18245 consensus_queue.cc:1059] T af57996a8be84d398f0f023e8d08ef77 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.040s
I20260709 13:19:47.814702 16266 leader_election.cc:304] T 3884f864a96d428785895e5463e5728f P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:19:47.765363 18388 raft_consensus.cc:515] T 434bff3082d141838068f323bdb0d1ed P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:47.811400 18276 leader_election.cc:290] T 68a6838d82ef4cac83507f906139d90f P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 6 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:19:47.869822 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4dc6b0541ca431ebc757beac00521f1" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
W20260709 13:19:47.875212 16791 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:47.815464 18123 consensus_queue.cc:260] T fffab26427e24d398e2695aad40ca134 P 90f3da57796b488c9087a2f523adf045 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:19:47.857916 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 105790us
I20260709 13:19:47.873198 16147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a959dbecc2648eb85d1cc550f5a3245" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:19:47.829615 18145 raft_consensus.cc:2749] T 829b4ad8c5a54eababeea1181b960d4b P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:19:47.890985 16117 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 99602.5us
W20260709 13:19:47.891530 16116 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:47.892300 16790 leader_election.cc:304] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:19:47.864367 18256 raft_consensus.cc:2804] T e4dc6b0541ca431ebc757beac00521f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:19:47.896121 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8158bbd17b994d2eb96f560bd0f1b88d" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
W20260709 13:19:47.922786 16191 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.14.177.129:36741 timed out after 2.133s (SENT) [suppressed 51 similar messages]
I20260709 13:19:47.924098 16147 raft_consensus.cc:3060] T 5a959dbecc2648eb85d1cc550f5a3245 P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:19:47.940857 16192 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 180504us
W20260709 13:19:47.958616 18437 log.cc:927] Time spent T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 0.363s	user 0.004s	sys 0.000s
I20260709 13:19:47.903718 18318 leader_election.cc:290] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 5 election: Requested vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:19:47.977120 16145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a878999c0024d2abe517484180290fc" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:19:47.981053 16148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "829b4ad8c5a54eababeea1181b960d4b" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
W20260709 13:19:47.981534 18440 log.cc:927] Time spent T 66f75124215f4ca6bef831e174a792d8 P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 0.159s	user 0.001s	sys 0.000s
I20260709 13:19:47.981911 16148 raft_consensus.cc:2468] T 829b4ad8c5a54eababeea1181b960d4b P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 5.
I20260709 13:19:47.923718 18390 raft_consensus.cc:515] T d16dfad0c3b540e5b4b86492ddf87208 P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:47.956434 18400 raft_consensus.cc:3055] T 3a7ca61592e848ea9b0fb81a2ce33cdc P cb82cc7bd6db46d980c75237a1402860 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:19:47.953613 18256 raft_consensus.cc:697] T e4dc6b0541ca431ebc757beac00521f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 4 LEADER]: Becoming Leader. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
W20260709 13:19:47.988070 18444 log.cc:927] Time spent T 8c11074579b046d99e585c2818e18a2c P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.115s	user 0.000s	sys 0.001s
W20260709 13:19:47.989074 18446 log.cc:927] Time spent T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 0.097s	user 0.000s	sys 0.000s
W20260709 13:19:47.990072 18441 log.cc:927] Time spent T 84f2da98864b4425bf7c0055951fdd4b P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 0.087s	user 0.000s	sys 0.000s
I20260709 13:19:47.963349 18262 raft_consensus.cc:2749] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:19:47.924789 18352 raft_consensus.cc:515] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:47.962291 17992 raft_consensus.cc:2804] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:19:47.961596 18123 raft_consensus.cc:3060] T fffab26427e24d398e2695aad40ca134 P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:19:48.005367 18181 raft_consensus.cc:493] T fffab26427e24d398e2695aad40ca134 P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:19:47.961341 18401 raft_consensus.cc:493] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:19:47.993633 18400 raft_consensus.cc:740] T 3a7ca61592e848ea9b0fb81a2ce33cdc P cb82cc7bd6db46d980c75237a1402860 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:19:48.040761 16151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ad5dbfa99ee4b8d9e28be92b3f36db5" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:19:48.041913 16151 raft_consensus.cc:2393] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 4: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
I20260709 13:19:48.013185 16142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fffab26427e24d398e2695aad40ca134" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:19:48.021826 17992 raft_consensus.cc:493] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:48.053933 16148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:19:47.963408 18272 consensus_queue.cc:237] T 19baa7bc367844c288bca8f81439c413 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } }
I20260709 13:19:48.065799 18145 raft_consensus.cc:493] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:48.078310 18145 raft_consensus.cc:515] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:48.079366 18434 raft_consensus.cc:493] T 5a96373bb0e14dce8a266b2fc6c58507 P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:48.052606 17992 raft_consensus.cc:3060] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:19:48.067232 16151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18ad7e3af5504571a69d10c00750a01c" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:19:48.081660 16149 raft_consensus.cc:2468] T 8158bbd17b994d2eb96f560bd0f1b88d P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 5.
I20260709 13:19:48.083459 16147 raft_consensus.cc:2468] T 5a959dbecc2648eb85d1cc550f5a3245 P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 3.
I20260709 13:19:48.085170 16147 raft_consensus.cc:1240] T 8158bbd17b994d2eb96f560bd0f1b88d P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 4. Current term is 5. Ops: [4.4-4.4]
I20260709 13:19:48.079622 18256 consensus_queue.cc:237] T e4dc6b0541ca431ebc757beac00521f1 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:48.060905 18181 raft_consensus.cc:515] T fffab26427e24d398e2695aad40ca134 P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:48.099472 16148 raft_consensus.cc:2393] T ecc1e363230043fe9fef390f0b979f20 P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 4: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
I20260709 13:19:48.107322 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4dc6b0541ca431ebc757beac00521f1" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
W20260709 13:19:48.108385 16266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 314839us
I20260709 13:19:48.109159 16266 leader_election.cc:304] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860
I20260709 13:19:48.110539 16148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:19:48.111335 16148 raft_consensus.cc:2393] T ecc1e363230043fe9fef390f0b979f20 P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 4: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
I20260709 13:19:48.064981 18388 leader_election.cc:290] T 434bff3082d141838068f323bdb0d1ed P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 5 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:19:48.043190 18401 raft_consensus.cc:515] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:48.080029 18434 raft_consensus.cc:515] T 5a96373bb0e14dce8a266b2fc6c58507 P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:48.132067 16295 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48684 (request call id 1964) took 1249 ms. Trace:
I20260709 13:19:48.117050 16148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f82224f4564fea93572e8206bad1fe" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:19:48.269158 16148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a878999c0024d2abe517484180290fc" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:19:48.234556 16151 raft_consensus.cc:2468] T 18ad7e3af5504571a69d10c00750a01c P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 4.
I20260709 13:19:48.270140 16143 raft_consensus.cc:1275] T 829b4ad8c5a54eababeea1181b960d4b P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:19:48.270253 16151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8158bbd17b994d2eb96f560bd0f1b88d" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:19:48.270946 16151 raft_consensus.cc:2376] T 8158bbd17b994d2eb96f560bd0f1b88d P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 5. Re-sending same reply.
I20260709 13:19:48.268209 16295 rpcz_store.cc:276] 0709 13:19:46.882576 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:47.021649 (+139073us) service_pool.cc:224] Handling call
0709 13:19:47.342411 (+320762us) spinlock_profiling.cc:243] Waited 980 us on lock 0x7b5800090098. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69ad8fadb1 00007f69ada3ab71 00007f69ada3aeee 0000000000375939 00007f69b5aae1e0 00007f69b5b770c4 00007f69b5b77131 00007f69b5b38fc0 00007f69b5b23929 00007f69b5b233d2 00007f69af09abd4
0709 13:19:47.342585 (+   174us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:47.453735 (+111150us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:19:47.512220 (+ 58485us) log.cc:844] Serialized 14 byte log entry
0709 13:19:47.691525 (+179305us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:47.733116 (+ 41591us) log.cc:844] Serialized 26 byte log entry
0709 13:19:47.733290 (+   174us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:19:47.733537 (+   247us) log.cc:844] Serialized 14 byte log entry
0709 13:19:48.057565 (+324028us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:48.057649 (+    84us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:48.131750 (+ 74101us) raft_consensus.cc:1685] finished
0709 13:19:48.131785 (+    35us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:48.131986 (+   201us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":671119,"prepare.queue_time_us":55294,"prepare.run_cpu_time_us":129,"prepare.run_wall_time_us":4877,"raft.queue_time_us":393163,"raft.run_cpu_time_us":384,"raft.run_wall_time_us":22578,"spinlock_wait_cycles":2254208,"thread_start_us":232227,"threads_started":2,"wal-append.queue_time_us":249828}
I20260709 13:19:48.249657 16220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fffab26427e24d398e2695aad40ca134" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:48.100350 18104 raft_consensus.cc:493] T 8cc959fb7f704fba9009c0595467d074 P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:48.277673 16220 raft_consensus.cc:2393] T fffab26427e24d398e2695aad40ca134 P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 3: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
I20260709 13:19:48.117401 16223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68a6838d82ef4cac83507f906139d90f" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:19:48.162389 16822 raft_consensus.cc:1275] T 806d7690c2b644959152074bb2cab0a7 P cb82cc7bd6db46d980c75237a1402860 [term 3 LEARNER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 13:19:48.209522 16150 raft_consensus.cc:2468] T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 4.
I20260709 13:19:48.279366 16822 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49896 (request call id 831) took 1051 ms. Trace:
I20260709 13:19:48.279955 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a959dbecc2648eb85d1cc550f5a3245" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:19:48.280819 16150 raft_consensus.cc:2468] T 5a959dbecc2648eb85d1cc550f5a3245 P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 3.
W20260709 13:19:48.201933 18465 log.cc:927] Time spent T 746aabf856ab401c9ef5e859371e7b67 P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 0.051s	user 0.001s	sys 0.000s
I20260709 13:19:48.279793 16822 rpcz_store.cc:276] 0709 13:19:47.228144 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:47.297006 (+ 68862us) service_pool.cc:224] Handling call
0709 13:19:48.049585 (+752579us) spinlock_profiling.cc:243] Waited 1.91 ms on lock 0x7b5800004298. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69ad8fadb1 00007f69ada3ab71 00007f69ada3aa22 00007f69b6953c09 00007f69b5aadee0 00007f69b5b770ab 00007f69b5b77131 00007f69b5b38fc0 00007f69b5b23929 00007f69b5b233d2 00007f69af09abd4
0709 13:19:48.054967 (+  5382us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:48.279081 (+224114us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:48.279306 (+   225us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4390144}
W20260709 13:19:48.263814 18458 log.cc:927] Time spent T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 0.243s	user 0.001s	sys 0.000s
I20260709 13:19:48.260552 18390 leader_election.cc:290] T d16dfad0c3b540e5b4b86492ddf87208 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 4 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:19:48.133656 16224 raft_consensus.cc:1217] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac [term 4 LEADER]: Deduplicated request from leader. Original: 2.5->[3.6-3.6]   Dedup: 3.6->[]
I20260709 13:19:48.295903 16224 raft_consensus.cc:1240] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac [term 4 LEADER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 3. Current term is 4. Ops: [3.6-3.6]
I20260709 13:19:48.132665 18276 raft_consensus.cc:2804] T 3884f864a96d428785895e5463e5728f P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:19:48.290329 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02838c86531d4f98bf58d1c22b8baf77" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:19:48.292027 16148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87ac0d1f7e8a419f9bcbc04aee0e067e" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:19:48.283378 16265 leader_election.cc:304] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:19:48.259852 18352 leader_election.cc:290] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 5 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:19:48.306133 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee9ed67006144797b1de6ea8c2013a68" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:19:48.125726 18181 leader_election.cc:290] T fffab26427e24d398e2695aad40ca134 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:48.108794 16142 raft_consensus.cc:3060] T fffab26427e24d398e2695aad40ca134 P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:19:48.175500 18438 raft_consensus.cc:2749] T 8c11074579b046d99e585c2818e18a2c P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:19:48.303543 16266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 194433us
I20260709 13:19:48.304881 16145 raft_consensus.cc:2468] T 8a878999c0024d2abe517484180290fc P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 4.
I20260709 13:19:48.311561 16266 leader_election.cc:304] T cdbd78acb4cf46298e0de8ba0d130e19 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860
I20260709 13:19:48.316332 16190 leader_election.cc:304] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: cb82cc7bd6db46d980c75237a1402860
I20260709 13:19:48.285126 18388 raft_consensus.cc:2749] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:19:48.283452 18346 raft_consensus.cc:493] T c4f65cdf00b748bb963ee6e81ef43b79 P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:19:48.323977 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5d59407eb545b895d8d922e0dd69fd" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:19:48.326799 16150 raft_consensus.cc:2376] T ee9ed67006144797b1de6ea8c2013a68 P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 4. Re-sending same reply.
I20260709 13:19:48.327490 16148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c61a39fd4f84424b129c1a6f9ba404d" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:19:48.305482 18276 raft_consensus.cc:493] T 3884f864a96d428785895e5463e5728f P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:19:48.334556 16264 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:48.336227 16148 raft_consensus.cc:3060] T 9c61a39fd4f84424b129c1a6f9ba404d P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:19:48.336161 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 436528us
I20260709 13:19:48.344398 18432 raft_consensus.cc:2749] T cdbd78acb4cf46298e0de8ba0d130e19 P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:19:48.347698 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4f65cdf00b748bb963ee6e81ef43b79" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:19:48.348982 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68a6838d82ef4cac83507f906139d90f" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:19:48.350204 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4dc6b0541ca431ebc757beac00521f1" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:19:48.350757 16149 raft_consensus.cc:2376] T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 4. Re-sending same reply.
I20260709 13:19:48.352336 18434 leader_election.cc:290] T 5a96373bb0e14dce8a266b2fc6c58507 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
W20260709 13:19:48.359261 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 75983.2us
I20260709 13:19:48.360142 16265 leader_election.cc:304] T c0eab25ce4c64e018a041beeda2867db P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:19:48.294811 18104 raft_consensus.cc:515] T 8cc959fb7f704fba9009c0595467d074 P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:48.369174 18104 leader_election.cc:290] T 8cc959fb7f704fba9009c0595467d074 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:48.370378 16144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434bff3082d141838068f323bdb0d1ed" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:19:48.371766 16144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cc959fb7f704fba9009c0595467d074" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:19:48.372355 16144 raft_consensus.cc:2468] T 8cc959fb7f704fba9009c0595467d074 P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 4.
I20260709 13:19:48.373201 16144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fffab26427e24d398e2695aad40ca134" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:19:48.339566 18272 consensus_queue.cc:1048] T c0eab25ce4c64e018a041beeda2867db P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.036s
W20260709 13:19:48.377118 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 45412.2us
I20260709 13:19:48.332010 18346 raft_consensus.cc:515] T c4f65cdf00b748bb963ee6e81ef43b79 P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:48.385351 18346 leader_election.cc:290] T c4f65cdf00b748bb963ee6e81ef43b79 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:19:48.386799 18145 leader_election.cc:290] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:19:48.330658 18347 raft_consensus.cc:2749] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:19:48.388175 18426 raft_consensus.cc:3055] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:19:48.388764 18426 raft_consensus.cc:740] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:19:48.389941 18426 consensus_queue.cc:260] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:48.393929 18426 raft_consensus.cc:3060] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:19:48.313223 16143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a959dbecc2648eb85d1cc550f5a3245" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:19:48.395610 16143 raft_consensus.cc:2376] T 5a959dbecc2648eb85d1cc550f5a3245 P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 3. Re-sending same reply.
I20260709 13:19:48.396859 16143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ad5dbfa99ee4b8d9e28be92b3f36db5" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:19:48.397711 16143 raft_consensus.cc:2393] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 4: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
I20260709 13:19:48.398569 16143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5ad4eb281564936a59f41e387a47156" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:19:48.405771 18254 leader_election.cc:290] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
W20260709 13:19:48.410488 18439 log.cc:927] Time spent T ada5a004ba7c46819051e3ccec71ffdb P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.600s	user 0.001s	sys 0.000s
I20260709 13:19:48.175956 18400 consensus_queue.cc:260] T 3a7ca61592e848ea9b0fb81a2ce33cdc P cb82cc7bd6db46d980c75237a1402860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:48.419096 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c2095a5f4ad49408dbc7f69c606120d" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:19:48.419819 16149 raft_consensus.cc:3060] T 7c2095a5f4ad49408dbc7f69c606120d P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:19:48.398226 18424 consensus_queue.cc:260] T e5ad4eb281564936a59f41e387a47156 P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:48.422856 18424 raft_consensus.cc:3060] T e5ad4eb281564936a59f41e387a47156 P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:19:48.425258 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "829b4ad8c5a54eababeea1181b960d4b" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:19:48.426340 16150 raft_consensus.cc:2468] T 829b4ad8c5a54eababeea1181b960d4b P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 5.
I20260709 13:19:48.337502 18276 raft_consensus.cc:3060] T 3884f864a96d428785895e5463e5728f P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:19:48.436540 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 76484.3us
I20260709 13:19:48.339401 18452 raft_consensus.cc:2804] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:19:48.446255 16265 leader_election.cc:304] T b87147731a68400e8cce9b879daa7c5d P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: cb82cc7bd6db46d980c75237a1402860
I20260709 13:19:48.447926 16144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f5f89186204242bf377f7c42b2e2da" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 7 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:19:48.425187 18390 raft_consensus.cc:493] T ecc1e363230043fe9fef390f0b979f20 P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:48.452869 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af57996a8be84d398f0f023e8d08ef77" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
W20260709 13:19:48.449432 18422 consensus_meta.cc:287] T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b: Time spent flushing consensus metadata: real 0.656s	user 0.019s	sys 0.002s
I20260709 13:19:48.426354 18323 raft_consensus.cc:493] T e5ad4eb281564936a59f41e387a47156 P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:19:48.462922 16144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62fe3346cd3c42c4a55e14af9073bca4" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:19:48.450799 18434 raft_consensus.cc:2749] T c0eab25ce4c64e018a041beeda2867db P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:19:48.456063 18400 raft_consensus.cc:3060] T 3a7ca61592e848ea9b0fb81a2ce33cdc P cb82cc7bd6db46d980c75237a1402860 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:19:48.468971 16144 raft_consensus.cc:2393] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 6: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
I20260709 13:19:48.452458 18452 raft_consensus.cc:697] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 1ca2725d290e47d68b1a4b609c9816ac [term 7 LEADER]: Becoming Leader. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:19:48.470960 16265 leader_election.cc:304] T e4dc6b0541ca431ebc757beac00521f1 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:19:48.464910 16149 raft_consensus.cc:2468] T 7c2095a5f4ad49408dbc7f69c606120d P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 4.
W20260709 13:19:48.483404 16116 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:48.491261 18452 consensus_queue.cc:237] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:48.496410 18472 raft_consensus.cc:2749] T e4dc6b0541ca431ebc757beac00521f1 P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:19:48.472620 16265 leader_election.cc:304] T 5a959dbecc2648eb85d1cc550f5a3245 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:19:48.499917 16265 leader_election.cc:304] T 02838c86531d4f98bf58d1c22b8baf77 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:19:48.501761 16265 leader_election.cc:304] T 8cc959fb7f704fba9009c0595467d074 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:19:48.502568 18432 raft_consensus.cc:2804] T 5a959dbecc2648eb85d1cc550f5a3245 P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:19:48.502955 18347 raft_consensus.cc:2749] T 02838c86531d4f98bf58d1c22b8baf77 P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:19:48.502952 18432 raft_consensus.cc:493] T 5a959dbecc2648eb85d1cc550f5a3245 P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:48.503755 18432 raft_consensus.cc:3060] T 5a959dbecc2648eb85d1cc550f5a3245 P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:19:48.503948 18347 raft_consensus.cc:2804] T 8cc959fb7f704fba9009c0595467d074 P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:19:48.504524 18347 raft_consensus.cc:493] T 8cc959fb7f704fba9009c0595467d074 P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:48.508613 16142 raft_consensus.cc:2410] T fffab26427e24d398e2695aad40ca134 P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac for term 3 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:19:48.509570 16149 raft_consensus.cc:1275] T c0eab25ce4c64e018a041beeda2867db P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 3. (term mismatch)
W20260709 13:19:48.512296 18433 consensus_meta.cc:287] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 90f3da57796b488c9087a2f523adf045: Time spent flushing consensus metadata: real 0.694s	user 0.008s	sys 0.004s
I20260709 13:19:48.512633 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02838c86531d4f98bf58d1c22b8baf77" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:48.513454 16292 raft_consensus.cc:1275] T 1df93779209748b9860f6d893dd3de53 P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:19:48.514539 16293 raft_consensus.cc:1240] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 4. Current term is 5. Ops: []
I20260709 13:19:48.515530 16294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a959dbecc2648eb85d1cc550f5a3245" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:48.515897 16291 raft_consensus.cc:1240] T d16dfad0c3b540e5b4b86492ddf87208 P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 3. Current term is 4. Ops: []
I20260709 13:19:48.516762 16296 raft_consensus.cc:1240] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045 [term 4 LEADER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 3. Current term is 4. Ops: []
I20260709 13:19:48.518702 16300 raft_consensus.cc:1240] T 8a878999c0024d2abe517484180290fc P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 3. Current term is 4. Ops: []
I20260709 13:19:48.520222 16265 leader_election.cc:304] T 7c2095a5f4ad49408dbc7f69c606120d P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:19:48.528607 16298 raft_consensus.cc:2393] T 02838c86531d4f98bf58d1c22b8baf77 P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 5: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
I20260709 13:19:48.527607 16149 pending_rounds.cc:85] T c0eab25ce4c64e018a041beeda2867db P 647a343a542d4a139fe106c19095129b: Aborting all ops after (but not including) 2
I20260709 13:19:48.554519 16149 pending_rounds.cc:107] T c0eab25ce4c64e018a041beeda2867db P 647a343a542d4a139fe106c19095129b: Aborting uncommitted NO_OP operation due to leader change: 3.3
I20260709 13:19:48.528797 16789 leader_election.cc:304] T 3884f864a96d428785895e5463e5728f P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 647a343a542d4a139fe106c19095129b
I20260709 13:19:48.531195 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a96373bb0e14dce8a266b2fc6c58507" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:19:48.466058 18346 raft_consensus.cc:2804] T b87147731a68400e8cce9b879daa7c5d P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Leader election won for term 6
W20260709 13:19:48.553709 16266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 220642us
I20260709 13:19:48.554879 16149 raft_consensus.cc:2889] T c0eab25ce4c64e018a041beeda2867db P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:19:48.542811 16295 raft_consensus.cc:1275] T 84f2da98864b4425bf7c0055951fdd4b P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
W20260709 13:19:48.564159 16266 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:48.563926 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 61298.4us
W20260709 13:19:48.567138 16115 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:48.527411 18347 raft_consensus.cc:3060] T 8cc959fb7f704fba9009c0595467d074 P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:19:48.571347 16789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 990612us
I20260709 13:19:48.573381 16296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0eab25ce4c64e018a041beeda2867db" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:48.574218 16296 raft_consensus.cc:2468] T c0eab25ce4c64e018a041beeda2867db P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 4.
I20260709 13:19:48.464721 18390 raft_consensus.cc:515] T ecc1e363230043fe9fef390f0b979f20 P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } }
I20260709 13:19:48.579465 16150 raft_consensus.cc:2468] T 5a96373bb0e14dce8a266b2fc6c58507 P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 3.
I20260709 13:19:48.582417 16300 raft_consensus.cc:1275] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 4. (term mismatch)
I20260709 13:19:48.582581 16220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87ac0d1f7e8a419f9bcbc04aee0e067e" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:19:48.583098 16789 leader_election.cc:304] T 8c11074579b046d99e585c2818e18a2c P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:19:48.585079 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87ac0d1f7e8a419f9bcbc04aee0e067e" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:19:48.586293 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4f65cdf00b748bb963ee6e81ef43b79" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:19:48.519133 18323 raft_consensus.cc:515] T e5ad4eb281564936a59f41e387a47156 P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:48.591254 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d16dfad0c3b540e5b4b86492ddf87208" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:19:48.569548 18123 raft_consensus.cc:2804] T 7c2095a5f4ad49408dbc7f69c606120d P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:19:48.582837 16300 pending_rounds.cc:85] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045: Aborting all ops after (but not including) 3
I20260709 13:19:48.595528 16300 pending_rounds.cc:107] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045: Aborting uncommitted NO_OP operation due to leader change: 5.4
I20260709 13:19:48.595826 16300 raft_consensus.cc:2889] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:19:48.542339 18433 raft_consensus.cc:515] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:48.602686 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a96373bb0e14dce8a266b2fc6c58507" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:19:48.592111 18013 raft_consensus.cc:493] T 5a96373bb0e14dce8a266b2fc6c58507 P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:48.608870 16220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d16dfad0c3b540e5b4b86492ddf87208" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:19:48.597746 18145 raft_consensus.cc:493] T ee9ed67006144797b1de6ea8c2013a68 P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:48.600396 18346 raft_consensus.cc:697] T b87147731a68400e8cce9b879daa7c5d P 90f3da57796b488c9087a2f523adf045 [term 6 LEADER]: Becoming Leader. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
W20260709 13:19:48.621732 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 88145.4us
I20260709 13:19:48.632330 18272 raft_consensus.cc:493] T b6f07842d3aa47858df5665874c86f3a P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:48.645640 16789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 62664.2us
I20260709 13:19:48.622731 16191 leader_election.cc:304] T ecc1e363230043fe9fef390f0b979f20 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:19:48.616109 18388 raft_consensus.cc:493] T 9e65676b21b748728ad39668cfe722ec P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:48.658241 16789 leader_election.cc:304] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 647a343a542d4a139fe106c19095129b
I20260709 13:19:48.615406 18401 leader_election.cc:290] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:19:48.620211 18352 raft_consensus.cc:493] T 84f2da98864b4425bf7c0055951fdd4b P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:48.619838 18041 raft_consensus.cc:493] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:48.604393 18432 raft_consensus.cc:515] T 5a959dbecc2648eb85d1cc550f5a3245 P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:19:48.668155 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 45542.7us
I20260709 13:19:48.668880 16191 leader_election.cc:304] T 18ad7e3af5504571a69d10c00750a01c P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 90f3da57796b488c9087a2f523adf045
I20260709 13:19:48.647760 16292 raft_consensus.cc:1240] T 68a6838d82ef4cac83507f906139d90f P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 5. Current term is 6. Ops: []
I20260709 13:19:48.633040 18473 raft_consensus.cc:2804] T 3884f864a96d428785895e5463e5728f P cb82cc7bd6db46d980c75237a1402860 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:19:48.665360 16300 raft_consensus.cc:1275] T 66ae176961b74813a58bc15c9c4c6f2a P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:19:48.677016 16149 raft_consensus.cc:1275] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 6. (term mismatch)
I20260709 13:19:48.614699 18013 raft_consensus.cc:515] T 5a96373bb0e14dce8a266b2fc6c58507 P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:48.678617 16149 pending_rounds.cc:85] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b: Aborting all ops after (but not including) 5
I20260709 13:19:48.683192 16149 pending_rounds.cc:107] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b: Aborting uncommitted NO_OP operation due to leader change: 3.6
I20260709 13:19:48.683466 16149 raft_consensus.cc:2889] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:19:48.685895 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fffab26427e24d398e2695aad40ca134" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:48.686748 16300 raft_consensus.cc:2468] T fffab26427e24d398e2695aad40ca134 P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 3.
I20260709 13:19:48.688548 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4f65cdf00b748bb963ee6e81ef43b79" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:48.669399 18254 raft_consensus.cc:2749] T ecc1e363230043fe9fef390f0b979f20 P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:19:48.669449 18401 raft_consensus.cc:2749] T 8c11074579b046d99e585c2818e18a2c P cb82cc7bd6db46d980c75237a1402860 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:19:48.696767 16150 raft_consensus.cc:2468] T 5a96373bb0e14dce8a266b2fc6c58507 P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 3.
I20260709 13:19:48.657569 18123 raft_consensus.cc:697] T 7c2095a5f4ad49408dbc7f69c606120d P 90f3da57796b488c9087a2f523adf045 [term 4 LEADER]: Becoming Leader. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:19:48.698261 16265 leader_election.cc:304] T c9944d5e4b8c4109af9955c4f08c9095 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:19:48.653893 18272 raft_consensus.cc:515] T b6f07842d3aa47858df5665874c86f3a P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:48.670511 18041 raft_consensus.cc:515] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:48.724277 18272 leader_election.cc:290] T b6f07842d3aa47858df5665874c86f3a P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:48.678049 18473 raft_consensus.cc:493] T 3884f864a96d428785895e5463e5728f P cb82cc7bd6db46d980c75237a1402860 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:48.724848 18473 raft_consensus.cc:3060] T 3884f864a96d428785895e5463e5728f P cb82cc7bd6db46d980c75237a1402860 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:19:48.724895 18485 raft_consensus.cc:2804] T c9944d5e4b8c4109af9955c4f08c9095 P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:19:48.725564 18485 raft_consensus.cc:493] T c9944d5e4b8c4109af9955c4f08c9095 P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Starting leader election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:19:48.725939 18485 raft_consensus.cc:3060] T c9944d5e4b8c4109af9955c4f08c9095 P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:19:48.704645 16294 raft_consensus.cc:2368] T 5a959dbecc2648eb85d1cc550f5a3245 P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b for earlier term 3. Current term is 4.
I20260709 13:19:48.669571 18352 raft_consensus.cc:515] T 84f2da98864b4425bf7c0055951fdd4b P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:48.736686 16223 raft_consensus.cc:1275] T b6f07842d3aa47858df5665874c86f3a P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:19:48.737006 18352 leader_election.cc:290] T 84f2da98864b4425bf7c0055951fdd4b P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:19:48.737624 18347 raft_consensus.cc:515] T 8cc959fb7f704fba9009c0595467d074 P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:48.657788 18460 consensus_queue.cc:1059] T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.031s
I20260709 13:19:48.673282 18145 raft_consensus.cc:515] T ee9ed67006144797b1de6ea8c2013a68 P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:48.712474 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f5f89186204242bf377f7c42b2e2da" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 7 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:48.718647 16148 raft_consensus.cc:2468] T 9c61a39fd4f84424b129c1a6f9ba404d P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 5.
I20260709 13:19:48.706032 18013 leader_election.cc:290] T 5a96373bb0e14dce8a266b2fc6c58507 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:48.665841 18388 raft_consensus.cc:515] T 9e65676b21b748728ad39668cfe722ec P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:48.687480 16219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a96373bb0e14dce8a266b2fc6c58507" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
W20260709 13:19:48.730517 16117 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:48.745788 16117 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:48.732034 18484 log.cc:927] Time spent T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 0.055s	user 0.000s	sys 0.001s
I20260709 13:19:48.732522 18041 leader_election.cc:290] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:19:48.733093 16292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:48.737905 16115 leader_election.cc:304] T 02838c86531d4f98bf58d1c22b8baf77 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:19:48.677107 18463 consensus_queue.cc:1059] T 23f82224f4564fea93572e8206bad1fe P 647a343a542d4a139fe106c19095129b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.034s
I20260709 13:19:48.698489 18454 raft_consensus.cc:2804] T 18ad7e3af5504571a69d10c00750a01c P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:19:48.710143 16191 leader_election.cc:304] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 90f3da57796b488c9087a2f523adf045
I20260709 13:19:48.743722 16300 raft_consensus.cc:2468] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 6.
I20260709 13:19:48.764580 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6f07842d3aa47858df5665874c86f3a" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:48.687867 18372 consensus_queue.cc:1048] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.052s
I20260709 13:19:48.765817 16190 leader_election.cc:304] T c2f5f89186204242bf377f7c42b2e2da P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:19:48.747123 16265 leader_election.cc:304] T 1df93779209748b9860f6d893dd3de53 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:19:48.755626 16148 raft_consensus.cc:1240] T 7c2095a5f4ad49408dbc7f69c606120d P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 3. Current term is 4. Ops: []
I20260709 13:19:48.767573 18489 raft_consensus.cc:493] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:48.768014 16148 raft_consensus.cc:1275] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:19:48.768131 18489 raft_consensus.cc:515] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } }
I20260709 13:19:48.769570 16148 raft_consensus.cc:1275] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 4 index: 4. (term mismatch)
I20260709 13:19:48.769918 16148 pending_rounds.cc:85] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b: Aborting all ops after (but not including) 3
I20260709 13:19:48.770146 16148 pending_rounds.cc:107] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b: Aborting uncommitted CHANGE_CONFIG_OP operation due to leader change: 3.4
W20260709 13:19:48.754693 17992 consensus_meta.cc:287] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860: Time spent flushing consensus metadata: real 0.641s	user 0.003s	sys 0.007s
I20260709 13:19:48.742542 16218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4f65cdf00b748bb963ee6e81ef43b79" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:48.719491 18346 consensus_queue.cc:237] T b87147731a68400e8cce9b879daa7c5d P 90f3da57796b488c9087a2f523adf045 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:48.770483 16148 raft_consensus.cc:2917] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Aborting config change with OpId 3.4: Aborted: Op aborted by new leader
I20260709 13:19:49.336423 18453 raft_consensus.cc:3055] T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:19:49.034706 18450 raft_consensus.cc:493] T 746aabf856ab401c9ef5e859371e7b67 P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:19:49.371827 18453 raft_consensus.cc:740] T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
W20260709 13:19:49.372114 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 662050us
W20260709 13:19:49.158344 18276 consensus_meta.cc:287] T 3884f864a96d428785895e5463e5728f P 90f3da57796b488c9087a2f523adf045: Time spent flushing consensus metadata: real 0.699s	user 0.006s	sys 0.002s
I20260709 13:19:48.770789 18457 raft_consensus.cc:493] T ee9ed67006144797b1de6ea8c2013a68 P cb82cc7bd6db46d980c75237a1402860 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:49.289247 16222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62fe3346cd3c42c4a55e14af9073bca4" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:48.766059 16219 raft_consensus.cc:2468] T 5a96373bb0e14dce8a266b2fc6c58507 P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 3.
I20260709 13:19:49.378484 16222 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33050 (request call id 434) took 1670 ms. Trace:
I20260709 13:19:49.378964 16222 rpcz_store.cc:276] 0709 13:19:47.708482 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:47.728538 (+ 20056us) service_pool.cc:224] Handling call
0709 13:19:49.289048 (+1560510us) spinlock_profiling.cc:243] Waited 5.83 ms on lock 0x7b5800010b98. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69ad8fadb1 00007f69ada3ab71 00007f69ada3aa22 00007f69b6953c09 00007f69b5aadee0 00007f69b5b770ab 00007f69b5b77131 00007f69b5b250a0 00007f69b5b24984 00007f69af09f6d4 00007f69af09f662
0709 13:19:49.378415 (+ 89367us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":13411200}
I20260709 13:19:49.373584 18457 raft_consensus.cc:515] T ee9ed67006144797b1de6ea8c2013a68 P cb82cc7bd6db46d980c75237a1402860 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:48.824353 18390 leader_election.cc:290] T ecc1e363230043fe9fef390f0b979f20 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:19:48.930164 16149 raft_consensus.cc:1275] T 8d4ab870a05347868956aa8b841b2981 P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 5 index: 2. (term mismatch)
I20260709 13:19:49.292167 18401 raft_consensus.cc:493] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:19:49.380796 16149 pending_rounds.cc:85] T 8d4ab870a05347868956aa8b841b2981 P 647a343a542d4a139fe106c19095129b: Aborting all ops after (but not including) 1
W20260709 13:19:48.829581 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 82582.1us
W20260709 13:19:49.193778 18424 consensus_meta.cc:287] T e5ad4eb281564936a59f41e387a47156 P 647a343a542d4a139fe106c19095129b: Time spent flushing consensus metadata: real 0.758s	user 0.009s	sys 0.002s
W20260709 13:19:48.771198 18483 log.cc:927] Time spent T 829b4ad8c5a54eababeea1181b960d4b P cb82cc7bd6db46d980c75237a1402860: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 13:19:48.763238 18460 consensus_queue.cc:1243] T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Peer 90f3da57796b488c9087a2f523adf045 log is divergent from this leader: its last log entry 4.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 13:19:49.039798 18140 raft_consensus.cc:3055] T 23f82224f4564fea93572e8206bad1fe P 647a343a542d4a139fe106c19095129b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:19:49.374847 16218 raft_consensus.cc:2468] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 6.
I20260709 13:19:48.925617 18456 raft_consensus.cc:2804] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:19:49.369933 18145 leader_election.cc:290] T ee9ed67006144797b1de6ea8c2013a68 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
W20260709 13:19:49.369866 16115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 632008us
I20260709 13:19:49.375497 17992 raft_consensus.cc:515] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:48.843709 18041 raft_consensus.cc:2804] T 1df93779209748b9860f6d893dd3de53 P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:19:48.772236 18013 raft_consensus.cc:3060] T 02838c86531d4f98bf58d1c22b8baf77 P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:19:49.372306 18450 raft_consensus.cc:515] T 746aabf856ab401c9ef5e859371e7b67 P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:19:48.927775 16297 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40474 (request call id 332) took 1451 ms (client timeout 1806 ms). Trace:
W20260709 13:19:49.398532 16297 rpcz_store.cc:269] 0709 13:19:47.475878 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:47.480525 (+  4647us) service_pool.cc:224] Handling call
0709 13:19:48.927495 (+1446970us) spinlock_profiling.cc:243] Waited 181 ms on lock 0x7b540024ca54. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:19:48.927661 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":416887168}
W20260709 13:19:48.832227 16266 leader_election.cc:336] T 57a744f5aaf944678a63469cbf514015 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923): Timed out: RequestConsensusVote RPC to 127.14.177.132:38923 timed out after 1.803s (SENT)
I20260709 13:19:48.844704 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a96373bb0e14dce8a266b2fc6c58507" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:49.400005 16300 raft_consensus.cc:2410] T 5a96373bb0e14dce8a266b2fc6c58507 P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b for term 4 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:19:48.928313 18255 raft_consensus.cc:493] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Starting leader election (detected failure of leader 647a343a542d4a139fe106c19095129b)
W20260709 13:19:48.770534 18477 log.cc:927] Time spent T af57996a8be84d398f0f023e8d08ef77 P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 0.223s	user 0.000s	sys 0.001s
I20260709 13:19:49.030733 18109 raft_consensus.cc:493] T 66f75124215f4ca6bef831e174a792d8 P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
W20260709 13:19:49.343648 18245 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.514s	user 0.010s	sys 0.017s
W20260709 13:19:49.160831 18447 log.cc:927] Time spent T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860: Append to log took a long time: real 1.476s	user 0.000s	sys 0.006s
I20260709 13:19:49.411697 16822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5d59407eb545b895d8d922e0dd69fd" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:19:48.761476 18352 consensus_queue.cc:1048] T a378d7f796f94615b3b347119a9880fd P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.618s
I20260709 13:19:48.856961 18454 raft_consensus.cc:697] T 18ad7e3af5504571a69d10c00750a01c P 1ca2725d290e47d68b1a4b609c9816ac [term 4 LEADER]: Becoming Leader. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:19:49.381076 18433 leader_election.cc:290] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 3 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:19:49.104206 18387 raft_consensus.cc:493] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
W20260709 13:19:49.182749 16117 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:49.418915 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74ca8140f35c4ae29e8a664f6bdfc829" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:19:49.424775 16300 raft_consensus.cc:3060] T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:19:49.105609 16144 raft_consensus.cc:1275] T b6f07842d3aa47858df5665874c86f3a P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:19:49.381175 16149 pending_rounds.cc:107] T 8d4ab870a05347868956aa8b841b2981 P 647a343a542d4a139fe106c19095129b: Aborting uncommitted NO_OP operation due to leader change: 4.2
I20260709 13:19:49.425722 16149 raft_consensus.cc:2889] T 8d4ab870a05347868956aa8b841b2981 P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:19:49.370774 16789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 712549us
W20260709 13:19:49.167400 16295 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.500s	user 0.007s	sys 0.001s
I20260709 13:19:49.374658 18323 leader_election.cc:290] T e5ad4eb281564936a59f41e387a47156 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:19:49.427110 16822 raft_consensus.cc:2468] T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 5.
W20260709 13:19:49.411664 18245 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.579s	user 0.010s	sys 0.018s
I20260709 13:19:49.374007 16191 leader_election.cc:304] T ee9ed67006144797b1de6ea8c2013a68 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: cb82cc7bd6db46d980c75237a1402860
I20260709 13:19:49.106897 16145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1df93779209748b9860f6d893dd3de53" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
W20260709 13:19:49.370532 16114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:49.381183 18401 raft_consensus.cc:515] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:49.385064 16148 raft_consensus.cc:2889] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: CHANGE_CONFIG_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:19:49.390235 16296 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.598s	user 0.004s	sys 0.006s
W20260709 13:19:49.467567 16296 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.702s	user 0.004s	sys 0.007s
I20260709 13:19:48.935693 16293 raft_consensus.cc:1240] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 2. Current term is 3. Ops: []
W20260709 13:19:49.155411 18400 consensus_meta.cc:287] T 3a7ca61592e848ea9b0fb81a2ce33cdc P cb82cc7bd6db46d980c75237a1402860: Time spent flushing consensus metadata: real 0.636s	user 0.005s	sys 0.006s
I20260709 13:19:49.038115 18389 raft_consensus.cc:3055] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:19:49.395744 16115 leader_election.cc:304] T c0eab25ce4c64e018a041beeda2867db P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac
I20260709 13:19:49.443549 18421 raft_consensus.cc:493] T 8d4ab870a05347868956aa8b841b2981 P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:49.474254 16115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 78575.9us
W20260709 13:19:48.926468 18474 log.cc:927] Time spent T 87ac0d1f7e8a419f9bcbc04aee0e067e P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 0.280s	user 0.001s	sys 0.000s
W20260709 13:19:48.883503 18482 log.cc:927] Time spent T 57a744f5aaf944678a63469cbf514015 P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.107s	user 0.000s	sys 0.000s
I20260709 13:19:48.859388 18426 raft_consensus.cc:493] T 2794af5b99984efebdee564035841cb0 P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:49.103693 18388 leader_election.cc:290] T 9e65676b21b748728ad39668cfe722ec P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:19:48.840224 18256 raft_consensus.cc:2804] T c2f5f89186204242bf377f7c42b2e2da P 1ca2725d290e47d68b1a4b609c9816ac [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:19:48.925597 16217 raft_consensus.cc:1275] T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 5 index: 2. (index mismatch)
I20260709 13:19:49.517468 16296 raft_consensus.cc:1240] T 02838c86531d4f98bf58d1c22b8baf77 P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 4. Current term is 5. Ops: []
I20260709 13:19:48.856715 18313 raft_consensus.cc:2804] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:19:49.384402 18487 raft_consensus.cc:493] T 2794af5b99984efebdee564035841cb0 P cb82cc7bd6db46d980c75237a1402860 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:49.406749 16266 leader_election.cc:304] T 57a744f5aaf944678a63469cbf514015 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
I20260709 13:19:48.927290 18432 leader_election.cc:290] T 5a959dbecc2648eb85d1cc550f5a3245 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 4 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:19:48.884025 18488 raft_consensus.cc:493] T 18ad7e3af5504571a69d10c00750a01c P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:19:48.838891 16292 raft_consensus.cc:2468] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 5.
I20260709 13:19:49.068204 18347 leader_election.cc:290] T 8cc959fb7f704fba9009c0595467d074 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 5 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:19:49.396322 18041 raft_consensus.cc:493] T 1df93779209748b9860f6d893dd3de53 P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:49.523862 18041 raft_consensus.cc:3060] T 1df93779209748b9860f6d893dd3de53 P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:19:49.523469 18488 raft_consensus.cc:515] T 18ad7e3af5504571a69d10c00750a01c P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } }
I20260709 13:19:49.525840 18488 leader_election.cc:290] T 18ad7e3af5504571a69d10c00750a01c P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:19:49.526628 16146 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46224 (request call id 293) took 1832 ms. Trace:
I20260709 13:19:49.527026 16146 rpcz_store.cc:276] 0709 13:19:47.694318 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:47.700310 (+  5992us) service_pool.cc:224] Handling call
0709 13:19:47.725302 (+ 24992us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:47.748221 (+ 22919us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:47.748269 (+    48us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:48.400507 (+652238us) spinlock_profiling.cc:243] Waited 5.59 ms on lock 0x7b2c00000658. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69af458a11 00007f69af468372 00007f69b0b9aa01 00007f69b0b5f43a 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2
0709 13:19:48.401018 (+   511us) log.cc:844] Serialized 26 byte log entry
0709 13:19:48.447493 (+ 46475us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:19:48.447527 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:48.447592 (+    65us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:49.521253 (+1073661us) raft_consensus.cc:1685] finished
0709 13:19:49.521285 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:49.526538 (+  5253us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6659,"raft.queue_time_us":12421,"raft.run_cpu_time_us":192,"raft.run_wall_time_us":23880,"spinlock_wait_cycles":12855680,"thread_start_us":45964,"threads_started":1,"wal-append.queue_time_us":60518}
I20260709 13:19:49.388586 18140 raft_consensus.cc:740] T 23f82224f4564fea93572e8206bad1fe P 647a343a542d4a139fe106c19095129b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:19:49.393970 18460 raft_consensus.cc:493] T b87147731a68400e8cce9b879daa7c5d P cb82cc7bd6db46d980c75237a1402860 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:49.373202 18276 raft_consensus.cc:515] T 3884f864a96d428785895e5463e5728f P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:19:49.426558 16295 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.759s	user 0.007s	sys 0.005s
I20260709 13:19:49.098241 16220 raft_consensus.cc:1275] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:19:49.382555 16265 leader_election.cc:304] T 68a6838d82ef4cac83507f906139d90f P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:19:49.407205 18255 raft_consensus.cc:3060] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:19:49.430080 16820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74ca8140f35c4ae29e8a664f6bdfc829" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:19:49.537752 16820 raft_consensus.cc:3060] T 74ca8140f35c4ae29e8a664f6bdfc829 P cb82cc7bd6db46d980c75237a1402860 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:19:49.431351 16192 leader_election.cc:304] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 647a343a542d4a139fe106c19095129b
I20260709 13:19:48.982091 18473 raft_consensus.cc:515] T 3884f864a96d428785895e5463e5728f P cb82cc7bd6db46d980c75237a1402860 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:49.441526 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746aabf856ab401c9ef5e859371e7b67" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:19:49.404464 18456 raft_consensus.cc:697] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860 [term 5 LEADER]: Becoming Leader. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:19:49.443923 16145 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40830 (request call id 1088) took 1136 ms. Trace:
I20260709 13:19:48.915892 18123 consensus_queue.cc:237] T 7c2095a5f4ad49408dbc7f69c606120d P 90f3da57796b488c9087a2f523adf045 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:49.442842 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cc959fb7f704fba9009c0595467d074" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:49.403879 18463 consensus_queue.cc:1048] T 84f2da98864b4425bf7c0055951fdd4b P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.028s
I20260709 13:19:48.463049 15046 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:19:49.407055 18145 consensus_queue.cc:1059] T d16dfad0c3b540e5b4b86492ddf87208 P 647a343a542d4a139fe106c19095129b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.032s
I20260709 13:19:49.427265 18387 raft_consensus.cc:3060] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:19:49.418069 18489 leader_election.cc:290] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:19:49.429351 16789 leader_election.cc:304] T 74ca8140f35c4ae29e8a664f6bdfc829 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac
I20260709 13:19:49.428653 18496 raft_consensus.cc:493] T d2ab0ac8347641e6955e9bce7d90c1a2 P cb82cc7bd6db46d980c75237a1402860 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:49.439666 18406 raft_consensus.cc:493] T b6f07842d3aa47858df5665874c86f3a P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:49.545892 18406 raft_consensus.cc:515] T b6f07842d3aa47858df5665874c86f3a P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:49.547003 18276 leader_election.cc:290] T 3884f864a96d428785895e5463e5728f P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 4 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:19:49.547855 16298 raft_consensus.cc:1240] T 3884f864a96d428785895e5463e5728f P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 3. Current term is 4. Ops: [3.6-3.6]
I20260709 13:19:49.548247 18406 leader_election.cc:290] T b6f07842d3aa47858df5665874c86f3a P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:49.539350 18487 raft_consensus.cc:515] T 2794af5b99984efebdee564035841cb0 P cb82cc7bd6db46d980c75237a1402860 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:49.550199 18406 raft_consensus.cc:493] T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:19:49.550571 18487 leader_election.cc:290] T 2794af5b99984efebdee564035841cb0 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:49.424063 18109 raft_consensus.cc:515] T 66f75124215f4ca6bef831e174a792d8 P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:49.551561 18456 consensus_queue.cc:237] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:49.552657 18109 leader_election.cc:290] T 66f75124215f4ca6bef831e174a792d8 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:19:49.553154 18109 raft_consensus.cc:493] T 8cc959fb7f704fba9009c0595467d074 P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:49.553553 18109 raft_consensus.cc:515] T 8cc959fb7f704fba9009c0595467d074 P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:49.538285 18457 leader_election.cc:290] T ee9ed67006144797b1de6ea8c2013a68 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:19:49.555548 18450 leader_election.cc:290] T 746aabf856ab401c9ef5e859371e7b67 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:19:49.468954 18433 raft_consensus.cc:493] T 8638ee363273426db1393dd81e5c7b72 P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:19:49.461974 18372 raft_consensus.cc:493] T d16dfad0c3b540e5b4b86492ddf87208 P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:19:49.473723 18389 raft_consensus.cc:740] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:19:49.477639 18057 raft_consensus.cc:2804] T c0eab25ce4c64e018a041beeda2867db P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:19:49.480716 18431 raft_consensus.cc:493] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:49.494498 16816 raft_consensus.cc:1240] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [term 7 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 6. Current term is 7. Ops: [6.7-6.7]
I20260709 13:19:49.480569 18256 raft_consensus.cc:493] T c2f5f89186204242bf377f7c42b2e2da P 1ca2725d290e47d68b1a4b609c9816ac [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:49.567272 18256 raft_consensus.cc:3060] T c2f5f89186204242bf377f7c42b2e2da P 1ca2725d290e47d68b1a4b609c9816ac [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:19:49.531857 16266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 699630us
I20260709 13:19:49.538494 16819 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49906 (request call id 515) took 2133 ms. Trace:
W20260709 13:19:49.570667 16266 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:49.535784 18313 raft_consensus.cc:697] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac [term 5 LEADER]: Becoming Leader. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:19:49.540385 16296 raft_consensus.cc:1240] T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 4. Current term is 5. Ops: []
I20260709 13:19:49.539532 16145 rpcz_store.cc:276] 0709 13:19:48.307118 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:48.311675 (+  4557us) service_pool.cc:224] Handling call
0709 13:19:49.106719 (+795044us) spinlock_profiling.cc:243] Waited 1.32 ms on lock 0x7b5800011198. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69ad8fadb1 00007f69ada3ab71 00007f69ada3aa22 00007f69b6953c09 00007f69b5aadee0 00007f69b5b770ab 00007f69b5b77131 00007f69b5b250a0 00007f69b5b24984 00007f69af09f6d4 00007f69af09f662
0709 13:19:49.443841 (+337122us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3042048}
I20260709 13:19:49.542124 16297 raft_consensus.cc:2393] T 8cc959fb7f704fba9009c0595467d074 P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 5: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
I20260709 13:19:49.535319 18140 consensus_queue.cc:260] T 23f82224f4564fea93572e8206bad1fe P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:19:49.542609 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 160118us
I20260709 13:19:49.545360 16299 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48674 (request call id 2091) took 1031 ms. Trace:
I20260709 13:19:49.574805 16299 rpcz_store.cc:276] 0709 13:19:48.513927 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:48.516318 (+  2391us) service_pool.cc:224] Handling call
0709 13:19:48.516669 (+   351us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:48.517157 (+   488us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:48.517197 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:48.517805 (+   608us) log.cc:844] Serialized 26 byte log entry
0709 13:19:48.602257 (+ 84452us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:19:48.602310 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:48.602374 (+    64us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:49.545149 (+942775us) raft_consensus.cc:1685] finished
0709 13:19:49.545168 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:49.545303 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":798,"raft.run_cpu_time_us":140,"raft.run_wall_time_us":140,"thread_start_us":84011,"threads_started":1,"wal-append.queue_time_us":135202}
I20260709 13:19:49.532225 18472 raft_consensus.cc:2749] T 57a744f5aaf944678a63469cbf514015 P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:19:49.478451 18421 raft_consensus.cc:515] T 8d4ab870a05347868956aa8b841b2981 P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:49.550690 18406 raft_consensus.cc:515] T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:49.564291 18389 consensus_queue.cc:260] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:49.542521 18013 raft_consensus.cc:2749] T 02838c86531d4f98bf58d1c22b8baf77 P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:19:49.574379 16265 leader_election.cc:304] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:19:49.565297 18057 raft_consensus.cc:493] T c0eab25ce4c64e018a041beeda2867db P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:49.570108 18450 raft_consensus.cc:2804] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 13:19:49.580060 16789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 150824us
I20260709 13:19:49.571676 18114 raft_consensus.cc:2804] T ee9ed67006144797b1de6ea8c2013a68 P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:19:49.556193 18347 raft_consensus.cc:2749] T 68a6838d82ef4cac83507f906139d90f P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:19:49.617368 16265 leader_election.cc:304] T d16dfad0c3b540e5b4b86492ddf87208 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:19:49.565433 17992 leader_election.cc:290] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 7 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:19:49.544163 18424 consensus_queue.cc:1048] T 1df93779209748b9860f6d893dd3de53 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.040s
I20260709 13:19:49.570577 18488 raft_consensus.cc:493] T 8a878999c0024d2abe517484180290fc P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:49.578644 18493 log.cc:927] Time spent T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.113s	user 0.000s	sys 0.000s
I20260709 13:19:49.582859 18401 leader_election.cc:290] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:19:49.516139 18426 raft_consensus.cc:515] T 2794af5b99984efebdee564035841cb0 P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:49.638715 16296 raft_consensus.cc:1240] T 7c2095a5f4ad49408dbc7f69c606120d P 90f3da57796b488c9087a2f523adf045 [term 4 LEADER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 13:19:49.644738 18245 raft_consensus.cc:493] T 9e65676b21b748728ad39668cfe722ec P cb82cc7bd6db46d980c75237a1402860 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:19:49.645816 16790 leader_election.cc:304] T 66f75124215f4ca6bef831e174a792d8 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
W20260709 13:19:49.650170 16192 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 218936us
I20260709 13:19:49.660324 18255 raft_consensus.cc:515] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:49.664541 16265 leader_election.cc:304] T 5a96373bb0e14dce8a266b2fc6c58507 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:19:49.665997 16789 leader_election.cc:304] T 26354361e9b044cea1c2b86c4882fe92 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
W20260709 13:19:49.666604 16791 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:49.576881 18372 raft_consensus.cc:515] T d16dfad0c3b540e5b4b86492ddf87208 P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:49.674329 16221 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57604 (request call id 2011) took 1812 ms. Trace:
I20260709 13:19:49.607316 18432 raft_consensus.cc:493] T 66ae176961b74813a58bc15c9c4c6f2a P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:49.674808 18504 consensus_queue.cc:1059] T 02838c86531d4f98bf58d1c22b8baf77 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.005s
I20260709 13:19:49.676918 18140 raft_consensus.cc:3060] T 23f82224f4564fea93572e8206bad1fe P 647a343a542d4a139fe106c19095129b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:19:49.674396 18109 leader_election.cc:290] T 8cc959fb7f704fba9009c0595467d074 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:49.577456 18454 consensus_queue.cc:237] T 18ad7e3af5504571a69d10c00750a01c P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } }
I20260709 13:19:49.724705 16294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87ac0d1f7e8a419f9bcbc04aee0e067e" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:19:49.541721 18460 raft_consensus.cc:515] T b87147731a68400e8cce9b879daa7c5d P cb82cc7bd6db46d980c75237a1402860 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:49.583424 18489 raft_consensus.cc:493] T ada5a004ba7c46819051e3ccec71ffdb P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
W20260709 13:19:49.685945 16115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 210909us
W20260709 13:19:49.736479 16789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 70611.5us
I20260709 13:19:49.688527 16226 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57604 (request call id 2002) took 2195 ms. Trace:
I20260709 13:19:49.565270 18433 raft_consensus.cc:515] T 8638ee363273426db1393dd81e5c7b72 P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:49.627739 18057 raft_consensus.cc:3060] T c0eab25ce4c64e018a041beeda2867db P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:19:49.749709 16222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cc959fb7f704fba9009c0595467d074" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:49.615545 18400 raft_consensus.cc:493] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:19:49.621294 18276 raft_consensus.cc:2749] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:19:49.625459 18450 raft_consensus.cc:493] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:49.703099 16265 leader_election.cc:304] T c4f65cdf00b748bb963ee6e81ef43b79 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 647a343a542d4a139fe106c19095129b
W20260709 13:19:49.705152 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 331211us
I20260709 13:19:49.624243 18272 consensus_queue.cc:1059] T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: INVALID_TERM, Last received: 4.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.067s
I20260709 13:19:49.672663 18352 raft_consensus.cc:2749] T d16dfad0c3b540e5b4b86492ddf87208 P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:19:49.575073 18431 raft_consensus.cc:515] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } }
I20260709 13:19:49.708694 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cc959fb7f704fba9009c0595467d074" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:49.756825 16145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee9ed67006144797b1de6ea8c2013a68" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:19:49.683514 18041 raft_consensus.cc:515] T 1df93779209748b9860f6d893dd3de53 P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:49.710007 16217 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57600 (request call id 1069) took 1973 ms. Trace:
I20260709 13:19:49.711956 16292 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48674 (request call id 2102) took 1003 ms. Trace:
I20260709 13:19:49.714627 16293 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48674 (request call id 2098) took 1119 ms. Trace:
I20260709 13:19:49.689580 18389 raft_consensus.cc:3060] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:19:49.715230 18123 raft_consensus.cc:2804] T 5a96373bb0e14dce8a266b2fc6c58507 P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:19:49.716188 18459 raft_consensus.cc:3055] T af57996a8be84d398f0f023e8d08ef77 P cb82cc7bd6db46d980c75237a1402860 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:19:49.562482 18496 raft_consensus.cc:515] T d2ab0ac8347641e6955e9bce7d90c1a2 P cb82cc7bd6db46d980c75237a1402860 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:49.716904 18255 leader_election.cc:290] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 5 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:49.718425 16291 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48674 (request call id 2095) took 1203 ms. Trace:
I20260709 13:19:49.718950 18421 leader_election.cc:290] T 8d4ab870a05347868956aa8b841b2981 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:49.720582 18313 consensus_queue.cc:237] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:49.722385 18406 leader_election.cc:290] T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
W20260709 13:19:49.695822 18485 consensus_meta.cc:287] T c9944d5e4b8c4109af9955c4f08c9095 P 90f3da57796b488c9087a2f523adf045: Time spent flushing consensus metadata: real 0.954s	user 0.009s	sys 0.003s
I20260709 13:19:49.733515 16820 raft_consensus.cc:2468] T 74ca8140f35c4ae29e8a664f6bdfc829 P cb82cc7bd6db46d980c75237a1402860 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 5.
I20260709 13:19:49.685531 16295 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48674 (request call id 2094) took 1170 ms. Trace:
I20260709 13:19:49.746768 18463 raft_consensus.cc:493] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:49.747133 18504 raft_consensus.cc:3055] T 02838c86531d4f98bf58d1c22b8baf77 P 1ca2725d290e47d68b1a4b609c9816ac [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:19:49.752661 18489 raft_consensus.cc:515] T ada5a004ba7c46819051e3ccec71ffdb P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:49.759333 18013 consensus_queue.cc:1059] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 647a343a542d4a139fe106c19095129b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.017s
I20260709 13:19:49.748085 18433 leader_election.cc:290] T 8638ee363273426db1393dd81e5c7b72 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:19:49.757351 18511 consensus_queue.cc:1059] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.030s
W20260709 13:19:49.750741 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 105044us
I20260709 13:19:49.701195 18347 raft_consensus.cc:493] T 31969392ccdb43b491911a319e8bc7e1 P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:49.752306 16789 leader_election.cc:304] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 647a343a542d4a139fe106c19095129b
W20260709 13:19:49.753767 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 50754.8us
I20260709 13:19:49.750422 16222 raft_consensus.cc:2468] T 8cc959fb7f704fba9009c0595467d074 P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 4.
I20260709 13:19:49.754448 16115 leader_election.cc:304] T fffab26427e24d398e2695aad40ca134 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac
I20260709 13:19:49.683820 18472 consensus_queue.cc:1048] T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.055s
I20260709 13:19:49.704850 18518 raft_consensus.cc:493] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:49.686556 18488 raft_consensus.cc:515] T 8a878999c0024d2abe517484180290fc P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:49.766825 18513 consensus_queue.cc:1048] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.069s
I20260709 13:19:49.742580 18401 raft_consensus.cc:2749] T 66f75124215f4ca6bef831e174a792d8 P cb82cc7bd6db46d980c75237a1402860 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:19:49.736181 17992 raft_consensus.cc:2749] T 26354361e9b044cea1c2b86c4882fe92 P cb82cc7bd6db46d980c75237a1402860 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:19:49.761510 16299 raft_consensus.cc:2393] T 8cc959fb7f704fba9009c0595467d074 P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 5: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
I20260709 13:19:49.761201 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62fe3346cd3c42c4a55e14af9073bca4" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:19:49.741395 16226 rpcz_store.cc:276] 0709 13:19:47.493409 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:47.500190 (+  6781us) service_pool.cc:224] Handling call
0709 13:19:48.673406 (+1173216us) spinlock_profiling.cc:243] Waited 1.67 ms on lock 0x7b5800010b98. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69ad8fadb1 00007f69ada3ab71 00007f69ada3aa22 00007f69b6953c09 00007f69b5aadee0 00007f69b5b770ab 00007f69b5b77131 00007f69b5b38fc0 00007f69b5b23929 00007f69b5b233d2 00007f69af09abd4
0709 13:19:48.673596 (+   190us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:19:48.698042 (+ 24446us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:48.698087 (+    45us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:19:48.738735 (+ 40648us) log.cc:844] Serialized 52 byte log entry
0709 13:19:49.673215 (+934480us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:19:49.673273 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:49.673362 (+    89us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:49.673415 (+    53us) raft_consensus.cc:1685] finished
0709 13:19:49.673432 (+    17us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:49.688465 (+ 15033us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1106,"raft.queue_time_us":10361,"raft.run_cpu_time_us":180,"raft.run_wall_time_us":19229,"spinlock_wait_cycles":3852032,"thread_start_us":302954,"threads_started":1,"wal-append.queue_time_us":902941}
I20260709 13:19:49.761087 16823 raft_consensus.cc:3060] T 19baa7bc367844c288bca8f81439c413 P cb82cc7bd6db46d980c75237a1402860 [term 4 LEARNER]: Advancing to term 5
I20260709 13:19:49.765595 16148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5718c623ebee4d2bb770f0f9ddb2f164" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:19:49.701388 18245 raft_consensus.cc:515] T 9e65676b21b748728ad39668cfe722ec P cb82cc7bd6db46d980c75237a1402860 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:49.764611 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a96373bb0e14dce8a266b2fc6c58507" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:49.767642 16146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2794af5b99984efebdee564035841cb0" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:19:49.756671 18323 raft_consensus.cc:2804] T c4f65cdf00b748bb963ee6e81ef43b79 P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:19:49.754925 18450 raft_consensus.cc:3060] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:19:49.770093 18123 raft_consensus.cc:493] T 5a96373bb0e14dce8a266b2fc6c58507 P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:49.782648 16219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:49.926895 18123 raft_consensus.cc:3060] T 5a96373bb0e14dce8a266b2fc6c58507 P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:19:49.927999 16219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d4ab870a05347868956aa8b841b2981" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
W20260709 13:19:49.929250 18507 log.cc:927] Time spent T 3a7ca61592e848ea9b0fb81a2ce33cdc P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 0.159s	user 0.000s	sys 0.001s
I20260709 13:19:49.930540 18323 raft_consensus.cc:493] T c4f65cdf00b748bb963ee6e81ef43b79 P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:19:49.726127 18432 raft_consensus.cc:515] T 66ae176961b74813a58bc15c9c4c6f2a P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:49.930956 18323 raft_consensus.cc:3060] T c4f65cdf00b748bb963ee6e81ef43b79 P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:19:49.765134 16293 rpcz_store.cc:276] 0709 13:19:48.594808 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:48.601705 (+  6897us) service_pool.cc:224] Handling call
0709 13:19:48.935405 (+333700us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:49.471440 (+536035us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:49.714411 (+242971us) spinlock_profiling.cc:243] Waited 281 ms on lock 0x7b540024ca50. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5e8ad 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:19:49.714567 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":646418816}
I20260709 13:19:49.933182 18432 leader_election.cc:290] T 66ae176961b74813a58bc15c9c4c6f2a P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:19:49.756558 18522 raft_consensus.cc:493] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:49.933923 18522 raft_consensus.cc:515] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:49.761528 18452 raft_consensus.cc:3055] T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:19:49.935313 18452 raft_consensus.cc:740] T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:19:49.936324 18452 consensus_queue.cc:260] T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:49.937577 18452 raft_consensus.cc:3060] T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:19:49.761884 16816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66f75124215f4ca6bef831e174a792d8" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:19:49.761936 16217 rpcz_store.cc:276] 0709 13:19:47.736737 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:47.776699 (+ 39962us) service_pool.cc:224] Handling call
0709 13:19:47.777042 (+   343us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:49.520431 (+1743389us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:49.709724 (+189293us) spinlock_profiling.cc:243] Waited 1.15 s on lock 0x7b5400396054. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:19:49.709940 (+   216us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2640091008}
I20260709 13:19:49.940465 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee9ed67006144797b1de6ea8c2013a68" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:49.941447 18245 leader_election.cc:290] T 9e65676b21b748728ad39668cfe722ec P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
W20260709 13:19:49.831399 16117 leader_election.cc:336] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923): Timed out: RequestConsensusVote RPC to 127.14.177.132:38923 timed out after 1.814s (SENT)
I20260709 13:19:49.946082 16117 leader_election.cc:304] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860
I20260709 13:19:49.947098 16148 raft_consensus.cc:2468] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 4.
W20260709 13:19:49.950340 16115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 195952us
I20260709 13:19:49.943204 16146 raft_consensus.cc:2468] T 2794af5b99984efebdee564035841cb0 P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 3.
W20260709 13:19:49.959831 16114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:49.991678 16300 consensus_meta.cc:287] T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045: Time spent flushing consensus metadata: real 0.562s	user 0.010s	sys 0.000s
W20260709 13:19:49.991959 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 47987.1us
I20260709 13:19:49.992146 16300 raft_consensus.cc:2468] T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 5.
I20260709 13:19:49.992772 16300 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48684 (request call id 1983) took 1343 ms. Trace:
I20260709 13:19:49.992918 16790 leader_election.cc:304] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:19:49.993353 16300 rpcz_store.cc:276] 0709 13:19:48.649503 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:49.417670 (+768167us) service_pool.cc:224] Handling call
0709 13:19:49.992715 (+575045us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:49.945159 16224 raft_consensus.cc:2468] T 5a96373bb0e14dce8a266b2fc6c58507 P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 3.
W20260709 13:19:50.004736 16789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 252506us
I20260709 13:19:50.005201 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f2da98864b4425bf7c0055951fdd4b" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:49.770220 18459 raft_consensus.cc:740] T af57996a8be84d398f0f023e8d08ef77 P cb82cc7bd6db46d980c75237a1402860 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
W20260709 13:19:50.016996 16265 connection.cc:256] RPC call timeout handler was delayed by -1.1300113780000629s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1527s for a response
W20260709 13:19:50.020084 16266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 133861us
I20260709 13:19:49.778654 18422 raft_consensus.cc:3055] T d16dfad0c3b540e5b4b86492ddf87208 P 647a343a542d4a139fe106c19095129b [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:19:50.023850 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6f07842d3aa47858df5665874c86f3a" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:49.782577 18487 raft_consensus.cc:2764] T 74ca8140f35c4ae29e8a664f6bdfc829 P cb82cc7bd6db46d980c75237a1402860 [term 5 FOLLOWER]: Leader election decision vote started in defunct term 4: won
I20260709 13:19:49.746371 18114 raft_consensus.cc:697] T ee9ed67006144797b1de6ea8c2013a68 P 1ca2725d290e47d68b1a4b609c9816ac [term 4 LEADER]: Becoming Leader. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:19:49.751350 18387 raft_consensus.cc:515] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:49.761416 18400 raft_consensus.cc:515] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:49.753777 16191 leader_election.cc:304] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:19:49.757457 16145 raft_consensus.cc:2468] T ee9ed67006144797b1de6ea8c2013a68 P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 4.
I20260709 13:19:49.933414 16293 raft_consensus.cc:1275] T cdbd78acb4cf46298e0de8ba0d130e19 P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
W20260709 13:19:50.022270 16117 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 190901us
W20260709 13:19:50.052963 16117 leader_election.cc:336] T ecc1e363230043fe9fef390f0b979f20 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923): Timed out: RequestConsensusVote RPC to 127.14.177.132:38923 timed out after 1.881s (SENT)
I20260709 13:19:49.780634 16295 rpcz_store.cc:276] 0709 13:19:48.514854 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:48.569295 (+ 54441us) service_pool.cc:224] Handling call
0709 13:19:48.583795 (+ 14500us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:48.602601 (+ 18806us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:19:48.667079 (+ 64478us) log.cc:844] Serialized 14 byte log entry
0709 13:19:49.650808 (+983729us) log.cc:844] Serialized 14 byte log entry
0709 13:19:49.685085 (+ 34277us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:49.685154 (+    69us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:19:49.685213 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:49.685301 (+    88us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:49.685454 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":35580,"prepare.queue_time_us":188921,"prepare.run_cpu_time_us":137,"prepare.run_wall_time_us":17528,"raft.queue_time_us":34739,"raft.run_cpu_time_us":223,"raft.run_wall_time_us":8707,"thread_start_us":1028536,"threads_started":3,"wal-append.queue_time_us":786893}
I20260709 13:19:50.025633 16300 raft_consensus.cc:1275] T 8a878999c0024d2abe517484180290fc P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:19:49.759755 18524 raft_consensus.cc:493] T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:50.020872 18388 raft_consensus.cc:493] T 2794af5b99984efebdee564035841cb0 P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:49.893651 18523 raft_consensus.cc:493] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:19:50.023726 18422 raft_consensus.cc:740] T d16dfad0c3b540e5b4b86492ddf87208 P 647a343a542d4a139fe106c19095129b [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:19:50.040866 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a959dbecc2648eb85d1cc550f5a3245" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:19:49.835016 18473 leader_election.cc:290] T 3884f864a96d428785895e5463e5728f P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 4 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
W20260709 13:19:50.042927 16115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 91337.7us
I20260709 13:19:50.004050 18418 raft_consensus.cc:2764] T fffab26427e24d398e2695aad40ca134 P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Leader pre-election decision vote started in defunct term 2: won
I20260709 13:19:49.949664 18504 raft_consensus.cc:740] T 02838c86531d4f98bf58d1c22b8baf77 P 1ca2725d290e47d68b1a4b609c9816ac [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:19:49.830004 18495 raft_consensus.cc:493] T 3a7ca61592e848ea9b0fb81a2ce33cdc P cb82cc7bd6db46d980c75237a1402860 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:49.763267 16292 rpcz_store.cc:276] 0709 13:19:48.708961 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:48.726260 (+ 17299us) service_pool.cc:224] Handling call
0709 13:19:49.711785 (+985525us) spinlock_profiling.cc:243] Waited 11.1 ms on lock 0x7b5400399ed4. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:19:49.711896 (+   111us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":25510784}
I20260709 13:19:50.132475 16295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3884f864a96d428785895e5463e5728f" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:19:50.048558 16190 leader_election.cc:304] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:19:50.045512 16116 leader_election.cc:304] T 5a96373bb0e14dce8a266b2fc6c58507 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
I20260709 13:19:50.021169 18457 raft_consensus.cc:2804] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:19:50.024537 16226 raft_consensus.cc:2468] T b6f07842d3aa47858df5665874c86f3a P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 4.
I20260709 13:19:49.990718 18245 consensus_queue.cc:1048] T 829b4ad8c5a54eababeea1181b960d4b P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.057s
I20260709 13:19:50.049492 18105 consensus_queue.cc:1059] T 68a6838d82ef4cac83507f906139d90f P 647a343a542d4a139fe106c19095129b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 2, Time since last communication: 0.060s
I20260709 13:19:50.053570 16117 leader_election.cc:304] T ecc1e363230043fe9fef390f0b979f20 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860
W20260709 13:19:50.145888 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 100470us
W20260709 13:19:50.059793 16266 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:49.572160 16819 rpcz_store.cc:276] 0709 13:19:47.398260 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:47.398418 (+   158us) service_pool.cc:224] Handling call
0709 13:19:47.431382 (+ 32964us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:47.431894 (+   512us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:47.431936 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:47.432529 (+   593us) log.cc:844] Serialized 26 byte log entry
0709 13:19:47.684255 (+251726us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:19:47.684302 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:47.684384 (+    82us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:49.160099 (+1475715us) spinlock_profiling.cc:243] Waited 16.1 ms on lock 0x7b4400022b4c. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69ad8fadb1 00007f69ad998c85 00007f69ad9996a2 00007f69b070b1db 00007f69b0706aaf 00007f69b06fa86e 00007f69b06f9bb6 00007f69b06f97d8 00007f69b0703221 00007f69b07031d9 00007f69b0703169
0709 13:19:49.510321 (+350222us) raft_consensus.cc:1685] finished
0709 13:19:49.510349 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:49.521993 (+ 11644us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":11732,"raft.run_cpu_time_us":169,"raft.run_wall_time_us":6080,"spinlock_wait_cycles":36985728,"thread_start_us":251276,"threads_started":1,"wal-append.queue_time_us":251641}
I20260709 13:19:50.068964 16818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746aabf856ab401c9ef5e859371e7b67" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
W20260709 13:19:50.023428 16142 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40844 (request call id 1831) took 2078 ms (client timeout 2157 ms). Trace:
I20260709 13:19:49.990888 18421 raft_consensus.cc:493] T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:49.991611 18518 raft_consensus.cc:515] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } }
I20260709 13:19:50.008678 18352 consensus_queue.cc:1048] T 8d4ab870a05347868956aa8b841b2981 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.044s
I20260709 13:19:50.007653 18432 consensus_queue.cc:1048] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.053s
I20260709 13:19:50.043939 18346 raft_consensus.cc:493] T 8158bbd17b994d2eb96f560bd0f1b88d P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:50.079499 16224 raft_consensus.cc:3060] T 5a959dbecc2648eb85d1cc550f5a3245 P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:19:50.046911 18417 raft_consensus.cc:2749] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:19:50.050937 17992 raft_consensus.cc:2804] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:19:50.137015 18460 leader_election.cc:290] T b87147731a68400e8cce9b879daa7c5d P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:49.797008 18485 raft_consensus.cc:515] T c9944d5e4b8c4109af9955c4f08c9095 P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:50.044589 16816 raft_consensus.cc:2468] T 66f75124215f4ca6bef831e174a792d8 P cb82cc7bd6db46d980c75237a1402860 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 5.
I20260709 13:19:50.050546 18426 leader_election.cc:290] T 2794af5b99984efebdee564035841cb0 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:19:50.008401 18450 raft_consensus.cc:515] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:50.052136 18433 raft_consensus.cc:493] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:50.069823 18323 raft_consensus.cc:515] T c4f65cdf00b748bb963ee6e81ef43b79 P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:49.828264 16294 raft_consensus.cc:2393] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 5: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
I20260709 13:19:50.041558 18254 raft_consensus.cc:493] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:19:50.007238 18347 raft_consensus.cc:515] T 31969392ccdb43b491911a319e8bc7e1 P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:49.828590 18463 raft_consensus.cc:515] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:50.078173 18388 raft_consensus.cc:515] T 2794af5b99984efebdee564035841cb0 P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:50.070223 18524 raft_consensus.cc:515] T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:50.230098 16824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f82224f4564fea93572e8206bad1fe" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:19:50.267486 16295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e65676b21b748728ad39668cfe722ec" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:50.369697 16295 raft_consensus.cc:2468] T 9e65676b21b748728ad39668cfe722ec P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 5.
I20260709 13:19:50.245393 16147 raft_consensus.cc:1240] T fffab26427e24d398e2695aad40ca134 P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 2. Current term is 3. Ops: []
I20260709 13:19:50.110737 16293 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48684 (request call id 1985) took 1460 ms. Trace:
I20260709 13:19:49.702378 16221 rpcz_store.cc:276] 0709 13:19:47.863521 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:47.873237 (+  9716us) service_pool.cc:224] Handling call
0709 13:19:47.924427 (+ 51190us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:47.994189 (+ 69762us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:48.001807 (+  7618us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:48.059560 (+ 57753us) log.cc:844] Serialized 26 byte log entry
0709 13:19:48.454343 (+394783us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:19:48.455308 (+   965us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:48.461650 (+  6342us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:49.618164 (+1156514us) raft_consensus.cc:1685] finished
0709 13:19:49.621367 (+  3203us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:49.670081 (+ 48714us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":3008,"raft.run_cpu_time_us":141,"raft.run_wall_time_us":11053,"thread_start_us":259928,"threads_started":1,"wal-append.queue_time_us":454703}
W20260709 13:19:50.346848 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 593117us
W20260709 13:19:50.358109 16117 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 305164us
W20260709 13:19:50.374631 16117 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:50.095659 18523 raft_consensus.cc:515] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:19:50.375262 16117 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:50.181108 18372 leader_election.cc:290] T d16dfad0c3b540e5b4b86492ddf87208 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:50.375919 16191 leader_election.cc:304] T 5a959dbecc2648eb85d1cc550f5a3245 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 90f3da57796b488c9087a2f523adf045
I20260709 13:19:50.377274 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5ad4eb281564936a59f41e387a47156" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:50.135593 18457 raft_consensus.cc:493] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 647a343a542d4a139fe106c19095129b)
W20260709 13:19:50.278401 18256 consensus_meta.cc:287] T c2f5f89186204242bf377f7c42b2e2da P 1ca2725d290e47d68b1a4b609c9816ac: Time spent flushing consensus metadata: real 0.711s	user 0.006s	sys 0.002s
I20260709 13:19:50.205708 18473 raft_consensus.cc:493] T 23f82224f4564fea93572e8206bad1fe P cb82cc7bd6db46d980c75237a1402860 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:19:50.207027 18041 leader_election.cc:290] T 1df93779209748b9860f6d893dd3de53 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 5 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:19:50.405238 16223 raft_consensus.cc:1240] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac [term 6 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 5. Current term is 6. Ops: []
W20260709 13:19:50.425483 16117 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 47056us
I20260709 13:19:50.431510 16226 raft_consensus.cc:2393] T e5ad4eb281564936a59f41e387a47156 P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 5: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
I20260709 13:19:50.278285 16292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2794af5b99984efebdee564035841cb0" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:50.437198 16148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f5f89186204242bf377f7c42b2e2da" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 7 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
W20260709 13:19:50.438158 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 62344us
I20260709 13:19:50.439131 16191 leader_election.cc:304] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:19:50.439239 16148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f2da98864b4425bf7c0055951fdd4b" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:19:50.274716 18522 leader_election.cc:290] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:19:50.277884 18488 leader_election.cc:290] T 8a878999c0024d2abe517484180290fc P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:19:50.267035 18456 raft_consensus.cc:493] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:50.280911 18418 raft_consensus.cc:2749] T ecc1e363230043fe9fef390f0b979f20 P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:19:50.270896 18109 raft_consensus.cc:2749] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:19:50.369204 18514 log.cc:927] Time spent T d16dfad0c3b540e5b4b86492ddf87208 P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 0.562s	user 0.001s	sys 0.000s
I20260709 13:19:50.458037 16151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cc959fb7f704fba9009c0595467d074" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:19:50.276975 18496 leader_election.cc:290] T d2ab0ac8347641e6955e9bce7d90c1a2 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:50.404659 16149 raft_consensus.cc:3060] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:19:50.243564 16818 raft_consensus.cc:2468] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 4.
I20260709 13:19:50.425196 16819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3884f864a96d428785895e5463e5728f" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
W20260709 13:19:50.425225 18389 consensus_meta.cc:287] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860: Time spent flushing consensus metadata: real 0.634s	user 0.009s	sys 0.000s
I20260709 13:19:50.361217 17992 raft_consensus.cc:493] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
W20260709 13:19:50.453267 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 404751us
I20260709 13:19:50.458638 16151 raft_consensus.cc:3060] T 8cc959fb7f704fba9009c0595467d074 P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:19:50.437109 18501 raft_consensus.cc:493] T 434bff3082d141838068f323bdb0d1ed P cb82cc7bd6db46d980c75237a1402860 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:19:50.480754 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66ae176961b74813a58bc15c9c4c6f2a" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
W20260709 13:19:50.468776 16220 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.708s	user 0.012s	sys 0.007s
I20260709 13:19:50.206112 18013 raft_consensus.cc:2804] T 5a96373bb0e14dce8a266b2fc6c58507 P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:19:50.449649 16219 raft_consensus.cc:2468] T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 5.
I20260709 13:19:50.272135 18421 raft_consensus.cc:515] T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:49.771919 16291 rpcz_store.cc:276] 0709 13:19:48.522066 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:48.582114 (+ 60048us) service_pool.cc:224] Handling call
0709 13:19:48.606638 (+ 24524us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:48.656721 (+ 50083us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:48.658091 (+  1370us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:48.676880 (+ 18789us) log.cc:844] Serialized 26 byte log entry
0709 13:19:48.910519 (+233639us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:19:48.910588 (+    69us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:48.910699 (+   111us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:49.718181 (+807482us) raft_consensus.cc:1685] finished
0709 13:19:49.718209 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:49.718367 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":8937,"raft.run_cpu_time_us":107,"raft.run_wall_time_us":7179,"thread_start_us":202509,"threads_started":1,"wal-append.queue_time_us":703809}
I20260709 13:19:50.474776 16145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 5 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:19:50.472170 16191 leader_election.cc:304] T fffab26427e24d398e2695aad40ca134 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:19:50.516307 16145 raft_consensus.cc:2410] T ecc1e363230043fe9fef390f0b979f20 P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 for term 5 because replica has last-logged OpId of term: 2 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 3.
I20260709 13:19:50.474311 16146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5ad4eb281564936a59f41e387a47156" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:19:50.474431 16144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:19:50.518340 16144 raft_consensus.cc:2410] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 for term 6 because replica has last-logged OpId of term: 4 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 4.
I20260709 13:19:50.374550 18433 raft_consensus.cc:515] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:50.461512 16148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3884f864a96d428785895e5463e5728f" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:19:50.522274 16148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e65676b21b748728ad39668cfe722ec" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:19:50.522856 16148 raft_consensus.cc:2468] T 9e65676b21b748728ad39668cfe722ec P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 5.
I20260709 13:19:50.446022 18473 raft_consensus.cc:515] T 23f82224f4564fea93572e8206bad1fe P cb82cc7bd6db46d980c75237a1402860 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:50.373572 16293 rpcz_store.cc:276] 0709 13:19:48.650044 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:49.932692 (+1282648us) service_pool.cc:224] Handling call
0709 13:19:49.933033 (+   341us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:50.059216 (+126183us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:50.107445 (+ 48229us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:19:50.489468 18521 log.cc:927] Time spent T d2ab0ac8347641e6955e9bce7d90c1a2 P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.247s	user 0.000s	sys 0.001s
I20260709 13:19:50.489750 18323 leader_election.cc:290] T c4f65cdf00b748bb963ee6e81ef43b79 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 7 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:19:50.448312 18417 raft_consensus.cc:493] T 8a878999c0024d2abe517484180290fc P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:50.469760 18454 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.744s	user 0.010s	sys 0.010s
I20260709 13:19:50.371374 18346 raft_consensus.cc:515] T 8158bbd17b994d2eb96f560bd0f1b88d P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:50.450533 18372 raft_consensus.cc:2804] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Leader election won for term 5
W20260709 13:19:50.495776 16220 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.735s	user 0.012s	sys 0.007s
I20260709 13:19:50.496634 16151 raft_consensus.cc:2468] T 8cc959fb7f704fba9009c0595467d074 P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 5.
I20260709 13:19:50.448791 16817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9944d5e4b8c4109af9955c4f08c9095" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:19:50.498068 16822 raft_consensus.cc:1275] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 [term 5 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:19:50.273619 18422 consensus_queue.cc:260] T d16dfad0c3b540e5b4b86492ddf87208 P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:50.140000 18495 raft_consensus.cc:515] T 3a7ca61592e848ea9b0fb81a2ce33cdc P cb82cc7bd6db46d980c75237a1402860 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:50.470546 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a959dbecc2648eb85d1cc550f5a3245" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:19:50.591807 16150 raft_consensus.cc:3060] T 5a959dbecc2648eb85d1cc550f5a3245 P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:19:50.452093 18522 raft_consensus.cc:493] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:19:50.374289 18254 raft_consensus.cc:515] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:50.462221 18041 raft_consensus.cc:493] T fffab26427e24d398e2695aad40ca134 P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:50.204953 16142 rpcz_store.cc:269] 0709 13:19:47.945043 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:47.954882 (+  9839us) service_pool.cc:224] Handling call
0709 13:19:50.003814 (+2048932us) spinlock_profiling.cc:243] Waited 52.3 ms on lock 0x7b540013ecd4. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:19:50.020696 (+ 16882us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":120351104}
I20260709 13:19:50.436458 18504 consensus_queue.cc:260] T 02838c86531d4f98bf58d1c22b8baf77 P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:50.505352 16149 raft_consensus.cc:2468] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 7.
I20260709 13:19:50.437584 18457 raft_consensus.cc:3060] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:19:50.512221 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 519241us
I20260709 13:19:50.513993 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cc959fb7f704fba9009c0595467d074" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:19:50.615384 16225 raft_consensus.cc:3060] T 8cc959fb7f704fba9009c0595467d074 P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:19:50.475947 18387 leader_election.cc:290] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 7 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:50.490520 17992 raft_consensus.cc:3060] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:19:50.437456 18256 raft_consensus.cc:515] T c2f5f89186204242bf377f7c42b2e2da P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:50.514915 16789 leader_election.cc:304] T 9e65676b21b748728ad39668cfe722ec P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:19:50.523759 16148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e9b2bf7011f4ae688a79d7bb5a447b4" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:19:50.358428 18114 consensus_queue.cc:237] T ee9ed67006144797b1de6ea8c2013a68 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:50.526467 16265 leader_election.cc:304] T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac
I20260709 13:19:50.505900 18013 raft_consensus.cc:493] T 5a96373bb0e14dce8a266b2fc6c58507 P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:50.511816 16292 raft_consensus.cc:2410] T 2794af5b99984efebdee564035841cb0 P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 for term 4 because replica has last-logged OpId of term: 2 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 13:19:50.498585 18489 leader_election.cc:290] T ada5a004ba7c46819051e3ccec71ffdb P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:19:50.535907 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18ad7e3af5504571a69d10c00750a01c" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:50.503482 18109 raft_consensus.cc:2749] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:19:50.499617 16221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:50.639001 16218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f5f89186204242bf377f7c42b2e2da" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 7 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:50.645532 16221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2372981006d4b218215c28964dea623" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:50.645872 18254 leader_election.cc:290] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:19:50.551352 18417 raft_consensus.cc:515] T 8a878999c0024d2abe517484180290fc P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:50.646199 16221 raft_consensus.cc:2468] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 4.
I20260709 13:19:50.646713 18323 raft_consensus.cc:2804] T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:19:50.641917 18347 leader_election.cc:290] T 31969392ccdb43b491911a319e8bc7e1 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:19:50.647352 18323 raft_consensus.cc:697] T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045 [term 5 LEADER]: Becoming Leader. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:19:50.648283 18346 leader_election.cc:290] T 8158bbd17b994d2eb96f560bd0f1b88d P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:19:50.648407 18323 consensus_queue.cc:237] T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 4.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:50.540807 16145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18ad7e3af5504571a69d10c00750a01c" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:19:50.650079 16145 raft_consensus.cc:2393] T 18ad7e3af5504571a69d10c00750a01c P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 4: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
W20260709 13:19:50.542956 16823 consensus_meta.cc:287] T 19baa7bc367844c288bca8f81439c413 P cb82cc7bd6db46d980c75237a1402860: Time spent flushing consensus metadata: real 0.552s	user 0.011s	sys 0.000s
I20260709 13:19:50.513804 18276 raft_consensus.cc:493] T 02838c86531d4f98bf58d1c22b8baf77 P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:50.521149 18472 raft_consensus.cc:493] T cdbd78acb4cf46298e0de8ba0d130e19 P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:50.569118 16824 raft_consensus.cc:3060] T 23f82224f4564fea93572e8206bad1fe P cb82cc7bd6db46d980c75237a1402860 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:19:50.515123 18418 raft_consensus.cc:493] T b87147731a68400e8cce9b879daa7c5d P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:50.538538 16819 raft_consensus.cc:2393] T 3884f864a96d428785895e5463e5728f P cb82cc7bd6db46d980c75237a1402860 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 4: Already voted for candidate cb82cc7bd6db46d980c75237a1402860 in this term.
W20260709 13:19:50.543692 18454 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.818s	user 0.010s	sys 0.010s
I20260709 13:19:50.522761 18400 leader_election.cc:290] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:19:50.490463 18456 raft_consensus.cc:515] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:50.527432 18432 raft_consensus.cc:493] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:50.537127 18488 raft_consensus.cc:493] T e4dc6b0541ca431ebc757beac00521f1 P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:50.604354 16117 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 157923us
I20260709 13:19:50.503482 18501 raft_consensus.cc:515] T 434bff3082d141838068f323bdb0d1ed P cb82cc7bd6db46d980c75237a1402860 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:19:50.618846 16789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 104029us
I20260709 13:19:50.582532 16818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66ae176961b74813a58bc15c9c4c6f2a" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:19:50.627884 16297 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48684 (request call id 1980) took 1979 ms. Trace:
I20260709 13:19:50.614730 18041 raft_consensus.cc:515] T fffab26427e24d398e2695aad40ca134 P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:50.632550 18524 leader_election.cc:290] T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:50.633325 18518 leader_election.cc:290] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:50.684084 16818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:19:50.683394 18463 leader_election.cc:290] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:19:50.659533 18472 raft_consensus.cc:515] T cdbd78acb4cf46298e0de8ba0d130e19 P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:19:50.686712 18537 log.cc:927] Time spent T c0eab25ce4c64e018a041beeda2867db P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.079s	user 0.001s	sys 0.000s
I20260709 13:19:50.657135 18276 raft_consensus.cc:515] T 02838c86531d4f98bf58d1c22b8baf77 P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:50.689254 18276 leader_election.cc:290] T 02838c86531d4f98bf58d1c22b8baf77 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:19:50.690138 16790 leader_election.cc:304] T 2794af5b99984efebdee564035841cb0 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:19:50.690994 16142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e65676b21b748728ad39668cfe722ec" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:19:50.691186 16790 leader_election.cc:304] T ee9ed67006144797b1de6ea8c2013a68 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:19:50.691561 16298 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48674 (request call id 2093) took 2177 ms. Trace:
I20260709 13:19:50.691992 16298 rpcz_store.cc:276] 0709 13:19:48.514550 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:48.569105 (+ 54555us) service_pool.cc:224] Handling call
0709 13:19:48.583423 (+ 14318us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:49.548256 (+964833us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:50.691296 (+1143040us) spinlock_profiling.cc:243] Waited 964 ms on lock 0x7b5400178c54. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:19:50.691498 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2217468160}
W20260709 13:19:50.640863 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 114541us
W20260709 13:19:50.641505 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 169393us
W20260709 13:19:50.629447 18123 consensus_meta.cc:287] T 5a96373bb0e14dce8a266b2fc6c58507 P 90f3da57796b488c9087a2f523adf045: Time spent flushing consensus metadata: real 0.696s	user 0.009s	sys 0.001s
I20260709 13:19:50.622874 18422 raft_consensus.cc:3060] T d16dfad0c3b540e5b4b86492ddf87208 P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:19:50.707530 18123 raft_consensus.cc:515] T 5a96373bb0e14dce8a266b2fc6c58507 P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:50.708751 18459 consensus_queue.cc:260] T af57996a8be84d398f0f023e8d08ef77 P cb82cc7bd6db46d980c75237a1402860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:50.710011 18123 leader_election.cc:290] T 5a96373bb0e14dce8a266b2fc6c58507 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 4 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:19:50.654269 16823 raft_consensus.cc:1275] T 19baa7bc367844c288bca8f81439c413 P cb82cc7bd6db46d980c75237a1402860 [term 5 LEARNER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 13:19:50.712483 18515 raft_consensus.cc:3055] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 647a343a542d4a139fe106c19095129b [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:19:50.712996 18515 raft_consensus.cc:740] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 647a343a542d4a139fe106c19095129b [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:19:50.713651 16823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434bff3082d141838068f323bdb0d1ed" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:19:50.615545 18522 raft_consensus.cc:515] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:50.713876 18515 consensus_queue.cc:260] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:50.639683 18013 raft_consensus.cc:3060] T 5a96373bb0e14dce8a266b2fc6c58507 P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:19:50.694343 16299 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48684 (request call id 1986) took 2044 ms. Trace:
I20260709 13:19:50.692787 16218 raft_consensus.cc:2393] T c2f5f89186204242bf377f7c42b2e2da P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 7: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
I20260709 13:19:50.697719 18512 raft_consensus.cc:493] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:50.692128 18400 raft_consensus.cc:2804] T 2794af5b99984efebdee564035841cb0 P cb82cc7bd6db46d980c75237a1402860 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:19:50.698249 18473 leader_election.cc:290] T 23f82224f4564fea93572e8206bad1fe P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:50.691949 16142 raft_consensus.cc:2393] T 9e65676b21b748728ad39668cfe722ec P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 in current term 5: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
I20260709 13:19:50.661455 18418 raft_consensus.cc:515] T b87147731a68400e8cce9b879daa7c5d P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:50.693370 18432 raft_consensus.cc:515] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:50.679873 18488 raft_consensus.cc:515] T e4dc6b0541ca431ebc757beac00521f1 P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:50.628990 18389 raft_consensus.cc:2804] T 9e65676b21b748728ad39668cfe722ec P cb82cc7bd6db46d980c75237a1402860 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:19:50.660128 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8638ee363273426db1393dd81e5c7b72" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:50.625197 18372 raft_consensus.cc:697] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [term 5 LEADER]: Becoming Leader. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:19:50.642393 18450 leader_election.cc:290] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 6 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:19:50.649665 18388 leader_election.cc:290] T 2794af5b99984efebdee564035841cb0 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
W20260709 13:19:50.654737 18452 consensus_meta.cc:287] T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac: Time spent flushing consensus metadata: real 0.717s	user 0.012s	sys 0.000s
I20260709 13:19:50.651093 16221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 5 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:50.675268 16297 rpcz_store.cc:276] 0709 13:19:48.648601 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:49.583647 (+935046us) service_pool.cc:224] Handling call
0709 13:19:49.583964 (+   317us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:49.584314 (+   350us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:49.584344 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:49.621804 (+ 37460us) log.cc:844] Serialized 26 byte log entry
0709 13:19:49.772194 (+150390us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:19:49.772235 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:49.772300 (+    65us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:50.627673 (+855373us) raft_consensus.cc:1685] finished
0709 13:19:50.627696 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:50.627827 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":23497,"raft.run_cpu_time_us":120,"raft.run_wall_time_us":6043,"thread_start_us":64175,"threads_started":1,"wal-append.queue_time_us":370778}
I20260709 13:19:50.678109 16817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a9de347c711477e87c582d003f36925" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:19:50.657562 18109 raft_consensus.cc:2749] T fffab26427e24d398e2695aad40ca134 P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:19:50.658190 18485 leader_election.cc:290] T c9944d5e4b8c4109af9955c4f08c9095 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 6 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:19:50.680691 18523 leader_election.cc:290] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:19:50.680763 18457 raft_consensus.cc:515] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:19:50.682516 18498 log.cc:927] Time spent T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860: Append to log took a long time: real 1.225s	user 0.012s	sys 0.026s
I20260709 13:19:50.683344 18041 leader_election.cc:290] T fffab26427e24d398e2695aad40ca134 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:19:50.696506 16191 leader_election.cc:304] T 829b4ad8c5a54eababeea1181b960d4b P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: cb82cc7bd6db46d980c75237a1402860
I20260709 13:19:50.699090 16150 raft_consensus.cc:2468] T 5a959dbecc2648eb85d1cc550f5a3245 P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 4.
I20260709 13:19:50.998672 16146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b7d1ab519044eda2d5e04a07242f25" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
W20260709 13:19:50.716521 16115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 619353us
I20260709 13:19:50.730183 16823 raft_consensus.cc:3060] T 434bff3082d141838068f323bdb0d1ed P cb82cc7bd6db46d980c75237a1402860 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:19:50.724642 16145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3884f864a96d428785895e5463e5728f" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:19:50.897142 16151 raft_consensus.cc:1275] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:19:50.739948 16298 raft_consensus.cc:1240] T a378d7f796f94615b3b347119a9880fd P 90f3da57796b488c9087a2f523adf045 [term 4 LEADER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 3. Current term is 4. Ops: []
W20260709 13:19:51.018143 16266 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.755s	user 0.014s	sys 0.022s
W20260709 13:19:51.018656 16266 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.756s	user 0.014s	sys 0.022s
I20260709 13:19:51.020826 16220 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57604 (request call id 2016) took 1272 ms. Trace:
I20260709 13:19:50.729723 16225 raft_consensus.cc:2468] T 8cc959fb7f704fba9009c0595467d074 P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 5.
I20260709 13:19:50.716964 16148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87147731a68400e8cce9b879daa7c5d" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:19:51.047217 16225 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57600 (request call id 1083) took 1046 ms. Trace:
I20260709 13:19:51.047325 16148 raft_consensus.cc:2468] T b87147731a68400e8cce9b879daa7c5d P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 6.
I20260709 13:19:50.759418 16224 raft_consensus.cc:2468] T 5a959dbecc2648eb85d1cc550f5a3245 P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 4.
I20260709 13:19:50.718218 16299 rpcz_store.cc:276] 0709 13:19:48.650332 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:49.946071 (+1295739us) service_pool.cc:224] Handling call
0709 13:19:50.015825 (+ 69754us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:50.022367 (+  6542us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:50.022399 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:50.042238 (+ 19839us) log.cc:844] Serialized 26 byte log entry
0709 13:19:50.457411 (+415173us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:19:50.457456 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:50.457526 (+    70us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:50.694134 (+236608us) raft_consensus.cc:1685] finished
0709 13:19:50.694155 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:50.694285 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6152,"raft.queue_time_us":6633,"raft.run_cpu_time_us":166,"raft.run_wall_time_us":23932,"thread_start_us":348870,"threads_started":1,"wal-append.queue_time_us":440761}
W20260709 13:19:50.774289 16116 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:51.049069 16224 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57600 (request call id 1082) took 1048 ms. Trace:
I20260709 13:19:50.729050 16824 raft_consensus.cc:2468] T 23f82224f4564fea93572e8206bad1fe P cb82cc7bd6db46d980c75237a1402860 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 2.
I20260709 13:19:51.021987 16298 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40474 (request call id 340) took 1010 ms. Trace:
I20260709 13:19:50.734892 16265 leader_election.cc:304] T 9e65676b21b748728ad39668cfe722ec P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:19:51.050441 16298 rpcz_store.cc:276] 0709 13:19:50.011401 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:50.739451 (+728050us) service_pool.cc:224] Handling call
0709 13:19:50.739770 (+   319us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:51.017677 (+277907us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:51.021914 (+  4237us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:51.023892 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ab0ac8347641e6955e9bce7d90c1a2" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
W20260709 13:19:51.037191 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 340735us
I20260709 13:19:51.047840 16225 rpcz_store.cc:276] 0709 13:19:50.001097 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:50.503195 (+502098us) service_pool.cc:224] Handling call
0709 13:19:51.047112 (+543917us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:51.056555 16226 raft_consensus.cc:2468] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 4.
I20260709 13:19:51.056931 16225 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57604 (request call id 2024) took 1281 ms. Trace:
I20260709 13:19:51.057076 16226 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57600 (request call id 1087) took 1054 ms. Trace:
I20260709 13:19:51.057420 16226 rpcz_store.cc:276] 0709 13:19:50.002449 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:50.650874 (+648425us) service_pool.cc:224] Handling call
0709 13:19:51.057022 (+406148us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:51.057331 16225 rpcz_store.cc:276] 0709 13:19:49.775470 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:51.056088 (+1280618us) service_pool.cc:224] Handling call
0709 13:19:51.056442 (+   354us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:51.056608 (+   166us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:51.056635 (+    27us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:51.056679 (+    44us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:19:51.056712 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:51.056770 (+    58us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:51.056875 (+   105us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:51.049635 16224 rpcz_store.cc:276] 0709 13:19:50.000822 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:50.040579 (+ 39757us) service_pool.cc:224] Handling call
0709 13:19:51.048967 (+1008388us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:19:50.740927 16143 consensus_meta.cc:287] T e5ad4eb281564936a59f41e387a47156 P 647a343a542d4a139fe106c19095129b: Time spent flushing consensus metadata: real 1.358s	user 0.013s	sys 0.000s
I20260709 13:19:51.048729 16220 rpcz_store.cc:276] 0709 13:19:49.748689 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:49.754784 (+  6095us) service_pool.cc:224] Handling call
0709 13:19:49.759055 (+  4271us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:49.759286 (+   231us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 13:19:50.621953 (+862667us) log.cc:844] Serialized 14 byte log entry
0709 13:19:51.020399 (+398446us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:51.020481 (+    82us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:19:51.020545 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:51.020637 (+    92us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:51.020761 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3093,"prepare.queue_time_us":296571,"prepare.run_cpu_time_us":36,"prepare.run_wall_time_us":36,"raft.queue_time_us":11344,"thread_start_us":1203435,"threads_started":2}
I20260709 13:19:50.698660 18504 raft_consensus.cc:3060] T 02838c86531d4f98bf58d1c22b8baf77 P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:19:50.718595 18057 consensus_meta.cc:287] T c0eab25ce4c64e018a041beeda2867db P 647a343a542d4a139fe106c19095129b: Time spent flushing consensus metadata: real 0.970s	user 0.005s	sys 0.005s
I20260709 13:19:51.067397 16115 leader_election.cc:400] T 5a959dbecc2648eb85d1cc550f5a3245 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 3 pre-election: Vote denied by peer 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389) with higher term. Message: Invalid argument: T 5a959dbecc2648eb85d1cc550f5a3245 P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b for earlier term 3. Current term is 4.
I20260709 13:19:50.735623 16292 raft_consensus.cc:1240] T cdbd78acb4cf46298e0de8ba0d130e19 P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 4. Current term is 5. Ops: []
I20260709 13:19:51.104516 16115 leader_election.cc:403] T 5a959dbecc2648eb85d1cc550f5a3245 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 3 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:19:50.950376 18538 raft_consensus.cc:493] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:50.732795 16818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a378d7f796f94615b3b347119a9880fd" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:19:50.735975 16819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:19:50.727828 18501 leader_election.cc:290] T 434bff3082d141838068f323bdb0d1ed P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:50.716145 18522 leader_election.cc:290] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:19:51.072381 16143 raft_consensus.cc:2468] T e5ad4eb281564936a59f41e387a47156 P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 5.
I20260709 13:19:50.720952 18256 leader_election.cc:290] T c2f5f89186204242bf377f7c42b2e2da P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 7 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:50.788084 18463 raft_consensus.cc:493] T 18ad7e3af5504571a69d10c00750a01c P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:50.721695 16218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62fe3346cd3c42c4a55e14af9073bca4" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:51.011495 16291 raft_consensus.cc:1275] T e4dc6b0541ca431ebc757beac00521f1 P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:19:51.082319 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66ae176961b74813a58bc15c9c4c6f2a" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:19:51.095343 18530 raft_consensus.cc:2764] T 5a959dbecc2648eb85d1cc550f5a3245 P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Leader election decision vote started in defunct term 3: won
W20260709 13:19:51.076566 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 341710us
I20260709 13:19:50.715130 18515 raft_consensus.cc:3060] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:19:50.725900 18472 leader_election.cc:290] T cdbd78acb4cf46298e0de8ba0d130e19 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:19:51.087399 16146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f82224f4564fea93572e8206bad1fe" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:19:50.741956 18487 raft_consensus.cc:2804] T ee9ed67006144797b1de6ea8c2013a68 P cb82cc7bd6db46d980c75237a1402860 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:19:50.788702 18473 raft_consensus.cc:493] T 8c11074579b046d99e585c2818e18a2c P cb82cc7bd6db46d980c75237a1402860 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:19:51.091773 18524 raft_consensus.cc:493] T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:51.092407 18488 leader_election.cc:290] T e4dc6b0541ca431ebc757beac00521f1 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:19:51.092799 16148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31969392ccdb43b491911a319e8bc7e1" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:19:51.169004 16148 raft_consensus.cc:2410] T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 for term 5 because replica has last-logged OpId of term: 3 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 13:19:50.715639 18433 leader_election.cc:290] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:19:51.062618 16191 leader_election.cc:304] T af57996a8be84d398f0f023e8d08ef77 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
I20260709 13:19:51.091825 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7ca61592e848ea9b0fb81a2ce33cdc" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:51.097769 16823 raft_consensus.cc:2468] T 434bff3082d141838068f323bdb0d1ed P cb82cc7bd6db46d980c75237a1402860 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 5.
I20260709 13:19:51.100592 18457 leader_election.cc:290] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 3 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:50.738149 18418 leader_election.cc:290] T b87147731a68400e8cce9b879daa7c5d P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:19:50.717882 18495 leader_election.cc:290] T 3a7ca61592e848ea9b0fb81a2ce33cdc P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:50.722781 18400 raft_consensus.cc:493] T 2794af5b99984efebdee564035841cb0 P cb82cc7bd6db46d980c75237a1402860 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:51.021999 18535 raft_consensus.cc:2804] T 829b4ad8c5a54eababeea1181b960d4b P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:19:51.186888 18400 raft_consensus.cc:3060] T 2794af5b99984efebdee564035841cb0 P cb82cc7bd6db46d980c75237a1402860 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:19:51.186987 18535 raft_consensus.cc:493] T 829b4ad8c5a54eababeea1181b960d4b P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Starting leader election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:19:51.187507 18535 raft_consensus.cc:3060] T 829b4ad8c5a54eababeea1181b960d4b P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:19:51.134976 18463 raft_consensus.cc:515] T 18ad7e3af5504571a69d10c00750a01c P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } }
I20260709 13:19:50.720014 18456 leader_election.cc:290] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:50.728078 18421 leader_election.cc:290] T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:19:51.190156 18463 leader_election.cc:290] T 18ad7e3af5504571a69d10c00750a01c P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:19:51.161748 18487 raft_consensus.cc:493] T ee9ed67006144797b1de6ea8c2013a68 P cb82cc7bd6db46d980c75237a1402860 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:51.190524 18421 raft_consensus.cc:2749] T 5a959dbecc2648eb85d1cc550f5a3245 P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Leader pre-election lost for term 3. Reason: Vote denied by peer 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389) with higher term. Message: Invalid argument: T 5a959dbecc2648eb85d1cc550f5a3245 P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b for earlier term 3. Current term is 4.
I20260709 13:19:51.190743 18487 raft_consensus.cc:3060] T ee9ed67006144797b1de6ea8c2013a68 P cb82cc7bd6db46d980c75237a1402860 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:19:51.072023 18140 raft_consensus.cc:493] T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:50.735087 18347 consensus_queue.cc:1048] T 8d4ab870a05347868956aa8b841b2981 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.369s
I20260709 13:19:51.193010 18140 raft_consensus.cc:515] T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:50.735438 18539 consensus_queue.cc:1048] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.374s
I20260709 13:19:51.087960 18485 raft_consensus.cc:2804] T 9e65676b21b748728ad39668cfe722ec P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:19:50.740347 18432 leader_election.cc:290] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:19:51.123997 16151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434bff3082d141838068f323bdb0d1ed" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:19:51.196230 18485 raft_consensus.cc:493] T 9e65676b21b748728ad39668cfe722ec P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:51.196089 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87147731a68400e8cce9b879daa7c5d" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:51.079902 18389 raft_consensus.cc:493] T 9e65676b21b748728ad39668cfe722ec P cb82cc7bd6db46d980c75237a1402860 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:19:51.196683 18485 raft_consensus.cc:3060] T 9e65676b21b748728ad39668cfe722ec P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:19:51.197034 18389 raft_consensus.cc:3060] T 9e65676b21b748728ad39668cfe722ec P cb82cc7bd6db46d980c75237a1402860 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:19:51.197470 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434bff3082d141838068f323bdb0d1ed" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:51.198235 16299 raft_consensus.cc:2393] T 434bff3082d141838068f323bdb0d1ed P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 in current term 5: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
I20260709 13:19:50.741173 18013 raft_consensus.cc:515] T 5a96373bb0e14dce8a266b2fc6c58507 P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:51.201221 18013 leader_election.cc:290] T 5a96373bb0e14dce8a266b2fc6c58507 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 4 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:51.116250 16221 raft_consensus.cc:2410] T ecc1e363230043fe9fef390f0b979f20 P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 for term 5 because replica has last-logged OpId of term: 2 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 3.
I20260709 13:19:51.202203 16221 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57600 (request call id 1080) took 1202 ms. Trace:
I20260709 13:19:51.199007 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31969392ccdb43b491911a319e8bc7e1" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:51.204341 16299 raft_consensus.cc:2410] T 31969392ccdb43b491911a319e8bc7e1 P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 for term 5 because replica has last-logged OpId of term: 3 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
W20260709 13:19:51.205220 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 269329us
W20260709 13:19:51.205806 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 143292us
I20260709 13:19:50.724433 18123 consensus_queue.cc:1048] T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.390s
I20260709 13:19:50.722345 18512 raft_consensus.cc:515] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:51.103008 17992 raft_consensus.cc:515] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:51.083463 16225 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57604 (request call id 2027) took 1283 ms. Trace:
I20260709 13:19:51.218827 16225 rpcz_store.cc:276] 0709 13:19:49.800380 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:51.058663 (+1258283us) service_pool.cc:224] Handling call
0709 13:19:51.082692 (+ 24029us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:51.082921 (+   229us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:51.082961 (+    40us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:51.083021 (+    60us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:19:51.083071 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:51.083148 (+    77us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:51.083388 (+   240us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:51.088147 18450 consensus_queue.cc:1048] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.041s
I20260709 13:19:51.113070 18057 raft_consensus.cc:515] T c0eab25ce4c64e018a041beeda2867db P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:51.142524 16265 leader_election.cc:304] T 8cc959fb7f704fba9009c0595467d074 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:19:51.154492 16146 raft_consensus.cc:2468] T 23f82224f4564fea93572e8206bad1fe P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 2.
W20260709 13:19:51.222637 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 80223.1us
I20260709 13:19:51.167569 16220 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57604 (request call id 2030) took 1019 ms. Trace:
I20260709 13:19:51.227324 16220 rpcz_store.cc:276] 0709 13:19:50.147738 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:51.093487 (+945749us) service_pool.cc:224] Handling call
0709 13:19:51.167027 (+ 73540us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:51.167231 (+   204us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:51.167258 (+    27us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:51.167302 (+    44us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:19:51.167336 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:51.167391 (+    55us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:51.167509 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:51.131448 16292 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40474 (request call id 338) took 1118 ms. Trace:
I20260709 13:19:51.175256 18372 consensus_queue.cc:237] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:51.129200 18501 consensus_queue.cc:1059] T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.036s
I20260709 13:19:51.121243 18538 raft_consensus.cc:515] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:51.184425 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f82224f4564fea93572e8206bad1fe" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:51.232316 16298 raft_consensus.cc:2393] T 23f82224f4564fea93572e8206bad1fe P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 in current term 2: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
I20260709 13:19:51.185894 18418 raft_consensus.cc:493] T 26354361e9b044cea1c2b86c4882fe92 P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:19:51.170850 16817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e65676b21b748728ad39668cfe722ec" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:19:51.214876 18522 raft_consensus.cc:2749] T af57996a8be84d398f0f023e8d08ef77 P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:19:51.177956 18524 raft_consensus.cc:515] T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:51.215523 16790 leader_election.cc:304] T 3884f864a96d428785895e5463e5728f P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:19:51.174090 18473 raft_consensus.cc:515] T 8c11074579b046d99e585c2818e18a2c P cb82cc7bd6db46d980c75237a1402860 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:51.230551 16191 leader_election.cc:304] T 5a96373bb0e14dce8a266b2fc6c58507 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 90f3da57796b488c9087a2f523adf045
I20260709 13:19:51.242949 16790 leader_election.cc:304] T b87147731a68400e8cce9b879daa7c5d P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860; no voters: 
W20260709 13:19:51.232946 16789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 147148us
W20260709 13:19:51.171238 18511 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.034s	user 0.013s	sys 0.009s
I20260709 13:19:51.240425 16821 raft_consensus.cc:1240] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 [term 5 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 4. Current term is 5. Ops: []
I20260709 13:19:51.195122 18140 leader_election.cc:290] T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:51.202587 16221 rpcz_store.cc:276] 0709 13:19:50.000233 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:50.646899 (+646666us) service_pool.cc:224] Handling call
0709 13:19:51.202143 (+555244us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:51.253176 16299 raft_consensus.cc:1240] T c9944d5e4b8c4109af9955c4f08c9095 P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 5. Current term is 6. Ops: []
I20260709 13:19:51.254127 16299 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40474 (request call id 342) took 1242 ms. Trace:
I20260709 13:19:51.237172 18539 raft_consensus.cc:2804] T 8cc959fb7f704fba9009c0595467d074 P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:19:51.231663 18463 raft_consensus.cc:493] T 23f82224f4564fea93572e8206bad1fe P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:51.254568 16299 rpcz_store.cc:276] 0709 13:19:50.012069 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:51.236520 (+1224451us) service_pool.cc:224] Handling call
0709 13:19:51.252937 (+ 16417us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:51.253801 (+   864us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:51.254054 (+   253us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:51.246307 16298 raft_consensus.cc:1275] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
W20260709 13:19:51.248860 18511 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.118s	user 0.013s	sys 0.009s
I20260709 13:19:51.273036 16265 leader_election.cc:304] T 84f2da98864b4425bf7c0055951fdd4b P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:19:51.249655 18456 raft_consensus.cc:2749] T 3884f864a96d428785895e5463e5728f P cb82cc7bd6db46d980c75237a1402860 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:19:51.252444 18495 consensus_queue.cc:1048] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.036s
I20260709 13:19:51.249102 18501 consensus_queue.cc:1243] T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Peer 90f3da57796b488c9087a2f523adf045 log is divergent from this leader: its last log entry 4.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 13:19:51.260804 18457 raft_consensus.cc:2804] T b87147731a68400e8cce9b879daa7c5d P cb82cc7bd6db46d980c75237a1402860 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:19:51.268379 18114 raft_consensus.cc:2804] T 5a96373bb0e14dce8a266b2fc6c58507 P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 13:19:51.329239 16191 connection.cc:256] RPC call timeout handler was delayed by -2.2171880210000836s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22060000000000002s for a response
I20260709 13:19:51.232177 16292 rpcz_store.cc:276] 0709 13:19:50.010622 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:50.674284 (+663662us) service_pool.cc:224] Handling call
0709 13:19:50.735386 (+ 61102us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:51.114296 (+378910us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:51.128881 (+ 14585us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:19:51.345343 16117 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:51.288054 18524 leader_election.cc:290] T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:51.278672 18463 raft_consensus.cc:515] T 23f82224f4564fea93572e8206bad1fe P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:51.290688 18539 raft_consensus.cc:697] T 8cc959fb7f704fba9009c0595467d074 P 90f3da57796b488c9087a2f523adf045 [term 5 LEADER]: Becoming Leader. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:19:51.261662 18418 raft_consensus.cc:515] T 26354361e9b044cea1c2b86c4882fe92 P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:51.357290 16144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e9b2bf7011f4ae688a79d7bb5a447b4" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:19:51.354033 18114 raft_consensus.cc:493] T 5a96373bb0e14dce8a266b2fc6c58507 P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:51.330713 16819 raft_consensus.cc:2393] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 4: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
W20260709 13:19:51.356431 16117 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:51.305218 16789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 44887.3us
W20260709 13:19:51.362936 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 90021us
I20260709 13:19:51.267895 18522 consensus_queue.cc:1048] T c0eab25ce4c64e018a041beeda2867db P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.084s
W20260709 13:19:51.363922 16115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 353907us
W20260709 13:19:51.430560 16266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 281725us
I20260709 13:19:51.426441 16220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee9ed67006144797b1de6ea8c2013a68" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:51.327756 18457 raft_consensus.cc:493] T b87147731a68400e8cce9b879daa7c5d P cb82cc7bd6db46d980c75237a1402860 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:51.435267 16265 leader_election.cc:304] T 5a959dbecc2648eb85d1cc550f5a3245 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:19:51.437732 16221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746aabf856ab401c9ef5e859371e7b67" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
W20260709 13:19:51.433917 16117 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:51.355154 18456 raft_consensus.cc:493] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:51.447793 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4f65cdf00b748bb963ee6e81ef43b79" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
W20260709 13:19:51.450969 16266 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:51.407584 18123 raft_consensus.cc:493] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:19:51.459810 16146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1df93779209748b9860f6d893dd3de53" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:19:51.436832 18114 raft_consensus.cc:3060] T 5a96373bb0e14dce8a266b2fc6c58507 P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:19:51.409040 18450 raft_consensus.cc:493] T 84f2da98864b4425bf7c0055951fdd4b P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:19:51.463788 16148 raft_consensus.cc:1275] T b87147731a68400e8cce9b879daa7c5d P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 6 index: 6. (index mismatch)
W20260709 13:19:51.462920 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 200537us
I20260709 13:19:51.435570 18432 raft_consensus.cc:2749] T 84f2da98864b4425bf7c0055951fdd4b P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:19:51.466655 16144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a878999c0024d2abe517484180290fc" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:19:51.905290 16820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e9b2bf7011f4ae688a79d7bb5a447b4" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
W20260709 13:19:51.888543 18564 log.cc:927] Time spent T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 0.294s	user 0.001s	sys 0.000s
I20260709 13:19:51.906217 16820 raft_consensus.cc:2393] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 3: Already voted for candidate cb82cc7bd6db46d980c75237a1402860 in this term.
W20260709 13:19:51.887483 16143 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40844 (request call id 1839) took 3619 ms (client timeout 2206 ms). Trace:
W20260709 13:19:51.906826 16820 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49892 (request call id 362) took 1741 ms (client timeout 2107 ms). Trace:
W20260709 13:19:51.907262 16820 rpcz_store.cc:269] 0709 13:19:50.165933 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:51.895920 (+1729987us) service_pool.cc:224] Handling call
0709 13:19:51.906771 (+ 10851us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:51.883968 18473 leader_election.cc:290] T 8c11074579b046d99e585c2818e18a2c P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:51.908480 16820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8638ee363273426db1393dd81e5c7b72" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:19:51.908563 18473 raft_consensus.cc:493] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P cb82cc7bd6db46d980c75237a1402860 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:19:51.909161 18473 raft_consensus.cc:515] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P cb82cc7bd6db46d980c75237a1402860 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:51.911028 18538 leader_election.cc:290] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:51.893646 18513 raft_consensus.cc:3055] T 68a6838d82ef4cac83507f906139d90f P 647a343a542d4a139fe106c19095129b [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:19:51.911666 16300 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48684 (request call id 1987) took 3261 ms. Trace:
I20260709 13:19:51.911928 18513 raft_consensus.cc:740] T 68a6838d82ef4cac83507f906139d90f P 647a343a542d4a139fe106c19095129b [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
W20260709 13:19:51.863822 18559 log.cc:927] Time spent T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 0.385s	user 0.000s	sys 0.001s
W20260709 13:19:51.917883 16192 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:51.756536 16266 leader_election.cc:304] T 3884f864a96d428785895e5463e5728f P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
I20260709 13:19:51.905681 16144 raft_consensus.cc:2468] T 8a878999c0024d2abe517484180290fc P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 4.
W20260709 13:19:51.906958 16143 rpcz_store.cc:269] 0709 13:19:48.267791 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:48.398322 (+130531us) service_pool.cc:224] Handling call
0709 13:19:51.887135 (+3488813us) spinlock_profiling.cc:243] Waited 984 ms on lock 0x7b54000ea454. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:19:51.887293 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2262551424}
I20260709 13:19:51.912092 16300 rpcz_store.cc:276] 0709 13:19:48.650618 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:50.007443 (+1356825us) service_pool.cc:224] Handling call
0709 13:19:50.007721 (+   278us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:50.054500 (+ 46779us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:51.911465 (+1856965us) spinlock_profiling.cc:243] Waited 17.3 ms on lock 0x7b540041c3d4. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:19:51.911617 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":39786240}
I20260709 13:19:51.880976 16821 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49906 (request call id 526) took 2438 ms. Trace:
W20260709 13:19:51.909184 16820 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49892 (request call id 364) took 1742 ms (client timeout 2188 ms). Trace:
I20260709 13:19:51.883143 16222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66ae176961b74813a58bc15c9c4c6f2a" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:19:51.881841 16825 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49906 (request call id 523) took 2469 ms. Trace:
I20260709 13:19:51.875649 16147 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40830 (request call id 1081) took 3811 ms. Trace:
I20260709 13:19:51.919051 16790 leader_election.cc:304] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:19:51.948467 16147 rpcz_store.cc:276] 0709 13:19:48.064141 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:48.086149 (+ 22008us) service_pool.cc:224] Handling call
0709 13:19:50.006999 (+1920850us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:50.370632 (+363633us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:51.874461 (+1503829us) spinlock_profiling.cc:243] Waited 238 ms on lock 0x7b540013ecd4. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:19:51.875345 (+   884us) spinlock_profiling.cc:243] Waited 1.89 s on lock 0x7b540013ecd0. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5e8ad 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:19:51.875548 (+   203us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4892376832}
I20260709 13:19:51.949517 16790 leader_election.cc:304] T 434bff3082d141838068f323bdb0d1ed P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
W20260709 13:19:51.950783 16820 rpcz_store.cc:269] 0709 13:19:50.166539 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:51.908256 (+1741717us) service_pool.cc:224] Handling call
0709 13:19:51.909113 (+   857us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:51.477156 16220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5718c623ebee4d2bb770f0f9ddb2f164" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:51.493434 16146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31969392ccdb43b491911a319e8bc7e1" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:19:51.953001 16146 raft_consensus.cc:2468] T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 4.
I20260709 13:19:51.829219 16296 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48684 (request call id 1984) took 3179 ms. Trace:
W20260709 13:19:51.516009 16115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 49565us
I20260709 13:19:51.933112 16265 leader_election.cc:304] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:19:51.493331 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ab0ac8347641e6955e9bce7d90c1a2" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:51.954033 16296 rpcz_store.cc:276] 0709 13:19:48.649776 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:49.587805 (+938029us) service_pool.cc:224] Handling call
0709 13:19:49.622215 (+ 34410us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:49.688329 (+ 66114us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:51.828869 (+2140540us) spinlock_profiling.cc:243] Waited 16.2 ms on lock 0x7b54000cdd54. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:19:51.829143 (+   274us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":37197056}
W20260709 13:19:51.458158 18550 log.cc:927] Time spent T 66ae176961b74813a58bc15c9c4c6f2a P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.255s	user 0.000s	sys 0.001s
I20260709 13:19:51.448730 18347 raft_consensus.cc:2804] T 5a959dbecc2648eb85d1cc550f5a3245 P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:19:51.950986 16790 leader_election.cc:304] T 23f82224f4564fea93572e8206bad1fe P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860; no voters: 90f3da57796b488c9087a2f523adf045
W20260709 13:19:51.845948 16824 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49892 (request call id 354) took 1700 ms (client timeout 1993 ms). Trace:
I20260709 13:19:51.447554 18457 raft_consensus.cc:3060] T b87147731a68400e8cce9b879daa7c5d P cb82cc7bd6db46d980c75237a1402860 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:19:51.902894 18567 log.cc:927] Time spent T 829b4ad8c5a54eababeea1181b960d4b P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 0.073s	user 0.000s	sys 0.000s
W20260709 13:19:51.892328 18504 consensus_meta.cc:287] T 02838c86531d4f98bf58d1c22b8baf77 P 1ca2725d290e47d68b1a4b609c9816ac: Time spent flushing consensus metadata: real 0.783s	user 0.007s	sys 0.003s
W20260709 13:19:51.531481 18548 log.cc:927] Time spent T 84f2da98864b4425bf7c0055951fdd4b P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 0.368s	user 0.001s	sys 0.000s
I20260709 13:19:51.944604 16821 rpcz_store.cc:276] 0709 13:19:49.442529 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:49.442655 (+   126us) service_pool.cc:224] Handling call
0709 13:19:51.121954 (+1679299us) spinlock_profiling.cc:243] Waited 3.07 ms on lock 0x7b5800004298. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69ad8fadb1 00007f69ada3ab71 00007f69ada3aa22 00007f69b6953c09 00007f69b5aadee0 00007f69b5b770ab 00007f69b5b77131 00007f69b5b38fc0 00007f69b5b23929 00007f69b5b233d2 00007f69af09abd4
0709 13:19:51.122195 (+   241us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:51.249238 (+127043us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:51.880690 (+631452us) spinlock_profiling.cc:243] Waited 118 ms on lock 0x7b54005c35d4. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:19:51.880899 (+   209us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":278463488}
W20260709 13:19:51.885916 18389 consensus_meta.cc:287] T 9e65676b21b748728ad39668cfe722ec P cb82cc7bd6db46d980c75237a1402860: Time spent flushing consensus metadata: real 0.688s	user 0.008s	sys 0.001s
W20260709 13:19:51.458379 18547 log.cc:927] Time spent T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 0.296s	user 0.001s	sys 0.001s
I20260709 13:19:51.494127 16150 raft_consensus.cc:1275] T 7c2095a5f4ad49408dbc7f69c606120d P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 2. (index mismatch)
I20260709 13:19:51.466777 16292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e9b2bf7011f4ae688a79d7bb5a447b4" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:19:51.890466 18255 raft_consensus.cc:3055] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [term 6 LEADER]: Stepping down as leader of term 6
W20260709 13:19:51.992358 16824 rpcz_store.cc:269] 0709 13:19:50.138401 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:50.138557 (+   156us) service_pool.cc:224] Handling call
0709 13:19:51.838228 (+1699671us) spinlock_profiling.cc:243] Waited 200 ms on lock 0x7b54007d0354. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:19:51.838406 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":459429760}
I20260709 13:19:52.046100 16824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2372981006d4b218215c28964dea623" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
W20260709 13:19:51.510845 18553 log.cc:927] Time spent T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.242s	user 0.001s	sys 0.000s
W20260709 13:19:51.509876 18555 log.cc:927] Time spent T af57996a8be84d398f0f023e8d08ef77 P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 0.070s	user 0.000s	sys 0.000s
I20260709 13:19:52.054445 16824 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49896 (request call id 849) took 1557 ms. Trace:
I20260709 13:19:52.054925 16824 rpcz_store.cc:276] 0709 13:19:50.497426 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:52.045801 (+1548375us) service_pool.cc:224] Handling call
0709 13:19:52.054369 (+  8568us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:19:52.005247 16189 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:51.943398 16218 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.536s	user 0.016s	sys 0.044s
I20260709 13:19:52.010195 18473 leader_election.cc:290] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:51.517307 16142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fffab26427e24d398e2695aad40ca134" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:19:51.838667 18561 raft_consensus.cc:493] T fffab26427e24d398e2695aad40ca134 P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:51.946614 16825 rpcz_store.cc:276] 0709 13:19:49.412953 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:49.418266 (+  5313us) service_pool.cc:224] Handling call
0709 13:19:49.462808 (+ 44542us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:49.797540 (+334732us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:49.797568 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:49.835809 (+ 38241us) log.cc:844] Serialized 26 byte log entry
0709 13:19:50.211719 (+375910us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:19:50.211781 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:51.881362 (+1669581us) spinlock_profiling.cc:243] Waited 334 ms on lock 0x7b54003df254. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:19:51.881437 (+    75us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:51.881498 (+    61us) raft_consensus.cc:1685] finished
0709 13:19:51.881517 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:51.881774 (+   257us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":273,"raft.run_cpu_time_us":115,"raft.run_wall_time_us":3588,"spinlock_wait_cycles":769004288,"thread_start_us":221055,"threads_started":1,"wal-append.queue_time_us":635579,"wal-append.run_cpu_time_us":1880,"wal-append.run_wall_time_us":1185066}
I20260709 13:19:51.499521 18558 consensus_queue.cc:1059] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 647a343a542d4a139fe106c19095129b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.574s
W20260709 13:19:51.884141 16816 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49906 (request call id 527) took 2179 ms (client timeout 1833 ms). Trace:
I20260709 13:19:51.523478 18488 raft_consensus.cc:493] T c0eab25ce4c64e018a041beeda2867db P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:19:51.943627 18424 raft_consensus.cc:493] T 8158bbd17b994d2eb96f560bd0f1b88d P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:51.593729 16221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62fe3346cd3c42c4a55e14af9073bca4" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:19:52.086237 18424 raft_consensus.cc:515] T 8158bbd17b994d2eb96f560bd0f1b88d P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:52.087553 16221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1df93779209748b9860f6d893dd3de53" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:19:52.088500 18424 leader_election.cc:290] T 8158bbd17b994d2eb96f560bd0f1b88d P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:52.089186 16221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b7d1ab519044eda2d5e04a07242f25" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:52.089402 18488 raft_consensus.cc:515] T c0eab25ce4c64e018a041beeda2867db P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:51.920189 18557 raft_consensus.cc:493] T 57a744f5aaf944678a63469cbf514015 P cb82cc7bd6db46d980c75237a1402860 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:52.091596 18488 leader_election.cc:290] T c0eab25ce4c64e018a041beeda2867db P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:19:52.091430 18557 raft_consensus.cc:515] T 57a744f5aaf944678a63469cbf514015 P cb82cc7bd6db46d980c75237a1402860 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:52.092141 18488 raft_consensus.cc:2749] T 3884f864a96d428785895e5463e5728f P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:19:51.511724 18512 leader_election.cc:290] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
W20260709 13:19:52.017587 16115 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:51.494679 18495 consensus_queue.cc:1059] T c9944d5e4b8c4109af9955c4f08c9095 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.518s
I20260709 13:19:52.095642 16224 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57600 (request call id 1086) took 2093 ms. Trace:
I20260709 13:19:51.977631 18450 raft_consensus.cc:515] T 84f2da98864b4425bf7c0055951fdd4b P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:52.048931 16220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7ca61592e848ea9b0fb81a2ce33cdc" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:52.003293 18565 raft_consensus.cc:493] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:51.515324 16044 catalog_manager.cc:5697] T e5ad4eb281564936a59f41e387a47156 P 647a343a542d4a139fe106c19095129b reported cstate change: term changed from 3 to 4, leader changed from 90f3da57796b488c9087a2f523adf045 (127.14.177.131) to 647a343a542d4a139fe106c19095129b (127.14.177.129). New cstate: current_term: 4 leader_uuid: "647a343a542d4a139fe106c19095129b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } health_report { overall_health: UNKNOWN } } }
I20260709 13:19:51.515664 18539 consensus_queue.cc:237] T 8cc959fb7f704fba9009c0595467d074 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:52.015553 18406 raft_consensus.cc:493] T 9c61a39fd4f84424b129c1a6f9ba404d P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:52.018793 18400 consensus_meta.cc:287] T 2794af5b99984efebdee564035841cb0 P cb82cc7bd6db46d980c75237a1402860: Time spent flushing consensus metadata: real 0.829s	user 0.006s	sys 0.004s
I20260709 13:19:52.116918 18400 raft_consensus.cc:515] T 2794af5b99984efebdee564035841cb0 P cb82cc7bd6db46d980c75237a1402860 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:52.118891 18400 leader_election.cc:290] T 2794af5b99984efebdee564035841cb0 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 4 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:51.993363 16820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee9ed67006144797b1de6ea8c2013a68" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:19:52.014113 18538 raft_consensus.cc:493] T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:52.123430 18538 raft_consensus.cc:515] T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:19:52.125036 16294 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48684 (request call id 1994) took 2405 ms (client timeout 2034 ms). Trace:
I20260709 13:19:52.125437 18557 leader_election.cc:290] T 57a744f5aaf944678a63469cbf514015 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:52.125802 18538 leader_election.cc:290] T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
W20260709 13:19:52.125476 16294 rpcz_store.cc:269] 0709 13:19:49.719689 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:49.724406 (+  4717us) service_pool.cc:224] Handling call
0709 13:19:52.124831 (+2400425us) spinlock_profiling.cc:243] Waited 35.9 ms on lock 0x7b54002e0c54. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:19:52.124969 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":82584448}
W20260709 13:19:52.017755 18535 consensus_meta.cc:287] T 829b4ad8c5a54eababeea1181b960d4b P 1ca2725d290e47d68b1a4b609c9816ac: Time spent flushing consensus metadata: real 0.830s	user 0.008s	sys 0.003s
I20260709 13:19:52.126899 18535 raft_consensus.cc:515] T 829b4ad8c5a54eababeea1181b960d4b P 1ca2725d290e47d68b1a4b609c9816ac [term 6 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:52.129352 18535 leader_election.cc:290] T 829b4ad8c5a54eababeea1181b960d4b P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 6 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:19:52.058821 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8158bbd17b994d2eb96f560bd0f1b88d" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:19:52.130256 16150 raft_consensus.cc:2468] T 8158bbd17b994d2eb96f560bd0f1b88d P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 5.
I20260709 13:19:52.131382 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2794af5b99984efebdee564035841cb0" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:19:52.132105 16150 raft_consensus.cc:2468] T 2794af5b99984efebdee564035841cb0 P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 3.
I20260709 13:19:52.133208 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5718c623ebee4d2bb770f0f9ddb2f164" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:19:52.121392 16293 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48684 (request call id 1992) took 2536 ms. Trace:
I20260709 13:19:52.135040 16293 rpcz_store.cc:276] 0709 13:19:49.585265 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:50.550701 (+965436us) service_pool.cc:224] Handling call
0709 13:19:50.648048 (+ 97347us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:50.648480 (+   432us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:50.648517 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:50.649095 (+   578us) log.cc:844] Serialized 26 byte log entry
0709 13:19:51.108588 (+459493us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:19:51.111457 (+  2869us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:51.123071 (+ 11614us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:52.096323 (+973252us) raft_consensus.cc:1685] finished
0709 13:19:52.096617 (+   294us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:52.110690 (+ 14073us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":621,"raft.run_cpu_time_us":69,"raft.run_wall_time_us":70,"thread_start_us":422126,"threads_started":1,"wal-append.queue_time_us":485277}
W20260709 13:19:52.136160 16819 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49896 (request call id 837) took 1945 ms (client timeout 1881 ms). Trace:
W20260709 13:19:52.136745 16819 rpcz_store.cc:269] 0709 13:19:50.191227 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:50.735652 (+544425us) service_pool.cc:224] Handling call
0709 13:19:52.135867 (+1400215us) spinlock_profiling.cc:243] Waited 134 ms on lock 0x7b54007ffdd4. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:19:52.136061 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":308527360}
I20260709 13:19:52.138053 16819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31969392ccdb43b491911a319e8bc7e1" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:19:52.140213 16147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a9de347c711477e87c582d003f36925" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:19:52.140926 16147 raft_consensus.cc:2468] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 5.
I20260709 13:19:52.141855 16147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02838c86531d4f98bf58d1c22b8baf77" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:19:52.142480 16147 raft_consensus.cc:2468] T 02838c86531d4f98bf58d1c22b8baf77 P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 5.
I20260709 13:19:51.531210 18123 raft_consensus.cc:515] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:52.074170 16824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada5a004ba7c46819051e3ccec71ffdb" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:19:51.725606 18057 leader_election.cc:290] T c0eab25ce4c64e018a041beeda2867db P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 5 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:51.889827 18453 consensus_queue.cc:260] T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:52.047410 18255 raft_consensus.cc:740] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:19:51.794044 18433 raft_consensus.cc:493] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:19:52.052850 17992 leader_election.cc:290] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 5 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:19:52.053277 16143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a96373bb0e14dce8a266b2fc6c58507" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:19:52.162220 16146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ad5dbfa99ee4b8d9e28be92b3f36db5" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:19:52.053161 18418 leader_election.cc:290] T 26354361e9b044cea1c2b86c4882fe92 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:19:52.094058 16145 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46224 (request call id 308) took 1094 ms. Trace:
I20260709 13:19:52.144603 18495 raft_consensus.cc:2749] T 434bff3082d141838068f323bdb0d1ed P cb82cc7bd6db46d980c75237a1402860 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:19:51.529541 18487 raft_consensus.cc:515] T ee9ed67006144797b1de6ea8c2013a68 P cb82cc7bd6db46d980c75237a1402860 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:52.096063 16224 rpcz_store.cc:276] 0709 13:19:50.001992 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:51.059577 (+1057585us) service_pool.cc:224] Handling call
0709 13:19:51.059917 (+   340us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:51.086818 (+ 26901us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:51.086865 (+    47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:51.092140 (+  5275us) log.cc:844] Serialized 26 byte log entry
0709 13:19:51.240398 (+148258us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:19:51.240437 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:51.240502 (+    65us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:52.095444 (+854942us) raft_consensus.cc:1685] finished
0709 13:19:52.095462 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:52.095584 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":20913,"raft.queue_time_us":10589,"raft.run_cpu_time_us":113,"raft.run_wall_time_us":4678,"thread_start_us":147876,"threads_started":1,"wal-append.queue_time_us":265632}
I20260709 13:19:52.184698 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a959dbecc2648eb85d1cc550f5a3245" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:19:52.185271 16224 raft_consensus.cc:2376] T 5a959dbecc2648eb85d1cc550f5a3245 P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 4. Re-sending same reply.
I20260709 13:19:52.047931 18347 raft_consensus.cc:697] T 5a959dbecc2648eb85d1cc550f5a3245 P 90f3da57796b488c9087a2f523adf045 [term 4 LEADER]: Becoming Leader. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:19:52.016525 18562 consensus_queue.cc:1059] T 3884f864a96d428785895e5463e5728f P 647a343a542d4a139fe106c19095129b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.083s
I20260709 13:19:51.458055 18456 raft_consensus.cc:515] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:19:52.103868 16226 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.979s	user 0.005s	sys 0.021s
I20260709 13:19:52.101099 16825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:19:52.095443 16292 raft_consensus.cc:2393] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 in current term 3: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
W20260709 13:19:52.079494 16816 rpcz_store.cc:269] 0709 13:19:49.704468 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:49.706407 (+  1939us) service_pool.cc:224] Handling call
0709 13:19:51.883890 (+2177483us) spinlock_profiling.cc:243] Waited 84 ms on lock 0x7b54005302d4. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:19:51.884029 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":193150208}
I20260709 13:19:52.115636 18450 leader_election.cc:290] T 84f2da98864b4425bf7c0055951fdd4b P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:52.116120 18406 raft_consensus.cc:515] T 9c61a39fd4f84424b129c1a6f9ba404d P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:52.057366 16821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9944d5e4b8c4109af9955c4f08c9095" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:19:52.050763 18513 consensus_queue.cc:260] T 68a6838d82ef4cac83507f906139d90f P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:52.120584 18431 leader_election.cc:290] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
W20260709 13:19:52.183301 18515 consensus_meta.cc:287] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 647a343a542d4a139fe106c19095129b: Time spent flushing consensus metadata: real 1.038s	user 0.007s	sys 0.004s
I20260709 13:19:52.144044 16220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9944d5e4b8c4109af9955c4f08c9095" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:19:52.248225 16220 raft_consensus.cc:3060] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:19:52.151671 16144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 5 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:19:52.249390 16144 raft_consensus.cc:2468] T ecc1e363230043fe9fef390f0b979f20 P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 4.
I20260709 13:19:52.156425 16824 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49892 (request call id 368) took 1092 ms. Trace:
W20260709 13:19:52.121773 18485 consensus_meta.cc:287] T 9e65676b21b748728ad39668cfe722ec P 90f3da57796b488c9087a2f523adf045: Time spent flushing consensus metadata: real 0.925s	user 0.009s	sys 0.003s
W20260709 13:19:52.157866 16191 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.536s	user 0.010s	sys 0.008s
I20260709 13:19:52.158855 16147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4dc6b0541ca431ebc757beac00521f1" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:19:52.100558 18494 raft_consensus.cc:2749] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:19:52.162968 16146 raft_consensus.cc:2468] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 4.
W20260709 13:19:52.283332 16191 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.662s	user 0.011s	sys 0.008s
I20260709 13:19:52.283556 16147 raft_consensus.cc:2468] T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 4.
W20260709 13:19:52.283725 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 662497us
I20260709 13:19:52.284531 16147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f5f89186204242bf377f7c42b2e2da" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 7 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:19:52.228760 16143 raft_consensus.cc:2393] T 5a96373bb0e14dce8a266b2fc6c58507 P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 4: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
I20260709 13:19:52.019062 18432 raft_consensus.cc:493] T 57a744f5aaf944678a63469cbf514015 P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
W20260709 13:19:52.167338 16116 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.758s	user 0.062s	sys 0.120s
W20260709 13:19:52.289983 16116 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.881s	user 0.062s	sys 0.120s
W20260709 13:19:52.290377 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 881684us
W20260709 13:19:52.175551 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 242417us
I20260709 13:19:52.164923 16145 rpcz_store.cc:276] 0709 13:19:51.000163 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:51.020742 (+ 20579us) service_pool.cc:224] Handling call
0709 13:19:51.069810 (+ 49068us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:51.220881 (+151071us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:51.220912 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:51.239923 (+ 19011us) log.cc:844] Serialized 26 byte log entry
0709 13:19:51.753749 (+513826us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:19:51.991144 (+237395us) log.cc:844] Serialized 14 byte log entry
0709 13:19:52.010431 (+ 19287us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:52.010548 (+   117us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:52.093805 (+ 83257us) raft_consensus.cc:1685] finished
0709 13:19:52.093833 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:52.094000 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":47519,"prepare.queue_time_us":222022,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":40,"raft.queue_time_us":137554,"raft.run_cpu_time_us":111,"raft.run_wall_time_us":6171,"spinlock_wait_cycles":60306304,"thread_start_us":625344,"threads_started":2,"wal-append.queue_time_us":362180}
I20260709 13:19:52.152689 18488 raft_consensus.cc:3060] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:19:52.186940 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 236083us
W20260709 13:19:52.215443 16226 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.091s	user 0.005s	sys 0.030s
I20260709 13:19:52.201258 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada5a004ba7c46819051e3ccec71ffdb" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:52.200416 16223 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57604 (request call id 2020) took 2438 ms. Trace:
I20260709 13:19:52.374990 16223 rpcz_store.cc:276] 0709 13:19:49.761868 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:49.761992 (+   124us) service_pool.cc:224] Handling call
0709 13:19:49.769596 (+  7604us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:50.443309 (+673713us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:52.200157 (+1756848us) spinlock_profiling.cc:243] Waited 635 ms on lock 0x7b54002c8b54. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:19:52.200353 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1461394816}
I20260709 13:19:52.376228 16223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5ad4eb281564936a59f41e387a47156" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:52.376498 16145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f2da98864b4425bf7c0055951fdd4b" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:19:52.376873 16223 raft_consensus.cc:2393] T e5ad4eb281564936a59f41e387a47156 P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 5: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
I20260709 13:19:52.158388 18400 raft_consensus.cc:3055] T c9944d5e4b8c4109af9955c4f08c9095 P cb82cc7bd6db46d980c75237a1402860 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:19:52.215443 16825 raft_consensus.cc:2468] T 19baa7bc367844c288bca8f81439c413 P cb82cc7bd6db46d980c75237a1402860 [term 5 LEARNER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 5.
I20260709 13:19:52.157372 18557 raft_consensus.cc:2764] T 23f82224f4564fea93572e8206bad1fe P cb82cc7bd6db46d980c75237a1402860 [term 2 FOLLOWER]: Leader pre-election decision vote started in defunct term 1: won
I20260709 13:19:52.079052 18561 raft_consensus.cc:515] T fffab26427e24d398e2695aad40ca134 P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:52.024870 18389 raft_consensus.cc:515] T 9e65676b21b748728ad39668cfe722ec P cb82cc7bd6db46d980c75237a1402860 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:52.189940 16822 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49906 (request call id 525) took 2759 ms. Trace:
I20260709 13:19:52.227236 16820 raft_consensus.cc:2393] T ee9ed67006144797b1de6ea8c2013a68 P cb82cc7bd6db46d980c75237a1402860 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 5: Already voted for candidate cb82cc7bd6db46d980c75237a1402860 in this term.
I20260709 13:19:52.122237 18565 raft_consensus.cc:515] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:19:52.229490 16821 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49892 (request call id 366) took 1209 ms (client timeout 1570 ms). Trace:
W20260709 13:19:52.180545 18574 log.cc:927] Time spent T 57a744f5aaf944678a63469cbf514015 P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 0.089s	user 0.001s	sys 0.000s
I20260709 13:19:52.071637 18272 raft_consensus.cc:493] T 08c74490c40243a88e62d1760b5cd814 P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:19:52.250254 16824 rpcz_store.cc:276] 0709 13:19:51.064070 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:52.055751 (+991681us) service_pool.cc:224] Handling call
0709 13:19:52.156350 (+100599us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:19:52.163585 16266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 407151us
I20260709 13:19:52.186172 16221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fffab26427e24d398e2695aad40ca134" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:52.366591 16144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66f75124215f4ca6bef831e174a792d8" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:19:52.363716 16146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d16dfad0c3b540e5b4b86492ddf87208" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:19:52.371563 16265 leader_election.cc:304] T 8638ee363273426db1393dd81e5c7b72 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:19:52.372718 16143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6f07842d3aa47858df5665874c86f3a" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:19:52.285907 18513 raft_consensus.cc:3060] T 68a6838d82ef4cac83507f906139d90f P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:19:52.456441 18272 raft_consensus.cc:515] T 08c74490c40243a88e62d1760b5cd814 P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:52.372877 16816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9944d5e4b8c4109af9955c4f08c9095" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:19:52.305661 16147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746aabf856ab401c9ef5e859371e7b67" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:19:52.282421 18568 raft_consensus.cc:493] T 74ca8140f35c4ae29e8a664f6bdfc829 P cb82cc7bd6db46d980c75237a1402860 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:52.183605 18433 raft_consensus.cc:515] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:52.381705 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:52.425483 16822 rpcz_store.cc:276] 0709 13:19:49.430332 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:49.430460 (+   128us) service_pool.cc:224] Handling call
0709 13:19:49.439267 (+  8807us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:50.591203 (+1151936us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:52.189690 (+1598487us) spinlock_profiling.cc:243] Waited 1.05 s on lock 0x7b54004b5d54. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:19:52.189878 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2406977792}
W20260709 13:19:52.381392 16149 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46224 (request call id 298) took 2624 ms (client timeout 2237 ms). Trace:
W20260709 13:19:52.429375 16823 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49892 (request call id 357) took 2265 ms (client timeout 2015 ms). Trace:
W20260709 13:19:52.430122 16821 rpcz_store.cc:269] 0709 13:19:51.019936 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:52.020786 (+1000850us) service_pool.cc:224] Handling call
0709 13:19:52.229424 (+208638us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:52.434147 16790 leader_election.cc:304] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:19:52.440151 16825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9944d5e4b8c4109af9955c4f08c9095" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:19:52.247795 18495 consensus_queue.cc:1048] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.130s
I20260709 13:19:52.450026 16266 leader_election.cc:304] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
I20260709 13:19:52.461339 16146 raft_consensus.cc:2468] T d16dfad0c3b540e5b4b86492ddf87208 P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 4.
W20260709 13:19:52.461453 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 89989.2us
I20260709 13:19:52.364398 18255 consensus_queue.cc:260] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:52.288211 18485 raft_consensus.cc:515] T 9e65676b21b748728ad39668cfe722ec P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:52.374125 16819 raft_consensus.cc:2468] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 4.
W20260709 13:19:52.422088 16217 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57604 (request call id 2023) took 2650 ms (client timeout 2048 ms). Trace:
W20260709 13:19:52.684684 16115 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:52.648939 16145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5d59407eb545b895d8d922e0dd69fd" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
W20260709 13:19:52.463832 18114 consensus_meta.cc:287] T 5a96373bb0e14dce8a266b2fc6c58507 P 1ca2725d290e47d68b1a4b609c9816ac: Time spent flushing consensus metadata: real 0.952s	user 0.010s	sys 0.002s
W20260709 13:19:52.491335 16189 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:52.480918 16117 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:52.738765 16265 leader_election.cc:304] T ecc1e363230043fe9fef390f0b979f20 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
W20260709 13:19:52.756738 16117 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:52.757172 16117 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:52.471112 16824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2794af5b99984efebdee564035841cb0" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
W20260709 13:19:52.741456 16266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 291516us
I20260709 13:19:52.492038 16144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87ac0d1f7e8a419f9bcbc04aee0e067e" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:19:52.757977 16821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2794af5b99984efebdee564035841cb0" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:19:52.695326 16220 raft_consensus.cc:2468] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 6.
I20260709 13:19:52.740363 16147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "829b4ad8c5a54eababeea1181b960d4b" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:19:52.759439 16147 raft_consensus.cc:3060] T 829b4ad8c5a54eababeea1181b960d4b P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:19:52.491829 16150 raft_consensus.cc:3060] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:19:52.759799 16220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4dc6b0541ca431ebc757beac00521f1" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:52.470172 16223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a96373bb0e14dce8a266b2fc6c58507" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
W20260709 13:19:52.739398 16823 rpcz_store.cc:269] 0709 13:19:50.164247 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:50.713390 (+549143us) service_pool.cc:224] Handling call
0709 13:19:52.429198 (+1715808us) spinlock_profiling.cc:243] Waited 15.9 ms on lock 0x7b54007553d4. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:19:52.429305 (+   107us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":36539264}
I20260709 13:19:52.760586 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d16dfad0c3b540e5b4b86492ddf87208" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
W20260709 13:19:52.731194 16149 rpcz_store.cc:269] 0709 13:19:49.758093 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:49.760966 (+  2873us) service_pool.cc:224] Handling call
0709 13:19:52.381225 (+2620259us) spinlock_profiling.cc:243] Waited 460 ms on lock 0x7b540020c454. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:19:52.381327 (+   102us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1058015104}
I20260709 13:19:52.676570 16226 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57604 (request call id 2025) took 2899 ms. Trace:
W20260709 13:19:52.746731 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 312630us
I20260709 13:19:52.762252 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c61a39fd4f84424b129c1a6f9ba404d" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:19:52.763115 16149 raft_consensus.cc:2468] T 9c61a39fd4f84424b129c1a6f9ba404d P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 5.
I20260709 13:19:52.485697 18457 raft_consensus.cc:515] T b87147731a68400e8cce9b879daa7c5d P cb82cc7bd6db46d980c75237a1402860 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:52.750419 18401 raft_consensus.cc:2749] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:19:52.456943 16142 raft_consensus.cc:2468] T fffab26427e24d398e2695aad40ca134 P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 3.
I20260709 13:19:52.765424 18457 leader_election.cc:290] T b87147731a68400e8cce9b879daa7c5d P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 7 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:52.765988 16822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62fe3346cd3c42c4a55e14af9073bca4" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:19:52.766788 16822 raft_consensus.cc:2393] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 7: Already voted for candidate cb82cc7bd6db46d980c75237a1402860 in this term.
I20260709 13:19:52.767189 16822 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49896 (request call id 851) took 1327 ms. Trace:
I20260709 13:19:52.767373 18580 raft_consensus.cc:2749] T ecc1e363230043fe9fef390f0b979f20 P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:19:52.463554 18463 leader_election.cc:290] T 23f82224f4564fea93572e8206bad1fe P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:19:52.767689 16292 raft_consensus.cc:1240] T 02838c86531d4f98bf58d1c22b8baf77 P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 4. Current term is 5. Ops: []
I20260709 13:19:52.693248 18488 raft_consensus.cc:2749] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:19:52.475768 18573 log.cc:927] Time spent T d2ab0ac8347641e6955e9bce7d90c1a2 P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 0.100s	user 0.000s	sys 0.000s
I20260709 13:19:52.691392 18585 raft_consensus.cc:493] T 9e65676b21b748728ad39668cfe722ec P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:52.472908 18575 log.cc:927] Time spent T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 0.349s	user 0.003s	sys 0.000s
I20260709 13:19:52.767513 16822 rpcz_store.cc:276] 0709 13:19:51.439708 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:52.747886 (+1308178us) service_pool.cc:224] Handling call
0709 13:19:52.767111 (+ 19225us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:52.689883 16116 leader_election.cc:304] T 8cc959fb7f704fba9009c0595467d074 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 90f3da57796b488c9087a2f523adf045
I20260709 13:19:52.739413 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66ae176961b74813a58bc15c9c4c6f2a" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:52.691546 18558 raft_consensus.cc:493] T 7c2095a5f4ad49408dbc7f69c606120d P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:52.769153 16790 leader_election.cc:304] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:19:52.769721 16824 raft_consensus.cc:2393] T 2794af5b99984efebdee564035841cb0 P cb82cc7bd6db46d980c75237a1402860 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 4: Already voted for candidate cb82cc7bd6db46d980c75237a1402860 in this term.
I20260709 13:19:52.380021 18400 raft_consensus.cc:740] T c9944d5e4b8c4109af9955c4f08c9095 P cb82cc7bd6db46d980c75237a1402860 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:19:52.769682 16265 leader_election.cc:304] T ada5a004ba7c46819051e3ccec71ffdb P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860
I20260709 13:19:52.380071 18417 leader_election.cc:290] T 8a878999c0024d2abe517484180290fc P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:52.751600 16825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdbd78acb4cf46298e0de8ba0d130e19" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:19:52.688155 18421 raft_consensus.cc:493] T af57996a8be84d398f0f023e8d08ef77 P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:19:52.778370 18463 raft_consensus.cc:2764] T 8cc959fb7f704fba9009c0595467d074 P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Leader pre-election decision vote started in defunct term 4: won
I20260709 13:19:52.772959 16117 leader_election.cc:304] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac
I20260709 13:19:52.765415 18272 leader_election.cc:290] T 08c74490c40243a88e62d1760b5cd814 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:52.730837 18565 leader_election.cc:290] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:19:52.489600 18487 leader_election.cc:290] T ee9ed67006144797b1de6ea8c2013a68 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 5 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:19:52.470206 16218 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57604 (request call id 2031) took 1843 ms. Trace:
I20260709 13:19:52.769069 16297 raft_consensus.cc:2393] T d16dfad0c3b540e5b4b86492ddf87208 P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 4: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
I20260709 13:19:52.471050 18123 leader_election.cc:290] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:19:52.747846 18582 raft_consensus.cc:2749] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:19:52.634490 18406 leader_election.cc:290] T 9c61a39fd4f84424b129c1a6f9ba404d P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:52.648566 18456 leader_election.cc:290] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:52.378443 18432 raft_consensus.cc:515] T 57a744f5aaf944678a63469cbf514015 P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:52.775430 16146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2372981006d4b218215c28964dea623" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
W20260709 13:19:52.776202 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 86437us
I20260709 13:19:52.773662 16823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5d59407eb545b895d8d922e0dd69fd" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:19:52.832365 16116 leader_election.cc:304] T b6f07842d3aa47858df5665874c86f3a P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
I20260709 13:19:52.832648 16823 raft_consensus.cc:3060] T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:19:52.796440 18488 raft_consensus.cc:2749] T ada5a004ba7c46819051e3ccec71ffdb P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:19:52.777032 16790 leader_election.cc:304] T d2ab0ac8347641e6955e9bce7d90c1a2 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:19:52.640053 16817 raft_consensus.cc:2468] T 9e65676b21b748728ad39668cfe722ec P cb82cc7bd6db46d980c75237a1402860 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 5.
I20260709 13:19:52.777153 16821 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49896 (request call id 850) took 2268 ms. Trace:
I20260709 13:19:52.843479 16821 rpcz_store.cc:276] 0709 13:19:50.508444 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:52.749851 (+2241407us) service_pool.cc:224] Handling call
0709 13:19:52.777068 (+ 27217us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:19:52.846715 16047 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:49250 (request call id 319) took 15533 ms (client timeout 15000 ms). Trace:
I20260709 13:19:52.784132 16145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d4ab870a05347868956aa8b841b2981" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:19:52.894255 16145 raft_consensus.cc:2468] T 8d4ab870a05347868956aa8b841b2981 P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 5.
I20260709 13:19:52.783823 16822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31969392ccdb43b491911a319e8bc7e1" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
W20260709 13:19:52.479148 16219 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57604 (request call id 2028) took 2577 ms (client timeout 1538 ms). Trace:
I20260709 13:19:52.786823 16147 raft_consensus.cc:2468] T 829b4ad8c5a54eababeea1181b960d4b P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 6.
I20260709 13:19:52.790427 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8158bbd17b994d2eb96f560bd0f1b88d" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:52.791034 16296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806d7690c2b644959152074bb2cab0a7" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:52.747641 16143 raft_consensus.cc:2468] T b6f07842d3aa47858df5665874c86f3a P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 4.
I20260709 13:19:52.647300 18578 raft_consensus.cc:2804] T 8638ee363273426db1393dd81e5c7b72 P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:19:52.761819 16226 rpcz_store.cc:276] 0709 13:19:49.777548 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:51.058265 (+1280717us) service_pool.cc:224] Handling call
0709 13:19:51.108109 (+ 49844us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:51.108374 (+   265us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 13:19:52.374505 (+1266131us) log.cc:844] Serialized 14 byte log entry
0709 13:19:52.675831 (+301326us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:52.675979 (+   148us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:19:52.676117 (+   138us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:52.676299 (+   182us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:52.676488 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4768,"prepare.queue_time_us":1067013,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":40,"raft.queue_time_us":8882,"raft.run_cpu_time_us":177,"raft.run_wall_time_us":7892,"thread_start_us":1550349,"threads_started":2,"wal-append.queue_time_us":376115}
I20260709 13:19:52.786855 16825 raft_consensus.cc:2468] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 5.
I20260709 13:19:52.905099 16825 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49892 (request call id 373) took 1463 ms. Trace:
I20260709 13:19:52.905431 16825 rpcz_store.cc:276] 0709 13:19:51.441774 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:52.750210 (+1308436us) service_pool.cc:224] Handling call
0709 13:19:52.905046 (+154836us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:52.906286 16825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2372981006d4b218215c28964dea623" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:19:52.430083 18347 consensus_queue.cc:237] T 5a959dbecc2648eb85d1cc550f5a3245 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:19:52.800876 16192 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 150142us
I20260709 13:19:52.798710 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f5f89186204242bf377f7c42b2e2da" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 7 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:19:52.783828 18457 raft_consensus.cc:2749] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:19:52.785468 16816 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49892 (request call id 371) took 1344 ms. Trace:
I20260709 13:19:52.806515 16220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4f65cdf00b748bb963ee6e81ef43b79" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:19:52.915603 16816 rpcz_store.cc:276] 0709 13:19:51.441093 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:52.372566 (+931473us) service_pool.cc:224] Handling call
0709 13:19:52.777609 (+405043us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:52.915974 16220 raft_consensus.cc:3060] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:19:52.916883 16816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada5a004ba7c46819051e3ccec71ffdb" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:19:52.917740 16816 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49906 (request call id 535) took 1031 ms. Trace:
I20260709 13:19:52.918088 16816 rpcz_store.cc:276] 0709 13:19:51.886128 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:52.916577 (+1030449us) service_pool.cc:224] Handling call
0709 13:19:52.917679 (+  1102us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:52.743935 18568 raft_consensus.cc:515] T 74ca8140f35c4ae29e8a664f6bdfc829 P cb82cc7bd6db46d980c75237a1402860 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:19:52.897032 16219 rpcz_store.cc:269] 0709 13:19:49.900558 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:49.927712 (+ 27154us) service_pool.cc:224] Handling call
0709 13:19:52.477309 (+2549597us) spinlock_profiling.cc:243] Waited 508 ms on lock 0x7b5400396054. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:19:52.477427 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1168737024}
I20260709 13:19:52.779222 16824 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49892 (request call id 370) took 1338 ms. Trace:
I20260709 13:19:52.832098 16146 raft_consensus.cc:2468] T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 4.
I20260709 13:19:52.920682 16824 rpcz_store.cc:276] 0709 13:19:51.440847 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:52.470829 (+1029982us) service_pool.cc:224] Handling call
0709 13:19:52.779146 (+308317us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:19:52.838021 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 61067.1us
I20260709 13:19:52.922169 16790 leader_election.cc:304] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:19:52.771008 18114 raft_consensus.cc:515] T 5a96373bb0e14dce8a266b2fc6c58507 P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:52.835958 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c61a39fd4f84424b129c1a6f9ba404d" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
W20260709 13:19:52.748329 16217 rpcz_store.cc:269] 0709 13:19:49.771478 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:49.940151 (+168673us) service_pool.cc:224] Handling call
0709 13:19:52.362924 (+2422773us) spinlock_profiling.cc:243] Waited 1.21 s on lock 0x7b54002b4654. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:19:52.380820 (+ 17896us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2774978944}
I20260709 13:19:52.839561 18456 raft_consensus.cc:2749] T d2ab0ac8347641e6955e9bce7d90c1a2 P cb82cc7bd6db46d980c75237a1402860 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:19:52.840853 16299 raft_consensus.cc:3060] T 18ad7e3af5504571a69d10c00750a01c P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:19:52.841632 16291 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48684 (request call id 1991) took 3265 ms. Trace:
I20260709 13:19:52.486055 16221 raft_consensus.cc:2468] T fffab26427e24d398e2695aad40ca134 P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 3.
I20260709 13:19:52.847961 16044 catalog_manager.cc:5697] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b reported cstate change: term changed from 3 to 4, leader changed from 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130) to 647a343a542d4a139fe106c19095129b (127.14.177.129), now has a pending config: VOTER 647a343a542d4a139fe106c19095129b (127.14.177.129), VOTER 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130), VOTER 90f3da57796b488c9087a2f523adf045 (127.14.177.131), VOTER cb82cc7bd6db46d980c75237a1402860 (127.14.177.132). New cstate: current_term: 4 leader_uuid: "647a343a542d4a139fe106c19095129b" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } health_report { overall_health: UNKNOWN } } } pending_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:19:52.851814 18582 raft_consensus.cc:493] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:52.833040 18463 raft_consensus.cc:2804] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:19:52.852514 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cc959fb7f704fba9009c0595467d074" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:19:52.806154 18421 raft_consensus.cc:515] T af57996a8be84d398f0f023e8d08ef77 P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:52.856779 18432 leader_election.cc:290] T 57a744f5aaf944678a63469cbf514015 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:19:52.857450 18459 raft_consensus.cc:3060] T af57996a8be84d398f0f023e8d08ef77 P cb82cc7bd6db46d980c75237a1402860 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:19:52.804112 16218 rpcz_store.cc:276] 0709 13:19:50.626718 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:51.216535 (+589817us) service_pool.cc:224] Handling call
0709 13:19:51.261345 (+ 44810us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:51.292039 (+ 30694us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:19:52.073655 (+781616us) log.cc:844] Serialized 14 byte log entry
0709 13:19:52.247721 (+174066us) log.cc:844] Serialized 14 byte log entry
0709 13:19:52.397382 (+149661us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:52.424232 (+ 26850us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:19:52.450640 (+ 26408us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:52.460827 (+ 10187us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:52.470142 (+  9315us) inbound_call.cc:177] Queueing success response
0709 13:19:52.474921 (+  4779us) spinlock_profiling.cc:243] Waited 4.58 ms on lock 0x7b18004458a0. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b693ade1 00007f69b69475b1 00007f69ada723f3 00007f69ada747c1 00007f69ada74779 00007f69ada74709 00007f69ada746d1 00007f69ada739cd 000000000038f354 000000000038f199 00007f69ada5583a
Metrics: {"mutex_wait_us":79175,"prepare.queue_time_us":916467,"prepare.run_wall_time_us":37,"raft.queue_time_us":140780,"raft.run_cpu_time_us":129,"raft.run_wall_time_us":13629,"spinlock_wait_cycles":47003904,"thread_start_us":797651,"threads_started":2,"wal-append.queue_time_us":293836}
I20260709 13:19:52.788168 18585 raft_consensus.cc:515] T 9e65676b21b748728ad39668cfe722ec P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:52.831449 16294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d4ab870a05347868956aa8b841b2981" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:52.784873 18558 raft_consensus.cc:515] T 7c2095a5f4ad49408dbc7f69c606120d P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:52.838469 16116 leader_election.cc:304] T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
I20260709 13:19:52.967717 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdbd78acb4cf46298e0de8ba0d130e19" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:52.870363 16148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cc959fb7f704fba9009c0595467d074" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
W20260709 13:19:52.968215 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 129812us
I20260709 13:19:52.968580 16148 raft_consensus.cc:2393] T 8cc959fb7f704fba9009c0595467d074 P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 5: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
W20260709 13:19:52.872836 16820 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49896 (request call id 847) took 2417 ms (client timeout 2048 ms). Trace:
W20260709 13:19:52.968904 16820 rpcz_store.cc:269] 0709 13:19:50.455656 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:51.992976 (+1537320us) service_pool.cc:224] Handling call
0709 13:19:52.872614 (+879638us) spinlock_profiling.cc:243] Waited 104 ms on lock 0x7b5400721854. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:19:52.872765 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":238413184}
I20260709 13:19:52.873786 16142 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40830 (request call id 1119) took 1379 ms. Trace:
I20260709 13:19:52.846997 16823 raft_consensus.cc:2468] T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 6.
I20260709 13:19:52.970021 16823 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49906 (request call id 533) took 1084 ms. Trace:
I20260709 13:19:52.920912 18485 leader_election.cc:290] T 9e65676b21b748728ad39668cfe722ec P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 6 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:19:52.882131 18433 leader_election.cc:290] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:19:52.884614 16222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a878999c0024d2abe517484180290fc" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
W20260709 13:19:52.885679 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 116060us
I20260709 13:19:52.977237 16265 leader_election.cc:304] T e5ad4eb281564936a59f41e387a47156 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:19:52.938755 18495 raft_consensus.cc:2749] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:19:52.978685 16265 leader_election.cc:304] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:19:52.888465 16144 raft_consensus.cc:2468] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 5.
I20260709 13:19:52.958814 18463 raft_consensus.cc:493] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:19:52.893601 16266 leader_election.cc:336] T a378d7f796f94615b3b347119a9880fd P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923): Timed out: RequestConsensusVote RPC to 127.14.177.132:38923 timed out after 2.040s (SENT)
W20260709 13:19:52.985399 16266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 91825.1us
I20260709 13:19:52.902441 16822 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49892 (request call id 369) took 1831 ms. Trace:
I20260709 13:19:52.904969 16296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87147731a68400e8cce9b879daa7c5d" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:19:52.979697 16820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746aabf856ab401c9ef5e859371e7b67" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:19:52.986832 16820 raft_consensus.cc:2468] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 4.
I20260709 13:19:52.987097 16296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdbd78acb4cf46298e0de8ba0d130e19" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:52.795789 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada5a004ba7c46819051e3ccec71ffdb" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:52.987725 16296 raft_consensus.cc:2468] T cdbd78acb4cf46298e0de8ba0d130e19 P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 5.
I20260709 13:19:52.988178 16300 raft_consensus.cc:2468] T ada5a004ba7c46819051e3ccec71ffdb P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 5.
I20260709 13:19:52.988543 16296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e4bfb9090c8460eb14fa7e0d818e7f1" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:52.988960 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c11074579b046d99e585c2818e18a2c" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:52.906931 16825 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49906 (request call id 534) took 1021 ms. Trace:
I20260709 13:19:52.989542 16300 raft_consensus.cc:2468] T 8c11074579b046d99e585c2818e18a2c P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 5.
I20260709 13:19:52.989516 16825 rpcz_store.cc:276] 0709 13:19:51.885832 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:52.906071 (+1020239us) service_pool.cc:224] Handling call
0709 13:19:52.906884 (+   813us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:52.912266 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18ad7e3af5504571a69d10c00750a01c" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:52.990461 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2794af5b99984efebdee564035841cb0" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:19:52.923789 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d4ab870a05347868956aa8b841b2981" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:52.990962 16300 raft_consensus.cc:3060] T 2794af5b99984efebdee564035841cb0 P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:19:52.991560 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6f07842d3aa47858df5665874c86f3a" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
W20260709 13:19:52.935895 16192 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:52.935132 16790 leader_election.cc:304] T 3a7ca61592e848ea9b0fb81a2ce33cdc P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:19:52.941964 16220 raft_consensus.cc:2468] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 7.
I20260709 13:19:52.957412 16225 raft_consensus.cc:2376] T 8cc959fb7f704fba9009c0595467d074 P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 5. Re-sending same reply.
I20260709 13:19:52.958997 16824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87147731a68400e8cce9b879daa7c5d" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 7 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:19:52.908628 18406 raft_consensus.cc:2804] T b6f07842d3aa47858df5665874c86f3a P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:19:52.904539 18578 raft_consensus.cc:493] T 8638ee363273426db1393dd81e5c7b72 P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Starting leader election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:19:52.964728 16146 raft_consensus.cc:1275] T 9c61a39fd4f84424b129c1a6f9ba404d P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:19:52.966361 16223 raft_consensus.cc:2393] T 5a96373bb0e14dce8a266b2fc6c58507 P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 4: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
I20260709 13:19:52.970324 16823 rpcz_store.cc:276] 0709 13:19:51.885494 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:52.773436 (+887942us) service_pool.cc:224] Handling call
0709 13:19:52.969968 (+196532us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:52.881517 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a378d7f796f94615b3b347119a9880fd" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:52.890645 16150 raft_consensus.cc:2468] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 5.
I20260709 13:19:52.980100 16265 leader_election.cc:304] T c9944d5e4b8c4109af9955c4f08c9095 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:19:52.972651 16116 leader_election.cc:304] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860
I20260709 13:19:52.985983 16822 rpcz_store.cc:276] 0709 13:19:51.071306 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:52.775876 (+1704570us) service_pool.cc:224] Handling call
0709 13:19:52.902383 (+126507us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:52.907816 16145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806d7690c2b644959152074bb2cab0a7" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:19:53.007649 16823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a378d7f796f94615b3b347119a9880fd" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:19:53.007797 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4dc6b0541ca431ebc757beac00521f1" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:52.978127 16298 raft_consensus.cc:2468] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 7.
I20260709 13:19:53.009088 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 5 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:53.009318 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a878999c0024d2abe517484180290fc" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:53.009672 16298 raft_consensus.cc:2468] T ecc1e363230043fe9fef390f0b979f20 P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 4.
I20260709 13:19:53.010365 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0eab25ce4c64e018a041beeda2867db" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
W20260709 13:19:52.995599 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 60582.2us
I20260709 13:19:52.956681 18535 consensus_queue.cc:1059] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.040s
I20260709 13:19:52.993266 16192 leader_election.cc:304] T 746aabf856ab401c9ef5e859371e7b67 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:19:53.013864 16220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02838c86531d4f98bf58d1c22b8baf77" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:52.977661 16816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a9de347c711477e87c582d003f36925" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:19:53.017726 16825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "829b4ad8c5a54eababeea1181b960d4b" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:19:53.018215 16822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74ca8140f35c4ae29e8a664f6bdfc829" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:19:53.002542 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2794af5b99984efebdee564035841cb0" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:53.003441 16820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66f75124215f4ca6bef831e174a792d8" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:19:53.019800 16820 raft_consensus.cc:2468] T 66f75124215f4ca6bef831e174a792d8 P cb82cc7bd6db46d980c75237a1402860 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 5.
I20260709 13:19:53.001998 16146 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40830 (request call id 1127) took 1504 ms. Trace:
I20260709 13:19:53.020726 16146 rpcz_store.cc:276] 0709 13:19:51.497212 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:52.921497 (+1424285us) service_pool.cc:224] Handling call
0709 13:19:52.959223 (+ 37726us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:53.001746 (+ 42523us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:53.001945 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:53.010468 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57a744f5aaf944678a63469cbf514015" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:53.022804 16298 raft_consensus.cc:2410] T 57a744f5aaf944678a63469cbf514015 P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 for term 4 because replica has last-logged OpId of term: 3 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 13:19:52.993974 16824 raft_consensus.cc:2393] T b87147731a68400e8cce9b879daa7c5d P cb82cc7bd6db46d980c75237a1402860 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 7: Already voted for candidate cb82cc7bd6db46d980c75237a1402860 in this term.
I20260709 13:19:53.004748 18456 raft_consensus.cc:2749] T 3a7ca61592e848ea9b0fb81a2ce33cdc P cb82cc7bd6db46d980c75237a1402860 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:19:53.013943 16145 raft_consensus.cc:1240] T 9c61a39fd4f84424b129c1a6f9ba404d P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 4. Current term is 5. Ops: []
I20260709 13:19:53.025822 16145 raft_consensus.cc:1275] T 8a878999c0024d2abe517484180290fc P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:19:52.976867 18417 raft_consensus.cc:2804] T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:19:53.014616 16220 raft_consensus.cc:2468] T 02838c86531d4f98bf58d1c22b8baf77 P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 5.
I20260709 13:19:53.014672 16823 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49906 (request call id 532) took 1129 ms. Trace:
I20260709 13:19:53.026979 16823 rpcz_store.cc:276] 0709 13:19:51.885093 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:53.007419 (+1122326us) service_pool.cc:224] Handling call
0709 13:19:53.014618 (+  7199us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:52.939318 18450 consensus_queue.cc:1048] T 19baa7bc367844c288bca8f81439c413 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.045s
W20260709 13:19:52.996346 16266 connection.cc:256] RPC call timeout handler was delayed by -1.4305933119999281s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2073s for a response
I20260709 13:19:52.989100 16296 raft_consensus.cc:2468] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 3.
I20260709 13:19:53.023463 16824 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49896 (request call id 854) took 1582 ms. Trace:
I20260709 13:19:53.028855 16824 rpcz_store.cc:276] 0709 13:19:51.440609 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:52.921627 (+1481018us) service_pool.cc:224] Handling call
0709 13:19:53.023411 (+101784us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:19:53.015060 16790 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.14.177.129:36741 timed out after 2.237s (SENT) [suppressed 21 similar messages]
I20260709 13:19:53.015600 16044 catalog_manager.cc:5697] T 68a6838d82ef4cac83507f906139d90f P 647a343a542d4a139fe106c19095129b reported cstate change: term changed from 4 to 5, leader changed from 90f3da57796b488c9087a2f523adf045 (127.14.177.131) to 647a343a542d4a139fe106c19095129b (127.14.177.129). New cstate: current_term: 5 leader_uuid: "647a343a542d4a139fe106c19095129b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } health_report { overall_health: UNKNOWN } } }
I20260709 13:19:53.016686 16265 leader_election.cc:304] T 66ae176961b74813a58bc15c9c4c6f2a P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860
I20260709 13:19:53.000823 16300 raft_consensus.cc:2410] T 2794af5b99984efebdee564035841cb0 P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 for term 4 because replica has last-logged OpId of term: 2 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 13:19:52.997207 18463 raft_consensus.cc:3060] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:19:52.995934 18433 raft_consensus.cc:3060] T e5ad4eb281564936a59f41e387a47156 P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:19:53.033865 16265 leader_election.cc:304] T fffab26427e24d398e2695aad40ca134 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:19:53.018878 16822 raft_consensus.cc:2376] T 74ca8140f35c4ae29e8a664f6bdfc829 P cb82cc7bd6db46d980c75237a1402860 [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 5. Re-sending same reply.
I20260709 13:19:52.960626 18582 raft_consensus.cc:515] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:53.033563 18457 raft_consensus.cc:493] T 26354361e9b044cea1c2b86c4882fe92 P cb82cc7bd6db46d980c75237a1402860 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:53.006903 18432 raft_consensus.cc:2749] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:19:53.036550 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ad5dbfa99ee4b8d9e28be92b3f36db5" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:53.039716 16823 raft_consensus.cc:3060] T 806d7690c2b644959152074bb2cab0a7 P cb82cc7bd6db46d980c75237a1402860 [term 3 LEARNER]: Advancing to term 4
I20260709 13:19:53.005363 18578 raft_consensus.cc:3060] T 8638ee363273426db1393dd81e5c7b72 P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:19:53.038197 16297 raft_consensus.cc:1240] T 434bff3082d141838068f323bdb0d1ed P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 4. Current term is 5. Ops: []
I20260709 13:19:53.063129 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31969392ccdb43b491911a319e8bc7e1" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:53.063889 16297 raft_consensus.cc:2468] T 31969392ccdb43b491911a319e8bc7e1 P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 4.
I20260709 13:19:53.065212 16297 raft_consensus.cc:1240] T 23f82224f4564fea93572e8206bad1fe P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 1. Current term is 2. Ops: []
I20260709 13:19:52.957831 18400 consensus_queue.cc:260] T c9944d5e4b8c4109af9955c4f08c9095 P cb82cc7bd6db46d980c75237a1402860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:52.965827 18587 consensus_queue.cc:1048] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.058s
I20260709 13:19:53.054759 16265 leader_election.cc:304] T 18ad7e3af5504571a69d10c00750a01c P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:19:53.055332 16044 catalog_manager.cc:5697] T 434bff3082d141838068f323bdb0d1ed P 647a343a542d4a139fe106c19095129b reported cstate change: term changed from 3 to 4, leader changed from cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) to 647a343a542d4a139fe106c19095129b (127.14.177.129). New cstate: current_term: 4 leader_uuid: "647a343a542d4a139fe106c19095129b" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:19:53.036280 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8158bbd17b994d2eb96f560bd0f1b88d" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:52.936664 16291 rpcz_store.cc:276] 0709 13:19:49.576692 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:50.532095 (+955403us) service_pool.cc:224] Handling call
0709 13:19:50.610192 (+ 78097us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:51.134799 (+524607us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:52.841387 (+1706588us) spinlock_profiling.cc:243] Waited 325 ms on lock 0x7b540030bbd4. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:19:52.841572 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":748277632}
I20260709 13:19:53.032550 18485 raft_consensus.cc:2804] T c9944d5e4b8c4109af9955c4f08c9095 P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:19:53.085327 16146 raft_consensus.cc:1275] T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:19:53.092708 16191 leader_election.cc:304] T d16dfad0c3b540e5b4b86492ddf87208 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
I20260709 13:19:52.972645 16142 rpcz_store.cc:276] 0709 13:19:51.494716 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:51.516991 (+ 22275us) service_pool.cc:224] Handling call
0709 13:19:52.873620 (+1356629us) spinlock_profiling.cc:243] Waited 356 ms on lock 0x7b540013ecd4. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:19:52.873728 (+   108us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":819823360}
I20260709 13:19:53.072446 16044 catalog_manager.cc:5697] T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b reported cstate change: term changed from 3 to 6, leader changed from 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130) to 647a343a542d4a139fe106c19095129b (127.14.177.129). New cstate: current_term: 6 leader_uuid: "647a343a542d4a139fe106c19095129b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } health_report { overall_health: UNKNOWN } } }
I20260709 13:19:53.074793 16825 raft_consensus.cc:1240] T 3884f864a96d428785895e5463e5728f P cb82cc7bd6db46d980c75237a1402860 [term 4 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 3. Current term is 4. Ops: [3.6-3.6]
I20260709 13:19:53.075337 16219 raft_consensus.cc:1240] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 3. Current term is 4. Ops: []
W20260709 13:19:53.076465 16817 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49892 (request call id 361) took 2911 ms (client timeout 2073 ms). Trace:
I20260709 13:19:53.102330 16220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ad5dbfa99ee4b8d9e28be92b3f36db5" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:53.053592 18551 raft_consensus.cc:2804] T 746aabf856ab401c9ef5e859371e7b67 P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:19:53.035069 18488 raft_consensus.cc:2749] T 66ae176961b74813a58bc15c9c4c6f2a P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:19:53.077507 18424 raft_consensus.cc:3055] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 647a343a542d4a139fe106c19095129b [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:19:53.077888 16298 raft_consensus.cc:1275] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
W20260709 13:19:53.078783 16819 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49896 (request call id 853) took 1638 ms (client timeout 1681 ms). Trace:
I20260709 13:19:53.079866 18565 raft_consensus.cc:2749] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:19:53.080241 16143 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40830 (request call id 1126) took 1583 ms. Trace:
I20260709 13:19:53.058173 18457 raft_consensus.cc:515] T 26354361e9b044cea1c2b86c4882fe92 P cb82cc7bd6db46d980c75237a1402860 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:53.080657 16823 raft_consensus.cc:1275] T 806d7690c2b644959152074bb2cab0a7 P cb82cc7bd6db46d980c75237a1402860 [term 4 LEARNER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:19:53.033816 18417 raft_consensus.cc:493] T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Starting leader election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:19:53.076713 18582 leader_election.cc:290] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:19:53.104175 16222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a96373bb0e14dce8a266b2fc6c58507" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:19:53.054517 18504 raft_consensus.cc:493] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:53.105818 16825 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49896 (request call id 838) took 2914 ms. Trace:
I20260709 13:19:53.140563 16825 rpcz_store.cc:276] 0709 13:19:50.191568 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:53.058180 (+2866612us) service_pool.cc:224] Handling call
0709 13:19:53.074579 (+ 16399us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:53.105564 (+ 30985us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:53.105760 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:19:53.086789 16190 connection.cc:256] RPC call timeout handler was delayed by -1.5364930180001011s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2034s for a response
I20260709 13:19:53.077534 18272 raft_consensus.cc:3055] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:19:53.061856 18580 raft_consensus.cc:2804] T fffab26427e24d398e2695aad40ca134 P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:19:53.060214 16218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806d7690c2b644959152074bb2cab0a7" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
W20260709 13:19:52.895282 16047 rpcz_store.cc:269] 0709 13:19:37.313531 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:37.313782 (+   251us) service_pool.cc:224] Handling call
0709 13:19:52.846483 (+15532701us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:19:45.030749 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:19:45.044187 (+ 13438us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:19:45.044226 (+    39us) write_op.cc:435] Acquired schema lock
0709 13:19:45.044246 (+    20us) tablet.cc:662] Decoding operations
0709 13:19:45.124738 (+ 80492us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:19:45.134090 (+  9352us) write_op.cc:641] Partition lock acquired for write op
0709 13:19:45.136472 (+  2382us) tablet.cc:685] Acquiring locks for 5 operations
0709 13:19:45.155217 (+ 18745us) tablet.cc:701] Row locks acquired
0709 13:19:45.155239 (+    22us) write_op.cc:260] PREPARE: finished
0709 13:19:45.155383 (+   144us) write_op.cc:270] Start()
0709 13:19:45.155494 (+   111us) write_op.cc:276] Timestamp: P: 1783603185155337 usec, L: 0
0709 13:19:45.155513 (+    19us) op_driver.cc:348] REPLICATION: starting
0709 13:19:45.235491 (+ 79978us) log.cc:844] Serialized 2094 byte log entry
0709 13:19:46.227896 (+992405us) op_driver.cc:464] REPLICATION: finished
0709 13:19:46.494427 (+266531us) write_op.cc:301] APPLY: starting
0709 13:19:46.494535 (+   108us) tablet.cc:1367] starting BulkCheckPresence
0709 13:19:46.494743 (+   208us) tablet.cc:1370] finished BulkCheckPresence
0709 13:19:46.494917 (+   174us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:19:46.496079 (+  1162us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:19:46.496336 (+   257us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=5
0709 13:19:46.496398 (+    62us) write_op.cc:312] APPLY: finished
0709 13:19:46.496707 (+   309us) log.cc:844] Serialized 47 byte log entry
0709 13:19:46.497002 (+   295us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:19:46.497140 (+   138us) write_op.cc:454] Released schema lock
0709 13:19:46.497193 (+    53us) write_op.cc:341] FINISH: Updating metrics
0709 13:19:46.497606 (+   413us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:19:46.497642 (+    36us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":37474,"reported_tablets":22,"child_traces":[["op",{"apply.queue_time_us":253393,"apply.run_cpu_time_us":2921,"apply.run_wall_time_us":3153,"num_ops":5,"prepare.queue_time_us":420721,"prepare.run_cpu_time_us":20745,"prepare.run_wall_time_us":331910,"raft.queue_time_us":367446,"raft.run_cpu_time_us":27400,"raft.run_wall_time_us":331876,"replication_time_us":1015072,"thread_start_us":939389,"threads_started":4,"wal-append.queue_time_us":205178,"wal-append.run_cpu_time_us":41632,"wal-append.run_wall_time_us":2069688}]]}
I20260709 13:19:53.109509 16044 catalog_manager.cc:5697] T 3884f864a96d428785895e5463e5728f P 647a343a542d4a139fe106c19095129b reported cstate change: term changed from 1 to 2, leader changed from 647a343a542d4a139fe106c19095129b (127.14.177.129) to cb82cc7bd6db46d980c75237a1402860 (127.14.177.132). New cstate: current_term: 2 leader_uuid: "cb82cc7bd6db46d980c75237a1402860" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:19:53.071141 18568 leader_election.cc:290] T 74ca8140f35c4ae29e8a664f6bdfc829 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:53.124894 16146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87147731a68400e8cce9b879daa7c5d" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:19:53.143615 16146 raft_consensus.cc:3060] T b87147731a68400e8cce9b879daa7c5d P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:19:53.103281 16816 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49892 (request call id 372) took 1653 ms. Trace:
I20260709 13:19:53.139883 16823 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49892 (request call id 358) took 2963 ms. Trace:
I20260709 13:19:53.106071 18485 raft_consensus.cc:697] T c9944d5e4b8c4109af9955c4f08c9095 P 90f3da57796b488c9087a2f523adf045 [term 6 LEADER]: Becoming Leader. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:19:53.107589 18551 raft_consensus.cc:493] T 746aabf856ab401c9ef5e859371e7b67 P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Starting leader election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:19:53.150189 16823 rpcz_store.cc:276] 0709 13:19:50.164534 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:53.039246 (+2874712us) service_pool.cc:224] Handling call
0709 13:19:53.039580 (+   334us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:53.127925 (+ 88345us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:53.128122 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:19:53.142326 16190 connection.cc:256] RPC call timeout handler was delayed by -1.2323034529999859s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15180000000000002s for a response
I20260709 13:19:53.114476 18424 raft_consensus.cc:740] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 647a343a542d4a139fe106c19095129b [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:19:53.105423 18452 consensus_queue.cc:1048] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.047s
I20260709 13:19:53.152187 16294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4dc6b0541ca431ebc757beac00521f1" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:53.171998 18421 leader_election.cc:290] T af57996a8be84d398f0f023e8d08ef77 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:19:53.121176 18530 raft_consensus.cc:2804] T d16dfad0c3b540e5b4b86492ddf87208 P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:19:53.177124 16222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e4bfb9090c8460eb14fa7e0d818e7f1" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:53.184520 18424 consensus_queue.cc:260] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:53.170691 16296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806d7690c2b644959152074bb2cab0a7" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:53.185756 18424 raft_consensus.cc:3060] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:19:53.186141 16222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee9ed67006144797b1de6ea8c2013a68" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:19:53.186254 16296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a96373bb0e14dce8a266b2fc6c58507" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:19:53.103137 18535 consensus_queue.cc:1059] T 7c2095a5f4ad49408dbc7f69c606120d P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.036s
I20260709 13:19:53.187031 16296 raft_consensus.cc:2393] T 5a96373bb0e14dce8a266b2fc6c58507 P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 4: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
I20260709 13:19:53.183636 18114 leader_election.cc:290] T 5a96373bb0e14dce8a266b2fc6c58507 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 4 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:53.149346 18272 raft_consensus.cc:740] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:19:53.187830 16296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26354361e9b044cea1c2b86c4882fe92" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:53.129495 18513 raft_consensus.cc:493] T 68a6838d82ef4cac83507f906139d90f P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:53.188448 16222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdbd78acb4cf46298e0de8ba0d130e19" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:53.149559 18580 raft_consensus.cc:493] T fffab26427e24d398e2695aad40ca134 P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:53.184877 18530 raft_consensus.cc:493] T d16dfad0c3b540e5b4b86492ddf87208 P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Starting leader election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:19:53.130378 18417 raft_consensus.cc:3060] T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:19:53.122022 16143 rpcz_store.cc:276] 0709 13:19:51.496922 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:52.919431 (+1422509us) service_pool.cc:224] Handling call
0709 13:19:52.919766 (+   335us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:52.920153 (+   387us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:52.920184 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:52.920733 (+   549us) log.cc:844] Serialized 26 byte log entry
0709 13:19:53.012598 (+ 91865us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:19:53.018870 (+  6272us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:53.035139 (+ 16269us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:53.080025 (+ 44886us) raft_consensus.cc:1685] finished
0709 13:19:53.080053 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:53.080182 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1007,"raft.run_cpu_time_us":135,"raft.run_wall_time_us":2101,"thread_start_us":74656,"threads_started":1,"wal-append.queue_time_us":156891}
I20260709 13:19:53.165067 18551 raft_consensus.cc:3060] T 746aabf856ab401c9ef5e859371e7b67 P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:19:53.191247 16143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5ad4eb281564936a59f41e387a47156" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:19:53.191553 16296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f82224f4564fea93572e8206bad1fe" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:53.191838 16143 raft_consensus.cc:2376] T e5ad4eb281564936a59f41e387a47156 P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 5. Re-sending same reply.
I20260709 13:19:53.192181 16296 raft_consensus.cc:2468] T 23f82224f4564fea93572e8206bad1fe P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 2.
I20260709 13:19:53.192736 16143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e65676b21b748728ad39668cfe722ec" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:19:53.193014 16296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c61a39fd4f84424b129c1a6f9ba404d" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:53.193498 16143 raft_consensus.cc:2393] T 9e65676b21b748728ad39668cfe722ec P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 in current term 5: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
I20260709 13:19:53.193872 18585 leader_election.cc:290] T 9e65676b21b748728ad39668cfe722ec P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:19:53.169971 16147 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40830 (request call id 1125) took 1673 ms. Trace:
I20260709 13:19:53.171280 16142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee9ed67006144797b1de6ea8c2013a68" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:19:53.188575 18513 raft_consensus.cc:515] T 68a6838d82ef4cac83507f906139d90f P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:19:53.115270 16819 rpcz_store.cc:269] 0709 13:19:51.440366 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:52.137763 (+697397us) service_pool.cc:224] Handling call
0709 13:19:53.078606 (+940843us) spinlock_profiling.cc:243] Waited 225 ms on lock 0x7b5400625554. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:19:53.078717 (+   111us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":517354368}
I20260709 13:19:53.207652 18561 leader_election.cc:290] T fffab26427e24d398e2695aad40ca134 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:53.207962 16819 raft_consensus.cc:1240] T 434bff3082d141838068f323bdb0d1ed P cb82cc7bd6db46d980c75237a1402860 [term 5 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 4. Current term is 5. Ops: []
I20260709 13:19:53.208683 16819 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49896 (request call id 842) took 3016 ms. Trace:
I20260709 13:19:53.209061 16819 rpcz_store.cc:276] 0709 13:19:50.192660 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:53.207432 (+3014772us) service_pool.cc:224] Handling call
0709 13:19:53.207747 (+   315us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:53.208389 (+   642us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:53.208606 (+   217us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:53.106222 18532 consensus_queue.cc:1048] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.046s
I20260709 13:19:53.210716 16819 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49896 (request call id 843) took 3017 ms. Trace:
I20260709 13:19:53.211659 18433 raft_consensus.cc:2749] T e5ad4eb281564936a59f41e387a47156 P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:19:53.194514 16147 rpcz_store.cc:276] 0709 13:19:51.496610 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:52.897391 (+1400781us) service_pool.cc:224] Handling call
0709 13:19:52.897687 (+   296us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:52.897995 (+   308us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:52.898022 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:52.901886 (+  3864us) log.cc:844] Serialized 26 byte log entry
0709 13:19:52.969389 (+ 67503us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:19:52.969448 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:52.969538 (+    90us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:53.169741 (+200203us) raft_consensus.cc:1685] finished
0709 13:19:53.169766 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:53.169912 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1100,"raft.run_cpu_time_us":112,"raft.run_wall_time_us":6293,"thread_start_us":48994,"threads_started":1,"wal-append.queue_time_us":185053}
I20260709 13:19:53.220340 16147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57a744f5aaf944678a63469cbf514015" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:19:53.200788 16222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74ca8140f35c4ae29e8a664f6bdfc829" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:53.201452 16115 leader_election.cc:304] T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
W20260709 13:19:53.225558 16266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 172627us
W20260709 13:19:53.227030 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 254435us
I20260709 13:19:53.223142 16147 raft_consensus.cc:1240] T c0eab25ce4c64e018a041beeda2867db P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 13:19:53.228202 16147 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40844 (request call id 1873) took 1021 ms. Trace:
I20260709 13:19:53.229125 16147 rpcz_store.cc:276] 0709 13:19:52.207209 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:53.222534 (+1015325us) service_pool.cc:224] Handling call
0709 13:19:53.222934 (+   400us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:53.227975 (+  5041us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:53.228154 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:53.165311 16823 raft_consensus.cc:1275] T 66f75124215f4ca6bef831e174a792d8 P cb82cc7bd6db46d980c75237a1402860 [term 5 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 13:19:53.203100 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a878999c0024d2abe517484180290fc" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:53.232059 16299 raft_consensus.cc:2468] T 8a878999c0024d2abe517484180290fc P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 4.
I20260709 13:19:53.205225 16146 raft_consensus.cc:2468] T b87147731a68400e8cce9b879daa7c5d P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 7.
I20260709 13:19:53.160466 16218 raft_consensus.cc:2468] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 4.
I20260709 13:19:53.205519 18587 raft_consensus.cc:493] T ecc1e363230043fe9fef390f0b979f20 P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:53.176786 18457 leader_election.cc:290] T 26354361e9b044cea1c2b86c4882fe92 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:53.205888 16822 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49896 (request call id 839) took 3014 ms. Trace:
I20260709 13:19:53.252377 16822 rpcz_store.cc:276] 0709 13:19:50.191837 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:53.075805 (+2883968us) service_pool.cc:224] Handling call
0709 13:19:53.076099 (+   294us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:53.076360 (+   261us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:53.076383 (+    23us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:53.076799 (+   416us) log.cc:844] Serialized 26 byte log entry
0709 13:19:53.119631 (+ 42832us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:19:53.119675 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:53.119751 (+    76us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:53.205697 (+ 85946us) raft_consensus.cc:1685] finished
0709 13:19:53.205716 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:53.205835 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":42476,"threads_started":1,"wal-append.queue_time_us":88289}
I20260709 13:19:53.254186 16822 raft_consensus.cc:1217] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 [term 5 FOLLOWER]: Deduplicated request from leader. Original: 2.5->[3.6-3.6]   Dedup: 3.6->[]
I20260709 13:19:53.184543 16294 raft_consensus.cc:2468] T e4dc6b0541ca431ebc757beac00521f1 P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 4.
I20260709 13:19:53.189365 18530 raft_consensus.cc:3060] T d16dfad0c3b540e5b4b86492ddf87208 P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:19:53.102605 16817 rpcz_store.cc:269] 0709 13:19:50.165511 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:51.128058 (+962547us) service_pool.cc:224] Handling call
0709 13:19:53.076293 (+1948235us) spinlock_profiling.cc:243] Waited 1.25 s on lock 0x7b540047fc54. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:19:53.076400 (+   107us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2877990784}
I20260709 13:19:53.212435 18585 consensus_queue.cc:1059] T 434bff3082d141838068f323bdb0d1ed P 647a343a542d4a139fe106c19095129b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: INVALID_TERM, Last received: 4.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:19:53.212232 18272 consensus_queue.cc:260] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:53.189209 18580 raft_consensus.cc:3060] T fffab26427e24d398e2695aad40ca134 P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:19:53.194379 16143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2794af5b99984efebdee564035841cb0" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:19:53.153048 18504 raft_consensus.cc:515] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:53.212270 18485 consensus_queue.cc:237] T c9944d5e4b8c4109af9955c4f08c9095 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:53.139744 16219 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33050 (request call id 446) took 1662 ms. Trace:
I20260709 13:19:53.160858 16820 raft_consensus.cc:1240] T 26354361e9b044cea1c2b86c4882fe92 P cb82cc7bd6db46d980c75237a1402860 [term 3 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 2. Current term is 3. Ops: []
I20260709 13:19:53.214251 16151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5d59407eb545b895d8d922e0dd69fd" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:19:53.283886 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f2da98864b4425bf7c0055951fdd4b" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:53.284835 16151 raft_consensus.cc:1275] T 8158bbd17b994d2eb96f560bd0f1b88d P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:19:53.194729 16044 catalog_manager.cc:5697] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b reported cstate change: term changed from 2 to 3, leader changed from 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130) to 647a343a542d4a139fe106c19095129b (127.14.177.129). New cstate: current_term: 3 leader_uuid: "647a343a542d4a139fe106c19095129b" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
W20260709 13:19:53.284144 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 229438us
I20260709 13:19:53.214814 16295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9944d5e4b8c4109af9955c4f08c9095" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:53.206534 18432 raft_consensus.cc:2749] T 18ad7e3af5504571a69d10c00750a01c P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:19:53.179795 18558 leader_election.cc:290] T 7c2095a5f4ad49408dbc7f69c606120d P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:53.205551 18513 leader_election.cc:290] T 68a6838d82ef4cac83507f906139d90f P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:53.216440 16150 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46224 (request call id 315) took 1299 ms. Trace:
I20260709 13:19:53.216786 16142 raft_consensus.cc:3060] T ee9ed67006144797b1de6ea8c2013a68 P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:19:53.216373 18551 raft_consensus.cc:515] T 746aabf856ab401c9ef5e859371e7b67 P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:53.215510 16819 rpcz_store.cc:276] 0709 13:19:50.192933 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:53.209832 (+3016899us) service_pool.cc:224] Handling call
0709 13:19:53.210142 (+   310us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:53.210385 (+   243us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:53.210414 (+    29us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:53.210457 (+    43us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:19:53.210494 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:53.210556 (+    62us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:53.210668 (+   112us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:53.217370 16223 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57600 (request call id 1101) took 1118 ms. Trace:
I20260709 13:19:53.291394 16223 rpcz_store.cc:276] 0709 13:19:52.099485 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:52.456937 (+357452us) service_pool.cc:224] Handling call
0709 13:19:53.217177 (+760240us) spinlock_profiling.cc:243] Waited 205 ms on lock 0x7b54002977d4. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:19:53.217313 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":472552832}
I20260709 13:19:53.203087 16296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0eab25ce4c64e018a041beeda2867db" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:19:53.292797 16296 raft_consensus.cc:3060] T c0eab25ce4c64e018a041beeda2867db P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:19:53.244388 16115 leader_election.cc:304] T 23f82224f4564fea93572e8206bad1fe P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
W20260709 13:19:53.246562 16266 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:53.251530 16147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c74490c40243a88e62d1760b5cd814" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:19:53.252393 16823 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49896 (request call id 841) took 3060 ms. Trace:
I20260709 13:19:53.296034 16823 rpcz_store.cc:276] 0709 13:19:50.192362 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:53.164709 (+2972347us) service_pool.cc:224] Handling call
0709 13:19:53.164980 (+   271us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:53.231602 (+ 66622us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:53.252337 (+ 20735us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:53.297781 16823 raft_consensus.cc:1240] T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860 [term 6 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 5. Current term is 6. Ops: []
I20260709 13:19:53.298461 16823 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49896 (request call id 848) took 2818 ms. Trace:
I20260709 13:19:53.255713 16222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5718c623ebee4d2bb770f0f9ddb2f164" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:19:53.223515 18532 raft_consensus.cc:493] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:19:53.262249 18450 raft_consensus.cc:493] T 1df93779209748b9860f6d893dd3de53 P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:19:53.150000 16816 rpcz_store.cc:276] 0709 13:19:51.441514 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:52.958169 (+1516655us) service_pool.cc:224] Handling call
0709 13:19:53.094720 (+136551us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:53.261598 16822 raft_consensus.cc:1240] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 [term 5 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 3. Current term is 5. Ops: [3.6-3.6]
I20260709 13:19:53.262604 16146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a96373bb0e14dce8a266b2fc6c58507" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:19:53.313455 16146 raft_consensus.cc:2393] T 5a96373bb0e14dce8a266b2fc6c58507 P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 4: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
I20260709 13:19:53.313248 16295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fffab26427e24d398e2695aad40ca134" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:53.254037 18452 raft_consensus.cc:493] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:19:53.245311 18433 raft_consensus.cc:493] T d2ab0ac8347641e6955e9bce7d90c1a2 P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:19:53.316754 16223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5d59407eb545b895d8d922e0dd69fd" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:53.317752 16223 raft_consensus.cc:2393] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 in current term 6: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
I20260709 13:19:53.290558 16151 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40844 (request call id 1874) took 1003 ms. Trace:
W20260709 13:19:53.288704 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 273676us
I20260709 13:19:53.305158 16044 catalog_manager.cc:5697] T b87147731a68400e8cce9b879daa7c5d P 647a343a542d4a139fe106c19095129b reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "90f3da57796b488c9087a2f523adf045" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:19:53.250353 18595 raft_consensus.cc:2804] T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 13:19:53.260856 16789 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:53.289006 16150 rpcz_store.cc:276] 0709 13:19:51.917456 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:52.132884 (+215428us) service_pool.cc:224] Handling call
0709 13:19:53.216251 (+1083367us) spinlock_profiling.cc:243] Waited 358 ms on lock 0x7b540028e954. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:19:53.216370 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":823195008}
I20260709 13:19:53.310529 16266 leader_election.cc:304] T cdbd78acb4cf46298e0de8ba0d130e19 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac
I20260709 13:19:53.315148 16225 raft_consensus.cc:2468] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 4.
I20260709 13:19:53.316669 16822 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49896 (request call id 844) took 2944 ms. Trace:
I20260709 13:19:53.298836 16823 rpcz_store.cc:276] 0709 13:19:50.482189 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:53.297241 (+2815052us) service_pool.cc:224] Handling call
0709 13:19:53.297570 (+   329us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:53.298201 (+   631us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:53.298400 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:53.286366 16265 leader_election.cc:304] T c4f65cdf00b748bb963ee6e81ef43b79 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:19:53.324055 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18ad7e3af5504571a69d10c00750a01c" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:53.322322 16822 rpcz_store.cc:276] 0709 13:19:50.372824 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:53.253610 (+2880786us) service_pool.cc:224] Handling call
0709 13:19:53.254083 (+   473us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:53.316416 (+ 62333us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:53.316614 (+   198us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:53.287971 18114 raft_consensus.cc:493] T 68a6838d82ef4cac83507f906139d90f P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:19:53.329053 16790 leader_election.cc:304] T b87147731a68400e8cce9b879daa7c5d P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:19:53.336666 16146 raft_consensus.cc:1275] T 18ad7e3af5504571a69d10c00750a01c P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:19:53.290982 16143 raft_consensus.cc:3060] T 2794af5b99984efebdee564035841cb0 P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:19:53.319422 16151 rpcz_store.cc:276] 0709 13:19:52.286803 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:53.284221 (+997418us) service_pool.cc:224] Handling call
0709 13:19:53.284508 (+   287us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:53.285330 (+   822us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:53.290498 (+  5168us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:19:53.336503 16115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 92169.8us
I20260709 13:19:53.338281 16790 leader_election.cc:304] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:19:53.350185 16223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62fe3346cd3c42c4a55e14af9073bca4" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:53.316917 18272 raft_consensus.cc:3060] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:19:53.346607 18114 raft_consensus.cc:515] T 68a6838d82ef4cac83507f906139d90f P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:53.364722 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 5 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:53.365008 18504 leader_election.cc:290] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:53.365443 18114 leader_election.cc:290] T 68a6838d82ef4cac83507f906139d90f P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:53.366106 18568 raft_consensus.cc:2804] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:19:53.366705 18568 raft_consensus.cc:697] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 [term 5 LEADER]: Becoming Leader. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:19:53.367873 16148 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40830 (request call id 1128) took 1870 ms. Trace:
I20260709 13:19:53.313083 16218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c11074579b046d99e585c2818e18a2c" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:53.352262 18560 raft_consensus.cc:2804] T 23f82224f4564fea93572e8206bad1fe P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:19:53.336804 18433 raft_consensus.cc:515] T d2ab0ac8347641e6955e9bce7d90c1a2 P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:53.323652 18452 raft_consensus.cc:515] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:53.356460 16146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26354361e9b044cea1c2b86c4882fe92" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:19:53.358952 18255 raft_consensus.cc:3060] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:19:53.369439 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87147731a68400e8cce9b879daa7c5d" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 7 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:53.324177 18495 raft_consensus.cc:493] T af57996a8be84d398f0f023e8d08ef77 P cb82cc7bd6db46d980c75237a1402860 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:53.381577 16822 raft_consensus.cc:1275] T 3a7ca61592e848ea9b0fb81a2ce33cdc P cb82cc7bd6db46d980c75237a1402860 [term 7 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:19:53.382340 16822 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49906 (request call id 536) took 1496 ms. Trace:
I20260709 13:19:53.382701 16822 rpcz_store.cc:276] 0709 13:19:51.886425 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:53.330578 (+1444153us) service_pool.cc:224] Handling call
0709 13:19:53.354474 (+ 23896us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:53.382051 (+ 27577us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:53.382278 (+   227us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:53.288139 18591 consensus_queue.cc:1059] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.071s
I20260709 13:19:53.368350 16148 rpcz_store.cc:276] 0709 13:19:51.497502 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:52.977812 (+1480310us) service_pool.cc:224] Handling call
0709 13:19:53.004986 (+ 27174us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:53.131891 (+126905us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:53.131934 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:53.132542 (+   608us) log.cc:844] Serialized 26 byte log entry
0709 13:19:53.285525 (+152983us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:19:53.285607 (+    82us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:53.285697 (+    90us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:53.367650 (+ 81953us) raft_consensus.cc:1685] finished
0709 13:19:53.367671 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:53.367814 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":74444,"raft.run_cpu_time_us":47,"raft.run_wall_time_us":48,"thread_start_us":266574,"threads_started":2,"wal-append.queue_time_us":180615}
I20260709 13:19:53.264532 18587 raft_consensus.cc:515] T ecc1e363230043fe9fef390f0b979f20 P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:19:53.376225 16117 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:53.367331 16220 raft_consensus.cc:3055] T 7c2095a5f4ad49408dbc7f69c606120d P 1ca2725d290e47d68b1a4b609c9816ac [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:19:53.394353 16220 raft_consensus.cc:740] T 7c2095a5f4ad49408dbc7f69c606120d P 1ca2725d290e47d68b1a4b609c9816ac [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:19:53.345921 16151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0eab25ce4c64e018a041beeda2867db" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:19:53.395941 16151 raft_consensus.cc:2393] T c0eab25ce4c64e018a041beeda2867db P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 5: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
I20260709 13:19:53.353641 16294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c74490c40243a88e62d1760b5cd814" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:53.319689 18450 raft_consensus.cc:515] T 1df93779209748b9860f6d893dd3de53 P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:53.312848 18532 raft_consensus.cc:515] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:53.389698 16265 leader_election.cc:304] T 1df93779209748b9860f6d893dd3de53 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
W20260709 13:19:53.386626 16790 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:53.347219 18457 raft_consensus.cc:2804] T b87147731a68400e8cce9b879daa7c5d P cb82cc7bd6db46d980c75237a1402860 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:19:53.347540 18539 raft_consensus.cc:2804] T cdbd78acb4cf46298e0de8ba0d130e19 P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:19:53.343801 18123 raft_consensus.cc:2804] T c4f65cdf00b748bb963ee6e81ef43b79 P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:19:53.422009 16146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57a744f5aaf944678a63469cbf514015" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:19:53.287549 16219 rpcz_store.cc:276] 0709 13:19:51.477331 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:53.074823 (+1597492us) service_pool.cc:224] Handling call
0709 13:19:53.075147 (+   324us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:53.108376 (+ 33229us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:53.129509 (+ 21133us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:19:53.400825 16116 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:53.351178 16044 catalog_manager.cc:5697] T 8a878999c0024d2abe517484180290fc P 647a343a542d4a139fe106c19095129b reported cstate change: term changed from 1 to 3, leader changed from 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130) to 647a343a542d4a139fe106c19095129b (127.14.177.129). New cstate: current_term: 3 leader_uuid: "647a343a542d4a139fe106c19095129b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } health_report { overall_health: HEALTHY } } }
I20260709 13:19:53.441632 18532 leader_election.cc:290] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:19:53.441875 16044 catalog_manager.cc:5697] T 18ad7e3af5504571a69d10c00750a01c P 647a343a542d4a139fe106c19095129b reported cstate change: term changed from 2 to 3, leader changed from 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130) to 647a343a542d4a139fe106c19095129b (127.14.177.129). New cstate: current_term: 3 leader_uuid: "647a343a542d4a139fe106c19095129b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: UNKNOWN } } }
I20260709 13:19:53.379731 18582 raft_consensus.cc:493] T 68a6838d82ef4cac83507f906139d90f P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:53.318142 18598 consensus_queue.cc:1059] T 3884f864a96d428785895e5463e5728f P 647a343a542d4a139fe106c19095129b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.085s
I20260709 13:19:53.379447 18560 raft_consensus.cc:493] T 23f82224f4564fea93572e8206bad1fe P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:53.385335 18418 raft_consensus.cc:3055] T 3884f864a96d428785895e5463e5728f P 647a343a542d4a139fe106c19095129b [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:19:53.443773 18560 raft_consensus.cc:3060] T 23f82224f4564fea93572e8206bad1fe P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:19:53.443843 18418 raft_consensus.cc:740] T 3884f864a96d428785895e5463e5728f P 647a343a542d4a139fe106c19095129b [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:19:53.444072 16224 raft_consensus.cc:2468] T ecc1e363230043fe9fef390f0b979f20 P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 4.
I20260709 13:19:53.443416 16146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c74490c40243a88e62d1760b5cd814" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:19:53.446007 16146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87ac0d1f7e8a419f9bcbc04aee0e067e" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:19:53.324865 18595 raft_consensus.cc:493] T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:53.393215 16817 raft_consensus.cc:1240] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860 [term 3 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 2. Current term is 3. Ops: []
I20260709 13:19:53.447490 18595 raft_consensus.cc:3060] T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:19:53.447932 16817 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49896 (request call id 846) took 3007 ms. Trace:
I20260709 13:19:53.448295 16817 rpcz_store.cc:276] 0709 13:19:50.440654 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:53.289268 (+2848614us) service_pool.cc:224] Handling call
0709 13:19:53.352603 (+ 63335us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:53.447671 (+ 95068us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:53.447875 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:53.449725 16817 raft_consensus.cc:1240] T ee9ed67006144797b1de6ea8c2013a68 P cb82cc7bd6db46d980c75237a1402860 [term 5 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 4. Current term is 5. Ops: []
I20260709 13:19:53.450346 16817 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49906 (request call id 539) took 1531 ms. Trace:
I20260709 13:19:53.400279 16220 consensus_queue.cc:260] T 7c2095a5f4ad49408dbc7f69c606120d P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:53.451718 16220 raft_consensus.cc:3060] T 7c2095a5f4ad49408dbc7f69c606120d P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:19:53.418244 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ad5dbfa99ee4b8d9e28be92b3f36db5" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:53.455952 16299 raft_consensus.cc:2468] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 4.
I20260709 13:19:53.456838 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8158bbd17b994d2eb96f560bd0f1b88d" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:53.456279 16044 catalog_manager.cc:5697] T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b reported cstate change: term changed from 2 to 3, leader changed from cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) to 647a343a542d4a139fe106c19095129b (127.14.177.129). New cstate: current_term: 3 leader_uuid: "647a343a542d4a139fe106c19095129b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:19:53.457583 16299 raft_consensus.cc:2410] T 8158bbd17b994d2eb96f560bd0f1b88d P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b for term 6 because replica has last-logged OpId of term: 4 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 13:19:53.443456 18582 raft_consensus.cc:515] T 68a6838d82ef4cac83507f906139d90f P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:53.460325 18582 leader_election.cc:290] T 68a6838d82ef4cac83507f906139d90f P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:19:53.460863 18582 raft_consensus.cc:2749] T 1df93779209748b9860f6d893dd3de53 P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:19:53.461505 18582 consensus_queue.cc:1048] T b87147731a68400e8cce9b879daa7c5d P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:19:53.450677 16817 rpcz_store.cc:276] 0709 13:19:51.919151 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:53.449126 (+1529975us) service_pool.cc:224] Handling call
0709 13:19:53.449520 (+   394us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:53.450107 (+   587us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:53.450292 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:53.463510 18560 raft_consensus.cc:515] T 23f82224f4564fea93572e8206bad1fe P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:53.464542 16817 raft_consensus.cc:1240] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [term 7 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 6. Current term is 7. Ops: []
I20260709 13:19:53.466084 18560 leader_election.cc:290] T 23f82224f4564fea93572e8206bad1fe P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 3 election: Requested vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:19:53.356520 18372 consensus_queue.cc:1048] T 8a878999c0024d2abe517484180290fc P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.042s
I20260709 13:19:53.470433 18595 raft_consensus.cc:515] T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:53.474172 16146 raft_consensus.cc:2410] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 for term 6 because replica has last-logged OpId of term: 4 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 13:19:53.393507 18488 raft_consensus.cc:493] T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:53.475355 16148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e65676b21b748728ad39668cfe722ec" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:19:53.503897 16148 raft_consensus.cc:3060] T 9e65676b21b748728ad39668cfe722ec P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:19:53.486295 16220 raft_consensus.cc:1275] T 7c2095a5f4ad49408dbc7f69c606120d P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 2. (term mismatch)
I20260709 13:19:53.505060 18488 raft_consensus.cc:515] T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:53.507833 16297 raft_consensus.cc:1240] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 6. Current term is 7. Ops: [6.4-6.4]
W20260709 13:19:53.508538 18601 log.cc:927] Time spent T 5a959dbecc2648eb85d1cc550f5a3245 P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.120s	user 0.000s	sys 0.000s
I20260709 13:19:53.515079 16148 raft_consensus.cc:2468] T 9e65676b21b748728ad39668cfe722ec P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 6.
I20260709 13:19:53.379819 18588 consensus_queue.cc:1048] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.068s
I20260709 13:19:53.515450 16226 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57600 (request call id 1089) took 1455 ms. Trace:
I20260709 13:19:53.476082 18457 raft_consensus.cc:697] T b87147731a68400e8cce9b879daa7c5d P cb82cc7bd6db46d980c75237a1402860 [term 7 LEADER]: Becoming Leader. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:19:53.475360 18421 consensus_queue.cc:1059] T 434bff3082d141838068f323bdb0d1ed P 647a343a542d4a139fe106c19095129b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.081s
I20260709 13:19:53.515218 18452 leader_election.cc:290] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:19:53.519920 16218 raft_consensus.cc:1240] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 6. Current term is 7. Ops: []
I20260709 13:19:53.520810 18595 leader_election.cc:290] T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 5 election: Requested vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
W20260709 13:19:53.477807 16116 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:53.522271 16825 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49896 (request call id 840) took 3330 ms. Trace:
I20260709 13:19:53.476325 16822 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49906 (request call id 537) took 1589 ms. Trace:
I20260709 13:19:53.399912 18347 raft_consensus.cc:493] T d16dfad0c3b540e5b4b86492ddf87208 P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:53.609741 16825 rpcz_store.cc:276] 0709 13:19:50.192099 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:53.141486 (+2949387us) service_pool.cc:224] Handling call
0709 13:19:53.164877 (+ 23391us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:53.166200 (+  1323us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:53.166232 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:53.178314 (+ 12082us) log.cc:844] Serialized 26 byte log entry
0709 13:19:53.418572 (+240258us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:19:53.418613 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:53.418675 (+    62us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:53.522048 (+103373us) raft_consensus.cc:1685] finished
0709 13:19:53.522066 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:53.522198 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":5226,"raft.run_cpu_time_us":92,"raft.run_wall_time_us":93,"thread_start_us":116096,"threads_started":1,"wal-append.queue_time_us":221827}
I20260709 13:19:53.611047 16822 rpcz_store.cc:276] 0709 13:19:51.886735 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:53.404896 (+1518161us) service_pool.cc:224] Handling call
0709 13:19:53.475821 (+ 70925us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:53.476020 (+   199us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:53.476045 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:53.476083 (+    38us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:19:53.476110 (+    27us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:53.476161 (+    51us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:53.476267 (+   106us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:53.401978 18591 consensus_queue.cc:1243] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [LEADER]: Peer 1ca2725d290e47d68b1a4b609c9816ac log is divergent from this leader: its last log entry 4.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
W20260709 13:19:53.617061 16189 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:53.535382 16220 pending_rounds.cc:85] T 7c2095a5f4ad49408dbc7f69c606120d P 1ca2725d290e47d68b1a4b609c9816ac: Aborting all ops after (but not including) 1
I20260709 13:19:53.546272 18450 leader_election.cc:290] T 1df93779209748b9860f6d893dd3de53 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
W20260709 13:19:53.538955 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 283763us
I20260709 13:19:53.546756 16297 raft_consensus.cc:1240] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 4. Current term is 5. Ops: [4.4-4.4]
I20260709 13:19:53.396481 18568 consensus_queue.cc:237] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:53.394881 18495 raft_consensus.cc:515] T af57996a8be84d398f0f023e8d08ef77 P cb82cc7bd6db46d980c75237a1402860 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:53.560545 18578 raft_consensus.cc:515] T 8638ee363273426db1393dd81e5c7b72 P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:19:53.562280 18463 consensus_meta.cc:287] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b: Time spent flushing consensus metadata: real 0.504s	user 0.008s	sys 0.000s
I20260709 13:19:53.562950 16144 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46224 (request call id 314) took 1648 ms. Trace:
I20260709 13:19:53.421875 16294 raft_consensus.cc:2468] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 5.
I20260709 13:19:53.643285 16223 raft_consensus.cc:2410] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 for term 7 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 13:19:53.566066 18488 leader_election.cc:290] T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:19:53.568328 18558 raft_consensus.cc:3055] T 434bff3082d141838068f323bdb0d1ed P 647a343a542d4a139fe106c19095129b [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:19:53.399657 18417 raft_consensus.cc:515] T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:53.576872 18433 leader_election.cc:290] T d2ab0ac8347641e6955e9bce7d90c1a2 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:19:53.579583 18587 leader_election.cc:290] T ecc1e363230043fe9fef390f0b979f20 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:19:53.580919 18453 raft_consensus.cc:3060] T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:19:53.584622 16291 raft_consensus.cc:1240] T 68a6838d82ef4cac83507f906139d90f P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 5. Current term is 6. Ops: []
I20260709 13:19:53.587359 16226 rpcz_store.cc:276] 0709 13:19:52.059964 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:53.081601 (+1021637us) service_pool.cc:224] Handling call
0709 13:19:53.081920 (+   319us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:53.082530 (+   610us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:53.082559 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:53.082969 (+   410us) log.cc:844] Serialized 26 byte log entry
0709 13:19:53.294809 (+211840us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:19:53.294846 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:53.294901 (+    55us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:53.515228 (+220327us) raft_consensus.cc:1685] finished
0709 13:19:53.515246 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:53.515392 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":112,"raft.run_cpu_time_us":98,"raft.run_wall_time_us":98,"thread_start_us":199370,"threads_started":1,"wal-append.queue_time_us":269824}
I20260709 13:19:53.420392 18539 raft_consensus.cc:493] T cdbd78acb4cf46298e0de8ba0d130e19 P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:19:53.591078 16192 leader_election.cc:304] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 647a343a542d4a139fe106c19095129b
I20260709 13:19:53.591266 16823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f82224f4564fea93572e8206bad1fe" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:19:53.669670 16823 raft_consensus.cc:2376] T 23f82224f4564fea93572e8206bad1fe P cb82cc7bd6db46d980c75237a1402860 [term 2 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 2. Re-sending same reply.
I20260709 13:19:53.544716 16142 raft_consensus.cc:2468] T ee9ed67006144797b1de6ea8c2013a68 P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 5.
I20260709 13:19:53.523669 18123 raft_consensus.cc:697] T c4f65cdf00b748bb963ee6e81ef43b79 P 90f3da57796b488c9087a2f523adf045 [term 7 LEADER]: Becoming Leader. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:19:53.598183 16115 leader_election.cc:304] T 8a878999c0024d2abe517484180290fc P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac
I20260709 13:19:53.610632 16296 raft_consensus.cc:2410] T c0eab25ce4c64e018a041beeda2867db P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b for term 5 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:19:53.528672 18459 raft_consensus.cc:493] T 3884f864a96d428785895e5463e5728f P cb82cc7bd6db46d980c75237a1402860 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:53.534336 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 131039us
I20260709 13:19:53.679093 16265 leader_election.cc:304] T 31969392ccdb43b491911a319e8bc7e1 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: cb82cc7bd6db46d980c75237a1402860
I20260709 13:19:53.620551 16220 pending_rounds.cc:107] T 7c2095a5f4ad49408dbc7f69c606120d P 1ca2725d290e47d68b1a4b609c9816ac: Aborting uncommitted NO_OP operation due to leader change: 3.2
I20260709 13:19:53.680265 16265 leader_election.cc:304] T 8a878999c0024d2abe517484180290fc P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:19:53.680687 16220 raft_consensus.cc:2889] T 7c2095a5f4ad49408dbc7f69c606120d P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:19:53.622843 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26354361e9b044cea1c2b86c4882fe92" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:53.563694 16816 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49906 (request call id 531) took 1679 ms. Trace:
I20260709 13:19:53.625236 16116 leader_election.cc:304] T 5a96373bb0e14dce8a266b2fc6c58507 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:19:53.632894 16144 rpcz_store.cc:276] 0709 13:19:51.914288 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:52.980640 (+1066352us) service_pool.cc:224] Handling call
0709 13:19:52.981008 (+   368us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:53.114016 (+133008us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:53.114054 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:53.124196 (+ 10142us) log.cc:844] Serialized 26 byte log entry
0709 13:19:53.386848 (+262652us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:19:53.386889 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:53.386950 (+    61us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:53.562613 (+175663us) raft_consensus.cc:1685] finished
0709 13:19:53.562633 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:53.562867 (+   234us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":98178,"raft.run_cpu_time_us":93,"raft.run_wall_time_us":94,"thread_start_us":304126,"threads_started":2,"wal-append.queue_time_us":373606}
I20260709 13:19:53.654683 16294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74ca8140f35c4ae29e8a664f6bdfc829" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:53.634361 16190 leader_election.cc:304] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: cb82cc7bd6db46d980c75237a1402860
I20260709 13:19:53.639153 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d4ab870a05347868956aa8b841b2981" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
W20260709 13:19:53.644683 16114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:53.646517 16117 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:53.627400 18347 raft_consensus.cc:515] T d16dfad0c3b540e5b4b86492ddf87208 P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:53.685950 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f82224f4564fea93572e8206bad1fe" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
W20260709 13:19:53.686239 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 61128us
I20260709 13:19:53.686695 16297 raft_consensus.cc:3060] T 23f82224f4564fea93572e8206bad1fe P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:19:53.686982 16116 leader_election.cc:304] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 90f3da57796b488c9087a2f523adf045
I20260709 13:19:53.624872 18406 raft_consensus.cc:493] T b6f07842d3aa47858df5665874c86f3a P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Starting leader election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:19:53.689496 18406 raft_consensus.cc:3060] T b6f07842d3aa47858df5665874c86f3a P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:19:53.683928 16816 rpcz_store.cc:276] 0709 13:19:51.884756 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:53.347572 (+1462816us) service_pool.cc:224] Handling call
0709 13:19:53.393617 (+ 46045us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:53.424238 (+ 30621us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:53.477014 (+ 52776us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:53.504531 (+ 27517us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:19:53.528481 (+ 23950us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:53.539533 (+ 11052us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:53.563605 (+ 24072us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:53.621315 18515 raft_consensus.cc:493] T e5ad4eb281564936a59f41e387a47156 P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:53.691465 18515 raft_consensus.cc:515] T e5ad4eb281564936a59f41e387a47156 P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:53.649415 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e65676b21b748728ad39668cfe722ec" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:53.693971 16298 raft_consensus.cc:2393] T 9e65676b21b748728ad39668cfe722ec P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 6: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
I20260709 13:19:53.694876 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31969392ccdb43b491911a319e8bc7e1" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:19:53.653358 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5718c623ebee4d2bb770f0f9ddb2f164" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:53.688371 16220 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57600 (request call id 1091) took 1627 ms. Trace:
I20260709 13:19:53.665246 16300 raft_consensus.cc:1240] T 8638ee363273426db1393dd81e5c7b72 P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 4. Current term is 5. Ops: []
I20260709 13:19:53.670913 16150 raft_consensus.cc:1240] T ee9ed67006144797b1de6ea8c2013a68 P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 4. Current term is 5. Ops: []
I20260709 13:19:53.668601 18551 leader_election.cc:290] T 746aabf856ab401c9ef5e859371e7b67 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 5 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:19:53.676406 16296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68a6838d82ef4cac83507f906139d90f" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:53.700999 16294 raft_consensus.cc:2468] T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 5.
I20260709 13:19:53.701416 16296 raft_consensus.cc:2468] T 68a6838d82ef4cac83507f906139d90f P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 6.
I20260709 13:19:53.660831 18558 raft_consensus.cc:740] T 434bff3082d141838068f323bdb0d1ed P 647a343a542d4a139fe106c19095129b [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:19:53.671267 18539 raft_consensus.cc:3060] T cdbd78acb4cf46298e0de8ba0d130e19 P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:19:53.681799 16265 leader_election.cc:304] T 8158bbd17b994d2eb96f560bd0f1b88d P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac
W20260709 13:19:53.683388 16192 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 92424.7us
I20260709 13:19:53.706809 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fffab26427e24d398e2695aad40ca134" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
W20260709 13:19:53.685833 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 51595.2us
I20260709 13:19:53.707345 16225 raft_consensus.cc:2468] T fffab26427e24d398e2695aad40ca134 P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 3.
I20260709 13:19:53.707926 16190 leader_election.cc:304] T c2f5f89186204242bf377f7c42b2e2da P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 647a343a542d4a139fe106c19095129b
I20260709 13:19:53.710199 18347 leader_election.cc:290] T d16dfad0c3b540e5b4b86492ddf87208 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:19:53.710376 18272 raft_consensus.cc:2804] T c2f5f89186204242bf377f7c42b2e2da P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:19:53.710721 16299 raft_consensus.cc:1240] T d16dfad0c3b540e5b4b86492ddf87208 P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 13:19:53.710382 16820 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49892 (request call id 356) took 3546 ms. Trace:
I20260709 13:19:53.711330 16820 rpcz_store.cc:276] 0709 13:19:50.163942 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:53.040634 (+2876692us) service_pool.cc:224] Handling call
0709 13:19:53.040937 (+   303us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:53.283472 (+242535us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:53.710166 (+426694us) spinlock_profiling.cc:243] Waited 120 ms on lock 0x7b5400520dd4. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:19:53.710322 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":275340544}
I20260709 13:19:53.713924 18578 leader_election.cc:290] T 8638ee363273426db1393dd81e5c7b72 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 5 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:19:53.713563 18418 consensus_queue.cc:260] T 3884f864a96d428785895e5463e5728f P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:53.664358 16147 raft_consensus.cc:1240] T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 4. Current term is 5. Ops: []
I20260709 13:19:53.684430 18424 raft_consensus.cc:2804] T 8a878999c0024d2abe517484180290fc P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:19:53.718685 18424 raft_consensus.cc:493] T 8a878999c0024d2abe517484180290fc P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:53.719122 18424 raft_consensus.cc:3060] T 8a878999c0024d2abe517484180290fc P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:19:53.720323 18400 raft_consensus.cc:3060] T c9944d5e4b8c4109af9955c4f08c9095 P cb82cc7bd6db46d980c75237a1402860 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:19:53.721899 16143 raft_consensus.cc:2468] T 2794af5b99984efebdee564035841cb0 P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 4.
I20260709 13:19:53.661825 18463 raft_consensus.cc:515] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:53.724027 18418 raft_consensus.cc:3060] T 3884f864a96d428785895e5463e5728f P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:19:53.724350 16217 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57600 (request call id 1096) took 1654 ms. Trace:
I20260709 13:19:53.725749 18463 leader_election.cc:290] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 6 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:19:53.724754 16217 rpcz_store.cc:276] 0709 13:19:52.069875 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:53.512774 (+1442899us) service_pool.cc:224] Handling call
0709 13:19:53.513145 (+   371us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:53.513538 (+   393us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:53.513573 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:53.514057 (+   484us) log.cc:844] Serialized 26 byte log entry
0709 13:19:53.664442 (+150385us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:19:53.667405 (+  2963us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:53.672508 (+  5103us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:53.724125 (+ 51617us) raft_consensus.cc:1685] finished
0709 13:19:53.724149 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:53.724291 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1851,"raft.run_cpu_time_us":106,"raft.run_wall_time_us":107,"thread_start_us":146433,"threads_started":1,"wal-append.queue_time_us":173498}
I20260709 13:19:53.726327 18463 raft_consensus.cc:493] T 5a959dbecc2648eb85d1cc550f5a3245 P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:53.728142 16145 raft_consensus.cc:1240] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 5. Current term is 6. Ops: []
I20260709 13:19:53.688120 16191 leader_election.cc:304] T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 90f3da57796b488c9087a2f523adf045
I20260709 13:19:53.695974 16220 rpcz_store.cc:276] 0709 13:19:52.060621 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:53.103331 (+1042710us) service_pool.cc:224] Handling call
0709 13:19:53.122507 (+ 19176us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:53.367235 (+244728us) spinlock_profiling.cc:243] Waited 65.1 ms on lock 0x7b540038f520. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0ad6dc0 00007f69b0b611c0 00007f69b0b62231 00007f69b0b5ed8f 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c
0709 13:19:53.688136 (+320901us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:53.688318 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":149688192}
I20260709 13:19:53.729507 16265 leader_election.cc:304] T 2794af5b99984efebdee564035841cb0 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:19:53.698127 16116 leader_election.cc:304] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
I20260709 13:19:53.673856 16790 leader_election.cc:304] T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:19:53.699188 18591 raft_consensus.cc:2804] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:19:53.687011 18587 raft_consensus.cc:2804] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac [term 6 FOLLOWER]: Leader election won for term 6
W20260709 13:19:53.701802 18535 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 4 current term: 4 is higher.
W20260709 13:19:53.731351 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 57585.2us
I20260709 13:19:53.699810 18123 consensus_queue.cc:237] T c4f65cdf00b748bb963ee6e81ef43b79 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:53.702387 16296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e65676b21b748728ad39668cfe722ec" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
W20260709 13:19:53.677806 16115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 79996.8us
I20260709 13:19:53.738330 16296 raft_consensus.cc:2368] T 9e65676b21b748728ad39668cfe722ec P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 for earlier term 5. Current term is 6.
I20260709 13:19:53.738737 16115 leader_election.cc:304] T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac
I20260709 13:19:53.734958 18560 raft_consensus.cc:2804] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:19:53.740332 16142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62fe3346cd3c42c4a55e14af9073bca4" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:19:53.740983 16142 raft_consensus.cc:2376] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 7. Re-sending same reply.
I20260709 13:19:53.708096 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee9ed67006144797b1de6ea8c2013a68" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:53.715688 18558 consensus_queue.cc:260] T 434bff3082d141838068f323bdb0d1ed P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:53.688820 18588 raft_consensus.cc:2804] T 31969392ccdb43b491911a319e8bc7e1 P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:19:53.707265 16192 leader_election.cc:304] T 66f75124215f4ca6bef831e174a792d8 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 647a343a542d4a139fe106c19095129b
I20260709 13:19:53.748042 16192 leader_election.cc:304] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860
I20260709 13:19:53.698741 16219 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57600 (request call id 1095) took 1637 ms. Trace:
I20260709 13:19:53.731295 16222 raft_consensus.cc:2468] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 5.
I20260709 13:19:53.699625 18417 leader_election.cc:290] T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 5 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:19:53.726704 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c2095a5f4ad49408dbc7f69c606120d" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:53.729151 16190 leader_election.cc:304] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860
I20260709 13:19:53.746649 18614 consensus_queue.cc:1059] T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.008s
I20260709 13:19:53.750510 18532 raft_consensus.cc:2749] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:19:53.750829 18614 raft_consensus.cc:3055] T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:19:53.751255 18614 raft_consensus.cc:740] T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:19:53.751914 16190 leader_election.cc:304] T 84f2da98864b4425bf7c0055951fdd4b P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:19:53.730561 16265 leader_election.cc:304] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:19:53.752066 18614 consensus_queue.cc:260] T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:53.753293 18614 raft_consensus.cc:3060] T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:19:53.695809 18433 raft_consensus.cc:2804] T 8a878999c0024d2abe517484180290fc P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:19:53.753890 18433 raft_consensus.cc:493] T 8a878999c0024d2abe517484180290fc P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:53.754478 18433 raft_consensus.cc:3060] T 8a878999c0024d2abe517484180290fc P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:19:53.694695 18255 raft_consensus.cc:2804] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:19:53.689280 18595 raft_consensus.cc:2749] T 5a96373bb0e14dce8a266b2fc6c58507 P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:19:53.696424 18611 raft_consensus.cc:493] T af57996a8be84d398f0f023e8d08ef77 P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:53.742556 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 54527us
I20260709 13:19:53.737123 18272 raft_consensus.cc:697] T c2f5f89186204242bf377f7c42b2e2da P 1ca2725d290e47d68b1a4b609c9816ac [term 7 LEADER]: Becoming Leader. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:19:53.744019 16115 leader_election.cc:304] T b87147731a68400e8cce9b879daa7c5d P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860
I20260709 13:19:53.704154 16294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c2095a5f4ad49408dbc7f69c606120d" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:53.702479 16218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c61a39fd4f84424b129c1a6f9ba404d" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:53.749330 16217 raft_consensus.cc:2468] T 7c2095a5f4ad49408dbc7f69c606120d P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 4.
I20260709 13:19:53.746769 18513 raft_consensus.cc:2804] T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:19:53.753198 16265 leader_election.cc:304] T 02838c86531d4f98bf58d1c22b8baf77 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
W20260709 13:19:53.808812 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 55693us
I20260709 13:19:53.809839 16265 leader_election.cc:304] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:19:53.753824 18532 raft_consensus.cc:2749] T 84f2da98864b4425bf7c0055951fdd4b P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:19:53.733369 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af57996a8be84d398f0f023e8d08ef77" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:53.740461 18560 raft_consensus.cc:493] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:53.749078 18612 raft_consensus.cc:2804] T 66f75124215f4ca6bef831e174a792d8 P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:19:53.732093 17992 raft_consensus.cc:2749] T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:19:53.696544 18485 consensus_queue.cc:1048] T 7c2095a5f4ad49408dbc7f69c606120d P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.047s
I20260709 13:19:53.653643 18457 consensus_queue.cc:237] T b87147731a68400e8cce9b879daa7c5d P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:53.762893 16191 leader_election.cc:304] T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 90f3da57796b488c9087a2f523adf045
W20260709 13:19:53.826823 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 64048.2us
I20260709 13:19:53.739625 16790 leader_election.cc:304] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860; no voters: 
W20260709 13:19:53.827718 16115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 83798.9us
W20260709 13:19:53.828233 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 88728.2us
I20260709 13:19:53.827766 16191 leader_election.cc:304] T 829b4ad8c5a54eababeea1181b960d4b P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
I20260709 13:19:53.829981 16790 leader_election.cc:304] T ee9ed67006144797b1de6ea8c2013a68 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac
I20260709 13:19:53.829367 16115 leader_election.cc:304] T 8158bbd17b994d2eb96f560bd0f1b88d P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:19:53.732005 18609 raft_consensus.cc:2804] T 8158bbd17b994d2eb96f560bd0f1b88d P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:19:53.765340 18347 raft_consensus.cc:2804] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:19:53.736680 18578 raft_consensus.cc:2764] T 2794af5b99984efebdee564035841cb0 P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Leader pre-election decision vote started in defunct term 3: won
I20260709 13:19:53.811024 16265 leader_election.cc:304] T e4dc6b0541ca431ebc757beac00521f1 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac
I20260709 13:19:53.840420 16191 leader_election.cc:304] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: cb82cc7bd6db46d980c75237a1402860
I20260709 13:19:53.814910 18513 raft_consensus.cc:493] T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:53.742309 18515 leader_election.cc:290] T e5ad4eb281564936a59f41e387a47156 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:53.731717 18591 raft_consensus.cc:493] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Starting leader election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:19:53.755872 16291 raft_consensus.cc:1240] T 1df93779209748b9860f6d893dd3de53 P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 4. Current term is 5. Ops: [4.4-4.4]
I20260709 13:19:53.726768 18463 raft_consensus.cc:515] T 5a959dbecc2648eb85d1cc550f5a3245 P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:19:53.821652 16117 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:53.755754 18587 raft_consensus.cc:697] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac [term 6 LEADER]: Becoming Leader. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:19:53.732847 18452 consensus_queue.cc:1048] T e4dc6b0541ca431ebc757beac00521f1 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.072s
I20260709 13:19:53.749291 18114 raft_consensus.cc:2804] T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:19:53.761490 18255 raft_consensus.cc:493] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Starting leader election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:19:53.729487 18530 raft_consensus.cc:515] T d16dfad0c3b540e5b4b86492ddf87208 P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:53.688324 18459 raft_consensus.cc:515] T 3884f864a96d428785895e5463e5728f P cb82cc7bd6db46d980c75237a1402860 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:53.833247 16790 leader_election.cc:304] T 2794af5b99984efebdee564035841cb0 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:19:53.755232 18588 raft_consensus.cc:493] T 31969392ccdb43b491911a319e8bc7e1 P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:53.839216 18347 raft_consensus.cc:493] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:53.836714 18565 raft_consensus.cc:3055] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:19:53.837340 18538 raft_consensus.cc:2804] T ee9ed67006144797b1de6ea8c2013a68 P cb82cc7bd6db46d980c75237a1402860 [term 5 FOLLOWER]: Leader election won for term 5
W20260709 13:19:53.837590 16251 heartbeater.cc:646] Failed to heartbeat to 127.14.177.190:44079 (0 consecutive failures): Timed out: Failed to send heartbeat to master: TSHeartbeat RPC to 127.14.177.190:44079 timed out after 15.000s (SENT)
I20260709 13:19:53.832185 18417 raft_consensus.cc:2749] T 8158bbd17b994d2eb96f560bd0f1b88d P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:19:53.842258 16115 leader_election.cc:304] T c0eab25ce4c64e018a041beeda2867db P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:19:53.888370 18538 raft_consensus.cc:697] T ee9ed67006144797b1de6ea8c2013a68 P cb82cc7bd6db46d980c75237a1402860 [term 5 LEADER]: Becoming Leader. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
W20260709 13:19:53.889235 16115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 47022.2us
I20260709 13:19:53.817282 18560 raft_consensus.cc:3060] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:19:53.813203 18611 raft_consensus.cc:515] T af57996a8be84d398f0f023e8d08ef77 P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:53.821008 18582 raft_consensus.cc:2804] T 02838c86531d4f98bf58d1c22b8baf77 P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:19:53.840793 18551 raft_consensus.cc:2749] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:19:53.832366 18609 raft_consensus.cc:493] T 8158bbd17b994d2eb96f560bd0f1b88d P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:53.817731 18612 raft_consensus.cc:493] T 66f75124215f4ca6bef831e174a792d8 P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Starting leader election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:19:53.907415 18612 raft_consensus.cc:3060] T 66f75124215f4ca6bef831e174a792d8 P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:19:53.862818 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 107906us
I20260709 13:19:53.928526 16190 leader_election.cc:304] T 08c74490c40243a88e62d1760b5cd814 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:19:53.823792 18558 raft_consensus.cc:3060] T 434bff3082d141838068f323bdb0d1ed P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:19:53.841171 18595 raft_consensus.cc:2749] T b87147731a68400e8cce9b879daa7c5d P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:19:53.838928 18453 raft_consensus.cc:2804] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:19:53.844842 18532 raft_consensus.cc:2804] T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:19:53.845377 18513 raft_consensus.cc:3060] T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:19:53.839490 18617 raft_consensus.cc:2804] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 13:19:53.880741 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 47587.5us
I20260709 13:19:53.835098 18389 leader_election.cc:290] T 9e65676b21b748728ad39668cfe722ec P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 5 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:53.860536 18488 raft_consensus.cc:2804] T e4dc6b0541ca431ebc757beac00521f1 P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:19:53.851235 18591 raft_consensus.cc:3060] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:19:53.863744 18114 raft_consensus.cc:493] T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:53.862792 18432 consensus_queue.cc:1048] T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.026s
I20260709 13:19:53.890231 16115 leader_election.cc:304] T 68a6838d82ef4cac83507f906139d90f P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:19:53.908073 18609 raft_consensus.cc:3060] T 8158bbd17b994d2eb96f560bd0f1b88d P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:19:53.865675 18255 raft_consensus.cc:3060] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:19:53.885633 18347 raft_consensus.cc:3060] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:19:53.884723 18588 raft_consensus.cc:3060] T 31969392ccdb43b491911a319e8bc7e1 P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:19:53.884543 18504 raft_consensus.cc:2804] T 829b4ad8c5a54eababeea1181b960d4b P 1ca2725d290e47d68b1a4b609c9816ac [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:19:53.848976 18585 consensus_queue.cc:1059] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.042s
I20260709 13:19:53.900104 16225 raft_consensus.cc:1240] T af57996a8be84d398f0f023e8d08ef77 P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 13:19:53.884543 18568 raft_consensus.cc:2804] T 2794af5b99984efebdee564035841cb0 P cb82cc7bd6db46d980c75237a1402860 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:19:53.885507 18565 raft_consensus.cc:740] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:19:53.901892 18628 raft_consensus.cc:2804] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 13:19:53.984534 16116 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:53.904791 18622 log.cc:927] Time spent T c9944d5e4b8c4109af9955c4f08c9095 P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.083s	user 0.001s	sys 0.000s
I20260709 13:19:53.907836 16822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e65676b21b748728ad39668cfe722ec" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:19:53.988195 16822 raft_consensus.cc:3060] T 9e65676b21b748728ad39668cfe722ec P cb82cc7bd6db46d980c75237a1402860 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:19:53.813956 18450 consensus_queue.cc:1048] T 19baa7bc367844c288bca8f81439c413 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.030s
I20260709 13:19:53.927328 16825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9944d5e4b8c4109af9955c4f08c9095" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:19:53.891942 18417 raft_consensus.cc:2749] T c0eab25ce4c64e018a041beeda2867db P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:19:53.934535 16823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e4bfb9090c8460eb14fa7e0d818e7f1" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:19:53.934387 16820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434bff3082d141838068f323bdb0d1ed" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:19:53.940106 16816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57a744f5aaf944678a63469cbf514015" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:19:53.940428 16821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e65676b21b748728ad39668cfe722ec" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:19:53.957093 16790 leader_election.cc:304] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac
I20260709 13:19:53.895063 18582 raft_consensus.cc:493] T 02838c86531d4f98bf58d1c22b8baf77 P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:53.754877 16219 rpcz_store.cc:276] 0709 13:19:52.061800 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:53.440303 (+1378503us) service_pool.cc:224] Handling call
0709 13:19:53.440674 (+   371us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:53.441735 (+  1061us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:53.441767 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:53.442405 (+   638us) log.cc:844] Serialized 26 byte log entry
0709 13:19:53.637246 (+194841us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:19:53.637297 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:53.637383 (+    86us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:53.694513 (+ 57130us) raft_consensus.cc:1685] finished
0709 13:19:53.698531 (+  4018us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:53.698684 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":488,"raft.run_cpu_time_us":92,"raft.run_wall_time_us":93,"thread_start_us":194413,"threads_started":1,"wal-append.queue_time_us":226702}
W20260709 13:19:53.964154 16115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 73985.9us
I20260709 13:19:53.737712 18580 raft_consensus.cc:515] T fffab26427e24d398e2695aad40ca134 P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:53.939560 16789 leader_election.cc:304] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:19:53.886092 18272 consensus_queue.cc:237] T c2f5f89186204242bf377f7c42b2e2da P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:53.956679 18617 raft_consensus.cc:493] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:53.889531 18538 consensus_queue.cc:237] T ee9ed67006144797b1de6ea8c2013a68 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:54.018389 16821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5718c623ebee4d2bb770f0f9ddb2f164" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:19:53.995857 18568 raft_consensus.cc:697] T 2794af5b99984efebdee564035841cb0 P cb82cc7bd6db46d980c75237a1402860 [term 4 LEADER]: Becoming Leader. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:19:53.959122 18488 raft_consensus.cc:493] T e4dc6b0541ca431ebc757beac00521f1 P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:53.935737 18532 raft_consensus.cc:493] T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Starting leader election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:19:53.958290 18551 raft_consensus.cc:2804] T 08c74490c40243a88e62d1760b5cd814 P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 13:19:53.978763 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 119822us
I20260709 13:19:53.963536 18114 raft_consensus.cc:3060] T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:19:53.936462 18453 raft_consensus.cc:493] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:54.013142 18598 raft_consensus.cc:3055] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:19:53.999372 16820 raft_consensus.cc:2376] T 434bff3082d141838068f323bdb0d1ed P cb82cc7bd6db46d980c75237a1402860 [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 5. Re-sending same reply.
I20260709 13:19:54.002614 16823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f82224f4564fea93572e8206bad1fe" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
W20260709 13:19:53.962620 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 122297us
W20260709 13:19:54.004470 16114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:54.050022 16823 raft_consensus.cc:2468] T 23f82224f4564fea93572e8206bad1fe P cb82cc7bd6db46d980c75237a1402860 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 2.
W20260709 13:19:54.004837 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 47858.7us
W20260709 13:19:54.007506 16789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 68064.3us
I20260709 13:19:53.979513 16817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2794af5b99984efebdee564035841cb0" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
W20260709 13:19:54.010242 16192 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 262249us
I20260709 13:19:54.018306 18617 raft_consensus.cc:3060] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:19:53.993976 18628 raft_consensus.cc:493] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Starting leader election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:19:54.036866 18532 raft_consensus.cc:3060] T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:19:54.020581 16117 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:54.020494 16821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ab0ac8347641e6955e9bce7d90c1a2" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:19:54.022001 16296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68a6838d82ef4cac83507f906139d90f" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:54.069407 16821 raft_consensus.cc:2468] T d2ab0ac8347641e6955e9bce7d90c1a2 P cb82cc7bd6db46d980c75237a1402860 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 5.
I20260709 13:19:54.058495 18628 raft_consensus.cc:3060] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:19:54.070521 16821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:19:54.023907 16297 raft_consensus.cc:2468] T 23f82224f4564fea93572e8206bad1fe P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 3.
I20260709 13:19:54.072170 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8638ee363273426db1393dd81e5c7b72" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:54.038143 18453 raft_consensus.cc:3060] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:19:54.004287 16294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1df93779209748b9860f6d893dd3de53" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:54.030737 18611 leader_election.cc:290] T af57996a8be84d398f0f023e8d08ef77 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:19:54.009204 18389 raft_consensus.cc:2804] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:19:54.031283 16295 raft_consensus.cc:2393] T fffab26427e24d398e2695aad40ca134 P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 4: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
I20260709 13:19:54.008049 18463 leader_election.cc:290] T 5a959dbecc2648eb85d1cc550f5a3245 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:54.032657 16818 raft_consensus.cc:2468] T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 4.
I20260709 13:19:54.020988 18565 consensus_queue.cc:260] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:54.033145 18551 raft_consensus.cc:493] T 08c74490c40243a88e62d1760b5cd814 P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Starting leader election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:19:54.009423 17992 raft_consensus.cc:2749] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:19:54.013563 16291 raft_consensus.cc:1275] T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 13:19:54.025564 18433 raft_consensus.cc:515] T 8a878999c0024d2abe517484180290fc P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:54.022816 18424 raft_consensus.cc:515] T 8a878999c0024d2abe517484180290fc P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:54.034392 18530 leader_election.cc:290] T d16dfad0c3b540e5b4b86492ddf87208 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 4 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:54.034960 18459 leader_election.cc:290] T 3884f864a96d428785895e5463e5728f P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:53.961189 18587 consensus_queue.cc:237] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:53.968575 18595 raft_consensus.cc:2804] T 68a6838d82ef4cac83507f906139d90f P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:19:54.111392 18595 raft_consensus.cc:493] T 68a6838d82ef4cac83507f906139d90f P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:54.111824 18595 raft_consensus.cc:3060] T 68a6838d82ef4cac83507f906139d90f P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:19:54.113027 16291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a96373bb0e14dce8a266b2fc6c58507" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:19:54.113926 16291 raft_consensus.cc:2393] T 5a96373bb0e14dce8a266b2fc6c58507 P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 4: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
I20260709 13:19:54.114846 16291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 5 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:54.115640 16291 raft_consensus.cc:2468] T ecc1e363230043fe9fef390f0b979f20 P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 4.
I20260709 13:19:54.116204 16294 raft_consensus.cc:2393] T 1df93779209748b9860f6d893dd3de53 P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 5: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
I20260709 13:19:54.088398 16816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26354361e9b044cea1c2b86c4882fe92" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:19:54.117588 16816 raft_consensus.cc:2393] T 26354361e9b044cea1c2b86c4882fe92 P cb82cc7bd6db46d980c75237a1402860 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 3: Already voted for candidate cb82cc7bd6db46d980c75237a1402860 in this term.
I20260709 13:19:54.118469 16816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f82224f4564fea93572e8206bad1fe" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:19:54.118218 18568 consensus_queue.cc:237] T 2794af5b99984efebdee564035841cb0 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:54.119110 16816 raft_consensus.cc:3060] T 23f82224f4564fea93572e8206bad1fe P cb82cc7bd6db46d980c75237a1402860 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:19:54.058430 18452 consensus_queue.cc:1048] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.041s
I20260709 13:19:54.099978 18389 raft_consensus.cc:697] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [term 7 LEADER]: Becoming Leader. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:19:54.080286 16143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3884f864a96d428785895e5463e5728f" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:19:54.088096 18487 raft_consensus.cc:493] T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:54.036536 16265 leader_election.cc:304] T 9e65676b21b748728ad39668cfe722ec P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
W20260709 13:19:54.139204 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 102708us
I20260709 13:19:54.096776 18551 raft_consensus.cc:3060] T 08c74490c40243a88e62d1760b5cd814 P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:19:54.121228 18389 consensus_queue.cc:237] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:54.050318 16191 leader_election.cc:304] T 8cc959fb7f704fba9009c0595467d074 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:19:54.053368 16789 leader_election.cc:304] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac
I20260709 13:19:53.988945 18504 raft_consensus.cc:697] T 829b4ad8c5a54eababeea1181b960d4b P 1ca2725d290e47d68b1a4b609c9816ac [term 6 LEADER]: Becoming Leader. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:19:54.026751 18488 raft_consensus.cc:3060] T e4dc6b0541ca431ebc757beac00521f1 P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:19:54.056958 16820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e65676b21b748728ad39668cfe722ec" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:19:54.057019 16823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31969392ccdb43b491911a319e8bc7e1" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:19:54.150095 16144 raft_consensus.cc:1275] T 8cc959fb7f704fba9009c0595467d074 P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:19:54.150069 16823 raft_consensus.cc:2468] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 4.
I20260709 13:19:54.154572 16292 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48674 (request call id 2121) took 1071 ms. Trace:
W20260709 13:19:54.063925 16192 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:54.067744 16114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:54.069602 16296 raft_consensus.cc:2393] T 68a6838d82ef4cac83507f906139d90f P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 6: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
I20260709 13:19:54.004516 18582 raft_consensus.cc:3060] T 02838c86531d4f98bf58d1c22b8baf77 P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:19:54.014675 18625 log.cc:927] Time spent T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 0.119s	user 0.001s	sys 0.000s
I20260709 13:19:54.073268 16821 raft_consensus.cc:3060] T 19baa7bc367844c288bca8f81439c413 P cb82cc7bd6db46d980c75237a1402860 [term 5 LEARNER]: Advancing to term 6
I20260709 13:19:54.038729 18598 raft_consensus.cc:740] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:19:54.085743 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ad5dbfa99ee4b8d9e28be92b3f36db5" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:54.143843 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68a6838d82ef4cac83507f906139d90f" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:19:54.039394 16822 raft_consensus.cc:2468] T 9e65676b21b748728ad39668cfe722ec P cb82cc7bd6db46d980c75237a1402860 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 6.
I20260709 13:19:54.144639 18487 raft_consensus.cc:515] T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:54.083793 18585 consensus_queue.cc:1059] T 1df93779209748b9860f6d893dd3de53 P 647a343a542d4a139fe106c19095129b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.061s
I20260709 13:19:54.155944 16823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee9ed67006144797b1de6ea8c2013a68" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:19:54.155891 16820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8638ee363273426db1393dd81e5c7b72" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:19:54.157557 16151 raft_consensus.cc:1240] T b87147731a68400e8cce9b879daa7c5d P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 6. Current term is 7. Ops: [6.6-6.6]
I20260709 13:19:54.159710 16148 raft_consensus.cc:1275] T 5a959dbecc2648eb85d1cc550f5a3245 P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:19:53.909691 16221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18ad7e3af5504571a69d10c00750a01c" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:54.141552 18457 raft_consensus.cc:493] T 434bff3082d141838068f323bdb0d1ed P cb82cc7bd6db46d980c75237a1402860 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:54.083452 18632 consensus_queue.cc:1059] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.078s
W20260709 13:19:54.381590 16114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:54.235431 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 185184us
I20260709 13:19:54.148849 16819 raft_consensus.cc:1240] T b87147731a68400e8cce9b879daa7c5d P cb82cc7bd6db46d980c75237a1402860 [term 7 LEADER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 6. Current term is 7. Ops: []
W20260709 13:19:54.235736 16789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 182467us
W20260709 13:19:54.167873 18637 log.cc:927] Time spent T d2ab0ac8347641e6955e9bce7d90c1a2 P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.129s	user 0.001s	sys 0.000s
I20260709 13:19:54.312266 18424 leader_election.cc:290] T 8a878999c0024d2abe517484180290fc P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 5 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:54.149214 18452 raft_consensus.cc:2749] T 8cc959fb7f704fba9009c0595467d074 P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:19:54.357224 18418 consensus_meta.cc:287] T 3884f864a96d428785895e5463e5728f P 647a343a542d4a139fe106c19095129b: Time spent flushing consensus metadata: real 0.632s	user 0.007s	sys 0.012s
I20260709 13:19:54.360289 16820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2372981006d4b218215c28964dea623" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:19:54.361235 16823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31969392ccdb43b491911a319e8bc7e1" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:19:54.397032 16823 raft_consensus.cc:3060] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:19:54.362032 16296 raft_consensus.cc:1240] T 3884f864a96d428785895e5463e5728f P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 3. Current term is 4. Ops: [3.6-3.6]
I20260709 13:19:54.226330 16146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d16dfad0c3b540e5b4b86492ddf87208" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
W20260709 13:19:54.280058 16117 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:54.167338 16223 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33050 (request call id 450) took 1059 ms. Trace:
I20260709 13:19:54.379590 16822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2372981006d4b218215c28964dea623" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
W20260709 13:19:54.313957 18406 consensus_meta.cc:287] T b6f07842d3aa47858df5665874c86f3a P 647a343a542d4a139fe106c19095129b: Time spent flushing consensus metadata: real 0.624s	user 0.010s	sys 0.000s
I20260709 13:19:54.402427 16296 raft_consensus.cc:1240] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 2. Current term is 3. Ops: []
I20260709 13:19:54.089080 18633 consensus_queue.cc:1059] T ee9ed67006144797b1de6ea8c2013a68 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.126s
I20260709 13:19:54.382995 16191 leader_election.cc:304] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:19:54.385133 16148 raft_consensus.cc:1275] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:19:54.387408 16789 leader_election.cc:304] T 8c11074579b046d99e585c2818e18a2c P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac
I20260709 13:19:54.147217 18432 raft_consensus.cc:2804] T 9e65676b21b748728ad39668cfe722ec P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:19:54.154994 18473 raft_consensus.cc:2804] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:19:54.187922 18433 leader_election.cc:290] T 8a878999c0024d2abe517484180290fc P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 5 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:19:54.390488 18530 raft_consensus.cc:3055] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:19:54.396147 16220 raft_consensus.cc:1240] T c2f5f89186204242bf377f7c42b2e2da P 1ca2725d290e47d68b1a4b609c9816ac [term 7 LEADER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 6. Current term is 7. Ops: []
W20260709 13:19:54.399627 16117 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:54.399778 16146 raft_consensus.cc:2410] T d16dfad0c3b540e5b4b86492ddf87208 P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 for term 5 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:19:54.357290 18255 raft_consensus.cc:515] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:54.485365 16224 raft_consensus.cc:1240] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac [term 6 FOLLOWER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 5. Current term is 6. Ops: []
I20260709 13:19:54.188844 16817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af57996a8be84d398f0f023e8d08ef77" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
W20260709 13:19:54.460034 16114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:54.486313 16817 raft_consensus.cc:2468] T af57996a8be84d398f0f023e8d08ef77 P cb82cc7bd6db46d980c75237a1402860 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 5.
I20260709 13:19:54.311722 18609 raft_consensus.cc:515] T 8158bbd17b994d2eb96f560bd0f1b88d P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:54.457505 18473 raft_consensus.cc:493] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:54.478029 16820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74ca8140f35c4ae29e8a664f6bdfc829" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:19:54.437352 18456 raft_consensus.cc:2804] T 8c11074579b046d99e585c2818e18a2c P cb82cc7bd6db46d980c75237a1402860 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:19:54.540334 16150 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40844 (request call id 1879) took 1364 ms. Trace:
I20260709 13:19:54.540760 18456 raft_consensus.cc:493] T 8c11074579b046d99e585c2818e18a2c P cb82cc7bd6db46d980c75237a1402860 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:19:54.540795 16146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a878999c0024d2abe517484180290fc" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:19:54.541407 18456 raft_consensus.cc:3060] T 8c11074579b046d99e585c2818e18a2c P cb82cc7bd6db46d980c75237a1402860 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:19:54.541745 16146 raft_consensus.cc:2393] T 8a878999c0024d2abe517484180290fc P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 5: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
W20260709 13:19:54.519061 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 156016us
I20260709 13:19:54.541692 16147 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46224 (request call id 321) took 1453 ms. Trace:
I20260709 13:19:54.521595 16822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2794af5b99984efebdee564035841cb0" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:19:54.518383 18424 raft_consensus.cc:493] T ecc1e363230043fe9fef390f0b979f20 P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:54.539469 18609 leader_election.cc:290] T 8158bbd17b994d2eb96f560bd0f1b88d P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 6 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:19:54.540140 18473 raft_consensus.cc:3060] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:19:54.533324 16820 raft_consensus.cc:2468] T 74ca8140f35c4ae29e8a664f6bdfc829 P cb82cc7bd6db46d980c75237a1402860 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 5.
W20260709 13:19:54.566656 16789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 179324us
I20260709 13:19:54.540828 16150 rpcz_store.cc:276] 0709 13:19:53.175939 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:53.321556 (+145617us) service_pool.cc:224] Handling call
0709 13:19:53.670731 (+349175us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:53.696193 (+ 25462us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:54.540109 (+843916us) spinlock_profiling.cc:243] Waited 324 ms on lock 0x7b54002d0d50. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5e8ad 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:19:54.540256 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":745264896}
I20260709 13:19:54.458151 18452 raft_consensus.cc:2749] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:19:54.459257 18580 leader_election.cc:290] T fffab26427e24d398e2695aad40ca134 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 4 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:19:54.543421 16147 rpcz_store.cc:276] 0709 13:19:53.088421 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:53.295453 (+207032us) service_pool.cc:224] Handling call
0709 13:19:53.295751 (+   298us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:53.718109 (+422358us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:54.541464 (+823355us) spinlock_profiling.cc:243] Waited 368 ms on lock 0x7b54000e9054. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:19:54.541633 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":847316864}
I20260709 13:19:54.404940 16223 rpcz_store.cc:276] 0709 13:19:53.108885 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:53.321431 (+212546us) service_pool.cc:224] Handling call
0709 13:19:54.161046 (+839615us) spinlock_profiling.cc:243] Waited 292 ms on lock 0x7b5400367c54. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:19:54.167242 (+  6196us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":671405952}
W20260709 13:19:54.461644 16221 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57604 (request call id 2034) took 2086 ms (client timeout 2138 ms). Trace:
W20260709 13:19:54.568614 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 185700us
W20260709 13:19:54.604830 16114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:54.592242 16789 leader_election.cc:304] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac
I20260709 13:19:54.460939 18530 raft_consensus.cc:740] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:19:54.435699 17992 consensus_queue.cc:1048] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.130s
I20260709 13:19:54.601730 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68a6838d82ef4cac83507f906139d90f" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:54.626183 16226 raft_consensus.cc:2468] T 68a6838d82ef4cac83507f906139d90f P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 5.
I20260709 13:19:54.435971 18406 raft_consensus.cc:515] T b6f07842d3aa47858df5665874c86f3a P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:54.624424 16191 leader_election.cc:304] T e5ad4eb281564936a59f41e387a47156 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 90f3da57796b488c9087a2f523adf045
I20260709 13:19:54.559660 18432 raft_consensus.cc:697] T 9e65676b21b748728ad39668cfe722ec P 90f3da57796b488c9087a2f523adf045 [term 6 LEADER]: Becoming Leader. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:19:54.596000 16143 raft_consensus.cc:2410] T 3884f864a96d428785895e5463e5728f P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 for term 5 because replica has last-logged OpId of term: 3 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
W20260709 13:19:54.601631 18610 log.cc:927] Time spent T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 0.105s	user 0.000s	sys 0.001s
I20260709 13:19:54.447415 18504 consensus_queue.cc:237] T 829b4ad8c5a54eababeea1181b960d4b P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:54.637269 16223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ab0ac8347641e6955e9bce7d90c1a2" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:54.386859 18457 raft_consensus.cc:515] T 434bff3082d141838068f323bdb0d1ed P cb82cc7bd6db46d980c75237a1402860 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:19:54.633140 18619 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.766s	user 0.007s	sys 0.021s
I20260709 13:19:54.659487 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e4bfb9090c8460eb14fa7e0d818e7f1" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:54.594977 16294 raft_consensus.cc:1240] T 8158bbd17b994d2eb96f560bd0f1b88d P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 5. Current term is 6. Ops: []
W20260709 13:19:54.656374 16115 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.640s	user 0.008s	sys 0.010s
W20260709 13:19:54.729434 16115 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.713s	user 0.013s	sys 0.013s
W20260709 13:19:54.729979 16115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 713993us
W20260709 13:19:54.732612 16115 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:54.626967 18372 consensus_queue.cc:1048] T 8a878999c0024d2abe517484180290fc P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.090s
W20260709 13:19:54.604077 16221 rpcz_store.cc:269] 0709 13:19:52.372419 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:53.882550 (+1510131us) service_pool.cc:224] Handling call
0709 13:19:54.458709 (+576159us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:54.533701 18598 consensus_queue.cc:260] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:54.654798 18632 raft_consensus.cc:2804] T e5ad4eb281564936a59f41e387a47156 P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:19:54.743943 16823 raft_consensus.cc:2468] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 5.
I20260709 13:19:54.631635 18255 leader_election.cc:290] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 6 election: Requested vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:19:54.592031 18424 raft_consensus.cc:515] T ecc1e363230043fe9fef390f0b979f20 P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:19:54.716010 18619 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.881s	user 0.007s	sys 0.021s
I20260709 13:19:54.635708 18485 raft_consensus.cc:493] T 84f2da98864b4425bf7c0055951fdd4b P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:19:54.763521 16223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fffab26427e24d398e2695aad40ca134" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:19:54.791751 16223 raft_consensus.cc:3060] T fffab26427e24d398e2695aad40ca134 P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:19:54.714146 18616 raft_consensus.cc:493] T ada5a004ba7c46819051e3ccec71ffdb P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:19:54.654457 18609 consensus_queue.cc:1048] T 7c2095a5f4ad49408dbc7f69c606120d P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.094s
I20260709 13:19:54.701152 18452 consensus_queue.cc:1059] T c0eab25ce4c64e018a041beeda2867db P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.052s
W20260709 13:19:54.770115 18642 log.cc:927] Time spent T b87147731a68400e8cce9b879daa7c5d P cb82cc7bd6db46d980c75237a1402860: Append to log took a long time: real 0.203s	user 0.000s	sys 0.001s
I20260709 13:19:54.825023 16115 leader_election.cc:304] T 23f82224f4564fea93572e8206bad1fe P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:19:54.771122 16147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ad5dbfa99ee4b8d9e28be92b3f36db5" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
W20260709 13:19:54.778600 18400 consensus_meta.cc:287] T c9944d5e4b8c4109af9955c4f08c9095 P cb82cc7bd6db46d980c75237a1402860: Time spent flushing consensus metadata: real 1.058s	user 0.011s	sys 0.020s
I20260709 13:19:54.766647 18418 raft_consensus.cc:493] T d16dfad0c3b540e5b4b86492ddf87208 P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:54.630861 18633 consensus_queue.cc:1048] T e4dc6b0541ca431ebc757beac00521f1 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.115s
W20260709 13:19:54.780862 18628 consensus_meta.cc:287] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac: Time spent flushing consensus metadata: real 0.710s	user 0.011s	sys 0.004s
W20260709 13:19:54.832664 16789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 240519us
I20260709 13:19:54.718262 18641 consensus_queue.cc:1048] T 8cc959fb7f704fba9009c0595467d074 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.096s
I20260709 13:19:54.766847 18639 raft_consensus.cc:493] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:54.782284 16293 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.777s	user 0.027s	sys 0.009s
W20260709 13:19:54.808373 18560 consensus_meta.cc:287] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b: Time spent flushing consensus metadata: real 0.911s	user 0.015s	sys 0.018s
W20260709 13:19:54.841632 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 411457us
I20260709 13:19:54.756174 18433 consensus_queue.cc:1059] T b87147731a68400e8cce9b879daa7c5d P 90f3da57796b488c9087a2f523adf045 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.067s
I20260709 13:19:54.774420 18495 leader_election.cc:290] T af57996a8be84d398f0f023e8d08ef77 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:19:54.843956 16299 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48674 (request call id 2109) took 1905 ms. Trace:
I20260709 13:19:54.853081 16299 rpcz_store.cc:276] 0709 13:19:52.938727 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:53.458143 (+519416us) service_pool.cc:224] Handling call
0709 13:19:53.458519 (+   376us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:53.711126 (+252607us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:54.843617 (+1132491us) spinlock_profiling.cc:243] Waited 252 ms on lock 0x7b5400163854. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:19:54.843893 (+   276us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":579465472}
I20260709 13:19:54.842684 16789 leader_election.cc:304] T 57a744f5aaf944678a63469cbf514015 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:19:54.870951 16225 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33050 (request call id 458) took 1697 ms. Trace:
I20260709 13:19:54.871603 16225 rpcz_store.cc:276] 0709 13:19:53.173297 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:53.745071 (+571774us) service_pool.cc:224] Handling call
0709 13:19:53.745321 (+   250us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:53.974307 (+228986us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:54.870527 (+896220us) spinlock_profiling.cc:243] Waited 155 ms on lock 0x7b5400395654. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:19:54.870849 (+   322us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":355382528}
I20260709 13:19:54.873359 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c74490c40243a88e62d1760b5cd814" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:54.875908 16145 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40844 (request call id 1864) took 2673 ms. Trace:
I20260709 13:19:54.876483 16145 rpcz_store.cc:276] 0709 13:19:52.203194 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:53.043257 (+840063us) service_pool.cc:224] Handling call
0709 13:19:53.058534 (+ 15277us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:53.728532 (+669998us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:54.875600 (+1147068us) spinlock_profiling.cc:243] Waited 669 ms on lock 0x7b540028f354. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:19:54.875832 (+   232us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1539567872}
W20260709 13:19:54.881474 16190 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:54.883322 18255 raft_consensus.cc:493] T 5a959dbecc2648eb85d1cc550f5a3245 P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:54.884052 18255 raft_consensus.cc:515] T 5a959dbecc2648eb85d1cc550f5a3245 P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:54.878103 16145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1df93779209748b9860f6d893dd3de53" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:19:54.898365 16145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746aabf856ab401c9ef5e859371e7b67" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:19:54.900274 16145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b7d1ab519044eda2d5e04a07242f25" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:19:54.887029 18255 leader_election.cc:290] T 5a959dbecc2648eb85d1cc550f5a3245 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:54.874971 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8638ee363273426db1393dd81e5c7b72" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:19:54.805168 18632 raft_consensus.cc:697] T e5ad4eb281564936a59f41e387a47156 P 1ca2725d290e47d68b1a4b609c9816ac [term 5 LEADER]: Becoming Leader. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:19:54.911232 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8158bbd17b994d2eb96f560bd0f1b88d" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:19:54.919906 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74ca8140f35c4ae29e8a664f6bdfc829" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
W20260709 13:19:54.844890 16293 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.841s	user 0.027s	sys 0.014s
I20260709 13:19:54.921811 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a878999c0024d2abe517484180290fc" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:19:54.918340 16145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 5 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:19:54.748759 18530 consensus_queue.cc:260] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:54.930612 16145 raft_consensus.cc:2468] T ecc1e363230043fe9fef390f0b979f20 P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 4.
W20260709 13:19:54.931831 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 296201us
I20260709 13:19:54.852026 16221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0eab25ce4c64e018a041beeda2867db" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:54.939893 16224 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33050 (request call id 459) took 1766 ms. Trace:
I20260709 13:19:54.948060 16224 rpcz_store.cc:276] 0709 13:19:53.173544 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:53.815817 (+642273us) service_pool.cc:224] Handling call
0709 13:19:53.826372 (+ 10555us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:54.485860 (+659488us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:54.939562 (+453702us) spinlock_profiling.cc:243] Waited 659 ms on lock 0x7b54001414d4. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:19:54.939815 (+   253us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1515103616}
I20260709 13:19:54.941263 16220 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57604 (request call id 2041) took 2185 ms. Trace:
W20260709 13:19:54.930095 18539 consensus_meta.cc:287] T cdbd78acb4cf46298e0de8ba0d130e19 P 90f3da57796b488c9087a2f523adf045: Time spent flushing consensus metadata: real 1.194s	user 0.021s	sys 0.008s
I20260709 13:19:54.328890 16292 rpcz_store.cc:276] 0709 13:19:53.082900 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:53.711084 (+628184us) service_pool.cc:224] Handling call
0709 13:19:53.711407 (+   323us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:53.711623 (+   216us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:19:53.848140 (+136517us) log.cc:844] Serialized 14 byte log entry
0709 13:19:54.141596 (+293456us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:54.145153 (+  3557us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:19:54.150241 (+  5088us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:54.152988 (+  2747us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:54.154509 (+  1521us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":135038,"prepare.run_cpu_time_us":51,"prepare.run_wall_time_us":53,"raft.queue_time_us":52792,"raft.run_cpu_time_us":149,"raft.run_wall_time_us":17314,"thread_start_us":371607,"threads_started":3,"wal-append.queue_time_us":76149}
I20260709 13:19:54.910212 18400 raft_consensus.cc:2804] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P cb82cc7bd6db46d980c75237a1402860 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:19:54.808221 18485 raft_consensus.cc:515] T 84f2da98864b4425bf7c0055951fdd4b P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:54.950212 16220 rpcz_store.cc:276] 0709 13:19:52.755741 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:53.730162 (+974421us) service_pool.cc:224] Handling call
0709 13:19:53.730445 (+   283us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:54.461224 (+730779us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:54.940966 (+479742us) spinlock_profiling.cc:243] Waited 665 ms on lock 0x7b5400138654. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:19:54.941187 (+   221us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1530504448}
I20260709 13:19:54.977463 16220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d16dfad0c3b540e5b4b86492ddf87208" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:54.978775 16220 raft_consensus.cc:2410] T d16dfad0c3b540e5b4b86492ddf87208 P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 for term 5 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
W20260709 13:19:54.941712 16117 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:54.980011 16117 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:54.980670 16117 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:54.844261 18628 raft_consensus.cc:515] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:54.821200 18616 raft_consensus.cc:515] T ada5a004ba7c46819051e3ccec71ffdb P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:54.840991 18655 raft_consensus.cc:2804] T 23f82224f4564fea93572e8206bad1fe P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:19:54.931677 16142 raft_consensus.cc:1240] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 13:19:54.975028 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87ac0d1f7e8a419f9bcbc04aee0e067e" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:55.000013 16224 raft_consensus.cc:2410] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 for term 6 because replica has last-logged OpId of term: 4 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 13:19:55.000102 16145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fffab26427e24d398e2695aad40ca134" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:19:55.000691 16145 raft_consensus.cc:3060] T fffab26427e24d398e2695aad40ca134 P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:19:54.844998 18418 raft_consensus.cc:515] T d16dfad0c3b540e5b4b86492ddf87208 P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:54.909583 18560 raft_consensus.cc:515] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:54.944298 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68a6838d82ef4cac83507f906139d90f" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:54.963675 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a878999c0024d2abe517484180290fc" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:19:55.033152 16299 raft_consensus.cc:2393] T 8a878999c0024d2abe517484180290fc P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 5: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
I20260709 13:19:54.959885 18634 raft_consensus.cc:493] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:54.919724 18495 consensus_queue.cc:1059] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.045s
I20260709 13:19:54.809331 18432 consensus_queue.cc:237] T 9e65676b21b748728ad39668cfe722ec P 90f3da57796b488c9087a2f523adf045 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:54.853987 18639 raft_consensus.cc:515] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:55.001618 16224 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57604 (request call id 2054) took 1336 ms. Trace:
I20260709 13:19:55.042689 16224 rpcz_store.cc:276] 0709 13:19:53.665570 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:55.000645 (+1335075us) service_pool.cc:224] Handling call
0709 13:19:55.000911 (+   266us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:55.001081 (+   170us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:55.001105 (+    24us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:55.001143 (+    38us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:19:55.001169 (+    26us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:55.001220 (+    51us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:55.001564 (+   344us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:55.044880 16224 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57604 (request call id 2055) took 1376 ms. Trace:
I20260709 13:19:55.045305 16224 rpcz_store.cc:276] 0709 13:19:53.668262 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:55.043900 (+1375638us) service_pool.cc:224] Handling call
0709 13:19:55.044259 (+   359us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:55.044480 (+   221us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:55.044520 (+    40us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:55.044574 (+    54us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:19:55.044624 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:55.044693 (+    69us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:55.044819 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:19:54.962829 18513 consensus_meta.cc:287] T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b: Time spent flushing consensus metadata: real 0.999s	user 0.017s	sys 0.014s
I20260709 13:19:54.973204 18651 raft_consensus.cc:2749] T 57a744f5aaf944678a63469cbf514015 P cb82cc7bd6db46d980c75237a1402860 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:19:55.026412 16225 raft_consensus.cc:2468] T 68a6838d82ef4cac83507f906139d90f P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 5.
W20260709 13:19:54.953329 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 324955us
I20260709 13:19:54.984848 18530 raft_consensus.cc:3060] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:19:55.110761 16789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 254867us
I20260709 13:19:54.975359 18400 raft_consensus.cc:493] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P cb82cc7bd6db46d980c75237a1402860 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:19:55.112290 16789 leader_election.cc:304] T 26354361e9b044cea1c2b86c4882fe92 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
W20260709 13:19:55.112824 16117 leader_election.cc:336] T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923): Timed out: RequestConsensusVote RPC to 127.14.177.132:38923 timed out after 1.681s (SENT)
I20260709 13:19:55.117699 16147 raft_consensus.cc:2393] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 5: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
W20260709 13:19:55.121978 16821 consensus_meta.cc:287] T 19baa7bc367844c288bca8f81439c413 P cb82cc7bd6db46d980c75237a1402860: Time spent flushing consensus metadata: real 0.742s	user 0.020s	sys 0.006s
I20260709 13:19:55.122623 16821 raft_consensus.cc:2468] T 19baa7bc367844c288bca8f81439c413 P cb82cc7bd6db46d980c75237a1402860 [term 6 LEARNER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 6.
I20260709 13:19:55.126523 16820 raft_consensus.cc:1240] T 19baa7bc367844c288bca8f81439c413 P cb82cc7bd6db46d980c75237a1402860 [term 6 LEARNER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 5. Current term is 6. Ops: [5.5-5.5]
I20260709 13:19:55.128262 16821 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49896 (request call id 867) took 1189 ms. Trace:
I20260709 13:19:54.983525 16220 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57604 (request call id 2053) took 1321 ms. Trace:
I20260709 13:19:54.974056 18539 raft_consensus.cc:515] T cdbd78acb4cf46298e0de8ba0d130e19 P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:19:55.110421 18645 log.cc:927] Time spent T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.418s	user 0.000s	sys 0.001s
I20260709 13:19:55.111315 16292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5ad4eb281564936a59f41e387a47156" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:55.124430 16225 raft_consensus.cc:1275] T 8cc959fb7f704fba9009c0595467d074 P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:19:54.983892 18255 consensus_queue.cc:1048] T 18ad7e3af5504571a69d10c00750a01c P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.133s
I20260709 13:19:55.138975 16789 leader_election.cc:304] T 74ca8140f35c4ae29e8a664f6bdfc829 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac
I20260709 13:19:55.115099 18400 raft_consensus.cc:3060] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P cb82cc7bd6db46d980c75237a1402860 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:19:55.156360 16220 rpcz_store.cc:276] 0709 13:19:53.662343 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:54.982312 (+1319969us) service_pool.cc:224] Handling call
0709 13:19:54.982711 (+   399us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:54.983104 (+   393us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:54.983145 (+    41us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:54.983223 (+    78us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:19:54.983268 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:54.983337 (+    69us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:54.983465 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:19:55.192785 18657 log.cc:927] Time spent T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 0.106s	user 0.000s	sys 0.000s
I20260709 13:19:55.040997 18535 consensus_queue.cc:1048] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.101s
I20260709 13:19:55.137751 18659 raft_consensus.cc:493] T 3a7ca61592e848ea9b0fb81a2ce33cdc P cb82cc7bd6db46d980c75237a1402860 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:55.192631 18660 raft_consensus.cc:3055] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:19:55.138350 18651 raft_consensus.cc:493] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:55.193522 18560 leader_election.cc:290] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 5 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:55.193794 16292 raft_consensus.cc:2468] T e5ad4eb281564936a59f41e387a47156 P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 5.
I20260709 13:19:55.184321 16225 raft_consensus.cc:1275] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac [term 6 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 13:19:55.199973 16300 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40474 (request call id 356) took 2327 ms. Trace:
W20260709 13:19:55.209121 16789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 70297.3us
I20260709 13:19:55.219174 16300 rpcz_store.cc:276] 0709 13:19:52.872606 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:53.032416 (+159810us) service_pool.cc:224] Handling call
0709 13:19:53.032725 (+   309us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:53.696090 (+663365us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:55.199720 (+1503630us) spinlock_profiling.cc:243] Waited 632 ms on lock 0x7b540010ed54. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:19:55.199910 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1454093568}
I20260709 13:19:55.220818 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d16dfad0c3b540e5b4b86492ddf87208" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:19:55.221805 16300 raft_consensus.cc:2393] T d16dfad0c3b540e5b4b86492ddf87208 P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 4: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
I20260709 13:19:55.211329 16821 rpcz_store.cc:276] 0709 13:19:53.939150 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:54.070244 (+131094us) service_pool.cc:224] Handling call
0709 13:19:55.128158 (+1057914us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:55.224068 16821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8638ee363273426db1393dd81e5c7b72" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:19:55.181562 18406 leader_election.cc:290] T b6f07842d3aa47858df5665874c86f3a P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 5 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:55.228304 16789 leader_election.cc:400] T 9e65676b21b748728ad39668cfe722ec P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 5 election: Vote denied by peer 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389) with higher term. Message: Invalid argument: T 9e65676b21b748728ad39668cfe722ec P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 for earlier term 5. Current term is 6.
I20260709 13:19:55.228760 16789 leader_election.cc:403] T 9e65676b21b748728ad39668cfe722ec P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 5 election: Cancelling election due to peer responding with higher term
W20260709 13:19:55.212195 16117 leader_election.cc:336] T ee9ed67006144797b1de6ea8c2013a68 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923): Timed out: RequestConsensusVote RPC to 127.14.177.132:38923 timed out after 2.048s (SENT)
I20260709 13:19:55.131589 18513 raft_consensus.cc:515] T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:19:55.173146 16816 consensus_meta.cc:287] T 23f82224f4564fea93572e8206bad1fe P cb82cc7bd6db46d980c75237a1402860: Time spent flushing consensus metadata: real 1.053s	user 0.010s	sys 0.023s
I20260709 13:19:55.183034 18655 raft_consensus.cc:697] T 23f82224f4564fea93572e8206bad1fe P 647a343a542d4a139fe106c19095129b [term 3 LEADER]: Becoming Leader. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
W20260709 13:19:55.230393 16117 leader_election.cc:336] T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923): Timed out: RequestConsensusVote RPC to 127.14.177.132:38923 timed out after 2.173s (SENT)
I20260709 13:19:55.230290 16823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada5a004ba7c46819051e3ccec71ffdb" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:19:55.248481 16823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b7d1ab519044eda2d5e04a07242f25" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
W20260709 13:19:55.206758 18114 consensus_meta.cc:287] T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac: Time spent flushing consensus metadata: real 1.156s	user 0.008s	sys 0.021s
I20260709 13:19:55.249603 16823 raft_consensus.cc:2468] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 3.
I20260709 13:19:55.226636 16821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746aabf856ab401c9ef5e859371e7b67" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:19:55.205571 18568 raft_consensus.cc:2749] T 26354361e9b044cea1c2b86c4882fe92 P cb82cc7bd6db46d980c75237a1402860 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:19:55.090025 18634 raft_consensus.cc:515] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:55.256374 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a959dbecc2648eb85d1cc550f5a3245" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:55.199693 18661 raft_consensus.cc:493] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:55.208321 16142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8158bbd17b994d2eb96f560bd0f1b88d" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:19:55.259620 16823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 5 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:19:55.198421 18457 leader_election.cc:290] T 434bff3082d141838068f323bdb0d1ed P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:55.261307 16821 raft_consensus.cc:3060] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:19:55.261226 16192 leader_election.cc:304] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 
W20260709 13:19:55.262307 16114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:55.207386 18628 leader_election.cc:290] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 5 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:19:55.204597 18485 leader_election.cc:290] T 84f2da98864b4425bf7c0055951fdd4b P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:19:55.206454 18616 leader_election.cc:290] T ada5a004ba7c46819051e3ccec71ffdb P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:19:55.289255 18628 raft_consensus.cc:493] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:55.289813 18628 raft_consensus.cc:515] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:55.290525 18616 raft_consensus.cc:493] T 18ad7e3af5504571a69d10c00750a01c P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:19:55.214680 18660 raft_consensus.cc:740] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:19:55.268920 16145 raft_consensus.cc:2468] T fffab26427e24d398e2695aad40ca134 P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 4.
I20260709 13:19:55.251657 16816 raft_consensus.cc:2468] T 23f82224f4564fea93572e8206bad1fe P cb82cc7bd6db46d980c75237a1402860 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 3.
W20260709 13:19:55.274026 18347 consensus_meta.cc:287] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045: Time spent flushing consensus metadata: real 1.296s	user 0.026s	sys 0.026s
I20260709 13:19:55.251464 18560 raft_consensus.cc:493] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:55.283737 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 327858us
I20260709 13:19:55.114971 18632 consensus_queue.cc:237] T e5ad4eb281564936a59f41e387a47156 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.4, Last appended by leader: 4, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:55.295266 16145 raft_consensus.cc:1240] T b6f07842d3aa47858df5665874c86f3a P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 4. Current term is 5. Ops: []
W20260709 13:19:55.264477 16789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 44901.8us
I20260709 13:19:55.268198 18406 raft_consensus.cc:493] T 829b4ad8c5a54eababeea1181b960d4b P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:55.305074 16816 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49896 (request call id 863) took 1367 ms. Trace:
I20260709 13:19:55.274541 18538 raft_consensus.cc:2749] T 9e65676b21b748728ad39668cfe722ec P cb82cc7bd6db46d980c75237a1402860 [term 6 FOLLOWER]: Leader election lost for term 5. Reason: Vote denied by peer 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389) with higher term. Message: Invalid argument: T 9e65676b21b748728ad39668cfe722ec P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 for earlier term 5. Current term is 6.
I20260709 13:19:55.292214 16142 raft_consensus.cc:3060] T 8158bbd17b994d2eb96f560bd0f1b88d P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:19:55.258844 18114 raft_consensus.cc:515] T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:55.289255 18485 raft_consensus.cc:493] T 3884f864a96d428785895e5463e5728f P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:55.294425 18666 raft_consensus.cc:3055] T b87147731a68400e8cce9b879daa7c5d P 90f3da57796b488c9087a2f523adf045 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:19:55.274478 18665 raft_consensus.cc:493] T 66ae176961b74813a58bc15c9c4c6f2a P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:19:55.263379 18670 raft_consensus.cc:2804] T 74ca8140f35c4ae29e8a664f6bdfc829 P cb82cc7bd6db46d980c75237a1402860 [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 13:19:55.314354 18456 consensus_meta.cc:287] T 8c11074579b046d99e585c2818e18a2c P cb82cc7bd6db46d980c75237a1402860: Time spent flushing consensus metadata: real 0.770s	user 0.014s	sys 0.009s
I20260709 13:19:55.325687 16145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af57996a8be84d398f0f023e8d08ef77" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:19:55.321976 16825 raft_consensus.cc:2468] T c9944d5e4b8c4109af9955c4f08c9095 P cb82cc7bd6db46d980c75237a1402860 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 6.
I20260709 13:19:55.301674 16190 leader_election.cc:304] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860
I20260709 13:19:55.349723 16145 raft_consensus.cc:2468] T af57996a8be84d398f0f023e8d08ef77 P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 4.
I20260709 13:19:55.297597 16292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada5a004ba7c46819051e3ccec71ffdb" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:19:55.227394 18659 raft_consensus.cc:515] T 3a7ca61592e848ea9b0fb81a2ce33cdc P cb82cc7bd6db46d980c75237a1402860 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:55.322904 16146 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "19baa7bc367844c288bca8f81439c413"
dest_uuid: "647a343a542d4a139fe106c19095129b"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:40822
I20260709 13:19:55.370074 16146 raft_consensus.cc:606] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Received request to transfer leadership
I20260709 13:19:55.370441 16146 raft_consensus.cc:614] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:19:55.291090 18616 raft_consensus.cc:515] T 18ad7e3af5504571a69d10c00750a01c P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } }
I20260709 13:19:55.342115 18666 raft_consensus.cc:740] T b87147731a68400e8cce9b879daa7c5d P 90f3da57796b488c9087a2f523adf045 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
W20260709 13:19:55.350565 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 49006.7us
I20260709 13:19:55.314739 16816 rpcz_store.cc:276] 0709 13:19:53.937741 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:54.118190 (+180449us) service_pool.cc:224] Handling call
0709 13:19:55.305001 (+1186811us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:55.351933 16145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434bff3082d141838068f323bdb0d1ed" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
W20260709 13:19:55.355151 16266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 55461.6us
W20260709 13:19:55.338469 18488 consensus_meta.cc:287] T e4dc6b0541ca431ebc757beac00521f1 P 90f3da57796b488c9087a2f523adf045: Time spent flushing consensus metadata: real 1.182s	user 0.004s	sys 0.011s
W20260709 13:19:55.374580 16266 leader_election.cc:336] T 23f82224f4564fea93572e8206bad1fe P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923): Timed out: RequestConsensusVote RPC to 127.14.177.132:38923 timed out after 1.993s (SENT)
I20260709 13:19:55.340127 18418 leader_election.cc:290] T d16dfad0c3b540e5b4b86492ddf87208 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
W20260709 13:19:55.375145 16265 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.696s	user 0.009s	sys 0.022s
W20260709 13:19:55.383881 16265 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.705s	user 0.009s	sys 0.022s
I20260709 13:19:55.252249 18651 raft_consensus.cc:515] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:55.377377 16266 leader_election.cc:304] T 23f82224f4564fea93572e8206bad1fe P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
W20260709 13:19:55.372829 18654 log.cc:927] Time spent T 57a744f5aaf944678a63469cbf514015 P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.070s	user 0.001s	sys 0.000s
W20260709 13:19:55.384208 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 705129us
I20260709 13:19:55.312541 18580 raft_consensus.cc:493] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
W20260709 13:19:55.386411 16192 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 125293us
I20260709 13:19:55.344854 18670 raft_consensus.cc:493] T 74ca8140f35c4ae29e8a664f6bdfc829 P cb82cc7bd6db46d980c75237a1402860 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:19:55.387229 18551 consensus_meta.cc:287] T 08c74490c40243a88e62d1760b5cd814 P 1ca2725d290e47d68b1a4b609c9816ac: Time spent flushing consensus metadata: real 1.233s	user 0.016s	sys 0.010s
I20260709 13:19:55.394340 16816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada5a004ba7c46819051e3ccec71ffdb" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:19:55.394971 18255 raft_consensus.cc:2804] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:19:55.433130 18255 raft_consensus.cc:493] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Starting leader election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:19:55.382959 18488 raft_consensus.cc:515] T e4dc6b0541ca431ebc757beac00521f1 P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:55.433558 18255 raft_consensus.cc:3060] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:19:55.433751 16816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a9de347c711477e87c582d003f36925" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:19:55.435101 18488 leader_election.cc:290] T e4dc6b0541ca431ebc757beac00521f1 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 5 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:19:55.311621 18347 raft_consensus.cc:515] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:55.439698 18513 leader_election.cc:290] T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 5 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:55.440761 18347 leader_election.cc:290] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 6 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
W20260709 13:19:55.441219 18646 log.cc:927] Time spent T c2f5f89186204242bf377f7c42b2e2da P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 0.738s	user 0.003s	sys 0.000s
W20260709 13:19:55.441431 18663 log.cc:927] Time spent T 2794af5b99984efebdee564035841cb0 P cb82cc7bd6db46d980c75237a1402860: Append to log took a long time: real 0.065s	user 0.000s	sys 0.000s
I20260709 13:19:55.321305 18406 raft_consensus.cc:515] T 829b4ad8c5a54eababeea1181b960d4b P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:55.442579 18666 consensus_queue.cc:260] T b87147731a68400e8cce9b879daa7c5d P 90f3da57796b488c9087a2f523adf045 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.6, Last appended by leader: 6, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:55.443431 18539 leader_election.cc:290] T cdbd78acb4cf46298e0de8ba0d130e19 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 6 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:19:55.443760 18666 raft_consensus.cc:3060] T b87147731a68400e8cce9b879daa7c5d P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:19:55.444725 16296 raft_consensus.cc:1240] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 5. Current term is 6. Ops: [5.7-5.7]
I20260709 13:19:55.396488 18114 leader_election.cc:290] T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 6 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
W20260709 13:19:55.449591 18650 log.cc:927] Time spent T 7c2095a5f4ad49408dbc7f69c606120d P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 0.257s	user 0.000s	sys 0.001s
I20260709 13:19:55.449805 18114 consensus_queue.cc:1048] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:19:55.450371 16265 leader_election.cc:304] T fffab26427e24d398e2695aad40ca134 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:19:55.450946 16144 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40830 (request call id 1140) took 1290 ms. Trace:
I20260709 13:19:55.451427 16144 rpcz_store.cc:276] 0709 13:19:54.160716 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:54.226259 (+ 65543us) service_pool.cc:224] Handling call
0709 13:19:54.301795 (+ 75536us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:54.317849 (+ 16054us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:54.317899 (+    50us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:54.344910 (+ 27011us) log.cc:844] Serialized 26 byte log entry
0709 13:19:54.725704 (+380794us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:19:54.725977 (+   273us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:54.726478 (+   501us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:55.450719 (+724241us) raft_consensus.cc:1685] finished
0709 13:19:55.450739 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:55.450883 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":14416,"raft.run_cpu_time_us":280,"raft.run_wall_time_us":57694,"thread_start_us":326201,"threads_started":1,"wal-append.queue_time_us":647385}
I20260709 13:19:55.396683 18634 leader_election.cc:290] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:19:55.360914 18652 raft_consensus.cc:493] T c9944d5e4b8c4109af9955c4f08c9095 P cb82cc7bd6db46d980c75237a1402860 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:55.314040 18560 raft_consensus.cc:515] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:55.347714 18485 raft_consensus.cc:515] T 3884f864a96d428785895e5463e5728f P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:19:55.404706 16116 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.559s	user 0.007s	sys 0.054s
W20260709 13:19:55.478423 16116 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.633s	user 0.008s	sys 0.055s
I20260709 13:19:55.403084 18530 raft_consensus.cc:2749] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:19:55.408619 18453 consensus_meta.cc:287] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860: Time spent flushing consensus metadata: real 1.321s	user 0.025s	sys 0.008s
W20260709 13:19:55.413098 18591 consensus_meta.cc:287] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b: Time spent flushing consensus metadata: real 1.445s	user 0.010s	sys 0.006s
I20260709 13:19:55.415948 18659 leader_election.cc:290] T 3a7ca61592e848ea9b0fb81a2ce33cdc P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:55.454079 18628 leader_election.cc:290] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:19:55.345224 18655 consensus_queue.cc:237] T 23f82224f4564fea93572e8206bad1fe P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:55.417587 16294 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48684 (request call id 2013) took 2171 ms. Trace:
W20260709 13:19:55.417840 18612 consensus_meta.cc:287] T 66f75124215f4ca6bef831e174a792d8 P 1ca2725d290e47d68b1a4b609c9816ac: Time spent flushing consensus metadata: real 1.461s	user 0.012s	sys 0.023s
I20260709 13:19:55.526957 18612 raft_consensus.cc:515] T 66f75124215f4ca6bef831e174a792d8 P 1ca2725d290e47d68b1a4b609c9816ac [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:55.373685 18389 raft_consensus.cc:493] T 66f75124215f4ca6bef831e174a792d8 P cb82cc7bd6db46d980c75237a1402860 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:55.427896 16266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 53348us
W20260709 13:19:55.552363 16266 leader_election.cc:336] T 434bff3082d141838068f323bdb0d1ed P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923): Timed out: RequestConsensusVote RPC to 127.14.177.132:38923 timed out after 2.015s (SENT)
I20260709 13:19:55.552896 16266 leader_election.cc:304] T 434bff3082d141838068f323bdb0d1ed P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
I20260709 13:19:55.378046 16821 raft_consensus.cc:2468] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 5.
I20260709 13:19:55.383409 16292 raft_consensus.cc:3060] T ada5a004ba7c46819051e3ccec71ffdb P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:19:55.432219 18424 leader_election.cc:290] T ecc1e363230043fe9fef390f0b979f20 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
W20260709 13:19:55.393421 18558 consensus_meta.cc:287] T 434bff3082d141838068f323bdb0d1ed P 647a343a542d4a139fe106c19095129b: Time spent flushing consensus metadata: real 1.453s	user 0.004s	sys 0.012s
I20260709 13:19:55.493917 18639 leader_election.cc:290] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:55.453604 16225 raft_consensus.cc:1275] T 5a959dbecc2648eb85d1cc550f5a3245 P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
W20260709 13:19:55.397835 18473 consensus_meta.cc:287] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860: Time spent flushing consensus metadata: real 0.775s	user 0.005s	sys 0.010s
I20260709 13:19:55.276732 18661 raft_consensus.cc:515] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:55.363790 18456 raft_consensus.cc:515] T 8c11074579b046d99e585c2818e18a2c P cb82cc7bd6db46d980c75237a1402860 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:55.462771 16220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a96373bb0e14dce8a266b2fc6c58507" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:19:55.586122 16220 raft_consensus.cc:2393] T 5a96373bb0e14dce8a266b2fc6c58507 P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 4: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
I20260709 13:19:55.587150 16220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a878999c0024d2abe517484180290fc" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:19:55.588652 16220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4dc6b0541ca431ebc757beac00521f1" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:19:55.589902 16220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87ac0d1f7e8a419f9bcbc04aee0e067e" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:55.590540 16220 raft_consensus.cc:2468] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 5.
I20260709 13:19:55.384117 18660 consensus_queue.cc:260] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:55.475418 18123 raft_consensus.cc:2804] T fffab26427e24d398e2695aad40ca134 P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:19:55.430341 16291 raft_consensus.cc:1240] T c4f65cdf00b748bb963ee6e81ef43b79 P 90f3da57796b488c9087a2f523adf045 [term 7 LEADER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 6. Current term is 7. Ops: []
W20260709 13:19:55.482842 16116 leader_election.cc:336] T 8d4ab870a05347868956aa8b841b2981 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943): Timed out: RequestConsensusVote RPC to 127.14.177.130:35943 timed out after 1.538s (SENT)
I20260709 13:19:55.592702 16116 leader_election.cc:304] T 8d4ab870a05347868956aa8b841b2981 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
W20260709 13:19:55.488046 18668 log.cc:927] Time spent T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860: Append to log took a long time: real 0.126s	user 0.000s	sys 0.001s
I20260709 13:19:55.488476 18680 raft_consensus.cc:2749] T 23f82224f4564fea93572e8206bad1fe P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:19:55.349010 18665 raft_consensus.cc:515] T 66ae176961b74813a58bc15c9c4c6f2a P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:55.464718 16218 raft_consensus.cc:2393] T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 6: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
I20260709 13:19:55.430733 18670 raft_consensus.cc:3060] T 74ca8140f35c4ae29e8a664f6bdfc829 P cb82cc7bd6db46d980c75237a1402860 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:19:55.488034 18406 leader_election.cc:290] T 829b4ad8c5a54eababeea1181b960d4b P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:19:55.475732 18672 consensus_queue.cc:1048] T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.169s
I20260709 13:19:55.499388 18652 raft_consensus.cc:515] T c9944d5e4b8c4109af9955c4f08c9095 P cb82cc7bd6db46d980c75237a1402860 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:55.416199 16819 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49892 (request call id 367) took 4364 ms. Trace:
W20260709 13:19:55.506717 16223 consensus_meta.cc:287] T fffab26427e24d398e2695aad40ca134 P 1ca2725d290e47d68b1a4b609c9816ac: Time spent flushing consensus metadata: real 0.714s	user 0.008s	sys 0.006s
I20260709 13:19:55.512070 18255 raft_consensus.cc:515] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:55.519047 18643 consensus_queue.cc:1059] T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.026s
I20260709 13:19:55.477926 16816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af57996a8be84d398f0f023e8d08ef77" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
W20260709 13:19:55.522114 18582 consensus_meta.cc:287] T 02838c86531d4f98bf58d1c22b8baf77 P 90f3da57796b488c9087a2f523adf045: Time spent flushing consensus metadata: real 1.148s	user 0.017s	sys 0.004s
I20260709 13:19:55.523667 18487 leader_election.cc:290] T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
W20260709 13:19:55.526252 18649 log.cc:927] Time spent T 9c61a39fd4f84424b129c1a6f9ba404d P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 0.388s	user 0.001s	sys 0.000s
I20260709 13:19:55.431663 18580 raft_consensus.cc:515] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:19:55.545048 16115 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.571s	user 0.001s	sys 0.034s
W20260709 13:19:55.625602 16115 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.652s	user 0.001s	sys 0.034s
I20260709 13:19:55.609755 18651 leader_election.cc:290] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:55.447077 18551 raft_consensus.cc:515] T 08c74490c40243a88e62d1760b5cd814 P 1ca2725d290e47d68b1a4b609c9816ac [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:19:55.563316 16264 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:55.636719 16264 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:55.564950 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 263397us
W20260709 13:19:55.639752 16790 leader_election.cc:336] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943): Timed out: RequestConsensusVote RPC to 127.14.177.130:35943 timed out after 3.194s (SENT)
I20260709 13:19:55.567241 16192 leader_election.cc:304] T 746aabf856ab401c9ef5e859371e7b67 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:19:55.573163 18612 leader_election.cc:290] T 66f75124215f4ca6bef831e174a792d8 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 6 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:19:55.573897 18389 raft_consensus.cc:515] T 66f75124215f4ca6bef831e174a792d8 P cb82cc7bd6db46d980c75237a1402860 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:55.401330 16142 raft_consensus.cc:2468] T 8158bbd17b994d2eb96f560bd0f1b88d P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 6.
I20260709 13:19:55.486346 18609 consensus_queue.cc:1048] T 5a959dbecc2648eb85d1cc550f5a3245 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.046s
I20260709 13:19:55.576064 16217 raft_consensus.cc:1240] T 829b4ad8c5a54eababeea1181b960d4b P 1ca2725d290e47d68b1a4b609c9816ac [term 6 LEADER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 5. Current term is 6. Ops: []
W20260709 13:19:55.582037 16266 leader_election.cc:336] T c9944d5e4b8c4109af9955c4f08c9095 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923): Timed out: RequestConsensusVote RPC to 127.14.177.132:38923 timed out after 1.867s (SENT)
I20260709 13:19:55.583745 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6f07842d3aa47858df5665874c86f3a" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:19:55.657403 16225 raft_consensus.cc:3060] T b6f07842d3aa47858df5665874c86f3a P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:19:55.657343 16266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 75416.1us
I20260709 13:19:55.586050 16145 raft_consensus.cc:2468] T 434bff3082d141838068f323bdb0d1ed P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 5.
W20260709 13:19:55.597023 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 114438us
I20260709 13:19:55.600697 18674 raft_consensus.cc:2749] T 434bff3082d141838068f323bdb0d1ed P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:19:55.495551 18634 raft_consensus.cc:493] T 2794af5b99984efebdee564035841cb0 P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:55.505450 18453 raft_consensus.cc:515] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:55.608741 18123 raft_consensus.cc:697] T fffab26427e24d398e2695aad40ca134 P 90f3da57796b488c9087a2f523adf045 [term 4 LEADER]: Becoming Leader. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:19:55.615907 16223 raft_consensus.cc:2468] T fffab26427e24d398e2695aad40ca134 P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 4.
I20260709 13:19:55.504339 18591 raft_consensus.cc:515] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } }
I20260709 13:19:55.595753 16219 raft_consensus.cc:1240] T 66f75124215f4ca6bef831e174a792d8 P 1ca2725d290e47d68b1a4b609c9816ac [term 6 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 5. Current term is 6. Ops: []
I20260709 13:19:55.617165 16220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5ad4eb281564936a59f41e387a47156" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:55.602238 18558 raft_consensus.cc:2749] T 8d4ab870a05347868956aa8b841b2981 P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:19:55.550344 16191 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.595s	user 0.014s	sys 0.014s
W20260709 13:19:55.675863 16191 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.720s	user 0.014s	sys 0.014s
W20260709 13:19:55.628685 16115 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:55.629674 18681 raft_consensus.cc:493] T 3884f864a96d428785895e5463e5728f P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:55.630036 18532 consensus_meta.cc:287] T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac: Time spent flushing consensus metadata: real 1.550s	user 0.021s	sys 0.009s
I20260709 13:19:55.632464 18580 leader_election.cc:290] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
W20260709 13:19:55.620501 18595 consensus_meta.cc:287] T 68a6838d82ef4cac83507f906139d90f P 647a343a542d4a139fe106c19095129b: Time spent flushing consensus metadata: real 1.507s	user 0.012s	sys 0.012s
I20260709 13:19:55.594476 18473 raft_consensus.cc:515] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:55.518162 16294 rpcz_store.cc:276] 0709 13:19:53.245914 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:54.116825 (+870911us) service_pool.cc:224] Handling call
0709 13:19:54.117126 (+   301us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:54.757303 (+640177us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:55.417272 (+659969us) spinlock_profiling.cc:243] Waited 426 ms on lock 0x7b54000cf154. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:19:55.417514 (+   242us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":979969920}
W20260709 13:19:55.645509 16192 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 78423.7us
I20260709 13:19:55.643922 18675 raft_consensus.cc:493] T 8c11074579b046d99e585c2818e18a2c P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
W20260709 13:19:55.649247 18588 consensus_meta.cc:287] T 31969392ccdb43b491911a319e8bc7e1 P 90f3da57796b488c9087a2f523adf045: Time spent flushing consensus metadata: real 1.660s	user 0.017s	sys 0.007s
I20260709 13:19:55.649933 18665 leader_election.cc:290] T 66ae176961b74813a58bc15c9c4c6f2a P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:19:55.650408 16222 raft_consensus.cc:1240] T 08c74490c40243a88e62d1760b5cd814 P 1ca2725d290e47d68b1a4b609c9816ac [term 6 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 5. Current term is 6. Ops: []
I20260709 13:19:55.640934 16790 leader_election.cc:304] T af57996a8be84d398f0f023e8d08ef77 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860; no voters: 
W20260709 13:19:55.658092 16266 leader_election.cc:336] T 9e65676b21b748728ad39668cfe722ec P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923): Timed out: RequestConsensusVote RPC to 127.14.177.132:38923 timed out after 2.073s (SENT)
W20260709 13:19:55.744067 16266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 86002.1us
W20260709 13:19:55.658663 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 284976us
I20260709 13:19:55.745355 16190 leader_election.cc:304] T 5a96373bb0e14dce8a266b2fc6c58507 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:19:55.620190 18660 raft_consensus.cc:3060] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:19:55.659523 16116 leader_election.cc:304] T fffab26427e24d398e2695aad40ca134 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
I20260709 13:19:55.667434 16223 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57600 (request call id 1124) took 1026 ms. Trace:
I20260709 13:19:55.746388 16223 rpcz_store.cc:276] 0709 13:19:54.640939 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:54.724805 (+ 83866us) service_pool.cc:224] Handling call
0709 13:19:55.667377 (+942572us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:55.747406 16223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "829b4ad8c5a54eababeea1181b960d4b" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
W20260709 13:19:55.669713 16117 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 356465us
I20260709 13:19:55.650861 18551 leader_election.cc:290] T 08c74490c40243a88e62d1760b5cd814 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 6 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
W20260709 13:19:55.676203 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 720727us
W20260709 13:19:55.750020 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 109182us
I20260709 13:19:55.681475 16823 raft_consensus.cc:2393] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 5: Already voted for candidate cb82cc7bd6db46d980c75237a1402860 in this term.
I20260709 13:19:55.752708 16266 leader_election.cc:304] T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:19:55.754050 16117 leader_election.cc:304] T 2794af5b99984efebdee564035841cb0 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860
I20260709 13:19:55.681912 18678 raft_consensus.cc:493] T 02838c86531d4f98bf58d1c22b8baf77 P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:55.682500 18255 leader_election.cc:290] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 4 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
W20260709 13:19:55.687695 18679 log.cc:927] Time spent T c4f65cdf00b748bb963ee6e81ef43b79 P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.065s	user 0.001s	sys 0.000s
I20260709 13:19:55.660122 18463 raft_consensus.cc:493] T 434bff3082d141838068f323bdb0d1ed P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:55.764885 16225 raft_consensus.cc:2468] T b6f07842d3aa47858df5665874c86f3a P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 5.
I20260709 13:19:55.628651 18582 raft_consensus.cc:515] T 02838c86531d4f98bf58d1c22b8baf77 P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:55.675917 16816 raft_consensus.cc:2410] T af57996a8be84d398f0f023e8d08ef77 P cb82cc7bd6db46d980c75237a1402860 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac for term 6 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:19:55.691056 16148 raft_consensus.cc:1240] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 4. Current term is 5. Ops: []
I20260709 13:19:55.733229 18123 consensus_queue.cc:237] T fffab26427e24d398e2695aad40ca134 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:55.734016 18661 leader_election.cc:290] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:55.615483 16819 rpcz_store.cc:276] 0709 13:19:51.051819 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:53.319260 (+2267441us) service_pool.cc:224] Handling call
0709 13:19:53.572555 (+253295us) spinlock_profiling.cc:243] Waited 3.75 ms on lock 0x7b5800004298. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69ad8fadb1 00007f69ada3ab71 00007f69ada3aa22 00007f69b6953c09 00007f69b5aadee0 00007f69b5b770ab 00007f69b5b77131 00007f69b5b38fc0 00007f69b5b23929 00007f69b5b233d2 00007f69af09abd4
0709 13:19:53.572747 (+   192us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:54.388775 (+816028us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:55.415960 (+1027185us) spinlock_profiling.cc:243] Waited 552 ms on lock 0x7b54004e0cd4. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:19:55.416138 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1278645760}
W20260709 13:19:55.735391 18687 log.cc:927] Time spent T 829b4ad8c5a54eababeea1181b960d4b P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 0.061s	user 0.001s	sys 0.000s
I20260709 13:19:55.735827 18473 leader_election.cc:290] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 6 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:55.736183 16294 raft_consensus.cc:1240] T c0eab25ce4c64e018a041beeda2867db P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 4. Current term is 5. Ops: []
I20260709 13:19:55.736686 18456 leader_election.cc:290] T 8c11074579b046d99e585c2818e18a2c P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 6 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:55.656193 18676 raft_consensus.cc:2804] T 746aabf856ab401c9ef5e859371e7b67 P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:19:55.764109 18609 raft_consensus.cc:2804] T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:19:55.656010 18400 raft_consensus.cc:515] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P cb82cc7bd6db46d980c75237a1402860 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:55.739071 16822 raft_consensus.cc:1240] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 [term 6 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 5. Current term is 6. Ops: [5.7-5.7]
I20260709 13:19:55.739126 16151 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40830 (request call id 1141) took 1487 ms. Trace:
W20260709 13:19:55.749119 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 89700.1us
I20260709 13:19:55.794593 16116 leader_election.cc:304] T ee9ed67006144797b1de6ea8c2013a68 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860
I20260709 13:19:55.794190 16151 rpcz_store.cc:276] 0709 13:19:54.251955 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:54.364191 (+112236us) service_pool.cc:224] Handling call
0709 13:19:54.364571 (+   380us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:54.365022 (+   451us) raft_consensus.cc:1534] Early marking committed up to index 1
0709 13:19:54.365060 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:54.381646 (+ 16586us) log.cc:844] Serialized 26 byte log entry
0709 13:19:54.795365 (+413719us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:19:54.795432 (+    67us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:54.795537 (+   105us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:55.525311 (+729774us) spinlock_profiling.cc:243] Waited 17.2 ms on lock 0x7b4400022b4c. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69ad8fadb1 00007f69ad998c85 00007f69ad9996a2 00007f69b070b1db 00007f69b0706aaf 00007f69b06fa86e 00007f69b06f9bb6 00007f69b06f97d8 00007f69b0703221 00007f69b07031d9 00007f69b0703169
0709 13:19:55.692491 (+167180us) raft_consensus.cc:1685] finished
0709 13:19:55.692513 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:55.739051 (+ 46538us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":112,"raft.run_cpu_time_us":127,"raft.run_wall_time_us":13491,"spinlock_wait_cycles":39639808,"thread_start_us":394925,"threads_started":1,"wal-append.queue_time_us":595309}
W20260709 13:19:55.680585 16789 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:55.756095 16117 leader_election.cc:304] T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:19:55.758934 16224 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57600 (request call id 1113) took 1379 ms. Trace:
I20260709 13:19:55.761750 16191 leader_election.cc:304] T 1df93779209748b9860f6d893dd3de53 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:19:55.766441 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7ca61592e848ea9b0fb81a2ce33cdc" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:55.767589 16223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f2da98864b4425bf7c0055951fdd4b" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
W20260709 13:19:55.768563 16818 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49892 (request call id 360) took 5603 ms (client timeout 2040 ms). Trace:
I20260709 13:19:55.772958 16824 raft_consensus.cc:1275] T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860 [term 4 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:19:55.675400 18634 raft_consensus.cc:515] T 2794af5b99984efebdee564035841cb0 P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:19:55.759429 18617 consensus_meta.cc:287] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 90f3da57796b488c9087a2f523adf045: Time spent flushing consensus metadata: real 1.686s	user 0.015s	sys 0.019s
I20260709 13:19:55.689464 18681 raft_consensus.cc:515] T 3884f864a96d428785895e5463e5728f P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:55.767609 18463 raft_consensus.cc:515] T 434bff3082d141838068f323bdb0d1ed P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:55.827666 18694 raft_consensus.cc:2749] T 5a96373bb0e14dce8a266b2fc6c58507 P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:19:55.760093 18558 raft_consensus.cc:2764] T fffab26427e24d398e2695aad40ca134 P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Leader pre-election decision vote started in defunct term 3: won
I20260709 13:19:55.691339 18532 raft_consensus.cc:515] T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:55.758359 18613 raft_consensus.cc:493] T 8cc959fb7f704fba9009c0595467d074 P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:55.738478 18560 leader_election.cc:290] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:19:55.691063 18595 raft_consensus.cc:515] T 68a6838d82ef4cac83507f906139d90f P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:55.796113 16116 leader_election.cc:304] T 7c2095a5f4ad49408dbc7f69c606120d P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 90f3da57796b488c9087a2f523adf045
I20260709 13:19:55.783929 16819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a9de347c711477e87c582d003f36925" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:19:55.762701 18665 consensus_queue.cc:1059] T 26354361e9b044cea1c2b86c4882fe92 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.031s
I20260709 13:19:55.783468 18688 consensus_queue.cc:1059] T ee9ed67006144797b1de6ea8c2013a68 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.019s
I20260709 13:19:55.765188 18504 consensus_queue.cc:1059] T 19baa7bc367844c288bca8f81439c413 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.031s
I20260709 13:19:55.793512 18609 raft_consensus.cc:493] T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:55.750584 18588 raft_consensus.cc:515] T 31969392ccdb43b491911a319e8bc7e1 P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:55.793752 18691 raft_consensus.cc:2749] T 2794af5b99984efebdee564035841cb0 P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:19:55.815328 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdbd78acb4cf46298e0de8ba0d130e19" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:19:55.814348 18421 raft_consensus.cc:2749] T ee9ed67006144797b1de6ea8c2013a68 P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:19:55.822222 18582 leader_election.cc:290] T 02838c86531d4f98bf58d1c22b8baf77 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 6 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:19:55.822332 18587 raft_consensus.cc:3055] T c0eab25ce4c64e018a041beeda2867db P 1ca2725d290e47d68b1a4b609c9816ac [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:19:55.822475 18453 leader_election.cc:290] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 5 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:55.824090 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68a6838d82ef4cac83507f906139d90f" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:19:55.824410 16223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:19:55.784387 18487 raft_consensus.cc:2804] T af57996a8be84d398f0f023e8d08ef77 P cb82cc7bd6db46d980c75237a1402860 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:19:55.782940 16817 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49892 (request call id 384) took 1871 ms. Trace:
I20260709 13:19:55.983824 16150 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40844 (request call id 1884) took 1209 ms. Trace:
I20260709 13:19:55.815788 18672 raft_consensus.cc:2804] T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:19:55.815631 16294 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48684 (request call id 2016) took 1883 ms. Trace:
I20260709 13:19:55.864943 16265 leader_election.cc:304] T 8158bbd17b994d2eb96f560bd0f1b88d P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
W20260709 13:19:55.876432 16117 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:55.944008 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 147938us
W20260709 13:19:55.985687 16117 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:55.799515 16818 rpcz_store.cc:269] 0709 13:19:50.165066 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:50.732518 (+567452us) service_pool.cc:224] Handling call
0709 13:19:55.768328 (+5035810us) spinlock_profiling.cc:243] Waited 2.64 s on lock 0x7b54006f2054. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:19:55.768465 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6064912640}
I20260709 13:19:55.980135 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada5a004ba7c46819051e3ccec71ffdb" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:55.862903 16816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e4bfb9090c8460eb14fa7e0d818e7f1" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:19:55.982959 16149 raft_consensus.cc:2393] T 68a6838d82ef4cac83507f906139d90f P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 7: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
W20260709 13:19:55.998553 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 133681us
I20260709 13:19:55.943380 18417 raft_consensus.cc:493] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:55.772406 18689 consensus_queue.cc:1059] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.103s
I20260709 13:19:55.987324 16142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f2da98864b4425bf7c0055951fdd4b" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:19:55.987509 18672 raft_consensus.cc:697] T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b [term 5 LEADER]: Becoming Leader. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:19:55.988909 16147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e9b2bf7011f4ae688a79d7bb5a447b4" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:19:55.989564 16223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2372981006d4b218215c28964dea623" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:19:55.990142 16144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a9de347c711477e87c582d003f36925" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
W20260709 13:19:55.991936 18696 log.cc:927] Time spent T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 0.173s	user 0.000s	sys 0.000s
I20260709 13:19:55.993420 18634 leader_election.cc:290] T 2794af5b99984efebdee564035841cb0 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:19:55.995396 18272 raft_consensus.cc:3055] T ee9ed67006144797b1de6ea8c2013a68 P 1ca2725d290e47d68b1a4b609c9816ac [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:19:55.995576 16819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f82224f4564fea93572e8206bad1fe" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:19:55.987329 18558 consensus_queue.cc:1059] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.010s
I20260709 13:19:55.996308 16294 rpcz_store.cc:276] 0709 13:19:53.932330 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:55.735635 (+1803305us) service_pool.cc:224] Handling call
0709 13:19:55.735990 (+   355us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:55.792947 (+ 56957us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:55.814047 (+ 21100us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:55.984257 16150 rpcz_store.cc:276] 0709 13:19:54.774260 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:54.774430 (+   170us) service_pool.cc:224] Handling call
0709 13:19:55.823932 (+1049502us) spinlock_profiling.cc:243] Waited 2.44 ms on lock 0x7b5800011198. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69ad8fadb1 00007f69ada3ab71 00007f69ada3aa22 00007f69b6953c09 00007f69b5aadee0 00007f69b5b770ab 00007f69b5b77131 00007f69b5b250a0 00007f69b5b24984 00007f69af09f6d4 00007f69af09f662
0709 13:19:55.983760 (+159828us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5616512}
I20260709 13:19:55.999466 16298 raft_consensus.cc:2393] T 31969392ccdb43b491911a319e8bc7e1 P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 5: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
I20260709 13:19:55.999832 16225 raft_consensus.cc:2393] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 6: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
I20260709 13:19:55.967365 18609 raft_consensus.cc:3060] T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:19:55.965505 18560 raft_consensus.cc:2804] T 7c2095a5f4ad49408dbc7f69c606120d P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:19:56.000990 16116 leader_election.cc:304] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
I20260709 13:19:55.862425 18617 raft_consensus.cc:515] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:55.982419 18587 raft_consensus.cc:740] T c0eab25ce4c64e018a041beeda2867db P 1ca2725d290e47d68b1a4b609c9816ac [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:19:56.005091 16223 raft_consensus.cc:2393] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 5: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
I20260709 13:19:56.001345 18123 raft_consensus.cc:2804] T 8158bbd17b994d2eb96f560bd0f1b88d P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:19:56.058215 16223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d16dfad0c3b540e5b4b86492ddf87208" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:56.059058 16223 raft_consensus.cc:2468] T d16dfad0c3b540e5b4b86492ddf87208 P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 4.
I20260709 13:19:56.006873 18681 leader_election.cc:290] T 3884f864a96d428785895e5463e5728f P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:19:56.007230 18595 leader_election.cc:290] T 68a6838d82ef4cac83507f906139d90f P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 7 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:56.003914 16147 raft_consensus.cc:2468] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 3.
I20260709 13:19:56.009558 16142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4dc6b0541ca431ebc757beac00521f1" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
W20260709 13:19:56.012048 16295 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48674 (request call id 2129) took 2823 ms (client timeout 1996 ms). Trace:
I20260709 13:19:55.747886 18675 raft_consensus.cc:515] T 8c11074579b046d99e585c2818e18a2c P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:55.963734 18613 raft_consensus.cc:515] T 8cc959fb7f704fba9009c0595467d074 P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:55.765712 18678 raft_consensus.cc:515] T 02838c86531d4f98bf58d1c22b8baf77 P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:56.056226 16819 raft_consensus.cc:2376] T 23f82224f4564fea93572e8206bad1fe P cb82cc7bd6db46d980c75237a1402860 [term 3 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 647a343a542d4a139fe106c19095129b in term 3. Re-sending same reply.
I20260709 13:19:56.058988 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18ad7e3af5504571a69d10c00750a01c" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:56.099082 16223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4dc6b0541ca431ebc757beac00521f1" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
W20260709 13:19:56.065714 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 64782.8us
I20260709 13:19:56.075660 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18ad7e3af5504571a69d10c00750a01c" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:19:56.101481 16150 raft_consensus.cc:2468] T 18ad7e3af5504571a69d10c00750a01c P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 4.
I20260709 13:19:56.007004 18532 leader_election.cc:290] T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 6 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
W20260709 13:19:56.094249 16295 rpcz_store.cc:269] 0709 13:19:53.188724 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:53.290000 (+101276us) service_pool.cc:224] Handling call
0709 13:19:56.011845 (+2721845us) spinlock_profiling.cc:243] Waited 709 ms on lock 0x7b54002cfbd4. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:19:56.011985 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1630847232}
I20260709 13:19:56.106635 16265 leader_election.cc:304] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:19:56.022680 18560 raft_consensus.cc:493] T 7c2095a5f4ad49408dbc7f69c606120d P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Starting leader election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:19:56.098574 16297 raft_consensus.cc:2393] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 5: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
I20260709 13:19:56.075659 18558 consensus_queue.cc:1243] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Peer 1ca2725d290e47d68b1a4b609c9816ac log is divergent from this leader: its last log entry 7.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260709 13:19:56.100693 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 5 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:56.110507 16225 raft_consensus.cc:2468] T ecc1e363230043fe9fef390f0b979f20 P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 4.
I20260709 13:19:56.111516 18665 raft_consensus.cc:2804] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:19:56.111752 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ad5dbfa99ee4b8d9e28be92b3f36db5" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:19:56.111995 18665 raft_consensus.cc:493] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:56.112310 16225 raft_consensus.cc:3060] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:19:56.092051 18698 log.cc:927] Time spent T e5ad4eb281564936a59f41e387a47156 P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 0.071s	user 0.000s	sys 0.011s
W20260709 13:19:56.113706 18638 log.cc:927] Time spent T 3a7ca61592e848ea9b0fb81a2ce33cdc P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 1.813s	user 0.000s	sys 0.013s
W20260709 13:19:56.106261 16116 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:56.010777 18272 raft_consensus.cc:740] T ee9ed67006144797b1de6ea8c2013a68 P 1ca2725d290e47d68b1a4b609c9816ac [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:19:56.082161 16816 raft_consensus.cc:2393] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P cb82cc7bd6db46d980c75237a1402860 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 4: Already voted for candidate cb82cc7bd6db46d980c75237a1402860 in this term.
I20260709 13:19:56.003386 18417 raft_consensus.cc:515] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } }
I20260709 13:19:56.112349 18665 raft_consensus.cc:3060] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:19:56.122525 16295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c61a39fd4f84424b129c1a6f9ba404d" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:19:56.004024 16144 raft_consensus.cc:3060] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:19:56.010057 18670 raft_consensus.cc:515] T 74ca8140f35c4ae29e8a664f6bdfc829 P cb82cc7bd6db46d980c75237a1402860 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:56.092862 18421 raft_consensus.cc:2804] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:19:56.094762 16818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57a744f5aaf944678a63469cbf514015" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:19:56.097999 18123 raft_consensus.cc:697] T 8158bbd17b994d2eb96f560bd0f1b88d P 90f3da57796b488c9087a2f523adf045 [term 6 LEADER]: Becoming Leader. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:19:56.124470 16223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a959dbecc2648eb85d1cc550f5a3245" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:56.132091 16223 raft_consensus.cc:2468] T 5a959dbecc2648eb85d1cc550f5a3245 P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 4.
I20260709 13:19:56.140249 16295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4f65cdf00b748bb963ee6e81ef43b79" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:56.110931 18560 raft_consensus.cc:3060] T 7c2095a5f4ad49408dbc7f69c606120d P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:19:56.151321 16144 raft_consensus.cc:2410] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 for term 6 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 6.
I20260709 13:19:56.152307 18587 consensus_queue.cc:260] T c0eab25ce4c64e018a041beeda2867db P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:56.154508 16816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ab0ac8347641e6955e9bce7d90c1a2" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:19:56.127357 18672 consensus_queue.cc:237] T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:56.154932 18123 consensus_queue.cc:237] T 8158bbd17b994d2eb96f560bd0f1b88d P 90f3da57796b488c9087a2f523adf045 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:55.983810 16817 rpcz_store.cc:276] 0709 13:19:53.912030 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:54.487067 (+575037us) service_pool.cc:224] Handling call
0709 13:19:54.590320 (+103253us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:55.228693 (+638373us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:55.246675 (+ 17982us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:55.303908 (+ 57233us) log.cc:844] Serialized 26 byte log entry
0709 13:19:55.565117 (+261209us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:19:55.565174 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:55.565274 (+   100us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:55.782722 (+217448us) raft_consensus.cc:1685] finished
0709 13:19:55.782747 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:55.782880 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":54293,"raft.queue_time_us":293547,"raft.run_cpu_time_us":142,"raft.run_wall_time_us":13408,"thread_start_us":660460,"threads_started":2,"wal-append.queue_time_us":302675}
I20260709 13:19:56.160092 18588 leader_election.cc:290] T 31969392ccdb43b491911a319e8bc7e1 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 5 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
W20260709 13:19:56.160372 16190 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:56.162149 18389 leader_election.cc:290] T 66f75124215f4ca6bef831e174a792d8 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:19:56.163304 18485 leader_election.cc:290] T 3884f864a96d428785895e5463e5728f P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:19:56.163852 18632 raft_consensus.cc:493] T 7c2095a5f4ad49408dbc7f69c606120d P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:56.164441 18676 raft_consensus.cc:697] T 746aabf856ab401c9ef5e859371e7b67 P 1ca2725d290e47d68b1a4b609c9816ac [term 5 LEADER]: Becoming Leader. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:19:56.166999 16265 leader_election.cc:304] T 18ad7e3af5504571a69d10c00750a01c P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:19:56.270910 18613 leader_election.cc:290] T 8cc959fb7f704fba9009c0595467d074 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:56.173949 16821 raft_consensus.cc:1240] T 74ca8140f35c4ae29e8a664f6bdfc829 P cb82cc7bd6db46d980c75237a1402860 [term 6 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 5. Current term is 6. Ops: []
W20260709 13:19:56.291613 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 124733us
I20260709 13:19:56.292925 16295 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48684 (request call id 2022) took 2330 ms. Trace:
I20260709 13:19:56.176206 18675 leader_election.cc:290] T 8c11074579b046d99e585c2818e18a2c P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:19:56.125684 16294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c74490c40243a88e62d1760b5cd814" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:19:56.298282 18689 consensus_queue.cc:1243] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Peer 90f3da57796b488c9087a2f523adf045 log is divergent from this leader: its last log entry 4.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260709 13:19:56.298488 16294 raft_consensus.cc:3060] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:19:56.178050 16292 consensus_meta.cc:287] T ada5a004ba7c46819051e3ccec71ffdb P 90f3da57796b488c9087a2f523adf045: Time spent flushing consensus metadata: real 0.591s	user 0.010s	sys 0.000s
I20260709 13:19:56.179977 16818 raft_consensus.cc:2468] T 57a744f5aaf944678a63469cbf514015 P cb82cc7bd6db46d980c75237a1402860 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 3.
W20260709 13:19:56.180215 16297 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48684 (request call id 2017) took 2247 ms (client timeout 1920 ms). Trace:
I20260709 13:19:56.126605 18421 raft_consensus.cc:493] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:56.180835 16819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5718c623ebee4d2bb770f0f9ddb2f164" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:19:56.181845 16217 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33050 (request call id 461) took 3007 ms. Trace:
I20260709 13:19:56.182709 16823 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49906 (request call id 551) took 1288 ms. Trace:
I20260709 13:19:56.307250 16823 rpcz_store.cc:276] 0709 13:19:54.894600 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:55.259353 (+364753us) service_pool.cc:224] Handling call
0709 13:19:56.182507 (+923154us) spinlock_profiling.cc:243] Waited 407 ms on lock 0x7b54007ffdd4. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:19:56.182630 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":935000448}
W20260709 13:19:56.187153 18700 log.cc:927] Time spent T fffab26427e24d398e2695aad40ca134 P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.078s	user 0.000s	sys 0.000s
I20260709 13:19:56.187247 18614 raft_consensus.cc:3055] T 1df93779209748b9860f6d893dd3de53 P 647a343a542d4a139fe106c19095129b [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:19:56.125753 18565 raft_consensus.cc:3060] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:19:56.298950 18689 raft_consensus.cc:3055] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:19:56.193336 18616 leader_election.cc:290] T 18ad7e3af5504571a69d10c00750a01c P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:19:56.193871 18617 leader_election.cc:290] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 5 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:19:56.128268 16142 raft_consensus.cc:2393] T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 5: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
W20260709 13:19:56.209128 16097 auto_rebalancer.cc:893] Removing replace marker failed: Replica 647a343a542d4a139fe106c19095129b is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 6 committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } } } pending_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
W20260709 13:19:56.213478 16114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:56.218814 18487 raft_consensus.cc:493] T af57996a8be84d398f0f023e8d08ef77 P cb82cc7bd6db46d980c75237a1402860 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:56.240969 16225 raft_consensus.cc:2468] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 5.
I20260709 13:19:56.241309 16223 raft_consensus.cc:1240] T b6f07842d3aa47858df5665874c86f3a P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 4. Current term is 5. Ops: []
I20260709 13:19:56.246157 16220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5718c623ebee4d2bb770f0f9ddb2f164" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:56.249991 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a959dbecc2648eb85d1cc550f5a3245" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:56.355984 16220 raft_consensus.cc:2393] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 5: Already voted for candidate cb82cc7bd6db46d980c75237a1402860 in this term.
I20260709 13:19:56.356051 16300 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48674 (request call id 2141) took 1623 ms. Trace:
I20260709 13:19:56.342283 18580 raft_consensus.cc:493] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:56.311661 18614 raft_consensus.cc:740] T 1df93779209748b9860f6d893dd3de53 P 647a343a542d4a139fe106c19095129b [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:19:56.356752 16300 rpcz_store.cc:276] 0709 13:19:54.732234 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:55.277145 (+544911us) service_pool.cc:224] Handling call
0709 13:19:56.249783 (+972638us) spinlock_profiling.cc:243] Waited 1.02 ms on lock 0x7b5800090098. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69ad8fadb1 00007f69ada3ab71 00007f69ada3aa22 00007f69b6953c09 00007f69b5aadee0 00007f69b5b770ab 00007f69b5b77131 00007f69b5b250a0 00007f69b5b24984 00007f69af09f6d4 00007f69af09f662
0709 13:19:56.355974 (+106191us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2343680}
I20260709 13:19:56.311751 18417 leader_election.cc:290] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:56.251250 16148 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40830 (request call id 1142) took 1788 ms. Trace:
I20260709 13:19:56.251506 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:19:56.358654 16220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e4bfb9090c8460eb14fa7e0d818e7f1" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:56.253793 18400 leader_election.cc:290] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 4 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
W20260709 13:19:56.254725 16218 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57604 (request call id 2046) took 2635 ms (client timeout 1828 ms). Trace:
I20260709 13:19:56.258266 16221 raft_consensus.cc:1240] T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac [term 6 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 5. Current term is 6. Ops: []
I20260709 13:19:56.260515 16293 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48674 (request call id 2122) took 3177 ms. Trace:
W20260709 13:19:56.269796 16791 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:56.271521 16222 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57604 (request call id 2048) took 2651 ms. Trace:
W20260709 13:19:56.276814 16789 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:56.367281 16222 rpcz_store.cc:276] 0709 13:19:53.620221 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:53.981361 (+361140us) service_pool.cc:224] Handling call
0709 13:19:53.981736 (+   375us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:55.743686 (+1761950us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:56.271142 (+527456us) spinlock_profiling.cc:243] Waited 1.67 s on lock 0x7b5400368154. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:19:56.271431 (+   289us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3837220608}
I20260709 13:19:56.368841 16222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c61a39fd4f84424b129c1a6f9ba404d" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:56.370221 16222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c11074579b046d99e585c2818e18a2c" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
W20260709 13:19:56.284409 16190 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:56.173161 18670 leader_election.cc:290] T 74ca8140f35c4ae29e8a664f6bdfc829 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 6 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
W20260709 13:19:56.291595 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 529921us
I20260709 13:19:56.164392 18591 leader_election.cc:290] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 5 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:56.272753 18463 leader_election.cc:290] T 434bff3082d141838068f323bdb0d1ed P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:19:56.295907 16144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3884f864a96d428785895e5463e5728f" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:19:56.378400 16144 raft_consensus.cc:1240] T b6f07842d3aa47858df5665874c86f3a P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 4. Current term is 5. Ops: []
I20260709 13:19:56.312196 18665 raft_consensus.cc:515] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:56.297356 16146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02838c86531d4f98bf58d1c22b8baf77" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:19:56.293556 18485 raft_consensus.cc:2804] T 18ad7e3af5504571a69d10c00750a01c P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:19:56.288552 18504 consensus_queue.cc:1243] T 19baa7bc367844c288bca8f81439c413 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Peer 647a343a542d4a139fe106c19095129b log is divergent from this leader: its last log entry 4.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
W20260709 13:19:56.302809 16297 rpcz_store.cc:269] 0709 13:19:53.933246 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:54.085507 (+152261us) service_pool.cc:224] Handling call
0709 13:19:56.179996 (+2094489us) spinlock_profiling.cc:243] Waited 1.72 s on lock 0x7b54000be5d4. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:19:56.180143 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3957850880}
I20260709 13:19:56.304402 16115 leader_election.cc:304] T e5ad4eb281564936a59f41e387a47156 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:19:56.307184 16217 rpcz_store.cc:276] 0709 13:19:53.174028 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:53.968163 (+794135us) service_pool.cc:224] Handling call
0709 13:19:54.001940 (+ 33777us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:55.657053 (+1655113us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:56.181580 (+524527us) spinlock_profiling.cc:243] Waited 1.57 s on lock 0x7b54001419d4. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:19:56.181752 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3614130944}
I20260709 13:19:56.186360 16296 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40474 (request call id 363) took 2597 ms. Trace:
W20260709 13:19:56.191710 16825 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49892 (request call id 379) took 2284 ms (client timeout 1867 ms). Trace:
I20260709 13:19:56.280695 18609 raft_consensus.cc:515] T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:19:56.304333 18629 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.988s	user 0.014s	sys 0.022s
I20260709 13:19:56.300501 18612 raft_consensus.cc:3060] T 1df93779209748b9860f6d893dd3de53 P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:19:56.306481 16292 raft_consensus.cc:2468] T ada5a004ba7c46819051e3ccec71ffdb P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 6.
I20260709 13:19:56.342114 16823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdbd78acb4cf46298e0de8ba0d130e19" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:19:56.308133 18421 raft_consensus.cc:3060] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:19:56.290894 18632 raft_consensus.cc:515] T 7c2095a5f4ad49408dbc7f69c606120d P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:19:56.353667 16097 auto_rebalancer.cc:901] Could not move replica: Illegal state: not currently leader
W20260709 13:19:56.250806 16264 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:56.341566 18674 raft_consensus.cc:493] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:56.342724 18588 raft_consensus.cc:493] T 1df93779209748b9860f6d893dd3de53 P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:56.360136 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5ad4eb281564936a59f41e387a47156" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:56.548621 16823 raft_consensus.cc:2393] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 6: Already voted for candidate cb82cc7bd6db46d980c75237a1402860 in this term.
W20260709 13:19:56.548177 16097 auto_rebalancer.cc:330] scheduled replica move failed to complete: Illegal state: not currently leader
I20260709 13:19:56.252205 16822 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49906 (request call id 547) took 1359 ms. Trace:
I20260709 13:19:56.553143 16822 rpcz_store.cc:276] 0709 13:19:54.892483 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:54.892653 (+   170us) service_pool.cc:224] Handling call
0709 13:19:54.893081 (+   428us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:55.794138 (+901057us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:56.251950 (+457812us) spinlock_profiling.cc:243] Waited 846 ms on lock 0x7b54004b5d54. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:19:56.252141 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1945032448}
I20260709 13:19:56.361490 16148 rpcz_store.cc:276] 0709 13:19:54.462449 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:54.463581 (+  1132us) service_pool.cc:224] Handling call
0709 13:19:54.463936 (+   355us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:55.770423 (+1306487us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:56.250963 (+480540us) spinlock_profiling.cc:243] Waited 1.23 s on lock 0x7b540011add4. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:19:56.251186 (+   223us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2821755648}
I20260709 13:19:56.299263 18676 consensus_queue.cc:237] T 746aabf856ab401c9ef5e859371e7b67 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:56.564263 16823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26354361e9b044cea1c2b86c4882fe92" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:19:56.359506 16818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e65676b21b748728ad39668cfe722ec" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:19:56.373703 16191 leader_election.cc:304] T ecc1e363230043fe9fef390f0b979f20 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:19:56.366288 16293 rpcz_store.cc:276] 0709 13:19:53.099247 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:53.726917 (+627670us) service_pool.cc:224] Handling call
0709 13:19:53.727305 (+   388us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:53.727547 (+   242us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:19:53.979378 (+251831us) log.cc:844] Serialized 14 byte log entry
0709 13:19:56.260085 (+2280707us) spinlock_profiling.cc:243] Waited 16.1 ms on lock 0x7b1800433180. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b693ade1 00007f69b69475b1 00007f69b07074bc 00007f69ada5548b 00007f69b68afab9 00007f69ada7169b 00007f69ada6f93f 00007f69ada6f455 00007f69b0b57060 00007f69b0b69716 00007f69b0b7a287
0709 13:19:56.260186 (+   101us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:56.260234 (+    48us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:19:56.260280 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:56.260343 (+    63us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:56.260464 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":119483,"prepare.run_cpu_time_us":36,"prepare.run_wall_time_us":37,"raft.queue_time_us":342945,"raft.run_cpu_time_us":121,"raft.run_wall_time_us":7060,"spinlock_wait_cycles":37015424,"thread_start_us":1518112,"threads_started":3,"wal-append.queue_time_us":1113863}
I20260709 13:19:56.374434 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6f07842d3aa47858df5665874c86f3a" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:19:56.357158 18487 raft_consensus.cc:3060] T af57996a8be84d398f0f023e8d08ef77 P cb82cc7bd6db46d980c75237a1402860 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:19:56.365976 16218 rpcz_store.cc:269] 0709 13:19:53.619598 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:53.672242 (+ 52644us) service_pool.cc:224] Handling call
0709 13:19:56.254477 (+2582235us) spinlock_profiling.cc:243] Waited 1.57 s on lock 0x7b54000154d4. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:19:56.254635 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3610570112}
I20260709 13:19:56.345247 18689 raft_consensus.cc:740] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:19:56.346370 16817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7ca61592e848ea9b0fb81a2ce33cdc" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:19:56.293265 16295 rpcz_store.cc:276] 0709 13:19:53.962285 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:56.176100 (+2213815us) service_pool.cc:224] Handling call
0709 13:19:56.176443 (+   343us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:56.176668 (+   225us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:19:56.176989 (+   321us) log.cc:844] Serialized 14 byte log entry
0709 13:19:56.292653 (+115664us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:56.292703 (+    50us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:19:56.292740 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:56.292783 (+    43us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:56.292866 (+    83us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":162,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":41,"raft.queue_time_us":100,"thread_start_us":115287,"threads_started":1}
I20260709 13:19:56.596737 16295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434bff3082d141838068f323bdb0d1ed" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:56.381327 16146 raft_consensus.cc:3060] T 02838c86531d4f98bf58d1c22b8baf77 P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:19:56.597354 16295 raft_consensus.cc:2468] T 434bff3082d141838068f323bdb0d1ed P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 5.
I20260709 13:19:56.372233 18684 raft_consensus.cc:493] T c0eab25ce4c64e018a041beeda2867db P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:56.386237 16223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:19:56.388037 18670 raft_consensus.cc:493] T d2ab0ac8347641e6955e9bce7d90c1a2 P cb82cc7bd6db46d980c75237a1402860 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:19:56.387416 16116 leader_election.cc:304] T b6f07842d3aa47858df5665874c86f3a P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
I20260709 13:19:56.476907 18609 leader_election.cc:290] T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 6 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:19:56.387101 18485 raft_consensus.cc:493] T 18ad7e3af5504571a69d10c00750a01c P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:19:56.607831 18485 raft_consensus.cc:3060] T 18ad7e3af5504571a69d10c00750a01c P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:19:56.483851 16825 rpcz_store.cc:269] 0709 13:19:53.907710 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:53.908342 (+   632us) service_pool.cc:224] Handling call
0709 13:19:56.191466 (+2283124us) spinlock_profiling.cc:243] Waited 846 ms on lock 0x7b54004b8cd4. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:19:56.191614 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1945138048}
I20260709 13:19:56.484550 16296 rpcz_store.cc:276] 0709 13:19:53.589166 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:54.409566 (+820400us) service_pool.cc:224] Handling call
0709 13:19:54.410015 (+   449us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:55.449326 (+1039311us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:56.186038 (+736712us) spinlock_profiling.cc:243] Waited 1.03 s on lock 0x7b5400353ed4. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:19:56.186240 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2379101056}
I20260709 13:19:56.612123 16296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e4bfb9090c8460eb14fa7e0d818e7f1" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:19:56.612686 16296 raft_consensus.cc:3060] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:19:56.374475 18400 raft_consensus.cc:493] T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:56.485968 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02838c86531d4f98bf58d1c22b8baf77" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:56.569621 18588 raft_consensus.cc:515] T 1df93779209748b9860f6d893dd3de53 P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:56.615896 16217 raft_consensus.cc:2468] T 02838c86531d4f98bf58d1c22b8baf77 P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 5.
I20260709 13:19:56.615804 18400 raft_consensus.cc:515] T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:56.380861 18662 raft_consensus.cc:493] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:56.616911 18662 raft_consensus.cc:515] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:56.486999 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87ac0d1f7e8a419f9bcbc04aee0e067e" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
W20260709 13:19:56.488937 16115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 184582us
W20260709 13:19:56.547312 16117 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:56.549212 16142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2794af5b99984efebdee564035841cb0" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
W20260709 13:19:56.547829 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 237701us
I20260709 13:19:56.555774 16148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31969392ccdb43b491911a319e8bc7e1" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:19:56.552959 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68a6838d82ef4cac83507f906139d90f" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
W20260709 13:19:56.487789 18629 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 2.211s	user 0.015s	sys 0.022s
I20260709 13:19:56.565181 16823 raft_consensus.cc:2393] T 26354361e9b044cea1c2b86c4882fe92 P cb82cc7bd6db46d980c75237a1402860 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 3: Already voted for candidate cb82cc7bd6db46d980c75237a1402860 in this term.
I20260709 13:19:57.319705 16225 raft_consensus.cc:3060] T 68a6838d82ef4cac83507f906139d90f P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Advancing to term 7
I20260709 13:19:57.320163 16297 raft_consensus.cc:2468] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 5.
W20260709 13:19:56.570581 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 196969us
I20260709 13:19:57.070683 16223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18ad7e3af5504571a69d10c00750a01c" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:57.325744 16223 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57600 (request call id 1139) took 1031 ms. Trace:
I20260709 13:19:56.989722 16146 raft_consensus.cc:2468] T 02838c86531d4f98bf58d1c22b8baf77 P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 6.
I20260709 13:19:57.326328 16191 leader_election.cc:304] T 68a6838d82ef4cac83507f906139d90f P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:19:57.326946 16146 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40830 (request call id 1159) took 1030 ms. Trace:
W20260709 13:19:56.585703 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 213798us
I20260709 13:19:57.319801 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a378d7f796f94615b3b347119a9880fd" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:57.327390 16146 rpcz_store.cc:276] 0709 13:19:56.296948 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:56.297111 (+   163us) service_pool.cc:224] Handling call
0709 13:19:57.326875 (+1029764us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:57.328446 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434bff3082d141838068f323bdb0d1ed" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:57.328702 16146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ad5dbfa99ee4b8d9e28be92b3f36db5" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:19:56.386039 18389 consensus_queue.cc:1059] T 8638ee363273426db1393dd81e5c7b72 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.182s
I20260709 13:19:57.329077 16300 raft_consensus.cc:2468] T 434bff3082d141838068f323bdb0d1ed P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 5.
I20260709 13:19:56.587684 16822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3884f864a96d428785895e5463e5728f" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
W20260709 13:19:56.382406 18640 log.cc:927] Time spent T 8d4ab870a05347868956aa8b841b2981 P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 2.008s	user 0.007s	sys 0.005s
I20260709 13:19:56.552904 18591 raft_consensus.cc:2804] T e5ad4eb281564936a59f41e387a47156 P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:19:57.117527 18665 leader_election.cc:290] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 4 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:19:56.606067 16818 raft_consensus.cc:2393] T 9e65676b21b748728ad39668cfe722ec P cb82cc7bd6db46d980c75237a1402860 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 6: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
I20260709 13:19:56.555655 16292 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48684 (request call id 2030) took 1638 ms. Trace:
I20260709 13:19:56.363925 18580 raft_consensus.cc:515] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:19:56.726651 18708 log.cc:927] Time spent T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.145s	user 0.000s	sys 0.001s
I20260709 13:19:57.327014 16225 raft_consensus.cc:2468] T 68a6838d82ef4cac83507f906139d90f P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 7.
I20260709 13:19:56.569648 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3884f864a96d428785895e5463e5728f" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:57.322619 16825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af57996a8be84d398f0f023e8d08ef77" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
W20260709 13:19:57.189457 18421 consensus_meta.cc:287] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 647a343a542d4a139fe106c19095129b: Time spent flushing consensus metadata: real 0.625s	user 0.011s	sys 0.000s
I20260709 13:19:56.987819 18457 raft_consensus.cc:493] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:57.391626 16298 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48674 (request call id 2138) took 3790 ms (client timeout 2101 ms). Trace:
W20260709 13:19:57.392211 16298 rpcz_store.cc:269] 0709 13:19:53.601223 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:53.694608 (+ 93385us) service_pool.cc:224] Handling call
0709 13:19:57.391398 (+3696790us) spinlock_profiling.cc:243] Waited 2.3 s on lock 0x7b54001eee54. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:19:57.391552 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5298191744}
I20260709 13:19:57.394556 16820 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49906 (request call id 546) took 2529 ms. Trace:
I20260709 13:19:57.371567 18421 raft_consensus.cc:515] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:57.395131 16820 rpcz_store.cc:276] 0709 13:19:54.864703 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:54.891400 (+ 26697us) service_pool.cc:224] Handling call
0709 13:19:55.126176 (+234776us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:55.127033 (+   857us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:57.394306 (+2267273us) spinlock_profiling.cc:243] Waited 234 ms on lock 0x7b54006eebd0. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5e8ad 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:19:57.394484 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":538681216}
I20260709 13:19:57.396483 16820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434bff3082d141838068f323bdb0d1ed" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:19:57.396852 18114 raft_consensus.cc:2749] T 68a6838d82ef4cac83507f906139d90f P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:19:57.397001 16820 raft_consensus.cc:2468] T 434bff3082d141838068f323bdb0d1ed P cb82cc7bd6db46d980c75237a1402860 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 5.
W20260709 13:19:56.925235 16791 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:57.397425 18421 leader_election.cc:290] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 6 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:57.397564 16820 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49896 (request call id 880) took 1210 ms. Trace:
I20260709 13:19:57.397971 16820 rpcz_store.cc:276] 0709 13:19:56.187278 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:57.396227 (+1208949us) service_pool.cc:224] Handling call
0709 13:19:57.397509 (+  1282us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:57.398042 18421 raft_consensus.cc:493] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:19:57.335188 16292 rpcz_store.cc:276] 0709 13:19:54.917220 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:55.269967 (+352747us) service_pool.cc:224] Handling call
0709 13:19:56.549990 (+1280023us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:57.322158 18662 leader_election.cc:290] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:57.399312 16292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68a6838d82ef4cac83507f906139d90f" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:19:57.399583 18629 raft_consensus.cc:3055] T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:19:57.398798 18421 raft_consensus.cc:515] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:57.400017 16292 raft_consensus.cc:3060] T 68a6838d82ef4cac83507f906139d90f P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:19:56.570263 18674 raft_consensus.cc:515] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } }
I20260709 13:19:57.329761 16822 raft_consensus.cc:2468] T 3884f864a96d428785895e5463e5728f P cb82cc7bd6db46d980c75237a1402860 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 4.
W20260709 13:19:57.400931 16822 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49896 (request call id 879) took 1393 ms (client timeout 1640 ms). Trace:
W20260709 13:19:57.401338 16822 rpcz_store.cc:269] 0709 13:19:56.007526 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:56.554507 (+546981us) service_pool.cc:224] Handling call
0709 13:19:57.400871 (+846364us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:57.402281 18674 leader_election.cc:290] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:19:57.402485 16822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e9b2bf7011f4ae688a79d7bb5a447b4" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:19:57.403112 16822 raft_consensus.cc:2468] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 3.
I20260709 13:19:57.403646 16822 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49896 (request call id 876) took 1529 ms. Trace:
I20260709 13:19:57.404008 16822 rpcz_store.cc:276] 0709 13:19:55.874596 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:57.402210 (+1527614us) service_pool.cc:224] Handling call
0709 13:19:57.403589 (+  1379us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:57.405131 16822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2372981006d4b218215c28964dea623" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:19:57.400951 17992 raft_consensus.cc:493] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:56.805523 16789 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:56.623097 16147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 5 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:19:57.406706 16147 raft_consensus.cc:3060] T ecc1e363230043fe9fef390f0b979f20 P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:19:56.991413 16296 raft_consensus.cc:2468] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 4.
I20260709 13:19:56.783613 18689 consensus_queue.cc:260] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:56.723409 16218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806d7690c2b644959152074bb2cab0a7" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:19:56.766067 16294 raft_consensus.cc:2468] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 6.
W20260709 13:19:56.840363 16192 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 290652us
I20260709 13:19:56.586683 18633 raft_consensus.cc:2804] T ecc1e363230043fe9fef390f0b979f20 P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:19:56.840950 18585 raft_consensus.cc:493] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:56.625564 16817 raft_consensus.cc:2468] T 3a7ca61592e848ea9b0fb81a2ce33cdc P cb82cc7bd6db46d980c75237a1402860 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 7.
W20260709 13:19:57.250233 16226 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.657s	user 0.002s	sys 0.015s
W20260709 13:19:57.413461 16226 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.820s	user 0.002s	sys 0.018s
I20260709 13:19:57.326051 16223 rpcz_store.cc:276] 0709 13:19:56.293985 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:57.010901 (+716916us) service_pool.cc:224] Handling call
0709 13:19:57.325687 (+314786us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:56.619655 18560 raft_consensus.cc:515] T 7c2095a5f4ad49408dbc7f69c606120d P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:57.335335 16297 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48674 (request call id 2150) took 1023 ms. Trace:
W20260709 13:19:57.406064 18485 consensus_meta.cc:287] T 18ad7e3af5504571a69d10c00750a01c P 90f3da57796b488c9087a2f523adf045: Time spent flushing consensus metadata: real 0.798s	user 0.007s	sys 0.002s
I20260709 13:19:57.345304 16265 leader_election.cc:304] T 8a878999c0024d2abe517484180290fc P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:19:57.330086 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3884f864a96d428785895e5463e5728f" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:57.348150 16225 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57604 (request call id 2072) took 1067 ms. Trace:
I20260709 13:19:56.792937 18678 leader_election.cc:290] T 02838c86531d4f98bf58d1c22b8baf77 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:56.617419 18588 leader_election.cc:290] T 1df93779209748b9860f6d893dd3de53 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:19:56.722940 18613 raft_consensus.cc:493] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:57.243691 18652 leader_election.cc:290] T c9944d5e4b8c4109af9955c4f08c9095 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:57.427403 18652 raft_consensus.cc:493] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:57.411670 18457 raft_consensus.cc:515] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:57.428188 18580 leader_election.cc:290] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
W20260709 13:19:57.344522 16823 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49896 (request call id 874) took 1481 ms (client timeout 1822 ms). Trace:
W20260709 13:19:57.428766 16823 rpcz_store.cc:269] 0709 13:19:55.863222 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:56.564023 (+700801us) service_pool.cc:224] Handling call
0709 13:19:57.344459 (+780436us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:57.429406 18457 leader_election.cc:290] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:19:57.429821 16823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2794af5b99984efebdee564035841cb0" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:19:57.430608 16823 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49892 (request call id 398) took 1432 ms. Trace:
I20260709 13:19:57.322124 18400 leader_election.cc:290] T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
W20260709 13:19:57.335477 16818 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49896 (request call id 873) took 1490 ms (client timeout 1753 ms). Trace:
I20260709 13:19:57.400059 18629 raft_consensus.cc:740] T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
W20260709 13:19:57.400393 18667 log.cc:927] Time spent T ee9ed67006144797b1de6ea8c2013a68 P cb82cc7bd6db46d980c75237a1402860: Append to log took a long time: real 2.024s	user 0.006s	sys 0.006s
I20260709 13:19:57.401521 16299 raft_consensus.cc:3060] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:19:56.694766 18681 raft_consensus.cc:2804] T b6f07842d3aa47858df5665874c86f3a P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:19:57.408008 16296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fffab26427e24d398e2695aad40ca134" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:56.626564 18684 raft_consensus.cc:515] T c0eab25ce4c64e018a041beeda2867db P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:57.393733 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c11074579b046d99e585c2818e18a2c" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:19:57.423041 18674 raft_consensus.cc:2749] T 8a878999c0024d2abe517484180290fc P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:19:57.434944 16298 raft_consensus.cc:3060] T 8c11074579b046d99e585c2818e18a2c P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:19:57.348048 16293 raft_consensus.cc:2468] T 3884f864a96d428785895e5463e5728f P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 4.
W20260709 13:19:57.412263 16294 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48684 (request call id 2034) took 1654 ms (client timeout 1534 ms). Trace:
I20260709 13:19:57.411533 17992 raft_consensus.cc:515] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:57.436501 16293 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48674 (request call id 2155) took 1090 ms. Trace:
W20260709 13:19:57.436486 16294 rpcz_store.cc:269] 0709 13:19:55.757612 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:56.095466 (+337854us) service_pool.cc:224] Handling call
0709 13:19:57.412193 (+1316727us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:57.437059 16293 rpcz_store.cc:276] 0709 13:19:56.346034 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:56.569389 (+223355us) service_pool.cc:224] Handling call
0709 13:19:57.436420 (+867031us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:57.413059 18633 raft_consensus.cc:493] T ecc1e363230043fe9fef390f0b979f20 P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:57.437866 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74ca8140f35c4ae29e8a664f6bdfc829" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:19:57.438207 18633 raft_consensus.cc:3060] T ecc1e363230043fe9fef390f0b979f20 P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:19:57.438649 16293 raft_consensus.cc:2393] T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 in current term 6: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
I20260709 13:19:57.416903 18585 raft_consensus.cc:515] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:57.419541 16297 rpcz_store.cc:276] 0709 13:19:56.311500 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:56.486765 (+175265us) service_pool.cc:224] Handling call
0709 13:19:57.335253 (+848488us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:57.439901 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdbd78acb4cf46298e0de8ba0d130e19" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:19:57.440531 16297 raft_consensus.cc:2393] T cdbd78acb4cf46298e0de8ba0d130e19 P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 in current term 6: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
I20260709 13:19:57.439085 16293 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40474 (request call id 378) took 1051 ms. Trace:
W20260709 13:19:56.756503 18705 log.cc:927] Time spent T 8cc959fb7f704fba9009c0595467d074 P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 0.150s	user 0.000s	sys 0.001s
I20260709 13:19:57.448710 18421 leader_election.cc:290] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
W20260709 13:19:57.415498 18612 consensus_meta.cc:287] T 1df93779209748b9860f6d893dd3de53 P 1ca2725d290e47d68b1a4b609c9816ac: Time spent flushing consensus metadata: real 0.868s	user 0.000s	sys 0.011s
I20260709 13:19:57.450238 18612 raft_consensus.cc:2749] T 1df93779209748b9860f6d893dd3de53 P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:19:57.427938 18652 raft_consensus.cc:515] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:57.330191 18591 raft_consensus.cc:493] T e5ad4eb281564936a59f41e387a47156 P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:57.431056 16823 rpcz_store.cc:276] 0709 13:19:55.998346 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:57.429501 (+1431155us) service_pool.cc:224] Handling call
0709 13:19:57.430551 (+  1050us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:57.433866 16296 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48674 (request call id 2146) took 1123 ms. Trace:
I20260709 13:19:57.455677 16296 rpcz_store.cc:276] 0709 13:19:56.310379 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:57.407788 (+1097409us) service_pool.cc:224] Handling call
0709 13:19:57.433792 (+ 26004us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:19:57.425138 16817 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49892 (request call id 395) took 1607 ms (client timeout 1657 ms). Trace:
I20260709 13:19:57.399071 16820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e65676b21b748728ad39668cfe722ec" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:19:56.695377 18670 raft_consensus.cc:515] T d2ab0ac8347641e6955e9bce7d90c1a2 P cb82cc7bd6db46d980c75237a1402860 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:57.441551 16297 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40474 (request call id 375) took 1073 ms. Trace:
I20260709 13:19:57.439281 18681 raft_consensus.cc:697] T b6f07842d3aa47858df5665874c86f3a P 647a343a542d4a139fe106c19095129b [term 5 LEADER]: Becoming Leader. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:19:57.443269 17992 leader_election.cc:290] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:57.443922 18629 consensus_queue.cc:260] T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:57.444928 18684 leader_election.cc:290] T c0eab25ce4c64e018a041beeda2867db P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:57.432024 18613 raft_consensus.cc:515] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:57.432204 18632 leader_election.cc:290] T 7c2095a5f4ad49408dbc7f69c606120d P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
W20260709 13:19:57.449586 18487 consensus_meta.cc:287] T af57996a8be84d398f0f023e8d08ef77 P cb82cc7bd6db46d980c75237a1402860: Time spent flushing consensus metadata: real 0.851s	user 0.009s	sys 0.002s
I20260709 13:19:57.454345 16822 raft_consensus.cc:2468] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 5.
I20260709 13:19:57.424943 18485 raft_consensus.cc:515] T 18ad7e3af5504571a69d10c00750a01c P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } }
W20260709 13:19:57.431919 16818 rpcz_store.cc:269] 0709 13:19:55.845288 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:56.342025 (+496737us) service_pool.cc:224] Handling call
0709 13:19:57.335391 (+993366us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:57.446949 18560 leader_election.cc:290] T 7c2095a5f4ad49408dbc7f69c606120d P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 5 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:57.456666 18591 raft_consensus.cc:3060] T e5ad4eb281564936a59f41e387a47156 P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:19:56.999877 16114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:56.602878 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 215524us
W20260709 13:19:56.564539 16143 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46224 (request call id 326) took 2511 ms (client timeout 1745 ms). Trace:
I20260709 13:19:57.347457 16146 raft_consensus.cc:2393] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 5: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
I20260709 13:19:57.386384 16142 raft_consensus.cc:2468] T 2794af5b99984efebdee564035841cb0 P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 4.
I20260709 13:19:57.321197 16148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66f75124215f4ca6bef831e174a792d8" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:19:57.440979 16293 rpcz_store.cc:276] 0709 13:19:56.387898 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:57.437649 (+1049751us) service_pool.cc:224] Handling call
0709 13:19:57.439034 (+  1385us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:57.437783 16294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806d7690c2b644959152074bb2cab0a7" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:19:55.797073 16224 rpcz_store.cc:276] 0709 13:19:54.379488 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:55.046366 (+666878us) service_pool.cc:224] Handling call
0709 13:19:55.046710 (+   344us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:55.046927 (+   217us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:19:55.610691 (+563764us) log.cc:844] Serialized 14 byte log entry
0709 13:19:55.758265 (+147574us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:55.758340 (+    75us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:19:55.758658 (+   318us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:55.758722 (+    64us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:55.758864 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":11126,"prepare.queue_time_us":699941,"prepare.run_cpu_time_us":42,"prepare.run_wall_time_us":1864,"raft.queue_time_us":8063,"raft.run_cpu_time_us":401,"raft.run_wall_time_us":6421,"spinlock_wait_cycles":25694976,"thread_start_us":616432,"threads_started":2,"wal-append.queue_time_us":181911}
I20260709 13:19:57.452414 16299 raft_consensus.cc:2468] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 6.
I20260709 13:19:57.417680 16218 raft_consensus.cc:3060] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:19:57.421023 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 75814.1us
I20260709 13:19:57.420907 16225 rpcz_store.cc:276] 0709 13:19:56.281101 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:56.552626 (+271525us) service_pool.cc:224] Handling call
0709 13:19:57.348070 (+795444us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:57.418694 16223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0eab25ce4c64e018a041beeda2867db" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:57.460983 16297 rpcz_store.cc:276] 0709 13:19:56.368043 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:57.439648 (+1071605us) service_pool.cc:224] Handling call
0709 13:19:57.441490 (+  1842us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:57.564481 18684 raft_consensus.cc:493] T 5a96373bb0e14dce8a266b2fc6c58507 P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:57.422593 16300 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40474 (request call id 367) took 1074 ms. Trace:
I20260709 13:19:57.475198 18272 consensus_queue.cc:260] T ee9ed67006144797b1de6ea8c2013a68 P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:57.565490 18684 raft_consensus.cc:515] T 5a96373bb0e14dce8a266b2fc6c58507 P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:57.565944 16300 rpcz_store.cc:276] 0709 13:19:56.348218 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:57.329794 (+981576us) service_pool.cc:224] Handling call
0709 13:19:57.422530 (+ 92736us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:19:57.484556 18704 log.cc:927] Time spent T 66ae176961b74813a58bc15c9c4c6f2a P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.938s	user 0.003s	sys 0.000s
I20260709 13:19:57.567664 18560 raft_consensus.cc:493] T ee9ed67006144797b1de6ea8c2013a68 P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:19:57.526320 16144 raft_consensus.cc:1240] T 7c2095a5f4ad49408dbc7f69c606120d P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 4. Current term is 5. Ops: []
I20260709 13:19:57.568899 18560 raft_consensus.cc:515] T ee9ed67006144797b1de6ea8c2013a68 P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:19:57.569737 16145 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46224 (request call id 330) took 2245 ms (client timeout 1672 ms). Trace:
W20260709 13:19:57.570413 16145 rpcz_store.cc:269] 0709 13:19:55.324724 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:55.351612 (+ 26888us) service_pool.cc:224] Handling call
0709 13:19:57.569237 (+2217625us) spinlock_profiling.cc:243] Waited 212 ms on lock 0x7b540020ba54. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:19:57.569492 (+   255us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":487533952}
I20260709 13:19:57.571079 16818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62fe3346cd3c42c4a55e14af9073bca4" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:19:57.461042 16820 raft_consensus.cc:2376] T 9e65676b21b748728ad39668cfe722ec P cb82cc7bd6db46d980c75237a1402860 [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 6. Re-sending same reply.
I20260709 13:19:57.463649 18681 consensus_queue.cc:237] T b6f07842d3aa47858df5665874c86f3a P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:57.466742 18652 leader_election.cc:290] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:19:57.572568 16818 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49906 (request call id 561) took 1084 ms. Trace:
W20260709 13:19:57.572670 16820 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49892 (request call id 394) took 1754 ms (client timeout 2059 ms). Trace:
W20260709 13:19:57.478194 18669 log.cc:927] Time spent T 8d4ab870a05347868956aa8b841b2981 P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 2.250s	user 0.006s	sys 0.006s
I20260709 13:19:57.553871 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d16dfad0c3b540e5b4b86492ddf87208" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:57.574515 16293 raft_consensus.cc:2468] T d16dfad0c3b540e5b4b86492ddf87208 P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 4.
I20260709 13:19:57.471025 18633 raft_consensus.cc:515] T ecc1e363230043fe9fef390f0b979f20 P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:57.526736 16142 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40830 (request call id 1162) took 1213 ms. Trace:
W20260709 13:19:57.573414 16820 rpcz_store.cc:269] 0709 13:19:55.817819 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:57.398855 (+1581036us) service_pool.cc:224] Handling call
0709 13:19:57.572435 (+173580us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:57.575799 16296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31969392ccdb43b491911a319e8bc7e1" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:19:57.576922 16296 raft_consensus.cc:2393] T 31969392ccdb43b491911a319e8bc7e1 P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 5: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
I20260709 13:19:57.576799 16820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b7d1ab519044eda2d5e04a07242f25" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:19:57.577375 16296 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48674 (request call id 2153) took 1236 ms. Trace:
I20260709 13:19:57.577934 16296 rpcz_store.cc:276] 0709 13:19:56.341481 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:57.456292 (+1114811us) service_pool.cc:224] Handling call
0709 13:19:57.577305 (+121013us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:57.523197 16116 leader_election.cc:304] T d16dfad0c3b540e5b4b86492ddf87208 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
I20260709 13:19:57.473570 18485 leader_election.cc:290] T 18ad7e3af5504571a69d10c00750a01c P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 5 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
W20260709 13:19:57.580338 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 57711.5us
I20260709 13:19:57.467409 18487 raft_consensus.cc:515] T af57996a8be84d398f0f023e8d08ef77 P cb82cc7bd6db46d980c75237a1402860 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:57.581712 16116 leader_election.cc:304] T 18ad7e3af5504571a69d10c00750a01c P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:19:57.447027 16298 raft_consensus.cc:2468] T 8c11074579b046d99e585c2818e18a2c P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 6.
I20260709 13:19:57.560869 17992 raft_consensus.cc:493] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:19:57.472430 18551 raft_consensus.cc:3055] T 19baa7bc367844c288bca8f81439c413 P 1ca2725d290e47d68b1a4b609c9816ac [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:19:57.562374 18389 raft_consensus.cc:493] T 23f82224f4564fea93572e8206bad1fe P cb82cc7bd6db46d980c75237a1402860 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:57.583947 18551 raft_consensus.cc:740] T 19baa7bc367844c288bca8f81439c413 P 1ca2725d290e47d68b1a4b609c9816ac [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:19:57.583806 17992 raft_consensus.cc:515] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:57.575929 16142 rpcz_store.cc:276] 0709 13:19:56.313306 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:56.380508 (+ 67202us) service_pool.cc:224] Handling call
0709 13:19:57.526509 (+1146001us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:57.584690 18655 raft_consensus.cc:2804] T d16dfad0c3b540e5b4b86492ddf87208 P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:19:57.585318 18655 raft_consensus.cc:493] T d16dfad0c3b540e5b4b86492ddf87208 P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:57.585788 18655 raft_consensus.cc:3060] T d16dfad0c3b540e5b4b86492ddf87208 P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:19:57.585852 18551 consensus_queue.cc:260] T 19baa7bc367844c288bca8f81439c413 P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.5, Last appended by leader: 5, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } }
I20260709 13:19:57.466886 18629 raft_consensus.cc:3060] T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:19:57.587658 18406 raft_consensus.cc:2749] T 18ad7e3af5504571a69d10c00750a01c P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:19:57.471549 18347 raft_consensus.cc:3055] T 26354361e9b044cea1c2b86c4882fe92 P 90f3da57796b488c9087a2f523adf045 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:19:57.467352 16823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e9b2bf7011f4ae688a79d7bb5a447b4" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:19:57.588971 18347 raft_consensus.cc:740] T 26354361e9b044cea1c2b86c4882fe92 P 90f3da57796b488c9087a2f523adf045 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:19:57.589498 16823 raft_consensus.cc:2468] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 3.
W20260709 13:19:57.590318 16823 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49892 (request call id 389) took 2019 ms (client timeout 2689 ms). Trace:
I20260709 13:19:57.477948 18662 raft_consensus.cc:3055] T 8638ee363273426db1393dd81e5c7b72 P cb82cc7bd6db46d980c75237a1402860 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:19:57.467139 18613 leader_election.cc:290] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:19:57.591794 16825 raft_consensus.cc:2368] T af57996a8be84d398f0f023e8d08ef77 P cb82cc7bd6db46d980c75237a1402860 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b for earlier term 5. Current term is 6.
I20260709 13:19:57.583410 16298 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40474 (request call id 376) took 1208 ms. Trace:
I20260709 13:19:57.526705 16146 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40830 (request call id 1166) took 1123 ms. Trace:
I20260709 13:19:57.592947 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66ae176961b74813a58bc15c9c4c6f2a" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
W20260709 13:19:57.525606 16143 rpcz_store.cc:269] 0709 13:19:54.053141 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:54.054468 (+  1327us) service_pool.cc:224] Handling call
0709 13:19:56.489930 (+2435462us) spinlock_profiling.cc:243] Waited 289 ms on lock 0x7b54000730d4. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:19:56.552149 (+ 62219us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":665788288}
I20260709 13:19:57.477164 18585 leader_election.cc:290] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:19:57.485169 18598 raft_consensus.cc:3060] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:19:57.594038 16294 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48674 (request call id 2154) took 1213 ms. Trace:
I20260709 13:19:57.595341 16294 rpcz_store.cc:276] 0709 13:19:56.345479 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:57.437524 (+1092045us) service_pool.cc:224] Handling call
0709 13:19:57.554963 (+117439us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:57.596035 16219 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57604 (request call id 2052) took 3935 ms. Trace:
I20260709 13:19:57.478937 18614 consensus_queue.cc:260] T 1df93779209748b9860f6d893dd3de53 P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:57.596899 16294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02838c86531d4f98bf58d1c22b8baf77" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:57.597468 18614 raft_consensus.cc:3060] T 1df93779209748b9860f6d893dd3de53 P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:19:57.562949 18558 raft_consensus.cc:493] T 8d4ab870a05347868956aa8b841b2981 P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:19:57.598237 16294 raft_consensus.cc:2393] T 02838c86531d4f98bf58d1c22b8baf77 P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 6: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
I20260709 13:19:57.599069 16221 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57600 (request call id 1116) took 3218 ms. Trace:
W20260709 13:19:57.460871 16817 rpcz_store.cc:269] 0709 13:19:55.824492 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:56.308216 (+483724us) service_pool.cc:224] Handling call
0709 13:19:57.423343 (+1115127us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:57.596522 16219 rpcz_store.cc:276] 0709 13:19:53.660469 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:54.018627 (+358158us) service_pool.cc:224] Handling call
0709 13:19:54.067822 (+ 49195us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:55.675430 (+1607608us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:57.595353 (+1919923us) spinlock_profiling.cc:243] Waited 1.5 s on lock 0x7b54001428d4. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:19:57.595770 (+   417us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3454385920}
I20260709 13:19:57.599766 16221 rpcz_store.cc:276] 0709 13:19:54.380308 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:55.110924 (+730616us) service_pool.cc:224] Handling call
0709 13:19:56.258033 (+1147109us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:56.376386 (+118353us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:57.598635 (+1222249us) spinlock_profiling.cc:243] Waited 1.07 s on lock 0x7b54000154d0. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5e8ad 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:19:57.598884 (+   249us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2457144576}
I20260709 13:19:57.600859 16817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66f75124215f4ca6bef831e174a792d8" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:19:57.601004 16219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02838c86531d4f98bf58d1c22b8baf77" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:19:57.601632 16221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cc959fb7f704fba9009c0595467d074" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:57.601976 16219 raft_consensus.cc:3060] T 02838c86531d4f98bf58d1c22b8baf77 P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:19:57.602730 16221 raft_consensus.cc:2468] T 8cc959fb7f704fba9009c0595467d074 P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 5.
I20260709 13:19:57.592690 16298 rpcz_store.cc:276] 0709 13:19:56.374732 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:57.393409 (+1018677us) service_pool.cc:224] Handling call
0709 13:19:57.583141 (+189732us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:19:57.590798 16823 rpcz_store.cc:269] 0709 13:19:55.571402 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:57.465354 (+1893952us) service_pool.cc:224] Handling call
0709 13:19:57.590192 (+124838us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:57.565515 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e9b2bf7011f4ae688a79d7bb5a447b4" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:57.561913 16824 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49892 (request call id 359) took 7396 ms. Trace:
I20260709 13:19:57.590831 18347 consensus_queue.cc:260] T 26354361e9b044cea1c2b86c4882fe92 P 90f3da57796b488c9087a2f523adf045 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:57.591809 18662 raft_consensus.cc:740] T 8638ee363273426db1393dd81e5c7b72 P cb82cc7bd6db46d980c75237a1402860 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:19:57.593708 16146 rpcz_store.cc:276] 0709 13:19:56.404262 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:57.328443 (+924181us) service_pool.cc:224] Handling call
0709 13:19:57.526414 (+197971us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:57.549623 16151 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40830 (request call id 1149) took 1563 ms. Trace:
I20260709 13:19:57.567878 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4dc6b0541ca431ebc757beac00521f1" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:19:57.558203 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806d7690c2b644959152074bb2cab0a7" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:57.558023 16265 leader_election.cc:304] T d16dfad0c3b540e5b4b86492ddf87208 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
W20260709 13:19:57.552410 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 225983us
I20260709 13:19:57.609360 18487 leader_election.cc:290] T af57996a8be84d398f0f023e8d08ef77 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 6 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:19:57.606690 16819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "829b4ad8c5a54eababeea1181b960d4b" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
W20260709 13:19:57.611151 16149 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40830 (request call id 1136) took 3467 ms (client timeout 2801 ms). Trace:
I20260709 13:19:57.613314 18587 raft_consensus.cc:3060] T c0eab25ce4c64e018a041beeda2867db P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:19:57.556772 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c74490c40243a88e62d1760b5cd814" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:57.596536 16143 raft_consensus.cc:1240] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 5. Current term is 6. Ops: []
I20260709 13:19:57.594241 16217 raft_consensus.cc:2468] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 7.
I20260709 13:19:57.577493 16293 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48674 (request call id 2147) took 1266 ms. Trace:
I20260709 13:19:57.578751 18633 leader_election.cc:290] T ecc1e363230043fe9fef390f0b979f20 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 5 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:19:57.614921 16820 raft_consensus.cc:3060] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:19:57.579046 16296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 5 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:19:57.590938 16144 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40830 (request call id 1161) took 1281 ms. Trace:
I20260709 13:19:57.624859 16144 rpcz_store.cc:276] 0709 13:19:56.309692 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:56.379140 (+ 69448us) service_pool.cc:224] Handling call
0709 13:19:56.379395 (+   255us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:57.569106 (+1189711us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:57.590570 (+ 21464us) spinlock_profiling.cc:243] Waited 1.15 s on lock 0x7b5400072bd4. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:19:57.590882 (+   312us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2636418432}
I20260709 13:19:57.586918 17992 leader_election.cc:290] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
W20260709 13:19:57.593887 16825 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49896 (request call id 875) took 1730 ms (client timeout 1931 ms). Trace:
I20260709 13:19:57.601797 16817 raft_consensus.cc:3060] T 66f75124215f4ca6bef831e174a792d8 P cb82cc7bd6db46d980c75237a1402860 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:19:57.485888 18673 log.cc:927] Time spent T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860: Append to log took a long time: real 2.102s	user 0.008s	sys 0.007s
W20260709 13:19:57.627799 16825 rpcz_store.cc:269] 0709 13:19:55.863505 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:56.611579 (+748074us) service_pool.cc:224] Handling call
0709 13:19:57.593635 (+982056us) spinlock_profiling.cc:243] Waited 243 ms on lock 0x7b54007c01d4. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:19:57.593813 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":558336896}
I20260709 13:19:57.623909 16293 rpcz_store.cc:276] 0709 13:19:56.310597 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:57.552975 (+1242378us) service_pool.cc:224] Handling call
0709 13:19:57.575156 (+ 22181us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:57.630455 16217 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57600 (request call id 1140) took 1330 ms. Trace:
I20260709 13:19:57.598798 18558 raft_consensus.cc:515] T 8d4ab870a05347868956aa8b841b2981 P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:57.546121 16117 leader_election.cc:304] T 434bff3082d141838068f323bdb0d1ed P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:19:57.593888 16116 leader_election.cc:304] T 5a959dbecc2648eb85d1cc550f5a3245 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 90f3da57796b488c9087a2f523adf045
I20260709 13:19:57.633775 18272 raft_consensus.cc:3060] T ee9ed67006144797b1de6ea8c2013a68 P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:19:57.635766 16300 raft_consensus.cc:2393] T e4dc6b0541ca431ebc757beac00521f1 P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 5: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
I20260709 13:19:57.636399 16300 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48674 (request call id 2149) took 1325 ms. Trace:
I20260709 13:19:57.636823 16300 rpcz_store.cc:276] 0709 13:19:56.311220 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:57.567353 (+1256133us) service_pool.cc:224] Handling call
0709 13:19:57.636324 (+ 68971us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:57.638072 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 5 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:57.639707 18347 raft_consensus.cc:3060] T 26354361e9b044cea1c2b86c4882fe92 P 90f3da57796b488c9087a2f523adf045 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:19:57.640431 16146 raft_consensus.cc:1275] T 9e65676b21b748728ad39668cfe722ec P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 13:19:57.640031 18662 consensus_queue.cc:260] T 8638ee363273426db1393dd81e5c7b72 P cb82cc7bd6db46d980c75237a1402860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:57.641142 16146 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40830 (request call id 1167) took 1169 ms. Trace:
I20260709 13:19:57.641573 16146 rpcz_store.cc:276] 0709 13:19:56.471586 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:57.639396 (+1167810us) service_pool.cc:224] Handling call
0709 13:19:57.639971 (+   575us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:57.640843 (+   872us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:57.641056 (+   213us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:57.642073 18662 raft_consensus.cc:3060] T 8638ee363273426db1393dd81e5c7b72 P cb82cc7bd6db46d980c75237a1402860 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:19:57.642710 16146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d16dfad0c3b540e5b4b86492ddf87208" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:19:57.537755 16147 raft_consensus.cc:2468] T ecc1e363230043fe9fef390f0b979f20 P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 5.
W20260709 13:19:57.486804 18710 log.cc:927] Time spent T 5a959dbecc2648eb85d1cc550f5a3245 P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 0.716s	user 0.003s	sys 0.008s
I20260709 13:19:57.643898 16297 raft_consensus.cc:2393] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 4: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
I20260709 13:19:57.617657 16824 rpcz_store.cc:276] 0709 13:19:50.164800 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:53.041532 (+2876732us) service_pool.cc:224] Handling call
0709 13:19:55.770105 (+2728573us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:55.826127 (+ 56022us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:57.478362 (+1652235us) spinlock_profiling.cc:243] Waited 2.71 s on lock 0x7b54006f2050. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5e8ad 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:19:57.561604 (+ 83242us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6230409984}
I20260709 13:19:57.613536 18684 leader_election.cc:290] T 5a96373bb0e14dce8a266b2fc6c58507 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
W20260709 13:19:57.620069 16149 rpcz_store.cc:269] 0709 13:19:54.143276 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:54.143468 (+   192us) service_pool.cc:224] Handling call
0709 13:19:57.610674 (+3467206us) spinlock_profiling.cc:243] Waited 1.63 s on lock 0x7b54003c0ad4. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:19:57.610946 (+   272us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3745823488}
I20260709 13:19:57.541708 18720 raft_consensus.cc:493] T b87147731a68400e8cce9b879daa7c5d P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:57.650643 18688 consensus_queue.cc:1059] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:19:57.651592 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87ac0d1f7e8a419f9bcbc04aee0e067e" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:19:57.651852 18688 raft_consensus.cc:3055] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:19:57.652307 18688 raft_consensus.cc:740] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:19:57.646873 16294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a878999c0024d2abe517484180290fc" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:19:57.653400 16294 raft_consensus.cc:2393] T 8a878999c0024d2abe517484180290fc P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 5: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
I20260709 13:19:57.653707 18688 consensus_queue.cc:260] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:57.649778 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6f07842d3aa47858df5665874c86f3a" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:19:57.649951 16823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66ae176961b74813a58bc15c9c4c6f2a" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:19:57.632632 16217 rpcz_store.cc:276] 0709 13:19:56.294340 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:57.592348 (+1298008us) service_pool.cc:224] Handling call
0709 13:19:57.625018 (+ 32670us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:57.654440 16294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ad5dbfa99ee4b8d9e28be92b3f36db5" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:19:57.654877 18485 raft_consensus.cc:2749] T d16dfad0c3b540e5b4b86492ddf87208 P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:19:57.655311 18585 raft_consensus.cc:493] T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:19:57.619457 16151 rpcz_store.cc:276] 0709 13:19:55.986403 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:55.986530 (+   127us) service_pool.cc:224] Handling call
0709 13:19:55.986867 (+   337us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:19:56.117053 (+130186us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:56.117083 (+    30us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:19:56.131277 (+ 14194us) log.cc:844] Serialized 52 byte log entry
0709 13:19:56.345150 (+213873us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:19:56.346590 (+  1440us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:57.404653 (+1058063us) spinlock_profiling.cc:243] Waited 123 ms on lock 0x7b540028da54. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:19:57.404697 (+    44us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:57.548928 (+144231us) raft_consensus.cc:1685] finished
0709 13:19:57.548972 (+    44us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:57.549412 (+   440us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6221,"raft.queue_time_us":6388,"raft.run_cpu_time_us":162,"raft.run_wall_time_us":10765,"spinlock_wait_cycles":283913600,"thread_start_us":167309,"threads_started":1,"wal-append.queue_time_us":312088}
I20260709 13:19:57.655953 18661 raft_consensus.cc:493] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:19:57.656570 18560 leader_election.cc:290] T ee9ed67006144797b1de6ea8c2013a68 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:19:57.659332 18689 raft_consensus.cc:3060] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:19:57.649595 16819 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49896 (request call id 872) took 1823 ms (client timeout 1777 ms). Trace:
I20260709 13:19:57.614283 18389 raft_consensus.cc:515] T 23f82224f4564fea93572e8206bad1fe P cb82cc7bd6db46d980c75237a1402860 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:19:57.682621 16819 rpcz_store.cc:269] 0709 13:19:55.825895 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:56.343564 (+517669us) service_pool.cc:224] Handling call
0709 13:19:57.606324 (+1262760us) spinlock_profiling.cc:243] Waited 9.12 ms on lock 0x7b5800004298. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69ad8fadb1 00007f69ada3ab71 00007f69ada3aeee 0000000000375939 00007f69b5aae1e0 00007f69b5b770c4 00007f69b5b77131 00007f69b5b250a0 00007f69b5b24984 00007f69af09f6d4 00007f69af09f662
0709 13:19:57.649494 (+ 43170us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":20964736}
I20260709 13:19:57.684650 16147 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46224 (request call id 338) took 1084 ms. Trace:
I20260709 13:19:57.684775 16819 raft_consensus.cc:1240] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860 [term 5 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 4. Current term is 5. Ops: []
I20260709 13:19:57.685120 16147 rpcz_store.cc:276] 0709 13:19:56.578562 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:56.622697 (+ 44135us) service_pool.cc:224] Handling call
0709 13:19:57.659960 (+1037263us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:57.686909 16150 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40830 (request call id 1158) took 1391 ms. Trace:
I20260709 13:19:57.687284 16150 rpcz_store.cc:276] 0709 13:19:56.295819 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:56.295979 (+   160us) service_pool.cc:224] Handling call
0709 13:19:56.296372 (+   393us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:19:56.296742 (+   370us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:56.296769 (+    27us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:19:56.297327 (+   558us) log.cc:844] Serialized 52 byte log entry
0709 13:19:56.595008 (+297681us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:19:56.595063 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:56.595151 (+    88us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:57.486315 (+891164us) spinlock_profiling.cc:243] Waited 2.54 ms on lock 0x7b4400022b4c. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69ad8fadb1 00007f69ad998c85 00007f69ad9996a2 00007f69b070b1db 00007f69b0706aaf 00007f69b06fa86e 00007f69b06f9bb6 00007f69b06f97d8 00007f69b0703221 00007f69b07031d9 00007f69b0703169
0709 13:19:57.686395 (+200080us) raft_consensus.cc:1685] finished
0709 13:19:57.686433 (+    38us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:57.686733 (+   300us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":4815,"raft.run_cpu_time_us":135,"raft.run_wall_time_us":4231,"spinlock_wait_cycles":5843712,"thread_start_us":249121,"threads_started":1,"wal-append.queue_time_us":383423}
I20260709 13:19:57.688723 16219 raft_consensus.cc:2468] T 02838c86531d4f98bf58d1c22b8baf77 P 1ca2725d290e47d68b1a4b609c9816ac [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 6.
I20260709 13:19:57.689282 16219 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57600 (request call id 1144) took 1391 ms. Trace:
I20260709 13:19:57.689548 16150 raft_consensus.cc:1275] T 2794af5b99984efebdee564035841cb0 P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
I20260709 13:19:57.689718 16300 raft_consensus.cc:2468] T ecc1e363230043fe9fef390f0b979f20 P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 4.
I20260709 13:19:57.690153 16150 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46224 (request call id 339) took 1066 ms. Trace:
I20260709 13:19:57.685544 16819 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49896 (request call id 868) took 2323 ms. Trace:
I20260709 13:19:57.691375 16819 rpcz_store.cc:276] 0709 13:19:55.361672 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:57.683952 (+2322280us) service_pool.cc:224] Handling call
0709 13:19:57.684475 (+   523us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:57.685259 (+   784us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:57.685469 (+   210us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:19:57.691931 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 98240.6us
I20260709 13:19:57.693429 16116 leader_election.cc:304] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
I20260709 13:19:57.694176 16819 raft_consensus.cc:1275] T c9944d5e4b8c4109af9955c4f08c9095 P cb82cc7bd6db46d980c75237a1402860 [term 6 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 13:19:57.648604 16297 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48674 (request call id 2152) took 1336 ms. Trace:
I20260709 13:19:57.695672 16297 rpcz_store.cc:276] 0709 13:19:56.311952 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:57.564850 (+1252898us) service_pool.cc:224] Handling call
0709 13:19:57.648550 (+ 83700us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:57.695557 18661 raft_consensus.cc:515] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:57.573324 16818 rpcz_store.cc:276] 0709 13:19:56.488102 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:57.570151 (+1082049us) service_pool.cc:224] Handling call
0709 13:19:57.572452 (+  2301us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:57.698561 18661 leader_election.cc:290] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:19:57.652366 16298 raft_consensus.cc:3060] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:19:57.648079 16225 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57604 (request call id 2078) took 1282 ms. Trace:
I20260709 13:19:57.650673 18720 raft_consensus.cc:515] T b87147731a68400e8cce9b879daa7c5d P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:57.699666 16225 rpcz_store.cc:276] 0709 13:19:56.364235 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:57.557600 (+1193365us) service_pool.cc:224] Handling call
0709 13:19:57.646077 (+ 88477us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:57.660571 16296 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40474 (request call id 377) took 1283 ms. Trace:
I20260709 13:19:57.700963 16296 rpcz_store.cc:276] 0709 13:19:56.376808 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:57.578672 (+1201864us) service_pool.cc:224] Handling call
0709 13:19:57.660512 (+ 81840us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:57.476557 18591 raft_consensus.cc:515] T e5ad4eb281564936a59f41e387a47156 P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:57.701803 18720 leader_election.cc:290] T b87147731a68400e8cce9b879daa7c5d P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:19:57.665762 18551 raft_consensus.cc:3060] T 19baa7bc367844c288bca8f81439c413 P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:19:57.665652 16825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 5 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:19:57.703815 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7ca61592e848ea9b0fb81a2ce33cdc" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:57.704444 16825 raft_consensus.cc:2393] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 5: Already voted for candidate cb82cc7bd6db46d980c75237a1402860 in this term.
I20260709 13:19:57.705039 16297 raft_consensus.cc:2410] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 for term 8 because replica has last-logged OpId of term: 7 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
I20260709 13:19:57.705008 16825 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49896 (request call id 870) took 1887 ms. Trace:
I20260709 13:19:57.666383 18613 raft_consensus.cc:2804] T 5a959dbecc2648eb85d1cc550f5a3245 P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:19:57.705602 16297 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40474 (request call id 371) took 1350 ms. Trace:
I20260709 13:19:57.705804 18613 raft_consensus.cc:493] T 5a959dbecc2648eb85d1cc550f5a3245 P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Starting leader election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:19:57.706018 16297 rpcz_store.cc:276] 0709 13:19:56.354844 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:57.703415 (+1348571us) service_pool.cc:224] Handling call
0709 13:19:57.705548 (+  2133us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:57.706274 18613 raft_consensus.cc:3060] T 5a959dbecc2648eb85d1cc550f5a3245 P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:19:57.666054 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c2095a5f4ad49408dbc7f69c606120d" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:19:57.712340 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87ac0d1f7e8a419f9bcbc04aee0e067e" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:57.712707 18389 leader_election.cc:290] T 23f82224f4564fea93572e8206bad1fe P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
W20260709 13:19:57.670037 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 112473us
I20260709 13:19:57.714059 16151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87ac0d1f7e8a419f9bcbc04aee0e067e" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:19:57.670832 16224 raft_consensus.cc:2393] T 08c74490c40243a88e62d1760b5cd814 P 1ca2725d290e47d68b1a4b609c9816ac [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 6: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
I20260709 13:19:57.715561 16151 raft_consensus.cc:2368] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac for earlier term 5. Current term is 6.
I20260709 13:19:57.715983 16224 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57600 (request call id 1143) took 1420 ms. Trace:
W20260709 13:19:57.716416 16299 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48674 (request call id 2139) took 2984 ms (client timeout 1871 ms). Trace:
I20260709 13:19:57.716766 16142 raft_consensus.cc:1275] T b87147731a68400e8cce9b879daa7c5d P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 13:19:57.716425 16224 rpcz_store.cc:276] 0709 13:19:56.295653 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:57.556006 (+1260353us) service_pool.cc:224] Handling call
0709 13:19:57.715898 (+159892us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:57.718662 16224 raft_consensus.cc:1240] T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac [term 6 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 5. Current term is 6. Ops: []
I20260709 13:19:57.718847 16220 raft_consensus.cc:1240] T 746aabf856ab401c9ef5e859371e7b67 P 1ca2725d290e47d68b1a4b609c9816ac [term 5 LEADER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 4. Current term is 5. Ops: []
I20260709 13:19:57.719134 16142 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46224 (request call id 333) took 1352 ms. Trace:
I20260709 13:19:57.719350 16224 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57600 (request call id 1134) took 2156 ms. Trace:
I20260709 13:19:57.719530 16142 rpcz_store.cc:276] 0709 13:19:56.366401 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:57.585343 (+1218942us) service_pool.cc:224] Handling call
0709 13:19:57.585960 (+   617us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:57.717196 (+131236us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:57.718899 (+  1703us) spinlock_profiling.cc:243] Waited 130 ms on lock 0x7b540041dcd4. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:19:57.719057 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":299951360}
I20260709 13:19:57.719830 16224 rpcz_store.cc:276] 0709 13:19:55.565778 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:57.717859 (+2152081us) service_pool.cc:224] Handling call
0709 13:19:57.718335 (+   476us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:57.719052 (+   717us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:57.719272 (+   220us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1920}
I20260709 13:19:57.721521 16294 raft_consensus.cc:2393] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 5: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
I20260709 13:19:57.722036 16294 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48674 (request call id 2145) took 1411 ms. Trace:
I20260709 13:19:57.722381 16294 rpcz_store.cc:276] 0709 13:19:56.310204 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:57.654073 (+1343869us) service_pool.cc:224] Handling call
0709 13:19:57.721982 (+ 67909us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:57.630321 16221 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57604 (request call id 2074) took 1330 ms. Trace:
I20260709 13:19:57.672935 16823 raft_consensus.cc:2393] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 7: Already voted for candidate cb82cc7bd6db46d980c75237a1402860 in this term.
I20260709 13:19:57.673183 16143 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46224 (request call id 334) took 1281 ms. Trace:
I20260709 13:19:57.676896 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af57996a8be84d398f0f023e8d08ef77" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:19:57.690210 16300 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48674 (request call id 2148) took 1379 ms. Trace:
I20260709 13:19:57.701912 18688 raft_consensus.cc:3060] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:19:57.763954 18655 raft_consensus.cc:515] T d16dfad0c3b540e5b4b86492ddf87208 P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:57.765532 16146 raft_consensus.cc:2368] T d16dfad0c3b540e5b4b86492ddf87208 P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac for earlier term 4. Current term is 5.
W20260709 13:19:57.723659 16190 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:57.767185 16190 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:57.719556 16220 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57604 (request call id 2067) took 2150 ms. Trace:
I20260709 13:19:57.768802 16220 rpcz_store.cc:276] 0709 13:19:55.568673 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:57.717880 (+2149207us) service_pool.cc:224] Handling call
0709 13:19:57.718361 (+   481us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:57.719302 (+   941us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:57.719486 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":32384}
I20260709 13:19:57.723798 16294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:19:57.771667 16294 raft_consensus.cc:2376] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 647a343a542d4a139fe106c19095129b in term 6. Re-sending same reply.
I20260709 13:19:57.772791 16294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ab0ac8347641e6955e9bce7d90c1a2" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
W20260709 13:19:57.773460 16791 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:57.725338 16144 raft_consensus.cc:1240] T 829b4ad8c5a54eababeea1181b960d4b P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 5. Current term is 6. Ops: []
I20260709 13:19:57.705463 16825 rpcz_store.cc:276] 0709 13:19:55.817535 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:57.665216 (+1847681us) service_pool.cc:224] Handling call
0709 13:19:57.704926 (+ 39710us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:57.774945 16144 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46224 (request call id 335) took 1386 ms. Trace:
I20260709 13:19:57.775398 16144 rpcz_store.cc:276] 0709 13:19:56.388856 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:57.626851 (+1237995us) service_pool.cc:224] Handling call
0709 13:19:57.646658 (+ 19807us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:57.774640 (+127982us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:57.774876 (+   236us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:57.776257 16298 raft_consensus.cc:2468] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 6.
I20260709 13:19:57.671447 18585 raft_consensus.cc:515] T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:57.776724 16144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3884f864a96d428785895e5463e5728f" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:19:57.777334 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0eab25ce4c64e018a041beeda2867db" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:57.778023 16144 raft_consensus.cc:2410] T 3884f864a96d428785895e5463e5728f P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 for term 5 because replica has last-logged OpId of term: 3 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:19:57.778386 16298 raft_consensus.cc:2410] T c0eab25ce4c64e018a041beeda2867db P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b for term 6 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:19:57.778767 18585 leader_election.cc:290] T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
W20260709 13:19:57.724685 16264 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:57.668131 18558 leader_election.cc:290] T 8d4ab870a05347868956aa8b841b2981 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:57.715553 16265 leader_election.cc:304] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:19:57.773550 16294 raft_consensus.cc:2468] T d2ab0ac8347641e6955e9bce7d90c1a2 P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 5.
W20260709 13:19:57.782192 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 67393.6us
I20260709 13:19:57.782481 16294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d4ab870a05347868956aa8b841b2981" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 6 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:57.783432 16265 leader_election.cc:304] T 68a6838d82ef4cac83507f906139d90f P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:19:57.690583 16150 rpcz_store.cc:276] 0709 13:19:56.624010 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:57.688776 (+1064766us) service_pool.cc:224] Handling call
0709 13:19:57.689111 (+   335us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:57.689919 (+   808us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:57.690087 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:57.712364 18670 leader_election.cc:290] T d2ab0ac8347641e6955e9bce7d90c1a2 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:57.722352 16224 raft_consensus.cc:1275] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:19:57.727313 16143 rpcz_store.cc:276] 0709 13:19:56.380067 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:57.595685 (+1215618us) service_pool.cc:224] Handling call
0709 13:19:57.596191 (+   506us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:57.629949 (+ 33758us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:57.661603 (+ 31654us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:57.787503 16224 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57600 (request call id 1137) took 2049 ms. Trace:
I20260709 13:19:57.728117 16823 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49892 (request call id 397) took 1903 ms. Trace:
I20260709 13:19:57.788465 16823 rpcz_store.cc:276] 0709 13:19:55.825035 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:57.649566 (+1824531us) service_pool.cc:224] Handling call
0709 13:19:57.728051 (+ 78485us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:19:57.716883 16299 rpcz_store.cc:269] 0709 13:19:54.731538 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:55.041982 (+310444us) service_pool.cc:224] Handling call
0709 13:19:56.251365 (+1209383us) spinlock_profiling.cc:243] Waited 4.34 ms on lock 0x7b5800090098. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69ad8fadb1 00007f69ada3ab71 00007f69ada3aa22 00007f69b6953c09 00007f69b5aadee0 00007f69b5b770ab 00007f69b5b77131 00007f69b5b250a0 00007f69b5b24984 00007f69af09f6d4 00007f69af09f662
0709 13:19:57.716100 (+1464735us) spinlock_profiling.cc:243] Waited 1.04 s on lock 0x7b5400399ed4. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:19:57.716239 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2401962240}
I20260709 13:19:57.728797 16116 leader_election.cc:304] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860
I20260709 13:19:57.729221 16149 raft_consensus.cc:2410] T af57996a8be84d398f0f023e8d08ef77 P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac for term 6 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:19:57.731495 18599 raft_consensus.cc:493] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:57.732172 18720 raft_consensus.cc:2804] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:19:57.734227 18591 leader_election.cc:290] T e5ad4eb281564936a59f41e387a47156 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 6 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:57.735103 16292 raft_consensus.cc:2468] T 68a6838d82ef4cac83507f906139d90f P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 7.
I20260709 13:19:57.735679 16222 raft_consensus.cc:1240] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 3. Current term is 4. Ops: []
I20260709 13:19:57.740386 16296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9944d5e4b8c4109af9955c4f08c9095" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:57.743491 16291 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48674 (request call id 2131) took 4155 ms. Trace:
I20260709 13:19:57.745137 16821 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49906 (request call id 548) took 2851 ms. Trace:
I20260709 13:19:57.726104 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66ae176961b74813a58bc15c9c4c6f2a" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:57.746173 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ad5dbfa99ee4b8d9e28be92b3f36db5" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:19:57.746654 16295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a9de347c711477e87c582d003f36925" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:57.746117 18613 raft_consensus.cc:515] T 5a959dbecc2648eb85d1cc550f5a3245 P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:57.727852 18485 raft_consensus.cc:493] T b87147731a68400e8cce9b879daa7c5d P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:57.751989 16115 leader_election.cc:336] T fffab26427e24d398e2695aad40ca134 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389): Timed out: RequestConsensusVote RPC to 127.14.177.131:41389 timed out after 1.996s (SENT)
I20260709 13:19:57.725656 16221 rpcz_store.cc:276] 0709 13:19:56.290136 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:57.601190 (+1311054us) service_pool.cc:224] Handling call
0709 13:19:57.620624 (+ 19434us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:19:57.754302 16266 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:57.758618 18724 raft_consensus.cc:493] T 26354361e9b044cea1c2b86c4882fe92 P cb82cc7bd6db46d980c75237a1402860 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:57.783913 16294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e9b2bf7011f4ae688a79d7bb5a447b4" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:57.770924 16220 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57604 (request call id 2080) took 1378 ms. Trace:
I20260709 13:19:57.821785 16294 raft_consensus.cc:2393] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 in current term 4: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
W20260709 13:19:57.821724 16115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 69972.8us
I20260709 13:19:57.787523 18655 leader_election.cc:290] T d16dfad0c3b540e5b4b86492ddf87208 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 5 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:57.714080 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806d7690c2b644959152074bb2cab0a7" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:57.689579 16219 rpcz_store.cc:276] 0709 13:19:56.297782 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:57.600560 (+1302778us) service_pool.cc:224] Handling call
0709 13:19:57.689214 (+ 88654us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:57.788035 16224 rpcz_store.cc:276] 0709 13:19:55.737906 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:57.721391 (+1983485us) service_pool.cc:224] Handling call
0709 13:19:57.721849 (+   458us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:57.787101 (+ 65252us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:57.787407 (+   306us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:57.791404 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cc959fb7f704fba9009c0595467d074" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:57.794147 16820 raft_consensus.cc:2468] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 4.
I20260709 13:19:57.821511 16220 rpcz_store.cc:276] 0709 13:19:56.392192 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:57.769712 (+1377520us) service_pool.cc:224] Handling call
0709 13:19:57.770155 (+   443us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:57.770531 (+   376us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:57.770558 (+    27us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:57.770634 (+    76us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:19:57.770671 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:57.770736 (+    65us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:57.770862 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:57.770725 16218 raft_consensus.cc:2468] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 5.
I20260709 13:19:57.802438 16142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e9b2bf7011f4ae688a79d7bb5a447b4" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:19:57.805847 16222 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57604 (request call id 2073) took 1518 ms. Trace:
W20260709 13:19:57.814570 16117 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 268840us
I20260709 13:19:57.805297 16145 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40830 (request call id 1164) took 1460 ms. Trace:
I20260709 13:19:57.727720 16300 rpcz_store.cc:276] 0709 13:19:56.310911 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:57.637692 (+1326781us) service_pool.cc:224] Handling call
0709 13:19:57.690159 (+ 52467us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:57.806600 16821 rpcz_store.cc:276] 0709 13:19:54.893617 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:55.554654 (+661037us) service_pool.cc:224] Handling call
0709 13:19:55.592832 (+ 38178us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:56.293031 (+700199us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:57.744519 (+1451488us) spinlock_profiling.cc:243] Waited 581 ms on lock 0x7b54005a0d54. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:19:57.744877 (+   358us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1335679360}
W20260709 13:19:57.825022 16791 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:57.817641 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31969392ccdb43b491911a319e8bc7e1" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 6 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:57.779060 16144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2372981006d4b218215c28964dea623" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:19:57.824455 16145 rpcz_store.cc:276] 0709 13:19:56.361986 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:57.578289 (+1216303us) service_pool.cc:224] Handling call
0709 13:19:57.578706 (+   417us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:57.724646 (+145940us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:57.724682 (+    36us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:57.724753 (+    71us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:19:57.724789 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:57.805034 (+ 80245us) spinlock_profiling.cc:243] Waited 101 ms on lock 0x7b540041e6d4. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:19:57.805086 (+    52us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:57.805239 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":232606464}
I20260709 13:19:57.824323 16222 rpcz_store.cc:276] 0709 13:19:56.287543 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:57.734522 (+1446979us) service_pool.cc:224] Handling call
0709 13:19:57.735039 (+   517us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:57.805601 (+ 70562us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:57.805793 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:57.829006 16298 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48674 (request call id 2157) took 1481 ms. Trace:
I20260709 13:19:57.829638 16298 rpcz_store.cc:276] 0709 13:19:56.347531 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:57.779007 (+1431476us) service_pool.cc:224] Handling call
0709 13:19:57.828937 (+ 49930us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:57.804889 18347 raft_consensus.cc:2749] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:19:57.831202 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5d59407eb545b895d8d922e0dd69fd" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 7 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:19:57.816921 16819 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49892 (request call id 388) took 2229 ms. Trace:
I20260709 13:19:57.832962 16295 raft_consensus.cc:2468] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 6.
I20260709 13:19:57.839529 16148 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40830 (request call id 1163) took 1457 ms. Trace:
I20260709 13:19:57.839988 16148 rpcz_store.cc:276] 0709 13:19:56.342535 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:57.539252 (+1196717us) service_pool.cc:224] Handling call
0709 13:19:57.539939 (+   687us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:57.540552 (+   613us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:19:57.547758 (+  7206us) log.cc:844] Serialized 14 byte log entry
0709 13:19:57.727491 (+179733us) log.cc:844] Serialized 14 byte log entry
0709 13:19:57.790162 (+ 62671us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:57.790227 (+    65us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:19:57.790276 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:57.790355 (+    79us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:57.799972 (+  9617us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":88986,"prepare.queue_time_us":7095,"prepare.run_cpu_time_us":101,"prepare.run_wall_time_us":105,"raft.queue_time_us":167139,"spinlock_wait_cycles":6337408,"thread_start_us":89157,"threads_started":2,"wal-append.queue_time_us":77663}
I20260709 13:19:57.841341 16295 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48674 (request call id 2142) took 3108 ms. Trace:
W20260709 13:19:57.842855 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 114223us
I20260709 13:19:57.843127 18114 raft_consensus.cc:493] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:57.843787 18114 raft_consensus.cc:515] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } }
I20260709 13:19:57.844650 16116 leader_election.cc:304] T 9c61a39fd4f84424b129c1a6f9ba404d P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:19:57.805863 16817 raft_consensus.cc:2468] T 66f75124215f4ca6bef831e174a792d8 P cb82cc7bd6db46d980c75237a1402860 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 6.
I20260709 13:19:57.845997 16818 raft_consensus.cc:1240] T 66f75124215f4ca6bef831e174a792d8 P cb82cc7bd6db46d980c75237a1402860 [term 6 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 5. Current term is 6. Ops: [5.6-5.6]
I20260709 13:19:57.847384 16823 raft_consensus.cc:1240] T ee9ed67006144797b1de6ea8c2013a68 P cb82cc7bd6db46d980c75237a1402860 [term 5 LEADER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 13:19:57.848287 16823 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49906 (request call id 556) took 1470 ms. Trace:
I20260709 13:19:57.848671 16823 rpcz_store.cc:276] 0709 13:19:56.377762 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:57.805243 (+1427481us) service_pool.cc:224] Handling call
0709 13:19:57.846936 (+ 41693us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:57.847966 (+  1030us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:57.848173 (+   207us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:57.849861 16823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 5 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:19:57.850750 16823 raft_consensus.cc:2393] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 5: Already voted for candidate cb82cc7bd6db46d980c75237a1402860 in this term.
I20260709 13:19:57.851256 16217 raft_consensus.cc:2393] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 5: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
I20260709 13:19:57.851491 18674 raft_consensus.cc:2764] T 68a6838d82ef4cac83507f906139d90f P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Leader pre-election decision vote started in defunct term 6: won
I20260709 13:19:57.851799 16217 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57600 (request call id 1146) took 1547 ms. Trace:
I20260709 13:19:57.847275 16817 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49906 (request call id 562) took 1299 ms. Trace:
I20260709 13:19:57.852200 16217 rpcz_store.cc:276] 0709 13:19:56.304135 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:57.680164 (+1376029us) service_pool.cc:224] Handling call
0709 13:19:57.851737 (+171573us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:57.845649 18485 raft_consensus.cc:515] T b87147731a68400e8cce9b879daa7c5d P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:57.853261 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87147731a68400e8cce9b879daa7c5d" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 8 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:58.289659 18114 leader_election.cc:290] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
W20260709 13:19:57.932358 16190 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:57.853611 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a96373bb0e14dce8a266b2fc6c58507" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:58.291507 16300 raft_consensus.cc:2410] T 5a96373bb0e14dce8a266b2fc6c58507 P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b for term 5 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:19:57.830890 18684 raft_consensus.cc:2749] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:19:58.292706 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a378d7f796f94615b3b347119a9880fd" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:57.942622 16117 leader_election.cc:304] T 3884f864a96d428785895e5463e5728f P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:19:58.294128 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c74490c40243a88e62d1760b5cd814" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
W20260709 13:19:58.294188 16117 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 456223us
I20260709 13:19:57.842039 16295 rpcz_store.cc:276] 0709 13:19:54.732519 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:57.839754 (+3107235us) service_pool.cc:224] Handling call
0709 13:19:57.840346 (+   592us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:57.840824 (+   478us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:57.840869 (+    45us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:57.840941 (+    72us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:19:57.840993 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:57.841080 (+    87us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:57.841237 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:58.295396 16117 leader_election.cc:304] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:19:58.296361 18558 raft_consensus.cc:2804] T 3884f864a96d428785895e5463e5728f P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 13:19:58.296458 16115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 333668us
I20260709 13:19:58.296912 18558 raft_consensus.cc:493] T 3884f864a96d428785895e5463e5728f P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:57.959618 18662 raft_consensus.cc:493] T 9e65676b21b748728ad39668cfe722ec P cb82cc7bd6db46d980c75237a1402860 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:58.297567 18558 raft_consensus.cc:3060] T 3884f864a96d428785895e5463e5728f P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:19:58.297632 16220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7ca61592e848ea9b0fb81a2ce33cdc" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:58.298079 18662 raft_consensus.cc:515] T 9e65676b21b748728ad39668cfe722ec P cb82cc7bd6db46d980c75237a1402860 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:58.299597 16220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a378d7f796f94615b3b347119a9880fd" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:58.300038 16115 leader_election.cc:304] T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:19:58.300484 16220 raft_consensus.cc:2410] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 for term 5 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 5.
I20260709 13:19:57.878799 16299 raft_consensus.cc:2410] T 31969392ccdb43b491911a319e8bc7e1 P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 for term 6 because replica has last-logged OpId of term: 3 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
W20260709 13:19:57.953765 16822 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49906 (request call id 557) took 1501 ms (client timeout 1686 ms). Trace:
I20260709 13:19:57.956596 16823 raft_consensus.cc:1275] T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860 [term 6 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
W20260709 13:19:58.081519 16820 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49906 (request call id 563) took 1532 ms (client timeout 2017 ms). Trace:
I20260709 13:19:58.185241 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af57996a8be84d398f0f023e8d08ef77" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:57.874025 16298 raft_consensus.cc:1240] T cdbd78acb4cf46298e0de8ba0d130e19 P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 5. Current term is 6. Ops: []
W20260709 13:19:57.931977 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 87524us
I20260709 13:19:58.064364 16266 leader_election.cc:304] T d2ab0ac8347641e6955e9bce7d90c1a2 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860
W20260709 13:19:57.932404 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 101358us
W20260709 13:19:58.311839 16820 rpcz_store.cc:269] 0709 13:19:56.548767 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:57.576319 (+1027552us) service_pool.cc:224] Handling call
0709 13:19:58.081262 (+504943us) spinlock_profiling.cc:243] Waited 37.5 ms on lock 0x7b54005032d4. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:19:58.081433 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":86320896}
I20260709 13:19:57.953426 16296 raft_consensus.cc:1240] T 18ad7e3af5504571a69d10c00750a01c P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 4. Current term is 5. Ops: [4.4-4.4]
I20260709 13:19:58.285504 16294 raft_consensus.cc:1240] T 8a878999c0024d2abe517484180290fc P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 13:19:58.296361 16295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c61a39fd4f84424b129c1a6f9ba404d" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:19:57.875243 16225 raft_consensus.cc:1240] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 4. Current term is 5. Ops: []
I20260709 13:19:57.877518 16821 raft_consensus.cc:1240] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P cb82cc7bd6db46d980c75237a1402860 [term 4 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 3. Current term is 4. Ops: []
W20260709 13:19:58.041478 16789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 54872.8us
I20260709 13:19:57.960516 16150 raft_consensus.cc:2410] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 for term 7 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 6.
W20260709 13:19:58.083066 16189 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:57.875491 16142 raft_consensus.cc:2468] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 3.
W20260709 13:19:57.830332 16292 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48674 (request call id 2156) took 1475 ms (client timeout 1951 ms). Trace:
W20260709 13:19:58.311146 16822 rpcz_store.cc:269] 0709 13:19:56.378102 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:57.404817 (+1026715us) service_pool.cc:224] Handling call
0709 13:19:57.824183 (+419366us) spinlock_profiling.cc:243] Waited 24.3 ms on lock 0x7b54006239d4. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:19:57.879138 (+ 54955us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":55785344}
I20260709 13:19:58.317809 16298 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48684 (request call id 2024) took 4346 ms. Trace:
I20260709 13:19:58.318333 16298 rpcz_store.cc:276] 0709 13:19:53.967074 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:57.832484 (+3865410us) service_pool.cc:224] Handling call
0709 13:19:57.873611 (+ 41127us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:58.313140 (+439529us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:58.313346 (+   206us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:19:57.873090 16218 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57604 (request call id 2071) took 1565 ms (client timeout 1965 ms). Trace:
W20260709 13:19:58.290957 16790 connection.cc:256] RPC call timeout handler was delayed by -1.1177507180002522s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16720000000000002s for a response
I20260709 13:19:58.311895 16823 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49906 (request call id 560) took 1841 ms. Trace:
I20260709 13:19:58.322184 16823 rpcz_store.cc:276] 0709 13:19:56.471077 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:57.955602 (+1484525us) service_pool.cc:224] Handling call
0709 13:19:57.956103 (+   501us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:58.311559 (+355456us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:58.311817 (+   258us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:57.961318 16297 raft_consensus.cc:2468] T 66ae176961b74813a58bc15c9c4c6f2a P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 6.
W20260709 13:19:58.320335 16218 rpcz_store.cc:269] 0709 13:19:56.273039 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:56.694443 (+421404us) service_pool.cc:224] Handling call
0709 13:19:57.833518 (+1139075us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:57.853765 16145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31969392ccdb43b491911a319e8bc7e1" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 6 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:19:58.326462 16821 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49906 (request call id 559) took 1939 ms. Trace:
I20260709 13:19:58.326913 16821 rpcz_store.cc:276] 0709 13:19:56.387024 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:57.876772 (+1489748us) service_pool.cc:224] Handling call
0709 13:19:57.877218 (+   446us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:58.326201 (+448983us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:58.326407 (+   206us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:58.309347 16819 rpcz_store.cc:276] 0709 13:19:55.570998 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:57.692772 (+2121774us) service_pool.cc:224] Handling call
0709 13:19:57.693487 (+   715us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:57.694805 (+  1318us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:57.799954 (+105149us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:58.328508 16226 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57600 (request call id 1130) took 2873 ms. Trace:
I20260709 13:19:58.330629 16820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a378d7f796f94615b3b347119a9880fd" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
W20260709 13:19:58.189659 18587 consensus_meta.cc:287] T c0eab25ce4c64e018a041beeda2867db P 1ca2725d290e47d68b1a4b609c9816ac: Time spent flushing consensus metadata: real 0.546s	user 0.008s	sys 0.004s
I20260709 13:19:58.331518 16820 raft_consensus.cc:2468] T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 4.
I20260709 13:19:58.332293 16223 raft_consensus.cc:2468] T c0eab25ce4c64e018a041beeda2867db P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 5.
I20260709 13:19:58.333024 18633 raft_consensus.cc:493] T c0eab25ce4c64e018a041beeda2867db P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:58.333683 18633 raft_consensus.cc:515] T c0eab25ce4c64e018a041beeda2867db P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:58.334482 18485 leader_election.cc:290] T b87147731a68400e8cce9b879daa7c5d P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:19:57.960831 18421 consensus_queue.cc:1059] T 08c74490c40243a88e62d1760b5cd814 P 647a343a542d4a139fe106c19095129b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.375s
I20260709 13:19:58.003804 18681 raft_consensus.cc:2749] T 9c61a39fd4f84424b129c1a6f9ba404d P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:19:57.961112 18347 raft_consensus.cc:493] T e5ad4eb281564936a59f41e387a47156 P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:58.336361 18114 raft_consensus.cc:493] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:19:58.329098 16226 rpcz_store.cc:276] 0709 13:19:55.454957 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:56.261959 (+807002us) service_pool.cc:224] Handling call
0709 13:19:56.262302 (+   343us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:56.592428 (+330126us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:56.592473 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:56.593102 (+   629us) log.cc:844] Serialized 26 byte log entry
0709 13:19:57.413895 (+820793us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:19:57.413938 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:57.833743 (+419805us) spinlock_profiling.cc:243] Waited 330 ms on lock 0x7b54002c8154. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:19:57.837634 (+  3891us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:57.942662 (+105028us) raft_consensus.cc:1685] finished
0709 13:19:57.942691 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:58.328258 (+385567us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1553,"raft.run_cpu_time_us":118,"raft.run_wall_time_us":119,"spinlock_wait_cycles":757974272,"thread_start_us":820255,"threads_started":1,"wal-append.queue_time_us":793321}
I20260709 13:19:58.323038 16294 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48684 (request call id 2023) took 4348 ms. Trace:
I20260709 13:19:58.325671 16296 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48684 (request call id 2025) took 4338 ms. Trace:
I20260709 13:19:57.828207 18585 raft_consensus.cc:2804] T 434bff3082d141838068f323bdb0d1ed P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:19:58.364560 16297 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40474 (request call id 374) took 1968 ms. Trace:
I20260709 13:19:58.365031 16297 rpcz_store.cc:276] 0709 13:19:56.356978 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:57.706833 (+1349855us) service_pool.cc:224] Handling call
0709 13:19:58.325658 (+618825us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:58.347148 16293 raft_consensus.cc:2410] T b87147731a68400e8cce9b879daa7c5d P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b for term 8 because replica has last-logged OpId of term: 6 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 5.
I20260709 13:19:57.959746 18580 consensus_queue.cc:1048] T 8cc959fb7f704fba9009c0595467d074 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.376s
I20260709 13:19:58.367905 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4f65cdf00b748bb963ee6e81ef43b79" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:57.834690 18535 raft_consensus.cc:493] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:58.369670 16791 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:58.296895 16117 leader_election.cc:304] T 26354361e9b044cea1c2b86c4882fe92 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860
I20260709 13:19:57.877830 16144 raft_consensus.cc:2468] T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 5.
I20260709 13:19:57.876493 16816 raft_consensus.cc:1240] T 8c11074579b046d99e585c2818e18a2c P cb82cc7bd6db46d980c75237a1402860 [term 6 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 5. Current term is 6. Ops: []
I20260709 13:19:58.300639 18662 leader_election.cc:290] T 9e65676b21b748728ad39668cfe722ec P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:58.073645 16818 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49896 (request call id 877) took 2198 ms. Trace:
I20260709 13:19:58.376359 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ad5dbfa99ee4b8d9e28be92b3f36db5" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:57.835615 18724 raft_consensus.cc:515] T 26354361e9b044cea1c2b86c4882fe92 P cb82cc7bd6db46d980c75237a1402860 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:58.336865 18347 raft_consensus.cc:515] T e5ad4eb281564936a59f41e387a47156 P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:58.374771 16147 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46224 (request call id 336) took 1813 ms. Trace:
W20260709 13:19:58.336378 16192 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.14.177.132:38923 timed out after 1.686s (SENT) [suppressed 42 similar messages]
I20260709 13:19:58.383010 16147 rpcz_store.cc:276] 0709 13:19:56.551892 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:57.685982 (+1134090us) service_pool.cc:224] Handling call
0709 13:19:57.686455 (+   473us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:57.686866 (+   411us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:57.686897 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:57.783494 (+ 96597us) log.cc:844] Serialized 26 byte log entry
0709 13:19:58.012993 (+229499us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:19:58.013072 (+    79us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:58.013159 (+    87us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:58.365473 (+352314us) raft_consensus.cc:1685] finished
0709 13:19:58.365492 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:58.365636 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":228846,"threads_started":1,"wal-append.queue_time_us":517697}
I20260709 13:19:58.372483 16224 raft_consensus.cc:2468] T af57996a8be84d398f0f023e8d08ef77 P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 5.
I20260709 13:19:58.364928 18585 raft_consensus.cc:493] T 434bff3082d141838068f323bdb0d1ed P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:58.373409 16146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746aabf856ab401c9ef5e859371e7b67" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:19:57.835089 18720 raft_consensus.cc:697] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b [term 5 LEADER]: Becoming Leader. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
W20260709 13:19:58.376657 16190 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:58.387130 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d4ab870a05347868956aa8b841b2981" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 6 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:19:58.367704 16294 rpcz_store.cc:276] 0709 13:19:53.965535 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:57.858271 (+3892736us) service_pool.cc:224] Handling call
0709 13:19:58.285186 (+426915us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:58.313278 (+ 28092us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:58.313543 (+   265us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:58.366614 16296 rpcz_store.cc:276] 0709 13:19:53.976246 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:57.837795 (+3861549us) service_pool.cc:224] Handling call
0709 13:19:57.952897 (+115102us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:58.314143 (+361246us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:58.314429 (+   286us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:19:58.317603 16292 rpcz_store.cc:269] 0709 13:19:56.347139 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:57.399022 (+1051883us) service_pool.cc:224] Handling call
0709 13:19:57.817143 (+418121us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:58.255736 15046 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:19:58.325353 18684 raft_consensus.cc:2804] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:19:58.364108 18114 raft_consensus.cc:515] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:58.390154 16300 raft_consensus.cc:2376] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 6. Re-sending same reply.
I20260709 13:19:57.828537 18599 raft_consensus.cc:515] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:58.389344 16299 raft_consensus.cc:2393] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 5: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
I20260709 13:19:58.395251 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c2095a5f4ad49408dbc7f69c606120d" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:58.389801 16295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada5a004ba7c46819051e3ccec71ffdb" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:19:58.319531 18613 leader_election.cc:290] T 5a959dbecc2648eb85d1cc550f5a3245 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 5 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:58.388132 18585 raft_consensus.cc:3060] T 434bff3082d141838068f323bdb0d1ed P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:19:58.393164 16264 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:57.806375 16291 rpcz_store.cc:276] 0709 13:19:53.587482 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:54.143257 (+555775us) service_pool.cc:224] Handling call
0709 13:19:54.311957 (+168700us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:55.595332 (+1283375us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:57.604060 (+2008728us) spinlock_profiling.cc:243] Waited 5.3 ms on lock 0x7b5400306720. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0ad6dc0 00007f69b0b62a02 00007f69b0b5edd6 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2
0709 13:19:57.742861 (+138801us) spinlock_profiling.cc:243] Waited 1.08 s on lock 0x7b54002e1654. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:19:57.743223 (+   362us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2494480384}
I20260709 13:19:58.397668 18580 raft_consensus.cc:2749] T d2ab0ac8347641e6955e9bce7d90c1a2 P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:19:58.398208 16292 raft_consensus.cc:1240] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 3. Current term is 4. Ops: []
I20260709 13:19:58.398968 18347 leader_election.cc:290] T e5ad4eb281564936a59f41e387a47156 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:19:58.396814 16294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 5 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:19:58.313653 16817 rpcz_store.cc:276] 0709 13:19:56.548399 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:57.600285 (+1051886us) service_pool.cc:224] Handling call
0709 13:19:57.847181 (+246896us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:58.405261 18724 leader_election.cc:290] T 26354361e9b044cea1c2b86c4882fe92 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:58.407552 18114 leader_election.cc:290] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:19:58.407279 18535 raft_consensus.cc:515] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:19:58.410399 16115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 110440us
I20260709 13:19:58.411123 16116 leader_election.cc:304] T 02838c86531d4f98bf58d1c22b8baf77 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 90f3da57796b488c9087a2f523adf045
I20260709 13:19:58.399829 18560 raft_consensus.cc:2749] T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:19:58.409879 18421 raft_consensus.cc:3060] T 26354361e9b044cea1c2b86c4882fe92 P 647a343a542d4a139fe106c19095129b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:19:58.413044 18733 consensus_queue.cc:1048] T c9944d5e4b8c4109af9955c4f08c9095 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.022s
I20260709 13:19:58.382938 16818 rpcz_store.cc:276] 0709 13:19:55.874921 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:57.697094 (+1822173us) service_pool.cc:224] Handling call
0709 13:19:57.845521 (+148427us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:57.846597 (+  1076us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:58.073258 (+226661us) spinlock_profiling.cc:243] Waited 148 ms on lock 0x7b54005302d0. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5e8ad 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:19:58.073521 (+   263us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":340030208}
W20260709 13:19:58.415700 16266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 351450us
I20260709 13:19:58.415711 18720 consensus_queue.cc:237] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:58.400475 16816 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49906 (request call id 558) took 2017 ms. Trace:
I20260709 13:19:58.417271 16295 raft_consensus.cc:2376] T ada5a004ba7c46819051e3ccec71ffdb P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 6. Re-sending same reply.
I20260709 13:19:58.419185 16294 raft_consensus.cc:3060] T ecc1e363230043fe9fef390f0b979f20 P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:19:58.419564 16297 raft_consensus.cc:1240] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 5. Current term is 6. Ops: [5.5-5.5]
I20260709 13:19:58.424120 16297 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48684 (request call id 2028) took 3507 ms. Trace:
I20260709 13:19:58.424520 16297 rpcz_store.cc:276] 0709 13:19:54.916562 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:58.393500 (+3476938us) service_pool.cc:224] Handling call
0709 13:19:58.411648 (+ 18148us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:58.423852 (+ 12204us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:58.424064 (+   212us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:58.425160 18535 leader_election.cc:290] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:19:58.420262 16265 leader_election.cc:304] T 5a96373bb0e14dce8a266b2fc6c58507 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:19:58.420279 16116 leader_election.cc:304] T 68a6838d82ef4cac83507f906139d90f P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
W20260709 13:19:58.421864 16263 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:58.428579 16265 leader_election.cc:304] T 84f2da98864b4425bf7c0055951fdd4b P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:19:58.429036 16790 leader_election.cc:304] T 434bff3082d141838068f323bdb0d1ed P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:19:58.421902 18558 raft_consensus.cc:515] T 3884f864a96d428785895e5463e5728f P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:58.418334 16292 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48684 (request call id 2032) took 3500 ms. Trace:
I20260709 13:19:58.429692 16292 rpcz_store.cc:276] 0709 13:19:54.917773 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:58.397600 (+3479827us) service_pool.cc:224] Handling call
0709 13:19:58.397987 (+   387us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:58.412287 (+ 14300us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:58.417099 (+  4812us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:58.431738 16300 raft_consensus.cc:1240] T e4dc6b0541ca431ebc757beac00521f1 P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 13:19:58.431435 16292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d16dfad0c3b540e5b4b86492ddf87208" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:19:58.431238 16266 leader_election.cc:304] T 57a744f5aaf944678a63469cbf514015 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 647a343a542d4a139fe106c19095129b
I20260709 13:19:58.432654 16292 raft_consensus.cc:3060] T d16dfad0c3b540e5b4b86492ddf87208 P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:19:58.431131 18485 raft_consensus.cc:2749] T 5a96373bb0e14dce8a266b2fc6c58507 P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:19:58.433283 18734 raft_consensus.cc:2749] T 84f2da98864b4425bf7c0055951fdd4b P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:19:58.435271 18485 raft_consensus.cc:2804] T 57a744f5aaf944678a63469cbf514015 P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:19:58.435722 18485 raft_consensus.cc:493] T 57a744f5aaf944678a63469cbf514015 P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 647a343a542d4a139fe106c19095129b)
W20260709 13:19:58.435961 16117 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 139243us
I20260709 13:19:58.436691 16266 leader_election.cc:304] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:19:58.436951 16117 leader_election.cc:304] T 9e65676b21b748728ad39668cfe722ec P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860
I20260709 13:19:58.436175 18485 raft_consensus.cc:3060] T 57a744f5aaf944678a63469cbf514015 P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:19:58.434303 16821 raft_consensus.cc:1275] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860 [term 5 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:19:58.421880 16818 raft_consensus.cc:3055] T 829b4ad8c5a54eababeea1181b960d4b P cb82cc7bd6db46d980c75237a1402860 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:19:58.427006 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5ad4eb281564936a59f41e387a47156" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:19:58.448529 16297 raft_consensus.cc:3060] T e5ad4eb281564936a59f41e387a47156 P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:19:58.429702 18347 consensus_queue.cc:1048] T 5a959dbecc2648eb85d1cc550f5a3245 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.018s
I20260709 13:19:58.447916 16818 raft_consensus.cc:740] T 829b4ad8c5a54eababeea1181b960d4b P cb82cc7bd6db46d980c75237a1402860 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:19:58.440269 18724 consensus_queue.cc:1059] T ada5a004ba7c46819051e3ccec71ffdb P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.012s
I20260709 13:19:58.454634 16300 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48684 (request call id 2027) took 3541 ms. Trace:
I20260709 13:19:58.455232 16300 rpcz_store.cc:276] 0709 13:19:54.909104 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:58.392802 (+3483698us) service_pool.cc:224] Handling call
0709 13:19:58.417899 (+ 25097us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:58.449911 (+ 32012us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:58.450197 (+   286us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:58.453508 16115 leader_election.cc:304] T 8a878999c0024d2abe517484180290fc P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:19:58.455667 16292 raft_consensus.cc:2468] T d16dfad0c3b540e5b4b86492ddf87208 P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 5.
I20260709 13:19:58.459657 18535 raft_consensus.cc:493] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:58.463549 18535 raft_consensus.cc:515] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:19:58.465600 16223 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57600 (request call id 1142) took 2170 ms (client timeout 2027 ms). Trace:
W20260709 13:19:58.466028 16223 rpcz_store.cc:269] 0709 13:19:56.295297 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:57.414305 (+1119008us) service_pool.cc:224] Handling call
0709 13:19:58.465370 (+1051065us) spinlock_profiling.cc:243] Waited 772 ms on lock 0x7b54002981d4. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:19:58.465527 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1776317312}
I20260709 13:19:58.467845 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f82224f4564fea93572e8206bad1fe" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:58.468569 16300 raft_consensus.cc:2468] T 23f82224f4564fea93572e8206bad1fe P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 3.
I20260709 13:19:58.469928 18347 raft_consensus.cc:1064] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045: attempting to promote NON_VOTER cb82cc7bd6db46d980c75237a1402860 to VOTER
I20260709 13:19:58.422406 16816 rpcz_store.cc:276] 0709 13:19:56.378411 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:57.875529 (+1497118us) service_pool.cc:224] Handling call
0709 13:19:57.876096 (+   567us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:58.382116 (+506020us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:58.396017 (+ 13901us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:58.477566 16298 raft_consensus.cc:1275] T b87147731a68400e8cce9b879daa7c5d P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 13:19:58.477946 16295 raft_consensus.cc:1240] T ada5a004ba7c46819051e3ccec71ffdb P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 5. Current term is 6. Ops: []
I20260709 13:19:58.478660 16295 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40474 (request call id 372) took 2123 ms. Trace:
I20260709 13:19:58.479027 16295 rpcz_store.cc:276] 0709 13:19:56.355223 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:58.423517 (+2068294us) service_pool.cc:224] Handling call
0709 13:19:58.473672 (+ 50155us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:58.478406 (+  4734us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:58.478602 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:19:58.480273 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 51917.2us
I20260709 13:19:58.482545 16299 raft_consensus.cc:1240] T 1df93779209748b9860f6d893dd3de53 P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 4. Current term is 5. Ops: [4.4-4.4]
I20260709 13:19:58.448055 18613 raft_consensus.cc:2764] T 02838c86531d4f98bf58d1c22b8baf77 P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Leader pre-election decision vote started in defunct term 5: won
I20260709 13:19:58.486004 16292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4f65cdf00b748bb963ee6e81ef43b79" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:58.487893 18724 raft_consensus.cc:2804] T 434bff3082d141838068f323bdb0d1ed P cb82cc7bd6db46d980c75237a1402860 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:19:58.490406 18724 raft_consensus.cc:493] T 434bff3082d141838068f323bdb0d1ed P cb82cc7bd6db46d980c75237a1402860 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:58.490669 16265 leader_election.cc:304] T cdbd78acb4cf46298e0de8ba0d130e19 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860
I20260709 13:19:58.492107 16265 leader_election.cc:304] T ada5a004ba7c46819051e3ccec71ffdb P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860
I20260709 13:19:58.494065 16265 leader_election.cc:304] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860
I20260709 13:19:58.495561 18580 raft_consensus.cc:2749] T cdbd78acb4cf46298e0de8ba0d130e19 P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:19:58.496279 18580 raft_consensus.cc:2749] T ada5a004ba7c46819051e3ccec71ffdb P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:19:58.496464 16225 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57600 (request call id 1133) took 2945 ms. Trace:
I20260709 13:19:58.460456 18560 raft_consensus.cc:2749] T 9e65676b21b748728ad39668cfe722ec P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:19:58.456826 18662 consensus_queue.cc:1059] T 829b4ad8c5a54eababeea1181b960d4b P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.027s
I20260709 13:19:58.490836 18724 raft_consensus.cc:3060] T 434bff3082d141838068f323bdb0d1ed P cb82cc7bd6db46d980c75237a1402860 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:19:58.488520 16299 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48674 (request call id 2151) took 2171 ms. Trace:
I20260709 13:19:58.483213 18614 raft_consensus.cc:2804] T 68a6838d82ef4cac83507f906139d90f P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:19:58.511572 16291 raft_consensus.cc:1275] T 2794af5b99984efebdee564035841cb0 P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 5. (term mismatch)
I20260709 13:19:58.511757 18614 raft_consensus.cc:697] T 68a6838d82ef4cac83507f906139d90f P 647a343a542d4a139fe106c19095129b [term 7 LEADER]: Becoming Leader. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:19:58.511837 18681 raft_consensus.cc:3055] T 08c74490c40243a88e62d1760b5cd814 P 647a343a542d4a139fe106c19095129b [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:19:58.512607 18681 raft_consensus.cc:740] T 08c74490c40243a88e62d1760b5cd814 P 647a343a542d4a139fe106c19095129b [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:19:58.512966 18614 consensus_queue.cc:237] T 68a6838d82ef4cac83507f906139d90f P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:58.513877 18681 consensus_queue.cc:260] T 08c74490c40243a88e62d1760b5cd814 P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:58.492223 18684 raft_consensus.cc:493] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:58.511601 18560 consensus_queue.cc:1059] T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.004s
I20260709 13:19:58.515410 18684 raft_consensus.cc:3060] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:19:58.516320 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c11074579b046d99e585c2818e18a2c" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:19:58.517418 16223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdbd78acb4cf46298e0de8ba0d130e19" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:19:58.517992 16218 raft_consensus.cc:1275] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:19:58.518399 16223 raft_consensus.cc:2410] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 for term 6 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 13:19:58.472421 18347 consensus_queue.cc:237] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 3, Current term: 4, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:58.519845 16298 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40474 (request call id 368) took 2171 ms. Trace:
I20260709 13:19:58.520236 16298 rpcz_store.cc:276] 0709 13:19:56.348544 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:58.410259 (+2061715us) service_pool.cc:224] Handling call
0709 13:19:58.473806 (+ 63547us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:58.478151 (+  4345us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:58.519657 (+ 41506us) spinlock_profiling.cc:243] Waited 24.1 ms on lock 0x7b54001ee950. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5e8ad 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:19:58.519789 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":55332736}
I20260709 13:19:58.512081 16291 pending_rounds.cc:85] T 2794af5b99984efebdee564035841cb0 P 90f3da57796b488c9087a2f523adf045: Aborting all ops after (but not including) 4
I20260709 13:19:58.521476 16291 pending_rounds.cc:107] T 2794af5b99984efebdee564035841cb0 P 90f3da57796b488c9087a2f523adf045: Aborting uncommitted NO_OP operation due to leader change: 2.5
I20260709 13:19:58.521806 16291 raft_consensus.cc:2889] T 2794af5b99984efebdee564035841cb0 P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:19:58.522603 16291 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40474 (request call id 370) took 2168 ms. Trace:
I20260709 13:19:58.522939 16291 rpcz_store.cc:276] 0709 13:19:56.354529 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:58.418137 (+2063608us) service_pool.cc:224] Handling call
0709 13:19:58.498441 (+ 80304us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:58.522347 (+ 23906us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:58.522546 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:58.524699 16295 raft_consensus.cc:1240] T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 5. Current term is 6. Ops: [5.6-5.6]
I20260709 13:19:58.525079 18599 leader_election.cc:290] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:58.466054 18535 leader_election.cc:290] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:58.505991 16299 rpcz_store.cc:276] 0709 13:19:56.311721 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:58.419001 (+2107280us) service_pool.cc:224] Handling call
0709 13:19:58.477447 (+ 58446us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:58.482969 (+  5522us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:58.483150 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:58.525835 18599 raft_consensus.cc:3055] T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:19:58.496858 16225 rpcz_store.cc:276] 0709 13:19:55.551057 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:57.700580 (+2149523us) service_pool.cc:224] Handling call
0709 13:19:57.828117 (+127537us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:58.319834 (+491717us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:58.453459 (+133625us) spinlock_profiling.cc:243] Waited 24.7 ms on lock 0x7b54000159d4. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:19:58.496255 (+ 42796us) spinlock_profiling.cc:243] Waited 108 ms on lock 0x7b54000159d0. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5e8ad 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:19:58.496385 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":304831616}
I20260709 13:19:58.526316 18599 raft_consensus.cc:740] T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
W20260709 13:19:58.527765 16117 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 90927.1us
I20260709 13:19:58.529559 18599 consensus_queue.cc:260] T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:58.507561 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66ae176961b74813a58bc15c9c4c6f2a" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:58.547384 16217 raft_consensus.cc:2468] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 7.
I20260709 13:19:58.507869 18629 consensus_queue.cc:1048] T a378d7f796f94615b3b347119a9880fd P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.028s
W20260709 13:19:58.511042 16114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:58.549582 16114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:58.533149 16300 raft_consensus.cc:1240] T 8c11074579b046d99e585c2818e18a2c P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 5. Current term is 6. Ops: []
W20260709 13:19:58.535390 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 73974.3us
I20260709 13:19:58.505283 18674 raft_consensus.cc:2749] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:19:58.552234 16819 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49896 (request call id 881) took 1138 ms. Trace:
I20260709 13:19:58.557334 16824 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49906 (request call id 554) took 3652 ms. Trace:
I20260709 13:19:58.561098 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 5 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:19:58.565092 16825 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49896 (request call id 878) took 2689 ms. Trace:
I20260709 13:19:58.565148 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b7d1ab519044eda2d5e04a07242f25" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
W20260709 13:19:58.545735 16789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 220884us
I20260709 13:19:58.552704 16819 rpcz_store.cc:276] 0709 13:19:57.408447 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:58.328754 (+920307us) service_pool.cc:224] Handling call
0709 13:19:58.329291 (+   537us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:58.546629 (+217338us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:58.546663 (+    34us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:58.546716 (+    53us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:19:58.546750 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:58.546855 (+   105us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:58.546979 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":82841,"raft.run_cpu_time_us":186,"raft.run_wall_time_us":15867,"thread_start_us":209041,"threads_started":1}
W20260709 13:19:58.548568 16192 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 212358us
I20260709 13:19:58.570888 16145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af57996a8be84d398f0f023e8d08ef77" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:19:58.548930 18613 raft_consensus.cc:2749] T 8a878999c0024d2abe517484180290fc P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:19:58.572028 16225 raft_consensus.cc:2393] T ecc1e363230043fe9fef390f0b979f20 P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 in current term 5: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
I20260709 13:19:58.573567 16144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f82224f4564fea93572e8206bad1fe" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:19:58.577452 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5718c623ebee4d2bb770f0f9ddb2f164" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:19:58.579468 16145 raft_consensus.cc:3060] T af57996a8be84d398f0f023e8d08ef77 P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Advancing to term 6
W20260709 13:19:58.580550 16044 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:49246 (request call id 329) took 13280 ms (client timeout 15000 ms). Trace:
I20260709 13:19:58.581848 16117 leader_election.cc:400] T af57996a8be84d398f0f023e8d08ef77 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 5 pre-election: Vote denied by peer cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923) with higher term. Message: Invalid argument: T af57996a8be84d398f0f023e8d08ef77 P cb82cc7bd6db46d980c75237a1402860 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b for earlier term 5. Current term is 6.
I20260709 13:19:58.582201 16117 leader_election.cc:403] T af57996a8be84d398f0f023e8d08ef77 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 5 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:19:58.579288 18535 raft_consensus.cc:493] T fffab26427e24d398e2695aad40ca134 P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:58.583467 16117 leader_election.cc:304] T 829b4ad8c5a54eababeea1181b960d4b P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860
I20260709 13:19:58.583726 18535 raft_consensus.cc:515] T fffab26427e24d398e2695aad40ca134 P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:58.588989 18633 leader_election.cc:290] T c0eab25ce4c64e018a041beeda2867db P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:58.591624 18535 leader_election.cc:290] T fffab26427e24d398e2695aad40ca134 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:58.592710 18724 raft_consensus.cc:515] T 434bff3082d141838068f323bdb0d1ed P cb82cc7bd6db46d980c75237a1402860 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:58.555929 18662 consensus_queue.cc:1243] T 829b4ad8c5a54eababeea1181b960d4b P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Peer 1ca2725d290e47d68b1a4b609c9816ac log is divergent from this leader: its last log entry 6.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 13:19:58.594506 18662 raft_consensus.cc:3055] T ada5a004ba7c46819051e3ccec71ffdb P cb82cc7bd6db46d980c75237a1402860 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:19:58.594887 18662 raft_consensus.cc:740] T ada5a004ba7c46819051e3ccec71ffdb P cb82cc7bd6db46d980c75237a1402860 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:19:58.595223 18724 leader_election.cc:290] T 434bff3082d141838068f323bdb0d1ed P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 6 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:58.595966 18662 consensus_queue.cc:260] T ada5a004ba7c46819051e3ccec71ffdb P cb82cc7bd6db46d980c75237a1402860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:58.597487 16818 consensus_queue.cc:260] T 829b4ad8c5a54eababeea1181b960d4b P cb82cc7bd6db46d980c75237a1402860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:58.565686 16825 rpcz_store.cc:276] 0709 13:19:55.875252 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:57.775425 (+1900173us) service_pool.cc:224] Handling call
0709 13:19:57.775841 (+   416us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:57.776142 (+   301us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:19:57.832678 (+ 56536us) log.cc:844] Serialized 14 byte log entry
0709 13:19:58.554743 (+722065us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:58.554793 (+    50us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:19:58.564755 (+  9962us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:58.564862 (+   107us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:58.565020 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":223084,"prepare.run_cpu_time_us":43,"prepare.run_wall_time_us":45,"raft.queue_time_us":67530,"raft.run_cpu_time_us":117,"raft.run_wall_time_us":6480,"thread_start_us":723292,"threads_started":3,"wal-append.queue_time_us":446721}
I20260709 13:19:58.601466 16217 raft_consensus.cc:2376] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 5. Re-sending same reply.
I20260709 13:19:58.571698 18558 leader_election.cc:290] T 3884f864a96d428785895e5463e5728f P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 5 election: Requested vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:19:58.560556 16824 rpcz_store.cc:276] 0709 13:19:54.895641 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:57.755309 (+2859668us) service_pool.cc:224] Handling call
0709 13:19:57.755792 (+   483us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:57.756413 (+   621us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:57.756449 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:57.757223 (+   774us) log.cc:844] Serialized 26 byte log entry
0709 13:19:57.947295 (+190072us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:19:58.304998 (+357703us) log.cc:844] Serialized 14 byte log entry
0709 13:19:58.548031 (+243033us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:58.548134 (+   103us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:58.548194 (+    60us) raft_consensus.cc:1685] finished
0709 13:19:58.548213 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:58.548371 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":56094,"prepare.run_cpu_time_us":45,"prepare.run_wall_time_us":39,"raft.queue_time_us":79440,"raft.run_cpu_time_us":331,"raft.run_wall_time_us":4675,"thread_start_us":710967,"threads_started":3,"wal-append.queue_time_us":571085}
I20260709 13:19:58.612392 16296 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48684 (request call id 2026) took 3699 ms. Trace:
I20260709 13:19:58.599151 16818 raft_consensus.cc:3060] T 829b4ad8c5a54eababeea1181b960d4b P cb82cc7bd6db46d980c75237a1402860 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:19:58.597510 18662 raft_consensus.cc:3060] T ada5a004ba7c46819051e3ccec71ffdb P cb82cc7bd6db46d980c75237a1402860 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:19:58.617250 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e4bfb9090c8460eb14fa7e0d818e7f1" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:19:58.624220 18738 consensus_queue.cc:1048] T b87147731a68400e8cce9b879daa7c5d P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:19:58.625352 18633 consensus_queue.cc:1059] T 18ad7e3af5504571a69d10c00750a01c P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:19:58.626407 18633 raft_consensus.cc:3055] T 18ad7e3af5504571a69d10c00750a01c P 1ca2725d290e47d68b1a4b609c9816ac [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:19:58.626487 16292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806d7690c2b644959152074bb2cab0a7" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:58.611104 18114 consensus_queue.cc:1059] T 8a878999c0024d2abe517484180290fc P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.018s
I20260709 13:19:58.627899 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66f75124215f4ca6bef831e174a792d8" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:58.627993 18114 raft_consensus.cc:3055] T 8a878999c0024d2abe517484180290fc P 1ca2725d290e47d68b1a4b609c9816ac [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:19:58.628513 18114 raft_consensus.cc:740] T 8a878999c0024d2abe517484180290fc P 1ca2725d290e47d68b1a4b609c9816ac [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:19:58.628707 16225 raft_consensus.cc:2393] T 66f75124215f4ca6bef831e174a792d8 P 1ca2725d290e47d68b1a4b609c9816ac [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 in current term 6: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
I20260709 13:19:58.629545 18114 consensus_queue.cc:260] T 8a878999c0024d2abe517484180290fc P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:58.625344 16047 catalog_manager.cc:5697] T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045 reported cstate change: term changed from 1 to 5. New cstate: current_term: 5 leader_uuid: "90f3da57796b488c9087a2f523adf045" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } health_report { overall_health: HEALTHY } } }
I20260709 13:19:58.609404 18734 raft_consensus.cc:2804] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:19:58.631589 16192 leader_election.cc:304] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:19:58.626875 18633 raft_consensus.cc:740] T 18ad7e3af5504571a69d10c00750a01c P 1ca2725d290e47d68b1a4b609c9816ac [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:19:58.633776 16190 leader_election.cc:304] T 08c74490c40243a88e62d1760b5cd814 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:19:58.639156 16047 catalog_manager.cc:5697] T 8c11074579b046d99e585c2818e18a2c P 90f3da57796b488c9087a2f523adf045 reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "1ca2725d290e47d68b1a4b609c9816ac" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:19:58.630963 18114 raft_consensus.cc:3060] T 8a878999c0024d2abe517484180290fc P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:19:58.656126 16297 raft_consensus.cc:2468] T e5ad4eb281564936a59f41e387a47156 P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 6.
W20260709 13:19:58.660040 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 176982us
I20260709 13:19:58.661154 16293 raft_consensus.cc:1240] T e5ad4eb281564936a59f41e387a47156 P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 5. Current term is 6. Ops: []
I20260709 13:19:58.613727 16296 rpcz_store.cc:276] 0709 13:19:54.908674 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:58.411354 (+3502680us) service_pool.cc:224] Handling call
0709 13:19:58.434152 (+ 22798us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:58.434452 (+   300us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:19:58.487676 (+ 53224us) log.cc:844] Serialized 14 byte log entry
0709 13:19:58.602480 (+114804us) log.cc:844] Serialized 14 byte log entry
0709 13:19:58.602799 (+   319us) log.cc:844] Serialized 14 byte log entry
0709 13:19:58.604610 (+  1811us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:58.604659 (+    49us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:19:58.607534 (+  2875us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:58.607670 (+   136us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:58.607856 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3376,"prepare.queue_time_us":179089,"raft.queue_time_us":2445,"raft.run_cpu_time_us":238,"raft.run_wall_time_us":4139,"spinlock_wait_cycles":4146048,"thread_start_us":132317,"threads_started":2}
I20260709 13:19:58.664403 16294 raft_consensus.cc:2468] T ecc1e363230043fe9fef390f0b979f20 P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 5.
I20260709 13:19:58.665059 16825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3884f864a96d428785895e5463e5728f" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
W20260709 13:19:58.663594 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 107825us
W20260709 13:19:58.666644 16192 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 52184.1us
I20260709 13:19:58.671231 18587 raft_consensus.cc:2804] T 08c74490c40243a88e62d1760b5cd814 P 1ca2725d290e47d68b1a4b609c9816ac [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:19:58.671348 16217 raft_consensus.cc:1275] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 7. (term mismatch)
I20260709 13:19:58.671830 16192 leader_election.cc:304] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 90f3da57796b488c9087a2f523adf045
I20260709 13:19:58.672188 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8638ee363273426db1393dd81e5c7b72" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:58.673058 16297 raft_consensus.cc:2393] T 8638ee363273426db1393dd81e5c7b72 P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 5: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
I20260709 13:19:58.672909 18689 consensus_queue.cc:1048] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:19:58.674690 18689 raft_consensus.cc:2804] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:19:58.675199 18689 raft_consensus.cc:493] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Starting leader election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:19:58.675725 18689 raft_consensus.cc:3060] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:19:58.675815 16115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 244312us
I20260709 13:19:58.677174 18735 consensus_queue.cc:1048] T 2794af5b99984efebdee564035841cb0 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:19:58.678514 16115 leader_election.cc:304] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac
I20260709 13:19:58.671833 16217 pending_rounds.cc:85] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac: Aborting all ops after (but not including) 6
I20260709 13:19:58.679225 18735 consensus_queue.cc:1059] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:19:58.679874 16217 pending_rounds.cc:107] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac: Aborting uncommitted NO_OP operation due to leader change: 6.7
I20260709 13:19:58.633716 18558 raft_consensus.cc:2749] T 829b4ad8c5a54eababeea1181b960d4b P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:19:58.680302 16217 raft_consensus.cc:2889] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:19:58.640334 18681 raft_consensus.cc:3060] T 08c74490c40243a88e62d1760b5cd814 P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:19:58.684177 16819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c11074579b046d99e585c2818e18a2c" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:19:58.670192 16824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31969392ccdb43b491911a319e8bc7e1" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:19:58.705943 16819 raft_consensus.cc:2393] T 8c11074579b046d99e585c2818e18a2c P cb82cc7bd6db46d980c75237a1402860 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 6: Already voted for candidate cb82cc7bd6db46d980c75237a1402860 in this term.
I20260709 13:19:58.617652 18535 consensus_queue.cc:1059] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.064s
I20260709 13:19:58.681331 16047 catalog_manager.cc:5697] T a378d7f796f94615b3b347119a9880fd P 90f3da57796b488c9087a2f523adf045 reported cstate change: term changed from 3 to 4, leader changed from cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) to 90f3da57796b488c9087a2f523adf045 (127.14.177.131). New cstate: current_term: 4 leader_uuid: "90f3da57796b488c9087a2f523adf045" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 13:19:58.718827 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 224854us
I20260709 13:19:58.712821 16825 raft_consensus.cc:2468] T 3884f864a96d428785895e5463e5728f P cb82cc7bd6db46d980c75237a1402860 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 4.
I20260709 13:19:58.720247 16790 leader_election.cc:304] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:19:58.723440 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74ca8140f35c4ae29e8a664f6bdfc829" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:19:58.725150 18558 raft_consensus.cc:2804] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:19:58.724903 18741 consensus_queue.cc:1059] T 746aabf856ab401c9ef5e859371e7b67 P 647a343a542d4a139fe106c19095129b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
W20260709 13:19:58.727268 16791 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:58.723515 18535 raft_consensus.cc:3055] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:19:58.727934 18750 consensus_queue.cc:1059] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:19:58.747689 16823 raft_consensus.cc:1240] T 26354361e9b044cea1c2b86c4882fe92 P cb82cc7bd6db46d980c75237a1402860 [term 3 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 2. Current term is 3. Ops: []
I20260709 13:19:58.762713 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9944d5e4b8c4109af9955c4f08c9095" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
W20260709 13:19:58.748432 16117 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 163730us
W20260709 13:19:58.765179 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 84760.1us
I20260709 13:19:58.749961 16819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e9b2bf7011f4ae688a79d7bb5a447b4" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:19:58.767073 16819 raft_consensus.cc:3060] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:19:58.753000 16145 raft_consensus.cc:2468] T af57996a8be84d398f0f023e8d08ef77 P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 6.
W20260709 13:19:58.581002 16044 rpcz_store.cc:269] 0709 13:19:45.299475 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:45.299658 (+   183us) service_pool.cc:224] Handling call
0709 13:19:45.440661 (+141003us) spinlock_profiling.cc:243] Waited 30.9 ms on lock 0x7b54003cdf54. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac9090 00007f69b0b67f3a 00007f69b691e1cc 00007f69b6a1c37e 00007f69b4edd2a4 00007f69b4edd232 00007f69b4edd161 00007f69b4edd0dc 00007f69b4edc392 00007f69aee9a48c 00007f69aee998c6
0709 13:19:58.580283 (+13139622us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:19:53.700428 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:19:53.701084 (+   656us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:19:53.701132 (+    48us) write_op.cc:435] Acquired schema lock
0709 13:19:53.701154 (+    22us) tablet.cc:662] Decoding operations
0709 13:19:53.704358 (+  3204us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:19:53.704437 (+    79us) write_op.cc:641] Partition lock acquired for write op
0709 13:19:53.704464 (+    27us) tablet.cc:685] Acquiring locks for 11 operations
0709 13:19:53.715450 (+ 10986us) tablet.cc:701] Row locks acquired
0709 13:19:53.715500 (+    50us) write_op.cc:260] PREPARE: finished
0709 13:19:53.715668 (+   168us) write_op.cc:270] Start()
0709 13:19:53.715862 (+   194us) write_op.cc:276] Timestamp: P: 1783603193715611 usec, L: 0
0709 13:19:53.715891 (+    29us) op_driver.cc:348] REPLICATION: starting
0709 13:19:53.716566 (+   675us) log.cc:844] Serialized 4804 byte log entry
0709 13:19:54.080525 (+363959us) op_driver.cc:464] REPLICATION: finished
0709 13:19:54.540410 (+459885us) write_op.cc:301] APPLY: starting
0709 13:19:54.540526 (+   116us) tablet.cc:1367] starting BulkCheckPresence
0709 13:19:54.571037 (+ 30511us) tablet.cc:1370] finished BulkCheckPresence
0709 13:19:54.636984 (+ 65947us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:19:54.923505 (+286521us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:19:54.924015 (+   510us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=11
0709 13:19:54.924094 (+    79us) write_op.cc:312] APPLY: finished
0709 13:19:54.924609 (+   515us) log.cc:844] Serialized 83 byte log entry
0709 13:19:54.924816 (+   207us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:19:54.924965 (+   149us) write_op.cc:454] Released schema lock
0709 13:19:54.925030 (+    65us) write_op.cc:341] FINISH: Updating metrics
0709 13:19:54.925513 (+   483us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:19:54.925555 (+    42us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":4686640,"reported_tablets":21,"spinlock_wait_cycles":71146752,"child_traces":[["op",{"apply.queue_time_us":375837,"apply.run_cpu_time_us":13081,"apply.run_wall_time_us":385040,"num_ops":11,"prepare.queue_time_us":177458,"prepare.run_cpu_time_us":102530,"prepare.run_wall_time_us":165853,"raft.queue_time_us":120389,"raft.run_cpu_time_us":12290,"raft.run_wall_time_us":360561,"replication_time_us":339237,"thread_start_us":579257,"threads_started":4,"wal-append.queue_time_us":182825,"wal-append.run_cpu_time_us":31179,"wal-append.run_wall_time_us":2043415}]]}
I20260709 13:19:58.748972 16190 leader_election.cc:304] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
W20260709 13:19:58.757055 16115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 78603.9us
I20260709 13:19:58.685851 18551 raft_consensus.cc:2804] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:19:58.744971 16817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74ca8140f35c4ae29e8a664f6bdfc829" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:19:58.770288 16824 raft_consensus.cc:2393] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 5: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
I20260709 13:19:58.768802 16115 leader_election.cc:304] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:19:58.769728 16047 catalog_manager.cc:5697] T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045 reported cstate change: term changed from 2 to 4, leader changed from 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130) to 90f3da57796b488c9087a2f523adf045 (127.14.177.131). New cstate: current_term: 4 leader_uuid: "90f3da57796b488c9087a2f523adf045" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } health_report { overall_health: HEALTHY } } }
I20260709 13:19:58.773491 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a378d7f796f94615b3b347119a9880fd" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:58.769588 16116 leader_election.cc:304] T 8cc959fb7f704fba9009c0595467d074 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
I20260709 13:19:58.777029 16217 raft_consensus.cc:2468] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 6.
I20260709 13:19:58.760535 18535 raft_consensus.cc:740] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:19:58.780696 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5d59407eb545b895d8d922e0dd69fd" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 7 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:58.779186 16047 catalog_manager.cc:5697] T fffab26427e24d398e2695aad40ca134 P 90f3da57796b488c9087a2f523adf045 reported cstate change: term changed from 1 to 2, leader changed from 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130) to 90f3da57796b488c9087a2f523adf045 (127.14.177.131). New cstate: current_term: 2 leader_uuid: "90f3da57796b488c9087a2f523adf045" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } health_report { overall_health: HEALTHY } } }
I20260709 13:19:58.782388 16817 raft_consensus.cc:2393] T 74ca8140f35c4ae29e8a664f6bdfc829 P cb82cc7bd6db46d980c75237a1402860 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 6: Already voted for candidate cb82cc7bd6db46d980c75237a1402860 in this term.
I20260709 13:19:58.777267 18558 raft_consensus.cc:697] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [term 6 LEADER]: Becoming Leader. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:19:58.782402 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434bff3082d141838068f323bdb0d1ed" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:58.783402 16292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26354361e9b044cea1c2b86c4882fe92" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:58.787496 16192 leader_election.cc:304] T 66f75124215f4ca6bef831e174a792d8 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:19:58.786252 16823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5d59407eb545b895d8d922e0dd69fd" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:19:58.787953 16292 raft_consensus.cc:2468] T 26354361e9b044cea1c2b86c4882fe92 P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 3.
I20260709 13:19:58.784777 18741 raft_consensus.cc:2804] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:19:58.773542 18613 raft_consensus.cc:2749] T af57996a8be84d398f0f023e8d08ef77 P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Leader pre-election lost for term 5. Reason: Vote denied by peer cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923) with higher term. Message: Invalid argument: T af57996a8be84d398f0f023e8d08ef77 P cb82cc7bd6db46d980c75237a1402860 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b for earlier term 5. Current term is 6.
I20260709 13:19:58.781669 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434bff3082d141838068f323bdb0d1ed" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:19:58.795848 16790 leader_election.cc:304] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:19:58.793659 16825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62fe3346cd3c42c4a55e14af9073bca4" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:19:58.797009 16823 raft_consensus.cc:2468] T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 6.
I20260709 13:19:58.771179 18587 raft_consensus.cc:697] T 08c74490c40243a88e62d1760b5cd814 P 1ca2725d290e47d68b1a4b609c9816ac [term 6 LEADER]: Becoming Leader. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
W20260709 13:19:58.803261 16266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 69994.9us
I20260709 13:19:58.804314 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c11074579b046d99e585c2818e18a2c" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:19:58.806221 16292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3884f864a96d428785895e5463e5728f" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:58.807940 16115 leader_election.cc:304] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:19:58.821089 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2372981006d4b218215c28964dea623" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:58.803732 16299 raft_consensus.cc:2468] T 434bff3082d141838068f323bdb0d1ed P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 5.
I20260709 13:19:58.821542 16823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a9de347c711477e87c582d003f36925" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:19:58.829275 16297 raft_consensus.cc:2376] T 8c11074579b046d99e585c2818e18a2c P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 6. Re-sending same reply.
W20260709 13:19:58.829030 16266 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:58.832216 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 48105.5us
I20260709 13:19:58.832820 18756 raft_consensus.cc:2749] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:19:58.833770 16116 leader_election.cc:304] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 90f3da57796b488c9087a2f523adf045
I20260709 13:19:58.834024 18756 consensus_queue.cc:1059] T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:19:58.829030 16225 raft_consensus.cc:2468] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 5.
I20260709 13:19:58.838596 18758 raft_consensus.cc:2749] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:19:58.840894 18758 raft_consensus.cc:2804] T 66f75124215f4ca6bef831e174a792d8 P 1ca2725d290e47d68b1a4b609c9816ac [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:19:58.841730 18758 raft_consensus.cc:697] T 66f75124215f4ca6bef831e174a792d8 P 1ca2725d290e47d68b1a4b609c9816ac [term 6 LEADER]: Becoming Leader. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
W20260709 13:19:58.843416 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 48237.4us
I20260709 13:19:58.847512 16292 raft_consensus.cc:2468] T 3884f864a96d428785895e5463e5728f P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 4.
I20260709 13:19:58.848611 16266 leader_election.cc:304] T 8c11074579b046d99e585c2818e18a2c P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860
W20260709 13:19:58.848626 16192 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 71423.1us
I20260709 13:19:58.787856 18684 raft_consensus.cc:515] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:58.814626 18741 raft_consensus.cc:697] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 647a343a542d4a139fe106c19095129b [term 6 LEADER]: Becoming Leader. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:19:58.788712 18633 consensus_queue.cc:260] T 18ad7e3af5504571a69d10c00750a01c P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 2, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } }
I20260709 13:19:58.856081 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ab0ac8347641e6955e9bce7d90c1a2" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:58.859457 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746aabf856ab401c9ef5e859371e7b67" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:58.853439 18724 raft_consensus.cc:2804] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:19:58.863350 16296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8158bbd17b994d2eb96f560bd0f1b88d" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:58.861222 18724 raft_consensus.cc:493] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:58.855779 18535 consensus_queue.cc:260] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:58.864100 18724 raft_consensus.cc:3060] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:19:58.864900 18535 raft_consensus.cc:3060] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:19:58.867573 18749 consensus_queue.cc:1048] T 746aabf856ab401c9ef5e859371e7b67 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.047s
I20260709 13:19:58.867671 18558 consensus_queue.cc:237] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.5, Last appended by leader: 5, Current term: 6, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:58.869066 18633 raft_consensus.cc:3060] T 18ad7e3af5504571a69d10c00750a01c P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:19:58.869556 18747 raft_consensus.cc:2804] T 8cc959fb7f704fba9009c0595467d074 P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:19:58.870085 18747 raft_consensus.cc:493] T 8cc959fb7f704fba9009c0595467d074 P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Starting leader election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:19:58.870512 18747 raft_consensus.cc:3060] T 8cc959fb7f704fba9009c0595467d074 P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:19:58.871801 16817 raft_consensus.cc:1240] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 [term 4 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 3. Current term is 4. Ops: []
I20260709 13:19:58.871536 18587 consensus_queue.cc:237] T 08c74490c40243a88e62d1760b5cd814 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:58.862893 16223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b7d1ab519044eda2d5e04a07242f25" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:58.868162 18421 raft_consensus.cc:2749] T 26354361e9b044cea1c2b86c4882fe92 P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:19:58.881539 16819 raft_consensus.cc:2468] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 4.
I20260709 13:19:58.882911 18743 raft_consensus.cc:2749] T 8c11074579b046d99e585c2818e18a2c P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:19:58.885104 18684 leader_election.cc:290] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 4 election: Requested vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:19:58.886267 18585 raft_consensus.cc:515] T 434bff3082d141838068f323bdb0d1ed P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:58.887322 18745 consensus_queue.cc:1059] T 746aabf856ab401c9ef5e859371e7b67 P 647a343a542d4a139fe106c19095129b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.109s
I20260709 13:19:58.887771 16294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e65676b21b748728ad39668cfe722ec" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:58.888114 18735 raft_consensus.cc:2764] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860 [term 4 FOLLOWER]: Leader pre-election decision vote started in defunct term 3: won
I20260709 13:19:58.888612 18585 leader_election.cc:290] T 434bff3082d141838068f323bdb0d1ed P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 6 election: Requested vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:19:58.889218 16825 raft_consensus.cc:1275] T 806d7690c2b644959152074bb2cab0a7 P cb82cc7bd6db46d980c75237a1402860 [term 4 LEARNER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
I20260709 13:19:58.855376 18485 raft_consensus.cc:515] T 57a744f5aaf944678a63469cbf514015 P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:58.889668 18689 raft_consensus.cc:515] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:58.891324 18485 leader_election.cc:290] T 57a744f5aaf944678a63469cbf514015 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 4 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:19:58.889117 18734 raft_consensus.cc:493] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
W20260709 13:19:58.892547 16266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 43996us
W20260709 13:19:58.893539 16115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 85701.4us
I20260709 13:19:58.888667 18753 raft_consensus.cc:3055] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 3 LEADER]: Stepping down as leader of term 3
W20260709 13:19:58.904000 16191 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:58.909062 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 201149us
I20260709 13:19:58.900113 16115 leader_election.cc:304] T ecc1e363230043fe9fef390f0b979f20 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:19:58.890522 18741 consensus_queue.cc:237] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:58.892238 18689 leader_election.cc:290] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 5 election: Requested vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:19:58.915937 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af57996a8be84d398f0f023e8d08ef77" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:19:58.917641 16217 raft_consensus.cc:3060] T af57996a8be84d398f0f023e8d08ef77 P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:19:58.856566 18744 consensus_queue.cc:1048] T c9944d5e4b8c4109af9955c4f08c9095 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.048s
I20260709 13:19:58.919463 16819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5718c623ebee4d2bb770f0f9ddb2f164" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:19:58.916976 18745 consensus_queue.cc:1243] T 746aabf856ab401c9ef5e859371e7b67 P 647a343a542d4a139fe106c19095129b [LEADER]: Peer 1ca2725d290e47d68b1a4b609c9816ac log is divergent from this leader: its last log entry 5.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 13:19:58.895066 18753 raft_consensus.cc:740] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:19:58.917165 18485 raft_consensus.cc:493] T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:58.894554 16047 catalog_manager.cc:5697] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045 reported cstate change: term changed from 3 to 4, leader changed from 647a343a542d4a139fe106c19095129b (127.14.177.129) to 90f3da57796b488c9087a2f523adf045 (127.14.177.131). New cstate: current_term: 4 leader_uuid: "90f3da57796b488c9087a2f523adf045" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:19:58.896987 18734 raft_consensus.cc:3060] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:19:58.921669 18347 consensus_queue.cc:1059] T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.004s
I20260709 13:19:58.920208 16265 leader_election.cc:304] T 66ae176961b74813a58bc15c9c4c6f2a P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
W20260709 13:19:58.929181 16114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:58.931306 16825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8638ee363273426db1393dd81e5c7b72" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
W20260709 13:19:58.933712 16191 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:58.931804 16190 leader_election.cc:304] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: cb82cc7bd6db46d980c75237a1402860
I20260709 13:19:58.933059 16825 raft_consensus.cc:2468] T 8638ee363273426db1393dd81e5c7b72 P cb82cc7bd6db46d980c75237a1402860 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 5.
I20260709 13:19:58.857529 18758 consensus_queue.cc:237] T 66f75124215f4ca6bef831e174a792d8 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:19:58.939891 16117 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:58.940828 18743 raft_consensus.cc:3055] T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:19:58.941267 18743 raft_consensus.cc:740] T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:19:58.943701 16265 leader_election.cc:304] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:19:58.945531 16265 leader_election.cc:304] T 02838c86531d4f98bf58d1c22b8baf77 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:19:58.897320 18749 consensus_queue.cc:1059] T 8c11074579b046d99e585c2818e18a2c P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.039s
W20260709 13:19:58.947822 16114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:58.951262 16115 leader_election.cc:304] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:19:58.915975 18551 raft_consensus.cc:697] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac [term 4 LEADER]: Becoming Leader. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:19:58.926108 18681 raft_consensus.cc:2764] T ecc1e363230043fe9fef390f0b979f20 P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Leader pre-election decision vote started in defunct term 4: won
I20260709 13:19:58.942402 18743 consensus_queue.cc:260] T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:58.931036 18684 raft_consensus.cc:493] T fffab26427e24d398e2695aad40ca134 P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:58.962332 16116 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:58.963517 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 116387us
I20260709 13:19:58.962731 16192 leader_election.cc:304] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:19:58.967790 16047 catalog_manager.cc:5697] T 26354361e9b044cea1c2b86c4882fe92 P 90f3da57796b488c9087a2f523adf045 reported cstate change: term changed from 1 to 2, leader changed from 647a343a542d4a139fe106c19095129b (127.14.177.129) to 90f3da57796b488c9087a2f523adf045 (127.14.177.131). New cstate: current_term: 2 leader_uuid: "90f3da57796b488c9087a2f523adf045" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:19:58.971052 16819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2372981006d4b218215c28964dea623" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:19:58.958946 18674 raft_consensus.cc:2804] T 66ae176961b74813a58bc15c9c4c6f2a P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 13:19:58.978426 16191 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:58.968677 18347 raft_consensus.cc:2804] T 02838c86531d4f98bf58d1c22b8baf77 P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:19:58.936103 18485 raft_consensus.cc:515] T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:58.963121 18580 raft_consensus.cc:2749] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:19:58.966423 18720 raft_consensus.cc:2804] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:19:58.969010 18689 raft_consensus.cc:2804] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac [term 6 FOLLOWER]: Leader election won for term 6
W20260709 13:19:58.986994 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 60680.4us
I20260709 13:19:58.981632 18750 raft_consensus.cc:2804] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:19:58.978621 18674 raft_consensus.cc:493] T 66ae176961b74813a58bc15c9c4c6f2a P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:19:58.966279 18733 consensus_queue.cc:1059] T 8d4ab870a05347868956aa8b841b2981 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: INVALID_TERM, Last received: 5.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.023s
I20260709 13:19:59.009632 18485 leader_election.cc:290] T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:19:58.984879 18720 raft_consensus.cc:493] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:59.017155 16191 leader_election.cc:304] T b6f07842d3aa47858df5665874c86f3a P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:19:58.986686 18347 raft_consensus.cc:697] T 02838c86531d4f98bf58d1c22b8baf77 P 90f3da57796b488c9087a2f523adf045 [term 6 LEADER]: Becoming Leader. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:19:59.010515 16821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a378d7f796f94615b3b347119a9880fd" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
W20260709 13:19:59.023286 16264 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:59.018352 16047 catalog_manager.cc:5697] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045 reported cstate change: term changed from 3 to 5, leader changed from 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130) to 90f3da57796b488c9087a2f523adf045 (127.14.177.131). New cstate: current_term: 5 leader_uuid: "90f3da57796b488c9087a2f523adf045" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } health_report { overall_health: HEALTHY } } }
I20260709 13:19:58.986275 18662 consensus_queue.cc:1059] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.033s
I20260709 13:19:59.027604 18760 raft_consensus.cc:2804] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:19:58.991693 18744 consensus_queue.cc:1048] T c4f65cdf00b748bb963ee6e81ef43b79 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.034s
I20260709 13:19:58.984375 18753 consensus_queue.cc:260] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:59.026070 16047 catalog_manager.cc:5697] T ada5a004ba7c46819051e3ccec71ffdb P 90f3da57796b488c9087a2f523adf045 reported cstate change: term changed from 4 to 5, leader changed from 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130) to cb82cc7bd6db46d980c75237a1402860 (127.14.177.132). New cstate: current_term: 5 leader_uuid: "cb82cc7bd6db46d980c75237a1402860" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:19:59.005432 18750 raft_consensus.cc:493] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Starting leader election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:19:59.011183 18674 raft_consensus.cc:3060] T 66ae176961b74813a58bc15c9c4c6f2a P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:19:58.986470 18684 raft_consensus.cc:515] T fffab26427e24d398e2695aad40ca134 P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:59.018296 18720 raft_consensus.cc:3060] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:19:59.018941 18743 raft_consensus.cc:3060] T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:19:59.070891 16192 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 108252us
I20260709 13:19:59.071681 16047 catalog_manager.cc:5697] T 9e65676b21b748728ad39668cfe722ec P 90f3da57796b488c9087a2f523adf045 reported cstate change: term changed from 3 to 4, leader changed from cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) to 90f3da57796b488c9087a2f523adf045 (127.14.177.131). New cstate: current_term: 4 leader_uuid: "90f3da57796b488c9087a2f523adf045" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 13:19:59.076102 16789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 183957us
I20260709 13:19:59.062191 18750 raft_consensus.cc:3060] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:19:59.056269 18760 raft_consensus.cc:697] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b [term 5 LEADER]: Becoming Leader. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
W20260709 13:19:59.091768 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 145397us
W20260709 13:19:59.099546 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 90783.4us
I20260709 13:19:59.099972 18760 consensus_queue.cc:237] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.4, Last appended by leader: 4, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } }
I20260709 13:19:59.101714 18587 consensus_queue.cc:1059] T e4dc6b0541ca431ebc757beac00521f1 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:19:59.103318 18587 raft_consensus.cc:3055] T e4dc6b0541ca431ebc757beac00521f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:19:59.103825 18587 raft_consensus.cc:740] T e4dc6b0541ca431ebc757beac00521f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
W20260709 13:19:59.108851 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 102709us
I20260709 13:19:59.112905 16191 leader_election.cc:400] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 5 election: Vote denied by peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741) with higher term. Message: Invalid argument: T 87ac0d1f7e8a419f9bcbc04aee0e067e P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac for earlier term 5. Current term is 6.
I20260709 13:19:59.113234 16191 leader_election.cc:403] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 5 election: Cancelling election due to peer responding with higher term
I20260709 13:19:59.123257 16047 catalog_manager.cc:5697] T 2794af5b99984efebdee564035841cb0 P 90f3da57796b488c9087a2f523adf045 reported cstate change: term changed from 1 to 2, leader changed from 647a343a542d4a139fe106c19095129b (127.14.177.129) to 90f3da57796b488c9087a2f523adf045 (127.14.177.131). New cstate: current_term: 2 leader_uuid: "90f3da57796b488c9087a2f523adf045" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:19:59.097729 18585 raft_consensus.cc:493] T 1df93779209748b9860f6d893dd3de53 P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:59.127561 18762 raft_consensus.cc:3060] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:19:59.131565 16191 leader_election.cc:304] T af57996a8be84d398f0f023e8d08ef77 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
I20260709 13:19:59.134672 16816 raft_consensus.cc:1275] T 9e65676b21b748728ad39668cfe722ec P cb82cc7bd6db46d980c75237a1402860 [term 6 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 13:19:59.135294 18757 consensus_queue.cc:1059] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.044s
W20260709 13:19:59.141273 16189 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:59.142016 16189 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:59.105530 18767 raft_consensus.cc:2749] T b6f07842d3aa47858df5665874c86f3a P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:19:59.144480 16225 raft_consensus.cc:1240] T 1df93779209748b9860f6d893dd3de53 P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 4. Current term is 5. Ops: []
W20260709 13:19:59.171908 16818 consensus_meta.cc:287] T 829b4ad8c5a54eababeea1181b960d4b P cb82cc7bd6db46d980c75237a1402860: Time spent flushing consensus metadata: real 0.548s	user 0.010s	sys 0.000s
I20260709 13:19:59.173708 16818 raft_consensus.cc:1275] T 829b4ad8c5a54eababeea1181b960d4b P cb82cc7bd6db46d980c75237a1402860 [term 6 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 13:19:59.174576 16818 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49906 (request call id 565) took 1425 ms. Trace:
I20260709 13:19:59.176999 16818 rpcz_store.cc:276] 0709 13:19:57.749334 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:58.420337 (+671003us) service_pool.cc:224] Handling call
0709 13:19:58.420722 (+   385us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:58.596853 (+176131us) spinlock_profiling.cc:243] Waited 144 ms on lock 0x7b54007c1120. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0add06f 00007f69b0b55f60 00007f69b0b577de 00007f69b0b61b49 00007f69b0b62416 00007f69b0b5ed8f 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62
0709 13:19:59.174218 (+577365us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:59.174499 (+   281us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":331693312}
W20260709 13:19:59.189248 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 222356us
I20260709 13:19:59.108143 18689 raft_consensus.cc:697] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac [term 6 LEADER]: Becoming Leader. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:19:59.071789 18551 consensus_queue.cc:237] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:59.194506 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2372981006d4b218215c28964dea623" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:19:59.196718 18535 raft_consensus.cc:493] T 5a96373bb0e14dce8a266b2fc6c58507 P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:59.200960 16225 raft_consensus.cc:2393] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 5: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
I20260709 13:19:59.203559 16265 leader_election.cc:304] T c0eab25ce4c64e018a041beeda2867db P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 647a343a542d4a139fe106c19095129b
W20260709 13:19:59.206807 18765 log.cc:927] Time spent T 806d7690c2b644959152074bb2cab0a7 P cb82cc7bd6db46d980c75237a1402860: Append to log took a long time: real 0.118s	user 0.001s	sys 0.000s
I20260709 13:19:59.206529 18535 raft_consensus.cc:515] T 5a96373bb0e14dce8a266b2fc6c58507 P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:59.208746 18535 leader_election.cc:290] T 5a96373bb0e14dce8a266b2fc6c58507 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:59.211272 18580 raft_consensus.cc:3055] T 8d4ab870a05347868956aa8b841b2981 P 90f3da57796b488c9087a2f523adf045 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:19:59.211740 18580 raft_consensus.cc:740] T 8d4ab870a05347868956aa8b841b2981 P 90f3da57796b488c9087a2f523adf045 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:19:59.212893 18580 consensus_queue.cc:260] T 8d4ab870a05347868956aa8b841b2981 P 90f3da57796b488c9087a2f523adf045 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.2, Last appended by leader: 2, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:59.214200 18580 raft_consensus.cc:3060] T 8d4ab870a05347868956aa8b841b2981 P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:19:59.216925 16217 raft_consensus.cc:2468] T af57996a8be84d398f0f023e8d08ef77 P 1ca2725d290e47d68b1a4b609c9816ac [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 6.
I20260709 13:19:59.221160 18684 leader_election.cc:290] T fffab26427e24d398e2695aad40ca134 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:59.143630 18347 consensus_queue.cc:237] T 02838c86531d4f98bf58d1c22b8baf77 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:59.171626 18585 raft_consensus.cc:515] T 1df93779209748b9860f6d893dd3de53 P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:59.229692 18585 leader_election.cc:290] T 1df93779209748b9860f6d893dd3de53 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:59.230134 18599 raft_consensus.cc:3060] T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:19:59.230585 18736 raft_consensus.cc:3055] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:19:59.230769 16789 leader_election.cc:304] T d2ab0ac8347641e6955e9bce7d90c1a2 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:19:59.229768 18614 raft_consensus.cc:3055] T 746aabf856ab401c9ef5e859371e7b67 P 647a343a542d4a139fe106c19095129b [term 4 LEADER]: Stepping down as leader of term 4
W20260709 13:19:59.235809 16263 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:59.252884 16263 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:59.251525 16218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87ac0d1f7e8a419f9bcbc04aee0e067e" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
W20260709 13:19:59.253412 16263 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:59.253803 16263 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:59.254299 18614 raft_consensus.cc:740] T 746aabf856ab401c9ef5e859371e7b67 P 647a343a542d4a139fe106c19095129b [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:19:59.256172 18614 consensus_queue.cc:260] T 746aabf856ab401c9ef5e859371e7b67 P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:59.256906 18747 raft_consensus.cc:515] T 8cc959fb7f704fba9009c0595467d074 P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:59.257485 18614 raft_consensus.cc:3060] T 746aabf856ab401c9ef5e859371e7b67 P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:19:59.258823 18777 raft_consensus.cc:2764] T c0eab25ce4c64e018a041beeda2867db P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Leader pre-election decision vote started in defunct term 4: won
I20260709 13:19:59.259941 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0eab25ce4c64e018a041beeda2867db" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:59.260978 16225 raft_consensus.cc:2410] T c0eab25ce4c64e018a041beeda2867db P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b for term 6 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:19:59.264912 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62fe3346cd3c42c4a55e14af9073bca4" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:59.265640 16217 raft_consensus.cc:2468] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 7.
I20260709 13:19:59.266592 16820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87147731a68400e8cce9b879daa7c5d" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
W20260709 13:19:59.231084 18114 consensus_meta.cc:287] T 8a878999c0024d2abe517484180290fc P 1ca2725d290e47d68b1a4b609c9816ac: Time spent flushing consensus metadata: real 0.562s	user 0.008s	sys 0.000s
W20260709 13:19:59.269294 16266 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:59.282433 16266 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:59.270958 16115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 261165us
I20260709 13:19:59.280349 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c2095a5f4ad49408dbc7f69c606120d" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:19:59.122895 18587 consensus_queue.cc:260] T e4dc6b0541ca431ebc757beac00521f1 P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:59.283797 16115 leader_election.cc:304] T d16dfad0c3b540e5b4b86492ddf87208 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:19:59.219048 18757 raft_consensus.cc:2749] T af57996a8be84d398f0f023e8d08ef77 P 1ca2725d290e47d68b1a4b609c9816ac [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:19:59.231060 18736 raft_consensus.cc:740] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:19:59.235021 18753 raft_consensus.cc:3060] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:19:59.290616 16117 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 214568us
I20260709 13:19:59.284830 16225 raft_consensus.cc:3060] T 7c2095a5f4ad49408dbc7f69c606120d P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:19:59.295702 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 103010us
I20260709 13:19:59.266346 18749 consensus_queue.cc:1059] T 8c11074579b046d99e585c2818e18a2c P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.033s
I20260709 13:19:59.236608 18759 raft_consensus.cc:3055] T 8c11074579b046d99e585c2818e18a2c P 1ca2725d290e47d68b1a4b609c9816ac [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:19:59.298063 16789 leader_election.cc:304] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
W20260709 13:19:59.298125 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 94661us
I20260709 13:19:59.264312 18747 leader_election.cc:290] T 8cc959fb7f704fba9009c0595467d074 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 6 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:59.286209 18735 raft_consensus.cc:2804] T d2ab0ac8347641e6955e9bce7d90c1a2 P cb82cc7bd6db46d980c75237a1402860 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:19:59.220084 18734 raft_consensus.cc:515] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:59.295552 18599 raft_consensus.cc:493] T 2794af5b99984efebdee564035841cb0 P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:59.328555 18689 consensus_queue.cc:237] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:59.329294 18599 raft_consensus.cc:515] T 2794af5b99984efebdee564035841cb0 P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:59.329430 18720 raft_consensus.cc:515] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:59.331740 16789 leader_election.cc:304] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:19:59.332751 16789 leader_election.cc:304] T 23f82224f4564fea93572e8206bad1fe P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 647a343a542d4a139fe106c19095129b
I20260709 13:19:59.334517 18785 raft_consensus.cc:2804] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:19:59.306648 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5d59407eb545b895d8d922e0dd69fd" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:59.340165 16225 raft_consensus.cc:2468] T 7c2095a5f4ad49408dbc7f69c606120d P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 5.
I20260709 13:19:59.311126 16790 leader_election.cc:304] T af57996a8be84d398f0f023e8d08ef77 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:19:59.299669 18558 raft_consensus.cc:2804] T d16dfad0c3b540e5b4b86492ddf87208 P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:19:59.316888 16297 raft_consensus.cc:1275] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 13:19:59.317550 16292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a378d7f796f94615b3b347119a9880fd" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:19:59.343878 16789 leader_election.cc:304] T 26354361e9b044cea1c2b86c4882fe92 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:19:59.317353 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a96373bb0e14dce8a266b2fc6c58507" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:59.318444 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a96373bb0e14dce8a266b2fc6c58507" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:59.345150 16224 raft_consensus.cc:2468] T 5a96373bb0e14dce8a266b2fc6c58507 P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 4.
I20260709 13:19:59.323562 16296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fffab26427e24d398e2695aad40ca134" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:59.324753 16294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0eab25ce4c64e018a041beeda2867db" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:19:59.239411 18778 consensus_queue.cc:1048] T 829b4ad8c5a54eababeea1181b960d4b P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.091s
W20260709 13:19:59.325978 16791 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:59.335000 18785 raft_consensus.cc:493] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:59.320463 18734 leader_election.cc:290] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 8 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:19:59.346334 16294 raft_consensus.cc:2468] T c0eab25ce4c64e018a041beeda2867db P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 5.
W20260709 13:19:59.346588 16115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 44083us
I20260709 13:19:59.354435 16115 leader_election.cc:304] T e5ad4eb281564936a59f41e387a47156 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:19:59.300606 18759 raft_consensus.cc:740] T 8c11074579b046d99e585c2818e18a2c P 1ca2725d290e47d68b1a4b609c9816ac [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:19:59.327610 18724 raft_consensus.cc:515] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:59.350469 16293 raft_consensus.cc:2410] T 5a96373bb0e14dce8a266b2fc6c58507 P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac for term 5 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:19:59.351854 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee9ed67006144797b1de6ea8c2013a68" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:59.307410 18662 raft_consensus.cc:2749] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:19:59.331740 18599 leader_election.cc:290] T 2794af5b99984efebdee564035841cb0 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:19:59.359377 18633 consensus_queue.cc:1059] T e5ad4eb281564936a59f41e387a47156 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:19:59.366868 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26354361e9b044cea1c2b86c4882fe92" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:19:59.368285 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5ad4eb281564936a59f41e387a47156" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:19:59.359829 16142 raft_consensus.cc:1240] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 5. Current term is 7. Ops: []
I20260709 13:19:59.360370 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4f65cdf00b748bb963ee6e81ef43b79" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:59.370386 16217 raft_consensus.cc:2468] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 7.
I20260709 13:19:59.360472 16221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f5f89186204242bf377f7c42b2e2da" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:59.373934 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d4ab870a05347868956aa8b841b2981" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 6 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:59.306692 18735 raft_consensus.cc:493] T d2ab0ac8347641e6955e9bce7d90c1a2 P cb82cc7bd6db46d980c75237a1402860 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:19:59.361296 16220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d16dfad0c3b540e5b4b86492ddf87208" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
W20260709 13:19:59.363811 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 76451us
I20260709 13:19:59.375830 16145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e65676b21b748728ad39668cfe722ec" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:19:59.376322 16146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434bff3082d141838068f323bdb0d1ed" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:19:59.376657 16145 raft_consensus.cc:2468] T 9e65676b21b748728ad39668cfe722ec P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 6.
I20260709 13:19:59.368800 16149 raft_consensus.cc:2468] T 26354361e9b044cea1c2b86c4882fe92 P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 3.
I20260709 13:19:59.347864 18762 raft_consensus.cc:2749] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 1ca2725d290e47d68b1a4b609c9816ac [term 6 FOLLOWER]: Leader election lost for term 5. Reason: Vote denied by peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741) with higher term. Message: Invalid argument: T 87ac0d1f7e8a419f9bcbc04aee0e067e P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac for earlier term 5. Current term is 6.
I20260709 13:19:59.380843 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cc959fb7f704fba9009c0595467d074" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:19:59.386912 16225 raft_consensus.cc:3060] T 8cc959fb7f704fba9009c0595467d074 P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:19:59.384909 16220 raft_consensus.cc:3060] T d16dfad0c3b540e5b4b86492ddf87208 P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:19:59.383078 16146 raft_consensus.cc:2393] T 434bff3082d141838068f323bdb0d1ed P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 in current term 6: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
I20260709 13:19:59.389704 16219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fffab26427e24d398e2695aad40ca134" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:59.390177 16819 raft_consensus.cc:2368] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac for earlier term 5. Current term is 6.
I20260709 13:19:59.391222 16224 raft_consensus.cc:2393] T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 6: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
I20260709 13:19:59.390558 16219 raft_consensus.cc:2468] T fffab26427e24d398e2695aad40ca134 P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 4.
I20260709 13:19:59.352869 18785 raft_consensus.cc:3060] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:19:59.395668 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1df93779209748b9860f6d893dd3de53" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:19:59.390799 16222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a959dbecc2648eb85d1cc550f5a3245" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:19:59.396509 16222 raft_consensus.cc:3060] T 5a959dbecc2648eb85d1cc550f5a3245 P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:19:59.390486 16151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a959dbecc2648eb85d1cc550f5a3245" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:19:59.397581 16151 raft_consensus.cc:2393] T 5a959dbecc2648eb85d1cc550f5a3245 P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 5: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
W20260709 13:19:59.400638 16189 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:59.389170 16265 leader_election.cc:304] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
I20260709 13:19:59.375123 18749 raft_consensus.cc:493] T 1df93779209748b9860f6d893dd3de53 P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:59.380020 18735 raft_consensus.cc:3060] T d2ab0ac8347641e6955e9bce7d90c1a2 P cb82cc7bd6db46d980c75237a1402860 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:19:59.378808 18558 raft_consensus.cc:697] T d16dfad0c3b540e5b4b86492ddf87208 P 647a343a542d4a139fe106c19095129b [term 5 LEADER]: Becoming Leader. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:19:59.345093 18750 raft_consensus.cc:515] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:59.467027 16224 raft_consensus.cc:1275] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:19:59.388558 18599 raft_consensus.cc:2804] T e5ad4eb281564936a59f41e387a47156 P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:19:59.427529 18746 raft_consensus.cc:2804] T af57996a8be84d398f0f023e8d08ef77 P cb82cc7bd6db46d980c75237a1402860 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:19:59.478053 16192 leader_election.cc:400] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 5 election: Vote denied by peer cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923) with higher term. Message: Invalid argument: T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac for earlier term 5. Current term is 6.
I20260709 13:19:59.469177 16225 raft_consensus.cc:2468] T 8cc959fb7f704fba9009c0595467d074 P 1ca2725d290e47d68b1a4b609c9816ac [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 6.
I20260709 13:19:59.384399 18761 raft_consensus.cc:2804] T 26354361e9b044cea1c2b86c4882fe92 P cb82cc7bd6db46d980c75237a1402860 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:19:59.467262 16151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62fe3346cd3c42c4a55e14af9073bca4" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:19:59.470361 16145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2372981006d4b218215c28964dea623" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:19:59.484788 16145 raft_consensus.cc:2393] T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 5: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
W20260709 13:19:59.470847 16789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 106962us
I20260709 13:19:59.475406 18580 raft_consensus.cc:493] T 8a878999c0024d2abe517484180290fc P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:59.401295 18769 raft_consensus.cc:2804] T 23f82224f4564fea93572e8206bad1fe P cb82cc7bd6db46d980c75237a1402860 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:19:59.478451 16192 leader_election.cc:403] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 5 election: Cancelling election due to peer responding with higher term
W20260709 13:19:59.481197 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 99215.3us
I20260709 13:19:59.482318 16820 raft_consensus.cc:1240] T 8c11074579b046d99e585c2818e18a2c P cb82cc7bd6db46d980c75237a1402860 [term 6 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 5. Current term is 6. Ops: []
I20260709 13:19:59.486207 16789 leader_election.cc:304] T 8c11074579b046d99e585c2818e18a2c P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:19:59.491906 16149 raft_consensus.cc:1240] T 8a878999c0024d2abe517484180290fc P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 13:19:59.466909 18720 leader_election.cc:290] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 7 election: Requested vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:19:59.473209 18734 raft_consensus.cc:2749] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
W20260709 13:19:59.492333 16790 connection.cc:256] RPC call timeout handler was delayed by -1.334219234000102s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.244s for a response
I20260709 13:19:59.488083 18769 raft_consensus.cc:493] T 23f82224f4564fea93572e8206bad1fe P cb82cc7bd6db46d980c75237a1402860 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:19:59.499907 18769 raft_consensus.cc:3060] T 23f82224f4564fea93572e8206bad1fe P cb82cc7bd6db46d980c75237a1402860 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:19:59.504011 18789 log.cc:927] Time spent T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860: Append to log took a long time: real 0.102s	user 0.000s	sys 0.000s
I20260709 13:19:59.391992 18736 consensus_queue.cc:260] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:59.509349 18759 consensus_queue.cc:260] T 8c11074579b046d99e585c2818e18a2c P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:59.512904 18750 leader_election.cc:290] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 5 election: Requested vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:19:59.515343 16265 leader_election.cc:304] T e4dc6b0541ca431ebc757beac00521f1 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:19:59.486573 18599 raft_consensus.cc:697] T e5ad4eb281564936a59f41e387a47156 P 647a343a542d4a139fe106c19095129b [term 6 LEADER]: Becoming Leader. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:19:59.500012 18766 consensus_queue.cc:1048] T b87147731a68400e8cce9b879daa7c5d P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.157s
I20260709 13:19:59.500676 18776 consensus_queue.cc:1048] T 2794af5b99984efebdee564035841cb0 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.151s
I20260709 13:19:59.514321 16116 leader_election.cc:304] T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860
I20260709 13:19:59.507611 18558 consensus_queue.cc:237] T d16dfad0c3b540e5b4b86492ddf87208 P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:59.541229 16149 raft_consensus.cc:1240] T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 13:19:59.538380 18759 raft_consensus.cc:3060] T 8c11074579b046d99e585c2818e18a2c P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:19:59.542332 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4f65cdf00b748bb963ee6e81ef43b79" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:19:59.545025 16142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b7d1ab519044eda2d5e04a07242f25" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:19:59.543123 16149 raft_consensus.cc:2410] T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac for term 8 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 13:19:59.497161 18778 raft_consensus.cc:493] T 19baa7bc367844c288bca8f81439c413 P 1ca2725d290e47d68b1a4b609c9816ac [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:59.470988 18633 raft_consensus.cc:493] T 84f2da98864b4425bf7c0055951fdd4b P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
W20260709 13:19:59.518584 16790 connection.cc:256] RPC call timeout handler was delayed by -1.0917946470001425s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2916s for a response
W20260709 13:19:59.567034 16790 connection.cc:256] RPC call timeout handler was delayed by -1.0863946470001338s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2922s for a response
W20260709 13:19:59.521075 16191 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:59.513787 16147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66f75124215f4ca6bef831e174a792d8" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:19:59.529784 16145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c74490c40243a88e62d1760b5cd814" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:19:59.531589 16222 raft_consensus.cc:2468] T 5a959dbecc2648eb85d1cc550f5a3245 P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 5.
I20260709 13:19:59.488770 18746 raft_consensus.cc:697] T af57996a8be84d398f0f023e8d08ef77 P cb82cc7bd6db46d980c75237a1402860 [term 6 LEADER]: Becoming Leader. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
W20260709 13:19:59.535631 16192 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 57720.2us
I20260709 13:19:59.529808 16190 leader_election.cc:304] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860
I20260709 13:19:59.484632 18761 raft_consensus.cc:493] T 26354361e9b044cea1c2b86c4882fe92 P cb82cc7bd6db46d980c75237a1402860 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:59.523386 18662 raft_consensus.cc:2804] T 8c11074579b046d99e585c2818e18a2c P cb82cc7bd6db46d980c75237a1402860 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:19:59.538364 16151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c61a39fd4f84424b129c1a6f9ba404d" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
W20260709 13:19:59.538777 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 44359.6us
I20260709 13:19:59.539420 18777 raft_consensus.cc:2749] T e4dc6b0541ca431ebc757beac00521f1 P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:19:59.540000 18421 raft_consensus.cc:3055] T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:19:59.495478 18580 raft_consensus.cc:515] T 8a878999c0024d2abe517484180290fc P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:59.517195 16789 leader_election.cc:304] T 3884f864a96d428785895e5463e5728f P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 647a343a542d4a139fe106c19095129b
I20260709 13:19:59.577275 16151 raft_consensus.cc:3060] T 9c61a39fd4f84424b129c1a6f9ba404d P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:19:59.575135 16218 raft_consensus.cc:2468] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 1ca2725d290e47d68b1a4b609c9816ac [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 6.
I20260709 13:19:59.544246 18766 consensus_queue.cc:1243] T b87147731a68400e8cce9b879daa7c5d P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Peer 90f3da57796b488c9087a2f523adf045 log is divergent from this leader: its last log entry 6.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 13:19:59.556079 18762 raft_consensus.cc:3060] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:19:59.581869 18756 raft_consensus.cc:2749] T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:19:59.586161 18114 raft_consensus.cc:3055] T e5ad4eb281564936a59f41e387a47156 P 1ca2725d290e47d68b1a4b609c9816ac [term 5 LEADER]: Stepping down as leader of term 5
W20260709 13:19:59.619045 16789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 101957us
I20260709 13:19:59.589546 18580 leader_election.cc:290] T 8a878999c0024d2abe517484180290fc P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:19:59.592772 18587 raft_consensus.cc:3060] T e4dc6b0541ca431ebc757beac00521f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:19:59.594005 18743 consensus_meta.cc:287] T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045: Time spent flushing consensus metadata: real 0.522s	user 0.007s	sys 0.002s
I20260709 13:19:59.595402 18662 raft_consensus.cc:697] T 8c11074579b046d99e585c2818e18a2c P cb82cc7bd6db46d980c75237a1402860 [term 6 LEADER]: Becoming Leader. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:19:59.595052 18599 consensus_queue.cc:237] T e5ad4eb281564936a59f41e387a47156 P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:59.484247 18749 raft_consensus.cc:515] T 1df93779209748b9860f6d893dd3de53 P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:19:59.607527 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 77793.4us
W20260709 13:19:59.599737 16264 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:59.608419 16145 raft_consensus.cc:2468] T 08c74490c40243a88e62d1760b5cd814 P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 6.
I20260709 13:19:59.580309 18421 raft_consensus.cc:740] T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
W20260709 13:19:59.612824 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 125159us
I20260709 13:19:59.612874 16142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d4ab870a05347868956aa8b841b2981" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 6 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:19:59.567133 18674 raft_consensus.cc:515] T 66ae176961b74813a58bc15c9c4c6f2a P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:59.620834 16266 leader_election.cc:304] T 3884f864a96d428785895e5463e5728f P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 647a343a542d4a139fe106c19095129b
I20260709 13:19:59.621558 16117 leader_election.cc:304] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:19:59.596930 18736 raft_consensus.cc:3060] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:19:59.648432 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e4bfb9090c8460eb14fa7e0d818e7f1" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:19:59.672916 16297 raft_consensus.cc:2376] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 4. Re-sending same reply.
I20260709 13:19:59.579962 18761 raft_consensus.cc:3060] T 26354361e9b044cea1c2b86c4882fe92 P cb82cc7bd6db46d980c75237a1402860 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:19:59.657255 16142 raft_consensus.cc:3060] T 8d4ab870a05347868956aa8b841b2981 P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:19:59.651317 16790 leader_election.cc:304] T 3a7ca61592e848ea9b0fb81a2ce33cdc P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:19:59.658551 16149 raft_consensus.cc:1275] T 829b4ad8c5a54eababeea1181b960d4b P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 13:19:59.659418 16116 leader_election.cc:304] T c0eab25ce4c64e018a041beeda2867db P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:19:59.619735 18114 raft_consensus.cc:740] T e5ad4eb281564936a59f41e387a47156 P 1ca2725d290e47d68b1a4b609c9816ac [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:19:59.645810 18776 raft_consensus.cc:2804] T 3884f864a96d428785895e5463e5728f P cb82cc7bd6db46d980c75237a1402860 [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 13:19:59.670876 16117 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 49375us
I20260709 13:19:59.635689 18750 raft_consensus.cc:2749] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:19:59.606201 18633 raft_consensus.cc:515] T 84f2da98864b4425bf7c0055951fdd4b P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:59.568846 18778 raft_consensus.cc:515] T 19baa7bc367844c288bca8f81439c413 P 1ca2725d290e47d68b1a4b609c9816ac [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } }
I20260709 13:19:59.676975 16217 raft_consensus.cc:2468] T 1df93779209748b9860f6d893dd3de53 P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 5.
I20260709 13:19:59.644004 16817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8638ee363273426db1393dd81e5c7b72" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:19:59.693524 16147 raft_consensus.cc:2468] T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 6.
I20260709 13:19:59.674206 16296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdbd78acb4cf46298e0de8ba0d130e19" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:19:59.701061 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c2095a5f4ad49408dbc7f69c606120d" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
W20260709 13:19:59.701860 16266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 81128.9us
I20260709 13:19:59.705897 16266 leader_election.cc:304] T 31969392ccdb43b491911a319e8bc7e1 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
I20260709 13:19:59.679713 18745 raft_consensus.cc:2804] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:19:59.680796 18776 raft_consensus.cc:493] T 3884f864a96d428785895e5463e5728f P cb82cc7bd6db46d980c75237a1402860 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:59.715394 16296 raft_consensus.cc:2393] T cdbd78acb4cf46298e0de8ba0d130e19 P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 in current term 6: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
W20260709 13:19:59.716001 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 56664.6us
I20260709 13:19:59.730288 16116 leader_election.cc:304] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: cb82cc7bd6db46d980c75237a1402860
I20260709 13:19:59.681828 18743 raft_consensus.cc:2804] T 3884f864a96d428785895e5463e5728f P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:19:59.710680 16149 raft_consensus.cc:2393] T 7c2095a5f4ad49408dbc7f69c606120d P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 5: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
I20260709 13:19:59.727698 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8638ee363273426db1393dd81e5c7b72" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:19:59.731923 16297 raft_consensus.cc:2393] T 8638ee363273426db1393dd81e5c7b72 P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 5: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
I20260709 13:19:59.738582 16220 raft_consensus.cc:2468] T d16dfad0c3b540e5b4b86492ddf87208 P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 5.
I20260709 13:19:59.701203 18747 raft_consensus.cc:2749] T c0eab25ce4c64e018a041beeda2867db P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:19:59.740669 16293 raft_consensus.cc:1275] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 13:19:59.719470 18745 raft_consensus.cc:493] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:59.746686 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 5 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:19:59.747478 16265 leader_election.cc:304] T 2794af5b99984efebdee564035841cb0 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: cb82cc7bd6db46d980c75237a1402860
I20260709 13:19:59.717986 18724 leader_election.cc:290] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 6 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:19:59.754463 18770 raft_consensus.cc:493] T 68a6838d82ef4cac83507f906139d90f P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:59.736325 18743 raft_consensus.cc:493] T 3884f864a96d428785895e5463e5728f P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:59.749660 16149 raft_consensus.cc:2393] T ecc1e363230043fe9fef390f0b979f20 P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 5: Already voted for candidate cb82cc7bd6db46d980c75237a1402860 in this term.
I20260709 13:19:59.731927 18776 raft_consensus.cc:3060] T 3884f864a96d428785895e5463e5728f P cb82cc7bd6db46d980c75237a1402860 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:19:59.730293 18777 raft_consensus.cc:2749] T 31969392ccdb43b491911a319e8bc7e1 P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:19:59.744717 18766 raft_consensus.cc:493] T 57a744f5aaf944678a63469cbf514015 P cb82cc7bd6db46d980c75237a1402860 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:19:59.757375 16820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66ae176961b74813a58bc15c9c4c6f2a" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:19:59.749719 16144 raft_consensus.cc:1240] T 8158bbd17b994d2eb96f560bd0f1b88d P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 5. Current term is 6. Ops: [5.4-5.4]
I20260709 13:19:59.764686 16266 leader_election.cc:304] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:19:59.766125 16824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7ca61592e848ea9b0fb81a2ce33cdc" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:19:59.751864 18745 raft_consensus.cc:3060] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:19:59.771387 16189 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:59.780776 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806d7690c2b644959152074bb2cab0a7" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:19:59.783448 16818 raft_consensus.cc:1275] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860 [term 5 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
W20260709 13:19:59.791563 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 147001us
I20260709 13:19:59.792049 16825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57a744f5aaf944678a63469cbf514015" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:19:59.758348 18743 raft_consensus.cc:3060] T 3884f864a96d428785895e5463e5728f P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:19:59.794106 16190 leader_election.cc:304] T c0eab25ce4c64e018a041beeda2867db P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:19:59.796571 16790 leader_election.cc:304] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
W20260709 13:19:59.796681 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 66489.5us
I20260709 13:19:59.797453 16265 leader_election.cc:304] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:19:59.683411 18746 consensus_queue.cc:237] T af57996a8be84d398f0f023e8d08ef77 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:59.809021 16824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3884f864a96d428785895e5463e5728f" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:19:59.810614 16144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5718c623ebee4d2bb770f0f9ddb2f164" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:19:59.813982 16116 leader_election.cc:304] T 7c2095a5f4ad49408dbc7f69c606120d P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 90f3da57796b488c9087a2f523adf045
I20260709 13:19:59.780038 18724 raft_consensus.cc:2749] T 3a7ca61592e848ea9b0fb81a2ce33cdc P cb82cc7bd6db46d980c75237a1402860 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:19:59.746409 18421 consensus_queue.cc:260] T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:59.800751 18777 raft_consensus.cc:2804] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:19:59.807377 18580 raft_consensus.cc:2804] T 2794af5b99984efebdee564035841cb0 P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:19:59.760905 18770 raft_consensus.cc:515] T 68a6838d82ef4cac83507f906139d90f P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:59.805918 18758 raft_consensus.cc:2804] T c0eab25ce4c64e018a041beeda2867db P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:19:59.833689 16818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a9de347c711477e87c582d003f36925" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:19:59.757515 18662 consensus_queue.cc:237] T 8c11074579b046d99e585c2818e18a2c P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:19:59.837635 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 50990.2us
I20260709 13:19:59.837991 16116 leader_election.cc:304] T 5a96373bb0e14dce8a266b2fc6c58507 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 90f3da57796b488c9087a2f523adf045
I20260709 13:19:59.764010 18114 consensus_queue.cc:260] T e5ad4eb281564936a59f41e387a47156 P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.5, Last appended by leader: 5, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:19:59.800217 16816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee9ed67006144797b1de6ea8c2013a68" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:19:59.846276 16144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8158bbd17b994d2eb96f560bd0f1b88d" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:19:59.860054 16116 leader_election.cc:304] T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 90f3da57796b488c9087a2f523adf045
I20260709 13:19:59.857368 16144 raft_consensus.cc:2468] T 8158bbd17b994d2eb96f560bd0f1b88d P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 6.
I20260709 13:19:59.780997 18766 raft_consensus.cc:515] T 57a744f5aaf944678a63469cbf514015 P cb82cc7bd6db46d980c75237a1402860 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:59.867367 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0eab25ce4c64e018a041beeda2867db" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:19:59.868000 16150 raft_consensus.cc:2468] T c0eab25ce4c64e018a041beeda2867db P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 5.
I20260709 13:19:59.778477 16220 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "02838c86531d4f98bf58d1c22b8baf77"
dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
mode: GRACEFUL
new_leader_uuid: "90f3da57796b488c9087a2f523adf045"
 from {username='slave'} at 127.0.0.1:39144
I20260709 13:19:59.869189 16220 raft_consensus.cc:606] T 02838c86531d4f98bf58d1c22b8baf77 P 1ca2725d290e47d68b1a4b609c9816ac [term 6 FOLLOWER]: Received request to transfer leadership to TS 90f3da57796b488c9087a2f523adf045
I20260709 13:19:59.869496 16220 raft_consensus.cc:614] T 02838c86531d4f98bf58d1c22b8baf77 P 1ca2725d290e47d68b1a4b609c9816ac [term 6 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:19:59.838435 18558 raft_consensus.cc:2804] T 7c2095a5f4ad49408dbc7f69c606120d P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:19:59.838706 18758 raft_consensus.cc:493] T c0eab25ce4c64e018a041beeda2867db P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:59.843711 18778 leader_election.cc:290] T 19baa7bc367844c288bca8f81439c413 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:59.870162 18747 raft_consensus.cc:2804] T 5a96373bb0e14dce8a266b2fc6c58507 P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:19:59.830346 18781 consensus_queue.cc:1048] T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.044s
I20260709 13:19:59.833206 18756 consensus_queue.cc:1059] T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.048s
I20260709 13:19:59.924077 16825 raft_consensus.cc:3060] T 57a744f5aaf944678a63469cbf514015 P cb82cc7bd6db46d980c75237a1402860 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:19:59.867174 18347 raft_consensus.cc:2749] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:19:59.928232 16816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e9b2bf7011f4ae688a79d7bb5a447b4" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:19:59.962733 16816 raft_consensus.cc:2393] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 4: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
I20260709 13:19:59.963650 18347 consensus_queue.cc:1059] T 7c2095a5f4ad49408dbc7f69c606120d P 90f3da57796b488c9087a2f523adf045 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: INVALID_TERM, Last received: 4.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:19:59.963935 16816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434bff3082d141838068f323bdb0d1ed" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:19:59.964677 18347 raft_consensus.cc:3055] T 7c2095a5f4ad49408dbc7f69c606120d P 90f3da57796b488c9087a2f523adf045 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:19:59.965147 18347 raft_consensus.cc:740] T 7c2095a5f4ad49408dbc7f69c606120d P 90f3da57796b488c9087a2f523adf045 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:19:59.942373 18772 consensus_queue.cc:1048] T c2f5f89186204242bf377f7c42b2e2da P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.134s
W20260709 13:19:59.944334 16788 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:59.874893 18769 raft_consensus.cc:515] T 23f82224f4564fea93572e8206bad1fe P cb82cc7bd6db46d980c75237a1402860 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:19:59.969342 18769 leader_election.cc:290] T 23f82224f4564fea93572e8206bad1fe P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 4 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:59.945864 16789 leader_election.cc:304] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:19:59.974035 16144 raft_consensus.cc:1275] T 746aabf856ab401c9ef5e859371e7b67 P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
W20260709 13:19:59.975980 18801 log.cc:927] Time spent T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 0.153s	user 0.000s	sys 0.001s
I20260709 13:19:59.949503 16142 raft_consensus.cc:2468] T 8d4ab870a05347868956aa8b841b2981 P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 6.
I20260709 13:19:59.954146 16151 raft_consensus.cc:2410] T 9c61a39fd4f84424b129c1a6f9ba404d P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac for term 6 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 2.
W20260709 13:19:59.921545 18753 consensus_meta.cc:287] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 1ca2725d290e47d68b1a4b609c9816ac: Time spent flushing consensus metadata: real 0.621s	user 0.009s	sys 0.003s
I20260709 13:19:59.958868 18746 raft_consensus.cc:493] T ada5a004ba7c46819051e3ccec71ffdb P cb82cc7bd6db46d980c75237a1402860 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:59.964833 16816 raft_consensus.cc:2393] T 434bff3082d141838068f323bdb0d1ed P cb82cc7bd6db46d980c75237a1402860 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 6: Already voted for candidate cb82cc7bd6db46d980c75237a1402860 in this term.
W20260709 13:19:59.941809 16263 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:59.890589 18758 raft_consensus.cc:3060] T c0eab25ce4c64e018a041beeda2867db P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:19:59.920838 18114 raft_consensus.cc:3060] T e5ad4eb281564936a59f41e387a47156 P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:19:59.986029 16266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 232707us
I20260709 13:19:59.970893 18633 leader_election.cc:290] T 84f2da98864b4425bf7c0055951fdd4b P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
W20260709 13:19:59.993185 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 69275us
I20260709 13:19:59.946585 18747 raft_consensus.cc:493] T 5a96373bb0e14dce8a266b2fc6c58507 P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:19:59.998955 16191 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:00.001283 16151 raft_consensus.cc:1275] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 4. (term mismatch)
I20260709 13:19:59.956759 18778 raft_consensus.cc:493] T ee9ed67006144797b1de6ea8c2013a68 P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:19:59.866952 18674 leader_election.cc:290] T 66ae176961b74813a58bc15c9c4c6f2a P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 7 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:00.001816 16151 pending_rounds.cc:85] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b: Aborting all ops after (but not including) 3
I20260709 13:20:00.010509 16151 pending_rounds.cc:107] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b: Aborting uncommitted NO_OP operation due to leader change: 6.4
I20260709 13:19:59.987282 18769 raft_consensus.cc:2804] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P cb82cc7bd6db46d980c75237a1402860 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:20:00.010929 16151 raft_consensus.cc:2889] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:19:59.883257 18799 log.cc:927] Time spent T 02838c86531d4f98bf58d1c22b8baf77 P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.202s	user 0.001s	sys 0.001s
I20260709 13:19:59.889376 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fffab26427e24d398e2695aad40ca134" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:19:59.957178 18749 leader_election.cc:290] T 1df93779209748b9860f6d893dd3de53 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:59.884559 18808 raft_consensus.cc:2749] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:20:00.000872 18747 raft_consensus.cc:3060] T 5a96373bb0e14dce8a266b2fc6c58507 P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:20:00.019469 16150 raft_consensus.cc:2410] T fffab26427e24d398e2695aad40ca134 P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac for term 5 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
W20260709 13:20:00.011312 18804 log.cc:927] Time spent T ee9ed67006144797b1de6ea8c2013a68 P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 0.254s	user 0.001s	sys 0.000s
W20260709 13:20:00.035395 16264 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:20:00.057082 18785 consensus_meta.cc:287] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860: Time spent flushing consensus metadata: real 0.626s	user 0.005s	sys 0.005s
I20260709 13:19:59.996480 18746 raft_consensus.cc:515] T ada5a004ba7c46819051e3ccec71ffdb P cb82cc7bd6db46d980c75237a1402860 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:00.061532 18746 leader_election.cc:290] T ada5a004ba7c46819051e3ccec71ffdb P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:19:59.879295 18805 consensus_queue.cc:1048] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.191s
I20260709 13:20:00.011766 18778 raft_consensus.cc:515] T ee9ed67006144797b1de6ea8c2013a68 P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:00.066433 16816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2794af5b99984efebdee564035841cb0" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:00.000850 18347 consensus_queue.cc:260] T 7c2095a5f4ad49408dbc7f69c606120d P 90f3da57796b488c9087a2f523adf045 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:00.068238 18347 raft_consensus.cc:3060] T 7c2095a5f4ad49408dbc7f69c606120d P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:20:00.031616 18769 raft_consensus.cc:697] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P cb82cc7bd6db46d980c75237a1402860 [term 4 LEADER]: Becoming Leader. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:20:00.072247 18769 consensus_queue.cc:237] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:00.072992 18778 leader_election.cc:290] T ee9ed67006144797b1de6ea8c2013a68 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
W20260709 13:20:00.088053 18810 log.cc:927] Time spent T 829b4ad8c5a54eababeea1181b960d4b P cb82cc7bd6db46d980c75237a1402860: Append to log took a long time: real 0.070s	user 0.000s	sys 0.000s
I20260709 13:20:00.089480 18770 leader_election.cc:290] T 68a6838d82ef4cac83507f906139d90f P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:00.091704 18762 raft_consensus.cc:2749] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [term 6 FOLLOWER]: Leader election lost for term 5. Reason: Vote denied by peer cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923) with higher term. Message: Invalid argument: T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac for earlier term 5. Current term is 6.
W20260709 13:20:00.095086 18735 consensus_meta.cc:287] T d2ab0ac8347641e6955e9bce7d90c1a2 P cb82cc7bd6db46d980c75237a1402860: Time spent flushing consensus metadata: real 0.614s	user 0.007s	sys 0.002s
I20260709 13:20:00.095477 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a96373bb0e14dce8a266b2fc6c58507" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:00.095615 18735 raft_consensus.cc:515] T d2ab0ac8347641e6955e9bce7d90c1a2 P cb82cc7bd6db46d980c75237a1402860 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:00.097590 16819 raft_consensus.cc:1240] T d2ab0ac8347641e6955e9bce7d90c1a2 P cb82cc7bd6db46d980c75237a1402860 [term 6 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 5. Current term is 6. Ops: []
I20260709 13:20:00.097663 16823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee9ed67006144797b1de6ea8c2013a68" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:00.099555 16823 raft_consensus.cc:1240] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860 [term 5 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 4. Current term is 5. Ops: []
I20260709 13:20:00.103070 18812 consensus_queue.cc:1048] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:20:00.110003 16825 raft_consensus.cc:2468] T 57a744f5aaf944678a63469cbf514015 P cb82cc7bd6db46d980c75237a1402860 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 4.
I20260709 13:20:00.041109 18753 raft_consensus.cc:493] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:00.127051 16145 raft_consensus.cc:1240] T e5ad4eb281564936a59f41e387a47156 P 647a343a542d4a139fe106c19095129b [term 6 LEADER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 5. Current term is 6. Ops: []
I20260709 13:20:00.128479 16823 raft_consensus.cc:1240] T 23f82224f4564fea93572e8206bad1fe P cb82cc7bd6db46d980c75237a1402860 [term 4 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 3. Current term is 4. Ops: [3.5-3.5]
I20260709 13:20:00.100380 18749 raft_consensus.cc:493] T 02838c86531d4f98bf58d1c22b8baf77 P 1ca2725d290e47d68b1a4b609c9816ac [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:20:00.136739 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 340291us
I20260709 13:20:00.057693 18785 raft_consensus.cc:515] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:00.137727 18558 raft_consensus.cc:697] T 7c2095a5f4ad49408dbc7f69c606120d P 647a343a542d4a139fe106c19095129b [term 5 LEADER]: Becoming Leader. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:20:00.163105 16148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c74490c40243a88e62d1760b5cd814" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
W20260709 13:20:00.194041 16189 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:20:00.195832 16189 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:00.125229 18766 leader_election.cc:290] T 57a744f5aaf944678a63469cbf514015 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:00.126892 18777 raft_consensus.cc:697] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 90f3da57796b488c9087a2f523adf045 [term 4 LEADER]: Becoming Leader. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:20:00.117882 18674 consensus_queue.cc:1048] T 9e65676b21b748728ad39668cfe722ec P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.043s
I20260709 13:20:00.141000 18599 raft_consensus.cc:2804] T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:20:00.095077 18743 raft_consensus.cc:515] T 3884f864a96d428785895e5463e5728f P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:20:00.212363 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 69114.3us
I20260709 13:20:00.213656 16790 leader_election.cc:304] T 66f75124215f4ca6bef831e174a792d8 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
W20260709 13:20:00.198674 16143 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.617s	user 0.002s	sys 0.014s
I20260709 13:20:00.204497 16823 raft_consensus.cc:1275] T 66f75124215f4ca6bef831e174a792d8 P cb82cc7bd6db46d980c75237a1402860 [term 6 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:20:00.204813 16820 raft_consensus.cc:2368] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 for earlier term 7. Current term is 8.
I20260709 13:20:00.195458 16148 raft_consensus.cc:2376] T 08c74490c40243a88e62d1760b5cd814 P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 6. Re-sending same reply.
W20260709 13:20:00.204525 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 344561us
I20260709 13:20:00.206491 16151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2372981006d4b218215c28964dea623" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:20:00.217170 16151 raft_consensus.cc:2393] T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 5: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
I20260709 13:20:00.218278 16116 leader_election.cc:304] T ee9ed67006144797b1de6ea8c2013a68 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860
I20260709 13:20:00.220366 16116 leader_election.cc:304] T fffab26427e24d398e2695aad40ca134 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
W20260709 13:20:00.222009 16143 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.653s	user 0.003s	sys 0.014s
I20260709 13:20:00.139277 18753 raft_consensus.cc:515] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:00.194180 16817 raft_consensus.cc:2468] T 8638ee363273426db1393dd81e5c7b72 P cb82cc7bd6db46d980c75237a1402860 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 5.
I20260709 13:20:00.240844 16144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c61a39fd4f84424b129c1a6f9ba404d" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:20:00.282212 16144 raft_consensus.cc:2410] T 9c61a39fd4f84424b129c1a6f9ba404d P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac for term 6 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 2.
I20260709 13:20:00.209604 18599 raft_consensus.cc:493] T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Starting leader election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:20:00.285012 18599 raft_consensus.cc:3060] T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:20:00.243965 18761 consensus_meta.cc:287] T 26354361e9b044cea1c2b86c4882fe92 P cb82cc7bd6db46d980c75237a1402860: Time spent flushing consensus metadata: real 0.560s	user 0.007s	sys 0.004s
I20260709 13:20:00.288367 18761 raft_consensus.cc:515] T 26354361e9b044cea1c2b86c4882fe92 P cb82cc7bd6db46d980c75237a1402860 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:20:00.290102 18803 log.cc:927] Time spent T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 0.492s	user 0.002s	sys 0.002s
I20260709 13:20:00.248054 18558 consensus_queue.cc:237] T 7c2095a5f4ad49408dbc7f69c606120d P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 4.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:00.249848 18766 raft_consensus.cc:2749] T 66f75124215f4ca6bef831e174a792d8 P cb82cc7bd6db46d980c75237a1402860 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:20:00.252802 18720 raft_consensus.cc:2749] T ee9ed67006144797b1de6ea8c2013a68 P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:20:00.261476 18580 raft_consensus.cc:493] T 2794af5b99984efebdee564035841cb0 P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Starting leader election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:20:00.263427 16218 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57604 (request call id 2083) took 1058 ms. Trace:
I20260709 13:20:00.266980 18807 raft_consensus.cc:493] T 18ad7e3af5504571a69d10c00750a01c P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:00.310614 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada5a004ba7c46819051e3ccec71ffdb" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:00.275328 18812 consensus_queue.cc:1048] T 9e65676b21b748728ad39668cfe722ec P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:20:00.277493 18735 leader_election.cc:290] T d2ab0ac8347641e6955e9bce7d90c1a2 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 6 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
W20260709 13:20:00.290921 16266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 289491us
I20260709 13:20:00.145315 18749 raft_consensus.cc:515] T 02838c86531d4f98bf58d1c22b8baf77 P 1ca2725d290e47d68b1a4b609c9816ac [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:20:00.298385 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 258464us
I20260709 13:20:00.305675 16226 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "08c74490c40243a88e62d1760b5cd814"
dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
mode: GRACEFUL
new_leader_uuid: "90f3da57796b488c9087a2f523adf045"
 from {username='slave'} at 127.0.0.1:39144
I20260709 13:20:00.306195 16218 rpcz_store.cc:276] 0709 13:19:59.204739 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:59.205073 (+   334us) service_pool.cc:224] Handling call
0709 13:20:00.263181 (+1058108us) spinlock_profiling.cc:243] Waited 149 ms on lock 0x7b54003d2054. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:20:00.263337 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":343261056}
I20260709 13:20:00.323086 16226 raft_consensus.cc:606] T 08c74490c40243a88e62d1760b5cd814 P 1ca2725d290e47d68b1a4b609c9816ac [term 6 LEADER]: Received request to transfer leadership to TS 90f3da57796b488c9087a2f523adf045
I20260709 13:20:00.333410 16151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62fe3346cd3c42c4a55e14af9073bca4" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:00.333608 16144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4f65cdf00b748bb963ee6e81ef43b79" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
W20260709 13:20:00.306532 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 92998.3us
I20260709 13:20:00.306885 16823 raft_consensus.cc:1275] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 [term 4 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260709 13:20:00.314915 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f82224f4564fea93572e8206bad1fe" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:20:00.335866 16293 raft_consensus.cc:3060] T 23f82224f4564fea93572e8206bad1fe P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:20:00.315716 16296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57a744f5aaf944678a63469cbf514015" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:00.279260 18768 raft_consensus.cc:2804] T fffab26427e24d398e2695aad40ca134 P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:20:00.336855 16296 raft_consensus.cc:2393] T 57a744f5aaf944678a63469cbf514015 P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 in current term 4: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
I20260709 13:20:00.314352 16294 pending_rounds.cc:85] T b87147731a68400e8cce9b879daa7c5d P 90f3da57796b488c9087a2f523adf045: Aborting all ops after (but not including) 5
I20260709 13:20:00.338119 16294 pending_rounds.cc:107] T b87147731a68400e8cce9b879daa7c5d P 90f3da57796b488c9087a2f523adf045: Aborting uncommitted NO_OP operation due to leader change: 6.6
I20260709 13:20:00.338516 16294 raft_consensus.cc:2889] T b87147731a68400e8cce9b879daa7c5d P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:20:00.317309 16297 raft_consensus.cc:2468] T ada5a004ba7c46819051e3ccec71ffdb P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 6.
I20260709 13:20:00.317349 16292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ab0ac8347641e6955e9bce7d90c1a2" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:20:00.321121 16146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d16dfad0c3b540e5b4b86492ddf87208" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:20:00.347448 16789 leader_election.cc:304] T ada5a004ba7c46819051e3ccec71ffdb P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 
W20260709 13:20:00.359510 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 139267us
I20260709 13:20:00.360654 16116 leader_election.cc:304] T 8d4ab870a05347868956aa8b841b2981 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:20:00.310434 18580 raft_consensus.cc:3060] T 2794af5b99984efebdee564035841cb0 P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:20:00.376936 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 54852.4us
I20260709 13:20:00.329763 18761 leader_election.cc:290] T 26354361e9b044cea1c2b86c4882fe92 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 4 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:00.380678 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26354361e9b044cea1c2b86c4882fe92" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:20:00.382391 16823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66ae176961b74813a58bc15c9c4c6f2a" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:20:00.381336 16299 raft_consensus.cc:3060] T 26354361e9b044cea1c2b86c4882fe92 P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:20:00.400259 16146 raft_consensus.cc:1275] T 08c74490c40243a88e62d1760b5cd814 P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 13:20:00.400478 16148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f2da98864b4425bf7c0055951fdd4b" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:00.300436 18758 raft_consensus.cc:515] T c0eab25ce4c64e018a041beeda2867db P 1ca2725d290e47d68b1a4b609c9816ac [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:00.353053 18720 raft_consensus.cc:493] T 9c61a39fd4f84424b129c1a6f9ba404d P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:20:00.403213 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 67305.2us
I20260709 13:20:00.351506 18768 raft_consensus.cc:493] T fffab26427e24d398e2695aad40ca134 P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:00.398437 18766 raft_consensus.cc:2804] T ada5a004ba7c46819051e3ccec71ffdb P cb82cc7bd6db46d980c75237a1402860 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:20:00.318193 18807 raft_consensus.cc:515] T 18ad7e3af5504571a69d10c00750a01c P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } }
W20260709 13:20:00.435750 18776 consensus_meta.cc:287] T 3884f864a96d428785895e5463e5728f P cb82cc7bd6db46d980c75237a1402860: Time spent flushing consensus metadata: real 0.674s	user 0.002s	sys 0.009s
I20260709 13:20:00.441852 18766 raft_consensus.cc:493] T ada5a004ba7c46819051e3ccec71ffdb P cb82cc7bd6db46d980c75237a1402860 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:00.435289 18768 raft_consensus.cc:3060] T fffab26427e24d398e2695aad40ca134 P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:20:00.508059 18822 log.cc:927] Time spent T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 0.072s	user 0.000s	sys 0.000s
I20260709 13:20:00.624617 16143 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46224 (request call id 357) took 1267 ms. Trace:
I20260709 13:20:00.460857 18766 raft_consensus.cc:3060] T ada5a004ba7c46819051e3ccec71ffdb P cb82cc7bd6db46d980c75237a1402860 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:20:00.587466 16788 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:20:00.613889 18587 consensus_meta.cc:287] T e4dc6b0541ca431ebc757beac00521f1 P 1ca2725d290e47d68b1a4b609c9816ac: Time spent flushing consensus metadata: real 0.963s	user 0.013s	sys 0.004s
I20260709 13:20:00.614491 18770 raft_consensus.cc:493] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:00.626215 18770 raft_consensus.cc:515] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:00.627796 16292 raft_consensus.cc:1275] T 8c11074579b046d99e585c2818e18a2c P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 13:20:00.628646 18770 leader_election.cc:290] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:00.629081 16293 raft_consensus.cc:2468] T 23f82224f4564fea93572e8206bad1fe P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 4.
W20260709 13:20:00.615104 16189 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:20:00.640002 16189 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:20:00.640578 16189 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:00.498646 18816 raft_consensus.cc:493] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:00.641430 18816 raft_consensus.cc:515] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:00.643918 18816 leader_election.cc:290] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:00.644691 18837 raft_consensus.cc:493] T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:00.645285 18837 raft_consensus.cc:515] T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:00.647475 18837 leader_election.cc:290] T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
W20260709 13:20:00.648524 16191 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:00.403811 18750 consensus_queue.cc:1048] T 08c74490c40243a88e62d1760b5cd814 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.051s
I20260709 13:20:00.642856 18812 raft_consensus.cc:493] T cdbd78acb4cf46298e0de8ba0d130e19 P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:00.494558 18821 consensus_queue.cc:1059] T 23f82224f4564fea93572e8206bad1fe P 647a343a542d4a139fe106c19095129b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.114s
I20260709 13:20:00.615691 18825 raft_consensus.cc:493] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:20:00.618084 18747 consensus_meta.cc:287] T 5a96373bb0e14dce8a266b2fc6c58507 P 647a343a542d4a139fe106c19095129b: Time spent flushing consensus metadata: real 0.528s	user 0.006s	sys 0.013s
I20260709 13:20:00.618127 18614 raft_consensus.cc:2749] T 8d4ab870a05347868956aa8b841b2981 P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:20:00.619064 18753 leader_election.cc:290] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:00.619211 18743 leader_election.cc:290] T 3884f864a96d428785895e5463e5728f P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 5 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
W20260709 13:20:00.619493 16266 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:00.673578 18836 raft_consensus.cc:493] T 746aabf856ab401c9ef5e859371e7b67 P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:00.620118 18832 consensus_queue.cc:1048] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:20:00.714622 16192 leader_election.cc:304] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 90f3da57796b488c9087a2f523adf045
I20260709 13:20:00.442504 18777 consensus_queue.cc:237] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:20:00.621379 16217 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57604 (request call id 2097) took 1262 ms (client timeout 1659 ms). Trace:
I20260709 13:20:00.633996 18720 raft_consensus.cc:515] T 9c61a39fd4f84424b129c1a6f9ba404d P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:00.622570 16790 leader_election.cc:304] T 74ca8140f35c4ae29e8a664f6bdfc829 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:20:00.623342 18755 raft_consensus.cc:2804] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:20:00.623807 18824 consensus_queue.cc:1048] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:20:00.624032 18758 leader_election.cc:290] T c0eab25ce4c64e018a041beeda2867db P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 6 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:00.460510 18776 raft_consensus.cc:515] T 3884f864a96d428785895e5463e5728f P cb82cc7bd6db46d980c75237a1402860 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:00.629508 18797 raft_consensus.cc:493] T e4dc6b0541ca431ebc757beac00521f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:00.629707 18761 consensus_queue.cc:1048] T 8c11074579b046d99e585c2818e18a2c P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:20:00.632972 18770 raft_consensus.cc:493] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:00.633193 18587 raft_consensus.cc:493] T 8a878999c0024d2abe517484180290fc P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:00.633036 16146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:00.634061 16148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a959dbecc2648eb85d1cc550f5a3245" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:00.613885 16116 leader_election.cc:304] T 8cc959fb7f704fba9009c0595467d074 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
I20260709 13:20:00.750407 16148 raft_consensus.cc:2393] T 5a959dbecc2648eb85d1cc550f5a3245 P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 5: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
I20260709 13:20:00.650749 16299 raft_consensus.cc:2468] T 26354361e9b044cea1c2b86c4882fe92 P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 4.
W20260709 13:20:00.615844 18347 consensus_meta.cc:287] T 7c2095a5f4ad49408dbc7f69c606120d P 90f3da57796b488c9087a2f523adf045: Time spent flushing consensus metadata: real 0.547s	user 0.007s	sys 0.007s
I20260709 13:20:00.669044 16151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b7d1ab519044eda2d5e04a07242f25" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:20:00.675338 18833 raft_consensus.cc:493] T 8a878999c0024d2abe517484180290fc P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:00.721614 18836 raft_consensus.cc:515] T 746aabf856ab401c9ef5e859371e7b67 P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:00.759358 18833 raft_consensus.cc:515] T 8a878999c0024d2abe517484180290fc P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:00.631966 16143 rpcz_store.cc:276] 0709 13:19:59.357330 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:59.361154 (+  3824us) service_pool.cc:224] Handling call
0709 13:19:59.380860 (+ 19706us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:59.381144 (+   284us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:19:59.569105 (+187961us) log.cc:844] Serialized 14 byte log entry
0709 13:20:00.435360 (+866255us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:20:00.623076 (+187716us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:20:00.623133 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:00.624407 (+  1274us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:00.624557 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":157817,"prepare.run_cpu_time_us":25,"prepare.run_wall_time_us":26,"raft.queue_time_us":63903,"raft.run_cpu_time_us":96,"raft.run_wall_time_us":96,"thread_start_us":1033746,"threads_started":3,"wal-append.queue_time_us":751037}
I20260709 13:20:00.761603 18836 leader_election.cc:290] T 746aabf856ab401c9ef5e859371e7b67 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:00.726001 18753 consensus_queue.cc:1048] T 66f75124215f4ca6bef831e174a792d8 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.036s
I20260709 13:20:00.762452 16143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66f75124215f4ca6bef831e174a792d8" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
W20260709 13:20:00.660835 18745 consensus_meta.cc:287] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b: Time spent flushing consensus metadata: real 0.860s	user 0.015s	sys 0.004s
I20260709 13:20:00.763023 16143 raft_consensus.cc:2376] T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 6. Re-sending same reply.
I20260709 13:20:00.699806 16147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1df93779209748b9860f6d893dd3de53" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:00.763566 18745 raft_consensus.cc:515] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:00.763993 16147 raft_consensus.cc:2468] T 1df93779209748b9860f6d893dd3de53 P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 5.
I20260709 13:20:00.705230 16149 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1df93779209748b9860f6d893dd3de53"
dest_uuid: "647a343a542d4a139fe106c19095129b"
mode: GRACEFUL
new_leader_uuid: "90f3da57796b488c9087a2f523adf045"
 from {username='slave'} at 127.0.0.1:40290
W20260709 13:20:00.704768 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 336653us
I20260709 13:20:00.764828 16149 raft_consensus.cc:606] T 1df93779209748b9860f6d893dd3de53 P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Received request to transfer leadership to TS 90f3da57796b488c9087a2f523adf045
W20260709 13:20:00.620693 18736 consensus_meta.cc:287] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860: Time spent flushing consensus metadata: real 0.928s	user 0.020s	sys 0.010s
I20260709 13:20:00.698873 18825 raft_consensus.cc:515] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:00.765784 16818 raft_consensus.cc:3060] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:20:00.765906 18797 raft_consensus.cc:515] T e4dc6b0541ca431ebc757beac00521f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:00.767369 16142 raft_consensus.cc:1240] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 6. Current term is 7. Ops: []
I20260709 13:20:00.720399 16821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87147731a68400e8cce9b879daa7c5d" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 8 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:00.768378 16821 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49896 (request call id 888) took 1011 ms. Trace:
I20260709 13:20:00.768735 16821 rpcz_store.cc:276] 0709 13:19:59.758527 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:59.759555 (+  1028us) service_pool.cc:224] Handling call
0709 13:20:00.720223 (+960668us) spinlock_profiling.cc:243] Waited 5.29 ms on lock 0x7b5800004298. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69ad8fadb1 00007f69ada3ab71 00007f69ada3aa22 00007f69b6953c09 00007f69b5aadee0 00007f69b5b770ab 00007f69b5b77131 00007f69b5b250a0 00007f69b5b24984 00007f69af09f6d4 00007f69af09f662
0709 13:20:00.768316 (+ 48093us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":12167936}
I20260709 13:20:00.770105 16821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e4bfb9090c8460eb14fa7e0d818e7f1" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
W20260709 13:20:00.721184 16217 rpcz_store.cc:269] 0709 13:19:59.358823 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:59.380241 (+ 21418us) service_pool.cc:224] Handling call
0709 13:20:00.407517 (+1027276us) spinlock_profiling.cc:243] Waited 281 ms on lock 0x7b5400246654. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:20:00.621310 (+213793us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":645239680}
I20260709 13:20:00.621713 18834 raft_consensus.cc:493] T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:00.771875 18834 raft_consensus.cc:515] T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:00.723939 16292 raft_consensus.cc:1240] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 7. Current term is 8. Ops: []
I20260709 13:20:00.723513 16295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f2da98864b4425bf7c0055951fdd4b" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:00.727396 18114 raft_consensus.cc:493] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:00.677223 18812 raft_consensus.cc:515] T cdbd78acb4cf46298e0de8ba0d130e19 P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:00.727676 18750 consensus_queue.cc:1048] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:20:00.728677 18580 raft_consensus.cc:515] T 2794af5b99984efebdee564035841cb0 P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:00.729346 18807 leader_election.cc:290] T 18ad7e3af5504571a69d10c00750a01c P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:00.701556 18747 raft_consensus.cc:515] T 5a96373bb0e14dce8a266b2fc6c58507 P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:00.730006 16296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1df93779209748b9860f6d893dd3de53" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:00.730257 18749 leader_election.cc:290] T 02838c86531d4f98bf58d1c22b8baf77 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:00.730736 18816 raft_consensus.cc:2804] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:20:00.731031 18766 raft_consensus.cc:515] T ada5a004ba7c46819051e3ccec71ffdb P cb82cc7bd6db46d980c75237a1402860 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:00.732402 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:00.733698 16293 raft_consensus.cc:1275] T ada5a004ba7c46819051e3ccec71ffdb P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
W20260709 13:20:00.746188 16117 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 110195us
I20260709 13:20:00.747459 16291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68a6838d82ef4cac83507f906139d90f" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:00.708942 18785 leader_election.cc:290] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 8 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
W20260709 13:20:00.750670 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 348270us
I20260709 13:20:00.753134 18828 raft_consensus.cc:493] T 26354361e9b044cea1c2b86c4882fe92 P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:00.765231 16149 raft_consensus.cc:614] T 1df93779209748b9860f6d893dd3de53 P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:20:00.773155 16295 raft_consensus.cc:2468] T 84f2da98864b4425bf7c0055951fdd4b P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 3.
W20260709 13:20:00.722474 16264 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:00.777813 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdbd78acb4cf46298e0de8ba0d130e19" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
W20260709 13:20:00.803176 16189 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:00.844885 16292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e4bfb9090c8460eb14fa7e0d818e7f1" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:00.918269 16821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66ae176961b74813a58bc15c9c4c6f2a" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
W20260709 13:20:00.959445 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 194367us
W20260709 13:20:00.960108 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 337617us
I20260709 13:20:00.960949 16790 leader_election.cc:304] T c9944d5e4b8c4109af9955c4f08c9095 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 90f3da57796b488c9087a2f523adf045
I20260709 13:20:00.825471 16824 raft_consensus.cc:2393] T 3884f864a96d428785895e5463e5728f P cb82cc7bd6db46d980c75237a1402860 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 5: Already voted for candidate cb82cc7bd6db46d980c75237a1402860 in this term.
I20260709 13:20:00.867735 16818 raft_consensus.cc:2468] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 7.
I20260709 13:20:00.958571 16116 leader_election.cc:304] T 5a959dbecc2648eb85d1cc550f5a3245 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
I20260709 13:20:00.958925 16292 raft_consensus.cc:2468] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 4.
I20260709 13:20:00.964990 16297 raft_consensus.cc:2468] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 6.
W20260709 13:20:00.752930 18792 log.cc:927] Time spent T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 1.099s	user 0.003s	sys 0.003s
W20260709 13:20:00.806497 18599 consensus_meta.cc:287] T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b: Time spent flushing consensus metadata: real 0.521s	user 0.009s	sys 0.003s
I20260709 13:20:00.776163 18832 raft_consensus.cc:2804] T 8cc959fb7f704fba9009c0595467d074 P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:20:00.838627 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0eab25ce4c64e018a041beeda2867db" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:20:00.964522 16291 raft_consensus.cc:2468] T 68a6838d82ef4cac83507f906139d90f P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 7.
I20260709 13:20:00.963548 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18ad7e3af5504571a69d10c00750a01c" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:00.971618 16822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdbd78acb4cf46298e0de8ba0d130e19" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:00.964901 16296 raft_consensus.cc:2468] T 1df93779209748b9860f6d893dd3de53 P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 5.
I20260709 13:20:00.966485 16117 leader_election.cc:304] T b87147731a68400e8cce9b879daa7c5d P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860
I20260709 13:20:00.973687 16266 leader_election.cc:304] T 57a744f5aaf944678a63469cbf514015 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:20:00.979044 16295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66ae176961b74813a58bc15c9c4c6f2a" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:01.031762 16150 raft_consensus.cc:2393] T 5a96373bb0e14dce8a266b2fc6c58507 P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 5: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
I20260709 13:20:00.988811 16298 raft_consensus.cc:3060] T c0eab25ce4c64e018a041beeda2867db P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:20:01.034017 16266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 60414.1us
I20260709 13:20:00.840409 18776 leader_election.cc:290] T 3884f864a96d428785895e5463e5728f P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 5 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:01.035971 16299 raft_consensus.cc:2468] T cdbd78acb4cf46298e0de8ba0d130e19 P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 6.
I20260709 13:20:01.036167 16789 leader_election.cc:304] T 23f82224f4564fea93572e8206bad1fe P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 
W20260709 13:20:01.040789 16117 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 74392.7us
I20260709 13:20:01.041085 16789 leader_election.cc:304] T 26354361e9b044cea1c2b86c4882fe92 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:20:01.036921 16292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02838c86531d4f98bf58d1c22b8baf77" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:01.037573 16146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d4ab870a05347868956aa8b841b2981" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 6 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:20:00.981631 16190 leader_election.cc:304] T 84f2da98864b4425bf7c0055951fdd4b P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:20:00.962554 18823 raft_consensus.cc:493] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:01.043150 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee9ed67006144797b1de6ea8c2013a68" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:01.042361 16146 raft_consensus.cc:2376] T 8d4ab870a05347868956aa8b841b2981 P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 6. Re-sending same reply.
I20260709 13:20:01.033246 16295 raft_consensus.cc:2468] T 66ae176961b74813a58bc15c9c4c6f2a P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 7.
W20260709 13:20:01.040174 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 71804.1us
I20260709 13:20:01.043310 18753 raft_consensus.cc:493] T b6f07842d3aa47858df5665874c86f3a P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:01.046219 18753 raft_consensus.cc:515] T b6f07842d3aa47858df5665874c86f3a P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:00.806164 18825 leader_election.cc:290] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:00.970762 18558 raft_consensus.cc:2804] T 5a959dbecc2648eb85d1cc550f5a3245 P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:20:01.044430 16149 raft_consensus.cc:2410] T ee9ed67006144797b1de6ea8c2013a68 P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac for term 6 because replica has last-logged OpId of term: 5 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 13:20:01.046530 18662 raft_consensus.cc:2804] T 23f82224f4564fea93572e8206bad1fe P cb82cc7bd6db46d980c75237a1402860 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:20:01.043423 16293 raft_consensus.cc:2468] T 18ad7e3af5504571a69d10c00750a01c P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 5.
W20260709 13:20:01.049623 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 88767.7us
I20260709 13:20:00.967901 18674 raft_consensus.cc:493] T 8c11074579b046d99e585c2818e18a2c P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:01.057602 16821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746aabf856ab401c9ef5e859371e7b67" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:01.062326 16823 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49906 (request call id 583) took 1076 ms. Trace:
I20260709 13:20:01.062744 16116 leader_election.cc:304] T 1df93779209748b9860f6d893dd3de53 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
I20260709 13:20:01.058964 16790 leader_election.cc:304] T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:20:00.968461 18738 raft_consensus.cc:2749] T 74ca8140f35c4ae29e8a664f6bdfc829 P cb82cc7bd6db46d980c75237a1402860 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:20:01.048369 18753 leader_election.cc:290] T b6f07842d3aa47858df5665874c86f3a P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:00.971933 18821 raft_consensus.cc:493] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:20:00.968979 18816 raft_consensus.cc:697] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [term 5 LEADER]: Becoming Leader. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
W20260709 13:20:01.071666 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 90092.4us
I20260709 13:20:00.977640 18347 raft_consensus.cc:493] T 5a96373bb0e14dce8a266b2fc6c58507 P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:00.799062 18587 raft_consensus.cc:515] T 8a878999c0024d2abe517484180290fc P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:01.059930 16822 raft_consensus.cc:2468] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 6.
I20260709 13:20:01.068596 16217 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "66ae176961b74813a58bc15c9c4c6f2a"
dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
mode: GRACEFUL
new_leader_uuid: "90f3da57796b488c9087a2f523adf045"
 from {username='slave'} at 127.0.0.1:39144
I20260709 13:20:00.823836 18114 raft_consensus.cc:515] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:01.038895 18824 raft_consensus.cc:2804] T c9944d5e4b8c4109af9955c4f08c9095 P cb82cc7bd6db46d980c75237a1402860 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 13:20:01.051388 18768 consensus_meta.cc:287] T fffab26427e24d398e2695aad40ca134 P 647a343a542d4a139fe106c19095129b: Time spent flushing consensus metadata: real 0.554s	user 0.009s	sys 0.003s
I20260709 13:20:01.075805 16217 raft_consensus.cc:606] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Received request to transfer leadership to TS 90f3da57796b488c9087a2f523adf045
I20260709 13:20:01.074676 16298 raft_consensus.cc:2468] T c0eab25ce4c64e018a041beeda2867db P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 6.
I20260709 13:20:01.089618 16217 raft_consensus.cc:614] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:20:01.079772 16146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c2095a5f4ad49408dbc7f69c606120d" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:01.084594 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 5 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:20:01.093739 16149 raft_consensus.cc:2393] T ecc1e363230043fe9fef390f0b979f20 P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 5: Already voted for candidate cb82cc7bd6db46d980c75237a1402860 in this term.
I20260709 13:20:01.049286 18743 raft_consensus.cc:2804] T 57a744f5aaf944678a63469cbf514015 P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:20:00.799154 18770 raft_consensus.cc:515] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:20:01.098340 16788 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:01.112263 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3884f864a96d428785895e5463e5728f" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
W20260709 13:20:01.104128 16191 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:01.113088 16298 raft_consensus.cc:2393] T 3884f864a96d428785895e5463e5728f P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 in current term 5: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
I20260709 13:20:01.062705 16823 rpcz_store.cc:276] 0709 13:19:59.985962 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:00.458125 (+472163us) service_pool.cc:224] Handling call
0709 13:20:00.458508 (+   383us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:00.616429 (+157921us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:20:00.674758 (+ 58329us) log.cc:844] Serialized 14 byte log entry
0709 13:20:01.061987 (+387229us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:20:01.062041 (+    54us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:20:01.062082 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:01.062158 (+    76us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:01.062274 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4401,"prepare.queue_time_us":58310,"prepare.run_cpu_time_us":58,"prepare.run_wall_time_us":59,"raft.queue_time_us":217849,"raft.run_cpu_time_us":115,"raft.run_wall_time_us":8191,"spinlock_wait_cycles":18176,"thread_start_us":235077,"threads_started":3,"wal-append.queue_time_us":288940}
I20260709 13:20:01.057390 16817 raft_consensus.cc:1240] T ada5a004ba7c46819051e3ccec71ffdb P cb82cc7bd6db46d980c75237a1402860 [term 7 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 6. Current term is 7. Ops: []
I20260709 13:20:01.118984 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0eab25ce4c64e018a041beeda2867db" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:20:01.119663 16149 raft_consensus.cc:3060] T c0eab25ce4c64e018a041beeda2867db P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:20:01.056968 18736 raft_consensus.cc:2804] T 26354361e9b044cea1c2b86c4882fe92 P cb82cc7bd6db46d980c75237a1402860 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:20:01.110281 16190 leader_election.cc:304] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: cb82cc7bd6db46d980c75237a1402860
I20260709 13:20:01.124882 16190 leader_election.cc:304] T 19baa7bc367844c288bca8f81439c413 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:20:01.114933 16790 leader_election.cc:304] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:20:01.083546 18347 raft_consensus.cc:515] T 5a96373bb0e14dce8a266b2fc6c58507 P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:01.117408 16291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9944d5e4b8c4109af9955c4f08c9095" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:01.118112 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66ae176961b74813a58bc15c9c4c6f2a" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:20:01.120044 16822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3884f864a96d428785895e5463e5728f" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:20:01.154973 16298 raft_consensus.cc:3060] T 66ae176961b74813a58bc15c9c4c6f2a P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:20:01.061146 18762 raft_consensus.cc:493] T 8c11074579b046d99e585c2818e18a2c P 1ca2725d290e47d68b1a4b609c9816ac [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:00.981092 18599 raft_consensus.cc:515] T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:20:01.129879 16189 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:00.969156 18828 raft_consensus.cc:515] T 26354361e9b044cea1c2b86c4882fe92 P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:01.043809 18778 consensus_queue.cc:1059] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.074s
I20260709 13:20:01.126019 16190 leader_election.cc:304] T 68a6838d82ef4cac83507f906139d90f P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:20:01.098960 18776 raft_consensus.cc:2749] T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:20:01.179716 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74ca8140f35c4ae29e8a664f6bdfc829" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:20:01.077719 18745 leader_election.cc:290] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 7 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
W20260709 13:20:01.212662 16189 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:01.085492 18807 raft_consensus.cc:2804] T 84f2da98864b4425bf7c0055951fdd4b P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:20:01.087397 18824 raft_consensus.cc:493] T c9944d5e4b8c4109af9955c4f08c9095 P cb82cc7bd6db46d980c75237a1402860 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:20:01.116878 18558 raft_consensus.cc:697] T 5a959dbecc2648eb85d1cc550f5a3245 P 647a343a542d4a139fe106c19095129b [term 5 LEADER]: Becoming Leader. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:20:01.110213 18662 raft_consensus.cc:697] T 23f82224f4564fea93572e8206bad1fe P cb82cc7bd6db46d980c75237a1402860 [term 4 LEADER]: Becoming Leader. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:20:01.058164 18823 raft_consensus.cc:515] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:01.073052 18674 raft_consensus.cc:515] T 8c11074579b046d99e585c2818e18a2c P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:01.078907 18821 raft_consensus.cc:515] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:20:01.296624 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 234116us
I20260709 13:20:01.372985 16143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1df93779209748b9860f6d893dd3de53" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:01.264285 16298 raft_consensus.cc:2468] T 66ae176961b74813a58bc15c9c4c6f2a P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 8.
W20260709 13:20:01.332790 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 217965us
I20260709 13:20:01.511909 16226 raft_consensus.cc:2410] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 for term 6 because replica has last-logged OpId of term: 5 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:20:01.246142 16821 raft_consensus.cc:1275] T 19baa7bc367844c288bca8f81439c413 P cb82cc7bd6db46d980c75237a1402860 [term 6 LEARNER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 6 index: 6. (index mismatch)
I20260709 13:20:01.528434 16821 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49896 (request call id 897) took 1275 ms. Trace:
I20260709 13:20:01.528779 16821 rpcz_store.cc:276] 0709 13:20:00.253513 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:01.124888 (+871375us) service_pool.cc:224] Handling call
0709 13:20:01.181415 (+ 56527us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:01.528166 (+346751us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:01.528379 (+   213us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:20:01.529747 16821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3884f864a96d428785895e5463e5728f" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
W20260709 13:20:01.513000 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 387043us
W20260709 13:20:01.530823 16116 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:20:01.410477 16824 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49896 (request call id 889) took 1651 ms (client timeout 1664 ms). Trace:
W20260709 13:20:01.531291 16824 rpcz_store.cc:269] 0709 13:19:59.759926 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:59.799295 (+ 39369us) service_pool.cc:224] Handling call
0709 13:20:01.410173 (+1610878us) spinlock_profiling.cc:243] Waited 992 ms on lock 0x7b54006bf654. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:20:01.410390 (+   217us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2280881920}
I20260709 13:20:01.532152 16146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68a6838d82ef4cac83507f906139d90f" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
W20260709 13:20:01.513827 16189 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:01.320225 16144 raft_consensus.cc:2393] T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 8: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
I20260709 13:20:01.351565 16822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5d59407eb545b895d8d922e0dd69fd" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:20:01.504578 16149 raft_consensus.cc:2468] T c0eab25ce4c64e018a041beeda2867db P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 6.
W20260709 13:20:01.533296 16189 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:20:01.521440 16818 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49896 (request call id 895) took 1729 ms (client timeout 1655 ms). Trace:
I20260709 13:20:01.315693 16218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8638ee363273426db1393dd81e5c7b72" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:20:01.118754 18842 raft_consensus.cc:493] T ecc1e363230043fe9fef390f0b979f20 P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:01.123016 18614 raft_consensus.cc:2749] T b87147731a68400e8cce9b879daa7c5d P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:20:01.537181 16821 raft_consensus.cc:2393] T 3884f864a96d428785895e5463e5728f P cb82cc7bd6db46d980c75237a1402860 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 5: Already voted for candidate cb82cc7bd6db46d980c75237a1402860 in this term.
I20260709 13:20:01.538796 16790 leader_election.cc:304] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:20:01.543391 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02838c86531d4f98bf58d1c22b8baf77" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
W20260709 13:20:01.536952 16818 rpcz_store.cc:269] 0709 13:19:59.791667 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:59.822070 (+ 30403us) service_pool.cc:224] Handling call
0709 13:20:01.521191 (+1699121us) spinlock_profiling.cc:243] Waited 929 ms on lock 0x7b5400787154. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:20:01.521349 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2137761152}
I20260709 13:20:01.545686 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1df93779209748b9860f6d893dd3de53" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:01.546904 16217 raft_consensus.cc:2410] T 1df93779209748b9860f6d893dd3de53 P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 for term 6 because replica has last-logged OpId of term: 4 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 13:20:01.122820 18743 raft_consensus.cc:697] T 57a744f5aaf944678a63469cbf514015 P 90f3da57796b488c9087a2f523adf045 [term 4 LEADER]: Becoming Leader. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:20:01.273654 18736 raft_consensus.cc:697] T 26354361e9b044cea1c2b86c4882fe92 P cb82cc7bd6db46d980c75237a1402860 [term 4 LEADER]: Becoming Leader. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:20:01.221694 18750 raft_consensus.cc:2804] T 19baa7bc367844c288bca8f81439c413 P 1ca2725d290e47d68b1a4b609c9816ac [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:20:01.552960 16822 raft_consensus.cc:3060] T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:20:01.371248 18818 raft_consensus.cc:493] T e5ad4eb281564936a59f41e387a47156 P 1ca2725d290e47d68b1a4b609c9816ac [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:01.215639 18738 raft_consensus.cc:2749] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:20:01.524859 18747 leader_election.cc:290] T 5a96373bb0e14dce8a266b2fc6c58507 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 5 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
W20260709 13:20:01.549098 16266 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:01.536609 18758 raft_consensus.cc:2804] T 68a6838d82ef4cac83507f906139d90f P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:20:01.392387 18580 leader_election.cc:290] T 2794af5b99984efebdee564035841cb0 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 5 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:01.183970 18753 raft_consensus.cc:2804] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:20:01.395001 18599 leader_election.cc:290] T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 8 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:01.334664 18836 raft_consensus.cc:2804] T 1df93779209748b9860f6d893dd3de53 P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:20:01.531512 18824 raft_consensus.cc:3060] T c9944d5e4b8c4109af9955c4f08c9095 P cb82cc7bd6db46d980c75237a1402860 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:20:01.561807 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d16dfad0c3b540e5b4b86492ddf87208" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:01.563679 16291 raft_consensus.cc:1240] T 2794af5b99984efebdee564035841cb0 P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 4. Current term is 5. Ops: []
I20260709 13:20:01.181591 18785 raft_consensus.cc:493] T 829b4ad8c5a54eababeea1181b960d4b P cb82cc7bd6db46d980c75237a1402860 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:20:01.275233 18720 leader_election.cc:290] T 9c61a39fd4f84424b129c1a6f9ba404d P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:01.514190 18807 raft_consensus.cc:493] T 84f2da98864b4425bf7c0055951fdd4b P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Starting leader election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:20:01.555598 16149 raft_consensus.cc:2468] T 02838c86531d4f98bf58d1c22b8baf77 P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 6.
I20260709 13:20:01.119287 18749 consensus_queue.cc:1048] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.429s
I20260709 13:20:01.573809 16220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:01.335536 18843 raft_consensus.cc:493] T e4dc6b0541ca431ebc757beac00521f1 P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:01.585881 16146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5718c623ebee4d2bb770f0f9ddb2f164" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
W20260709 13:20:01.587936 16788 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:01.114249 18768 raft_consensus.cc:515] T fffab26427e24d398e2695aad40ca134 P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:01.622920 16298 raft_consensus.cc:2376] T d16dfad0c3b540e5b4b86492ddf87208 P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 647a343a542d4a139fe106c19095129b in term 5. Re-sending same reply.
I20260709 13:20:01.553788 18347 leader_election.cc:290] T 5a96373bb0e14dce8a266b2fc6c58507 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:20:01.564580 18750 raft_consensus.cc:493] T 19baa7bc367844c288bca8f81439c413 P 1ca2725d290e47d68b1a4b609c9816ac [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:01.571738 18776 raft_consensus.cc:2749] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:20:01.244782 18762 raft_consensus.cc:515] T 8c11074579b046d99e585c2818e18a2c P 1ca2725d290e47d68b1a4b609c9816ac [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:01.525574 18816 consensus_queue.cc:237] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:01.554456 18842 raft_consensus.cc:515] T ecc1e363230043fe9fef390f0b979f20 P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } }
W20260709 13:20:01.833920 16144 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40844 (request call id 1922) took 1534 ms (client timeout 1796 ms). Trace:
W20260709 13:20:02.044587 16266 leader_election.cc:336] T 66ae176961b74813a58bc15c9c4c6f2a P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923): Timed out: RequestConsensusVote RPC to 127.14.177.132:38923 timed out after 2.890s (SENT)
W20260709 13:20:02.044734 16144 rpcz_store.cc:269] 0709 13:20:00.299530 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:00.309680 (+ 10150us) service_pool.cc:224] Handling call
0709 13:20:01.833685 (+1524005us) spinlock_profiling.cc:243] Waited 974 ms on lock 0x7b54000d1454. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:20:01.833855 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2240547072}
I20260709 13:20:02.045187 16790 leader_election.cc:304] T 9e65676b21b748728ad39668cfe722ec P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860; no voters: 90f3da57796b488c9087a2f523adf045
I20260709 13:20:02.046136 16146 raft_consensus.cc:2410] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac for term 6 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
W20260709 13:20:01.908378 16189 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:20:02.048161 16189 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:02.050156 18750 raft_consensus.cc:3060] T 19baa7bc367844c288bca8f81439c413 P 1ca2725d290e47d68b1a4b609c9816ac [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:20:02.050196 18854 raft_consensus.cc:493] T ecc1e363230043fe9fef390f0b979f20 P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:01.622875 18836 raft_consensus.cc:493] T 1df93779209748b9860f6d893dd3de53 P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:02.051667 18836 raft_consensus.cc:3060] T 1df93779209748b9860f6d893dd3de53 P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:20:02.051064 18854 raft_consensus.cc:515] T ecc1e363230043fe9fef390f0b979f20 P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:02.052096 18842 leader_election.cc:290] T ecc1e363230043fe9fef390f0b979f20 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:20:02.052378 16225 raft_consensus.cc:1275] T d16dfad0c3b540e5b4b86492ddf87208 P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:20:02.053544 18823 leader_election.cc:290] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:01.862903 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2372981006d4b218215c28964dea623" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:20:02.054664 16149 raft_consensus.cc:3060] T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:20:02.055864 16221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746aabf856ab401c9ef5e859371e7b67" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:01.634261 18812 leader_election.cc:290] T cdbd78acb4cf46298e0de8ba0d130e19 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:02.057464 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a878999c0024d2abe517484180290fc" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:01.623045 18614 raft_consensus.cc:493] T 18ad7e3af5504571a69d10c00750a01c P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:20:02.058411 16224 raft_consensus.cc:2410] T 8a878999c0024d2abe517484180290fc P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b for term 6 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 13:20:02.058290 18614 raft_consensus.cc:515] T 18ad7e3af5504571a69d10c00750a01c P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } }
I20260709 13:20:01.923060 18736 consensus_queue.cc:237] T 26354361e9b044cea1c2b86c4882fe92 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:02.060791 16219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4dc6b0541ca431ebc757beac00521f1" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:02.061589 16219 raft_consensus.cc:2410] T e4dc6b0541ca431ebc757beac00521f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b for term 6 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
W20260709 13:20:01.887671 18848 log.cc:927] Time spent T b87147731a68400e8cce9b879daa7c5d P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.666s	user 0.000s	sys 0.001s
I20260709 13:20:02.062659 18762 leader_election.cc:290] T 8c11074579b046d99e585c2818e18a2c P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:02.062769 16219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5d59407eb545b895d8d922e0dd69fd" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:20:01.579592 18818 raft_consensus.cc:515] T e5ad4eb281564936a59f41e387a47156 P 1ca2725d290e47d68b1a4b609c9816ac [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:02.064389 16219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c61a39fd4f84424b129c1a6f9ba404d" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:02.065002 16219 raft_consensus.cc:2468] T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 6.
I20260709 13:20:02.066222 18818 leader_election.cc:290] T e5ad4eb281564936a59f41e387a47156 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:02.066218 16219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a96373bb0e14dce8a266b2fc6c58507" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:20:02.066663 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74ca8140f35c4ae29e8a664f6bdfc829" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:02.067025 16219 raft_consensus.cc:3060] T 5a96373bb0e14dce8a266b2fc6c58507 P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:20:02.067261 16293 raft_consensus.cc:2468] T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 6.
I20260709 13:20:01.617926 18753 raft_consensus.cc:493] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:02.068308 16294 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40474 (request call id 395) took 1772 ms. Trace:
I20260709 13:20:02.068475 18753 raft_consensus.cc:3060] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:20:02.068699 16294 rpcz_store.cc:276] 0709 13:20:00.295694 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:00.295844 (+   150us) service_pool.cc:224] Handling call
0709 13:20:00.305791 (+  9947us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:20:00.624505 (+318714us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:00.624547 (+    42us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:20:00.632464 (+  7917us) log.cc:844] Serialized 52 byte log entry
0709 13:20:00.862528 (+230064us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:20:00.862590 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:00.862697 (+   107us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:20:02.068088 (+1205391us) raft_consensus.cc:1685] finished
0709 13:20:02.068111 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:02.068230 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":303044,"raft.run_cpu_time_us":86,"raft.run_wall_time_us":86,"thread_start_us":410945,"threads_started":2,"wal-append.queue_time_us":396127}
I20260709 13:20:02.046839 16790 leader_election.cc:304] T 434bff3082d141838068f323bdb0d1ed P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:20:02.071823 18834 leader_election.cc:290] T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:01.670611 18785 raft_consensus.cc:515] T 829b4ad8c5a54eababeea1181b960d4b P cb82cc7bd6db46d980c75237a1402860 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:02.073889 16219 raft_consensus.cc:2468] T 5a96373bb0e14dce8a266b2fc6c58507 P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 5.
I20260709 13:20:02.075843 18834 consensus_queue.cc:1048] T d16dfad0c3b540e5b4b86492ddf87208 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:20:02.076333 18817 raft_consensus.cc:3055] T 23f82224f4564fea93572e8206bad1fe P 647a343a542d4a139fe106c19095129b [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:20:02.076800 18817 raft_consensus.cc:740] T 23f82224f4564fea93572e8206bad1fe P 647a343a542d4a139fe106c19095129b [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:20:02.077576 18817 consensus_queue.cc:260] T 23f82224f4564fea93572e8206bad1fe P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:01.635005 18747 raft_consensus.cc:493] T 26354361e9b044cea1c2b86c4882fe92 P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:02.078680 18817 raft_consensus.cc:3060] T 23f82224f4564fea93572e8206bad1fe P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:20:01.835772 16820 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49892 (request call id 410) took 2087 ms. Trace:
I20260709 13:20:02.079967 16116 leader_election.cc:304] T 746aabf856ab401c9ef5e859371e7b67 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860
I20260709 13:20:02.080417 16820 rpcz_store.cc:276] 0709 13:19:59.748803 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:59.748958 (+   155us) service_pool.cc:224] Handling call
0709 13:20:01.835514 (+2086556us) spinlock_profiling.cc:243] Waited 439 ms on lock 0x7b540054bad4. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:20:01.835695 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1008573696}
I20260709 13:20:02.082554 16219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4f65cdf00b748bb963ee6e81ef43b79" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:20:02.077574 16225 raft_consensus.cc:1275] T e5ad4eb281564936a59f41e387a47156 P 1ca2725d290e47d68b1a4b609c9816ac [term 6 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 6 index: 5. (term mismatch)
I20260709 13:20:02.083859 16225 pending_rounds.cc:85] T e5ad4eb281564936a59f41e387a47156 P 1ca2725d290e47d68b1a4b609c9816ac: Aborting all ops after (but not including) 4
I20260709 13:20:02.083904 16219 raft_consensus.cc:3060] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:20:02.084199 16225 pending_rounds.cc:107] T e5ad4eb281564936a59f41e387a47156 P 1ca2725d290e47d68b1a4b609c9816ac: Aborting uncommitted NO_OP operation due to leader change: 5.5
I20260709 13:20:02.084488 16225 raft_consensus.cc:2889] T e5ad4eb281564936a59f41e387a47156 P 1ca2725d290e47d68b1a4b609c9816ac [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:20:02.086089 16149 raft_consensus.cc:2468] T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 6.
I20260709 13:20:01.580255 18758 raft_consensus.cc:493] T 68a6838d82ef4cac83507f906139d90f P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:01.822919 16291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6f07842d3aa47858df5665874c86f3a" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:02.104733 16291 raft_consensus.cc:2468] T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 5.
W20260709 13:20:02.094058 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 47254us
I20260709 13:20:02.102614 16297 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48684 (request call id 2072) took 1350 ms. Trace:
W20260709 13:20:02.029107 18830 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.824s	user 0.001s	sys 0.014s
W20260709 13:20:01.999145 18849 log.cc:927] Time spent T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860: Append to log took a long time: real 0.141s	user 0.000s	sys 0.000s
I20260709 13:20:02.095077 16116 leader_election.cc:304] T 9c61a39fd4f84424b129c1a6f9ba404d P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
I20260709 13:20:02.113641 16820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9944d5e4b8c4109af9955c4f08c9095" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
W20260709 13:20:02.019136 18859 log.cc:927] Time spent T c9944d5e4b8c4109af9955c4f08c9095 P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.190s	user 0.000s	sys 0.000s
I20260709 13:20:02.107687 16297 rpcz_store.cc:276] 0709 13:20:00.752414 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:00.966111 (+213697us) service_pool.cc:224] Handling call
0709 13:20:00.981426 (+ 15315us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:20:01.038700 (+ 57274us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:01.038744 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:20:01.050574 (+ 11830us) log.cc:844] Serialized 26 byte log entry
0709 13:20:01.556353 (+505779us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:20:01.556475 (+   122us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:01.556552 (+    77us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:20:02.102355 (+545803us) raft_consensus.cc:1685] finished
0709 13:20:02.102383 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:02.102543 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":26714,"raft.queue_time_us":26571,"raft.run_cpu_time_us":130,"raft.run_wall_time_us":9194,"thread_start_us":375599,"threads_started":1,"wal-append.queue_time_us":1016135}
W20260709 13:20:02.109567 16822 consensus_meta.cc:287] T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860: Time spent flushing consensus metadata: real 0.541s	user 0.007s	sys 0.002s
W20260709 13:20:02.108621 16266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 64081.6us
I20260709 13:20:02.000226 18674 leader_election.cc:290] T 8c11074579b046d99e585c2818e18a2c P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:02.092434 18766 leader_election.cc:290] T ada5a004ba7c46819051e3ccec71ffdb P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 7 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:02.108438 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8158bbd17b994d2eb96f560bd0f1b88d" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:02.112421 16226 raft_consensus.cc:1275] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 1ca2725d290e47d68b1a4b609c9816ac [term 6 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 13:20:01.640888 18807 raft_consensus.cc:3060] T 84f2da98864b4425bf7c0055951fdd4b P 1ca2725d290e47d68b1a4b609c9816ac [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:20:01.862864 18114 leader_election.cc:290] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:01.786576 18599 raft_consensus.cc:493] T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:01.835608 18858 raft_consensus.cc:493] T af57996a8be84d398f0f023e8d08ef77 P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:20:01.671530 18832 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.606s	user 0.007s	sys 0.009s
I20260709 13:20:01.827569 18720 raft_consensus.cc:493] T 08c74490c40243a88e62d1760b5cd814 P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:02.123325 16822 raft_consensus.cc:2410] T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b for term 7 because replica has last-logged OpId of term: 6 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 7.
I20260709 13:20:02.078984 18747 raft_consensus.cc:515] T 26354361e9b044cea1c2b86c4882fe92 P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:02.080058 18755 raft_consensus.cc:493] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Starting leader election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:20:02.124637 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ad5dbfa99ee4b8d9e28be92b3f36db5" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:02.149350 16297 raft_consensus.cc:2468] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 5.
I20260709 13:20:02.126691 16146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18ad7e3af5504571a69d10c00750a01c" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:02.126698 16144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57a744f5aaf944678a63469cbf514015" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:02.107353 18758 raft_consensus.cc:3060] T 68a6838d82ef4cac83507f906139d90f P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:20:01.888096 18770 leader_election.cc:290] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:02.104343 18745 raft_consensus.cc:2749] T 746aabf856ab401c9ef5e859371e7b67 P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:20:02.128399 16116 leader_election.cc:304] T 5a96373bb0e14dce8a266b2fc6c58507 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
W20260709 13:20:02.140811 18832 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.082s	user 0.012s	sys 0.032s
W20260709 13:20:02.113543 18830 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.379s	user 0.001s	sys 0.016s
I20260709 13:20:02.129967 16219 raft_consensus.cc:2468] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 8.
I20260709 13:20:02.108008 18823 raft_consensus.cc:493] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:02.136336 16221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5718c623ebee4d2bb770f0f9ddb2f164" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:02.111256 18738 raft_consensus.cc:2749] T 434bff3082d141838068f323bdb0d1ed P cb82cc7bd6db46d980c75237a1402860 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:20:02.136690 16294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5ad4eb281564936a59f41e387a47156" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:02.169673 16294 raft_consensus.cc:2468] T e5ad4eb281564936a59f41e387a47156 P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 6.
I20260709 13:20:02.137652 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c11074579b046d99e585c2818e18a2c" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:02.128772 16149 raft_consensus.cc:2468] T 8158bbd17b994d2eb96f560bd0f1b88d P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 6.
I20260709 13:20:02.170805 18823 raft_consensus.cc:515] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:02.171554 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18ad7e3af5504571a69d10c00750a01c" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:20:02.172705 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6f07842d3aa47858df5665874c86f3a" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:02.173038 18823 leader_election.cc:290] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:02.173828 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f82224f4564fea93572e8206bad1fe" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:20:02.151795 16146 raft_consensus.cc:2468] T 18ad7e3af5504571a69d10c00750a01c P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 4.
I20260709 13:20:02.175058 16146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f5f89186204242bf377f7c42b2e2da" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 8 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:02.175295 18855 raft_consensus.cc:2804] T 9e65676b21b748728ad39668cfe722ec P cb82cc7bd6db46d980c75237a1402860 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:20:02.148767 18821 leader_election.cc:290] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:02.175658 16146 raft_consensus.cc:2468] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 7.
I20260709 13:20:02.176126 18821 raft_consensus.cc:2804] T 5a96373bb0e14dce8a266b2fc6c58507 P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:20:02.176576 18821 raft_consensus.cc:697] T 5a96373bb0e14dce8a266b2fc6c58507 P 647a343a542d4a139fe106c19095129b [term 5 LEADER]: Becoming Leader. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:20:01.854168 18743 consensus_queue.cc:237] T 57a744f5aaf944678a63469cbf514015 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:02.177603 18821 consensus_queue.cc:237] T 5a96373bb0e14dce8a266b2fc6c58507 P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:02.104202 18843 raft_consensus.cc:515] T e4dc6b0541ca431ebc757beac00521f1 P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:01.787103 18662 consensus_queue.cc:237] T 23f82224f4564fea93572e8206bad1fe P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:02.176515 16146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57a744f5aaf944678a63469cbf514015" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:20:01.667793 18558 consensus_queue.cc:237] T 5a959dbecc2648eb85d1cc550f5a3245 P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:02.152240 18755 raft_consensus.cc:3060] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:20:02.156220 18832 raft_consensus.cc:697] T 8cc959fb7f704fba9009c0595467d074 P 647a343a542d4a139fe106c19095129b [term 6 LEADER]: Becoming Leader. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:20:02.163800 18750 raft_consensus.cc:515] T 19baa7bc367844c288bca8f81439c413 P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } }
I20260709 13:20:02.193207 16821 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49906 (request call id 591) took 1349 ms. Trace:
I20260709 13:20:02.195015 16293 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40474 (request call id 405) took 1068 ms. Trace:
I20260709 13:20:02.148132 18599 raft_consensus.cc:515] T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:02.195451 16293 rpcz_store.cc:276] 0709 13:20:01.126946 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:02.068079 (+941133us) service_pool.cc:224] Handling call
0709 13:20:02.068382 (+   303us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:20:02.068693 (+   311us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:02.068717 (+    24us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:20:02.069153 (+   436us) log.cc:844] Serialized 26 byte log entry
0709 13:20:02.148051 (+ 78898us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:20:02.148087 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:02.148157 (+    70us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:20:02.194806 (+ 46649us) raft_consensus.cc:1685] finished
0709 13:20:02.194823 (+    17us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:02.194962 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":9104,"raft.queue_time_us":11300,"raft.run_cpu_time_us":152,"raft.run_wall_time_us":18470,"thread_start_us":78542,"threads_started":1,"wal-append.queue_time_us":123674}
I20260709 13:20:02.149829 18720 raft_consensus.cc:515] T 08c74490c40243a88e62d1760b5cd814 P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:02.181679 16221 raft_consensus.cc:2468] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 5.
I20260709 13:20:02.199668 16816 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49892 (request call id 417) took 1224 ms. Trace:
W20260709 13:20:02.212260 18862 log.cc:927] Time spent T 806d7690c2b644959152074bb2cab0a7 P cb82cc7bd6db46d980c75237a1402860: Append to log took a long time: real 0.100s	user 0.000s	sys 0.001s
I20260709 13:20:02.193830 16821 rpcz_store.cc:276] 0709 13:20:00.844032 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:01.554696 (+710664us) service_pool.cc:224] Handling call
0709 13:20:01.555206 (+   510us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:20:01.566802 (+ 11596us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:01.566842 (+    40us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:20:01.760907 (+194065us) log.cc:844] Serialized 52 byte log entry
0709 13:20:02.070557 (+309650us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:20:02.122892 (+ 52335us) log.cc:844] Serialized 14 byte log entry
0709 13:20:02.192867 (+ 69975us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:02.192942 (+    75us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:20:02.192994 (+    52us) raft_consensus.cc:1685] finished
0709 13:20:02.193005 (+    11us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:02.193146 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7787,"prepare.queue_time_us":35579,"prepare.run_cpu_time_us":44,"prepare.run_wall_time_us":44,"raft.queue_time_us":37781,"raft.run_cpu_time_us":375,"raft.run_wall_time_us":50062,"thread_start_us":326952,"threads_started":3,"wal-append.queue_time_us":265299}
I20260709 13:20:02.214636 16146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a878999c0024d2abe517484180290fc" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:02.149533 18858 raft_consensus.cc:515] T af57996a8be84d398f0f023e8d08ef77 P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:02.218434 16825 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49896 (request call id 896) took 2007 ms. Trace:
I20260709 13:20:02.216506 16146 raft_consensus.cc:2468] T 8a878999c0024d2abe517484180290fc P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 5.
I20260709 13:20:02.221165 18819 raft_consensus.cc:493] T 7c2095a5f4ad49408dbc7f69c606120d P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:02.221720 18819 raft_consensus.cc:515] T 7c2095a5f4ad49408dbc7f69c606120d P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:02.222529 18825 raft_consensus.cc:3055] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 1ca2725d290e47d68b1a4b609c9816ac [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:20:02.222920 18825 raft_consensus.cc:740] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 1ca2725d290e47d68b1a4b609c9816ac [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:20:02.223798 18825 consensus_queue.cc:260] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:02.224984 18825 raft_consensus.cc:3060] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:20:02.226294 18828 leader_election.cc:290] T 26354361e9b044cea1c2b86c4882fe92 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:02.214175 16816 rpcz_store.cc:276] 0709 13:20:00.974855 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:01.667778 (+692923us) service_pool.cc:224] Handling call
0709 13:20:01.668259 (+   481us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:20:01.668837 (+   578us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:20:01.668876 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:20:01.669579 (+   703us) log.cc:844] Serialized 26 byte log entry
0709 13:20:02.127018 (+457439us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:20:02.130263 (+  3245us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:02.137095 (+  6832us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:20:02.194617 (+ 57522us) raft_consensus.cc:1685] finished
0709 13:20:02.194641 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:02.199584 (+  4943us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1933,"raft.run_cpu_time_us":147,"raft.run_wall_time_us":24742,"thread_start_us":442043,"threads_started":1,"wal-append.queue_time_us":523311}
W20260709 13:20:02.229182 18856 log.cc:927] Time spent T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.631s	user 0.003s	sys 0.000s
W20260709 13:20:02.232537 16191 connection.cc:256] RPC call timeout handler was delayed by -1.0988351569999395s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1686s for a response
W20260709 13:20:02.232985 16191 connection.cc:256] RPC call timeout handler was delayed by -1.0772351569999046s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.171s for a response
W20260709 13:20:02.233661 16191 leader_election.cc:336] T 08c74490c40243a88e62d1760b5cd814 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Timed out: RequestConsensusVote RPC to 127.14.177.129:36741 timed out after 1.534s (SENT)
I20260709 13:20:02.234305 18747 leader_election.cc:290] T 26354361e9b044cea1c2b86c4882fe92 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:02.234440 18843 leader_election.cc:290] T e4dc6b0541ca431ebc757beac00521f1 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:20:02.220099 16825 rpcz_store.cc:276] 0709 13:20:00.206595 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:00.645937 (+439342us) service_pool.cc:224] Handling call
0709 13:20:00.646354 (+   417us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:20:00.646798 (+   444us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:00.646827 (+    29us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:20:00.661238 (+ 14411us) log.cc:844] Serialized 52 byte log entry
0709 13:20:00.959225 (+297987us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:20:00.959264 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:00.959326 (+    62us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:20:02.209106 (+1249780us) raft_consensus.cc:1685] finished
0709 13:20:02.209127 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:02.213708 (+  4581us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":10806,"raft.run_cpu_time_us":186,"raft.run_wall_time_us":20316,"thread_start_us":252915,"threads_started":1,"wal-append.queue_time_us":953304}
I20260709 13:20:02.246526 16220 raft_consensus.cc:2368] T 19baa7bc367844c288bca8f81439c413 P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 for earlier term 6. Current term is 7.
W20260709 13:20:02.247665 16191 connection.cc:256] RPC call timeout handler was delayed by -1.0422258649998639s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2546s for a response
I20260709 13:20:02.211478 18868 raft_consensus.cc:493] T 8d4ab870a05347868956aa8b841b2981 P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:02.254609 16146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:02.208599 18674 raft_consensus.cc:493] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:20:02.240300 18824 consensus_meta.cc:287] T c9944d5e4b8c4109af9955c4f08c9095 P cb82cc7bd6db46d980c75237a1402860: Time spent flushing consensus metadata: real 0.588s	user 0.003s	sys 0.007s
I20260709 13:20:02.263149 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c11074579b046d99e585c2818e18a2c" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:20:02.263360 16219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a378d7f796f94615b3b347119a9880fd" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:02.254462 18770 raft_consensus.cc:493] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:02.199764 18870 raft_consensus.cc:2804] T 9c61a39fd4f84424b129c1a6f9ba404d P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:20:02.267843 16146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5ad4eb281564936a59f41e387a47156" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
W20260709 13:20:02.270789 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 142465us
W20260709 13:20:02.272375 16116 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:02.274258 16221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada5a004ba7c46819051e3ccec71ffdb" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:02.275499 16221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2372981006d4b218215c28964dea623" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
W20260709 13:20:02.278963 16190 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:02.269402 18870 raft_consensus.cc:493] T 9c61a39fd4f84424b129c1a6f9ba404d P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:02.269124 18551 raft_consensus.cc:493] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 1ca2725d290e47d68b1a4b609c9816ac [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:02.279516 18870 raft_consensus.cc:3060] T 9c61a39fd4f84424b129c1a6f9ba404d P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:20:02.270417 18770 raft_consensus.cc:515] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:02.282222 18770 leader_election.cc:290] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:02.283599 16146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26354361e9b044cea1c2b86c4882fe92" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:02.284013 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a378d7f796f94615b3b347119a9880fd" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:02.284252 16146 raft_consensus.cc:2468] T 26354361e9b044cea1c2b86c4882fe92 P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 3.
I20260709 13:20:02.284704 18832 consensus_queue.cc:237] T 8cc959fb7f704fba9009c0595467d074 P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:02.285287 16146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2794af5b99984efebdee564035841cb0" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:20:02.285822 16146 raft_consensus.cc:3060] T 2794af5b99984efebdee564035841cb0 P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:20:02.285434 16189 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:20:02.286911 16189 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:02.287535 18819 leader_election.cc:290] T 7c2095a5f4ad49408dbc7f69c606120d P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:20:02.288502 16219 raft_consensus.cc:2393] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 in current term 5: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
W20260709 13:20:02.288589 16299 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48684 (request call id 2068) took 1567 ms (client timeout 1775 ms). Trace:
W20260709 13:20:02.290480 16299 rpcz_store.cc:269] 0709 13:20:00.720727 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:00.777468 (+ 56741us) service_pool.cc:224] Handling call
0709 13:20:02.288408 (+1510940us) spinlock_profiling.cc:243] Waited 77 ms on lock 0x7b54002e1b54. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:20:02.288505 (+    97us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":177142528}
I20260709 13:20:02.292222 16226 raft_consensus.cc:2468] T 8c11074579b046d99e585c2818e18a2c P 1ca2725d290e47d68b1a4b609c9816ac [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 6.
I20260709 13:20:02.293458 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ab0ac8347641e6955e9bce7d90c1a2" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:20:02.294384 16191 leader_election.cc:304] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
W20260709 13:20:02.294694 16116 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:02.268532 18785 leader_election.cc:290] T 829b4ad8c5a54eababeea1181b960d4b P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:20:02.297580 16116 leader_election.cc:304] T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
I20260709 13:20:02.299727 16265 leader_election.cc:304] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:20:02.304315 16116 leader_election.cc:304] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
I20260709 13:20:02.305315 16265 leader_election.cc:304] T 8a878999c0024d2abe517484180290fc P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:20:02.307142 18881 consensus_queue.cc:1048] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.005s
I20260709 13:20:02.298913 18770 consensus_queue.cc:1048] T 829b4ad8c5a54eababeea1181b960d4b P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.010s
I20260709 13:20:02.310956 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af57996a8be84d398f0f023e8d08ef77" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:02.311672 16226 raft_consensus.cc:2376] T af57996a8be84d398f0f023e8d08ef77 P 1ca2725d290e47d68b1a4b609c9816ac [term 6 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 6. Re-sending same reply.
I20260709 13:20:02.313829 16191 leader_election.cc:304] T fffab26427e24d398e2695aad40ca134 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:20:02.314320 16265 leader_election.cc:304] T 26354361e9b044cea1c2b86c4882fe92 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:20:02.312013 16823 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49892 (request call id 415) took 1980 ms. Trace:
I20260709 13:20:02.315280 16823 rpcz_store.cc:276] 0709 13:20:00.331382 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:01.187327 (+855945us) service_pool.cc:224] Handling call
0709 13:20:01.274184 (+ 86857us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:20:01.314047 (+ 39863us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:01.314088 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:20:01.354616 (+ 40528us) log.cc:844] Serialized 572 byte log entry
0709 13:20:01.775971 (+421355us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:20:01.776007 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:01.776077 (+    70us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:20:02.311827 (+535750us) raft_consensus.cc:1685] finished
0709 13:20:02.311844 (+    17us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:02.311961 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":421014,"threads_started":1,"wal-append.queue_time_us":705601}
I20260709 13:20:02.313064 18747 raft_consensus.cc:2804] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:20:02.317713 18747 raft_consensus.cc:493] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Starting leader election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:20:02.318086 18747 raft_consensus.cc:3060] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:20:02.323225 18843 raft_consensus.cc:2804] T 8a878999c0024d2abe517484180290fc P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:20:02.323681 18770 consensus_queue.cc:1048] T 746aabf856ab401c9ef5e859371e7b67 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:20:02.325265 18869 consensus_queue.cc:1048] T c2f5f89186204242bf377f7c42b2e2da P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:20:02.323702 18843 raft_consensus.cc:493] T 8a878999c0024d2abe517484180290fc P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:02.327126 18843 raft_consensus.cc:3060] T 8a878999c0024d2abe517484180290fc P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:20:02.289568 16221 raft_consensus.cc:2468] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 6.
I20260709 13:20:02.268599 18674 raft_consensus.cc:515] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:02.341915 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada5a004ba7c46819051e3ccec71ffdb" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:20:02.359807 16821 raft_consensus.cc:1240] T 23f82224f4564fea93572e8206bad1fe P cb82cc7bd6db46d980c75237a1402860 [term 4 LEADER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 3. Current term is 4. Ops: [3.5-3.5]
W20260709 13:20:02.360134 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 201534us
I20260709 13:20:02.361724 16790 leader_election.cc:304] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:20:02.361933 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdbd78acb4cf46298e0de8ba0d130e19" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:20:02.362509 18785 raft_consensus.cc:2804] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:20:02.362964 16226 raft_consensus.cc:2410] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 for term 6 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 13:20:02.363050 18785 raft_consensus.cc:697] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 [term 6 LEADER]: Becoming Leader. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:20:02.364339 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66ae176961b74813a58bc15c9c4c6f2a" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:20:02.365083 16226 raft_consensus.cc:3060] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:20:02.364362 18785 consensus_queue.cc:237] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:02.371305 18878 raft_consensus.cc:493] T 8638ee363273426db1393dd81e5c7b72 P cb82cc7bd6db46d980c75237a1402860 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:02.371882 18878 raft_consensus.cc:515] T 8638ee363273426db1393dd81e5c7b72 P cb82cc7bd6db46d980c75237a1402860 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:02.374711 18878 leader_election.cc:290] T 8638ee363273426db1393dd81e5c7b72 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:02.284282 18858 leader_election.cc:290] T af57996a8be84d398f0f023e8d08ef77 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:02.279728 18551 raft_consensus.cc:515] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 1ca2725d290e47d68b1a4b609c9816ac [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:02.302781 18720 leader_election.cc:290] T 08c74490c40243a88e62d1760b5cd814 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:02.314379 18745 raft_consensus.cc:2804] T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:20:02.314538 18114 raft_consensus.cc:2749] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:20:02.267958 18868 raft_consensus.cc:515] T 8d4ab870a05347868956aa8b841b2981 P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:02.314085 18819 raft_consensus.cc:2804] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:20:02.319489 18755 raft_consensus.cc:515] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:02.395192 18599 leader_election.cc:290] T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:02.397887 16225 pending_rounds.cc:85] T ee9ed67006144797b1de6ea8c2013a68 P 1ca2725d290e47d68b1a4b609c9816ac: Aborting all ops after (but not including) 5
I20260709 13:20:02.429678 16225 pending_rounds.cc:107] T ee9ed67006144797b1de6ea8c2013a68 P 1ca2725d290e47d68b1a4b609c9816ac: Aborting uncommitted NO_OP operation due to leader change: 4.6
I20260709 13:20:02.429978 16225 raft_consensus.cc:2889] T ee9ed67006144797b1de6ea8c2013a68 P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:20:02.397547 18836 raft_consensus.cc:515] T 1df93779209748b9860f6d893dd3de53 P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:02.398098 18858 consensus_queue.cc:1048] T e5ad4eb281564936a59f41e387a47156 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:20:02.403409 16226 raft_consensus.cc:2468] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 8.
I20260709 13:20:02.436988 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fffab26427e24d398e2695aad40ca134" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:20:02.404914 16190 leader_election.cc:304] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:20:02.269446 18824 raft_consensus.cc:515] T c9944d5e4b8c4109af9955c4f08c9095 P cb82cc7bd6db46d980c75237a1402860 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:02.407547 16149 raft_consensus.cc:2410] T 23f82224f4564fea93572e8206bad1fe P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 for term 4 because replica has last-logged OpId of term: 3 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
W20260709 13:20:02.399135 18880 log.cc:927] Time spent T 829b4ad8c5a54eababeea1181b960d4b P cb82cc7bd6db46d980c75237a1402860: Append to log took a long time: real 0.092s	user 0.000s	sys 0.000s
I20260709 13:20:02.399955 18745 raft_consensus.cc:697] T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b [term 8 LEADER]: Becoming Leader. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:20:02.416692 18421 raft_consensus.cc:3060] T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:20:02.418082 18797 leader_election.cc:290] T e4dc6b0541ca431ebc757beac00521f1 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:02.418270 18768 leader_election.cc:290] T fffab26427e24d398e2695aad40ca134 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 5 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:02.418759 16296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fffab26427e24d398e2695aad40ca134" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:02.360077 18743 raft_consensus.cc:2804] T 26354361e9b044cea1c2b86c4882fe92 P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:20:02.343861 18770 raft_consensus.cc:2749] T fffab26427e24d398e2695aad40ca134 P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:20:02.420130 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434bff3082d141838068f323bdb0d1ed" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
W20260709 13:20:02.432107 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 52379.8us
I20260709 13:20:02.455962 16191 leader_election.cc:304] T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:20:02.435448 16294 raft_consensus.cc:1240] T 23f82224f4564fea93572e8206bad1fe P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 3. Current term is 4. Ops: []
I20260709 13:20:02.408530 16146 raft_consensus.cc:2468] T 2794af5b99984efebdee564035841cb0 P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 5.
I20260709 13:20:02.458551 16146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3884f864a96d428785895e5463e5728f" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:20:02.459642 16146 raft_consensus.cc:2393] T 3884f864a96d428785895e5463e5728f P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 in current term 5: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
I20260709 13:20:02.441445 16295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a959dbecc2648eb85d1cc550f5a3245" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:20:02.442248 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f5f89186204242bf377f7c42b2e2da" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:02.429280 18812 raft_consensus.cc:493] T 434bff3082d141838068f323bdb0d1ed P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:02.447773 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e9b2bf7011f4ae688a79d7bb5a447b4" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:20:02.448369 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3884f864a96d428785895e5463e5728f" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:20:02.608039 16299 raft_consensus.cc:2393] T 3884f864a96d428785895e5463e5728f P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 5: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
I20260709 13:20:02.608201 16294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1df93779209748b9860f6d893dd3de53" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:02.608803 16294 raft_consensus.cc:2468] T 1df93779209748b9860f6d893dd3de53 P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 5.
I20260709 13:20:02.608950 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4dc6b0541ca431ebc757beac00521f1" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:02.609540 16299 raft_consensus.cc:2468] T e4dc6b0541ca431ebc757beac00521f1 P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 5.
W20260709 13:20:02.610205 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 154368us
I20260709 13:20:02.610743 16299 raft_consensus.cc:1275] T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:20:02.419288 18828 raft_consensus.cc:493] T 31969392ccdb43b491911a319e8bc7e1 P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:02.610961 16191 leader_election.cc:304] T d16dfad0c3b540e5b4b86492ddf87208 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:20:02.611542 18869 raft_consensus.cc:2749] T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:20:02.611459 18828 raft_consensus.cc:515] T 31969392ccdb43b491911a319e8bc7e1 P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:02.612329 16293 raft_consensus.cc:1275] T 68a6838d82ef4cac83507f906139d90f P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:20:02.613720 18828 leader_election.cc:290] T 31969392ccdb43b491911a319e8bc7e1 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:02.613857 16293 raft_consensus.cc:1275] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:20:02.461397 16265 leader_election.cc:304] T 2794af5b99984efebdee564035841cb0 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:20:02.454422 16143 raft_consensus.cc:2393] T 1df93779209748b9860f6d893dd3de53 P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 6: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
W20260709 13:20:02.615021 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 153732us
I20260709 13:20:02.429601 18819 raft_consensus.cc:493] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:20:02.615691 18819 raft_consensus.cc:3060] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:20:02.612353 18869 raft_consensus.cc:2749] T d16dfad0c3b540e5b4b86492ddf87208 P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:20:02.439481 16190 leader_election.cc:304] T 1df93779209748b9860f6d893dd3de53 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
W20260709 13:20:02.616983 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 177556us
I20260709 13:20:02.617604 18869 raft_consensus.cc:2804] T 1df93779209748b9860f6d893dd3de53 P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:20:02.617677 16190 leader_election.cc:304] T 18ad7e3af5504571a69d10c00750a01c P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:20:02.618140 18869 raft_consensus.cc:493] T 1df93779209748b9860f6d893dd3de53 P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:02.618639 18869 raft_consensus.cc:3060] T 1df93779209748b9860f6d893dd3de53 P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:20:02.378356 18758 raft_consensus.cc:515] T 68a6838d82ef4cac83507f906139d90f P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:02.619014 16190 leader_election.cc:304] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:20:02.629199 16294 raft_consensus.cc:1275] T 31969392ccdb43b491911a319e8bc7e1 P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 13:20:02.612268 16191 leader_election.cc:304] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
I20260709 13:20:02.439472 18114 raft_consensus.cc:2764] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Leader pre-election decision vote started in defunct term 7: won
I20260709 13:20:02.622287 16820 raft_consensus.cc:2393] T c9944d5e4b8c4109af9955c4f08c9095 P cb82cc7bd6db46d980c75237a1402860 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 7: Already voted for candidate cb82cc7bd6db46d980c75237a1402860 in this term.
I20260709 13:20:02.399672 18870 raft_consensus.cc:515] T 9c61a39fd4f84424b129c1a6f9ba404d P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:02.644586 16146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87147731a68400e8cce9b879daa7c5d" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:02.648540 16146 raft_consensus.cc:2410] T b87147731a68400e8cce9b879daa7c5d P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 for term 8 because replica has last-logged OpId of term: 7 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 6.
I20260709 13:20:02.650143 16296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cc959fb7f704fba9009c0595467d074" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:20:02.675691 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2794af5b99984efebdee564035841cb0" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:02.454831 18743 raft_consensus.cc:493] T 26354361e9b044cea1c2b86c4882fe92 P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:02.619609 18720 raft_consensus.cc:493] T 8d4ab870a05347868956aa8b841b2981 P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:02.674576 16819 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49906 (request call id 576) took 3197 ms. Trace:
I20260709 13:20:02.613987 18755 leader_election.cc:290] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 8 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:02.443352 18753 raft_consensus.cc:515] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:02.715135 16226 raft_consensus.cc:3060] T fffab26427e24d398e2695aad40ca134 P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:20:02.698863 16819 rpcz_store.cc:276] 0709 13:19:59.477304 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:59.477447 (+   143us) service_pool.cc:224] Handling call
0709 13:19:59.489259 (+ 11812us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:00.097988 (+608729us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:02.674307 (+2576319us) spinlock_profiling.cc:243] Waited 608 ms on lock 0x7b54003df254. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:20:02.674511 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1398626688}
I20260709 13:20:02.641445 18828 raft_consensus.cc:2804] T 2794af5b99984efebdee564035841cb0 P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:20:02.642293 18797 raft_consensus.cc:2804] T 18ad7e3af5504571a69d10c00750a01c P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:20:02.718266 16295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a9de347c711477e87c582d003f36925" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:20:02.721692 16295 raft_consensus.cc:3060] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:20:02.715086 16146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6f07842d3aa47858df5665874c86f3a" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:02.733022 16146 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40844 (request call id 1943) took 1343 ms. Trace:
I20260709 13:20:02.733716 16146 rpcz_store.cc:276] 0709 13:20:01.389339 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:02.678194 (+1288855us) service_pool.cc:224] Handling call
0709 13:20:02.732939 (+ 54745us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:20:02.699416 18743 raft_consensus.cc:3060] T 26354361e9b044cea1c2b86c4882fe92 P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:20:02.749889 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 130937us
I20260709 13:20:02.624230 18812 raft_consensus.cc:515] T 434bff3082d141838068f323bdb0d1ed P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:02.714190 18736 raft_consensus.cc:493] T 3a7ca61592e848ea9b0fb81a2ce33cdc P cb82cc7bd6db46d980c75237a1402860 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:20:02.721460 18770 raft_consensus.cc:2804] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:20:02.721297 18797 raft_consensus.cc:493] T 18ad7e3af5504571a69d10c00750a01c P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:02.719813 18842 raft_consensus.cc:493] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:02.731485 18755 raft_consensus.cc:493] T 829b4ad8c5a54eababeea1181b960d4b P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:02.763463 16296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26354361e9b044cea1c2b86c4882fe92" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:02.769937 16146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c2095a5f4ad49408dbc7f69c606120d" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:02.815419 16297 raft_consensus.cc:3060] T 434bff3082d141838068f323bdb0d1ed P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:20:02.816627 18812 leader_election.cc:290] T 434bff3082d141838068f323bdb0d1ed P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:02.817193 18812 raft_consensus.cc:493] T c0eab25ce4c64e018a041beeda2867db P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:02.817657 18812 raft_consensus.cc:515] T c0eab25ce4c64e018a041beeda2867db P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:02.737345 18832 raft_consensus.cc:493] T ee9ed67006144797b1de6ea8c2013a68 P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:20:02.744551 18750 leader_election.cc:290] T 19baa7bc367844c288bca8f81439c413 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 7 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:02.741518 18828 raft_consensus.cc:697] T 2794af5b99984efebdee564035841cb0 P 90f3da57796b488c9087a2f523adf045 [term 5 LEADER]: Becoming Leader. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:20:02.760176 18745 consensus_queue.cc:237] T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:02.822906 16297 raft_consensus.cc:2468] T 434bff3082d141838068f323bdb0d1ed P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 6.
I20260709 13:20:02.750886 16190 leader_election.cc:304] T b6f07842d3aa47858df5665874c86f3a P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:20:02.720463 18720 raft_consensus.cc:515] T 8d4ab870a05347868956aa8b841b2981 P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:02.833706 16295 raft_consensus.cc:2468] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 7.
I20260709 13:20:02.834170 16115 leader_election.cc:304] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:20:02.836812 16226 raft_consensus.cc:2468] T fffab26427e24d398e2695aad40ca134 P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 5.
I20260709 13:20:02.838130 16146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e9b2bf7011f4ae688a79d7bb5a447b4" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:20:02.764446 18770 raft_consensus.cc:493] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:02.843330 16146 raft_consensus.cc:2393] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 4: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
W20260709 13:20:02.844765 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 93997us
I20260709 13:20:02.786494 18797 raft_consensus.cc:3060] T 18ad7e3af5504571a69d10c00750a01c P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:20:02.851776 16146 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40830 (request call id 1169) took 1531 ms. Trace:
I20260709 13:20:02.852337 16146 rpcz_store.cc:276] 0709 13:20:01.319983 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:02.830552 (+1510569us) service_pool.cc:224] Handling call
0709 13:20:02.851692 (+ 21140us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:20:02.850302 16190 leader_election.cc:304] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:20:02.853504 16146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e9b2bf7011f4ae688a79d7bb5a447b4" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:02.853724 16295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fffab26427e24d398e2695aad40ca134" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:20:02.854450 16146 raft_consensus.cc:2468] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 4.
I20260709 13:20:02.873909 16146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c61a39fd4f84424b129c1a6f9ba404d" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
W20260709 13:20:02.873965 16266 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:02.874811 16146 raft_consensus.cc:2393] T 9c61a39fd4f84424b129c1a6f9ba404d P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 7: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
I20260709 13:20:02.835493 18755 raft_consensus.cc:515] T 829b4ad8c5a54eababeea1181b960d4b P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:02.847978 18770 raft_consensus.cc:3060] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:20:02.769150 18736 raft_consensus.cc:515] T 3a7ca61592e848ea9b0fb81a2ce33cdc P cb82cc7bd6db46d980c75237a1402860 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:02.852119 18785 raft_consensus.cc:493] T 74ca8140f35c4ae29e8a664f6bdfc829 P cb82cc7bd6db46d980c75237a1402860 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:20:02.858173 18747 consensus_meta.cc:287] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b: Time spent flushing consensus metadata: real 0.540s	user 0.010s	sys 0.000s
I20260709 13:20:02.866711 18858 raft_consensus.cc:2749] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:20:02.831559 18842 raft_consensus.cc:515] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:02.849251 18896 raft_consensus.cc:2749] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:20:02.907980 16151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a9de347c711477e87c582d003f36925" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:02.908617 16151 raft_consensus.cc:2468] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 7.
I20260709 13:20:02.909406 16151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "829b4ad8c5a54eababeea1181b960d4b" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:02.909915 16151 raft_consensus.cc:2468] T 829b4ad8c5a54eababeea1181b960d4b P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 6.
I20260709 13:20:02.910642 16151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a96373bb0e14dce8a266b2fc6c58507" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:02.911998 18812 leader_election.cc:290] T c0eab25ce4c64e018a041beeda2867db P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:20:02.878232 18674 leader_election.cc:290] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:20:02.915408 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66f75124215f4ca6bef831e174a792d8" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:02.916975 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62fe3346cd3c42c4a55e14af9073bca4" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:20:02.917552 16225 raft_consensus.cc:3060] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:20:02.920192 18884 log.cc:927] Time spent T 57a744f5aaf944678a63469cbf514015 P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.167s	user 0.000s	sys 0.001s
I20260709 13:20:02.921092 18883 consensus_queue.cc:1048] T b6f07842d3aa47858df5665874c86f3a P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
W20260709 13:20:02.921512 18891 log.cc:927] Time spent T b87147731a68400e8cce9b879daa7c5d P cb82cc7bd6db46d980c75237a1402860: Append to log took a long time: real 0.197s	user 0.001s	sys 0.000s
W20260709 13:20:02.922941 18889 log.cc:927] Time spent T 23f82224f4564fea93572e8206bad1fe P cb82cc7bd6db46d980c75237a1402860: Append to log took a long time: real 0.205s	user 0.001s	sys 0.000s
I20260709 13:20:02.885460 18836 leader_election.cc:290] T 1df93779209748b9860f6d893dd3de53 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 6 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
W20260709 13:20:02.924355 18887 log.cc:927] Time spent T ada5a004ba7c46819051e3ccec71ffdb P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.090s	user 0.000s	sys 0.000s
I20260709 13:20:02.925096 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18ad7e3af5504571a69d10c00750a01c" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:02.925372 16225 raft_consensus.cc:2468] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 8.
W20260709 13:20:02.925745 18890 log.cc:927] Time spent T 2794af5b99984efebdee564035841cb0 P cb82cc7bd6db46d980c75237a1402860: Append to log took a long time: real 0.274s	user 0.000s	sys 0.001s
I20260709 13:20:02.902420 18755 leader_election.cc:290] T 829b4ad8c5a54eababeea1181b960d4b P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:02.926429 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18ad7e3af5504571a69d10c00750a01c" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:02.835783 18832 raft_consensus.cc:515] T ee9ed67006144797b1de6ea8c2013a68 P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:02.925899 16299 raft_consensus.cc:2393] T 18ad7e3af5504571a69d10c00750a01c P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 5: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
I20260709 13:20:02.878377 18892 consensus_queue.cc:1048] T 68a6838d82ef4cac83507f906139d90f P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 2, Time since last communication: 0.031s
I20260709 13:20:02.904539 16295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c61a39fd4f84424b129c1a6f9ba404d" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:02.880074 18750 raft_consensus.cc:2804] T b6f07842d3aa47858df5665874c86f3a P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:20:02.928548 16116 leader_election.cc:304] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
I20260709 13:20:02.931994 16151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3884f864a96d428785895e5463e5728f" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:20:02.936352 18868 leader_election.cc:290] T 8d4ab870a05347868956aa8b841b2981 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:20:02.936966 18758 leader_election.cc:290] T 68a6838d82ef4cac83507f906139d90f P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 8 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:02.961151 18736 leader_election.cc:290] T 3a7ca61592e848ea9b0fb81a2ce33cdc P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
W20260709 13:20:02.964919 18853 log.cc:927] Time spent T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 1.569s	user 0.003s	sys 0.013s
I20260709 13:20:02.894979 18785 raft_consensus.cc:515] T 74ca8140f35c4ae29e8a664f6bdfc829 P cb82cc7bd6db46d980c75237a1402860 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:02.968001 18785 leader_election.cc:290] T 74ca8140f35c4ae29e8a664f6bdfc829 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:02.900279 18747 raft_consensus.cc:515] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:02.968909 16821 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49896 (request call id 899) took 1916 ms. Trace:
I20260709 13:20:02.969329 16821 rpcz_store.cc:276] 0709 13:20:01.052070 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:02.216627 (+1164557us) service_pool.cc:224] Handling call
0709 13:20:02.216965 (+   338us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:20:02.375516 (+158551us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:02.968643 (+593127us) spinlock_profiling.cc:243] Waited 138 ms on lock 0x7b54007d0354. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:20:02.968850 (+   207us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":317412736}
I20260709 13:20:02.970546 18747 leader_election.cc:290] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 6 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:02.899552 18551 leader_election.cc:290] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:02.970989 16148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4dc6b0541ca431ebc757beac00521f1" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:02.971697 16148 raft_consensus.cc:2468] T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 5.
I20260709 13:20:02.968007 16219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c74490c40243a88e62d1760b5cd814" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:02.937817 18870 leader_election.cc:290] T 9c61a39fd4f84424b129c1a6f9ba404d P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 7 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:02.898910 18828 consensus_queue.cc:237] T 2794af5b99984efebdee564035841cb0 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:02.974532 16148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26354361e9b044cea1c2b86c4882fe92" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:20:02.975229 16148 raft_consensus.cc:3060] T 26354361e9b044cea1c2b86c4882fe92 P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:20:02.932945 16146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31969392ccdb43b491911a319e8bc7e1" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:02.939980 16295 raft_consensus.cc:2468] T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 6.
I20260709 13:20:02.942009 18750 raft_consensus.cc:493] T b6f07842d3aa47858df5665874c86f3a P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:02.940835 18900 consensus_queue.cc:1048] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.024s
W20260709 13:20:02.937012 18843 consensus_meta.cc:287] T 8a878999c0024d2abe517484180290fc P 90f3da57796b488c9087a2f523adf045: Time spent flushing consensus metadata: real 0.609s	user 0.008s	sys 0.002s
W20260709 13:20:02.977684 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 49211.4us
I20260709 13:20:02.978801 16151 raft_consensus.cc:2393] T 3884f864a96d428785895e5463e5728f P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 5: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
W20260709 13:20:02.980038 16115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 111969us
I20260709 13:20:03.012259 16115 leader_election.cc:304] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860
I20260709 13:20:03.019840 16291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4f65cdf00b748bb963ee6e81ef43b79" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
W20260709 13:20:03.021656 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 171445us
I20260709 13:20:02.986337 16151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 6 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:02.980767 18870 raft_consensus.cc:2804] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:20:03.026191 16146 raft_consensus.cc:1275] T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:20:03.048020 16151 raft_consensus.cc:2410] T ecc1e363230043fe9fef390f0b979f20 P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 for term 6 because replica has last-logged OpId of term: 2 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 3.
I20260709 13:20:03.078920 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c74490c40243a88e62d1760b5cd814" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:03.049049 16190 leader_election.cc:304] T e5ad4eb281564936a59f41e387a47156 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:20:03.083360 16146 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40844 (request call id 1932) took 2365 ms. Trace:
I20260709 13:20:03.079749 16219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "829b4ad8c5a54eababeea1181b960d4b" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:03.083906 16146 rpcz_store.cc:276] 0709 13:20:00.726423 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:03.012993 (+2286570us) service_pool.cc:224] Handling call
0709 13:20:03.025635 (+ 12642us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:03.082963 (+ 57328us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:03.083286 (+   323us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:20:02.980747 18750 raft_consensus.cc:3060] T b6f07842d3aa47858df5665874c86f3a P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:20:03.029992 16115 leader_election.cc:304] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:20:03.104750 16146 raft_consensus.cc:1275] T af57996a8be84d398f0f023e8d08ef77 P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:20:03.106446 16151 raft_consensus.cc:1275] T 8158bbd17b994d2eb96f560bd0f1b88d P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 13:20:03.107280 16151 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40830 (request call id 1175) took 1056 ms. Trace:
I20260709 13:20:03.107662 16151 rpcz_store.cc:276] 0709 13:20:02.051115 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:03.105778 (+1054663us) service_pool.cc:224] Handling call
0709 13:20:03.106095 (+   317us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:03.106971 (+   876us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:03.107208 (+   237us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:20:03.102414 16190 leader_election.cc:304] T e4dc6b0541ca431ebc757beac00521f1 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:20:03.112241 16146 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46224 (request call id 363) took 1461 ms. Trace:
I20260709 13:20:03.112783 16146 rpcz_store.cc:276] 0709 13:20:01.650911 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:03.085644 (+1434733us) service_pool.cc:224] Handling call
0709 13:20:03.104317 (+ 18673us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:03.105366 (+  1049us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:03.112142 (+  6776us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:20:03.115795 16151 raft_consensus.cc:1240] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b [term 5 LEADER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 4. Current term is 5. Ops: []
W20260709 13:20:03.120152 16115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 90276us
I20260709 13:20:03.125694 18901 raft_consensus.cc:2804] T e4dc6b0541ca431ebc757beac00521f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:20:03.126215 18901 raft_consensus.cc:493] T e4dc6b0541ca431ebc757beac00521f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:20:03.126293 18893 log.cc:927] Time spent T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860: Append to log took a long time: real 0.232s	user 0.001s	sys 0.000s
I20260709 13:20:03.126611 18901 raft_consensus.cc:3060] T e4dc6b0541ca431ebc757beac00521f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:20:03.128566 18895 log.cc:927] Time spent T 5a96373bb0e14dce8a266b2fc6c58507 P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 0.188s	user 0.000s	sys 0.000s
I20260709 13:20:03.130285 16219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1df93779209748b9860f6d893dd3de53" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:20:03.131009 18892 raft_consensus.cc:2804] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:20:03.131541 18892 raft_consensus.cc:697] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b [term 7 LEADER]: Becoming Leader. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:20:03.132390 18892 consensus_queue.cc:237] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:20:03.135917 16190 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:03.137219 18903 raft_consensus.cc:493] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:03.137321 16151 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40830 (request call id 1176) took 1025 ms. Trace:
I20260709 13:20:03.137923 16151 rpcz_store.cc:276] 0709 13:20:02.111562 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:03.114774 (+1003212us) service_pool.cc:224] Handling call
0709 13:20:03.115447 (+   673us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:03.137063 (+ 21616us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:03.137253 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:20:03.139004 18879 raft_consensus.cc:493] T 66f75124215f4ca6bef831e174a792d8 P cb82cc7bd6db46d980c75237a1402860 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:20:03.139639 18879 raft_consensus.cc:515] T 66f75124215f4ca6bef831e174a792d8 P cb82cc7bd6db46d980c75237a1402860 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:03.142068 18879 leader_election.cc:290] T 66f75124215f4ca6bef831e174a792d8 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:20:03.144040 16220 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57600 (request call id 1150) took 1600 ms. Trace:
I20260709 13:20:03.144469 16220 rpcz_store.cc:276] 0709 13:20:01.543566 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:01.543771 (+   205us) service_pool.cc:224] Handling call
0709 13:20:03.143821 (+1600050us) spinlock_profiling.cc:243] Waited 189 ms on lock 0x7b5400246b54. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:20:03.143977 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":434407296}
I20260709 13:20:03.145766 16220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af57996a8be84d398f0f023e8d08ef77" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:03.146533 16220 raft_consensus.cc:2468] T af57996a8be84d398f0f023e8d08ef77 P 1ca2725d290e47d68b1a4b609c9816ac [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 6.
I20260709 13:20:03.147662 16223 raft_consensus.cc:1240] T 19baa7bc367844c288bca8f81439c413 P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 6. Current term is 7. Ops: []
W20260709 13:20:03.149040 16822 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49896 (request call id 900) took 2023 ms (client timeout 1605 ms). Trace:
I20260709 13:20:03.149605 16151 raft_consensus.cc:1275] T 02838c86531d4f98bf58d1c22b8baf77 P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
W20260709 13:20:03.149554 16822 rpcz_store.cc:269] 0709 13:20:01.125717 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:01.333858 (+208141us) service_pool.cc:224] Handling call
0709 13:20:03.148772 (+1814914us) spinlock_profiling.cc:243] Waited 18.9 ms on lock 0x7b54005320d4. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:20:03.148951 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":43437824}
I20260709 13:20:03.021564 18753 leader_election.cc:290] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 5 election: Requested vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
W20260709 13:20:03.081533 18853 log.cc:927] Time spent T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.100s	user 0.000s	sys 0.000s
I20260709 13:20:03.019332 18747 raft_consensus.cc:2749] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:20:03.137971 18903 raft_consensus.cc:515] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:03.174620 16824 raft_consensus.cc:1240] T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860 [term 7 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 6. Current term is 7. Ops: []
I20260709 13:20:03.104750 18551 raft_consensus.cc:2804] T e5ad4eb281564936a59f41e387a47156 P 1ca2725d290e47d68b1a4b609c9816ac [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:20:03.011690 18843 raft_consensus.cc:515] T 8a878999c0024d2abe517484180290fc P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:03.261116 16291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4dc6b0541ca431ebc757beac00521f1" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:03.267027 16291 raft_consensus.cc:2468] T e4dc6b0541ca431ebc757beac00521f1 P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 5.
I20260709 13:20:03.109578 18870 raft_consensus.cc:697] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [term 8 LEADER]: Becoming Leader. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:20:03.295176 16220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c61a39fd4f84424b129c1a6f9ba404d" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:20:03.313598 16220 raft_consensus.cc:3060] T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:20:03.296013 16266 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:03.280702 18551 raft_consensus.cc:493] T e5ad4eb281564936a59f41e387a47156 P 1ca2725d290e47d68b1a4b609c9816ac [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:20:03.306908 18807 consensus_meta.cc:287] T 84f2da98864b4425bf7c0055951fdd4b P 1ca2725d290e47d68b1a4b609c9816ac: Time spent flushing consensus metadata: real 1.166s	user 0.002s	sys 0.010s
I20260709 13:20:03.363108 16291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68a6838d82ef4cac83507f906139d90f" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:20:03.364066 16291 raft_consensus.cc:3060] T 68a6838d82ef4cac83507f906139d90f P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:20:03.381902 18421 consensus_meta.cc:287] T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b: Time spent flushing consensus metadata: real 0.927s	user 0.010s	sys 0.000s
I20260709 13:20:03.333890 18868 raft_consensus.cc:493] T 84f2da98864b4425bf7c0055951fdd4b P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:20:03.355979 18551 raft_consensus.cc:3060] T e5ad4eb281564936a59f41e387a47156 P 1ca2725d290e47d68b1a4b609c9816ac [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:20:03.403885 16300 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48684 (request call id 2064) took 2699 ms. Trace:
I20260709 13:20:03.404706 18720 leader_election.cc:290] T 8d4ab870a05347868956aa8b841b2981 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:03.408542 18870 consensus_queue.cc:237] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:03.411628 18901 raft_consensus.cc:515] T e4dc6b0541ca431ebc757beac00521f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:03.425100 18832 leader_election.cc:290] T ee9ed67006144797b1de6ea8c2013a68 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:03.490022 16146 raft_consensus.cc:1240] T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b [term 8 LEADER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 7. Current term is 8. Ops: []
W20260709 13:20:03.502221 18819 consensus_meta.cc:287] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045: Time spent flushing consensus metadata: real 0.861s	user 0.006s	sys 0.006s
I20260709 13:20:03.512766 18843 leader_election.cc:290] T 8a878999c0024d2abe517484180290fc P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 6 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:20:03.520229 16145 raft_consensus.cc:1275] T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
W20260709 13:20:03.498528 16191 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.849s	user 0.000s	sys 0.029s
W20260709 13:20:03.503901 18558 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.584s	user 0.002s	sys 0.027s
W20260709 13:20:03.534212 16148 consensus_meta.cc:287] T 26354361e9b044cea1c2b86c4882fe92 P 647a343a542d4a139fe106c19095129b: Time spent flushing consensus metadata: real 0.558s	user 0.007s	sys 0.002s
I20260709 13:20:03.322676 18797 raft_consensus.cc:515] T 18ad7e3af5504571a69d10c00750a01c P 1ca2725d290e47d68b1a4b609c9816ac [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } }
I20260709 13:20:03.424019 18842 leader_election.cc:290] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
W20260709 13:20:03.503753 18114 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.580s	user 0.008s	sys 0.020s
I20260709 13:20:03.463604 18807 raft_consensus.cc:515] T 84f2da98864b4425bf7c0055951fdd4b P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:20:03.577839 16191 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.928s	user 0.000s	sys 0.044s
W20260709 13:20:03.573901 16266 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.14.177.132:38923 timed out after 2.586s (SENT) [suppressed 31 similar messages]
W20260709 13:20:03.602559 18114 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.684s	user 0.008s	sys 0.023s
I20260709 13:20:03.517148 16300 rpcz_store.cc:276] 0709 13:20:00.704144 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:00.710252 (+  6108us) service_pool.cc:224] Handling call
0709 13:20:00.730813 (+ 20561us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:20:00.731313 (+   500us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:20:00.731636 (+   323us) log.cc:844] Serialized 14 byte log entry
0709 13:20:00.990526 (+258890us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:20:01.037072 (+ 46546us) log.cc:844] Serialized 26 byte log entry
0709 13:20:01.037201 (+   129us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:20:01.037238 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:01.037301 (+    63us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:20:02.963961 (+1926660us) spinlock_profiling.cc:243] Waited 246 ms on lock 0x7b4400022b4c. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69ad8fadb1 00007f69ad998c85 00007f69ad9996a2 00007f69b070b1db 00007f69b0706aaf 00007f69b06fa86e 00007f69b06f9bb6 00007f69b06f97d8 00007f69b0703221 00007f69b07031d9 00007f69b0703169
0709 13:20:03.403502 (+439541us) raft_consensus.cc:1685] finished
0709 13:20:03.403623 (+   121us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:03.403807 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8527,"prepare.queue_time_us":5786,"prepare.run_cpu_time_us":78,"prepare.run_wall_time_us":69,"raft.queue_time_us":43408,"raft.run_cpu_time_us":248,"raft.run_wall_time_us":14758,"spinlock_wait_cycles":566465408,"thread_start_us":258344,"threads_started":1,"wal-append.queue_time_us":603946}
W20260709 13:20:03.587323 18825 consensus_meta.cc:287] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 1ca2725d290e47d68b1a4b609c9816ac: Time spent flushing consensus metadata: real 1.362s	user 0.009s	sys 0.009s
I20260709 13:20:03.611681 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e4bfb9090c8460eb14fa7e0d818e7f1" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:20:03.612506 16300 raft_consensus.cc:3060] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:20:03.589442 16150 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40844 (request call id 1918) took 3536 ms (client timeout 1915 ms). Trace:
W20260709 13:20:03.613518 16150 rpcz_store.cc:269] 0709 13:20:00.053565 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:00.053775 (+   210us) service_pool.cc:224] Handling call
0709 13:20:03.589225 (+3535450us) spinlock_profiling.cc:243] Waited 891 ms on lock 0x7b54002daad4. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:20:03.589367 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2048727808}
I20260709 13:20:03.590896 16147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a878999c0024d2abe517484180290fc" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:03.615216 16147 raft_consensus.cc:2468] T 8a878999c0024d2abe517484180290fc P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 5.
I20260709 13:20:03.523437 18421 raft_consensus.cc:493] T b87147731a68400e8cce9b879daa7c5d P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:20:03.617036 16225 raft_consensus.cc:2368] T 18ad7e3af5504571a69d10c00750a01c P 1ca2725d290e47d68b1a4b609c9816ac [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b for earlier term 5. Current term is 6.
I20260709 13:20:03.597222 16291 raft_consensus.cc:2468] T 68a6838d82ef4cac83507f906139d90f P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 8.
I20260709 13:20:03.598098 16142 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40844 (request call id 1917) took 3581 ms. Trace:
I20260709 13:20:03.598793 18901 leader_election.cc:290] T e4dc6b0541ca431ebc757beac00521f1 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 6 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
W20260709 13:20:03.598522 18770 consensus_meta.cc:287] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac: Time spent flushing consensus metadata: real 0.709s	user 0.006s	sys 0.003s
I20260709 13:20:03.579352 16148 raft_consensus.cc:2468] T 26354361e9b044cea1c2b86c4882fe92 P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 4.
I20260709 13:20:03.622458 16148 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46224 (request call id 362) took 1996 ms. Trace:
I20260709 13:20:03.622859 16148 rpcz_store.cc:276] 0709 13:20:01.626264 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:02.974262 (+1347998us) service_pool.cc:224] Handling call
0709 13:20:03.622379 (+648117us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:20:03.623951 16148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ad5dbfa99ee4b8d9e28be92b3f36db5" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
W20260709 13:20:03.579959 18558 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.665s	user 0.003s	sys 0.028s
I20260709 13:20:03.635106 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0eab25ce4c64e018a041beeda2867db" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
W20260709 13:20:03.536803 18828 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.560s	user 0.014s	sys 0.020s
I20260709 13:20:03.610663 16220 raft_consensus.cc:2468] T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 7.
I20260709 13:20:03.635150 16147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5ad4eb281564936a59f41e387a47156" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:03.638264 16220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 6 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:03.536643 18821 raft_consensus.cc:493] T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:03.595983 16223 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57604 (request call id 2102) took 1546 ms. Trace:
I20260709 13:20:03.640444 16223 rpcz_store.cc:276] 0709 13:20:02.049212 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:02.049310 (+    98us) service_pool.cc:224] Handling call
0709 13:20:03.147418 (+1098108us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:03.148185 (+   767us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:03.595766 (+447581us) spinlock_profiling.cc:243] Waited 1.04 s on lock 0x7b5400246b50. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5e8ad 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:20:03.595921 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2382736256}
I20260709 13:20:03.618124 16291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:20:03.618014 16142 rpcz_store.cc:276] 0709 13:20:00.016611 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:00.016760 (+   149us) service_pool.cc:224] Handling call
0709 13:20:00.017091 (+   331us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:00.767830 (+750739us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:03.597857 (+2830027us) spinlock_profiling.cc:243] Waited 750 ms on lock 0x7b540012f7d4. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:20:03.598022 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1725056384}
I20260709 13:20:03.521826 18868 raft_consensus.cc:515] T 84f2da98864b4425bf7c0055951fdd4b P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:20:03.619948 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 422333us
W20260709 13:20:03.626241 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 1.01396e+06us
I20260709 13:20:03.630617 16148 raft_consensus.cc:1240] T 8cc959fb7f704fba9009c0595467d074 P 647a343a542d4a139fe106c19095129b [term 6 LEADER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 5. Current term is 6. Ops: []
I20260709 13:20:03.636529 16224 raft_consensus.cc:1240] T 84f2da98864b4425bf7c0055951fdd4b P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 3. Current term is 4. Ops: []
I20260709 13:20:03.635833 16150 raft_consensus.cc:2376] T c0eab25ce4c64e018a041beeda2867db P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 6. Re-sending same reply.
I20260709 13:20:03.604910 18842 raft_consensus.cc:493] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:20:03.638914 16220 raft_consensus.cc:2468] T ecc1e363230043fe9fef390f0b979f20 P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 5.
I20260709 13:20:03.641515 16291 raft_consensus.cc:3060] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:20:03.535636 18819 raft_consensus.cc:515] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:03.670509 16220 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57604 (request call id 2117) took 1247 ms. Trace:
I20260709 13:20:03.681087 16220 rpcz_store.cc:276] 0709 13:20:02.422727 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:03.638012 (+1215285us) service_pool.cc:224] Handling call
0709 13:20:03.670439 (+ 32427us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:20:03.671782 16190 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:03.671985 16148 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40830 (request call id 1184) took 1397 ms. Trace:
I20260709 13:20:03.673580 18807 leader_election.cc:290] T 84f2da98864b4425bf7c0055951fdd4b P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 4 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
W20260709 13:20:03.674517 18912 log.cc:927] Time spent T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 0.093s	user 0.000s	sys 0.000s
I20260709 13:20:03.675654 16223 raft_consensus.cc:1240] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 4. Current term is 5. Ops: []
I20260709 13:20:03.679847 16147 raft_consensus.cc:1240] T 5a959dbecc2648eb85d1cc550f5a3245 P 647a343a542d4a139fe106c19095129b [term 5 LEADER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 4. Current term is 5. Ops: []
I20260709 13:20:03.682303 16148 rpcz_store.cc:276] 0709 13:20:02.274346 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:03.629981 (+1355635us) service_pool.cc:224] Handling call
0709 13:20:03.630378 (+   397us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:03.671739 (+ 41361us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:03.671932 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:20:03.693239 16223 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33050 (request call id 494) took 1435 ms. Trace:
I20260709 13:20:03.693358 16147 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40830 (request call id 1186) took 1418 ms. Trace:
I20260709 13:20:03.693792 16147 rpcz_store.cc:276] 0709 13:20:02.274902 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:03.639269 (+1364367us) service_pool.cc:224] Handling call
0709 13:20:03.679678 (+ 40409us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:03.693118 (+ 13440us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:03.693310 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:20:03.637485 18828 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.661s	user 0.015s	sys 0.020s
I20260709 13:20:03.693598 16223 rpcz_store.cc:276] 0709 13:20:02.257661 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:03.675211 (+1417550us) service_pool.cc:224] Handling call
0709 13:20:03.675503 (+   292us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:03.692985 (+ 17482us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:03.693182 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:20:03.697746 16220 raft_consensus.cc:1240] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 7. Current term is 8. Ops: [7.7-7.7]
I20260709 13:20:03.698148 18855 raft_consensus.cc:493] T 9e65676b21b748728ad39668cfe722ec P cb82cc7bd6db46d980c75237a1402860 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:20:03.698477 16220 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33050 (request call id 495) took 1440 ms. Trace:
I20260709 13:20:03.698608 18855 raft_consensus.cc:3060] T 9e65676b21b748728ad39668cfe722ec P cb82cc7bd6db46d980c75237a1402860 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:20:03.698899 16220 rpcz_store.cc:276] 0709 13:20:02.257952 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:03.681937 (+1423985us) service_pool.cc:224] Handling call
0709 13:20:03.697512 (+ 15575us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:20:03.698197 (+   685us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:03.698426 (+   229us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:20:03.703115 18797 leader_election.cc:290] T 18ad7e3af5504571a69d10c00750a01c P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 6 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:03.703933 18842 raft_consensus.cc:515] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:03.704535 16295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1df93779209748b9860f6d893dd3de53" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:20:03.705114 16295 raft_consensus.cc:3060] T 1df93779209748b9860f6d893dd3de53 P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:20:03.706019 16291 raft_consensus.cc:2468] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 7.
I20260709 13:20:03.706250 18842 leader_election.cc:290] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:20:03.706982 16291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a878999c0024d2abe517484180290fc" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:03.697746 16150 raft_consensus.cc:1240] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 5. Current term is 6. Ops: []
I20260709 13:20:03.698647 18903 leader_election.cc:290] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:03.707898 16150 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46224 (request call id 370) took 1327 ms. Trace:
I20260709 13:20:03.708186 16145 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40844 (request call id 1924) took 3379 ms. Trace:
I20260709 13:20:03.708304 16150 rpcz_store.cc:276] 0709 13:20:02.380480 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:03.697354 (+1316874us) service_pool.cc:224] Handling call
0709 13:20:03.697587 (+   233us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:03.707658 (+ 10071us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:03.707843 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:20:03.708598 16145 rpcz_store.cc:276] 0709 13:20:00.328825 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:00.328993 (+   168us) service_pool.cc:224] Handling call
0709 13:20:00.366018 (+ 37025us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:03.536312 (+3170294us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:03.707964 (+171652us) spinlock_profiling.cc:243] Waited 3.15 s on lock 0x7b5400073fd4. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:20:03.708123 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":7252930304}
I20260709 13:20:03.709389 16292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d4ab870a05347868956aa8b841b2981" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 7 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:03.710013 16292 raft_consensus.cc:2468] T 8d4ab870a05347868956aa8b841b2981 P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 6.
I20260709 13:20:03.711773 16820 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49906 (request call id 592) took 1638 ms. Trace:
I20260709 13:20:03.712147 16820 rpcz_store.cc:276] 0709 13:20:02.072963 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:02.104075 (+ 31112us) service_pool.cc:224] Handling call
0709 13:20:03.711587 (+1607512us) spinlock_profiling.cc:243] Waited 492 ms on lock 0x7b54004b8cd4. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:20:03.711714 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1132742528}
I20260709 13:20:03.710994 16292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a96373bb0e14dce8a266b2fc6c58507" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:20:03.715830 16292 raft_consensus.cc:3060] T 5a96373bb0e14dce8a266b2fc6c58507 P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:20:03.716307 16146 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40830 (request call id 1180) took 1530 ms. Trace:
I20260709 13:20:03.716717 16146 rpcz_store.cc:276] 0709 13:20:02.185624 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:03.113757 (+928133us) service_pool.cc:224] Handling call
0709 13:20:03.114139 (+   382us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:03.527338 (+413199us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:03.716034 (+188696us) spinlock_profiling.cc:243] Waited 376 ms on lock 0x7b54000d1454. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:20:03.716246 (+   212us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":863859968}
I20260709 13:20:03.718837 16226 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57604 (request call id 2115) took 1296 ms. Trace:
I20260709 13:20:03.719276 16226 rpcz_store.cc:276] 0709 13:20:02.422075 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:02.436741 (+ 14666us) service_pool.cc:224] Handling call
0709 13:20:03.718640 (+1281899us) spinlock_profiling.cc:243] Waited 261 ms on lock 0x7b5400140fd4. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:20:03.718778 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":599330688}
I20260709 13:20:03.719542 16291 raft_consensus.cc:2393] T 8a878999c0024d2abe517484180290fc P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 6: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
I20260709 13:20:03.721001 16291 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48674 (request call id 2193) took 1105 ms. Trace:
I20260709 13:20:03.721312 16291 rpcz_store.cc:276] 0709 13:20:02.615964 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:03.706743 (+1090779us) service_pool.cc:224] Handling call
0709 13:20:03.720948 (+ 14205us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:20:03.722289 16291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c61a39fd4f84424b129c1a6f9ba404d" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:20:03.722457 16218 raft_consensus.cc:1275] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 13:20:03.722836 16291 raft_consensus.cc:3060] T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:20:03.723142 16218 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57604 (request call id 2120) took 1292 ms. Trace:
I20260709 13:20:03.723558 16218 rpcz_store.cc:276] 0709 13:20:02.430586 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:03.721781 (+1291195us) service_pool.cc:224] Handling call
0709 13:20:03.722112 (+   331us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:03.722890 (+   778us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:03.723086 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:20:03.725402 16818 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49906 (request call id 587) took 3006 ms. Trace:
W20260709 13:20:03.727746 18869 consensus_meta.cc:287] T 1df93779209748b9860f6d893dd3de53 P 1ca2725d290e47d68b1a4b609c9816ac: Time spent flushing consensus metadata: real 1.109s	user 0.014s	sys 0.000s
I20260709 13:20:03.728296 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87ac0d1f7e8a419f9bcbc04aee0e067e" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:03.728186 18869 raft_consensus.cc:515] T 1df93779209748b9860f6d893dd3de53 P 1ca2725d290e47d68b1a4b609c9816ac [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:03.728901 16298 raft_consensus.cc:2468] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 6.
I20260709 13:20:03.729368 16219 raft_consensus.cc:2393] T 1df93779209748b9860f6d893dd3de53 P 1ca2725d290e47d68b1a4b609c9816ac [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 6: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
I20260709 13:20:03.730037 16298 raft_consensus.cc:1240] T 57a744f5aaf944678a63469cbf514015 P 90f3da57796b488c9087a2f523adf045 [term 4 LEADER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 3. Current term is 4. Ops: []
I20260709 13:20:03.730372 18869 leader_election.cc:290] T 1df93779209748b9860f6d893dd3de53 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 6 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:03.730671 16298 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48674 (request call id 2191) took 1115 ms. Trace:
I20260709 13:20:03.688891 16142 raft_consensus.cc:1240] T fffab26427e24d398e2695aad40ca134 P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 4. Current term is 5. Ops: []
I20260709 13:20:03.763833 16142 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40830 (request call id 1190) took 1487 ms. Trace:
I20260709 13:20:03.764230 16142 rpcz_store.cc:276] 0709 13:20:02.276056 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:03.641643 (+1365587us) service_pool.cc:224] Handling call
0709 13:20:03.688722 (+ 47079us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:03.763540 (+ 74818us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:03.763764 (+   224us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:20:03.642853 18850 log.cc:927] Time spent T 829b4ad8c5a54eababeea1181b960d4b P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 2.290s	user 0.013s	sys 0.000s
I20260709 13:20:03.669533 18558 raft_consensus.cc:493] T 23f82224f4564fea93572e8206bad1fe P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:03.686724 16300 raft_consensus.cc:2468] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 5.
I20260709 13:20:03.738500 16116 leader_election.cc:304] T 08c74490c40243a88e62d1760b5cd814 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:20:03.739451 16291 raft_consensus.cc:2468] T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 7.
I20260709 13:20:03.739759 16824 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49906 (request call id 586) took 3021 ms. Trace:
I20260709 13:20:03.791671 16824 rpcz_store.cc:276] 0709 13:20:00.718535 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:01.532431 (+813896us) service_pool.cc:224] Handling call
0709 13:20:03.150960 (+1618529us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:03.265841 (+114881us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:03.739559 (+473718us) spinlock_profiling.cc:243] Waited 1.61 s on lock 0x7b54005320d0. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5e8ad 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:20:03.739697 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3699804032}
I20260709 13:20:03.740865 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5718c623ebee4d2bb770f0f9ddb2f164" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
W20260709 13:20:03.793520 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 55125.4us
I20260709 13:20:03.794445 16116 leader_election.cc:304] T af57996a8be84d398f0f023e8d08ef77 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
I20260709 13:20:03.743023 18868 leader_election.cc:290] T 84f2da98864b4425bf7c0055951fdd4b P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:20:03.643621 18821 raft_consensus.cc:515] T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:03.746970 18551 raft_consensus.cc:515] T e5ad4eb281564936a59f41e387a47156 P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:03.621891 18421 raft_consensus.cc:515] T b87147731a68400e8cce9b879daa7c5d P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:03.800024 16219 raft_consensus.cc:1240] T e5ad4eb281564936a59f41e387a47156 P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 6. Current term is 7. Ops: [6.5-6.5]
I20260709 13:20:03.800449 16291 raft_consensus.cc:1275] T e5ad4eb281564936a59f41e387a47156 P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
W20260709 13:20:03.754144 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 72616.6us
I20260709 13:20:03.725813 16818 rpcz_store.cc:276] 0709 13:20:00.718911 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:01.546138 (+827227us) service_pool.cc:224] Handling call
0709 13:20:01.546616 (+   478us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:02.072710 (+526094us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:20:02.110118 (+ 37408us) log.cc:844] Serialized 14 byte log entry
0709 13:20:02.185376 (+ 75258us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:20:02.185430 (+    54us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:20:02.185469 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:03.725169 (+1539700us) spinlock_profiling.cc:243] Waited 526 ms on lock 0x7b54007c0e54. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:20:03.725218 (+    49us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:03.725343 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4743,"prepare.queue_time_us":33712,"prepare.run_cpu_time_us":31,"prepare.run_wall_time_us":31,"raft.queue_time_us":9472,"raft.run_cpu_time_us":124,"raft.run_wall_time_us":8787,"spinlock_wait_cycles":1209453312,"thread_start_us":107210,"threads_started":2,"wal-append.queue_time_us":158522,"wal-append.run_cpu_time_us":2193,"wal-append.run_wall_time_us":1180201}
I20260709 13:20:03.802066 16191 leader_election.cc:304] T 5a959dbecc2648eb85d1cc550f5a3245 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:20:03.707971 18903 raft_consensus.cc:493] T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:03.763408 16298 rpcz_store.cc:276] 0709 13:20:02.615378 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:03.729568 (+1114190us) service_pool.cc:224] Handling call
0709 13:20:03.729869 (+   301us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:03.730433 (+   564us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:03.730617 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:20:03.787971 16147 raft_consensus.cc:1240] T 2794af5b99984efebdee564035841cb0 P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 4. Current term is 5. Ops: []
I20260709 13:20:03.804967 16147 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46224 (request call id 369) took 1438 ms. Trace:
W20260709 13:20:03.671299 16790 leader_election.cc:336] T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943): Timed out: RequestConsensusVote RPC to 127.14.177.130:35943 timed out after 1.649s (SENT)
I20260709 13:20:03.794610 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d4ab870a05347868956aa8b841b2981" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 7 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:03.796267 18768 raft_consensus.cc:2749] T 08c74490c40243a88e62d1760b5cd814 P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:20:03.796852 18892 raft_consensus.cc:2804] T af57996a8be84d398f0f023e8d08ef77 P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:20:03.795714 16116 leader_election.cc:304] T 9c61a39fd4f84424b129c1a6f9ba404d P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
W20260709 13:20:03.902657 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 107030us
I20260709 13:20:03.904418 16116 leader_election.cc:304] T fffab26427e24d398e2695aad40ca134 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 90f3da57796b488c9087a2f523adf045
I20260709 13:20:03.637068 18770 raft_consensus.cc:515] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:03.805120 16298 raft_consensus.cc:1240] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 6. Current term is 7. Ops: []
I20260709 13:20:03.805594 16790 leader_election.cc:304] T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:20:03.912034 16298 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48674 (request call id 2199) took 1294 ms. Trace:
W20260709 13:20:03.912495 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 241225us
I20260709 13:20:03.912957 16298 rpcz_store.cc:276] 0709 13:20:02.617519 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:03.804503 (+1186984us) service_pool.cc:224] Handling call
0709 13:20:03.804898 (+   395us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:03.911502 (+106604us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:03.911902 (+   400us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:20:03.913692 18769 raft_consensus.cc:2749] T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:20:03.915253 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 113272us
W20260709 13:20:03.900125 18750 consensus_meta.cc:287] T b6f07842d3aa47858df5665874c86f3a P 1ca2725d290e47d68b1a4b609c9816ac: Time spent flushing consensus metadata: real 0.781s	user 0.016s	sys 0.000s
I20260709 13:20:03.815961 16223 raft_consensus.cc:1275] T af57996a8be84d398f0f023e8d08ef77 P 1ca2725d290e47d68b1a4b609c9816ac [term 6 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:20:03.805459 16147 rpcz_store.cc:276] 0709 13:20:02.366525 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:03.694848 (+1328323us) service_pool.cc:224] Handling call
0709 13:20:03.782222 (+ 87374us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:03.804721 (+ 22499us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:03.804915 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:20:03.787952 16300 raft_consensus.cc:1275] T d16dfad0c3b540e5b4b86492ddf87208 P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
W20260709 13:20:03.917443 18743 consensus_meta.cc:287] T 26354361e9b044cea1c2b86c4882fe92 P 90f3da57796b488c9087a2f523adf045: Time spent flushing consensus metadata: real 1.159s	user 0.009s	sys 0.006s
I20260709 13:20:03.902658 18868 raft_consensus.cc:493] T 23f82224f4564fea93572e8206bad1fe P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:03.805938 16217 raft_consensus.cc:2468] T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 6.
I20260709 13:20:03.938650 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee9ed67006144797b1de6ea8c2013a68" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:03.922322 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada5a004ba7c46819051e3ccec71ffdb" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:20:03.694016 18901 consensus_queue.cc:1048] T 08c74490c40243a88e62d1760b5cd814 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.208s
I20260709 13:20:03.787585 18825 raft_consensus.cc:493] T 8cc959fb7f704fba9009c0595467d074 P 1ca2725d290e47d68b1a4b609c9816ac [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:03.791186 18828 raft_consensus.cc:493] T d2ab0ac8347641e6955e9bce7d90c1a2 P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:20:03.929415 16300 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48674 (request call id 2192) took 1313 ms. Trace:
I20260709 13:20:03.968160 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a9de347c711477e87c582d003f36925" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:03.968461 16300 rpcz_store.cc:276] 0709 13:20:02.615679 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:03.769269 (+1153590us) service_pool.cc:224] Handling call
0709 13:20:03.787617 (+ 18348us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:03.929078 (+141461us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:03.929339 (+   261us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:20:03.901912 16291 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48674 (request call id 2195) took 1184 ms. Trace:
I20260709 13:20:03.929512 16191 leader_election.cc:304] T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 90f3da57796b488c9087a2f523adf045
W20260709 13:20:03.918531 16790 leader_election.cc:336] T 8c11074579b046d99e585c2818e18a2c P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943): Timed out: RequestConsensusVote RPC to 127.14.177.130:35943 timed out after 1.750s (SENT)
W20260709 13:20:03.933688 16115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 253609us
I20260709 13:20:03.910902 18892 raft_consensus.cc:493] T af57996a8be84d398f0f023e8d08ef77 P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:03.935571 16223 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33050 (request call id 496) took 1670 ms. Trace:
I20260709 13:20:03.972430 16223 rpcz_store.cc:276] 0709 13:20:02.258227 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:03.696775 (+1438548us) service_pool.cc:224] Handling call
0709 13:20:03.766569 (+ 69794us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:03.928473 (+161904us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:03.928716 (+   243us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:20:03.923491 18857 raft_consensus.cc:2804] T 9c61a39fd4f84424b129c1a6f9ba404d P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:20:03.974414 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8638ee363273426db1393dd81e5c7b72" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:03.925829 18768 raft_consensus.cc:2804] T fffab26427e24d398e2695aad40ca134 P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:20:03.955112 16292 raft_consensus.cc:2410] T 5a96373bb0e14dce8a266b2fc6c58507 P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b for term 5 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:20:03.948228 16116 leader_election.cc:304] T 8d4ab870a05347868956aa8b841b2981 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
I20260709 13:20:03.940115 16217 raft_consensus.cc:1275] T 5a96373bb0e14dce8a266b2fc6c58507 P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 5 index: 2. (index mismatch)
W20260709 13:20:04.020726 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 72587.1us
I20260709 13:20:03.807384 18558 raft_consensus.cc:515] T 23f82224f4564fea93572e8206bad1fe P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:20:03.964031 16265 leader_election.cc:336] T 8638ee363273426db1393dd81e5c7b72 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943): Timed out: RequestConsensusVote RPC to 127.14.177.130:35943 timed out after 2.104s (SENT)
I20260709 13:20:04.022735 16265 leader_election.cc:304] T 8638ee363273426db1393dd81e5c7b72 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860
W20260709 13:20:03.970041 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 51539.7us
I20260709 13:20:03.970870 16295 raft_consensus.cc:2468] T 1df93779209748b9860f6d893dd3de53 P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 6.
I20260709 13:20:03.971025 16115 leader_election.cc:304] T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac
I20260709 13:20:03.933033 16817 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49906 (request call id 582) took 3962 ms. Trace:
I20260709 13:20:03.969370 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7ca61592e848ea9b0fb81a2ce33cdc" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:03.969348 16291 rpcz_store.cc:276] 0709 13:20:02.616553 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:03.791902 (+1175349us) service_pool.cc:224] Handling call
0709 13:20:03.792200 (+   298us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:03.800863 (+  8663us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:03.801077 (+   214us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:20:04.026571 16115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 55641.2us
I20260709 13:20:04.027256 16291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74ca8140f35c4ae29e8a664f6bdfc829" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 7 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
W20260709 13:20:04.024274 16790 leader_election.cc:336] T af57996a8be84d398f0f023e8d08ef77 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943): Timed out: RequestConsensusVote RPC to 127.14.177.130:35943 timed out after 1.952s (SENT)
I20260709 13:20:03.917827 18797 raft_consensus.cc:2749] T 5a959dbecc2648eb85d1cc550f5a3245 P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:20:03.978809 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 56239.6us
I20260709 13:20:04.027915 16115 leader_election.cc:304] T 8a878999c0024d2abe517484180290fc P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
W20260709 13:20:03.994225 16266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 420327us
I20260709 13:20:04.027963 16291 raft_consensus.cc:2468] T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 6.
I20260709 13:20:04.029626 16300 raft_consensus.cc:2468] T 8638ee363273426db1393dd81e5c7b72 P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 5.
I20260709 13:20:04.029956 16291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 6 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
W20260709 13:20:04.030563 16266 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:04.031558 16298 raft_consensus.cc:2393] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 in current term 8: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
I20260709 13:20:04.030797 16291 raft_consensus.cc:2468] T ecc1e363230043fe9fef390f0b979f20 P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 5.
I20260709 13:20:04.028616 16191 leader_election.cc:304] T ee9ed67006144797b1de6ea8c2013a68 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
I20260709 13:20:04.033754 16191 leader_election.cc:304] T ecc1e363230043fe9fef390f0b979f20 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
I20260709 13:20:04.032070 16190 leader_election.cc:304] T 68a6838d82ef4cac83507f906139d90f P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:20:03.911811 18903 raft_consensus.cc:515] T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:03.968844 16299 raft_consensus.cc:2468] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 7.
I20260709 13:20:03.922367 18858 consensus_queue.cc:1059] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.044s
I20260709 13:20:03.925534 18781 consensus_queue.cc:1048] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.044s
I20260709 13:20:04.024237 16292 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48674 (request call id 2197) took 1407 ms. Trace:
I20260709 13:20:04.041836 16295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e9b2bf7011f4ae688a79d7bb5a447b4" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:04.019105 18662 raft_consensus.cc:493] T 806d7690c2b644959152074bb2cab0a7 P cb82cc7bd6db46d980c75237a1402860 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:20:04.027307 18869 raft_consensus.cc:2804] T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:20:04.043077 16300 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48674 (request call id 2200) took 1425 ms. Trace:
I20260709 13:20:03.944998 18743 raft_consensus.cc:515] T 26354361e9b044cea1c2b86c4882fe92 P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:20:04.044615 16297 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48674 (request call id 2180) took 1626 ms (client timeout 1988 ms). Trace:
I20260709 13:20:03.975613 18892 raft_consensus.cc:3060] T af57996a8be84d398f0f023e8d08ef77 P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:20:03.960319 18868 raft_consensus.cc:515] T 23f82224f4564fea93572e8206bad1fe P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:04.022593 18807 raft_consensus.cc:493] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:04.042200 16115 leader_election.cc:304] T 1df93779209748b9860f6d893dd3de53 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac
I20260709 13:20:03.995335 18825 raft_consensus.cc:515] T 8cc959fb7f704fba9009c0595467d074 P 1ca2725d290e47d68b1a4b609c9816ac [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:04.052070 16300 rpcz_store.cc:276] 0709 13:20:02.617759 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:04.030345 (+1412586us) service_pool.cc:224] Handling call
0709 13:20:04.042620 (+ 12275us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:04.042789 (+   169us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:04.042812 (+    23us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:20:04.042851 (+    39us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:20:04.042881 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:04.042930 (+    49us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:04.043027 (+    97us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:20:04.029381 18785 raft_consensus.cc:493] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:20:04.026837 18918 raft_consensus.cc:2804] T 8d4ab870a05347868956aa8b841b2981 P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:20:04.026121 18857 raft_consensus.cc:697] T 9c61a39fd4f84424b129c1a6f9ba404d P 647a343a542d4a139fe106c19095129b [term 7 LEADER]: Becoming Leader. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:20:04.026146 18769 raft_consensus.cc:493] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:20:03.954239 18750 raft_consensus.cc:515] T b6f07842d3aa47858df5665874c86f3a P 1ca2725d290e47d68b1a4b609c9816ac [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:04.038873 18834 raft_consensus.cc:2804] T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 13:20:04.184762 18841 log.cc:927] Time spent T 7c2095a5f4ad49408dbc7f69c606120d P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 3.508s	user 0.009s	sys 0.025s
I20260709 13:20:04.233381 16818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a378d7f796f94615b3b347119a9880fd" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:04.234366 16818 raft_consensus.cc:2468] T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 4.
I20260709 13:20:04.047343 16292 rpcz_store.cc:276] 0709 13:20:02.617071 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:03.710783 (+1093712us) service_pool.cc:224] Handling call
0709 13:20:04.024148 (+313365us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:20:04.039320 18114 raft_consensus.cc:2749] T ee9ed67006144797b1de6ea8c2013a68 P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:20:04.176721 16297 rpcz_store.cc:269] 0709 13:20:02.417726 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:02.419622 (+  1896us) service_pool.cc:224] Handling call
0709 13:20:04.044437 (+1624815us) spinlock_profiling.cc:243] Waited 360 ms on lock 0x7b54000a9954. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:20:04.044551 (+   114us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":828150144}
I20260709 13:20:04.186509 18896 raft_consensus.cc:2804] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:20:04.197928 18918 raft_consensus.cc:493] T 8d4ab870a05347868956aa8b841b2981 P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:20:04.192269 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 168007us
W20260709 13:20:04.120946 18852 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 2.686s	user 0.013s	sys 0.036s
I20260709 13:20:04.205415 16300 raft_consensus.cc:3055] T 8cc959fb7f704fba9009c0595467d074 P 90f3da57796b488c9087a2f523adf045 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:20:04.047502 18919 raft_consensus.cc:2749] T 8638ee363273426db1393dd81e5c7b72 P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:20:04.053889 18858 consensus_queue.cc:1243] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [LEADER]: Peer 1ca2725d290e47d68b1a4b609c9816ac log is divergent from this leader: its last log entry 5.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 13:20:04.045552 18720 raft_consensus.cc:3060] T 8a878999c0024d2abe517484180290fc P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:20:04.175562 18836 raft_consensus.cc:493] T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:04.220491 18834 raft_consensus.cc:493] T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:04.175809 18797 raft_consensus.cc:2804] T 68a6838d82ef4cac83507f906139d90f P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Leader election won for term 8
W20260709 13:20:04.220005 16192 connection.cc:256] RPC call timeout handler was delayed by -1.9741642479998518s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1775s for a response
W20260709 13:20:04.280452 16192 connection.cc:256] RPC call timeout handler was delayed by -1.9454621909999332s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1777s for a response
I20260709 13:20:04.183612 18807 raft_consensus.cc:515] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:04.281781 16192 leader_election.cc:304] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:20:04.047406 18819 leader_election.cc:290] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 8 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:20:04.222114 16296 raft_consensus.cc:2368] T 26354361e9b044cea1c2b86c4882fe92 P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b for earlier term 4. Current term is 5.
I20260709 13:20:03.976452 18828 raft_consensus.cc:515] T d2ab0ac8347641e6955e9bce7d90c1a2 P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:20:04.224524 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 276010us
I20260709 13:20:04.198163 18785 raft_consensus.cc:515] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:04.242015 16824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74ca8140f35c4ae29e8a664f6bdfc829" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:04.302625 16824 raft_consensus.cc:2468] T 74ca8140f35c4ae29e8a664f6bdfc829 P cb82cc7bd6db46d980c75237a1402860 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 6.
I20260709 13:20:04.047727 18745 consensus_queue.cc:1059] T 57a744f5aaf944678a63469cbf514015 P 647a343a542d4a139fe106c19095129b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.177s
I20260709 13:20:04.245087 16297 raft_consensus.cc:1240] T 68a6838d82ef4cac83507f906139d90f P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 7. Current term is 8. Ops: [7.5-7.5]
I20260709 13:20:04.037639 18768 raft_consensus.cc:697] T fffab26427e24d398e2695aad40ca134 P 647a343a542d4a139fe106c19095129b [term 5 LEADER]: Becoming Leader. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:20:04.040298 18551 leader_election.cc:290] T e5ad4eb281564936a59f41e387a47156 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 7 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:04.047924 18832 consensus_queue.cc:1048] T d16dfad0c3b540e5b4b86492ddf87208 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.172s
I20260709 13:20:04.026367 16817 rpcz_store.cc:276] 0709 13:19:59.966771 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:00.312224 (+345453us) service_pool.cc:224] Handling call
0709 13:20:00.376516 (+ 64292us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:01.134755 (+758239us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:03.911832 (+2777077us) spinlock_profiling.cc:243] Waited 656 ms on lock 0x7b54005519d4. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:20:03.929152 (+ 17320us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1508663680}
I20260709 13:20:04.249523 16820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c11074579b046d99e585c2818e18a2c" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:04.279474 18858 raft_consensus.cc:2804] T 1df93779209748b9860f6d893dd3de53 P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:20:04.251963 16292 raft_consensus.cc:3055] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:20:04.321795 16292 raft_consensus.cc:740] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:20:04.322912 16292 consensus_queue.cc:260] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.5, Last appended by leader: 5, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:04.324347 16292 raft_consensus.cc:3060] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:20:04.254391 16226 raft_consensus.cc:1240] T b6f07842d3aa47858df5665874c86f3a P 1ca2725d290e47d68b1a4b609c9816ac [term 6 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 5. Current term is 6. Ops: []
I20260709 13:20:04.262665 16300 raft_consensus.cc:740] T 8cc959fb7f704fba9009c0595467d074 P 90f3da57796b488c9087a2f523adf045 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
W20260709 13:20:04.257870 16790 leader_election.cc:336] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943): Timed out: RequestConsensusVote RPC to 127.14.177.130:35943 timed out after 2.037s (SENT)
I20260709 13:20:04.330837 16790 leader_election.cc:304] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:20:04.206272 18869 raft_consensus.cc:697] T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac [term 6 LEADER]: Becoming Leader. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:20:04.039564 18917 consensus_queue.cc:1048] T 5a96373bb0e14dce8a266b2fc6c58507 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.160s
W20260709 13:20:04.262375 18852 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 2.899s	user 0.014s	sys 0.039s
I20260709 13:20:04.173760 18662 raft_consensus.cc:515] T 806d7690c2b644959152074bb2cab0a7 P cb82cc7bd6db46d980c75237a1402860 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:04.243175 18421 leader_election.cc:290] T b87147731a68400e8cce9b879daa7c5d P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
W20260709 13:20:04.296663 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 264648us
I20260709 13:20:04.180497 18905 raft_consensus.cc:2749] T ecc1e363230043fe9fef390f0b979f20 P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:20:04.275578 18906 consensus_queue.cc:1059] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.025s
I20260709 13:20:04.264437 18918 raft_consensus.cc:3060] T 8d4ab870a05347868956aa8b841b2981 P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:20:04.279145 18836 raft_consensus.cc:515] T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:04.220850 18769 raft_consensus.cc:515] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:04.281152 18834 raft_consensus.cc:3060] T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:20:04.250207 16818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e4bfb9090c8460eb14fa7e0d818e7f1" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
W20260709 13:20:04.318600 16266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 278479us
I20260709 13:20:04.296514 18770 leader_election.cc:290] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 7 election: Requested vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:04.339653 16190 leader_election.cc:304] T 19baa7bc367844c288bca8f81439c413 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:20:04.328496 16300 consensus_queue.cc:260] T 8cc959fb7f704fba9009c0595467d074 P 90f3da57796b488c9087a2f523adf045 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:04.352268 16300 raft_consensus.cc:3060] T 8cc959fb7f704fba9009c0595467d074 P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:20:04.328405 18924 log.cc:927] Time spent T d16dfad0c3b540e5b4b86492ddf87208 P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 0.152s	user 0.000s	sys 0.001s
W20260709 13:20:04.350195 16264 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:20:04.352684 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 94859us
I20260709 13:20:04.351410 16150 raft_consensus.cc:1275] T 23f82224f4564fea93572e8206bad1fe P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 5. (term mismatch)
I20260709 13:20:04.353652 16150 pending_rounds.cc:85] T 23f82224f4564fea93572e8206bad1fe P 647a343a542d4a139fe106c19095129b: Aborting all ops after (but not including) 4
I20260709 13:20:04.354089 16150 pending_rounds.cc:107] T 23f82224f4564fea93572e8206bad1fe P 647a343a542d4a139fe106c19095129b: Aborting uncommitted NO_OP operation due to leader change: 3.5
I20260709 13:20:04.354499 16150 raft_consensus.cc:2889] T 23f82224f4564fea93572e8206bad1fe P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:20:04.351020 16149 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46224 (request call id 360) took 2755 ms (client timeout 2203 ms). Trace:
W20260709 13:20:04.355785 16149 rpcz_store.cc:269] 0709 13:20:01.595476 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:02.173608 (+578132us) service_pool.cc:224] Handling call
0709 13:20:04.350764 (+2177156us) spinlock_profiling.cc:243] Waited 233 ms on lock 0x7b54000749d4. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:20:04.350947 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":535640448}
I20260709 13:20:04.334069 18745 consensus_queue.cc:1243] T 57a744f5aaf944678a63469cbf514015 P 647a343a542d4a139fe106c19095129b [LEADER]: Peer 90f3da57796b488c9087a2f523adf045 log is divergent from this leader: its last log entry 4.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 13:20:04.360846 16190 leader_election.cc:304] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:20:04.361919 18905 raft_consensus.cc:2804] T 19baa7bc367844c288bca8f81439c413 P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:20:04.362532 16220 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57604 (request call id 2118) took 1932 ms. Trace:
I20260709 13:20:04.363106 16190 leader_election.cc:304] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:20:04.334483 18858 raft_consensus.cc:697] T 1df93779209748b9860f6d893dd3de53 P 647a343a542d4a139fe106c19095129b [term 6 LEADER]: Becoming Leader. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:20:04.362927 16220 rpcz_store.cc:276] 0709 13:20:02.430034 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:03.699724 (+1269690us) service_pool.cc:224] Handling call
0709 13:20:03.700085 (+   361us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:20:03.707848 (+  7763us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:03.707874 (+    26us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:20:03.708343 (+   469us) log.cc:844] Serialized 26 byte log entry
0709 13:20:03.930215 (+221872us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:20:03.930272 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:03.930359 (+    87us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:20:04.362295 (+431936us) raft_consensus.cc:1685] finished
0709 13:20:04.362319 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:04.362471 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":7216,"raft.run_cpu_time_us":77,"raft.run_wall_time_us":78,"thread_start_us":221397,"threads_started":1,"wal-append.queue_time_us":317530}
I20260709 13:20:04.340472 18551 raft_consensus.cc:2804] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 13:20:04.367177 16190 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:20:04.373831 16115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 331672us
I20260709 13:20:04.357718 16292 raft_consensus.cc:1275] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 5 index: 5. (term mismatch)
I20260709 13:20:04.376415 16292 pending_rounds.cc:85] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045: Aborting all ops after (but not including) 4
W20260709 13:20:04.367826 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 334132us
I20260709 13:20:04.377331 18551 raft_consensus.cc:493] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:04.378638 18769 leader_election.cc:290] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:04.343650 18923 raft_consensus.cc:2749] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:20:04.380060 16191 leader_election.cc:304] T 7c2095a5f4ad49408dbc7f69c606120d P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:20:04.387435 16300 raft_consensus.cc:1275] T 8cc959fb7f704fba9009c0595467d074 P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
W20260709 13:20:04.389338 16192 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 96401.4us
I20260709 13:20:04.404747 16300 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48674 (request call id 2205) took 1191 ms. Trace:
I20260709 13:20:04.405164 16300 rpcz_store.cc:276] 0709 13:20:03.213666 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:04.179072 (+965406us) service_pool.cc:224] Handling call
0709 13:20:04.179521 (+   449us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:04.404471 (+224950us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:04.404691 (+   220us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:20:04.406386 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4dc6b0541ca431ebc757beac00521f1" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
W20260709 13:20:04.369393 18916 log.cc:927] Time spent T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 0.337s	user 0.000s	sys 0.001s
I20260709 13:20:04.396485 18797 raft_consensus.cc:697] T 68a6838d82ef4cac83507f906139d90f P 1ca2725d290e47d68b1a4b609c9816ac [term 8 LEADER]: Becoming Leader. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:20:04.408937 18797 consensus_queue.cc:237] T 68a6838d82ef4cac83507f906139d90f P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:20:04.410900 16264 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:04.411482 16300 raft_consensus.cc:3060] T e4dc6b0541ca431ebc757beac00521f1 P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:20:04.402915 16222 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57604 (request call id 2105) took 2347 ms. Trace:
I20260709 13:20:04.362561 18905 raft_consensus.cc:697] T 19baa7bc367844c288bca8f81439c413 P 1ca2725d290e47d68b1a4b609c9816ac [term 7 LEADER]: Becoming Leader. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:20:04.339694 18857 consensus_queue.cc:237] T 9c61a39fd4f84424b129c1a6f9ba404d P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:20:04.406750 18926 log.cc:927] Time spent T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 0.067s	user 0.000s	sys 0.001s
I20260709 13:20:04.369071 18852 raft_consensus.cc:493] T d16dfad0c3b540e5b4b86492ddf87208 P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:04.419979 16191 leader_election.cc:304] T c0eab25ce4c64e018a041beeda2867db P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
I20260709 13:20:04.390830 18770 raft_consensus.cc:2804] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:20:04.376802 16292 pending_rounds.cc:107] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045: Aborting uncommitted CHANGE_CONFIG_OP operation due to leader change: 4.5
I20260709 13:20:04.450618 16292 raft_consensus.cc:2917] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Aborting config change with OpId 4.5: Aborted: Op aborted by new leader
I20260709 13:20:04.451152 16292 raft_consensus.cc:2889] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: CHANGE_CONFIG_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:20:04.452014 16292 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48674 (request call id 2207) took 1236 ms. Trace:
I20260709 13:20:04.452466 16292 rpcz_store.cc:276] 0709 13:20:03.215279 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:04.251434 (+1036155us) service_pool.cc:224] Handling call
0709 13:20:04.251810 (+   376us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:04.451685 (+199875us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:04.451947 (+   262us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:20:04.453647 16292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0eab25ce4c64e018a041beeda2867db" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:20:04.454305 16292 raft_consensus.cc:2376] T c0eab25ce4c64e018a041beeda2867db P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 6. Re-sending same reply.
I20260709 13:20:04.372982 18903 leader_election.cc:290] T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
W20260709 13:20:04.444955 16218 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.531s	user 0.025s	sys 0.028s
I20260709 13:20:04.413861 16222 rpcz_store.cc:276] 0709 13:20:02.055143 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:02.055275 (+   132us) service_pool.cc:224] Handling call
0709 13:20:02.055622 (+   347us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:20:02.055931 (+   309us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:02.055956 (+    25us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:20:03.716827 (+1660871us) spinlock_profiling.cc:243] Waited 43.5 ms on lock 0x7b2c00115ac8. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69af458a11 00007f69af468372 00007f69b0b9aa01 00007f69b0b5f43a 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2
0709 13:20:03.717231 (+   404us) log.cc:844] Serialized 26 byte log entry
0709 13:20:04.179703 (+462472us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:20:04.179741 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:04.179811 (+    70us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:20:04.379307 (+199496us) raft_consensus.cc:1685] finished
0709 13:20:04.379328 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:04.402820 (+ 23492us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":7416,"raft.run_cpu_time_us":114,"raft.run_wall_time_us":104,"spinlock_wait_cycles":99951872,"thread_start_us":462087,"threads_started":1,"wal-append.queue_time_us":597501}
I20260709 13:20:04.395834 18551 raft_consensus.cc:3060] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:20:04.461920 16295 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48684 (request call id 2090) took 1292 ms. Trace:
W20260709 13:20:04.448997 16148 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.533s	user 0.005s	sys 0.013s
W20260709 13:20:04.455790 16218 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.542s	user 0.025s	sys 0.028s
W20260709 13:20:04.463853 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 43943.7us
W20260709 13:20:04.467043 16148 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.551s	user 0.005s	sys 0.014s
I20260709 13:20:04.467300 16191 leader_election.cc:304] T 02838c86531d4f98bf58d1c22b8baf77 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 90f3da57796b488c9087a2f523adf045
I20260709 13:20:04.457589 16292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdbd78acb4cf46298e0de8ba0d130e19" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:04.468417 16292 raft_consensus.cc:2468] T cdbd78acb4cf46298e0de8ba0d130e19 P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 6.
W20260709 13:20:04.469671 16143 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40830 (request call id 1168) took 3249 ms (client timeout 1689 ms). Trace:
I20260709 13:20:04.389266 18745 consensus_queue.cc:1048] T e5ad4eb281564936a59f41e387a47156 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.066s
W20260709 13:20:04.470187 16143 rpcz_store.cc:269] 0709 13:20:01.220771 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:01.372545 (+151774us) service_pool.cc:224] Handling call
0709 13:20:04.469431 (+3096886us) spinlock_profiling.cc:243] Waited 922 ms on lock 0x7b540011b2d4. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:20:04.469603 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2121538304}
I20260709 13:20:04.473686 16298 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48684 (request call id 2088) took 1337 ms. Trace:
I20260709 13:20:04.472929 16192 leader_election.cc:304] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860
I20260709 13:20:04.440467 18558 leader_election.cc:290] T 23f82224f4564fea93572e8206bad1fe P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:04.447220 18825 leader_election.cc:290] T 8cc959fb7f704fba9009c0595467d074 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:04.360525 18768 consensus_queue.cc:237] T fffab26427e24d398e2695aad40ca134 P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:04.492040 16300 raft_consensus.cc:2468] T e4dc6b0541ca431ebc757beac00521f1 P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 6.
I20260709 13:20:04.470547 18931 raft_consensus.cc:2804] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 1ca2725d290e47d68b1a4b609c9816ac [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:20:04.471509 16292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f82224f4564fea93572e8206bad1fe" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:04.373694 18869 consensus_queue.cc:237] T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 5.2, Last appended by leader: 2, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:20:04.477317 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 72973.3us
I20260709 13:20:04.499372 16292 raft_consensus.cc:2468] T 23f82224f4564fea93572e8206bad1fe P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 4.
I20260709 13:20:04.500732 16292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f2da98864b4425bf7c0055951fdd4b" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:20:04.504725 16298 rpcz_store.cc:276] 0709 13:20:03.135989 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:04.042035 (+906046us) service_pool.cc:224] Handling call
0709 13:20:04.042343 (+   308us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:04.042547 (+   204us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:20:04.308922 (+266375us) log.cc:844] Serialized 14 byte log entry
0709 13:20:04.473184 (+164262us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:20:04.473261 (+    77us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:20:04.473366 (+   105us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:04.473449 (+    83us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:04.473594 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":319894,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":41,"raft.queue_time_us":106,"raft.run_cpu_time_us":120,"raft.run_wall_time_us":21316,"thread_start_us":417061,"threads_started":2}
I20260709 13:20:04.477782 18114 raft_consensus.cc:2804] T c0eab25ce4c64e018a041beeda2867db P 1ca2725d290e47d68b1a4b609c9816ac [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:20:04.517655 16292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434bff3082d141838068f323bdb0d1ed" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:20:04.518446 16292 raft_consensus.cc:2376] T 434bff3082d141838068f323bdb0d1ed P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 647a343a542d4a139fe106c19095129b in term 6. Re-sending same reply.
I20260709 13:20:04.471421 18935 raft_consensus.cc:2749] T 7c2095a5f4ad49408dbc7f69c606120d P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:20:04.540349 18833 leader_election.cc:290] T 8a878999c0024d2abe517484180290fc P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
W20260709 13:20:04.540805 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 73592.3us
I20260709 13:20:04.541774 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18ad7e3af5504571a69d10c00750a01c" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:20:04.476089 18770 raft_consensus.cc:697] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 5 LEADER]: Becoming Leader. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:20:04.543606 16300 raft_consensus.cc:3060] T 18ad7e3af5504571a69d10c00750a01c P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:20:04.543392 16191 leader_election.cc:304] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
I20260709 13:20:04.463290 18852 raft_consensus.cc:515] T d16dfad0c3b540e5b4b86492ddf87208 P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:04.551117 16292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87147731a68400e8cce9b879daa7c5d" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:04.551817 16292 raft_consensus.cc:2468] T b87147731a68400e8cce9b879daa7c5d P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 7.
I20260709 13:20:04.552623 16292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2794af5b99984efebdee564035841cb0" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:04.544010 18114 raft_consensus.cc:697] T c0eab25ce4c64e018a041beeda2867db P 1ca2725d290e47d68b1a4b609c9816ac [term 6 LEADER]: Becoming Leader. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:20:04.556816 18114 consensus_queue.cc:237] T c0eab25ce4c64e018a041beeda2867db P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:04.560052 18770 consensus_queue.cc:237] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:04.561756 18892 raft_consensus.cc:515] T af57996a8be84d398f0f023e8d08ef77 P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:04.546066 18558 raft_consensus.cc:493] T 02838c86531d4f98bf58d1c22b8baf77 P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:04.563439 16297 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48674 (request call id 2210) took 1249 ms. Trace:
I20260709 13:20:04.563520 18558 raft_consensus.cc:515] T 02838c86531d4f98bf58d1c22b8baf77 P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:04.564458 18662 leader_election.cc:290] T 806d7690c2b644959152074bb2cab0a7 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:04.566857 16292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1df93779209748b9860f6d893dd3de53" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:20:04.563911 16297 rpcz_store.cc:276] 0709 13:20:03.314385 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:04.304606 (+990221us) service_pool.cc:224] Handling call
0709 13:20:04.305040 (+   434us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:20:04.316088 (+ 11048us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:04.316133 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:20:04.322879 (+  6746us) log.cc:844] Serialized 26 byte log entry
0709 13:20:04.445799 (+122920us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:20:04.445842 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:04.445909 (+    67us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:20:04.563228 (+117319us) raft_consensus.cc:1685] finished
0709 13:20:04.563248 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:04.563385 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":12039,"raft.run_cpu_time_us":164,"raft.run_wall_time_us":10271,"thread_start_us":93851,"threads_started":1,"wal-append.queue_time_us":219942}
I20260709 13:20:04.570843 16300 raft_consensus.cc:2468] T 18ad7e3af5504571a69d10c00750a01c P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 6.
I20260709 13:20:04.572345 16300 raft_consensus.cc:1240] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 4. Current term is 5. Ops: []
I20260709 13:20:04.575551 16292 raft_consensus.cc:2393] T 1df93779209748b9860f6d893dd3de53 P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 6: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
I20260709 13:20:04.576824 16292 raft_consensus.cc:3055] T c4f65cdf00b748bb963ee6e81ef43b79 P 90f3da57796b488c9087a2f523adf045 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:20:04.577224 16292 raft_consensus.cc:740] T c4f65cdf00b748bb963ee6e81ef43b79 P 90f3da57796b488c9087a2f523adf045 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:20:04.578099 16292 consensus_queue.cc:260] T c4f65cdf00b748bb963ee6e81ef43b79 P 90f3da57796b488c9087a2f523adf045 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:04.579368 16292 raft_consensus.cc:3060] T c4f65cdf00b748bb963ee6e81ef43b79 P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:20:04.580571 16223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2372981006d4b218215c28964dea623" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:04.581228 16223 raft_consensus.cc:2468] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 6.
W20260709 13:20:04.555222 18855 consensus_meta.cc:287] T 9e65676b21b748728ad39668cfe722ec P cb82cc7bd6db46d980c75237a1402860: Time spent flushing consensus metadata: real 0.856s	user 0.010s	sys 0.001s
I20260709 13:20:04.623135 16292 raft_consensus.cc:1275] T c4f65cdf00b748bb963ee6e81ef43b79 P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:20:04.669797 16297 raft_consensus.cc:1275] T 7c2095a5f4ad49408dbc7f69c606120d P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:20:04.695859 16297 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48674 (request call id 2212) took 1046 ms. Trace:
I20260709 13:20:04.696350 16297 rpcz_store.cc:276] 0709 13:20:03.649701 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:04.568531 (+918830us) service_pool.cc:224] Handling call
0709 13:20:04.669444 (+100913us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:04.695555 (+ 26111us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:04.695792 (+   237us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:20:04.683034 18937 log.cc:927] Time spent T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.100s	user 0.000s	sys 0.000s
I20260709 13:20:04.499506 18858 consensus_queue.cc:237] T 1df93779209748b9860f6d893dd3de53 P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:04.685364 18836 leader_election.cc:290] T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:04.566002 18807 leader_election.cc:290] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:04.582870 18935 raft_consensus.cc:3060] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:20:04.582717 18936 raft_consensus.cc:2804] T 02838c86531d4f98bf58d1c22b8baf77 P 1ca2725d290e47d68b1a4b609c9816ac [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:20:04.514230 18825 consensus_queue.cc:1059] T d2ab0ac8347641e6955e9bce7d90c1a2 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.052s
W20260709 13:20:04.778995 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 235634us
I20260709 13:20:04.779846 16191 leader_election.cc:304] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:20:04.768920 16190 leader_election.cc:304] T e4dc6b0541ca431ebc757beac00521f1 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
W20260709 13:20:04.783180 16266 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:04.545881 18905 consensus_queue.cc:237] T 19baa7bc367844c288bca8f81439c413 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.5, Last appended by leader: 5, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } }
I20260709 13:20:04.745103 18824 leader_election.cc:290] T c9944d5e4b8c4109af9955c4f08c9095 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 7 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:04.747635 16226 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57604 (request call id 2119) took 2317 ms. Trace:
I20260709 13:20:04.794407 16226 rpcz_store.cc:276] 0709 13:20:02.430318 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:03.720171 (+1289853us) service_pool.cc:224] Handling call
0709 13:20:03.720459 (+   288us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:04.327423 (+606964us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:04.747382 (+419959us) spinlock_profiling.cc:243] Waited 534 ms on lock 0x7b5400014fd4. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:20:04.747571 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1227500288}
I20260709 13:20:04.796329 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02838c86531d4f98bf58d1c22b8baf77" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:04.748826 16223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18ad7e3af5504571a69d10c00750a01c" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:04.750922 16143 raft_consensus.cc:1240] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [term 8 LEADER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 7. Current term is 8. Ops: []
I20260709 13:20:04.751415 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66f75124215f4ca6bef831e174a792d8" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:04.798096 16149 raft_consensus.cc:2468] T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 6.
I20260709 13:20:04.751901 18587 leader_election.cc:290] T 8a878999c0024d2abe517484180290fc P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:04.770844 18936 raft_consensus.cc:493] T 02838c86531d4f98bf58d1c22b8baf77 P 1ca2725d290e47d68b1a4b609c9816ac [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:04.770246 18807 raft_consensus.cc:493] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:04.752840 18931 raft_consensus.cc:493] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 1ca2725d290e47d68b1a4b609c9816ac [term 6 FOLLOWER]: Starting leader election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:20:04.753083 16147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 6 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:04.808481 16147 raft_consensus.cc:2468] T ecc1e363230043fe9fef390f0b979f20 P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 5.
I20260709 13:20:04.809513 16147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f82224f4564fea93572e8206bad1fe" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:20:04.812021 18956 raft_consensus.cc:2749] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:20:04.813050 18956 raft_consensus.cc:2804] T e4dc6b0541ca431ebc757beac00521f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:20:04.813715 18956 raft_consensus.cc:697] T e4dc6b0541ca431ebc757beac00521f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 6 LEADER]: Becoming Leader. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
W20260709 13:20:04.754716 18943 log.cc:927] Time spent T 87ac0d1f7e8a419f9bcbc04aee0e067e P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 0.067s	user 0.000s	sys 0.000s
I20260709 13:20:04.758177 18854 leader_election.cc:290] T ecc1e363230043fe9fef390f0b979f20 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:04.817077 16218 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57604 (request call id 2123) took 2370 ms. Trace:
I20260709 13:20:04.758526 16146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5d59407eb545b895d8d922e0dd69fd" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:04.819607 16146 raft_consensus.cc:2468] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 7.
W20260709 13:20:04.762856 16296 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48674 (request call id 2202) took 2113 ms (client timeout 1807 ms). Trace:
W20260709 13:20:04.820561 16296 rpcz_store.cc:269] 0709 13:20:02.649448 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:02.747317 (+ 97869us) service_pool.cc:224] Handling call
0709 13:20:04.762591 (+2015274us) spinlock_profiling.cc:243] Waited 1.43 s on lock 0x7b54000cf654. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:20:04.762768 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3292281728}
I20260709 13:20:04.758119 16145 raft_consensus.cc:1275] T 26354361e9b044cea1c2b86c4882fe92 P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
W20260709 13:20:04.787088 16266 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:20:04.822477 16266 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:20:04.822906 16266 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:04.805605 16142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806d7690c2b644959152074bb2cab0a7" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
W20260709 13:20:04.727725 16192 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 254874us
I20260709 13:20:04.806264 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31969392ccdb43b491911a319e8bc7e1" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:04.812175 16191 leader_election.cc:304] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 90f3da57796b488c9087a2f523adf045
I20260709 13:20:04.817595 16218 rpcz_store.cc:276] 0709 13:20:02.447115 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:03.724299 (+1277184us) service_pool.cc:224] Handling call
0709 13:20:03.724613 (+   314us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:20:03.777384 (+ 52771us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:20:03.897069 (+119685us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:20:03.897784 (+   715us) log.cc:844] Serialized 26 byte log entry
0709 13:20:04.467611 (+569827us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:20:04.467647 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:04.467725 (+    78us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:20:04.816787 (+349062us) raft_consensus.cc:1685] finished
0709 13:20:04.816815 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:04.817003 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":14652,"raft.run_cpu_time_us":117,"raft.run_wall_time_us":117,"thread_start_us":553734,"threads_started":1,"wal-append.queue_time_us":631578}
I20260709 13:20:04.726840 18855 raft_consensus.cc:515] T 9e65676b21b748728ad39668cfe722ec P cb82cc7bd6db46d980c75237a1402860 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:04.773674 18951 raft_consensus.cc:2749] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:20:04.829624 16143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57a744f5aaf944678a63469cbf514015" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:04.833081 16146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3884f864a96d428785895e5463e5728f" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:20:04.833860 16146 raft_consensus.cc:2393] T 3884f864a96d428785895e5463e5728f P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 in current term 5: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
I20260709 13:20:04.832396 16145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746aabf856ab401c9ef5e859371e7b67" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:04.835922 16145 raft_consensus.cc:2468] T 746aabf856ab401c9ef5e859371e7b67 P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 5.
I20260709 13:20:04.790294 18614 leader_election.cc:290] T 18ad7e3af5504571a69d10c00750a01c P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:20:04.823048 18931 raft_consensus.cc:3060] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 1ca2725d290e47d68b1a4b609c9816ac [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:20:04.824851 18785 leader_election.cc:290] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:20:04.845614 16192 leader_election.cc:304] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 90f3da57796b488c9087a2f523adf045
I20260709 13:20:04.842283 16296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26354361e9b044cea1c2b86c4882fe92" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:04.822517 18936 raft_consensus.cc:3060] T 02838c86531d4f98bf58d1c22b8baf77 P 1ca2725d290e47d68b1a4b609c9816ac [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:20:04.848068 16296 raft_consensus.cc:2368] T 26354361e9b044cea1c2b86c4882fe92 P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b for earlier term 4. Current term is 5.
I20260709 13:20:04.853381 16147 raft_consensus.cc:2468] T 23f82224f4564fea93572e8206bad1fe P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 4.
I20260709 13:20:04.855878 16191 leader_election.cc:304] T 8a878999c0024d2abe517484180290fc P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
W20260709 13:20:04.849606 18907 log.cc:927] Time spent T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 1.479s	user 0.000s	sys 0.005s
I20260709 13:20:04.866631 16192 leader_election.cc:304] T 8c11074579b046d99e585c2818e18a2c P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860
I20260709 13:20:04.827314 18807 raft_consensus.cc:515] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:04.869874 16296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cc959fb7f704fba9009c0595467d074" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 7 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
W20260709 13:20:04.873091 16264 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:04.874758 16296 raft_consensus.cc:2410] T 8cc959fb7f704fba9009c0595467d074 P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac for term 7 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
W20260709 13:20:04.860741 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 50903.7us
I20260709 13:20:04.875767 16190 leader_election.cc:304] T 18ad7e3af5504571a69d10c00750a01c P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:20:04.880018 16218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f5f89186204242bf377f7c42b2e2da" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 8 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:04.841274 18825 consensus_queue.cc:1059] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.8, Next index: 9, Last known committed idx: 6, Time since last communication: 0.028s
I20260709 13:20:04.860064 18952 raft_consensus.cc:2804] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:20:04.815016 18956 consensus_queue.cc:237] T e4dc6b0541ca431ebc757beac00521f1 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:04.888072 18953 consensus_queue.cc:1048] T 66f75124215f4ca6bef831e174a792d8 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.020s
I20260709 13:20:04.888302 16223 raft_consensus.cc:1240] T 68a6838d82ef4cac83507f906139d90f P 1ca2725d290e47d68b1a4b609c9816ac [term 8 LEADER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 7. Current term is 8. Ops: []
I20260709 13:20:04.865402 18951 raft_consensus.cc:2804] T 8a878999c0024d2abe517484180290fc P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:20:04.904421 16218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18ad7e3af5504571a69d10c00750a01c" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:20:04.887859 18858 raft_consensus.cc:493] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:04.912793 16218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5ad4eb281564936a59f41e387a47156" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
W20260709 13:20:04.914867 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 59069.7us
I20260709 13:20:04.895104 18952 raft_consensus.cc:493] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:04.916291 16296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74ca8140f35c4ae29e8a664f6bdfc829" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:20:04.916977 16296 raft_consensus.cc:3060] T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:20:04.902498 18852 leader_election.cc:290] T d16dfad0c3b540e5b4b86492ddf87208 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:20:04.905992 18797 raft_consensus.cc:2749] T 8c11074579b046d99e585c2818e18a2c P 1ca2725d290e47d68b1a4b609c9816ac [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:20:04.902884 18951 raft_consensus.cc:493] T 8a878999c0024d2abe517484180290fc P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:20:04.911990 18834 consensus_meta.cc:287] T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b: Time spent flushing consensus metadata: real 0.542s	user 0.009s	sys 0.002s
W20260709 13:20:04.912616 18918 consensus_meta.cc:287] T 8d4ab870a05347868956aa8b841b2981 P 647a343a542d4a139fe106c19095129b: Time spent flushing consensus metadata: real 0.557s	user 0.007s	sys 0.004s
I20260709 13:20:04.924290 16218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6f07842d3aa47858df5665874c86f3a" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:04.930027 16218 raft_consensus.cc:2393] T b6f07842d3aa47858df5665874c86f3a P 1ca2725d290e47d68b1a4b609c9816ac [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 6: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
W20260709 13:20:04.917001 18720 consensus_meta.cc:287] T 8a878999c0024d2abe517484180290fc P 647a343a542d4a139fe106c19095129b: Time spent flushing consensus metadata: real 0.638s	user 0.010s	sys 0.000s
I20260709 13:20:04.908349 18825 raft_consensus.cc:3055] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:20:04.942672 18964 consensus_queue.cc:1048] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.005s
I20260709 13:20:04.944686 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:20:04.945986 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68a6838d82ef4cac83507f906139d90f" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:20:04.947418 16142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87ac0d1f7e8a419f9bcbc04aee0e067e" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:04.947990 16143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4dc6b0541ca431ebc757beac00521f1" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
W20260709 13:20:04.951793 16264 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:04.918329 18952 raft_consensus.cc:3060] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:20:04.953753 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4dc6b0541ca431ebc757beac00521f1" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:20:04.955968 18939 raft_consensus.cc:2804] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:20:04.956249 18955 raft_consensus.cc:2804] T 18ad7e3af5504571a69d10c00750a01c P 1ca2725d290e47d68b1a4b609c9816ac [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:20:04.956489 18939 raft_consensus.cc:493] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:04.956840 18955 raft_consensus.cc:697] T 18ad7e3af5504571a69d10c00750a01c P 1ca2725d290e47d68b1a4b609c9816ac [term 6 LEADER]: Becoming Leader. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:20:04.957940 16144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cc959fb7f704fba9009c0595467d074" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 7 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:04.957901 18955 consensus_queue.cc:237] T 18ad7e3af5504571a69d10c00750a01c P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } }
I20260709 13:20:04.936555 18953 consensus_queue.cc:1059] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.023s
W20260709 13:20:04.959656 18947 log.cc:927] Time spent T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 0.135s	user 0.000s	sys 0.001s
I20260709 13:20:04.959844 18953 consensus_queue.cc:1243] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Peer 647a343a542d4a139fe106c19095129b log is divergent from this leader: its last log entry 5.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
W20260709 13:20:04.961073 18941 log.cc:927] Time spent T c2f5f89186204242bf377f7c42b2e2da P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 0.130s	user 0.001s	sys 0.000s
I20260709 13:20:04.961339 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7ca61592e848ea9b0fb81a2ce33cdc" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:04.962069 16217 raft_consensus.cc:2468] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 8.
I20260709 13:20:04.963912 16292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7ca61592e848ea9b0fb81a2ce33cdc" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:04.929268 18834 raft_consensus.cc:515] T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:04.964578 16292 raft_consensus.cc:2468] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 8.
I20260709 13:20:04.965510 16292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5ad4eb281564936a59f41e387a47156" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:20:04.927676 18951 raft_consensus.cc:3060] T 8a878999c0024d2abe517484180290fc P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:20:04.966485 16292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a96373bb0e14dce8a266b2fc6c58507" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:20:04.935830 18720 raft_consensus.cc:2749] T 8a878999c0024d2abe517484180290fc P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:20:04.962949 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9944d5e4b8c4109af9955c4f08c9095" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:20:04.956887 18939 raft_consensus.cc:3060] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:20:04.968732 16143 raft_consensus.cc:2393] T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 6: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
W20260709 13:20:04.969077 16114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:04.971540 16292 raft_consensus.cc:2410] T 5a96373bb0e14dce8a266b2fc6c58507 P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b for term 5 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:20:04.978127 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18ad7e3af5504571a69d10c00750a01c" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:20:04.978731 16292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 6 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:04.978835 16149 raft_consensus.cc:3060] T 18ad7e3af5504571a69d10c00750a01c P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Advancing to term 6
I20260709 13:20:04.979404 16292 raft_consensus.cc:2468] T ecc1e363230043fe9fef390f0b979f20 P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 5.
I20260709 13:20:04.980562 16292 raft_consensus.cc:1240] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 7. Current term is 8. Ops: []
I20260709 13:20:04.930831 16221 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33050 (request call id 490) took 2679 ms. Trace:
I20260709 13:20:04.936323 18825 raft_consensus.cc:740] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:20:04.992683 16142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a96373bb0e14dce8a266b2fc6c58507" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:04.921416 18858 raft_consensus.cc:515] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:04.998766 16191 leader_election.cc:304] T 5a96373bb0e14dce8a266b2fc6c58507 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:20:04.999787 16292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4dc6b0541ca431ebc757beac00521f1" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:20:04.935930 18918 raft_consensus.cc:515] T 8d4ab870a05347868956aa8b841b2981 P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:05.006242 16292 raft_consensus.cc:2393] T e4dc6b0541ca431ebc757beac00521f1 P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 6: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
I20260709 13:20:05.007771 16142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f2da98864b4425bf7c0055951fdd4b" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
W20260709 13:20:05.009444 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 133759us
I20260709 13:20:05.011075 16190 leader_election.cc:304] T 8cc959fb7f704fba9009c0595467d074 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:20:05.021894 16190 leader_election.cc:304] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:20:04.990975 18855 leader_election.cc:290] T 9e65676b21b748728ad39668cfe722ec P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 7 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:05.026347 18962 raft_consensus.cc:2804] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 13:20:05.028862 18551 consensus_meta.cc:287] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac: Time spent flushing consensus metadata: real 0.569s	user 0.010s	sys 0.000s
I20260709 13:20:05.029294 18551 raft_consensus.cc:515] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:05.029999 18821 leader_election.cc:290] T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:05.030933 16142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5ad4eb281564936a59f41e387a47156" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:20:05.032224 16142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6f07842d3aa47858df5665874c86f3a" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:20:05.033226 16142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1df93779209748b9860f6d893dd3de53" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:20:05.019997 18797 raft_consensus.cc:2749] T 5a96373bb0e14dce8a266b2fc6c58507 P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:20:05.024971 18964 raft_consensus.cc:2749] T 8cc959fb7f704fba9009c0595467d074 P 1ca2725d290e47d68b1a4b609c9816ac [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:20:05.050557 18931 raft_consensus.cc:515] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:05.052107 16220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada5a004ba7c46819051e3ccec71ffdb" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:05.052383 18896 raft_consensus.cc:493] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Starting leader election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:20:05.055532 18807 leader_election.cc:290] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:05.054948 16191 leader_election.cc:304] T 84f2da98864b4425bf7c0055951fdd4b P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:20:04.985407 16221 rpcz_store.cc:276] 0709 13:20:02.251267 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:02.380082 (+128815us) service_pool.cc:224] Handling call
0709 13:20:02.449632 (+ 69550us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:20:02.870651 (+421019us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:02.875261 (+  4610us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:20:02.889815 (+ 14554us) log.cc:844] Serialized 26 byte log entry
0709 13:20:03.103659 (+213844us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:20:03.310571 (+206912us) log.cc:844] Serialized 14 byte log entry
0709 13:20:03.355815 (+ 45244us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:03.368925 (+ 13110us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:20:03.742915 (+373990us) spinlock_profiling.cc:243] Waited 2.56 ms on lock 0x7b180046c480. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b693ade1 00007f69b69475b1 00007f69ada402a6 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b693ade1 00007f69b69475b1 00007f69ada723f3 00007f69ada747c1
0709 13:20:03.744645 (+  1730us) spinlock_profiling.cc:243] Waited 13.9 ms on lock 0x7b180046c480. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b693ade1 00007f69b69475b1 00007f69ada723f3 00007f69ada747c1 00007f69ada74779 00007f69ada74709 00007f69ada746d1 00007f69ada739cd 000000000038f354 000000000038f199 00007f69ada5583a
0709 13:20:04.784268 (+1039623us) spinlock_profiling.cc:243] Waited 66.5 ms on lock 0x7b4400022b4c. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69ad8fadb1 00007f69ad998c85 00007f69ad9996a2 00007f69b070b1db 00007f69b0706aaf 00007f69b06fa86e 00007f69b06f9bb6 00007f69b06f97d8 00007f69b0703221 00007f69b07031d9 00007f69b0703169
0709 13:20:04.927803 (+143535us) raft_consensus.cc:1685] finished
0709 13:20:04.927837 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:04.930770 (+  2933us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2314,"prepare.queue_time_us":225484,"prepare.run_cpu_time_us":48,"prepare.run_wall_time_us":40,"raft.queue_time_us":216851,"raft.run_cpu_time_us":316,"raft.run_wall_time_us":23106,"spinlock_wait_cycles":190604928,"thread_start_us":283504,"threads_started":2,"wal-append.queue_time_us":279806}
I20260709 13:20:05.026892 18962 raft_consensus.cc:493] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:05.088444 16191 leader_election.cc:304] T e5ad4eb281564936a59f41e387a47156 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:20:05.088871 16220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2372981006d4b218215c28964dea623" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:20:05.102875 16220 raft_consensus.cc:2376] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 6. Re-sending same reply.
I20260709 13:20:05.103847 16220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806d7690c2b644959152074bb2cab0a7" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:05.105381 16220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ab0ac8347641e6955e9bce7d90c1a2" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:20:05.106748 16220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada5a004ba7c46819051e3ccec71ffdb" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:20:05.108145 16220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8638ee363273426db1393dd81e5c7b72" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:05.109277 16220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66f75124215f4ca6bef831e174a792d8" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:05.110394 16220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8638ee363273426db1393dd81e5c7b72" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
W20260709 13:20:05.101094 16117 leader_election.cc:336] T 3884f864a96d428785895e5463e5728f P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923): Timed out: RequestConsensusVote RPC to 127.14.177.132:38923 timed out after 1.664s (SENT)
I20260709 13:20:05.111111 16117 leader_election.cc:304] T 3884f864a96d428785895e5463e5728f P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860
I20260709 13:20:05.112905 16191 leader_election.cc:304] T 1df93779209748b9860f6d893dd3de53 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
W20260709 13:20:05.113900 16117 connection.cc:256] RPC call timeout handler was delayed by -2.1461801580001065s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1655s for a response
W20260709 13:20:05.114763 16117 connection.cc:256] RPC call timeout handler was delayed by -1.2716415980000875s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15480000000000002s for a response
I20260709 13:20:05.063910 18896 raft_consensus.cc:3060] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:20:05.116297 16117 connection.cc:256] RPC call timeout handler was delayed by -1.1690328480001426s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1807s for a response
I20260709 13:20:05.115682 18881 raft_consensus.cc:2749] T 3884f864a96d428785895e5463e5728f P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:20:05.116776 18953 raft_consensus.cc:2749] T e5ad4eb281564936a59f41e387a47156 P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:20:05.122725 16149 raft_consensus.cc:2468] T 18ad7e3af5504571a69d10c00750a01c P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 6.
I20260709 13:20:05.081503 18551 leader_election.cc:290] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 7 election: Requested vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:05.059325 18936 raft_consensus.cc:515] T 02838c86531d4f98bf58d1c22b8baf77 P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:05.111804 18807 raft_consensus.cc:2749] T 84f2da98864b4425bf7c0055951fdd4b P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:20:05.126282 16220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66ae176961b74813a58bc15c9c4c6f2a" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:20:05.111972 18962 raft_consensus.cc:3060] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:20:05.140172 16220 raft_consensus.cc:2368] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 for earlier term 7. Current term is 8.
I20260709 13:20:05.139966 16294 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48674 (request call id 2183) took 2721 ms. Trace:
I20260709 13:20:05.119732 18869 raft_consensus.cc:2749] T 1df93779209748b9860f6d893dd3de53 P 1ca2725d290e47d68b1a4b609c9816ac [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:20:05.062682 18825 consensus_queue.cc:260] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:05.067806 18948 consensus_queue.cc:1059] T c2f5f89186204242bf377f7c42b2e2da P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.058s
I20260709 13:20:05.140712 16294 rpcz_store.cc:276] 0709 13:20:02.418608 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:02.609413 (+190805us) service_pool.cc:224] Handling call
0709 13:20:02.609698 (+   285us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:02.629615 (+ 19917us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:05.139651 (+2510036us) spinlock_profiling.cc:243] Waited 19.2 ms on lock 0x7b54001eee54. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:20:05.139880 (+   229us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":44046080}
I20260709 13:20:05.088907 18852 consensus_queue.cc:1048] T 8158bbd17b994d2eb96f560bd0f1b88d P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.037s
I20260709 13:20:05.142242 16226 raft_consensus.cc:2393] T 02838c86531d4f98bf58d1c22b8baf77 P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 7: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
I20260709 13:20:05.142424 16220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a878999c0024d2abe517484180290fc" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:05.103116 18844 consensus_queue.cc:1048] T 02838c86531d4f98bf58d1c22b8baf77 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.027s
I20260709 13:20:05.112347 18919 consensus_queue.cc:1059] T fffab26427e24d398e2695aad40ca134 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.024s
I20260709 13:20:05.112756 18827 consensus_queue.cc:1059] T 5a959dbecc2648eb85d1cc550f5a3245 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 1, Time since last communication: 0.027s
I20260709 13:20:05.123389 18881 consensus_queue.cc:1048] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.023s
I20260709 13:20:05.165477 16294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e9b2bf7011f4ae688a79d7bb5a447b4" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:05.180693 16294 raft_consensus.cc:1240] T 57a744f5aaf944678a63469cbf514015 P 90f3da57796b488c9087a2f523adf045 [term 4 LEADER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 3. Current term is 4. Ops: []
I20260709 13:20:05.171855 18827 consensus_queue.cc:1243] T 5a959dbecc2648eb85d1cc550f5a3245 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Peer 647a343a542d4a139fe106c19095129b log is divergent from this leader: its last log entry 5.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 1
W20260709 13:20:05.194902 18970 log.cc:927] Time spent T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 0.101s	user 0.000s	sys 0.001s
I20260709 13:20:05.194796 18939 raft_consensus.cc:515] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:05.197341 18939 leader_election.cc:290] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 6 election: Requested vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
W20260709 13:20:05.197592 18972 log.cc:927] Time spent T 746aabf856ab401c9ef5e859371e7b67 P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 0.102s	user 0.000s	sys 0.000s
I20260709 13:20:05.201634 18828 leader_election.cc:290] T d2ab0ac8347641e6955e9bce7d90c1a2 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:05.211565 16296 raft_consensus.cc:2468] T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 7.
I20260709 13:20:05.203881 16116 leader_election.cc:304] T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
I20260709 13:20:05.214746 16296 raft_consensus.cc:1275] T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 13:20:05.213208 16190 leader_election.cc:304] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:20:05.217244 16116 leader_election.cc:304] T 18ad7e3af5504571a69d10c00750a01c P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:20:05.219084 16222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7ca61592e848ea9b0fb81a2ce33cdc" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:20:05.218384 18821 raft_consensus.cc:2804] T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:20:05.223345 18821 raft_consensus.cc:493] T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Starting leader election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:20:05.223868 18821 raft_consensus.cc:3060] T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:20:05.228003 18886 log.cc:927] Time spent T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860: Append to log took a long time: real 2.825s	user 0.013s	sys 0.010s
I20260709 13:20:05.229328 16816 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49906 (request call id 593) took 3156 ms. Trace:
I20260709 13:20:05.229194 18747 consensus_queue.cc:1059] T b6f07842d3aa47858df5665874c86f3a P 647a343a542d4a139fe106c19095129b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.002s
I20260709 13:20:05.229297 18817 raft_consensus.cc:2749] T 18ad7e3af5504571a69d10c00750a01c P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:20:05.230376 18747 raft_consensus.cc:3055] T b6f07842d3aa47858df5665874c86f3a P 647a343a542d4a139fe106c19095129b [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:20:05.230811 18747 raft_consensus.cc:740] T b6f07842d3aa47858df5665874c86f3a P 647a343a542d4a139fe106c19095129b [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:20:05.231709 18747 consensus_queue.cc:260] T b6f07842d3aa47858df5665874c86f3a P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:05.233045 18747 raft_consensus.cc:3060] T b6f07842d3aa47858df5665874c86f3a P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:20:05.238212 16117 leader_election.cc:336] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 7 election: RPC error from VoteRequest() call to peer cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923): Timed out: RequestConsensusVote RPC to 127.14.177.132:38923 timed out after 1.655s (SENT)
W20260709 13:20:05.257030 16264 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:05.259110 18931 leader_election.cc:290] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 7 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:05.268656 16823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31969392ccdb43b491911a319e8bc7e1" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:05.269600 16823 raft_consensus.cc:2410] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 for term 6 because replica has last-logged OpId of term: 5 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 5.
I20260709 13:20:05.269552 18558 leader_election.cc:290] T 02838c86531d4f98bf58d1c22b8baf77 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:05.271076 16818 raft_consensus.cc:1275] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860 [term 4 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
W20260709 13:20:05.271283 16266 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:05.272360 16820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2794af5b99984efebdee564035841cb0" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:20:05.214344 18858 leader_election.cc:290] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:05.221266 18969 raft_consensus.cc:2804] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:20:05.292855 16822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdbd78acb4cf46298e0de8ba0d130e19" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:05.293856 16822 raft_consensus.cc:2468] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 6.
I20260709 13:20:05.295562 16821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c11074579b046d99e585c2818e18a2c" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:05.299639 16819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26354361e9b044cea1c2b86c4882fe92" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:05.300953 16825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434bff3082d141838068f323bdb0d1ed" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:05.301793 16825 raft_consensus.cc:2393] T 434bff3082d141838068f323bdb0d1ed P cb82cc7bd6db46d980c75237a1402860 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 6: Already voted for candidate cb82cc7bd6db46d980c75237a1402860 in this term.
I20260709 13:20:05.306095 18892 leader_election.cc:290] T af57996a8be84d398f0f023e8d08ef77 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 7 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:05.308423 18825 raft_consensus.cc:3060] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:20:05.229910 16816 rpcz_store.cc:276] 0709 13:20:02.073221 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:02.229029 (+155808us) service_pool.cc:224] Handling call
0709 13:20:02.229415 (+   386us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:20:02.230318 (+   903us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:02.230352 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:20:02.230851 (+   499us) log.cc:844] Serialized 26 byte log entry
0709 13:20:02.272129 (+ 41278us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:20:02.273989 (+  1860us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:02.296594 (+ 22605us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:20:05.227356 (+2930762us) spinlock_profiling.cc:243] Waited 5.96 ms on lock 0x7b4400022b4c. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69ad8fadb1 00007f69ad998c85 00007f69ad9996a2 00007f69b070b1db 00007f69b0706aaf 00007f69b06fa86e 00007f69b06f9bb6 00007f69b06f97d8 00007f69b0703221 00007f69b07031d9 00007f69b0703169
0709 13:20:05.228976 (+  1620us) raft_consensus.cc:1685] finished
0709 13:20:05.229014 (+    38us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:05.229261 (+   247us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":345,"raft.run_cpu_time_us":77,"raft.run_wall_time_us":77,"spinlock_wait_cycles":13716736,"thread_start_us":32627,"threads_started":1,"wal-append.queue_time_us":65769}
I20260709 13:20:05.318583 18936 leader_election.cc:290] T 02838c86531d4f98bf58d1c22b8baf77 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 7 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
W20260709 13:20:05.320230 18920 log.cc:927] Time spent T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 1.051s	user 0.000s	sys 0.002s
I20260709 13:20:05.234846 18918 leader_election.cc:290] T 8d4ab870a05347868956aa8b841b2981 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 7 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:05.318933 16221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87ac0d1f7e8a419f9bcbc04aee0e067e" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 7 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:05.326078 16221 raft_consensus.cc:2393] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 7: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
I20260709 13:20:05.327396 16221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "829b4ad8c5a54eababeea1181b960d4b" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:05.327879 16824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e4bfb9090c8460eb14fa7e0d818e7f1" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:05.328912 16221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5d59407eb545b895d8d922e0dd69fd" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:05.244486 18935 raft_consensus.cc:2749] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:20:05.332881 18950 log.cc:927] Time spent T d16dfad0c3b540e5b4b86492ddf87208 P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 0.500s	user 0.002s	sys 0.000s
I20260709 13:20:05.345327 16296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a878999c0024d2abe517484180290fc" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:05.349845 16296 raft_consensus.cc:2393] T 8a878999c0024d2abe517484180290fc P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 6: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
I20260709 13:20:05.349768 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdbd78acb4cf46298e0de8ba0d130e19" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:20:05.350520 16297 raft_consensus.cc:3060] T cdbd78acb4cf46298e0de8ba0d130e19 P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:20:05.350867 16294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66ae176961b74813a58bc15c9c4c6f2a" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:05.351527 16294 raft_consensus.cc:2468] T 66ae176961b74813a58bc15c9c4c6f2a P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 8.
I20260709 13:20:05.354905 18843 raft_consensus.cc:3055] T fffab26427e24d398e2695aad40ca134 P 90f3da57796b488c9087a2f523adf045 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:20:05.355387 18843 raft_consensus.cc:740] T fffab26427e24d398e2695aad40ca134 P 90f3da57796b488c9087a2f523adf045 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:20:05.355567 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a378d7f796f94615b3b347119a9880fd" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:20:05.356313 18843 consensus_queue.cc:260] T fffab26427e24d398e2695aad40ca134 P 90f3da57796b488c9087a2f523adf045 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:05.357537 18843 raft_consensus.cc:3060] T fffab26427e24d398e2695aad40ca134 P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:20:05.333963 18969 raft_consensus.cc:697] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [term 7 LEADER]: Becoming Leader. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:20:05.345903 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6f07842d3aa47858df5665874c86f3a" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:20:05.334450 18956 raft_consensus.cc:493] T af57996a8be84d398f0f023e8d08ef77 P 1ca2725d290e47d68b1a4b609c9816ac [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:05.338029 16144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1df93779209748b9860f6d893dd3de53" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:05.339645 16219 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57604 (request call id 2126) took 2592 ms. Trace:
W20260709 13:20:05.340979 16225 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57604 (request call id 2116) took 2918 ms (client timeout 2135 ms). Trace:
I20260709 13:20:05.342483 18868 leader_election.cc:290] T 23f82224f4564fea93572e8206bad1fe P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:05.343564 18750 leader_election.cc:290] T b6f07842d3aa47858df5665874c86f3a P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 6 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:05.360042 18969 consensus_queue.cc:237] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:05.360224 16297 raft_consensus.cc:2468] T cdbd78acb4cf46298e0de8ba0d130e19 P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 7.
I20260709 13:20:05.387645 18743 leader_election.cc:290] T 26354361e9b044cea1c2b86c4882fe92 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 5 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:05.388616 16224 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57604 (request call id 2113) took 3313 ms. Trace:
I20260709 13:20:05.360361 18956 raft_consensus.cc:515] T af57996a8be84d398f0f023e8d08ef77 P 1ca2725d290e47d68b1a4b609c9816ac [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:05.389000 16224 rpcz_store.cc:276] 0709 13:20:02.075093 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:02.080177 (+  5084us) service_pool.cc:224] Handling call
0709 13:20:02.109019 (+ 28842us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:03.670810 (+1561791us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:05.388387 (+1717577us) spinlock_profiling.cc:243] Waited 1.53 s on lock 0x7b5400396554. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:20:05.388555 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3512465152}
I20260709 13:20:05.390200 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b7d1ab519044eda2d5e04a07242f25" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:05.361514 16144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87ac0d1f7e8a419f9bcbc04aee0e067e" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 7 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:05.392658 16144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f5f89186204242bf377f7c42b2e2da" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:20:04.463835 16295 rpcz_store.cc:276] 0709 13:20:03.169593 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:04.047096 (+877503us) service_pool.cc:224] Handling call
0709 13:20:04.175574 (+128478us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:04.175838 (+   264us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:20:04.326879 (+151041us) log.cc:844] Serialized 14 byte log entry
0709 13:20:04.461535 (+134656us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:20:04.461606 (+    71us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:20:04.461668 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:04.461757 (+    89us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:04.461869 (+   112us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":186814,"prepare.run_cpu_time_us":24,"prepare.run_wall_time_us":25,"raft.queue_time_us":475,"thread_start_us":272532,"threads_started":2}
I20260709 13:20:05.393918 16144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26354361e9b044cea1c2b86c4882fe92" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:20:05.394568 16144 raft_consensus.cc:3060] T 26354361e9b044cea1c2b86c4882fe92 P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:20:05.390942 18956 leader_election.cc:290] T af57996a8be84d398f0f023e8d08ef77 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:05.394652 16150 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46224 (request call id 371) took 2522 ms. Trace:
I20260709 13:20:05.400152 16150 rpcz_store.cc:276] 0709 13:20:02.871881 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:03.709482 (+837601us) service_pool.cc:224] Handling call
0709 13:20:04.350916 (+641434us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:04.355285 (+  4369us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:05.394485 (+1039200us) spinlock_profiling.cc:243] Waited 641 ms on lock 0x7b54000749d0. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5e8ad 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:20:05.394601 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1474397952}
W20260709 13:20:05.361862 16225 rpcz_store.cc:269] 0709 13:20:02.422429 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:02.926088 (+503659us) service_pool.cc:224] Handling call
0709 13:20:05.340768 (+2414680us) spinlock_profiling.cc:243] Waited 690 ms on lock 0x7b5400367254. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:20:05.340907 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1586427648}
I20260709 13:20:05.402215 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c2095a5f4ad49408dbc7f69c606120d" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:05.402724 16821 raft_consensus.cc:1275] T 57a744f5aaf944678a63469cbf514015 P cb82cc7bd6db46d980c75237a1402860 [term 4 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:20:05.402799 16224 raft_consensus.cc:2468] T 7c2095a5f4ad49408dbc7f69c606120d P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 5.
I20260709 13:20:05.417923 16190 leader_election.cc:304] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:20:05.420557 16298 raft_consensus.cc:1240] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 7. Current term is 8. Ops: []
W20260709 13:20:05.420931 16266 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:05.359107 16217 raft_consensus.cc:2468] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 7.
I20260709 13:20:05.361685 16219 rpcz_store.cc:276] 0709 13:20:02.747613 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:03.730533 (+982920us) service_pool.cc:224] Handling call
0709 13:20:03.730851 (+   318us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:20:03.800444 (+ 69593us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:05.339314 (+1538870us) spinlock_profiling.cc:243] Waited 68.9 ms on lock 0x7b54003d2a54. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:20:05.339567 (+   253us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":158419840}
I20260709 13:20:05.426386 16190 leader_election.cc:304] T b6f07842d3aa47858df5665874c86f3a P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:20:05.424495 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d4ab870a05347868956aa8b841b2981" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 7 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:05.427443 16149 raft_consensus.cc:2393] T 8d4ab870a05347868956aa8b841b2981 P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 7: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
I20260709 13:20:05.428341 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0eab25ce4c64e018a041beeda2867db" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:05.431449 16149 raft_consensus.cc:2468] T c0eab25ce4c64e018a041beeda2867db P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 6.
I20260709 13:20:05.434478 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c74490c40243a88e62d1760b5cd814" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:05.437667 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a878999c0024d2abe517484180290fc" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
W20260709 13:20:05.442268 16293 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48674 (request call id 2189) took 2827 ms (client timeout 2089 ms). Trace:
I20260709 13:20:05.443141 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c11074579b046d99e585c2818e18a2c" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:05.444717 16225 raft_consensus.cc:2468] T 8c11074579b046d99e585c2818e18a2c P 1ca2725d290e47d68b1a4b609c9816ac [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 6.
I20260709 13:20:05.429751 16190 leader_election.cc:304] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:20:05.445749 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0eab25ce4c64e018a041beeda2867db" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:05.443372 16294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02838c86531d4f98bf58d1c22b8baf77" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
W20260709 13:20:05.447053 16265 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:20:05.442737 16293 rpcz_store.cc:269] 0709 13:20:02.614693 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:02.622180 (+  7487us) service_pool.cc:224] Handling call
0709 13:20:05.407741 (+2785561us) spinlock_profiling.cc:243] Waited 1.06 s on lock 0x7b54000aa854. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:20:05.442197 (+ 34456us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2443138304}
I20260709 13:20:05.450431 16265 leader_election.cc:304] T c0eab25ce4c64e018a041beeda2867db P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:20:05.456593 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c61a39fd4f84424b129c1a6f9ba404d" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:05.457552 16224 raft_consensus.cc:2393] T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 7: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
I20260709 13:20:05.459121 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66ae176961b74813a58bc15c9c4c6f2a" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:20:05.375136 18870 raft_consensus.cc:3055] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:20:05.462803 16295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87ac0d1f7e8a419f9bcbc04aee0e067e" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:20:05.478487 16295 raft_consensus.cc:3060] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:20:05.466267 16821 raft_consensus.cc:1240] T c9944d5e4b8c4109af9955c4f08c9095 P cb82cc7bd6db46d980c75237a1402860 [term 7 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 6. Current term is 7. Ops: []
I20260709 13:20:05.367946 18807 raft_consensus.cc:3055] T c2f5f89186204242bf377f7c42b2e2da P 1ca2725d290e47d68b1a4b609c9816ac [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:20:05.364603 18927 raft_consensus.cc:3055] T 5a959dbecc2648eb85d1cc550f5a3245 P 90f3da57796b488c9087a2f523adf045 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:20:05.480155 16300 raft_consensus.cc:1275] T 68a6838d82ef4cac83507f906139d90f P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:20:05.481246 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87ac0d1f7e8a419f9bcbc04aee0e067e" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 7 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:05.379578 18781 raft_consensus.cc:3055] T 57a744f5aaf944678a63469cbf514015 P 647a343a542d4a139fe106c19095129b [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:20:05.383265 18834 leader_election.cc:290] T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 6 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:05.453258 18750 raft_consensus.cc:2804] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:20:05.499495 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a96373bb0e14dce8a266b2fc6c58507" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:05.504491 16149 raft_consensus.cc:2468] T 8a878999c0024d2abe517484180290fc P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 6.
I20260709 13:20:05.460266 18935 raft_consensus.cc:2804] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:20:05.507862 16821 raft_consensus.cc:1240] T 9e65676b21b748728ad39668cfe722ec P cb82cc7bd6db46d980c75237a1402860 [term 7 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 6. Current term is 7. Ops: []
W20260709 13:20:05.513722 16264 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:05.477921 18897 raft_consensus.cc:2804] T c0eab25ce4c64e018a041beeda2867db P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:20:05.482126 18807 raft_consensus.cc:740] T c2f5f89186204242bf377f7c42b2e2da P 1ca2725d290e47d68b1a4b609c9816ac [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:20:05.555801 18929 raft_consensus.cc:2749] T b6f07842d3aa47858df5665874c86f3a P 1ca2725d290e47d68b1a4b609c9816ac [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:20:05.493857 18927 raft_consensus.cc:740] T 5a959dbecc2648eb85d1cc550f5a3245 P 90f3da57796b488c9087a2f523adf045 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:20:05.558105 18897 raft_consensus.cc:493] T c0eab25ce4c64e018a041beeda2867db P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:05.558525 18897 raft_consensus.cc:3060] T c0eab25ce4c64e018a041beeda2867db P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:20:05.560308 16817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3884f864a96d428785895e5463e5728f" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:20:05.561340 16817 raft_consensus.cc:2393] T 3884f864a96d428785895e5463e5728f P cb82cc7bd6db46d980c75237a1402860 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 5: Already voted for candidate cb82cc7bd6db46d980c75237a1402860 in this term.
I20260709 13:20:05.568179 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d4ab870a05347868956aa8b841b2981" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 7 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:05.569154 18927 consensus_queue.cc:260] T 5a959dbecc2648eb85d1cc550f5a3245 P 90f3da57796b488c9087a2f523adf045 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:05.442658 18855 consensus_queue.cc:1048] T af57996a8be84d398f0f023e8d08ef77 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.063s
W20260709 13:20:05.492980 18971 log.cc:927] Time spent T 829b4ad8c5a54eababeea1181b960d4b P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 0.397s	user 0.004s	sys 0.000s
I20260709 13:20:05.477496 18870 raft_consensus.cc:740] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
W20260709 13:20:05.794036 16142 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.671s	user 0.027s	sys 0.097s
I20260709 13:20:05.493227 16221 raft_consensus.cc:2468] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 7.
W20260709 13:20:05.786374 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 375784us
W20260709 13:20:05.895913 16142 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.773s	user 0.031s	sys 0.110s
W20260709 13:20:05.655507 16116 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:05.612974 16295 raft_consensus.cc:2468] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 7.
W20260709 13:20:05.699522 16264 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:05.569504 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "829b4ad8c5a54eababeea1181b960d4b" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:05.663340 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31969392ccdb43b491911a319e8bc7e1" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:05.789150 16146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87ac0d1f7e8a419f9bcbc04aee0e067e" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:20:05.917137 16146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87147731a68400e8cce9b879daa7c5d" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:05.802928 16223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6f07842d3aa47858df5665874c86f3a" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:05.923873 16223 raft_consensus.cc:2393] T b6f07842d3aa47858df5665874c86f3a P 1ca2725d290e47d68b1a4b609c9816ac [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 6: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
I20260709 13:20:05.505507 18781 raft_consensus.cc:740] T 57a744f5aaf944678a63469cbf514015 P 647a343a542d4a139fe106c19095129b [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:20:05.930269 16145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c2095a5f4ad49408dbc7f69c606120d" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
W20260709 13:20:05.596244 18978 log.cc:927] Time spent T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 0.083s	user 0.000s	sys 0.000s
W20260709 13:20:05.602008 18952 consensus_meta.cc:287] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac: Time spent flushing consensus metadata: real 0.619s	user 0.011s	sys 0.000s
W20260709 13:20:05.678341 18962 consensus_meta.cc:287] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 1ca2725d290e47d68b1a4b609c9816ac: Time spent flushing consensus metadata: real 0.534s	user 0.008s	sys 0.001s
W20260709 13:20:05.823959 18720 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.506s	user 0.019s	sys 0.045s
W20260709 13:20:05.847424 18946 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.549s	user 0.041s	sys 0.059s
I20260709 13:20:05.505973 18750 raft_consensus.cc:493] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:05.935230 16146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e65676b21b748728ad39668cfe722ec" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:20:05.952211 16146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c61a39fd4f84424b129c1a6f9ba404d" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:05.935686 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1df93779209748b9860f6d893dd3de53" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:05.954173 16224 raft_consensus.cc:2393] T 1df93779209748b9860f6d893dd3de53 P 1ca2725d290e47d68b1a4b609c9816ac [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 6: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
I20260709 13:20:05.946903 16223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ab0ac8347641e6955e9bce7d90c1a2" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
W20260709 13:20:05.922994 18969 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.527s	user 0.011s	sys 0.021s
I20260709 13:20:05.949451 16821 raft_consensus.cc:3055] T 2794af5b99984efebdee564035841cb0 P cb82cc7bd6db46d980c75237a1402860 [term 4 LEADER]: Stepping down as leader of term 4
W20260709 13:20:05.947986 18946 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.689s	user 0.042s	sys 0.060s
W20260709 13:20:05.937373 18720 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.619s	user 0.021s	sys 0.046s
I20260709 13:20:05.939177 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e9b2bf7011f4ae688a79d7bb5a447b4" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:05.692807 18897 raft_consensus.cc:515] T c0eab25ce4c64e018a041beeda2867db P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:05.952127 16145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a9de347c711477e87c582d003f36925" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
W20260709 13:20:05.956774 18951 consensus_meta.cc:287] T 8a878999c0024d2abe517484180290fc P 1ca2725d290e47d68b1a4b609c9816ac: Time spent flushing consensus metadata: real 0.990s	user 0.008s	sys 0.003s
I20260709 13:20:05.949137 18952 raft_consensus.cc:515] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:05.973656 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f5f89186204242bf377f7c42b2e2da" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:20:05.931409 18932 consensus_queue.cc:1059] T 84f2da98864b4425bf7c0055951fdd4b P 647a343a542d4a139fe106c19095129b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.035s
I20260709 13:20:05.980679 16223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66ae176961b74813a58bc15c9c4c6f2a" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:20:05.570343 18807 consensus_queue.cc:260] T c2f5f89186204242bf377f7c42b2e2da P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:05.930720 18927 raft_consensus.cc:3060] T 5a959dbecc2648eb85d1cc550f5a3245 P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:20:05.958225 18969 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.562s	user 0.012s	sys 0.021s
I20260709 13:20:05.966914 16150 raft_consensus.cc:2468] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 4.
I20260709 13:20:05.958441 16821 raft_consensus.cc:740] T 2794af5b99984efebdee564035841cb0 P cb82cc7bd6db46d980c75237a1402860 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:20:05.927388 18770 raft_consensus.cc:3055] T d2ab0ac8347641e6955e9bce7d90c1a2 P 1ca2725d290e47d68b1a4b609c9816ac [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:20:05.928254 18935 raft_consensus.cc:697] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [term 7 LEADER]: Becoming Leader. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:20:06.003437 16218 raft_consensus.cc:1240] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 6. Current term is 7. Ops: []
I20260709 13:20:05.953593 18750 raft_consensus.cc:3060] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:20:05.983019 16147 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40844 (request call id 1949) took 1050 ms. Trace:
I20260709 13:20:05.986340 16145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a96373bb0e14dce8a266b2fc6c58507" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:05.960712 16146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "829b4ad8c5a54eababeea1181b960d4b" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
W20260709 13:20:05.999486 16144 consensus_meta.cc:287] T 26354361e9b044cea1c2b86c4882fe92 P 647a343a542d4a139fe106c19095129b: Time spent flushing consensus metadata: real 0.604s	user 0.011s	sys 0.003s
W20260709 13:20:05.988044 18981 log.cc:927] Time spent T 19baa7bc367844c288bca8f81439c413 P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 0.101s	user 0.000s	sys 0.001s
I20260709 13:20:06.036377 16818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31969392ccdb43b491911a319e8bc7e1" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:06.037774 16818 raft_consensus.cc:2410] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 for term 6 because replica has last-logged OpId of term: 5 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 5.
I20260709 13:20:06.008730 16223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a878999c0024d2abe517484180290fc" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:06.014093 16144 raft_consensus.cc:2468] T 26354361e9b044cea1c2b86c4882fe92 P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 5.
I20260709 13:20:06.039608 16144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f2da98864b4425bf7c0055951fdd4b" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:06.010823 16147 rpcz_store.cc:276] 0709 13:20:04.932338 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:04.932492 (+   154us) service_pool.cc:224] Handling call
0709 13:20:04.932801 (+   309us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:20:04.942249 (+  9448us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:04.942268 (+    19us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:20:04.955764 (+ 13496us) log.cc:844] Serialized 26 byte log entry
0709 13:20:05.096667 (+140903us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:20:05.168063 (+ 71396us) log.cc:844] Serialized 14 byte log entry
0709 13:20:05.168335 (+   272us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:05.168426 (+    91us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:20:05.386932 (+218506us) spinlock_profiling.cc:243] Waited 14.5 ms on lock 0x7b4400022b4c. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69ad8fadb1 00007f69ad998c85 00007f69ad9996a2 00007f69b070b1db 00007f69b0706aaf 00007f69b06fa86e 00007f69b06f9bb6 00007f69b06f97d8 00007f69b0703221 00007f69b07031d9 00007f69b0703169
0709 13:20:05.982675 (+595743us) raft_consensus.cc:1685] finished
0709 13:20:05.982787 (+   112us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:05.982964 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":9023,"prepare.queue_time_us":139425,"prepare.run_cpu_time_us":56,"prepare.run_wall_time_us":18768,"raft.queue_time_us":9657,"raft.run_cpu_time_us":220,"raft.run_wall_time_us":12724,"spinlock_wait_cycles":33298688,"thread_start_us":161494,"threads_started":2,"wal-append.queue_time_us":103693}
I20260709 13:20:05.997435 16817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f82224f4564fea93572e8206bad1fe" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:06.031862 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3884f864a96d428785895e5463e5728f" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:20:06.048471 18935 consensus_queue.cc:237] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:06.049579 16150 raft_consensus.cc:2393] T 3884f864a96d428785895e5463e5728f P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 5: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
I20260709 13:20:06.050539 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e9b2bf7011f4ae688a79d7bb5a447b4" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:20:06.051312 16150 raft_consensus.cc:2393] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 4: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
I20260709 13:20:06.052397 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d16dfad0c3b540e5b4b86492ddf87208" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:05.987726 18781 consensus_queue.cc:260] T 57a744f5aaf944678a63469cbf514015 P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:06.055286 18781 raft_consensus.cc:3060] T 57a744f5aaf944678a63469cbf514015 P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:20:06.057632 18969 raft_consensus.cc:493] T 5a959dbecc2648eb85d1cc550f5a3245 P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:06.058130 18969 raft_consensus.cc:515] T 5a959dbecc2648eb85d1cc550f5a3245 P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:06.060312 18969 leader_election.cc:290] T 5a959dbecc2648eb85d1cc550f5a3245 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:06.060513 16146 raft_consensus.cc:1275] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:20:06.060812 18969 raft_consensus.cc:493] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:06.061314 18969 raft_consensus.cc:515] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:06.000342 18770 raft_consensus.cc:740] T d2ab0ac8347641e6955e9bce7d90c1a2 P 1ca2725d290e47d68b1a4b609c9816ac [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:20:06.062871 18770 consensus_queue.cc:260] T d2ab0ac8347641e6955e9bce7d90c1a2 P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:06.063494 16816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ab0ac8347641e6955e9bce7d90c1a2" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:06.064201 18969 leader_election.cc:290] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:06.064296 18770 raft_consensus.cc:3060] T d2ab0ac8347641e6955e9bce7d90c1a2 P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:20:06.068689 18952 leader_election.cc:290] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 7 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
W20260709 13:20:06.071537 18821 consensus_meta.cc:287] T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b: Time spent flushing consensus metadata: real 0.847s	user 0.006s	sys 0.007s
I20260709 13:20:06.071945 16142 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40844 (request call id 1962) took 1121 ms. Trace:
I20260709 13:20:06.072021 18821 raft_consensus.cc:515] T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:06.072471 16142 rpcz_store.cc:276] 0709 13:20:04.950983 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:05.061852 (+110869us) service_pool.cc:224] Handling call
0709 13:20:05.102160 (+ 40308us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:20:05.103410 (+  1250us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:05.103437 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:20:05.103914 (+   477us) log.cc:844] Serialized 26 byte log entry
0709 13:20:05.914025 (+810111us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:20:05.955784 (+ 41759us) log.cc:844] Serialized 14 byte log entry
0709 13:20:05.956290 (+   506us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:05.956366 (+    76us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:20:06.071723 (+115357us) raft_consensus.cc:1685] finished
0709 13:20:06.071744 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:06.071873 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":15735,"prepare.queue_time_us":40769,"raft.queue_time_us":2796,"raft.run_cpu_time_us":102,"raft.run_wall_time_us":16992,"spinlock_wait_cycles":12362880,"thread_start_us":790521,"threads_started":1,"wal-append.queue_time_us":949618}
I20260709 13:20:06.074365 18821 leader_election.cc:290] T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 7 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:06.075505 16144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f82224f4564fea93572e8206bad1fe" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:06.076162 16144 raft_consensus.cc:2468] T 23f82224f4564fea93572e8206bad1fe P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 4.
W20260709 13:20:06.081799 18747 consensus_meta.cc:287] T b6f07842d3aa47858df5665874c86f3a P 647a343a542d4a139fe106c19095129b: Time spent flushing consensus metadata: real 0.848s	user 0.009s	sys 0.000s
I20260709 13:20:06.087316 16190 leader_election.cc:304] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:20:06.088510 18969 consensus_queue.cc:1048] T 68a6838d82ef4cac83507f906139d90f P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:20:06.089870 18969 raft_consensus.cc:2804] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:20:06.090313 18969 raft_consensus.cc:697] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 1ca2725d290e47d68b1a4b609c9816ac [term 7 LEADER]: Becoming Leader. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:20:06.091111 16145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4dc6b0541ca431ebc757beac00521f1" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:05.947675 18962 raft_consensus.cc:515] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:06.003937 16821 consensus_queue.cc:260] T 2794af5b99984efebdee564035841cb0 P cb82cc7bd6db46d980c75237a1402860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.5, Last appended by leader: 5, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:06.033095 16789 leader_election.cc:304] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:20:06.034276 16822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26354361e9b044cea1c2b86c4882fe92" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:06.034999 18897 leader_election.cc:290] T c0eab25ce4c64e018a041beeda2867db P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 7 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:20:06.035364 16820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2794af5b99984efebdee564035841cb0" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:20:06.074990 16142 raft_consensus.cc:1275] T 2794af5b99984efebdee564035841cb0 P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 13:20:06.125895 16819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7ca61592e848ea9b0fb81a2ce33cdc" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
W20260709 13:20:06.129511 16789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 96565.6us
I20260709 13:20:06.130023 16819 raft_consensus.cc:2468] T 3a7ca61592e848ea9b0fb81a2ce33cdc P cb82cc7bd6db46d980c75237a1402860 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 7.
I20260709 13:20:06.042817 16147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434bff3082d141838068f323bdb0d1ed" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:05.963387 18870 consensus_queue.cc:260] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.6, Last appended by leader: 6, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:06.048738 16223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18ad7e3af5504571a69d10c00750a01c" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
W20260709 13:20:06.131536 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 215319us
W20260709 13:20:06.096904 16264 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:06.092249 16226 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57604 (request call id 2139) took 1359 ms. Trace:
I20260709 13:20:06.135226 16226 rpcz_store.cc:276] 0709 13:20:04.733179 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:04.795987 (+ 62808us) service_pool.cc:224] Handling call
0709 13:20:06.092063 (+1296076us) spinlock_profiling.cc:243] Waited 345 ms on lock 0x7b54002b3754. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:20:06.092192 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":793161600}
I20260709 13:20:06.136427 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c74490c40243a88e62d1760b5cd814" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:06.132560 16147 raft_consensus.cc:2393] T 434bff3082d141838068f323bdb0d1ed P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 6: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
I20260709 13:20:06.137256 16147 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40830 (request call id 1197) took 1011 ms. Trace:
I20260709 13:20:06.137881 16147 rpcz_store.cc:276] 0709 13:20:05.126410 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:06.042478 (+916068us) service_pool.cc:224] Handling call
0709 13:20:06.137199 (+ 94721us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:20:06.137972 16820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdbd78acb4cf46298e0de8ba0d130e19" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:20:06.139098 16820 raft_consensus.cc:3060] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:20:06.139703 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a878999c0024d2abe517484180290fc" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:20:06.140856 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4dc6b0541ca431ebc757beac00521f1" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:20:06.143671 16825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74ca8140f35c4ae29e8a664f6bdfc829" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:20:06.144651 16825 raft_consensus.cc:3060] T 74ca8140f35c4ae29e8a664f6bdfc829 P cb82cc7bd6db46d980c75237a1402860 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:20:06.142081 18825 raft_consensus.cc:493] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:20:06.064388 16816 raft_consensus.cc:2393] T d2ab0ac8347641e6955e9bce7d90c1a2 P cb82cc7bd6db46d980c75237a1402860 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 6: Already voted for candidate cb82cc7bd6db46d980c75237a1402860 in this term.
W20260709 13:20:06.121625 18896 consensus_meta.cc:287] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 1ca2725d290e47d68b1a4b609c9816ac: Time spent flushing consensus metadata: real 0.995s	user 0.010s	sys 0.000s
I20260709 13:20:06.153851 16226 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57604 (request call id 2141) took 1177 ms. Trace:
I20260709 13:20:06.156098 16222 raft_consensus.cc:2368] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 for earlier term 8. Current term is 9.
I20260709 13:20:05.977517 18951 raft_consensus.cc:515] T 8a878999c0024d2abe517484180290fc P 1ca2725d290e47d68b1a4b609c9816ac [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:06.156669 16226 rpcz_store.cc:276] 0709 13:20:04.976778 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:06.140656 (+1163878us) service_pool.cc:224] Handling call
0709 13:20:06.153789 (+ 13133us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:20:06.158746 16223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7ca61592e848ea9b0fb81a2ce33cdc" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:20:06.016443 18807 raft_consensus.cc:3060] T c2f5f89186204242bf377f7c42b2e2da P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:20:06.001377 18855 consensus_queue.cc:1059] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.154s
W20260709 13:20:06.166234 16264 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:06.153441 18587 consensus_queue.cc:1059] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.027s
I20260709 13:20:06.181111 16822 raft_consensus.cc:3055] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P cb82cc7bd6db46d980c75237a1402860 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:20:06.194598 16822 raft_consensus.cc:740] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P cb82cc7bd6db46d980c75237a1402860 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:20:06.149255 18897 raft_consensus.cc:493] T c4f65cdf00b748bb963ee6e81ef43b79 P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:06.149067 18988 consensus_queue.cc:1059] T 2794af5b99984efebdee564035841cb0 P cb82cc7bd6db46d980c75237a1402860 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: INVALID_TERM, Last received: 4.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.056s
W20260709 13:20:06.158453 18843 consensus_meta.cc:287] T fffab26427e24d398e2695aad40ca134 P 90f3da57796b488c9087a2f523adf045: Time spent flushing consensus metadata: real 0.801s	user 0.011s	sys 0.004s
I20260709 13:20:06.183653 16817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66ae176961b74813a58bc15c9c4c6f2a" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:06.188540 16823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af57996a8be84d398f0f023e8d08ef77" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 7 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:06.190503 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2372981006d4b218215c28964dea623" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:20:06.204818 16226 raft_consensus.cc:3060] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:20:06.147615 18825 raft_consensus.cc:515] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } }
I20260709 13:20:06.091128 18969 consensus_queue.cc:237] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.5, Last appended by leader: 5, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:06.159458 18896 raft_consensus.cc:515] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 1ca2725d290e47d68b1a4b609c9816ac [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:06.216666 16220 raft_consensus.cc:2393] T 8a878999c0024d2abe517484180290fc P 1ca2725d290e47d68b1a4b609c9816ac [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 6: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
I20260709 13:20:06.197664 18897 raft_consensus.cc:515] T c4f65cdf00b748bb963ee6e81ef43b79 P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:06.213763 16822 consensus_queue.cc:260] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P cb82cc7bd6db46d980c75237a1402860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:06.239048 16223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e4bfb9090c8460eb14fa7e0d818e7f1" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:06.240175 16822 raft_consensus.cc:3060] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P cb82cc7bd6db46d980c75237a1402860 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:20:06.226832 16820 raft_consensus.cc:2468] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 7.
I20260709 13:20:06.233332 16226 raft_consensus.cc:2468] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 7.
I20260709 13:20:06.243189 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d4ab870a05347868956aa8b841b2981" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 7 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
W20260709 13:20:06.246285 18989 log.cc:927] Time spent T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 0.052s	user 0.001s	sys 0.000s
I20260709 13:20:06.248106 18897 leader_election.cc:290] T c4f65cdf00b748bb963ee6e81ef43b79 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
W20260709 13:20:06.252166 18928 log.cc:927] Time spent T 9e65676b21b748728ad39668cfe722ec P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 1.883s	user 0.008s	sys 0.011s
I20260709 13:20:06.246302 16823 raft_consensus.cc:3060] T 19baa7bc367844c288bca8f81439c413 P cb82cc7bd6db46d980c75237a1402860 [term 6 LEARNER]: Advancing to term 7
I20260709 13:20:06.226850 16817 raft_consensus.cc:2468] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 8.
I20260709 13:20:06.256551 16223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4dc6b0541ca431ebc757beac00521f1" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:06.258394 18951 leader_election.cc:290] T 8a878999c0024d2abe517484180290fc P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 6 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
W20260709 13:20:06.260040 16143 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40844 (request call id 1952) took 1325 ms (client timeout 1737 ms). Trace:
I20260709 13:20:06.216506 18936 raft_consensus.cc:3055] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:20:06.276163 16226 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57604 (request call id 2146) took 1103 ms. Trace:
I20260709 13:20:06.327262 16226 rpcz_store.cc:276] 0709 13:20:05.172623 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:06.242836 (+1070213us) service_pool.cc:224] Handling call
0709 13:20:06.276078 (+ 33242us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:20:06.328619 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af57996a8be84d398f0f023e8d08ef77" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
W20260709 13:20:06.288208 16143 rpcz_store.cc:269] 0709 13:20:04.934236 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:04.934339 (+   103us) service_pool.cc:224] Handling call
0709 13:20:06.259846 (+1325507us) spinlock_profiling.cc:243] Waited 20 ms on lock 0x7b54001f22d4. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:20:06.259982 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":45950208}
I20260709 13:20:06.275141 16822 raft_consensus.cc:1275] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P cb82cc7bd6db46d980c75237a1402860 [term 5 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 6. (term mismatch)
I20260709 13:20:06.292804 16825 raft_consensus.cc:2468] T 74ca8140f35c4ae29e8a664f6bdfc829 P cb82cc7bd6db46d980c75237a1402860 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 7.
I20260709 13:20:06.330493 16822 pending_rounds.cc:85] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P cb82cc7bd6db46d980c75237a1402860: Aborting all ops after (but not including) 5
W20260709 13:20:06.301895 16791 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:06.330755 16820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62fe3346cd3c42c4a55e14af9073bca4" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 9 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:06.330821 16822 pending_rounds.cc:107] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P cb82cc7bd6db46d980c75237a1402860: Aborting uncommitted NO_OP operation due to leader change: 4.6
W20260709 13:20:06.313856 16116 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:06.332876 16822 raft_consensus.cc:2889] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P cb82cc7bd6db46d980c75237a1402860 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:20:06.287582 16223 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57600 (request call id 1172) took 1082 ms. Trace:
I20260709 13:20:06.333485 16223 rpcz_store.cc:276] 0709 13:20:05.205055 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:06.256233 (+1051178us) service_pool.cc:224] Handling call
0709 13:20:06.287497 (+ 31264us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:20:06.329342 16226 raft_consensus.cc:3060] T af57996a8be84d398f0f023e8d08ef77 P 1ca2725d290e47d68b1a4b609c9816ac [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:20:06.326264 18114 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.501s	user 0.058s	sys 0.092s
I20260709 13:20:06.359043 16226 raft_consensus.cc:2468] T af57996a8be84d398f0f023e8d08ef77 P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 7.
I20260709 13:20:06.360205 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c2095a5f4ad49408dbc7f69c606120d" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:06.360870 16226 raft_consensus.cc:2468] T 7c2095a5f4ad49408dbc7f69c606120d P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 5.
I20260709 13:20:06.363682 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5ad4eb281564936a59f41e387a47156" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:06.365869 16226 raft_consensus.cc:2368] T e5ad4eb281564936a59f41e387a47156 P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 for earlier term 6. Current term is 7.
I20260709 13:20:06.376479 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74ca8140f35c4ae29e8a664f6bdfc829" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 7 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:06.377816 16226 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33050 (request call id 503) took 1437 ms. Trace:
I20260709 13:20:06.378361 16226 rpcz_store.cc:276] 0709 13:20:04.940508 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:06.376168 (+1435660us) service_pool.cc:224] Handling call
0709 13:20:06.377743 (+  1575us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:20:06.326664 18936 raft_consensus.cc:740] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:20:06.379987 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c61a39fd4f84424b129c1a6f9ba404d" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:06.340752 18587 raft_consensus.cc:493] T d16dfad0c3b540e5b4b86492ddf87208 P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:20:06.382743 16226 raft_consensus.cc:2393] T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 7: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
W20260709 13:20:06.384232 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 235864us
W20260709 13:20:06.381417 16266 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:20:06.358877 18817 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.050s	user 0.031s	sys 0.027s
W20260709 13:20:06.395195 18817 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.087s	user 0.032s	sys 0.027s
W20260709 13:20:06.410367 16116 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:06.357717 18992 raft_consensus.cc:2804] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:20:06.411089 16298 raft_consensus.cc:1275] T 8d4ab870a05347868956aa8b841b2981 P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 5 index: 2. Preceding OpId from leader: term: 6 index: 3. (index mismatch)
W20260709 13:20:06.411326 16789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 280983us
I20260709 13:20:06.348611 18956 consensus_queue.cc:1048] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.046s
I20260709 13:20:06.416167 16298 raft_consensus.cc:1275] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 13:20:06.421284 16294 raft_consensus.cc:1275] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260709 13:20:06.403280 18962 leader_election.cc:290] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 9 election: Requested vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:06.427392 16151 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40830 (request call id 1182) took 4181 ms. Trace:
I20260709 13:20:06.428977 16151 rpcz_store.cc:276] 0709 13:20:02.245739 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:03.482632 (+1236893us) service_pool.cc:224] Handling call
0709 13:20:03.483145 (+   513us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:20:03.639689 (+156544us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:03.639723 (+    34us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:20:03.670665 (+ 30942us) log.cc:844] Serialized 52 byte log entry
0709 13:20:04.184907 (+514242us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:20:04.185019 (+   112us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:04.185096 (+    77us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:20:06.213081 (+2027985us) spinlock_profiling.cc:243] Waited 17.1 ms on lock 0x7b4400022b4c. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69ad8fadb1 00007f69ad998c85 00007f69ad9996a2 00007f69b070b1db 00007f69b0706aaf 00007f69b06fa86e 00007f69b06f9bb6 00007f69b06f97d8 00007f69b0703221 00007f69b07031d9 00007f69b0703169
0709 13:20:06.423860 (+210779us) raft_consensus.cc:1685] finished
0709 13:20:06.423881 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:06.427322 (+  3441us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2345,"raft.queue_time_us":20217,"raft.run_cpu_time_us":193,"raft.run_wall_time_us":16874,"spinlock_wait_cycles":39310080,"thread_start_us":429317,"threads_started":2,"wal-append.queue_time_us":530145}
I20260709 13:20:06.394132 18587 raft_consensus.cc:515] T d16dfad0c3b540e5b4b86492ddf87208 P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:06.420996 18758 consensus_queue.cc:1048] T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.023s
W20260709 13:20:06.459609 16266 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:20:06.462296 18991 log.cc:927] Time spent T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 0.068s	user 0.000s	sys 0.000s
I20260709 13:20:06.468741 16148 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46224 (request call id 366) took 4103 ms. Trace:
W20260709 13:20:06.469772 16264 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:20:06.470329 16217 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33050 (request call id 501) took 1530 ms (client timeout 1513 ms). Trace:
W20260709 13:20:06.470768 16217 rpcz_store.cc:269] 0709 13:20:04.940114 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:04.962712 (+ 22598us) service_pool.cc:224] Handling call
0709 13:20:06.470106 (+1507394us) spinlock_profiling.cc:243] Waited 381 ms on lock 0x7b5400368654. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:20:06.470256 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":876514688}
I20260709 13:20:06.472098 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c11074579b046d99e585c2818e18a2c" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:06.472761 16217 raft_consensus.cc:2468] T 8c11074579b046d99e585c2818e18a2c P 1ca2725d290e47d68b1a4b609c9816ac [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 6.
I20260709 13:20:06.479115 16146 raft_consensus.cc:1275] T 57a744f5aaf944678a63469cbf514015 P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
W20260709 13:20:06.481379 16117 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:06.482105 18750 raft_consensus.cc:515] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:06.483331 16225 raft_consensus.cc:2368] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 for earlier term 8. Current term is 9.
I20260709 13:20:06.448802 18980 consensus_queue.cc:1048] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.040s
I20260709 13:20:06.487233 16223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a96373bb0e14dce8a266b2fc6c58507" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
W20260709 13:20:06.497591 16191 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:06.498054 16223 raft_consensus.cc:2393] T 5a96373bb0e14dce8a266b2fc6c58507 P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 5: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
I20260709 13:20:06.491072 18825 leader_election.cc:290] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:06.493388 18896 leader_election.cc:290] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 6 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:06.495783 18944 raft_consensus.cc:3055] T 84f2da98864b4425bf7c0055951fdd4b P 647a343a542d4a139fe106c19095129b [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:20:06.501513 18944 raft_consensus.cc:740] T 84f2da98864b4425bf7c0055951fdd4b P 647a343a542d4a139fe106c19095129b [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:20:06.425139 18936 consensus_queue.cc:260] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:06.502658 18944 consensus_queue.cc:260] T 84f2da98864b4425bf7c0055951fdd4b P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:06.504410 18944 raft_consensus.cc:3060] T 84f2da98864b4425bf7c0055951fdd4b P 647a343a542d4a139fe106c19095129b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:20:06.504634 19002 raft_consensus.cc:3055] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:20:06.505180 19002 raft_consensus.cc:740] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:20:06.506531 18587 leader_election.cc:290] T d16dfad0c3b540e5b4b86492ddf87208 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:06.507180 16294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d4ab870a05347868956aa8b841b2981" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 7 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:20:06.507450 16144 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40830 (request call id 1214) took 1106 ms. Trace:
I20260709 13:20:06.507650 16297 raft_consensus.cc:1275] T 1df93779209748b9860f6d893dd3de53 P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 13:20:06.508157 16298 raft_consensus.cc:1240] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 6. Current term is 7. Ops: []
I20260709 13:20:06.508458 16295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02838c86531d4f98bf58d1c22b8baf77" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:06.509495 16300 raft_consensus.cc:1275] T 5a959dbecc2648eb85d1cc550f5a3245 P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:20:06.509848 16299 raft_consensus.cc:1240] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 6. Current term is 7. Ops: [4.5-6.6]
I20260709 13:20:06.510701 16219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f5f89186204242bf377f7c42b2e2da" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 8 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:06.511631 16223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdbd78acb4cf46298e0de8ba0d130e19" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:06.511708 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:06.511919 16219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4dc6b0541ca431ebc757beac00521f1" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
W20260709 13:20:06.496865 16116 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:06.512929 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746aabf856ab401c9ef5e859371e7b67" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:06.496928 16150 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40844 (request call id 1966) took 1500 ms. Trace:
I20260709 13:20:06.513772 16226 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33050 (request call id 515) took 1100 ms. Trace:
I20260709 13:20:06.514214 16226 rpcz_store.cc:276] 0709 13:20:05.413874 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:06.512696 (+1098822us) service_pool.cc:224] Handling call
0709 13:20:06.513706 (+  1010us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:20:06.513718 16150 rpcz_store.cc:276] 0709 13:20:04.996703 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:06.053351 (+1056648us) service_pool.cc:224] Handling call
0709 13:20:06.053719 (+   368us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:20:06.113442 (+ 59723us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:06.113493 (+    51us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:20:06.114158 (+   665us) log.cc:844] Serialized 26 byte log entry
0709 13:20:06.415933 (+301775us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:20:06.415971 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:06.416034 (+    63us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:20:06.496684 (+ 80650us) raft_consensus.cc:1685] finished
0709 13:20:06.496711 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:06.496863 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":69478,"raft.run_cpu_time_us":96,"raft.run_wall_time_us":98,"thread_start_us":254629,"threads_started":1,"wal-append.queue_time_us":353380}
W20260709 13:20:06.513113 16116 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:06.516439 18758 raft_consensus.cc:493] T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:06.516927 18758 raft_consensus.cc:515] T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:06.517346 16116 leader_election.cc:304] T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
W20260709 13:20:06.518111 18959 log.cc:927] Time spent T 1df93779209748b9860f6d893dd3de53 P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 1.541s	user 0.004s	sys 0.020s
I20260709 13:20:06.518586 16116 leader_election.cc:304] T af57996a8be84d398f0f023e8d08ef77 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
I20260709 13:20:06.519411 18781 consensus_queue.cc:1059] T e5ad4eb281564936a59f41e387a47156 P 647a343a542d4a139fe106c19095129b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: INVALID_TERM, Last received: 4.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:20:06.519615 18421 consensus_queue.cc:1059] T 68a6838d82ef4cac83507f906139d90f P 647a343a542d4a139fe106c19095129b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:20:06.520305 18421 consensus_queue.cc:1243] T 68a6838d82ef4cac83507f906139d90f P 647a343a542d4a139fe106c19095129b [LEADER]: Peer 1ca2725d290e47d68b1a4b609c9816ac log is divergent from this leader: its last log entry 8.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 13:20:06.520650 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74ca8140f35c4ae29e8a664f6bdfc829" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:20:06.520815 18421 raft_consensus.cc:2804] T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:20:06.521348 18421 raft_consensus.cc:697] T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b [term 7 LEADER]: Becoming Leader. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:20:06.521472 18946 raft_consensus.cc:2804] T af57996a8be84d398f0f023e8d08ef77 P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:20:06.521993 18946 raft_consensus.cc:697] T af57996a8be84d398f0f023e8d08ef77 P 647a343a542d4a139fe106c19095129b [term 7 LEADER]: Becoming Leader. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
W20260709 13:20:06.522634 16264 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:06.523000 18946 consensus_queue.cc:237] T af57996a8be84d398f0f023e8d08ef77 P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:06.507866 16144 rpcz_store.cc:276] 0709 13:20:05.401141 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:06.076810 (+675669us) service_pool.cc:224] Handling call
0709 13:20:06.077125 (+   315us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:20:06.127753 (+ 50628us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:06.127781 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:20:06.151284 (+ 23503us) log.cc:844] Serialized 26 byte log entry
0709 13:20:06.413871 (+262587us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:20:06.413906 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:06.413970 (+    64us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:20:06.507246 (+ 93276us) raft_consensus.cc:1685] finished
0709 13:20:06.507266 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:06.507396 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":50199,"raft.queue_time_us":50827,"raft.run_cpu_time_us":147,"raft.run_wall_time_us":14415,"thread_start_us":227355,"threads_started":1,"wal-append.queue_time_us":332098}
I20260709 13:20:06.515151 16220 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57600 (request call id 1164) took 1435 ms. Trace:
I20260709 13:20:06.525583 16220 rpcz_store.cc:276] 0709 13:20:05.080199 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:05.140897 (+ 60698us) service_pool.cc:224] Handling call
0709 13:20:06.514975 (+1374078us) spinlock_profiling.cc:243] Waited 1.06 s on lock 0x7b5400394754. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:20:06.515098 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2441901952}
I20260709 13:20:06.526247 18857 raft_consensus.cc:3055] T e5ad4eb281564936a59f41e387a47156 P 647a343a542d4a139fe106c19095129b [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:20:06.526599 16220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 6 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:06.526614 18857 raft_consensus.cc:740] T e5ad4eb281564936a59f41e387a47156 P 647a343a542d4a139fe106c19095129b [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:20:06.528621 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 6 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:06.529538 16223 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57600 (request call id 1168) took 1392 ms. Trace:
I20260709 13:20:06.529878 16223 rpcz_store.cc:276] 0709 13:20:05.136925 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:06.498859 (+1361934us) service_pool.cc:224] Handling call
0709 13:20:06.529477 (+ 30618us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:20:06.530433 16220 raft_consensus.cc:2410] T ecc1e363230043fe9fef390f0b979f20 P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 for term 6 because replica has last-logged OpId of term: 2 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 3.
I20260709 13:20:06.530895 16223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d16dfad0c3b540e5b4b86492ddf87208" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:06.531131 16220 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57600 (request call id 1171) took 1339 ms. Trace:
I20260709 13:20:06.531734 16223 raft_consensus.cc:2410] T d16dfad0c3b540e5b4b86492ddf87208 P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 for term 6 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:20:06.531697 16220 rpcz_store.cc:276] 0709 13:20:05.192149 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:06.526349 (+1334200us) service_pool.cc:224] Handling call
0709 13:20:06.531075 (+  4726us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:20:06.532917 16220 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "84f2da98864b4425bf7c0055951fdd4b"
dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
mode: GRACEFUL
new_leader_uuid: "90f3da57796b488c9087a2f523adf045"
 from {username='slave'} at 127.0.0.1:39144
I20260709 13:20:06.534093 16220 raft_consensus.cc:606] T 84f2da98864b4425bf7c0055951fdd4b P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Received request to transfer leadership to TS 90f3da57796b488c9087a2f523adf045
I20260709 13:20:06.534641 16220 raft_consensus.cc:614] T 84f2da98864b4425bf7c0055951fdd4b P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:20:06.535709 16224 raft_consensus.cc:2468] T c2f5f89186204242bf377f7c42b2e2da P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 8.
I20260709 13:20:06.506088 19002 consensus_queue.cc:260] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:06.554335 16821 raft_consensus.cc:3060] T 2794af5b99984efebdee564035841cb0 P cb82cc7bd6db46d980c75237a1402860 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:20:06.556452 18750 leader_election.cc:290] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 9 election: Requested vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:06.518939 18758 leader_election.cc:290] T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:06.469226 16148 rpcz_store.cc:276] 0709 13:20:02.365700 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:03.693297 (+1327597us) service_pool.cc:224] Handling call
0709 13:20:03.693578 (+   281us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:03.803407 (+109829us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:20:04.467631 (+664224us) log.cc:844] Serialized 14 byte log entry
0709 13:20:04.689068 (+221437us) log.cc:844] Serialized 14 byte log entry
0709 13:20:04.691121 (+  2053us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:20:04.691190 (+    69us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:20:04.691236 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:05.372171 (+680935us) spinlock_profiling.cc:243] Waited 6.36 ms on lock 0x7b18002f10a0. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b693ade1 00007f69b69475b1 00007f69ada402a6 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b693ade1 00007f69b69475b1 00007f69ada402a6 00007f69ada40240
0709 13:20:05.373027 (+   856us) spinlock_profiling.cc:243] Waited 34.1 ms on lock 0x7b18002f10a0. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b693ade1 00007f69b69475b1 00007f69ada402a6 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b693ade1 00007f69b69475b1 00007f69ada723f3 00007f69ada747c1
0709 13:20:05.373508 (+   481us) spinlock_profiling.cc:243] Waited 23.8 ms on lock 0x7b18002f10a0. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b693ade1 00007f69b69475b1 00007f69ada723f3 00007f69ada747c1 00007f69ada74779 00007f69ada74709 00007f69ada746d1 00007f69ada739cd 000000000038f354 000000000038f199 00007f69ada5583a
0709 13:20:05.375644 (+  2136us) spinlock_profiling.cc:243] Waited 1.7 ms on lock 0x7b18002f10a0. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b693ade1 00007f69b69475b1 00007f69ada723f3 00007f69ada747c1 00007f69ada74779 00007f69ada74709 00007f69ada746d1 00007f69ada739cd 000000000038f354 000000000038f199 00007f69ada5583a
0709 13:20:06.468465 (+1092821us) spinlock_profiling.cc:243] Waited 109 ms on lock 0x7b540041dcd4. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:20:06.468515 (+    50us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:06.468676 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":15420,"prepare.queue_time_us":705183,"prepare.run_cpu_time_us":87,"prepare.run_wall_time_us":6723,"raft.queue_time_us":33483,"raft.run_cpu_time_us":227,"raft.run_wall_time_us":11817,"spinlock_wait_cycles":403455616,"thread_start_us":771968,"threads_started":2,"wal-append.queue_time_us":312385,"wal-append.run_cpu_time_us":1460,"wal-append.run_wall_time_us":1003147}
W20260709 13:20:06.564054 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 143943us
I20260709 13:20:06.542969 16823 raft_consensus.cc:1275] T 19baa7bc367844c288bca8f81439c413 P cb82cc7bd6db46d980c75237a1402860 [term 7 LEARNER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260709 13:20:06.544078 16226 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33050 (request call id 505) took 1603 ms. Trace:
I20260709 13:20:06.565328 16226 rpcz_store.cc:276] 0709 13:20:04.940929 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:06.528389 (+1587460us) service_pool.cc:224] Handling call
0709 13:20:06.544014 (+ 15625us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:20:06.528339 18857 consensus_queue.cc:260] T e5ad4eb281564936a59f41e387a47156 P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:06.544499 18881 raft_consensus.cc:3055] T 68a6838d82ef4cac83507f906139d90f P 647a343a542d4a139fe106c19095129b [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:20:06.545210 16115 leader_election.cc:304] T 23f82224f4564fea93572e8206bad1fe P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:20:06.545430 16221 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33050 (request call id 511) took 1591 ms. Trace:
W20260709 13:20:06.568938 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 174478us
I20260709 13:20:06.545275 18745 consensus_queue.cc:1059] T 68a6838d82ef4cac83507f906139d90f P 647a343a542d4a139fe106c19095129b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:20:06.546697 16295 raft_consensus.cc:1275] T 5a96373bb0e14dce8a266b2fc6c58507 P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 5 index: 2. (term mismatch)
I20260709 13:20:06.571708 16295 pending_rounds.cc:85] T 5a96373bb0e14dce8a266b2fc6c58507 P 90f3da57796b488c9087a2f523adf045: Aborting all ops after (but not including) 1
I20260709 13:20:06.572008 16295 pending_rounds.cc:107] T 5a96373bb0e14dce8a266b2fc6c58507 P 90f3da57796b488c9087a2f523adf045: Aborting uncommitted NO_OP operation due to leader change: 2.2
I20260709 13:20:06.572319 16295 raft_consensus.cc:2889] T 5a96373bb0e14dce8a266b2fc6c58507 P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:20:06.522334 18421 consensus_queue.cc:237] T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:06.547065 18983 consensus_queue.cc:1048] T 8cc959fb7f704fba9009c0595467d074 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:20:06.547333 16219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f2da98864b4425bf7c0055951fdd4b" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:06.584350 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a959dbecc2648eb85d1cc550f5a3245" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:06.585078 16219 raft_consensus.cc:2393] T 84f2da98864b4425bf7c0055951fdd4b P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 4: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
I20260709 13:20:06.547559 16220 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:39144 (request call id 23) took 1596 ms. Trace:
I20260709 13:20:06.548079 18833 consensus_queue.cc:1048] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:20:06.549094 16218 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57600 (request call id 1153) took 1680 ms. Trace:
I20260709 13:20:06.570785 18720 raft_consensus.cc:2804] T 23f82224f4564fea93572e8206bad1fe P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:20:06.586958 16218 rpcz_store.cc:276] 0709 13:20:04.868624 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:04.936990 (+ 68366us) service_pool.cc:224] Handling call
0709 13:20:04.937281 (+   291us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:06.004040 (+1066759us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:06.548844 (+544804us) spinlock_profiling.cc:243] Waited 1.07 s on lock 0x7b54002c9554. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:20:06.549028 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2451522816}
I20260709 13:20:06.588937 16218 raft_consensus.cc:1240] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 4. Current term is 5. Ops: []
I20260709 13:20:06.589637 16218 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57600 (request call id 1158) took 1682 ms. Trace:
I20260709 13:20:06.590133 16218 rpcz_store.cc:276] 0709 13:20:04.907513 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:06.588322 (+1680809us) service_pool.cc:224] Handling call
0709 13:20:06.588677 (+   355us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:06.589356 (+   679us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:06.589571 (+   215us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:20:06.592446 16219 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57600 (request call id 1181) took 1016 ms. Trace:
I20260709 13:20:06.592947 16219 rpcz_store.cc:276] 0709 13:20:05.576363 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:06.512763 (+936400us) service_pool.cc:224] Handling call
0709 13:20:06.592371 (+ 79608us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:20:06.549376 16223 raft_consensus.cc:1240] T 5a959dbecc2648eb85d1cc550f5a3245 P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 4. Current term is 5. Ops: [2.2-4.3]
I20260709 13:20:06.594599 16223 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57600 (request call id 1156) took 1718 ms. Trace:
I20260709 13:20:06.594971 16223 rpcz_store.cc:276] 0709 13:20:04.876301 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:06.548785 (+1672484us) service_pool.cc:224] Handling call
0709 13:20:06.549114 (+   329us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:20:06.594304 (+ 45190us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:06.594540 (+   236us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:20:06.549738 18954 log.cc:927] Time spent T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 1.657s	user 0.003s	sys 0.015s
I20260709 13:20:06.540130 16217 raft_consensus.cc:1240] T 8cc959fb7f704fba9009c0595467d074 P 1ca2725d290e47d68b1a4b609c9816ac [term 6 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 5. Current term is 6. Ops: [3.3-5.4]
I20260709 13:20:06.597627 16217 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57600 (request call id 1154) took 1728 ms. Trace:
I20260709 13:20:06.598052 16217 rpcz_store.cc:276] 0709 13:20:04.868872 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:06.539349 (+1670477us) service_pool.cc:224] Handling call
0709 13:20:06.539794 (+   445us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:20:06.597353 (+ 57559us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:06.597568 (+   215us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:20:06.507741 16294 raft_consensus.cc:3060] T 8d4ab870a05347868956aa8b841b2981 P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:20:06.568042 16115 leader_election.cc:304] T 434bff3082d141838068f323bdb0d1ed P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: cb82cc7bd6db46d980c75237a1402860
I20260709 13:20:06.570039 16790 leader_election.cc:304] T 66f75124215f4ca6bef831e174a792d8 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:20:06.573607 16226 raft_consensus.cc:1240] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 6. Current term is 7. Ops: [6.8-6.8]
I20260709 13:20:06.597991 16300 raft_consensus.cc:1240] T c0eab25ce4c64e018a041beeda2867db P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 6. Current term is 7. Ops: []
I20260709 13:20:06.601287 16226 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57600 (request call id 1157) took 1713 ms. Trace:
I20260709 13:20:06.601768 16226 rpcz_store.cc:276] 0709 13:20:04.888327 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:06.572990 (+1684663us) service_pool.cc:224] Handling call
0709 13:20:06.573401 (+   411us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:20:06.600982 (+ 27581us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:06.601225 (+   243us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:20:06.574757 16299 raft_consensus.cc:1275] T 18ad7e3af5504571a69d10c00750a01c P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 13:20:06.585649 16298 raft_consensus.cc:2410] T 5a959dbecc2648eb85d1cc550f5a3245 P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac for term 6 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:20:06.585507 16220 rpcz_store.cc:276] 0709 13:20:04.939224 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:06.532765 (+1593541us) service_pool.cc:224] Handling call
0709 13:20:06.535252 (+  2487us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:20:06.605978 16218 raft_consensus.cc:1240] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac [term 6 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 4. Current term is 6. Ops: []
I20260709 13:20:06.607663 16218 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57600 (request call id 1160) took 1679 ms. Trace:
I20260709 13:20:06.609086 16218 rpcz_store.cc:276] 0709 13:20:04.928338 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:06.591019 (+1662681us) service_pool.cc:224] Handling call
0709 13:20:06.605748 (+ 14729us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:06.607368 (+  1620us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:06.607596 (+   228us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:20:06.567065 18881 raft_consensus.cc:740] T 68a6838d82ef4cac83507f906139d90f P 647a343a542d4a139fe106c19095129b [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:20:06.597154 18758 raft_consensus.cc:493] T ee9ed67006144797b1de6ea8c2013a68 P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:20:06.569161 16221 rpcz_store.cc:276] 0709 13:20:04.954305 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:05.328625 (+374320us) service_pool.cc:224] Handling call
0709 13:20:06.545258 (+1216633us) spinlock_profiling.cc:243] Waited 95.4 ms on lock 0x7b54002c8b54. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:20:06.545371 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":219454208}
I20260709 13:20:06.621425 18758 raft_consensus.cc:515] T ee9ed67006144797b1de6ea8c2013a68 P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:06.622473 16221 raft_consensus.cc:1275] T 8c11074579b046d99e585c2818e18a2c P 1ca2725d290e47d68b1a4b609c9816ac [term 6 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 13:20:06.623098 16221 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33050 (request call id 499) took 1683 ms. Trace:
I20260709 13:20:06.623449 16221 rpcz_store.cc:276] 0709 13:20:04.939710 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:06.621774 (+1682064us) service_pool.cc:224] Handling call
0709 13:20:06.622117 (+   343us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:06.622844 (+   727us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:06.623043 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:20:06.623886 16220 raft_consensus.cc:1240] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 6. Current term is 7. Ops: []
I20260709 13:20:06.624500 16220 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33050 (request call id 506) took 1683 ms. Trace:
I20260709 13:20:06.624788 16220 rpcz_store.cc:276] 0709 13:20:04.941121 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:06.623438 (+1682317us) service_pool.cc:224] Handling call
0709 13:20:06.623718 (+   280us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:06.624269 (+   551us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:06.624451 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:20:06.625876 16220 raft_consensus.cc:1240] T 7c2095a5f4ad49408dbc7f69c606120d P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 4. Current term is 5. Ops: []
I20260709 13:20:06.626411 16220 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57600 (request call id 1162) took 1631 ms. Trace:
I20260709 13:20:06.576840 19002 raft_consensus.cc:3060] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:20:06.628401 16264 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:06.628737 16219 raft_consensus.cc:1275] T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 13:20:06.629328 16219 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57604 (request call id 2143) took 1567 ms. Trace:
I20260709 13:20:06.629676 18855 raft_consensus.cc:2804] T 66f75124215f4ca6bef831e174a792d8 P cb82cc7bd6db46d980c75237a1402860 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:20:06.630112 18855 raft_consensus.cc:493] T 66f75124215f4ca6bef831e174a792d8 P cb82cc7bd6db46d980c75237a1402860 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:20:06.630473 18855 raft_consensus.cc:3060] T 66f75124215f4ca6bef831e174a792d8 P cb82cc7bd6db46d980c75237a1402860 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:20:06.595022 18720 raft_consensus.cc:493] T 23f82224f4564fea93572e8206bad1fe P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:06.631630 18720 raft_consensus.cc:3060] T 23f82224f4564fea93572e8206bad1fe P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:20:06.635680 16115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 67748us
I20260709 13:20:06.629628 16219 rpcz_store.cc:276] 0709 13:20:05.062348 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:06.628136 (+1565788us) service_pool.cc:224] Handling call
0709 13:20:06.628439 (+   303us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:06.629109 (+   670us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:06.629270 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:20:06.636655 16115 leader_election.cc:304] T b87147731a68400e8cce9b879daa7c5d P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:20:06.637676 18747 raft_consensus.cc:2804] T 434bff3082d141838068f323bdb0d1ed P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:20:06.637751 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8158bbd17b994d2eb96f560bd0f1b88d" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:20:06.638288 18747 raft_consensus.cc:697] T 434bff3082d141838068f323bdb0d1ed P 647a343a542d4a139fe106c19095129b [term 6 LEADER]: Becoming Leader. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:20:06.639338 18747 consensus_queue.cc:237] T 434bff3082d141838068f323bdb0d1ed P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:06.643990 16115 leader_election.cc:304] T 2794af5b99984efebdee564035841cb0 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860
I20260709 13:20:06.644829 18599 raft_consensus.cc:2804] T b87147731a68400e8cce9b879daa7c5d P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:20:06.645243 18599 raft_consensus.cc:493] T b87147731a68400e8cce9b879daa7c5d P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Starting leader election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:20:06.645382 18745 raft_consensus.cc:2749] T 2794af5b99984efebdee564035841cb0 P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:20:06.645624 18599 raft_consensus.cc:3060] T b87147731a68400e8cce9b879daa7c5d P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:20:06.596364 18770 consensus_meta.cc:287] T d2ab0ac8347641e6955e9bce7d90c1a2 P 1ca2725d290e47d68b1a4b609c9816ac: Time spent flushing consensus metadata: real 0.531s	user 0.011s	sys 0.000s
I20260709 13:20:06.647254 18833 consensus_queue.cc:1048] T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:20:06.648253 16293 raft_consensus.cc:1275] T 8638ee363273426db1393dd81e5c7b72 P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:20:06.648677 16115 leader_election.cc:400] T 26354361e9b044cea1c2b86c4882fe92 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 4 pre-election: Vote denied by peer 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389) with higher term. Message: Invalid argument: T 26354361e9b044cea1c2b86c4882fe92 P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b for earlier term 4. Current term is 5.
I20260709 13:20:06.648978 16115 leader_election.cc:403] T 26354361e9b044cea1c2b86c4882fe92 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 4 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:20:06.649290 16296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a878999c0024d2abe517484180290fc" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:20:06.650040 16296 raft_consensus.cc:2393] T 8a878999c0024d2abe517484180290fc P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 6: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
I20260709 13:20:06.650903 16291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7ca61592e848ea9b0fb81a2ce33cdc" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:20:06.651497 16291 raft_consensus.cc:3060] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:20:06.651515 18932 consensus_queue.cc:1048] T 7c2095a5f4ad49408dbc7f69c606120d P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:20:06.652025 16221 raft_consensus.cc:1240] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 7. Current term is 8. Ops: [7.5-7.5]
I20260709 13:20:06.652670 16221 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57600 (request call id 1161) took 1697 ms. Trace:
I20260709 13:20:06.653053 18932 raft_consensus.cc:2749] T 26354361e9b044cea1c2b86c4882fe92 P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Leader pre-election lost for term 4. Reason: Vote denied by peer 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389) with higher term. Message: Invalid argument: T 26354361e9b044cea1c2b86c4882fe92 P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b for earlier term 4. Current term is 5.
I20260709 13:20:06.653179 18599 raft_consensus.cc:515] T b87147731a68400e8cce9b879daa7c5d P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:06.654111 16292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806d7690c2b644959152074bb2cab0a7" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:06.654668 16292 raft_consensus.cc:2468] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 5.
I20260709 13:20:06.655401 18599 leader_election.cc:290] T b87147731a68400e8cce9b879daa7c5d P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 8 election: Requested vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:06.656715 16115 leader_election.cc:304] T ecc1e363230043fe9fef390f0b979f20 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:20:06.657464 18599 raft_consensus.cc:2804] T ecc1e363230043fe9fef390f0b979f20 P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:20:06.657827 18599 raft_consensus.cc:493] T ecc1e363230043fe9fef390f0b979f20 P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:06.658170 18599 raft_consensus.cc:3060] T ecc1e363230043fe9fef390f0b979f20 P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:20:06.661854 16820 raft_consensus.cc:1275] T 74ca8140f35c4ae29e8a664f6bdfc829 P cb82cc7bd6db46d980c75237a1402860 [term 7 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 13:20:06.661751 16825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66ae176961b74813a58bc15c9c4c6f2a" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:20:06.663447 16822 raft_consensus.cc:1275] T 8638ee363273426db1393dd81e5c7b72 P cb82cc7bd6db46d980c75237a1402860 [term 5 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:20:06.663514 16823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7ca61592e848ea9b0fb81a2ce33cdc" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:20:06.664208 16823 raft_consensus.cc:3060] T 3a7ca61592e848ea9b0fb81a2ce33cdc P cb82cc7bd6db46d980c75237a1402860 [term 7 FOLLOWER]: Advancing to term 9
I20260709 13:20:06.666522 16219 raft_consensus.cc:1275] T 7c2095a5f4ad49408dbc7f69c606120d P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:20:06.626693 16220 rpcz_store.cc:276] 0709 13:20:04.994647 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:06.625457 (+1630810us) service_pool.cc:224] Handling call
0709 13:20:06.625729 (+   272us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:06.626194 (+   465us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:06.626363 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:20:06.676811 16220 raft_consensus.cc:1240] T 02838c86531d4f98bf58d1c22b8baf77 P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 6. Current term is 7. Ops: []
I20260709 13:20:06.677551 16220 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57600 (request call id 1167) took 1558 ms. Trace:
I20260709 13:20:06.677909 16220 rpcz_store.cc:276] 0709 13:20:05.118955 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:06.676233 (+1557278us) service_pool.cc:224] Handling call
0709 13:20:06.676605 (+   372us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:06.677254 (+   649us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:06.677487 (+   233us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:20:06.691722 16294 raft_consensus.cc:2468] T 8d4ab870a05347868956aa8b841b2981 P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 7.
I20260709 13:20:06.692727 16219 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57604 (request call id 2144) took 1628 ms. Trace:
I20260709 13:20:06.693212 16219 rpcz_store.cc:276] 0709 13:20:05.064586 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:06.659156 (+1594570us) service_pool.cc:224] Handling call
0709 13:20:06.666147 (+  6991us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:06.666896 (+   749us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:06.692675 (+ 25779us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:20:06.695618 16821 raft_consensus.cc:1275] T 2794af5b99984efebdee564035841cb0 P cb82cc7bd6db46d980c75237a1402860 [term 5 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 13:20:06.705613 16294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c61a39fd4f84424b129c1a6f9ba404d" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 8 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:06.706857 16294 raft_consensus.cc:2410] T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac for term 8 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 2.
I20260709 13:20:06.707893 16291 raft_consensus.cc:2468] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 9.
I20260709 13:20:06.704125 16115 leader_election.cc:304] T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:20:06.707973 16220 raft_consensus.cc:1275] T 8cc959fb7f704fba9009c0595467d074 P 1ca2725d290e47d68b1a4b609c9816ac [term 6 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 13:20:06.713940 16220 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57604 (request call id 2145) took 1593 ms. Trace:
I20260709 13:20:06.714529 16220 rpcz_store.cc:276] 0709 13:20:05.120470 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:06.678777 (+1558307us) service_pool.cc:224] Handling call
0709 13:20:06.679105 (+   328us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:06.713543 (+ 34438us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:06.713874 (+   331us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:20:06.717190 16220 raft_consensus.cc:1240] T fffab26427e24d398e2695aad40ca134 P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 4. Current term is 5. Ops: []
I20260709 13:20:06.733412 16823 raft_consensus.cc:2468] T 3a7ca61592e848ea9b0fb81a2ce33cdc P cb82cc7bd6db46d980c75237a1402860 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 9.
I20260709 13:20:06.745543 16220 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57600 (request call id 1176) took 1287 ms. Trace:
I20260709 13:20:06.746035 16220 rpcz_store.cc:276] 0709 13:20:05.458225 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:06.716615 (+1258390us) service_pool.cc:224] Handling call
0709 13:20:06.716960 (+   345us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:06.745208 (+ 28248us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:06.745487 (+   279us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:20:06.749332 16219 raft_consensus.cc:1240] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 1ca2725d290e47d68b1a4b609c9816ac [term 7 LEADER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 6. Current term is 7. Ops: []
W20260709 13:20:06.756688 16192 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 75691.3us
I20260709 13:20:06.759078 16219 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57604 (request call id 2147) took 1521 ms. Trace:
I20260709 13:20:06.759629 16219 rpcz_store.cc:276] 0709 13:20:05.238254 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:06.701654 (+1463400us) service_pool.cc:224] Handling call
0709 13:20:06.744773 (+ 43119us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:06.758654 (+ 13881us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:06.759008 (+   354us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:20:06.762429 18750 raft_consensus.cc:493] T e5ad4eb281564936a59f41e387a47156 P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:06.763448 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66ae176961b74813a58bc15c9c4c6f2a" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:20:06.763339 18935 consensus_queue.cc:1048] T 19baa7bc367844c288bca8f81439c413 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:20:06.764006 16300 raft_consensus.cc:3060] T 66ae176961b74813a58bc15c9c4c6f2a P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:20:06.765097 18935 consensus_queue.cc:1048] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:20:06.763011 18750 raft_consensus.cc:515] T e5ad4eb281564936a59f41e387a47156 P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:06.768512 18750 leader_election.cc:290] T e5ad4eb281564936a59f41e387a47156 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:06.770174 16192 leader_election.cc:304] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:20:06.770634 18825 consensus_queue.cc:1048] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:20:06.771426 18587 raft_consensus.cc:2804] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:20:06.749600 18770 raft_consensus.cc:493] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:20:06.712414 18900 consensus_queue.cc:1048] T 7c2095a5f4ad49408dbc7f69c606120d P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.060s
I20260709 13:20:06.772233 18587 raft_consensus.cc:697] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 1ca2725d290e47d68b1a4b609c9816ac [term 9 LEADER]: Becoming Leader. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:20:06.772315 18770 raft_consensus.cc:515] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:06.745048 18983 consensus_queue.cc:1048] T 8cc959fb7f704fba9009c0595467d074 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 2, Time since last communication: 0.028s
I20260709 13:20:06.773138 18587 consensus_queue.cc:237] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:06.744848 18821 raft_consensus.cc:2749] T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
W20260709 13:20:06.780220 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 44018.3us
I20260709 13:20:06.653014 16221 rpcz_store.cc:276] 0709 13:20:04.955127 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:06.624265 (+1669138us) service_pool.cc:224] Handling call
0709 13:20:06.624566 (+   301us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:20:06.652409 (+ 27843us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:06.652615 (+   206us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:20:06.787853 16300 raft_consensus.cc:2468] T 66ae176961b74813a58bc15c9c4c6f2a P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 9.
I20260709 13:20:06.788990 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ad5dbfa99ee4b8d9e28be92b3f36db5" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:20:06.789680 16300 raft_consensus.cc:3060] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:20:06.695327 18944 consensus_queue.cc:1048] T 9c61a39fd4f84424b129c1a6f9ba404d P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.080s
W20260709 13:20:06.790104 16789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 378163us
I20260709 13:20:06.794349 16824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26354361e9b044cea1c2b86c4882fe92" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:20:06.794903 16190 leader_election.cc:304] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:20:06.802147 16789 leader_election.cc:304] T 74ca8140f35c4ae29e8a664f6bdfc829 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:20:06.821734 16789 leader_election.cc:304] T 8638ee363273426db1393dd81e5c7b72 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:20:06.828172 16219 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8cc959fb7f704fba9009c0595467d074"
dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
mode: GRACEFUL
new_leader_uuid: "90f3da57796b488c9087a2f523adf045"
 from {username='slave'} at 127.0.0.1:39144
I20260709 13:20:06.828994 16219 raft_consensus.cc:606] T 8cc959fb7f704fba9009c0595467d074 P 1ca2725d290e47d68b1a4b609c9816ac [term 6 FOLLOWER]: Received request to transfer leadership to TS 90f3da57796b488c9087a2f523adf045
I20260709 13:20:06.829419 16219 raft_consensus.cc:614] T 8cc959fb7f704fba9009c0595467d074 P 1ca2725d290e47d68b1a4b609c9816ac [term 6 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:20:06.838971 16300 raft_consensus.cc:2468] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 6.
I20260709 13:20:06.848457 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d16dfad0c3b540e5b4b86492ddf87208" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:06.849325 16300 raft_consensus.cc:2468] T d16dfad0c3b540e5b4b86492ddf87208 P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 5.
I20260709 13:20:06.790521 18821 raft_consensus.cc:493] T b6f07842d3aa47858df5665874c86f3a P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:06.781461 18770 leader_election.cc:290] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:06.853020 16789 leader_election.cc:304] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:20:06.790109 18750 raft_consensus.cc:493] T fffab26427e24d398e2695aad40ca134 P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:06.770135 19002 consensus_queue.cc:1059] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.040s
I20260709 13:20:06.781629 18854 consensus_queue.cc:1048] T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.070s
I20260709 13:20:06.789783 18929 consensus_queue.cc:1048] T 19baa7bc367844c288bca8f81439c413 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.084s
I20260709 13:20:06.851305 18896 raft_consensus.cc:493] T ecc1e363230043fe9fef390f0b979f20 P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:06.792832 18807 consensus_queue.cc:1059] T c0eab25ce4c64e018a041beeda2867db P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.086s
I20260709 13:20:06.800534 18935 consensus_queue.cc:1048] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.090s
I20260709 13:20:06.800695 18825 consensus_queue.cc:1048] T 18ad7e3af5504571a69d10c00750a01c P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 2, Time since last communication: 0.090s
W20260709 13:20:06.901733 16266 leader_election.cc:336] T 26354361e9b044cea1c2b86c4882fe92 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923): Timed out: RequestConsensusVote RPC to 127.14.177.132:38923 timed out after 1.551s (SENT)
W20260709 13:20:06.911720 16791 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:20:06.916669 16264 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:06.891659 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5ad4eb281564936a59f41e387a47156" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 8 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:06.923734 16300 raft_consensus.cc:2410] T e5ad4eb281564936a59f41e387a47156 P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac for term 8 because replica has last-logged OpId of term: 6 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 4.
W20260709 13:20:06.938903 16224 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57600 (request call id 1182) took 1362 ms (client timeout 1758 ms). Trace:
W20260709 13:20:06.939515 16224 rpcz_store.cc:269] 0709 13:20:05.576752 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:05.959382 (+382630us) service_pool.cc:224] Handling call
0709 13:20:06.938717 (+979335us) spinlock_profiling.cc:243] Waited 543 ms on lock 0x7b5400138654. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:20:06.938843 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1248423808}
W20260709 13:20:06.930445 19006 log.cc:927] Time spent T 8158bbd17b994d2eb96f560bd0f1b88d P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 0.057s	user 0.000s	sys 0.001s
I20260709 13:20:06.879388 18821 raft_consensus.cc:515] T b6f07842d3aa47858df5665874c86f3a P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:06.878957 18750 raft_consensus.cc:515] T fffab26427e24d398e2695aad40ca134 P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:06.941099 16224 raft_consensus.cc:1275] T 5a959dbecc2648eb85d1cc550f5a3245 P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
W20260709 13:20:06.954551 16117 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 404908us
W20260709 13:20:06.956312 16117 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:06.960191 16300 raft_consensus.cc:1240] T 8d4ab870a05347868956aa8b841b2981 P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 6. Current term is 7. Ops: [6.3-6.3]
W20260709 13:20:06.969343 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 368842us
W20260709 13:20:06.970930 16266 leader_election.cc:336] T 3884f864a96d428785895e5463e5728f P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923): Timed out: RequestConsensusVote RPC to 127.14.177.132:38923 timed out after 2.586s (SENT)
W20260709 13:20:06.972239 16264 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:06.974572 16790 leader_election.cc:304] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:20:06.976197 16224 raft_consensus.cc:1275] T 1df93779209748b9860f6d893dd3de53 P 1ca2725d290e47d68b1a4b609c9816ac [term 6 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 13:20:06.918944 18896 raft_consensus.cc:515] T ecc1e363230043fe9fef390f0b979f20 P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:06.951530 19009 raft_consensus.cc:493] T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:20:06.941017 18983 consensus_queue.cc:1059] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 647a343a542d4a139fe106c19095129b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 3, Time since last communication: 0.036s
I20260709 13:20:06.984278 16221 raft_consensus.cc:1275] T fffab26427e24d398e2695aad40ca134 P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 13:20:06.961778 18992 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.524s	user 0.006s	sys 0.012s
I20260709 13:20:06.952430 18946 raft_consensus.cc:493] T 746aabf856ab401c9ef5e859371e7b67 P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:20:06.988549 16224 raft_consensus.cc:1275] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:20:07.000190 16224 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57604 (request call id 2154) took 1014 ms. Trace:
I20260709 13:20:07.000854 16224 rpcz_store.cc:276] 0709 13:20:05.985666 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:06.977209 (+991543us) service_pool.cc:224] Handling call
0709 13:20:06.988233 (+ 11024us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:06.999792 (+ 11559us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:07.000109 (+   317us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:20:06.983536 16115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 205070us
I20260709 13:20:06.973697 18421 raft_consensus.cc:493] T 08c74490c40243a88e62d1760b5cd814 P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
W20260709 13:20:06.992614 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 200442us
I20260709 13:20:06.948171 18944 consensus_queue.cc:1048] T 9c61a39fd4f84424b129c1a6f9ba404d P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.034s
I20260709 13:20:06.993526 16300 raft_consensus.cc:1275] T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
W20260709 13:20:07.003775 16789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 150862us
I20260709 13:20:07.035122 16224 raft_consensus.cc:1275] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
W20260709 13:20:07.038371 16264 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:07.045251 16224 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57604 (request call id 2155) took 1059 ms. Trace:
I20260709 13:20:07.045804 16224 rpcz_store.cc:276] 0709 13:20:05.985976 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:07.002038 (+1016062us) service_pool.cc:224] Handling call
0709 13:20:07.033816 (+ 31778us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:07.035860 (+  2044us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:07.045169 (+  9309us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:20:07.031303 18843 raft_consensus.cc:493] T 8638ee363273426db1393dd81e5c7b72 P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:20:07.003413 18992 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.568s	user 0.006s	sys 0.012s
W20260709 13:20:07.044520 16266 leader_election.cc:336] T 2794af5b99984efebdee564035841cb0 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923): Timed out: RequestConsensusVote RPC to 127.14.177.132:38923 timed out after 1.710s (SENT)
I20260709 13:20:07.049888 19002 consensus_queue.cc:1243] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Peer 647a343a542d4a139fe106c19095129b log is divergent from this leader: its last log entry 8.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 13:20:07.006213 18946 raft_consensus.cc:515] T 746aabf856ab401c9ef5e859371e7b67 P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:07.052209 18946 leader_election.cc:290] T 746aabf856ab401c9ef5e859371e7b67 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:07.056741 16142 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40830 (request call id 1215) took 1655 ms. Trace:
I20260709 13:20:07.057310 16142 rpcz_store.cc:276] 0709 13:20:05.401388 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:06.128995 (+727607us) service_pool.cc:224] Handling call
0709 13:20:06.157669 (+ 28674us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:20:06.196183 (+ 38514us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:06.196214 (+    31us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:20:06.196778 (+   564us) log.cc:844] Serialized 52 byte log entry
0709 13:20:06.467619 (+270841us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:20:06.467692 (+    73us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:06.467797 (+   105us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:20:07.056450 (+588653us) raft_consensus.cc:1685] finished
0709 13:20:07.056480 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:07.056663 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":14116,"raft.run_cpu_time_us":204,"raft.run_wall_time_us":5278,"thread_start_us":222216,"threads_started":1,"wal-append.queue_time_us":580000}
I20260709 13:20:07.053113 18854 raft_consensus.cc:493] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:20:07.061292 18854 raft_consensus.cc:515] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:07.064356 18855 raft_consensus.cc:515] T 66f75124215f4ca6bef831e174a792d8 P cb82cc7bd6db46d980c75237a1402860 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:07.066325 18855 leader_election.cc:290] T 66f75124215f4ca6bef831e174a792d8 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 7 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:20:07.066859 18855 raft_consensus.cc:2804] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:20:07.050953 18821 leader_election.cc:290] T b6f07842d3aa47858df5665874c86f3a P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:07.016264 18983 consensus_queue.cc:1243] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 647a343a542d4a139fe106c19095129b [LEADER]: Peer 1ca2725d290e47d68b1a4b609c9816ac log is divergent from this leader: its last log entry 7.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260709 13:20:07.072213 16224 raft_consensus.cc:1240] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 1ca2725d290e47d68b1a4b609c9816ac [term 6 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 5. Current term is 6. Ops: []
I20260709 13:20:07.074318 16300 raft_consensus.cc:1275] T e4dc6b0541ca431ebc757beac00521f1 P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:20:07.048804 18996 raft_consensus.cc:493] T 8cc959fb7f704fba9009c0595467d074 P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:07.047449 19021 raft_consensus.cc:2764] T 74ca8140f35c4ae29e8a664f6bdfc829 P cb82cc7bd6db46d980c75237a1402860 [term 7 FOLLOWER]: Leader pre-election decision vote started in defunct term 6: won
I20260709 13:20:07.044425 18743 raft_consensus.cc:493] T ada5a004ba7c46819051e3ccec71ffdb P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:20:07.084782 19002 raft_consensus.cc:3055] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:20:07.091784 16220 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57604 (request call id 2149) took 1124 ms. Trace:
I20260709 13:20:07.049517 18992 raft_consensus.cc:697] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [term 8 LEADER]: Becoming Leader. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:20:07.090024 16824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a9de347c711477e87c582d003f36925" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:20:07.096675 16300 raft_consensus.cc:1275] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260709 13:20:07.016045 18421 raft_consensus.cc:515] T 08c74490c40243a88e62d1760b5cd814 P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:07.067745 18855 raft_consensus.cc:493] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:07.070614 19017 raft_consensus.cc:2804] T 8638ee363273426db1393dd81e5c7b72 P cb82cc7bd6db46d980c75237a1402860 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:20:07.092520 16224 raft_consensus.cc:1240] T 19baa7bc367844c288bca8f81439c413 P 1ca2725d290e47d68b1a4b609c9816ac [term 7 LEADER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 6. Current term is 7. Ops: [4.5-6.6]
I20260709 13:20:07.154526 16823 raft_consensus.cc:1240] T 57a744f5aaf944678a63469cbf514015 P cb82cc7bd6db46d980c75237a1402860 [term 4 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 3. Current term is 4. Ops: []
I20260709 13:20:07.106945 16190 leader_election.cc:304] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:20:07.106252 16220 rpcz_store.cc:276] 0709 13:20:05.967583 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:06.748212 (+780629us) service_pool.cc:224] Handling call
0709 13:20:06.748589 (+   377us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:06.748825 (+   236us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:20:06.957576 (+208751us) log.cc:844] Serialized 14 byte log entry
0709 13:20:07.088488 (+130912us) log.cc:844] Serialized 14 byte log entry
0709 13:20:07.091464 (+  2976us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:20:07.091518 (+    54us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:20:07.091564 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:07.091622 (+    58us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:07.091731 (+   109us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":250399,"prepare.run_cpu_time_us":47,"prepare.run_wall_time_us":70,"raft.queue_time_us":2863,"raft.run_wall_time_us":12045,"spinlock_wait_cycles":20428288,"thread_start_us":305491,"threads_started":2}
W20260709 13:20:07.155695 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 48817.3us
I20260709 13:20:07.156270 16220 raft_consensus.cc:1240] T 68a6838d82ef4cac83507f906139d90f P 1ca2725d290e47d68b1a4b609c9816ac [term 8 LEADER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 7. Current term is 8. Ops: [7.5-7.5]
I20260709 13:20:07.095006 18821 consensus_queue.cc:1048] T 1df93779209748b9860f6d893dd3de53 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.057s
I20260709 13:20:07.098335 18952 raft_consensus.cc:2804] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:20:07.068967 18843 raft_consensus.cc:515] T 8638ee363273426db1393dd81e5c7b72 P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:07.151109 16821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5718c623ebee4d2bb770f0f9ddb2f164" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:07.165819 16821 raft_consensus.cc:2468] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 6.
I20260709 13:20:07.095191 19024 raft_consensus.cc:493] T 3884f864a96d428785895e5463e5728f P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:07.097015 18983 consensus_queue.cc:1059] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 647a343a542d4a139fe106c19095129b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.062s
I20260709 13:20:07.156385 16190 leader_election.cc:304] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
W20260709 13:20:07.172892 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 198404us
I20260709 13:20:07.105893 19002 raft_consensus.cc:740] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:20:07.164361 16822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26354361e9b044cea1c2b86c4882fe92" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:07.000272 19009 raft_consensus.cc:515] T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:07.167480 16821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62fe3346cd3c42c4a55e14af9073bca4" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:20:07.094647 18854 leader_election.cc:290] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:07.148516 18855 raft_consensus.cc:3060] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:20:07.154090 19021 raft_consensus.cc:2804] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:20:07.157279 16300 raft_consensus.cc:1275] T cdbd78acb4cf46298e0de8ba0d130e19 P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
W20260709 13:20:07.162973 19015 log.cc:927] Time spent T af57996a8be84d398f0f023e8d08ef77 P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 0.074s	user 0.000s	sys 0.001s
I20260709 13:20:07.154141 18900 consensus_queue.cc:1048] T 5a959dbecc2648eb85d1cc550f5a3245 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.027s
W20260709 13:20:07.176659 16222 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57600 (request call id 1163) took 2154 ms (client timeout 2077 ms). Trace:
W20260709 13:20:07.195237 16222 rpcz_store.cc:269] 0709 13:20:05.022553 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:05.203154 (+180601us) service_pool.cc:224] Handling call
0709 13:20:07.176463 (+1973309us) spinlock_profiling.cc:243] Waited 936 ms on lock 0x7b54002c8654. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:20:07.176590 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2152787328}
I20260709 13:20:07.197499 16821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee9ed67006144797b1de6ea8c2013a68" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:07.171358 16817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af57996a8be84d398f0f023e8d08ef77" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:07.153831 18992 consensus_queue.cc:237] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:07.161795 18770 raft_consensus.cc:2804] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:20:07.103538 18996 raft_consensus.cc:515] T 8cc959fb7f704fba9009c0595467d074 P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:07.162348 19017 raft_consensus.cc:493] T 8638ee363273426db1393dd81e5c7b72 P cb82cc7bd6db46d980c75237a1402860 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:07.184509 18750 leader_election.cc:290] T fffab26427e24d398e2695aad40ca134 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:07.186820 16820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "829b4ad8c5a54eababeea1181b960d4b" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:07.187144 18901 raft_consensus.cc:2804] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 1ca2725d290e47d68b1a4b609c9816ac [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:20:07.187947 19002 consensus_queue.cc:260] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:07.188956 19009 leader_election.cc:290] T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:07.189570 16816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5718c623ebee4d2bb770f0f9ddb2f164" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:20:07.237712 19017 raft_consensus.cc:3060] T 8638ee363273426db1393dd81e5c7b72 P cb82cc7bd6db46d980c75237a1402860 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:20:07.191280 16266 leader_election.cc:336] T 31969392ccdb43b491911a319e8bc7e1 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923): Timed out: RequestConsensusVote RPC to 127.14.177.132:38923 timed out after 1.867s (SENT)
W20260709 13:20:07.261313 16266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 70057.5us
I20260709 13:20:07.105741 18743 raft_consensus.cc:515] T ada5a004ba7c46819051e3ccec71ffdb P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:07.191802 16822 raft_consensus.cc:3060] T 806d7690c2b644959152074bb2cab0a7 P cb82cc7bd6db46d980c75237a1402860 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:20:07.173919 16819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a378d7f796f94615b3b347119a9880fd" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
W20260709 13:20:07.179639 19011 log.cc:927] Time spent T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 0.220s	user 0.001s	sys 0.000s
I20260709 13:20:07.202824 16821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f82224f4564fea93572e8206bad1fe" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:07.177493 19024 raft_consensus.cc:515] T 3884f864a96d428785895e5463e5728f P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:07.210330 18421 leader_election.cc:290] T 08c74490c40243a88e62d1760b5cd814 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:07.211604 16817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87147731a68400e8cce9b879daa7c5d" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:07.190536 19021 raft_consensus.cc:493] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:07.315858 19021 raft_consensus.cc:3060] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:20:07.317070 19024 leader_election.cc:290] T 3884f864a96d428785895e5463e5728f P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:07.316780 16817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 6 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:07.192617 18870 raft_consensus.cc:3060] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:20:07.177086 16790 leader_election.cc:304] T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:20:07.231510 16823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66f75124215f4ca6bef831e174a792d8" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
W20260709 13:20:07.210589 18720 consensus_meta.cc:287] T 23f82224f4564fea93572e8206bad1fe P 647a343a542d4a139fe106c19095129b: Time spent flushing consensus metadata: real 0.578s	user 0.010s	sys 0.002s
I20260709 13:20:07.323856 18720 raft_consensus.cc:515] T 23f82224f4564fea93572e8206bad1fe P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:07.326664 18720 leader_election.cc:290] T 23f82224f4564fea93572e8206bad1fe P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 5 election: Requested vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:07.244709 16820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2372981006d4b218215c28964dea623" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:07.267557 16819 raft_consensus.cc:3060] T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860 [term 4 FOLLOWER]: Advancing to term 6
I20260709 13:20:07.332110 19021 raft_consensus.cc:515] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:20:07.332868 18599 consensus_meta.cc:287] T ecc1e363230043fe9fef390f0b979f20 P 647a343a542d4a139fe106c19095129b: Time spent flushing consensus metadata: real 0.674s	user 0.013s	sys 0.004s
I20260709 13:20:07.333356 18599 raft_consensus.cc:515] T ecc1e363230043fe9fef390f0b979f20 P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:20:07.275580 16266 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:20:07.336673 16266 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:07.307487 18843 leader_election.cc:290] T 8638ee363273426db1393dd81e5c7b72 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:07.337265 19021 leader_election.cc:290] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 6 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:07.338084 18843 raft_consensus.cc:493] T fffab26427e24d398e2695aad40ca134 P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:07.338774 18843 raft_consensus.cc:515] T fffab26427e24d398e2695aad40ca134 P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:20:07.340003 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 163008us
I20260709 13:20:07.237955 18770 raft_consensus.cc:697] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [term 9 LEADER]: Becoming Leader. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:20:07.309418 16821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b7d1ab519044eda2d5e04a07242f25" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:07.341509 16790 leader_election.cc:304] T 57a744f5aaf944678a63469cbf514015 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:20:07.353744 16819 raft_consensus.cc:2468] T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 6.
I20260709 13:20:07.354640 16819 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49906 (request call id 604) took 1203 ms. Trace:
I20260709 13:20:07.355039 16819 rpcz_store.cc:276] 0709 13:20:06.151514 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:06.152015 (+   501us) service_pool.cc:224] Handling call
0709 13:20:07.173756 (+1021741us) spinlock_profiling.cc:243] Waited 1.48 ms on lock 0x7b5800004298. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69ad8fadb1 00007f69ada3ab71 00007f69ada3aa22 00007f69b6953c09 00007f69b5aadee0 00007f69b5b770ab 00007f69b5b77131 00007f69b5b250a0 00007f69b5b24984 00007f69af09f6d4 00007f69af09f662
0709 13:20:07.354575 (+180819us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3405696}
I20260709 13:20:07.358982 16816 raft_consensus.cc:2468] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 6.
I20260709 13:20:07.360334 16816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee9ed67006144797b1de6ea8c2013a68" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
W20260709 13:20:07.369751 16116 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:20:07.375527 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 1.76956e+06us
I20260709 13:20:07.377136 16790 leader_election.cc:304] T 3884f864a96d428785895e5463e5728f P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:20:07.250998 18952 raft_consensus.cc:493] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Starting leader election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:20:07.382230 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0eab25ce4c64e018a041beeda2867db" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:20:07.388814 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4f65cdf00b748bb963ee6e81ef43b79" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:07.389478 16226 raft_consensus.cc:2468] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 8.
I20260709 13:20:07.388630 18952 raft_consensus.cc:3060] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:20:07.380957 16218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdbd78acb4cf46298e0de8ba0d130e19" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:07.389107 16817 raft_consensus.cc:2393] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 6: Already voted for candidate cb82cc7bd6db46d980c75237a1402860 in this term.
W20260709 13:20:07.398715 16264 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:07.361593 18901 raft_consensus.cc:697] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 1ca2725d290e47d68b1a4b609c9816ac [term 6 LEADER]: Becoming Leader. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:20:07.369626 18843 leader_election.cc:290] T fffab26427e24d398e2695aad40ca134 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
W20260709 13:20:07.379295 16115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 371322us
I20260709 13:20:07.378031 19008 raft_consensus.cc:2804] T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860 [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 13:20:07.480945 16116 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:20:07.435272 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 278933us
I20260709 13:20:07.828450 16190 leader_election.cc:304] T d16dfad0c3b540e5b4b86492ddf87208 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:20:07.441994 18770 consensus_queue.cc:237] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:07.417510 16146 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40830 (request call id 1199) took 2290 ms. Trace:
I20260709 13:20:07.832109 16223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6f07842d3aa47858df5665874c86f3a" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:07.844262 16146 rpcz_store.cc:276] 0709 13:20:05.127233 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:06.061413 (+934180us) service_pool.cc:224] Handling call
0709 13:20:06.061727 (+   314us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:06.479681 (+417954us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:07.410908 (+931227us) spinlock_profiling.cc:243] Waited 417 ms on lock 0x7b54001f1dd4. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:20:07.417425 (+  6517us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":958713088}
I20260709 13:20:07.393447 19009 raft_consensus.cc:2749] T 3884f864a96d428785895e5463e5728f P cb82cc7bd6db46d980c75237a1402860 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:20:07.489321 16820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5d59407eb545b895d8d922e0dd69fd" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:20:07.387334 19038 raft_consensus.cc:2749] T 57a744f5aaf944678a63469cbf514015 P cb82cc7bd6db46d980c75237a1402860 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:20:07.498625 19002 raft_consensus.cc:3060] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:20:07.551062 16115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 55189us
W20260709 13:20:07.576428 16264 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:07.410612 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8638ee363273426db1393dd81e5c7b72" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:07.856518 16115 leader_election.cc:304] T 02838c86531d4f98bf58d1c22b8baf77 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
W20260709 13:20:07.483568 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 106519us
I20260709 13:20:07.857358 16790 leader_election.cc:304] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:20:07.857826 16115 leader_election.cc:304] T 8d4ab870a05347868956aa8b841b2981 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac
I20260709 13:20:07.484957 16822 raft_consensus.cc:1275] T 806d7690c2b644959152074bb2cab0a7 P cb82cc7bd6db46d980c75237a1402860 [term 5 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 5 index: 5. (term mismatch)
W20260709 13:20:07.483273 16265 connection.cc:256] RPC call timeout handler was delayed by -1.7640917929998068s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20830000000000004s for a response
I20260709 13:20:07.387557 19035 raft_consensus.cc:493] T 2794af5b99984efebdee564035841cb0 P cb82cc7bd6db46d980c75237a1402860 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:07.540922 19008 raft_consensus.cc:493] T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:07.478965 18996 leader_election.cc:290] T 8cc959fb7f704fba9009c0595467d074 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:20:07.479043 18743 leader_election.cc:290] T ada5a004ba7c46819051e3ccec71ffdb P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:07.449184 19017 raft_consensus.cc:515] T 8638ee363273426db1393dd81e5c7b72 P cb82cc7bd6db46d980c75237a1402860 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:07.782593 19030 raft_consensus.cc:493] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P cb82cc7bd6db46d980c75237a1402860 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:07.886581 18114 consensus_queue.cc:1059] T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: INVALID_TERM, Last received: 5.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.003s
I20260709 13:20:07.886616 18956 consensus_queue.cc:1048] T e4dc6b0541ca431ebc757beac00521f1 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.003s
I20260709 13:20:07.889425 19030 raft_consensus.cc:515] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P cb82cc7bd6db46d980c75237a1402860 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:07.833634 16220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746aabf856ab401c9ef5e859371e7b67" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:07.764084 18896 leader_election.cc:290] T ecc1e363230043fe9fef390f0b979f20 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:07.891357 18956 consensus_queue.cc:1048] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:20:07.836936 16192 leader_election.cc:304] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 90f3da57796b488c9087a2f523adf045
W20260709 13:20:07.892757 16192 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 55943.3us
I20260709 13:20:07.892524 19009 consensus_queue.cc:1048] T 23f82224f4564fea93572e8206bad1fe P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:20:07.893829 19043 raft_consensus.cc:3060] T 02838c86531d4f98bf58d1c22b8baf77 P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:20:07.895875 19030 leader_election.cc:290] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:07.840624 16221 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57604 (request call id 2150) took 1872 ms. Trace:
W20260709 13:20:07.856639 16264 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:20:07.896895 16264 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:07.896646 16221 rpcz_store.cc:276] 0709 13:20:05.967814 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:06.801522 (+833708us) service_pool.cc:224] Handling call
0709 13:20:06.891232 (+ 89710us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:06.992443 (+101211us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:07.840391 (+847948us) spinlock_profiling.cc:243] Waited 87 ms on lock 0x7b5400140fd4. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:20:07.840568 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":200014592}
W20260709 13:20:07.750595 16266 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:20:07.898306 16266 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:07.387862 19033 consensus_queue.cc:1048] T 26354361e9b044cea1c2b86c4882fe92 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.427s
W20260709 13:20:07.879753 16222 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.501s	user 0.006s	sys 0.040s
W20260709 13:20:07.899302 16222 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.524s	user 0.007s	sys 0.040s
W20260709 13:20:07.858786 16265 connection.cc:256] RPC call timeout handler was delayed by -1.7514917929997864s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2097s for a response
W20260709 13:20:07.900040 16265 connection.cc:256] RPC call timeout handler was delayed by -1.7514917929997864s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2097s for a response
I20260709 13:20:07.858584 16822 pending_rounds.cc:85] T 806d7690c2b644959152074bb2cab0a7 P cb82cc7bd6db46d980c75237a1402860: Aborting all ops after (but not including) 4
I20260709 13:20:07.900825 16822 pending_rounds.cc:107] T 806d7690c2b644959152074bb2cab0a7 P cb82cc7bd6db46d980c75237a1402860: Aborting uncommitted CHANGE_CONFIG_OP operation due to leader change: 4.5
I20260709 13:20:07.900854 16221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c74490c40243a88e62d1760b5cd814" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:07.844627 16223 raft_consensus.cc:2468] T b6f07842d3aa47858df5665874c86f3a P 1ca2725d290e47d68b1a4b609c9816ac [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 6.
I20260709 13:20:07.806463 18599 leader_election.cc:290] T ecc1e363230043fe9fef390f0b979f20 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 6 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
W20260709 13:20:07.885599 16117 leader_election.cc:336] T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923): Timed out: RequestConsensusVote RPC to 127.14.177.132:38923 timed out after 1.548s (SENT)
I20260709 13:20:07.901186 16822 raft_consensus.cc:2917] T 806d7690c2b644959152074bb2cab0a7 P cb82cc7bd6db46d980c75237a1402860 [term 5 FOLLOWER]: Aborting config change with OpId 4.5: Aborted: Op aborted by new leader
I20260709 13:20:07.853866 18720 raft_consensus.cc:493] T 18ad7e3af5504571a69d10c00750a01c P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:07.902297 16220 raft_consensus.cc:1275] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
W20260709 13:20:07.901639 16265 connection.cc:256] RPC call timeout handler was delayed by -1.6155917929997941s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22480000000000003s for a response
W20260709 13:20:07.903188 16225 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33050 (request call id 512) took 2923 ms (client timeout 2573 ms). Trace:
I20260709 13:20:07.907333 16220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 6 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:20:07.904696 16117 leader_election.cc:304] T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860
I20260709 13:20:07.904556 16822 raft_consensus.cc:2889] T 806d7690c2b644959152074bb2cab0a7 P cb82cc7bd6db46d980c75237a1402860 [term 5 LEARNER]: CHANGE_CONFIG_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:20:07.882074 19021 raft_consensus.cc:2804] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:20:07.905571 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f5f89186204242bf377f7c42b2e2da" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:07.881536 19008 raft_consensus.cc:3060] T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:20:07.908806 16220 raft_consensus.cc:3060] T ecc1e363230043fe9fef390f0b979f20 P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:20:07.882251 19039 raft_consensus.cc:2804] T d16dfad0c3b540e5b4b86492ddf87208 P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 13:20:07.910029 16265 connection.cc:256] RPC call timeout handler was delayed by -1.587526261999983s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.3275s for a response
W20260709 13:20:07.914335 16265 connection.cc:256] RPC call timeout handler was delayed by -1.5632715619999544s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.41100000000000003s for a response
W20260709 13:20:07.915026 16265 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:07.909958 16822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66f75124215f4ca6bef831e174a792d8" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
W20260709 13:20:07.915571 16265 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:07.910066 16224 raft_consensus.cc:2393] T c2f5f89186204242bf377f7c42b2e2da P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 8: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
W20260709 13:20:07.916203 16265 connection.cc:256] RPC call timeout handler was delayed by -1.4999160009999741s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1758s for a response
I20260709 13:20:07.909385 16223 raft_consensus.cc:1275] T af57996a8be84d398f0f023e8d08ef77 P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
W20260709 13:20:07.907976 16225 rpcz_store.cc:269] 0709 13:20:04.980256 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:05.458853 (+478597us) service_pool.cc:224] Handling call
0709 13:20:07.900596 (+2441743us) spinlock_profiling.cc:243] Waited 1.02 s on lock 0x7b5400395154. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:20:07.902916 (+  2320us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2351201152}
W20260709 13:20:07.916682 16265 connection.cc:256] RPC call timeout handler was delayed by -1.4510076739998112s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1626s for a response
I20260709 13:20:07.915602 16822 raft_consensus.cc:2393] T 66f75124215f4ca6bef831e174a792d8 P cb82cc7bd6db46d980c75237a1402860 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 7: Already voted for candidate cb82cc7bd6db46d980c75237a1402860 in this term.
W20260709 13:20:07.917145 16265 connection.cc:256] RPC call timeout handler was delayed by -1.2796111959999052s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1865s for a response
I20260709 13:20:07.918226 16822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2372981006d4b218215c28964dea623" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:20:07.919463 16822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87147731a68400e8cce9b879daa7c5d" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:20:07.911062 19039 raft_consensus.cc:493] T d16dfad0c3b540e5b4b86492ddf87208 P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Starting leader election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:20:07.887022 19010 consensus_queue.cc:1048] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.029s
I20260709 13:20:07.924960 19039 raft_consensus.cc:3060] T d16dfad0c3b540e5b4b86492ddf87208 P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:20:07.925755 19040 raft_consensus.cc:2804] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:20:07.926278 19040 raft_consensus.cc:697] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac [term 6 LEADER]: Becoming Leader. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:20:07.909377 18720 raft_consensus.cc:515] T 18ad7e3af5504571a69d10c00750a01c P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } }
I20260709 13:20:07.927155 19040 consensus_queue.cc:237] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:07.928121 18720 leader_election.cc:290] T 18ad7e3af5504571a69d10c00750a01c P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:20:07.913324 19021 raft_consensus.cc:493] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:20:07.928869 19021 raft_consensus.cc:3060] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:20:07.929279 16295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31969392ccdb43b491911a319e8bc7e1" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:07.929867 16295 raft_consensus.cc:2468] T 31969392ccdb43b491911a319e8bc7e1 P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 5.
I20260709 13:20:07.930684 19042 raft_consensus.cc:493] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:07.930836 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806d7690c2b644959152074bb2cab0a7" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:07.931510 16299 raft_consensus.cc:2468] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 5.
I20260709 13:20:07.931237 19042 raft_consensus.cc:515] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:07.932389 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9944d5e4b8c4109af9955c4f08c9095" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
W20260709 13:20:07.933513 18952 consensus_meta.cc:287] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac: Time spent flushing consensus metadata: real 0.542s	user 0.007s	sys 0.000s
I20260709 13:20:07.933518 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e65676b21b748728ad39668cfe722ec" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:20:07.934748 16294 raft_consensus.cc:1240] T 26354361e9b044cea1c2b86c4882fe92 P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 4. Current term is 5. Ops: []
I20260709 13:20:07.941985 16265 leader_election.cc:304] T 8a878999c0024d2abe517484180290fc P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:20:07.911288 18599 raft_consensus.cc:2804] T 8d4ab870a05347868956aa8b841b2981 P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:20:07.943805 16822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62fe3346cd3c42c4a55e14af9073bca4" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:20:07.945516 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a378d7f796f94615b3b347119a9880fd" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:07.946743 16822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746aabf856ab401c9ef5e859371e7b67" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:07.944926 18743 consensus_queue.cc:1048] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.012s
I20260709 13:20:07.957764 18551 raft_consensus.cc:3055] T c0eab25ce4c64e018a041beeda2867db P 1ca2725d290e47d68b1a4b609c9816ac [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:20:07.957851 18743 consensus_queue.cc:1059] T 9e65676b21b748728ad39668cfe722ec P 90f3da57796b488c9087a2f523adf045 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.8, Next index: 9, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:20:07.958360 18551 raft_consensus.cc:740] T c0eab25ce4c64e018a041beeda2867db P 1ca2725d290e47d68b1a4b609c9816ac [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:20:07.904898 18932 consensus_queue.cc:1048] T 5a96373bb0e14dce8a266b2fc6c58507 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.041s
I20260709 13:20:07.887465 19035 raft_consensus.cc:515] T 2794af5b99984efebdee564035841cb0 P cb82cc7bd6db46d980c75237a1402860 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:07.965229 18551 consensus_queue.cc:260] T c0eab25ce4c64e018a041beeda2867db P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:07.965893 19017 leader_election.cc:290] T 8638ee363273426db1393dd81e5c7b72 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 6 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:07.959414 18599 raft_consensus.cc:697] T 8d4ab870a05347868956aa8b841b2981 P 647a343a542d4a139fe106c19095129b [term 7 LEADER]: Becoming Leader. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:20:07.953583 18843 consensus_queue.cc:1048] T 57a744f5aaf944678a63469cbf514015 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.013s
I20260709 13:20:07.907914 18901 consensus_queue.cc:237] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:07.961534 19048 consensus_queue.cc:1059] T c9944d5e4b8c4109af9955c4f08c9095 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.8, Next index: 9, Last known committed idx: 6, Time since last communication: 0.014s
I20260709 13:20:07.933882 19042 leader_election.cc:290] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:07.903972 19037 consensus_queue.cc:1048] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 3, Time since last communication: 0.061s
I20260709 13:20:07.944717 19050 raft_consensus.cc:493] T 5a959dbecc2648eb85d1cc550f5a3245 P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:07.945266 18952 raft_consensus.cc:515] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac [term 6 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } }
I20260709 13:20:07.985479 18996 raft_consensus.cc:2804] T 8a878999c0024d2abe517484180290fc P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:20:08.006381 19066 raft_consensus.cc:3055] T c9944d5e4b8c4109af9955c4f08c9095 P 90f3da57796b488c9087a2f523adf045 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:20:08.007308 19066 raft_consensus.cc:740] T c9944d5e4b8c4109af9955c4f08c9095 P 90f3da57796b488c9087a2f523adf045 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:20:07.993213 19048 raft_consensus.cc:3055] T 9e65676b21b748728ad39668cfe722ec P 90f3da57796b488c9087a2f523adf045 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:20:08.018158 19066 consensus_queue.cc:260] T c9944d5e4b8c4109af9955c4f08c9095 P 90f3da57796b488c9087a2f523adf045 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:08.021258 19066 raft_consensus.cc:3060] T c9944d5e4b8c4109af9955c4f08c9095 P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:20:08.024272 19035 leader_election.cc:290] T 2794af5b99984efebdee564035841cb0 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:07.989154 19050 raft_consensus.cc:515] T 5a959dbecc2648eb85d1cc550f5a3245 P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:08.026996 19050 leader_election.cc:290] T 5a959dbecc2648eb85d1cc550f5a3245 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:20:08.010622 19048 raft_consensus.cc:740] T 9e65676b21b748728ad39668cfe722ec P 90f3da57796b488c9087a2f523adf045 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:20:08.030592 19048 consensus_queue.cc:260] T 9e65676b21b748728ad39668cfe722ec P 90f3da57796b488c9087a2f523adf045 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:08.013444 19039 raft_consensus.cc:515] T d16dfad0c3b540e5b4b86492ddf87208 P 1ca2725d290e47d68b1a4b609c9816ac [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:08.036381 19043 raft_consensus.cc:2749] T 02838c86531d4f98bf58d1c22b8baf77 P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:20:08.070399 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 128261us
I20260709 13:20:08.072980 16265 leader_election.cc:304] T 31969392ccdb43b491911a319e8bc7e1 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
W20260709 13:20:08.076470 16117 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 190780us
I20260709 13:20:08.077039 16265 leader_election.cc:304] T b87147731a68400e8cce9b879daa7c5d P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
W20260709 13:20:08.077611 16117 leader_election.cc:336] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 7 election: RPC error from VoteRequest() call to peer cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923): Timed out: RequestConsensusVote RPC to 127.14.177.132:38923 timed out after 1.605s (SENT)
I20260709 13:20:08.078495 16117 leader_election.cc:304] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860
W20260709 13:20:08.079715 16266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 141149us
I20260709 13:20:08.083441 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18ad7e3af5504571a69d10c00750a01c" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:08.086139 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:08.095814 16292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e4bfb9090c8460eb14fa7e0d818e7f1" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:08.096122 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 6 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:20:08.097070 16292 raft_consensus.cc:2468] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 5.
I20260709 13:20:08.097491 16300 raft_consensus.cc:3060] T ecc1e363230043fe9fef390f0b979f20 P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:20:08.108974 16789 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:08.111068 16265 leader_election.cc:304] T 26354361e9b044cea1c2b86c4882fe92 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
W20260709 13:20:08.111697 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 253007us
I20260709 13:20:08.116020 16821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26354361e9b044cea1c2b86c4882fe92" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:08.117523 16821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 6 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:08.119540 16821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ab0ac8347641e6955e9bce7d90c1a2" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:08.121107 16821 raft_consensus.cc:2393] T d2ab0ac8347641e6955e9bce7d90c1a2 P cb82cc7bd6db46d980c75237a1402860 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 6: Already voted for candidate cb82cc7bd6db46d980c75237a1402860 in this term.
I20260709 13:20:08.122603 16295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2794af5b99984efebdee564035841cb0" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:08.120965 16789 leader_election.cc:304] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 647a343a542d4a139fe106c19095129b
I20260709 13:20:08.122686 16821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f82224f4564fea93572e8206bad1fe" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
W20260709 13:20:08.133775 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 250018us
W20260709 13:20:08.135481 16115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 277551us
I20260709 13:20:08.148768 19043 raft_consensus.cc:2749] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:20:08.152228 19039 leader_election.cc:290] T d16dfad0c3b540e5b4b86492ddf87208 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 6 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:08.149605 18599 consensus_queue.cc:237] T 8d4ab870a05347868956aa8b841b2981 P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.2, Last appended by leader: 2, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:08.154050 19004 raft_consensus.cc:2804] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:20:08.154659 19004 raft_consensus.cc:493] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:20:08.152436 19003 raft_consensus.cc:493] T 434bff3082d141838068f323bdb0d1ed P cb82cc7bd6db46d980c75237a1402860 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:08.155980 19003 raft_consensus.cc:515] T 434bff3082d141838068f323bdb0d1ed P cb82cc7bd6db46d980c75237a1402860 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:08.156806 18843 consensus_queue.cc:1048] T 2794af5b99984efebdee564035841cb0 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:20:08.159204 19004 raft_consensus.cc:3060] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:20:08.163383 16292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8638ee363273426db1393dd81e5c7b72" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:20:08.163764 19003 leader_election.cc:290] T 434bff3082d141838068f323bdb0d1ed P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:08.165083 16790 leader_election.cc:304] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860; no voters: 90f3da57796b488c9087a2f523adf045
I20260709 13:20:08.165910 19021 raft_consensus.cc:515] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:08.171428 19050 raft_consensus.cc:2749] T 31969392ccdb43b491911a319e8bc7e1 P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:20:08.170253 16789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 49580.8us
W20260709 13:20:08.150329 16266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 68809.2us
I20260709 13:20:08.159091 18843 raft_consensus.cc:2804] T 26354361e9b044cea1c2b86c4882fe92 P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:20:08.192703 19042 raft_consensus.cc:2749] T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:20:08.198307 16821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada5a004ba7c46819051e3ccec71ffdb" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:08.195564 16146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02838c86531d4f98bf58d1c22b8baf77" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:20:07.988507 18551 raft_consensus.cc:3060] T c0eab25ce4c64e018a041beeda2867db P 1ca2725d290e47d68b1a4b609c9816ac [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:20:08.200233 16300 raft_consensus.cc:2468] T ecc1e363230043fe9fef390f0b979f20 P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 6.
I20260709 13:20:08.174772 18952 leader_election.cc:290] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 6 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:08.197683 16789 leader_election.cc:304] T 9e65676b21b748728ad39668cfe722ec P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
W20260709 13:20:08.204989 16117 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 117556us
I20260709 13:20:08.210228 16822 raft_consensus.cc:2393] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 6: Already voted for candidate cb82cc7bd6db46d980c75237a1402860 in this term.
I20260709 13:20:08.217752 19021 leader_election.cc:290] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 6 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:20:08.220163 16148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af57996a8be84d398f0f023e8d08ef77" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 7 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:08.221698 16144 raft_consensus.cc:1275] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 7 index: 6. (term mismatch)
I20260709 13:20:08.222244 16144 pending_rounds.cc:85] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b: Aborting all ops after (but not including) 5
I20260709 13:20:08.222537 16150 raft_consensus.cc:1275] T 18ad7e3af5504571a69d10c00750a01c P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 13:20:08.225473 19040 raft_consensus.cc:493] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:08.225792 16148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a959dbecc2648eb85d1cc550f5a3245" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:08.226399 19040 raft_consensus.cc:515] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:08.227754 16821 raft_consensus.cc:2393] T ada5a004ba7c46819051e3ccec71ffdb P cb82cc7bd6db46d980c75237a1402860 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 7: Already voted for candidate cb82cc7bd6db46d980c75237a1402860 in this term.
I20260709 13:20:08.227651 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62fe3346cd3c42c4a55e14af9073bca4" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 9 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:08.229684 16151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8158bbd17b994d2eb96f560bd0f1b88d" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:20:08.232683 16151 raft_consensus.cc:1275] T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:20:08.235752 16817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26354361e9b044cea1c2b86c4882fe92" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:20:08.237756 16790 leader_election.cc:304] T 3a7ca61592e848ea9b0fb81a2ce33cdc P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 90f3da57796b488c9087a2f523adf045
I20260709 13:20:08.239110 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fffab26427e24d398e2695aad40ca134" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:08.240162 16225 raft_consensus.cc:2468] T fffab26427e24d398e2695aad40ca134 P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 5.
I20260709 13:20:08.240221 18992 consensus_queue.cc:1048] T af57996a8be84d398f0f023e8d08ef77 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:20:08.240931 18114 raft_consensus.cc:493] T d2ab0ac8347641e6955e9bce7d90c1a2 P 1ca2725d290e47d68b1a4b609c9816ac [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:08.242714 18992 raft_consensus.cc:2764] T 3a7ca61592e848ea9b0fb81a2ce33cdc P cb82cc7bd6db46d980c75237a1402860 [term 9 FOLLOWER]: Leader pre-election decision vote started in defunct term 7: won
I20260709 13:20:08.246037 18758 leader_election.cc:290] T ee9ed67006144797b1de6ea8c2013a68 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:08.010113 18996 raft_consensus.cc:697] T 8a878999c0024d2abe517484180290fc P 90f3da57796b488c9087a2f523adf045 [term 6 LEADER]: Becoming Leader. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:20:08.248775 18996 consensus_queue.cc:237] T 8a878999c0024d2abe517484180290fc P 90f3da57796b488c9087a2f523adf045 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:08.251555 16296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fffab26427e24d398e2695aad40ca134" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:08.252825 16296 raft_consensus.cc:2410] T fffab26427e24d398e2695aad40ca134 P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac for term 6 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
W20260709 13:20:08.254343 16263 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:08.254819 18896 raft_consensus.cc:3055] T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:20:08.254447 18881 consensus_queue.cc:260] T 68a6838d82ef4cac83507f906139d90f P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:08.193487 16265 leader_election.cc:304] T 3884f864a96d428785895e5463e5728f P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
I20260709 13:20:08.255368 18896 raft_consensus.cc:740] T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:20:08.244119 16146 raft_consensus.cc:2410] T 02838c86531d4f98bf58d1c22b8baf77 P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac for term 7 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 13:20:08.255928 18881 raft_consensus.cc:3060] T 68a6838d82ef4cac83507f906139d90f P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:20:08.256742 18896 consensus_queue.cc:260] T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 6.3, Last appended by leader: 3, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:08.257779 16150 raft_consensus.cc:1275] T c0eab25ce4c64e018a041beeda2867db P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:20:08.258514 18896 raft_consensus.cc:3060] T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:20:08.257706 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 6 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:08.260474 16298 raft_consensus.cc:2393] T ecc1e363230043fe9fef390f0b979f20 P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 6: Already voted for candidate cb82cc7bd6db46d980c75237a1402860 in this term.
W20260709 13:20:08.268414 16789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 70886.6us
I20260709 13:20:08.273069 18999 raft_consensus.cc:2749] T 9e65676b21b748728ad39668cfe722ec P cb82cc7bd6db46d980c75237a1402860 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:20:08.274358 18999 consensus_queue.cc:1059] T 26354361e9b044cea1c2b86c4882fe92 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:20:08.284006 18999 raft_consensus.cc:3055] T 26354361e9b044cea1c2b86c4882fe92 P cb82cc7bd6db46d980c75237a1402860 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:20:08.284667 18999 raft_consensus.cc:740] T 26354361e9b044cea1c2b86c4882fe92 P cb82cc7bd6db46d980c75237a1402860 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:20:08.276386 19033 raft_consensus.cc:2804] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:20:08.285868 19033 raft_consensus.cc:493] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:08.286399 19033 raft_consensus.cc:3060] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:20:08.278651 19012 log.cc:927] Time spent T 66f75124215f4ca6bef831e174a792d8 P cb82cc7bd6db46d980c75237a1402860: Append to log took a long time: real 1.362s	user 0.010s	sys 0.020s
I20260709 13:20:08.222553 16144 pending_rounds.cc:107] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b: Aborting uncommitted NO_OP operation due to leader change: 6.6
I20260709 13:20:08.293574 16144 raft_consensus.cc:2889] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:20:08.298177 18946 raft_consensus.cc:3055] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 647a343a542d4a139fe106c19095129b [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:20:08.270713 16789 leader_election.cc:304] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 
W20260709 13:20:08.302629 16266 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:08.303560 15046 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 6
W20260709 13:20:08.219007 16190 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:08.229687 19040 leader_election.cc:290] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:08.307350 16144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806d7690c2b644959152074bb2cab0a7" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:08.309602 16220 raft_consensus.cc:2468] T ecc1e363230043fe9fef390f0b979f20 P 1ca2725d290e47d68b1a4b609c9816ac [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 6.
I20260709 13:20:08.314297 16222 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57604 (request call id 2163) took 1307 ms. Trace:
I20260709 13:20:08.316350 18952 raft_consensus.cc:493] T 1df93779209748b9860f6d893dd3de53 P 1ca2725d290e47d68b1a4b609c9816ac [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:08.317193 18952 raft_consensus.cc:515] T 1df93779209748b9860f6d893dd3de53 P 1ca2725d290e47d68b1a4b609c9816ac [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:08.317734 18929 raft_consensus.cc:493] T 84f2da98864b4425bf7c0055951fdd4b P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:08.318707 18929 raft_consensus.cc:515] T 84f2da98864b4425bf7c0055951fdd4b P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:20:08.321283 16264 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:08.322117 16222 rpcz_store.cc:276] 0709 13:20:07.006700 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:07.196371 (+189671us) service_pool.cc:224] Handling call
0709 13:20:07.208256 (+ 11885us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:20:07.208772 (+   516us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:07.208813 (+    41us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:20:07.239419 (+ 30606us) log.cc:844] Serialized 52 byte log entry
0709 13:20:07.899672 (+660253us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:20:08.311075 (+411403us) log.cc:844] Serialized 14 byte log entry
0709 13:20:08.311600 (+   525us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:08.311762 (+   162us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:20:08.311851 (+    89us) raft_consensus.cc:1685] finished
0709 13:20:08.311873 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:08.314061 (+  2188us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":8364,"raft.run_cpu_time_us":228,"raft.run_wall_time_us":229,"thread_start_us":826229,"threads_started":2,"wal-append.queue_time_us":611490}
I20260709 13:20:08.298789 18946 raft_consensus.cc:740] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 647a343a542d4a139fe106c19095129b [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:20:08.336004 19073 raft_consensus.cc:2804] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P cb82cc7bd6db46d980c75237a1402860 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:20:08.336416 16291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d16dfad0c3b540e5b4b86492ddf87208" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:20:08.337581 16291 raft_consensus.cc:3060] T d16dfad0c3b540e5b4b86492ddf87208 P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:20:08.338311 19021 raft_consensus.cc:493] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:20:08.336669 19073 raft_consensus.cc:493] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P cb82cc7bd6db46d980c75237a1402860 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:20:08.341066 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 66011.4us
I20260709 13:20:08.339082 19021 raft_consensus.cc:515] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:08.346175 16151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a878999c0024d2abe517484180290fc" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:20:08.344058 19073 raft_consensus.cc:3060] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P cb82cc7bd6db46d980c75237a1402860 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:20:08.342082 16817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5718c623ebee4d2bb770f0f9ddb2f164" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:08.264778 19004 raft_consensus.cc:515] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:08.349417 16790 leader_election.cc:304] T 806d7690c2b644959152074bb2cab0a7 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:20:08.287809 18999 consensus_queue.cc:260] T 26354361e9b044cea1c2b86c4882fe92 P cb82cc7bd6db46d980c75237a1402860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:08.350538 16823 raft_consensus.cc:1275] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [term 8 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
W20260709 13:20:08.352381 16117 leader_election.cc:336] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 8 election: RPC error from VoteRequest() call to peer cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923): Timed out: RequestConsensusVote RPC to 127.14.177.132:38923 timed out after 1.799s (SENT)
I20260709 13:20:08.321322 18929 leader_election.cc:290] T 84f2da98864b4425bf7c0055951fdd4b P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:08.367027 16817 raft_consensus.cc:2376] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 [term 6 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 647a343a542d4a139fe106c19095129b in term 6. Re-sending same reply.
W20260709 13:20:08.377672 16266 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:08.352527 18946 consensus_queue.cc:260] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:08.399273 16151 raft_consensus.cc:2393] T 8a878999c0024d2abe517484180290fc P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 6: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
I20260709 13:20:08.380020 19003 raft_consensus.cc:493] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:08.400966 16142 raft_consensus.cc:1240] T 8d4ab870a05347868956aa8b841b2981 P 647a343a542d4a139fe106c19095129b [term 7 LEADER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 6. Current term is 7. Ops: []
I20260709 13:20:08.399596 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ad5dbfa99ee4b8d9e28be92b3f36db5" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:20:08.370127 18114 raft_consensus.cc:515] T d2ab0ac8347641e6955e9bce7d90c1a2 P 1ca2725d290e47d68b1a4b609c9816ac [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:08.408414 16144 raft_consensus.cc:1275] T 68a6838d82ef4cac83507f906139d90f P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 5. (term mismatch)
I20260709 13:20:08.393576 16190 leader_election.cc:304] T ecc1e363230043fe9fef390f0b979f20 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860
I20260709 13:20:08.419450 16151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d16dfad0c3b540e5b4b86492ddf87208" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:08.421150 18114 leader_election.cc:290] T d2ab0ac8347641e6955e9bce7d90c1a2 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:08.399904 16817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5d59407eb545b895d8d922e0dd69fd" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:08.427975 16817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e4bfb9090c8460eb14fa7e0d818e7f1" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
W20260709 13:20:08.431926 16825 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49906 (request call id 615) took 1770 ms (client timeout 1934 ms). Trace:
I20260709 13:20:08.395926 18952 leader_election.cc:290] T 1df93779209748b9860f6d893dd3de53 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:08.415356 16144 pending_rounds.cc:85] T 68a6838d82ef4cac83507f906139d90f P 647a343a542d4a139fe106c19095129b: Aborting all ops after (but not including) 4
W20260709 13:20:08.435475 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 64680.9us
I20260709 13:20:08.464668 16144 pending_rounds.cc:107] T 68a6838d82ef4cac83507f906139d90f P 647a343a542d4a139fe106c19095129b: Aborting uncommitted NO_OP operation due to leader change: 7.5
I20260709 13:20:08.466410 16144 raft_consensus.cc:2889] T 68a6838d82ef4cac83507f906139d90f P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:20:08.443800 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434bff3082d141838068f323bdb0d1ed" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:08.442495 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806d7690c2b644959152074bb2cab0a7" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:20:08.443693 16823 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49896 (request call id 915) took 1241 ms. Trace:
I20260709 13:20:08.448673 16821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c11074579b046d99e585c2818e18a2c" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:08.469357 16291 raft_consensus.cc:2468] T d16dfad0c3b540e5b4b86492ddf87208 P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 6.
W20260709 13:20:08.432478 16825 rpcz_store.cc:269] 0709 13:20:06.661374 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:06.661483 (+   109us) service_pool.cc:224] Handling call
0709 13:20:08.431509 (+1770026us) spinlock_profiling.cc:243] Waited 1.47 s on lock 0x7b540054bad4. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:20:08.431698 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3378672384}
I20260709 13:20:08.435053 16818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5718c623ebee4d2bb770f0f9ddb2f164" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:08.468664 16823 rpcz_store.cc:276] 0709 13:20:07.202496 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:07.324093 (+121597us) service_pool.cc:224] Handling call
0709 13:20:07.324429 (+   336us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:08.356898 (+1032469us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:08.443297 (+ 86399us) spinlock_profiling.cc:243] Waited 1.03 s on lock 0x7b54006423d4. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:20:08.443604 (+   307us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2358406528}
W20260709 13:20:08.473999 16264 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:08.436512 16816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af57996a8be84d398f0f023e8d08ef77" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:20:08.397820 18843 raft_consensus.cc:697] T 26354361e9b044cea1c2b86c4882fe92 P 90f3da57796b488c9087a2f523adf045 [term 5 LEADER]: Becoming Leader. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
W20260709 13:20:08.481752 16264 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:20:08.454943 19014 log.cc:927] Time spent T 8c11074579b046d99e585c2818e18a2c P cb82cc7bd6db46d980c75237a1402860: Append to log took a long time: real 1.455s	user 0.010s	sys 0.021s
W20260709 13:20:08.450538 19019 log.cc:927] Time spent T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 1.292s	user 0.005s	sys 0.015s
I20260709 13:20:08.435885 18551 raft_consensus.cc:2749] T ecc1e363230043fe9fef390f0b979f20 P 1ca2725d290e47d68b1a4b609c9816ac [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:20:08.446709 18936 raft_consensus.cc:3060] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:20:08.500206 16266 leader_election.cc:336] T 8c11074579b046d99e585c2818e18a2c P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923): Timed out: RequestConsensusVote RPC to 127.14.177.132:38923 timed out after 2.183s (SENT)
I20260709 13:20:08.501271 16190 leader_election.cc:304] T d16dfad0c3b540e5b4b86492ddf87208 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:20:08.450922 19021 leader_election.cc:290] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:20:08.495325 16299 raft_consensus.cc:3060] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:20:08.503799 18551 raft_consensus.cc:2804] T d16dfad0c3b540e5b4b86492ddf87208 P 1ca2725d290e47d68b1a4b609c9816ac [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:20:08.504371 18551 raft_consensus.cc:697] T d16dfad0c3b540e5b4b86492ddf87208 P 1ca2725d290e47d68b1a4b609c9816ac [term 6 LEADER]: Becoming Leader. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
W20260709 13:20:08.505380 16191 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:08.505695 18551 consensus_queue.cc:237] T d16dfad0c3b540e5b4b86492ddf87208 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:08.456642 19048 raft_consensus.cc:3060] T 9e65676b21b748728ad39668cfe722ec P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:20:08.507948 19003 raft_consensus.cc:515] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:08.510344 19003 leader_election.cc:290] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
W20260709 13:20:08.511526 16117 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 159184us
I20260709 13:20:08.511723 16823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af57996a8be84d398f0f023e8d08ef77" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
W20260709 13:20:08.512372 16117 leader_election.cc:336] T 26354361e9b044cea1c2b86c4882fe92 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923): Timed out: RequestConsensusVote RPC to 127.14.177.132:38923 timed out after 1.807s (SENT)
I20260709 13:20:08.513099 16823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdbd78acb4cf46298e0de8ba0d130e19" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:08.513888 16823 raft_consensus.cc:2393] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 7: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
I20260709 13:20:08.515277 19008 raft_consensus.cc:515] T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:08.516532 16221 raft_consensus.cc:1217] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 1ca2725d290e47d68b1a4b609c9816ac [term 7 LEADER]: Deduplicated request from leader. Original: 3.3->[4.4-6.5]   Dedup: 6.5->[]
I20260709 13:20:08.517282 16221 raft_consensus.cc:1240] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 1ca2725d290e47d68b1a4b609c9816ac [term 7 LEADER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 6. Current term is 7. Ops: [4.4-6.5]
I20260709 13:20:08.517975 19008 leader_election.cc:290] T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 8 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
W20260709 13:20:08.519156 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 169852us
I20260709 13:20:08.520102 16790 leader_election.cc:304] T d2ab0ac8347641e6955e9bce7d90c1a2 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:20:08.521553 16790 leader_election.cc:304] T ada5a004ba7c46819051e3ccec71ffdb P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
W20260709 13:20:08.522239 16116 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:20:08.523654 16116 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:08.523730 18946 raft_consensus.cc:3060] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:20:08.525421 16820 raft_consensus.cc:2368] T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b for earlier term 7. Current term is 8.
I20260709 13:20:08.526695 16820 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49896 (request call id 919) took 1186 ms. Trace:
I20260709 13:20:08.527406 18999 raft_consensus.cc:3060] T 26354361e9b044cea1c2b86c4882fe92 P cb82cc7bd6db46d980c75237a1402860 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:20:08.528357 16822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434bff3082d141838068f323bdb0d1ed" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:08.529121 16822 raft_consensus.cc:2393] T 434bff3082d141838068f323bdb0d1ed P cb82cc7bd6db46d980c75237a1402860 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 6: Already voted for candidate cb82cc7bd6db46d980c75237a1402860 in this term.
I20260709 13:20:08.529322 16821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ab0ac8347641e6955e9bce7d90c1a2" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:08.529947 16821 raft_consensus.cc:2468] T d2ab0ac8347641e6955e9bce7d90c1a2 P cb82cc7bd6db46d980c75237a1402860 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 6.
I20260709 13:20:08.531144 16821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 6 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:20:08.531970 16821 raft_consensus.cc:2393] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 6: Already voted for candidate cb82cc7bd6db46d980c75237a1402860 in this term.
I20260709 13:20:08.534690 16819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e9b2bf7011f4ae688a79d7bb5a447b4" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:08.534495 18843 consensus_queue.cc:237] T 26354361e9b044cea1c2b86c4882fe92 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:08.440330 18857 raft_consensus.cc:3060] T e5ad4eb281564936a59f41e387a47156 P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:20:08.493436 16293 raft_consensus.cc:2468] T 434bff3082d141838068f323bdb0d1ed P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 6.
I20260709 13:20:08.498874 16816 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49896 (request call id 923) took 1014 ms. Trace:
I20260709 13:20:08.496129 16825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3884f864a96d428785895e5463e5728f" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:08.444157 19002 raft_consensus.cc:2749] T 806d7690c2b644959152074bb2cab0a7 P cb82cc7bd6db46d980c75237a1402860 [term 5 LEARNER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:20:08.529898 16822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8638ee363273426db1393dd81e5c7b72" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:08.538865 16816 rpcz_store.cc:276] 0709 13:20:07.484422 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:07.486420 (+  1998us) service_pool.cc:224] Handling call
0709 13:20:08.436214 (+949794us) spinlock_profiling.cc:243] Waited 241 ms on lock 0x7b5800004298. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69ad8fadb1 00007f69ada3ab71 00007f69ada3aa22 00007f69b6953c09 00007f69b5aadee0 00007f69b5b770ab 00007f69b5b77131 00007f69b5b250a0 00007f69b5b24984 00007f69af09f6d4 00007f69af09f662
0709 13:20:08.498789 (+ 62575us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":554265856}
I20260709 13:20:08.514658 16823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f82224f4564fea93572e8206bad1fe" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:08.549561 16818 raft_consensus.cc:2393] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 6: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
I20260709 13:20:08.527011 16820 rpcz_store.cc:276] 0709 13:20:07.340166 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:07.341785 (+  1619us) service_pool.cc:224] Handling call
0709 13:20:08.526437 (+1184652us) spinlock_profiling.cc:243] Waited 679 ms on lock 0x7b54005320d4. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:20:08.526646 (+   209us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1561832704}
W20260709 13:20:08.552495 16117 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:08.555244 19004 leader_election.cc:290] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 6 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:08.557754 18892 consensus_queue.cc:1048] T 1df93779209748b9860f6d893dd3de53 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:20:08.559628 16148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fffab26427e24d398e2695aad40ca134" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:08.559693 16142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee9ed67006144797b1de6ea8c2013a68" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:08.560700 16151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5ad4eb281564936a59f41e387a47156" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 8 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:08.561535 16142 raft_consensus.cc:2468] T ee9ed67006144797b1de6ea8c2013a68 P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 5.
I20260709 13:20:08.561748 18599 consensus_queue.cc:1048] T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:20:08.562479 16143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 6 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:08.563580 16143 raft_consensus.cc:2393] T ecc1e363230043fe9fef390f0b979f20 P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 6: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
I20260709 13:20:08.564340 18817 consensus_queue.cc:1048] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:20:08.565842 16144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c61a39fd4f84424b129c1a6f9ba404d" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 8 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:08.566077 19033 raft_consensus.cc:515] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:08.567086 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5718c623ebee4d2bb770f0f9ddb2f164" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:08.567389 16819 raft_consensus.cc:2393] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 5: Already voted for candidate cb82cc7bd6db46d980c75237a1402860 in this term.
I20260709 13:20:08.568005 16150 raft_consensus.cc:2393] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 6: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
I20260709 13:20:08.568814 16819 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49896 (request call id 921) took 1227 ms. Trace:
I20260709 13:20:08.569386 16819 rpcz_store.cc:276] 0709 13:20:07.340932 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:07.372970 (+ 32038us) service_pool.cc:224] Handling call
0709 13:20:08.534316 (+1161346us) spinlock_profiling.cc:243] Waited 895 us on lock 0x7b5800004298. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69ad8fadb1 00007f69ada3ab71 00007f69ada3aa22 00007f69b6953c09 00007f69b5aadee0 00007f69b5b770ab 00007f69b5b77131 00007f69b5b250a0 00007f69b5b24984 00007f69af09f6d4 00007f69af09f662
0709 13:20:08.568564 (+ 34248us) spinlock_profiling.cc:243] Waited 31.7 ms on lock 0x7b5400771354. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:20:08.568739 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":74892800}
I20260709 13:20:08.571911 16818 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49906 (request call id 620) took 1196 ms. Trace:
I20260709 13:20:08.574183 16818 rpcz_store.cc:276] 0709 13:20:07.375771 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:07.391321 (+ 15550us) service_pool.cc:224] Handling call
0709 13:20:08.434626 (+1043305us) spinlock_profiling.cc:243] Waited 226 ms on lock 0x7b5800004298. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69ad8fadb1 00007f69ada3ab71 00007f69ada3aa22 00007f69b6953c09 00007f69b5aadee0 00007f69b5b770ab 00007f69b5b77131 00007f69b5b250a0 00007f69b5b24984 00007f69af09f6d4 00007f69af09f662
0709 13:20:08.571841 (+137215us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":519396608}
I20260709 13:20:08.577458 16822 raft_consensus.cc:2393] T 8638ee363273426db1393dd81e5c7b72 P cb82cc7bd6db46d980c75237a1402860 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 6: Already voted for candidate cb82cc7bd6db46d980c75237a1402860 in this term.
I20260709 13:20:08.546868 19008 raft_consensus.cc:2749] T d2ab0ac8347641e6955e9bce7d90c1a2 P cb82cc7bd6db46d980c75237a1402860 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:20:08.551838 19003 raft_consensus.cc:2749] T ada5a004ba7c46819051e3ccec71ffdb P cb82cc7bd6db46d980c75237a1402860 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:20:08.592511 16819 raft_consensus.cc:1240] T 8638ee363273426db1393dd81e5c7b72 P cb82cc7bd6db46d980c75237a1402860 [term 6 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 5. Current term is 6. Ops: [5.7-5.7]
I20260709 13:20:08.602017 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f2da98864b4425bf7c0055951fdd4b" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
W20260709 13:20:08.606081 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 83202.6us
I20260709 13:20:08.586740 16299 raft_consensus.cc:2468] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 6.
I20260709 13:20:08.605155 16825 raft_consensus.cc:2468] T 3884f864a96d428785895e5463e5728f P cb82cc7bd6db46d980c75237a1402860 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 5.
I20260709 13:20:08.621825 16820 raft_consensus.cc:1240] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860 [term 6 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 5. Current term is 6. Ops: []
W20260709 13:20:08.623294 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 117097us
I20260709 13:20:08.626842 16291 raft_consensus.cc:1275] T 23f82224f4564fea93572e8206bad1fe P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
I20260709 13:20:08.631978 16190 leader_election.cc:304] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:20:08.632445 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ab0ac8347641e6955e9bce7d90c1a2" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:08.630131 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806d7690c2b644959152074bb2cab0a7" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:20:08.630033 16293 raft_consensus.cc:2468] T 84f2da98864b4425bf7c0055951fdd4b P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 3.
I20260709 13:20:08.637281 16299 raft_consensus.cc:2468] T d2ab0ac8347641e6955e9bce7d90c1a2 P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 5.
I20260709 13:20:08.639434 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1df93779209748b9860f6d893dd3de53" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:08.639331 18720 consensus_queue.cc:1059] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.019s
I20260709 13:20:08.640527 16293 raft_consensus.cc:2468] T 1df93779209748b9860f6d893dd3de53 P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 6.
I20260709 13:20:08.641114 18720 raft_consensus.cc:3055] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:20:08.641628 18720 raft_consensus.cc:740] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
W20260709 13:20:08.643280 18855 consensus_meta.cc:287] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860: Time spent flushing consensus metadata: real 1.380s	user 0.006s	sys 0.014s
I20260709 13:20:08.643723 18855 raft_consensus.cc:515] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:08.646179 18936 raft_consensus.cc:2804] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:20:08.646701 18936 raft_consensus.cc:697] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac [term 6 LEADER]: Becoming Leader. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:20:08.643294 16145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d16dfad0c3b540e5b4b86492ddf87208" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:20:08.647873 18936 consensus_queue.cc:237] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.5, Last appended by leader: 5, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } }
I20260709 13:20:08.650331 19065 raft_consensus.cc:2749] T b87147731a68400e8cce9b879daa7c5d P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:20:08.651317 16192 leader_election.cc:304] T d2ab0ac8347641e6955e9bce7d90c1a2 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:20:08.651506 16291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdbd78acb4cf46298e0de8ba0d130e19" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
W20260709 13:20:08.651966 16192 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 100408us
I20260709 13:20:08.652403 16291 raft_consensus.cc:2410] T cdbd78acb4cf46298e0de8ba0d130e19 P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 for term 8 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 13:20:08.618366 19050 raft_consensus.cc:2749] T 3884f864a96d428785895e5463e5728f P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:20:08.654238 18770 raft_consensus.cc:2804] T d2ab0ac8347641e6955e9bce7d90c1a2 P 1ca2725d290e47d68b1a4b609c9816ac [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:20:08.654691 18770 raft_consensus.cc:493] T d2ab0ac8347641e6955e9bce7d90c1a2 P 1ca2725d290e47d68b1a4b609c9816ac [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:08.634898 16820 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49906 (request call id 622) took 1237 ms. Trace:
I20260709 13:20:08.656021 16820 rpcz_store.cc:276] 0709 13:20:07.390219 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:08.573632 (+1183413us) service_pool.cc:224] Handling call
0709 13:20:08.605495 (+ 31863us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:08.627840 (+ 22345us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:08.628054 (+   214us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:20:08.655081 18770 raft_consensus.cc:3060] T d2ab0ac8347641e6955e9bce7d90c1a2 P 1ca2725d290e47d68b1a4b609c9816ac [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:20:08.600651 19043 consensus_queue.cc:1048] T 5a959dbecc2648eb85d1cc550f5a3245 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 1, Time since last communication: 0.048s
W20260709 13:20:08.671111 16117 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:08.669512 16789 leader_election.cc:304] T 434bff3082d141838068f323bdb0d1ed P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:20:08.666308 15046 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:20:08.676461 16190 leader_election.cc:304] T 84f2da98864b4425bf7c0055951fdd4b P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:20:08.679454 16819 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49906 (request call id 619) took 1302 ms. Trace:
I20260709 13:20:08.682669 16819 rpcz_store.cc:276] 0709 13:20:07.371691 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:08.570410 (+1198719us) service_pool.cc:224] Handling call
0709 13:20:08.579372 (+  8962us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:20:08.593299 (+ 13927us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:08.674031 (+ 80732us) spinlock_profiling.cc:243] Waited 8.48 ms on lock 0x7b54007706d0. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5e8ad 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:20:08.674254 (+   223us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":19498880}
I20260709 13:20:08.680619 16145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5d59407eb545b895d8d922e0dd69fd" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:08.686652 19003 raft_consensus.cc:2804] T 434bff3082d141838068f323bdb0d1ed P cb82cc7bd6db46d980c75237a1402860 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:20:08.687196 19003 raft_consensus.cc:493] T 434bff3082d141838068f323bdb0d1ed P cb82cc7bd6db46d980c75237a1402860 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:08.687330 16824 raft_consensus.cc:2368] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b for earlier term 7. Current term is 8.
I20260709 13:20:08.687609 19003 raft_consensus.cc:3060] T 434bff3082d141838068f323bdb0d1ed P cb82cc7bd6db46d980c75237a1402860 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:20:08.688215 16293 raft_consensus.cc:1240] T 66ae176961b74813a58bc15c9c4c6f2a P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 8. Current term is 9. Ops: []
I20260709 13:20:08.682719 18551 raft_consensus.cc:493] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:08.666666 18855 leader_election.cc:290] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 8 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:08.701808 18939 raft_consensus.cc:493] T c2f5f89186204242bf377f7c42b2e2da P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:08.702004 16145 raft_consensus.cc:2468] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 7.
I20260709 13:20:08.702210 18952 raft_consensus.cc:2804] T 84f2da98864b4425bf7c0055951fdd4b P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:20:08.702632 18952 raft_consensus.cc:493] T 84f2da98864b4425bf7c0055951fdd4b P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:08.701838 16818 raft_consensus.cc:1275] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 [term 7 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:20:08.667410 18892 raft_consensus.cc:493] T 84f2da98864b4425bf7c0055951fdd4b P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:08.707410 16190 leader_election.cc:304] T 1df93779209748b9860f6d893dd3de53 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
W20260709 13:20:08.719301 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 570712us
I20260709 13:20:08.719573 19033 leader_election.cc:290] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 5 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:08.702562 18939 raft_consensus.cc:515] T c2f5f89186204242bf377f7c42b2e2da P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:08.720800 16818 raft_consensus.cc:1240] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860 [term 5 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 13:20:08.722335 18939 leader_election.cc:290] T c2f5f89186204242bf377f7c42b2e2da P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:08.723156 18892 raft_consensus.cc:515] T 84f2da98864b4425bf7c0055951fdd4b P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:08.725694 18892 leader_election.cc:290] T 84f2da98864b4425bf7c0055951fdd4b P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
W20260709 13:20:08.726292 16266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 44842.4us
I20260709 13:20:08.703063 18952 raft_consensus.cc:3060] T 84f2da98864b4425bf7c0055951fdd4b P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:20:08.727591 19010 raft_consensus.cc:493] T 8cc959fb7f704fba9009c0595467d074 P 1ca2725d290e47d68b1a4b609c9816ac [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:20:08.696350 18551 raft_consensus.cc:515] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:08.737171 18551 leader_election.cc:290] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:08.643023 18720 consensus_queue.cc:260] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:08.738370 18720 raft_consensus.cc:3060] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:20:08.740967 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 103970us
I20260709 13:20:08.741850 16790 leader_election.cc:304] T 829b4ad8c5a54eababeea1181b960d4b P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:20:08.673010 19073 raft_consensus.cc:515] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P cb82cc7bd6db46d980c75237a1402860 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:08.753088 16820 raft_consensus.cc:3055] T af57996a8be84d398f0f023e8d08ef77 P cb82cc7bd6db46d980c75237a1402860 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:20:08.739149 16116 leader_election.cc:304] T b6f07842d3aa47858df5665874c86f3a P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
I20260709 13:20:08.728186 18935 consensus_queue.cc:1059] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:20:08.759667 19043 raft_consensus.cc:2804] T b6f07842d3aa47858df5665874c86f3a P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:20:08.760286 19043 raft_consensus.cc:493] T b6f07842d3aa47858df5665874c86f3a P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:08.760637 19043 raft_consensus.cc:3060] T b6f07842d3aa47858df5665874c86f3a P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:20:08.763816 18900 consensus_queue.cc:1048] T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:20:08.766027 18900 consensus_queue.cc:1048] T af57996a8be84d398f0f023e8d08ef77 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:20:08.797710 16219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f2da98864b4425bf7c0055951fdd4b" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
W20260709 13:20:08.800155 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 78960.6us
W20260709 13:20:08.800608 16266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 54685.2us
I20260709 13:20:08.756323 16820 raft_consensus.cc:740] T af57996a8be84d398f0f023e8d08ef77 P cb82cc7bd6db46d980c75237a1402860 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
W20260709 13:20:08.805552 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 51029.2us
I20260709 13:20:08.754761 18758 raft_consensus.cc:2804] T 1df93779209748b9860f6d893dd3de53 P 1ca2725d290e47d68b1a4b609c9816ac [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 13:20:08.831795 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 90024.5us
I20260709 13:20:08.805660 19008 raft_consensus.cc:2749] T 829b4ad8c5a54eababeea1181b960d4b P cb82cc7bd6db46d980c75237a1402860 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:20:08.728205 19010 raft_consensus.cc:515] T 8cc959fb7f704fba9009c0595467d074 P 1ca2725d290e47d68b1a4b609c9816ac [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:08.842226 16265 leader_election.cc:304] T 23f82224f4564fea93572e8206bad1fe P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: cb82cc7bd6db46d980c75237a1402860
W20260709 13:20:08.840976 16191 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.14.177.129:36741 timed out after 1.592s (SENT) [suppressed 70 similar messages]
W20260709 13:20:08.863828 16266 leader_election.cc:336] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923): Timed out: RequestConsensusVote RPC to 127.14.177.132:38923 timed out after 2.248s (SENT)
I20260709 13:20:08.835470 16820 consensus_queue.cc:260] T af57996a8be84d398f0f023e8d08ef77 P cb82cc7bd6db46d980c75237a1402860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:08.831174 18758 raft_consensus.cc:493] T 1df93779209748b9860f6d893dd3de53 P 1ca2725d290e47d68b1a4b609c9816ac [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:08.833740 19042 raft_consensus.cc:493] T 9e65676b21b748728ad39668cfe722ec P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:20:08.876467 16820 raft_consensus.cc:3060] T af57996a8be84d398f0f023e8d08ef77 P cb82cc7bd6db46d980c75237a1402860 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:20:08.870545 16266 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:08.743533 18944 consensus_queue.cc:1048] T fffab26427e24d398e2695aad40ca134 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.121s
I20260709 13:20:08.866045 16817 raft_consensus.cc:2368] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P cb82cc7bd6db46d980c75237a1402860 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 for earlier term 5. Current term is 6.
W20260709 13:20:08.891080 19054 log.cc:927] Time spent T 18ad7e3af5504571a69d10c00750a01c P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.073s	user 0.000s	sys 0.000s
W20260709 13:20:08.891439 16114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:20:08.892354 19083 log.cc:927] Time spent T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.180s	user 0.000s	sys 0.000s
I20260709 13:20:08.896798 19062 consensus_queue.cc:1059] T 746aabf856ab401c9ef5e859371e7b67 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.186s
I20260709 13:20:08.898452 18114 raft_consensus.cc:493] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:08.901671 19088 consensus_queue.cc:1048] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.076s
W20260709 13:20:08.913785 19085 log.cc:927] Time spent T 8a878999c0024d2abe517484180290fc P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 13:20:08.863843 18939 raft_consensus.cc:493] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:08.883517 18758 raft_consensus.cc:3060] T 1df93779209748b9860f6d893dd3de53 P 1ca2725d290e47d68b1a4b609c9816ac [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:20:08.918577 19089 raft_consensus.cc:2804] T 23f82224f4564fea93572e8206bad1fe P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 13:20:08.926733 16264 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:20:08.973150 16264 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:08.925302 16151 raft_consensus.cc:2410] T e5ad4eb281564936a59f41e387a47156 P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac for term 8 because replica has last-logged OpId of term: 6 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 4.
W20260709 13:20:08.937875 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 54409.6us
I20260709 13:20:08.924769 16143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fffab26427e24d398e2695aad40ca134" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
W20260709 13:20:08.940503 16266 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:08.970484 18114 raft_consensus.cc:515] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:08.922108 16145 raft_consensus.cc:1217] T 5a959dbecc2648eb85d1cc550f5a3245 P 647a343a542d4a139fe106c19095129b [term 5 LEADER]: Deduplicated request from leader. Original: 1.1->[2.2-4.3]   Dedup: 4.3->[]
I20260709 13:20:08.977260 16145 raft_consensus.cc:1240] T 5a959dbecc2648eb85d1cc550f5a3245 P 647a343a542d4a139fe106c19095129b [term 5 LEADER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 4. Current term is 5. Ops: [2.2-4.3]
I20260709 13:20:08.958194 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0eab25ce4c64e018a041beeda2867db" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:20:08.964535 16144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4f65cdf00b748bb963ee6e81ef43b79" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
W20260709 13:20:08.978736 16114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:20:08.929853 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 45001us
I20260709 13:20:08.917874 19073 leader_election.cc:290] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 6 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:08.840502 19057 consensus_queue.cc:1048] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.212s
W20260709 13:20:08.942888 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 122523us
I20260709 13:20:08.916684 19010 leader_election.cc:290] T 8cc959fb7f704fba9009c0595467d074 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:08.989290 16150 raft_consensus.cc:3060] T c0eab25ce4c64e018a041beeda2867db P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:20:08.967726 18935 raft_consensus.cc:3055] T 746aabf856ab401c9ef5e859371e7b67 P 1ca2725d290e47d68b1a4b609c9816ac [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:20:08.975759 19089 raft_consensus.cc:493] T 23f82224f4564fea93572e8206bad1fe P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:08.976938 18939 raft_consensus.cc:515] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:09.002233 16265 leader_election.cc:304] T 434bff3082d141838068f323bdb0d1ed P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
I20260709 13:20:09.002719 18114 leader_election.cc:290] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:08.995543 18857 raft_consensus.cc:493] T 26354361e9b044cea1c2b86c4882fe92 P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:08.999280 18935 raft_consensus.cc:740] T 746aabf856ab401c9ef5e859371e7b67 P 1ca2725d290e47d68b1a4b609c9816ac [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:20:09.028733 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f5f89186204242bf377f7c42b2e2da" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
W20260709 13:20:09.032307 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 143775us
I20260709 13:20:09.015851 19089 raft_consensus.cc:3060] T 23f82224f4564fea93572e8206bad1fe P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:20:09.033874 16291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cc959fb7f704fba9009c0595467d074" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:09.032701 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9944d5e4b8c4109af9955c4f08c9095" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:09.007268 19042 raft_consensus.cc:515] T 9e65676b21b748728ad39668cfe722ec P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:08.996286 19048 consensus_queue.cc:1059] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.032s
I20260709 13:20:09.018896 18817 raft_consensus.cc:493] T 2794af5b99984efebdee564035841cb0 P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:09.013350 18996 consensus_queue.cc:1048] T 2794af5b99984efebdee564035841cb0 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.033s
I20260709 13:20:08.915132 19003 raft_consensus.cc:515] T 434bff3082d141838068f323bdb0d1ed P cb82cc7bd6db46d980c75237a1402860 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:09.041327 16298 raft_consensus.cc:2468] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 8.
W20260709 13:20:09.059863 16190 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:20:09.286123 19043 consensus_meta.cc:287] T b6f07842d3aa47858df5665874c86f3a P 647a343a542d4a139fe106c19095129b: Time spent flushing consensus metadata: real 0.525s	user 0.015s	sys 0.000s
I20260709 13:20:09.336094 18939 leader_election.cc:290] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:09.613437 19043 raft_consensus.cc:515] T b6f07842d3aa47858df5665874c86f3a P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:20:09.457549 16114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:09.136689 16294 raft_consensus.cc:1275] T 66ae176961b74813a58bc15c9c4c6f2a P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
W20260709 13:20:09.507587 19053 log.cc:927] Time spent T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 0.096s	user 0.000s	sys 0.001s
I20260709 13:20:09.250550 18932 raft_consensus.cc:493] T e5ad4eb281564936a59f41e387a47156 P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:09.616030 19043 leader_election.cc:290] T b6f07842d3aa47858df5665874c86f3a P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 7 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
W20260709 13:20:09.458513 16116 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:20:09.616674 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 614495us
W20260709 13:20:09.050112 16264 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:20:09.327696 16788 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:09.339862 18935 consensus_queue.cc:260] T 746aabf856ab401c9ef5e859371e7b67 P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:20:09.618716 16264 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:09.619217 18935 raft_consensus.cc:3060] T 746aabf856ab401c9ef5e859371e7b67 P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:20:09.619858 19068 raft_consensus.cc:3055] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 90f3da57796b488c9087a2f523adf045 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:20:09.620537 19068 raft_consensus.cc:740] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 90f3da57796b488c9087a2f523adf045 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:20:09.620868 19010 raft_consensus.cc:493] T c0eab25ce4c64e018a041beeda2867db P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:09.621646 19068 consensus_queue.cc:260] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 90f3da57796b488c9087a2f523adf045 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:09.159344 16291 raft_consensus.cc:2468] T 8cc959fb7f704fba9009c0595467d074 P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 6.
I20260709 13:20:09.621428 19010 raft_consensus.cc:515] T c0eab25ce4c64e018a041beeda2867db P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:09.110301 16150 raft_consensus.cc:2468] T c0eab25ce4c64e018a041beeda2867db P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 7.
W20260709 13:20:09.199839 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 98530.8us
W20260709 13:20:09.322180 19100 log.cc:927] Time spent T 26354361e9b044cea1c2b86c4882fe92 P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.096s	user 0.000s	sys 0.001s
I20260709 13:20:09.232618 18944 raft_consensus.cc:493] T 8158bbd17b994d2eb96f560bd0f1b88d P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
W20260709 13:20:09.619269 16264 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:09.090360 16299 raft_consensus.cc:2410] T c9944d5e4b8c4109af9955c4f08c9095 P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac for term 8 because replica has last-logged OpId of term: 6 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 7.
W20260709 13:20:09.602149 18952 consensus_meta.cc:287] T 84f2da98864b4425bf7c0055951fdd4b P 1ca2725d290e47d68b1a4b609c9816ac: Time spent flushing consensus metadata: real 0.874s	user 0.008s	sys 0.006s
I20260709 13:20:09.331955 18745 raft_consensus.cc:493] T 8a878999c0024d2abe517484180290fc P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:09.287918 16823 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49906 (request call id 618) took 1925 ms. Trace:
W20260709 13:20:09.071964 18946 consensus_meta.cc:287] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 647a343a542d4a139fe106c19095129b: Time spent flushing consensus metadata: real 0.548s	user 0.011s	sys 0.002s
W20260709 13:20:09.625293 16820 consensus_meta.cc:287] T af57996a8be84d398f0f023e8d08ef77 P cb82cc7bd6db46d980c75237a1402860: Time spent flushing consensus metadata: real 0.740s	user 0.010s	sys 0.000s
W20260709 13:20:09.627456 16116 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:09.478514 19073 raft_consensus.cc:493] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:09.616752 19043 raft_consensus.cc:493] T 68a6838d82ef4cac83507f906139d90f P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:20:09.672273 16191 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:09.118700 16790 leader_election.cc:304] T c9944d5e4b8c4109af9955c4f08c9095 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 90f3da57796b488c9087a2f523adf045
W20260709 13:20:09.596339 18758 consensus_meta.cc:287] T 1df93779209748b9860f6d893dd3de53 P 1ca2725d290e47d68b1a4b609c9816ac: Time spent flushing consensus metadata: real 0.598s	user 0.009s	sys 0.000s
W20260709 13:20:09.563995 19037 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.935s	user 0.017s	sys 0.009s
W20260709 13:20:09.342279 18720 consensus_meta.cc:287] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b: Time spent flushing consensus metadata: real 0.603s	user 0.006s	sys 0.006s
W20260709 13:20:09.657442 16220 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.731s	user 0.003s	sys 0.023s
W20260709 13:20:09.687217 16220 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.761s	user 0.003s	sys 0.024s
W20260709 13:20:09.660707 18770 consensus_meta.cc:287] T d2ab0ac8347641e6955e9bce7d90c1a2 P 1ca2725d290e47d68b1a4b609c9816ac: Time spent flushing consensus metadata: real 0.986s	user 0.013s	sys 0.004s
I20260709 13:20:09.330161 19091 raft_consensus.cc:2749] T 434bff3082d141838068f323bdb0d1ed P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:20:09.680433 16820 raft_consensus.cc:1275] T af57996a8be84d398f0f023e8d08ef77 P cb82cc7bd6db46d980c75237a1402860 [term 7 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 7. (term mismatch)
I20260709 13:20:09.694227 16820 pending_rounds.cc:85] T af57996a8be84d398f0f023e8d08ef77 P cb82cc7bd6db46d980c75237a1402860: Aborting all ops after (but not including) 6
I20260709 13:20:09.694579 16820 pending_rounds.cc:107] T af57996a8be84d398f0f023e8d08ef77 P cb82cc7bd6db46d980c75237a1402860: Aborting uncommitted NO_OP operation due to leader change: 6.7
I20260709 13:20:09.694871 16820 raft_consensus.cc:2889] T af57996a8be84d398f0f023e8d08ef77 P cb82cc7bd6db46d980c75237a1402860 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:20:09.660259 19040 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.593s	user 0.000s	sys 0.013s
I20260709 13:20:09.616485 18932 raft_consensus.cc:515] T e5ad4eb281564936a59f41e387a47156 P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:20:09.679044 19037 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.158s	user 0.022s	sys 0.011s
I20260709 13:20:09.703024 16820 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49896 (request call id 935) took 1777 ms. Trace:
I20260709 13:20:09.031518 18857 raft_consensus.cc:515] T 26354361e9b044cea1c2b86c4882fe92 P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:09.698606 16146 raft_consensus.cc:1275] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
W20260709 13:20:09.703590 16293 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.566s	user 0.009s	sys 0.017s
W20260709 13:20:09.704487 16293 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.567s	user 0.009s	sys 0.017s
I20260709 13:20:09.703612 16820 rpcz_store.cc:276] 0709 13:20:07.925594 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:08.679880 (+754286us) service_pool.cc:224] Handling call
0709 13:20:08.729322 (+ 49442us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:09.702643 (+973321us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:09.702952 (+   309us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:20:09.700464 19040 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.660s	user 0.000s	sys 0.014s
I20260709 13:20:09.679334 18758 raft_consensus.cc:515] T 1df93779209748b9860f6d893dd3de53 P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:20:09.718887 18768 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.854s	user 0.019s	sys 0.023s
W20260709 13:20:09.719331 18768 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.878s	user 0.020s	sys 0.023s
I20260709 13:20:09.700987 18817 raft_consensus.cc:515] T 2794af5b99984efebdee564035841cb0 P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:09.721500 18817 leader_election.cc:290] T 2794af5b99984efebdee564035841cb0 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:09.732563 16820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
W20260709 13:20:09.701653 19099 log.cc:927] Time spent T e4dc6b0541ca431ebc757beac00521f1 P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.149s	user 0.000s	sys 0.001s
I20260709 13:20:09.668749 18952 raft_consensus.cc:515] T 84f2da98864b4425bf7c0055951fdd4b P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:09.661020 16823 rpcz_store.cc:276] 0709 13:20:07.362126 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:08.548332 (+1186206us) service_pool.cc:224] Handling call
0709 13:20:08.592247 (+ 43915us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:20:08.674977 (+ 82730us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:08.675009 (+    32us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:20:08.675606 (+   597us) log.cc:844] Serialized 52 byte log entry
0709 13:20:08.873563 (+197957us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:20:08.873611 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:08.873674 (+    63us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:20:09.285700 (+412026us) raft_consensus.cc:1685] finished
0709 13:20:09.285722 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:09.287840 (+  2118us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4429,"raft.queue_time_us":23183,"raft.run_cpu_time_us":187,"raft.run_wall_time_us":16013,"thread_start_us":197507,"threads_started":1,"wal-append.queue_time_us":581356}
I20260709 13:20:09.662459 18944 raft_consensus.cc:515] T 8158bbd17b994d2eb96f560bd0f1b88d P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:09.675134 19073 raft_consensus.cc:515] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:09.745039 16221 raft_consensus.cc:1240] T 1df93779209748b9860f6d893dd3de53 P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 6. Current term is 7. Ops: [6.5-6.5]
I20260709 13:20:09.678267 19043 raft_consensus.cc:515] T 68a6838d82ef4cac83507f906139d90f P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:20:09.757572 16117 leader_election.cc:336] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923): Timed out: RequestConsensusVote RPC to 127.14.177.132:38923 timed out after 2.163s (SENT)
I20260709 13:20:09.672590 18745 raft_consensus.cc:515] T 8a878999c0024d2abe517484180290fc P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:09.762356 16219 raft_consensus.cc:2393] T 84f2da98864b4425bf7c0055951fdd4b P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 5: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
I20260709 13:20:09.766119 16143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f2da98864b4425bf7c0055951fdd4b" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:09.766763 16143 raft_consensus.cc:2468] T 84f2da98864b4425bf7c0055951fdd4b P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 4.
I20260709 13:20:09.697669 18770 raft_consensus.cc:515] T d2ab0ac8347641e6955e9bce7d90c1a2 P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:09.771271 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02838c86531d4f98bf58d1c22b8baf77" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:20:09.771271 18900 raft_consensus.cc:493] T 57a744f5aaf944678a63469cbf514015 P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:09.774000 16143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1df93779209748b9860f6d893dd3de53" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:09.776327 16823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "829b4ad8c5a54eababeea1181b960d4b" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:09.774443 18900 raft_consensus.cc:515] T 57a744f5aaf944678a63469cbf514015 P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:09.777709 16142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f5f89186204242bf377f7c42b2e2da" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:09.778398 16142 raft_consensus.cc:2468] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 7.
I20260709 13:20:09.779417 18900 leader_election.cc:290] T 57a744f5aaf944678a63469cbf514015 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:09.780081 18745 leader_election.cc:290] T 8a878999c0024d2abe517484180290fc P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:09.751924 18817 raft_consensus.cc:493] T 02838c86531d4f98bf58d1c22b8baf77 P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:09.785632 16142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c61a39fd4f84424b129c1a6f9ba404d" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 8 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:09.794096 16820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2372981006d4b218215c28964dea623" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:09.795358 16142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2372981006d4b218215c28964dea623" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:09.754541 19062 raft_consensus.cc:493] T 8c11074579b046d99e585c2818e18a2c P 1ca2725d290e47d68b1a4b609c9816ac [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:09.797530 16823 raft_consensus.cc:2410] T 829b4ad8c5a54eababeea1181b960d4b P cb82cc7bd6db46d980c75237a1402860 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b for term 7 because replica has last-logged OpId of term: 6 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 7.
I20260709 13:20:09.800598 16143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cc959fb7f704fba9009c0595467d074" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:09.807413 16145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62fe3346cd3c42c4a55e14af9073bca4" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 9 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:09.812865 16142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a878999c0024d2abe517484180290fc" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:20:09.813866 16142 raft_consensus.cc:2393] T 8a878999c0024d2abe517484180290fc P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 6: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
I20260709 13:20:09.814173 18758 leader_election.cc:290] T 1df93779209748b9860f6d893dd3de53 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 7 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:09.807235 18817 raft_consensus.cc:515] T 02838c86531d4f98bf58d1c22b8baf77 P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:09.853225 16143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8158bbd17b994d2eb96f560bd0f1b88d" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:20:09.853690 16300 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48684 (request call id 2143) took 1532 ms. Trace:
W20260709 13:20:09.874647 16117 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 121695us
I20260709 13:20:09.878031 16143 raft_consensus.cc:3060] T 8158bbd17b994d2eb96f560bd0f1b88d P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:20:09.854130 16300 rpcz_store.cc:276] 0709 13:20:08.291218 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:08.292668 (+  1450us) service_pool.cc:224] Handling call
0709 13:20:08.293182 (+   514us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:20:08.633443 (+340261us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:08.633578 (+   135us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:20:08.678795 (+ 45217us) log.cc:844] Serialized 52 byte log entry
0709 13:20:08.886538 (+207743us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:20:08.886579 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:08.886651 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:20:09.823364 (+936713us) raft_consensus.cc:1685] finished
0709 13:20:09.823384 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:09.823528 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":277285,"raft.queue_time_us":311337,"raft.run_cpu_time_us":305,"raft.run_wall_time_us":43732,"thread_start_us":190440,"threads_started":2,"wal-append.queue_time_us":600412}
I20260709 13:20:09.905284 16823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f82224f4564fea93572e8206bad1fe" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:09.907670 16150 raft_consensus.cc:2410] T 02838c86531d4f98bf58d1c22b8baf77 P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac for term 7 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
W20260709 13:20:09.908102 19111 log.cc:927] Time spent T 68a6838d82ef4cac83507f906139d90f P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 0.093s	user 0.000s	sys 0.000s
I20260709 13:20:09.909901 18770 leader_election.cc:290] T d2ab0ac8347641e6955e9bce7d90c1a2 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 7 election: Requested vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:09.910089 16219 raft_consensus.cc:1240] T d16dfad0c3b540e5b4b86492ddf87208 P 1ca2725d290e47d68b1a4b609c9816ac [term 6 LEADER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 5. Current term is 6. Ops: []
W20260709 13:20:09.915043 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 61341.3us
I20260709 13:20:09.906901 16823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5718c623ebee4d2bb770f0f9ddb2f164" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:20:09.919032 19010 leader_election.cc:290] T c0eab25ce4c64e018a041beeda2867db P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:09.917837 16265 leader_election.cc:304] T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:20:09.958725 16820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e9b2bf7011f4ae688a79d7bb5a447b4" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:09.959929 16820 raft_consensus.cc:2393] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 5: Already voted for candidate cb82cc7bd6db46d980c75237a1402860 in this term.
I20260709 13:20:09.918882 16823 raft_consensus.cc:2376] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 647a343a542d4a139fe106c19095129b in term 6. Re-sending same reply.
I20260709 13:20:09.961696 16823 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49896 (request call id 942) took 1019 ms. Trace:
I20260709 13:20:09.962124 16823 rpcz_store.cc:276] 0709 13:20:08.941886 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:09.906638 (+964752us) service_pool.cc:224] Handling call
0709 13:20:09.961622 (+ 54984us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:20:09.976380 16820 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49896 (request call id 940) took 1048 ms. Trace:
I20260709 13:20:09.976861 16820 rpcz_store.cc:276] 0709 13:20:08.928317 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:09.819303 (+890986us) service_pool.cc:224] Handling call
0709 13:20:09.976250 (+156947us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:20:09.980533 16142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af57996a8be84d398f0f023e8d08ef77" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 7 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:09.986347 18951 raft_consensus.cc:493] T 5a96373bb0e14dce8a266b2fc6c58507 P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:20:09.986931 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6f07842d3aa47858df5665874c86f3a" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:20:09.987130 18951 raft_consensus.cc:515] T 5a96373bb0e14dce8a266b2fc6c58507 P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:09.987813 16226 raft_consensus.cc:3060] T b6f07842d3aa47858df5665874c86f3a P 1ca2725d290e47d68b1a4b609c9816ac [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:20:09.988895 16219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a878999c0024d2abe517484180290fc" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
W20260709 13:20:09.991043 16190 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:09.876394 19003 leader_election.cc:290] T 434bff3082d141838068f323bdb0d1ed P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 7 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:09.919664 19065 consensus_queue.cc:1048] T 57a744f5aaf944678a63469cbf514015 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.064s
I20260709 13:20:10.005343 19066 raft_consensus.cc:3060] T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:20:10.002915 16219 raft_consensus.cc:2410] T 8a878999c0024d2abe517484180290fc P 1ca2725d290e47d68b1a4b609c9816ac [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b for term 7 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
W20260709 13:20:10.006157 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 88454.4us
I20260709 13:20:09.909893 18817 leader_election.cc:290] T 02838c86531d4f98bf58d1c22b8baf77 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
W20260709 13:20:10.008970 16264 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:10.007899 18951 leader_election.cc:290] T 5a96373bb0e14dce8a266b2fc6c58507 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:10.019577 16143 raft_consensus.cc:2410] T 8158bbd17b994d2eb96f560bd0f1b88d P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac for term 7 because replica has last-logged OpId of term: 6 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260709 13:20:10.020146 16219 raft_consensus.cc:1240] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac [term 6 LEADER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 5. Current term is 6. Ops: []
I20260709 13:20:10.020643 19073 leader_election.cc:290] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:20:10.021235 19113 raft_consensus.cc:493] T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:10.021727 19043 leader_election.cc:290] T 68a6838d82ef4cac83507f906139d90f P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:10.021792 19113 raft_consensus.cc:515] T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:10.022102 16226 raft_consensus.cc:2468] T b6f07842d3aa47858df5665874c86f3a P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 7.
I20260709 13:20:10.022476 16265 leader_election.cc:304] T 1df93779209748b9860f6d893dd3de53 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:20:10.023638 19065 raft_consensus.cc:2749] T 1df93779209748b9860f6d893dd3de53 P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:20:10.024488 16293 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48684 (request call id 2162) took 1002 ms. Trace:
I20260709 13:20:10.025206 19113 leader_election.cc:290] T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:10.024901 16293 rpcz_store.cc:276] 0709 13:20:09.027705 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:09.036285 (+  8580us) service_pool.cc:224] Handling call
0709 13:20:09.096015 (+ 59730us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:20:09.097231 (+  1216us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:09.097269 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:20:09.137411 (+ 40142us) log.cc:844] Serialized 26 byte log entry
0709 13:20:09.704988 (+567577us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:20:09.705013 (+    25us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:09.705057 (+    44us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:20:10.024273 (+319216us) raft_consensus.cc:1685] finished
0709 13:20:10.024293 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:10.024420 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":567187,"threads_started":1,"wal-append.queue_time_us":884888}
I20260709 13:20:10.026844 19003 raft_consensus.cc:2804] T c9944d5e4b8c4109af9955c4f08c9095 P cb82cc7bd6db46d980c75237a1402860 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:20:10.027462 19003 raft_consensus.cc:697] T c9944d5e4b8c4109af9955c4f08c9095 P cb82cc7bd6db46d980c75237a1402860 [term 7 LEADER]: Becoming Leader. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:20:10.028438 19003 consensus_queue.cc:237] T c9944d5e4b8c4109af9955c4f08c9095 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.8, Last appended by leader: 8, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:10.025166 16265 leader_election.cc:304] T d2ab0ac8347641e6955e9bce7d90c1a2 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860
I20260709 13:20:10.030233 19065 raft_consensus.cc:3060] T d2ab0ac8347641e6955e9bce7d90c1a2 P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:20:10.031006 19108 raft_consensus.cc:493] T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:10.031665 19108 raft_consensus.cc:515] T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:10.034070 19108 leader_election.cc:290] T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:09.831735 19062 raft_consensus.cc:515] T 8c11074579b046d99e585c2818e18a2c P 1ca2725d290e47d68b1a4b609c9816ac [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:10.036067 16265 leader_election.cc:304] T 66ae176961b74813a58bc15c9c4c6f2a P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860
I20260709 13:20:10.036715 19062 leader_election.cc:290] T 8c11074579b046d99e585c2818e18a2c P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:10.037130 18843 raft_consensus.cc:2749] T 66ae176961b74813a58bc15c9c4c6f2a P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:20:10.039338 16265 leader_election.cc:304] T 18ad7e3af5504571a69d10c00750a01c P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:20:10.043627 18952 leader_election.cc:290] T 84f2da98864b4425bf7c0055951fdd4b P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 5 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:10.019551 18857 leader_election.cc:290] T 26354361e9b044cea1c2b86c4882fe92 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
W20260709 13:20:10.045979 16788 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:10.048373 18944 leader_election.cc:290] T 8158bbd17b994d2eb96f560bd0f1b88d P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:09.989363 19042 leader_election.cc:290] T 9e65676b21b748728ad39668cfe722ec P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:10.061005 16818 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49892 (request call id 448) took 1721 ms. Trace:
W20260709 13:20:10.062692 19089 consensus_meta.cc:287] T 23f82224f4564fea93572e8206bad1fe P 90f3da57796b488c9087a2f523adf045: Time spent flushing consensus metadata: real 0.967s	user 0.014s	sys 0.000s
I20260709 13:20:10.055935 16820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af57996a8be84d398f0f023e8d08ef77" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:20:10.063508 16225 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57604 (request call id 2178) took 1200 ms. Trace:
W20260709 13:20:10.071249 19097 log.cc:927] Time spent T 19baa7bc367844c288bca8f81439c413 P cb82cc7bd6db46d980c75237a1402860: Append to log took a long time: real 0.169s	user 0.001s	sys 0.000s
I20260709 13:20:10.070838 19073 consensus_queue.cc:1048] T 23f82224f4564fea93572e8206bad1fe P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
W20260709 13:20:10.085268 16264 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:20:10.090018 16190 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:20:10.101148 16191 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:20:10.102792 16266 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:20:10.112473 16264 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:10.104679 19089 raft_consensus.cc:515] T 23f82224f4564fea93572e8206bad1fe P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:10.110848 16299 raft_consensus.cc:1240] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 5. Current term is 6. Ops: [5.5-5.5]
I20260709 13:20:10.088493 16818 rpcz_store.cc:276] 0709 13:20:08.339146 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:08.721596 (+382450us) service_pool.cc:224] Handling call
0709 13:20:08.722017 (+   421us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:20:08.744330 (+ 22313us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:08.744374 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:20:08.805394 (+ 61020us) log.cc:844] Serialized 26 byte log entry
0709 13:20:09.047346 (+241952us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:20:09.047396 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:09.047469 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:20:10.060764 (+1013295us) raft_consensus.cc:1685] finished
0709 13:20:10.060785 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:10.060940 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7757,"raft.queue_time_us":25722,"raft.run_cpu_time_us":138,"raft.run_wall_time_us":2079,"thread_start_us":142719,"threads_started":1,"wal-append.queue_time_us":1205927}
I20260709 13:20:10.083411 19066 raft_consensus.cc:2749] T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:20:10.118248 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 1.01971e+06us
I20260709 13:20:10.117038 16190 leader_election.cc:304] T c2f5f89186204242bf377f7c42b2e2da P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:20:10.099717 16225 rpcz_store.cc:276] 0709 13:20:08.862811 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:08.866569 (+  3758us) service_pool.cc:224] Handling call
0709 13:20:08.961441 (+ 94872us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 13:20:09.618258 (+656817us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:09.618289 (+    31us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 13:20:09.619089 (+   800us) log.cc:844] Serialized 78 byte log entry
0709 13:20:09.788326 (+169237us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:20:09.788364 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:09.788431 (+    67us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:20:10.063273 (+274842us) raft_consensus.cc:1685] finished
0709 13:20:10.063292 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:10.063451 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":15584,"raft.queue_time_us":31425,"raft.run_cpu_time_us":198,"raft.run_wall_time_us":33888,"thread_start_us":168763,"threads_started":1,"wal-append.queue_time_us":436062}
W20260709 13:20:10.125864 16789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 53754.5us
I20260709 13:20:10.127807 16190 leader_election.cc:304] T 8cc959fb7f704fba9009c0595467d074 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:20:10.121970 16294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87147731a68400e8cce9b879daa7c5d" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
W20260709 13:20:10.128777 16117 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 144366us
I20260709 13:20:10.106374 19065 raft_consensus.cc:2749] T d2ab0ac8347641e6955e9bce7d90c1a2 P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:20:10.104813 18843 raft_consensus.cc:493] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:10.145781 16220 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57604 (request call id 2176) took 1349 ms. Trace:
I20260709 13:20:10.146255 16299 raft_consensus.cc:1217] T 68a6838d82ef4cac83507f906139d90f P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Deduplicated request from leader. Original: 5.4->[7.5-7.5]   Dedup: 7.5->[]
I20260709 13:20:10.147183 16299 raft_consensus.cc:1240] T 68a6838d82ef4cac83507f906139d90f P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 7. Current term is 8. Ops: [7.5-7.5]
I20260709 13:20:10.140161 16818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87147731a68400e8cce9b879daa7c5d" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:10.168918 16294 raft_consensus.cc:3060] T b87147731a68400e8cce9b879daa7c5d P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:20:10.146451 16220 rpcz_store.cc:276] 0709 13:20:08.796295 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:08.796421 (+   126us) service_pool.cc:224] Handling call
0709 13:20:08.796835 (+   414us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:20:08.833867 (+ 37032us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:20:08.885189 (+ 51322us) log.cc:844] Serialized 14 byte log entry
0709 13:20:09.691116 (+805927us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:20:09.701005 (+  9889us) log.cc:844] Serialized 52 byte log entry
0709 13:20:09.701228 (+   223us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:20:10.122264 (+421036us) log.cc:844] Serialized 14 byte log entry
0709 13:20:10.145314 (+ 23050us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:10.145427 (+   113us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:20:10.145487 (+    60us) raft_consensus.cc:1685] finished
0709 13:20:10.145507 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:10.145704 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":56927,"prepare.queue_time_us":86933,"prepare.run_cpu_time_us":69,"prepare.run_wall_time_us":70,"raft.queue_time_us":38919,"raft.run_cpu_time_us":271,"raft.run_wall_time_us":16432,"thread_start_us":1178638,"threads_started":2,"wal-append.queue_time_us":1076638}
W20260709 13:20:10.176800 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 49296.5us
I20260709 13:20:10.179435 16818 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49896 (request call id 943) took 1206 ms. Trace:
I20260709 13:20:10.179999 16818 rpcz_store.cc:276] 0709 13:20:08.973030 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:10.123603 (+1150573us) service_pool.cc:224] Handling call
0709 13:20:10.179348 (+ 55745us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:20:10.191996 16299 raft_consensus.cc:1275] T fffab26427e24d398e2695aad40ca134 P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 3. (term mismatch)
I20260709 13:20:10.192479 16299 pending_rounds.cc:85] T fffab26427e24d398e2695aad40ca134 P 90f3da57796b488c9087a2f523adf045: Aborting all ops after (but not including) 2
I20260709 13:20:10.192749 16299 pending_rounds.cc:107] T fffab26427e24d398e2695aad40ca134 P 90f3da57796b488c9087a2f523adf045: Aborting uncommitted NO_OP operation due to leader change: 4.3
I20260709 13:20:10.193001 16299 raft_consensus.cc:2889] T fffab26427e24d398e2695aad40ca134 P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:20:10.194444 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c74490c40243a88e62d1760b5cd814" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:10.195231 18843 raft_consensus.cc:515] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:20:10.196280 16824 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49896 (request call id 902) took 3147 ms (client timeout 1655 ms). Trace:
W20260709 13:20:10.196775 16824 rpcz_store.cc:269] 0709 13:20:07.076085 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:07.076268 (+   183us) service_pool.cc:224] Handling call
0709 13:20:10.196003 (+3119735us) spinlock_profiling.cc:243] Waited 1.55 s on lock 0x7b5400787154. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:20:10.196175 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3567915392}
I20260709 13:20:10.197599 18843 leader_election.cc:290] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:10.198167 19113 consensus_queue.cc:1048] T e4dc6b0541ca431ebc757beac00521f1 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:20:10.198779 16825 raft_consensus.cc:1240] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860 [term 8 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 7. Current term is 8. Ops: []
I20260709 13:20:10.192044 16292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6f07842d3aa47858df5665874c86f3a" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:10.136634 18951 raft_consensus.cc:2804] T c2f5f89186204242bf377f7c42b2e2da P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:20:10.200161 16292 raft_consensus.cc:2468] T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 6.
I20260709 13:20:10.195122 16299 raft_consensus.cc:2468] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 6.
I20260709 13:20:10.202960 16293 raft_consensus.cc:1275] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:20:10.129928 18935 consensus_queue.cc:1048] T 18ad7e3af5504571a69d10c00750a01c P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 2, Time since last communication: 0.070s
I20260709 13:20:10.174335 18952 raft_consensus.cc:2804] T 8cc959fb7f704fba9009c0595467d074 P 1ca2725d290e47d68b1a4b609c9816ac [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:20:10.175227 19066 raft_consensus.cc:2749] T 18ad7e3af5504571a69d10c00750a01c P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:20:10.140161 18770 consensus_queue.cc:1048] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.064s
W20260709 13:20:10.212136 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 74673.1us
W20260709 13:20:10.212918 16114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:20:10.214691 19116 log.cc:927] Time spent T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 0.063s	user 0.000s	sys 0.000s
I20260709 13:20:10.202853 18951 raft_consensus.cc:493] T c2f5f89186204242bf377f7c42b2e2da P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:20:10.220054 16266 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:10.220356 16818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9944d5e4b8c4109af9955c4f08c9095" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
W20260709 13:20:10.220657 16266 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:10.221462 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f82224f4564fea93572e8206bad1fe" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:20:10.222250 16299 raft_consensus.cc:2393] T 23f82224f4564fea93572e8206bad1fe P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 5: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
I20260709 13:20:10.224150 16818 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49906 (request call id 632) took 1478 ms. Trace:
I20260709 13:20:10.224557 16818 rpcz_store.cc:276] 0709 13:20:08.745552 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:10.209083 (+1463531us) service_pool.cc:224] Handling call
0709 13:20:10.224087 (+ 15004us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:20:10.223378 16222 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57604 (request call id 2175) took 1430 ms. Trace:
W20260709 13:20:10.228626 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 189364us
I20260709 13:20:10.231000 16292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f5f89186204242bf377f7c42b2e2da" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:10.231377 16820 raft_consensus.cc:2468] T af57996a8be84d398f0f023e8d08ef77 P cb82cc7bd6db46d980c75237a1402860 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 7.
I20260709 13:20:10.231855 16292 raft_consensus.cc:2393] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 8: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
I20260709 13:20:10.231923 16820 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49896 (request call id 944) took 1240 ms. Trace:
I20260709 13:20:10.232369 16820 rpcz_store.cc:276] 0709 13:20:08.991443 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:10.009714 (+1018271us) service_pool.cc:224] Handling call
0709 13:20:10.231863 (+222149us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:20:10.233348 16820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ab0ac8347641e6955e9bce7d90c1a2" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:20:10.233843 16820 raft_consensus.cc:3060] T d2ab0ac8347641e6955e9bce7d90c1a2 P cb82cc7bd6db46d980c75237a1402860 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:20:10.230123 16821 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49906 (request call id 617) took 2883 ms. Trace:
I20260709 13:20:10.240774 16818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b7d1ab519044eda2d5e04a07242f25" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:10.213363 18952 raft_consensus.cc:493] T 8cc959fb7f704fba9009c0595467d074 P 1ca2725d290e47d68b1a4b609c9816ac [term 6 FOLLOWER]: Starting leader election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:20:10.242861 16818 raft_consensus.cc:2410] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b for term 5 because replica has last-logged OpId of term: 4 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 6.
I20260709 13:20:10.234941 19066 raft_consensus.cc:493] T c9944d5e4b8c4109af9955c4f08c9095 P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:10.226110 16222 rpcz_store.cc:276] 0709 13:20:08.793031 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:08.793176 (+   145us) service_pool.cc:224] Handling call
0709 13:20:08.793624 (+   448us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:20:08.841488 (+ 47864us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:08.841530 (+    42us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:20:08.918917 (+ 77387us) log.cc:844] Serialized 52 byte log entry
0709 13:20:09.970652 (+1051735us) spinlock_profiling.cc:243] Waited 8.01 ms on lock 0x7b1800336a00. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b693ade1 00007f69b69475b1 00007f69ada54e07 00007f69b68afab9 00007f69ada7169b 00007f69ada6f93f 00007f69ada71c02 00007f69b06f920b 00007f69b06fed23 00007f69b06ff0d1 00007f69b0b1aa0f
0709 13:20:10.002397 (+ 31745us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:20:10.002440 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:10.010826 (+  8386us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:20:10.217061 (+206235us) raft_consensus.cc:1685] finished
0709 13:20:10.217083 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:10.223309 (+  6226us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":36471,"raft.queue_time_us":68990,"raft.run_cpu_time_us":201,"raft.run_wall_time_us":20716,"spinlock_wait_cycles":18413824,"thread_start_us":31530,"threads_started":1,"wal-append.queue_time_us":1187764}
I20260709 13:20:10.248972 16818 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49896 (request call id 945) took 1238 ms. Trace:
I20260709 13:20:10.249802 16818 rpcz_store.cc:276] 0709 13:20:09.010421 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:10.225234 (+1214813us) service_pool.cc:224] Handling call
0709 13:20:10.248908 (+ 23674us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:20:10.239722 16821 rpcz_store.cc:276] 0709 13:20:07.347229 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:08.532499 (+1185270us) service_pool.cc:224] Handling call
0709 13:20:08.585052 (+ 52553us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:20:08.675305 (+ 90253us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:08.675347 (+    42us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:20:08.676007 (+   660us) log.cc:844] Serialized 52 byte log entry
0709 13:20:09.215816 (+539809us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:20:09.215882 (+    66us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:09.215975 (+    93us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:20:10.227205 (+1011230us) raft_consensus.cc:1685] finished
0709 13:20:10.227226 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:10.230061 (+  2835us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1766,"raft.queue_time_us":14922,"spinlock_wait_cycles":18192768,"thread_start_us":445645,"threads_started":1,"wal-append.queue_time_us":1216241}
I20260709 13:20:10.223224 18951 raft_consensus.cc:3060] T c2f5f89186204242bf377f7c42b2e2da P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:20:10.254204 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3884f864a96d428785895e5463e5728f" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:10.257395 16822 raft_consensus.cc:1240] T 19baa7bc367844c288bca8f81439c413 P cb82cc7bd6db46d980c75237a1402860 [term 7 LEARNER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 6. Current term is 7. Ops: [4.5-6.6]
I20260709 13:20:10.264932 16151 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40844 (request call id 1985) took 1708 ms. Trace:
I20260709 13:20:10.265372 16151 rpcz_store.cc:276] 0709 13:20:08.556858 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:08.560418 (+  3560us) service_pool.cc:224] Handling call
0709 13:20:10.255489 (+1695071us) spinlock_profiling.cc:243] Waited 363 ms on lock 0x7b54000ea454. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:20:10.264844 (+  9355us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":835950976}
I20260709 13:20:10.272153 19066 raft_consensus.cc:515] T c9944d5e4b8c4109af9955c4f08c9095 P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:10.273782 16292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 6 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:20:10.270398 16299 raft_consensus.cc:2468] T 3884f864a96d428785895e5463e5728f P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 5.
I20260709 13:20:10.276331 16292 raft_consensus.cc:2393] T ecc1e363230043fe9fef390f0b979f20 P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 6: Already voted for candidate cb82cc7bd6db46d980c75237a1402860 in this term.
I20260709 13:20:10.277442 16821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c11074579b046d99e585c2818e18a2c" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:10.275425 16818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada5a004ba7c46819051e3ccec71ffdb" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:10.278952 16818 raft_consensus.cc:2468] T ada5a004ba7c46819051e3ccec71ffdb P cb82cc7bd6db46d980c75237a1402860 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 7.
I20260709 13:20:10.253439 18952 raft_consensus.cc:3060] T 8cc959fb7f704fba9009c0595467d074 P 1ca2725d290e47d68b1a4b609c9816ac [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:20:10.283216 16265 leader_election.cc:304] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:20:10.215233 19126 consensus_queue.cc:1059] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.037s
I20260709 13:20:10.287740 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18ad7e3af5504571a69d10c00750a01c" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:10.235649 19089 leader_election.cc:290] T 23f82224f4564fea93572e8206bad1fe P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 5 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:10.290102 16299 raft_consensus.cc:2410] T 18ad7e3af5504571a69d10c00750a01c P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b for term 7 because replica has last-logged OpId of term: 6 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 4.
I20260709 13:20:10.236522 18114 consensus_queue.cc:1048] T 19baa7bc367844c288bca8f81439c413 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.044s
I20260709 13:20:10.291865 16192 leader_election.cc:304] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860
I20260709 13:20:10.289507 16292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada5a004ba7c46819051e3ccec71ffdb" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
W20260709 13:20:10.297168 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 73133.7us
I20260709 13:20:10.304936 18944 consensus_queue.cc:1048] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:20:10.310344 18768 consensus_queue.cc:1048] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:20:10.320254 16117 leader_election.cc:304] T 746aabf856ab401c9ef5e859371e7b67 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860
I20260709 13:20:10.274138 19135 raft_consensus.cc:493] T 9e65676b21b748728ad39668cfe722ec P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:10.326630 16820 raft_consensus.cc:2468] T d2ab0ac8347641e6955e9bce7d90c1a2 P cb82cc7bd6db46d980c75237a1402860 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 7.
I20260709 13:20:10.329247 16294 raft_consensus.cc:2468] T b87147731a68400e8cce9b879daa7c5d P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 8.
W20260709 13:20:10.331822 16789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 111180us
W20260709 13:20:10.359647 16192 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 67860.7us
I20260709 13:20:10.361258 16292 raft_consensus.cc:2468] T ada5a004ba7c46819051e3ccec71ffdb P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 6.
I20260709 13:20:10.361992 19089 raft_consensus.cc:2749] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:20:10.352429 19066 leader_election.cc:290] T c9944d5e4b8c4109af9955c4f08c9095 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:10.323431 19113 consensus_queue.cc:1048] T 68a6838d82ef4cac83507f906139d90f P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.040s
I20260709 13:20:10.394743 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a96373bb0e14dce8a266b2fc6c58507" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 6 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:10.395422 16292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c11074579b046d99e585c2818e18a2c" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:10.395691 16819 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49892 (request call id 449) took 2055 ms. Trace:
I20260709 13:20:10.396400 16292 raft_consensus.cc:2410] T 8c11074579b046d99e585c2818e18a2c P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac for term 7 because replica has last-logged OpId of term: 6 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 7.
I20260709 13:20:10.396337 16819 rpcz_store.cc:276] 0709 13:20:08.340622 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:10.264826 (+1924204us) service_pool.cc:224] Handling call
0709 13:20:10.265233 (+   407us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:20:10.272000 (+  6767us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:10.272036 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:20:10.272618 (+   582us) log.cc:844] Serialized 26 byte log entry
0709 13:20:10.333318 (+ 60700us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:20:10.333386 (+    68us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:10.333562 (+   176us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:20:10.395439 (+ 61877us) raft_consensus.cc:1685] finished
0709 13:20:10.395464 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:10.395605 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":8185,"raft.run_cpu_time_us":217,"raft.run_wall_time_us":44039,"thread_start_us":60117,"threads_started":1,"wal-append.queue_time_us":120797}
I20260709 13:20:10.397567 16292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0eab25ce4c64e018a041beeda2867db" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:10.397202 19126 consensus_queue.cc:1059] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:20:10.398304 16292 raft_consensus.cc:2468] T c0eab25ce4c64e018a041beeda2867db P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 7.
I20260709 13:20:10.398319 16819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9944d5e4b8c4109af9955c4f08c9095" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:10.351147 18114 consensus_queue.cc:1243] T 19baa7bc367844c288bca8f81439c413 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Peer 647a343a542d4a139fe106c19095129b log is divergent from this leader: its last log entry 4.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 13:20:10.395649 16300 raft_consensus.cc:2468] T 5a96373bb0e14dce8a266b2fc6c58507 P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 5.
I20260709 13:20:10.358630 16294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
W20260709 13:20:10.393448 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 102366us
W20260709 13:20:10.399785 16117 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 79616.8us
I20260709 13:20:10.403311 16191 leader_election.cc:304] T fffab26427e24d398e2695aad40ca134 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:20:10.388070 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ab0ac8347641e6955e9bce7d90c1a2" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:20:10.400633 16821 raft_consensus.cc:1240] T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860 [term 6 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 4. Current term is 6. Ops: []
W20260709 13:20:10.401955 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 147995us
I20260709 13:20:10.404851 16299 raft_consensus.cc:3060] T d2ab0ac8347641e6955e9bce7d90c1a2 P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:20:10.406564 16821 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49892 (request call id 450) took 2059 ms. Trace:
I20260709 13:20:10.407047 16821 rpcz_store.cc:276] 0709 13:20:08.347378 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:10.291527 (+1944149us) service_pool.cc:224] Handling call
0709 13:20:10.353158 (+ 61631us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:10.406294 (+ 53136us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:10.406499 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:20:10.389986 16790 leader_election.cc:400] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 8 election: Vote denied by peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943) with higher term. Message: Invalid argument: T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 for earlier term 8. Current term is 9.
I20260709 13:20:10.412706 16819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f82224f4564fea93572e8206bad1fe" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:20:10.421490 16823 raft_consensus.cc:1240] T 66f75124215f4ca6bef831e174a792d8 P cb82cc7bd6db46d980c75237a1402860 [term 7 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 6. Current term is 7. Ops: []
W20260709 13:20:10.423619 16817 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49892 (request call id 441) took 2307 ms (client timeout 2248 ms). Trace:
I20260709 13:20:10.365289 18857 raft_consensus.cc:3060] T 746aabf856ab401c9ef5e859371e7b67 P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:20:10.424484 16816 raft_consensus.cc:1240] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P cb82cc7bd6db46d980c75237a1402860 [term 6 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 5. Current term is 6. Ops: [5.6-5.6]
I20260709 13:20:10.455271 18551 raft_consensus.cc:3055] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:20:10.455833 18551 raft_consensus.cc:740] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:20:10.454869 16823 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49906 (request call id 628) took 1909 ms. Trace:
W20260709 13:20:10.451406 16192 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 87713.2us
W20260709 13:20:10.430189 16817 rpcz_store.cc:269] 0709 13:20:08.116198 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:08.427560 (+311362us) service_pool.cc:224] Handling call
0709 13:20:10.423365 (+1995805us) spinlock_profiling.cc:243] Waited 394 ms on lock 0x7b54007c3b54. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:20:10.423537 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":906859904}
I20260709 13:20:10.430967 16291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f2da98864b4425bf7c0055951fdd4b" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:20:10.461040 16821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a9de347c711477e87c582d003f36925" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 8 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:10.461462 16291 raft_consensus.cc:3060] T 84f2da98864b4425bf7c0055951fdd4b P 90f3da57796b488c9087a2f523adf045 [term 3 FOLLOWER]: Advancing to term 5
I20260709 13:20:10.462007 16817 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49906 (request call id 629) took 1916 ms. Trace:
I20260709 13:20:10.459674 16823 rpcz_store.cc:276] 0709 13:20:08.545083 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:10.420820 (+1875737us) service_pool.cc:224] Handling call
0709 13:20:10.421191 (+   371us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:10.454559 (+ 33368us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:10.454804 (+   245us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:20:10.428063 18939 raft_consensus.cc:2749] T fffab26427e24d398e2695aad40ca134 P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:20:10.435402 16292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d4ab870a05347868956aa8b841b2981" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:10.362980 18770 raft_consensus.cc:2749] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:20:10.461788 16821 raft_consensus.cc:2393] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 8: Already voted for candidate cb82cc7bd6db46d980c75237a1402860 in this term.
I20260709 13:20:10.469695 16294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1df93779209748b9860f6d893dd3de53" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:20:10.472432 16117 leader_election.cc:304] T 3884f864a96d428785895e5463e5728f P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:20:10.494258 16294 raft_consensus.cc:3060] T 1df93779209748b9860f6d893dd3de53 P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:20:10.477921 16265 leader_election.cc:304] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860
I20260709 13:20:10.390297 19133 consensus_queue.cc:1048] T 8c11074579b046d99e585c2818e18a2c P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.079s
I20260709 13:20:10.362298 19135 raft_consensus.cc:515] T 9e65676b21b748728ad39668cfe722ec P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:20:10.474529 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 71335.6us
I20260709 13:20:10.456686 18551 consensus_queue.cc:260] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:10.501056 16823 raft_consensus.cc:1275] T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860 [term 6 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:20:10.503139 16823 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49906 (request call id 631) took 1784 ms. Trace:
I20260709 13:20:10.503509 16823 rpcz_store.cc:276] 0709 13:20:08.719260 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:10.495346 (+1776086us) service_pool.cc:224] Handling call
0709 13:20:10.495712 (+   366us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:10.502804 (+  7092us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:10.503085 (+   281us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:20:10.504320 16264 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:10.506043 16819 raft_consensus.cc:1275] T 3a7ca61592e848ea9b0fb81a2ce33cdc P cb82cc7bd6db46d980c75237a1402860 [term 9 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:20:10.506847 16819 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49906 (request call id 630) took 1906 ms. Trace:
I20260709 13:20:10.507253 16819 rpcz_store.cc:276] 0709 13:20:08.600075 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:10.473978 (+1873903us) service_pool.cc:224] Handling call
0709 13:20:10.480274 (+  6296us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:10.506527 (+ 26253us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:10.506778 (+   251us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:20:10.510669 16299 raft_consensus.cc:2468] T d2ab0ac8347641e6955e9bce7d90c1a2 P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 7.
I20260709 13:20:10.512533 16299 raft_consensus.cc:1240] T d16dfad0c3b540e5b4b86492ddf87208 P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 5. Current term is 6. Ops: []
W20260709 13:20:10.526098 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 136359us
W20260709 13:20:10.531814 16117 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 100434us
I20260709 13:20:10.532269 16819 raft_consensus.cc:1240] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860 [term 6 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 5. Current term is 6. Ops: []
I20260709 13:20:10.532582 16292 raft_consensus.cc:2468] T 8d4ab870a05347868956aa8b841b2981 P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 7.
I20260709 13:20:10.532917 16819 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49906 (request call id 635) took 1029 ms. Trace:
I20260709 13:20:10.533236 16819 rpcz_store.cc:276] 0709 13:20:09.503452 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:10.508155 (+1004703us) service_pool.cc:224] Handling call
0709 13:20:10.532103 (+ 23948us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:10.532674 (+   571us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:10.532864 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:20:10.535599 16788 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:10.536738 16821 raft_consensus.cc:1240] T 434bff3082d141838068f323bdb0d1ed P cb82cc7bd6db46d980c75237a1402860 [term 7 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 6. Current term is 7. Ops: []
I20260709 13:20:10.547406 16819 raft_consensus.cc:1240] T 8638ee363273426db1393dd81e5c7b72 P cb82cc7bd6db46d980c75237a1402860 [term 6 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 5. Current term is 6. Ops: [5.7-5.7]
I20260709 13:20:10.524806 18551 raft_consensus.cc:3060] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:20:10.502388 19066 raft_consensus.cc:2749] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:20:10.548732 16191 leader_election.cc:304] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
I20260709 13:20:10.556025 16117 leader_election.cc:304] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860
I20260709 13:20:10.550144 16294 raft_consensus.cc:2468] T 1df93779209748b9860f6d893dd3de53 P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 7.
I20260709 13:20:10.560321 16821 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49896 (request call id 938) took 1632 ms. Trace:
I20260709 13:20:10.557572 16820 raft_consensus.cc:3055] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [term 8 LEADER]: Stepping down as leader of term 8
W20260709 13:20:10.552322 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 74597.4us
I20260709 13:20:10.562623 16820 raft_consensus.cc:740] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:20:10.562547 16821 rpcz_store.cc:276] 0709 13:20:08.927739 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:10.496173 (+1568434us) service_pool.cc:224] Handling call
0709 13:20:10.526932 (+ 30759us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:10.560053 (+ 33121us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:10.560257 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:20:10.564334 16820 consensus_queue.cc:260] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:10.556288 19042 consensus_queue.cc:1048] T af57996a8be84d398f0f023e8d08ef77 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.009s
I20260709 13:20:10.565673 16820 raft_consensus.cc:3060] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:20:10.568017 16299 raft_consensus.cc:1240] T 1df93779209748b9860f6d893dd3de53 P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 6. Current term is 7. Ops: [4.4-6.5]
I20260709 13:20:10.563915 16192 leader_election.cc:304] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:20:10.572053 16192 leader_election.cc:304] T d2ab0ac8347641e6955e9bce7d90c1a2 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:20:10.573930 16265 leader_election.cc:304] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860
I20260709 13:20:10.570551 16117 leader_election.cc:304] T 829b4ad8c5a54eababeea1181b960d4b P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860
I20260709 13:20:10.584367 19113 raft_consensus.cc:2804] T d2ab0ac8347641e6955e9bce7d90c1a2 P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:20:10.586812 16291 raft_consensus.cc:2468] T 84f2da98864b4425bf7c0055951fdd4b P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 5.
I20260709 13:20:10.579808 19126 raft_consensus.cc:493] T 3a7ca61592e848ea9b0fb81a2ce33cdc P cb82cc7bd6db46d980c75237a1402860 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:10.589839 19126 raft_consensus.cc:515] T 3a7ca61592e848ea9b0fb81a2ce33cdc P cb82cc7bd6db46d980c75237a1402860 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:10.592075 19126 leader_election.cc:290] T 3a7ca61592e848ea9b0fb81a2ce33cdc P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:10.593461 18857 raft_consensus.cc:2749] T 746aabf856ab401c9ef5e859371e7b67 P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:20:10.595904 16151 raft_consensus.cc:1240] T af57996a8be84d398f0f023e8d08ef77 P 647a343a542d4a139fe106c19095129b [term 7 LEADER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 6. Current term is 7. Ops: [6.7-6.7]
I20260709 13:20:10.597060 18951 raft_consensus.cc:515] T c2f5f89186204242bf377f7c42b2e2da P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:20:10.600340 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 63567.7us
I20260709 13:20:10.566468 18770 raft_consensus.cc:493] T 746aabf856ab401c9ef5e859371e7b67 P 1ca2725d290e47d68b1a4b609c9816ac [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:10.569270 18599 raft_consensus.cc:2804] T 3884f864a96d428785895e5463e5728f P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:20:10.569048 18892 raft_consensus.cc:2749] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:20:10.490470 16817 rpcz_store.cc:276] 0709 13:20:08.545381 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:10.461174 (+1915793us) service_pool.cc:224] Handling call
0709 13:20:10.461483 (+   309us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:10.461680 (+   197us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:10.461708 (+    28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:20:10.461748 (+    40us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:20:10.461781 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:10.461834 (+    53us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:10.461936 (+   102us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:20:10.587000 19010 raft_consensus.cc:2804] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:20:10.581701 18939 raft_consensus.cc:3060] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:20:10.621682 16117 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 51207.4us
I20260709 13:20:10.621111 16820 raft_consensus.cc:1275] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [term 9 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 8. (term mismatch)
I20260709 13:20:10.631179 16820 pending_rounds.cc:85] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860: Aborting all ops after (but not including) 7
I20260709 13:20:10.631474 16820 pending_rounds.cc:107] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860: Aborting uncommitted NO_OP operation due to leader change: 8.8
I20260709 13:20:10.631703 16820 raft_consensus.cc:2889] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:20:10.632187 16791 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:10.577555 19136 raft_consensus.cc:2749] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:20:10.632216 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66f75124215f4ca6bef831e174a792d8" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:20:10.609741 18944 raft_consensus.cc:2749] T 829b4ad8c5a54eababeea1181b960d4b P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:20:10.620496 16790 connection.cc:256] RPC call timeout handler was delayed by -1.9440592509999988s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 1.5s for a response
I20260709 13:20:10.615336 18599 raft_consensus.cc:493] T 3884f864a96d428785895e5463e5728f P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:09.760203 15046 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:20:10.641600 16265 leader_election.cc:304] T 7c2095a5f4ad49408dbc7f69c606120d P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 647a343a542d4a139fe106c19095129b
W20260709 13:20:10.693397 16192 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 109752us
I20260709 13:20:10.695706 16046 catalog_manager.cc:5697] T 829b4ad8c5a54eababeea1181b960d4b P cb82cc7bd6db46d980c75237a1402860 reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "cb82cc7bd6db46d980c75237a1402860" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:20:10.699599 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 83261us
I20260709 13:20:10.628933 19010 raft_consensus.cc:493] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:10.637580 19148 raft_consensus.cc:493] T 26354361e9b044cea1c2b86c4882fe92 P cb82cc7bd6db46d980c75237a1402860 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:20:10.651818 16264 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:10.701992 18599 raft_consensus.cc:3060] T 3884f864a96d428785895e5463e5728f P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:20:10.635582 19146 raft_consensus.cc:670] T 806d7690c2b644959152074bb2cab0a7 P cb82cc7bd6db46d980c75237a1402860: failed to trigger leader election: Illegal state: only voting members can start elections: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } }
I20260709 13:20:10.706059 16150 raft_consensus.cc:3060] T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:20:10.706583 16264 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:10.636595 18951 leader_election.cc:290] T c2f5f89186204242bf377f7c42b2e2da P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 9 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:10.708170 16144 raft_consensus.cc:1240] T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b [term 7 LEADER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 6. Current term is 7. Ops: [6.7-6.7]
I20260709 13:20:10.618031 18952 raft_consensus.cc:515] T 8cc959fb7f704fba9009c0595467d074 P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:10.615938 18770 raft_consensus.cc:515] T 746aabf856ab401c9ef5e859371e7b67 P 1ca2725d290e47d68b1a4b609c9816ac [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:10.698920 16046 catalog_manager.cc:5697] T d2ab0ac8347641e6955e9bce7d90c1a2 P cb82cc7bd6db46d980c75237a1402860 reported cstate change: term changed from 4 to 5, leader changed from cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) to 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130). New cstate: current_term: 5 leader_uuid: "1ca2725d290e47d68b1a4b609c9816ac" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:20:10.707151 19010 raft_consensus.cc:3060] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:20:10.720283 16117 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 90144.1us
I20260709 13:20:10.721952 16151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 6 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
W20260709 13:20:10.727231 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 85695.8us
I20260709 13:20:10.737987 16143 raft_consensus.cc:1240] T 26354361e9b044cea1c2b86c4882fe92 P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 13:20:10.739619 16117 leader_election.cc:304] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860
I20260709 13:20:10.743325 16151 raft_consensus.cc:2393] T ecc1e363230043fe9fef390f0b979f20 P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 in current term 6: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
I20260709 13:20:10.750560 16265 leader_election.cc:400] T e5ad4eb281564936a59f41e387a47156 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 6 pre-election: Vote denied by peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943) with higher term. Message: Invalid argument: T e5ad4eb281564936a59f41e387a47156 P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 for earlier term 6. Current term is 7.
I20260709 13:20:10.751257 16265 leader_election.cc:403] T e5ad4eb281564936a59f41e387a47156 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 6 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:20:10.713522 18743 raft_consensus.cc:2804] T 7c2095a5f4ad49408dbc7f69c606120d P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:20:10.760994 16143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2794af5b99984efebdee564035841cb0" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:10.761613 16143 raft_consensus.cc:2468] T 2794af5b99984efebdee564035841cb0 P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 5.
I20260709 13:20:10.711299 19147 consensus_queue.cc:1059] T 66f75124215f4ca6bef831e174a792d8 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.047s
I20260709 13:20:10.710915 19148 raft_consensus.cc:515] T 26354361e9b044cea1c2b86c4882fe92 P cb82cc7bd6db46d980c75237a1402860 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:10.765882 16117 leader_election.cc:304] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac
I20260709 13:20:10.755728 18944 raft_consensus.cc:3060] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:20:10.756174 18743 raft_consensus.cc:493] T 7c2095a5f4ad49408dbc7f69c606120d P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:20:10.772732 16143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434bff3082d141838068f323bdb0d1ed" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
W20260709 13:20:10.776000 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 55415.5us
W20260709 13:20:10.778728 19101 log.cc:927] Time spent T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 1.677s	user 0.004s	sys 0.022s
I20260709 13:20:10.775818 18892 raft_consensus.cc:2804] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:20:10.743448 19157 consensus_queue.cc:1048] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.036s
I20260709 13:20:10.804270 18770 leader_election.cc:290] T 746aabf856ab401c9ef5e859371e7b67 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:10.804452 19147 raft_consensus.cc:3055] T 66f75124215f4ca6bef831e174a792d8 P 1ca2725d290e47d68b1a4b609c9816ac [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:20:10.781486 16191 leader_election.cc:304] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: cb82cc7bd6db46d980c75237a1402860
W20260709 13:20:10.806646 16117 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 45779.5us
I20260709 13:20:10.808643 16300 raft_consensus.cc:3055] T a378d7f796f94615b3b347119a9880fd P 90f3da57796b488c9087a2f523adf045 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:20:10.809054 16300 raft_consensus.cc:740] T a378d7f796f94615b3b347119a9880fd P 90f3da57796b488c9087a2f523adf045 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:20:10.809971 16300 consensus_queue.cc:260] T a378d7f796f94615b3b347119a9880fd P 90f3da57796b488c9087a2f523adf045 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:10.771540 18743 raft_consensus.cc:3060] T 7c2095a5f4ad49408dbc7f69c606120d P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:20:10.817441 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 73588.3us
I20260709 13:20:10.818293 16265 leader_election.cc:304] T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:20:10.756369 19153 consensus_queue.cc:1048] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.043s
I20260709 13:20:10.783497 16150 raft_consensus.cc:2468] T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 7.
I20260709 13:20:10.767776 19135 leader_election.cc:290] T 9e65676b21b748728ad39668cfe722ec P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:10.820245 18952 leader_election.cc:290] T 8cc959fb7f704fba9009c0595467d074 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 7 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:10.779346 19091 raft_consensus.cc:3060] T e5ad4eb281564936a59f41e387a47156 P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:20:10.803624 18892 raft_consensus.cc:697] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b [term 6 LEADER]: Becoming Leader. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:20:10.804908 19147 raft_consensus.cc:740] T 66f75124215f4ca6bef831e174a792d8 P 1ca2725d290e47d68b1a4b609c9816ac [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
W20260709 13:20:10.828374 16264 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:10.823123 18892 consensus_queue.cc:237] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:10.827600 16296 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48684 (request call id 2144) took 2535 ms. Trace:
I20260709 13:20:10.831256 16296 rpcz_store.cc:276] 0709 13:20:08.292715 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:08.294831 (+  2116us) service_pool.cc:224] Handling call
0709 13:20:08.295391 (+   560us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:20:08.677127 (+381736us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:08.677154 (+    27us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:20:08.750060 (+ 72906us) log.cc:844] Serialized 52 byte log entry
0709 13:20:08.945768 (+195708us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:20:08.945826 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:08.945927 (+   101us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:20:10.621502 (+1675575us) spinlock_profiling.cc:243] Waited 9.73 ms on lock 0x7b4800130158. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b5a88aa1 00007f69b5a7fb37 00007f69b5a7f9ca 00007f69b5b7726f 00007f69b5b76f37 00007f69b5b8b55c 00007f69b5b8b508 00007f69b5b8b488 00007f69b5b8b443 00007f69b5b8a6ec 00007f69b4ab4a96
0709 13:20:10.776576 (+155074us) spinlock_profiling.cc:243] Waited 15.6 ms on lock 0x7b4400022b4c. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69ad8fadb1 00007f69ad998c85 00007f69ad9996a2 00007f69b070b1db 00007f69b0706aaf 00007f69b06fa86e 00007f69b06f9bb6 00007f69b06f97d8 00007f69b0703221 00007f69b07031d9 00007f69b0703169
0709 13:20:10.827374 (+ 50798us) raft_consensus.cc:1685] finished
0709 13:20:10.827393 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:10.827522 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":275753,"raft.queue_time_us":332166,"raft.run_cpu_time_us":25621,"raft.run_wall_time_us":2005571,"spinlock_wait_cycles":58339968,"thread_start_us":206137,"threads_started":2,"wal-append.queue_time_us":198630}
I20260709 13:20:10.811076 16300 raft_consensus.cc:3060] T a378d7f796f94615b3b347119a9880fd P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Advancing to term 6
I20260709 13:20:10.827502 16142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5d59407eb545b895d8d922e0dd69fd" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:20:10.835007 16142 raft_consensus.cc:3060] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:20:10.829424 16151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b7d1ab519044eda2d5e04a07242f25" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:10.830039 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a9de347c711477e87c582d003f36925" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:20:10.836252 16151 raft_consensus.cc:2468] T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 4.
I20260709 13:20:10.827423 16143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746aabf856ab401c9ef5e859371e7b67" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:20:10.829717 16148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31969392ccdb43b491911a319e8bc7e1" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:20:10.824829 16144 raft_consensus.cc:1240] T 23f82224f4564fea93572e8206bad1fe P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 4. Current term is 5. Ops: [4.5-4.5]
I20260709 13:20:10.829284 16790 leader_election.cc:304] T 2794af5b99984efebdee564035841cb0 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860; no voters: 90f3da57796b488c9087a2f523adf045
W20260709 13:20:10.839596 16192 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 145330us
I20260709 13:20:10.831092 16147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e9b2bf7011f4ae688a79d7bb5a447b4" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:20:10.847474 16191 leader_election.cc:304] T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:20:10.845696 16291 raft_consensus.cc:1275] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:20:10.854795 16150 raft_consensus.cc:1240] T b87147731a68400e8cce9b879daa7c5d P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 7. Current term is 8. Ops: []
I20260709 13:20:10.817490 18551 raft_consensus.cc:2804] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 13:20:10.862926 16117 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:10.860100 16823 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49906 (request call id 634) took 1414 ms. Trace:
I20260709 13:20:10.864629 16823 rpcz_store.cc:276] 0709 13:20:09.439425 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:10.504233 (+1064808us) service_pool.cc:224] Handling call
0709 13:20:10.504573 (+   340us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:20:10.570280 (+ 65707us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:10.570311 (+    31us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:20:10.621633 (+ 51322us) log.cc:844] Serialized 52 byte log entry
0709 13:20:10.773797 (+152164us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:20:10.852921 (+ 79124us) log.cc:844] Serialized 14 byte log entry
0709 13:20:10.853265 (+   344us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:10.853362 (+    97us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:20:10.853419 (+    57us) raft_consensus.cc:1685] finished
0709 13:20:10.853437 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:10.853593 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7912,"prepare.queue_time_us":78465,"raft.queue_time_us":106111,"raft.run_cpu_time_us":200,"raft.run_wall_time_us":26860,"spinlock_wait_cycles":8679296,"thread_start_us":128332,"threads_started":2,"wal-append.queue_time_us":186558}
I20260709 13:20:10.868875 16790 leader_election.cc:304] T 66f75124215f4ca6bef831e174a792d8 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:20:10.847990 19149 raft_consensus.cc:2804] T 2794af5b99984efebdee564035841cb0 P cb82cc7bd6db46d980c75237a1402860 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:20:10.872879 16191 leader_election.cc:304] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
I20260709 13:20:10.869895 16151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26354361e9b044cea1c2b86c4882fe92" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:10.829924 19144 consensus_queue.cc:1059] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.038s
I20260709 13:20:10.875057 16191 leader_election.cc:304] T 8a878999c0024d2abe517484180290fc P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:20:10.875562 16151 raft_consensus.cc:2468] T 26354361e9b044cea1c2b86c4882fe92 P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 5.
I20260709 13:20:10.875756 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434bff3082d141838068f323bdb0d1ed" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:20:10.876188 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5718c623ebee4d2bb770f0f9ddb2f164" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:10.862028 18551 raft_consensus.cc:493] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:10.887444 18551 raft_consensus.cc:3060] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:20:10.889496 16825 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49896 (request call id 928) took 2991 ms. Trace:
I20260709 13:20:10.889873 16825 rpcz_store.cc:276] 0709 13:20:07.898502 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:08.625798 (+727296us) service_pool.cc:224] Handling call
0709 13:20:10.198577 (+1572779us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:10.199283 (+   706us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:10.889287 (+690004us) spinlock_profiling.cc:243] Waited 1.55 s on lock 0x7b5400787150. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5e8ad 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:20:10.889417 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3575345920}
I20260709 13:20:10.875270 18857 raft_consensus.cc:493] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:10.871848 19149 raft_consensus.cc:493] T 2794af5b99984efebdee564035841cb0 P cb82cc7bd6db46d980c75237a1402860 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:20:10.877558 18951 raft_consensus.cc:2749] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:20:10.880897 19148 leader_election.cc:290] T 26354361e9b044cea1c2b86c4882fe92 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:10.878539 19126 raft_consensus.cc:2804] T 66f75124215f4ca6bef831e174a792d8 P cb82cc7bd6db46d980c75237a1402860 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:20:10.882284 19144 consensus_queue.cc:1243] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Peer 647a343a542d4a139fe106c19095129b log is divergent from this leader: its last log entry 7.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 13:20:10.894825 16142 raft_consensus.cc:2468] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 8.
I20260709 13:20:10.886231 18932 leader_election.cc:290] T e5ad4eb281564936a59f41e387a47156 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:10.823179 19147 consensus_queue.cc:260] T 66f75124215f4ca6bef831e174a792d8 P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:10.898489 19126 raft_consensus.cc:697] T 66f75124215f4ca6bef831e174a792d8 P cb82cc7bd6db46d980c75237a1402860 [term 7 LEADER]: Becoming Leader. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:20:10.898715 19147 raft_consensus.cc:3060] T 66f75124215f4ca6bef831e174a792d8 P 1ca2725d290e47d68b1a4b609c9816ac [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:20:10.869391 19163 raft_consensus.cc:2749] T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:20:10.854063 18599 raft_consensus.cc:515] T 3884f864a96d428785895e5463e5728f P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:10.896296 19149 raft_consensus.cc:3060] T 2794af5b99984efebdee564035841cb0 P cb82cc7bd6db46d980c75237a1402860 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:20:10.902309 18599 leader_election.cc:290] T 3884f864a96d428785895e5463e5728f P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 6 election: Requested vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:10.909034 19124 raft_consensus.cc:2749] T 8a878999c0024d2abe517484180290fc P 1ca2725d290e47d68b1a4b609c9816ac [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:20:10.914662 18944 raft_consensus.cc:2749] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:20:10.899569 18857 raft_consensus.cc:515] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:10.917178 18857 leader_election.cc:290] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:10.872169 19010 raft_consensus.cc:515] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:10.914693 16221 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57604 (request call id 2174) took 2123 ms. Trace:
I20260709 13:20:10.919267 19010 leader_election.cc:290] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 8 election: Requested vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:10.919068 16221 rpcz_store.cc:276] 0709 13:20:08.791196 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:08.791423 (+   227us) service_pool.cc:224] Handling call
0709 13:20:08.791895 (+   472us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:20:09.753703 (+961808us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:10.914426 (+1160723us) spinlock_profiling.cc:243] Waited 953 ms on lock 0x7b5400246654. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:20:10.914607 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2191250304}
I20260709 13:20:10.907305 19126 consensus_queue.cc:237] T 66f75124215f4ca6bef831e174a792d8 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:10.923950 16224 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57604 (request call id 2179) took 2056 ms. Trace:
I20260709 13:20:10.924404 16224 rpcz_store.cc:276] 0709 13:20:08.868022 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:08.868346 (+   324us) service_pool.cc:224] Handling call
0709 13:20:08.903626 (+ 35280us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:08.957083 (+ 53457us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:20:08.993941 (+ 36858us) log.cc:844] Serialized 14 byte log entry
0709 13:20:09.328538 (+334597us) log.cc:844] Serialized 14 byte log entry
0709 13:20:09.332777 (+  4239us) log.cc:844] Serialized 14 byte log entry
0709 13:20:09.334115 (+  1338us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:20:09.334167 (+    52us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:20:09.334227 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:10.923611 (+1589384us) spinlock_profiling.cc:243] Waited 25.5 ms on lock 0x7b5400245254. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:20:10.923658 (+    47us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:10.923887 (+   229us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":98284,"prepare.queue_time_us":21403,"prepare.run_wall_time_us":5646,"raft.queue_time_us":102854,"raft.run_cpu_time_us":275,"raft.run_wall_time_us":276,"spinlock_wait_cycles":91192320,"thread_start_us":195435,"threads_started":1,"wal-append.queue_time_us":732324}
W20260709 13:20:10.927003 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 52060.1us
I20260709 13:20:10.927845 16191 leader_election.cc:304] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
I20260709 13:20:10.930236 16295 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48684 (request call id 2150) took 2581 ms. Trace:
I20260709 13:20:10.931344 18599 consensus_queue.cc:1059] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: INVALID_TERM, Last received: 4.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:20:10.932441 18599 consensus_queue.cc:1048] T fffab26427e24d398e2695aad40ca134 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:20:10.932857 18896 raft_consensus.cc:1064] T 19baa7bc367844c288bca8f81439c413 P 1ca2725d290e47d68b1a4b609c9816ac: attempting to promote NON_VOTER cb82cc7bd6db46d980c75237a1402860 to VOTER
I20260709 13:20:10.933818 16191 leader_election.cc:304] T af57996a8be84d398f0f023e8d08ef77 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
I20260709 13:20:10.930882 16115 leader_election.cc:304] T ecc1e363230043fe9fef390f0b979f20 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860
W20260709 13:20:10.937958 16263 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:10.939059 16147 raft_consensus.cc:2468] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 5.
I20260709 13:20:10.930605 16295 rpcz_store.cc:276] 0709 13:20:08.348595 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:08.651715 (+303120us) service_pool.cc:224] Handling call
0709 13:20:08.655498 (+  3783us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:20:08.703390 (+ 47892us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:20:08.703658 (+   268us) log.cc:844] Serialized 14 byte log entry
0709 13:20:09.100734 (+397076us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:20:09.254240 (+153506us) log.cc:844] Serialized 26 byte log entry
0709 13:20:09.268382 (+ 14142us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:20:09.268442 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:09.268536 (+    94us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:20:10.055312 (+786776us) raft_consensus.cc:1685] finished
0709 13:20:10.055334 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:10.473352 (+418018us) spinlock_profiling.cc:243] Waited 45.6 ms on lock 0x7b18002ef3c0. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b693ade1 00007f69b69475b1 00007f69ad9ecc0a 00007f69b712c219 00007f69b4a3358f 00007f69ada72571 00007f69ada747c1 00007f69ada74779 00007f69ada74709 00007f69ada746d1 00007f69ada739cd
0709 13:20:10.930077 (+456725us) spinlock_profiling.cc:243] Waited 3.38 ms on lock 0x7b54002e1b50. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5e8ad 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:20:10.930180 (+   103us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":104767,"prepare.queue_time_us":526,"prepare.run_cpu_time_us":33,"prepare.run_wall_time_us":33,"raft.queue_time_us":25720,"raft.run_cpu_time_us":369,"raft.run_wall_time_us":57150,"spinlock_wait_cycles":112606848,"thread_start_us":391752,"threads_started":1,"wal-append.queue_time_us":1349482}
I20260709 13:20:10.933843 18599 raft_consensus.cc:3055] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:20:10.958141 16300 raft_consensus.cc:1275] T a378d7f796f94615b3b347119a9880fd P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:20:10.894598 18743 raft_consensus.cc:515] T 7c2095a5f4ad49408dbc7f69c606120d P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:10.963128 16295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57a744f5aaf944678a63469cbf514015" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:10.946823 19010 raft_consensus.cc:2749] T af57996a8be84d398f0f023e8d08ef77 P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:20:10.964681 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e9b2bf7011f4ae688a79d7bb5a447b4" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:20:10.956321 18944 raft_consensus.cc:2749] T ecc1e363230043fe9fef390f0b979f20 P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:20:10.970708 19145 raft_consensus.cc:2749] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:20:10.971622 19167 log.cc:927] Time spent T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.054s	user 0.000s	sys 0.001s
I20260709 13:20:10.976338 18939 raft_consensus.cc:2749] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:20:10.977325 16295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31969392ccdb43b491911a319e8bc7e1" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:20:10.956810 18599 raft_consensus.cc:740] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:20:10.977566 19133 raft_consensus.cc:3055] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:20:10.980561 16191 leader_election.cc:304] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:20:10.974347 19149 raft_consensus.cc:515] T 2794af5b99984efebdee564035841cb0 P cb82cc7bd6db46d980c75237a1402860 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:10.969089 19113 raft_consensus.cc:697] T d2ab0ac8347641e6955e9bce7d90c1a2 P 1ca2725d290e47d68b1a4b609c9816ac [term 7 LEADER]: Becoming Leader. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:20:10.992336 16295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18ad7e3af5504571a69d10c00750a01c" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:10.981967 19133 raft_consensus.cc:740] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
W20260709 13:20:10.994724 19092 log.cc:927] Time spent T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 1.841s	user 0.009s	sys 0.013s
I20260709 13:20:10.995225 16822 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49896 (request call id 922) took 3586 ms. Trace:
I20260709 13:20:10.998826 19068 raft_consensus.cc:3060] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:20:10.986369 18944 consensus_queue.cc:1059] T d16dfad0c3b540e5b4b86492ddf87208 P 647a343a542d4a139fe106c19095129b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.023s
I20260709 13:20:11.008945 16222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8158bbd17b994d2eb96f560bd0f1b88d" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:11.009747 16222 raft_consensus.cc:2393] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 7: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
I20260709 13:20:11.009853 18944 raft_consensus.cc:3055] T d16dfad0c3b540e5b4b86492ddf87208 P 647a343a542d4a139fe106c19095129b [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:20:11.010233 18944 raft_consensus.cc:740] T d16dfad0c3b540e5b4b86492ddf87208 P 647a343a542d4a139fe106c19095129b [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:20:11.010835 16219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87ac0d1f7e8a419f9bcbc04aee0e067e" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:11.011137 18944 consensus_queue.cc:260] T d16dfad0c3b540e5b4b86492ddf87208 P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:11.012331 18944 raft_consensus.cc:3060] T d16dfad0c3b540e5b4b86492ddf87208 P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:20:11.001353 18551 raft_consensus.cc:515] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:11.004974 16218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4dc6b0541ca431ebc757beac00521f1" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:11.007433 16223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68a6838d82ef4cac83507f906139d90f" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:11.007874 18932 raft_consensus.cc:493] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:11.008198 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5ad4eb281564936a59f41e387a47156" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
W20260709 13:20:10.999058 16117 leader_election.cc:336] T af57996a8be84d398f0f023e8d08ef77 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 7 election: RPC error from VoteRequest() call to peer cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923): Timed out: RequestConsensusVote RPC to 127.14.177.132:38923 timed out after 1.899s (SENT)
I20260709 13:20:11.008569 16822 rpcz_store.cc:276] 0709 13:20:07.409357 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:08.589730 (+1180373us) service_pool.cc:224] Handling call
0709 13:20:10.236177 (+1646447us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:20:10.266400 (+ 30223us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:10.995022 (+728622us) spinlock_profiling.cc:243] Waited 1.57 s on lock 0x7b54006eebd0. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5e8ad 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:20:10.995145 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3601276160}
W20260709 13:20:11.017341 16192 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:11.022330 18932 raft_consensus.cc:515] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:11.023149 16226 raft_consensus.cc:1275] T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 7 index: 3. (term mismatch)
I20260709 13:20:11.023550 16226 pending_rounds.cc:85] T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac: Aborting all ops after (but not including) 2
I20260709 13:20:11.023801 16226 pending_rounds.cc:107] T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac: Aborting uncommitted NO_OP operation due to leader change: 6.3
I20260709 13:20:11.024027 16226 raft_consensus.cc:2889] T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:20:11.024863 16116 leader_election.cc:304] T b6f07842d3aa47858df5665874c86f3a P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
I20260709 13:20:11.025120 16221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02838c86531d4f98bf58d1c22b8baf77" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:11.025817 16221 raft_consensus.cc:2468] T 02838c86531d4f98bf58d1c22b8baf77 P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 7.
I20260709 13:20:11.025887 19042 raft_consensus.cc:2804] T b6f07842d3aa47858df5665874c86f3a P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:20:11.014271 19172 raft_consensus.cc:2749] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:20:11.025585 18599 consensus_queue.cc:260] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.5, Last appended by leader: 5, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } }
I20260709 13:20:10.992075 19091 raft_consensus.cc:2749] T e5ad4eb281564936a59f41e387a47156 P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Leader pre-election lost for term 6. Reason: Vote denied by peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943) with higher term. Message: Invalid argument: T e5ad4eb281564936a59f41e387a47156 P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 for earlier term 6. Current term is 7.
I20260709 13:20:11.031368 16225 raft_consensus.cc:2468] T e5ad4eb281564936a59f41e387a47156 P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 7.
I20260709 13:20:11.032021 16295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3884f864a96d428785895e5463e5728f" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:20:11.030030 19113 consensus_queue.cc:237] T d2ab0ac8347641e6955e9bce7d90c1a2 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:11.004810 18896 consensus_queue.cc:237] T 19baa7bc367844c288bca8f81439c413 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 5, Current term: 7, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:20:11.036717 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 56238.9us
I20260709 13:20:11.036031 16295 raft_consensus.cc:3060] T 3884f864a96d428785895e5463e5728f P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:20:11.039525 16292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8158bbd17b994d2eb96f560bd0f1b88d" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:11.030934 18892 consensus_queue.cc:1059] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true }, Status: INVALID_TERM, Last received: 5.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.010s
I20260709 13:20:11.041214 16292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e65676b21b748728ad39668cfe722ec" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:11.041332 18892 consensus_queue.cc:1243] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Peer 1ca2725d290e47d68b1a4b609c9816ac log is divergent from this leader: its last log entry 6.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 13:20:11.041797 16292 raft_consensus.cc:2468] T 9e65676b21b748728ad39668cfe722ec P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 7.
I20260709 13:20:11.042670 16292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5ad4eb281564936a59f41e387a47156" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:11.044799 18551 leader_election.cc:290] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 8 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:11.046746 19149 leader_election.cc:290] T 2794af5b99984efebdee564035841cb0 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 6 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:11.048545 19133 consensus_queue.cc:260] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:11.053766 16191 leader_election.cc:304] T 02838c86531d4f98bf58d1c22b8baf77 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:20:11.058029 19147 raft_consensus.cc:2749] T 02838c86531d4f98bf58d1c22b8baf77 P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:20:11.056011 16292 raft_consensus.cc:2468] T e5ad4eb281564936a59f41e387a47156 P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 7.
I20260709 13:20:11.031219 18932 leader_election.cc:290] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:11.063958 16191 leader_election.cc:304] T e5ad4eb281564936a59f41e387a47156 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:20:11.070815 18901 raft_consensus.cc:2749] T e5ad4eb281564936a59f41e387a47156 P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:20:11.074419 16145 raft_consensus.cc:1275] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:20:11.074586 16294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87147731a68400e8cce9b879daa7c5d" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:20:11.069273 18743 leader_election.cc:290] T 7c2095a5f4ad49408dbc7f69c606120d P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 6 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:20:11.075256 16294 raft_consensus.cc:2376] T b87147731a68400e8cce9b879daa7c5d P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 647a343a542d4a139fe106c19095129b in term 8. Re-sending same reply.
I20260709 13:20:11.075846 19133 raft_consensus.cc:3060] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:20:11.076061 16300 raft_consensus.cc:2410] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 for term 5 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:20:11.076210 16294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:11.078296 16294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6f07842d3aa47858df5665874c86f3a" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:11.078858 16294 raft_consensus.cc:2468] T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 6.
I20260709 13:20:11.079651 16294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26354361e9b044cea1c2b86c4882fe92" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:11.080621 16294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e4bfb9090c8460eb14fa7e0d818e7f1" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:20:11.081091 16294 raft_consensus.cc:3060] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:20:11.083125 16117 leader_election.cc:304] T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860
W20260709 13:20:11.082996 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 56335.5us
I20260709 13:20:11.087016 16142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1df93779209748b9860f6d893dd3de53" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:20:11.087491 16819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2794af5b99984efebdee564035841cb0" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
W20260709 13:20:11.088145 19003 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 9 current term: 9 is higher.
I20260709 13:20:11.088163 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ad5dbfa99ee4b8d9e28be92b3f36db5" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:20:11.088960 16824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57a744f5aaf944678a63469cbf514015" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:11.089730 16145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0eab25ce4c64e018a041beeda2867db" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:11.087509 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee9ed67006144797b1de6ea8c2013a68" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:11.095297 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c74490c40243a88e62d1760b5cd814" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:11.096767 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2794af5b99984efebdee564035841cb0" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:11.090313 16145 raft_consensus.cc:2468] T c0eab25ce4c64e018a041beeda2867db P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 7.
I20260709 13:20:11.097908 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87ac0d1f7e8a419f9bcbc04aee0e067e" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:11.099238 16150 raft_consensus.cc:2468] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 6.
W20260709 13:20:11.100123 16264 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:11.102265 16790 leader_election.cc:304] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860; no voters: 
W20260709 13:20:11.103900 16114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:11.105050 16295 raft_consensus.cc:2468] T 3884f864a96d428785895e5463e5728f P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 6.
I20260709 13:20:11.105858 16295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f82224f4564fea93572e8206bad1fe" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:20:11.106496 16295 raft_consensus.cc:2393] T 23f82224f4564fea93572e8206bad1fe P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 5: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
I20260709 13:20:11.106385 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f2da98864b4425bf7c0055951fdd4b" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:11.108764 16299 raft_consensus.cc:2393] T 84f2da98864b4425bf7c0055951fdd4b P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 5: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
I20260709 13:20:11.113801 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a96373bb0e14dce8a266b2fc6c58507" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 6 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:11.114326 16142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d4ab870a05347868956aa8b841b2981" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:11.084091 19042 raft_consensus.cc:697] T b6f07842d3aa47858df5665874c86f3a P 647a343a542d4a139fe106c19095129b [term 7 LEADER]: Becoming Leader. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:20:11.116742 16790 leader_election.cc:304] T 26354361e9b044cea1c2b86c4882fe92 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:20:11.121065 16790 leader_election.cc:304] T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:20:11.116998 16825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee9ed67006144797b1de6ea8c2013a68" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:11.118018 16823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2372981006d4b218215c28964dea623" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:11.122483 16151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d16dfad0c3b540e5b4b86492ddf87208" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:11.122483 16819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e65676b21b748728ad39668cfe722ec" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:11.123203 16151 raft_consensus.cc:2468] T d16dfad0c3b540e5b4b86492ddf87208 P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 6.
I20260709 13:20:11.124819 16145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f2da98864b4425bf7c0055951fdd4b" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:20:11.125379 16145 raft_consensus.cc:3060] T 84f2da98864b4425bf7c0055951fdd4b P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:20:11.125507 16142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2794af5b99984efebdee564035841cb0" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:20:11.126053 16142 raft_consensus.cc:3060] T 2794af5b99984efebdee564035841cb0 P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:20:11.128589 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6f07842d3aa47858df5665874c86f3a" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:20:11.129737 16299 raft_consensus.cc:3060] T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:20:11.146857 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f5f89186204242bf377f7c42b2e2da" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:20:11.203328 16149 raft_consensus.cc:3060] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Advancing to term 9
I20260709 13:20:11.148422 16295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68a6838d82ef4cac83507f906139d90f" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:11.124598 18999 raft_consensus.cc:2804] T 26354361e9b044cea1c2b86c4882fe92 P cb82cc7bd6db46d980c75237a1402860 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:20:11.187515 16151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806d7690c2b644959152074bb2cab0a7" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
W20260709 13:20:11.194279 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 109783us
I20260709 13:20:11.114879 19149 consensus_queue.cc:1059] T 23f82224f4564fea93572e8206bad1fe P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.061s
I20260709 13:20:11.201279 16823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3884f864a96d428785895e5463e5728f" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:20:11.206424 16823 raft_consensus.cc:3060] T 3884f864a96d428785895e5463e5728f P cb82cc7bd6db46d980c75237a1402860 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:20:11.113976 19003 raft_consensus.cc:2804] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:20:11.202361 16822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26354361e9b044cea1c2b86c4882fe92" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:11.115971 18901 raft_consensus.cc:493] T ecc1e363230043fe9fef390f0b979f20 P 1ca2725d290e47d68b1a4b609c9816ac [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:11.146339 16819 raft_consensus.cc:2393] T 9e65676b21b748728ad39668cfe722ec P cb82cc7bd6db46d980c75237a1402860 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 7: Already voted for candidate cb82cc7bd6db46d980c75237a1402860 in this term.
I20260709 13:20:11.122186 19146 consensus_queue.cc:1059] T b87147731a68400e8cce9b879daa7c5d P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.074s
I20260709 13:20:11.208912 16299 raft_consensus.cc:2468] T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 7.
W20260709 13:20:11.224478 16117 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 141415us
I20260709 13:20:11.225879 16142 raft_consensus.cc:2468] T 2794af5b99984efebdee564035841cb0 P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 6.
I20260709 13:20:11.225167 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a9de347c711477e87c582d003f36925" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:20:11.217597 16294 raft_consensus.cc:2410] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 for term 6 because replica has last-logged OpId of term: 5 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 5.
I20260709 13:20:11.226570 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66f75124215f4ca6bef831e174a792d8" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:20:11.209926 19003 raft_consensus.cc:493] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:20:11.230548 18901 raft_consensus.cc:515] T ecc1e363230043fe9fef390f0b979f20 P 1ca2725d290e47d68b1a4b609c9816ac [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:11.203204 19144 raft_consensus.cc:2804] T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:20:11.238165 16295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f5f89186204242bf377f7c42b2e2da" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:11.243048 16295 raft_consensus.cc:2393] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 8: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
I20260709 13:20:11.206230 18999 raft_consensus.cc:493] T 26354361e9b044cea1c2b86c4882fe92 P cb82cc7bd6db46d980c75237a1402860 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:11.240468 16142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cc959fb7f704fba9009c0595467d074" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:20:11.240936 16294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02838c86531d4f98bf58d1c22b8baf77" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:11.242909 16223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a378d7f796f94615b3b347119a9880fd" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:11.241530 16790 leader_election.cc:304] T 2794af5b99984efebdee564035841cb0 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:20:11.249830 16294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434bff3082d141838068f323bdb0d1ed" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:20:11.250617 16294 raft_consensus.cc:3060] T 434bff3082d141838068f323bdb0d1ed P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:20:11.251569 16142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746aabf856ab401c9ef5e859371e7b67" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
W20260709 13:20:11.252592 16264 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:11.253379 16142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5d59407eb545b895d8d922e0dd69fd" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:20:11.235680 16817 raft_consensus.cc:1240] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860 [term 6 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 5. Current term is 6. Ops: []
W20260709 13:20:11.255434 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 49479.9us
I20260709 13:20:11.256044 16142 raft_consensus.cc:2393] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 8: Already voted for candidate cb82cc7bd6db46d980c75237a1402860 in this term.
I20260709 13:20:11.258976 16221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 6 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:20:11.265161 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4dc6b0541ca431ebc757beac00521f1" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:11.241786 19003 raft_consensus.cc:3060] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:20:11.278735 16142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a959dbecc2648eb85d1cc550f5a3245" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:11.278900 16295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a878999c0024d2abe517484180290fc" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:11.279867 16299 raft_consensus.cc:2410] T e4dc6b0541ca431ebc757beac00521f1 P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b for term 7 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 13:20:11.280833 16824 raft_consensus.cc:1275] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 [term 7 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 7. (term mismatch)
I20260709 13:20:11.281279 16824 pending_rounds.cc:85] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860: Aborting all ops after (but not including) 6
I20260709 13:20:11.281430 16221 raft_consensus.cc:2393] T ecc1e363230043fe9fef390f0b979f20 P 1ca2725d290e47d68b1a4b609c9816ac [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 in current term 6: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
I20260709 13:20:11.281623 16824 pending_rounds.cc:107] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860: Aborting uncommitted NO_OP operation due to leader change: 6.7
I20260709 13:20:11.247766 18999 raft_consensus.cc:3060] T 26354361e9b044cea1c2b86c4882fe92 P cb82cc7bd6db46d980c75237a1402860 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:20:11.281955 16824 raft_consensus.cc:2889] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:20:11.282891 16223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e4bfb9090c8460eb14fa7e0d818e7f1" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:11.284590 16294 raft_consensus.cc:2468] T 434bff3082d141838068f323bdb0d1ed P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 7.
W20260709 13:20:11.296289 16264 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:20:11.300418 16117 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 75171.1us
I20260709 13:20:11.304425 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8638ee363273426db1393dd81e5c7b72" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:20:11.305553 16226 raft_consensus.cc:2410] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 for term 6 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 13:20:11.306826 16299 raft_consensus.cc:1240] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 5. Current term is 6. Ops: []
W20260709 13:20:11.308441 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 66968.7us
W20260709 13:20:11.308923 16264 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:11.314011 16823 raft_consensus.cc:2468] T 3884f864a96d428785895e5463e5728f P cb82cc7bd6db46d980c75237a1402860 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 6.
I20260709 13:20:11.316304 16223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746aabf856ab401c9ef5e859371e7b67" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
W20260709 13:20:11.337785 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 76870.9us
I20260709 13:20:11.352126 16299 raft_consensus.cc:1240] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 5. Current term is 6. Ops: []
I20260709 13:20:11.358356 16149 raft_consensus.cc:2468] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 9.
I20260709 13:20:11.359314 16226 raft_consensus.cc:1217] T af57996a8be84d398f0f023e8d08ef77 P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Deduplicated request from leader. Original: 2.5->[4.6-6.7]   Dedup: 4.6->[6.7-6.7]
I20260709 13:20:11.362018 16223 raft_consensus.cc:2468] T 746aabf856ab401c9ef5e859371e7b67 P 1ca2725d290e47d68b1a4b609c9816ac [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 6.
I20260709 13:20:11.363762 18932 consensus_queue.cc:1059] T 434bff3082d141838068f323bdb0d1ed P 647a343a542d4a139fe106c19095129b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:20:11.388597 16145 raft_consensus.cc:2468] T 84f2da98864b4425bf7c0055951fdd4b P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 5.
I20260709 13:20:11.367707 16226 raft_consensus.cc:1240] T af57996a8be84d398f0f023e8d08ef77 P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 6. Current term is 7. Ops: [4.6-6.7]
I20260709 13:20:11.399134 18114 consensus_queue.cc:1048] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:20:11.399609 16790 leader_election.cc:304] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:20:11.403155 16819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746aabf856ab401c9ef5e859371e7b67" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:11.403849 16819 raft_consensus.cc:2468] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 6.
I20260709 13:20:11.291548 18857 consensus_queue.cc:1059] T 1df93779209748b9860f6d893dd3de53 P 647a343a542d4a139fe106c19095129b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: INVALID_TERM, Last received: 4.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.062s
I20260709 13:20:11.406396 16192 leader_election.cc:304] T 746aabf856ab401c9ef5e859371e7b67 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:20:11.415354 16819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada5a004ba7c46819051e3ccec71ffdb" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:20:11.416119 16819 raft_consensus.cc:3060] T ada5a004ba7c46819051e3ccec71ffdb P cb82cc7bd6db46d980c75237a1402860 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:20:11.419665 16299 raft_consensus.cc:1240] T 434bff3082d141838068f323bdb0d1ed P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 6. Current term is 7. Ops: []
I20260709 13:20:11.368009 19133 raft_consensus.cc:2804] T 2794af5b99984efebdee564035841cb0 P cb82cc7bd6db46d980c75237a1402860 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:20:11.426498 16144 raft_consensus.cc:1275] T d16dfad0c3b540e5b4b86492ddf87208 P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:20:11.432091 16292 raft_consensus.cc:1275] T 8d4ab870a05347868956aa8b841b2981 P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 5 index: 2. Preceding OpId from leader: term: 7 index: 3. (index mismatch)
I20260709 13:20:11.300853 19042 consensus_queue.cc:237] T b6f07842d3aa47858df5665874c86f3a P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:11.447386 16223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b7d1ab519044eda2d5e04a07242f25" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
W20260709 13:20:11.430879 19180 log.cc:927] Time spent T e4dc6b0541ca431ebc757beac00521f1 P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 0.196s	user 0.001s	sys 0.001s
W20260709 13:20:11.454425 16192 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:11.435585 19147 raft_consensus.cc:2804] T 746aabf856ab401c9ef5e859371e7b67 P 1ca2725d290e47d68b1a4b609c9816ac [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:20:11.459955 16144 pending_rounds.cc:85] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b: Aborting all ops after (but not including) 4
I20260709 13:20:11.429723 19144 raft_consensus.cc:697] T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860 [term 8 LEADER]: Becoming Leader. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:20:11.472450 16223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdbd78acb4cf46298e0de8ba0d130e19" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:11.457645 16817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5d59407eb545b895d8d922e0dd69fd" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:20:11.463748 19147 raft_consensus.cc:493] T 746aabf856ab401c9ef5e859371e7b67 P 1ca2725d290e47d68b1a4b609c9816ac [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:20:11.485571 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 125004us
I20260709 13:20:11.490439 16225 raft_consensus.cc:2468] T 66f75124215f4ca6bef831e174a792d8 P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 7.
I20260709 13:20:11.496238 16146 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40844 (request call id 1987) took 2930 ms. Trace:
I20260709 13:20:11.466050 16144 pending_rounds.cc:107] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b: Aborting uncommitted CHANGE_CONFIG_OP operation due to leader change: 4.5
W20260709 13:20:11.497653 16114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:11.502760 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5d59407eb545b895d8d922e0dd69fd" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:20:11.503780 16226 raft_consensus.cc:2393] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 in current term 8: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
I20260709 13:20:11.499282 16146 rpcz_store.cc:276] 0709 13:20:08.557552 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:08.561513 (+  3961us) service_pool.cc:224] Handling call
0709 13:20:08.561856 (+   343us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:09.704287 (+1142431us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:11.487120 (+1782833us) spinlock_profiling.cc:243] Waited 1.12 s on lock 0x7b540028e454. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:20:11.487507 (+   387us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2566537600}
I20260709 13:20:11.507069 16144 raft_consensus.cc:2917] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Aborting config change with OpId 4.5: Aborted: Op aborted by new leader
I20260709 13:20:11.477245 19183 raft_consensus.cc:493] T 9e65676b21b748728ad39668cfe722ec P cb82cc7bd6db46d980c75237a1402860 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:20:11.509022 16189 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:11.508280 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e4bfb9090c8460eb14fa7e0d818e7f1" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:20:11.513923 16144 raft_consensus.cc:2889] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: CHANGE_CONFIG_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:20:11.536052 16790 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:11.506145 16116 leader_election.cc:304] T 02838c86531d4f98bf58d1c22b8baf77 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
I20260709 13:20:11.497637 19147 raft_consensus.cc:3060] T 746aabf856ab401c9ef5e859371e7b67 P 1ca2725d290e47d68b1a4b609c9816ac [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:20:11.504117 19192 raft_consensus.cc:2804] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:20:11.594165 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5718c623ebee4d2bb770f0f9ddb2f164" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:11.598726 16293 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48674 (request call id 2235) took 1516 ms. Trace:
I20260709 13:20:11.601665 16293 rpcz_store.cc:276] 0709 13:20:10.089283 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:10.101233 (+ 11950us) service_pool.cc:224] Handling call
0709 13:20:10.101656 (+   423us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:10.203419 (+101763us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:11.598386 (+1394967us) spinlock_profiling.cc:243] Waited 97.7 ms on lock 0x7b5400353ed4. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:20:11.598649 (+   263us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":224781696}
I20260709 13:20:11.613037 16226 raft_consensus.cc:2468] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 6.
I20260709 13:20:11.613524 16046 catalog_manager.cc:5697] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 reported cstate change: term changed from 2 to 5. New cstate: current_term: 5 leader_uuid: "cb82cc7bd6db46d980c75237a1402860" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:20:11.617913 16117 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 313277us
W20260709 13:20:11.629914 16264 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:20:11.623049 19112 log.cc:927] Time spent T 8cc959fb7f704fba9009c0595467d074 P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 1.823s	user 0.013s	sys 0.008s
I20260709 13:20:11.635138 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7ca61592e848ea9b0fb81a2ce33cdc" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:11.640599 16790 leader_election.cc:304] T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:20:11.550437 19183 raft_consensus.cc:515] T 9e65676b21b748728ad39668cfe722ec P cb82cc7bd6db46d980c75237a1402860 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:11.619460 19192 raft_consensus.cc:697] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860 [term 5 LEADER]: Becoming Leader. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:20:11.622430 19176 raft_consensus.cc:2749] T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:20:11.647316 19197 consensus_queue.cc:1059] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.020s
W20260709 13:20:11.648098 19120 log.cc:927] Time spent T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 1.850s	user 0.002s	sys 0.004s
I20260709 13:20:11.666646 16217 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57604 (request call id 2181) took 2710 ms. Trace:
I20260709 13:20:11.667137 16217 rpcz_store.cc:276] 0709 13:20:08.956401 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:08.970535 (+ 14134us) service_pool.cc:224] Handling call
0709 13:20:08.971043 (+   508us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:20:09.026350 (+ 55307us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:09.026393 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:20:09.100820 (+ 74427us) log.cc:844] Serialized 26 byte log entry
0709 13:20:09.703963 (+603143us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:20:09.704016 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:11.396645 (+1692629us) spinlock_profiling.cc:243] Waited 37.3 ms on lock 0x7b540023fad4. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:20:11.396703 (+    58us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:20:11.520828 (+124125us) spinlock_profiling.cc:243] Waited 6.75 ms on lock 0x7b4400022b4c. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69ad8fadb1 00007f69ad998c85 00007f69ad9996a2 00007f69b070b1db 00007f69b0706aaf 00007f69b06fa86e 00007f69b06f9bb6 00007f69b06f97d8 00007f69b0703221 00007f69b07031d9 00007f69b0703169
0709 13:20:11.666180 (+145352us) raft_consensus.cc:1685] finished
0709 13:20:11.666221 (+    41us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:11.666438 (+   217us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3882,"raft.queue_time_us":14428,"raft.run_cpu_time_us":142,"raft.run_wall_time_us":15334,"spinlock_wait_cycles":101325824,"thread_start_us":417682,"threads_started":1,"wal-append.queue_time_us":632217}
I20260709 13:20:11.670718 16223 raft_consensus.cc:1240] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 6. Current term is 7. Ops: [6.7-6.7]
I20260709 13:20:11.670131 19003 raft_consensus.cc:515] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:11.673614 19003 leader_election.cc:290] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 5 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:20:11.652313 18587 raft_consensus.cc:493] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:20:11.676091 18587 raft_consensus.cc:515] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:11.678748 18587 leader_election.cc:290] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:11.655355 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e65676b21b748728ad39668cfe722ec" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:11.686012 16150 raft_consensus.cc:2468] T 9e65676b21b748728ad39668cfe722ec P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 6.
I20260709 13:20:11.621119 19144 consensus_queue.cc:237] T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:11.653630 16218 raft_consensus.cc:1275] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 9. (index mismatch)
I20260709 13:20:11.648155 16117 leader_election.cc:304] T ee9ed67006144797b1de6ea8c2013a68 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860
I20260709 13:20:11.655493 16790 leader_election.cc:304] T 8638ee363273426db1393dd81e5c7b72 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:20:11.690196 16819 raft_consensus.cc:2468] T ada5a004ba7c46819051e3ccec71ffdb P cb82cc7bd6db46d980c75237a1402860 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 8.
I20260709 13:20:11.705861 16190 leader_election.cc:304] T 8c11074579b046d99e585c2818e18a2c P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860
W20260709 13:20:11.711593 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 56197.4us
W20260709 13:20:11.712008 19194 log.cc:927] Time spent T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 0.163s	user 0.001s	sys 0.000s
I20260709 13:20:11.655891 18901 leader_election.cc:290] T ecc1e363230043fe9fef390f0b979f20 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:11.727319 16190 leader_election.cc:304] T c0eab25ce4c64e018a041beeda2867db P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:20:11.712334 19149 raft_consensus.cc:2749] T 8638ee363273426db1393dd81e5c7b72 P cb82cc7bd6db46d980c75237a1402860 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
W20260709 13:20:11.714519 16117 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 66416.9us
I20260709 13:20:11.650480 19183 leader_election.cc:290] T 9e65676b21b748728ad39668cfe722ec P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:11.716100 19200 consensus_queue.cc:1048] T 8d4ab870a05347868956aa8b841b2981 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:20:11.717554 19173 log.cc:927] Time spent T 66f75124215f4ca6bef831e174a792d8 P cb82cc7bd6db46d980c75237a1402860: Append to log took a long time: real 0.710s	user 0.002s	sys 0.000s
I20260709 13:20:11.725010 16790 leader_election.cc:304] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 647a343a542d4a139fe106c19095129b
I20260709 13:20:11.729092 16190 leader_election.cc:304] T 5a96373bb0e14dce8a266b2fc6c58507 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:20:11.738284 16790 leader_election.cc:304] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:20:11.734761 19153 raft_consensus.cc:2804] T c0eab25ce4c64e018a041beeda2867db P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 13:20:11.768013 16264 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:11.739578 19133 raft_consensus.cc:697] T 2794af5b99984efebdee564035841cb0 P cb82cc7bd6db46d980c75237a1402860 [term 6 LEADER]: Becoming Leader. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:20:11.736344 18587 raft_consensus.cc:2804] T 5a96373bb0e14dce8a266b2fc6c58507 P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:20:11.791971 16790 leader_election.cc:304] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 647a343a542d4a139fe106c19095129b
I20260709 13:20:11.740708 18857 raft_consensus.cc:2804] T 02838c86531d4f98bf58d1c22b8baf77 P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:20:11.743561 19003 raft_consensus.cc:2804] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860 [term 6 FOLLOWER]: Leader election won for term 6
W20260709 13:20:11.794637 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 288567us
W20260709 13:20:11.812409 16117 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 83351.2us
I20260709 13:20:11.787420 19153 raft_consensus.cc:493] T c0eab25ce4c64e018a041beeda2867db P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:11.788617 19149 raft_consensus.cc:2749] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:20:11.841087 16264 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:11.817314 16116 leader_election.cc:304] T e5ad4eb281564936a59f41e387a47156 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
I20260709 13:20:11.739276 18999 raft_consensus.cc:515] T 26354361e9b044cea1c2b86c4882fe92 P cb82cc7bd6db46d980c75237a1402860 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:11.823812 16148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b7d1ab519044eda2d5e04a07242f25" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:20:11.798808 18857 raft_consensus.cc:493] T 02838c86531d4f98bf58d1c22b8baf77 P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:11.799957 18587 raft_consensus.cc:493] T 5a96373bb0e14dce8a266b2fc6c58507 P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Starting leader election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:20:11.793886 19198 raft_consensus.cc:3055] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b [term 7 LEADER]: Stepping down as leader of term 7
W20260709 13:20:11.822767 19146 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.591s	user 0.020s	sys 0.010s
I20260709 13:20:11.803248 18932 raft_consensus.cc:2749] T ee9ed67006144797b1de6ea8c2013a68 P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:20:11.862416 16115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 185321us
I20260709 13:20:11.867576 16822 raft_consensus.cc:2393] T 26354361e9b044cea1c2b86c4882fe92 P cb82cc7bd6db46d980c75237a1402860 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 6: Already voted for candidate cb82cc7bd6db46d980c75237a1402860 in this term.
I20260709 13:20:11.842298 19153 raft_consensus.cc:3060] T c0eab25ce4c64e018a041beeda2867db P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:20:11.861896 19198 raft_consensus.cc:740] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
W20260709 13:20:11.867727 16115 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:20:11.819764 18944 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.537s	user 0.001s	sys 0.012s
I20260709 13:20:11.819038 19003 raft_consensus.cc:697] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860 [term 6 LEADER]: Becoming Leader. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:20:11.849665 18896 raft_consensus.cc:2749] T 8c11074579b046d99e585c2818e18a2c P 1ca2725d290e47d68b1a4b609c9816ac [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:20:11.860576 18857 raft_consensus.cc:3060] T 02838c86531d4f98bf58d1c22b8baf77 P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:20:11.863118 18587 raft_consensus.cc:3060] T 5a96373bb0e14dce8a266b2fc6c58507 P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:20:11.865976 19146 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.634s	user 0.020s	sys 0.010s
W20260709 13:20:11.873865 19175 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.502s	user 0.007s	sys 0.003s
I20260709 13:20:11.869205 18599 raft_consensus.cc:3060] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:20:11.879632 18944 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.598s	user 0.002s	sys 0.012s
W20260709 13:20:11.908586 16264 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:20:11.896350 19175 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.605s	user 0.009s	sys 0.009s
I20260709 13:20:11.824817 19192 consensus_queue.cc:237] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:20:11.936398 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 119159us
I20260709 13:20:11.884339 19209 consensus_queue.cc:1059] T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.100s
I20260709 13:20:11.975622 19149 raft_consensus.cc:493] T 829b4ad8c5a54eababeea1181b960d4b P cb82cc7bd6db46d980c75237a1402860 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:20:11.975793 19209 raft_consensus.cc:3055] T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:20:11.976292 19209 raft_consensus.cc:740] T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:20:11.976193 19149 raft_consensus.cc:515] T 829b4ad8c5a54eababeea1181b960d4b P cb82cc7bd6db46d980c75237a1402860 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:11.977665 16150 raft_consensus.cc:1240] T 23f82224f4564fea93572e8206bad1fe P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 4. Current term is 5. Ops: [4.5-4.5]
I20260709 13:20:11.977294 19209 consensus_queue.cc:260] T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:11.938891 16142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26354361e9b044cea1c2b86c4882fe92" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:20:11.978624 19209 raft_consensus.cc:3060] T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:20:11.944344 19126 raft_consensus.cc:3055] T 23f82224f4564fea93572e8206bad1fe P cb82cc7bd6db46d980c75237a1402860 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:20:11.979369 19126 raft_consensus.cc:740] T 23f82224f4564fea93572e8206bad1fe P cb82cc7bd6db46d980c75237a1402860 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:20:11.980238 19126 consensus_queue.cc:260] T 23f82224f4564fea93572e8206bad1fe P cb82cc7bd6db46d980c75237a1402860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.5, Last appended by leader: 5, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:11.981307 19126 raft_consensus.cc:3060] T 23f82224f4564fea93572e8206bad1fe P cb82cc7bd6db46d980c75237a1402860 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:20:11.981412 19204 log.cc:927] Time spent T d2ab0ac8347641e6955e9bce7d90c1a2 P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 0.083s	user 0.000s	sys 0.000s
I20260709 13:20:11.945166 19187 raft_consensus.cc:2804] T e5ad4eb281564936a59f41e387a47156 P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:20:11.983378 19187 raft_consensus.cc:493] T e5ad4eb281564936a59f41e387a47156 P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:11.983767 19187 raft_consensus.cc:3060] T e5ad4eb281564936a59f41e387a47156 P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:20:11.984881 16191 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:11.986196 16191 leader_election.cc:304] T ee9ed67006144797b1de6ea8c2013a68 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
I20260709 13:20:11.904333 19133 consensus_queue.cc:237] T 2794af5b99984efebdee564035841cb0 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:11.988878 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "829b4ad8c5a54eababeea1181b960d4b" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:11.989611 16150 raft_consensus.cc:2468] T 829b4ad8c5a54eababeea1181b960d4b P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 6.
I20260709 13:20:11.990379 19040 consensus_queue.cc:1048] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:20:11.978843 16142 raft_consensus.cc:3060] T 26354361e9b044cea1c2b86c4882fe92 P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:20:11.991271 16117 leader_election.cc:304] T 3884f864a96d428785895e5463e5728f P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:20:11.945539 19188 raft_consensus.cc:2804] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:20:11.954212 16143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 7 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:11.954453 16300 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40474 (request call id 433) took 1026 ms. Trace:
I20260709 13:20:11.952459 19185 raft_consensus.cc:3055] T b87147731a68400e8cce9b879daa7c5d P cb82cc7bd6db46d980c75237a1402860 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:20:11.961429 19003 consensus_queue.cc:237] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:11.966266 18999 leader_election.cc:290] T 26354361e9b044cea1c2b86c4882fe92 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 6 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:11.886586 19198 consensus_queue.cc:260] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:11.978374 19149 leader_election.cc:290] T 829b4ad8c5a54eababeea1181b960d4b P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
W20260709 13:20:11.991583 16189 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:11.993821 16143 raft_consensus.cc:2468] T ecc1e363230043fe9fef390f0b979f20 P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 6.
I20260709 13:20:11.993736 16300 rpcz_store.cc:276] 0709 13:20:10.928290 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:10.964370 (+ 36080us) service_pool.cc:224] Handling call
0709 13:20:11.954228 (+989858us) spinlock_profiling.cc:243] Waited 108 ms on lock 0x7b540024ca54. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:20:11.954389 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":248429824}
W20260709 13:20:11.957479 19186 log.cc:927] Time spent T 19baa7bc367844c288bca8f81439c413 P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 0.551s	user 0.000s	sys 0.003s
I20260709 13:20:11.999130 16191 leader_election.cc:304] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
I20260709 13:20:12.009550 18114 raft_consensus.cc:2749] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:20:11.998360 18551 raft_consensus.cc:2749] T ee9ed67006144797b1de6ea8c2013a68 P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:20:12.021109 16191 leader_election.cc:304] T 5a959dbecc2648eb85d1cc550f5a3245 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
W20260709 13:20:12.018796 16790 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:11.998675 19188 raft_consensus.cc:493] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:11.999413 19185 raft_consensus.cc:740] T b87147731a68400e8cce9b879daa7c5d P cb82cc7bd6db46d980c75237a1402860 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:20:12.035504 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ad5dbfa99ee4b8d9e28be92b3f36db5" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:12.041934 16191 leader_election.cc:304] T c2f5f89186204242bf377f7c42b2e2da P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
I20260709 13:20:12.053193 16224 raft_consensus.cc:1240] T 1df93779209748b9860f6d893dd3de53 P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 6. Current term is 7. Ops: [6.5-6.5]
I20260709 13:20:12.049798 16191 leader_election.cc:304] T 84f2da98864b4425bf7c0055951fdd4b P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
I20260709 13:20:12.056915 18114 raft_consensus.cc:2804] T 84f2da98864b4425bf7c0055951fdd4b P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:20:12.062125 16221 raft_consensus.cc:1240] T 8cc959fb7f704fba9009c0595467d074 P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 6. Current term is 7. Ops: []
I20260709 13:20:12.035096 19188 raft_consensus.cc:3060] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:20:12.065589 18551 raft_consensus.cc:2804] T c2f5f89186204242bf377f7c42b2e2da P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:20:12.087186 16823 raft_consensus.cc:1240] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 [term 5 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 4. Current term is 5. Ops: []
I20260709 13:20:12.098276 19193 raft_consensus.cc:2749] T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860 [term 6 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:20:12.097967 19185 consensus_queue.cc:260] T b87147731a68400e8cce9b879daa7c5d P cb82cc7bd6db46d980c75237a1402860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:20:12.117657 16115 leader_election.cc:336] T 18ad7e3af5504571a69d10c00750a01c P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389): Timed out: RequestConsensusVote RPC to 127.14.177.131:41389 timed out after 1.638s (SENT)
I20260709 13:20:12.118263 16115 leader_election.cc:304] T 18ad7e3af5504571a69d10c00750a01c P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:20:12.105667 16224 raft_consensus.cc:1275] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac [term 6 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 13:20:12.034428 19199 consensus_queue.cc:1048] T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.081s
I20260709 13:20:12.114311 16145 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40844 (request call id 2022) took 1001 ms. Trace:
I20260709 13:20:12.133180 16145 rpcz_store.cc:276] 0709 13:20:11.113039 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:11.405400 (+292361us) service_pool.cc:224] Handling call
0709 13:20:11.420919 (+ 15519us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:20:11.721499 (+300580us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:11.721542 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:20:11.722059 (+   517us) log.cc:844] Serialized 26 byte log entry
0709 13:20:11.858504 (+136445us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:20:12.037579 (+179075us) log.cc:844] Serialized 14 byte log entry
0709 13:20:12.038208 (+   629us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:12.041500 (+  3292us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:20:12.114029 (+ 72529us) raft_consensus.cc:1685] finished
0709 13:20:12.114059 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:12.114233 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":45476,"prepare.queue_time_us":121143,"prepare.run_cpu_time_us":25,"prepare.run_wall_time_us":25,"raft.queue_time_us":299588,"raft.run_cpu_time_us":96,"raft.run_wall_time_us":96,"spinlock_wait_cycles":15472384,"thread_start_us":473642,"threads_started":3,"wal-append.queue_time_us":259324}
I20260709 13:20:12.144769 16816 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49906 (request call id 621) took 4761 ms. Trace:
I20260709 13:20:12.146088 16816 rpcz_store.cc:276] 0709 13:20:07.383026 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:08.572534 (+1189508us) service_pool.cc:224] Handling call
0709 13:20:10.424207 (+1851673us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:20:10.459589 (+ 35382us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:12.144571 (+1684982us) spinlock_profiling.cc:243] Waited 1.84 s on lock 0x7b54007c3b50. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5e8ad 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:20:12.144705 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4223694720}
I20260709 13:20:12.150539 16142 raft_consensus.cc:2468] T 26354361e9b044cea1c2b86c4882fe92 P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 6.
I20260709 13:20:12.099133 19201 raft_consensus.cc:2804] T 3884f864a96d428785895e5463e5728f P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:20:12.072240 19113 consensus_queue.cc:1048] T d16dfad0c3b540e5b4b86492ddf87208 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.066s
W20260709 13:20:12.143960 19212 log.cc:927] Time spent T fffab26427e24d398e2695aad40ca134 P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
W20260709 13:20:12.130237 19215 log.cc:927] Time spent T 5a959dbecc2648eb85d1cc550f5a3245 P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
W20260709 13:20:12.179554 16115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 61950us
W20260709 13:20:12.214726 16115 leader_election.cc:336] T b87147731a68400e8cce9b879daa7c5d P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 8 election: RPC error from VoteRequest() call to peer 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389): Timed out: RequestConsensusVote RPC to 127.14.177.131:41389 timed out after 1.692s (SENT)
I20260709 13:20:12.215308 16115 leader_election.cc:304] T b87147731a68400e8cce9b879daa7c5d P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860
I20260709 13:20:12.081568 18896 consensus_queue.cc:1048] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.064s
I20260709 13:20:12.099699 19185 raft_consensus.cc:3060] T b87147731a68400e8cce9b879daa7c5d P cb82cc7bd6db46d980c75237a1402860 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:20:12.183627 16151 raft_consensus.cc:2468] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 6.
W20260709 13:20:12.210026 16264 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:12.129314 18944 raft_consensus.cc:2749] T 18ad7e3af5504571a69d10c00750a01c P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:20:12.129999 18999 raft_consensus.cc:493] T 74ca8140f35c4ae29e8a664f6bdfc829 P cb82cc7bd6db46d980c75237a1402860 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:20:12.138267 19210 raft_consensus.cc:493] T 57a744f5aaf944678a63469cbf514015 P cb82cc7bd6db46d980c75237a1402860 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:20:12.319479 19179 raft_consensus.cc:493] T 746aabf856ab401c9ef5e859371e7b67 P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:12.102980 19184 consensus_queue.cc:1059] T 1df93779209748b9860f6d893dd3de53 P 647a343a542d4a139fe106c19095129b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: INVALID_TERM, Last received: 4.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.109s
W20260709 13:20:12.347823 16264 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:20:12.348958 16116 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:12.103106 19176 consensus_queue.cc:1059] T 8cc959fb7f704fba9009c0595467d074 P 647a343a542d4a139fe106c19095129b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.208s
W20260709 13:20:12.365777 16789 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:12.107151 18587 raft_consensus.cc:515] T 5a96373bb0e14dce8a266b2fc6c58507 P 1ca2725d290e47d68b1a4b609c9816ac [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:12.296486 19201 raft_consensus.cc:697] T 3884f864a96d428785895e5463e5728f P 647a343a542d4a139fe106c19095129b [term 6 LEADER]: Becoming Leader. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:20:12.371320 18551 raft_consensus.cc:697] T c2f5f89186204242bf377f7c42b2e2da P 1ca2725d290e47d68b1a4b609c9816ac [term 9 LEADER]: Becoming Leader. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
W20260709 13:20:12.375459 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 311660us
W20260709 13:20:12.380676 16115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 166000us
W20260709 13:20:12.381477 16115 leader_election.cc:336] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389): Timed out: RequestConsensusVote RPC to 127.14.177.131:41389 timed out after 1.701s (SENT)
W20260709 13:20:12.378774 19147 consensus_meta.cc:287] T 746aabf856ab401c9ef5e859371e7b67 P 1ca2725d290e47d68b1a4b609c9816ac: Time spent flushing consensus metadata: real 0.788s	user 0.011s	sys 0.001s
I20260709 13:20:12.392446 16219 raft_consensus.cc:1240] T 5a96373bb0e14dce8a266b2fc6c58507 P 1ca2725d290e47d68b1a4b609c9816ac [term 6 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 5. Current term is 6. Ops: []
W20260709 13:20:12.395319 16115 leader_election.cc:336] T b6f07842d3aa47858df5665874c86f3a P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389): Timed out: RequestConsensusVote RPC to 127.14.177.131:41389 timed out after 1.714s (SENT)
I20260709 13:20:12.355731 18999 raft_consensus.cc:515] T 74ca8140f35c4ae29e8a664f6bdfc829 P cb82cc7bd6db46d980c75237a1402860 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:20:12.401233 16115 leader_election.cc:336] T 08c74490c40243a88e62d1760b5cd814 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389): Timed out: RequestConsensusVote RPC to 127.14.177.131:41389 timed out after 1.793s (SENT)
I20260709 13:20:12.403049 18114 raft_consensus.cc:697] T 84f2da98864b4425bf7c0055951fdd4b P 1ca2725d290e47d68b1a4b609c9816ac [term 5 LEADER]: Becoming Leader. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:20:12.402688 16115 leader_election.cc:304] T 08c74490c40243a88e62d1760b5cd814 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:20:12.396818 18944 raft_consensus.cc:2749] T b87147731a68400e8cce9b879daa7c5d P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:20:12.407018 18114 consensus_queue.cc:237] T 84f2da98864b4425bf7c0055951fdd4b P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:20:12.420257 16791 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:12.422509 16816 raft_consensus.cc:1240] T 434bff3082d141838068f323bdb0d1ed P cb82cc7bd6db46d980c75237a1402860 [term 7 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 6. Current term is 7. Ops: [6.8-6.8]
W20260709 13:20:12.434809 19195 log.cc:927] Time spent T 806d7690c2b644959152074bb2cab0a7 P cb82cc7bd6db46d980c75237a1402860: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
W20260709 13:20:12.436982 16266 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:12.437476 19184 raft_consensus.cc:3055] T 8cc959fb7f704fba9009c0595467d074 P 647a343a542d4a139fe106c19095129b [term 6 LEADER]: Stepping down as leader of term 6
W20260709 13:20:12.437773 16266 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:12.437970 19184 raft_consensus.cc:740] T 8cc959fb7f704fba9009c0595467d074 P 647a343a542d4a139fe106c19095129b [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:20:12.443841 16816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e65676b21b748728ad39668cfe722ec" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:12.439412 16190 leader_election.cc:304] T 1df93779209748b9860f6d893dd3de53 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 647a343a542d4a139fe106c19095129b
I20260709 13:20:12.444587 16816 raft_consensus.cc:2468] T 9e65676b21b748728ad39668cfe722ec P cb82cc7bd6db46d980c75237a1402860 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 7.
I20260709 13:20:12.401158 18857 raft_consensus.cc:515] T 02838c86531d4f98bf58d1c22b8baf77 P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:12.401492 19147 raft_consensus.cc:515] T 746aabf856ab401c9ef5e859371e7b67 P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:12.454635 16291 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48684 (request call id 2178) took 2065 ms. Trace:
I20260709 13:20:12.455084 16291 rpcz_store.cc:276] 0709 13:20:10.389637 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:10.874827 (+485190us) service_pool.cc:224] Handling call
0709 13:20:10.879937 (+  5110us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:10.880132 (+   195us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:20:10.895823 (+ 15691us) log.cc:844] Serialized 14 byte log entry
0709 13:20:12.454294 (+1558471us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:20:12.454349 (+    55us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:20:12.454396 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:12.454464 (+    68us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:12.454579 (+   115us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1565035,"prepare.queue_time_us":16495,"prepare.run_cpu_time_us":32,"prepare.run_wall_time_us":33}
I20260709 13:20:12.457285 16824 raft_consensus.cc:3060] T 806d7690c2b644959152074bb2cab0a7 P cb82cc7bd6db46d980c75237a1402860 [term 5 LEARNER]: Advancing to term 6
W20260709 13:20:12.455367 16115 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:12.422060 19206 consensus_queue.cc:1048] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.295s
W20260709 13:20:12.458885 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 1.64055e+06us
I20260709 13:20:12.456858 16818 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49896 (request call id 955) took 1338 ms. Trace:
I20260709 13:20:12.350476 19210 raft_consensus.cc:515] T 57a744f5aaf944678a63469cbf514015 P cb82cc7bd6db46d980c75237a1402860 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:12.439028 19184 consensus_queue.cc:260] T 8cc959fb7f704fba9009c0595467d074 P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:12.359302 19179 raft_consensus.cc:515] T 746aabf856ab401c9ef5e859371e7b67 P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:12.465293 16818 rpcz_store.cc:276] 0709 13:20:11.118834 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:11.118937 (+   103us) service_pool.cc:224] Handling call
0709 13:20:11.202813 (+ 83876us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:20:11.261749 (+ 58936us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:11.261774 (+    25us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:20:11.282921 (+ 21147us) log.cc:844] Serialized 26 byte log entry
0709 13:20:11.445202 (+162281us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:20:11.457452 (+ 12250us) log.cc:844] Serialized 14 byte log entry
0709 13:20:11.816682 (+359230us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:11.816761 (+    79us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:20:12.356262 (+539501us) spinlock_profiling.cc:243] Waited 1.81 ms on lock 0x7b180031d4a0. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b693ade1 00007f69b69475b1 00007f69ada723f3 00007f69ada747c1 00007f69ada74779 00007f69ada74709 00007f69ada746d1 00007f69ada739cd 000000000038f354 000000000038f199 00007f69ada5583a
0709 13:20:12.439239 (+ 82977us) raft_consensus.cc:1685] finished
0709 13:20:12.456635 (+ 17396us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:12.456798 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7781,"prepare.queue_time_us":11790,"prepare.run_cpu_time_us":49,"prepare.run_wall_time_us":1634,"raft.queue_time_us":178472,"raft.run_cpu_time_us":264,"raft.run_wall_time_us":10671,"spinlock_wait_cycles":4166400,"thread_start_us":477875,"threads_started":3,"wal-append.queue_time_us":239334}
I20260709 13:20:12.451485 19135 raft_consensus.cc:2749] T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:20:12.456553 18758 raft_consensus.cc:2804] T 1df93779209748b9860f6d893dd3de53 P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:20:12.533725 16265 leader_election.cc:304] T 8c11074579b046d99e585c2818e18a2c P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: cb82cc7bd6db46d980c75237a1402860
W20260709 13:20:12.563537 19224 log.cc:927] Time spent T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860: Append to log took a long time: real 0.121s	user 0.000s	sys 0.001s
W20260709 13:20:12.564018 19219 log.cc:927] Time spent T 3a7ca61592e848ea9b0fb81a2ce33cdc P cb82cc7bd6db46d980c75237a1402860: Append to log took a long time: real 0.085s	user 0.000s	sys 0.000s
I20260709 13:20:12.564697 19214 raft_consensus.cc:2749] T 08c74490c40243a88e62d1760b5cd814 P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:20:12.565898 16820 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49906 (request call id 646) took 1165 ms. Trace:
W20260709 13:20:12.566723 19229 log.cc:927] Time spent T 2794af5b99984efebdee564035841cb0 P cb82cc7bd6db46d980c75237a1402860: Append to log took a long time: real 0.170s	user 0.001s	sys 0.000s
I20260709 13:20:12.566373 16820 rpcz_store.cc:276] 0709 13:20:11.400388 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:11.685539 (+285151us) service_pool.cc:224] Handling call
0709 13:20:11.685960 (+   421us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:20:11.709657 (+ 23697us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:11.709689 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:20:11.728910 (+ 19221us) log.cc:844] Serialized 26 byte log entry
0709 13:20:11.904812 (+175902us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:20:11.929515 (+ 24703us) log.cc:844] Serialized 14 byte log entry
0709 13:20:11.930123 (+   608us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:11.930241 (+   118us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:20:12.565655 (+635414us) raft_consensus.cc:1685] finished
0709 13:20:12.565687 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:12.565827 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":19086,"prepare.queue_time_us":24543,"prepare.run_cpu_time_us":34,"prepare.run_wall_time_us":35,"raft.queue_time_us":25423,"raft.run_cpu_time_us":376,"raft.run_wall_time_us":61771,"thread_start_us":115577,"threads_started":1,"wal-append.queue_time_us":708515}
W20260709 13:20:12.533535 19221 log.cc:927] Time spent T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
W20260709 13:20:12.572165 19223 log.cc:927] Time spent T 18ad7e3af5504571a69d10c00750a01c P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 0.107s	user 0.000s	sys 0.000s
I20260709 13:20:12.571625 16824 raft_consensus.cc:1275] T 806d7690c2b644959152074bb2cab0a7 P cb82cc7bd6db46d980c75237a1402860 [term 6 LEARNER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 6 index: 6. (index mismatch)
I20260709 13:20:12.574872 16817 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49906 (request call id 644) took 1175 ms. Trace:
I20260709 13:20:12.575554 16817 rpcz_store.cc:276] 0709 13:20:11.399890 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:11.612865 (+212975us) service_pool.cc:224] Handling call
0709 13:20:11.651471 (+ 38606us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:20:11.709685 (+ 58214us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:11.712570 (+  2885us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:20:11.741483 (+ 28913us) log.cc:844] Serialized 26 byte log entry
0709 13:20:12.014504 (+273021us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:20:12.024593 (+ 10089us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:12.034544 (+  9951us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:20:12.574514 (+539970us) raft_consensus.cc:1685] finished
0709 13:20:12.574546 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:12.574785 (+   239us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":29038,"raft.queue_time_us":52762,"raft.run_cpu_time_us":141,"raft.run_wall_time_us":5666,"thread_start_us":179420,"threads_started":1,"wal-append.queue_time_us":526543}
I20260709 13:20:12.578616 16149 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40844 (request call id 2021) took 1465 ms. Trace:
I20260709 13:20:12.581235 19147 leader_election.cc:290] T 746aabf856ab401c9ef5e859371e7b67 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 7 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
W20260709 13:20:12.581617 16189 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:12.459232 19187 raft_consensus.cc:515] T e5ad4eb281564936a59f41e387a47156 P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:20:12.584355 19190 log.cc:927] Time spent T af57996a8be84d398f0f023e8d08ef77 P cb82cc7bd6db46d980c75237a1402860: Append to log took a long time: real 0.957s	user 0.015s	sys 0.009s
I20260709 13:20:12.585249 19187 leader_election.cc:290] T e5ad4eb281564936a59f41e387a47156 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 8 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
W20260709 13:20:12.585603 19126 consensus_meta.cc:287] T 23f82224f4564fea93572e8206bad1fe P cb82cc7bd6db46d980c75237a1402860: Time spent flushing consensus metadata: real 0.604s	user 0.007s	sys 0.002s
I20260709 13:20:12.579210 16149 rpcz_store.cc:276] 0709 13:20:11.112742 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:11.403884 (+291142us) service_pool.cc:224] Handling call
0709 13:20:11.418497 (+ 14613us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:20:11.846681 (+428184us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:20:11.866763 (+ 20082us) log.cc:844] Serialized 14 byte log entry
0709 13:20:12.043133 (+176370us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:20:12.070923 (+ 27790us) log.cc:844] Serialized 26 byte log entry
0709 13:20:12.071097 (+   174us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:20:12.071393 (+   296us) log.cc:844] Serialized 14 byte log entry
0709 13:20:12.107374 (+ 35981us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:12.107469 (+    95us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:20:12.578273 (+470804us) raft_consensus.cc:1685] finished
0709 13:20:12.578303 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:12.578504 (+   201us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":190412,"prepare.queue_time_us":37839,"prepare.run_cpu_time_us":71,"prepare.run_wall_time_us":74,"raft.queue_time_us":331165,"raft.run_cpu_time_us":292,"raft.run_wall_time_us":16551,"thread_start_us":551996,"threads_started":2,"wal-append.queue_time_us":549716}
I20260709 13:20:12.570873 16819 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49906 (request call id 649) took 1162 ms. Trace:
I20260709 13:20:12.592366 16819 rpcz_store.cc:276] 0709 13:20:11.408851 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:11.731020 (+322169us) service_pool.cc:224] Handling call
0709 13:20:11.731442 (+   422us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:20:11.779605 (+ 48163us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:20:11.801744 (+ 22139us) log.cc:844] Serialized 14 byte log entry
0709 13:20:11.872245 (+ 70501us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:20:11.872719 (+   474us) log.cc:844] Serialized 26 byte log entry
0709 13:20:11.872846 (+   127us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:20:11.872877 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:11.872944 (+    67us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:20:11.981023 (+108079us) spinlock_profiling.cc:243] Waited 14.4 ms on lock 0x7b1800306be0. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b693ade1 00007f69b69475b1 00007f69ada723f3 00007f69ada747c1 00007f69ada74779 00007f69ada74709 00007f69ada746d1 00007f69ada739cd 000000000038f354 000000000038f199 00007f69ada5583a
0709 13:20:12.570445 (+589422us) raft_consensus.cc:1685] finished
0709 13:20:12.570594 (+   149us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:12.570793 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":49162,"prepare.queue_time_us":21938,"prepare.run_cpu_time_us":31,"prepare.run_wall_time_us":32,"raft.queue_time_us":56426,"raft.run_cpu_time_us":228,"raft.run_wall_time_us":10768,"spinlock_wait_cycles":33215744,"thread_start_us":25691,"threads_started":1,"wal-append.queue_time_us":581688}
W20260709 13:20:12.566051 19239 log.cc:927] Time spent T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 0.111s	user 0.000s	sys 0.000s
I20260709 13:20:12.601691 16825 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49896 (request call id 956) took 1474 ms. Trace:
I20260709 13:20:12.455741 19201 consensus_queue.cc:237] T 3884f864a96d428785895e5463e5728f P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:12.585887 19197 raft_consensus.cc:3055] T 1df93779209748b9860f6d893dd3de53 P 647a343a542d4a139fe106c19095129b [term 6 LEADER]: Stepping down as leader of term 6
W20260709 13:20:12.617954 19003 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.505s	user 0.026s	sys 0.024s
I20260709 13:20:12.480095 18551 consensus_queue.cc:237] T c2f5f89186204242bf377f7c42b2e2da P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:12.608820 16825 rpcz_store.cc:276] 0709 13:20:11.127140 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:11.163555 (+ 36415us) service_pool.cc:224] Handling call
0709 13:20:11.201807 (+ 38252us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:20:11.202174 (+   367us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:20:11.224661 (+ 22487us) log.cc:844] Serialized 14 byte log entry
0709 13:20:11.388179 (+163518us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:20:11.404803 (+ 16624us) log.cc:844] Serialized 26 byte log entry
0709 13:20:11.404950 (+   147us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:20:11.404982 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:11.405052 (+    70us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:20:12.583700 (+1178648us) spinlock_profiling.cc:243] Waited 2.5 ms on lock 0x7b4400022b4c. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69ad8fadb1 00007f69ad998c85 00007f69ad9996a2 00007f69b070b1db 00007f69b0706aaf 00007f69b06fa86e 00007f69b06f9bb6 00007f69b06f97d8 00007f69b0703221 00007f69b07031d9 00007f69b0703169
0709 13:20:12.586914 (+  3214us) raft_consensus.cc:1685] finished
0709 13:20:12.601411 (+ 14497us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:12.601611 (+   200us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":23519,"prepare.run_cpu_time_us":42,"prepare.run_wall_time_us":44,"raft.queue_time_us":5301,"raft.run_wall_time_us":21254,"spinlock_wait_cycles":8846592,"thread_start_us":88159,"threads_started":1,"wal-append.queue_time_us":276625}
I20260709 13:20:12.535005 19202 consensus_queue.cc:1059] T 434bff3082d141838068f323bdb0d1ed P 647a343a542d4a139fe106c19095129b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.089s
W20260709 13:20:12.685869 16117 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 234224us
W20260709 13:20:12.678042 19209 consensus_meta.cc:287] T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b: Time spent flushing consensus metadata: real 0.653s	user 0.006s	sys 0.008s
W20260709 13:20:12.679895 19003 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.564s	user 0.027s	sys 0.024s
I20260709 13:20:12.632999 19197 raft_consensus.cc:740] T 1df93779209748b9860f6d893dd3de53 P 647a343a542d4a139fe106c19095129b [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
W20260709 13:20:12.687475 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 308775us
W20260709 13:20:12.698769 19177 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.272s	user 0.041s	sys 0.070s
W20260709 13:20:12.699285 19177 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.292s	user 0.041s	sys 0.070s
W20260709 13:20:12.700704 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 232098us
I20260709 13:20:12.684556 18587 leader_election.cc:290] T 5a96373bb0e14dce8a266b2fc6c58507 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 6 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:12.703722 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b7d1ab519044eda2d5e04a07242f25" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:20:12.709764 16218 raft_consensus.cc:1275] T 66f75124215f4ca6bef831e174a792d8 P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:20:12.715134 16221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "829b4ad8c5a54eababeea1181b960d4b" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
W20260709 13:20:12.706837 19142 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.305s	user 0.008s	sys 0.012s
W20260709 13:20:12.719022 16264 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:20:12.719550 16264 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:12.722229 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cc959fb7f704fba9009c0595467d074" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:20:12.722860 16293 raft_consensus.cc:3060] T 8cc959fb7f704fba9009c0595467d074 P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:20:12.702625 18999 leader_election.cc:290] T 74ca8140f35c4ae29e8a664f6bdfc829 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
W20260709 13:20:12.718308 19142 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.317s	user 0.008s	sys 0.013s
I20260709 13:20:12.734140 16292 raft_consensus.cc:1275] T d16dfad0c3b540e5b4b86492ddf87208 P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:20:12.739974 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada5a004ba7c46819051e3ccec71ffdb" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:20:12.740542 16299 raft_consensus.cc:3060] T ada5a004ba7c46819051e3ccec71ffdb P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Advancing to term 8
I20260709 13:20:12.736764 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f5f89186204242bf377f7c42b2e2da" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:20:12.741294 16300 raft_consensus.cc:3060] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:20:12.732066 18843 raft_consensus.cc:493] T a378d7f796f94615b3b347119a9880fd P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:12.737668 18857 leader_election.cc:290] T 02838c86531d4f98bf58d1c22b8baf77 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 8 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
W20260709 13:20:12.752328 19230 log.cc:927] Time spent T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 0.183s	user 0.000s	sys 0.001s
I20260709 13:20:12.746852 16292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ab0ac8347641e6955e9bce7d90c1a2" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:20:12.744467 19066 raft_consensus.cc:2804] T 8c11074579b046d99e585c2818e18a2c P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 13:20:12.778158 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 296353us
I20260709 13:20:12.777374 16292 raft_consensus.cc:2376] T d2ab0ac8347641e6955e9bce7d90c1a2 P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 7. Re-sending same reply.
W20260709 13:20:12.793706 16264 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:12.798076 16299 raft_consensus.cc:2468] T ada5a004ba7c46819051e3ccec71ffdb P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 8.
I20260709 13:20:12.815707 16292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 7 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:12.816840 16300 raft_consensus.cc:2468] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 9.
I20260709 13:20:12.820245 16292 raft_consensus.cc:2468] T ecc1e363230043fe9fef390f0b979f20 P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 6.
I20260709 13:20:12.823989 16265 leader_election.cc:304] T 5a96373bb0e14dce8a266b2fc6c58507 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
W20260709 13:20:12.824899 16264 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:20:12.783918 19241 log.cc:927] Time spent T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 0.073s	user 0.000s	sys 0.001s
I20260709 13:20:12.831442 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e9b2bf7011f4ae688a79d7bb5a447b4" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:20:12.773655 19179 leader_election.cc:290] T 746aabf856ab401c9ef5e859371e7b67 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:12.834126 16292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31969392ccdb43b491911a319e8bc7e1" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:20:12.778323 19066 raft_consensus.cc:493] T 8c11074579b046d99e585c2818e18a2c P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Starting leader election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:20:12.775840 19210 leader_election.cc:290] T 57a744f5aaf944678a63469cbf514015 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:12.841030 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57a744f5aaf944678a63469cbf514015" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:12.843132 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8158bbd17b994d2eb96f560bd0f1b88d" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:12.809738 19135 raft_consensus.cc:493] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:12.818604 19136 raft_consensus.cc:493] T b87147731a68400e8cce9b879daa7c5d P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:20:12.847934 16266 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:12.868214 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e65676b21b748728ad39668cfe722ec" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:12.858201 16297 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48684 (request call id 2160) took 3809 ms. Trace:
I20260709 13:20:12.869174 16300 raft_consensus.cc:2468] T 9e65676b21b748728ad39668cfe722ec P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 7.
I20260709 13:20:12.818850 19080 raft_consensus.cc:493] T 18ad7e3af5504571a69d10c00750a01c P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:20:12.868528 16825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 7 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:12.899881 16825 raft_consensus.cc:2468] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 6.
W20260709 13:20:12.839267 19238 log.cc:927] Time spent T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860: Append to log took a long time: real 0.295s	user 0.001s	sys 0.001s
I20260709 13:20:12.877030 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e4bfb9090c8460eb14fa7e0d818e7f1" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:20:12.772312 18843 raft_consensus.cc:515] T a378d7f796f94615b3b347119a9880fd P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:12.829226 19134 raft_consensus.cc:493] T 31969392ccdb43b491911a319e8bc7e1 P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:20:12.872179 16292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26354361e9b044cea1c2b86c4882fe92" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:12.903092 16820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746aabf856ab401c9ef5e859371e7b67" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:20:12.809485 19234 consensus_queue.cc:1048] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.208s
I20260709 13:20:12.901866 16300 raft_consensus.cc:2410] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 for term 6 because replica has last-logged OpId of term: 5 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 5.
W20260709 13:20:12.883546 16266 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:20:12.904831 16266 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:20:12.907419 19254 log.cc:927] Time spent T 5a959dbecc2648eb85d1cc550f5a3245 P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 0.076s	user 0.001s	sys 0.000s
I20260709 13:20:12.907582 16817 raft_consensus.cc:1275] T d2ab0ac8347641e6955e9bce7d90c1a2 P cb82cc7bd6db46d980c75237a1402860 [term 7 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:20:12.908345 19134 raft_consensus.cc:515] T 31969392ccdb43b491911a319e8bc7e1 P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:20:12.909318 19252 log.cc:927] Time spent T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 0.070s	user 0.000s	sys 0.000s
I20260709 13:20:12.844168 19066 raft_consensus.cc:3060] T 8c11074579b046d99e585c2818e18a2c P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:20:12.898255 19136 raft_consensus.cc:515] T b87147731a68400e8cce9b879daa7c5d P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:12.916378 19136 leader_election.cc:290] T b87147731a68400e8cce9b879daa7c5d P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:12.898048 19135 raft_consensus.cc:515] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:20:12.920655 19245 log.cc:927] Time spent T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860: Append to log took a long time: real 0.147s	user 0.000s	sys 0.000s
W20260709 13:20:12.922132 19240 log.cc:927] Time spent T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 0.198s	user 0.001s	sys 0.000s
W20260709 13:20:12.923246 19242 log.cc:927] Time spent T fffab26427e24d398e2695aad40ca134 P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 0.215s	user 0.000s	sys 0.001s
I20260709 13:20:12.924642 18996 raft_consensus.cc:493] T 434bff3082d141838068f323bdb0d1ed P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:12.925341 18996 raft_consensus.cc:515] T 434bff3082d141838068f323bdb0d1ed P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:12.933239 16299 raft_consensus.cc:2410] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 for term 5 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:20:12.846786 19065 raft_consensus.cc:2749] T 5a96373bb0e14dce8a266b2fc6c58507 P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:20:12.938874 16292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26354361e9b044cea1c2b86c4882fe92" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:12.940938 16292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2794af5b99984efebdee564035841cb0" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:12.951537 16295 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48684 (request call id 2180) took 2561 ms. Trace:
I20260709 13:20:12.952095 16295 rpcz_store.cc:276] 0709 13:20:10.390240 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:11.291527 (+901287us) service_pool.cc:224] Handling call
0709 13:20:11.314782 (+ 23255us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:20:12.712779 (+1397997us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:12.712808 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:20:12.727227 (+ 14419us) log.cc:844] Serialized 26 byte log entry
0709 13:20:12.870247 (+143020us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:20:12.870290 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:12.870376 (+    86us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:20:12.951238 (+ 80862us) raft_consensus.cc:1685] finished
0709 13:20:12.951266 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:12.951458 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1415673,"raft.queue_time_us":1403693,"raft.run_cpu_time_us":113,"raft.run_wall_time_us":6219,"thread_start_us":112836,"threads_started":1,"wal-append.queue_time_us":201900}
I20260709 13:20:12.954560 16295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f2da98864b4425bf7c0055951fdd4b" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
W20260709 13:20:12.959601 19243 log.cc:927] Time spent T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 0.258s	user 0.000s	sys 0.001s
I20260709 13:20:12.962486 16293 raft_consensus.cc:2468] T 8cc959fb7f704fba9009c0595467d074 P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 7.
I20260709 13:20:12.962925 16298 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48674 (request call id 2241) took 2819 ms. Trace:
I20260709 13:20:12.976289 16298 rpcz_store.cc:276] 0709 13:20:10.143794 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:10.143945 (+   151us) service_pool.cc:224] Handling call
0709 13:20:10.183233 (+ 39288us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:20:10.187365 (+  4132us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:20:10.728095 (+540730us) log.cc:844] Serialized 14 byte log entry
0709 13:20:12.454168 (+1726073us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:20:12.458760 (+  4592us) log.cc:844] Serialized 26 byte log entry
0709 13:20:12.739363 (+280603us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:20:12.823075 (+ 83712us) log.cc:844] Serialized 14 byte log entry
0709 13:20:12.826157 (+  3082us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:12.826277 (+   120us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:20:12.962732 (+136455us) raft_consensus.cc:1685] finished
0709 13:20:12.962749 (+    17us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:12.962869 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2230454,"prepare.queue_time_us":589441,"prepare.run_cpu_time_us":53,"prepare.run_wall_time_us":53,"raft.queue_time_us":1610757,"raft.run_cpu_time_us":313,"raft.run_wall_time_us":20059,"thread_start_us":610060,"threads_started":4,"wal-append.queue_time_us":531667,"wal-append.run_cpu_time_us":1212,"wal-append.run_wall_time_us":1005433}
I20260709 13:20:12.773167 19197 consensus_queue.cc:260] T 1df93779209748b9860f6d893dd3de53 P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:12.979666 19197 raft_consensus.cc:3060] T 1df93779209748b9860f6d893dd3de53 P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:20:12.982605 16296 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48674 (request call id 2255) took 2686 ms. Trace:
I20260709 13:20:12.984090 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ad5dbfa99ee4b8d9e28be92b3f36db5" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:12.983006 16296 rpcz_store.cc:276] 0709 13:20:10.296497 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:10.839780 (+543283us) service_pool.cc:224] Handling call
0709 13:20:10.854308 (+ 14528us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:20:12.454146 (+1599838us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:12.454181 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:20:12.473130 (+ 18949us) log.cc:844] Serialized 26 byte log entry
0709 13:20:12.715300 (+242170us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:20:12.775236 (+ 59936us) log.cc:844] Serialized 14 byte log entry
0709 13:20:12.809917 (+ 34681us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:12.809990 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:20:12.982372 (+172382us) raft_consensus.cc:1685] finished
0709 13:20:12.982392 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:12.982547 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1893740,"prepare.queue_time_us":111356,"prepare.run_cpu_time_us":71,"prepare.run_wall_time_us":5833,"raft.queue_time_us":1643476,"raft.run_cpu_time_us":257,"raft.run_wall_time_us":12429,"thread_start_us":179207,"threads_started":2,"wal-append.queue_time_us":284964}
I20260709 13:20:12.985134 16298 raft_consensus.cc:2468] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 6.
I20260709 13:20:12.919697 19191 raft_consensus.cc:3055] T 434bff3082d141838068f323bdb0d1ed P 647a343a542d4a139fe106c19095129b [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:20:12.988138 19191 raft_consensus.cc:740] T 434bff3082d141838068f323bdb0d1ed P 647a343a542d4a139fe106c19095129b [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:20:12.989249 19191 consensus_queue.cc:260] T 434bff3082d141838068f323bdb0d1ed P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:12.990743 19191 raft_consensus.cc:3060] T 434bff3082d141838068f323bdb0d1ed P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:20:12.964118 18996 leader_election.cc:290] T 434bff3082d141838068f323bdb0d1ed P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:12.965036 16295 raft_consensus.cc:2393] T 84f2da98864b4425bf7c0055951fdd4b P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 5: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
W20260709 13:20:12.970436 19153 consensus_meta.cc:287] T c0eab25ce4c64e018a041beeda2867db P 1ca2725d290e47d68b1a4b609c9816ac: Time spent flushing consensus metadata: real 1.092s	user 0.007s	sys 0.004s
I20260709 13:20:12.972129 18843 leader_election.cc:290] T a378d7f796f94615b3b347119a9880fd P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:12.942910 19080 raft_consensus.cc:515] T 18ad7e3af5504571a69d10c00750a01c P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } }
I20260709 13:20:13.000751 16296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7ca61592e848ea9b0fb81a2ce33cdc" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:12.919646 19135 leader_election.cc:290] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:13.016381 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d4ab870a05347868956aa8b841b2981" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:13.023216 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a9de347c711477e87c582d003f36925" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:20:13.023360 16295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2794af5b99984efebdee564035841cb0" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:20:13.032895 16292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4f65cdf00b748bb963ee6e81ef43b79" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:13.045965 16292 raft_consensus.cc:2468] T c4f65cdf00b748bb963ee6e81ef43b79 P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 8.
I20260709 13:20:13.049706 16293 raft_consensus.cc:2468] T 8d4ab870a05347868956aa8b841b2981 P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 7.
I20260709 13:20:13.051689 16292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e65676b21b748728ad39668cfe722ec" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:13.053977 16295 raft_consensus.cc:1275] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 6 index: 6. (index mismatch)
I20260709 13:20:13.051708 16300 raft_consensus.cc:3060] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:20:13.016832 19185 consensus_meta.cc:287] T b87147731a68400e8cce9b879daa7c5d P cb82cc7bd6db46d980c75237a1402860: Time spent flushing consensus metadata: real 0.708s	user 0.008s	sys 0.005s
I20260709 13:20:13.054381 16292 raft_consensus.cc:2468] T 9e65676b21b748728ad39668cfe722ec P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 7.
I20260709 13:20:13.057289 16291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26354361e9b044cea1c2b86c4882fe92" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:20:13.067237 16292 raft_consensus.cc:1240] T b87147731a68400e8cce9b879daa7c5d P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 7. Current term is 8. Ops: []
I20260709 13:20:13.058655 16295 raft_consensus.cc:1240] T 23f82224f4564fea93572e8206bad1fe P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 4. Current term is 5. Ops: [4.5-4.5]
I20260709 13:20:13.055780 16296 raft_consensus.cc:2410] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 for term 10 because replica has last-logged OpId of term: 9 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 8.
I20260709 13:20:13.054877 16293 raft_consensus.cc:1275] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
W20260709 13:20:13.062439 16822 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49896 (request call id 950) took 1946 ms (client timeout 1744 ms). Trace:
W20260709 13:20:13.074887 16822 rpcz_store.cc:269] 0709 13:20:11.116270 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:11.122688 (+  6418us) service_pool.cc:224] Handling call
0709 13:20:13.062232 (+1939544us) spinlock_profiling.cc:243] Waited 657 ms on lock 0x7b5400520dd4. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:20:13.062372 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1510969088}
I20260709 13:20:13.078047 16294 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48674 (request call id 2256) took 2157 ms. Trace:
I20260709 13:20:13.078577 16294 rpcz_store.cc:276] 0709 13:20:10.920630 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:11.301641 (+381011us) service_pool.cc:224] Handling call
0709 13:20:11.308238 (+  6597us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:20:12.718716 (+1410478us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:12.718752 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:20:12.729538 (+ 10786us) log.cc:844] Serialized 26 byte log entry
0709 13:20:12.832213 (+102675us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:20:12.832269 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:12.832359 (+    90us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:20:12.924258 (+ 91899us) spinlock_profiling.cc:243] Waited 3.1 ms on lock 0x7b4800130158. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b5a88aa1 00007f69b5a7fb37 00007f69b5a7f9ca 00007f69b5b7726f 00007f69b5b76f37 00007f69b5b8b55c 00007f69b5b8b508 00007f69b5b8b488 00007f69b5b8b443 00007f69b5b8a6ec 00007f69b4ab4a96
0709 13:20:13.065617 (+141359us) raft_consensus.cc:1685] finished
0709 13:20:13.065636 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:13.077964 (+ 12328us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1307554,"raft.queue_time_us":1319309,"raft.run_cpu_time_us":2689,"raft.run_wall_time_us":197399,"spinlock_wait_cycles":7134464,"thread_start_us":95634,"threads_started":1,"wal-append.queue_time_us":315893}
I20260709 13:20:13.084115 16293 raft_consensus.cc:1275] T c9944d5e4b8c4109af9955c4f08c9095 P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 7 index: 9. (index mismatch)
I20260709 13:20:13.025403 19153 raft_consensus.cc:515] T c0eab25ce4c64e018a041beeda2867db P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:20:13.088923 16263 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:13.089478 16300 raft_consensus.cc:2468] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 8.
W20260709 13:20:13.095881 19189 log.cc:927] Time spent T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 1.468s	user 0.017s	sys 0.027s
W20260709 13:20:13.097008 19262 log.cc:927] Time spent T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.078s	user 0.000s	sys 0.000s
I20260709 13:20:13.099071 19080 leader_election.cc:290] T 18ad7e3af5504571a69d10c00750a01c P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:20:13.055291 19066 raft_consensus.cc:515] T 8c11074579b046d99e585c2818e18a2c P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:13.099480 19179 raft_consensus.cc:493] T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:13.110751 19179 raft_consensus.cc:515] T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:13.115985 19179 leader_election.cc:290] T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
W20260709 13:20:13.118323 19207 log.cc:927] Time spent T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 1.000s	user 0.003s	sys 0.000s
W20260709 13:20:13.128834 16264 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:20:13.119488 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 295520us
I20260709 13:20:13.148378 16223 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33050 (request call id 532) took 1525 ms. Trace:
I20260709 13:20:13.148824 16223 rpcz_store.cc:276] 0709 13:20:11.623069 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:11.623327 (+   258us) service_pool.cc:224] Handling call
0709 13:20:11.670386 (+ 47059us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:20:11.671285 (+   899us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:13.146607 (+1475322us) spinlock_profiling.cc:243] Waited 36.4 ms on lock 0x7b540023fad0. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5e8ad 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:20:13.148304 (+  1697us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":83772672}
I20260709 13:20:13.175809 19066 leader_election.cc:290] T 8c11074579b046d99e585c2818e18a2c P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 7 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
W20260709 13:20:13.176287 16117 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:20:13.177968 16115 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:20:13.183369 16115 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:13.184325 19153 leader_election.cc:290] T c0eab25ce4c64e018a041beeda2867db P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 8 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:13.184533 16291 raft_consensus.cc:1240] T 8cc959fb7f704fba9009c0595467d074 P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 6. Current term is 7. Ops: []
W20260709 13:20:13.187582 19188 consensus_meta.cc:287] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860: Time spent flushing consensus metadata: real 1.057s	user 0.006s	sys 0.006s
I20260709 13:20:13.188107 19188 raft_consensus.cc:515] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:13.189610 16295 raft_consensus.cc:1240] T 7c2095a5f4ad49408dbc7f69c606120d P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 5. Current term is 6. Ops: []
I20260709 13:20:13.195456 16149 raft_consensus.cc:1275] T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:20:13.196334 16142 raft_consensus.cc:1275] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
W20260709 13:20:13.190707 19263 log.cc:927] Time spent T e4dc6b0541ca431ebc757beac00521f1 P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.059s	user 0.001s	sys 0.000s
I20260709 13:20:13.197278 16143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57a744f5aaf944678a63469cbf514015" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:13.197932 16143 raft_consensus.cc:2468] T 57a744f5aaf944678a63469cbf514015 P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 4.
I20260709 13:20:13.199332 19175 raft_consensus.cc:493] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:13.199834 19175 raft_consensus.cc:515] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:13.200915 19209 raft_consensus.cc:493] T d16dfad0c3b540e5b4b86492ddf87208 P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:13.201380 19209 raft_consensus.cc:515] T d16dfad0c3b540e5b4b86492ddf87208 P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:13.203374 19209 leader_election.cc:290] T d16dfad0c3b540e5b4b86492ddf87208 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:13.204448 19202 raft_consensus.cc:493] T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:13.204896 19202 raft_consensus.cc:515] T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:13.206974 19202 leader_election.cc:290] T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
W20260709 13:20:13.207285 16789 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:20:13.191735 16264 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:13.211855 16266 leader_election.cc:304] T 9e65676b21b748728ad39668cfe722ec P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 
W20260709 13:20:13.192852 16790 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:13.234582 16825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434bff3082d141838068f323bdb0d1ed" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 8 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:13.234848 16819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a378d7f796f94615b3b347119a9880fd" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:13.235647 16825 raft_consensus.cc:2468] T 434bff3082d141838068f323bdb0d1ed P cb82cc7bd6db46d980c75237a1402860 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 7.
I20260709 13:20:13.237599 16146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a96373bb0e14dce8a266b2fc6c58507" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 6 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:20:13.237727 16143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4f65cdf00b748bb963ee6e81ef43b79" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:13.240820 16819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c11074579b046d99e585c2818e18a2c" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:20:12.699615 15046 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:20:13.242429 16115 leader_election.cc:304] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:20:13.243721 16823 raft_consensus.cc:1240] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 [term 7 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 6. Current term is 7. Ops: []
I20260709 13:20:13.245169 16825 raft_consensus.cc:1240] T 2794af5b99984efebdee564035841cb0 P cb82cc7bd6db46d980c75237a1402860 [term 6 LEADER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 5. Current term is 6. Ops: []
W20260709 13:20:13.247535 16114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:13.250900 16820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e9b2bf7011f4ae688a79d7bb5a447b4" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:13.250201 16825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26354361e9b044cea1c2b86c4882fe92" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
W20260709 13:20:13.251550 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 550025us
I20260709 13:20:13.252450 16265 leader_election.cc:304] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
W20260709 13:20:13.298734 16264 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:13.253844 16819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746aabf856ab401c9ef5e859371e7b67" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
W20260709 13:20:13.314464 16264 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:13.314996 16819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5d59407eb545b895d8d922e0dd69fd" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 9 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:13.306335 16789 leader_election.cc:304] T 434bff3082d141838068f323bdb0d1ed P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 647a343a542d4a139fe106c19095129b
I20260709 13:20:13.316599 19144 raft_consensus.cc:2804] T 434bff3082d141838068f323bdb0d1ed P cb82cc7bd6db46d980c75237a1402860 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:20:13.317137 19144 raft_consensus.cc:697] T 434bff3082d141838068f323bdb0d1ed P cb82cc7bd6db46d980c75237a1402860 [term 7 LEADER]: Becoming Leader. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:20:13.317905 16789 leader_election.cc:304] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:20:13.318012 19144 consensus_queue.cc:237] T 434bff3082d141838068f323bdb0d1ed P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:13.318758 19185 raft_consensus.cc:2749] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:20:13.319998 16789 leader_election.cc:304] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:20:13.320662 19185 raft_consensus.cc:2749] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P cb82cc7bd6db46d980c75237a1402860 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:20:13.322536 16789 leader_election.cc:304] T 9e65676b21b748728ad39668cfe722ec P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:20:13.313660 16820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31969392ccdb43b491911a319e8bc7e1" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
W20260709 13:20:13.284507 16266 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:13.323345 16820 raft_consensus.cc:2468] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 6.
I20260709 13:20:13.324661 16266 leader_election.cc:304] T 434bff3082d141838068f323bdb0d1ed P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:20:13.254426 16822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87147731a68400e8cce9b879daa7c5d" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:13.253367 16046 catalog_manager.cc:5697] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "cb82cc7bd6db46d980c75237a1402860" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:20:13.214520 19088 raft_consensus.cc:2804] T 9e65676b21b748728ad39668cfe722ec P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:20:13.254712 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746aabf856ab401c9ef5e859371e7b67" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:20:13.330147 16150 raft_consensus.cc:3060] T 746aabf856ab401c9ef5e859371e7b67 P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:20:13.268985 16825 raft_consensus.cc:2393] T 26354361e9b044cea1c2b86c4882fe92 P cb82cc7bd6db46d980c75237a1402860 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 6: Already voted for candidate cb82cc7bd6db46d980c75237a1402860 in this term.
I20260709 13:20:13.325630 16822 raft_consensus.cc:2468] T b87147731a68400e8cce9b879daa7c5d P cb82cc7bd6db46d980c75237a1402860 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 8.
I20260709 13:20:13.329144 16818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66f75124215f4ca6bef831e174a792d8" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:13.333330 19068 consensus_queue.cc:1059] T 2794af5b99984efebdee564035841cb0 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
W20260709 13:20:13.333559 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 117209us
I20260709 13:20:13.334111 19068 consensus_queue.cc:1243] T 2794af5b99984efebdee564035841cb0 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Peer cb82cc7bd6db46d980c75237a1402860 log is divergent from this leader: its last log entry 6.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
W20260709 13:20:13.334260 16790 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:13.334611 19068 raft_consensus.cc:3055] T 2794af5b99984efebdee564035841cb0 P 90f3da57796b488c9087a2f523adf045 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:20:13.334995 19068 raft_consensus.cc:740] T 2794af5b99984efebdee564035841cb0 P 90f3da57796b488c9087a2f523adf045 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:20:13.335595 19210 raft_consensus.cc:493] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:13.335834 19068 consensus_queue.cc:260] T 2794af5b99984efebdee564035841cb0 P 90f3da57796b488c9087a2f523adf045 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:13.336236 19210 raft_consensus.cc:515] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:13.337105 19068 raft_consensus.cc:3060] T 2794af5b99984efebdee564035841cb0 P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:20:13.341566 16192 leader_election.cc:304] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:20:13.241389 19175 leader_election.cc:290] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
W20260709 13:20:13.345820 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 93445.5us
I20260709 13:20:13.346583 16265 leader_election.cc:304] T e4dc6b0541ca431ebc757beac00521f1 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:20:13.347352 19269 raft_consensus.cc:2749] T e4dc6b0541ca431ebc757beac00521f1 P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:20:13.348631 16265 leader_election.cc:304] T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860
I20260709 13:20:13.349388 19269 raft_consensus.cc:2749] T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:20:13.350816 16265 leader_election.cc:304] T ecc1e363230043fe9fef390f0b979f20 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:20:13.297266 19066 raft_consensus.cc:2749] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:20:13.356825 16817 raft_consensus.cc:1240] T 26354361e9b044cea1c2b86c4882fe92 P cb82cc7bd6db46d980c75237a1402860 [term 6 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 5. Current term is 6. Ops: []
W20260709 13:20:13.323365 16115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 81024.4us
I20260709 13:20:13.352952 16046 catalog_manager.cc:5697] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860 reported cstate change: term changed from 2 to 4, leader changed from 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130) to 647a343a542d4a139fe106c19095129b (127.14.177.129). New cstate: current_term: 4 leader_uuid: "647a343a542d4a139fe106c19095129b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:20:13.358202 16115 leader_election.cc:304] T 23f82224f4564fea93572e8206bad1fe P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860
I20260709 13:20:13.339562 19148 raft_consensus.cc:2804] T 9e65676b21b748728ad39668cfe722ec P cb82cc7bd6db46d980c75237a1402860 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:20:13.358152 16046 catalog_manager.cc:5697] T c9944d5e4b8c4109af9955c4f08c9095 P cb82cc7bd6db46d980c75237a1402860 reported cstate change: term changed from 3 to 5, leader changed from 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130) to cb82cc7bd6db46d980c75237a1402860 (127.14.177.132). New cstate: current_term: 5 leader_uuid: "cb82cc7bd6db46d980c75237a1402860" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:20:13.359989 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 107608us
I20260709 13:20:13.339228 19210 leader_election.cc:290] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:20:13.331923 19088 raft_consensus.cc:493] T 9e65676b21b748728ad39668cfe722ec P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:13.370203 16150 raft_consensus.cc:2468] T 746aabf856ab401c9ef5e859371e7b67 P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 7.
I20260709 13:20:13.343798 18743 raft_consensus.cc:2804] T 434bff3082d141838068f323bdb0d1ed P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:20:13.397053 19175 raft_consensus.cc:2749] T 23f82224f4564fea93572e8206bad1fe P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:20:13.398604 16115 leader_election.cc:304] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:20:13.399189 18587 raft_consensus.cc:493] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:13.399425 19175 raft_consensus.cc:2749] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:20:13.399715 18587 raft_consensus.cc:515] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:13.400643 16115 leader_election.cc:304] T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:20:13.390079 19134 leader_election.cc:290] T 31969392ccdb43b491911a319e8bc7e1 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:13.394964 16046 catalog_manager.cc:5697] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P cb82cc7bd6db46d980c75237a1402860 reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "1ca2725d290e47d68b1a4b609c9816ac" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
W20260709 13:20:13.398516 16191 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:13.402520 16115 leader_election.cc:304] T 8a878999c0024d2abe517484180290fc P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:20:13.355530 18551 raft_consensus.cc:493] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:13.401481 18743 raft_consensus.cc:493] T 434bff3082d141838068f323bdb0d1ed P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:13.409431 18587 leader_election.cc:290] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:13.409655 18743 raft_consensus.cc:3060] T 434bff3082d141838068f323bdb0d1ed P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:20:13.410164 19175 raft_consensus.cc:2749] T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:20:13.389951 19148 raft_consensus.cc:493] T 9e65676b21b748728ad39668cfe722ec P cb82cc7bd6db46d980c75237a1402860 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:20:13.411188 19270 log.cc:927] Time spent T b6f07842d3aa47858df5665874c86f3a P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 0.156s	user 0.001s	sys 0.000s
I20260709 13:20:13.411430 19148 raft_consensus.cc:3060] T 9e65676b21b748728ad39668cfe722ec P cb82cc7bd6db46d980c75237a1402860 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:20:13.414439 19250 raft_consensus.cc:493] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:20:13.416714 16189 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:20:13.410311 19268 log.cc:927] Time spent T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860: Append to log took a long time: real 0.165s	user 0.000s	sys 0.000s
I20260709 13:20:13.360553 19153 raft_consensus.cc:2804] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:20:13.410072 19113 raft_consensus.cc:493] T 02838c86531d4f98bf58d1c22b8baf77 P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:13.424714 16790 leader_election.cc:304] T 829b4ad8c5a54eababeea1181b960d4b P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:20:13.343868 19244 consensus_queue.cc:1048] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.085s
I20260709 13:20:13.428965 16790 leader_election.cc:304] T 26354361e9b044cea1c2b86c4882fe92 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:20:13.414956 19250 raft_consensus.cc:515] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:13.432278 16115 leader_election.cc:304] T 57a744f5aaf944678a63469cbf514015 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860
I20260709 13:20:13.432998 16190 leader_election.cc:304] T ecc1e363230043fe9fef390f0b979f20 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:20:13.437963 18114 raft_consensus.cc:493] T 66f75124215f4ca6bef831e174a792d8 P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:13.403060 19088 raft_consensus.cc:3060] T 9e65676b21b748728ad39668cfe722ec P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:20:13.434984 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 84271.7us
I20260709 13:20:13.416010 19066 raft_consensus.cc:2749] T ecc1e363230043fe9fef390f0b979f20 P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:20:13.440796 16266 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:13.438452 18114 raft_consensus.cc:515] T 66f75124215f4ca6bef831e174a792d8 P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:13.424675 19210 raft_consensus.cc:493] T 23f82224f4564fea93572e8206bad1fe P cb82cc7bd6db46d980c75237a1402860 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:13.423133 19193 raft_consensus.cc:493] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:20:13.468621 16115 leader_election.cc:304] T 8158bbd17b994d2eb96f560bd0f1b88d P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:20:13.470084 16115 leader_election.cc:304] T 9e65676b21b748728ad39668cfe722ec P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: cb82cc7bd6db46d980c75237a1402860
I20260709 13:20:13.439425 19185 raft_consensus.cc:2804] T 829b4ad8c5a54eababeea1181b960d4b P cb82cc7bd6db46d980c75237a1402860 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:20:13.440614 18587 raft_consensus.cc:493] T 8a878999c0024d2abe517484180290fc P 1ca2725d290e47d68b1a4b609c9816ac [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:13.481412 16265 leader_election.cc:304] T d16dfad0c3b540e5b4b86492ddf87208 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
W20260709 13:20:13.483194 16266 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:13.487529 18114 leader_election.cc:290] T 66f75124215f4ca6bef831e174a792d8 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:13.420276 18551 raft_consensus.cc:515] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:13.490216 16219 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57604 (request call id 2205) took 1448 ms. Trace:
W20260709 13:20:13.493721 16151 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40844 (request call id 2018) took 2379 ms (client timeout 2230 ms). Trace:
W20260709 13:20:13.494161 16151 rpcz_store.cc:269] 0709 13:20:11.110173 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:11.131639 (+ 21466us) service_pool.cc:224] Handling call
0709 13:20:13.489732 (+2358093us) spinlock_profiling.cc:243] Waited 978 ms on lock 0x7b540011add4. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:20:13.489854 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2249333120}
I20260709 13:20:13.429925 19113 raft_consensus.cc:515] T 02838c86531d4f98bf58d1c22b8baf77 P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:13.495796 19250 leader_election.cc:290] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:13.491835 16219 rpcz_store.cc:276] 0709 13:20:12.041847 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:12.042062 (+   215us) service_pool.cc:224] Handling call
0709 13:20:12.070558 (+ 28496us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:12.400017 (+329459us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:13.489995 (+1089978us) spinlock_profiling.cc:243] Waited 316 ms on lock 0x7b54002977d4. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:20:13.490158 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":725665536}
I20260709 13:20:13.497597 19113 leader_election.cc:290] T 02838c86531d4f98bf58d1c22b8baf77 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:13.497869 19134 raft_consensus.cc:2749] T d16dfad0c3b540e5b4b86492ddf87208 P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:20:13.483714 19193 raft_consensus.cc:515] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:13.489758 16821 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49896 (request call id 953) took 2371 ms. Trace:
I20260709 13:20:13.500289 16821 rpcz_store.cc:276] 0709 13:20:11.118300 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:11.118405 (+   105us) service_pool.cc:224] Handling call
0709 13:20:11.202151 (+ 83746us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:20:11.261948 (+ 59797us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:11.261990 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:20:11.299465 (+ 37475us) log.cc:844] Serialized 26 byte log entry
0709 13:20:12.870434 (+1570969us) spinlock_profiling.cc:243] Waited 1.69 ms on lock 0x7b180038a5e0. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b693ade1 00007f69b69475b1 00007f69ada54e07 00007f69b68afab9 00007f69ada7169b 00007f69ada6f93f 00007f69ada71c02 00007f69b06f920b 00007f69b06fed23 00007f69b06ff0d1 00007f69b0b1aa0f
0709 13:20:13.151879 (+281445us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:20:13.151930 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:13.152004 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:20:13.483720 (+331716us) raft_consensus.cc:1685] finished
0709 13:20:13.483738 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:13.489691 (+  5953us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":77218,"raft.run_cpu_time_us":138,"raft.run_wall_time_us":10531,"spinlock_wait_cycles":3876224,"thread_start_us":329223,"threads_started":2,"wal-append.queue_time_us":1868151}
I20260709 13:20:13.487196 19244 raft_consensus.cc:2804] T ecc1e363230043fe9fef390f0b979f20 P 1ca2725d290e47d68b1a4b609c9816ac [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:20:13.480273 19066 raft_consensus.cc:493] T d2ab0ac8347641e6955e9bce7d90c1a2 P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:13.488276 18944 raft_consensus.cc:2804] T 9e65676b21b748728ad39668cfe722ec P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:20:13.472055 19153 raft_consensus.cc:697] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac [term 8 LEADER]: Becoming Leader. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:20:13.523941 16217 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57604 (request call id 2201) took 1514 ms. Trace:
I20260709 13:20:13.483541 16225 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57604 (request call id 2206) took 1429 ms. Trace:
I20260709 13:20:13.479699 19275 consensus_queue.cc:1048] T d16dfad0c3b540e5b4b86492ddf87208 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.034s
I20260709 13:20:13.437547 19202 consensus_queue.cc:1048] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.046s
I20260709 13:20:13.402886 19235 consensus_queue.cc:1059] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.042s
I20260709 13:20:13.458202 19209 consensus_queue.cc:1048] T 8d4ab870a05347868956aa8b841b2981 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.046s
I20260709 13:20:13.482921 19191 raft_consensus.cc:2749] T 8158bbd17b994d2eb96f560bd0f1b88d P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:20:13.486851 19197 raft_consensus.cc:2749] T 57a744f5aaf944678a63469cbf514015 P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:20:13.485016 19185 raft_consensus.cc:493] T 829b4ad8c5a54eababeea1181b960d4b P cb82cc7bd6db46d980c75237a1402860 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:20:13.558864 19185 raft_consensus.cc:3060] T 829b4ad8c5a54eababeea1181b960d4b P cb82cc7bd6db46d980c75237a1402860 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:20:13.564534 19193 leader_election.cc:290] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
W20260709 13:20:13.537874 16047 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:49252 (request call id 287) took 19869 ms (client timeout 15000 ms). Trace:
I20260709 13:20:13.529929 19042 consensus_queue.cc:1059] T 5a96373bb0e14dce8a266b2fc6c58507 P 647a343a542d4a139fe106c19095129b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: INVALID_TERM, Last received: 5.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.013s
I20260709 13:20:13.566977 19197 raft_consensus.cc:3055] T 5a96373bb0e14dce8a266b2fc6c58507 P 647a343a542d4a139fe106c19095129b [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:20:13.567485 19197 raft_consensus.cc:740] T 5a96373bb0e14dce8a266b2fc6c58507 P 647a343a542d4a139fe106c19095129b [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:20:13.568452 19197 consensus_queue.cc:260] T 5a96373bb0e14dce8a266b2fc6c58507 P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.2, Last appended by leader: 2, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:13.569774 19197 raft_consensus.cc:3060] T 5a96373bb0e14dce8a266b2fc6c58507 P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:20:13.469447 19210 raft_consensus.cc:515] T 23f82224f4564fea93572e8206bad1fe P cb82cc7bd6db46d980c75237a1402860 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:13.572727 19210 leader_election.cc:290] T 23f82224f4564fea93572e8206bad1fe P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:13.536038 16217 rpcz_store.cc:276] 0709 13:20:12.009517 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:12.009725 (+   208us) service_pool.cc:224] Handling call
0709 13:20:12.029599 (+ 19874us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:12.029844 (+   245us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:20:12.364492 (+334648us) log.cc:844] Serialized 14 byte log entry
0709 13:20:13.211099 (+846607us) spinlock_profiling.cc:243] Waited 7.16 ms on lock 0x7b1800336ca0. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b693ade1 00007f69b69475b1 00007f69ad9ecc0a 00007f69b712c219 00007f69ada74a40 00007f69ada6f4e4 00007f69ada6f455 00007f69b0b57060 00007f69b0b69716 00007f69b0b7a287 00007f69b0b7a208
0709 13:20:13.521776 (+310677us) log.cc:844] Serialized 14 byte log entry
0709 13:20:13.523442 (+  1666us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:20:13.523516 (+    74us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:20:13.523577 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:13.523697 (+   120us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:13.523869 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":287234,"prepare.queue_time_us":248554,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":38,"raft.queue_time_us":736738,"raft.run_cpu_time_us":131,"raft.run_wall_time_us":8343,"spinlock_wait_cycles":24324352,"thread_start_us":718233,"threads_started":4,"wal-append.queue_time_us":251467}
W20260709 13:20:13.543324 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 62037.6us
I20260709 13:20:13.573755 18551 leader_election.cc:290] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:13.534369 18944 raft_consensus.cc:493] T 9e65676b21b748728ad39668cfe722ec P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Starting leader election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:20:13.574738 18944 raft_consensus.cc:3060] T 9e65676b21b748728ad39668cfe722ec P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:20:13.575420 16265 leader_election.cc:304] T 84f2da98864b4425bf7c0055951fdd4b P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:20:13.576036 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68a6838d82ef4cac83507f906139d90f" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:13.576336 19264 raft_consensus.cc:493] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:20:13.576946 16298 raft_consensus.cc:2410] T 68a6838d82ef4cac83507f906139d90f P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b for term 9 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260709 13:20:13.576996 19264 raft_consensus.cc:515] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:13.522832 19244 raft_consensus.cc:493] T ecc1e363230043fe9fef390f0b979f20 P 1ca2725d290e47d68b1a4b609c9816ac [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:13.580024 19244 raft_consensus.cc:3060] T ecc1e363230043fe9fef390f0b979f20 P 1ca2725d290e47d68b1a4b609c9816ac [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:20:13.550441 19250 raft_consensus.cc:3055] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:20:13.583339 19250 raft_consensus.cc:740] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:20:13.584280 19250 consensus_queue.cc:260] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:13.526620 18758 raft_consensus.cc:697] T 1df93779209748b9860f6d893dd3de53 P 1ca2725d290e47d68b1a4b609c9816ac [term 7 LEADER]: Becoming Leader. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:20:13.535045 19066 raft_consensus.cc:515] T d2ab0ac8347641e6955e9bce7d90c1a2 P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:13.541746 19282 raft_consensus.cc:2804] T 26354361e9b044cea1c2b86c4882fe92 P cb82cc7bd6db46d980c75237a1402860 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:20:13.603936 19282 raft_consensus.cc:697] T 26354361e9b044cea1c2b86c4882fe92 P cb82cc7bd6db46d980c75237a1402860 [term 6 LEADER]: Becoming Leader. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:20:13.605302 19066 leader_election.cc:290] T d2ab0ac8347641e6955e9bce7d90c1a2 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:13.604944 19282 consensus_queue.cc:237] T 26354361e9b044cea1c2b86c4882fe92 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:13.538697 19188 leader_election.cc:290] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 7 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:20:13.554186 16219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74ca8140f35c4ae29e8a664f6bdfc829" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:13.607419 19134 consensus_queue.cc:1059] T 8158bbd17b994d2eb96f560bd0f1b88d P 90f3da57796b488c9087a2f523adf045 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:20:13.491796 18587 raft_consensus.cc:515] T 8a878999c0024d2abe517484180290fc P 1ca2725d290e47d68b1a4b609c9816ac [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:13.591907 19251 raft_consensus.cc:493] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:13.609196 19251 raft_consensus.cc:515] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:13.610116 18587 leader_election.cc:290] T 8a878999c0024d2abe517484180290fc P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:13.555374 19246 raft_consensus.cc:2749] T 8a878999c0024d2abe517484180290fc P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:20:13.555799 19267 log.cc:927] Time spent T 3884f864a96d428785895e5463e5728f P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 0.307s	user 0.000s	sys 0.001s
I20260709 13:20:13.538640 16225 rpcz_store.cc:276] 0709 13:20:12.053850 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:12.054239 (+   389us) service_pool.cc:224] Handling call
0709 13:20:12.084564 (+ 30325us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:12.084767 (+   203us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:20:12.100020 (+ 15253us) log.cc:844] Serialized 14 byte log entry
0709 13:20:13.482948 (+1382928us) log.cc:844] Serialized 14 byte log entry
0709 13:20:13.483142 (+   194us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:20:13.483218 (+    76us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:20:13.483280 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:13.483358 (+    78us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:13.483482 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":841669,"prepare.queue_time_us":176933,"prepare.run_cpu_time_us":100,"prepare.run_wall_time_us":103,"raft.queue_time_us":778453,"raft.run_cpu_time_us":273,"raft.run_wall_time_us":55017,"thread_start_us":655875,"threads_started":3,"wal-append.queue_time_us":592753}
I20260709 13:20:13.572803 16293 raft_consensus.cc:1275] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 6. (term mismatch)
I20260709 13:20:13.612653 16293 pending_rounds.cc:85] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 90f3da57796b488c9087a2f523adf045: Aborting all ops after (but not including) 5
I20260709 13:20:13.612911 16293 pending_rounds.cc:107] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 90f3da57796b488c9087a2f523adf045: Aborting uncommitted NO_OP operation due to leader change: 4.6
I20260709 13:20:13.613140 16293 raft_consensus.cc:2889] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:20:13.586886 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74ca8140f35c4ae29e8a664f6bdfc829" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:13.587404 16147 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40844 (request call id 2026) took 2460 ms. Trace:
I20260709 13:20:13.550164 19153 consensus_queue.cc:237] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:13.552058 18743 raft_consensus.cc:515] T 434bff3082d141838068f323bdb0d1ed P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:13.591437 16291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5ad4eb281564936a59f41e387a47156" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:20:13.626021 16291 raft_consensus.cc:3060] T e5ad4eb281564936a59f41e387a47156 P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:20:13.591578 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6f07842d3aa47858df5665874c86f3a" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:20:13.627095 16299 raft_consensus.cc:2376] T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 647a343a542d4a139fe106c19095129b in term 7. Re-sending same reply.
I20260709 13:20:13.585618 19250 raft_consensus.cc:3060] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:20:13.624388 19048 consensus_queue.cc:1059] T 02838c86531d4f98bf58d1c22b8baf77 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.004s
I20260709 13:20:13.593776 19080 raft_consensus.cc:2749] T 84f2da98864b4425bf7c0055951fdd4b P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:20:13.594766 16265 leader_election.cc:304] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 647a343a542d4a139fe106c19095129b
I20260709 13:20:13.613752 16147 rpcz_store.cc:276] 0709 13:20:11.126928 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:11.395968 (+269040us) service_pool.cc:224] Handling call
0709 13:20:13.490837 (+2094869us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:13.496581 (+  5744us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:13.496613 (+    32us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:20:13.496656 (+    43us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:20:13.496690 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:13.496755 (+    65us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:13.587205 (+ 90450us) spinlock_profiling.cc:243] Waited 2.09 s on lock 0x7b540011add0. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5e8ad 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:20:13.587340 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5405,"raft.queue_time_us":9002,"raft.run_cpu_time_us":140,"raft.run_wall_time_us":11215,"spinlock_wait_cycles":4816857856}
W20260709 13:20:13.626477 16191 leader_election.cc:336] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Timed out: RequestConsensusVote RPC to 127.14.177.129:36741 timed out after 2.230s (SENT)
I20260709 13:20:13.632836 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02838c86531d4f98bf58d1c22b8baf77" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:13.632833 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57a744f5aaf944678a63469cbf514015" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:13.636220 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02838c86531d4f98bf58d1c22b8baf77" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
W20260709 13:20:13.637431 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 198846us
I20260709 13:20:13.638566 16190 leader_election.cc:304] T 8cc959fb7f704fba9009c0595467d074 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 647a343a542d4a139fe106c19095129b
I20260709 13:20:13.640609 16190 leader_election.cc:304] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:20:13.612279 19209 raft_consensus.cc:493] T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:20:13.630586 19134 consensus_queue.cc:1243] T 8158bbd17b994d2eb96f560bd0f1b88d P 90f3da57796b488c9087a2f523adf045 [LEADER]: Peer 1ca2725d290e47d68b1a4b609c9816ac log is divergent from this leader: its last log entry 5.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260709 13:20:13.630196 18896 raft_consensus.cc:493] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:20:13.661213 16296 raft_consensus.cc:1275] T 2794af5b99984efebdee564035841cb0 P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
W20260709 13:20:13.648823 16115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 178822us
W20260709 13:20:13.661767 16117 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:20:13.662391 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 226977us
I20260709 13:20:13.677214 16115 leader_election.cc:304] T 2794af5b99984efebdee564035841cb0 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860
W20260709 13:20:13.665306 16192 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 322954us
I20260709 13:20:13.643332 18587 raft_consensus.cc:2804] T 8cc959fb7f704fba9009c0595467d074 P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:20:13.643792 19251 leader_election.cc:290] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:12.869127 16297 rpcz_store.cc:276] 0709 13:20:09.017556 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:09.021811 (+  4255us) service_pool.cc:224] Handling call
0709 13:20:09.027191 (+  5380us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:09.027414 (+   223us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:20:10.490628 (+1463214us) log.cc:844] Serialized 14 byte log entry
0709 13:20:12.774645 (+2284017us) log.cc:844] Serialized 14 byte log entry
0709 13:20:12.825708 (+ 51063us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:20:12.825834 (+   126us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:20:12.826427 (+   593us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:12.826519 (+    92us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:12.826667 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3348127,"prepare.queue_time_us":1675894,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":40,"raft.queue_time_us":1894788,"raft.run_cpu_time_us":167,"raft.run_wall_time_us":15632,"spinlock_wait_cycles":4177792,"thread_start_us":441835,"threads_started":3,"wal-append.queue_time_us":402300,"wal-append.run_cpu_time_us":1813,"wal-append.run_wall_time_us":1084273}
I20260709 13:20:13.673902 16291 raft_consensus.cc:2468] T e5ad4eb281564936a59f41e387a47156 P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 8.
I20260709 13:20:13.646541 19264 leader_election.cc:290] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:20:13.661250 19068 raft_consensus.cc:2764] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Leader election decision vote started in defunct term 8: won
I20260709 13:20:13.698513 16220 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57604 (request call id 2198) took 1691 ms. Trace:
W20260709 13:20:13.686981 16117 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:20:13.699532 16117 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:13.678068 16790 leader_election.cc:304] T 57a744f5aaf944678a63469cbf514015 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:20:13.700811 19193 raft_consensus.cc:2804] T 57a744f5aaf944678a63469cbf514015 P cb82cc7bd6db46d980c75237a1402860 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:20:13.701318 19193 raft_consensus.cc:493] T 57a744f5aaf944678a63469cbf514015 P cb82cc7bd6db46d980c75237a1402860 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:20:13.701910 19193 raft_consensus.cc:3060] T 57a744f5aaf944678a63469cbf514015 P cb82cc7bd6db46d980c75237a1402860 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:20:13.698964 16220 rpcz_store.cc:276] 0709 13:20:12.007269 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:12.007408 (+   139us) service_pool.cc:224] Handling call
0709 13:20:12.026393 (+ 18985us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:12.026650 (+   257us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:20:12.527243 (+500593us) log.cc:844] Serialized 14 byte log entry
0709 13:20:13.389911 (+862668us) log.cc:844] Serialized 14 byte log entry
0709 13:20:13.436901 (+ 46990us) log.cc:844] Serialized 14 byte log entry
0709 13:20:13.698142 (+261241us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:20:13.698204 (+    62us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:20:13.698258 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:13.698337 (+    79us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:13.698457 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":681696,"prepare.queue_time_us":331712,"prepare.run_cpu_time_us":163,"prepare.run_wall_time_us":4314,"raft.queue_time_us":747135,"raft.run_cpu_time_us":310,"raft.run_wall_time_us":19655,"thread_start_us":908843,"threads_started":4,"wal-append.queue_time_us":246109}
I20260709 13:20:13.710170 16790 leader_election.cc:304] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
W20260709 13:20:13.565874 16047 rpcz_store.cc:269] 0709 13:19:53.522554 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:53.522840 (+   286us) service_pool.cc:224] Handling call
0709 13:20:13.358650 (+19835810us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:19:59.537825 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:19:59.538594 (+   769us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:19:59.538640 (+    46us) write_op.cc:435] Acquired schema lock
0709 13:19:59.538658 (+    18us) tablet.cc:662] Decoding operations
0709 13:19:59.548407 (+  9749us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:19:59.577577 (+ 29170us) write_op.cc:641] Partition lock acquired for write op
0709 13:19:59.580800 (+  3223us) tablet.cc:685] Acquiring locks for 11 operations
0709 13:19:59.879532 (+298732us) tablet.cc:701] Row locks acquired
0709 13:19:59.945879 (+ 66347us) write_op.cc:260] PREPARE: finished
0709 13:20:00.000170 (+ 54291us) write_op.cc:270] Start()
0709 13:20:00.022046 (+ 21876us) write_op.cc:276] Timestamp: P: 1783603199988432 usec, L: 0
0709 13:20:00.031046 (+  9000us) op_driver.cc:348] REPLICATION: starting
0709 13:20:00.064710 (+ 33664us) log.cc:844] Serialized 4484 byte log entry
0709 13:20:02.199383 (+2134673us) op_driver.cc:464] REPLICATION: finished
0709 13:20:02.308857 (+109474us) write_op.cc:301] APPLY: starting
0709 13:20:02.308952 (+    95us) tablet.cc:1367] starting BulkCheckPresence
0709 13:20:02.309239 (+   287us) tablet.cc:1370] finished BulkCheckPresence
0709 13:20:02.309374 (+   135us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:20:02.388086 (+ 78712us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:20:02.388469 (+   383us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=11
0709 13:20:02.388522 (+    53us) write_op.cc:312] APPLY: finished
0709 13:20:02.388953 (+   431us) log.cc:844] Serialized 83 byte log entry
0709 13:20:02.389152 (+   199us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:20:02.389298 (+   146us) write_op.cc:454] Released schema lock
0709 13:20:02.389366 (+    68us) write_op.cc:341] FINISH: Updating metrics
0709 13:20:02.392678 (+  3312us) spinlock_profiling.cc:243] Waited 3.73 ms on lock 0x7b54003cdf54. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5ae28 00007f69b4595581 00007f69b4594d58 00007f69b45999f1 00007f69b45999a9 00007f69b4599939 00007f69b4599901 00007f69b4598b2d 000000000038f354 000000000038f199
0709 13:20:02.392898 (+   220us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:20:02.392924 (+    26us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":17611,"reported_tablets":32,"child_traces":[["op",{"apply.queue_time_us":95885,"apply.run_cpu_time_us":4240,"apply.run_wall_time_us":80906,"num_ops":11,"prepare.queue_time_us":263016,"prepare.run_cpu_time_us":55575,"prepare.run_wall_time_us":2855104,"raft.queue_time_us":1417378,"raft.run_cpu_time_us":22024,"raft.run_wall_time_us":140494,"replication_time_us":2148777,"spinlock_wait_cycles":8580352,"thread_start_us":1842494,"threads_started":4,"wal-append.queue_time_us":615286,"wal-append.run_cpu_time_us":20769,"wal-append.run_wall_time_us":2690075}]]}
I20260709 13:20:13.645154 19148 raft_consensus.cc:515] T 9e65676b21b748728ad39668cfe722ec P cb82cc7bd6db46d980c75237a1402860 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:20:13.736845 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 96323.6us
I20260709 13:20:13.737722 16190 leader_election.cc:304] T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 647a343a542d4a139fe106c19095129b
I20260709 13:20:13.708472 19057 raft_consensus.cc:2749] T 5a959dbecc2648eb85d1cc550f5a3245 P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:20:13.685027 18857 raft_consensus.cc:2749] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:20:13.747128 16115 leader_election.cc:304] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac
I20260709 13:20:13.748879 16115 leader_election.cc:304] T 84f2da98864b4425bf7c0055951fdd4b P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:20:13.715891 19274 consensus_queue.cc:1059] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.031s
I20260709 13:20:13.708145 19246 raft_consensus.cc:2749] T 2794af5b99984efebdee564035841cb0 P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:20:13.750401 19037 raft_consensus.cc:2804] T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:20:13.746554 19253 raft_consensus.cc:2804] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:20:13.741802 18758 consensus_queue.cc:237] T 1df93779209748b9860f6d893dd3de53 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:13.695624 19188 consensus_queue.cc:1048] T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.057s
I20260709 13:20:13.755993 19275 consensus_queue.cc:1048] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.018s
I20260709 13:20:13.695858 19210 consensus_queue.cc:1048] T 66f75124215f4ca6bef831e174a792d8 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.056s
I20260709 13:20:13.694803 19209 raft_consensus.cc:515] T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:13.695142 18896 raft_consensus.cc:515] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:13.746686 18587 raft_consensus.cc:697] T 8cc959fb7f704fba9009c0595467d074 P 1ca2725d290e47d68b1a4b609c9816ac [term 7 LEADER]: Becoming Leader. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:20:13.753587 19179 raft_consensus.cc:2804] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 13:20:13.768738 16264 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:20:13.783488 19208 log.cc:927] Time spent T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 0.636s	user 0.005s	sys 0.000s
W20260709 13:20:13.781231 16117 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:13.804512 16115 leader_election.cc:304] T 68a6838d82ef4cac83507f906139d90f P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:20:13.761206 18743 leader_election.cc:290] T 434bff3082d141838068f323bdb0d1ed P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 8 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:13.771564 19188 consensus_queue.cc:1243] T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Peer 647a343a542d4a139fe106c19095129b log is divergent from this leader: its last log entry 5.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 13:20:13.759570 19253 raft_consensus.cc:493] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Starting leader election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:20:13.802199 19275 raft_consensus.cc:493] T e5ad4eb281564936a59f41e387a47156 P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:13.759718 19142 raft_consensus.cc:3055] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:20:13.760864 19148 leader_election.cc:290] T 9e65676b21b748728ad39668cfe722ec P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 8 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:13.821174 16115 leader_election.cc:304] T e5ad4eb281564936a59f41e387a47156 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
W20260709 13:20:13.869596 16115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 48509.4us
I20260709 13:20:13.763404 19235 consensus_queue.cc:1048] T 19baa7bc367844c288bca8f81439c413 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.062s
I20260709 13:20:13.821233 19213 raft_consensus.cc:2749] T 68a6838d82ef4cac83507f906139d90f P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:20:13.759410 19037 raft_consensus.cc:493] T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Starting leader election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:20:13.806981 19179 raft_consensus.cc:493] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Starting leader election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:20:13.872714 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31969392ccdb43b491911a319e8bc7e1" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:13.875022 16191 leader_election.cc:304] T 746aabf856ab401c9ef5e859371e7b67 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
I20260709 13:20:13.758903 19292 raft_consensus.cc:2749] T 84f2da98864b4425bf7c0055951fdd4b P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:20:13.803354 19113 raft_consensus.cc:493] T af57996a8be84d398f0f023e8d08ef77 P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:20:13.783631 19289 consensus_queue.cc:1048] T c9944d5e4b8c4109af9955c4f08c9095 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.095s
I20260709 13:20:13.812227 19247 consensus_queue.cc:1059] T 7c2095a5f4ad49408dbc7f69c606120d P 647a343a542d4a139fe106c19095129b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 1, Time since last communication: 0.055s
I20260709 13:20:13.885911 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d16dfad0c3b540e5b4b86492ddf87208" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:13.902040 16298 raft_consensus.cc:2468] T d16dfad0c3b540e5b4b86492ddf87208 P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 6.
I20260709 13:20:13.862015 19253 raft_consensus.cc:3060] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:20:13.869429 19280 log.cc:927] Time spent T 434bff3082d141838068f323bdb0d1ed P cb82cc7bd6db46d980c75237a1402860: Append to log took a long time: real 0.115s	user 0.001s	sys 0.000s
I20260709 13:20:13.818285 19088 raft_consensus.cc:515] T 9e65676b21b748728ad39668cfe722ec P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:13.737190 19185 raft_consensus.cc:515] T 829b4ad8c5a54eababeea1181b960d4b P cb82cc7bd6db46d980c75237a1402860 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:20:13.881908 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 287197us
W20260709 13:20:13.888082 16189 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:13.888831 16293 raft_consensus.cc:2468] T 31969392ccdb43b491911a319e8bc7e1 P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 5.
I20260709 13:20:13.892560 16291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5ad4eb281564936a59f41e387a47156" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:13.924333 16291 raft_consensus.cc:2376] T e5ad4eb281564936a59f41e387a47156 P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 647a343a542d4a139fe106c19095129b in term 8. Re-sending same reply.
W20260709 13:20:13.924585 16264 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:13.892989 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3884f864a96d428785895e5463e5728f" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:20:13.895581 16824 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49906 (request call id 648) took 2494 ms. Trace:
W20260709 13:20:13.887845 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 177773us
I20260709 13:20:13.866467 19142 raft_consensus.cc:740] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:20:13.884879 19176 raft_consensus.cc:2804] T e5ad4eb281564936a59f41e387a47156 P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:20:13.925315 16297 raft_consensus.cc:2376] T 3884f864a96d428785895e5463e5728f P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 647a343a542d4a139fe106c19095129b in term 6. Re-sending same reply.
I20260709 13:20:13.919427 16145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0eab25ce4c64e018a041beeda2867db" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
W20260709 13:20:13.920524 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 45603us
I20260709 13:20:13.925796 16824 rpcz_store.cc:276] 0709 13:20:11.401026 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:11.705296 (+304270us) service_pool.cc:224] Handling call
0709 13:20:12.457163 (+751867us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:12.574514 (+117351us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:13.895321 (+1320807us) spinlock_profiling.cc:243] Waited 745 ms on lock 0x7b5400769150. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5e8ad 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:20:13.895510 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1714458880}
I20260709 13:20:13.933598 16142 raft_consensus.cc:1275] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 13:20:13.892666 19179 raft_consensus.cc:3060] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:20:13.889146 19148 raft_consensus.cc:2749] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:20:13.888978 19037 raft_consensus.cc:3060] T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:20:13.900995 18599 raft_consensus.cc:493] T c0eab25ce4c64e018a041beeda2867db P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:13.871285 19275 raft_consensus.cc:515] T e5ad4eb281564936a59f41e387a47156 P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:13.955188 16151 raft_consensus.cc:1240] T 746aabf856ab401c9ef5e859371e7b67 P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 6. Current term is 7. Ops: []
I20260709 13:20:13.948866 16142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62fe3346cd3c42c4a55e14af9073bca4" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:13.956574 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2372981006d4b218215c28964dea623" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:13.954779 16146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66f75124215f4ca6bef831e174a792d8" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:13.957546 16146 raft_consensus.cc:2468] T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 7.
I20260709 13:20:13.956481 16143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5718c623ebee4d2bb770f0f9ddb2f164" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:20:13.955396 16265 leader_election.cc:304] T c4f65cdf00b748bb963ee6e81ef43b79 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:20:13.951051 16149 raft_consensus.cc:1275] T 2794af5b99984efebdee564035841cb0 P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:20:13.963676 16265 leader_election.cc:304] T cdbd78acb4cf46298e0de8ba0d130e19 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860
I20260709 13:20:13.933755 19296 raft_consensus.cc:2804] T 746aabf856ab401c9ef5e859371e7b67 P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:20:13.966202 16115 leader_election.cc:304] T d16dfad0c3b540e5b4b86492ddf87208 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:20:13.974349 16146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e65676b21b748728ad39668cfe722ec" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:20:13.907442 19113 raft_consensus.cc:515] T af57996a8be84d398f0f023e8d08ef77 P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:13.955220 19176 raft_consensus.cc:697] T e5ad4eb281564936a59f41e387a47156 P 647a343a542d4a139fe106c19095129b [term 8 LEADER]: Becoming Leader. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:20:13.905108 18587 consensus_queue.cc:237] T 8cc959fb7f704fba9009c0595467d074 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:14.006364 16151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a878999c0024d2abe517484180290fc" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:14.010186 16151 raft_consensus.cc:2468] T 8a878999c0024d2abe517484180290fc P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 6.
I20260709 13:20:13.971266 19048 raft_consensus.cc:2749] T cdbd78acb4cf46298e0de8ba0d130e19 P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:20:13.962074 18599 raft_consensus.cc:515] T c0eab25ce4c64e018a041beeda2867db P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:13.988998 19251 raft_consensus.cc:2804] T c4f65cdf00b748bb963ee6e81ef43b79 P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:20:13.989699 19201 raft_consensus.cc:2804] T d16dfad0c3b540e5b4b86492ddf87208 P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:20:14.021368 16144 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40844 (request call id 2028) took 2809 ms. Trace:
I20260709 13:20:13.966326 19148 consensus_queue.cc:1048] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.041s
I20260709 13:20:13.993744 19296 raft_consensus.cc:697] T 746aabf856ab401c9ef5e859371e7b67 P 1ca2725d290e47d68b1a4b609c9816ac [term 7 LEADER]: Becoming Leader. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:20:14.029160 16148 raft_consensus.cc:1275] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 13:20:13.990684 19291 raft_consensus.cc:493] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:20:13.965482 19300 consensus_queue.cc:1048] T 66f75124215f4ca6bef831e174a792d8 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.057s
I20260709 13:20:14.023993 16144 rpcz_store.cc:276] 0709 13:20:11.211808 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:11.450469 (+238661us) service_pool.cc:224] Handling call
0709 13:20:11.459697 (+  9228us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:20:11.546944 (+ 87247us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:11.546974 (+    30us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:20:11.548164 (+  1190us) log.cc:844] Serialized 52 byte log entry
0709 13:20:11.789241 (+241077us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:20:11.848973 (+ 59732us) log.cc:844] Serialized 14 byte log entry
0709 13:20:11.977486 (+128513us) spinlock_profiling.cc:243] Waited 5.1 ms on lock 0x7b180009e860. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b693ade1 00007f69b69475b1 00007f69ad9ecc0a 00007f69b712c219 00007f69ada74a40 00007f69ada6f4e4 00007f69ada6f455 00007f69b45573c7 00007f69b0b699a7 00007f69b0b790bf 00007f69b0b79038
0709 13:20:11.977686 (+   200us) log.cc:844] Serialized 14 byte log entry
0709 13:20:11.977878 (+   192us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:11.977950 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:20:12.823611 (+845661us) spinlock_profiling.cc:243] Waited 2.57 ms on lock 0x7b180009e860. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b693ade1 00007f69b69475b1 00007f69ada723f3 00007f69ada747c1 00007f69ada74779 00007f69ada74709 00007f69ada746d1 00007f69ada739cd 000000000038f354 000000000038f199 00007f69ada5583a
0709 13:20:13.115412 (+291801us) spinlock_profiling.cc:243] Waited 18.3 ms on lock 0x7b180009e860. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b693ade1 00007f69b69475b1 00007f69ada723f3 00007f69ada747c1 00007f69ada74779 00007f69ada74709 00007f69ada746d1 00007f69ada739cd 000000000038f354 000000000038f199 00007f69ada5583a
0709 13:20:13.633263 (+517851us) spinlock_profiling.cc:243] Waited 1.35 ms on lock 0x7b4400022b4c. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69ad8fadb1 00007f69ad998c85 00007f69ad9996a2 00007f69b070b1db 00007f69b0706aaf 00007f69b06fa86e 00007f69b06f9bb6 00007f69b06f97d8 00007f69b0703221 00007f69b07031d9 00007f69b0703169
0709 13:20:14.021009 (+387746us) raft_consensus.cc:1685] finished
0709 13:20:14.021042 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:14.021265 (+   223us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":134505,"prepare.queue_time_us":178538,"prepare.run_cpu_time_us":61,"prepare.run_wall_time_us":63,"raft.queue_time_us":226580,"raft.run_cpu_time_us":355,"raft.run_wall_time_us":14432,"spinlock_wait_cycles":62859520,"thread_start_us":173744,"threads_started":1,"wal-append.queue_time_us":281718}
I20260709 13:20:14.043195 16145 raft_consensus.cc:3060] T c0eab25ce4c64e018a041beeda2867db P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:20:14.044420 16151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f82224f4564fea93572e8206bad1fe" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:14.045182 16151 raft_consensus.cc:2468] T 23f82224f4564fea93572e8206bad1fe P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 5.
I20260709 13:20:14.047895 16191 leader_election.cc:304] T 66f75124215f4ca6bef831e174a792d8 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
I20260709 13:20:14.039546 18896 leader_election.cc:290] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:14.055439 19284 raft_consensus.cc:493] T 8c11074579b046d99e585c2818e18a2c P 1ca2725d290e47d68b1a4b609c9816ac [term 6 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:20:14.056241 19284 raft_consensus.cc:515] T 8c11074579b046d99e585c2818e18a2c P 1ca2725d290e47d68b1a4b609c9816ac [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:14.011636 19142 consensus_queue.cc:260] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:14.060771 19088 leader_election.cc:290] T 9e65676b21b748728ad39668cfe722ec P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 8 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:14.065737 16265 leader_election.cc:304] T 8638ee363273426db1393dd81e5c7b72 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860
I20260709 13:20:14.034610 19209 leader_election.cc:290] T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:14.030239 19201 raft_consensus.cc:493] T d16dfad0c3b540e5b4b86492ddf87208 P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Starting leader election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:20:14.028059 18758 raft_consensus.cc:493] T fffab26427e24d398e2695aad40ca134 P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:20:14.073560 16147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8158bbd17b994d2eb96f560bd0f1b88d" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:14.082056 16147 raft_consensus.cc:2410] T 8158bbd17b994d2eb96f560bd0f1b88d P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac for term 8 because replica has last-logged OpId of term: 6 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260709 13:20:14.084636 16219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5ad4eb281564936a59f41e387a47156" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:20:14.084882 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02838c86531d4f98bf58d1c22b8baf77" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:20:14.085247 16219 raft_consensus.cc:3060] T e5ad4eb281564936a59f41e387a47156 P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:20:14.085466 19066 raft_consensus.cc:3055] T 8158bbd17b994d2eb96f560bd0f1b88d P 90f3da57796b488c9087a2f523adf045 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:20:14.085956 19066 raft_consensus.cc:740] T 8158bbd17b994d2eb96f560bd0f1b88d P 90f3da57796b488c9087a2f523adf045 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:20:14.086200 19185 leader_election.cc:290] T 829b4ad8c5a54eababeea1181b960d4b P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 7 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:20:14.088281 16220 raft_consensus.cc:1240] T 1df93779209748b9860f6d893dd3de53 P 1ca2725d290e47d68b1a4b609c9816ac [term 7 LEADER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 6. Current term is 7. Ops: [6.5-6.5]
W20260709 13:20:14.090322 19303 log.cc:927] Time spent T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 0.097s	user 0.001s	sys 0.000s
W20260709 13:20:14.092371 19304 log.cc:927] Time spent T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 0.070s	user 0.001s	sys 0.000s
W20260709 13:20:14.081282 19197 consensus_meta.cc:287] T 5a96373bb0e14dce8a266b2fc6c58507 P 647a343a542d4a139fe106c19095129b: Time spent flushing consensus metadata: real 0.511s	user 0.012s	sys 0.000s
I20260709 13:20:14.085508 16217 raft_consensus.cc:3060] T 02838c86531d4f98bf58d1c22b8baf77 P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:20:14.095690 16151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02838c86531d4f98bf58d1c22b8baf77" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:14.096630 16151 raft_consensus.cc:2393] T 02838c86531d4f98bf58d1c22b8baf77 P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 8: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
I20260709 13:20:14.092439 19310 raft_consensus.cc:493] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:20:14.097342 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 49529us
I20260709 13:20:14.099593 16191 leader_election.cc:304] T 8a878999c0024d2abe517484180290fc P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
W20260709 13:20:14.101437 16117 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:14.100235 16265 leader_election.cc:304] T fffab26427e24d398e2695aad40ca134 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:20:14.110311 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66f75124215f4ca6bef831e174a792d8" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:14.110940 16226 raft_consensus.cc:2468] T 66f75124215f4ca6bef831e174a792d8 P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 7.
I20260709 13:20:14.086925 19066 consensus_queue.cc:260] T 8158bbd17b994d2eb96f560bd0f1b88d P 90f3da57796b488c9087a2f523adf045 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:14.109644 16145 raft_consensus.cc:2468] T c0eab25ce4c64e018a041beeda2867db P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 8.
I20260709 13:20:14.119218 16144 raft_consensus.cc:1275] T 84f2da98864b4425bf7c0055951fdd4b P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:20:14.120990 16824 raft_consensus.cc:1275] T 19baa7bc367844c288bca8f81439c413 P cb82cc7bd6db46d980c75237a1402860 [term 7 LEARNER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 13:20:14.083712 19088 raft_consensus.cc:3060] T 8638ee363273426db1393dd81e5c7b72 P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:20:14.117105 16220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746aabf856ab401c9ef5e859371e7b67" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:14.083071 19201 raft_consensus.cc:3060] T d16dfad0c3b540e5b4b86492ddf87208 P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:20:14.130209 16818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9944d5e4b8c4109af9955c4f08c9095" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:14.131810 16220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5d59407eb545b895d8d922e0dd69fd" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 9 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:14.133853 16824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8638ee363273426db1393dd81e5c7b72" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:14.116353 19048 raft_consensus.cc:2804] T fffab26427e24d398e2695aad40ca134 P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:20:14.143013 16222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d16dfad0c3b540e5b4b86492ddf87208" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:14.121987 18896 raft_consensus.cc:2804] T 66f75124215f4ca6bef831e174a792d8 P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:20:14.150525 16219 raft_consensus.cc:2468] T e5ad4eb281564936a59f41e387a47156 P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 8.
I20260709 13:20:14.157776 19057 consensus_queue.cc:1048] T d2ab0ac8347641e6955e9bce7d90c1a2 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:20:14.150873 18896 raft_consensus.cc:493] T 66f75124215f4ca6bef831e174a792d8 P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:14.159430 18896 raft_consensus.cc:3060] T 66f75124215f4ca6bef831e174a792d8 P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:20:14.126524 19306 raft_consensus.cc:493] T af57996a8be84d398f0f023e8d08ef77 P cb82cc7bd6db46d980c75237a1402860 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:20:14.160607 19306 raft_consensus.cc:515] T af57996a8be84d398f0f023e8d08ef77 P cb82cc7bd6db46d980c75237a1402860 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:14.165337 19113 leader_election.cc:290] T af57996a8be84d398f0f023e8d08ef77 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:14.165997 19113 consensus_queue.cc:1048] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
W20260709 13:20:14.166771 18944 consensus_meta.cc:287] T 9e65676b21b748728ad39668cfe722ec P 647a343a542d4a139fe106c19095129b: Time spent flushing consensus metadata: real 0.591s	user 0.010s	sys 0.000s
I20260709 13:20:14.167261 18944 raft_consensus.cc:515] T 9e65676b21b748728ad39668cfe722ec P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:14.168357 16146 raft_consensus.cc:3060] T 9e65676b21b748728ad39668cfe722ec P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:20:14.168043 19193 raft_consensus.cc:515] T 57a744f5aaf944678a63469cbf514015 P cb82cc7bd6db46d980c75237a1402860 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:14.170272 19193 leader_election.cc:290] T 57a744f5aaf944678a63469cbf514015 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 5 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:14.144002 16817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66f75124215f4ca6bef831e174a792d8" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:14.155447 16221 raft_consensus.cc:1275] T b6f07842d3aa47858df5665874c86f3a P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:20:14.121536 19185 raft_consensus.cc:493] T 8638ee363273426db1393dd81e5c7b72 P cb82cc7bd6db46d980c75237a1402860 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:20:14.171411 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 70774.7us
I20260709 13:20:14.094727 18758 raft_consensus.cc:515] T fffab26427e24d398e2695aad40ca134 P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:14.112609 19310 raft_consensus.cc:515] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:14.122203 19289 raft_consensus.cc:493] T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:20:14.044847 19291 raft_consensus.cc:515] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:14.185557 16116 leader_election.cc:304] T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
W20260709 13:20:14.189972 16192 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 69953.9us
I20260709 13:20:14.149801 19066 raft_consensus.cc:3060] T 8158bbd17b994d2eb96f560bd0f1b88d P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:20:14.188448 16191 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:14.197928 16221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b7d1ab519044eda2d5e04a07242f25" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:14.200286 16218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2372981006d4b218215c28964dea623" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:14.201103 16218 raft_consensus.cc:2410] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 for term 8 because replica has last-logged OpId of term: 7 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 13:20:14.149246 19048 raft_consensus.cc:493] T fffab26427e24d398e2695aad40ca134 P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:20:14.210160 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 6 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:20:14.163018 19306 leader_election.cc:290] T af57996a8be84d398f0f023e8d08ef77 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:20:14.209888 16143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2372981006d4b218215c28964dea623" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:14.171659 19235 raft_consensus.cc:2804] T 8a878999c0024d2abe517484180290fc P 1ca2725d290e47d68b1a4b609c9816ac [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:20:14.076537 19176 consensus_queue.cc:237] T e5ad4eb281564936a59f41e387a47156 P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:14.211546 16144 raft_consensus.cc:1275] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
W20260709 13:20:14.185628 19250 consensus_meta.cc:287] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 1ca2725d290e47d68b1a4b609c9816ac: Time spent flushing consensus metadata: real 0.557s	user 0.008s	sys 0.004s
I20260709 13:20:14.218345 16220 raft_consensus.cc:2410] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b for term 9 because replica has last-logged OpId of term: 6 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 7.
I20260709 13:20:14.226925 16823 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49896 (request call id 958) took 2280 ms. Trace:
I20260709 13:20:14.230525 16821 raft_consensus.cc:1240] T 829b4ad8c5a54eababeea1181b960d4b P cb82cc7bd6db46d980c75237a1402860 [term 7 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 6. Current term is 7. Ops: []
I20260709 13:20:14.230670 16823 rpcz_store.cc:276] 0709 13:20:11.946982 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:12.117188 (+170206us) service_pool.cc:224] Handling call
0709 13:20:12.117568 (+   380us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:13.244108 (+1126540us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:14.226692 (+982584us) spinlock_profiling.cc:243] Waited 1.13 s on lock 0x7b54005c35d4. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:20:14.226866 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2589491968}
I20260709 13:20:14.232633 16146 raft_consensus.cc:2468] T 9e65676b21b748728ad39668cfe722ec P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 8.
I20260709 13:20:14.233489 16218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5718c623ebee4d2bb770f0f9ddb2f164" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:20:14.239790 16143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5ad4eb281564936a59f41e387a47156" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 8 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:14.191720 19284 leader_election.cc:290] T 8c11074579b046d99e585c2818e18a2c P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:14.219496 19048 raft_consensus.cc:3060] T fffab26427e24d398e2695aad40ca134 P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:20:14.210570 18944 leader_election.cc:290] T 9e65676b21b748728ad39668cfe722ec P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 7 election: Requested vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
W20260709 13:20:14.254755 19307 log.cc:927] Time spent T 8d4ab870a05347868956aa8b841b2981 P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 0.064s	user 0.000s	sys 0.001s
I20260709 13:20:14.198668 19289 raft_consensus.cc:515] T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:14.256119 19310 leader_election.cc:290] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:14.256770 16144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af57996a8be84d398f0f023e8d08ef77" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:14.138258 19296 consensus_queue.cc:237] T 746aabf856ab401c9ef5e859371e7b67 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:14.260433 19080 raft_consensus.cc:3055] T 02838c86531d4f98bf58d1c22b8baf77 P 90f3da57796b488c9087a2f523adf045 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:20:14.262323 19184 raft_consensus.cc:3060] T 8cc959fb7f704fba9009c0595467d074 P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:20:14.268914 19291 leader_election.cc:290] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:14.273623 19088 raft_consensus.cc:2749] T 8638ee363273426db1393dd81e5c7b72 P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:20:14.191560 19185 raft_consensus.cc:515] T 8638ee363273426db1393dd81e5c7b72 P cb82cc7bd6db46d980c75237a1402860 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:14.223810 19235 raft_consensus.cc:493] T 8a878999c0024d2abe517484180290fc P 1ca2725d290e47d68b1a4b609c9816ac [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:14.305352 19235 raft_consensus.cc:3060] T 8a878999c0024d2abe517484180290fc P 1ca2725d290e47d68b1a4b609c9816ac [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:20:14.307801 16300 raft_consensus.cc:2468] T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 7.
I20260709 13:20:14.309489 19185 leader_election.cc:290] T 8638ee363273426db1393dd81e5c7b72 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:14.234549 19275 leader_election.cc:290] T e5ad4eb281564936a59f41e387a47156 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:14.311450 16824 raft_consensus.cc:2468] T 8638ee363273426db1393dd81e5c7b72 P cb82cc7bd6db46d980c75237a1402860 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 6.
W20260709 13:20:14.315016 19273 log.cc:927] Time spent T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.896s	user 0.000s	sys 0.005s
I20260709 13:20:14.288633 19080 raft_consensus.cc:740] T 02838c86531d4f98bf58d1c22b8baf77 P 90f3da57796b488c9087a2f523adf045 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:20:14.293324 19310 consensus_queue.cc:1048] T 19baa7bc367844c288bca8f81439c413 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.026s
I20260709 13:20:14.316373 19080 consensus_queue.cc:260] T 02838c86531d4f98bf58d1c22b8baf77 P 90f3da57796b488c9087a2f523adf045 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:14.317696 19080 raft_consensus.cc:3060] T 02838c86531d4f98bf58d1c22b8baf77 P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:20:14.316401 19308 log.cc:927] Time spent T c4f65cdf00b748bb963ee6e81ef43b79 P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.188s	user 0.000s	sys 0.001s
W20260709 13:20:14.319845 19312 log.cc:927] Time spent T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 0.148s	user 0.000s	sys 0.001s
I20260709 13:20:14.320981 19315 consensus_queue.cc:1059] T 829b4ad8c5a54eababeea1181b960d4b P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:20:14.321168 19235 raft_consensus.cc:515] T 8a878999c0024d2abe517484180290fc P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:14.322209 18758 leader_election.cc:290] T fffab26427e24d398e2695aad40ca134 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
W20260709 13:20:14.326212 19244 consensus_meta.cc:287] T ecc1e363230043fe9fef390f0b979f20 P 1ca2725d290e47d68b1a4b609c9816ac: Time spent flushing consensus metadata: real 0.746s	user 0.010s	sys 0.002s
W20260709 13:20:14.286302 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 336895us
I20260709 13:20:14.326689 19244 raft_consensus.cc:515] T ecc1e363230043fe9fef390f0b979f20 P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:14.287439 18599 leader_election.cc:290] T c0eab25ce4c64e018a041beeda2867db P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:14.330013 19144 raft_consensus.cc:493] T d2ab0ac8347641e6955e9bce7d90c1a2 P cb82cc7bd6db46d980c75237a1402860 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:14.330621 19144 raft_consensus.cc:515] T d2ab0ac8347641e6955e9bce7d90c1a2 P cb82cc7bd6db46d980c75237a1402860 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:20:14.333534 19305 log.cc:927] Time spent T 26354361e9b044cea1c2b86c4882fe92 P cb82cc7bd6db46d980c75237a1402860: Append to log took a long time: real 0.294s	user 0.000s	sys 0.001s
I20260709 13:20:14.333652 19144 leader_election.cc:290] T d2ab0ac8347641e6955e9bce7d90c1a2 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:14.339264 19235 leader_election.cc:290] T 8a878999c0024d2abe517484180290fc P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 7 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:14.339551 19289 leader_election.cc:290] T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
W20260709 13:20:14.339901 19297 log.cc:927] Time spent T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 0.368s	user 0.003s	sys 0.000s
I20260709 13:20:14.342722 16217 raft_consensus.cc:2468] T 02838c86531d4f98bf58d1c22b8baf77 P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 8.
I20260709 13:20:14.212217 19311 raft_consensus.cc:493] T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
W20260709 13:20:14.344691 16790 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:14.345714 16224 raft_consensus.cc:2368] T ecc1e363230043fe9fef390f0b979f20 P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 for earlier term 6. Current term is 7.
I20260709 13:20:14.346900 16816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2372981006d4b218215c28964dea623" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:14.322247 19315 raft_consensus.cc:3055] T 829b4ad8c5a54eababeea1181b960d4b P 1ca2725d290e47d68b1a4b609c9816ac [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:20:14.347609 16816 raft_consensus.cc:2468] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 7.
I20260709 13:20:14.347658 19315 raft_consensus.cc:740] T 829b4ad8c5a54eababeea1181b960d4b P 1ca2725d290e47d68b1a4b609c9816ac [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:20:14.348727 19315 consensus_queue.cc:260] T 829b4ad8c5a54eababeea1181b960d4b P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:14.350701 19210 consensus_queue.cc:1059] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:20:14.351534 19210 raft_consensus.cc:3055] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:20:14.351835 19210 raft_consensus.cc:740] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:20:14.352661 19210 consensus_queue.cc:260] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:14.353778 19210 raft_consensus.cc:3060] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:20:14.356104 19037 raft_consensus.cc:515] T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:14.357263 16817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c11074579b046d99e585c2818e18a2c" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:14.358464 16825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5d59407eb545b895d8d922e0dd69fd" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:14.359696 16820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62fe3346cd3c42c4a55e14af9073bca4" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:14.360245 16820 raft_consensus.cc:2468] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 8.
I20260709 13:20:14.362922 16790 leader_election.cc:304] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
W20260709 13:20:14.362957 16265 connection.cc:256] RPC call timeout handler was delayed by -5.3945551780000187s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1531s for a response
W20260709 13:20:14.364722 16265 connection.cc:256] RPC call timeout handler was delayed by -5.29645517799986s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.164s for a response
I20260709 13:20:14.358916 19042 raft_consensus.cc:3055] T 7c2095a5f4ad49408dbc7f69c606120d P 647a343a542d4a139fe106c19095129b [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:20:14.379338 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a959dbecc2648eb85d1cc550f5a3245" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:14.382872 16217 raft_consensus.cc:2410] T 5a959dbecc2648eb85d1cc550f5a3245 P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 for term 6 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
W20260709 13:20:14.383983 16189 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:14.395720 16817 raft_consensus.cc:1240] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 [term 5 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 4. Current term is 5. Ops: []
W20260709 13:20:14.395887 16264 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:14.396080 19037 leader_election.cc:290] T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 8 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:14.387254 19042 raft_consensus.cc:740] T 7c2095a5f4ad49408dbc7f69c606120d P 647a343a542d4a139fe106c19095129b [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:20:14.397873 18896 raft_consensus.cc:515] T 66f75124215f4ca6bef831e174a792d8 P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:14.398919 19042 consensus_queue.cc:260] T 7c2095a5f4ad49408dbc7f69c606120d P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:14.399881 16192 leader_election.cc:304] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:20:14.400650 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87ac0d1f7e8a419f9bcbc04aee0e067e" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 7 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:14.400861 16192 leader_election.cc:304] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 
W20260709 13:20:14.401633 16264 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:14.358930 19201 raft_consensus.cc:515] T d16dfad0c3b540e5b4b86492ddf87208 P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:14.378573 19185 raft_consensus.cc:2749] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:20:14.404449 19201 leader_election.cc:290] T d16dfad0c3b540e5b4b86492ddf87208 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 7 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:14.400071 19042 raft_consensus.cc:3060] T 7c2095a5f4ad49408dbc7f69c606120d P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:20:14.401875 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c11074579b046d99e585c2818e18a2c" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:20:14.405920 19310 raft_consensus.cc:2804] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:20:14.411480 19310 raft_consensus.cc:493] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:14.411895 19310 raft_consensus.cc:3060] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:20:14.412703 16192 leader_election.cc:304] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:20:14.413493 18114 raft_consensus.cc:2804] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:20:14.413970 18114 raft_consensus.cc:493] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Starting leader election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:20:14.414525 18114 raft_consensus.cc:3060] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:20:14.406848 18551 raft_consensus.cc:2804] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:20:14.415472 16143 raft_consensus.cc:1240] T d16dfad0c3b540e5b4b86492ddf87208 P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 6. Current term is 7. Ops: []
I20260709 13:20:14.415639 18551 raft_consensus.cc:493] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Starting leader election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:20:14.416364 18551 raft_consensus.cc:3060] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:20:14.417191 19253 raft_consensus.cc:515] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:14.418524 16219 raft_consensus.cc:1240] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 8. Current term is 9. Ops: []
W20260709 13:20:14.420758 16264 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:14.389086 16821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af57996a8be84d398f0f023e8d08ef77" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:14.421864 16217 raft_consensus.cc:3060] T 8c11074579b046d99e585c2818e18a2c P 1ca2725d290e47d68b1a4b609c9816ac [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:20:14.422349 16821 raft_consensus.cc:2468] T af57996a8be84d398f0f023e8d08ef77 P cb82cc7bd6db46d980c75237a1402860 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 7.
I20260709 13:20:14.452584 18114 raft_consensus.cc:515] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:20:14.458110 16265 connection.cc:256] RPC call timeout handler was delayed by -5.021955177999871s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1945s for a response
I20260709 13:20:14.459928 16192 leader_election.cc:304] T af57996a8be84d398f0f023e8d08ef77 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:20:14.406435 19179 raft_consensus.cc:515] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:14.470650 18114 leader_election.cc:290] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 9 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:14.473340 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62fe3346cd3c42c4a55e14af9073bca4" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:14.361553 19311 raft_consensus.cc:515] T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:14.483659 19142 raft_consensus.cc:3060] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:20:14.496361 18896 leader_election.cc:290] T 66f75124215f4ca6bef831e174a792d8 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 8 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:14.461607 19037 raft_consensus.cc:2804] T af57996a8be84d398f0f023e8d08ef77 P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:20:14.511497 19037 raft_consensus.cc:493] T af57996a8be84d398f0f023e8d08ef77 P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Starting leader election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:20:14.511986 19037 raft_consensus.cc:3060] T af57996a8be84d398f0f023e8d08ef77 P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:20:14.511914 19048 raft_consensus.cc:515] T fffab26427e24d398e2695aad40ca134 P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:14.514597 19048 leader_election.cc:290] T fffab26427e24d398e2695aad40ca134 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 6 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:20:14.499580 16821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62fe3346cd3c42c4a55e14af9073bca4" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:20:14.515285 16821 raft_consensus.cc:3060] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:20:14.515372 16295 raft_consensus.cc:1240] T fffab26427e24d398e2695aad40ca134 P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 5. Current term is 6. Ops: []
I20260709 13:20:14.517059 16220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e4bfb9090c8460eb14fa7e0d818e7f1" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:14.518055 16220 raft_consensus.cc:2468] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 6.
I20260709 13:20:14.520716 16222 raft_consensus.cc:1275] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:20:14.500259 19253 leader_election.cc:290] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 9 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:14.500005 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada5a004ba7c46819051e3ccec71ffdb" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:14.522776 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0eab25ce4c64e018a041beeda2867db" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:14.502751 19311 leader_election.cc:290] T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:14.504609 19179 leader_election.cc:290] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 7 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
W20260709 13:20:14.507320 16192 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:14.507408 19080 raft_consensus.cc:493] T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:14.523110 16222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c74490c40243a88e62d1760b5cd814" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
W20260709 13:20:14.542729 19274 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.552s	user 0.012s	sys 0.020s
W20260709 13:20:14.543526 19274 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.567s	user 0.013s	sys 0.020s
I20260709 13:20:14.548122 16222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18ad7e3af5504571a69d10c00750a01c" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:14.549113 16220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d4ab870a05347868956aa8b841b2981" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 7 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:14.559685 16220 raft_consensus.cc:2368] T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 for earlier term 7. Current term is 8.
I20260709 13:20:14.549741 16217 raft_consensus.cc:2468] T 8c11074579b046d99e585c2818e18a2c P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 7.
I20260709 13:20:14.531322 18551 raft_consensus.cc:515] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:14.533246 16265 leader_election.cc:304] T c0eab25ce4c64e018a041beeda2867db P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac
I20260709 13:20:14.536976 16291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57a744f5aaf944678a63469cbf514015" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:20:14.538048 16296 raft_consensus.cc:1240] T 8c11074579b046d99e585c2818e18a2c P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 6. Current term is 7. Ops: []
I20260709 13:20:14.538345 16294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f82224f4564fea93572e8206bad1fe" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:14.552937 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a378d7f796f94615b3b347119a9880fd" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:14.565809 16294 raft_consensus.cc:2468] T 23f82224f4564fea93572e8206bad1fe P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 5.
I20260709 13:20:14.566013 16299 raft_consensus.cc:2468] T a378d7f796f94615b3b347119a9880fd P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 6.
I20260709 13:20:14.523476 16225 raft_consensus.cc:2368] T c0eab25ce4c64e018a041beeda2867db P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 for earlier term 7. Current term is 8.
I20260709 13:20:14.539000 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8638ee363273426db1393dd81e5c7b72" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:14.554565 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cc959fb7f704fba9009c0595467d074" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:14.539608 19080 raft_consensus.cc:515] T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:14.538672 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e65676b21b748728ad39668cfe722ec" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:20:14.605860 16297 raft_consensus.cc:2393] T 9e65676b21b748728ad39668cfe722ec P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 in current term 8: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
I20260709 13:20:14.539348 16292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ab0ac8347641e6955e9bce7d90c1a2" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
W20260709 13:20:14.537266 16789 leader_election.cc:336] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389): Timed out: RequestConsensusVote RPC to 127.14.177.131:41389 timed out after 1.554s (SENT)
I20260709 13:20:14.556761 16790 leader_election.cc:304] T 23f82224f4564fea93572e8206bad1fe P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:20:14.606794 16292 raft_consensus.cc:2468] T d2ab0ac8347641e6955e9bce7d90c1a2 P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 7.
W20260709 13:20:14.607153 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 50484.9us
I20260709 13:20:14.563614 16821 raft_consensus.cc:2468] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 9.
W20260709 13:20:14.608984 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 423490us
W20260709 13:20:14.612936 16789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 75693.6us
W20260709 13:20:14.613838 16789 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.14.177.131:41389 timed out after 2.111s (SENT) [suppressed 37 similar messages]
W20260709 13:20:14.614277 16115 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:14.603850 16220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9944d5e4b8c4109af9955c4f08c9095" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:14.604189 16300 raft_consensus.cc:2410] T 8638ee363273426db1393dd81e5c7b72 P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 for term 7 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 13:20:14.615963 16220 raft_consensus.cc:2468] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 7.
I20260709 13:20:14.617766 16223 raft_consensus.cc:1240] T 746aabf856ab401c9ef5e859371e7b67 P 1ca2725d290e47d68b1a4b609c9816ac [term 7 LEADER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 6. Current term is 7. Ops: []
I20260709 13:20:14.617673 16220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ab0ac8347641e6955e9bce7d90c1a2" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:14.620196 16818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66f75124215f4ca6bef831e174a792d8" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:20:14.619597 19048 raft_consensus.cc:2804] T c0eab25ce4c64e018a041beeda2867db P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:20:14.567201 19185 consensus_queue.cc:1048] T 2794af5b99984efebdee564035841cb0 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.057s
I20260709 13:20:14.617177 19314 raft_consensus.cc:2804] T 23f82224f4564fea93572e8206bad1fe P cb82cc7bd6db46d980c75237a1402860 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:20:14.628954 16219 raft_consensus.cc:1240] T 8a878999c0024d2abe517484180290fc P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 6. Current term is 7. Ops: []
I20260709 13:20:14.629127 19314 raft_consensus.cc:493] T 23f82224f4564fea93572e8206bad1fe P cb82cc7bd6db46d980c75237a1402860 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:14.629509 19314 raft_consensus.cc:3060] T 23f82224f4564fea93572e8206bad1fe P cb82cc7bd6db46d980c75237a1402860 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:20:14.608651 19197 raft_consensus.cc:493] T 8cc959fb7f704fba9009c0595467d074 P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:14.630357 16219 raft_consensus.cc:1240] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 6. Current term is 7. Ops: [4.4-6.5]
W20260709 13:20:14.623106 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 89998.6us
I20260709 13:20:14.622301 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a378d7f796f94615b3b347119a9880fd" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:14.623638 16222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c2095a5f4ad49408dbc7f69c606120d" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:20:14.603348 19274 raft_consensus.cc:493] T ee9ed67006144797b1de6ea8c2013a68 P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:20:14.644105 16265 leader_election.cc:304] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:20:14.623840 19048 raft_consensus.cc:697] T c0eab25ce4c64e018a041beeda2867db P 90f3da57796b488c9087a2f523adf045 [term 7 LEADER]: Becoming Leader. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:20:14.628489 18551 leader_election.cc:290] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 8 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:14.652037 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fffab26427e24d398e2695aad40ca134" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:20:14.653398 16145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57a744f5aaf944678a63469cbf514015" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:20:14.657577 16217 raft_consensus.cc:3060] T fffab26427e24d398e2695aad40ca134 P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:20:14.657624 16145 raft_consensus.cc:3060] T 57a744f5aaf944678a63469cbf514015 P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:20:14.652945 16151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af57996a8be84d398f0f023e8d08ef77" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:14.549145 19310 raft_consensus.cc:515] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:14.637857 18599 raft_consensus.cc:493] T 18ad7e3af5504571a69d10c00750a01c P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:20:14.660840 19080 leader_election.cc:290] T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:20:14.661831 19205 raft_consensus.cc:493] T b87147731a68400e8cce9b879daa7c5d P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:14.680308 19205 raft_consensus.cc:515] T b87147731a68400e8cce9b879daa7c5d P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:14.682781 19205 leader_election.cc:290] T b87147731a68400e8cce9b879daa7c5d P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:14.683898 16151 raft_consensus.cc:1275] T 26354361e9b044cea1c2b86c4882fe92 P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:20:14.685465 16151 raft_consensus.cc:1275] T 434bff3082d141838068f323bdb0d1ed P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 7 index: 8. (term mismatch)
I20260709 13:20:14.685787 16151 pending_rounds.cc:85] T 434bff3082d141838068f323bdb0d1ed P 647a343a542d4a139fe106c19095129b: Aborting all ops after (but not including) 7
I20260709 13:20:14.686015 16151 pending_rounds.cc:107] T 434bff3082d141838068f323bdb0d1ed P 647a343a542d4a139fe106c19095129b: Aborting uncommitted NO_OP operation due to leader change: 6.8
I20260709 13:20:14.686232 16151 raft_consensus.cc:2889] T 434bff3082d141838068f323bdb0d1ed P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:20:14.660851 16150 pending_rounds.cc:85] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b: Aborting all ops after (but not including) 6
I20260709 13:20:14.687634 16150 pending_rounds.cc:107] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b: Aborting uncommitted NO_OP operation due to leader change: 5.7
I20260709 13:20:14.687928 16150 raft_consensus.cc:2889] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:20:14.689019 16148 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40844 (request call id 2030) took 2735 ms. Trace:
I20260709 13:20:14.689422 16148 rpcz_store.cc:276] 0709 13:20:11.953537 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:11.953700 (+   163us) service_pool.cc:224] Handling call
0709 13:20:14.023140 (+2069440us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:14.029582 (+  6442us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:14.688837 (+659255us) spinlock_profiling.cc:243] Waited 2.07 s on lock 0x7b540028f350. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5e8ad 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:20:14.688964 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4755666688}
I20260709 13:20:14.654102 19178 raft_consensus.cc:493] T 1df93779209748b9860f6d893dd3de53 P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:14.691685 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0eab25ce4c64e018a041beeda2867db" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 8 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:14.691604 19178 raft_consensus.cc:515] T 1df93779209748b9860f6d893dd3de53 P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:14.692615 16225 raft_consensus.cc:2393] T c0eab25ce4c64e018a041beeda2867db P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 8: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
I20260709 13:20:14.692332 19048 consensus_queue.cc:237] T c0eab25ce4c64e018a041beeda2867db P 90f3da57796b488c9087a2f523adf045 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:14.693568 16222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d16dfad0c3b540e5b4b86492ddf87208" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:20:14.693914 19178 leader_election.cc:290] T 1df93779209748b9860f6d893dd3de53 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:14.643383 19274 raft_consensus.cc:515] T ee9ed67006144797b1de6ea8c2013a68 P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:14.696902 16222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ad5dbfa99ee4b8d9e28be92b3f36db5" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:20:14.698712 19274 leader_election.cc:290] T ee9ed67006144797b1de6ea8c2013a68 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:14.664156 16117 leader_election.cc:304] T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:20:14.700265 16117 leader_election.cc:304] T 26354361e9b044cea1c2b86c4882fe92 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860
I20260709 13:20:14.700598 19310 leader_election.cc:290] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 7 election: Requested vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:14.691483 18599 raft_consensus.cc:515] T 18ad7e3af5504571a69d10c00750a01c P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } }
I20260709 13:20:14.701519 19213 raft_consensus.cc:2804] T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:20:14.646224 19269 raft_consensus.cc:2749] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:20:14.701964 19213 raft_consensus.cc:493] T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:14.702374 19213 raft_consensus.cc:3060] T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:20:14.702050 19314 raft_consensus.cc:515] T 23f82224f4564fea93572e8206bad1fe P cb82cc7bd6db46d980c75237a1402860 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:14.702872 18599 leader_election.cc:290] T 18ad7e3af5504571a69d10c00750a01c P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:20:14.703495 18599 raft_consensus.cc:2749] T 26354361e9b044cea1c2b86c4882fe92 P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:20:14.704770 19176 consensus_queue.cc:1059] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:20:14.705260 16142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cc959fb7f704fba9009c0595467d074" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:14.705750 19176 raft_consensus.cc:3055] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:20:14.666961 16116 leader_election.cc:304] T 746aabf856ab401c9ef5e859371e7b67 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860
I20260709 13:20:14.667116 19201 raft_consensus.cc:493] T ecc1e363230043fe9fef390f0b979f20 P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:14.656816 16265 leader_election.cc:304] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:20:14.707094 19209 raft_consensus.cc:2749] T 746aabf856ab401c9ef5e859371e7b67 P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:20:14.707433 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 50721.8us
I20260709 13:20:14.707284 19201 raft_consensus.cc:515] T ecc1e363230043fe9fef390f0b979f20 P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:14.708170 16265 leader_election.cc:304] T ada5a004ba7c46819051e3ccec71ffdb P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860
I20260709 13:20:14.709538 16265 leader_election.cc:400] T 8d4ab870a05347868956aa8b841b2981 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 7 pre-election: Vote denied by peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943) with higher term. Message: Invalid argument: T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 for earlier term 7. Current term is 8.
I20260709 13:20:14.709796 16265 leader_election.cc:403] T 8d4ab870a05347868956aa8b841b2981 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 7 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:20:14.642447 19066 raft_consensus.cc:493] T 68a6838d82ef4cac83507f906139d90f P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:20:14.710413 19066 raft_consensus.cc:515] T 68a6838d82ef4cac83507f906139d90f P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:14.710718 16148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a959dbecc2648eb85d1cc550f5a3245" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:14.712524 19066 leader_election.cc:290] T 68a6838d82ef4cac83507f906139d90f P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:20:14.639415 19197 raft_consensus.cc:515] T 8cc959fb7f704fba9009c0595467d074 P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:14.714301 19197 leader_election.cc:290] T 8cc959fb7f704fba9009c0595467d074 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:14.714655 19088 raft_consensus.cc:2804] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:20:14.715106 16142 raft_consensus.cc:2410] T 8cc959fb7f704fba9009c0595467d074 P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 for term 7 because replica has last-logged OpId of term: 6 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260709 13:20:14.716454 16151 raft_consensus.cc:1240] T 2794af5b99984efebdee564035841cb0 P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 5. Current term is 6. Ops: [5.6-5.6]
I20260709 13:20:14.674966 16145 raft_consensus.cc:2468] T 57a744f5aaf944678a63469cbf514015 P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 5.
I20260709 13:20:14.663287 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "829b4ad8c5a54eababeea1181b960d4b" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:20:14.718341 16149 raft_consensus.cc:3060] T 829b4ad8c5a54eababeea1181b960d4b P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:20:14.706171 19176 raft_consensus.cc:740] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:20:14.720177 19176 consensus_queue.cc:260] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:14.720643 19289 consensus_queue.cc:1048] T 26354361e9b044cea1c2b86c4882fe92 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:20:14.721330 19176 raft_consensus.cc:3060] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:20:14.722105 19289 consensus_queue.cc:1048] T 434bff3082d141838068f323bdb0d1ed P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:20:14.715319 19269 raft_consensus.cc:2749] T ada5a004ba7c46819051e3ccec71ffdb P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:20:14.715111 19088 raft_consensus.cc:493] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:14.724138 19088 raft_consensus.cc:3060] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:20:14.726161 19213 raft_consensus.cc:515] T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:14.727092 16148 raft_consensus.cc:1275] T 8a878999c0024d2abe517484180290fc P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 6 index: 3. (index mismatch)
I20260709 13:20:14.728425 19213 leader_election.cc:290] T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 7 election: Requested vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:14.732175 19080 raft_consensus.cc:3060] T 8d4ab870a05347868956aa8b841b2981 P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:20:14.709910 19201 leader_election.cc:290] T ecc1e363230043fe9fef390f0b979f20 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:14.735483 16217 raft_consensus.cc:2468] T fffab26427e24d398e2695aad40ca134 P 1ca2725d290e47d68b1a4b609c9816ac [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 6.
I20260709 13:20:14.736414 16790 leader_election.cc:304] T 57a744f5aaf944678a63469cbf514015 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:20:14.738036 19289 raft_consensus.cc:2804] T 57a744f5aaf944678a63469cbf514015 P cb82cc7bd6db46d980c75237a1402860 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:20:14.738705 19289 raft_consensus.cc:697] T 57a744f5aaf944678a63469cbf514015 P cb82cc7bd6db46d980c75237a1402860 [term 5 LEADER]: Becoming Leader. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:20:14.739655 19289 consensus_queue.cc:237] T 57a744f5aaf944678a63469cbf514015 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:14.742487 16221 raft_consensus.cc:2468] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 5.
W20260709 13:20:14.743854 16789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 130059us
W20260709 13:20:14.744495 16789 leader_election.cc:336] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 8 election: RPC error from VoteRequest() call to peer 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389): Timed out: RequestConsensusVote RPC to 127.14.177.131:41389 timed out after 2.181s (SENT)
I20260709 13:20:14.744946 16789 leader_election.cc:304] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:20:14.745875 16116 leader_election.cc:304] T 02838c86531d4f98bf58d1c22b8baf77 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 90f3da57796b488c9087a2f523adf045
I20260709 13:20:14.746281 16149 raft_consensus.cc:2468] T 829b4ad8c5a54eababeea1181b960d4b P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 7.
I20260709 13:20:14.721261 16265 leader_election.cc:304] T 8c11074579b046d99e585c2818e18a2c P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: cb82cc7bd6db46d980c75237a1402860
I20260709 13:20:14.747687 16116 leader_election.cc:304] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860
I20260709 13:20:14.749433 16790 leader_election.cc:304] T 829b4ad8c5a54eababeea1181b960d4b P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:20:14.750044 16143 raft_consensus.cc:1240] T 02838c86531d4f98bf58d1c22b8baf77 P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 6. Current term is 8. Ops: []
I20260709 13:20:14.752126 16265 leader_election.cc:304] T c9944d5e4b8c4109af9955c4f08c9095 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: cb82cc7bd6db46d980c75237a1402860
I20260709 13:20:14.754232 16143 raft_consensus.cc:1240] T 8158bbd17b994d2eb96f560bd0f1b88d P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 6. Current term is 7. Ops: []
I20260709 13:20:14.756794 16143 raft_consensus.cc:1240] T 57a744f5aaf944678a63469cbf514015 P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 13:20:14.640151 19037 raft_consensus.cc:515] T af57996a8be84d398f0f023e8d08ef77 P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:20:14.762732 16789 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:14.765491 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68a6838d82ef4cac83507f906139d90f" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:14.769819 16143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a9de347c711477e87c582d003f36925" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 8 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:14.777837 16789 leader_election.cc:304] T 74ca8140f35c4ae29e8a664f6bdfc829 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac
I20260709 13:20:14.778856 16142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e65676b21b748728ad39668cfe722ec" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:14.785606 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f82224f4564fea93572e8206bad1fe" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:20:14.786692 16149 raft_consensus.cc:2393] T 23f82224f4564fea93572e8206bad1fe P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 5: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
I20260709 13:20:14.758599 19269 raft_consensus.cc:2804] T 8c11074579b046d99e585c2818e18a2c P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:20:14.791420 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c2095a5f4ad49408dbc7f69c606120d" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:20:14.792485 19037 leader_election.cc:290] T af57996a8be84d398f0f023e8d08ef77 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 8 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:14.788264 16221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6f07842d3aa47858df5665874c86f3a" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:14.763126 19197 raft_consensus.cc:2804] T 02838c86531d4f98bf58d1c22b8baf77 P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:20:14.793983 16221 raft_consensus.cc:2468] T b6f07842d3aa47858df5665874c86f3a P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 7.
I20260709 13:20:14.794049 19197 raft_consensus.cc:697] T 02838c86531d4f98bf58d1c22b8baf77 P 647a343a542d4a139fe106c19095129b [term 8 LEADER]: Becoming Leader. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:20:14.777274 19210 raft_consensus.cc:2804] T 829b4ad8c5a54eababeea1181b960d4b P cb82cc7bd6db46d980c75237a1402860 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:20:14.795106 19197 consensus_queue.cc:237] T 02838c86531d4f98bf58d1c22b8baf77 P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:14.795991 19210 raft_consensus.cc:697] T 829b4ad8c5a54eababeea1181b960d4b P cb82cc7bd6db46d980c75237a1402860 [term 7 LEADER]: Becoming Leader. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:20:14.783129 19213 consensus_queue.cc:1059] T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.011s
I20260709 13:20:14.796950 19210 consensus_queue.cc:237] T 829b4ad8c5a54eababeea1181b960d4b P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:14.798089 19213 raft_consensus.cc:3055] T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:20:14.798499 19213 raft_consensus.cc:740] T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:20:14.762655 19295 raft_consensus.cc:2749] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:20:14.799463 19213 consensus_queue.cc:260] T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:14.801189 19213 raft_consensus.cc:3060] T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:20:14.803776 16192 leader_election.cc:304] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:20:14.804555 19068 raft_consensus.cc:493] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:14.804862 19295 consensus_queue.cc:1048] T 2794af5b99984efebdee564035841cb0 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:20:14.805200 19068 raft_consensus.cc:515] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } }
I20260709 13:20:14.785450 19148 consensus_queue.cc:1048] T c9944d5e4b8c4109af9955c4f08c9095 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.005s
I20260709 13:20:14.788971 19306 raft_consensus.cc:2804] T 74ca8140f35c4ae29e8a664f6bdfc829 P cb82cc7bd6db46d980c75237a1402860 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:20:14.807256 19306 raft_consensus.cc:493] T 74ca8140f35c4ae29e8a664f6bdfc829 P cb82cc7bd6db46d980c75237a1402860 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:20:14.807646 19306 raft_consensus.cc:3060] T 74ca8140f35c4ae29e8a664f6bdfc829 P cb82cc7bd6db46d980c75237a1402860 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:20:14.810808 16263 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:14.805078 19291 raft_consensus.cc:2804] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:20:14.812769 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d16dfad0c3b540e5b4b86492ddf87208" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:20:14.813511 16300 raft_consensus.cc:3060] T d16dfad0c3b540e5b4b86492ddf87208 P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:20:14.812700 16824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e65676b21b748728ad39668cfe722ec" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:20:14.818434 16824 raft_consensus.cc:2368] T 9e65676b21b748728ad39668cfe722ec P cb82cc7bd6db46d980c75237a1402860 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b for earlier term 7. Current term is 8.
I20260709 13:20:14.829629 16825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee9ed67006144797b1de6ea8c2013a68" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:14.790330 19314 leader_election.cc:290] T 23f82224f4564fea93572e8206bad1fe P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 6 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:14.834695 16789 leader_election.cc:304] T d2ab0ac8347641e6955e9bce7d90c1a2 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:20:14.841205 16292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e65676b21b748728ad39668cfe722ec" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:20:14.841226 16149 raft_consensus.cc:2468] T 7c2095a5f4ad49408dbc7f69c606120d P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 6.
I20260709 13:20:14.844719 16825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8638ee363273426db1393dd81e5c7b72" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:20:14.845504 16825 raft_consensus.cc:3060] T 8638ee363273426db1393dd81e5c7b72 P cb82cc7bd6db46d980c75237a1402860 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:20:14.776772 19066 raft_consensus.cc:2804] T c9944d5e4b8c4109af9955c4f08c9095 P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:20:14.850415 16292 raft_consensus.cc:2368] T 9e65676b21b748728ad39668cfe722ec P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b for earlier term 7. Current term is 8.
I20260709 13:20:14.851547 16824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2372981006d4b218215c28964dea623" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:20:14.758864 19201 consensus_queue.cc:1048] T b6f07842d3aa47858df5665874c86f3a P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 2, Time since last communication: 0.059s
I20260709 13:20:14.856380 16816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b7d1ab519044eda2d5e04a07242f25" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:14.856704 16789 leader_election.cc:304] T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 
W20260709 13:20:14.875784 16791 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:14.865015 16824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31969392ccdb43b491911a319e8bc7e1" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:14.858979 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ad5dbfa99ee4b8d9e28be92b3f36db5" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:20:14.879264 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0eab25ce4c64e018a041beeda2867db" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 8 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:14.879870 16297 raft_consensus.cc:3060] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:20:14.860360 16821 raft_consensus.cc:1275] T 3884f864a96d428785895e5463e5728f P cb82cc7bd6db46d980c75237a1402860 [term 6 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:20:14.880913 16298 raft_consensus.cc:1275] T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:20:14.881337 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f82224f4564fea93572e8206bad1fe" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:20:14.854038 19066 raft_consensus.cc:493] T c9944d5e4b8c4109af9955c4f08c9095 P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:14.881913 16149 raft_consensus.cc:3060] T 23f82224f4564fea93572e8206bad1fe P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:20:14.884822 16816 raft_consensus.cc:2393] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 5: Already voted for candidate cb82cc7bd6db46d980c75237a1402860 in this term.
I20260709 13:20:14.882320 19066 raft_consensus.cc:3060] T c9944d5e4b8c4109af9955c4f08c9095 P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:20:14.889688 19244 leader_election.cc:290] T ecc1e363230043fe9fef390f0b979f20 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 7 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:14.894055 16823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e4bfb9090c8460eb14fa7e0d818e7f1" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:14.894938 16823 raft_consensus.cc:2468] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P cb82cc7bd6db46d980c75237a1402860 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 6.
I20260709 13:20:14.881624 16300 raft_consensus.cc:2468] T d16dfad0c3b540e5b4b86492ddf87208 P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 7.
I20260709 13:20:14.785089 19144 consensus_queue.cc:1048] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.094s
I20260709 13:20:14.879417 16824 raft_consensus.cc:2393] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 6: Already voted for candidate cb82cc7bd6db46d980c75237a1402860 in this term.
I20260709 13:20:14.905299 16115 leader_election.cc:304] T d16dfad0c3b540e5b4b86492ddf87208 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac
I20260709 13:20:14.840515 18599 raft_consensus.cc:2749] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:20:14.915318 16264 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:14.880497 19148 raft_consensus.cc:2804] T d2ab0ac8347641e6955e9bce7d90c1a2 P cb82cc7bd6db46d980c75237a1402860 [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 13:20:14.912348 16789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 55737.1us
I20260709 13:20:14.966920 16291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1df93779209748b9860f6d893dd3de53" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:14.969164 16291 raft_consensus.cc:2468] T 1df93779209748b9860f6d893dd3de53 P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 7.
I20260709 13:20:14.916649 16825 raft_consensus.cc:2468] T 8638ee363273426db1393dd81e5c7b72 P cb82cc7bd6db46d980c75237a1402860 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 7.
I20260709 13:20:14.916860 16149 raft_consensus.cc:2468] T 23f82224f4564fea93572e8206bad1fe P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 6.
I20260709 13:20:14.966627 19148 raft_consensus.cc:493] T d2ab0ac8347641e6955e9bce7d90c1a2 P cb82cc7bd6db46d980c75237a1402860 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:20:14.987534 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e9b2bf7011f4ae688a79d7bb5a447b4" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
W20260709 13:20:14.944105 16264 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:14.970260 16291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31969392ccdb43b491911a319e8bc7e1" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:20:14.988404 16291 raft_consensus.cc:3060] T 31969392ccdb43b491911a319e8bc7e1 P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Advancing to term 7
I20260709 13:20:14.988775 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18ad7e3af5504571a69d10c00750a01c" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:14.967391 16824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ab0ac8347641e6955e9bce7d90c1a2" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:14.989492 16149 raft_consensus.cc:2468] T 18ad7e3af5504571a69d10c00750a01c P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 6.
I20260709 13:20:14.990283 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87147731a68400e8cce9b879daa7c5d" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:14.966307 19209 raft_consensus.cc:2804] T d16dfad0c3b540e5b4b86492ddf87208 P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:20:14.966902 19068 leader_election.cc:290] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:20:14.973994 18944 raft_consensus.cc:2804] T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:20:14.978801 16790 leader_election.cc:304] T 23f82224f4564fea93572e8206bad1fe P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:20:14.883985 19314 consensus_queue.cc:1059] T 8c11074579b046d99e585c2818e18a2c P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: INVALID_TERM, Last received: 6.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.093s
I20260709 13:20:14.979645 16825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 7 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:20:14.982336 16192 leader_election.cc:304] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:20:14.904693 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f82224f4564fea93572e8206bad1fe" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:20:15.000064 16300 raft_consensus.cc:3060] T 23f82224f4564fea93572e8206bad1fe P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:20:15.001782 16825 raft_consensus.cc:3060] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:20:14.990854 16149 raft_consensus.cc:2468] T b87147731a68400e8cce9b879daa7c5d P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 8.
I20260709 13:20:14.968310 19295 raft_consensus.cc:2804] T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:20:14.972939 16821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af57996a8be84d398f0f023e8d08ef77" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
W20260709 13:20:14.970306 19233 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.942s	user 0.020s	sys 0.022s
I20260709 13:20:14.843016 19088 raft_consensus.cc:515] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:14.970813 19188 raft_consensus.cc:493] T ada5a004ba7c46819051e3ccec71ffdb P cb82cc7bd6db46d980c75237a1402860 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:14.850337 19291 raft_consensus.cc:697] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [term 9 LEADER]: Becoming Leader. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:20:15.003580 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31969392ccdb43b491911a319e8bc7e1" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:14.991117 19148 raft_consensus.cc:3060] T d2ab0ac8347641e6955e9bce7d90c1a2 P cb82cc7bd6db46d980c75237a1402860 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:20:15.017876 16149 raft_consensus.cc:2368] T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 for earlier term 6. Current term is 7.
I20260709 13:20:15.025942 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434bff3082d141838068f323bdb0d1ed" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 8 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:15.021317 16300 raft_consensus.cc:2468] T 23f82224f4564fea93572e8206bad1fe P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 6.
I20260709 13:20:15.023802 16291 raft_consensus.cc:2468] T 31969392ccdb43b491911a319e8bc7e1 P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 7.
I20260709 13:20:15.028152 16291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cc959fb7f704fba9009c0595467d074" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:15.029949 16291 raft_consensus.cc:2468] T 8cc959fb7f704fba9009c0595467d074 P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 7.
I20260709 13:20:15.031042 16291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18ad7e3af5504571a69d10c00750a01c" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:15.031812 16291 raft_consensus.cc:2468] T 18ad7e3af5504571a69d10c00750a01c P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 6.
I20260709 13:20:15.032819 16291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 7 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:15.034291 16291 raft_consensus.cc:2468] T ecc1e363230043fe9fef390f0b979f20 P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 6.
I20260709 13:20:15.029184 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87147731a68400e8cce9b879daa7c5d" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:15.035740 16300 raft_consensus.cc:2468] T b87147731a68400e8cce9b879daa7c5d P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 8.
I20260709 13:20:15.036059 16825 raft_consensus.cc:2468] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 7.
W20260709 13:20:15.009449 19233 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.985s	user 0.020s	sys 0.022s
I20260709 13:20:15.083737 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806d7690c2b644959152074bb2cab0a7" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:15.090440 16291 raft_consensus.cc:3055] T 26354361e9b044cea1c2b86c4882fe92 P 90f3da57796b488c9087a2f523adf045 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:20:15.094777 16291 raft_consensus.cc:740] T 26354361e9b044cea1c2b86c4882fe92 P 90f3da57796b488c9087a2f523adf045 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:20:15.058581 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c74490c40243a88e62d1760b5cd814" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:15.095815 16149 raft_consensus.cc:2468] T 08c74490c40243a88e62d1760b5cd814 P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 6.
I20260709 13:20:15.097016 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434bff3082d141838068f323bdb0d1ed" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 8 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:20:15.018419 19068 raft_consensus.cc:493] T 5a96373bb0e14dce8a266b2fc6c58507 P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:20:15.037535 16300 raft_consensus.cc:1275] T 3884f864a96d428785895e5463e5728f P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:20:15.099234 16300 raft_consensus.cc:1240] T 434bff3082d141838068f323bdb0d1ed P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 7. Current term is 8. Ops: []
I20260709 13:20:15.000962 19209 raft_consensus.cc:697] T d16dfad0c3b540e5b4b86492ddf87208 P 647a343a542d4a139fe106c19095129b [term 7 LEADER]: Becoming Leader. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:20:15.103031 16291 consensus_queue.cc:260] T 26354361e9b044cea1c2b86c4882fe92 P 90f3da57796b488c9087a2f523adf045 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:15.104871 16291 raft_consensus.cc:3060] T 26354361e9b044cea1c2b86c4882fe92 P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:20:15.008111 19295 raft_consensus.cc:493] T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:20:15.009742 19244 raft_consensus.cc:2804] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:20:15.026311 19289 raft_consensus.cc:2804] T 23f82224f4564fea93572e8206bad1fe P cb82cc7bd6db46d980c75237a1402860 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:20:15.132105 16265 leader_election.cc:400] T c0eab25ce4c64e018a041beeda2867db P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 7 pre-election: Vote denied by peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943) with higher term. Message: Invalid argument: T c0eab25ce4c64e018a041beeda2867db P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 for earlier term 7. Current term is 8.
I20260709 13:20:15.133466 19209 consensus_queue.cc:237] T d16dfad0c3b540e5b4b86492ddf87208 P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:20:15.140626 16192 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:15.079507 19251 raft_consensus.cc:493] T c4f65cdf00b748bb963ee6e81ef43b79 P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:20:15.024915 19188 raft_consensus.cc:515] T ada5a004ba7c46819051e3ccec71ffdb P cb82cc7bd6db46d980c75237a1402860 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:15.095438 19269 raft_consensus.cc:697] T 8c11074579b046d99e585c2818e18a2c P 90f3da57796b488c9087a2f523adf045 [term 7 LEADER]: Becoming Leader. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:20:15.129616 19295 raft_consensus.cc:3060] T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:20:15.167490 16291 raft_consensus.cc:1275] T 26354361e9b044cea1c2b86c4882fe92 P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:20:15.138055 19244 raft_consensus.cc:697] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [term 7 LEADER]: Becoming Leader. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:20:15.120433 19068 raft_consensus.cc:515] T 5a96373bb0e14dce8a266b2fc6c58507 P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:15.150408 19251 raft_consensus.cc:3060] T c4f65cdf00b748bb963ee6e81ef43b79 P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:20:15.185261 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 206575us
I20260709 13:20:15.204421 19066 raft_consensus.cc:515] T c9944d5e4b8c4109af9955c4f08c9095 P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:15.205587 19080 raft_consensus.cc:2749] T 8d4ab870a05347868956aa8b841b2981 P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Leader pre-election lost for term 7. Reason: Vote denied by peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943) with higher term. Message: Invalid argument: T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 for earlier term 7. Current term is 8.
I20260709 13:20:15.211328 19269 consensus_queue.cc:237] T 8c11074579b046d99e585c2818e18a2c P 90f3da57796b488c9087a2f523adf045 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:15.217803 19188 leader_election.cc:290] T ada5a004ba7c46819051e3ccec71ffdb P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
W20260709 13:20:15.219823 19348 log.cc:927] Time spent T 434bff3082d141838068f323bdb0d1ed P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 0.064s	user 0.000s	sys 0.000s
I20260709 13:20:15.223877 19088 leader_election.cc:290] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 7 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:15.224763 19244 consensus_queue.cc:237] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:15.154064 19291 consensus_queue.cc:237] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:15.231971 18944 raft_consensus.cc:493] T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Starting leader election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
W20260709 13:20:15.269106 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 162611us
I20260709 13:20:15.285251 16265 leader_election.cc:304] T a378d7f796f94615b3b347119a9880fd P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860
I20260709 13:20:15.246526 16790 leader_election.cc:304] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
W20260709 13:20:15.270618 16266 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:15.287432 16790 leader_election.cc:304] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:20:15.275149 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e65676b21b748728ad39668cfe722ec" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:20:15.280303 16300 raft_consensus.cc:1240] T c9944d5e4b8c4109af9955c4f08c9095 P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 7. Current term is 8. Ops: [7.9-7.9]
W20260709 13:20:15.265520 19339 log.cc:927] Time spent T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 0.248s	user 0.001s	sys 0.000s
I20260709 13:20:15.250350 19314 raft_consensus.cc:3055] T 8c11074579b046d99e585c2818e18a2c P cb82cc7bd6db46d980c75237a1402860 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:20:15.293560 16265 leader_election.cc:304] T fffab26427e24d398e2695aad40ca134 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:20:15.287474 16266 leader_election.cc:304] T b87147731a68400e8cce9b879daa7c5d P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:20:15.295816 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fffab26427e24d398e2695aad40ca134" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:20:15.297860 16265 leader_election.cc:304] T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:20:15.303874 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6f07842d3aa47858df5665874c86f3a" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
W20260709 13:20:15.291947 19176 consensus_meta.cc:287] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b: Time spent flushing consensus metadata: real 0.570s	user 0.010s	sys 0.000s
I20260709 13:20:15.308925 16218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e4bfb9090c8460eb14fa7e0d818e7f1" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:20:15.309520 16218 raft_consensus.cc:3060] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:20:15.296753 19314 raft_consensus.cc:740] T 8c11074579b046d99e585c2818e18a2c P cb82cc7bd6db46d980c75237a1402860 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:20:15.291421 18944 raft_consensus.cc:3060] T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:20:15.311219 16265 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:15.311496 16266 leader_election.cc:304] T d2ab0ac8347641e6955e9bce7d90c1a2 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860
I20260709 13:20:15.366343 16297 raft_consensus.cc:2468] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 7.
I20260709 13:20:15.297644 19198 raft_consensus.cc:3060] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:20:15.296489 19088 raft_consensus.cc:2749] T a378d7f796f94615b3b347119a9880fd P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:20:15.295805 19080 consensus_queue.cc:1059] T 8a878999c0024d2abe517484180290fc P 90f3da57796b488c9087a2f523adf045 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.071s
I20260709 13:20:15.308004 19344 raft_consensus.cc:2804] T fffab26427e24d398e2695aad40ca134 P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:20:15.313520 19048 raft_consensus.cc:2804] T b87147731a68400e8cce9b879daa7c5d P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:20:15.380789 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68a6838d82ef4cac83507f906139d90f" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
W20260709 13:20:15.386544 16146 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46224 (request call id 426) took 1448 ms (client timeout 1839 ms). Trace:
W20260709 13:20:15.386950 16146 rpcz_store.cc:269] 0709 13:20:13.938047 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:13.963286 (+ 25239us) service_pool.cc:224] Handling call
0709 13:20:15.386378 (+1423092us) spinlock_profiling.cc:243] Waited 193 ms on lock 0x7b54003f0f54. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:20:15.386489 (+   111us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":444554624}
I20260709 13:20:15.314507 19185 raft_consensus.cc:2749] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
W20260709 13:20:15.373255 19336 log.cc:927] Time spent T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 0.065s	user 0.000s	sys 0.001s
I20260709 13:20:15.398321 16117 leader_election.cc:400] T 9e65676b21b748728ad39668cfe722ec P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 7 election: Vote denied by peer cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923) with higher term. Message: Invalid argument: T 9e65676b21b748728ad39668cfe722ec P cb82cc7bd6db46d980c75237a1402860 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b for earlier term 7. Current term is 8.
I20260709 13:20:15.400132 16117 leader_election.cc:403] T 9e65676b21b748728ad39668cfe722ec P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 7 election: Cancelling election due to peer responding with higher term
I20260709 13:20:15.386152 19048 raft_consensus.cc:493] T b87147731a68400e8cce9b879daa7c5d P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:15.372092 19210 raft_consensus.cc:2749] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:20:15.415104 16825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 7 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:15.417384 16146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806d7690c2b644959152074bb2cab0a7" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:15.313634 19306 raft_consensus.cc:515] T 74ca8140f35c4ae29e8a664f6bdfc829 P cb82cc7bd6db46d980c75237a1402860 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:20:15.401896 19341 log.cc:927] Time spent T 57a744f5aaf944678a63469cbf514015 P cb82cc7bd6db46d980c75237a1402860: Append to log took a long time: real 0.263s	user 0.000s	sys 0.001s
I20260709 13:20:15.420609 16825 raft_consensus.cc:2393] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 7: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
I20260709 13:20:15.389752 19344 raft_consensus.cc:697] T fffab26427e24d398e2695aad40ca134 P 90f3da57796b488c9087a2f523adf045 [term 6 LEADER]: Becoming Leader. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:20:15.348174 19314 consensus_queue.cc:260] T 8c11074579b046d99e585c2818e18a2c P cb82cc7bd6db46d980c75237a1402860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:15.399025 19178 raft_consensus.cc:493] T 5a96373bb0e14dce8a266b2fc6c58507 P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:15.400501 19088 raft_consensus.cc:2804] T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 13:20:15.427984 16791 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:15.431634 16822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e65676b21b748728ad39668cfe722ec" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:20:15.437753 16822 raft_consensus.cc:2393] T 9e65676b21b748728ad39668cfe722ec P cb82cc7bd6db46d980c75237a1402860 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 8: Already voted for candidate cb82cc7bd6db46d980c75237a1402860 in this term.
I20260709 13:20:15.402976 19205 raft_consensus.cc:2749] T 9e65676b21b748728ad39668cfe722ec P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Leader election lost for term 7. Reason: Vote denied by peer cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923) with higher term. Message: Invalid argument: T 9e65676b21b748728ad39668cfe722ec P cb82cc7bd6db46d980c75237a1402860 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b for earlier term 7. Current term is 8.
I20260709 13:20:15.433121 16824 raft_consensus.cc:1240] T 57a744f5aaf944678a63469cbf514015 P cb82cc7bd6db46d980c75237a1402860 [term 5 LEADER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 4. Current term is 5. Ops: []
I20260709 13:20:15.434291 16818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31969392ccdb43b491911a319e8bc7e1" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:20:15.445781 16818 raft_consensus.cc:3060] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:20:15.402180 19113 consensus_queue.cc:1048] T 84f2da98864b4425bf7c0055951fdd4b P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.036s
I20260709 13:20:15.420048 19048 raft_consensus.cc:3060] T b87147731a68400e8cce9b879daa7c5d P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:20:15.435807 16821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434bff3082d141838068f323bdb0d1ed" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 8 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
W20260709 13:20:15.438650 16266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 127246us
I20260709 13:20:15.420987 19201 consensus_queue.cc:1048] T 3884f864a96d428785895e5463e5728f P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.019s
I20260709 13:20:15.453883 16294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a96373bb0e14dce8a266b2fc6c58507" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 6 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:20:15.452365 16822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e4bfb9090c8460eb14fa7e0d818e7f1" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:20:15.455042 16822 raft_consensus.cc:3060] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P cb82cc7bd6db46d980c75237a1402860 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:20:15.432991 16816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87147731a68400e8cce9b879daa7c5d" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:15.449196 16823 raft_consensus.cc:1240] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [term 9 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 8. Current term is 9. Ops: []
I20260709 13:20:15.454499 16294 raft_consensus.cc:3060] T 5a96373bb0e14dce8a266b2fc6c58507 P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:20:15.456458 16147 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46224 (request call id 427) took 1511 ms. Trace:
I20260709 13:20:15.438586 19088 raft_consensus.cc:493] T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:15.455389 16297 raft_consensus.cc:1240] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 6. Current term is 7. Ops: [6.4-6.4]
I20260709 13:20:15.457057 16147 rpcz_store.cc:276] 0709 13:20:13.945372 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:14.113127 (+167755us) service_pool.cc:224] Handling call
0709 13:20:14.128323 (+ 15196us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:20:14.608061 (+479738us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:14.608105 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:20:14.608577 (+   472us) log.cc:844] Serialized 26 byte log entry
0709 13:20:14.970246 (+361669us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:20:14.970290 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:14.970354 (+    64us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:20:15.456174 (+485820us) raft_consensus.cc:1685] finished
0709 13:20:15.456203 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:15.456381 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":511514,"raft.queue_time_us":488690,"raft.run_cpu_time_us":121,"raft.run_wall_time_us":10280,"thread_start_us":323322,"threads_started":1,"wal-append.queue_time_us":556454}
I20260709 13:20:15.456410 16816 raft_consensus.cc:2468] T b87147731a68400e8cce9b879daa7c5d P cb82cc7bd6db46d980c75237a1402860 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 8.
I20260709 13:20:15.419975 19289 raft_consensus.cc:697] T 23f82224f4564fea93572e8206bad1fe P cb82cc7bd6db46d980c75237a1402860 [term 6 LEADER]: Becoming Leader. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:20:15.461323 16296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0eab25ce4c64e018a041beeda2867db" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:20:15.461911 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9944d5e4b8c4109af9955c4f08c9095" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:15.455662 19068 leader_election.cc:290] T 5a96373bb0e14dce8a266b2fc6c58507 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:20:15.469357 16218 raft_consensus.cc:2468] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 7.
I20260709 13:20:15.452921 19314 raft_consensus.cc:3060] T 8c11074579b046d99e585c2818e18a2c P cb82cc7bd6db46d980c75237a1402860 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:20:15.458227 19066 leader_election.cc:290] T c9944d5e4b8c4109af9955c4f08c9095 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 8 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:15.494886 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8638ee363273426db1393dd81e5c7b72" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:20:15.495342 19066 raft_consensus.cc:2749] T d2ab0ac8347641e6955e9bce7d90c1a2 P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:20:15.495483 16297 raft_consensus.cc:3060] T 8638ee363273426db1393dd81e5c7b72 P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:20:15.476754 19088 raft_consensus.cc:3060] T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:20:15.471626 16820 raft_consensus.cc:1240] T 74ca8140f35c4ae29e8a664f6bdfc829 P cb82cc7bd6db46d980c75237a1402860 [term 8 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 7. Current term is 8. Ops: []
I20260709 13:20:15.501207 19306 leader_election.cc:290] T 74ca8140f35c4ae29e8a664f6bdfc829 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 8 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:15.501895 19344 consensus_queue.cc:237] T fffab26427e24d398e2695aad40ca134 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:15.502549 16297 raft_consensus.cc:2468] T 8638ee363273426db1393dd81e5c7b72 P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 7.
I20260709 13:20:15.503546 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4f65cdf00b748bb963ee6e81ef43b79" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:20:15.472862 16822 raft_consensus.cc:2468] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P cb82cc7bd6db46d980c75237a1402860 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 7.
I20260709 13:20:15.505409 16820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9944d5e4b8c4109af9955c4f08c9095" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:20:15.481820 16295 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48674 (request call id 2303) took 1321 ms. Trace:
I20260709 13:20:15.483067 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fffab26427e24d398e2695aad40ca134" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:15.506991 16295 rpcz_store.cc:276] 0709 13:20:14.160697 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:14.175737 (+ 15040us) service_pool.cc:224] Handling call
0709 13:20:14.176029 (+   292us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:14.515782 (+339753us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:15.481579 (+965797us) spinlock_profiling.cc:243] Waited 339 ms on lock 0x7b54002cfbd4. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:20:15.481763 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":779497856}
I20260709 13:20:15.510042 16295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a878999c0024d2abe517484180290fc" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:20:15.448256 19345 raft_consensus.cc:493] T 84f2da98864b4425bf7c0055951fdd4b P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:15.488674 16117 leader_election.cc:304] T b87147731a68400e8cce9b879daa7c5d P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:20:15.508391 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c61a39fd4f84424b129c1a6f9ba404d" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:15.511492 19345 raft_consensus.cc:515] T 84f2da98864b4425bf7c0055951fdd4b P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:15.512284 19176 raft_consensus.cc:2804] T b87147731a68400e8cce9b879daa7c5d P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:20:15.512347 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a878999c0024d2abe517484180290fc" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:15.512907 19176 raft_consensus.cc:493] T b87147731a68400e8cce9b879daa7c5d P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:15.513303 19176 raft_consensus.cc:3060] T b87147731a68400e8cce9b879daa7c5d P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:20:15.513677 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d4ab870a05347868956aa8b841b2981" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 8 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:20:15.514048 19345 leader_election.cc:290] T 84f2da98864b4425bf7c0055951fdd4b P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:20:15.515312 19201 raft_consensus.cc:493] T 8158bbd17b994d2eb96f560bd0f1b88d P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:15.515545 16295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c11074579b046d99e585c2818e18a2c" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:15.515854 19201 raft_consensus.cc:515] T 8158bbd17b994d2eb96f560bd0f1b88d P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:15.489447 19289 consensus_queue.cc:237] T 23f82224f4564fea93572e8206bad1fe P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.5, Last appended by leader: 5, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:15.491621 16294 raft_consensus.cc:2468] T 5a96373bb0e14dce8a266b2fc6c58507 P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 6.
I20260709 13:20:15.516741 16295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8158bbd17b994d2eb96f560bd0f1b88d" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:15.517611 16294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02838c86531d4f98bf58d1c22b8baf77" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:15.517771 16295 raft_consensus.cc:2410] T 8158bbd17b994d2eb96f560bd0f1b88d P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac for term 8 because replica has last-logged OpId of term: 6 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260709 13:20:15.518438 16294 raft_consensus.cc:2410] T 02838c86531d4f98bf58d1c22b8baf77 P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac for term 8 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 13:20:15.477029 16148 raft_consensus.cc:1240] T 26354361e9b044cea1c2b86c4882fe92 P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 5. Current term is 6. Ops: []
I20260709 13:20:15.521291 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 7 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:20:15.521826 16298 raft_consensus.cc:3060] T ecc1e363230043fe9fef390f0b979f20 P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:20:15.524632 16294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5ad4eb281564936a59f41e387a47156" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 8 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:15.443828 19178 raft_consensus.cc:515] T 5a96373bb0e14dce8a266b2fc6c58507 P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:15.525355 16294 raft_consensus.cc:2393] T e5ad4eb281564936a59f41e387a47156 P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 8: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
I20260709 13:20:15.525942 16266 leader_election.cc:304] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:20:15.528177 16295 raft_consensus.cc:1275] T d2ab0ac8347641e6955e9bce7d90c1a2 P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:20:15.538936 16296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8638ee363273426db1393dd81e5c7b72" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:15.539788 16296 raft_consensus.cc:2376] T 8638ee363273426db1393dd81e5c7b72 P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 7. Re-sending same reply.
I20260709 13:20:15.514956 19345 consensus_queue.cc:1059] T 57a744f5aaf944678a63469cbf514015 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.027s
I20260709 13:20:15.477909 19148 raft_consensus.cc:515] T d2ab0ac8347641e6955e9bce7d90c1a2 P cb82cc7bd6db46d980c75237a1402860 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:20:15.568861 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 281522us
W20260709 13:20:15.569347 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 264800us
I20260709 13:20:15.545511 19068 raft_consensus.cc:2804] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Leader election won for term 7
W20260709 13:20:15.570186 16264 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:15.577454 16296 raft_consensus.cc:1240] T d16dfad0c3b540e5b4b86492ddf87208 P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 6. Current term is 7. Ops: [6.4-6.4]
I20260709 13:20:15.578263 19351 consensus_queue.cc:1048] T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:20:15.579171 16790 leader_election.cc:400] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 6 election: Vote denied by peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943) with higher term. Message: Invalid argument: T ecc1e363230043fe9fef390f0b979f20 P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 for earlier term 6. Current term is 7.
I20260709 13:20:15.542088 19338 consensus_queue.cc:1059] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.033s
I20260709 13:20:15.593256 16817 raft_consensus.cc:1240] T d2ab0ac8347641e6955e9bce7d90c1a2 P cb82cc7bd6db46d980c75237a1402860 [term 8 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 7. Current term is 8. Ops: []
I20260709 13:20:15.589394 16191 leader_election.cc:304] T c0eab25ce4c64e018a041beeda2867db P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
I20260709 13:20:15.590931 16790 leader_election.cc:403] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 6 election: Cancelling election due to peer responding with higher term
I20260709 13:20:15.582543 16299 raft_consensus.cc:2468] T 8d4ab870a05347868956aa8b841b2981 P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 8.
W20260709 13:20:15.644658 16266 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:15.589547 19345 consensus_queue.cc:1243] T 57a744f5aaf944678a63469cbf514015 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Peer cb82cc7bd6db46d980c75237a1402860 log is divergent from this leader: its last log entry 5.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 13:20:15.669510 16296 raft_consensus.cc:1275] T 84f2da98864b4425bf7c0055951fdd4b P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:20:15.594592 19068 raft_consensus.cc:697] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045 [term 7 LEADER]: Becoming Leader. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
W20260709 13:20:15.693219 16192 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:20:15.664675 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 75363.5us
W20260709 13:20:15.706137 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 127086us
I20260709 13:20:15.595690 19235 consensus_queue.cc:1048] T c2f5f89186204242bf377f7c42b2e2da P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.099s
I20260709 13:20:15.719050 16296 raft_consensus.cc:1275] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
W20260709 13:20:15.720476 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 394222us
W20260709 13:20:15.721627 16190 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:15.682092 19345 raft_consensus.cc:3055] T 57a744f5aaf944678a63469cbf514015 P 90f3da57796b488c9087a2f523adf045 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:20:15.691828 19362 raft_consensus.cc:493] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:20:15.728886 16116 leader_election.cc:304] T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: cb82cc7bd6db46d980c75237a1402860
I20260709 13:20:15.699383 19314 raft_consensus.cc:2749] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860 [term 7 FOLLOWER]: Leader election lost for term 6. Reason: Vote denied by peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943) with higher term. Message: Invalid argument: T ecc1e363230043fe9fef390f0b979f20 P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 for earlier term 6. Current term is 7.
I20260709 13:20:15.695178 19338 raft_consensus.cc:3055] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:20:15.732908 19142 raft_consensus.cc:2804] T c0eab25ce4c64e018a041beeda2867db P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:20:15.725098 19345 raft_consensus.cc:740] T 57a744f5aaf944678a63469cbf514015 P 90f3da57796b488c9087a2f523adf045 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
W20260709 13:20:15.763748 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 49185.7us
I20260709 13:20:15.742978 19205 raft_consensus.cc:2804] T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:20:15.743487 19338 raft_consensus.cc:740] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:20:15.769099 16790 leader_election.cc:304] T 9e65676b21b748728ad39668cfe722ec P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860; no voters: 90f3da57796b488c9087a2f523adf045
I20260709 13:20:15.762049 18901 consensus_queue.cc:1059] T d16dfad0c3b540e5b4b86492ddf87208 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.012s
I20260709 13:20:15.714670 19080 consensus_queue.cc:1243] T 8a878999c0024d2abe517484180290fc P 90f3da57796b488c9087a2f523adf045 [LEADER]: Peer 1ca2725d290e47d68b1a4b609c9816ac log is divergent from this leader: its last log entry 4.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260709 13:20:15.785485 16143 raft_consensus.cc:2410] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 for term 8 because replica has last-logged OpId of term: 7 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 6.
I20260709 13:20:15.755095 19201 leader_election.cc:290] T 8158bbd17b994d2eb96f560bd0f1b88d P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:15.752024 19363 raft_consensus.cc:493] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:20:15.795825 16818 raft_consensus.cc:2468] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 7.
W20260709 13:20:15.775048 19354 log.cc:927] Time spent T 2794af5b99984efebdee564035841cb0 P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.250s	user 0.001s	sys 0.000s
I20260709 13:20:15.771122 19205 raft_consensus.cc:493] T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Starting leader election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:20:15.770221 19351 raft_consensus.cc:493] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:20:15.786929 19306 raft_consensus.cc:2804] T 9e65676b21b748728ad39668cfe722ec P cb82cc7bd6db46d980c75237a1402860 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:20:15.787352 19187 raft_consensus.cc:493] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:15.743716 19362 raft_consensus.cc:515] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } }
I20260709 13:20:15.801478 19344 raft_consensus.cc:493] T cdbd78acb4cf46298e0de8ba0d130e19 P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
W20260709 13:20:15.858522 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 136135us
I20260709 13:20:15.862079 16190 leader_election.cc:304] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860
W20260709 13:20:15.859759 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 159642us
I20260709 13:20:15.801040 19316 consensus_queue.cc:1059] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.043s
I20260709 13:20:15.833652 19205 raft_consensus.cc:3060] T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:20:15.815497 19080 raft_consensus.cc:493] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:20:15.831293 19359 raft_consensus.cc:493] T e4dc6b0541ca431ebc757beac00521f1 P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:20:15.855777 19148 leader_election.cc:290] T d2ab0ac8347641e6955e9bce7d90c1a2 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 8 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:15.874173 16298 raft_consensus.cc:2468] T ecc1e363230043fe9fef390f0b979f20 P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 7.
I20260709 13:20:15.842566 19365 raft_consensus.cc:493] T 8158bbd17b994d2eb96f560bd0f1b88d P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:15.838160 19178 leader_election.cc:290] T 5a96373bb0e14dce8a266b2fc6c58507 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:15.878996 16145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a96373bb0e14dce8a266b2fc6c58507" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 6 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:15.908299 16145 raft_consensus.cc:2468] T 5a96373bb0e14dce8a266b2fc6c58507 P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 6.
W20260709 13:20:15.880326 16266 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:15.870421 16265 leader_election.cc:304] T 5a959dbecc2648eb85d1cc550f5a3245 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:20:15.846796 19306 raft_consensus.cc:697] T 9e65676b21b748728ad39668cfe722ec P cb82cc7bd6db46d980c75237a1402860 [term 8 LEADER]: Becoming Leader. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:20:15.796929 19068 consensus_queue.cc:237] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:15.840127 19351 raft_consensus.cc:515] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:15.920755 16298 raft_consensus.cc:1275] T ada5a004ba7c46819051e3ccec71ffdb P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
W20260709 13:20:15.875715 19352 log.cc:927] Time spent T 2794af5b99984efebdee564035841cb0 P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 0.280s	user 0.000s	sys 0.001s
I20260709 13:20:15.881968 16117 leader_election.cc:304] T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860; no voters: 
W20260709 13:20:15.927138 16117 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 45288.3us
W20260709 13:20:15.927343 16192 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 92427.3us
I20260709 13:20:15.847203 19187 raft_consensus.cc:515] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:15.808687 19363 raft_consensus.cc:515] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:15.915926 16190 leader_election.cc:304] T 8c11074579b046d99e585c2818e18a2c P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860
I20260709 13:20:15.896332 16147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f2da98864b4425bf7c0055951fdd4b" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:15.932200 16147 raft_consensus.cc:2468] T 84f2da98864b4425bf7c0055951fdd4b P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 5.
I20260709 13:20:15.885782 19291 raft_consensus.cc:493] T 5a959dbecc2648eb85d1cc550f5a3245 P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:20:15.922971 16291 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40474 (request call id 462) took 1086 ms. Trace:
I20260709 13:20:15.894212 19357 raft_consensus.cc:493] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:15.860245 19344 raft_consensus.cc:515] T cdbd78acb4cf46298e0de8ba0d130e19 P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:15.928807 16218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9944d5e4b8c4109af9955c4f08c9095" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:20:15.956775 16218 raft_consensus.cc:3060] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:20:15.832508 19345 consensus_queue.cc:260] T 57a744f5aaf944678a63469cbf514015 P 90f3da57796b488c9087a2f523adf045 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:15.959468 16190 leader_election.cc:304] T 5a96373bb0e14dce8a266b2fc6c58507 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 647a343a542d4a139fe106c19095129b
I20260709 13:20:15.943560 16265 leader_election.cc:304] T 8cc959fb7f704fba9009c0595467d074 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:20:15.946743 16291 rpcz_store.cc:276] 0709 13:20:14.836418 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:15.178940 (+342522us) service_pool.cc:224] Handling call
0709 13:20:15.180620 (+  1680us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:20:15.180996 (+   376us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:15.181032 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:20:15.181572 (+   540us) log.cc:844] Serialized 26 byte log entry
0709 13:20:15.370136 (+188564us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:20:15.370209 (+    73us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:15.370299 (+    90us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:20:15.922706 (+552407us) raft_consensus.cc:1685] finished
0709 13:20:15.922736 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:15.922894 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":87,"raft.run_cpu_time_us":222,"raft.run_wall_time_us":79257,"thread_start_us":187042,"threads_started":1,"wal-append.queue_time_us":273701}
I20260709 13:20:15.842355 19338 consensus_queue.cc:260] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:15.986265 19338 raft_consensus.cc:3060] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:20:15.990641 19187 leader_election.cc:290] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:15.991930 19363 leader_election.cc:290] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:15.992559 19363 consensus_queue.cc:1048] T 19baa7bc367844c288bca8f81439c413 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:20:15.993911 19306 consensus_queue.cc:237] T 9e65676b21b748728ad39668cfe722ec P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:15.947029 18599 raft_consensus.cc:2804] T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:20:15.946698 19178 consensus_queue.cc:1059] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.060s
I20260709 13:20:15.957140 19291 raft_consensus.cc:515] T 5a959dbecc2648eb85d1cc550f5a3245 P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:20:16.008051 19295 consensus_meta.cc:287] T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860: Time spent flushing consensus metadata: real 0.829s	user 0.010s	sys 0.002s
I20260709 13:20:16.008630 19295 raft_consensus.cc:515] T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:16.009310 19291 leader_election.cc:290] T 5a959dbecc2648eb85d1cc550f5a3245 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:16.010092 19291 raft_consensus.cc:2749] T 8c11074579b046d99e585c2818e18a2c P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:20:16.010694 19291 raft_consensus.cc:2804] T 5a96373bb0e14dce8a266b2fc6c58507 P 1ca2725d290e47d68b1a4b609c9816ac [term 6 FOLLOWER]: Leader election won for term 6
W20260709 13:20:16.014564 19251 consensus_meta.cc:287] T c4f65cdf00b748bb963ee6e81ef43b79 P 90f3da57796b488c9087a2f523adf045: Time spent flushing consensus metadata: real 0.746s	user 0.008s	sys 0.001s
I20260709 13:20:15.914815 19365 raft_consensus.cc:515] T 8158bbd17b994d2eb96f560bd0f1b88d P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:15.907081 19080 raft_consensus.cc:515] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:15.958089 19357 raft_consensus.cc:515] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:15.926683 19375 raft_consensus.cc:2749] T 5a959dbecc2648eb85d1cc550f5a3245 P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:20:16.024281 16300 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40474 (request call id 463) took 1187 ms. Trace:
I20260709 13:20:16.031412 16218 raft_consensus.cc:2468] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 8.
I20260709 13:20:16.031695 16300 rpcz_store.cc:276] 0709 13:20:14.836641 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:15.273467 (+436826us) service_pool.cc:224] Handling call
0709 13:20:15.273840 (+   373us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:20:15.288774 (+ 14934us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:16.024031 (+735257us) spinlock_profiling.cc:243] Waited 6.07 ms on lock 0x7b540010f254. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:20:16.024221 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":13964160}
I20260709 13:20:16.033473 16300 raft_consensus.cc:1275] T 1df93779209748b9860f6d893dd3de53 P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:20:16.033586 16821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5718c623ebee4d2bb770f0f9ddb2f164" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:16.034375 16821 raft_consensus.cc:2393] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 7: Already voted for candidate cb82cc7bd6db46d980c75237a1402860 in this term.
I20260709 13:20:16.030560 16820 raft_consensus.cc:1240] T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860 [term 7 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 6. Current term is 7. Ops: []
I20260709 13:20:15.964771 16190 leader_election.cc:304] T 02838c86531d4f98bf58d1c22b8baf77 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:20:15.970264 16221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f2da98864b4425bf7c0055951fdd4b" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:15.970515 19345 raft_consensus.cc:3060] T 57a744f5aaf944678a63469cbf514015 P 90f3da57796b488c9087a2f523adf045 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:20:15.959415 19142 raft_consensus.cc:697] T c0eab25ce4c64e018a041beeda2867db P 1ca2725d290e47d68b1a4b609c9816ac [term 8 LEADER]: Becoming Leader. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:20:15.972573 19315 raft_consensus.cc:3060] T 829b4ad8c5a54eababeea1181b960d4b P 1ca2725d290e47d68b1a4b609c9816ac [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:20:16.006935 19352 log.cc:927] Time spent T 2794af5b99984efebdee564035841cb0 P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
W20260709 13:20:15.980746 19368 log.cc:927] Time spent T 02838c86531d4f98bf58d1c22b8baf77 P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
W20260709 13:20:15.981578 19369 log.cc:927] Time spent T fffab26427e24d398e2695aad40ca134 P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.095s	user 0.000s	sys 0.000s
W20260709 13:20:15.982398 19370 log.cc:927] Time spent T d16dfad0c3b540e5b4b86492ddf87208 P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 0.069s	user 0.000s	sys 0.000s
I20260709 13:20:15.896482 19359 raft_consensus.cc:515] T e4dc6b0541ca431ebc757beac00521f1 P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:20:16.043426 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 78743.8us
I20260709 13:20:15.978896 19269 raft_consensus.cc:2749] T 8cc959fb7f704fba9009c0595467d074 P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:20:16.048256 16818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a9de347c711477e87c582d003f36925" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:16.020824 19080 leader_election.cc:290] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:16.073272 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a96373bb0e14dce8a266b2fc6c58507" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 6 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:16.074708 16816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7ca61592e848ea9b0fb81a2ce33cdc" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:16.111841 16145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5d59407eb545b895d8d922e0dd69fd" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
W20260709 13:20:16.075275 19048 consensus_meta.cc:287] T b87147731a68400e8cce9b879daa7c5d P 90f3da57796b488c9087a2f523adf045: Time spent flushing consensus metadata: real 0.617s	user 0.009s	sys 0.003s
I20260709 13:20:16.118924 16221 raft_consensus.cc:1240] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 7. Current term is 8. Ops: []
I20260709 13:20:16.097075 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fffab26427e24d398e2695aad40ca134" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:15.977264 19088 raft_consensus.cc:515] T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:16.124414 16220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 7 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:16.125890 16145 raft_consensus.cc:1275] T 1df93779209748b9860f6d893dd3de53 P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 7 index: 5. (term mismatch)
I20260709 13:20:16.126374 16145 pending_rounds.cc:85] T 1df93779209748b9860f6d893dd3de53 P 647a343a542d4a139fe106c19095129b: Aborting all ops after (but not including) 4
I20260709 13:20:16.126662 16145 pending_rounds.cc:107] T 1df93779209748b9860f6d893dd3de53 P 647a343a542d4a139fe106c19095129b: Aborting uncommitted NO_OP operation due to leader change: 6.5
I20260709 13:20:16.126968 16145 raft_consensus.cc:2889] T 1df93779209748b9860f6d893dd3de53 P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:20:16.099192 16146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a878999c0024d2abe517484180290fc" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:20:16.011098 19291 raft_consensus.cc:697] T 5a96373bb0e14dce8a266b2fc6c58507 P 1ca2725d290e47d68b1a4b609c9816ac [term 6 LEADER]: Becoming Leader. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:20:16.103976 16190 leader_election.cc:304] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:20:16.108210 16818 raft_consensus.cc:2468] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 8.
I20260709 13:20:16.074236 19361 raft_consensus.cc:493] T 66ae176961b74813a58bc15c9c4c6f2a P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
W20260709 13:20:16.068782 16263 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:16.129110 16221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cc959fb7f704fba9009c0595467d074" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:16.133184 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62fe3346cd3c42c4a55e14af9073bca4" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:16.021184 19357 leader_election.cc:290] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:20:16.016813 19344 leader_election.cc:290] T cdbd78acb4cf46298e0de8ba0d130e19 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:16.018987 19365 leader_election.cc:290] T 8158bbd17b994d2eb96f560bd0f1b88d P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:20:16.092535 16823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdbd78acb4cf46298e0de8ba0d130e19" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:16.042678 19377 raft_consensus.cc:2749] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:20:16.083029 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 211258us
W20260709 13:20:16.098096 19244 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.504s	user 0.003s	sys 0.013s
I20260709 13:20:16.046797 19295 leader_election.cc:290] T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 7 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:16.044107 18599 raft_consensus.cc:697] T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b [term 7 LEADER]: Becoming Leader. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:20:16.120939 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1df93779209748b9860f6d893dd3de53" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:16.015130 19251 raft_consensus.cc:515] T c4f65cdf00b748bb963ee6e81ef43b79 P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:16.122314 16147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d4ab870a05347868956aa8b841b2981" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 8 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:20:16.069322 19364 raft_consensus.cc:2749] T 02838c86531d4f98bf58d1c22b8baf77 P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:20:16.130357 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62fe3346cd3c42c4a55e14af9073bca4" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:20:16.131228 16220 raft_consensus.cc:2393] T ecc1e363230043fe9fef390f0b979f20 P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 7: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
I20260709 13:20:16.133288 16294 raft_consensus.cc:1275] T 8cc959fb7f704fba9009c0595467d074 P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260709 13:20:16.023413 19362 leader_election.cc:290] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:20:16.092126 19197 raft_consensus.cc:493] T 8a878999c0024d2abe517484180290fc P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:16.183739 19295 consensus_queue.cc:1059] T c9944d5e4b8c4109af9955c4f08c9095 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: INVALID_TERM, Last received: 6.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.027s
I20260709 13:20:16.212558 19295 raft_consensus.cc:3055] T c9944d5e4b8c4109af9955c4f08c9095 P cb82cc7bd6db46d980c75237a1402860 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:20:16.212963 16220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74ca8140f35c4ae29e8a664f6bdfc829" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:20:16.101778 19198 raft_consensus.cc:493] T 68a6838d82ef4cac83507f906139d90f P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:20:16.187060 19251 leader_election.cc:290] T c4f65cdf00b748bb963ee6e81ef43b79 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 9 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:20:16.213802 19198 raft_consensus.cc:515] T 68a6838d82ef4cac83507f906139d90f P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:16.216217 19198 leader_election.cc:290] T 68a6838d82ef4cac83507f906139d90f P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:16.128863 19048 raft_consensus.cc:515] T b87147731a68400e8cce9b879daa7c5d P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:16.218308 19048 leader_election.cc:290] T b87147731a68400e8cce9b879daa7c5d P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 9 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:16.219112 16151 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46224 (request call id 435) took 1503 ms. Trace:
I20260709 13:20:16.219548 16151 rpcz_store.cc:276] 0709 13:20:14.715710 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:14.717342 (+  1632us) service_pool.cc:224] Handling call
0709 13:20:14.717613 (+   271us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:20:14.717953 (+   340us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:20:14.991138 (+273185us) log.cc:844] Serialized 14 byte log entry
0709 13:20:15.390502 (+399364us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:20:15.411445 (+ 20943us) log.cc:844] Serialized 52 byte log entry
0709 13:20:15.421183 (+  9738us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:20:15.421217 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:15.421282 (+    65us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:20:16.218873 (+797591us) raft_consensus.cc:1685] finished
0709 13:20:16.218897 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:16.219050 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":83697,"prepare.run_cpu_time_us":44,"prepare.run_wall_time_us":45,"raft.queue_time_us":7205,"raft.run_cpu_time_us":216,"raft.run_wall_time_us":17062,"thread_start_us":513923,"threads_started":2,"wal-append.queue_time_us":501622}
I20260709 13:20:16.067587 19148 consensus_queue.cc:1059] T 434bff3082d141838068f323bdb0d1ed P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.063s
I20260709 13:20:16.155349 19361 raft_consensus.cc:515] T 66ae176961b74813a58bc15c9c4c6f2a P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:16.224120 19351 leader_election.cc:290] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:16.224480 19361 leader_election.cc:290] T 66ae176961b74813a58bc15c9c4c6f2a P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:16.177063 16296 raft_consensus.cc:1275] T e5ad4eb281564936a59f41e387a47156 P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:20:16.185889 16297 raft_consensus.cc:2393] T c4f65cdf00b748bb963ee6e81ef43b79 P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 9: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
W20260709 13:20:16.182417 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 78540.9us
I20260709 13:20:16.226256 16190 leader_election.cc:304] T e5ad4eb281564936a59f41e387a47156 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:20:16.177102 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74ca8140f35c4ae29e8a664f6bdfc829" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:20:16.205842 16219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18ad7e3af5504571a69d10c00750a01c" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:16.210628 19197 raft_consensus.cc:515] T 8a878999c0024d2abe517484180290fc P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:16.184487 18599 consensus_queue.cc:237] T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:16.049669 19210 consensus_queue.cc:1048] T 26354361e9b044cea1c2b86c4882fe92 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.074s
I20260709 13:20:16.207829 19316 raft_consensus.cc:493] T 7c2095a5f4ad49408dbc7f69c606120d P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:20:16.216262 19246 raft_consensus.cc:3055] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:20:16.227252 16117 leader_election.cc:304] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:20:16.205971 16217 raft_consensus.cc:1240] T fffab26427e24d398e2695aad40ca134 P 1ca2725d290e47d68b1a4b609c9816ac [term 6 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 5. Current term is 6. Ops: []
I20260709 13:20:16.164078 16145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4f65cdf00b748bb963ee6e81ef43b79" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:20:16.229609 16151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2372981006d4b218215c28964dea623" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:20:16.245265 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee9ed67006144797b1de6ea8c2013a68" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
W20260709 13:20:16.298591 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 72420.7us
I20260709 13:20:16.298934 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af57996a8be84d398f0f023e8d08ef77" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:20:16.157025 19382 raft_consensus.cc:493] T 02838c86531d4f98bf58d1c22b8baf77 P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:16.310015 16149 raft_consensus.cc:1275] T 8cc959fb7f704fba9009c0595467d074 P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260709 13:20:16.213009 19295 raft_consensus.cc:740] T c9944d5e4b8c4109af9955c4f08c9095 P cb82cc7bd6db46d980c75237a1402860 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:20:16.310912 16146 raft_consensus.cc:3060] T 8a878999c0024d2abe517484180290fc P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:20:16.324898 16219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af57996a8be84d398f0f023e8d08ef77" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:16.325817 16219 raft_consensus.cc:2393] T af57996a8be84d398f0f023e8d08ef77 P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 in current term 8: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
I20260709 13:20:16.326807 16219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8158bbd17b994d2eb96f560bd0f1b88d" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:16.316461 16145 raft_consensus.cc:2468] T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 9.
I20260709 13:20:16.180052 19359 leader_election.cc:290] T e4dc6b0541ca431ebc757beac00521f1 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:20:16.247529 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66f75124215f4ca6bef831e174a792d8" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:20:16.254040 16147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c61a39fd4f84424b129c1a6f9ba404d" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:16.324107 16151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 7 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:20:16.330011 16148 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40830 (request call id 1247) took 1027 ms. Trace:
I20260709 13:20:16.245605 19210 consensus_queue.cc:1243] T 26354361e9b044cea1c2b86c4882fe92 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Peer 90f3da57796b488c9087a2f523adf045 log is divergent from this leader: its last log entry 5.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 13:20:16.331115 16149 raft_consensus.cc:1275] T 746aabf856ab401c9ef5e859371e7b67 P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:20:16.328439 16145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5718c623ebee4d2bb770f0f9ddb2f164" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:16.335148 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66ae176961b74813a58bc15c9c4c6f2a" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:16.333598 16190 leader_election.cc:304] T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:20:16.348997 16145 raft_consensus.cc:2468] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 7.
I20260709 13:20:16.351118 16145 raft_consensus.cc:1240] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 6. Current term is 7. Ops: []
I20260709 13:20:16.346381 16148 rpcz_store.cc:276] 0709 13:20:15.302385 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:15.351628 (+ 49243us) service_pool.cc:224] Handling call
0709 13:20:15.476880 (+125252us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:15.519071 (+ 42191us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:16.329809 (+810738us) spinlock_profiling.cc:243] Waited 125 ms on lock 0x7b54003d9ad0. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5e8ad 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:20:16.329952 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":287083264}
I20260709 13:20:16.238054 19306 raft_consensus.cc:493] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:16.350001 16151 raft_consensus.cc:3060] T ecc1e363230043fe9fef390f0b979f20 P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:20:16.357758 16190 leader_election.cc:304] T ecc1e363230043fe9fef390f0b979f20 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:20:16.182641 19388 raft_consensus.cc:2749] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:20:16.318296 16117 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 91143.9us
I20260709 13:20:16.305928 19316 raft_consensus.cc:515] T 7c2095a5f4ad49408dbc7f69c606120d P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:16.311676 19198 consensus_queue.cc:1059] T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.105s
I20260709 13:20:16.359714 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a959dbecc2648eb85d1cc550f5a3245" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:16.494695 16146 raft_consensus.cc:2468] T 8a878999c0024d2abe517484180290fc P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 7.
I20260709 13:20:16.496388 19113 raft_consensus.cc:3055] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 1ca2725d290e47d68b1a4b609c9816ac [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:20:16.496536 19403 raft_consensus.cc:493] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:20:16.500332 16218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a96373bb0e14dce8a266b2fc6c58507" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 7 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:16.503839 19197 leader_election.cc:290] T 8a878999c0024d2abe517484180290fc P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:16.504158 16115 leader_election.cc:304] T c0eab25ce4c64e018a041beeda2867db P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
W20260709 13:20:16.414090 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 232155us
I20260709 13:20:16.633833 16218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada5a004ba7c46819051e3ccec71ffdb" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 9 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:16.494459 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8158bbd17b994d2eb96f560bd0f1b88d" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:16.392370 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "829b4ad8c5a54eababeea1181b960d4b" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:20:16.333011 19209 raft_consensus.cc:2804] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:20:16.393327 18901 raft_consensus.cc:2804] T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:20:16.328392 19246 raft_consensus.cc:740] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:20:16.608244 16219 raft_consensus.cc:2468] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 7.
I20260709 13:20:16.332798 19393 raft_consensus.cc:2749] T e5ad4eb281564936a59f41e387a47156 P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:20:16.635512 16300 raft_consensus.cc:2468] T 8158bbd17b994d2eb96f560bd0f1b88d P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 7.
W20260709 13:20:16.636579 16115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 132530us
I20260709 13:20:16.246513 19142 consensus_queue.cc:237] T c0eab25ce4c64e018a041beeda2867db P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:16.656028 16218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a378d7f796f94615b3b347119a9880fd" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:16.329905 19244 consensus_queue.cc:1048] T d2ab0ac8347641e6955e9bce7d90c1a2 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.297s
I20260709 13:20:16.325469 19382 raft_consensus.cc:515] T 02838c86531d4f98bf58d1c22b8baf77 P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:16.605942 19113 raft_consensus.cc:740] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 1ca2725d290e47d68b1a4b609c9816ac [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:20:16.619577 19378 raft_consensus.cc:493] T 829b4ad8c5a54eababeea1181b960d4b P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:16.619256 19388 raft_consensus.cc:493] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:20:16.656028 18944 consensus_meta.cc:287] T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b: Time spent flushing consensus metadata: real 1.308s	user 0.007s	sys 0.006s
W20260709 13:20:16.676579 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 320824us
I20260709 13:20:16.393425 19364 consensus_queue.cc:1048] T 84f2da98864b4425bf7c0055951fdd4b P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.268s
I20260709 13:20:16.391969 19383 consensus_queue.cc:1059] T d16dfad0c3b540e5b4b86492ddf87208 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.320s
I20260709 13:20:16.693338 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a9de347c711477e87c582d003f36925" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:16.655735 19197 raft_consensus.cc:3055] T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:20:16.688853 16218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8638ee363273426db1393dd81e5c7b72" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:16.655121 19315 raft_consensus.cc:2804] T ecc1e363230043fe9fef390f0b979f20 P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:20:16.651440 19209 raft_consensus.cc:493] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:20:16.703539 16265 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.741s	user 0.003s	sys 0.013s
I20260709 13:20:16.694255 16300 raft_consensus.cc:2468] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 8.
I20260709 13:20:16.712188 16218 raft_consensus.cc:1240] T af57996a8be84d398f0f023e8d08ef77 P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 7. Current term is 8. Ops: []
I20260709 13:20:16.619344 19385 consensus_queue.cc:1048] T c2f5f89186204242bf377f7c42b2e2da P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.066s
I20260709 13:20:16.667371 19361 raft_consensus.cc:493] T ada5a004ba7c46819051e3ccec71ffdb P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:20:16.713106 16265 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.751s	user 0.004s	sys 0.014s
I20260709 13:20:16.607106 19306 raft_consensus.cc:515] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:16.713380 16218 raft_consensus.cc:1240] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [term 9 LEADER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 8. Current term is 9. Ops: []
I20260709 13:20:16.718396 16218 raft_consensus.cc:1240] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 7. Current term is 8. Ops: [6.8-7.9]
I20260709 13:20:16.671737 18901 raft_consensus.cc:697] T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac [term 8 LEADER]: Becoming Leader. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
W20260709 13:20:16.719859 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 794665us
I20260709 13:20:16.729032 16218 raft_consensus.cc:1240] T 66f75124215f4ca6bef831e174a792d8 P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 7. Current term is 8. Ops: [7.8-7.8]
I20260709 13:20:16.730511 16218 raft_consensus.cc:1240] T 8c11074579b046d99e585c2818e18a2c P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 6. Current term is 7. Ops: []
I20260709 13:20:16.737711 16265 leader_election.cc:304] T 23f82224f4564fea93572e8206bad1fe P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
I20260709 13:20:16.738790 16265 leader_election.cc:304] T 7c2095a5f4ad49408dbc7f69c606120d P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac
W20260709 13:20:16.714375 19176 consensus_meta.cc:287] T b87147731a68400e8cce9b879daa7c5d P 647a343a542d4a139fe106c19095129b: Time spent flushing consensus metadata: real 1.201s	user 0.012s	sys 0.001s
I20260709 13:20:16.740736 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a378d7f796f94615b3b347119a9880fd" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:20:16.604573 19291 consensus_queue.cc:237] T 5a96373bb0e14dce8a266b2fc6c58507 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.2, Last appended by leader: 2, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:16.604763 19295 consensus_queue.cc:260] T c9944d5e4b8c4109af9955c4f08c9095 P cb82cc7bd6db46d980c75237a1402860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.9, Last appended by leader: 9, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:16.703982 19197 raft_consensus.cc:740] T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:20:16.741886 16265 leader_election.cc:304] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
I20260709 13:20:16.707279 19201 raft_consensus.cc:493] T 26354361e9b044cea1c2b86c4882fe92 P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:20:16.715583 19209 raft_consensus.cc:3060] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:20:16.708240 19404 raft_consensus.cc:493] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
W20260709 13:20:16.732993 19068 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.501s	user 0.037s	sys 0.073s
I20260709 13:20:16.764076 16265 leader_election.cc:304] T 18ad7e3af5504571a69d10c00750a01c P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac
I20260709 13:20:16.715452 19088 leader_election.cc:290] T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 8 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:20:16.719799 19315 raft_consensus.cc:697] T ecc1e363230043fe9fef390f0b979f20 P 1ca2725d290e47d68b1a4b609c9816ac [term 7 LEADER]: Becoming Leader. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
W20260709 13:20:16.781162 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 423454us
I20260709 13:20:16.692310 18944 raft_consensus.cc:515] T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:16.790040 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ab0ac8347641e6955e9bce7d90c1a2" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:20:16.633406 19403 raft_consensus.cc:515] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:20:16.780274 19068 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.555s	user 0.037s	sys 0.073s
I20260709 13:20:16.753237 19359 raft_consensus.cc:2749] T 23f82224f4564fea93572e8206bad1fe P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:20:16.805485 16300 raft_consensus.cc:3060] T d2ab0ac8347641e6955e9bce7d90c1a2 P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:20:16.732184 19080 consensus_queue.cc:1048] T 8a878999c0024d2abe517484180290fc P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.060s
I20260709 13:20:16.759574 19402 raft_consensus.cc:2749] T c0eab25ce4c64e018a041beeda2867db P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:20:16.724078 19361 raft_consensus.cc:515] T ada5a004ba7c46819051e3ccec71ffdb P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:16.691359 19378 raft_consensus.cc:515] T 829b4ad8c5a54eababeea1181b960d4b P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:16.842680 16150 raft_consensus.cc:2393] T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 8: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
I20260709 13:20:16.833087 16116 leader_election.cc:304] T ecc1e363230043fe9fef390f0b979f20 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860
I20260709 13:20:16.761014 19176 raft_consensus.cc:515] T b87147731a68400e8cce9b879daa7c5d P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:16.696678 19388 raft_consensus.cc:515] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:16.820520 19188 raft_consensus.cc:493] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:20:16.780867 19201 raft_consensus.cc:515] T 26354361e9b044cea1c2b86c4882fe92 P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:16.830004 19088 raft_consensus.cc:2804] T 7c2095a5f4ad49408dbc7f69c606120d P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:20:16.796900 19198 consensus_queue.cc:1048] T b6f07842d3aa47858df5665874c86f3a P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 2, Time since last communication: 0.049s
I20260709 13:20:16.835362 19314 raft_consensus.cc:3055] T 434bff3082d141838068f323bdb0d1ed P cb82cc7bd6db46d980c75237a1402860 [term 7 LEADER]: Stepping down as leader of term 7
W20260709 13:20:16.846263 19394 log.cc:927] Time spent T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.130s	user 0.000s	sys 0.001s
I20260709 13:20:16.785548 19404 raft_consensus.cc:515] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:20:16.897027 19396 log.cc:927] Time spent T cdbd78acb4cf46298e0de8ba0d130e19 P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.235s	user 0.001s	sys 0.000s
I20260709 13:20:16.897047 16299 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48684 (request call id 2208) took 1432 ms. Trace:
I20260709 13:20:16.768576 19113 consensus_queue.cc:260] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:16.889948 19088 raft_consensus.cc:697] T 7c2095a5f4ad49408dbc7f69c606120d P 90f3da57796b488c9087a2f523adf045 [term 6 LEADER]: Becoming Leader. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:20:16.887914 19314 raft_consensus.cc:740] T 434bff3082d141838068f323bdb0d1ed P cb82cc7bd6db46d980c75237a1402860 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
W20260709 13:20:16.898808 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 65788.8us
W20260709 13:20:16.868007 16115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 201760us
I20260709 13:20:16.889124 16191 leader_election.cc:304] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
W20260709 13:20:16.850162 19345 consensus_meta.cc:287] T 57a744f5aaf944678a63469cbf514015 P 90f3da57796b488c9087a2f523adf045: Time spent flushing consensus metadata: real 0.803s	user 0.011s	sys 0.000s
I20260709 13:20:16.724563 19246 consensus_queue.cc:260] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:20:16.868661 19395 log.cc:927] Time spent T 9e65676b21b748728ad39668cfe722ec P cb82cc7bd6db46d980c75237a1402860: Append to log took a long time: real 0.403s	user 0.001s	sys 0.000s
I20260709 13:20:16.916101 16115 leader_election.cc:400] T 9e65676b21b748728ad39668cfe722ec P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 7 election: Vote denied by peer 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389) with higher term. Message: Invalid argument: T 9e65676b21b748728ad39668cfe722ec P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b for earlier term 7. Current term is 8.
I20260709 13:20:16.931792 16115 leader_election.cc:304] T 1df93779209748b9860f6d893dd3de53 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac
I20260709 13:20:16.933394 16115 leader_election.cc:304] T 8cc959fb7f704fba9009c0595467d074 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac
I20260709 13:20:16.934489 16115 leader_election.cc:304] T 18ad7e3af5504571a69d10c00750a01c P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac
I20260709 13:20:16.918536 16218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68a6838d82ef4cac83507f906139d90f" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:16.897701 16299 rpcz_store.cc:276] 0709 13:20:15.464541 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:15.599956 (+135415us) service_pool.cc:224] Handling call
0709 13:20:15.648232 (+ 48276us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:20:15.894916 (+246684us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:15.894948 (+    32us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:20:15.913195 (+ 18247us) log.cc:844] Serialized 52 byte log entry
0709 13:20:16.112146 (+198951us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:20:16.132902 (+ 20756us) log.cc:844] Serialized 14 byte log entry
0709 13:20:16.137994 (+  5092us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:16.138065 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:20:16.896805 (+758740us) raft_consensus.cc:1685] finished
0709 13:20:16.896824 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:16.896979 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4872,"prepare.queue_time_us":21359,"prepare.run_cpu_time_us":61,"prepare.run_wall_time_us":62,"raft.queue_time_us":153724,"raft.run_cpu_time_us":323,"raft.run_wall_time_us":28492,"thread_start_us":399173,"threads_started":2,"wal-append.queue_time_us":700982}
W20260709 13:20:16.897300 19401 log.cc:927] Time spent T 2794af5b99984efebdee564035841cb0 P cb82cc7bd6db46d980c75237a1402860: Append to log took a long time: real 0.185s	user 0.001s	sys 0.000s
I20260709 13:20:16.936408 16300 raft_consensus.cc:2468] T d2ab0ac8347641e6955e9bce7d90c1a2 P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 8.
I20260709 13:20:16.937245 16151 raft_consensus.cc:2468] T ecc1e363230043fe9fef390f0b979f20 P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 7.
I20260709 13:20:16.897513 19404 leader_election.cc:290] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:16.938436 16218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a878999c0024d2abe517484180290fc" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:16.939181 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a96373bb0e14dce8a266b2fc6c58507" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 7 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:16.950011 16299 raft_consensus.cc:2468] T 5a96373bb0e14dce8a266b2fc6c58507 P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 6.
I20260709 13:20:16.950686 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7ca61592e848ea9b0fb81a2ce33cdc" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:16.950989 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68a6838d82ef4cac83507f906139d90f" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:16.951366 19345 raft_consensus.cc:493] T e5ad4eb281564936a59f41e387a47156 P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:16.951982 19345 raft_consensus.cc:515] T e5ad4eb281564936a59f41e387a47156 P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:16.847291 19359 raft_consensus.cc:2804] T 18ad7e3af5504571a69d10c00750a01c P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:20:16.939409 19314 consensus_queue.cc:260] T 434bff3082d141838068f323bdb0d1ed P cb82cc7bd6db46d980c75237a1402860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:16.844069 19414 raft_consensus.cc:2749] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:20:16.929461 16217 raft_consensus.cc:2468] T 829b4ad8c5a54eababeea1181b960d4b P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 7.
W20260709 13:20:16.909773 19391 log.cc:927] Time spent T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 0.290s	user 0.000s	sys 0.001s
I20260709 13:20:16.899093 19316 leader_election.cc:290] T 7c2095a5f4ad49408dbc7f69c606120d P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:16.953416 19359 raft_consensus.cc:493] T 18ad7e3af5504571a69d10c00750a01c P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
W20260709 13:20:16.982142 16117 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:16.840292 18901 consensus_queue.cc:237] T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 7.3, Last appended by leader: 3, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:16.868685 19188 raft_consensus.cc:515] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:16.954581 16218 raft_consensus.cc:2393] T 8a878999c0024d2abe517484180290fc P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 7: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
I20260709 13:20:16.951977 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada5a004ba7c46819051e3ccec71ffdb" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 9 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:16.940424 18944 leader_election.cc:290] T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 8 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:16.907649 19358 consensus_queue.cc:1048] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.051s
W20260709 13:20:16.954551 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 65549.8us
I20260709 13:20:17.019855 16191 leader_election.cc:304] T fffab26427e24d398e2695aad40ca134 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:20:17.003712 19403 leader_election.cc:290] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:20:16.956187 19113 raft_consensus.cc:3060] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 1ca2725d290e47d68b1a4b609c9816ac [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:20:16.840077 19197 consensus_queue.cc:260] T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:16.954120 19198 raft_consensus.cc:2749] T ecc1e363230043fe9fef390f0b979f20 P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:20:17.020879 19418 raft_consensus.cc:2804] T 18ad7e3af5504571a69d10c00750a01c P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:20:17.022576 19393 raft_consensus.cc:2749] T fffab26427e24d398e2695aad40ca134 P 1ca2725d290e47d68b1a4b609c9816ac [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:20:17.012925 19402 raft_consensus.cc:2804] T 8cc959fb7f704fba9009c0595467d074 P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:20:17.024616 19407 consensus_queue.cc:1048] T 1df93779209748b9860f6d893dd3de53 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:20:16.986974 19385 raft_consensus.cc:2749] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:20:17.027535 16191 leader_election.cc:304] T ee9ed67006144797b1de6ea8c2013a68 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
I20260709 13:20:17.059487 16191 leader_election.cc:304] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
W20260709 13:20:17.012650 19401 log.cc:927] Time spent T 2794af5b99984efebdee564035841cb0 P cb82cc7bd6db46d980c75237a1402860: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
I20260709 13:20:17.028506 19388 leader_election.cc:290] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:17.029510 19361 leader_election.cc:290] T ada5a004ba7c46819051e3ccec71ffdb P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:17.031111 19404 consensus_queue.cc:1059] T af57996a8be84d398f0f023e8d08ef77 P 647a343a542d4a139fe106c19095129b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:20:16.955535 19315 consensus_queue.cc:237] T ecc1e363230043fe9fef390f0b979f20 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.4, Last appended by leader: 4, Current term: 7, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:17.032333 19306 leader_election.cc:290] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
W20260709 13:20:17.033860 19205 consensus_meta.cc:287] T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b: Time spent flushing consensus metadata: real 1.137s	user 0.011s	sys 0.000s
I20260709 13:20:17.010380 19359 raft_consensus.cc:3060] T 18ad7e3af5504571a69d10c00750a01c P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:20:17.034657 19088 consensus_queue.cc:237] T 7c2095a5f4ad49408dbc7f69c606120d P 90f3da57796b488c9087a2f523adf045 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:17.035580 16291 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48684 (request call id 2217) took 1565 ms. Trace:
I20260709 13:20:17.058384 19402 raft_consensus.cc:493] T 8cc959fb7f704fba9009c0595467d074 P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:16.980412 19380 consensus_queue.cc:1059] T fffab26427e24d398e2695aad40ca134 P 647a343a542d4a139fe106c19095129b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.062s
I20260709 13:20:17.042488 19417 raft_consensus.cc:493] T 3884f864a96d428785895e5463e5728f P cb82cc7bd6db46d980c75237a1402860 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:20:17.014029 19382 leader_election.cc:290] T 02838c86531d4f98bf58d1c22b8baf77 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:20:17.005951 19416 raft_consensus.cc:2804] T 1df93779209748b9860f6d893dd3de53 P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:20:16.985996 19246 raft_consensus.cc:3060] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:20:17.018128 16300 raft_consensus.cc:2410] T ada5a004ba7c46819051e3ccec71ffdb P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 for term 9 because replica has last-logged OpId of term: 6 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 7.
W20260709 13:20:17.052721 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 248191us
I20260709 13:20:17.044258 19201 leader_election.cc:290] T 26354361e9b044cea1c2b86c4882fe92 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:17.067728 16191 leader_election.cc:304] T af57996a8be84d398f0f023e8d08ef77 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
I20260709 13:20:17.051463 19080 raft_consensus.cc:493] T a378d7f796f94615b3b347119a9880fd P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:20:17.056641 19418 raft_consensus.cc:493] T 18ad7e3af5504571a69d10c00750a01c P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Starting leader election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
W20260709 13:20:17.139755 16190 connection.cc:256] RPC call timeout handler was delayed by -1.1779001889997289s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16190000000000002s for a response
W20260709 13:20:17.041931 19338 consensus_meta.cc:287] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac: Time spent flushing consensus metadata: real 1.055s	user 0.001s	sys 0.009s
I20260709 13:20:17.073479 19205 raft_consensus.cc:515] T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:17.066280 19406 raft_consensus.cc:2749] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:20:17.080989 16295 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48684 (request call id 2211) took 1612 ms. Trace:
I20260709 13:20:17.177474 16295 rpcz_store.cc:276] 0709 13:20:15.468369 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:15.844321 (+375952us) service_pool.cc:224] Handling call
0709 13:20:15.845010 (+   689us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:20:15.863681 (+ 18671us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:15.863726 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:20:15.889742 (+ 26016us) log.cc:844] Serialized 572 byte log entry
0709 13:20:16.083732 (+193990us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:20:16.083777 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:17.080578 (+996801us) spinlock_profiling.cc:243] Waited 18.2 ms on lock 0x7b5400399ed4. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:20:17.080629 (+    51us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:20:17.080677 (+    48us) raft_consensus.cc:1685] finished
0709 13:20:17.080692 (+    15us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:17.080924 (+   232us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":41815936,"thread_start_us":193678,"threads_started":1,"wal-append.queue_time_us":288663}
I20260709 13:20:17.081877 16820 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49906 (request call id 686) took 1559 ms. Trace:
I20260709 13:20:17.084784 19359 raft_consensus.cc:515] T 18ad7e3af5504571a69d10c00750a01c P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } }
W20260709 13:20:17.089469 19346 log.cc:927] Time spent T af57996a8be84d398f0f023e8d08ef77 P cb82cc7bd6db46d980c75237a1402860: Append to log took a long time: real 1.888s	user 0.000s	sys 0.005s
I20260709 13:20:17.091889 19295 raft_consensus.cc:3060] T c9944d5e4b8c4109af9955c4f08c9095 P cb82cc7bd6db46d980c75237a1402860 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:20:17.063931 19414 raft_consensus.cc:493] T 57a744f5aaf944678a63469cbf514015 P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:20:17.129865 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 188860us
I20260709 13:20:17.069087 19423 raft_consensus.cc:2749] T ee9ed67006144797b1de6ea8c2013a68 P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:20:17.140165 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 72548.6us
I20260709 13:20:17.072105 16291 rpcz_store.cc:276] 0709 13:20:15.469894 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:15.962513 (+492619us) service_pool.cc:224] Handling call
0709 13:20:16.072640 (+110127us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:16.135821 (+ 63181us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:16.135866 (+    45us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:20:16.135919 (+    53us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:20:16.135964 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:17.035265 (+899301us) spinlock_profiling.cc:243] Waited 62.9 ms on lock 0x7b540030bbd4. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:20:17.035325 (+    60us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:17.035501 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":144651520}
I20260709 13:20:17.149858 19176 leader_election.cc:290] T b87147731a68400e8cce9b879daa7c5d P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 9 election: Requested vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
W20260709 13:20:17.122488 19340 log.cc:927] Time spent T c0eab25ce4c64e018a041beeda2867db P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 1.826s	user 0.006s	sys 0.009s
I20260709 13:20:17.070037 19345 leader_election.cc:290] T e5ad4eb281564936a59f41e387a47156 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:20:17.132742 19402 raft_consensus.cc:3060] T 8cc959fb7f704fba9009c0595467d074 P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:20:17.154661 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 390761us
W20260709 13:20:17.155830 16115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 221392us
I20260709 13:20:17.179412 16820 rpcz_store.cc:276] 0709 13:20:15.522171 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:15.530725 (+  8554us) service_pool.cc:224] Handling call
0709 13:20:15.531074 (+   349us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:16.035093 (+504019us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:17.081636 (+1046543us) spinlock_profiling.cc:243] Waited 499 ms on lock 0x7b54006f2054. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:20:17.081821 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1148312832}
I20260709 13:20:17.123016 19351 raft_consensus.cc:493] T 8c11074579b046d99e585c2818e18a2c P cb82cc7bd6db46d980c75237a1402860 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:17.131572 19416 raft_consensus.cc:493] T 1df93779209748b9860f6d893dd3de53 P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:17.131304 19068 raft_consensus.cc:493] T d16dfad0c3b540e5b4b86492ddf87208 P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:17.146394 16300 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40474 (request call id 466) took 1004 ms. Trace:
I20260709 13:20:17.152523 19364 raft_consensus.cc:2749] T af57996a8be84d398f0f023e8d08ef77 P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:20:17.203683 16191 leader_election.cc:304] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
I20260709 13:20:17.135610 19417 raft_consensus.cc:515] T 3884f864a96d428785895e5463e5728f P cb82cc7bd6db46d980c75237a1402860 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:17.183288 19382 raft_consensus.cc:493] T 8d4ab870a05347868956aa8b841b2981 P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:17.151819 19418 raft_consensus.cc:3060] T 18ad7e3af5504571a69d10c00750a01c P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:20:17.220820 16291 raft_consensus.cc:1240] T 434bff3082d141838068f323bdb0d1ed P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 7. Current term is 8. Ops: [7.8-7.8]
I20260709 13:20:17.228821 16300 rpcz_store.cc:276] 0709 13:20:16.142115 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:16.951713 (+809598us) service_pool.cc:224] Handling call
0709 13:20:17.146322 (+194609us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:20:17.230953 16220 raft_consensus.cc:2468] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 8.
I20260709 13:20:17.176245 19080 raft_consensus.cc:515] T a378d7f796f94615b3b347119a9880fd P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:17.233278 16265 leader_election.cc:400] T 31969392ccdb43b491911a319e8bc7e1 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 6 pre-election: Vote denied by peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741) with higher term. Message: Invalid argument: T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 for earlier term 6. Current term is 7.
I20260709 13:20:17.237725 16191 leader_election.cc:304] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: cb82cc7bd6db46d980c75237a1402860
I20260709 13:20:17.151695 19037 consensus_queue.cc:1048] T 1df93779209748b9860f6d893dd3de53 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.057s
I20260709 13:20:17.180152 19361 raft_consensus.cc:493] T 26354361e9b044cea1c2b86c4882fe92 P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:17.240913 19351 raft_consensus.cc:515] T 8c11074579b046d99e585c2818e18a2c P cb82cc7bd6db46d980c75237a1402860 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:17.204514 19414 raft_consensus.cc:515] T 57a744f5aaf944678a63469cbf514015 P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:17.268978 16265 leader_election.cc:403] T 31969392ccdb43b491911a319e8bc7e1 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 6 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:20:17.270654 16293 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48684 (request call id 2195) took 1818 ms. Trace:
I20260709 13:20:17.271100 16293 rpcz_store.cc:276] 0709 13:20:15.452718 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:15.453378 (+   660us) service_pool.cc:224] Handling call
0709 13:20:15.460252 (+  6874us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:15.460501 (+   249us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:20:15.762285 (+301784us) log.cc:844] Serialized 14 byte log entry
0709 13:20:15.920887 (+158602us) log.cc:844] Serialized 14 byte log entry
0709 13:20:17.084275 (+1163388us) spinlock_profiling.cc:243] Waited 12.8 ms on lock 0x7b180043a620. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b693ade1 00007f69b69475b1 00007f69ada54e07 00007f69b68afab9 00007f69ada7169b 00007f69ada6f93f 00007f69ada6f455 00007f69b0b57060 00007f69b0b69716 00007f69b0b7a287 00007f69b0b7a208
0709 13:20:17.270142 (+185867us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:20:17.270196 (+    54us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:20:17.270246 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:17.270479 (+   233us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:17.270597 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6459,"prepare.queue_time_us":388446,"prepare.run_cpu_time_us":77,"prepare.run_wall_time_us":79,"raft.queue_time_us":121922,"raft.run_cpu_time_us":151,"raft.run_wall_time_us":151,"spinlock_wait_cycles":29497728,"thread_start_us":638667,"threads_started":3,"wal-append.queue_time_us":360375,"wal-append.run_cpu_time_us":1428,"wal-append.run_wall_time_us":1055562}
W20260709 13:20:17.285514 16115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 49958.6us
I20260709 13:20:17.223088 19210 raft_consensus.cc:493] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:17.211268 19407 consensus_queue.cc:1048] T 8cc959fb7f704fba9009c0595467d074 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.061s
W20260709 13:20:17.284617 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 56381us
I20260709 13:20:17.291348 16222 raft_consensus.cc:1240] T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac [term 8 LEADER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 7. Current term is 8. Ops: []
I20260709 13:20:17.290060 16265 leader_election.cc:304] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac
I20260709 13:20:17.293694 16218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5d59407eb545b895d8d922e0dd69fd" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:17.255592 19432 raft_consensus.cc:493] T 746aabf856ab401c9ef5e859371e7b67 P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:17.238703 19416 raft_consensus.cc:3060] T 1df93779209748b9860f6d893dd3de53 P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:20:17.286885 16115 leader_election.cc:304] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac
I20260709 13:20:17.243245 19423 raft_consensus.cc:2804] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: Leader election won for term 9
W20260709 13:20:17.255429 19337 log.cc:927] Time spent T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 2.055s	user 0.001s	sys 0.007s
I20260709 13:20:17.294409 16218 raft_consensus.cc:2468] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 8.
I20260709 13:20:17.295455 16148 raft_consensus.cc:1275] T 829b4ad8c5a54eababeea1181b960d4b P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 7 index: 9. (index mismatch)
I20260709 13:20:17.360224 16116 leader_election.cc:304] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
I20260709 13:20:17.362592 16218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b7d1ab519044eda2d5e04a07242f25" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:20:17.295941 16265 leader_election.cc:304] T 434bff3082d141838068f323bdb0d1ed P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
I20260709 13:20:17.297739 16223 raft_consensus.cc:1240] T c0eab25ce4c64e018a041beeda2867db P 1ca2725d290e47d68b1a4b609c9816ac [term 8 LEADER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 7. Current term is 8. Ops: []
I20260709 13:20:17.257443 19205 leader_election.cc:290] T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 5 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:17.257507 19188 leader_election.cc:290] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:17.273005 19088 raft_consensus.cc:2749] T 31969392ccdb43b491911a319e8bc7e1 P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Leader pre-election lost for term 6. Reason: Vote denied by peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741) with higher term. Message: Invalid argument: T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 for earlier term 6. Current term is 7.
I20260709 13:20:17.317173 19359 leader_election.cc:290] T 18ad7e3af5504571a69d10c00750a01c P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 7 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:20:17.317335 18944 raft_consensus.cc:2804] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:20:17.266955 19382 raft_consensus.cc:515] T 8d4ab870a05347868956aa8b841b2981 P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:17.392824 19382 leader_election.cc:290] T 8d4ab870a05347868956aa8b841b2981 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:20:17.288348 19364 raft_consensus.cc:2804] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:20:17.393919 19364 raft_consensus.cc:493] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac [term 6 FOLLOWER]: Starting leader election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:20:17.394371 19364 raft_consensus.cc:3060] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:20:17.325254 19431 consensus_queue.cc:1059] T fffab26427e24d398e2695aad40ca134 P 647a343a542d4a139fe106c19095129b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:20:17.298637 19201 raft_consensus.cc:493] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:20:17.400080 19201 raft_consensus.cc:515] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } }
I20260709 13:20:17.402565 19201 leader_election.cc:290] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:17.298864 19413 raft_consensus.cc:493] T 434bff3082d141838068f323bdb0d1ed P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:20:17.403298 19413 raft_consensus.cc:515] T 434bff3082d141838068f323bdb0d1ed P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:17.405784 19413 leader_election.cc:290] T 434bff3082d141838068f323bdb0d1ed P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:17.328735 16147 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40844 (request call id 2079) took 1005 ms. Trace:
I20260709 13:20:17.331027 16145 raft_consensus.cc:1275] T 57a744f5aaf944678a63469cbf514015 P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:20:17.336694 19414 leader_election.cc:290] T 57a744f5aaf944678a63469cbf514015 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:17.406900 16147 rpcz_store.cc:276] 0709 13:20:16.323073 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:16.505260 (+182187us) service_pool.cc:224] Handling call
0709 13:20:16.626039 (+120779us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:20:16.954747 (+328708us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:16.954784 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:20:16.955314 (+   530us) log.cc:844] Serialized 26 byte log entry
0709 13:20:17.190333 (+235019us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:20:17.190391 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:17.190491 (+   100us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:20:17.328498 (+138007us) raft_consensus.cc:1685] finished
0709 13:20:17.328520 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:17.328675 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":107559,"raft.run_cpu_time_us":169,"raft.run_wall_time_us":13757,"thread_start_us":469721,"threads_started":2,"wal-append.queue_time_us":338211}
I20260709 13:20:17.347646 16226 raft_consensus.cc:3055] T d16dfad0c3b540e5b4b86492ddf87208 P 1ca2725d290e47d68b1a4b609c9816ac [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:20:17.408489 16226 raft_consensus.cc:740] T d16dfad0c3b540e5b4b86492ddf87208 P 1ca2725d290e47d68b1a4b609c9816ac [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:20:17.350198 19351 leader_election.cc:290] T 8c11074579b046d99e585c2818e18a2c P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:17.409610 16226 consensus_queue.cc:260] T d16dfad0c3b540e5b4b86492ddf87208 P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:17.411252 16226 raft_consensus.cc:3060] T d16dfad0c3b540e5b4b86492ddf87208 P 1ca2725d290e47d68b1a4b609c9816ac [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:20:17.351894 19417 leader_election.cc:290] T 3884f864a96d428785895e5463e5728f P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:17.352545 16219 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57604 (request call id 2239) took 1196 ms. Trace:
I20260709 13:20:17.293306 19178 consensus_queue.cc:1048] T 3884f864a96d428785895e5463e5728f P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.062s
I20260709 13:20:17.352867 19080 leader_election.cc:290] T a378d7f796f94615b3b347119a9880fd P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
W20260709 13:20:17.354817 19209 consensus_meta.cc:287] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b: Time spent flushing consensus metadata: real 0.568s	user 0.008s	sys 0.002s
I20260709 13:20:17.243609 19275 consensus_queue.cc:1048] T 746aabf856ab401c9ef5e859371e7b67 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.051s
I20260709 13:20:17.299407 19345 raft_consensus.cc:2804] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:20:17.282625 19361 raft_consensus.cc:515] T 26354361e9b044cea1c2b86c4882fe92 P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:20:17.382347 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 86488.2us
I20260709 13:20:17.366827 19176 raft_consensus.cc:2804] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 13:20:17.387920 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 89246.7us
I20260709 13:20:17.240013 19068 raft_consensus.cc:515] T d16dfad0c3b540e5b4b86492ddf87208 P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:17.393371 16295 raft_consensus.cc:1240] T 8d4ab870a05347868956aa8b841b2981 P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 7. Current term is 8. Ops: []
I20260709 13:20:17.360620 19423 raft_consensus.cc:697] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac [term 9 LEADER]: Becoming Leader. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:20:17.294034 19210 raft_consensus.cc:515] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:17.415446 16219 rpcz_store.cc:276] 0709 13:20:16.156596 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:16.326578 (+169982us) service_pool.cc:224] Handling call
0709 13:20:17.352369 (+1025791us) spinlock_profiling.cc:243] Waited 280 ms on lock 0x7b54002c9554. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:20:17.352482 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":644381952}
I20260709 13:20:17.354410 18599 raft_consensus.cc:3055] T af57996a8be84d398f0f023e8d08ef77 P 647a343a542d4a139fe106c19095129b [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:20:17.423951 16218 raft_consensus.cc:2468] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 5.
I20260709 13:20:17.462309 16116 leader_election.cc:304] T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
I20260709 13:20:17.428057 16226 raft_consensus.cc:1275] T d16dfad0c3b540e5b4b86492ddf87208 P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:20:17.463593 16218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66f75124215f4ca6bef831e174a792d8" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:20:17.464354 16218 raft_consensus.cc:2393] T 66f75124215f4ca6bef831e174a792d8 P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 8: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
W20260709 13:20:17.430408 16115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 125953us
I20260709 13:20:17.430734 16265 leader_election.cc:304] T 9e65676b21b748728ad39668cfe722ec P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
I20260709 13:20:17.456785 16142 raft_consensus.cc:1240] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 13:20:17.415588 18944 raft_consensus.cc:697] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b [term 7 LEADER]: Becoming Leader. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:20:17.383823 19439 raft_consensus.cc:2749] T 434bff3082d141838068f323bdb0d1ed P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:20:17.469084 16115 leader_election.cc:304] T 8158bbd17b994d2eb96f560bd0f1b88d P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:20:17.467283 16144 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46224 (request call id 432) took 2811 ms. Trace:
I20260709 13:20:17.469481 16265 leader_election.cc:304] T 68a6838d82ef4cac83507f906139d90f P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:20:17.471951 16265 leader_election.cc:304] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:20:17.474716 16115 leader_election.cc:304] T 5a96373bb0e14dce8a266b2fc6c58507 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac
I20260709 13:20:17.477695 16218 raft_consensus.cc:1275] T 02838c86531d4f98bf58d1c22b8baf77 P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:20:17.471266 16144 rpcz_store.cc:276] 0709 13:20:14.655396 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:14.655523 (+   127us) service_pool.cc:224] Handling call
0709 13:20:14.661517 (+  5994us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:20:14.676043 (+ 14526us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:14.676073 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:20:14.676530 (+   457us) log.cc:844] Serialized 26 byte log entry
0709 13:20:14.973233 (+296703us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:20:14.973306 (+    73us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:14.973391 (+    85us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:20:17.093485 (+2120094us) spinlock_profiling.cc:243] Waited 62.3 ms on lock 0x7b4400022b4c. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69ad8fadb1 00007f69ad998c85 00007f69ad9996a2 00007f69b070b1db 00007f69b0706aaf 00007f69b06fa86e 00007f69b06f9bb6 00007f69b06f97d8 00007f69b0703221 00007f69b07031d9 00007f69b0703169
0709 13:20:17.456112 (+362627us) raft_consensus.cc:1685] finished
0709 13:20:17.456143 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:17.467214 (+ 11071us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7387,"raft.queue_time_us":15505,"raft.run_cpu_time_us":89,"raft.run_wall_time_us":88,"spinlock_wait_cycles":143338240,"thread_start_us":296262,"threads_started":1,"wal-append.queue_time_us":459385}
I20260709 13:20:17.478420 16300 pending_rounds.cc:85] T 26354361e9b044cea1c2b86c4882fe92 P 90f3da57796b488c9087a2f523adf045: Aborting all ops after (but not including) 5
I20260709 13:20:17.478744 16300 pending_rounds.cc:107] T 26354361e9b044cea1c2b86c4882fe92 P 90f3da57796b488c9087a2f523adf045: Aborting uncommitted NO_OP operation due to leader change: 5.6
I20260709 13:20:17.479120 16300 raft_consensus.cc:2889] T 26354361e9b044cea1c2b86c4882fe92 P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:20:17.479667 16265 leader_election.cc:304] T 5a96373bb0e14dce8a266b2fc6c58507 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:20:17.481123 16817 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49906 (request call id 677) took 2653 ms. Trace:
I20260709 13:20:17.425920 19246 raft_consensus.cc:493] T 2794af5b99984efebdee564035841cb0 P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:20:17.481809 16265 leader_election.cc:304] T 84f2da98864b4425bf7c0055951fdd4b P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:20:17.430084 19345 raft_consensus.cc:493] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:20:17.481560 16817 rpcz_store.cc:276] 0709 13:20:14.827502 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:14.831923 (+  4421us) service_pool.cc:224] Handling call
0709 13:20:14.903960 (+ 72037us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:15.596549 (+692589us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:17.480634 (+1884085us) spinlock_profiling.cc:243] Waited 689 ms on lock 0x7b54003df254. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:20:17.481008 (+   374us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1584626560}
I20260709 13:20:17.470919 19413 raft_consensus.cc:2804] T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:20:17.302320 19432 raft_consensus.cc:515] T 746aabf856ab401c9ef5e859371e7b67 P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:17.466771 18599 raft_consensus.cc:740] T af57996a8be84d398f0f023e8d08ef77 P 647a343a542d4a139fe106c19095129b [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:20:17.479321 19080 raft_consensus.cc:2749] T 9e65676b21b748728ad39668cfe722ec P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:20:17.420293 19209 raft_consensus.cc:515] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:17.511929 16149 raft_consensus.cc:1240] T c0eab25ce4c64e018a041beeda2867db P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 7. Current term is 8. Ops: []
I20260709 13:20:17.483649 16115 leader_election.cc:304] T 68a6838d82ef4cac83507f906139d90f P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:20:17.500612 16148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4dc6b0541ca431ebc757beac00521f1" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:17.501629 16145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:17.514665 16145 raft_consensus.cc:2468] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 7.
W20260709 13:20:17.502713 19435 log.cc:927] Time spent T 3884f864a96d428785895e5463e5728f P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 0.114s	user 0.001s	sys 0.000s
I20260709 13:20:17.491791 19382 raft_consensus.cc:2764] T 5a96373bb0e14dce8a266b2fc6c58507 P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Leader pre-election decision vote started in defunct term 5: won
I20260709 13:20:17.492599 19430 raft_consensus.cc:2804] T 5a96373bb0e14dce8a266b2fc6c58507 P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:20:17.504185 19423 consensus_queue.cc:237] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:17.520723 19430 raft_consensus.cc:493] T 5a96373bb0e14dce8a266b2fc6c58507 P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:17.521263 19430 raft_consensus.cc:3060] T 5a96373bb0e14dce8a266b2fc6c58507 P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:20:17.521847 19432 leader_election.cc:290] T 746aabf856ab401c9ef5e859371e7b67 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:17.522396 19418 raft_consensus.cc:515] T 18ad7e3af5504571a69d10c00750a01c P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } }
I20260709 13:20:17.506662 19068 leader_election.cc:290] T d16dfad0c3b540e5b4b86492ddf87208 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:20:17.461184 19176 raft_consensus.cc:493] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Starting leader election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:20:17.493981 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:17.482947 19213 raft_consensus.cc:2804] T 8158bbd17b994d2eb96f560bd0f1b88d P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:20:17.510066 16151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87147731a68400e8cce9b879daa7c5d" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:20:17.535018 16151 raft_consensus.cc:2393] T b87147731a68400e8cce9b879daa7c5d P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 9: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
I20260709 13:20:17.482798 19088 raft_consensus.cc:2749] T 68a6838d82ef4cac83507f906139d90f P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:20:17.502475 16147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8158bbd17b994d2eb96f560bd0f1b88d" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:17.500638 19345 raft_consensus.cc:3060] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:20:17.517443 16822 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49906 (request call id 685) took 1997 ms. Trace:
I20260709 13:20:17.517999 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6f07842d3aa47858df5665874c86f3a" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:20:17.518220 16144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4f65cdf00b748bb963ee6e81ef43b79" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:20:17.500293 19246 raft_consensus.cc:515] T 2794af5b99984efebdee564035841cb0 P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:17.500157 19439 raft_consensus.cc:2804] T 84f2da98864b4425bf7c0055951fdd4b P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 13:20:17.523612 19434 log.cc:927] Time spent T c0eab25ce4c64e018a041beeda2867db P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 0.162s	user 0.001s	sys 0.000s
I20260709 13:20:17.481134 19414 raft_consensus.cc:2749] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:20:17.470342 19205 consensus_queue.cc:1048] T d16dfad0c3b540e5b4b86492ddf87208 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.040s
W20260709 13:20:17.506574 19427 log.cc:927] Time spent T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 0.262s	user 0.000s	sys 0.000s
I20260709 13:20:17.529208 16148 raft_consensus.cc:3055] T fffab26427e24d398e2695aad40ca134 P 647a343a542d4a139fe106c19095129b [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:20:17.530000 16145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02838c86531d4f98bf58d1c22b8baf77" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
W20260709 13:20:17.535508 16189 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:17.477985 19419 consensus_queue.cc:1048] T e5ad4eb281564936a59f41e387a47156 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.041s
I20260709 13:20:17.559520 16816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b7d1ab519044eda2d5e04a07242f25" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:17.610592 16144 raft_consensus.cc:2393] T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 9: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
I20260709 13:20:17.546029 16291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a878999c0024d2abe517484180290fc" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:17.610951 16816 raft_consensus.cc:2468] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 5.
I20260709 13:20:17.611693 16145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18ad7e3af5504571a69d10c00750a01c" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:20:17.560217 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ab0ac8347641e6955e9bce7d90c1a2" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:17.613180 16145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5ad4eb281564936a59f41e387a47156" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:17.611560 16822 rpcz_store.cc:276] 0709 13:20:15.519621 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:15.519952 (+   331us) service_pool.cc:224] Handling call
0709 13:20:15.543020 (+ 23068us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:20:15.543486 (+   466us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:15.543527 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:20:15.560116 (+ 16589us) log.cc:844] Serialized 26 byte log entry
0709 13:20:17.086273 (+1526157us) spinlock_profiling.cc:243] Waited 4.79 ms on lock 0x7b1800350080. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b693ade1 00007f69b69475b1 00007f69ada54e07 00007f69b68afab9 00007f69ada7169b 00007f69ada6f93f 00007f69ada71c02 00007f69b06f920b 00007f69b06fed23 00007f69b06ff0d1 00007f69b0b1aa0f
0709 13:20:17.297415 (+211142us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:20:17.297472 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:17.297559 (+    87us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:20:17.511222 (+213663us) raft_consensus.cc:1685] finished
0709 13:20:17.511240 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:17.517375 (+  6135us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3029,"raft.queue_time_us":157,"raft.run_cpu_time_us":122,"raft.run_wall_time_us":13974,"spinlock_wait_cycles":11027840,"thread_start_us":210488,"threads_started":1,"wal-append.queue_time_us":1823220}
I20260709 13:20:17.548099 16292 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48684 (request call id 2213) took 2079 ms. Trace:
W20260709 13:20:17.557507 16192 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:17.623720 19412 raft_consensus.cc:493] T b87147731a68400e8cce9b879daa7c5d P cb82cc7bd6db46d980c75237a1402860 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:17.624408 19412 raft_consensus.cc:515] T b87147731a68400e8cce9b879daa7c5d P cb82cc7bd6db46d980c75237a1402860 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:17.627110 19412 leader_election.cc:290] T b87147731a68400e8cce9b879daa7c5d P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:17.627275 19439 raft_consensus.cc:493] T 84f2da98864b4425bf7c0055951fdd4b P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:17.627833 19209 leader_election.cc:290] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 9 election: Requested vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:17.562516 16820 raft_consensus.cc:1275] T ada5a004ba7c46819051e3ccec71ffdb P cb82cc7bd6db46d980c75237a1402860 [term 8 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:20:17.492832 19364 raft_consensus.cc:515] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:17.630303 19364 leader_election.cc:290] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 7 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:17.630571 19246 leader_election.cc:290] T 2794af5b99984efebdee564035841cb0 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
W20260709 13:20:17.627853 16790 connection.cc:256] RPC call timeout handler was delayed by -1.6871951919999901s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17420000000000002s for a response
I20260709 13:20:17.552940 16221 raft_consensus.cc:1275] T e5ad4eb281564936a59f41e387a47156 P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:20:17.632197 19210 leader_election.cc:290] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:20:17.632555 16221 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57604 (request call id 2237) took 1494 ms. Trace:
I20260709 13:20:17.633004 16221 rpcz_store.cc:276] 0709 13:20:16.138417 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:16.193550 (+ 55133us) service_pool.cc:224] Handling call
0709 13:20:17.552377 (+1358827us) spinlock_profiling.cc:243] Waited 11.7 ms on lock 0x7b5800010b98. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69ad8fadb1 00007f69ada3ab71 00007f69ada3aa22 00007f69b6953c09 00007f69b5aadee0 00007f69b5b770ab 00007f69b5b77131 00007f69b5b38fc0 00007f69b5b23929 00007f69b5b233d2 00007f69af09abd4
0709 13:20:17.552586 (+   209us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:17.632264 (+ 79678us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:17.632491 (+   227us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":26828544}
I20260709 13:20:17.635903 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d4ab870a05347868956aa8b841b2981" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 9 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:17.636865 19418 leader_election.cc:290] T 18ad7e3af5504571a69d10c00750a01c P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 7 election: Requested vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:20:17.636029 19378 leader_election.cc:290] T 829b4ad8c5a54eababeea1181b960d4b P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
W20260709 13:20:17.637071 16143 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40830 (request call id 1233) took 2867 ms (client timeout 3114 ms). Trace:
I20260709 13:20:17.561497 16817 raft_consensus.cc:1275] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860 [term 7 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 7 index: 8. (term mismatch)
I20260709 13:20:17.637679 16817 pending_rounds.cc:85] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860: Aborting all ops after (but not including) 7
I20260709 13:20:17.638049 16817 pending_rounds.cc:107] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860: Aborting uncommitted NO_OP operation due to leader change: 6.8
W20260709 13:20:17.637529 16143 rpcz_store.cc:269] 0709 13:20:14.769254 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:14.769496 (+   242us) service_pool.cc:224] Handling call
0709 13:20:17.636879 (+2867383us) spinlock_profiling.cc:243] Waited 1.01 s on lock 0x7b5400073ad4. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:20:17.637018 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2316337024}
I20260709 13:20:17.638357 16817 raft_consensus.cc:2889] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:20:17.627857 19439 raft_consensus.cc:3060] T 84f2da98864b4425bf7c0055951fdd4b P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:20:17.642819 19066 raft_consensus.cc:3055] T 8a878999c0024d2abe517484180290fc P 90f3da57796b488c9087a2f523adf045 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:20:17.643347 19066 raft_consensus.cc:740] T 8a878999c0024d2abe517484180290fc P 90f3da57796b488c9087a2f523adf045 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:20:17.644300 19066 consensus_queue.cc:260] T 8a878999c0024d2abe517484180290fc P 90f3da57796b488c9087a2f523adf045 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 6.3, Last appended by leader: 3, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:17.645433 19066 raft_consensus.cc:3060] T 8a878999c0024d2abe517484180290fc P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:20:17.564971 16819 raft_consensus.cc:1275] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [term 9 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:20:17.649695 16222 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57604 (request call id 2230) took 1530 ms. Trace:
I20260709 13:20:17.650151 16222 rpcz_store.cc:276] 0709 13:20:16.119249 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:16.119461 (+   212us) service_pool.cc:224] Handling call
0709 13:20:17.083078 (+963617us) spinlock_profiling.cc:243] Waited 23.1 ms on lock 0x7b5800010b98. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69ad8fadb1 00007f69ada3ab71 00007f69ada3aa22 00007f69b6953c09 00007f69b5aadee0 00007f69b5b770ab 00007f69b5b77131 00007f69b5b38fc0 00007f69b5b23929 00007f69b5b233d2 00007f69af09abd4
0709 13:20:17.083302 (+   224us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:17.296634 (+213332us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:17.649476 (+352842us) spinlock_profiling.cc:243] Waited 208 ms on lock 0x7b5400396054. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:20:17.649637 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":531169280}
I20260709 13:20:17.651281 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdbd78acb4cf46298e0de8ba0d130e19" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:17.610271 16148 raft_consensus.cc:740] T fffab26427e24d398e2695aad40ca134 P 647a343a542d4a139fe106c19095129b [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:20:17.656430 16298 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48684 (request call id 2216) took 2186 ms. Trace:
I20260709 13:20:17.517627 19359 raft_consensus.cc:493] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:20:17.663080 16298 rpcz_store.cc:276] 0709 13:20:15.469650 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:15.956783 (+487133us) service_pool.cc:224] Handling call
0709 13:20:16.006645 (+ 49862us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:16.035610 (+ 28965us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:16.035640 (+    30us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:20:16.035690 (+    50us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:20:16.035723 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:17.656187 (+1620464us) spinlock_profiling.cc:243] Waited 28.8 ms on lock 0x7b5400163354. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:20:17.656240 (+    53us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:17.656368 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":66136448}
I20260709 13:20:17.653558 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c11074579b046d99e585c2818e18a2c" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:17.540532 16151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57a744f5aaf944678a63469cbf514015" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:17.666112 16226 raft_consensus.cc:2468] T 8c11074579b046d99e585c2818e18a2c P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 7.
I20260709 13:20:17.666152 16151 raft_consensus.cc:2468] T 57a744f5aaf944678a63469cbf514015 P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 5.
I20260709 13:20:17.614581 16292 rpcz_store.cc:276] 0709 13:20:15.468888 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:15.847326 (+378438us) service_pool.cc:224] Handling call
0709 13:20:15.847672 (+   346us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:16.028939 (+181267us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:20:16.029174 (+   235us) log.cc:844] Serialized 14 byte log entry
0709 13:20:16.302810 (+273636us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:20:16.302859 (+    49us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:20:16.302905 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:17.547794 (+1244889us) spinlock_profiling.cc:243] Waited 181 ms on lock 0x7b54002e1b54. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:20:17.547855 (+    61us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:17.548025 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":13899,"prepare.run_cpu_time_us":32,"prepare.run_wall_time_us":34,"raft.queue_time_us":89,"raft.run_cpu_time_us":135,"raft.run_wall_time_us":5857,"spinlock_wait_cycles":416379136,"thread_start_us":271977,"threads_started":1,"wal-append.queue_time_us":436171}
W20260709 13:20:17.671625 19113 consensus_meta.cc:287] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 1ca2725d290e47d68b1a4b609c9816ac: Time spent flushing consensus metadata: real 0.614s	user 0.007s	sys 0.003s
I20260709 13:20:17.533612 19178 raft_consensus.cc:2749] T 68a6838d82ef4cac83507f906139d90f P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:20:17.656087 16222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66ae176961b74813a58bc15c9c4c6f2a" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
W20260709 13:20:17.687341 16225 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33050 (request call id 550) took 1488 ms (client timeout 1507 ms). Trace:
I20260709 13:20:17.559305 18944 consensus_queue.cc:237] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:17.534576 19213 raft_consensus.cc:493] T 8158bbd17b994d2eb96f560bd0f1b88d P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:17.682740 19414 raft_consensus.cc:493] T 1df93779209748b9860f6d893dd3de53 P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:17.707831 19414 raft_consensus.cc:515] T 1df93779209748b9860f6d893dd3de53 P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:17.710493 19414 leader_election.cc:290] T 1df93779209748b9860f6d893dd3de53 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:20:17.696456 19113 raft_consensus.cc:493] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:17.714079 19113 raft_consensus.cc:515] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:17.715675 19205 raft_consensus.cc:493] T 829b4ad8c5a54eababeea1181b960d4b P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:17.716270 19205 raft_consensus.cc:515] T 829b4ad8c5a54eababeea1181b960d4b P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:17.716780 19113 leader_election.cc:290] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:17.716423 19439 raft_consensus.cc:515] T 84f2da98864b4425bf7c0055951fdd4b P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:17.717957 16293 raft_consensus.cc:1275] T 31969392ccdb43b491911a319e8bc7e1 P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 13:20:17.718626 19205 leader_election.cc:290] T 829b4ad8c5a54eababeea1181b960d4b P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:17.719280 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434bff3082d141838068f323bdb0d1ed" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:17.719837 19205 raft_consensus.cc:493] T 7c2095a5f4ad49408dbc7f69c606120d P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:17.720167 16293 raft_consensus.cc:2393] T 434bff3082d141838068f323bdb0d1ed P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 8: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
I20260709 13:20:17.720453 19205 raft_consensus.cc:515] T 7c2095a5f4ad49408dbc7f69c606120d P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:17.721040 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:17.721912 16293 raft_consensus.cc:2410] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b for term 8 because replica has last-logged OpId of term: 7 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 6.
I20260709 13:20:17.723227 19205 leader_election.cc:290] T 7c2095a5f4ad49408dbc7f69c606120d P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:17.724211 19439 leader_election.cc:290] T 84f2da98864b4425bf7c0055951fdd4b P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 6 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:20:17.724745 19205 raft_consensus.cc:493] T ee9ed67006144797b1de6ea8c2013a68 P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:20:17.725366 19205 raft_consensus.cc:515] T ee9ed67006144797b1de6ea8c2013a68 P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:20:17.726704 19376 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 7 current term: 7 is higher.
I20260709 13:20:17.727762 19213 raft_consensus.cc:3060] T 8158bbd17b994d2eb96f560bd0f1b88d P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:20:17.729588 19209 raft_consensus.cc:493] T 23f82224f4564fea93572e8206bad1fe P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:17.730319 19209 raft_consensus.cc:515] T 23f82224f4564fea93572e8206bad1fe P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:17.731122 19246 raft_consensus.cc:493] T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:20:17.731527 16296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87147731a68400e8cce9b879daa7c5d" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:20:17.731779 19246 raft_consensus.cc:515] T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:17.732366 16296 raft_consensus.cc:2393] T b87147731a68400e8cce9b879daa7c5d P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 9: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
I20260709 13:20:17.682214 19382 raft_consensus.cc:493] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:20:17.688083 16225 rpcz_store.cc:269] 0709 13:20:16.184844 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:16.185076 (+   232us) service_pool.cc:224] Handling call
0709 13:20:17.560058 (+1374982us) spinlock_profiling.cc:243] Waited 20.3 ms on lock 0x7b5800010b98. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69ad8fadb1 00007f69ada3ab71 00007f69ada3aa22 00007f69b6953c09 00007f69b5aadee0 00007f69b5b770ab 00007f69b5b77131 00007f69b5b250a0 00007f69b5b24984 00007f69af09f6d4 00007f69af09f662
0709 13:20:17.673609 (+113551us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":46576384}
I20260709 13:20:17.553642 19361 leader_election.cc:290] T 26354361e9b044cea1c2b86c4882fe92 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:17.695621 19068 raft_consensus.cc:493] T 8638ee363273426db1393dd81e5c7b72 P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:17.535974 19176 raft_consensus.cc:3060] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:20:17.696892 16222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66ae176961b74813a58bc15c9c4c6f2a" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:17.694546 16265 leader_election.cc:304] T c9944d5e4b8c4109af9955c4f08c9095 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: cb82cc7bd6db46d980c75237a1402860
I20260709 13:20:17.740918 19068 raft_consensus.cc:515] T 8638ee363273426db1393dd81e5c7b72 P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:17.703514 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4f65cdf00b748bb963ee6e81ef43b79" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:20:17.741472 16144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746aabf856ab401c9ef5e859371e7b67" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:17.742198 16144 raft_consensus.cc:2468] T 746aabf856ab401c9ef5e859371e7b67 P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 7.
I20260709 13:20:17.742653 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2372981006d4b218215c28964dea623" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:17.743451 16226 raft_consensus.cc:2393] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 in current term 8: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
I20260709 13:20:17.705364 19430 raft_consensus.cc:515] T 5a96373bb0e14dce8a266b2fc6c58507 P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:17.706064 19363 consensus_queue.cc:1059] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:20:17.706766 16291 raft_consensus.cc:1240] T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 7. Current term is 8. Ops: [7.5-7.5]
I20260709 13:20:17.706907 19233 consensus_queue.cc:1059] T d2ab0ac8347641e6955e9bce7d90c1a2 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:20:17.714447 19198 raft_consensus.cc:493] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:20:17.658249 16148 consensus_queue.cc:260] T fffab26427e24d398e2695aad40ca134 P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:17.755023 16148 raft_consensus.cc:3060] T fffab26427e24d398e2695aad40ca134 P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:20:17.684209 16220 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33050 (request call id 555) took 1498 ms (client timeout 1537 ms). Trace:
I20260709 13:20:17.706985 16192 leader_election.cc:304] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:20:17.739590 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b7d1ab519044eda2d5e04a07242f25" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
W20260709 13:20:17.756147 16220 rpcz_store.cc:269] 0709 13:20:16.186065 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:16.212712 (+ 26647us) service_pool.cc:224] Handling call
0709 13:20:17.683984 (+1471272us) spinlock_profiling.cc:243] Waited 967 ms on lock 0x7b5400245c54. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:20:17.684110 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2224399744}
W20260709 13:20:17.741181 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 46768.9us
I20260709 13:20:17.701252 16292 raft_consensus.cc:1275] T d16dfad0c3b540e5b4b86492ddf87208 P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:20:17.745213 16144 raft_consensus.cc:1275] T 9e65676b21b748728ad39668cfe722ec P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:20:17.745550 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26354361e9b044cea1c2b86c4882fe92" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:17.748894 19430 leader_election.cc:290] T 5a96373bb0e14dce8a266b2fc6c58507 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 7 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:17.749799 16143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c2095a5f4ad49408dbc7f69c606120d" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:17.766662 16143 raft_consensus.cc:2393] T 7c2095a5f4ad49408dbc7f69c606120d P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 6: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
I20260709 13:20:17.733278 19209 leader_election.cc:290] T 23f82224f4564fea93572e8206bad1fe P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:17.750855 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a378d7f796f94615b3b347119a9880fd" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:20:17.756541 16225 raft_consensus.cc:2468] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 5.
I20260709 13:20:17.770375 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746aabf856ab401c9ef5e859371e7b67" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:17.771585 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a378d7f796f94615b3b347119a9880fd" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:17.753490 19361 raft_consensus.cc:2804] T c9944d5e4b8c4109af9955c4f08c9095 P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:20:17.696918 19080 raft_consensus.cc:493] T 3884f864a96d428785895e5463e5728f P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:17.762115 16222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ab0ac8347641e6955e9bce7d90c1a2" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:20:17.763752 19205 leader_election.cc:290] T ee9ed67006144797b1de6ea8c2013a68 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:17.741210 19246 leader_election.cc:290] T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
W20260709 13:20:17.777658 19402 consensus_meta.cc:287] T 8cc959fb7f704fba9009c0595467d074 P 647a343a542d4a139fe106c19095129b: Time spent flushing consensus metadata: real 0.526s	user 0.005s	sys 0.006s
I20260709 13:20:17.778446 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7ca61592e848ea9b0fb81a2ce33cdc" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:17.778475 16220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18ad7e3af5504571a69d10c00750a01c" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:20:17.754004 19413 raft_consensus.cc:697] T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b [term 5 LEADER]: Becoming Leader. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:20:17.781559 16144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b7d1ab519044eda2d5e04a07242f25" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:17.776556 19198 raft_consensus.cc:515] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:17.782436 16143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b7d1ab519044eda2d5e04a07242f25" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:17.743508 19382 raft_consensus.cc:515] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:17.783737 19068 leader_election.cc:290] T 8638ee363273426db1393dd81e5c7b72 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:17.784133 19376 raft_consensus.cc:3055] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:20:17.686168 19359 raft_consensus.cc:515] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:17.811604 16143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5718c623ebee4d2bb770f0f9ddb2f164" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:20:17.813841 19359 leader_election.cc:290] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:20:17.787657 16222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8158bbd17b994d2eb96f560bd0f1b88d" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:17.814682 16222 raft_consensus.cc:2468] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 7.
I20260709 13:20:17.787918 19361 raft_consensus.cc:697] T c9944d5e4b8c4109af9955c4f08c9095 P 90f3da57796b488c9087a2f523adf045 [term 8 LEADER]: Becoming Leader. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:20:17.815759 16222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:17.815893 19361 consensus_queue.cc:237] T c9944d5e4b8c4109af9955c4f08c9095 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:17.816992 16222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada5a004ba7c46819051e3ccec71ffdb" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:17.818081 16222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d4ab870a05347868956aa8b841b2981" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 9 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:17.818842 16291 raft_consensus.cc:1240] T fffab26427e24d398e2695aad40ca134 P 90f3da57796b488c9087a2f523adf045 [term 6 LEADER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 5. Current term is 6. Ops: []
I20260709 13:20:17.782243 19176 raft_consensus.cc:515] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:17.819836 16145 raft_consensus.cc:1240] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 8. Current term is 9. Ops: []
I20260709 13:20:17.819900 16291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4dc6b0541ca431ebc757beac00521f1" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:17.820494 16291 raft_consensus.cc:2468] T e4dc6b0541ca431ebc757beac00521f1 P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 6.
I20260709 13:20:17.820907 19176 leader_election.cc:290] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 9 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:17.821261 16291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2794af5b99984efebdee564035841cb0" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:17.551988 18599 consensus_queue.cc:260] T af57996a8be84d398f0f023e8d08ef77 P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:17.745613 16151 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0a9de347c711477e87c582d003f36925"
dest_uuid: "647a343a542d4a139fe106c19095129b"
mode: GRACEFUL
new_leader_uuid: "90f3da57796b488c9087a2f523adf045"
 from {username='slave'} at 127.0.0.1:40290
I20260709 13:20:17.822116 16151 raft_consensus.cc:606] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Received request to transfer leadership to TS 90f3da57796b488c9087a2f523adf045
I20260709 13:20:17.822273 18599 raft_consensus.cc:3060] T af57996a8be84d398f0f023e8d08ef77 P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:20:17.822348 16291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a9de347c711477e87c582d003f36925" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:20:17.823237 16143 raft_consensus.cc:2468] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 7.
I20260709 13:20:17.824326 16143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66f75124215f4ca6bef831e174a792d8" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:20:17.794001 16220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02838c86531d4f98bf58d1c22b8baf77" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:17.827683 16220 raft_consensus.cc:2393] T 02838c86531d4f98bf58d1c22b8baf77 P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 8: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
W20260709 13:20:17.828289 16224 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57600 (request call id 1227) took 1899 ms (client timeout 2388 ms). Trace:
W20260709 13:20:17.828691 16224 rpcz_store.cc:269] 0709 13:20:15.928741 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:15.963317 (+ 34576us) service_pool.cc:224] Handling call
0709 13:20:17.828125 (+1864808us) spinlock_profiling.cc:243] Waited 1.08 s on lock 0x7b54002977d4. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:20:17.828235 (+   110us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2476845952}
I20260709 13:20:17.829388 16143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c61a39fd4f84424b129c1a6f9ba404d" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:17.798540 16296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18ad7e3af5504571a69d10c00750a01c" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:20:17.831795 16296 raft_consensus.cc:2393] T 18ad7e3af5504571a69d10c00750a01c P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 7: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
I20260709 13:20:17.832487 16296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c2095a5f4ad49408dbc7f69c606120d" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:17.833405 16296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f82224f4564fea93572e8206bad1fe" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 7 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:17.833892 16296 raft_consensus.cc:2468] T 23f82224f4564fea93572e8206bad1fe P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 6.
I20260709 13:20:17.834728 16296 raft_consensus.cc:3060] T 02838c86531d4f98bf58d1c22b8baf77 P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:20:17.800730 16115 leader_election.cc:304] T 8a878999c0024d2abe517484180290fc P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:20:17.835245 16294 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48684 (request call id 2224) took 2341 ms. Trace:
I20260709 13:20:17.793486 19402 raft_consensus.cc:515] T 8cc959fb7f704fba9009c0595467d074 P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:17.835614 16294 rpcz_store.cc:276] 0709 13:20:15.493608 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:16.013947 (+520339us) service_pool.cc:224] Handling call
0709 13:20:16.014178 (+   231us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:16.192011 (+177833us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:17.834975 (+1642964us) spinlock_profiling.cc:243] Waited 119 ms on lock 0x7b5400178754. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:20:17.835133 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":273214720}
I20260709 13:20:17.837262 19450 raft_consensus.cc:2749] T 8a878999c0024d2abe517484180290fc P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:20:17.784930 19066 consensus_queue.cc:1059] T c0eab25ce4c64e018a041beeda2867db P 90f3da57796b488c9087a2f523adf045 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.018s
I20260709 13:20:17.838210 19314 raft_consensus.cc:3060] T 434bff3082d141838068f323bdb0d1ed P cb82cc7bd6db46d980c75237a1402860 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:20:17.838824 19066 raft_consensus.cc:3055] T c0eab25ce4c64e018a041beeda2867db P 90f3da57796b488c9087a2f523adf045 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:20:17.839188 19066 raft_consensus.cc:740] T c0eab25ce4c64e018a041beeda2867db P 90f3da57796b488c9087a2f523adf045 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:20:17.839963 19066 consensus_queue.cc:260] T c0eab25ce4c64e018a041beeda2867db P 90f3da57796b488c9087a2f523adf045 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:17.773401 16265 leader_election.cc:304] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
W20260709 13:20:17.840801 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 369618us
W20260709 13:20:17.841291 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 67971.4us
I20260709 13:20:17.843590 16142 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40830 (request call id 1235) took 3060 ms. Trace:
I20260709 13:20:17.841013 19066 raft_consensus.cc:3060] T c0eab25ce4c64e018a041beeda2867db P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:20:17.843963 16142 rpcz_store.cc:276] 0709 13:20:14.783565 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:14.783778 (+   213us) service_pool.cc:224] Handling call
0709 13:20:17.456550 (+2672772us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:20:17.465455 (+  8905us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:17.843418 (+377963us) spinlock_profiling.cc:243] Waited 2.67 s on lock 0x7b54000d0550. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5e8ad 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:20:17.843538 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6146002176}
I20260709 13:20:17.845216 16142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2372981006d4b218215c28964dea623" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:17.843977 19068 raft_consensus.cc:2804] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:20:17.847823 19068 raft_consensus.cc:493] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:20:17.848389 19068 raft_consensus.cc:3060] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:20:17.851418 16144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a878999c0024d2abe517484180290fc" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:20:17.851601 16223 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57600 (request call id 1228) took 1890 ms. Trace:
I20260709 13:20:17.852622 16144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1df93779209748b9860f6d893dd3de53" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 8 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
W20260709 13:20:17.853317 16217 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33050 (request call id 546) took 1669 ms (client timeout 1742 ms). Trace:
W20260709 13:20:17.853863 16217 rpcz_store.cc:269] 0709 13:20:16.183870 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:16.334814 (+150944us) service_pool.cc:224] Handling call
0709 13:20:17.853091 (+1518277us) spinlock_profiling.cc:243] Waited 237 ms on lock 0x7b54001419d4. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:20:17.853242 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":545005440}
I20260709 13:20:17.854876 16296 raft_consensus.cc:1275] T 02838c86531d4f98bf58d1c22b8baf77 P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:20:17.855760 18599 consensus_queue.cc:1048] T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:20:17.857033 19198 leader_election.cc:290] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:17.857543 19198 consensus_queue.cc:1059] T b6f07842d3aa47858df5665874c86f3a P 647a343a542d4a139fe106c19095129b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:20:17.858279 19198 consensus_queue.cc:1048] T d16dfad0c3b540e5b4b86492ddf87208 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:20:17.858495 16143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3884f864a96d428785895e5463e5728f" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:17.859321 19246 raft_consensus.cc:3055] T b6f07842d3aa47858df5665874c86f3a P 647a343a542d4a139fe106c19095129b [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:20:17.859674 19246 raft_consensus.cc:740] T b6f07842d3aa47858df5665874c86f3a P 647a343a542d4a139fe106c19095129b [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:20:17.860386 19246 consensus_queue.cc:260] T b6f07842d3aa47858df5665874c86f3a P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:17.861470 19246 raft_consensus.cc:3060] T b6f07842d3aa47858df5665874c86f3a P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:20:17.788223 19080 raft_consensus.cc:515] T 3884f864a96d428785895e5463e5728f P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:17.864316 19363 raft_consensus.cc:3055] T d2ab0ac8347641e6955e9bce7d90c1a2 P 1ca2725d290e47d68b1a4b609c9816ac [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:20:17.864787 19363 raft_consensus.cc:740] T d2ab0ac8347641e6955e9bce7d90c1a2 P 1ca2725d290e47d68b1a4b609c9816ac [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:20:17.865057 19080 leader_election.cc:290] T 3884f864a96d428785895e5463e5728f P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:17.866140 16143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26354361e9b044cea1c2b86c4882fe92" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:17.865711 19363 consensus_queue.cc:260] T d2ab0ac8347641e6955e9bce7d90c1a2 P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:17.867919 16143 raft_consensus.cc:2410] T 26354361e9b044cea1c2b86c4882fe92 P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 for term 7 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 6.
I20260709 13:20:17.868741 16295 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48674 (request call id 2325) took 1366 ms. Trace:
I20260709 13:20:17.824900 16221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdbd78acb4cf46298e0de8ba0d130e19" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:17.869077 16143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "829b4ad8c5a54eababeea1181b960d4b" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:17.794888 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5ad4eb281564936a59f41e387a47156" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:17.869762 16143 raft_consensus.cc:2468] T 829b4ad8c5a54eababeea1181b960d4b P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 7.
I20260709 13:20:17.869364 16295 rpcz_store.cc:276] 0709 13:20:16.502511 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:17.190844 (+688333us) service_pool.cc:224] Handling call
0709 13:20:17.191237 (+   393us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:17.461007 (+269770us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:17.868416 (+407409us) spinlock_profiling.cc:243] Waited 202 ms on lock 0x7b5400210054. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:20:17.868660 (+   244us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":464400640}
I20260709 13:20:17.852164 16223 rpcz_store.cc:276] 0709 13:20:15.960958 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:15.961077 (+   119us) service_pool.cc:224] Handling call
0709 13:20:15.961361 (+   284us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:17.382039 (+1420678us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:17.851269 (+469230us) spinlock_profiling.cc:243] Waited 1.34 s on lock 0x7b54002981d4. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:20:17.851524 (+   255us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3072948992}
I20260709 13:20:17.870899 16143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d16dfad0c3b540e5b4b86492ddf87208" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:17.826696 16151 raft_consensus.cc:614] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 13:20:17.871532 16146 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40844 (request call id 2060) took 1773 ms (client timeout 1906 ms). Trace:
I20260709 13:20:17.824501 19382 leader_election.cc:290] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:20:17.803267 19364 raft_consensus.cc:2804] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 13:20:17.871958 16146 rpcz_store.cc:269] 0709 13:20:16.098118 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:16.098916 (+   798us) service_pool.cc:224] Handling call
0709 13:20:17.871368 (+1772452us) spinlock_profiling.cc:243] Waited 183 ms on lock 0x7b540006ccd4. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:20:17.871483 (+   115us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":421210496}
I20260709 13:20:17.872668 16220 raft_consensus.cc:1275] T 829b4ad8c5a54eababeea1181b960d4b P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 7 index: 9. (index mismatch)
I20260709 13:20:17.872797 16225 raft_consensus.cc:2468] T e5ad4eb281564936a59f41e387a47156 P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 8.
I20260709 13:20:17.892774 16146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62fe3346cd3c42c4a55e14af9073bca4" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:17.893399 16146 raft_consensus.cc:2468] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 9.
I20260709 13:20:17.894510 16146 raft_consensus.cc:1275] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:20:17.779865 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6f07842d3aa47858df5665874c86f3a" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
W20260709 13:20:17.873978 16789 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:17.897274 16789 leader_election.cc:304] T d2ab0ac8347641e6955e9bce7d90c1a2 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:20:17.898526 16789 leader_election.cc:304] T 3a7ca61592e848ea9b0fb81a2ce33cdc P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
W20260709 13:20:17.797039 16192 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 90077.8us
I20260709 13:20:17.875569 16221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4dc6b0541ca431ebc757beac00521f1" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:17.830080 16224 raft_consensus.cc:1240] T 8a878999c0024d2abe517484180290fc P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 6. Current term is 7. Ops: [6.3-6.3]
I20260709 13:20:17.877940 16291 raft_consensus.cc:3060] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:20:17.843588 16265 leader_election.cc:304] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
I20260709 13:20:17.738524 19416 raft_consensus.cc:515] T 1df93779209748b9860f6d893dd3de53 P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:17.904024 19416 leader_election.cc:290] T 1df93779209748b9860f6d893dd3de53 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 8 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:17.904610 19306 raft_consensus.cc:493] T 19baa7bc367844c288bca8f81439c413 P cb82cc7bd6db46d980c75237a1402860 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:20:17.905432 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c2095a5f4ad49408dbc7f69c606120d" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:17.905177 19306 raft_consensus.cc:515] T 19baa7bc367844c288bca8f81439c413 P cb82cc7bd6db46d980c75237a1402860 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:17.905632 19213 raft_consensus.cc:515] T 8158bbd17b994d2eb96f560bd0f1b88d P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:17.906072 16226 raft_consensus.cc:2468] T 7c2095a5f4ad49408dbc7f69c606120d P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 5.
I20260709 13:20:17.811594 19376 raft_consensus.cc:740] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:20:17.907291 19398 raft_consensus.cc:2804] T d2ab0ac8347641e6955e9bce7d90c1a2 P cb82cc7bd6db46d980c75237a1402860 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:20:17.907859 19398 raft_consensus.cc:697] T d2ab0ac8347641e6955e9bce7d90c1a2 P cb82cc7bd6db46d980c75237a1402860 [term 8 LEADER]: Becoming Leader. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:20:17.907946 19402 leader_election.cc:290] T 8cc959fb7f704fba9009c0595467d074 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 8 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:17.908038 19376 consensus_queue.cc:260] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:17.908978 19423 consensus_queue.cc:1048] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:20:17.908851 19398 consensus_queue.cc:237] T d2ab0ac8347641e6955e9bce7d90c1a2 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:17.909459 19376 raft_consensus.cc:3060] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:20:17.910341 19423 consensus_queue.cc:1048] T 746aabf856ab401c9ef5e859371e7b67 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:20:17.911644 19233 raft_consensus.cc:1064] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac: attempting to promote NON_VOTER cb82cc7bd6db46d980c75237a1402860 to VOTER
I20260709 13:20:17.912858 16191 leader_election.cc:304] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
I20260709 13:20:17.913745 16191 leader_election.cc:304] T 66f75124215f4ca6bef831e174a792d8 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
I20260709 13:20:17.914395 16151 raft_consensus.cc:1240] T d16dfad0c3b540e5b4b86492ddf87208 P 647a343a542d4a139fe106c19095129b [term 7 LEADER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 6. Current term is 7. Ops: []
I20260709 13:20:17.914016 19233 consensus_queue.cc:237] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.6, Last appended by leader: 5, Current term: 6, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:17.876374 19363 raft_consensus.cc:3060] T d2ab0ac8347641e6955e9bce7d90c1a2 P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:20:17.915426 19385 raft_consensus.cc:2804] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:20:17.915498 19037 raft_consensus.cc:2749] T 66f75124215f4ca6bef831e174a792d8 P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:20:17.916003 19385 raft_consensus.cc:697] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac [term 7 LEADER]: Becoming Leader. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:20:17.916626 19364 raft_consensus.cc:493] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:17.917119 19364 raft_consensus.cc:3060] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:20:17.916882 19385 consensus_queue.cc:237] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.8, Last appended by leader: 8, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:17.917806 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5d59407eb545b895d8d922e0dd69fd" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:20:17.918989 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746aabf856ab401c9ef5e859371e7b67" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:17.919843 16191 leader_election.cc:304] T 8a878999c0024d2abe517484180290fc P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:20:17.883486 16299 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40474 (request call id 464) took 1742 ms. Trace:
I20260709 13:20:17.920406 16224 raft_consensus.cc:1275] T fffab26427e24d398e2695aad40ca134 P 1ca2725d290e47d68b1a4b609c9816ac [term 6 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:20:17.920485 16299 rpcz_store.cc:276] 0709 13:20:16.140673 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:16.953079 (+812406us) service_pool.cc:224] Handling call
0709 13:20:16.953410 (+   331us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:20:17.055706 (+102296us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:17.055754 (+    48us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:20:17.059675 (+  3921us) log.cc:844] Serialized 26 byte log entry
0709 13:20:17.229552 (+169877us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:20:17.229606 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:17.867389 (+637783us) spinlock_profiling.cc:243] Waited 92.8 ms on lock 0x7b540024ca54. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:20:17.867441 (+    52us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:20:17.867487 (+    46us) raft_consensus.cc:1685] finished
0709 13:20:17.867505 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:17.883404 (+ 15899us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5399,"raft.queue_time_us":9738,"raft.run_cpu_time_us":109,"raft.run_wall_time_us":3663,"spinlock_wait_cycles":213523712,"thread_start_us":134958,"threads_started":1,"wal-append.queue_time_us":259397}
I20260709 13:20:17.884524 16143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f2da98864b4425bf7c0055951fdd4b" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:20:17.923305 16143 raft_consensus.cc:1240] T 8cc959fb7f704fba9009c0595467d074 P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 7. Current term is 8. Ops: [7.6-7.6]
W20260709 13:20:17.923243 19315 consensus_queue.cc:1175] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "647a343a542d4a139fe106c19095129b" responder_term: 6 status { last_received { term: 6 index: 6 } last_committed_idx: 6 last_received_current_leader { term: 6 index: 6 } }
I20260709 13:20:17.924592 16143 raft_consensus.cc:1240] T 18ad7e3af5504571a69d10c00750a01c P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 6. Current term is 7. Ops: []
I20260709 13:20:17.876093 16223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18ad7e3af5504571a69d10c00750a01c" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:20:17.878386 16222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4dc6b0541ca431ebc757beac00521f1" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:17.906899 16147 raft_consensus.cc:2393] T 8158bbd17b994d2eb96f560bd0f1b88d P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 8: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
I20260709 13:20:17.886727 16295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a96373bb0e14dce8a266b2fc6c58507" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 7 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
W20260709 13:20:17.927086 16189 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:17.927296 16295 raft_consensus.cc:3060] T 5a96373bb0e14dce8a266b2fc6c58507 P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:20:17.927822 16264 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:17.873404 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a96373bb0e14dce8a266b2fc6c58507" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 7 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
W20260709 13:20:17.929373 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 85843.7us
I20260709 13:20:17.931026 16817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e4bfb9090c8460eb14fa7e0d818e7f1" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:17.931636 16817 raft_consensus.cc:2468] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P cb82cc7bd6db46d980c75237a1402860 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 7.
W20260709 13:20:17.885675 16116 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:17.901839 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "829b4ad8c5a54eababeea1181b960d4b" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:17.933889 16191 leader_election.cc:304] T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:20:17.934361 16225 raft_consensus.cc:2468] T 829b4ad8c5a54eababeea1181b960d4b P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 7.
I20260709 13:20:17.919505 19413 consensus_queue.cc:237] T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:17.935374 16292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87ac0d1f7e8a419f9bcbc04aee0e067e" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:17.935983 16292 raft_consensus.cc:2468] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 7.
I20260709 13:20:17.937462 16292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1df93779209748b9860f6d893dd3de53" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 8 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:20:17.937943 16292 raft_consensus.cc:3060] T 1df93779209748b9860f6d893dd3de53 P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:20:17.900601 16148 raft_consensus.cc:1275] T fffab26427e24d398e2695aad40ca134 P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:20:17.906939 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee9ed67006144797b1de6ea8c2013a68" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:17.926340 16149 raft_consensus.cc:1240] T 8a878999c0024d2abe517484180290fc P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 6. Current term is 7. Ops: [6.3-6.3]
I20260709 13:20:17.940954 16148 raft_consensus.cc:1240] T 5a96373bb0e14dce8a266b2fc6c58507 P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 6. Current term is 7. Ops: []
I20260709 13:20:17.930754 16265 leader_election.cc:304] T 57a744f5aaf944678a63469cbf514015 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:20:17.944461 16191 leader_election.cc:304] T 829b4ad8c5a54eababeea1181b960d4b P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
I20260709 13:20:17.948804 16819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "829b4ad8c5a54eababeea1181b960d4b" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
W20260709 13:20:17.949956 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 314546us
I20260709 13:20:17.961486 16144 raft_consensus.cc:2393] T 1df93779209748b9860f6d893dd3de53 P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 8: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
I20260709 13:20:17.962558 16822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5d59407eb545b895d8d922e0dd69fd" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
W20260709 13:20:17.962749 16790 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:20:17.963354 16790 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:17.965530 16822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66f75124215f4ca6bef831e174a792d8" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
W20260709 13:20:17.966439 16297 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48684 (request call id 2221) took 2457 ms (client timeout 1619 ms). Trace:
W20260709 13:20:17.966980 16297 rpcz_store.cc:269] 0709 13:20:15.492819 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:15.503329 (+ 10510us) service_pool.cc:224] Handling call
0709 13:20:17.950264 (+2446935us) spinlock_profiling.cc:243] Waited 682 ms on lock 0x7b54002e1654. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:20:17.950416 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1567710080}
I20260709 13:20:17.969607 16819 raft_consensus.cc:1275] T 8638ee363273426db1393dd81e5c7b72 P cb82cc7bd6db46d980c75237a1402860 [term 7 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:20:17.960820 19198 raft_consensus.cc:493] T 08c74490c40243a88e62d1760b5cd814 P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:20:17.972185 19198 raft_consensus.cc:515] T 08c74490c40243a88e62d1760b5cd814 P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:17.977573 16222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1df93779209748b9860f6d893dd3de53" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 8 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:17.978207 16115 leader_election.cc:304] T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:20:17.979382 16116 leader_election.cc:304] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:20:17.978684 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8638ee363273426db1393dd81e5c7b72" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:17.982698 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d16dfad0c3b540e5b4b86492ddf87208" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:17.986601 16823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5718c623ebee4d2bb770f0f9ddb2f164" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:20:17.981328 16115 leader_election.cc:304] T 23f82224f4564fea93572e8206bad1fe P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:20:17.990896 16819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26354361e9b044cea1c2b86c4882fe92" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:17.991911 16222 raft_consensus.cc:1275] T 8c11074579b046d99e585c2818e18a2c P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 7 index: 9. (index mismatch)
I20260709 13:20:17.992069 16820 raft_consensus.cc:1275] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860 [term 7 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:20:17.991987 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f2da98864b4425bf7c0055951fdd4b" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
W20260709 13:20:17.996838 16150 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40844 (request call id 2066) took 1795 ms (client timeout 1705 ms). Trace:
I20260709 13:20:17.925910 19357 raft_consensus.cc:2749] T 3a7ca61592e848ea9b0fb81a2ce33cdc P cb82cc7bd6db46d980c75237a1402860 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:20:17.964635 19421 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 6 current term: 6 is higher.
I20260709 13:20:18.001387 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cc959fb7f704fba9009c0595467d074" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:20:18.012465 16297 raft_consensus.cc:3060] T 8cc959fb7f704fba9009c0595467d074 P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:20:18.015955 16115 leader_election.cc:304] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
W20260709 13:20:17.997489 16150 rpcz_store.cc:269] 0709 13:20:16.201109 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:16.201266 (+   157us) service_pool.cc:224] Handling call
0709 13:20:17.996584 (+1795318us) spinlock_profiling.cc:243] Waited 493 ms on lock 0x7b540041ebd4. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:20:17.996757 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1132909952}
I20260709 13:20:17.964679 19390 raft_consensus.cc:2749] T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:20:17.960502 19315 raft_consensus.cc:2749] T 8a878999c0024d2abe517484180290fc P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:20:18.024300 16816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87147731a68400e8cce9b879daa7c5d" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:20:18.025094 16816 raft_consensus.cc:3060] T b87147731a68400e8cce9b879daa7c5d P cb82cc7bd6db46d980c75237a1402860 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:20:17.977070 19210 raft_consensus.cc:493] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:17.990200 18944 raft_consensus.cc:2804] T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:20:17.990932 19080 raft_consensus.cc:2804] T 57a744f5aaf944678a63469cbf514015 P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:20:17.976572 19316 raft_consensus.cc:2804] T 829b4ad8c5a54eababeea1181b960d4b P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:20:17.986857 19412 raft_consensus.cc:493] T c9944d5e4b8c4109af9955c4f08c9095 P cb82cc7bd6db46d980c75237a1402860 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:17.960408 19407 consensus_queue.cc:1048] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.065s
I20260709 13:20:18.061053 19415 consensus_queue.cc:1048] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:20:18.066450 19198 leader_election.cc:290] T 08c74490c40243a88e62d1760b5cd814 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:17.989974 19088 consensus_queue.cc:1048] T fffab26427e24d398e2695aad40ca134 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.064s
W20260709 13:20:18.069083 16264 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:17.996716 19383 consensus_queue.cc:1059] T 8cc959fb7f704fba9009c0595467d074 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.057s
I20260709 13:20:18.070014 19383 raft_consensus.cc:3055] T 8cc959fb7f704fba9009c0595467d074 P 1ca2725d290e47d68b1a4b609c9816ac [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:20:18.070377 19383 raft_consensus.cc:740] T 8cc959fb7f704fba9009c0595467d074 P 1ca2725d290e47d68b1a4b609c9816ac [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:20:18.071203 19383 consensus_queue.cc:260] T 8cc959fb7f704fba9009c0595467d074 P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.6, Last appended by leader: 6, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:18.072374 19383 raft_consensus.cc:3060] T 8cc959fb7f704fba9009c0595467d074 P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:20:18.077360 16144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87ac0d1f7e8a419f9bcbc04aee0e067e" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:18.078513 16144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3884f864a96d428785895e5463e5728f" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 7 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:18.079461 16151 raft_consensus.cc:1240] T 1df93779209748b9860f6d893dd3de53 P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 7. Current term is 8. Ops: [7.5-7.5]
I20260709 13:20:18.041246 19380 raft_consensus.cc:2804] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:20:18.081669 19380 raft_consensus.cc:493] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Starting leader election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:20:18.082042 19380 raft_consensus.cc:3060] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:20:18.084921 19197 raft_consensus.cc:3060] T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:20:18.033854 18944 raft_consensus.cc:493] T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Starting leader election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:20:18.089089 18944 raft_consensus.cc:3060] T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:20:18.092120 19213 leader_election.cc:290] T 8158bbd17b994d2eb96f560bd0f1b88d P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 8 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:18.093046 19412 raft_consensus.cc:515] T c9944d5e4b8c4109af9955c4f08c9095 P cb82cc7bd6db46d980c75237a1402860 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:18.095374 19412 leader_election.cc:290] T c9944d5e4b8c4109af9955c4f08c9095 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
W20260709 13:20:18.040693 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 49530.6us
I20260709 13:20:18.028223 19205 raft_consensus.cc:2804] T 23f82224f4564fea93572e8206bad1fe P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:20:17.998060 19378 consensus_queue.cc:1059] T 5a96373bb0e14dce8a266b2fc6c58507 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.055s
I20260709 13:20:18.054304 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdbd78acb4cf46298e0de8ba0d130e19" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:18.056242 16821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b7d1ab519044eda2d5e04a07242f25" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:20:17.974242 19176 consensus_queue.cc:1048] T 02838c86531d4f98bf58d1c22b8baf77 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.057s
I20260709 13:20:17.996888 19113 consensus_queue.cc:1059] T 18ad7e3af5504571a69d10c00750a01c P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.058s
I20260709 13:20:18.110313 16116 leader_election.cc:304] T 7c2095a5f4ad49408dbc7f69c606120d P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 90f3da57796b488c9087a2f523adf045
I20260709 13:20:18.112859 16116 leader_election.cc:304] T 18ad7e3af5504571a69d10c00750a01c P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:20:18.113963 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ad5dbfa99ee4b8d9e28be92b3f36db5" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:18.052139 19306 leader_election.cc:290] T 19baa7bc367844c288bca8f81439c413 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:18.107352 16298 raft_consensus.cc:2468] T cdbd78acb4cf46298e0de8ba0d130e19 P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 7.
W20260709 13:20:18.100890 16266 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:18.121210 16824 raft_consensus.cc:1275] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860 [term 7 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 13:20:18.041932 19080 raft_consensus.cc:493] T 57a744f5aaf944678a63469cbf514015 P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:18.108233 16821 raft_consensus.cc:2393] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 5: Already voted for candidate cb82cc7bd6db46d980c75237a1402860 in this term.
I20260709 13:20:18.123248 16116 leader_election.cc:304] T 829b4ad8c5a54eababeea1181b960d4b P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
I20260709 13:20:18.020198 16295 raft_consensus.cc:2468] T 5a96373bb0e14dce8a266b2fc6c58507 P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 7.
I20260709 13:20:18.053545 19316 raft_consensus.cc:493] T 829b4ad8c5a54eababeea1181b960d4b P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:18.053515 19404 raft_consensus.cc:493] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:20:18.104655 16291 raft_consensus.cc:2468] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 9.
I20260709 13:20:18.156015 16151 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "19baa7bc367844c288bca8f81439c413"
dest_uuid: "647a343a542d4a139fe106c19095129b"
mode: GRACEFUL
new_leader_uuid: "90f3da57796b488c9087a2f523adf045"
 from {username='slave'} at 127.0.0.1:40290
I20260709 13:20:18.161571 16821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434bff3082d141838068f323bdb0d1ed" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:18.169092 16821 raft_consensus.cc:2468] T 434bff3082d141838068f323bdb0d1ed P cb82cc7bd6db46d980c75237a1402860 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 8.
W20260709 13:20:18.172374 16115 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:18.041436 18599 consensus_queue.cc:1059] T 8d4ab870a05347868956aa8b841b2981 P 647a343a542d4a139fe106c19095129b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: INVALID_TERM, Last received: 7.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.093s
I20260709 13:20:18.113540 19205 raft_consensus.cc:493] T 23f82224f4564fea93572e8206bad1fe P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:18.120174 19414 raft_consensus.cc:493] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:18.042156 19210 raft_consensus.cc:515] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:18.130270 19213 raft_consensus.cc:2804] T 7c2095a5f4ad49408dbc7f69c606120d P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:20:18.204787 16291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0eab25ce4c64e018a041beeda2867db" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:20:18.120812 19088 raft_consensus.cc:493] T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:20:18.141152 19080 raft_consensus.cc:3060] T 57a744f5aaf944678a63469cbf514015 P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:20:18.135133 19198 raft_consensus.cc:2749] T 18ad7e3af5504571a69d10c00750a01c P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:20:18.215912 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c11074579b046d99e585c2818e18a2c" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:18.230664 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1df93779209748b9860f6d893dd3de53" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 8 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:20:18.231675 16824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746aabf856ab401c9ef5e859371e7b67" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:18.232282 16824 raft_consensus.cc:2468] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 7.
I20260709 13:20:18.144217 19421 raft_consensus.cc:2804] T 829b4ad8c5a54eababeea1181b960d4b P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:20:18.221321 16220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cc959fb7f704fba9009c0595467d074" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:20:18.222112 16222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8158bbd17b994d2eb96f560bd0f1b88d" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:20:18.223620 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c74490c40243a88e62d1760b5cd814" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:18.238679 16222 raft_consensus.cc:3060] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:20:18.228660 16297 raft_consensus.cc:2468] T 8cc959fb7f704fba9009c0595467d074 P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 8.
I20260709 13:20:18.239725 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4f65cdf00b748bb963ee6e81ef43b79" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:20:18.240756 16297 raft_consensus.cc:2393] T c4f65cdf00b748bb963ee6e81ef43b79 P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 9: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
I20260709 13:20:18.240808 16221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87ac0d1f7e8a419f9bcbc04aee0e067e" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:18.219770 16291 raft_consensus.cc:2410] T c0eab25ce4c64e018a041beeda2867db P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac for term 8 because replica has last-logged OpId of term: 7 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 13:20:18.128073 19407 consensus_queue.cc:1059] T 1df93779209748b9860f6d893dd3de53 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: INVALID_TERM, Last received: 4.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.086s
I20260709 13:20:18.150208 19359 raft_consensus.cc:2749] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:20:18.145318 19316 raft_consensus.cc:3060] T 829b4ad8c5a54eababeea1181b960d4b P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:20:18.231950 19142 raft_consensus.cc:3055] T 5a96373bb0e14dce8a266b2fc6c58507 P 1ca2725d290e47d68b1a4b609c9816ac [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:20:18.267547 19142 raft_consensus.cc:740] T 5a96373bb0e14dce8a266b2fc6c58507 P 1ca2725d290e47d68b1a4b609c9816ac [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:20:18.129346 19416 consensus_queue.cc:1048] T e5ad4eb281564936a59f41e387a47156 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 3, Time since last communication: 0.087s
I20260709 13:20:18.204274 19205 raft_consensus.cc:3060] T 23f82224f4564fea93572e8206bad1fe P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:20:18.247289 16823 raft_consensus.cc:2393] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 7: Already voted for candidate cb82cc7bd6db46d980c75237a1402860 in this term.
I20260709 13:20:18.279660 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c2095a5f4ad49408dbc7f69c606120d" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:18.227957 19377 raft_consensus.cc:3055] T 18ad7e3af5504571a69d10c00750a01c P 1ca2725d290e47d68b1a4b609c9816ac [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:20:18.143951 19419 consensus_queue.cc:1048] T 02838c86531d4f98bf58d1c22b8baf77 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.094s
W20260709 13:20:18.283507 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 139856us
I20260709 13:20:18.284468 16820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a9de347c711477e87c582d003f36925" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:20:18.309157 16820 raft_consensus.cc:3060] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:20:18.285807 16791 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:18.217854 19213 raft_consensus.cc:493] T 7c2095a5f4ad49408dbc7f69c606120d P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:18.238034 19421 raft_consensus.cc:493] T 829b4ad8c5a54eababeea1181b960d4b P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:18.319365 16824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2794af5b99984efebdee564035841cb0" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:18.243031 19315 raft_consensus.cc:493] T 02838c86531d4f98bf58d1c22b8baf77 P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:18.336292 16151 raft_consensus.cc:606] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Received request to transfer leadership to TS 90f3da57796b488c9087a2f523adf045
I20260709 13:20:18.336826 16151 raft_consensus.cc:614] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:20:18.232666 19450 consensus_queue.cc:1059] T 8d4ab870a05347868956aa8b841b2981 P 647a343a542d4a139fe106c19095129b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: INVALID_TERM, Last received: 7.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.079s
I20260709 13:20:18.186012 19404 raft_consensus.cc:515] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } }
I20260709 13:20:18.227789 19414 raft_consensus.cc:515] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:18.347957 16824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "829b4ad8c5a54eababeea1181b960d4b" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
W20260709 13:20:18.343575 16264 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:18.345180 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c74490c40243a88e62d1760b5cd814" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:18.353933 16226 raft_consensus.cc:1275] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:20:18.239460 19088 raft_consensus.cc:515] T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:18.321136 19213 raft_consensus.cc:3060] T 7c2095a5f4ad49408dbc7f69c606120d P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:20:18.308612 19377 raft_consensus.cc:740] T 18ad7e3af5504571a69d10c00750a01c P 1ca2725d290e47d68b1a4b609c9816ac [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:20:18.337852 19378 raft_consensus.cc:3055] T 1df93779209748b9860f6d893dd3de53 P 1ca2725d290e47d68b1a4b609c9816ac [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:20:18.328473 19421 raft_consensus.cc:3060] T 829b4ad8c5a54eababeea1181b960d4b P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:20:18.374625 16820 raft_consensus.cc:2468] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 9.
I20260709 13:20:18.375026 16824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f82224f4564fea93572e8206bad1fe" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 7 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:18.360913 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3884f864a96d428785895e5463e5728f" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:18.340415 19431 raft_consensus.cc:2749] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:20:18.389976 16816 raft_consensus.cc:2468] T b87147731a68400e8cce9b879daa7c5d P cb82cc7bd6db46d980c75237a1402860 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 9.
I20260709 13:20:18.393095 16292 raft_consensus.cc:2468] T 1df93779209748b9860f6d893dd3de53 P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 8.
I20260709 13:20:18.398641 16292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87147731a68400e8cce9b879daa7c5d" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:18.399866 16292 raft_consensus.cc:2393] T b87147731a68400e8cce9b879daa7c5d P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 in current term 9: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
I20260709 13:20:18.401268 16292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e4bfb9090c8460eb14fa7e0d818e7f1" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:18.365458 19450 consensus_queue.cc:1243] T 8d4ab870a05347868956aa8b841b2981 P 647a343a542d4a139fe106c19095129b [LEADER]: Peer 1ca2725d290e47d68b1a4b609c9816ac log is divergent from this leader: its last log entry 8.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260709 13:20:18.335958 19315 raft_consensus.cc:515] T 02838c86531d4f98bf58d1c22b8baf77 P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:18.420034 16824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee9ed67006144797b1de6ea8c2013a68" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:18.419301 16292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8158bbd17b994d2eb96f560bd0f1b88d" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:20:18.425225 16292 raft_consensus.cc:3060] T 8158bbd17b994d2eb96f560bd0f1b88d P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:20:18.426102 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a959dbecc2648eb85d1cc550f5a3245" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:18.370143 19378 raft_consensus.cc:740] T 1df93779209748b9860f6d893dd3de53 P 1ca2725d290e47d68b1a4b609c9816ac [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
W20260709 13:20:18.400044 19068 consensus_meta.cc:287] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045: Time spent flushing consensus metadata: real 0.548s	user 0.008s	sys 0.004s
W20260709 13:20:18.537160 16790 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.549s	user 0.021s	sys 0.024s
I20260709 13:20:18.541105 16824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5d59407eb545b895d8d922e0dd69fd" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:20:18.542668 16297 raft_consensus.cc:1275] T 9e65676b21b748728ad39668cfe722ec P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
W20260709 13:20:18.539439 19454 log.cc:927] Time spent T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 0.429s	user 0.001s	sys 0.001s
I20260709 13:20:18.373189 18599 consensus_queue.cc:1048] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.060s
W20260709 13:20:18.560254 16790 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.571s	user 0.021s	sys 0.024s
W20260709 13:20:18.572432 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 594111us
I20260709 13:20:18.540266 19068 raft_consensus.cc:515] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:18.549882 19377 consensus_queue.cc:260] T 18ad7e3af5504571a69d10c00750a01c P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } }
W20260709 13:20:18.675076 19455 log.cc:927] Time spent T b6f07842d3aa47858df5665874c86f3a P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 0.376s	user 0.001s	sys 0.000s
W20260709 13:20:18.700106 19361 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.796s	user 0.049s	sys 0.034s
I20260709 13:20:18.613901 19378 consensus_queue.cc:260] T 1df93779209748b9860f6d893dd3de53 P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:18.338069 19407 raft_consensus.cc:2955] T 19baa7bc367844c288bca8f81439c413 P 1ca2725d290e47d68b1a4b609c9816ac [term 7 LEADER]: Committing config change with OpId 7.7: config changed from index 3 to 7, cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:20:18.675372 19210 leader_election.cc:290] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
W20260709 13:20:18.804857 19380 consensus_meta.cc:287] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 647a343a542d4a139fe106c19095129b: Time spent flushing consensus metadata: real 0.722s	user 0.010s	sys 0.000s
I20260709 13:20:18.805181 19418 raft_consensus.cc:3055] T 8d4ab870a05347868956aa8b841b2981 P 647a343a542d4a139fe106c19095129b [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:20:18.805688 19418 raft_consensus.cc:740] T 8d4ab870a05347868956aa8b841b2981 P 647a343a542d4a139fe106c19095129b [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:20:18.805387 19380 raft_consensus.cc:515] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:18.809103 19315 leader_election.cc:290] T 02838c86531d4f98bf58d1c22b8baf77 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:18.806746 19418 consensus_queue.cc:260] T 8d4ab870a05347868956aa8b841b2981 P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.3, Last appended by leader: 3, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:20:18.815479 19376 consensus_meta.cc:287] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac: Time spent flushing consensus metadata: real 0.906s	user 0.007s	sys 0.007s
W20260709 13:20:18.821050 19316 consensus_meta.cc:287] T 829b4ad8c5a54eababeea1181b960d4b P 1ca2725d290e47d68b1a4b609c9816ac: Time spent flushing consensus metadata: real 0.556s	user 0.008s	sys 0.000s
I20260709 13:20:18.821569 19316 raft_consensus.cc:515] T 829b4ad8c5a54eababeea1181b960d4b P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:20:18.799984 19361 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.896s	user 0.049s	sys 0.035s
I20260709 13:20:18.823870 19404 leader_election.cc:290] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
W20260709 13:20:18.825191 19205 consensus_meta.cc:287] T 23f82224f4564fea93572e8206bad1fe P 647a343a542d4a139fe106c19095129b: Time spent flushing consensus metadata: real 0.507s	user 0.000s	sys 0.010s
W20260709 13:20:18.825623 19345 consensus_meta.cc:287] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045: Time spent flushing consensus metadata: real 1.128s	user 0.010s	sys 0.001s
I20260709 13:20:18.825749 19205 raft_consensus.cc:515] T 23f82224f4564fea93572e8206bad1fe P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:18.826216 19345 raft_consensus.cc:515] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:18.828594 19377 raft_consensus.cc:3060] T 18ad7e3af5504571a69d10c00750a01c P 1ca2725d290e47d68b1a4b609c9816ac [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:20:18.800135 19364 consensus_meta.cc:287] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac: Time spent flushing consensus metadata: real 0.883s	user 0.008s	sys 0.003s
I20260709 13:20:18.835108 19364 raft_consensus.cc:515] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:20:18.836194 19383 consensus_meta.cc:287] T 8cc959fb7f704fba9009c0595467d074 P 1ca2725d290e47d68b1a4b609c9816ac: Time spent flushing consensus metadata: real 0.763s	user 0.008s	sys 0.004s
W20260709 13:20:18.837632 19363 consensus_meta.cc:287] T d2ab0ac8347641e6955e9bce7d90c1a2 P 1ca2725d290e47d68b1a4b609c9816ac: Time spent flushing consensus metadata: real 0.922s	user 0.007s	sys 0.003s
I20260709 13:20:18.839448 19345 leader_election.cc:290] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 7 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:20:18.848506 19213 raft_consensus.cc:515] T 7c2095a5f4ad49408dbc7f69c606120d P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:18.807798 19380 leader_election.cc:290] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 8 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:18.850523 19421 raft_consensus.cc:515] T 829b4ad8c5a54eababeea1181b960d4b P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:20:18.905908 16115 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:20:18.910545 16144 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.774s	user 0.010s	sys 0.003s
W20260709 13:20:18.911211 16144 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.775s	user 0.010s	sys 0.004s
W20260709 13:20:18.912495 16116 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:20:18.914011 16265 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.928s	user 0.005s	sys 0.020s
W20260709 13:20:18.914251 16264 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:20:18.914434 16192 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.927s	user 0.014s	sys 0.009s
W20260709 13:20:18.914988 16192 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.928s	user 0.014s	sys 0.010s
W20260709 13:20:18.916747 16266 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:20:18.917086 16264 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:20:18.919426 16219 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.679s	user 0.000s	sys 0.000s
W20260709 13:20:18.919838 16219 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.691s	user 0.000s	sys 0.000s
I20260709 13:20:18.922148 16146 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40844 (request call id 2083) took 1175 ms. Trace:
I20260709 13:20:18.922761 16146 rpcz_store.cc:276] 0709 13:20:17.746524 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:17.896984 (+150460us) service_pool.cc:224] Handling call
0709 13:20:17.897456 (+   472us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:20:17.897783 (+   327us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:17.897810 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:20:17.940011 (+ 42201us) log.cc:844] Serialized 572 byte log entry
0709 13:20:18.336141 (+396130us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:20:18.336183 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:18.337271 (+  1088us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:20:18.921409 (+584138us) raft_consensus.cc:1685] finished
0709 13:20:18.921468 (+    59us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:18.921873 (+   405us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":395807,"threads_started":1,"wal-append.queue_time_us":878412}
I20260709 13:20:18.924520 16220 raft_consensus.cc:2410] T 8cc959fb7f704fba9009c0595467d074 P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b for term 8 because replica has last-logged OpId of term: 7 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 5.
I20260709 13:20:18.925801 16143 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40844 (request call id 2090) took 1154 ms. Trace:
I20260709 13:20:18.926671 16146 raft_consensus.cc:1275] T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 13:20:18.926285 16143 rpcz_store.cc:276] 0709 13:20:17.770913 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:17.925263 (+154350us) service_pool.cc:224] Handling call
0709 13:20:17.925560 (+   297us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:20:17.925866 (+   306us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:17.925892 (+    26us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:20:17.968234 (+ 42342us) log.cc:844] Serialized 26 byte log entry
0709 13:20:18.340083 (+371849us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:20:18.340142 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:18.340246 (+   104us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:20:18.925238 (+584992us) raft_consensus.cc:1685] finished
0709 13:20:18.925290 (+    52us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:18.925603 (+   313us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1077,"raft.run_cpu_time_us":161,"raft.run_wall_time_us":45574,"thread_start_us":312151,"threads_started":1,"wal-append.queue_time_us":824455}
I20260709 13:20:18.928140 16296 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48674 (request call id 2344) took 1029 ms. Trace:
I20260709 13:20:18.929489 16298 raft_consensus.cc:2393] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 7: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
W20260709 13:20:18.930581 16293 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.932s	user 0.015s	sys 0.022s
W20260709 13:20:18.914557 16265 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.928s	user 0.005s	sys 0.021s
W20260709 13:20:19.002318 16293 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.004s	user 0.015s	sys 0.022s
I20260709 13:20:19.012545 16292 raft_consensus.cc:2468] T 8158bbd17b994d2eb96f560bd0f1b88d P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 8.
I20260709 13:20:19.013779 16265 leader_election.cc:304] T c4f65cdf00b748bb963ee6e81ef43b79 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:20:18.941519 16296 rpcz_store.cc:276] 0709 13:20:17.898803 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:17.899038 (+   235us) service_pool.cc:224] Handling call
0709 13:20:17.899331 (+   293us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:20:17.905211 (+  5880us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:20:17.905246 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:20:17.905682 (+   436us) log.cc:844] Serialized 26 byte log entry
0709 13:20:18.240730 (+335048us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:20:18.240777 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:18.240845 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:20:18.791868 (+551023us) spinlock_profiling.cc:243] Waited 20.6 ms on lock 0x7b4800130158. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b5a88aa1 00007f69b5a7fb37 00007f69b5a7f9ca 00007f69b5b7726f 00007f69b5b76f37 00007f69b5b8b55c 00007f69b5b8b508 00007f69b5b8b488 00007f69b5b8b443 00007f69b5b8a6ec 00007f69b4ab4a96
0709 13:20:18.927447 (+135579us) raft_consensus.cc:1685] finished
0709 13:20:18.927493 (+    46us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:18.927892 (+   399us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":6516,"raft.run_cpu_time_us":4347,"raft.run_wall_time_us":894570,"spinlock_wait_cycles":47425792,"thread_start_us":334116,"threads_started":1,"wal-append.queue_time_us":910786}
W20260709 13:20:19.038550 16299 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.990s	user 0.005s	sys 0.004s
W20260709 13:20:19.039108 16299 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.038s	user 0.005s	sys 0.004s
I20260709 13:20:19.040194 16296 raft_consensus.cc:1240] T 84f2da98864b4425bf7c0055951fdd4b P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 5. Current term is 6. Ops: [5.3-5.3]
W20260709 13:20:19.055073 16117 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:19.060261 19316 leader_election.cc:290] T 829b4ad8c5a54eababeea1181b960d4b P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 8 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:19.062026 19378 raft_consensus.cc:3060] T 1df93779209748b9860f6d893dd3de53 P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:20:19.072224 16297 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40474 (request call id 474) took 1088 ms. Trace:
I20260709 13:20:19.072852 16297 rpcz_store.cc:276] 0709 13:20:17.983250 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:18.428616 (+445366us) service_pool.cc:224] Handling call
0709 13:20:18.542373 (+113757us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:18.559451 (+ 17078us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:19.071986 (+512535us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:20:19.073508 16117 leader_election.cc:304] T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860
I20260709 13:20:19.074782 16297 raft_consensus.cc:1240] T d2ab0ac8347641e6955e9bce7d90c1a2 P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 7. Current term is 8. Ops: [7.8-7.8]
I20260709 13:20:19.078704 16147 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40830 (request call id 1252) took 1579 ms. Trace:
W20260709 13:20:19.080725 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 506958us
I20260709 13:20:19.082320 19345 raft_consensus.cc:493] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
W20260709 13:20:19.084205 16192 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:19.088740 16224 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57604 (request call id 2253) took 1240 ms. Trace:
I20260709 13:20:19.088920 19205 leader_election.cc:290] T 23f82224f4564fea93572e8206bad1fe P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 7 election: Requested vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:19.040460 19361 raft_consensus.cc:2749] T c4f65cdf00b748bb963ee6e81ef43b79 P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
W20260709 13:20:19.092002 16791 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:19.096691 16147 rpcz_store.cc:276] 0709 13:20:17.499102 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:17.501250 (+  2148us) service_pool.cc:224] Handling call
0709 13:20:19.078172 (+1576922us) spinlock_profiling.cc:243] Waited 296 ms on lock 0x7b54000e9f54. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:20:19.078483 (+   311us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":681730816}
I20260709 13:20:19.099395 16219 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57604 (request call id 2247) took 1793 ms. Trace:
W20260709 13:20:19.111145 16189 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:19.104336 16219 rpcz_store.cc:276] 0709 13:20:17.293532 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:17.470425 (+176893us) service_pool.cc:224] Handling call
0709 13:20:17.470745 (+   320us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:17.470965 (+   220us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:20:17.794664 (+323699us) log.cc:844] Serialized 14 byte log entry
0709 13:20:18.921045 (+1126381us) log.cc:844] Serialized 14 byte log entry
0709 13:20:19.086413 (+165368us) log.cc:844] Serialized 14 byte log entry
0709 13:20:19.086669 (+   256us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:20:19.086750 (+    81us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:20:19.086838 (+    88us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:19.086956 (+   118us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:19.087131 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":256499,"prepare.queue_time_us":510239,"prepare.run_cpu_time_us":102,"prepare.run_wall_time_us":11526,"raft.queue_time_us":269842,"raft.run_cpu_time_us":449,"raft.run_wall_time_us":30884,"thread_start_us":1335143,"threads_started":3,"wal-append.queue_time_us":409023}
I20260709 13:20:19.113368 16192 leader_election.cc:304] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 
W20260709 13:20:19.115466 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 102046us
I20260709 13:20:19.115731 16297 raft_consensus.cc:1240] T 18ad7e3af5504571a69d10c00750a01c P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 6. Current term is 7. Ops: []
I20260709 13:20:19.104758 19364 leader_election.cc:290] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 6 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:19.117871 16265 leader_election.cc:304] T 18ad7e3af5504571a69d10c00750a01c P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:20:19.098599 16224 rpcz_store.cc:276] 0709 13:20:17.847933 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:17.921211 (+ 73278us) service_pool.cc:224] Handling call
0709 13:20:17.939910 (+ 18699us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:20:17.977059 (+ 37149us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:17.977092 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:20:17.977557 (+   465us) log.cc:844] Serialized 26 byte log entry
0709 13:20:18.405052 (+427495us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:20:18.405109 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:18.405207 (+    98us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:20:19.083585 (+678378us) raft_consensus.cc:1685] finished
0709 13:20:19.088220 (+  4635us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:19.088467 (+   247us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":36679,"raft.queue_time_us":37685,"raft.run_cpu_time_us":119,"raft.run_wall_time_us":12631,"thread_start_us":427166,"threads_started":1,"wal-append.queue_time_us":655712}
I20260709 13:20:19.120329 16816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87147731a68400e8cce9b879daa7c5d" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:20:19.121726 16816 raft_consensus.cc:2393] T b87147731a68400e8cce9b879daa7c5d P cb82cc7bd6db46d980c75237a1402860 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 9: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
W20260709 13:20:19.120369 16114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:19.123667 19361 raft_consensus.cc:2749] T 18ad7e3af5504571a69d10c00750a01c P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:20:19.123577 19425 raft_consensus.cc:493] T 2794af5b99984efebdee564035841cb0 P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:20:19.125030 19425 raft_consensus.cc:515] T 2794af5b99984efebdee564035841cb0 P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:19.126204 19450 raft_consensus.cc:493] T fffab26427e24d398e2695aad40ca134 P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:19.124150 19380 raft_consensus.cc:493] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:20:19.126895 19450 raft_consensus.cc:515] T fffab26427e24d398e2695aad40ca134 P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:19.127592 19380 raft_consensus.cc:515] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:19.129485 19450 leader_election.cc:290] T fffab26427e24d398e2695aad40ca134 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:19.129673 16816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66ae176961b74813a58bc15c9c4c6f2a" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:19.130115 19450 raft_consensus.cc:493] T 9e65676b21b748728ad39668cfe722ec P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:19.130286 19380 leader_election.cc:290] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:19.125171 19345 raft_consensus.cc:515] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:19.145696 16825 raft_consensus.cc:1275] T 8c11074579b046d99e585c2818e18a2c P cb82cc7bd6db46d980c75237a1402860 [term 7 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 7 index: 9. (index mismatch)
I20260709 13:20:19.130868 16816 raft_consensus.cc:2468] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 9.
I20260709 13:20:19.152133 16297 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48684 (request call id 2236) took 1027 ms. Trace:
I20260709 13:20:19.154472 16821 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49896 (request call id 985) took 1001 ms. Trace:
I20260709 13:20:19.141628 16300 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40474 (request call id 473) took 2406 ms. Trace:
I20260709 13:20:19.134521 16824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada5a004ba7c46819051e3ccec71ffdb" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:19.185012 16297 rpcz_store.cc:276] 0709 13:20:18.124292 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:19.118350 (+994058us) service_pool.cc:224] Handling call
0709 13:20:19.118871 (+   521us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:19.151665 (+ 32794us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:19.151698 (+    33us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:20:19.151799 (+   101us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:20:19.151832 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:19.151897 (+    65us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:19.152034 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:20:19.188045 16192 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 74986.4us
W20260709 13:20:19.157486 19455 log.cc:927] Time spent T b6f07842d3aa47858df5665874c86f3a P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 13:20:19.133124 19380 raft_consensus.cc:493] T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:20:19.156155 16823 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49906 (request call id 697) took 1227 ms (client timeout 1576 ms). Trace:
I20260709 13:20:19.135304 16820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57a744f5aaf944678a63469cbf514015" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:19.140940 19425 leader_election.cc:290] T 2794af5b99984efebdee564035841cb0 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:19.139670 16818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a378d7f796f94615b3b347119a9880fd" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:19.132407 19414 leader_election.cc:290] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:20:19.141010 19068 leader_election.cc:290] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 8 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:19.133306 19418 raft_consensus.cc:3060] T 8d4ab870a05347868956aa8b841b2981 P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:20:19.186160 16821 rpcz_store.cc:276] 0709 13:20:18.152771 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:18.161247 (+  8476us) service_pool.cc:224] Handling call
0709 13:20:19.154068 (+992821us) spinlock_profiling.cc:243] Waited 6.68 ms on lock 0x7b54007553d4. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:20:19.154261 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":15358336}
I20260709 13:20:19.141830 19088 leader_election.cc:290] T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:19.131067 19450 raft_consensus.cc:515] T 9e65676b21b748728ad39668cfe722ec P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:19.150636 19404 raft_consensus.cc:2749] T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
W20260709 13:20:19.203245 16117 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 129958us
W20260709 13:20:19.192140 16823 rpcz_store.cc:269] 0709 13:20:17.928651 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:17.930661 (+  2010us) service_pool.cc:224] Handling call
0709 13:20:19.155691 (+1225030us) spinlock_profiling.cc:243] Waited 260 ms on lock 0x7b54005c35d4. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:20:19.155955 (+   264us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":597678464}
I20260709 13:20:19.205472 16151 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:40290 (request call id 18) took 1105 ms. Trace:
I20260709 13:20:19.212891 16151 rpcz_store.cc:276] 0709 13:20:18.097574 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:18.155831 (+ 58257us) service_pool.cc:224] Handling call
0709 13:20:19.202910 (+1047079us) spinlock_profiling.cc:243] Waited 179 ms on lock 0x7b540028f354. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b59853 00007f69b5b2760a 00007f69af0ad7d4 00007f69af0ad762 00007f69af0ad691 00007f69af0ad60c 00007f69af0ac8c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:20:19.203176 (+   266us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":412517760}
W20260709 13:20:19.219851 16190 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.987s	user 0.017s	sys 0.022s
W20260709 13:20:19.220078 16191 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.895s	user 0.019s	sys 0.021s
W20260709 13:20:19.220329 16190 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.996s	user 0.017s	sys 0.022s
W20260709 13:20:19.220568 16191 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.903s	user 0.019s	sys 0.021s
W20260709 13:20:19.220482 16789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 69732.1us
W20260709 13:20:19.221704 16191 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:19.223636 19357 raft_consensus.cc:493] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:19.225277 19316 consensus_queue.cc:1048] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:20:19.218838 16825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26354361e9b044cea1c2b86c4882fe92" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:19.239094 16820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3884f864a96d428785895e5463e5728f" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 7 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:19.239435 16818 raft_consensus.cc:2393] T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 7: Already voted for candidate cb82cc7bd6db46d980c75237a1402860 in this term.
I20260709 13:20:19.240629 16824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8638ee363273426db1393dd81e5c7b72" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:19.187119 16300 rpcz_store.cc:276] 0709 13:20:16.735503 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:17.244048 (+508545us) service_pool.cc:224] Handling call
0709 13:20:17.284714 (+ 40666us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:20:17.480478 (+195764us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:17.480515 (+    37us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:20:17.481381 (+   866us) log.cc:844] Serialized 52 byte log entry
0709 13:20:17.961716 (+480335us) spinlock_profiling.cc:243] Waited 2.24 ms on lock 0x7b180012bf80. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b693ade1 00007f69b69475b1 00007f69ada54e07 00007f69b68afab9 00007f69ada7169b 00007f69ada6f93f 00007f69ada71c02 00007f69b06f920b 00007f69b06fed23 00007f69b06ff0d1 00007f69b0b1aa0f
0709 13:20:18.325452 (+363736us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:20:18.325506 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:19.140889 (+815383us) spinlock_profiling.cc:243] Waited 193 ms on lock 0x7b54000cf654. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:20:19.140970 (+    81us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:20:19.141067 (+    97us) raft_consensus.cc:1685] finished
0709 13:20:19.141086 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:19.141421 (+   335us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":880,"raft.run_cpu_time_us":155,"raft.run_wall_time_us":13915,"spinlock_wait_cycles":450090112,"thread_start_us":363518,"threads_started":1,"wal-append.queue_time_us":1340274}
I20260709 13:20:19.199136 19380 raft_consensus.cc:515] T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:20:19.263455 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 145843us
I20260709 13:20:19.224787 19357 raft_consensus.cc:515] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:20:19.265724 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 1.13115e+06us
W20260709 13:20:19.278646 16817 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.816s	user 0.012s	sys 0.028s
W20260709 13:20:19.279213 16817 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.852s	user 0.013s	sys 0.028s
I20260709 13:20:19.280898 16825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:20:19.283421 19414 raft_consensus.cc:493] T 8a878999c0024d2abe517484180290fc P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:19.283967 16816 raft_consensus.cc:1275] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P cb82cc7bd6db46d980c75237a1402860 [term 7 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 13:20:19.284222 19414 raft_consensus.cc:515] T 8a878999c0024d2abe517484180290fc P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:19.286696 16117 leader_election.cc:304] T 26354361e9b044cea1c2b86c4882fe92 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860
I20260709 13:20:19.267810 19088 raft_consensus.cc:493] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:20:19.287186 16144 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40844 (request call id 2102) took 1272 ms. Trace:
I20260709 13:20:19.287312 19345 leader_election.cc:290] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:19.287918 19088 raft_consensus.cc:515] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } }
I20260709 13:20:19.287672 16144 rpcz_store.cc:276] 0709 13:20:18.014641 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:18.102886 (+ 88245us) service_pool.cc:224] Handling call
0709 13:20:18.135826 (+ 32940us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:18.136016 (+   190us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:20:18.912038 (+776022us) log.cc:844] Serialized 14 byte log entry
0709 13:20:19.286672 (+374634us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:20:19.286764 (+    92us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:20:19.286858 (+    94us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:19.286946 (+    88us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:19.287103 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":691386,"prepare.run_cpu_time_us":44,"prepare.run_wall_time_us":44,"thread_start_us":1120734,"threads_started":2}
I20260709 13:20:19.289474 19380 leader_election.cc:290] T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:19.289857 19404 raft_consensus.cc:2749] T 26354361e9b044cea1c2b86c4882fe92 P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:20:19.283450 16824 raft_consensus.cc:2468] T 8638ee363273426db1393dd81e5c7b72 P cb82cc7bd6db46d980c75237a1402860 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 7.
I20260709 13:20:19.289892 19213 leader_election.cc:290] T 7c2095a5f4ad49408dbc7f69c606120d P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 7 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:19.290555 19088 leader_election.cc:290] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:20:19.283457 18901 consensus_queue.cc:1048] T ecc1e363230043fe9fef390f0b979f20 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.008s
I20260709 13:20:19.292479 16151 raft_consensus.cc:1240] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 7. Current term is 8. Ops: []
I20260709 13:20:19.291360 16266 leader_election.cc:304] T b87147731a68400e8cce9b879daa7c5d P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
I20260709 13:20:19.287060 19414 leader_election.cc:290] T 8a878999c0024d2abe517484180290fc P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
W20260709 13:20:19.302764 16150 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.024s	user 0.004s	sys 0.014s
W20260709 13:20:19.303508 16150 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.075s	user 0.004s	sys 0.015s
W20260709 13:20:19.295058 16222 consensus_meta.cc:287] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac: Time spent flushing consensus metadata: real 1.056s	user 0.010s	sys 0.007s
I20260709 13:20:19.296097 19068 raft_consensus.cc:2749] T b87147731a68400e8cce9b879daa7c5d P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:20:19.297616 16266 leader_election.cc:304] T 66ae176961b74813a58bc15c9c4c6f2a P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac
I20260709 13:20:19.298393 19345 consensus_queue.cc:1048] T 8c11074579b046d99e585c2818e18a2c P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:20:19.271081 19364 raft_consensus.cc:2804] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:20:19.299041 16190 leader_election.cc:304] T 7c2095a5f4ad49408dbc7f69c606120d P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:20:19.320730 16300 raft_consensus.cc:1240] T 1df93779209748b9860f6d893dd3de53 P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 7. Current term is 8. Ops: [4.4-7.5]
I20260709 13:20:19.312964 19364 raft_consensus.cc:493] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:19.325069 19425 raft_consensus.cc:2804] T 66ae176961b74813a58bc15c9c4c6f2a P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:20:19.326503 16300 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48684 (request call id 2239) took 1201 ms. Trace:
I20260709 13:20:19.328750 16300 rpcz_store.cc:276] 0709 13:20:18.125128 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:19.275415 (+1150287us) service_pool.cc:224] Handling call
0709 13:20:19.304992 (+ 29577us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:20:19.326193 (+ 21201us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:19.326449 (+   256us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:20:19.317054 16222 raft_consensus.cc:2468] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 8.
W20260709 13:20:19.325991 19233 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.517s	user 0.009s	sys 0.013s
W20260709 13:20:19.338136 16264 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:19.328516 19364 raft_consensus.cc:3060] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:20:19.334822 16265 leader_election.cc:304] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860
W20260709 13:20:19.343523 16266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 46022.7us
I20260709 13:20:19.330457 19425 raft_consensus.cc:493] T 66ae176961b74813a58bc15c9c4c6f2a P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:20:19.344897 16222 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57604 (request call id 2266) took 1119 ms. Trace:
W20260709 13:20:19.346187 16117 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 59718.1us
I20260709 13:20:19.345548 16222 rpcz_store.cc:276] 0709 13:20:18.221583 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:18.221853 (+   270us) service_pool.cc:224] Handling call
0709 13:20:19.338463 (+1116610us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:20:19.346160 19450 leader_election.cc:290] T 9e65676b21b748728ad39668cfe722ec P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:19.368994 16266 leader_election.cc:304] T a378d7f796f94615b3b347119a9880fd P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860
I20260709 13:20:19.372670 16266 leader_election.cc:304] T 26354361e9b044cea1c2b86c4882fe92 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
I20260709 13:20:19.370620 16824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e9b2bf7011f4ae688a79d7bb5a447b4" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
W20260709 13:20:19.348172 19233 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.652s	user 0.009s	sys 0.013s
I20260709 13:20:19.375392 16144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02838c86531d4f98bf58d1c22b8baf77" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 9 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:19.376953 19412 consensus_queue.cc:1048] T 829b4ad8c5a54eababeea1181b960d4b P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.066s
I20260709 13:20:19.381533 19357 leader_election.cc:290] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:19.381992 19450 raft_consensus.cc:493] T af57996a8be84d398f0f023e8d08ef77 P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:19.343416 19289 consensus_queue.cc:1048] T 57a744f5aaf944678a63469cbf514015 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.106s
I20260709 13:20:19.382897 19450 raft_consensus.cc:515] T af57996a8be84d398f0f023e8d08ef77 P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:19.346846 19425 raft_consensus.cc:3060] T 66ae176961b74813a58bc15c9c4c6f2a P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:20:19.385541 19414 raft_consensus.cc:2749] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:20:19.389122 16146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b7d1ab519044eda2d5e04a07242f25" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:20:19.390079 19450 leader_election.cc:290] T af57996a8be84d398f0f023e8d08ef77 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
W20260709 13:20:19.428383 16191 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:19.396837 16299 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48674 (request call id 2345) took 1458 ms. Trace:
I20260709 13:20:19.348201 19316 raft_consensus.cc:3060] T 7c2095a5f4ad49408dbc7f69c606120d P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:20:19.434608 16146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "829b4ad8c5a54eababeea1181b960d4b" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:20:19.434918 19414 raft_consensus.cc:2749] T 26354361e9b044cea1c2b86c4882fe92 P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:20:19.435946 16146 raft_consensus.cc:2393] T 829b4ad8c5a54eababeea1181b960d4b P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 8: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
W20260709 13:20:19.431627 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 206789us
I20260709 13:20:19.436846 16790 leader_election.cc:304] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:20:19.438720 19345 raft_consensus.cc:3060] T a378d7f796f94615b3b347119a9880fd P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:20:19.440704 19412 raft_consensus.cc:2804] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860 [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 13:20:19.440897 19473 log.cc:927] Time spent T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 0.110s	user 0.001s	sys 0.000s
I20260709 13:20:19.441200 19412 raft_consensus.cc:493] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:19.441629 19412 raft_consensus.cc:3060] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:20:19.442602 18944 consensus_meta.cc:287] T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b: Time spent flushing consensus metadata: real 1.353s	user 0.009s	sys 0.008s
I20260709 13:20:19.443082 18944 raft_consensus.cc:515] T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:19.444993 16144 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4e9b2bf7011f4ae688a79d7bb5a447b4"
dest_uuid: "647a343a542d4a139fe106c19095129b"
mode: GRACEFUL
new_leader_uuid: "90f3da57796b488c9087a2f523adf045"
 from {username='slave'} at 127.0.0.1:40290
I20260709 13:20:19.445864 16144 raft_consensus.cc:606] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Received request to transfer leadership to TS 90f3da57796b488c9087a2f523adf045
I20260709 13:20:19.446314 16144 raft_consensus.cc:614] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:20:19.447656 16148 raft_consensus.cc:1240] T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 6. Current term is 7. Ops: []
W20260709 13:20:19.451635 16117 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 55281us
I20260709 13:20:19.452167 19465 consensus_queue.cc:1048] T 9e65676b21b748728ad39668cfe722ec P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:20:19.452764 16117 leader_election.cc:304] T 746aabf856ab401c9ef5e859371e7b67 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac
I20260709 13:20:19.454602 16117 leader_election.cc:304] T 2794af5b99984efebdee564035841cb0 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860
I20260709 13:20:19.455449 19404 raft_consensus.cc:2804] T 746aabf856ab401c9ef5e859371e7b67 P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:20:19.455922 19404 raft_consensus.cc:493] T 746aabf856ab401c9ef5e859371e7b67 P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Starting leader election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:20:19.456355 19404 raft_consensus.cc:3060] T 746aabf856ab401c9ef5e859371e7b67 P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:20:19.457101 19412 raft_consensus.cc:515] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:19.456480 16295 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48684 (request call id 2237) took 1308 ms. Trace:
I20260709 13:20:19.458343 19289 consensus_queue.cc:1059] T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:20:19.462733 19289 raft_consensus.cc:3055] T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:20:19.459587 16790 leader_election.cc:304] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:20:19.463137 19289 raft_consensus.cc:740] T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
W20260709 13:20:19.464198 19197 consensus_meta.cc:287] T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b: Time spent flushing consensus metadata: real 1.379s	user 0.009s	sys 0.016s
I20260709 13:20:19.464682 19314 raft_consensus.cc:2804] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:20:19.465171 19314 raft_consensus.cc:493] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:19.464612 19289 consensus_queue.cc:260] T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:19.465502 19314 raft_consensus.cc:3060] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:20:19.466601 16142 raft_consensus.cc:2468] T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 8.
I20260709 13:20:19.467360 18599 raft_consensus.cc:493] T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:19.467820 18599 raft_consensus.cc:515] T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:19.469447 19289 raft_consensus.cc:3060] T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:20:19.475863 16790 leader_election.cc:304] T ada5a004ba7c46819051e3ccec71ffdb P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:20:19.478152 19412 leader_election.cc:290] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 8 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:20:19.483727 18944 leader_election.cc:290] T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 7 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
W20260709 13:20:19.480254 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 157503us
I20260709 13:20:19.482764 16790 leader_election.cc:304] T 8638ee363273426db1393dd81e5c7b72 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:20:19.544003 16265 leader_election.cc:304] T 8158bbd17b994d2eb96f560bd0f1b88d P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
W20260709 13:20:19.544277 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 61798.3us
I20260709 13:20:19.544982 16293 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48674 (request call id 2334) took 1844 ms. Trace:
I20260709 13:20:19.546083 16265 leader_election.cc:304] T ada5a004ba7c46819051e3ccec71ffdb P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860
I20260709 13:20:19.474468 19314 raft_consensus.cc:515] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:19.547227 19068 raft_consensus.cc:2764] T 8158bbd17b994d2eb96f560bd0f1b88d P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Leader pre-election decision vote started in defunct term 7: won
I20260709 13:20:19.547976 19068 raft_consensus.cc:2749] T ada5a004ba7c46819051e3ccec71ffdb P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:20:19.548296 16298 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48674 (request call id 2352) took 1446 ms (client timeout 1849 ms). Trace:
I20260709 13:20:19.548781 19314 leader_election.cc:290] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 10 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
W20260709 13:20:19.548705 16298 rpcz_store.cc:269] 0709 13:20:18.102172 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:18.316489 (+214317us) service_pool.cc:224] Handling call
0709 13:20:19.548053 (+1231564us) spinlock_profiling.cc:243] Waited 579 ms on lock 0x7b5400240254. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:20:19.548219 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1330572160}
I20260709 13:20:19.549459 16265 leader_election.cc:304] T 8d4ab870a05347868956aa8b841b2981 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
W20260709 13:20:19.484906 16192 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 142844us
I20260709 13:20:19.549475 19314 consensus_queue.cc:1059] T 66f75124215f4ca6bef831e174a792d8 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:20:19.550639 16298 raft_consensus.cc:1275] T 8638ee363273426db1393dd81e5c7b72 P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:20:19.551322 19314 raft_consensus.cc:3055] T 66f75124215f4ca6bef831e174a792d8 P cb82cc7bd6db46d980c75237a1402860 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:20:19.551775 19314 raft_consensus.cc:740] T 66f75124215f4ca6bef831e174a792d8 P cb82cc7bd6db46d980c75237a1402860 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:20:19.553015 19314 consensus_queue.cc:260] T 66f75124215f4ca6bef831e174a792d8 P cb82cc7bd6db46d980c75237a1402860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:19.554054 16300 raft_consensus.cc:1240] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 6. Current term is 7. Ops: []
I20260709 13:20:19.554512 19314 raft_consensus.cc:3060] T 66f75124215f4ca6bef831e174a792d8 P cb82cc7bd6db46d980c75237a1402860 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:20:19.487632 16266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 115172us
I20260709 13:20:19.555387 19068 raft_consensus.cc:2749] T 8d4ab870a05347868956aa8b841b2981 P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:20:19.492085 16791 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:19.556218 16300 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48684 (request call id 2240) took 1430 ms. Trace:
I20260709 13:20:19.556291 19068 consensus_queue.cc:1048] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:20:19.556639 16300 rpcz_store.cc:276] 0709 13:20:18.125380 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:19.336983 (+1211603us) service_pool.cc:224] Handling call
0709 13:20:19.553829 (+216846us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:19.554502 (+   673us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:19.555994 (+  1492us) spinlock_profiling.cc:243] Waited 157 ms on lock 0x7b5400240250. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5e8ad 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:20:19.556117 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":361622784}
I20260709 13:20:19.558221 16816 raft_consensus.cc:1240] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [term 10 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 9. Current term is 10. Ops: [9.9-9.9]
I20260709 13:20:19.558619 16823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "829b4ad8c5a54eababeea1181b960d4b" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:20:19.559110 16265 leader_election.cc:304] T 02838c86531d4f98bf58d1c22b8baf77 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:20:19.559904 16824 raft_consensus.cc:1240] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860 [term 8 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 7. Current term is 8. Ops: [7.8-7.8]
I20260709 13:20:19.560029 16816 raft_consensus.cc:2955] T 19baa7bc367844c288bca8f81439c413 P cb82cc7bd6db46d980c75237a1402860 [term 7 FOLLOWER]: Committing config change with OpId 7.7: config changed from index 3 to 7, cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:20:19.561543 16265 leader_election.cc:304] T cdbd78acb4cf46298e0de8ba0d130e19 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860
I20260709 13:20:19.562143 16821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b7d1ab519044eda2d5e04a07242f25" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:20:19.562783 16821 raft_consensus.cc:3060] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:20:19.493855 19080 consensus_meta.cc:287] T 57a744f5aaf944678a63469cbf514015 P 90f3da57796b488c9087a2f523adf045: Time spent flushing consensus metadata: real 1.253s	user 0.003s	sys 0.013s
W20260709 13:20:19.503340 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 204440us
I20260709 13:20:19.570677 16298 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48684 (request call id 2242) took 1444 ms. Trace:
I20260709 13:20:19.479247 19213 consensus_queue.cc:1048] T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:20:19.516048 16151 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "68a6838d82ef4cac83507f906139d90f"
dest_uuid: "647a343a542d4a139fe106c19095129b"
mode: GRACEFUL
new_leader_uuid: "90f3da57796b488c9087a2f523adf045"
 from {username='slave'} at 127.0.0.1:40290
I20260709 13:20:19.518072 19142 consensus_queue.cc:260] T 5a96373bb0e14dce8a266b2fc6c58507 P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 6.3, Last appended by leader: 3, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:20:19.520519 16264 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:19.521744 19416 raft_consensus.cc:493] T b6f07842d3aa47858df5665874c86f3a P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:19.429338 16299 rpcz_store.cc:276] 0709 13:20:17.934470 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:17.962819 (+ 28349us) service_pool.cc:224] Handling call
0709 13:20:18.000188 (+ 37369us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:20:18.000487 (+   299us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:18.000514 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:20:18.000937 (+   423us) log.cc:844] Serialized 26 byte log entry
0709 13:20:19.039835 (+1038898us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:20:19.039931 (+    96us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:19.040026 (+    95us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:20:19.385933 (+345907us) raft_consensus.cc:1685] finished
0709 13:20:19.385951 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:19.392965 (+  7014us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":61273,"raft.run_cpu_time_us":85,"raft.run_wall_time_us":78,"thread_start_us":1038301,"threads_started":1,"wal-append.queue_time_us":1311509}
I20260709 13:20:19.524580 16297 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48684 (request call id 2238) took 1399 ms. Trace:
I20260709 13:20:19.524868 16145 raft_consensus.cc:1275] T ecc1e363230043fe9fef390f0b979f20 P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:20:19.527266 16296 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48684 (request call id 2231) took 1409 ms. Trace:
I20260709 13:20:19.538275 16820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74ca8140f35c4ae29e8a664f6bdfc829" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:19.546842 19357 raft_consensus.cc:2749] T 8638ee363273426db1393dd81e5c7b72 P cb82cc7bd6db46d980c75237a1402860 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:20:19.556352 16266 leader_election.cc:304] T 8638ee363273426db1393dd81e5c7b72 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 
W20260709 13:20:19.511669 16191 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:19.580920 16298 rpcz_store.cc:276] 0709 13:20:18.125917 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:19.549846 (+1423929us) service_pool.cc:224] Handling call
0709 13:20:19.550245 (+   399us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:19.551087 (+   842us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:19.570600 (+ 19513us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:20:19.608693 16821 raft_consensus.cc:2468] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 6.
I20260709 13:20:19.612901 16144 raft_consensus.cc:1275] T 8d4ab870a05347868956aa8b841b2981 P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 7 index: 3. Preceding OpId from leader: term: 8 index: 4. (index mismatch)
I20260709 13:20:19.475262 16295 rpcz_store.cc:276] 0709 13:20:18.124577 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:19.136590 (+1012013us) service_pool.cc:224] Handling call
0709 13:20:19.137187 (+   597us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:19.137723 (+   536us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:20:19.317963 (+180240us) log.cc:844] Serialized 14 byte log entry
0709 13:20:19.429487 (+111524us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:20:19.429820 (+   333us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:20:19.432605 (+  2785us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:19.432779 (+   174us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:19.432980 (+   201us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":16937,"prepare.run_cpu_time_us":56,"prepare.run_wall_time_us":71,"raft.queue_time_us":4188,"raft.run_cpu_time_us":134,"raft.run_wall_time_us":3096,"thread_start_us":258926,"threads_started":2}
I20260709 13:20:19.602371 16297 rpcz_store.cc:276] 0709 13:20:18.124855 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:19.219145 (+1094290us) service_pool.cc:224] Handling call
0709 13:20:19.283727 (+ 64582us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:20:19.284463 (+   736us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:19.284509 (+    46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:20:19.285176 (+   667us) log.cc:844] Serialized 26 byte log entry
0709 13:20:19.432917 (+147741us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:20:19.432948 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:19.433012 (+    64us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:20:19.524356 (+ 91344us) raft_consensus.cc:1685] finished
0709 13:20:19.524380 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:19.524523 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":3744,"raft.run_cpu_time_us":138,"raft.run_wall_time_us":138,"thread_start_us":147249,"threads_started":1,"wal-append.queue_time_us":222349}
I20260709 13:20:19.609110 16145 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40844 (request call id 2100) took 1726 ms. Trace:
I20260709 13:20:19.604012 16296 rpcz_store.cc:276] 0709 13:20:18.118069 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:19.055546 (+937477us) service_pool.cc:224] Handling call
0709 13:20:19.056338 (+   792us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:20:19.095380 (+ 39042us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:20:19.107477 (+ 12097us) log.cc:844] Serialized 14 byte log entry
0709 13:20:19.337474 (+229997us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:20:19.423423 (+ 85949us) log.cc:844] Serialized 26 byte log entry
0709 13:20:19.423581 (+   158us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:20:19.423617 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:19.423700 (+    83us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:20:19.527040 (+103340us) raft_consensus.cc:1685] finished
0709 13:20:19.527059 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:19.527207 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":15116,"prepare.queue_time_us":4730,"prepare.run_cpu_time_us":55,"prepare.run_wall_time_us":55,"raft.queue_time_us":17692,"raft.run_cpu_time_us":120,"raft.run_wall_time_us":14474,"spinlock_wait_cycles":100130688,"thread_start_us":224577,"threads_started":1,"wal-append.queue_time_us":310818}
I20260709 13:20:19.612336 16151 raft_consensus.cc:606] T 68a6838d82ef4cac83507f906139d90f P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Received request to transfer leadership to TS 90f3da57796b488c9087a2f523adf045
I20260709 13:20:19.545425 16293 rpcz_store.cc:276] 0709 13:20:17.700768 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:17.722471 (+ 21703us) service_pool.cc:224] Handling call
0709 13:20:17.722809 (+   338us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:20:17.877674 (+154865us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:20:17.896480 (+ 18806us) log.cc:844] Serialized 14 byte log entry
0709 13:20:19.038749 (+1142269us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:20:19.040064 (+  1315us) log.cc:844] Serialized 52 byte log entry
0709 13:20:19.040284 (+   220us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:20:19.040649 (+   365us) log.cc:844] Serialized 14 byte log entry
0709 13:20:19.057027 (+ 16378us) log.cc:844] Serialized 14 byte log entry
0709 13:20:19.084634 (+ 27607us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:19.149962 (+ 65328us) spinlock_profiling.cc:243] Waited 5.11 ms on lock 0x7b4800130158. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b5a88aa1 00007f69b5a7fb37 00007f69b5a7f9ca 00007f69b5b7726f 00007f69b5b76f37 00007f69b5b8b55c 00007f69b5b8b508 00007f69b5b8b488 00007f69b5b8b443 00007f69b5b8a6ec 00007f69b4ab4a96
0709 13:20:19.540061 (+390099us) spinlock_profiling.cc:243] Waited 15.3 ms on lock 0x7b18003b0920. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b693ade1 00007f69b69475b1 00007f69ada72522 00007f69ada747c1 00007f69ada74779 00007f69ada74709 00007f69ada746d1 00007f69ada739cd 000000000038f354 000000000038f199 00007f69ada5583a
0709 13:20:19.544620 (+  4559us) spinlock_profiling.cc:243] Waited 154 ms on lock 0x7b5400178c54. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:20:19.544667 (+    47us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:20:19.544716 (+    49us) raft_consensus.cc:1685] finished
0709 13:20:19.544728 (+    12us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:19.544900 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":27320,"prepare.queue_time_us":77746,"prepare.run_cpu_time_us":114,"prepare.run_wall_time_us":116,"raft.queue_time_us":69052,"raft.run_cpu_time_us":3745,"raft.run_wall_time_us":101897,"spinlock_wait_cycles":402196992,"thread_start_us":1022957,"threads_started":1,"wal-append.queue_time_us":1354496}
I20260709 13:20:19.617667 16145 rpcz_store.cc:276] 0709 13:20:17.877126 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:19.523807 (+1646681us) service_pool.cc:224] Handling call
0709 13:20:19.524331 (+   524us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:19.603578 (+ 79247us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:19.603803 (+   225us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:20:19.625576 16790 leader_election.cc:304] T 8c11074579b046d99e585c2818e18a2c P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:20:19.626413 16151 raft_consensus.cc:614] T 68a6838d82ef4cac83507f906139d90f P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 13:20:19.619858 16266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 63631.6us
W20260709 13:20:19.632908 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 85737.2us
I20260709 13:20:19.610863 19412 raft_consensus.cc:493] T 3a7ca61592e848ea9b0fb81a2ce33cdc P cb82cc7bd6db46d980c75237a1402860 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:20:19.637331 16820 raft_consensus.cc:2393] T 74ca8140f35c4ae29e8a664f6bdfc829 P cb82cc7bd6db46d980c75237a1402860 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 8: Already voted for candidate cb82cc7bd6db46d980c75237a1402860 in this term.
I20260709 13:20:19.640192 16298 raft_consensus.cc:1275] T ecc1e363230043fe9fef390f0b979f20 P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:20:19.640405 16817 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49896 (request call id 977) took 1696 ms. Trace:
I20260709 13:20:19.640962 16298 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48684 (request call id 2245) took 1429 ms. Trace:
I20260709 13:20:19.641371 16298 rpcz_store.cc:276] 0709 13:20:18.211063 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:19.619360 (+1408297us) service_pool.cc:224] Handling call
0709 13:20:19.639628 (+ 20268us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:19.640692 (+  1064us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:19.640906 (+   214us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:20:19.645401 19306 raft_consensus.cc:2804] T 8c11074579b046d99e585c2818e18a2c P cb82cc7bd6db46d980c75237a1402860 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:20:19.645933 19306 raft_consensus.cc:493] T 8c11074579b046d99e585c2818e18a2c P cb82cc7bd6db46d980c75237a1402860 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:19.646332 19306 raft_consensus.cc:3060] T 8c11074579b046d99e585c2818e18a2c P cb82cc7bd6db46d980c75237a1402860 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:20:19.648149 16297 raft_consensus.cc:1275] T c0eab25ce4c64e018a041beeda2867db P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:20:19.648797 16218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87ac0d1f7e8a419f9bcbc04aee0e067e" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:20:19.648918 16297 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48684 (request call id 2247) took 1437 ms. Trace:
I20260709 13:20:19.650339 16297 rpcz_store.cc:276] 0709 13:20:18.211565 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:19.616514 (+1404949us) service_pool.cc:224] Handling call
0709 13:20:19.647575 (+ 31061us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:19.648643 (+  1068us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:19.648858 (+   215us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:20:19.651525 19382 raft_consensus.cc:2804] T 8638ee363273426db1393dd81e5c7b72 P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:20:19.652087 19382 raft_consensus.cc:493] T 8638ee363273426db1393dd81e5c7b72 P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:19.652204 19142 raft_consensus.cc:3060] T 5a96373bb0e14dce8a266b2fc6c58507 P 1ca2725d290e47d68b1a4b609c9816ac [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:20:19.652621 19382 raft_consensus.cc:3060] T 8638ee363273426db1393dd81e5c7b72 P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:20:19.654603 19421 leader_election.cc:290] T 829b4ad8c5a54eababeea1181b960d4b P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 8 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:19.655342 19416 raft_consensus.cc:515] T b6f07842d3aa47858df5665874c86f3a P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:19.655953 19359 raft_consensus.cc:493] T d2ab0ac8347641e6955e9bce7d90c1a2 P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:19.656409 16190 leader_election.cc:304] T 5a959dbecc2648eb85d1cc550f5a3245 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:20:19.656615 19359 raft_consensus.cc:515] T d2ab0ac8347641e6955e9bce7d90c1a2 P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:19.658372 19416 leader_election.cc:290] T b6f07842d3aa47858df5665874c86f3a P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:19.612994 19088 raft_consensus.cc:2749] T 02838c86531d4f98bf58d1c22b8baf77 P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:20:19.659365 19233 raft_consensus.cc:2749] T 5a959dbecc2648eb85d1cc550f5a3245 P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:20:19.611078 19080 raft_consensus.cc:515] T 57a744f5aaf944678a63469cbf514015 P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:19.659956 19480 consensus_queue.cc:1059] T 84f2da98864b4425bf7c0055951fdd4b P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:20:19.660391 16300 raft_consensus.cc:1275] T 8d4ab870a05347868956aa8b841b2981 P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 7 index: 3. Preceding OpId from leader: term: 8 index: 4. (index mismatch)
I20260709 13:20:19.661010 19480 raft_consensus.cc:3055] T 84f2da98864b4425bf7c0055951fdd4b P 1ca2725d290e47d68b1a4b609c9816ac [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:20:19.661643 19480 raft_consensus.cc:740] T 84f2da98864b4425bf7c0055951fdd4b P 1ca2725d290e47d68b1a4b609c9816ac [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:20:19.662130 16220 raft_consensus.cc:1240] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac [term 6 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 5. Current term is 6. Ops: []
I20260709 13:20:19.662016 19412 raft_consensus.cc:515] T 3a7ca61592e848ea9b0fb81a2ce33cdc P cb82cc7bd6db46d980c75237a1402860 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:19.664083 16220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fffab26427e24d398e2695aad40ca134" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:19.640800 16817 rpcz_store.cc:276] 0709 13:20:17.944602 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:17.944752 (+   150us) service_pool.cc:224] Handling call
0709 13:20:17.945088 (+   336us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:17.945373 (+   285us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:20:18.426749 (+481376us) log.cc:844] Serialized 14 byte log entry
0709 13:20:19.280182 (+853433us) log.cc:844] Serialized 14 byte log entry
0709 13:20:19.605384 (+325202us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:20:19.611768 (+  6384us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:20:19.611819 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:19.625887 (+ 14068us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:19.640321 (+ 14434us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":1100758,"prepare.run_cpu_time_us":106,"prepare.run_wall_time_us":2901,"raft.queue_time_us":170478,"raft.run_cpu_time_us":169,"raft.run_wall_time_us":170,"thread_start_us":1595488,"threads_started":3,"wal-append.queue_time_us":921007}
I20260709 13:20:19.664865 16220 raft_consensus.cc:2468] T fffab26427e24d398e2695aad40ca134 P 1ca2725d290e47d68b1a4b609c9816ac [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 6.
I20260709 13:20:19.664920 19412 leader_election.cc:290] T 3a7ca61592e848ea9b0fb81a2ce33cdc P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:19.665968 16822 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49906 (request call id 703) took 1695 ms. Trace:
I20260709 13:20:19.666275 16298 raft_consensus.cc:1275] T 23f82224f4564fea93572e8206bad1fe P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 6 index: 6. (index mismatch)
I20260709 13:20:19.667542 19359 leader_election.cc:290] T d2ab0ac8347641e6955e9bce7d90c1a2 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:19.666426 16822 rpcz_store.cc:276] 0709 13:20:17.970903 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:17.971029 (+   126us) service_pool.cc:224] Handling call
0709 13:20:17.971309 (+   280us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:17.971494 (+   185us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:20:18.535689 (+564195us) log.cc:844] Serialized 14 byte log entry
0709 13:20:19.611388 (+1075699us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:20:19.624258 (+ 12870us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:20:19.665436 (+ 41178us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:19.665559 (+   123us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:19.665755 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":76396,"prepare.queue_time_us":1082939,"prepare.run_cpu_time_us":74,"prepare.run_wall_time_us":76,"raft.queue_time_us":52151,"raft.run_cpu_time_us":90,"raft.run_wall_time_us":89,"thread_start_us":1435757,"threads_started":3,"wal-append.queue_time_us":893834}
W20260709 13:20:19.667577 16117 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 213173us
I20260709 13:20:19.670547 16117 leader_election.cc:304] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860; no voters: 
W20260709 13:20:19.670682 16142 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46224 (request call id 448) took 1910 ms (client timeout 1860 ms). Trace:
W20260709 13:20:19.671103 16142 rpcz_store.cc:269] 0709 13:20:17.760385 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:17.844952 (+ 84567us) service_pool.cc:224] Handling call
0709 13:20:19.670454 (+1825502us) spinlock_profiling.cc:243] Waited 1.62 s on lock 0x7b54000e9054. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:20:19.670615 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3727241600}
I20260709 13:20:19.671665 16117 leader_election.cc:304] T b87147731a68400e8cce9b879daa7c5d P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860; no voters: 90f3da57796b488c9087a2f523adf045
I20260709 13:20:19.661051 16300 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48684 (request call id 2244) took 1534 ms. Trace:
I20260709 13:20:19.673797 16300 rpcz_store.cc:276] 0709 13:20:18.210731 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:19.557540 (+1346809us) service_pool.cc:224] Handling call
0709 13:20:19.659997 (+102457us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:19.660773 (+   776us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:19.660962 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:20:19.677158 16222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806d7690c2b644959152074bb2cab0a7" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:19.677299 19213 raft_consensus.cc:2804] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:20:19.677407 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4dc6b0541ca431ebc757beac00521f1" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:20:19.678135 19213 raft_consensus.cc:697] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b [term 9 LEADER]: Becoming Leader. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:20:19.679028 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c2095a5f4ad49408dbc7f69c606120d" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:20:19.679364 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "829b4ad8c5a54eababeea1181b960d4b" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:20:19.679795 16265 leader_election.cc:304] T e5ad4eb281564936a59f41e387a47156 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 647a343a542d4a139fe106c19095129b
I20260709 13:20:19.613804 19414 raft_consensus.cc:2749] T cdbd78acb4cf46298e0de8ba0d130e19 P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:20:19.684099 16217 raft_consensus.cc:2393] T 829b4ad8c5a54eababeea1181b960d4b P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 8: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
I20260709 13:20:19.623715 19197 raft_consensus.cc:2749] T 2794af5b99984efebdee564035841cb0 P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:20:19.705521 16296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 8 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:19.706887 16219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4f65cdf00b748bb963ee6e81ef43b79" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
W20260709 13:20:19.687397 16115 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.14.177.131:41389 timed out after 1.665s (SENT) [suppressed 8 similar messages]
I20260709 13:20:19.706305 16296 raft_consensus.cc:2468] T ecc1e363230043fe9fef390f0b979f20 P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 7.
I20260709 13:20:19.708014 16790 leader_election.cc:304] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:20:19.630846 19418 raft_consensus.cc:493] T c0eab25ce4c64e018a041beeda2867db P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:19.720541 16819 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49906 (request call id 704) took 1708 ms. Trace:
W20260709 13:20:19.722937 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 123583us
I20260709 13:20:19.725013 16790 leader_election.cc:304] T 74ca8140f35c4ae29e8a664f6bdfc829 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 90f3da57796b488c9087a2f523adf045
I20260709 13:20:19.724825 19306 raft_consensus.cc:515] T 8c11074579b046d99e585c2818e18a2c P cb82cc7bd6db46d980c75237a1402860 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:19.725376 19501 raft_consensus.cc:2749] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:20:19.727840 16295 raft_consensus.cc:1275] T c4f65cdf00b748bb963ee6e81ef43b79 P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 13:20:19.701587 19345 raft_consensus.cc:2749] T a378d7f796f94615b3b347119a9880fd P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:20:19.759119 16295 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48684 (request call id 2246) took 1547 ms. Trace:
I20260709 13:20:19.759568 16295 rpcz_store.cc:276] 0709 13:20:18.211334 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:19.621140 (+1409806us) service_pool.cc:224] Handling call
0709 13:20:19.698309 (+ 77169us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:19.755666 (+ 57357us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:19.759061 (+  3395us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:20:19.748215 16789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 244188us
I20260709 13:20:19.758262 16294 raft_consensus.cc:1240] T 57a744f5aaf944678a63469cbf514015 P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 5. Current term is 6. Ops: []
I20260709 13:20:19.758335 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6f07842d3aa47858df5665874c86f3a" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:19.706596 19421 raft_consensus.cc:493] T 68a6838d82ef4cac83507f906139d90f P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:20:19.727416 19465 raft_consensus.cc:2749] T ada5a004ba7c46819051e3ccec71ffdb P cb82cc7bd6db46d980c75237a1402860 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:20:19.772261 16266 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:19.662786 19480 consensus_queue.cc:260] T 84f2da98864b4425bf7c0055951fdd4b P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:19.777566 16298 raft_consensus.cc:1275] T d2ab0ac8347641e6955e9bce7d90c1a2 P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:20:19.728547 16819 rpcz_store.cc:276] 0709 13:20:18.011917 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:18.013899 (+  1982us) service_pool.cc:224] Handling call
0709 13:20:18.022029 (+  8130us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:20:18.022349 (+   320us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:18.022377 (+    28us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:20:18.033218 (+ 10841us) log.cc:844] Serialized 52 byte log entry
0709 13:20:18.557989 (+524771us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:20:19.313106 (+755117us) log.cc:844] Serialized 14 byte log entry
0709 13:20:19.711927 (+398821us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:19.720023 (+  8096us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:20:19.720134 (+   111us) raft_consensus.cc:1685] finished
0709 13:20:19.720147 (+    13us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:19.720351 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":497165,"prepare.run_cpu_time_us":76,"prepare.run_wall_time_us":72,"raft.queue_time_us":50132,"raft.run_cpu_time_us":243,"raft.run_wall_time_us":8678,"thread_start_us":1010981,"threads_started":3,"wal-append.queue_time_us":625702}
I20260709 13:20:19.726544 19482 raft_consensus.cc:493] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:19.758402 19450 raft_consensus.cc:2804] T b87147731a68400e8cce9b879daa7c5d P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Leader election won for term 9
W20260709 13:20:19.791865 16117 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 120375us
I20260709 13:20:19.730022 19233 raft_consensus.cc:493] T d16dfad0c3b540e5b4b86492ddf87208 P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:20:19.750355 19359 raft_consensus.cc:493] T 31969392ccdb43b491911a319e8bc7e1 P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:20:19.790319 16142 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c4f65cdf00b748bb963ee6e81ef43b79"
dest_uuid: "647a343a542d4a139fe106c19095129b"
mode: GRACEFUL
new_leader_uuid: "90f3da57796b488c9087a2f523adf045"
 from {username='slave'} at 127.0.0.1:40290
I20260709 13:20:19.679364 19213 consensus_queue.cc:237] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:19.791487 16221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af57996a8be84d398f0f023e8d08ef77" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:19.802240 16142 raft_consensus.cc:606] T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Received request to transfer leadership to TS 90f3da57796b488c9087a2f523adf045
I20260709 13:20:19.788810 16150 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40844 (request call id 2103) took 1773 ms. Trace:
I20260709 13:20:19.811679 16142 raft_consensus.cc:614] T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 13:20:19.792790 19378 consensus_meta.cc:287] T 1df93779209748b9860f6d893dd3de53 P 1ca2725d290e47d68b1a4b609c9816ac: Time spent flushing consensus metadata: real 0.674s	user 0.005s	sys 0.008s
I20260709 13:20:19.810829 16217 raft_consensus.cc:2468] T b6f07842d3aa47858df5665874c86f3a P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 7.
I20260709 13:20:19.824568 16824 raft_consensus.cc:1275] T 806d7690c2b644959152074bb2cab0a7 P cb82cc7bd6db46d980c75237a1402860 [term 6 LEARNER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:20:19.784453 19148 raft_consensus.cc:2804] T 74ca8140f35c4ae29e8a664f6bdfc829 P cb82cc7bd6db46d980c75237a1402860 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:20:19.781471 19068 raft_consensus.cc:2804] T e5ad4eb281564936a59f41e387a47156 P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:20:19.783831 19306 leader_election.cc:290] T 8c11074579b046d99e585c2818e18a2c P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 8 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:19.783143 19316 raft_consensus.cc:2749] T 7c2095a5f4ad49408dbc7f69c606120d P 1ca2725d290e47d68b1a4b609c9816ac [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:20:19.831437 16221 raft_consensus.cc:2468] T af57996a8be84d398f0f023e8d08ef77 P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 8.
I20260709 13:20:19.793123 18901 raft_consensus.cc:493] T ee9ed67006144797b1de6ea8c2013a68 P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:20:19.787405 19418 raft_consensus.cc:515] T c0eab25ce4c64e018a041beeda2867db P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:19.788244 19425 raft_consensus.cc:515] T 66ae176961b74813a58bc15c9c4c6f2a P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:19.811782 16150 rpcz_store.cc:276] 0709 13:20:18.014976 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:18.146731 (+131755us) service_pool.cc:224] Handling call
0709 13:20:18.204054 (+ 57323us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:18.204260 (+   206us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:20:19.304314 (+1100054us) log.cc:844] Serialized 14 byte log entry
0709 13:20:19.780905 (+476591us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:20:19.788346 (+  7441us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:20:19.788476 (+   130us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:19.788575 (+    99us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:19.788751 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5181,"prepare.queue_time_us":623383,"prepare.run_cpu_time_us":27,"prepare.run_wall_time_us":27,"raft.queue_time_us":18346,"raft.run_cpu_time_us":200,"raft.run_wall_time_us":17685,"thread_start_us":1518821,"threads_started":2}
I20260709 13:20:19.826578 19450 raft_consensus.cc:697] T b87147731a68400e8cce9b879daa7c5d P 647a343a542d4a139fe106c19095129b [term 9 LEADER]: Becoming Leader. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:20:19.835602 19068 raft_consensus.cc:493] T e5ad4eb281564936a59f41e387a47156 P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 647a343a542d4a139fe106c19095129b)
W20260709 13:20:19.866050 16789 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:19.883987 16226 raft_consensus.cc:3060] T 7c2095a5f4ad49408dbc7f69c606120d P 1ca2725d290e47d68b1a4b609c9816ac [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:20:19.851621 16116 leader_election.cc:304] T 8158bbd17b994d2eb96f560bd0f1b88d P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
W20260709 13:20:19.884025 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 124559us
W20260709 13:20:19.887373 16789 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:19.887512 16265 leader_election.cc:304] T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:20:19.853644 19148 raft_consensus.cc:697] T 74ca8140f35c4ae29e8a664f6bdfc829 P cb82cc7bd6db46d980c75237a1402860 [term 8 LEADER]: Becoming Leader. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:20:19.947093 16294 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40474 (request call id 478) took 1960 ms. Trace:
I20260709 13:20:19.857142 19378 raft_consensus.cc:493] T af57996a8be84d398f0f023e8d08ef77 P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:19.947625 16294 rpcz_store.cc:276] 0709 13:20:17.986276 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:19.014430 (+1028154us) service_pool.cc:224] Handling call
0709 13:20:19.015151 (+   721us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:19.769281 (+754130us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:19.944869 (+175588us) spinlock_profiling.cc:243] Waited 657 ms on lock 0x7b540035f7d4. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:20:19.947016 (+  2147us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1510969600}
I20260709 13:20:19.890767 19068 raft_consensus.cc:3060] T e5ad4eb281564936a59f41e387a47156 P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:20:19.892625 19364 consensus_meta.cc:287] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 1ca2725d290e47d68b1a4b609c9816ac: Time spent flushing consensus metadata: real 0.506s	user 0.003s	sys 0.008s
W20260709 13:20:19.951514 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 99997us
W20260709 13:20:19.960903 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 66588.8us
W20260709 13:20:19.961735 16115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 322934us
W20260709 13:20:19.962337 16115 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:19.891579 19306 raft_consensus.cc:493] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:19.952884 16296 raft_consensus.cc:1275] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:20:19.847052 19233 raft_consensus.cc:515] T d16dfad0c3b540e5b4b86492ddf87208 P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:20:19.965788 16117 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 125411us
I20260709 13:20:19.846861 19421 raft_consensus.cc:515] T 68a6838d82ef4cac83507f906139d90f P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:19.855846 19359 raft_consensus.cc:515] T 31969392ccdb43b491911a319e8bc7e1 P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:20:19.963932 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 239016us
I20260709 13:20:19.966506 16820 raft_consensus.cc:1240] T 8c11074579b046d99e585c2818e18a2c P cb82cc7bd6db46d980c75237a1402860 [term 8 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 7. Current term is 8. Ops: [7.9-7.9]
I20260709 13:20:19.969373 16149 raft_consensus.cc:1275] T c0eab25ce4c64e018a041beeda2867db P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:20:19.974208 16816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ab0ac8347641e6955e9bce7d90c1a2" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 9 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:19.983336 19496 raft_consensus.cc:493] T 434bff3082d141838068f323bdb0d1ed P cb82cc7bd6db46d980c75237a1402860 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:19.984050 19496 raft_consensus.cc:515] T 434bff3082d141838068f323bdb0d1ed P cb82cc7bd6db46d980c75237a1402860 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:19.986624 19407 raft_consensus.cc:493] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:19.986191 19450 consensus_queue.cc:237] T b87147731a68400e8cce9b879daa7c5d P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:19.986868 16116 leader_election.cc:304] T fffab26427e24d398e2695aad40ca134 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
I20260709 13:20:19.987339 19407 raft_consensus.cc:515] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:19.888664 18901 raft_consensus.cc:515] T ee9ed67006144797b1de6ea8c2013a68 P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:19.867116 19482 raft_consensus.cc:515] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:19.963486 19414 raft_consensus.cc:2749] T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
W20260709 13:20:19.979940 19404 consensus_meta.cc:287] T 746aabf856ab401c9ef5e859371e7b67 P 647a343a542d4a139fe106c19095129b: Time spent flushing consensus metadata: real 0.523s	user 0.011s	sys 0.002s
W20260709 13:20:20.003710 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 180675us
I20260709 13:20:20.004312 16789 leader_election.cc:304] T 3884f864a96d428785895e5463e5728f P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:20:20.005968 16117 leader_election.cc:304] T ee9ed67006144797b1de6ea8c2013a68 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860
I20260709 13:20:19.965189 19364 raft_consensus.cc:515] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:20.014732 16221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87ac0d1f7e8a419f9bcbc04aee0e067e" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
W20260709 13:20:20.015378 16192 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:19.970929 19378 raft_consensus.cc:515] T af57996a8be84d398f0f023e8d08ef77 P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:20.011334 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ad5dbfa99ee4b8d9e28be92b3f36db5" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:20.012265 16820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57a744f5aaf944678a63469cbf514015" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:20:20.017328 16790 leader_election.cc:304] T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:20:19.997296 19197 raft_consensus.cc:2804] T 8158bbd17b994d2eb96f560bd0f1b88d P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:20:19.978857 19306 raft_consensus.cc:515] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:20:20.038352 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 51456.7us
I20260709 13:20:20.041249 16115 leader_election.cc:304] T 8cc959fb7f704fba9009c0595467d074 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac
I20260709 13:20:20.021090 19289 raft_consensus.cc:2749] T 3884f864a96d428785895e5463e5728f P cb82cc7bd6db46d980c75237a1402860 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:20:20.055548 16789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 55790us
I20260709 13:20:20.006969 19404 raft_consensus.cc:515] T 746aabf856ab401c9ef5e859371e7b67 P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:20.066493 16816 raft_consensus.cc:1275] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 [term 7 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 9. (index mismatch)
I20260709 13:20:20.059653 19522 consensus_queue.cc:1059] T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
W20260709 13:20:20.060357 19507 log.cc:927] Time spent T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860: Append to log took a long time: real 0.079s	user 0.000s	sys 0.001s
I20260709 13:20:20.057422 19148 consensus_queue.cc:237] T 74ca8140f35c4ae29e8a664f6bdfc829 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:20.061190 19306 leader_election.cc:290] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:20.062405 19364 leader_election.cc:290] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 8 election: Requested vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:20.051648 19465 raft_consensus.cc:2749] T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:20:20.063326 19515 raft_consensus.cc:493] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:20.064656 16265 leader_election.cc:304] T e4dc6b0541ca431ebc757beac00521f1 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:20:20.065114 19512 raft_consensus.cc:493] T 8cc959fb7f704fba9009c0595467d074 P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:20.065414 19377 raft_consensus.cc:493] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:20:20.100461 19414 consensus_queue.cc:1048] T fffab26427e24d398e2695aad40ca134 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:20:20.101167 19275 consensus_queue.cc:1048] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:20:20.101358 19377 raft_consensus.cc:515] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:20.101639 19509 consensus_queue.cc:1059] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.008s
I20260709 13:20:20.103956 19377 leader_election.cc:290] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:20.105139 19404 leader_election.cc:290] T 746aabf856ab401c9ef5e859371e7b67 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 8 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:20.100348 19478 raft_consensus.cc:493] T d2ab0ac8347641e6955e9bce7d90c1a2 P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:20.106477 19478 raft_consensus.cc:515] T d2ab0ac8347641e6955e9bce7d90c1a2 P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:20.065423 19510 consensus_queue.cc:1059] T 08c74490c40243a88e62d1760b5cd814 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.042s
I20260709 13:20:20.100338 19512 raft_consensus.cc:515] T 8cc959fb7f704fba9009c0595467d074 P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:20.098999 19496 leader_election.cc:290] T 434bff3082d141838068f323bdb0d1ed P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:20.104404 19509 consensus_queue.cc:1059] T 746aabf856ab401c9ef5e859371e7b67 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.011s
I20260709 13:20:20.074651 19316 raft_consensus.cc:493] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:20.113761 19404 raft_consensus.cc:2804] T fffab26427e24d398e2695aad40ca134 P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 13:20:20.136752 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 143676us
I20260709 13:20:20.108331 19510 raft_consensus.cc:3055] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:20:20.112226 19425 leader_election.cc:290] T 66ae176961b74813a58bc15c9c4c6f2a P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 10 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:20.065848 19419 raft_consensus.cc:493] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:20.119263 19418 leader_election.cc:290] T c0eab25ce4c64e018a041beeda2867db P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:20.126969 19414 raft_consensus.cc:2749] T e4dc6b0541ca431ebc757beac00521f1 P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:20:20.190443 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 152809us
I20260709 13:20:20.123476 19465 consensus_queue.cc:1048] T 9e65676b21b748728ad39668cfe722ec P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.022s
W20260709 13:20:20.221249 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 79563.4us
I20260709 13:20:20.147843 19197 raft_consensus.cc:697] T 8158bbd17b994d2eb96f560bd0f1b88d P 647a343a542d4a139fe106c19095129b [term 8 LEADER]: Becoming Leader. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:20:20.221107 16790 leader_election.cc:304] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 647a343a542d4a139fe106c19095129b
I20260709 13:20:20.139802 19404 raft_consensus.cc:493] T fffab26427e24d398e2695aad40ca134 P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:20:20.231855 16192 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 166162us
I20260709 13:20:20.141987 19510 raft_consensus.cc:740] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
W20260709 13:20:20.241827 16117 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 249176us
I20260709 13:20:20.250330 16192 leader_election.cc:304] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 
W20260709 13:20:20.252441 19506 log.cc:927] Time spent T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860: Append to log took a long time: real 0.250s	user 0.001s	sys 0.000s
W20260709 13:20:20.252556 16115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 215093us
I20260709 13:20:20.238983 19404 raft_consensus.cc:3060] T fffab26427e24d398e2695aad40ca134 P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:20:20.257414 15046 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:20:20.236578 19421 leader_election.cc:290] T 68a6838d82ef4cac83507f906139d90f P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:20.153895 19316 raft_consensus.cc:515] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:20.243745 19359 leader_election.cc:290] T 31969392ccdb43b491911a319e8bc7e1 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:20.267445 19378 leader_election.cc:290] T af57996a8be84d398f0f023e8d08ef77 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:20.267628 19522 raft_consensus.cc:2804] T 8cc959fb7f704fba9009c0595467d074 P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Leader election won for term 8
W20260709 13:20:20.261629 19505 log.cc:927] Time spent T 84f2da98864b4425bf7c0055951fdd4b P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 0.148s	user 0.001s	sys 0.000s
I20260709 13:20:20.268334 19522 raft_consensus.cc:697] T 8cc959fb7f704fba9009c0595467d074 P 647a343a542d4a139fe106c19095129b [term 8 LEADER]: Becoming Leader. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:20:20.269731 19522 consensus_queue.cc:237] T 8cc959fb7f704fba9009c0595467d074 P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:20.242781 19418 raft_consensus.cc:2749] T ee9ed67006144797b1de6ea8c2013a68 P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:20:20.263252 19210 raft_consensus.cc:2764] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 [term 6 FOLLOWER]: Leader pre-election decision vote started in defunct term 5: won
I20260709 13:20:20.271376 16115 leader_election.cc:304] T 1df93779209748b9860f6d893dd3de53 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac
I20260709 13:20:20.263304 19519 raft_consensus.cc:3055] T 08c74490c40243a88e62d1760b5cd814 P 1ca2725d290e47d68b1a4b609c9816ac [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:20:20.272367 19519 raft_consensus.cc:740] T 08c74490c40243a88e62d1760b5cd814 P 1ca2725d290e47d68b1a4b609c9816ac [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:20:20.273778 19519 consensus_queue.cc:260] T 08c74490c40243a88e62d1760b5cd814 P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:20.266640 19529 raft_consensus.cc:493] T 84f2da98864b4425bf7c0055951fdd4b P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:20:20.275934 19529 raft_consensus.cc:515] T 84f2da98864b4425bf7c0055951fdd4b P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:20.276691 16115 leader_election.cc:304] T 08c74490c40243a88e62d1760b5cd814 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:20:20.277105 16117 leader_election.cc:304] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860
I20260709 13:20:20.278514 19531 raft_consensus.cc:3060] T 08c74490c40243a88e62d1760b5cd814 P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:20:20.278631 19529 leader_election.cc:290] T 84f2da98864b4425bf7c0055951fdd4b P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:20.126457 19515 raft_consensus.cc:515] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:20.272465 19418 raft_consensus.cc:2804] T 1df93779209748b9860f6d893dd3de53 P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:20:20.283260 19418 raft_consensus.cc:697] T 1df93779209748b9860f6d893dd3de53 P 647a343a542d4a139fe106c19095129b [term 8 LEADER]: Becoming Leader. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:20:20.284628 19515 leader_election.cc:290] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:20.284578 19418 consensus_queue.cc:237] T 1df93779209748b9860f6d893dd3de53 P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:20.285596 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
W20260709 13:20:20.283282 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 218832us
I20260709 13:20:20.286546 16299 raft_consensus.cc:2393] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 in current term 8: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
I20260709 13:20:20.287865 16265 leader_election.cc:304] T d16dfad0c3b540e5b4b86492ddf87208 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:20:20.288190 16822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e4bfb9090c8460eb14fa7e0d818e7f1" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:20:20.290000 16265 leader_election.cc:304] T 84f2da98864b4425bf7c0055951fdd4b P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:20:20.290935 19425 raft_consensus.cc:2749] T d16dfad0c3b540e5b4b86492ddf87208 P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:20:20.291281 19517 consensus_queue.cc:1048] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.046s
I20260709 13:20:20.291867 19425 consensus_queue.cc:1048] T 8c11074579b046d99e585c2818e18a2c P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:20:20.254115 19419 raft_consensus.cc:515] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:20.293322 19425 raft_consensus.cc:2749] T 84f2da98864b4425bf7c0055951fdd4b P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:20:20.293797 19531 raft_consensus.cc:2749] T 08c74490c40243a88e62d1760b5cd814 P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:20:20.294523 19531 raft_consensus.cc:2749] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:20:20.295318 19531 raft_consensus.cc:493] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:20:20.295851 19531 raft_consensus.cc:515] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:20.298539 19531 leader_election.cc:290] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:20.299896 16191 leader_election.cc:304] T 829b4ad8c5a54eababeea1181b960d4b P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
I20260709 13:20:20.303437 16221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74ca8140f35c4ae29e8a664f6bdfc829" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:20.303658 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c74490c40243a88e62d1760b5cd814" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:20:20.305606 16219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f5f89186204242bf377f7c42b2e2da" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:20.305866 16220 raft_consensus.cc:1240] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 7. Current term is 8. Ops: []
I20260709 13:20:20.307982 19482 leader_election.cc:290] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:20.309598 16219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a878999c0024d2abe517484180290fc" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:20.310355 16219 raft_consensus.cc:2468] T 8a878999c0024d2abe517484180290fc P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 7.
I20260709 13:20:20.311673 16219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806d7690c2b644959152074bb2cab0a7" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:20.311545 19510 consensus_queue.cc:260] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:20.313148 19510 raft_consensus.cc:3060] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:20:20.313342 16219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ab0ac8347641e6955e9bce7d90c1a2" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 9 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:20.314260 16219 raft_consensus.cc:2410] T d2ab0ac8347641e6955e9bce7d90c1a2 P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 for term 9 because replica has last-logged OpId of term: 7 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 7.
I20260709 13:20:20.314597 18901 leader_election.cc:290] T ee9ed67006144797b1de6ea8c2013a68 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:20.315497 16219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66ae176961b74813a58bc15c9c4c6f2a" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:20:20.321532 19414 raft_consensus.cc:493] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:20.322129 19414 raft_consensus.cc:515] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:20.325590 19414 leader_election.cc:290] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:20.326357 19414 raft_consensus.cc:493] T 5a959dbecc2648eb85d1cc550f5a3245 P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
W20260709 13:20:20.326915 16189 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:20.327280 19359 raft_consensus.cc:493] T 9e65676b21b748728ad39668cfe722ec P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:20.327848 19359 raft_consensus.cc:515] T 9e65676b21b748728ad39668cfe722ec P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:20.330170 19359 leader_election.cc:290] T 9e65676b21b748728ad39668cfe722ec P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:20.330976 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87ac0d1f7e8a419f9bcbc04aee0e067e" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:20:20.334403 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fffab26427e24d398e2695aad40ca134" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:20.335469 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e65676b21b748728ad39668cfe722ec" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:20.335740 16294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4f65cdf00b748bb963ee6e81ef43b79" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:20.259377 19478 leader_election.cc:290] T d2ab0ac8347641e6955e9bce7d90c1a2 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:20.336067 16300 raft_consensus.cc:2468] T 9e65676b21b748728ad39668cfe722ec P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 8.
I20260709 13:20:20.336292 16294 raft_consensus.cc:2468] T c4f65cdf00b748bb963ee6e81ef43b79 P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 9.
I20260709 13:20:20.337600 16291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c2095a5f4ad49408dbc7f69c606120d" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:20:20.338692 16296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e9b2bf7011f4ae688a79d7bb5a447b4" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:20.333439 16818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2794af5b99984efebdee564035841cb0" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:20.340073 16818 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49892 (request call id 484) took 1062 ms. Trace:
I20260709 13:20:20.340683 16816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af57996a8be84d398f0f023e8d08ef77" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:20.341070 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4dc6b0541ca431ebc757beac00521f1" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:20:20.341353 16816 raft_consensus.cc:2468] T af57996a8be84d398f0f023e8d08ef77 P cb82cc7bd6db46d980c75237a1402860 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 7.
W20260709 13:20:20.341708 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 120699us
I20260709 13:20:20.342109 16817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9944d5e4b8c4109af9955c4f08c9095" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 9 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:20.342435 16820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a378d7f796f94615b3b347119a9880fd" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:20.341354 16148 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40844 (request call id 2096) took 2489 ms. Trace:
I20260709 13:20:20.343330 16822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee9ed67006144797b1de6ea8c2013a68" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:20.343484 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f82224f4564fea93572e8206bad1fe" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 7 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:20:20.343575 16148 rpcz_store.cc:276] 0709 13:20:17.792198 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:17.951391 (+159193us) service_pool.cc:224] Handling call
0709 13:20:17.994011 (+ 42620us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:19.448187 (+1454176us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:20.248380 (+800193us) spinlock_profiling.cc:243] Waited 1.45 s on lock 0x7b54001f22d4. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:20:20.262158 (+ 13778us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3342190976}
I20260709 13:20:20.344290 16298 raft_consensus.cc:3060] T 23f82224f4564fea93572e8206bad1fe P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:20:20.344800 19421 raft_consensus.cc:3055] T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:20:20.343081 16820 raft_consensus.cc:2468] T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 7.
I20260709 13:20:20.345292 19421 raft_consensus.cc:740] T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:20:20.340431 16818 rpcz_store.cc:276] 0709 13:20:19.277644 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:20.333186 (+1055542us) service_pool.cc:224] Handling call
0709 13:20:20.340016 (+  6830us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:20:20.346378 19421 consensus_queue.cc:260] T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.8, Last appended by leader: 8, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:20.280050 19519 raft_consensus.cc:3060] T 08c74490c40243a88e62d1760b5cd814 P 1ca2725d290e47d68b1a4b609c9816ac [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:20:20.343047 16817 raft_consensus.cc:2410] T c9944d5e4b8c4109af9955c4f08c9095 P cb82cc7bd6db46d980c75237a1402860 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac for term 9 because replica has last-logged OpId of term: 7 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 7.
I20260709 13:20:20.341758 16297 raft_consensus.cc:3060] T e4dc6b0541ca431ebc757beac00521f1 P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:20:20.348039 19197 consensus_queue.cc:237] T 8158bbd17b994d2eb96f560bd0f1b88d P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:20.364718 16226 raft_consensus.cc:2468] T 7c2095a5f4ad49408dbc7f69c606120d P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 7.
I20260709 13:20:20.279284 16117 leader_election.cc:304] T 434bff3082d141838068f323bdb0d1ed P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860; no voters: 90f3da57796b488c9087a2f523adf045
I20260709 13:20:20.272203 19512 leader_election.cc:290] T 8cc959fb7f704fba9009c0595467d074 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:20.373075 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806d7690c2b644959152074bb2cab0a7" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:20.390166 16220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:20:20.391006 16819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5d59407eb545b895d8d922e0dd69fd" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:20.372913 16825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ab0ac8347641e6955e9bce7d90c1a2" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:20.390563 16299 raft_consensus.cc:2468] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 6.
I20260709 13:20:20.391687 16819 raft_consensus.cc:2468] T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 9.
I20260709 13:20:20.393818 16221 raft_consensus.cc:2393] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 8: Already voted for candidate cb82cc7bd6db46d980c75237a1402860 in this term.
I20260709 13:20:20.295053 19419 leader_election.cc:290] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:20.399962 16295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6f07842d3aa47858df5665874c86f3a" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:20.401400 16265 leader_election.cc:304] T 1df93779209748b9860f6d893dd3de53 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:20:20.357132 19359 raft_consensus.cc:493] T c0eab25ce4c64e018a041beeda2867db P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:20:20.404045 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 104224us
I20260709 13:20:20.406790 18599 leader_election.cc:290] T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:20.400684 16295 raft_consensus.cc:2468] T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 8.
I20260709 13:20:20.370893 19480 raft_consensus.cc:3060] T 84f2da98864b4425bf7c0055951fdd4b P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:20:20.364289 19465 consensus_queue.cc:1048] T 829b4ad8c5a54eababeea1181b960d4b P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.034s
I20260709 13:20:20.393857 19482 raft_consensus.cc:2749] T 829b4ad8c5a54eababeea1181b960d4b P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:20:20.326932 19414 raft_consensus.cc:515] T 5a959dbecc2648eb85d1cc550f5a3245 P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:20:20.418799 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 50222.5us
I20260709 13:20:20.419555 16148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87147731a68400e8cce9b879daa7c5d" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
W20260709 13:20:20.423373 16117 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 144265us
I20260709 13:20:20.399515 19526 raft_consensus.cc:2804] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:20:20.412670 19421 raft_consensus.cc:3060] T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:20:20.412459 19316 leader_election.cc:290] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:20.416832 19531 raft_consensus.cc:2804] T 434bff3082d141838068f323bdb0d1ed P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:20:20.433665 16142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:20.420915 19345 raft_consensus.cc:2749] T 1df93779209748b9860f6d893dd3de53 P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:20:20.440300 16142 raft_consensus.cc:2468] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 7.
I20260709 13:20:20.445967 16298 raft_consensus.cc:2468] T 23f82224f4564fea93572e8206bad1fe P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 7.
I20260709 13:20:20.448065 16148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5718c623ebee4d2bb770f0f9ddb2f164" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:20.453516 16148 raft_consensus.cc:2410] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 for term 8 because replica has last-logged OpId of term: 6 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 7.
W20260709 13:20:20.454120 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 88547.9us
I20260709 13:20:20.452463 16295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0eab25ce4c64e018a041beeda2867db" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 9 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:20.417373 19359 raft_consensus.cc:515] T c0eab25ce4c64e018a041beeda2867db P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:20.443063 19531 raft_consensus.cc:493] T 434bff3082d141838068f323bdb0d1ed P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Starting leader election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:20:20.461907 16265 leader_election.cc:304] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:20:20.257639 19530 consensus_queue.cc:1059] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: INVALID_TERM, Last received: 7.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.191s
W20260709 13:20:20.492628 16117 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 49273.7us
W20260709 13:20:20.494843 16791 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:20.442660 19526 raft_consensus.cc:697] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac [term 6 LEADER]: Becoming Leader. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:20:20.494958 16192 leader_election.cc:304] T af57996a8be84d398f0f023e8d08ef77 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:20:20.454933 19414 leader_election.cc:290] T 5a959dbecc2648eb85d1cc550f5a3245 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
W20260709 13:20:20.461712 19382 consensus_meta.cc:287] T 8638ee363273426db1393dd81e5c7b72 P 90f3da57796b488c9087a2f523adf045: Time spent flushing consensus metadata: real 0.808s	user 0.014s	sys 0.001s
W20260709 13:20:20.505288 16264 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:20.440073 19482 consensus_queue.cc:1048] T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.022s
W20260709 13:20:20.510915 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 57588.8us
I20260709 13:20:20.509193 16295 raft_consensus.cc:2410] T c0eab25ce4c64e018a041beeda2867db P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b for term 9 because replica has last-logged OpId of term: 7 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
W20260709 13:20:20.513109 16789 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:20.514127 16825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f82224f4564fea93572e8206bad1fe" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 7 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:20:20.496560 19531 raft_consensus.cc:3060] T 434bff3082d141838068f323bdb0d1ed P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:20:20.511096 19345 raft_consensus.cc:2749] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:20:20.523919 16296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68a6838d82ef4cac83507f906139d90f" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:20.524341 16151 raft_consensus.cc:1240] T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 7. Current term is 8. Ops: []
I20260709 13:20:20.526788 16148 raft_consensus.cc:1240] T 23f82224f4564fea93572e8206bad1fe P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 6. Current term is 7. Ops: []
I20260709 13:20:20.527581 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a96373bb0e14dce8a266b2fc6c58507" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 7 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
W20260709 13:20:20.524626 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 468083us
W20260709 13:20:20.560074 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 43779.8us
W20260709 13:20:20.563258 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 104983us
W20260709 13:20:20.563342 16192 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 68504.4us
I20260709 13:20:20.567986 16790 leader_election.cc:304] T b87147731a68400e8cce9b879daa7c5d P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
W20260709 13:20:20.530290 19142 consensus_meta.cc:287] T 5a96373bb0e14dce8a266b2fc6c58507 P 1ca2725d290e47d68b1a4b609c9816ac: Time spent flushing consensus metadata: real 0.877s	user 0.009s	sys 0.006s
W20260709 13:20:20.577356 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 118701us
I20260709 13:20:20.579195 16817 raft_consensus.cc:1240] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 [term 6 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 5. Current term is 6. Ops: []
I20260709 13:20:20.579130 16818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e65676b21b748728ad39668cfe722ec" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:20.576056 16192 leader_election.cc:304] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:20:20.582686 16298 raft_consensus.cc:2376] T 5a96373bb0e14dce8a266b2fc6c58507 P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 647a343a542d4a139fe106c19095129b in term 7. Re-sending same reply.
I20260709 13:20:20.584156 16825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e9b2bf7011f4ae688a79d7bb5a447b4" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:20.584580 16820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af57996a8be84d398f0f023e8d08ef77" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:20.585232 16820 raft_consensus.cc:2468] T af57996a8be84d398f0f023e8d08ef77 P cb82cc7bd6db46d980c75237a1402860 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 7.
I20260709 13:20:20.586212 16116 leader_election.cc:304] T b6f07842d3aa47858df5665874c86f3a P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
I20260709 13:20:20.587083 16116 leader_election.cc:304] T af57996a8be84d398f0f023e8d08ef77 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
I20260709 13:20:20.514801 19382 raft_consensus.cc:515] T 8638ee363273426db1393dd81e5c7b72 P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:20:20.576586 19527 log.cc:927] Time spent T e5ad4eb281564936a59f41e387a47156 P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
I20260709 13:20:20.590219 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9944d5e4b8c4109af9955c4f08c9095" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:20.591229 16822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "829b4ad8c5a54eababeea1181b960d4b" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:20:20.598362 16265 leader_election.cc:304] T 3884f864a96d428785895e5463e5728f P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
I20260709 13:20:20.582060 19478 raft_consensus.cc:2804] T af57996a8be84d398f0f023e8d08ef77 P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:20:20.579761 19465 raft_consensus.cc:2749] T b87147731a68400e8cce9b879daa7c5d P cb82cc7bd6db46d980c75237a1402860 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:20:20.605975 16825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746aabf856ab401c9ef5e859371e7b67" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:20:20.578814 19390 consensus_queue.cc:1048] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.133s
I20260709 13:20:20.600593 16151 raft_consensus.cc:1240] T 829b4ad8c5a54eababeea1181b960d4b P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 7. Current term is 8. Ops: [7.9-7.9]
I20260709 13:20:20.602370 19345 raft_consensus.cc:2749] T 3884f864a96d428785895e5463e5728f P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:20:20.602771 19522 raft_consensus.cc:2804] T af57996a8be84d398f0f023e8d08ef77 P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:20:20.602994 16820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:20.515604 19512 consensus_queue.cc:1048] T ecc1e363230043fe9fef390f0b979f20 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.083s
I20260709 13:20:20.610036 16825 raft_consensus.cc:2393] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 8: Already voted for candidate cb82cc7bd6db46d980c75237a1402860 in this term.
I20260709 13:20:20.609937 16818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2372981006d4b218215c28964dea623" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:20.611418 16822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5718c623ebee4d2bb770f0f9ddb2f164" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:20.612638 16817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74ca8140f35c4ae29e8a664f6bdfc829" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:20.590909 18599 raft_consensus.cc:2804] T b6f07842d3aa47858df5665874c86f3a P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 13:20:20.621587 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 53695.9us
I20260709 13:20:20.601066 18901 raft_consensus.cc:2804] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:20:20.602507 19418 raft_consensus.cc:493] T 18ad7e3af5504571a69d10c00750a01c P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:20.619061 16822 raft_consensus.cc:2468] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 7.
I20260709 13:20:20.604233 19478 raft_consensus.cc:493] T af57996a8be84d398f0f023e8d08ef77 P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:20.603281 19142 consensus_queue.cc:1059] T e4dc6b0541ca431ebc757beac00521f1 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.007s
I20260709 13:20:20.630435 16220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0eab25ce4c64e018a041beeda2867db" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 9 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:20.630617 16147 raft_consensus.cc:1240] T 7c2095a5f4ad49408dbc7f69c606120d P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 6. Current term is 7. Ops: []
I20260709 13:20:20.632284 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f2da98864b4425bf7c0055951fdd4b" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:20.631364 16151 raft_consensus.cc:1275] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 7 index: 9. (index mismatch)
I20260709 13:20:20.581521 19515 consensus_queue.cc:1048] T c0eab25ce4c64e018a041beeda2867db P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 2, Time since last communication: 0.033s
I20260709 13:20:20.624542 18599 raft_consensus.cc:493] T b6f07842d3aa47858df5665874c86f3a P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:20.633049 16299 raft_consensus.cc:2393] T 84f2da98864b4425bf7c0055951fdd4b P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 6: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
I20260709 13:20:20.633432 18599 raft_consensus.cc:3060] T b6f07842d3aa47858df5665874c86f3a P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:20:20.603818 19530 consensus_queue.cc:1048] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.031s
I20260709 13:20:20.614979 19522 raft_consensus.cc:493] T af57996a8be84d398f0f023e8d08ef77 P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:20.642119 16144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 8 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:20.642761 16143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5d59407eb545b895d8d922e0dd69fd" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:20.643587 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746aabf856ab401c9ef5e859371e7b67" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
W20260709 13:20:20.643997 19534 log.cc:927] Time spent T 74ca8140f35c4ae29e8a664f6bdfc829 P cb82cc7bd6db46d980c75237a1402860: Append to log took a long time: real 0.114s	user 0.000s	sys 0.001s
I20260709 13:20:20.649842 19390 raft_consensus.cc:493] T 5a96373bb0e14dce8a266b2fc6c58507 P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:20.650502 16265 leader_election.cc:304] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:20:20.650393 19390 raft_consensus.cc:515] T 5a96373bb0e14dce8a266b2fc6c58507 P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:20.652714 19390 leader_election.cc:290] T 5a96373bb0e14dce8a266b2fc6c58507 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:20.654328 16265 leader_election.cc:304] T 8a878999c0024d2abe517484180290fc P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:20:20.625895 18901 raft_consensus.cc:493] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:20.656167 16144 raft_consensus.cc:2468] T ecc1e363230043fe9fef390f0b979f20 P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 7.
I20260709 13:20:20.635486 19478 raft_consensus.cc:3060] T af57996a8be84d398f0f023e8d08ef77 P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:20:20.661638 18599 raft_consensus.cc:515] T b6f07842d3aa47858df5665874c86f3a P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:20.654209 19414 raft_consensus.cc:2749] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:20:20.669432 16192 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:20.614123 19482 consensus_queue.cc:1048] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.031s
I20260709 13:20:20.673151 16224 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57604 (request call id 2268) took 1558 ms. Trace:
I20260709 13:20:20.674312 16265 leader_election.cc:304] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860
I20260709 13:20:20.674417 16220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2372981006d4b218215c28964dea623" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:20.749575 16220 raft_consensus.cc:2468] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 8.
I20260709 13:20:20.613874 19514 consensus_queue.cc:1048] T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.061s
I20260709 13:20:20.648658 19522 raft_consensus.cc:3060] T af57996a8be84d398f0f023e8d08ef77 P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:20:20.695742 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87ac0d1f7e8a419f9bcbc04aee0e067e" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:20:20.636787 19515 consensus_queue.cc:1243] T c0eab25ce4c64e018a041beeda2867db P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Peer 90f3da57796b488c9087a2f523adf045 log is divergent from this leader: its last log entry 7.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260709 13:20:20.693295 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:20.678561 16148 raft_consensus.cc:1275] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:20:20.712726 16143 raft_consensus.cc:2393] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 9: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
I20260709 13:20:20.676920 16146 raft_consensus.cc:2955] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Committing config change with OpId 7.7: config changed from index 3 to 7, cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
W20260709 13:20:20.715665 19068 consensus_meta.cc:287] T e5ad4eb281564936a59f41e387a47156 P 90f3da57796b488c9087a2f523adf045: Time spent flushing consensus metadata: real 0.739s	user 0.011s	sys 0.000s
W20260709 13:20:20.717724 19539 log.cc:927] Time spent T 8c11074579b046d99e585c2818e18a2c P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 0.083s	user 0.000s	sys 0.000s
I20260709 13:20:20.637795 19418 raft_consensus.cc:515] T 18ad7e3af5504571a69d10c00750a01c P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } }
I20260709 13:20:20.718878 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f2da98864b4425bf7c0055951fdd4b" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:20.719821 16291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806d7690c2b644959152074bb2cab0a7" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:20.723295 19359 leader_election.cc:290] T c0eab25ce4c64e018a041beeda2867db P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:20:20.723877 19382 leader_election.cc:290] T 8638ee363273426db1393dd81e5c7b72 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 8 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:20.614704 19526 consensus_queue.cc:237] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:20.655294 19540 raft_consensus.cc:493] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:20:20.733407 16144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c74490c40243a88e62d1760b5cd814" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:20:20.734021 19531 raft_consensus.cc:515] T 434bff3082d141838068f323bdb0d1ed P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:20.662113 18901 raft_consensus.cc:3060] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:20:20.760861 18599 leader_election.cc:290] T b6f07842d3aa47858df5665874c86f3a P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 9 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:20.677824 16224 rpcz_store.cc:276] 0709 13:20:19.114702 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:19.683774 (+569072us) service_pool.cc:224] Handling call
0709 13:20:19.697532 (+ 13758us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:20:19.892766 (+195234us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:20:20.018093 (+125327us) log.cc:844] Serialized 14 byte log entry
0709 13:20:20.498066 (+479973us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:20:20.501911 (+  3845us) log.cc:844] Serialized 52 byte log entry
0709 13:20:20.502101 (+   190us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:20:20.502140 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:20.502212 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:20:20.672889 (+170677us) raft_consensus.cc:1685] finished
0709 13:20:20.672917 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:20.673080 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":41460,"prepare.queue_time_us":42759,"prepare.run_cpu_time_us":28,"prepare.run_wall_time_us":29,"raft.queue_time_us":257353,"raft.run_cpu_time_us":323,"raft.run_wall_time_us":13523,"thread_start_us":583709,"threads_started":4,"wal-append.queue_time_us":299630}
I20260709 13:20:20.675952 19425 raft_consensus.cc:2804] T 8a878999c0024d2abe517484180290fc P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:20:20.811369 16299 raft_consensus.cc:2393] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 8: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
I20260709 13:20:20.820174 16293 raft_consensus.cc:3060] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:20:20.661316 19345 consensus_queue.cc:1059] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.100s
I20260709 13:20:20.844760 16116 leader_election.cc:304] T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
I20260709 13:20:20.855014 16220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:20.855014 16291 raft_consensus.cc:2468] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 6.
W20260709 13:20:20.860543 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 271612us
W20260709 13:20:20.862879 16115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 144338us
W20260709 13:20:20.863436 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 189170us
I20260709 13:20:20.881314 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6f07842d3aa47858df5665874c86f3a" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:20:20.881969 16224 raft_consensus.cc:3060] T b6f07842d3aa47858df5665874c86f3a P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Advancing to term 9
I20260709 13:20:20.810937 19530 raft_consensus.cc:493] T 18ad7e3af5504571a69d10c00750a01c P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:20.869203 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdbd78acb4cf46298e0de8ba0d130e19" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:20.872326 16148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cc959fb7f704fba9009c0595467d074" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
W20260709 13:20:20.867313 19404 consensus_meta.cc:287] T fffab26427e24d398e2695aad40ca134 P 647a343a542d4a139fe106c19095129b: Time spent flushing consensus metadata: real 0.510s	user 0.007s	sys 0.003s
I20260709 13:20:20.886795 16294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c11074579b046d99e585c2818e18a2c" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:20:20.884018 16143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
W20260709 13:20:20.887549 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 125084us
I20260709 13:20:20.887755 18599 raft_consensus.cc:2804] T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:20:20.927114 18599 raft_consensus.cc:493] T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:20.891836 16150 raft_consensus.cc:1240] T 434bff3082d141838068f323bdb0d1ed P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 7. Current term is 8. Ops: []
I20260709 13:20:20.927580 18599 raft_consensus.cc:3060] T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:20:20.926396 16292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7ca61592e848ea9b0fb81a2ce33cdc" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:20.810166 19511 consensus_queue.cc:1048] T ecc1e363230043fe9fef390f0b979f20 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.124s
I20260709 13:20:20.931784 16300 raft_consensus.cc:2468] T cdbd78acb4cf46298e0de8ba0d130e19 P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 7.
I20260709 13:20:20.934240 16148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746aabf856ab401c9ef5e859371e7b67" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:20:20.948168 16220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ab0ac8347641e6955e9bce7d90c1a2" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:20.949364 16220 raft_consensus.cc:2410] T d2ab0ac8347641e6955e9bce7d90c1a2 P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 for term 8 because replica has last-logged OpId of term: 7 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 7.
I20260709 13:20:20.949784 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:20.950555 16294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6f07842d3aa47858df5665874c86f3a" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:20:20.951107 16294 raft_consensus.cc:3060] T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:20:20.856940 19068 raft_consensus.cc:515] T e5ad4eb281564936a59f41e387a47156 P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:20.877041 19425 raft_consensus.cc:493] T 8a878999c0024d2abe517484180290fc P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:20.879577 19414 raft_consensus.cc:2749] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:20:20.926683 19404 raft_consensus.cc:515] T fffab26427e24d398e2695aad40ca134 P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:20.955974 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c11074579b046d99e585c2818e18a2c" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:20.959089 16292 raft_consensus.cc:2468] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 9.
W20260709 13:20:20.964504 19537 log.cc:927] Time spent T 8cc959fb7f704fba9009c0595467d074 P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 0.177s	user 0.000s	sys 0.001s
W20260709 13:20:20.965619 19431 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.859s	user 0.000s	sys 0.037s
W20260709 13:20:20.966125 19431 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.981s	user 0.000s	sys 0.037s
W20260709 13:20:20.966300 19538 log.cc:927] Time spent T 1df93779209748b9860f6d893dd3de53 P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 0.122s	user 0.001s	sys 0.000s
I20260709 13:20:20.968716 16298 raft_consensus.cc:1240] T e5ad4eb281564936a59f41e387a47156 P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 8. Current term is 9. Ops: []
I20260709 13:20:20.973789 16220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746aabf856ab401c9ef5e859371e7b67" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:20:20.955773 16148 raft_consensus.cc:2393] T 746aabf856ab401c9ef5e859371e7b67 P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 in current term 8: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
I20260709 13:20:20.935207 19419 raft_consensus.cc:493] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:20:20.888312 19465 consensus_queue.cc:1048] T d2ab0ac8347641e6955e9bce7d90c1a2 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.074s
I20260709 13:20:20.884526 19496 consensus_queue.cc:1048] T 23f82224f4564fea93572e8206bad1fe P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.074s
I20260709 13:20:20.991987 18599 raft_consensus.cc:515] T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:20.989773 16115 leader_election.cc:304] T 9e65676b21b748728ad39668cfe722ec P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:20:20.932636 19148 consensus_queue.cc:1059] T 57a744f5aaf944678a63469cbf514015 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.031s
I20260709 13:20:20.930832 19540 raft_consensus.cc:515] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:21.003262 16300 raft_consensus.cc:2393] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 in current term 8: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
I20260709 13:20:20.998301 16147 raft_consensus.cc:1240] T fffab26427e24d398e2695aad40ca134 P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 6. Current term is 7. Ops: [6.4-6.4]
I20260709 13:20:20.961933 19425 raft_consensus.cc:3060] T 8a878999c0024d2abe517484180290fc P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:20:20.999249 16115 leader_election.cc:304] T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac
W20260709 13:20:20.994794 16789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 65670.5us
I20260709 13:20:21.006219 18599 leader_election.cc:290] T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 9 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:21.006453 16143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a878999c0024d2abe517484180290fc" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:20.951990 19530 raft_consensus.cc:515] T 18ad7e3af5504571a69d10c00750a01c P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } }
W20260709 13:20:21.000677 19510 consensus_meta.cc:287] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac: Time spent flushing consensus metadata: real 0.687s	user 0.012s	sys 0.000s
I20260709 13:20:21.018556 16220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af57996a8be84d398f0f023e8d08ef77" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:21.016047 16148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f5f89186204242bf377f7c42b2e2da" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:21.019941 16789 leader_election.cc:304] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac
I20260709 13:20:21.000718 19535 raft_consensus.cc:493] T 1df93779209748b9860f6d893dd3de53 P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:21.028110 16148 raft_consensus.cc:2410] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 for term 10 because replica has last-logged OpId of term: 9 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
I20260709 13:20:21.032716 19530 leader_election.cc:290] T 18ad7e3af5504571a69d10c00750a01c P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:21.009033 19532 consensus_queue.cc:1048] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 2, Time since last communication: 0.025s
I20260709 13:20:21.034997 19532 raft_consensus.cc:2804] T 9e65676b21b748728ad39668cfe722ec P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:20:21.035454 19532 raft_consensus.cc:493] T 9e65676b21b748728ad39668cfe722ec P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:21.035872 19532 raft_consensus.cc:3060] T 9e65676b21b748728ad39668cfe722ec P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:20:21.046819 16297 consensus_meta.cc:287] T e4dc6b0541ca431ebc757beac00521f1 P 90f3da57796b488c9087a2f523adf045: Time spent flushing consensus metadata: real 0.679s	user 0.005s	sys 0.003s
I20260709 13:20:21.047426 16297 raft_consensus.cc:2468] T e4dc6b0541ca431ebc757beac00521f1 P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 7.
I20260709 13:20:21.039670 16143 raft_consensus.cc:2468] T 8a878999c0024d2abe517484180290fc P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 7.
I20260709 13:20:21.057300 16293 raft_consensus.cc:2468] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 8.
I20260709 13:20:21.058622 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434bff3082d141838068f323bdb0d1ed" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:21.060105 16294 raft_consensus.cc:2468] T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 9.
W20260709 13:20:21.060971 16114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:21.049268 15046 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:20:21.061777 16294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9944d5e4b8c4109af9955c4f08c9095" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:21.064544 16297 raft_consensus.cc:2468] T 434bff3082d141838068f323bdb0d1ed P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 8.
I20260709 13:20:20.997239 19419 raft_consensus.cc:515] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:21.078390 16143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806d7690c2b644959152074bb2cab0a7" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:21.078114 16294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31969392ccdb43b491911a319e8bc7e1" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:21.080469 16143 raft_consensus.cc:2468] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 6.
I20260709 13:20:21.083392 16294 raft_consensus.cc:2410] T 31969392ccdb43b491911a319e8bc7e1 P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 for term 8 because replica has last-logged OpId of term: 7 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 6.
I20260709 13:20:21.120024 16296 raft_consensus.cc:1275] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:20:21.125465 16224 raft_consensus.cc:2468] T b6f07842d3aa47858df5665874c86f3a P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 9.
I20260709 13:20:21.066578 19197 raft_consensus.cc:2804] T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:20:21.126984 16144 raft_consensus.cc:2468] T 08c74490c40243a88e62d1760b5cd814 P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 7.
I20260709 13:20:21.130635 16143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee9ed67006144797b1de6ea8c2013a68" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
W20260709 13:20:21.133407 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 66996.3us
I20260709 13:20:21.136066 16790 leader_election.cc:304] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:20:21.143361 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:21.081310 19559 raft_consensus.cc:3055] T 57a744f5aaf944678a63469cbf514015 P cb82cc7bd6db46d980c75237a1402860 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:20:21.069759 19148 consensus_queue.cc:1059] T 23f82224f4564fea93572e8206bad1fe P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.063s
I20260709 13:20:21.153581 16144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
W20260709 13:20:21.128918 19480 consensus_meta.cc:287] T 84f2da98864b4425bf7c0055951fdd4b P 1ca2725d290e47d68b1a4b609c9816ac: Time spent flushing consensus metadata: real 0.709s	user 0.007s	sys 0.004s
I20260709 13:20:21.062487 19535 raft_consensus.cc:515] T 1df93779209748b9860f6d893dd3de53 P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:21.122654 19531 leader_election.cc:290] T 434bff3082d141838068f323bdb0d1ed P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 8 election: Requested vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
W20260709 13:20:21.137732 19213 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.884s	user 0.009s	sys 0.019s
I20260709 13:20:21.149994 19559 raft_consensus.cc:740] T 57a744f5aaf944678a63469cbf514015 P cb82cc7bd6db46d980c75237a1402860 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:20:21.132789 19197 raft_consensus.cc:493] T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:21.191946 19524 raft_consensus.cc:493] T 8a878999c0024d2abe517484180290fc P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:21.195472 19210 consensus_queue.cc:1059] T 829b4ad8c5a54eababeea1181b960d4b P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: INVALID_TERM, Last received: 6.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:20:21.195812 19068 leader_election.cc:290] T e5ad4eb281564936a59f41e387a47156 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 9 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:20:21.185469 19496 raft_consensus.cc:2804] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:20:21.197870 16144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2372981006d4b218215c28964dea623" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:21.225129 16144 raft_consensus.cc:2368] T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 for earlier term 8. Current term is 9.
I20260709 13:20:21.188766 16225 raft_consensus.cc:2468] T 84f2da98864b4425bf7c0055951fdd4b P 1ca2725d290e47d68b1a4b609c9816ac [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 6.
I20260709 13:20:21.159931 19418 leader_election.cc:290] T 18ad7e3af5504571a69d10c00750a01c P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
W20260709 13:20:21.199314 19213 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.946s	user 0.009s	sys 0.019s
I20260709 13:20:21.197229 19509 raft_consensus.cc:3055] T 746aabf856ab401c9ef5e859371e7b67 P 1ca2725d290e47d68b1a4b609c9816ac [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:20:21.224242 16117 leader_election.cc:304] T 829b4ad8c5a54eababeea1181b960d4b P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860
I20260709 13:20:21.204162 16151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62fe3346cd3c42c4a55e14af9073bca4" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:20:21.154165 19530 consensus_queue.cc:1048] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.058s
I20260709 13:20:21.229890 16151 raft_consensus.cc:3060] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:20:21.224476 16143 raft_consensus.cc:2468] T ee9ed67006144797b1de6ea8c2013a68 P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 5.
I20260709 13:20:21.204798 19080 leader_election.cc:290] T 57a744f5aaf944678a63469cbf514015 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 6 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:21.207305 19197 raft_consensus.cc:3060] T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:20:21.268617 16265 leader_election.cc:304] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac
W20260709 13:20:21.269783 16117 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 46558.7us
I20260709 13:20:21.276531 16117 leader_election.cc:304] T 746aabf856ab401c9ef5e859371e7b67 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860
W20260709 13:20:21.273135 16789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 267403us
W20260709 13:20:21.274436 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 138487us
I20260709 13:20:21.274698 16144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2794af5b99984efebdee564035841cb0" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:21.277885 16789 leader_election.cc:304] T 3a7ca61592e848ea9b0fb81a2ce33cdc P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:20:21.280494 16294 raft_consensus.cc:1240] T 23f82224f4564fea93572e8206bad1fe P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 6. Current term is 7. Ops: [4.5-6.6]
I20260709 13:20:21.276790 16265 leader_election.cc:304] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:20:21.227871 19465 raft_consensus.cc:2804] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:20:21.224783 19496 raft_consensus.cc:493] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:20:21.294446 16790 leader_election.cc:400] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 8 pre-election: Vote denied by peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741) with higher term. Message: Invalid argument: T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 for earlier term 8. Current term is 9.
I20260709 13:20:21.294873 16790 leader_election.cc:403] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 8 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:20:21.294853 16143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e9b2bf7011f4ae688a79d7bb5a447b4" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
W20260709 13:20:21.275804 19519 consensus_meta.cc:287] T 08c74490c40243a88e62d1760b5cd814 P 1ca2725d290e47d68b1a4b609c9816ac: Time spent flushing consensus metadata: real 0.905s	user 0.008s	sys 0.004s
I20260709 13:20:21.296320 16143 raft_consensus.cc:2468] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 5.
I20260709 13:20:21.336555 16294 raft_consensus.cc:1240] T 57a744f5aaf944678a63469cbf514015 P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 5. Current term is 6. Ops: [5.7-5.7]
I20260709 13:20:21.236996 19509 raft_consensus.cc:740] T 746aabf856ab401c9ef5e859371e7b67 P 1ca2725d290e47d68b1a4b609c9816ac [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
W20260709 13:20:21.293448 16115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 294292us
I20260709 13:20:21.323079 16265 leader_election.cc:304] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
W20260709 13:20:21.328835 16117 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 52375.5us
I20260709 13:20:21.292483 19080 raft_consensus.cc:2804] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:20:21.281626 19514 raft_consensus.cc:493] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:21.340728 16115 leader_election.cc:304] T 23f82224f4564fea93572e8206bad1fe P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: cb82cc7bd6db46d980c75237a1402860
I20260709 13:20:21.220860 19524 raft_consensus.cc:515] T 8a878999c0024d2abe517484180290fc P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:21.273949 19345 consensus_queue.cc:1059] T 7c2095a5f4ad49408dbc7f69c606120d P 90f3da57796b488c9087a2f523adf045 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.083s
I20260709 13:20:21.342979 16117 leader_election.cc:304] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:20:21.367091 16823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434bff3082d141838068f323bdb0d1ed" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
W20260709 13:20:21.364789 19522 consensus_meta.cc:287] T af57996a8be84d398f0f023e8d08ef77 P 647a343a542d4a139fe106c19095129b: Time spent flushing consensus metadata: real 0.504s	user 0.009s	sys 0.005s
I20260709 13:20:21.290527 19210 raft_consensus.cc:493] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:21.365127 16144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e65676b21b748728ad39668cfe722ec" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:21.370257 16825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2372981006d4b218215c28964dea623" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:20:21.296056 19465 raft_consensus.cc:493] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:20:21.339644 19213 raft_consensus.cc:2749] T 746aabf856ab401c9ef5e859371e7b67 P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:20:21.370384 19551 raft_consensus.cc:493] T 66f75124215f4ca6bef831e174a792d8 P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:21.297957 19496 raft_consensus.cc:3060] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:20:21.339983 16296 raft_consensus.cc:3060] T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:20:21.379920 16143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57a744f5aaf944678a63469cbf514015" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:20:21.376770 19514 raft_consensus.cc:515] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:21.428618 16143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434bff3082d141838068f323bdb0d1ed" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:21.427136 19551 raft_consensus.cc:515] T 66f75124215f4ca6bef831e174a792d8 P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:21.430022 16143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31969392ccdb43b491911a319e8bc7e1" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:21.383870 16146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31969392ccdb43b491911a319e8bc7e1" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:21.432447 16146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a959dbecc2648eb85d1cc550f5a3245" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:21.341410 19068 raft_consensus.cc:2804] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:20:21.346326 19080 raft_consensus.cc:493] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:20:21.388572 19540 leader_election.cc:290] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:21.422493 16825 raft_consensus.cc:3060] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 [term 7 FOLLOWER]: Advancing to term 9
I20260709 13:20:21.434356 19080 raft_consensus.cc:3060] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:20:21.433698 16146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0eab25ce4c64e018a041beeda2867db" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:21.436056 16143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af57996a8be84d398f0f023e8d08ef77" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:21.437951 16296 raft_consensus.cc:1275] T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:20:21.438901 16146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2372981006d4b218215c28964dea623" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:21.442842 16825 raft_consensus.cc:2468] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 9.
I20260709 13:20:21.443722 16146 raft_consensus.cc:2393] T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 9: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
I20260709 13:20:21.417806 19359 raft_consensus.cc:2749] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:20:21.446394 19465 raft_consensus.cc:3060] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:20:21.487926 16117 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 145012us
W20260709 13:20:21.488076 16115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 147446us
I20260709 13:20:21.482837 19345 raft_consensus.cc:493] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:21.481808 19419 leader_election.cc:290] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
W20260709 13:20:21.504370 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 194888us
I20260709 13:20:21.504201 16115 leader_election.cc:304] T 5a96373bb0e14dce8a266b2fc6c58507 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac
W20260709 13:20:21.504254 16189 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.619s	user 0.032s	sys 0.051s
W20260709 13:20:21.518391 16189 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.633s	user 0.033s	sys 0.051s
I20260709 13:20:21.491824 19418 raft_consensus.cc:2804] T 23f82224f4564fea93572e8206bad1fe P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:20:21.448423 19545 consensus_queue.cc:1048] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.300s
I20260709 13:20:21.447592 19522 raft_consensus.cc:515] T af57996a8be84d398f0f023e8d08ef77 P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:21.521453 16116 leader_election.cc:304] T b6f07842d3aa47858df5665874c86f3a P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
I20260709 13:20:21.474735 19210 raft_consensus.cc:515] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:20:21.544409 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 266139us
I20260709 13:20:21.503561 19213 raft_consensus.cc:2804] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:20:21.504122 19576 raft_consensus.cc:3055] T 829b4ad8c5a54eababeea1181b960d4b P cb82cc7bd6db46d980c75237a1402860 [term 7 LEADER]: Stepping down as leader of term 7
W20260709 13:20:21.540050 19550 log.cc:927] Time spent T fffab26427e24d398e2695aad40ca134 P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 0.339s	user 0.001s	sys 0.001s
W20260709 13:20:21.549436 16117 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 57903.5us
W20260709 13:20:21.548259 16818 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.600s	user 0.013s	sys 0.038s
W20260709 13:20:21.557938 16818 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.621s	user 0.014s	sys 0.038s
I20260709 13:20:21.541213 19512 raft_consensus.cc:493] T e5ad4eb281564936a59f41e387a47156 P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:20:21.539477 19571 raft_consensus.cc:493] T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:21.511201 19535 leader_election.cc:290] T 1df93779209748b9860f6d893dd3de53 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:21.559638 16143 raft_consensus.cc:2393] T af57996a8be84d398f0f023e8d08ef77 P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 9: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
I20260709 13:20:21.364086 19559 consensus_queue.cc:260] T 57a744f5aaf944678a63469cbf514015 P cb82cc7bd6db46d980c75237a1402860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:21.551589 19580 raft_consensus.cc:3055] T 7c2095a5f4ad49408dbc7f69c606120d P 90f3da57796b488c9087a2f523adf045 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:20:21.577907 16816 raft_consensus.cc:1275] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860 [term 9 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:20:21.552516 19213 raft_consensus.cc:493] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:20:21.563328 19421 consensus_meta.cc:287] T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b: Time spent flushing consensus metadata: real 1.130s	user 0.013s	sys 0.000s
I20260709 13:20:21.548967 19566 raft_consensus.cc:2749] T 829b4ad8c5a54eababeea1181b960d4b P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
W20260709 13:20:21.585508 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 64181.8us
I20260709 13:20:21.586298 19580 raft_consensus.cc:740] T 7c2095a5f4ad49408dbc7f69c606120d P 90f3da57796b488c9087a2f523adf045 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:20:21.615357 19210 leader_election.cc:290] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:21.616103 19580 consensus_queue.cc:260] T 7c2095a5f4ad49408dbc7f69c606120d P 90f3da57796b488c9087a2f523adf045 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:21.617467 19580 raft_consensus.cc:3060] T 7c2095a5f4ad49408dbc7f69c606120d P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:20:21.617491 19210 raft_consensus.cc:493] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P cb82cc7bd6db46d980c75237a1402860 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
W20260709 13:20:21.619462 19553 log.cc:927] Time spent T 8158bbd17b994d2eb96f560bd0f1b88d P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 0.504s	user 0.000s	sys 0.003s
I20260709 13:20:21.607725 19578 raft_consensus.cc:2804] T 3a7ca61592e848ea9b0fb81a2ce33cdc P cb82cc7bd6db46d980c75237a1402860 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:20:21.620038 19578 raft_consensus.cc:493] T 3a7ca61592e848ea9b0fb81a2ce33cdc P cb82cc7bd6db46d980c75237a1402860 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:20:21.583280 19418 raft_consensus.cc:697] T 23f82224f4564fea93572e8206bad1fe P 647a343a542d4a139fe106c19095129b [term 7 LEADER]: Becoming Leader. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:20:21.619936 19210 raft_consensus.cc:515] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P cb82cc7bd6db46d980c75237a1402860 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:21.620504 19578 raft_consensus.cc:3060] T 3a7ca61592e848ea9b0fb81a2ce33cdc P cb82cc7bd6db46d980c75237a1402860 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:20:21.621273 19418 consensus_queue.cc:237] T 23f82224f4564fea93572e8206bad1fe P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 5, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:21.622574 19210 leader_election.cc:290] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:21.623072 16823 raft_consensus.cc:2468] T 434bff3082d141838068f323bdb0d1ed P cb82cc7bd6db46d980c75237a1402860 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 8.
I20260709 13:20:21.499109 19509 consensus_queue.cc:260] T 746aabf856ab401c9ef5e859371e7b67 P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:20:21.592473 19558 log.cc:927] Time spent T 26354361e9b044cea1c2b86c4882fe92 P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 0.400s	user 0.000s	sys 0.001s
I20260709 13:20:21.623772 16151 raft_consensus.cc:2410] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 for term 10 because replica has last-logged OpId of term: 9 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 8.
I20260709 13:20:21.593950 19522 leader_election.cc:290] T af57996a8be84d398f0f023e8d08ef77 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 9 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:21.554782 19576 raft_consensus.cc:740] T 829b4ad8c5a54eababeea1181b960d4b P cb82cc7bd6db46d980c75237a1402860 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:20:21.624068 19509 raft_consensus.cc:3060] T 746aabf856ab401c9ef5e859371e7b67 P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:20:21.625037 19576 consensus_queue.cc:260] T 829b4ad8c5a54eababeea1181b960d4b P cb82cc7bd6db46d980c75237a1402860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.9, Last appended by leader: 9, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:21.625859 19585 raft_consensus.cc:2749] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 [term 9 FOLLOWER]: Leader pre-election lost for term 8. Reason: Vote denied by peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741) with higher term. Message: Invalid argument: T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 for earlier term 8. Current term is 9.
W20260709 13:20:21.595438 19561 log.cc:927] Time spent T 806d7690c2b644959152074bb2cab0a7 P cb82cc7bd6db46d980c75237a1402860: Append to log took a long time: real 0.249s	user 0.000s	sys 0.001s
I20260709 13:20:21.596125 19524 leader_election.cc:290] T 8a878999c0024d2abe517484180290fc P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:21.583930 19080 raft_consensus.cc:515] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } }
I20260709 13:20:21.542223 19345 raft_consensus.cc:515] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:21.599308 19571 raft_consensus.cc:515] T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:21.600930 16217 raft_consensus.cc:2468] T 08c74490c40243a88e62d1760b5cd814 P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 7.
I20260709 13:20:21.601547 19575 raft_consensus.cc:2804] T b6f07842d3aa47858df5665874c86f3a P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Leader election won for term 9
W20260709 13:20:21.602029 16117 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 44351.6us
I20260709 13:20:21.634856 16117 leader_election.cc:304] T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:20:21.603583 16219 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57600 (request call id 1263) took 1286 ms. Trace:
I20260709 13:20:21.600989 19560 raft_consensus.cc:2804] T 5a96373bb0e14dce8a266b2fc6c58507 P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Leader election won for term 7
W20260709 13:20:21.605971 19425 consensus_meta.cc:287] T 8a878999c0024d2abe517484180290fc P 90f3da57796b488c9087a2f523adf045: Time spent flushing consensus metadata: real 0.577s	user 0.005s	sys 0.002s
I20260709 13:20:21.607818 19142 raft_consensus.cc:493] T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:20:21.649475 19142 raft_consensus.cc:515] T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:21.651604 19142 leader_election.cc:290] T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:21.651760 19575 raft_consensus.cc:697] T b6f07842d3aa47858df5665874c86f3a P 647a343a542d4a139fe106c19095129b [term 9 LEADER]: Becoming Leader. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:20:21.603000 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62fe3346cd3c42c4a55e14af9073bca4" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:20:21.612792 16149 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40844 (request call id 2101) took 3735 ms. Trace:
W20260709 13:20:21.619544 19564 log.cc:927] Time spent T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 0.230s	user 0.001s	sys 0.000s
W20260709 13:20:21.620473 18901 consensus_meta.cc:287] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac: Time spent flushing consensus metadata: real 0.737s	user 0.010s	sys 0.000s
W20260709 13:20:21.599627 16115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 95491.7us
W20260709 13:20:21.663447 16145 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.604s	user 0.014s	sys 0.016s
I20260709 13:20:21.640568 16117 leader_election.cc:304] T 434bff3082d141838068f323bdb0d1ed P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860; no voters: 
W20260709 13:20:21.663895 16145 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.623s	user 0.015s	sys 0.017s
I20260709 13:20:21.665307 16145 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46224 (request call id 458) took 1141 ms. Trace:
I20260709 13:20:21.626150 19576 raft_consensus.cc:3060] T 829b4ad8c5a54eababeea1181b960d4b P cb82cc7bd6db46d980c75237a1402860 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:20:21.613654 19213 raft_consensus.cc:3060] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:20:21.635772 16219 rpcz_store.cc:276] 0709 13:20:20.316735 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:20.318748 (+  2013us) service_pool.cc:224] Handling call
0709 13:20:20.319075 (+   327us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:20:20.407260 (+ 88185us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:20.407295 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:20:20.407896 (+   601us) log.cc:844] Serialized 26 byte log entry
0709 13:20:20.784127 (+376231us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:20:20.784198 (+    71us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:20.784299 (+   101us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:20:21.603352 (+819053us) raft_consensus.cc:1685] finished
0709 13:20:21.603374 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:21.603516 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":77928,"raft.queue_time_us":88957,"raft.run_cpu_time_us":138,"raft.run_wall_time_us":6095,"thread_start_us":375790,"threads_started":1,"wal-append.queue_time_us":457285}
I20260709 13:20:21.670468 16115 leader_election.cc:304] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860
I20260709 13:20:21.671252 16115 leader_election.cc:304] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac
I20260709 13:20:21.644408 19583 raft_consensus.cc:493] T ada5a004ba7c46819051e3ccec71ffdb P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
W20260709 13:20:21.644589 19557 log.cc:927] Time spent T 02838c86531d4f98bf58d1c22b8baf77 P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 0.289s	user 0.001s	sys 0.001s
W20260709 13:20:21.644420 19577 log.cc:927] Time spent T 26354361e9b044cea1c2b86c4882fe92 P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.118s	user 0.001s	sys 0.000s
I20260709 13:20:21.645159 19425 raft_consensus.cc:515] T 8a878999c0024d2abe517484180290fc P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:21.646939 19345 leader_election.cc:290] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:21.647786 16296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a9de347c711477e87c582d003f36925" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 10 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:21.674815 16296 raft_consensus.cc:2468] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 9.
I20260709 13:20:21.649750 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e4bfb9090c8460eb14fa7e0d818e7f1" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:21.652966 16224 raft_consensus.cc:2410] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 for term 10 because replica has last-logged OpId of term: 9 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 8.
I20260709 13:20:21.550778 19068 raft_consensus.cc:697] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045 [term 7 LEADER]: Becoming Leader. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
W20260709 13:20:21.654168 16191 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:21.652621 16149 rpcz_store.cc:276] 0709 13:20:17.877473 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:19.677776 (+1800303us) service_pool.cc:224] Handling call
0709 13:20:19.678190 (+   414us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:19.971929 (+293739us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:21.612520 (+1640591us) spinlock_profiling.cc:243] Waited 222 ms on lock 0x7b54000735d4. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:20:21.612725 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":509725952}
I20260709 13:20:21.662194 19575 consensus_queue.cc:237] T b6f07842d3aa47858df5665874c86f3a P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:21.668748 18901 raft_consensus.cc:515] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:21.665814 16145 rpcz_store.cc:276] 0709 13:20:20.523891 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:20.559276 (+ 35385us) service_pool.cc:224] Handling call
0709 13:20:20.574869 (+ 15593us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:20.577123 (+  2254us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:20:20.828354 (+251231us) log.cc:844] Serialized 14 byte log entry
0709 13:20:21.664530 (+836176us) log.cc:844] Serialized 14 byte log entry
0709 13:20:21.664843 (+   313us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:20:21.664907 (+    64us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:20:21.664969 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:21.665060 (+    91us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:21.665229 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":43258,"prepare.queue_time_us":83563,"prepare.run_cpu_time_us":82,"prepare.run_wall_time_us":13197,"raft.queue_time_us":194835,"raft.run_cpu_time_us":63,"raft.run_wall_time_us":71,"thread_start_us":977640,"threads_started":3,"wal-append.queue_time_us":215724}
I20260709 13:20:21.609678 19512 raft_consensus.cc:515] T e5ad4eb281564936a59f41e387a47156 P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:21.645632 19080 leader_election.cc:290] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 7 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
W20260709 13:20:21.713932 16789 leader_election.cc:336] T 8c11074579b046d99e585c2818e18a2c P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389): Timed out: RequestConsensusVote RPC to 127.14.177.131:41389 timed out after 1.550s (SENT)
W20260709 13:20:21.962046 16115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 290869us
I20260709 13:20:21.652810 19522 raft_consensus.cc:2804] T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:20:21.721484 16219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68a6838d82ef4cac83507f906139d90f" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
W20260709 13:20:21.970563 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 300168us
W20260709 13:20:21.713398 16221 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57600 (request call id 1255) took 1410 ms (client timeout 1604 ms). Trace:
I20260709 13:20:21.971800 16790 leader_election.cc:304] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:20:21.895566 16818 raft_consensus.cc:1275] T b87147731a68400e8cce9b879daa7c5d P cb82cc7bd6db46d980c75237a1402860 [term 9 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 13:20:21.718602 16143 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40844 (request call id 2118) took 1002 ms. Trace:
I20260709 13:20:21.658910 19560 raft_consensus.cc:697] T 5a96373bb0e14dce8a266b2fc6c58507 P 647a343a542d4a139fe106c19095129b [term 7 LEADER]: Becoming Leader. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
W20260709 13:20:21.974680 16790 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:21.977435 16265 leader_election.cc:304] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: cb82cc7bd6db46d980c75237a1402860
W20260709 13:20:21.713636 19478 consensus_meta.cc:287] T af57996a8be84d398f0f023e8d08ef77 P 1ca2725d290e47d68b1a4b609c9816ac: Time spent flushing consensus metadata: real 0.978s	user 0.011s	sys 0.000s
I20260709 13:20:21.670169 19524 raft_consensus.cc:2804] T 434bff3082d141838068f323bdb0d1ed P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Leader election won for term 8
W20260709 13:20:21.712065 16226 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57604 (request call id 2277) took 2045 ms (client timeout 1812 ms). Trace:
I20260709 13:20:21.714741 16295 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48674 (request call id 2368) took 1275 ms. Trace:
I20260709 13:20:21.988283 16295 rpcz_store.cc:276] 0709 13:20:20.439746 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:20.439917 (+   171us) service_pool.cc:224] Handling call
0709 13:20:21.714549 (+1274632us) spinlock_profiling.cc:243] Waited 53.1 ms on lock 0x7b54000aa354. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:20:21.714685 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":122142080}
W20260709 13:20:21.972115 16221 rpcz_store.cc:269] 0709 13:20:20.302975 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:20.303098 (+   123us) service_pool.cc:224] Handling call
0709 13:20:21.713207 (+1410109us) spinlock_profiling.cc:243] Waited 66.2 ms on lock 0x7b5400245c54. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:20:21.713332 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":152275328}
I20260709 13:20:21.975669 16143 rpcz_store.cc:276] 0709 13:20:20.715832 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:21.431017 (+715185us) service_pool.cc:224] Handling call
0709 13:20:21.718418 (+287401us) spinlock_profiling.cc:243] Waited 119 ms on lock 0x7b54000e9a54. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:20:21.718545 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":273619712}
I20260709 13:20:21.989629 16219 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57604 (request call id 2285) took 1340 ms. Trace:
I20260709 13:20:21.721794 19421 raft_consensus.cc:3060] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:20:21.990109 16219 rpcz_store.cc:276] 0709 13:20:20.643006 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:21.673831 (+1030825us) service_pool.cc:224] Handling call
0709 13:20:21.983045 (+309214us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:20:21.713825 16223 raft_consensus.cc:1240] T 7c2095a5f4ad49408dbc7f69c606120d P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 6. Current term is 7. Ops: []
I20260709 13:20:21.978752 16790 leader_election.cc:304] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:20:21.982069 19568 raft_consensus.cc:2749] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
W20260709 13:20:21.688791 19532 consensus_meta.cc:287] T 9e65676b21b748728ad39668cfe722ec P 647a343a542d4a139fe106c19095129b: Time spent flushing consensus metadata: real 0.652s	user 0.011s	sys 0.000s
W20260709 13:20:21.976686 19416 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.994s	user 0.006s	sys 0.006s
W20260709 13:20:21.986814 16789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 272902us
W20260709 13:20:21.709610 19562 log.cc:927] Time spent T d16dfad0c3b540e5b4b86492ddf87208 P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 0.332s	user 0.000s	sys 0.000s
I20260709 13:20:21.699669 19390 raft_consensus.cc:3055] T e4dc6b0541ca431ebc757beac00521f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 6 LEADER]: Stepping down as leader of term 6
W20260709 13:20:21.920428 19148 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.721s	user 0.000s	sys 0.014s
W20260709 13:20:21.694829 19523 log.cc:927] Time spent T 8638ee363273426db1393dd81e5c7b72 P cb82cc7bd6db46d980c75237a1402860: Append to log took a long time: real 1.184s	user 0.004s	sys 0.000s
I20260709 13:20:21.989349 19345 raft_consensus.cc:2804] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:20:21.992425 16221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a959dbecc2648eb85d1cc550f5a3245" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:22.039601 16221 raft_consensus.cc:2468] T 5a959dbecc2648eb85d1cc550f5a3245 P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 5.
I20260709 13:20:22.040163 16221 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57600 (request call id 1267) took 1345 ms. Trace:
I20260709 13:20:21.685892 19197 raft_consensus.cc:515] T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:21.992542 16145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5ad4eb281564936a59f41e387a47156" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:20:22.038836 16191 leader_election.cc:304] T ee9ed67006144797b1de6ea8c2013a68 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: cb82cc7bd6db46d980c75237a1402860
I20260709 13:20:21.993278 16143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806d7690c2b644959152074bb2cab0a7" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
W20260709 13:20:21.990195 16226 rpcz_store.cc:269] 0709 13:20:19.667036 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:19.678600 (+ 11564us) service_pool.cc:224] Handling call
0709 13:20:21.711858 (+2033258us) spinlock_profiling.cc:243] Waited 194 ms on lock 0x7b54002c8154. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:20:21.711976 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":445377408}
I20260709 13:20:21.961174 19578 raft_consensus.cc:515] T 3a7ca61592e848ea9b0fb81a2ce33cdc P cb82cc7bd6db46d980c75237a1402860 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:21.999518 19554 raft_consensus.cc:493] T a378d7f796f94615b3b347119a9880fd P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:22.000121 19414 raft_consensus.cc:493] T 8cc959fb7f704fba9009c0595467d074 P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:22.000044 16219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:21.999614 19213 raft_consensus.cc:515] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:20:21.999543 16263 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:22.061574 16219 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57604 (request call id 2288) took 1417 ms. Trace:
I20260709 13:20:21.680008 19551 leader_election.cc:290] T 66f75124215f4ca6bef831e174a792d8 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:21.861994 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5718c623ebee4d2bb770f0f9ddb2f164" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:21.881114 19450 raft_consensus.cc:2804] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:20:21.700107 19382 raft_consensus.cc:3055] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:20:21.717852 19559 raft_consensus.cc:3060] T 57a744f5aaf944678a63469cbf514015 P cb82cc7bd6db46d980c75237a1402860 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:20:21.983750 19522 raft_consensus.cc:697] T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b [term 9 LEADER]: Becoming Leader. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:20:22.017638 16789 leader_election.cc:304] T 434bff3082d141838068f323bdb0d1ed P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 647a343a542d4a139fe106c19095129b
W20260709 13:20:22.102506 16789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 84972.9us
I20260709 13:20:22.103477 16789 leader_election.cc:304] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:20:22.020785 16822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af57996a8be84d398f0f023e8d08ef77" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:20:22.104420 16822 raft_consensus.cc:3060] T af57996a8be84d398f0f023e8d08ef77 P cb82cc7bd6db46d980c75237a1402860 [term 7 FOLLOWER]: Advancing to term 9
I20260709 13:20:21.696609 19514 leader_election.cc:290] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:21.715873 19407 leader_election.cc:290] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:21.705982 19496 raft_consensus.cc:515] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:21.989487 19478 raft_consensus.cc:515] T af57996a8be84d398f0f023e8d08ef77 P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:22.127667 16820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66f75124215f4ca6bef831e174a792d8" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:22.040517 16221 rpcz_store.cc:276] 0709 13:20:20.695235 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:21.992158 (+1296923us) service_pool.cc:224] Handling call
0709 13:20:22.040104 (+ 47946us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:20:21.984977 19580 raft_consensus.cc:493] T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:20:22.040325 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a9de347c711477e87c582d003f36925" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 10 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
W20260709 13:20:22.011674 19416 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 2.029s	user 0.006s	sys 0.009s
I20260709 13:20:22.057068 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9944d5e4b8c4109af9955c4f08c9095" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:22.057559 16818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62fe3346cd3c42c4a55e14af9073bca4" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:22.012728 19532 raft_consensus.cc:515] T 9e65676b21b748728ad39668cfe722ec P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:22.153573 16221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c11074579b046d99e585c2818e18a2c" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:20:22.154659 16221 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33050 (request call id 577) took 1217 ms. Trace:
I20260709 13:20:22.155084 16221 rpcz_store.cc:276] 0709 13:20:20.936957 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:22.133067 (+1196110us) service_pool.cc:224] Handling call
0709 13:20:22.154580 (+ 21513us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:20:22.155799 19585 consensus_queue.cc:1048] T 74ca8140f35c4ae29e8a664f6bdfc829 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:20:22.156221 16221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0eab25ce4c64e018a041beeda2867db" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:22.157210 16221 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57600 (request call id 1268) took 1274 ms. Trace:
I20260709 13:20:22.157546 16819 raft_consensus.cc:1240] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860 [term 8 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 7. Current term is 8. Ops: [7.8-7.8]
I20260709 13:20:22.157629 16221 rpcz_store.cc:276] 0709 13:20:20.883281 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:22.155882 (+1272601us) service_pool.cc:224] Handling call
0709 13:20:22.157143 (+  1261us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:20:22.158661 16819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2372981006d4b218215c28964dea623" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:22.159453 16144 raft_consensus.cc:2393] T 9e65676b21b748728ad39668cfe722ec P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 9: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
I20260709 13:20:22.158916 16221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af57996a8be84d398f0f023e8d08ef77" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:20:22.060680 19578 leader_election.cc:290] T 3a7ca61592e848ea9b0fb81a2ce33cdc P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 10 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:22.004881 19524 raft_consensus.cc:697] T 434bff3082d141838068f323bdb0d1ed P 647a343a542d4a139fe106c19095129b [term 8 LEADER]: Becoming Leader. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:20:21.702059 19465 raft_consensus.cc:515] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:22.061959 16219 rpcz_store.cc:276] 0709 13:20:20.643889 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:21.999763 (+1355874us) service_pool.cc:224] Handling call
0709 13:20:22.061519 (+ 61756us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:20:21.720170 19583 raft_consensus.cc:515] T ada5a004ba7c46819051e3ccec71ffdb P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:22.093400 19382 raft_consensus.cc:740] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:20:22.062282 16294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e9b2bf7011f4ae688a79d7bb5a447b4" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:21.973148 19589 raft_consensus.cc:493] T 02838c86531d4f98bf58d1c22b8baf77 P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:20:22.015887 19568 raft_consensus.cc:2749] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:20:22.040805 19345 raft_consensus.cc:493] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Starting leader election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:20:22.170801 16220 raft_consensus.cc:2368] T af57996a8be84d398f0f023e8d08ef77 P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 for earlier term 8. Current term is 9.
I20260709 13:20:22.065495 19364 raft_consensus.cc:2804] T ee9ed67006144797b1de6ea8c2013a68 P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:20:22.065644 16296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f5f89186204242bf377f7c42b2e2da" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:22.066476 19512 leader_election.cc:290] T e5ad4eb281564936a59f41e387a47156 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:22.038770 19390 raft_consensus.cc:740] T e4dc6b0541ca431ebc757beac00521f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:20:22.179385 19522 consensus_queue.cc:237] T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:22.180066 19390 consensus_queue.cc:260] T e4dc6b0541ca431ebc757beac00521f1 P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:22.181470 19390 raft_consensus.cc:3060] T e4dc6b0541ca431ebc757beac00521f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:20:22.068627 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 54225.1us
I20260709 13:20:22.073036 19414 raft_consensus.cc:515] T 8cc959fb7f704fba9009c0595467d074 P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:20:22.187777 19595 log.cc:927] Time spent T 23f82224f4564fea93572e8206bad1fe P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 0.064s	user 0.000s	sys 0.000s
I20260709 13:20:22.188813 19414 leader_election.cc:290] T 8cc959fb7f704fba9009c0595467d074 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:20:22.172634 19589 raft_consensus.cc:515] T 02838c86531d4f98bf58d1c22b8baf77 P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:22.074313 16295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434bff3082d141838068f323bdb0d1ed" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:20:22.192088 16295 raft_consensus.cc:2393] T 434bff3082d141838068f323bdb0d1ed P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 8: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
I20260709 13:20:22.193142 16295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a878999c0024d2abe517484180290fc" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 8 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:22.194062 16295 raft_consensus.cc:2393] T 8a878999c0024d2abe517484180290fc P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 8: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
W20260709 13:20:22.074918 19592 log.cc:927] Time spent T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 0.178s	user 0.001s	sys 0.000s
W20260709 13:20:22.195379 16788 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:22.077165 19213 leader_election.cc:290] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 8 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:22.155028 19532 leader_election.cc:290] T 9e65676b21b748728ad39668cfe722ec P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 9 election: Requested vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:22.040315 19529 consensus_queue.cc:1048] T b87147731a68400e8cce9b879daa7c5d P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.038s
I20260709 13:20:22.063802 19554 raft_consensus.cc:515] T a378d7f796f94615b3b347119a9880fd P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:22.198205 16297 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40474 (request call id 494) took 1328 ms. Trace:
I20260709 13:20:22.199313 19554 leader_election.cc:290] T a378d7f796f94615b3b347119a9880fd P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:22.198604 16297 rpcz_store.cc:276] 0709 13:20:20.870217 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:21.066058 (+195841us) service_pool.cc:224] Handling call
0709 13:20:21.088886 (+ 22828us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:20:21.128211 (+ 39325us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:21.128248 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:20:21.155871 (+ 27623us) log.cc:844] Serialized 26 byte log entry
0709 13:20:21.670144 (+514273us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:20:21.670191 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:21.670267 (+    76us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:20:22.197998 (+527731us) raft_consensus.cc:1685] finished
0709 13:20:22.198025 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:22.198150 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":91,"raft.run_cpu_time_us":137,"raft.run_wall_time_us":138,"thread_start_us":405084,"threads_started":1,"wal-append.queue_time_us":921362}
I20260709 13:20:22.079398 16300 raft_consensus.cc:1275] T b87147731a68400e8cce9b879daa7c5d P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 13:20:22.170926 19560 consensus_queue.cc:237] T 5a96373bb0e14dce8a266b2fc6c58507 P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.2, Last appended by leader: 2, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:20:22.079581 16148 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40830 (request call id 1276) took 1344 ms (client timeout 1697 ms). Trace:
W20260709 13:20:22.200484 16148 rpcz_store.cc:269] 0709 13:20:20.735141 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:20.990736 (+255595us) service_pool.cc:224] Handling call
0709 13:20:22.079326 (+1088590us) spinlock_profiling.cc:243] Waited 4.13 ms on lock 0x7b540018f954. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:20:22.079504 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":9489152}
I20260709 13:20:22.201524 19524 consensus_queue.cc:237] T 434bff3082d141838068f323bdb0d1ed P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:22.083341 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18ad7e3af5504571a69d10c00750a01c" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:22.083732 16145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a9de347c711477e87c582d003f36925" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 10 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:22.173784 19364 raft_consensus.cc:493] T ee9ed67006144797b1de6ea8c2013a68 P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Starting leader election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:20:22.204098 16145 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46224 (request call id 466) took 1195 ms. Trace:
I20260709 13:20:22.204629 16145 rpcz_store.cc:276] 0709 13:20:21.008304 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:22.203252 (+1194948us) service_pool.cc:224] Handling call
0709 13:20:22.203606 (+   354us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:22.203767 (+   161us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:22.203790 (+    23us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:20:22.203831 (+    41us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:20:22.203863 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:22.203920 (+    57us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:22.204022 (+   102us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:20:22.084328 19567 raft_consensus.cc:493] T 26354361e9b044cea1c2b86c4882fe92 P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:20:22.085749 19581 raft_consensus.cc:493] T 18ad7e3af5504571a69d10c00750a01c P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:22.205854 19567 raft_consensus.cc:515] T 26354361e9b044cea1c2b86c4882fe92 P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:22.205946 19581 raft_consensus.cc:515] T 18ad7e3af5504571a69d10c00750a01c P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } }
I20260709 13:20:22.086300 16821 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49906 (request call id 713) took 2445 ms. Trace:
I20260709 13:20:22.086416 19080 raft_consensus.cc:493] T 5a96373bb0e14dce8a266b2fc6c58507 P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:22.208032 19567 leader_election.cc:290] T 26354361e9b044cea1c2b86c4882fe92 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:22.208135 19581 leader_election.cc:290] T 18ad7e3af5504571a69d10c00750a01c P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:20:22.088447 16146 raft_consensus.cc:1240] T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b [term 10 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 9. Current term is 10. Ops: [9.7-9.7]
I20260709 13:20:22.208338 19080 raft_consensus.cc:515] T 5a96373bb0e14dce8a266b2fc6c58507 P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:22.085202 19425 leader_election.cc:290] T 8a878999c0024d2abe517484180290fc P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 8 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:20:22.048751 19596 raft_consensus.cc:493] T 2794af5b99984efebdee564035841cb0 P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:20:22.091543 16143 raft_consensus.cc:3060] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:20:22.097929 16192 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:22.130295 16789 leader_election.cc:304] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 
W20260709 13:20:22.131174 16790 leader_election.cc:336] T af57996a8be84d398f0f023e8d08ef77 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943): Timed out: RequestConsensusVote RPC to 127.14.177.130:35943 timed out after 1.546s (SENT)
I20260709 13:20:22.134536 16820 raft_consensus.cc:2410] T 66f75124215f4ca6bef831e174a792d8 P cb82cc7bd6db46d980c75237a1402860 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac for term 9 because replica has last-logged OpId of term: 7 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
I20260709 13:20:22.134903 16825 raft_consensus.cc:1240] T 3a7ca61592e848ea9b0fb81a2ce33cdc P cb82cc7bd6db46d980c75237a1402860 [term 10 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 9. Current term is 10. Ops: []
I20260709 13:20:22.211412 16790 leader_election.cc:304] T af57996a8be84d398f0f023e8d08ef77 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:20:22.211817 16822 raft_consensus.cc:2468] T af57996a8be84d398f0f023e8d08ef77 P cb82cc7bd6db46d980c75237a1402860 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 9.
I20260709 13:20:22.152516 16226 raft_consensus.cc:2468] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 8.
I20260709 13:20:22.161494 16819 raft_consensus.cc:2393] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 9: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
I20260709 13:20:22.212293 16292 raft_consensus.cc:1275] T 8cc959fb7f704fba9009c0595467d074 P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:20:22.212940 16226 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33050 (request call id 568) took 1338 ms. Trace:
I20260709 13:20:22.212972 16299 raft_consensus.cc:2468] T 18ad7e3af5504571a69d10c00750a01c P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 7.
I20260709 13:20:22.213392 16226 rpcz_store.cc:276] 0709 13:20:20.874184 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:22.056812 (+1182628us) service_pool.cc:224] Handling call
0709 13:20:22.212884 (+156072us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:20:22.037581 19148 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.839s	user 0.001s	sys 0.020s
I20260709 13:20:22.169981 16817 raft_consensus.cc:1240] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 [term 8 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 7. Current term is 8. Ops: []
I20260709 13:20:22.168807 16219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7ca61592e848ea9b0fb81a2ce33cdc" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:22.214666 16219 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33050 (request call id 576) took 1289 ms. Trace:
I20260709 13:20:22.214978 16219 rpcz_store.cc:276] 0709 13:20:20.925402 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:22.160951 (+1235549us) service_pool.cc:224] Handling call
0709 13:20:22.214608 (+ 53657us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:20:22.174012 16296 raft_consensus.cc:2468] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 9.
I20260709 13:20:22.175460 16221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62fe3346cd3c42c4a55e14af9073bca4" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:22.216655 16221 raft_consensus.cc:2468] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 10.
I20260709 13:20:22.166695 19580 raft_consensus.cc:515] T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:22.204136 19364 raft_consensus.cc:3060] T ee9ed67006144797b1de6ea8c2013a68 P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:20:22.089398 16151 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40844 (request call id 2121) took 1372 ms. Trace:
I20260709 13:20:22.209223 16818 raft_consensus.cc:2393] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 10: Already voted for candidate cb82cc7bd6db46d980c75237a1402860 in this term.
I20260709 13:20:22.207444 16821 rpcz_store.cc:276] 0709 13:20:19.641276 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:19.684295 (+ 43019us) service_pool.cc:224] Handling call
0709 13:20:19.684733 (+   438us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:20:19.725989 (+ 41256us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:19.726021 (+    32us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:20:19.726763 (+   742us) log.cc:844] Serialized 52 byte log entry
0709 13:20:20.140447 (+413684us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:20:20.256198 (+115751us) log.cc:844] Serialized 14 byte log entry
0709 13:20:20.261357 (+  5159us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:20.261724 (+   367us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:20:21.694362 (+1432638us) spinlock_profiling.cc:243] Waited 17.1 ms on lock 0x7b2400303158. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69ad8fadb1 00007f69ad8f89c4 00007f69ad8f6acb 00007f69adab1dc9 00007f69adab1a55 00007f69adab109d 00007f69b06e310e 00007f69b06fa88c 00007f69b06f9bb6 00007f69b06f97d8 00007f69b0703221
0709 13:20:22.085911 (+391549us) raft_consensus.cc:1685] finished
0709 13:20:22.086044 (+   133us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:22.086210 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":39620,"prepare.queue_time_us":300902,"prepare.run_cpu_time_us":64,"prepare.run_wall_time_us":4292,"raft.queue_time_us":25108,"raft.run_cpu_time_us":344,"raft.run_wall_time_us":40035,"spinlock_wait_cycles":39261952,"thread_start_us":367574,"threads_started":2,"wal-append.queue_time_us":662443}
I20260709 13:20:22.220528 16151 rpcz_store.cc:276] 0709 13:20:20.716856 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:21.624876 (+908020us) service_pool.cc:224] Handling call
0709 13:20:21.625372 (+   496us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:20:21.641330 (+ 15958us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:21.641379 (+    49us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:20:21.664042 (+ 22663us) log.cc:844] Serialized 26 byte log entry
0709 13:20:21.919630 (+255588us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:20:21.919687 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:21.919770 (+    83us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:20:22.089095 (+169325us) raft_consensus.cc:1685] finished
0709 13:20:22.089127 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:22.089321 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8641,"raft.queue_time_us":14579,"raft.run_cpu_time_us":132,"raft.run_wall_time_us":17034,"thread_start_us":241612,"threads_started":1,"wal-append.queue_time_us":402186}
I20260709 13:20:22.094134 16224 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57604 (request call id 2287) took 1450 ms. Trace:
I20260709 13:20:22.222786 16224 rpcz_store.cc:276] 0709 13:20:20.643626 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:21.676644 (+1033018us) service_pool.cc:224] Handling call
0709 13:20:22.094052 (+417408us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:20:22.223743 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8638ee363273426db1393dd81e5c7b72" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:20:22.127437 19306 raft_consensus.cc:2804] T 434bff3082d141838068f323bdb0d1ed P cb82cc7bd6db46d980c75237a1402860 [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 13:20:22.215417 16789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 85234.2us
I20260709 13:20:22.229825 16224 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57600 (request call id 1269) took 1346 ms. Trace:
I20260709 13:20:22.230209 16224 rpcz_store.cc:276] 0709 13:20:20.883605 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:22.223502 (+1339897us) service_pool.cc:224] Handling call
0709 13:20:22.229770 (+  6268us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:20:22.173589 19576 consensus_meta.cc:287] T 829b4ad8c5a54eababeea1181b960d4b P cb82cc7bd6db46d980c75237a1402860: Time spent flushing consensus metadata: real 0.501s	user 0.011s	sys 0.000s
I20260709 13:20:22.227489 16789 leader_election.cc:304] T 19baa7bc367844c288bca8f81439c413 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:20:22.172672 19345 raft_consensus.cc:3060] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:20:22.233443 16192 leader_election.cc:304] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 647a343a542d4a139fe106c19095129b
I20260709 13:20:22.128024 19068 consensus_queue.cc:237] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:22.244715 16221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18ad7e3af5504571a69d10c00750a01c" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:22.199326 19496 leader_election.cc:290] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 8 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:22.217258 19414 raft_consensus.cc:493] T 68a6838d82ef4cac83507f906139d90f P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:20:22.248092 16221 raft_consensus.cc:2468] T 18ad7e3af5504571a69d10c00750a01c P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 7.
I20260709 13:20:22.244745 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c2095a5f4ad49408dbc7f69c606120d" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:20:22.252167 16224 raft_consensus.cc:2376] T 7c2095a5f4ad49408dbc7f69c606120d P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 647a343a542d4a139fe106c19095129b in term 7. Re-sending same reply.
I20260709 13:20:22.248237 16151 raft_consensus.cc:1240] T 829b4ad8c5a54eababeea1181b960d4b P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 7. Current term is 8. Ops: [7.9-7.9]
I20260709 13:20:22.245332 19584 raft_consensus.cc:3060] T 19baa7bc367844c288bca8f81439c413 P cb82cc7bd6db46d980c75237a1402860 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:20:22.220877 18901 leader_election.cc:290] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 8 election: Requested vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
W20260709 13:20:22.241838 16266 connection.cc:256] RPC call timeout handler was delayed by -1.0674066809999658s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2056s for a response
I20260709 13:20:22.257984 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdbd78acb4cf46298e0de8ba0d130e19" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:20:22.258067 16117 leader_election.cc:304] T af57996a8be84d398f0f023e8d08ef77 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:20:22.258545 16300 raft_consensus.cc:3060] T cdbd78acb4cf46298e0de8ba0d130e19 P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:20:22.220232 19578 raft_consensus.cc:493] T 66f75124215f4ca6bef831e174a792d8 P cb82cc7bd6db46d980c75237a1402860 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:22.253285 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fffab26427e24d398e2695aad40ca134" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:20:22.220094 19596 raft_consensus.cc:515] T 2794af5b99984efebdee564035841cb0 P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:22.282929 16143 raft_consensus.cc:2468] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 7.
I20260709 13:20:22.226568 19568 raft_consensus.cc:2804] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:20:22.255733 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2372981006d4b218215c28964dea623" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:20:22.293318 16224 raft_consensus.cc:3060] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:20:22.229041 19551 raft_consensus.cc:493] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:22.235491 19148 raft_consensus.cc:2749] T af57996a8be84d398f0f023e8d08ef77 P cb82cc7bd6db46d980c75237a1402860 [term 9 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:20:22.259577 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 128461us
I20260709 13:20:22.245800 19512 raft_consensus.cc:493] T 8a878999c0024d2abe517484180290fc P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:22.241895 19421 raft_consensus.cc:2749] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:20:22.259905 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5718c623ebee4d2bb770f0f9ddb2f164" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:22.319478 16226 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33050 (request call id 572) took 1444 ms. Trace:
I20260709 13:20:22.319849 16226 rpcz_store.cc:276] 0709 13:20:20.875354 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:22.259617 (+1384263us) service_pool.cc:224] Handling call
0709 13:20:22.319416 (+ 59799us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:20:22.229271 19306 raft_consensus.cc:493] T 434bff3082d141838068f323bdb0d1ed P cb82cc7bd6db46d980c75237a1402860 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:22.261029 16221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f5f89186204242bf377f7c42b2e2da" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:22.261191 16295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7ca61592e848ea9b0fb81a2ce33cdc" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:20:22.321544 16295 raft_consensus.cc:3060] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:20:22.261777 16296 raft_consensus.cc:1240] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 7. Current term is 8. Ops: []
I20260709 13:20:22.261139 16819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66ae176961b74813a58bc15c9c4c6f2a" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
W20260709 13:20:22.263657 19509 consensus_meta.cc:287] T 746aabf856ab401c9ef5e859371e7b67 P 1ca2725d290e47d68b1a4b609c9816ac: Time spent flushing consensus metadata: real 0.639s	user 0.012s	sys 0.000s
I20260709 13:20:22.266479 19197 leader_election.cc:290] T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 10 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:22.247457 19514 raft_consensus.cc:2804] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:20:22.267750 19583 leader_election.cc:290] T ada5a004ba7c46819051e3ccec71ffdb P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:22.268160 19567 raft_consensus.cc:2804] T af57996a8be84d398f0f023e8d08ef77 P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:20:22.259147 19414 raft_consensus.cc:515] T 68a6838d82ef4cac83507f906139d90f P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:22.286901 19450 raft_consensus.cc:493] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:22.274456 19404 leader_election.cc:290] T fffab26427e24d398e2695aad40ca134 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 7 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:22.276134 16223 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57600 (request call id 1254) took 1973 ms. Trace:
I20260709 13:20:22.276572 16825 raft_consensus.cc:1275] T c9944d5e4b8c4109af9955c4f08c9095 P cb82cc7bd6db46d980c75237a1402860 [term 8 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 7 index: 9. Preceding OpId from leader: term: 8 index: 9. (term mismatch)
I20260709 13:20:22.278316 16822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31969392ccdb43b491911a319e8bc7e1" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:22.332837 16825 pending_rounds.cc:85] T c9944d5e4b8c4109af9955c4f08c9095 P cb82cc7bd6db46d980c75237a1402860: Aborting all ops after (but not including) 8
I20260709 13:20:22.281260 16816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5718c623ebee4d2bb770f0f9ddb2f164" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:20:22.333504 16816 raft_consensus.cc:3060] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:20:22.244346 19591 consensus_queue.cc:1059] T 23f82224f4564fea93572e8206bad1fe P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.023s
I20260709 13:20:22.332075 16223 rpcz_store.cc:276] 0709 13:20:20.302674 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:20.304784 (+  2110us) service_pool.cc:224] Handling call
0709 13:20:21.713539 (+1408755us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:21.993702 (+280163us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:22.275951 (+282249us) spinlock_profiling.cc:243] Waited 1.41 s on lock 0x7b54002c8150. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5e8ad 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:20:22.276079 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3239144832}
I20260709 13:20:22.316669 16790 leader_election.cc:304] T 8c11074579b046d99e585c2818e18a2c P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:20:22.317354 18599 consensus_queue.cc:1048] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:20:22.291258 16116 leader_election.cc:304] T 68a6838d82ef4cac83507f906139d90f P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:20:22.325094 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62fe3346cd3c42c4a55e14af9073bca4" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
W20260709 13:20:22.272819 16225 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57604 (request call id 2286) took 1629 ms (client timeout 2117 ms). Trace:
I20260709 13:20:22.330832 16291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02838c86531d4f98bf58d1c22b8baf77" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 9 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:22.312676 19568 raft_consensus.cc:493] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:22.335135 16296 raft_consensus.cc:1240] T 5a96373bb0e14dce8a266b2fc6c58507 P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 6. Current term is 7. Ops: []
I20260709 13:20:22.323876 19306 raft_consensus.cc:3060] T 434bff3082d141838068f323bdb0d1ed P cb82cc7bd6db46d980c75237a1402860 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:20:22.338348 16225 rpcz_store.cc:269] 0709 13:20:20.643364 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:20.697552 (+ 54188us) service_pool.cc:224] Handling call
0709 13:20:22.272643 (+1575091us) spinlock_profiling.cc:243] Waited 307 ms on lock 0x7b5400396554. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:20:22.272756 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":705703168}
I20260709 13:20:22.342942 16224 raft_consensus.cc:2468] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 9.
W20260709 13:20:22.349278 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 58125us
I20260709 13:20:22.327574 19514 raft_consensus.cc:493] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Starting leader election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:20:22.354542 16820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:22.355208 16146 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40844 (request call id 2120) took 1638 ms. Trace:
I20260709 13:20:22.357046 19478 leader_election.cc:290] T af57996a8be84d398f0f023e8d08ef77 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 9 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
W20260709 13:20:22.358136 16046 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:58400 (request call id 189) took 27165 ms (client timeout 15000 ms). Trace:
W20260709 13:20:22.360630 16220 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33050 (request call id 570) took 1485 ms (client timeout 1546 ms). Trace:
I20260709 13:20:22.205281 15046 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:20:22.315492 19578 raft_consensus.cc:515] T 66f75124215f4ca6bef831e174a792d8 P cb82cc7bd6db46d980c75237a1402860 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:20:22.406018 16220 rpcz_store.cc:269] 0709 13:20:20.874753 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:20.999863 (+125110us) service_pool.cc:224] Handling call
0709 13:20:22.360445 (+1360582us) spinlock_profiling.cc:243] Waited 1.14 s on lock 0x7b5400395654. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:20:22.360574 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2630847360}
I20260709 13:20:22.368278 19580 leader_election.cc:290] T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:20:22.368911 19529 raft_consensus.cc:2749] T 68a6838d82ef4cac83507f906139d90f P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:20:22.333083 16825 pending_rounds.cc:107] T c9944d5e4b8c4109af9955c4f08c9095 P cb82cc7bd6db46d980c75237a1402860: Aborting uncommitted NO_OP operation due to leader change: 7.9
I20260709 13:20:22.260115 19382 consensus_queue.cc:260] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:22.354063 16222 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57600 (request call id 1264) took 2034 ms. Trace:
I20260709 13:20:22.378715 16300 raft_consensus.cc:2468] T cdbd78acb4cf46298e0de8ba0d130e19 P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 8.
I20260709 13:20:22.425642 16222 rpcz_store.cc:276] 0709 13:20:20.319691 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:20.323363 (+  3672us) service_pool.cc:224] Handling call
0709 13:20:20.323685 (+   322us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:20:20.405832 (+ 82147us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:20.405877 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:20:20.406490 (+   613us) log.cc:844] Serialized 26 byte log entry
0709 13:20:20.497756 (+ 91266us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:20:20.497805 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:20.497899 (+    94us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:20:22.352543 (+1854644us) spinlock_profiling.cc:243] Waited 7.43 ms on lock 0x7b240028f3e8. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69ad8fadb1 00007f69ad8f89c4 00007f69ad8f6acb 00007f69adab1dc9 00007f69adab1a55 00007f69adab0fb0 00007f69b0727488 00007f69b0727be8 00007f69b06ff398 00007f69b06fa76a 00007f69b06f9bb6
0709 13:20:22.353731 (+  1188us) raft_consensus.cc:1685] finished
0709 13:20:22.353754 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:22.353992 (+   238us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":76331,"raft.queue_time_us":84529,"raft.run_cpu_time_us":251,"raft.run_wall_time_us":6268,"spinlock_wait_cycles":17083648,"thread_start_us":90810,"threads_started":1,"wal-append.queue_time_us":185812}
I20260709 13:20:22.428102 16222 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57604 (request call id 2291) took 1770 ms. Trace:
I20260709 13:20:22.428457 16222 rpcz_store.cc:276] 0709 13:20:20.657962 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:22.427312 (+1769350us) service_pool.cc:224] Handling call
0709 13:20:22.427626 (+   314us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:22.427791 (+   165us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:22.427816 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:20:22.427855 (+    39us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:20:22.427883 (+    28us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:22.427929 (+    46us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:22.428030 (+   101us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:20:22.405691 16046 rpcz_store.cc:269] 0709 13:19:55.192594 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:55.198748 (+  6154us) service_pool.cc:224] Handling call
0709 13:20:22.357892 (+27159144us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:20:13.869630 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:20:14.042456 (+172826us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:20:14.064472 (+ 22016us) write_op.cc:435] Acquired schema lock
0709 13:20:14.075956 (+ 11484us) tablet.cc:662] Decoding operations
0709 13:20:14.213417 (+137461us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:20:14.213509 (+    92us) write_op.cc:641] Partition lock acquired for write op
0709 13:20:14.213537 (+    28us) tablet.cc:685] Acquiring locks for 7 operations
0709 13:20:14.214462 (+   925us) tablet.cc:701] Row locks acquired
0709 13:20:14.214529 (+    67us) write_op.cc:260] PREPARE: finished
0709 13:20:14.214787 (+   258us) write_op.cc:270] Start()
0709 13:20:14.215018 (+   231us) write_op.cc:276] Timestamp: P: 1783603214214700 usec, L: 0
0709 13:20:14.215054 (+    36us) op_driver.cc:348] REPLICATION: starting
0709 13:20:14.294204 (+ 79150us) log.cc:844] Serialized 2856 byte log entry
0709 13:20:14.789614 (+495410us) op_driver.cc:464] REPLICATION: finished
0709 13:20:15.431729 (+642115us) write_op.cc:301] APPLY: starting
0709 13:20:15.439307 (+  7578us) tablet.cc:1367] starting BulkCheckPresence
0709 13:20:15.475297 (+ 35990us) tablet.cc:1370] finished BulkCheckPresence
0709 13:20:15.475439 (+   142us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:20:15.492004 (+ 16565us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:20:15.492301 (+   297us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=7
0709 13:20:15.492354 (+    53us) write_op.cc:312] APPLY: finished
0709 13:20:15.492704 (+   350us) log.cc:844] Serialized 59 byte log entry
0709 13:20:15.492973 (+   269us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:20:15.493082 (+   109us) write_op.cc:454] Released schema lock
0709 13:20:15.493149 (+    67us) write_op.cc:341] FINISH: Updating metrics
0709 13:20:15.493535 (+   386us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:20:15.493563 (+    28us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":7128916,"reported_tablets":23,"child_traces":[["op",{"apply.queue_time_us":597610,"apply.run_cpu_time_us":6441,"apply.run_wall_time_us":75829,"num_ops":7,"prepare.queue_time_us":389428,"prepare.run_cpu_time_us":28954,"prepare.run_wall_time_us":676964,"raft.queue_time_us":210688,"raft.run_cpu_time_us":16647,"raft.run_wall_time_us":340133,"replication_time_us":547035,"thread_start_us":785187,"threads_started":4,"wal-append.queue_time_us":230399,"wal-append.run_cpu_time_us":34532,"wal-append.run_wall_time_us":1952157}]]}
I20260709 13:20:22.429502 16147 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40830 (request call id 1274) took 1793 ms. Trace:
I20260709 13:20:22.429942 16147 rpcz_store.cc:276] 0709 13:20:20.635990 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:20.656789 (+ 20799us) service_pool.cc:224] Handling call
0709 13:20:20.697881 (+ 41092us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:20:21.015613 (+317732us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:22.429272 (+1413659us) spinlock_profiling.cc:243] Waited 300 ms on lock 0x7b540013ecd4. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:20:22.429448 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":690338176}
I20260709 13:20:22.430585 16824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26354361e9b044cea1c2b86c4882fe92" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:22.431833 19378 raft_consensus.cc:3055] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 1ca2725d290e47d68b1a4b609c9816ac [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:20:22.432226 19378 raft_consensus.cc:740] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 1ca2725d290e47d68b1a4b609c9816ac [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:20:22.432806 19306 raft_consensus.cc:515] T 434bff3082d141838068f323bdb0d1ed P cb82cc7bd6db46d980c75237a1402860 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:22.433043 19378 consensus_queue.cc:260] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:22.433970 19378 raft_consensus.cc:3060] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:20:22.434757 19576 raft_consensus.cc:2749] T 8c11074579b046d99e585c2818e18a2c P cb82cc7bd6db46d980c75237a1402860 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:20:22.397887 19080 leader_election.cc:290] T 5a96373bb0e14dce8a266b2fc6c58507 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:20:22.379276 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a878999c0024d2abe517484180290fc" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 8 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:22.396911 19233 leader_election.cc:290] T d16dfad0c3b540e5b4b86492ddf87208 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:22.396834 16221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e4bfb9090c8460eb14fa7e0d818e7f1" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:22.320362 19551 raft_consensus.cc:515] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:22.331449 19450 raft_consensus.cc:3060] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:20:22.397475 19584 raft_consensus.cc:2749] T 19baa7bc367844c288bca8f81439c413 P cb82cc7bd6db46d980c75237a1402860 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:20:22.396700 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 8 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:22.404438 16146 rpcz_store.cc:276] 0709 13:20:20.716548 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:21.473747 (+757199us) service_pool.cc:224] Handling call
0709 13:20:21.487417 (+ 13670us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:20:22.208857 (+721440us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:22.354903 (+146046us) spinlock_profiling.cc:243] Waited 601 ms on lock 0x7b54000d1454. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:20:22.355123 (+   220us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1381463936}
I20260709 13:20:22.413780 16292 raft_consensus.cc:1240] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 6. Current term is 7. Ops: []
I20260709 13:20:22.347025 19568 raft_consensus.cc:3060] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:20:22.406826 16223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7ca61592e848ea9b0fb81a2ce33cdc" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:20:22.405537 19514 raft_consensus.cc:3060] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:20:22.465353 16221 raft_consensus.cc:2393] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 in current term 8: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
I20260709 13:20:22.423323 16825 raft_consensus.cc:2889] T c9944d5e4b8c4109af9955c4f08c9095 P cb82cc7bd6db46d980c75237a1402860 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:20:22.465478 16292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434bff3082d141838068f323bdb0d1ed" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:20:22.377121 16116 leader_election.cc:304] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
I20260709 13:20:22.466122 16292 raft_consensus.cc:3060] T 434bff3082d141838068f323bdb0d1ed P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:20:22.323933 19512 raft_consensus.cc:515] T 8a878999c0024d2abe517484180290fc P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:20:22.447954 16791 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:22.421806 19364 raft_consensus.cc:515] T ee9ed67006144797b1de6ea8c2013a68 P 1ca2725d290e47d68b1a4b609c9816ac [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:22.436273 16789 leader_election.cc:304] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:20:22.429914 16222 raft_consensus.cc:1275] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:20:22.464088 16225 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33050 (request call id 573) took 1588 ms. Trace:
W20260709 13:20:22.491207 16789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 54993.4us
I20260709 13:20:22.491604 16225 rpcz_store.cc:276] 0709 13:20:20.875620 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:22.371581 (+1495961us) service_pool.cc:224] Handling call
0709 13:20:22.464026 (+ 92445us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:20:22.464386 16816 raft_consensus.cc:2410] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b for term 8 because replica has last-logged OpId of term: 7 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 8.
I20260709 13:20:22.344308 19567 raft_consensus.cc:697] T af57996a8be84d398f0f023e8d08ef77 P 647a343a542d4a139fe106c19095129b [term 9 LEADER]: Becoming Leader. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:20:22.402694 16044 catalog_manager.cc:5697] T 829b4ad8c5a54eababeea1181b960d4b P 1ca2725d290e47d68b1a4b609c9816ac reported cstate change: term changed from 5 to 6, leader changed from cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) to 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130). New cstate: current_term: 6 leader_uuid: "1ca2725d290e47d68b1a4b609c9816ac" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:20:22.437191 16291 raft_consensus.cc:1240] T e4dc6b0541ca431ebc757beac00521f1 P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 6. Current term is 7. Ops: []
W20260709 13:20:22.437155 16217 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57600 (request call id 1256) took 2133 ms (client timeout 2090 ms). Trace:
I20260709 13:20:22.398644 19414 leader_election.cc:290] T 68a6838d82ef4cac83507f906139d90f P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:20:22.475854 16221 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33050 (request call id 571) took 1600 ms. Trace:
I20260709 13:20:22.476936 16825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a9de347c711477e87c582d003f36925" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 10 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
W20260709 13:20:22.477492 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 100439us
I20260709 13:20:22.497265 16295 raft_consensus.cc:2468] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 10.
I20260709 13:20:22.497501 16222 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57600 (request call id 1265) took 1797 ms. Trace:
I20260709 13:20:22.498123 16116 leader_election.cc:304] T 18ad7e3af5504571a69d10c00750a01c P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
I20260709 13:20:22.498323 16222 rpcz_store.cc:276] 0709 13:20:20.694435 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:22.429323 (+1734888us) service_pool.cc:224] Handling call
0709 13:20:22.429602 (+   279us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:22.491118 (+ 61516us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:22.491348 (+   230us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:20:22.498968 16116 leader_election.cc:304] T 7c2095a5f4ad49408dbc7f69c606120d P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 90f3da57796b488c9087a2f523adf045
I20260709 13:20:22.483443 19233 raft_consensus.cc:493] T 7c2095a5f4ad49408dbc7f69c606120d P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:22.435017 19306 leader_election.cc:290] T 434bff3082d141838068f323bdb0d1ed P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 9 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
W20260709 13:20:22.485986 19508 log.cc:927] Time spent T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 2.234s	user 0.007s	sys 0.006s
I20260709 13:20:22.486828 19522 raft_consensus.cc:2764] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [term 10 FOLLOWER]: Leader pre-election decision vote started in defunct term 9: won
I20260709 13:20:22.495890 16226 raft_consensus.cc:2410] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 for term 10 because replica has last-logged OpId of term: 9 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 8.
I20260709 13:20:22.497902 16820 raft_consensus.cc:2468] T 19baa7bc367844c288bca8f81439c413 P cb82cc7bd6db46d980c75237a1402860 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 8.
I20260709 13:20:22.496287 16221 rpcz_store.cc:276] 0709 13:20:20.875047 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:22.474861 (+1599814us) service_pool.cc:224] Handling call
0709 13:20:22.475224 (+   363us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:22.475520 (+   296us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:22.475547 (+    27us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:20:22.475600 (+    53us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:20:22.475630 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:22.475681 (+    51us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:22.475779 (+    98us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":7820,"raft.run_cpu_time_us":117,"raft.run_wall_time_us":114}
I20260709 13:20:22.504091 16222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdbd78acb4cf46298e0de8ba0d130e19" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:20:22.437426 19382 raft_consensus.cc:3060] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:20:22.400738 19345 raft_consensus.cc:515] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:22.504938 16789 leader_election.cc:304] T 3a7ca61592e848ea9b0fb81a2ce33cdc P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:20:22.520257 16224 raft_consensus.cc:2410] T 8a878999c0024d2abe517484180290fc P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b for term 8 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
W20260709 13:20:22.495510 16217 rpcz_store.cc:269] 0709 13:20:20.303299 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:20.303404 (+   105us) service_pool.cc:224] Handling call
0709 13:20:22.436976 (+2133572us) spinlock_profiling.cc:243] Waited 1.04 s on lock 0x7b5400368154. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:20:22.437069 (+    93us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2391571712}
I20260709 13:20:22.525282 16217 raft_consensus.cc:1275] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:20:22.526306 16217 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33050 (request call id 580) took 1474 ms. Trace:
I20260709 13:20:22.526847 16217 rpcz_store.cc:276] 0709 13:20:21.051770 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:22.524438 (+1472668us) service_pool.cc:224] Handling call
0709 13:20:22.524824 (+   386us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:22.525938 (+  1114us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:22.526230 (+   292us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:20:22.528411 16295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e4bfb9090c8460eb14fa7e0d818e7f1" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:20:22.530434 16291 raft_consensus.cc:1240] T 66ae176961b74813a58bc15c9c4c6f2a P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 9. Current term is 10. Ops: []
I20260709 13:20:22.534600 16221 raft_consensus.cc:1240] T ee9ed67006144797b1de6ea8c2013a68 P 1ca2725d290e47d68b1a4b609c9816ac [term 6 FOLLOWER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 5. Current term is 6. Ops: []
I20260709 13:20:22.534739 16223 raft_consensus.cc:1240] T 829b4ad8c5a54eababeea1181b960d4b P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 7. Current term is 8. Ops: [7.9-7.9]
I20260709 13:20:22.533797 19587 raft_consensus.cc:2749] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:20:22.541085 16192 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 295434us
W20260709 13:20:22.537494 19113 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.582s	user 0.009s	sys 0.030s
I20260709 13:20:22.508744 19584 raft_consensus.cc:2804] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:20:22.544659 16044 catalog_manager.cc:5697] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac reported cstate change: term changed from 4 to 5, leader changed from 90f3da57796b488c9087a2f523adf045 (127.14.177.131) to 647a343a542d4a139fe106c19095129b (127.14.177.129). New cstate: current_term: 5 leader_uuid: "647a343a542d4a139fe106c19095129b" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } } }
I20260709 13:20:22.547526 16294 raft_consensus.cc:2393] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 6: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
I20260709 13:20:22.528201 19529 raft_consensus.cc:2804] T 18ad7e3af5504571a69d10c00750a01c P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:20:22.556420 16221 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33050 (request call id 578) took 1592 ms. Trace:
I20260709 13:20:22.533103 19404 raft_consensus.cc:2804] T 7c2095a5f4ad49408dbc7f69c606120d P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:20:22.569413 16221 rpcz_store.cc:276] 0709 13:20:20.963954 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:22.504422 (+1540468us) service_pool.cc:224] Handling call
0709 13:20:22.534358 (+ 29936us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:22.556157 (+ 21799us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:22.556358 (+   201us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:20:22.503971 19233 raft_consensus.cc:515] T 7c2095a5f4ad49408dbc7f69c606120d P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:20:22.590534 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 273929us
I20260709 13:20:22.597442 16291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cc959fb7f704fba9009c0595467d074" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:22.598356 16291 raft_consensus.cc:2468] T 8cc959fb7f704fba9009c0595467d074 P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 8.
I20260709 13:20:22.545288 19576 raft_consensus.cc:2804] T 3a7ca61592e848ea9b0fb81a2ce33cdc P cb82cc7bd6db46d980c75237a1402860 [term 10 FOLLOWER]: Leader election won for term 10
W20260709 13:20:22.567229 19113 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.620s	user 0.010s	sys 0.030s
I20260709 13:20:22.607795 16221 raft_consensus.cc:1275] T 1df93779209748b9860f6d893dd3de53 P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 5. (term mismatch)
I20260709 13:20:22.608508 16221 pending_rounds.cc:85] T 1df93779209748b9860f6d893dd3de53 P 1ca2725d290e47d68b1a4b609c9816ac: Aborting all ops after (but not including) 4
I20260709 13:20:22.609536 16221 pending_rounds.cc:107] T 1df93779209748b9860f6d893dd3de53 P 1ca2725d290e47d68b1a4b609c9816ac: Aborting uncommitted NO_OP operation due to leader change: 7.5
I20260709 13:20:22.610361 16221 raft_consensus.cc:2889] T 1df93779209748b9860f6d893dd3de53 P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:20:22.611616 16221 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57604 (request call id 2297) took 1059 ms. Trace:
I20260709 13:20:22.612947 16221 rpcz_store.cc:276] 0709 13:20:21.551834 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:22.570325 (+1018491us) service_pool.cc:224] Handling call
0709 13:20:22.602024 (+ 31699us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:22.611319 (+  9295us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:22.611552 (+   233us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:20:22.608385 16148 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46224 (request call id 465) took 1647 ms. Trace:
I20260709 13:20:22.615510 16044 catalog_manager.cc:5697] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac reported cstate change: term changed from 4 to 5, leader changed from cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) to 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130). New cstate: current_term: 5 leader_uuid: "1ca2725d290e47d68b1a4b609c9816ac" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:20:22.593050 19529 raft_consensus.cc:493] T 18ad7e3af5504571a69d10c00750a01c P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:22.643151 19233 leader_election.cc:290] T 7c2095a5f4ad49408dbc7f69c606120d P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:22.645228 16150 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46224 (request call id 452) took 2225 ms. Trace:
I20260709 13:20:22.645668 16150 rpcz_store.cc:276] 0709 13:20:20.420316 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:20.426053 (+  5737us) service_pool.cc:224] Handling call
0709 13:20:20.438671 (+ 12618us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:20.936617 (+497946us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:22.645001 (+1708384us) spinlock_profiling.cc:243] Waited 441 ms on lock 0x7b540020ba54. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:20:22.645169 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1013559168}
I20260709 13:20:22.596982 19584 raft_consensus.cc:697] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 [term 8 LEADER]: Becoming Leader. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:20:22.648073 19584 consensus_queue.cc:237] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:22.650944 16044 catalog_manager.cc:5697] T ee9ed67006144797b1de6ea8c2013a68 P 1ca2725d290e47d68b1a4b609c9816ac reported cstate change: term changed from 3 to 5, leader changed from 647a343a542d4a139fe106c19095129b (127.14.177.129) to cb82cc7bd6db46d980c75237a1402860 (127.14.177.132). New cstate: current_term: 5 leader_uuid: "cb82cc7bd6db46d980c75237a1402860" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:20:22.653194 16044 catalog_manager.cc:5697] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac reported cstate change: term changed from 3 to 6, leader changed from cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) to 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130). New cstate: current_term: 6 leader_uuid: "1ca2725d290e47d68b1a4b609c9816ac" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:20:22.655340 16192 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 54305us
I20260709 13:20:22.654150 16292 raft_consensus.cc:2468] T 434bff3082d141838068f323bdb0d1ed P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 9.
I20260709 13:20:22.663213 16291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
W20260709 13:20:22.685745 16263 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:22.601070 19404 raft_consensus.cc:697] T 7c2095a5f4ad49408dbc7f69c606120d P 647a343a542d4a139fe106c19095129b [term 7 LEADER]: Becoming Leader. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
W20260709 13:20:22.693019 16190 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:20:22.693265 16189 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:22.694151 16116 leader_election.cc:304] T 84f2da98864b4425bf7c0055951fdd4b P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 90f3da57796b488c9087a2f523adf045
I20260709 13:20:22.694250 16789 leader_election.cc:304] T 434bff3082d141838068f323bdb0d1ed P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 
W20260709 13:20:22.694661 16191 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:22.695113 16292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a378d7f796f94615b3b347119a9880fd" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:22.712076 16298 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48674 (request call id 2372) took 2108 ms. Trace:
I20260709 13:20:22.712185 16292 raft_consensus.cc:2468] T a378d7f796f94615b3b347119a9880fd P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 7.
I20260709 13:20:22.696969 16818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e65676b21b748728ad39668cfe722ec" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:22.698896 16192 leader_election.cc:304] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
I20260709 13:20:22.712600 16298 rpcz_store.cc:276] 0709 13:20:20.591291 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:20.592334 (+  1043us) service_pool.cc:224] Handling call
0709 13:20:20.607479 (+ 15145us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:20.969299 (+361820us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:22.696635 (+1727336us) spinlock_profiling.cc:243] Waited 361 ms on lock 0x7b54001ef354. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:20:22.699711 (+  3076us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":830161152}
I20260709 13:20:22.699453 16291 raft_consensus.cc:3060] T ecc1e363230043fe9fef390f0b979f20 P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:20:22.678658 16221 raft_consensus.cc:1275] T 8cc959fb7f704fba9009c0595467d074 P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 8 index: 6. (term mismatch)
I20260709 13:20:22.700549 16223 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33050 (request call id 579) took 1673 ms. Trace:
I20260709 13:20:22.714674 16223 rpcz_store.cc:276] 0709 13:20:21.002111 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:22.504450 (+1502339us) service_pool.cc:224] Handling call
0709 13:20:22.534519 (+ 30069us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:20:22.544653 (+ 10134us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:22.675028 (+130375us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:20:22.717108 16223 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57604 (request call id 2304) took 1098 ms. Trace:
I20260709 13:20:22.717521 16223 rpcz_store.cc:276] 0709 13:20:21.619008 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:22.715871 (+1096863us) service_pool.cc:224] Handling call
0709 13:20:22.716345 (+   474us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:22.716606 (+   261us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:22.716662 (+    56us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:20:22.716730 (+    68us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:20:22.716787 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:22.716872 (+    85us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:22.717031 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:20:22.720085 16296 raft_consensus.cc:1240] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 9. Current term is 10. Ops: []
I20260709 13:20:22.726981 16824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8638ee363273426db1393dd81e5c7b72" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:20:22.729238 16819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e65676b21b748728ad39668cfe722ec" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:20:22.628803 19576 raft_consensus.cc:697] T 3a7ca61592e848ea9b0fb81a2ce33cdc P cb82cc7bd6db46d980c75237a1402860 [term 10 LEADER]: Becoming Leader. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:20:22.721547 16218 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57604 (request call id 2271) took 3049 ms. Trace:
I20260709 13:20:22.661162 19529 raft_consensus.cc:3060] T 18ad7e3af5504571a69d10c00750a01c P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:20:22.729322 16292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ab0ac8347641e6955e9bce7d90c1a2" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:22.706629 19113 raft_consensus.cc:493] T 829b4ad8c5a54eababeea1181b960d4b P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:22.699152 19414 raft_consensus.cc:493] T 8d4ab870a05347868956aa8b841b2981 P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:22.758595 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74ca8140f35c4ae29e8a664f6bdfc829" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:22.740109 16218 rpcz_store.cc:276] 0709 13:20:19.657344 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:19.658250 (+   906us) service_pool.cc:224] Handling call
0709 13:20:19.658703 (+   453us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:20:19.660338 (+  1635us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:19.660370 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:20:19.660865 (+   495us) log.cc:844] Serialized 26 byte log entry
0709 13:20:19.815904 (+155039us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:20:19.815957 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:19.816038 (+    81us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:20:22.485407 (+2669369us) spinlock_profiling.cc:243] Waited 3.08 ms on lock 0x7b4400022b4c. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69ad8fadb1 00007f69ad998c85 00007f69ad9996a2 00007f69b070b1db 00007f69b0706aaf 00007f69b06fa86e 00007f69b06f9bb6 00007f69b06f97d8 00007f69b0703221 00007f69b07031d9 00007f69b0703169
0709 13:20:22.693333 (+207926us) raft_consensus.cc:1685] finished
0709 13:20:22.693355 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:22.707113 (+ 13758us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":983,"raft.run_cpu_time_us":108,"raft.run_wall_time_us":108,"spinlock_wait_cycles":7083392,"thread_start_us":154630,"threads_started":1,"wal-append.queue_time_us":575422}
I20260709 13:20:22.759749 16296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a96373bb0e14dce8a266b2fc6c58507" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:22.764305 16292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9944d5e4b8c4109af9955c4f08c9095" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 9 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:22.764500 16296 raft_consensus.cc:2468] T 5a96373bb0e14dce8a266b2fc6c58507 P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 7.
I20260709 13:20:22.696987 19589 leader_election.cc:290] T 02838c86531d4f98bf58d1c22b8baf77 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:20:22.737761 19509 raft_consensus.cc:493] T 746aabf856ab401c9ef5e859371e7b67 P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:22.725811 16221 pending_rounds.cc:85] T 8cc959fb7f704fba9009c0595467d074 P 1ca2725d290e47d68b1a4b609c9816ac: Aborting all ops after (but not including) 5
I20260709 13:20:22.736522 19421 raft_consensus.cc:2804] T 84f2da98864b4425bf7c0055951fdd4b P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:20:22.774420 16221 pending_rounds.cc:107] T 8cc959fb7f704fba9009c0595467d074 P 1ca2725d290e47d68b1a4b609c9816ac: Aborting uncommitted NO_OP operation due to leader change: 7.6
I20260709 13:20:22.774817 16221 raft_consensus.cc:2889] T 8cc959fb7f704fba9009c0595467d074 P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:20:22.784448 19549 raft_consensus.cc:2749] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:20:22.656461 19567 consensus_queue.cc:237] T af57996a8be84d398f0f023e8d08ef77 P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:22.784633 19404 consensus_queue.cc:237] T 7c2095a5f4ad49408dbc7f69c606120d P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:20:22.785959 19607 log.cc:927] Time spent T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
I20260709 13:20:22.711520 19526 consensus_queue.cc:1059] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.062s
I20260709 13:20:22.787847 16220 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57604 (request call id 2290) took 2123 ms. Trace:
I20260709 13:20:22.790557 16296 raft_consensus.cc:1240] T cdbd78acb4cf46298e0de8ba0d130e19 P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 7. Current term is 8. Ops: []
I20260709 13:20:22.791021 16223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5ad4eb281564936a59f41e387a47156" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:20:22.812863 19496 raft_consensus.cc:2804] T 434bff3082d141838068f323bdb0d1ed P cb82cc7bd6db46d980c75237a1402860 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:20:22.817116 16218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a378d7f796f94615b3b347119a9880fd" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:22.818289 16218 raft_consensus.cc:2393] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 8: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
I20260709 13:20:22.824999 16221 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57604 (request call id 2301) took 1206 ms. Trace:
I20260709 13:20:22.825672 16221 rpcz_store.cc:276] 0709 13:20:21.618242 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:22.614634 (+996392us) service_pool.cc:224] Handling call
0709 13:20:22.641081 (+ 26447us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:22.806933 (+165852us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:22.824906 (+ 17973us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:20:22.789583 16044 catalog_manager.cc:5697] T 66f75124215f4ca6bef831e174a792d8 P 1ca2725d290e47d68b1a4b609c9816ac reported cstate change: term changed from 3 to 6, leader changed from 647a343a542d4a139fe106c19095129b (127.14.177.129) to 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130). New cstate: current_term: 6 leader_uuid: "1ca2725d290e47d68b1a4b609c9816ac" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:20:22.775276 19421 raft_consensus.cc:493] T 84f2da98864b4425bf7c0055951fdd4b P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Starting leader election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:20:22.900796 16223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c61a39fd4f84424b129c1a6f9ba404d" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:22.901746 19514 raft_consensus.cc:515] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:22.903716 16221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a96373bb0e14dce8a266b2fc6c58507" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 8 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:22.977524 16221 raft_consensus.cc:2410] T 5a96373bb0e14dce8a266b2fc6c58507 P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 for term 8 because replica has last-logged OpId of term: 6 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 2.
I20260709 13:20:22.904633 16296 raft_consensus.cc:1240] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 7. Current term is 8. Ops: []
I20260709 13:20:22.794829 16220 rpcz_store.cc:276] 0709 13:20:20.657390 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:22.423151 (+1765761us) service_pool.cc:224] Handling call
0709 13:20:22.465522 (+ 42371us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:22.465706 (+   184us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:20:22.497042 (+ 31336us) log.cc:844] Serialized 14 byte log entry
0709 13:20:22.774712 (+277670us) log.cc:844] Serialized 14 byte log entry
0709 13:20:22.780687 (+  5975us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:20:22.780739 (+    52us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:20:22.780782 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:22.780850 (+    68us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:22.780989 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":15920,"prepare.queue_time_us":35936,"prepare.run_cpu_time_us":64,"prepare.run_wall_time_us":66,"raft.queue_time_us":25094,"raft.run_cpu_time_us":128,"raft.run_wall_time_us":5229,"spinlock_wait_cycles":68224768,"thread_start_us":266305,"threads_started":1}
I20260709 13:20:22.909430 19526 raft_consensus.cc:3055] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 1ca2725d290e47d68b1a4b609c9816ac [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:20:22.989154 19526 raft_consensus.cc:740] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 1ca2725d290e47d68b1a4b609c9816ac [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:20:22.920869 19576 consensus_queue.cc:237] T 3a7ca61592e848ea9b0fb81a2ce33cdc P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:22.993450 16044 catalog_manager.cc:5697] T c2f5f89186204242bf377f7c42b2e2da P 1ca2725d290e47d68b1a4b609c9816ac reported cstate change: term changed from 5 to 7, leader changed from 90f3da57796b488c9087a2f523adf045 (127.14.177.131) to 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130). New cstate: current_term: 7 leader_uuid: "1ca2725d290e47d68b1a4b609c9816ac" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } health_report { overall_health: UNKNOWN } } }
I20260709 13:20:22.943821 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18ad7e3af5504571a69d10c00750a01c" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:22.997017 16299 raft_consensus.cc:2468] T 18ad7e3af5504571a69d10c00750a01c P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 7.
I20260709 13:20:22.777777 19509 raft_consensus.cc:515] T 746aabf856ab401c9ef5e859371e7b67 P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:22.744501 19522 consensus_queue.cc:1048] T 1df93779209748b9860f6d893dd3de53 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.039s
W20260709 13:20:22.953563 16192 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:22.793841 19364 leader_election.cc:290] T ee9ed67006144797b1de6ea8c2013a68 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 6 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:22.768054 19113 raft_consensus.cc:515] T 829b4ad8c5a54eababeea1181b960d4b P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:22.982726 19514 leader_election.cc:290] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 9 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:22.983896 16221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cc959fb7f704fba9009c0595467d074" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:23.062371 16221 raft_consensus.cc:2468] T 8cc959fb7f704fba9009c0595467d074 P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 8.
I20260709 13:20:22.985036 16220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68a6838d82ef4cac83507f906139d90f" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:22.979943 19421 raft_consensus.cc:3060] T 84f2da98864b4425bf7c0055951fdd4b P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:20:22.986384 19604 log.cc:927] Time spent T 02838c86531d4f98bf58d1c22b8baf77 P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.157s	user 0.000s	sys 0.000s
W20260709 13:20:22.986840 19378 consensus_meta.cc:287] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 1ca2725d290e47d68b1a4b609c9816ac: Time spent flushing consensus metadata: real 0.552s	user 0.015s	sys 0.005s
I20260709 13:20:22.987703 16223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a878999c0024d2abe517484180290fc" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:20:22.770411 19414 raft_consensus.cc:515] T 8d4ab870a05347868956aa8b841b2981 P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:23.141533 19414 leader_election.cc:290] T 8d4ab870a05347868956aa8b841b2981 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:20:23.037675 16291 raft_consensus.cc:2468] T ecc1e363230043fe9fef390f0b979f20 P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 8.
I20260709 13:20:23.142961 16291 raft_consensus.cc:1240] T 8638ee363273426db1393dd81e5c7b72 P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 7. Current term is 8. Ops: [7.8-7.8]
W20260709 13:20:23.122108 16189 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:23.146005 16296 raft_consensus.cc:1240] T ecc1e363230043fe9fef390f0b979f20 P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 7. Current term is 8. Ops: [2.4-7.5]
I20260709 13:20:22.990348 19526 consensus_queue.cc:260] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:22.926033 19450 raft_consensus.cc:515] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } }
I20260709 13:20:23.138522 16221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada5a004ba7c46819051e3ccec71ffdb" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:23.133332 16299 raft_consensus.cc:1217] T c0eab25ce4c64e018a041beeda2867db P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Deduplicated request from leader. Original: 2.2->[4.3-8.5]   Dedup: 4.3->[6.4-8.5]
I20260709 13:20:23.153224 16299 pending_rounds.cc:85] T c0eab25ce4c64e018a041beeda2867db P 90f3da57796b488c9087a2f523adf045: Aborting all ops after (but not including) 3
I20260709 13:20:23.153556 16299 pending_rounds.cc:107] T c0eab25ce4c64e018a041beeda2867db P 90f3da57796b488c9087a2f523adf045: Aborting uncommitted NO_OP operation due to leader change: 7.4
I20260709 13:20:23.153843 16299 raft_consensus.cc:2889] T c0eab25ce4c64e018a041beeda2867db P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:20:23.155028 16044 catalog_manager.cc:5697] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac reported cstate change: term changed from 5 to 6, leader changed from cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) to 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130). New cstate: current_term: 6 leader_uuid: "1ca2725d290e47d68b1a4b609c9816ac" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:20:23.127082 19233 raft_consensus.cc:493] T 84f2da98864b4425bf7c0055951fdd4b P 1ca2725d290e47d68b1a4b609c9816ac [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:23.126554 19496 raft_consensus.cc:697] T 434bff3082d141838068f323bdb0d1ed P cb82cc7bd6db46d980c75237a1402860 [term 9 LEADER]: Becoming Leader. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:20:23.173288 16291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1df93779209748b9860f6d893dd3de53" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:23.176280 16291 raft_consensus.cc:2468] T 1df93779209748b9860f6d893dd3de53 P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 8.
I20260709 13:20:23.179973 16223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18ad7e3af5504571a69d10c00750a01c" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
W20260709 13:20:23.183053 19601 log.cc:927] Time spent T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 0.532s	user 0.002s	sys 0.000s
W20260709 13:20:23.185211 19606 log.cc:927] Time spent T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 0.449s	user 0.008s	sys 0.001s
I20260709 13:20:23.191603 19589 raft_consensus.cc:493] T b87147731a68400e8cce9b879daa7c5d P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:23.192422 19572 raft_consensus.cc:3055] T 23f82224f4564fea93572e8206bad1fe P cb82cc7bd6db46d980c75237a1402860 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:20:23.192340 19589 raft_consensus.cc:515] T b87147731a68400e8cce9b879daa7c5d P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:23.193135 19572 raft_consensus.cc:740] T 23f82224f4564fea93572e8206bad1fe P cb82cc7bd6db46d980c75237a1402860 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:20:23.180697 16223 raft_consensus.cc:2468] T 18ad7e3af5504571a69d10c00750a01c P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 7.
W20260709 13:20:23.201164 16263 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:23.203896 19571 leader_election.cc:290] T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
W20260709 13:20:23.204962 16790 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.512s	user 0.008s	sys 0.016s
W20260709 13:20:23.205462 16790 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.512s	user 0.008s	sys 0.016s
W20260709 13:20:23.206318 16790 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:20:23.207182 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 426488us
I20260709 13:20:23.210758 16217 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57604 (request call id 2296) took 1663 ms. Trace:
I20260709 13:20:23.212110 16044 catalog_manager.cc:5697] T 746aabf856ab401c9ef5e859371e7b67 P 1ca2725d290e47d68b1a4b609c9816ac reported cstate change: term changed from 4 to 5, leader changed from 647a343a542d4a139fe106c19095129b (127.14.177.129) to 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130). New cstate: current_term: 5 leader_uuid: "1ca2725d290e47d68b1a4b609c9816ac" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:20:23.222918 16223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806d7690c2b644959152074bb2cab0a7" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:20:23.168129 19233 raft_consensus.cc:515] T 84f2da98864b4425bf7c0055951fdd4b P 1ca2725d290e47d68b1a4b609c9816ac [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:20:23.230250 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 516182us
I20260709 13:20:23.194377 19572 consensus_queue.cc:260] T 23f82224f4564fea93572e8206bad1fe P cb82cc7bd6db46d980c75237a1402860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.6, Last appended by leader: 6, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:20:23.207435 19568 consensus_meta.cc:287] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860: Time spent flushing consensus metadata: real 0.706s	user 0.012s	sys 0.000s
I20260709 13:20:23.208777 19589 leader_election.cc:290] T b87147731a68400e8cce9b879daa7c5d P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
W20260709 13:20:23.244952 16190 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:23.245411 16044 catalog_manager.cc:5697] T e4dc6b0541ca431ebc757beac00521f1 P 1ca2725d290e47d68b1a4b609c9816ac reported cstate change: term changed from 3 to 4, leader changed from 647a343a542d4a139fe106c19095129b (127.14.177.129) to 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130). New cstate: current_term: 4 leader_uuid: "1ca2725d290e47d68b1a4b609c9816ac" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } health_report { overall_health: UNKNOWN } } }
I20260709 13:20:23.235584 19549 raft_consensus.cc:493] T fffab26427e24d398e2695aad40ca134 P 1ca2725d290e47d68b1a4b609c9816ac [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:20:23.262199 16223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:20:23.255657 19549 raft_consensus.cc:515] T fffab26427e24d398e2695aad40ca134 P 1ca2725d290e47d68b1a4b609c9816ac [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:23.237217 19568 raft_consensus.cc:515] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:23.264961 19568 leader_election.cc:290] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 10 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:23.265130 19549 leader_election.cc:290] T fffab26427e24d398e2695aad40ca134 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:23.265822 16825 raft_consensus.cc:2393] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 10: Already voted for candidate cb82cc7bd6db46d980c75237a1402860 in this term.
I20260709 13:20:23.260380 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a878999c0024d2abe517484180290fc" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:20:23.268455 16150 raft_consensus.cc:3060] T 8a878999c0024d2abe517484180290fc P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:20:23.216527 16217 rpcz_store.cc:276] 0709 13:20:21.547527 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:22.544055 (+996528us) service_pool.cc:224] Handling call
0709 13:20:22.591924 (+ 47869us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:22.592183 (+   259us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:20:22.592481 (+   298us) log.cc:844] Serialized 14 byte log entry
0709 13:20:23.210023 (+617542us) log.cc:844] Serialized 14 byte log entry
0709 13:20:23.210272 (+   249us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:20:23.210342 (+    70us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:20:23.210404 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:23.210498 (+    94us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:23.210676 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":89607,"prepare.queue_time_us":10329,"prepare.run_cpu_time_us":68,"prepare.run_wall_time_us":68,"raft.queue_time_us":9049,"raft.run_cpu_time_us":110,"raft.run_wall_time_us":8729,"spinlock_wait_cycles":3840,"thread_start_us":605436,"threads_started":2,"wal-append.queue_time_us":401103}
I20260709 13:20:23.270921 19113 leader_election.cc:290] T 829b4ad8c5a54eababeea1181b960d4b P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:23.235657 19524 consensus_queue.cc:1048] T 8cc959fb7f704fba9009c0595467d074 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.029s
I20260709 13:20:23.274506 16151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18ad7e3af5504571a69d10c00750a01c" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:23.274567 16146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cc959fb7f704fba9009c0595467d074" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:23.278589 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5718c623ebee4d2bb770f0f9ddb2f164" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:20:23.277100 16790 leader_election.cc:304] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:20:23.261008 19532 raft_consensus.cc:493] T 8d4ab870a05347868956aa8b841b2981 P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:20:23.272163 19572 raft_consensus.cc:3060] T 23f82224f4564fea93572e8206bad1fe P cb82cc7bd6db46d980c75237a1402860 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:20:23.284714 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c61a39fd4f84424b129c1a6f9ba404d" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:23.285593 16223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66f75124215f4ca6bef831e174a792d8" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:23.286207 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02838c86531d4f98bf58d1c22b8baf77" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:23.286254 16223 raft_consensus.cc:2468] T 66f75124215f4ca6bef831e174a792d8 P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 8.
I20260709 13:20:23.288671 16790 leader_election.cc:304] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:20:23.294258 16146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a96373bb0e14dce8a266b2fc6c58507" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 8 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:23.296159 19233 leader_election.cc:290] T 84f2da98864b4425bf7c0055951fdd4b P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:23.296778 19233 consensus_queue.cc:1059] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:20:23.298086 19113 raft_consensus.cc:3055] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:20:23.298488 19113 raft_consensus.cc:740] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:20:23.298930 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68a6838d82ef4cac83507f906139d90f" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:23.299659 16149 raft_consensus.cc:2468] T 68a6838d82ef4cac83507f906139d90f P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 8.
I20260709 13:20:23.299353 19113 consensus_queue.cc:260] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:23.300607 19113 raft_consensus.cc:3060] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:20:23.310916 16192 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:23.312768 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4f65cdf00b748bb963ee6e81ef43b79" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:20:23.322254 16817 raft_consensus.cc:1240] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860 [term 8 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 7. Current term is 8. Ops: []
I20260709 13:20:23.323772 16821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a378d7f796f94615b3b347119a9880fd" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:20:23.325678 19551 leader_election.cc:290] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:23.316676 19585 raft_consensus.cc:493] T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:23.326534 19585 raft_consensus.cc:515] T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:22.615428 16148 rpcz_store.cc:276] 0709 13:20:20.961251 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:22.201457 (+1240206us) service_pool.cc:224] Handling call
0709 13:20:22.201780 (+   323us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:22.202076 (+   296us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:20:22.495146 (+293070us) log.cc:844] Serialized 14 byte log entry
0709 13:20:22.607916 (+112770us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:20:22.607986 (+    70us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:20:22.608050 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:22.608142 (+    92us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:22.608298 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":282833,"prepare.run_cpu_time_us":36,"prepare.run_wall_time_us":37,"raft.queue_time_us":7487,"thread_start_us":363349,"threads_started":2}
I20260709 13:20:23.328085 19609 raft_consensus.cc:493] T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:23.328806 19585 leader_election.cc:290] T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:20:23.328905 19609 raft_consensus.cc:515] T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:20:23.331686 19625 log.cc:927] Time spent T 7c2095a5f4ad49408dbc7f69c606120d P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 13:20:23.291913 19532 raft_consensus.cc:515] T 8d4ab870a05347868956aa8b841b2981 P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:23.332885 19609 leader_election.cc:290] T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:23.333942 19559 raft_consensus.cc:493] T 57a744f5aaf944678a63469cbf514015 P cb82cc7bd6db46d980c75237a1402860 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:23.334482 19532 leader_election.cc:290] T 8d4ab870a05347868956aa8b841b2981 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:23.335119 16820 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49892 (request call id 494) took 1059 ms. Trace:
I20260709 13:20:23.334638 19559 raft_consensus.cc:515] T 57a744f5aaf944678a63469cbf514015 P cb82cc7bd6db46d980c75237a1402860 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:23.335539 16820 rpcz_store.cc:276] 0709 13:20:22.276047 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:22.354273 (+ 78226us) service_pool.cc:224] Handling call
0709 13:20:23.334969 (+980696us) spinlock_profiling.cc:243] Waited 93.4 ms on lock 0x7b54006eebd4. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:20:23.335066 (+    97us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":214844800}
I20260709 13:20:23.336796 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434bff3082d141838068f323bdb0d1ed" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:20:23.337455 19559 leader_election.cc:290] T 57a744f5aaf944678a63469cbf514015 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:23.337930 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a9de347c711477e87c582d003f36925" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 10 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:20:23.289685 19620 raft_consensus.cc:2749] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:20:23.340476 19624 raft_consensus.cc:493] T 3884f864a96d428785895e5463e5728f P cb82cc7bd6db46d980c75237a1402860 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:20:23.341012 16823 raft_consensus.cc:1240] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 [term 8 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 7. Current term is 8. Ops: []
I20260709 13:20:23.340129 16816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee9ed67006144797b1de6ea8c2013a68" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:20:23.343891 16821 raft_consensus.cc:3060] T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:20:23.345672 16148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:20:23.346365 16148 raft_consensus.cc:3060] T ecc1e363230043fe9fef390f0b979f20 P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:20:23.319770 19568 raft_consensus.cc:493] T 806d7690c2b644959152074bb2cab0a7 P cb82cc7bd6db46d980c75237a1402860 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:20:23.292246 19509 leader_election.cc:290] T 746aabf856ab401c9ef5e859371e7b67 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:23.354584 16044 catalog_manager.cc:5697] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 1ca2725d290e47d68b1a4b609c9816ac reported cstate change: term changed from 4 to 7, leader changed from cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) to 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130). New cstate: current_term: 7 leader_uuid: "1ca2725d290e47d68b1a4b609c9816ac" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:20:23.307327 19589 raft_consensus.cc:493] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:23.363479 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9944d5e4b8c4109af9955c4f08c9095" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:23.274060 19496 consensus_queue.cc:237] T 434bff3082d141838068f323bdb0d1ed P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:23.373459 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5d59407eb545b895d8d922e0dd69fd" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:23.323239 19576 raft_consensus.cc:3060] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P cb82cc7bd6db46d980c75237a1402860 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:20:23.368023 16217 raft_consensus.cc:2468] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 8.
I20260709 13:20:23.339777 19559 raft_consensus.cc:493] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:20:23.369092 16816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5d59407eb545b895d8d922e0dd69fd" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:20:23.311685 19591 raft_consensus.cc:493] T ada5a004ba7c46819051e3ccec71ffdb P cb82cc7bd6db46d980c75237a1402860 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:20:23.408478 19591 raft_consensus.cc:515] T ada5a004ba7c46819051e3ccec71ffdb P cb82cc7bd6db46d980c75237a1402860 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:23.410539 19591 leader_election.cc:290] T ada5a004ba7c46819051e3ccec71ffdb P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:23.353935 19624 raft_consensus.cc:515] T 3884f864a96d428785895e5463e5728f P cb82cc7bd6db46d980c75237a1402860 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:23.380360 16817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af57996a8be84d398f0f023e8d08ef77" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:20:23.359320 19568 raft_consensus.cc:515] T 806d7690c2b644959152074bb2cab0a7 P cb82cc7bd6db46d980c75237a1402860 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:23.392268 19567 raft_consensus.cc:493] T 746aabf856ab401c9ef5e859371e7b67 P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:23.380815 19589 raft_consensus.cc:515] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:20:23.396248 19382 consensus_meta.cc:287] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045: Time spent flushing consensus metadata: real 0.867s	user 0.011s	sys 0.000s
I20260709 13:20:23.399457 19559 raft_consensus.cc:515] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:23.400339 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5d59407eb545b895d8d922e0dd69fd" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:23.425280 16217 raft_consensus.cc:2468] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 9.
I20260709 13:20:23.404357 16816 raft_consensus.cc:2468] T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 9.
I20260709 13:20:23.405031 19421 raft_consensus.cc:515] T 84f2da98864b4425bf7c0055951fdd4b P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:23.413681 16817 raft_consensus.cc:2393] T af57996a8be84d398f0f023e8d08ef77 P cb82cc7bd6db46d980c75237a1402860 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 9: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
I20260709 13:20:23.414359 16149 raft_consensus.cc:2468] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 9.
I20260709 13:20:23.417795 16821 raft_consensus.cc:2468] T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 8.
I20260709 13:20:23.419770 16150 raft_consensus.cc:2468] T 8a878999c0024d2abe517484180290fc P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 8.
I20260709 13:20:23.427201 16790 leader_election.cc:304] T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:20:23.428046 16822 raft_consensus.cc:1275] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 [term 6 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 13:20:23.431835 16044 catalog_manager.cc:5697] T 7c2095a5f4ad49408dbc7f69c606120d P 1ca2725d290e47d68b1a4b609c9816ac reported cstate change: term changed from 1 to 4, leader changed from 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130) to 90f3da57796b488c9087a2f523adf045 (127.14.177.131). New cstate: current_term: 4 leader_uuid: "90f3da57796b488c9087a2f523adf045" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } }
I20260709 13:20:23.435204 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada5a004ba7c46819051e3ccec71ffdb" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:23.439018 16145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57a744f5aaf944678a63469cbf514015" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:23.357805 19551 consensus_queue.cc:1059] T ecc1e363230043fe9fef390f0b979f20 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 3, Time since last communication: 0.067s
I20260709 13:20:23.441821 16295 raft_consensus.cc:2410] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac for term 8 because replica has last-logged OpId of term: 7 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 6.
I20260709 13:20:23.444439 16044 catalog_manager.cc:5697] T 18ad7e3af5504571a69d10c00750a01c P 1ca2725d290e47d68b1a4b609c9816ac reported cstate change: term changed from 3 to 4, leader changed from 647a343a542d4a139fe106c19095129b (127.14.177.129) to 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130). New cstate: current_term: 4 leader_uuid: "1ca2725d290e47d68b1a4b609c9816ac" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: HEALTHY } } }
I20260709 13:20:23.443428 16145 raft_consensus.cc:2468] T 57a744f5aaf944678a63469cbf514015 P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 5.
I20260709 13:20:23.427536 19567 raft_consensus.cc:515] T 746aabf856ab401c9ef5e859371e7b67 P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:23.456362 19589 leader_election.cc:290] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:20:23.457140 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a378d7f796f94615b3b347119a9880fd" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:23.457895 16217 raft_consensus.cc:2393] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 in current term 8: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
I20260709 13:20:23.458910 16217 raft_consensus.cc:1240] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 7. Current term is 8. Ops: [7.7-7.7]
I20260709 13:20:23.442144 19620 raft_consensus.cc:2804] T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:20:23.463326 19620 raft_consensus.cc:493] T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:23.463670 19620 raft_consensus.cc:3060] T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:20:23.464394 19624 leader_election.cc:290] T 3884f864a96d428785895e5463e5728f P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:23.466575 19568 leader_election.cc:290] T 806d7690c2b644959152074bb2cab0a7 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
W20260709 13:20:23.468044 19529 consensus_meta.cc:287] T 18ad7e3af5504571a69d10c00750a01c P 647a343a542d4a139fe106c19095129b: Time spent flushing consensus metadata: real 0.699s	user 0.010s	sys 0.000s
I20260709 13:20:23.468153 16291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c61a39fd4f84424b129c1a6f9ba404d" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:23.468366 19559 leader_election.cc:290] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:23.468896 16291 raft_consensus.cc:2468] T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 7.
I20260709 13:20:23.470124 16790 leader_election.cc:304] T 57a744f5aaf944678a63469cbf514015 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:20:23.471096 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806d7690c2b644959152074bb2cab0a7" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:23.481472 16291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18ad7e3af5504571a69d10c00750a01c" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:23.486541 16291 raft_consensus.cc:2468] T 18ad7e3af5504571a69d10c00750a01c P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 7.
I20260709 13:20:23.481921 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66ae176961b74813a58bc15c9c4c6f2a" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:23.482162 19559 raft_consensus.cc:2804] T 57a744f5aaf944678a63469cbf514015 P cb82cc7bd6db46d980c75237a1402860 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:20:23.482343 16145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3884f864a96d428785895e5463e5728f" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:23.484683 16218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806d7690c2b644959152074bb2cab0a7" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:23.485217 16266 leader_election.cc:304] T d2ab0ac8347641e6955e9bce7d90c1a2 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860
I20260709 13:20:23.489065 16142 raft_consensus.cc:1240] T 9e65676b21b748728ad39668cfe722ec P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 8. Current term is 9. Ops: [8.9-8.9]
W20260709 13:20:23.491899 16144 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40830 (request call id 1285) took 2359 ms (client timeout 1813 ms). Trace:
W20260709 13:20:23.492389 16144 rpcz_store.cc:269] 0709 13:20:21.127078 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:21.302006 (+174928us) service_pool.cc:224] Handling call
0709 13:20:23.485768 (+2183762us) spinlock_profiling.cc:243] Waited 711 ms on lock 0x7b54003f0f54. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:20:23.485912 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1634968448}
I20260709 13:20:23.495118 16266 leader_election.cc:304] T 57a744f5aaf944678a63469cbf514015 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
I20260709 13:20:23.506052 16217 raft_consensus.cc:1275] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:20:23.509672 16266 leader_election.cc:304] T 2794af5b99984efebdee564035841cb0 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
I20260709 13:20:23.495548 16148 raft_consensus.cc:2468] T ecc1e363230043fe9fef390f0b979f20 P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 8.
I20260709 13:20:23.511646 16291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c61a39fd4f84424b129c1a6f9ba404d" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:23.512259 16291 raft_consensus.cc:2468] T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 7.
W20260709 13:20:23.493472 19478 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.848s	user 0.006s	sys 0.012s
I20260709 13:20:23.489462 19559 raft_consensus.cc:493] T 57a744f5aaf944678a63469cbf514015 P cb82cc7bd6db46d980c75237a1402860 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:23.537081 16291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cc959fb7f704fba9009c0595467d074" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:23.537874 16291 raft_consensus.cc:2468] T 8cc959fb7f704fba9009c0595467d074 P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 8.
I20260709 13:20:23.513913 19414 raft_consensus.cc:2749] T 57a744f5aaf944678a63469cbf514015 P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
W20260709 13:20:23.514524 19478 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.871s	user 0.006s	sys 0.012s
I20260709 13:20:23.541465 16291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a378d7f796f94615b3b347119a9880fd" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:20:23.542069 16291 raft_consensus.cc:3060] T a378d7f796f94615b3b347119a9880fd P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:20:23.491544 19529 raft_consensus.cc:515] T 18ad7e3af5504571a69d10c00750a01c P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } }
I20260709 13:20:23.533927 19559 raft_consensus.cc:3060] T 57a744f5aaf944678a63469cbf514015 P cb82cc7bd6db46d980c75237a1402860 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:20:23.507483 19382 consensus_queue.cc:1059] T 8c11074579b046d99e585c2818e18a2c P 90f3da57796b488c9087a2f523adf045 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.034s
W20260709 13:20:23.550936 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 65056.6us
I20260709 13:20:23.555526 16151 raft_consensus.cc:2393] T 18ad7e3af5504571a69d10c00750a01c P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 8: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
I20260709 13:20:23.549010 19414 raft_consensus.cc:2749] T 2794af5b99984efebdee564035841cb0 P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:20:23.566936 16265 leader_election.cc:304] T 5a959dbecc2648eb85d1cc550f5a3245 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 647a343a542d4a139fe106c19095129b
I20260709 13:20:23.555112 19576 raft_consensus.cc:2749] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P cb82cc7bd6db46d980c75237a1402860 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:20:23.558112 19382 raft_consensus.cc:3055] T 8c11074579b046d99e585c2818e18a2c P 90f3da57796b488c9087a2f523adf045 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:20:23.575416 19382 raft_consensus.cc:740] T 8c11074579b046d99e585c2818e18a2c P 90f3da57796b488c9087a2f523adf045 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:20:23.576529 19382 consensus_queue.cc:260] T 8c11074579b046d99e585c2818e18a2c P 90f3da57796b488c9087a2f523adf045 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.9, Last appended by leader: 9, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:23.578122 19382 raft_consensus.cc:3060] T 8c11074579b046d99e585c2818e18a2c P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:20:23.580667 19414 raft_consensus.cc:2804] T 5a959dbecc2648eb85d1cc550f5a3245 P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:20:23.581149 19414 raft_consensus.cc:493] T 5a959dbecc2648eb85d1cc550f5a3245 P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:20:23.581665 19414 raft_consensus.cc:3060] T 5a959dbecc2648eb85d1cc550f5a3245 P 90f3da57796b488c9087a2f523adf045 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:20:23.584033 19567 leader_election.cc:290] T 746aabf856ab401c9ef5e859371e7b67 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:23.593585 19345 leader_election.cc:290] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 6 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:23.604831 16222 raft_consensus.cc:1275] T b6f07842d3aa47858df5665874c86f3a P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
W20260709 13:20:23.618191 19565 log.cc:927] Time spent T d16dfad0c3b540e5b4b86492ddf87208 P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 2.413s	user 0.008s	sys 0.010s
I20260709 13:20:23.590794 19512 leader_election.cc:290] T 8a878999c0024d2abe517484180290fc P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
W20260709 13:20:23.618371 19635 log.cc:927] Time spent T 5a96373bb0e14dce8a266b2fc6c58507 P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 0.082s	user 0.000s	sys 0.000s
W20260709 13:20:23.633334 19632 log.cc:927] Time spent T af57996a8be84d398f0f023e8d08ef77 P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 0.349s	user 0.002s	sys 0.000s
W20260709 13:20:23.663612 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 96715.1us
I20260709 13:20:23.680759 16265 leader_election.cc:304] T c0eab25ce4c64e018a041beeda2867db P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
W20260709 13:20:23.694205 16790 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:20:23.702611 19636 log.cc:927] Time spent T 8d4ab870a05347868956aa8b841b2981 P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.145s	user 0.000s	sys 0.001s
W20260709 13:20:23.708662 19638 log.cc:927] Time spent T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 0.135s	user 0.001s	sys 0.000s
I20260709 13:20:23.737221 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66f75124215f4ca6bef831e174a792d8" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:23.767758 16222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62fe3346cd3c42c4a55e14af9073bca4" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:23.768070 16150 raft_consensus.cc:2410] T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac for term 9 because replica has last-logged OpId of term: 7 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
W20260709 13:20:23.769214 16266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 259633us
I20260709 13:20:23.738651 16144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18ad7e3af5504571a69d10c00750a01c" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:23.764253 16146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1df93779209748b9860f6d893dd3de53" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:23.581149 16291 raft_consensus.cc:2468] T a378d7f796f94615b3b347119a9880fd P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 8.
I20260709 13:20:23.738550 16147 raft_consensus.cc:1240] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 7. Current term is 8. Ops: []
I20260709 13:20:23.769959 16266 leader_election.cc:304] T 66ae176961b74813a58bc15c9c4c6f2a P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860
I20260709 13:20:23.770897 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8158bbd17b994d2eb96f560bd0f1b88d" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:23.771929 16266 leader_election.cc:304] T 31969392ccdb43b491911a319e8bc7e1 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
W20260709 13:20:23.771662 16263 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:23.773422 16145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a96373bb0e14dce8a266b2fc6c58507" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:23.776548 16291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a378d7f796f94615b3b347119a9880fd" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:23.780997 16291 raft_consensus.cc:2376] T a378d7f796f94615b3b347119a9880fd P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 8. Re-sending same reply.
W20260709 13:20:23.764253 19605 log.cc:927] Time spent T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 1.122s	user 0.008s	sys 0.004s
I20260709 13:20:23.783843 16149 raft_consensus.cc:1240] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 7. Current term is 8. Ops: [7.9-7.9]
I20260709 13:20:23.785744 16143 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40830 (request call id 1289) took 2120 ms. Trace:
I20260709 13:20:23.786136 16143 rpcz_store.cc:276] 0709 13:20:21.665016 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:21.993026 (+328010us) service_pool.cc:224] Handling call
0709 13:20:23.785566 (+1792540us) spinlock_profiling.cc:243] Waited 38.9 ms on lock 0x7b540011add4. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:20:23.785688 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":89525504}
I20260709 13:20:23.795751 16144 raft_consensus.cc:1240] T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 6. Current term is 7. Ops: []
I20260709 13:20:23.734586 19540 raft_consensus.cc:493] T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:23.737509 19620 raft_consensus.cc:515] T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:23.795670 16147 raft_consensus.cc:1240] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 6. Current term is 8. Ops: [6.7-6.7]
I20260709 13:20:23.765692 19596 leader_election.cc:290] T 2794af5b99984efebdee564035841cb0 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:23.802309 16790 leader_election.cc:304] T c9944d5e4b8c4109af9955c4f08c9095 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 90f3da57796b488c9087a2f523adf045
I20260709 13:20:23.807008 16146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c61a39fd4f84424b129c1a6f9ba404d" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:23.769364 19465 leader_election.cc:290] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 8 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
W20260709 13:20:23.814710 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 134070us
I20260709 13:20:23.771019 19382 raft_consensus.cc:493] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:23.738348 19418 raft_consensus.cc:493] T ee9ed67006144797b1de6ea8c2013a68 P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:20:23.820734 16291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a96373bb0e14dce8a266b2fc6c58507" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:23.821408 16291 raft_consensus.cc:2468] T 5a96373bb0e14dce8a266b2fc6c58507 P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 7.
I20260709 13:20:23.820734 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5ad4eb281564936a59f41e387a47156" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:23.772984 19568 consensus_queue.cc:1048] T 74ca8140f35c4ae29e8a664f6bdfc829 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.043s
I20260709 13:20:23.783759 19496 consensus_queue.cc:1059] T ee9ed67006144797b1de6ea8c2013a68 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.042s
I20260709 13:20:23.833544 16291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f2da98864b4425bf7c0055951fdd4b" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:23.835888 16291 raft_consensus.cc:2468] T 84f2da98864b4425bf7c0055951fdd4b P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 6.
W20260709 13:20:23.839890 16266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 68037us
I20260709 13:20:23.840548 16147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af57996a8be84d398f0f023e8d08ef77" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:20:23.844506 16790 leader_election.cc:304] T 806d7690c2b644959152074bb2cab0a7 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:20:23.845403 16300 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40474 (request call id 506) took 1407 ms. Trace:
I20260709 13:20:23.846189 16300 rpcz_store.cc:276] 0709 13:20:22.437571 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:22.438004 (+   433us) service_pool.cc:224] Handling call
0709 13:20:22.438298 (+   294us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:22.438544 (+   246us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:20:22.438876 (+   332us) log.cc:844] Serialized 14 byte log entry
0709 13:20:23.596272 (+1157396us) spinlock_profiling.cc:243] Waited 15.1 ms on lock 0x7b18002fa160. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b693ade1 00007f69b69475b1 00007f69ada723f3 00007f69ada747c1 00007f69ada74779 00007f69ada74709 00007f69ada746d1 00007f69ada739cd 000000000038f354 000000000038f199 00007f69ada5583a
0709 13:20:23.626713 (+ 30441us) spinlock_profiling.cc:243] Waited 4.99 ms on lock 0x7b18002fa160. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b693ade1 00007f69b69475b1 00007f69ad9ecc0a 00007f69b712c219 00007f69ada74a40 00007f69ada6f4e4 00007f69ada6f455 00007f69b45573c7 00007f69b0b699a7 00007f69b0b790bf 00007f69b0b79038
0709 13:20:23.627049 (+   336us) log.cc:844] Serialized 14 byte log entry
0709 13:20:23.844832 (+217783us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:20:23.844902 (+    70us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:20:23.844960 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:23.845166 (+   206us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:23.845325 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":53873,"prepare.queue_time_us":1206083,"prepare.run_cpu_time_us":64,"prepare.run_wall_time_us":65,"raft.queue_time_us":16068,"raft.run_cpu_time_us":132,"raft.run_wall_time_us":22578,"spinlock_wait_cycles":46223872,"thread_start_us":217330,"threads_started":2}
I20260709 13:20:23.849905 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d16dfad0c3b540e5b4b86492ddf87208" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:23.850909 16300 raft_consensus.cc:2468] T d16dfad0c3b540e5b4b86492ddf87208 P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 7.
I20260709 13:20:23.828507 19113 raft_consensus.cc:493] T b6f07842d3aa47858df5665874c86f3a P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:23.856624 16291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e4bfb9090c8460eb14fa7e0d818e7f1" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:20:23.822403 19642 raft_consensus.cc:2749] T 66ae176961b74813a58bc15c9c4c6f2a P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:20:23.858150 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74ca8140f35c4ae29e8a664f6bdfc829" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:23.830650 19450 leader_election.cc:290] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 8 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:23.860361 16146 raft_consensus.cc:1240] T ecc1e363230043fe9fef390f0b979f20 P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 7. Current term is 8. Ops: [7.5-7.5]
I20260709 13:20:23.863996 16300 raft_consensus.cc:2410] T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac for term 9 because replica has last-logged OpId of term: 8 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 7.
I20260709 13:20:23.866065 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c2095a5f4ad49408dbc7f69c606120d" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:23.866063 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d16dfad0c3b540e5b4b86492ddf87208" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:23.839083 19610 raft_consensus.cc:2804] T c9944d5e4b8c4109af9955c4f08c9095 P cb82cc7bd6db46d980c75237a1402860 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:20:23.819336 19540 raft_consensus.cc:515] T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:23.868337 16300 raft_consensus.cc:2410] T 7c2095a5f4ad49408dbc7f69c606120d P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac for term 8 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
I20260709 13:20:23.859326 19587 raft_consensus.cc:2749] T 806d7690c2b644959152074bb2cab0a7 P cb82cc7bd6db46d980c75237a1402860 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:20:23.869328 16291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ab0ac8347641e6955e9bce7d90c1a2" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:23.852121 19589 raft_consensus.cc:2749] T d2ab0ac8347641e6955e9bce7d90c1a2 P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:20:23.843137 19529 leader_election.cc:290] T 18ad7e3af5504571a69d10c00750a01c P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 8 election: Requested vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:20:23.878734 16147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c2095a5f4ad49408dbc7f69c606120d" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:23.883874 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fffab26427e24d398e2695aad40ca134" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:23.828466 19382 raft_consensus.cc:515] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:23.888826 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee9ed67006144797b1de6ea8c2013a68" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:20:23.888826 16291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5ad4eb281564936a59f41e387a47156" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:23.889890 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02838c86531d4f98bf58d1c22b8baf77" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 9 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:23.890250 16291 raft_consensus.cc:2393] T e5ad4eb281564936a59f41e387a47156 P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 9: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
I20260709 13:20:23.875415 19610 raft_consensus.cc:493] T c9944d5e4b8c4109af9955c4f08c9095 P cb82cc7bd6db46d980c75237a1402860 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:23.834748 19418 raft_consensus.cc:515] T ee9ed67006144797b1de6ea8c2013a68 P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:23.878748 19651 raft_consensus.cc:493] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:23.901877 16224 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57604 (request call id 2303) took 2283 ms. Trace:
I20260709 13:20:23.902419 16224 rpcz_store.cc:276] 0709 13:20:21.618756 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:22.379028 (+760272us) service_pool.cc:224] Handling call
0709 13:20:23.901618 (+1522590us) spinlock_profiling.cc:243] Waited 64.9 ms on lock 0x7b5400394754. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:20:23.901791 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":149360512}
I20260709 13:20:23.905606 16291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8158bbd17b994d2eb96f560bd0f1b88d" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:23.906716 16291 raft_consensus.cc:2410] T 8158bbd17b994d2eb96f560bd0f1b88d P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac for term 9 because replica has last-logged OpId of term: 6 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
W20260709 13:20:23.907469 16266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 66765.9us
I20260709 13:20:23.864667 19113 raft_consensus.cc:515] T b6f07842d3aa47858df5665874c86f3a P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:23.890623 19407 raft_consensus.cc:493] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
W20260709 13:20:23.919263 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 81476.7us
I20260709 13:20:23.919569 16144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62fe3346cd3c42c4a55e14af9073bca4" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:23.920506 16149 raft_consensus.cc:3060] T ee9ed67006144797b1de6ea8c2013a68 P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:20:23.920758 16144 raft_consensus.cc:2468] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 10.
I20260709 13:20:23.923028 16293 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48674 (request call id 2381) took 1860 ms. Trace:
I20260709 13:20:23.897495 19610 raft_consensus.cc:3060] T c9944d5e4b8c4109af9955c4f08c9095 P cb82cc7bd6db46d980c75237a1402860 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:20:23.923449 16293 rpcz_store.cc:276] 0709 13:20:22.062574 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:22.062729 (+   155us) service_pool.cc:224] Handling call
0709 13:20:22.065714 (+  2985us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:20:22.066076 (+   362us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:20:22.174018 (+107942us) log.cc:844] Serialized 14 byte log entry
0709 13:20:22.532218 (+358200us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:20:22.547336 (+ 15118us) log.cc:844] Serialized 52 byte log entry
0709 13:20:22.547709 (+   373us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:20:22.547745 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:22.547816 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:20:23.739792 (+1191976us) spinlock_profiling.cc:243] Waited 53.4 ms on lock 0x7b4400022b4c. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69ad8fadb1 00007f69ad998c85 00007f69ad9996a2 00007f69b070b1db 00007f69b0706aaf 00007f69b06fa86e 00007f69b06f9bb6 00007f69b06f97d8 00007f69b0703221 00007f69b07031d9 00007f69b0703169
0709 13:20:23.920009 (+180217us) raft_consensus.cc:1685] finished
0709 13:20:23.922803 (+  2794us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:23.922968 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":1771,"prepare.run_cpu_time_us":29,"prepare.run_wall_time_us":30,"raft.queue_time_us":5277,"raft.run_cpu_time_us":245,"raft.run_wall_time_us":5669,"spinlock_wait_cycles":122791424,"thread_start_us":383116,"threads_started":2,"wal-append.queue_time_us":350674}
I20260709 13:20:23.898890 19576 raft_consensus.cc:493] T 829b4ad8c5a54eababeea1181b960d4b P cb82cc7bd6db46d980c75237a1402860 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:23.897804 19642 raft_consensus.cc:2749] T 31969392ccdb43b491911a319e8bc7e1 P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:20:23.926822 16226 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33050 (request call id 566) took 3064 ms (client timeout 2903 ms). Trace:
I20260709 13:20:23.900434 19587 raft_consensus.cc:493] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:23.895359 19653 raft_consensus.cc:2749] T c0eab25ce4c64e018a041beeda2867db P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:20:23.898195 19645 raft_consensus.cc:493] T 3884f864a96d428785895e5463e5728f P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:20:23.931878 16144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a878999c0024d2abe517484180290fc" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
W20260709 13:20:23.930725 16226 rpcz_store.cc:269] 0709 13:20:20.873462 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:22.324841 (+1451379us) service_pool.cc:224] Handling call
0709 13:20:23.926579 (+1601738us) spinlock_profiling.cc:243] Waited 157 ms on lock 0x7b5400367c54. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:20:23.926732 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":361821056}
I20260709 13:20:23.935801 16265 leader_election.cc:304] T 8cc959fb7f704fba9009c0595467d074 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:20:23.938021 19559 raft_consensus.cc:515] T 57a744f5aaf944678a63469cbf514015 P cb82cc7bd6db46d980c75237a1402860 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:23.939453 16147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5d59407eb545b895d8d922e0dd69fd" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:20:23.940083 16147 raft_consensus.cc:3060] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:20:23.941149 19645 raft_consensus.cc:515] T 3884f864a96d428785895e5463e5728f P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:23.943668 16226 raft_consensus.cc:1240] T ee9ed67006144797b1de6ea8c2013a68 P 1ca2725d290e47d68b1a4b609c9816ac [term 6 FOLLOWER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 5. Current term is 6. Ops: []
I20260709 13:20:23.943778 19645 leader_election.cc:290] T 3884f864a96d428785895e5463e5728f P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:23.946045 16144 raft_consensus.cc:2393] T 8a878999c0024d2abe517484180290fc P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 8: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
I20260709 13:20:23.948786 19148 consensus_queue.cc:1059] T ee9ed67006144797b1de6ea8c2013a68 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.005s
I20260709 13:20:23.954332 16146 raft_consensus.cc:1240] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 7. Current term is 8. Ops: []
W20260709 13:20:23.956151 19643 log.cc:927] Time spent T 1df93779209748b9860f6d893dd3de53 P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
I20260709 13:20:23.956523 19113 leader_election.cc:290] T b6f07842d3aa47858df5665874c86f3a P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:23.957794 19652 raft_consensus.cc:493] T 8158bbd17b994d2eb96f560bd0f1b88d P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:20:23.954207 19654 log.cc:927] Time spent T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.095s	user 0.001s	sys 0.000s
I20260709 13:20:23.961980 16265 leader_election.cc:304] T e5ad4eb281564936a59f41e387a47156 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:20:23.929955 19407 raft_consensus.cc:515] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:23.909176 19651 raft_consensus.cc:515] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:23.969652 16265 leader_election.cc:304] T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:20:23.967796 16144 raft_consensus.cc:3055] T 434bff3082d141838068f323bdb0d1ed P 647a343a542d4a139fe106c19095129b [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:20:24.021149 16144 raft_consensus.cc:740] T 434bff3082d141838068f323bdb0d1ed P 647a343a542d4a139fe106c19095129b [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:20:23.957376 19113 raft_consensus.cc:493] T 8c11074579b046d99e585c2818e18a2c P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:20:23.973843 19540 leader_election.cc:290] T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:23.975524 19407 leader_election.cc:290] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
W20260709 13:20:23.976939 19647 log.cc:927] Time spent T d2ab0ac8347641e6955e9bce7d90c1a2 P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 13:20:23.981050 19578 leader_election.cc:290] T 66f75124215f4ca6bef831e174a792d8 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:20:24.021798 16144 consensus_queue.cc:260] T 434bff3082d141838068f323bdb0d1ed P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:24.043459 16144 raft_consensus.cc:3060] T 434bff3082d141838068f323bdb0d1ed P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:20:23.985199 19609 raft_consensus.cc:3055] T ee9ed67006144797b1de6ea8c2013a68 P cb82cc7bd6db46d980c75237a1402860 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:20:23.967730 19645 raft_consensus.cc:2749] T e5ad4eb281564936a59f41e387a47156 P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:20:23.988476 16148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66f75124215f4ca6bef831e174a792d8" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:24.055820 16192 leader_election.cc:304] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:20:23.995537 19148 raft_consensus.cc:493] T b87147731a68400e8cce9b879daa7c5d P cb82cc7bd6db46d980c75237a1402860 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:20:23.997874 16221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66f75124215f4ca6bef831e174a792d8" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:24.057662 16221 raft_consensus.cc:2468] T 66f75124215f4ca6bef831e174a792d8 P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 8.
I20260709 13:20:23.999259 19526 raft_consensus.cc:3060] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:20:24.060431 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3884f864a96d428785895e5463e5728f" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:24.061156 16293 raft_consensus.cc:2468] T 3884f864a96d428785895e5463e5728f P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 6.
I20260709 13:20:23.999687 19382 leader_election.cc:290] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:24.040688 19653 raft_consensus.cc:2749] T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:20:24.003870 16142 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46224 (request call id 462) took 3243 ms. Trace:
I20260709 13:20:24.004457 19559 leader_election.cc:290] T 57a744f5aaf944678a63469cbf514015 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 7 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:23.966593 19642 raft_consensus.cc:2804] T 8cc959fb7f704fba9009c0595467d074 P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:20:24.070609 19642 raft_consensus.cc:493] T 8cc959fb7f704fba9009c0595467d074 P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:20:24.006138 16295 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48684 (request call id 2255) took 1549 ms (client timeout 1860 ms). Trace:
I20260709 13:20:23.930259 19650 consensus_queue.cc:1048] T c9944d5e4b8c4109af9955c4f08c9095 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.037s
I20260709 13:20:23.963248 19652 raft_consensus.cc:515] T 8158bbd17b994d2eb96f560bd0f1b88d P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:24.007989 16825 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49892 (request call id 497) took 1731 ms. Trace:
I20260709 13:20:24.008111 19509 raft_consensus.cc:3055] T ecc1e363230043fe9fef390f0b979f20 P 1ca2725d290e47d68b1a4b609c9816ac [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:20:24.006150 19418 leader_election.cc:290] T ee9ed67006144797b1de6ea8c2013a68 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:24.008901 16147 raft_consensus.cc:2468] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 10.
I20260709 13:20:24.065836 19478 raft_consensus.cc:2804] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:20:24.009240 16143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5d59407eb545b895d8d922e0dd69fd" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:20:24.014446 16145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c61a39fd4f84424b129c1a6f9ba404d" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:24.016068 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "829b4ad8c5a54eababeea1181b960d4b" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:24.079955 16143 raft_consensus.cc:2368] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac for earlier term 9. Current term is 10.
I20260709 13:20:24.018399 16151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6f07842d3aa47858df5665874c86f3a" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:24.018584 16115 leader_election.cc:304] T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac
I20260709 13:20:24.018957 16297 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48674 (request call id 2388) took 1954 ms. Trace:
I20260709 13:20:24.080840 16143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a96373bb0e14dce8a266b2fc6c58507" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:24.019440 16266 leader_election.cc:304] T 8638ee363273426db1393dd81e5c7b72 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860
I20260709 13:20:24.081133 16151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f2da98864b4425bf7c0055951fdd4b" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:24.081874 16151 raft_consensus.cc:2468] T 84f2da98864b4425bf7c0055951fdd4b P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 6.
I20260709 13:20:24.081959 16143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ad5dbfa99ee4b8d9e28be92b3f36db5" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:24.012624 19656 consensus_queue.cc:1059] T fffab26427e24d398e2695aad40ca134 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:20:24.083014 19652 leader_election.cc:290] T 8158bbd17b994d2eb96f560bd0f1b88d P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:20:23.996526 19610 raft_consensus.cc:515] T c9944d5e4b8c4109af9955c4f08c9095 P cb82cc7bd6db46d980c75237a1402860 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:24.039440 16149 raft_consensus.cc:2468] T ee9ed67006144797b1de6ea8c2013a68 P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 6.
I20260709 13:20:23.983476 19620 leader_election.cc:290] T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 10 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:20:24.044565 16291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a9de347c711477e87c582d003f36925" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 10 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:20:24.045324 16821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2794af5b99984efebdee564035841cb0" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:24.173102 16291 raft_consensus.cc:3060] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:20:24.046291 16818 raft_consensus.cc:1240] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860 [term 10 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 9. Current term is 10. Ops: [7.8-9.9]
I20260709 13:20:23.935917 19587 raft_consensus.cc:515] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:20:24.048303 16294 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48674 (request call id 2379) took 1986 ms (client timeout 1657 ms). Trace:
I20260709 13:20:24.048592 16300 raft_consensus.cc:1240] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 5. Current term is 6. Ops: []
I20260709 13:20:24.050194 16820 raft_consensus.cc:1275] T 23f82224f4564fea93572e8206bad1fe P cb82cc7bd6db46d980c75237a1402860 [term 7 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 7 index: 5. (term mismatch)
W20260709 13:20:24.175752 16294 rpcz_store.cc:269] 0709 13:20:22.061949 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:22.062061 (+   112us) service_pool.cc:224] Handling call
0709 13:20:24.047998 (+1985937us) spinlock_profiling.cc:243] Waited 377 ms on lock 0x7b540024ca54. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:20:24.048232 (+   234us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":866922240}
I20260709 13:20:24.176323 16820 pending_rounds.cc:85] T 23f82224f4564fea93572e8206bad1fe P cb82cc7bd6db46d980c75237a1402860: Aborting all ops after (but not including) 4
I20260709 13:20:24.176720 16820 pending_rounds.cc:107] T 23f82224f4564fea93572e8206bad1fe P cb82cc7bd6db46d980c75237a1402860: Aborting uncommitted NO_OP operation due to leader change: 4.5
I20260709 13:20:24.177052 16820 raft_consensus.cc:2889] T 23f82224f4564fea93572e8206bad1fe P cb82cc7bd6db46d980c75237a1402860 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:20:24.177502 16820 pending_rounds.cc:107] T 23f82224f4564fea93572e8206bad1fe P cb82cc7bd6db46d980c75237a1402860: Aborting uncommitted NO_OP operation due to leader change: 6.6
I20260709 13:20:24.177309 16294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66ae176961b74813a58bc15c9c4c6f2a" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:24.177897 16820 raft_consensus.cc:2889] T 23f82224f4564fea93572e8206bad1fe P cb82cc7bd6db46d980c75237a1402860 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:20:24.050694 16189 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:24.178387 16294 raft_consensus.cc:2393] T 66ae176961b74813a58bc15c9c4c6f2a P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 in current term 10: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
I20260709 13:20:24.051774 16116 leader_election.cc:304] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860
W20260709 13:20:24.178977 16189 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:24.179208 18599 raft_consensus.cc:2749] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
W20260709 13:20:24.179433 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 127750us
I20260709 13:20:24.052098 16298 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48684 (request call id 2265) took 1290 ms. Trace:
I20260709 13:20:24.180811 16298 rpcz_store.cc:276] 0709 13:20:22.761358 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:22.781368 (+ 20010us) service_pool.cc:224] Handling call
0709 13:20:22.781691 (+   323us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:20:23.144162 (+362471us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:23.144191 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:20:23.144654 (+   463us) log.cc:844] Serialized 26 byte log entry
0709 13:20:23.534201 (+389547us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:20:23.534698 (+   497us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:24.051626 (+516928us) spinlock_profiling.cc:243] Waited 362 ms on lock 0x7b5400210054. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:20:24.051703 (+    77us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:20:24.051764 (+    61us) raft_consensus.cc:1685] finished
0709 13:20:24.051784 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:24.052028 (+   244us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":3898,"raft.run_cpu_time_us":127,"raft.run_wall_time_us":7452,"spinlock_wait_cycles":832785280,"thread_start_us":368514,"threads_started":1,"wal-append.queue_time_us":394246}
I20260709 13:20:24.183022 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada5a004ba7c46819051e3ccec71ffdb" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:24.183943 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d4ab870a05347868956aa8b841b2981" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:24.056013 16148 raft_consensus.cc:2468] T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 8.
I20260709 13:20:24.185509 16148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57a744f5aaf944678a63469cbf514015" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:20:24.088292 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806d7690c2b644959152074bb2cab0a7" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:24.186275 16148 raft_consensus.cc:3060] T 57a744f5aaf944678a63469cbf514015 P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Advancing to term 7
I20260709 13:20:24.186830 16293 raft_consensus.cc:2393] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 in current term 7: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
I20260709 13:20:23.991122 16296 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48684 (request call id 2268) took 1041 ms. Trace:
I20260709 13:20:24.080152 16150 raft_consensus.cc:2468] T 829b4ad8c5a54eababeea1181b960d4b P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 8.
I20260709 13:20:24.187690 16296 rpcz_store.cc:276] 0709 13:20:22.949697 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:22.983762 (+ 34065us) service_pool.cc:224] Handling call
0709 13:20:23.145637 (+161875us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:20:23.149724 (+  4087us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:23.990863 (+841139us) spinlock_profiling.cc:243] Waited 161 ms on lock 0x7b54003999d0. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5e8ad 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:20:23.991052 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":370697984}
I20260709 13:20:24.029299 19512 raft_consensus.cc:493] T 08c74490c40243a88e62d1760b5cd814 P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:24.060480 16817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746aabf856ab401c9ef5e859371e7b67" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
W20260709 13:20:24.087869 16115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 69360us
I20260709 13:20:24.064173 16819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66f75124215f4ca6bef831e174a792d8" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:24.180335 16116 leader_election.cc:304] T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
I20260709 13:20:24.194814 16819 raft_consensus.cc:2468] T 66f75124215f4ca6bef831e174a792d8 P cb82cc7bd6db46d980c75237a1402860 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 8.
I20260709 13:20:24.064558 16822 raft_consensus.cc:1240] T 434bff3082d141838068f323bdb0d1ed P cb82cc7bd6db46d980c75237a1402860 [term 9 LEADER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 8. Current term is 9. Ops: []
I20260709 13:20:24.195569 16294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57a744f5aaf944678a63469cbf514015" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:20:23.932667 19576 raft_consensus.cc:515] T 829b4ad8c5a54eababeea1181b960d4b P cb82cc7bd6db46d980c75237a1402860 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:24.065201 16824 raft_consensus.cc:1275] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 [term 9 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 13:20:24.065040 16790 leader_election.cc:304] T 66f75124215f4ca6bef831e174a792d8 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:20:24.066021 16816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b7d1ab519044eda2d5e04a07242f25" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
W20260709 13:20:24.067400 16190 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:24.069075 16142 rpcz_store.cc:276] 0709 13:20:20.760059 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:21.662964 (+902905us) service_pool.cc:224] Handling call
0709 13:20:23.486058 (+1823094us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:20:23.491247 (+  5189us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:24.003636 (+512389us) spinlock_profiling.cc:243] Waited 1.82 s on lock 0x7b54003f0f50. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5e8ad 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:20:24.003799 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4191900032}
I20260709 13:20:24.068035 19578 raft_consensus.cc:493] T af57996a8be84d398f0f023e8d08ef77 P cb82cc7bd6db46d980c75237a1402860 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:20:24.072620 16295 rpcz_store.cc:269] 0709 13:20:22.456243 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:22.498034 (+ 41791us) service_pool.cc:224] Handling call
0709 13:20:24.005929 (+1507895us) spinlock_profiling.cc:243] Waited 898 ms on lock 0x7b540030c0d4. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:20:24.006063 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2064554880}
I20260709 13:20:24.053948 19609 raft_consensus.cc:740] T ee9ed67006144797b1de6ea8c2013a68 P cb82cc7bd6db46d980c75237a1402860 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:20:24.075052 16292 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48684 (request call id 2262) took 1485 ms. Trace:
I20260709 13:20:24.073359 16789 leader_election.cc:304] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac
I20260709 13:20:24.071125 19642 raft_consensus.cc:3060] T 8cc959fb7f704fba9009c0595467d074 P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:20:24.078325 19509 raft_consensus.cc:740] T ecc1e363230043fe9fef390f0b979f20 P 1ca2725d290e47d68b1a4b609c9816ac [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:20:24.076511 16825 rpcz_store.cc:276] 0709 13:20:22.276872 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:22.476709 (+199837us) service_pool.cc:224] Handling call
0709 13:20:24.007797 (+1531088us) spinlock_profiling.cc:243] Waited 768 ms on lock 0x7b5400787154. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:20:24.007929 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1767116544}
W20260709 13:20:24.023379 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 53827.3us
I20260709 13:20:24.211544 16218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746aabf856ab401c9ef5e859371e7b67" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:24.211836 16265 leader_election.cc:304] T 5a96373bb0e14dce8a266b2fc6c58507 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:20:24.082820 16151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66f75124215f4ca6bef831e174a792d8" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:24.040335 19113 raft_consensus.cc:515] T 8c11074579b046d99e585c2818e18a2c P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:24.158499 19496 raft_consensus.cc:493] T 8c11074579b046d99e585c2818e18a2c P cb82cc7bd6db46d980c75237a1402860 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:24.085922 16823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee9ed67006144797b1de6ea8c2013a68" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
W20260709 13:20:24.106469 16117 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:24.217114 16821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62fe3346cd3c42c4a55e14af9073bca4" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:24.217804 16821 raft_consensus.cc:2468] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 10.
I20260709 13:20:24.163199 19407 consensus_queue.cc:1048] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.078s
I20260709 13:20:24.218168 16220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f2da98864b4425bf7c0055951fdd4b" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:20:23.985526 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5d59407eb545b895d8d922e0dd69fd" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
W20260709 13:20:24.174458 16266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 155080us
I20260709 13:20:24.083055 16143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1df93779209748b9860f6d893dd3de53" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
W20260709 13:20:24.194309 19414 consensus_meta.cc:287] T 5a959dbecc2648eb85d1cc550f5a3245 P 90f3da57796b488c9087a2f523adf045: Time spent flushing consensus metadata: real 0.593s	user 0.011s	sys 0.001s
I20260709 13:20:24.212200 16218 raft_consensus.cc:2468] T 746aabf856ab401c9ef5e859371e7b67 P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 8.
I20260709 13:20:24.188915 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746aabf856ab401c9ef5e859371e7b67" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:24.221134 16150 raft_consensus.cc:2468] T 746aabf856ab401c9ef5e859371e7b67 P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 8.
I20260709 13:20:24.221199 16820 raft_consensus.cc:1240] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860 [term 8 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 7. Current term is 8. Ops: []
I20260709 13:20:24.193039 16144 raft_consensus.cc:1275] T 434bff3082d141838068f323bdb0d1ed P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 9 index: 9. (term mismatch)
I20260709 13:20:24.222050 16144 pending_rounds.cc:85] T 434bff3082d141838068f323bdb0d1ed P 647a343a542d4a139fe106c19095129b: Aborting all ops after (but not including) 8
I20260709 13:20:24.221860 16115 leader_election.cc:304] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac
I20260709 13:20:24.222425 16144 pending_rounds.cc:107] T 434bff3082d141838068f323bdb0d1ed P 647a343a542d4a139fe106c19095129b: Aborting uncommitted NO_OP operation due to leader change: 8.9
I20260709 13:20:24.222769 16144 raft_consensus.cc:2889] T 434bff3082d141838068f323bdb0d1ed P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:20:24.196133 16294 raft_consensus.cc:3060] T 57a744f5aaf944678a63469cbf514015 P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:20:24.223122 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a378d7f796f94615b3b347119a9880fd" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:24.224303 16293 raft_consensus.cc:2393] T a378d7f796f94615b3b347119a9880fd P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 in current term 8: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
I20260709 13:20:24.080770 16297 rpcz_store.cc:276] 0709 13:20:22.064551 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:22.200178 (+135627us) service_pool.cc:224] Handling call
0709 13:20:22.200479 (+   301us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:22.235869 (+ 35390us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:20:22.249440 (+ 13571us) log.cc:844] Serialized 14 byte log entry
0709 13:20:22.548063 (+298623us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:20:22.548108 (+    45us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:20:22.548148 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:24.012402 (+1464254us) spinlock_profiling.cc:243] Waited 35.1 ms on lock 0x7b540035e8d4. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:20:24.018800 (+  6398us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:24.018907 (+   107us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":13378,"prepare.run_cpu_time_us":31,"prepare.run_wall_time_us":32,"raft.queue_time_us":7964,"raft.run_cpu_time_us":138,"raft.run_wall_time_us":28518,"spinlock_wait_cycles":80726784,"thread_start_us":298278,"threads_started":1,"wal-append.queue_time_us":519938,"wal-append.run_cpu_time_us":2261,"wal-append.run_wall_time_us":1382801}
I20260709 13:20:24.194274 16817 raft_consensus.cc:2468] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 8.
I20260709 13:20:24.206173 16292 rpcz_store.cc:276] 0709 13:20:22.590083 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:22.765255 (+175172us) service_pool.cc:224] Handling call
0709 13:20:22.765560 (+   305us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:22.765776 (+   216us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:20:23.787739 (+1021963us) log.cc:844] Serialized 14 byte log entry
0709 13:20:24.074639 (+286900us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:20:24.074687 (+    48us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:20:24.074732 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:24.074803 (+    71us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:24.074976 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":874418,"prepare.queue_time_us":862197,"prepare.run_cpu_time_us":49,"prepare.run_wall_time_us":46,"raft.queue_time_us":104828,"raft.run_cpu_time_us":161,"raft.run_wall_time_us":6759,"thread_start_us":446435,"threads_started":3,"wal-append.queue_time_us":47021}
I20260709 13:20:24.215051 16296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57a744f5aaf944678a63469cbf514015" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:24.228267 16295 raft_consensus.cc:1275] T 434bff3082d141838068f323bdb0d1ed P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:20:24.164999 16147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18ad7e3af5504571a69d10c00750a01c" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:24.231583 16816 raft_consensus.cc:2468] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 6.
I20260709 13:20:24.232153 16147 raft_consensus.cc:2393] T 18ad7e3af5504571a69d10c00750a01c P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 8: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
I20260709 13:20:24.228180 16145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fffab26427e24d398e2695aad40ca134" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:24.233736 16145 raft_consensus.cc:2393] T fffab26427e24d398e2695aad40ca134 P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 7: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
I20260709 13:20:24.214082 16148 raft_consensus.cc:2468] T 57a744f5aaf944678a63469cbf514015 P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 7.
I20260709 13:20:24.242687 16143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af57996a8be84d398f0f023e8d08ef77" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:24.244735 16294 raft_consensus.cc:2468] T 57a744f5aaf944678a63469cbf514015 P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 7.
I20260709 13:20:24.243149 16818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746aabf856ab401c9ef5e859371e7b67" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:24.207540 19512 raft_consensus.cc:515] T 08c74490c40243a88e62d1760b5cd814 P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:24.248987 19512 leader_election.cc:290] T 08c74490c40243a88e62d1760b5cd814 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:24.251438 19512 consensus_queue.cc:1059] T 19baa7bc367844c288bca8f81439c413 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
W20260709 13:20:24.253047 16116 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:24.253509 19569 raft_consensus.cc:2804] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:20:24.258006 19569 raft_consensus.cc:697] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 647a343a542d4a139fe106c19095129b [term 8 LEADER]: Becoming Leader. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:20:24.259147 19569 consensus_queue.cc:237] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:20:24.255734 16115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 60263.7us
I20260709 13:20:24.265970 19450 consensus_queue.cc:1048] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:20:24.269248 16115 leader_election.cc:304] T c0eab25ce4c64e018a041beeda2867db P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:20:24.272238 19450 raft_consensus.cc:2749] T c0eab25ce4c64e018a041beeda2867db P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:20:24.278113 16819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "829b4ad8c5a54eababeea1181b960d4b" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:24.194022 19418 raft_consensus.cc:2804] T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:20:24.282087 16822 raft_consensus.cc:1240] T 19baa7bc367844c288bca8f81439c413 P cb82cc7bd6db46d980c75237a1402860 [term 8 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 7. Current term is 8. Ops: []
I20260709 13:20:24.284121 16819 raft_consensus.cc:2410] T 829b4ad8c5a54eababeea1181b960d4b P cb82cc7bd6db46d980c75237a1402860 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac for term 9 because replica has last-logged OpId of term: 7 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 8.
I20260709 13:20:24.089393 19478 raft_consensus.cc:697] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac [term 9 LEADER]: Becoming Leader. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:20:24.226877 19465 raft_consensus.cc:493] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:24.173738 19657 consensus_queue.cc:1048] T c9944d5e4b8c4109af9955c4f08c9095 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.209s
I20260709 13:20:24.199911 19660 raft_consensus.cc:2749] T 8638ee363273426db1393dd81e5c7b72 P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:20:24.211905 19416 raft_consensus.cc:2804] T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:20:24.305086 16225 raft_consensus.cc:1275] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 13:20:24.295465 16219 raft_consensus.cc:1275] T af57996a8be84d398f0f023e8d08ef77 P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 13:20:24.071902 19148 raft_consensus.cc:515] T b87147731a68400e8cce9b879daa7c5d P cb82cc7bd6db46d980c75237a1402860 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:24.220021 19620 raft_consensus.cc:2804] T 66f75124215f4ca6bef831e174a792d8 P cb82cc7bd6db46d980c75237a1402860 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:20:24.307462 19345 raft_consensus.cc:3055] T fffab26427e24d398e2695aad40ca134 P 90f3da57796b488c9087a2f523adf045 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:20:24.308559 19421 leader_election.cc:290] T 84f2da98864b4425bf7c0055951fdd4b P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 6 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
W20260709 13:20:24.309989 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 98215.8us
I20260709 13:20:24.311408 16224 raft_consensus.cc:1275] T 7c2095a5f4ad49408dbc7f69c606120d P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 13:20:24.321355 16146 raft_consensus.cc:1240] T ee9ed67006144797b1de6ea8c2013a68 P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 5. Current term is 6. Ops: []
I20260709 13:20:24.317343 16222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b7d1ab519044eda2d5e04a07242f25" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:24.329305 16219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18ad7e3af5504571a69d10c00750a01c" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:20:24.210618 19578 raft_consensus.cc:515] T af57996a8be84d398f0f023e8d08ef77 P cb82cc7bd6db46d980c75237a1402860 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:20:24.354280 16115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 78329.5us
I20260709 13:20:24.368553 16115 leader_election.cc:304] T 8a878999c0024d2abe517484180290fc P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:20:24.215205 19532 consensus_queue.cc:1048] T b6f07842d3aa47858df5665874c86f3a P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.094s
I20260709 13:20:24.215406 19197 consensus_queue.cc:1048] T 8158bbd17b994d2eb96f560bd0f1b88d P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 3, Time since last communication: 0.095s
I20260709 13:20:24.358196 16291 raft_consensus.cc:2468] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 10.
I20260709 13:20:24.339529 16219 raft_consensus.cc:3060] T 18ad7e3af5504571a69d10c00750a01c P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:20:24.314007 16265 leader_election.cc:304] T 68a6838d82ef4cac83507f906139d90f P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac
I20260709 13:20:24.360489 16222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee9ed67006144797b1de6ea8c2013a68" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
W20260709 13:20:24.315212 16192 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 258246us
I20260709 13:20:24.379086 16222 raft_consensus.cc:2393] T ee9ed67006144797b1de6ea8c2013a68 P 1ca2725d290e47d68b1a4b609c9816ac [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 6: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
W20260709 13:20:24.379825 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 65896.2us
I20260709 13:20:24.245036 19414 raft_consensus.cc:515] T 5a959dbecc2648eb85d1cc550f5a3245 P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:24.304076 19416 raft_consensus.cc:493] T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:24.241577 19496 raft_consensus.cc:515] T 8c11074579b046d99e585c2818e18a2c P cb82cc7bd6db46d980c75237a1402860 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:24.310479 19418 raft_consensus.cc:697] T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b [term 7 LEADER]: Becoming Leader. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:20:24.313503 19345 raft_consensus.cc:740] T fffab26427e24d398e2695aad40ca134 P 90f3da57796b488c9087a2f523adf045 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:20:24.313344 19664 raft_consensus.cc:2749] T 5a96373bb0e14dce8a266b2fc6c58507 P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:20:24.380717 16192 leader_election.cc:304] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:20:24.314210 19620 raft_consensus.cc:493] T 66f75124215f4ca6bef831e174a792d8 P cb82cc7bd6db46d980c75237a1402860 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:24.383095 16265 leader_election.cc:304] T 8a878999c0024d2abe517484180290fc P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac
I20260709 13:20:24.409866 16265 leader_election.cc:304] T 9e65676b21b748728ad39668cfe722ec P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
W20260709 13:20:24.353397 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 288441us
I20260709 13:20:24.367794 19575 raft_consensus.cc:493] T 08c74490c40243a88e62d1760b5cd814 P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:24.293027 19465 raft_consensus.cc:515] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:20:24.400243 16266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 170975us
I20260709 13:20:24.367992 19529 raft_consensus.cc:493] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:24.377607 19197 consensus_queue.cc:1243] T 8158bbd17b994d2eb96f560bd0f1b88d P 647a343a542d4a139fe106c19095129b [LEADER]: Peer 1ca2725d290e47d68b1a4b609c9816ac log is divergent from this leader: its last log entry 5.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260709 13:20:24.374563 19450 raft_consensus.cc:2749] T 8a878999c0024d2abe517484180290fc P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:20:24.455482 16294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a878999c0024d2abe517484180290fc" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:24.385536 16115 leader_election.cc:304] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac
I20260709 13:20:24.418951 19345 consensus_queue.cc:260] T fffab26427e24d398e2695aad40ca134 P 90f3da57796b488c9087a2f523adf045 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:20:24.423432 19661 log.cc:927] Time spent T ada5a004ba7c46819051e3ccec71ffdb P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.120s	user 0.001s	sys 0.000s
I20260709 13:20:24.403708 19416 raft_consensus.cc:3060] T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:20:24.462757 16295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1df93779209748b9860f6d893dd3de53" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:24.424129 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806d7690c2b644959152074bb2cab0a7" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:20:24.463701 16295 raft_consensus.cc:2468] T 1df93779209748b9860f6d893dd3de53 P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 8.
I20260709 13:20:24.328151 19509 consensus_queue.cc:260] T ecc1e363230043fe9fef390f0b979f20 P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:20:24.427438 19665 log.cc:927] Time spent T d2ab0ac8347641e6955e9bce7d90c1a2 P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 0.052s	user 0.001s	sys 0.000s
I20260709 13:20:24.464921 16818 raft_consensus.cc:2468] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 8.
I20260709 13:20:24.400215 19421 consensus_queue.cc:1048] T b87147731a68400e8cce9b879daa7c5d P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.027s
I20260709 13:20:24.411583 19620 raft_consensus.cc:3060] T 66f75124215f4ca6bef831e174a792d8 P cb82cc7bd6db46d980c75237a1402860 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:20:24.469947 19642 raft_consensus.cc:515] T 8cc959fb7f704fba9009c0595467d074 P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:24.459753 19532 raft_consensus.cc:2804] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:20:24.471444 16223 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33050 (request call id 586) took 1132 ms. Trace:
I20260709 13:20:24.412137 19478 consensus_queue.cc:237] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.9, Last appended by leader: 9, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:24.435855 19587 leader_election.cc:290] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:20:24.436869 19576 leader_election.cc:290] T 829b4ad8c5a54eababeea1181b960d4b P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:20:24.412519 19651 leader_election.cc:290] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:24.438098 19113 leader_election.cc:290] T 8c11074579b046d99e585c2818e18a2c P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
W20260709 13:20:24.438761 19663 log.cc:927] Time spent T fffab26427e24d398e2695aad40ca134 P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.070s	user 0.002s	sys 0.000s
I20260709 13:20:24.438840 19526 raft_consensus.cc:3055] T 19baa7bc367844c288bca8f81439c413 P 1ca2725d290e47d68b1a4b609c9816ac [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:20:24.439265 19667 raft_consensus.cc:2804] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:20:24.471184 19532 raft_consensus.cc:493] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Starting leader election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:20:24.440493 19418 consensus_queue.cc:237] T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:24.447392 19575 raft_consensus.cc:515] T 08c74490c40243a88e62d1760b5cd814 P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:24.449461 19672 raft_consensus.cc:2804] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:20:24.452150 16265 leader_election.cc:304] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
W20260709 13:20:24.457055 16115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 71596.6us
W20260709 13:20:24.512684 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 60626.7us
I20260709 13:20:24.458606 16291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6f07842d3aa47858df5665874c86f3a" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:24.513435 16291 raft_consensus.cc:2468] T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 9.
I20260709 13:20:24.462383 16296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ad5dbfa99ee4b8d9e28be92b3f36db5" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:24.456009 19660 raft_consensus.cc:2804] T 8a878999c0024d2abe517484180290fc P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Leader election won for term 8
W20260709 13:20:24.472122 16191 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:24.375382 19609 consensus_queue.cc:260] T ee9ed67006144797b1de6ea8c2013a68 P cb82cc7bd6db46d980c75237a1402860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:24.457979 19664 raft_consensus.cc:3060] T 9e65676b21b748728ad39668cfe722ec P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:20:24.471875 16223 rpcz_store.cc:276] 0709 13:20:23.339532 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:23.605865 (+266333us) service_pool.cc:224] Handling call
0709 13:20:23.765309 (+159444us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:20:23.765573 (+   264us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:23.765602 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:20:23.766101 (+   499us) log.cc:844] Serialized 26 byte log entry
0709 13:20:24.041710 (+275609us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:20:24.063932 (+ 22222us) log.cc:844] Serialized 14 byte log entry
0709 13:20:24.368031 (+304099us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:24.368108 (+    77us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:20:24.471234 (+103126us) raft_consensus.cc:1685] finished
0709 13:20:24.471255 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:24.471390 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":22174,"prepare.run_cpu_time_us":45,"prepare.run_wall_time_us":47,"raft.queue_time_us":185858,"raft.run_cpu_time_us":89,"raft.run_wall_time_us":89,"thread_start_us":537633,"threads_started":2,"wal-append.queue_time_us":505133}
I20260709 13:20:24.521962 16790 leader_election.cc:304] T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860; no voters: 
W20260709 13:20:24.475531 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 269470us
I20260709 13:20:24.383605 19656 raft_consensus.cc:2804] T 68a6838d82ef4cac83507f906139d90f P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:20:24.487272 19610 leader_election.cc:290] T c9944d5e4b8c4109af9955c4f08c9095 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 9 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:24.496302 19575 leader_election.cc:290] T 08c74490c40243a88e62d1760b5cd814 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:24.496634 19642 leader_election.cc:290] T 8cc959fb7f704fba9009c0595467d074 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 9 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:20:24.491623 19667 raft_consensus.cc:697] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac [term 8 LEADER]: Becoming Leader. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:20:24.504052 19496 leader_election.cc:290] T 8c11074579b046d99e585c2818e18a2c P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:24.505260 19414 leader_election.cc:290] T 5a959dbecc2648eb85d1cc550f5a3245 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 6 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:20:24.489569 19509 raft_consensus.cc:3060] T ecc1e363230043fe9fef390f0b979f20 P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:20:24.539479 19575 consensus_queue.cc:1048] T 8cc959fb7f704fba9009c0595467d074 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:20:24.541237 19197 consensus_queue.cc:1059] T e5ad4eb281564936a59f41e387a47156 P 647a343a542d4a139fe106c19095129b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:20:24.542223 19197 raft_consensus.cc:3055] T e5ad4eb281564936a59f41e387a47156 P 647a343a542d4a139fe106c19095129b [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:20:24.542665 19197 raft_consensus.cc:740] T e5ad4eb281564936a59f41e387a47156 P 647a343a542d4a139fe106c19095129b [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:20:24.507189 16220 raft_consensus.cc:2468] T 84f2da98864b4425bf7c0055951fdd4b P 1ca2725d290e47d68b1a4b609c9816ac [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 6.
I20260709 13:20:24.517215 16144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87147731a68400e8cce9b879daa7c5d" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:24.543864 19197 consensus_queue.cc:260] T e5ad4eb281564936a59f41e387a47156 P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:24.544783 16144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a959dbecc2648eb85d1cc550f5a3245" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:20:24.545351 19197 raft_consensus.cc:3060] T e5ad4eb281564936a59f41e387a47156 P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:20:24.517994 16265 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:24.547710 16265 leader_election.cc:304] T 18ad7e3af5504571a69d10c00750a01c P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 647a343a542d4a139fe106c19095129b
I20260709 13:20:24.517895 16143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87ac0d1f7e8a419f9bcbc04aee0e067e" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:24.552444 19652 raft_consensus.cc:493] T e4dc6b0541ca431ebc757beac00521f1 P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:24.552959 19652 raft_consensus.cc:515] T e4dc6b0541ca431ebc757beac00521f1 P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:24.555296 19653 raft_consensus.cc:493] T 7c2095a5f4ad49408dbc7f69c606120d P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:24.555501 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e9b2bf7011f4ae688a79d7bb5a447b4" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:20:24.555810 19653 raft_consensus.cc:515] T 7c2095a5f4ad49408dbc7f69c606120d P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:24.556895 16142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8158bbd17b994d2eb96f560bd0f1b88d" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:24.558239 19653 leader_election.cc:290] T 7c2095a5f4ad49408dbc7f69c606120d P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:20:24.558372 16145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f5f89186204242bf377f7c42b2e2da" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:24.518930 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cc959fb7f704fba9009c0595467d074" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:20:24.525281 16148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3884f864a96d428785895e5463e5728f" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:24.560043 16146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d4ab870a05347868956aa8b841b2981" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 9 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:24.457899 19529 raft_consensus.cc:515] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:24.485035 19526 raft_consensus.cc:740] T 19baa7bc367844c288bca8f81439c413 P 1ca2725d290e47d68b1a4b609c9816ac [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:20:24.536803 16147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:24.537865 19416 raft_consensus.cc:515] T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:24.566247 16151 raft_consensus.cc:2393] T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 9: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
I20260709 13:20:24.538424 19496 raft_consensus.cc:2804] T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:20:24.506685 19148 leader_election.cc:290] T b87147731a68400e8cce9b879daa7c5d P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
W20260709 13:20:24.595283 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 70746us
I20260709 13:20:24.525049 19656 raft_consensus.cc:493] T 68a6838d82ef4cac83507f906139d90f P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:20:24.596390 16147 raft_consensus.cc:2468] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 8.
I20260709 13:20:24.597231 16146 raft_consensus.cc:2410] T 8d4ab870a05347868956aa8b841b2981 P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 for term 9 because replica has last-logged OpId of term: 8 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 3.
I20260709 13:20:24.490334 19532 raft_consensus.cc:3060] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:20:24.598178 16115 leader_election.cc:304] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860
I20260709 13:20:24.602547 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c2095a5f4ad49408dbc7f69c606120d" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
W20260709 13:20:24.592844 16191 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:24.604377 16148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4dc6b0541ca431ebc757beac00521f1" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:24.627211 19666 consensus_queue.cc:1048] T c0eab25ce4c64e018a041beeda2867db P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:20:24.600139 16265 leader_election.cc:304] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:20:24.524865 19651 raft_consensus.cc:2804] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:20:24.607910 19526 consensus_queue.cc:260] T 19baa7bc367844c288bca8f81439c413 P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:24.609248 16190 leader_election.cc:304] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860
I20260709 13:20:24.609429 19496 raft_consensus.cc:697] T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860 [term 10 LEADER]: Becoming Leader. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:20:24.609660 19113 consensus_queue.cc:1059] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
W20260709 13:20:24.593288 19663 log.cc:927] Time spent T fffab26427e24d398e2695aad40ca134 P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.104s	user 0.000s	sys 0.000s
I20260709 13:20:24.645510 16148 raft_consensus.cc:1275] T 08c74490c40243a88e62d1760b5cd814 P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260709 13:20:24.607369 19589 raft_consensus.cc:2804] T 18ad7e3af5504571a69d10c00750a01c P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:20:24.616847 19465 leader_election.cc:290] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:20:24.535404 19672 raft_consensus.cc:697] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860 [term 8 LEADER]: Becoming Leader. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:20:24.653637 16295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e65676b21b748728ad39668cfe722ec" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:20:24.620076 16219 raft_consensus.cc:2468] T 18ad7e3af5504571a69d10c00750a01c P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 8.
I20260709 13:20:24.619726 19667 consensus_queue.cc:237] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:24.621025 19578 leader_election.cc:290] T af57996a8be84d398f0f023e8d08ef77 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:20:24.621877 19509 consensus_queue.cc:1059] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:20:24.622392 19609 raft_consensus.cc:3060] T ee9ed67006144797b1de6ea8c2013a68 P cb82cc7bd6db46d980c75237a1402860 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:20:24.622493 19656 raft_consensus.cc:3060] T 68a6838d82ef4cac83507f906139d90f P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:20:24.631011 19653 raft_consensus.cc:2749] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:20:24.678488 16295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2794af5b99984efebdee564035841cb0" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:24.645084 19512 consensus_queue.cc:1059] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:20:24.645262 16296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fffab26427e24d398e2695aad40ca134" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:20:24.613381 19345 raft_consensus.cc:3060] T fffab26427e24d398e2695aad40ca134 P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:20:24.604359 19660 raft_consensus.cc:697] T 8a878999c0024d2abe517484180290fc P 90f3da57796b488c9087a2f523adf045 [term 8 LEADER]: Becoming Leader. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:20:24.630090 19450 raft_consensus.cc:3060] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:20:24.645399 16190 leader_election.cc:304] T 18ad7e3af5504571a69d10c00750a01c P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:20:24.634738 19676 raft_consensus.cc:2749] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:20:24.652892 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26354361e9b044cea1c2b86c4882fe92" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:24.653075 16291 raft_consensus.cc:1275] T 8158bbd17b994d2eb96f560bd0f1b88d P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:20:24.654110 19672 consensus_queue.cc:237] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.5, Last appended by leader: 5, Current term: 8, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:24.659497 19113 raft_consensus.cc:3055] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:20:24.659950 19526 raft_consensus.cc:3060] T 19baa7bc367844c288bca8f81439c413 P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:20:24.661439 19671 consensus_queue.cc:1059] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:20:24.629909 19651 raft_consensus.cc:697] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045 [term 7 LEADER]: Becoming Leader. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:20:24.675226 19652 leader_election.cc:290] T e4dc6b0541ca431ebc757beac00521f1 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
W20260709 13:20:24.669036 16789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 599031us
I20260709 13:20:24.670744 16298 raft_consensus.cc:1240] T 434bff3082d141838068f323bdb0d1ed P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 8. Current term is 9. Ops: []
I20260709 13:20:24.675868 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4f65cdf00b748bb963ee6e81ef43b79" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:20:24.655054 19645 consensus_queue.cc:1048] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.033s
I20260709 13:20:24.681381 16297 raft_consensus.cc:1275] T 23f82224f4564fea93572e8206bad1fe P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
W20260709 13:20:24.694169 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 292149us
W20260709 13:20:24.726101 16117 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.14.177.132:38923 timed out after 1.581s (SENT) [suppressed 30 similar messages]
I20260709 13:20:24.710867 16146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c74490c40243a88e62d1760b5cd814" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:24.713696 19657 raft_consensus.cc:493] T c4f65cdf00b748bb963ee6e81ef43b79 P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:20:24.732692 16116 leader_election.cc:304] T 746aabf856ab401c9ef5e859371e7b67 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
I20260709 13:20:24.694401 19113 raft_consensus.cc:740] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:20:24.745790 16299 raft_consensus.cc:1240] T 5a959dbecc2648eb85d1cc550f5a3245 P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 5. Current term is 6. Ops: []
I20260709 13:20:24.722716 19676 raft_consensus.cc:3055] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:20:24.749799 18599 raft_consensus.cc:2804] T 746aabf856ab401c9ef5e859371e7b67 P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:20:24.726325 16789 leader_election.cc:304] T 3884f864a96d428785895e5463e5728f P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 647a343a542d4a139fe106c19095129b
I20260709 13:20:24.750962 18599 raft_consensus.cc:493] T 746aabf856ab401c9ef5e859371e7b67 P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:24.714862 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d4ab870a05347868956aa8b841b2981" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:24.752025 16293 raft_consensus.cc:2468] T 8d4ab870a05347868956aa8b841b2981 P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 8.
W20260709 13:20:24.723662 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 78316.7us
I20260709 13:20:24.647397 19589 raft_consensus.cc:493] T 18ad7e3af5504571a69d10c00750a01c P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:24.755234 16297 raft_consensus.cc:1275] T 5a96373bb0e14dce8a266b2fc6c58507 P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 5 index: 2. Preceding OpId from leader: term: 7 index: 3. (index mismatch)
I20260709 13:20:24.752846 16190 leader_election.cc:304] T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:20:24.728626 16291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806d7690c2b644959152074bb2cab0a7" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:20:24.727238 16146 raft_consensus.cc:2468] T 08c74490c40243a88e62d1760b5cd814 P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 7.
I20260709 13:20:24.729593 16298 raft_consensus.cc:1275] T 1df93779209748b9860f6d893dd3de53 P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:20:24.724925 19666 raft_consensus.cc:3055] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:20:24.683064 19620 raft_consensus.cc:515] T 66f75124215f4ca6bef831e174a792d8 P cb82cc7bd6db46d980c75237a1402860 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:24.727308 16295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18ad7e3af5504571a69d10c00750a01c" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:20:24.752970 18599 raft_consensus.cc:3060] T 746aabf856ab401c9ef5e859371e7b67 P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:20:24.715559 16192 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 306921us
I20260709 13:20:24.759315 16789 leader_election.cc:304] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:20:24.727896 19416 leader_election.cc:290] T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 9 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:24.771566 16789 leader_election.cc:304] T ada5a004ba7c46819051e3ccec71ffdb P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:20:24.731765 19512 raft_consensus.cc:493] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:24.774777 16115 leader_election.cc:304] T 26354361e9b044cea1c2b86c4882fe92 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860
W20260709 13:20:24.776608 16189 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:20:24.777559 16189 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:24.779815 16789 leader_election.cc:304] T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:20:24.791131 18599 raft_consensus.cc:515] T 746aabf856ab401c9ef5e859371e7b67 P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:24.799887 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f2da98864b4425bf7c0055951fdd4b" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:20:24.800963 16299 raft_consensus.cc:2393] T 84f2da98864b4425bf7c0055951fdd4b P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 6: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
I20260709 13:20:24.749606 19676 raft_consensus.cc:740] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:20:24.802634 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c74490c40243a88e62d1760b5cd814" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:24.754760 19652 raft_consensus.cc:493] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:20:24.775287 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 253389us
I20260709 13:20:24.722877 19496 consensus_queue.cc:237] T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:24.769121 19589 raft_consensus.cc:3060] T 18ad7e3af5504571a69d10c00750a01c P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:20:24.768983 19666 raft_consensus.cc:740] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:20:24.776407 19529 leader_election.cc:290] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:24.743727 19657 raft_consensus.cc:515] T c4f65cdf00b748bb963ee6e81ef43b79 P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:24.778350 19478 raft_consensus.cc:2764] T 18ad7e3af5504571a69d10c00750a01c P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Leader pre-election decision vote started in defunct term 7: won
I20260709 13:20:24.817983 16298 raft_consensus.cc:1240] T 8cc959fb7f704fba9009c0595467d074 P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 8. Current term is 9. Ops: [8.6-8.6]
I20260709 13:20:24.803071 16789 leader_election.cc:304] T 57a744f5aaf944678a63469cbf514015 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:20:24.799615 19576 raft_consensus.cc:2749] T ada5a004ba7c46819051e3ccec71ffdb P cb82cc7bd6db46d980c75237a1402860 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:20:24.776984 19673 raft_consensus.cc:2804] T 3884f864a96d428785895e5463e5728f P cb82cc7bd6db46d980c75237a1402860 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:20:24.777671 19465 raft_consensus.cc:3060] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:20:24.822010 18599 leader_election.cc:290] T 746aabf856ab401c9ef5e859371e7b67 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 9 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:24.807528 19416 raft_consensus.cc:2749] T 26354361e9b044cea1c2b86c4882fe92 P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:20:24.803951 19568 raft_consensus.cc:2749] T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:20:24.843324 16299 raft_consensus.cc:1240] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 9. Current term is 10. Ops: [5.7-9.9]
I20260709 13:20:24.842553 16115 leader_election.cc:304] T 8d4ab870a05347868956aa8b841b2981 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac
W20260709 13:20:24.883554 16789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 80569.7us
W20260709 13:20:24.893352 16115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 50924.9us
I20260709 13:20:24.802045 19148 consensus_queue.cc:1048] T 434bff3082d141838068f323bdb0d1ed P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.014s
W20260709 13:20:24.848620 16266 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:24.853943 16291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
W20260709 13:20:24.856536 16117 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 130452us
W20260709 13:20:24.865670 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 208458us
I20260709 13:20:24.787923 19512 raft_consensus.cc:515] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:24.894755 16789 leader_election.cc:304] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 647a343a542d4a139fe106c19095129b
I20260709 13:20:24.896201 18599 raft_consensus.cc:2804] T 8d4ab870a05347868956aa8b841b2981 P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:20:24.902527 18599 raft_consensus.cc:493] T 8d4ab870a05347868956aa8b841b2981 P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Starting leader election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:20:24.902879 18599 raft_consensus.cc:3060] T 8d4ab870a05347868956aa8b841b2981 P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:20:24.897727 16291 raft_consensus.cc:2468] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 8.
I20260709 13:20:24.904798 16266 leader_election.cc:304] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
I20260709 13:20:24.905788 16300 raft_consensus.cc:3060] T c4f65cdf00b748bb963ee6e81ef43b79 P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:20:24.899731 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 167145us
I20260709 13:20:24.900075 19529 consensus_queue.cc:1048] T 5a96373bb0e14dce8a266b2fc6c58507 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.009s
I20260709 13:20:24.908737 19645 raft_consensus.cc:2749] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:20:24.900195 19569 consensus_queue.cc:1059] T 8cc959fb7f704fba9009c0595467d074 P 647a343a542d4a139fe106c19095129b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 2, Time since last communication: 0.009s
I20260709 13:20:24.840708 19522 consensus_queue.cc:1048] T 8158bbd17b994d2eb96f560bd0f1b88d P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.064s
I20260709 13:20:24.813479 19652 raft_consensus.cc:515] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:24.926220 18599 raft_consensus.cc:515] T 8d4ab870a05347868956aa8b841b2981 P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:24.897089 19587 raft_consensus.cc:2804] T 57a744f5aaf944678a63469cbf514015 P cb82cc7bd6db46d980c75237a1402860 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:20:24.939314 16146 raft_consensus.cc:1240] T 8d4ab870a05347868956aa8b841b2981 P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 8. Current term is 9. Ops: []
I20260709 13:20:24.850175 19578 consensus_queue.cc:1059] T 9e65676b21b748728ad39668cfe722ec P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: INVALID_TERM, Last received: 6.8, Next index: 9, Last known committed idx: 6, Time since last communication: 0.060s
W20260709 13:20:24.930083 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 177309us
I20260709 13:20:24.883188 19673 raft_consensus.cc:493] T 3884f864a96d428785895e5463e5728f P cb82cc7bd6db46d980c75237a1402860 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:20:24.849658 19418 consensus_queue.cc:1048] T 23f82224f4564fea93572e8206bad1fe P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.061s
I20260709 13:20:24.897140 19407 raft_consensus.cc:2804] T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 13:20:24.941427 16192 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 181754us
I20260709 13:20:24.883266 19575 consensus_queue.cc:1048] T 1df93779209748b9860f6d893dd3de53 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.063s
W20260709 13:20:24.943742 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 44489.8us
I20260709 13:20:24.943912 16190 leader_election.cc:304] T 8cc959fb7f704fba9009c0595467d074 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 647a343a542d4a139fe106c19095129b
I20260709 13:20:24.916515 19576 raft_consensus.cc:2804] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:20:24.955433 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9944d5e4b8c4109af9955c4f08c9095" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:20:24.956027 16291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87147731a68400e8cce9b879daa7c5d" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:24.956564 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c11074579b046d99e585c2818e18a2c" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:24.957458 16293 raft_consensus.cc:2410] T 8c11074579b046d99e585c2818e18a2c P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 for term 9 because replica has last-logged OpId of term: 7 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 8.
I20260709 13:20:24.957599 18599 leader_election.cc:290] T 8d4ab870a05347868956aa8b841b2981 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 9 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:24.815848 19113 consensus_queue.cc:260] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:24.948395 16818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a378d7f796f94615b3b347119a9880fd" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:24.966720 16818 raft_consensus.cc:2393] T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 8: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
I20260709 13:20:24.969511 16816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada5a004ba7c46819051e3ccec71ffdb" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
W20260709 13:20:24.951982 19681 log.cc:927] Time spent T b87147731a68400e8cce9b879daa7c5d P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 0.135s	user 0.000s	sys 0.001s
I20260709 13:20:24.952764 16819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e9b2bf7011f4ae688a79d7bb5a447b4" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:20:24.983599 16266 leader_election.cc:304] T a378d7f796f94615b3b347119a9880fd P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860
I20260709 13:20:24.951503 16820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87147731a68400e8cce9b879daa7c5d" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:24.984735 16820 raft_consensus.cc:2410] T b87147731a68400e8cce9b879daa7c5d P cb82cc7bd6db46d980c75237a1402860 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 for term 10 because replica has last-logged OpId of term: 9 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 7.
I20260709 13:20:24.810925 19660 consensus_queue.cc:237] T 8a878999c0024d2abe517484180290fc P 90f3da57796b488c9087a2f523adf045 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 6.3, Last appended by leader: 3, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:24.900935 19610 consensus_queue.cc:1059] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.042s
I20260709 13:20:24.883047 19682 consensus_queue.cc:1059] T 5a959dbecc2648eb85d1cc550f5a3245 P 647a343a542d4a139fe106c19095129b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.090s
I20260709 13:20:24.957168 16190 leader_election.cc:304] T 5a96373bb0e14dce8a266b2fc6c58507 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:20:24.970238 16816 raft_consensus.cc:2468] T ada5a004ba7c46819051e3ccec71ffdb P cb82cc7bd6db46d980c75237a1402860 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 8.
I20260709 13:20:24.972563 16293 raft_consensus.cc:1275] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:20:24.949013 16192 leader_election.cc:304] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:20:24.979892 18599 consensus_queue.cc:1048] T af57996a8be84d398f0f023e8d08ef77 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:20:24.969656 16818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66f75124215f4ca6bef831e174a792d8" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:20:25.022394 16818 raft_consensus.cc:2393] T 66f75124215f4ca6bef831e174a792d8 P cb82cc7bd6db46d980c75237a1402860 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 9: Already voted for candidate cb82cc7bd6db46d980c75237a1402860 in this term.
I20260709 13:20:24.916344 19584 consensus_queue.cc:1048] T 434bff3082d141838068f323bdb0d1ed P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.041s
I20260709 13:20:24.989755 15046 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:20:24.953719 19407 raft_consensus.cc:493] T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Starting leader election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:20:24.990387 16300 raft_consensus.cc:2410] T c4f65cdf00b748bb963ee6e81ef43b79 P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b for term 10 because replica has last-logged OpId of term: 9 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 6.
W20260709 13:20:24.955368 19677 log.cc:927] Time spent T 87ac0d1f7e8a419f9bcbc04aee0e067e P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 0.213s	user 0.000s	sys 0.001s
I20260709 13:20:24.991916 16822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3884f864a96d428785895e5463e5728f" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
W20260709 13:20:24.992837 16788 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:25.009181 16824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:25.030757 16824 raft_consensus.cc:2468] T 19baa7bc367844c288bca8f81439c413 P cb82cc7bd6db46d980c75237a1402860 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 8.
I20260709 13:20:25.015908 16821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:25.031891 16821 raft_consensus.cc:2468] T 19baa7bc367844c288bca8f81439c413 P cb82cc7bd6db46d980c75237a1402860 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 8.
I20260709 13:20:24.948877 19673 raft_consensus.cc:3060] T 3884f864a96d428785895e5463e5728f P cb82cc7bd6db46d980c75237a1402860 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:20:25.016191 16825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e65676b21b748728ad39668cfe722ec" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:20:25.016903 16823 raft_consensus.cc:1240] T c9944d5e4b8c4109af9955c4f08c9095 P cb82cc7bd6db46d980c75237a1402860 [term 9 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 8. Current term is 9. Ops: [8.9-8.9]
I20260709 13:20:25.017537 16817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e4bfb9090c8460eb14fa7e0d818e7f1" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
W20260709 13:20:25.025352 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 81943.6us
I20260709 13:20:25.016991 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d4ab870a05347868956aa8b841b2981" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:20:25.035892 16299 raft_consensus.cc:3060] T 8d4ab870a05347868956aa8b841b2981 P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:20:25.035856 16817 raft_consensus.cc:2468] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P cb82cc7bd6db46d980c75237a1402860 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 8.
I20260709 13:20:24.982398 19569 raft_consensus.cc:3055] T 8cc959fb7f704fba9009c0595467d074 P 647a343a542d4a139fe106c19095129b [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:20:24.820755 19651 consensus_queue.cc:237] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } }
I20260709 13:20:24.990109 19688 raft_consensus.cc:493] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:20:25.049094 16117 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 105526us
I20260709 13:20:25.010113 19645 raft_consensus.cc:2749] T a378d7f796f94615b3b347119a9880fd P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:20:25.017382 19540 consensus_queue.cc:1059] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.044s
I20260709 13:20:25.016093 19664 raft_consensus.cc:2749] T 9e65676b21b748728ad39668cfe722ec P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:20:25.033165 19605 log.cc:927] Time spent T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.062s	user 0.001s	sys 0.000s
I20260709 13:20:25.026966 19407 raft_consensus.cc:3060] T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:20:25.086676 16825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746aabf856ab401c9ef5e859371e7b67" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:20:25.087536 16825 raft_consensus.cc:3060] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:20:25.088966 16191 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:25.090104 16266 leader_election.cc:304] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
I20260709 13:20:25.092486 16819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:25.093119 16819 raft_consensus.cc:2468] T 19baa7bc367844c288bca8f81439c413 P cb82cc7bd6db46d980c75237a1402860 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 8.
I20260709 13:20:24.898479 19676 consensus_queue.cc:260] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:20:25.096352 16117 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 43743us
I20260709 13:20:25.096956 19676 raft_consensus.cc:3060] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:20:25.043840 19113 raft_consensus.cc:3060] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:20:25.085702 19587 raft_consensus.cc:697] T 57a744f5aaf944678a63469cbf514015 P cb82cc7bd6db46d980c75237a1402860 [term 7 LEADER]: Becoming Leader. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
W20260709 13:20:25.100478 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 143260us
I20260709 13:20:25.104866 16190 leader_election.cc:304] T 84f2da98864b4425bf7c0055951fdd4b P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:20:24.958552 19692 consensus_queue.cc:1059] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.9, Next index: 10, Last known committed idx: 7, Time since last communication: 0.093s
I20260709 13:20:25.045197 19569 raft_consensus.cc:740] T 8cc959fb7f704fba9009c0595467d074 P 647a343a542d4a139fe106c19095129b [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:20:25.050769 19620 leader_election.cc:290] T 66f75124215f4ca6bef831e174a792d8 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 9 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:20:25.078346 19610 raft_consensus.cc:3055] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:20:25.121245 16825 raft_consensus.cc:2468] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 9.
I20260709 13:20:25.058207 19642 raft_consensus.cc:493] T ecc1e363230043fe9fef390f0b979f20 P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:20:25.128541 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 44322.2us
I20260709 13:20:24.937558 19666 consensus_queue.cc:260] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:25.093976 19653 raft_consensus.cc:493] T 8c11074579b046d99e585c2818e18a2c P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:20:25.156751 16117 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 59284.9us
I20260709 13:20:25.097966 19414 raft_consensus.cc:493] T 23f82224f4564fea93572e8206bad1fe P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:20:25.154722 16115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 182564us
I20260709 13:20:25.113132 19416 raft_consensus.cc:3055] T 5a959dbecc2648eb85d1cc550f5a3245 P 647a343a542d4a139fe106c19095129b [term 5 LEADER]: Stepping down as leader of term 5
W20260709 13:20:25.157634 16266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 67623.7us
W20260709 13:20:25.178189 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 360982us
I20260709 13:20:25.178224 16266 leader_election.cc:304] T ada5a004ba7c46819051e3ccec71ffdb P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac
I20260709 13:20:25.181593 16266 leader_election.cc:304] T b87147731a68400e8cce9b879daa7c5d P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
I20260709 13:20:25.157490 16116 leader_election.cc:304] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
W20260709 13:20:25.164707 16192 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 215626us
I20260709 13:20:25.115779 19645 raft_consensus.cc:2749] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:20:25.093789 19529 consensus_queue.cc:1048] T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.066s
I20260709 13:20:25.126904 19702 raft_consensus.cc:2804] T 8cc959fb7f704fba9009c0595467d074 P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:20:25.190021 16117 leader_election.cc:304] T ee9ed67006144797b1de6ea8c2013a68 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860
I20260709 13:20:25.132107 19610 raft_consensus.cc:740] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:20:25.155951 19576 raft_consensus.cc:697] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860 [term 10 LEADER]: Becoming Leader. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:20:25.162642 19620 raft_consensus.cc:493] T 19baa7bc367844c288bca8f81439c413 P cb82cc7bd6db46d980c75237a1402860 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:25.200762 16192 leader_election.cc:304] T 746aabf856ab401c9ef5e859371e7b67 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:20:25.204289 16151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b7d1ab519044eda2d5e04a07242f25" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:25.131732 19575 consensus_queue.cc:1048] T 7c2095a5f4ad49408dbc7f69c606120d P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.061s
I20260709 13:20:25.127843 19421 consensus_queue.cc:1059] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.064s
I20260709 13:20:25.228039 16151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746aabf856ab401c9ef5e859371e7b67" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 9 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:25.229058 16151 raft_consensus.cc:2393] T 746aabf856ab401c9ef5e859371e7b67 P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 in current term 9: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
I20260709 13:20:25.232606 16151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66f75124215f4ca6bef831e174a792d8" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:20:25.192889 19664 raft_consensus.cc:2804] T ada5a004ba7c46819051e3ccec71ffdb P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:20:25.238579 16145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87147731a68400e8cce9b879daa7c5d" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:25.239425 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af57996a8be84d398f0f023e8d08ef77" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
W20260709 13:20:25.198486 19697 log.cc:927] Time spent T c0eab25ce4c64e018a041beeda2867db P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 0.139s	user 0.001s	sys 0.000s
I20260709 13:20:25.177531 19416 raft_consensus.cc:740] T 5a959dbecc2648eb85d1cc550f5a3245 P 647a343a542d4a139fe106c19095129b [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:20:25.234741 16151 raft_consensus.cc:2393] T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 in current term 9: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
W20260709 13:20:25.235061 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 145267us
I20260709 13:20:25.241753 16147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "829b4ad8c5a54eababeea1181b960d4b" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:25.245754 16147 raft_consensus.cc:2468] T 829b4ad8c5a54eababeea1181b960d4b P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 8.
W20260709 13:20:25.249922 16189 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:25.198210 19702 raft_consensus.cc:493] T 8cc959fb7f704fba9009c0595467d074 P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Starting leader election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:20:25.199139 19714 raft_consensus.cc:2804] T 5a96373bb0e14dce8a266b2fc6c58507 P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:20:25.255049 16191 leader_election.cc:304] T af57996a8be84d398f0f023e8d08ef77 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
I20260709 13:20:25.185952 19418 consensus_queue.cc:1048] T 23f82224f4564fea93572e8206bad1fe P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.053s
W20260709 13:20:25.257910 16266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 76396us
I20260709 13:20:25.179329 19653 raft_consensus.cc:515] T 8c11074579b046d99e585c2818e18a2c P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:25.186748 19414 raft_consensus.cc:515] T 23f82224f4564fea93572e8206bad1fe P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:25.126943 19688 raft_consensus.cc:515] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:25.157081 19642 raft_consensus.cc:515] T ecc1e363230043fe9fef390f0b979f20 P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } }
W20260709 13:20:25.271893 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 167591us
I20260709 13:20:25.254717 19692 raft_consensus.cc:3055] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac [term 7 LEADER]: Stepping down as leader of term 7
W20260709 13:20:25.331439 16117 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 141474us
I20260709 13:20:25.332926 16117 leader_election.cc:304] T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac
W20260709 13:20:25.335857 19701 log.cc:927] Time spent T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 0.069s	user 0.000s	sys 0.000s
I20260709 13:20:25.240823 19587 consensus_queue.cc:237] T 57a744f5aaf944678a63469cbf514015 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:20:25.337040 19450 consensus_meta.cc:287] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 647a343a542d4a139fe106c19095129b: Time spent flushing consensus metadata: real 0.642s	user 0.009s	sys 0.001s
I20260709 13:20:25.337533 19450 raft_consensus.cc:2749] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:20:25.305231 19526 consensus_meta.cc:287] T 19baa7bc367844c288bca8f81439c413 P 1ca2725d290e47d68b1a4b609c9816ac: Time spent flushing consensus metadata: real 0.587s	user 0.009s	sys 0.001s
I20260709 13:20:25.238394 19664 raft_consensus.cc:493] T ada5a004ba7c46819051e3ccec71ffdb P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:20:25.242779 19512 leader_election.cc:290] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:25.206496 19645 consensus_queue.cc:1059] T c9944d5e4b8c4109af9955c4f08c9095 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.062s
I20260709 13:20:25.234321 19620 raft_consensus.cc:515] T 19baa7bc367844c288bca8f81439c413 P cb82cc7bd6db46d980c75237a1402860 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:25.266453 19695 consensus_queue.cc:1048] T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.087s
I20260709 13:20:25.259631 19718 raft_consensus.cc:2749] T b87147731a68400e8cce9b879daa7c5d P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:20:25.271006 16266 leader_election.cc:304] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 
W20260709 13:20:25.380126 16266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 109318us
I20260709 13:20:25.381255 16266 leader_election.cc:304] T 3884f864a96d428785895e5463e5728f P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
I20260709 13:20:25.382659 16266 leader_election.cc:304] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:20:25.264807 19710 raft_consensus.cc:2804] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:20:25.383433 19710 raft_consensus.cc:493] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:25.383834 19710 raft_consensus.cc:3060] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:20:25.301220 19687 log.cc:927] Time spent T 1df93779209748b9860f6d893dd3de53 P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 0.095s	user 0.000s	sys 0.000s
W20260709 13:20:25.300597 19689 log.cc:927] Time spent T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 0.185s	user 0.000s	sys 0.000s
I20260709 13:20:25.261471 19714 raft_consensus.cc:493] T 5a96373bb0e14dce8a266b2fc6c58507 P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Starting leader election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:20:25.388631 19714 raft_consensus.cc:3060] T 5a96373bb0e14dce8a266b2fc6c58507 P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:20:25.346161 16117 leader_election.cc:304] T 9e65676b21b748728ad39668cfe722ec P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860
I20260709 13:20:25.272145 19716 raft_consensus.cc:493] T 23f82224f4564fea93572e8206bad1fe P cb82cc7bd6db46d980c75237a1402860 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:25.296669 19699 raft_consensus.cc:2804] T 746aabf856ab401c9ef5e859371e7b67 P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 13:20:25.355014 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 197595us
I20260709 13:20:25.356284 16191 leader_election.cc:400] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 9 election: Vote denied by peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741) with higher term. Message: Invalid argument: T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac for earlier term 9. Current term is 10.
W20260709 13:20:25.330585 19693 log.cc:927] Time spent T 31969392ccdb43b491911a319e8bc7e1 P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.280s	user 0.000s	sys 0.001s
I20260709 13:20:25.303141 19414 leader_election.cc:290] T 23f82224f4564fea93572e8206bad1fe P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:25.304723 19653 leader_election.cc:290] T 8c11074579b046d99e585c2818e18a2c P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:25.301316 19416 consensus_queue.cc:260] T 5a959dbecc2648eb85d1cc550f5a3245 P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:25.417254 19706 raft_consensus.cc:2749] T ee9ed67006144797b1de6ea8c2013a68 P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:20:25.420357 19416 raft_consensus.cc:3060] T 5a959dbecc2648eb85d1cc550f5a3245 P 647a343a542d4a139fe106c19095129b [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:20:25.420749 19682 raft_consensus.cc:2749] T 9e65676b21b748728ad39668cfe722ec P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
W20260709 13:20:25.395044 19609 consensus_meta.cc:287] T ee9ed67006144797b1de6ea8c2013a68 P cb82cc7bd6db46d980c75237a1402860: Time spent flushing consensus metadata: real 0.718s	user 0.007s	sys 0.001s
I20260709 13:20:25.365747 19526 raft_consensus.cc:2749] T af57996a8be84d398f0f023e8d08ef77 P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:20:25.351555 19700 raft_consensus.cc:493] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:25.366452 19727 raft_consensus.cc:2749] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:20:25.277999 19652 leader_election.cc:290] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:25.274987 19559 raft_consensus.cc:493] T 8638ee363273426db1393dd81e5c7b72 P cb82cc7bd6db46d980c75237a1402860 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:20:25.329699 16190 leader_election.cc:304] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860
I20260709 13:20:25.274294 19715 raft_consensus.cc:2804] T 84f2da98864b4425bf7c0055951fdd4b P 1ca2725d290e47d68b1a4b609c9816ac [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:20:25.323752 19692 raft_consensus.cc:740] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:20:25.300783 19721 consensus_queue.cc:1059] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: INVALID_TERM, Last received: 6.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.074s
I20260709 13:20:25.258607 19702 raft_consensus.cc:3060] T 8cc959fb7f704fba9009c0595467d074 P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:20:25.399719 16299 raft_consensus.cc:2468] T 8d4ab870a05347868956aa8b841b2981 P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 9.
I20260709 13:20:25.403120 16116 leader_election.cc:304] T 84f2da98864b4425bf7c0055951fdd4b P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 90f3da57796b488c9087a2f523adf045
W20260709 13:20:25.453560 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 50538.9us
I20260709 13:20:25.454463 16116 leader_election.cc:304] T 18ad7e3af5504571a69d10c00750a01c P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
W20260709 13:20:25.403676 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 60527.2us
I20260709 13:20:25.457818 16191 leader_election.cc:304] T 829b4ad8c5a54eababeea1181b960d4b P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: cb82cc7bd6db46d980c75237a1402860
I20260709 13:20:25.360386 19664 raft_consensus.cc:3060] T ada5a004ba7c46819051e3ccec71ffdb P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:20:25.395732 19197 consensus_meta.cc:287] T e5ad4eb281564936a59f41e387a47156 P 647a343a542d4a139fe106c19095129b: Time spent flushing consensus metadata: real 0.850s	user 0.012s	sys 0.008s
W20260709 13:20:25.406137 16117 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 60083.4us
I20260709 13:20:25.459007 19197 raft_consensus.cc:2804] T 84f2da98864b4425bf7c0055951fdd4b P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:20:25.459517 19197 raft_consensus.cc:697] T 84f2da98864b4425bf7c0055951fdd4b P 647a343a542d4a139fe106c19095129b [term 6 LEADER]: Becoming Leader. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:20:25.460386 19197 consensus_queue.cc:237] T 84f2da98864b4425bf7c0055951fdd4b P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:25.304890 19576 consensus_queue.cc:237] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:25.463555 19418 raft_consensus.cc:2804] T 18ad7e3af5504571a69d10c00750a01c P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:20:25.464263 19418 raft_consensus.cc:697] T 18ad7e3af5504571a69d10c00750a01c P 647a343a542d4a139fe106c19095129b [term 8 LEADER]: Becoming Leader. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:20:25.465276 16823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2794af5b99984efebdee564035841cb0" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:25.465363 19418 consensus_queue.cc:237] T 18ad7e3af5504571a69d10c00750a01c P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } }
I20260709 13:20:25.420519 19706 raft_consensus.cc:2804] T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 13:20:25.422077 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 212468us
W20260709 13:20:25.401348 19708 log.cc:927] Time spent T 08c74490c40243a88e62d1760b5cd814 P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 0.110s	user 0.000s	sys 0.001s
W20260709 13:20:25.401069 19656 consensus_meta.cc:287] T 68a6838d82ef4cac83507f906139d90f P 90f3da57796b488c9087a2f523adf045: Time spent flushing consensus metadata: real 0.674s	user 0.011s	sys 0.000s
I20260709 13:20:25.375811 19645 raft_consensus.cc:3055] T c9944d5e4b8c4109af9955c4f08c9095 P 90f3da57796b488c9087a2f523adf045 [term 8 LEADER]: Stepping down as leader of term 8
W20260709 13:20:25.417128 19345 consensus_meta.cc:287] T fffab26427e24d398e2695aad40ca134 P 90f3da57796b488c9087a2f523adf045: Time spent flushing consensus metadata: real 0.730s	user 0.012s	sys 0.000s
I20260709 13:20:25.396250 19718 raft_consensus.cc:2804] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:20:25.400449 19642 leader_election.cc:290] T ecc1e363230043fe9fef390f0b979f20 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:20:25.442217 19724 raft_consensus.cc:2749] T 3884f864a96d428785895e5463e5728f P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:20:25.446291 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 116743us
W20260709 13:20:25.450498 16791 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.505s	user 0.026s	sys 0.014s
I20260709 13:20:25.471889 16148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4f65cdf00b748bb963ee6e81ef43b79" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:25.493472 16148 raft_consensus.cc:2393] T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 10: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
I20260709 13:20:25.433593 19414 raft_consensus.cc:2804] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:20:25.443255 19715 raft_consensus.cc:493] T 84f2da98864b4425bf7c0055951fdd4b P 1ca2725d290e47d68b1a4b609c9816ac [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:25.402662 19699 raft_consensus.cc:493] T 746aabf856ab401c9ef5e859371e7b67 P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:25.496073 16146 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "18ad7e3af5504571a69d10c00750a01c"
dest_uuid: "647a343a542d4a139fe106c19095129b"
mode: GRACEFUL
new_leader_uuid: "90f3da57796b488c9087a2f523adf045"
 from {username='slave'} at 127.0.0.1:40290
I20260709 13:20:25.498466 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746aabf856ab401c9ef5e859371e7b67" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:20:25.498912 16222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d4ab870a05347868956aa8b841b2981" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:20:25.493032 16190 leader_election.cc:304] T d16dfad0c3b540e5b4b86492ddf87208 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:20:25.500751 16296 raft_consensus.cc:2410] T fffab26427e24d398e2695aad40ca134 P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b for term 7 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
I20260709 13:20:25.470252 19667 raft_consensus.cc:2749] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
W20260709 13:20:25.506032 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 48298.9us
I20260709 13:20:25.474534 19706 raft_consensus.cc:493] T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Starting leader election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:20:25.516142 16219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c74490c40243a88e62d1760b5cd814" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:25.518373 16191 leader_election.cc:304] T ee9ed67006144797b1de6ea8c2013a68 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: cb82cc7bd6db46d980c75237a1402860
I20260709 13:20:25.413925 19716 raft_consensus.cc:515] T 23f82224f4564fea93572e8206bad1fe P cb82cc7bd6db46d980c75237a1402860 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:25.446460 19559 raft_consensus.cc:515] T 8638ee363273426db1393dd81e5c7b72 P cb82cc7bd6db46d980c75237a1402860 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:25.527103 16220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:25.541672 16220 raft_consensus.cc:2468] T 19baa7bc367844c288bca8f81439c413 P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 8.
I20260709 13:20:25.527448 16223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66f75124215f4ca6bef831e174a792d8" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:20:25.542794 16223 raft_consensus.cc:3060] T 66f75124215f4ca6bef831e174a792d8 P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:20:25.530727 16219 raft_consensus.cc:2468] T 08c74490c40243a88e62d1760b5cd814 P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 7.
I20260709 13:20:25.528136 16218 pending_rounds.cc:85] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac: Aborting all ops after (but not including) 3
I20260709 13:20:25.544009 16218 pending_rounds.cc:107] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac: Aborting uncommitted NO_OP operation due to leader change: 5.4
I20260709 13:20:25.544327 16218 raft_consensus.cc:2889] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:20:25.490551 19718 raft_consensus.cc:493] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:25.481900 19645 raft_consensus.cc:740] T c9944d5e4b8c4109af9955c4f08c9095 P 90f3da57796b488c9087a2f523adf045 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:20:25.500590 19414 raft_consensus.cc:493] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:25.516460 19699 raft_consensus.cc:3060] T 746aabf856ab401c9ef5e859371e7b67 P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:20:25.558728 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 65778us
I20260709 13:20:25.433389 19700 raft_consensus.cc:515] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:25.510612 19715 raft_consensus.cc:3060] T 84f2da98864b4425bf7c0055951fdd4b P 1ca2725d290e47d68b1a4b609c9816ac [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:20:25.517879 19674 raft_consensus.cc:3055] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:20:25.587612 16190 leader_election.cc:304] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860
I20260709 13:20:25.484100 19656 raft_consensus.cc:515] T 68a6838d82ef4cac83507f906139d90f P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:25.590976 16817 raft_consensus.cc:1240] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860 [term 10 LEADER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 9. Current term is 10. Ops: [7.8-9.9]
I20260709 13:20:25.592610 16116 leader_election.cc:304] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:20:25.631000 16116 leader_election.cc:304] T 08c74490c40243a88e62d1760b5cd814 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 90f3da57796b488c9087a2f523adf045
I20260709 13:20:25.550241 19725 raft_consensus.cc:2804] T 829b4ad8c5a54eababeea1181b960d4b P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:20:25.540041 19706 raft_consensus.cc:3060] T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:20:25.599481 19674 raft_consensus.cc:740] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:20:25.559316 19414 raft_consensus.cc:3060] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:20:25.551689 19718 raft_consensus.cc:3060] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:20:25.657166 16220 raft_consensus.cc:1275] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
I20260709 13:20:25.590164 19667 raft_consensus.cc:2804] T d16dfad0c3b540e5b4b86492ddf87208 P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 13:20:25.667724 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 158644us
I20260709 13:20:25.635454 19721 raft_consensus.cc:2804] T ee9ed67006144797b1de6ea8c2013a68 P 1ca2725d290e47d68b1a4b609c9816ac [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:20:25.677637 16220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74ca8140f35c4ae29e8a664f6bdfc829" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:25.679116 16790 leader_election.cc:304] T 829b4ad8c5a54eababeea1181b960d4b P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:20:25.633339 19725 raft_consensus.cc:493] T 829b4ad8c5a54eababeea1181b960d4b P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:25.664646 19345 raft_consensus.cc:493] T cdbd78acb4cf46298e0de8ba0d130e19 P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:25.662461 19450 raft_consensus.cc:2804] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:20:25.651930 19526 raft_consensus.cc:493] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:25.665809 19667 raft_consensus.cc:493] T d16dfad0c3b540e5b4b86492ddf87208 P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Starting leader election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:20:25.669673 19719 raft_consensus.cc:2749] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:20:25.661784 19688 leader_election.cc:290] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
W20260709 13:20:25.717418 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 199125us
W20260709 13:20:25.688259 19532 consensus_meta.cc:287] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b: Time spent flushing consensus metadata: real 1.054s	user 0.013s	sys 0.000s
I20260709 13:20:25.723205 16191 leader_election.cc:304] T 66f75124215f4ca6bef831e174a792d8 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
I20260709 13:20:25.729501 16191 leader_election.cc:304] T 08c74490c40243a88e62d1760b5cd814 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
I20260709 13:20:25.688128 19725 raft_consensus.cc:3060] T 829b4ad8c5a54eababeea1181b960d4b P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:20:25.731473 19698 log.cc:927] Time spent T 74ca8140f35c4ae29e8a664f6bdfc829 P cb82cc7bd6db46d980c75237a1402860: Append to log took a long time: real 0.283s	user 0.000s	sys 0.001s
W20260709 13:20:25.732473 16191 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:25.710073 19450 raft_consensus.cc:493] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:25.733176 19720 raft_consensus.cc:493] T e4dc6b0541ca431ebc757beac00521f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:25.706499 19721 raft_consensus.cc:697] T ee9ed67006144797b1de6ea8c2013a68 P 1ca2725d290e47d68b1a4b609c9816ac [term 6 LEADER]: Becoming Leader. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:20:25.688222 19707 raft_consensus.cc:2804] T 08c74490c40243a88e62d1760b5cd814 P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 13:20:25.764482 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 85454.7us
I20260709 13:20:25.714541 19667 raft_consensus.cc:3060] T d16dfad0c3b540e5b4b86492ddf87208 P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:20:25.679445 19714 raft_consensus.cc:515] T 5a96373bb0e14dce8a266b2fc6c58507 P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:25.765014 16220 raft_consensus.cc:2393] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 8: Already voted for candidate cb82cc7bd6db46d980c75237a1402860 in this term.
I20260709 13:20:25.711283 19712 raft_consensus.cc:493] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:20:25.738902 19532 raft_consensus.cc:515] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:20:25.755592 19407 consensus_meta.cc:287] T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac: Time spent flushing consensus metadata: real 0.640s	user 0.008s	sys 0.003s
I20260709 13:20:25.682906 19674 consensus_queue.cc:260] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:25.706024 19345 raft_consensus.cc:515] T cdbd78acb4cf46298e0de8ba0d130e19 P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:25.662096 19645 consensus_queue.cc:260] T c9944d5e4b8c4109af9955c4f08c9095 P 90f3da57796b488c9087a2f523adf045 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:25.796049 19723 raft_consensus.cc:493] T 68a6838d82ef4cac83507f906139d90f P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:20:25.806445 16223 raft_consensus.cc:2468] T 66f75124215f4ca6bef831e174a792d8 P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 9.
I20260709 13:20:25.807920 19656 leader_election.cc:290] T 68a6838d82ef4cac83507f906139d90f P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 9 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:20:25.788679 19738 raft_consensus.cc:2804] T 829b4ad8c5a54eababeea1181b960d4b P cb82cc7bd6db46d980c75237a1402860 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:20:25.788221 19719 raft_consensus.cc:2804] T 08c74490c40243a88e62d1760b5cd814 P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:20:25.788537 19707 raft_consensus.cc:493] T 08c74490c40243a88e62d1760b5cd814 P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Starting leader election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:20:25.768827 19450 raft_consensus.cc:3060] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:20:25.852319 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02838c86531d4f98bf58d1c22b8baf77" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
W20260709 13:20:25.853422 16115 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.589s	user 0.011s	sys 0.049s
I20260709 13:20:25.722138 19526 raft_consensus.cc:515] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:20:25.763916 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 176370us
I20260709 13:20:25.928529 16226 raft_consensus.cc:1240] T 5a96373bb0e14dce8a266b2fc6c58507 P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 7. Current term is 8. Ops: [7.3-7.3]
W20260709 13:20:26.530335 16819 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.869s	user 0.058s	sys 0.091s
W20260709 13:20:26.106500 16191 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:26.536751 16190 leader_election.cc:304] T d2ab0ac8347641e6955e9bce7d90c1a2 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860
I20260709 13:20:26.158371 16816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7ca61592e848ea9b0fb81a2ce33cdc" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
W20260709 13:20:26.118819 16265 leader_election.cc:336] T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943): Timed out: RequestConsensusVote RPC to 127.14.177.130:35943 timed out after 1.604s (SENT)
W20260709 13:20:26.111418 16266 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.715s	user 0.011s	sys 0.005s
W20260709 13:20:26.539602 16266 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.143s	user 0.014s	sys 0.012s
W20260709 13:20:26.540118 16266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 1.15744e+06us
I20260709 13:20:26.075273 16218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "829b4ad8c5a54eababeea1181b960d4b" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
W20260709 13:20:26.542120 16266 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:20:26.543181 16115 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.279s	user 0.015s	sys 0.064s
W20260709 13:20:26.543340 16191 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:20:26.530687 16819 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.869s	user 0.058s	sys 0.091s
I20260709 13:20:26.530025 16217 raft_consensus.cc:2468] T 02838c86531d4f98bf58d1c22b8baf77 P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 8.
I20260709 13:20:26.550432 16820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c11074579b046d99e585c2818e18a2c" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:26.551723 16820 raft_consensus.cc:2393] T 8c11074579b046d99e585c2818e18a2c P cb82cc7bd6db46d980c75237a1402860 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 8: Already voted for candidate cb82cc7bd6db46d980c75237a1402860 in this term.
I20260709 13:20:26.538885 16265 leader_election.cc:304] T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860
I20260709 13:20:26.538076 16223 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57604 (request call id 2334) took 1040 ms. Trace:
I20260709 13:20:26.573508 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e4bfb9090c8460eb14fa7e0d818e7f1" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
W20260709 13:20:26.568996 16296 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48674 (request call id 2389) took 1925 ms (client timeout 1957 ms). Trace:
W20260709 13:20:26.574457 16296 rpcz_store.cc:269] 0709 13:20:24.643855 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:24.645037 (+  1182us) service_pool.cc:224] Handling call
0709 13:20:26.568727 (+1923690us) spinlock_profiling.cc:243] Waited 819 ms on lock 0x7b54002cfbd4. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:20:26.568913 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1883849472}
W20260709 13:20:25.816129 19735 log.cc:927] Time spent T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 0.162s	user 0.001s	sys 0.000s
W20260709 13:20:26.271744 19673 consensus_meta.cc:287] T 3884f864a96d428785895e5463e5728f P cb82cc7bd6db46d980c75237a1402860: Time spent flushing consensus metadata: real 1.215s	user 0.013s	sys 0.000s
W20260709 13:20:26.384862 19667 consensus_meta.cc:287] T d16dfad0c3b540e5b4b86492ddf87208 P 1ca2725d290e47d68b1a4b609c9816ac: Time spent flushing consensus metadata: real 0.523s	user 0.011s	sys 0.000s
W20260709 13:20:25.830119 19113 consensus_meta.cc:287] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac: Time spent flushing consensus metadata: real 0.714s	user 0.011s	sys 0.000s
W20260709 13:20:25.840730 19729 log.cc:927] Time spent T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 0.178s	user 0.001s	sys 0.000s
I20260709 13:20:25.815554 19660 raft_consensus.cc:493] T fffab26427e24d398e2695aad40ca134 P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:20:26.181891 19686 log.cc:927] Time spent T 8cc959fb7f704fba9009c0595467d074 P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 1.213s	user 0.000s	sys 0.003s
W20260709 13:20:26.403718 19706 consensus_meta.cc:287] T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b: Time spent flushing consensus metadata: real 0.738s	user 0.010s	sys 0.000s
W20260709 13:20:25.929299 19465 consensus_meta.cc:287] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860: Time spent flushing consensus metadata: real 1.030s	user 0.012s	sys 0.001s
I20260709 13:20:26.575794 16223 rpcz_store.cc:276] 0709 13:20:25.497490 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:25.497607 (+   117us) service_pool.cc:224] Handling call
0709 13:20:26.537995 (+1040388us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:20:26.615851 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 497065us
W20260709 13:20:26.271308 19664 consensus_meta.cc:287] T ada5a004ba7c46819051e3ccec71ffdb P 90f3da57796b488c9087a2f523adf045: Time spent flushing consensus metadata: real 0.811s	user 0.007s	sys 0.007s
I20260709 13:20:26.528928 19666 raft_consensus.cc:3060] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:20:26.621987 19197 raft_consensus.cc:493] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:26.622548 19197 raft_consensus.cc:515] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:26.074170 19540 raft_consensus.cc:3055] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:20:26.623353 19540 raft_consensus.cc:740] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:20:26.624347 19540 consensus_queue.cc:260] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:26.625484 19540 raft_consensus.cc:3060] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:20:26.616062 19656 raft_consensus.cc:2749] T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:20:25.818974 19589 consensus_meta.cc:287] T 18ad7e3af5504571a69d10c00750a01c P 90f3da57796b488c9087a2f523adf045: Time spent flushing consensus metadata: real 0.967s	user 0.005s	sys 0.005s
I20260709 13:20:26.627676 19589 raft_consensus.cc:515] T 18ad7e3af5504571a69d10c00750a01c P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } }
I20260709 13:20:26.110759 19715 raft_consensus.cc:515] T 84f2da98864b4425bf7c0055951fdd4b P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:26.629899 19589 leader_election.cc:290] T 18ad7e3af5504571a69d10c00750a01c P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 8 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:20:26.630640 16295 raft_consensus.cc:2393] T 18ad7e3af5504571a69d10c00750a01c P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 8: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
W20260709 13:20:26.037202 19672 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.699s	user 0.001s	sys 0.017s
I20260709 13:20:26.631505 19715 leader_election.cc:290] T 84f2da98864b4425bf7c0055951fdd4b P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 7 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:25.929971 19587 raft_consensus.cc:493] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:26.632262 19587 raft_consensus.cc:515] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:26.601951 19673 raft_consensus.cc:515] T 3884f864a96d428785895e5463e5728f P cb82cc7bd6db46d980c75237a1402860 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:20:26.634318 19702 consensus_meta.cc:287] T 8cc959fb7f704fba9009c0595467d074 P 1ca2725d290e47d68b1a4b609c9816ac: Time spent flushing consensus metadata: real 1.154s	user 0.011s	sys 0.000s
I20260709 13:20:26.634500 19587 leader_election.cc:290] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:20:26.126551 19726 raft_consensus.cc:493] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:26.635370 16825 raft_consensus.cc:1240] T 3884f864a96d428785895e5463e5728f P cb82cc7bd6db46d980c75237a1402860 [term 7 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 6. Current term is 7. Ops: []
I20260709 13:20:26.635335 19726 raft_consensus.cc:515] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:26.048740 19714 leader_election.cc:290] T 5a96373bb0e14dce8a266b2fc6c58507 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 8 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:26.576519 19751 raft_consensus.cc:2749] T d2ab0ac8347641e6955e9bce7d90c1a2 P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:20:26.373471 19650 raft_consensus.cc:493] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
W20260709 13:20:25.834339 19676 consensus_meta.cc:287] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac: Time spent flushing consensus metadata: real 0.737s	user 0.010s	sys 0.002s
I20260709 13:20:26.604204 16816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7ca61592e848ea9b0fb81a2ce33cdc" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:26.130604 19532 leader_election.cc:290] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 10 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:26.565152 16190 leader_election.cc:304] T 7c2095a5f4ad49408dbc7f69c606120d P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
W20260709 13:20:26.662061 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 97041us
I20260709 13:20:26.574514 16217 raft_consensus.cc:2468] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 8.
I20260709 13:20:26.662511 16816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c11074579b046d99e585c2818e18a2c" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:26.662847 16217 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57600 (request call id 1289) took 1060 ms. Trace:
I20260709 13:20:25.825841 19725 raft_consensus.cc:515] T 829b4ad8c5a54eababeea1181b960d4b P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:26.665845 19345 leader_election.cc:290] T cdbd78acb4cf46298e0de8ba0d130e19 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:26.667197 19725 leader_election.cc:290] T 829b4ad8c5a54eababeea1181b960d4b P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 9 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:26.551640 19738 raft_consensus.cc:493] T 829b4ad8c5a54eababeea1181b960d4b P cb82cc7bd6db46d980c75237a1402860 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:25.828187 19716 leader_election.cc:290] T 23f82224f4564fea93572e8206bad1fe P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:25.847827 19559 leader_election.cc:290] T 8638ee363273426db1393dd81e5c7b72 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:26.553498 19707 raft_consensus.cc:3060] T 08c74490c40243a88e62d1760b5cd814 P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:20:26.616866 16223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8158bbd17b994d2eb96f560bd0f1b88d" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:26.536875 19719 raft_consensus.cc:493] T 08c74490c40243a88e62d1760b5cd814 P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:25.821439 19700 leader_election.cc:290] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:25.876581 19712 raft_consensus.cc:515] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:26.126289 19610 consensus_queue.cc:260] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:20:26.631680 19672 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.293s	user 0.001s	sys 0.025s
I20260709 13:20:26.039494 19609 raft_consensus.cc:493] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:26.637683 16825 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49896 (request call id 1034) took 1172 ms. Trace:
I20260709 13:20:26.107239 19721 consensus_queue.cc:237] T ee9ed67006144797b1de6ea8c2013a68 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:26.578850 19674 raft_consensus.cc:3060] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:20:25.877127 19620 leader_election.cc:290] T 19baa7bc367844c288bca8f81439c413 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:25.815151 19699 raft_consensus.cc:515] T 746aabf856ab401c9ef5e859371e7b67 P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:25.795343 19720 raft_consensus.cc:515] T e4dc6b0541ca431ebc757beac00521f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:26.609211 19651 raft_consensus.cc:493] T 26354361e9b044cea1c2b86c4882fe92 P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:20:26.663220 16217 rpcz_store.cc:276] 0709 13:20:25.602096 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:26.573153 (+971057us) service_pool.cc:224] Handling call
0709 13:20:26.662790 (+ 89637us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:20:26.686405 19718 consensus_meta.cc:287] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045: Time spent flushing consensus metadata: real 1.007s	user 0.012s	sys 0.000s
I20260709 13:20:26.041055 19732 raft_consensus.cc:493] T ee9ed67006144797b1de6ea8c2013a68 P cb82cc7bd6db46d980c75237a1402860 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:26.712643 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87ac0d1f7e8a419f9bcbc04aee0e067e" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:26.712875 16219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cc959fb7f704fba9009c0595467d074" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:20:26.663717 16816 raft_consensus.cc:2468] T 8c11074579b046d99e585c2818e18a2c P cb82cc7bd6db46d980c75237a1402860 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 8.
I20260709 13:20:26.713610 16219 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57600 (request call id 1290) took 1111 ms. Trace:
I20260709 13:20:26.715271 16219 rpcz_store.cc:276] 0709 13:20:25.602393 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:26.691384 (+1088991us) service_pool.cc:224] Handling call
0709 13:20:26.713558 (+ 22174us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:20:26.638125 19645 raft_consensus.cc:3060] T c9944d5e4b8c4109af9955c4f08c9095 P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:20:25.839013 19710 raft_consensus.cc:515] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:26.668455 16218 raft_consensus.cc:2393] T 829b4ad8c5a54eababeea1181b960d4b P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 in current term 9: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
I20260709 13:20:26.554004 19407 raft_consensus.cc:515] T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:20:26.660683 19414 consensus_meta.cc:287] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045: Time spent flushing consensus metadata: real 0.982s	user 0.003s	sys 0.009s
I20260709 13:20:26.571444 19723 raft_consensus.cc:515] T 68a6838d82ef4cac83507f906139d90f P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:26.672794 16223 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57600 (request call id 1287) took 1071 ms. Trace:
I20260709 13:20:26.731773 16223 rpcz_store.cc:276] 0709 13:20:25.601474 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:26.616603 (+1015129us) service_pool.cc:224] Handling call
0709 13:20:26.672740 (+ 56137us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:20:26.618378 19465 raft_consensus.cc:2749] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:20:26.634768 19702 raft_consensus.cc:515] T 8cc959fb7f704fba9009c0595467d074 P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:26.602816 19667 raft_consensus.cc:515] T d16dfad0c3b540e5b4b86492ddf87208 P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:26.614666 19660 raft_consensus.cc:515] T fffab26427e24d398e2695aad40ca134 P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:26.684083 16825 rpcz_store.cc:276] 0709 13:20:25.465191 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:25.467368 (+  2177us) service_pool.cc:224] Handling call
0709 13:20:25.467696 (+   328us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:26.635912 (+1168216us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:26.637475 (+  1563us) spinlock_profiling.cc:243] Waited 1.17 s on lock 0x7b54006bf654. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:20:26.637630 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2684960000}
I20260709 13:20:26.662778 19650 raft_consensus.cc:515] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:26.685657 19610 raft_consensus.cc:3060] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:20:26.685626 19609 raft_consensus.cc:515] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:26.670595 19738 raft_consensus.cc:3060] T 829b4ad8c5a54eababeea1181b960d4b P cb82cc7bd6db46d980c75237a1402860 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:20:26.691661 16817 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49896 (request call id 1036) took 1225 ms. Trace:
I20260709 13:20:26.750828 16817 rpcz_store.cc:276] 0709 13:20:25.465737 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:25.467923 (+  2186us) service_pool.cc:224] Handling call
0709 13:20:25.468328 (+   405us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:20:25.632192 (+163864us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:26.691436 (+1059244us) spinlock_profiling.cc:243] Waited 122 ms on lock 0x7b5400787154. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:20:26.691607 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":281383808}
W20260709 13:20:26.691991 16300 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48674 (request call id 2393) took 2039 ms (client timeout 2180 ms). Trace:
W20260709 13:20:26.697117 16189 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:26.699931 16117 leader_election.cc:304] T 746aabf856ab401c9ef5e859371e7b67 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:20:26.703351 16221 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57604 (request call id 2312) took 2523 ms. Trace:
W20260709 13:20:26.752318 16189 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:26.707875 16294 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48684 (request call id 2292) took 2236 ms. Trace:
W20260709 13:20:26.752386 16117 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 52527.3us
I20260709 13:20:26.549610 19717 raft_consensus.cc:2749] T 66f75124215f4ca6bef831e174a792d8 P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:20:26.753563 16117 leader_election.cc:304] T 2794af5b99984efebdee564035841cb0 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860
I20260709 13:20:26.642352 19664 raft_consensus.cc:515] T ada5a004ba7c46819051e3ccec71ffdb P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:26.709385 16819 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49896 (request call id 1035) took 1244 ms. Trace:
I20260709 13:20:26.679082 19719 raft_consensus.cc:3060] T 08c74490c40243a88e62d1760b5cd814 P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:20:26.637530 19706 raft_consensus.cc:515] T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:26.713305 16217 raft_consensus.cc:2468] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 8.
I20260709 13:20:26.714740 16818 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49896 (request call id 1038) took 1076 ms. Trace:
I20260709 13:20:26.664357 16190 leader_election.cc:304] T fffab26427e24d398e2695aad40ca134 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:20:26.717691 16824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f82224f4564fea93572e8206bad1fe" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 8 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:26.718336 16822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdbd78acb4cf46298e0de8ba0d130e19" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:26.719960 16224 raft_consensus.cc:2393] T 746aabf856ab401c9ef5e859371e7b67 P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 9: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
I20260709 13:20:26.730952 16266 leader_election.cc:304] T 8c11074579b046d99e585c2818e18a2c P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:20:26.733091 16219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c2095a5f4ad49408dbc7f69c606120d" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:26.702567 19657 leader_election.cc:290] T c4f65cdf00b748bb963ee6e81ef43b79 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:20:26.739553 16816 raft_consensus.cc:1240] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [term 10 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 9. Current term is 10. Ops: []
I20260709 13:20:26.741395 19421 raft_consensus.cc:493] T c0eab25ce4c64e018a041beeda2867db P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:20:26.718863 19718 raft_consensus.cc:515] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:26.740317 16223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9944d5e4b8c4109af9955c4f08c9095" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:20:26.766786 16223 raft_consensus.cc:3060] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:20:26.737135 19700 consensus_queue.cc:1059] T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: INVALID_TERM, Last received: 8.4, Next index: 5, Last known committed idx: 2, Time since last communication: 0.008s
W20260709 13:20:26.747656 19680 log.cc:927] Time spent T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 1.929s	user 0.010s	sys 0.014s
I20260709 13:20:26.743814 19660 leader_election.cc:290] T fffab26427e24d398e2695aad40ca134 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:20:26.740803 19726 leader_election.cc:290] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:26.758004 19751 raft_consensus.cc:493] T ecc1e363230043fe9fef390f0b979f20 P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:26.747485 19609 leader_election.cc:290] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
W20260709 13:20:26.742424 19758 log.cc:927] Time spent T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
W20260709 13:20:26.743779 19752 log.cc:927] Time spent T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860: Append to log took a long time: real 0.076s	user 0.000s	sys 0.000s
I20260709 13:20:26.752796 16294 rpcz_store.cc:276] 0709 13:20:24.455093 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:24.455218 (+   125us) service_pool.cc:224] Handling call
0709 13:20:26.690889 (+2235671us) spinlock_profiling.cc:243] Waited 1.05 s on lock 0x7b540041c3d4. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:20:26.691014 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2423829888}
I20260709 13:20:26.766842 19666 raft_consensus.cc:3060] T fffab26427e24d398e2695aad40ca134 P 1ca2725d290e47d68b1a4b609c9816ac [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:20:26.752004 16300 rpcz_store.cc:269] 0709 13:20:24.652202 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:24.652337 (+   135us) service_pool.cc:224] Handling call
0709 13:20:24.675728 (+ 23391us) spinlock_profiling.cc:243] Waited 2.35 ms on lock 0x7b5800090098. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69ad8fadb1 00007f69ada3ab71 00007f69ada3aeee 0000000000375939 00007f69b5aae1e0 00007f69b5b770c4 00007f69b5b77131 00007f69b5b250a0 00007f69b5b24984 00007f69af09f6d4 00007f69af09f662
0709 13:20:26.691757 (+2016029us) spinlock_profiling.cc:243] Waited 183 ms on lock 0x7b54002e1654. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:20:26.691917 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":426196224}
I20260709 13:20:26.792917 16824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62fe3346cd3c42c4a55e14af9073bca4" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
W20260709 13:20:26.755385 16265 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:26.793571 16824 raft_consensus.cc:2468] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 10.
I20260709 13:20:26.752511 16221 rpcz_store.cc:276] 0709 13:20:24.179765 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:24.183397 (+  3632us) service_pool.cc:224] Handling call
0709 13:20:24.216050 (+ 32653us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:20:24.409650 (+193600us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:24.409677 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:20:24.454674 (+ 44997us) log.cc:844] Serialized 26 byte log entry
0709 13:20:24.743518 (+288844us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:20:24.743559 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:24.744055 (+   496us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:20:26.130740 (+1386685us) spinlock_profiling.cc:243] Waited 7.78 ms on lock 0x7b4400022b4c. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69ad8fadb1 00007f69ad998c85 00007f69ad9996a2 00007f69b070b1db 00007f69b0706aaf 00007f69b06fa86e 00007f69b06f9bb6 00007f69b06f97d8 00007f69b0703221 00007f69b07031d9 00007f69b0703169
0709 13:20:26.689442 (+558702us) raft_consensus.cc:1685] finished
0709 13:20:26.689461 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:26.703253 (+ 13792us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":196616,"raft.run_cpu_time_us":120,"raft.run_wall_time_us":123,"spinlock_wait_cycles":17885952,"thread_start_us":425443,"threads_started":2,"wal-append.queue_time_us":427593}
I20260709 13:20:26.766108 19345 raft_consensus.cc:2804] T 8c11074579b046d99e585c2818e18a2c P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:20:26.712918 19651 raft_consensus.cc:515] T 26354361e9b044cea1c2b86c4882fe92 P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:26.761709 16217 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57600 (request call id 1283) took 1169 ms. Trace:
I20260709 13:20:26.796921 16217 rpcz_store.cc:276] 0709 13:20:25.592240 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:26.708483 (+1116243us) service_pool.cc:224] Handling call
0709 13:20:26.761651 (+ 53168us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:20:26.761873 16818 rpcz_store.cc:276] 0709 13:20:25.638028 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:26.694422 (+1056394us) service_pool.cc:224] Handling call
0709 13:20:26.714194 (+ 19772us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:26.714393 (+   199us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:26.714428 (+    35us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:20:26.714474 (+    46us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:20:26.714511 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:26.714580 (+    69us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:26.714680 (+   100us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:20:26.756174 16819 rpcz_store.cc:276] 0709 13:20:25.465462 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:25.468783 (+  3321us) service_pool.cc:224] Handling call
0709 13:20:25.486754 (+ 17971us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:20:25.588678 (+101924us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:25.588723 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:20:25.589288 (+   565us) log.cc:844] Serialized 26 byte log entry
0709 13:20:26.549280 (+959992us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:20:26.549339 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:26.695089 (+145750us) spinlock_profiling.cc:243] Waited 102 ms on lock 0x7b54007d0354. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:20:26.695153 (+    64us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:20:26.695241 (+    88us) raft_consensus.cc:1685] finished
0709 13:20:26.695262 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:26.709333 (+ 14071us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":46060,"raft.queue_time_us":124963,"raft.run_cpu_time_us":141,"raft.run_wall_time_us":7075,"spinlock_wait_cycles":233457536,"thread_start_us":887831,"threads_started":1,"wal-append.queue_time_us":998020}
W20260709 13:20:26.764353 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 100124us
I20260709 13:20:26.739310 19760 raft_consensus.cc:493] T 26354361e9b044cea1c2b86c4882fe92 P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:20:26.767524 16219 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57600 (request call id 1292) took 1164 ms. Trace:
I20260709 13:20:26.719838 19732 raft_consensus.cc:515] T ee9ed67006144797b1de6ea8c2013a68 P cb82cc7bd6db46d980c75237a1402860 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:26.771718 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 9 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:26.808015 16150 raft_consensus.cc:2410] T ecc1e363230043fe9fef390f0b979f20 P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 for term 9 because replica has last-logged OpId of term: 2 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 3.
I20260709 13:20:26.773497 16143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18ad7e3af5504571a69d10c00750a01c" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:20:26.741967 19526 leader_election.cc:290] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:26.731317 19724 raft_consensus.cc:493] T 9e65676b21b748728ad39668cfe722ec P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:26.786046 16142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68a6838d82ef4cac83507f906139d90f" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:20:26.820017 16142 raft_consensus.cc:3060] T 68a6838d82ef4cac83507f906139d90f P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:20:26.786543 16144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fffab26427e24d398e2695aad40ca134" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:26.821209 16144 raft_consensus.cc:2468] T fffab26427e24d398e2695aad40ca134 P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 7.
I20260709 13:20:26.787947 16145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f82224f4564fea93572e8206bad1fe" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 8 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:26.736291 19414 raft_consensus.cc:515] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:26.759948 19754 raft_consensus.cc:2804] T 746aabf856ab401c9ef5e859371e7b67 P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:20:26.776288 19532 raft_consensus.cc:493] T e5ad4eb281564936a59f41e387a47156 P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:26.804862 16219 rpcz_store.cc:276] 0709 13:20:25.602984 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:26.715928 (+1112944us) service_pool.cc:224] Handling call
0709 13:20:26.767447 (+ 51519us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:20:26.806639 16190 leader_election.cc:304] T 02838c86531d4f98bf58d1c22b8baf77 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:20:26.775511 19762 raft_consensus.cc:2749] T 2794af5b99984efebdee564035841cb0 P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:20:26.831940 16824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5718c623ebee4d2bb770f0f9ddb2f164" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:26.833274 16824 raft_consensus.cc:2410] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b for term 9 because replica has last-logged OpId of term: 7 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 8.
I20260709 13:20:26.833521 19522 raft_consensus.cc:493] T 434bff3082d141838068f323bdb0d1ed P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:26.834008 19522 raft_consensus.cc:515] T 434bff3082d141838068f323bdb0d1ed P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:26.836849 19702 leader_election.cc:290] T 8cc959fb7f704fba9009c0595467d074 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 9 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:26.837523 19710 leader_election.cc:290] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 11 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:26.837668 19526 raft_consensus.cc:2749] T 02838c86531d4f98bf58d1c22b8baf77 P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:20:26.809715 16817 raft_consensus.cc:1240] T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860 [term 10 LEADER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 9. Current term is 10. Ops: []
I20260709 13:20:26.782387 19725 raft_consensus.cc:3055] T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:20:26.818266 16221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746aabf856ab401c9ef5e859371e7b67" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 9 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:26.818928 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a959dbecc2648eb85d1cc550f5a3245" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:20:26.825434 16823 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49896 (request call id 1037) took 1288 ms. Trace:
I20260709 13:20:26.770860 19421 raft_consensus.cc:515] T c0eab25ce4c64e018a041beeda2867db P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:26.830816 16219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66f75124215f4ca6bef831e174a792d8" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:20:26.853314 16221 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33050 (request call id 601) took 1065 ms. Trace:
I20260709 13:20:26.798137 19345 raft_consensus.cc:493] T 8c11074579b046d99e585c2818e18a2c P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:26.849699 16217 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57600 (request call id 1291) took 1247 ms. Trace:
I20260709 13:20:26.795455 19696 consensus_queue.cc:1059] T 3884f864a96d428785895e5463e5728f P 647a343a542d4a139fe106c19095129b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.036s
I20260709 13:20:26.807034 19197 leader_election.cc:290] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:26.795640 19726 consensus_queue.cc:1048] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.036s
I20260709 13:20:26.855249 16217 rpcz_store.cc:276] 0709 13:20:25.602694 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:26.797609 (+1194915us) service_pool.cc:224] Handling call
0709 13:20:26.849639 (+ 52030us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:20:26.855511 16221 rpcz_store.cc:276] 0709 13:20:25.787794 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:26.810490 (+1022696us) service_pool.cc:224] Handling call
0709 13:20:26.853250 (+ 42760us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:20:26.849491 16823 rpcz_store.cc:276] 0709 13:20:25.537150 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:25.537700 (+   550us) service_pool.cc:224] Handling call
0709 13:20:25.538096 (+   396us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:20:25.854557 (+316461us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:25.854602 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:20:26.534084 (+679482us) log.cc:844] Serialized 26 byte log entry
0709 13:20:26.738290 (+204206us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:20:26.738327 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:26.739670 (+  1343us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:20:26.825232 (+ 85562us) raft_consensus.cc:1685] finished
0709 13:20:26.825253 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:26.825383 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":44424,"raft.queue_time_us":170356,"raft.run_cpu_time_us":112,"raft.run_wall_time_us":114,"thread_start_us":441289,"threads_started":2,"wal-append.queue_time_us":187127}
I20260709 13:20:26.853684 16219 raft_consensus.cc:2393] T 66f75124215f4ca6bef831e174a792d8 P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 in current term 9: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
I20260709 13:20:26.856637 19345 raft_consensus.cc:3060] T 8c11074579b046d99e585c2818e18a2c P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:20:26.858202 19699 leader_election.cc:290] T 746aabf856ab401c9ef5e859371e7b67 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 9 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:26.868484 16218 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33050 (request call id 598) took 1109 ms. Trace:
I20260709 13:20:26.810603 19760 raft_consensus.cc:515] T 26354361e9b044cea1c2b86c4882fe92 P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:26.910259 16221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4dc6b0541ca431ebc757beac00521f1" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:26.910959 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b7d1ab519044eda2d5e04a07242f25" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:26.824609 19569 consensus_queue.cc:260] T 8cc959fb7f704fba9009c0595467d074 P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:26.875387 19660 raft_consensus.cc:493] T 57a744f5aaf944678a63469cbf514015 P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:26.830113 19724 raft_consensus.cc:515] T 9e65676b21b748728ad39668cfe722ec P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:26.908887 16218 rpcz_store.cc:276] 0709 13:20:25.758663 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:26.074911 (+316248us) service_pool.cc:224] Handling call
0709 13:20:26.868261 (+793350us) spinlock_profiling.cc:243] Waited 126 ms on lock 0x7b54001419d4. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:20:26.868410 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":289959680}
I20260709 13:20:26.869899 19673 leader_election.cc:290] T 3884f864a96d428785895e5463e5728f P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 7 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:26.910960 16221 raft_consensus.cc:2468] T e4dc6b0541ca431ebc757beac00521f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 7.
I20260709 13:20:26.920583 16221 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57600 (request call id 1294) took 1284 ms. Trace:
I20260709 13:20:26.920921 16221 rpcz_store.cc:276] 0709 13:20:25.636645 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:26.907800 (+1271155us) service_pool.cc:224] Handling call
0709 13:20:26.920525 (+ 12725us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:20:26.915627 16219 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33050 (request call id 605) took 1124 ms. Trace:
W20260709 13:20:26.915359 19578 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.948s	user 0.020s	sys 0.029s
I20260709 13:20:26.916076 16217 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33050 (request call id 597) took 1157 ms. Trace:
I20260709 13:20:26.922544 16217 rpcz_store.cc:276] 0709 13:20:25.758311 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:26.882445 (+1124134us) service_pool.cc:224] Handling call
0709 13:20:26.916022 (+ 33577us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:20:26.844463 19754 raft_consensus.cc:697] T 746aabf856ab401c9ef5e859371e7b67 P 647a343a542d4a139fe106c19095129b [term 9 LEADER]: Becoming Leader. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
W20260709 13:20:26.917639 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 111078us
I20260709 13:20:26.852932 19725 raft_consensus.cc:740] T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:20:26.921435 16219 rpcz_store.cc:276] 0709 13:20:25.791028 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:26.829366 (+1038338us) service_pool.cc:224] Handling call
0709 13:20:26.915539 (+ 86173us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:20:26.928691 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5ad4eb281564936a59f41e387a47156" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:26.929767 16151 raft_consensus.cc:1240] T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b [term 10 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 9. Current term is 10. Ops: [9.7-9.7]
I20260709 13:20:26.930240 16143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8158bbd17b994d2eb96f560bd0f1b88d" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:26.916255 16117 leader_election.cc:304] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860; no voters: 
W20260709 13:20:26.924736 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 152520us
I20260709 13:20:26.925189 16824 raft_consensus.cc:1275] T af57996a8be84d398f0f023e8d08ef77 P cb82cc7bd6db46d980c75237a1402860 [term 9 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 13:20:26.925812 19589 raft_consensus.cc:493] T 434bff3082d141838068f323bdb0d1ed P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:26.933148 16221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7ca61592e848ea9b0fb81a2ce33cdc" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:26.933873 16221 raft_consensus.cc:2468] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 10.
I20260709 13:20:26.934327 16221 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57600 (request call id 1295) took 1295 ms. Trace:
I20260709 13:20:26.934612 16221 rpcz_store.cc:276] 0709 13:20:25.639255 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:26.921514 (+1282259us) service_pool.cc:224] Handling call
0709 13:20:26.934270 (+ 12756us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:20:26.931953 16143 raft_consensus.cc:1240] T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 6. Current term is 7. Ops: [6.8-6.8]
I20260709 13:20:26.841986 19532 raft_consensus.cc:515] T e5ad4eb281564936a59f41e387a47156 P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:26.926222 19753 raft_consensus.cc:493] T c0eab25ce4c64e018a041beeda2867db P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:20:26.935452 16116 leader_election.cc:304] T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: cb82cc7bd6db46d980c75237a1402860
I20260709 13:20:26.933193 16218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f5f89186204242bf377f7c42b2e2da" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:20:26.940135 16190 leader_election.cc:304] T 1df93779209748b9860f6d893dd3de53 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 647a343a542d4a139fe106c19095129b
W20260709 13:20:26.922184 19578 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.967s	user 0.020s	sys 0.029s
I20260709 13:20:26.947877 19753 raft_consensus.cc:515] T c0eab25ce4c64e018a041beeda2867db P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:26.938911 19589 raft_consensus.cc:515] T 434bff3082d141838068f323bdb0d1ed P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:26.926350 19656 raft_consensus.cc:493] T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
W20260709 13:20:26.980849 16266 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:26.955786 16144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d16dfad0c3b540e5b4b86492ddf87208" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:26.982585 16144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62fe3346cd3c42c4a55e14af9073bca4" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:20:26.984455 19421 leader_election.cc:290] T c0eab25ce4c64e018a041beeda2867db P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:26.986912 16150 raft_consensus.cc:2468] T e5ad4eb281564936a59f41e387a47156 P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 9.
I20260709 13:20:26.956233 16218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af57996a8be84d398f0f023e8d08ef77" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:26.989705 16218 raft_consensus.cc:2468] T af57996a8be84d398f0f023e8d08ef77 P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 9.
I20260709 13:20:26.990365 16218 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33050 (request call id 603) took 1200 ms. Trace:
I20260709 13:20:26.990796 16218 rpcz_store.cc:276] 0709 13:20:25.790427 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:26.955074 (+1164647us) service_pool.cc:224] Handling call
0709 13:20:26.990282 (+ 35208us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:20:26.992285 19657 raft_consensus.cc:493] T 2794af5b99984efebdee564035841cb0 P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:20:26.992476 19672 raft_consensus.cc:493] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:26.992898 19657 raft_consensus.cc:515] T 2794af5b99984efebdee564035841cb0 P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:26.923703 19660 raft_consensus.cc:515] T 57a744f5aaf944678a63469cbf514015 P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:26.958031 16143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87ac0d1f7e8a419f9bcbc04aee0e067e" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:26.926359 19770 raft_consensus.cc:493] T 57a744f5aaf944678a63469cbf514015 P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:26.959198 16221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62fe3346cd3c42c4a55e14af9073bca4" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:26.789860 19751 raft_consensus.cc:515] T ecc1e363230043fe9fef390f0b979f20 P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:26.930281 19569 raft_consensus.cc:3060] T 8cc959fb7f704fba9009c0595467d074 P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:20:26.962517 16151 raft_consensus.cc:1240] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 7. Current term is 8. Ops: [7.9-7.9]
I20260709 13:20:26.966804 16223 raft_consensus.cc:2468] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 9.
I20260709 13:20:26.936990 19696 raft_consensus.cc:2804] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:20:27.098708 16218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c11074579b046d99e585c2818e18a2c" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
W20260709 13:20:27.215299 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 275237us
I20260709 13:20:27.309638 16190 leader_election.cc:304] T b6f07842d3aa47858df5665874c86f3a P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 647a343a542d4a139fe106c19095129b
I20260709 13:20:27.024586 16142 raft_consensus.cc:2468] T 68a6838d82ef4cac83507f906139d90f P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 9.
W20260709 13:20:27.220628 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 260552us
I20260709 13:20:27.311467 16142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c61a39fd4f84424b129c1a6f9ba404d" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:20:27.214839 16221 raft_consensus.cc:2393] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 11: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
I20260709 13:20:27.309661 16218 raft_consensus.cc:2410] T 8c11074579b046d99e585c2818e18a2c P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 for term 9 because replica has last-logged OpId of term: 7 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 8.
I20260709 13:20:27.313470 16218 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33050 (request call id 600) took 1532 ms. Trace:
I20260709 13:20:26.959474 19651 leader_election.cc:290] T 26354361e9b044cea1c2b86c4882fe92 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:27.301506 16144 raft_consensus.cc:3060] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:20:26.961136 19407 leader_election.cc:290] T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 8 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
W20260709 13:20:27.071630 16189 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:27.221491 16223 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33050 (request call id 599) took 1457 ms. Trace:
I20260709 13:20:27.334766 16223 rpcz_store.cc:276] 0709 13:20:25.764575 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:26.732507 (+967932us) service_pool.cc:224] Handling call
0709 13:20:27.221410 (+488903us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:20:26.998656 16117 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:27.336895 16223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5718c623ebee4d2bb770f0f9ddb2f164" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:26.979759 19733 raft_consensus.cc:2804] T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:20:27.313884 16218 rpcz_store.cc:276] 0709 13:20:25.781111 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:26.991679 (+1210568us) service_pool.cc:224] Handling call
0709 13:20:27.313405 (+321726us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:20:27.324257 16190 leader_election.cc:304] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:20:26.952339 19667 leader_election.cc:290] T d16dfad0c3b540e5b4b86492ddf87208 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 8 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
W20260709 13:20:27.306012 16789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 388268us
I20260709 13:20:27.201128 19760 leader_election.cc:290] T 26354361e9b044cea1c2b86c4882fe92 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:27.200685 19732 leader_election.cc:290] T ee9ed67006144797b1de6ea8c2013a68 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:20:27.393158 16221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d4ab870a05347868956aa8b841b2981" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 9 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:27.303819 19682 raft_consensus.cc:493] T ee9ed67006144797b1de6ea8c2013a68 P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:27.038144 19666 raft_consensus.cc:2749] T fffab26427e24d398e2695aad40ca134 P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:20:27.201399 19718 leader_election.cc:290] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 9 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:27.067497 19664 leader_election.cc:290] T ada5a004ba7c46819051e3ccec71ffdb P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 9 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:27.100220 19702 raft_consensus.cc:493] T 19baa7bc367844c288bca8f81439c413 P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:27.401554 16144 raft_consensus.cc:2468] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 11.
I20260709 13:20:26.985601 19700 raft_consensus.cc:2804] T 1df93779209748b9860f6d893dd3de53 P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:20:27.420212 16218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
W20260709 13:20:27.404592 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 80447.2us
W20260709 13:20:27.405535 16789 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:27.421253 16147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cc959fb7f704fba9009c0595467d074" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:20:27.421583 16190 leader_election.cc:304] T 8a878999c0024d2abe517484180290fc P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:20:27.341241 19651 raft_consensus.cc:493] T d16dfad0c3b540e5b4b86492ddf87208 P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:20:27.410388 16292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3884f864a96d428785895e5463e5728f" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:20:27.434244 16292 raft_consensus.cc:3060] T 3884f864a96d428785895e5463e5728f P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:20:27.303893 19650 leader_election.cc:290] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:20:27.432771 16296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:27.412649 16298 raft_consensus.cc:1240] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 5. Current term is 6. Ops: []
I20260709 13:20:27.435876 16296 raft_consensus.cc:2393] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 in current term 9: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
I20260709 13:20:27.435997 19724 leader_election.cc:290] T 9e65676b21b748728ad39668cfe722ec P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:27.436995 16296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7ca61592e848ea9b0fb81a2ce33cdc" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:27.437202 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87ac0d1f7e8a419f9bcbc04aee0e067e" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:27.437678 16296 raft_consensus.cc:2468] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 10.
I20260709 13:20:27.437832 16298 raft_consensus.cc:2468] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 8.
I20260709 13:20:27.438730 16296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c61a39fd4f84424b129c1a6f9ba404d" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:20:26.992985 19672 raft_consensus.cc:515] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:27.413219 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a96373bb0e14dce8a266b2fc6c58507" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:20:27.333690 19754 consensus_queue.cc:237] T 746aabf856ab401c9ef5e859371e7b67 P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:27.343416 19671 raft_consensus.cc:2804] T b6f07842d3aa47858df5665874c86f3a P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:20:27.416028 16294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f82224f4564fea93572e8206bad1fe" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 8 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:27.453228 16294 raft_consensus.cc:2468] T 23f82224f4564fea93572e8206bad1fe P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 7.
I20260709 13:20:27.454329 16294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cc959fb7f704fba9009c0595467d074" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:20:27.417186 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31969392ccdb43b491911a319e8bc7e1" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:27.313122 19770 raft_consensus.cc:515] T 57a744f5aaf944678a63469cbf514015 P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:27.217012 19712 leader_election.cc:290] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:27.002070 19656 raft_consensus.cc:515] T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:27.420269 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8638ee363273426db1393dd81e5c7b72" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:27.463056 16300 raft_consensus.cc:2393] T 8638ee363273426db1393dd81e5c7b72 P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 in current term 8: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
I20260709 13:20:27.420126 19696 raft_consensus.cc:493] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:27.432686 19667 raft_consensus.cc:3060] T 8a878999c0024d2abe517484180290fc P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:20:27.217290 19706 leader_election.cc:290] T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 7 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
W20260709 13:20:27.431525 16115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 217664us
W20260709 13:20:27.456375 19707 consensus_meta.cc:287] T 08c74490c40243a88e62d1760b5cd814 P 647a343a542d4a139fe106c19095129b: Time spent flushing consensus metadata: real 0.776s	user 0.010s	sys 0.006s
I20260709 13:20:26.996906 19720 leader_election.cc:290] T e4dc6b0541ca431ebc757beac00521f1 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:27.412493 19682 raft_consensus.cc:515] T ee9ed67006144797b1de6ea8c2013a68 P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:27.438755 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a96373bb0e14dce8a266b2fc6c58507" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:20:27.481150 16296 raft_consensus.cc:3060] T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:20:27.419996 19660 leader_election.cc:290] T 57a744f5aaf944678a63469cbf514015 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:27.402850 19723 leader_election.cc:290] T 68a6838d82ef4cac83507f906139d90f P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:27.408183 19760 raft_consensus.cc:493] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:27.193120 19725 consensus_queue.cc:260] T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.4, Last appended by leader: 4, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:27.000871 19450 raft_consensus.cc:515] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:27.407348 19407 raft_consensus.cc:2749] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:20:27.441208 16190 leader_election.cc:304] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
W20260709 13:20:27.517509 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 76424us
I20260709 13:20:27.455123 16294 raft_consensus.cc:2393] T 8cc959fb7f704fba9009c0595467d074 P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 9: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
I20260709 13:20:27.413094 19733 raft_consensus.cc:697] T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b [term 9 LEADER]: Becoming Leader. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:20:27.518564 16190 leader_election.cc:304] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:20:27.407557 19610 consensus_queue.cc:1048] T 3a7ca61592e848ea9b0fb81a2ce33cdc P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.043s
I20260709 13:20:27.419916 19700 raft_consensus.cc:493] T 1df93779209748b9860f6d893dd3de53 P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Starting leader election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:20:27.473565 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d16dfad0c3b540e5b4b86492ddf87208" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:20:27.441532 19718 raft_consensus.cc:493] T b87147731a68400e8cce9b879daa7c5d P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:20:27.455569 16297 raft_consensus.cc:2468] T 31969392ccdb43b491911a319e8bc7e1 P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 7.
W20260709 13:20:27.456367 19645 consensus_meta.cc:287] T c9944d5e4b8c4109af9955c4f08c9095 P 90f3da57796b488c9087a2f523adf045: Time spent flushing consensus metadata: real 0.737s	user 0.007s	sys 0.003s
I20260709 13:20:27.432986 19699 raft_consensus.cc:493] T 5a959dbecc2648eb85d1cc550f5a3245 P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:20:27.540786 19699 raft_consensus.cc:515] T 5a959dbecc2648eb85d1cc550f5a3245 P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:27.543292 19699 leader_election.cc:290] T 5a959dbecc2648eb85d1cc550f5a3245 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
W20260709 13:20:27.409710 19768 log.cc:927] Time spent T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860: Append to log took a long time: real 0.405s	user 0.001s	sys 0.000s
I20260709 13:20:27.543963 19770 leader_election.cc:290] T 57a744f5aaf944678a63469cbf514015 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:27.543728 19733 consensus_queue.cc:237] T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:27.479728 16292 raft_consensus.cc:2468] T 3884f864a96d428785895e5463e5728f P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 7.
I20260709 13:20:27.545806 16821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b7d1ab519044eda2d5e04a07242f25" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:20:27.516618 19414 leader_election.cc:290] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 9 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:27.547278 16821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada5a004ba7c46819051e3ccec71ffdb" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:20:27.548431 16294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c11074579b046d99e585c2818e18a2c" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
W20260709 13:20:27.549113 19776 log.cc:927] Time spent T ee9ed67006144797b1de6ea8c2013a68 P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 0.070s	user 0.000s	sys 0.000s
I20260709 13:20:27.549429 19732 raft_consensus.cc:493] T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:27.549983 19732 raft_consensus.cc:515] T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:20:27.550463 19674 consensus_meta.cc:287] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac: Time spent flushing consensus metadata: real 0.849s	user 0.007s	sys 0.004s
I20260709 13:20:27.550295 19718 raft_consensus.cc:515] T b87147731a68400e8cce9b879daa7c5d P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:27.550989 19683 raft_consensus.cc:493] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:27.551792 19751 leader_election.cc:290] T ecc1e363230043fe9fef390f0b979f20 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:27.551548 19683 raft_consensus.cc:515] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:27.552205 19732 leader_election.cc:290] T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:27.552721 19718 leader_election.cc:290] T b87147731a68400e8cce9b879daa7c5d P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:27.553481 16817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26354361e9b044cea1c2b86c4882fe92" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:27.452998 19671 raft_consensus.cc:493] T b6f07842d3aa47858df5665874c86f3a P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: Starting leader election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:20:27.562847 19671 raft_consensus.cc:3060] T b6f07842d3aa47858df5665874c86f3a P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:20:27.485010 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d16dfad0c3b540e5b4b86492ddf87208" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:20:27.495579 19660 raft_consensus.cc:493] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:27.564949 19450 leader_election.cc:290] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 9 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:27.472759 19696 raft_consensus.cc:3060] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:20:27.565214 19692 consensus_queue.cc:260] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 7.9, Last appended by leader: 9, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:27.566336 19692 raft_consensus.cc:3060] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:20:27.504405 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 704906us
I20260709 13:20:27.446650 19651 raft_consensus.cc:515] T d16dfad0c3b540e5b4b86492ddf87208 P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:27.569792 19651 leader_election.cc:290] T d16dfad0c3b540e5b4b86492ddf87208 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:20:27.514781 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f2da98864b4425bf7c0055951fdd4b" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:20:27.570541 16298 raft_consensus.cc:3060] T 84f2da98864b4425bf7c0055951fdd4b P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:20:27.571204 16822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e65676b21b748728ad39668cfe722ec" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:27.571391 19753 leader_election.cc:290] T c0eab25ce4c64e018a041beeda2867db P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:20:27.439368 19702 raft_consensus.cc:515] T 19baa7bc367844c288bca8f81439c413 P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:27.527956 16823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434bff3082d141838068f323bdb0d1ed" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:27.531513 16818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26354361e9b044cea1c2b86c4882fe92" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
W20260709 13:20:27.521253 19540 consensus_meta.cc:287] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b: Time spent flushing consensus metadata: real 0.895s	user 0.010s	sys 0.003s
I20260709 13:20:27.602998 16298 raft_consensus.cc:2468] T 84f2da98864b4425bf7c0055951fdd4b P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 7.
I20260709 13:20:27.604139 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0eab25ce4c64e018a041beeda2867db" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:27.604853 16298 raft_consensus.cc:2468] T c0eab25ce4c64e018a041beeda2867db P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 8.
W20260709 13:20:27.532284 16266 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:27.538046 16296 raft_consensus.cc:2468] T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 8.
I20260709 13:20:27.539551 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f2da98864b4425bf7c0055951fdd4b" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:20:27.545884 16292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26354361e9b044cea1c2b86c4882fe92" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:27.555011 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdbd78acb4cf46298e0de8ba0d130e19" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:27.555558 16789 leader_election.cc:304] T 23f82224f4564fea93572e8206bad1fe P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:20:27.556651 16825 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49906 (request call id 751) took 1635 ms. Trace:
I20260709 13:20:27.608570 16825 rpcz_store.cc:276] 0709 13:20:25.921421 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:26.738165 (+816744us) service_pool.cc:224] Handling call
0709 13:20:26.748269 (+ 10104us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:20:26.748597 (+   328us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:26.748625 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:20:26.749066 (+   441us) log.cc:844] Serialized 26 byte log entry
0709 13:20:26.783788 (+ 34722us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:20:26.783831 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:26.783892 (+    61us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:20:27.556370 (+772478us) raft_consensus.cc:1685] finished
0709 13:20:27.556400 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:27.556568 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1037,"raft.run_cpu_time_us":114,"raft.run_wall_time_us":6179,"thread_start_us":34351,"threads_started":1,"wal-append.queue_time_us":106253}
I20260709 13:20:27.610244 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "829b4ad8c5a54eababeea1181b960d4b" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:20:27.558743 16817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57a744f5aaf944678a63469cbf514015" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:27.608132 16297 raft_consensus.cc:2468] T cdbd78acb4cf46298e0de8ba0d130e19 P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 8.
I20260709 13:20:27.611325 16150 raft_consensus.cc:3060] T 829b4ad8c5a54eababeea1181b960d4b P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:20:27.611958 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f5f89186204242bf377f7c42b2e2da" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
W20260709 13:20:27.612476 16265 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:27.612594 16297 raft_consensus.cc:3060] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:20:27.613878 16789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 58390.6us
I20260709 13:20:27.560675 19656 leader_election.cc:290] T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:20:27.562229 19657 leader_election.cc:290] T 2794af5b99984efebdee564035841cb0 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:27.564848 19660 raft_consensus.cc:515] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:27.500984 19197 raft_consensus.cc:493] T ecc1e363230043fe9fef390f0b979f20 P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:20:27.569885 16191 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:27.637715 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a959dbecc2648eb85d1cc550f5a3245" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
W20260709 13:20:27.487726 19771 log.cc:927] Time spent T c0eab25ce4c64e018a041beeda2867db P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 0.528s	user 0.001s	sys 0.000s
I20260709 13:20:27.572459 16822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:20:27.548472 16821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e4bfb9090c8460eb14fa7e0d818e7f1" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:20:27.641551 16822 raft_consensus.cc:3060] T 19baa7bc367844c288bca8f81439c413 P cb82cc7bd6db46d980c75237a1402860 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:20:27.524127 19720 raft_consensus.cc:2804] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:20:27.644275 19720 raft_consensus.cc:493] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:27.608132 16292 raft_consensus.cc:2468] T 26354361e9b044cea1c2b86c4882fe92 P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 6.
I20260709 13:20:27.481906 16115 leader_election.cc:304] T 8d4ab870a05347868956aa8b841b2981 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac
I20260709 13:20:27.486860 19707 raft_consensus.cc:515] T 08c74490c40243a88e62d1760b5cd814 P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:27.530215 19700 raft_consensus.cc:3060] T 1df93779209748b9860f6d893dd3de53 P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:20:27.526700 19666 raft_consensus.cc:2804] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:20:27.615031 16789 leader_election.cc:304] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:20:27.614463 16265 leader_election.cc:304] T fffab26427e24d398e2695aad40ca134 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
W20260709 13:20:27.743657 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 129272us
W20260709 13:20:27.744935 16789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 129985us
I20260709 13:20:27.745618 16789 leader_election.cc:304] T 3884f864a96d428785895e5463e5728f P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 
W20260709 13:20:27.711695 16192 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:20:27.712541 16295 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48674 (request call id 2401) took 3036 ms (client timeout 1505 ms). Trace:
W20260709 13:20:27.748410 16295 rpcz_store.cc:269] 0709 13:20:24.676124 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:24.719199 (+ 43075us) service_pool.cc:224] Handling call
0709 13:20:27.712257 (+2993058us) spinlock_profiling.cc:243] Waited 1.87 s on lock 0x7b540024c554. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:20:27.712455 (+   198us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4303790976}
I20260709 13:20:27.749635 16822 raft_consensus.cc:2468] T 19baa7bc367844c288bca8f81439c413 P cb82cc7bd6db46d980c75237a1402860 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 9.
I20260709 13:20:27.633687 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746aabf856ab401c9ef5e859371e7b67" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:20:27.638375 16218 raft_consensus.cc:2468] T 19baa7bc367844c288bca8f81439c413 P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 8.
I20260709 13:20:27.638512 16298 raft_consensus.cc:2393] T 5a959dbecc2648eb85d1cc550f5a3245 P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 6: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
I20260709 13:20:27.550400 19682 leader_election.cc:290] T ee9ed67006144797b1de6ea8c2013a68 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
W20260709 13:20:27.638350 16191 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:27.641602 16821 raft_consensus.cc:3060] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P cb82cc7bd6db46d980c75237a1402860 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:20:27.574931 16300 raft_consensus.cc:3060] T d16dfad0c3b540e5b4b86492ddf87208 P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:20:27.756908 16117 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:27.574699 19532 leader_election.cc:290] T e5ad4eb281564936a59f41e387a47156 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:27.755096 16265 leader_election.cc:304] T 23f82224f4564fea93572e8206bad1fe P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
I20260709 13:20:27.760648 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87147731a68400e8cce9b879daa7c5d" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:27.761102 16292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c74490c40243a88e62d1760b5cd814" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:27.761412 16298 raft_consensus.cc:2468] T b87147731a68400e8cce9b879daa7c5d P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 9.
I20260709 13:20:27.761704 16265 leader_election.cc:304] T 68a6838d82ef4cac83507f906139d90f P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:20:27.762392 16292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4dc6b0541ca431ebc757beac00521f1" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:27.763207 16292 raft_consensus.cc:2468] T e4dc6b0541ca431ebc757beac00521f1 P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 7.
I20260709 13:20:27.764037 16292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:27.764598 16292 raft_consensus.cc:2468] T ecc1e363230043fe9fef390f0b979f20 P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 8.
I20260709 13:20:27.643359 19197 raft_consensus.cc:515] T ecc1e363230043fe9fef390f0b979f20 P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:27.514758 19760 raft_consensus.cc:515] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } }
I20260709 13:20:27.749599 16295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434bff3082d141838068f323bdb0d1ed" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:27.669802 16147 raft_consensus.cc:2468] T 8cc959fb7f704fba9009c0595467d074 P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 9.
I20260709 13:20:27.632490 16297 raft_consensus.cc:2468] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 10.
I20260709 13:20:27.664740 16226 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57604 (request call id 2339) took 2163 ms. Trace:
I20260709 13:20:27.754426 16150 raft_consensus.cc:2468] T 829b4ad8c5a54eababeea1181b960d4b P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 9.
W20260709 13:20:27.770740 16115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 288875us
I20260709 13:20:27.767948 16226 rpcz_store.cc:276] 0709 13:20:25.501117 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:25.501247 (+   130us) service_pool.cc:224] Handling call
0709 13:20:25.501528 (+   281us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:20:26.530409 (+1028881us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:27.664473 (+1134064us) spinlock_profiling.cc:243] Waited 427 ms on lock 0x7b54002977d4. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:20:27.664674 (+   201us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":981093248}
I20260709 13:20:27.774127 16821 raft_consensus.cc:2468] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P cb82cc7bd6db46d980c75237a1402860 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 9.
I20260709 13:20:27.760736 19407 raft_consensus.cc:493] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:27.662775 19774 raft_consensus.cc:493] T 66ae176961b74813a58bc15c9c4c6f2a P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:27.775339 19407 raft_consensus.cc:515] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:27.775547 19774 raft_consensus.cc:515] T 66ae176961b74813a58bc15c9c4c6f2a P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:27.775928 16298 raft_consensus.cc:1240] T c4f65cdf00b748bb963ee6e81ef43b79 P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 9. Current term is 10. Ops: []
W20260709 13:20:27.762650 19719 consensus_meta.cc:287] T 08c74490c40243a88e62d1760b5cd814 P 1ca2725d290e47d68b1a4b609c9816ac: Time spent flushing consensus metadata: real 0.987s	user 0.006s	sys 0.010s
I20260709 13:20:27.777572 19719 raft_consensus.cc:515] T 08c74490c40243a88e62d1760b5cd814 P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:27.633857 19414 raft_consensus.cc:493] T e5ad4eb281564936a59f41e387a47156 P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:27.637609 19450 raft_consensus.cc:493] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:20:27.703022 19345 consensus_meta.cc:287] T 8c11074579b046d99e585c2818e18a2c P 90f3da57796b488c9087a2f523adf045: Time spent flushing consensus metadata: real 0.783s	user 0.011s	sys 0.000s
I20260709 13:20:27.622296 19785 raft_consensus.cc:2804] T 23f82224f4564fea93572e8206bad1fe P cb82cc7bd6db46d980c75237a1402860 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:20:27.820685 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26354361e9b044cea1c2b86c4882fe92" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:27.755941 19720 raft_consensus.cc:3060] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:20:27.703246 19656 raft_consensus.cc:493] T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:27.847987 16822 raft_consensus.cc:1275] T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860 [term 8 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:20:27.823103 16821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62fe3346cd3c42c4a55e14af9073bca4" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:20:27.844923 16149 raft_consensus.cc:2468] T 26354361e9b044cea1c2b86c4882fe92 P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 6.
I20260709 13:20:27.645689 19779 raft_consensus.cc:493] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:27.808346 16300 raft_consensus.cc:2468] T d16dfad0c3b540e5b4b86492ddf87208 P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 8.
I20260709 13:20:27.846390 16824 raft_consensus.cc:1240] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 [term 8 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 7. Current term is 8. Ops: []
I20260709 13:20:27.787964 16115 leader_election.cc:304] T c0eab25ce4c64e018a041beeda2867db P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:20:27.755460 19674 raft_consensus.cc:493] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:27.846973 16825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdbd78acb4cf46298e0de8ba0d130e19" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:27.780112 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87147731a68400e8cce9b879daa7c5d" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:27.822454 16148 raft_consensus.cc:1240] T 08c74490c40243a88e62d1760b5cd814 P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 7. Current term is 8. Ops: []
I20260709 13:20:27.755677 19666 raft_consensus.cc:493] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:27.848656 16823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746aabf856ab401c9ef5e859371e7b67" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:20:27.848389 16817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "829b4ad8c5a54eababeea1181b960d4b" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:20:27.853289 16823 raft_consensus.cc:2393] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 9: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
I20260709 13:20:27.849722 16821 raft_consensus.cc:3060] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:20:27.767458 19766 raft_consensus.cc:2804] T 3884f864a96d428785895e5463e5728f P cb82cc7bd6db46d980c75237a1402860 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:20:27.880098 16821 raft_consensus.cc:2468] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 11.
I20260709 13:20:27.769958 19657 raft_consensus.cc:2749] T 23f82224f4564fea93572e8206bad1fe P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:20:27.781035 19664 raft_consensus.cc:2804] T 68a6838d82ef4cac83507f906139d90f P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:20:27.822000 19529 raft_consensus.cc:2804] T 8d4ab870a05347868956aa8b841b2981 P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:20:27.767588 19783 raft_consensus.cc:2804] T fffab26427e24d398e2695aad40ca134 P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:20:27.874545 16292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a378d7f796f94615b3b347119a9880fd" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:27.755337 19751 raft_consensus.cc:493] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:27.760102 19732 raft_consensus.cc:2804] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:20:27.766678 19699 raft_consensus.cc:493] T 18ad7e3af5504571a69d10c00750a01c P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:27.771036 19540 consensus_queue.cc:1048] T af57996a8be84d398f0f023e8d08ef77 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.218s
I20260709 13:20:27.863435 19674 raft_consensus.cc:515] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:27.846704 19345 raft_consensus.cc:515] T 8c11074579b046d99e585c2818e18a2c P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:27.644426 19660 leader_election.cc:290] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:27.905215 16145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c61a39fd4f84424b129c1a6f9ba404d" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:27.911518 16151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ad5dbfa99ee4b8d9e28be92b3f36db5" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
W20260709 13:20:27.906389 16220 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57600 (request call id 1284) took 2306 ms (client timeout 1604 ms). Trace:
I20260709 13:20:28.181828 16148 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40830 (request call id 1315) took 1409 ms. Trace:
I20260709 13:20:28.175028 16816 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49906 (request call id 749) took 2254 ms. Trace:
I20260709 13:20:28.210481 16151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2794af5b99984efebdee564035841cb0" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
W20260709 13:20:27.965168 16115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 177326us
I20260709 13:20:28.003232 19724 raft_consensus.cc:493] T a378d7f796f94615b3b347119a9880fd P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:28.188951 16218 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57600 (request call id 1286) took 2587 ms. Trace:
I20260709 13:20:28.211535 16151 raft_consensus.cc:2468] T 2794af5b99984efebdee564035841cb0 P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 6.
I20260709 13:20:28.212137 19724 raft_consensus.cc:515] T a378d7f796f94615b3b347119a9880fd P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:28.212888 16151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
W20260709 13:20:28.209424 16220 rpcz_store.cc:269] 0709 13:20:25.600495 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:25.677301 (+ 76806us) service_pool.cc:224] Handling call
0709 13:20:27.906151 (+2228850us) spinlock_profiling.cc:243] Waited 82.3 ms on lock 0x7b5400245c54. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:20:27.906310 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":189334272}
I20260709 13:20:28.213843 16151 raft_consensus.cc:2393] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 in current term 9: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
I20260709 13:20:28.214152 16220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68a6838d82ef4cac83507f906139d90f" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:20:28.214638 19724 leader_election.cc:290] T a378d7f796f94615b3b347119a9880fd P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:28.214258 19720 raft_consensus.cc:515] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:27.854988 19666 raft_consensus.cc:3060] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:20:28.210067 16816 rpcz_store.cc:276] 0709 13:20:25.920740 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:26.735225 (+814485us) service_pool.cc:224] Handling call
0709 13:20:26.735529 (+   304us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:26.763492 (+ 27963us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:28.174652 (+1411160us) spinlock_profiling.cc:243] Waited 3.79 ms on lock 0x7b540054bad4. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:20:28.174934 (+   282us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":8720128}
I20260709 13:20:28.215008 16151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2372981006d4b218215c28964dea623" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:28.237248 16265 leader_election.cc:304] T e4dc6b0541ca431ebc757beac00521f1 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 647a343a542d4a139fe106c19095129b
I20260709 13:20:28.239050 16151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d16dfad0c3b540e5b4b86492ddf87208" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:28.239387 16265 leader_election.cc:304] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: cb82cc7bd6db46d980c75237a1402860
I20260709 13:20:28.209448 16148 rpcz_store.cc:276] 0709 13:20:26.772355 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:26.772458 (+   103us) service_pool.cc:224] Handling call
0709 13:20:26.775869 (+  3411us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:27.852772 (+1076903us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:28.181467 (+328695us) spinlock_profiling.cc:243] Waited 1.05 s on lock 0x7b540006e0d4. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:20:28.181736 (+   269us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2406307072}
I20260709 13:20:28.211680 16115 leader_election.cc:304] T 26354361e9b044cea1c2b86c4882fe92 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: cb82cc7bd6db46d980c75237a1402860
I20260709 13:20:28.212209 16218 rpcz_store.cc:276] 0709 13:20:25.601168 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:27.403965 (+1802797us) service_pool.cc:224] Handling call
0709 13:20:28.188706 (+784741us) spinlock_profiling.cc:243] Waited 217 ms on lock 0x7b5400246b54. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:20:28.188869 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":500036864}
I20260709 13:20:28.219650 16292 raft_consensus.cc:2468] T a378d7f796f94615b3b347119a9880fd P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 8.
I20260709 13:20:28.241257 16266 leader_election.cc:304] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:20:28.241580 16816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 9 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:28.217120 16144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0eab25ce4c64e018a041beeda2867db" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
W20260709 13:20:28.187045 19788 log.cc:927] Time spent T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 0.639s	user 0.002s	sys 0.000s
W20260709 13:20:28.243774 16789 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:28.244176 16115 leader_election.cc:304] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac
W20260709 13:20:28.035678 19795 log.cc:927] Time spent T 8158bbd17b994d2eb96f560bd0f1b88d P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.066s	user 0.000s	sys 0.000s
I20260709 13:20:28.075225 16223 raft_consensus.cc:2410] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b for term 9 because replica has last-logged OpId of term: 7 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 8.
I20260709 13:20:28.225116 16824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee9ed67006144797b1de6ea8c2013a68" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
W20260709 13:20:27.855331 19738 consensus_meta.cc:287] T 829b4ad8c5a54eababeea1181b960d4b P cb82cc7bd6db46d980c75237a1402860: Time spent flushing consensus metadata: real 1.087s	user 0.016s	sys 0.004s
I20260709 13:20:27.844662 19785 raft_consensus.cc:493] T 23f82224f4564fea93572e8206bad1fe P cb82cc7bd6db46d980c75237a1402860 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:20:28.228439 16818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57a744f5aaf944678a63469cbf514015" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:28.231595 16823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:27.848623 19414 raft_consensus.cc:515] T e5ad4eb281564936a59f41e387a47156 P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:28.231966 16820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:27.868535 19569 raft_consensus.cc:493] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:28.270387 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57a744f5aaf944678a63469cbf514015" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:28.010066 19529 raft_consensus.cc:697] T 8d4ab870a05347868956aa8b841b2981 P 647a343a542d4a139fe106c19095129b [term 9 LEADER]: Becoming Leader. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
W20260709 13:20:28.177742 19757 log.cc:927] Time spent T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 1.517s	user 0.014s	sys 0.001s
I20260709 13:20:27.856545 19656 raft_consensus.cc:515] T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:20:28.221503 16191 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:20:28.140111 19793 log.cc:927] Time spent T 23f82224f4564fea93572e8206bad1fe P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 0.065s	user 0.001s	sys 0.000s
I20260709 13:20:27.846827 19450 raft_consensus.cc:515] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:28.091775 19645 raft_consensus.cc:493] T 5a96373bb0e14dce8a266b2fc6c58507 P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:20:28.228014 19522 leader_election.cc:290] T 434bff3082d141838068f323bdb0d1ed P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:28.247326 16145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f82224f4564fea93572e8206bad1fe" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 8 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:28.247568 16220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18ad7e3af5504571a69d10c00750a01c" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:20:28.247977 19794 raft_consensus.cc:2804] T e4dc6b0541ca431ebc757beac00521f1 P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:20:28.248214 16218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdbd78acb4cf46298e0de8ba0d130e19" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:28.350068 19794 raft_consensus.cc:493] T e4dc6b0541ca431ebc757beac00521f1 P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:28.350263 16218 raft_consensus.cc:2468] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 8.
I20260709 13:20:28.351311 16218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada5a004ba7c46819051e3ccec71ffdb" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:20:27.868485 19789 raft_consensus.cc:493] T 8a878999c0024d2abe517484180290fc P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:28.250159 16294 raft_consensus.cc:2368] T 8c11074579b046d99e585c2818e18a2c P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac for earlier term 8. Current term is 9.
I20260709 13:20:28.352483 16218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4f65cdf00b748bb963ee6e81ef43b79" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:28.352743 19414 leader_election.cc:290] T e5ad4eb281564936a59f41e387a47156 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
W20260709 13:20:28.352974 16224 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57604 (request call id 2336) took 2854 ms (client timeout 2517 ms). Trace:
I20260709 13:20:28.353097 16218 raft_consensus.cc:2468] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 10.
I20260709 13:20:28.352483 19789 raft_consensus.cc:515] T 8a878999c0024d2abe517484180290fc P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:28.354092 16218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02838c86531d4f98bf58d1c22b8baf77" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:28.354671 16218 raft_consensus.cc:2468] T 02838c86531d4f98bf58d1c22b8baf77 P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 8.
I20260709 13:20:28.354733 19720 leader_election.cc:290] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 11 election: Requested vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:28.228612 19699 raft_consensus.cc:515] T 18ad7e3af5504571a69d10c00750a01c P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } }
I20260709 13:20:28.355752 19789 leader_election.cc:290] T 8a878999c0024d2abe517484180290fc P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:27.986858 19733 raft_consensus.cc:2804] T c0eab25ce4c64e018a041beeda2867db P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:20:27.909904 19683 leader_election.cc:290] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:28.210292 19783 raft_consensus.cc:493] T fffab26427e24d398e2695aad40ca134 P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:27.987079 19766 raft_consensus.cc:697] T 3884f864a96d428785895e5463e5728f P cb82cc7bd6db46d980c75237a1402860 [term 7 LEADER]: Becoming Leader. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:20:28.183655 19568 raft_consensus.cc:3055] T 9e65676b21b748728ad39668cfe722ec P cb82cc7bd6db46d980c75237a1402860 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:20:28.039156 19700 raft_consensus.cc:515] T 1df93779209748b9860f6d893dd3de53 P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:27.905712 19774 leader_election.cc:290] T 66ae176961b74813a58bc15c9c4c6f2a P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:28.215229 19760 leader_election.cc:290] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:28.255927 19660 raft_consensus.cc:2804] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:20:28.256031 19575 raft_consensus.cc:2804] T 26354361e9b044cea1c2b86c4882fe92 P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:20:28.378793 19575 raft_consensus.cc:493] T 26354361e9b044cea1c2b86c4882fe92 P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Starting leader election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:20:28.379344 19575 raft_consensus.cc:3060] T 26354361e9b044cea1c2b86c4882fe92 P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:20:28.381855 19672 leader_election.cc:290] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:28.382575 19672 raft_consensus.cc:493] T 806d7690c2b644959152074bb2cab0a7 P cb82cc7bd6db46d980c75237a1402860 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:20:28.039889 19702 leader_election.cc:290] T 19baa7bc367844c288bca8f81439c413 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:28.266108 16820 raft_consensus.cc:2393] T 19baa7bc367844c288bca8f81439c413 P cb82cc7bd6db46d980c75237a1402860 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 9: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
I20260709 13:20:28.211090 19732 raft_consensus.cc:493] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:20:28.383985 19732 raft_consensus.cc:3060] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:20:28.384922 19765 log.cc:927] Time spent T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 1.528s	user 0.009s	sys 0.011s
I20260709 13:20:28.035856 19197 leader_election.cc:290] T ecc1e363230043fe9fef390f0b979f20 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:28.387706 16296 raft_consensus.cc:1240] T 84f2da98864b4425bf7c0055951fdd4b P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 6. Current term is 7. Ops: []
I20260709 13:20:28.269248 16823 raft_consensus.cc:1275] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860 [term 9 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:20:28.388592 16223 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57604 (request call id 2349) took 1610 ms. Trace:
I20260709 13:20:28.389323 16296 raft_consensus.cc:1240] T d16dfad0c3b540e5b4b86492ddf87208 P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 7. Current term is 8. Ops: []
W20260709 13:20:28.270766 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 48731.5us
I20260709 13:20:28.117281 19707 leader_election.cc:290] T 08c74490c40243a88e62d1760b5cd814 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 8 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:28.271384 16150 raft_consensus.cc:2410] T 57a744f5aaf944678a63469cbf514015 P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 for term 8 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 13:20:28.390897 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee9ed67006144797b1de6ea8c2013a68" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:28.391567 16150 raft_consensus.cc:2468] T ee9ed67006144797b1de6ea8c2013a68 P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 6.
I20260709 13:20:28.392571 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e65676b21b748728ad39668cfe722ec" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:28.393093 16150 raft_consensus.cc:2468] T 9e65676b21b748728ad39668cfe722ec P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 9.
I20260709 13:20:28.393797 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e9b2bf7011f4ae688a79d7bb5a447b4" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:28.394286 16150 raft_consensus.cc:2468] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 6.
I20260709 13:20:28.256906 19724 raft_consensus.cc:2804] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:20:28.321436 19529 consensus_queue.cc:237] T 8d4ab870a05347868956aa8b841b2981 P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 8.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:28.264510 19754 raft_consensus.cc:2804] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:20:28.256166 19710 raft_consensus.cc:493] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:28.269313 19785 raft_consensus.cc:3060] T 23f82224f4564fea93572e8206bad1fe P cb82cc7bd6db46d980c75237a1402860 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:20:28.237087 19664 raft_consensus.cc:697] T 68a6838d82ef4cac83507f906139d90f P 90f3da57796b488c9087a2f523adf045 [term 9 LEADER]: Becoming Leader. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:20:28.269140 19738 raft_consensus.cc:515] T 829b4ad8c5a54eababeea1181b960d4b P cb82cc7bd6db46d980c75237a1402860 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:20:28.257175 19667 consensus_meta.cc:287] T 8a878999c0024d2abe517484180290fc P 1ca2725d290e47d68b1a4b609c9816ac: Time spent flushing consensus metadata: real 0.781s	user 0.011s	sys 0.000s
I20260709 13:20:28.326478 19656 leader_election.cc:290] T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:20:28.329171 16299 raft_consensus.cc:1240] T 8cc959fb7f704fba9009c0595467d074 P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 8. Current term is 9. Ops: [8.6-8.6]
I20260709 13:20:28.398833 16296 raft_consensus.cc:1240] T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 7. Current term is 8. Ops: []
I20260709 13:20:28.358103 19794 raft_consensus.cc:3060] T e4dc6b0541ca431ebc757beac00521f1 P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:20:28.333580 16117 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:28.337284 19763 raft_consensus.cc:3055] T 3884f864a96d428785895e5463e5728f P 647a343a542d4a139fe106c19095129b [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:20:27.864233 19779 raft_consensus.cc:515] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:28.344521 19450 leader_election.cc:290] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:28.345647 16143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31969392ccdb43b491911a319e8bc7e1" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:28.346858 16219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c61a39fd4f84424b129c1a6f9ba404d" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:28.253074 16115 leader_election.cc:304] T 434bff3082d141838068f323bdb0d1ed P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860
W20260709 13:20:28.353350 16224 rpcz_store.cc:269] 0709 13:20:25.498075 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:25.498176 (+   101us) service_pool.cc:224] Handling call
0709 13:20:28.352793 (+2854617us) spinlock_profiling.cc:243] Waited 1.21 s on lock 0x7b5400139554. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:20:28.352916 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2775659904}
I20260709 13:20:28.429754 16219 raft_consensus.cc:2393] T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 8: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
I20260709 13:20:28.253958 16824 raft_consensus.cc:2468] T ee9ed67006144797b1de6ea8c2013a68 P cb82cc7bd6db46d980c75237a1402860 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 6.
I20260709 13:20:28.357739 16291 raft_consensus.cc:1240] T 3884f864a96d428785895e5463e5728f P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 6. Current term is 7. Ops: []
I20260709 13:20:28.348966 16145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3884f864a96d428785895e5463e5728f" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:20:28.360199 16191 leader_election.cc:304] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
W20260709 13:20:28.433737 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 73628.5us
I20260709 13:20:28.232358 19751 raft_consensus.cc:515] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:28.364439 16142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5718c623ebee4d2bb770f0f9ddb2f164" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
W20260709 13:20:28.369186 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 129854us
I20260709 13:20:28.369637 16789 leader_election.cc:304] T b87147731a68400e8cce9b879daa7c5d P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 647a343a542d4a139fe106c19095129b
I20260709 13:20:28.371506 16220 raft_consensus.cc:2393] T 18ad7e3af5504571a69d10c00750a01c P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 8: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
I20260709 13:20:28.449445 16142 raft_consensus.cc:2468] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 8.
W20260709 13:20:28.449815 16790 connection.cc:256] RPC call timeout handler was delayed by -1.6045858229999794s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17410000000000003s for a response
I20260709 13:20:28.227784 19651 raft_consensus.cc:493] T 8638ee363273426db1393dd81e5c7b72 P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:20:28.176748 19671 consensus_meta.cc:287] T b6f07842d3aa47858df5665874c86f3a P 1ca2725d290e47d68b1a4b609c9816ac: Time spent flushing consensus metadata: real 0.613s	user 0.005s	sys 0.005s
I20260709 13:20:28.405120 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fffab26427e24d398e2695aad40ca134" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:28.458343 16226 raft_consensus.cc:2468] T fffab26427e24d398e2695aad40ca134 P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 7.
I20260709 13:20:28.389752 19660 raft_consensus.cc:493] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:28.459213 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e4bfb9090c8460eb14fa7e0d818e7f1" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:28.459856 16226 raft_consensus.cc:2468] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 8.
I20260709 13:20:28.460604 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ad5dbfa99ee4b8d9e28be92b3f36db5" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:28.461110 16226 raft_consensus.cc:2468] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 7.
I20260709 13:20:28.461799 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d16dfad0c3b540e5b4b86492ddf87208" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:28.389052 16223 rpcz_store.cc:276] 0709 13:20:26.778340 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:27.335991 (+557651us) service_pool.cc:224] Handling call
0709 13:20:28.388446 (+1052455us) spinlock_profiling.cc:243] Waited 736 ms on lock 0x7b54002c9054. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:20:28.388544 (+    98us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1693005696}
I20260709 13:20:28.414028 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:20:28.465737 16150 raft_consensus.cc:2393] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 9: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
W20260709 13:20:28.314814 16192 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 96627.9us
I20260709 13:20:28.420463 16299 raft_consensus.cc:1275] T 7c2095a5f4ad49408dbc7f69c606120d P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 4. (term mismatch)
I20260709 13:20:28.466288 16299 pending_rounds.cc:85] T 7c2095a5f4ad49408dbc7f69c606120d P 90f3da57796b488c9087a2f523adf045: Aborting all ops after (but not including) 3
I20260709 13:20:28.466203 16219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e4bfb9090c8460eb14fa7e0d818e7f1" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:20:28.467155 16219 raft_consensus.cc:3060] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:20:28.467358 16192 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:28.429734 16821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:20:28.430208 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c11074579b046d99e585c2818e18a2c" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:28.469291 16821 raft_consensus.cc:2393] T 19baa7bc367844c288bca8f81439c413 P cb82cc7bd6db46d980c75237a1402860 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 9: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
I20260709 13:20:28.469202 16224 raft_consensus.cc:2468] T 8c11074579b046d99e585c2818e18a2c P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 7.
I20260709 13:20:28.470182 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 9 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:28.419885 19667 raft_consensus.cc:2749] T 8a878999c0024d2abe517484180290fc P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:20:28.366804 19783 raft_consensus.cc:3060] T fffab26427e24d398e2695aad40ca134 P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:20:28.370833 19345 leader_election.cc:290] T 8c11074579b046d99e585c2818e18a2c P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 9 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:28.301928 19569 raft_consensus.cc:515] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:28.437431 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62fe3346cd3c42c4a55e14af9073bca4" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:28.437656 19766 consensus_queue.cc:237] T 3884f864a96d428785895e5463e5728f P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:28.470461 19671 raft_consensus.cc:515] T b6f07842d3aa47858df5665874c86f3a P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:28.439604 19664 consensus_queue.cc:237] T 68a6838d82ef4cac83507f906139d90f P 90f3da57796b488c9087a2f523adf045 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:28.441756 16225 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57604 (request call id 2338) took 2943 ms. Trace:
I20260709 13:20:28.483743 19671 leader_election.cc:290] T b6f07842d3aa47858df5665874c86f3a P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 10 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:28.464439 19651 raft_consensus.cc:515] T 8638ee363273426db1393dd81e5c7b72 P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:28.486450 19651 leader_election.cc:290] T 8638ee363273426db1393dd81e5c7b72 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
W20260709 13:20:28.487596 16147 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40844 (request call id 2174) took 1527 ms (client timeout 1801 ms). Trace:
W20260709 13:20:28.488024 16147 rpcz_store.cc:269] 0709 13:20:26.959962 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:27.403778 (+443816us) service_pool.cc:224] Handling call
0709 13:20:28.487429 (+1083651us) spinlock_profiling.cc:243] Waited 198 ms on lock 0x7b540028da54. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:20:28.487537 (+   108us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":454267136}
I20260709 13:20:28.488404 19779 leader_election.cc:290] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:28.448833 19683 raft_consensus.cc:2749] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:20:28.489271 16147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6f07842d3aa47858df5665874c86f3a" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 10 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:28.483470 16225 rpcz_store.cc:276] 0709 13:20:25.498587 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:25.499023 (+   436us) service_pool.cc:224] Handling call
0709 13:20:25.528486 (+ 29463us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:20:26.037063 (+508577us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:26.037107 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:20:26.578654 (+541547us) log.cc:844] Serialized 26 byte log entry
0709 13:20:26.765050 (+186396us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:20:26.765087 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:26.765156 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:20:28.372090 (+1606934us) spinlock_profiling.cc:243] Waited 2.82 ms on lock 0x7b4400022b4c. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69ad8fadb1 00007f69ad998c85 00007f69ad9996a2 00007f69b070b1db 00007f69b0706aaf 00007f69b06fa86e 00007f69b06f9bb6 00007f69b06f97d8 00007f69b0703221 00007f69b07031d9 00007f69b0703169
0709 13:20:28.441409 (+ 69319us) raft_consensus.cc:1685] finished
0709 13:20:28.441435 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:28.441688 (+   253us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":48361,"raft.queue_time_us":61972,"raft.run_cpu_time_us":155,"raft.run_wall_time_us":28652,"spinlock_wait_cycles":6478080,"thread_start_us":624567,"threads_started":2,"wal-append.queue_time_us":226882}
I20260709 13:20:28.490693 16147 raft_consensus.cc:1240] T 68a6838d82ef4cac83507f906139d90f P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 8. Current term is 9. Ops: []
I20260709 13:20:28.491107 16225 raft_consensus.cc:1240] T fffab26427e24d398e2695aad40ca134 P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 6. Current term is 7. Ops: []
I20260709 13:20:28.491729 16225 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57600 (request call id 1285) took 2891 ms. Trace:
I20260709 13:20:28.492074 16225 rpcz_store.cc:276] 0709 13:20:25.600857 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:28.490685 (+2889828us) service_pool.cc:224] Handling call
0709 13:20:28.490965 (+   280us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:28.491492 (+   527us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:28.491670 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:20:28.493327 16225 raft_consensus.cc:1240] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 8. Current term is 9. Ops: [6.8-8.9]
I20260709 13:20:28.493883 16225 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57600 (request call id 1288) took 2892 ms. Trace:
I20260709 13:20:28.494170 16225 rpcz_store.cc:276] 0709 13:20:25.601791 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:28.492762 (+2890971us) service_pool.cc:224] Handling call
0709 13:20:28.493093 (+   331us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:20:28.493638 (+   545us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:28.493832 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:20:28.495505 16223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:20:28.496102 16223 raft_consensus.cc:3060] T 19baa7bc367844c288bca8f81439c413 P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:20:28.496587 19589 leader_election.cc:290] T 434bff3082d141838068f323bdb0d1ed P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:28.501530 16217 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33050 (request call id 602) took 2711 ms. Trace:
I20260709 13:20:28.501916 16217 rpcz_store.cc:276] 0709 13:20:25.790096 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:28.500625 (+2710529us) service_pool.cc:224] Handling call
0709 13:20:28.500914 (+   289us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:28.501190 (+   276us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:28.501230 (+    40us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:20:28.501287 (+    57us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:20:28.501319 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:28.501373 (+    54us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:28.501478 (+   105us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:20:28.502767 19569 leader_election.cc:290] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:28.503875 19699 leader_election.cc:290] T 18ad7e3af5504571a69d10c00750a01c P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:28.507913 19407 leader_election.cc:290] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:28.508972 19751 leader_election.cc:290] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:28.494952 19725 raft_consensus.cc:3060] T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:20:28.511317 16217 raft_consensus.cc:1240] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 10. Current term is 11. Ops: []
W20260709 13:20:28.511772 19797 log.cc:927] Time spent T 5a96373bb0e14dce8a266b2fc6c58507 P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.055s	user 0.001s	sys 0.000s
I20260709 13:20:28.512020 16217 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33050 (request call id 604) took 2721 ms. Trace:
I20260709 13:20:28.512367 16217 rpcz_store.cc:276] 0709 13:20:25.790715 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:28.502766 (+2712051us) service_pool.cc:224] Handling call
0709 13:20:28.503038 (+   272us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:28.511751 (+  8713us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:28.511961 (+   210us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:20:28.513314 16293 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48674 (request call id 2415) took 1102 ms. Trace:
I20260709 13:20:28.513697 16293 rpcz_store.cc:276] 0709 13:20:27.410920 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:27.411038 (+   118us) service_pool.cc:224] Handling call
0709 13:20:27.411369 (+   331us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:20:27.759114 (+347745us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:27.759151 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:20:27.770456 (+ 11305us) log.cc:844] Serialized 26 byte log entry
0709 13:20:28.085893 (+315437us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:20:28.085950 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:28.086062 (+   112us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:20:28.513070 (+427008us) raft_consensus.cc:1685] finished
0709 13:20:28.513096 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:28.513247 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":23872,"raft.run_cpu_time_us":83,"raft.run_wall_time_us":83,"thread_start_us":564904,"threads_started":2,"wal-append.queue_time_us":519441}
I20260709 13:20:28.514752 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434bff3082d141838068f323bdb0d1ed" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 10 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:28.448359 16294 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48684 (request call id 2296) took 1024 ms. Trace:
I20260709 13:20:28.515341 16294 rpcz_store.cc:276] 0709 13:20:27.424422 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:27.548172 (+123750us) service_pool.cc:224] Handling call
0709 13:20:28.448157 (+899985us) spinlock_profiling.cc:243] Waited 701 ms on lock 0x7b540032f854. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:20:28.448298 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1611667328}
I20260709 13:20:28.357321 19645 raft_consensus.cc:515] T 5a96373bb0e14dce8a266b2fc6c58507 P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:20:28.449937 16789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 80384.3us
I20260709 13:20:28.431532 19754 raft_consensus.cc:697] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b [term 10 LEADER]: Becoming Leader. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
W20260709 13:20:28.452601 16265 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:28.538245 16265 leader_election.cc:304] T 26354361e9b044cea1c2b86c4882fe92 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: cb82cc7bd6db46d980c75237a1402860
W20260709 13:20:28.454468 16115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 201448us
I20260709 13:20:28.541965 16294 raft_consensus.cc:1275] T ecc1e363230043fe9fef390f0b979f20 P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:20:28.542114 16293 raft_consensus.cc:1275] T 57a744f5aaf944678a63469cbf514015 P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:20:28.369697 19568 raft_consensus.cc:740] T 9e65676b21b748728ad39668cfe722ec P cb82cc7bd6db46d980c75237a1402860 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:20:28.543341 19779 raft_consensus.cc:2804] T 26354361e9b044cea1c2b86c4882fe92 P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:20:28.534957 19569 consensus_queue.cc:1059] T 5a96373bb0e14dce8a266b2fc6c58507 P 647a343a542d4a139fe106c19095129b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: INVALID_TERM, Last received: 5.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.009s
I20260709 13:20:28.543912 19779 raft_consensus.cc:493] T 26354361e9b044cea1c2b86c4882fe92 P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Starting leader election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:20:28.544329 19779 raft_consensus.cc:3060] T 26354361e9b044cea1c2b86c4882fe92 P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:20:28.456723 16817 raft_consensus.cc:2393] T 829b4ad8c5a54eababeea1181b960d4b P cb82cc7bd6db46d980c75237a1402860 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 9: Already voted for candidate cb82cc7bd6db46d980c75237a1402860 in this term.
I20260709 13:20:28.465188 16226 raft_consensus.cc:2393] T d16dfad0c3b540e5b4b86492ddf87208 P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 8: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
I20260709 13:20:28.466765 16299 pending_rounds.cc:107] T 7c2095a5f4ad49408dbc7f69c606120d P 90f3da57796b488c9087a2f523adf045: Aborting uncommitted NO_OP operation due to leader change: 6.4
I20260709 13:20:28.361088 19733 raft_consensus.cc:493] T c0eab25ce4c64e018a041beeda2867db P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Starting leader election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:20:28.470995 16224 raft_consensus.cc:2410] T ecc1e363230043fe9fef390f0b979f20 P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 for term 9 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 3.
I20260709 13:20:28.355629 16218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0eab25ce4c64e018a041beeda2867db" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:28.471294 16145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5ad4eb281564936a59f41e387a47156" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:28.554231 16145 raft_consensus.cc:2468] T e5ad4eb281564936a59f41e387a47156 P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 9.
I20260709 13:20:28.468801 19660 raft_consensus.cc:3060] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:20:28.472517 16191 leader_election.cc:336] T e5ad4eb281564936a59f41e387a47156 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Timed out: RequestConsensusVote RPC to 127.14.177.129:36741 timed out after 2.232s (SENT)
I20260709 13:20:28.555472 16191 leader_election.cc:304] T e5ad4eb281564936a59f41e387a47156 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
W20260709 13:20:28.556337 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 83849.6us
W20260709 13:20:28.556916 16191 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:20:28.557212 16191 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:28.457633 19414 raft_consensus.cc:493] T c9944d5e4b8c4109af9955c4f08c9095 P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:28.428624 19763 raft_consensus.cc:740] T 3884f864a96d428785895e5463e5728f P 647a343a542d4a139fe106c19095129b [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:20:28.517592 16225 raft_consensus.cc:1240] T 08c74490c40243a88e62d1760b5cd814 P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 7. Current term is 8. Ops: []
I20260709 13:20:28.544065 19568 consensus_queue.cc:260] T 9e65676b21b748728ad39668cfe722ec P cb82cc7bd6db46d980c75237a1402860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:28.521401 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b7d1ab519044eda2d5e04a07242f25" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:20:28.444069 16296 raft_consensus.cc:1275] T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:20:28.524374 16116 leader_election.cc:304] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860
I20260709 13:20:28.529162 16223 raft_consensus.cc:2468] T 19baa7bc367844c288bca8f81439c413 P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 9.
I20260709 13:20:28.442076 19724 raft_consensus.cc:697] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045 [term 9 LEADER]: Becoming Leader. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:20:28.443675 19715 raft_consensus.cc:2749] T 7c2095a5f4ad49408dbc7f69c606120d P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:20:28.547099 16299 raft_consensus.cc:2889] T 7c2095a5f4ad49408dbc7f69c606120d P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:20:28.464308 19789 raft_consensus.cc:2749] T 434bff3082d141838068f323bdb0d1ed P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:20:28.540385 16265 leader_election.cc:304] T 2794af5b99984efebdee564035841cb0 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
W20260709 13:20:28.594972 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 54669.4us
I20260709 13:20:28.596838 16265 leader_election.cc:304] T 57a744f5aaf944678a63469cbf514015 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
I20260709 13:20:28.463994 19702 raft_consensus.cc:493] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:20:28.464818 19806 raft_consensus.cc:2804] T b87147731a68400e8cce9b879daa7c5d P cb82cc7bd6db46d980c75237a1402860 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:20:28.558609 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68a6838d82ef4cac83507f906139d90f" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:28.558727 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0eab25ce4c64e018a041beeda2867db" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:28.559104 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68a6838d82ef4cac83507f906139d90f" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:28.610458 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18ad7e3af5504571a69d10c00750a01c" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:20:28.398298 19672 raft_consensus.cc:515] T 806d7690c2b644959152074bb2cab0a7 P cb82cc7bd6db46d980c75237a1402860 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:28.611009 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5718c623ebee4d2bb770f0f9ddb2f164" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
W20260709 13:20:28.450304 16790 connection.cc:256] RPC call timeout handler was delayed by -1.3318858229999933s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20440000000000003s for a response
I20260709 13:20:28.599859 19651 raft_consensus.cc:2749] T 57a744f5aaf944678a63469cbf514015 P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:20:28.611866 16790 connection.cc:256] RPC call timeout handler was delayed by -1.3021858229999452s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2077s for a response
W20260709 13:20:28.612222 16790 connection.cc:256] RPC call timeout handler was delayed by -1.243685823000078s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2142s for a response
I20260709 13:20:28.608278 19806 raft_consensus.cc:493] T b87147731a68400e8cce9b879daa7c5d P cb82cc7bd6db46d980c75237a1402860 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 647a343a542d4a139fe106c19095129b)
W20260709 13:20:28.612543 16790 connection.cc:256] RPC call timeout handler was delayed by -1.1824858229999791s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.221s for a response
W20260709 13:20:28.612869 16790 connection.cc:256] RPC call timeout handler was delayed by -1.0007830329998342s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1765s for a response
W20260709 13:20:28.613234 16790 connection.cc:256] RPC call timeout handler was delayed by -1.0007830329998342s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1765s for a response
I20260709 13:20:28.517680 16147 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40844 (request call id 2166) took 1560 ms. Trace:
I20260709 13:20:28.613803 16790 leader_election.cc:304] T 19baa7bc367844c288bca8f81439c413 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:20:28.576251 19738 leader_election.cc:290] T 829b4ad8c5a54eababeea1181b960d4b P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 9 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:20:28.572223 16218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8158bbd17b994d2eb96f560bd0f1b88d" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:28.621920 16218 raft_consensus.cc:2410] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 for term 9 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 5.
I20260709 13:20:28.581656 16225 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57600 (request call id 1293) took 2952 ms. Trace:
I20260709 13:20:28.582517 16296 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48674 (request call id 2429) took 1042 ms. Trace:
I20260709 13:20:28.622298 16225 rpcz_store.cc:276] 0709 13:20:25.629074 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:28.494848 (+2865774us) service_pool.cc:224] Handling call
0709 13:20:28.517422 (+ 22574us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:28.581351 (+ 63929us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:28.581581 (+   230us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:20:28.622440 16296 rpcz_store.cc:276] 0709 13:20:27.540188 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:28.410746 (+870558us) service_pool.cc:224] Handling call
0709 13:20:28.411313 (+   567us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:28.582272 (+170959us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:28.582463 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:20:28.623996 16296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a9de347c711477e87c582d003f36925" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:28.584501 16146 raft_consensus.cc:606] T 18ad7e3af5504571a69d10c00750a01c P 647a343a542d4a139fe106c19095129b [term 8 LEADER]: Received request to transfer leadership to TS 90f3da57796b488c9087a2f523adf045
I20260709 13:20:28.463831 19720 consensus_queue.cc:1059] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: INVALID_TERM, Last received: 4.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.094s
I20260709 13:20:28.585160 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c11074579b046d99e585c2818e18a2c" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
W20260709 13:20:28.585862 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 67103.7us
I20260709 13:20:28.630999 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c74490c40243a88e62d1760b5cd814" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:20:28.631268 16217 raft_consensus.cc:3060] T 8c11074579b046d99e585c2818e18a2c P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Advancing to term 9
W20260709 13:20:28.572660 16789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 43788.9us
I20260709 13:20:28.632495 16789 leader_election.cc:304] T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:20:28.559939 19733 raft_consensus.cc:3060] T c0eab25ce4c64e018a041beeda2867db P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:20:28.612808 19806 raft_consensus.cc:3060] T b87147731a68400e8cce9b879daa7c5d P cb82cc7bd6db46d980c75237a1402860 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:20:28.598445 16265 leader_election.cc:304] T 9e65676b21b748728ad39668cfe722ec P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: cb82cc7bd6db46d980c75237a1402860
I20260709 13:20:28.598855 16299 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48674 (request call id 2428) took 1059 ms. Trace:
W20260709 13:20:28.567999 19696 consensus_meta.cc:287] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b: Time spent flushing consensus metadata: real 1.002s	user 0.006s	sys 0.006s
I20260709 13:20:28.566028 19667 raft_consensus.cc:3060] T e5ad4eb281564936a59f41e387a47156 P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:20:28.610639 19702 raft_consensus.cc:515] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:28.607880 19589 consensus_queue.cc:1059] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: INVALID_TERM, Last received: 7.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.004s
I20260709 13:20:28.592686 19700 leader_election.cc:290] T 1df93779209748b9860f6d893dd3de53 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 9 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:28.431762 19710 raft_consensus.cc:515] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:28.614077 16147 rpcz_store.cc:276] 0709 13:20:26.956995 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:28.490264 (+1533269us) service_pool.cc:224] Handling call
0709 13:20:28.490539 (+   275us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:28.491037 (+   498us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:28.517623 (+ 26586us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:20:28.544204 19569 consensus_queue.cc:1243] T 5a96373bb0e14dce8a266b2fc6c58507 P 647a343a542d4a139fe106c19095129b [LEADER]: Peer 1ca2725d290e47d68b1a4b609c9816ac log is divergent from this leader: its last log entry 6.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260709 13:20:28.571980 19540 raft_consensus.cc:493] T 5a959dbecc2648eb85d1cc550f5a3245 P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:28.615111 18599 consensus_queue.cc:1059] T d16dfad0c3b540e5b4b86492ddf87208 P 647a343a542d4a139fe106c19095129b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:20:28.572751 19532 raft_consensus.cc:493] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:28.545032 19575 raft_consensus.cc:515] T 26354361e9b044cea1c2b86c4882fe92 P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:28.625943 16296 raft_consensus.cc:2468] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 10.
I20260709 13:20:28.630515 16150 raft_consensus.cc:2410] T 434bff3082d141838068f323bdb0d1ed P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 for term 10 because replica has last-logged OpId of term: 7 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 7.
I20260709 13:20:28.591869 19197 raft_consensus.cc:2749] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:20:28.611115 19754 consensus_queue.cc:237] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:28.636766 16143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:28.701399 18599 raft_consensus.cc:3055] T d16dfad0c3b540e5b4b86492ddf87208 P 647a343a542d4a139fe106c19095129b [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:20:28.701956 18599 raft_consensus.cc:740] T d16dfad0c3b540e5b4b86492ddf87208 P 647a343a542d4a139fe106c19095129b [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:20:28.701920 16143 raft_consensus.cc:2393] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 9: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
I20260709 13:20:28.633855 16142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4f65cdf00b748bb963ee6e81ef43b79" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:28.704867 16142 raft_consensus.cc:2468] T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 10.
I20260709 13:20:28.710587 16296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e9b2bf7011f4ae688a79d7bb5a447b4" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:28.715844 16296 raft_consensus.cc:2468] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 6.
I20260709 13:20:28.717096 16219 raft_consensus.cc:2468] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 9.
I20260709 13:20:28.722136 16296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fffab26427e24d398e2695aad40ca134" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:20:28.723968 16824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8638ee363273426db1393dd81e5c7b72" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:28.731146 16824 raft_consensus.cc:2468] T 8638ee363273426db1393dd81e5c7b72 P cb82cc7bd6db46d980c75237a1402860 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 7.
I20260709 13:20:28.670133 16821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7ca61592e848ea9b0fb81a2ce33cdc" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
W20260709 13:20:28.725081 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 108986us
I20260709 13:20:28.734259 16191 leader_election.cc:304] T 829b4ad8c5a54eababeea1181b960d4b P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
I20260709 13:20:28.735610 16191 leader_election.cc:304] T 8cc959fb7f704fba9009c0595467d074 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 90f3da57796b488c9087a2f523adf045
I20260709 13:20:28.656215 16116 leader_election.cc:304] T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860
I20260709 13:20:28.725903 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8638ee363273426db1393dd81e5c7b72" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:28.714207 16300 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48674 (request call id 2422) took 1240 ms. Trace:
I20260709 13:20:28.737033 16226 raft_consensus.cc:2468] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 8.
I20260709 13:20:28.650528 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5ad4eb281564936a59f41e387a47156" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:28.737082 16300 rpcz_store.cc:276] 0709 13:20:27.474268 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:28.346415 (+872147us) service_pool.cc:224] Handling call
0709 13:20:28.346691 (+   276us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:20:28.346874 (+   183us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:28.346897 (+    23us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:20:28.360421 (+ 13524us) log.cc:844] Serialized 26 byte log entry
0709 13:20:28.582380 (+221959us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:20:28.582416 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:28.582482 (+    66us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:20:28.713900 (+131418us) raft_consensus.cc:1685] finished
0709 13:20:28.713930 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:28.714129 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":176288,"threads_started":1,"wal-append.queue_time_us":285319}
I20260709 13:20:28.730181 16217 raft_consensus.cc:2468] T 8c11074579b046d99e585c2818e18a2c P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 9.
I20260709 13:20:28.598186 19568 raft_consensus.cc:3060] T 9e65676b21b748728ad39668cfe722ec P cb82cc7bd6db46d980c75237a1402860 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:20:28.739701 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:20:28.761413 16217 raft_consensus.cc:2393] T 19baa7bc367844c288bca8f81439c413 P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 9: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
I20260709 13:20:28.762735 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2372981006d4b218215c28964dea623" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:28.753959 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
W20260709 13:20:28.755151 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 99006.4us
I20260709 13:20:28.579367 19414 raft_consensus.cc:515] T c9944d5e4b8c4109af9955c4f08c9095 P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:28.764848 16225 raft_consensus.cc:2393] T 19baa7bc367844c288bca8f81439c413 P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 in current term 9: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
I20260709 13:20:28.738729 16293 raft_consensus.cc:2468] T e5ad4eb281564936a59f41e387a47156 P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 9.
I20260709 13:20:28.767073 16823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806d7690c2b644959152074bb2cab0a7" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:28.609719 19345 raft_consensus.cc:2804] T 2794af5b99984efebdee564035841cb0 P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:20:28.667977 16117 leader_election.cc:304] T ee9ed67006144797b1de6ea8c2013a68 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:20:28.773548 16816 raft_consensus.cc:1240] T 3884f864a96d428785895e5463e5728f P cb82cc7bd6db46d980c75237a1402860 [term 7 LEADER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 6. Current term is 7. Ops: []
W20260709 13:20:28.774025 16117 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 106143us
I20260709 13:20:28.774789 16117 leader_election.cc:304] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860
W20260709 13:20:28.776046 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 177674us
I20260709 13:20:28.767992 16823 raft_consensus.cc:2468] T 806d7690c2b644959152074bb2cab0a7 P cb82cc7bd6db46d980c75237a1402860 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 6.
I20260709 13:20:28.776872 16265 leader_election.cc:304] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:20:28.781268 16143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c74490c40243a88e62d1760b5cd814" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:20:28.782019 16219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c74490c40243a88e62d1760b5cd814" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:20:28.782730 16219 raft_consensus.cc:2393] T 08c74490c40243a88e62d1760b5cd814 P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 8: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
I20260709 13:20:28.783465 16218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5ad4eb281564936a59f41e387a47156" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
W20260709 13:20:28.654879 19799 log.cc:927] Time spent T 1df93779209748b9860f6d893dd3de53 P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.181s	user 0.001s	sys 0.000s
I20260709 13:20:28.801412 16818 raft_consensus.cc:1275] T ee9ed67006144797b1de6ea8c2013a68 P cb82cc7bd6db46d980c75237a1402860 [term 6 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:20:28.791210 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cc959fb7f704fba9009c0595467d074" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:20:28.793651 16224 raft_consensus.cc:2468] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 8.
I20260709 13:20:28.786259 16820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c11074579b046d99e585c2818e18a2c" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:28.780907 18599 consensus_queue.cc:260] T d16dfad0c3b540e5b4b86492ddf87208 P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:28.781638 19345 raft_consensus.cc:493] T 2794af5b99984efebdee564035841cb0 P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Starting leader election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:20:28.654191 19610 consensus_queue.cc:1048] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 3, Time since last communication: 0.138s
I20260709 13:20:28.621129 19726 consensus_queue.cc:1059] T 9c61a39fd4f84424b129c1a6f9ba404d P 647a343a542d4a139fe106c19095129b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.171s
I20260709 13:20:28.765776 19699 raft_consensus.cc:2804] T 829b4ad8c5a54eababeea1181b960d4b P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:20:28.773147 19683 raft_consensus.cc:2804] T 8cc959fb7f704fba9009c0595467d074 P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:20:28.773342 19522 raft_consensus.cc:2749] T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:20:28.775426 19532 raft_consensus.cc:515] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:28.882479 16298 raft_consensus.cc:1240] T 68a6838d82ef4cac83507f906139d90f P 90f3da57796b488c9087a2f523adf045 [term 9 LEADER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 8. Current term is 9. Ops: []
I20260709 13:20:28.637045 19673 raft_consensus.cc:2749] T 19baa7bc367844c288bca8f81439c413 P cb82cc7bd6db46d980c75237a1402860 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:20:28.639366 19666 raft_consensus.cc:515] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:28.791145 16143 raft_consensus.cc:2393] T 08c74490c40243a88e62d1760b5cd814 P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 8: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
I20260709 13:20:28.792768 19575 leader_election.cc:290] T 26354361e9b044cea1c2b86c4882fe92 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 7 election: Requested vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:28.877197 19718 raft_consensus.cc:2804] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:20:28.887884 19718 raft_consensus.cc:493] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:28.888254 19718 raft_consensus.cc:3060] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:20:28.888144 19751 consensus_queue.cc:1048] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:20:28.855050 19700 raft_consensus.cc:493] T d2ab0ac8347641e6955e9bce7d90c1a2 P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:20:28.889258 19700 raft_consensus.cc:515] T d2ab0ac8347641e6955e9bce7d90c1a2 P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:28.792042 16219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c2095a5f4ad49408dbc7f69c606120d" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
W20260709 13:20:28.890133 16791 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:28.890573 16219 raft_consensus.cc:2410] T 7c2095a5f4ad49408dbc7f69c606120d P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 for term 8 because replica has last-logged OpId of term: 7 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 13:20:28.891464 19700 leader_election.cc:290] T d2ab0ac8347641e6955e9bce7d90c1a2 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:28.892297 19672 leader_election.cc:290] T 806d7690c2b644959152074bb2cab0a7 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:28.688687 19450 consensus_queue.cc:1048] T 746aabf856ab401c9ef5e859371e7b67 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.182s
I20260709 13:20:28.855113 19197 raft_consensus.cc:2749] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:20:28.765061 19540 raft_consensus.cc:515] T 5a959dbecc2648eb85d1cc550f5a3245 P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:28.656562 19720 consensus_queue.cc:1243] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Peer 647a343a542d4a139fe106c19095129b log is divergent from this leader: its last log entry 5.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 13:20:28.654458 19802 raft_consensus.cc:2804] T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:20:28.825489 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a959dbecc2648eb85d1cc550f5a3245" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:20:28.609947 19760 consensus_queue.cc:1059] T 84f2da98864b4425bf7c0055951fdd4b P 647a343a542d4a139fe106c19095129b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.173s
I20260709 13:20:28.906278 16221 raft_consensus.cc:2410] T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 for term 9 because replica has last-logged OpId of term: 8 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 3.
I20260709 13:20:28.794651 16819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66ae176961b74813a58bc15c9c4c6f2a" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:28.768564 19696 raft_consensus.cc:515] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:28.906833 16819 raft_consensus.cc:2468] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 10.
I20260709 13:20:28.654282 19807 consensus_queue.cc:1048] T 57a744f5aaf944678a63469cbf514015 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.240s
I20260709 13:20:28.683765 19724 consensus_queue.cc:237] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:20:28.809679 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 187050us
I20260709 13:20:28.856003 18599 raft_consensus.cc:3060] T d16dfad0c3b540e5b4b86492ddf87208 P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:20:28.814569 19345 raft_consensus.cc:3060] T 2794af5b99984efebdee564035841cb0 P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:20:28.803798 16299 rpcz_store.cc:276] 0709 13:20:27.539880 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:28.409469 (+869589us) service_pool.cc:224] Handling call
0709 13:20:28.409759 (+   290us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:28.598595 (+188836us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:28.598799 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:20:28.913452 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57a744f5aaf944678a63469cbf514015" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:28.914021 16299 raft_consensus.cc:2468] T 57a744f5aaf944678a63469cbf514015 P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 7.
I20260709 13:20:28.856384 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:20:28.915609 16300 raft_consensus.cc:2393] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 9: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
W20260709 13:20:28.862353 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 85624us
W20260709 13:20:28.869796 16115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 254585us
I20260709 13:20:28.871811 16147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a959dbecc2648eb85d1cc550f5a3245" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:28.917821 16147 raft_consensus.cc:2468] T 5a959dbecc2648eb85d1cc550f5a3245 P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 6.
W20260709 13:20:28.873755 16266 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:28.875653 16818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2794af5b99984efebdee564035841cb0" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
W20260709 13:20:28.880236 16189 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:28.856458 16223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87ac0d1f7e8a419f9bcbc04aee0e067e" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:28.919358 16223 raft_consensus.cc:2393] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 9: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
I20260709 13:20:28.794616 16222 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57604 (request call id 2340) took 3289 ms. Trace:
I20260709 13:20:28.802295 19418 raft_consensus.cc:2804] T ee9ed67006144797b1de6ea8c2013a68 P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:20:28.919703 16222 rpcz_store.cc:276] 0709 13:20:25.505516 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:25.516035 (+ 10519us) service_pool.cc:224] Handling call
0709 13:20:25.516321 (+   286us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:20:25.727352 (+211031us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:25.727381 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:20:28.228558 (+2501177us) spinlock_profiling.cc:243] Waited 2.64 ms on lock 0x7b2c00115ac8. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69af458a11 00007f69af468372 00007f69b0b9aa01 00007f69b0b5f43a 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2
0709 13:20:28.252462 (+ 23904us) log.cc:844] Serialized 26 byte log entry
0709 13:20:28.436685 (+184223us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:20:28.436725 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:28.436793 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:20:28.794353 (+357560us) raft_consensus.cc:1685] finished
0709 13:20:28.794372 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:28.794564 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":47090,"raft.queue_time_us":71805,"raft.run_cpu_time_us":137,"raft.run_wall_time_us":13415,"spinlock_wait_cycles":6079872,"thread_start_us":257693,"threads_started":2,"wal-append.queue_time_us":484181}
I20260709 13:20:28.921561 16222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746aabf856ab401c9ef5e859371e7b67" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:20:28.922250 16222 raft_consensus.cc:2393] T 746aabf856ab401c9ef5e859371e7b67 P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 9: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
I20260709 13:20:28.885190 16146 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:40290 (request call id 22) took 3380 ms. Trace:
I20260709 13:20:28.800797 19651 raft_consensus.cc:2804] T 9e65676b21b748728ad39668cfe722ec P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:20:28.918219 16265 leader_election.cc:304] T e5ad4eb281564936a59f41e387a47156 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:20:28.884316 16143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4dc6b0541ca431ebc757beac00521f1" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:28.791605 16142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:28.931696 16142 raft_consensus.cc:2468] T ecc1e363230043fe9fef390f0b979f20 P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 8.
I20260709 13:20:28.891424 16219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee9ed67006144797b1de6ea8c2013a68" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:28.842448 16820 raft_consensus.cc:2393] T 8c11074579b046d99e585c2818e18a2c P cb82cc7bd6db46d980c75237a1402860 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 8: Already voted for candidate cb82cc7bd6db46d980c75237a1402860 in this term.
I20260709 13:20:28.894340 16297 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48674 (request call id 2417) took 1459 ms. Trace:
I20260709 13:20:28.572796 19763 consensus_queue.cc:260] T 3884f864a96d428785895e5463e5728f P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:28.895782 16789 leader_election.cc:304] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:20:28.896813 19766 raft_consensus.cc:493] T ada5a004ba7c46819051e3ccec71ffdb P cb82cc7bd6db46d980c75237a1402860 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:20:28.902944 16151 raft_consensus.cc:1240] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b [term 10 LEADER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 9. Current term is 10. Ops: []
I20260709 13:20:28.903609 16225 raft_consensus.cc:3060] T 5a959dbecc2648eb85d1cc550f5a3245 P 1ca2725d290e47d68b1a4b609c9816ac [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:20:28.791620 16824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87147731a68400e8cce9b879daa7c5d" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:28.922554 16146 rpcz_store.cc:276] 0709 13:20:25.495749 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:25.495906 (+   157us) service_pool.cc:224] Handling call
0709 13:20:28.875915 (+3380009us) spinlock_profiling.cc:243] Waited 3.07 s on lock 0x7b540028d554. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b59853 00007f69b5b2760a 00007f69af0ad7d4 00007f69af0ad762 00007f69af0ad691 00007f69af0ad60c 00007f69af0ac8c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:20:28.876006 (+    91us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":7064838528}
I20260709 13:20:28.946262 16146 raft_consensus.cc:1275] T ecc1e363230043fe9fef390f0b979f20 P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:20:28.946494 16149 raft_consensus.cc:2368] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 for earlier term 11. Current term is 12.
I20260709 13:20:28.873052 19732 raft_consensus.cc:515] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:28.914909 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4f65cdf00b748bb963ee6e81ef43b79" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
W20260709 13:20:28.951176 16192 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 150016us
I20260709 13:20:28.951977 16299 raft_consensus.cc:2410] T c4f65cdf00b748bb963ee6e81ef43b79 P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b for term 10 because replica has last-logged OpId of term: 9 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 6.
I20260709 13:20:28.870594 19407 consensus_queue.cc:1059] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.036s
I20260709 13:20:28.927793 16819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c11074579b046d99e585c2818e18a2c" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:20:28.935375 16222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5ad4eb281564936a59f41e387a47156" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:28.933153 16297 rpcz_store.cc:276] 0709 13:20:27.434746 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:28.040220 (+605474us) service_pool.cc:224] Handling call
0709 13:20:28.040568 (+   348us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:20:28.040968 (+   400us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:28.040995 (+    27us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:20:28.041551 (+   556us) log.cc:844] Serialized 52 byte log entry
0709 13:20:28.362294 (+320743us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:20:28.427129 (+ 64835us) log.cc:844] Serialized 14 byte log entry
0709 13:20:28.427548 (+   419us) log.cc:844] Serialized 14 byte log entry
0709 13:20:28.427703 (+   155us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:28.427783 (+    80us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:20:28.654266 (+226483us) spinlock_profiling.cc:243] Waited 13.8 ms on lock 0x7b4400022b4c. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69ad8fadb1 00007f69ad998c85 00007f69ad9996a2 00007f69b070b1db 00007f69b0706aaf 00007f69b06fa86e 00007f69b06f9bb6 00007f69b06f97d8 00007f69b0703221 00007f69b07031d9 00007f69b0703169
0709 13:20:28.894139 (+239873us) raft_consensus.cc:1685] finished
0709 13:20:28.894160 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:28.894286 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":32989,"prepare.queue_time_us":96720,"prepare.run_cpu_time_us":88,"prepare.run_wall_time_us":91,"raft.queue_time_us":11397,"raft.run_cpu_time_us":252,"raft.run_wall_time_us":11433,"spinlock_wait_cycles":48025088,"thread_start_us":320360,"threads_started":1,"wal-append.queue_time_us":330579}
I20260709 13:20:28.905784 19802 raft_consensus.cc:493] T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:29.000070 19802 raft_consensus.cc:3060] T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:20:29.005997 16225 raft_consensus.cc:2468] T 5a959dbecc2648eb85d1cc550f5a3245 P 1ca2725d290e47d68b1a4b609c9816ac [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 6.
I20260709 13:20:29.012048 16818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a9de347c711477e87c582d003f36925" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:29.013423 16266 leader_election.cc:304] T b87147731a68400e8cce9b879daa7c5d P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
I20260709 13:20:29.015343 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7ca61592e848ea9b0fb81a2ce33cdc" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
W20260709 13:20:29.016193 16789 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:20:29.016906 16789 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:28.928362 19651 raft_consensus.cc:493] T 9e65676b21b748728ad39668cfe722ec P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:28.944412 16820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26354361e9b044cea1c2b86c4882fe92" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:20:28.947602 16146 raft_consensus.cc:1275] T 57a744f5aaf944678a63469cbf514015 P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:20:28.911444 16790 leader_election.cc:304] T ee9ed67006144797b1de6ea8c2013a68 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860; no voters: 
W20260709 13:20:29.021662 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 110294us
I20260709 13:20:28.947211 16190 leader_election.cc:304] T 84f2da98864b4425bf7c0055951fdd4b P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 647a343a542d4a139fe106c19095129b
I20260709 13:20:28.982263 16819 raft_consensus.cc:3060] T 8c11074579b046d99e585c2818e18a2c P cb82cc7bd6db46d980c75237a1402860 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:20:28.953470 16219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806d7690c2b644959152074bb2cab0a7" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:29.023801 16219 raft_consensus.cc:2410] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b for term 9 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 6.
I20260709 13:20:29.024331 16818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a9de347c711477e87c582d003f36925" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 11 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:29.024777 16219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a959dbecc2648eb85d1cc550f5a3245" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:29.025445 16219 raft_consensus.cc:2468] T 5a959dbecc2648eb85d1cc550f5a3245 P 1ca2725d290e47d68b1a4b609c9816ac [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 6.
I20260709 13:20:29.025911 16818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66ae176961b74813a58bc15c9c4c6f2a" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:29.026237 16219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a378d7f796f94615b3b347119a9880fd" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:29.026763 16818 raft_consensus.cc:2468] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 10.
I20260709 13:20:28.954982 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806d7690c2b644959152074bb2cab0a7" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:29.030776 16219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a378d7f796f94615b3b347119a9880fd" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:28.984041 16222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee9ed67006144797b1de6ea8c2013a68" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:29.033010 16295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8638ee363273426db1393dd81e5c7b72" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:28.952826 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4f65cdf00b748bb963ee6e81ef43b79" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:28.985472 16824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8638ee363273426db1393dd81e5c7b72" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 9 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:29.033609 16295 raft_consensus.cc:2468] T 8638ee363273426db1393dd81e5c7b72 P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 8.
I20260709 13:20:29.034096 16824 raft_consensus.cc:2410] T 8638ee363273426db1393dd81e5c7b72 P cb82cc7bd6db46d980c75237a1402860 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 for term 9 because replica has last-logged OpId of term: 7 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 7.
I20260709 13:20:29.028694 16116 leader_election.cc:304] T 5a959dbecc2648eb85d1cc550f5a3245 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
I20260709 13:20:29.033900 16299 raft_consensus.cc:2468] T c4f65cdf00b748bb963ee6e81ef43b79 P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 10.
I20260709 13:20:28.953681 19725 raft_consensus.cc:493] T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:28.986106 16192 leader_election.cc:304] T 746aabf856ab401c9ef5e859371e7b67 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
W20260709 13:20:28.986850 19810 log.cc:927] Time spent T 18ad7e3af5504571a69d10c00750a01c P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 0.114s	user 0.000s	sys 0.000s
W20260709 13:20:28.987704 19812 log.cc:927] Time spent T 8d4ab870a05347868956aa8b841b2981 P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 0.105s	user 0.000s	sys 0.000s
I20260709 13:20:28.988684 19666 leader_election.cc:290] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 9 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:28.992024 16142 raft_consensus.cc:1275] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [term 10 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:20:28.993228 19657 raft_consensus.cc:3055] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:20:28.994360 19660 raft_consensus.cc:515] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:28.995705 19540 leader_election.cc:290] T 5a959dbecc2648eb85d1cc550f5a3245 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
W20260709 13:20:29.011049 16115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 93161.8us
I20260709 13:20:29.018630 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62fe3346cd3c42c4a55e14af9073bca4" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:28.929450 19418 raft_consensus.cc:493] T ee9ed67006144797b1de6ea8c2013a68 P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:28.944476 19664 raft_consensus.cc:2804] T e5ad4eb281564936a59f41e387a47156 P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:20:29.022691 16790 leader_election.cc:304] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:20:28.950911 16143 raft_consensus.cc:1240] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 5. Current term is 6. Ops: []
I20260709 13:20:29.099184 16143 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46224 (request call id 498) took 1096 ms. Trace:
I20260709 13:20:29.099651 16143 rpcz_store.cc:276] 0709 13:20:28.002991 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:28.944952 (+941961us) service_pool.cc:224] Handling call
0709 13:20:28.945291 (+   339us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:29.098897 (+153606us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:29.099097 (+   200us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:20:29.030043 19651 raft_consensus.cc:3060] T 9e65676b21b748728ad39668cfe722ec P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:20:28.954541 16147 raft_consensus.cc:1275] T 8a878999c0024d2abe517484180290fc P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 8 index: 4. (index mismatch)
I20260709 13:20:29.131603 16147 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40830 (request call id 1321) took 1495 ms. Trace:
I20260709 13:20:29.132158 16147 rpcz_store.cc:276] 0709 13:20:27.635783 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:28.918416 (+1282633us) service_pool.cc:224] Handling call
0709 13:20:28.954147 (+ 35731us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:29.131185 (+177038us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:29.131517 (+   332us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:20:29.029258 16822 raft_consensus.cc:1240] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860 [term 8 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 7. Current term is 8. Ops: []
I20260709 13:20:28.948513 19673 raft_consensus.cc:2804] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:20:28.985913 16223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66ae176961b74813a58bc15c9c4c6f2a" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:29.134820 16223 raft_consensus.cc:2468] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 10.
I20260709 13:20:28.952710 19702 leader_election.cc:290] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
W20260709 13:20:29.021402 19785 consensus_meta.cc:287] T 23f82224f4564fea93572e8206bad1fe P cb82cc7bd6db46d980c75237a1402860: Time spent flushing consensus metadata: real 0.589s	user 0.004s	sys 0.008s
W20260709 13:20:29.021025 19794 consensus_meta.cc:287] T e4dc6b0541ca431ebc757beac00521f1 P 90f3da57796b488c9087a2f523adf045: Time spent flushing consensus metadata: real 0.573s	user 0.011s	sys 0.000s
I20260709 13:20:28.948709 19789 consensus_queue.cc:1059] T 3884f864a96d428785895e5463e5728f P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.087s
I20260709 13:20:29.091526 16266 leader_election.cc:304] T 66ae176961b74813a58bc15c9c4c6f2a P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 
W20260709 13:20:29.153086 16266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 61631.7us
I20260709 13:20:29.032016 19407 consensus_queue.cc:1243] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Peer cb82cc7bd6db46d980c75237a1402860 log is divergent from this leader: its last log entry 10.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260709 13:20:29.154907 19774 raft_consensus.cc:2804] T 66ae176961b74813a58bc15c9c4c6f2a P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:20:29.155705 19774 raft_consensus.cc:493] T 66ae176961b74813a58bc15c9c4c6f2a P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:29.156239 19774 raft_consensus.cc:3060] T 66ae176961b74813a58bc15c9c4c6f2a P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:20:29.061295 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 114184us
I20260709 13:20:29.158692 16190 leader_election.cc:304] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: cb82cc7bd6db46d980c75237a1402860
I20260709 13:20:29.160904 16190 leader_election.cc:304] T e4dc6b0541ca431ebc757beac00521f1 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:20:29.161538 19702 raft_consensus.cc:2804] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:20:29.092828 16219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a878999c0024d2abe517484180290fc" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 9 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:29.112185 19744 raft_consensus.cc:2749] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:20:29.163349 16219 raft_consensus.cc:2410] T 8a878999c0024d2abe517484180290fc P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b for term 9 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
W20260709 13:20:29.054949 16192 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 68911.1us
I20260709 13:20:29.095904 16225 raft_consensus.cc:2393] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 in current term 11: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
I20260709 13:20:29.164912 16192 leader_election.cc:304] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:20:29.166647 16192 leader_election.cc:304] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 
W20260709 13:20:29.027943 16189 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:20:29.102587 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 80027.3us
I20260709 13:20:29.170099 16790 leader_election.cc:304] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:20:29.103631 16115 leader_election.cc:304] T 68a6838d82ef4cac83507f906139d90f P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
W20260709 13:20:29.171362 16115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 67828.1us
I20260709 13:20:29.101065 16220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6f07842d3aa47858df5665874c86f3a" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 10 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:29.173036 16220 raft_consensus.cc:2393] T b6f07842d3aa47858df5665874c86f3a P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 10: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
I20260709 13:20:29.106052 16142 raft_consensus.cc:1240] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 7. Current term is 8. Ops: []
W20260709 13:20:29.053992 19779 consensus_meta.cc:287] T 26354361e9b044cea1c2b86c4882fe92 P 90f3da57796b488c9087a2f523adf045: Time spent flushing consensus metadata: real 0.509s	user 0.009s	sys 0.004s
I20260709 13:20:29.109290 16295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:29.177906 16115 leader_election.cc:304] T 08c74490c40243a88e62d1760b5cd814 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:20:29.178745 16295 raft_consensus.cc:2393] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 9: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
I20260709 13:20:29.031827 19610 raft_consensus.cc:2804] T ee9ed67006144797b1de6ea8c2013a68 P cb82cc7bd6db46d980c75237a1402860 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:20:29.113771 16294 raft_consensus.cc:1240] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 10. Current term is 11. Ops: [10.10-10.10]
I20260709 13:20:29.113991 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806d7690c2b644959152074bb2cab0a7" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:29.032513 19804 raft_consensus.cc:2749] T b87147731a68400e8cce9b879daa7c5d P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:20:29.083402 19725 raft_consensus.cc:515] T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:29.090840 19657 raft_consensus.cc:740] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:20:29.101797 19664 raft_consensus.cc:493] T e5ad4eb281564936a59f41e387a47156 P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:29.117211 16143 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "31969392ccdb43b491911a319e8bc7e1"
dest_uuid: "647a343a542d4a139fe106c19095129b"
mode: GRACEFUL
new_leader_uuid: "90f3da57796b488c9087a2f523adf045"
 from {username='slave'} at 127.0.0.1:40290
I20260709 13:20:29.117647 19718 raft_consensus.cc:515] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:29.098284 19418 raft_consensus.cc:3060] T ee9ed67006144797b1de6ea8c2013a68 P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:20:28.956135 19766 raft_consensus.cc:515] T ada5a004ba7c46819051e3ccec71ffdb P cb82cc7bd6db46d980c75237a1402860 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:29.083664 19720 raft_consensus.cc:2749] T 746aabf856ab401c9ef5e859371e7b67 P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:20:29.036527 19532 leader_election.cc:290] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:29.085857 19820 raft_consensus.cc:2804] T 84f2da98864b4425bf7c0055951fdd4b P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:20:29.140053 16818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e9b2bf7011f4ae688a79d7bb5a447b4" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:29.035351 16222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 9 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:29.227413 16222 raft_consensus.cc:2410] T ecc1e363230043fe9fef390f0b979f20 P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b for term 9 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 4.
I20260709 13:20:29.163538 19414 leader_election.cc:290] T c9944d5e4b8c4109af9955c4f08c9095 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:29.098930 19816 raft_consensus.cc:2804] T 5a959dbecc2648eb85d1cc550f5a3245 P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:20:29.179940 16790 leader_election.cc:304] T 66f75124215f4ca6bef831e174a792d8 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:20:29.184695 19672 raft_consensus.cc:2749] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:20:29.233583 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 53734.6us
I20260709 13:20:29.183123 16115 leader_election.cc:304] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:20:29.143748 19699 raft_consensus.cc:697] T 829b4ad8c5a54eababeea1181b960d4b P 1ca2725d290e47d68b1a4b609c9816ac [term 9 LEADER]: Becoming Leader. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:20:29.139281 19673 raft_consensus.cc:493] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:29.234448 16790 leader_election.cc:304] T af57996a8be84d398f0f023e8d08ef77 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 647a343a542d4a139fe106c19095129b
I20260709 13:20:29.184223 16225 raft_consensus.cc:1240] T 1df93779209748b9860f6d893dd3de53 P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 8. Current term is 9. Ops: []
I20260709 13:20:29.184094 16295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18ad7e3af5504571a69d10c00750a01c" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:29.210461 16143 raft_consensus.cc:606] T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b [term 7 LEADER]: Received request to transfer leadership to TS 90f3da57796b488c9087a2f523adf045
I20260709 13:20:29.237010 16225 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57604 (request call id 2344) took 2480 ms. Trace:
I20260709 13:20:29.210662 16220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5d59407eb545b895d8d922e0dd69fd" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:29.212388 16219 raft_consensus.cc:1240] T d16dfad0c3b540e5b4b86492ddf87208 P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 7. Current term is 8. Ops: []
I20260709 13:20:29.237349 16225 rpcz_store.cc:276] 0709 13:20:26.756130 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:29.165019 (+2408889us) service_pool.cc:224] Handling call
0709 13:20:29.184039 (+ 19020us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:29.236773 (+ 52734us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:29.236958 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:20:29.237461 16295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806d7690c2b644959152074bb2cab0a7" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
W20260709 13:20:29.189005 16116 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:29.238716 16220 raft_consensus.cc:1240] T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 7. Current term is 8. Ops: []
I20260709 13:20:29.238812 16295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6f07842d3aa47858df5665874c86f3a" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:20:29.239481 16295 raft_consensus.cc:3060] T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:20:29.239559 16116 leader_election.cc:304] T ecc1e363230043fe9fef390f0b979f20 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860
I20260709 13:20:29.240339 16220 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57604 (request call id 2345) took 2480 ms. Trace:
I20260709 13:20:29.214046 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66ae176961b74813a58bc15c9c4c6f2a" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:29.240725 16220 rpcz_store.cc:276] 0709 13:20:26.760119 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:29.238184 (+2478065us) service_pool.cc:224] Handling call
0709 13:20:29.238521 (+   337us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:29.240099 (+  1578us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:29.240287 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1792}
I20260709 13:20:29.242785 19197 raft_consensus.cc:2749] T ecc1e363230043fe9fef390f0b979f20 P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:20:29.150841 19683 raft_consensus.cc:697] T 8cc959fb7f704fba9009c0595467d074 P 1ca2725d290e47d68b1a4b609c9816ac [term 9 LEADER]: Becoming Leader. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:20:29.186233 19575 raft_consensus.cc:2749] T 08c74490c40243a88e62d1760b5cd814 P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:20:29.211548 19664 raft_consensus.cc:3060] T e5ad4eb281564936a59f41e387a47156 P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:20:29.219584 19818 log.cc:927] Time spent T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 0.092s	user 0.000s	sys 0.001s
I20260709 13:20:29.184330 19610 raft_consensus.cc:493] T ee9ed67006144797b1de6ea8c2013a68 P cb82cc7bd6db46d980c75237a1402860 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:29.180641 19754 raft_consensus.cc:2749] T 68a6838d82ef4cac83507f906139d90f P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:20:29.223910 19815 log.cc:927] Time spent T 23f82224f4564fea93572e8206bad1fe P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.126s	user 0.001s	sys 0.000s
W20260709 13:20:29.224730 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 48214.4us
W20260709 13:20:29.144959 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 215098us
I20260709 13:20:29.182365 19819 raft_consensus.cc:2804] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:20:29.234457 16222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8638ee363273426db1393dd81e5c7b72" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 9 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:29.162031 19666 raft_consensus.cc:2804] T e4dc6b0541ca431ebc757beac00521f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:20:29.215284 19822 raft_consensus.cc:2804] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:20:29.243369 19760 consensus_queue.cc:1059] T 1df93779209748b9860f6d893dd3de53 P 647a343a542d4a139fe106c19095129b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:20:29.162066 19702 raft_consensus.cc:493] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Starting leader election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:20:29.184191 19540 consensus_queue.cc:1048] T b6f07842d3aa47858df5665874c86f3a P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 2, Time since last communication: 0.049s
I20260709 13:20:29.232774 19820 raft_consensus.cc:697] T 84f2da98864b4425bf7c0055951fdd4b P 1ca2725d290e47d68b1a4b609c9816ac [term 7 LEADER]: Becoming Leader. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:20:29.176847 19794 raft_consensus.cc:515] T e4dc6b0541ca431ebc757beac00521f1 P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:29.247515 16295 raft_consensus.cc:2468] T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 10.
I20260709 13:20:29.248633 16220 raft_consensus.cc:1275] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 7. (term mismatch)
I20260709 13:20:29.249047 18599 consensus_queue.cc:1059] T 9c61a39fd4f84424b129c1a6f9ba404d P 647a343a542d4a139fe106c19095129b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:20:29.249835 16219 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57604 (request call id 2343) took 2495 ms. Trace:
W20260709 13:20:29.262481 19783 consensus_meta.cc:287] T fffab26427e24d398e2695aad40ca134 P 90f3da57796b488c9087a2f523adf045: Time spent flushing consensus metadata: real 0.785s	user 0.009s	sys 0.004s
I20260709 13:20:29.303843 19783 raft_consensus.cc:515] T fffab26427e24d398e2695aad40ca134 P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:29.282137 19804 raft_consensus.cc:493] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:29.232532 19718 leader_election.cc:290] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 7 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:29.186558 19779 raft_consensus.cc:515] T 26354361e9b044cea1c2b86c4882fe92 P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:29.306051 19783 leader_election.cc:290] T fffab26427e24d398e2695aad40ca134 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 8 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:20:29.244966 19532 raft_consensus.cc:2804] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:20:29.306666 19779 leader_election.cc:290] T 26354361e9b044cea1c2b86c4882fe92 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 7 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:29.257705 16190 leader_election.cc:304] T d16dfad0c3b540e5b4b86492ddf87208 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 647a343a542d4a139fe106c19095129b
I20260709 13:20:29.307430 19702 raft_consensus.cc:3060] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:20:29.308069 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 50452us
I20260709 13:20:29.308897 16190 leader_election.cc:304] T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 647a343a542d4a139fe106c19095129b
I20260709 13:20:29.297582 16192 leader_election.cc:304] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:20:29.309808 19710 leader_election.cc:290] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:29.310112 16190 leader_election.cc:400] T 8c11074579b046d99e585c2818e18a2c P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 8 pre-election: Vote denied by peer 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389) with higher term. Message: Invalid argument: T 8c11074579b046d99e585c2818e18a2c P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac for earlier term 8. Current term is 9.
I20260709 13:20:29.310402 19710 raft_consensus.cc:2804] T d16dfad0c3b540e5b4b86492ddf87208 P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:20:29.245405 19673 raft_consensus.cc:3060] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:20:29.258258 19672 raft_consensus.cc:2804] T af57996a8be84d398f0f023e8d08ef77 P cb82cc7bd6db46d980c75237a1402860 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:20:29.265687 19725 leader_election.cc:290] T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:29.265512 19683 consensus_queue.cc:237] T 8cc959fb7f704fba9009c0595467d074 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.6, Last appended by leader: 6, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:29.267014 19674 leader_election.cc:290] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:29.268183 19766 leader_election.cc:290] T ada5a004ba7c46819051e3ccec71ffdb P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:29.268527 19822 raft_consensus.cc:493] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:29.336002 19822 raft_consensus.cc:3060] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:20:29.269497 19819 raft_consensus.cc:697] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [term 11 LEADER]: Becoming Leader. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:20:29.261802 19666 raft_consensus.cc:493] T e4dc6b0541ca431ebc757beac00521f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:29.346493 19418 raft_consensus.cc:515] T ee9ed67006144797b1de6ea8c2013a68 P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:29.347371 19666 raft_consensus.cc:3060] T e4dc6b0541ca431ebc757beac00521f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:20:29.349161 19418 leader_election.cc:290] T ee9ed67006144797b1de6ea8c2013a68 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 7 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:29.254789 19809 raft_consensus.cc:2749] T 66f75124215f4ca6bef831e174a792d8 P cb82cc7bd6db46d980c75237a1402860 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:20:29.272480 16821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74ca8140f35c4ae29e8a664f6bdfc829" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:29.272367 19699 consensus_queue.cc:237] T 829b4ad8c5a54eababeea1181b960d4b P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:29.273454 16292 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48674 (request call id 2418) took 1834 ms. Trace:
W20260709 13:20:29.291734 19803 log.cc:927] Time spent T c0eab25ce4c64e018a041beeda2867db P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.793s	user 0.005s	sys 0.005s
I20260709 13:20:29.352262 16292 rpcz_store.cc:276] 0709 13:20:27.439363 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:28.254291 (+814928us) service_pool.cc:224] Handling call
0709 13:20:28.254763 (+   472us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:20:28.256371 (+  1608us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:28.256404 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:20:28.256879 (+   475us) log.cc:844] Serialized 26 byte log entry
0709 13:20:28.579205 (+322326us) spinlock_profiling.cc:243] Waited 5.82 ms on lock 0x7b1800388600. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b693ade1 00007f69b69475b1 00007f69ada54e07 00007f69b68afab9 00007f69ada7169b 00007f69ada6f93f 00007f69ada71c02 00007f69b06f920b 00007f69b06fed23 00007f69b06ff0d1 00007f69b0b1aa0f
0709 13:20:28.855500 (+276295us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:20:28.855537 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:28.855604 (+    67us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:20:29.273103 (+417499us) raft_consensus.cc:1685] finished
0709 13:20:29.273129 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:29.273380 (+   251us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":579,"raft.queue_time_us":1183,"raft.run_cpu_time_us":204,"raft.run_wall_time_us":37559,"spinlock_wait_cycles":13386624,"thread_start_us":253924,"threads_started":1,"wal-append.queue_time_us":690101}
I20260709 13:20:29.353552 16821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e9b2bf7011f4ae688a79d7bb5a447b4" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:29.354266 16292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ab0ac8347641e6955e9bce7d90c1a2" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:29.355367 16294 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40474 (request call id 540) took 1607 ms. Trace:
I20260709 13:20:29.355738 16294 rpcz_store.cc:276] 0709 13:20:27.748393 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:28.542855 (+794462us) service_pool.cc:224] Handling call
0709 13:20:28.545926 (+  3071us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:20:29.181124 (+635198us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:29.355087 (+173963us) spinlock_profiling.cc:243] Waited 568 ms on lock 0x7b5400163354. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:20:29.355300 (+   213us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1305288960}
I20260709 13:20:29.356920 16822 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49896 (request call id 1048) took 1132 ms. Trace:
I20260709 13:20:29.357224 16822 rpcz_store.cc:276] 0709 13:20:28.224099 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:28.224218 (+   119us) service_pool.cc:224] Handling call
0709 13:20:28.243345 (+ 19127us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:29.133656 (+890311us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:29.356731 (+223075us) spinlock_profiling.cc:243] Waited 786 ms on lock 0x7b5400625554. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:20:29.356876 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1806892288}
I20260709 13:20:29.360338 16821 raft_consensus.cc:3060] T 806d7690c2b644959152074bb2cab0a7 P cb82cc7bd6db46d980c75237a1402860 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:20:29.294057 16823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ab0ac8347641e6955e9bce7d90c1a2" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:29.305832 19345 raft_consensus.cc:515] T 2794af5b99984efebdee564035841cb0 P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:29.306758 19532 raft_consensus.cc:493] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:29.310628 19720 raft_consensus.cc:2804] T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:20:29.152010 19785 raft_consensus.cc:515] T 23f82224f4564fea93572e8206bad1fe P cb82cc7bd6db46d980c75237a1402860 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:29.310796 19823 raft_consensus.cc:2804] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:20:29.372862 19823 raft_consensus.cc:493] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:29.373584 19823 raft_consensus.cc:3060] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:20:29.311187 19671 raft_consensus.cc:3055] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:20:29.251335 19657 consensus_queue.cc:260] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:29.279889 19414 raft_consensus.cc:493] T d2ab0ac8347641e6955e9bce7d90c1a2 P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:20:29.233009 19816 raft_consensus.cc:493] T 5a959dbecc2648eb85d1cc550f5a3245 P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:29.294178 19660 leader_election.cc:290] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 11 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
W20260709 13:20:29.296474 19667 consensus_meta.cc:287] T e5ad4eb281564936a59f41e387a47156 P 1ca2725d290e47d68b1a4b609c9816ac: Time spent flushing consensus metadata: real 0.532s	user 0.006s	sys 0.003s
W20260709 13:20:29.322059 19806 consensus_meta.cc:287] T b87147731a68400e8cce9b879daa7c5d P cb82cc7bd6db46d980c75237a1402860: Time spent flushing consensus metadata: real 0.689s	user 0.011s	sys 0.000s
I20260709 13:20:29.321758 19820 consensus_queue.cc:237] T 84f2da98864b4425bf7c0055951fdd4b P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:20:29.323316 19733 consensus_meta.cc:287] T c0eab25ce4c64e018a041beeda2867db P 647a343a542d4a139fe106c19095129b: Time spent flushing consensus metadata: real 0.690s	user 0.005s	sys 0.006s
I20260709 13:20:29.324766 19774 raft_consensus.cc:515] T 66ae176961b74813a58bc15c9c4c6f2a P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:29.326280 19719 leader_election.cc:290] T 08c74490c40243a88e62d1760b5cd814 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 8 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:29.292420 19707 raft_consensus.cc:3055] T 5a96373bb0e14dce8a266b2fc6c58507 P 647a343a542d4a139fe106c19095129b [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:20:29.331809 19696 leader_election.cc:290] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 12 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:29.332765 19726 raft_consensus.cc:3055] T 84f2da98864b4425bf7c0055951fdd4b P 647a343a542d4a139fe106c19095129b [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:20:29.333344 19751 raft_consensus.cc:3055] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:20:29.333498 19672 raft_consensus.cc:493] T af57996a8be84d398f0f023e8d08ef77 P cb82cc7bd6db46d980c75237a1402860 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:29.345894 19819 consensus_queue.cc:237] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:29.256965 19610 raft_consensus.cc:3060] T ee9ed67006144797b1de6ea8c2013a68 P cb82cc7bd6db46d980c75237a1402860 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:20:29.416199 19707 raft_consensus.cc:740] T 5a96373bb0e14dce8a266b2fc6c58507 P 647a343a542d4a139fe106c19095129b [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:20:29.422859 19645 leader_election.cc:290] T 5a96373bb0e14dce8a266b2fc6c58507 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:20:29.422199 19707 consensus_queue.cc:260] T 5a96373bb0e14dce8a266b2fc6c58507 P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.3, Last appended by leader: 3, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:29.419785 19726 raft_consensus.cc:740] T 84f2da98864b4425bf7c0055951fdd4b P 647a343a542d4a139fe106c19095129b [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:20:29.425261 19726 consensus_queue.cc:260] T 84f2da98864b4425bf7c0055951fdd4b P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:29.420053 19751 raft_consensus.cc:740] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:20:29.428644 19751 consensus_queue.cc:260] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.10, Last appended by leader: 10, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:29.430401 19751 raft_consensus.cc:3060] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:20:29.431308 19672 raft_consensus.cc:3060] T af57996a8be84d398f0f023e8d08ef77 P cb82cc7bd6db46d980c75237a1402860 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:20:29.431740 19774 leader_election.cc:290] T 66ae176961b74813a58bc15c9c4c6f2a P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 11 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:29.364921 19532 raft_consensus.cc:3060] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:20:29.345072 19804 raft_consensus.cc:515] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:29.353650 19710 raft_consensus.cc:697] T d16dfad0c3b540e5b4b86492ddf87208 P 1ca2725d290e47d68b1a4b609c9816ac [term 8 LEADER]: Becoming Leader. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:20:29.371827 19720 raft_consensus.cc:697] T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac [term 8 LEADER]: Becoming Leader. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:20:29.417218 19657 raft_consensus.cc:3060] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:20:29.350852 19673 raft_consensus.cc:515] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:29.398886 19816 raft_consensus.cc:3060] T 5a959dbecc2648eb85d1cc550f5a3245 P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:20:29.413597 19733 raft_consensus.cc:515] T c0eab25ce4c64e018a041beeda2867db P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:29.441479 19664 raft_consensus.cc:515] T e5ad4eb281564936a59f41e387a47156 P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:29.426783 19726 raft_consensus.cc:3060] T 84f2da98864b4425bf7c0055951fdd4b P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:20:29.402803 19667 raft_consensus.cc:2749] T e5ad4eb281564936a59f41e387a47156 P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:20:29.449904 19710 consensus_queue.cc:237] T d16dfad0c3b540e5b4b86492ddf87208 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:29.388394 19671 raft_consensus.cc:740] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:20:29.413012 19785 leader_election.cc:290] T 23f82224f4564fea93572e8206bad1fe P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 8 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:29.450076 19720 consensus_queue.cc:237] T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:29.411525 19806 raft_consensus.cc:515] T b87147731a68400e8cce9b879daa7c5d P cb82cc7bd6db46d980c75237a1402860 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:29.398347 19702 raft_consensus.cc:515] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:29.458815 19671 consensus_queue.cc:260] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:29.460274 19733 leader_election.cc:290] T c0eab25ce4c64e018a041beeda2867db P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 9 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:29.461187 19804 leader_election.cc:290] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:20:29.462126 19806 leader_election.cc:290] T b87147731a68400e8cce9b879daa7c5d P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 10 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:29.461835 19794 leader_election.cc:290] T e4dc6b0541ca431ebc757beac00521f1 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 8 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:20:29.477747 19666 raft_consensus.cc:515] T e4dc6b0541ca431ebc757beac00521f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:20:29.480494 19829 log.cc:927] Time spent T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.069s	user 0.001s	sys 0.000s
I20260709 13:20:29.484622 19702 leader_election.cc:290] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 9 election: Requested vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:29.457850 19822 raft_consensus.cc:515] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:29.493315 19822 leader_election.cc:290] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 9 election: Requested vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:29.485795 19823 raft_consensus.cc:515] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:20:29.234895 16115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 51834.8us
I20260709 13:20:29.537333 16144 raft_consensus.cc:2393] T c0eab25ce4c64e018a041beeda2867db P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 9: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
I20260709 13:20:29.537739 16115 leader_election.cc:304] T e5ad4eb281564936a59f41e387a47156 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:20:29.551810 16143 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4e9b2bf7011f4ae688a79d7bb5a447b4"
dest_uuid: "647a343a542d4a139fe106c19095129b"
mode: GRACEFUL
new_leader_uuid: "90f3da57796b488c9087a2f523adf045"
 from {username='slave'} at 127.0.0.1:40290
I20260709 13:20:29.553123 16143 raft_consensus.cc:606] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Received request to transfer leadership to TS 90f3da57796b488c9087a2f523adf045
I20260709 13:20:29.553577 16143 raft_consensus.cc:614] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:20:29.558851 16115 leader_election.cc:304] T 57a744f5aaf944678a63469cbf514015 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: cb82cc7bd6db46d980c75237a1402860
I20260709 13:20:29.312273 16220 pending_rounds.cc:85] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac: Aborting all ops after (but not including) 6
I20260709 13:20:29.275143 16217 raft_consensus.cc:1240] T b6f07842d3aa47858df5665874c86f3a P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 9. Current term is 10. Ops: []
I20260709 13:20:29.584206 16220 pending_rounds.cc:107] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac: Aborting uncommitted CHANGE_CONFIG_OP operation due to leader change: 6.7
I20260709 13:20:29.584353 16218 raft_consensus.cc:2468] T e5ad4eb281564936a59f41e387a47156 P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 9.
I20260709 13:20:29.314589 16219 rpcz_store.cc:276] 0709 13:20:26.754311 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:29.164165 (+2409854us) service_pool.cc:224] Handling call
0709 13:20:29.212190 (+ 48025us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:29.249564 (+ 37374us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:29.249774 (+   210us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:20:29.585137 16217 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57604 (request call id 2354) took 1077 ms. Trace:
I20260709 13:20:29.585804 16217 rpcz_store.cc:276] 0709 13:20:28.507115 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:29.274545 (+767430us) service_pool.cc:224] Handling call
0709 13:20:29.274916 (+   371us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:29.584434 (+309518us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:29.584906 (+   472us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:20:29.310626 16190 leader_election.cc:403] T 8c11074579b046d99e585c2818e18a2c P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 8 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:20:29.586974 16115 leader_election.cc:304] T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:20:29.587925 16224 raft_consensus.cc:1240] T e4dc6b0541ca431ebc757beac00521f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 7. Current term is 8. Ops: []
I20260709 13:20:29.588191 16222 raft_consensus.cc:2393] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 9: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
I20260709 13:20:29.296065 16223 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57604 (request call id 2342) took 2542 ms. Trace:
W20260709 13:20:29.588363 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 278229us
I20260709 13:20:29.584944 16220 raft_consensus.cc:2917] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Aborting config change with OpId 6.7: Aborted: Op aborted by new leader
W20260709 13:20:29.329206 16221 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57600 (request call id 1282) took 3737 ms (client timeout 4824 ms). Trace:
I20260709 13:20:29.589238 16223 rpcz_store.cc:276] 0709 13:20:26.754059 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:29.141660 (+2387601us) service_pool.cc:224] Handling call
0709 13:20:29.159916 (+ 18256us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:20:29.160320 (+   404us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:29.160360 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:20:29.160849 (+   489us) log.cc:844] Serialized 26 byte log entry
0709 13:20:29.235332 (+ 74483us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:20:29.235370 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:29.235435 (+    65us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:20:29.295868 (+ 60433us) raft_consensus.cc:1685] finished
0709 13:20:29.295886 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:29.296006 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1493,"raft.run_cpu_time_us":101,"raft.run_wall_time_us":102,"thread_start_us":74042,"threads_started":1,"wal-append.queue_time_us":95464}
I20260709 13:20:29.448410 19651 raft_consensus.cc:515] T 9e65676b21b748728ad39668cfe722ec P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:29.320791 16291 raft_consensus.cc:1240] T e4dc6b0541ca431ebc757beac00521f1 P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 7. Current term is 8. Ops: []
I20260709 13:20:29.608448 16294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e9b2bf7011f4ae688a79d7bb5a447b4" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:29.313268 16295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1df93779209748b9860f6d893dd3de53" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:20:29.610054 16294 raft_consensus.cc:2393] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 7: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
I20260709 13:20:29.610287 16295 raft_consensus.cc:3060] T 1df93779209748b9860f6d893dd3de53 P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:20:29.610498 19666 leader_election.cc:290] T e4dc6b0541ca431ebc757beac00521f1 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 8 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:29.616931 16225 raft_consensus.cc:1240] T 84f2da98864b4425bf7c0055951fdd4b P 1ca2725d290e47d68b1a4b609c9816ac [term 7 LEADER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 6. Current term is 7. Ops: []
I20260709 13:20:29.276499 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada5a004ba7c46819051e3ccec71ffdb" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:29.313055 16296 raft_consensus.cc:2368] T fffab26427e24d398e2695aad40ca134 P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b for earlier term 7. Current term is 8.
I20260709 13:20:29.251480 16790 leader_election.cc:304] T c9944d5e4b8c4109af9955c4f08c9095 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 90f3da57796b488c9087a2f523adf045
I20260709 13:20:29.325608 16816 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49896 (request call id 1052) took 1100 ms. Trace:
I20260709 13:20:29.331130 16817 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49906 (request call id 758) took 1564 ms. Trace:
I20260709 13:20:29.648757 16816 rpcz_store.cc:276] 0709 13:20:28.225350 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:28.243765 (+ 18415us) service_pool.cc:224] Handling call
0709 13:20:28.244121 (+   356us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:28.774050 (+529929us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:29.325367 (+551317us) spinlock_profiling.cc:243] Waited 529 ms on lock 0x7b54006bf654. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:20:29.325545 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1217112320}
I20260709 13:20:29.648926 16817 rpcz_store.cc:276] 0709 13:20:27.766259 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:27.767195 (+   936us) service_pool.cc:224] Handling call
0709 13:20:29.330915 (+1563720us) spinlock_profiling.cc:243] Waited 603 ms on lock 0x7b54007c0e54. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:20:29.331056 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1387279104}
I20260709 13:20:29.650293 16818 raft_consensus.cc:2393] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 7: Already voted for candidate cb82cc7bd6db46d980c75237a1402860 in this term.
I20260709 13:20:29.652482 16816 raft_consensus.cc:1275] T 66f75124215f4ca6bef831e174a792d8 P cb82cc7bd6db46d980c75237a1402860 [term 9 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:20:29.362958 16820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434bff3082d141838068f323bdb0d1ed" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 10 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:29.656170 19695 raft_consensus.cc:2804] T 57a744f5aaf944678a63469cbf514015 P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:20:29.656868 19695 raft_consensus.cc:493] T 57a744f5aaf944678a63469cbf514015 P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:29.657408 19695 raft_consensus.cc:3060] T 57a744f5aaf944678a63469cbf514015 P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:20:29.314951 16819 raft_consensus.cc:2468] T 8c11074579b046d99e585c2818e18a2c P cb82cc7bd6db46d980c75237a1402860 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 9.
I20260709 13:20:29.375240 16822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a378d7f796f94615b3b347119a9880fd" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:29.474483 19671 raft_consensus.cc:3060] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:20:29.373121 16825 raft_consensus.cc:1240] T 23f82224f4564fea93572e8206bad1fe P cb82cc7bd6db46d980c75237a1402860 [term 8 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 7. Current term is 8. Ops: []
I20260709 13:20:29.297444 19407 consensus_queue.cc:1048] T ee9ed67006144797b1de6ea8c2013a68 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.007s
I20260709 13:20:29.461817 19610 raft_consensus.cc:515] T ee9ed67006144797b1de6ea8c2013a68 P cb82cc7bd6db46d980c75237a1402860 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:29.397900 19802 raft_consensus.cc:515] T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:20:29.604851 16221 rpcz_store.cc:269] 0709 13:20:25.591956 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:27.392870 (+1800914us) service_pool.cc:224] Handling call
0709 13:20:29.328931 (+1936061us) spinlock_profiling.cc:243] Waited 1.5 s on lock 0x7b5400396054. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:20:29.329108 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3452862720}
I20260709 13:20:29.608964 16823 raft_consensus.cc:1240] T b87147731a68400e8cce9b879daa7c5d P cb82cc7bd6db46d980c75237a1402860 [term 10 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 9. Current term is 10. Ops: []
I20260709 13:20:29.312265 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c74490c40243a88e62d1760b5cd814" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:20:29.296803 16265 leader_election.cc:304] T ada5a004ba7c46819051e3ccec71ffdb P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860
I20260709 13:20:29.698414 16190 leader_election.cc:304] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860
W20260709 13:20:29.699874 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 403004us
I20260709 13:20:29.700657 19407 raft_consensus.cc:2749] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:20:29.701211 16265 leader_election.cc:304] T c4f65cdf00b748bb963ee6e81ef43b79 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 647a343a542d4a139fe106c19095129b
I20260709 13:20:29.610720 16294 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48674 (request call id 2447) took 1037 ms. Trace:
I20260709 13:20:29.702310 16820 raft_consensus.cc:1240] T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860 [term 9 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 8. Current term is 9. Ops: [8.8-8.8]
I20260709 13:20:29.604684 16220 raft_consensus.cc:2889] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: CHANGE_CONFIG_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:20:29.606979 16223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5ad4eb281564936a59f41e387a47156" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:29.706807 16223 raft_consensus.cc:2468] T e5ad4eb281564936a59f41e387a47156 P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 9.
I20260709 13:20:29.634181 19673 leader_election.cc:290] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 7 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:29.634672 19823 leader_election.cc:290] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 11 election: Requested vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:29.635129 19651 leader_election.cc:290] T 9e65676b21b748728ad39668cfe722ec P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 10 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:29.702113 16294 rpcz_store.cc:276] 0709 13:20:28.573344 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:29.607471 (+1034127us) service_pool.cc:224] Handling call
0709 13:20:29.610581 (+  3110us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:20:29.713431 16220 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57600 (request call id 1308) took 1848 ms. Trace:
I20260709 13:20:29.713718 16294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdbd78acb4cf46298e0de8ba0d130e19" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:20:29.713932 16220 rpcz_store.cc:276] 0709 13:20:27.866593 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:29.248004 (+1381411us) service_pool.cc:224] Handling call
0709 13:20:29.248314 (+   310us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:29.713056 (+464742us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:29.713368 (+   312us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:20:29.714551 16294 raft_consensus.cc:3060] T cdbd78acb4cf46298e0de8ba0d130e19 P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:20:29.714974 16296 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48674 (request call id 2439) took 1163 ms. Trace:
I20260709 13:20:29.675500 16222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee9ed67006144797b1de6ea8c2013a68" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:20:29.635979 19822 raft_consensus.cc:2749] T 8c11074579b046d99e585c2818e18a2c P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: Leader pre-election lost for term 8. Reason: Vote denied by peer 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389) with higher term. Message: Invalid argument: T 8c11074579b046d99e585c2818e18a2c P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac for earlier term 8. Current term is 9.
I20260709 13:20:29.669131 19664 leader_election.cc:290] T e5ad4eb281564936a59f41e387a47156 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 10 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:20:29.648272 16293 raft_consensus.cc:2393] T ada5a004ba7c46819051e3ccec71ffdb P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 in current term 9: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
I20260709 13:20:29.669306 16226 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57600 (request call id 1314) took 1140 ms. Trace:
W20260709 13:20:29.676239 16791 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:29.684820 16300 raft_consensus.cc:2393] T 66ae176961b74813a58bc15c9c4c6f2a P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 11: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
W20260709 13:20:29.686707 16189 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:20:29.680902 16116 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:20:29.720114 16116 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:29.671227 16225 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57604 (request call id 2346) took 2909 ms. Trace:
I20260709 13:20:29.680708 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0eab25ce4c64e018a041beeda2867db" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:20:29.672142 16224 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57604 (request call id 2351) took 1165 ms. Trace:
I20260709 13:20:29.673167 16291 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48674 (request call id 2431) took 2132 ms. Trace:
I20260709 13:20:29.679313 16819 raft_consensus.cc:1240] T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860 [term 10 LEADER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 9. Current term is 10. Ops: [9.10-9.10]
I20260709 13:20:29.675379 16219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62fe3346cd3c42c4a55e14af9073bca4" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:20:29.675997 19569 raft_consensus.cc:3055] T 9c61a39fd4f84424b129c1a6f9ba404d P 647a343a542d4a139fe106c19095129b [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:20:29.670369 18599 raft_consensus.cc:3055] T 1df93779209748b9860f6d893dd3de53 P 647a343a542d4a139fe106c19095129b [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:20:29.656065 19575 consensus_queue.cc:1048] T 7c2095a5f4ad49408dbc7f69c606120d P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.041s
I20260709 13:20:29.684103 19610 leader_election.cc:290] T ee9ed67006144797b1de6ea8c2013a68 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 7 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:20:29.704725 19540 raft_consensus.cc:2804] T e5ad4eb281564936a59f41e387a47156 P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:20:29.673928 16218 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57604 (request call id 2357) took 1126 ms. Trace:
I20260709 13:20:29.652526 19345 leader_election.cc:290] T 2794af5b99984efebdee564035841cb0 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 7 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
W20260709 13:20:29.684103 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 432851us
I20260709 13:20:29.677223 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d4ab870a05347868956aa8b841b2981" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:29.704039 19802 leader_election.cc:290] T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 9 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:29.715509 16296 rpcz_store.cc:276] 0709 13:20:28.551044 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:28.721886 (+170842us) service_pool.cc:224] Handling call
0709 13:20:29.714569 (+992683us) spinlock_profiling.cc:243] Waited 576 ms on lock 0x7b54002cfbd4. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:20:29.714752 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1324774656}
W20260709 13:20:29.637157 16789 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:29.724351 19569 raft_consensus.cc:740] T 9c61a39fd4f84424b129c1a6f9ba404d P 647a343a542d4a139fe106c19095129b [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:20:29.726585 19529 consensus_queue.cc:1059] T b6f07842d3aa47858df5665874c86f3a P 647a343a542d4a139fe106c19095129b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:20:29.728951 16219 raft_consensus.cc:1275] T 8a878999c0024d2abe517484180290fc P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 8 index: 4. (index mismatch)
I20260709 13:20:29.721236 16224 rpcz_store.cc:276] 0709 13:20:28.506426 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:29.269593 (+763167us) service_pool.cc:224] Handling call
0709 13:20:29.269949 (+   356us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:29.588538 (+318589us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:29.671815 (+ 83277us) spinlock_profiling.cc:243] Waited 317 ms on lock 0x7b54002986d4. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:20:29.672075 (+   260us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":729667328}
I20260709 13:20:29.721627 16291 rpcz_store.cc:276] 0709 13:20:27.540795 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:28.480552 (+939757us) service_pool.cc:224] Handling call
0709 13:20:28.480855 (+   303us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:29.607767 (+1126912us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:29.672701 (+ 64934us) spinlock_profiling.cc:243] Waited 840 ms on lock 0x7b540030bbd4. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:20:29.672946 (+   245us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1931250560}
I20260709 13:20:29.732143 16218 rpcz_store.cc:276] 0709 13:20:28.547711 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:28.672537 (+124826us) service_pool.cc:224] Handling call
0709 13:20:29.673703 (+1001166us) spinlock_profiling.cc:243] Waited 800 ms on lock 0x7b54003d2a54. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:20:29.673868 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1839079168}
I20260709 13:20:29.667086 19696 raft_consensus.cc:2749] T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:20:29.726702 16217 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33050 (request call id 609) took 1159 ms. Trace:
W20260709 13:20:29.734210 16115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 144470us
I20260709 13:20:29.686316 19806 raft_consensus.cc:2804] T c9944d5e4b8c4109af9955c4f08c9095 P cb82cc7bd6db46d980c75237a1402860 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:20:29.716045 16295 raft_consensus.cc:2468] T 1df93779209748b9860f6d893dd3de53 P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 9.
I20260709 13:20:29.720717 16225 rpcz_store.cc:276] 0709 13:20:26.761802 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:29.238221 (+2476419us) service_pool.cc:224] Handling call
0709 13:20:29.238550 (+   329us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:29.617674 (+379124us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:29.670857 (+ 53183us) spinlock_profiling.cc:243] Waited 378 ms on lock 0x7b5400396554. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:20:29.671119 (+   262us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":868588288}
I20260709 13:20:29.722481 18599 raft_consensus.cc:740] T 1df93779209748b9860f6d893dd3de53 P 647a343a542d4a139fe106c19095129b [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:20:29.738863 16217 rpcz_store.cc:276] 0709 13:20:28.567520 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:29.725432 (+1157912us) service_pool.cc:224] Handling call
0709 13:20:29.725956 (+   524us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:29.726238 (+   282us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:29.726262 (+    24us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:20:29.726332 (+    70us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:20:29.726392 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:29.726472 (+    80us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:29.726614 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:20:29.718652 16226 rpcz_store.cc:276] 0709 13:20:28.528901 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:28.738196 (+209295us) service_pool.cc:224] Handling call
0709 13:20:29.668903 (+930707us) spinlock_profiling.cc:243] Waited 804 ms on lock 0x7b5400245254. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:20:29.669052 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1848349056}
I20260709 13:20:29.741087 16219 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57600 (request call id 1323) took 1165 ms. Trace:
I20260709 13:20:29.735392 19540 raft_consensus.cc:493] T e5ad4eb281564936a59f41e387a47156 P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:29.753114 16143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a9de347c711477e87c582d003f36925" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 11 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:29.753465 19540 raft_consensus.cc:3060] T e5ad4eb281564936a59f41e387a47156 P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:20:29.751775 16145 raft_consensus.cc:1240] T c0eab25ce4c64e018a041beeda2867db P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 8. Current term is 9. Ops: []
I20260709 13:20:29.753517 18599 consensus_queue.cc:260] T 1df93779209748b9860f6d893dd3de53 P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:29.755110 16298 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48684 (request call id 2299) took 2328 ms. Trace:
I20260709 13:20:29.755779 16145 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40844 (request call id 2172) took 2797 ms. Trace:
I20260709 13:20:29.756260 16145 rpcz_store.cc:276] 0709 13:20:26.958898 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:28.554932 (+1596034us) service_pool.cc:224] Handling call
0709 13:20:29.751414 (+1196482us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:29.754464 (+  3050us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:29.755512 (+  1048us) spinlock_profiling.cc:243] Waited 1.19 s on lock 0x7b54000735d0. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5e8ad 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:20:29.755682 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2739022208}
I20260709 13:20:29.756807 19840 raft_consensus.cc:3055] T b6f07842d3aa47858df5665874c86f3a P 647a343a542d4a139fe106c19095129b [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:20:29.757381 19840 raft_consensus.cc:740] T b6f07842d3aa47858df5665874c86f3a P 647a343a542d4a139fe106c19095129b [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:20:29.755573 16298 rpcz_store.cc:276] 0709 13:20:27.426521 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:27.776942 (+350421us) service_pool.cc:224] Handling call
0709 13:20:27.777230 (+   288us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:28.883055 (+1105825us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:29.754606 (+871551us) spinlock_profiling.cc:243] Waited 1.1 s on lock 0x7b540010de54. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:20:29.754937 (+   331us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2541303680}
I20260709 13:20:29.759482 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8638ee363273426db1393dd81e5c7b72" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:20:29.759755 16297 raft_consensus.cc:2468] T 8d4ab870a05347868956aa8b841b2981 P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 9.
I20260709 13:20:29.760349 16298 raft_consensus.cc:3060] T 8638ee363273426db1393dd81e5c7b72 P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:20:29.761278 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87ac0d1f7e8a419f9bcbc04aee0e067e" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:20:29.762605 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66ae176961b74813a58bc15c9c4c6f2a" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:20:29.754953 18599 raft_consensus.cc:3060] T 1df93779209748b9860f6d893dd3de53 P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:20:29.685767 19763 raft_consensus.cc:3060] T 3884f864a96d428785895e5463e5728f P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:20:29.747980 16144 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40830 (request call id 1326) took 2079 ms (client timeout 1885 ms). Trace:
I20260709 13:20:29.747758 19569 consensus_queue.cc:260] T 9c61a39fd4f84424b129c1a6f9ba404d P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:29.734988 19651 raft_consensus.cc:2764] T c4f65cdf00b748bb963ee6e81ef43b79 P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Leader pre-election decision vote started in defunct term 9: won
I20260709 13:20:29.399349 19414 raft_consensus.cc:515] T d2ab0ac8347641e6955e9bce7d90c1a2 P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:29.768357 19569 raft_consensus.cc:3060] T 9c61a39fd4f84424b129c1a6f9ba404d P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:20:29.770117 16292 raft_consensus.cc:2468] T d2ab0ac8347641e6955e9bce7d90c1a2 P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 8.
I20260709 13:20:29.746588 16115 leader_election.cc:304] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:20:29.771293 19664 raft_consensus.cc:493] T 23f82224f4564fea93572e8206bad1fe P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
W20260709 13:20:29.771468 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 70498.7us
I20260709 13:20:29.772111 19664 raft_consensus.cc:515] T 23f82224f4564fea93572e8206bad1fe P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:29.772616 16265 leader_election.cc:304] T 02838c86531d4f98bf58d1c22b8baf77 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 647a343a542d4a139fe106c19095129b
I20260709 13:20:29.773932 19529 consensus_queue.cc:1059] T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:20:29.774575 16265 leader_election.cc:304] T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:20:29.775080 19529 raft_consensus.cc:3055] T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:20:29.775595 19529 raft_consensus.cc:740] T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:20:29.776232 19345 raft_consensus.cc:2804] T 02838c86531d4f98bf58d1c22b8baf77 P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:20:29.776878 19345 raft_consensus.cc:493] T 02838c86531d4f98bf58d1c22b8baf77 P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:20:29.777498 19345 raft_consensus.cc:3060] T 02838c86531d4f98bf58d1c22b8baf77 P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:20:29.777369 19529 consensus_queue.cc:260] T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:29.779811 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74ca8140f35c4ae29e8a664f6bdfc829" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:29.780725 16293 raft_consensus.cc:2468] T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 8.
W20260709 13:20:29.781980 16189 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:20:29.751128 16192 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 440652us
I20260709 13:20:29.782410 19822 raft_consensus.cc:493] T 66f75124215f4ca6bef831e174a792d8 P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:29.751343 19794 raft_consensus.cc:2749] T ada5a004ba7c46819051e3ccec71ffdb P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:20:29.783293 19822 raft_consensus.cc:515] T 66f75124215f4ca6bef831e174a792d8 P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:29.783938 16224 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57604 (request call id 2361) took 1202 ms. Trace:
W20260709 13:20:29.784150 16191 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:29.784420 16224 rpcz_store.cc:276] 0709 13:20:28.581158 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:29.738550 (+1157392us) service_pool.cc:224] Handling call
0709 13:20:29.783219 (+ 44669us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:29.783579 (+   360us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:29.783613 (+    34us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:20:29.783657 (+    44us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:20:29.783694 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:29.783746 (+    52us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:29.783870 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:20:29.785652 16224 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "84f2da98864b4425bf7c0055951fdd4b"
dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
mode: GRACEFUL
new_leader_uuid: "90f3da57796b488c9087a2f523adf045"
 from {username='slave'} at 127.0.0.1:39144
I20260709 13:20:29.758690 19840 consensus_queue.cc:260] T b6f07842d3aa47858df5665874c86f3a P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:29.786002 19822 leader_election.cc:290] T 66f75124215f4ca6bef831e174a792d8 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:29.786237 16224 raft_consensus.cc:606] T 84f2da98864b4425bf7c0055951fdd4b P 1ca2725d290e47d68b1a4b609c9816ac [term 7 LEADER]: Received request to transfer leadership to TS 90f3da57796b488c9087a2f523adf045
W20260709 13:20:29.750191 16789 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:29.782444 19733 raft_consensus.cc:2749] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:20:29.787002 19840 raft_consensus.cc:3060] T b6f07842d3aa47858df5665874c86f3a P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:20:29.752388 19806 raft_consensus.cc:697] T c9944d5e4b8c4109af9955c4f08c9095 P cb82cc7bd6db46d980c75237a1402860 [term 9 LEADER]: Becoming Leader. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:20:29.789211 19568 raft_consensus.cc:493] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P cb82cc7bd6db46d980c75237a1402860 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:29.789884 19568 raft_consensus.cc:515] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P cb82cc7bd6db46d980c75237a1402860 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:29.791453 19823 consensus_queue.cc:1059] T 68a6838d82ef4cac83507f906139d90f P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:20:29.792598 16820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62fe3346cd3c42c4a55e14af9073bca4" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:20:29.792843 19407 raft_consensus.cc:3055] T 68a6838d82ef4cac83507f906139d90f P 1ca2725d290e47d68b1a4b609c9816ac [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:20:29.748708 16790 leader_election.cc:304] T 8c11074579b046d99e585c2818e18a2c P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:20:29.793063 16822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5718c623ebee4d2bb770f0f9ddb2f164" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:29.793335 19407 raft_consensus.cc:740] T 68a6838d82ef4cac83507f906139d90f P 1ca2725d290e47d68b1a4b609c9816ac [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:20:29.794004 16822 raft_consensus.cc:2468] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 8.
I20260709 13:20:29.794198 16296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4dc6b0541ca431ebc757beac00521f1" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
W20260709 13:20:29.794360 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 45830.8us
I20260709 13:20:29.795289 16296 raft_consensus.cc:2393] T e4dc6b0541ca431ebc757beac00521f1 P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 8: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
I20260709 13:20:29.795493 19673 raft_consensus.cc:2749] T 8c11074579b046d99e585c2818e18a2c P cb82cc7bd6db46d980c75237a1402860 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:20:29.793332 16820 raft_consensus.cc:3060] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:20:29.796294 16296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a878999c0024d2abe517484180290fc" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 9 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:29.796470 19673 consensus_queue.cc:1059] T 3a7ca61592e848ea9b0fb81a2ce33cdc P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:20:29.797176 16296 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48674 (request call id 2448) took 1223 ms. Trace:
I20260709 13:20:29.797569 16296 rpcz_store.cc:276] 0709 13:20:28.573665 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:29.795935 (+1222270us) service_pool.cc:224] Handling call
0709 13:20:29.797122 (+  1187us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:20:29.798070 19809 raft_consensus.cc:3055] T 3a7ca61592e848ea9b0fb81a2ce33cdc P cb82cc7bd6db46d980c75237a1402860 [term 10 LEADER]: Stepping down as leader of term 10
W20260709 13:20:29.764459 16144 rpcz_store.cc:269] 0709 13:20:27.668194 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:27.907022 (+238828us) service_pool.cc:224] Handling call
0709 13:20:29.747676 (+1840654us) spinlock_profiling.cc:243] Waited 1.29 s on lock 0x7b54000735d4. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:20:29.747881 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2973009280}
I20260709 13:20:29.794627 19407 consensus_queue.cc:260] T 68a6838d82ef4cac83507f906139d90f P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:29.800151 16816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee9ed67006144797b1de6ea8c2013a68" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:20:29.800917 16225 raft_consensus.cc:1275] T ecc1e363230043fe9fef390f0b979f20 P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:20:29.801620 16225 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33050 (request call id 615) took 1204 ms. Trace:
I20260709 13:20:29.802022 16225 rpcz_store.cc:276] 0709 13:20:28.597094 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:29.743214 (+1146120us) service_pool.cc:224] Handling call
0709 13:20:29.800533 (+ 57319us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:29.801362 (+   829us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:29.801565 (+   203us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:20:29.801133 16191 leader_election.cc:304] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
I20260709 13:20:29.801017 16816 raft_consensus.cc:2393] T ee9ed67006144797b1de6ea8c2013a68 P cb82cc7bd6db46d980c75237a1402860 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 7: Already voted for candidate cb82cc7bd6db46d980c75237a1402860 in this term.
I20260709 13:20:29.743414 16219 rpcz_store.cc:276] 0709 13:20:28.573572 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:29.727886 (+1154314us) service_pool.cc:224] Handling call
0709 13:20:29.728409 (+   523us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:29.736524 (+  8115us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:29.739419 (+  2895us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:20:29.781879 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 9 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:29.789789 19806 consensus_queue.cc:237] T c9944d5e4b8c4109af9955c4f08c9095 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.8, Last appended by leader: 9, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:29.793440 19568 leader_election.cc:290] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:29.803095 16825 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49896 (request call id 1046) took 1579 ms. Trace:
W20260709 13:20:29.820562 16116 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:29.821372 16296 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48674 (request call id 2443) took 1257 ms. Trace:
I20260709 13:20:29.806859 16151 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40844 (request call id 2169) took 2848 ms. Trace:
I20260709 13:20:29.806561 16295 raft_consensus.cc:1240] T 26354361e9b044cea1c2b86c4882fe92 P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 6. Current term is 7. Ops: []
I20260709 13:20:29.798445 16790 leader_election.cc:304] T 8638ee363273426db1393dd81e5c7b72 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 90f3da57796b488c9087a2f523adf045
I20260709 13:20:29.823558 16296 rpcz_store.cc:276] 0709 13:20:28.552212 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:29.798245 (+1246033us) service_pool.cc:224] Handling call
0709 13:20:29.798694 (+   449us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:29.799051 (+   357us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:29.799075 (+    24us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:20:29.799136 (+    61us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:20:29.809856 (+ 10720us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:29.809919 (+    63us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:29.810012 (+    93us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:20:29.823875 16151 rpcz_store.cc:276] 0709 13:20:26.957785 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:28.499702 (+1541917us) service_pool.cc:224] Handling call
0709 13:20:28.499976 (+   274us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:28.944393 (+444417us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:29.806350 (+861957us) spinlock_profiling.cc:243] Waited 403 ms on lock 0x7b540018f954. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:20:29.806653 (+   303us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":926306560}
I20260709 13:20:29.826869 16816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8638ee363273426db1393dd81e5c7b72" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:20:29.827764 16816 raft_consensus.cc:3060] T 8638ee363273426db1393dd81e5c7b72 P cb82cc7bd6db46d980c75237a1402860 [term 7 FOLLOWER]: Advancing to term 9
I20260709 13:20:29.831513 16220 raft_consensus.cc:1240] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 8. Current term is 9. Ops: [8.7-8.7]
I20260709 13:20:29.832309 16220 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57604 (request call id 2359) took 1269 ms. Trace:
I20260709 13:20:29.832670 16220 rpcz_store.cc:276] 0709 13:20:28.562965 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:29.714901 (+1151936us) service_pool.cc:224] Handling call
0709 13:20:29.740144 (+ 25243us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:20:29.831975 (+ 91831us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:29.832229 (+   254us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:20:29.834478 16218 raft_consensus.cc:1275] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 10 index: 10. (term mismatch)
I20260709 13:20:29.834952 16218 pending_rounds.cc:85] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac: Aborting all ops after (but not including) 9
I20260709 13:20:29.835304 16218 pending_rounds.cc:107] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac: Aborting uncommitted NO_OP operation due to leader change: 9.10
I20260709 13:20:29.835577 16218 raft_consensus.cc:2889] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:20:29.835997 16790 leader_election.cc:304] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:20:29.837532 19802 raft_consensus.cc:2749] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:20:29.839372 16822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdbd78acb4cf46298e0de8ba0d130e19" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:20:29.840197 19529 raft_consensus.cc:3060] T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:20:29.840983 16822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66ae176961b74813a58bc15c9c4c6f2a" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:20:29.841784 16822 raft_consensus.cc:3060] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:20:29.798532 19809 raft_consensus.cc:740] T 3a7ca61592e848ea9b0fb81a2ce33cdc P cb82cc7bd6db46d980c75237a1402860 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:20:29.847230 16150 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40830 (request call id 1325) took 2206 ms. Trace:
I20260709 13:20:29.847759 16150 rpcz_store.cc:276] 0709 13:20:27.649901 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:28.513577 (+863676us) service_pool.cc:224] Handling call
0709 13:20:29.846726 (+1333149us) spinlock_profiling.cc:243] Waited 113 ms on lock 0x7b540020ba54. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:20:29.847001 (+   275us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":260802944}
W20260709 13:20:29.849166 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 48494.1us
I20260709 13:20:29.850489 19197 consensus_queue.cc:1059] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 647a343a542d4a139fe106c19095129b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: INVALID_TERM, Last received: 7.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:20:29.850993 16191 leader_election.cc:304] T 19baa7bc367844c288bca8f81439c413 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
I20260709 13:20:29.851909 19197 raft_consensus.cc:3055] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 647a343a542d4a139fe106c19095129b [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:20:29.852356 19197 raft_consensus.cc:740] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 647a343a542d4a139fe106c19095129b [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:20:29.852572 19666 raft_consensus.cc:2804] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:20:29.852769 19751 raft_consensus.cc:2749] T 19baa7bc367844c288bca8f81439c413 P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:20:29.853184 19666 raft_consensus.cc:493] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:29.853677 19666 raft_consensus.cc:3060] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:20:29.853399 19197 consensus_queue.cc:260] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:29.855369 16293 raft_consensus.cc:2468] T ecc1e363230043fe9fef390f0b979f20 P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 8.
I20260709 13:20:29.855912 16293 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48674 (request call id 2445) took 1295 ms. Trace:
I20260709 13:20:29.856208 16293 rpcz_store.cc:276] 0709 13:20:28.560895 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:29.781500 (+1220605us) service_pool.cc:224] Handling call
0709 13:20:29.855853 (+ 74353us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:20:29.857466 16293 raft_consensus.cc:1240] T 9e65676b21b748728ad39668cfe722ec P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 8. Current term is 10. Ops: []
I20260709 13:20:29.858074 16293 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40474 (request call id 545) took 1178 ms. Trace:
I20260709 13:20:29.858397 16293 rpcz_store.cc:276] 0709 13:20:28.680114 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:29.856869 (+1176755us) service_pool.cc:224] Handling call
0709 13:20:29.857255 (+   386us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:29.857823 (+   568us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:29.857995 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:20:29.859521 19414 leader_election.cc:290] T d2ab0ac8347641e6955e9bce7d90c1a2 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:29.821348 16825 rpcz_store.cc:276] 0709 13:20:28.223243 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:28.223405 (+   162us) service_pool.cc:224] Handling call
0709 13:20:28.242945 (+ 19540us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:29.681067 (+1438122us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:29.802655 (+121588us) spinlock_profiling.cc:243] Waited 1.13 s on lock 0x7b54007d0354. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:20:29.802908 (+   253us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2598760704}
I20260709 13:20:29.865550 16825 raft_consensus.cc:1240] T ee9ed67006144797b1de6ea8c2013a68 P cb82cc7bd6db46d980c75237a1402860 [term 7 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 6. Current term is 7. Ops: [6.7-6.7]
I20260709 13:20:29.817036 19664 leader_election.cc:290] T 23f82224f4564fea93572e8206bad1fe P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:29.805835 19732 leader_election.cc:290] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 8 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:29.847246 19809 consensus_queue.cc:260] T 3a7ca61592e848ea9b0fb81a2ce33cdc P cb82cc7bd6db46d980c75237a1402860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:29.867753 16300 raft_consensus.cc:1275] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 13:20:29.868307 19809 raft_consensus.cc:3060] T 3a7ca61592e848ea9b0fb81a2ce33cdc P cb82cc7bd6db46d980c75237a1402860 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:20:29.868479 16300 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40474 (request call id 547) took 1080 ms. Trace:
I20260709 13:20:29.868911 16300 rpcz_store.cc:276] 0709 13:20:28.788173 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:29.866741 (+1078568us) service_pool.cc:224] Handling call
0709 13:20:29.867256 (+   515us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:29.868209 (+   953us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:29.868418 (+   209us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:20:29.870347 16300 raft_consensus.cc:1240] T 2794af5b99984efebdee564035841cb0 P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 6. Current term is 7. Ops: []
I20260709 13:20:29.870920 16300 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40474 (request call id 548) took 1068 ms. Trace:
I20260709 13:20:29.858839 19707 raft_consensus.cc:3060] T 5a96373bb0e14dce8a266b2fc6c58507 P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:20:29.872467 16818 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49892 (request call id 520) took 1082 ms. Trace:
I20260709 13:20:29.872888 16818 rpcz_store.cc:276] 0709 13:20:28.790289 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:29.099716 (+309427us) service_pool.cc:224] Handling call
0709 13:20:29.872247 (+772531us) spinlock_profiling.cc:243] Waited 423 ms on lock 0x7b5400771354. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:20:29.872408 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":972418816}
I20260709 13:20:29.874125 16192 leader_election.cc:304] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:20:29.875238 16300 rpcz_store.cc:276] 0709 13:20:28.802640 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:29.869728 (+1067088us) service_pool.cc:224] Handling call
0709 13:20:29.870077 (+   349us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:29.870689 (+   612us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:29.870869 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:20:29.799113 19816 raft_consensus.cc:515] T 5a959dbecc2648eb85d1cc550f5a3245 P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:29.878408 19816 leader_election.cc:290] T 5a959dbecc2648eb85d1cc550f5a3245 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 7 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:29.871632 16295 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40474 (request call id 542) took 1183 ms. Trace:
I20260709 13:20:29.881816 16295 rpcz_store.cc:276] 0709 13:20:28.642472 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:29.805700 (+1163228us) service_pool.cc:224] Handling call
0709 13:20:29.806183 (+   483us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:29.825223 (+ 19040us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:29.825479 (+   256us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:20:29.883193 16295 raft_consensus.cc:1240] T ada5a004ba7c46819051e3ccec71ffdb P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 8. Current term is 9. Ops: []
W20260709 13:20:29.884711 16821 consensus_meta.cc:287] T 806d7690c2b644959152074bb2cab0a7 P cb82cc7bd6db46d980c75237a1402860: Time spent flushing consensus metadata: real 0.524s	user 0.000s	sys 0.012s
I20260709 13:20:29.884874 16295 raft_consensus.cc:1240] T 8d4ab870a05347868956aa8b841b2981 P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 8. Current term is 9. Ops: []
I20260709 13:20:29.885722 16821 raft_consensus.cc:1275] T 806d7690c2b644959152074bb2cab0a7 P cb82cc7bd6db46d980c75237a1402860 [term 7 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 7. (term mismatch)
I20260709 13:20:29.886049 16821 pending_rounds.cc:85] T 806d7690c2b644959152074bb2cab0a7 P cb82cc7bd6db46d980c75237a1402860: Aborting all ops after (but not including) 6
I20260709 13:20:29.886296 16821 pending_rounds.cc:107] T 806d7690c2b644959152074bb2cab0a7 P cb82cc7bd6db46d980c75237a1402860: Aborting uncommitted CHANGE_CONFIG_OP operation due to leader change: 6.7
I20260709 13:20:29.886546 16821 raft_consensus.cc:2917] T 806d7690c2b644959152074bb2cab0a7 P cb82cc7bd6db46d980c75237a1402860 [term 7 FOLLOWER]: Aborting config change with OpId 6.7: Aborted: Op aborted by new leader
I20260709 13:20:29.886906 16821 raft_consensus.cc:2889] T 806d7690c2b644959152074bb2cab0a7 P cb82cc7bd6db46d980c75237a1402860 [term 7 LEARNER]: CHANGE_CONFIG_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:20:29.887478 19822 consensus_queue.cc:1059] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:20:29.854696 16790 leader_election.cc:304] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:20:29.910640 16298 raft_consensus.cc:2468] T 8638ee363273426db1393dd81e5c7b72 P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 9.
I20260709 13:20:29.914337 19702 raft_consensus.cc:2804] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:20:29.915921 16191 leader_election.cc:304] T 5a959dbecc2648eb85d1cc550f5a3245 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 90f3da57796b488c9087a2f523adf045
I20260709 13:20:29.916297 16296 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40474 (request call id 544) took 1236 ms. Trace:
I20260709 13:20:29.916785 19822 consensus_queue.cc:1059] T ee9ed67006144797b1de6ea8c2013a68 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:20:29.919327 16217 raft_consensus.cc:1275] T 66f75124215f4ca6bef831e174a792d8 P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:20:29.886557 16295 raft_consensus.cc:1240] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 9. Current term is 10. Ops: []
I20260709 13:20:29.955910 16217 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57604 (request call id 2365) took 1303 ms. Trace:
I20260709 13:20:29.920382 16821 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49892 (request call id 522) took 1063 ms. Trace:
I20260709 13:20:29.920784 19532 raft_consensus.cc:515] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:29.923208 19700 raft_consensus.cc:3055] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:20:29.923794 16150 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c4f65cdf00b748bb963ee6e81ef43b79"
dest_uuid: "647a343a542d4a139fe106c19095129b"
mode: GRACEFUL
new_leader_uuid: "90f3da57796b488c9087a2f523adf045"
 from {username='slave'} at 127.0.0.1:40290
I20260709 13:20:29.956559 16821 rpcz_store.cc:276] 0709 13:20:28.857122 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:29.359806 (+502684us) service_pool.cc:224] Handling call
0709 13:20:29.360183 (+   377us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:29.920060 (+559877us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:29.920303 (+   243us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:20:29.959190 16150 raft_consensus.cc:606] T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b [term 10 FOLLOWER]: Received request to transfer leadership to TS 90f3da57796b488c9087a2f523adf045
I20260709 13:20:29.959722 16150 raft_consensus.cc:614] T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b [term 10 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:20:29.928170 16149 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46224 (request call id 506) took 1608 ms. Trace:
I20260709 13:20:29.929580 16297 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40474 (request call id 543) took 1263 ms. Trace:
W20260709 13:20:29.931809 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 157419us
I20260709 13:20:29.932550 19657 raft_consensus.cc:2749] T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:20:29.932919 19672 consensus_meta.cc:287] T af57996a8be84d398f0f023e8d08ef77 P cb82cc7bd6db46d980c75237a1402860: Time spent flushing consensus metadata: real 0.500s	user 0.017s	sys 0.000s
I20260709 13:20:29.933498 16820 raft_consensus.cc:2468] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 12.
I20260709 13:20:29.933633 16294 raft_consensus.cc:2468] T cdbd78acb4cf46298e0de8ba0d130e19 P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 9.
I20260709 13:20:29.960539 16149 rpcz_store.cc:276] 0709 13:20:28.319924 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:28.437134 (+117210us) service_pool.cc:224] Handling call
0709 13:20:29.927595 (+1490461us) spinlock_profiling.cc:243] Waited 467 ms on lock 0x7b540020c454. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:20:29.927920 (+   325us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1073786752}
I20260709 13:20:29.965749 16291 raft_consensus.cc:1240] T cdbd78acb4cf46298e0de8ba0d130e19 P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 8. Current term is 9. Ops: []
I20260709 13:20:29.960672 16297 rpcz_store.cc:276] 0709 13:20:28.665902 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:29.817338 (+1151436us) service_pool.cc:224] Handling call
0709 13:20:29.894526 (+ 77188us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:29.927281 (+ 32755us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:29.927320 (+    39us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:20:29.927403 (+    83us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:20:29.927444 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:29.927525 (+    81us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:29.929300 (+  1775us) spinlock_profiling.cc:243] Waited 61.3 ms on lock 0x7b54000a9e50. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5e8ad 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:20:29.929469 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":140896128}
I20260709 13:20:29.934757 16299 raft_consensus.cc:2410] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac for term 8 because replica has last-logged OpId of term: 7 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 5.
I20260709 13:20:29.936614 16218 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33050 (request call id 611) took 1354 ms. Trace:
I20260709 13:20:29.966646 16218 rpcz_store.cc:276] 0709 13:20:28.582690 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:29.743102 (+1160412us) service_pool.cc:224] Handling call
0709 13:20:29.746608 (+  3506us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:29.836205 (+ 89597us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:29.936330 (+100125us) spinlock_profiling.cc:243] Waited 87.3 ms on lock 0x7b54002c8b54. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:20:29.936536 (+   206us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":200878848}
I20260709 13:20:29.937798 19418 raft_consensus.cc:493] T d16dfad0c3b540e5b4b86492ddf87208 P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:29.938647 16117 leader_election.cc:304] T ee9ed67006144797b1de6ea8c2013a68 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860
I20260709 13:20:29.942246 16226 raft_consensus.cc:1275] T 746aabf856ab401c9ef5e859371e7b67 P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:20:29.947145 16825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66f75124215f4ca6bef831e174a792d8" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 10 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:29.970805 16226 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57604 (request call id 2368) took 1317 ms. Trace:
I20260709 13:20:29.948314 16219 raft_consensus.cc:1240] T 1df93779209748b9860f6d893dd3de53 P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 8. Current term is 9. Ops: []
I20260709 13:20:29.971050 16825 raft_consensus.cc:2410] T 66f75124215f4ca6bef831e174a792d8 P cb82cc7bd6db46d980c75237a1402860 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac for term 10 because replica has last-logged OpId of term: 7 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
I20260709 13:20:29.949425 16816 raft_consensus.cc:2468] T 8638ee363273426db1393dd81e5c7b72 P cb82cc7bd6db46d980c75237a1402860 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 9.
W20260709 13:20:29.949576 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 95065.3us
I20260709 13:20:29.950166 16822 raft_consensus.cc:2468] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 11.
I20260709 13:20:29.943912 16190 leader_election.cc:304] T b6f07842d3aa47858df5665874c86f3a P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:20:29.956261 16217 rpcz_store.cc:276] 0709 13:20:28.652490 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:29.741571 (+1089081us) service_pool.cc:224] Handling call
0709 13:20:29.892143 (+150572us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:29.955675 (+ 63532us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:29.955859 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1920}
I20260709 13:20:29.954519 16296 rpcz_store.cc:276] 0709 13:20:28.679703 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:29.894355 (+1214652us) service_pool.cc:224] Handling call
0709 13:20:29.915647 (+ 21292us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:29.915898 (+   251us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:29.915951 (+    53us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:20:29.916011 (+    60us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:20:29.916044 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:29.916097 (+    53us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:29.916230 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:20:29.956419 16295 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48684 (request call id 2321) took 1013 ms. Trace:
I20260709 13:20:29.971256 16226 rpcz_store.cc:276] 0709 13:20:28.653262 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:29.745464 (+1092202us) service_pool.cc:224] Handling call
0709 13:20:29.941400 (+195936us) spinlock_profiling.cc:243] Waited 65.8 ms on lock 0x7b5800010b98. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69ad8fadb1 00007f69ada3ab71 00007f69ada3aa22 00007f69b6953c09 00007f69b5aadee0 00007f69b5b770ab 00007f69b5b77131 00007f69b5b38fc0 00007f69b5b23929 00007f69b5b233d2 00007f69af09abd4
0709 13:20:29.941717 (+   317us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:29.970542 (+ 28825us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:29.970750 (+   208us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":151271296}
I20260709 13:20:29.974116 16295 rpcz_store.cc:276] 0709 13:20:28.942524 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:29.885828 (+943304us) service_pool.cc:224] Handling call
0709 13:20:29.886319 (+   491us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:29.956087 (+ 69768us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:29.956348 (+   261us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:20:29.953966 19673 raft_consensus.cc:2804] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:20:29.957280 19702 raft_consensus.cc:493] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:29.978010 16192 leader_election.cc:304] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:20:29.983762 16143 raft_consensus.cc:2393] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 11: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
W20260709 13:20:30.030514 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 49954.3us
I20260709 13:20:29.964826 19700 raft_consensus.cc:740] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
W20260709 13:20:30.034236 16788 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:20:30.029661 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 53851us
I20260709 13:20:30.026988 16218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a959dbecc2648eb85d1cc550f5a3245" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:20:29.973367 19568 raft_consensus.cc:2764] T 8638ee363273426db1393dd81e5c7b72 P cb82cc7bd6db46d980c75237a1402860 [term 9 FOLLOWER]: Leader pre-election decision vote started in defunct term 7: won
I20260709 13:20:30.040874 16224 raft_consensus.cc:1240] T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac [term 8 LEADER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 7. Current term is 8. Ops: []
I20260709 13:20:30.041515 16219 raft_consensus.cc:3055] T c2f5f89186204242bf377f7c42b2e2da P 1ca2725d290e47d68b1a4b609c9816ac [term 9 LEADER]: Stepping down as leader of term 9
W20260709 13:20:30.038713 16115 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:29.951344 19540 raft_consensus.cc:515] T e5ad4eb281564936a59f41e387a47156 P 647a343a542d4a139fe106c19095129b [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:20:30.041010 16266 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:30.041953 16219 raft_consensus.cc:740] T c2f5f89186204242bf377f7c42b2e2da P 1ca2725d290e47d68b1a4b609c9816ac [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:20:29.954187 19695 raft_consensus.cc:515] T 57a744f5aaf944678a63469cbf514015 P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:29.985569 16823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5d59407eb545b895d8d922e0dd69fd" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
W20260709 13:20:30.058996 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 115243us
W20260709 13:20:30.061251 16117 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 122777us
I20260709 13:20:30.066403 16298 raft_consensus.cc:1275] T a378d7f796f94615b3b347119a9880fd P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:20:29.978382 19673 raft_consensus.cc:493] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:30.068571 16218 raft_consensus.cc:3060] T 5a959dbecc2648eb85d1cc550f5a3245 P 1ca2725d290e47d68b1a4b609c9816ac [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:20:30.040074 19702 raft_consensus.cc:3060] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:20:30.034807 19754 consensus_queue.cc:1059] T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.106s
W20260709 13:20:30.076604 16192 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 102539us
W20260709 13:20:30.076656 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 160895us
I20260709 13:20:29.974017 19707 consensus_queue.cc:1048] T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.089s
I20260709 13:20:29.978442 19672 raft_consensus.cc:515] T af57996a8be84d398f0f023e8d08ef77 P cb82cc7bd6db46d980c75237a1402860 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:30.068285 19667 raft_consensus.cc:3055] T ee9ed67006144797b1de6ea8c2013a68 P 1ca2725d290e47d68b1a4b609c9816ac [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:20:30.056784 19575 raft_consensus.cc:2749] T ee9ed67006144797b1de6ea8c2013a68 P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:20:30.097086 16825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9944d5e4b8c4109af9955c4f08c9095" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:30.097391 16823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e9b2bf7011f4ae688a79d7bb5a447b4" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:20:30.142802 16823 raft_consensus.cc:2393] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 7: Already voted for candidate cb82cc7bd6db46d980c75237a1402860 in this term.
I20260709 13:20:30.055950 19345 raft_consensus.cc:515] T 02838c86531d4f98bf58d1c22b8baf77 P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:30.053263 19418 raft_consensus.cc:515] T d16dfad0c3b540e5b4b86492ddf87208 P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:30.088814 19751 raft_consensus.cc:2804] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:20:30.087832 19673 raft_consensus.cc:3060] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:20:30.105794 16821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7ca61592e848ea9b0fb81a2ce33cdc" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:20:30.108444 16190 leader_election.cc:304] T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:20:29.979242 19816 consensus_queue.cc:1059] T b87147731a68400e8cce9b879daa7c5d P 647a343a542d4a139fe106c19095129b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.115s
I20260709 13:20:30.110138 16192 leader_election.cc:304] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:20:30.138695 16820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26354361e9b044cea1c2b86c4882fe92" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:20:30.099489 16219 consensus_queue.cc:260] T c2f5f89186204242bf377f7c42b2e2da P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:20:30.140142 16116 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:20:30.144381 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 116546us
I20260709 13:20:30.095350 19802 raft_consensus.cc:493] T 9e65676b21b748728ad39668cfe722ec P cb82cc7bd6db46d980c75237a1402860 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:30.149013 16822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66ae176961b74813a58bc15c9c4c6f2a" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:20:30.154290 16822 raft_consensus.cc:2393] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 11: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
W20260709 13:20:30.149814 16266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 74281.8us
W20260709 13:20:30.161371 16116 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:30.158968 16816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e65676b21b748728ad39668cfe722ec" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:20:30.162392 16823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f82224f4564fea93572e8206bad1fe" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:30.166507 16817 raft_consensus.cc:1240] T af57996a8be84d398f0f023e8d08ef77 P cb82cc7bd6db46d980c75237a1402860 [term 10 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 9. Current term is 10. Ops: [9.8-9.8]
I20260709 13:20:30.139523 19667 raft_consensus.cc:740] T ee9ed67006144797b1de6ea8c2013a68 P 1ca2725d290e47d68b1a4b609c9816ac [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:20:30.176590 16144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806d7690c2b644959152074bb2cab0a7" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:30.177747 16144 raft_consensus.cc:2393] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 8: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
I20260709 13:20:30.169840 16151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18ad7e3af5504571a69d10c00750a01c" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:30.178622 16299 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48684 (request call id 2314) took 1252 ms. Trace:
I20260709 13:20:30.179039 16825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ab0ac8347641e6955e9bce7d90c1a2" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:30.165383 16818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2794af5b99984efebdee564035841cb0" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
W20260709 13:20:30.180464 16116 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:30.181607 16219 raft_consensus.cc:3060] T c2f5f89186204242bf377f7c42b2e2da P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:20:30.170382 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6f07842d3aa47858df5665874c86f3a" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:20:30.179415 16299 rpcz_store.cc:276] 0709 13:20:28.925822 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:29.312012 (+386190us) service_pool.cc:224] Handling call
0709 13:20:30.178377 (+866365us) spinlock_profiling.cc:243] Waited 237 ms on lock 0x7b5400240254. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:20:30.178558 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":544033024}
I20260709 13:20:30.195221 19671 raft_consensus.cc:2804] T b6f07842d3aa47858df5665874c86f3a P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:20:30.195701 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31969392ccdb43b491911a319e8bc7e1" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:20:30.196234 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f82224f4564fea93572e8206bad1fe" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:20:30.196573 16299 raft_consensus.cc:3060] T 31969392ccdb43b491911a319e8bc7e1 P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:20:30.196921 16298 raft_consensus.cc:3060] T 23f82224f4564fea93572e8206bad1fe P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:20:30.196588 16142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1df93779209748b9860f6d893dd3de53" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:20:30.198735 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a96373bb0e14dce8a266b2fc6c58507" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:20:30.199842 16148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62fe3346cd3c42c4a55e14af9073bca4" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:20:30.200543 16150 raft_consensus.cc:2410] T 5a96373bb0e14dce8a266b2fc6c58507 P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac for term 8 because replica has last-logged OpId of term: 7 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 3.
I20260709 13:20:30.200704 16148 raft_consensus.cc:2368] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac for earlier term 11. Current term is 12.
I20260709 13:20:30.201537 16147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87ac0d1f7e8a419f9bcbc04aee0e067e" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:20:30.201808 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d4ab870a05347868956aa8b841b2981" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:30.201865 16148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5718c623ebee4d2bb770f0f9ddb2f164" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:30.202670 16148 raft_consensus.cc:2468] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 8.
I20260709 13:20:30.203081 19789 consensus_queue.cc:1048] T 746aabf856ab401c9ef5e859371e7b67 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.106s
I20260709 13:20:30.203361 16265 leader_election.cc:304] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 647a343a542d4a139fe106c19095129b
I20260709 13:20:30.203655 16150 raft_consensus.cc:1240] T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 6. Current term is 7. Ops: [6.8-6.8]
W20260709 13:20:30.204973 16189 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:30.164692 19785 consensus_queue.cc:1048] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 3, Time since last communication: 0.034s
I20260709 13:20:30.138887 19696 consensus_queue.cc:1059] T 23f82224f4564fea93572e8206bad1fe P 647a343a542d4a139fe106c19095129b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.051s
I20260709 13:20:30.189136 19532 leader_election.cc:290] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 11 election: Requested vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:30.172358 19802 raft_consensus.cc:515] T 9e65676b21b748728ad39668cfe722ec P cb82cc7bd6db46d980c75237a1402860 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:30.166193 19700 consensus_queue.cc:260] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:30.207836 16145 raft_consensus.cc:1240] T ee9ed67006144797b1de6ea8c2013a68 P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 6. Current term is 7. Ops: []
W20260709 13:20:30.212477 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 61731.5us
I20260709 13:20:30.251745 16816 raft_consensus.cc:3060] T 9e65676b21b748728ad39668cfe722ec P cb82cc7bd6db46d980c75237a1402860 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:20:30.213274 16150 raft_consensus.cc:1240] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [term 10 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 9. Current term is 10. Ops: []
I20260709 13:20:30.217363 16299 raft_consensus.cc:2468] T 31969392ccdb43b491911a319e8bc7e1 P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 8.
W20260709 13:20:30.253597 16789 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:20:30.224939 16117 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:20:30.254218 16789 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:30.225209 16817 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49896 (request call id 1058) took 1335 ms. Trace:
I20260709 13:20:30.255573 16817 rpcz_store.cc:276] 0709 13:20:28.889301 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:29.650735 (+761434us) service_pool.cc:224] Handling call
0709 13:20:29.651372 (+   637us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:20:30.166961 (+515589us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:30.224911 (+ 57950us) spinlock_profiling.cc:243] Waited 501 ms on lock 0x7b54007c01d4. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:20:30.225143 (+   232us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1152465664}
W20260709 13:20:30.235214 16116 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:20:30.251225 16189 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:30.237614 16151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e9b2bf7011f4ae688a79d7bb5a447b4" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:30.239879 16143 raft_consensus.cc:1240] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 10. Current term is 11. Ops: []
I20260709 13:20:30.241750 16220 raft_consensus.cc:1240] T 5a96373bb0e14dce8a266b2fc6c58507 P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 7. Current term is 8. Ops: [7.3-7.3]
I20260709 13:20:30.254688 16117 leader_election.cc:304] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac
I20260709 13:20:30.207834 16823 raft_consensus.cc:2393] T 23f82224f4564fea93572e8206bad1fe P cb82cc7bd6db46d980c75237a1402860 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 8: Already voted for candidate cb82cc7bd6db46d980c75237a1402860 in this term.
I20260709 13:20:30.207129 16265 leader_election.cc:304] T ecc1e363230043fe9fef390f0b979f20 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:20:30.211815 16291 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40474 (request call id 541) took 1569 ms. Trace:
I20260709 13:20:30.273308 16291 rpcz_store.cc:276] 0709 13:20:28.642091 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:29.799529 (+1157438us) service_pool.cc:224] Handling call
0709 13:20:29.965522 (+165993us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:29.966256 (+   734us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:30.211571 (+245315us) spinlock_profiling.cc:243] Waited 166 ms on lock 0x7b54002e1b50. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5e8ad 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:20:30.211712 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":380907904}
I20260709 13:20:30.243572 16144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87ac0d1f7e8a419f9bcbc04aee0e067e" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:30.188934 16266 leader_election.cc:304] T 8c11074579b046d99e585c2818e18a2c P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:20:30.218425 16146 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46224 (request call id 507) took 1898 ms. Trace:
I20260709 13:20:30.280503 16149 raft_consensus.cc:2468] T b6f07842d3aa47858df5665874c86f3a P 647a343a542d4a139fe106c19095129b [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 10.
I20260709 13:20:30.265149 16145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4dc6b0541ca431ebc757beac00521f1" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:20:30.281939 16145 raft_consensus.cc:2410] T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac for term 8 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
W20260709 13:20:30.292740 16791 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:30.294739 16298 raft_consensus.cc:2468] T 23f82224f4564fea93572e8206bad1fe P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 8.
I20260709 13:20:30.300958 16296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87147731a68400e8cce9b879daa7c5d" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
W20260709 13:20:30.250422 19846 log.cc:927] Time spent T af57996a8be84d398f0f023e8d08ef77 P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 0.090s	user 0.000s	sys 0.000s
I20260709 13:20:30.212001 19695 leader_election.cc:290] T 57a744f5aaf944678a63469cbf514015 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 8 election: Requested vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:30.207466 19414 raft_consensus.cc:2804] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:20:30.236688 15046 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:20:30.227397 19407 raft_consensus.cc:3060] T 68a6838d82ef4cac83507f906139d90f P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:20:30.313618 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806d7690c2b644959152074bb2cab0a7" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:30.314544 16148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "829b4ad8c5a54eababeea1181b960d4b" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:20:30.234933 19802 leader_election.cc:290] T 9e65676b21b748728ad39668cfe722ec P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:30.216428 19672 leader_election.cc:290] T af57996a8be84d398f0f023e8d08ef77 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 10 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:20:30.328677 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31969392ccdb43b491911a319e8bc7e1" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:20:30.319599 16292 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40474 (request call id 551) took 1316 ms. Trace:
I20260709 13:20:30.249281 19699 raft_consensus.cc:2804] T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:20:30.297021 19657 raft_consensus.cc:2749] T ecc1e363230043fe9fef390f0b979f20 P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:20:30.243985 19197 raft_consensus.cc:3060] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:20:30.230882 19345 leader_election.cc:290] T 02838c86531d4f98bf58d1c22b8baf77 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 9 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:20:30.219734 19823 raft_consensus.cc:2804] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:20:30.360118 19672 consensus_queue.cc:1048] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:20:30.248723 19700 raft_consensus.cc:3060] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:20:30.321058 16222 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57604 (request call id 2369) took 1526 ms. Trace:
I20260709 13:20:30.314807 16218 raft_consensus.cc:2468] T 5a959dbecc2648eb85d1cc550f5a3245 P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 7.
I20260709 13:20:30.361840 19672 consensus_queue.cc:1059] T 2794af5b99984efebdee564035841cb0 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:20:30.231771 19789 raft_consensus.cc:3055] T 23f82224f4564fea93572e8206bad1fe P 647a343a542d4a139fe106c19095129b [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:20:30.328455 16191 leader_election.cc:304] T ecc1e363230043fe9fef390f0b979f20 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: cb82cc7bd6db46d980c75237a1402860
I20260709 13:20:30.232209 19540 leader_election.cc:290] T e5ad4eb281564936a59f41e387a47156 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 10 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:30.236399 19816 raft_consensus.cc:493] T 829b4ad8c5a54eababeea1181b960d4b P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:30.232347 19671 raft_consensus.cc:697] T b6f07842d3aa47858df5665874c86f3a P 1ca2725d290e47d68b1a4b609c9816ac [term 10 LEADER]: Becoming Leader. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:20:30.240067 19754 raft_consensus.cc:3055] T b87147731a68400e8cce9b879daa7c5d P 647a343a542d4a139fe106c19095129b [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:20:30.343817 19699 raft_consensus.cc:493] T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:30.331279 16149 raft_consensus.cc:2368] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 for earlier term 7. Current term is 8.
I20260709 13:20:30.249066 19760 raft_consensus.cc:3055] T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:20:30.280217 16146 rpcz_store.cc:276] 0709 13:20:28.320304 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:29.028501 (+708197us) service_pool.cc:224] Handling call
0709 13:20:29.847005 (+818504us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:20:29.847709 (+   704us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:20:29.848051 (+   342us) log.cc:844] Serialized 14 byte log entry
0709 13:20:30.036344 (+188293us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:20:30.057213 (+ 20869us) log.cc:844] Serialized 26 byte log entry
0709 13:20:30.057403 (+   190us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:20:30.057452 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:30.061506 (+  4054us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:20:30.178953 (+117447us) raft_consensus.cc:1685] finished
0709 13:20:30.178973 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:30.218090 (+ 39117us) spinlock_profiling.cc:243] Waited 818 ms on lock 0x7b540020ba50. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5e8ad 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:20:30.218237 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":48122,"prepare.queue_time_us":716,"prepare.run_cpu_time_us":63,"prepare.run_wall_time_us":64,"raft.queue_time_us":42531,"raft.run_cpu_time_us":316,"raft.run_wall_time_us":5898,"spinlock_wait_cycles":1881073536,"thread_start_us":167617,"threads_started":1,"wal-append.queue_time_us":242852}
I20260709 13:20:30.335592 16145 raft_consensus.cc:1240] T 9e65676b21b748728ad39668cfe722ec P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 8. Current term is 9. Ops: [8.9-8.9]
I20260709 13:20:30.336444 16144 raft_consensus.cc:1240] T 26354361e9b044cea1c2b86c4882fe92 P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 6. Current term is 7. Ops: []
I20260709 13:20:30.339605 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d16dfad0c3b540e5b4b86492ddf87208" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:30.220752 19418 leader_election.cc:290] T d16dfad0c3b540e5b4b86492ddf87208 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
W20260709 13:20:30.321631 16266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 157689us
I20260709 13:20:30.343281 16151 raft_consensus.cc:2468] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 6.
I20260709 13:20:30.361189 16222 rpcz_store.cc:276] 0709 13:20:28.776429 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:29.749404 (+972975us) service_pool.cc:224] Handling call
0709 13:20:29.749877 (+   473us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:29.750367 (+   490us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:20:29.953891 (+203524us) log.cc:844] Serialized 14 byte log entry
0709 13:20:30.302199 (+348308us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:20:30.302254 (+    55us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:20:30.302407 (+   153us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:30.302469 (+    62us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:30.302579 (+   110us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8182,"prepare.queue_time_us":329701,"prepare.run_cpu_time_us":76,"prepare.run_wall_time_us":62,"thread_start_us":454187,"threads_started":2,"wal-append.queue_time_us":343004}
I20260709 13:20:30.207971 19726 consensus_queue.cc:1048] T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.173s
I20260709 13:20:30.353619 16821 raft_consensus.cc:2410] T 3a7ca61592e848ea9b0fb81a2ce33cdc P cb82cc7bd6db46d980c75237a1402860 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 for term 11 because replica has last-logged OpId of term: 10 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 9.
I20260709 13:20:30.357707 16819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a9de347c711477e87c582d003f36925" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:20:30.359333 19664 raft_consensus.cc:2804] T 8c11074579b046d99e585c2818e18a2c P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:20:30.360538 16823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57a744f5aaf944678a63469cbf514015" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
W20260709 13:20:30.363137 16117 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 108577us
I20260709 13:20:30.406124 16117 leader_election.cc:304] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860
I20260709 13:20:30.376197 16816 raft_consensus.cc:2468] T 9e65676b21b748728ad39668cfe722ec P cb82cc7bd6db46d980c75237a1402860 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 10.
I20260709 13:20:30.231612 19666 raft_consensus.cc:515] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:30.378747 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66f75124215f4ca6bef831e174a792d8" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 10 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:30.333115 16292 rpcz_store.cc:276] 0709 13:20:29.003445 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:29.888778 (+885333us) service_pool.cc:224] Handling call
0709 13:20:29.889189 (+   411us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:20:29.889710 (+   521us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:29.889744 (+    34us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:20:29.890401 (+   657us) log.cc:844] Serialized 52 byte log entry
0709 13:20:30.074025 (+183624us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:20:30.074075 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:30.088673 (+ 14598us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:20:30.319304 (+230631us) raft_consensus.cc:1685] finished
0709 13:20:30.319335 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:30.319516 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":20943,"raft.queue_time_us":1409,"raft.run_cpu_time_us":323,"raft.run_wall_time_us":17780,"thread_start_us":183218,"threads_started":1,"wal-append.queue_time_us":413748}
I20260709 13:20:30.326498 19414 raft_consensus.cc:493] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:20:30.390435 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a378d7f796f94615b3b347119a9880fd" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:20:30.411255 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e4bfb9090c8460eb14fa7e0d818e7f1" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:30.362536 19822 raft_consensus.cc:2764] T 5a959dbecc2648eb85d1cc550f5a3245 P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Leader pre-election decision vote started in defunct term 5: won
I20260709 13:20:30.389128 16291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e9b2bf7011f4ae688a79d7bb5a447b4" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:20:30.396152 16300 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40474 (request call id 549) took 1537 ms. Trace:
W20260709 13:20:30.403050 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 196083us
I20260709 13:20:30.414572 16295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e65676b21b748728ad39668cfe722ec" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
W20260709 13:20:30.406114 16192 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 296055us
W20260709 13:20:30.397759 16790 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.14.177.129:36741 timed out after 1.765s (SENT) [suppressed 23 similar messages]
I20260709 13:20:30.411345 16298 raft_consensus.cc:3060] T a378d7f796f94615b3b347119a9880fd P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:20:30.330457 18599 raft_consensus.cc:2804] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:20:30.415798 16295 raft_consensus.cc:2393] T 9e65676b21b748728ad39668cfe722ec P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 in current term 10: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
I20260709 13:20:30.407860 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26354361e9b044cea1c2b86c4882fe92" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:20:30.376791 19789 raft_consensus.cc:740] T 23f82224f4564fea93572e8206bad1fe P 647a343a542d4a139fe106c19095129b [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:20:30.418318 16297 raft_consensus.cc:2393] T 26354361e9b044cea1c2b86c4882fe92 P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 7: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
W20260709 13:20:30.419179 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 327559us
I20260709 13:20:30.391525 19754 raft_consensus.cc:740] T b87147731a68400e8cce9b879daa7c5d P 647a343a542d4a139fe106c19095129b [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:20:30.400439 19699 raft_consensus.cc:3060] T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:20:30.452242 16219 raft_consensus.cc:1275] T c2f5f89186204242bf377f7c42b2e2da P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:20:30.440953 16223 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33050 (request call id 606) took 1879 ms. Trace:
I20260709 13:20:30.399821 19760 raft_consensus.cc:740] T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:20:30.456754 16190 leader_election.cc:304] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: cb82cc7bd6db46d980c75237a1402860
I20260709 13:20:30.459304 18599 raft_consensus.cc:697] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [term 12 LEADER]: Becoming Leader. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:20:30.452590 16220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5ad4eb281564936a59f41e387a47156" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:20:30.463529 16291 raft_consensus.cc:2393] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 in current term 7: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
I20260709 13:20:30.463954 16220 raft_consensus.cc:3060] T e5ad4eb281564936a59f41e387a47156 P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:20:30.414997 16300 rpcz_store.cc:276] 0709 13:20:28.858689 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:29.876410 (+1017721us) service_pool.cc:224] Handling call
0709 13:20:29.876857 (+   447us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:29.877245 (+   388us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:20:30.107277 (+230032us) log.cc:844] Serialized 14 byte log entry
0709 13:20:30.395549 (+288272us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:20:30.395734 (+   185us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:20:30.395823 (+    89us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:30.395962 (+   139us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:30.396093 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":453494,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":40,"raft.queue_time_us":11051,"thread_start_us":409171,"threads_started":2}
I20260709 13:20:30.469765 16142 raft_consensus.cc:2468] T 1df93779209748b9860f6d893dd3de53 P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 9.
I20260709 13:20:30.470775 16296 raft_consensus.cc:1275] T 3884f864a96d428785895e5463e5728f P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
W20260709 13:20:30.450490 19763 consensus_meta.cc:287] T 3884f864a96d428785895e5463e5728f P 647a343a542d4a139fe106c19095129b: Time spent flushing consensus metadata: real 0.556s	user 0.014s	sys 0.000s
W20260709 13:20:30.471607 16114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:30.413952 19414 raft_consensus.cc:3060] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:20:30.425104 19664 raft_consensus.cc:697] T 8c11074579b046d99e585c2818e18a2c P 90f3da57796b488c9087a2f523adf045 [term 9 LEADER]: Becoming Leader. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:20:30.482899 16265 leader_election.cc:304] T d16dfad0c3b540e5b4b86492ddf87208 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:20:30.493410 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a959dbecc2648eb85d1cc550f5a3245" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:30.460980 19540 raft_consensus.cc:2749] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:20:30.498335 18599 consensus_queue.cc:237] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:30.504680 16143 raft_consensus.cc:1275] T 3884f864a96d428785895e5463e5728f P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:20:30.511601 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87147731a68400e8cce9b879daa7c5d" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:20:30.514012 16115 leader_election.cc:304] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860
I20260709 13:20:30.508616 16145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e9b2bf7011f4ae688a79d7bb5a447b4" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:20:30.515591 16145 raft_consensus.cc:3060] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:20:30.450196 19816 raft_consensus.cc:515] T 829b4ad8c5a54eababeea1181b960d4b P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:30.517952 16142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f82224f4564fea93572e8206bad1fe" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
W20260709 13:20:30.521534 16117 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 115488us
I20260709 13:20:30.522244 16047 catalog_manager.cc:5697] T af57996a8be84d398f0f023e8d08ef77 P cb82cc7bd6db46d980c75237a1402860 reported cstate change: term changed from 2 to 7, leader changed from cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) to 647a343a542d4a139fe106c19095129b (127.14.177.129). New cstate: current_term: 7 leader_uuid: "647a343a542d4a139fe106c19095129b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:20:30.523559 16144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee9ed67006144797b1de6ea8c2013a68" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:20:30.524384 16144 raft_consensus.cc:2393] T ee9ed67006144797b1de6ea8c2013a68 P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 in current term 7: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
I20260709 13:20:30.524204 16047 catalog_manager.cc:5697] T 434bff3082d141838068f323bdb0d1ed P cb82cc7bd6db46d980c75237a1402860 reported cstate change: term changed from 4 to 7, leader changed from 647a343a542d4a139fe106c19095129b (127.14.177.129) to cb82cc7bd6db46d980c75237a1402860 (127.14.177.132). New cstate: current_term: 7 leader_uuid: "cb82cc7bd6db46d980c75237a1402860" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:20:30.525367 16151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af57996a8be84d398f0f023e8d08ef77" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:20:30.525599 19732 raft_consensus.cc:3055] T 2794af5b99984efebdee564035841cb0 P cb82cc7bd6db46d980c75237a1402860 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:20:30.526181 19732 raft_consensus.cc:740] T 2794af5b99984efebdee564035841cb0 P cb82cc7bd6db46d980c75237a1402860 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:20:30.528769 16299 raft_consensus.cc:2468] T 5a959dbecc2648eb85d1cc550f5a3245 P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 6.
I20260709 13:20:30.402818 19667 consensus_queue.cc:260] T ee9ed67006144797b1de6ea8c2013a68 P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:30.524574 19856 raft_consensus.cc:2804] T ecc1e363230043fe9fef390f0b979f20 P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:20:30.535506 16116 leader_election.cc:304] T 5a959dbecc2648eb85d1cc550f5a3245 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
I20260709 13:20:30.526921 16047 catalog_manager.cc:5697] T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860 reported cstate change: term changed from 4 to 6, leader changed from 90f3da57796b488c9087a2f523adf045 (127.14.177.131) to 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130). New cstate: current_term: 6 leader_uuid: "1ca2725d290e47d68b1a4b609c9816ac" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:20:30.546861 19822 raft_consensus.cc:2804] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:20:30.534446 19823 raft_consensus.cc:697] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [term 11 LEADER]: Becoming Leader. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:20:30.458478 16223 rpcz_store.cc:276] 0709 13:20:28.561761 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:29.707668 (+1145907us) service_pool.cc:224] Handling call
0709 13:20:29.708175 (+   507us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:29.708641 (+   466us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:20:29.964265 (+255624us) log.cc:844] Serialized 14 byte log entry
0709 13:20:30.440490 (+476225us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:20:30.440544 (+    54us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:20:30.440585 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:30.440758 (+   173us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:30.440894 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7425,"prepare.queue_time_us":360857,"prepare.run_cpu_time_us":56,"prepare.run_wall_time_us":2396,"thread_start_us":635798,"threads_started":3,"wal-append.queue_time_us":430874}
I20260709 13:20:30.551608 19760 consensus_queue.cc:260] T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:20:30.559300 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 274064us
I20260709 13:20:30.527156 19732 consensus_queue.cc:260] T 2794af5b99984efebdee564035841cb0 P cb82cc7bd6db46d980c75237a1402860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:30.572554 15046 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
W20260709 13:20:30.573920 16266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 91732.9us
W20260709 13:20:30.575896 16044 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:49250 (request call id 320) took 25083 ms (client timeout 15000 ms). Trace:
I20260709 13:20:30.526484 19789 consensus_queue.cc:260] T 23f82224f4564fea93572e8206bad1fe P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:30.584407 16300 raft_consensus.cc:1240] T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 7. Current term is 8. Ops: []
I20260709 13:20:30.586522 19418 consensus_queue.cc:1059] T af57996a8be84d398f0f023e8d08ef77 P 647a343a542d4a139fe106c19095129b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:20:30.565651 19751 raft_consensus.cc:697] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [term 9 LEADER]: Becoming Leader. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:20:30.589459 19666 leader_election.cc:290] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 11 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:30.573479 16226 raft_consensus.cc:1275] T 18ad7e3af5504571a69d10c00750a01c P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:20:30.607802 16148 raft_consensus.cc:2393] T 829b4ad8c5a54eababeea1181b960d4b P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 in current term 9: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
I20260709 13:20:30.492870 19672 consensus_queue.cc:1059] T 26354361e9b044cea1c2b86c4882fe92 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.044s
I20260709 13:20:30.611430 16298 raft_consensus.cc:2468] T a378d7f796f94615b3b347119a9880fd P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 9.
I20260709 13:20:30.616135 16225 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57604 (request call id 2372) took 1376 ms. Trace:
W20260709 13:20:30.620165 16189 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:30.583851 19667 raft_consensus.cc:3060] T ee9ed67006144797b1de6ea8c2013a68 P 1ca2725d290e47d68b1a4b609c9816ac [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:20:30.606454 19822 raft_consensus.cc:493] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Starting leader election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
W20260709 13:20:30.627298 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 91883.4us
I20260709 13:20:30.502722 19673 raft_consensus.cc:515] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:30.531037 19657 raft_consensus.cc:2749] T d16dfad0c3b540e5b4b86492ddf87208 P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:20:30.596742 19856 raft_consensus.cc:493] T ecc1e363230043fe9fef390f0b979f20 P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:30.644400 19672 raft_consensus.cc:3055] T 26354361e9b044cea1c2b86c4882fe92 P cb82cc7bd6db46d980c75237a1402860 [term 6 LEADER]: Stepping down as leader of term 6
W20260709 13:20:30.626050 16115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 121512us
I20260709 13:20:30.645689 16191 leader_election.cc:304] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:20:30.637537 16047 catalog_manager.cc:5697] T 8c11074579b046d99e585c2818e18a2c P cb82cc7bd6db46d980c75237a1402860 reported cstate change: term changed from 5 to 6, leader changed from 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130) to cb82cc7bd6db46d980c75237a1402860 (127.14.177.132). New cstate: current_term: 6 leader_uuid: "cb82cc7bd6db46d980c75237a1402860" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:20:30.635555 19763 raft_consensus.cc:2804] T 5a959dbecc2648eb85d1cc550f5a3245 P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:20:30.641321 16225 rpcz_store.cc:276] 0709 13:20:29.239982 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:29.802885 (+562903us) service_pool.cc:224] Handling call
0709 13:20:29.943665 (+140780us) spinlock_profiling.cc:243] Waited 67.8 ms on lock 0x7b5800010b98. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69ad8fadb1 00007f69ada3ab71 00007f69ada3aeee 0000000000375939 00007f69b5aae1e0 00007f69b5b770c4 00007f69b5b77131 00007f69b5b38fc0 00007f69b5b23929 00007f69b5b233d2 00007f69af09abd4
0709 13:20:29.943871 (+   206us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:29.947047 (+  3176us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:20:30.281609 (+334562us) log.cc:844] Serialized 14 byte log entry
0709 13:20:30.615377 (+333768us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:20:30.615514 (+   137us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:20:30.615605 (+    91us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:30.615785 (+   180us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:30.616010 (+   225us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":196990,"prepare.run_cpu_time_us":61,"prepare.run_wall_time_us":4862,"spinlock_wait_cycles":155882240,"thread_start_us":564084,"threads_started":2}
I20260709 13:20:30.679119 16265 leader_election.cc:304] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 647a343a542d4a139fe106c19095129b
I20260709 13:20:30.692656 16298 raft_consensus.cc:1240] T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 9. Current term is 10. Ops: [7.5-9.6]
I20260709 13:20:30.572917 19664 consensus_queue.cc:237] T 8c11074579b046d99e585c2818e18a2c P 90f3da57796b488c9087a2f523adf045 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.9, Last appended by leader: 9, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:20:30.693639 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 48041.7us
I20260709 13:20:30.693539 16047 catalog_manager.cc:5697] T 3a7ca61592e848ea9b0fb81a2ce33cdc P cb82cc7bd6db46d980c75237a1402860 reported cstate change: term changed from 7 to 9. New cstate: current_term: 9 leader_uuid: "1ca2725d290e47d68b1a4b609c9816ac" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:20:30.695536 19857 raft_consensus.cc:3060] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:20:30.696017 16115 leader_election.cc:400] T fffab26427e24d398e2695aad40ca134 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 7 election: Vote denied by peer 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389) with higher term. Message: Invalid argument: T fffab26427e24d398e2695aad40ca134 P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b for earlier term 7. Current term is 8.
I20260709 13:20:30.696504 16115 leader_election.cc:403] T fffab26427e24d398e2695aad40ca134 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 7 election: Cancelling election due to peer responding with higher term
I20260709 13:20:30.696992 19450 raft_consensus.cc:493] T 5a96373bb0e14dce8a266b2fc6c58507 P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:30.697556 19450 raft_consensus.cc:515] T 5a96373bb0e14dce8a266b2fc6c58507 P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:30.698276 19816 leader_election.cc:290] T 829b4ad8c5a54eababeea1181b960d4b P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:30.699455 16819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "829b4ad8c5a54eababeea1181b960d4b" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 10 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:30.665606 19856 raft_consensus.cc:3060] T ecc1e363230043fe9fef390f0b979f20 P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:20:30.700363 16819 raft_consensus.cc:2410] T 829b4ad8c5a54eababeea1181b960d4b P cb82cc7bd6db46d980c75237a1402860 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b for term 10 because replica has last-logged OpId of term: 7 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 8.
I20260709 13:20:30.700562 16291 raft_consensus.cc:1275] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:20:30.701519 16297 raft_consensus.cc:1240] T 31969392ccdb43b491911a319e8bc7e1 P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 7. Current term is 8. Ops: []
I20260709 13:20:30.701740 16291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a959dbecc2648eb85d1cc550f5a3245" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:20:30.702366 16291 raft_consensus.cc:3060] T 5a959dbecc2648eb85d1cc550f5a3245 P 90f3da57796b488c9087a2f523adf045 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:20:30.655726 19822 raft_consensus.cc:3060] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:20:30.694706 16191 leader_election.cc:304] T 5a96373bb0e14dce8a266b2fc6c58507 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:20:30.702908 16297 raft_consensus.cc:1275] T 8d4ab870a05347868956aa8b841b2981 P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
W20260709 13:20:30.688112 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 231350us
I20260709 13:20:30.660826 19666 consensus_queue.cc:1059] T c0eab25ce4c64e018a041beeda2867db P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 3, Time since last communication: 0.074s
I20260709 13:20:30.735405 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0eab25ce4c64e018a041beeda2867db" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:20:30.734999 19823 consensus_queue.cc:237] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:30.736079 16298 raft_consensus.cc:3060] T c0eab25ce4c64e018a041beeda2867db P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:20:30.739295 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 44700.8us
I20260709 13:20:30.541446 19671 consensus_queue.cc:237] T b6f07842d3aa47858df5665874c86f3a P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:30.739668 16300 raft_consensus.cc:1240] T 84f2da98864b4425bf7c0055951fdd4b P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 6. Current term is 7. Ops: [5.3-6.4]
I20260709 13:20:30.748576 16291 raft_consensus.cc:2468] T 5a959dbecc2648eb85d1cc550f5a3245 P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 7.
W20260709 13:20:30.685513 19702 consensus_meta.cc:287] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac: Time spent flushing consensus metadata: real 0.538s	user 0.004s	sys 0.009s
I20260709 13:20:30.567464 19754 consensus_queue.cc:260] T b87147731a68400e8cce9b879daa7c5d P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:30.669888 19672 raft_consensus.cc:740] T 26354361e9b044cea1c2b86c4882fe92 P cb82cc7bd6db46d980c75237a1402860 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:20:30.685897 19789 raft_consensus.cc:3060] T 23f82224f4564fea93572e8206bad1fe P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:20:30.625537 16044 rpcz_store.cc:269] 0709 13:20:05.492195 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:05.505382 (+ 13187us) service_pool.cc:224] Handling call
0709 13:20:30.575647 (+25070265us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:20:23.941827 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:20:23.942508 (+   681us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:20:23.942563 (+    55us) write_op.cc:435] Acquired schema lock
0709 13:20:23.942579 (+    16us) tablet.cc:662] Decoding operations
0709 13:20:23.943808 (+  1229us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:20:23.943881 (+    73us) write_op.cc:641] Partition lock acquired for write op
0709 13:20:23.943910 (+    29us) tablet.cc:685] Acquiring locks for 13 operations
0709 13:20:23.988699 (+ 44789us) tablet.cc:701] Row locks acquired
0709 13:20:23.988759 (+    60us) write_op.cc:260] PREPARE: finished
0709 13:20:24.057643 (+ 68884us) write_op.cc:270] Start()
0709 13:20:24.070357 (+ 12714us) write_op.cc:276] Timestamp: P: 1783603224043541 usec, L: 0
0709 13:20:24.070393 (+    36us) op_driver.cc:348] REPLICATION: starting
0709 13:20:24.071297 (+   904us) log.cc:844] Serialized 5252 byte log entry
0709 13:20:24.909356 (+838059us) op_driver.cc:464] REPLICATION: finished
0709 13:20:25.501118 (+591762us) write_op.cc:301] APPLY: starting
0709 13:20:25.517243 (+ 16125us) tablet.cc:1367] starting BulkCheckPresence
0709 13:20:25.677832 (+160589us) tablet.cc:1370] finished BulkCheckPresence
0709 13:20:25.727640 (+ 49808us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:20:26.075329 (+347689us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:20:26.075691 (+   362us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=13
0709 13:20:26.075748 (+    57us) write_op.cc:312] APPLY: finished
0709 13:20:26.127181 (+ 51433us) log.cc:844] Serialized 95 byte log entry
0709 13:20:26.616710 (+489529us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:20:26.646037 (+ 29327us) write_op.cc:454] Released schema lock
0709 13:20:26.683838 (+ 37801us) write_op.cc:341] FINISH: Updating metrics
0709 13:20:26.739979 (+ 56141us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:20:26.741547 (+  1568us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":2041127,"reported_tablets":43,"child_traces":[["op",{"apply.queue_time_us":530238,"apply.run_cpu_time_us":34873,"apply.run_wall_time_us":1239184,"num_ops":13,"prepare.queue_time_us":147889,"prepare.run_cpu_time_us":37606,"prepare.run_wall_time_us":438419,"raft.queue_time_us":439262,"raft.run_cpu_time_us":7352,"raft.run_wall_time_us":46445,"replication_time_us":838674,"thread_start_us":1241406,"threads_started":5,"wal-append.queue_time_us":898536,"wal-append.run_cpu_time_us":42524,"wal-append.run_wall_time_us":2519548}]]}
I20260709 13:20:30.755781 19751 consensus_queue.cc:237] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:20:30.757342 16266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 80409.9us
I20260709 13:20:30.675179 19345 consensus_queue.cc:1048] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.085s
I20260709 13:20:30.750329 19860 raft_consensus.cc:2749] T 5a96373bb0e14dce8a266b2fc6c58507 P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:20:30.763548 16789 leader_election.cc:304] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:20:30.764535 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5ad4eb281564936a59f41e387a47156" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
W20260709 13:20:30.788934 16115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 93198.5us
I20260709 13:20:30.788793 19816 consensus_queue.cc:1048] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:20:30.790262 16115 leader_election.cc:304] T 8a878999c0024d2abe517484180290fc P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:20:30.790753 19754 raft_consensus.cc:3060] T b87147731a68400e8cce9b879daa7c5d P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:20:30.794624 19707 raft_consensus.cc:3055] T af57996a8be84d398f0f023e8d08ef77 P 647a343a542d4a139fe106c19095129b [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:20:30.795182 19707 raft_consensus.cc:740] T af57996a8be84d398f0f023e8d08ef77 P 647a343a542d4a139fe106c19095129b [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:20:30.795516 19732 raft_consensus.cc:3060] T 2794af5b99984efebdee564035841cb0 P cb82cc7bd6db46d980c75237a1402860 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:20:30.797055 19673 leader_election.cc:290] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 9 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:20:30.797674 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57a744f5aaf944678a63469cbf514015" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:20:30.798059 19672 consensus_queue.cc:260] T 26354361e9b044cea1c2b86c4882fe92 P cb82cc7bd6db46d980c75237a1402860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:30.799587 19672 raft_consensus.cc:3060] T 26354361e9b044cea1c2b86c4882fe92 P cb82cc7bd6db46d980c75237a1402860 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:20:30.765753 19702 raft_consensus.cc:515] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:30.802234 19760 raft_consensus.cc:3060] T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:20:30.804248 19702 leader_election.cc:290] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 9 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:30.804374 19861 raft_consensus.cc:3055] T c0eab25ce4c64e018a041beeda2867db P 1ca2725d290e47d68b1a4b609c9816ac [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:20:30.806922 16191 leader_election.cc:400] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 11 election: Vote denied by peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741) with higher term. Message: Invalid argument: T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac for earlier term 11. Current term is 12.
I20260709 13:20:30.810576 16293 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40474 (request call id 552) took 1798 ms. Trace:
I20260709 13:20:30.811177 16293 rpcz_store.cc:276] 0709 13:20:29.012448 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:29.893209 (+880761us) service_pool.cc:224] Handling call
0709 13:20:29.919936 (+ 26727us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 13:20:29.947126 (+ 27190us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:29.947175 (+    49us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 13:20:29.948573 (+  1398us) log.cc:844] Serialized 624 byte log entry
0709 13:20:30.287468 (+338895us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:20:30.287509 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:30.287581 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:20:30.810250 (+522669us) raft_consensus.cc:1685] finished
0709 13:20:30.810278 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:30.810478 (+   200us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":23678,"raft.queue_time_us":29876,"raft.run_cpu_time_us":594,"raft.run_wall_time_us":109471,"thread_start_us":338424,"threads_started":1,"wal-append.queue_time_us":858767}
I20260709 13:20:30.813180 16297 raft_consensus.cc:2393] T e5ad4eb281564936a59f41e387a47156 P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 10: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
W20260709 13:20:30.813556 16789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 71834.3us
I20260709 13:20:30.815985 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d16dfad0c3b540e5b4b86492ddf87208" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:30.795307 19657 raft_consensus.cc:2804] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Leader election won for term 9
W20260709 13:20:30.821633 16116 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:20:30.822264 16116 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:30.822314 16220 raft_consensus.cc:2468] T e5ad4eb281564936a59f41e387a47156 P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 10.
W20260709 13:20:30.822724 16116 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:20:30.823289 16116 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:30.818645 16145 raft_consensus.cc:2468] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 7.
I20260709 13:20:30.825574 16190 leader_election.cc:304] T 1df93779209748b9860f6d893dd3de53 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:20:30.840397 16291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a9de347c711477e87c582d003f36925" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:20:30.841305 16291 raft_consensus.cc:3060] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:20:30.804970 16047 catalog_manager.cc:5697] T 19baa7bc367844c288bca8f81439c413 P cb82cc7bd6db46d980c75237a1402860 reported cstate change: term changed from 4 to 7, leader changed from 647a343a542d4a139fe106c19095129b (127.14.177.129) to 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130), pending config changed. New cstate: current_term: 7 leader_uuid: "1ca2725d290e47d68b1a4b609c9816ac" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } } } pending_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:20:30.766001 19763 raft_consensus.cc:697] T 5a959dbecc2648eb85d1cc550f5a3245 P 647a343a542d4a139fe106c19095129b [term 7 LEADER]: Becoming Leader. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:20:30.866814 16300 raft_consensus.cc:2468] T d16dfad0c3b540e5b4b86492ddf87208 P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 8.
I20260709 13:20:30.868006 16266 leader_election.cc:304] T 9e65676b21b748728ad39668cfe722ec P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 
W20260709 13:20:30.868396 16189 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:30.856858 16298 raft_consensus.cc:2468] T c0eab25ce4c64e018a041beeda2867db P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 9.
I20260709 13:20:30.771296 19418 raft_consensus.cc:3060] T fffab26427e24d398e2695aad40ca134 P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:20:30.873343 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 475582us
I20260709 13:20:30.804972 19861 raft_consensus.cc:740] T c0eab25ce4c64e018a041beeda2867db P 1ca2725d290e47d68b1a4b609c9816ac [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:20:30.847772 16819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5718c623ebee4d2bb770f0f9ddb2f164" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
W20260709 13:20:30.880044 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 121210us
I20260709 13:20:30.838732 19696 raft_consensus.cc:2749] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:20:30.883025 16789 leader_election.cc:304] T 23f82224f4564fea93572e8206bad1fe P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:20:30.904704 19568 raft_consensus.cc:2804] T 23f82224f4564fea93572e8206bad1fe P cb82cc7bd6db46d980c75237a1402860 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:20:30.905313 19568 raft_consensus.cc:697] T 23f82224f4564fea93572e8206bad1fe P cb82cc7bd6db46d980c75237a1402860 [term 8 LEADER]: Becoming Leader. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:20:30.883553 16293 raft_consensus.cc:1240] T 23f82224f4564fea93572e8206bad1fe P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 7. Current term is 8. Ops: []
I20260709 13:20:30.883661 19673 consensus_queue.cc:1059] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
W20260709 13:20:30.884748 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 76842.1us
I20260709 13:20:30.919062 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "829b4ad8c5a54eababeea1181b960d4b" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 10 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:30.919133 16116 leader_election.cc:304] T e5ad4eb281564936a59f41e387a47156 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
I20260709 13:20:30.887324 19863 raft_consensus.cc:493] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:30.890035 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a96373bb0e14dce8a266b2fc6c58507" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 9 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:30.923877 16226 raft_consensus.cc:2468] T 5a96373bb0e14dce8a266b2fc6c58507 P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 8.
W20260709 13:20:30.891475 16189 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:30.838819 19726 consensus_queue.cc:1048] T 18ad7e3af5504571a69d10c00750a01c P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.068s
I20260709 13:20:30.927865 16191 leader_election.cc:304] T 18ad7e3af5504571a69d10c00750a01c P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:20:30.892642 19540 raft_consensus.cc:493] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:30.893263 19529 raft_consensus.cc:493] T 84f2da98864b4425bf7c0055951fdd4b P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:30.895907 19857 raft_consensus.cc:2749] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:20:30.895996 19575 raft_consensus.cc:2749] T 8a878999c0024d2abe517484180290fc P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:20:30.896936 19700 consensus_meta.cc:287] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac: Time spent flushing consensus metadata: real 0.536s	user 0.011s	sys 0.003s
I20260709 13:20:30.838644 19450 leader_election.cc:290] T 5a96373bb0e14dce8a266b2fc6c58507 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
W20260709 13:20:30.901579 19407 consensus_meta.cc:287] T 68a6838d82ef4cac83507f906139d90f P 1ca2725d290e47d68b1a4b609c9816ac: Time spent flushing consensus metadata: real 0.573s	user 0.011s	sys 0.002s
I20260709 13:20:30.902192 19569 raft_consensus.cc:493] T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:20:30.909547 16117 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:30.910419 16819 raft_consensus.cc:2393] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 9: Already voted for candidate cb82cc7bd6db46d980c75237a1402860 in this term.
I20260709 13:20:30.912288 16297 raft_consensus.cc:1275] T c9944d5e4b8c4109af9955c4f08c9095 P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 9 index: 9. (term mismatch)
I20260709 13:20:30.918002 16291 raft_consensus.cc:2468] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 11.
I20260709 13:20:30.867264 19657 raft_consensus.cc:697] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045 [term 9 LEADER]: Becoming Leader. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:20:30.880882 19785 raft_consensus.cc:2804] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:20:30.953292 16297 pending_rounds.cc:85] T c9944d5e4b8c4109af9955c4f08c9095 P 90f3da57796b488c9087a2f523adf045: Aborting all ops after (but not including) 8
I20260709 13:20:30.961130 16297 pending_rounds.cc:107] T c9944d5e4b8c4109af9955c4f08c9095 P 90f3da57796b488c9087a2f523adf045: Aborting uncommitted NO_OP operation due to leader change: 8.9
I20260709 13:20:30.961499 16297 raft_consensus.cc:2889] T c9944d5e4b8c4109af9955c4f08c9095 P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:20:30.937557 16115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 147477us
I20260709 13:20:30.948537 16293 raft_consensus.cc:1240] T c0eab25ce4c64e018a041beeda2867db P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 8. Current term is 9. Ops: []
I20260709 13:20:30.954523 16298 raft_consensus.cc:1240] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 10. Current term is 11. Ops: [5.7-10.8]
I20260709 13:20:30.938151 19345 raft_consensus.cc:2804] T 9e65676b21b748728ad39668cfe722ec P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:20:30.958249 16047 catalog_manager.cc:5697] T d2ab0ac8347641e6955e9bce7d90c1a2 P cb82cc7bd6db46d980c75237a1402860 reported cstate change: term changed from 5 to 7. New cstate: current_term: 7 leader_uuid: "1ca2725d290e47d68b1a4b609c9816ac" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:20:30.925463 19702 raft_consensus.cc:2804] T 1df93779209748b9860f6d893dd3de53 P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:20:30.940541 19707 consensus_queue.cc:260] T af57996a8be84d398f0f023e8d08ef77 P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:30.906335 19568 consensus_queue.cc:237] T 23f82224f4564fea93572e8206bad1fe P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:20:30.991358 16117 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:20:30.993762 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 56793.4us
I20260709 13:20:30.945654 19754 raft_consensus.cc:2804] T e5ad4eb281564936a59f41e387a47156 P 647a343a542d4a139fe106c19095129b [term 10 FOLLOWER]: Leader election won for term 10
W20260709 13:20:30.998090 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 72867.1us
W20260709 13:20:31.000217 16117 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:30.934299 19806 consensus_queue.cc:1048] T 3884f864a96d428785895e5463e5728f P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.047s
W20260709 13:20:31.001116 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 364965us
I20260709 13:20:31.004591 16221 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57604 (request call id 2356) took 2464 ms. Trace:
I20260709 13:20:30.940734 19529 raft_consensus.cc:515] T 84f2da98864b4425bf7c0055951fdd4b P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:30.935250 19863 raft_consensus.cc:515] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:30.989470 19785 raft_consensus.cc:697] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860 [term 8 LEADER]: Becoming Leader. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:20:30.976176 19866 raft_consensus.cc:2749] T 18ad7e3af5504571a69d10c00750a01c P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:20:31.046052 16790 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:31.010977 16221 rpcz_store.cc:276] 0709 13:20:28.535530 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:29.707154 (+1171624us) service_pool.cc:224] Handling call
0709 13:20:29.707770 (+   616us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:29.708360 (+   590us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:20:29.973420 (+265060us) log.cc:844] Serialized 14 byte log entry
0709 13:20:30.870985 (+897565us) log.cc:844] Serialized 14 byte log entry
0709 13:20:30.933231 (+ 62246us) log.cc:844] Serialized 14 byte log entry
0709 13:20:30.999087 (+ 65856us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:20:30.999134 (+    47us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:20:30.999192 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:30.999286 (+    94us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:30.999418 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":116917,"prepare.queue_time_us":331303,"prepare.run_cpu_time_us":169,"prepare.run_wall_time_us":6271,"raft.queue_time_us":265288,"raft.run_cpu_time_us":303,"raft.run_wall_time_us":23074,"spinlock_wait_cycles":1084672,"thread_start_us":1029750,"threads_started":3,"wal-append.queue_time_us":856739}
W20260709 13:20:31.047811 19868 log.cc:927] Time spent T 8158bbd17b994d2eb96f560bd0f1b88d P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
I20260709 13:20:31.049177 19529 leader_election.cc:290] T 84f2da98864b4425bf7c0055951fdd4b P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:30.960788 19569 raft_consensus.cc:515] T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:31.051867 19569 leader_election.cc:290] T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
W20260709 13:20:31.052515 19699 consensus_meta.cc:287] T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac: Time spent flushing consensus metadata: real 0.564s	user 0.011s	sys 0.000s
I20260709 13:20:31.052767 19863 leader_election.cc:290] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:31.053045 19699 raft_consensus.cc:515] T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:31.053390 19763 consensus_queue.cc:237] T 5a959dbecc2648eb85d1cc550f5a3245 P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:30.966018 19856 raft_consensus.cc:515] T ecc1e363230043fe9fef390f0b979f20 P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:31.044756 19816 raft_consensus.cc:493] T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:31.042240 19754 raft_consensus.cc:697] T e5ad4eb281564936a59f41e387a47156 P 647a343a542d4a139fe106c19095129b [term 10 LEADER]: Becoming Leader. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:20:31.058893 19816 raft_consensus.cc:515] T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:31.060048 19754 consensus_queue.cc:237] T e5ad4eb281564936a59f41e387a47156 P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:20:31.062847 19197 consensus_meta.cc:287] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 647a343a542d4a139fe106c19095129b: Time spent flushing consensus metadata: real 0.707s	user 0.008s	sys 0.005s
I20260709 13:20:30.985750 19540 raft_consensus.cc:515] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:31.068790 16147 raft_consensus.cc:2410] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac for term 9 because replica has last-logged OpId of term: 8 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 6.
I20260709 13:20:31.058133 19856 leader_election.cc:290] T ecc1e363230043fe9fef390f0b979f20 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 9 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:31.065445 16819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5d59407eb545b895d8d922e0dd69fd" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:31.064724 16191 leader_election.cc:304] T e4dc6b0541ca431ebc757beac00521f1 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
W20260709 13:20:31.064039 16115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 60578.4us
I20260709 13:20:30.975423 19822 raft_consensus.cc:515] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:20:31.069095 16789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 160268us
I20260709 13:20:31.014052 19345 raft_consensus.cc:697] T 9e65676b21b748728ad39668cfe722ec P 90f3da57796b488c9087a2f523adf045 [term 10 LEADER]: Becoming Leader. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:20:31.065119 16219 raft_consensus.cc:1240] T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 9. Current term is 10. Ops: []
W20260709 13:20:31.070561 16790 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:31.014688 19702 raft_consensus.cc:697] T 1df93779209748b9860f6d893dd3de53 P 1ca2725d290e47d68b1a4b609c9816ac [term 9 LEADER]: Becoming Leader. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:20:31.079115 16789 leader_election.cc:304] T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:20:31.084034 16047 catalog_manager.cc:5697] T 74ca8140f35c4ae29e8a664f6bdfc829 P cb82cc7bd6db46d980c75237a1402860 reported cstate change: term changed from 2 to 7, leader changed from cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) to 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130). New cstate: current_term: 7 leader_uuid: "1ca2725d290e47d68b1a4b609c9816ac" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:20:31.087549 16150 raft_consensus.cc:2468] T b87147731a68400e8cce9b879daa7c5d P 647a343a542d4a139fe106c19095129b [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 10.
W20260709 13:20:31.088596 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 178347us
I20260709 13:20:31.090193 16190 leader_election.cc:304] T d2ab0ac8347641e6955e9bce7d90c1a2 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: cb82cc7bd6db46d980c75237a1402860
I20260709 13:20:31.090281 16821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:20:31.068519 19699 leader_election.cc:290] T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 10 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:31.094604 16116 leader_election.cc:304] T 5a96373bb0e14dce8a266b2fc6c58507 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
W20260709 13:20:31.081975 16790 connection.cc:256] RPC call timeout handler was delayed by -1.3098609500000293s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.3406s for a response
I20260709 13:20:31.100205 16047 catalog_manager.cc:5697] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 reported cstate change: term changed from 4 to 7, leader changed from cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) to 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130). New cstate: current_term: 7 leader_uuid: "1ca2725d290e47d68b1a4b609c9816ac" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:20:31.109724 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4dc6b0541ca431ebc757beac00521f1" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:31.115454 16265 leader_election.cc:304] T 8158bbd17b994d2eb96f560bd0f1b88d P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:20:31.080001 19707 raft_consensus.cc:3060] T af57996a8be84d398f0f023e8d08ef77 P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:20:31.011246 19861 consensus_queue.cc:260] T c0eab25ce4c64e018a041beeda2867db P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:31.092600 19866 raft_consensus.cc:2749] T e4dc6b0541ca431ebc757beac00521f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:20:31.108497 19802 raft_consensus.cc:2804] T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:20:31.106560 19696 raft_consensus.cc:2804] T 5a96373bb0e14dce8a266b2fc6c58507 P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:20:31.070978 19863 consensus_queue.cc:1059] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.050s
I20260709 13:20:31.131467 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f2da98864b4425bf7c0055951fdd4b" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:31.132730 16225 raft_consensus.cc:2468] T e4dc6b0541ca431ebc757beac00521f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 8.
I20260709 13:20:31.131765 16117 leader_election.cc:304] T 829b4ad8c5a54eababeea1181b960d4b P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860
W20260709 13:20:31.132181 16789 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:20:31.136202 16789 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:31.136068 16047 catalog_manager.cc:5697] T 66f75124215f4ca6bef831e174a792d8 P cb82cc7bd6db46d980c75237a1402860 reported cstate change: term changed from 6 to 7, leader changed from 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130) to cb82cc7bd6db46d980c75237a1402860 (127.14.177.132). New cstate: current_term: 7 leader_uuid: "cb82cc7bd6db46d980c75237a1402860" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:20:31.137153 19885 consensus_queue.cc:1048] T c9944d5e4b8c4109af9955c4f08c9095 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:20:31.138077 16047 catalog_manager.cc:5697] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 reported cstate change: term changed from 4 to 9. New cstate: current_term: 9 leader_uuid: "1ca2725d290e47d68b1a4b609c9816ac" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:20:31.133613 19664 raft_consensus.cc:2749] T 8158bbd17b994d2eb96f560bd0f1b88d P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:20:31.139899 16265 leader_election.cc:304] T 7c2095a5f4ad49408dbc7f69c606120d P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:20:31.140581 19873 raft_consensus.cc:493] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:31.141332 16265 leader_election.cc:304] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:20:31.141189 19873 raft_consensus.cc:515] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:31.143955 16819 raft_consensus.cc:1275] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P cb82cc7bd6db46d980c75237a1402860 [term 9 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
W20260709 13:20:31.144775 19872 log.cc:927] Time spent T 66f75124215f4ca6bef831e174a792d8 P cb82cc7bd6db46d980c75237a1402860: Append to log took a long time: real 0.129s	user 0.000s	sys 0.001s
I20260709 13:20:31.113113 19856 raft_consensus.cc:2804] T d2ab0ac8347641e6955e9bce7d90c1a2 P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:20:31.145953 19856 raft_consensus.cc:493] T d2ab0ac8347641e6955e9bce7d90c1a2 P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Starting leader election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:20:31.146379 19856 raft_consensus.cc:3060] T d2ab0ac8347641e6955e9bce7d90c1a2 P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:20:31.147823 16190 leader_election.cc:304] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: cb82cc7bd6db46d980c75237a1402860
I20260709 13:20:31.151376 19816 leader_election.cc:290] T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:31.152455 19673 consensus_queue.cc:1048] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:20:31.116888 19859 raft_consensus.cc:493] T 8a878999c0024d2abe517484180290fc P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:20:31.135226 19414 consensus_meta.cc:287] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 90f3da57796b488c9087a2f523adf045: Time spent flushing consensus metadata: real 0.629s	user 0.006s	sys 0.006s
W20260709 13:20:31.167989 16115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 65610.7us
W20260709 13:20:31.144037 19853 log.cc:927] Time spent T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 0.182s	user 0.000s	sys 0.000s
I20260709 13:20:31.130975 19696 raft_consensus.cc:493] T 5a96373bb0e14dce8a266b2fc6c58507 P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:20:31.182269 16117 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 50589.9us
W20260709 13:20:31.184726 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 54495.9us
I20260709 13:20:31.176975 16115 leader_election.cc:304] T 26354361e9b044cea1c2b86c4882fe92 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860
W20260709 13:20:31.190762 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 68304.1us
I20260709 13:20:31.181560 19696 raft_consensus.cc:3060] T 5a96373bb0e14dce8a266b2fc6c58507 P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:20:31.076746 19657 consensus_queue.cc:237] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.7, Last appended by leader: 7, Current term: 9, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:31.096020 19785 consensus_queue.cc:237] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:20:31.222033 16116 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:31.159543 19885 consensus_queue.cc:1048] T 57a744f5aaf944678a63469cbf514015 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.034s
I20260709 13:20:31.199720 19816 raft_consensus.cc:2749] T 829b4ad8c5a54eababeea1181b960d4b P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:20:31.183943 19866 raft_consensus.cc:2804] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:20:31.194096 19861 raft_consensus.cc:3060] T c0eab25ce4c64e018a041beeda2867db P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:20:31.258631 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b7d1ab519044eda2d5e04a07242f25" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 8 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
W20260709 13:20:31.234102 19879 log.cc:927] Time spent T 746aabf856ab401c9ef5e859371e7b67 P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 0.117s	user 0.001s	sys 0.000s
I20260709 13:20:31.221717 19873 leader_election.cc:290] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:20:31.175661 19732 consensus_queue.cc:1048] T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 6, Time since last communication: 0.058s
W20260709 13:20:31.235751 19875 log.cc:927] Time spent T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 0.105s	user 0.000s	sys 0.001s
I20260709 13:20:31.289672 16222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
W20260709 13:20:31.238770 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 97524.9us
I20260709 13:20:31.241671 19699 raft_consensus.cc:3055] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:20:31.243335 19575 raft_consensus.cc:2749] T 26354361e9b044cea1c2b86c4882fe92 P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:20:31.173629 19414 raft_consensus.cc:515] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:31.200268 19532 raft_consensus.cc:493] T 9c61a39fd4f84424b129c1a6f9ba404d P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:31.243660 19751 raft_consensus.cc:493] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:20:31.192622 19345 consensus_queue.cc:237] T 9e65676b21b748728ad39668cfe722ec P 90f3da57796b488c9087a2f523adf045 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:31.245688 19540 leader_election.cc:290] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:31.247046 16816 raft_consensus.cc:1240] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [term 12 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 11. Current term is 12. Ops: []
I20260709 13:20:31.184682 19702 consensus_queue.cc:237] T 1df93779209748b9860f6d893dd3de53 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:31.263685 16226 raft_consensus.cc:2410] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b for term 8 because replica has last-logged OpId of term: 6 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 8.
W20260709 13:20:31.263679 16115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 86830.7us
W20260709 13:20:31.273543 19878 log.cc:927] Time spent T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 0.094s	user 0.000s	sys 0.000s
I20260709 13:20:31.262866 19726 raft_consensus.cc:493] T 68a6838d82ef4cac83507f906139d90f P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:31.278123 16190 leader_election.cc:304] T 08c74490c40243a88e62d1760b5cd814 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
W20260709 13:20:31.283007 16192 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:31.283820 16219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada5a004ba7c46819051e3ccec71ffdb" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:31.263794 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806d7690c2b644959152074bb2cab0a7" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:31.287098 16218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee9ed67006144797b1de6ea8c2013a68" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:20:31.292265 16047 catalog_manager.cc:5697] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860 reported cstate change: term changed from 2 to 5, leader changed from 647a343a542d4a139fe106c19095129b (127.14.177.129) to cb82cc7bd6db46d980c75237a1402860 (127.14.177.132). New cstate: current_term: 5 leader_uuid: "cb82cc7bd6db46d980c75237a1402860" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:20:31.293713 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "829b4ad8c5a54eababeea1181b960d4b" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:20:31.295665 19418 raft_consensus.cc:2749] T fffab26427e24d398e2695aad40ca134 P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Leader election lost for term 7. Reason: Vote denied by peer 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389) with higher term. Message: Invalid argument: T fffab26427e24d398e2695aad40ca134 P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b for earlier term 7. Current term is 8.
I20260709 13:20:31.296169 16222 raft_consensus.cc:2468] T 19baa7bc367844c288bca8f81439c413 P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 9.
I20260709 13:20:31.297302 19696 raft_consensus.cc:515] T 5a96373bb0e14dce8a266b2fc6c58507 P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:20:31.316843 16116 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:31.350944 16224 raft_consensus.cc:2368] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 for earlier term 7. Current term is 8.
I20260709 13:20:31.298496 19699 raft_consensus.cc:740] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:20:31.320374 16295 raft_consensus.cc:1240] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 7. Current term is 8. Ops: []
I20260709 13:20:31.172356 19859 raft_consensus.cc:515] T 8a878999c0024d2abe517484180290fc P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:20:31.261413 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 108522us
I20260709 13:20:31.286540 19866 raft_consensus.cc:697] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [term 9 LEADER]: Becoming Leader. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:20:31.355825 16219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e4bfb9090c8460eb14fa7e0d818e7f1" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:31.357136 16142 raft_consensus.cc:2468] T 23f82224f4564fea93572e8206bad1fe P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 8.
W20260709 13:20:31.349133 19879 log.cc:927] Time spent T 746aabf856ab401c9ef5e859371e7b67 P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 0.056s	user 0.000s	sys 0.001s
I20260709 13:20:31.335505 19414 leader_election.cc:290] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 8 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:20:31.367012 16221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a378d7f796f94615b3b347119a9880fd" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:20:31.326565 19532 raft_consensus.cc:515] T 9c61a39fd4f84424b129c1a6f9ba404d P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:31.314827 19760 consensus_queue.cc:1059] T 8d4ab870a05347868956aa8b841b2981 P 647a343a542d4a139fe106c19095129b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.046s
I20260709 13:20:31.351055 19816 raft_consensus.cc:493] T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:31.375722 16222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af57996a8be84d398f0f023e8d08ef77" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
W20260709 13:20:31.377388 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 99388.1us
I20260709 13:20:31.377785 16219 raft_consensus.cc:2468] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 9.
I20260709 13:20:31.369699 16221 raft_consensus.cc:2410] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 for term 9 because replica has last-logged OpId of term: 8 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
I20260709 13:20:31.360723 19822 leader_election.cc:290] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 9 election: Requested vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:31.381390 16047 catalog_manager.cc:5697] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 reported cstate change: term changed from 4 to 8. New cstate: current_term: 8 leader_uuid: "647a343a542d4a139fe106c19095129b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:20:31.361100 19695 raft_consensus.cc:493] T 434bff3082d141838068f323bdb0d1ed P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
W20260709 13:20:31.418788 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 82856.7us
W20260709 13:20:31.418990 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 43985.4us
W20260709 13:20:31.410140 16116 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:31.336417 19751 raft_consensus.cc:515] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:31.427294 19855 raft_consensus.cc:2749] T 08c74490c40243a88e62d1760b5cd814 P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:20:31.428241 19751 leader_election.cc:290] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:31.429328 16790 leader_election.cc:400] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 11 pre-election: Vote denied by peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741) with higher term. Message: Invalid argument: T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 for earlier term 11. Current term is 12.
I20260709 13:20:31.429688 16790 leader_election.cc:403] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 11 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:20:31.430292 19898 raft_consensus.cc:493] T 19baa7bc367844c288bca8f81439c413 P cb82cc7bd6db46d980c75237a1402860 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:31.430944 19898 raft_consensus.cc:515] T 19baa7bc367844c288bca8f81439c413 P cb82cc7bd6db46d980c75237a1402860 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:20:31.431816 19888 log.cc:927] Time spent T 66f75124215f4ca6bef831e174a792d8 P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 0.239s	user 0.001s	sys 0.000s
I20260709 13:20:31.433574 19699 consensus_queue.cc:260] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:31.434713 19866 consensus_queue.cc:237] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:31.356982 19883 consensus_queue.cc:1059] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.264s
I20260709 13:20:31.375154 19802 raft_consensus.cc:697] T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860 [term 9 LEADER]: Becoming Leader. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
W20260709 13:20:31.452661 19869 log.cc:927] Time spent T 746aabf856ab401c9ef5e859371e7b67 P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
W20260709 13:20:31.453380 16789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 295869us
I20260709 13:20:31.419672 19861 raft_consensus.cc:493] T 8c11074579b046d99e585c2818e18a2c P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:31.360838 19726 raft_consensus.cc:515] T 68a6838d82ef4cac83507f906139d90f P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:31.452916 16047 catalog_manager.cc:5697] T 2794af5b99984efebdee564035841cb0 P cb82cc7bd6db46d980c75237a1402860 reported cstate change: term changed from 2 to 6, leader changed from 90f3da57796b488c9087a2f523adf045 (127.14.177.131) to cb82cc7bd6db46d980c75237a1402860 (127.14.177.132). New cstate: current_term: 6 leader_uuid: "cb82cc7bd6db46d980c75237a1402860" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:20:31.454249 16217 raft_consensus.cc:1240] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [term 9 LEADER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 8. Current term is 9. Ops: [8.9-8.9]
I20260709 13:20:31.452708 16265 leader_election.cc:304] T 5a959dbecc2648eb85d1cc550f5a3245 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 647a343a542d4a139fe106c19095129b
W20260709 13:20:31.455493 16117 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:31.447829 16222 raft_consensus.cc:3055] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:20:31.473922 16222 raft_consensus.cc:740] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:20:31.464322 19696 leader_election.cc:290] T 5a96373bb0e14dce8a266b2fc6c58507 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 9 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
W20260709 13:20:31.465113 19667 consensus_meta.cc:287] T ee9ed67006144797b1de6ea8c2013a68 P 1ca2725d290e47d68b1a4b609c9816ac: Time spent flushing consensus metadata: real 0.810s	user 0.011s	sys 0.004s
I20260709 13:20:31.435056 19699 raft_consensus.cc:3060] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:20:31.494621 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 65445.6us
I20260709 13:20:31.437651 19751 raft_consensus.cc:493] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:31.470269 16220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a96373bb0e14dce8a266b2fc6c58507" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 9 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:20:31.496675 16220 raft_consensus.cc:3060] T 5a96373bb0e14dce8a266b2fc6c58507 P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:20:31.494729 16117 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:31.383586 19816 raft_consensus.cc:515] T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:31.497473 16047 catalog_manager.cc:5697] T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860 reported cstate change: term changed from 6 to 8, leader changed from 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130) to cb82cc7bd6db46d980c75237a1402860 (127.14.177.132). New cstate: current_term: 8 leader_uuid: "cb82cc7bd6db46d980c75237a1402860" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:20:31.504922 16218 raft_consensus.cc:2410] T ee9ed67006144797b1de6ea8c2013a68 P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 for term 7 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 6.
I20260709 13:20:31.471081 19883 raft_consensus.cc:3055] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:20:31.456840 19568 raft_consensus.cc:2749] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [term 12 FOLLOWER]: Leader pre-election lost for term 11. Reason: Vote denied by peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741) with higher term. Message: Invalid argument: T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 for earlier term 11. Current term is 12.
I20260709 13:20:31.558135 16047 catalog_manager.cc:5697] T c9944d5e4b8c4109af9955c4f08c9095 P cb82cc7bd6db46d980c75237a1402860 reported cstate change: term changed from 5 to 7. New cstate: current_term: 7 leader_uuid: "cb82cc7bd6db46d980c75237a1402860" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:20:31.559705 16821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:31.606187 16821 raft_consensus.cc:2468] T 19baa7bc367844c288bca8f81439c413 P cb82cc7bd6db46d980c75237a1402860 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 9.
I20260709 13:20:31.457697 19540 consensus_queue.cc:1048] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.080s
W20260709 13:20:31.604476 16115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 76078.9us
I20260709 13:20:31.505323 19751 raft_consensus.cc:515] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:20:31.604524 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 74915.2us
I20260709 13:20:31.606554 16047 catalog_manager.cc:5697] T 26354361e9b044cea1c2b86c4882fe92 P cb82cc7bd6db46d980c75237a1402860 reported cstate change: term changed from 2 to 6, leader changed from 90f3da57796b488c9087a2f523adf045 (127.14.177.131) to cb82cc7bd6db46d980c75237a1402860 (127.14.177.132). New cstate: current_term: 6 leader_uuid: "cb82cc7bd6db46d980c75237a1402860" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:20:31.458405 19695 raft_consensus.cc:515] T 434bff3082d141838068f323bdb0d1ed P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:20:31.626204 16116 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:31.528345 19885 raft_consensus.cc:493] T 3a7ca61592e848ea9b0fb81a2ce33cdc P cb82cc7bd6db46d980c75237a1402860 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:20:31.633620 16147 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40844 (request call id 2192) took 1434 ms (client timeout 1895 ms). Trace:
I20260709 13:20:31.533947 19883 raft_consensus.cc:740] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:20:31.499963 19861 raft_consensus.cc:515] T 8c11074579b046d99e585c2818e18a2c P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:31.504468 19529 consensus_queue.cc:1048] T 8d4ab870a05347868956aa8b841b2981 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 2, Time since last communication: 0.104s
I20260709 13:20:31.681715 19859 leader_election.cc:290] T 8a878999c0024d2abe517484180290fc P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:31.655887 16115 leader_election.cc:304] T 57a744f5aaf944678a63469cbf514015 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860
I20260709 13:20:31.604859 16816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b7d1ab519044eda2d5e04a07242f25" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 8 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:31.660365 16220 raft_consensus.cc:2468] T 5a96373bb0e14dce8a266b2fc6c58507 P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 9.
W20260709 13:20:31.670001 19901 log.cc:927] Time spent T 57a744f5aaf944678a63469cbf514015 P cb82cc7bd6db46d980c75237a1402860: Append to log took a long time: real 0.109s	user 0.000s	sys 0.000s
W20260709 13:20:31.670737 19877 log.cc:927] Time spent T 18ad7e3af5504571a69d10c00750a01c P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.341s	user 0.000s	sys 0.010s
W20260709 13:20:31.687559 19897 log.cc:927] Time spent T 5a959dbecc2648eb85d1cc550f5a3245 P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 0.083s	user 0.000s	sys 0.001s
I20260709 13:20:31.683277 19802 consensus_queue.cc:237] T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:31.688107 19861 leader_election.cc:290] T 8c11074579b046d99e585c2818e18a2c P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
W20260709 13:20:31.688597 19856 consensus_meta.cc:287] T d2ab0ac8347641e6955e9bce7d90c1a2 P 1ca2725d290e47d68b1a4b609c9816ac: Time spent flushing consensus metadata: real 0.542s	user 0.005s	sys 0.006s
I20260709 13:20:31.689250 16790 leader_election.cc:400] T 806d7690c2b644959152074bb2cab0a7 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 7 pre-election: Vote denied by peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741) with higher term. Message: Invalid argument: T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 for earlier term 7. Current term is 8.
I20260709 13:20:31.724931 16790 leader_election.cc:403] T 806d7690c2b644959152074bb2cab0a7 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 7 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:20:31.690605 19664 raft_consensus.cc:3060] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:20:31.691555 19894 consensus_queue.cc:1048] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:20:31.662868 19785 consensus_queue.cc:1059] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.002s
W20260709 13:20:31.665469 16147 rpcz_store.cc:269] 0709 13:20:30.199028 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:30.201074 (+  2046us) service_pool.cc:224] Handling call
0709 13:20:31.558415 (+1357341us) spinlock_profiling.cc:243] Waited 816 ms on lock 0x7b540028e454. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:20:31.633552 (+ 75137us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1876116224}
W20260709 13:20:31.720853 16115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 65060.9us
I20260709 13:20:31.737227 16115 leader_election.cc:304] T d16dfad0c3b540e5b4b86492ddf87208 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac
I20260709 13:20:31.721493 16821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74ca8140f35c4ae29e8a664f6bdfc829" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:20:31.723757 16816 raft_consensus.cc:2410] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b for term 8 because replica has last-logged OpId of term: 6 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 8.
W20260709 13:20:31.741436 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 319086us
I20260709 13:20:31.717047 19532 leader_election.cc:290] T 9c61a39fd4f84424b129c1a6f9ba404d P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:31.737458 19568 raft_consensus.cc:3060] T 806d7690c2b644959152074bb2cab0a7 P cb82cc7bd6db46d980c75237a1402860 [term 7 LEARNER]: Advancing to term 8
I20260709 13:20:31.735239 19197 raft_consensus.cc:2749] T 57a744f5aaf944678a63469cbf514015 P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:20:31.749047 15046 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:20:31.762892 16297 raft_consensus.cc:1275] T 84f2da98864b4425bf7c0055951fdd4b P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 13:20:31.735277 19856 raft_consensus.cc:515] T d2ab0ac8347641e6955e9bce7d90c1a2 P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:31.740095 19898 leader_election.cc:290] T 19baa7bc367844c288bca8f81439c413 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:31.766508 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4f65cdf00b748bb963ee6e81ef43b79" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:20:31.766858 19856 leader_election.cc:290] T d2ab0ac8347641e6955e9bce7d90c1a2 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 9 election: Requested vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:31.767115 16293 raft_consensus.cc:3060] T c4f65cdf00b748bb963ee6e81ef43b79 P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:20:31.768131 19751 leader_election.cc:290] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:31.768769 19890 raft_consensus.cc:493] T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:31.769630 19893 raft_consensus.cc:1064] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045: attempting to promote NON_VOTER cb82cc7bd6db46d980c75237a1402860 to VOTER
I20260709 13:20:31.769273 19890 raft_consensus.cc:515] T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:20:31.770056 19893 raft_consensus.cc:1068] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045: Unable to promote non-voter cb82cc7bd6db46d980c75237a1402860: Illegal state: Leader has not yet committed an operation in its own term
W20260709 13:20:31.772857 16150 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46224 (request call id 520) took 1266 ms (client timeout 1619 ms). Trace:
W20260709 13:20:31.773207 16150 rpcz_store.cc:269] 0709 13:20:30.506400 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:30.511303 (+  4903us) service_pool.cc:224] Handling call
0709 13:20:31.772740 (+1261437us) spinlock_profiling.cc:243] Waited 419 ms on lock 0x7b540041dcd4. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:20:31.772817 (+    77us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":964169600}
I20260709 13:20:31.774318 16823 raft_consensus.cc:1275] T 829b4ad8c5a54eababeea1181b960d4b P cb82cc7bd6db46d980c75237a1402860 [term 9 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 7 index: 9. Preceding OpId from leader: term: 9 index: 9. (term mismatch)
I20260709 13:20:31.774632 16823 pending_rounds.cc:85] T 829b4ad8c5a54eababeea1181b960d4b P cb82cc7bd6db46d980c75237a1402860: Aborting all ops after (but not including) 8
I20260709 13:20:31.774843 16823 pending_rounds.cc:107] T 829b4ad8c5a54eababeea1181b960d4b P cb82cc7bd6db46d980c75237a1402860: Aborting uncommitted NO_OP operation due to leader change: 7.9
I20260709 13:20:31.717360 19885 raft_consensus.cc:515] T 3a7ca61592e848ea9b0fb81a2ce33cdc P cb82cc7bd6db46d980c75237a1402860 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:31.775168 16823 raft_consensus.cc:2889] T 829b4ad8c5a54eababeea1181b960d4b P cb82cc7bd6db46d980c75237a1402860 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:20:31.747346 19785 consensus_queue.cc:1243] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Peer 647a343a542d4a139fe106c19095129b log is divergent from this leader: its last log entry 9.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 13:20:31.760016 19345 raft_consensus.cc:493] T b87147731a68400e8cce9b879daa7c5d P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:20:31.786069 16298 raft_consensus.cc:1275] T d16dfad0c3b540e5b4b86492ddf87208 P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:20:31.786605 19345 raft_consensus.cc:515] T b87147731a68400e8cce9b879daa7c5d P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:31.783237 16293 raft_consensus.cc:2468] T c4f65cdf00b748bb963ee6e81ef43b79 P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 11.
I20260709 13:20:31.784652 16294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d4ab870a05347868956aa8b841b2981" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:20:31.789728 16294 raft_consensus.cc:3060] T 8d4ab870a05347868956aa8b841b2981 P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:20:31.790849 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 340919us
I20260709 13:20:31.791287 19914 raft_consensus.cc:493] T 2794af5b99984efebdee564035841cb0 P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:20:31.791698 16265 leader_election.cc:304] T a378d7f796f94615b3b347119a9880fd P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860
I20260709 13:20:31.791865 19914 raft_consensus.cc:515] T 2794af5b99984efebdee564035841cb0 P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:31.794023 19914 leader_election.cc:290] T 2794af5b99984efebdee564035841cb0 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:31.794543 19892 raft_consensus.cc:2764] T 5a959dbecc2648eb85d1cc550f5a3245 P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Leader election decision vote started in defunct term 6: won
I20260709 13:20:31.795112 19876 raft_consensus.cc:2749] T 7c2095a5f4ad49408dbc7f69c606120d P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:20:31.796631 19707 consensus_meta.cc:287] T af57996a8be84d398f0f023e8d08ef77 P 647a343a542d4a139fe106c19095129b: Time spent flushing consensus metadata: real 0.669s	user 0.009s	sys 0.003s
I20260709 13:20:31.807013 19695 leader_election.cc:290] T 434bff3082d141838068f323bdb0d1ed P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:31.807593 19893 raft_consensus.cc:493] T ada5a004ba7c46819051e3ccec71ffdb P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:31.807751 16291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:20:31.808866 16291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8158bbd17b994d2eb96f560bd0f1b88d" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:31.785485 19197 raft_consensus.cc:493] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:31.771492 19890 leader_election.cc:290] T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:20:31.786072 16297 raft_consensus.cc:1275] T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:20:31.802073 16292 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48674 (request call id 2462) took 1101 ms. Trace:
I20260709 13:20:31.764348 19532 raft_consensus.cc:2804] T d16dfad0c3b540e5b4b86492ddf87208 P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:20:31.802939 16296 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48674 (request call id 2453) took 1323 ms. Trace:
I20260709 13:20:31.805918 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74ca8140f35c4ae29e8a664f6bdfc829" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:20:31.820405 16299 raft_consensus.cc:1275] T 8cc959fb7f704fba9009c0595467d074 P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 13:20:31.809587 19657 raft_consensus.cc:2749] T a378d7f796f94615b3b347119a9880fd P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:20:31.799007 16294 raft_consensus.cc:2468] T 8d4ab870a05347868956aa8b841b2981 P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 10.
I20260709 13:20:31.824708 19785 consensus_queue.cc:1048] T 3884f864a96d428785895e5463e5728f P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:20:31.818167 16296 rpcz_store.cc:276] 0709 13:20:30.479755 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:30.486508 (+  6753us) service_pool.cc:224] Handling call
0709 13:20:30.509081 (+ 22573us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:20:30.595369 (+ 86288us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:30.595407 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:20:30.633669 (+ 38262us) log.cc:844] Serialized 26 byte log entry
0709 13:20:31.008887 (+375218us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:20:31.008948 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:31.009051 (+   103us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:20:31.802671 (+793620us) raft_consensus.cc:1685] finished
0709 13:20:31.802698 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:31.802867 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":323518,"threads_started":1,"wal-append.queue_time_us":424326}
I20260709 13:20:31.820164 19532 raft_consensus.cc:493] T d16dfad0c3b540e5b4b86492ddf87208 P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:31.832259 19532 raft_consensus.cc:3060] T d16dfad0c3b540e5b4b86492ddf87208 P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:20:31.832648 16265 leader_election.cc:304] T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:20:31.834295 16265 leader_election.cc:304] T 8638ee363273426db1393dd81e5c7b72 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860
I20260709 13:20:31.835556 19892 raft_consensus.cc:2749] T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:20:31.836266 19892 raft_consensus.cc:2749] T 8638ee363273426db1393dd81e5c7b72 P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:20:31.838281 19418 raft_consensus.cc:3055] T 8d4ab870a05347868956aa8b841b2981 P 647a343a542d4a139fe106c19095129b [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:20:31.838717 19418 raft_consensus.cc:740] T 8d4ab870a05347868956aa8b841b2981 P 647a343a542d4a139fe106c19095129b [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:20:31.839694 19418 consensus_queue.cc:260] T 8d4ab870a05347868956aa8b841b2981 P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:31.841086 19418 raft_consensus.cc:3060] T 8d4ab870a05347868956aa8b841b2981 P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:20:31.846639 16790 leader_election.cc:304] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860; no voters: 90f3da57796b488c9087a2f523adf045
I20260709 13:20:31.814414 16292 rpcz_store.cc:276] 0709 13:20:30.700785 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:30.703262 (+  2477us) service_pool.cc:224] Handling call
0709 13:20:30.707249 (+  3987us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:20:31.009318 (+302069us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:20:31.066346 (+ 57028us) log.cc:844] Serialized 14 byte log entry
0709 13:20:31.247949 (+181603us) spinlock_profiling.cc:243] Waited 6.07 ms on lock 0x7b180042e320. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b693ade1 00007f69b69475b1 00007f69ada54e07 00007f69b68afab9 00007f69ada7169b 00007f69ada6f93f 00007f69ada71c02 00007f69b06f920b 00007f69b06fed23 00007f69b06ff23a 00007f69b0b69a2a
0709 13:20:31.743152 (+495203us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:20:31.743723 (+   571us) log.cc:844] Serialized 26 byte log entry
0709 13:20:31.743893 (+   170us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:20:31.785843 (+ 41950us) log.cc:844] Serialized 14 byte log entry
0709 13:20:31.786108 (+   265us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:31.786182 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:20:31.801921 (+ 15739us) raft_consensus.cc:1685] finished
0709 13:20:31.801945 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:31.802033 (+    88us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":373695,"prepare.queue_time_us":51463,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":39,"raft.queue_time_us":486205,"raft.run_cpu_time_us":347,"raft.run_wall_time_us":26174,"spinlock_wait_cycles":13978880,"thread_start_us":464342,"threads_started":3,"wal-append.queue_time_us":417545}
W20260709 13:20:31.861745 19912 log.cc:927] Time spent T 18ad7e3af5504571a69d10c00750a01c P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
I20260709 13:20:31.862063 19529 raft_consensus.cc:493] T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:31.862655 19529 raft_consensus.cc:515] T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:31.863229 19822 raft_consensus.cc:493] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:31.863718 19822 raft_consensus.cc:515] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:31.864833 19529 leader_election.cc:290] T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:31.865777 19822 leader_election.cc:290] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:31.810163 19893 raft_consensus.cc:515] T ada5a004ba7c46819051e3ccec71ffdb P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:31.869488 19893 leader_election.cc:290] T ada5a004ba7c46819051e3ccec71ffdb P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:31.810184 19197 raft_consensus.cc:515] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } }
I20260709 13:20:31.869856 19532 raft_consensus.cc:515] T d16dfad0c3b540e5b4b86492ddf87208 P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:31.872133 19197 leader_election.cc:290] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
W20260709 13:20:31.873507 16191 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:31.879657 16219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c61a39fd4f84424b129c1a6f9ba404d" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:31.878824 16116 leader_election.cc:304] T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
W20260709 13:20:31.884213 16117 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:31.890268 16223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4f65cdf00b748bb963ee6e81ef43b79" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:31.891208 16223 raft_consensus.cc:2393] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 11: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
I20260709 13:20:31.893875 16116 leader_election.cc:304] T 5a96373bb0e14dce8a266b2fc6c58507 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
I20260709 13:20:31.899633 16265 leader_election.cc:304] T 8d4ab870a05347868956aa8b841b2981 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:20:31.904075 16219 raft_consensus.cc:1240] T d2ab0ac8347641e6955e9bce7d90c1a2 P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 8. Current term is 9. Ops: []
I20260709 13:20:31.873379 19673 raft_consensus.cc:2804] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:20:31.909940 16221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68a6838d82ef4cac83507f906139d90f" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:31.914534 16221 raft_consensus.cc:2468] T 68a6838d82ef4cac83507f906139d90f P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 9.
I20260709 13:20:31.892387 19785 raft_consensus.cc:493] T 8638ee363273426db1393dd81e5c7b72 P cb82cc7bd6db46d980c75237a1402860 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:20:31.911063 16115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 160619us
I20260709 13:20:31.915889 16225 raft_consensus.cc:1240] T ecc1e363230043fe9fef390f0b979f20 P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 8. Current term is 9. Ops: [8.6-8.6]
I20260709 13:20:31.899118 19197 raft_consensus.cc:2804] T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:20:31.900905 19568 raft_consensus.cc:2749] T 806d7690c2b644959152074bb2cab0a7 P cb82cc7bd6db46d980c75237a1402860 [term 8 LEARNER]: Leader pre-election lost for term 7. Reason: Vote denied by peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741) with higher term. Message: Invalid argument: T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 for earlier term 7. Current term is 8.
I20260709 13:20:31.920396 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e65676b21b748728ad39668cfe722ec" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:31.918794 16223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5718c623ebee4d2bb770f0f9ddb2f164" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:20:31.923840 16223 raft_consensus.cc:2393] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 in current term 9: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
I20260709 13:20:31.906060 19789 raft_consensus.cc:2804] T 5a96373bb0e14dce8a266b2fc6c58507 P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:20:31.909761 19893 raft_consensus.cc:2749] T 8d4ab870a05347868956aa8b841b2981 P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:20:31.928897 16151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3884f864a96d428785895e5463e5728f" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:20:31.927274 16219 raft_consensus.cc:1275] T 5a959dbecc2648eb85d1cc550f5a3245 P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:20:31.929034 16145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5718c623ebee4d2bb770f0f9ddb2f164" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:31.930032 16145 raft_consensus.cc:2468] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 8.
I20260709 13:20:31.930473 16221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:31.931207 16221 raft_consensus.cc:2468] T 19baa7bc367844c288bca8f81439c413 P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 9.
I20260709 13:20:31.921111 16115 leader_election.cc:304] T c0eab25ce4c64e018a041beeda2867db P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac
I20260709 13:20:31.932598 16147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5718c623ebee4d2bb770f0f9ddb2f164" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:20:31.932921 16150 raft_consensus.cc:2468] T 9e65676b21b748728ad39668cfe722ec P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 9.
I20260709 13:20:31.933171 16147 raft_consensus.cc:3060] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:20:31.934059 16225 raft_consensus.cc:1275] T e5ad4eb281564936a59f41e387a47156 P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:20:31.938766 16144 raft_consensus.cc:1240] T 57a744f5aaf944678a63469cbf514015 P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 7. Current term is 8. Ops: [7.8-7.8]
I20260709 13:20:31.939949 19809 raft_consensus.cc:3055] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:20:31.940344 19809 raft_consensus.cc:740] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:20:31.941124 19809 consensus_queue.cc:260] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:31.942353 19809 raft_consensus.cc:3060] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:20:31.943555 19885 leader_election.cc:290] T 3a7ca61592e848ea9b0fb81a2ce33cdc P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:31.943728 19726 leader_election.cc:290] T 68a6838d82ef4cac83507f906139d90f P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:31.935685 19903 consensus_queue.cc:1048] T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.009s
I20260709 13:20:31.936606 16148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31969392ccdb43b491911a319e8bc7e1" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:31.946390 16148 raft_consensus.cc:2468] T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 8.
W20260709 13:20:31.946753 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 55278.3us
I20260709 13:20:31.948621 16147 raft_consensus.cc:2468] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 9.
I20260709 13:20:31.922107 19532 leader_election.cc:290] T d16dfad0c3b540e5b4b86492ddf87208 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 9 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:31.949046 19789 raft_consensus.cc:697] T 5a96373bb0e14dce8a266b2fc6c58507 P 647a343a542d4a139fe106c19095129b [term 9 LEADER]: Becoming Leader. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:20:31.952198 19915 raft_consensus.cc:2804] T c0eab25ce4c64e018a041beeda2867db P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:20:31.924225 19197 raft_consensus.cc:493] T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:31.924805 19673 raft_consensus.cc:697] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860 [term 7 LEADER]: Becoming Leader. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:20:31.962636 19664 raft_consensus.cc:2749] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:20:31.927737 19802 raft_consensus.cc:493] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:20:31.957017 16221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806d7690c2b644959152074bb2cab0a7" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:31.967857 16221 raft_consensus.cc:2468] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 8.
I20260709 13:20:31.959530 16223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d16dfad0c3b540e5b4b86492ddf87208" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:20:31.961751 19893 consensus_queue.cc:1048] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:20:31.962488 19197 raft_consensus.cc:3060] T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:20:31.968977 16151 raft_consensus.cc:2468] T 3884f864a96d428785895e5463e5728f P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 7.
I20260709 13:20:31.970037 16265 leader_election.cc:304] T 8cc959fb7f704fba9009c0595467d074 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:20:31.963532 19673 consensus_queue.cc:237] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:31.961277 16145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:31.973624 16145 raft_consensus.cc:2468] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 9.
W20260709 13:20:31.974221 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 51100.4us
W20260709 13:20:31.971191 16115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 50286.6us
I20260709 13:20:31.923200 19785 raft_consensus.cc:515] T 8638ee363273426db1393dd81e5c7b72 P cb82cc7bd6db46d980c75237a1402860 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:31.976469 19657 raft_consensus.cc:2749] T 8cc959fb7f704fba9009c0595467d074 P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:20:31.977551 16115 leader_election.cc:304] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: cb82cc7bd6db46d980c75237a1402860
I20260709 13:20:31.990324 19532 raft_consensus.cc:2804] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:20:31.990828 19532 raft_consensus.cc:697] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b [term 11 LEADER]: Becoming Leader. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:20:31.991657 19532 consensus_queue.cc:237] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:31.992331 19789 consensus_queue.cc:237] T 5a96373bb0e14dce8a266b2fc6c58507 P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 7.3, Last appended by leader: 3, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:31.977299 16116 leader_election.cc:304] T 68a6838d82ef4cac83507f906139d90f P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
I20260709 13:20:32.002992 18599 raft_consensus.cc:2804] T 68a6838d82ef4cac83507f906139d90f P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:20:32.003643 18599 raft_consensus.cc:493] T 68a6838d82ef4cac83507f906139d90f P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:32.003993 18599 raft_consensus.cc:3060] T 68a6838d82ef4cac83507f906139d90f P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:20:32.010205 16191 leader_election.cc:304] T 66f75124215f4ca6bef831e174a792d8 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
I20260709 13:20:32.012007 16222 consensus_queue.cc:260] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:32.013209 16222 raft_consensus.cc:3060] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:20:32.013242 19863 raft_consensus.cc:2749] T 66f75124215f4ca6bef831e174a792d8 P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:20:32.013733 16292 raft_consensus.cc:1240] T 02838c86531d4f98bf58d1c22b8baf77 P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 8. Current term is 9. Ops: []
I20260709 13:20:32.015872 16292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a96373bb0e14dce8a266b2fc6c58507" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 9 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:32.018195 16292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f2da98864b4425bf7c0055951fdd4b" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:32.018837 16292 raft_consensus.cc:2468] T 84f2da98864b4425bf7c0055951fdd4b P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 7.
I20260709 13:20:32.021864 16294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a96373bb0e14dce8a266b2fc6c58507" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 9 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:20:32.023813 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c61a39fd4f84424b129c1a6f9ba404d" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:32.024420 16299 raft_consensus.cc:2468] T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 8.
I20260709 13:20:32.025924 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68a6838d82ef4cac83507f906139d90f" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:32.025663 18599 raft_consensus.cc:515] T 68a6838d82ef4cac83507f906139d90f P 647a343a542d4a139fe106c19095129b [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:32.030247 16296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:32.032831 19922 consensus_queue.cc:1048] T e5ad4eb281564936a59f41e387a47156 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:20:32.033025 16291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4f65cdf00b748bb963ee6e81ef43b79" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:32.035212 16116 leader_election.cc:304] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
I20260709 13:20:32.040060 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4dc6b0541ca431ebc757beac00521f1" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:32.040933 16293 raft_consensus.cc:2468] T e4dc6b0541ca431ebc757beac00521f1 P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 8.
I20260709 13:20:32.042619 16291 raft_consensus.cc:2393] T c4f65cdf00b748bb963ee6e81ef43b79 P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 11: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
I20260709 13:20:31.975021 19915 raft_consensus.cc:697] T c0eab25ce4c64e018a041beeda2867db P 647a343a542d4a139fe106c19095129b [term 9 LEADER]: Becoming Leader. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
W20260709 13:20:32.032581 19882 log.cc:927] Time spent T 87ac0d1f7e8a419f9bcbc04aee0e067e P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.675s	user 0.002s	sys 0.003s
I20260709 13:20:32.048943 18599 leader_election.cc:290] T 68a6838d82ef4cac83507f906139d90f P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 10 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:32.049216 16296 raft_consensus.cc:1240] T 8d4ab870a05347868956aa8b841b2981 P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 9. Current term is 10. Ops: [9.5-9.5]
I20260709 13:20:32.047318 16790 leader_election.cc:304] T ada5a004ba7c46819051e3ccec71ffdb P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:20:32.055743 16293 raft_consensus.cc:1275] T e5ad4eb281564936a59f41e387a47156 P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:20:32.056681 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31969392ccdb43b491911a319e8bc7e1" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:32.057219 16293 raft_consensus.cc:2468] T 31969392ccdb43b491911a319e8bc7e1 P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 8.
I20260709 13:20:32.057999 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d16dfad0c3b540e5b4b86492ddf87208" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:20:32.058480 16293 raft_consensus.cc:3060] T d16dfad0c3b540e5b4b86492ddf87208 P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:20:32.059398 16291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434bff3082d141838068f323bdb0d1ed" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:32.060438 16291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806d7690c2b644959152074bb2cab0a7" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:32.062533 19919 raft_consensus.cc:493] T 8cc959fb7f704fba9009c0595467d074 P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:32.063021 19919 raft_consensus.cc:515] T 8cc959fb7f704fba9009c0595467d074 P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:32.065290 19919 leader_election.cc:290] T 8cc959fb7f704fba9009c0595467d074 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:32.069011 16149 raft_consensus.cc:1240] T 68a6838d82ef4cac83507f906139d90f P 647a343a542d4a139fe106c19095129b [term 10 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 9. Current term is 10. Ops: []
I20260709 13:20:32.071466 16292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2794af5b99984efebdee564035841cb0" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:32.071671 19915 consensus_queue.cc:237] T c0eab25ce4c64e018a041beeda2867db P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 8.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:32.072521 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e9b2bf7011f4ae688a79d7bb5a447b4" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:20:32.074087 16149 raft_consensus.cc:2393] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 7: Already voted for candidate cb82cc7bd6db46d980c75237a1402860 in this term.
I20260709 13:20:32.010847 19414 consensus_queue.cc:1048] T 8a878999c0024d2abe517484180290fc P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.023s
I20260709 13:20:32.075572 19414 consensus_queue.cc:1243] T 8a878999c0024d2abe517484180290fc P 90f3da57796b488c9087a2f523adf045 [LEADER]: Peer 1ca2725d290e47d68b1a4b609c9816ac log is divergent from this leader: its last log entry 4.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260709 13:20:32.076112 19785 leader_election.cc:290] T 8638ee363273426db1393dd81e5c7b72 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:32.072235 16292 raft_consensus.cc:2393] T 2794af5b99984efebdee564035841cb0 P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 7: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
I20260709 13:20:32.075030 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fffab26427e24d398e2695aad40ca134" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:20:32.072610 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68a6838d82ef4cac83507f906139d90f" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:20:32.078225 16225 raft_consensus.cc:3060] T 68a6838d82ef4cac83507f906139d90f P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:20:32.054404 16222 raft_consensus.cc:1275] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 10. (term mismatch)
I20260709 13:20:32.082885 16222 pending_rounds.cc:85] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac: Aborting all ops after (but not including) 9
I20260709 13:20:32.083210 16222 pending_rounds.cc:107] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac: Aborting uncommitted NO_OP operation due to leader change: 11.10
I20260709 13:20:32.083513 16222 raft_consensus.cc:2889] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:20:31.973558 19802 raft_consensus.cc:515] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:32.128696 16221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cc959fb7f704fba9009c0595467d074" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:32.047729 19884 raft_consensus.cc:2804] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 13:20:32.134636 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 87398.2us
I20260709 13:20:32.137041 16790 leader_election.cc:304] T 829b4ad8c5a54eababeea1181b960d4b P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:20:32.031981 19763 consensus_queue.cc:1048] T 5a959dbecc2648eb85d1cc550f5a3245 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.092s
I20260709 13:20:32.144456 16149 raft_consensus.cc:2468] T fffab26427e24d398e2695aad40ca134 P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 8.
I20260709 13:20:32.145511 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2794af5b99984efebdee564035841cb0" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:20:32.146140 16149 raft_consensus.cc:3060] T 2794af5b99984efebdee564035841cb0 P 647a343a542d4a139fe106c19095129b [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:20:32.148819 16115 leader_election.cc:304] T 84f2da98864b4425bf7c0055951fdd4b P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac
I20260709 13:20:32.120630 19904 raft_consensus.cc:493] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:20:32.157115 16115 leader_election.cc:304] T 9c61a39fd4f84424b129c1a6f9ba404d P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac
I20260709 13:20:32.134361 19898 raft_consensus.cc:3060] T ada5a004ba7c46819051e3ccec71ffdb P cb82cc7bd6db46d980c75237a1402860 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:20:32.128424 19760 raft_consensus.cc:493] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:32.134311 19884 raft_consensus.cc:493] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:32.128165 19529 raft_consensus.cc:493] T 08c74490c40243a88e62d1760b5cd814 P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:32.167714 16293 raft_consensus.cc:2468] T d16dfad0c3b540e5b4b86492ddf87208 P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 9.
I20260709 13:20:32.147854 19751 raft_consensus.cc:493] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:20:32.171227 18599 raft_consensus.cc:2804] T 9c61a39fd4f84424b129c1a6f9ba404d P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:20:32.175823 18599 raft_consensus.cc:493] T 9c61a39fd4f84424b129c1a6f9ba404d P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:32.176312 18599 raft_consensus.cc:3060] T 9c61a39fd4f84424b129c1a6f9ba404d P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:20:32.173203 16115 leader_election.cc:304] T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:20:32.164498 19754 raft_consensus.cc:2804] T 84f2da98864b4425bf7c0055951fdd4b P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:20:32.161746 19673 raft_consensus.cc:2749] T 829b4ad8c5a54eababeea1181b960d4b P cb82cc7bd6db46d980c75237a1402860 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:20:32.182090 16222 raft_consensus.cc:1275] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
W20260709 13:20:32.189884 16265 connection.cc:256] RPC call timeout handler was delayed by -2.7447881829998551s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1885s for a response
I20260709 13:20:32.191794 19657 raft_consensus.cc:493] T 1df93779209748b9860f6d893dd3de53 P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:32.192312 19657 raft_consensus.cc:515] T 1df93779209748b9860f6d893dd3de53 P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:32.193605 16149 raft_consensus.cc:2468] T 2794af5b99984efebdee564035841cb0 P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 7.
I20260709 13:20:32.194396 19657 leader_election.cc:290] T 1df93779209748b9860f6d893dd3de53 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:20:32.197394 16145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ad5dbfa99ee4b8d9e28be92b3f36db5" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:20:32.198850 16145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f5f89186204242bf377f7c42b2e2da" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:32.199150 19529 raft_consensus.cc:515] T 08c74490c40243a88e62d1760b5cd814 P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:32.200116 16145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02838c86531d4f98bf58d1c22b8baf77" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:20:32.196446 16790 leader_election.cc:304] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:20:32.201257 16145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:20:32.201642 19809 raft_consensus.cc:2804] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:20:32.202191 19809 raft_consensus.cc:697] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 [term 9 LEADER]: Becoming Leader. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:20:32.202459 16145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a96373bb0e14dce8a266b2fc6c58507" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 8 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:32.203022 19809 consensus_queue.cc:237] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.9, Last appended by leader: 9, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:32.206825 19785 consensus_queue.cc:1059] T 57a744f5aaf944678a63469cbf514015 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:20:32.207944 19923 raft_consensus.cc:3055] T 57a744f5aaf944678a63469cbf514015 P cb82cc7bd6db46d980c75237a1402860 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:20:32.208370 19923 raft_consensus.cc:740] T 57a744f5aaf944678a63469cbf514015 P cb82cc7bd6db46d980c75237a1402860 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:20:32.209239 19923 consensus_queue.cc:260] T 57a744f5aaf944678a63469cbf514015 P cb82cc7bd6db46d980c75237a1402860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:32.209775 16143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d4ab870a05347868956aa8b841b2981" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:20:32.210637 19923 raft_consensus.cc:3060] T 57a744f5aaf944678a63469cbf514015 P cb82cc7bd6db46d980c75237a1402860 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:20:32.210820 16143 raft_consensus.cc:2410] T 8d4ab870a05347868956aa8b841b2981 P 647a343a542d4a139fe106c19095129b [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac for term 10 because replica has last-logged OpId of term: 9 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 4.
I20260709 13:20:32.170068 19884 raft_consensus.cc:3060] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:20:32.211558 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8638ee363273426db1393dd81e5c7b72" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:32.216547 16151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c74490c40243a88e62d1760b5cd814" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:32.182709 19754 raft_consensus.cc:493] T 84f2da98864b4425bf7c0055951fdd4b P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:32.217787 19754 raft_consensus.cc:3060] T 84f2da98864b4425bf7c0055951fdd4b P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:20:32.218528 16790 leader_election.cc:304] T 9e65676b21b748728ad39668cfe722ec P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860; no voters: 90f3da57796b488c9087a2f523adf045
I20260709 13:20:32.170214 19760 raft_consensus.cc:515] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:32.219919 19673 raft_consensus.cc:2764] T 9e65676b21b748728ad39668cfe722ec P cb82cc7bd6db46d980c75237a1402860 [term 10 FOLLOWER]: Leader pre-election decision vote started in defunct term 9: won
I20260709 13:20:32.221400 19657 consensus_queue.cc:1048] T 8a878999c0024d2abe517484180290fc P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:20:32.222114 19414 consensus_queue.cc:1059] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:20:32.217228 16151 raft_consensus.cc:2468] T 08c74490c40243a88e62d1760b5cd814 P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 8.
I20260709 13:20:32.223017 19657 raft_consensus.cc:3055] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:20:32.223491 19657 raft_consensus.cc:740] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:20:32.223574 16265 leader_election.cc:304] T c0eab25ce4c64e018a041beeda2867db P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:20:32.224046 16151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4f65cdf00b748bb963ee6e81ef43b79" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:20:32.224691 16151 raft_consensus.cc:3060] T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:20:32.224606 19657 consensus_queue.cc:260] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } }
I20260709 13:20:32.225610 19883 consensus_queue.cc:260] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.8, Last appended by leader: 8, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:32.226651 19657 raft_consensus.cc:3060] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:20:32.164546 19532 consensus_queue.cc:1059] T 02838c86531d4f98bf58d1c22b8baf77 P 647a343a542d4a139fe106c19095129b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.031s
I20260709 13:20:32.238729 16292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a878999c0024d2abe517484180290fc" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 9 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:32.239989 16296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c11074579b046d99e585c2818e18a2c" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 10 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:32.244807 16293 raft_consensus.cc:1275] T 66ae176961b74813a58bc15c9c4c6f2a P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
W20260709 13:20:32.261113 16117 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:32.261013 16143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5d59407eb545b895d8d922e0dd69fd" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:32.262022 16117 leader_election.cc:304] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860; no voters: 90f3da57796b488c9087a2f523adf045
W20260709 13:20:32.264536 16115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 91399.3us
I20260709 13:20:32.265287 16221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7ca61592e848ea9b0fb81a2ce33cdc" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:32.266067 16221 raft_consensus.cc:2468] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 11.
W20260709 13:20:32.275516 16789 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:32.277755 16821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ab0ac8347641e6955e9bce7d90c1a2" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:20:32.278771 18599 raft_consensus.cc:515] T 9c61a39fd4f84424b129c1a6f9ba404d P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:32.277669 16143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26354361e9b044cea1c2b86c4882fe92" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:20:32.282084 16151 raft_consensus.cc:2468] T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 11.
I20260709 13:20:32.282835 16151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5718c623ebee4d2bb770f0f9ddb2f164" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:20:32.283814 16823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c11074579b046d99e585c2818e18a2c" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 10 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:32.283761 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87147731a68400e8cce9b879daa7c5d" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:32.284729 16149 raft_consensus.cc:2393] T b87147731a68400e8cce9b879daa7c5d P 647a343a542d4a139fe106c19095129b [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 10: Already voted for candidate cb82cc7bd6db46d980c75237a1402860 in this term.
I20260709 13:20:32.284771 16291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ab0ac8347641e6955e9bce7d90c1a2" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:20:32.285307 16151 raft_consensus.cc:2393] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 9: Already voted for candidate cb82cc7bd6db46d980c75237a1402860 in this term.
I20260709 13:20:32.227058 19883 raft_consensus.cc:3060] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:20:32.287844 16143 raft_consensus.cc:2393] T 26354361e9b044cea1c2b86c4882fe92 P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 7: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
I20260709 13:20:32.287755 16816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7ca61592e848ea9b0fb81a2ce33cdc" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:32.292340 16190 leader_election.cc:304] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:20:32.292523 16816 raft_consensus.cc:2410] T 3a7ca61592e848ea9b0fb81a2ce33cdc P cb82cc7bd6db46d980c75237a1402860 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac for term 12 because replica has last-logged OpId of term: 10 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 9.
I20260709 13:20:32.291309 16818 raft_consensus.cc:1275] T 8c11074579b046d99e585c2818e18a2c P cb82cc7bd6db46d980c75237a1402860 [term 9 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
I20260709 13:20:32.292973 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c74490c40243a88e62d1760b5cd814" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:32.293599 16115 leader_election.cc:304] T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:20:32.291963 16821 raft_consensus.cc:1240] T 806d7690c2b644959152074bb2cab0a7 P cb82cc7bd6db46d980c75237a1402860 [term 8 LEARNER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 7. Current term is 8. Ops: []
I20260709 13:20:32.293706 16226 raft_consensus.cc:2468] T 08c74490c40243a88e62d1760b5cd814 P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 8.
W20260709 13:20:32.294751 16788 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:32.293152 16190 leader_election.cc:304] T ecc1e363230043fe9fef390f0b979f20 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860
I20260709 13:20:32.294649 16822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada5a004ba7c46819051e3ccec71ffdb" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:32.283296 19414 raft_consensus.cc:2749] T c0eab25ce4c64e018a041beeda2867db P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:20:32.297567 16820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b7d1ab519044eda2d5e04a07242f25" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:32.298269 16820 raft_consensus.cc:2468] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 6.
I20260709 13:20:32.299026 16143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e65676b21b748728ad39668cfe722ec" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:20:32.299252 16145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f82224f4564fea93572e8206bad1fe" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:32.293143 16117 leader_election.cc:304] T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860
I20260709 13:20:32.299741 16143 raft_consensus.cc:3060] T 9e65676b21b748728ad39668cfe722ec P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:20:32.263806 19863 raft_consensus.cc:493] T ee9ed67006144797b1de6ea8c2013a68 P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:32.277395 19726 raft_consensus.cc:493] T b6f07842d3aa47858df5665874c86f3a P 647a343a542d4a139fe106c19095129b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:32.304991 16221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7ca61592e848ea9b0fb81a2ce33cdc" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
W20260709 13:20:32.301865 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 78367.1us
I20260709 13:20:32.164857 19904 raft_consensus.cc:515] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:32.309093 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5ad4eb281564936a59f41e387a47156" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:20:32.303995 16825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87147731a68400e8cce9b879daa7c5d" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:32.302105 16151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c61a39fd4f84424b129c1a6f9ba404d" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:32.305781 16296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7ca61592e848ea9b0fb81a2ce33cdc" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:32.269824 19859 raft_consensus.cc:493] T af57996a8be84d398f0f023e8d08ef77 P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:20:32.313390 16222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9944d5e4b8c4109af9955c4f08c9095" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:32.314016 16222 raft_consensus.cc:2468] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 9.
I20260709 13:20:32.306474 16821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2794af5b99984efebdee564035841cb0" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:32.310376 16825 raft_consensus.cc:2393] T b87147731a68400e8cce9b879daa7c5d P cb82cc7bd6db46d980c75237a1402860 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 10: Already voted for candidate cb82cc7bd6db46d980c75237a1402860 in this term.
I20260709 13:20:32.285108 19418 raft_consensus.cc:2749] T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:20:32.311976 19726 raft_consensus.cc:515] T b6f07842d3aa47858df5665874c86f3a P 647a343a542d4a139fe106c19095129b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:32.308439 16146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4dc6b0541ca431ebc757beac00521f1" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:20:32.310868 16221 raft_consensus.cc:2393] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 11: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
I20260709 13:20:32.290206 19856 raft_consensus.cc:493] T fffab26427e24d398e2695aad40ca134 P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:32.314783 16821 raft_consensus.cc:2468] T 2794af5b99984efebdee564035841cb0 P cb82cc7bd6db46d980c75237a1402860 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 7.
I20260709 13:20:32.311113 16151 raft_consensus.cc:2393] T 9c61a39fd4f84424b129c1a6f9ba404d P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 9: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
I20260709 13:20:32.316085 16822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434bff3082d141838068f323bdb0d1ed" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:32.319511 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fffab26427e24d398e2695aad40ca134" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:32.321305 16291 raft_consensus.cc:1275] T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:20:32.196201 19751 raft_consensus.cc:515] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:20:32.323952 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 63386.1us
I20260709 13:20:32.327764 16825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31969392ccdb43b491911a319e8bc7e1" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:32.331745 16149 raft_consensus.cc:1275] T 829b4ad8c5a54eababeea1181b960d4b P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:20:32.301800 19914 raft_consensus.cc:2804] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:20:32.334136 16145 raft_consensus.cc:1275] T 8cc959fb7f704fba9009c0595467d074 P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 13:20:32.334728 16296 raft_consensus.cc:2468] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 11.
I20260709 13:20:32.336843 16149 raft_consensus.cc:1240] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b [term 11 LEADER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 10. Current term is 11. Ops: [10.8-10.8]
I20260709 13:20:32.337212 19345 leader_election.cc:290] T b87147731a68400e8cce9b879daa7c5d P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:32.337479 16115 leader_election.cc:304] T d16dfad0c3b540e5b4b86492ddf87208 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac
I20260709 13:20:32.338344 16265 leader_election.cc:304] T 434bff3082d141838068f323bdb0d1ed P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
I20260709 13:20:32.339396 19414 raft_consensus.cc:2749] T 434bff3082d141838068f323bdb0d1ed P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:20:32.339653 16265 leader_election.cc:304] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
I20260709 13:20:32.341212 16265 leader_election.cc:304] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
I20260709 13:20:32.341673 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:32.342288 16265 leader_election.cc:304] T fffab26427e24d398e2695aad40ca134 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:20:32.343297 19664 raft_consensus.cc:2749] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:20:32.343681 16222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18ad7e3af5504571a69d10c00750a01c" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:20:32.343799 19802 leader_election.cc:290] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:20:32.343993 19664 consensus_queue.cc:1059] T 68a6838d82ef4cac83507f906139d90f P 90f3da57796b488c9087a2f523adf045 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:20:32.344849 19664 raft_consensus.cc:2749] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:20:32.344861 16222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdbd78acb4cf46298e0de8ba0d130e19" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:32.345619 19664 raft_consensus.cc:2804] T fffab26427e24d398e2695aad40ca134 P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:20:32.346100 19664 raft_consensus.cc:697] T fffab26427e24d398e2695aad40ca134 P 90f3da57796b488c9087a2f523adf045 [term 8 LEADER]: Becoming Leader. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:20:32.303932 19866 raft_consensus.cc:2804] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:20:32.305466 19915 raft_consensus.cc:3055] T 02838c86531d4f98bf58d1c22b8baf77 P 647a343a542d4a139fe106c19095129b [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:20:32.350389 16151 raft_consensus.cc:1240] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [term 12 LEADER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 11. Current term is 12. Ops: []
I20260709 13:20:32.342994 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68a6838d82ef4cac83507f906139d90f" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:20:32.357013 16291 raft_consensus.cc:1275] T 8638ee363273426db1393dd81e5c7b72 P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:20:32.309911 19861 raft_consensus.cc:2749] T ecc1e363230043fe9fef390f0b979f20 P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:20:32.359565 16820 raft_consensus.cc:1275] T 8638ee363273426db1393dd81e5c7b72 P cb82cc7bd6db46d980c75237a1402860 [term 9 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:20:32.363206 16824 raft_consensus.cc:1275] T 9e65676b21b748728ad39668cfe722ec P cb82cc7bd6db46d980c75237a1402860 [term 10 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:20:32.304232 19667 consensus_queue.cc:1048] T d16dfad0c3b540e5b4b86492ddf87208 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.052s
I20260709 13:20:32.333407 19575 raft_consensus.cc:2804] T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:20:32.365496 19904 leader_election.cc:290] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:20:32.357399 19856 raft_consensus.cc:515] T fffab26427e24d398e2695aad40ca134 P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:32.357590 19915 raft_consensus.cc:740] T 02838c86531d4f98bf58d1c22b8baf77 P 647a343a542d4a139fe106c19095129b [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:20:32.369277 19754 raft_consensus.cc:515] T 84f2da98864b4425bf7c0055951fdd4b P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:32.371069 16296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8638ee363273426db1393dd81e5c7b72" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:32.371286 19763 raft_consensus.cc:493] T af57996a8be84d398f0f023e8d08ef77 P 647a343a542d4a139fe106c19095129b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:32.323951 19859 raft_consensus.cc:515] T af57996a8be84d398f0f023e8d08ef77 P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:32.371742 16296 raft_consensus.cc:2468] T 8638ee363273426db1393dd81e5c7b72 P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 9.
I20260709 13:20:32.371966 16226 raft_consensus.cc:2468] T fffab26427e24d398e2695aad40ca134 P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 7.
I20260709 13:20:32.363224 16145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8158bbd17b994d2eb96f560bd0f1b88d" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:32.300952 19822 consensus_queue.cc:1048] T 84f2da98864b4425bf7c0055951fdd4b P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.047s
I20260709 13:20:32.373404 19859 leader_election.cc:290] T af57996a8be84d398f0f023e8d08ef77 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:32.305208 19707 consensus_queue.cc:1048] T e5ad4eb281564936a59f41e387a47156 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.053s
I20260709 13:20:32.371743 19763 raft_consensus.cc:515] T af57996a8be84d398f0f023e8d08ef77 P 647a343a542d4a139fe106c19095129b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:32.374830 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1df93779209748b9860f6d893dd3de53" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:20:32.375497 16150 raft_consensus.cc:2376] T 1df93779209748b9860f6d893dd3de53 P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 9. Re-sending same reply.
I20260709 13:20:32.375840 16291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7ca61592e848ea9b0fb81a2ce33cdc" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:32.376094 19763 leader_election.cc:290] T af57996a8be84d398f0f023e8d08ef77 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:32.376500 16291 raft_consensus.cc:2468] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 11.
I20260709 13:20:32.377542 19751 leader_election.cc:290] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:32.379446 19197 raft_consensus.cc:515] T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:32.381225 16220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87ac0d1f7e8a419f9bcbc04aee0e067e" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:32.347985 19914 raft_consensus.cc:493] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:32.381801 16220 raft_consensus.cc:2468] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 9.
I20260709 13:20:32.381909 19914 raft_consensus.cc:3060] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:20:32.382792 16220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af57996a8be84d398f0f023e8d08ef77" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:32.387904 16220 raft_consensus.cc:2468] T af57996a8be84d398f0f023e8d08ef77 P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 9.
I20260709 13:20:32.389860 16144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87147731a68400e8cce9b879daa7c5d" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:20:32.383517 16222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c61a39fd4f84424b129c1a6f9ba404d" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:20:32.390441 16144 raft_consensus.cc:2376] T b87147731a68400e8cce9b879daa7c5d P 647a343a542d4a139fe106c19095129b [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 10. Re-sending same reply.
I20260709 13:20:32.359606 19789 raft_consensus.cc:2749] T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:20:32.391502 16144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2372981006d4b218215c28964dea623" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:32.392875 16144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b7d1ab519044eda2d5e04a07242f25" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:32.385705 19866 raft_consensus.cc:697] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac [term 11 LEADER]: Becoming Leader. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:20:32.386269 16146 raft_consensus.cc:2368] T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 for earlier term 8. Current term is 9.
I20260709 13:20:32.309013 19863 raft_consensus.cc:515] T ee9ed67006144797b1de6ea8c2013a68 P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:32.392338 19806 raft_consensus.cc:2955] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860 [term 8 LEADER]: Committing config change with OpId 2.4: config changed from index 3 to 4, cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
W20260709 13:20:32.393848 16142 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46224 (request call id 519) took 1887 ms (client timeout 1971 ms). Trace:
I20260709 13:20:32.365830 19575 raft_consensus.cc:493] T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:32.383941 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cc959fb7f704fba9009c0595467d074" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:32.414155 16298 raft_consensus.cc:2468] T 8cc959fb7f704fba9009c0595467d074 P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 9.
I20260709 13:20:32.401732 16219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f2da98864b4425bf7c0055951fdd4b" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:20:32.403604 16149 raft_consensus.cc:1275] T 23f82224f4564fea93572e8206bad1fe P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:20:32.404663 16225 raft_consensus.cc:2468] T 68a6838d82ef4cac83507f906139d90f P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 10.
I20260709 13:20:32.405874 16221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fffab26427e24d398e2695aad40ca134" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:20:32.407366 16144 raft_consensus.cc:2393] T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 in current term 7: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
I20260709 13:20:32.395862 16790 leader_election.cc:304] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 90f3da57796b488c9087a2f523adf045
I20260709 13:20:32.412675 16151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87ac0d1f7e8a419f9bcbc04aee0e067e" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:20:32.419869 16790 leader_election.cc:304] T af57996a8be84d398f0f023e8d08ef77 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:20:32.376531 16295 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48674 (request call id 2461) took 1676 ms. Trace:
I20260709 13:20:32.420730 16295 rpcz_store.cc:276] 0709 13:20:30.700449 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:30.703275 (+  2826us) service_pool.cc:224] Handling call
0709 13:20:30.703547 (+   272us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:31.354434 (+650887us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:32.376306 (+1021872us) spinlock_profiling.cc:243] Waited 612 ms on lock 0x7b54000be5d4. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:20:32.376475 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1407624064}
W20260709 13:20:32.411077 16142 rpcz_store.cc:269] 0709 13:20:30.506055 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:30.517660 (+ 11605us) service_pool.cc:224] Handling call
0709 13:20:32.393675 (+1876015us) spinlock_profiling.cc:243] Waited 764 ms on lock 0x7b54000749d4. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:20:32.393775 (+   100us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1757466880}
I20260709 13:20:32.412498 16222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4dc6b0541ca431ebc757beac00521f1" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:20:32.422835 16144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a878999c0024d2abe517484180290fc" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 9 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:32.423365 16222 raft_consensus.cc:3060] T e4dc6b0541ca431ebc757beac00521f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:20:32.361934 19540 raft_consensus.cc:2804] T d16dfad0c3b540e5b4b86492ddf87208 P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:20:32.427800 16220 raft_consensus.cc:1275] T 5a96373bb0e14dce8a266b2fc6c58507 P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 9 index: 4. (index mismatch)
I20260709 13:20:32.361581 19414 raft_consensus.cc:3055] T 68a6838d82ef4cac83507f906139d90f P 90f3da57796b488c9087a2f523adf045 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:20:32.434301 16144 raft_consensus.cc:2468] T 8a878999c0024d2abe517484180290fc P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 8.
I20260709 13:20:32.434803 16223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a96373bb0e14dce8a266b2fc6c58507" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 8 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:32.405243 19760 leader_election.cc:290] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:32.438174 16116 leader_election.cc:304] T 08c74490c40243a88e62d1760b5cd814 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
I20260709 13:20:32.420773 16151 raft_consensus.cc:2410] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac for term 9 because replica has last-logged OpId of term: 8 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 6.
I20260709 13:20:32.442972 16223 raft_consensus.cc:2368] T 5a96373bb0e14dce8a266b2fc6c58507 P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 for earlier term 8. Current term is 9.
I20260709 13:20:32.446596 16144 raft_consensus.cc:1240] T 84f2da98864b4425bf7c0055951fdd4b P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 7. Current term is 8. Ops: []
I20260709 13:20:32.448561 16142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b7d1ab519044eda2d5e04a07242f25" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:32.449366 16142 raft_consensus.cc:2468] T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 7.
I20260709 13:20:32.421347 19575 raft_consensus.cc:3060] T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:20:32.455757 16298 raft_consensus.cc:1275] T 23f82224f4564fea93572e8206bad1fe P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:20:32.452139 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66ae176961b74813a58bc15c9c4c6f2a" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:20:32.463291 16222 raft_consensus.cc:2468] T e4dc6b0541ca431ebc757beac00521f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 9.
I20260709 13:20:32.346990 19664 consensus_queue.cc:237] T fffab26427e24d398e2695aad40ca134 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:32.437135 19414 raft_consensus.cc:740] T 68a6838d82ef4cac83507f906139d90f P 90f3da57796b488c9087a2f523adf045 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:20:32.470050 16142 raft_consensus.cc:1240] T 9c61a39fd4f84424b129c1a6f9ba404d P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 8. Current term is 9. Ops: []
I20260709 13:20:32.449954 19540 raft_consensus.cc:697] T d16dfad0c3b540e5b4b86492ddf87208 P 647a343a542d4a139fe106c19095129b [term 9 LEADER]: Becoming Leader. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:20:32.416196 19899 consensus_queue.cc:1048] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.041s
I20260709 13:20:32.456962 19760 raft_consensus.cc:2804] T 08c74490c40243a88e62d1760b5cd814 P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:20:32.474967 16223 raft_consensus.cc:1240] T 68a6838d82ef4cac83507f906139d90f P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 9. Current term is 10. Ops: []
I20260709 13:20:32.444936 19754 leader_election.cc:290] T 84f2da98864b4425bf7c0055951fdd4b P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 8 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:32.477746 16298 raft_consensus.cc:1240] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 10. Current term is 11. Ops: [5.7-10.8]
W20260709 13:20:32.477029 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 134831us
I20260709 13:20:32.482056 16265 leader_election.cc:304] T 2794af5b99984efebdee564035841cb0 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: cb82cc7bd6db46d980c75237a1402860
I20260709 13:20:32.483109 19345 raft_consensus.cc:2804] T 2794af5b99984efebdee564035841cb0 P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:20:32.483683 19345 raft_consensus.cc:697] T 2794af5b99984efebdee564035841cb0 P 90f3da57796b488c9087a2f523adf045 [term 7 LEADER]: Becoming Leader. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:20:32.484349 16265 leader_election.cc:304] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:20:32.484711 19345 consensus_queue.cc:237] T 2794af5b99984efebdee564035841cb0 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:32.485863 16265 leader_election.cc:304] T 26354361e9b044cea1c2b86c4882fe92 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
I20260709 13:20:32.487500 19657 raft_consensus.cc:2804] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:20:32.488077 19657 raft_consensus.cc:493] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:32.487871 19673 consensus_queue.cc:1059] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:20:32.488687 19657 raft_consensus.cc:3060] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:20:32.489156 19673 consensus_queue.cc:1059] T d2ab0ac8347641e6955e9bce7d90c1a2 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:20:32.499992 19816 leader_election.cc:290] T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:32.502454 19657 raft_consensus.cc:515] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:32.507042 18599 leader_election.cc:290] T 9c61a39fd4f84424b129c1a6f9ba404d P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 9 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:32.512362 19876 raft_consensus.cc:2749] T 26354361e9b044cea1c2b86c4882fe92 P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:20:32.509063 19657 leader_election.cc:290] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 9 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:20:32.465175 19898 raft_consensus.cc:2749] T ada5a004ba7c46819051e3ccec71ffdb P cb82cc7bd6db46d980c75237a1402860 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:20:32.545431 19915 consensus_queue.cc:260] T 02838c86531d4f98bf58d1c22b8baf77 P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:32.548882 16191 leader_election.cc:304] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
I20260709 13:20:32.549616 19859 raft_consensus.cc:493] T c0eab25ce4c64e018a041beeda2867db P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:32.550154 19751 raft_consensus.cc:493] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:32.465659 19856 leader_election.cc:290] T fffab26427e24d398e2695aad40ca134 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:32.551810 16222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f5f89186204242bf377f7c42b2e2da" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:32.562091 16191 leader_election.cc:304] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
I20260709 13:20:32.473886 19568 raft_consensus.cc:2749] T af57996a8be84d398f0f023e8d08ef77 P cb82cc7bd6db46d980c75237a1402860 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:20:32.531273 19760 raft_consensus.cc:493] T 08c74490c40243a88e62d1760b5cd814 P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:32.569991 16298 raft_consensus.cc:1275] T 31969392ccdb43b491911a319e8bc7e1 P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:20:32.498998 19529 leader_election.cc:290] T 08c74490c40243a88e62d1760b5cd814 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:32.552667 19726 leader_election.cc:290] T b6f07842d3aa47858df5665874c86f3a P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:32.567997 19855 raft_consensus.cc:2749] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:20:32.591813 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4dc6b0541ca431ebc757beac00521f1" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
W20260709 13:20:32.592672 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 299566us
W20260709 13:20:32.595868 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 157785us
I20260709 13:20:32.596084 16226 raft_consensus.cc:2368] T e4dc6b0541ca431ebc757beac00521f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 for earlier term 8. Current term is 9.
I20260709 13:20:32.596827 16222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5ad4eb281564936a59f41e387a47156" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:20:32.574837 19760 raft_consensus.cc:3060] T 08c74490c40243a88e62d1760b5cd814 P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:20:32.598264 16190 leader_election.cc:304] T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 647a343a542d4a139fe106c19095129b
I20260709 13:20:32.575711 19883 raft_consensus.cc:3055] T d2ab0ac8347641e6955e9bce7d90c1a2 P cb82cc7bd6db46d980c75237a1402860 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:20:32.598889 16116 leader_election.cc:304] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
I20260709 13:20:32.600988 19883 raft_consensus.cc:740] T d2ab0ac8347641e6955e9bce7d90c1a2 P cb82cc7bd6db46d980c75237a1402860 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:20:32.585706 19856 raft_consensus.cc:2749] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
W20260709 13:20:32.611775 16218 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33050 (request call id 620) took 1313 ms (client timeout 1731 ms). Trace:
W20260709 13:20:32.612383 16218 rpcz_store.cc:269] 0709 13:20:31.286649 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:31.286833 (+   184us) service_pool.cc:224] Handling call
0709 13:20:32.600091 (+1313258us) spinlock_profiling.cc:243] Waited 155 ms on lock 0x7b54002b4654. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:20:32.600238 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":356550400}
I20260709 13:20:32.602958 16116 leader_election.cc:304] T af57996a8be84d398f0f023e8d08ef77 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
I20260709 13:20:32.604696 16222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b7d1ab519044eda2d5e04a07242f25" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:32.614568 16222 raft_consensus.cc:2468] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 7.
I20260709 13:20:32.569140 19859 raft_consensus.cc:515] T c0eab25ce4c64e018a041beeda2867db P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:20:32.626883 16789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 299224us
I20260709 13:20:32.632076 16789 leader_election.cc:304] T 3a7ca61592e848ea9b0fb81a2ce33cdc P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:20:32.633770 16190 leader_election.cc:304] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
W20260709 13:20:32.635779 16263 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:32.637073 16789 leader_election.cc:304] T 8638ee363273426db1393dd81e5c7b72 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:20:32.598623 19915 raft_consensus.cc:3060] T 02838c86531d4f98bf58d1c22b8baf77 P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:20:32.572671 19751 raft_consensus.cc:515] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:32.651953 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fffab26427e24d398e2695aad40ca134" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:32.585290 19822 consensus_queue.cc:1048] T 829b4ad8c5a54eababeea1181b960d4b P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.041s
I20260709 13:20:32.651981 16218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ab0ac8347641e6955e9bce7d90c1a2" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:32.658614 16222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2372981006d4b218215c28964dea623" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:32.645936 16266 leader_election.cc:304] T b87147731a68400e8cce9b879daa7c5d P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
I20260709 13:20:32.658010 16295 raft_consensus.cc:1240] T d16dfad0c3b540e5b4b86492ddf87208 P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 8. Current term is 9. Ops: [8.6-8.6]
I20260709 13:20:32.658818 16218 raft_consensus.cc:2393] T d2ab0ac8347641e6955e9bce7d90c1a2 P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 9: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
I20260709 13:20:32.654366 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ad5dbfa99ee4b8d9e28be92b3f36db5" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:32.660389 16293 raft_consensus.cc:2393] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 8: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
W20260709 13:20:32.661053 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 175272us
I20260709 13:20:32.661854 16265 leader_election.cc:304] T 23f82224f4564fea93572e8206bad1fe P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
I20260709 13:20:32.627784 19855 raft_consensus.cc:2804] T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:20:32.626969 19696 raft_consensus.cc:2804] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:20:32.670784 16265 leader_election.cc:400] T e4dc6b0541ca431ebc757beac00521f1 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 8 election: Vote denied by peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741) with higher term. Message: Invalid argument: T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 for earlier term 8. Current term is 9.
I20260709 13:20:32.671448 16265 leader_election.cc:403] T e4dc6b0541ca431ebc757beac00521f1 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 8 election: Cancelling election due to peer responding with higher term
I20260709 13:20:32.572543 19866 consensus_queue.cc:237] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:32.597971 19667 consensus_queue.cc:1048] T 8cc959fb7f704fba9009c0595467d074 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.059s
I20260709 13:20:32.688325 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02838c86531d4f98bf58d1c22b8baf77" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:20:32.602121 19702 consensus_queue.cc:1048] T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.062s
I20260709 13:20:32.650388 19899 raft_consensus.cc:2804] T af57996a8be84d398f0f023e8d08ef77 P 647a343a542d4a139fe106c19095129b [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:20:32.650647 19856 raft_consensus.cc:2804] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:20:32.628284 19823 consensus_queue.cc:1048] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.044s
I20260709 13:20:32.654011 19938 raft_consensus.cc:493] T 57a744f5aaf944678a63469cbf514015 P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
W20260709 13:20:32.672122 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 211876us
I20260709 13:20:32.699223 16116 leader_election.cc:304] T 68a6838d82ef4cac83507f906139d90f P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
I20260709 13:20:32.670581 19657 raft_consensus.cc:2749] T 23f82224f4564fea93572e8206bad1fe P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:20:32.705709 16191 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:32.649432 19699 consensus_queue.cc:1048] T b6f07842d3aa47858df5665874c86f3a P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 2, Time since last communication: 0.045s
I20260709 13:20:32.667596 19809 raft_consensus.cc:2804] T 8638ee363273426db1393dd81e5c7b72 P cb82cc7bd6db46d980c75237a1402860 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:20:32.670692 19696 raft_consensus.cc:493] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:32.709268 16790 leader_election.cc:304] T 19baa7bc367844c288bca8f81439c413 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860; no voters: 90f3da57796b488c9087a2f523adf045
I20260709 13:20:32.678157 19863 leader_election.cc:290] T ee9ed67006144797b1de6ea8c2013a68 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:32.692462 19899 raft_consensus.cc:493] T af57996a8be84d398f0f023e8d08ef77 P 647a343a542d4a139fe106c19095129b [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:32.605515 19414 consensus_queue.cc:260] T 68a6838d82ef4cac83507f906139d90f P 90f3da57796b488c9087a2f523adf045 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:32.748484 16226 raft_consensus.cc:1275] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 8. (term mismatch)
W20260709 13:20:32.743278 16191 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:32.763885 16226 pending_rounds.cc:85] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac: Aborting all ops after (but not including) 7
W20260709 13:20:32.753584 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 85615.8us
I20260709 13:20:32.747771 16218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c74490c40243a88e62d1760b5cd814" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:32.764180 16226 pending_rounds.cc:107] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac: Aborting uncommitted NO_OP operation due to leader change: 8.8
I20260709 13:20:32.764452 16226 raft_consensus.cc:2889] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:20:32.764539 16218 raft_consensus.cc:2468] T 08c74490c40243a88e62d1760b5cd814 P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 8.
I20260709 13:20:32.670590 19345 raft_consensus.cc:3060] T b87147731a68400e8cce9b879daa7c5d P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:20:32.653762 19893 consensus_queue.cc:1048] T 8c11074579b046d99e585c2818e18a2c P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.044s
I20260709 13:20:32.671881 19942 raft_consensus.cc:2804] T 3a7ca61592e848ea9b0fb81a2ce33cdc P cb82cc7bd6db46d980c75237a1402860 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:20:32.697430 19856 raft_consensus.cc:493] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:32.756420 16116 leader_election.cc:304] T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
W20260709 13:20:32.766397 16191 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:32.701871 19883 consensus_queue.cc:260] T d2ab0ac8347641e6955e9bce7d90c1a2 P cb82cc7bd6db46d980c75237a1402860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:20:32.767517 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 58297.2us
I20260709 13:20:32.779886 16223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ad5dbfa99ee4b8d9e28be92b3f36db5" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:20:32.773546 16143 raft_consensus.cc:2468] T 9e65676b21b748728ad39668cfe722ec P 647a343a542d4a139fe106c19095129b [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 10.
I20260709 13:20:32.749874 19754 raft_consensus.cc:2804] T 68a6838d82ef4cac83507f906139d90f P 647a343a542d4a139fe106c19095129b [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:20:32.708118 19802 raft_consensus.cc:2804] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P cb82cc7bd6db46d980c75237a1402860 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:20:32.753332 19809 raft_consensus.cc:493] T 8638ee363273426db1393dd81e5c7b72 P cb82cc7bd6db46d980c75237a1402860 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:32.800025 16223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6f07842d3aa47858df5665874c86f3a" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
W20260709 13:20:32.781982 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 114724us
I20260709 13:20:32.742461 19938 raft_consensus.cc:515] T 57a744f5aaf944678a63469cbf514015 P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:32.785974 16117 leader_election.cc:304] T 2794af5b99984efebdee564035841cb0 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860; no voters: 90f3da57796b488c9087a2f523adf045
I20260709 13:20:32.755374 19941 raft_consensus.cc:493] T 26354361e9b044cea1c2b86c4882fe92 P cb82cc7bd6db46d980c75237a1402860 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:20:32.839849 16117 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 53982.8us
I20260709 13:20:32.840715 16117 leader_election.cc:304] T 434bff3082d141838068f323bdb0d1ed P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860
I20260709 13:20:32.755127 19696 raft_consensus.cc:3060] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:20:32.665545 19540 consensus_queue.cc:237] T d16dfad0c3b540e5b4b86492ddf87208 P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:32.699216 19861 consensus_queue.cc:1048] T 8cc959fb7f704fba9009c0595467d074 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 2, Time since last communication: 0.074s
I20260709 13:20:32.790666 16790 leader_election.cc:304] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 
W20260709 13:20:32.851974 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 61388.1us
I20260709 13:20:32.675442 19876 consensus_queue.cc:1048] T 9e65676b21b748728ad39668cfe722ec P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.084s
I20260709 13:20:32.765012 19899 raft_consensus.cc:3060] T af57996a8be84d398f0f023e8d08ef77 P 647a343a542d4a139fe106c19095129b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:20:32.774061 19945 consensus_queue.cc:1048] T 5a96373bb0e14dce8a266b2fc6c58507 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.082s
I20260709 13:20:32.859795 16145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c74490c40243a88e62d1760b5cd814" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:20:32.776326 19942 raft_consensus.cc:493] T 3a7ca61592e848ea9b0fb81a2ce33cdc P cb82cc7bd6db46d980c75237a1402860 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:32.780241 19948 raft_consensus.cc:3060] T e4dc6b0541ca431ebc757beac00521f1 P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:20:32.779677 19856 raft_consensus.cc:3060] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:20:32.838508 18599 raft_consensus.cc:493] T 26354361e9b044cea1c2b86c4882fe92 P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:32.844197 16220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c61a39fd4f84424b129c1a6f9ba404d" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:32.788172 19672 raft_consensus.cc:2804] T 19baa7bc367844c288bca8f81439c413 P cb82cc7bd6db46d980c75237a1402860 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:20:32.768430 19863 consensus_queue.cc:1048] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.035s
I20260709 13:20:32.793511 19414 raft_consensus.cc:3060] T 68a6838d82ef4cac83507f906139d90f P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:20:32.854737 16143 raft_consensus.cc:1275] T 9e65676b21b748728ad39668cfe722ec P 647a343a542d4a139fe106c19095129b [term 10 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:20:32.894006 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a378d7f796f94615b3b347119a9880fd" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:32.894627 16299 raft_consensus.cc:2468] T a378d7f796f94615b3b347119a9880fd P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 9.
I20260709 13:20:32.856326 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1df93779209748b9860f6d893dd3de53" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:32.798030 19809 raft_consensus.cc:3060] T 8638ee363273426db1393dd81e5c7b72 P cb82cc7bd6db46d980c75237a1402860 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:20:32.804957 19883 raft_consensus.cc:3060] T d2ab0ac8347641e6955e9bce7d90c1a2 P cb82cc7bd6db46d980c75237a1402860 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:20:32.838868 19922 raft_consensus.cc:493] T 1df93779209748b9860f6d893dd3de53 P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:32.802911 19754 raft_consensus.cc:697] T 68a6838d82ef4cac83507f906139d90f P 647a343a542d4a139fe106c19095129b [term 10 LEADER]: Becoming Leader. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:20:32.800609 19816 raft_consensus.cc:2804] T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:20:32.865589 19945 consensus_queue.cc:1243] T 5a96373bb0e14dce8a266b2fc6c58507 P 647a343a542d4a139fe106c19095129b [LEADER]: Peer 1ca2725d290e47d68b1a4b609c9816ac log is divergent from this leader: its last log entry 6.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
W20260709 13:20:32.872063 19937 log.cc:927] Time spent T 00000000000000000000000000000000 P 667a34b27a39474ebefc1af3cf5b4271: Append to log took a long time: real 0.070s	user 0.002s	sys 0.000s
W20260709 13:20:32.873664 16791 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:32.879513 16223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada5a004ba7c46819051e3ccec71ffdb" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:32.845948 19569 raft_consensus.cc:2764] T 2794af5b99984efebdee564035841cb0 P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Leader pre-election decision vote started in defunct term 6: won
I20260709 13:20:32.850311 19789 raft_consensus.cc:2749] T 434bff3082d141838068f323bdb0d1ed P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:20:32.897565 16190 leader_election.cc:304] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:20:32.867072 16149 raft_consensus.cc:2955] T ecc1e363230043fe9fef390f0b979f20 P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
W20260709 13:20:32.877130 19936 log.cc:927] Time spent T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.109s	user 0.000s	sys 0.000s
I20260709 13:20:32.944095 18599 raft_consensus.cc:515] T 26354361e9b044cea1c2b86c4882fe92 P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:20:32.954288 16266 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:32.866389 19942 raft_consensus.cc:3060] T 3a7ca61592e848ea9b0fb81a2ce33cdc P cb82cc7bd6db46d980c75237a1402860 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:20:32.981324 16220 raft_consensus.cc:1240] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 7. Current term is 8. Ops: [7.7-7.7]
I20260709 13:20:32.914472 19903 raft_consensus.cc:2804] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:20:32.836045 19939 consensus_queue.cc:1059] T d16dfad0c3b540e5b4b86492ddf87208 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.113s
I20260709 13:20:32.889128 19672 raft_consensus.cc:493] T 19baa7bc367844c288bca8f81439c413 P cb82cc7bd6db46d980c75237a1402860 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:32.888492 19532 raft_consensus.cc:493] T 829b4ad8c5a54eababeea1181b960d4b P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:32.975804 16223 raft_consensus.cc:2468] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 9.
I20260709 13:20:33.001667 16819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af57996a8be84d398f0f023e8d08ef77" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:32.887574 19568 consensus_queue.cc:1048] T c9944d5e4b8c4109af9955c4f08c9095 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.093s
I20260709 13:20:32.853775 19941 raft_consensus.cc:515] T 26354361e9b044cea1c2b86c4882fe92 P cb82cc7bd6db46d980c75237a1402860 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:32.977108 19529 raft_consensus.cc:493] T 8d4ab870a05347868956aa8b841b2981 P 647a343a542d4a139fe106c19095129b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:33.002543 16819 raft_consensus.cc:2468] T af57996a8be84d398f0f023e8d08ef77 P cb82cc7bd6db46d980c75237a1402860 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 10.
I20260709 13:20:32.962700 19898 raft_consensus.cc:493] T 66f75124215f4ca6bef831e174a792d8 P cb82cc7bd6db46d980c75237a1402860 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:20:33.019006 16144 raft_consensus.cc:1240] T d16dfad0c3b540e5b4b86492ddf87208 P 647a343a542d4a139fe106c19095129b [term 9 LEADER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 8. Current term is 9. Ops: []
I20260709 13:20:32.984120 19859 leader_election.cc:290] T c0eab25ce4c64e018a041beeda2867db P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:32.991369 19816 raft_consensus.cc:697] T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b [term 9 LEADER]: Becoming Leader. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:20:33.023275 16220 raft_consensus.cc:1275] T 8c11074579b046d99e585c2818e18a2c P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 7 index: 9. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
I20260709 13:20:32.992141 19903 raft_consensus.cc:493] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:20:33.004578 19672 raft_consensus.cc:3060] T 19baa7bc367844c288bca8f81439c413 P cb82cc7bd6db46d980c75237a1402860 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:20:33.045404 16221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1df93779209748b9860f6d893dd3de53" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:33.024744 16223 pending_rounds.cc:85] T 8a878999c0024d2abe517484180290fc P 1ca2725d290e47d68b1a4b609c9816ac: Aborting all ops after (but not including) 2
I20260709 13:20:33.041543 18599 leader_election.cc:290] T 26354361e9b044cea1c2b86c4882fe92 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:33.034237 16824 raft_consensus.cc:1275] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860 [term 11 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:20:33.048441 16223 pending_rounds.cc:107] T 8a878999c0024d2abe517484180290fc P 1ca2725d290e47d68b1a4b609c9816ac: Aborting uncommitted NO_OP operation due to leader change: 4.3
I20260709 13:20:33.049898 16790 leader_election.cc:304] T b87147731a68400e8cce9b879daa7c5d P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860; no voters: 90f3da57796b488c9087a2f523adf045
I20260709 13:20:33.005561 19751 leader_election.cc:290] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:33.057744 16223 raft_consensus.cc:2889] T 8a878999c0024d2abe517484180290fc P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:20:33.044646 19903 raft_consensus.cc:3060] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:20:32.966774 19922 raft_consensus.cc:515] T 1df93779209748b9860f6d893dd3de53 P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:33.013495 19532 raft_consensus.cc:515] T 829b4ad8c5a54eababeea1181b960d4b P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:33.066495 16221 raft_consensus.cc:1275] T 19baa7bc367844c288bca8f81439c413 P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 13:20:33.021330 19960 consensus_queue.cc:1048] T 9e65676b21b748728ad39668cfe722ec P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 6, Time since last communication: 0.064s
I20260709 13:20:33.070624 16219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c74490c40243a88e62d1760b5cd814" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:20:33.062741 19197 leader_election.cc:290] T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 9 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:33.034659 19529 raft_consensus.cc:515] T 8d4ab870a05347868956aa8b841b2981 P 647a343a542d4a139fe106c19095129b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:33.091269 16219 raft_consensus.cc:3060] T 08c74490c40243a88e62d1760b5cd814 P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:20:33.035075 19898 raft_consensus.cc:515] T 66f75124215f4ca6bef831e174a792d8 P cb82cc7bd6db46d980c75237a1402860 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:33.070293 16192 leader_election.cc:304] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860
W20260709 13:20:33.065555 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 170324us
I20260709 13:20:33.093902 16150 raft_consensus.cc:2468] T 1df93779209748b9860f6d893dd3de53 P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 9.
I20260709 13:20:33.098479 16295 raft_consensus.cc:1275] T a378d7f796f94615b3b347119a9880fd P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
W20260709 13:20:33.068372 19940 log.cc:927] Time spent T 5a959dbecc2648eb85d1cc550f5a3245 P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 0.320s	user 0.000s	sys 0.001s
I20260709 13:20:33.107581 16819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26354361e9b044cea1c2b86c4882fe92" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:33.108232 16819 raft_consensus.cc:2468] T 26354361e9b044cea1c2b86c4882fe92 P cb82cc7bd6db46d980c75237a1402860 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 7.
I20260709 13:20:33.107041 16297 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48674 (request call id 2482) took 1078 ms. Trace:
I20260709 13:20:33.111647 16297 rpcz_store.cc:276] 0709 13:20:32.029056 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:32.052711 (+ 23655us) service_pool.cc:224] Handling call
0709 13:20:32.053005 (+   294us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:20:32.132917 (+ 79912us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:32.132955 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:20:32.158712 (+ 25757us) log.cc:844] Serialized 26 byte log entry
0709 13:20:32.473936 (+315224us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:20:32.567007 (+ 93071us) log.cc:844] Serialized 14 byte log entry
0709 13:20:32.567337 (+   330us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:32.567434 (+    97us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:20:33.106747 (+539313us) raft_consensus.cc:1685] finished
0709 13:20:33.106798 (+    51us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:33.106964 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":93482,"prepare.run_cpu_time_us":66,"prepare.run_wall_time_us":8705,"raft.queue_time_us":29300,"raft.run_cpu_time_us":256,"raft.run_wall_time_us":10806,"thread_start_us":209207,"threads_started":1,"wal-append.queue_time_us":507147}
I20260709 13:20:33.114058 19529 leader_election.cc:290] T 8d4ab870a05347868956aa8b841b2981 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:33.114789 19898 leader_election.cc:290] T 66f75124215f4ca6bef831e174a792d8 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
W20260709 13:20:33.115783 19951 log.cc:927] Time spent T 3884f864a96d428785895e5463e5728f P cb82cc7bd6db46d980c75237a1402860: Append to log took a long time: real 0.166s	user 0.000s	sys 0.001s
W20260709 13:20:33.118450 19884 consensus_meta.cc:287] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b: Time spent flushing consensus metadata: real 0.907s	user 0.010s	sys 0.005s
I20260709 13:20:33.110975 16117 leader_election.cc:304] T 26354361e9b044cea1c2b86c4882fe92 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:20:33.119788 16298 raft_consensus.cc:1275] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 13:20:33.121853 16220 pending_rounds.cc:85] T 5a96373bb0e14dce8a266b2fc6c58507 P 1ca2725d290e47d68b1a4b609c9816ac: Aborting all ops after (but not including) 2
I20260709 13:20:33.122385 16220 pending_rounds.cc:107] T 5a96373bb0e14dce8a266b2fc6c58507 P 1ca2725d290e47d68b1a4b609c9816ac: Aborting uncommitted NO_OP operation due to leader change: 6.3
I20260709 13:20:33.122711 16220 raft_consensus.cc:2889] T 5a96373bb0e14dce8a266b2fc6c58507 P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:20:33.110226 19949 log.cc:927] Time spent T 2794af5b99984efebdee564035841cb0 P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.122s	user 0.001s	sys 0.001s
I20260709 13:20:33.161906 16819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57a744f5aaf944678a63469cbf514015" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:33.088750 19962 consensus_queue.cc:1048] T 23f82224f4564fea93572e8206bad1fe P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.064s
I20260709 13:20:33.129935 19529 raft_consensus.cc:2804] T 26354361e9b044cea1c2b86c4882fe92 P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:20:33.130439 19855 raft_consensus.cc:697] T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac [term 10 LEADER]: Becoming Leader. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
W20260709 13:20:33.165921 19914 consensus_meta.cc:287] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b: Time spent flushing consensus metadata: real 0.784s	user 0.012s	sys 0.000s
I20260709 13:20:33.119499 19569 consensus_queue.cc:1048] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.052s
I20260709 13:20:33.072018 19754 consensus_queue.cc:237] T 68a6838d82ef4cac83507f906139d90f P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:33.136585 19965 raft_consensus.cc:2804] T b87147731a68400e8cce9b879daa7c5d P cb82cc7bd6db46d980c75237a1402860 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:20:33.106175 19947 consensus_queue.cc:1048] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.094s
I20260709 13:20:33.198321 16142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57a744f5aaf944678a63469cbf514015" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:33.104388 19816 consensus_queue.cc:237] T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:33.129158 19884 raft_consensus.cc:515] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } }
I20260709 13:20:33.162243 19938 leader_election.cc:290] T 57a744f5aaf944678a63469cbf514015 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:33.219347 16142 raft_consensus.cc:2393] T 57a744f5aaf944678a63469cbf514015 P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 8: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
I20260709 13:20:33.199528 19529 raft_consensus.cc:493] T 26354361e9b044cea1c2b86c4882fe92 P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:33.217404 19885 consensus_queue.cc:1048] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.005s
W20260709 13:20:33.222805 16264 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:33.157783 19568 consensus_queue.cc:1048] T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.056s
I20260709 13:20:33.213477 19569 consensus_queue.cc:1243] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b [LEADER]: Peer cb82cc7bd6db46d980c75237a1402860 log is divergent from this leader: its last log entry 10.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
W20260709 13:20:33.223078 19973 log.cc:927] Time spent T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.065s	user 0.000s	sys 0.000s
W20260709 13:20:33.246063 19954 log.cc:927] Time spent T 9e4bfb9090c8460eb14fa7e0d818e7f1 P cb82cc7bd6db46d980c75237a1402860: Append to log took a long time: real 0.255s	user 0.000s	sys 0.001s
I20260709 13:20:33.213126 19914 raft_consensus.cc:515] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [term 10 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:33.248847 16148 raft_consensus.cc:1240] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [term 10 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 9. Current term is 10. Ops: []
I20260709 13:20:33.262373 16219 raft_consensus.cc:2468] T 08c74490c40243a88e62d1760b5cd814 P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 9.
I20260709 13:20:33.229180 19529 raft_consensus.cc:3060] T 26354361e9b044cea1c2b86c4882fe92 P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:20:33.226878 19965 raft_consensus.cc:697] T b87147731a68400e8cce9b879daa7c5d P cb82cc7bd6db46d980c75237a1402860 [term 10 LEADER]: Becoming Leader. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:20:33.282042 19914 leader_election.cc:290] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 10 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:33.300318 16144 raft_consensus.cc:1275] T fffab26427e24d398e2695aad40ca134 P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:20:33.306414 16817 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49892 (request call id 538) took 1030 ms. Trace:
I20260709 13:20:33.334128 16144 raft_consensus.cc:1275] T 2794af5b99984efebdee564035841cb0 P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
W20260709 13:20:33.293092 19575 consensus_meta.cc:287] T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b: Time spent flushing consensus metadata: real 0.827s	user 0.014s	sys 0.007s
W20260709 13:20:33.294627 19760 consensus_meta.cc:287] T 08c74490c40243a88e62d1760b5cd814 P 647a343a542d4a139fe106c19095129b: Time spent flushing consensus metadata: real 0.667s	user 0.008s	sys 0.002s
I20260709 13:20:33.309521 16817 rpcz_store.cc:276] 0709 13:20:32.276400 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:32.276534 (+   134us) service_pool.cc:224] Handling call
0709 13:20:32.292458 (+ 15924us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:20:32.588799 (+296341us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:20:32.588829 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:20:32.602148 (+ 13319us) log.cc:844] Serialized 26 byte log entry
0709 13:20:32.697016 (+ 94868us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:20:32.697084 (+    68us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:32.697835 (+   751us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:20:33.306187 (+608352us) raft_consensus.cc:1685] finished
0709 13:20:33.306207 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:33.306353 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5761,"raft.queue_time_us":53785,"raft.run_cpu_time_us":120,"raft.run_wall_time_us":37480,"thread_start_us":301025,"threads_started":2,"wal-append.queue_time_us":137777}
I20260709 13:20:33.265513 19418 raft_consensus.cc:493] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:20:33.287115 19923 consensus_meta.cc:287] T 57a744f5aaf944678a63469cbf514015 P cb82cc7bd6db46d980c75237a1402860: Time spent flushing consensus metadata: real 1.076s	user 0.018s	sys 0.005s
I20260709 13:20:33.350656 16144 raft_consensus.cc:1275] T b6f07842d3aa47858df5665874c86f3a P 647a343a542d4a139fe106c19095129b [term 10 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
W20260709 13:20:33.352198 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 302371us
I20260709 13:20:33.303668 19884 leader_election.cc:290] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 9 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:33.351259 16115 leader_election.cc:304] T 8cc959fb7f704fba9009c0595467d074 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac
I20260709 13:20:33.362123 16790 leader_election.cc:304] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:20:33.361682 16819 raft_consensus.cc:2468] T 57a744f5aaf944678a63469cbf514015 P cb82cc7bd6db46d980c75237a1402860 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 8.
I20260709 13:20:33.363149 16144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fffab26427e24d398e2695aad40ca134" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:33.377553 16144 raft_consensus.cc:2393] T fffab26427e24d398e2695aad40ca134 P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 8: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
I20260709 13:20:33.348327 19945 raft_consensus.cc:493] T b87147731a68400e8cce9b879daa7c5d P 647a343a542d4a139fe106c19095129b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:33.390372 19785 consensus_queue.cc:1048] T 23f82224f4564fea93572e8206bad1fe P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.006s
I20260709 13:20:33.398751 19945 raft_consensus.cc:515] T b87147731a68400e8cce9b879daa7c5d P 647a343a542d4a139fe106c19095129b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:20:33.402067 19959 log.cc:927] Time spent T 84f2da98864b4425bf7c0055951fdd4b P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.304s	user 0.001s	sys 0.000s
I20260709 13:20:33.356438 19418 raft_consensus.cc:515] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:20:33.408108 16263 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:33.365835 19914 raft_consensus.cc:2804] T 8cc959fb7f704fba9009c0595467d074 P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:20:33.414963 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4dc6b0541ca431ebc757beac00521f1" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:20:33.419497 16144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66f75124215f4ca6bef831e174a792d8" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:33.420356 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f2da98864b4425bf7c0055951fdd4b" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:20:33.420707 16144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ad5dbfa99ee4b8d9e28be92b3f36db5" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:33.348965 19575 raft_consensus.cc:515] T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:33.422029 16291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6f07842d3aa47858df5665874c86f3a" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:33.425580 16291 raft_consensus.cc:2468] T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 10.
I20260709 13:20:33.387820 19923 raft_consensus.cc:2749] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:20:33.420662 16295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c74490c40243a88e62d1760b5cd814" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
W20260709 13:20:33.418689 19915 consensus_meta.cc:287] T 02838c86531d4f98bf58d1c22b8baf77 P 647a343a542d4a139fe106c19095129b: Time spent flushing consensus metadata: real 0.768s	user 0.009s	sys 0.004s
I20260709 13:20:33.428071 16291 raft_consensus.cc:1275] T c0eab25ce4c64e018a041beeda2867db P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:20:33.354048 19760 raft_consensus.cc:515] T 08c74490c40243a88e62d1760b5cd814 P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:33.427153 16295 raft_consensus.cc:2393] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 9: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
I20260709 13:20:33.431950 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87ac0d1f7e8a419f9bcbc04aee0e067e" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:33.436921 16146 raft_consensus.cc:1240] T 68a6838d82ef4cac83507f906139d90f P 647a343a542d4a139fe106c19095129b [term 10 LEADER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 9. Current term is 10. Ops: [9.6-9.6]
I20260709 13:20:33.435946 16293 raft_consensus.cc:2468] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 9.
I20260709 13:20:33.438740 16296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c61a39fd4f84424b129c1a6f9ba404d" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:20:33.442436 16144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af57996a8be84d398f0f023e8d08ef77" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:33.418087 19914 raft_consensus.cc:493] T 8cc959fb7f704fba9009c0595467d074 P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:33.405973 19816 consensus_queue.cc:1059] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.029s
I20260709 13:20:33.452587 16151 raft_consensus.cc:1240] T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 8. Current term is 9. Ops: []
I20260709 13:20:33.459014 16297 raft_consensus.cc:3060] T 5a96373bb0e14dce8a266b2fc6c58507 P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Advancing to term 9
W20260709 13:20:33.443699 19899 consensus_meta.cc:287] T af57996a8be84d398f0f023e8d08ef77 P 647a343a542d4a139fe106c19095129b: Time spent flushing consensus metadata: real 0.566s	user 0.021s	sys 0.009s
I20260709 13:20:33.348791 19855 consensus_queue.cc:237] T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.4, Last appended by leader: 4, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:33.416824 19965 consensus_queue.cc:237] T b87147731a68400e8cce9b879daa7c5d P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:33.426174 19942 raft_consensus.cc:515] T 3a7ca61592e848ea9b0fb81a2ce33cdc P cb82cc7bd6db46d980c75237a1402860 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:33.614727 16148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26354361e9b044cea1c2b86c4882fe92" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:33.537426 16291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:20:33.531960 16294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806d7690c2b644959152074bb2cab0a7" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:20:33.463608 16296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26354361e9b044cea1c2b86c4882fe92" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:33.470340 16147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f82224f4564fea93572e8206bad1fe" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
W20260709 13:20:33.536258 16791 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:33.524799 16115 leader_election.cc:304] T b6f07842d3aa47858df5665874c86f3a P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac
I20260709 13:20:33.558538 16297 raft_consensus.cc:1275] T 5a96373bb0e14dce8a266b2fc6c58507 P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 7 index: 3. Preceding OpId from leader: term: 9 index: 4. (index mismatch)
I20260709 13:20:33.578886 16295 raft_consensus.cc:1275] T 5a959dbecc2648eb85d1cc550f5a3245 P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:20:33.480610 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d4ab870a05347868956aa8b841b2981" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
W20260709 13:20:33.449491 19414 consensus_meta.cc:287] T 68a6838d82ef4cac83507f906139d90f P 90f3da57796b488c9087a2f523adf045: Time spent flushing consensus metadata: real 0.504s	user 0.015s	sys 0.006s
I20260709 13:20:33.630657 16293 raft_consensus.cc:2468] T 8d4ab870a05347868956aa8b841b2981 P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 10.
I20260709 13:20:33.628368 16294 raft_consensus.cc:3060] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:20:33.473242 16145 raft_consensus.cc:2393] T 08c74490c40243a88e62d1760b5cd814 P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 9: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
I20260709 13:20:33.628870 16296 raft_consensus.cc:2468] T 26354361e9b044cea1c2b86c4882fe92 P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 7.
W20260709 13:20:33.573256 19943 log.cc:927] Time spent T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860: Append to log took a long time: real 0.737s	user 0.003s	sys 0.000s
W20260709 13:20:33.532459 19948 consensus_meta.cc:287] T e4dc6b0541ca431ebc757beac00521f1 P 90f3da57796b488c9087a2f523adf045: Time spent flushing consensus metadata: real 0.637s	user 0.010s	sys 0.005s
I20260709 13:20:33.449285 19914 raft_consensus.cc:3060] T 8cc959fb7f704fba9009c0595467d074 P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:20:33.462764 19856 consensus_meta.cc:287] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 1ca2725d290e47d68b1a4b609c9816ac: Time spent flushing consensus metadata: real 0.548s	user 0.016s	sys 0.005s
W20260709 13:20:33.536944 19975 log.cc:927] Time spent T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 0.168s	user 0.000s	sys 0.000s
W20260709 13:20:33.633376 16115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 108700us
W20260709 13:20:33.631916 16789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 222076us
I20260709 13:20:33.468593 19922 leader_election.cc:290] T 1df93779209748b9860f6d893dd3de53 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
W20260709 13:20:33.572080 19806 consensus_meta.cc:287] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860: Time spent flushing consensus metadata: real 0.997s	user 0.012s	sys 0.002s
W20260709 13:20:33.534819 19883 consensus_meta.cc:287] T d2ab0ac8347641e6955e9bce7d90c1a2 P cb82cc7bd6db46d980c75237a1402860: Time spent flushing consensus metadata: real 0.572s	user 0.010s	sys 0.000s
W20260709 13:20:33.569540 19696 consensus_meta.cc:287] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 647a343a542d4a139fe106c19095129b: Time spent flushing consensus metadata: real 0.712s	user 0.007s	sys 0.004s
I20260709 13:20:33.615396 19673 raft_consensus.cc:493] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:20:33.532778 19345 consensus_meta.cc:287] T b87147731a68400e8cce9b879daa7c5d P 90f3da57796b488c9087a2f523adf045: Time spent flushing consensus metadata: real 0.758s	user 0.009s	sys 0.013s
W20260709 13:20:33.530148 19809 consensus_meta.cc:287] T 8638ee363273426db1393dd81e5c7b72 P cb82cc7bd6db46d980c75237a1402860: Time spent flushing consensus metadata: real 0.568s	user 0.001s	sys 0.019s
I20260709 13:20:33.528177 19569 raft_consensus.cc:493] T 23f82224f4564fea93572e8206bad1fe P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:33.570335 19903 raft_consensus.cc:515] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:33.636221 19726 raft_consensus.cc:2804] T b6f07842d3aa47858df5665874c86f3a P 647a343a542d4a139fe106c19095129b [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:20:33.711906 19726 raft_consensus.cc:493] T b6f07842d3aa47858df5665874c86f3a P 647a343a542d4a139fe106c19095129b [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:33.712376 19726 raft_consensus.cc:3060] T b6f07842d3aa47858df5665874c86f3a P 647a343a542d4a139fe106c19095129b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:20:33.712532 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26354361e9b044cea1c2b86c4882fe92" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:33.463981 19754 raft_consensus.cc:493] T 57a744f5aaf944678a63469cbf514015 P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:33.533942 19575 leader_election.cc:290] T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 9 election: Requested vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:33.536509 19760 leader_election.cc:290] T 08c74490c40243a88e62d1760b5cd814 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 9 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:33.650600 16115 leader_election.cc:304] T 84f2da98864b4425bf7c0055951fdd4b P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
W20260709 13:20:33.733877 16115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 83396.6us
I20260709 13:20:33.631879 19899 raft_consensus.cc:515] T af57996a8be84d398f0f023e8d08ef77 P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:33.535140 19418 leader_election.cc:290] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:33.674638 19942 leader_election.cc:290] T 3a7ca61592e848ea9b0fb81a2ce33cdc P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 12 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:33.680029 16294 raft_consensus.cc:2410] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b for term 9 because replica has last-logged OpId of term: 7 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 6.
I20260709 13:20:33.680212 19785 raft_consensus.cc:493] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:33.633664 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1df93779209748b9860f6d893dd3de53" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:33.531636 19945 leader_election.cc:290] T b87147731a68400e8cce9b879daa7c5d P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:33.684684 16291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87147731a68400e8cce9b879daa7c5d" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:33.686211 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee9ed67006144797b1de6ea8c2013a68" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
W20260709 13:20:33.686617 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 789115us
I20260709 13:20:33.688892 19696 raft_consensus.cc:515] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 647a343a542d4a139fe106c19095129b [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:33.690415 19939 raft_consensus.cc:2804] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:20:33.692085 19673 raft_consensus.cc:515] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:33.546777 19529 raft_consensus.cc:515] T 26354361e9b044cea1c2b86c4882fe92 P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:33.706956 16219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "829b4ad8c5a54eababeea1181b960d4b" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 10 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:33.708122 16217 raft_consensus.cc:1275] T c0eab25ce4c64e018a041beeda2867db P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
W20260709 13:20:33.708489 16264 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:33.709374 16221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1df93779209748b9860f6d893dd3de53" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:33.710299 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d4ab870a05347868956aa8b841b2981" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:33.713343 16298 raft_consensus.cc:2468] T 26354361e9b044cea1c2b86c4882fe92 P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 7.
I20260709 13:20:33.683804 19948 raft_consensus.cc:2749] T e4dc6b0541ca431ebc757beac00521f1 P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Leader election lost for term 8. Reason: Vote denied by peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741) with higher term. Message: Invalid argument: T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 for earlier term 8. Current term is 9.
I20260709 13:20:33.572567 19672 raft_consensus.cc:515] T 19baa7bc367844c288bca8f81439c413 P cb82cc7bd6db46d980c75237a1402860 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:33.759089 19540 raft_consensus.cc:2749] T 84f2da98864b4425bf7c0055951fdd4b P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
W20260709 13:20:33.715087 16192 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 644832us
I20260709 13:20:33.690430 19702 raft_consensus.cc:3055] T d16dfad0c3b540e5b4b86492ddf87208 P 1ca2725d290e47d68b1a4b609c9816ac [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:20:33.721952 19972 raft_consensus.cc:2749] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:20:33.796481 19702 raft_consensus.cc:740] T d16dfad0c3b540e5b4b86492ddf87208 P 1ca2725d290e47d68b1a4b609c9816ac [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:20:33.797180 19997 consensus_queue.cc:1048] T 829b4ad8c5a54eababeea1181b960d4b P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:20:33.729943 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806d7690c2b644959152074bb2cab0a7" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:20:33.797420 19702 consensus_queue.cc:260] T d16dfad0c3b540e5b4b86492ddf87208 P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:33.798125 16226 raft_consensus.cc:3060] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:20:33.798642 19702 raft_consensus.cc:3060] T d16dfad0c3b540e5b4b86492ddf87208 P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:20:33.738837 16115 leader_election.cc:304] T 9c61a39fd4f84424b129c1a6f9ba404d P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
W20260709 13:20:33.799784 16115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 61074.9us
I20260709 13:20:33.801333 16115 leader_election.cc:304] T 8d4ab870a05347868956aa8b841b2981 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:20:33.802067 19575 raft_consensus.cc:2749] T 9c61a39fd4f84424b129c1a6f9ba404d P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:20:33.696166 19345 raft_consensus.cc:2749] T b87147731a68400e8cce9b879daa7c5d P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:20:33.683771 19982 consensus_queue.cc:1048] T c0eab25ce4c64e018a041beeda2867db P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.044s
I20260709 13:20:33.765021 16297 raft_consensus.cc:2468] T 1df93779209748b9860f6d893dd3de53 P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 9.
I20260709 13:20:33.684520 19822 raft_consensus.cc:493] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:33.803721 19945 consensus_queue.cc:1048] T 5a96373bb0e14dce8a266b2fc6c58507 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:20:33.803967 19982 raft_consensus.cc:2804] T 8d4ab870a05347868956aa8b841b2981 P 647a343a542d4a139fe106c19095129b [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:20:33.803717 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdbd78acb4cf46298e0de8ba0d130e19" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:33.804610 19982 raft_consensus.cc:493] T 8d4ab870a05347868956aa8b841b2981 P 647a343a542d4a139fe106c19095129b [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:33.804838 16297 raft_consensus.cc:2468] T cdbd78acb4cf46298e0de8ba0d130e19 P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 9.
I20260709 13:20:33.804986 19982 raft_consensus.cc:3060] T 8d4ab870a05347868956aa8b841b2981 P 647a343a542d4a139fe106c19095129b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:20:33.805697 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:20:33.766739 16191 leader_election.cc:304] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:20:33.685330 19856 raft_consensus.cc:515] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:33.765931 16294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7ca61592e848ea9b0fb81a2ce33cdc" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:20:33.807057 16294 raft_consensus.cc:3060] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:20:33.719125 19992 consensus_queue.cc:1048] T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.045s
I20260709 13:20:33.714648 19569 raft_consensus.cc:515] T 23f82224f4564fea93572e8206bad1fe P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:33.711041 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:20:33.816883 16224 raft_consensus.cc:3060] T 19baa7bc367844c288bca8f81439c413 P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:20:33.784247 16144 raft_consensus.cc:2368] T af57996a8be84d398f0f023e8d08ef77 P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac for earlier term 10. Current term is 11.
I20260709 13:20:33.785562 19966 raft_consensus.cc:493] T 18ad7e3af5504571a69d10c00750a01c P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:20:33.786891 19529 leader_election.cc:290] T 26354361e9b044cea1c2b86c4882fe92 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 8 election: Requested vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:33.787649 19673 leader_election.cc:290] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:33.787945 19672 leader_election.cc:290] T 19baa7bc367844c288bca8f81439c413 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 10 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:33.789477 16816 raft_consensus.cc:1240] T 19baa7bc367844c288bca8f81439c413 P cb82cc7bd6db46d980c75237a1402860 [term 10 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 9. Current term is 10. Ops: []
I20260709 13:20:33.789772 19696 leader_election.cc:290] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 10 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:33.790699 16148 raft_consensus.cc:2393] T 26354361e9b044cea1c2b86c4882fe92 P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 in current term 8: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
I20260709 13:20:33.793782 16192 leader_election.cc:304] T d2ab0ac8347641e6955e9bce7d90c1a2 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860
I20260709 13:20:33.698776 19809 raft_consensus.cc:515] T 8638ee363273426db1393dd81e5c7b72 P cb82cc7bd6db46d980c75237a1402860 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:33.778645 19785 raft_consensus.cc:515] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:33.780529 16218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c74490c40243a88e62d1760b5cd814" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:20:33.824057 16218 raft_consensus.cc:2393] T 08c74490c40243a88e62d1760b5cd814 P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 9: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
I20260709 13:20:33.810819 16226 raft_consensus.cc:2468] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 9.
I20260709 13:20:33.824967 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87ac0d1f7e8a419f9bcbc04aee0e067e" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:20:33.825529 16226 raft_consensus.cc:3060] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:20:33.808367 16291 raft_consensus.cc:2468] T b87147731a68400e8cce9b879daa7c5d P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 10.
I20260709 13:20:33.777772 16149 raft_consensus.cc:2468] T ee9ed67006144797b1de6ea8c2013a68 P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 7.
W20260709 13:20:33.814050 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 47395.8us
I20260709 13:20:33.779953 19939 raft_consensus.cc:493] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:20:33.811908 19883 raft_consensus.cc:670] T 806d7690c2b644959152074bb2cab0a7 P cb82cc7bd6db46d980c75237a1402860: failed to trigger leader election: Illegal state: only voting members can start elections: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } } [suppressed 3 similar messages]
I20260709 13:20:33.729604 19754 raft_consensus.cc:515] T 57a744f5aaf944678a63469cbf514015 P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:33.828161 16192 leader_election.cc:304] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:20:33.838207 16226 raft_consensus.cc:2468] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 10.
I20260709 13:20:33.835327 16147 raft_consensus.cc:2376] T 23f82224f4564fea93572e8206bad1fe P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 8. Re-sending same reply.
I20260709 13:20:33.831137 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0eab25ce4c64e018a041beeda2867db" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:33.845793 19785 leader_election.cc:290] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:33.837867 19939 raft_consensus.cc:3060] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:20:33.819414 19972 raft_consensus.cc:493] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:20:33.847494 19972 raft_consensus.cc:515] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:33.848495 19856 leader_election.cc:290] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 12 election: Requested vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:33.804037 19822 raft_consensus.cc:515] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:33.849670 19972 leader_election.cc:290] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:33.851466 16149 raft_consensus.cc:1275] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
I20260709 13:20:33.851655 19822 leader_election.cc:290] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:33.851863 16191 leader_election.cc:304] T 8a878999c0024d2abe517484180290fc P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 90f3da57796b488c9087a2f523adf045
I20260709 13:20:33.852949 16149 raft_consensus.cc:1275] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 13:20:33.853726 19702 raft_consensus.cc:2804] T 8a878999c0024d2abe517484180290fc P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:20:33.854203 19702 raft_consensus.cc:493] T 8a878999c0024d2abe517484180290fc P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Starting leader election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:20:33.854592 19702 raft_consensus.cc:3060] T 8a878999c0024d2abe517484180290fc P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:20:33.815387 19919 raft_consensus.cc:493] T 02838c86531d4f98bf58d1c22b8baf77 P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:33.855074 19809 leader_election.cc:290] T 8638ee363273426db1393dd81e5c7b72 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 10 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:33.855347 19919 raft_consensus.cc:515] T 02838c86531d4f98bf58d1c22b8baf77 P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:33.855731 19809 consensus_queue.cc:1048] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:20:33.856137 19947 consensus_queue.cc:1048] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:20:33.857666 19919 leader_election.cc:290] T 02838c86531d4f98bf58d1c22b8baf77 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:33.860039 16295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31969392ccdb43b491911a319e8bc7e1" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:20:33.860143 19569 leader_election.cc:290] T 23f82224f4564fea93572e8206bad1fe P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:33.860792 19754 leader_election.cc:290] T 57a744f5aaf944678a63469cbf514015 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:33.861225 16295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c74490c40243a88e62d1760b5cd814" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:20:33.861356 19822 consensus_queue.cc:1059] T 84f2da98864b4425bf7c0055951fdd4b P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:20:33.861927 16295 raft_consensus.cc:2393] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 9: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
I20260709 13:20:33.862162 19903 leader_election.cc:290] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 7 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:20:33.862677 16295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26354361e9b044cea1c2b86c4882fe92" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:20:33.861356 19532 leader_election.cc:290] T 829b4ad8c5a54eababeea1181b960d4b P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:33.863730 19899 leader_election.cc:290] T af57996a8be84d398f0f023e8d08ef77 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 11 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:33.865574 16192 leader_election.cc:304] T 8c11074579b046d99e585c2818e18a2c P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860
I20260709 13:20:33.853751 16191 leader_election.cc:304] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:20:33.862205 19822 consensus_queue.cc:1243] T 84f2da98864b4425bf7c0055951fdd4b P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Peer 647a343a542d4a139fe106c19095129b log is divergent from this leader: its last log entry 6.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260709 13:20:33.871912 16295 raft_consensus.cc:3060] T 26354361e9b044cea1c2b86c4882fe92 P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:20:33.876356 16294 raft_consensus.cc:2468] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 12.
I20260709 13:20:33.888425 19699 consensus_queue.cc:1048] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:20:33.889959 19823 raft_consensus.cc:2749] T 8c11074579b046d99e585c2818e18a2c P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:20:33.890599 16818 raft_consensus.cc:1275] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [term 11 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 13:20:33.892247 16824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee9ed67006144797b1de6ea8c2013a68" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:33.892935 16824 raft_consensus.cc:2468] T ee9ed67006144797b1de6ea8c2013a68 P cb82cc7bd6db46d980c75237a1402860 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 7.
I20260709 13:20:33.855283 19942 raft_consensus.cc:493] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:33.893571 16151 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46224 (request call id 542) took 1039 ms. Trace:
I20260709 13:20:33.894063 16151 rpcz_store.cc:276] 0709 13:20:32.854001 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:32.854422 (+   421us) service_pool.cc:224] Handling call
0709 13:20:32.866928 (+ 12506us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:33.456733 (+589805us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:33.893309 (+436576us) spinlock_profiling.cc:243] Waited 585 ms on lock 0x7b540020c954. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:20:33.893506 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1346431872}
I20260709 13:20:33.899416 16296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e9b2bf7011f4ae688a79d7bb5a447b4" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
W20260709 13:20:33.899631 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 45947.8us
I20260709 13:20:33.900197 16296 raft_consensus.cc:2468] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 7.
I20260709 13:20:33.900507 19726 raft_consensus.cc:515] T b6f07842d3aa47858df5665874c86f3a P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:33.901197 19861 raft_consensus.cc:2749] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:20:33.901314 16191 leader_election.cc:304] T fffab26427e24d398e2695aad40ca134 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:20:33.902002 19861 consensus_queue.cc:1059] T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:20:33.902546 16191 leader_election.cc:304] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:20:33.822379 19997 raft_consensus.cc:2749] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:20:33.902962 19861 consensus_queue.cc:1048] T b6f07842d3aa47858df5665874c86f3a P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:20:33.903611 19997 raft_consensus.cc:3060] T fffab26427e24d398e2695aad40ca134 P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:20:33.902873 19726 leader_election.cc:290] T b6f07842d3aa47858df5665874c86f3a P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 11 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:33.904609 19861 raft_consensus.cc:3055] T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:20:33.905201 19861 raft_consensus.cc:740] T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:20:33.907500 16789 leader_election.cc:304] T 26354361e9b044cea1c2b86c4882fe92 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:20:33.908412 16789 leader_election.cc:304] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:20:33.909125 16192 leader_election.cc:304] T ee9ed67006144797b1de6ea8c2013a68 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:20:33.909453 16789 leader_election.cc:304] T 3a7ca61592e848ea9b0fb81a2ce33cdc P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:20:33.910154 19977 raft_consensus.cc:2804] T ee9ed67006144797b1de6ea8c2013a68 P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:20:33.910671 19977 raft_consensus.cc:493] T ee9ed67006144797b1de6ea8c2013a68 P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:33.910753 19785 raft_consensus.cc:2804] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:20:33.911299 16294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87ac0d1f7e8a419f9bcbc04aee0e067e" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:20:33.911387 19785 raft_consensus.cc:493] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:33.911617 16822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2372981006d4b218215c28964dea623" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:33.911911 19785 raft_consensus.cc:3060] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:20:33.912107 19904 raft_consensus.cc:2804] T 3a7ca61592e848ea9b0fb81a2ce33cdc P cb82cc7bd6db46d980c75237a1402860 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:20:33.912308 16822 raft_consensus.cc:2468] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 9.
I20260709 13:20:33.912725 19904 raft_consensus.cc:697] T 3a7ca61592e848ea9b0fb81a2ce33cdc P cb82cc7bd6db46d980c75237a1402860 [term 12 LEADER]: Becoming Leader. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:20:33.910521 19883 raft_consensus.cc:2804] T 26354361e9b044cea1c2b86c4882fe92 P cb82cc7bd6db46d980c75237a1402860 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:20:33.913619 19883 raft_consensus.cc:493] T 26354361e9b044cea1c2b86c4882fe92 P cb82cc7bd6db46d980c75237a1402860 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:33.913977 19883 raft_consensus.cc:3060] T 26354361e9b044cea1c2b86c4882fe92 P cb82cc7bd6db46d980c75237a1402860 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:20:33.914362 16821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada5a004ba7c46819051e3ccec71ffdb" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:33.909178 16825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7ca61592e848ea9b0fb81a2ce33cdc" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:20:33.915076 16821 raft_consensus.cc:2468] T ada5a004ba7c46819051e3ccec71ffdb P cb82cc7bd6db46d980c75237a1402860 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 9.
I20260709 13:20:33.837637 19859 raft_consensus.cc:2749] T d2ab0ac8347641e6955e9bce7d90c1a2 P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:20:33.913985 16822 raft_consensus.cc:1240] T 8638ee363273426db1393dd81e5c7b72 P cb82cc7bd6db46d980c75237a1402860 [term 10 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 9. Current term is 10. Ops: [9.9-9.9]
I20260709 13:20:33.894075 19942 raft_consensus.cc:515] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:33.919876 19667 raft_consensus.cc:493] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:33.920033 19345 raft_consensus.cc:493] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:33.920535 19667 raft_consensus.cc:515] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:33.920643 19345 raft_consensus.cc:515] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } }
I20260709 13:20:33.921430 19942 leader_election.cc:290] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:20:33.911914 16294 raft_consensus.cc:3060] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:20:33.923561 16296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f82224f4564fea93572e8206bad1fe" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:33.924101 19977 raft_consensus.cc:3060] T ee9ed67006144797b1de6ea8c2013a68 P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:20:33.925311 19785 raft_consensus.cc:515] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:33.928918 16824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a378d7f796f94615b3b347119a9880fd" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:33.930528 19922 consensus_queue.cc:1048] T 5a959dbecc2648eb85d1cc550f5a3245 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:20:33.909996 19955 consensus_queue.cc:1048] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:20:33.932935 19955 raft_consensus.cc:493] T 02838c86531d4f98bf58d1c22b8baf77 P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:20:33.933319 16115 leader_election.cc:304] T 1df93779209748b9860f6d893dd3de53 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:20:33.933669 19955 raft_consensus.cc:515] T 02838c86531d4f98bf58d1c22b8baf77 P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:33.934428 19899 raft_consensus.cc:2804] T 1df93779209748b9860f6d893dd3de53 P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:20:33.934906 19899 raft_consensus.cc:493] T 1df93779209748b9860f6d893dd3de53 P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:33.935286 19899 raft_consensus.cc:3060] T 1df93779209748b9860f6d893dd3de53 P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:20:33.936579 16296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02838c86531d4f98bf58d1c22b8baf77" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
W20260709 13:20:33.836208 19996 log.cc:927] Time spent T 5a959dbecc2648eb85d1cc550f5a3245 P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 0.056s	user 0.001s	sys 0.000s
I20260709 13:20:33.937768 16115 leader_election.cc:304] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:20:33.949509 16823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af57996a8be84d398f0f023e8d08ef77" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:33.950387 16823 raft_consensus.cc:2393] T af57996a8be84d398f0f023e8d08ef77 P cb82cc7bd6db46d980c75237a1402860 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 10: Already voted for candidate cb82cc7bd6db46d980c75237a1402860 in this term.
I20260709 13:20:33.950484 19785 leader_election.cc:290] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 10 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:33.922981 19345 leader_election.cc:290] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:20:33.828697 19966 raft_consensus.cc:515] T 18ad7e3af5504571a69d10c00750a01c P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } }
I20260709 13:20:33.957988 19966 leader_election.cc:290] T 18ad7e3af5504571a69d10c00750a01c P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:33.838313 19672 raft_consensus.cc:493] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:33.958726 16223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6f07842d3aa47858df5665874c86f3a" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:20:33.958838 19672 raft_consensus.cc:515] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:33.959637 19904 consensus_queue.cc:237] T 3a7ca61592e848ea9b0fb81a2ce33cdc P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:33.960589 16223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02838c86531d4f98bf58d1c22b8baf77" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:33.961221 19672 leader_election.cc:290] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:33.961392 16223 raft_consensus.cc:2468] T 02838c86531d4f98bf58d1c22b8baf77 P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 8.
I20260709 13:20:33.877116 19751 raft_consensus.cc:2804] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:20:33.962155 19751 raft_consensus.cc:493] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:33.941015 16192 leader_election.cc:304] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:20:33.964308 16192 leader_election.cc:304] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:20:33.938522 16296 raft_consensus.cc:2468] T 02838c86531d4f98bf58d1c22b8baf77 P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 9.
I20260709 13:20:33.922963 19667 leader_election.cc:290] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:33.942620 16294 raft_consensus.cc:2468] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 10.
I20260709 13:20:33.962538 19751 raft_consensus.cc:3060] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:20:33.972880 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 70140.6us
I20260709 13:20:33.973160 16824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e4bfb9090c8460eb14fa7e0d818e7f1" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 10 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:33.973259 16223 raft_consensus.cc:1275] T 68a6838d82ef4cac83507f906139d90f P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:20:33.942849 16295 raft_consensus.cc:2468] T 26354361e9b044cea1c2b86c4882fe92 P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 8.
I20260709 13:20:33.906279 19861 consensus_queue.cc:260] T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:33.970073 16224 raft_consensus.cc:2468] T 19baa7bc367844c288bca8f81439c413 P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 10.
I20260709 13:20:33.980849 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66f75124215f4ca6bef831e174a792d8" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:33.982581 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee9ed67006144797b1de6ea8c2013a68" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:20:33.984237 16115 leader_election.cc:304] T 02838c86531d4f98bf58d1c22b8baf77 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:20:33.985486 16115 leader_election.cc:304] T 26354361e9b044cea1c2b86c4882fe92 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
W20260709 13:20:33.991255 19964 log.cc:927] Time spent T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 0.830s	user 0.002s	sys 0.000s
W20260709 13:20:33.994735 19976 log.cc:927] Time spent T fffab26427e24d398e2695aad40ca134 P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.634s	user 0.001s	sys 0.000s
I20260709 13:20:33.995673 16222 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33050 (request call id 634) took 1311 ms. Trace:
I20260709 13:20:34.002250 16818 raft_consensus.cc:1240] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 [term 10 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 9. Current term is 10. Ops: []
I20260709 13:20:33.987135 16218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af57996a8be84d398f0f023e8d08ef77" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:20:33.996120 16222 rpcz_store.cc:276] 0709 13:20:32.684091 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:32.684303 (+   212us) service_pool.cc:224] Handling call
0709 13:20:32.684727 (+   424us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:20:32.778121 (+ 93394us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:20:32.788718 (+ 10597us) log.cc:844] Serialized 14 byte log entry
0709 13:20:33.694788 (+906070us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:20:33.695263 (+   475us) log.cc:844] Serialized 26 byte log entry
0709 13:20:33.695391 (+   128us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:20:33.695425 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:33.695491 (+    66us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:20:33.990542 (+295051us) spinlock_profiling.cc:243] Waited 30.8 ms on lock 0x7b4400022b4c. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69ad8fadb1 00007f69ad998c85 00007f69ad9996a2 00007f69b070b1db 00007f69b0706aaf 00007f69b06fa86e 00007f69b06f9bb6 00007f69b06f97d8 00007f69b0703221 00007f69b07031d9 00007f69b0703169
0709 13:20:33.995313 (+  4771us) raft_consensus.cc:1685] finished
0709 13:20:33.995416 (+   103us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:33.995613 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":696326,"prepare.queue_time_us":15296,"prepare.run_cpu_time_us":33,"prepare.run_wall_time_us":34,"raft.queue_time_us":1692783,"raft.run_cpu_time_us":292,"raft.run_wall_time_us":4077,"spinlock_wait_cycles":70918400,"thread_start_us":240885,"threads_started":2,"wal-append.queue_time_us":295465}
I20260709 13:20:34.003254 16218 raft_consensus.cc:3060] T af57996a8be84d398f0f023e8d08ef77 P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: Advancing to term 11
I20260709 13:20:33.977025 19822 raft_consensus.cc:2804] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:20:34.009413 16222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdbd78acb4cf46298e0de8ba0d130e19" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:33.972124 19966 raft_consensus.cc:2804] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:20:33.977758 19726 raft_consensus.cc:2804] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:20:33.985281 19667 raft_consensus.cc:3060] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:20:34.081691 16223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7ca61592e848ea9b0fb81a2ce33cdc" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:20:34.082928 16296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66ae176961b74813a58bc15c9c4c6f2a" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:34.024377 16300 raft_consensus.cc:2468] T e4dc6b0541ca431ebc757beac00521f1 P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 9.
W20260709 13:20:34.056200 16263 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:33.999819 19569 raft_consensus.cc:2804] T 02838c86531d4f98bf58d1c22b8baf77 P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:20:34.000349 19986 consensus_queue.cc:1059] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.142s
I20260709 13:20:34.005470 19942 raft_consensus.cc:493] T 2794af5b99984efebdee564035841cb0 P cb82cc7bd6db46d980c75237a1402860 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:34.115029 16222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66ae176961b74813a58bc15c9c4c6f2a" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:33.998395 19345 raft_consensus.cc:493] T 8158bbd17b994d2eb96f560bd0f1b88d P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:20:33.967247 19914 raft_consensus.cc:515] T 8cc959fb7f704fba9009c0595467d074 P 647a343a542d4a139fe106c19095129b [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:34.120110 16223 raft_consensus.cc:2393] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 in current term 12: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
I20260709 13:20:34.001343 19754 raft_consensus.cc:2804] T 26354361e9b044cea1c2b86c4882fe92 P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:20:34.125857 16296 raft_consensus.cc:2468] T 66ae176961b74813a58bc15c9c4c6f2a P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 11.
I20260709 13:20:34.137250 19754 raft_consensus.cc:697] T 26354361e9b044cea1c2b86c4882fe92 P 647a343a542d4a139fe106c19095129b [term 8 LEADER]: Becoming Leader. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:20:34.137542 16296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdbd78acb4cf46298e0de8ba0d130e19" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:20:34.138135 16296 raft_consensus.cc:3060] T cdbd78acb4cf46298e0de8ba0d130e19 P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:20:34.138226 19754 consensus_queue.cc:237] T 26354361e9b044cea1c2b86c4882fe92 P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:34.078580 19414 raft_consensus.cc:493] T 434bff3082d141838068f323bdb0d1ed P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:20:34.140749 19414 raft_consensus.cc:515] T 434bff3082d141838068f323bdb0d1ed P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:34.142865 19414 leader_election.cc:290] T 434bff3082d141838068f323bdb0d1ed P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:34.084956 16219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806d7690c2b644959152074bb2cab0a7" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:34.191955 16219 raft_consensus.cc:2368] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 for earlier term 7. Current term is 9.
I20260709 13:20:34.136390 16223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8638ee363273426db1393dd81e5c7b72" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:20:33.979614 19699 consensus_queue.cc:1059] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.113s
I20260709 13:20:34.114576 19861 raft_consensus.cc:3060] T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:20:34.149755 16220 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57604 (request call id 2443) took 1056 ms. Trace:
I20260709 13:20:34.160651 19939 raft_consensus.cc:515] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:34.162163 19977 raft_consensus.cc:515] T ee9ed67006144797b1de6ea8c2013a68 P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:34.130530 16218 raft_consensus.cc:2468] T af57996a8be84d398f0f023e8d08ef77 P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 11.
I20260709 13:20:34.078368 19938 raft_consensus.cc:493] T 68a6838d82ef4cac83507f906139d90f P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:34.168475 16296 raft_consensus.cc:2468] T cdbd78acb4cf46298e0de8ba0d130e19 P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 10.
I20260709 13:20:34.170912 16222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:20:34.208949 16222 raft_consensus.cc:2393] T 19baa7bc367844c288bca8f81439c413 P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 in current term 10: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
W20260709 13:20:34.119810 19940 log.cc:927] Time spent T 5a959dbecc2648eb85d1cc550f5a3245 P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 0.118s	user 0.000s	sys 0.000s
I20260709 13:20:34.016410 19726 raft_consensus.cc:493] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:34.010923 19822 raft_consensus.cc:493] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: Starting leader election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:20:34.015836 19966 raft_consensus.cc:493] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:34.165573 19982 raft_consensus.cc:515] T 8d4ab870a05347868956aa8b841b2981 P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:34.123610 19569 raft_consensus.cc:493] T 02838c86531d4f98bf58d1c22b8baf77 P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:34.169654 19899 raft_consensus.cc:515] T 1df93779209748b9860f6d893dd3de53 P 647a343a542d4a139fe106c19095129b [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:34.156884 19702 raft_consensus.cc:515] T 8a878999c0024d2abe517484180290fc P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:34.205750 16226 raft_consensus.cc:1240] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 8. Current term is 9. Ops: []
I20260709 13:20:34.227456 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62fe3346cd3c42c4a55e14af9073bca4" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 13 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:34.229010 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b7d1ab519044eda2d5e04a07242f25" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:20:34.193429 16220 rpcz_store.cc:276] 0709 13:20:33.093749 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:33.093986 (+   237us) service_pool.cc:224] Handling call
0709 13:20:33.121662 (+ 27676us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:20:33.760373 (+638711us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:33.760403 (+    30us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:20:33.760956 (+   553us) log.cc:844] Serialized 52 byte log entry
0709 13:20:33.969446 (+208490us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:20:33.969513 (+    67us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:33.969610 (+    97us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:20:34.149501 (+179891us) raft_consensus.cc:1685] finished
0709 13:20:34.149528 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:34.149692 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":624641,"raft.queue_time_us":622606,"raft.run_cpu_time_us":157,"raft.run_wall_time_us":1250,"thread_start_us":180802,"threads_started":1,"wal-append.queue_time_us":300058}
I20260709 13:20:34.204967 19948 raft_consensus.cc:493] T c0eab25ce4c64e018a041beeda2867db P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:34.209333 16296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6f07842d3aa47858df5665874c86f3a" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:20:34.193169 19823 raft_consensus.cc:493] T ecc1e363230043fe9fef390f0b979f20 P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:34.190670 19997 raft_consensus.cc:2749] T fffab26427e24d398e2695aad40ca134 P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:20:34.220731 16219 raft_consensus.cc:1275] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 13:20:34.172119 19345 raft_consensus.cc:515] T 8158bbd17b994d2eb96f560bd0f1b88d P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:34.219409 19785 raft_consensus.cc:3055] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860 [term 8 LEADER]: Stepping down as leader of term 8
W20260709 13:20:34.209628 16264 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:34.225899 19569 raft_consensus.cc:3060] T 02838c86531d4f98bf58d1c22b8baf77 P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:20:34.238003 16296 raft_consensus.cc:3060] T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:20:34.217681 19726 raft_consensus.cc:3060] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:20:34.239349 16222 raft_consensus.cc:1240] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 9. Current term is 10. Ops: [9.8-9.8]
I20260709 13:20:34.225144 16224 raft_consensus.cc:2368] T ee9ed67006144797b1de6ea8c2013a68 P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 for earlier term 7. Current term is 8.
W20260709 13:20:34.244228 16192 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:34.247503 16790 leader_election.cc:304] T 66f75124215f4ca6bef831e174a792d8 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:20:34.219939 19822 raft_consensus.cc:3060] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:20:34.187587 19942 raft_consensus.cc:515] T 2794af5b99984efebdee564035841cb0 P cb82cc7bd6db46d980c75237a1402860 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:34.219739 19966 raft_consensus.cc:3060] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:20:34.268261 16220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdbd78acb4cf46298e0de8ba0d130e19" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:20:34.225739 19972 raft_consensus.cc:3055] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:20:34.271322 16296 raft_consensus.cc:2468] T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 11.
I20260709 13:20:34.275285 16790 leader_election.cc:400] T 806d7690c2b644959152074bb2cab0a7 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 7 pre-election: Vote denied by peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943) with higher term. Message: Invalid argument: T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 for earlier term 7. Current term is 9.
I20260709 13:20:34.278319 16296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a9de347c711477e87c582d003f36925" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:34.279142 16296 raft_consensus.cc:2468] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 11.
I20260709 13:20:34.270886 19903 raft_consensus.cc:2749] T 66f75124215f4ca6bef831e174a792d8 P cb82cc7bd6db46d980c75237a1402860 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:20:34.218766 19938 raft_consensus.cc:515] T 68a6838d82ef4cac83507f906139d90f P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:34.251029 19785 raft_consensus.cc:740] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:20:34.260764 19955 leader_election.cc:290] T 02838c86531d4f98bf58d1c22b8baf77 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:34.285727 16790 leader_election.cc:304] T 19baa7bc367844c288bca8f81439c413 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:20:34.284821 16296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8638ee363273426db1393dd81e5c7b72" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:20:34.242033 19948 raft_consensus.cc:515] T c0eab25ce4c64e018a041beeda2867db P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:34.272117 19972 raft_consensus.cc:740] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:20:34.308933 16143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18ad7e3af5504571a69d10c00750a01c" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:34.289559 16296 raft_consensus.cc:3060] T 8638ee363273426db1393dd81e5c7b72 P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:20:34.292642 19802 raft_consensus.cc:493] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P cb82cc7bd6db46d980c75237a1402860 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:20:34.318346 19802 raft_consensus.cc:3060] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P cb82cc7bd6db46d980c75237a1402860 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:20:34.293382 16151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a9de347c711477e87c582d003f36925" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:34.320549 16143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62fe3346cd3c42c4a55e14af9073bca4" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 13 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:34.294252 19345 leader_election.cc:290] T 8158bbd17b994d2eb96f560bd0f1b88d P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:20:34.253123 19823 raft_consensus.cc:515] T ecc1e363230043fe9fef390f0b979f20 P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:34.296370 19939 leader_election.cc:290] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 10 election: Requested vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:34.296572 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57a744f5aaf944678a63469cbf514015" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 9 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:34.325716 16297 raft_consensus.cc:2410] T 57a744f5aaf944678a63469cbf514015 P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b for term 9 because replica has last-logged OpId of term: 7 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 7.
I20260709 13:20:34.298192 19785 consensus_queue.cc:260] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:34.301957 19914 leader_election.cc:290] T 8cc959fb7f704fba9009c0595467d074 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 10 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:34.305861 19806 raft_consensus.cc:3055] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:20:34.306875 19702 leader_election.cc:290] T 8a878999c0024d2abe517484180290fc P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 9 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:34.324452 16151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2372981006d4b218215c28964dea623" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:34.325815 16143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02838c86531d4f98bf58d1c22b8baf77" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:34.331357 16296 raft_consensus.cc:2468] T 8638ee363273426db1393dd81e5c7b72 P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 10.
I20260709 13:20:34.332202 16219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cc959fb7f704fba9009c0595467d074" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:20:34.333369 16151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b7d1ab519044eda2d5e04a07242f25" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:20:34.334401 16151 raft_consensus.cc:2393] T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 in current term 7: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
I20260709 13:20:34.337026 16790 leader_election.cc:304] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:20:34.312983 19414 raft_consensus.cc:493] T 5a959dbecc2648eb85d1cc550f5a3245 P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:34.339274 16265 leader_election.cc:304] T c9944d5e4b8c4109af9955c4f08c9095 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: cb82cc7bd6db46d980c75237a1402860
I20260709 13:20:34.339172 16151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:20:34.319847 19992 raft_consensus.cc:2749] T 19baa7bc367844c288bca8f81439c413 P cb82cc7bd6db46d980c75237a1402860 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:20:34.341723 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cc959fb7f704fba9009c0595467d074" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:20:34.342370 16297 raft_consensus.cc:3060] T 8cc959fb7f704fba9009c0595467d074 P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:20:34.328040 19345 raft_consensus.cc:493] T c4f65cdf00b748bb963ee6e81ef43b79 P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:34.290158 19883 raft_consensus.cc:515] T 26354361e9b044cea1c2b86c4882fe92 P cb82cc7bd6db46d980c75237a1402860 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:34.345950 20014 raft_consensus.cc:493] T ecc1e363230043fe9fef390f0b979f20 P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:20:34.356122 20014 raft_consensus.cc:515] T ecc1e363230043fe9fef390f0b979f20 P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:34.357172 19883 leader_election.cc:290] T 26354361e9b044cea1c2b86c4882fe92 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 8 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
W20260709 13:20:34.358716 19987 log.cc:927] Time spent T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 0.828s	user 0.000s	sys 0.031s
I20260709 13:20:34.327725 19806 raft_consensus.cc:740] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:20:34.338147 19965 raft_consensus.cc:2749] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:20:34.348836 16151 raft_consensus.cc:2393] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 in current term 10: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
I20260709 13:20:34.366091 16151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26354361e9b044cea1c2b86c4882fe92" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:20:34.321566 19809 consensus_queue.cc:1059] T 74ca8140f35c4ae29e8a664f6bdfc829 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.026s
I20260709 13:20:34.374109 19938 leader_election.cc:290] T 68a6838d82ef4cac83507f906139d90f P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:20:34.375265 19938 raft_consensus.cc:2804] T c9944d5e4b8c4109af9955c4f08c9095 P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:20:34.375694 19938 raft_consensus.cc:493] T c9944d5e4b8c4109af9955c4f08c9095 P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Starting leader election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:20:34.376111 19938 raft_consensus.cc:3060] T c9944d5e4b8c4109af9955c4f08c9095 P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:20:34.361752 19667 raft_consensus.cc:2749] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:20:34.368237 19345 raft_consensus.cc:515] T c4f65cdf00b748bb963ee6e81ef43b79 P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:34.386345 19982 leader_election.cc:290] T 8d4ab870a05347868956aa8b841b2981 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 11 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:34.386656 19345 leader_election.cc:290] T c4f65cdf00b748bb963ee6e81ef43b79 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:20:34.351007 19785 raft_consensus.cc:3060] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:20:34.347358 19414 raft_consensus.cc:515] T 5a959dbecc2648eb85d1cc550f5a3245 P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:34.389667 19414 leader_election.cc:290] T 5a959dbecc2648eb85d1cc550f5a3245 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:20:34.392112 16220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1df93779209748b9860f6d893dd3de53" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:20:34.394398 19948 leader_election.cc:290] T c0eab25ce4c64e018a041beeda2867db P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:20:34.399586 19899 leader_election.cc:290] T 1df93779209748b9860f6d893dd3de53 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 10 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:34.399669 16219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d4ab870a05347868956aa8b841b2981" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:20:34.390236 19806 consensus_queue.cc:260] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:20:34.403473 16115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 78130.8us
I20260709 13:20:34.351428 19972 consensus_queue.cc:260] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:20:34.372746 20005 log.cc:927] Time spent T 31969392ccdb43b491911a319e8bc7e1 P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.103s	user 0.001s	sys 0.000s
I20260709 13:20:34.373157 19942 leader_election.cc:290] T 2794af5b99984efebdee564035841cb0 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:34.374200 19809 raft_consensus.cc:3055] T 74ca8140f35c4ae29e8a664f6bdfc829 P cb82cc7bd6db46d980c75237a1402860 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:20:34.371872 19977 leader_election.cc:290] T ee9ed67006144797b1de6ea8c2013a68 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 8 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:34.411242 16151 raft_consensus.cc:1240] T 8cc959fb7f704fba9009c0595467d074 P 647a343a542d4a139fe106c19095129b [term 10 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 9. Current term is 10. Ops: [9.7-9.7]
I20260709 13:20:34.382550 19569 raft_consensus.cc:515] T 02838c86531d4f98bf58d1c22b8baf77 P 647a343a542d4a139fe106c19095129b [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:34.422760 16151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2794af5b99984efebdee564035841cb0" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:34.420720 16297 raft_consensus.cc:2468] T 8cc959fb7f704fba9009c0595467d074 P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 10.
I20260709 13:20:34.423632 16151 raft_consensus.cc:2468] T 2794af5b99984efebdee564035841cb0 P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 7.
W20260709 13:20:34.427320 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 88148.7us
I20260709 13:20:34.428249 16265 leader_election.cc:304] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860
I20260709 13:20:34.429275 19948 raft_consensus.cc:2749] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:20:34.431066 16265 leader_election.cc:304] T 18ad7e3af5504571a69d10c00750a01c P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:20:34.431901 19948 raft_consensus.cc:2749] T 18ad7e3af5504571a69d10c00750a01c P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:20:34.432708 19941 leader_election.cc:290] T 26354361e9b044cea1c2b86c4882fe92 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
W20260709 13:20:34.433357 16145 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40830 (request call id 1355) took 1584 ms (client timeout 1571 ms). Trace:
I20260709 13:20:34.407742 19345 raft_consensus.cc:493] T d16dfad0c3b540e5b4b86492ddf87208 P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:34.425873 16790 leader_election.cc:304] T 2794af5b99984efebdee564035841cb0 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860; no voters: 
W20260709 13:20:34.433764 16145 rpcz_store.cc:269] 0709 13:20:32.849123 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:32.849274 (+   151us) service_pool.cc:224] Handling call
0709 13:20:34.433129 (+1583855us) spinlock_profiling.cc:243] Waited 596 ms on lock 0x7b540006e0d4. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:20:34.433279 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1371307392}
I20260709 13:20:34.435458 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d4ab870a05347868956aa8b841b2981" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:20:34.411818 19899 raft_consensus.cc:493] T 3884f864a96d428785895e5463e5728f P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:20:34.436936 16297 raft_consensus.cc:3060] T 8d4ab870a05347868956aa8b841b2981 P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:20:34.416563 19809 raft_consensus.cc:740] T 74ca8140f35c4ae29e8a664f6bdfc829 P cb82cc7bd6db46d980c75237a1402860 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:20:34.441613 16143 raft_consensus.cc:2368] T 02838c86531d4f98bf58d1c22b8baf77 P 647a343a542d4a139fe106c19095129b [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac for earlier term 9. Current term is 10.
I20260709 13:20:34.439615 16265 leader_election.cc:304] T cdbd78acb4cf46298e0de8ba0d130e19 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860
I20260709 13:20:34.433827 19972 raft_consensus.cc:3060] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:20:34.440238 19942 raft_consensus.cc:2804] T 2794af5b99984efebdee564035841cb0 P cb82cc7bd6db46d980c75237a1402860 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:20:34.448179 19809 consensus_queue.cc:260] T 74ca8140f35c4ae29e8a664f6bdfc829 P cb82cc7bd6db46d980c75237a1402860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:34.412992 19802 raft_consensus.cc:515] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P cb82cc7bd6db46d980c75237a1402860 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:34.418905 19751 raft_consensus.cc:515] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:34.473991 20014 leader_election.cc:290] T ecc1e363230043fe9fef390f0b979f20 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:34.477303 19948 raft_consensus.cc:2749] T cdbd78acb4cf46298e0de8ba0d130e19 P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:20:34.490011 19966 raft_consensus.cc:515] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:34.459997 19942 raft_consensus.cc:493] T 2794af5b99984efebdee564035841cb0 P cb82cc7bd6db46d980c75237a1402860 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:34.522306 19726 raft_consensus.cc:515] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:34.501853 16297 raft_consensus.cc:2468] T 8d4ab870a05347868956aa8b841b2981 P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 11.
I20260709 13:20:34.589721 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4dc6b0541ca431ebc757beac00521f1" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:20:34.546823 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1df93779209748b9860f6d893dd3de53" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:20:34.554457 19992 raft_consensus.cc:493] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:20:34.555918 19941 raft_consensus.cc:493] T 8c11074579b046d99e585c2818e18a2c P cb82cc7bd6db46d980c75237a1402860 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:20:34.557251 19822 raft_consensus.cc:515] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:34.558199 19809 raft_consensus.cc:3060] T 74ca8140f35c4ae29e8a664f6bdfc829 P cb82cc7bd6db46d980c75237a1402860 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:20:34.441069 19345 raft_consensus.cc:515] T d16dfad0c3b540e5b4b86492ddf87208 P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:34.442461 19899 raft_consensus.cc:515] T 3884f864a96d428785895e5463e5728f P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:34.559924 16226 raft_consensus.cc:2368] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 for earlier term 7. Current term is 8.
I20260709 13:20:34.560375 19942 raft_consensus.cc:3060] T 2794af5b99984efebdee564035841cb0 P cb82cc7bd6db46d980c75237a1402860 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:20:34.492122 19982 raft_consensus.cc:493] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [term 10 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:20:34.590692 16298 raft_consensus.cc:3060] T 1df93779209748b9860f6d893dd3de53 P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:20:34.617072 16150 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40830 (request call id 1353) took 1768 ms (client timeout 1729 ms). Trace:
W20260709 13:20:34.655565 16150 rpcz_store.cc:269] 0709 13:20:32.848254 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:32.848405 (+   151us) service_pool.cc:224] Handling call
0709 13:20:34.616854 (+1768449us) spinlock_profiling.cc:243] Waited 198 ms on lock 0x7b540011b2d4. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:20:34.616982 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":455918336}
I20260709 13:20:34.631470 16146 raft_consensus.cc:1240] T 1df93779209748b9860f6d893dd3de53 P 647a343a542d4a139fe106c19095129b [term 10 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 9. Current term is 10. Ops: []
W20260709 13:20:34.686220 16264 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:34.718838 16151 raft_consensus.cc:1240] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 7. Current term is 8. Ops: [7.7-7.7]
I20260709 13:20:34.636116 19992 raft_consensus.cc:515] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:34.756873 19982 raft_consensus.cc:515] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:20:34.766356 16788 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:34.876860 16147 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46224 (request call id 547) took 1453 ms. Trace:
I20260709 13:20:34.877417 16147 rpcz_store.cc:276] 0709 13:20:33.423421 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:33.470017 (+ 46596us) service_pool.cc:224] Handling call
0709 13:20:34.876630 (+1406613us) spinlock_profiling.cc:243] Waited 206 ms on lock 0x7b54000749d4. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:20:34.876789 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":474423680}
I20260709 13:20:34.642011 19941 raft_consensus.cc:515] T 8c11074579b046d99e585c2818e18a2c P cb82cc7bd6db46d980c75237a1402860 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:20:34.937736 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 502540us
I20260709 13:20:34.949448 16298 raft_consensus.cc:2468] T 1df93779209748b9860f6d893dd3de53 P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 10.
I20260709 13:20:34.958205 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2794af5b99984efebdee564035841cb0" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:34.920663 19992 leader_election.cc:290] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:20:34.964954 16265 leader_election.cc:400] T 5a96373bb0e14dce8a266b2fc6c58507 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 8 pre-election: Vote denied by peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943) with higher term. Message: Invalid argument: T 5a96373bb0e14dce8a266b2fc6c58507 P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 for earlier term 8. Current term is 9.
I20260709 13:20:34.965342 16265 leader_election.cc:403] T 5a96373bb0e14dce8a266b2fc6c58507 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 8 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:20:34.968222 16147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746aabf856ab401c9ef5e859371e7b67" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:34.969004 16265 leader_election.cc:304] T 66ae176961b74813a58bc15c9c4c6f2a P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860
I20260709 13:20:34.972970 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26354361e9b044cea1c2b86c4882fe92" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:20:34.973728 16298 raft_consensus.cc:2393] T 26354361e9b044cea1c2b86c4882fe92 P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 in current term 8: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
I20260709 13:20:34.975524 19822 leader_election.cc:290] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 10 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
W20260709 13:20:34.976472 19785 consensus_meta.cc:287] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860: Time spent flushing consensus metadata: real 0.589s	user 0.009s	sys 0.000s
I20260709 13:20:34.979496 16220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02838c86531d4f98bf58d1c22b8baf77" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:20:34.980088 16220 raft_consensus.cc:3060] T 02838c86531d4f98bf58d1c22b8baf77 P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Advancing to term 10
I20260709 13:20:34.981467 19941 leader_election.cc:290] T 8c11074579b046d99e585c2818e18a2c P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:34.981961 19941 raft_consensus.cc:493] T 57a744f5aaf944678a63469cbf514015 P cb82cc7bd6db46d980c75237a1402860 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:34.982372 19941 raft_consensus.cc:515] T 57a744f5aaf944678a63469cbf514015 P cb82cc7bd6db46d980c75237a1402860 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:34.975068 16219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5d59407eb545b895d8d922e0dd69fd" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:34.984772 16219 raft_consensus.cc:2468] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 10.
I20260709 13:20:34.988298 16220 raft_consensus.cc:2468] T 02838c86531d4f98bf58d1c22b8baf77 P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 10.
I20260709 13:20:34.959023 19823 leader_election.cc:290] T ecc1e363230043fe9fef390f0b979f20 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:34.970021 19982 leader_election.cc:290] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:34.999205 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c11074579b046d99e585c2818e18a2c" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:34.972215 20016 raft_consensus.cc:2749] T 5a96373bb0e14dce8a266b2fc6c58507 P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Leader pre-election lost for term 8. Reason: Vote denied by peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943) with higher term. Message: Invalid argument: T 5a96373bb0e14dce8a266b2fc6c58507 P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 for earlier term 8. Current term is 9.
I20260709 13:20:35.008323 16298 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40474 (request call id 593) took 1211 ms. Trace:
I20260709 13:20:35.008728 16298 rpcz_store.cc:276] 0709 13:20:33.796927 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:34.997030 (+1200103us) service_pool.cc:224] Handling call
0709 13:20:35.007967 (+ 10937us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:35.008106 (+   139us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:35.008120 (+    14us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:20:35.008148 (+    28us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:20:35.008169 (+    21us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:35.008202 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:35.008274 (+    72us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:20:34.999878 20014 raft_consensus.cc:2749] T 66ae176961b74813a58bc15c9c4c6f2a P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:20:35.015049 16147 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46224 (request call id 557) took 1025 ms. Trace:
I20260709 13:20:35.003441 19569 leader_election.cc:290] T 02838c86531d4f98bf58d1c22b8baf77 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 10 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:35.016733 16147 rpcz_store.cc:276] 0709 13:20:33.989566 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:35.005042 (+1015476us) service_pool.cc:224] Handling call
0709 13:20:35.014689 (+  9647us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:35.014817 (+   128us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:35.014850 (+    33us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:20:35.014878 (+    28us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:20:35.014900 (+    22us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:35.014932 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:35.015007 (+    75us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:20:35.007901 19966 leader_election.cc:290] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 10 election: Requested vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:35.021302 16265 leader_election.cc:304] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:20:35.011749 19751 leader_election.cc:290] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 8 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:35.023291 19345 leader_election.cc:290] T d16dfad0c3b540e5b4b86492ddf87208 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:20:35.025154 19915 raft_consensus.cc:3055] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:20:35.025574 19915 raft_consensus.cc:740] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:20:35.026417 19915 consensus_queue.cc:260] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:35.027650 20014 raft_consensus.cc:2749] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:20:35.018438 19802 leader_election.cc:290] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 10 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:35.019299 19726 leader_election.cc:290] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 8 election: Requested vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:35.035456 16147 raft_consensus.cc:1240] T b6f07842d3aa47858df5665874c86f3a P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 10. Current term is 11. Ops: [10.7-10.7]
I20260709 13:20:35.036870 16147 raft_consensus.cc:1275] T b87147731a68400e8cce9b879daa7c5d P 647a343a542d4a139fe106c19095129b [term 10 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:20:35.044473 19941 leader_election.cc:290] T 57a744f5aaf944678a63469cbf514015 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:35.046700 16296 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40474 (request call id 586) took 1381 ms. Trace:
I20260709 13:20:35.047120 16296 rpcz_store.cc:276] 0709 13:20:33.665160 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:34.340370 (+675210us) service_pool.cc:224] Handling call
0709 13:20:34.340690 (+   320us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:20:34.439646 (+ 98956us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:34.439676 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:20:34.444973 (+  5297us) log.cc:844] Serialized 26 byte log entry
0709 13:20:34.848671 (+403698us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:20:34.848731 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:34.848828 (+    97us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:20:35.046482 (+197654us) raft_consensus.cc:1685] finished
0709 13:20:35.046505 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:35.046639 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":361,"raft.queue_time_us":33989,"raft.run_cpu_time_us":87,"raft.run_wall_time_us":88,"thread_start_us":435073,"threads_started":2,"wal-append.queue_time_us":497509}
I20260709 13:20:35.049000 16296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e4bfb9090c8460eb14fa7e0d818e7f1" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:20:35.051062 16296 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48674 (request call id 2517) took 1151 ms. Trace:
I20260709 13:20:35.051527 16296 rpcz_store.cc:276] 0709 13:20:33.900077 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:35.050110 (+1150033us) service_pool.cc:224] Handling call
0709 13:20:35.050478 (+   368us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:35.050715 (+   237us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:35.050751 (+    36us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:20:35.050799 (+    48us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:20:35.050838 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:35.050888 (+    50us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:35.050996 (+   108us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:20:35.053514 16296 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48674 (request call id 2519) took 1153 ms. Trace:
I20260709 13:20:35.054283 16296 rpcz_store.cc:276] 0709 13:20:33.900556 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:35.052638 (+1152082us) service_pool.cc:224] Handling call
0709 13:20:35.052941 (+   303us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:35.053152 (+   211us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:35.053194 (+    42us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:20:35.053248 (+    54us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:20:35.053291 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:35.053358 (+    67us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:35.053459 (+   101us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:20:35.055894 16296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57a744f5aaf944678a63469cbf514015" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:35.056723 16296 raft_consensus.cc:2468] T 57a744f5aaf944678a63469cbf514015 P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 7.
I20260709 13:20:35.053612 19915 raft_consensus.cc:3060] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:20:35.075484 20019 log.cc:927] Time spent T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
I20260709 13:20:35.079463 16145 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46224 (request call id 552) took 1160 ms. Trace:
I20260709 13:20:35.079841 16145 rpcz_store.cc:276] 0709 13:20:33.919202 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:34.436491 (+517289us) service_pool.cc:224] Handling call
0709 13:20:34.443287 (+  6796us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:20:34.443756 (+   469us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:20:34.443785 (+    29us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:20:34.468711 (+ 24926us) log.cc:844] Serialized 52 byte log entry
0709 13:20:34.727234 (+258523us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:20:34.727298 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:34.727403 (+   105us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:20:35.079234 (+351831us) raft_consensus.cc:1685] finished
0709 13:20:35.079260 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:35.079403 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":676,"raft.run_cpu_time_us":138,"raft.run_wall_time_us":12623,"thread_start_us":197714,"threads_started":1,"wal-append.queue_time_us":505815}
W20260709 13:20:35.081058 19938 consensus_meta.cc:287] T c9944d5e4b8c4109af9955c4f08c9095 P 90f3da57796b488c9087a2f523adf045: Time spent flushing consensus metadata: real 0.704s	user 0.011s	sys 0.000s
I20260709 13:20:35.081486 19938 raft_consensus.cc:515] T c9944d5e4b8c4109af9955c4f08c9095 P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:35.082540 16297 raft_consensus.cc:1240] T c9944d5e4b8c4109af9955c4f08c9095 P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 9. Current term is 10. Ops: []
W20260709 13:20:35.083385 16264 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:35.091423 19938 leader_election.cc:290] T c9944d5e4b8c4109af9955c4f08c9095 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 10 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:35.096956 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3884f864a96d428785895e5463e5728f" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:35.098242 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02838c86531d4f98bf58d1c22b8baf77" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:20:35.098796 16293 raft_consensus.cc:3060] T 02838c86531d4f98bf58d1c22b8baf77 P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:20:35.100335 16151 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46224 (request call id 553) took 1180 ms. Trace:
I20260709 13:20:35.100751 16151 rpcz_store.cc:276] 0709 13:20:33.919543 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:34.436851 (+517308us) service_pool.cc:224] Handling call
0709 13:20:34.437154 (+   303us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:20:34.726782 (+289628us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:35.100067 (+373285us) spinlock_profiling.cc:243] Waited 281 ms on lock 0x7b54000d0554. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:20:35.100276 (+   209us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":647117696}
W20260709 13:20:35.074950 16115 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:34.198746 15046 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:20:35.124310 16292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e9b2bf7011f4ae688a79d7bb5a447b4" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:20:35.125252 16150 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46224 (request call id 554) took 1177 ms. Trace:
I20260709 13:20:35.125743 16150 rpcz_store.cc:276] 0709 13:20:33.947771 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:34.719926 (+772155us) service_pool.cc:224] Handling call
0709 13:20:34.720322 (+   396us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:34.795021 (+ 74699us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:34.795066 (+    45us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:20:34.795132 (+    66us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:20:34.795192 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:35.125003 (+329811us) spinlock_profiling.cc:243] Waited 74.4 ms on lock 0x7b54000730d4. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:20:35.125052 (+    49us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:35.125189 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":171167488}
I20260709 13:20:35.132792 16295 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40474 (request call id 587) took 1450 ms. Trace:
I20260709 13:20:35.133481 16295 rpcz_store.cc:276] 0709 13:20:33.682585 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:34.615761 (+933176us) service_pool.cc:224] Handling call
0709 13:20:34.616172 (+   411us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:20:34.616619 (+   447us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:34.616642 (+    23us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:20:34.962258 (+345616us) log.cc:844] Serialized 26 byte log entry
0709 13:20:35.073135 (+110877us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:20:35.073198 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:35.073280 (+    82us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:20:35.132450 (+ 59170us) raft_consensus.cc:1685] finished
0709 13:20:35.132485 (+    35us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:35.132706 (+   221us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":9125,"raft.run_cpu_time_us":176,"raft.run_wall_time_us":8071,"thread_start_us":82986,"threads_started":1,"wal-append.queue_time_us":118983}
I20260709 13:20:35.159283 16789 leader_election.cc:304] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac
I20260709 13:20:35.160391 16789 leader_election.cc:304] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:20:35.161198 16789 leader_election.cc:304] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 647a343a542d4a139fe106c19095129b
I20260709 13:20:35.138075 16146 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40844 (request call id 2217) took 1788 ms. Trace:
I20260709 13:20:35.162138 16293 raft_consensus.cc:2468] T 02838c86531d4f98bf58d1c22b8baf77 P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 10.
I20260709 13:20:35.144359 19955 raft_consensus.cc:493] T 08c74490c40243a88e62d1760b5cd814 P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:20:35.147933 16148 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46224 (request call id 548) took 1723 ms (client timeout 2053 ms). Trace:
I20260709 13:20:35.150678 19939 raft_consensus.cc:493] T 68a6838d82ef4cac83507f906139d90f P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:35.151425 19702 raft_consensus.cc:493] T e5ad4eb281564936a59f41e387a47156 P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:20:35.152074 19952 raft_consensus.cc:493] T e4dc6b0541ca431ebc757beac00521f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:35.162421 16146 rpcz_store.cc:276] 0709 13:20:33.349400 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:33.849801 (+500401us) service_pool.cc:224] Handling call
0709 13:20:34.617280 (+767479us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:34.657071 (+ 39791us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:35.137873 (+480802us) spinlock_profiling.cc:243] Waited 767 ms on lock 0x7b540011b2d0. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5e8ad 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:20:35.138013 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1764241152}
I20260709 13:20:35.140201 16294 raft_consensus.cc:1275] T 68a6838d82ef4cac83507f906139d90f P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 6. (term mismatch)
I20260709 13:20:35.181955 16294 pending_rounds.cc:85] T 68a6838d82ef4cac83507f906139d90f P 90f3da57796b488c9087a2f523adf045: Aborting all ops after (but not including) 5
I20260709 13:20:35.182209 16294 pending_rounds.cc:107] T 68a6838d82ef4cac83507f906139d90f P 90f3da57796b488c9087a2f523adf045: Aborting uncommitted NO_OP operation due to leader change: 9.6
I20260709 13:20:35.182439 16294 raft_consensus.cc:2889] T 68a6838d82ef4cac83507f906139d90f P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:20:35.154199 16116 leader_election.cc:304] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 90f3da57796b488c9087a2f523adf045
I20260709 13:20:35.183918 16116 leader_election.cc:304] T 08c74490c40243a88e62d1760b5cd814 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:20:35.164240 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806d7690c2b644959152074bb2cab0a7" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:35.185703 16225 raft_consensus.cc:2468] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 9.
I20260709 13:20:35.165338 16222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8158bbd17b994d2eb96f560bd0f1b88d" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:35.166214 16115 leader_election.cc:304] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:20:35.165344 16226 raft_consensus.cc:1275] T fffab26427e24d398e2695aad40ca134 P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:20:35.167651 16789 leader_election.cc:304] T 8638ee363273426db1393dd81e5c7b72 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac
W20260709 13:20:35.167138 16148 rpcz_store.cc:269] 0709 13:20:33.424212 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:33.596482 (+172270us) service_pool.cc:224] Handling call
0709 13:20:35.147690 (+1551208us) spinlock_profiling.cc:243] Waited 164 ms on lock 0x7b54003d9ad4. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:20:35.147856 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":377836288}
I20260709 13:20:35.174188 16293 raft_consensus.cc:1275] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:20:35.195756 16220 raft_consensus.cc:1275] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 7 index: 9. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
I20260709 13:20:35.195557 19953 raft_consensus.cc:2804] T 8638ee363273426db1393dd81e5c7b72 P cb82cc7bd6db46d980c75237a1402860 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:20:35.198922 19953 raft_consensus.cc:697] T 8638ee363273426db1393dd81e5c7b72 P cb82cc7bd6db46d980c75237a1402860 [term 10 LEADER]: Becoming Leader. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:20:35.199704 19856 raft_consensus.cc:3055] T 84f2da98864b4425bf7c0055951fdd4b P 1ca2725d290e47d68b1a4b609c9816ac [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:20:35.200488 19856 raft_consensus.cc:740] T 84f2da98864b4425bf7c0055951fdd4b P 1ca2725d290e47d68b1a4b609c9816ac [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:20:35.199887 19953 consensus_queue.cc:237] T 8638ee363273426db1393dd81e5c7b72 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:35.201860 19856 consensus_queue.cc:260] T 84f2da98864b4425bf7c0055951fdd4b P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:35.203918 16293 raft_consensus.cc:1275] T 26354361e9b044cea1c2b86c4882fe92 P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:20:35.206246 16294 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48674 (request call id 2525) took 1038 ms. Trace:
I20260709 13:20:35.206442 16789 leader_election.cc:304] T 26354361e9b044cea1c2b86c4882fe92 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:20:35.207044 16115 leader_election.cc:304] T b6f07842d3aa47858df5665874c86f3a P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:20:35.207778 19197 raft_consensus.cc:2804] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 647a343a542d4a139fe106c19095129b [term 10 FOLLOWER]: Leader election won for term 10
W20260709 13:20:35.217592 16291 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48674 (request call id 2509) took 1667 ms (client timeout 1676 ms). Trace:
W20260709 13:20:35.222661 16300 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48674 (request call id 2494) took 1808 ms (client timeout 1927 ms). Trace:
W20260709 13:20:35.276870 16291 rpcz_store.cc:269] 0709 13:20:33.550332 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:33.640416 (+ 90084us) service_pool.cc:224] Handling call
0709 13:20:35.217369 (+1576953us) spinlock_profiling.cc:243] Waited 29 ms on lock 0x7b54001ee954. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:20:35.217499 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":66702208}
W20260709 13:20:35.277505 16300 rpcz_store.cc:269] 0709 13:20:33.414448 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:33.414591 (+   143us) service_pool.cc:224] Handling call
0709 13:20:35.222453 (+1807862us) spinlock_profiling.cc:243] Waited 373 ms on lock 0x7b540030bbd4. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:20:35.222592 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":857370496}
I20260709 13:20:35.224531 16296 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48674 (request call id 2523) took 1254 ms. Trace:
W20260709 13:20:35.278420 16789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 72103.4us
I20260709 13:20:35.274765 16294 rpcz_store.cc:276] 0709 13:20:34.167938 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:35.139443 (+971505us) service_pool.cc:224] Handling call
0709 13:20:35.139798 (+   355us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:35.205904 (+ 66106us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:35.206173 (+   269us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:20:35.278716 16296 rpcz_store.cc:276] 0709 13:20:33.969785 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:35.057498 (+1087713us) service_pool.cc:224] Handling call
0709 13:20:35.057967 (+   469us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:20:35.058491 (+   524us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:35.058533 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:20:35.066719 (+  8186us) log.cc:844] Serialized 26 byte log entry
0709 13:20:35.171049 (+104330us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:20:35.223418 (+ 52369us) log.cc:844] Serialized 14 byte log entry
0709 13:20:35.224172 (+   754us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:35.224256 (+    84us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:20:35.224300 (+    44us) raft_consensus.cc:1685] finished
0709 13:20:35.224314 (+    14us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:35.224464 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":36361,"prepare.run_cpu_time_us":36,"prepare.run_wall_time_us":36,"raft.queue_time_us":398,"raft.run_cpu_time_us":231,"raft.run_wall_time_us":2040,"thread_start_us":155506,"threads_started":2,"wal-append.queue_time_us":147845}
W20260709 13:20:35.232120 19957 log.cc:927] Time spent T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860: Append to log took a long time: real 2.198s	user 0.007s	sys 0.009s
I20260709 13:20:35.234071 16149 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46224 (request call id 549) took 1688 ms. Trace:
W20260709 13:20:35.235401 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 53907us
W20260709 13:20:35.236398 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 215137us
I20260709 13:20:35.286397 16265 leader_election.cc:400] T e4dc6b0541ca431ebc757beac00521f1 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 8 election: Vote denied by peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943) with higher term. Message: Invalid argument: T e4dc6b0541ca431ebc757beac00521f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 for earlier term 8. Current term is 9.
I20260709 13:20:35.284530 16149 rpcz_store.cc:276] 0709 13:20:33.546085 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:33.853748 (+307663us) service_pool.cc:224] Handling call
0709 13:20:34.878025 (+1024277us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:20:35.066990 (+188965us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:35.067022 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:20:35.067520 (+   498us) log.cc:844] Serialized 26 byte log entry
0709 13:20:35.169855 (+102335us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:20:35.170452 (+   597us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:35.170540 (+    88us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:20:35.232144 (+ 61604us) raft_consensus.cc:1685] finished
0709 13:20:35.232176 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:35.233695 (+  1519us) spinlock_profiling.cc:243] Waited 1.02 s on lock 0x7b54000749d0. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5e8ad 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:20:35.233966 (+   271us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":88115,"raft.queue_time_us":117945,"raft.run_cpu_time_us":128,"raft.run_wall_time_us":5956,"spinlock_wait_cycles":2354899200,"thread_start_us":199539,"threads_started":2,"wal-append.queue_time_us":160544}
I20260709 13:20:35.235414 16295 raft_consensus.cc:1275] T b87147731a68400e8cce9b879daa7c5d P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:20:35.170500 19991 raft_consensus.cc:2804] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:20:35.205631 19974 raft_consensus.cc:2804] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:20:35.289028 16789 leader_election.cc:304] T 57a744f5aaf944678a63469cbf514015 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:20:35.170562 19962 raft_consensus.cc:2804] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:20:35.182989 19992 raft_consensus.cc:2804] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:20:35.265363 16224 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33050 (request call id 637) took 1290 ms. Trace:
I20260709 13:20:35.293740 16224 rpcz_store.cc:276] 0709 13:20:33.974882 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:33.982293 (+  7411us) service_pool.cc:224] Handling call
0709 13:20:35.265028 (+1282735us) spinlock_profiling.cc:243] Waited 242 ms on lock 0x7b54002b4654. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:20:35.265270 (+   242us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":555914496}
I20260709 13:20:35.294549 16293 raft_consensus.cc:1275] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:20:35.275130 16298 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48674 (request call id 2515) took 1375 ms. Trace:
I20260709 13:20:35.295581 16265 leader_election.cc:304] T e5ad4eb281564936a59f41e387a47156 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
W20260709 13:20:35.275601 16115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 68654.8us
I20260709 13:20:35.298708 16115 leader_election.cc:304] T 8cc959fb7f704fba9009c0595467d074 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:20:35.205085 19760 raft_consensus.cc:2749] T 08c74490c40243a88e62d1760b5cd814 P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:20:35.169979 19575 consensus_queue.cc:1048] T c0eab25ce4c64e018a041beeda2867db P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.094s
I20260709 13:20:35.302703 16115 leader_election.cc:304] T 8d4ab870a05347868956aa8b841b2981 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:20:35.179255 19955 raft_consensus.cc:515] T 08c74490c40243a88e62d1760b5cd814 P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:35.295858 16298 rpcz_store.cc:276] 0709 13:20:33.899540 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:35.009502 (+1109962us) service_pool.cc:224] Handling call
0709 13:20:35.018119 (+  8617us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:20:35.020165 (+  2046us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:35.021881 (+  1716us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:20:35.022319 (+   438us) log.cc:844] Serialized 26 byte log entry
0709 13:20:35.167330 (+145011us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:20:35.167367 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:35.167427 (+    60us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:20:35.274930 (+107503us) raft_consensus.cc:1685] finished
0709 13:20:35.274951 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:35.275071 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":552,"raft.queue_time_us":1466,"raft.run_cpu_time_us":129,"raft.run_wall_time_us":8087,"thread_start_us":134025,"threads_started":1,"wal-append.queue_time_us":183995}
I20260709 13:20:35.183146 19939 raft_consensus.cc:515] T 68a6838d82ef4cac83507f906139d90f P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:35.283390 19966 raft_consensus.cc:493] T d16dfad0c3b540e5b4b86492ddf87208 P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:35.308979 16115 leader_election.cc:304] T 1df93779209748b9860f6d893dd3de53 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:20:35.263526 19699 consensus_queue.cc:1059] T 8cc959fb7f704fba9009c0595467d074 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.060s
I20260709 13:20:35.294582 19197 raft_consensus.cc:697] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 647a343a542d4a139fe106c19095129b [term 10 LEADER]: Becoming Leader. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:20:35.337937 16218 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33050 (request call id 639) took 1362 ms. Trace:
I20260709 13:20:35.324450 19789 raft_consensus.cc:2804] T 8d4ab870a05347868956aa8b841b2981 P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:20:35.195878 19702 raft_consensus.cc:515] T e5ad4eb281564936a59f41e387a47156 P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:35.294970 19962 raft_consensus.cc:493] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:20:35.326274 19955 leader_election.cc:290] T 08c74490c40243a88e62d1760b5cd814 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:35.276716 19816 raft_consensus.cc:2804] T b6f07842d3aa47858df5665874c86f3a P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:20:35.310065 19914 raft_consensus.cc:2804] T 8cc959fb7f704fba9009c0595467d074 P 647a343a542d4a139fe106c19095129b [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:20:35.263983 19942 raft_consensus.cc:515] T 2794af5b99984efebdee564035841cb0 P cb82cc7bd6db46d980c75237a1402860 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:35.315857 19974 raft_consensus.cc:697] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b [term 9 LEADER]: Becoming Leader. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:20:35.331974 16265 leader_election.cc:304] T d2ab0ac8347641e6955e9bce7d90c1a2 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860
I20260709 13:20:35.276381 19855 raft_consensus.cc:493] T 5a96373bb0e14dce8a266b2fc6c58507 P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:20:35.288110 19990 raft_consensus.cc:2749] T 26354361e9b044cea1c2b86c4882fe92 P cb82cc7bd6db46d980c75237a1402860 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:20:35.338462 16218 rpcz_store.cc:276] 0709 13:20:33.975541 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:34.214645 (+239104us) service_pool.cc:224] Handling call
0709 13:20:34.215112 (+   467us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:20:35.124508 (+909396us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:35.124542 (+    34us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:20:35.125206 (+   664us) log.cc:844] Serialized 52 byte log entry
0709 13:20:35.174298 (+ 49092us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:20:35.306947 (+132649us) log.cc:844] Serialized 14 byte log entry
0709 13:20:35.328152 (+ 21205us) log.cc:844] Serialized 14 byte log entry
0709 13:20:35.337558 (+  9406us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:35.337655 (+    97us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:20:35.337697 (+    42us) raft_consensus.cc:1685] finished
0709 13:20:35.337710 (+    13us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:35.337840 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":940768,"raft.queue_time_us":936939,"raft.run_cpu_time_us":129,"raft.run_wall_time_us":17414,"thread_start_us":156740,"threads_started":2,"wal-append.queue_time_us":47832}
W20260709 13:20:35.369840 16819 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49892 (request call id 547) took 2196 ms (client timeout 1715 ms). Trace:
I20260709 13:20:35.293673 19992 raft_consensus.cc:493] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:35.294695 19822 raft_consensus.cc:493] T c2f5f89186204242bf377f7c42b2e2da P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:20:35.296173 19899 leader_election.cc:290] T 3884f864a96d428785895e5463e5728f P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:35.307049 19802 raft_consensus.cc:2804] T 57a744f5aaf944678a63469cbf514015 P cb82cc7bd6db46d980c75237a1402860 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:20:35.328022 19863 raft_consensus.cc:493] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:35.328617 19785 consensus_queue.cc:1048] T 3a7ca61592e848ea9b0fb81a2ce33cdc P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.057s
I20260709 13:20:35.301962 19823 consensus_queue.cc:1059] T b6f07842d3aa47858df5665874c86f3a P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.060s
I20260709 13:20:35.371306 16296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada5a004ba7c46819051e3ccec71ffdb" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:35.404673 16296 raft_consensus.cc:2468] T ada5a004ba7c46819051e3ccec71ffdb P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 9.
I20260709 13:20:35.405606 16296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a878999c0024d2abe517484180290fc" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:20:35.204710 19952 raft_consensus.cc:515] T e4dc6b0541ca431ebc757beac00521f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:20:35.370232 16819 rpcz_store.cc:269] 0709 13:20:33.161458 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:33.161601 (+   143us) service_pool.cc:224] Handling call
0709 13:20:35.357854 (+2196253us) spinlock_profiling.cc:243] Waited 193 ms on lock 0x7b54006dbfd4. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:20:35.357984 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":442914688}
I20260709 13:20:35.342943 19760 raft_consensus.cc:2804] T 1df93779209748b9860f6d893dd3de53 P 647a343a542d4a139fe106c19095129b [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:20:35.364440 19789 raft_consensus.cc:697] T 8d4ab870a05347868956aa8b841b2981 P 647a343a542d4a139fe106c19095129b [term 11 LEADER]: Becoming Leader. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:20:35.375401 16265 leader_election.cc:304] T 02838c86531d4f98bf58d1c22b8baf77 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:20:35.337144 19977 raft_consensus.cc:493] T 5a959dbecc2648eb85d1cc550f5a3245 P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:20:35.433112 16296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a378d7f796f94615b3b347119a9880fd" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 10 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:20:35.381373 16115 leader_election.cc:304] T 02838c86531d4f98bf58d1c22b8baf77 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:20:35.315578 16298 raft_consensus.cc:1240] T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 10. Current term is 11. Ops: [7.5-10.7]
I20260709 13:20:35.355182 19962 raft_consensus.cc:3060] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:20:35.365517 19856 raft_consensus.cc:3060] T 84f2da98864b4425bf7c0055951fdd4b P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:20:35.377256 19992 raft_consensus.cc:3060] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:20:35.386729 20034 log.cc:927] Time spent T 8638ee363273426db1393dd81e5c7b72 P cb82cc7bd6db46d980c75237a1402860: Append to log took a long time: real 0.152s	user 0.001s	sys 0.000s
I20260709 13:20:35.437076 16143 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40844 (request call id 2227) took 1191 ms. Trace:
I20260709 13:20:35.344192 19966 raft_consensus.cc:515] T d16dfad0c3b540e5b4b86492ddf87208 P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:35.373566 19816 raft_consensus.cc:697] T b6f07842d3aa47858df5665874c86f3a P 647a343a542d4a139fe106c19095129b [term 11 LEADER]: Becoming Leader. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:20:35.386451 19982 raft_consensus.cc:493] T 8a878999c0024d2abe517484180290fc P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
W20260709 13:20:35.481523 16266 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.14.177.132:38923 timed out after 1.715s (SENT) [suppressed 10 similar messages]
I20260709 13:20:35.384263 20016 raft_consensus.cc:3060] T d2ab0ac8347641e6955e9bce7d90c1a2 P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:20:35.383991 19802 raft_consensus.cc:493] T 57a744f5aaf944678a63469cbf514015 P cb82cc7bd6db46d980c75237a1402860 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:35.379575 19914 raft_consensus.cc:697] T 8cc959fb7f704fba9009c0595467d074 P 647a343a542d4a139fe106c19095129b [term 10 LEADER]: Becoming Leader. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:20:35.386443 19699 raft_consensus.cc:493] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:20:35.511049 16143 rpcz_store.cc:276] 0709 13:20:34.245797 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:34.325565 (+ 79768us) service_pool.cc:224] Handling call
0709 13:20:35.436892 (+1111327us) spinlock_profiling.cc:243] Waited 111 ms on lock 0x7b5400427a54. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:20:35.437021 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":255932288}
I20260709 13:20:35.385202 19955 raft_consensus.cc:3055] T 8cc959fb7f704fba9009c0595467d074 P 1ca2725d290e47d68b1a4b609c9816ac [term 9 LEADER]: Stepping down as leader of term 9
W20260709 13:20:35.520956 16189 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:35.373162 19855 raft_consensus.cc:515] T 5a96373bb0e14dce8a266b2fc6c58507 P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:20:35.530308 16263 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:35.506196 19861 raft_consensus.cc:493] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:35.479681 19760 raft_consensus.cc:697] T 1df93779209748b9860f6d893dd3de53 P 647a343a542d4a139fe106c19095129b [term 10 LEADER]: Becoming Leader. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:20:35.506307 19990 raft_consensus.cc:493] T 829b4ad8c5a54eababeea1181b960d4b P cb82cc7bd6db46d980c75237a1402860 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:20:35.532500 16291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada5a004ba7c46819051e3ccec71ffdb" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:20:35.560544 16291 raft_consensus.cc:3060] T ada5a004ba7c46819051e3ccec71ffdb P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:20:35.509354 19823 raft_consensus.cc:493] T 66f75124215f4ca6bef831e174a792d8 P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:20:35.557323 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18ad7e3af5504571a69d10c00750a01c" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:35.510769 20022 raft_consensus.cc:493] T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:35.403868 19863 raft_consensus.cc:515] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:35.568774 16298 raft_consensus.cc:2468] T 18ad7e3af5504571a69d10c00750a01c P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 8.
I20260709 13:20:35.386549 19822 raft_consensus.cc:515] T c2f5f89186204242bf377f7c42b2e2da P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:35.525702 19955 raft_consensus.cc:740] T 8cc959fb7f704fba9009c0595467d074 P 1ca2725d290e47d68b1a4b609c9816ac [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:20:35.523085 19802 raft_consensus.cc:3060] T 57a744f5aaf944678a63469cbf514015 P cb82cc7bd6db46d980c75237a1402860 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:20:35.583971 19939 leader_election.cc:290] T 68a6838d82ef4cac83507f906139d90f P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
W20260709 13:20:35.566710 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 198277us
I20260709 13:20:35.479136 19977 raft_consensus.cc:515] T 5a959dbecc2648eb85d1cc550f5a3245 P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:20:35.529147 16115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 147838us
I20260709 13:20:35.590596 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02838c86531d4f98bf58d1c22b8baf77" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:35.590984 16115 leader_election.cc:304] T b87147731a68400e8cce9b879daa7c5d P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:20:35.592202 16298 raft_consensus.cc:2368] T 02838c86531d4f98bf58d1c22b8baf77 P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac for earlier term 9. Current term is 10.
I20260709 13:20:35.529615 19699 raft_consensus.cc:515] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:35.516271 19197 consensus_queue.cc:237] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:35.587664 16292 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48674 (request call id 2524) took 1470 ms. Trace:
I20260709 13:20:35.557909 19899 raft_consensus.cc:2804] T 02838c86531d4f98bf58d1c22b8baf77 P 647a343a542d4a139fe106c19095129b [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:20:35.606357 16292 rpcz_store.cc:276] 0709 13:20:34.116879 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:35.125507 (+1008628us) service_pool.cc:224] Handling call
0709 13:20:35.125833 (+   326us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:35.126086 (+   253us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:20:35.277692 (+151606us) log.cc:844] Serialized 14 byte log entry
0709 13:20:35.587203 (+309511us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:20:35.587277 (+    74us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:20:35.587343 (+    66us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:35.587432 (+    89us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:35.587589 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":81440,"prepare.run_cpu_time_us":76,"prepare.run_wall_time_us":77,"thread_start_us":460217,"threads_started":2}
W20260709 13:20:35.611276 16190 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:20:35.617923 16264 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:35.622995 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0eab25ce4c64e018a041beeda2867db" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:35.624830 16292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:35.627427 16291 raft_consensus.cc:2468] T ada5a004ba7c46819051e3ccec71ffdb P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 10.
I20260709 13:20:35.628021 16292 raft_consensus.cc:2410] T ecc1e363230043fe9fef390f0b979f20 P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac for term 10 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 13:20:35.628815 16291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e4bfb9090c8460eb14fa7e0d818e7f1" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 10 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:35.630245 16291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c74490c40243a88e62d1760b5cd814" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 10 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:35.632046 16291 raft_consensus.cc:2410] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac for term 10 because replica has last-logged OpId of term: 7 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 5.
I20260709 13:20:35.636943 16291 raft_consensus.cc:1240] T 8638ee363273426db1393dd81e5c7b72 P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 9. Current term is 10. Ops: [7.8-9.9]
I20260709 13:20:35.638690 16291 raft_consensus.cc:1240] T 8cc959fb7f704fba9009c0595467d074 P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 9. Current term is 10. Ops: [8.6-9.7]
I20260709 13:20:35.640066 16291 raft_consensus.cc:1240] T 1df93779209748b9860f6d893dd3de53 P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 9. Current term is 10. Ops: []
I20260709 13:20:35.561389 19861 raft_consensus.cc:515] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:35.589175 19938 raft_consensus.cc:2749] T e5ad4eb281564936a59f41e387a47156 P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:20:35.652072 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746aabf856ab401c9ef5e859371e7b67" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:35.526505 19982 raft_consensus.cc:515] T 8a878999c0024d2abe517484180290fc P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:35.641498 16822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:20:35.660211 16822 raft_consensus.cc:2393] T 19baa7bc367844c288bca8f81439c413 P cb82cc7bd6db46d980c75237a1402860 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 10: Already voted for candidate cb82cc7bd6db46d980c75237a1402860 in this term.
I20260709 13:20:35.595655 19939 raft_consensus.cc:493] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:35.652674 16143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57a744f5aaf944678a63469cbf514015" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:35.661075 16143 raft_consensus.cc:2468] T 57a744f5aaf944678a63469cbf514015 P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 8.
I20260709 13:20:35.569785 19823 raft_consensus.cc:515] T 66f75124215f4ca6bef831e174a792d8 P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:35.653005 16817 raft_consensus.cc:1275] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [term 12 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:20:35.654388 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7ca61592e848ea9b0fb81a2ce33cdc" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:20:35.649703 16291 raft_consensus.cc:1240] T 8d4ab870a05347868956aa8b841b2981 P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 10. Current term is 11. Ops: []
I20260709 13:20:35.656837 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c11074579b046d99e585c2818e18a2c" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:35.663872 16224 raft_consensus.cc:2468] T 8c11074579b046d99e585c2818e18a2c P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 9.
I20260709 13:20:35.657533 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2794af5b99984efebdee564035841cb0" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:20:35.657998 16825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e9b2bf7011f4ae688a79d7bb5a447b4" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:35.664889 16821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31969392ccdb43b491911a319e8bc7e1" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:20:35.658656 16221 raft_consensus.cc:1240] T ecc1e363230043fe9fef390f0b979f20 P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 8. Current term is 9. Ops: [8.6-8.6]
I20260709 13:20:35.665759 16821 raft_consensus.cc:2410] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b for term 9 because replica has last-logged OpId of term: 8 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 7.
I20260709 13:20:35.664911 16149 raft_consensus.cc:3060] T 2794af5b99984efebdee564035841cb0 P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:20:35.659027 16816 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49892 (request call id 544) took 2647 ms. Trace:
I20260709 13:20:35.671391 16816 rpcz_store.cc:276] 0709 13:20:33.000050 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:33.000266 (+   216us) service_pool.cc:224] Handling call
0709 13:20:33.000637 (+   371us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:33.819096 (+818459us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:35.647312 (+1828216us) spinlock_profiling.cc:243] Waited 789 ms on lock 0x7b54006eebd4. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:20:35.647471 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1813700864}
I20260709 13:20:35.682288 16298 raft_consensus.cc:2393] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 12: Already voted for candidate cb82cc7bd6db46d980c75237a1402860 in this term.
I20260709 13:20:35.683866 16218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e4bfb9090c8460eb14fa7e0d818e7f1" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:20:35.684453 16218 raft_consensus.cc:3060] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:20:35.685660 16790 leader_election.cc:400] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 7 election: Vote denied by peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943) with higher term. Message: Invalid argument: T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 for earlier term 7. Current term is 8.
I20260709 13:20:35.685573 16816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26354361e9b044cea1c2b86c4882fe92" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:20:35.686203 16790 leader_election.cc:403] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 7 election: Cancelling election due to peer responding with higher term
I20260709 13:20:35.686770 16816 raft_consensus.cc:2393] T 26354361e9b044cea1c2b86c4882fe92 P cb82cc7bd6db46d980c75237a1402860 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 8: Already voted for candidate cb82cc7bd6db46d980c75237a1402860 in this term.
W20260709 13:20:35.684927 16266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 203428us
I20260709 13:20:35.688285 16823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87147731a68400e8cce9b879daa7c5d" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:35.688980 16223 raft_consensus.cc:1275] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:20:35.564725 19990 raft_consensus.cc:515] T 829b4ad8c5a54eababeea1181b960d4b P cb82cc7bd6db46d980c75237a1402860 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:35.570693 19789 consensus_queue.cc:237] T 8d4ab870a05347868956aa8b841b2981 P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 9.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:35.700313 16190 leader_election.cc:304] T 18ad7e3af5504571a69d10c00750a01c P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 647a343a542d4a139fe106c19095129b
I20260709 13:20:35.598173 20021 consensus_queue.cc:1048] T 26354361e9b044cea1c2b86c4882fe92 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.068s
I20260709 13:20:35.642874 19899 raft_consensus.cc:697] T 02838c86531d4f98bf58d1c22b8baf77 P 647a343a542d4a139fe106c19095129b [term 10 LEADER]: Becoming Leader. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:20:35.654125 19863 leader_election.cc:290] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:35.691946 16790 leader_election.cc:400] T ee9ed67006144797b1de6ea8c2013a68 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 7 election: Vote denied by peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943) with higher term. Message: Invalid argument: T ee9ed67006144797b1de6ea8c2013a68 P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 for earlier term 7. Current term is 8.
I20260709 13:20:35.643889 19754 consensus_queue.cc:1048] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 6, Time since last communication: 0.074s
I20260709 13:20:35.563764 19974 consensus_queue.cc:237] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.6, Last appended by leader: 6, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } }
I20260709 13:20:35.573834 20022 raft_consensus.cc:515] T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:20:35.733323 16144 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40844 (request call id 2216) took 2372 ms (client timeout 2023 ms). Trace:
I20260709 13:20:35.718696 19942 leader_election.cc:290] T 2794af5b99984efebdee564035841cb0 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 8 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
W20260709 13:20:35.751592 16144 rpcz_store.cc:269] 0709 13:20:33.348990 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:33.442031 (+ 93041us) service_pool.cc:224] Handling call
0709 13:20:35.717288 (+2275257us) spinlock_profiling.cc:243] Waited 339 ms on lock 0x7b54000e9a54. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:20:35.721098 (+  3810us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":778661632}
I20260709 13:20:35.734356 16790 leader_election.cc:403] T ee9ed67006144797b1de6ea8c2013a68 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 7 election: Cancelling election due to peer responding with higher term
I20260709 13:20:35.744124 16149 raft_consensus.cc:2468] T 2794af5b99984efebdee564035841cb0 P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 8.
I20260709 13:20:35.600579 19816 consensus_queue.cc:237] T b6f07842d3aa47858df5665874c86f3a P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:35.733912 16823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f82224f4564fea93572e8206bad1fe" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:35.749255 16822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "829b4ad8c5a54eababeea1181b960d4b" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 10 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:35.686797 19991 raft_consensus.cc:697] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 [term 10 LEADER]: Becoming Leader. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
W20260709 13:20:35.723613 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 348151us
I20260709 13:20:35.767212 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68a6838d82ef4cac83507f906139d90f" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:35.767836 16298 raft_consensus.cc:2468] T 68a6838d82ef4cac83507f906139d90f P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 10.
I20260709 13:20:35.765949 16821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af57996a8be84d398f0f023e8d08ef77" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:20:35.768898 16821 raft_consensus.cc:3060] T af57996a8be84d398f0f023e8d08ef77 P cb82cc7bd6db46d980c75237a1402860 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:20:35.769392 16265 leader_election.cc:304] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:20:35.753449 16817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57a744f5aaf944678a63469cbf514015" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 9 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:35.634928 19914 consensus_queue.cc:237] T 8cc959fb7f704fba9009c0595467d074 P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:35.683696 19939 raft_consensus.cc:515] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:35.781334 16298 raft_consensus.cc:1240] T cdbd78acb4cf46298e0de8ba0d130e19 P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 9. Current term is 10. Ops: []
I20260709 13:20:35.651046 19760 consensus_queue.cc:237] T 1df93779209748b9860f6d893dd3de53 P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:35.796657 16824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5d59407eb545b895d8d922e0dd69fd" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:35.744614 19953 raft_consensus.cc:493] T d2ab0ac8347641e6955e9bce7d90c1a2 P cb82cc7bd6db46d980c75237a1402860 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:35.786425 16218 raft_consensus.cc:2468] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 10.
I20260709 13:20:35.667276 19955 consensus_queue.cc:260] T 8cc959fb7f704fba9009c0595467d074 P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.7, Last appended by leader: 7, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:35.799968 16822 raft_consensus.cc:1275] T 26354361e9b044cea1c2b86c4882fe92 P cb82cc7bd6db46d980c75237a1402860 [term 8 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:20:35.800863 16298 raft_consensus.cc:1275] T c4f65cdf00b748bb963ee6e81ef43b79 P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 13:20:35.748304 19863 raft_consensus.cc:2804] T 18ad7e3af5504571a69d10c00750a01c P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:20:35.811803 19863 raft_consensus.cc:493] T 18ad7e3af5504571a69d10c00750a01c P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Starting leader election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:20:35.746187 20038 raft_consensus.cc:3060] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:20:35.816430 16821 raft_consensus.cc:2468] T af57996a8be84d398f0f023e8d08ef77 P cb82cc7bd6db46d980c75237a1402860 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 11.
W20260709 13:20:35.816232 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 124450us
I20260709 13:20:35.886736 16823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e9b2bf7011f4ae688a79d7bb5a447b4" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:20:36.055663 16821 raft_consensus.cc:1240] T 2794af5b99984efebdee564035841cb0 P cb82cc7bd6db46d980c75237a1402860 [term 8 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 7. Current term is 8. Ops: []
I20260709 13:20:35.929306 19855 leader_election.cc:290] T 5a96373bb0e14dce8a266b2fc6c58507 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:35.721333 20041 consensus_queue.cc:1059] T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.094s
I20260709 13:20:35.801488 16293 raft_consensus.cc:2955] T ecc1e363230043fe9fef390f0b979f20 P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
W20260709 13:20:35.927840 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 227617us
W20260709 13:20:36.020010 16263 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:35.812371 19863 raft_consensus.cc:3060] T 18ad7e3af5504571a69d10c00750a01c P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:20:35.755046 19751 consensus_queue.cc:1059] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.070s
W20260709 13:20:36.038949 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 269662us
I20260709 13:20:36.245313 16265 leader_election.cc:304] T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:20:35.730523 20046 consensus_queue.cc:1059] T b6f07842d3aa47858df5665874c86f3a P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 2, Time since last communication: 0.171s
I20260709 13:20:36.046018 16822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3884f864a96d428785895e5463e5728f" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:35.822852 19966 leader_election.cc:290] T d16dfad0c3b540e5b4b86492ddf87208 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
W20260709 13:20:36.254385 20055 log.cc:927] Time spent T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 0.162s	user 0.000s	sys 0.000s
I20260709 13:20:36.103997 19904 consensus_queue.cc:1048] T b87147731a68400e8cce9b879daa7c5d P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.151s
I20260709 13:20:36.256731 19904 raft_consensus.cc:3055] T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:20:36.257303 19904 raft_consensus.cc:740] T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:20:36.221222 20057 raft_consensus.cc:493] T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:36.261890 20057 raft_consensus.cc:515] T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:36.264847 20057 leader_election.cc:290] T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:36.058928 19991 consensus_queue.cc:237] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:36.271628 20062 raft_consensus.cc:3060] T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:20:36.040208 20021 raft_consensus.cc:2804] T b87147731a68400e8cce9b879daa7c5d P 647a343a542d4a139fe106c19095129b [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:20:36.137173 16824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a378d7f796f94615b3b347119a9880fd" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 10 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:20:36.099767 16821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
W20260709 13:20:36.101040 20040 log.cc:927] Time spent T 23f82224f4564fea93572e8206bad1fe P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.363s	user 0.001s	sys 0.000s
I20260709 13:20:36.093751 16823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434bff3082d141838068f323bdb0d1ed" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:36.103965 20049 raft_consensus.cc:2749] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:20:36.071816 19977 leader_election.cc:290] T 5a959dbecc2648eb85d1cc550f5a3245 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
W20260709 13:20:36.090271 19802 consensus_meta.cc:287] T 57a744f5aaf944678a63469cbf514015 P cb82cc7bd6db46d980c75237a1402860: Time spent flushing consensus metadata: real 0.510s	user 0.009s	sys 0.000s
I20260709 13:20:36.100302 19667 raft_consensus.cc:3055] T b6f07842d3aa47858df5665874c86f3a P 1ca2725d290e47d68b1a4b609c9816ac [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:20:35.928568 19899 consensus_queue.cc:237] T 02838c86531d4f98bf58d1c22b8baf77 P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:20:36.097615 20036 log.cc:927] Time spent T 23f82224f4564fea93572e8206bad1fe P cb82cc7bd6db46d980c75237a1402860: Append to log took a long time: real 0.429s	user 0.000s	sys 0.001s
I20260709 13:20:36.053155 19898 raft_consensus.cc:3060] T ee9ed67006144797b1de6ea8c2013a68 P cb82cc7bd6db46d980c75237a1402860 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:20:36.112946 16189 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:20:36.101740 20037 log.cc:927] Time spent T e5ad4eb281564936a59f41e387a47156 P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.289s	user 0.000s	sys 0.000s
W20260709 13:20:36.066252 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 880902us
W20260709 13:20:36.077008 16264 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:20:36.102533 20051 log.cc:927] Time spent T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.062s	user 0.000s	sys 0.000s
I20260709 13:20:36.047585 20053 raft_consensus.cc:2749] T 02838c86531d4f98bf58d1c22b8baf77 P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:20:35.812893 19953 raft_consensus.cc:515] T d2ab0ac8347641e6955e9bce7d90c1a2 P cb82cc7bd6db46d980c75237a1402860 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:36.067394 19822 leader_election.cc:290] T c2f5f89186204242bf377f7c42b2e2da P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:36.103451 19823 leader_election.cc:290] T 66f75124215f4ca6bef831e174a792d8 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:36.050895 19942 raft_consensus.cc:493] T 9e65676b21b748728ad39668cfe722ec P cb82cc7bd6db46d980c75237a1402860 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
W20260709 13:20:36.070284 19915 consensus_meta.cc:287] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b: Time spent flushing consensus metadata: real 0.999s	user 0.008s	sys 0.004s
I20260709 13:20:36.065248 19699 leader_election.cc:290] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:36.044327 19952 leader_election.cc:290] T e4dc6b0541ca431ebc757beac00521f1 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:36.330614 20022 leader_election.cc:290] T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
W20260709 13:20:36.158391 16115 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.566s	user 0.063s	sys 0.098s
I20260709 13:20:36.093364 19861 leader_election.cc:290] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
W20260709 13:20:36.219326 20017 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.573s	user 0.004s	sys 0.008s
I20260709 13:20:35.749305 19568 consensus_queue.cc:1048] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.075s
I20260709 13:20:36.276772 16265 leader_election.cc:304] T ada5a004ba7c46819051e3ccec71ffdb P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: cb82cc7bd6db46d980c75237a1402860
W20260709 13:20:36.392541 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 115849us
I20260709 13:20:36.294862 16292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2794af5b99984efebdee564035841cb0" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:20:36.324117 16822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2372981006d4b218215c28964dea623" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:20:36.323983 19802 raft_consensus.cc:515] T 57a744f5aaf944678a63469cbf514015 P cb82cc7bd6db46d980c75237a1402860 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:36.325249 16190 leader_election.cc:400] T 02838c86531d4f98bf58d1c22b8baf77 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 9 pre-election: Vote denied by peer 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389) with higher term. Message: Invalid argument: T 02838c86531d4f98bf58d1c22b8baf77 P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac for earlier term 9. Current term is 10.
I20260709 13:20:36.396322 16190 leader_election.cc:403] T 02838c86531d4f98bf58d1c22b8baf77 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 9 pre-election: Cancelling election due to peer responding with higher term
W20260709 13:20:36.396898 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 102804us
I20260709 13:20:36.397583 16190 leader_election.cc:304] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:20:36.184522 19982 leader_election.cc:290] T 8a878999c0024d2abe517484180290fc P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
W20260709 13:20:36.328706 16192 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.599s	user 0.029s	sys 0.101s
W20260709 13:20:36.399145 16192 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.669s	user 0.029s	sys 0.101s
W20260709 13:20:36.399501 16192 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 669526us
I20260709 13:20:36.293612 19667 raft_consensus.cc:740] T b6f07842d3aa47858df5665874c86f3a P 1ca2725d290e47d68b1a4b609c9816ac [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:20:36.278555 20021 raft_consensus.cc:493] T b87147731a68400e8cce9b879daa7c5d P 647a343a542d4a139fe106c19095129b [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:20:36.339713 16789 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:36.331338 19751 consensus_queue.cc:1243] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Peer cb82cc7bd6db46d980c75237a1402860 log is divergent from this leader: its last log entry 8.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 13:20:36.324152 19990 leader_election.cc:290] T 829b4ad8c5a54eababeea1181b960d4b P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
W20260709 13:20:36.349020 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 287102us
I20260709 13:20:35.907998 19955 raft_consensus.cc:3060] T 8cc959fb7f704fba9009c0595467d074 P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:20:36.403846 19823 raft_consensus.cc:493] T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:36.381917 19953 leader_election.cc:290] T d2ab0ac8347641e6955e9bce7d90c1a2 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:36.385109 20064 raft_consensus.cc:493] T 434bff3082d141838068f323bdb0d1ed P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
W20260709 13:20:36.387369 16115 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.795s	user 0.063s	sys 0.098s
I20260709 13:20:36.430253 16144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806d7690c2b644959152074bb2cab0a7" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:36.342244 20046 raft_consensus.cc:493] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:20:36.350692 20020 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.948s	user 0.012s	sys 0.005s
W20260709 13:20:36.356611 20024 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.824s	user 0.004s	sys 0.012s
W20260709 13:20:36.453418 20024 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.924s	user 0.004s	sys 0.013s
I20260709 13:20:36.454713 20024 consensus_queue.cc:1059] T 2794af5b99984efebdee564035841cb0 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:20:36.455186 16817 raft_consensus.cc:2393] T 57a744f5aaf944678a63469cbf514015 P cb82cc7bd6db46d980c75237a1402860 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 9: Already voted for candidate cb82cc7bd6db46d980c75237a1402860 in this term.
I20260709 13:20:36.455785 20024 raft_consensus.cc:2764] T ada5a004ba7c46819051e3ccec71ffdb P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Leader pre-election decision vote started in defunct term 9: won
I20260709 13:20:36.456548 20071 raft_consensus.cc:493] T 19baa7bc367844c288bca8f81439c413 P cb82cc7bd6db46d980c75237a1402860 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:36.457115 20071 raft_consensus.cc:515] T 19baa7bc367844c288bca8f81439c413 P cb82cc7bd6db46d980c75237a1402860 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:36.458436 16817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66f75124215f4ca6bef831e174a792d8" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:36.459630 20074 consensus_queue.cc:1048] T 8c11074579b046d99e585c2818e18a2c P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:20:36.344672 19855 raft_consensus.cc:493] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:36.460606 19855 raft_consensus.cc:515] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:36.447818 20054 raft_consensus.cc:493] T fffab26427e24d398e2695aad40ca134 P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:36.462060 20054 raft_consensus.cc:515] T fffab26427e24d398e2695aad40ca134 P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:20:36.382809 19856 consensus_meta.cc:287] T 84f2da98864b4425bf7c0055951fdd4b P 1ca2725d290e47d68b1a4b609c9816ac: Time spent flushing consensus metadata: real 0.876s	user 0.004s	sys 0.004s
I20260709 13:20:36.351688 20053 raft_consensus.cc:493] T a378d7f796f94615b3b347119a9880fd P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:20:36.464418 20054 leader_election.cc:290] T fffab26427e24d398e2695aad40ca134 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:36.401034 16190 leader_election.cc:304] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860
I20260709 13:20:36.342109 19977 raft_consensus.cc:493] T 746aabf856ab401c9ef5e859371e7b67 P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:20:36.356881 19942 raft_consensus.cc:515] T 9e65676b21b748728ad39668cfe722ec P cb82cc7bd6db46d980c75237a1402860 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:20:36.391634 20017 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.745s	user 0.004s	sys 0.008s
I20260709 13:20:36.340691 20049 consensus_queue.cc:1059] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.063s
I20260709 13:20:36.415139 16822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada5a004ba7c46819051e3ccec71ffdb" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:20:36.486557 16822 raft_consensus.cc:3060] T ada5a004ba7c46819051e3ccec71ffdb P cb82cc7bd6db46d980c75237a1402860 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:20:36.413893 19699 raft_consensus.cc:2804] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:20:36.426795 16292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ab0ac8347641e6955e9bce7d90c1a2" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:36.412140 20065 raft_consensus.cc:493] T b87147731a68400e8cce9b879daa7c5d P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:36.429740 16143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434bff3082d141838068f323bdb0d1ed" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
W20260709 13:20:36.430742 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 96344.3us
W20260709 13:20:36.430683 16115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 839715us
I20260709 13:20:36.339426 19904 consensus_queue.cc:260] T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:20:36.432003 20020 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.046s	user 0.013s	sys 0.006s
I20260709 13:20:36.401993 19861 raft_consensus.cc:2749] T 02838c86531d4f98bf58d1c22b8baf77 P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Leader pre-election lost for term 9. Reason: Vote denied by peer 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389) with higher term. Message: Invalid argument: T 02838c86531d4f98bf58d1c22b8baf77 P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac for earlier term 9. Current term is 10.
I20260709 13:20:36.452742 16146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8158bbd17b994d2eb96f560bd0f1b88d" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
W20260709 13:20:36.466046 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 65101.1us
W20260709 13:20:36.491511 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 97597.4us
I20260709 13:20:36.491986 16190 leader_election.cc:304] T c0eab25ce4c64e018a041beeda2867db P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
W20260709 13:20:36.403483 16192 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:36.492373 16265 leader_election.cc:304] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:20:36.468428 16790 leader_election.cc:304] T 2794af5b99984efebdee564035841cb0 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860; no voters: 90f3da57796b488c9087a2f523adf045
I20260709 13:20:36.466646 16817 raft_consensus.cc:2468] T 66f75124215f4ca6bef831e174a792d8 P cb82cc7bd6db46d980c75237a1402860 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 9.
W20260709 13:20:36.471370 16789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 43634.5us
W20260709 13:20:36.474984 16117 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 45078.3us
W20260709 13:20:36.414310 19962 consensus_meta.cc:287] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860: Time spent flushing consensus metadata: real 0.896s	user 0.011s	sys 0.000s
I20260709 13:20:36.412737 19809 raft_consensus.cc:493] T 74ca8140f35c4ae29e8a664f6bdfc829 P cb82cc7bd6db46d980c75237a1402860 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:36.412500 20021 raft_consensus.cc:3060] T b87147731a68400e8cce9b879daa7c5d P 647a343a542d4a139fe106c19095129b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:20:36.475080 19856 raft_consensus.cc:2749] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:20:36.350358 19939 leader_election.cc:290] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:36.462972 19855 leader_election.cc:290] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:36.427063 19863 raft_consensus.cc:515] T 18ad7e3af5504571a69d10c00750a01c P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } }
I20260709 13:20:36.483327 20024 raft_consensus.cc:3055] T 2794af5b99984efebdee564035841cb0 P 90f3da57796b488c9087a2f523adf045 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:20:36.490406 20062 raft_consensus.cc:2749] T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:20:36.428092 19823 raft_consensus.cc:515] T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:36.464540 20053 raft_consensus.cc:515] T a378d7f796f94615b3b347119a9880fd P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:36.470832 19941 consensus_queue.cc:1059] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.038s
I20260709 13:20:36.537880 16190 leader_election.cc:304] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860
W20260709 13:20:36.538689 16115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 48787us
W20260709 13:20:36.527227 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 58842.9us
W20260709 13:20:36.538755 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 48593us
I20260709 13:20:36.541791 16144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "829b4ad8c5a54eababeea1181b960d4b" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:36.542425 16144 raft_consensus.cc:2468] T 829b4ad8c5a54eababeea1181b960d4b P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 9.
W20260709 13:20:36.524473 19992 consensus_meta.cc:287] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860: Time spent flushing consensus metadata: real 1.002s	user 0.005s	sys 0.009s
W20260709 13:20:36.545106 16117 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 51312.7us
I20260709 13:20:36.481487 19977 raft_consensus.cc:515] T 746aabf856ab401c9ef5e859371e7b67 P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:36.521023 16822 raft_consensus.cc:2468] T ada5a004ba7c46819051e3ccec71ffdb P cb82cc7bd6db46d980c75237a1402860 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 10.
I20260709 13:20:36.546777 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 11 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:36.539113 16190 leader_election.cc:304] T 68a6838d82ef4cac83507f906139d90f P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:20:36.547951 16817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62fe3346cd3c42c4a55e14af9073bca4" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
W20260709 13:20:36.526360 20016 consensus_meta.cc:287] T d2ab0ac8347641e6955e9bce7d90c1a2 P 90f3da57796b488c9087a2f523adf045: Time spent flushing consensus metadata: real 1.001s	user 0.007s	sys 0.006s
I20260709 13:20:36.525660 20024 raft_consensus.cc:740] T 2794af5b99984efebdee564035841cb0 P 90f3da57796b488c9087a2f523adf045 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:20:36.504463 20074 consensus_queue.cc:1048] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.056s
I20260709 13:20:36.562808 16817 raft_consensus.cc:2468] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 12.
I20260709 13:20:36.563529 16117 leader_election.cc:304] T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860
I20260709 13:20:36.563733 16822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9944d5e4b8c4109af9955c4f08c9095" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:20:36.588742 16822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57a744f5aaf944678a63469cbf514015" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:36.589570 16822 raft_consensus.cc:2368] T 57a744f5aaf944678a63469cbf514015 P cb82cc7bd6db46d980c75237a1402860 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 for earlier term 8. Current term is 9.
I20260709 13:20:36.582124 16150 raft_consensus.cc:2468] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 10.
W20260709 13:20:36.583793 16115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 44253.4us
I20260709 13:20:36.590986 16266 leader_election.cc:400] T 57a744f5aaf944678a63469cbf514015 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 8 pre-election: Vote denied by peer cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923) with higher term. Message: Invalid argument: T 57a744f5aaf944678a63469cbf514015 P cb82cc7bd6db46d980c75237a1402860 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 for earlier term 8. Current term is 9.
I20260709 13:20:36.591372 16266 leader_election.cc:403] T 57a744f5aaf944678a63469cbf514015 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 8 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:20:36.465633 20064 raft_consensus.cc:515] T 434bff3082d141838068f323bdb0d1ed P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:36.579329 19855 raft_consensus.cc:2749] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:20:36.585920 20044 raft_consensus.cc:2804] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Leader election won for term 9
W20260709 13:20:36.586592 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 45871.8us
I20260709 13:20:36.452957 20046 raft_consensus.cc:515] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:36.526122 20056 raft_consensus.cc:2749] T c0eab25ce4c64e018a041beeda2867db P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:20:36.583789 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 86510.7us
W20260709 13:20:36.593787 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 54705.3us
I20260709 13:20:36.630367 16116 leader_election.cc:304] T af57996a8be84d398f0f023e8d08ef77 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
W20260709 13:20:36.633186 16190 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:36.593828 16117 leader_election.cc:304] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860
I20260709 13:20:36.560863 16192 leader_election.cc:304] T 66f75124215f4ca6bef831e174a792d8 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:20:36.597254 16822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b7d1ab519044eda2d5e04a07242f25" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:20:36.577791 16790 leader_election.cc:304] T 829b4ad8c5a54eababeea1181b960d4b P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:20:36.580883 19861 raft_consensus.cc:493] T d2ab0ac8347641e6955e9bce7d90c1a2 P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:36.508816 19809 raft_consensus.cc:515] T 74ca8140f35c4ae29e8a664f6bdfc829 P cb82cc7bd6db46d980c75237a1402860 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:36.563269 19939 consensus_queue.cc:1059] T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 2, Time since last communication: 0.030s
W20260709 13:20:36.646360 16117 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 52626.2us
W20260709 13:20:36.641144 16192 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 80352.5us
I20260709 13:20:36.548944 19904 raft_consensus.cc:3060] T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:20:36.526535 19923 raft_consensus.cc:493] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:36.497788 20065 raft_consensus.cc:515] T b87147731a68400e8cce9b879daa7c5d P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:36.548198 20054 consensus_queue.cc:1048] T 68a6838d82ef4cac83507f906139d90f P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.056s
I20260709 13:20:36.522658 19667 consensus_queue.cc:260] T b6f07842d3aa47858df5665874c86f3a P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:36.680579 16817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee9ed67006144797b1de6ea8c2013a68" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:20:36.560165 19751 consensus_queue.cc:1059] T 1df93779209748b9860f6d893dd3de53 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.079s
I20260709 13:20:36.593263 20016 raft_consensus.cc:2749] T d2ab0ac8347641e6955e9bce7d90c1a2 P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:20:36.684222 16143 raft_consensus.cc:2468] T 434bff3082d141838068f323bdb0d1ed P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 9.
I20260709 13:20:36.600485 19822 raft_consensus.cc:2804] T 68a6838d82ef4cac83507f906139d90f P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:20:36.607601 19914 raft_consensus.cc:2749] T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:20:36.602675 20017 raft_consensus.cc:493] T 7c2095a5f4ad49408dbc7f69c606120d P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
W20260709 13:20:36.647581 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 69941.6us
I20260709 13:20:36.564214 19992 raft_consensus.cc:515] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:36.708305 16297 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40474 (request call id 590) took 2987 ms. Trace:
I20260709 13:20:36.721886 16297 rpcz_store.cc:276] 0709 13:20:33.721003 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:34.637333 (+916330us) service_pool.cc:224] Handling call
0709 13:20:34.948359 (+311026us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:35.082914 (+134555us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:36.708043 (+1625129us) spinlock_profiling.cc:243] Waited 134 ms on lock 0x7b540010f254. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:20:36.708241 (+   198us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":308176768}
I20260709 13:20:36.508333 19962 raft_consensus.cc:515] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:36.630095 20062 raft_consensus.cc:493] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
W20260709 13:20:36.684321 19197 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.538s	user 0.002s	sys 0.029s
I20260709 13:20:36.702843 19699 raft_consensus.cc:697] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac [term 10 LEADER]: Becoming Leader. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:20:36.640771 19855 raft_consensus.cc:493] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
W20260709 13:20:36.708214 20051 log.cc:927] Time spent T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
I20260709 13:20:36.677525 20068 raft_consensus.cc:2804] T af57996a8be84d398f0f023e8d08ef77 P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:20:36.753100 16292 raft_consensus.cc:2468] T d2ab0ac8347641e6955e9bce7d90c1a2 P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 9.
I20260709 13:20:36.676935 19861 raft_consensus.cc:515] T d2ab0ac8347641e6955e9bce7d90c1a2 P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:36.789472 16818 raft_consensus.cc:1240] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860 [term 12 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 11. Current term is 12. Ops: [7.8-11.10]
I20260709 13:20:36.717624 20056 raft_consensus.cc:2804] T 66f75124215f4ca6bef831e174a792d8 P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 13:20:36.759579 19197 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.607s	user 0.003s	sys 0.029s
I20260709 13:20:36.792326 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57a744f5aaf944678a63469cbf514015" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:20:36.793071 16144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e65676b21b748728ad39668cfe722ec" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:36.791828 19699 consensus_queue.cc:237] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:36.719340 19822 raft_consensus.cc:493] T 68a6838d82ef4cac83507f906139d90f P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:36.820004 16150 raft_consensus.cc:3060] T 57a744f5aaf944678a63469cbf514015 P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:20:36.646728 19972 consensus_queue.cc:1048] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.090s
I20260709 13:20:36.752758 19990 raft_consensus.cc:3055] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860 [term 7 LEADER]: Stepping down as leader of term 7
W20260709 13:20:36.803112 16264 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:36.706719 19923 raft_consensus.cc:515] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:20:36.752103 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 121999us
I20260709 13:20:36.808295 16192 leader_election.cc:304] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:20:36.828195 16265 leader_election.cc:304] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:20:36.724639 20024 consensus_queue.cc:260] T 2794af5b99984efebdee564035841cb0 P 90f3da57796b488c9087a2f523adf045 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:36.803795 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f5f89186204242bf377f7c42b2e2da" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:36.835371 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9944d5e4b8c4109af9955c4f08c9095" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
W20260709 13:20:36.816540 16117 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 138650us
I20260709 13:20:36.737799 20058 raft_consensus.cc:493] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:20:36.821297 16145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee9ed67006144797b1de6ea8c2013a68" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:20:36.836977 16145 raft_consensus.cc:3060] T ee9ed67006144797b1de6ea8c2013a68 P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:20:36.761021 19667 raft_consensus.cc:3060] T b6f07842d3aa47858df5665874c86f3a P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:20:36.743435 20066 raft_consensus.cc:493] T ecc1e363230043fe9fef390f0b979f20 P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:20:36.741160 19955 raft_consensus.cc:493] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:20:36.823254 16144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2372981006d4b218215c28964dea623" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:20:36.842795 19955 raft_consensus.cc:515] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:36.843608 16144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68a6838d82ef4cac83507f906139d90f" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:36.844892 16144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b7d1ab519044eda2d5e04a07242f25" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:20:36.845016 19955 leader_election.cc:290] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:36.845443 16144 raft_consensus.cc:3060] T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:20:36.845592 19955 raft_consensus.cc:2804] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:20:36.846020 19955 raft_consensus.cc:493] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Starting leader election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:20:36.846431 19955 raft_consensus.cc:3060] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:20:36.799494 16151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a878999c0024d2abe517484180290fc" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
W20260709 13:20:36.828701 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 145807us
I20260709 13:20:36.831238 16218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4f65cdf00b748bb963ee6e81ef43b79" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:36.848197 16221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0eab25ce4c64e018a041beeda2867db" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:36.833156 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
W20260709 13:20:36.833483 16265 connection.cc:256] RPC call timeout handler was delayed by -2.4698273619999327s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17290000000000003s for a response
I20260709 13:20:36.848845 16221 raft_consensus.cc:2468] T c0eab25ce4c64e018a041beeda2867db P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 9.
I20260709 13:20:36.833554 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d16dfad0c3b540e5b4b86492ddf87208" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
W20260709 13:20:36.849527 16265 connection.cc:256] RPC call timeout handler was delayed by -2.1924760449999212s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1715s for a response
I20260709 13:20:36.838346 16117 leader_election.cc:304] T 23f82224f4564fea93572e8206bad1fe P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860
I20260709 13:20:36.841061 16147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:36.796340 19751 raft_consensus.cc:493] T 8cc959fb7f704fba9009c0595467d074 P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:36.847955 16151 raft_consensus.cc:3060] T 8a878999c0024d2abe517484180290fc P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:20:36.866508 16223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a959dbecc2648eb85d1cc550f5a3245" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:36.859089 16217 raft_consensus.cc:2393] T ecc1e363230043fe9fef390f0b979f20 P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 9: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
I20260709 13:20:36.849786 16224 raft_consensus.cc:2410] T d16dfad0c3b540e5b4b86492ddf87208 P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 for term 10 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 13:20:36.867376 16223 raft_consensus.cc:2410] T 5a959dbecc2648eb85d1cc550f5a3245 P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 for term 8 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260709 13:20:36.803217 20016 raft_consensus.cc:3060] T 57a744f5aaf944678a63469cbf514015 P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Advancing to term 9
I20260709 13:20:36.863407 16117 leader_election.cc:304] T 829b4ad8c5a54eababeea1181b960d4b P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860
I20260709 13:20:36.864272 16116 leader_election.cc:304] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
I20260709 13:20:36.864272 16220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68a6838d82ef4cac83507f906139d90f" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:36.753022 19789 consensus_queue.cc:1048] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.081s
I20260709 13:20:36.814271 19802 leader_election.cc:290] T 57a744f5aaf944678a63469cbf514015 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 9 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:36.804118 20056 raft_consensus.cc:493] T 66f75124215f4ca6bef831e174a792d8 P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: Starting leader election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:20:36.814355 19575 raft_consensus.cc:493] T 84f2da98864b4425bf7c0055951fdd4b P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:36.819521 20068 raft_consensus.cc:697] T af57996a8be84d398f0f023e8d08ef77 P 647a343a542d4a139fe106c19095129b [term 11 LEADER]: Becoming Leader. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:20:36.822222 19822 raft_consensus.cc:3060] T 68a6838d82ef4cac83507f906139d90f P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:20:36.874316 16150 raft_consensus.cc:2468] T 57a744f5aaf944678a63469cbf514015 P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 9.
I20260709 13:20:36.882514 16151 raft_consensus.cc:2468] T 8a878999c0024d2abe517484180290fc P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 9.
I20260709 13:20:36.762041 20062 raft_consensus.cc:515] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:36.828328 19942 leader_election.cc:290] T 9e65676b21b748728ad39668cfe722ec P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:36.835323 20060 raft_consensus.cc:2804] T 2794af5b99984efebdee564035841cb0 P cb82cc7bd6db46d980c75237a1402860 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:20:36.826511 19990 raft_consensus.cc:740] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
W20260709 13:20:36.933231 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 69015.1us
W20260709 13:20:36.873816 20038 consensus_meta.cc:287] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860: Time spent flushing consensus metadata: real 0.807s	user 0.006s	sys 0.005s
I20260709 13:20:36.682513 20071 leader_election.cc:290] T 19baa7bc367844c288bca8f81439c413 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:36.934031 20038 raft_consensus.cc:2749] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 [term 8 FOLLOWER]: Leader election lost for term 7. Reason: Vote denied by peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943) with higher term. Message: Invalid argument: T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 for earlier term 7. Current term is 8.
W20260709 13:20:36.934803 20082 log.cc:927] Time spent T 87ac0d1f7e8a419f9bcbc04aee0e067e P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
I20260709 13:20:36.735652 20017 raft_consensus.cc:515] T 7c2095a5f4ad49408dbc7f69c606120d P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:36.927700 16145 raft_consensus.cc:2468] T ee9ed67006144797b1de6ea8c2013a68 P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 8.
I20260709 13:20:36.931936 16295 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40474 (request call id 598) took 2035 ms. Trace:
W20260709 13:20:36.932648 16117 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 69298.3us
I20260709 13:20:36.860812 20058 raft_consensus.cc:515] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:36.944862 16295 rpcz_store.cc:276] 0709 13:20:34.896129 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:35.169127 (+272998us) service_pool.cc:224] Handling call
0709 13:20:35.234884 (+ 65757us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:35.286963 (+ 52079us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:36.931710 (+1644747us) spinlock_profiling.cc:243] Waited 65.3 ms on lock 0x7b54001ee950. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5e8ad 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:20:36.931868 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":150262016}
I20260709 13:20:36.931376 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a96373bb0e14dce8a266b2fc6c58507" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 10 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:36.946201 16295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8638ee363273426db1393dd81e5c7b72" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:36.946353 16300 raft_consensus.cc:2468] T 5a96373bb0e14dce8a266b2fc6c58507 P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 9.
I20260709 13:20:36.946831 16295 raft_consensus.cc:2468] T 8638ee363273426db1393dd81e5c7b72 P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 10.
I20260709 13:20:36.947317 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4f65cdf00b748bb963ee6e81ef43b79" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 12 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:36.947840 16822 raft_consensus.cc:2468] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 8.
I20260709 13:20:36.947928 16295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18ad7e3af5504571a69d10c00750a01c" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
W20260709 13:20:36.936040 20085 log.cc:927] Time spent T 8d4ab870a05347868956aa8b841b2981 P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 0.050s	user 0.000s	sys 0.001s
I20260709 13:20:36.948609 16295 raft_consensus.cc:3060] T 18ad7e3af5504571a69d10c00750a01c P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:20:36.936128 16298 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.601s	user 0.003s	sys 0.011s
I20260709 13:20:36.936646 20057 raft_consensus.cc:2749] T 829b4ad8c5a54eababeea1181b960d4b P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:20:36.815469 19855 raft_consensus.cc:515] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:36.950266 16226 raft_consensus.cc:3060] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:20:36.950304 16298 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.615s	user 0.004s	sys 0.012s
I20260709 13:20:36.938673 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c74490c40243a88e62d1760b5cd814" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 10 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:36.890071 20024 raft_consensus.cc:3060] T 2794af5b99984efebdee564035841cb0 P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:20:36.916671 16299 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48674 (request call id 2516) took 3016 ms. Trace:
I20260709 13:20:36.951373 16150 raft_consensus.cc:2410] T 08c74490c40243a88e62d1760b5cd814 P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac for term 10 because replica has last-logged OpId of term: 7 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 5.
I20260709 13:20:36.954241 16297 raft_consensus.cc:2468] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 10.
I20260709 13:20:36.948158 16300 raft_consensus.cc:2410] T c4f65cdf00b748bb963ee6e81ef43b79 P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b for term 12 because replica has last-logged OpId of term: 9 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 6.
I20260709 13:20:36.949251 16190 leader_election.cc:304] T 5a96373bb0e14dce8a266b2fc6c58507 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:20:36.923045 16151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5718c623ebee4d2bb770f0f9ddb2f164" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 10 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:36.921716 20056 raft_consensus.cc:3060] T 66f75124215f4ca6bef831e174a792d8 P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:20:36.977420 16226 raft_consensus.cc:2468] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 10.
I20260709 13:20:36.954638 16299 rpcz_store.cc:276] 0709 13:20:33.899840 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:35.009942 (+1110102us) service_pool.cc:224] Handling call
0709 13:20:35.015653 (+  5711us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:20:35.018891 (+  3238us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:35.018925 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:20:35.029797 (+ 10872us) log.cc:844] Serialized 26 byte log entry
0709 13:20:35.166978 (+137181us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:20:35.167018 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:35.343454 (+176436us) spinlock_profiling.cc:243] Waited 1.84 ms on lock 0x7b4800130158. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b5a88aa1 00007f69b5a7fb37 00007f69b5a7f9ca 00007f69b5b7726f 00007f69b5b76f37 00007f69b5b8b55c 00007f69b5b8b508 00007f69b5b8b488 00007f69b5b8b443 00007f69b5b8a6ec 00007f69b4ab4a96
0709 13:20:36.916311 (+1572857us) spinlock_profiling.cc:243] Waited 1.84 ms on lock 0x7b540032fd54. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:20:36.916390 (+    79us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:20:36.916442 (+    52us) raft_consensus.cc:1685] finished
0709 13:20:36.916456 (+    14us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:36.916605 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1018,"raft.queue_time_us":1141,"raft.run_cpu_time_us":2988,"raft.run_wall_time_us":334578,"spinlock_wait_cycles":8456704,"thread_start_us":136881,"threads_started":1,"wal-append.queue_time_us":141796,"wal-append.run_cpu_time_us":15562,"wal-append.run_wall_time_us":1053538}
I20260709 13:20:36.980103 16151 raft_consensus.cc:2410] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac for term 10 because replica has last-logged OpId of term: 9 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 9.
I20260709 13:20:36.990285 16295 raft_consensus.cc:2468] T 18ad7e3af5504571a69d10c00750a01c P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 9.
I20260709 13:20:36.931352 19863 leader_election.cc:290] T 18ad7e3af5504571a69d10c00750a01c P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 9 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:36.875543 19751 raft_consensus.cc:515] T 8cc959fb7f704fba9009c0595467d074 P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:20:36.969712 16265 connection.cc:256] RPC call timeout handler was delayed by -1.173788297000101s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15000000000000002s for a response
I20260709 13:20:36.991885 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8158bbd17b994d2eb96f560bd0f1b88d" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:36.995394 16300 raft_consensus.cc:2468] T 8158bbd17b994d2eb96f560bd0f1b88d P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 8.
W20260709 13:20:36.995692 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 61139.8us
I20260709 13:20:36.979697 16190 leader_election.cc:304] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:20:36.998157 16190 leader_election.cc:304] T 18ad7e3af5504571a69d10c00750a01c P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:20:37.002031 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fffab26427e24d398e2695aad40ca134" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 9 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:37.002172 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9944d5e4b8c4109af9955c4f08c9095" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:36.992017 16295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a959dbecc2648eb85d1cc550f5a3245" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:37.003417 16225 raft_consensus.cc:1275] T d16dfad0c3b540e5b4b86492ddf87208 P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 9 index: 6. (term mismatch)
I20260709 13:20:37.003851 16225 pending_rounds.cc:85] T d16dfad0c3b540e5b4b86492ddf87208 P 1ca2725d290e47d68b1a4b609c9816ac: Aborting all ops after (but not including) 5
I20260709 13:20:37.004083 16225 pending_rounds.cc:107] T d16dfad0c3b540e5b4b86492ddf87208 P 1ca2725d290e47d68b1a4b609c9816ac: Aborting uncommitted NO_OP operation due to leader change: 8.6
I20260709 13:20:37.004338 16225 raft_consensus.cc:2889] T d16dfad0c3b540e5b4b86492ddf87208 P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:20:37.005085 16117 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 59765.2us
W20260709 13:20:36.994704 16265 connection.cc:256] RPC call timeout handler was delayed by -1.0909882969999671s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1592s for a response
I20260709 13:20:36.944290 20060 raft_consensus.cc:697] T 2794af5b99984efebdee564035841cb0 P cb82cc7bd6db46d980c75237a1402860 [term 8 LEADER]: Becoming Leader. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:20:36.998997 16190 leader_election.cc:304] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:20:37.008085 16300 raft_consensus.cc:2393] T c9944d5e4b8c4109af9955c4f08c9095 P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 10: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
I20260709 13:20:37.007772 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d16dfad0c3b540e5b4b86492ddf87208" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:36.938140 19575 raft_consensus.cc:515] T 84f2da98864b4425bf7c0055951fdd4b P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:36.976446 20073 raft_consensus.cc:2804] T 829b4ad8c5a54eababeea1181b960d4b P cb82cc7bd6db46d980c75237a1402860 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:20:37.009858 16295 raft_consensus.cc:2468] T 5a959dbecc2648eb85d1cc550f5a3245 P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 7.
I20260709 13:20:36.950927 20077 raft_consensus.cc:2749] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:20:36.874490 20066 raft_consensus.cc:515] T ecc1e363230043fe9fef390f0b979f20 P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:37.039026 16117 leader_election.cc:304] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860
I20260709 13:20:37.078894 19923 leader_election.cc:290] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:37.010910 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4f65cdf00b748bb963ee6e81ef43b79" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 12 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:37.068948 16299 raft_consensus.cc:2468] T d16dfad0c3b540e5b4b86492ddf87208 P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 9.
I20260709 13:20:37.073971 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a878999c0024d2abe517484180290fc" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:37.074609 20084 raft_consensus.cc:493] T 26354361e9b044cea1c2b86c4882fe92 P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:20:37.011570 20057 consensus_queue.cc:1048] T 26354361e9b044cea1c2b86c4882fe92 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.064s
I20260709 13:20:37.074610 16295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c61a39fd4f84424b129c1a6f9ba404d" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:36.976446 20053 leader_election.cc:290] T a378d7f796f94615b3b347119a9880fd P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:36.987004 19977 leader_election.cc:290] T 746aabf856ab401c9ef5e859371e7b67 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:36.976614 19823 leader_election.cc:290] T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:37.081327 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4dc6b0541ca431ebc757beac00521f1" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:37.081568 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87ac0d1f7e8a419f9bcbc04aee0e067e" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 11 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:37.080102 16147 raft_consensus.cc:2410] T ecc1e363230043fe9fef390f0b979f20 P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac for term 10 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 13:20:37.008924 19863 raft_consensus.cc:2804] T 18ad7e3af5504571a69d10c00750a01c P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:20:37.005676 19856 raft_consensus.cc:2804] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:20:37.082067 16117 leader_election.cc:304] T 3884f864a96d428785895e5463e5728f P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860
I20260709 13:20:37.084106 16295 raft_consensus.cc:2468] T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 9.
I20260709 13:20:37.073211 20073 raft_consensus.cc:493] T 829b4ad8c5a54eababeea1181b960d4b P cb82cc7bd6db46d980c75237a1402860 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:20:37.100282 20073 raft_consensus.cc:3060] T 829b4ad8c5a54eababeea1181b960d4b P cb82cc7bd6db46d980c75237a1402860 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:20:37.092806 16151 raft_consensus.cc:1240] T 84f2da98864b4425bf7c0055951fdd4b P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 7. Current term is 8. Ops: [7.4-7.4]
I20260709 13:20:37.095430 16300 raft_consensus.cc:2410] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac for term 11 because replica has last-logged OpId of term: 8 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 6.
I20260709 13:20:37.143054 20105 raft_consensus.cc:2749] T 3884f864a96d428785895e5463e5728f P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:20:37.094264 16299 raft_consensus.cc:2468] T e4dc6b0541ca431ebc757beac00521f1 P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 9.
I20260709 13:20:37.097461 16295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74ca8140f35c4ae29e8a664f6bdfc829" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
W20260709 13:20:37.090875 20078 log.cc:927] Time spent T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.374s	user 0.000s	sys 0.001s
I20260709 13:20:37.148098 16144 raft_consensus.cc:2468] T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 8.
I20260709 13:20:36.970217 20045 consensus_queue.cc:1048] T 68a6838d82ef4cac83507f906139d90f P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.374s
I20260709 13:20:37.099846 19856 raft_consensus.cc:493] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:37.155754 16144 raft_consensus.cc:1275] T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 13:20:37.139001 16219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fffab26427e24d398e2695aad40ca134" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 9 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:37.070353 19809 leader_election.cc:290] T 74ca8140f35c4ae29e8a664f6bdfc829 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:37.141415 19823 raft_consensus.cc:2804] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:20:37.090706 20077 raft_consensus.cc:2749] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:20:37.142613 20088 consensus_queue.cc:1048] T fffab26427e24d398e2695aad40ca134 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 2, Time since last communication: 0.054s
I20260709 13:20:37.012526 19990 consensus_queue.cc:260] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:37.143924 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 11 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
W20260709 13:20:37.152966 20076 log.cc:927] Time spent T 9e65676b21b748728ad39668cfe722ec P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.351s	user 0.000s	sys 0.001s
I20260709 13:20:37.080834 20087 raft_consensus.cc:3055] T 1df93779209748b9860f6d893dd3de53 P 1ca2725d290e47d68b1a4b609c9816ac [term 9 LEADER]: Stepping down as leader of term 9
W20260709 13:20:37.099422 20083 log.cc:927] Time spent T b6f07842d3aa47858df5665874c86f3a P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 0.360s	user 0.001s	sys 0.000s
I20260709 13:20:37.098150 19962 leader_election.cc:290] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 12 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
W20260709 13:20:37.136364 16117 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 54381.5us
I20260709 13:20:37.164160 16219 raft_consensus.cc:2410] T fffab26427e24d398e2695aad40ca134 P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b for term 9 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
I20260709 13:20:37.109931 20053 raft_consensus.cc:493] T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:20:36.950912 20067 consensus_queue.cc:1059] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.440s
I20260709 13:20:37.136634 20046 leader_election.cc:290] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:37.061342 20090 raft_consensus.cc:2804] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:20:37.136955 20084 raft_consensus.cc:515] T 26354361e9b044cea1c2b86c4882fe92 P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:37.075711 20060 consensus_queue.cc:237] T 2794af5b99984efebdee564035841cb0 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:37.179471 16117 leader_election.cc:304] T 57a744f5aaf944678a63469cbf514015 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860
I20260709 13:20:37.163966 19856 raft_consensus.cc:3060] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:20:37.175884 20087 raft_consensus.cc:740] T 1df93779209748b9860f6d893dd3de53 P 1ca2725d290e47d68b1a4b609c9816ac [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:20:37.168944 19823 raft_consensus.cc:493] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Starting leader election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:20:37.197647 16300 raft_consensus.cc:1240] T 18ad7e3af5504571a69d10c00750a01c P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 8. Current term is 9. Ops: []
I20260709 13:20:37.203437 20017 leader_election.cc:290] T 7c2095a5f4ad49408dbc7f69c606120d P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:20:37.204272 20017 consensus_queue.cc:1048] T fffab26427e24d398e2695aad40ca134 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:20:37.206060 20017 raft_consensus.cc:493] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:37.207901 16219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a878999c0024d2abe517484180290fc" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:37.208940 16219 raft_consensus.cc:2393] T 8a878999c0024d2abe517484180290fc P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 9: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
I20260709 13:20:37.139689 19863 raft_consensus.cc:697] T 18ad7e3af5504571a69d10c00750a01c P 1ca2725d290e47d68b1a4b609c9816ac [term 9 LEADER]: Becoming Leader. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
W20260709 13:20:37.196023 20081 log.cc:927] Time spent T ecc1e363230043fe9fef390f0b979f20 P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.216s	user 0.000s	sys 0.000s
I20260709 13:20:37.076363 20068 consensus_queue.cc:237] T af57996a8be84d398f0f023e8d08ef77 P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:37.182653 20065 leader_election.cc:290] T b87147731a68400e8cce9b879daa7c5d P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:37.213522 16219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b7d1ab519044eda2d5e04a07242f25" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 8 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:37.221972 16300 raft_consensus.cc:1275] T 8638ee363273426db1393dd81e5c7b72 P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:20:37.195794 19569 raft_consensus.cc:2749] T 57a744f5aaf944678a63469cbf514015 P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:20:37.216980 19953 raft_consensus.cc:493] T 26354361e9b044cea1c2b86c4882fe92 P cb82cc7bd6db46d980c75237a1402860 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:37.222131 16219 raft_consensus.cc:2393] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 8: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
I20260709 13:20:37.188537 20053 raft_consensus.cc:515] T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:37.208873 19823 raft_consensus.cc:3060] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:20:37.233137 16295 raft_consensus.cc:2468] T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 8.
I20260709 13:20:37.209293 20090 raft_consensus.cc:493] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:37.224340 19953 raft_consensus.cc:515] T 26354361e9b044cea1c2b86c4882fe92 P cb82cc7bd6db46d980c75237a1402860 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:37.184351 20096 raft_consensus.cc:2804] T 5a96373bb0e14dce8a266b2fc6c58507 P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:20:37.230638 20053 leader_election.cc:290] T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:37.223682 19863 consensus_queue.cc:237] T 18ad7e3af5504571a69d10c00750a01c P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } }
I20260709 13:20:37.206619 20017 raft_consensus.cc:515] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:37.211112 20105 raft_consensus.cc:493] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:37.236527 19938 raft_consensus.cc:493] T 8cc959fb7f704fba9009c0595467d074 P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:37.242429 20096 raft_consensus.cc:493] T 5a96373bb0e14dce8a266b2fc6c58507 P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: Starting leader election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:20:37.181571 20093 raft_consensus.cc:2804] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 13:20:37.243518 20035 log.cc:927] Time spent T 829b4ad8c5a54eababeea1181b960d4b P cb82cc7bd6db46d980c75237a1402860: Append to log took a long time: real 1.582s	user 0.004s	sys 0.000s
I20260709 13:20:37.243880 19953 leader_election.cc:290] T 26354361e9b044cea1c2b86c4882fe92 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:37.260249 19822 raft_consensus.cc:515] T 68a6838d82ef4cac83507f906139d90f P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:20:37.263743 16264 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:37.150539 20095 consensus_queue.cc:1048] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.352s
I20260709 13:20:37.264307 20105 raft_consensus.cc:515] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:37.276983 16790 leader_election.cc:304] T 57a744f5aaf944678a63469cbf514015 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:20:37.265940 19938 raft_consensus.cc:515] T 8cc959fb7f704fba9009c0595467d074 P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:37.265949 19751 leader_election.cc:290] T 8cc959fb7f704fba9009c0595467d074 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:37.272408 20093 raft_consensus.cc:493] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [term 10 FOLLOWER]: Starting leader election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:20:37.272641 20096 raft_consensus.cc:3060] T 5a96373bb0e14dce8a266b2fc6c58507 P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:20:37.275070 19966 raft_consensus.cc:3055] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [term 9 LEADER]: Stepping down as leader of term 9
W20260709 13:20:37.275333 20101 log.cc:927] Time spent T 3a7ca61592e848ea9b0fb81a2ce33cdc P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.066s	user 0.000s	sys 0.000s
I20260709 13:20:37.223625 20087 consensus_queue.cc:260] T 1df93779209748b9860f6d893dd3de53 P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:37.235386 20090 raft_consensus.cc:3060] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:20:37.238142 20062 leader_election.cc:290] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:20:37.283347 16220 raft_consensus.cc:2393] T 68a6838d82ef4cac83507f906139d90f P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 11: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
I20260709 13:20:37.281037 19966 raft_consensus.cc:740] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:20:37.279632 20093 raft_consensus.cc:3060] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:20:37.328858 19938 leader_election.cc:290] T 8cc959fb7f704fba9009c0595467d074 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:20:37.290305 19966 consensus_queue.cc:260] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:37.331310 19966 raft_consensus.cc:3060] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:20:37.333144 19822 leader_election.cc:290] T 68a6838d82ef4cac83507f906139d90f P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 11 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
W20260709 13:20:37.336149 19898 consensus_meta.cc:287] T ee9ed67006144797b1de6ea8c2013a68 P cb82cc7bd6db46d980c75237a1402860: Time spent flushing consensus metadata: real 1.000s	user 0.009s	sys 0.003s
I20260709 13:20:37.336748 19898 raft_consensus.cc:2749] T ee9ed67006144797b1de6ea8c2013a68 P cb82cc7bd6db46d980c75237a1402860 [term 8 FOLLOWER]: Leader election lost for term 7. Reason: Vote denied by peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943) with higher term. Message: Invalid argument: T ee9ed67006144797b1de6ea8c2013a68 P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 for earlier term 7. Current term is 8.
I20260709 13:20:37.322599 16038 maintenance_manager.cc:419] P 667a34b27a39474ebefc1af3cf5b4271: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.033386
I20260709 13:20:37.322391 20087 raft_consensus.cc:3060] T 1df93779209748b9860f6d893dd3de53 P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:20:37.339241 19938 raft_consensus.cc:493] T 8d4ab870a05347868956aa8b841b2981 P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:37.345222 16820 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49906 (request call id 798) took 3436 ms. Trace:
I20260709 13:20:37.359936 16820 rpcz_store.cc:276] 0709 13:20:33.908479 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:33.931278 (+ 22799us) service_pool.cc:224] Handling call
0709 13:20:33.931620 (+   342us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:20:33.931970 (+   350us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:33.932000 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:20:35.225679 (+1293679us) spinlock_profiling.cc:243] Waited 6.42 ms on lock 0x7b2c00047148. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69af458a11 00007f69af468372 00007f69b0b9aa01 00007f69b0b5f43a 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2
0709 13:20:35.226326 (+   647us) log.cc:844] Serialized 26 byte log entry
0709 13:20:35.288545 (+ 62219us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:20:35.288589 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:35.288663 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:20:37.100311 (+1811648us) spinlock_profiling.cc:243] Waited 59 ms on lock 0x7b4400022b4c. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69ad8fadb1 00007f69ad998c85 00007f69ad9996a2 00007f69b070b1db 00007f69b0706aaf 00007f69b06fa86e 00007f69b06f9bb6 00007f69b06f97d8 00007f69b0703221 00007f69b07031d9 00007f69b0703169
0709 13:20:37.344997 (+244686us) raft_consensus.cc:1685] finished
0709 13:20:37.345017 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:37.345159 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2217,"raft.run_cpu_time_us":87,"raft.run_wall_time_us":87,"spinlock_wait_cycles":150572672,"thread_start_us":61754,"threads_started":1,"wal-append.queue_time_us":116166}
W20260709 13:20:37.367486 16142 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.503s	user 0.086s	sys 0.070s
W20260709 13:20:37.368292 16142 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.504s	user 0.086s	sys 0.070s
I20260709 13:20:37.368326 16298 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40474 (request call id 607) took 1768 ms. Trace:
I20260709 13:20:37.369472 16817 raft_consensus.cc:2468] T ee9ed67006144797b1de6ea8c2013a68 P cb82cc7bd6db46d980c75237a1402860 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 8.
I20260709 13:20:37.368878 16298 rpcz_store.cc:276] 0709 13:20:35.600295 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:35.802861 (+202566us) service_pool.cc:224] Handling call
0709 13:20:35.803198 (+   337us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:20:36.156356 (+353158us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:36.156405 (+    49us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:20:36.211437 (+ 55032us) log.cc:844] Serialized 52 byte log entry
0709 13:20:36.951586 (+740149us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:20:36.951729 (+   143us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:36.953866 (+  2137us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:20:37.368103 (+414237us) raft_consensus.cc:1685] finished
0709 13:20:37.368125 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:37.368270 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":380346,"raft.run_cpu_time_us":160,"raft.run_wall_time_us":39802,"thread_start_us":949006,"threads_started":2,"wal-append.queue_time_us":777436}
I20260709 13:20:37.360905 19938 raft_consensus.cc:515] T 8d4ab870a05347868956aa8b841b2981 P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:37.361244 20017 leader_election.cc:290] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:20:37.376310 19938 leader_election.cc:290] T 8d4ab870a05347868956aa8b841b2981 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:20:37.380834 16820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8638ee363273426db1393dd81e5c7b72" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:37.388242 16820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:37.389637 16820 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49906 (request call id 807) took 1251 ms. Trace:
I20260709 13:20:37.390046 16820 rpcz_store.cc:276] 0709 13:20:36.138491 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:37.387949 (+1249458us) service_pool.cc:224] Handling call
0709 13:20:37.389581 (+  1632us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:20:37.391026 16820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5718c623ebee4d2bb770f0f9ddb2f164" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 10 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:37.391770 16820 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49906 (request call id 811) took 1193 ms. Trace:
I20260709 13:20:37.392061 16820 rpcz_store.cc:276] 0709 13:20:36.198763 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:37.390768 (+1192005us) service_pool.cc:224] Handling call
0709 13:20:37.391697 (+   929us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:20:37.392879 16820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74ca8140f35c4ae29e8a664f6bdfc829" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:37.396234 16820 raft_consensus.cc:2468] T 74ca8140f35c4ae29e8a664f6bdfc829 P cb82cc7bd6db46d980c75237a1402860 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 9.
I20260709 13:20:37.381882 20048 raft_consensus.cc:2804] T 57a744f5aaf944678a63469cbf514015 P cb82cc7bd6db46d980c75237a1402860 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:20:37.384732 20105 leader_election.cc:290] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:37.416141 16142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d16dfad0c3b540e5b4b86492ddf87208" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:37.434628 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a959dbecc2648eb85d1cc550f5a3245" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:37.444500 16790 leader_election.cc:304] T 8c11074579b046d99e585c2818e18a2c P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 90f3da57796b488c9087a2f523adf045
I20260709 13:20:37.446159 16790 leader_election.cc:304] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:20:37.448637 19966 raft_consensus.cc:493] T 7c2095a5f4ad49408dbc7f69c606120d P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
W20260709 13:20:37.479772 16191 connection.cc:256] RPC call timeout handler was delayed by -1.2591759740000725s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21520000000000003s for a response
I20260709 13:20:37.422078 20048 raft_consensus.cc:697] T 57a744f5aaf944678a63469cbf514015 P cb82cc7bd6db46d980c75237a1402860 [term 9 LEADER]: Becoming Leader. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
W20260709 13:20:37.492723 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 46678.9us
W20260709 13:20:37.493273 16294 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.680s	user 0.002s	sys 0.023s
W20260709 13:20:37.494160 16294 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.681s	user 0.002s	sys 0.023s
I20260709 13:20:37.494731 16191 leader_election.cc:400] T 02838c86531d4f98bf58d1c22b8baf77 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 9 pre-election: Vote denied by peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741) with higher term. Message: Invalid argument: T 02838c86531d4f98bf58d1c22b8baf77 P 647a343a542d4a139fe106c19095129b [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac for earlier term 9. Current term is 10.
I20260709 13:20:37.497519 16191 leader_election.cc:304] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
I20260709 13:20:37.409932 20073 raft_consensus.cc:515] T 829b4ad8c5a54eababeea1181b960d4b P cb82cc7bd6db46d980c75237a1402860 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:37.449256 19966 raft_consensus.cc:515] T 7c2095a5f4ad49408dbc7f69c606120d P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:37.428828 19806 raft_consensus.cc:3060] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:20:37.435269 20094 raft_consensus.cc:2749] T 23f82224f4564fea93572e8206bad1fe P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:20:37.520541 16790 leader_election.cc:304] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 90f3da57796b488c9087a2f523adf045
I20260709 13:20:37.491494 19667 raft_consensus.cc:493] T 19baa7bc367844c288bca8f81439c413 P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:37.431142 19856 raft_consensus.cc:515] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:20:37.533900 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 534939us
W20260709 13:20:37.678663 16190 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:20:37.535400 16222 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.670s	user 0.031s	sys 0.075s
W20260709 13:20:37.679358 16222 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.814s	user 0.031s	sys 0.075s
I20260709 13:20:37.527230 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68a6838d82ef4cac83507f906139d90f" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
W20260709 13:20:37.649030 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 151581us
I20260709 13:20:37.500903 19673 raft_consensus.cc:2749] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:20:37.530284 16144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5ad4eb281564936a59f41e387a47156" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:37.690876 16144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4f65cdf00b748bb963ee6e81ef43b79" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:37.691669 16144 raft_consensus.cc:2468] T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 11.
I20260709 13:20:37.499936 19822 raft_consensus.cc:493] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
W20260709 13:20:37.659449 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 139051us
W20260709 13:20:37.693579 16190 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:37.703189 16190 leader_election.cc:304] T 5a959dbecc2648eb85d1cc550f5a3245 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
W20260709 13:20:37.644503 20049 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.838s	user 0.008s	sys 0.018s
I20260709 13:20:37.707345 16190 leader_election.cc:304] T d16dfad0c3b540e5b4b86492ddf87208 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:20:37.659260 19751 raft_consensus.cc:2749] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:20:37.713550 16190 leader_election.cc:304] T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:20:37.717576 16144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62fe3346cd3c42c4a55e14af9073bca4" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:37.719201 16144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0eab25ce4c64e018a041beeda2867db" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:37.697292 16191 leader_election.cc:304] T 8a878999c0024d2abe517484180290fc P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 90f3da57796b488c9087a2f523adf045
I20260709 13:20:37.676774 19789 raft_consensus.cc:493] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
W20260709 13:20:37.740033 16293 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.003s	user 0.012s	sys 0.012s
W20260709 13:20:37.752442 16293 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.016s	user 0.013s	sys 0.012s
I20260709 13:20:37.753930 16293 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40474 (request call id 606) took 2154 ms. Trace:
W20260709 13:20:37.740545 16192 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.730s	user 0.013s	sys 0.029s
W20260709 13:20:37.755481 16192 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.745s	user 0.013s	sys 0.030s
W20260709 13:20:37.755937 16192 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 746077us
W20260709 13:20:37.741061 16265 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.689s	user 0.014s	sys 0.009s
I20260709 13:20:37.698011 19923 raft_consensus.cc:2804] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P cb82cc7bd6db46d980c75237a1402860 [term 10 FOLLOWER]: Leader election won for term 10
W20260709 13:20:37.757153 16265 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.705s	user 0.014s	sys 0.009s
I20260709 13:20:37.765091 20067 raft_consensus.cc:2804] T 5a959dbecc2648eb85d1cc550f5a3245 P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:20:37.698141 20095 raft_consensus.cc:493] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
W20260709 13:20:37.783071 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 731431us
I20260709 13:20:37.784581 16144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a96373bb0e14dce8a266b2fc6c58507" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 10 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:37.638119 19672 consensus_queue.cc:1048] T 3a7ca61592e848ea9b0fb81a2ce33cdc P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.180s
W20260709 13:20:37.712917 20049 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.997s	user 0.009s	sys 0.018s
W20260709 13:20:37.792452 19699 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.500s	user 0.013s	sys 0.025s
I20260709 13:20:37.794875 19702 leader_election.cc:290] T e5ad4eb281564936a59f41e387a47156 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
W20260709 13:20:37.808697 16264 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:37.824534 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f5f89186204242bf377f7c42b2e2da" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:37.687860 19667 raft_consensus.cc:515] T 19baa7bc367844c288bca8f81439c413 P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:20:37.831619 20099 log.cc:927] Time spent T 26354361e9b044cea1c2b86c4882fe92 P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.078s	user 0.001s	sys 0.001s
I20260709 13:20:37.706777 19822 raft_consensus.cc:515] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:37.816000 19789 raft_consensus.cc:515] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:20:37.748246 20044 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.041s	user 0.001s	sys 0.051s
I20260709 13:20:37.843451 16265 leader_election.cc:304] T 1df93779209748b9860f6d893dd3de53 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac
I20260709 13:20:37.754357 16293 rpcz_store.cc:276] 0709 13:20:35.599306 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:35.799322 (+200016us) service_pool.cc:224] Handling call
0709 13:20:35.800947 (+  1625us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:35.801321 (+   374us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:20:36.282163 (+480842us) log.cc:844] Serialized 14 byte log entry
0709 13:20:36.736534 (+454371us) log.cc:844] Serialized 14 byte log entry
0709 13:20:37.753283 (+1016749us) log.cc:844] Serialized 14 byte log entry
0709 13:20:37.753541 (+   258us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:20:37.753593 (+    52us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:20:37.753651 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:37.753756 (+   105us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:37.753878 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":15437,"prepare.queue_time_us":66158,"prepare.run_cpu_time_us":109,"prepare.run_wall_time_us":113,"raft.queue_time_us":378806,"raft.run_cpu_time_us":173,"raft.run_wall_time_us":76340,"spinlock_wait_cycles":54417536,"thread_start_us":1417882,"threads_started":3,"wal-append.queue_time_us":396903}
W20260709 13:20:37.817613 20016 consensus_meta.cc:287] T 57a744f5aaf944678a63469cbf514015 P 90f3da57796b488c9087a2f523adf045: Time spent flushing consensus metadata: real 0.932s	user 0.006s	sys 0.006s
I20260709 13:20:38.043021 20016 raft_consensus.cc:2749] T 57a744f5aaf944678a63469cbf514015 P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Leader pre-election lost for term 8. Reason: Vote denied by peer cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923) with higher term. Message: Invalid argument: T 57a744f5aaf944678a63469cbf514015 P cb82cc7bd6db46d980c75237a1402860 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 for earlier term 8. Current term is 9.
I20260709 13:20:37.852418 16144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18ad7e3af5504571a69d10c00750a01c" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:20:37.818014 20067 raft_consensus.cc:493] T 5a959dbecc2648eb85d1cc550f5a3245 P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Starting leader election (detected failure of leader 647a343a542d4a139fe106c19095129b)
W20260709 13:20:37.855377 19699 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.570s	user 0.013s	sys 0.025s
W20260709 13:20:37.864858 19823 consensus_meta.cc:287] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac: Time spent flushing consensus metadata: real 0.627s	user 0.011s	sys 0.000s
I20260709 13:20:37.841181 19751 raft_consensus.cc:2804] T d16dfad0c3b540e5b4b86492ddf87208 P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:20:37.838083 19923 raft_consensus.cc:697] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P cb82cc7bd6db46d980c75237a1402860 [term 10 LEADER]: Becoming Leader. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:20:38.068560 19923 consensus_queue.cc:237] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:38.057693 20067 raft_consensus.cc:3060] T 5a959dbecc2648eb85d1cc550f5a3245 P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:20:37.918422 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cc959fb7f704fba9009c0595467d074" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:38.076186 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:38.077666 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af57996a8be84d398f0f023e8d08ef77" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:37.875972 19702 raft_consensus.cc:2804] T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 13:20:37.915077 20081 log.cc:927] Time spent T ecc1e363230043fe9fef390f0b979f20 P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.064s	user 0.000s	sys 0.000s
W20260709 13:20:37.898103 20044 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.217s	user 0.001s	sys 0.052s
W20260709 13:20:37.895119 20089 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.615s	user 0.007s	sys 0.013s
W20260709 13:20:37.903508 19863 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.575s	user 0.009s	sys 0.020s
W20260709 13:20:37.958984 19345 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.522s	user 0.011s	sys 0.020s
W20260709 13:20:37.968634 20060 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.736s	user 0.014s	sys 0.018s
I20260709 13:20:37.842612 20095 raft_consensus.cc:515] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:20:37.990660 20100 log.cc:927] Time spent T 23f82224f4564fea93572e8206bad1fe P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 13:20:37.990505 19966 leader_election.cc:290] T 7c2095a5f4ad49408dbc7f69c606120d P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:37.816907 20048 consensus_queue.cc:237] T 57a744f5aaf944678a63469cbf514015 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:20:38.090746 16300 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.805s	user 0.009s	sys 0.018s
W20260709 13:20:38.016489 16116 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.961s	user 0.013s	sys 0.037s
W20260709 13:20:38.091883 16116 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.036s	user 0.013s	sys 0.037s
I20260709 13:20:38.024194 16291 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48674 (request call id 2535) took 1391 ms. Trace:
I20260709 13:20:38.092828 16291 rpcz_store.cc:276] 0709 13:20:36.633381 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:36.633531 (+   150us) service_pool.cc:224] Handling call
0709 13:20:36.633900 (+   369us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:20:36.778100 (+144200us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:36.778155 (+    55us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:20:36.790442 (+ 12287us) log.cc:844] Serialized 26 byte log entry
0709 13:20:37.063587 (+273145us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:20:37.063638 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:37.063726 (+    88us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:20:38.023842 (+960116us) raft_consensus.cc:1685] finished
0709 13:20:38.023861 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:38.024116 (+   255us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":249397,"raft.run_cpu_time_us":182,"raft.run_wall_time_us":52185,"thread_start_us":341126,"threads_started":2,"wal-append.queue_time_us":510716}
I20260709 13:20:37.994439 20109 raft_consensus.cc:2804] T 8a878999c0024d2abe517484180290fc P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: Leader election won for term 9
W20260709 13:20:38.085000 19863 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.756s	user 0.009s	sys 0.023s
W20260709 13:20:38.088071 20060 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.855s	user 0.015s	sys 0.018s
W20260709 13:20:38.051694 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 208347us
W20260709 13:20:38.054495 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 357314us
I20260709 13:20:38.110359 16191 leader_election.cc:304] T 08c74490c40243a88e62d1760b5cd814 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:20:38.056789 16144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66f75124215f4ca6bef831e174a792d8" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
W20260709 13:20:38.041206 20075 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.645s	user 0.006s	sys 0.016s
I20260709 13:20:38.080435 20044 raft_consensus.cc:697] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045 [term 9 LEADER]: Becoming Leader. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:20:38.067260 19751 raft_consensus.cc:493] T d16dfad0c3b540e5b4b86492ddf87208 P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:20:38.046262 19955 consensus_meta.cc:287] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac: Time spent flushing consensus metadata: real 1.199s	user 0.014s	sys 0.000s
I20260709 13:20:38.079135 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8158bbd17b994d2eb96f560bd0f1b88d" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
W20260709 13:20:38.052191 19760 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.561s	user 0.016s	sys 0.004s
I20260709 13:20:38.029942 19856 leader_election.cc:290] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 11 election: Requested vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
W20260709 13:20:38.088503 16266 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.573s	user 0.039s	sys 0.026s
W20260709 13:20:38.159822 16266 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.644s	user 0.040s	sys 0.026s
W20260709 13:20:38.160203 16266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 645205us
W20260709 13:20:38.084996 19345 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.645s	user 0.011s	sys 0.020s
I20260709 13:20:37.873318 19938 consensus_queue.cc:1059] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.086s
W20260709 13:20:38.040881 19974 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.958s	user 0.003s	sys 0.019s
W20260709 13:20:38.080612 20089 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.841s	user 0.008s	sys 0.013s
I20260709 13:20:38.050537 20073 leader_election.cc:290] T 829b4ad8c5a54eababeea1181b960d4b P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 10 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
W20260709 13:20:38.029384 20024 consensus_meta.cc:287] T 2794af5b99984efebdee564035841cb0 P 90f3da57796b488c9087a2f523adf045: Time spent flushing consensus metadata: real 1.052s	user 0.006s	sys 0.006s
I20260709 13:20:38.079505 19702 raft_consensus.cc:493] T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:20:38.041380 16298 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.664s	user 0.001s	sys 0.019s
W20260709 13:20:38.101715 16300 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.816s	user 0.010s	sys 0.018s
I20260709 13:20:38.055836 20114 raft_consensus.cc:2764] T 1df93779209748b9860f6d893dd3de53 P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Leader pre-election decision vote started in defunct term 9: won
I20260709 13:20:38.107143 20105 raft_consensus.cc:493] T 9c61a39fd4f84424b129c1a6f9ba404d P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:20:38.095917 20096 consensus_meta.cc:287] T 5a96373bb0e14dce8a266b2fc6c58507 P 1ca2725d290e47d68b1a4b609c9816ac: Time spent flushing consensus metadata: real 0.811s	user 0.017s	sys 0.000s
W20260709 13:20:38.058214 20014 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 2.550s	user 0.022s	sys 0.044s
W20260709 13:20:38.208019 20021 consensus_meta.cc:287] T b87147731a68400e8cce9b879daa7c5d P 647a343a542d4a139fe106c19095129b: Time spent flushing consensus metadata: real 1.682s	user 0.010s	sys 0.005s
I20260709 13:20:38.090227 19699 raft_consensus.cc:493] T 84f2da98864b4425bf7c0055951fdd4b P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:20:38.151816 16218 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.013s	user 0.011s	sys 0.045s
W20260709 13:20:38.224642 16218 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.086s	user 0.012s	sys 0.045s
I20260709 13:20:38.152654 16191 leader_election.cc:304] T ee9ed67006144797b1de6ea8c2013a68 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
W20260709 13:20:38.225515 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 72944.5us
I20260709 13:20:38.226382 16191 leader_election.cc:304] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
I20260709 13:20:38.151201 16265 leader_election.cc:304] T 8158bbd17b994d2eb96f560bd0f1b88d P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:20:38.161473 16144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d16dfad0c3b540e5b4b86492ddf87208" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:38.166100 16291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ad5dbfa99ee4b8d9e28be92b3f36db5" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:38.166612 16296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68a6838d82ef4cac83507f906139d90f" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:20:38.227800 16296 raft_consensus.cc:3060] T 68a6838d82ef4cac83507f906139d90f P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:20:38.228142 16144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a959dbecc2648eb85d1cc550f5a3245" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:38.166708 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cc959fb7f704fba9009c0595467d074" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:38.229031 16293 raft_consensus.cc:2468] T 8cc959fb7f704fba9009c0595467d074 P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 10.
W20260709 13:20:38.116132 20056 consensus_meta.cc:287] T 66f75124215f4ca6bef831e174a792d8 P 1ca2725d290e47d68b1a4b609c9816ac: Time spent flushing consensus metadata: real 1.111s	user 0.009s	sys 0.004s
W20260709 13:20:38.220652 20122 log.cc:927] Time spent T c4f65cdf00b748bb963ee6e81ef43b79 P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.160s	user 0.000s	sys 0.001s
W20260709 13:20:38.169086 16294 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.673s	user 0.010s	sys 0.014s
W20260709 13:20:38.232376 16294 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.736s	user 0.010s	sys 0.014s
I20260709 13:20:38.147979 20109 raft_consensus.cc:697] T 8a878999c0024d2abe517484180290fc P 1ca2725d290e47d68b1a4b609c9816ac [term 9 LEADER]: Becoming Leader. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
W20260709 13:20:38.181002 16298 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.804s	user 0.002s	sys 0.019s
W20260709 13:20:38.163828 19760 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.708s	user 0.017s	sys 0.004s
I20260709 13:20:38.181845 16300 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48684 (request call id 2385) took 1277 ms. Trace:
W20260709 13:20:38.222862 20014 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 2.717s	user 0.022s	sys 0.044s
I20260709 13:20:38.154129 19751 raft_consensus.cc:3060] T d16dfad0c3b540e5b4b86492ddf87208 P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:20:38.238322 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 87252.4us
I20260709 13:20:38.187603 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c61a39fd4f84424b129c1a6f9ba404d" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:38.174846 20112 raft_consensus.cc:493] T e5ad4eb281564936a59f41e387a47156 P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
W20260709 13:20:38.227290 20093 consensus_meta.cc:287] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b: Time spent flushing consensus metadata: real 0.901s	user 0.012s	sys 0.000s
W20260709 13:20:38.213701 16822 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49906 (request call id 810) took 2044 ms (client timeout 2040 ms). Trace:
W20260709 13:20:38.177945 19974 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.096s	user 0.004s	sys 0.019s
I20260709 13:20:38.195114 20044 consensus_queue.cc:237] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:38.184440 19702 raft_consensus.cc:3060] T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:20:38.170212 19863 raft_consensus.cc:2749] T 08c74490c40243a88e62d1760b5cd814 P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:20:38.179277 19789 leader_election.cc:290] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:38.172335 19822 leader_election.cc:290] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:38.227758 16291 raft_consensus.cc:2468] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 8.
W20260709 13:20:38.220139 20123 log.cc:927] Time spent T 5a96373bb0e14dce8a266b2fc6c58507 P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 0.189s	user 0.000s	sys 0.001s
W20260709 13:20:38.147835 20075 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.752s	user 0.006s	sys 0.016s
W20260709 13:20:38.220534 20121 log.cc:927] Time spent T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 0.138s	user 0.000s	sys 0.001s
I20260709 13:20:38.168196 19955 raft_consensus.cc:515] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:38.080559 19823 raft_consensus.cc:515] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:38.237540 16294 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48674 (request call id 2536) took 1603 ms. Trace:
I20260709 13:20:38.233634 19856 raft_consensus.cc:2804] T ee9ed67006144797b1de6ea8c2013a68 P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:20:38.214388 20105 raft_consensus.cc:515] T 9c61a39fd4f84424b129c1a6f9ba404d P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:38.236171 19966 raft_consensus.cc:2804] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:20:38.236425 16300 rpcz_store.cc:276] 0709 13:20:36.904942 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:37.222982 (+318040us) service_pool.cc:224] Handling call
0709 13:20:37.269903 (+ 46921us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:37.270147 (+   244us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:20:37.284526 (+ 14379us) log.cc:844] Serialized 14 byte log entry
0709 13:20:38.181372 (+896846us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:20:38.181447 (+    75us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:20:38.181510 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:38.181609 (+    99us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:38.181772 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":25299,"prepare.queue_time_us":13841,"prepare.run_cpu_time_us":49,"prepare.run_wall_time_us":50,"raft.queue_time_us":1205,"raft.run_cpu_time_us":121,"raft.run_wall_time_us":7184,"thread_start_us":895191,"threads_started":1}
I20260709 13:20:38.227476 20096 raft_consensus.cc:515] T 5a96373bb0e14dce8a266b2fc6c58507 P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:38.231621 19699 raft_consensus.cc:515] T 84f2da98864b4425bf7c0055951fdd4b P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:38.229393 20021 raft_consensus.cc:515] T b87147731a68400e8cce9b879daa7c5d P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:38.233525 20056 raft_consensus.cc:515] T 66f75124215f4ca6bef831e174a792d8 P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:38.291958 16296 raft_consensus.cc:2468] T 68a6838d82ef4cac83507f906139d90f P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 11.
I20260709 13:20:38.251456 16144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746aabf856ab401c9ef5e859371e7b67" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:38.271675 16294 rpcz_store.cc:276] 0709 13:20:36.633711 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:36.633870 (+   159us) service_pool.cc:224] Handling call
0709 13:20:36.697238 (+ 63368us) raft_consensus.cc:1471] Updating replica for 4 ops
0709 13:20:36.787485 (+ 90247us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:36.790247 (+  2762us) raft_consensus.cc:1539] Triggering prepare for 4 ops
0709 13:20:36.813029 (+ 22782us) log.cc:844] Serialized 104 byte log entry
0709 13:20:37.495685 (+682656us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:20:38.232735 (+737050us) log.cc:844] Serialized 14 byte log entry
0709 13:20:38.237231 (+  4496us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:38.237303 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:20:38.237344 (+    41us) raft_consensus.cc:1685] finished
0709 13:20:38.237355 (+    11us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:38.237489 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":27414,"prepare.queue_time_us":24577,"prepare.run_cpu_time_us":72,"prepare.run_wall_time_us":9054,"raft.queue_time_us":315694,"raft.run_cpu_time_us":333,"raft.run_wall_time_us":69816,"thread_start_us":1472748,"threads_started":3,"wal-append.queue_time_us":1082260}
I20260709 13:20:38.301921 16150 raft_consensus.cc:2468] T 9c61a39fd4f84424b129c1a6f9ba404d P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 9.
W20260709 13:20:38.245759 16822 rpcz_store.cc:269] 0709 13:20:36.162179 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:36.596969 (+434790us) service_pool.cc:224] Handling call
0709 13:20:38.196894 (+1599925us) spinlock_profiling.cc:243] Waited 301 ms on lock 0x7b54005032d4. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:20:38.206793 (+  9899us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":693226240}
W20260709 13:20:38.264665 16820 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.864s	user 0.009s	sys 0.045s
I20260709 13:20:38.261015 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5718c623ebee4d2bb770f0f9ddb2f164" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
W20260709 13:20:38.302696 16820 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.902s	user 0.009s	sys 0.045s
I20260709 13:20:38.302898 16225 raft_consensus.cc:2468] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 9.
I20260709 13:20:38.304184 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "829b4ad8c5a54eababeea1181b960d4b" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
W20260709 13:20:38.264684 16190 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.548s	user 0.006s	sys 0.029s
W20260709 13:20:38.307407 16190 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.591s	user 0.006s	sys 0.029s
W20260709 13:20:38.307713 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 594225us
I20260709 13:20:38.316242 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f2da98864b4425bf7c0055951fdd4b" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:38.314224 16190 leader_election.cc:304] T e4dc6b0541ca431ebc757beac00521f1 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:20:38.318436 16190 leader_election.cc:304] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:20:38.320739 16822 raft_consensus.cc:1240] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [term 12 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 11. Current term is 12. Ops: []
I20260709 13:20:38.321344 16822 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49906 (request call id 816) took 1492 ms. Trace:
I20260709 13:20:38.321641 16822 rpcz_store.cc:276] 0709 13:20:36.828963 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:38.320192 (+1491229us) service_pool.cc:224] Handling call
0709 13:20:38.320553 (+   361us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:38.321128 (+   575us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:38.321293 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:20:38.257609 20090 consensus_meta.cc:287] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045: Time spent flushing consensus metadata: real 0.964s	user 0.002s	sys 0.008s
I20260709 13:20:38.325335 16225 raft_consensus.cc:2410] T 84f2da98864b4425bf7c0055951fdd4b P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b for term 9 because replica has last-logged OpId of term: 7 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 13:20:38.325919 16190 leader_election.cc:304] T c2f5f89186204242bf377f7c42b2e2da P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
W20260709 13:20:38.279424 19806 consensus_meta.cc:287] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860: Time spent flushing consensus metadata: real 0.607s	user 0.005s	sys 0.006s
I20260709 13:20:38.327471 16144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68a6838d82ef4cac83507f906139d90f" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
W20260709 13:20:38.273638 19898 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.895s	user 0.009s	sys 0.029s
I20260709 13:20:38.330773 16148 raft_consensus.cc:1240] T b87147731a68400e8cce9b879daa7c5d P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 10. Current term is 11. Ops: [10.9-10.9]
I20260709 13:20:38.244715 19667 leader_election.cc:290] T 19baa7bc367844c288bca8f81439c413 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:38.301448 19966 raft_consensus.cc:697] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac [term 8 LEADER]: Becoming Leader. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:20:38.331745 16190 leader_election.cc:304] T 8cc959fb7f704fba9009c0595467d074 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
W20260709 13:20:38.393153 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 61483.5us
I20260709 13:20:38.368274 16298 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48684 (request call id 2388) took 1368 ms. Trace:
W20260709 13:20:38.239733 20068 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.524s	user 0.012s	sys 0.021s
I20260709 13:20:38.344162 19823 leader_election.cc:290] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 9 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:38.278314 19856 raft_consensus.cc:697] T ee9ed67006144797b1de6ea8c2013a68 P 1ca2725d290e47d68b1a4b609c9816ac [term 8 LEADER]: Becoming Leader. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:20:38.347579 16148 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46224 (request call id 567) took 1805 ms. Trace:
I20260709 13:20:38.350584 19751 raft_consensus.cc:515] T d16dfad0c3b540e5b4b86492ddf87208 P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:38.270489 19863 consensus_queue.cc:1048] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.081s
I20260709 13:20:38.351859 19760 consensus_queue.cc:1048] T d16dfad0c3b540e5b4b86492ddf87208 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:20:38.352013 16144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5d59407eb545b895d8d922e0dd69fd" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
W20260709 13:20:38.353782 19898 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.965s	user 0.009s	sys 0.030s
I20260709 13:20:38.353711 19955 leader_election.cc:290] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 13 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
W20260709 13:20:38.354326 20118 log.cc:927] Time spent T 18ad7e3af5504571a69d10c00750a01c P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 0.184s	user 0.000s	sys 0.001s
I20260709 13:20:38.355604 20105 leader_election.cc:290] T 9c61a39fd4f84424b129c1a6f9ba404d P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:38.356160 20129 raft_consensus.cc:493] T fffab26427e24d398e2695aad40ca134 P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:20:38.356858 20125 log.cc:927] Time spent T 8c11074579b046d99e585c2818e18a2c P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.174s	user 0.001s	sys 0.000s
I20260709 13:20:38.356626 19702 raft_consensus.cc:515] T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:38.359459 16222 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57600 (request call id 1360) took 1524 ms. Trace:
I20260709 13:20:38.360108 20109 consensus_queue.cc:237] T 8a878999c0024d2abe517484180290fc P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 8.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:20:38.365146 20131 log.cc:927] Time spent T 66ae176961b74813a58bc15c9c4c6f2a P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.082s	user 0.000s	sys 0.000s
I20260709 13:20:38.367026 20056 leader_election.cc:290] T 66f75124215f4ca6bef831e174a792d8 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 10 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
W20260709 13:20:38.391307 16192 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.634s	user 0.029s	sys 0.020s
W20260709 13:20:38.434237 16192 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.677s	user 0.029s	sys 0.020s
I20260709 13:20:38.369045 16821 raft_consensus.cc:2468] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 9.
I20260709 13:20:38.393941 16190 leader_election.cc:304] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:20:38.255299 20049 raft_consensus.cc:2749] T 8158bbd17b994d2eb96f560bd0f1b88d P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:20:38.393787 16298 rpcz_store.cc:276] 0709 13:20:36.999908 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:37.370556 (+370648us) service_pool.cc:224] Handling call
0709 13:20:37.371011 (+   455us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:20:37.371503 (+   492us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:20:37.377082 (+  5579us) log.cc:844] Serialized 14 byte log entry
0709 13:20:38.242917 (+865835us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:20:38.267844 (+ 24927us) log.cc:844] Serialized 26 byte log entry
0709 13:20:38.268022 (+   178us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:20:38.268070 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:38.268145 (+    75us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:20:38.368010 (+ 99865us) raft_consensus.cc:1685] finished
0709 13:20:38.368038 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:38.368205 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":9201,"prepare.queue_time_us":5099,"prepare.run_cpu_time_us":35,"prepare.run_wall_time_us":36,"raft.queue_time_us":18368,"raft.run_cpu_time_us":221,"raft.run_wall_time_us":6237,"thread_start_us":855824,"threads_started":1,"wal-append.queue_time_us":643729}
I20260709 13:20:38.398701 16819 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49892 (request call id 554) took 1892 ms. Trace:
I20260709 13:20:38.400803 16116 leader_election.cc:304] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
I20260709 13:20:38.402619 16148 rpcz_store.cc:276] 0709 13:20:36.541788 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:36.541959 (+   171us) service_pool.cc:224] Handling call
0709 13:20:36.562408 (+ 20449us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:20:38.345926 (+1783518us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:38.347326 (+  1400us) spinlock_profiling.cc:243] Waited 1.77 s on lock 0x7b540041dcd4. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:20:38.347519 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4066133376}
I20260709 13:20:38.337877 20130 raft_consensus.cc:2804] T e4dc6b0541ca431ebc757beac00521f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:20:38.394798 20069 raft_consensus.cc:3055] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:20:38.338354 19822 raft_consensus.cc:2804] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 13:20:38.367368 20127 log.cc:927] Time spent T 2794af5b99984efebdee564035841cb0 P cb82cc7bd6db46d980c75237a1402860: Append to log took a long time: real 0.146s	user 0.000s	sys 0.001s
I20260709 13:20:38.441814 16819 rpcz_store.cc:276] 0709 13:20:36.505964 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:37.794476 (+1288512us) service_pool.cc:224] Handling call
0709 13:20:37.794887 (+   411us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:37.795104 (+   217us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:20:38.222129 (+427025us) log.cc:844] Serialized 14 byte log entry
0709 13:20:38.398365 (+176236us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:20:38.398421 (+    56us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:20:38.398466 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:38.398533 (+    67us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:38.398646 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3753,"prepare.queue_time_us":549782,"prepare.run_cpu_time_us":29,"prepare.run_wall_time_us":30,"raft.queue_time_us":75,"raft.run_cpu_time_us":237,"raft.run_wall_time_us":5776,"thread_start_us":602429,"threads_started":2}
W20260709 13:20:38.452092 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 58231.6us
W20260709 13:20:38.396111 20068 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.684s	user 0.013s	sys 0.021s
I20260709 13:20:38.353742 19699 leader_election.cc:290] T 84f2da98864b4425bf7c0055951fdd4b P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:38.413192 19997 raft_consensus.cc:2804] T 8cc959fb7f704fba9009c0595467d074 P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:20:38.261905 20093 raft_consensus.cc:515] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:20:38.402062 19904 consensus_meta.cc:287] T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860: Time spent flushing consensus metadata: real 1.695s	user 0.015s	sys 0.000s
I20260709 13:20:38.399897 20014 raft_consensus.cc:493] T 2794af5b99984efebdee564035841cb0 P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:38.260923 20112 raft_consensus.cc:515] T e5ad4eb281564936a59f41e387a47156 P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:38.403999 19667 raft_consensus.cc:2804] T c2f5f89186204242bf377f7c42b2e2da P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:20:38.448203 20041 raft_consensus.cc:2804] T 8c11074579b046d99e585c2818e18a2c P cb82cc7bd6db46d980c75237a1402860 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:20:38.446095 16820 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49892 (request call id 552) took 2213 ms. Trace:
I20260709 13:20:38.453047 16190 leader_election.cc:304] T 68a6838d82ef4cac83507f906139d90f P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:20:38.493907 16820 rpcz_store.cc:276] 0709 13:20:36.232942 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:37.399518 (+1166576us) service_pool.cc:224] Handling call
0709 13:20:37.399812 (+   294us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:37.400001 (+   189us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:20:38.303064 (+903063us) log.cc:844] Serialized 14 byte log entry
0709 13:20:38.445238 (+142174us) log.cc:844] Serialized 14 byte log entry
0709 13:20:38.445561 (+   323us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:20:38.445665 (+   104us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:20:38.445728 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:38.445842 (+   114us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:38.446002 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5528,"prepare.queue_time_us":945705,"prepare.run_cpu_time_us":29,"prepare.run_wall_time_us":80,"raft.queue_time_us":7399,"raft.run_cpu_time_us":82,"raft.run_wall_time_us":82,"spinlock_wait_cycles":34464000,"thread_start_us":1025204,"threads_started":2}
W20260709 13:20:38.448205 20087 consensus_meta.cc:287] T 1df93779209748b9860f6d893dd3de53 P 1ca2725d290e47d68b1a4b609c9816ac: Time spent flushing consensus metadata: real 1.088s	user 0.010s	sys 0.003s
I20260709 13:20:38.434993 19958 raft_consensus.cc:493] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:38.341872 20090 raft_consensus.cc:515] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } }
I20260709 13:20:38.428102 16222 rpcz_store.cc:276] 0709 13:20:36.836776 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:36.836892 (+   116us) service_pool.cc:224] Handling call
0709 13:20:36.837185 (+   293us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:20:36.837528 (+   343us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:20:37.679836 (+842308us) log.cc:844] Serialized 14 byte log entry
0709 13:20:38.095888 (+416052us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:20:38.150944 (+ 55056us) log.cc:844] Serialized 26 byte log entry
0709 13:20:38.151104 (+   160us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:20:38.151416 (+   312us) log.cc:844] Serialized 14 byte log entry
0709 13:20:38.153246 (+  1830us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:38.153349 (+   103us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:20:38.359252 (+205903us) raft_consensus.cc:1685] finished
0709 13:20:38.359272 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:38.359408 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1561,"prepare.queue_time_us":1111513,"prepare.run_cpu_time_us":100,"prepare.run_wall_time_us":4245,"raft.queue_time_us":71690,"raft.run_cpu_time_us":455,"raft.run_wall_time_us":55025,"thread_start_us":1157299,"threads_started":2,"wal-append.queue_time_us":628456}
I20260709 13:20:38.502202 16222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ab0ac8347641e6955e9bce7d90c1a2" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:38.448850 20140 raft_consensus.cc:2804] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:20:38.502792 16222 raft_consensus.cc:2468] T d2ab0ac8347641e6955e9bce7d90c1a2 P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 9.
I20260709 13:20:38.445966 20130 raft_consensus.cc:493] T e4dc6b0541ca431ebc757beac00521f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:38.451499 19789 raft_consensus.cc:493] T 08c74490c40243a88e62d1760b5cd814 P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:38.465024 19823 raft_consensus.cc:2804] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:20:38.499181 16144 raft_consensus.cc:2393] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 11: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
I20260709 13:20:38.453163 19822 raft_consensus.cc:493] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:38.448890 20069 raft_consensus.cc:740] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:20:38.454704 19997 raft_consensus.cc:493] T 8cc959fb7f704fba9009c0595467d074 P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:38.486239 16148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8158bbd17b994d2eb96f560bd0f1b88d" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
W20260709 13:20:38.500819 16192 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 65528.4us
I20260709 13:20:38.476554 20073 raft_consensus.cc:493] T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:38.485204 20014 raft_consensus.cc:515] T 2794af5b99984efebdee564035841cb0 P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:38.517778 16192 leader_election.cc:304] T ecc1e363230043fe9fef390f0b979f20 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860
I20260709 13:20:38.431806 20129 raft_consensus.cc:515] T fffab26427e24d398e2695aad40ca134 P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:38.519274 16192 leader_election.cc:304] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
I20260709 13:20:38.485445 19667 raft_consensus.cc:493] T c2f5f89186204242bf377f7c42b2e2da P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Starting leader election (detected failure of leader 647a343a542d4a139fe106c19095129b)
W20260709 13:20:38.521649 16115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 81063.8us
I20260709 13:20:38.498817 20093 leader_election.cc:290] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 11 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:38.514477 19823 raft_consensus.cc:493] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:38.508018 20140 raft_consensus.cc:493] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Starting leader election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:20:38.525775 20087 raft_consensus.cc:2749] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:20:38.526222 16143 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40830 (request call id 1363) took 2096 ms (client timeout 1592 ms). Trace:
W20260709 13:20:38.529280 16143 rpcz_store.cc:269] 0709 13:20:36.429378 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:36.429489 (+   111us) service_pool.cc:224] Handling call
0709 13:20:38.526003 (+2096514us) spinlock_profiling.cc:243] Waited 195 ms on lock 0x7b540020ba54. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:20:38.526147 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":447556992}
I20260709 13:20:38.526824 20066 leader_election.cc:290] T ecc1e363230043fe9fef390f0b979f20 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:38.503403 19966 consensus_queue.cc:237] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:38.488595 20067 raft_consensus.cc:515] T 5a959dbecc2648eb85d1cc550f5a3245 P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:38.504535 19699 raft_consensus.cc:493] T 8c11074579b046d99e585c2818e18a2c P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:20:38.530529 16143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c2095a5f4ad49408dbc7f69c606120d" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:38.504410 20139 raft_consensus.cc:2804] T 68a6838d82ef4cac83507f906139d90f P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:20:38.512444 20022 raft_consensus.cc:493] T 829b4ad8c5a54eababeea1181b960d4b P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:20:38.532240 16148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66f75124215f4ca6bef831e174a792d8" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:20:38.511404 20130 raft_consensus.cc:3060] T e4dc6b0541ca431ebc757beac00521f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:20:38.556552 19699 raft_consensus.cc:515] T 8c11074579b046d99e585c2818e18a2c P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:38.560468 19702 leader_election.cc:290] T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 10 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:38.562142 19699 leader_election.cc:290] T 8c11074579b046d99e585c2818e18a2c P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:38.535699 19863 consensus_queue.cc:1059] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: INVALID_TERM, Last received: 11.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:20:38.510860 19958 raft_consensus.cc:515] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:38.562609 20022 raft_consensus.cc:515] T 829b4ad8c5a54eababeea1181b960d4b P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:38.563126 19861 leader_election.cc:290] T d2ab0ac8347641e6955e9bce7d90c1a2 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:38.555054 20087 raft_consensus.cc:493] T c0eab25ce4c64e018a041beeda2867db P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:20:38.564368 20087 raft_consensus.cc:515] T c0eab25ce4c64e018a041beeda2867db P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:38.564720 20022 leader_election.cc:290] T 829b4ad8c5a54eababeea1181b960d4b P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:38.520157 20056 raft_consensus.cc:2749] T ecc1e363230043fe9fef390f0b979f20 P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:20:38.538333 20014 leader_election.cc:290] T 2794af5b99984efebdee564035841cb0 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:38.516327 19822 raft_consensus.cc:3060] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:20:38.566004 20090 leader_election.cc:290] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 10 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:20:38.566367 20087 leader_election.cc:290] T c0eab25ce4c64e018a041beeda2867db P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
W20260709 13:20:38.522576 20120 log.cc:927] Time spent T c0eab25ce4c64e018a041beeda2867db P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 0.207s	user 0.000s	sys 0.001s
I20260709 13:20:38.566730 19751 leader_election.cc:290] T d16dfad0c3b540e5b4b86492ddf87208 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 10 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:38.527485 16147 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40844 (request call id 2230) took 1727 ms. Trace:
I20260709 13:20:38.567312 16147 rpcz_store.cc:276] 0709 13:20:36.799864 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:36.799984 (+   120us) service_pool.cc:224] Handling call
0709 13:20:38.527302 (+1727318us) spinlock_profiling.cc:243] Waited 217 ms on lock 0x7b54003d9fd4. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:20:38.527432 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":498865024}
I20260709 13:20:38.568305 20129 leader_election.cc:290] T fffab26427e24d398e2695aad40ca134 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:38.568459 16147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a96373bb0e14dce8a266b2fc6c58507" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 10 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:20:38.569751 16147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f2da98864b4425bf7c0055951fdd4b" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:38.571666 16148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4dc6b0541ca431ebc757beac00521f1" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:38.571964 16218 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57604 (request call id 2474) took 1575 ms. Trace:
I20260709 13:20:38.572422 16148 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40844 (request call id 2246) took 1072 ms. Trace:
I20260709 13:20:38.572921 16148 rpcz_store.cc:276] 0709 13:20:37.499894 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:38.571423 (+1071529us) service_pool.cc:224] Handling call
0709 13:20:38.572368 (+   945us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:20:38.572399 16218 rpcz_store.cc:276] 0709 13:20:36.997027 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:36.997151 (+   124us) service_pool.cc:224] Handling call
0709 13:20:37.010569 (+ 13418us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:20:37.010976 (+   407us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:37.011015 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:20:37.057403 (+ 46388us) log.cc:844] Serialized 26 byte log entry
0709 13:20:38.225022 (+1167619us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:20:38.225059 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:38.225142 (+    83us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:20:38.571749 (+346607us) raft_consensus.cc:1685] finished
0709 13:20:38.571770 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:38.571902 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":85593,"raft.run_cpu_time_us":138,"raft.run_wall_time_us":8240,"spinlock_wait_cycles":42752,"thread_start_us":1086048,"threads_started":1,"wal-append.queue_time_us":1189353}
I20260709 13:20:38.574118 16148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87ac0d1f7e8a419f9bcbc04aee0e067e" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 11 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:38.518528 19997 raft_consensus.cc:3060] T 8cc959fb7f704fba9009c0595467d074 P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:20:38.575846 20139 raft_consensus.cc:697] T 68a6838d82ef4cac83507f906139d90f P 1ca2725d290e47d68b1a4b609c9816ac [term 11 LEADER]: Becoming Leader. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:20:38.576856 20139 consensus_queue.cc:237] T 68a6838d82ef4cac83507f906139d90f P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:38.578104 16148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:38.544878 20069 consensus_queue.cc:260] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:20:38.548736 16789 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:38.517378 19789 raft_consensus.cc:515] T 08c74490c40243a88e62d1760b5cd814 P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:38.528504 19667 raft_consensus.cc:3060] T c2f5f89186204242bf377f7c42b2e2da P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:20:38.587803 20112 leader_election.cc:290] T e5ad4eb281564936a59f41e387a47156 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:20:38.588356 20069 raft_consensus.cc:3060] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:20:38.557106 16143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62fe3346cd3c42c4a55e14af9073bca4" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:20:38.592330 16143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 11 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:38.593164 16143 raft_consensus.cc:2468] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 10.
I20260709 13:20:38.550699 20021 leader_election.cc:290] T b87147731a68400e8cce9b879daa7c5d P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 11 election: Requested vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:38.481673 19856 consensus_queue.cc:237] T ee9ed67006144797b1de6ea8c2013a68 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:38.558774 16191 leader_election.cc:304] T af57996a8be84d398f0f023e8d08ef77 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
I20260709 13:20:38.574995 16218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:38.570873 16147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ad5dbfa99ee4b8d9e28be92b3f36db5" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:38.598191 16147 raft_consensus.cc:2468] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 8.
I20260709 13:20:38.523866 20073 raft_consensus.cc:515] T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:38.530798 20140 raft_consensus.cc:3060] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:20:38.602859 16218 raft_consensus.cc:2393] T ecc1e363230043fe9fef390f0b979f20 P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 9: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
W20260709 13:20:38.605664 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 47028.4us
I20260709 13:20:38.586404 16148 raft_consensus.cc:2410] T ecc1e363230043fe9fef390f0b979f20 P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac for term 10 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
W20260709 13:20:38.586472 16789 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:38.611539 16820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b7d1ab519044eda2d5e04a07242f25" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 8 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:38.612309 16820 raft_consensus.cc:2393] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 8: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
I20260709 13:20:38.613684 16822 raft_consensus.cc:3060] T 806d7690c2b644959152074bb2cab0a7 P cb82cc7bd6db46d980c75237a1402860 [term 8 LEARNER]: Advancing to term 9
W20260709 13:20:38.594604 20117 log.cc:927] Time spent T 8a878999c0024d2abe517484180290fc P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.562s	user 0.003s	sys 0.002s
I20260709 13:20:38.530987 19823 raft_consensus.cc:3060] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:20:38.595675 16143 raft_consensus.cc:1240] T 8d4ab870a05347868956aa8b841b2981 P 647a343a542d4a139fe106c19095129b [term 11 LEADER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 10. Current term is 11. Ops: []
W20260709 13:20:38.619552 16226 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57600 (request call id 1359) took 1785 ms (client timeout 1991 ms). Trace:
I20260709 13:20:38.619796 16218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66ae176961b74813a58bc15c9c4c6f2a" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
W20260709 13:20:38.620004 16226 rpcz_store.cc:269] 0709 13:20:36.834618 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:36.835035 (+   417us) service_pool.cc:224] Handling call
0709 13:20:38.619406 (+1784371us) spinlock_profiling.cc:243] Waited 114 ms on lock 0x7b5400368654. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:20:38.619499 (+    93us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":262581120}
I20260709 13:20:38.593412 19861 raft_consensus.cc:3055] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:20:38.586993 20048 raft_consensus.cc:493] T 66f75124215f4ca6bef831e174a792d8 P cb82cc7bd6db46d980c75237a1402860 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:20:38.620659 16191 leader_election.cc:304] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:20:38.620548 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74ca8140f35c4ae29e8a664f6bdfc829" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:38.619592 16143 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40844 (request call id 2235) took 1799 ms. Trace:
I20260709 13:20:38.622716 16824 raft_consensus.cc:2410] T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac for term 10 because replica has last-logged OpId of term: 9 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
I20260709 13:20:38.633157 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5d59407eb545b895d8d922e0dd69fd" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:20:38.633785 16226 raft_consensus.cc:3060] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:20:38.623114 16218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ad5dbfa99ee4b8d9e28be92b3f36db5" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:38.590595 20130 raft_consensus.cc:515] T e4dc6b0541ca431ebc757beac00521f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:38.616720 16820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a378d7f796f94615b3b347119a9880fd" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:38.632787 16143 rpcz_store.cc:276] 0709 13:20:36.820046 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:38.595206 (+1775160us) service_pool.cc:224] Handling call
0709 13:20:38.595506 (+   300us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:38.619374 (+ 23868us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:38.619545 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:20:38.632377 16825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87147731a68400e8cce9b879daa7c5d" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:38.642551 16822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74ca8140f35c4ae29e8a664f6bdfc829" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:38.615924 19863 raft_consensus.cc:2749] T af57996a8be84d398f0f023e8d08ef77 P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:20:38.643242 16822 raft_consensus.cc:2468] T 74ca8140f35c4ae29e8a664f6bdfc829 P cb82cc7bd6db46d980c75237a1402860 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 9.
I20260709 13:20:38.615576 19806 raft_consensus.cc:493] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:20:38.644446 16143 raft_consensus.cc:1240] T b6f07842d3aa47858df5665874c86f3a P 647a343a542d4a139fe106c19095129b [term 11 LEADER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 10. Current term is 11. Ops: [10.7-10.7]
I20260709 13:20:38.645766 16143 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40844 (request call id 2244) took 1154 ms. Trace:
W20260709 13:20:38.646245 20151 log.cc:927] Time spent T 8a878999c0024d2abe517484180290fc P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 0.059s	user 0.001s	sys 0.000s
I20260709 13:20:38.646145 16143 rpcz_store.cc:276] 0709 13:20:37.490951 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:38.641689 (+1150738us) service_pool.cc:224] Handling call
0709 13:20:38.644190 (+  2501us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:20:38.645492 (+  1302us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:38.645691 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:20:38.646730 19789 leader_election.cc:290] T 08c74490c40243a88e62d1760b5cd814 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:38.645911 19806 raft_consensus.cc:515] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:38.649893 19806 leader_election.cc:290] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:20:38.641366 20109 raft_consensus.cc:2749] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:20:38.640130 20048 raft_consensus.cc:515] T 66f75124215f4ca6bef831e174a792d8 P cb82cc7bd6db46d980c75237a1402860 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:38.665179 16148 raft_consensus.cc:1240] T 8cc959fb7f704fba9009c0595467d074 P 647a343a542d4a139fe106c19095129b [term 10 LEADER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 9. Current term is 10. Ops: [9.7-9.7]
I20260709 13:20:38.666152 16148 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40844 (request call id 2240) took 1814 ms. Trace:
I20260709 13:20:38.666601 16148 rpcz_store.cc:276] 0709 13:20:36.851599 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:38.635601 (+1784002us) service_pool.cc:224] Handling call
0709 13:20:38.664920 (+ 29319us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:20:38.665775 (+   855us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:38.666065 (+   290us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:20:38.654788 19997 raft_consensus.cc:515] T 8cc959fb7f704fba9009c0595467d074 P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:38.670141 19855 leader_election.cc:290] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:38.633127 19861 raft_consensus.cc:740] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:20:38.678364 19861 consensus_queue.cc:260] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:38.679505 19997 leader_election.cc:290] T 8cc959fb7f704fba9009c0595467d074 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 11 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:38.679838 19861 raft_consensus.cc:3060] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:20:38.683070 19823 raft_consensus.cc:515] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:38.684474 16218 raft_consensus.cc:2393] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 9: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
I20260709 13:20:38.686324 16226 raft_consensus.cc:2468] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 11.
I20260709 13:20:38.687455 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c61a39fd4f84424b129c1a6f9ba404d" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:38.688354 16226 raft_consensus.cc:2393] T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 10: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
I20260709 13:20:38.690222 16218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 11 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:38.690843 16218 raft_consensus.cc:2468] T 19baa7bc367844c288bca8f81439c413 P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 10.
I20260709 13:20:38.670504 20130 leader_election.cc:290] T e4dc6b0541ca431ebc757beac00521f1 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 10 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:38.702574 20067 leader_election.cc:290] T 5a959dbecc2648eb85d1cc550f5a3245 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 8 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:38.703660 19667 raft_consensus.cc:515] T c2f5f89186204242bf377f7c42b2e2da P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:38.707908 16218 raft_consensus.cc:1240] T 5a96373bb0e14dce8a266b2fc6c58507 P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 9. Current term is 10. Ops: []
I20260709 13:20:38.708765 16218 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57604 (request call id 2484) took 1708 ms. Trace:
I20260709 13:20:38.709074 16218 rpcz_store.cc:276] 0709 13:20:36.999905 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:38.705100 (+1705195us) service_pool.cc:224] Handling call
0709 13:20:38.707742 (+  2642us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:38.708541 (+   799us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:38.708715 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:20:38.706130 16226 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57604 (request call id 2485) took 1706 ms. Trace:
I20260709 13:20:38.712993 19822 raft_consensus.cc:515] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:38.714529 20058 leader_election.cc:290] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:38.715441 16217 raft_consensus.cc:2393] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 in current term 10: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
W20260709 13:20:38.716102 20147 log.cc:927] Time spent T 57a744f5aaf944678a63469cbf514015 P cb82cc7bd6db46d980c75237a1402860: Append to log took a long time: real 0.191s	user 0.001s	sys 0.000s
I20260709 13:20:38.695662 20064 leader_election.cc:290] T 434bff3082d141838068f323bdb0d1ed P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:38.692981 20140 raft_consensus.cc:515] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:38.722690 20140 leader_election.cc:290] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 10 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:38.723903 16218 raft_consensus.cc:1275] T b6f07842d3aa47858df5665874c86f3a P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 11 index: 7. (term mismatch)
I20260709 13:20:38.727305 16218 pending_rounds.cc:85] T b6f07842d3aa47858df5665874c86f3a P 1ca2725d290e47d68b1a4b609c9816ac: Aborting all ops after (but not including) 6
I20260709 13:20:38.727872 16218 pending_rounds.cc:107] T b6f07842d3aa47858df5665874c86f3a P 1ca2725d290e47d68b1a4b609c9816ac: Aborting uncommitted NO_OP operation due to leader change: 10.7
I20260709 13:20:38.728457 16218 raft_consensus.cc:2889] T b6f07842d3aa47858df5665874c86f3a P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:20:38.728976 19941 consensus_queue.cc:1059] T b87147731a68400e8cce9b879daa7c5d P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:20:38.729892 19941 raft_consensus.cc:3055] T b87147731a68400e8cce9b879daa7c5d P cb82cc7bd6db46d980c75237a1402860 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:20:38.730245 19941 raft_consensus.cc:740] T b87147731a68400e8cce9b879daa7c5d P cb82cc7bd6db46d980c75237a1402860 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:20:38.731541 16218 raft_consensus.cc:1240] T e4dc6b0541ca431ebc757beac00521f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 9. Current term is 10. Ops: []
I20260709 13:20:38.724766 16223 raft_consensus.cc:1240] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 9. Current term is 10. Ops: []
W20260709 13:20:38.725929 16220 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57600 (request call id 1353) took 1895 ms (client timeout 1767 ms). Trace:
I20260709 13:20:38.726277 19667 leader_election.cc:290] T c2f5f89186204242bf377f7c42b2e2da P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 11 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
W20260709 13:20:38.734704 16220 rpcz_store.cc:269] 0709 13:20:36.830337 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:36.830470 (+   133us) service_pool.cc:224] Handling call
0709 13:20:38.725741 (+1895271us) spinlock_profiling.cc:243] Waited 406 ms on lock 0x7b5400297cd4. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:20:38.725865 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":934660480}
I20260709 13:20:38.711272 16226 rpcz_store.cc:276] 0709 13:20:37.000139 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:38.705358 (+1705219us) service_pool.cc:224] Handling call
0709 13:20:38.705660 (+   302us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:38.705835 (+   175us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:38.705857 (+    22us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:20:38.705897 (+    40us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:20:38.705925 (+    28us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:38.705973 (+    48us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:38.706078 (+   105us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:20:38.740665 16824 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49906 (request call id 805) took 2604 ms (client timeout 1810 ms). Trace:
W20260709 13:20:38.741250 16824 rpcz_store.cc:269] 0709 13:20:36.136625 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:36.136813 (+   188us) service_pool.cc:224] Handling call
0709 13:20:38.740432 (+2603619us) spinlock_profiling.cc:243] Waited 2.34 s on lock 0x7b54006f2054. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:20:38.740574 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5388909824}
I20260709 13:20:38.751036 19823 leader_election.cc:290] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 9 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:38.751636 16218 raft_consensus.cc:1240] T 5a959dbecc2648eb85d1cc550f5a3245 P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 7. Current term is 8. Ops: []
I20260709 13:20:38.749976 19822 leader_election.cc:290] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 10 election: Requested vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:38.768208 16221 raft_consensus.cc:1240] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 12. Current term is 13. Ops: []
W20260709 13:20:38.847622 20157 log.cc:927] Time spent T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860: Append to log took a long time: real 0.085s	user 0.001s	sys 0.000s
I20260709 13:20:38.850987 20080 raft_consensus.cc:493] T 18ad7e3af5504571a69d10c00750a01c P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:38.851645 20080 raft_consensus.cc:515] T 18ad7e3af5504571a69d10c00750a01c P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } }
I20260709 13:20:38.731073 19941 consensus_queue.cc:260] T b87147731a68400e8cce9b879daa7c5d P cb82cc7bd6db46d980c75237a1402860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:38.893800 16218 raft_consensus.cc:1240] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 8. Current term is 9. Ops: []
I20260709 13:20:38.879302 19822 raft_consensus.cc:493] T b6f07842d3aa47858df5665874c86f3a P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:38.901400 19822 raft_consensus.cc:515] T b6f07842d3aa47858df5665874c86f3a P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:38.907226 16225 raft_consensus.cc:1240] T 66f75124215f4ca6bef831e174a792d8 P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 9. Current term is 10. Ops: []
I20260709 13:20:38.912776 16117 leader_election.cc:304] T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860
I20260709 13:20:38.914179 19941 raft_consensus.cc:3060] T b87147731a68400e8cce9b879daa7c5d P cb82cc7bd6db46d980c75237a1402860 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:20:38.917029 20057 raft_consensus.cc:493] T 2794af5b99984efebdee564035841cb0 P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:38.917299 19914 raft_consensus.cc:2749] T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:20:38.924614 19939 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 2.202s	user 0.009s	sys 0.028s
W20260709 13:20:38.927111 16789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 168873us
I20260709 13:20:38.925359 20057 raft_consensus.cc:515] T 2794af5b99984efebdee564035841cb0 P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:38.932916 19938 raft_consensus.cc:493] T 66ae176961b74813a58bc15c9c4c6f2a P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:20:38.933203 20057 leader_election.cc:290] T 2794af5b99984efebdee564035841cb0 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:38.933497 19938 raft_consensus.cc:515] T 66ae176961b74813a58bc15c9c4c6f2a P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:38.934058 19822 leader_election.cc:290] T b6f07842d3aa47858df5665874c86f3a P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:38.935560 19938 leader_election.cc:290] T 66ae176961b74813a58bc15c9c4c6f2a P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
W20260709 13:20:38.937853 16817 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49906 (request call id 806) took 2799 ms (client timeout 2087 ms). Trace:
I20260709 13:20:38.938074 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74ca8140f35c4ae29e8a664f6bdfc829" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:38.938469 19938 raft_consensus.cc:493] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
W20260709 13:20:38.938318 16817 rpcz_store.cc:269] 0709 13:20:36.138160 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:36.628090 (+489930us) service_pool.cc:224] Handling call
0709 13:20:38.937682 (+2309592us) spinlock_profiling.cc:243] Waited 636 ms on lock 0x7b5400721854. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:20:38.937783 (+   101us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1462455680}
I20260709 13:20:38.938894 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31969392ccdb43b491911a319e8bc7e1" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:38.939545 16297 raft_consensus.cc:2468] T 31969392ccdb43b491911a319e8bc7e1 P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 8.
I20260709 13:20:38.939798 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57a744f5aaf944678a63469cbf514015" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:20:38.940491 16293 raft_consensus.cc:2468] T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 8.
I20260709 13:20:38.939208 19938 raft_consensus.cc:515] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:38.942518 19938 leader_election.cc:290] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:38.943502 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26354361e9b044cea1c2b86c4882fe92" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:38.947129 16816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5718c623ebee4d2bb770f0f9ddb2f164" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:38.947247 16291 raft_consensus.cc:1275] T d16dfad0c3b540e5b4b86492ddf87208 P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:20:38.948788 16816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5d59407eb545b895d8d922e0dd69fd" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:20:38.949002 16824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:38.949414 16298 raft_consensus.cc:2468] T 57a744f5aaf944678a63469cbf514015 P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 9.
I20260709 13:20:38.952495 16816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87147731a68400e8cce9b879daa7c5d" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:20:38.952803 16824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "829b4ad8c5a54eababeea1181b960d4b" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:38.953496 16824 raft_consensus.cc:2393] T 829b4ad8c5a54eababeea1181b960d4b P cb82cc7bd6db46d980c75237a1402860 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 10: Already voted for candidate cb82cc7bd6db46d980c75237a1402860 in this term.
I20260709 13:20:38.954211 16825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5718c623ebee4d2bb770f0f9ddb2f164" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:20:38.955955 16297 raft_consensus.cc:1275] T 57a744f5aaf944678a63469cbf514015 P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:20:38.957310 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434bff3082d141838068f323bdb0d1ed" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:38.959524 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e65676b21b748728ad39668cfe722ec" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:38.960597 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f2da98864b4425bf7c0055951fdd4b" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:38.961390 16300 raft_consensus.cc:2410] T 84f2da98864b4425bf7c0055951fdd4b P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b for term 9 because replica has last-logged OpId of term: 7 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 13:20:38.962142 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:38.962703 16300 raft_consensus.cc:2468] T ecc1e363230043fe9fef390f0b979f20 P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 8.
I20260709 13:20:38.964010 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ad5dbfa99ee4b8d9e28be92b3f36db5" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:38.964607 16293 raft_consensus.cc:2468] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 8.
I20260709 13:20:38.965730 16293 raft_consensus.cc:1275] T e4dc6b0541ca431ebc757beac00521f1 P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:20:38.966698 16295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66ae176961b74813a58bc15c9c4c6f2a" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:20:38.967288 16295 raft_consensus.cc:3060] T 66ae176961b74813a58bc15c9c4c6f2a P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:20:38.970611 16291 raft_consensus.cc:1275] T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:20:38.978534 16293 raft_consensus.cc:1240] T 68a6838d82ef4cac83507f906139d90f P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 10. Current term is 11. Ops: [10.6-10.6]
W20260709 13:20:38.944187 19939 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 2.235s	user 0.009s	sys 0.029s
I20260709 13:20:38.984557 16823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434bff3082d141838068f323bdb0d1ed" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:38.979638 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c61a39fd4f84424b129c1a6f9ba404d" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:38.982589 16291 raft_consensus.cc:1275] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 13:20:38.977833 16298 raft_consensus.cc:1275] T 8d4ab870a05347868956aa8b841b2981 P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 13:20:38.987929 16294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87147731a68400e8cce9b879daa7c5d" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:20:38.988521 16294 raft_consensus.cc:3060] T b87147731a68400e8cce9b879daa7c5d P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:20:38.991292 16291 raft_consensus.cc:1240] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 9. Current term is 10. Ops: []
I20260709 13:20:39.039863 16817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2794af5b99984efebdee564035841cb0" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:38.950693 20073 leader_election.cc:290] T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:38.980065 20058 raft_consensus.cc:493] T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:39.061043 16291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4f65cdf00b748bb963ee6e81ef43b79" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 12 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:39.062992 16298 raft_consensus.cc:1275] T 1df93779209748b9860f6d893dd3de53 P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:20:39.064659 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c74490c40243a88e62d1760b5cd814" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:39.064719 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fffab26427e24d398e2695aad40ca134" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 9 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:39.066778 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2794af5b99984efebdee564035841cb0" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:38.977731 20080 leader_election.cc:290] T 18ad7e3af5504571a69d10c00750a01c P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:20:39.070261 16291 raft_consensus.cc:2410] T c4f65cdf00b748bb963ee6e81ef43b79 P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b for term 12 because replica has last-logged OpId of term: 11 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 6.
I20260709 13:20:39.070500 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a878999c0024d2abe517484180290fc" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:39.076565 16295 raft_consensus.cc:2468] T 66ae176961b74813a58bc15c9c4c6f2a P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 12.
I20260709 13:20:39.077260 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a378d7f796f94615b3b347119a9880fd" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:38.976642 20060 consensus_queue.cc:1048] T 8638ee363273426db1393dd81e5c7b72 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.086s
I20260709 13:20:39.078318 16298 raft_consensus.cc:2468] T a378d7f796f94615b3b347119a9880fd P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 9.
I20260709 13:20:39.082762 16294 raft_consensus.cc:2468] T b87147731a68400e8cce9b879daa7c5d P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 11.
I20260709 13:20:38.977579 19904 consensus_queue.cc:1059] T c9944d5e4b8c4109af9955c4f08c9095 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.085s
I20260709 13:20:39.074824 16789 leader_election.cc:304] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:20:39.084996 20095 leader_election.cc:290] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:39.087961 20095 raft_consensus.cc:493] T ada5a004ba7c46819051e3ccec71ffdb P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:39.088467 20095 raft_consensus.cc:515] T ada5a004ba7c46819051e3ccec71ffdb P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:39.089355 16115 leader_election.cc:304] T 434bff3082d141838068f323bdb0d1ed P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860
I20260709 13:20:39.090451 20095 leader_election.cc:290] T ada5a004ba7c46819051e3ccec71ffdb P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:39.091986 20094 raft_consensus.cc:2749] T 434bff3082d141838068f323bdb0d1ed P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:20:39.090814 19726 consensus_queue.cc:1048] T d16dfad0c3b540e5b4b86492ddf87208 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.004s
I20260709 13:20:39.107843 16789 leader_election.cc:304] T 74ca8140f35c4ae29e8a664f6bdfc829 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:20:39.112782 20136 raft_consensus.cc:2804] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:20:39.128136 20136 raft_consensus.cc:493] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:39.128547 20136 raft_consensus.cc:3060] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:20:39.121752 19904 raft_consensus.cc:2804] T 74ca8140f35c4ae29e8a664f6bdfc829 P cb82cc7bd6db46d980c75237a1402860 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:20:39.131642 19904 raft_consensus.cc:493] T 74ca8140f35c4ae29e8a664f6bdfc829 P cb82cc7bd6db46d980c75237a1402860 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:39.132030 19904 raft_consensus.cc:3060] T 74ca8140f35c4ae29e8a664f6bdfc829 P cb82cc7bd6db46d980c75237a1402860 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:20:39.134155 20119 log.cc:927] Time spent T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 1.196s	user 0.000s	sys 0.002s
I20260709 13:20:39.074676 20058 raft_consensus.cc:515] T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:39.098311 20140 consensus_queue.cc:1059] T 18ad7e3af5504571a69d10c00750a01c P 647a343a542d4a139fe106c19095129b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.037s
W20260709 13:20:39.128159 20158 log.cc:927] Time spent T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860: Append to log took a long time: real 0.087s	user 0.000s	sys 0.001s
I20260709 13:20:39.140753 16818 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49896 (request call id 1085) took 3536 ms. Trace:
I20260709 13:20:39.145071 16818 rpcz_store.cc:276] 0709 13:20:35.604720 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:35.604870 (+   150us) service_pool.cc:224] Handling call
0709 13:20:35.653942 (+ 49072us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 13:20:36.803383 (+1149441us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:39.140469 (+2337086us) spinlock_profiling.cc:243] Waited 1.14 s on lock 0x7b5400787154. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:20:39.140696 (+   227us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2610688256}
I20260709 13:20:39.142184 20140 raft_consensus.cc:3055] T 18ad7e3af5504571a69d10c00750a01c P 647a343a542d4a139fe106c19095129b [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:20:39.143908 20058 leader_election.cc:290] T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:20:39.144450 16151 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40844 (request call id 2233) took 2342 ms. Trace:
I20260709 13:20:39.153409 16151 rpcz_store.cc:276] 0709 13:20:36.802138 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:36.988240 (+186102us) service_pool.cc:224] Handling call
0709 13:20:36.988572 (+   332us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:20:37.142549 (+153977us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:39.144184 (+2001635us) spinlock_profiling.cc:243] Waited 104 ms on lock 0x7b54000d1954. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:20:39.144363 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":238912768}
I20260709 13:20:39.144649 16293 raft_consensus.cc:2410] T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b for term 10 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
W20260709 13:20:39.149997 16190 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:39.154332 19822 raft_consensus.cc:493] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:39.074891 20161 consensus_queue.cc:1059] T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.106s
I20260709 13:20:39.153220 20140 raft_consensus.cc:740] T 18ad7e3af5504571a69d10c00750a01c P 647a343a542d4a139fe106c19095129b [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:20:39.186420 20161 consensus_queue.cc:1243] T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Peer 647a343a542d4a139fe106c19095129b log is divergent from this leader: its last log entry 11.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260709 13:20:39.198825 20140 consensus_queue.cc:260] T 18ad7e3af5504571a69d10c00750a01c P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } }
I20260709 13:20:39.156477 19822 raft_consensus.cc:515] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:20:39.211529 16292 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40474 (request call id 612) took 2779 ms (client timeout 1738 ms). Trace:
W20260709 13:20:39.212004 16292 rpcz_store.cc:269] 0709 13:20:36.426331 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:36.426535 (+   204us) service_pool.cc:224] Handling call
0709 13:20:39.205120 (+2778585us) spinlock_profiling.cc:243] Waited 264 ms on lock 0x7b54000a9e54. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:20:39.205236 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":607121280}
W20260709 13:20:39.224756 16789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 109729us
I20260709 13:20:39.226450 16789 leader_election.cc:304] T 9e65676b21b748728ad39668cfe722ec P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:20:39.227797 16789 leader_election.cc:304] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:20:39.229573 16292 raft_consensus.cc:3055] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:20:39.229929 16292 raft_consensus.cc:740] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:20:39.230616 16292 consensus_queue.cc:260] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:39.231546 16292 raft_consensus.cc:3060] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:20:39.235275 19990 raft_consensus.cc:3060] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:20:39.239229 16292 raft_consensus.cc:1275] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:20:39.242117 16292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ab0ac8347641e6955e9bce7d90c1a2" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:39.242789 16292 raft_consensus.cc:2468] T d2ab0ac8347641e6955e9bce7d90c1a2 P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 9.
I20260709 13:20:39.204720 20140 raft_consensus.cc:3060] T 18ad7e3af5504571a69d10c00750a01c P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:20:39.249457 20020 consensus_queue.cc:1048] T 57a744f5aaf944678a63469cbf514015 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.003s
W20260709 13:20:39.248740 20155 log.cc:927] Time spent T 26354361e9b044cea1c2b86c4882fe92 P cb82cc7bd6db46d980c75237a1402860: Append to log took a long time: real 0.207s	user 0.000s	sys 0.001s
I20260709 13:20:39.224817 19575 leader_election.cc:290] T 84f2da98864b4425bf7c0055951fdd4b P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:39.248598 19885 raft_consensus.cc:2749] T 9e65676b21b748728ad39668cfe722ec P cb82cc7bd6db46d980c75237a1402860 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:20:39.250298 20073 raft_consensus.cc:2804] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:20:39.262929 16297 raft_consensus.cc:1275] T cdbd78acb4cf46298e0de8ba0d130e19 P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:20:39.310912 16217 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33050 (request call id 660) took 1079 ms. Trace:
I20260709 13:20:39.311504 16217 rpcz_store.cc:276] 0709 13:20:38.231034 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:38.620311 (+389277us) service_pool.cc:224] Handling call
0709 13:20:39.310634 (+690323us) spinlock_profiling.cc:243] Waited 80.1 ms on lock 0x7b5400245c54. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:20:39.310826 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":184236416}
I20260709 13:20:39.315762 16299 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48674 (request call id 2539) took 2672 ms. Trace:
I20260709 13:20:39.316200 16299 rpcz_store.cc:276] 0709 13:20:36.643788 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:37.203072 (+559284us) service_pool.cc:224] Handling call
0709 13:20:37.393402 (+190330us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:37.393588 (+   186us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:37.393612 (+    24us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:20:37.393651 (+    39us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:20:37.393686 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:37.393792 (+   106us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:39.315558 (+1921766us) spinlock_profiling.cc:243] Waited 190 ms on lock 0x7b5400162e50. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5e8ad 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:20:39.315677 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":436602112}
W20260709 13:20:39.321978 16821 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49892 (request call id 550) took 3226 ms (client timeout 2204 ms). Trace:
W20260709 13:20:39.322398 16821 rpcz_store.cc:269] 0709 13:20:36.095410 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:36.095578 (+   168us) service_pool.cc:224] Handling call
0709 13:20:39.321812 (+3226234us) spinlock_profiling.cc:243] Waited 2.09 s on lock 0x7b54007ffdd4. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:20:39.321911 (+    99us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4805107968}
I20260709 13:20:39.323578 16820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9944d5e4b8c4109af9955c4f08c9095" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:39.326169 19904 raft_consensus.cc:515] T 74ca8140f35c4ae29e8a664f6bdfc829 P cb82cc7bd6db46d980c75237a1402860 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:39.327382 20073 raft_consensus.cc:697] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [term 12 LEADER]: Becoming Leader. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:20:39.327519 16822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746aabf856ab401c9ef5e859371e7b67" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:39.327561 16818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5d59407eb545b895d8d922e0dd69fd" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:39.328760 16816 raft_consensus.cc:2410] T b87147731a68400e8cce9b879daa7c5d P cb82cc7bd6db46d980c75237a1402860 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b for term 11 because replica has last-logged OpId of term: 10 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 8.
I20260709 13:20:39.328358 20073 consensus_queue.cc:237] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:39.329252 16823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 11 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:39.330065 16823 raft_consensus.cc:2468] T 19baa7bc367844c288bca8f81439c413 P cb82cc7bd6db46d980c75237a1402860 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 10.
I20260709 13:20:39.333190 16825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62fe3346cd3c42c4a55e14af9073bca4" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:20:39.335510 19822 leader_election.cc:290] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:39.336584 20084 leader_election.cc:290] T 26354361e9b044cea1c2b86c4882fe92 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:39.337471 16821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8638ee363273426db1393dd81e5c7b72" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:20:39.338901 16818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c11074579b046d99e585c2818e18a2c" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:39.339744 16818 raft_consensus.cc:2468] T 8c11074579b046d99e585c2818e18a2c P cb82cc7bd6db46d980c75237a1402860 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 9.
I20260709 13:20:39.336489 16820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a378d7f796f94615b3b347119a9880fd" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 10 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
W20260709 13:20:39.341830 16115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 252547us
I20260709 13:20:39.344836 16818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ab0ac8347641e6955e9bce7d90c1a2" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:39.345777 16818 raft_consensus.cc:2468] T d2ab0ac8347641e6955e9bce7d90c1a2 P cb82cc7bd6db46d980c75237a1402860 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 9.
I20260709 13:20:39.342947 16115 leader_election.cc:304] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:20:39.346738 16820 raft_consensus.cc:2410] T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac for term 10 because replica has last-logged OpId of term: 9 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
I20260709 13:20:39.347625 16821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee9ed67006144797b1de6ea8c2013a68" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:20:39.348495 16821 raft_consensus.cc:2376] T ee9ed67006144797b1de6ea8c2013a68 P cb82cc7bd6db46d980c75237a1402860 [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 8. Re-sending same reply.
I20260709 13:20:39.351337 16821 raft_consensus.cc:1240] T 829b4ad8c5a54eababeea1181b960d4b P cb82cc7bd6db46d980c75237a1402860 [term 10 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 9. Current term is 10. Ops: []
I20260709 13:20:39.352151 16818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74ca8140f35c4ae29e8a664f6bdfc829" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:20:39.354455 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a378d7f796f94615b3b347119a9880fd" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:39.356177 16820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdbd78acb4cf46298e0de8ba0d130e19" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:39.356935 16224 raft_consensus.cc:2393] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 10: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
I20260709 13:20:39.357208 16818 raft_consensus.cc:2393] T 74ca8140f35c4ae29e8a664f6bdfc829 P cb82cc7bd6db46d980c75237a1402860 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 10: Already voted for candidate cb82cc7bd6db46d980c75237a1402860 in this term.
I20260709 13:20:39.357623 16821 raft_consensus.cc:1275] T ee9ed67006144797b1de6ea8c2013a68 P cb82cc7bd6db46d980c75237a1402860 [term 8 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:20:39.357828 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c2095a5f4ad49408dbc7f69c606120d" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
W20260709 13:20:39.347337 16265 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:39.358523 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f5f89186204242bf377f7c42b2e2da" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:39.359290 16224 raft_consensus.cc:2393] T c2f5f89186204242bf377f7c42b2e2da P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 11: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
I20260709 13:20:39.359823 16823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b7d1ab519044eda2d5e04a07242f25" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:20:39.360227 16789 leader_election.cc:304] T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:20:39.361150 16220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87ac0d1f7e8a419f9bcbc04aee0e067e" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:39.360375 16823 raft_consensus.cc:2376] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 8. Re-sending same reply.
I20260709 13:20:39.362411 16115 leader_election.cc:304] T fffab26427e24d398e2695aad40ca134 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:20:39.363621 16115 leader_election.cc:304] T 2794af5b99984efebdee564035841cb0 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860
I20260709 13:20:39.364655 16115 leader_election.cc:304] T 8a878999c0024d2abe517484180290fc P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:20:39.361742 16220 raft_consensus.cc:2468] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 10.
I20260709 13:20:39.361318 19575 consensus_queue.cc:1048] T b6f07842d3aa47858df5665874c86f3a P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 2, Time since last communication: 0.005s
I20260709 13:20:39.366173 16221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74ca8140f35c4ae29e8a664f6bdfc829" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:39.366820 16221 raft_consensus.cc:2368] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 for earlier term 9. Current term is 10.
I20260709 13:20:39.367383 16223 raft_consensus.cc:1240] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 9. Current term is 10. Ops: [7.7-9.8]
I20260709 13:20:39.367573 20084 raft_consensus.cc:493] T 84f2da98864b4425bf7c0055951fdd4b P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:20:39.368075 20084 raft_consensus.cc:515] T 84f2da98864b4425bf7c0055951fdd4b P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:39.356344 20064 consensus_queue.cc:1048] T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.013s
I20260709 13:20:39.370200 20084 leader_election.cc:290] T 84f2da98864b4425bf7c0055951fdd4b P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:20:39.354988 20107 raft_consensus.cc:2804] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:20:39.371126 20107 raft_consensus.cc:493] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:39.371662 20136 raft_consensus.cc:515] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:39.372466 16218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cc959fb7f704fba9009c0595467d074" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 11 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:39.373304 16218 raft_consensus.cc:2393] T 8cc959fb7f704fba9009c0595467d074 P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 11: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
I20260709 13:20:39.335035 20048 leader_election.cc:290] T 66f75124215f4ca6bef831e174a792d8 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:20:39.371629 20107 raft_consensus.cc:3060] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:20:39.399456 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 110954us
I20260709 13:20:39.392789 19822 raft_consensus.cc:493] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:39.389528 19754 consensus_queue.cc:1059] T 68a6838d82ef4cac83507f906139d90f P 647a343a542d4a139fe106c19095129b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.009s
I20260709 13:20:39.404003 16218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d4ab870a05347868956aa8b841b2981" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 12 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:39.405227 16220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806d7690c2b644959152074bb2cab0a7" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:20:39.405625 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66ae176961b74813a58bc15c9c4c6f2a" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:39.405824 16220 raft_consensus.cc:3060] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:20:39.406270 16225 raft_consensus.cc:2468] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 12.
I20260709 13:20:39.406330 16115 leader_election.cc:304] T b87147731a68400e8cce9b879daa7c5d P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:20:39.407277 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18ad7e3af5504571a69d10c00750a01c" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
W20260709 13:20:39.407725 16789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 47650.1us
I20260709 13:20:39.409895 16218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c61a39fd4f84424b129c1a6f9ba404d" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:39.410681 16218 raft_consensus.cc:2393] T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 10: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
I20260709 13:20:39.411545 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada5a004ba7c46819051e3ccec71ffdb" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:39.415692 16789 leader_election.cc:304] T d2ab0ac8347641e6955e9bce7d90c1a2 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:20:39.409029 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5ad4eb281564936a59f41e387a47156" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:39.420847 16224 raft_consensus.cc:2468] T e5ad4eb281564936a59f41e387a47156 P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 10.
I20260709 13:20:39.394232 20068 consensus_queue.cc:1048] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.009s
I20260709 13:20:39.421058 16221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7ca61592e848ea9b0fb81a2ce33cdc" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:39.421741 16221 raft_consensus.cc:2468] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 12.
I20260709 13:20:39.395000 19915 consensus_queue.cc:1059] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.008s
I20260709 13:20:39.400923 20057 raft_consensus.cc:2749] T fffab26427e24d398e2695aad40ca134 P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:20:39.429713 16223 raft_consensus.cc:1240] T 8a878999c0024d2abe517484180290fc P 1ca2725d290e47d68b1a4b609c9816ac [term 9 LEADER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 8. Current term is 9. Ops: []
I20260709 13:20:39.423342 16220 raft_consensus.cc:2468] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 10.
I20260709 13:20:39.433660 16265 leader_election.cc:304] T d16dfad0c3b540e5b4b86492ddf87208 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:20:39.403357 20084 raft_consensus.cc:493] T cdbd78acb4cf46298e0de8ba0d130e19 P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:39.403149 19822 raft_consensus.cc:515] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:39.402235 19575 raft_consensus.cc:2749] T 2794af5b99984efebdee564035841cb0 P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:20:39.403199 20064 raft_consensus.cc:2749] T 8a878999c0024d2abe517484180290fc P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:20:39.438153 16192 leader_election.cc:304] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860
W20260709 13:20:39.485558 16192 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 47502.1us
I20260709 13:20:39.441089 16220 raft_consensus.cc:1275] T 08c74490c40243a88e62d1760b5cd814 P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 13:20:39.404770 20058 raft_consensus.cc:493] T 5a96373bb0e14dce8a266b2fc6c58507 P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:20:39.431900 19990 raft_consensus.cc:2804] T d2ab0ac8347641e6955e9bce7d90c1a2 P cb82cc7bd6db46d980c75237a1402860 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:20:39.400583 20021 consensus_queue.cc:1048] T 1df93779209748b9860f6d893dd3de53 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.032s
I20260709 13:20:39.451972 19904 leader_election.cc:290] T 74ca8140f35c4ae29e8a664f6bdfc829 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 10 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:39.496820 16823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:39.497540 16823 raft_consensus.cc:2468] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 9.
I20260709 13:20:39.453963 20045 raft_consensus.cc:2804] T b87147731a68400e8cce9b879daa7c5d P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:20:39.457823 20022 raft_consensus.cc:493] T 9e65676b21b748728ad39668cfe722ec P 647a343a542d4a139fe106c19095129b [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:20:39.458417 16149 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40830 (request call id 1361) took 3029 ms. Trace:
I20260709 13:20:39.399828 19789 consensus_queue.cc:1048] T 8d4ab870a05347868956aa8b841b2981 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 2, Time since last communication: 0.030s
I20260709 13:20:39.463064 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74ca8140f35c4ae29e8a664f6bdfc829" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:20:39.501852 16297 raft_consensus.cc:3060] T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Advancing to term 10
I20260709 13:20:39.475505 16821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2794af5b99984efebdee564035841cb0" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:39.499902 16149 rpcz_store.cc:276] 0709 13:20:36.428697 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:36.429494 (+   797us) service_pool.cc:224] Handling call
0709 13:20:36.429879 (+   385us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:20:36.533661 (+103782us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:20:39.149543 (+2615882us) spinlock_profiling.cc:243] Waited 2.45 ms on lock 0x7b1800346600. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b693ade1 00007f69b69475b1 00007f69ada54e07 00007f69b68afab9 00007f69ada7169b 00007f69ada6f93f 00007f69ada6f455 00007f69b45573c7 00007f69b0b699a7 00007f69b0b790bf 00007f69b0b79038
0709 13:20:39.260226 (+110683us) log.cc:844] Serialized 14 byte log entry
0709 13:20:39.422647 (+162421us) log.cc:844] Serialized 14 byte log entry
0709 13:20:39.453535 (+ 30888us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:20:39.454225 (+   690us) log.cc:844] Serialized 52 byte log entry
0709 13:20:39.454418 (+   193us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:20:39.454682 (+   264us) log.cc:844] Serialized 14 byte log entry
0709 13:20:39.454886 (+   204us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:39.455102 (+   216us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:20:39.458211 (+  3109us) raft_consensus.cc:1685] finished
0709 13:20:39.458229 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:39.458362 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":134155,"prepare.queue_time_us":399491,"prepare.run_cpu_time_us":164,"prepare.run_wall_time_us":2741,"raft.queue_time_us":192097,"raft.run_cpu_time_us":410,"raft.run_wall_time_us":24786,"spinlock_wait_cycles":5632896,"thread_start_us":256544,"threads_started":2,"wal-append.queue_time_us":168681}
I20260709 13:20:39.476209 16825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e9b2bf7011f4ae688a79d7bb5a447b4" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:39.481552 20065 raft_consensus.cc:2749] T d16dfad0c3b540e5b4b86492ddf87208 P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:20:39.504459 16825 raft_consensus.cc:2468] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 8.
I20260709 13:20:39.474753 16820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7ca61592e848ea9b0fb81a2ce33cdc" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
W20260709 13:20:39.477275 16266 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:39.474045 16265 leader_election.cc:304] T c0eab25ce4c64e018a041beeda2867db P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:20:39.507498 16265 leader_election.cc:304] T 5a959dbecc2648eb85d1cc550f5a3245 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:20:39.471269 20174 raft_consensus.cc:3055] T 68a6838d82ef4cac83507f906139d90f P 647a343a542d4a139fe106c19095129b [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:20:39.483295 20084 raft_consensus.cc:515] T cdbd78acb4cf46298e0de8ba0d130e19 P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:39.491614 20136 leader_election.cc:290] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 10 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:39.487443 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31969392ccdb43b491911a319e8bc7e1" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:20:39.513469 16298 raft_consensus.cc:3060] T 31969392ccdb43b491911a319e8bc7e1 P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Advancing to term 10
I20260709 13:20:39.517138 16823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26354361e9b044cea1c2b86c4882fe92" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:39.509810 20174 raft_consensus.cc:740] T 68a6838d82ef4cac83507f906139d90f P 647a343a542d4a139fe106c19095129b [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:20:39.489368 16822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66ae176961b74813a58bc15c9c4c6f2a" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:39.432698 19942 consensus_queue.cc:1048] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.048s
I20260709 13:20:39.413278 19941 raft_consensus.cc:2804] T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:20:39.506687 16266 leader_election.cc:304] T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:20:39.494451 19667 raft_consensus.cc:2749] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:20:39.525987 19941 raft_consensus.cc:493] T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:39.505244 20022 raft_consensus.cc:515] T 9e65676b21b748728ad39668cfe722ec P 647a343a542d4a139fe106c19095129b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:39.527261 19941 raft_consensus.cc:3060] T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:20:39.486347 16192 leader_election.cc:304] T 746aabf856ab401c9ef5e859371e7b67 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
I20260709 13:20:39.493975 19990 raft_consensus.cc:493] T d2ab0ac8347641e6955e9bce7d90c1a2 P cb82cc7bd6db46d980c75237a1402860 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:39.527952 19990 raft_consensus.cc:3060] T d2ab0ac8347641e6955e9bce7d90c1a2 P cb82cc7bd6db46d980c75237a1402860 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:20:39.432368 20048 consensus_queue.cc:1048] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.046s
I20260709 13:20:39.532615 20174 consensus_queue.cc:260] T 68a6838d82ef4cac83507f906139d90f P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:39.533951 20174 raft_consensus.cc:3060] T 68a6838d82ef4cac83507f906139d90f P 647a343a542d4a139fe106c19095129b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:20:39.535563 19667 raft_consensus.cc:2749] T 746aabf856ab401c9ef5e859371e7b67 P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:20:39.535772 16192 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 49518.9us
I20260709 13:20:39.538208 19990 raft_consensus.cc:515] T d2ab0ac8347641e6955e9bce7d90c1a2 P cb82cc7bd6db46d980c75237a1402860 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:39.540063 16192 leader_election.cc:304] T 19baa7bc367844c288bca8f81439c413 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:20:39.519675 16297 raft_consensus.cc:2468] T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 10.
I20260709 13:20:39.520474 16298 raft_consensus.cc:2468] T 31969392ccdb43b491911a319e8bc7e1 P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 10.
W20260709 13:20:39.523504 16115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 117254us
I20260709 13:20:39.511899 19915 consensus_queue.cc:1243] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b [LEADER]: Peer 90f3da57796b488c9087a2f523adf045 log is divergent from this leader: its last log entry 7.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 13:20:39.542958 19990 leader_election.cc:290] T d2ab0ac8347641e6955e9bce7d90c1a2 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 10 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:39.546859 16192 leader_election.cc:304] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
I20260709 13:20:39.547245 16115 leader_election.cc:304] T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:20:39.554646 16822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada5a004ba7c46819051e3ccec71ffdb" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:39.547875 16266 leader_election.cc:304] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:20:39.555358 16822 raft_consensus.cc:2468] T ada5a004ba7c46819051e3ccec71ffdb P cb82cc7bd6db46d980c75237a1402860 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 10.
I20260709 13:20:39.557231 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c2095a5f4ad49408dbc7f69c606120d" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:39.557690 16295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 11 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:39.558091 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5ad4eb281564936a59f41e387a47156" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:39.558604 16295 raft_consensus.cc:2410] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac for term 11 because replica has last-logged OpId of term: 9 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 7.
I20260709 13:20:39.512557 20045 raft_consensus.cc:697] T b87147731a68400e8cce9b879daa7c5d P 647a343a542d4a139fe106c19095129b [term 11 LEADER]: Becoming Leader. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:20:39.513576 20095 raft_consensus.cc:2804] T c0eab25ce4c64e018a041beeda2867db P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:20:39.496912 20058 raft_consensus.cc:515] T 5a96373bb0e14dce8a266b2fc6c58507 P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:39.512771 20069 raft_consensus.cc:2749] T 5a959dbecc2648eb85d1cc550f5a3245 P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:20:39.548657 16823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a9de347c711477e87c582d003f36925" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
W20260709 13:20:39.564218 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 55561.5us
I20260709 13:20:39.572237 16823 raft_consensus.cc:2393] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 12: Already voted for candidate cb82cc7bd6db46d980c75237a1402860 in this term.
I20260709 13:20:39.572342 16265 leader_election.cc:304] T c9944d5e4b8c4109af9955c4f08c9095 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: cb82cc7bd6db46d980c75237a1402860
W20260709 13:20:39.578999 16114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:39.559248 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8158bbd17b994d2eb96f560bd0f1b88d" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:20:39.589610 16298 raft_consensus.cc:3060] T 8158bbd17b994d2eb96f560bd0f1b88d P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:20:39.560799 16192 leader_election.cc:304] T 8c11074579b046d99e585c2818e18a2c P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:20:39.563997 16295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f2da98864b4425bf7c0055951fdd4b" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:39.592787 16295 raft_consensus.cc:2468] T 84f2da98864b4425bf7c0055951fdd4b P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 7.
I20260709 13:20:39.593771 16295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c61a39fd4f84424b129c1a6f9ba404d" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:20:39.564641 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a96373bb0e14dce8a266b2fc6c58507" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 10 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:20:39.550892 19667 raft_consensus.cc:2804] T 19baa7bc367844c288bca8f81439c413 P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:20:39.549715 20088 raft_consensus.cc:2764] T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Leader pre-election decision vote started in defunct term 8: won
I20260709 13:20:39.596743 16295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c11074579b046d99e585c2818e18a2c" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:39.599821 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8638ee363273426db1393dd81e5c7b72" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:20:39.606549 16299 raft_consensus.cc:3060] T 5a96373bb0e14dce8a266b2fc6c58507 P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:20:39.607650 16790 leader_election.cc:304] T 19baa7bc367844c288bca8f81439c413 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860; no voters: 90f3da57796b488c9087a2f523adf045
I20260709 13:20:39.561664 19856 raft_consensus.cc:2749] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:20:39.562103 20095 raft_consensus.cc:493] T c0eab25ce4c64e018a041beeda2867db P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:20:39.613138 20095 raft_consensus.cc:3060] T c0eab25ce4c64e018a041beeda2867db P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:20:39.593253 19915 raft_consensus.cc:2749] T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:20:39.603175 19667 raft_consensus.cc:493] T 19baa7bc367844c288bca8f81439c413 P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:39.620321 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ab0ac8347641e6955e9bce7d90c1a2" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:39.620922 16298 raft_consensus.cc:2468] T 8158bbd17b994d2eb96f560bd0f1b88d P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 9.
I20260709 13:20:39.621191 16300 raft_consensus.cc:2468] T d2ab0ac8347641e6955e9bce7d90c1a2 P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 9.
I20260709 13:20:39.606019 20069 raft_consensus.cc:2804] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 13:20:39.624118 16115 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:39.621910 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d16dfad0c3b540e5b4b86492ddf87208" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:20:39.625999 16298 raft_consensus.cc:3060] T d16dfad0c3b540e5b4b86492ddf87208 P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:20:39.629843 16145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a9de347c711477e87c582d003f36925" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:20:39.618984 19990 raft_consensus.cc:2804] T 19baa7bc367844c288bca8f81439c413 P cb82cc7bd6db46d980c75237a1402860 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:20:39.638752 16298 raft_consensus.cc:2468] T d16dfad0c3b540e5b4b86492ddf87208 P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 10.
I20260709 13:20:39.641958 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cc959fb7f704fba9009c0595467d074" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:20:39.643386 16298 raft_consensus.cc:3060] T 8cc959fb7f704fba9009c0595467d074 P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:20:39.644842 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0eab25ce4c64e018a041beeda2867db" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:39.652972 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d16dfad0c3b540e5b4b86492ddf87208" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:20:39.654521 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cc959fb7f704fba9009c0595467d074" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:20:39.653169 16151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c61a39fd4f84424b129c1a6f9ba404d" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:20:39.656340 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a959dbecc2648eb85d1cc550f5a3245" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:20:39.657253 16148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4dc6b0541ca431ebc757beac00521f1" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:20:39.657830 16151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6f07842d3aa47858df5665874c86f3a" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:39.659755 16151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c61a39fd4f84424b129c1a6f9ba404d" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:39.660123 16299 raft_consensus.cc:2468] T 5a96373bb0e14dce8a266b2fc6c58507 P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 10.
I20260709 13:20:39.623420 19667 raft_consensus.cc:3060] T 19baa7bc367844c288bca8f81439c413 P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:20:39.661274 16192 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 100508us
I20260709 13:20:39.661401 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31969392ccdb43b491911a319e8bc7e1" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:39.661567 16143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0eab25ce4c64e018a041beeda2867db" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
W20260709 13:20:39.662416 16190 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:39.623348 20080 raft_consensus.cc:2804] T c9944d5e4b8c4109af9955c4f08c9095 P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:20:39.664554 16192 leader_election.cc:304] T d2ab0ac8347641e6955e9bce7d90c1a2 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:20:39.666903 16151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ad5dbfa99ee4b8d9e28be92b3f36db5" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:20:39.625751 20069 raft_consensus.cc:493] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Starting leader election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:20:39.670969 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4dc6b0541ca431ebc757beac00521f1" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
W20260709 13:20:39.671434 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 99176.2us
I20260709 13:20:39.672043 16300 raft_consensus.cc:3060] T e4dc6b0541ca431ebc757beac00521f1 P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:20:39.668700 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a959dbecc2648eb85d1cc550f5a3245" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:20:39.676826 16142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fffab26427e24d398e2695aad40ca134" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:39.677736 16142 raft_consensus.cc:2468] T fffab26427e24d398e2695aad40ca134 P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 8.
I20260709 13:20:39.658008 19789 raft_consensus.cc:493] T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:39.682255 16145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f5f89186204242bf377f7c42b2e2da" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:20:39.657605 19822 leader_election.cc:290] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:39.653716 19990 raft_consensus.cc:493] T 19baa7bc367844c288bca8f81439c413 P cb82cc7bd6db46d980c75237a1402860 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:39.672621 20069 raft_consensus.cc:3060] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:20:39.690043 16148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26354361e9b044cea1c2b86c4882fe92" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
W20260709 13:20:39.691753 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 79851.4us
I20260709 13:20:39.694937 16295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f5f89186204242bf377f7c42b2e2da" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:20:39.697016 16295 raft_consensus.cc:3060] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:20:39.675961 20080 raft_consensus.cc:697] T c9944d5e4b8c4109af9955c4f08c9095 P 90f3da57796b488c9087a2f523adf045 [term 10 LEADER]: Becoming Leader. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:20:39.687759 19992 leader_election.cc:290] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 12 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:39.700947 16148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2372981006d4b218215c28964dea623" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:39.704866 16148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "829b4ad8c5a54eababeea1181b960d4b" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:20:39.706018 16148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66f75124215f4ca6bef831e174a792d8" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:39.687911 19789 raft_consensus.cc:515] T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:39.707406 16149 raft_consensus.cc:2468] T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 9.
I20260709 13:20:39.708669 19789 leader_election.cc:290] T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:39.708747 20184 raft_consensus.cc:2804] T d2ab0ac8347641e6955e9bce7d90c1a2 P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:20:39.709390 20184 raft_consensus.cc:493] T d2ab0ac8347641e6955e9bce7d90c1a2 P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:39.709637 16149 raft_consensus.cc:1275] T ee9ed67006144797b1de6ea8c2013a68 P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:20:39.709762 20184 raft_consensus.cc:3060] T d2ab0ac8347641e6955e9bce7d90c1a2 P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:20:39.712433 16148 raft_consensus.cc:1275] T 8a878999c0024d2abe517484180290fc P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 13:20:39.712910 19939 consensus_queue.cc:1059] T 829b4ad8c5a54eababeea1181b960d4b P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:20:39.713725 16148 raft_consensus.cc:1240] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b [term 10 FOLLOWER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 9. Current term is 10. Ops: []
I20260709 13:20:39.716163 20041 raft_consensus.cc:493] T 8c11074579b046d99e585c2818e18a2c P cb82cc7bd6db46d980c75237a1402860 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:20:39.716671 20041 raft_consensus.cc:3060] T 8c11074579b046d99e585c2818e18a2c P cb82cc7bd6db46d980c75237a1402860 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:20:39.717617 16148 raft_consensus.cc:1275] T 57a744f5aaf944678a63469cbf514015 P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:20:39.701298 16300 raft_consensus.cc:2468] T e4dc6b0541ca431ebc757beac00521f1 P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 10.
W20260709 13:20:39.706614 20176 log.cc:927] Time spent T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860: Append to log took a long time: real 0.090s	user 0.000s	sys 0.000s
I20260709 13:20:39.719795 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ab0ac8347641e6955e9bce7d90c1a2" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:20:39.702780 20080 consensus_queue.cc:237] T c9944d5e4b8c4109af9955c4f08c9095 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:39.720340 16300 raft_consensus.cc:3060] T d2ab0ac8347641e6955e9bce7d90c1a2 P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:20:39.686472 19990 raft_consensus.cc:3060] T 19baa7bc367844c288bca8f81439c413 P cb82cc7bd6db46d980c75237a1402860 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:20:39.701805 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74ca8140f35c4ae29e8a664f6bdfc829" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:20:39.724869 20181 raft_consensus.cc:3055] T 829b4ad8c5a54eababeea1181b960d4b P 1ca2725d290e47d68b1a4b609c9816ac [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:20:39.725330 20181 raft_consensus.cc:740] T 829b4ad8c5a54eababeea1181b960d4b P 1ca2725d290e47d68b1a4b609c9816ac [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:20:39.726226 20181 consensus_queue.cc:260] T 829b4ad8c5a54eababeea1181b960d4b P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:39.691510 19822 raft_consensus.cc:2804] T 8c11074579b046d99e585c2818e18a2c P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:20:39.727415 19822 raft_consensus.cc:493] T 8c11074579b046d99e585c2818e18a2c P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: Starting leader election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:20:39.727850 19822 raft_consensus.cc:3060] T 8c11074579b046d99e585c2818e18a2c P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:20:39.729668 16790 leader_election.cc:304] T 26354361e9b044cea1c2b86c4882fe92 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:20:39.731549 20084 leader_election.cc:290] T cdbd78acb4cf46298e0de8ba0d130e19 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:39.732381 20136 raft_consensus.cc:2749] T 26354361e9b044cea1c2b86c4882fe92 P cb82cc7bd6db46d980c75237a1402860 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:20:39.732609 20107 raft_consensus.cc:515] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:39.651693 20045 consensus_queue.cc:237] T b87147731a68400e8cce9b879daa7c5d P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:39.734190 16297 raft_consensus.cc:2393] T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 10: Already voted for candidate cb82cc7bd6db46d980c75237a1402860 in this term.
I20260709 13:20:39.734990 16222 raft_consensus.cc:1240] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 9. Current term is 10. Ops: []
I20260709 13:20:39.736748 16222 raft_consensus.cc:1240] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 10. Current term is 11. Ops: []
I20260709 13:20:39.739711 16220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "829b4ad8c5a54eababeea1181b960d4b" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
W20260709 13:20:39.740302 16265 connection.cc:256] RPC call timeout handler was delayed by -1.252630070999885s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1767s for a response
W20260709 13:20:39.740731 16265 connection.cc:256] RPC call timeout handler was delayed by -1.1446300709999377s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1887s for a response
W20260709 13:20:39.741029 16265 connection.cc:256] RPC call timeout handler was delayed by -1.0042300709999381s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20430000000000004s for a response
I20260709 13:20:39.741657 16265 leader_election.cc:304] T 68a6838d82ef4cac83507f906139d90f P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:20:39.742370 16218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2372981006d4b218215c28964dea623" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:39.742748 16298 raft_consensus.cc:2468] T 8cc959fb7f704fba9009c0595467d074 P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 11.
I20260709 13:20:39.743156 16218 raft_consensus.cc:2393] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 in current term 10: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
I20260709 13:20:39.746658 20096 leader_election.cc:290] T 5a96373bb0e14dce8a266b2fc6c58507 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 10 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:39.747035 16265 leader_election.cc:304] T c4f65cdf00b748bb963ee6e81ef43b79 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac
I20260709 13:20:39.749022 16265 leader_election.cc:304] T ecc1e363230043fe9fef390f0b979f20 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:20:39.749132 16147 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40844 (request call id 2237) took 2921 ms. Trace:
I20260709 13:20:39.749666 16147 rpcz_store.cc:276] 0709 13:20:36.828130 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:38.599031 (+1770901us) service_pool.cc:224] Handling call
0709 13:20:38.604263 (+  5232us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:38.604556 (+   293us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:20:38.645046 (+ 40490us) log.cc:844] Serialized 14 byte log entry
0709 13:20:39.736656 (+1091610us) spinlock_profiling.cc:243] Waited 9.17 ms on lock 0x7b18003f7de0. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b693ade1 00007f69b69475b1 00007f69b07074bc 00007f69ada5548b 00007f69b68afab9 00007f69ada7169b 00007f69ada6f93f 00007f69ada71c02 00007f69b06f920b 00007f69b06fed23 00007f69b06ff23a
0709 13:20:39.748709 (+ 12053us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:20:39.748781 (+    72us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:20:39.748840 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:39.748925 (+    85us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:39.749067 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":151994,"prepare.run_cpu_time_us":65,"prepare.run_wall_time_us":451,"spinlock_wait_cycles":21098752,"thread_start_us":155848,"threads_started":2,"wal-append.queue_time_us":333952}
I20260709 13:20:39.757239 16147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31969392ccdb43b491911a319e8bc7e1" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:20:39.757869 16147 raft_consensus.cc:3060] T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:20:39.764129 16192 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 94919.4us
I20260709 13:20:39.784885 16149 raft_consensus.cc:1240] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 10. Current term is 11. Ops: []
I20260709 13:20:39.733285 20136 consensus_queue.cc:1059] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: INVALID_TERM, Last received: 9.10, Next index: 11, Last known committed idx: 7, Time since last communication: 0.032s
I20260709 13:20:39.765925 16223 raft_consensus.cc:1240] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 10. Current term is 11. Ops: []
I20260709 13:20:39.767668 20183 raft_consensus.cc:3060] T ecc1e363230043fe9fef390f0b979f20 P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:20:39.763790 20084 raft_consensus.cc:2804] T c4f65cdf00b748bb963ee6e81ef43b79 P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:20:39.761857 20065 raft_consensus.cc:2749] T 68a6838d82ef4cac83507f906139d90f P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:20:39.773295 19809 raft_consensus.cc:3055] T c9944d5e4b8c4109af9955c4f08c9095 P cb82cc7bd6db46d980c75237a1402860 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:20:39.775601 20096 raft_consensus.cc:493] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:20:39.778079 16144 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40844 (request call id 2260) took 1518 ms (client timeout 1708 ms). Trace:
W20260709 13:20:39.780181 16150 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40844 (request call id 2253) took 1945 ms (client timeout 1849 ms). Trace:
I20260709 13:20:39.780617 20184 raft_consensus.cc:515] T d2ab0ac8347641e6955e9bce7d90c1a2 P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:39.782174 16226 raft_consensus.cc:1275] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:20:39.732378 20058 leader_election.cc:290] T 5a96373bb0e14dce8a266b2fc6c58507 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:20:39.764278 16151 raft_consensus.cc:2393] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 9: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
I20260709 13:20:39.786115 16225 raft_consensus.cc:1275] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 8 index: 9. (term mismatch)
I20260709 13:20:39.787209 16192 leader_election.cc:304] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860
I20260709 13:20:39.787086 16789 leader_election.cc:304] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:20:39.791488 16300 raft_consensus.cc:2468] T d2ab0ac8347641e6955e9bce7d90c1a2 P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 10.
I20260709 13:20:39.747865 19904 consensus_queue.cc:1048] T 57a744f5aaf944678a63469cbf514015 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.044s
I20260709 13:20:39.789948 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a9de347c711477e87c582d003f36925" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:20:39.795728 20084 raft_consensus.cc:493] T c4f65cdf00b748bb963ee6e81ef43b79 P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:20:39.795253 16295 raft_consensus.cc:2468] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 11.
W20260709 13:20:39.803099 16150 rpcz_store.cc:269] 0709 13:20:37.834697 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:38.161509 (+326812us) service_pool.cc:224] Handling call
0709 13:20:39.779957 (+1618448us) spinlock_profiling.cc:243] Waited 62.3 ms on lock 0x7b54000d0f54. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:20:39.780095 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":143255808}
I20260709 13:20:39.814662 16295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fffab26427e24d398e2695aad40ca134" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
W20260709 13:20:39.803314 16144 rpcz_store.cc:269] 0709 13:20:38.260043 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:38.339451 (+ 79408us) service_pool.cc:224] Handling call
0709 13:20:39.777800 (+1438349us) spinlock_profiling.cc:243] Waited 86.8 ms on lock 0x7b540012f7d4. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:20:39.778002 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":199565568}
I20260709 13:20:39.815761 16223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a378d7f796f94615b3b347119a9880fd" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:39.816108 20084 raft_consensus.cc:3060] T c4f65cdf00b748bb963ee6e81ef43b79 P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:20:39.804041 19785 raft_consensus.cc:3055] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:20:39.816820 19785 raft_consensus.cc:740] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:20:39.817694 19785 consensus_queue.cc:260] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 9.10, Last appended by leader: 10, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:39.818991 19785 raft_consensus.cc:3060] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:20:39.819396 20187 consensus_queue.cc:1048] T ee9ed67006144797b1de6ea8c2013a68 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:20:39.804970 16225 pending_rounds.cc:85] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac: Aborting all ops after (but not including) 8
I20260709 13:20:39.820119 16225 pending_rounds.cc:107] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac: Aborting uncommitted NO_OP operation due to leader change: 9.9
I20260709 13:20:39.820360 16225 raft_consensus.cc:2889] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:20:39.803030 19809 raft_consensus.cc:740] T c9944d5e4b8c4109af9955c4f08c9095 P cb82cc7bd6db46d980c75237a1402860 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:20:39.822149 19809 consensus_queue.cc:260] T c9944d5e4b8c4109af9955c4f08c9095 P cb82cc7bd6db46d980c75237a1402860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:39.806797 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ab0ac8347641e6955e9bce7d90c1a2" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:39.806890 16297 raft_consensus.cc:3060] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:20:39.808044 19939 raft_consensus.cc:2749] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:20:39.808148 20096 raft_consensus.cc:515] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:39.827845 20096 leader_election.cc:290] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:39.829854 20069 raft_consensus.cc:515] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:39.809443 16789 leader_election.cc:304] T 74ca8140f35c4ae29e8a664f6bdfc829 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:20:39.832320 16789 leader_election.cc:304] T d2ab0ac8347641e6955e9bce7d90c1a2 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:20:39.800800 16147 raft_consensus.cc:2468] T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 10.
I20260709 13:20:39.833907 19806 raft_consensus.cc:2804] T 74ca8140f35c4ae29e8a664f6bdfc829 P cb82cc7bd6db46d980c75237a1402860 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:20:39.834478 19806 raft_consensus.cc:697] T 74ca8140f35c4ae29e8a664f6bdfc829 P cb82cc7bd6db46d980c75237a1402860 [term 10 LEADER]: Becoming Leader. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:20:39.810422 16265 leader_election.cc:304] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
W20260709 13:20:39.835551 16265 leader_election.cc:336] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 9 election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Timed out: RequestConsensusVote RPC to 127.14.177.129:36741 timed out after 1.571s (SENT)
I20260709 13:20:39.835446 19806 consensus_queue.cc:237] T 74ca8140f35c4ae29e8a664f6bdfc829 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:39.836201 19942 raft_consensus.cc:2804] T d2ab0ac8347641e6955e9bce7d90c1a2 P cb82cc7bd6db46d980c75237a1402860 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:20:39.836597 16265 leader_election.cc:400] T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 9 pre-election: Vote denied by peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943) with higher term. Message: Invalid argument: T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 for earlier term 9. Current term is 10.
I20260709 13:20:39.836246 19667 raft_consensus.cc:515] T 19baa7bc367844c288bca8f81439c413 P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:39.836917 19942 raft_consensus.cc:697] T d2ab0ac8347641e6955e9bce7d90c1a2 P cb82cc7bd6db46d980c75237a1402860 [term 10 LEADER]: Becoming Leader. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:20:39.837953 20065 raft_consensus.cc:2804] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:20:39.837783 19942 consensus_queue.cc:237] T d2ab0ac8347641e6955e9bce7d90c1a2 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:39.838364 20065 raft_consensus.cc:493] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:39.838758 20065 raft_consensus.cc:3060] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:20:39.839294 19809 raft_consensus.cc:3060] T c9944d5e4b8c4109af9955c4f08c9095 P cb82cc7bd6db46d980c75237a1402860 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:20:39.839532 20058 raft_consensus.cc:493] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:39.840049 20058 raft_consensus.cc:515] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:39.842819 20058 leader_election.cc:290] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:39.843607 16265 leader_election.cc:304] T 66ae176961b74813a58bc15c9c4c6f2a P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:20:39.864295 16265 leader_election.cc:304] T e5ad4eb281564936a59f41e387a47156 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:20:39.865116 16265 leader_election.cc:304] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:20:39.865224 19822 raft_consensus.cc:515] T 8c11074579b046d99e585c2818e18a2c P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:39.866222 16190 leader_election.cc:304] T e5ad4eb281564936a59f41e387a47156 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:20:39.866879 20182 raft_consensus.cc:2764] T 66ae176961b74813a58bc15c9c4c6f2a P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Leader pre-election decision vote started in defunct term 11: won
I20260709 13:20:39.867462 19822 leader_election.cc:290] T 8c11074579b046d99e585c2818e18a2c P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 10 election: Requested vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:39.867606 20182 raft_consensus.cc:2804] T e5ad4eb281564936a59f41e387a47156 P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:20:39.867988 20182 raft_consensus.cc:493] T e5ad4eb281564936a59f41e387a47156 P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:20:39.868031 19822 raft_consensus.cc:2749] T e5ad4eb281564936a59f41e387a47156 P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:20:39.868422 20182 raft_consensus.cc:3060] T e5ad4eb281564936a59f41e387a47156 P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:20:39.869655 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6f07842d3aa47858df5665874c86f3a" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:39.848564 16223 raft_consensus.cc:2468] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 10.
I20260709 13:20:39.870268 16298 raft_consensus.cc:2468] T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 11.
I20260709 13:20:39.870836 20088 raft_consensus.cc:2804] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:20:39.847688 20065 raft_consensus.cc:515] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:39.855866 20107 leader_election.cc:290] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 9 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:39.856357 20073 raft_consensus.cc:2804] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:20:39.880069 20073 raft_consensus.cc:697] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860 [term 10 LEADER]: Becoming Leader. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:20:39.880946 20073 consensus_queue.cc:237] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:20:39.881567 20166 log.cc:927] Time spent T 26354361e9b044cea1c2b86c4882fe92 P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 0.513s	user 0.000s	sys 0.003s
I20260709 13:20:39.857887 16297 raft_consensus.cc:2410] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 for term 12 because replica has last-logged OpId of term: 11 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 8.
I20260709 13:20:39.859244 16824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66f75124215f4ca6bef831e174a792d8" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:20:39.860633 16219 raft_consensus.cc:3055] T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:20:39.862111 19990 raft_consensus.cc:515] T 19baa7bc367844c288bca8f81439c413 P cb82cc7bd6db46d980c75237a1402860 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:39.866349 16265 leader_election.cc:304] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:20:39.888408 16824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdbd78acb4cf46298e0de8ba0d130e19" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 11 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:39.888836 16821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a378d7f796f94615b3b347119a9880fd" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:39.869428 19941 raft_consensus.cc:515] T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:39.849995 20095 raft_consensus.cc:515] T c0eab25ce4c64e018a041beeda2867db P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:39.848002 19667 leader_election.cc:290] T 19baa7bc367844c288bca8f81439c413 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 11 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:39.871258 20088 raft_consensus.cc:493] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Starting leader election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:20:39.871140 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdbd78acb4cf46298e0de8ba0d130e19" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:39.872189 16295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 11 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:20:39.891440 16789 leader_election.cc:304] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:20:39.869444 20069 leader_election.cc:290] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 8 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:39.893535 20088 raft_consensus.cc:3060] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:20:39.879043 16190 leader_election.cc:304] T 7c2095a5f4ad49408dbc7f69c606120d P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:20:39.884837 16219 raft_consensus.cc:740] T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:20:39.908478 16190 leader_election.cc:304] T 84f2da98864b4425bf7c0055951fdd4b P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 647a343a542d4a139fe106c19095129b
I20260709 13:20:39.910145 16266 leader_election.cc:304] T ada5a004ba7c46819051e3ccec71ffdb P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac
I20260709 13:20:39.885504 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ad5dbfa99ee4b8d9e28be92b3f36db5" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:20:39.890153 16824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:39.875584 16226 raft_consensus.cc:2393] T d2ab0ac8347641e6955e9bce7d90c1a2 P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 in current term 10: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
I20260709 13:20:39.891844 16821 raft_consensus.cc:2368] T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 for earlier term 10. Current term is 11.
I20260709 13:20:39.891868 16295 raft_consensus.cc:3060] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:20:39.909252 16219 consensus_queue.cc:260] T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 10.5, Last appended by leader: 5, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:39.913496 16219 raft_consensus.cc:3060] T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:20:39.913807 16190 leader_election.cc:304] T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:20:39.892570 16298 raft_consensus.cc:2468] T cdbd78acb4cf46298e0de8ba0d130e19 P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 10.
I20260709 13:20:39.893846 16223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66ae176961b74813a58bc15c9c4c6f2a" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:20:39.895910 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e4bfb9090c8460eb14fa7e0d818e7f1" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 11 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:39.877518 20183 raft_consensus.cc:2749] T ecc1e363230043fe9fef390f0b979f20 P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:20:39.905982 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74ca8140f35c4ae29e8a664f6bdfc829" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:39.916201 16297 raft_consensus.cc:2410] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac for term 11 because replica has last-logged OpId of term: 9 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 6.
I20260709 13:20:39.916676 16225 raft_consensus.cc:2393] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 in current term 10: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
I20260709 13:20:39.891551 20179 raft_consensus.cc:2804] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:20:39.911237 16300 raft_consensus.cc:3060] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:20:39.917886 16298 raft_consensus.cc:1240] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 11. Current term is 12. Ops: []
I20260709 13:20:39.919968 19823 raft_consensus.cc:2804] T 84f2da98864b4425bf7c0055951fdd4b P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:20:39.920395 19823 raft_consensus.cc:493] T 84f2da98864b4425bf7c0055951fdd4b P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:39.920714 19823 raft_consensus.cc:3060] T 84f2da98864b4425bf7c0055951fdd4b P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:20:39.921098 19941 leader_election.cc:290] T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 11 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:39.893548 20058 consensus_queue.cc:1059] T 8a878999c0024d2abe517484180290fc P 90f3da57796b488c9087a2f523adf045 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: INVALID_TERM, Last received: 8.4, Next index: 5, Last known committed idx: 2, Time since last communication: 0.019s
I20260709 13:20:39.921931 16221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66f75124215f4ca6bef831e174a792d8" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:39.922761 16221 raft_consensus.cc:2393] T 66f75124215f4ca6bef831e174a792d8 P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 in current term 10: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
I20260709 13:20:39.887691 20041 raft_consensus.cc:515] T 8c11074579b046d99e585c2818e18a2c P cb82cc7bd6db46d980c75237a1402860 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:39.918957 19667 raft_consensus.cc:2749] T 7c2095a5f4ad49408dbc7f69c606120d P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:20:39.924819 20041 leader_election.cc:290] T 8c11074579b046d99e585c2818e18a2c P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 10 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:39.926167 20065 leader_election.cc:290] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 11 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:20:39.926788 20065 raft_consensus.cc:2804] T ada5a004ba7c46819051e3ccec71ffdb P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:20:39.927140 20065 raft_consensus.cc:493] T ada5a004ba7c46819051e3ccec71ffdb P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:20:39.902410 20184 leader_election.cc:290] T d2ab0ac8347641e6955e9bce7d90c1a2 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 10 election: Requested vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:39.927556 20065 raft_consensus.cc:3060] T ada5a004ba7c46819051e3ccec71ffdb P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:20:39.928478 16225 raft_consensus.cc:1240] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 9. Current term is 10. Ops: [9.7-9.7]
I20260709 13:20:39.929476 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "829b4ad8c5a54eababeea1181b960d4b" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:39.929656 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c74490c40243a88e62d1760b5cd814" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:39.930308 16224 raft_consensus.cc:2468] T 08c74490c40243a88e62d1760b5cd814 P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 9.
I20260709 13:20:39.930562 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5718c623ebee4d2bb770f0f9ddb2f164" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:20:39.931476 16224 raft_consensus.cc:1275] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 13:20:39.932219 20095 leader_election.cc:290] T c0eab25ce4c64e018a041beeda2867db P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 10 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:20:39.932592 16224 raft_consensus.cc:1240] T 18ad7e3af5504571a69d10c00750a01c P 1ca2725d290e47d68b1a4b609c9816ac [term 9 LEADER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 8. Current term is 9. Ops: []
I20260709 13:20:39.933911 16224 raft_consensus.cc:1240] T 8cc959fb7f704fba9009c0595467d074 P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 10. Current term is 11. Ops: []
I20260709 13:20:39.935209 16224 raft_consensus.cc:1240] T 68a6838d82ef4cac83507f906139d90f P 1ca2725d290e47d68b1a4b609c9816ac [term 11 LEADER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 10. Current term is 11. Ops: [10.6-10.6]
I20260709 13:20:39.898587 20072 raft_consensus.cc:2749] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:20:39.935988 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74ca8140f35c4ae29e8a664f6bdfc829" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:20:39.936932 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a378d7f796f94615b3b347119a9880fd" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:20:39.944255 16298 raft_consensus.cc:3060] T a378d7f796f94615b3b347119a9880fd P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Advancing to term 11
I20260709 13:20:39.941247 16223 raft_consensus.cc:2468] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 12.
I20260709 13:20:39.947993 16223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ad5dbfa99ee4b8d9e28be92b3f36db5" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:20:39.948912 16223 raft_consensus.cc:2393] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 9: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
I20260709 13:20:39.942396 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c11074579b046d99e585c2818e18a2c" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:20:39.943535 16824 raft_consensus.cc:2393] T 19baa7bc367844c288bca8f81439c413 P cb82cc7bd6db46d980c75237a1402860 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 11: Already voted for candidate cb82cc7bd6db46d980c75237a1402860 in this term.
I20260709 13:20:39.951814 16225 raft_consensus.cc:2393] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 in current term 10: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
I20260709 13:20:39.953635 16790 leader_election.cc:304] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:20:39.956081 16191 leader_election.cc:304] T fffab26427e24d398e2695aad40ca134 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
I20260709 13:20:39.956544 16190 leader_election.cc:304] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860
I20260709 13:20:39.957149 16295 raft_consensus.cc:2410] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac for term 11 because replica has last-logged OpId of term: 9 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 7.
I20260709 13:20:39.958891 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 11 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:20:39.959009 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ab0ac8347641e6955e9bce7d90c1a2" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:20:39.960160 16297 raft_consensus.cc:2410] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 for term 11 because replica has last-logged OpId of term: 9 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 7.
I20260709 13:20:39.900547 20069 consensus_queue.cc:1048] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.043s
I20260709 13:20:39.963644 16190 leader_election.cc:304] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 647a343a542d4a139fe106c19095129b
I20260709 13:20:39.967716 16221 raft_consensus.cc:1240] T d16dfad0c3b540e5b4b86492ddf87208 P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 9. Current term is 10. Ops: [9.6-9.6]
I20260709 13:20:39.969616 16116 leader_election.cc:304] T ecc1e363230043fe9fef390f0b979f20 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860
I20260709 13:20:39.971140 16224 raft_consensus.cc:1240] T c2f5f89186204242bf377f7c42b2e2da P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 10. Current term is 11. Ops: []
I20260709 13:20:39.987607 16190 leader_election.cc:304] T d16dfad0c3b540e5b4b86492ddf87208 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 647a343a542d4a139fe106c19095129b
I20260709 13:20:39.990543 16190 leader_election.cc:304] T c0eab25ce4c64e018a041beeda2867db P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:20:39.975898 16790 leader_election.cc:304] T 66f75124215f4ca6bef831e174a792d8 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:20:39.984889 16297 raft_consensus.cc:1240] T d16dfad0c3b540e5b4b86492ddf87208 P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 9. Current term is 10. Ops: [9.6-9.6]
I20260709 13:20:39.995759 16219 raft_consensus.cc:1275] T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 10 index: 5. Preceding OpId from leader: term: 9 index: 5. (term mismatch)
I20260709 13:20:39.996166 16219 pending_rounds.cc:85] T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac: Aborting all ops after (but not including) 4
I20260709 13:20:39.996477 16219 pending_rounds.cc:107] T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac: Aborting uncommitted NO_OP operation due to leader change: 10.5
I20260709 13:20:39.996771 16219 raft_consensus.cc:2889] T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:20:39.998132 16150 raft_consensus.cc:1275] T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 5 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:20:40.000198 16142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2372981006d4b218215c28964dea623" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:39.943991 19990 leader_election.cc:290] T 19baa7bc367844c288bca8f81439c413 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 11 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:39.992542 16298 raft_consensus.cc:2468] T a378d7f796f94615b3b347119a9880fd P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 11.
I20260709 13:20:40.005357 16789 leader_election.cc:304] T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:20:39.951516 20179 raft_consensus.cc:697] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045 [term 10 LEADER]: Becoming Leader. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:20:39.957679 19789 raft_consensus.cc:493] T 57a744f5aaf944678a63469cbf514015 P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:39.995522 16300 raft_consensus.cc:2468] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 9.
I20260709 13:20:40.008143 16219 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57604 (request call id 2483) took 2999 ms. Trace:
W20260709 13:20:40.008821 16266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 89423.3us
I20260709 13:20:39.957472 20041 consensus_queue.cc:1059] T 8638ee363273426db1393dd81e5c7b72 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.032s
I20260709 13:20:39.967567 19941 raft_consensus.cc:3060] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:20:39.999326 19939 raft_consensus.cc:2804] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:20:39.963352 19667 raft_consensus.cc:2749] T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:20:40.011936 16296 raft_consensus.cc:1240] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 10. Current term is 11. Ops: [10.8-10.8]
I20260709 13:20:40.002700 19991 raft_consensus.cc:3060] T 66f75124215f4ca6bef831e174a792d8 P cb82cc7bd6db46d980c75237a1402860 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:20:39.971277 20021 raft_consensus.cc:493] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:20:40.006393 19952 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 11 current term: 11 is higher.
I20260709 13:20:40.019695 16824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e9b2bf7011f4ae688a79d7bb5a447b4" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:20:39.971131 20189 raft_consensus.cc:2804] T fffab26427e24d398e2695aad40ca134 P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:20:40.007747 16298 raft_consensus.cc:1240] T e4dc6b0541ca431ebc757beac00521f1 P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 9. Current term is 10. Ops: [7.5-9.6]
I20260709 13:20:39.998451 20136 consensus_queue.cc:1048] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.030s
I20260709 13:20:39.957507 20178 consensus_queue.cc:1059] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 7, Time since last communication: 0.056s
I20260709 13:20:40.007738 20187 raft_consensus.cc:2804] T d16dfad0c3b540e5b4b86492ddf87208 P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:20:40.010942 19822 raft_consensus.cc:2749] T c0eab25ce4c64e018a041beeda2867db P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:20:39.999084 19785 consensus_queue.cc:1048] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.024s
I20260709 13:20:39.967492 20184 raft_consensus.cc:2749] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:20:39.989578 20072 consensus_queue.cc:1059] T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.030s
I20260709 13:20:39.971248 20161 consensus_queue.cc:1048] T ee9ed67006144797b1de6ea8c2013a68 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.049s
I20260709 13:20:40.056999 20179 consensus_queue.cc:237] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.7, Last appended by leader: 7, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } }
I20260709 13:20:39.987380 20096 consensus_queue.cc:1048] T 8a878999c0024d2abe517484180290fc P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 2, Time since last communication: 0.034s
I20260709 13:20:40.023566 20041 consensus_queue.cc:1243] T 8638ee363273426db1393dd81e5c7b72 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Peer 1ca2725d290e47d68b1a4b609c9816ac log is divergent from this leader: its last log entry 9.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 13:20:40.054219 16819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e4bfb9090c8460eb14fa7e0d818e7f1" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 11 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:40.013655 19823 raft_consensus.cc:515] T 84f2da98864b4425bf7c0055951fdd4b P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:40.062110 20088 raft_consensus.cc:515] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:40.011862 20060 raft_consensus.cc:2804] T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:20:40.017243 19789 raft_consensus.cc:515] T 57a744f5aaf944678a63469cbf514015 P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:40.063103 19823 leader_election.cc:290] T 84f2da98864b4425bf7c0055951fdd4b P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 9 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:40.064136 16818 raft_consensus.cc:1240] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P cb82cc7bd6db46d980c75237a1402860 [term 10 LEADER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 9. Current term is 10. Ops: []
I20260709 13:20:40.064795 19789 leader_election.cc:290] T 57a744f5aaf944678a63469cbf514015 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:40.072459 19990 raft_consensus.cc:3055] T 8638ee363273426db1393dd81e5c7b72 P cb82cc7bd6db46d980c75237a1402860 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:20:40.072952 19990 raft_consensus.cc:740] T 8638ee363273426db1393dd81e5c7b72 P cb82cc7bd6db46d980c75237a1402860 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:20:40.073812 19990 consensus_queue.cc:260] T 8638ee363273426db1393dd81e5c7b72 P cb82cc7bd6db46d980c75237a1402860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:40.077116 20065 raft_consensus.cc:515] T ada5a004ba7c46819051e3ccec71ffdb P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:40.078814 20072 raft_consensus.cc:3055] T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:20:40.079241 20072 raft_consensus.cc:740] T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:20:40.080025 20072 consensus_queue.cc:260] T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:40.081194 20072 raft_consensus.cc:3060] T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:20:40.081929 20065 leader_election.cc:290] T ada5a004ba7c46819051e3ccec71ffdb P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 11 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:40.055279 20187 raft_consensus.cc:697] T d16dfad0c3b540e5b4b86492ddf87208 P 1ca2725d290e47d68b1a4b609c9816ac [term 10 LEADER]: Becoming Leader. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:20:40.083660 20088 leader_election.cc:290] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 13 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:40.083652 20060 raft_consensus.cc:697] T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860 [term 11 LEADER]: Becoming Leader. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:20:40.084416 20187 consensus_queue.cc:237] T d16dfad0c3b540e5b4b86492ddf87208 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:40.085480 19991 raft_consensus.cc:2749] T 66f75124215f4ca6bef831e174a792d8 P cb82cc7bd6db46d980c75237a1402860 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:20:40.085405 20060 consensus_queue.cc:237] T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:20:40.087999 20190 log.cc:927] Time spent T 3a7ca61592e848ea9b0fb81a2ce33cdc P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 0.084s	user 0.000s	sys 0.000s
I20260709 13:20:40.063082 16816 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49896 (request call id 1111) took 1111 ms. Trace:
I20260709 13:20:40.090907 16816 rpcz_store.cc:276] 0709 13:20:38.952142 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:38.952274 (+   132us) service_pool.cc:224] Handling call
0709 13:20:40.062913 (+1110639us) spinlock_profiling.cc:243] Waited 375 ms on lock 0x7b54004e0cd4. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:20:40.063028 (+   115us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":863400192}
I20260709 13:20:40.034930 20189 raft_consensus.cc:493] T fffab26427e24d398e2695aad40ca134 P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Starting leader election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:20:40.038527 19939 raft_consensus.cc:697] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac [term 9 LEADER]: Becoming Leader. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:20:40.033295 20021 raft_consensus.cc:515] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:40.075023 19990 raft_consensus.cc:3060] T 8638ee363273426db1393dd81e5c7b72 P cb82cc7bd6db46d980c75237a1402860 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:20:40.095700 20189 raft_consensus.cc:3060] T fffab26427e24d398e2695aad40ca134 P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:20:40.101795 20058 consensus_queue.cc:1243] T 8a878999c0024d2abe517484180290fc P 90f3da57796b488c9087a2f523adf045 [LEADER]: Peer 1ca2725d290e47d68b1a4b609c9816ac log is divergent from this leader: its last log entry 9.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
W20260709 13:20:40.116966 20094 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.024s	user 0.005s	sys 0.000s
W20260709 13:20:40.125422 20094 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.033s	user 0.005s	sys 0.000s
I20260709 13:20:40.140376 20182 raft_consensus.cc:515] T e5ad4eb281564936a59f41e387a47156 P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:40.153282 20182 leader_election.cc:290] T e5ad4eb281564936a59f41e387a47156 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 11 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
W20260709 13:20:40.184506 16115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 190253us
W20260709 13:20:40.187578 16115 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:40.194953 16148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 11 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:20:40.015725 16219 rpcz_store.cc:276] 0709 13:20:36.999655 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:37.260484 (+260829us) service_pool.cc:224] Handling call
0709 13:20:37.260790 (+   306us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:39.860511 (+2599721us) spinlock_profiling.cc:243] Waited 1.73 s on lock 0x7b5400422820. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0ad6dc0 00007f69b0b611c0 00007f69b0b62231 00007f69b0b5ed8f 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c
0709 13:20:39.999340 (+138829us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:39.999520 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3975704832}
I20260709 13:20:40.196862 16148 raft_consensus.cc:3060] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:20:40.201160 16150 raft_consensus.cc:1275] T 68a6838d82ef4cac83507f906139d90f P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 6. (term mismatch)
I20260709 13:20:40.201642 16150 pending_rounds.cc:85] T 68a6838d82ef4cac83507f906139d90f P 647a343a542d4a139fe106c19095129b: Aborting all ops after (but not including) 5
I20260709 13:20:40.202010 16150 pending_rounds.cc:107] T 68a6838d82ef4cac83507f906139d90f P 647a343a542d4a139fe106c19095129b: Aborting uncommitted NO_OP operation due to leader change: 10.6
I20260709 13:20:40.202431 16150 raft_consensus.cc:2889] T 68a6838d82ef4cac83507f906139d90f P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:20:40.206189 16789 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:40.208702 16222 raft_consensus.cc:1275] T af57996a8be84d398f0f023e8d08ef77 P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 13:20:40.209362 16219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a378d7f796f94615b3b347119a9880fd" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:20:40.210417 16219 raft_consensus.cc:3060] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:20:40.209159 16223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c11074579b046d99e585c2818e18a2c" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:20:40.211577 16296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c11074579b046d99e585c2818e18a2c" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:20:40.213503 16266 leader_election.cc:400] T a378d7f796f94615b3b347119a9880fd P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 10 pre-election: Vote denied by peer cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923) with higher term. Message: Invalid argument: T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 for earlier term 10. Current term is 11.
I20260709 13:20:40.213758 16296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ab0ac8347641e6955e9bce7d90c1a2" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:20:40.214048 16266 leader_election.cc:403] T a378d7f796f94615b3b347119a9880fd P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 10 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:20:40.211937 16223 raft_consensus.cc:2393] T 8c11074579b046d99e585c2818e18a2c P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 in current term 10: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
W20260709 13:20:40.210421 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 321703us
W20260709 13:20:40.193408 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 222372us
W20260709 13:20:40.246114 19861 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.724s	user 0.005s	sys 0.002s
I20260709 13:20:40.246573 16817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7ca61592e848ea9b0fb81a2ce33cdc" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:20:40.247804 16296 raft_consensus.cc:2393] T d2ab0ac8347641e6955e9bce7d90c1a2 P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 10: Already voted for candidate cb82cc7bd6db46d980c75237a1402860 in this term.
I20260709 13:20:40.248669 19962 raft_consensus.cc:493] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:20:40.250022 19962 raft_consensus.cc:515] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:40.122239 20084 raft_consensus.cc:515] T c4f65cdf00b748bb963ee6e81ef43b79 P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:40.246577 16816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada5a004ba7c46819051e3ccec71ffdb" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:20:40.252976 16816 raft_consensus.cc:3060] T ada5a004ba7c46819051e3ccec71ffdb P cb82cc7bd6db46d980c75237a1402860 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:20:40.246869 19861 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.728s	user 0.005s	sys 0.002s
I20260709 13:20:40.263088 19962 leader_election.cc:290] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:20:40.265754 19726 consensus_queue.cc:1059] T d16dfad0c3b540e5b4b86492ddf87208 P 647a343a542d4a139fe106c19095129b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:20:40.246879 19806 raft_consensus.cc:493] T b87147731a68400e8cce9b879daa7c5d P cb82cc7bd6db46d980c75237a1402860 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:40.268167 19726 raft_consensus.cc:3055] T d16dfad0c3b540e5b4b86492ddf87208 P 647a343a542d4a139fe106c19095129b [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:20:40.268743 19726 raft_consensus.cc:740] T d16dfad0c3b540e5b4b86492ddf87208 P 647a343a542d4a139fe106c19095129b [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:20:40.268656 19806 raft_consensus.cc:515] T b87147731a68400e8cce9b879daa7c5d P cb82cc7bd6db46d980c75237a1402860 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:40.270289 19726 consensus_queue.cc:260] T d16dfad0c3b540e5b4b86492ddf87208 P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:40.273377 20021 leader_election.cc:290] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:40.254932 19958 leader_election.cc:290] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:40.284721 19939 consensus_queue.cc:237] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:40.254804 20084 leader_election.cc:290] T c4f65cdf00b748bb963ee6e81ef43b79 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 12 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:20:40.282940 16146 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46224 (request call id 565) took 3741 ms. Trace:
I20260709 13:20:40.292738 16146 rpcz_store.cc:276] 0709 13:20:36.541002 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:36.543476 (+  2474us) service_pool.cc:224] Handling call
0709 13:20:38.527270 (+1983794us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:38.527456 (+   186us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:38.527485 (+    29us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:20:38.527532 (+    47us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:20:38.527567 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:38.527625 (+    58us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:40.282440 (+1754815us) spinlock_profiling.cc:243] Waited 1.98 s on lock 0x7b540020ba50. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5e8ad 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:20:40.282709 (+   269us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4561498880}
I20260709 13:20:40.295529 19726 raft_consensus.cc:3060] T d16dfad0c3b540e5b4b86492ddf87208 P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:20:40.296061 16266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 83177us
W20260709 13:20:40.304839 16114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:40.305523 16291 raft_consensus.cc:1275] T 18ad7e3af5504571a69d10c00750a01c P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 13:20:40.317428 16816 raft_consensus.cc:2468] T ada5a004ba7c46819051e3ccec71ffdb P cb82cc7bd6db46d980c75237a1402860 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 11.
I20260709 13:20:40.322397 16292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31969392ccdb43b491911a319e8bc7e1" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:40.288906 19806 leader_election.cc:290] T b87147731a68400e8cce9b879daa7c5d P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:40.329653 16295 raft_consensus.cc:1240] T 5a96373bb0e14dce8a266b2fc6c58507 P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 9. Current term is 10. Ops: []
I20260709 13:20:40.339365 20196 raft_consensus.cc:2749] T a378d7f796f94615b3b347119a9880fd P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Leader pre-election lost for term 10. Reason: Vote denied by peer cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923) with higher term. Message: Invalid argument: T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 for earlier term 10. Current term is 11.
I20260709 13:20:40.341990 16218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 11 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:20:40.343091 16218 raft_consensus.cc:2393] T 19baa7bc367844c288bca8f81439c413 P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 in current term 11: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
I20260709 13:20:40.306645 20189 raft_consensus.cc:515] T fffab26427e24d398e2695aad40ca134 P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:40.348639 20177 raft_consensus.cc:3060] T ecc1e363230043fe9fef390f0b979f20 P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:20:40.349505 19806 raft_consensus.cc:493] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:40.350270 19806 raft_consensus.cc:515] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:40.353363 16116 leader_election.cc:304] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: cb82cc7bd6db46d980c75237a1402860
I20260709 13:20:40.354559 19806 leader_election.cc:290] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:20:40.357875 19754 raft_consensus.cc:3055] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:20:40.358566 19754 raft_consensus.cc:740] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:20:40.360373 19754 consensus_queue.cc:260] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.7, Last appended by leader: 7, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } }
I20260709 13:20:40.362524 19754 raft_consensus.cc:3060] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:20:40.377071 16788 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:40.382489 16219 raft_consensus.cc:2468] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 11.
I20260709 13:20:40.387390 16148 raft_consensus.cc:2468] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 11.
I20260709 13:20:40.395642 19915 raft_consensus.cc:2804] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:20:40.414127 20181 raft_consensus.cc:3060] T 829b4ad8c5a54eababeea1181b960d4b P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:20:40.417380 20183 raft_consensus.cc:3055] T 8a878999c0024d2abe517484180290fc P 90f3da57796b488c9087a2f523adf045 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:20:40.424292 20080 raft_consensus.cc:493] T 1df93779209748b9860f6d893dd3de53 P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
W20260709 13:20:40.430395 16788 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:40.396631 19915 raft_consensus.cc:697] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [term 11 LEADER]: Becoming Leader. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:20:40.434577 20094 raft_consensus.cc:3055] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:20:40.447999 16293 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48674 (request call id 2559) took 1469 ms. Trace:
I20260709 13:20:40.455260 16292 raft_consensus.cc:2393] T 31969392ccdb43b491911a319e8bc7e1 P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 10: Already voted for candidate cb82cc7bd6db46d980c75237a1402860 in this term.
I20260709 13:20:40.407303 20189 leader_election.cc:290] T fffab26427e24d398e2695aad40ca134 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 9 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:40.464771 16293 rpcz_store.cc:276] 0709 13:20:38.978090 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:38.979403 (+  1313us) service_pool.cc:224] Handling call
0709 13:20:40.447433 (+1468030us) spinlock_profiling.cc:243] Waited 158 ms on lock 0x7b5400347e54. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:20:40.447759 (+   326us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":363494272}
I20260709 13:20:40.474509 19915 consensus_queue.cc:237] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:40.464329 20094 raft_consensus.cc:740] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:20:40.480751 20094 consensus_queue.cc:260] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:40.482410 20094 raft_consensus.cc:3060] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:20:40.487008 20080 raft_consensus.cc:515] T 1df93779209748b9860f6d893dd3de53 P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:40.459913 20183 raft_consensus.cc:740] T 8a878999c0024d2abe517484180290fc P 90f3da57796b488c9087a2f523adf045 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
W20260709 13:20:40.495366 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 142230us
I20260709 13:20:40.499328 16148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 11 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:20:40.500172 16148 raft_consensus.cc:2393] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 in current term 11: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
I20260709 13:20:40.503969 16266 leader_election.cc:304] T ada5a004ba7c46819051e3ccec71ffdb P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:20:40.518380 20022 leader_election.cc:290] T 9e65676b21b748728ad39668cfe722ec P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
W20260709 13:20:40.523639 16115 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:40.525914 16145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f2da98864b4425bf7c0055951fdd4b" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:20:40.526778 16145 raft_consensus.cc:3060] T 84f2da98864b4425bf7c0055951fdd4b P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:20:40.536888 16295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ad5dbfa99ee4b8d9e28be92b3f36db5" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:20:40.538247 16295 raft_consensus.cc:2393] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 9: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
I20260709 13:20:40.538847 16144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
W20260709 13:20:40.506794 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 380711us
I20260709 13:20:40.554044 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57a744f5aaf944678a63469cbf514015" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 10 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
W20260709 13:20:40.562989 16266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 91916.7us
I20260709 13:20:40.567027 16296 raft_consensus.cc:1240] T c9944d5e4b8c4109af9955c4f08c9095 P 90f3da57796b488c9087a2f523adf045 [term 10 LEADER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 9. Current term is 10. Ops: []
I20260709 13:20:40.586380 16146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87147731a68400e8cce9b879daa7c5d" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:40.587853 16816 raft_consensus.cc:1240] T 8c11074579b046d99e585c2818e18a2c P cb82cc7bd6db46d980c75237a1402860 [term 10 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 9. Current term is 10. Ops: []
W20260709 13:20:40.588349 16824 consensus_meta.cc:287] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860: Time spent flushing consensus metadata: real 0.557s	user 0.011s	sys 0.008s
I20260709 13:20:40.588811 16824 raft_consensus.cc:2468] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 8.
I20260709 13:20:40.591005 20183 consensus_queue.cc:260] T 8a878999c0024d2abe517484180290fc P 90f3da57796b488c9087a2f523adf045 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.4, Last appended by leader: 4, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:40.593060 20073 raft_consensus.cc:493] T ee9ed67006144797b1de6ea8c2013a68 P cb82cc7bd6db46d980c75237a1402860 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:20:40.593571 20206 log.cc:927] Time spent T d16dfad0c3b540e5b4b86492ddf87208 P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 0.231s	user 0.000s	sys 0.001s
I20260709 13:20:40.593729 20073 raft_consensus.cc:515] T ee9ed67006144797b1de6ea8c2013a68 P cb82cc7bd6db46d980c75237a1402860 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:40.595897 16266 leader_election.cc:304] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:20:40.597131 20065 consensus_queue.cc:1059] T 8c11074579b046d99e585c2818e18a2c P 90f3da57796b488c9087a2f523adf045 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:20:40.598232 20065 raft_consensus.cc:2804] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:20:40.598807 20065 raft_consensus.cc:697] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 90f3da57796b488c9087a2f523adf045 [term 8 LEADER]: Becoming Leader. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:20:40.601831 16190 leader_election.cc:304] T 5a959dbecc2648eb85d1cc550f5a3245 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:20:40.602547 20065 consensus_queue.cc:237] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:40.604447 20197 raft_consensus.cc:2804] T ada5a004ba7c46819051e3ccec71ffdb P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:20:40.599620 20088 raft_consensus.cc:3055] T 8c11074579b046d99e585c2818e18a2c P 90f3da57796b488c9087a2f523adf045 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:20:40.605324 20197 raft_consensus.cc:697] T ada5a004ba7c46819051e3ccec71ffdb P 90f3da57796b488c9087a2f523adf045 [term 11 LEADER]: Becoming Leader. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:20:40.605628 16292 raft_consensus.cc:1275] T 66ae176961b74813a58bc15c9c4c6f2a P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
W20260709 13:20:40.607523 20211 log.cc:927] Time spent T 02838c86531d4f98bf58d1c22b8baf77 P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.093s	user 0.000s	sys 0.002s
I20260709 13:20:40.605484 20088 raft_consensus.cc:740] T 8c11074579b046d99e585c2818e18a2c P 90f3da57796b488c9087a2f523adf045 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:20:40.609040 20088 consensus_queue.cc:260] T 8c11074579b046d99e585c2818e18a2c P 90f3da57796b488c9087a2f523adf045 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 9.10, Last appended by leader: 10, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:40.609778 16817 raft_consensus.cc:1275] T af57996a8be84d398f0f023e8d08ef77 P cb82cc7bd6db46d980c75237a1402860 [term 11 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 13:20:40.610473 20088 raft_consensus.cc:3060] T 8c11074579b046d99e585c2818e18a2c P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:20:40.569797 16151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62fe3346cd3c42c4a55e14af9073bca4" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
W20260709 13:20:40.599874 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 147029us
I20260709 13:20:40.604300 20187 raft_consensus.cc:493] T 02838c86531d4f98bf58d1c22b8baf77 P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
W20260709 13:20:40.616411 16114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:40.617282 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e65676b21b748728ad39668cfe722ec" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:40.600829 20183 raft_consensus.cc:3060] T 8a878999c0024d2abe517484180290fc P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:20:40.628599 16219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a96373bb0e14dce8a266b2fc6c58507" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 10 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:40.626536 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87ac0d1f7e8a419f9bcbc04aee0e067e" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:20:40.628942 16218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
W20260709 13:20:40.623799 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 56371.2us
I20260709 13:20:40.639532 16218 raft_consensus.cc:2393] T 19baa7bc367844c288bca8f81439c413 P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 11: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
I20260709 13:20:40.621806 16145 raft_consensus.cc:2468] T 84f2da98864b4425bf7c0055951fdd4b P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 9.
I20260709 13:20:40.611918 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87147731a68400e8cce9b879daa7c5d" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:40.627799 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdbd78acb4cf46298e0de8ba0d130e19" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 11 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:40.627140 16223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada5a004ba7c46819051e3ccec71ffdb" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:20:40.624899 16221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f2da98864b4425bf7c0055951fdd4b" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:40.645713 16300 raft_consensus.cc:2468] T b87147731a68400e8cce9b879daa7c5d P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 11.
I20260709 13:20:40.646055 16292 raft_consensus.cc:1240] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 12. Current term is 13. Ops: []
I20260709 13:20:40.646770 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e9b2bf7011f4ae688a79d7bb5a447b4" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 9 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:40.618875 20177 raft_consensus.cc:2749] T ecc1e363230043fe9fef390f0b979f20 P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:20:40.627096 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0eab25ce4c64e018a041beeda2867db" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:20:40.649340 16225 raft_consensus.cc:3060] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:20:40.649384 16224 raft_consensus.cc:3060] T c0eab25ce4c64e018a041beeda2867db P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:20:40.611105 19990 consensus_meta.cc:287] T 8638ee363273426db1393dd81e5c7b72 P cb82cc7bd6db46d980c75237a1402860: Time spent flushing consensus metadata: real 0.507s	user 0.008s	sys 0.008s
I20260709 13:20:40.650965 16222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7ca61592e848ea9b0fb81a2ce33cdc" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:20:40.619069 20080 leader_election.cc:290] T 1df93779209748b9860f6d893dd3de53 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:20:40.673141 16144 raft_consensus.cc:2468] T ecc1e363230043fe9fef390f0b979f20 P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 9.
W20260709 13:20:40.674183 16789 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:20:40.667007 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 264285us
I20260709 13:20:40.678895 16226 raft_consensus.cc:2468] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 10.
I20260709 13:20:40.606724 20197 consensus_queue.cc:237] T ada5a004ba7c46819051e3ccec71ffdb P 90f3da57796b488c9087a2f523adf045 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:40.685032 16219 raft_consensus.cc:2393] T 5a96373bb0e14dce8a266b2fc6c58507 P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 10: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
I20260709 13:20:40.667574 19939 raft_consensus.cc:493] T 1df93779209748b9860f6d893dd3de53 P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:20:40.679260 20189 raft_consensus.cc:2749] T 5a959dbecc2648eb85d1cc550f5a3245 P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:20:40.702672 16221 raft_consensus.cc:2368] T 84f2da98864b4425bf7c0055951fdd4b P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 for earlier term 8. Current term is 9.
I20260709 13:20:40.717113 16219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4f65cdf00b748bb963ee6e81ef43b79" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:20:40.730078 16825 pending_rounds.cc:85] T 806d7690c2b644959152074bb2cab0a7 P cb82cc7bd6db46d980c75237a1402860: Aborting all ops after (but not including) 6
I20260709 13:20:40.730479 16825 pending_rounds.cc:107] T 806d7690c2b644959152074bb2cab0a7 P cb82cc7bd6db46d980c75237a1402860: Aborting uncommitted NO_OP operation due to leader change: 7.7
I20260709 13:20:40.679489 20187 raft_consensus.cc:515] T 02838c86531d4f98bf58d1c22b8baf77 P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:40.731133 16825 raft_consensus.cc:2889] T 806d7690c2b644959152074bb2cab0a7 P cb82cc7bd6db46d980c75237a1402860 [term 9 LEARNER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:20:40.738977 16300 raft_consensus.cc:1240] T 8cc959fb7f704fba9009c0595467d074 P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 10. Current term is 11. Ops: []
W20260709 13:20:40.732157 16265 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.14.177.129:36741 timed out after 1.824s (SENT) [suppressed 18 similar messages]
I20260709 13:20:40.727771 19923 raft_consensus.cc:493] T af57996a8be84d398f0f023e8d08ef77 P cb82cc7bd6db46d980c75237a1402860 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:40.746237 16292 raft_consensus.cc:1217] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045 [term 10 LEADER]: Deduplicated request from leader. Original: 5.5->[6.6-9.7]   Dedup: 6.6->[9.7-9.7]
I20260709 13:20:40.771359 16292 raft_consensus.cc:1240] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045 [term 10 LEADER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 9. Current term is 10. Ops: [6.6-9.7]
W20260709 13:20:40.782438 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 50304.8us
W20260709 13:20:40.790156 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 52857.2us
W20260709 13:20:40.802706 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 201112us
I20260709 13:20:40.800206 19982 consensus_queue.cc:1048] T b6f07842d3aa47858df5665874c86f3a P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:20:40.798667 16790 leader_election.cc:304] T 19baa7bc367844c288bca8f81439c413 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:20:40.803320 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:40.805730 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5ad4eb281564936a59f41e387a47156" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:20:40.743275 19939 raft_consensus.cc:515] T 1df93779209748b9860f6d893dd3de53 P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:40.760365 20177 consensus_queue.cc:1059] T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.061s
W20260709 13:20:40.830891 16047 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:58400 (request call id 190) took 25821 ms (client timeout 15000 ms). Trace:
W20260709 13:20:40.843534 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 54033.7us
I20260709 13:20:40.769133 19754 consensus_queue.cc:1059] T 5a96373bb0e14dce8a266b2fc6c58507 P 647a343a542d4a139fe106c19095129b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: INVALID_TERM, Last received: 9.4, Next index: 5, Last known committed idx: 2, Time since last communication: 0.070s
I20260709 13:20:40.792953 19923 raft_consensus.cc:515] T af57996a8be84d398f0f023e8d08ef77 P cb82cc7bd6db46d980c75237a1402860 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:40.830391 20073 leader_election.cc:290] T ee9ed67006144797b1de6ea8c2013a68 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
W20260709 13:20:40.853631 16266 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:40.863960 19914 raft_consensus.cc:493] T 18ad7e3af5504571a69d10c00750a01c P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:40.859542 19915 consensus_queue.cc:1059] T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.007s
W20260709 13:20:40.865360 16117 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:40.866039 16816 raft_consensus.cc:1275] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 [term 8 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:20:40.866096 16151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b7d1ab519044eda2d5e04a07242f25" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:40.867051 16151 raft_consensus.cc:2468] T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 8.
I20260709 13:20:40.850651 19806 raft_consensus.cc:2749] T 19baa7bc367844c288bca8f81439c413 P cb82cc7bd6db46d980c75237a1402860 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:20:40.859838 16190 leader_election.cc:304] T e4dc6b0541ca431ebc757beac00521f1 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 647a343a542d4a139fe106c19095129b
W20260709 13:20:40.865463 20214 log.cc:927] Time spent T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 0.249s	user 0.001s	sys 0.000s
W20260709 13:20:40.879962 19941 consensus_meta.cc:287] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860: Time spent flushing consensus metadata: real 0.858s	user 0.006s	sys 0.010s
I20260709 13:20:40.891448 19941 raft_consensus.cc:2749] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:20:40.864631 19914 raft_consensus.cc:515] T 18ad7e3af5504571a69d10c00750a01c P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } }
I20260709 13:20:40.893321 20187 leader_election.cc:290] T 02838c86531d4f98bf58d1c22b8baf77 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:40.894394 19914 leader_election.cc:290] T 18ad7e3af5504571a69d10c00750a01c P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:20:40.895341 16816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2372981006d4b218215c28964dea623" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:40.896045 16816 raft_consensus.cc:2468] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 10.
W20260709 13:20:40.899749 20216 log.cc:927] Time spent T 57a744f5aaf944678a63469cbf514015 P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 0.174s	user 0.000s	sys 0.001s
I20260709 13:20:40.901243 19939 leader_election.cc:290] T 1df93779209748b9860f6d893dd3de53 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
W20260709 13:20:40.903764 20218 log.cc:927] Time spent T cdbd78acb4cf46298e0de8ba0d130e19 P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.110s	user 0.000s	sys 0.001s
I20260709 13:20:40.906189 16299 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40474 (request call id 635) took 1195 ms. Trace:
I20260709 13:20:40.906754 16299 rpcz_store.cc:276] 0709 13:20:39.711113 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:40.324888 (+613775us) service_pool.cc:224] Handling call
0709 13:20:40.325477 (+   589us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:20:40.395313 (+ 69836us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:40.395344 (+    31us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:20:40.396191 (+   847us) log.cc:844] Serialized 52 byte log entry
0709 13:20:40.648100 (+251909us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:20:40.648143 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:40.648210 (+    67us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:20:40.905702 (+257492us) raft_consensus.cc:1685] finished
0709 13:20:40.905726 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:40.905947 (+   221us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":60103,"raft.queue_time_us":73333,"raft.run_cpu_time_us":216,"raft.run_wall_time_us":22321,"thread_start_us":246752,"threads_started":1,"wal-append.queue_time_us":311078}
W20260709 13:20:40.880146 20215 log.cc:927] Time spent T ee9ed67006144797b1de6ea8c2013a68 P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 0.111s	user 0.000s	sys 0.001s
W20260709 13:20:40.885632 20217 log.cc:927] Time spent T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.092s	user 0.001s	sys 0.000s
I20260709 13:20:40.886422 16298 raft_consensus.cc:2468] T ecc1e363230043fe9fef390f0b979f20 P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 9.
W20260709 13:20:40.886976 16263 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:40.895939 16115 leader_election.cc:304] T 9c61a39fd4f84424b129c1a6f9ba404d P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:20:40.829511 20107 consensus_queue.cc:1059] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 647a343a542d4a139fe106c19095129b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: INVALID_TERM, Last received: 8.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.161s
W20260709 13:20:40.884529 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 50304.8us
W20260709 13:20:40.924401 16294 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.523s	user 0.012s	sys 0.024s
W20260709 13:20:40.925143 16790 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:20:40.925639 16294 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.620s	user 0.013s	sys 0.024s
W20260709 13:20:40.926604 16788 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:40.920468 16224 raft_consensus.cc:2468] T c0eab25ce4c64e018a041beeda2867db P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 10.
I20260709 13:20:40.847712 20192 consensus_queue.cc:1048] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.152s
W20260709 13:20:40.947078 16114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:20:40.950117 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 74152.3us
W20260709 13:20:40.954039 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 110710us
I20260709 13:20:40.964318 20022 raft_consensus.cc:3055] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 647a343a542d4a139fe106c19095129b [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:20:40.964898 20022 raft_consensus.cc:740] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 647a343a542d4a139fe106c19095129b [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:20:40.966041 20022 consensus_queue.cc:260] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.8, Last appended by leader: 8, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:40.971324 20187 consensus_queue.cc:1048] T 68a6838d82ef4cac83507f906139d90f P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:20:40.973860 16790 leader_election.cc:304] T 8c11074579b046d99e585c2818e18a2c P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:20:40.976253 20073 raft_consensus.cc:2749] T 8c11074579b046d99e585c2818e18a2c P cb82cc7bd6db46d980c75237a1402860 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:20:40.978366 16294 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48674 (request call id 2576) took 1354 ms. Trace:
I20260709 13:20:40.978662 16151 raft_consensus.cc:1275] T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b [term 10 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:20:40.979707 20022 raft_consensus.cc:3060] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 647a343a542d4a139fe106c19095129b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:20:40.978791 16294 rpcz_store.cc:276] 0709 13:20:39.623480 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:40.302902 (+679422us) service_pool.cc:224] Handling call
0709 13:20:40.303450 (+   548us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:20:40.304276 (+   826us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:20:40.304303 (+    27us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:20:40.305103 (+   800us) log.cc:844] Serialized 52 byte log entry
0709 13:20:40.926023 (+620920us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:20:40.926066 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:40.926146 (+    80us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:20:40.978106 (+ 51960us) raft_consensus.cc:1685] finished
0709 13:20:40.978129 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:40.978298 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":58952,"raft.queue_time_us":8960,"raft.run_cpu_time_us":157,"raft.run_wall_time_us":157,"thread_start_us":620501,"threads_started":1,"wal-append.queue_time_us":573426}
I20260709 13:20:40.980221 16816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 11 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:20:40.980994 16816 raft_consensus.cc:2393] T 19baa7bc367844c288bca8f81439c413 P cb82cc7bd6db46d980c75237a1402860 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 11: Already voted for candidate cb82cc7bd6db46d980c75237a1402860 in this term.
I20260709 13:20:40.981083 16294 raft_consensus.cc:1275] T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:20:40.981927 16295 raft_consensus.cc:1240] T 8d4ab870a05347868956aa8b841b2981 P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 10. Current term is 11. Ops: [10.5-10.5]
I20260709 13:20:40.984223 16294 raft_consensus.cc:1275] T 31969392ccdb43b491911a319e8bc7e1 P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:20:40.995236 20021 raft_consensus.cc:3055] T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b [term 9 LEADER]: Stepping down as leader of term 9
W20260709 13:20:40.995502 20181 consensus_meta.cc:287] T 829b4ad8c5a54eababeea1181b960d4b P 1ca2725d290e47d68b1a4b609c9816ac: Time spent flushing consensus metadata: real 0.528s	user 0.010s	sys 0.002s
I20260709 13:20:40.995792 19914 raft_consensus.cc:3060] T 9c61a39fd4f84424b129c1a6f9ba404d P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:20:40.995831 20021 raft_consensus.cc:740] T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:20:41.000437 20094 consensus_queue.cc:1059] T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:20:41.000373 20021 consensus_queue.cc:260] T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:41.001585 20021 raft_consensus.cc:3060] T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:20:41.002447 20177 consensus_queue.cc:1059] T 5a959dbecc2648eb85d1cc550f5a3245 P 647a343a542d4a139fe106c19095129b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
W20260709 13:20:41.006069 19726 consensus_meta.cc:287] T d16dfad0c3b540e5b4b86492ddf87208 P 647a343a542d4a139fe106c19095129b: Time spent flushing consensus metadata: real 0.709s	user 0.003s	sys 0.007s
I20260709 13:20:41.006709 19726 raft_consensus.cc:3055] T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:20:41.007503 16297 raft_consensus.cc:1240] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 10. Current term is 11. Ops: []
I20260709 13:20:41.010455 19915 raft_consensus.cc:3055] T 5a959dbecc2648eb85d1cc550f5a3245 P 647a343a542d4a139fe106c19095129b [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:20:41.012394 16295 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48684 (request call id 2417) took 1147 ms. Trace:
I20260709 13:20:41.012846 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee9ed67006144797b1de6ea8c2013a68" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 9 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:41.012801 16295 rpcz_store.cc:276] 0709 13:20:39.865375 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:40.539108 (+673733us) service_pool.cc:224] Handling call
0709 13:20:40.981678 (+442570us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:20:40.982348 (+   670us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:41.012165 (+ 29817us) spinlock_profiling.cc:243] Waited 421 ms on lock 0x7b5400210050. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5e8ad 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:20:41.012335 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":967832832}
I20260709 13:20:40.955886 20192 consensus_queue.cc:1243] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Peer 647a343a542d4a139fe106c19095129b log is divergent from this leader: its last log entry 5.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
W20260709 13:20:41.017102 16115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 121247us
I20260709 13:20:41.023698 16293 raft_consensus.cc:1240] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 11. Current term is 12. Ops: []
W20260709 13:20:41.037815 16114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:41.089797 16220 raft_consensus.cc:2468] T 829b4ad8c5a54eababeea1181b960d4b P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 10.
I20260709 13:20:41.010963 19915 raft_consensus.cc:740] T 5a959dbecc2648eb85d1cc550f5a3245 P 647a343a542d4a139fe106c19095129b [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:20:40.961314 19939 raft_consensus.cc:2804] T e4dc6b0541ca431ebc757beac00521f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Leader election won for term 10
W20260709 13:20:41.099496 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 75282.8us
I20260709 13:20:40.960724 19923 leader_election.cc:290] T af57996a8be84d398f0f023e8d08ef77 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:20:41.007086 19726 raft_consensus.cc:740] T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:20:41.099485 16825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c11074579b046d99e585c2818e18a2c" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:20:41.104951 16816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ab0ac8347641e6955e9bce7d90c1a2" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
W20260709 13:20:41.106462 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 258423us
I20260709 13:20:41.109709 16817 raft_consensus.cc:1240] T ada5a004ba7c46819051e3ccec71ffdb P cb82cc7bd6db46d980c75237a1402860 [term 11 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 10. Current term is 11. Ops: []
I20260709 13:20:41.111928 16825 raft_consensus.cc:2393] T 8c11074579b046d99e585c2818e18a2c P cb82cc7bd6db46d980c75237a1402860 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 10: Already voted for candidate cb82cc7bd6db46d980c75237a1402860 in this term.
W20260709 13:20:40.837023 16047 rpcz_store.cc:269] 0709 13:20:14.903068 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:14.903266 (+   198us) service_pool.cc:224] Handling call
0709 13:20:40.716982 (+25813716us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:20:31.992640 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:20:31.993345 (+   705us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:20:31.993394 (+    49us) write_op.cc:435] Acquired schema lock
0709 13:20:31.993417 (+    23us) tablet.cc:662] Decoding operations
0709 13:20:31.995122 (+  1705us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:20:32.039497 (+ 44375us) write_op.cc:641] Partition lock acquired for write op
0709 13:20:32.045664 (+  6167us) tablet.cc:685] Acquiring locks for 17 operations
0709 13:20:32.218631 (+172967us) tablet.cc:701] Row locks acquired
0709 13:20:32.218698 (+    67us) write_op.cc:260] PREPARE: finished
0709 13:20:32.218896 (+   198us) write_op.cc:270] Start()
0709 13:20:32.219069 (+   173us) write_op.cc:276] Timestamp: P: 1783603232218824 usec, L: 0
0709 13:20:32.219095 (+    26us) op_driver.cc:348] REPLICATION: starting
0709 13:20:32.335990 (+116895us) log.cc:844] Serialized 7035 byte log entry
0709 13:20:33.453881 (+1117891us) op_driver.cc:464] REPLICATION: finished
0709 13:20:33.550859 (+ 96978us) write_op.cc:301] APPLY: starting
0709 13:20:33.550967 (+   108us) tablet.cc:1367] starting BulkCheckPresence
0709 13:20:33.551377 (+   410us) tablet.cc:1370] finished BulkCheckPresence
0709 13:20:33.551515 (+   138us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:20:33.594015 (+ 42500us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:20:33.594555 (+   540us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=17
0709 13:20:33.594630 (+    75us) write_op.cc:312] APPLY: finished
0709 13:20:33.632608 (+ 37978us) log.cc:844] Serialized 119 byte log entry
0709 13:20:33.673578 (+ 40970us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:20:33.673757 (+   179us) write_op.cc:454] Released schema lock
0709 13:20:33.673834 (+    77us) write_op.cc:341] FINISH: Updating metrics
0709 13:20:33.674317 (+   483us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:20:33.674343 (+    26us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":22456,"reported_tablets":34,"child_traces":[["op",{"apply.queue_time_us":88453,"apply.run_cpu_time_us":6265,"apply.run_wall_time_us":123423,"num_ops":17,"prepare.queue_time_us":150453,"prepare.run_cpu_time_us":33040,"prepare.run_wall_time_us":581834,"raft.queue_time_us":307263,"raft.run_cpu_time_us":11801,"raft.run_wall_time_us":177748,"replication_time_us":1204581,"thread_start_us":581689,"threads_started":4,"wal-append.queue_time_us":355740,"wal-append.run_cpu_time_us":71106,"wal-append.run_wall_time_us":2022581}]]}
I20260709 13:20:41.107364 19939 raft_consensus.cc:697] T e4dc6b0541ca431ebc757beac00521f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 10 LEADER]: Becoming Leader. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:20:41.146165 16291 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40474 (request call id 636) took 1434 ms. Trace:
I20260709 13:20:41.153745 16225 raft_consensus.cc:2468] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 11.
I20260709 13:20:41.149794 16298 pending_rounds.cc:85] T 2794af5b99984efebdee564035841cb0 P 90f3da57796b488c9087a2f523adf045: Aborting all ops after (but not including) 7
I20260709 13:20:41.155589 16298 pending_rounds.cc:107] T 2794af5b99984efebdee564035841cb0 P 90f3da57796b488c9087a2f523adf045: Aborting uncommitted NO_OP operation due to leader change: 7.8
I20260709 13:20:41.155862 16298 raft_consensus.cc:2889] T 2794af5b99984efebdee564035841cb0 P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:20:41.146624 16291 rpcz_store.cc:276] 0709 13:20:39.711422 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:40.432948 (+721526us) service_pool.cc:224] Handling call
0709 13:20:40.433487 (+   539us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:20:40.434078 (+   591us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:40.434106 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:20:40.434672 (+   566us) log.cc:844] Serialized 26 byte log entry
0709 13:20:40.556352 (+121680us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:20:40.556402 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:40.556479 (+    77us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:20:41.145955 (+589476us) raft_consensus.cc:1685] finished
0709 13:20:41.145973 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:41.146105 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":4263,"raft.run_cpu_time_us":119,"raft.run_wall_time_us":118,"thread_start_us":121243,"threads_started":1,"wal-append.queue_time_us":279815}
W20260709 13:20:41.163792 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 174945us
I20260709 13:20:41.166621 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:41.173475 16217 raft_consensus.cc:1275] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
W20260709 13:20:41.174353 16192 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 119937us
I20260709 13:20:41.174614 20228 consensus_queue.cc:1059] T 5a96373bb0e14dce8a266b2fc6c58507 P 647a343a542d4a139fe106c19095129b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: INVALID_TERM, Last received: 9.4, Next index: 5, Last known committed idx: 2, Time since last communication: 0.006s
I20260709 13:20:41.175298 16192 leader_election.cc:304] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860
I20260709 13:20:41.176872 16192 leader_election.cc:304] T 66f75124215f4ca6bef831e174a792d8 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
I20260709 13:20:41.177824 19923 raft_consensus.cc:493] T c9944d5e4b8c4109af9955c4f08c9095 P cb82cc7bd6db46d980c75237a1402860 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:41.177909 19939 consensus_queue.cc:237] T e4dc6b0541ca431ebc757beac00521f1 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:41.179740 20181 raft_consensus.cc:2749] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:20:41.180693 20181 raft_consensus.cc:2749] T 66f75124215f4ca6bef831e174a792d8 P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:20:41.185330 16190 leader_election.cc:304] T 8cc959fb7f704fba9009c0595467d074 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 647a343a542d4a139fe106c19095129b
I20260709 13:20:41.186565 20231 consensus_queue.cc:1059] T 8cc959fb7f704fba9009c0595467d074 P 647a343a542d4a139fe106c19095129b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 2, Time since last communication: 0.001s
W20260709 13:20:41.188886 16115 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:20:41.189445 16115 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:41.178558 19923 raft_consensus.cc:515] T c9944d5e4b8c4109af9955c4f08c9095 P cb82cc7bd6db46d980c75237a1402860 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:41.192358 19923 leader_election.cc:290] T c9944d5e4b8c4109af9955c4f08c9095 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:41.193850 20192 consensus_queue.cc:1048] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:20:41.195881 16190 leader_election.cc:304] T c2f5f89186204242bf377f7c42b2e2da P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 647a343a542d4a139fe106c19095129b
I20260709 13:20:41.197448 16190 leader_election.cc:304] T b6f07842d3aa47858df5665874c86f3a P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 647a343a542d4a139fe106c19095129b
I20260709 13:20:41.198328 19861 raft_consensus.cc:2804] T 8cc959fb7f704fba9009c0595467d074 P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:20:41.198858 19861 raft_consensus.cc:697] T 8cc959fb7f704fba9009c0595467d074 P 1ca2725d290e47d68b1a4b609c9816ac [term 11 LEADER]: Becoming Leader. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:20:41.199512 20200 raft_consensus.cc:2804] T c2f5f89186204242bf377f7c42b2e2da P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:20:41.199994 19861 consensus_queue.cc:237] T 8cc959fb7f704fba9009c0595467d074 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.7, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:41.200786 20181 raft_consensus.cc:2804] T b6f07842d3aa47858df5665874c86f3a P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:20:41.201247 20181 raft_consensus.cc:493] T b6f07842d3aa47858df5665874c86f3a P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:41.201612 20181 raft_consensus.cc:3060] T b6f07842d3aa47858df5665874c86f3a P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:20:41.207355 16192 leader_election.cc:304] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 647a343a542d4a139fe106c19095129b
I20260709 13:20:41.167673 19914 raft_consensus.cc:2749] T 9c61a39fd4f84424b129c1a6f9ba404d P 647a343a542d4a139fe106c19095129b [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:20:41.218079 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62fe3346cd3c42c4a55e14af9073bca4" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:41.218093 16190 leader_election.cc:304] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: cb82cc7bd6db46d980c75237a1402860
I20260709 13:20:41.233640 16221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b7d1ab519044eda2d5e04a07242f25" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:41.234127 16190 leader_election.cc:304] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860
I20260709 13:20:41.230145 20189 raft_consensus.cc:2804] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:20:41.235116 20189 raft_consensus.cc:493] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:41.235504 20189 raft_consensus.cc:3060] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:20:41.241750 16293 raft_consensus.cc:1240] T c0eab25ce4c64e018a041beeda2867db P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 9. Current term is 10. Ops: []
I20260709 13:20:41.245453 16295 raft_consensus.cc:1240] T e4dc6b0541ca431ebc757beac00521f1 P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 9. Current term is 10. Ops: [7.5-9.6]
I20260709 13:20:41.245862 16291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18ad7e3af5504571a69d10c00750a01c" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:41.246523 16291 raft_consensus.cc:2468] T 18ad7e3af5504571a69d10c00750a01c P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 9.
I20260709 13:20:41.248762 16217 raft_consensus.cc:2393] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 in current term 13: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
I20260709 13:20:41.162539 19915 consensus_queue.cc:260] T 5a959dbecc2648eb85d1cc550f5a3245 P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:41.259148 16225 raft_consensus.cc:2468] T ecc1e363230043fe9fef390f0b979f20 P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 9.
I20260709 13:20:41.260835 16295 raft_consensus.cc:1240] T e5ad4eb281564936a59f41e387a47156 P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 10. Current term is 11. Ops: []
W20260709 13:20:41.262187 16192 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 54925.2us
W20260709 13:20:41.262614 16266 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:20:41.269550 16220 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33050 (request call id 665) took 1522 ms (client timeout 1771 ms). Trace:
W20260709 13:20:41.269999 16220 rpcz_store.cc:269] 0709 13:20:39.739308 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:39.739435 (+   127us) service_pool.cc:224] Handling call
0709 13:20:41.252344 (+1512909us) spinlock_profiling.cc:243] Waited 1.26 s on lock 0x7b54001419d4. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:20:41.261527 (+  9183us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2904653696}
I20260709 13:20:41.273489 19726 consensus_queue.cc:260] T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:41.274778 19726 raft_consensus.cc:3060] T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:20:41.288125 20212 log.cc:927] Time spent T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 0.672s	user 0.008s	sys 0.000s
W20260709 13:20:41.288401 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 115493us
I20260709 13:20:41.289244 19982 raft_consensus.cc:3055] T 5a96373bb0e14dce8a266b2fc6c58507 P 647a343a542d4a139fe106c19095129b [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:20:41.289713 19982 raft_consensus.cc:740] T 5a96373bb0e14dce8a266b2fc6c58507 P 647a343a542d4a139fe106c19095129b [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:20:41.290637 16790 leader_election.cc:304] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
W20260709 13:20:41.291363 20183 consensus_meta.cc:287] T 8a878999c0024d2abe517484180290fc P 90f3da57796b488c9087a2f523adf045: Time spent flushing consensus metadata: real 0.651s	user 0.011s	sys 0.000s
I20260709 13:20:41.293630 16816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e9b2bf7011f4ae688a79d7bb5a447b4" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 9 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:41.293828 16818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e65676b21b748728ad39668cfe722ec" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:41.294488 16816 raft_consensus.cc:2410] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b for term 9 because replica has last-logged OpId of term: 7 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 6.
I20260709 13:20:41.294227 16789 leader_election.cc:304] T b87147731a68400e8cce9b879daa7c5d P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 647a343a542d4a139fe106c19095129b
I20260709 13:20:41.301940 16291 raft_consensus.cc:1240] T 8158bbd17b994d2eb96f560bd0f1b88d P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 8. Current term is 9. Ops: []
I20260709 13:20:41.230458 19914 consensus_queue.cc:1059] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.032s
I20260709 13:20:41.290648 19982 consensus_queue.cc:260] T 5a96373bb0e14dce8a266b2fc6c58507 P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 9.4, Last appended by leader: 4, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:41.307706 16824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31969392ccdb43b491911a319e8bc7e1" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:41.307811 16221 raft_consensus.cc:1275] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 13:20:41.308584 19982 raft_consensus.cc:3060] T 5a96373bb0e14dce8a266b2fc6c58507 P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:20:41.310099 16217 raft_consensus.cc:1275] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
W20260709 13:20:41.311225 16192 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 48103.9us
I20260709 13:20:41.309337 16225 raft_consensus.cc:1275] T d2ab0ac8347641e6955e9bce7d90c1a2 P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:20:41.319685 16220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee9ed67006144797b1de6ea8c2013a68" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 9 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:41.318723 19673 raft_consensus.cc:2804] T b87147731a68400e8cce9b879daa7c5d P cb82cc7bd6db46d980c75237a1402860 [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 13:20:41.336244 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 102205us
I20260709 13:20:41.337270 16190 leader_election.cc:304] T 19baa7bc367844c288bca8f81439c413 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860
I20260709 13:20:41.338778 16190 leader_election.cc:304] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 647a343a542d4a139fe106c19095129b
I20260709 13:20:41.346225 16190 leader_election.cc:304] T d2ab0ac8347641e6955e9bce7d90c1a2 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860
I20260709 13:20:41.341504 20246 raft_consensus.cc:2749] T 19baa7bc367844c288bca8f81439c413 P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:20:41.347648 16190 leader_election.cc:304] T 5a96373bb0e14dce8a266b2fc6c58507 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 647a343a542d4a139fe106c19095129b
I20260709 13:20:41.350361 20246 raft_consensus.cc:2804] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:20:41.351016 20246 raft_consensus.cc:697] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 1ca2725d290e47d68b1a4b609c9816ac [term 9 LEADER]: Becoming Leader. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
W20260709 13:20:41.352298 16115 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:20:41.363579 16115 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:41.352187 20246 consensus_queue.cc:237] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:41.369839 20254 raft_consensus.cc:493] T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:41.372898 20245 raft_consensus.cc:2749] T d2ab0ac8347641e6955e9bce7d90c1a2 P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:20:41.378705 16145 raft_consensus.cc:1275] T 18ad7e3af5504571a69d10c00750a01c P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 13:20:41.370846 20254 raft_consensus.cc:515] T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:41.384264 16147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fffab26427e24d398e2695aad40ca134" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:20:41.323213 16825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57a744f5aaf944678a63469cbf514015" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 10 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:41.382808 20245 raft_consensus.cc:2804] T 5a96373bb0e14dce8a266b2fc6c58507 P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:20:41.319140 19923 raft_consensus.cc:3060] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:20:41.397950 16790 leader_election.cc:304] T 829b4ad8c5a54eababeea1181b960d4b P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 647a343a542d4a139fe106c19095129b
I20260709 13:20:41.396390 16147 raft_consensus.cc:3060] T fffab26427e24d398e2695aad40ca134 P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:20:41.400207 16145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5d59407eb545b895d8d922e0dd69fd" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:41.388466 20187 raft_consensus.cc:2804] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:20:41.385979 19673 raft_consensus.cc:493] T b87147731a68400e8cce9b879daa7c5d P cb82cc7bd6db46d980c75237a1402860 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:41.405972 16192 leader_election.cc:304] T 8c11074579b046d99e585c2818e18a2c P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860
W20260709 13:20:41.397044 20065 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.775s	user 0.005s	sys 0.009s
W20260709 13:20:41.407217 20065 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.786s	user 0.006s	sys 0.009s
W20260709 13:20:41.408309 20045 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.804s	user 0.011s	sys 0.004s
W20260709 13:20:41.408689 20045 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.805s	user 0.011s	sys 0.004s
I20260709 13:20:41.409521 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1df93779209748b9860f6d893dd3de53" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:41.415517 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02838c86531d4f98bf58d1c22b8baf77" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:41.410385 16226 raft_consensus.cc:2468] T 1df93779209748b9860f6d893dd3de53 P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 10.
I20260709 13:20:41.394735 20200 raft_consensus.cc:697] T c2f5f89186204242bf377f7c42b2e2da P 1ca2725d290e47d68b1a4b609c9816ac [term 11 LEADER]: Becoming Leader. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:20:41.423377 16151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1df93779209748b9860f6d893dd3de53" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
W20260709 13:20:41.423986 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 431384us
I20260709 13:20:41.426015 16217 raft_consensus.cc:1240] T 8c11074579b046d99e585c2818e18a2c P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 9. Current term is 10. Ops: []
I20260709 13:20:41.424454 16148 pending_rounds.cc:85] T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b: Aborting all ops after (but not including) 7
I20260709 13:20:41.434310 16148 pending_rounds.cc:107] T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b: Aborting uncommitted NO_OP operation due to leader change: 5.8
I20260709 13:20:41.434612 16148 raft_consensus.cc:2889] T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:20:41.436616 16146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c74490c40243a88e62d1760b5cd814" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:20:41.425032 16191 leader_election.cc:304] T 84f2da98864b4425bf7c0055951fdd4b P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
I20260709 13:20:41.409222 19673 raft_consensus.cc:3060] T b87147731a68400e8cce9b879daa7c5d P cb82cc7bd6db46d980c75237a1402860 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:20:41.439131 16221 raft_consensus.cc:1275] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:20:41.393354 20073 consensus_queue.cc:1059] T 3a7ca61592e848ea9b0fb81a2ce33cdc P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 9, Time since last communication: 0.023s
I20260709 13:20:41.443138 16116 leader_election.cc:304] T 84f2da98864b4425bf7c0055951fdd4b P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:20:41.412247 20245 raft_consensus.cc:697] T 5a96373bb0e14dce8a266b2fc6c58507 P 1ca2725d290e47d68b1a4b609c9816ac [term 10 LEADER]: Becoming Leader. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
W20260709 13:20:41.441275 16789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 47364us
I20260709 13:20:41.407797 20065 raft_consensus.cc:493] T 68a6838d82ef4cac83507f906139d90f P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:20:41.442364 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 190194us
I20260709 13:20:41.463003 16115 leader_election.cc:304] T 18ad7e3af5504571a69d10c00750a01c P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
W20260709 13:20:41.471975 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 72588.6us
I20260709 13:20:41.474802 20065 raft_consensus.cc:515] T 68a6838d82ef4cac83507f906139d90f P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:41.477142 20065 leader_election.cc:290] T 68a6838d82ef4cac83507f906139d90f P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:20:41.478276 20245 consensus_queue.cc:237] T 5a96373bb0e14dce8a266b2fc6c58507 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 9.4, Last appended by leader: 4, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:41.444257 19861 consensus_queue.cc:1059] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.035s
I20260709 13:20:41.485162 16145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f5f89186204242bf377f7c42b2e2da" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:41.460815 20045 raft_consensus.cc:2749] T 84f2da98864b4425bf7c0055951fdd4b P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:20:41.461549 19990 raft_consensus.cc:2804] T 829b4ad8c5a54eababeea1181b960d4b P cb82cc7bd6db46d980c75237a1402860 [term 10 FOLLOWER]: Leader election won for term 10
W20260709 13:20:41.487627 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 140096us
I20260709 13:20:41.496443 16143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87147731a68400e8cce9b879daa7c5d" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:41.501807 16146 raft_consensus.cc:2393] T 08c74490c40243a88e62d1760b5cd814 P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 9: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
I20260709 13:20:41.505568 20252 raft_consensus.cc:493] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:41.506276 20229 raft_consensus.cc:2804] T 18ad7e3af5504571a69d10c00750a01c P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:20:41.506157 20252 raft_consensus.cc:515] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:41.506749 20229 raft_consensus.cc:493] T 18ad7e3af5504571a69d10c00750a01c P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:41.507128 20229 raft_consensus.cc:3060] T 18ad7e3af5504571a69d10c00750a01c P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:20:41.507710 20257 raft_consensus.cc:2804] T 84f2da98864b4425bf7c0055951fdd4b P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:20:41.509869 16145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c2095a5f4ad49408dbc7f69c606120d" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
W20260709 13:20:41.520431 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 193448us
I20260709 13:20:41.521445 16147 raft_consensus.cc:2468] T fffab26427e24d398e2695aad40ca134 P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 9.
I20260709 13:20:41.440037 16220 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "02838c86531d4f98bf58d1c22b8baf77"
dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
mode: GRACEFUL
new_leader_uuid: "90f3da57796b488c9087a2f523adf045"
 from {username='slave'} at 127.0.0.1:39144
I20260709 13:20:41.481424 20181 raft_consensus.cc:515] T b6f07842d3aa47858df5665874c86f3a P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:20:41.578866 16114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:20:41.565147 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 122129us
W20260709 13:20:41.562680 16115 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:20:41.586181 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 63158.8us
I20260709 13:20:41.587736 16220 raft_consensus.cc:606] T 02838c86531d4f98bf58d1c22b8baf77 P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Received request to transfer leadership to TS 90f3da57796b488c9087a2f523adf045
I20260709 13:20:41.588183 16220 raft_consensus.cc:614] T 02838c86531d4f98bf58d1c22b8baf77 P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 13:20:41.594048 16192 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 188145us
I20260709 13:20:41.518672 19939 raft_consensus.cc:2749] T 8c11074579b046d99e585c2818e18a2c P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:20:41.487715 20060 consensus_queue.cc:1048] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.253s
I20260709 13:20:41.508236 20257 raft_consensus.cc:697] T 84f2da98864b4425bf7c0055951fdd4b P 1ca2725d290e47d68b1a4b609c9816ac [term 9 LEADER]: Becoming Leader. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:20:41.611689 16143 raft_consensus.cc:1240] T fffab26427e24d398e2695aad40ca134 P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 8. Current term is 9. Ops: [6.4-8.5]
I20260709 13:20:41.518224 19990 raft_consensus.cc:697] T 829b4ad8c5a54eababeea1181b960d4b P cb82cc7bd6db46d980c75237a1402860 [term 10 LEADER]: Becoming Leader. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
W20260709 13:20:41.612641 16114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:41.615347 16145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af57996a8be84d398f0f023e8d08ef77" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 12 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:41.518255 20189 raft_consensus.cc:515] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:41.638064 19990 consensus_queue.cc:237] T 829b4ad8c5a54eababeea1181b960d4b P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:41.639523 20189 leader_election.cc:290] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 11 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:41.522723 20252 leader_election.cc:290] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:41.642155 20243 raft_consensus.cc:493] T 66ae176961b74813a58bc15c9c4c6f2a P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:41.642743 20243 raft_consensus.cc:515] T 66ae176961b74813a58bc15c9c4c6f2a P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:41.644982 20243 leader_election.cc:290] T 66ae176961b74813a58bc15c9c4c6f2a P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:41.613970 19914 raft_consensus.cc:493] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:41.648742 16817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2372981006d4b218215c28964dea623" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:20:41.655542 16817 raft_consensus.cc:3060] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:20:41.652067 20257 consensus_queue.cc:237] T 84f2da98864b4425bf7c0055951fdd4b P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:41.555903 20237 consensus_queue.cc:1059] T c0eab25ce4c64e018a041beeda2867db P 647a343a542d4a139fe106c19095129b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.168s
I20260709 13:20:41.663687 16825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9944d5e4b8c4109af9955c4f08c9095" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:41.664265 16825 raft_consensus.cc:2468] T c9944d5e4b8c4109af9955c4f08c9095 P cb82cc7bd6db46d980c75237a1402860 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 10.
I20260709 13:20:41.695303 16145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cc959fb7f704fba9009c0595467d074" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 11 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:41.706635 16220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74ca8140f35c4ae29e8a664f6bdfc829" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:41.710088 16221 raft_consensus.cc:3055] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:20:41.710649 16221 raft_consensus.cc:740] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:20:41.666921 20197 raft_consensus.cc:493] T 02838c86531d4f98bf58d1c22b8baf77 P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
W20260709 13:20:41.722479 16263 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:41.717942 16220 raft_consensus.cc:2468] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 10.
W20260709 13:20:41.718308 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 219044us
I20260709 13:20:41.610356 20200 consensus_queue.cc:237] T c2f5f89186204242bf377f7c42b2e2da P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:41.726804 16146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d4ab870a05347868956aa8b841b2981" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 12 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:41.733385 16144 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46224 (request call id 589) took 1222 ms. Trace:
I20260709 13:20:41.703231 20235 raft_consensus.cc:493] T d16dfad0c3b540e5b4b86492ddf87208 P 647a343a542d4a139fe106c19095129b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:41.703313 20237 raft_consensus.cc:493] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:41.667281 20189 consensus_queue.cc:1048] T 18ad7e3af5504571a69d10c00750a01c P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.043s
I20260709 13:20:41.665795 19914 raft_consensus.cc:515] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } }
I20260709 13:20:41.680970 19941 consensus_queue.cc:1048] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.046s
W20260709 13:20:41.728969 20021 consensus_meta.cc:287] T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b: Time spent flushing consensus metadata: real 0.723s	user 0.011s	sys 0.001s
I20260709 13:20:41.750317 16144 rpcz_store.cc:276] 0709 13:20:40.499824 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:40.502760 (+  2936us) service_pool.cc:224] Handling call
0709 13:20:41.722356 (+1219596us) spinlock_profiling.cc:243] Waited 133 ms on lock 0x7b54003d9fd4. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:20:41.722502 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":306814208}
I20260709 13:20:41.675383 19726 consensus_queue.cc:1059] T 8158bbd17b994d2eb96f560bd0f1b88d P 647a343a542d4a139fe106c19095129b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.076s
I20260709 13:20:41.758992 16191 leader_election.cc:304] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
W20260709 13:20:41.764124 16789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 267092us
I20260709 13:20:41.770445 16789 leader_election.cc:304] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:20:41.783663 16790 leader_election.cc:304] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac
I20260709 13:20:41.795101 16145 raft_consensus.cc:1240] T 8a878999c0024d2abe517484180290fc P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 8. Current term is 9. Ops: []
I20260709 13:20:41.795693 16293 raft_consensus.cc:1240] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 10. Current term is 11. Ops: []
W20260709 13:20:41.793883 16115 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:20:41.834353 16115 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:20:41.857249 20258 log.cc:927] Time spent T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860: Append to log took a long time: real 0.204s	user 0.000s	sys 0.001s
I20260709 13:20:41.859704 20260 consensus_queue.cc:1048] T 18ad7e3af5504571a69d10c00750a01c P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.067s
I20260709 13:20:41.861271 16142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87ac0d1f7e8a419f9bcbc04aee0e067e" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
W20260709 13:20:42.035634 16789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 233790us
I20260709 13:20:41.758780 20237 raft_consensus.cc:515] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:41.758922 20235 raft_consensus.cc:515] T d16dfad0c3b540e5b4b86492ddf87208 P 647a343a542d4a139fe106c19095129b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:41.867910 16221 consensus_queue.cc:260] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:41.785145 20197 raft_consensus.cc:515] T 02838c86531d4f98bf58d1c22b8baf77 P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:41.749187 20107 consensus_queue.cc:1059] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.044s
I20260709 13:20:41.859236 20253 raft_consensus.cc:493] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:20:41.958359 20181 leader_election.cc:290] T b6f07842d3aa47858df5665874c86f3a P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 12 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
W20260709 13:20:41.959373 20255 log.cc:927] Time spent T 02838c86531d4f98bf58d1c22b8baf77 P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 0.292s	user 0.007s	sys 0.000s
I20260709 13:20:41.762758 19990 raft_consensus.cc:493] T 26354361e9b044cea1c2b86c4882fe92 P cb82cc7bd6db46d980c75237a1402860 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:20:41.803382 16265 leader_election.cc:304] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860
I20260709 13:20:41.858047 20230 consensus_queue.cc:1059] T e5ad4eb281564936a59f41e387a47156 P 647a343a542d4a139fe106c19095129b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.100s
W20260709 13:20:42.030068 16115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 181084us
W20260709 13:20:42.072851 16115 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:20:42.073475 16115 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:41.972556 16145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5ad4eb281564936a59f41e387a47156" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:42.036175 16142 raft_consensus.cc:2410] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 for term 11 because replica has last-logged OpId of term: 10 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 7.
I20260709 13:20:42.050493 16144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2794af5b99984efebdee564035841cb0" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
W20260709 13:20:42.058432 16190 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:42.061538 16221 raft_consensus.cc:3060] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:20:42.035382 19673 consensus_meta.cc:287] T b87147731a68400e8cce9b879daa7c5d P cb82cc7bd6db46d980c75237a1402860: Time spent flushing consensus metadata: real 0.596s	user 0.000s	sys 0.021s
I20260709 13:20:42.065625 16817 raft_consensus.cc:2468] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 11.
W20260709 13:20:42.042737 19982 consensus_meta.cc:287] T 5a96373bb0e14dce8a266b2fc6c58507 P 647a343a542d4a139fe106c19095129b: Time spent flushing consensus metadata: real 0.734s	user 0.018s	sys 0.000s
W20260709 13:20:42.066924 20261 log.cc:927] Time spent T 8d4ab870a05347868956aa8b841b2981 P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 0.306s	user 0.000s	sys 0.005s
I20260709 13:20:42.067242 20107 raft_consensus.cc:3055] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:20:42.067744 16291 raft_consensus.cc:1240] T c4f65cdf00b748bb963ee6e81ef43b79 P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 11. Current term is 12. Ops: []
I20260709 13:20:42.041834 19939 raft_consensus.cc:2749] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:20:42.246626 16115 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:42.116060 16146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806d7690c2b644959152074bb2cab0a7" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
W20260709 13:20:42.078691 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 277430us
I20260709 13:20:42.314596 16147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e9b2bf7011f4ae688a79d7bb5a447b4" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
W20260709 13:20:42.301743 16789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 263948us
W20260709 13:20:42.090600 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 316231us
I20260709 13:20:42.078498 19990 raft_consensus.cc:515] T 26354361e9b044cea1c2b86c4882fe92 P cb82cc7bd6db46d980c75237a1402860 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:42.352715 20271 raft_consensus.cc:2804] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:20:42.352061 16147 raft_consensus.cc:2393] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 8: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
W20260709 13:20:42.145416 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 390803us
I20260709 13:20:42.353257 20271 raft_consensus.cc:493] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:42.089960 16296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f2da98864b4425bf7c0055951fdd4b" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
W20260709 13:20:42.321214 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 554364us
I20260709 13:20:42.355326 16146 raft_consensus.cc:2410] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 for term 10 because replica has last-logged OpId of term: 9 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 7.
I20260709 13:20:42.355336 16191 leader_election.cc:304] T fffab26427e24d398e2695aad40ca134 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
I20260709 13:20:42.351939 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fffab26427e24d398e2695aad40ca134" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
W20260709 13:20:42.338395 20256 log.cc:927] Time spent T e4dc6b0541ca431ebc757beac00521f1 P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 0.333s	user 0.000s	sys 0.005s
I20260709 13:20:42.154121 20197 leader_election.cc:290] T 02838c86531d4f98bf58d1c22b8baf77 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
W20260709 13:20:42.370549 16115 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:20:42.355084 19726 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.552s	user 0.007s	sys 0.008s
W20260709 13:20:42.352192 20263 log.cc:927] Time spent T 08c74490c40243a88e62d1760b5cd814 P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 0.314s	user 0.000s	sys 0.001s
I20260709 13:20:42.107853 20272 raft_consensus.cc:493] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:42.379714 16144 raft_consensus.cc:1275] T d16dfad0c3b540e5b4b86492ddf87208 P 647a343a542d4a139fe106c19095129b [term 10 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 6. (term mismatch)
I20260709 13:20:42.354039 16265 leader_election.cc:304] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860
I20260709 13:20:42.339962 20275 raft_consensus.cc:493] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:42.383198 16296 raft_consensus.cc:3060] T 84f2da98864b4425bf7c0055951fdd4b P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Advancing to term 9
I20260709 13:20:42.402585 16291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02838c86531d4f98bf58d1c22b8baf77" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:42.384598 16116 leader_election.cc:304] T 829b4ad8c5a54eababeea1181b960d4b P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860
I20260709 13:20:42.385478 16143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6f07842d3aa47858df5665874c86f3a" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:20:42.386071 20280 raft_consensus.cc:2804] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:20:42.386847 16146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2372981006d4b218215c28964dea623" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:20:42.339697 19914 leader_election.cc:290] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:42.387024 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1df93779209748b9860f6d893dd3de53" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:42.354559 20065 raft_consensus.cc:2749] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:20:42.396986 16144 pending_rounds.cc:85] T d16dfad0c3b540e5b4b86492ddf87208 P 647a343a542d4a139fe106c19095129b: Aborting all ops after (but not including) 5
I20260709 13:20:42.408288 16144 pending_rounds.cc:107] T d16dfad0c3b540e5b4b86492ddf87208 P 647a343a542d4a139fe106c19095129b: Aborting uncommitted NO_OP operation due to leader change: 9.6
I20260709 13:20:42.408614 16144 raft_consensus.cc:2889] T d16dfad0c3b540e5b4b86492ddf87208 P 647a343a542d4a139fe106c19095129b [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:20:42.354557 20107 raft_consensus.cc:740] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
W20260709 13:20:42.385499 19726 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.591s	user 0.008s	sys 0.008s
I20260709 13:20:42.405076 16146 raft_consensus.cc:3060] T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:20:42.365366 20271 raft_consensus.cc:3060] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:20:42.397646 16142 raft_consensus.cc:1275] T 08c74490c40243a88e62d1760b5cd814 P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 13:20:42.414386 16143 raft_consensus.cc:1275] T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b [term 10 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 5. (term mismatch)
I20260709 13:20:42.381330 19939 raft_consensus.cc:2804] T fffab26427e24d398e2695aad40ca134 P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:20:42.386833 20248 raft_consensus.cc:3055] T e5ad4eb281564936a59f41e387a47156 P 647a343a542d4a139fe106c19095129b [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:20:42.387529 20197 raft_consensus.cc:493] T a378d7f796f94615b3b347119a9880fd P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:42.386927 20252 raft_consensus.cc:493] T 746aabf856ab401c9ef5e859371e7b67 P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:20:42.091486 19754 consensus_queue.cc:1059] T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.339s
I20260709 13:20:42.414781 16143 pending_rounds.cc:85] T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b: Aborting all ops after (but not including) 4
W20260709 13:20:42.415179 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 61234.5us
I20260709 13:20:42.354627 19673 raft_consensus.cc:515] T b87147731a68400e8cce9b879daa7c5d P cb82cc7bd6db46d980c75237a1402860 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:42.421823 16297 raft_consensus.cc:1240] T ada5a004ba7c46819051e3ccec71ffdb P 90f3da57796b488c9087a2f523adf045 [term 11 LEADER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 10. Current term is 11. Ops: [10.10-10.10]
I20260709 13:20:42.417853 20248 raft_consensus.cc:740] T e5ad4eb281564936a59f41e387a47156 P 647a343a542d4a139fe106c19095129b [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:20:42.428902 20197 raft_consensus.cc:515] T a378d7f796f94615b3b347119a9880fd P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:42.429808 20252 raft_consensus.cc:515] T 746aabf856ab401c9ef5e859371e7b67 P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:42.427075 16142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18ad7e3af5504571a69d10c00750a01c" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
W20260709 13:20:42.427102 20277 log.cc:927] Time spent T 68a6838d82ef4cac83507f906139d90f P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 0.080s	user 0.000s	sys 0.000s
I20260709 13:20:42.095980 20253 raft_consensus.cc:515] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:42.432852 16147 raft_consensus.cc:1275] T 5a96373bb0e14dce8a266b2fc6c58507 P 647a343a542d4a139fe106c19095129b [term 10 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 9 index: 4. Preceding OpId from leader: term: 10 index: 5. (index mismatch)
W20260709 13:20:42.435694 20266 log.cc:927] Time spent T 829b4ad8c5a54eababeea1181b960d4b P cb82cc7bd6db46d980c75237a1402860: Append to log took a long time: real 0.375s	user 0.000s	sys 0.001s
I20260709 13:20:42.435971 20237 leader_election.cc:290] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:42.457104 16146 raft_consensus.cc:2468] T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 11.
I20260709 13:20:42.437759 20278 raft_consensus.cc:493] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
W20260709 13:20:42.439280 20264 log.cc:927] Time spent T 1df93779209748b9860f6d893dd3de53 P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 0.688s	user 0.004s	sys 0.004s
I20260709 13:20:42.440825 20260 raft_consensus.cc:493] T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:42.441450 20281 raft_consensus.cc:493] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:20:42.441960 20250 log.cc:927] Time spent T b87147731a68400e8cce9b879daa7c5d P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 1.037s	user 0.002s	sys 0.000s
W20260709 13:20:42.445415 20244 log.cc:927] Time spent T 2794af5b99984efebdee564035841cb0 P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 1.016s	user 0.002s	sys 0.000s
I20260709 13:20:42.446405 20045 raft_consensus.cc:3055] T c0eab25ce4c64e018a041beeda2867db P 647a343a542d4a139fe106c19095129b [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:20:42.492501 20045 raft_consensus.cc:740] T c0eab25ce4c64e018a041beeda2867db P 647a343a542d4a139fe106c19095129b [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:20:42.492748 16298 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40474 (request call id 649) took 1883 ms. Trace:
I20260709 13:20:42.493389 20045 consensus_queue.cc:260] T c0eab25ce4c64e018a041beeda2867db P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:42.493317 16298 rpcz_store.cc:276] 0709 13:20:40.608931 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:40.956695 (+347764us) service_pool.cc:224] Handling call
0709 13:20:41.090666 (+133971us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:20:41.215855 (+125189us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:41.215902 (+    47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:20:41.219536 (+  3634us) log.cc:844] Serialized 26 byte log entry
0709 13:20:41.432128 (+212592us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:20:41.432187 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:41.432302 (+   115us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:20:42.444742 (+1012440us) spinlock_profiling.cc:243] Waited 47.3 ms on lock 0x7b4400022b4c. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69ad8fadb1 00007f69ad998c85 00007f69ad9996a2 00007f69b070b1db 00007f69b0706aaf 00007f69b06fa86e 00007f69b06f9bb6 00007f69b06f97d8 00007f69b0703221 00007f69b07031d9 00007f69b0703169
0709 13:20:42.492332 (+ 47590us) raft_consensus.cc:1685] finished
0709 13:20:42.492366 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:42.492647 (+   281us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":108800384,"thread_start_us":199188,"threads_started":1,"wal-append.queue_time_us":202340}
I20260709 13:20:42.449563 16143 pending_rounds.cc:107] T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b: Aborting uncommitted NO_OP operation due to leader change: 7.5
I20260709 13:20:42.414193 20181 raft_consensus.cc:493] T af57996a8be84d398f0f023e8d08ef77 P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:42.495561 20181 raft_consensus.cc:515] T af57996a8be84d398f0f023e8d08ef77 P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:42.495988 20107 consensus_queue.cc:260] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:42.496496 20281 raft_consensus.cc:515] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:42.497318 20107 raft_consensus.cc:3060] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:20:42.497977 20181 leader_election.cc:290] T af57996a8be84d398f0f023e8d08ef77 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:42.498764 20281 leader_election.cc:290] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:42.407914 20280 raft_consensus.cc:493] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:20:42.499526 20280 raft_consensus.cc:3060] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:20:42.500475 20252 leader_election.cc:290] T 746aabf856ab401c9ef5e859371e7b67 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:42.503352 19990 leader_election.cc:290] T 26354361e9b044cea1c2b86c4882fe92 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:42.508263 16150 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40844 (request call id 2296) took 1090 ms. Trace:
I20260709 13:20:42.508633 16150 rpcz_store.cc:276] 0709 13:20:41.417750 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:41.417921 (+   171us) service_pool.cc:224] Handling call
0709 13:20:41.439897 (+ 21976us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:20:41.496981 (+ 57084us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:41.497010 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:20:41.499540 (+  2530us) log.cc:844] Serialized 26 byte log entry
0709 13:20:41.946500 (+446960us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:20:41.946555 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:41.946631 (+    76us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:20:42.508006 (+561375us) raft_consensus.cc:1685] finished
0709 13:20:42.508032 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:42.508185 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":11400,"raft.queue_time_us":43555,"raft.run_cpu_time_us":133,"raft.run_wall_time_us":134,"thread_start_us":363374,"threads_started":1,"wal-append.queue_time_us":537675}
I20260709 13:20:42.495340 16143 raft_consensus.cc:2889] T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:20:42.510341 16143 pending_rounds.cc:107] T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b: Aborting uncommitted NO_OP operation due to leader change: 9.6
I20260709 13:20:42.510666 16143 raft_consensus.cc:2889] T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:20:42.447628 20285 consensus_queue.cc:1048] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.088s
W20260709 13:20:42.512045 16114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:20:42.516675 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 132201us
I20260709 13:20:42.517637 16116 leader_election.cc:304] T 08c74490c40243a88e62d1760b5cd814 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 90f3da57796b488c9087a2f523adf045
I20260709 13:20:42.518579 20237 raft_consensus.cc:2804] T 08c74490c40243a88e62d1760b5cd814 P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:20:42.519054 20237 raft_consensus.cc:493] T 08c74490c40243a88e62d1760b5cd814 P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:42.519543 20237 raft_consensus.cc:3060] T 08c74490c40243a88e62d1760b5cd814 P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:20:42.520406 16147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a9de347c711477e87c582d003f36925" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:42.521081 16147 raft_consensus.cc:2468] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 12.
I20260709 13:20:42.521677 20197 leader_election.cc:290] T a378d7f796f94615b3b347119a9880fd P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:42.523762 16225 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57600 (request call id 1390) took 1123 ms. Trace:
I20260709 13:20:42.524169 16225 rpcz_store.cc:276] 0709 13:20:41.400769 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:41.400890 (+   121us) service_pool.cc:224] Handling call
0709 13:20:41.415797 (+ 14907us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:20:41.458190 (+ 42393us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:41.458211 (+    21us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:20:41.474115 (+ 15904us) log.cc:844] Serialized 52 byte log entry
0709 13:20:41.614910 (+140795us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:20:41.953996 (+339086us) log.cc:844] Serialized 14 byte log entry
0709 13:20:42.142909 (+188913us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:42.142996 (+    87us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:20:42.523454 (+380458us) raft_consensus.cc:1685] finished
0709 13:20:42.523482 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:42.523668 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3540,"prepare.queue_time_us":772262,"prepare.run_cpu_time_us":34,"prepare.run_wall_time_us":36,"raft.queue_time_us":292759,"raft.run_cpu_time_us":357,"raft.run_wall_time_us":106861,"thread_start_us":631856,"threads_started":4,"wal-append.queue_time_us":281550}
I20260709 13:20:42.527479 20253 leader_election.cc:290] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:42.419773 20270 consensus_queue.cc:1048] T 74ca8140f35c4ae29e8a664f6bdfc829 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.071s
I20260709 13:20:42.533437 20278 raft_consensus.cc:515] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:42.448428 20235 leader_election.cc:290] T d16dfad0c3b540e5b4b86492ddf87208 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:42.453233 16296 raft_consensus.cc:2468] T 84f2da98864b4425bf7c0055951fdd4b P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 9.
W20260709 13:20:42.447031 20229 consensus_meta.cc:287] T 18ad7e3af5504571a69d10c00750a01c P 647a343a542d4a139fe106c19095129b: Time spent flushing consensus metadata: real 0.939s	user 0.009s	sys 0.001s
I20260709 13:20:42.479310 16297 raft_consensus.cc:1275] T 8158bbd17b994d2eb96f560bd0f1b88d P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 13:20:42.464033 19914 raft_consensus.cc:3060] T 829b4ad8c5a54eababeea1181b960d4b P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:20:42.448294 19915 raft_consensus.cc:3060] T 5a959dbecc2648eb85d1cc550f5a3245 P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:20:42.485158 16291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9944d5e4b8c4109af9955c4f08c9095" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
W20260709 13:20:42.487613 16115 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:42.494675 20045 raft_consensus.cc:3060] T c0eab25ce4c64e018a041beeda2867db P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:20:42.495646 20260 raft_consensus.cc:515] T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:42.519349 16265 leader_election.cc:304] T cdbd78acb4cf46298e0de8ba0d130e19 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: cb82cc7bd6db46d980c75237a1402860
W20260709 13:20:42.458617 16789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 103692us
I20260709 13:20:42.477170 16221 raft_consensus.cc:1275] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 11 index: 10. (term mismatch)
I20260709 13:20:42.614328 16221 pending_rounds.cc:85] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac: Aborting all ops after (but not including) 9
I20260709 13:20:42.614665 16221 pending_rounds.cc:107] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac: Aborting uncommitted NO_OP operation due to leader change: 10.10
I20260709 13:20:42.614985 16221 raft_consensus.cc:2889] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:20:42.510950 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c61a39fd4f84424b129c1a6f9ba404d" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:42.476133 16292 raft_consensus.cc:1275] T 68a6838d82ef4cac83507f906139d90f P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
W20260709 13:20:42.596944 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 77900.4us
I20260709 13:20:42.469923 20065 raft_consensus.cc:2749] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:20:42.600694 16300 raft_consensus.cc:1275] T d16dfad0c3b540e5b4b86492ddf87208 P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:20:42.601634 19673 leader_election.cc:290] T b87147731a68400e8cce9b879daa7c5d P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 12 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:42.603328 20094 raft_consensus.cc:3055] T 8cc959fb7f704fba9009c0595467d074 P 647a343a542d4a139fe106c19095129b [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:20:42.408751 20272 raft_consensus.cc:515] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:42.457719 20231 consensus_queue.cc:1059] T 8158bbd17b994d2eb96f560bd0f1b88d P 647a343a542d4a139fe106c19095129b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.078s
I20260709 13:20:42.611461 16143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f2da98864b4425bf7c0055951fdd4b" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:42.616572 16150 raft_consensus.cc:2468] T 9c61a39fd4f84424b129c1a6f9ba404d P 647a343a542d4a139fe106c19095129b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 10.
I20260709 13:20:42.622628 16147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26354361e9b044cea1c2b86c4882fe92" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:42.625279 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6f07842d3aa47858df5665874c86f3a" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:20:42.480788 20283 raft_consensus.cc:493] T 57a744f5aaf944678a63469cbf514015 P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:20:42.631721 20283 raft_consensus.cc:515] T 57a744f5aaf944678a63469cbf514015 P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:42.632795 20285 raft_consensus.cc:493] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:42.633195 16824 raft_consensus.cc:1275] T c9944d5e4b8c4109af9955c4f08c9095 P cb82cc7bd6db46d980c75237a1402860 [term 10 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:20:42.633868 20283 leader_election.cc:290] T 57a744f5aaf944678a63469cbf514015 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
W20260709 13:20:42.627707 16115 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:42.633576 20285 raft_consensus.cc:515] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:42.418730 20275 raft_consensus.cc:515] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:42.636776 20275 leader_election.cc:290] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:42.642089 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8638ee363273426db1393dd81e5c7b72" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:42.641655 20282 consensus_queue.cc:1048] T c9944d5e4b8c4109af9955c4f08c9095 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:20:42.642935 16298 raft_consensus.cc:2410] T 8638ee363273426db1393dd81e5c7b72 P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac for term 12 because replica has last-logged OpId of term: 10 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 9.
I20260709 13:20:42.644315 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1df93779209748b9860f6d893dd3de53" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:42.677954 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4f65cdf00b748bb963ee6e81ef43b79" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:20:42.679384 20187 raft_consensus.cc:493] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Starting leader election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:20:42.680153 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:42.681303 16149 raft_consensus.cc:2393] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 11: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
I20260709 13:20:42.648994 16299 raft_consensus.cc:1275] T 8a878999c0024d2abe517484180290fc P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 13:20:42.649664 20254 leader_election.cc:290] T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:42.684463 16296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66ae176961b74813a58bc15c9c4c6f2a" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:42.685343 16296 raft_consensus.cc:2468] T 66ae176961b74813a58bc15c9c4c6f2a P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 12.
I20260709 13:20:42.687449 16143 raft_consensus.cc:2368] T 84f2da98864b4425bf7c0055951fdd4b P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 for earlier term 8. Current term is 9.
I20260709 13:20:42.654798 20285 leader_election.cc:290] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:42.658222 16190 leader_election.cc:304] T 1df93779209748b9860f6d893dd3de53 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
W20260709 13:20:42.695922 16294 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.539s	user 0.013s	sys 0.030s
W20260709 13:20:42.696242 16294 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.539s	user 0.013s	sys 0.030s
I20260709 13:20:42.696082 16190 leader_election.cc:304] T 02838c86531d4f98bf58d1c22b8baf77 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:20:42.660167 20275 consensus_queue.cc:1059] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: INVALID_TERM, Last received: 11.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
W20260709 13:20:42.662752 16263 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:42.671510 20267 raft_consensus.cc:493] T 8638ee363273426db1393dd81e5c7b72 P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:20:42.674376 20281 raft_consensus.cc:493] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:42.675228 20253 raft_consensus.cc:493] T 7c2095a5f4ad49408dbc7f69c606120d P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:20:42.629396 20094 raft_consensus.cc:740] T 8cc959fb7f704fba9009c0595467d074 P 647a343a542d4a139fe106c19095129b [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:20:42.691136 16817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74ca8140f35c4ae29e8a664f6bdfc829" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:42.706955 16299 raft_consensus.cc:1275] T a378d7f796f94615b3b347119a9880fd P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 13:20:42.692169 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a96373bb0e14dce8a266b2fc6c58507" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 10 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
W20260709 13:20:42.621975 20223 log.cc:927] Time spent T 8d4ab870a05347868956aa8b841b2981 P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.140s	user 0.001s	sys 0.000s
I20260709 13:20:42.639753 16292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7ca61592e848ea9b0fb81a2ce33cdc" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
W20260709 13:20:42.697324 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 75783.4us
I20260709 13:20:42.710419 16816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a378d7f796f94615b3b347119a9880fd" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:42.711582 16292 raft_consensus.cc:2393] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 13: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
W20260709 13:20:42.697618 16115 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:42.625743 20197 raft_consensus.cc:2804] T cdbd78acb4cf46298e0de8ba0d130e19 P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:20:42.680084 20187 raft_consensus.cc:3060] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:20:42.588581 19990 consensus_queue.cc:1048] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.100s
I20260709 13:20:42.702751 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434bff3082d141838068f323bdb0d1ed" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:42.704120 16825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66ae176961b74813a58bc15c9c4c6f2a" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
W20260709 13:20:42.645901 20262 log.cc:927] Time spent T 5a96373bb0e14dce8a266b2fc6c58507 P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 0.800s	user 0.009s	sys 0.006s
I20260709 13:20:42.705785 20065 raft_consensus.cc:493] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:20:42.706845 16296 raft_consensus.cc:1275] T d2ab0ac8347641e6955e9bce7d90c1a2 P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:20:42.691622 16824 raft_consensus.cc:3060] T 806d7690c2b644959152074bb2cab0a7 P cb82cc7bd6db46d980c75237a1402860 [term 9 LEARNER]: Advancing to term 10
I20260709 13:20:42.693756 19673 raft_consensus.cc:493] T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:42.716614 16149 raft_consensus.cc:2468] T 5a96373bb0e14dce8a266b2fc6c58507 P 647a343a542d4a139fe106c19095129b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 10.
I20260709 13:20:42.720374 16143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18ad7e3af5504571a69d10c00750a01c" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:42.598640 20229 raft_consensus.cc:515] T 18ad7e3af5504571a69d10c00750a01c P 647a343a542d4a139fe106c19095129b [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } }
I20260709 13:20:42.723667 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26354361e9b044cea1c2b86c4882fe92" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
W20260709 13:20:42.725580 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 247315us
W20260709 13:20:42.711787 20246 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.341s	user 0.019s	sys 0.015s
I20260709 13:20:42.734705 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9944d5e4b8c4109af9955c4f08c9095" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:42.737994 16297 raft_consensus.cc:2468] T 26354361e9b044cea1c2b86c4882fe92 P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 8.
I20260709 13:20:42.738091 16147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0eab25ce4c64e018a041beeda2867db" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:20:42.707595 20252 raft_consensus.cc:2749] T 1df93779209748b9860f6d893dd3de53 P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:20:42.735903 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87147731a68400e8cce9b879daa7c5d" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:20:42.709190 19939 raft_consensus.cc:697] T fffab26427e24d398e2695aad40ca134 P 1ca2725d290e47d68b1a4b609c9816ac [term 9 LEADER]: Becoming Leader. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:20:42.742246 16817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57a744f5aaf944678a63469cbf514015" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:42.695719 19726 consensus_queue.cc:1059] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.031s
I20260709 13:20:42.717686 20197 raft_consensus.cc:493] T cdbd78acb4cf46298e0de8ba0d130e19 P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Starting leader election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
W20260709 13:20:42.737660 20246 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.363s	user 0.019s	sys 0.015s
I20260709 13:20:42.750010 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e9b2bf7011f4ae688a79d7bb5a447b4" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:20:42.751955 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a9de347c711477e87c582d003f36925" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
W20260709 13:20:42.730811 19923 consensus_meta.cc:287] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860: Time spent flushing consensus metadata: real 1.325s	user 0.009s	sys 0.005s
W20260709 13:20:42.753523 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 359042us
I20260709 13:20:42.761885 16143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5ad4eb281564936a59f41e387a47156" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:20:42.763887 16142 raft_consensus.cc:2393] T 18ad7e3af5504571a69d10c00750a01c P 647a343a542d4a139fe106c19095129b [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 10: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
W20260709 13:20:42.768635 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 56182.1us
I20260709 13:20:42.772936 16149 raft_consensus.cc:2393] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 8: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
I20260709 13:20:42.763473 16150 raft_consensus.cc:2468] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 12.
I20260709 13:20:42.765009 16292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a9de347c711477e87c582d003f36925" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:42.774008 16292 raft_consensus.cc:2468] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 12.
W20260709 13:20:42.772367 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 44651.5us
I20260709 13:20:42.743752 20200 raft_consensus.cc:3055] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:20:42.740299 20275 raft_consensus.cc:2749] T 02838c86531d4f98bf58d1c22b8baf77 P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:20:42.790300 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d16dfad0c3b540e5b4b86492ddf87208" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:42.797737 16299 raft_consensus.cc:2468] T d16dfad0c3b540e5b4b86492ddf87208 P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 10.
I20260709 13:20:42.728820 20253 raft_consensus.cc:515] T 7c2095a5f4ad49408dbc7f69c606120d P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:42.711364 20281 raft_consensus.cc:515] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } }
I20260709 13:20:42.799657 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e9b2bf7011f4ae688a79d7bb5a447b4" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:42.795449 16116 leader_election.cc:304] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860
I20260709 13:20:42.764474 20197 raft_consensus.cc:3060] T cdbd78acb4cf46298e0de8ba0d130e19 P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:20:42.804276 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26354361e9b044cea1c2b86c4882fe92" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:42.734306 20065 raft_consensus.cc:515] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:20:42.815433 16266 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:42.815659 16143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87ac0d1f7e8a419f9bcbc04aee0e067e" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:20:42.737655 19673 raft_consensus.cc:515] T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:42.822954 16265 leader_election.cc:400] T 84f2da98864b4425bf7c0055951fdd4b P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 8 pre-election: Vote denied by peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943) with higher term. Message: Invalid argument: T 84f2da98864b4425bf7c0055951fdd4b P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 for earlier term 8. Current term is 9.
I20260709 13:20:42.780238 19923 raft_consensus.cc:2749] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:20:42.826000 16265 leader_election.cc:403] T 84f2da98864b4425bf7c0055951fdd4b P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 8 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:20:42.716317 20267 raft_consensus.cc:515] T 8638ee363273426db1393dd81e5c7b72 P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:42.814782 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87147731a68400e8cce9b879daa7c5d" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:20:42.798774 20192 raft_consensus.cc:2749] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:20:42.789606 20200 raft_consensus.cc:740] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:20:42.798866 20257 raft_consensus.cc:493] T 829b4ad8c5a54eababeea1181b960d4b P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:42.841742 16824 raft_consensus.cc:1275] T 806d7690c2b644959152074bb2cab0a7 P cb82cc7bd6db46d980c75237a1402860 [term 10 LEARNER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 13:20:42.832669 16292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806d7690c2b644959152074bb2cab0a7" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
W20260709 13:20:42.843412 16789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 60629.6us
I20260709 13:20:42.851430 16144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e9b2bf7011f4ae688a79d7bb5a447b4" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:42.852223 20229 leader_election.cc:290] T 18ad7e3af5504571a69d10c00750a01c P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 10 election: Requested vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:20:42.852737 20275 consensus_queue.cc:1048] T 68a6838d82ef4cac83507f906139d90f P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:20:42.853590 20269 consensus_queue.cc:1048] T d16dfad0c3b540e5b4b86492ddf87208 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:20:42.854643 20260 leader_election.cc:290] T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
W20260709 13:20:42.855242 16264 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.542s	user 0.002s	sys 0.010s
W20260709 13:20:42.869025 16264 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.672s	user 0.003s	sys 0.011s
I20260709 13:20:42.856535 16221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af57996a8be84d398f0f023e8d08ef77" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 12 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:42.870186 16221 raft_consensus.cc:2410] T af57996a8be84d398f0f023e8d08ef77 P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 for term 12 because replica has last-logged OpId of term: 9 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 7.
W20260709 13:20:42.860859 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 60601.9us
I20260709 13:20:42.861904 16144 raft_consensus.cc:2393] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 8: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
I20260709 13:20:42.860450 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e9b2bf7011f4ae688a79d7bb5a447b4" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:42.789234 20252 consensus_queue.cc:1048] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.044s
I20260709 13:20:42.859740 16789 leader_election.cc:304] T 26354361e9b044cea1c2b86c4882fe92 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:20:42.943506 20303 raft_consensus.cc:493] T 8a878999c0024d2abe517484180290fc P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:42.943902 20288 raft_consensus.cc:493] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:42.803721 20094 consensus_queue.cc:260] T 8cc959fb7f704fba9009c0595467d074 P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:42.944378 20288 raft_consensus.cc:515] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:42.944329 20303 raft_consensus.cc:515] T 8a878999c0024d2abe517484180290fc P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:42.945219 20094 raft_consensus.cc:3060] T 8cc959fb7f704fba9009c0595467d074 P 647a343a542d4a139fe106c19095129b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:20:42.945346 16265 leader_election.cc:304] T c0eab25ce4c64e018a041beeda2867db P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:20:42.946323 16265 leader_election.cc:304] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:20:42.946735 20303 leader_election.cc:290] T 8a878999c0024d2abe517484180290fc P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:20:42.872081 16220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9944d5e4b8c4109af9955c4f08c9095" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:42.947149 16265 leader_election.cc:304] T 1df93779209748b9860f6d893dd3de53 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:20:42.944370 16144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5ad4eb281564936a59f41e387a47156" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:42.939997 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c2095a5f4ad49408dbc7f69c606120d" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:42.947726 16143 raft_consensus.cc:2410] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 for term 11 because replica has last-logged OpId of term: 10 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 7.
I20260709 13:20:42.948148 16220 raft_consensus.cc:2468] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 10.
W20260709 13:20:42.979766 16115 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:43.008976 16265 leader_election.cc:304] T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: cb82cc7bd6db46d980c75237a1402860
W20260709 13:20:43.010807 16789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 151143us
I20260709 13:20:42.869316 20254 raft_consensus.cc:2749] T 84f2da98864b4425bf7c0055951fdd4b P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Leader pre-election lost for term 8. Reason: Vote denied by peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943) with higher term. Message: Invalid argument: T 84f2da98864b4425bf7c0055951fdd4b P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 for earlier term 8. Current term is 9.
I20260709 13:20:42.947700 20183 raft_consensus.cc:2804] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:20:42.959357 20270 consensus_queue.cc:1048] T d2ab0ac8347641e6955e9bce7d90c1a2 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.059s
I20260709 13:20:42.908885 20230 raft_consensus.cc:493] T 5a959dbecc2648eb85d1cc550f5a3245 P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:20:43.007984 20280 consensus_meta.cc:287] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860: Time spent flushing consensus metadata: real 0.508s	user 0.010s	sys 0.000s
I20260709 13:20:42.865495 20257 raft_consensus.cc:515] T 829b4ad8c5a54eababeea1181b960d4b P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:42.959051 20267 leader_election.cc:290] T 8638ee363273426db1393dd81e5c7b72 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:43.018453 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af57996a8be84d398f0f023e8d08ef77" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:43.018757 20080 consensus_queue.cc:1048] T c9944d5e4b8c4109af9955c4f08c9095 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:20:43.020018 16142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5718c623ebee4d2bb770f0f9ddb2f164" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:43.020643 20268 raft_consensus.cc:2804] T 26354361e9b044cea1c2b86c4882fe92 P cb82cc7bd6db46d980c75237a1402860 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:20:42.955544 20200 consensus_queue.cc:260] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:43.021946 16144 raft_consensus.cc:1240] T 18ad7e3af5504571a69d10c00750a01c P 647a343a542d4a139fe106c19095129b [term 10 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 9. Current term is 10. Ops: [9.7-9.7]
W20260709 13:20:42.956041 20301 log.cc:927] Time spent T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 0.084s	user 0.000s	sys 0.000s
I20260709 13:20:43.025626 20236 raft_consensus.cc:3055] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:20:43.062155 16142 raft_consensus.cc:2393] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 10: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
I20260709 13:20:43.063380 16790 leader_election.cc:304] T ee9ed67006144797b1de6ea8c2013a68 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:20:42.947516 20289 raft_consensus.cc:2804] T c0eab25ce4c64e018a041beeda2867db P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:20:43.038009 16817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8638ee363273426db1393dd81e5c7b72" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:43.026523 16115 leader_election.cc:304] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:20:43.066355 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746aabf856ab401c9ef5e859371e7b67" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
W20260709 13:20:43.068418 16191 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:20:43.058315 20237 consensus_meta.cc:287] T 08c74490c40243a88e62d1760b5cd814 P 647a343a542d4a139fe106c19095129b: Time spent flushing consensus metadata: real 0.537s	user 0.007s	sys 0.004s
I20260709 13:20:43.072577 16144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806d7690c2b644959152074bb2cab0a7" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:20:43.075260 16144 raft_consensus.cc:2410] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 for term 10 because replica has last-logged OpId of term: 9 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 7.
I20260709 13:20:43.076058 20080 consensus_queue.cc:1048] T ada5a004ba7c46819051e3ccec71ffdb P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:20:43.076912 20310 raft_consensus.cc:493] T 08c74490c40243a88e62d1760b5cd814 P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:20:43.077471 20310 raft_consensus.cc:515] T 08c74490c40243a88e62d1760b5cd814 P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:42.960256 19939 consensus_queue.cc:237] T fffab26427e24d398e2695aad40ca134 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:20:43.080272 20271 consensus_meta.cc:287] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860: Time spent flushing consensus metadata: real 0.642s	user 0.005s	sys 0.005s
I20260709 13:20:43.080684 20271 raft_consensus.cc:515] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:43.055828 20268 raft_consensus.cc:493] T 26354361e9b044cea1c2b86c4882fe92 P cb82cc7bd6db46d980c75237a1402860 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:20:42.944351 20253 leader_election.cc:290] T 7c2095a5f4ad49408dbc7f69c606120d P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:42.994184 20303 raft_consensus.cc:2804] T 1df93779209748b9860f6d893dd3de53 P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:20:42.948879 20065 leader_election.cc:290] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:43.044473 16824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e4bfb9090c8460eb14fa7e0d818e7f1" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
W20260709 13:20:43.046272 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 275719us
I20260709 13:20:43.065138 20073 raft_consensus.cc:493] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:20:43.073959 20237 raft_consensus.cc:515] T 08c74490c40243a88e62d1760b5cd814 P 647a343a542d4a139fe106c19095129b [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:42.946736 20288 leader_election.cc:290] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:43.039475 20260 raft_consensus.cc:493] T c0eab25ce4c64e018a041beeda2867db P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:43.076108 20274 consensus_queue.cc:1048] T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.141s
I20260709 13:20:43.059239 20272 leader_election.cc:290] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:20:43.073969 16142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f5f89186204242bf377f7c42b2e2da" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:43.082880 16790 leader_election.cc:304] T b87147731a68400e8cce9b879daa7c5d P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:20:42.872848 20278 leader_election.cc:290] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
W20260709 13:20:43.135835 20259 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.277s	user 0.013s	sys 0.013s
I20260709 13:20:42.911523 20290 raft_consensus.cc:493] T 8638ee363273426db1393dd81e5c7b72 P cb82cc7bd6db46d980c75237a1402860 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:43.060768 20252 raft_consensus.cc:493] T 5a959dbecc2648eb85d1cc550f5a3245 P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:20:43.088356 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 293028us
I20260709 13:20:43.062338 20236 raft_consensus.cc:740] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:20:43.089416 16220 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8158bbd17b994d2eb96f560bd0f1b88d"
dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
mode: GRACEFUL
new_leader_uuid: "90f3da57796b488c9087a2f523adf045"
 from {username='slave'} at 127.0.0.1:39144
W20260709 13:20:43.138106 20259 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.279s	user 0.014s	sys 0.013s
I20260709 13:20:42.950060 20281 leader_election.cc:290] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:43.096915 20316 consensus_queue.cc:1048] T 74ca8140f35c4ae29e8a664f6bdfc829 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.026s
W20260709 13:20:42.949033 20287 consensus_queue.cc:1175] T 8cc959fb7f704fba9009c0595467d074 P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "1ca2725d290e47d68b1a4b609c9816ac" responder_term: 11 status { last_received { term: 9 index: 7 } last_committed_idx: 5 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 10. Current term is 11. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:20:43.098030 16144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87147731a68400e8cce9b879daa7c5d" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:43.099042 20310 leader_election.cc:290] T 08c74490c40243a88e62d1760b5cd814 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:43.103434 16221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5718c623ebee4d2bb770f0f9ddb2f164" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:43.103863 20315 raft_consensus.cc:493] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:43.046018 20230 raft_consensus.cc:515] T 5a959dbecc2648eb85d1cc550f5a3245 P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:43.106161 20253 consensus_queue.cc:1048] T d16dfad0c3b540e5b4b86492ddf87208 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:20:43.089525 20268 raft_consensus.cc:3060] T 26354361e9b044cea1c2b86c4882fe92 P cb82cc7bd6db46d980c75237a1402860 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:20:43.110836 20237 leader_election.cc:290] T 08c74490c40243a88e62d1760b5cd814 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 10 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:43.087309 20289 raft_consensus.cc:697] T c0eab25ce4c64e018a041beeda2867db P 90f3da57796b488c9087a2f523adf045 [term 10 LEADER]: Becoming Leader. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:20:43.048619 20280 raft_consensus.cc:515] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:43.092677 20303 raft_consensus.cc:493] T 1df93779209748b9860f6d893dd3de53 P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:20:43.071722 20285 consensus_queue.cc:1048] T d2ab0ac8347641e6955e9bce7d90c1a2 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.039s
I20260709 13:20:43.073539 19923 raft_consensus.cc:2749] T ee9ed67006144797b1de6ea8c2013a68 P cb82cc7bd6db46d980c75237a1402860 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:20:43.111841 16142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c61a39fd4f84424b129c1a6f9ba404d" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
W20260709 13:20:43.132766 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 49955.1us
I20260709 13:20:43.240631 16142 raft_consensus.cc:2468] T 9c61a39fd4f84424b129c1a6f9ba404d P 647a343a542d4a139fe106c19095129b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 10.
I20260709 13:20:43.138499 20290 raft_consensus.cc:515] T 8638ee363273426db1393dd81e5c7b72 P cb82cc7bd6db46d980c75237a1402860 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:43.139623 16220 raft_consensus.cc:606] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac [term 9 LEADER]: Received request to transfer leadership to TS 90f3da57796b488c9087a2f523adf045
I20260709 13:20:43.093539 16190 leader_election.cc:304] T b6f07842d3aa47858df5665874c86f3a P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:20:43.142273 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2794af5b99984efebdee564035841cb0" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
W20260709 13:20:43.143394 16266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 108834us
I20260709 13:20:43.149708 20283 raft_consensus.cc:2804] T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:20:43.109812 20260 raft_consensus.cc:515] T c0eab25ce4c64e018a041beeda2867db P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:43.110299 20306 raft_consensus.cc:493] T e5ad4eb281564936a59f41e387a47156 P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:20:43.304807 20282 consensus_queue.cc:1048] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.009s
W20260709 13:20:43.314365 20240 log.cc:927] Time spent T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 1.955s	user 0.010s	sys 0.002s
I20260709 13:20:43.314538 20282 consensus_queue.cc:1243] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Peer cb82cc7bd6db46d980c75237a1402860 log is divergent from this leader: its last log entry 9.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 13:20:43.315275 20290 leader_election.cc:290] T 8638ee363273426db1393dd81e5c7b72 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:43.139269 20236 consensus_queue.cc:260] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:43.317883 16817 raft_consensus.cc:2468] T 8638ee363273426db1393dd81e5c7b72 P cb82cc7bd6db46d980c75237a1402860 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 11.
I20260709 13:20:43.153928 20278 consensus_queue.cc:1048] T e4dc6b0541ca431ebc757beac00521f1 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.007s
I20260709 13:20:43.154238 20048 raft_consensus.cc:3055] T 3a7ca61592e848ea9b0fb81a2ce33cdc P cb82cc7bd6db46d980c75237a1402860 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:20:43.275614 20270 raft_consensus.cc:493] T 8c11074579b046d99e585c2818e18a2c P cb82cc7bd6db46d980c75237a1402860 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:43.109437 20073 raft_consensus.cc:515] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:43.145365 20275 consensus_queue.cc:1048] T 5a96373bb0e14dce8a266b2fc6c58507 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.014s
I20260709 13:20:43.107132 20248 consensus_queue.cc:260] T e5ad4eb281564936a59f41e387a47156 P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:43.157765 20254 raft_consensus.cc:493] T d2ab0ac8347641e6955e9bce7d90c1a2 P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:20:43.158187 20261 log.cc:927] Time spent T 8d4ab870a05347868956aa8b841b2981 P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 0.749s	user 0.000s	sys 0.007s
I20260709 13:20:43.160940 16221 raft_consensus.cc:2468] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 9.
I20260709 13:20:43.168497 16293 raft_consensus.cc:1240] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 12. Current term is 13. Ops: []
W20260709 13:20:43.141734 20045 consensus_meta.cc:287] T c0eab25ce4c64e018a041beeda2867db P 647a343a542d4a139fe106c19095129b: Time spent flushing consensus metadata: real 0.538s	user 0.007s	sys 0.003s
I20260709 13:20:43.108775 20314 raft_consensus.cc:493] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:43.107650 19673 leader_election.cc:290] T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
W20260709 13:20:43.143935 19914 consensus_meta.cc:287] T 829b4ad8c5a54eababeea1181b960d4b P 647a343a542d4a139fe106c19095129b: Time spent flushing consensus metadata: real 0.518s	user 0.007s	sys 0.003s
I20260709 13:20:43.140877 20274 raft_consensus.cc:2749] T b87147731a68400e8cce9b879daa7c5d P cb82cc7bd6db46d980c75237a1402860 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
W20260709 13:20:43.418298 20238 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 11 current term: 11 is higher.
I20260709 13:20:43.418553 19914 raft_consensus.cc:2749] T 829b4ad8c5a54eababeea1181b960d4b P 647a343a542d4a139fe106c19095129b [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:20:43.160049 20107 consensus_meta.cc:287] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b: Time spent flushing consensus metadata: real 0.662s	user 0.011s	sys 0.000s
I20260709 13:20:43.390566 19939 raft_consensus.cc:2749] T b6f07842d3aa47858df5665874c86f3a P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
W20260709 13:20:43.237630 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 68440.8us
I20260709 13:20:43.276044 16790 leader_election.cc:304] T af57996a8be84d398f0f023e8d08ef77 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:20:43.277465 16142 raft_consensus.cc:1275] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 13:20:43.236832 20303 raft_consensus.cc:3060] T 1df93779209748b9860f6d893dd3de53 P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:20:43.110879 20312 consensus_queue.cc:1048] T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.122s
W20260709 13:20:43.298367 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 254489us
I20260709 13:20:43.233062 20310 raft_consensus.cc:493] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:43.300460 16149 raft_consensus.cc:2410] T 2794af5b99984efebdee564035841cb0 P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 for term 9 because replica has last-logged OpId of term: 8 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 8.
I20260709 13:20:43.305004 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5d59407eb545b895d8d922e0dd69fd" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:43.314036 20306 raft_consensus.cc:515] T e5ad4eb281564936a59f41e387a47156 P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:43.147059 20200 raft_consensus.cc:3060] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:20:43.319089 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 225667us
I20260709 13:20:43.229216 20252 raft_consensus.cc:515] T 5a959dbecc2648eb85d1cc550f5a3245 P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:20:43.333132 16266 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:43.108040 20257 leader_election.cc:290] T 829b4ad8c5a54eababeea1181b960d4b P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:43.372581 16294 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48674 (request call id 2596) took 1642 ms. Trace:
I20260709 13:20:43.391438 16224 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33050 (request call id 681) took 2237 ms. Trace:
I20260709 13:20:43.304234 20183 raft_consensus.cc:697] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 90f3da57796b488c9087a2f523adf045 [term 11 LEADER]: Becoming Leader. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:20:43.370584 20048 raft_consensus.cc:740] T 3a7ca61592e848ea9b0fb81a2ce33cdc P cb82cc7bd6db46d980c75237a1402860 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:20:43.167977 20315 raft_consensus.cc:515] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:20:43.411587 20304 log.cc:927] Time spent T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 0.178s	user 0.001s	sys 0.000s
W20260709 13:20:43.398538 20187 consensus_meta.cc:287] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac: Time spent flushing consensus metadata: real 0.658s	user 0.012s	sys 0.000s
I20260709 13:20:43.234519 16144 raft_consensus.cc:1275] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b [term 10 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
W20260709 13:20:43.427783 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 172744us
I20260709 13:20:43.469473 16790 leader_election.cc:304] T c9944d5e4b8c4109af9955c4f08c9095 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 90f3da57796b488c9087a2f523adf045
I20260709 13:20:43.452298 16224 rpcz_store.cc:276] 0709 13:20:41.131664 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:41.131860 (+   196us) service_pool.cc:224] Handling call
0709 13:20:41.132300 (+   440us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:20:41.142709 (+ 10409us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:41.142751 (+    42us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:20:41.164143 (+ 21392us) log.cc:844] Serialized 52 byte log entry
0709 13:20:41.358829 (+194686us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:20:41.358882 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:41.358955 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:20:43.159012 (+1800057us) spinlock_profiling.cc:243] Waited 123 ms on lock 0x7b4400022b4c. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69ad8fadb1 00007f69ad998c85 00007f69ad9996a2 00007f69b070b1db 00007f69b0706aaf 00007f69b06fa86e 00007f69b06f9bb6 00007f69b06f97d8 00007f69b0703221 00007f69b07031d9 00007f69b0703169
0709 13:20:43.318886 (+159874us) raft_consensus.cc:1685] finished
0709 13:20:43.318905 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:43.369373 (+ 50468us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":8301,"raft.run_cpu_time_us":162,"raft.run_wall_time_us":7388,"spinlock_wait_cycles":281921920,"thread_start_us":134655,"threads_started":1,"wal-append.queue_time_us":185655}
I20260709 13:20:43.428539 16116 leader_election.cc:304] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:20:43.471649 20285 raft_consensus.cc:2804] T c9944d5e4b8c4109af9955c4f08c9095 P cb82cc7bd6db46d980c75237a1402860 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:20:43.472138 20285 raft_consensus.cc:493] T c9944d5e4b8c4109af9955c4f08c9095 P cb82cc7bd6db46d980c75237a1402860 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:20:43.472570 20285 raft_consensus.cc:3060] T c9944d5e4b8c4109af9955c4f08c9095 P cb82cc7bd6db46d980c75237a1402860 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:20:43.472891 16190 leader_election.cc:304] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:20:43.474345 16218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a378d7f796f94615b3b347119a9880fd" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:43.450672 16294 rpcz_store.cc:276] 0709 13:20:41.729813 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:41.752597 (+ 22784us) service_pool.cc:224] Handling call
0709 13:20:41.772562 (+ 19965us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:42.074680 (+302118us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:20:42.696596 (+621916us) log.cc:844] Serialized 14 byte log entry
0709 13:20:42.788357 (+ 91761us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:20:42.788484 (+   127us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:20:42.788609 (+   125us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:43.372392 (+583783us) spinlock_profiling.cc:243] Waited 302 ms on lock 0x7b540035e8d4. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:20:43.372429 (+    37us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:43.372535 (+   106us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":30393,"prepare.queue_time_us":88616,"prepare.run_cpu_time_us":54,"prepare.run_wall_time_us":55,"raft.queue_time_us":17458,"raft.run_cpu_time_us":163,"raft.run_wall_time_us":27109,"spinlock_wait_cycles":694300928,"thread_start_us":621973,"threads_started":2,"wal-append.queue_time_us":154992}
I20260709 13:20:43.431285 20316 raft_consensus.cc:2749] T af57996a8be84d398f0f023e8d08ef77 P cb82cc7bd6db46d980c75237a1402860 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:20:43.425081 20236 raft_consensus.cc:3060] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:20:43.458415 16217 raft_consensus.cc:2468] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 11.
I20260709 13:20:43.409826 20254 raft_consensus.cc:515] T d2ab0ac8347641e6955e9bce7d90c1a2 P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:43.466722 16222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02838c86531d4f98bf58d1c22b8baf77" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:43.480804 16222 raft_consensus.cc:2468] T 02838c86531d4f98bf58d1c22b8baf77 P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 10.
I20260709 13:20:43.466604 16220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66ae176961b74813a58bc15c9c4c6f2a" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:43.481930 16220 raft_consensus.cc:2468] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 12.
I20260709 13:20:43.476547 16226 raft_consensus.cc:1275] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 13:20:43.438344 16221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68a6838d82ef4cac83507f906139d90f" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:43.446579 20248 raft_consensus.cc:3060] T e5ad4eb281564936a59f41e387a47156 P 647a343a542d4a139fe106c19095129b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:20:43.454174 20080 raft_consensus.cc:493] T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:20:43.397686 20270 raft_consensus.cc:515] T 8c11074579b046d99e585c2818e18a2c P cb82cc7bd6db46d980c75237a1402860 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:43.491288 16223 raft_consensus.cc:1240] T fffab26427e24d398e2695aad40ca134 P 1ca2725d290e47d68b1a4b609c9816ac [term 9 LEADER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 8. Current term is 9. Ops: []
I20260709 13:20:43.496411 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a878999c0024d2abe517484180290fc" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:43.499708 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8638ee363273426db1393dd81e5c7b72" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:43.505780 16222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e4bfb9090c8460eb14fa7e0d818e7f1" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:43.515115 16217 raft_consensus.cc:2393] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 11: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
I20260709 13:20:43.456972 20310 raft_consensus.cc:515] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:20:43.500191 20197 consensus_meta.cc:287] T cdbd78acb4cf46298e0de8ba0d130e19 P 90f3da57796b488c9087a2f523adf045: Time spent flushing consensus metadata: real 0.654s	user 0.010s	sys 0.002s
I20260709 13:20:43.507577 20080 raft_consensus.cc:515] T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:43.534629 20197 raft_consensus.cc:515] T cdbd78acb4cf46298e0de8ba0d130e19 P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:43.536039 20254 leader_election.cc:290] T d2ab0ac8347641e6955e9bce7d90c1a2 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:43.536806 20197 leader_election.cc:290] T cdbd78acb4cf46298e0de8ba0d130e19 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 11 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:43.536621 20298 consensus_queue.cc:1048] T 8a878999c0024d2abe517484180290fc P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.059s
I20260709 13:20:43.537281 20080 leader_election.cc:290] T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:20:43.537468 16298 raft_consensus.cc:1240] T cdbd78acb4cf46298e0de8ba0d130e19 P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 10. Current term is 11. Ops: []
I20260709 13:20:43.538421 20294 raft_consensus.cc:493] T 66f75124215f4ca6bef831e174a792d8 P cb82cc7bd6db46d980c75237a1402860 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:43.539067 20294 raft_consensus.cc:515] T 66f75124215f4ca6bef831e174a792d8 P cb82cc7bd6db46d980c75237a1402860 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:43.543717 20316 raft_consensus.cc:493] T ada5a004ba7c46819051e3ccec71ffdb P cb82cc7bd6db46d980c75237a1402860 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:43.544498 20316 raft_consensus.cc:515] T ada5a004ba7c46819051e3ccec71ffdb P cb82cc7bd6db46d980c75237a1402860 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:43.428040 20314 raft_consensus.cc:515] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:43.499061 20275 raft_consensus.cc:2804] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:20:43.476035 20187 raft_consensus.cc:515] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:20:43.705016 20241 log.cc:927] Time spent T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 2.297s	user 0.005s	sys 0.005s
W20260709 13:20:43.772913 20331 log.cc:927] Time spent T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.108s	user 0.000s	sys 0.000s
I20260709 13:20:43.775754 20260 leader_election.cc:290] T c0eab25ce4c64e018a041beeda2867db P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:43.780560 20314 leader_election.cc:290] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:43.781065 20280 leader_election.cc:290] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 9 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:20:43.784325 20073 leader_election.cc:290] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
W20260709 13:20:43.713084 20319 log.cc:927] Time spent T 31969392ccdb43b491911a319e8bc7e1 P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.413s	user 0.001s	sys 0.000s
I20260709 13:20:43.785566 20285 raft_consensus.cc:515] T c9944d5e4b8c4109af9955c4f08c9095 P cb82cc7bd6db46d980c75237a1402860 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:43.428552 20289 consensus_queue.cc:237] T c0eab25ce4c64e018a041beeda2867db P 90f3da57796b488c9087a2f523adf045 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:43.727375 16218 raft_consensus.cc:2468] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 11.
I20260709 13:20:43.724442 20252 leader_election.cc:290] T 5a959dbecc2648eb85d1cc550f5a3245 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:43.737694 20306 leader_election.cc:290] T e5ad4eb281564936a59f41e387a47156 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:43.794996 19982 raft_consensus.cc:3055] T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:20:43.799057 19982 raft_consensus.cc:740] T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:20:43.741425 20310 leader_election.cc:290] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
W20260709 13:20:43.750386 20263 log.cc:927] Time spent T 08c74490c40243a88e62d1760b5cd814 P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 0.070s	user 0.000s	sys 0.000s
W20260709 13:20:43.755659 20295 log.cc:927] Time spent T 8a878999c0024d2abe517484180290fc P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 0.298s	user 0.001s	sys 0.001s
I20260709 13:20:43.756214 16219 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33050 (request call id 683) took 2602 ms. Trace:
W20260709 13:20:43.756731 20318 log.cc:927] Time spent T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.290s	user 0.001s	sys 0.000s
I20260709 13:20:43.661334 19673 raft_consensus.cc:493] T 19baa7bc367844c288bca8f81439c413 P cb82cc7bd6db46d980c75237a1402860 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:43.759665 20187 leader_election.cc:290] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 11 election: Requested vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:43.761513 20271 leader_election.cc:290] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 10 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:43.663658 20048 consensus_queue.cc:260] T 3a7ca61592e848ea9b0fb81a2ce33cdc P cb82cc7bd6db46d980c75237a1402860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:43.670099 20275 raft_consensus.cc:493] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:43.763248 20307 raft_consensus.cc:493] T 66f75124215f4ca6bef831e174a792d8 P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:43.702339 20183 consensus_queue.cc:237] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 90f3da57796b488c9087a2f523adf045 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:43.767850 20294 leader_election.cc:290] T 66f75124215f4ca6bef831e174a792d8 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:20:43.786104 16300 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40474 (request call id 661) took 1078 ms. Trace:
I20260709 13:20:43.735658 20315 leader_election.cc:290] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
W20260709 13:20:43.732833 20329 log.cc:927] Time spent T fffab26427e24d398e2695aad40ca134 P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 0.185s	user 0.003s	sys 0.000s
I20260709 13:20:43.799322 16147 raft_consensus.cc:2468] T c0eab25ce4c64e018a041beeda2867db P 647a343a542d4a139fe106c19095129b [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 10.
I20260709 13:20:43.811676 16219 rpcz_store.cc:276] 0709 13:20:41.154081 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:41.154209 (+   128us) service_pool.cc:224] Handling call
0709 13:20:41.166414 (+ 12205us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 13:20:41.398741 (+232327us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:41.398771 (+    30us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 13:20:42.663606 (+1264835us) spinlock_profiling.cc:243] Waited 5.04 ms on lock 0x7b2c00115ac8. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69af458a11 00007f69af468372 00007f69b0b9aa01 00007f69b0b5f43a 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2
0709 13:20:42.664397 (+   791us) log.cc:844] Serialized 78 byte log entry
0709 13:20:42.798526 (+134129us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:20:42.798576 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:42.798664 (+    88us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:20:43.755853 (+957189us) raft_consensus.cc:1685] finished
0709 13:20:43.755874 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:43.756131 (+   257us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":64246,"raft.run_cpu_time_us":121,"raft.run_wall_time_us":121,"spinlock_wait_cycles":11588992,"thread_start_us":276217,"threads_started":2,"wal-append.queue_time_us":427944}
W20260709 13:20:43.831496 16266 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:20:43.836323 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 345178us
W20260709 13:20:43.853984 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 381136us
I20260709 13:20:43.783068 20316 leader_election.cc:290] T ada5a004ba7c46819051e3ccec71ffdb P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:43.848395 20275 raft_consensus.cc:3060] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:20:43.877545 16292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c61a39fd4f84424b129c1a6f9ba404d" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:43.876753 16294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c2095a5f4ad49408dbc7f69c606120d" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:43.864367 20307 raft_consensus.cc:515] T 66f75124215f4ca6bef831e174a792d8 P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:43.883229 16790 leader_election.cc:304] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
W20260709 13:20:43.887233 16265 leader_election.cc:336] T 434bff3082d141838068f323bdb0d1ed P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Timed out: RequestConsensusVote RPC to 127.14.177.129:36741 timed out after 1.592s (SENT)
I20260709 13:20:43.889453 16294 raft_consensus.cc:2468] T 7c2095a5f4ad49408dbc7f69c606120d P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 7.
I20260709 13:20:43.891234 16265 leader_election.cc:304] T 434bff3082d141838068f323bdb0d1ed P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
W20260709 13:20:43.875435 20239 log.cc:927] Time spent T 8cc959fb7f704fba9009c0595467d074 P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 2.467s	user 0.016s	sys 0.003s
I20260709 13:20:43.876341 19982 consensus_queue.cc:260] T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:43.848923 20330 raft_consensus.cc:493] T ecc1e363230043fe9fef390f0b979f20 P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:43.851202 16300 rpcz_store.cc:276] 0709 13:20:42.707314 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:42.718109 (+ 10795us) service_pool.cc:224] Handling call
0709 13:20:42.739558 (+ 21449us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:20:42.748621 (+  9063us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:20:42.748668 (+    47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:20:42.780535 (+ 31867us) log.cc:844] Serialized 26 byte log entry
0709 13:20:43.091385 (+310850us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:20:43.091423 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:43.091486 (+    63us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:20:43.785887 (+694401us) raft_consensus.cc:1685] finished
0709 13:20:43.785910 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:43.786044 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5593,"raft.queue_time_us":144,"raft.run_cpu_time_us":174,"raft.run_wall_time_us":14619,"thread_start_us":270949,"threads_started":1,"wal-append.queue_time_us":339556}
I20260709 13:20:43.876703 16145 raft_consensus.cc:3055] T 8158bbd17b994d2eb96f560bd0f1b88d P 647a343a542d4a139fe106c19095129b [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:20:43.896818 16145 raft_consensus.cc:740] T 8158bbd17b994d2eb96f560bd0f1b88d P 647a343a542d4a139fe106c19095129b [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:20:43.860646 20048 raft_consensus.cc:3060] T 3a7ca61592e848ea9b0fb81a2ce33cdc P cb82cc7bd6db46d980c75237a1402860 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:20:43.893697 16292 raft_consensus.cc:2468] T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 9.
I20260709 13:20:43.834640 19673 raft_consensus.cc:515] T 19baa7bc367844c288bca8f81439c413 P cb82cc7bd6db46d980c75237a1402860 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:43.908253 16145 consensus_queue.cc:260] T 8158bbd17b994d2eb96f560bd0f1b88d P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:43.899585 20316 raft_consensus.cc:2749] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:20:43.913342 16291 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40474 (request call id 662) took 1205 ms. Trace:
I20260709 13:20:43.916775 16291 rpcz_store.cc:276] 0709 13:20:42.707590 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:42.707825 (+   235us) service_pool.cc:224] Handling call
0709 13:20:42.726298 (+ 18473us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:20:42.734845 (+  8547us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:42.734876 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:20:42.762861 (+ 27985us) log.cc:844] Serialized 26 byte log entry
0709 13:20:43.149368 (+386507us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:20:43.149418 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:43.149501 (+    83us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:20:43.913093 (+763592us) raft_consensus.cc:1685] finished
0709 13:20:43.913122 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:43.913278 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":4074,"raft.run_cpu_time_us":165,"raft.run_wall_time_us":16723,"thread_start_us":327658,"threads_started":1,"wal-append.queue_time_us":443339}
I20260709 13:20:43.904597 20080 raft_consensus.cc:2749] T 434bff3082d141838068f323bdb0d1ed P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:20:43.925916 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 62559.3us
I20260709 13:20:43.929145 16145 raft_consensus.cc:3060] T 8158bbd17b994d2eb96f560bd0f1b88d P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:20:43.931361 16192 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 327458us
I20260709 13:20:43.931880 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0eab25ce4c64e018a041beeda2867db" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:43.937973 16291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c74490c40243a88e62d1760b5cd814" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:43.935756 16190 leader_election.cc:304] T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:20:43.940918 16190 leader_election.cc:304] T 7c2095a5f4ad49408dbc7f69c606120d P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:20:43.928879 20285 leader_election.cc:290] T c9944d5e4b8c4109af9955c4f08c9095 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 11 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:43.930995 20336 raft_consensus.cc:493] T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:43.945742 20245 raft_consensus.cc:2804] T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:20:43.943918 20338 raft_consensus.cc:493] T 8c11074579b046d99e585c2818e18a2c P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:43.947644 20187 raft_consensus.cc:2804] T 7c2095a5f4ad49408dbc7f69c606120d P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:20:43.954142 16291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806d7690c2b644959152074bb2cab0a7" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 11 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:43.952708 20245 raft_consensus.cc:493] T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:43.966817 20245 raft_consensus.cc:3060] T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:20:43.954974 20187 raft_consensus.cc:493] T 7c2095a5f4ad49408dbc7f69c606120d P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Starting leader election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:20:43.968719 16143 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40830 (request call id 1394) took 1442 ms. Trace:
I20260709 13:20:43.983139 20187 raft_consensus.cc:3060] T 7c2095a5f4ad49408dbc7f69c606120d P 1ca2725d290e47d68b1a4b609c9816ac [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:20:43.983134 16143 rpcz_store.cc:276] 0709 13:20:42.525999 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:42.780115 (+254116us) service_pool.cc:224] Handling call
0709 13:20:43.968476 (+1188361us) spinlock_profiling.cc:243] Waited 123 ms on lock 0x7b540028e454. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:20:43.968654 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":283007360}
I20260709 13:20:43.988662 16145 raft_consensus.cc:1275] T 8158bbd17b994d2eb96f560bd0f1b88d P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 13:20:43.916595 20330 raft_consensus.cc:515] T ecc1e363230043fe9fef390f0b979f20 P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:20:43.981297 16789 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:43.982221 20307 leader_election.cc:290] T 66f75124215f4ca6bef831e174a792d8 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:43.989823 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a959dbecc2648eb85d1cc550f5a3245" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:44.028182 16300 raft_consensus.cc:2468] T 5a959dbecc2648eb85d1cc550f5a3245 P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 7.
I20260709 13:20:43.986828 19673 leader_election.cc:290] T 19baa7bc367844c288bca8f81439c413 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:44.028975 16295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8638ee363273426db1393dd81e5c7b72" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:43.998361 16825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af57996a8be84d398f0f023e8d08ef77" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:43.973825 20338 raft_consensus.cc:515] T 8c11074579b046d99e585c2818e18a2c P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:43.999744 16824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66ae176961b74813a58bc15c9c4c6f2a" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
W20260709 13:20:44.002410 20268 consensus_meta.cc:287] T 26354361e9b044cea1c2b86c4882fe92 P cb82cc7bd6db46d980c75237a1402860: Time spent flushing consensus metadata: real 0.766s	user 0.011s	sys 0.000s
I20260709 13:20:43.952060 20336 raft_consensus.cc:515] T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:43.972743 20303 raft_consensus.cc:515] T 1df93779209748b9860f6d893dd3de53 P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:44.005306 16291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5ad4eb281564936a59f41e387a47156" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:44.038426 16291 raft_consensus.cc:2393] T e5ad4eb281564936a59f41e387a47156 P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 11: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
I20260709 13:20:44.006137 16192 leader_election.cc:304] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:20:44.012027 16818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746aabf856ab401c9ef5e859371e7b67" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:44.039228 16818 raft_consensus.cc:2468] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 9.
I20260709 13:20:44.017733 16145 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40844 (request call id 2299) took 1637 ms. Trace:
I20260709 13:20:44.029613 16295 raft_consensus.cc:2468] T 8638ee363273426db1393dd81e5c7b72 P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 10.
W20260709 13:20:44.028018 20021 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 9 current term: 9 is higher.
I20260709 13:20:44.029948 16825 raft_consensus.cc:2468] T af57996a8be84d398f0f023e8d08ef77 P cb82cc7bd6db46d980c75237a1402860 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 11.
I20260709 13:20:44.037567 16824 pending_rounds.cc:85] T 806d7690c2b644959152074bb2cab0a7 P cb82cc7bd6db46d980c75237a1402860: Aborting all ops after (but not including) 6
I20260709 13:20:44.045269 16824 pending_rounds.cc:107] T 806d7690c2b644959152074bb2cab0a7 P cb82cc7bd6db46d980c75237a1402860: Aborting uncommitted NO_OP operation due to leader change: 9.7
I20260709 13:20:44.045667 16824 raft_consensus.cc:2889] T 806d7690c2b644959152074bb2cab0a7 P cb82cc7bd6db46d980c75237a1402860 [term 10 LEARNER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:20:44.044138 16190 leader_election.cc:304] T 5a959dbecc2648eb85d1cc550f5a3245 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:20:44.041818 16145 rpcz_store.cc:276] 0709 13:20:42.380285 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:42.380451 (+   166us) service_pool.cc:224] Handling call
0709 13:20:42.402505 (+ 22054us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:43.876606 (+1474101us) spinlock_profiling.cc:243] Waited 47.6 ms on lock 0x7b5400319820. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0ad6dc0 00007f69b0b611c0 00007f69b0b62231 00007f69b0b5ed8f 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c
0709 13:20:44.017318 (+140712us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:44.017659 (+   341us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":109557120}
I20260709 13:20:44.030964 20230 leader_election.cc:290] T 5a959dbecc2648eb85d1cc550f5a3245 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
W20260709 13:20:44.037232 16265 connection.cc:256] RPC call timeout handler was delayed by -1.620754345000023s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1715s for a response
I20260709 13:20:44.034523 19941 raft_consensus.cc:493] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:20:44.054944 16265 connection.cc:256] RPC call timeout handler was delayed by -1.5973543449999852s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17410000000000003s for a response
I20260709 13:20:44.056187 16825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ab0ac8347641e6955e9bce7d90c1a2" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
W20260709 13:20:44.058543 16265 connection.cc:256] RPC call timeout handler was delayed by -1.4641543449999972s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1889s for a response
W20260709 13:20:44.059136 16265 connection.cc:256] RPC call timeout handler was delayed by -1.4326543449999463s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19240000000000002s for a response
W20260709 13:20:44.059608 16265 connection.cc:256] RPC call timeout handler was delayed by -1.3687543450000703s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1995s for a response
W20260709 13:20:44.060129 16265 connection.cc:256] RPC call timeout handler was delayed by -1.3407343069998205s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.42800000000000005s for a response
W20260709 13:20:44.060781 16265 connection.cc:256] RPC call timeout handler was delayed by -1.2640260670000316s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1625s for a response
W20260709 13:20:44.061079 16265 connection.cc:256] RPC call timeout handler was delayed by -1.0834543450000638s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2312s for a response
I20260709 13:20:44.065464 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9944d5e4b8c4109af9955c4f08c9095" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:20:44.066035 16295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:20:44.066893 16295 raft_consensus.cc:3060] T ecc1e363230043fe9fef390f0b979f20 P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:20:44.059912 16821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7ca61592e848ea9b0fb81a2ce33cdc" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:44.070067 16821 raft_consensus.cc:2468] T 3a7ca61592e848ea9b0fb81a2ce33cdc P cb82cc7bd6db46d980c75237a1402860 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 13.
I20260709 13:20:44.072190 20338 leader_election.cc:290] T 8c11074579b046d99e585c2818e18a2c P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:44.075532 16220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ab0ac8347641e6955e9bce7d90c1a2" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:44.076241 16220 raft_consensus.cc:2468] T d2ab0ac8347641e6955e9bce7d90c1a2 P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 10.
I20260709 13:20:44.077255 20336 leader_election.cc:290] T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
W20260709 13:20:44.079962 20265 log.cc:927] Time spent T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 2.008s	user 0.008s	sys 0.010s
I20260709 13:20:44.087692 16820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8638ee363273426db1393dd81e5c7b72" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:44.089286 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a378d7f796f94615b3b347119a9880fd" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 12 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:44.047221 20307 raft_consensus.cc:2804] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:20:44.090498 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c11074579b046d99e585c2818e18a2c" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:44.090689 20307 raft_consensus.cc:697] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [term 11 LEADER]: Becoming Leader. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:20:44.092227 20307 consensus_queue.cc:237] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:44.093932 16291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdbd78acb4cf46298e0de8ba0d130e19" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:20:44.108616 16148 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40844 (request call id 2295) took 2715 ms. Trace:
I20260709 13:20:44.131024 16789 leader_election.cc:304] T 8638ee363273426db1393dd81e5c7b72 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:20:44.059615 20269 raft_consensus.cc:2804] T 5a959dbecc2648eb85d1cc550f5a3245 P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:20:44.058149 20289 raft_consensus.cc:493] T 8c11074579b046d99e585c2818e18a2c P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:44.037901 20268 raft_consensus.cc:515] T 26354361e9b044cea1c2b86c4882fe92 P cb82cc7bd6db46d980c75237a1402860 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:20:44.105772 20340 log.cc:927] Time spent T b6f07842d3aa47858df5665874c86f3a P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 0.101s	user 0.001s	sys 0.000s
I20260709 13:20:44.152362 16821 raft_consensus.cc:1240] T c9944d5e4b8c4109af9955c4f08c9095 P cb82cc7bd6db46d980c75237a1402860 [term 11 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 10. Current term is 11. Ops: []
I20260709 13:20:44.109200 16148 rpcz_store.cc:276] 0709 13:20:41.397840 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:41.402733 (+  4893us) service_pool.cc:224] Handling call
0709 13:20:41.424180 (+ 21447us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:20:41.510560 (+ 86380us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:41.510600 (+    40us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:20:41.511135 (+   535us) log.cc:844] Serialized 52 byte log entry
0709 13:20:41.663117 (+151982us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:20:41.663359 (+   242us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:41.663496 (+   137us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:20:44.079558 (+2416062us) spinlock_profiling.cc:243] Waited 2.02 ms on lock 0x7b2400120518. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69ad8fadb1 00007f69ad8f89c4 00007f69ad8f6acb 00007f69adab1dc9 00007f69adab1a55 00007f69adab109d 00007f69b06e310e 00007f69b06fa88c 00007f69b06f9bb6 00007f69b06f97d8 00007f69b0703221
0709 13:20:44.101986 (+ 22428us) raft_consensus.cc:1685] finished
0709 13:20:44.102019 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:44.108530 (+  6511us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":47630,"raft.queue_time_us":57653,"raft.run_cpu_time_us":111,"raft.run_wall_time_us":2927,"spinlock_wait_cycles":4655488,"thread_start_us":70944,"threads_started":1,"wal-append.queue_time_us":208824}
I20260709 13:20:44.143427 16220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdbd78acb4cf46298e0de8ba0d130e19" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:20:44.150516 16192 leader_election.cc:304] T 746aabf856ab401c9ef5e859371e7b67 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 
W20260709 13:20:44.140674 20345 log.cc:927] Time spent T c0eab25ce4c64e018a041beeda2867db P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.114s	user 0.001s	sys 0.000s
I20260709 13:20:44.153990 16192 leader_election.cc:304] T af57996a8be84d398f0f023e8d08ef77 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:20:44.139721 16218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6f07842d3aa47858df5665874c86f3a" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:44.157200 16218 raft_consensus.cc:2393] T b6f07842d3aa47858df5665874c86f3a P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 12: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
I20260709 13:20:44.156004 16291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d4ab870a05347868956aa8b841b2981" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 12 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:44.157949 16291 raft_consensus.cc:2410] T 8d4ab870a05347868956aa8b841b2981 P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac for term 12 because replica has last-logged OpId of term: 11 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 4.
I20260709 13:20:44.063407 19941 raft_consensus.cc:515] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:44.159055 16192 leader_election.cc:304] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 90f3da57796b488c9087a2f523adf045
I20260709 13:20:44.158008 16818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5718c623ebee4d2bb770f0f9ddb2f164" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:44.162144 16818 raft_consensus.cc:2468] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 10.
I20260709 13:20:44.163786 16819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdbd78acb4cf46298e0de8ba0d130e19" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:20:44.141503 20269 raft_consensus.cc:493] T 5a959dbecc2648eb85d1cc550f5a3245 P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:44.165776 20269 raft_consensus.cc:3060] T 5a959dbecc2648eb85d1cc550f5a3245 P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:20:44.169377 16192 leader_election.cc:304] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860
I20260709 13:20:44.175806 20294 raft_consensus.cc:2804] T 8638ee363273426db1393dd81e5c7b72 P cb82cc7bd6db46d980c75237a1402860 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:20:44.176301 20294 raft_consensus.cc:493] T 8638ee363273426db1393dd81e5c7b72 P cb82cc7bd6db46d980c75237a1402860 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:44.176635 20294 raft_consensus.cc:3060] T 8638ee363273426db1393dd81e5c7b72 P cb82cc7bd6db46d980c75237a1402860 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:20:44.155681 20336 raft_consensus.cc:2804] T 746aabf856ab401c9ef5e859371e7b67 P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:20:44.152166 20289 raft_consensus.cc:515] T 8c11074579b046d99e585c2818e18a2c P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:44.166996 20338 raft_consensus.cc:2804] T af57996a8be84d398f0f023e8d08ef77 P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:20:44.171576 20278 raft_consensus.cc:2804] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:20:44.201536 16192 leader_election.cc:304] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:20:44.181969 20336 raft_consensus.cc:493] T 746aabf856ab401c9ef5e859371e7b67 P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: Starting leader election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:20:44.215102 16821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a378d7f796f94615b3b347119a9880fd" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 12 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:44.216467 19941 leader_election.cc:290] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:20:44.215101 16818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "829b4ad8c5a54eababeea1181b960d4b" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:44.217908 16825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdbd78acb4cf46298e0de8ba0d130e19" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
W20260709 13:20:44.219323 16265 leader_election.cc:336] T 18ad7e3af5504571a69d10c00750a01c P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Timed out: RequestConsensusVote RPC to 127.14.177.129:36741 timed out after 1.687s (SENT)
I20260709 13:20:44.220216 16300 raft_consensus.cc:2468] T 8c11074579b046d99e585c2818e18a2c P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 10.
I20260709 13:20:44.214352 20353 raft_consensus.cc:493] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:20:44.193005 20338 raft_consensus.cc:493] T af57996a8be84d398f0f023e8d08ef77 P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:44.222185 16819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62fe3346cd3c42c4a55e14af9073bca4" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:44.223771 16265 leader_election.cc:304] T 18ad7e3af5504571a69d10c00750a01c P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:20:44.200409 20278 raft_consensus.cc:493] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Starting leader election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:20:44.209120 20200 raft_consensus.cc:2749] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:20:44.212791 20336 raft_consensus.cc:3060] T 746aabf856ab401c9ef5e859371e7b67 P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:20:44.231830 16818 raft_consensus.cc:1240] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 [term 9 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 8. Current term is 9. Ops: []
I20260709 13:20:44.212419 20337 raft_consensus.cc:2804] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:20:44.224985 16819 raft_consensus.cc:2468] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 13.
I20260709 13:20:44.234257 16192 leader_election.cc:304] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860
I20260709 13:20:44.231544 20278 raft_consensus.cc:3060] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:20:44.238642 16825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9944d5e4b8c4109af9955c4f08c9095" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
W20260709 13:20:44.246956 16147 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40830 (request call id 1395) took 1720 ms (client timeout 1889 ms). Trace:
I20260709 13:20:44.238214 16192 leader_election.cc:304] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860
W20260709 13:20:44.252740 16147 rpcz_store.cc:269] 0709 13:20:42.526311 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:42.717264 (+190953us) service_pool.cc:224] Handling call
0709 13:20:44.246771 (+1529507us) spinlock_profiling.cc:243] Waited 682 ms on lock 0x7b54000735d4. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:20:44.246893 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1567898880}
I20260709 13:20:44.229240 20338 raft_consensus.cc:3060] T af57996a8be84d398f0f023e8d08ef77 P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:20:44.255098 16117 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 1.24859e+06us
I20260709 13:20:44.244347 16818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c11074579b046d99e585c2818e18a2c" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:44.257951 16818 raft_consensus.cc:2468] T 8c11074579b046d99e585c2818e18a2c P cb82cc7bd6db46d980c75237a1402860 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 10.
I20260709 13:20:44.212927 20187 raft_consensus.cc:515] T 7c2095a5f4ad49408dbc7f69c606120d P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:44.250322 16823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66f75124215f4ca6bef831e174a792d8" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:44.296005 16823 raft_consensus.cc:2468] T 66f75124215f4ca6bef831e174a792d8 P cb82cc7bd6db46d980c75237a1402860 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 10.
I20260709 13:20:44.229737 20353 raft_consensus.cc:515] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:44.297243 20187 leader_election.cc:290] T 7c2095a5f4ad49408dbc7f69c606120d P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 8 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:44.299198 20353 leader_election.cc:290] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:20:44.256608 20088 raft_consensus.cc:2749] T 18ad7e3af5504571a69d10c00750a01c P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:20:44.258832 20252 raft_consensus.cc:2749] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:20:44.255442 20306 raft_consensus.cc:2749] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:20:44.267270 20229 raft_consensus.cc:2749] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:20:44.279803 16150 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40844 (request call id 2307) took 1286 ms. Trace:
W20260709 13:20:44.280287 16115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 1.25375e+06us
I20260709 13:20:44.280494 19915 raft_consensus.cc:2804] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 13:20:44.248271 20296 log.cc:927] Time spent T ee9ed67006144797b1de6ea8c2013a68 P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 1.186s	user 0.005s	sys 0.005s
W20260709 13:20:44.283815 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 64516.2us
I20260709 13:20:44.284729 20260 consensus_queue.cc:1059] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 7, Time since last communication: 0.001s
W20260709 13:20:44.310724 16265 leader_election.cc:336] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Timed out: RequestConsensusVote RPC to 127.14.177.129:36741 timed out after 1.824s (SENT)
I20260709 13:20:44.284922 16192 leader_election.cc:304] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:20:44.223961 20275 raft_consensus.cc:515] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:44.317333 16298 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40474 (request call id 659) took 1610 ms. Trace:
I20260709 13:20:44.319444 16298 rpcz_store.cc:276] 0709 13:20:42.706724 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:42.718195 (+ 11471us) service_pool.cc:224] Handling call
0709 13:20:42.729526 (+ 11331us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:43.538105 (+808579us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:44.317101 (+778996us) spinlock_profiling.cc:243] Waited 808 ms on lock 0x7b54002e1b54. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:20:44.317273 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1857689856}
I20260709 13:20:44.307499 16150 rpcz_store.cc:276] 0709 13:20:42.992936 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:43.015635 (+ 22699us) service_pool.cc:224] Handling call
0709 13:20:43.031054 (+ 15419us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:44.279408 (+1248354us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:44.279438 (+    30us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:20:44.279488 (+    50us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:20:44.279526 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:44.279603 (+    77us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:44.279733 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1223928,"raft.queue_time_us":1248653,"raft.run_cpu_time_us":81,"raft.run_wall_time_us":81}
I20260709 13:20:44.309432 19915 raft_consensus.cc:493] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:44.307858 16115 leader_election.cc:304] T d16dfad0c3b540e5b4b86492ddf87208 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:20:44.311139 16265 leader_election.cc:304] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
I20260709 13:20:44.317342 16790 leader_election.cc:304] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:20:44.238348 20337 raft_consensus.cc:493] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: Starting leader election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:20:44.299268 20283 raft_consensus.cc:493] T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Starting leader election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:20:44.308341 16295 raft_consensus.cc:2468] T ecc1e363230043fe9fef390f0b979f20 P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 10.
I20260709 13:20:44.321952 19915 raft_consensus.cc:3060] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:20:44.325506 16192 leader_election.cc:304] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860
I20260709 13:20:44.344112 16293 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40474 (request call id 670) took 1300 ms. Trace:
I20260709 13:20:44.344383 16790 leader_election.cc:304] T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:20:44.310741 20260 raft_consensus.cc:3055] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:20:44.344383 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a9de347c711477e87c582d003f36925" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:44.347473 16192 leader_election.cc:304] T 8c11074579b046d99e585c2818e18a2c P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:20:44.311497 16048 catalog_manager.cc:5697] T fffab26427e24d398e2695aad40ca134 P 647a343a542d4a139fe106c19095129b reported cstate change: term changed from 2 to 5, leader changed from 90f3da57796b488c9087a2f523adf045 (127.14.177.131) to 647a343a542d4a139fe106c19095129b (127.14.177.129). New cstate: current_term: 5 leader_uuid: "647a343a542d4a139fe106c19095129b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } health_report { overall_health: UNKNOWN } } }
I20260709 13:20:44.350325 16220 raft_consensus.cc:1275] T 829b4ad8c5a54eababeea1181b960d4b P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:20:44.350955 16218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8638ee363273426db1393dd81e5c7b72" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:44.352164 16296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada5a004ba7c46819051e3ccec71ffdb" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 12 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:44.356442 16295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 12 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:44.357357 16295 raft_consensus.cc:2410] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 for term 12 because replica has last-logged OpId of term: 9 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 7.
I20260709 13:20:44.354897 16218 raft_consensus.cc:2468] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 11.
I20260709 13:20:44.355545 16192 leader_election.cc:304] T 66f75124215f4ca6bef831e174a792d8 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:20:44.346683 16293 rpcz_store.cc:276] 0709 13:20:43.043692 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:43.429429 (+385737us) service_pool.cc:224] Handling call
0709 13:20:43.456130 (+ 26701us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:20:43.519293 (+ 63163us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:43.519350 (+    57us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:20:43.551096 (+ 31746us) log.cc:844] Serialized 26 byte log entry
0709 13:20:43.944814 (+393718us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:20:43.944856 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:44.324008 (+379152us) spinlock_profiling.cc:243] Waited 54.4 ms on lock 0x7b54000a9e54. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:20:44.324055 (+    47us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:20:44.324106 (+    51us) raft_consensus.cc:1685] finished
0709 13:20:44.324120 (+    14us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:44.344029 (+ 19909us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":9041,"raft.run_cpu_time_us":180,"raft.run_wall_time_us":152071,"spinlock_wait_cycles":125048576,"thread_start_us":393195,"threads_started":1,"wal-append.queue_time_us":459557}
I20260709 13:20:44.378599 16048 catalog_manager.cc:5697] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b reported cstate change: term changed from 3 to 7. New cstate: current_term: 7 leader_uuid: "647a343a542d4a139fe106c19095129b" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:20:44.381268 20336 raft_consensus.cc:515] T 746aabf856ab401c9ef5e859371e7b67 P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:44.380817 16048 catalog_manager.cc:5697] T 8158bbd17b994d2eb96f560bd0f1b88d P 647a343a542d4a139fe106c19095129b reported cstate change: term changed from 3 to 6, leader changed from 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130) to 90f3da57796b488c9087a2f523adf045 (127.14.177.131). New cstate: current_term: 6 leader_uuid: "90f3da57796b488c9087a2f523adf045" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } }
I20260709 13:20:44.382959 16048 catalog_manager.cc:5697] T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b reported cstate change: term changed from 6 to 8. New cstate: current_term: 8 leader_uuid: "647a343a542d4a139fe106c19095129b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } health_report { overall_health: UNKNOWN } } }
I20260709 13:20:44.384017 16292 raft_consensus.cc:1275] T 84f2da98864b4425bf7c0055951fdd4b P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 13:20:44.356122 20189 raft_consensus.cc:2749] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:20:44.385546 16292 raft_consensus.cc:1275] T e4dc6b0541ca431ebc757beac00521f1 P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 10 index: 5. (index mismatch)
I20260709 13:20:44.385419 16048 catalog_manager.cc:5697] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 647a343a542d4a139fe106c19095129b reported cstate change: term changed from 3 to 6, leader changed from 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130) to 647a343a542d4a139fe106c19095129b (127.14.177.129). New cstate: current_term: 6 leader_uuid: "647a343a542d4a139fe106c19095129b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } health_report { overall_health: UNKNOWN } } }
I20260709 13:20:44.386476 16820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746aabf856ab401c9ef5e859371e7b67" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:20:44.352873 20260 raft_consensus.cc:740] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:20:44.387120 16820 raft_consensus.cc:3060] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:20:44.387418 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 76742.9us
W20260709 13:20:44.389087 16265 leader_election.cc:336] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Timed out: RequestConsensusVote RPC to 127.14.177.129:36741 timed out after 2.039s (SENT)
I20260709 13:20:44.348935 20283 raft_consensus.cc:3060] T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:20:44.343438 20337 raft_consensus.cc:3060] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:20:44.391713 20336 leader_election.cc:290] T 746aabf856ab401c9ef5e859371e7b67 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 10 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:44.392997 20338 raft_consensus.cc:515] T af57996a8be84d398f0f023e8d08ef77 P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:44.394833 20289 leader_election.cc:290] T 8c11074579b046d99e585c2818e18a2c P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
W20260709 13:20:44.389900 16265 leader_election.cc:336] T 7c2095a5f4ad49408dbc7f69c606120d P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Timed out: RequestConsensusVote RPC to 127.14.177.129:36741 timed out after 2.051s (SENT)
I20260709 13:20:44.354174 20280 raft_consensus.cc:2804] T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:20:44.352197 20310 raft_consensus.cc:2804] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:20:44.344813 20285 raft_consensus.cc:2804] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:20:44.343722 20274 consensus_queue.cc:1059] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 8, Time since last communication: 0.075s
W20260709 13:20:44.402940 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 54359.5us
I20260709 13:20:44.398078 16265 leader_election.cc:304] T 7c2095a5f4ad49408dbc7f69c606120d P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:20:44.401837 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26354361e9b044cea1c2b86c4882fe92" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:20:44.405227 16299 raft_consensus.cc:3060] T 26354361e9b044cea1c2b86c4882fe92 P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:20:44.358352 20268 leader_election.cc:290] T 26354361e9b044cea1c2b86c4882fe92 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 9 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:44.418807 16821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af57996a8be84d398f0f023e8d08ef77" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:20:44.419708 16821 raft_consensus.cc:3060] T af57996a8be84d398f0f023e8d08ef77 P cb82cc7bd6db46d980c75237a1402860 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:20:44.419103 16220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:20:44.419380 16820 raft_consensus.cc:2468] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 10.
I20260709 13:20:44.421624 16220 raft_consensus.cc:3060] T ecc1e363230043fe9fef390f0b979f20 P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:20:44.423230 16192 leader_election.cc:304] T 746aabf856ab401c9ef5e859371e7b67 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:20:44.351825 20267 raft_consensus.cc:2749] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:20:44.428377 16221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66f75124215f4ca6bef831e174a792d8" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:44.432323 16218 raft_consensus.cc:1240] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 12. Current term is 13. Ops: [12.10-12.10]
I20260709 13:20:44.399302 20252 raft_consensus.cc:2804] T 8c11074579b046d99e585c2818e18a2c P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:20:44.434140 16222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b7d1ab519044eda2d5e04a07242f25" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:20:44.395884 20323 raft_consensus.cc:2804] T 66f75124215f4ca6bef831e174a792d8 P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:20:44.400156 20280 raft_consensus.cc:493] T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:44.401146 20285 raft_consensus.cc:493] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:44.401785 20274 raft_consensus.cc:3055] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:20:44.436237 16294 raft_consensus.cc:1275] T 8cc959fb7f704fba9009c0595467d074 P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 13:20:44.429067 20306 raft_consensus.cc:2804] T 746aabf856ab401c9ef5e859371e7b67 P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:20:44.395141 20338 leader_election.cc:290] T af57996a8be84d398f0f023e8d08ef77 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 12 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:44.433471 20252 raft_consensus.cc:493] T 8c11074579b046d99e585c2818e18a2c P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:44.470793 16821 raft_consensus.cc:2468] T af57996a8be84d398f0f023e8d08ef77 P cb82cc7bd6db46d980c75237a1402860 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 12.
I20260709 13:20:44.470973 20252 raft_consensus.cc:3060] T 8c11074579b046d99e585c2818e18a2c P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:20:44.472289 16192 leader_election.cc:304] T af57996a8be84d398f0f023e8d08ef77 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:20:44.443054 16298 raft_consensus.cc:2410] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 for term 13 because replica has last-logged OpId of term: 11 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 8.
I20260709 13:20:44.445616 16218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada5a004ba7c46819051e3ccec71ffdb" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 12 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
W20260709 13:20:44.446009 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 56127.5us
I20260709 13:20:44.437872 16295 raft_consensus.cc:1275] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
W20260709 13:20:44.456393 20351 log.cc:927] Time spent T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860: Append to log took a long time: real 0.130s	user 0.000s	sys 0.000s
I20260709 13:20:44.456727 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746aabf856ab401c9ef5e859371e7b67" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:44.457876 16048 catalog_manager.cc:5697] T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b reported cstate change: term changed from 3 to 4, leader changed from 647a343a542d4a139fe106c19095129b (127.14.177.129) to 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130). New cstate: current_term: 4 leader_uuid: "1ca2725d290e47d68b1a4b609c9816ac" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:20:44.444989 20280 raft_consensus.cc:3060] T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:20:44.396714 20278 raft_consensus.cc:515] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:44.400501 20310 raft_consensus.cc:493] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:44.433943 20088 raft_consensus.cc:2749] T 7c2095a5f4ad49408dbc7f69c606120d P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:20:44.443409 20323 raft_consensus.cc:493] T 66f75124215f4ca6bef831e174a792d8 P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:44.397555 20260 consensus_queue.cc:260] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:44.529744 20283 raft_consensus.cc:515] T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:44.481689 20192 raft_consensus.cc:493] T 19baa7bc367844c288bca8f81439c413 P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:44.483577 16048 catalog_manager.cc:5697] T 1df93779209748b9860f6d893dd3de53 P 647a343a542d4a139fe106c19095129b reported cstate change: term changed from 2 to 6. New cstate: current_term: 6 leader_uuid: "647a343a542d4a139fe106c19095129b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } health_report { overall_health: UNKNOWN } } }
I20260709 13:20:44.443202 20330 leader_election.cc:290] T ecc1e363230043fe9fef390f0b979f20 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:44.490228 20269 raft_consensus.cc:515] T 5a959dbecc2648eb85d1cc550f5a3245 P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:44.501720 20310 raft_consensus.cc:3060] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:20:44.545069 16821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66ae176961b74813a58bc15c9c4c6f2a" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:20:44.545921 20269 leader_election.cc:290] T 5a959dbecc2648eb85d1cc550f5a3245 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 9 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:44.446597 20274 raft_consensus.cc:740] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
W20260709 13:20:44.491933 16265 leader_election.cc:336] T e5ad4eb281564936a59f41e387a47156 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Timed out: RequestConsensusVote RPC to 127.14.177.129:36741 timed out after 2.055s (SENT)
I20260709 13:20:44.494352 16299 raft_consensus.cc:2468] T 26354361e9b044cea1c2b86c4882fe92 P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 9.
I20260709 13:20:44.494731 16295 raft_consensus.cc:1275] T 5a96373bb0e14dce8a266b2fc6c58507 P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 9 index: 4. Preceding OpId from leader: term: 10 index: 5. (index mismatch)
I20260709 13:20:44.494971 20316 consensus_queue.cc:1059] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: INVALID_TERM, Last received: 11.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:20:44.499461 16224 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9e4bfb9090c8460eb14fa7e0d818e7f1"
dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
mode: GRACEFUL
new_leader_uuid: "90f3da57796b488c9087a2f523adf045"
 from {username='slave'} at 127.0.0.1:39144
I20260709 13:20:44.458509 20268 consensus_queue.cc:1048] T 829b4ad8c5a54eababeea1181b960d4b P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:20:44.504664 20323 raft_consensus.cc:3060] T 66f75124215f4ca6bef831e174a792d8 P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:20:44.517350 16789 leader_election.cc:304] T ada5a004ba7c46819051e3ccec71ffdb P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:20:44.517352 20270 leader_election.cc:290] T 8c11074579b046d99e585c2818e18a2c P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:44.476046 20306 raft_consensus.cc:697] T 746aabf856ab401c9ef5e859371e7b67 P 1ca2725d290e47d68b1a4b609c9816ac [term 10 LEADER]: Becoming Leader. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:20:44.524866 20278 leader_election.cc:290] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 13 election: Requested vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
W20260709 13:20:44.525528 16817 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49892 (request call id 586) took 1489 ms (client timeout 1812 ms). Trace:
I20260709 13:20:44.526320 20303 leader_election.cc:290] T 1df93779209748b9860f6d893dd3de53 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 11 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:20:44.527958 16818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7ca61592e848ea9b0fb81a2ce33cdc" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:20:44.527881 20337 raft_consensus.cc:515] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:20:44.589736 20347 log.cc:927] Time spent T fffab26427e24d398e2695aad40ca134 P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.369s	user 0.002s	sys 0.000s
W20260709 13:20:44.591287 20356 log.cc:927] Time spent T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 0.054s	user 0.001s	sys 0.000s
I20260709 13:20:44.591370 20107 raft_consensus.cc:2749] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:20:44.528482 16223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c11074579b046d99e585c2818e18a2c" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:44.529222 20275 leader_election.cc:290] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 13 election: Requested vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:44.530238 20260 raft_consensus.cc:3060] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:20:44.446219 20285 raft_consensus.cc:3060] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:20:44.603350 20246 consensus_queue.cc:1048] T 8cc959fb7f704fba9009c0595467d074 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:20:44.605165 20246 consensus_queue.cc:1048] T c2f5f89186204242bf377f7c42b2e2da P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:20:44.502519 20338 raft_consensus.cc:2804] T af57996a8be84d398f0f023e8d08ef77 P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:20:44.543337 19915 raft_consensus.cc:515] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:44.553299 16220 raft_consensus.cc:2468] T ecc1e363230043fe9fef390f0b979f20 P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 10.
I20260709 13:20:44.566484 16224 raft_consensus.cc:606] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Received request to transfer leadership to TS 90f3da57796b488c9087a2f523adf045
I20260709 13:20:44.609674 16224 raft_consensus.cc:614] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 13:20:44.570142 16192 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:44.551342 16048 catalog_manager.cc:5697] T 02838c86531d4f98bf58d1c22b8baf77 P 647a343a542d4a139fe106c19095129b reported cstate change: term changed from 4 to 6, leader changed from 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130) to 90f3da57796b488c9087a2f523adf045 (127.14.177.131). New cstate: current_term: 6 leader_uuid: "90f3da57796b488c9087a2f523adf045" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } }
I20260709 13:20:44.611050 16048 catalog_manager.cc:5697] T d16dfad0c3b540e5b4b86492ddf87208 P 647a343a542d4a139fe106c19095129b reported cstate change: term changed from 1 to 5. New cstate: current_term: 5 leader_uuid: "647a343a542d4a139fe106c19095129b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } health_report { overall_health: HEALTHY } } }
W20260709 13:20:44.577310 16817 rpcz_store.cc:269] 0709 13:20:43.036527 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:43.037792 (+  1265us) service_pool.cc:224] Handling call
0709 13:20:44.525337 (+1487545us) spinlock_profiling.cc:243] Waited 252 ms on lock 0x7b54007706d4. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:20:44.525436 (+    99us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":579905920}
W20260709 13:20:44.580289 16789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 63002.1us
I20260709 13:20:44.583352 16226 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57600 (request call id 1406) took 1087 ms. Trace:
W20260709 13:20:44.585129 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 1.15658e+06us
W20260709 13:20:44.588265 16266 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:44.588405 20252 raft_consensus.cc:515] T 8c11074579b046d99e585c2818e18a2c P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:20:44.591287 20350 log.cc:927] Time spent T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 0.388s	user 0.007s	sys 0.001s
I20260709 13:20:44.518694 20245 raft_consensus.cc:515] T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:44.568002 20330 raft_consensus.cc:493] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:44.581086 20192 raft_consensus.cc:515] T 19baa7bc367844c288bca8f81439c413 P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:44.602288 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2372981006d4b218215c28964dea623" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:44.602509 20297 consensus_queue.cc:1048] T e4dc6b0541ca431ebc757beac00521f1 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:20:44.610690 16790 leader_election.cc:304] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:20:44.613641 16789 leader_election.cc:304] T 26354361e9b044cea1c2b86c4882fe92 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:20:44.606212 20293 consensus_queue.cc:1048] T 5a96373bb0e14dce8a266b2fc6c58507 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 2, Time since last communication: 0.009s
I20260709 13:20:44.598390 20270 raft_consensus.cc:2749] T ada5a004ba7c46819051e3ccec71ffdb P cb82cc7bd6db46d980c75237a1402860 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:20:44.619184 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 127261us
W20260709 13:20:44.625211 16265 leader_election.cc:336] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 10 election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Timed out: RequestConsensusVote RPC to 127.14.177.129:36741 timed out after 2.061s (SENT)
I20260709 13:20:44.622396 16220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9944d5e4b8c4109af9955c4f08c9095" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:20:44.625587 16223 raft_consensus.cc:2393] T 8c11074579b046d99e585c2818e18a2c P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 in current term 11: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
I20260709 13:20:44.612852 16226 rpcz_store.cc:276] 0709 13:20:43.495712 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:43.495860 (+   148us) service_pool.cc:224] Handling call
0709 13:20:43.507060 (+ 11200us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:20:43.507842 (+   782us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:43.507882 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:20:43.508443 (+   561us) log.cc:844] Serialized 26 byte log entry
0709 13:20:44.506255 (+997812us) spinlock_profiling.cc:243] Waited 14 ms on lock 0x7b180032b120. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b693ade1 00007f69b69475b1 00007f69ada54e07 00007f69b68afab9 00007f69ada7169b 00007f69ada6f93f 00007f69ada71c02 00007f69b06f920b 00007f69b06fed23 00007f69b06ff0d1 00007f69b0b1aa0f
0709 13:20:44.572772 (+ 66517us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:20:44.572817 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:44.572886 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:20:44.577358 (+  4472us) raft_consensus.cc:1685] finished
0709 13:20:44.577377 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:44.583283 (+  5906us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2560,"raft.run_cpu_time_us":122,"raft.run_wall_time_us":2363,"spinlock_wait_cycles":32230016,"thread_start_us":66331,"threads_started":1,"wal-append.queue_time_us":1033616}
I20260709 13:20:44.628584 16818 raft_consensus.cc:2468] T 3a7ca61592e848ea9b0fb81a2ce33cdc P cb82cc7bd6db46d980c75237a1402860 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 13.
I20260709 13:20:44.630352 16192 leader_election.cc:304] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:20:44.616866 20269 raft_consensus.cc:493] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:44.610221 20278 consensus_queue.cc:1048] T 84f2da98864b4425bf7c0055951fdd4b P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.016s
W20260709 13:20:44.641718 16115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 333915us
I20260709 13:20:44.618667 20338 raft_consensus.cc:697] T af57996a8be84d398f0f023e8d08ef77 P 1ca2725d290e47d68b1a4b609c9816ac [term 12 LEADER]: Becoming Leader. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:20:44.640554 16048 catalog_manager.cc:5697] T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b reported cstate change: term changed from 4 to 7, leader changed from cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) to 647a343a542d4a139fe106c19095129b (127.14.177.129). New cstate: current_term: 7 leader_uuid: "647a343a542d4a139fe106c19095129b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:20:44.642347 20181 raft_consensus.cc:2804] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:20:44.662583 16296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a959dbecc2648eb85d1cc550f5a3245" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:20:44.663498 16296 raft_consensus.cc:3060] T 5a959dbecc2648eb85d1cc550f5a3245 P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Advancing to term 9
I20260709 13:20:44.647367 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 12 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:44.665282 16224 raft_consensus.cc:2468] T 19baa7bc367844c288bca8f81439c413 P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 11.
I20260709 13:20:44.654654 16145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:20:44.667635 16145 raft_consensus.cc:3060] T ecc1e363230043fe9fef390f0b979f20 P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:20:44.655221 16147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5d59407eb545b895d8d922e0dd69fd" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:44.656863 16143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b7d1ab519044eda2d5e04a07242f25" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:20:44.657212 16144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66f75124215f4ca6bef831e174a792d8" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
W20260709 13:20:44.649379 16117 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 391965us
I20260709 13:20:44.661823 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7ca61592e848ea9b0fb81a2ce33cdc" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:20:44.654872 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5718c623ebee4d2bb770f0f9ddb2f164" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:44.637631 19941 raft_consensus.cc:2804] T 26354361e9b044cea1c2b86c4882fe92 P cb82cc7bd6db46d980c75237a1402860 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:20:44.672652 16150 raft_consensus.cc:2468] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 10.
I20260709 13:20:44.672695 16298 raft_consensus.cc:2393] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 13: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
W20260709 13:20:44.678462 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 53291.1us
I20260709 13:20:44.677726 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806d7690c2b644959152074bb2cab0a7" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 11 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
W20260709 13:20:44.679247 16265 leader_election.cc:336] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Timed out: RequestConsensusVote RPC to 127.14.177.129:36741 timed out after 2.085s (SENT)
I20260709 13:20:44.679833 16149 raft_consensus.cc:2410] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac for term 11 because replica has last-logged OpId of term: 9 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 6.
I20260709 13:20:44.679687 16265 leader_election.cc:304] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:20:44.668452 16048 catalog_manager.cc:5697] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b reported cstate change: term changed from 2 to 5, leader changed from 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130) to 647a343a542d4a139fe106c19095129b (127.14.177.129). New cstate: current_term: 5 leader_uuid: "647a343a542d4a139fe106c19095129b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } health_report { overall_health: UNKNOWN } } }
I20260709 13:20:44.681999 16146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c74490c40243a88e62d1760b5cd814" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:44.682987 16146 raft_consensus.cc:2393] T 08c74490c40243a88e62d1760b5cd814 P 647a343a542d4a139fe106c19095129b [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 10: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
I20260709 13:20:44.683873 16148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c2095a5f4ad49408dbc7f69c606120d" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:44.621941 20267 consensus_queue.cc:1059] T c9944d5e4b8c4109af9955c4f08c9095 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.045s
I20260709 13:20:44.644940 20236 raft_consensus.cc:2804] T d16dfad0c3b540e5b4b86492ddf87208 P 647a343a542d4a139fe106c19095129b [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:20:44.703598 16143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "829b4ad8c5a54eababeea1181b960d4b" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:44.651450 19982 raft_consensus.cc:3060] T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:20:44.695907 16142 raft_consensus.cc:1275] T 8cc959fb7f704fba9009c0595467d074 P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 13:20:44.691450 16225 raft_consensus.cc:1240] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 10. Current term is 11. Ops: []
I20260709 13:20:44.666991 20021 raft_consensus.cc:493] T e5ad4eb281564936a59f41e387a47156 P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:44.648890 19915 leader_election.cc:290] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 13 election: Requested vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:44.654142 20337 leader_election.cc:290] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 10 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
W20260709 13:20:44.657826 20361 log.cc:927] Time spent T 68a6838d82ef4cac83507f906139d90f P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.115s	user 0.001s	sys 0.000s
I20260709 13:20:44.704397 16143 raft_consensus.cc:2468] T 829b4ad8c5a54eababeea1181b960d4b P 647a343a542d4a139fe106c19095129b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 10.
I20260709 13:20:44.657346 20231 raft_consensus.cc:493] T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:44.673858 20270 raft_consensus.cc:2804] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:20:44.660943 20283 leader_election.cc:290] T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 11 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:44.689462 16048 catalog_manager.cc:5697] T 9c61a39fd4f84424b129c1a6f9ba404d P 647a343a542d4a139fe106c19095129b reported cstate change: term changed from 5 to 7, leader changed from 90f3da57796b488c9087a2f523adf045 (127.14.177.131) to 647a343a542d4a139fe106c19095129b (127.14.177.129). New cstate: current_term: 7 leader_uuid: "647a343a542d4a139fe106c19095129b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } health_report { overall_health: UNKNOWN } } }
I20260709 13:20:44.659716 20181 raft_consensus.cc:697] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 1ca2725d290e47d68b1a4b609c9816ac [term 13 LEADER]: Becoming Leader. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:20:44.627995 20330 raft_consensus.cc:515] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:44.694180 19941 raft_consensus.cc:697] T 26354361e9b044cea1c2b86c4882fe92 P cb82cc7bd6db46d980c75237a1402860 [term 9 LEADER]: Becoming Leader. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:20:44.620426 20274 consensus_queue.cc:260] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:44.647547 20246 raft_consensus.cc:493] T 1df93779209748b9860f6d893dd3de53 P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:20:44.715571 16150 raft_consensus.cc:1275] T 84f2da98864b4425bf7c0055951fdd4b P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 13:20:44.718178 16145 raft_consensus.cc:2468] T ecc1e363230043fe9fef390f0b979f20 P 647a343a542d4a139fe106c19095129b [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 10.
I20260709 13:20:44.645998 20269 raft_consensus.cc:515] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:44.661379 20306 consensus_queue.cc:237] T 746aabf856ab401c9ef5e859371e7b67 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:20:44.772827 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 53482.4us
I20260709 13:20:44.973454 16293 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48684 (request call id 2449) took 1088 ms. Trace:
I20260709 13:20:45.317700 16293 rpcz_store.cc:276] 0709 13:20:43.884713 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:44.514908 (+630195us) service_pool.cc:224] Handling call
0709 13:20:44.515292 (+   384us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:20:44.515657 (+   365us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:44.515683 (+    26us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:20:44.516102 (+   419us) log.cc:844] Serialized 26 byte log entry
0709 13:20:44.629777 (+113675us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:20:44.971386 (+341609us) log.cc:844] Serialized 14 byte log entry
0709 13:20:44.972994 (+  1608us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:44.973105 (+   111us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:20:44.973162 (+    57us) raft_consensus.cc:1685] finished
0709 13:20:44.973183 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:44.973373 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":228961,"prepare.run_cpu_time_us":52,"prepare.run_wall_time_us":55,"raft.queue_time_us":233,"raft.run_cpu_time_us":183,"raft.run_wall_time_us":186,"thread_start_us":404207,"threads_started":2,"wal-append.queue_time_us":154256}
I20260709 13:20:45.189553 16292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c11074579b046d99e585c2818e18a2c" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
W20260709 13:20:45.153731 16790 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.517s	user 0.008s	sys 0.012s
I20260709 13:20:45.164775 16291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c2095a5f4ad49408dbc7f69c606120d" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
W20260709 13:20:44.803367 16789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 178612us
W20260709 13:20:45.305866 20260 consensus_meta.cc:287] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac: Time spent flushing consensus metadata: real 0.690s	user 0.009s	sys 0.000s
I20260709 13:20:45.223762 16818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5718c623ebee4d2bb770f0f9ddb2f164" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:20:45.402889 20192 leader_election.cc:290] T 19baa7bc367844c288bca8f81439c413 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
W20260709 13:20:45.406872 16789 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:45.407339 20246 raft_consensus.cc:515] T 1df93779209748b9860f6d893dd3de53 P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:45.408735 16222 raft_consensus.cc:2468] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 9.
W20260709 13:20:44.792505 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 166133us
I20260709 13:20:45.414418 16817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 12 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:45.415187 16817 raft_consensus.cc:2468] T 19baa7bc367844c288bca8f81439c413 P cb82cc7bd6db46d980c75237a1402860 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 11.
I20260709 13:20:45.416144 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66ae176961b74813a58bc15c9c4c6f2a" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:20:44.738025 16142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5ad4eb281564936a59f41e387a47156" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:44.744566 16146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0eab25ce4c64e018a041beeda2867db" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:45.418036 16146 raft_consensus.cc:2468] T c0eab25ce4c64e018a041beeda2867db P 647a343a542d4a139fe106c19095129b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 10.
W20260709 13:20:45.314895 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 635625us
I20260709 13:20:45.066589 20310 raft_consensus.cc:515] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:20:45.306105 20366 log.cc:927] Time spent T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.361s	user 0.000s	sys 0.002s
I20260709 13:20:44.742463 16296 raft_consensus.cc:2468] T 5a959dbecc2648eb85d1cc550f5a3245 P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 9.
W20260709 13:20:45.410857 20381 log.cc:927] Time spent T d16dfad0c3b540e5b4b86492ddf87208 P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.056s	user 0.001s	sys 0.000s
I20260709 13:20:45.323788 16297 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40474 (request call id 681) took 1342 ms. Trace:
I20260709 13:20:44.719594 20021 raft_consensus.cc:515] T e5ad4eb281564936a59f41e387a47156 P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:45.419350 16291 raft_consensus.cc:3060] T 7c2095a5f4ad49408dbc7f69c606120d P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:20:45.409677 20246 leader_election.cc:290] T 1df93779209748b9860f6d893dd3de53 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:45.428849 16817 raft_consensus.cc:1275] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 [term 11 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
W20260709 13:20:45.237854 20285 consensus_meta.cc:287] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860: Time spent flushing consensus metadata: real 0.634s	user 0.005s	sys 0.005s
W20260709 13:20:44.982761 20368 log.cc:927] Time spent T 9e4bfb9090c8460eb14fa7e0d818e7f1 P cb82cc7bd6db46d980c75237a1402860: Append to log took a long time: real 0.102s	user 0.001s	sys 0.000s
I20260709 13:20:44.703928 20236 raft_consensus.cc:493] T d16dfad0c3b540e5b4b86492ddf87208 P 647a343a542d4a139fe106c19095129b [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:20:44.783594 20304 log.cc:927] Time spent T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 0.073s	user 0.000s	sys 0.000s
I20260709 13:20:45.319178 16048 catalog_manager.cc:5697] T 08c74490c40243a88e62d1760b5cd814 P 647a343a542d4a139fe106c19095129b reported cstate change: term changed from 3 to 6. New cstate: current_term: 6 leader_uuid: "1ca2725d290e47d68b1a4b609c9816ac" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } }
I20260709 13:20:45.421926 16148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a959dbecc2648eb85d1cc550f5a3245" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
W20260709 13:20:45.421496 16790 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.793s	user 0.010s	sys 0.014s
I20260709 13:20:45.449747 16145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5718c623ebee4d2bb770f0f9ddb2f164" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:20:45.383405 20274 raft_consensus.cc:3060] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:20:45.016897 16266 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:44.704011 20065 raft_consensus.cc:2749] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:20:44.721115 20294 consensus_meta.cc:287] T 8638ee363273426db1393dd81e5c7b72 P cb82cc7bd6db46d980c75237a1402860: Time spent flushing consensus metadata: real 0.544s	user 0.008s	sys 0.003s
W20260709 13:20:45.428227 16117 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 724966us
W20260709 13:20:45.313365 19982 consensus_meta.cc:287] T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b: Time spent flushing consensus metadata: real 0.598s	user 0.005s	sys 0.005s
I20260709 13:20:45.431000 16296 raft_consensus.cc:1240] T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 10. Current term is 11. Ops: []
I20260709 13:20:44.872396 19861 raft_consensus.cc:993] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac: : Instructing follower 90f3da57796b488c9087a2f523adf045 to start an election
I20260709 13:20:45.397332 19941 consensus_queue.cc:237] T 26354361e9b044cea1c2b86c4882fe92 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:45.433210 16146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c2095a5f4ad49408dbc7f69c606120d" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:20:44.719791 19726 consensus_queue.cc:1048] T 8d4ab870a05347868956aa8b841b2981 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.714s
I20260709 13:20:45.425598 16297 rpcz_store.cc:276] 0709 13:20:43.981363 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:44.540751 (+559388us) service_pool.cc:224] Handling call
0709 13:20:44.541057 (+   306us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:20:44.575810 (+ 34753us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:44.575842 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:20:44.609246 (+ 33404us) log.cc:844] Serialized 26 byte log entry
0709 13:20:45.323319 (+714073us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:20:45.323378 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:45.323476 (+    98us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:20:45.323536 (+    60us) raft_consensus.cc:1685] finished
0709 13:20:45.323555 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:45.323722 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1917,"raft.run_cpu_time_us":140,"raft.run_wall_time_us":17433,"thread_start_us":91814,"threads_started":1,"wal-append.queue_time_us":221517}
I20260709 13:20:45.435827 20021 leader_election.cc:290] T e5ad4eb281564936a59f41e387a47156 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:45.436795 20246 consensus_queue.cc:1059] T 18ad7e3af5504571a69d10c00750a01c P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:20:45.437106 20267 raft_consensus.cc:493] T 84f2da98864b4425bf7c0055951fdd4b P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:45.437613 16142 raft_consensus.cc:2468] T e5ad4eb281564936a59f41e387a47156 P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 11.
I20260709 13:20:45.438750 16825 raft_consensus.cc:1275] T ada5a004ba7c46819051e3ccec71ffdb P cb82cc7bd6db46d980c75237a1402860 [term 11 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:20:45.440184 16298 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40474 (request call id 687) took 1195 ms. Trace:
I20260709 13:20:45.440550 16143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746aabf856ab401c9ef5e859371e7b67" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:44.710486 20303 raft_consensus.cc:3055] T c9944d5e4b8c4109af9955c4f08c9095 P 90f3da57796b488c9087a2f523adf045 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:20:45.443682 16147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a959dbecc2648eb85d1cc550f5a3245" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:20:44.870532 19990 raft_consensus.cc:3055] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [term 12 LEADER]: Stepping down as leader of term 12
W20260709 13:20:45.248777 20280 consensus_meta.cc:287] T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860: Time spent flushing consensus metadata: real 0.745s	user 0.009s	sys 0.000s
I20260709 13:20:45.420687 16292 raft_consensus.cc:3060] T 8c11074579b046d99e585c2818e18a2c P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:20:44.868371 20365 log.cc:927] Time spent T 87ac0d1f7e8a419f9bcbc04aee0e067e P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.103s	user 0.000s	sys 0.000s
I20260709 13:20:45.449477 16824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c11074579b046d99e585c2818e18a2c" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:45.538689 16824 raft_consensus.cc:2468] T 8c11074579b046d99e585c2818e18a2c P cb82cc7bd6db46d980c75237a1402860 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 10.
W20260709 13:20:45.450096 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 839423us
I20260709 13:20:45.539335 16819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
W20260709 13:20:45.469704 20323 consensus_meta.cc:287] T 66f75124215f4ca6bef831e174a792d8 P 1ca2725d290e47d68b1a4b609c9816ac: Time spent flushing consensus metadata: real 0.876s	user 0.009s	sys 0.003s
I20260709 13:20:45.450135 16148 raft_consensus.cc:2468] T 5a959dbecc2648eb85d1cc550f5a3245 P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 8.
I20260709 13:20:45.451056 16145 raft_consensus.cc:2393] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 10: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
I20260709 13:20:45.452560 16821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:45.452771 16817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c11074579b046d99e585c2818e18a2c" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:20:45.547380 16817 raft_consensus.cc:3060] T 8c11074579b046d99e585c2818e18a2c P cb82cc7bd6db46d980c75237a1402860 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:20:44.946063 20245 leader_election.cc:290] T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 11 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:44.978075 20094 raft_consensus.cc:493] T fffab26427e24d398e2695aad40ca134 P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:45.380445 20270 raft_consensus.cc:697] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860 [term 10 LEADER]: Becoming Leader. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:20:45.454996 16048 catalog_manager.cc:5697] T 7c2095a5f4ad49408dbc7f69c606120d P 647a343a542d4a139fe106c19095129b reported cstate change: term changed from 4 to 5, leader changed from 90f3da57796b488c9087a2f523adf045 (127.14.177.131) to 647a343a542d4a139fe106c19095129b (127.14.177.129). New cstate: current_term: 5 leader_uuid: "647a343a542d4a139fe106c19095129b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } health_report { overall_health: UNKNOWN } } }
I20260709 13:20:45.458381 16820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74ca8140f35c4ae29e8a664f6bdfc829" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:20:45.459378 16294 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48684 (request call id 2445) took 1575 ms. Trace:
W20260709 13:20:45.461369 16265 leader_election.cc:336] T 2794af5b99984efebdee564035841cb0 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Timed out: RequestConsensusVote RPC to 127.14.177.129:36741 timed out after 2.190s (SENT)
I20260709 13:20:45.420037 20231 raft_consensus.cc:515] T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:45.463007 16296 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40474 (request call id 686) took 1279 ms. Trace:
I20260709 13:20:44.840520 20022 consensus_queue.cc:1048] T af57996a8be84d398f0f023e8d08ef77 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.614s
I20260709 13:20:45.449328 20236 raft_consensus.cc:3060] T d16dfad0c3b540e5b4b86492ddf87208 P 647a343a542d4a139fe106c19095129b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:20:45.470158 16224 raft_consensus.cc:1240] T b6f07842d3aa47858df5665874c86f3a P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 11. Current term is 12. Ops: [11.7-11.7]
I20260709 13:20:45.581717 16298 rpcz_store.cc:276] 0709 13:20:44.244363 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:45.439341 (+1194978us) service_pool.cc:224] Handling call
0709 13:20:45.439654 (+   313us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:45.439822 (+   168us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:45.439851 (+    29us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:20:45.439895 (+    44us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:20:45.439927 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:45.439982 (+    55us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:45.440101 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:20:45.471189 16219 raft_consensus.cc:1240] T c0eab25ce4c64e018a041beeda2867db P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 9. Current term is 10. Ops: []
I20260709 13:20:45.471063 16146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746aabf856ab401c9ef5e859371e7b67" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:20:45.468859 16144 raft_consensus.cc:2468] T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 10.
I20260709 13:20:45.190085 20271 raft_consensus.cc:493] T b87147731a68400e8cce9b879daa7c5d P cb82cc7bd6db46d980c75237a1402860 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:45.568831 16296 rpcz_store.cc:276] 0709 13:20:44.183708 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:45.430370 (+1246662us) service_pool.cc:224] Handling call
0709 13:20:45.430762 (+   392us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:45.462476 (+ 31714us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:45.462934 (+   458us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:20:45.591949 16145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66f75124215f4ca6bef831e174a792d8" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:45.591694 16146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62fe3346cd3c42c4a55e14af9073bca4" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:45.546615 16147 raft_consensus.cc:3060] T 5a959dbecc2648eb85d1cc550f5a3245 P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:20:45.543012 16218 pending_rounds.cc:85] T 1df93779209748b9860f6d893dd3de53 P 1ca2725d290e47d68b1a4b609c9816ac: Aborting all ops after (but not including) 5
I20260709 13:20:45.543495 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c61a39fd4f84424b129c1a6f9ba404d" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:20:45.540170 16819 raft_consensus.cc:2468] T 19baa7bc367844c288bca8f81439c413 P cb82cc7bd6db46d980c75237a1402860 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 11.
I20260709 13:20:45.460481 20252 leader_election.cc:290] T 8c11074579b046d99e585c2818e18a2c P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 11 election: Requested vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:45.550741 16790 leader_election.cc:304] T c9944d5e4b8c4109af9955c4f08c9095 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:20:44.857275 20338 consensus_queue.cc:237] T af57996a8be84d398f0f023e8d08ef77 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:20:45.528100 20304 log.cc:927] Time spent T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 0.062s	user 0.000s	sys 0.000s
I20260709 13:20:45.445858 20285 raft_consensus.cc:515] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:45.553884 16220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18ad7e3af5504571a69d10c00750a01c" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:45.568804 16265 leader_election.cc:304] T 2794af5b99984efebdee564035841cb0 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
I20260709 13:20:45.570398 16148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af57996a8be84d398f0f023e8d08ef77" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:20:45.570627 16300 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48684 (request call id 2450) took 1685 ms. Trace:
I20260709 13:20:45.654618 16300 rpcz_store.cc:276] 0709 13:20:43.884977 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:44.530537 (+645560us) service_pool.cc:224] Handling call
0709 13:20:44.530988 (+   451us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:20:44.531291 (+   303us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:44.531328 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:20:44.531926 (+   598us) log.cc:844] Serialized 26 byte log entry
0709 13:20:44.626588 (+ 94662us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:20:44.626647 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:44.626734 (+    87us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:20:45.305944 (+679210us) spinlock_profiling.cc:243] Waited 120 ms on lock 0x7b20001a4e78. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b070db91 00007f69b072fa19 00007f69b06fa8c3 00007f69b06f9bb6 00007f69b06f97d8 00007f69b0703221 00007f69b07031d9 00007f69b0703169 00007f69b0703131 00007f69b070242d 000000000038f354
0709 13:20:45.553950 (+248006us) raft_consensus.cc:1685] finished
0709 13:20:45.553981 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:45.570544 (+ 16563us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":277139712,"thread_start_us":94276,"threads_started":1,"wal-append.queue_time_us":411307}
I20260709 13:20:45.570698 16294 rpcz_store.cc:276] 0709 13:20:43.883624 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:44.447108 (+563484us) service_pool.cc:224] Handling call
0709 13:20:44.453486 (+  6378us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:20:44.497807 (+ 44321us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:44.497835 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:20:44.498284 (+   449us) log.cc:844] Serialized 26 byte log entry
0709 13:20:44.556698 (+ 58414us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:20:44.556755 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:44.556851 (+    96us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:20:44.657259 (+100408us) spinlock_profiling.cc:243] Waited 20.1 ms on lock 0x7b4400022b4c. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69ad8fadb1 00007f69ad998c85 00007f69ad9996a2 00007f69b070b1db 00007f69b0706aaf 00007f69b06fa86e 00007f69b06f9bb6 00007f69b06f97d8 00007f69b0703221 00007f69b07031d9 00007f69b0703169
0709 13:20:45.453501 (+796242us) raft_consensus.cc:1685] finished
0709 13:20:45.453532 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:45.459314 (+  5782us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":43787,"raft.run_cpu_time_us":90,"raft.run_wall_time_us":91,"spinlock_wait_cycles":46326528,"thread_start_us":57984,"threads_started":1,"wal-append.queue_time_us":44317}
I20260709 13:20:45.558648 16295 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48684 (request call id 2456) took 1558 ms. Trace:
I20260709 13:20:45.581350 16225 raft_consensus.cc:1240] T 7c2095a5f4ad49408dbc7f69c606120d P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 7. Current term is 8. Ops: []
I20260709 13:20:45.470954 16823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8638ee363273426db1393dd81e5c7b72" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:45.594888 16145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a9de347c711477e87c582d003f36925" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:45.595228 16218 pending_rounds.cc:107] T 1df93779209748b9860f6d893dd3de53 P 1ca2725d290e47d68b1a4b609c9816ac: Aborting uncommitted NO_OP operation due to leader change: 9.6
I20260709 13:20:45.657071 16295 rpcz_store.cc:276] 0709 13:20:44.000088 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:44.548388 (+548300us) service_pool.cc:224] Handling call
0709 13:20:44.576041 (+ 27653us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:20:44.600485 (+ 24444us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:44.600515 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:20:44.622135 (+ 21620us) log.cc:844] Serialized 26 byte log entry
0709 13:20:45.091455 (+469320us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:20:45.091523 (+    68us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:45.091629 (+   106us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:20:45.410231 (+318602us) spinlock_profiling.cc:243] Waited 48.3 ms on lock 0x7b4400022b4c. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69ad8fadb1 00007f69ad998c85 00007f69ad9996a2 00007f69b070b1db 00007f69b0706aaf 00007f69b06fa86e 00007f69b06f9bb6 00007f69b06f97d8 00007f69b0703221 00007f69b07031d9 00007f69b0703169
0709 13:20:45.550360 (+140129us) raft_consensus.cc:1685] finished
0709 13:20:45.550380 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:45.558565 (+  8185us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3982,"raft.queue_time_us":5586,"raft.run_cpu_time_us":87,"raft.run_wall_time_us":87,"spinlock_wait_cycles":111018752,"thread_start_us":429160,"threads_started":1,"wal-append.queue_time_us":714961}
I20260709 13:20:45.658423 16145 raft_consensus.cc:2393] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 in current term 13: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
I20260709 13:20:45.594906 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d4ab870a05347868956aa8b841b2981" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 12 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:45.594933 16146 raft_consensus.cc:2468] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 13.
I20260709 13:20:45.466486 20294 raft_consensus.cc:515] T 8638ee363273426db1393dd81e5c7b72 P cb82cc7bd6db46d980c75237a1402860 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:45.595513 16297 raft_consensus.cc:3060] T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Advancing to term 11
I20260709 13:20:45.631821 16048 catalog_manager.cc:5697] T 8cc959fb7f704fba9009c0595467d074 P 647a343a542d4a139fe106c19095129b reported cstate change: term changed from 2 to 6, leader changed from 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130) to 647a343a542d4a139fe106c19095129b (127.14.177.129). New cstate: current_term: 6 leader_uuid: "647a343a542d4a139fe106c19095129b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } health_report { overall_health: UNKNOWN } } }
W20260709 13:20:45.556648 20368 log.cc:927] Time spent T 9e4bfb9090c8460eb14fa7e0d818e7f1 P cb82cc7bd6db46d980c75237a1402860: Append to log took a long time: real 0.088s	user 0.000s	sys 0.001s
I20260709 13:20:45.630231 16817 raft_consensus.cc:2468] T 8c11074579b046d99e585c2818e18a2c P cb82cc7bd6db46d980c75237a1402860 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 11.
I20260709 13:20:45.538131 19990 raft_consensus.cc:740] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:20:45.521106 20281 raft_consensus.cc:1081] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac [term 9 LEADER]: Signalling peer 90f3da57796b488c9087a2f523adf045 to start an election
I20260709 13:20:45.650243 16296 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40474 (request call id 688) took 1378 ms. Trace:
I20260709 13:20:45.682466 16296 rpcz_store.cc:276] 0709 13:20:44.272163 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:45.649023 (+1376860us) service_pool.cc:224] Handling call
0709 13:20:45.649592 (+   569us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:45.649825 (+   233us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:45.649863 (+    38us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:20:45.649916 (+    53us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:20:45.649957 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:45.650026 (+    69us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:45.650173 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:20:45.658308 16218 raft_consensus.cc:2889] T 1df93779209748b9860f6d893dd3de53 P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:20:45.584081 16222 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33050 (request call id 703) took 1153 ms. Trace:
I20260709 13:20:45.664386 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 12 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:45.665871 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2372981006d4b218215c28964dea623" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:45.662369 16219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806d7690c2b644959152074bb2cab0a7" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:45.686708 16219 raft_consensus.cc:2468] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 10.
I20260709 13:20:45.583482 20094 raft_consensus.cc:515] T fffab26427e24d398e2695aad40ca134 P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:20:45.674572 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 123913us
I20260709 13:20:45.671548 19982 raft_consensus.cc:493] T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:20:45.684787 16222 rpcz_store.cc:276] 0709 13:20:44.431031 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:44.431189 (+   158us) service_pool.cc:224] Handling call
0709 13:20:45.583873 (+1152684us) spinlock_profiling.cc:243] Waited 966 ms on lock 0x7b54002c9a54. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:20:45.584004 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2220754688}
I20260709 13:20:45.690850 16790 leader_election.cc:304] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:20:45.584240 20021 consensus_queue.cc:1048] T af57996a8be84d398f0f023e8d08ef77 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.103s
I20260709 13:20:45.692818 16298 raft_consensus.cc:2410] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac for term 12 because replica has last-logged OpId of term: 9 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 7.
I20260709 13:20:45.686017 16149 raft_consensus.cc:2468] T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 11.
I20260709 13:20:45.693197 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 12 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:45.592957 20303 raft_consensus.cc:740] T c9944d5e4b8c4109af9955c4f08c9095 P 90f3da57796b488c9087a2f523adf045 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:20:45.693909 16150 raft_consensus.cc:2468] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 11.
I20260709 13:20:45.544595 20280 raft_consensus.cc:515] T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:45.557868 20323 raft_consensus.cc:515] T 66f75124215f4ca6bef831e174a792d8 P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:45.704031 16220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d16dfad0c3b540e5b4b86492ddf87208" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:45.697299 16148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26354361e9b044cea1c2b86c4882fe92" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:20:45.699018 16144 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46224 (request call id 609) took 1044 ms. Trace:
I20260709 13:20:45.709972 16144 rpcz_store.cc:276] 0709 13:20:44.654476 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:44.656975 (+  2499us) service_pool.cc:224] Handling call
0709 13:20:45.698847 (+1041872us) spinlock_profiling.cc:243] Waited 798 ms on lock 0x7b540041ebd4. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:20:45.698958 (+   111us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1835042176}
I20260709 13:20:45.699141 16266 leader_election.cc:304] T 8c11074579b046d99e585c2818e18a2c P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:20:45.700789 16818 raft_consensus.cc:2368] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac for earlier term 10. Current term is 11.
I20260709 13:20:45.697306 19982 raft_consensus.cc:515] T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:45.467233 20181 consensus_queue.cc:237] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:45.732772 16823 raft_consensus.cc:2368] T 8638ee363273426db1393dd81e5c7b72 P cb82cc7bd6db46d980c75237a1402860 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 for earlier term 11. Current term is 12.
I20260709 13:20:45.716267 20270 consensus_queue.cc:237] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 10, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:45.712527 19990 consensus_queue.cc:260] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:20:45.718118 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 262078us
I20260709 13:20:45.742801 16221 raft_consensus.cc:2393] T 66f75124215f4ca6bef831e174a792d8 P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 in current term 11: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
I20260709 13:20:45.729285 16048 catalog_manager.cc:5697] T 5a959dbecc2648eb85d1cc550f5a3245 P 647a343a542d4a139fe106c19095129b reported cstate change: term changed from 1 to 5, leader changed from 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130) to 647a343a542d4a139fe106c19095129b (127.14.177.129). New cstate: current_term: 5 leader_uuid: "647a343a542d4a139fe106c19095129b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } health_report { overall_health: UNKNOWN } } }
I20260709 13:20:45.688613 20065 raft_consensus.cc:2749] T 2794af5b99984efebdee564035841cb0 P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:20:45.701642 19673 raft_consensus.cc:3060] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:20:45.648712 20267 raft_consensus.cc:515] T 84f2da98864b4425bf7c0055951fdd4b P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:45.652487 20271 raft_consensus.cc:515] T b87147731a68400e8cce9b879daa7c5d P cb82cc7bd6db46d980c75237a1402860 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:45.748147 16144 raft_consensus.cc:1275] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:20:45.767913 16048 catalog_manager.cc:5697] T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b reported cstate change: term changed from 1 to 5. New cstate: current_term: 5 leader_uuid: "647a343a542d4a139fe106c19095129b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:20:45.769925 16220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18ad7e3af5504571a69d10c00750a01c" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
W20260709 13:20:45.745600 16790 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.14.177.129:36741 timed out after 1.746s (SENT) [suppressed 27 similar messages]
I20260709 13:20:45.756299 16142 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40844 (request call id 2322) took 1045 ms. Trace:
W20260709 13:20:45.757241 16265 leader_election.cc:336] T b87147731a68400e8cce9b879daa7c5d P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Timed out: RequestConsensusVote RPC to 127.14.177.129:36741 timed out after 2.212s (SENT)
I20260709 13:20:45.776386 16265 leader_election.cc:304] T b87147731a68400e8cce9b879daa7c5d P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
I20260709 13:20:45.747696 16149 raft_consensus.cc:1275] T 829b4ad8c5a54eababeea1181b960d4b P 647a343a542d4a139fe106c19095129b [term 10 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:20:45.759924 16148 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46224 (request call id 616) took 1024 ms. Trace:
I20260709 13:20:45.777604 16148 rpcz_store.cc:276] 0709 13:20:44.735719 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:45.735937 (+1000218us) service_pool.cc:224] Handling call
0709 13:20:45.759319 (+ 23382us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:45.759536 (+   217us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:45.759571 (+    35us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:20:45.759624 (+    53us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:20:45.759666 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:45.759732 (+    66us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:45.759856 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:20:45.739370 20304 log.cc:927] Time spent T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 0.052s	user 0.001s	sys 0.000s
I20260709 13:20:45.764513 16222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87ac0d1f7e8a419f9bcbc04aee0e067e" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:45.776254 16142 rpcz_store.cc:276] 0709 13:20:44.711249 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:44.711464 (+   215us) service_pool.cc:224] Handling call
0709 13:20:45.756050 (+1044586us) spinlock_profiling.cc:243] Waited 20 ms on lock 0x7b54000ea454. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:20:45.756213 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":46053120}
I20260709 13:20:45.779868 16222 raft_consensus.cc:2468] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 11.
I20260709 13:20:45.779923 16144 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40844 (request call id 2324) took 1023 ms. Trace:
I20260709 13:20:45.694270 20374 raft_consensus.cc:2749] T c9944d5e4b8c4109af9955c4f08c9095 P cb82cc7bd6db46d980c75237a1402860 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:20:45.780993 16149 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46224 (request call id 615) took 1045 ms. Trace:
I20260709 13:20:45.780864 16144 rpcz_store.cc:276] 0709 13:20:44.756194 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:45.724328 (+968134us) service_pool.cc:224] Handling call
0709 13:20:45.744564 (+ 20236us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:45.767054 (+ 22490us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:45.779857 (+ 12803us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:20:45.763154 16221 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33050 (request call id 702) took 1338 ms (client timeout 1746 ms). Trace:
I20260709 13:20:45.781577 16149 rpcz_store.cc:276] 0709 13:20:44.735424 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:45.711490 (+976066us) service_pool.cc:224] Handling call
0709 13:20:45.742466 (+ 30976us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:45.777257 (+ 34791us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:45.780942 (+  3685us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:20:45.785079 20094 leader_election.cc:290] T fffab26427e24d398e2695aad40ca134 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:45.785517 20330 leader_election.cc:290] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
W20260709 13:20:45.782474 16221 rpcz_store.cc:269] 0709 13:20:44.424489 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:44.424615 (+   126us) service_pool.cc:224] Handling call
0709 13:20:45.762959 (+1338344us) spinlock_profiling.cc:243] Waited 1.31 s on lock 0x7b54001428d4. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:20:45.763097 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3017993600}
I20260709 13:20:45.787245 16048 catalog_manager.cc:5697] T 5a96373bb0e14dce8a266b2fc6c58507 P 647a343a542d4a139fe106c19095129b reported cstate change: term changed from 1 to 5, leader changed from 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130) to 647a343a542d4a139fe106c19095129b (127.14.177.129). New cstate: current_term: 5 leader_uuid: "647a343a542d4a139fe106c19095129b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } health_report { overall_health: UNKNOWN } } }
I20260709 13:20:45.794845 16147 raft_consensus.cc:2468] T 5a959dbecc2648eb85d1cc550f5a3245 P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 9.
W20260709 13:20:45.787283 20378 log.cc:927] Time spent T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 0.434s	user 0.001s	sys 0.004s
W20260709 13:20:45.850265 16151 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.079s	user 0.020s	sys 0.037s
I20260709 13:20:45.801916 16145 raft_consensus.cc:1275] T fffab26427e24d398e2695aad40ca134 P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
W20260709 13:20:45.850636 16151 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.079s	user 0.020s	sys 0.037s
I20260709 13:20:45.831048 16295 raft_consensus.cc:3055] T fffab26427e24d398e2695aad40ca134 P 90f3da57796b488c9087a2f523adf045 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:20:45.839715 20267 leader_election.cc:290] T 84f2da98864b4425bf7c0055951fdd4b P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:20:45.839825 16221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c74490c40243a88e62d1760b5cd814" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:20:45.857683 16221 raft_consensus.cc:3060] T 08c74490c40243a88e62d1760b5cd814 P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:20:45.840935 16220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a959dbecc2648eb85d1cc550f5a3245" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:45.843827 20283 raft_consensus.cc:2749] T b87147731a68400e8cce9b879daa7c5d P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:20:45.787376 20269 leader_election.cc:290] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:45.847904 20231 leader_election.cc:290] T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:45.854090 16295 raft_consensus.cc:740] T fffab26427e24d398e2695aad40ca134 P 90f3da57796b488c9087a2f523adf045 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
W20260709 13:20:45.786201 20268 raft_consensus.cc:670] T 806d7690c2b644959152074bb2cab0a7 P cb82cc7bd6db46d980c75237a1402860: failed to trigger leader election: Illegal state: only voting members can start elections: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } } [suppressed 3 similar messages]
I20260709 13:20:45.790083 19990 raft_consensus.cc:3060] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:20:45.866878 16297 raft_consensus.cc:2468] T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 11.
I20260709 13:20:45.886689 16220 raft_consensus.cc:2393] T 5a959dbecc2648eb85d1cc550f5a3245 P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 9: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
I20260709 13:20:45.891353 16048 catalog_manager.cc:5697] T 9e65676b21b748728ad39668cfe722ec P 647a343a542d4a139fe106c19095129b reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "90f3da57796b488c9087a2f523adf045" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:20:45.890137 16295 consensus_queue.cc:260] T fffab26427e24d398e2695aad40ca134 P 90f3da57796b488c9087a2f523adf045 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:45.916826 16295 raft_consensus.cc:3060] T fffab26427e24d398e2695aad40ca134 P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:20:45.886194 20257 raft_consensus.cc:493] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:20:45.807898 20330 consensus_queue.cc:1048] T ada5a004ba7c46819051e3ccec71ffdb P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.095s
W20260709 13:20:45.899258 20312 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.740s	user 0.025s	sys 0.034s
W20260709 13:20:45.906050 20369 log.cc:927] Time spent T 74ca8140f35c4ae29e8a664f6bdfc829 P cb82cc7bd6db46d980c75237a1402860: Append to log took a long time: real 0.440s	user 0.000s	sys 0.001s
I20260709 13:20:45.796779 20303 consensus_queue.cc:260] T c9944d5e4b8c4109af9955c4f08c9095 P 90f3da57796b488c9087a2f523adf045 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:45.903757 20310 leader_election.cc:290] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 14 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
W20260709 13:20:45.945048 16264 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:45.938609 16221 raft_consensus.cc:2410] T 08c74490c40243a88e62d1760b5cd814 P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b for term 10 because replica has last-logged OpId of term: 9 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 6.
W20260709 13:20:45.952212 16791 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:45.961539 16292 raft_consensus.cc:2468] T 8c11074579b046d99e585c2818e18a2c P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 11.
I20260709 13:20:45.967706 16292 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48684 (request call id 2469) took 1037 ms. Trace:
I20260709 13:20:45.946025 20236 raft_consensus.cc:515] T d16dfad0c3b540e5b4b86492ddf87208 P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:45.972018 20236 leader_election.cc:290] T d16dfad0c3b540e5b4b86492ddf87208 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 11 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:45.973207 16150 raft_consensus.cc:1240] T d16dfad0c3b540e5b4b86492ddf87208 P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 10. Current term is 11. Ops: [10.6-10.6]
W20260709 13:20:45.935526 20312 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.818s	user 0.025s	sys 0.035s
I20260709 13:20:45.968283 16292 rpcz_store.cc:276] 0709 13:20:44.930585 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:45.136989 (+206404us) service_pool.cc:224] Handling call
0709 13:20:45.967585 (+830596us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:20:45.993235 16295 raft_consensus.cc:1275] T fffab26427e24d398e2695aad40ca134 P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:20:45.998184 16295 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48684 (request call id 2467) took 1119 ms. Trace:
I20260709 13:20:45.998668 16295 rpcz_store.cc:276] 0709 13:20:44.874912 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:45.786916 (+912004us) service_pool.cc:224] Handling call
0709 13:20:45.830881 (+ 43965us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:45.993705 (+162824us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:45.993921 (+   216us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:20:46.002857 16791 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:45.973542 20065 raft_consensus.cc:2764] T 8c11074579b046d99e585c2818e18a2c P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Leader pre-election decision vote started in defunct term 10: won
W20260709 13:20:46.015580 16291 consensus_meta.cc:287] T 7c2095a5f4ad49408dbc7f69c606120d P 90f3da57796b488c9087a2f523adf045: Time spent flushing consensus metadata: real 0.590s	user 0.011s	sys 0.000s
I20260709 13:20:45.936945 20257 raft_consensus.cc:515] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:45.982805 20285 leader_election.cc:290] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 11 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:20:46.020535 16291 raft_consensus.cc:2468] T 7c2095a5f4ad49408dbc7f69c606120d P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 8.
I20260709 13:20:46.024513 16291 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48684 (request call id 2459) took 1410 ms. Trace:
I20260709 13:20:46.025070 16291 rpcz_store.cc:276] 0709 13:20:44.613913 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:45.118395 (+504482us) service_pool.cc:224] Handling call
0709 13:20:46.024437 (+906042us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:20:45.986500 20294 leader_election.cc:290] T 8638ee363273426db1393dd81e5c7b72 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 12 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:45.993326 20303 raft_consensus.cc:3060] T c9944d5e4b8c4109af9955c4f08c9095 P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:20:46.020069 20289 raft_consensus.cc:493] T 18ad7e3af5504571a69d10c00750a01c P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:20:46.040540 20323 leader_election.cc:290] T 66f75124215f4ca6bef831e174a792d8 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 11 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:46.043251 20280 leader_election.cc:290] T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 12 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
W20260709 13:20:46.103842 16788 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:20:46.155793 16192 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.726s	user 0.000s	sys 0.021s
W20260709 13:20:46.156416 16192 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.727s	user 0.000s	sys 0.021s
W20260709 13:20:46.157130 16192 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:46.021893 19982 leader_election.cc:290] T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:46.202425 16221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ad5dbfa99ee4b8d9e28be92b3f36db5" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:46.042917 20048 raft_consensus.cc:493] T ee9ed67006144797b1de6ea8c2013a68 P cb82cc7bd6db46d980c75237a1402860 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:20:46.206883 16823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62fe3346cd3c42c4a55e14af9073bca4" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:20:46.207701 16823 raft_consensus.cc:3060] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:20:46.208493 16818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66f75124215f4ca6bef831e174a792d8" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:20:46.209213 16818 raft_consensus.cc:3060] T 66f75124215f4ca6bef831e174a792d8 P cb82cc7bd6db46d980c75237a1402860 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:20:46.211848 16264 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:46.092293 20271 leader_election.cc:290] T b87147731a68400e8cce9b879daa7c5d P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:46.127028 20283 raft_consensus.cc:493] T d16dfad0c3b540e5b4b86492ddf87208 P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:20:46.074095 20289 raft_consensus.cc:515] T 18ad7e3af5504571a69d10c00750a01c P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } }
I20260709 13:20:46.248628 16222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1df93779209748b9860f6d893dd3de53" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:20:46.249174 16219 raft_consensus.cc:1275] T c0eab25ce4c64e018a041beeda2867db P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:20:46.249883 16220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:46.255879 16220 raft_consensus.cc:2393] T ecc1e363230043fe9fef390f0b979f20 P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 10: Already voted for candidate cb82cc7bd6db46d980c75237a1402860 in this term.
I20260709 13:20:46.253016 16221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c11074579b046d99e585c2818e18a2c" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:46.196655 20323 raft_consensus.cc:493] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:20:46.257486 16221 raft_consensus.cc:2393] T 8c11074579b046d99e585c2818e18a2c P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 11: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
W20260709 13:20:46.269816 16190 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.870s	user 0.015s	sys 0.031s
W20260709 13:20:46.270514 16190 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.871s	user 0.016s	sys 0.032s
I20260709 13:20:46.275592 16818 raft_consensus.cc:2468] T 66f75124215f4ca6bef831e174a792d8 P cb82cc7bd6db46d980c75237a1402860 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 11.
I20260709 13:20:46.278296 20315 raft_consensus.cc:493] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:46.278605 16151 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40844 (request call id 2318) took 1599 ms. Trace:
I20260709 13:20:46.279444 16221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74ca8140f35c4ae29e8a664f6bdfc829" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:20:46.278957 20315 raft_consensus.cc:515] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:46.280040 16221 raft_consensus.cc:3060] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:20:46.281946 20315 leader_election.cc:290] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:46.286772 16823 raft_consensus.cc:2468] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 14.
W20260709 13:20:46.293509 16115 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.879s	user 0.021s	sys 0.017s
W20260709 13:20:46.294730 16115 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.881s	user 0.021s	sys 0.017s
I20260709 13:20:46.295584 16220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
W20260709 13:20:46.296078 16115 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:20:46.296602 16115 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:46.296475 16220 raft_consensus.cc:2468] T 19baa7bc367844c288bca8f81439c413 P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 11.
W20260709 13:20:46.297240 16115 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:20:46.300191 20382 log.cc:927] Time spent T 31969392ccdb43b491911a319e8bc7e1 P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.318s	user 0.009s	sys 0.000s
I20260709 13:20:46.301367 16221 raft_consensus.cc:2468] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 11.
W20260709 13:20:46.301608 19673 consensus_meta.cc:287] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860: Time spent flushing consensus metadata: real 0.535s	user 0.008s	sys 0.000s
I20260709 13:20:46.306628 19673 raft_consensus.cc:2749] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:20:46.279042 16151 rpcz_store.cc:276] 0709 13:20:44.679609 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:44.681532 (+  1923us) service_pool.cc:224] Handling call
0709 13:20:44.704818 (+ 23286us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:20:44.705234 (+   416us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:44.705283 (+    49us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:20:44.705854 (+   571us) log.cc:844] Serialized 26 byte log entry
0709 13:20:45.851006 (+1145152us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:20:45.851059 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:45.851142 (+    83us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:20:46.278419 (+427277us) raft_consensus.cc:1685] finished
0709 13:20:46.278437 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:46.278550 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":36562,"raft.queue_time_us":227,"raft.run_cpu_time_us":170,"raft.run_wall_time_us":15321,"thread_start_us":1079285,"threads_started":1,"wal-append.queue_time_us":1543338}
W20260709 13:20:46.311338 16263 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.892s	user 0.012s	sys 0.033s
W20260709 13:20:46.311762 16263 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.892s	user 0.012s	sys 0.033s
I20260709 13:20:46.312878 16291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18ad7e3af5504571a69d10c00750a01c" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:20:46.313568 16291 raft_consensus.cc:3060] T 18ad7e3af5504571a69d10c00750a01c P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:20:46.315413 16293 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.519s	user 0.026s	sys 0.012s
I20260709 13:20:46.315490 16292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87ac0d1f7e8a419f9bcbc04aee0e067e" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
W20260709 13:20:46.315846 16293 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.519s	user 0.026s	sys 0.012s
W20260709 13:20:46.330775 16192 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 64334.6us
I20260709 13:20:46.258033 20283 raft_consensus.cc:515] T d16dfad0c3b540e5b4b86492ddf87208 P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:46.341593 16192 leader_election.cc:304] T 8c11074579b046d99e585c2818e18a2c P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:20:46.293514 20310 raft_consensus.cc:493] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:46.362335 16292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c74490c40243a88e62d1760b5cd814" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
W20260709 13:20:46.363626 16264 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:46.262902 20323 raft_consensus.cc:515] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:46.293649 20315 raft_consensus.cc:493] T b6f07842d3aa47858df5665874c86f3a P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:20:46.357923 16117 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.898s	user 0.018s	sys 0.016s
W20260709 13:20:46.389410 16117 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.929s	user 0.018s	sys 0.016s
I20260709 13:20:46.359220 16192 leader_election.cc:400] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 10 election: Vote denied by peer cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923) with higher term. Message: Invalid argument: T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac for earlier term 10. Current term is 11.
I20260709 13:20:46.390038 16192 leader_election.cc:403] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 10 election: Cancelling election due to peer responding with higher term
W20260709 13:20:46.389955 16117 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 929651us
W20260709 13:20:46.365701 16816 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.804s	user 0.008s	sys 0.028s
W20260709 13:20:46.392814 16816 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.831s	user 0.009s	sys 0.028s
I20260709 13:20:46.331488 16190 leader_election.cc:304] T 5a959dbecc2648eb85d1cc550f5a3245 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
W20260709 13:20:46.397570 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 66188.1us
I20260709 13:20:46.398602 16190 leader_election.cc:304] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860
I20260709 13:20:46.391053 16192 leader_election.cc:304] T 66f75124215f4ca6bef831e174a792d8 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:20:46.405143 16192 leader_election.cc:304] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:20:46.409163 16147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1df93779209748b9860f6d893dd3de53" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
W20260709 13:20:46.363840 20383 log.cc:927] Time spent T d2ab0ac8347641e6955e9bce7d90c1a2 P cb82cc7bd6db46d980c75237a1402860: Append to log took a long time: real 0.113s	user 0.002s	sys 0.000s
I20260709 13:20:46.295492 20048 raft_consensus.cc:515] T ee9ed67006144797b1de6ea8c2013a68 P cb82cc7bd6db46d980c75237a1402860 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:46.419602 16292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a9de347c711477e87c582d003f36925" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
W20260709 13:20:46.422400 16146 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.719s	user 0.014s	sys 0.026s
W20260709 13:20:46.422821 16146 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.726s	user 0.015s	sys 0.026s
I20260709 13:20:46.427496 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c61a39fd4f84424b129c1a6f9ba404d" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:20:46.428297 16150 raft_consensus.cc:3060] T 9c61a39fd4f84424b129c1a6f9ba404d P 647a343a542d4a139fe106c19095129b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:20:46.429242 16151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 12 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:46.429847 16151 raft_consensus.cc:2468] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 11.
I20260709 13:20:46.434235 16291 raft_consensus.cc:2410] T 18ad7e3af5504571a69d10c00750a01c P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b for term 10 because replica has last-logged OpId of term: 9 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 6.
I20260709 13:20:46.435760 16291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a959dbecc2648eb85d1cc550f5a3245" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:46.356794 20022 raft_consensus.cc:493] T 84f2da98864b4425bf7c0055951fdd4b P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:20:46.412873 16116 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.872s	user 0.004s	sys 0.033s
W20260709 13:20:46.437614 16294 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.518s	user 0.006s	sys 0.031s
I20260709 13:20:46.359825 19982 raft_consensus.cc:493] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:20:46.403316 20267 raft_consensus.cc:493] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:46.406975 20231 raft_consensus.cc:493] T 8158bbd17b994d2eb96f560bd0f1b88d P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:46.447305 16117 leader_election.cc:304] T 9e65676b21b748728ad39668cfe722ec P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860
W20260709 13:20:46.440899 16223 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.780s	user 0.010s	sys 0.025s
W20260709 13:20:46.442646 16791 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:20:46.453061 16223 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.793s	user 0.010s	sys 0.025s
W20260709 13:20:46.440511 20274 consensus_meta.cc:287] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860: Time spent flushing consensus metadata: real 0.983s	user 0.010s	sys 0.002s
I20260709 13:20:46.458489 16117 leader_election.cc:304] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860
W20260709 13:20:46.443140 16294 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.524s	user 0.006s	sys 0.031s
W20260709 13:20:46.460215 16299 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.018s	user 0.012s	sys 0.034s
W20260709 13:20:46.460799 16299 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.034s	user 0.012s	sys 0.035s
W20260709 13:20:46.461808 16266 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.729s	user 0.017s	sys 0.027s
W20260709 13:20:46.462327 16266 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.730s	user 0.018s	sys 0.027s
I20260709 13:20:46.442768 16291 raft_consensus.cc:2393] T 5a959dbecc2648eb85d1cc550f5a3245 P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 9: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
I20260709 13:20:46.402323 20294 consensus_queue.cc:1059] T 74ca8140f35c4ae29e8a664f6bdfc829 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.050s
I20260709 13:20:46.364693 20337 consensus_queue.cc:1048] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.081s
I20260709 13:20:46.389525 20310 raft_consensus.cc:515] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:20:46.468259 16266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 735668us
W20260709 13:20:46.388396 20293 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.133s	user 0.003s	sys 0.036s
I20260709 13:20:46.514559 16291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c11074579b046d99e585c2818e18a2c" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:46.515749 16291 raft_consensus.cc:2393] T 8c11074579b046d99e585c2818e18a2c P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 in current term 11: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
W20260709 13:20:46.482717 19861 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.634s	user 0.017s	sys 0.031s
W20260709 13:20:46.517318 19861 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.669s	user 0.018s	sys 0.032s
I20260709 13:20:46.460800 20253 raft_consensus.cc:2804] T 8c11074579b046d99e585c2818e18a2c P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:20:46.410508 20315 raft_consensus.cc:515] T b6f07842d3aa47858df5665874c86f3a P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:46.540385 20390 raft_consensus.cc:3060] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:20:46.541739 20315 leader_election.cc:290] T b6f07842d3aa47858df5665874c86f3a P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:46.542479 20315 raft_consensus.cc:2804] T 5a959dbecc2648eb85d1cc550f5a3245 P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: Leader election won for term 9
W20260709 13:20:46.543259 16300 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.774s	user 0.017s	sys 0.019s
W20260709 13:20:46.543558 16300 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.774s	user 0.017s	sys 0.019s
I20260709 13:20:46.451515 20022 raft_consensus.cc:515] T 84f2da98864b4425bf7c0055951fdd4b P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:46.454368 19982 raft_consensus.cc:515] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:20:46.537907 20293 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.283s	user 0.004s	sys 0.036s
I20260709 13:20:46.476784 20236 raft_consensus.cc:493] T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:20:46.542974 20315 raft_consensus.cc:697] T 5a959dbecc2648eb85d1cc550f5a3245 P 1ca2725d290e47d68b1a4b609c9816ac [term 9 LEADER]: Becoming Leader. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
W20260709 13:20:46.442800 16116 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.902s	user 0.004s	sys 0.034s
I20260709 13:20:46.460315 20257 leader_election.cc:290] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
W20260709 13:20:46.561316 16218 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.794s	user 0.006s	sys 0.032s
W20260709 13:20:46.561686 16218 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.794s	user 0.006s	sys 0.032s
W20260709 13:20:46.549098 16266 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:46.549875 16291 raft_consensus.cc:1240] T 7c2095a5f4ad49408dbc7f69c606120d P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 7. Current term is 8. Ops: []
W20260709 13:20:46.484823 16265 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.708s	user 0.014s	sys 0.018s
I20260709 13:20:46.456879 20267 raft_consensus.cc:515] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:20:46.569397 16265 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.792s	user 0.016s	sys 0.019s
W20260709 13:20:46.441649 20183 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.351s	user 0.003s	sys 0.032s
W20260709 13:20:46.554152 16296 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.825s	user 0.019s	sys 0.029s
I20260709 13:20:46.481885 16117 leader_election.cc:304] T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860
W20260709 13:20:46.545423 16790 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.722s	user 0.012s	sys 0.025s
W20260709 13:20:46.557366 16791 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:20:46.574674 16790 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.752s	user 0.012s	sys 0.025s
W20260709 13:20:46.575058 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 829453us
I20260709 13:20:46.457629 20231 raft_consensus.cc:515] T 8158bbd17b994d2eb96f560bd0f1b88d P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:20:46.581681 16296 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.853s	user 0.019s	sys 0.029s
W20260709 13:20:46.556387 19941 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.708s	user 0.003s	sys 0.061s
I20260709 13:20:46.591876 20021 raft_consensus.cc:2749] T 9e65676b21b748728ad39668cfe722ec P 647a343a542d4a139fe106c19095129b [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:20:46.612007 16790 leader_election.cc:304] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac
I20260709 13:20:46.613142 16266 leader_election.cc:400] T 8638ee363273426db1393dd81e5c7b72 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 11 pre-election: Vote denied by peer cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923) with higher term. Message: Invalid argument: T 8638ee363273426db1393dd81e5c7b72 P cb82cc7bd6db46d980c75237a1402860 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 for earlier term 11. Current term is 12.
I20260709 13:20:46.613792 16266 leader_election.cc:403] T 8638ee363273426db1393dd81e5c7b72 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 11 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:20:46.598562 20283 leader_election.cc:290] T d16dfad0c3b540e5b4b86492ddf87208 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:20:46.615613 16145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5d59407eb545b895d8d922e0dd69fd" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:20:46.617003 16151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5718c623ebee4d2bb770f0f9ddb2f164" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:20:46.617738 16151 raft_consensus.cc:3060] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:20:46.600894 16191 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.170s	user 0.001s	sys 0.050s
I20260709 13:20:46.605209 16789 leader_election.cc:304] T 8c11074579b046d99e585c2818e18a2c P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
W20260709 13:20:46.573378 20183 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.483s	user 0.004s	sys 0.033s
I20260709 13:20:46.635839 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87147731a68400e8cce9b879daa7c5d" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:46.637130 16292 raft_consensus.cc:3060] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:20:46.621555 16790 leader_election.cc:304] T 19baa7bc367844c288bca8f81439c413 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860; no voters: 90f3da57796b488c9087a2f523adf045
W20260709 13:20:46.621711 16191 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.191s	user 0.001s	sys 0.050s
I20260709 13:20:46.620083 16144 raft_consensus.cc:3055] T 26354361e9b044cea1c2b86c4882fe92 P 647a343a542d4a139fe106c19095129b [term 8 LEADER]: Stepping down as leader of term 8
W20260709 13:20:46.638173 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 1.21059e+06us
I20260709 13:20:46.638228 16144 raft_consensus.cc:740] T 26354361e9b044cea1c2b86c4882fe92 P 647a343a542d4a139fe106c19095129b [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
W20260709 13:20:46.612005 19941 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.764s	user 0.003s	sys 0.061s
W20260709 13:20:46.643894 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 901584us
I20260709 13:20:46.573495 20289 leader_election.cc:290] T 18ad7e3af5504571a69d10c00750a01c P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:20:46.593029 20257 raft_consensus.cc:2804] T 66f75124215f4ca6bef831e174a792d8 P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:20:46.599385 20293 raft_consensus.cc:2804] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:20:46.560168 20236 raft_consensus.cc:515] T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:46.639110 16291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5ad4eb281564936a59f41e387a47156" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:46.618521 20337 raft_consensus.cc:2749] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
W20260709 13:20:46.652750 16143 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.813s	user 0.019s	sys 0.023s
I20260709 13:20:46.653581 16265 leader_election.cc:304] T 02838c86531d4f98bf58d1c22b8baf77 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:20:46.640417 19673 raft_consensus.cc:2804] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:20:46.640511 20274 raft_consensus.cc:2749] T 8c11074579b046d99e585c2818e18a2c P cb82cc7bd6db46d980c75237a1402860 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:20:46.665678 16291 raft_consensus.cc:2468] T e5ad4eb281564936a59f41e387a47156 P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 11.
I20260709 13:20:46.653400 20294 raft_consensus.cc:2804] T 19baa7bc367844c288bca8f81439c413 P cb82cc7bd6db46d980c75237a1402860 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:20:46.672672 20048 leader_election.cc:290] T ee9ed67006144797b1de6ea8c2013a68 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:20:46.673460 20236 leader_election.cc:290] T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:46.657294 20021 raft_consensus.cc:2749] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:20:46.674345 20022 leader_election.cc:290] T 84f2da98864b4425bf7c0055951fdd4b P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:46.676234 20267 leader_election.cc:290] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:46.678306 19861 consensus_queue.cc:1048] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:20:46.678663 20231 leader_election.cc:290] T 8158bbd17b994d2eb96f560bd0f1b88d P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:46.667521 20315 consensus_queue.cc:237] T 5a959dbecc2648eb85d1cc550f5a3245 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:46.680105 19982 leader_election.cc:290] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:46.663525 19673 raft_consensus.cc:493] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:46.683907 19673 raft_consensus.cc:3060] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:20:46.686625 16293 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48684 (request call id 2463) took 2070 ms. Trace:
I20260709 13:20:46.687016 16293 rpcz_store.cc:276] 0709 13:20:44.616656 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:45.671287 (+1054631us) service_pool.cc:224] Handling call
0709 13:20:45.671672 (+   385us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 13:20:45.695876 (+ 24204us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:20:45.740456 (+ 44580us) log.cc:844] Serialized 14 byte log entry
0709 13:20:46.325279 (+584823us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 13:20:46.345825 (+ 20546us) log.cc:844] Serialized 78 byte log entry
0709 13:20:46.346055 (+   230us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:20:46.346088 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:46.346165 (+    77us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:20:46.686366 (+340201us) raft_consensus.cc:1685] finished
0709 13:20:46.686386 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:46.686564 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10214,"prepare.queue_time_us":47993,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":41,"raft.queue_time_us":43137,"raft.run_cpu_time_us":270,"raft.run_wall_time_us":28884,"thread_start_us":528469,"threads_started":1,"wal-append.queue_time_us":908813}
I20260709 13:20:46.688436 20389 raft_consensus.cc:493] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:46.689144 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fffab26427e24d398e2695aad40ca134" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 10 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:46.689706 20183 raft_consensus.cc:2804] T 02838c86531d4f98bf58d1c22b8baf77 P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:20:46.689968 16293 raft_consensus.cc:2410] T fffab26427e24d398e2695aad40ca134 P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b for term 10 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
I20260709 13:20:46.690110 20183 raft_consensus.cc:493] T 02838c86531d4f98bf58d1c22b8baf77 P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:20:46.690519 20183 raft_consensus.cc:3060] T 02838c86531d4f98bf58d1c22b8baf77 P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:20:46.640507 20065 raft_consensus.cc:3060] T 8638ee363273426db1393dd81e5c7b72 P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Advancing to term 12
I20260709 13:20:46.665409 16265 leader_election.cc:304] T 66ae176961b74813a58bc15c9c4c6f2a P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: cb82cc7bd6db46d980c75237a1402860
I20260709 13:20:46.691799 20310 leader_election.cc:290] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:46.692415 20310 consensus_queue.cc:1048] T 8cc959fb7f704fba9009c0595467d074 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:20:46.692807 20388 raft_consensus.cc:493] T 434bff3082d141838068f323bdb0d1ed P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:20:46.693041 20310 consensus_queue.cc:1243] T 8cc959fb7f704fba9009c0595467d074 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Peer 647a343a542d4a139fe106c19095129b log is divergent from this leader: its last log entry 10.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 13:20:46.693415 20388 raft_consensus.cc:515] T 434bff3082d141838068f323bdb0d1ed P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:46.688954 20389 raft_consensus.cc:515] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:46.697225 20389 leader_election.cc:290] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:46.698024 20389 raft_consensus.cc:2804] T 66ae176961b74813a58bc15c9c4c6f2a P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:20:46.698477 20389 raft_consensus.cc:493] T 66ae176961b74813a58bc15c9c4c6f2a P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Starting leader election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:20:46.699838 20337 raft_consensus.cc:3055] T 18ad7e3af5504571a69d10c00750a01c P 1ca2725d290e47d68b1a4b609c9816ac [term 9 LEADER]: Stepping down as leader of term 9
W20260709 13:20:46.700814 16264 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:20:46.658806 16143 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.829s	user 0.019s	sys 0.024s
I20260709 13:20:46.698923 20389 raft_consensus.cc:3060] T 66ae176961b74813a58bc15c9c4c6f2a P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:20:46.700244 20337 raft_consensus.cc:740] T 18ad7e3af5504571a69d10c00750a01c P 1ca2725d290e47d68b1a4b609c9816ac [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:20:46.667389 20323 leader_election.cc:290] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
W20260709 13:20:46.761823 16263 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:46.763445 20388 leader_election.cc:290] T 434bff3082d141838068f323bdb0d1ed P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
W20260709 13:20:46.667425 16037 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.689s	user 0.009s	sys 0.054s
I20260709 13:20:46.671017 20294 raft_consensus.cc:493] T 19baa7bc367844c288bca8f81439c413 P cb82cc7bd6db46d980c75237a1402860 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:20:46.767407 16297 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.779s	user 0.000s	sys 0.033s
W20260709 13:20:46.772923 20306 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.742s	user 0.010s	sys 0.037s
I20260709 13:20:46.694865 16265 leader_election.cc:304] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860
I20260709 13:20:46.773522 16291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d16dfad0c3b540e5b4b86492ddf87208" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:20:46.763643 16816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a9de347c711477e87c582d003f36925" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:46.874364 16291 raft_consensus.cc:3060] T d16dfad0c3b540e5b4b86492ddf87208 P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:20:46.749660 16298 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.723s	user 0.015s	sys 0.032s
I20260709 13:20:46.723507 20267 consensus_queue.cc:1048] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.047s
I20260709 13:20:46.783226 16292 raft_consensus.cc:2468] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 13.
W20260709 13:20:46.861773 16297 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.935s	user 0.001s	sys 0.034s
I20260709 13:20:46.834900 16150 raft_consensus.cc:2468] T 9c61a39fd4f84424b129c1a6f9ba404d P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 11.
W20260709 13:20:46.848678 16791 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:46.874455 16816 raft_consensus.cc:2468] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 13.
W20260709 13:20:46.875414 16298 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.849s	user 0.015s	sys 0.033s
W20260709 13:20:46.882820 16226 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.483s	user 0.002s	sys 0.055s
I20260709 13:20:46.882052 16823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7ca61592e848ea9b0fb81a2ce33cdc" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:46.883915 16823 raft_consensus.cc:2468] T 3a7ca61592e848ea9b0fb81a2ce33cdc P cb82cc7bd6db46d980c75237a1402860 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 13.
I20260709 13:20:46.793125 16144 consensus_queue.cc:260] T 26354361e9b044cea1c2b86c4882fe92 P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:20:46.769907 20303 consensus_meta.cc:287] T c9944d5e4b8c4109af9955c4f08c9095 P 90f3da57796b488c9087a2f523adf045: Time spent flushing consensus metadata: real 0.714s	user 0.009s	sys 0.002s
W20260709 13:20:46.883361 16226 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.484s	user 0.003s	sys 0.055s
W20260709 13:20:46.843545 19990 consensus_meta.cc:287] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860: Time spent flushing consensus metadata: real 0.934s	user 0.018s	sys 0.003s
W20260709 13:20:46.873996 16037 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.899s	user 0.016s	sys 0.056s
W20260709 13:20:46.873433 20306 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.850s	user 0.010s	sys 0.038s
I20260709 13:20:46.896021 16144 raft_consensus.cc:3060] T 26354361e9b044cea1c2b86c4882fe92 P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:20:46.874639 20294 raft_consensus.cc:3060] T 19baa7bc367844c288bca8f81439c413 P cb82cc7bd6db46d980c75237a1402860 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:20:46.902065 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 207269us
I20260709 13:20:46.873425 20236 raft_consensus.cc:2749] T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:20:46.903270 16297 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48684 (request call id 2470) took 1439 ms. Trace:
I20260709 13:20:46.910351 16297 rpcz_store.cc:276] 0709 13:20:45.464411 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:45.911274 (+446863us) service_pool.cc:224] Handling call
0709 13:20:45.911698 (+   424us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:45.911964 (+   266us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:20:45.926054 (+ 14090us) log.cc:844] Serialized 14 byte log entry
0709 13:20:46.902802 (+976748us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:20:46.902878 (+    76us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:20:46.902939 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:46.903024 (+    85us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:46.903198 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":22869,"prepare.queue_time_us":14327,"prepare.run_cpu_time_us":42,"prepare.run_wall_time_us":43,"raft.queue_time_us":24062,"thread_start_us":952193,"threads_started":1}
I20260709 13:20:46.916079 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8638ee363273426db1393dd81e5c7b72" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:20:46.920106 16291 raft_consensus.cc:2410] T d16dfad0c3b540e5b4b86492ddf87208 P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b for term 11 because replica has last-logged OpId of term: 10 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 13:20:46.921469 16291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4f65cdf00b748bb963ee6e81ef43b79" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:46.922415 16291 raft_consensus.cc:2393] T c4f65cdf00b748bb963ee6e81ef43b79 P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 12: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
I20260709 13:20:46.903599 16226 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33050 (request call id 709) took 2375 ms. Trace:
I20260709 13:20:46.923957 16265 leader_election.cc:304] T a378d7f796f94615b3b347119a9880fd P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: cb82cc7bd6db46d980c75237a1402860
W20260709 13:20:46.928117 16115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 443900us
I20260709 13:20:46.928818 16265 leader_election.cc:304] T d2ab0ac8347641e6955e9bce7d90c1a2 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: cb82cc7bd6db46d980c75237a1402860
I20260709 13:20:46.895366 20269 raft_consensus.cc:493] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
W20260709 13:20:46.931427 16115 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:20:46.926941 20181 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.660s	user 0.006s	sys 0.025s
I20260709 13:20:46.931092 16265 leader_election.cc:304] T cdbd78acb4cf46298e0de8ba0d130e19 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860
I20260709 13:20:46.930830 20289 raft_consensus.cc:2804] T a378d7f796f94615b3b347119a9880fd P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:20:46.925292 20401 raft_consensus.cc:2749] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:20:46.935521 20398 raft_consensus.cc:2804] T d2ab0ac8347641e6955e9bce7d90c1a2 P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 13:20:47.004774 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 73750us
I20260709 13:20:47.022223 20398 raft_consensus.cc:493] T d2ab0ac8347641e6955e9bce7d90c1a2 P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Starting leader election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:20:47.088083 20398 raft_consensus.cc:3060] T d2ab0ac8347641e6955e9bce7d90c1a2 P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:20:47.006268 16115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 64326.7us
I20260709 13:20:47.089066 16115 leader_election.cc:304] T e5ad4eb281564936a59f41e387a47156 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:20:47.026175 16292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87ac0d1f7e8a419f9bcbc04aee0e067e" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 12 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:47.005153 20303 raft_consensus.cc:2749] T cdbd78acb4cf46298e0de8ba0d130e19 P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:20:47.052531 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8158bbd17b994d2eb96f560bd0f1b88d" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
W20260709 13:20:47.055347 16190 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.649s	user 0.003s	sys 0.035s
W20260709 13:20:47.065901 16299 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.604s	user 0.031s	sys 0.053s
W20260709 13:20:47.104934 16299 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.643s	user 0.031s	sys 0.053s
W20260709 13:20:47.071770 16295 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.664s	user 0.005s	sys 0.038s
W20260709 13:20:47.105618 16295 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.698s	user 0.005s	sys 0.038s
I20260709 13:20:47.106017 16299 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40474 (request call id 685) took 2924 ms. Trace:
W20260709 13:20:46.946439 20181 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.685s	user 0.006s	sys 0.025s
W20260709 13:20:47.104873 16190 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.698s	user 0.005s	sys 0.035s
W20260709 13:20:47.074178 16791 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:20:47.080124 16146 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.638s	user 0.018s	sys 0.027s
I20260709 13:20:47.106468 16299 rpcz_store.cc:276] 0709 13:20:44.182091 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:45.163708 (+981617us) service_pool.cc:224] Handling call
0709 13:20:45.224678 (+ 60970us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:45.224951 (+   273us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:20:45.298181 (+ 73230us) log.cc:844] Serialized 14 byte log entry
0709 13:20:46.299569 (+1001388us) spinlock_profiling.cc:243] Waited 2.97 ms on lock 0x7b4400022b4c. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69ad8fadb1 00007f69ad998c85 00007f69ad9996a2 00007f69b070b1db 00007f69b0706aaf 00007f69b06fa86e 00007f69b06f9bb6 00007f69b06f97d8 00007f69b0703221 00007f69b07031d9 00007f69b0703169
0709 13:20:47.105447 (+805878us) log.cc:844] Serialized 14 byte log entry
0709 13:20:47.105662 (+   215us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:20:47.105716 (+    54us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:20:47.105774 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:47.105851 (+    77us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:47.105965 (+   114us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":51173,"prepare.queue_time_us":73465,"prepare.run_cpu_time_us":51,"prepare.run_wall_time_us":53,"raft.queue_time_us":68161,"raft.run_cpu_time_us":69,"raft.run_wall_time_us":68,"spinlock_wait_cycles":6840576,"thread_start_us":1748366,"threads_started":3,"wal-append.queue_time_us":627610}
W20260709 13:20:47.045375 20395 log.cc:927] Time spent T 02838c86531d4f98bf58d1c22b8baf77 P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 0.137s	user 0.000s	sys 0.000s
W20260709 13:20:47.047644 20407 log.cc:927] Time spent T 26354361e9b044cea1c2b86c4882fe92 P cb82cc7bd6db46d980c75237a1402860: Append to log took a long time: real 0.124s	user 0.000s	sys 0.000s
W20260709 13:20:47.110188 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 711673us
W20260709 13:20:47.044062 20409 log.cc:927] Time spent T 84f2da98864b4425bf7c0055951fdd4b P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.110s	user 0.000s	sys 0.000s
W20260709 13:20:47.070117 20403 log.cc:927] Time spent T 1df93779209748b9860f6d893dd3de53 P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 0.064s	user 0.000s	sys 0.000s
W20260709 13:20:47.092724 16048 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.575s	user 0.024s	sys 0.039s
W20260709 13:20:47.127542 16048 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.610s	user 0.024s	sys 0.039s
I20260709 13:20:47.002404 16291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4f65cdf00b748bb963ee6e81ef43b79" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:47.128199 16291 raft_consensus.cc:2468] T c4f65cdf00b748bb963ee6e81ef43b79 P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 12.
I20260709 13:20:47.066706 20280 raft_consensus.cc:3055] T 74ca8140f35c4ae29e8a664f6bdfc829 P cb82cc7bd6db46d980c75237a1402860 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:20:47.116496 16292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434bff3082d141838068f323bdb0d1ed" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:47.107596 20388 raft_consensus.cc:2804] T e5ad4eb281564936a59f41e387a47156 P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 13:20:47.117204 16189 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.586s	user 0.005s	sys 0.008s
W20260709 13:20:47.146858 16189 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.615s	user 0.005s	sys 0.009s
W20260709 13:20:47.117452 16192 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.711s	user 0.004s	sys 0.012s
W20260709 13:20:47.038863 16217 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33050 (request call id 705) took 2574 ms (client timeout 2046 ms). Trace:
W20260709 13:20:47.113744 16265 connection.cc:256] RPC call timeout handler was delayed by -1.6369208950000029s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15410000000000001s for a response
I20260709 13:20:47.149102 16221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5ad4eb281564936a59f41e387a47156" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
W20260709 13:20:47.149386 16265 connection.cc:256] RPC call timeout handler was delayed by -1.4857208949999858s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17090000000000002s for a response
W20260709 13:20:47.149936 16192 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.744s	user 0.004s	sys 0.012s
W20260709 13:20:47.110692 16146 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.669s	user 0.019s	sys 0.027s
W20260709 13:20:47.150238 16192 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 745166us
I20260709 13:20:47.054693 16151 raft_consensus.cc:2468] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 11.
W20260709 13:20:47.127849 16116 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.525s	user 0.015s	sys 0.027s
W20260709 13:20:47.150982 16116 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.549s	user 0.016s	sys 0.027s
W20260709 13:20:47.151455 16116 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:47.045332 16298 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48684 (request call id 2468) took 2124 ms. Trace:
I20260709 13:20:47.015998 20289 raft_consensus.cc:493] T a378d7f796f94615b3b347119a9880fd P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Starting leader election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:20:47.064836 16143 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46224 (request call id 612) took 2330 ms. Trace:
I20260709 13:20:47.101596 20236 raft_consensus.cc:493] T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:47.142555 16292 raft_consensus.cc:2468] T 434bff3082d141838068f323bdb0d1ed P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 9.
W20260709 13:20:47.149317 16217 rpcz_store.cc:269] 0709 13:20:44.455839 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:44.456340 (+   501us) service_pool.cc:224] Handling call
0709 13:20:47.030416 (+2574076us) spinlock_profiling.cc:243] Waited 2.44 s on lock 0x7b5400139554. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:20:47.030600 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5622783360}
I20260709 13:20:47.157642 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87147731a68400e8cce9b879daa7c5d" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
W20260709 13:20:47.105115 20253 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.551s	user 0.014s	sys 0.048s
I20260709 13:20:47.073779 20268 raft_consensus.cc:493] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
W20260709 13:20:47.083699 16037 tablet_replica.cc:1411] Time spent flushing the Log queue took a long time: real 1.293s	user 0.017s	sys 0.057s
I20260709 13:20:47.149991 16221 raft_consensus.cc:2468] T e5ad4eb281564936a59f41e387a47156 P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 10.
I20260709 13:20:47.182653 16816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74ca8140f35c4ae29e8a664f6bdfc829" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:47.151911 16298 rpcz_store.cc:276] 0709 13:20:44.920731 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:45.897658 (+976927us) service_pool.cc:224] Handling call
0709 13:20:45.916532 (+ 18874us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:20:45.941406 (+ 24874us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 13:20:45.941452 (+    46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:20:45.980058 (+ 38606us) log.cc:844] Serialized 26 byte log entry
0709 13:20:46.877923 (+897865us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:20:46.877977 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:46.878050 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:20:47.044898 (+166848us) raft_consensus.cc:1685] finished
0709 13:20:47.044926 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:47.045229 (+   303us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":17588,"raft.run_cpu_time_us":176,"raft.run_wall_time_us":28342,"thread_start_us":851355,"threads_started":1,"wal-append.queue_time_us":1052840}
W20260709 13:20:47.153981 16300 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.504s	user 0.006s	sys 0.005s
I20260709 13:20:47.184405 16220 raft_consensus.cc:1240] T 746aabf856ab401c9ef5e859371e7b67 P 1ca2725d290e47d68b1a4b609c9816ac [term 10 LEADER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 9. Current term is 10. Ops: []
I20260709 13:20:47.185736 16222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fffab26427e24d398e2695aad40ca134" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 10 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
W20260709 13:20:46.939869 20338 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.686s	user 0.006s	sys 0.014s
W20260709 13:20:47.117972 20270 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.582s	user 0.006s	sys 0.007s
W20260709 13:20:47.116850 16117 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.543s	user 0.004s	sys 0.008s
I20260709 13:20:47.167872 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d16dfad0c3b540e5b4b86492ddf87208" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:20:47.170042 16291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1df93779209748b9860f6d893dd3de53" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:47.170522 16146 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40844 (request call id 2323) took 2457 ms. Trace:
I20260709 13:20:47.157620 16143 rpcz_store.cc:276] 0709 13:20:44.734390 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:45.670469 (+936079us) service_pool.cc:224] Handling call
0709 13:20:45.703422 (+ 32953us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:20:45.741799 (+ 38377us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:45.741846 (+    47us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:20:45.746319 (+  4473us) log.cc:844] Serialized 52 byte log entry
0709 13:20:46.861543 (+1115224us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:20:46.861583 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:46.861652 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:20:47.064554 (+202902us) raft_consensus.cc:1685] finished
0709 13:20:47.064582 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:47.064760 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":27647,"raft.run_cpu_time_us":141,"raft.run_wall_time_us":8431,"thread_start_us":1032028,"threads_started":1,"wal-append.queue_time_us":1231571}
I20260709 13:20:46.942800 20269 raft_consensus.cc:515] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:47.145036 16190 leader_election.cc:304] T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
W20260709 13:20:47.179636 20382 log.cc:927] Time spent T 31969392ccdb43b491911a319e8bc7e1 P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
I20260709 13:20:47.115438 20293 raft_consensus.cc:697] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [term 14 LEADER]: Becoming Leader. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:20:47.118958 16226 rpcz_store.cc:276] 0709 13:20:44.528339 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:44.643019 (+114680us) service_pool.cc:224] Handling call
0709 13:20:44.667532 (+ 24513us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:20:44.687867 (+ 20335us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:44.687914 (+    47us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:20:44.706928 (+ 19014us) log.cc:844] Serialized 52 byte log entry
0709 13:20:45.090503 (+383575us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:20:45.191657 (+101154us) log.cc:844] Serialized 14 byte log entry
0709 13:20:45.467964 (+276307us) spinlock_profiling.cc:243] Waited 49.9 ms on lock 0x7b4400022b4c. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69ad8fadb1 00007f69ad998c85 00007f69ad9996a2 00007f69b070b1db 00007f69b0706aaf 00007f69b06fa86e 00007f69b06f9bb6 00007f69b06f97d8 00007f69b0703221 00007f69b07031d9 00007f69b0703169
0709 13:20:46.893237 (+1425273us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:46.893341 (+   104us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:20:46.893397 (+    56us) raft_consensus.cc:1685] finished
0709 13:20:46.903210 (+  9813us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:46.903491 (+   281us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":213232,"prepare.queue_time_us":96410,"prepare.run_cpu_time_us":65,"prepare.run_wall_time_us":59,"raft.queue_time_us":222031,"raft.run_cpu_time_us":254,"raft.run_wall_time_us":11233,"spinlock_wait_cycles":114775936,"thread_start_us":1836853,"threads_started":2,"wal-append.queue_time_us":631726,"wal-append.run_cpu_time_us":7942,"wal-append.run_wall_time_us":1678325}
I20260709 13:20:47.265591 16824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdbd78acb4cf46298e0de8ba0d130e19" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
W20260709 13:20:47.177356 16265 connection.cc:256] RPC call timeout handler was delayed by -1.3399208949999775s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18710000000000002s for a response
I20260709 13:20:47.266250 16824 raft_consensus.cc:2468] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 11.
W20260709 13:20:47.266417 16265 connection.cc:256] RPC call timeout handler was delayed by -1.0366208949999418s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22080000000000002s for a response
I20260709 13:20:47.023974 20393 consensus_queue.cc:1059] T 7c2095a5f4ad49408dbc7f69c606120d P 647a343a542d4a139fe106c19095129b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.127s
W20260709 13:20:47.184485 16300 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.544s	user 0.006s	sys 0.005s
W20260709 13:20:47.267750 16191 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.614s	user 0.004s	sys 0.012s
W20260709 13:20:47.172303 20253 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.620s	user 0.014s	sys 0.048s
W20260709 13:20:47.197434 20394 log.cc:927] Time spent T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860: Append to log took a long time: real 0.182s	user 0.000s	sys 0.001s
I20260709 13:20:47.195492 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f2da98864b4425bf7c0055951fdd4b" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
W20260709 13:20:47.272974 20315 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.543s	user 0.023s	sys 0.057s
W20260709 13:20:47.276908 20315 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.568s	user 0.023s	sys 0.057s
I20260709 13:20:47.195672 16817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada5a004ba7c46819051e3ccec71ffdb" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:47.142285 20280 raft_consensus.cc:740] T 74ca8140f35c4ae29e8a664f6bdfc829 P cb82cc7bd6db46d980c75237a1402860 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:20:47.009435 20337 consensus_queue.cc:260] T 18ad7e3af5504571a69d10c00750a01c P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } }
I20260709 13:20:47.171408 20289 raft_consensus.cc:3060] T a378d7f796f94615b3b347119a9880fd P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:20:47.171787 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4f65cdf00b748bb963ee6e81ef43b79" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:47.286588 16217 raft_consensus.cc:2468] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 12.
I20260709 13:20:47.228575 16292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ad5dbfa99ee4b8d9e28be92b3f36db5" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:47.287946 16292 raft_consensus.cc:2410] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b for term 10 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 13:20:47.191581 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2372981006d4b218215c28964dea623" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
W20260709 13:20:47.233650 16218 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.612s	user 0.013s	sys 0.031s
I20260709 13:20:47.229646 16298 raft_consensus.cc:1240] T cdbd78acb4cf46298e0de8ba0d130e19 P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 10. Current term is 11. Ops: []
W20260709 13:20:47.216404 16264 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:47.239151 16115 leader_election.cc:304] T 434bff3082d141838068f323bdb0d1ed P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
W20260709 13:20:47.288683 16218 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.676s	user 0.014s	sys 0.031s
W20260709 13:20:47.235638 16117 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.661s	user 0.006s	sys 0.014s
I20260709 13:20:47.262732 16291 raft_consensus.cc:2393] T 1df93779209748b9860f6d893dd3de53 P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 11: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
W20260709 13:20:47.237041 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 58786.3us
W20260709 13:20:47.267254 16265 leader_election.cc:336] T 26354361e9b044cea1c2b86c4882fe92 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Timed out: RequestConsensusVote RPC to 127.14.177.129:36741 timed out after 1.541s (SENT)
W20260709 13:20:47.268549 16191 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.615s	user 0.004s	sys 0.012s
I20260709 13:20:47.289505 16265 leader_election.cc:304] T 26354361e9b044cea1c2b86c4882fe92 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
W20260709 13:20:47.289944 16142 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.669s	user 0.009s	sys 0.022s
I20260709 13:20:47.287792 16294 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48684 (request call id 2466) took 2616 ms. Trace:
W20260709 13:20:47.290279 16142 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.670s	user 0.009s	sys 0.022s
W20260709 13:20:47.154299 16790 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.506s	user 0.006s	sys 0.006s
W20260709 13:20:47.273195 16266 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.506s	user 0.000s	sys 0.016s
W20260709 13:20:47.291218 16266 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.524s	user 0.000s	sys 0.016s
W20260709 13:20:47.291610 16266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 525025us
I20260709 13:20:47.290429 16294 rpcz_store.cc:276] 0709 13:20:44.671227 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:45.763891 (+1092664us) service_pool.cc:224] Handling call
0709 13:20:45.785951 (+ 22060us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:20:45.855576 (+ 69625us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:20:45.855621 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:20:45.856160 (+   539us) log.cc:844] Serialized 26 byte log entry
0709 13:20:46.459456 (+603296us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:20:46.459517 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:46.592035 (+132518us) spinlock_profiling.cc:243] Waited 44.1 ms on lock 0x7b540024de54. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:20:46.592087 (+    52us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:20:47.278627 (+686540us) raft_consensus.cc:1685] finished
0709 13:20:47.278653 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:47.287713 (+  9060us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":29170,"raft.run_cpu_time_us":152,"raft.run_wall_time_us":12880,"spinlock_wait_cycles":101473536,"thread_start_us":539857,"threads_started":1,"wal-append.queue_time_us":975625}
W20260709 13:20:47.273720 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 128765us
I20260709 13:20:47.145710 20388 raft_consensus.cc:493] T e5ad4eb281564936a59f41e387a47156 P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:20:47.263533 20270 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.728s	user 0.010s	sys 0.007s
I20260709 13:20:47.177829 20236 raft_consensus.cc:515] T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:47.174028 19673 raft_consensus.cc:515] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:20:47.302477 16117 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 820653us
I20260709 13:20:47.300210 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f2da98864b4425bf7c0055951fdd4b" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:47.301469 16192 leader_election.cc:304] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 
W20260709 13:20:47.194446 20338 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.953s	user 0.006s	sys 0.014s
W20260709 13:20:47.273243 20257 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.609s	user 0.000s	sys 0.015s
I20260709 13:20:47.236414 20021 raft_consensus.cc:493] T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:20:47.308537 20021 raft_consensus.cc:515] T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:20:47.309020 16265 leader_election.cc:336] T 1df93779209748b9860f6d893dd3de53 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Timed out: RequestConsensusVote RPC to 127.14.177.129:36741 timed out after 1.625s (SENT)
I20260709 13:20:47.190066 20268 raft_consensus.cc:515] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:20:47.294572 16790 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.646s	user 0.006s	sys 0.006s
I20260709 13:20:47.274089 20253 raft_consensus.cc:697] T 8c11074579b046d99e585c2818e18a2c P 1ca2725d290e47d68b1a4b609c9816ac [term 11 LEADER]: Becoming Leader. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
W20260709 13:20:47.311103 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 662572us
I20260709 13:20:47.311621 20253 consensus_queue.cc:237] T 8c11074579b046d99e585c2818e18a2c P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 9.10, Last appended by leader: 10, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:47.273586 20402 raft_consensus.cc:493] T 31969392ccdb43b491911a319e8bc7e1 P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:20:47.313664 20274 consensus_queue.cc:1048] T 829b4ad8c5a54eababeea1181b960d4b P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:20:47.314145 20402 raft_consensus.cc:515] T 31969392ccdb43b491911a319e8bc7e1 P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:47.299408 16217 raft_consensus.cc:1275] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
W20260709 13:20:47.289583 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 635661us
I20260709 13:20:47.299993 16220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d16dfad0c3b540e5b4b86492ddf87208" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:47.316634 16191 leader_election.cc:304] T 829b4ad8c5a54eababeea1181b960d4b P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: cb82cc7bd6db46d980c75237a1402860
I20260709 13:20:47.301008 16221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7ca61592e848ea9b0fb81a2ce33cdc" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:47.297276 16223 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57604 (request call id 2519) took 1815 ms. Trace:
I20260709 13:20:47.281044 20306 raft_consensus.cc:2804] T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:20:47.146327 20294 raft_consensus.cc:515] T 19baa7bc367844c288bca8f81439c413 P cb82cc7bd6db46d980c75237a1402860 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:47.307197 16117 leader_election.cc:304] T 57a744f5aaf944678a63469cbf514015 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860
W20260709 13:20:47.310612 16265 leader_election.cc:336] T 84f2da98864b4425bf7c0055951fdd4b P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Timed out: RequestConsensusVote RPC to 127.14.177.129:36741 timed out after 1.709s (SENT)
W20260709 13:20:47.319319 16265 leader_election.cc:336] T c4f65cdf00b748bb963ee6e81ef43b79 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 12 election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Timed out: RequestConsensusVote RPC to 127.14.177.129:36741 timed out after 1.715s (SENT)
I20260709 13:20:47.319803 16265 leader_election.cc:304] T c4f65cdf00b748bb963ee6e81ef43b79 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:20:47.300020 16222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18ad7e3af5504571a69d10c00750a01c" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:47.326151 16820 raft_consensus.cc:1240] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 [term 12 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 11. Current term is 12. Ops: [11.9-11.9]
I20260709 13:20:47.319741 16218 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57604 (request call id 2518) took 1850 ms. Trace:
I20260709 13:20:47.297588 20388 raft_consensus.cc:3060] T e5ad4eb281564936a59f41e387a47156 P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:20:47.331487 16265 leader_election.cc:336] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Timed out: RequestConsensusVote RPC to 127.14.177.129:36741 timed out after 1.741s (SENT)
W20260709 13:20:47.305227 20410 log.cc:927] Time spent T 8158bbd17b994d2eb96f560bd0f1b88d P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.232s	user 0.000s	sys 0.001s
I20260709 13:20:47.333616 20270 raft_consensus.cc:493] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:47.334519 20270 raft_consensus.cc:515] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:47.317746 16223 rpcz_store.cc:276] 0709 13:20:45.482233 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:45.482728 (+   495us) service_pool.cc:224] Handling call
0709 13:20:45.549946 (+ 67218us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:20:45.550221 (+   275us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:45.550260 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:20:45.563785 (+ 13525us) log.cc:844] Serialized 26 byte log entry
0709 13:20:46.453524 (+889739us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:20:46.453559 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:46.453638 (+    79us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:20:47.297070 (+843432us) raft_consensus.cc:1685] finished
0709 13:20:47.297087 (+    17us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:47.297220 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":793106,"threads_started":1,"wal-append.queue_time_us":1244835}
W20260709 13:20:47.306528 20257 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.648s	user 0.000s	sys 0.016s
I20260709 13:20:47.302284 20267 raft_consensus.cc:2749] T 26354361e9b044cea1c2b86c4882fe92 P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:20:47.327843 16218 rpcz_store.cc:276] 0709 13:20:45.469409 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:45.470577 (+  1168us) service_pool.cc:224] Handling call
0709 13:20:45.542798 (+ 72221us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:20:45.684942 (+142144us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:45.684985 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:20:45.696403 (+ 11418us) log.cc:844] Serialized 26 byte log entry
0709 13:20:46.562133 (+865730us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:20:47.289114 (+726981us) log.cc:844] Serialized 14 byte log entry
0709 13:20:47.319505 (+ 30391us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:47.319581 (+    76us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:20:47.319610 (+    29us) raft_consensus.cc:1685] finished
0709 13:20:47.319617 (+     7us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:47.319699 (+    82us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":30135,"thread_start_us":1471441,"threads_started":2,"wal-append.queue_time_us":1183238}
I20260709 13:20:47.337832 16190 leader_election.cc:304] T 7c2095a5f4ad49408dbc7f69c606120d P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:20:47.341308 16226 raft_consensus.cc:1240] T af57996a8be84d398f0f023e8d08ef77 P 1ca2725d290e47d68b1a4b609c9816ac [term 12 LEADER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 11. Current term is 12. Ops: [11.9-11.9]
I20260709 13:20:47.306661 20315 raft_consensus.cc:493] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:20:47.315981 20337 raft_consensus.cc:3060] T 18ad7e3af5504571a69d10c00750a01c P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:20:47.359802 16294 raft_consensus.cc:1240] T 18ad7e3af5504571a69d10c00750a01c P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 9. Current term is 10. Ops: []
I20260709 13:20:47.341910 20330 raft_consensus.cc:2749] T c4f65cdf00b748bb963ee6e81ef43b79 P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
W20260709 13:20:47.362702 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 50304.8us
I20260709 13:20:47.341511 20257 raft_consensus.cc:697] T 66f75124215f4ca6bef831e174a792d8 P 1ca2725d290e47d68b1a4b609c9816ac [term 11 LEADER]: Becoming Leader. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:20:47.363812 16790 leader_election.cc:304] T b87147731a68400e8cce9b879daa7c5d P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:20:47.364795 16790 leader_election.cc:304] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860; no voters: 
W20260709 13:20:47.367612 20415 log.cc:927] Time spent T 3a7ca61592e848ea9b0fb81a2ce33cdc P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 0.187s	user 0.001s	sys 0.000s
W20260709 13:20:47.368930 20414 log.cc:927] Time spent T af57996a8be84d398f0f023e8d08ef77 P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 0.136s	user 0.000s	sys 0.000s
I20260709 13:20:47.369308 20236 leader_election.cc:290] T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:47.358563 20424 raft_consensus.cc:2804] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 13:20:47.369843 20399 log.cc:927] Time spent T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.436s	user 0.005s	sys 0.000s
I20260709 13:20:47.370077 20424 raft_consensus.cc:493] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:47.370519 20424 raft_consensus.cc:3060] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:20:47.370882 16296 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48684 (request call id 2464) took 2754 ms. Trace:
I20260709 13:20:47.371770 19673 leader_election.cc:290] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 12 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:20:47.371254 16296 rpcz_store.cc:276] 0709 13:20:44.617823 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:45.683725 (+1065902us) service_pool.cc:224] Handling call
0709 13:20:45.684136 (+   411us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:20:45.696225 (+ 12089us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:45.696253 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:20:45.713228 (+ 16975us) log.cc:844] Serialized 26 byte log entry
0709 13:20:46.594002 (+880774us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:20:46.594059 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:46.594155 (+    96us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:20:47.369393 (+775238us) spinlock_profiling.cc:243] Waited 2.79 ms on lock 0x7b4400022b4c. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69ad8fadb1 00007f69ad998c85 00007f69ad9996a2 00007f69b070b1db 00007f69b0706aaf 00007f69b06fa86e 00007f69b06f9bb6 00007f69b06f97d8 00007f69b0703221 00007f69b07031d9 00007f69b0703169
0709 13:20:47.370601 (+  1208us) raft_consensus.cc:1685] finished
0709 13:20:47.370626 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:47.370826 (+   200us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":11643,"raft.queue_time_us":101596,"raft.run_cpu_time_us":84,"raft.run_wall_time_us":83,"spinlock_wait_cycles":6419840,"thread_start_us":864680,"threads_started":1,"wal-append.queue_time_us":1163558}
I20260709 13:20:47.373960 16265 leader_election.cc:304] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:20:47.375478 20392 raft_consensus.cc:493] T 8cc959fb7f704fba9009c0595467d074 P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:47.375936 16296 raft_consensus.cc:1240] T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 10. Current term is 11. Ops: []
I20260709 13:20:47.375690 20257 consensus_queue.cc:237] T 66f75124215f4ca6bef831e174a792d8 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:47.376124 20392 raft_consensus.cc:515] T 8cc959fb7f704fba9009c0595467d074 P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:47.378360 20392 leader_election.cc:290] T 8cc959fb7f704fba9009c0595467d074 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
W20260709 13:20:47.383764 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 46014.7us
I20260709 13:20:47.402230 16265 leader_election.cc:304] T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: cb82cc7bd6db46d980c75237a1402860
I20260709 13:20:47.400658 20306 raft_consensus.cc:697] T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac [term 11 LEADER]: Becoming Leader. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:20:47.388013 16294 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "8158bbd17b994d2eb96f560bd0f1b88d"
dest_uuid: "90f3da57796b488c9087a2f523adf045"
 from {username='slave'} at 127.0.0.1:48684
I20260709 13:20:47.365588 19990 raft_consensus.cc:2749] T b87147731a68400e8cce9b879daa7c5d P cb82cc7bd6db46d980c75237a1402860 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:20:47.415153 16294 raft_consensus.cc:493] T 8158bbd17b994d2eb96f560bd0f1b88d P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:20:47.356967 20422 consensus_queue.cc:1048] T fffab26427e24d398e2695aad40ca134 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.045s
I20260709 13:20:47.338344 20280 consensus_queue.cc:260] T 74ca8140f35c4ae29e8a664f6bdfc829 P cb82cc7bd6db46d980c75237a1402860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:47.455611 20280 raft_consensus.cc:3060] T 74ca8140f35c4ae29e8a664f6bdfc829 P cb82cc7bd6db46d980c75237a1402860 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:20:47.456209 16266 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:47.385808 20271 raft_consensus.cc:2804] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:20:47.378840 20392 consensus_queue.cc:1059] T b6f07842d3aa47858df5665874c86f3a P 647a343a542d4a139fe106c19095129b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.024s
I20260709 13:20:47.418401 16294 raft_consensus.cc:3060] T 8158bbd17b994d2eb96f560bd0f1b88d P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:20:47.365693 20432 raft_consensus.cc:2804] T 829b4ad8c5a54eababeea1181b960d4b P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:20:47.395522 20323 raft_consensus.cc:2804] T 7c2095a5f4ad49408dbc7f69c606120d P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:20:47.395496 20401 consensus_queue.cc:1048] T c0eab25ce4c64e018a041beeda2867db P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 3, Time since last communication: 0.024s
W20260709 13:20:47.421344 16115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 182301us
I20260709 13:20:47.421811 20231 raft_consensus.cc:3055] T 7c2095a5f4ad49408dbc7f69c606120d P 647a343a542d4a139fe106c19095129b [term 7 LEADER]: Stepping down as leader of term 7
W20260709 13:20:47.421921 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 101013us
W20260709 13:20:47.422850 16117 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 115835us
I20260709 13:20:47.484195 16116 leader_election.cc:304] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 90f3da57796b488c9087a2f523adf045
I20260709 13:20:47.421912 20393 raft_consensus.cc:2804] T 434bff3082d141838068f323bdb0d1ed P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:20:47.424672 20402 leader_election.cc:290] T 31969392ccdb43b491911a319e8bc7e1 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:47.491238 16116 leader_election.cc:304] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 90f3da57796b488c9087a2f523adf045
I20260709 13:20:47.234233 16146 rpcz_store.cc:276] 0709 13:20:44.713071 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:45.659682 (+946611us) service_pool.cc:224] Handling call
0709 13:20:45.665758 (+  6076us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:45.665950 (+   192us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:20:46.441891 (+775941us) log.cc:844] Serialized 14 byte log entry
0709 13:20:47.170029 (+728138us) log.cc:844] Serialized 14 byte log entry
0709 13:20:47.170211 (+   182us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:20:47.170254 (+    43us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:20:47.170295 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:47.170362 (+    67us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:47.170470 (+   108us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":1180477,"prepare.run_cpu_time_us":92,"prepare.run_wall_time_us":9873,"raft.queue_time_us":48742,"raft.run_cpu_time_us":327,"raft.run_wall_time_us":74752,"thread_start_us":1434417,"threads_started":2,"wal-append.queue_time_us":642549}
I20260709 13:20:47.427029 20434 consensus_queue.cc:1059] T 7c2095a5f4ad49408dbc7f69c606120d P 647a343a542d4a139fe106c19095129b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
W20260709 13:20:47.428624 20411 log.cc:927] Time spent T 00000000000000000000000000000000 P 667a34b27a39474ebefc1af3cf5b4271: Append to log took a long time: real 0.083s	user 0.000s	sys 0.001s
I20260709 13:20:47.430410 20294 leader_election.cc:290] T 19baa7bc367844c288bca8f81439c413 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 12 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:47.434576 20021 leader_election.cc:290] T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
W20260709 13:20:47.436343 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 51611.3us
I20260709 13:20:47.498157 16818 raft_consensus.cc:1240] T b87147731a68400e8cce9b879daa7c5d P cb82cc7bd6db46d980c75237a1402860 [term 12 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 11. Current term is 12. Ops: []
W20260709 13:20:47.437032 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 86542.3us
I20260709 13:20:47.443241 20270 leader_election.cc:290] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:47.443907 20268 leader_election.cc:290] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
W20260709 13:20:47.445264 20065 consensus_meta.cc:287] T 8638ee363273426db1393dd81e5c7b72 P 90f3da57796b488c9087a2f523adf045: Time spent flushing consensus metadata: real 0.754s	user 0.002s	sys 0.008s
I20260709 13:20:47.355027 20315 raft_consensus.cc:515] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:47.444909 20398 raft_consensus.cc:515] T d2ab0ac8347641e6955e9bce7d90c1a2 P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:20:47.445576 20418 log.cc:927] Time spent T b87147731a68400e8cce9b879daa7c5d P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.120s	user 0.001s	sys 0.000s
I20260709 13:20:47.427065 20236 raft_consensus.cc:2749] T 57a744f5aaf944678a63469cbf514015 P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:20:47.517344 20315 leader_election.cc:290] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:47.493240 20323 raft_consensus.cc:697] T 7c2095a5f4ad49408dbc7f69c606120d P 1ca2725d290e47d68b1a4b609c9816ac [term 8 LEADER]: Becoming Leader. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:20:47.518733 20323 consensus_queue.cc:237] T 7c2095a5f4ad49408dbc7f69c606120d P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:47.488580 20393 raft_consensus.cc:493] T 434bff3082d141838068f323bdb0d1ed P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Starting leader election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:20:47.520222 20393 raft_consensus.cc:3060] T 434bff3082d141838068f323bdb0d1ed P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:20:47.447110 16293 raft_consensus.cc:1240] T 26354361e9b044cea1c2b86c4882fe92 P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 8. Current term is 9. Ops: []
I20260709 13:20:47.341087 20293 consensus_queue.cc:237] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:47.451460 16300 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48684 (request call id 2465) took 2786 ms. Trace:
I20260709 13:20:47.522382 16300 rpcz_store.cc:276] 0709 13:20:44.664696 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:45.706156 (+1041460us) service_pool.cc:224] Handling call
0709 13:20:45.747319 (+ 41163us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:20:46.543878 (+796559us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:46.543908 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:20:46.561625 (+ 17717us) log.cc:844] Serialized 26 byte log entry
0709 13:20:47.268320 (+706695us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:20:47.268359 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:47.268434 (+    75us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:20:47.451229 (+182795us) raft_consensus.cc:1685] finished
0709 13:20:47.451248 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:47.451399 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":95864,"raft.run_cpu_time_us":104,"raft.run_wall_time_us":104,"thread_start_us":1401858,"threads_started":2,"wal-append.queue_time_us":700324}
W20260709 13:20:47.451006 20389 consensus_meta.cc:287] T 66ae176961b74813a58bc15c9c4c6f2a P 90f3da57796b488c9087a2f523adf045: Time spent flushing consensus metadata: real 0.568s	user 0.008s	sys 0.000s
W20260709 13:20:47.416883 16265 leader_election.cc:336] T 5a96373bb0e14dce8a266b2fc6c58507 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Timed out: RequestConsensusVote RPC to 127.14.177.129:36741 timed out after 1.871s (SENT)
I20260709 13:20:47.478169 20443 raft_consensus.cc:493] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:20:47.532254 16818 raft_consensus.cc:1240] T 26354361e9b044cea1c2b86c4882fe92 P cb82cc7bd6db46d980c75237a1402860 [term 9 LEADER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 8. Current term is 9. Ops: []
I20260709 13:20:47.532025 20443 raft_consensus.cc:515] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:47.479413 20401 consensus_queue.cc:1048] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
W20260709 13:20:47.479918 20390 consensus_meta.cc:287] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac: Time spent flushing consensus metadata: real 0.939s	user 0.013s	sys 0.000s
I20260709 13:20:47.483099 16144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee9ed67006144797b1de6ea8c2013a68" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:47.538542 16299 raft_consensus.cc:1240] T d2ab0ac8347641e6955e9bce7d90c1a2 P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 10. Current term is 11. Ops: []
I20260709 13:20:47.538707 16144 raft_consensus.cc:2468] T ee9ed67006144797b1de6ea8c2013a68 P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 8.
I20260709 13:20:47.513162 20021 raft_consensus.cc:2804] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:20:47.487723 16266 leader_election.cc:304] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:20:47.426313 20269 leader_election.cc:290] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:47.418083 20267 raft_consensus.cc:2804] T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:20:47.491966 16291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4dc6b0541ca431ebc757beac00521f1" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:47.480271 19982 raft_consensus.cc:3055] T b6f07842d3aa47858df5665874c86f3a P 647a343a542d4a139fe106c19095129b [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:20:47.498157 16222 raft_consensus.cc:2468] T 18ad7e3af5504571a69d10c00750a01c P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 10.
W20260709 13:20:47.546741 16266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 59095.1us
I20260709 13:20:47.439204 20338 consensus_queue.cc:1048] T 84f2da98864b4425bf7c0055951fdd4b P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:20:47.504809 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 12 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:20:47.499667 16191 leader_election.cc:304] T c0eab25ce4c64e018a041beeda2867db P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 90f3da57796b488c9087a2f523adf045
I20260709 13:20:47.505043 16145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e9b2bf7011f4ae688a79d7bb5a447b4" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:47.547705 16266 leader_election.cc:304] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac
I20260709 13:20:47.504832 16143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66f75124215f4ca6bef831e174a792d8" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:47.416828 20303 raft_consensus.cc:2804] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:20:47.509801 16824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a9de347c711477e87c582d003f36925" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:20:47.555186 16143 raft_consensus.cc:2468] T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 10.
I20260709 13:20:47.498601 16190 leader_election.cc:304] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:20:47.474846 20432 raft_consensus.cc:493] T 829b4ad8c5a54eababeea1181b960d4b P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:47.480657 20271 raft_consensus.cc:697] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 [term 11 LEADER]: Becoming Leader. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:20:47.514559 16816 raft_consensus.cc:1240] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 [term 12 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 9. Current term is 12. Ops: []
I20260709 13:20:47.483399 20231 raft_consensus.cc:740] T 7c2095a5f4ad49408dbc7f69c606120d P 647a343a542d4a139fe106c19095129b [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:20:47.526621 16294 raft_consensus.cc:515] T 8158bbd17b994d2eb96f560bd0f1b88d P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:20:47.560678 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 62169.5us
W20260709 13:20:47.561066 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 61509.1us
I20260709 13:20:47.528705 16265 leader_election.cc:304] T 5a96373bb0e14dce8a266b2fc6c58507 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:20:47.530004 16817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2372981006d4b218215c28964dea623" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:47.562325 16817 raft_consensus.cc:2393] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 12: Already voted for candidate cb82cc7bd6db46d980c75237a1402860 in this term.
I20260709 13:20:47.499893 20392 raft_consensus.cc:2804] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:20:47.556133 20303 raft_consensus.cc:493] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:47.537305 16823 raft_consensus.cc:1240] T af57996a8be84d398f0f023e8d08ef77 P cb82cc7bd6db46d980c75237a1402860 [term 12 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 11. Current term is 12. Ops: [9.8-11.9]
I20260709 13:20:47.563731 16822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66f75124215f4ca6bef831e174a792d8" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:47.564589 16822 raft_consensus.cc:2393] T 66f75124215f4ca6bef831e174a792d8 P cb82cc7bd6db46d980c75237a1402860 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 11: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
I20260709 13:20:47.564183 20271 consensus_queue.cc:237] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 9.10, Last appended by leader: 10, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:47.546440 19982 raft_consensus.cc:740] T b6f07842d3aa47858df5665874c86f3a P 647a343a542d4a139fe106c19095129b [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:20:47.566241 19982 consensus_queue.cc:260] T b6f07842d3aa47858df5665874c86f3a P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:47.567454 19982 raft_consensus.cc:3060] T b6f07842d3aa47858df5665874c86f3a P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:20:47.556216 20267 raft_consensus.cc:697] T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045 [term 11 LEADER]: Becoming Leader. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
W20260709 13:20:47.537956 16791 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:47.568673 20267 consensus_queue.cc:237] T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:47.568837 20388 raft_consensus.cc:515] T e5ad4eb281564936a59f41e387a47156 P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:47.557265 20338 consensus_queue.cc:1243] T 84f2da98864b4425bf7c0055951fdd4b P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Peer 647a343a542d4a139fe106c19095129b log is divergent from this leader: its last log entry 6.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260709 13:20:47.541430 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a9de347c711477e87c582d003f36925" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:20:47.542457 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18ad7e3af5504571a69d10c00750a01c" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:20:47.545567 20390 raft_consensus.cc:2749] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Leader election lost for term 10. Reason: Vote denied by peer cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923) with higher term. Message: Invalid argument: T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac for earlier term 10. Current term is 11.
I20260709 13:20:47.527729 20434 raft_consensus.cc:493] T 68a6838d82ef4cac83507f906139d90f P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:20:47.555019 16145 raft_consensus.cc:2468] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 8.
I20260709 13:20:47.504247 16820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a9de347c711477e87c582d003f36925" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:47.555204 16818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b7d1ab519044eda2d5e04a07242f25" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:47.554450 16149 raft_consensus.cc:3060] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:20:47.583200 16818 raft_consensus.cc:2393] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 9: Already voted for candidate cb82cc7bd6db46d980c75237a1402860 in this term.
I20260709 13:20:47.583642 16294 leader_election.cc:290] T 8158bbd17b994d2eb96f560bd0f1b88d P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 10 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:20:47.552460 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e9b2bf7011f4ae688a79d7bb5a447b4" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:47.534339 20443 leader_election.cc:290] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:47.558472 16291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a959dbecc2648eb85d1cc550f5a3245" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:47.556303 16825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434bff3082d141838068f323bdb0d1ed" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:47.560189 16819 raft_consensus.cc:1240] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860 [term 10 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 9. Current term is 10. Ops: []
I20260709 13:20:47.586264 16291 raft_consensus.cc:2393] T 5a959dbecc2648eb85d1cc550f5a3245 P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 9: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
I20260709 13:20:47.587616 16291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cc959fb7f704fba9009c0595467d074" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:47.588716 16291 raft_consensus.cc:2410] T 8cc959fb7f704fba9009c0595467d074 P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b for term 12 because replica has last-logged OpId of term: 11 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 7.
I20260709 13:20:47.559618 20432 raft_consensus.cc:3060] T 829b4ad8c5a54eababeea1181b960d4b P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:20:47.563217 20392 raft_consensus.cc:493] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:47.534739 16295 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48674 (request call id 2605) took 1220 ms. Trace:
I20260709 13:20:47.596040 16191 leader_election.cc:304] T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
W20260709 13:20:47.569226 16791 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:47.604563 16295 rpcz_store.cc:276] 0709 13:20:46.314877 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:46.315004 (+   127us) service_pool.cc:224] Handling call
0709 13:20:46.315333 (+   329us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:20:46.315551 (+   218us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:46.315577 (+    26us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:20:46.342053 (+ 26476us) log.cc:844] Serialized 26 byte log entry
0709 13:20:47.105995 (+763942us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:20:47.106037 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:47.106102 (+    65us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:20:47.534518 (+428416us) raft_consensus.cc:1685] finished
0709 13:20:47.534542 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:47.534681 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":698134,"threads_started":1,"wal-append.queue_time_us":928353}
I20260709 13:20:47.534965 20389 raft_consensus.cc:515] T 66ae176961b74813a58bc15c9c4c6f2a P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:47.571412 16293 raft_consensus.cc:2410] T 18ad7e3af5504571a69d10c00750a01c P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b for term 10 because replica has last-logged OpId of term: 9 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 6.
I20260709 13:20:47.514365 20065 raft_consensus.cc:2749] T 8638ee363273426db1393dd81e5c7b72 P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Leader pre-election lost for term 11. Reason: Vote denied by peer cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923) with higher term. Message: Invalid argument: T 8638ee363273426db1393dd81e5c7b72 P cb82cc7bd6db46d980c75237a1402860 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 for earlier term 11. Current term is 12.
I20260709 13:20:47.573873 16817 raft_consensus.cc:1240] T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860 [term 12 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 11. Current term is 12. Ops: []
I20260709 13:20:47.542974 20021 raft_consensus.cc:493] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:20:47.577606 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 160818us
I20260709 13:20:47.544456 20422 consensus_queue.cc:1059] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.061s
I20260709 13:20:47.552374 20402 raft_consensus.cc:2764] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Leader pre-election decision vote started in defunct term 12: won
I20260709 13:20:47.600378 16291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66ae176961b74813a58bc15c9c4c6f2a" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:47.602623 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 12 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:20:47.485380 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2372981006d4b218215c28964dea623" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:20:47.641511 16820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9944d5e4b8c4109af9955c4f08c9095" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:47.641921 16150 raft_consensus.cc:3060] T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:20:47.642405 16820 raft_consensus.cc:2393] T c9944d5e4b8c4109af9955c4f08c9095 P cb82cc7bd6db46d980c75237a1402860 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 11: Already voted for candidate cb82cc7bd6db46d980c75237a1402860 in this term.
I20260709 13:20:47.606577 16294 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RunLeaderElection from 127.0.0.1:48684 (request call id 2475) took 1012 ms. Trace:
I20260709 13:20:47.606640 16818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a378d7f796f94615b3b347119a9880fd" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:47.608319 16824 raft_consensus.cc:2468] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 13.
I20260709 13:20:47.563779 20303 raft_consensus.cc:3060] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:20:47.643745 16189 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:47.577366 20315 raft_consensus.cc:2804] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:20:47.609730 20392 raft_consensus.cc:3060] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:20:47.604174 20401 raft_consensus.cc:2804] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:20:47.592885 20257 raft_consensus.cc:2804] T c0eab25ce4c64e018a041beeda2867db P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:20:47.673931 16300 raft_consensus.cc:2376] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 647a343a542d4a139fe106c19095129b in term 13. Re-sending same reply.
W20260709 13:20:47.681310 16266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 133719us
I20260709 13:20:47.643384 16294 rpcz_store.cc:276] 0709 13:20:46.612224 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:47.362269 (+750045us) service_pool.cc:224] Handling call
0709 13:20:47.584260 (+221991us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:20:47.634917 16265 leader_election.cc:336] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Timed out: RequestConsensusVote RPC to 127.14.177.129:36741 timed out after 2.912s (SENT)
W20260709 13:20:47.659652 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 63715.7us
I20260709 13:20:47.684401 16191 leader_election.cc:304] T e5ad4eb281564936a59f41e387a47156 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 90f3da57796b488c9087a2f523adf045
I20260709 13:20:47.607288 20444 raft_consensus.cc:2749] T 5a96373bb0e14dce8a266b2fc6c58507 P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:20:47.696935 16295 raft_consensus.cc:1275] T ecc1e363230043fe9fef390f0b979f20 P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
W20260709 13:20:47.697273 16791 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:47.662864 16291 raft_consensus.cc:2468] T 66ae176961b74813a58bc15c9c4c6f2a P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 13.
I20260709 13:20:47.667909 16825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e4bfb9090c8460eb14fa7e0d818e7f1" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:47.671021 16297 raft_consensus.cc:2468] T 8638ee363273426db1393dd81e5c7b72 P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 12.
I20260709 13:20:47.539916 20306 consensus_queue.cc:237] T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:20:47.707146 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 72265.1us
W20260709 13:20:47.708544 16265 leader_election.cc:336] T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Timed out: RequestConsensusVote RPC to 127.14.177.129:36741 timed out after 3.070s (SENT)
I20260709 13:20:47.641333 20388 leader_election.cc:290] T e5ad4eb281564936a59f41e387a47156 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 12 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:47.679670 16150 raft_consensus.cc:2468] T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 12.
I20260709 13:20:47.640784 20021 raft_consensus.cc:3060] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:20:47.709040 16265 leader_election.cc:304] T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:20:47.567690 20231 consensus_queue.cc:260] T 7c2095a5f4ad49408dbc7f69c606120d P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:47.668849 20315 raft_consensus.cc:493] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:47.604262 20424 raft_consensus.cc:515] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:47.685123 16299 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40474 (request call id 693) took 1261 ms. Trace:
I20260709 13:20:47.735342 16299 rpcz_store.cc:276] 0709 13:20:46.423631 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:47.275022 (+851391us) service_pool.cc:224] Handling call
0709 13:20:47.275379 (+   357us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:47.538946 (+263567us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:47.684811 (+145865us) spinlock_profiling.cc:243] Waited 263 ms on lock 0x7b54000a9e54. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:20:47.685049 (+   238us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":604833024}
I20260709 13:20:47.604197 20434 raft_consensus.cc:515] T 68a6838d82ef4cac83507f906139d90f P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:47.671131 20401 raft_consensus.cc:493] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:47.687120 20257 raft_consensus.cc:493] T c0eab25ce4c64e018a041beeda2867db P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:20:47.711530 20442 log.cc:927] Time spent T ada5a004ba7c46819051e3ccec71ffdb P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.165s	user 0.001s	sys 0.000s
I20260709 13:20:47.709807 20416 raft_consensus.cc:2749] T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:20:47.742942 16824 raft_consensus.cc:1275] T af57996a8be84d398f0f023e8d08ef77 P cb82cc7bd6db46d980c75237a1402860 [term 12 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
W20260709 13:20:47.743587 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 59317us
I20260709 13:20:47.716341 20422 raft_consensus.cc:3055] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:20:47.747676 16818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31969392ccdb43b491911a319e8bc7e1" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:47.748363 16825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:47.736011 20315 raft_consensus.cc:3060] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:20:47.744290 20401 raft_consensus.cc:3060] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:20:47.769610 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 278427us
I20260709 13:20:47.747206 20337 raft_consensus.cc:2804] T e5ad4eb281564936a59f41e387a47156 P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:20:47.747443 20257 raft_consensus.cc:3060] T c0eab25ce4c64e018a041beeda2867db P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:20:47.773662 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 65134.2us
I20260709 13:20:47.753245 20065 raft_consensus.cc:493] T fffab26427e24d398e2695aad40ca134 P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:47.778949 20236 raft_consensus.cc:493] T 2794af5b99984efebdee564035841cb0 P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:20:47.779605 20236 raft_consensus.cc:515] T 2794af5b99984efebdee564035841cb0 P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:20:47.780128 20439 log.cc:927] Time spent T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 0.361s	user 0.004s	sys 0.000s
I20260709 13:20:47.780937 16146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02838c86531d4f98bf58d1c22b8baf77" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:47.781438 16143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57a744f5aaf944678a63469cbf514015" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:47.781997 20236 leader_election.cc:290] T 2794af5b99984efebdee564035841cb0 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:47.782110 16143 raft_consensus.cc:2468] T 57a744f5aaf944678a63469cbf514015 P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 9.
I20260709 13:20:47.782619 20236 raft_consensus.cc:493] T 8a878999c0024d2abe517484180290fc P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:20:47.783716 16143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a878999c0024d2abe517484180290fc" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:47.784355 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68a6838d82ef4cac83507f906139d90f" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:47.785194 16146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6f07842d3aa47858df5665874c86f3a" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:47.783123 20236 raft_consensus.cc:515] T 8a878999c0024d2abe517484180290fc P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:47.788163 16143 raft_consensus.cc:2410] T 8a878999c0024d2abe517484180290fc P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 for term 10 because replica has last-logged OpId of term: 9 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 4.
I20260709 13:20:47.788684 16144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26354361e9b044cea1c2b86c4882fe92" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:47.790078 16144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1df93779209748b9860f6d893dd3de53" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:47.790206 16143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:47.791544 16143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4f65cdf00b748bb963ee6e81ef43b79" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:20:47.792250 16143 raft_consensus.cc:3060] T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:20:47.798617 20236 leader_election.cc:290] T 8a878999c0024d2abe517484180290fc P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:47.785283 16150 raft_consensus.cc:2410] T 68a6838d82ef4cac83507f906139d90f P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 for term 12 because replica has last-logged OpId of term: 11 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
I20260709 13:20:47.803617 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d16dfad0c3b540e5b4b86492ddf87208" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:47.804376 16150 raft_consensus.cc:2393] T d16dfad0c3b540e5b4b86492ddf87208 P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 11: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
I20260709 13:20:47.804858 20448 consensus_queue.cc:1059] T af57996a8be84d398f0f023e8d08ef77 P 647a343a542d4a139fe106c19095129b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:20:47.750849 20267 raft_consensus.cc:493] T c9944d5e4b8c4109af9955c4f08c9095 P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:47.806252 20267 raft_consensus.cc:515] T c9944d5e4b8c4109af9955c4f08c9095 P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:47.802042 20426 consensus_queue.cc:1059] T b87147731a68400e8cce9b879daa7c5d P 647a343a542d4a139fe106c19095129b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:20:47.807468 16117 leader_election.cc:304] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:20:47.805158 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:47.810086 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18ad7e3af5504571a69d10c00750a01c" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
W20260709 13:20:47.811493 16821 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49892 (request call id 596) took 2359 ms (client timeout 1625 ms). Trace:
I20260709 13:20:47.747256 20450 raft_consensus.cc:2749] T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:20:47.812193 16821 rpcz_store.cc:269] 0709 13:20:45.452159 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:45.452319 (+   160us) service_pool.cc:224] Handling call
0709 13:20:47.811301 (+2358982us) spinlock_profiling.cc:243] Waited 1.75 s on lock 0x7b54007ffdd4. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:20:47.811409 (+   108us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4031956352}
I20260709 13:20:47.761513 20422 raft_consensus.cc:740] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:20:47.813242 16150 raft_consensus.cc:2393] T 18ad7e3af5504571a69d10c00750a01c P 647a343a542d4a139fe106c19095129b [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 10: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
W20260709 13:20:47.819391 16265 leader_election.cc:336] T c0eab25ce4c64e018a041beeda2867db P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 10 election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Timed out: RequestConsensusVote RPC to 127.14.177.129:36741 timed out after 1.889s (SENT)
I20260709 13:20:47.776994 16116 leader_election.cc:304] T 5a959dbecc2648eb85d1cc550f5a3245 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:20:47.888396 16144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f2da98864b4425bf7c0055951fdd4b" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:47.893481 16144 raft_consensus.cc:2368] T 84f2da98864b4425bf7c0055951fdd4b P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 for earlier term 8. Current term is 9.
I20260709 13:20:47.887446 16825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2794af5b99984efebdee564035841cb0" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
W20260709 13:20:47.893005 16117 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 85713.4us
I20260709 13:20:47.887786 16143 raft_consensus.cc:2468] T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 12.
I20260709 13:20:47.773002 20337 raft_consensus.cc:493] T e5ad4eb281564936a59f41e387a47156 P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Starting leader election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:20:47.886684 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6f07842d3aa47858df5665874c86f3a" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
W20260709 13:20:47.892864 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 115952us
W20260709 13:20:47.904520 16264 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:47.812608 20398 leader_election.cc:290] T d2ab0ac8347641e6955e9bce7d90c1a2 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 11 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
W20260709 13:20:47.962141 16789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 62589.2us
I20260709 13:20:47.963716 16790 leader_election.cc:304] T ee9ed67006144797b1de6ea8c2013a68 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:20:47.964731 16295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada5a004ba7c46819051e3ccec71ffdb" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:47.777426 20181 consensus_queue.cc:1048] T c2f5f89186204242bf377f7c42b2e2da P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.128s
I20260709 13:20:47.770501 20443 consensus_queue.cc:1048] T af57996a8be84d398f0f023e8d08ef77 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.121s
W20260709 13:20:47.965025 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 145666us
I20260709 13:20:47.968961 16116 leader_election.cc:304] T 08c74490c40243a88e62d1760b5cd814 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:20:47.968611 16143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a96373bb0e14dce8a266b2fc6c58507" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 10 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:48.020992 16143 raft_consensus.cc:2410] T 5a96373bb0e14dce8a266b2fc6c58507 P 647a343a542d4a139fe106c19095129b [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 for term 10 because replica has last-logged OpId of term: 10 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 4.
I20260709 13:20:48.022154 16143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0eab25ce4c64e018a041beeda2867db" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:20:48.023032 16143 raft_consensus.cc:2376] T c0eab25ce4c64e018a041beeda2867db P 647a343a542d4a139fe106c19095129b [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 10. Re-sending same reply.
W20260709 13:20:48.024752 16265 leader_election.cc:336] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 8 election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Timed out: RequestConsensusVote RPC to 127.14.177.129:36741 timed out after 1.924s (SENT)
I20260709 13:20:47.978340 16294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdbd78acb4cf46298e0de8ba0d130e19" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:48.025763 16294 raft_consensus.cc:2468] T cdbd78acb4cf46298e0de8ba0d130e19 P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 11.
I20260709 13:20:47.966215 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1df93779209748b9860f6d893dd3de53" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:20:47.831246 20065 raft_consensus.cc:515] T fffab26427e24d398e2695aad40ca134 P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:47.968235 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74ca8140f35c4ae29e8a664f6bdfc829" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:48.035873 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5ad4eb281564936a59f41e387a47156" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:20:47.836778 20388 consensus_queue.cc:1059] T 746aabf856ab401c9ef5e859371e7b67 P 647a343a542d4a139fe106c19095129b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.149s
W20260709 13:20:47.979133 20449 log.cc:927] Time spent T 8cc959fb7f704fba9009c0595467d074 P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 0.093s	user 0.000s	sys 0.001s
I20260709 13:20:48.050343 16293 raft_consensus.cc:3060] T e5ad4eb281564936a59f41e387a47156 P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:20:48.062561 16294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a378d7f796f94615b3b347119a9880fd" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:48.064415 16143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ad5dbfa99ee4b8d9e28be92b3f36db5" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:48.065243 16143 raft_consensus.cc:2468] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 9.
I20260709 13:20:48.066428 16295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a878999c0024d2abe517484180290fc" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:48.067229 16295 raft_consensus.cc:2468] T 8a878999c0024d2abe517484180290fc P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 9.
I20260709 13:20:48.068581 16144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:47.983304 20294 raft_consensus.cc:493] T c9944d5e4b8c4109af9955c4f08c9095 P cb82cc7bd6db46d980c75237a1402860 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:48.075064 16143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e9b2bf7011f4ae688a79d7bb5a447b4" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:20:47.968361 20337 raft_consensus.cc:3060] T e5ad4eb281564936a59f41e387a47156 P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:20:48.075466 16190 leader_election.cc:304] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860
I20260709 13:20:48.075800 16143 raft_consensus.cc:2393] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 8: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
W20260709 13:20:48.079210 20439 log.cc:927] Time spent T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
I20260709 13:20:48.080310 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a878999c0024d2abe517484180290fc" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
W20260709 13:20:48.094918 20289 consensus_meta.cc:287] T a378d7f796f94615b3b347119a9880fd P 90f3da57796b488c9087a2f523adf045: Time spent flushing consensus metadata: real 0.799s	user 0.006s	sys 0.005s
W20260709 13:20:48.088425 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 63713.9us
W20260709 13:20:48.097899 16149 consensus_meta.cc:287] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b: Time spent flushing consensus metadata: real 0.515s	user 0.012s	sys 0.000s
I20260709 13:20:48.100842 16190 leader_election.cc:304] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860
I20260709 13:20:48.047255 20389 leader_election.cc:290] T 66ae176961b74813a58bc15c9c4c6f2a P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 13 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:48.102238 16295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e4bfb9090c8460eb14fa7e0d818e7f1" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
W20260709 13:20:48.092059 20445 log.cc:927] Time spent T 66f75124215f4ca6bef831e174a792d8 P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 0.448s	user 0.001s	sys 0.000s
I20260709 13:20:48.148801 16150 raft_consensus.cc:2410] T 8a878999c0024d2abe517484180290fc P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 for term 10 because replica has last-logged OpId of term: 9 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 4.
W20260709 13:20:48.049810 20183 consensus_meta.cc:287] T 02838c86531d4f98bf58d1c22b8baf77 P 90f3da57796b488c9087a2f523adf045: Time spent flushing consensus metadata: real 1.359s	user 0.017s	sys 0.005s
I20260709 13:20:48.182950 16818 raft_consensus.cc:1275] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860 [term 10 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:20:48.183030 20183 raft_consensus.cc:515] T 02838c86531d4f98bf58d1c22b8baf77 P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:48.185695 20183 leader_election.cc:290] T 02838c86531d4f98bf58d1c22b8baf77 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 11 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:20:48.150545 16294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2794af5b99984efebdee564035841cb0" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:48.169498 16144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5ad4eb281564936a59f41e387a47156" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:20:48.080996 20458 raft_consensus.cc:2804] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:20:48.048707 20457 consensus_queue.cc:1059] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.088s
W20260709 13:20:48.171559 16265 leader_election.cc:336] T e5ad4eb281564936a59f41e387a47156 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 11 election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Timed out: RequestConsensusVote RPC to 127.14.177.129:36741 timed out after 1.995s (SENT)
W20260709 13:20:48.163427 16148 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.611s	user 0.000s	sys 0.003s
I20260709 13:20:48.172256 16149 raft_consensus.cc:2468] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 12.
I20260709 13:20:48.188617 16144 raft_consensus.cc:2368] T e5ad4eb281564936a59f41e387a47156 P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 for earlier term 11. Current term is 12.
I20260709 13:20:48.094990 20402 raft_consensus.cc:493] T 8c11074579b046d99e585c2818e18a2c P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:48.197962 16294 raft_consensus.cc:1240] T 1df93779209748b9860f6d893dd3de53 P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 10. Current term is 11. Ops: []
I20260709 13:20:48.170734 20426 consensus_queue.cc:1243] T b87147731a68400e8cce9b879daa7c5d P 647a343a542d4a139fe106c19095129b [LEADER]: Peer cb82cc7bd6db46d980c75237a1402860 log is divergent from this leader: its last log entry 10.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 13:20:48.174855 20434 leader_election.cc:290] T 68a6838d82ef4cac83507f906139d90f P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
W20260709 13:20:48.176345 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 75617.6us
I20260709 13:20:48.176579 16291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9944d5e4b8c4109af9955c4f08c9095" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:48.257665 16291 raft_consensus.cc:2468] T c9944d5e4b8c4109af9955c4f08c9095 P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 11.
I20260709 13:20:48.257786 16190 leader_election.cc:304] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860
W20260709 13:20:48.106161 20391 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.631s	user 0.000s	sys 0.012s
I20260709 13:20:48.182015 16295 raft_consensus.cc:1240] T d16dfad0c3b540e5b4b86492ddf87208 P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 10. Current term is 11. Ops: []
I20260709 13:20:48.182304 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8158bbd17b994d2eb96f560bd0f1b88d" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: true dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:20:48.260588 16150 raft_consensus.cc:3060] T 8158bbd17b994d2eb96f560bd0f1b88d P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:20:48.098848 19673 raft_consensus.cc:2804] T ee9ed67006144797b1de6ea8c2013a68 P cb82cc7bd6db46d980c75237a1402860 [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 13:20:48.190904 16148 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.639s	user 0.000s	sys 0.004s
I20260709 13:20:48.188172 16296 raft_consensus.cc:1240] T 02838c86531d4f98bf58d1c22b8baf77 P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 10. Current term is 11. Ops: []
I20260709 13:20:48.190771 16265 leader_election.cc:304] T e5ad4eb281564936a59f41e387a47156 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:20:48.086794 20294 raft_consensus.cc:515] T c9944d5e4b8c4109af9955c4f08c9095 P cb82cc7bd6db46d980c75237a1402860 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:48.049034 20422 consensus_queue.cc:260] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:48.197719 20402 raft_consensus.cc:515] T 8c11074579b046d99e585c2818e18a2c P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:48.202989 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57a744f5aaf944678a63469cbf514015" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:48.270550 16149 raft_consensus.cc:2468] T 57a744f5aaf944678a63469cbf514015 P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 9.
I20260709 13:20:48.203035 16144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02838c86531d4f98bf58d1c22b8baf77" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
W20260709 13:20:48.190289 20454 log.cc:927] Time spent T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.298s	user 0.000s	sys 0.001s
W20260709 13:20:48.222064 16791 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:48.273124 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8158bbd17b994d2eb96f560bd0f1b88d" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: true dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
W20260709 13:20:48.273464 16791 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:48.273836 16226 raft_consensus.cc:3055] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:20:48.179965 20386 raft_consensus.cc:2749] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:20:48.221972 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 253176us
W20260709 13:20:48.199054 20452 log.cc:927] Time spent T 829b4ad8c5a54eababeea1181b960d4b P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 0.458s	user 0.001s	sys 0.000s
W20260709 13:20:48.227336 16115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 261664us
I20260709 13:20:48.187335 20467 raft_consensus.cc:2749] T 5a959dbecc2648eb85d1cc550f5a3245 P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:20:48.236609 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 272968us
I20260709 13:20:48.288153 16790 leader_election.cc:304] T 66f75124215f4ca6bef831e174a792d8 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:20:48.289676 16790 leader_election.cc:304] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860; no voters: 90f3da57796b488c9087a2f523adf045
I20260709 13:20:48.173403 20289 raft_consensus.cc:515] T a378d7f796f94615b3b347119a9880fd P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:48.290995 16790 leader_election.cc:304] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860; no voters: 
W20260709 13:20:48.243690 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 483668us
I20260709 13:20:48.154038 20269 raft_consensus.cc:2749] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:20:48.269619 16217 raft_consensus.cc:1240] T 08c74490c40243a88e62d1760b5cd814 P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 9. Current term is 10. Ops: []
W20260709 13:20:48.244468 20293 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.501s	user 0.002s	sys 0.003s
W20260709 13:20:48.419320 20293 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.683s	user 0.003s	sys 0.003s
I20260709 13:20:48.274235 16226 raft_consensus.cc:740] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
W20260709 13:20:48.243264 20411 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.673s	user 0.000s	sys 0.006s
I20260709 13:20:48.282546 16223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ab0ac8347641e6955e9bce7d90c1a2" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
W20260709 13:20:48.267827 20391 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.794s	user 0.001s	sys 0.012s
I20260709 13:20:48.283538 16219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
W20260709 13:20:48.281497 20271 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.691s	user 0.002s	sys 0.013s
W20260709 13:20:48.286834 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 115324us
I20260709 13:20:48.286981 16115 leader_election.cc:304] T 18ad7e3af5504571a69d10c00750a01c P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
W20260709 13:20:48.410106 20455 log.cc:927] Time spent T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 0.431s	user 0.000s	sys 0.001s
W20260709 13:20:48.281489 20456 log.cc:927] Time spent T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 0.453s	user 0.002s	sys 0.000s
I20260709 13:20:48.278421 20388 raft_consensus.cc:2749] T 08c74490c40243a88e62d1760b5cd814 P 647a343a542d4a139fe106c19095129b [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:20:48.270045 20443 raft_consensus.cc:2749] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:20:48.272104 19673 raft_consensus.cc:493] T ee9ed67006144797b1de6ea8c2013a68 P cb82cc7bd6db46d980c75237a1402860 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
W20260709 13:20:48.286432 20323 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.765s	user 0.003s	sys 0.019s
W20260709 13:20:48.430274 16192 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 148068us
W20260709 13:20:48.424074 16264 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:48.474081 16150 raft_consensus.cc:2468] T 8158bbd17b994d2eb96f560bd0f1b88d P 647a343a542d4a139fe106c19095129b [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 10.
I20260709 13:20:48.475332 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87ac0d1f7e8a419f9bcbc04aee0e067e" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:20:48.435460 16223 raft_consensus.cc:3060] T d2ab0ac8347641e6955e9bce7d90c1a2 P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:20:48.435730 16292 raft_consensus.cc:1240] T a378d7f796f94615b3b347119a9880fd P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 11. Current term is 12. Ops: []
I20260709 13:20:48.284552 20457 raft_consensus.cc:3055] T 746aabf856ab401c9ef5e859371e7b67 P 647a343a542d4a139fe106c19095129b [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:20:48.420460 16226 consensus_queue.cc:260] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:20:48.441856 16115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 154945us
I20260709 13:20:48.477502 16226 raft_consensus.cc:3060] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:20:48.478713 16115 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:20:48.396600 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 108995us
I20260709 13:20:48.479647 16116 leader_election.cc:304] T fffab26427e24d398e2695aad40ca134 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:20:48.480729 16116 leader_election.cc:304] T d16dfad0c3b540e5b4b86492ddf87208 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
W20260709 13:20:48.480983 20453 log.cc:927] Time spent T ecc1e363230043fe9fef390f0b979f20 P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 0.422s	user 0.000s	sys 0.001s
W20260709 13:20:48.444618 16297 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40474 (request call id 698) took 1895 ms (client timeout 1701 ms). Trace:
W20260709 13:20:48.487679 16297 rpcz_store.cc:269] 0709 13:20:46.549002 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:46.915759 (+366757us) service_pool.cc:224] Handling call
0709 13:20:48.444168 (+1528409us) spinlock_profiling.cc:243] Waited 728 ms on lock 0x7b540010ed54. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:20:48.444453 (+   285us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1673992064}
I20260709 13:20:48.449528 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31969392ccdb43b491911a319e8bc7e1" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:48.401294 20426 raft_consensus.cc:3055] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:20:48.500643 20426 raft_consensus.cc:740] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:20:48.501448 20426 consensus_queue.cc:260] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.11, Last appended by leader: 11, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:48.481714 20457 raft_consensus.cc:740] T 746aabf856ab401c9ef5e859371e7b67 P 647a343a542d4a139fe106c19095129b [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:20:48.502642 20426 raft_consensus.cc:3060] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:20:48.400595 19990 raft_consensus.cc:2764] T 66f75124215f4ca6bef831e174a792d8 P cb82cc7bd6db46d980c75237a1402860 [term 11 FOLLOWER]: Leader pre-election decision vote started in defunct term 10: won
W20260709 13:20:48.503221 20468 log.cc:927] Time spent T d16dfad0c3b540e5b4b86492ddf87208 P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 0.227s	user 0.000s	sys 0.001s
I20260709 13:20:48.503718 19990 raft_consensus.cc:2804] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:20:48.504194 19990 raft_consensus.cc:697] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 [term 12 LEADER]: Becoming Leader. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:20:48.505079 19990 consensus_queue.cc:237] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:20:48.506654 20393 consensus_meta.cc:287] T 434bff3082d141838068f323bdb0d1ed P 647a343a542d4a139fe106c19095129b: Time spent flushing consensus metadata: real 0.986s	user 0.004s	sys 0.006s
I20260709 13:20:48.510386 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdbd78acb4cf46298e0de8ba0d130e19" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:20:48.409479 20422 raft_consensus.cc:3060] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:20:48.435289 20411 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.865s	user 0.000s	sys 0.007s
W20260709 13:20:48.465754 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 71213.5us
W20260709 13:20:48.451128 20271 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.850s	user 0.002s	sys 0.013s
I20260709 13:20:48.472141 16293 raft_consensus.cc:2468] T e5ad4eb281564936a59f41e387a47156 P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 12.
I20260709 13:20:48.443701 16219 raft_consensus.cc:2393] T ecc1e363230043fe9fef390f0b979f20 P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 10: Already voted for candidate cb82cc7bd6db46d980c75237a1402860 in this term.
I20260709 13:20:48.481727 16116 leader_election.cc:304] T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 90f3da57796b488c9087a2f523adf045
W20260709 13:20:48.527019 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 45361.1us
I20260709 13:20:48.527918 16116 leader_election.cc:304] T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860
I20260709 13:20:48.461020 20183 raft_consensus.cc:1064] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045: attempting to promote NON_VOTER cb82cc7bd6db46d980c75237a1402860 to VOTER
I20260709 13:20:48.409284 20434 consensus_queue.cc:1059] T 746aabf856ab401c9ef5e859371e7b67 P 647a343a542d4a139fe106c19095129b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.058s
I20260709 13:20:48.450768 20467 raft_consensus.cc:2749] T 18ad7e3af5504571a69d10c00750a01c P 647a343a542d4a139fe106c19095129b [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:20:48.498379 20477 raft_consensus.cc:2749] T fffab26427e24d398e2695aad40ca134 P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:20:48.500151 20274 raft_consensus.cc:493] T ada5a004ba7c46819051e3ccec71ffdb P cb82cc7bd6db46d980c75237a1402860 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
W20260709 13:20:48.531942 20465 log.cc:927] Time spent T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 0.360s	user 0.001s	sys 0.000s
I20260709 13:20:48.511361 16299 raft_consensus.cc:1275] T 5a959dbecc2648eb85d1cc550f5a3245 P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:20:48.512130 20388 raft_consensus.cc:2749] T d16dfad0c3b540e5b4b86492ddf87208 P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:20:48.526464 20450 raft_consensus.cc:2749] T e5ad4eb281564936a59f41e387a47156 P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
W20260709 13:20:48.473205 20323 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.951s	user 0.004s	sys 0.020s
W20260709 13:20:48.533735 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 242816us
I20260709 13:20:48.474756 19673 raft_consensus.cc:3060] T ee9ed67006144797b1de6ea8c2013a68 P cb82cc7bd6db46d980c75237a1402860 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:20:48.511075 20432 consensus_meta.cc:287] T 829b4ad8c5a54eababeea1181b960d4b P 1ca2725d290e47d68b1a4b609c9816ac: Time spent flushing consensus metadata: real 0.883s	user 0.008s	sys 0.000s
I20260709 13:20:48.483018 20471 raft_consensus.cc:2804] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:20:48.559952 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6f07842d3aa47858df5665874c86f3a" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:48.454752 20386 consensus_queue.cc:1048] T 746aabf856ab401c9ef5e859371e7b67 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.077s
I20260709 13:20:48.507073 20393 raft_consensus.cc:515] T 434bff3082d141838068f323bdb0d1ed P 647a343a542d4a139fe106c19095129b [term 10 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:48.468530 20310 consensus_queue.cc:1048] T fffab26427e24d398e2695aad40ca134 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 3, Time since last communication: 0.081s
I20260709 13:20:48.516862 16297 raft_consensus.cc:1240] T 8158bbd17b994d2eb96f560bd0f1b88d P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 9. Current term is 10. Ops: []
I20260709 13:20:48.532374 16190 leader_election.cc:304] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860
W20260709 13:20:48.590044 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 64827.1us
I20260709 13:20:48.687734 16147 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46224 (request call id 634) took 1152 ms. Trace:
I20260709 13:20:48.593566 16265 leader_election.cc:304] T d16dfad0c3b540e5b4b86492ddf87208 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
W20260709 13:20:48.556972 20183 raft_consensus.cc:1068] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045: Unable to promote non-voter cb82cc7bd6db46d980c75237a1402860: Illegal state: Leader has not yet committed an operation in its own term
W20260709 13:20:48.589411 20021 consensus_meta.cc:287] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 647a343a542d4a139fe106c19095129b: Time spent flushing consensus metadata: real 0.854s	user 0.008s	sys 0.003s
I20260709 13:20:48.585131 20434 consensus_queue.cc:1243] T 746aabf856ab401c9ef5e859371e7b67 P 647a343a542d4a139fe106c19095129b [LEADER]: Peer 1ca2725d290e47d68b1a4b609c9816ac log is divergent from this leader: its last log entry 10.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260709 13:20:48.510980 16300 raft_consensus.cc:3060] T cdbd78acb4cf46298e0de8ba0d130e19 P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:20:48.652724 16192 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:48.682075 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
W20260709 13:20:48.685578 16789 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.705s	user 0.003s	sys 0.016s
W20260709 13:20:48.707821 16789 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.727s	user 0.003s	sys 0.016s
W20260709 13:20:48.708166 16789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 730495us
I20260709 13:20:48.708410 16149 raft_consensus.cc:2393] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 12: Already voted for candidate cb82cc7bd6db46d980c75237a1402860 in this term.
I20260709 13:20:48.709537 16789 leader_election.cc:304] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:20:48.709883 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f2da98864b4425bf7c0055951fdd4b" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:48.711083 16149 raft_consensus.cc:2468] T 84f2da98864b4425bf7c0055951fdd4b P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 9.
I20260709 13:20:48.712522 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87ac0d1f7e8a419f9bcbc04aee0e067e" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:48.714217 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a9de347c711477e87c582d003f36925" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:48.688262 16147 rpcz_store.cc:276] 0709 13:20:47.535215 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:47.535337 (+   122us) service_pool.cc:224] Handling call
0709 13:20:47.535681 (+   344us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:20:47.535841 (+   160us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:47.535856 (+    15us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:20:47.544676 (+  8820us) log.cc:844] Serialized 26 byte log entry
0709 13:20:47.733839 (+189163us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:20:47.733876 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:47.733945 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:20:48.687450 (+953505us) raft_consensus.cc:1685] finished
0709 13:20:48.687479 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:48.687659 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":105062,"threads_started":1,"wal-append.queue_time_us":301895}
I20260709 13:20:48.716192 16147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c61a39fd4f84424b129c1a6f9ba404d" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:48.717631 16147 raft_consensus.cc:2368] T 9c61a39fd4f84424b129c1a6f9ba404d P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 for earlier term 10. Current term is 11.
W20260709 13:20:48.686942 20478 log.cc:927] Time spent T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.446s	user 0.001s	sys 0.000s
I20260709 13:20:48.592370 20472 raft_consensus.cc:2764] T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Leader pre-election decision vote started in defunct term 11: won
I20260709 13:20:48.676787 20271 raft_consensus.cc:493] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:20:48.694639 20464 log.cc:927] Time spent T 9e65676b21b748728ad39668cfe722ec P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 0.470s	user 0.001s	sys 0.000s
I20260709 13:20:48.722654 16142 raft_consensus.cc:1240] T 434bff3082d141838068f323bdb0d1ed P 647a343a542d4a139fe106c19095129b [term 10 FOLLOWER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 9. Current term is 10. Ops: []
I20260709 13:20:48.742225 16224 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57600 (request call id 1422) took 1442 ms. Trace:
I20260709 13:20:48.588641 20447 consensus_queue.cc:1048] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.130s
I20260709 13:20:48.674834 20487 raft_consensus.cc:2749] T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:20:48.680581 20471 raft_consensus.cc:493] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:20:48.746438 16224 rpcz_store.cc:276] 0709 13:20:47.300199 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:47.321014 (+ 20815us) service_pool.cc:224] Handling call
0709 13:20:47.321275 (+   261us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:20:47.367012 (+ 45737us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:47.367073 (+    61us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:20:47.367803 (+   730us) log.cc:844] Serialized 52 byte log entry
0709 13:20:47.527451 (+159648us) spinlock_profiling.cc:243] Waited 11.1 ms on lock 0x7b18004e7d80. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b693ade1 00007f69b69475b1 00007f69ada54e07 00007f69b68afab9 00007f69ada7169b 00007f69ada6f93f 00007f69ada71c02 00007f69b06f920b 00007f69b06fed23 00007f69b06ff0d1 00007f69b0b1aa0f
0709 13:20:47.709000 (+181549us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:20:47.709055 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:47.709162 (+   107us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:20:48.704229 (+995067us) raft_consensus.cc:1685] finished
0709 13:20:48.704254 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:48.742136 (+ 37882us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":87520,"raft.queue_time_us":43979,"raft.run_cpu_time_us":107,"raft.run_wall_time_us":106,"spinlock_wait_cycles":25528704,"thread_start_us":181339,"threads_started":1,"wal-append.queue_time_us":382465}
I20260709 13:20:48.615625 20398 consensus_queue.cc:1048] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.124s
I20260709 13:20:48.705497 20460 raft_consensus.cc:2749] T d16dfad0c3b540e5b4b86492ddf87208 P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:20:48.725549 20392 consensus_meta.cc:287] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b: Time spent flushing consensus metadata: real 1.039s	user 0.011s	sys 0.000s
I20260709 13:20:48.675544 20484 consensus_queue.cc:1059] T d16dfad0c3b540e5b4b86492ddf87208 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.152s
I20260709 13:20:48.595810 20451 consensus_queue.cc:1048] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.155s
W20260709 13:20:48.748946 20280 consensus_meta.cc:287] T 74ca8140f35c4ae29e8a664f6bdfc829 P cb82cc7bd6db46d980c75237a1402860: Time spent flushing consensus metadata: real 1.293s	user 0.011s	sys 0.002s
W20260709 13:20:48.768887 16192 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:48.676002 20432 raft_consensus.cc:515] T 829b4ad8c5a54eababeea1181b960d4b P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:48.704775 20021 raft_consensus.cc:515] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:48.746863 20402 leader_election.cc:290] T 8c11074579b046d99e585c2818e18a2c P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:48.722298 20434 consensus_queue.cc:1059] T af57996a8be84d398f0f023e8d08ef77 P 647a343a542d4a139fe106c19095129b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.042s
I20260709 13:20:48.755401 20471 raft_consensus.cc:3060] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:20:48.760759 20398 consensus_queue.cc:1243] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Peer 647a343a542d4a139fe106c19095129b log is divergent from this leader: its last log entry 9.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
W20260709 13:20:48.780786 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 187302us
W20260709 13:20:48.781708 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 93354.6us
I20260709 13:20:48.759039 16147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d4ab870a05347868956aa8b841b2981" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 12 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
W20260709 13:20:48.782773 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 250478us
W20260709 13:20:48.787880 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 258554us
I20260709 13:20:48.797204 16191 leader_election.cc:304] T 1df93779209748b9860f6d893dd3de53 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:20:48.761323 20294 leader_election.cc:290] T c9944d5e4b8c4109af9955c4f08c9095 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:48.806066 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b7d1ab519044eda2d5e04a07242f25" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:48.806932 16224 raft_consensus.cc:2393] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 9: Already voted for candidate cb82cc7bd6db46d980c75237a1402860 in this term.
I20260709 13:20:48.807610 16219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f2da98864b4425bf7c0055951fdd4b" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:48.808688 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8158bbd17b994d2eb96f560bd0f1b88d" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:48.797821 16190 leader_election.cc:304] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: cb82cc7bd6db46d980c75237a1402860
I20260709 13:20:48.812561 16150 raft_consensus.cc:2368] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 for earlier term 11. Current term is 12.
I20260709 13:20:48.813529 16191 leader_election.cc:304] T 19baa7bc367844c288bca8f81439c413 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860; no voters: 90f3da57796b488c9087a2f523adf045
I20260709 13:20:48.799614 16265 leader_election.cc:304] T ecc1e363230043fe9fef390f0b979f20 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
I20260709 13:20:48.809533 16147 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40830 (request call id 1421) took 1021 ms. Trace:
I20260709 13:20:48.814955 16147 rpcz_store.cc:276] 0709 13:20:47.787780 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:48.758698 (+970918us) service_pool.cc:224] Handling call
0709 13:20:48.809407 (+ 50709us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:20:48.816969 16220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4dc6b0541ca431ebc757beac00521f1" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:48.819128 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66f75124215f4ca6bef831e174a792d8" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:48.821693 20393 leader_election.cc:290] T 434bff3082d141838068f323bdb0d1ed P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 10 election: Requested vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:48.822506 20484 raft_consensus.cc:3060] T 1df93779209748b9860f6d893dd3de53 P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:20:48.810324 20391 raft_consensus.cc:3055] T d16dfad0c3b540e5b4b86492ddf87208 P 1ca2725d290e47d68b1a4b609c9816ac [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:20:48.830194 20391 raft_consensus.cc:740] T d16dfad0c3b540e5b4b86492ddf87208 P 1ca2725d290e47d68b1a4b609c9816ac [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:20:48.780200 20289 leader_election.cc:290] T a378d7f796f94615b3b347119a9880fd P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 12 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:48.751226 20271 raft_consensus.cc:515] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:48.835564 16217 raft_consensus.cc:1275] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 13:20:48.835407 16219 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a378d7f796f94615b3b347119a9880fd"
dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
mode: GRACEFUL
new_leader_uuid: "90f3da57796b488c9087a2f523adf045"
 from {username='slave'} at 127.0.0.1:39144
I20260709 13:20:48.836400 16219 raft_consensus.cc:606] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Received request to transfer leadership to TS 90f3da57796b488c9087a2f523adf045
I20260709 13:20:48.836748 16219 raft_consensus.cc:614] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:20:48.816382 16147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cc959fb7f704fba9009c0595467d074" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 11 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:48.838338 16147 raft_consensus.cc:2410] T 8cc959fb7f704fba9009c0595467d074 P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 for term 11 because replica has last-logged OpId of term: 10 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 5.
I20260709 13:20:48.838871 16226 raft_consensus.cc:2468] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 10.
I20260709 13:20:48.795292 20434 consensus_queue.cc:1243] T af57996a8be84d398f0f023e8d08ef77 P 647a343a542d4a139fe106c19095129b [LEADER]: Peer 1ca2725d290e47d68b1a4b609c9816ac log is divergent from this leader: its last log entry 12.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 13:20:48.840145 16221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ad5dbfa99ee4b8d9e28be92b3f36db5" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:48.783615 20469 raft_consensus.cc:2804] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:20:48.842156 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cc959fb7f704fba9009c0595467d074" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:48.793447 20246 raft_consensus.cc:2749] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:20:48.854526 16820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66ae176961b74813a58bc15c9c4c6f2a" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:20:48.867666 16822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9944d5e4b8c4109af9955c4f08c9095" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:48.868644 16822 raft_consensus.cc:2468] T c9944d5e4b8c4109af9955c4f08c9095 P cb82cc7bd6db46d980c75237a1402860 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 11.
W20260709 13:20:48.869869 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 121484us
I20260709 13:20:48.588583 20274 raft_consensus.cc:515] T ada5a004ba7c46819051e3ccec71ffdb P cb82cc7bd6db46d980c75237a1402860 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:48.848873 16147 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40830 (request call id 1435) took 1054 ms. Trace:
I20260709 13:20:48.872357 16226 raft_consensus.cc:1240] T b6f07842d3aa47858df5665874c86f3a P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 11. Current term is 12. Ops: [11.7-11.7]
I20260709 13:20:48.872648 16147 rpcz_store.cc:276] 0709 13:20:47.794198 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:48.815955 (+1021757us) service_pool.cc:224] Handling call
0709 13:20:48.848804 (+ 32849us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:20:48.773432 20392 raft_consensus.cc:515] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } }
I20260709 13:20:48.778899 20270 consensus_queue.cc:1059] T d2ab0ac8347641e6955e9bce7d90c1a2 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.061s
I20260709 13:20:48.883775 16266 leader_election.cc:304] T c9944d5e4b8c4109af9955c4f08c9095 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:20:48.885996 16147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c2095a5f4ad49408dbc7f69c606120d" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:48.849212 20269 raft_consensus.cc:2804] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:20:48.888646 16816 raft_consensus.cc:1275] T 74ca8140f35c4ae29e8a664f6bdfc829 P cb82cc7bd6db46d980c75237a1402860 [term 11 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
W20260709 13:20:48.890090 16115 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:48.848603 20293 raft_consensus.cc:2804] T 19baa7bc367844c288bca8f81439c413 P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 13:20:48.894486 19982 consensus_meta.cc:287] T b6f07842d3aa47858df5665874c86f3a P 647a343a542d4a139fe106c19095129b: Time spent flushing consensus metadata: real 1.327s	user 0.009s	sys 0.005s
I20260709 13:20:48.897785 16296 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48674 (request call id 2618) took 1955 ms. Trace:
W20260709 13:20:48.898403 20401 consensus_meta.cc:287] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 90f3da57796b488c9087a2f523adf045: Time spent flushing consensus metadata: real 1.085s	user 0.006s	sys 0.003s
I20260709 13:20:48.784209 20495 consensus_queue.cc:1048] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.108s
W20260709 13:20:48.901021 20303 consensus_meta.cc:287] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045: Time spent flushing consensus metadata: real 1.257s	user 0.010s	sys 0.002s
I20260709 13:20:48.834393 20391 consensus_queue.cc:260] T d16dfad0c3b540e5b4b86492ddf87208 P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:48.901482 20303 raft_consensus.cc:515] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:48.902063 20391 raft_consensus.cc:3060] T d16dfad0c3b540e5b4b86492ddf87208 P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:20:48.902913 16142 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46224 (request call id 627) took 1419 ms. Trace:
I20260709 13:20:48.918717 16142 rpcz_store.cc:276] 0709 13:20:47.483841 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:47.483957 (+   116us) service_pool.cc:224] Handling call
0709 13:20:47.497481 (+ 13524us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:48.753662 (+1256181us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:48.902696 (+149034us) spinlock_profiling.cc:243] Waited 1.22 s on lock 0x7b540020ba54. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:20:48.902857 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2817170688}
W20260709 13:20:48.906766 20490 log.cc:927] Time spent T 8cc959fb7f704fba9009c0595467d074 P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
W20260709 13:20:48.907616 20483 log.cc:927] Time spent T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 0.319s	user 0.001s	sys 0.000s
I20260709 13:20:48.909016 16116 leader_election.cc:304] T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860
I20260709 13:20:48.911322 16146 raft_consensus.cc:2468] T b6f07842d3aa47858df5665874c86f3a P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 12.
I20260709 13:20:48.850404 20469 raft_consensus.cc:493] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:48.913826 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68a6838d82ef4cac83507f906139d90f" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:48.898229 16296 rpcz_store.cc:276] 0709 13:20:46.941960 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:47.376763 (+434803us) service_pool.cc:224] Handling call
0709 13:20:47.377064 (+   301us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:48.268069 (+891005us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:48.897533 (+629464us) spinlock_profiling.cc:243] Waited 811 ms on lock 0x7b540035e8d4. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:20:48.897711 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1864835840}
I20260709 13:20:48.919819 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5ad4eb281564936a59f41e387a47156" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
W20260709 13:20:48.920679 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 121150us
I20260709 13:20:48.934751 16265 leader_election.cc:304] T 8a878999c0024d2abe517484180290fc P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:20:48.920953 16824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ab0ac8347641e6955e9bce7d90c1a2" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:20:48.934880 16116 leader_election.cc:304] T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:20:48.921526 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a878999c0024d2abe517484180290fc" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
W20260709 13:20:48.932896 16115 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:48.938138 16298 raft_consensus.cc:2393] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 in current term 12: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
I20260709 13:20:48.939626 16116 leader_election.cc:304] T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860
W20260709 13:20:48.921276 20257 consensus_meta.cc:287] T c0eab25ce4c64e018a041beeda2867db P 1ca2725d290e47d68b1a4b609c9816ac: Time spent flushing consensus metadata: real 1.093s	user 0.012s	sys 0.000s
W20260709 13:20:48.955458 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 121457us
I20260709 13:20:48.945468 16116 leader_election.cc:304] T 8158bbd17b994d2eb96f560bd0f1b88d P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:20:48.892282 20269 raft_consensus.cc:493] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Starting leader election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
W20260709 13:20:48.942224 16266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 58604.9us
I20260709 13:20:48.988606 16116 leader_election.cc:304] T 84f2da98864b4425bf7c0055951fdd4b P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:20:48.898918 20401 raft_consensus.cc:515] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 90f3da57796b488c9087a2f523adf045 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:48.920059 20183 raft_consensus.cc:493] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:20:48.989753 16142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c74490c40243a88e62d1760b5cd814" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:48.912689 20293 raft_consensus.cc:493] T 19baa7bc367844c288bca8f81439c413 P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:20:48.985231 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 64133.4us
I20260709 13:20:48.998016 16142 raft_consensus.cc:2393] T 08c74490c40243a88e62d1760b5cd814 P 647a343a542d4a139fe106c19095129b [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 10: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
I20260709 13:20:48.922221 20289 raft_consensus.cc:2749] T ecc1e363230043fe9fef390f0b979f20 P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:20:48.922070 20268 raft_consensus.cc:3055] T d2ab0ac8347641e6955e9bce7d90c1a2 P cb82cc7bd6db46d980c75237a1402860 [term 10 LEADER]: Stepping down as leader of term 10
W20260709 13:20:49.000880 16223 consensus_meta.cc:287] T d2ab0ac8347641e6955e9bce7d90c1a2 P 1ca2725d290e47d68b1a4b609c9816ac: Time spent flushing consensus metadata: real 0.524s	user 0.008s	sys 0.002s
I20260709 13:20:48.938014 20469 raft_consensus.cc:3060] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:20:48.914662 20253 consensus_queue.cc:1175] T d16dfad0c3b540e5b4b86492ddf87208 P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "90f3da57796b488c9087a2f523adf045" responder_term: 11 status { last_received { term: 10 index: 6 } last_committed_idx: 5 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 10. Current term is 11. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:20:48.999527 16296 raft_consensus.cc:1275] T 7c2095a5f4ad49408dbc7f69c606120d P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:20:48.986716 20475 raft_consensus.cc:3060] T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:20:48.942759 19982 raft_consensus.cc:2749] T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:20:48.929873 20323 raft_consensus.cc:493] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:49.008499 16116 leader_election.cc:304] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:20:49.013685 20294 consensus_queue.cc:1059] T 434bff3082d141838068f323bdb0d1ed P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.006s
I20260709 13:20:49.019910 16116 leader_election.cc:304] T 8cc959fb7f704fba9009c0595467d074 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:20:49.020395 19982 raft_consensus.cc:2749] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:20:49.020785 20294 raft_consensus.cc:3055] T 434bff3082d141838068f323bdb0d1ed P cb82cc7bd6db46d980c75237a1402860 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:20:49.021068 19982 raft_consensus.cc:2749] T 8cc959fb7f704fba9009c0595467d074 P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:20:49.021337 20294 raft_consensus.cc:740] T 434bff3082d141838068f323bdb0d1ed P cb82cc7bd6db46d980c75237a1402860 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:20:49.022308 20432 leader_election.cc:290] T 829b4ad8c5a54eababeea1181b960d4b P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 11 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:48.997860 20402 consensus_queue.cc:1048] T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.025s
W20260709 13:20:48.938426 20337 consensus_meta.cc:287] T e5ad4eb281564936a59f41e387a47156 P 1ca2725d290e47d68b1a4b609c9816ac: Time spent flushing consensus metadata: real 0.789s	user 0.009s	sys 0.003s
I20260709 13:20:49.012280 16223 raft_consensus.cc:2468] T d2ab0ac8347641e6955e9bce7d90c1a2 P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 11.
I20260709 13:20:48.941517 20502 raft_consensus.cc:3060] T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:20:48.985086 20447 raft_consensus.cc:2804] T c9944d5e4b8c4109af9955c4f08c9095 P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:20:48.993224 20269 raft_consensus.cc:3060] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:20:49.004509 20293 raft_consensus.cc:3060] T 19baa7bc367844c288bca8f81439c413 P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:20:48.992959 20434 raft_consensus.cc:2749] T 8158bbd17b994d2eb96f560bd0f1b88d P 647a343a542d4a139fe106c19095129b [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:20:49.016422 16142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66f75124215f4ca6bef831e174a792d8" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:20:49.019246 20323 raft_consensus.cc:515] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:48.942581 20398 raft_consensus.cc:2749] T 8a878999c0024d2abe517484180290fc P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:20:49.013554 20504 raft_consensus.cc:2749] T 84f2da98864b4425bf7c0055951fdd4b P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:20:49.050338 20323 leader_election.cc:290] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:49.052285 20401 leader_election.cc:290] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 14 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
W20260709 13:20:49.053308 16191 connection.cc:256] RPC call timeout handler was delayed by -1.0923316040000373s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 1.5s for a response
I20260709 13:20:49.059899 16226 raft_consensus.cc:1240] T d2ab0ac8347641e6955e9bce7d90c1a2 P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 10. Current term is 11. Ops: []
I20260709 13:20:49.061460 16218 raft_consensus.cc:1240] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 12. Current term is 13. Ops: [12.10-12.10]
I20260709 13:20:49.031759 19982 raft_consensus.cc:493] T c0eab25ce4c64e018a041beeda2867db P 647a343a542d4a139fe106c19095129b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:49.027557 20271 leader_election.cc:290] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:20:48.992215 20257 raft_consensus.cc:515] T c0eab25ce4c64e018a041beeda2867db P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:20:49.029738 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 88221.8us
I20260709 13:20:49.003942 20416 consensus_queue.cc:1048] T 5a959dbecc2648eb85d1cc550f5a3245 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.029s
I20260709 13:20:49.045179 20434 raft_consensus.cc:493] T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b [term 10 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:20:49.032374 20447 raft_consensus.cc:493] T c9944d5e4b8c4109af9955c4f08c9095 P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:49.011971 20183 raft_consensus.cc:515] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:49.067454 16223 raft_consensus.cc:1240] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 10. Current term is 11. Ops: []
I20260709 13:20:49.012140 20268 raft_consensus.cc:740] T d2ab0ac8347641e6955e9bce7d90c1a2 P cb82cc7bd6db46d980c75237a1402860 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:20:49.086906 20398 raft_consensus.cc:493] T 5a959dbecc2648eb85d1cc550f5a3245 P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:49.089085 20268 consensus_queue.cc:260] T d2ab0ac8347641e6955e9bce7d90c1a2 P cb82cc7bd6db46d980c75237a1402860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:49.090555 20268 raft_consensus.cc:3060] T d2ab0ac8347641e6955e9bce7d90c1a2 P cb82cc7bd6db46d980c75237a1402860 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:20:49.090739 20183 leader_election.cc:290] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:49.087239 20447 raft_consensus.cc:3060] T c9944d5e4b8c4109af9955c4f08c9095 P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:20:49.068076 16219 raft_consensus.cc:1240] T 829b4ad8c5a54eababeea1181b960d4b P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 10. Current term is 11. Ops: [10.10-10.10]
I20260709 13:20:49.092909 16219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746aabf856ab401c9ef5e859371e7b67" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:49.094027 16219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66ae176961b74813a58bc15c9c4c6f2a" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:49.094595 16219 raft_consensus.cc:2468] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 13.
I20260709 13:20:49.069890 16218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8638ee363273426db1393dd81e5c7b72" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:20:49.095438 16218 raft_consensus.cc:3060] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:20:49.095301 20434 raft_consensus.cc:515] T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:49.097554 20434 leader_election.cc:290] T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:49.070745 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5718c623ebee4d2bb770f0f9ddb2f164" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:20:49.098636 20330 raft_consensus.cc:493] T 23f82224f4564fea93572e8206bad1fe P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:20:49.098783 20505 raft_consensus.cc:493] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:20:49.099319 16265 leader_election.cc:304] T 18ad7e3af5504571a69d10c00750a01c P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 647a343a542d4a139fe106c19095129b
I20260709 13:20:49.099222 20330 raft_consensus.cc:515] T 23f82224f4564fea93572e8206bad1fe P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:49.099615 20505 raft_consensus.cc:515] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:49.100884 20289 raft_consensus.cc:493] T b87147731a68400e8cce9b879daa7c5d P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:20:49.101413 20330 leader_election.cc:290] T 23f82224f4564fea93572e8206bad1fe P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:49.101384 20289 raft_consensus.cc:515] T b87147731a68400e8cce9b879daa7c5d P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:49.101979 20505 leader_election.cc:290] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:20:49.102288 20460 raft_consensus.cc:2764] T 18ad7e3af5504571a69d10c00750a01c P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Leader pre-election decision vote started in defunct term 9: won
I20260709 13:20:49.102559 20401 consensus_queue.cc:1059] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: INVALID_TERM, Last received: 9.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:20:49.103765 20402 raft_consensus.cc:3055] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:20:49.104130 20402 raft_consensus.cc:740] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:20:49.104242 20267 leader_election.cc:290] T c9944d5e4b8c4109af9955c4f08c9095 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:49.105392 20274 leader_election.cc:290] T ada5a004ba7c46819051e3ccec71ffdb P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:49.075428 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5d59407eb545b895d8d922e0dd69fd" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:20:49.106063 16225 raft_consensus.cc:3060] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:20:49.080425 19982 raft_consensus.cc:515] T c0eab25ce4c64e018a041beeda2867db P 647a343a542d4a139fe106c19095129b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:49.109922 19982 leader_election.cc:290] T c0eab25ce4c64e018a041beeda2867db P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:49.071547 16151 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46224 (request call id 629) took 1586 ms. Trace:
I20260709 13:20:49.076153 16220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee9ed67006144797b1de6ea8c2013a68" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:49.110602 16151 rpcz_store.cc:276] 0709 13:20:47.484649 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:47.484750 (+   101us) service_pool.cc:224] Handling call
0709 13:20:47.499240 (+ 14490us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:20:47.538724 (+ 39484us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:47.538753 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:20:47.546050 (+  7297us) log.cc:844] Serialized 26 byte log entry
0709 13:20:47.667150 (+121100us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:20:47.667225 (+    75us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:47.667300 (+    75us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:20:49.071091 (+1403791us) spinlock_profiling.cc:243] Waited 1 ms on lock 0x7b2c0010b6d8. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69aee0f291 00007f69aee2a0ca 00007f69b0b584ad 00007f69b0b5ff8e 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2
0709 13:20:49.071230 (+   139us) raft_consensus.cc:1685] finished
0709 13:20:49.071246 (+    16us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:49.071483 (+   237us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":28208,"raft.run_cpu_time_us":100,"raft.run_wall_time_us":3153,"spinlock_wait_cycles":2300160,"thread_start_us":99669,"threads_started":2,"wal-append.queue_time_us":143118}
I20260709 13:20:49.112617 16151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806d7690c2b644959152074bb2cab0a7" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 11 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:49.113351 16151 raft_consensus.cc:2393] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 11: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
I20260709 13:20:49.077649 20431 raft_consensus.cc:493] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:49.114899 16225 raft_consensus.cc:2468] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 12.
I20260709 13:20:49.114740 20431 raft_consensus.cc:515] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:49.117125 20431 leader_election.cc:290] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:49.088999 20398 raft_consensus.cc:515] T 5a959dbecc2648eb85d1cc550f5a3245 P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:49.099327 20451 raft_consensus.cc:493] T 2794af5b99984efebdee564035841cb0 P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:20:49.120478 20451 raft_consensus.cc:515] T 2794af5b99984efebdee564035841cb0 P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:49.121179 20398 leader_election.cc:290] T 5a959dbecc2648eb85d1cc550f5a3245 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:20:49.122268 16145 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40830 (request call id 1411) took 1342 ms. Trace:
I20260709 13:20:49.122520 20451 leader_election.cc:290] T 2794af5b99984efebdee564035841cb0 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:49.123912 20065 leader_election.cc:290] T fffab26427e24d398e2695aad40ca134 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:20:49.124622 16143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5718c623ebee4d2bb770f0f9ddb2f164" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:49.125286 16143 raft_consensus.cc:2368] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac for earlier term 10. Current term is 11.
I20260709 13:20:49.125978 16143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4f65cdf00b748bb963ee6e81ef43b79" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:49.126300 16300 raft_consensus.cc:2468] T cdbd78acb4cf46298e0de8ba0d130e19 P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 12.
I20260709 13:20:49.126503 16143 raft_consensus.cc:2468] T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 12.
I20260709 13:20:49.077682 20512 raft_consensus.cc:493] T 02838c86531d4f98bf58d1c22b8baf77 P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:20:49.127694 20512 raft_consensus.cc:515] T 02838c86531d4f98bf58d1c22b8baf77 P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:49.129699 16292 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40474 (request call id 696) took 2653 ms. Trace:
I20260709 13:20:49.114527 19673 raft_consensus.cc:515] T ee9ed67006144797b1de6ea8c2013a68 P cb82cc7bd6db46d980c75237a1402860 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:49.130131 16292 rpcz_store.cc:276] 0709 13:20:46.476303 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:47.301801 (+825498us) service_pool.cc:224] Handling call
0709 13:20:47.302102 (+   301us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:48.477183 (+1175081us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:49.129465 (+652282us) spinlock_profiling.cc:243] Waited 1.13 s on lock 0x7b54000ce254. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:20:49.129637 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2606633856}
I20260709 13:20:49.131250 16821 raft_consensus.cc:1240] T ee9ed67006144797b1de6ea8c2013a68 P cb82cc7bd6db46d980c75237a1402860 [term 9 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 8. Current term is 9. Ops: []
I20260709 13:20:49.132328 20257 leader_election.cc:290] T c0eab25ce4c64e018a041beeda2867db P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 11 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:49.132704 16150 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40830 (request call id 1434) took 1339 ms. Trace:
I20260709 13:20:49.132849 19673 leader_election.cc:290] T ee9ed67006144797b1de6ea8c2013a68 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 9 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:20:49.133144 16150 rpcz_store.cc:276] 0709 13:20:47.793766 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:48.475075 (+681309us) service_pool.cc:224] Handling call
0709 13:20:49.132502 (+657427us) spinlock_profiling.cc:243] Waited 336 ms on lock 0x7b540028e454. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:20:49.132646 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":773448960}
I20260709 13:20:49.133754 20502 raft_consensus.cc:2749] T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:20:49.134356 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6f07842d3aa47858df5665874c86f3a" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
W20260709 13:20:49.136543 20422 consensus_meta.cc:287] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac: Time spent flushing consensus metadata: real 0.604s	user 0.007s	sys 0.002s
W20260709 13:20:49.138559 20501 log.cc:927] Time spent T 5a96373bb0e14dce8a266b2fc6c58507 P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 0.349s	user 0.002s	sys 0.000s
I20260709 13:20:49.133163 16222 raft_consensus.cc:1240] T c0eab25ce4c64e018a041beeda2867db P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 10. Current term is 11. Ops: [10.7-10.7]
I20260709 13:20:49.139304 20289 leader_election.cc:290] T b87147731a68400e8cce9b879daa7c5d P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:49.033380 20337 raft_consensus.cc:515] T e5ad4eb281564936a59f41e387a47156 P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:49.141178 20424 leader_election.cc:290] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 12 election: Requested vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:49.141440 20337 leader_election.cc:290] T e5ad4eb281564936a59f41e387a47156 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 11 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:49.105042 20402 consensus_queue.cc:260] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:20:49.132450 20426 consensus_meta.cc:287] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b: Time spent flushing consensus metadata: real 0.629s	user 0.009s	sys 0.004s
I20260709 13:20:49.144349 16217 raft_consensus.cc:3060] T e5ad4eb281564936a59f41e387a47156 P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:20:49.145416 16143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87ac0d1f7e8a419f9bcbc04aee0e067e" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 12 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:49.037376 20303 leader_election.cc:290] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 12 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:49.022555 20294 consensus_queue.cc:260] T 434bff3082d141838068f323bdb0d1ed P cb82cc7bd6db46d980c75237a1402860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:49.147606 16115 leader_election.cc:304] T 8a878999c0024d2abe517484180290fc P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:20:49.173740 16115 leader_election.cc:304] T e5ad4eb281564936a59f41e387a47156 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:20:49.153636 16226 raft_consensus.cc:2468] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 11.
I20260709 13:20:49.176185 19982 raft_consensus.cc:2804] T 8a878999c0024d2abe517484180290fc P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:20:49.176651 19982 raft_consensus.cc:493] T 8a878999c0024d2abe517484180290fc P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Starting leader election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:20:49.177042 19982 raft_consensus.cc:3060] T 8a878999c0024d2abe517484180290fc P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:20:49.179100 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 78598.9us
I20260709 13:20:49.179785 20504 consensus_queue.cc:1059] T 1df93779209748b9860f6d893dd3de53 P 647a343a542d4a139fe106c19095129b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:20:49.160197 16143 raft_consensus.cc:2393] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 12: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
I20260709 13:20:49.180837 20504 raft_consensus.cc:2804] T e5ad4eb281564936a59f41e387a47156 P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:20:49.161777 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b7d1ab519044eda2d5e04a07242f25" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:20:49.181737 16225 raft_consensus.cc:2376] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 9. Re-sending same reply.
I20260709 13:20:49.182502 16219 raft_consensus.cc:1275] T ecc1e363230043fe9fef390f0b979f20 P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
W20260709 13:20:49.182721 16790 leader_election.cc:336] T 66f75124215f4ca6bef831e174a792d8 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943): Timed out: RequestConsensusVote RPC to 127.14.177.130:35943 timed out after 1.746s (SENT)
I20260709 13:20:49.162685 16294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9944d5e4b8c4109af9955c4f08c9095" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:49.183454 16151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee9ed67006144797b1de6ea8c2013a68" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:20:49.184123 16151 raft_consensus.cc:3060] T ee9ed67006144797b1de6ea8c2013a68 P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:20:49.122682 16145 rpcz_store.cc:276] 0709 13:20:47.779995 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:47.783302 (+  3307us) service_pool.cc:224] Handling call
0709 13:20:47.783618 (+   316us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:47.783816 (+   198us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:20:47.786848 (+  3032us) log.cc:844] Serialized 14 byte log entry
0709 13:20:49.121817 (+1334969us) spinlock_profiling.cc:243] Waited 13.5 ms on lock 0x7b18003be2a0. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b693ade1 00007f69b69475b1 00007f69b07074bc 00007f69ada5548b 00007f69b68afab9 00007f69ada7169b 00007f69ada6f93f 00007f69ada6f455 00007f69b0b57060 00007f69b0b69716 00007f69b0b7a287
0709 13:20:49.121916 (+    99us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:20:49.121964 (+    48us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:20:49.122016 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:49.122080 (+    64us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:49.122212 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":25963,"prepare.queue_time_us":23721,"prepare.run_cpu_time_us":35,"prepare.run_wall_time_us":35,"raft.queue_time_us":215835,"raft.run_cpu_time_us":174,"raft.run_wall_time_us":10136,"spinlock_wait_cycles":31162112,"thread_start_us":358070,"threads_started":3,"wal-append.queue_time_us":361481}
I20260709 13:20:49.156561 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434bff3082d141838068f323bdb0d1ed" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:20:49.171381 16223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66f75124215f4ca6bef831e174a792d8" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:49.181334 20434 consensus_queue.cc:1059] T 02838c86531d4f98bf58d1c22b8baf77 P 647a343a542d4a139fe106c19095129b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.010s
I20260709 13:20:49.191721 20440 consensus_queue.cc:1059] T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: INVALID_TERM, Last received: 11.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.020s
I20260709 13:20:49.192723 20440 raft_consensus.cc:3055] T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:20:49.193142 20440 raft_consensus.cc:740] T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:20:49.194069 20440 consensus_queue.cc:260] T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:49.195483 20472 raft_consensus.cc:3055] T 1df93779209748b9860f6d893dd3de53 P 647a343a542d4a139fe106c19095129b [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:20:49.195962 20472 raft_consensus.cc:740] T 1df93779209748b9860f6d893dd3de53 P 647a343a542d4a139fe106c19095129b [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:20:49.196841 20472 consensus_queue.cc:260] T 1df93779209748b9860f6d893dd3de53 P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:49.198141 20472 raft_consensus.cc:3060] T 1df93779209748b9860f6d893dd3de53 P 647a343a542d4a139fe106c19095129b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:20:49.200505 20294 raft_consensus.cc:3060] T 434bff3082d141838068f323bdb0d1ed P cb82cc7bd6db46d980c75237a1402860 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:20:49.204968 16295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8638ee363273426db1393dd81e5c7b72" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:20:49.205240 16218 raft_consensus.cc:2468] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 12.
I20260709 13:20:49.205689 16295 raft_consensus.cc:2376] T 8638ee363273426db1393dd81e5c7b72 P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 12. Re-sending same reply.
I20260709 13:20:49.190831 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62fe3346cd3c42c4a55e14af9073bca4" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:49.206705 16150 raft_consensus.cc:2468] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 13.
I20260709 13:20:49.181458 20504 raft_consensus.cc:697] T e5ad4eb281564936a59f41e387a47156 P 647a343a542d4a139fe106c19095129b [term 12 LEADER]: Becoming Leader. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:20:49.214622 20434 raft_consensus.cc:3055] T 02838c86531d4f98bf58d1c22b8baf77 P 647a343a542d4a139fe106c19095129b [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:20:49.215188 20434 raft_consensus.cc:740] T 02838c86531d4f98bf58d1c22b8baf77 P 647a343a542d4a139fe106c19095129b [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:20:49.215026 20504 consensus_queue.cc:237] T e5ad4eb281564936a59f41e387a47156 P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:49.216238 20434 consensus_queue.cc:260] T 02838c86531d4f98bf58d1c22b8baf77 P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:49.217542 20434 raft_consensus.cc:3060] T 02838c86531d4f98bf58d1c22b8baf77 P 647a343a542d4a139fe106c19095129b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:20:49.102013 20330 raft_consensus.cc:493] T 26354361e9b044cea1c2b86c4882fe92 P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:20:49.231787 16295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31969392ccdb43b491911a319e8bc7e1" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:49.164860 20183 raft_consensus.cc:493] T 7c2095a5f4ad49408dbc7f69c606120d P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:49.238014 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68a6838d82ef4cac83507f906139d90f" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:49.242074 16295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0eab25ce4c64e018a041beeda2867db" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:20:49.262122 20512 leader_election.cc:290] T 02838c86531d4f98bf58d1c22b8baf77 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:49.262374 20183 raft_consensus.cc:515] T 7c2095a5f4ad49408dbc7f69c606120d P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:49.265403 20183 leader_election.cc:290] T 7c2095a5f4ad49408dbc7f69c606120d P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:20:49.269706 20469 raft_consensus.cc:515] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:20:49.240630 20315 consensus_meta.cc:287] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac: Time spent flushing consensus metadata: real 1.470s	user 0.008s	sys 0.004s
I20260709 13:20:49.240224 20471 raft_consensus.cc:515] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:49.239049 20440 raft_consensus.cc:3060] T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:20:49.299836 20471 leader_election.cc:290] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 9 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:49.300707 16823 raft_consensus.cc:1240] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860 [term 9 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 8. Current term is 9. Ops: []
I20260709 13:20:49.303635 20457 consensus_queue.cc:260] T 746aabf856ab401c9ef5e859371e7b67 P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:49.270721 20269 raft_consensus.cc:515] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:49.278365 20484 raft_consensus.cc:2749] T 1df93779209748b9860f6d893dd3de53 P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:20:49.247357 20330 raft_consensus.cc:515] T 26354361e9b044cea1c2b86c4882fe92 P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:49.307596 20269 leader_election.cc:290] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 11 election: Requested vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:49.307891 20330 leader_election.cc:290] T 26354361e9b044cea1c2b86c4882fe92 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:49.286283 20293 raft_consensus.cc:515] T 19baa7bc367844c288bca8f81439c413 P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:49.308501 20457 raft_consensus.cc:3060] T 746aabf856ab401c9ef5e859371e7b67 P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:20:49.310343 20293 leader_election.cc:290] T 19baa7bc367844c288bca8f81439c413 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 12 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:49.290143 20447 raft_consensus.cc:515] T c9944d5e4b8c4109af9955c4f08c9095 P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:49.296069 20315 raft_consensus.cc:515] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:49.312752 20315 leader_election.cc:290] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 13 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:49.305441 16820 raft_consensus.cc:2368] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 for earlier term 13. Current term is 14.
I20260709 13:20:49.306874 20469 leader_election.cc:290] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 14 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:49.315109 20447 leader_election.cc:290] T c9944d5e4b8c4109af9955c4f08c9095 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 12 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:49.271914 20475 raft_consensus.cc:2749] T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:20:49.316761 20268 raft_consensus.cc:493] T 8c11074579b046d99e585c2818e18a2c P cb82cc7bd6db46d980c75237a1402860 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:49.310967 19982 raft_consensus.cc:515] T 8a878999c0024d2abe517484180290fc P 647a343a542d4a139fe106c19095129b [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:49.325456 20268 raft_consensus.cc:515] T 8c11074579b046d99e585c2818e18a2c P cb82cc7bd6db46d980c75237a1402860 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:49.339011 19982 leader_election.cc:290] T 8a878999c0024d2abe517484180290fc P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 10 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:49.349159 20268 leader_election.cc:290] T 8c11074579b046d99e585c2818e18a2c P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:49.350380 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5ad4eb281564936a59f41e387a47156" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:20:49.351262 16293 raft_consensus.cc:2368] T e5ad4eb281564936a59f41e387a47156 P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac for earlier term 11. Current term is 12.
I20260709 13:20:49.352088 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0eab25ce4c64e018a041beeda2867db" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:49.353540 16293 raft_consensus.cc:1240] T 7c2095a5f4ad49408dbc7f69c606120d P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 7. Current term is 8. Ops: []
I20260709 13:20:49.379407 20236 raft_consensus.cc:3055] T af57996a8be84d398f0f023e8d08ef77 P 647a343a542d4a139fe106c19095129b [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:20:49.379918 20236 raft_consensus.cc:740] T af57996a8be84d398f0f023e8d08ef77 P 647a343a542d4a139fe106c19095129b [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:20:49.380838 20236 consensus_queue.cc:260] T af57996a8be84d398f0f023e8d08ef77 P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:49.382299 20236 raft_consensus.cc:3060] T af57996a8be84d398f0f023e8d08ef77 P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:20:49.387075 20392 leader_election.cc:290] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 11 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
W20260709 13:20:49.394618 20458 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.205s	user 0.001s	sys 0.003s
I20260709 13:20:49.394946 20448 raft_consensus.cc:3055] T b87147731a68400e8cce9b879daa7c5d P 647a343a542d4a139fe106c19095129b [term 11 LEADER]: Stepping down as leader of term 11
W20260709 13:20:49.395079 20458 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.205s	user 0.001s	sys 0.003s
I20260709 13:20:49.395448 20448 raft_consensus.cc:740] T b87147731a68400e8cce9b879daa7c5d P 647a343a542d4a139fe106c19095129b [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:20:49.395533 20458 raft_consensus.cc:697] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b [term 13 LEADER]: Becoming Leader. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:20:49.396251 20448 consensus_queue.cc:260] T b87147731a68400e8cce9b879daa7c5d P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:49.396421 20458 consensus_queue.cc:237] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:49.397300 20448 raft_consensus.cc:3060] T b87147731a68400e8cce9b879daa7c5d P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:20:49.413774 16790 leader_election.cc:336] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943): Timed out: RequestConsensusVote RPC to 127.14.177.130:35943 timed out after 2.046s (SENT)
I20260709 13:20:49.414427 16790 leader_election.cc:304] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:20:49.417825 20268 raft_consensus.cc:2749] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:20:49.445088 20486 log.cc:927] Time spent T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860: Append to log took a long time: real 0.666s	user 0.003s	sys 0.000s
I20260709 13:20:49.450552 16217 raft_consensus.cc:2468] T e5ad4eb281564936a59f41e387a47156 P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 12.
I20260709 13:20:49.472982 20268 raft_consensus.cc:493] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:49.473836 20268 raft_consensus.cc:515] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:49.477422 20268 leader_election.cc:290] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:49.472509 16144 raft_consensus.cc:1275] T c0eab25ce4c64e018a041beeda2867db P 647a343a542d4a139fe106c19095129b [term 10 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:20:49.479698 16144 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40830 (request call id 1426) took 1689 ms. Trace:
I20260709 13:20:49.480419 20402 raft_consensus.cc:3060] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:20:49.480189 16144 rpcz_store.cc:276] 0709 13:20:47.789979 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:49.471758 (+1681779us) service_pool.cc:224] Handling call
0709 13:20:49.472114 (+   356us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:49.479342 (+  7228us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:49.479599 (+   257us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:20:49.488350 16144 raft_consensus.cc:1240] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 11. Current term is 12. Ops: []
I20260709 13:20:49.489120 16144 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40830 (request call id 1432) took 1696 ms. Trace:
I20260709 13:20:49.489514 16144 rpcz_store.cc:276] 0709 13:20:47.792789 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:49.487756 (+1694967us) service_pool.cc:224] Handling call
0709 13:20:49.488139 (+   383us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:49.488802 (+   663us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:49.489040 (+   238us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:20:49.491801 16265 leader_election.cc:336] T 8d4ab870a05347868956aa8b841b2981 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Timed out: RequestConsensusVote RPC to 127.14.177.129:36741 timed out after 3.074s (SENT)
I20260709 13:20:49.492357 16265 leader_election.cc:304] T 8d4ab870a05347868956aa8b841b2981 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:20:49.493161 20447 raft_consensus.cc:2749] T 8d4ab870a05347868956aa8b841b2981 P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:20:49.493774 16265 leader_election.cc:336] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 11 election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Timed out: RequestConsensusVote RPC to 127.14.177.129:36741 timed out after 2.312s (SENT)
I20260709 13:20:49.495187 16144 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46224 (request call id 636) took 1591 ms. Trace:
I20260709 13:20:49.495651 16144 rpcz_store.cc:276] 0709 13:20:47.967582 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:49.494154 (+1526572us) service_pool.cc:224] Handling call
0709 13:20:49.494576 (+   422us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:49.494776 (+   200us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:49.494826 (+    50us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:20:49.494875 (+    49us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:20:49.494910 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:49.494973 (+    63us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:49.495108 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:20:49.496701 16145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62fe3346cd3c42c4a55e14af9073bca4" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:20:49.497304 16145 raft_consensus.cc:3060] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:20:49.521629 20402 raft_consensus.cc:493] T c4f65cdf00b748bb963ee6e81ef43b79 P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:49.524281 20402 raft_consensus.cc:515] T c4f65cdf00b748bb963ee6e81ef43b79 P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:49.527854 16145 raft_consensus.cc:2468] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 14.
I20260709 13:20:49.076433 16037 maintenance_manager.cc:643] P 667a34b27a39474ebefc1af3cf5b4271: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 11.652s	user 0.197s	sys 0.131s Metrics: {"bytes_written":29845,"cfile_init":1,"dirs.queue_time_us":72633,"dirs.run_cpu_time_us":403,"dirs.run_wall_time_us":1226,"drs_written":1,"lbm_read_time_us":7352,"lbm_reads_1-10_ms":1,"lbm_reads_lt_1ms":3,"lbm_write_time_us":425742,"lbm_writes_10-100_ms":9,"lbm_writes_gt_100_ms":1,"lbm_writes_lt_1ms":20,"peak_mem_usage":0,"raft.queue_time_us":1130415,"raft.run_cpu_time_us":27656,"raft.run_wall_time_us":232364,"rows_written":63,"thread_start_us":1942192,"threads_started":3,"wal-append.queue_time_us":1128998}
I20260709 13:20:49.635499 16294 raft_consensus.cc:2393] T c9944d5e4b8c4109af9955c4f08c9095 P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 in current term 12: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
W20260709 13:20:49.763501 16264 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:20:50.231299 16264 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:20:50.291539 16263 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:50.355644 20257 raft_consensus.cc:493] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:20:50.356099 16295 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48684 (request call id 2502) took 1149 ms. Trace:
I20260709 13:20:50.356573 16295 rpcz_store.cc:276] 0709 13:20:49.206714 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:49.237164 (+ 30450us) service_pool.cc:224] Handling call
0709 13:20:50.356032 (+1118868us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:20:50.356595 20257 raft_consensus.cc:515] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:20:50.364347 16265 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.681s	user 0.010s	sys 0.017s
W20260709 13:20:50.364830 16265 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.682s	user 0.010s	sys 0.018s
W20260709 13:20:50.365314 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 682744us
W20260709 13:20:50.368008 16265 leader_election.cc:336] T 57a744f5aaf944678a63469cbf514015 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Timed out: RequestConsensusVote RPC to 127.14.177.129:36741 timed out after 2.008s (SENT)
I20260709 13:20:50.368721 16265 leader_election.cc:304] T 57a744f5aaf944678a63469cbf514015 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
I20260709 13:20:50.429574 20257 leader_election.cc:290] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
W20260709 13:20:50.549517 16321 debug-util.cc:398] Leaking SignalData structure 0x7b08002de380 after lost signal to thread 15046
W20260709 13:20:50.550299 16321 debug-util.cc:398] Leaking SignalData structure 0x7b08002de5a0 after lost signal to thread 15049
W20260709 13:20:50.734871 16264 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:20:50.821468 16190 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.14.177.131:41389 timed out after 2.433s (SENT) [suppressed 26 similar messages]
W20260709 13:20:50.986152 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 618114us
W20260709 13:20:50.987350 16265 leader_election.cc:336] T 02838c86531d4f98bf58d1c22b8baf77 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Timed out: RequestConsensusVote RPC to 127.14.177.129:36741 timed out after 2.082s (SENT)
W20260709 13:20:51.220671 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 232001us
I20260709 13:20:51.222424 16219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2372981006d4b218215c28964dea623" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:20:51.224787 16219 raft_consensus.cc:2410] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 for term 12 because replica has last-logged OpId of term: 11 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 8.
I20260709 13:20:51.228055 16225 raft_consensus.cc:1275] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 13:20:51.304826 16221 raft_consensus.cc:1240] T 1df93779209748b9860f6d893dd3de53 P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 10. Current term is 11. Ops: []
W20260709 13:20:51.315766 16264 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:20:51.553937 16115 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:20:51.555124 16115 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:20:51.556108 16115 connection.cc:256] RPC call timeout handler was delayed by -1.3867578370000047s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1876s for a response
W20260709 13:20:51.557237 16115 connection.cc:256] RPC call timeout handler was delayed by -1.2935100019999481s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15960000000000002s for a response
W20260709 13:20:51.558537 16116 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:51.572463 20529 raft_consensus.cc:493] T 9e65676b21b748728ad39668cfe722ec P 647a343a542d4a139fe106c19095129b [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:20:51.573706 20529 raft_consensus.cc:515] T 9e65676b21b748728ad39668cfe722ec P 647a343a542d4a139fe106c19095129b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:51.577548 20529 leader_election.cc:290] T 9e65676b21b748728ad39668cfe722ec P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
W20260709 13:20:51.581333 16117 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:51.582084 20529 raft_consensus.cc:493] T 746aabf856ab401c9ef5e859371e7b67 P 647a343a542d4a139fe106c19095129b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:51.583891 20529 raft_consensus.cc:515] T 746aabf856ab401c9ef5e859371e7b67 P 647a343a542d4a139fe106c19095129b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:51.593369 20529 leader_election.cc:290] T 746aabf856ab401c9ef5e859371e7b67 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:51.594038 20529 raft_consensus.cc:493] T fffab26427e24d398e2695aad40ca134 P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:51.594897 20529 raft_consensus.cc:515] T fffab26427e24d398e2695aad40ca134 P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:51.597847 20529 leader_election.cc:290] T fffab26427e24d398e2695aad40ca134 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:51.612049 20529 raft_consensus.cc:493] T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:51.612968 20529 raft_consensus.cc:515] T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:20:51.610085 16151 consensus_meta.cc:287] T ee9ed67006144797b1de6ea8c2013a68 P 647a343a542d4a139fe106c19095129b: Time spent flushing consensus metadata: real 2.425s	user 0.032s	sys 0.017s
W20260709 13:20:51.611435 16788 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:51.615691 20529 leader_election.cc:290] T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:51.616345 20529 raft_consensus.cc:493] T 8cc959fb7f704fba9009c0595467d074 P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:51.617200 16151 raft_consensus.cc:2468] T ee9ed67006144797b1de6ea8c2013a68 P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 9.
W20260709 13:20:51.618091 16151 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46224 (request call id 643) took 2476 ms (client timeout 2176 ms). Trace:
W20260709 13:20:51.618813 16151 rpcz_store.cc:269] 0709 13:20:49.141241 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:49.145608 (+  4367us) service_pool.cc:224] Handling call
0709 13:20:51.617925 (+2472317us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:20:51.618112 16791 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:20:51.627578 16791 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:20:51.629997 16265 leader_election.cc:336] T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Timed out: RequestConsensusVote RPC to 127.14.177.129:36741 timed out after 2.208s (SENT)
I20260709 13:20:51.633493 16265 leader_election.cc:304] T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:20:51.617221 20529 raft_consensus.cc:515] T 8cc959fb7f704fba9009c0595467d074 P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:20:51.635576 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 406151us
I20260709 13:20:51.637110 20021 leader_election.cc:290] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 12 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:51.637403 20529 leader_election.cc:290] T 8cc959fb7f704fba9009c0595467d074 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:51.667680 20529 raft_consensus.cc:493] T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:20:51.648136 20236 raft_consensus.cc:493] T 84f2da98864b4425bf7c0055951fdd4b P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:20:51.656730 16265 leader_election.cc:336] T 1df93779209748b9860f6d893dd3de53 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 11 election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Timed out: RequestConsensusVote RPC to 127.14.177.129:36741 timed out after 1.857s (SENT)
I20260709 13:20:51.656901 19673 raft_consensus.cc:493] T 434bff3082d141838068f323bdb0d1ed P cb82cc7bd6db46d980c75237a1402860 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:51.668731 16265 leader_election.cc:304] T 1df93779209748b9860f6d893dd3de53 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:20:51.668653 20529 raft_consensus.cc:515] T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:51.669312 19673 raft_consensus.cc:515] T 434bff3082d141838068f323bdb0d1ed P cb82cc7bd6db46d980c75237a1402860 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:51.671469 20529 leader_election.cc:290] T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
W20260709 13:20:51.671762 16265 leader_election.cc:336] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Timed out: RequestConsensusVote RPC to 127.14.177.129:36741 timed out after 1.900s (SENT)
I20260709 13:20:51.645015 16821 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49906 (request call id 882) took 3471 ms. Trace:
I20260709 13:20:51.672171 19673 leader_election.cc:290] T 434bff3082d141838068f323bdb0d1ed P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:51.672518 16265 leader_election.cc:304] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:20:51.668710 20236 raft_consensus.cc:515] T 84f2da98864b4425bf7c0055951fdd4b P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:51.672628 16821 rpcz_store.cc:276] 0709 13:20:48.173536 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:48.173662 (+   126us) service_pool.cc:224] Handling call
0709 13:20:48.173946 (+   284us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:49.131633 (+957687us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:51.644344 (+2512711us) spinlock_profiling.cc:243] Waited 957 ms on lock 0x7b5400721854. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:20:51.644742 (+   398us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2201133440}
I20260709 13:20:51.672096 20529 raft_consensus.cc:493] T ecc1e363230043fe9fef390f0b979f20 P 647a343a542d4a139fe106c19095129b [term 10 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:20:51.675410 20236 leader_election.cc:290] T 84f2da98864b4425bf7c0055951fdd4b P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:51.676152 20236 raft_consensus.cc:493] T b6f07842d3aa47858df5665874c86f3a P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:51.677002 20236 raft_consensus.cc:515] T b6f07842d3aa47858df5665874c86f3a P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:51.677558 16150 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46224 (request call id 638) took 2844 ms. Trace:
I20260709 13:20:51.667764 20021 raft_consensus.cc:493] T 3884f864a96d428785895e5463e5728f P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:20:51.776476 16150 rpcz_store.cc:276] 0709 13:20:48.832715 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:51.675906 (+2843191us) service_pool.cc:224] Handling call
0709 13:20:51.676642 (+   736us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:51.677058 (+   416us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:51.677098 (+    40us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:20:51.677160 (+    62us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:20:51.677194 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:51.677252 (+    58us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:51.677371 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:20:51.776934 20021 raft_consensus.cc:515] T 3884f864a96d428785895e5463e5728f P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:51.675190 20529 raft_consensus.cc:515] T ecc1e363230043fe9fef390f0b979f20 P 647a343a542d4a139fe106c19095129b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:51.780251 20021 leader_election.cc:290] T 3884f864a96d428785895e5463e5728f P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:51.679463 20236 leader_election.cc:290] T b6f07842d3aa47858df5665874c86f3a P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:51.680030 19673 raft_consensus.cc:493] T d2ab0ac8347641e6955e9bce7d90c1a2 P cb82cc7bd6db46d980c75237a1402860 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:51.700549 20392 raft_consensus.cc:493] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:51.781821 19673 raft_consensus.cc:515] T d2ab0ac8347641e6955e9bce7d90c1a2 P cb82cc7bd6db46d980c75237a1402860 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:51.782091 20392 raft_consensus.cc:515] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:51.784772 20392 leader_election.cc:290] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:51.784835 19673 leader_election.cc:290] T d2ab0ac8347641e6955e9bce7d90c1a2 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:51.693701 20447 raft_consensus.cc:2749] T 57a744f5aaf944678a63469cbf514015 P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:20:51.789858 20447 raft_consensus.cc:493] T e5ad4eb281564936a59f41e387a47156 P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:51.790534 20447 raft_consensus.cc:515] T e5ad4eb281564936a59f41e387a47156 P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:51.793432 20447 leader_election.cc:290] T e5ad4eb281564936a59f41e387a47156 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:20:51.794113 20447 raft_consensus.cc:493] T e4dc6b0541ca431ebc757beac00521f1 P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:20:51.730971 20402 leader_election.cc:290] T c4f65cdf00b748bb963ee6e81ef43b79 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:20:51.795699 20402 raft_consensus.cc:493] T 8a878999c0024d2abe517484180290fc P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:20:51.796310 20402 raft_consensus.cc:515] T 8a878999c0024d2abe517484180290fc P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:51.798504 20402 leader_election.cc:290] T 8a878999c0024d2abe517484180290fc P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:20:51.732725 20448 raft_consensus.cc:493] T 5a959dbecc2648eb85d1cc550f5a3245 P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:51.736917 20450 raft_consensus.cc:493] T 8158bbd17b994d2eb96f560bd0f1b88d P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:51.738440 20429 raft_consensus.cc:493] T 74ca8140f35c4ae29e8a664f6bdfc829 P cb82cc7bd6db46d980c75237a1402860 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:51.740995 20426 raft_consensus.cc:493] T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:51.709829 20401 raft_consensus.cc:493] T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
W20260709 13:20:51.753713 20474 log.cc:927] Time spent T 7c2095a5f4ad49408dbc7f69c606120d P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 3.162s	user 0.007s	sys 0.020s
I20260709 13:20:51.660171 16221 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57604 (request call id 2547) took 2756 ms. Trace:
W20260709 13:20:51.672825 16294 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40474 (request call id 717) took 2517 ms (client timeout 1757 ms). Trace:
I20260709 13:20:51.775938 16038 maintenance_manager.cc:419] P 667a34b27a39474ebefc1af3cf5b4271: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 9346 bytes on disk
W20260709 13:20:51.777719 16146 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40830 (request call id 1415) took 3992 ms (client timeout 2208 ms). Trace:
W20260709 13:20:51.820210 16146 rpcz_store.cc:269] 0709 13:20:47.784801 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:47.784967 (+   166us) service_pool.cc:224] Handling call
0709 13:20:51.777340 (+3992373us) spinlock_profiling.cc:243] Waited 1.11 s on lock 0x7b540012fa54. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:20:51.777525 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2551079808}
I20260709 13:20:51.781211 20529 leader_election.cc:290] T ecc1e363230043fe9fef390f0b979f20 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:51.702406 20440 raft_consensus.cc:493] T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:51.824503 20440 raft_consensus.cc:515] T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:20:51.745810 20480 log.cc:927] Time spent T 26354361e9b044cea1c2b86c4882fe92 P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 3.324s	user 0.005s	sys 0.026s
I20260709 13:20:51.829500 16148 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46224 (request call id 635) took 4283 ms. Trace:
I20260709 13:20:51.830044 16148 rpcz_store.cc:276] 0709 13:20:47.545491 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:47.545714 (+   223us) service_pool.cc:224] Handling call
0709 13:20:47.545998 (+   284us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:20:47.546215 (+   217us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:47.546246 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:20:47.551756 (+  5510us) log.cc:844] Serialized 26 byte log entry
0709 13:20:48.268089 (+716333us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:20:48.268126 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:48.268182 (+    56us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:20:51.744747 (+3476565us) spinlock_profiling.cc:243] Waited 7.15 ms on lock 0x7b4400022b4c. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69ad8fadb1 00007f69ad998c85 00007f69ad9996a2 00007f69b070b1db 00007f69b0706aaf 00007f69b06fa86e 00007f69b06f9bb6 00007f69b06f97d8 00007f69b0703221 00007f69b07031d9 00007f69b0703169
0709 13:20:51.828825 (+ 84078us) raft_consensus.cc:1685] finished
0709 13:20:51.828852 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:51.829254 (+   402us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":16445312,"thread_start_us":715924,"threads_started":1,"wal-append.queue_time_us":703766}
I20260709 13:20:51.832988 16148 raft_consensus.cc:1240] T 8a878999c0024d2abe517484180290fc P 647a343a542d4a139fe106c19095129b [term 10 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 9. Current term is 10. Ops: []
I20260709 13:20:51.833760 16148 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40844 (request call id 2341) took 2989 ms. Trace:
I20260709 13:20:51.834163 16148 rpcz_store.cc:276] 0709 13:20:48.844458 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:51.832135 (+2987677us) service_pool.cc:224] Handling call
0709 13:20:51.832693 (+   558us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:51.833478 (+   785us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:51.833696 (+   218us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:20:51.799692 20448 raft_consensus.cc:515] T 5a959dbecc2648eb85d1cc550f5a3245 P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:51.837366 20448 leader_election.cc:290] T 5a959dbecc2648eb85d1cc550f5a3245 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:51.804571 19673 raft_consensus.cc:493] T 9e65676b21b748728ad39668cfe722ec P cb82cc7bd6db46d980c75237a1402860 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:20:51.838315 19673 raft_consensus.cc:515] T 9e65676b21b748728ad39668cfe722ec P cb82cc7bd6db46d980c75237a1402860 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:51.841511 19673 leader_election.cc:290] T 9e65676b21b748728ad39668cfe722ec P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:51.794763 20447 raft_consensus.cc:515] T e4dc6b0541ca431ebc757beac00521f1 P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:20:51.844269 16217 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57604 (request call id 2544) took 2941 ms (client timeout 1671 ms). Trace:
I20260709 13:20:51.845306 20447 leader_election.cc:290] T e4dc6b0541ca431ebc757beac00521f1 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
W20260709 13:20:51.845463 16217 rpcz_store.cc:269] 0709 13:20:48.902767 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:48.919536 (+ 16769us) service_pool.cc:224] Handling call
0709 13:20:51.843686 (+2924150us) spinlock_profiling.cc:243] Waited 210 ms on lock 0x7b54003d2a54. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:20:51.843975 (+   289us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":482639104}
I20260709 13:20:51.846221 20447 raft_consensus.cc:493] T 8638ee363273426db1393dd81e5c7b72 P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:51.857342 16218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9944d5e4b8c4109af9955c4f08c9095" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:51.858559 16218 raft_consensus.cc:2468] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 11.
I20260709 13:20:51.805991 16221 rpcz_store.cc:276] 0709 13:20:48.903777 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:48.920289 (+ 16512us) service_pool.cc:224] Handling call
0709 13:20:48.929755 (+  9466us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:51.305634 (+2375879us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:51.659769 (+354135us) spinlock_profiling.cc:243] Waited 2.37 s on lock 0x7b5400246654. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:20:51.660043 (+   274us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5460808576}
I20260709 13:20:51.799019 20402 raft_consensus.cc:493] T cdbd78acb4cf46298e0de8ba0d130e19 P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:20:51.806427 16294 rpcz_store.cc:269] 0709 13:20:49.155541 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:49.162394 (+  6853us) service_pool.cc:224] Handling call
0709 13:20:51.672275 (+2509881us) spinlock_profiling.cc:243] Waited 452 ms on lock 0x7b540010f254. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:20:51.672557 (+   282us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1038715264}
I20260709 13:20:51.893003 20529 raft_consensus.cc:493] T 9c61a39fd4f84424b129c1a6f9ba404d P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:51.899641 20429 raft_consensus.cc:515] T 74ca8140f35c4ae29e8a664f6bdfc829 P cb82cc7bd6db46d980c75237a1402860 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:51.902647 20429 leader_election.cc:290] T 74ca8140f35c4ae29e8a664f6bdfc829 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:51.903365 20426 raft_consensus.cc:515] T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:51.905268 16150 raft_consensus.cc:1240] T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 11. Current term is 12. Ops: []
I20260709 13:20:51.907459 20429 raft_consensus.cc:493] T 66f75124215f4ca6bef831e174a792d8 P cb82cc7bd6db46d980c75237a1402860 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:51.908411 20429 raft_consensus.cc:515] T 66f75124215f4ca6bef831e174a792d8 P cb82cc7bd6db46d980c75237a1402860 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:51.911196 20429 leader_election.cc:290] T 66f75124215f4ca6bef831e174a792d8 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:20:51.912128 20440 leader_election.cc:290] T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:51.913251 20448 raft_consensus.cc:493] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:51.914016 20236 raft_consensus.cc:493] T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:51.914451 20448 raft_consensus.cc:515] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:51.914758 20236 raft_consensus.cc:515] T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:20:51.913146 16116 connection.cc:256] RPC call timeout handler was delayed by -1.2560077200000705s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16710000000000003s for a response
W20260709 13:20:51.918051 16116 connection.cc:256] RPC call timeout handler was delayed by -1.0715077199999996s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1876s for a response
I20260709 13:20:51.919215 20401 raft_consensus.cc:515] T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:51.922601 20401 leader_election.cc:290] T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:20:51.923661 20402 raft_consensus.cc:515] T cdbd78acb4cf46298e0de8ba0d130e19 P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:51.928131 20021 raft_consensus.cc:493] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:51.929311 20021 raft_consensus.cc:515] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:51.926965 20402 leader_election.cc:290] T cdbd78acb4cf46298e0de8ba0d130e19 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
W20260709 13:20:51.936020 16115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 222142us
I20260709 13:20:51.847108 20447 raft_consensus.cc:515] T 8638ee363273426db1393dd81e5c7b72 P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:20:52.008909 16788 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:51.935364 20426 leader_election.cc:290] T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:51.930109 16037 maintenance_manager.cc:643] P 667a34b27a39474ebefc1af3cf5b4271: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.003s	user 0.000s	sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":575,"lbm_reads_lt_1ms":4}
I20260709 13:20:51.939956 20448 leader_election.cc:290] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:51.895252 20450 raft_consensus.cc:515] T 8158bbd17b994d2eb96f560bd0f1b88d P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:52.037110 20530 raft_consensus.cc:493] T 5a96373bb0e14dce8a266b2fc6c58507 P 647a343a542d4a139fe106c19095129b [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:20:52.038134 20530 raft_consensus.cc:515] T 5a96373bb0e14dce8a266b2fc6c58507 P 647a343a542d4a139fe106c19095129b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:52.040695 20447 leader_election.cc:290] T 8638ee363273426db1393dd81e5c7b72 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:51.951916 20402 raft_consensus.cc:493] T 3884f864a96d428785895e5463e5728f P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:20:52.047833 20531 raft_consensus.cc:2749] T 1df93779209748b9860f6d893dd3de53 P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:20:51.982332 20401 raft_consensus.cc:3060] T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:20:52.049106 20531 raft_consensus.cc:2749] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:20:51.898993 20392 raft_consensus.cc:493] T 23f82224f4564fea93572e8206bad1fe P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
W20260709 13:20:52.051697 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 85167.8us
I20260709 13:20:52.053902 20529 raft_consensus.cc:515] T 9c61a39fd4f84424b129c1a6f9ba404d P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:20:52.055974 16791 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:52.056901 20529 leader_election.cc:290] T 9c61a39fd4f84424b129c1a6f9ba404d P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:52.059010 16222 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57600 (request call id 1425) took 3794 ms. Trace:
I20260709 13:20:52.059494 16222 rpcz_store.cc:276] 0709 13:20:48.264147 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:48.264302 (+   155us) service_pool.cc:224] Handling call
0709 13:20:48.280228 (+ 15926us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:20:49.139706 (+859478us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:52.058372 (+2918666us) spinlock_profiling.cc:243] Waited 853 ms on lock 0x7b54002981d4. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:20:52.058795 (+   423us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1961032064}
I20260709 13:20:52.057018 20450 leader_election.cc:290] T 8158bbd17b994d2eb96f560bd0f1b88d P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:20:52.047703 20402 raft_consensus.cc:515] T 3884f864a96d428785895e5463e5728f P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:52.059146 20021 leader_election.cc:290] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:52.050453 20392 raft_consensus.cc:515] T 23f82224f4564fea93572e8206bad1fe P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:52.069217 16151 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46224 (request call id 639) took 3234 ms. Trace:
W20260709 13:20:52.070849 16791 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:52.072813 20530 leader_election.cc:290] T 5a96373bb0e14dce8a266b2fc6c58507 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:52.073817 20402 leader_election.cc:290] T 3884f864a96d428785895e5463e5728f P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:52.069821 16151 rpcz_store.cc:276] 0709 13:20:48.832994 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:51.681613 (+2848619us) service_pool.cc:224] Handling call
0709 13:20:51.682174 (+   561us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:51.687247 (+  5073us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:20:51.934340 (+247093us) log.cc:844] Serialized 14 byte log entry
0709 13:20:52.066075 (+131735us) log.cc:844] Serialized 14 byte log entry
0709 13:20:52.066463 (+   388us) log.cc:844] Serialized 14 byte log entry
0709 13:20:52.066681 (+   218us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:20:52.066738 (+    57us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:20:52.066797 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:52.066887 (+    90us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:52.067048 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":377397,"prepare.run_cpu_time_us":37,"prepare.run_wall_time_us":38,"raft.queue_time_us":93665,"raft.run_cpu_time_us":220,"raft.run_wall_time_us":220,"thread_start_us":280662,"threads_started":2}
I20260709 13:20:52.086967 16292 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48684 (request call id 2500) took 2936 ms. Trace:
I20260709 13:20:52.088739 16292 rpcz_store.cc:276] 0709 13:20:49.150644 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:49.150873 (+   229us) service_pool.cc:224] Handling call
0709 13:20:49.211543 (+ 60670us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:49.211846 (+   303us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:20:51.788025 (+2576179us) log.cc:844] Serialized 14 byte log entry
0709 13:20:52.076259 (+288234us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:20:52.080366 (+  4107us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:20:52.084898 (+  4532us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:52.086502 (+  1604us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:52.086737 (+   235us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":42697,"prepare.queue_time_us":29024,"prepare.run_cpu_time_us":74,"prepare.run_wall_time_us":75,"raft.queue_time_us":32434,"raft.run_cpu_time_us":219,"raft.run_wall_time_us":8049,"thread_start_us":230372,"threads_started":1,"wal-append.queue_time_us":286316}
I20260709 13:20:52.100272 16151 raft_consensus.cc:1240] T 8158bbd17b994d2eb96f560bd0f1b88d P 647a343a542d4a139fe106c19095129b [term 10 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 9. Current term is 10. Ops: [9.7-9.7]
I20260709 13:20:52.102579 16151 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40844 (request call id 2350) took 3247 ms. Trace:
I20260709 13:20:52.103530 16151 rpcz_store.cc:276] 0709 13:20:48.855619 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:52.084993 (+3229374us) service_pool.cc:224] Handling call
0709 13:20:52.099335 (+ 14342us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:20:52.101855 (+  2520us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:52.102438 (+   583us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:20:52.107442 16151 raft_consensus.cc:1275] T 746aabf856ab401c9ef5e859371e7b67 P 647a343a542d4a139fe106c19095129b [term 10 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:20:52.108738 16151 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40844 (request call id 2351) took 3186 ms. Trace:
I20260709 13:20:52.109603 16151 rpcz_store.cc:276] 0709 13:20:48.922037 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:52.105648 (+3183611us) service_pool.cc:224] Handling call
0709 13:20:52.106469 (+   821us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:52.108208 (+  1739us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:52.108584 (+   376us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:20:52.112803 16151 raft_consensus.cc:1217] T 8cc959fb7f704fba9009c0595467d074 P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Deduplicated request from leader. Original: 6.5->[8.6-11.8]   Dedup: 8.6->[9.7-11.8]
I20260709 13:20:52.113833 16151 pending_rounds.cc:85] T 8cc959fb7f704fba9009c0595467d074 P 647a343a542d4a139fe106c19095129b: Aborting all ops after (but not including) 6
W20260709 13:20:52.119964 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 448274us
I20260709 13:20:52.114440 16151 pending_rounds.cc:107] T 8cc959fb7f704fba9009c0595467d074 P 647a343a542d4a139fe106c19095129b: Aborting uncommitted NO_OP operation due to leader change: 10.7
I20260709 13:20:52.126336 16151 raft_consensus.cc:2889] T 8cc959fb7f704fba9009c0595467d074 P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:20:52.108997 16223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 12 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:20:52.128804 20392 leader_election.cc:290] T 23f82224f4564fea93572e8206bad1fe P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:52.129951 16223 raft_consensus.cc:2393] T 19baa7bc367844c288bca8f81439c413 P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 in current term 12: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
W20260709 13:20:52.139782 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 390851us
I20260709 13:20:52.145501 16146 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40844 (request call id 2338) took 3307 ms. Trace:
I20260709 13:20:52.145961 16146 rpcz_store.cc:276] 0709 13:20:48.838104 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:51.821498 (+2983394us) service_pool.cc:224] Handling call
0709 13:20:51.822092 (+   594us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:51.822583 (+   491us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:20:52.064562 (+241979us) log.cc:844] Serialized 14 byte log entry
0709 13:20:52.145147 (+ 80585us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:20:52.145201 (+    54us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:20:52.145245 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:52.145314 (+    69us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:52.145449 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":251553,"prepare.run_cpu_time_us":44,"prepare.run_wall_time_us":44,"raft.queue_time_us":2017,"thread_start_us":298164,"threads_started":2}
I20260709 13:20:52.152853 16146 pending_rounds.cc:85] T 84f2da98864b4425bf7c0055951fdd4b P 647a343a542d4a139fe106c19095129b: Aborting all ops after (but not including) 3
I20260709 13:20:52.153272 16146 pending_rounds.cc:107] T 84f2da98864b4425bf7c0055951fdd4b P 647a343a542d4a139fe106c19095129b: Aborting uncommitted NO_OP operation due to leader change: 6.4
I20260709 13:20:52.154220 16146 raft_consensus.cc:2889] T 84f2da98864b4425bf7c0055951fdd4b P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:20:52.156668 16150 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40844 (request call id 2337) took 3320 ms. Trace:
I20260709 13:20:52.157156 16150 rpcz_store.cc:276] 0709 13:20:48.836163 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:51.778967 (+2942804us) service_pool.cc:224] Handling call
0709 13:20:51.785615 (+  6648us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:51.905754 (+120139us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:52.154922 (+249168us) spinlock_profiling.cc:243] Waited 119 ms on lock 0x7b54000e9054. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:20:52.156577 (+  1655us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":274578816}
I20260709 13:20:52.164495 16150 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46224 (request call id 640) took 3154 ms. Trace:
I20260709 13:20:52.168570 16148 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40844 (request call id 2349) took 3313 ms. Trace:
I20260709 13:20:52.169111 16148 rpcz_store.cc:276] 0709 13:20:48.855289 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:51.888114 (+3032825us) service_pool.cc:224] Handling call
0709 13:20:52.024569 (+136455us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:20:52.059726 (+ 35157us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:52.059753 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:20:52.060284 (+   531us) log.cc:844] Serialized 26 byte log entry
0709 13:20:52.079403 (+ 19119us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:20:52.079440 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:52.079506 (+    66us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:20:52.168301 (+ 88795us) raft_consensus.cc:1685] finished
0709 13:20:52.168326 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:52.168500 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":18695,"threads_started":1,"wal-append.queue_time_us":83575}
I20260709 13:20:52.175546 16148 raft_consensus.cc:1275] T af57996a8be84d398f0f023e8d08ef77 P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 12 index: 9. (term mismatch)
I20260709 13:20:52.176064 16148 pending_rounds.cc:85] T af57996a8be84d398f0f023e8d08ef77 P 647a343a542d4a139fe106c19095129b: Aborting all ops after (but not including) 8
I20260709 13:20:52.176535 16148 pending_rounds.cc:107] T af57996a8be84d398f0f023e8d08ef77 P 647a343a542d4a139fe106c19095129b: Aborting uncommitted NO_OP operation due to leader change: 11.9
I20260709 13:20:52.176853 16148 raft_consensus.cc:2889] T af57996a8be84d398f0f023e8d08ef77 P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:20:52.177740 16148 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40844 (request call id 2354) took 3160 ms. Trace:
I20260709 13:20:52.178117 16148 rpcz_store.cc:276] 0709 13:20:49.017818 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:52.173931 (+3156113us) service_pool.cc:224] Handling call
0709 13:20:52.174668 (+   737us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:52.177451 (+  2783us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:52.177674 (+   223us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:20:52.179719 16114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:52.165004 16150 rpcz_store.cc:276] 0709 13:20:49.008707 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:52.158246 (+3149539us) service_pool.cc:224] Handling call
0709 13:20:52.158766 (+   520us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:52.159155 (+   389us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:52.162643 (+  3488us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:20:52.162712 (+    69us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:20:52.162768 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:52.162857 (+    89us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:52.163031 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:20:52.194399 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 45896.1us
W20260709 13:20:52.204139 16114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:52.205331 20021 raft_consensus.cc:493] T 08c74490c40243a88e62d1760b5cd814 P 647a343a542d4a139fe106c19095129b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:20:52.204345 16190 connection.cc:256] RPC call timeout handler was delayed by -1.2808051819999946s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16060000000000002s for a response
I20260709 13:20:52.206068 20021 raft_consensus.cc:515] T 08c74490c40243a88e62d1760b5cd814 P 647a343a542d4a139fe106c19095129b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:52.208020 20504 raft_consensus.cc:493] T 829b4ad8c5a54eababeea1181b960d4b P 647a343a542d4a139fe106c19095129b [term 10 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:20:52.209528 20021 leader_election.cc:290] T 08c74490c40243a88e62d1760b5cd814 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:52.209192 20504 raft_consensus.cc:515] T 829b4ad8c5a54eababeea1181b960d4b P 647a343a542d4a139fe106c19095129b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:52.212353 20504 leader_election.cc:290] T 829b4ad8c5a54eababeea1181b960d4b P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
W20260709 13:20:52.212937 16190 connection.cc:256] RPC call timeout handler was delayed by -1.0315051820000463s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18830000000000002s for a response
I20260709 13:20:52.217367 20021 raft_consensus.cc:493] T 8158bbd17b994d2eb96f560bd0f1b88d P 647a343a542d4a139fe106c19095129b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:52.224107 20021 raft_consensus.cc:515] T 8158bbd17b994d2eb96f560bd0f1b88d P 647a343a542d4a139fe106c19095129b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:52.227321 20502 raft_consensus.cc:493] T d16dfad0c3b540e5b4b86492ddf87208 P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:52.223316 16150 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46224 (request call id 642) took 3153 ms. Trace:
I20260709 13:20:52.227855 16294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806d7690c2b644959152074bb2cab0a7" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:20:52.228127 16150 rpcz_store.cc:276] 0709 13:20:49.065719 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:52.180912 (+3115193us) service_pool.cc:224] Handling call
0709 13:20:52.201087 (+ 20175us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:52.217566 (+ 16479us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:52.217595 (+    29us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:20:52.219112 (+  1517us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:20:52.219153 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:52.219237 (+    84us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:52.219374 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:20:52.228211 20502 raft_consensus.cc:515] T d16dfad0c3b540e5b4b86492ddf87208 P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:52.235808 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87ac0d1f7e8a419f9bcbc04aee0e067e" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:20:52.231802 16292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a878999c0024d2abe517484180290fc" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:20:52.233850 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fffab26427e24d398e2695aad40ca134" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 10 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:52.234763 16294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e65676b21b748728ad39668cfe722ec" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:52.237686 16292 raft_consensus.cc:3060] T 8a878999c0024d2abe517484180290fc P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:20:52.238010 16299 raft_consensus.cc:2410] T fffab26427e24d398e2695aad40ca134 P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b for term 10 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
I20260709 13:20:52.216264 20401 raft_consensus.cc:2749] T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:20:52.227257 20021 leader_election.cc:290] T 8158bbd17b994d2eb96f560bd0f1b88d P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:52.259948 20502 leader_election.cc:290] T d16dfad0c3b540e5b4b86492ddf87208 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:52.263655 16294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6f07842d3aa47858df5665874c86f3a" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
W20260709 13:20:52.272910 16266 connection.cc:256] RPC call timeout handler was delayed by -2.5079815820001841s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1878s for a response
W20260709 13:20:52.273379 16266 connection.cc:256] RPC call timeout handler was delayed by -2.4665815820001171s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19240000000000002s for a response
I20260709 13:20:52.215090 20504 raft_consensus.cc:493] T 18ad7e3af5504571a69d10c00750a01c P 647a343a542d4a139fe106c19095129b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:52.273633 16222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806d7690c2b644959152074bb2cab0a7" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:20:52.273919 16146 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40844 (request call id 2353) took 3265 ms. Trace:
I20260709 13:20:52.274559 16222 raft_consensus.cc:3060] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:20:52.274380 16146 rpcz_store.cc:276] 0709 13:20:49.008621 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:52.152070 (+3143449us) service_pool.cc:224] Handling call
0709 13:20:52.152556 (+   486us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:20:52.159656 (+  7100us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:52.159688 (+    32us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:20:52.160310 (+   622us) log.cc:844] Serialized 52 byte log entry
0709 13:20:52.187839 (+ 27529us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:20:52.187868 (+    29us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:52.187919 (+    51us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:20:52.273663 (+ 85744us) raft_consensus.cc:1685] finished
0709 13:20:52.273681 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:52.273845 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1724,"raft.run_cpu_time_us":95,"raft.run_wall_time_us":94,"thread_start_us":27058,"threads_started":1,"wal-append.queue_time_us":108150}
I20260709 13:20:52.277613 16146 raft_consensus.cc:1275] T 9c61a39fd4f84424b129c1a6f9ba404d P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:20:52.278196 16146 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40844 (request call id 2357) took 3206 ms. Trace:
I20260709 13:20:52.278563 16146 rpcz_store.cc:276] 0709 13:20:49.072125 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:52.276781 (+3204656us) service_pool.cc:224] Handling call
0709 13:20:52.277216 (+   435us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:52.277934 (+   718us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:52.278132 (+   198us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:20:52.297783 16295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4f65cdf00b748bb963ee6e81ef43b79" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
W20260709 13:20:52.372651 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 155999us
W20260709 13:20:52.445504 16788 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:20:52.618110 16114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:20:52.618227 16791 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:52.360743 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e9b2bf7011f4ae688a79d7bb5a447b4" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:20:52.611912 16294 raft_consensus.cc:2468] T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 12.
W20260709 13:20:52.654508 16791 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:52.629341 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cc959fb7f704fba9009c0595467d074" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:52.657832 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4dc6b0541ca431ebc757beac00521f1" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:52.658754 16299 raft_consensus.cc:2468] T e4dc6b0541ca431ebc757beac00521f1 P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 10.
I20260709 13:20:52.656677 16295 raft_consensus.cc:2468] T c4f65cdf00b748bb963ee6e81ef43b79 P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 12.
W20260709 13:20:52.659711 16172 debug-util.cc:398] Leaking SignalData structure 0x7b08002cfcc0 after lost signal to thread 15046
I20260709 13:20:52.673962 16293 raft_consensus.cc:2410] T 8cc959fb7f704fba9009c0595467d074 P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b for term 12 because replica has last-logged OpId of term: 11 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 7.
I20260709 13:20:52.674423 16292 raft_consensus.cc:2468] T 8a878999c0024d2abe517484180290fc P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 10.
I20260709 13:20:52.274616 20504 raft_consensus.cc:515] T 18ad7e3af5504571a69d10c00750a01c P 647a343a542d4a139fe106c19095129b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } }
W20260709 13:20:52.676479 16172 debug-util.cc:398] Leaking SignalData structure 0x7b080010a9e0 after lost signal to thread 15049
I20260709 13:20:52.679095 16150 raft_consensus.cc:3060] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:20:52.659734 20495 raft_consensus.cc:493] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:52.686592 16221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87ac0d1f7e8a419f9bcbc04aee0e067e" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:20:52.687453 16221 raft_consensus.cc:3060] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:20:52.707487 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0eab25ce4c64e018a041beeda2867db" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:52.708963 16221 raft_consensus.cc:2410] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b for term 12 because replica has last-logged OpId of term: 11 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 8.
I20260709 13:20:52.710440 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02838c86531d4f98bf58d1c22b8baf77" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:52.710964 16222 raft_consensus.cc:2410] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b for term 11 because replica has last-logged OpId of term: 10 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 7.
I20260709 13:20:52.709332 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434bff3082d141838068f323bdb0d1ed" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:20:52.714059 16218 raft_consensus.cc:1240] T 746aabf856ab401c9ef5e859371e7b67 P 1ca2725d290e47d68b1a4b609c9816ac [term 10 LEADER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 9. Current term is 10. Ops: []
I20260709 13:20:52.716234 16299 raft_consensus.cc:2393] T 02838c86531d4f98bf58d1c22b8baf77 P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 11: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
I20260709 13:20:52.717443 16293 raft_consensus.cc:3060] T 434bff3082d141838068f323bdb0d1ed P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:20:52.720669 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a878999c0024d2abe517484180290fc" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
W20260709 13:20:52.728861 16116 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:52.733862 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3884f864a96d428785895e5463e5728f" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:52.734526 16299 raft_consensus.cc:2468] T 3884f864a96d428785895e5463e5728f P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 7.
I20260709 13:20:52.737156 16217 raft_consensus.cc:2393] T c0eab25ce4c64e018a041beeda2867db P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 11: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
I20260709 13:20:52.714957 20495 raft_consensus.cc:515] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:52.759688 16218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746aabf856ab401c9ef5e859371e7b67" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:52.789940 20495 leader_election.cc:290] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:52.800307 16221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4f65cdf00b748bb963ee6e81ef43b79" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:52.800506 16150 raft_consensus.cc:2468] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 9.
I20260709 13:20:52.803403 16219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6f07842d3aa47858df5665874c86f3a" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:52.803542 16222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4dc6b0541ca431ebc757beac00521f1" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:52.782956 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fffab26427e24d398e2695aad40ca134" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 10 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
W20260709 13:20:52.803560 16266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 70535.5us
I20260709 13:20:52.759716 20504 leader_election.cc:290] T 18ad7e3af5504571a69d10c00750a01c P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:20:52.806527 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cc959fb7f704fba9009c0595467d074" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:52.807266 16221 raft_consensus.cc:2393] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 13: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
I20260709 13:20:52.805629 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:52.821468 16218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f2da98864b4425bf7c0055951fdd4b" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:52.807368 20495 raft_consensus.cc:493] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:52.824836 16299 raft_consensus.cc:2468] T ecc1e363230043fe9fef390f0b979f20 P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 10.
I20260709 13:20:52.831723 16265 leader_election.cc:304] T 8cc959fb7f704fba9009c0595467d074 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:20:52.833238 20495 raft_consensus.cc:515] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:52.822693 20518 raft_consensus.cc:493] T 18ad7e3af5504571a69d10c00750a01c P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:52.835851 20401 raft_consensus.cc:2749] T 8cc959fb7f704fba9009c0595467d074 P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:20:52.838804 16823 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49892 (request call id 614) took 3876 ms. Trace:
I20260709 13:20:52.840016 16823 rpcz_store.cc:276] 0709 13:20:48.855048 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:48.855198 (+   150us) service_pool.cc:224] Handling call
0709 13:20:48.885458 (+ 30260us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:49.301240 (+415782us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:52.703522 (+3402282us) spinlock_profiling.cc:243] Waited 415 ms on lock 0x7b5400771354. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:20:52.731385 (+ 27863us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":954426112}
I20260709 13:20:52.840835 20518 raft_consensus.cc:515] T 18ad7e3af5504571a69d10c00750a01c P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } }
I20260709 13:20:52.844281 20518 leader_election.cc:290] T 18ad7e3af5504571a69d10c00750a01c P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:20:52.845840 20231 raft_consensus.cc:3060] T 7c2095a5f4ad49408dbc7f69c606120d P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:20:52.841966 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 13 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
W20260709 13:20:52.857882 16788 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:52.861022 20472 raft_consensus.cc:493] T 7c2095a5f4ad49408dbc7f69c606120d P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:52.861419 20460 consensus_queue.cc:1048] T c0eab25ce4c64e018a041beeda2867db P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:20:52.861660 20472 raft_consensus.cc:515] T 7c2095a5f4ad49408dbc7f69c606120d P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:52.863080 20460 consensus_queue.cc:1059] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 90f3da57796b488c9087a2f523adf045 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:20:52.863842 20460 consensus_queue.cc:1243] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 90f3da57796b488c9087a2f523adf045 [LEADER]: Peer 647a343a542d4a139fe106c19095129b log is divergent from this leader: its last log entry 10.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 13:20:52.869423 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68a6838d82ef4cac83507f906139d90f" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:52.869995 16299 raft_consensus.cc:2468] T 68a6838d82ef4cac83507f906139d90f P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 11.
I20260709 13:20:52.872612 16293 raft_consensus.cc:2468] T 434bff3082d141838068f323bdb0d1ed P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 10.
I20260709 13:20:52.871091 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a96373bb0e14dce8a266b2fc6c58507" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:52.873519 16299 raft_consensus.cc:2468] T 5a96373bb0e14dce8a266b2fc6c58507 P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 10.
I20260709 13:20:52.874301 20495 leader_election.cc:290] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:20:52.874349 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0eab25ce4c64e018a041beeda2867db" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:20:52.876408 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31969392ccdb43b491911a319e8bc7e1" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:52.877091 16299 raft_consensus.cc:2468] T 31969392ccdb43b491911a319e8bc7e1 P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 10.
I20260709 13:20:52.878562 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02838c86531d4f98bf58d1c22b8baf77" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:52.879446 16299 raft_consensus.cc:2393] T 02838c86531d4f98bf58d1c22b8baf77 P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 11: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
I20260709 13:20:52.880285 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f82224f4564fea93572e8206bad1fe" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:52.880900 16299 raft_consensus.cc:2468] T 23f82224f4564fea93572e8206bad1fe P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 8.
I20260709 13:20:52.882134 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4f65cdf00b748bb963ee6e81ef43b79" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:20:52.882813 16299 raft_consensus.cc:3060] T c4f65cdf00b748bb963ee6e81ef43b79 P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:20:52.882200 16265 connection.cc:256] RPC call timeout handler was delayed by -1.0417986200000087s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.3199s for a response
W20260709 13:20:52.884936 16265 leader_election.cc:336] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Timed out: RequestConsensusVote RPC to 127.14.177.129:36741 timed out after 2.504s (SENT)
I20260709 13:20:52.875082 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a959dbecc2648eb85d1cc550f5a3245" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:52.887775 16293 raft_consensus.cc:2410] T 5a959dbecc2648eb85d1cc550f5a3245 P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b for term 10 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 13:20:52.887840 16144 raft_consensus.cc:1275] T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 13:20:52.889065 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5ad4eb281564936a59f41e387a47156" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:20:52.891104 16299 raft_consensus.cc:2468] T c4f65cdf00b748bb963ee6e81ef43b79 P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 13.
I20260709 13:20:52.891570 16293 raft_consensus.cc:2368] T e5ad4eb281564936a59f41e387a47156 P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac for earlier term 11. Current term is 12.
I20260709 13:20:52.891484 16144 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46224 (request call id 644) took 3748 ms. Trace:
I20260709 13:20:52.892185 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4f65cdf00b748bb963ee6e81ef43b79" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:20:52.892482 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0eab25ce4c64e018a041beeda2867db" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:52.892040 16144 rpcz_store.cc:276] 0709 13:20:49.142951 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:52.886921 (+3743970us) service_pool.cc:224] Handling call
0709 13:20:52.887387 (+   466us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:52.891207 (+  3820us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:52.891420 (+   213us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:20:52.892814 16299 raft_consensus.cc:2376] T c4f65cdf00b748bb963ee6e81ef43b79 P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 13. Re-sending same reply.
I20260709 13:20:52.893622 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9944d5e4b8c4109af9955c4f08c9095" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:20:52.893983 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9944d5e4b8c4109af9955c4f08c9095" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:20:52.894443 16293 raft_consensus.cc:2368] T c9944d5e4b8c4109af9955c4f08c9095 P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac for earlier term 11. Current term is 12.
W20260709 13:20:52.895094 16790 leader_election.cc:336] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 9 election: RPC error from VoteRequest() call to peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943): Timed out: RequestConsensusVote RPC to 127.14.177.130:35943 timed out after 2.161s (SENT)
I20260709 13:20:52.895215 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d16dfad0c3b540e5b4b86492ddf87208" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 12 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:52.895833 16790 leader_election.cc:304] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:20:52.896343 16299 raft_consensus.cc:2410] T d16dfad0c3b540e5b4b86492ddf87208 P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b for term 12 because replica has last-logged OpId of term: 10 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 13:20:52.895253 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdbd78acb4cf46298e0de8ba0d130e19" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:20:52.897233 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f2da98864b4425bf7c0055951fdd4b" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:52.897583 16293 raft_consensus.cc:2376] T cdbd78acb4cf46298e0de8ba0d130e19 P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 12. Re-sending same reply.
I20260709 13:20:52.898173 16299 raft_consensus.cc:2410] T 84f2da98864b4425bf7c0055951fdd4b P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b for term 10 because replica has last-logged OpId of term: 9 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 13:20:52.898744 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c61a39fd4f84424b129c1a6f9ba404d" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 12 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:52.898996 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a378d7f796f94615b3b347119a9880fd" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:52.899811 16299 raft_consensus.cc:2393] T a378d7f796f94615b3b347119a9880fd P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 12: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
I20260709 13:20:52.899852 16293 raft_consensus.cc:2410] T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b for term 12 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
I20260709 13:20:52.900550 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8158bbd17b994d2eb96f560bd0f1b88d" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:52.903105 16299 raft_consensus.cc:2410] T 8158bbd17b994d2eb96f560bd0f1b88d P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b for term 11 because replica has last-logged OpId of term: 9 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 6.
I20260709 13:20:52.903203 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 12 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:20:52.900419 20268 raft_consensus.cc:2749] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:20:52.903955 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c74490c40243a88e62d1760b5cd814" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 11 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:52.905750 16299 raft_consensus.cc:2410] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b for term 11 because replica has last-logged OpId of term: 9 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 6.
W20260709 13:20:52.907043 16790 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:52.907914 20268 raft_consensus.cc:493] T b87147731a68400e8cce9b879daa7c5d P cb82cc7bd6db46d980c75237a1402860 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:52.908465 20268 raft_consensus.cc:515] T b87147731a68400e8cce9b879daa7c5d P cb82cc7bd6db46d980c75237a1402860 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:52.903913 20236 leader_election.cc:290] T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:52.910522 20268 leader_election.cc:290] T b87147731a68400e8cce9b879daa7c5d P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
W20260709 13:20:52.912504 16790 connection.cc:256] RPC call timeout handler was delayed by -2.6305181430000175s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1757s for a response
W20260709 13:20:52.913540 16790 connection.cc:256] RPC call timeout handler was delayed by -2.4684811520000949s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1513s for a response
W20260709 13:20:52.914013 16790 connection.cc:256] RPC call timeout handler was delayed by -2.3514811520001331s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1643s for a response
I20260709 13:20:52.914312 19861 raft_consensus.cc:493] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:52.914870 19861 raft_consensus.cc:515] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:52.917251 19861 leader_election.cc:290] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:52.917776 19861 raft_consensus.cc:493] T 18ad7e3af5504571a69d10c00750a01c P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:52.918354 19861 raft_consensus.cc:515] T 18ad7e3af5504571a69d10c00750a01c P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } }
I20260709 13:20:52.928267 19861 leader_election.cc:290] T 18ad7e3af5504571a69d10c00750a01c P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:52.928890 19861 raft_consensus.cc:493] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:20:52.931483 19861 raft_consensus.cc:515] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:52.934020 19861 leader_election.cc:290] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:52.935487 19861 raft_consensus.cc:493] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:20:52.914333 16790 connection.cc:256] RPC call timeout handler was delayed by -2.2992811520000487s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17010000000000003s for a response
W20260709 13:20:52.936081 16790 connection.cc:256] RPC call timeout handler was delayed by -2.1733823519998623s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1795s for a response
W20260709 13:20:52.936470 16790 connection.cc:256] RPC call timeout handler was delayed by -2.0462753550000343s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17020000000000002s for a response
W20260709 13:20:52.936741 16790 connection.cc:256] RPC call timeout handler was delayed by -1.8600811520000207s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2189s for a response
W20260709 13:20:52.937028 16790 connection.cc:256] RPC call timeout handler was delayed by -1.7952920550001181s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21760000000000002s for a response
W20260709 13:20:52.937235 16790 connection.cc:256] RPC call timeout handler was delayed by -1.7952920550001181s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21760000000000002s for a response
W20260709 13:20:52.937479 16790 connection.cc:256] RPC call timeout handler was delayed by -1.582775354999967s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2217s for a response
W20260709 13:20:52.937703 16790 connection.cc:256] RPC call timeout handler was delayed by -1.5775847330000943s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20779999999999998s for a response
W20260709 13:20:52.937942 16790 connection.cc:256] RPC call timeout handler was delayed by -1.453000202000112s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2183s for a response
W20260709 13:20:52.938164 16790 connection.cc:256] RPC call timeout handler was delayed by -1.4031320360002155s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2759s for a response
W20260709 13:20:52.938439 16790 connection.cc:256] RPC call timeout handler was delayed by -1.3824150270002065s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2759s for a response
W20260709 13:20:52.938669 16790 connection.cc:256] RPC call timeout handler was delayed by -1.3668946369998594s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.24209999999999998s for a response
I20260709 13:20:52.936084 19861 raft_consensus.cc:515] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:52.941018 16144 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46224 (request call id 645) took 3769 ms. Trace:
I20260709 13:20:52.941447 16144 rpcz_store.cc:276] 0709 13:20:49.171210 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:52.893722 (+3722512us) service_pool.cc:224] Handling call
0709 13:20:52.894192 (+   470us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:52.894457 (+   265us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:20:52.920308 (+ 25851us) log.cc:844] Serialized 14 byte log entry
0709 13:20:52.940699 (+ 20391us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:20:52.940761 (+    62us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:20:52.940812 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:52.940870 (+    58us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:52.940968 (+    98us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":27487,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":40,"raft.queue_time_us":106,"raft.run_cpu_time_us":115,"raft.run_wall_time_us":116,"thread_start_us":45270,"threads_started":2,"wal-append.queue_time_us":21265}
I20260709 13:20:52.944132 19861 leader_election.cc:290] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:52.944813 19861 raft_consensus.cc:493] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:52.945429 19861 raft_consensus.cc:515] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:20:52.946324 16115 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:52.947254 16790 leader_election.cc:304] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:20:52.948371 19861 leader_election.cc:290] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:52.948498 20268 raft_consensus.cc:2804] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:20:52.948956 20268 raft_consensus.cc:493] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:52.948989 19861 raft_consensus.cc:493] T d16dfad0c3b540e5b4b86492ddf87208 P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:52.949461 20268 raft_consensus.cc:3060] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:20:52.949726 19861 raft_consensus.cc:515] T d16dfad0c3b540e5b4b86492ddf87208 P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:52.951586 16294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18ad7e3af5504571a69d10c00750a01c" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:52.952102 19861 leader_election.cc:290] T d16dfad0c3b540e5b4b86492ddf87208 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:52.952373 16294 raft_consensus.cc:2468] T 18ad7e3af5504571a69d10c00750a01c P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 10.
I20260709 13:20:52.956388 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8158bbd17b994d2eb96f560bd0f1b88d" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:52.956841 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74ca8140f35c4ae29e8a664f6bdfc829" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:53.003057 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d16dfad0c3b540e5b4b86492ddf87208" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:52.957072 16293 raft_consensus.cc:2468] T 8158bbd17b994d2eb96f560bd0f1b88d P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 10.
I20260709 13:20:53.004019 16299 raft_consensus.cc:2468] T d16dfad0c3b540e5b4b86492ddf87208 P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 11.
I20260709 13:20:52.959055 20268 raft_consensus.cc:515] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:20:53.095857 16321 debug-util.cc:398] Leaking SignalData structure 0x7b08002dede0 after lost signal to thread 16083
I20260709 13:20:52.959739 16790 leader_election.cc:304] T ee9ed67006144797b1de6ea8c2013a68 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860; no voters: 
W20260709 13:20:53.097371 16321 debug-util.cc:398] Leaking SignalData structure 0x7b08002de1c0 after lost signal to thread 16175
W20260709 13:20:53.097747 16321 debug-util.cc:398] Leaking SignalData structure 0x7b08002de1e0 after lost signal to thread 16176
W20260709 13:20:53.098997 16321 debug-util.cc:398] Leaking SignalData structure 0x7b08000d14e0 after lost signal to thread 16250
W20260709 13:20:53.099323 16321 debug-util.cc:398] Leaking SignalData structure 0x7b08000d14c0 after lost signal to thread 16251
I20260709 13:20:53.106185 16294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada5a004ba7c46819051e3ccec71ffdb" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
W20260709 13:20:53.106858 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 147393us
I20260709 13:20:53.106364 20495 raft_consensus.cc:2804] T ee9ed67006144797b1de6ea8c2013a68 P cb82cc7bd6db46d980c75237a1402860 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:20:53.108582 20268 leader_election.cc:290] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 15 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
W20260709 13:20:53.108878 16265 leader_election.cc:336] T d2ab0ac8347641e6955e9bce7d90c1a2 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 11 election: RPC error from VoteRequest() call to peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943): Timed out: RequestConsensusVote RPC to 127.14.177.130:35943 timed out after 1.878s (SENT)
I20260709 13:20:53.108193 16822 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49892 (request call id 616) took 4242 ms. Trace:
I20260709 13:20:53.127722 16822 rpcz_store.cc:276] 0709 13:20:48.865843 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:53.106909 (+4241066us) service_pool.cc:224] Handling call
0709 13:20:53.107393 (+   484us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:53.107669 (+   276us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:20:53.107703 (+    34us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:20:53.107781 (+    78us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:20:53.107833 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:53.107895 (+    62us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:53.108010 (+   115us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:20:53.139567 16115 leader_election.cc:304] T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:20:53.143950 20236 raft_consensus.cc:2804] T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:20:53.144426 20236 raft_consensus.cc:493] T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b [term 10 FOLLOWER]: Starting leader election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:20:53.144956 20236 raft_consensus.cc:3060] T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:20:53.147634 16791 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:53.144361 16115 leader_election.cc:304] T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:20:53.156348 20434 raft_consensus.cc:2804] T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:20:53.156822 20434 raft_consensus.cc:493] T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:53.157191 20434 raft_consensus.cc:3060] T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:20:53.167773 16115 leader_election.cc:304] T 8a878999c0024d2abe517484180290fc P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
W20260709 13:20:53.176786 16172 debug-util.cc:398] Leaking SignalData structure 0x7b0800137f20 after lost signal to thread 16083
W20260709 13:20:53.179944 16265 leader_election.cc:336] T 8158bbd17b994d2eb96f560bd0f1b88d P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 10 election: RPC error from VoteRequest() call to peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943): Timed out: RequestConsensusVote RPC to 127.14.177.130:35943 timed out after 1.953s (SENT)
W20260709 13:20:53.207736 16265 leader_election.cc:336] T 8158bbd17b994d2eb96f560bd0f1b88d P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 10 election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Timed out: RequestConsensusVote RPC to 127.14.177.129:36741 timed out after 1.953s (SENT)
I20260709 13:20:53.208315 16265 leader_election.cc:304] T 8158bbd17b994d2eb96f560bd0f1b88d P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:20:53.209805 20398 raft_consensus.cc:2749] T 8158bbd17b994d2eb96f560bd0f1b88d P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
W20260709 13:20:53.210361 16265 leader_election.cc:336] T 68a6838d82ef4cac83507f906139d90f P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Timed out: RequestConsensusVote RPC to 127.14.177.129:36741 timed out after 3.199s (SENT)
W20260709 13:20:53.189469 20488 log.cc:927] Time spent T c2f5f89186204242bf377f7c42b2e2da P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 4.405s	user 0.029s	sys 0.022s
I20260709 13:20:53.191677 20495 raft_consensus.cc:697] T ee9ed67006144797b1de6ea8c2013a68 P cb82cc7bd6db46d980c75237a1402860 [term 9 LEADER]: Becoming Leader. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:20:53.213799 20495 consensus_queue.cc:237] T ee9ed67006144797b1de6ea8c2013a68 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:20:53.193886 16116 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:53.216238 16298 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40474 (request call id 707) took 5745 ms. Trace:
W20260709 13:20:53.216764 16117 connection.cc:256] RPC call timeout handler was delayed by -1.7995575149998331s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22480000000000003s for a response
I20260709 13:20:53.216727 16298 rpcz_store.cc:276] 0709 13:20:47.469518 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:47.602293 (+132775us) service_pool.cc:224] Handling call
0709 13:20:53.214851 (+5612558us) spinlock_profiling.cc:243] Waited 1.29 s on lock 0x7b5400399ed4. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:20:53.215136 (+   285us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2973743872}
I20260709 13:20:53.188272 16146 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40844 (request call id 2358) took 4115 ms. Trace:
I20260709 13:20:53.220175 20475 raft_consensus.cc:493] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:53.194617 20427 raft_consensus.cc:493] T b87147731a68400e8cce9b879daa7c5d P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:53.220887 20475 raft_consensus.cc:515] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:53.221148 20427 raft_consensus.cc:515] T b87147731a68400e8cce9b879daa7c5d P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:53.223588 16144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68a6838d82ef4cac83507f906139d90f" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:53.211053 16265 leader_election.cc:304] T 68a6838d82ef4cac83507f906139d90f P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:20:53.225176 16144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02838c86531d4f98bf58d1c22b8baf77" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
W20260709 13:20:53.225775 16265 leader_election.cc:336] T a378d7f796f94615b3b347119a9880fd P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 12 election: RPC error from VoteRequest() call to peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943): Timed out: RequestConsensusVote RPC to 127.14.177.130:35943 timed out after 1.578s (SENT)
I20260709 13:20:53.227638 20427 leader_election.cc:290] T b87147731a68400e8cce9b879daa7c5d P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:53.228351 20427 raft_consensus.cc:493] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:20:53.229065 20398 raft_consensus.cc:2749] T 68a6838d82ef4cac83507f906139d90f P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:20:53.229004 20427 raft_consensus.cc:515] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:20:53.172044 16321 debug-util.cc:398] Leaking SignalData structure 0x7b080034c160 after lost signal to thread 16324
I20260709 13:20:53.219031 16146 rpcz_store.cc:276] 0709 13:20:49.072504 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:52.617686 (+3545182us) service_pool.cc:224] Handling call
0709 13:20:52.677220 (+ 59534us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 13:20:52.693598 (+ 16378us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:52.693632 (+    34us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 13:20:52.719344 (+ 25712us) log.cc:844] Serialized 78 byte log entry
0709 13:20:53.185836 (+466492us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:20:53.187665 (+  1829us) log.cc:844] Serialized 14 byte log entry
0709 13:20:53.187901 (+   236us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:53.187979 (+    78us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:20:53.188019 (+    40us) raft_consensus.cc:1685] finished
0709 13:20:53.188031 (+    12us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:53.188184 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":1272,"prepare.run_cpu_time_us":53,"prepare.run_wall_time_us":55,"raft.queue_time_us":1066,"raft.run_cpu_time_us":242,"raft.run_wall_time_us":8268,"thread_start_us":465153,"threads_started":1,"wal-append.queue_time_us":147061}
I20260709 13:20:53.232221 16146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1df93779209748b9860f6d893dd3de53" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:20:53.231271 20475 leader_election.cc:290] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:53.237277 20427 leader_election.cc:290] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
W20260709 13:20:53.239707 16321 debug-util.cc:398] Leaking SignalData structure 0x7b08003e53c0 after lost signal to thread 16849
W20260709 13:20:53.240178 16321 debug-util.cc:398] Leaking SignalData structure 0x7b0800400180 after lost signal to thread 16850
W20260709 13:20:53.242272 16115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 74704.1us
I20260709 13:20:53.205377 20236 raft_consensus.cc:515] T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:53.243228 16115 leader_election.cc:304] T 3884f864a96d428785895e5463e5728f P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:20:53.253629 16146 raft_consensus.cc:2468] T 1df93779209748b9860f6d893dd3de53 P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 11.
I20260709 13:20:53.254396 16115 leader_election.cc:304] T 68a6838d82ef4cac83507f906139d90f P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:20:53.254985 16146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f5f89186204242bf377f7c42b2e2da" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:53.255335 20477 raft_consensus.cc:2804] T 3884f864a96d428785895e5463e5728f P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:20:53.255950 20475 raft_consensus.cc:2804] T 68a6838d82ef4cac83507f906139d90f P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:20:53.255795 20477 raft_consensus.cc:493] T 3884f864a96d428785895e5463e5728f P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Starting leader election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:20:53.257114 20477 raft_consensus.cc:3060] T 3884f864a96d428785895e5463e5728f P 647a343a542d4a139fe106c19095129b [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:20:53.257246 16115 leader_election.cc:304] T 5a96373bb0e14dce8a266b2fc6c58507 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:20:53.258054 20231 raft_consensus.cc:2804] T 5a96373bb0e14dce8a266b2fc6c58507 P 647a343a542d4a139fe106c19095129b [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:20:53.250949 20434 raft_consensus.cc:515] T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:53.258458 20231 raft_consensus.cc:493] T 5a96373bb0e14dce8a266b2fc6c58507 P 647a343a542d4a139fe106c19095129b [term 10 FOLLOWER]: Starting leader election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:20:53.258898 20231 raft_consensus.cc:3060] T 5a96373bb0e14dce8a266b2fc6c58507 P 647a343a542d4a139fe106c19095129b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:20:53.256484 20475 raft_consensus.cc:493] T 68a6838d82ef4cac83507f906139d90f P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Starting leader election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:20:53.261812 20475 raft_consensus.cc:3060] T 68a6838d82ef4cac83507f906139d90f P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:20:53.264994 16218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62fe3346cd3c42c4a55e14af9073bca4" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:53.265442 16823 raft_consensus.cc:1275] T 8c11074579b046d99e585c2818e18a2c P cb82cc7bd6db46d980c75237a1402860 [term 11 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 13:20:53.265841 16218 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33050 (request call id 730) took 2042 ms. Trace:
I20260709 13:20:53.246048 20427 raft_consensus.cc:2804] T 8a878999c0024d2abe517484180290fc P 647a343a542d4a139fe106c19095129b [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:20:53.266211 16218 rpcz_store.cc:276] 0709 13:20:51.223132 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:53.264775 (+2041643us) service_pool.cc:224] Handling call
0709 13:20:53.265763 (+   988us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:20:53.263005 20434 leader_election.cc:290] T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 13 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:53.267668 16223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5718c623ebee4d2bb770f0f9ddb2f164" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:20:53.267728 20427 raft_consensus.cc:697] T 8a878999c0024d2abe517484180290fc P 647a343a542d4a139fe106c19095129b [term 10 LEADER]: Becoming Leader. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:20:53.268384 16223 raft_consensus.cc:2376] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 11. Re-sending same reply.
I20260709 13:20:53.268972 20427 consensus_queue.cc:237] T 8a878999c0024d2abe517484180290fc P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:53.271138 16218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5d59407eb545b895d8d922e0dd69fd" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:20:53.271579 16223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8638ee363273426db1393dd81e5c7b72" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:20:53.272783 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee9ed67006144797b1de6ea8c2013a68" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:53.274547 16218 raft_consensus.cc:2376] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 12. Re-sending same reply.
W20260709 13:20:53.275797 16791 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:53.276719 16222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada5a004ba7c46819051e3ccec71ffdb" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:53.277417 16222 raft_consensus.cc:2468] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 11.
I20260709 13:20:53.277097 16223 raft_consensus.cc:2376] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 12. Re-sending same reply.
W20260709 13:20:53.282047 16264 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:53.255801 16146 raft_consensus.cc:2468] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 11.
I20260709 13:20:53.288949 16146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f5f89186204242bf377f7c42b2e2da" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:53.289639 16219 raft_consensus.cc:2468] T b6f07842d3aa47858df5665874c86f3a P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 12.
I20260709 13:20:53.289865 16146 raft_consensus.cc:2468] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 11.
I20260709 13:20:53.288920 16823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "829b4ad8c5a54eababeea1181b960d4b" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
W20260709 13:20:53.290481 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 166931us
I20260709 13:20:53.291190 16115 leader_election.cc:304] T 434bff3082d141838068f323bdb0d1ed P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:20:53.295491 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee9ed67006144797b1de6ea8c2013a68" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:20:53.298100 20457 raft_consensus.cc:2804] T 434bff3082d141838068f323bdb0d1ed P 647a343a542d4a139fe106c19095129b [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:20:53.298789 20457 raft_consensus.cc:697] T 434bff3082d141838068f323bdb0d1ed P 647a343a542d4a139fe106c19095129b [term 10 LEADER]: Becoming Leader. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:20:53.283082 20477 raft_consensus.cc:515] T 3884f864a96d428785895e5463e5728f P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:53.299938 20457 consensus_queue.cc:237] T 434bff3082d141838068f323bdb0d1ed P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:20:53.309373 16265 leader_election.cc:336] T 31969392ccdb43b491911a319e8bc7e1 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Timed out: RequestConsensusVote RPC to 127.14.177.129:36741 timed out after 2.082s (SENT)
I20260709 13:20:53.310281 16265 leader_election.cc:304] T 31969392ccdb43b491911a319e8bc7e1 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
I20260709 13:20:53.311257 20267 raft_consensus.cc:2749] T 31969392ccdb43b491911a319e8bc7e1 P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:20:53.311304 16789 connection.cc:256] RPC call timeout handler was delayed by -2.8930759270001545s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17010000000000003s for a response
I20260709 13:20:53.303570 16219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c11074579b046d99e585c2818e18a2c" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 12 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
W20260709 13:20:53.312124 16789 connection.cc:256] RPC call timeout handler was delayed by -2.842675927000073s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1757s for a response
I20260709 13:20:53.311456 20495 raft_consensus.cc:493] T af57996a8be84d398f0f023e8d08ef77 P cb82cc7bd6db46d980c75237a1402860 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:20:53.314746 20518 raft_consensus.cc:3055] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 90f3da57796b488c9087a2f523adf045 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:20:53.314420 20495 raft_consensus.cc:515] T af57996a8be84d398f0f023e8d08ef77 P cb82cc7bd6db46d980c75237a1402860 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:20:53.311067 16149 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40830 (request call id 1439) took 5515 ms (client timeout 2950 ms). Trace:
I20260709 13:20:53.245735 16144 raft_consensus.cc:2468] T 02838c86531d4f98bf58d1c22b8baf77 P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 11.
I20260709 13:20:53.300724 16115 leader_election.cc:304] T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:20:53.315231 20518 raft_consensus.cc:740] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 90f3da57796b488c9087a2f523adf045 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
W20260709 13:20:53.318413 16149 rpcz_store.cc:269] 0709 13:20:47.795983 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:48.713864 (+917881us) service_pool.cc:224] Handling call
0709 13:20:53.310641 (+4596777us) spinlock_profiling.cc:243] Waited 3.35 s on lock 0x7b5400073ad4. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:20:53.310818 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":7694735744}
I20260709 13:20:53.323911 16115 leader_election.cc:304] T 23f82224f4564fea93572e8206bad1fe P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:20:53.324767 20487 raft_consensus.cc:2804] T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:20:53.325213 20487 raft_consensus.cc:493] T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b [term 10 FOLLOWER]: Starting leader election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:20:53.325587 20487 raft_consensus.cc:3060] T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:20:53.326601 16222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ab0ac8347641e6955e9bce7d90c1a2" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:53.327322 16147 raft_consensus.cc:2468] T 7c2095a5f4ad49408dbc7f69c606120d P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 8.
I20260709 13:20:53.327699 20434 raft_consensus.cc:2804] T 23f82224f4564fea93572e8206bad1fe P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:20:53.328060 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18ad7e3af5504571a69d10c00750a01c" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:53.328644 20236 leader_election.cc:290] T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 11 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:53.327538 16219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66ae176961b74813a58bc15c9c4c6f2a" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:53.330220 16115 leader_election.cc:304] T b6f07842d3aa47858df5665874c86f3a P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:20:53.330400 16219 raft_consensus.cc:2468] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 13.
I20260709 13:20:53.328898 16223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdbd78acb4cf46298e0de8ba0d130e19" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:53.331698 16223 raft_consensus.cc:2393] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 in current term 12: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
I20260709 13:20:53.332147 16295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18ad7e3af5504571a69d10c00750a01c" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:53.333046 16151 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40844 (request call id 2352) took 4400 ms. Trace:
I20260709 13:20:53.333470 16151 rpcz_store.cc:276] 0709 13:20:48.932674 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:52.111401 (+3178727us) service_pool.cc:224] Handling call
0709 13:20:52.112199 (+   798us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 13:20:52.164004 (+ 51805us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:20:52.164054 (+    50us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:20:52.171126 (+  7072us) log.cc:844] Serialized 52 byte log entry
0709 13:20:52.256978 (+ 85852us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:20:52.257083 (+   105us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:52.257531 (+   448us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:20:53.332719 (+1075188us) raft_consensus.cc:1685] finished
0709 13:20:53.332737 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:53.332962 (+   225us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":186,"raft.run_cpu_time_us":101,"raft.run_wall_time_us":101,"thread_start_us":41299,"threads_started":1,"wal-append.queue_time_us":538927}
I20260709 13:20:53.334884 16294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66ae176961b74813a58bc15c9c4c6f2a" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:20:53.328140 20434 raft_consensus.cc:493] T 23f82224f4564fea93572e8206bad1fe P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Starting leader election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:20:53.335451 16294 raft_consensus.cc:3060] T 66ae176961b74813a58bc15c9c4c6f2a P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:20:53.335561 20434 raft_consensus.cc:3060] T 23f82224f4564fea93572e8206bad1fe P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:20:53.337329 16291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e9b2bf7011f4ae688a79d7bb5a447b4" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:20:53.338207 20267 consensus_queue.cc:1059] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:20:53.338747 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c11074579b046d99e585c2818e18a2c" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 12 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:53.339180 20267 raft_consensus.cc:3055] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 90f3da57796b488c9087a2f523adf045 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:20:53.339470 16299 raft_consensus.cc:2468] T 8c11074579b046d99e585c2818e18a2c P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 11.
I20260709 13:20:53.339574 20267 raft_consensus.cc:740] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 90f3da57796b488c9087a2f523adf045 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:20:53.340605 20267 consensus_queue.cc:260] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 90f3da57796b488c9087a2f523adf045 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:53.341770 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdbd78acb4cf46298e0de8ba0d130e19" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:53.342602 16299 raft_consensus.cc:2393] T cdbd78acb4cf46298e0de8ba0d130e19 P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 in current term 12: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
I20260709 13:20:53.342504 20518 consensus_queue.cc:260] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 90f3da57796b488c9087a2f523adf045 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:53.333076 16295 raft_consensus.cc:2410] T 18ad7e3af5504571a69d10c00750a01c P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b for term 11 because replica has last-logged OpId of term: 9 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 6.
I20260709 13:20:53.343775 20518 raft_consensus.cc:3060] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:20:53.335644 16296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada5a004ba7c46819051e3ccec71ffdb" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:53.347443 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66ae176961b74813a58bc15c9c4c6f2a" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:20:53.347994 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434bff3082d141838068f323bdb0d1ed" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:53.348692 16298 raft_consensus.cc:2468] T 434bff3082d141838068f323bdb0d1ed P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 10.
I20260709 13:20:53.348531 20487 raft_consensus.cc:515] T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:53.348003 16224 raft_consensus.cc:3060] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:20:53.355567 16294 raft_consensus.cc:2468] T 66ae176961b74813a58bc15c9c4c6f2a P 90f3da57796b488c9087a2f523adf045 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 14.
I20260709 13:20:53.327523 16144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:20:53.355908 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ab0ac8347641e6955e9bce7d90c1a2" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:53.357074 16298 raft_consensus.cc:2468] T d2ab0ac8347641e6955e9bce7d90c1a2 P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 11.
I20260709 13:20:53.320338 20477 leader_election.cc:290] T 3884f864a96d428785895e5463e5728f P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 8 election: Requested vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:53.357683 16822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5d59407eb545b895d8d922e0dd69fd" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:53.360003 20477 raft_consensus.cc:2804] T b6f07842d3aa47858df5665874c86f3a P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:20:53.360251 16822 raft_consensus.cc:2393] T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 12: Already voted for candidate cb82cc7bd6db46d980c75237a1402860 in this term.
I20260709 13:20:53.360394 20477 raft_consensus.cc:493] T b6f07842d3aa47858df5665874c86f3a P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:53.360911 20477 raft_consensus.cc:3060] T b6f07842d3aa47858df5665874c86f3a P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:20:53.358713 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:53.361863 16226 raft_consensus.cc:2468] T ecc1e363230043fe9fef390f0b979f20 P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 10.
I20260709 13:20:53.365676 20505 raft_consensus.cc:1064] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045: attempting to promote NON_VOTER cb82cc7bd6db46d980c75237a1402860 to VOTER
I20260709 13:20:53.367259 16823 raft_consensus.cc:1240] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [term 15 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 14. Current term is 15. Ops: []
I20260709 13:20:53.367355 20231 raft_consensus.cc:515] T 5a96373bb0e14dce8a266b2fc6c58507 P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:53.368618 20505 consensus_queue.cc:237] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 7, Current term: 10, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:53.369555 16823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdbd78acb4cf46298e0de8ba0d130e19" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:20:53.369953 20231 leader_election.cc:290] T 5a96373bb0e14dce8a266b2fc6c58507 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 11 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:53.370424 16823 raft_consensus.cc:3060] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:20:53.370492 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e65676b21b748728ad39668cfe722ec" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:53.372133 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74ca8140f35c4ae29e8a664f6bdfc829" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:53.373531 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a378d7f796f94615b3b347119a9880fd" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:53.375401 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a378d7f796f94615b3b347119a9880fd" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
W20260709 13:20:53.375732 16172 debug-util.cc:398] Leaking SignalData structure 0x7b08000c48c0 after lost signal to thread 16175
I20260709 13:20:53.376161 16293 raft_consensus.cc:2468] T a378d7f796f94615b3b347119a9880fd P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 12.
W20260709 13:20:53.376156 16172 debug-util.cc:398] Leaking SignalData structure 0x7b080039f660 after lost signal to thread 16176
I20260709 13:20:53.330524 20495 leader_election.cc:290] T af57996a8be84d398f0f023e8d08ef77 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
W20260709 13:20:53.382122 16172 debug-util.cc:398] Leaking SignalData structure 0x7b0800377320 after lost signal to thread 16250
W20260709 13:20:53.382493 16172 debug-util.cc:398] Leaking SignalData structure 0x7b08002959e0 after lost signal to thread 16251
I20260709 13:20:53.384891 16822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b7d1ab519044eda2d5e04a07242f25" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:53.328846 16149 raft_consensus.cc:2468] T 18ad7e3af5504571a69d10c00750a01c P 647a343a542d4a139fe106c19095129b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 10.
I20260709 13:20:53.385844 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5d59407eb545b895d8d922e0dd69fd" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:53.385855 16822 raft_consensus.cc:2468] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 9.
I20260709 13:20:53.386859 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02838c86531d4f98bf58d1c22b8baf77" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:20:53.388343 16218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2372981006d4b218215c28964dea623" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 13 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:53.388639 16149 raft_consensus.cc:2376] T 02838c86531d4f98bf58d1c22b8baf77 P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 11. Re-sending same reply.
W20260709 13:20:53.389262 20459 log.cc:927] Time spent T 3a7ca61592e848ea9b0fb81a2ce33cdc P cb82cc7bd6db46d980c75237a1402860: Append to log took a long time: real 5.210s	user 0.006s	sys 0.039s
I20260709 13:20:53.389626 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a878999c0024d2abe517484180290fc" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:53.390066 16221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b7d1ab519044eda2d5e04a07242f25" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:53.390744 16221 raft_consensus.cc:2468] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 9.
I20260709 13:20:53.391611 16825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a378d7f796f94615b3b347119a9880fd" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:53.391835 16821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 12 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:20:53.392329 16825 raft_consensus.cc:2468] T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 12.
I20260709 13:20:53.392720 16821 raft_consensus.cc:2393] T 19baa7bc367844c288bca8f81439c413 P cb82cc7bd6db46d980c75237a1402860 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 12: Already voted for candidate cb82cc7bd6db46d980c75237a1402860 in this term.
I20260709 13:20:53.390892 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ad5dbfa99ee4b8d9e28be92b3f36db5" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:53.394075 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fffab26427e24d398e2695aad40ca134" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:53.394711 16300 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40474 (request call id 710) took 4242 ms. Trace:
I20260709 13:20:53.395299 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fffab26427e24d398e2695aad40ca134" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:53.396180 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26354361e9b044cea1c2b86c4882fe92" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:53.395193 16300 rpcz_store.cc:276] 0709 13:20:49.151712 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:49.158515 (+  6803us) service_pool.cc:224] Handling call
0709 13:20:49.158903 (+   388us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:20:52.727820 (+3568917us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:52.730843 (+  3023us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:20:52.758869 (+ 28026us) log.cc:844] Serialized 26 byte log entry
0709 13:20:52.832772 (+ 73903us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:20:52.832813 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:53.394115 (+561302us) spinlock_profiling.cc:243] Waited 3.55 s on lock 0x7b54000cf654. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:20:53.394170 (+    55us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:20:53.394231 (+    61us) raft_consensus.cc:1685] finished
0709 13:20:53.394248 (+    17us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:53.394484 (+   236us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":8174701952,"thread_start_us":73451,"threads_started":1,"wal-append.queue_time_us":83407}
I20260709 13:20:53.396567 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c61a39fd4f84424b129c1a6f9ba404d" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:53.395226 20267 raft_consensus.cc:3060] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:20:53.397248 16149 raft_consensus.cc:2468] T 9c61a39fd4f84424b129c1a6f9ba404d P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 11.
I20260709 13:20:53.398293 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26354361e9b044cea1c2b86c4882fe92" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:53.389284 16218 raft_consensus.cc:2410] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b for term 13 because replica has last-logged OpId of term: 11 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 8.
I20260709 13:20:53.400055 16218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5ad4eb281564936a59f41e387a47156" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
W20260709 13:20:53.400671 16147 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40844 (request call id 2344) took 4555 ms (client timeout 1543 ms). Trace:
W20260709 13:20:53.401145 16147 rpcz_store.cc:269] 0709 13:20:48.845545 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:48.885304 (+ 39759us) service_pool.cc:224] Handling call
0709 13:20:53.400383 (+4515079us) spinlock_profiling.cc:243] Waited 4.44 s on lock 0x7b5400072bd4. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:20:53.400587 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":10204561280}
I20260709 13:20:53.401503 16219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a959dbecc2648eb85d1cc550f5a3245" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:53.402369 16147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f82224f4564fea93572e8206bad1fe" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:53.365378 16789 leader_election.cc:304] T 8638ee363273426db1393dd81e5c7b72 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:20:53.402741 16219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68a6838d82ef4cac83507f906139d90f" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:53.403541 20268 raft_consensus.cc:2804] T 8638ee363273426db1393dd81e5c7b72 P cb82cc7bd6db46d980c75237a1402860 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:20:53.404109 20268 raft_consensus.cc:697] T 8638ee363273426db1393dd81e5c7b72 P cb82cc7bd6db46d980c75237a1402860 [term 12 LEADER]: Becoming Leader. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:20:53.404223 16219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a96373bb0e14dce8a266b2fc6c58507" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:53.405676 16821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2372981006d4b218215c28964dea623" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:53.405162 20268 consensus_queue.cc:237] T 8638ee363273426db1393dd81e5c7b72 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:53.406055 16825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada5a004ba7c46819051e3ccec71ffdb" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:53.401165 16221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 13 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:53.407543 16825 raft_consensus.cc:2468] T ada5a004ba7c46819051e3ccec71ffdb P cb82cc7bd6db46d980c75237a1402860 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 11.
I20260709 13:20:53.407956 16221 raft_consensus.cc:2468] T 19baa7bc367844c288bca8f81439c413 P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 12.
I20260709 13:20:53.408859 16221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62fe3346cd3c42c4a55e14af9073bca4" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:53.410280 16221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87ac0d1f7e8a419f9bcbc04aee0e067e" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
W20260709 13:20:53.410634 16789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 45354.9us
I20260709 13:20:53.397163 16150 raft_consensus.cc:2410] T 26354361e9b044cea1c2b86c4882fe92 P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 for term 10 because replica has last-logged OpId of term: 9 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 8.
I20260709 13:20:53.407568 16821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74ca8140f35c4ae29e8a664f6bdfc829" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:53.402918 16224 raft_consensus.cc:2468] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 14.
I20260709 13:20:53.414757 16823 raft_consensus.cc:2468] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 12.
I20260709 13:20:53.386567 16226 raft_consensus.cc:2468] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 12.
I20260709 13:20:53.415729 16219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806d7690c2b644959152074bb2cab0a7" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:20:53.416296 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "829b4ad8c5a54eababeea1181b960d4b" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:53.416958 16219 raft_consensus.cc:2410] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b for term 11 because replica has last-logged OpId of term: 10 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 7.
I20260709 13:20:53.411046 16221 raft_consensus.cc:2410] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b for term 12 because replica has last-logged OpId of term: 11 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 8.
I20260709 13:20:53.418916 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8158bbd17b994d2eb96f560bd0f1b88d" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: true dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:20:53.421671 16149 raft_consensus.cc:2376] T 8158bbd17b994d2eb96f560bd0f1b88d P 647a343a542d4a139fe106c19095129b [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 10. Re-sending same reply.
I20260709 13:20:53.402902 16822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9944d5e4b8c4109af9955c4f08c9095" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:20:53.356896 16144 raft_consensus.cc:2393] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 12: Already voted for candidate cb82cc7bd6db46d980c75237a1402860 in this term.
I20260709 13:20:53.427937 16146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:20:53.951253 16822 raft_consensus.cc:2393] T c9944d5e4b8c4109af9955c4f08c9095 P cb82cc7bd6db46d980c75237a1402860 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 11: Already voted for candidate cb82cc7bd6db46d980c75237a1402860 in this term.
I20260709 13:20:53.951622 16146 raft_consensus.cc:2393] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 12: Already voted for candidate cb82cc7bd6db46d980c75237a1402860 in this term.
I20260709 13:20:53.250730 16117 leader_election.cc:304] T 2794af5b99984efebdee564035841cb0 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860
I20260709 13:20:53.953079 19982 raft_consensus.cc:2749] T 2794af5b99984efebdee564035841cb0 P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:20:53.953177 16117 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 702568us
W20260709 13:20:53.954815 16263 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:53.327407 16222 raft_consensus.cc:2468] T d2ab0ac8347641e6955e9bce7d90c1a2 P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 11.
W20260709 13:20:53.955515 16263 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:53.956198 16222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4f65cdf00b748bb963ee6e81ef43b79" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:20:53.956629 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87147731a68400e8cce9b879daa7c5d" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:53.956959 16222 raft_consensus.cc:2393] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 13: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
I20260709 13:20:53.957269 16293 raft_consensus.cc:2468] T b87147731a68400e8cce9b879daa7c5d P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 11.
I20260709 13:20:53.957866 16222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5718c623ebee4d2bb770f0f9ddb2f164" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 12 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:53.958437 16222 raft_consensus.cc:2468] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 11.
I20260709 13:20:53.958550 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f5f89186204242bf377f7c42b2e2da" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:53.400678 16218 raft_consensus.cc:2376] T e5ad4eb281564936a59f41e387a47156 P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 647a343a542d4a139fe106c19095129b in term 12. Re-sending same reply.
I20260709 13:20:53.959458 16293 raft_consensus.cc:2410] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b for term 12 because replica has last-logged OpId of term: 11 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 6.
I20260709 13:20:53.423561 16819 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49906 (request call id 877) took 5817 ms. Trace:
I20260709 13:20:53.579453 16151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2794af5b99984efebdee564035841cb0" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:53.963599 16151 raft_consensus.cc:2468] T 2794af5b99984efebdee564035841cb0 P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 8.
I20260709 13:20:53.964582 16151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "829b4ad8c5a54eababeea1181b960d4b" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:20:53.965790 16151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0eab25ce4c64e018a041beeda2867db" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:20:53.966321 16151 raft_consensus.cc:3060] T c0eab25ce4c64e018a041beeda2867db P 647a343a542d4a139fe106c19095129b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:20:53.585637 16221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a878999c0024d2abe517484180290fc" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:53.597934 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0eab25ce4c64e018a041beeda2867db" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:53.967240 16148 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40844 (request call id 2355) took 4932 ms. Trace:
I20260709 13:20:53.967728 16224 raft_consensus.cc:2393] T c0eab25ce4c64e018a041beeda2867db P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 11: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
I20260709 13:20:53.584362 16223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74ca8140f35c4ae29e8a664f6bdfc829" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:53.967638 16148 rpcz_store.cc:276] 0709 13:20:49.034910 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:52.178972 (+3144062us) service_pool.cc:224] Handling call
0709 13:20:53.404354 (+1225382us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:53.840245 (+435891us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:53.840280 (+    35us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:20:53.840343 (+    63us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:20:53.840380 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:53.840451 (+    71us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:53.967028 (+126577us) spinlock_profiling.cc:243] Waited 1.22 s on lock 0x7b5400072bd0. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5e8ad 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:20:53.967181 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":435416,"raft.queue_time_us":436825,"raft.run_cpu_time_us":100,"raft.run_wall_time_us":101,"spinlock_wait_cycles":2799784192}
I20260709 13:20:53.968818 16223 raft_consensus.cc:2468] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 11.
I20260709 13:20:53.610522 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87147731a68400e8cce9b879daa7c5d" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:53.969668 16144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31969392ccdb43b491911a319e8bc7e1" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:53.969820 16148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02838c86531d4f98bf58d1c22b8baf77" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:53.969985 16150 raft_consensus.cc:2468] T b87147731a68400e8cce9b879daa7c5d P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 12.
I20260709 13:20:53.970465 16144 raft_consensus.cc:2393] T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 11: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
I20260709 13:20:53.970553 16148 raft_consensus.cc:2393] T 02838c86531d4f98bf58d1c22b8baf77 P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 11: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
I20260709 13:20:53.970803 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87147731a68400e8cce9b879daa7c5d" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:53.611992 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f82224f4564fea93572e8206bad1fe" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:53.971352 16150 raft_consensus.cc:2468] T b87147731a68400e8cce9b879daa7c5d P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 12.
I20260709 13:20:53.971786 16148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a959dbecc2648eb85d1cc550f5a3245" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:53.972203 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4dc6b0541ca431ebc757beac00521f1" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:53.972241 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3884f864a96d428785895e5463e5728f" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:53.972724 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4f65cdf00b748bb963ee6e81ef43b79" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:20:53.630054 16219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f5f89186204242bf377f7c42b2e2da" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:53.973152 16149 raft_consensus.cc:2393] T 3884f864a96d428785895e5463e5728f P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 8: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
I20260709 13:20:53.971360 16144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a959dbecc2648eb85d1cc550f5a3245" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:53.973968 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8158bbd17b994d2eb96f560bd0f1b88d" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
W20260709 13:20:53.670312 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 61159.3us
I20260709 13:20:53.974536 16144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d16dfad0c3b540e5b4b86492ddf87208" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:53.972335 16148 raft_consensus.cc:2468] T 5a959dbecc2648eb85d1cc550f5a3245 P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 9.
I20260709 13:20:53.627786 16226 raft_consensus.cc:2393] T 829b4ad8c5a54eababeea1181b960d4b P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 11: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
I20260709 13:20:53.975144 16144 raft_consensus.cc:2468] T d16dfad0c3b540e5b4b86492ddf87208 P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 11.
W20260709 13:20:53.630410 16192 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:53.975611 16148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b7d1ab519044eda2d5e04a07242f25" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:53.976080 16144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2372981006d4b218215c28964dea623" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:53.976476 16148 raft_consensus.cc:2468] T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 9.
I20260709 13:20:53.976619 16144 raft_consensus.cc:2468] T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 12.
I20260709 13:20:53.977659 16148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5ad4eb281564936a59f41e387a47156" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:20:53.978472 16192 leader_election.cc:304] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 
W20260709 13:20:53.670532 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 91790.1us
I20260709 13:20:53.979457 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3884f864a96d428785895e5463e5728f" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:20:53.980016 16299 raft_consensus.cc:3060] T 3884f864a96d428785895e5463e5728f P 90f3da57796b488c9087a2f523adf045 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:20:53.980288 16116 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:53.679113 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a96373bb0e14dce8a266b2fc6c58507" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:20:53.704034 16789 leader_election.cc:304] T 8c11074579b046d99e585c2818e18a2c P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:20:53.761410 16818 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49906 (request call id 883) took 5487 ms. Trace:
I20260709 13:20:53.780068 16220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c61a39fd4f84424b129c1a6f9ba404d" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 12 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
W20260709 13:20:53.797173 16791 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:53.806995 16300 raft_consensus.cc:1240] T 8a878999c0024d2abe517484180290fc P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 9. Current term is 10. Ops: []
W20260709 13:20:53.840731 16266 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:20:53.842123 16114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:53.853091 20487 leader_election.cc:290] T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 11 election: Requested vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
W20260709 13:20:53.857896 16292 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.509s	user 0.009s	sys 0.028s
W20260709 13:20:53.865099 16191 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:20:53.869143 16790 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.546s	user 0.003s	sys 0.005s
W20260709 13:20:53.881496 16264 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:53.889564 16225 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57604 (request call id 2557) took 1129 ms. Trace:
W20260709 13:20:53.892757 16172 debug-util.cc:398] Leaking SignalData structure 0x7b080016b4e0 after lost signal to thread 16324
I20260709 13:20:53.758947 16142 raft_consensus.cc:2468] T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 11.
I20260709 13:20:53.912495 20472 leader_election.cc:290] T 7c2095a5f4ad49408dbc7f69c606120d P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
W20260709 13:20:53.919903 20457 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.556s	user 0.004s	sys 0.020s
I20260709 13:20:53.926498 16296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a9de347c711477e87c582d003f36925" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 14 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:53.413539 16821 raft_consensus.cc:2468] T 74ca8140f35c4ae29e8a664f6bdfc829 P cb82cc7bd6db46d980c75237a1402860 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 11.
W20260709 13:20:53.930325 16265 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:20:53.933439 20475 consensus_meta.cc:287] T 68a6838d82ef4cac83507f906139d90f P 647a343a542d4a139fe106c19095129b: Time spent flushing consensus metadata: real 0.670s	user 0.011s	sys 0.002s
I20260709 13:20:53.977972 16144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4f65cdf00b748bb963ee6e81ef43b79" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
W20260709 13:20:53.992039 16191 connection.cc:256] RPC call timeout handler was delayed by -3.4498683560000245s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1543s for a response
W20260709 13:20:53.982942 16116 leader_election.cc:336] T e5ad4eb281564936a59f41e387a47156 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 12 election: RPC error from VoteRequest() call to peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943): Timed out: RequestConsensusVote RPC to 127.14.177.130:35943 timed out after 1.671s (SENT)
W20260709 13:20:53.992480 16191 connection.cc:256] RPC call timeout handler was delayed by -3.3499683560000904s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1654s for a response
W20260709 13:20:53.992843 16191 connection.cc:256] RPC call timeout handler was delayed by -3.3463683560000845s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1658s for a response
W20260709 13:20:53.993227 16191 connection.cc:256] RPC call timeout handler was delayed by -3.34366835600008s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16610000000000003s for a response
W20260709 13:20:53.993803 16191 connection.cc:256] RPC call timeout handler was delayed by -3.3283683560000554s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1678s for a response
W20260709 13:20:53.994125 16191 connection.cc:256] RPC call timeout handler was delayed by -2.8513683559999663s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22080000000000002s for a response
W20260709 13:20:53.994421 16191 connection.cc:256] RPC call timeout handler was delayed by -2.8324683559999357s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22290000000000001s for a response
W20260709 13:20:53.994485 16172 debug-util.cc:398] Leaking SignalData structure 0x7b080033bee0 after lost signal to thread 16849
W20260709 13:20:53.994731 16191 connection.cc:256] RPC call timeout handler was delayed by -2.7280683559999943s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.23450000000000004s for a response
W20260709 13:20:53.995083 16788 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:53.945217 20388 raft_consensus.cc:493] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:20:53.935083 16295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87147731a68400e8cce9b879daa7c5d" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:53.386565 16145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2794af5b99984efebdee564035841cb0" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:53.972992 16150 raft_consensus.cc:2393] T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 11: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
I20260709 13:20:53.974211 16293 raft_consensus.cc:2393] T c4f65cdf00b748bb963ee6e81ef43b79 P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 13: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
I20260709 13:20:53.993038 20475 raft_consensus.cc:515] T 68a6838d82ef4cac83507f906139d90f P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:53.766360 20434 raft_consensus.cc:515] T 23f82224f4564fea93572e8206bad1fe P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:20:53.908743 20470 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.524s	user 0.000s	sys 0.011s
W20260709 13:20:53.995146 16191 connection.cc:256] RPC call timeout handler was delayed by -2.5669683559999612s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2524s for a response
W20260709 13:20:53.991833 20268 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.582s	user 0.010s	sys 0.031s
W20260709 13:20:53.992039 16292 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.643s	user 0.011s	sys 0.030s
I20260709 13:20:54.006384 16191 leader_election.cc:304] T 08c74490c40243a88e62d1760b5cd814 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:20:53.998256 16818 rpcz_store.cc:276] 0709 13:20:48.274310 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:48.274697 (+   387us) service_pool.cc:224] Handling call
0709 13:20:48.392178 (+117481us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:20:48.694610 (+302432us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:48.694644 (+    34us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:20:48.695381 (+   737us) log.cc:844] Serialized 52 byte log entry
0709 13:20:48.887800 (+192419us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:20:48.996216 (+108416us) log.cc:844] Serialized 14 byte log entry
0709 13:20:48.996468 (+   252us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:48.996539 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:20:52.228529 (+3231990us) spinlock_profiling.cc:243] Waited 7.6 ms on lock 0x7b18003cbc20. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b693ade1 00007f69b69475b1 00007f69ada723f3 00007f69ada747c1 00007f69ada74779 00007f69ada74709 00007f69ada746d1 00007f69ada739cd 000000000038f354 000000000038f199 00007f69ada5583a
0709 13:20:53.760912 (+1532383us) raft_consensus.cc:1685] finished
0709 13:20:53.761009 (+    97us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:53.761222 (+   213us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3917,"prepare.queue_time_us":176210,"prepare.run_cpu_time_us":48,"prepare.run_wall_time_us":50,"raft.queue_time_us":114509,"raft.run_cpu_time_us":203,"raft.run_wall_time_us":6425,"spinlock_wait_cycles":17487232,"thread_start_us":491188,"threads_started":3,"wal-append.queue_time_us":249304,"wal-append.run_cpu_time_us":5285,"wal-append.run_wall_time_us":1501693}
I20260709 13:20:53.995381 16144 raft_consensus.cc:2393] T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 13: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
I20260709 13:20:54.010012 16144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:54.010725 16144 raft_consensus.cc:2393] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 12: Already voted for candidate cb82cc7bd6db46d980c75237a1402860 in this term.
W20260709 13:20:54.006222 20268 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.596s	user 0.011s	sys 0.031s
I20260709 13:20:54.011487 16144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c2095a5f4ad49408dbc7f69c606120d" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
W20260709 13:20:53.994876 16172 debug-util.cc:398] Leaking SignalData structure 0x7b08003c32c0 after lost signal to thread 16850
W20260709 13:20:53.940903 20477 consensus_meta.cc:287] T b6f07842d3aa47858df5665874c86f3a P 647a343a542d4a139fe106c19095129b: Time spent flushing consensus metadata: real 0.580s	user 0.010s	sys 0.002s
I20260709 13:20:54.012570 16144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c2095a5f4ad49408dbc7f69c606120d" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:54.013092 16146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8158bbd17b994d2eb96f560bd0f1b88d" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
W20260709 13:20:53.946970 20518 consensus_meta.cc:287] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 90f3da57796b488c9087a2f523adf045: Time spent flushing consensus metadata: real 0.603s	user 0.011s	sys 0.001s
W20260709 13:20:53.999595 16790 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.676s	user 0.005s	sys 0.007s
I20260709 13:20:54.000931 16296 raft_consensus.cc:2410] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 for term 14 because replica has last-logged OpId of term: 11 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 8.
I20260709 13:20:54.024201 20424 raft_consensus.cc:493] T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
W20260709 13:20:54.024498 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 703481us
I20260709 13:20:54.001879 16295 raft_consensus.cc:2410] T b87147731a68400e8cce9b879daa7c5d P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 for term 13 because replica has last-logged OpId of term: 11 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 9.
I20260709 13:20:54.024804 20424 raft_consensus.cc:515] T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:54.025756 16294 raft_consensus.cc:1275] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:20:54.014657 16146 raft_consensus.cc:2468] T 8158bbd17b994d2eb96f560bd0f1b88d P 647a343a542d4a139fe106c19095129b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 10.
W20260709 13:20:54.027433 16790 leader_election.cc:336] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 11 election: RPC error from VoteRequest() call to peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943): Timed out: RequestConsensusVote RPC to 127.14.177.130:35943 timed out after 1.513s (SENT)
I20260709 13:20:54.027781 16219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4dc6b0541ca431ebc757beac00521f1" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:20:54.028052 16146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f2da98864b4425bf7c0055951fdd4b" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
W20260709 13:20:54.028358 16790 leader_election.cc:336] T ee9ed67006144797b1de6ea8c2013a68 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943): Timed out: RequestConsensusVote RPC to 127.14.177.130:35943 timed out after 1.643s (SENT)
W20260709 13:20:54.001959 16142 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40844 (request call id 2342) took 5157 ms (client timeout 1658 ms). Trace:
I20260709 13:20:54.028980 16219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6f07842d3aa47858df5665874c86f3a" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
W20260709 13:20:54.029326 16790 leader_election.cc:336] T 8638ee363273426db1393dd81e5c7b72 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 12 election: RPC error from VoteRequest() call to peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943): Timed out: RequestConsensusVote RPC to 127.14.177.130:35943 timed out after 1.701s (SENT)
I20260709 13:20:54.029589 16219 raft_consensus.cc:2468] T b6f07842d3aa47858df5665874c86f3a P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 12.
W20260709 13:20:54.029157 16142 rpcz_store.cc:269] 0709 13:20:48.844869 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:49.004564 (+159695us) service_pool.cc:224] Handling call
0709 13:20:54.001759 (+4997195us) spinlock_profiling.cc:243] Waited 4.71 s on lock 0x7b540041ebd4. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:20:54.001897 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":10839242496}
I20260709 13:20:54.030480 16219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c2095a5f4ad49408dbc7f69c606120d" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:54.030826 16142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434bff3082d141838068f323bdb0d1ed" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:54.031432 20443 consensus_queue.cc:1059] T ee9ed67006144797b1de6ea8c2013a68 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:20:54.032379 20422 consensus_queue.cc:1048] T 8c11074579b046d99e585c2818e18a2c P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.003s
I20260709 13:20:54.032058 16142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a9de347c711477e87c582d003f36925" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:54.033167 16221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66f75124215f4ca6bef831e174a792d8" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:53.999526 16225 rpcz_store.cc:276] 0709 13:20:52.760139 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:52.762503 (+  2364us) service_pool.cc:224] Handling call
0709 13:20:53.889439 (+1126936us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":13184}
I20260709 13:20:54.015715 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4f65cdf00b748bb963ee6e81ef43b79" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
W20260709 13:20:54.002619 16264 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:54.005295 16147 raft_consensus.cc:2393] T 23f82224f4564fea93572e8206bad1fe P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 9: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
I20260709 13:20:54.002646 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18ad7e3af5504571a69d10c00750a01c" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
W20260709 13:20:53.998972 16791 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:54.017896 16144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e65676b21b748728ad39668cfe722ec" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:54.018019 16222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d16dfad0c3b540e5b4b86492ddf87208" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 12 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:54.018203 16299 raft_consensus.cc:2468] T 3884f864a96d428785895e5463e5728f P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 8.
I20260709 13:20:54.016424 16192 leader_election.cc:304] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:20:53.998945 20388 raft_consensus.cc:515] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:54.020586 20571 raft_consensus.cc:493] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:54.020936 20475 leader_election.cc:290] T 68a6838d82ef4cac83507f906139d90f P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 12 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
W20260709 13:20:54.021034 16789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 317060us
I20260709 13:20:54.013499 20487 raft_consensus.cc:493] T 26354361e9b044cea1c2b86c4882fe92 P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:20:53.995510 20267 raft_consensus.cc:493] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:20:54.161806 16300 raft_consensus.cc:1275] T 8c11074579b046d99e585c2818e18a2c P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 13:20:54.161990 16296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c2095a5f4ad49408dbc7f69c606120d" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:54.162644 16296 raft_consensus.cc:2468] T 7c2095a5f4ad49408dbc7f69c606120d P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 8.
I20260709 13:20:53.962970 16819 rpcz_store.cc:276] 0709 13:20:47.606150 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:47.606716 (+   566us) service_pool.cc:224] Handling call
0709 13:20:47.640212 (+ 33496us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:20:47.640527 (+   315us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:47.640595 (+    68us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:20:47.666586 (+ 25991us) log.cc:844] Serialized 26 byte log entry
0709 13:20:47.765363 (+ 98777us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:20:47.765399 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:47.765470 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:20:53.388604 (+5623134us) spinlock_profiling.cc:243] Waited 3.59 ms on lock 0x7b4400022b4c. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69ad8fadb1 00007f69ad998c85 00007f69ad9996a2 00007f69b070b1db 00007f69b0706aaf 00007f69b06fa86e 00007f69b06f9bb6 00007f69b06f97d8 00007f69b0703221 00007f69b07031d9 00007f69b0703169
0709 13:20:53.398109 (+  9505us) raft_consensus.cc:1685] finished
0709 13:20:53.398136 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:53.423309 (+ 25173us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":8258816,"thread_start_us":98360,"threads_started":1,"wal-append.queue_time_us":414973}
W20260709 13:20:53.985896 20505 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.593s	user 0.009s	sys 0.031s
W20260709 13:20:54.004980 20470 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.621s	user 0.000s	sys 0.013s
I20260709 13:20:54.001354 16116 leader_election.cc:304] T fffab26427e24d398e2695aad40ca134 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
W20260709 13:20:54.000397 20457 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.637s	user 0.005s	sys 0.021s
I20260709 13:20:54.021857 16298 raft_consensus.cc:1275] T e5ad4eb281564936a59f41e387a47156 P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:20:54.004786 20434 leader_election.cc:290] T 23f82224f4564fea93572e8206bad1fe P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 9 election: Requested vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:54.021768 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a96373bb0e14dce8a266b2fc6c58507" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
W20260709 13:20:53.998631 16251 heartbeater.cc:646] Failed to heartbeat to 127.14.177.190:44079 (2 consecutive failures): Timed out: Failed to send heartbeat to master: TSHeartbeat RPC to 127.14.177.190:44079 timed out after 15.000s (SENT)
I20260709 13:20:54.022500 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 12 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:20:54.177486 16293 raft_consensus.cc:2393] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 12: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
I20260709 13:20:54.023061 20477 raft_consensus.cc:515] T b6f07842d3aa47858df5665874c86f3a P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:54.023403 16292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4dc6b0541ca431ebc757beac00521f1" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:20:54.023626 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746aabf856ab401c9ef5e859371e7b67" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:54.002566 20472 raft_consensus.cc:493] T 57a744f5aaf944678a63469cbf514015 P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:20:54.022637 20268 raft_consensus.cc:2804] T 8c11074579b046d99e585c2818e18a2c P cb82cc7bd6db46d980c75237a1402860 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:20:54.002168 16145 raft_consensus.cc:2468] T 2794af5b99984efebdee564035841cb0 P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 8.
I20260709 13:20:54.034025 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c74490c40243a88e62d1760b5cd814" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 11 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:54.033437 20269 raft_consensus.cc:2804] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:20:54.026916 20424 leader_election.cc:290] T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:54.138753 16142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62fe3346cd3c42c4a55e14af9073bca4" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:20:54.187649 16142 raft_consensus.cc:3060] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:20:54.139492 16221 raft_consensus.cc:1275] T e5ad4eb281564936a59f41e387a47156 P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:20:54.139436 16218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2372981006d4b218215c28964dea623" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:20:54.038866 16149 raft_consensus.cc:2393] T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 13: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
I20260709 13:20:54.138432 16220 raft_consensus.cc:1275] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 12 index: 9. (term mismatch)
I20260709 13:20:54.143201 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8158bbd17b994d2eb96f560bd0f1b88d" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:54.190049 16226 raft_consensus.cc:2410] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b for term 11 because replica has last-logged OpId of term: 9 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 6.
I20260709 13:20:54.154099 16222 raft_consensus.cc:2410] T d16dfad0c3b540e5b4b86492ddf87208 P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b for term 12 because replica has last-logged OpId of term: 10 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 13:20:54.033715 20293 raft_consensus.cc:3055] T ee9ed67006144797b1de6ea8c2013a68 P 1ca2725d290e47d68b1a4b609c9816ac [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:20:54.033182 20570 consensus_queue.cc:1059] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.121s
W20260709 13:20:54.142024 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 132096us
I20260709 13:20:54.186228 20269 raft_consensus.cc:493] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:54.195670 20269 raft_consensus.cc:3060] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:20:54.197846 16821 raft_consensus.cc:1240] T b87147731a68400e8cce9b879daa7c5d P cb82cc7bd6db46d980c75237a1402860 [term 12 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 11. Current term is 12. Ops: [11.9-11.9]
I20260709 13:20:54.199292 16819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434bff3082d141838068f323bdb0d1ed" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:20:54.028991 16146 raft_consensus.cc:2410] T 84f2da98864b4425bf7c0055951fdd4b P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 for term 10 because replica has last-logged OpId of term: 9 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
I20260709 13:20:54.156217 16147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af57996a8be84d398f0f023e8d08ef77" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:54.203673 16147 raft_consensus.cc:2468] T af57996a8be84d398f0f023e8d08ef77 P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 12.
I20260709 13:20:54.203361 20472 raft_consensus.cc:515] T 57a744f5aaf944678a63469cbf514015 P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:54.192451 20293 raft_consensus.cc:740] T ee9ed67006144797b1de6ea8c2013a68 P 1ca2725d290e47d68b1a4b609c9816ac [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:20:54.204049 16146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87147731a68400e8cce9b879daa7c5d" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:54.204689 16147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746aabf856ab401c9ef5e859371e7b67" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:54.205076 16146 raft_consensus.cc:2410] T b87147731a68400e8cce9b879daa7c5d P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 for term 13 because replica has last-logged OpId of term: 11 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 9.
I20260709 13:20:54.204896 20293 consensus_queue.cc:260] T ee9ed67006144797b1de6ea8c2013a68 P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:54.205875 20472 leader_election.cc:290] T 57a744f5aaf944678a63469cbf514015 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:54.205988 16146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66f75124215f4ca6bef831e174a792d8" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:54.206130 20293 raft_consensus.cc:3060] T ee9ed67006144797b1de6ea8c2013a68 P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:20:54.043699 16294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31969392ccdb43b491911a319e8bc7e1" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:20:54.157094 16144 raft_consensus.cc:2468] T 9e65676b21b748728ad39668cfe722ec P 647a343a542d4a139fe106c19095129b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 10.
W20260709 13:20:54.207289 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 200998us
I20260709 13:20:54.208026 16144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 12 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:20:54.208397 16191 leader_election.cc:400] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 10 pre-election: Vote denied by peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741) with higher term. Message: Invalid argument: T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac for earlier term 10. Current term is 11.
I20260709 13:20:54.208782 16144 raft_consensus.cc:2393] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 12: Already voted for candidate cb82cc7bd6db46d980c75237a1402860 in this term.
I20260709 13:20:54.208945 16146 raft_consensus.cc:1240] T 08c74490c40243a88e62d1760b5cd814 P 647a343a542d4a139fe106c19095129b [term 10 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 9. Current term is 10. Ops: [9.7-9.7]
I20260709 13:20:54.210134 16144 raft_consensus.cc:1240] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 10. Current term is 11. Ops: [7.7-10.8]
I20260709 13:20:54.210458 16191 leader_election.cc:304] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:20:54.210753 16146 raft_consensus.cc:1240] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 8. Current term is 9. Ops: [7.7-8.8]
I20260709 13:20:54.140060 16223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18ad7e3af5504571a69d10c00750a01c" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:54.002444 16143 raft_consensus.cc:1275] T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:20:54.211665 16223 raft_consensus.cc:2410] T 18ad7e3af5504571a69d10c00750a01c P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b for term 11 because replica has last-logged OpId of term: 9 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 6.
I20260709 13:20:54.156889 16148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5ad4eb281564936a59f41e387a47156" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:54.214111 16818 raft_consensus.cc:1240] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860 [term 10 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 9. Current term is 10. Ops: []
W20260709 13:20:54.166919 16117 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:54.168882 20315 consensus_queue.cc:1048] T 7c2095a5f4ad49408dbc7f69c606120d P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.011s
I20260709 13:20:54.174160 16291 raft_consensus.cc:1240] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 90f3da57796b488c9087a2f523adf045 [term 14 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 13. Current term is 14. Ops: []
I20260709 13:20:54.156096 20573 consensus_queue.cc:1059] T c0eab25ce4c64e018a041beeda2867db P 90f3da57796b488c9087a2f523adf045 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 3, Time since last communication: 0.019s
W20260709 13:20:54.175618 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 145439us
I20260709 13:20:54.172116 16265 leader_election.cc:304] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
W20260709 13:20:54.182914 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 181618us
W20260709 13:20:54.169368 20505 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.776s	user 0.010s	sys 0.033s
I20260709 13:20:54.175678 20443 raft_consensus.cc:2804] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:20:54.189198 16220 pending_rounds.cc:85] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac: Aborting all ops after (but not including) 8
I20260709 13:20:54.193481 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f82224f4564fea93572e8206bad1fe" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:20:54.155467 20571 raft_consensus.cc:515] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:54.169939 20267 raft_consensus.cc:515] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:20:54.243057 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 83103.5us
I20260709 13:20:54.244297 16265 leader_election.cc:304] T 18ad7e3af5504571a69d10c00750a01c P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:20:54.205324 16147 raft_consensus.cc:2468] T 746aabf856ab401c9ef5e859371e7b67 P 647a343a542d4a139fe106c19095129b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 10.
I20260709 13:20:54.193360 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a9de347c711477e87c582d003f36925" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 14 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:54.157967 16150 raft_consensus.cc:2468] T 18ad7e3af5504571a69d10c00750a01c P 647a343a542d4a139fe106c19095129b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 10.
W20260709 13:20:54.167433 16192 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 151081us
I20260709 13:20:54.213539 16143 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40844 (request call id 2356) took 5146 ms. Trace:
I20260709 13:20:54.179910 20422 raft_consensus.cc:2749] T 08c74490c40243a88e62d1760b5cd814 P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:20:54.221138 16146 raft_consensus.cc:1240] T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 10. Current term is 11. Ops: []
I20260709 13:20:54.186913 20457 raft_consensus.cc:2749] T fffab26427e24d398e2695aad40ca134 P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:20:54.231405 16294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68a6838d82ef4cac83507f906139d90f" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:20:54.184542 20268 raft_consensus.cc:493] T 8c11074579b046d99e585c2818e18a2c P cb82cc7bd6db46d980c75237a1402860 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:54.236289 16145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5d59407eb545b895d8d922e0dd69fd" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:54.254451 16145 raft_consensus.cc:2410] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac for term 12 because replica has last-logged OpId of term: 11 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 10.
I20260709 13:20:54.239379 16220 pending_rounds.cc:107] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac: Aborting uncommitted NO_OP operation due to leader change: 11.9
I20260709 13:20:54.254835 16220 raft_consensus.cc:2889] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:20:54.234273 16115 leader_election.cc:304] T b87147731a68400e8cce9b879daa7c5d P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:20:54.245730 16821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31969392ccdb43b491911a319e8bc7e1" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:54.242681 16047 catalog_manager.cc:5697] T 434bff3082d141838068f323bdb0d1ed P cb82cc7bd6db46d980c75237a1402860 reported cstate change: term changed from 7 to 9. New cstate: current_term: 9 leader_uuid: "cb82cc7bd6db46d980c75237a1402860" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:20:54.247908 16147 raft_consensus.cc:1240] T 5a96373bb0e14dce8a266b2fc6c58507 P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 10. Current term is 11. Ops: []
I20260709 13:20:54.248835 16150 raft_consensus.cc:1240] T ee9ed67006144797b1de6ea8c2013a68 P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 8. Current term is 9. Ops: []
I20260709 13:20:54.246061 16143 rpcz_store.cc:276] 0709 13:20:49.066989 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:52.214130 (+3147141us) service_pool.cc:224] Handling call
0709 13:20:54.002038 (+1787908us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:54.211328 (+209290us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:54.213300 (+  1972us) spinlock_profiling.cc:243] Waited 1.79 s on lock 0x7b540041ebd0. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5e8ad 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:20:54.213469 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4110780800}
I20260709 13:20:54.252171 16142 raft_consensus.cc:2468] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 15.
I20260709 13:20:54.259750 16818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e65676b21b748728ad39668cfe722ec" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:54.258762 16295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57a744f5aaf944678a63469cbf514015" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:54.262538 16192 leader_election.cc:304] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:20:54.260063 16115 leader_election.cc:304] T 3884f864a96d428785895e5463e5728f P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:20:54.260380 16818 raft_consensus.cc:2468] T 9e65676b21b748728ad39668cfe722ec P cb82cc7bd6db46d980c75237a1402860 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 10.
I20260709 13:20:54.265775 16192 leader_election.cc:304] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:20:54.266170 16147 raft_consensus.cc:1275] T ee9ed67006144797b1de6ea8c2013a68 P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:20:54.189630 20487 raft_consensus.cc:515] T 26354361e9b044cea1c2b86c4882fe92 P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:54.244313 20578 raft_consensus.cc:2749] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:20:54.239109 20443 raft_consensus.cc:493] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Starting leader election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:20:54.268512 16116 leader_election.cc:304] T 84f2da98864b4425bf7c0055951fdd4b P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:20:54.278193 16116 leader_election.cc:304] T 5a959dbecc2648eb85d1cc550f5a3245 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:20:54.274024 16265 leader_election.cc:304] T 02838c86531d4f98bf58d1c22b8baf77 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:20:54.252372 20518 raft_consensus.cc:3055] T c0eab25ce4c64e018a041beeda2867db P 90f3da57796b488c9087a2f523adf045 [term 10 LEADER]: Stepping down as leader of term 10
W20260709 13:20:54.267907 16790 leader_election.cc:336] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943): Timed out: RequestConsensusVote RPC to 127.14.177.130:35943 timed out after 2.078s (SENT)
I20260709 13:20:54.280076 16822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746aabf856ab401c9ef5e859371e7b67" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:54.274936 16151 raft_consensus.cc:2468] T c0eab25ce4c64e018a041beeda2867db P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 11.
I20260709 13:20:54.283504 16144 raft_consensus.cc:1240] T c0eab25ce4c64e018a041beeda2867db P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 10. Current term is 11. Ops: [10.7-10.7]
I20260709 13:20:54.276010 16115 leader_election.cc:304] T 7c2095a5f4ad49408dbc7f69c606120d P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:20:54.285264 16217 raft_consensus.cc:2410] T 08c74490c40243a88e62d1760b5cd814 P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b for term 11 because replica has last-logged OpId of term: 9 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 6.
I20260709 13:20:54.258319 20268 raft_consensus.cc:3060] T 8c11074579b046d99e585c2818e18a2c P cb82cc7bd6db46d980c75237a1402860 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:20:54.290478 16116 leader_election.cc:304] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
I20260709 13:20:54.280416 20443 raft_consensus.cc:3060] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:20:54.293879 19982 raft_consensus.cc:2749] T 5a959dbecc2648eb85d1cc550f5a3245 P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:20:54.294204 20579 raft_consensus.cc:2804] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:20:54.294633 20231 raft_consensus.cc:2804] T 7c2095a5f4ad49408dbc7f69c606120d P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:20:54.295133 20231 raft_consensus.cc:493] T 7c2095a5f4ad49408dbc7f69c606120d P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Starting leader election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:20:54.295590 20231 raft_consensus.cc:3060] T 7c2095a5f4ad49408dbc7f69c606120d P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:20:54.296663 20565 consensus_queue.cc:1048] T e5ad4eb281564936a59f41e387a47156 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:20:54.296779 20269 raft_consensus.cc:515] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:54.298826 16823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2372981006d4b218215c28964dea623" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 13 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:54.299414 16822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3884f864a96d428785895e5463e5728f" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:54.300325 16817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b7d1ab519044eda2d5e04a07242f25" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:54.260469 20434 raft_consensus.cc:2804] T b87147731a68400e8cce9b879daa7c5d P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:20:54.272764 20475 consensus_queue.cc:1048] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 7, Time since last communication: 0.022s
I20260709 13:20:54.294765 20388 leader_election.cc:290] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:54.300967 16817 raft_consensus.cc:2468] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 9.
I20260709 13:20:54.305385 16116 leader_election.cc:304] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:20:54.305593 16818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:54.306386 16190 leader_election.cc:400] T e5ad4eb281564936a59f41e387a47156 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 11 election: Vote denied by peer 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389) with higher term. Message: Invalid argument: T e5ad4eb281564936a59f41e387a47156 P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac for earlier term 11. Current term is 12.
I20260709 13:20:54.306756 16190 leader_election.cc:403] T e5ad4eb281564936a59f41e387a47156 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 11 election: Cancelling election due to peer responding with higher term
I20260709 13:20:54.310149 16825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 13 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:54.320449 20586 raft_consensus.cc:2804] T 18ad7e3af5504571a69d10c00750a01c P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 13:20:54.321038 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 47078.4us
I20260709 13:20:54.323127 16821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5d59407eb545b895d8d922e0dd69fd" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:54.316208 16291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ad5dbfa99ee4b8d9e28be92b3f36db5" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:54.311015 16825 raft_consensus.cc:2468] T 19baa7bc367844c288bca8f81439c413 P cb82cc7bd6db46d980c75237a1402860 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 12.
I20260709 13:20:54.285480 20457 raft_consensus.cc:2749] T 84f2da98864b4425bf7c0055951fdd4b P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:20:54.286288 20472 raft_consensus.cc:2804] T 3884f864a96d428785895e5463e5728f P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:20:54.279731 20477 leader_election.cc:290] T b6f07842d3aa47858df5665874c86f3a P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 13 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:54.280066 20518 raft_consensus.cc:740] T c0eab25ce4c64e018a041beeda2867db P 90f3da57796b488c9087a2f523adf045 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:20:54.324065 16291 raft_consensus.cc:2468] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 9.
I20260709 13:20:54.332983 16817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3884f864a96d428785895e5463e5728f" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:20:54.328136 16825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87147731a68400e8cce9b879daa7c5d" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:54.303973 20434 raft_consensus.cc:493] T b87147731a68400e8cce9b879daa7c5d P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:54.329702 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6f07842d3aa47858df5665874c86f3a" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:20:54.336366 16298 raft_consensus.cc:3060] T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:20:54.294651 20579 raft_consensus.cc:493] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Starting leader election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:20:54.328166 16821 raft_consensus.cc:2468] T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 12.
I20260709 13:20:54.326148 16265 leader_election.cc:304] T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:20:54.334167 16825 raft_consensus.cc:2468] T b87147731a68400e8cce9b879daa7c5d P cb82cc7bd6db46d980c75237a1402860 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 12.
I20260709 13:20:54.334399 16115 leader_election.cc:304] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:20:54.337327 20586 raft_consensus.cc:493] T 18ad7e3af5504571a69d10c00750a01c P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:54.341140 20586 raft_consensus.cc:3060] T 18ad7e3af5504571a69d10c00750a01c P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:20:54.335537 16818 raft_consensus.cc:1275] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860 [term 13 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:20:54.309876 19982 raft_consensus.cc:2804] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:20:54.344888 19982 raft_consensus.cc:493] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:54.345255 19982 raft_consensus.cc:3060] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:20:54.347258 20589 raft_consensus.cc:2804] T 02838c86531d4f98bf58d1c22b8baf77 P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:20:54.301590 20576 raft_consensus.cc:2804] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:20:54.350646 20576 raft_consensus.cc:493] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:20:54.351044 20576 raft_consensus.cc:3060] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:20:54.352298 16265 leader_election.cc:304] T b87147731a68400e8cce9b879daa7c5d P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:20:54.353440 16265 leader_election.cc:304] T 5a959dbecc2648eb85d1cc550f5a3245 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:20:54.353674 19982 raft_consensus.cc:515] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:54.334487 16192 leader_election.cc:304] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:20:54.355619 20584 raft_consensus.cc:2804] T b87147731a68400e8cce9b879daa7c5d P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:20:54.356062 20584 raft_consensus.cc:493] T b87147731a68400e8cce9b879daa7c5d P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:20:54.349089 20586 raft_consensus.cc:515] T 18ad7e3af5504571a69d10c00750a01c P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } }
I20260709 13:20:54.302105 20269 leader_election.cc:290] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 10 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:54.325203 20422 raft_consensus.cc:2804] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:20:54.379117 16823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5718c623ebee4d2bb770f0f9ddb2f164" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 12 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
W20260709 13:20:54.380070 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 169706us
I20260709 13:20:54.349254 20588 raft_consensus.cc:2804] T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 13:20:54.358886 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 54320us
I20260709 13:20:54.381171 16116 leader_election.cc:304] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:20:54.382360 16144 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40830 (request call id 1485) took 1126 ms. Trace:
I20260709 13:20:54.382692 16144 rpcz_store.cc:276] 0709 13:20:53.250100 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:54.210966 (+960866us) service_pool.cc:224] Handling call
0709 13:20:54.283293 (+ 72327us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:20:54.283886 (+   593us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:54.358203 (+ 74317us) spinlock_profiling.cc:243] Waited 71.2 ms on lock 0x7b54000735d0. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5e8ad 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:20:54.376000 (+ 17797us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":163718528}
I20260709 13:20:54.368203 16190 leader_election.cc:304] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
W20260709 13:20:54.376116 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 114845us
W20260709 13:20:54.376607 16114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:54.329051 20388 raft_consensus.cc:2749] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
W20260709 13:20:54.386679 16788 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:54.375702 16820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62fe3346cd3c42c4a55e14af9073bca4" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:54.386615 16821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f82224f4564fea93572e8206bad1fe" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:20:54.389899 16820 raft_consensus.cc:2393] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 15: Already voted for candidate cb82cc7bd6db46d980c75237a1402860 in this term.
I20260709 13:20:54.390314 16819 raft_consensus.cc:2468] T 434bff3082d141838068f323bdb0d1ed P cb82cc7bd6db46d980c75237a1402860 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 10.
I20260709 13:20:54.339406 20434 raft_consensus.cc:3060] T b87147731a68400e8cce9b879daa7c5d P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:20:54.392189 16822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31969392ccdb43b491911a319e8bc7e1" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:20:54.337785 20472 raft_consensus.cc:697] T 3884f864a96d428785895e5463e5728f P 647a343a542d4a139fe106c19095129b [term 8 LEADER]: Becoming Leader. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:20:54.394356 19982 leader_election.cc:290] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 13 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:54.357347 20579 raft_consensus.cc:3060] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:20:54.402748 16295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26354361e9b044cea1c2b86c4882fe92" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:54.380782 20427 raft_consensus.cc:2804] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:20:54.378489 20584 raft_consensus.cc:3060] T b87147731a68400e8cce9b879daa7c5d P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:20:54.410367 16295 raft_consensus.cc:2468] T 26354361e9b044cea1c2b86c4882fe92 P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 9.
I20260709 13:20:54.381963 20589 raft_consensus.cc:697] T 02838c86531d4f98bf58d1c22b8baf77 P 90f3da57796b488c9087a2f523adf045 [term 11 LEADER]: Becoming Leader. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:20:54.405691 16823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57a744f5aaf944678a63469cbf514015" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:54.410594 16191 leader_election.cc:304] T b6f07842d3aa47858df5665874c86f3a P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:20:54.413381 16116 leader_election.cc:304] T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
W20260709 13:20:54.408736 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 53708.4us
I20260709 13:20:54.427026 16265 leader_election.cc:304] T 2794af5b99984efebdee564035841cb0 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:20:54.427249 16116 leader_election.cc:304] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
I20260709 13:20:54.383414 20573 raft_consensus.cc:2804] T 5a959dbecc2648eb85d1cc550f5a3245 P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:20:54.384390 20588 raft_consensus.cc:493] T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:20:54.446417 16115 leader_election.cc:304] T 26354361e9b044cea1c2b86c4882fe92 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:20:54.451084 16823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434bff3082d141838068f323bdb0d1ed" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:20:54.451850 16823 raft_consensus.cc:2376] T 434bff3082d141838068f323bdb0d1ed P cb82cc7bd6db46d980c75237a1402860 [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 647a343a542d4a139fe106c19095129b in term 10. Re-sending same reply.
I20260709 13:20:54.452252 16265 leader_election.cc:304] T 84f2da98864b4425bf7c0055951fdd4b P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:20:54.430655 20388 raft_consensus.cc:2749] T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:20:54.457280 16116 leader_election.cc:304] T c0eab25ce4c64e018a041beeda2867db P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:20:54.430601 20269 raft_consensus.cc:2804] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:20:54.430076 20427 raft_consensus.cc:493] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Starting leader election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:20:54.430444 19982 raft_consensus.cc:2749] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:20:54.430845 20573 raft_consensus.cc:493] T 5a959dbecc2648eb85d1cc550f5a3245 P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:20:54.437799 20578 raft_consensus.cc:2804] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:20:54.442435 20236 raft_consensus.cc:2804] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:20:54.460508 20269 raft_consensus.cc:493] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:54.438073 20588 raft_consensus.cc:3060] T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:20:54.458213 16790 leader_election.cc:336] T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 12 election: RPC error from VoteRequest() call to peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943): Timed out: RequestConsensusVote RPC to 127.14.177.130:35943 timed out after 2.189s (SENT)
I20260709 13:20:54.469522 16790 leader_election.cc:304] T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:20:54.450486 19861 raft_consensus.cc:2749] T b6f07842d3aa47858df5665874c86f3a P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:20:54.463819 16816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f82224f4564fea93572e8206bad1fe" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:54.465579 16294 raft_consensus.cc:1275] T 434bff3082d141838068f323bdb0d1ed P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:20:54.463591 16824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "829b4ad8c5a54eababeea1181b960d4b" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:54.463892 20427 raft_consensus.cc:3060] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:20:54.474475 16116 leader_election.cc:304] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:20:54.488914 16823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746aabf856ab401c9ef5e859371e7b67" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:54.489156 16116 leader_election.cc:304] T 5a96373bb0e14dce8a266b2fc6c58507 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
W20260709 13:20:54.495669 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 94114.8us
I20260709 13:20:54.496668 16116 leader_election.cc:304] T 9c61a39fd4f84424b129c1a6f9ba404d P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:20:54.468770 20573 raft_consensus.cc:3060] T 5a959dbecc2648eb85d1cc550f5a3245 P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:20:54.471505 20236 raft_consensus.cc:493] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Starting leader election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:20:54.471830 20269 raft_consensus.cc:3060] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:20:54.464810 20422 raft_consensus.cc:697] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [term 12 LEADER]: Becoming Leader. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
W20260709 13:20:54.502519 16192 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 170708us
I20260709 13:20:54.503786 16191 leader_election.cc:304] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:20:54.475519 20267 leader_election.cc:290] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:20:54.476181 20487 leader_election.cc:290] T 26354361e9b044cea1c2b86c4882fe92 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:54.490195 20565 raft_consensus.cc:2749] T c0eab25ce4c64e018a041beeda2867db P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:20:54.453469 20518 consensus_queue.cc:260] T c0eab25ce4c64e018a041beeda2867db P 90f3da57796b488c9087a2f523adf045 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:54.512405 16823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5d59407eb545b895d8d922e0dd69fd" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:20:54.498477 20560 raft_consensus.cc:2749] T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:20:54.499513 20388 raft_consensus.cc:2749] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:20:54.491355 20578 raft_consensus.cc:697] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac [term 13 LEADER]: Becoming Leader. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:20:54.514600 16116 leader_election.cc:304] T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:20:54.508803 20236 raft_consensus.cc:3060] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:20:54.521447 16824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e65676b21b748728ad39668cfe722ec" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:54.522220 16824 raft_consensus.cc:2468] T 9e65676b21b748728ad39668cfe722ec P cb82cc7bd6db46d980c75237a1402860 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 10.
I20260709 13:20:54.515333 16047 catalog_manager.cc:5697] T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860 reported cstate change: term changed from 6 to 11, leader changed from 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130) to cb82cc7bd6db46d980c75237a1402860 (127.14.177.132). New cstate: current_term: 11 leader_uuid: "cb82cc7bd6db46d980c75237a1402860" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:20:54.518702 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 65061.4us
I20260709 13:20:54.523397 20597 raft_consensus.cc:2749] T 5a96373bb0e14dce8a266b2fc6c58507 P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:20:54.523561 20583 raft_consensus.cc:493] T 1df93779209748b9860f6d893dd3de53 P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:54.524183 20583 raft_consensus.cc:515] T 1df93779209748b9860f6d893dd3de53 P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:54.525635 20422 consensus_queue.cc:237] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:54.526360 20583 leader_election.cc:290] T 1df93779209748b9860f6d893dd3de53 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:20:54.526805 20553 consensus_queue.cc:1059] T 829b4ad8c5a54eababeea1181b960d4b P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
W20260709 13:20:54.532837 16264 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:54.516877 20267 raft_consensus.cc:2804] T 2794af5b99984efebdee564035841cb0 P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:20:54.517526 19861 raft_consensus.cc:2749] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:20:54.536214 16823 raft_consensus.cc:3060] T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:20:54.526999 20583 raft_consensus.cc:2749] T 84f2da98864b4425bf7c0055951fdd4b P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:20:54.517026 20571 leader_election.cc:290] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:54.542912 16824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31969392ccdb43b491911a319e8bc7e1" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:54.555339 20598 raft_consensus.cc:2749] T 9c61a39fd4f84424b129c1a6f9ba404d P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:20:54.548774 16816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26354361e9b044cea1c2b86c4882fe92" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:54.519193 20487 consensus_queue.cc:1048] T 434bff3082d141838068f323bdb0d1ed P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.018s
I20260709 13:20:54.535344 20267 raft_consensus.cc:493] T 2794af5b99984efebdee564035841cb0 P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Starting leader election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:20:54.558069 20267 raft_consensus.cc:3060] T 2794af5b99984efebdee564035841cb0 P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:20:54.549571 20269 raft_consensus.cc:515] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:54.552701 16117 leader_election.cc:304] T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860
W20260709 13:20:54.553797 20590 log.cc:927] Time spent T e5ad4eb281564936a59f41e387a47156 P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.089s	user 0.001s	sys 0.000s
I20260709 13:20:54.521929 20424 raft_consensus.cc:3055] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:20:54.518803 20577 raft_consensus.cc:2749] T e5ad4eb281564936a59f41e387a47156 P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Leader election lost for term 11. Reason: Vote denied by peer 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389) with higher term. Message: Invalid argument: T e5ad4eb281564936a59f41e387a47156 P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac for earlier term 11. Current term is 12.
I20260709 13:20:54.563351 16291 raft_consensus.cc:1240] T 3884f864a96d428785895e5463e5728f P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 7. Current term is 8. Ops: []
I20260709 13:20:54.564852 16117 leader_election.cc:304] T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:20:54.577324 20602 raft_consensus.cc:2804] T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:20:54.577780 20602 raft_consensus.cc:493] T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:54.566452 16298 raft_consensus.cc:2468] T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 13.
I20260709 13:20:54.578245 20602 raft_consensus.cc:3060] T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:20:54.578828 20565 consensus_queue.cc:1048] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:20:54.579604 20565 consensus_queue.cc:1243] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b [LEADER]: Peer cb82cc7bd6db46d980c75237a1402860 log is divergent from this leader: its last log entry 10.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 13:20:54.582798 16115 leader_election.cc:304] T b6f07842d3aa47858df5665874c86f3a P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:20:54.509428 20472 consensus_queue.cc:237] T 3884f864a96d428785895e5463e5728f P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:54.536273 20388 raft_consensus.cc:2749] T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:20:54.578020 16117 leader_election.cc:304] T 23f82224f4564fea93572e8206bad1fe P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860
I20260709 13:20:54.591082 16117 leader_election.cc:304] T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860
I20260709 13:20:54.589784 16299 raft_consensus.cc:1275] T 8638ee363273426db1393dd81e5c7b72 P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
W20260709 13:20:54.587513 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 219428us
I20260709 13:20:54.592569 16190 leader_election.cc:304] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860
I20260709 13:20:54.563954 20586 leader_election.cc:290] T 18ad7e3af5504571a69d10c00750a01c P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 11 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:20:54.568156 20475 raft_consensus.cc:2804] T 26354361e9b044cea1c2b86c4882fe92 P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 13:20:54.595395 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 152507us
W20260709 13:20:54.596403 16266 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:20:54.596359 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 81853.2us
I20260709 13:20:54.597800 16298 raft_consensus.cc:1275] T 8a878999c0024d2abe517484180290fc P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:20:54.598223 16117 leader_election.cc:304] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860
I20260709 13:20:54.600739 16117 leader_election.cc:304] T 57a744f5aaf944678a63469cbf514015 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860
I20260709 13:20:54.602187 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66ae176961b74813a58bc15c9c4c6f2a" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:20:54.568063 20424 raft_consensus.cc:740] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:20:54.583842 20487 raft_consensus.cc:2749] T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:20:54.609536 16247 debug-util.cc:398] Leaking SignalData structure 0x7b0800093a80 after lost signal to thread 15046
I20260709 13:20:54.598462 19861 consensus_queue.cc:1059] T 8a878999c0024d2abe517484180290fc P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.012s
I20260709 13:20:54.605646 20388 raft_consensus.cc:2749] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:20:54.617404 20388 raft_consensus.cc:2749] T 57a744f5aaf944678a63469cbf514015 P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:20:54.598429 20231 raft_consensus.cc:515] T 7c2095a5f4ad49408dbc7f69c606120d P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:54.618453 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c11074579b046d99e585c2818e18a2c" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 12 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:54.612123 20443 raft_consensus.cc:515] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:54.627792 16222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02838c86531d4f98bf58d1c22b8baf77" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:20:54.596897 20598 raft_consensus.cc:2749] T 23f82224f4564fea93572e8206bad1fe P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:20:54.613389 16217 raft_consensus.cc:2368] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 for earlier term 13. Current term is 14.
I20260709 13:20:54.602447 20583 raft_consensus.cc:493] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:20:54.616096 16247 debug-util.cc:398] Leaking SignalData structure 0x7b0800099760 after lost signal to thread 15049
W20260709 13:20:54.638713 16247 debug-util.cc:398] Leaking SignalData structure 0x7b08000de0a0 after lost signal to thread 16083
I20260709 13:20:54.638863 16223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a378d7f796f94615b3b347119a9880fd" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
W20260709 13:20:54.640058 16247 debug-util.cc:398] Leaking SignalData structure 0x7b08000de120 after lost signal to thread 16175
W20260709 13:20:54.640358 16247 debug-util.cc:398] Leaking SignalData structure 0x7b08000de1a0 after lost signal to thread 16176
W20260709 13:20:54.641362 16247 debug-util.cc:398] Leaking SignalData structure 0x7b08003d3020 after lost signal to thread 16250
W20260709 13:20:54.641633 16247 debug-util.cc:398] Leaking SignalData structure 0x7b08003ddcc0 after lost signal to thread 16251
W20260709 13:20:54.642628 16247 debug-util.cc:398] Leaking SignalData structure 0x7b08003b19a0 after lost signal to thread 16324
W20260709 13:20:54.642902 16247 debug-util.cc:398] Leaking SignalData structure 0x7b0800108aa0 after lost signal to thread 16325
I20260709 13:20:54.622514 16117 leader_election.cc:304] T 829b4ad8c5a54eababeea1181b960d4b P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860
I20260709 13:20:54.592306 20597 raft_consensus.cc:2804] T b6f07842d3aa47858df5665874c86f3a P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:20:54.517292 20589 consensus_queue.cc:237] T 02838c86531d4f98bf58d1c22b8baf77 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:54.614967 16220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9944d5e4b8c4109af9955c4f08c9095" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:54.599330 20565 raft_consensus.cc:2749] T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
W20260709 13:20:54.636637 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 44169.9us
I20260709 13:20:54.656371 16190 leader_election.cc:304] T 18ad7e3af5504571a69d10c00750a01c P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:20:54.653951 20571 consensus_queue.cc:1048] T 746aabf856ab401c9ef5e859371e7b67 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.003s
I20260709 13:20:54.657750 16190 leader_election.cc:304] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:20:54.658241 20571 raft_consensus.cc:2804] T 18ad7e3af5504571a69d10c00750a01c P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:20:54.658619 20570 raft_consensus.cc:2804] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:20:54.658681 20571 raft_consensus.cc:493] T 18ad7e3af5504571a69d10c00750a01c P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:54.663919 16823 raft_consensus.cc:2468] T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 13.
W20260709 13:20:54.664517 16247 debug-util.cc:398] Leaking SignalData structure 0x7b0800355500 after lost signal to thread 16849
I20260709 13:20:54.651454 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:20:54.651247 20583 raft_consensus.cc:515] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:54.651742 20599 consensus_queue.cc:1059] T c0eab25ce4c64e018a041beeda2867db P 90f3da57796b488c9087a2f523adf045 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:20:54.652026 20603 raft_consensus.cc:493] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:54.656900 20443 leader_election.cc:290] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 12 election: Requested vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:54.659204 20570 raft_consensus.cc:493] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:54.659842 20576 raft_consensus.cc:515] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:54.644084 16222 raft_consensus.cc:3060] T 02838c86531d4f98bf58d1c22b8baf77 P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:20:54.694177 16263 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:54.635290 20293 raft_consensus.cc:2749] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
W20260709 13:20:54.647481 16265 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:54.664363 16117 leader_election.cc:304] T 9e65676b21b748728ad39668cfe722ec P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860; no voters: 90f3da57796b488c9087a2f523adf045
I20260709 13:20:54.650507 20236 raft_consensus.cc:515] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:54.707967 20475 raft_consensus.cc:493] T 26354361e9b044cea1c2b86c4882fe92 P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Starting leader election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:20:54.708523 20475 raft_consensus.cc:3060] T 26354361e9b044cea1c2b86c4882fe92 P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:20:54.708922 20236 leader_election.cc:290] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 12 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:54.665398 16217 raft_consensus.cc:2393] T 19baa7bc367844c288bca8f81439c413 P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 12: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
W20260709 13:20:54.665763 16247 debug-util.cc:398] Leaking SignalData structure 0x7b08003b0000 after lost signal to thread 16850
W20260709 13:20:54.710124 16247 debug-util.cc:398] Leaking SignalData structure 0x7b08001ea7e0 after lost signal to thread 20460
W20260709 13:20:54.710731 16247 debug-util.cc:398] Leaking SignalData structure 0x7b0800297560 after lost signal to thread 20528
I20260709 13:20:54.711536 20576 leader_election.cc:290] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 12 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:20:54.669293 20571 raft_consensus.cc:3060] T 18ad7e3af5504571a69d10c00750a01c P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:20:54.713728 20600 raft_consensus.cc:3060] T 829b4ad8c5a54eababeea1181b960d4b P 647a343a542d4a139fe106c19095129b [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:20:54.666330 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 58841.1us
I20260709 13:20:54.715636 16116 leader_election.cc:304] T 8cc959fb7f704fba9009c0595467d074 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:20:54.692406 16223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7ca61592e848ea9b0fb81a2ce33cdc" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:20:54.663218 16190 leader_election.cc:304] T d16dfad0c3b540e5b4b86492ddf87208 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
W20260709 13:20:54.719568 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 56462us
I20260709 13:20:54.693857 20570 raft_consensus.cc:3060] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:20:54.691918 20427 raft_consensus.cc:515] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:54.713052 20605 consensus_queue.cc:1048] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
W20260709 13:20:54.720707 16789 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:54.737443 16147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1df93779209748b9860f6d893dd3de53" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:54.738729 16819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:20:54.739490 16819 raft_consensus.cc:2393] T 19baa7bc367844c288bca8f81439c413 P cb82cc7bd6db46d980c75237a1402860 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 12: Already voted for candidate cb82cc7bd6db46d980c75237a1402860 in this term.
I20260709 13:20:54.739898 16818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2794af5b99984efebdee564035841cb0" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:54.740397 16144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18ad7e3af5504571a69d10c00750a01c" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:20:54.740978 16144 raft_consensus.cc:3060] T 18ad7e3af5504571a69d10c00750a01c P 647a343a542d4a139fe106c19095129b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:20:54.741503 16148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ad5dbfa99ee4b8d9e28be92b3f36db5" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:54.723419 16116 leader_election.cc:304] T 8158bbd17b994d2eb96f560bd0f1b88d P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
W20260709 13:20:54.695505 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 86124.2us
I20260709 13:20:54.687023 20597 raft_consensus.cc:697] T b6f07842d3aa47858df5665874c86f3a P 647a343a542d4a139fe106c19095129b [term 13 LEADER]: Becoming Leader. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:20:54.738121 16147 raft_consensus.cc:2468] T 1df93779209748b9860f6d893dd3de53 P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 11.
I20260709 13:20:54.750058 16116 leader_election.cc:304] T 18ad7e3af5504571a69d10c00750a01c P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:20:54.750783 16825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87147731a68400e8cce9b879daa7c5d" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:54.729996 20553 consensus_queue.cc:1048] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 3, Time since last communication: 0.018s
I20260709 13:20:54.751534 16116 leader_election.cc:304] T d16dfad0c3b540e5b4b86492ddf87208 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:20:54.730034 20315 raft_consensus.cc:2804] T d16dfad0c3b540e5b4b86492ddf87208 P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:20:54.755589 16790 leader_election.cc:304] T 19baa7bc367844c288bca8f81439c413 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:20:54.755980 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f5f89186204242bf377f7c42b2e2da" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:20:54.757345 16148 raft_consensus.cc:2393] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 10: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
I20260709 13:20:54.757090 16817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f82224f4564fea93572e8206bad1fe" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:54.752398 16822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7ca61592e848ea9b0fb81a2ce33cdc" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:20:54.729285 20598 raft_consensus.cc:2804] T 9e65676b21b748728ad39668cfe722ec P 647a343a542d4a139fe106c19095129b [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:20:54.755820 16825 raft_consensus.cc:2393] T b87147731a68400e8cce9b879daa7c5d P cb82cc7bd6db46d980c75237a1402860 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 12: Already voted for candidate cb82cc7bd6db46d980c75237a1402860 in this term.
I20260709 13:20:54.759884 16816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c11074579b046d99e585c2818e18a2c" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 12 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:54.760079 16818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8638ee363273426db1393dd81e5c7b72" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:54.731109 20565 raft_consensus.cc:2749] T 8cc959fb7f704fba9009c0595467d074 P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:20:54.755743 16824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a378d7f796f94615b3b347119a9880fd" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:20:54.692750 19861 consensus_queue.cc:1048] T af57996a8be84d398f0f023e8d08ef77 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.053s
I20260709 13:20:54.761807 16825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdbd78acb4cf46298e0de8ba0d130e19" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:54.770313 16825 raft_consensus.cc:2468] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 12.
I20260709 13:20:54.746429 16220 raft_consensus.cc:2468] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 11.
I20260709 13:20:54.760147 16819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26354361e9b044cea1c2b86c4882fe92" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:54.760699 16223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e4bfb9090c8460eb14fa7e0d818e7f1" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:54.764837 16821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e4bfb9090c8460eb14fa7e0d818e7f1" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:54.768711 16222 raft_consensus.cc:2468] T 02838c86531d4f98bf58d1c22b8baf77 P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 11.
I20260709 13:20:54.771512 16825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66ae176961b74813a58bc15c9c4c6f2a" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:20:54.773501 16265 leader_election.cc:304] T 1df93779209748b9860f6d893dd3de53 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:20:54.777760 16818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ab0ac8347641e6955e9bce7d90c1a2" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:20:54.644590 20578 consensus_queue.cc:237] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:54.759678 20315 raft_consensus.cc:493] T d16dfad0c3b540e5b4b86492ddf87208 P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:54.787283 20595 raft_consensus.cc:493] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:54.787278 16825 raft_consensus.cc:2368] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 for earlier term 13. Current term is 14.
I20260709 13:20:54.788031 20595 raft_consensus.cc:515] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:54.781721 20597 consensus_queue.cc:237] T b6f07842d3aa47858df5665874c86f3a P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:54.791085 20607 raft_consensus.cc:2749] T 19baa7bc367844c288bca8f81439c413 P cb82cc7bd6db46d980c75237a1402860 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:20:54.791888 20584 raft_consensus.cc:515] T b87147731a68400e8cce9b879daa7c5d P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:54.793452 20231 leader_election.cc:290] T 7c2095a5f4ad49408dbc7f69c606120d P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 9 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:54.782434 16822 raft_consensus.cc:3060] T 3a7ca61592e848ea9b0fb81a2ce33cdc P cb82cc7bd6db46d980c75237a1402860 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:20:54.713073 20602 raft_consensus.cc:515] T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:54.803853 20593 consensus_queue.cc:1048] T 8c11074579b046d99e585c2818e18a2c P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:20:54.804811 20602 leader_election.cc:290] T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 10 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:54.744078 20236 consensus_queue.cc:1048] T e5ad4eb281564936a59f41e387a47156 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.033s
I20260709 13:20:54.787526 20315 raft_consensus.cc:3060] T d16dfad0c3b540e5b4b86492ddf87208 P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:20:54.807641 16825 raft_consensus.cc:1275] T 806d7690c2b644959152074bb2cab0a7 P cb82cc7bd6db46d980c75237a1402860 [term 10 LEARNER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
W20260709 13:20:54.782531 16189 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:54.696321 20603 raft_consensus.cc:515] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:54.796684 16817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9944d5e4b8c4109af9955c4f08c9095" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:20:54.764020 20487 raft_consensus.cc:2749] T 8158bbd17b994d2eb96f560bd0f1b88d P 647a343a542d4a139fe106c19095129b [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:20:54.798416 20584 leader_election.cc:290] T b87147731a68400e8cce9b879daa7c5d P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 12 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:54.799422 20268 raft_consensus.cc:515] T 8c11074579b046d99e585c2818e18a2c P cb82cc7bd6db46d980c75237a1402860 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:54.806777 16222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f5f89186204242bf377f7c42b2e2da" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:54.767315 20598 raft_consensus.cc:493] T 9e65676b21b748728ad39668cfe722ec P 647a343a542d4a139fe106c19095129b [term 10 FOLLOWER]: Starting leader election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:20:54.805313 20595 leader_election.cc:290] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:54.795023 16292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c2095a5f4ad49408dbc7f69c606120d" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:20:54.831277 16292 raft_consensus.cc:3060] T 7c2095a5f4ad49408dbc7f69c606120d P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:20:54.750504 20422 consensus_queue.cc:1059] T 8a878999c0024d2abe517484180290fc P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.050s
I20260709 13:20:54.795756 16824 raft_consensus.cc:2468] T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 12.
I20260709 13:20:54.816701 16819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3884f864a96d428785895e5463e5728f" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:54.818606 16822 raft_consensus.cc:2410] T 3a7ca61592e848ea9b0fb81a2ce33cdc P cb82cc7bd6db46d980c75237a1402860 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 for term 14 because replica has last-logged OpId of term: 13 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 11.
I20260709 13:20:54.794624 20477 raft_consensus.cc:2749] T d16dfad0c3b540e5b4b86492ddf87208 P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:20:54.694967 20424 consensus_queue.cc:260] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:20:54.845300 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 89790.5us
W20260709 13:20:54.822707 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 71591.6us
I20260709 13:20:54.846439 16116 leader_election.cc:304] T 08c74490c40243a88e62d1760b5cd814 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:20:54.791910 20599 raft_consensus.cc:493] T d16dfad0c3b540e5b4b86492ddf87208 P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:54.851547 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746aabf856ab401c9ef5e859371e7b67" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:54.824591 16817 raft_consensus.cc:3060] T c9944d5e4b8c4109af9955c4f08c9095 P cb82cc7bd6db46d980c75237a1402860 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:20:54.824713 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a959dbecc2648eb85d1cc550f5a3245" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:54.795679 20576 raft_consensus.cc:2804] T 1df93779209748b9860f6d893dd3de53 P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:20:54.805420 20269 leader_election.cc:290] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 12 election: Requested vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:54.823071 16789 leader_election.cc:304] T 434bff3082d141838068f323bdb0d1ed P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 
W20260709 13:20:54.869729 16789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 46754.6us
I20260709 13:20:54.839811 20573 raft_consensus.cc:515] T 5a959dbecc2648eb85d1cc550f5a3245 P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:54.842900 16144 raft_consensus.cc:2468] T 18ad7e3af5504571a69d10c00750a01c P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 11.
I20260709 13:20:54.874241 16222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fffab26427e24d398e2695aad40ca134" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
W20260709 13:20:54.845765 16266 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:54.857717 16221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4dc6b0541ca431ebc757beac00521f1" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:54.860680 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:54.877848 16224 raft_consensus.cc:2468] T ecc1e363230043fe9fef390f0b979f20 P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 10.
I20260709 13:20:54.861524 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 13 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:54.859620 16219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b7d1ab519044eda2d5e04a07242f25" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:54.858702 16218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4f65cdf00b748bb963ee6e81ef43b79" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:54.805745 20267 raft_consensus.cc:515] T 2794af5b99984efebdee564035841cb0 P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:54.871532 16816 raft_consensus.cc:2393] T 8c11074579b046d99e585c2818e18a2c P cb82cc7bd6db46d980c75237a1402860 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 12: Already voted for candidate cb82cc7bd6db46d980c75237a1402860 in this term.
I20260709 13:20:54.922433 16789 leader_election.cc:304] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:20:54.882184 20573 leader_election.cc:290] T 5a959dbecc2648eb85d1cc550f5a3245 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 10 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:20:54.882721 20612 raft_consensus.cc:2804] T 434bff3082d141838068f323bdb0d1ed P cb82cc7bd6db46d980c75237a1402860 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:20:54.924448 16149 raft_consensus.cc:1240] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 10. Current term is 11. Ops: []
I20260709 13:20:54.887554 20599 raft_consensus.cc:515] T d16dfad0c3b540e5b4b86492ddf87208 P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:54.895457 20457 raft_consensus.cc:2749] T 18ad7e3af5504571a69d10c00750a01c P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:20:54.898047 20600 raft_consensus.cc:2749] T 829b4ad8c5a54eababeea1181b960d4b P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:20:54.900538 20603 leader_election.cc:290] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:20:54.905272 20472 consensus_queue.cc:1048] T 8a878999c0024d2abe517484180290fc P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:20:54.900882 16220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f5f89186204242bf377f7c42b2e2da" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:54.954879 16143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87ac0d1f7e8a419f9bcbc04aee0e067e" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:54.967763 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87147731a68400e8cce9b879daa7c5d" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:20:54.872422 20231 raft_consensus.cc:2749] T 08c74490c40243a88e62d1760b5cd814 P 647a343a542d4a139fe106c19095129b [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:20:54.908705 20268 leader_election.cc:290] T 8c11074579b046d99e585c2818e18a2c P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 12 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:54.911420 20571 raft_consensus.cc:515] T 18ad7e3af5504571a69d10c00750a01c P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } }
I20260709 13:20:54.912515 20583 leader_election.cc:290] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:20:54.913676 20601 raft_consensus.cc:3055] T 8a878999c0024d2abe517484180290fc P 1ca2725d290e47d68b1a4b609c9816ac [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:20:54.914764 20579 raft_consensus.cc:515] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:54.916579 16223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a378d7f796f94615b3b347119a9880fd" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
W20260709 13:20:54.919339 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 256598us
I20260709 13:20:54.974299 16223 raft_consensus.cc:2393] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 12: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
I20260709 13:20:54.878844 16225 raft_consensus.cc:2468] T 19baa7bc367844c288bca8f81439c413 P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 12.
I20260709 13:20:54.927400 16219 raft_consensus.cc:2393] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 10: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
I20260709 13:20:54.929049 16147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a959dbecc2648eb85d1cc550f5a3245" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:20:54.975720 16147 raft_consensus.cc:3060] T 5a959dbecc2648eb85d1cc550f5a3245 P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:20:54.929247 16144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e9b2bf7011f4ae688a79d7bb5a447b4" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:54.930130 16146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c74490c40243a88e62d1760b5cd814" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:54.841231 20598 raft_consensus.cc:3060] T 9e65676b21b748728ad39668cfe722ec P 647a343a542d4a139fe106c19095129b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:20:54.932005 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5d59407eb545b895d8d922e0dd69fd" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:20:54.935014 16818 raft_consensus.cc:2468] T d2ab0ac8347641e6955e9bce7d90c1a2 P cb82cc7bd6db46d980c75237a1402860 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 11.
I20260709 13:20:54.978433 16217 raft_consensus.cc:3060] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:20:54.939147 16265 leader_election.cc:336] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943): Timed out: RequestConsensusVote RPC to 127.14.177.130:35943 timed out after 1.539s (SENT)
I20260709 13:20:54.951697 16222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68a6838d82ef4cac83507f906139d90f" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
W20260709 13:20:54.955178 16789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 84614.5us
I20260709 13:20:54.869490 20576 raft_consensus.cc:493] T 1df93779209748b9860f6d893dd3de53 P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:54.982755 20571 leader_election.cc:290] T 18ad7e3af5504571a69d10c00750a01c P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 11 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:54.971859 16292 raft_consensus.cc:2468] T 7c2095a5f4ad49408dbc7f69c606120d P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 9.
I20260709 13:20:54.976431 16144 raft_consensus.cc:2468] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 9.
I20260709 13:20:54.977386 16143 raft_consensus.cc:2468] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 12.
I20260709 13:20:54.986434 16222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c2095a5f4ad49408dbc7f69c606120d" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:20:54.987711 16222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ad5dbfa99ee4b8d9e28be92b3f36db5" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:54.988149 16191 leader_election.cc:304] T 829b4ad8c5a54eababeea1181b960d4b P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
I20260709 13:20:54.988765 16222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b7d1ab519044eda2d5e04a07242f25" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:20:54.989639 16191 leader_election.cc:304] T 02838c86531d4f98bf58d1c22b8baf77 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:20:54.990633 16222 raft_consensus.cc:2393] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 10: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
I20260709 13:20:54.930863 20612 raft_consensus.cc:493] T 434bff3082d141838068f323bdb0d1ed P cb82cc7bd6db46d980c75237a1402860 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:54.979676 16789 leader_election.cc:304] T d2ab0ac8347641e6955e9bce7d90c1a2 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:20:54.986538 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:20:54.993263 16224 raft_consensus.cc:2393] T 19baa7bc367844c288bca8f81439c413 P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 12: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
I20260709 13:20:54.994331 16219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02838c86531d4f98bf58d1c22b8baf77" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:20:54.966162 20424 raft_consensus.cc:3060] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:20:55.003860 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 64774.8us
W20260709 13:20:55.004873 16265 leader_election.cc:336] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 12 election: RPC error from VoteRequest() call to peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943): Timed out: RequestConsensusVote RPC to 127.14.177.130:35943 timed out after 1.613s (SENT)
I20260709 13:20:55.005424 16265 leader_election.cc:304] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:20:54.994927 16219 raft_consensus.cc:2376] T 02838c86531d4f98bf58d1c22b8baf77 P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 11. Re-sending same reply.
I20260709 13:20:55.000006 16789 leader_election.cc:304] T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac
I20260709 13:20:55.003573 16220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9944d5e4b8c4109af9955c4f08c9095" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:20:55.008123 16220 raft_consensus.cc:3060] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:20:55.009743 20269 raft_consensus.cc:2749] T 829b4ad8c5a54eababeea1181b960d4b P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:20:55.010031 20599 leader_election.cc:290] T d16dfad0c3b540e5b4b86492ddf87208 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:20:55.017210 20268 raft_consensus.cc:2804] T d2ab0ac8347641e6955e9bce7d90c1a2 P cb82cc7bd6db46d980c75237a1402860 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:20:55.017714 20268 raft_consensus.cc:493] T d2ab0ac8347641e6955e9bce7d90c1a2 P cb82cc7bd6db46d980c75237a1402860 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:55.018082 20268 raft_consensus.cc:3060] T d2ab0ac8347641e6955e9bce7d90c1a2 P cb82cc7bd6db46d980c75237a1402860 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:20:55.024269 20422 raft_consensus.cc:2749] T 02838c86531d4f98bf58d1c22b8baf77 P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:20:55.027016 20588 consensus_meta.cc:287] T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045: Time spent flushing consensus metadata: real 0.527s	user 0.004s	sys 0.005s
I20260709 13:20:55.027513 20588 raft_consensus.cc:515] T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:55.029769 20588 leader_election.cc:290] T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 12 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:20:55.036644 20267 leader_election.cc:290] T 2794af5b99984efebdee564035841cb0 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 9 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:54.990199 20315 raft_consensus.cc:515] T d16dfad0c3b540e5b4b86492ddf87208 P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:55.037894 20570 raft_consensus.cc:515] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:55.038887 20315 leader_election.cc:290] T d16dfad0c3b540e5b4b86492ddf87208 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 12 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:54.967129 20553 raft_consensus.cc:2804] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:20:55.039757 20553 raft_consensus.cc:697] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [term 14 LEADER]: Becoming Leader. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:20:55.040048 20570 leader_election.cc:290] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 11 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:54.995473 20612 raft_consensus.cc:3060] T 434bff3082d141838068f323bdb0d1ed P cb82cc7bd6db46d980c75237a1402860 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:20:55.040740 20553 consensus_queue.cc:237] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:55.004189 20475 raft_consensus.cc:515] T 26354361e9b044cea1c2b86c4882fe92 P 647a343a542d4a139fe106c19095129b [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:55.045483 20475 leader_election.cc:290] T 26354361e9b044cea1c2b86c4882fe92 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 10 election: Requested vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:55.046025 20560 raft_consensus.cc:3055] T 829b4ad8c5a54eababeea1181b960d4b P cb82cc7bd6db46d980c75237a1402860 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:20:55.046442 20560 raft_consensus.cc:740] T 829b4ad8c5a54eababeea1181b960d4b P cb82cc7bd6db46d980c75237a1402860 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:20:55.048087 16221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f5f89186204242bf377f7c42b2e2da" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:20:55.049429 16221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8638ee363273426db1393dd81e5c7b72" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:55.049991 16221 raft_consensus.cc:2468] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 12.
I20260709 13:20:55.050756 16221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18ad7e3af5504571a69d10c00750a01c" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:55.051424 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c11074579b046d99e585c2818e18a2c" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 12 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:55.052433 16218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdbd78acb4cf46298e0de8ba0d130e19" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:55.003094 20592 consensus_queue.cc:1048] T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.057s
I20260709 13:20:55.054064 16218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18ad7e3af5504571a69d10c00750a01c" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:20:55.055431 16222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ab0ac8347641e6955e9bce7d90c1a2" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:20:55.056015 16222 raft_consensus.cc:2376] T d2ab0ac8347641e6955e9bce7d90c1a2 P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 11. Re-sending same reply.
I20260709 13:20:55.056859 16222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8158bbd17b994d2eb96f560bd0f1b88d" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: true dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:20:55.057485 16222 raft_consensus.cc:2368] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 for earlier term 10. Current term is 11.
I20260709 13:20:55.058197 16222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7ca61592e848ea9b0fb81a2ce33cdc" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:20:54.991627 20576 raft_consensus.cc:3060] T 1df93779209748b9860f6d893dd3de53 P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:20:55.055450 20560 consensus_queue.cc:260] T 829b4ad8c5a54eababeea1181b960d4b P cb82cc7bd6db46d980c75237a1402860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:20:54.997505 20434 consensus_meta.cc:287] T b87147731a68400e8cce9b879daa7c5d P 647a343a542d4a139fe106c19095129b: Time spent flushing consensus metadata: real 0.587s	user 0.008s	sys 0.002s
I20260709 13:20:55.064358 20434 raft_consensus.cc:515] T b87147731a68400e8cce9b879daa7c5d P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:55.066623 20434 leader_election.cc:290] T b87147731a68400e8cce9b879daa7c5d P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 13 election: Requested vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:55.067445 16150 raft_consensus.cc:2368] T b87147731a68400e8cce9b879daa7c5d P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 for earlier term 12. Current term is 13.
I20260709 13:20:55.004912 16115 leader_election.cc:304] T 7c2095a5f4ad49408dbc7f69c606120d P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
W20260709 13:20:55.068270 16115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 63433.6us
I20260709 13:20:55.071033 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26354361e9b044cea1c2b86c4882fe92" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:20:55.071343 20434 raft_consensus.cc:2804] T 7c2095a5f4ad49408dbc7f69c606120d P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:20:55.071676 16298 raft_consensus.cc:3060] T 26354361e9b044cea1c2b86c4882fe92 P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:20:55.072129 20434 raft_consensus.cc:697] T 7c2095a5f4ad49408dbc7f69c606120d P 647a343a542d4a139fe106c19095129b [term 9 LEADER]: Becoming Leader. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:20:55.073119 20434 consensus_queue.cc:237] T 7c2095a5f4ad49408dbc7f69c606120d P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:55.075336 16116 leader_election.cc:304] T ecc1e363230043fe9fef390f0b979f20 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: cb82cc7bd6db46d980c75237a1402860
I20260709 13:20:55.075508 16294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87147731a68400e8cce9b879daa7c5d" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:20:55.076184 16294 raft_consensus.cc:3060] T b87147731a68400e8cce9b879daa7c5d P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:20:55.077220 16116 leader_election.cc:304] T 746aabf856ab401c9ef5e859371e7b67 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860
I20260709 13:20:55.079409 20231 raft_consensus.cc:2804] T ecc1e363230043fe9fef390f0b979f20 P 647a343a542d4a139fe106c19095129b [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:20:55.079944 20231 raft_consensus.cc:493] T ecc1e363230043fe9fef390f0b979f20 P 647a343a542d4a139fe106c19095129b [term 10 FOLLOWER]: Starting leader election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:20:55.080319 20487 raft_consensus.cc:2749] T 746aabf856ab401c9ef5e859371e7b67 P 647a343a542d4a139fe106c19095129b [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:20:55.080785 20472 raft_consensus.cc:493] T ee9ed67006144797b1de6ea8c2013a68 P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:55.081293 20472 raft_consensus.cc:515] T ee9ed67006144797b1de6ea8c2013a68 P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:55.085026 16146 raft_consensus.cc:2468] T 08c74490c40243a88e62d1760b5cd814 P 647a343a542d4a139fe106c19095129b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 10.
I20260709 13:20:55.085803 16223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6f07842d3aa47858df5665874c86f3a" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:20:55.086017 20472 leader_election.cc:290] T ee9ed67006144797b1de6ea8c2013a68 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:55.086395 16223 raft_consensus.cc:3060] T b6f07842d3aa47858df5665874c86f3a P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:20:55.091254 16221 raft_consensus.cc:2393] T 18ad7e3af5504571a69d10c00750a01c P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 11: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
W20260709 13:20:55.094358 20613 log.cc:927] Time spent T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 0.302s	user 0.000s	sys 0.002s
I20260709 13:20:55.094551 16817 raft_consensus.cc:2468] T c9944d5e4b8c4109af9955c4f08c9095 P cb82cc7bd6db46d980c75237a1402860 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 12.
I20260709 13:20:55.080351 20231 raft_consensus.cc:3060] T ecc1e363230043fe9fef390f0b979f20 P 647a343a542d4a139fe106c19095129b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:20:55.052537 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a878999c0024d2abe517484180290fc" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:54.977180 20601 raft_consensus.cc:740] T 8a878999c0024d2abe517484180290fc P 1ca2725d290e47d68b1a4b609c9816ac [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:20:55.101022 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66ae176961b74813a58bc15c9c4c6f2a" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:20:55.101922 16224 raft_consensus.cc:2368] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 for earlier term 13. Current term is 14.
I20260709 13:20:55.101698 20601 consensus_queue.cc:260] T 8a878999c0024d2abe517484180290fc P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:55.061921 20560 raft_consensus.cc:3060] T 829b4ad8c5a54eababeea1181b960d4b P cb82cc7bd6db46d980c75237a1402860 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:20:55.105962 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9944d5e4b8c4109af9955c4f08c9095" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:20:55.106668 16116 leader_election.cc:304] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860; no voters: 90f3da57796b488c9087a2f523adf045
I20260709 13:20:55.107103 16220 raft_consensus.cc:2468] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 12.
I20260709 13:20:55.103107 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c61a39fd4f84424b129c1a6f9ba404d" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:55.108810 16220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4dc6b0541ca431ebc757beac00521f1" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:55.108901 16116 leader_election.cc:304] T 68a6838d82ef4cac83507f906139d90f P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:20:55.111271 16222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fffab26427e24d398e2695aad40ca134" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:55.071964 16047 catalog_manager.cc:5697] T 9e65676b21b748728ad39668cfe722ec P cb82cc7bd6db46d980c75237a1402860 reported cstate change: term changed from 6 to 10. New cstate: current_term: 10 leader_uuid: "90f3da57796b488c9087a2f523adf045" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
W20260709 13:20:55.108165 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 353379us
I20260709 13:20:55.116462 16190 leader_election.cc:304] T 19baa7bc367844c288bca8f81439c413 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860
W20260709 13:20:55.116407 16789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 116495us
I20260709 13:20:55.105293 16790 leader_election.cc:304] T af57996a8be84d398f0f023e8d08ef77 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:20:55.124781 16220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e4bfb9090c8460eb14fa7e0d818e7f1" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:55.125415 16220 raft_consensus.cc:2468] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 10.
I20260709 13:20:55.129796 16223 raft_consensus.cc:2468] T b6f07842d3aa47858df5665874c86f3a P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 13.
I20260709 13:20:55.125435 16790 leader_election.cc:304] T 9e65676b21b748728ad39668cfe722ec P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860; no voters: 90f3da57796b488c9087a2f523adf045
I20260709 13:20:55.133117 16294 raft_consensus.cc:2468] T b87147731a68400e8cce9b879daa7c5d P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 13.
I20260709 13:20:55.120550 16219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9944d5e4b8c4109af9955c4f08c9095" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:55.134337 16115 leader_election.cc:304] T b87147731a68400e8cce9b879daa7c5d P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
W20260709 13:20:55.121666 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 116809us
I20260709 13:20:55.121490 16147 raft_consensus.cc:2468] T 5a959dbecc2648eb85d1cc550f5a3245 P 647a343a542d4a139fe106c19095129b [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 10.
W20260709 13:20:55.135419 16265 leader_election.cc:336] T c9944d5e4b8c4109af9955c4f08c9095 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 12 election: RPC error from VoteRequest() call to peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943): Timed out: RequestConsensusVote RPC to 127.14.177.130:35943 timed out after 1.646s (SENT)
I20260709 13:20:55.136301 16298 raft_consensus.cc:2468] T 26354361e9b044cea1c2b86c4882fe92 P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 10.
I20260709 13:20:55.138685 16221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a959dbecc2648eb85d1cc550f5a3245" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:55.133888 16219 raft_consensus.cc:2376] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 12. Re-sending same reply.
I20260709 13:20:55.143848 16218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5718c623ebee4d2bb770f0f9ddb2f164" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:20:55.144025 16219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4f65cdf00b748bb963ee6e81ef43b79" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
W20260709 13:20:55.144052 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 135469us
I20260709 13:20:55.144711 16218 raft_consensus.cc:3060] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:20:55.145037 16191 leader_election.cc:304] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: cb82cc7bd6db46d980c75237a1402860
W20260709 13:20:55.141664 16265 leader_election.cc:336] T 02838c86531d4f98bf58d1c22b8baf77 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 11 election: RPC error from VoteRequest() call to peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943): Timed out: RequestConsensusVote RPC to 127.14.177.130:35943 timed out after 1.733s (SENT)
I20260709 13:20:55.147256 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ad5dbfa99ee4b8d9e28be92b3f36db5" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:55.147373 16219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1df93779209748b9860f6d893dd3de53" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:55.148504 16222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8158bbd17b994d2eb96f560bd0f1b88d" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:55.149405 16222 raft_consensus.cc:2393] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 11: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
I20260709 13:20:55.149515 16115 leader_election.cc:304] T 26354361e9b044cea1c2b86c4882fe92 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:20:55.149868 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c2095a5f4ad49408dbc7f69c606120d" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:55.150421 16222 raft_consensus.cc:1240] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 11. Current term is 12. Ops: []
I20260709 13:20:55.150475 16220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a959dbecc2648eb85d1cc550f5a3245" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:20:55.151145 16225 raft_consensus.cc:1240] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 10. Current term is 11. Ops: []
I20260709 13:20:55.151743 16222 raft_consensus.cc:1240] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 11. Current term is 12. Ops: []
I20260709 13:20:55.152022 16220 raft_consensus.cc:1240] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 11. Current term is 12. Ops: []
I20260709 13:20:55.152271 16225 raft_consensus.cc:1240] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 10. Current term is 11. Ops: []
I20260709 13:20:55.119640 20601 raft_consensus.cc:3060] T 8a878999c0024d2abe517484180290fc P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:20:55.123605 19982 raft_consensus.cc:2749] T 68a6838d82ef4cac83507f906139d90f P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:20:55.139101 20605 consensus_queue.cc:1048] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.016s
I20260709 13:20:55.155099 19982 raft_consensus.cc:2804] T 26354361e9b044cea1c2b86c4882fe92 P 647a343a542d4a139fe106c19095129b [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:20:55.155416 20604 raft_consensus.cc:2804] T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:20:55.155625 19982 raft_consensus.cc:697] T 26354361e9b044cea1c2b86c4882fe92 P 647a343a542d4a139fe106c19095129b [term 10 LEADER]: Becoming Leader. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:20:55.155844 20604 raft_consensus.cc:493] T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:55.156193 20604 raft_consensus.cc:3060] T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:20:55.156605 19982 consensus_queue.cc:237] T 26354361e9b044cea1c2b86c4882fe92 P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:55.157212 20518 raft_consensus.cc:3060] T c0eab25ce4c64e018a041beeda2867db P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:20:55.160535 16219 raft_consensus.cc:2468] T 1df93779209748b9860f6d893dd3de53 P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 11.
I20260709 13:20:55.164237 20607 raft_consensus.cc:2804] T 9e65676b21b748728ad39668cfe722ec P cb82cc7bd6db46d980c75237a1402860 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:20:55.164788 20607 raft_consensus.cc:493] T 9e65676b21b748728ad39668cfe722ec P cb82cc7bd6db46d980c75237a1402860 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:20:55.165244 20607 raft_consensus.cc:3060] T 9e65676b21b748728ad39668cfe722ec P cb82cc7bd6db46d980c75237a1402860 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:20:55.147902 20388 raft_consensus.cc:2804] T b87147731a68400e8cce9b879daa7c5d P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:20:55.167667 20388 raft_consensus.cc:697] T b87147731a68400e8cce9b879daa7c5d P 647a343a542d4a139fe106c19095129b [term 13 LEADER]: Becoming Leader. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:20:55.171231 20611 raft_consensus.cc:2804] T af57996a8be84d398f0f023e8d08ef77 P cb82cc7bd6db46d980c75237a1402860 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:20:55.171797 20611 raft_consensus.cc:493] T af57996a8be84d398f0f023e8d08ef77 P cb82cc7bd6db46d980c75237a1402860 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:20:55.172245 20611 raft_consensus.cc:3060] T af57996a8be84d398f0f023e8d08ef77 P cb82cc7bd6db46d980c75237a1402860 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:20:55.176424 16048 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:49246 (request call id 330) took 44168 ms (client timeout 15000 ms). Trace:
I20260709 13:20:55.120368 20475 raft_consensus.cc:2804] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:20:55.179389 20475 raft_consensus.cc:493] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:55.179755 20475 raft_consensus.cc:3060] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:20:55.180392 16297 raft_consensus.cc:1275] T 3884f864a96d428785895e5463e5728f P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:20:55.160997 16221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5ad4eb281564936a59f41e387a47156" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:55.181293 20579 leader_election.cc:290] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 12 election: Requested vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:55.181582 16221 raft_consensus.cc:2468] T e5ad4eb281564936a59f41e387a47156 P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 12.
I20260709 13:20:55.183887 20618 raft_consensus.cc:2749] T 19baa7bc367844c288bca8f81439c413 P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:20:55.184104 16219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c74490c40243a88e62d1760b5cd814" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:55.184499 20487 consensus_queue.cc:1048] T 3884f864a96d428785895e5463e5728f P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:20:55.184759 16219 raft_consensus.cc:2468] T 08c74490c40243a88e62d1760b5cd814 P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 10.
I20260709 13:20:55.159394 16047 catalog_manager.cc:5697] T 19baa7bc367844c288bca8f81439c413 P cb82cc7bd6db46d980c75237a1402860 reported cstate change: config changed from index 3 to 7, term changed from 7 to 11, leader changed from 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130) to <none>, cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) changed from NON_VOTER to VOTER, no longer has a pending config: VOTER 647a343a542d4a139fe106c19095129b (127.14.177.129), VOTER 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130), VOTER 90f3da57796b488c9087a2f523adf045 (127.14.177.131), VOTER cb82cc7bd6db46d980c75237a1402860 (127.14.177.132). New cstate: current_term: 11 committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
W20260709 13:20:55.191133 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 49509.8us
W20260709 13:20:55.192297 16265 leader_election.cc:336] T 8c11074579b046d99e585c2818e18a2c P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943): Timed out: RequestConsensusVote RPC to 127.14.177.130:35943 timed out after 1.783s (SENT)
W20260709 13:20:55.193106 16265 leader_election.cc:336] T c9944d5e4b8c4109af9955c4f08c9095 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943): Timed out: RequestConsensusVote RPC to 127.14.177.130:35943 timed out after 1.820s (SENT)
I20260709 13:20:55.195844 16217 raft_consensus.cc:2468] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 13.
I20260709 13:20:55.197515 16116 leader_election.cc:304] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
I20260709 13:20:55.210774 16047 catalog_manager.cc:5697] T 829b4ad8c5a54eababeea1181b960d4b P cb82cc7bd6db46d980c75237a1402860 reported cstate change: term changed from 6 to 10, leader changed from 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130) to cb82cc7bd6db46d980c75237a1402860 (127.14.177.132). New cstate: current_term: 10 leader_uuid: "cb82cc7bd6db46d980c75237a1402860" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:20:55.190842 20267 raft_consensus.cc:2749] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:20:55.215479 16047 catalog_manager.cc:5697] T d2ab0ac8347641e6955e9bce7d90c1a2 P cb82cc7bd6db46d980c75237a1402860 reported cstate change: term changed from 7 to 10, leader changed from 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130) to cb82cc7bd6db46d980c75237a1402860 (127.14.177.132). New cstate: current_term: 10 leader_uuid: "cb82cc7bd6db46d980c75237a1402860" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:20:55.215901 16221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87ac0d1f7e8a419f9bcbc04aee0e067e" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:55.240880 16221 raft_consensus.cc:2468] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 12.
I20260709 13:20:55.217058 20576 raft_consensus.cc:515] T 1df93779209748b9860f6d893dd3de53 P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:20:55.176796 16048 rpcz_store.cc:269] 0709 13:20:11.007630 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:11.007853 (+   223us) service_pool.cc:224] Handling call
0709 13:20:55.176209 (+44168356us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:20:47.086263 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:20:47.087032 (+   769us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:20:47.087089 (+    57us) write_op.cc:435] Acquired schema lock
0709 13:20:47.087117 (+    28us) tablet.cc:662] Decoding operations
0709 13:20:47.088748 (+  1631us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:20:47.088846 (+    98us) write_op.cc:641] Partition lock acquired for write op
0709 13:20:47.088872 (+    26us) tablet.cc:685] Acquiring locks for 19 operations
0709 13:20:47.090299 (+  1427us) tablet.cc:701] Row locks acquired
0709 13:20:47.090351 (+    52us) write_op.cc:260] PREPARE: finished
0709 13:20:47.119449 (+ 29098us) write_op.cc:270] Start()
0709 13:20:47.157331 (+ 37882us) write_op.cc:276] Timestamp: P: 1783603247090460 usec, L: 0
0709 13:20:47.169851 (+ 12520us) op_driver.cc:348] REPLICATION: starting
0709 13:20:47.341272 (+171421us) log.cc:844] Serialized 7558 byte log entry
0709 13:20:48.765272 (+1424000us) op_driver.cc:464] REPLICATION: finished
0709 13:20:49.061951 (+296679us) write_op.cc:301] APPLY: starting
0709 13:20:49.062087 (+   136us) tablet.cc:1367] starting BulkCheckPresence
0709 13:20:49.194086 (+131999us) tablet.cc:1370] finished BulkCheckPresence
0709 13:20:49.194218 (+   132us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:20:49.328294 (+134076us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:20:49.329667 (+  1373us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=38,key_file_lookups=38,delta_file_lookups=0,mrs_lookups=0
0709 13:20:49.329725 (+    58us) write_op.cc:312] APPLY: finished
0709 13:20:49.334910 (+  5185us) log.cc:844] Serialized 172 byte log entry
0709 13:20:49.337538 (+  2628us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:20:49.339252 (+  1714us) write_op.cc:454] Released schema lock
0709 13:20:49.339361 (+   109us) write_op.cc:341] FINISH: Updating metrics
0709 13:20:49.339790 (+   429us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:20:49.339826 (+    36us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":1899313,"reported_tablets":45,"child_traces":[["op",{"apply.queue_time_us":274316,"apply.run_cpu_time_us":28474,"apply.run_wall_time_us":277834,"cfile_cache_hit":74,"cfile_cache_hit_bytes":77256,"cfile_cache_miss":4,"cfile_cache_miss_bytes":6215,"cfile_init":1,"lbm_read_time_us":7623,"lbm_reads_1-10_ms":1,"lbm_reads_lt_1ms":7,"num_ops":19,"prepare.queue_time_us":568631,"prepare.run_cpu_time_us":6564,"prepare.run_wall_time_us":269592,"replication_time_us":1571957,"thread_start_us":759298,"threads_started":2}]]}
W20260709 13:20:55.219806 16265 leader_election.cc:336] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 14 election: RPC error from VoteRequest() call to peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943): Timed out: RequestConsensusVote RPC to 127.14.177.130:35943 timed out after 1.868s (SENT)
W20260709 13:20:55.223320 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 91720.2us
I20260709 13:20:55.170930 20231 raft_consensus.cc:515] T ecc1e363230043fe9fef390f0b979f20 P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:20:55.226151 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 81200.2us
I20260709 13:20:55.226692 19861 raft_consensus.cc:2804] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:20:55.227274 20427 leader_election.cc:290] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 10 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:55.227465 16223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c2095a5f4ad49408dbc7f69c606120d" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:55.228018 19982 raft_consensus.cc:2804] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:20:55.218600 20388 consensus_queue.cc:237] T b87147731a68400e8cce9b879daa7c5d P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:55.267026 16224 raft_consensus.cc:1275] T 8a878999c0024d2abe517484180290fc P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
W20260709 13:20:55.261436 16789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 121771us
I20260709 13:20:55.237957 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ad5dbfa99ee4b8d9e28be92b3f36db5" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:20:55.268786 16298 raft_consensus.cc:3060] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:20:55.243127 16294 raft_consensus.cc:1275] T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:20:55.244895 16218 raft_consensus.cc:2468] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 12.
W20260709 13:20:55.248284 16266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 294297us
W20260709 13:20:55.274699 16266 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:55.214476 20268 raft_consensus.cc:515] T d2ab0ac8347641e6955e9bce7d90c1a2 P cb82cc7bd6db46d980c75237a1402860 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:55.232558 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee9ed67006144797b1de6ea8c2013a68" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
W20260709 13:20:55.257813 16265 leader_election.cc:336] T fffab26427e24d398e2695aad40ca134 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943): Timed out: RequestConsensusVote RPC to 127.14.177.130:35943 timed out after 1.923s (SENT)
I20260709 13:20:55.235275 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ad5dbfa99ee4b8d9e28be92b3f36db5" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:20:55.280146 16217 raft_consensus.cc:2468] T ee9ed67006144797b1de6ea8c2013a68 P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 9.
I20260709 13:20:55.280185 16265 leader_election.cc:304] T fffab26427e24d398e2695aad40ca134 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
W20260709 13:20:55.281296 16265 leader_election.cc:336] T 66ae176961b74813a58bc15c9c4c6f2a P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 13 election: RPC error from VoteRequest() call to peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943): Timed out: RequestConsensusVote RPC to 127.14.177.130:35943 timed out after 1.924s (SENT)
I20260709 13:20:55.276789 16116 leader_election.cc:304] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
I20260709 13:20:55.285907 16789 leader_election.cc:304] T b87147731a68400e8cce9b879daa7c5d P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:20:55.259951 19861 raft_consensus.cc:493] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:55.295323 16116 leader_election.cc:304] T ee9ed67006144797b1de6ea8c2013a68 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
I20260709 13:20:55.295783 16294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d4ab870a05347868956aa8b841b2981" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
W20260709 13:20:55.296644 16263 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:55.296718 16296 raft_consensus.cc:1240] T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 11. Current term is 12. Ops: [11.6-11.6]
I20260709 13:20:55.290750 16265 leader_election.cc:304] T 18ad7e3af5504571a69d10c00750a01c P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:20:55.162984 20612 raft_consensus.cc:515] T 434bff3082d141838068f323bdb0d1ed P cb82cc7bd6db46d980c75237a1402860 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:55.311297 16293 raft_consensus.cc:1240] T 7c2095a5f4ad49408dbc7f69c606120d P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 8. Current term is 9. Ops: [8.5-8.5]
I20260709 13:20:55.312005 16265 leader_election.cc:304] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:20:55.282804 20599 raft_consensus.cc:2749] T fffab26427e24d398e2695aad40ca134 P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:20:55.316588 16265 leader_election.cc:304] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:20:55.320399 16224 raft_consensus.cc:1275] T b6f07842d3aa47858df5665874c86f3a P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:20:55.283389 19982 raft_consensus.cc:697] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [term 13 LEADER]: Becoming Leader. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:20:55.326589 16298 raft_consensus.cc:2468] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 10.
I20260709 13:20:55.302862 16296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74ca8140f35c4ae29e8a664f6bdfc829" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:20:55.362996 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a378d7f796f94615b3b347119a9880fd" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:55.362088 16819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74ca8140f35c4ae29e8a664f6bdfc829" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:20:55.359688 16820 raft_consensus.cc:1275] T 66f75124215f4ca6bef831e174a792d8 P cb82cc7bd6db46d980c75237a1402860 [term 11 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:20:55.360616 16825 raft_consensus.cc:1240] T 8c11074579b046d99e585c2818e18a2c P cb82cc7bd6db46d980c75237a1402860 [term 12 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 11. Current term is 12. Ops: [11.11-11.11]
I20260709 13:20:55.361316 16822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a378d7f796f94615b3b347119a9880fd" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:20:55.302139 19861 raft_consensus.cc:3060] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:20:55.364353 16299 raft_consensus.cc:2393] T a378d7f796f94615b3b347119a9880fd P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 12: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
I20260709 13:20:55.359643 16291 raft_consensus.cc:1240] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 9. Current term is 10. Ops: []
I20260709 13:20:55.327383 16300 raft_consensus.cc:1240] T 5a959dbecc2648eb85d1cc550f5a3245 P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 9. Current term is 10. Ops: []
W20260709 13:20:55.353905 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 58157.6us
W20260709 13:20:55.355103 16789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 52921.2us
I20260709 13:20:55.294893 20605 raft_consensus.cc:2749] T b87147731a68400e8cce9b879daa7c5d P cb82cc7bd6db46d980c75237a1402860 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:20:55.283741 20487 consensus_queue.cc:1048] T b6f07842d3aa47858df5665874c86f3a P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.032s
I20260709 13:20:55.299425 20597 raft_consensus.cc:493] T 02838c86531d4f98bf58d1c22b8baf77 P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:55.386895 16296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8158bbd17b994d2eb96f560bd0f1b88d" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:20:55.387816 16296 raft_consensus.cc:3060] T 8158bbd17b994d2eb96f560bd0f1b88d P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:20:55.361070 16816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b7d1ab519044eda2d5e04a07242f25" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:20:55.389380 16816 raft_consensus.cc:3060] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:20:55.277611 20593 consensus_queue.cc:1059] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 9, Time since last communication: 0.044s
I20260709 13:20:55.374641 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d16dfad0c3b540e5b4b86492ddf87208" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:20:55.390244 16298 raft_consensus.cc:3060] T d16dfad0c3b540e5b4b86492ddf87208 P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:20:55.374910 16818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8638ee363273426db1393dd81e5c7b72" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 13 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:55.369395 16115 leader_election.cc:304] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac
I20260709 13:20:55.297508 20589 raft_consensus.cc:493] T ecc1e363230043fe9fef390f0b979f20 P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:20:55.328058 16265 leader_election.cc:400] T b87147731a68400e8cce9b879daa7c5d P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 12 election: Vote denied by peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741) with higher term. Message: Invalid argument: T b87147731a68400e8cce9b879daa7c5d P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 for earlier term 12. Current term is 13.
I20260709 13:20:55.396039 16265 leader_election.cc:403] T b87147731a68400e8cce9b879daa7c5d P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 12 election: Cancelling election due to peer responding with higher term
W20260709 13:20:55.378201 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 252822us
I20260709 13:20:55.310497 20236 raft_consensus.cc:2804] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Leader election won for term 12
W20260709 13:20:55.401907 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 81052.1us
I20260709 13:20:55.402671 16265 leader_election.cc:304] T 5a959dbecc2648eb85d1cc550f5a3245 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:20:55.367961 20573 raft_consensus.cc:2804] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:20:55.396993 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18ad7e3af5504571a69d10c00750a01c" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:20:55.397022 16790 leader_election.cc:304] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:20:55.409305 16790 leader_election.cc:304] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:20:55.314688 20565 raft_consensus.cc:2804] T ee9ed67006144797b1de6ea8c2013a68 P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:20:55.327378 20583 raft_consensus.cc:2804] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:20:55.398665 16819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada5a004ba7c46819051e3ccec71ffdb" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:20:55.321240 20588 raft_consensus.cc:2804] T 18ad7e3af5504571a69d10c00750a01c P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Leader election won for term 11
W20260709 13:20:55.399374 16189 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:20:55.384469 20590 log.cc:927] Time spent T e5ad4eb281564936a59f41e387a47156 P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.072s	user 0.000s	sys 0.000s
I20260709 13:20:55.405992 16821 raft_consensus.cc:1240] T 3a7ca61592e848ea9b0fb81a2ce33cdc P cb82cc7bd6db46d980c75237a1402860 [term 14 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 13. Current term is 14. Ops: []
I20260709 13:20:55.327529 20472 consensus_queue.cc:1048] T 8a878999c0024d2abe517484180290fc P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.080s
I20260709 13:20:55.406817 20236 raft_consensus.cc:697] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b [term 12 LEADER]: Becoming Leader. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:20:55.365851 20594 raft_consensus.cc:493] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:20:55.412452 16298 raft_consensus.cc:2468] T d16dfad0c3b540e5b4b86492ddf87208 P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 12.
I20260709 13:20:55.327838 16047 catalog_manager.cc:5697] T 74ca8140f35c4ae29e8a664f6bdfc829 P cb82cc7bd6db46d980c75237a1402860 reported cstate change: term changed from 7 to 10, leader changed from 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130) to cb82cc7bd6db46d980c75237a1402860 (127.14.177.132). New cstate: current_term: 10 leader_uuid: "cb82cc7bd6db46d980c75237a1402860" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:20:55.414971 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8638ee363273426db1393dd81e5c7b72" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 13 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:55.436978 16299 raft_consensus.cc:2410] T 8638ee363273426db1393dd81e5c7b72 P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac for term 13 because replica has last-logged OpId of term: 10 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 9.
I20260709 13:20:55.413771 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a378d7f796f94615b3b347119a9880fd" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:20:55.319583 20424 consensus_queue.cc:1059] T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.089s
I20260709 13:20:55.368120 20601 consensus_queue.cc:1059] T 7c2095a5f4ad49408dbc7f69c606120d P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.045s
I20260709 13:20:55.408349 20573 raft_consensus.cc:493] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:55.438812 16300 raft_consensus.cc:2393] T a378d7f796f94615b3b347119a9880fd P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 12: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
I20260709 13:20:55.446347 16295 raft_consensus.cc:1240] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 11. Current term is 12. Ops: []
I20260709 13:20:55.405164 20600 raft_consensus.cc:2804] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:20:55.422881 20583 raft_consensus.cc:493] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:55.451498 16047 catalog_manager.cc:5697] T 23f82224f4564fea93572e8206bad1fe P cb82cc7bd6db46d980c75237a1402860 reported cstate change: term changed from 1 to 8, leader changed from 647a343a542d4a139fe106c19095129b (127.14.177.129) to cb82cc7bd6db46d980c75237a1402860 (127.14.177.132). New cstate: current_term: 8 leader_uuid: "cb82cc7bd6db46d980c75237a1402860" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:20:55.456018 20636 consensus_queue.cc:1048] T 8638ee363273426db1393dd81e5c7b72 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.017s
W20260709 13:20:55.463725 16789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 91925.9us
I20260709 13:20:55.443272 16790 leader_election.cc:304] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:20:55.433290 16294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada5a004ba7c46819051e3ccec71ffdb" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:20:55.404794 20599 raft_consensus.cc:2749] T b87147731a68400e8cce9b879daa7c5d P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Leader election lost for term 12. Reason: Vote denied by peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741) with higher term. Message: Invalid argument: T b87147731a68400e8cce9b879daa7c5d P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 for earlier term 12. Current term is 13.
I20260709 13:20:55.413797 20565 raft_consensus.cc:493] T ee9ed67006144797b1de6ea8c2013a68 P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:55.435107 20605 consensus_queue.cc:1059] T 3884f864a96d428785895e5463e5728f P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.039s
I20260709 13:20:55.452453 20573 raft_consensus.cc:3060] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:20:55.446043 20603 raft_consensus.cc:2804] T 5a959dbecc2648eb85d1cc550f5a3245 P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:20:55.399030 20597 raft_consensus.cc:515] T 02838c86531d4f98bf58d1c22b8baf77 P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:55.456297 20470 raft_consensus.cc:3055] T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:20:55.486610 16291 raft_consensus.cc:1240] T 2794af5b99984efebdee564035841cb0 P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 8. Current term is 9. Ops: []
I20260709 13:20:55.435541 20434 consensus_queue.cc:1048] T b6f07842d3aa47858df5665874c86f3a P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.040s
I20260709 13:20:55.473317 16818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5718c623ebee4d2bb770f0f9ddb2f164" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:20:55.476451 16295 raft_consensus.cc:1240] T 26354361e9b044cea1c2b86c4882fe92 P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 9. Current term is 10. Ops: []
W20260709 13:20:55.476982 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 78366.8us
I20260709 13:20:55.478106 16821 raft_consensus.cc:1240] T 434bff3082d141838068f323bdb0d1ed P cb82cc7bd6db46d980c75237a1402860 [term 11 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 10. Current term is 11. Ops: []
I20260709 13:20:55.488916 16192 leader_election.cc:304] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860
I20260709 13:20:55.404884 20589 raft_consensus.cc:515] T ecc1e363230043fe9fef390f0b979f20 P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:55.474445 20600 raft_consensus.cc:697] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b [term 10 LEADER]: Becoming Leader. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:20:55.477267 20565 raft_consensus.cc:3060] T ee9ed67006144797b1de6ea8c2013a68 P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:20:55.462616 20583 raft_consensus.cc:3060] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:20:55.494107 20593 raft_consensus.cc:3055] T 7c2095a5f4ad49408dbc7f69c606120d P 1ca2725d290e47d68b1a4b609c9816ac [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:20:55.536463 20593 raft_consensus.cc:740] T 7c2095a5f4ad49408dbc7f69c606120d P 1ca2725d290e47d68b1a4b609c9816ac [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:20:55.503650 20578 raft_consensus.cc:3055] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 1ca2725d290e47d68b1a4b609c9816ac [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:20:55.361776 19982 consensus_queue.cc:237] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.11, Last appended by leader: 11, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:20:55.470598 16266 leader_election.cc:336] T d2ab0ac8347641e6955e9bce7d90c1a2 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 11 election: RPC error from VoteRequest() call to peer cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923): Timed out: RequestConsensusVote RPC to 127.14.177.132:38923 timed out after 1.878s (SENT)
I20260709 13:20:55.473618 16294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c11074579b046d99e585c2818e18a2c" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 12 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:20:55.476670 16819 pending_rounds.cc:85] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860: Aborting all ops after (but not including) 7
I20260709 13:20:55.545998 16266 leader_election.cc:304] T d2ab0ac8347641e6955e9bce7d90c1a2 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860
I20260709 13:20:55.546386 16819 pending_rounds.cc:107] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860: Aborting uncommitted NO_OP operation due to leader change: 10.8
I20260709 13:20:55.546691 16819 raft_consensus.cc:2889] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860 [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:20:55.438457 20598 raft_consensus.cc:515] T 9e65676b21b748728ad39668cfe722ec P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:55.526248 20618 raft_consensus.cc:2749] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:20:55.526790 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:20:55.555974 16217 raft_consensus.cc:3060] T ecc1e363230043fe9fef390f0b979f20 P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:20:55.505580 16047 catalog_manager.cc:5697] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 reported cstate change: term changed from 7 to 10, leader changed from 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130) to cb82cc7bd6db46d980c75237a1402860 (127.14.177.132). New cstate: current_term: 10 leader_uuid: "cb82cc7bd6db46d980c75237a1402860" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:20:55.519907 20470 raft_consensus.cc:740] T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:20:55.535686 20424 consensus_queue.cc:1059] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:20:55.536131 20611 raft_consensus.cc:515] T af57996a8be84d398f0f023e8d08ef77 P cb82cc7bd6db46d980c75237a1402860 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:20:55.537725 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 94586.5us
I20260709 13:20:55.475150 16190 leader_election.cc:304] T d16dfad0c3b540e5b4b86492ddf87208 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:20:55.542163 16296 raft_consensus.cc:2468] T 8158bbd17b994d2eb96f560bd0f1b88d P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 11.
I20260709 13:20:55.542310 16818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26354361e9b044cea1c2b86c4882fe92" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:20:55.583364 16296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:20:55.583562 16818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e9b2bf7011f4ae688a79d7bb5a447b4" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:55.446178 20594 raft_consensus.cc:515] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:55.557010 16821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b7d1ab519044eda2d5e04a07242f25" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:20:55.504794 20475 raft_consensus.cc:515] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:55.559607 20579 consensus_queue.cc:1059] T 5a96373bb0e14dce8a266b2fc6c58507 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: INVALID_TERM, Last received: 10.5, Next index: 6, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:20:55.559461 20470 consensus_queue.cc:260] T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 4, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:55.590519 16818 raft_consensus.cc:2468] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 9.
I20260709 13:20:55.591992 16818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a378d7f796f94615b3b347119a9880fd" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:20:55.561046 16191 leader_election.cc:304] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
I20260709 13:20:55.562680 20597 leader_election.cc:290] T 02838c86531d4f98bf58d1c22b8baf77 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:55.563462 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02838c86531d4f98bf58d1c22b8baf77" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:55.593672 16226 raft_consensus.cc:2468] T 02838c86531d4f98bf58d1c22b8baf77 P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 11.
I20260709 13:20:55.564237 16047 catalog_manager.cc:5697] T 3884f864a96d428785895e5463e5728f P cb82cc7bd6db46d980c75237a1402860 reported cstate change: term changed from 2 to 7. New cstate: current_term: 7 leader_uuid: "cb82cc7bd6db46d980c75237a1402860" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:20:55.566030 16825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87147731a68400e8cce9b879daa7c5d" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:20:55.594758 16825 raft_consensus.cc:3060] T b87147731a68400e8cce9b879daa7c5d P cb82cc7bd6db46d980c75237a1402860 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:20:55.595445 16047 catalog_manager.cc:5697] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 reported cstate change: term changed from 9 to 12, leader changed from 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130) to cb82cc7bd6db46d980c75237a1402860 (127.14.177.132). New cstate: current_term: 12 leader_uuid: "cb82cc7bd6db46d980c75237a1402860" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:20:55.597460 16047 catalog_manager.cc:5697] T ee9ed67006144797b1de6ea8c2013a68 P cb82cc7bd6db46d980c75237a1402860 reported cstate change: term changed from 5 to 8, leader changed from cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) to 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130). New cstate: current_term: 8 leader_uuid: "1ca2725d290e47d68b1a4b609c9816ac" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:20:55.566607 16820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee9ed67006144797b1de6ea8c2013a68" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:55.599934 16820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87147731a68400e8cce9b879daa7c5d" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:20:55.598953 16047 catalog_manager.cc:5697] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P cb82cc7bd6db46d980c75237a1402860 reported cstate change: term changed from 3 to 10, leader changed from 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130) to cb82cc7bd6db46d980c75237a1402860 (127.14.177.132). New cstate: current_term: 10 leader_uuid: "cb82cc7bd6db46d980c75237a1402860" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:20:55.600998 16820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2794af5b99984efebdee564035841cb0" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:20:55.601418 16047 catalog_manager.cc:5697] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860 reported cstate change: config changed from index 3 to 4, term changed from 2 to 9, leader changed from 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130) to <none>, cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) changed from NON_VOTER to VOTER. New cstate: current_term: 9 committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:20:55.602890 16116 leader_election.cc:304] T 02838c86531d4f98bf58d1c22b8baf77 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
I20260709 13:20:55.602999 16047 catalog_manager.cc:5697] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860 reported cstate change: term changed from 5 to 9, leader changed from 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130) to 647a343a542d4a139fe106c19095129b (127.14.177.129). New cstate: current_term: 9 leader_uuid: "647a343a542d4a139fe106c19095129b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:20:55.604084 20457 raft_consensus.cc:2804] T 02838c86531d4f98bf58d1c22b8baf77 P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:20:55.604447 20457 raft_consensus.cc:493] T 02838c86531d4f98bf58d1c22b8baf77 P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:55.604851 20457 raft_consensus.cc:3060] T 02838c86531d4f98bf58d1c22b8baf77 P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:20:55.604350 16047 catalog_manager.cc:5697] T 57a744f5aaf944678a63469cbf514015 P cb82cc7bd6db46d980c75237a1402860 reported cstate change: term changed from 1 to 9, leader changed from 647a343a542d4a139fe106c19095129b (127.14.177.129) to cb82cc7bd6db46d980c75237a1402860 (127.14.177.132). New cstate: current_term: 9 leader_uuid: "cb82cc7bd6db46d980c75237a1402860" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:20:55.606020 16047 catalog_manager.cc:5697] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 reported cstate change: term changed from 8 to 12. New cstate: current_term: 12 leader_uuid: "647a343a542d4a139fe106c19095129b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:20:55.567096 20472 consensus_queue.cc:1059] T 434bff3082d141838068f323bdb0d1ed P 647a343a542d4a139fe106c19095129b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:20:55.607513 16047 catalog_manager.cc:5697] T 2794af5b99984efebdee564035841cb0 P cb82cc7bd6db46d980c75237a1402860 reported cstate change: term changed from 6 to 8. New cstate: current_term: 8 leader_uuid: "cb82cc7bd6db46d980c75237a1402860" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:20:55.609596 16047 catalog_manager.cc:5697] T 26354361e9b044cea1c2b86c4882fe92 P cb82cc7bd6db46d980c75237a1402860 reported cstate change: term changed from 6 to 8, leader changed from cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) to 647a343a542d4a139fe106c19095129b (127.14.177.129). New cstate: current_term: 8 leader_uuid: "647a343a542d4a139fe106c19095129b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:20:55.611060 16047 catalog_manager.cc:5697] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860 reported cstate change: term changed from 5 to 10, leader changed from 647a343a542d4a139fe106c19095129b (127.14.177.129) to cb82cc7bd6db46d980c75237a1402860 (127.14.177.132). New cstate: current_term: 10 leader_uuid: "cb82cc7bd6db46d980c75237a1402860" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:20:55.558096 20588 raft_consensus.cc:697] T 18ad7e3af5504571a69d10c00750a01c P 90f3da57796b488c9087a2f523adf045 [term 11 LEADER]: Becoming Leader. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:20:55.571322 20231 leader_election.cc:290] T ecc1e363230043fe9fef390f0b979f20 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 11 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:55.577966 20629 consensus_queue.cc:1059] T 2794af5b99984efebdee564035841cb0 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:20:55.622164 20588 consensus_queue.cc:237] T 18ad7e3af5504571a69d10c00750a01c P 90f3da57796b488c9087a2f523adf045 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.7, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } }
I20260709 13:20:55.584070 16296 raft_consensus.cc:3060] T ecc1e363230043fe9fef390f0b979f20 P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:20:55.584530 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 181945us
I20260709 13:20:55.471100 20622 consensus_queue.cc:1059] T 5a959dbecc2648eb85d1cc550f5a3245 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.100s
I20260709 13:20:55.602231 16820 raft_consensus.cc:3055] T 3884f864a96d428785895e5463e5728f P cb82cc7bd6db46d980c75237a1402860 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:20:55.567126 16291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02838c86531d4f98bf58d1c22b8baf77" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:55.526440 20601 consensus_queue.cc:1059] T 8c11074579b046d99e585c2818e18a2c P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.031s
W20260709 13:20:55.642453 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 50955us
W20260709 13:20:55.587157 16266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 116601us
W20260709 13:20:55.643710 16266 leader_election.cc:336] T 66ae176961b74813a58bc15c9c4c6f2a P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 13 election: RPC error from VoteRequest() call to peer cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923): Timed out: RequestConsensusVote RPC to 127.14.177.132:38923 timed out after 1.924s (SENT)
I20260709 13:20:55.644411 16266 leader_election.cc:304] T 66ae176961b74813a58bc15c9c4c6f2a P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860
I20260709 13:20:55.543203 20578 raft_consensus.cc:740] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 1ca2725d290e47d68b1a4b609c9816ac [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:20:55.641024 16265 leader_election.cc:304] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:20:55.550240 20599 raft_consensus.cc:493] T 31969392ccdb43b491911a319e8bc7e1 P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:20:55.649186 20599 raft_consensus.cc:515] T 31969392ccdb43b491911a319e8bc7e1 P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:55.650682 16824 raft_consensus.cc:1240] T af57996a8be84d398f0f023e8d08ef77 P cb82cc7bd6db46d980c75237a1402860 [term 13 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 12. Current term is 13. Ops: []
I20260709 13:20:55.651470 20599 leader_election.cc:290] T 31969392ccdb43b491911a319e8bc7e1 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:55.655689 16294 raft_consensus.cc:1275] T 26354361e9b044cea1c2b86c4882fe92 P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:20:55.642679 16820 raft_consensus.cc:740] T 3884f864a96d428785895e5463e5728f P cb82cc7bd6db46d980c75237a1402860 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:20:55.660475 16820 consensus_queue.cc:260] T 3884f864a96d428785895e5463e5728f P cb82cc7bd6db46d980c75237a1402860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:55.662621 16820 raft_consensus.cc:3060] T 3884f864a96d428785895e5463e5728f P cb82cc7bd6db46d980c75237a1402860 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:20:55.665509 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 190449us
W20260709 13:20:55.665794 16789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 84863.4us
I20260709 13:20:55.669797 16825 raft_consensus.cc:2468] T b87147731a68400e8cce9b879daa7c5d P cb82cc7bd6db46d980c75237a1402860 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 13.
I20260709 13:20:55.526274 20577 consensus_queue.cc:1048] T 66f75124215f4ca6bef831e174a792d8 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.111s
I20260709 13:20:55.537307 20593 consensus_queue.cc:260] T 7c2095a5f4ad49408dbc7f69c606120d P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:55.673976 20589 leader_election.cc:290] T ecc1e363230043fe9fef390f0b979f20 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:55.674507 20600 consensus_queue.cc:237] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:55.643357 20612 leader_election.cc:290] T 434bff3082d141838068f323bdb0d1ed P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 11 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
W20260709 13:20:55.675985 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 115026us
I20260709 13:20:55.585507 20576 leader_election.cc:290] T 1df93779209748b9860f6d893dd3de53 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 12 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:20:55.677852 16190 leader_election.cc:304] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860
I20260709 13:20:55.637260 20268 leader_election.cc:290] T d2ab0ac8347641e6955e9bce7d90c1a2 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 12 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:55.638825 20267 raft_consensus.cc:493] T 68a6838d82ef4cac83507f906139d90f P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:20:55.716415 16291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 13 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:20:55.753901 16291 raft_consensus.cc:3060] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:20:55.546528 20236 consensus_queue.cc:237] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.11, Last appended by leader: 11, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:55.714866 20424 raft_consensus.cc:2804] T d16dfad0c3b540e5b4b86492ddf87208 P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:20:55.555294 20571 consensus_queue.cc:1059] T e4dc6b0541ca431ebc757beac00521f1 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: INVALID_TERM, Last received: 10.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.122s
I20260709 13:20:55.636173 20641 consensus_queue.cc:1059] T 26354361e9b044cea1c2b86c4882fe92 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 7, Time since last communication: 0.122s
I20260709 13:20:55.742328 20518 consensus_queue.cc:1059] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:20:55.744671 16191 leader_election.cc:304] T c0eab25ce4c64e018a041beeda2867db P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 90f3da57796b488c9087a2f523adf045
I20260709 13:20:55.715584 20293 raft_consensus.cc:2749] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:20:55.730649 20269 raft_consensus.cc:3055] T 5a959dbecc2648eb85d1cc550f5a3245 P 1ca2725d290e47d68b1a4b609c9816ac [term 9 LEADER]: Stepping down as leader of term 9
W20260709 13:20:55.760550 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 82790us
I20260709 13:20:55.670928 20609 consensus_queue.cc:1048] T ee9ed67006144797b1de6ea8c2013a68 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.080s
I20260709 13:20:55.736742 20470 raft_consensus.cc:3060] T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:20:55.785912 16263 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:20:55.803035 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 159723us
I20260709 13:20:55.801796 16296 raft_consensus.cc:2468] T ecc1e363230043fe9fef390f0b979f20 P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 11.
W20260709 13:20:55.796480 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 155526us
I20260709 13:20:55.739542 20599 raft_consensus.cc:2749] T d2ab0ac8347641e6955e9bce7d90c1a2 P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:20:55.888509 16295 raft_consensus.cc:1275] T 7c2095a5f4ad49408dbc7f69c606120d P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 13:20:55.727821 16217 raft_consensus.cc:2468] T ecc1e363230043fe9fef390f0b979f20 P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 11.
I20260709 13:20:55.775122 20579 raft_consensus.cc:2749] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:20:55.856983 20269 raft_consensus.cc:740] T 5a959dbecc2648eb85d1cc550f5a3245 P 1ca2725d290e47d68b1a4b609c9816ac [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:20:55.744397 20434 consensus_queue.cc:1048] T 26354361e9b044cea1c2b86c4882fe92 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.081s
I20260709 13:20:55.797084 20640 raft_consensus.cc:2804] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:20:55.828079 20593 raft_consensus.cc:3060] T 7c2095a5f4ad49408dbc7f69c606120d P 1ca2725d290e47d68b1a4b609c9816ac [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:20:55.864748 16266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 221049us
I20260709 13:20:55.933794 16223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 13 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:20:55.823995 20424 raft_consensus.cc:697] T d16dfad0c3b540e5b4b86492ddf87208 P 1ca2725d290e47d68b1a4b609c9816ac [term 12 LEADER]: Becoming Leader. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:20:55.939697 16218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c61a39fd4f84424b129c1a6f9ba404d" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:20:55.741055 20618 consensus_queue.cc:1059] T af57996a8be84d398f0f023e8d08ef77 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.096s
W20260709 13:20:55.934106 16266 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.14.177.132:38923 timed out after 1.934s (SENT) [suppressed 78 similar messages]
I20260709 13:20:55.908012 16146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e9b2bf7011f4ae688a79d7bb5a447b4" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
W20260709 13:20:55.903342 16789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 234265us
I20260709 13:20:55.912654 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d16dfad0c3b540e5b4b86492ddf87208" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:55.767469 20267 raft_consensus.cc:515] T 68a6838d82ef4cac83507f906139d90f P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:55.855568 20603 raft_consensus.cc:697] T 5a959dbecc2648eb85d1cc550f5a3245 P 90f3da57796b488c9087a2f523adf045 [term 10 LEADER]: Becoming Leader. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:20:55.921530 16116 leader_election.cc:304] T ecc1e363230043fe9fef390f0b979f20 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:20:55.842203 20518 consensus_queue.cc:1243] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Peer 647a343a542d4a139fe106c19095129b log is divergent from this leader: its last log entry 9.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 13:20:55.934468 16223 raft_consensus.cc:3060] T 19baa7bc367844c288bca8f81439c413 P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:20:55.829375 20475 leader_election.cc:290] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 13 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:55.945413 16146 raft_consensus.cc:2376] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 9. Re-sending same reply.
I20260709 13:20:55.956617 16266 leader_election.cc:304] T 23f82224f4564fea93572e8206bad1fe P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
W20260709 13:20:55.957710 16265 leader_election.cc:336] T 5a959dbecc2648eb85d1cc550f5a3245 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943): Timed out: RequestConsensusVote RPC to 127.14.177.130:35943 timed out after 2.145s (SENT)
W20260709 13:20:55.958710 16265 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:55.905019 20639 raft_consensus.cc:2804] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [term 15 FOLLOWER]: Leader election won for term 15
W20260709 13:20:55.963050 16816 consensus_meta.cc:287] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860: Time spent flushing consensus metadata: real 0.566s	user 0.009s	sys 0.001s
I20260709 13:20:55.963577 16816 raft_consensus.cc:2468] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 10.
I20260709 13:20:55.924662 20553 raft_consensus.cc:2804] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:20:55.798110 20578 consensus_queue.cc:260] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:55.933427 20640 raft_consensus.cc:493] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:55.982949 16295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ab0ac8347641e6955e9bce7d90c1a2" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:20:55.983673 16296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434bff3082d141838068f323bdb0d1ed" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
W20260709 13:20:55.969871 16265 leader_election.cc:336] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943): Timed out: RequestConsensusVote RPC to 127.14.177.130:35943 timed out after 2.180s (SENT)
I20260709 13:20:55.887355 20571 consensus_queue.cc:1059] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.073s
I20260709 13:20:55.983680 16295 raft_consensus.cc:3060] T d2ab0ac8347641e6955e9bce7d90c1a2 P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:20:55.985812 16294 raft_consensus.cc:1240] T ecc1e363230043fe9fef390f0b979f20 P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 10. Current term is 11. Ops: []
I20260709 13:20:55.979382 16146 raft_consensus.cc:1240] T 3884f864a96d428785895e5463e5728f P 647a343a542d4a139fe106c19095129b [term 8 LEADER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 7. Current term is 8. Ops: []
I20260709 13:20:55.955596 20293 raft_consensus.cc:2804] T c0eab25ce4c64e018a041beeda2867db P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:20:55.933262 20431 consensus_queue.cc:1048] T 66f75124215f4ca6bef831e174a792d8 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.225s
I20260709 13:20:56.008723 16146 raft_consensus.cc:1240] T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 10. Current term is 11. Ops: []
I20260709 13:20:56.001089 16265 leader_election.cc:304] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:20:56.014874 16265 leader_election.cc:304] T c4f65cdf00b748bb963ee6e81ef43b79 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
W20260709 13:20:56.002473 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 81035.3us
I20260709 13:20:56.005795 16149 raft_consensus.cc:1240] T ecc1e363230043fe9fef390f0b979f20 P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 10. Current term is 11. Ops: []
W20260709 13:20:55.991156 20604 consensus_meta.cc:287] T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860: Time spent flushing consensus metadata: real 0.835s	user 0.000s	sys 0.011s
I20260709 13:20:55.980263 20518 raft_consensus.cc:2749] T 23f82224f4564fea93572e8206bad1fe P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:20:56.033286 20635 log.cc:927] Time spent T 3884f864a96d428785895e5463e5728f P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.423s	user 0.000s	sys 0.001s
I20260709 13:20:56.034150 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434bff3082d141838068f323bdb0d1ed" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:20:56.026551 16265 leader_election.cc:304] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:20:55.971886 20579 consensus_queue.cc:1059] T c2f5f89186204242bf377f7c42b2e2da P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.051s
I20260709 13:20:55.994351 20640 raft_consensus.cc:3060] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:20:56.042536 16816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:20:55.998517 20578 raft_consensus.cc:3060] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:20:55.934849 20269 consensus_queue.cc:260] T 5a959dbecc2648eb85d1cc550f5a3245 P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:56.057530 20657 raft_consensus.cc:493] T 8cc959fb7f704fba9009c0595467d074 P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:20:56.058230 20657 raft_consensus.cc:515] T 8cc959fb7f704fba9009c0595467d074 P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:56.016980 20599 raft_consensus.cc:2749] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
W20260709 13:20:56.065893 16790 connection.cc:256] RPC call timeout handler was delayed by -1.5794997780001268s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1515s for a response
I20260709 13:20:56.027961 20611 leader_election.cc:290] T af57996a8be84d398f0f023e8d08ef77 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 13 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
W20260709 13:20:56.097157 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 352560us
I20260709 13:20:55.981020 20659 consensus_queue.cc:1048] T 7c2095a5f4ad49408dbc7f69c606120d P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.104s
W20260709 13:20:56.098156 16788 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:56.100267 16146 raft_consensus.cc:1240] T 23f82224f4564fea93572e8206bad1fe P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 8. Current term is 9. Ops: []
I20260709 13:20:56.033949 20644 raft_consensus.cc:2749] T 66ae176961b74813a58bc15c9c4c6f2a P 90f3da57796b488c9087a2f523adf045 [term 14 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
W20260709 13:20:56.101085 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 187179us
I20260709 13:20:56.060523 20293 raft_consensus.cc:697] T c0eab25ce4c64e018a041beeda2867db P 1ca2725d290e47d68b1a4b609c9816ac [term 11 LEADER]: Becoming Leader. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:20:56.028414 20457 raft_consensus.cc:515] T 02838c86531d4f98bf58d1c22b8baf77 P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:20:56.120223 16190 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:20:56.109036 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 82575.8us
W20260709 13:20:56.154340 16190 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:56.042999 20639 raft_consensus.cc:697] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [term 15 LEADER]: Becoming Leader. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:20:56.154577 16265 leader_election.cc:304] T 7c2095a5f4ad49408dbc7f69c606120d P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:20:56.098680 20475 raft_consensus.cc:2804] T ecc1e363230043fe9fef390f0b979f20 P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:20:56.113453 20269 raft_consensus.cc:3060] T 5a959dbecc2648eb85d1cc550f5a3245 P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:20:56.113716 20518 raft_consensus.cc:2749] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:20:56.022353 20424 consensus_queue.cc:237] T d16dfad0c3b540e5b4b86492ddf87208 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:56.169384 16149 raft_consensus.cc:1240] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 14. Current term is 15. Ops: []
I20260709 13:20:56.169384 16146 raft_consensus.cc:1240] T 5a959dbecc2648eb85d1cc550f5a3245 P 647a343a542d4a139fe106c19095129b [term 10 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 9. Current term is 10. Ops: []
W20260709 13:20:56.166215 16192 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 251815us
I20260709 13:20:56.210835 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d16dfad0c3b540e5b4b86492ddf87208" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:20:56.211969 16149 raft_consensus.cc:3060] T d16dfad0c3b540e5b4b86492ddf87208 P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:20:56.213912 16192 leader_election.cc:304] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:20:56.035130 20604 raft_consensus.cc:515] T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:56.210806 16146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8158bbd17b994d2eb96f560bd0f1b88d" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:20:56.218961 16146 raft_consensus.cc:3060] T 8158bbd17b994d2eb96f560bd0f1b88d P 647a343a542d4a139fe106c19095129b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:20:56.219295 16142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18ad7e3af5504571a69d10c00750a01c" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:20:56.220016 16142 raft_consensus.cc:2393] T 18ad7e3af5504571a69d10c00750a01c P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 11: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
I20260709 13:20:56.220192 16219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8638ee363273426db1393dd81e5c7b72" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:56.181493 16144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b7d1ab519044eda2d5e04a07242f25" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:20:56.158895 16223 raft_consensus.cc:2468] T 19baa7bc367844c288bca8f81439c413 P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 13.
I20260709 13:20:56.221486 16144 raft_consensus.cc:2393] T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 10: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
I20260709 13:20:56.224797 16226 raft_consensus.cc:1275] T 02838c86531d4f98bf58d1c22b8baf77 P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:20:56.220815 16219 raft_consensus.cc:2468] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 12.
I20260709 13:20:56.233687 16219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a878999c0024d2abe517484180290fc" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:56.232494 16144 raft_consensus.cc:1240] T 7c2095a5f4ad49408dbc7f69c606120d P 647a343a542d4a139fe106c19095129b [term 9 LEADER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 8. Current term is 9. Ops: [8.5-8.5]
I20260709 13:20:56.233232 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9944d5e4b8c4109af9955c4f08c9095" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:56.231436 16220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18ad7e3af5504571a69d10c00750a01c" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:56.241004 16220 raft_consensus.cc:2393] T 18ad7e3af5504571a69d10c00750a01c P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 11: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
I20260709 13:20:56.243645 16219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada5a004ba7c46819051e3ccec71ffdb" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:56.244249 16143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d4ab870a05347868956aa8b841b2981" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:56.244544 16219 raft_consensus.cc:2393] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 in current term 12: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
I20260709 13:20:56.254022 16226 raft_consensus.cc:2393] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 in current term 12: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
I20260709 13:20:56.255139 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62fe3346cd3c42c4a55e14af9073bca4" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:20:56.260241 16223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:56.261082 16223 raft_consensus.cc:2393] T ecc1e363230043fe9fef390f0b979f20 P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 11: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
W20260709 13:20:56.262347 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 107877us
I20260709 13:20:56.132107 20611 consensus_queue.cc:1059] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.102s
I20260709 13:20:56.184165 20579 raft_consensus.cc:3055] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 1ca2725d290e47d68b1a4b609c9816ac [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:20:56.256393 16219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c61a39fd4f84424b129c1a6f9ba404d" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:56.263810 16226 raft_consensus.cc:2410] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 for term 15 because replica has last-logged OpId of term: 14 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 10.
I20260709 13:20:56.266737 16222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdbd78acb4cf46298e0de8ba0d130e19" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:56.267383 16822 raft_consensus.cc:2368] T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac for earlier term 12. Current term is 13.
I20260709 13:20:56.275338 20664 raft_consensus.cc:493] T fffab26427e24d398e2695aad40ca134 P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:20:56.276027 20664 raft_consensus.cc:515] T fffab26427e24d398e2695aad40ca134 P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:56.278784 20664 leader_election.cc:290] T fffab26427e24d398e2695aad40ca134 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
W20260709 13:20:56.279575 20565 consensus_meta.cc:287] T ee9ed67006144797b1de6ea8c2013a68 P 647a343a542d4a139fe106c19095129b: Time spent flushing consensus metadata: real 0.728s	user 0.001s	sys 0.013s
I20260709 13:20:56.279691 20670 raft_consensus.cc:493] T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:20:56.261557 20647 log.cc:927] Time spent T e5ad4eb281564936a59f41e387a47156 P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 0.348s	user 0.001s	sys 0.000s
W20260709 13:20:56.280640 20634 log.cc:927] Time spent T 8a878999c0024d2abe517484180290fc P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.453s	user 0.000s	sys 0.001s
I20260709 13:20:56.280911 20667 raft_consensus.cc:493] T 3a7ca61592e848ea9b0fb81a2ce33cdc P cb82cc7bd6db46d980c75237a1402860 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:56.181218 20661 raft_consensus.cc:2749] T c4f65cdf00b748bb963ee6e81ef43b79 P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:20:56.282568 16147 raft_consensus.cc:1275] T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 13:20:56.346463 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66ae176961b74813a58bc15c9c4c6f2a" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:20:56.347630 16224 raft_consensus.cc:2376] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 14. Re-sending same reply.
I20260709 13:20:56.347903 16147 raft_consensus.cc:1240] T 5a96373bb0e14dce8a266b2fc6c58507 P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 10. Current term is 11. Ops: []
I20260709 13:20:56.286339 16220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1df93779209748b9860f6d893dd3de53" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
W20260709 13:20:56.295871 16789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 275322us
I20260709 13:20:56.350714 16220 raft_consensus.cc:3060] T 1df93779209748b9860f6d893dd3de53 P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:20:56.296736 16295 raft_consensus.cc:2468] T d2ab0ac8347641e6955e9bce7d90c1a2 P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 12.
I20260709 13:20:56.298396 16223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af57996a8be84d398f0f023e8d08ef77" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
W20260709 13:20:56.280352 20650 log.cc:927] Time spent T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.407s	user 0.001s	sys 0.000s
I20260709 13:20:56.261157 20657 leader_election.cc:290] T 8cc959fb7f704fba9009c0595467d074 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:20:56.367964 20657 consensus_queue.cc:1059] T ada5a004ba7c46819051e3ccec71ffdb P 90f3da57796b488c9087a2f523adf045 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:20:56.368501 20268 raft_consensus.cc:3055] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:20:56.369035 20268 raft_consensus.cc:740] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:20:56.369128 20657 consensus_queue.cc:1059] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:20:56.370256 20657 raft_consensus.cc:3055] T ada5a004ba7c46819051e3ccec71ffdb P 90f3da57796b488c9087a2f523adf045 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:20:56.370711 20657 raft_consensus.cc:740] T ada5a004ba7c46819051e3ccec71ffdb P 90f3da57796b488c9087a2f523adf045 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:20:56.313776 16219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdbd78acb4cf46298e0de8ba0d130e19" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:56.330943 16146 raft_consensus.cc:2468] T 8158bbd17b994d2eb96f560bd0f1b88d P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 11.
I20260709 13:20:56.120895 20603 consensus_queue.cc:237] T 5a959dbecc2648eb85d1cc550f5a3245 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:20:56.337502 16299 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.700s	user 0.019s	sys 0.021s
W20260709 13:20:56.252259 16820 consensus_meta.cc:287] T 3884f864a96d428785895e5463e5728f P cb82cc7bd6db46d980c75237a1402860: Time spent flushing consensus metadata: real 0.589s	user 0.010s	sys 0.000s
I20260709 13:20:56.315975 20664 raft_consensus.cc:493] T 23f82224f4564fea93572e8206bad1fe P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:56.360455 16222 raft_consensus.cc:1275] T ee9ed67006144797b1de6ea8c2013a68 P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
W20260709 13:20:56.360772 20652 log.cc:927] Time spent T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 0.327s	user 0.000s	sys 0.001s
I20260709 13:20:56.361266 20267 leader_election.cc:290] T 68a6838d82ef4cac83507f906139d90f P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:20:56.362663 16265 leader_election.cc:304] T e5ad4eb281564936a59f41e387a47156 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 647a343a542d4a139fe106c19095129b
I20260709 13:20:56.362969 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c11074579b046d99e585c2818e18a2c" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 12 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
W20260709 13:20:56.412891 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 50327.6us
I20260709 13:20:56.414088 16789 leader_election.cc:304] T d2ab0ac8347641e6955e9bce7d90c1a2 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:20:56.414378 16295 raft_consensus.cc:1240] T 2794af5b99984efebdee564035841cb0 P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 8. Current term is 9. Ops: []
I20260709 13:20:56.364066 16223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62fe3346cd3c42c4a55e14af9073bca4" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:56.355302 20667 raft_consensus.cc:515] T 3a7ca61592e848ea9b0fb81a2ce33cdc P cb82cc7bd6db46d980c75237a1402860 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:20:56.365190 20645 log.cc:927] Time spent T b87147731a68400e8cce9b879daa7c5d P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 0.258s	user 0.000s	sys 0.001s
W20260709 13:20:56.366420 20668 log.cc:927] Time spent T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860: Append to log took a long time: real 0.190s	user 0.000s	sys 0.000s
W20260709 13:20:56.367242 20653 log.cc:927] Time spent T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860: Append to log took a long time: real 0.279s	user 0.001s	sys 0.000s
W20260709 13:20:56.371344 20665 log.cc:927] Time spent T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860: Append to log took a long time: real 0.068s	user 0.001s	sys 0.000s
I20260709 13:20:56.285248 20579 raft_consensus.cc:740] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 1ca2725d290e47d68b1a4b609c9816ac [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:20:56.335942 20670 raft_consensus.cc:515] T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:20:56.373075 20643 log.cc:927] Time spent T e5ad4eb281564936a59f41e387a47156 P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 0.437s	user 0.000s	sys 0.001s
I20260709 13:20:56.347903 20518 consensus_queue.cc:1059] T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.079s
I20260709 13:20:56.300101 20584 raft_consensus.cc:2749] T 7c2095a5f4ad49408dbc7f69c606120d P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:20:56.395829 16292 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48674 (request call id 2690) took 1304 ms. Trace:
I20260709 13:20:56.396194 16219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c11074579b046d99e585c2818e18a2c" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 12 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
W20260709 13:20:56.399646 16299 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.762s	user 0.019s	sys 0.021s
I20260709 13:20:56.401185 16820 raft_consensus.cc:1275] T 3884f864a96d428785895e5463e5728f P cb82cc7bd6db46d980c75237a1402860 [term 8 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
W20260709 13:20:56.389627 16291 consensus_meta.cc:287] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045: Time spent flushing consensus metadata: real 0.632s	user 0.010s	sys 0.002s
I20260709 13:20:56.415753 16223 raft_consensus.cc:2410] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 for term 15 because replica has last-logged OpId of term: 14 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 10.
I20260709 13:20:56.487401 16291 raft_consensus.cc:2410] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b for term 13 because replica has last-logged OpId of term: 9 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 7.
W20260709 13:20:56.230960 19982 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.661s	user 0.010s	sys 0.002s
I20260709 13:20:56.335278 20611 raft_consensus.cc:493] T 829b4ad8c5a54eababeea1181b960d4b P cb82cc7bd6db46d980c75237a1402860 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:20:56.421437 16192 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 207593us
I20260709 13:20:56.361816 20639 consensus_queue.cc:237] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:56.443179 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a378d7f796f94615b3b347119a9880fd" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:20:56.315649 20565 raft_consensus.cc:515] T ee9ed67006144797b1de6ea8c2013a68 P 647a343a542d4a139fe106c19095129b [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:56.447252 16222 raft_consensus.cc:1240] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 11. Current term is 12. Ops: [11.11-11.11]
I20260709 13:20:56.313058 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee9ed67006144797b1de6ea8c2013a68" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:20:56.453460 16224 raft_consensus.cc:1240] T ecc1e363230043fe9fef390f0b979f20 P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 10. Current term is 11. Ops: [8.6-10.7]
I20260709 13:20:56.456535 20685 raft_consensus.cc:493] T d2ab0ac8347641e6955e9bce7d90c1a2 P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:56.589963 16820 raft_consensus.cc:1275] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 [term 12 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:20:56.457901 20598 leader_election.cc:290] T 9e65676b21b748728ad39668cfe722ec P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 11 election: Requested vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:56.459697 20672 raft_consensus.cc:2804] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:20:56.460156 20676 raft_consensus.cc:2804] T d2ab0ac8347641e6955e9bce7d90c1a2 P cb82cc7bd6db46d980c75237a1402860 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:20:56.419956 20664 raft_consensus.cc:515] T 23f82224f4564fea93572e8206bad1fe P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:56.437979 20669 raft_consensus.cc:3055] T c2f5f89186204242bf377f7c42b2e2da P 1ca2725d290e47d68b1a4b609c9816ac [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:20:56.486554 16292 rpcz_store.cc:276] 0709 13:20:55.069968 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:55.133857 (+ 63889us) service_pool.cc:224] Handling call
0709 13:20:55.134179 (+   322us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:20:55.134554 (+   375us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:55.134583 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:20:55.188767 (+ 54184us) log.cc:844] Serialized 26 byte log entry
0709 13:20:55.322266 (+133499us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:20:55.322305 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:55.322365 (+    60us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:20:56.374159 (+1051794us) raft_consensus.cc:1685] finished
0709 13:20:56.374184 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:56.374353 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":108,"raft.run_cpu_time_us":120,"raft.run_wall_time_us":7879,"thread_start_us":100016,"threads_started":1,"wal-append.queue_time_us":264396}
I20260709 13:20:56.426594 20644 raft_consensus.cc:2804] T e5ad4eb281564936a59f41e387a47156 P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:20:56.576349 16297 raft_consensus.cc:3060] T a378d7f796f94615b3b347119a9880fd P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:20:56.573374 16265 leader_election.cc:304] T 8638ee363273426db1393dd81e5c7b72 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:20:56.656931 16222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68a6838d82ef4cac83507f906139d90f" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
W20260709 13:20:56.657959 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 84677.8us
I20260709 13:20:56.598428 16223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ab0ac8347641e6955e9bce7d90c1a2" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:20:56.603279 16220 raft_consensus.cc:2468] T 1df93779209748b9860f6d893dd3de53 P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 12.
I20260709 13:20:56.610554 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cc959fb7f704fba9009c0595467d074" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
W20260709 13:20:56.574692 19982 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.999s	user 0.011s	sys 0.003s
I20260709 13:20:56.611999 16225 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57604 (request call id 2607) took 1366 ms. Trace:
I20260709 13:20:56.611340 16300 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48674 (request call id 2699) took 1108 ms. Trace:
I20260709 13:20:56.399119 20293 consensus_queue.cc:237] T c0eab25ce4c64e018a041beeda2867db P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:56.573607 20659 raft_consensus.cc:493] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:56.570619 20475 raft_consensus.cc:697] T ecc1e363230043fe9fef390f0b979f20 P 647a343a542d4a139fe106c19095129b [term 11 LEADER]: Becoming Leader. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:20:56.585826 20661 raft_consensus.cc:3055] T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:20:56.570533 20622 raft_consensus.cc:493] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:56.370746 20268 consensus_queue.cc:260] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:56.651314 20598 raft_consensus.cc:493] T 5a96373bb0e14dce8a266b2fc6c58507 P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:20:56.659047 16789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 234453us
I20260709 13:20:56.585495 20611 raft_consensus.cc:515] T 829b4ad8c5a54eababeea1181b960d4b P cb82cc7bd6db46d980c75237a1402860 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:20:56.659137 16189 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.720s	user 0.019s	sys 0.084s
I20260709 13:20:56.651547 20457 leader_election.cc:290] T 02838c86531d4f98bf58d1c22b8baf77 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 12 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
W20260709 13:20:56.677358 16790 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.565s	user 0.002s	sys 0.010s
I20260709 13:20:56.646034 20676 raft_consensus.cc:697] T d2ab0ac8347641e6955e9bce7d90c1a2 P cb82cc7bd6db46d980c75237a1402860 [term 12 LEADER]: Becoming Leader. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:20:56.663839 16300 rpcz_store.cc:276] 0709 13:20:55.503327 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:55.503515 (+   188us) service_pool.cc:224] Handling call
0709 13:20:55.503851 (+   336us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:20:55.504177 (+   326us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:55.504218 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:20:55.504721 (+   503us) log.cc:844] Serialized 26 byte log entry
0709 13:20:55.670938 (+166217us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:20:55.670986 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:55.671061 (+    75us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:20:56.611011 (+939950us) raft_consensus.cc:1685] finished
0709 13:20:56.611035 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:56.611285 (+   250us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1007,"raft.run_cpu_time_us":139,"raft.run_wall_time_us":18205,"thread_start_us":165867,"threads_started":1,"wal-append.queue_time_us":207667}
I20260709 13:20:56.663722 16225 rpcz_store.cc:276] 0709 13:20:55.230957 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:55.236279 (+  5322us) service_pool.cc:224] Handling call
0709 13:20:55.257901 (+ 21622us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:20:55.277534 (+ 19633us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:55.277566 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:20:55.285905 (+  8339us) log.cc:844] Serialized 26 byte log entry
0709 13:20:55.443953 (+158048us) spinlock_profiling.cc:243] Waited 2.77 ms on lock 0x7b18002cd880. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b693ade1 00007f69b69475b1 00007f69ada54e07 00007f69b68afab9 00007f69ada7169b 00007f69ada6f93f 00007f69ada71c02 00007f69b06f920b 00007f69b06fed23 00007f69b06ff0d1 00007f69b0b1aa0f
0709 13:20:55.664793 (+220840us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:20:55.664832 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:55.664899 (+    67us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:20:56.571590 (+906691us) raft_consensus.cc:1685] finished
0709 13:20:56.571609 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:56.597186 (+ 25577us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4312,"raft.queue_time_us":9498,"raft.run_cpu_time_us":110,"raft.run_wall_time_us":6681,"spinlock_wait_cycles":6373120,"thread_start_us":220670,"threads_started":1,"wal-append.queue_time_us":512944}
I20260709 13:20:56.605624 20685 raft_consensus.cc:515] T d2ab0ac8347641e6955e9bce7d90c1a2 P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:56.568670 20657 consensus_queue.cc:260] T ada5a004ba7c46819051e3ccec71ffdb P 90f3da57796b488c9087a2f523adf045 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:20:56.682067 20583 consensus_meta.cc:287] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 90f3da57796b488c9087a2f523adf045: Time spent flushing consensus metadata: real 1.130s	user 0.007s	sys 0.007s
W20260709 13:20:56.779807 16189 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.841s	user 0.022s	sys 0.092s
W20260709 13:20:56.780207 16790 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.668s	user 0.003s	sys 0.018s
I20260709 13:20:56.755873 16151 raft_consensus.cc:1240] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b [term 10 LEADER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 9. Current term is 10. Ops: []
I20260709 13:20:56.649654 20672 raft_consensus.cc:697] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac [term 10 LEADER]: Becoming Leader. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:20:56.663101 20644 raft_consensus.cc:493] T e5ad4eb281564936a59f41e387a47156 P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 647a343a542d4a139fe106c19095129b)
W20260709 13:20:56.789593 16788 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:56.663178 20674 raft_consensus.cc:493] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:56.665274 16265 leader_election.cc:304] T 8a878999c0024d2abe517484180290fc P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
W20260709 13:20:56.799036 16142 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.507s	user 0.017s	sys 0.023s
W20260709 13:20:56.658550 20618 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.678s	user 0.004s	sys 0.011s
I20260709 13:20:56.649160 20579 consensus_queue.cc:260] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:56.781219 16220 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20"
dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
mode: GRACEFUL
new_leader_uuid: "90f3da57796b488c9087a2f523adf045"
 from {username='slave'} at 127.0.0.1:39144
I20260709 13:20:56.670644 20584 raft_consensus.cc:2804] T 8638ee363273426db1393dd81e5c7b72 P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:20:56.957698 16299 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48684 (request call id 2544) took 1659 ms. Trace:
I20260709 13:20:56.852015 16222 raft_consensus.cc:1275] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
W20260709 13:20:56.942389 16149 consensus_meta.cc:287] T d16dfad0c3b540e5b4b86492ddf87208 P 647a343a542d4a139fe106c19095129b: Time spent flushing consensus metadata: real 0.701s	user 0.008s	sys 0.000s
W20260709 13:20:57.134363 16142 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.843s	user 0.033s	sys 0.045s
I20260709 13:20:57.134703 16149 raft_consensus.cc:2468] T d16dfad0c3b540e5b4b86492ddf87208 P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 12.
W20260709 13:20:57.134845 16790 connection.cc:256] RPC call timeout handler was delayed by -1.2212997780000023s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19130000000000003s for a response
W20260709 13:20:57.135416 16790 connection.cc:256] RPC call timeout handler was delayed by -1.0034997780001049s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2155s for a response
W20260709 13:20:57.137372 16790 connection.cc:256] RPC call timeout handler was delayed by -1.176752333999957s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2149s for a response
I20260709 13:20:57.138401 16220 raft_consensus.cc:606] T ecc1e363230043fe9fef390f0b979f20 P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Received request to transfer leadership to TS 90f3da57796b488c9087a2f523adf045
I20260709 13:20:57.138779 16220 raft_consensus.cc:614] T ecc1e363230043fe9fef390f0b979f20 P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:20:57.133062 16299 rpcz_store.cc:276] 0709 13:20:55.298537 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:55.439447 (+140910us) service_pool.cc:224] Handling call
0709 13:20:55.460245 (+ 20798us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:20:55.471522 (+ 11277us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:55.471554 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:20:55.529092 (+ 57538us) log.cc:844] Serialized 26 byte log entry
0709 13:20:56.486913 (+957821us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:20:56.486958 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:56.487034 (+    76us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:20:56.957462 (+470428us) raft_consensus.cc:1685] finished
0709 13:20:56.957485 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:56.957637 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":17630,"raft.run_cpu_time_us":47,"raft.run_wall_time_us":46,"thread_start_us":849454,"threads_started":1,"wal-append.queue_time_us":1293097}
I20260709 13:20:56.685372 19982 raft_consensus.cc:493] T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:20:57.143235 16788 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:56.662662 20669 raft_consensus.cc:740] T c2f5f89186204242bf377f7c42b2e2da P 1ca2725d290e47d68b1a4b609c9816ac [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:20:57.147477 16297 raft_consensus.cc:2468] T a378d7f796f94615b3b347119a9880fd P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 13.
W20260709 13:20:57.161437 16789 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:20:56.909307 20269 consensus_meta.cc:287] T 5a959dbecc2648eb85d1cc550f5a3245 P 1ca2725d290e47d68b1a4b609c9816ac: Time spent flushing consensus metadata: real 0.727s	user 0.012s	sys 0.000s
W20260709 13:20:57.012085 20640 consensus_meta.cc:287] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045: Time spent flushing consensus metadata: real 0.898s	user 0.011s	sys 0.001s
W20260709 13:20:56.999655 20607 consensus_meta.cc:287] T 9e65676b21b748728ad39668cfe722ec P cb82cc7bd6db46d980c75237a1402860: Time spent flushing consensus metadata: real 1.834s	user 0.007s	sys 0.009s
I20260709 13:20:57.174613 16223 raft_consensus.cc:3060] T d2ab0ac8347641e6955e9bce7d90c1a2 P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:20:56.986223 20573 consensus_meta.cc:287] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 90f3da57796b488c9087a2f523adf045: Time spent flushing consensus metadata: real 1.463s	user 0.012s	sys 0.000s
W20260709 13:20:56.924461 20648 log.cc:927] Time spent T 806d7690c2b644959152074bb2cab0a7 P cb82cc7bd6db46d980c75237a1402860: Append to log took a long time: real 0.269s	user 0.000s	sys 0.001s
W20260709 13:20:57.140836 20618 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.157s	user 0.009s	sys 0.037s
I20260709 13:20:56.680297 20661 raft_consensus.cc:740] T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:20:56.873943 20268 raft_consensus.cc:3060] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:20:57.288146 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 150316us
W20260709 13:20:57.147398 20470 consensus_meta.cc:287] T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac: Time spent flushing consensus metadata: real 1.237s	user 0.007s	sys 0.005s
I20260709 13:20:56.676779 20518 consensus_queue.cc:1048] T 02838c86531d4f98bf58d1c22b8baf77 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.492s
W20260709 13:20:57.202693 16217 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.713s	user 0.000s	sys 0.018s
I20260709 13:20:56.993562 20667 leader_election.cc:290] T 3a7ca61592e848ea9b0fb81a2ce33cdc P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:56.883608 20683 raft_consensus.cc:493] T c0eab25ce4c64e018a041beeda2867db P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:56.911845 20691 raft_consensus.cc:493] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:20:56.998755 20670 leader_election.cc:290] T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
W20260709 13:20:56.910029 20666 raft_consensus.cc:670] T 806d7690c2b644959152074bb2cab0a7 P cb82cc7bd6db46d980c75237a1402860: failed to trigger leader election: Illegal state: only voting members can start elections: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } } [suppressed 3 similar messages]
I20260709 13:20:57.151842 20583 raft_consensus.cc:515] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:57.005769 20664 leader_election.cc:290] T 23f82224f4564fea93572e8206bad1fe P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
W20260709 13:20:57.160208 20593 consensus_meta.cc:287] T 7c2095a5f4ad49408dbc7f69c606120d P 1ca2725d290e47d68b1a4b609c9816ac: Time spent flushing consensus metadata: real 1.216s	user 0.000s	sys 0.012s
W20260709 13:20:57.209858 16115 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.789s	user 0.005s	sys 0.015s
W20260709 13:20:57.424248 16115 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.004s	user 0.005s	sys 0.015s
I20260709 13:20:57.424422 16220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fffab26427e24d398e2695aad40ca134" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:57.144693 20584 raft_consensus.cc:493] T 8638ee363273426db1393dd81e5c7b72 P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Starting leader election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:20:56.785792 20598 raft_consensus.cc:515] T 5a96373bb0e14dce8a266b2fc6c58507 P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:57.132987 20644 raft_consensus.cc:3060] T e5ad4eb281564936a59f41e387a47156 P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:20:56.962006 20657 raft_consensus.cc:3060] T ada5a004ba7c46819051e3ccec71ffdb P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:20:57.202260 20579 raft_consensus.cc:3060] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:20:57.188758 20688 raft_consensus.cc:493] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:57.404636 20470 raft_consensus.cc:493] T 8a878999c0024d2abe517484180290fc P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:20:57.434530 20688 raft_consensus.cc:515] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:57.434695 20470 raft_consensus.cc:515] T 8a878999c0024d2abe517484180290fc P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:57.435576 20553 raft_consensus.cc:493] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:20:57.436126 20553 raft_consensus.cc:3060] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:20:57.436834 20688 leader_election.cc:290] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:57.428124 20584 raft_consensus.cc:3060] T 8638ee363273426db1393dd81e5c7b72 P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:20:56.690876 20622 raft_consensus.cc:515] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:57.440874 16221 raft_consensus.cc:1275] T 7c2095a5f4ad49408dbc7f69c606120d P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 5. (term mismatch)
I20260709 13:20:57.441956 16221 pending_rounds.cc:85] T 7c2095a5f4ad49408dbc7f69c606120d P 1ca2725d290e47d68b1a4b609c9816ac: Aborting all ops after (but not including) 4
I20260709 13:20:57.442214 16221 pending_rounds.cc:107] T 7c2095a5f4ad49408dbc7f69c606120d P 1ca2725d290e47d68b1a4b609c9816ac: Aborting uncommitted NO_OP operation due to leader change: 8.5
I20260709 13:20:57.442452 16221 raft_consensus.cc:2889] T 7c2095a5f4ad49408dbc7f69c606120d P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:20:57.223937 20607 raft_consensus.cc:515] T 9e65676b21b748728ad39668cfe722ec P cb82cc7bd6db46d980c75237a1402860 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:56.683476 20659 raft_consensus.cc:515] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } }
I20260709 13:20:57.454663 16151 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40844 (request call id 2380) took 1280 ms. Trace:
I20260709 13:20:57.455286 16151 rpcz_store.cc:276] 0709 13:20:56.174661 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:56.174824 (+   163us) service_pool.cc:224] Handling call
0709 13:20:56.175194 (+   370us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:57.132569 (+957375us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:57.454322 (+321753us) spinlock_profiling.cc:243] Waited 580 ms on lock 0x7b54000744d4. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:20:57.454587 (+   265us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1334950784}
I20260709 13:20:57.003496 20684 consensus_queue.cc:1059] T 2794af5b99984efebdee564035841cb0 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.357s
W20260709 13:20:57.158545 19861 consensus_meta.cc:287] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac: Time spent flushing consensus metadata: real 1.761s	user 0.007s	sys 0.007s
I20260709 13:20:57.173075 19982 raft_consensus.cc:515] T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:57.205626 20640 raft_consensus.cc:515] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:57.387145 16266 leader_election.cc:304] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860
W20260709 13:20:57.292116 16790 leader_election.cc:336] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 12 election: RPC error from VoteRequest() call to peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943): Timed out: RequestConsensusVote RPC to 127.14.177.130:35943 timed out after 2.155s (SENT)
I20260709 13:20:57.378798 20672 consensus_queue.cc:237] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:20:57.300647 16144 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.868s	user 0.012s	sys 0.029s
I20260709 13:20:57.388147 16218 raft_consensus.cc:2410] T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 for term 12 because replica has last-logged OpId of term: 11 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
W20260709 13:20:57.396106 16217 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.906s	user 0.000s	sys 0.018s
I20260709 13:20:56.955713 20676 consensus_queue.cc:237] T d2ab0ac8347641e6955e9bce7d90c1a2 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:57.157917 20674 raft_consensus.cc:515] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } }
I20260709 13:20:57.284015 20573 raft_consensus.cc:515] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:57.417042 20605 raft_consensus.cc:3055] T 2794af5b99984efebdee564035841cb0 P cb82cc7bd6db46d980c75237a1402860 [term 8 LEADER]: Stepping down as leader of term 8
W20260709 13:20:57.422487 16226 consensus_meta.cc:287] T ee9ed67006144797b1de6ea8c2013a68 P 1ca2725d290e47d68b1a4b609c9816ac: Time spent flushing consensus metadata: real 0.836s	user 0.005s	sys 0.001s
I20260709 13:20:57.498847 16226 raft_consensus.cc:2468] T ee9ed67006144797b1de6ea8c2013a68 P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 9.
W20260709 13:20:57.495373 16144 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.063s	user 0.018s	sys 0.042s
I20260709 13:20:57.499454 16226 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33050 (request call id 755) took 1281 ms. Trace:
W20260709 13:20:57.484491 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 192312us
I20260709 13:20:57.210793 20475 consensus_queue.cc:237] T ecc1e363230043fe9fef390f0b979f20 P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.7, Last appended by leader: 7, Current term: 11, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:20:57.408558 20578 consensus_meta.cc:287] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 1ca2725d290e47d68b1a4b609c9816ac: Time spent flushing consensus metadata: real 1.344s	user 0.006s	sys 0.009s
I20260709 13:20:57.446811 20685 leader_election.cc:290] T d2ab0ac8347641e6955e9bce7d90c1a2 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
W20260709 13:20:57.428964 20707 log.cc:927] Time spent T 5a959dbecc2648eb85d1cc550f5a3245 P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.224s	user 0.001s	sys 0.003s
I20260709 13:20:57.436837 20470 leader_election.cc:290] T 8a878999c0024d2abe517484180290fc P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:57.407402 20683 raft_consensus.cc:515] T c0eab25ce4c64e018a041beeda2867db P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:57.447808 20518 raft_consensus.cc:2749] T 8a878999c0024d2abe517484180290fc P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:20:57.448159 20611 leader_election.cc:290] T 829b4ad8c5a54eababeea1181b960d4b P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
W20260709 13:20:57.449321 20694 log.cc:927] Time spent T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 0.120s	user 0.000s	sys 0.000s
W20260709 13:20:57.450209 20651 log.cc:927] Time spent T 8638ee363273426db1393dd81e5c7b72 P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.575s	user 0.001s	sys 0.000s
I20260709 13:20:57.462553 16149 raft_consensus.cc:1240] T 68a6838d82ef4cac83507f906139d90f P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 11. Current term is 12. Ops: []
I20260709 13:20:57.427752 20565 leader_election.cc:290] T ee9ed67006144797b1de6ea8c2013a68 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 10 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:57.531226 16298 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40474 (request call id 743) took 2028 ms. Trace:
I20260709 13:20:57.532337 20611 consensus_queue.cc:1048] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:20:57.531656 16298 rpcz_store.cc:276] 0709 13:20:55.502571 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:55.505127 (+  2556us) service_pool.cc:224] Handling call
0709 13:20:55.505433 (+   306us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:20:55.523135 (+ 17702us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:55.523192 (+    57us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:20:55.526281 (+  3089us) log.cc:844] Serialized 26 byte log entry
0709 13:20:55.715988 (+189707us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:20:55.716056 (+    68us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:55.716163 (+   107us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:20:57.449321 (+1733158us) spinlock_profiling.cc:243] Waited 4.54 ms on lock 0x7b4400022b4c. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69ad8fadb1 00007f69ad998c85 00007f69ad9996a2 00007f69b070b1db 00007f69b0706aaf 00007f69b06fa86e 00007f69b06f9bb6 00007f69b06f97d8 00007f69b0703221 00007f69b07031d9 00007f69b0703169
0709 13:20:57.530934 (+ 81613us) raft_consensus.cc:1685] finished
0709 13:20:57.530959 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:57.531142 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":35338,"raft.queue_time_us":117493,"raft.run_cpu_time_us":144,"raft.run_wall_time_us":23378,"spinlock_wait_cycles":10447744,"thread_start_us":189266,"threads_started":1,"wal-append.queue_time_us":1125344}
W20260709 13:20:57.497925 20702 log.cc:927] Time spent T 8cc959fb7f704fba9009c0595467d074 P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 0.075s	user 0.000s	sys 0.000s
I20260709 13:20:57.535104 20518 raft_consensus.cc:493] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:57.535743 20518 raft_consensus.cc:515] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:57.538805 20518 leader_election.cc:290] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:57.499766 16226 rpcz_store.cc:276] 0709 13:20:56.312483 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:56.312752 (+   269us) service_pool.cc:224] Handling call
0709 13:20:57.499399 (+1186647us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:20:57.508993 20693 log.cc:927] Time spent T 84f2da98864b4425bf7c0055951fdd4b P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 0.111s	user 0.000s	sys 0.000s
W20260709 13:20:57.509320 20630 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 8 current term: 8 is higher.
I20260709 13:20:57.511332 16790 leader_election.cc:304] T c9944d5e4b8c4109af9955c4f08c9095 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
W20260709 13:20:57.513675 16266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 1.52947e+06us
I20260709 13:20:57.498507 20605 raft_consensus.cc:740] T 2794af5b99984efebdee564035841cb0 P cb82cc7bd6db46d980c75237a1402860 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:20:57.382414 20669 consensus_queue.cc:260] T c2f5f89186204242bf377f7c42b2e2da P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:57.418674 20691 raft_consensus.cc:515] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:57.529822 20671 raft_consensus.cc:493] T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:20:57.491919 20705 log.cc:927] Time spent T b6f07842d3aa47858df5665874c86f3a P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 0.069s	user 0.000s	sys 0.000s
I20260709 13:20:57.543788 20659 leader_election.cc:290] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:57.545818 20683 leader_election.cc:290] T c0eab25ce4c64e018a041beeda2867db P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:57.501503 20684 raft_consensus.cc:493] T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:57.476603 19861 raft_consensus.cc:515] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:57.568862 20696 raft_consensus.cc:493] T 3884f864a96d428785895e5463e5728f P cb82cc7bd6db46d980c75237a1402860 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:57.505681 16223 raft_consensus.cc:2468] T d2ab0ac8347641e6955e9bce7d90c1a2 P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 12.
W20260709 13:20:57.584190 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 72961.8us
I20260709 13:20:57.583729 20605 consensus_queue.cc:260] T 2794af5b99984efebdee564035841cb0 P cb82cc7bd6db46d980c75237a1402860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:57.586920 20671 raft_consensus.cc:515] T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:57.591821 16790 leader_election.cc:304] T ada5a004ba7c46819051e3ccec71ffdb P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:20:57.567842 16266 leader_election.cc:304] T 3884f864a96d428785895e5463e5728f P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
I20260709 13:20:57.559756 20598 leader_election.cc:290] T 5a96373bb0e14dce8a266b2fc6c58507 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:57.562505 20583 leader_election.cc:290] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 10 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:57.579670 20630 raft_consensus.cc:2749] T c9944d5e4b8c4109af9955c4f08c9095 P cb82cc7bd6db46d980c75237a1402860 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:20:57.489894 20661 consensus_queue.cc:260] T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:20:57.599782 16788 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:57.569733 20711 raft_consensus.cc:2749] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:20:57.593441 20696 raft_consensus.cc:515] T 3884f864a96d428785895e5463e5728f P cb82cc7bd6db46d980c75237a1402860 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:57.602916 20708 raft_consensus.cc:2749] T 3884f864a96d428785895e5463e5728f P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:20:57.579543 19982 leader_election.cc:290] T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
W20260709 13:20:57.598372 20706 log.cc:927] Time spent T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860: Append to log took a long time: real 0.083s	user 0.001s	sys 0.000s
W20260709 13:20:57.610283 16788 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:57.602756 20711 consensus_queue.cc:1048] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:20:57.613023 16790 leader_election.cc:304] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:20:57.605449 20605 raft_consensus.cc:3060] T 2794af5b99984efebdee564035841cb0 P cb82cc7bd6db46d980c75237a1402860 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:20:57.608959 20669 raft_consensus.cc:3060] T c2f5f89186204242bf377f7c42b2e2da P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:20:57.628095 20666 raft_consensus.cc:2749] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:20:57.628371 20630 consensus_queue.cc:1048] T ee9ed67006144797b1de6ea8c2013a68 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:20:57.630483 16790 leader_election.cc:304] T 8c11074579b046d99e585c2818e18a2c P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:20:57.633410 16790 leader_election.cc:304] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860; no voters: 90f3da57796b488c9087a2f523adf045
I20260709 13:20:57.628950 20646 raft_consensus.cc:3060] T ada5a004ba7c46819051e3ccec71ffdb P cb82cc7bd6db46d980c75237a1402860 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:20:57.597599 20684 raft_consensus.cc:515] T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:57.628990 19861 leader_election.cc:290] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 13 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:57.647320 20674 leader_election.cc:290] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:57.648239 20640 leader_election.cc:290] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 11 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:20:57.648178 20553 raft_consensus.cc:515] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:57.688164 16144 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40844 (request call id 2387) took 1508 ms. Trace:
I20260709 13:20:57.689355 20553 leader_election.cc:290] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 11 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:20:57.650178 20684 leader_election.cc:290] T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:20:57.641600 20689 consensus_queue.cc:1059] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: INVALID_TERM, Last received: 9.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.014s
I20260709 13:20:57.690426 16820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ab0ac8347641e6955e9bce7d90c1a2" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:57.690537 20689 raft_consensus.cc:3055] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:20:57.691126 20689 raft_consensus.cc:740] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:20:57.692528 16825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2372981006d4b218215c28964dea623" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 13 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:20:57.692335 20689 consensus_queue.cc:260] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:20:57.693864 16115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 250040us
I20260709 13:20:57.694156 20689 raft_consensus.cc:3060] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:20:57.650245 16265 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.516s	user 0.009s	sys 0.042s
W20260709 13:20:57.696592 16265 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.562s	user 0.010s	sys 0.043s
W20260709 13:20:57.697137 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 1.03202e+06us
I20260709 13:20:57.654549 20671 leader_election.cc:290] T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:57.655366 20607 leader_election.cc:290] T 9e65676b21b748728ad39668cfe722ec P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 11 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:57.655975 16223 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33050 (request call id 764) took 1213 ms. Trace:
I20260709 13:20:57.637358 20691 leader_election.cc:290] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:57.644251 20268 raft_consensus.cc:2804] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:20:57.662103 16217 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57604 (request call id 2613) took 1745 ms. Trace:
I20260709 13:20:57.708737 16217 rpcz_store.cc:276] 0709 13:20:55.916780 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:55.917514 (+   734us) service_pool.cc:224] Handling call
0709 13:20:55.948447 (+ 30933us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:20:56.226386 (+277939us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:20:56.391814 (+165428us) log.cc:844] Serialized 14 byte log entry
0709 13:20:57.489082 (+1097268us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:20:57.491522 (+  2440us) log.cc:844] Serialized 52 byte log entry
0709 13:20:57.491735 (+   213us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:20:57.579219 (+ 87484us) log.cc:844] Serialized 14 byte log entry
0709 13:20:57.579459 (+   240us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:57.579544 (+    85us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:20:57.661847 (+ 82303us) raft_consensus.cc:1685] finished
0709 13:20:57.661869 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:57.662037 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":13112,"prepare.queue_time_us":600875,"prepare.run_cpu_time_us":97,"prepare.run_wall_time_us":26079,"raft.queue_time_us":171875,"raft.run_cpu_time_us":538,"raft.run_wall_time_us":26768,"thread_start_us":1468391,"threads_started":4,"wal-append.queue_time_us":786079}
I20260709 13:20:57.664019 20231 raft_consensus.cc:3055] T 434bff3082d141838068f323bdb0d1ed P 647a343a542d4a139fe106c19095129b [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:20:57.665349 20576 raft_consensus.cc:3055] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:20:57.666904 20579 raft_consensus.cc:493] T e5ad4eb281564936a59f41e387a47156 P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:20:57.671636 20644 raft_consensus.cc:515] T e5ad4eb281564936a59f41e387a47156 P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:57.681159 20584 raft_consensus.cc:515] T 8638ee363273426db1393dd81e5c7b72 P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:57.682504 16151 raft_consensus.cc:1240] T ee9ed67006144797b1de6ea8c2013a68 P 647a343a542d4a139fe106c19095129b [term 10 FOLLOWER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 9. Current term is 10. Ops: [9.9-9.9]
I20260709 13:20:57.683100 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af57996a8be84d398f0f023e8d08ef77" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:20:57.686025 16142 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40844 (request call id 2388) took 1505 ms. Trace:
I20260709 13:20:57.688611 16144 rpcz_store.cc:276] 0709 13:20:56.180213 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:56.236375 (+ 56162us) service_pool.cc:224] Handling call
0709 13:20:56.236693 (+   318us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:20:56.358111 (+121418us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:56.358164 (+    53us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:20:56.358817 (+   653us) log.cc:844] Serialized 26 byte log entry
0709 13:20:57.499552 (+1140735us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:20:57.543743 (+ 44191us) log.cc:844] Serialized 14 byte log entry
0709 13:20:57.687853 (+144110us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:57.687932 (+    79us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:20:57.687977 (+    45us) raft_consensus.cc:1685] finished
0709 13:20:57.687991 (+    14us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:57.688108 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":141738,"prepare.queue_time_us":41774,"prepare.run_cpu_time_us":42,"prepare.run_wall_time_us":43,"raft.queue_time_us":198517,"raft.run_cpu_time_us":169,"raft.run_wall_time_us":99112,"thread_start_us":1144112,"threads_started":2,"wal-append.queue_time_us":1119641}
I20260709 13:20:57.702648 16790 leader_election.cc:304] T 434bff3082d141838068f323bdb0d1ed P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:20:57.702092 16223 rpcz_store.cc:276] 0709 13:20:56.442735 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:56.568179 (+125444us) service_pool.cc:224] Handling call
0709 13:20:57.655803 (+1087624us) spinlock_profiling.cc:243] Waited 515 ms on lock 0x7b5400245754. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:20:57.655915 (+   112us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1185471232}
I20260709 13:20:57.635444 20630 raft_consensus.cc:2749] T 8c11074579b046d99e585c2818e18a2c P cb82cc7bd6db46d980c75237a1402860 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:20:57.693856 20674 raft_consensus.cc:493] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:57.718146 20576 raft_consensus.cc:740] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:20:57.719799 20579 raft_consensus.cc:515] T e5ad4eb281564936a59f41e387a47156 P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:20:57.722045 16190 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:20:57.755514 16190 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:20:57.756233 16190 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:57.708575 20700 consensus_queue.cc:1059] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.015s
I20260709 13:20:57.718861 20268 raft_consensus.cc:697] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860 [term 9 LEADER]: Becoming Leader. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:20:57.724992 16823 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49892 (request call id 634) took 2983 ms. Trace:
I20260709 13:20:57.762452 16823 rpcz_store.cc:276] 0709 13:20:54.741718 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:54.795316 (+ 53598us) service_pool.cc:224] Handling call
0709 13:20:55.090166 (+294850us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:55.090457 (+   291us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:20:55.277079 (+186622us) log.cc:844] Serialized 14 byte log entry
0709 13:20:55.888216 (+611137us) log.cc:844] Serialized 14 byte log entry
0709 13:20:55.945176 (+ 56960us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:20:55.945233 (+    57us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:20:55.945278 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:55.945343 (+    65us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:57.723320 (+1777977us) spinlock_profiling.cc:243] Waited 3.84 ms on lock 0x7b18003627a0. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b693ade1 00007f69b69475b1 00007f69ada402a6 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5e8ad 00007f69b5b23419 00007f69af09abd4
0709 13:20:57.724734 (+  1414us) spinlock_profiling.cc:243] Waited 295 ms on lock 0x7b5400769150. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5e8ad 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:20:57.724908 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":76565,"prepare.queue_time_us":308362,"prepare.run_cpu_time_us":141,"prepare.run_wall_time_us":11556,"raft.queue_time_us":125445,"raft.run_cpu_time_us":164,"raft.run_wall_time_us":6629,"spinlock_wait_cycles":711874432,"thread_start_us":759592,"threads_started":3,"wal-append.queue_time_us":1155025}
I20260709 13:20:57.699299 19861 consensus_queue.cc:1048] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.019s
W20260709 13:20:57.727134 20714 log.cc:927] Time spent T 18ad7e3af5504571a69d10c00750a01c P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.115s	user 0.000s	sys 0.000s
I20260709 13:20:57.729209 20594 leader_election.cc:290] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:57.715883 20231 raft_consensus.cc:740] T 434bff3082d141838068f323bdb0d1ed P 647a343a542d4a139fe106c19095129b [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:20:57.730859 20671 raft_consensus.cc:493] T b6f07842d3aa47858df5665874c86f3a P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
W20260709 13:20:57.732018 16822 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49906 (request call id 902) took 2371 ms (client timeout 2005 ms). Trace:
I20260709 13:20:57.736138 16265 leader_election.cc:304] T 1df93779209748b9860f6d893dd3de53 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:20:57.736158 16221 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57604 (request call id 2614) took 1819 ms. Trace:
W20260709 13:20:57.779601 16822 rpcz_store.cc:269] 0709 13:20:55.360930 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:55.361063 (+   133us) service_pool.cc:224] Handling call
0709 13:20:57.731760 (+2370697us) spinlock_profiling.cc:243] Waited 902 ms on lock 0x7b54006f2054. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:20:57.731914 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2074392448}
I20260709 13:20:57.780161 16221 rpcz_store.cc:276] 0709 13:20:55.917123 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:55.917233 (+   110us) service_pool.cc:224] Handling call
0709 13:20:55.947994 (+ 30761us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:57.442852 (+1494858us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:57.735930 (+293078us) spinlock_profiling.cc:243] Waited 1.49 s on lock 0x7b54002c8154. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:20:57.736099 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3432471296}
W20260709 13:20:57.781157 16189 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:57.781766 16817 raft_consensus.cc:1240] T 8c11074579b046d99e585c2818e18a2c P cb82cc7bd6db46d980c75237a1402860 [term 12 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 11. Current term is 12. Ops: [11.11-11.11]
W20260709 13:20:57.745501 16266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 143270us
I20260709 13:20:57.739219 16142 rpcz_store.cc:276] 0709 13:20:56.180535 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:56.256345 (+ 75810us) service_pool.cc:224] Handling call
0709 13:20:56.291016 (+ 34671us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:56.291295 (+   279us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:20:57.143625 (+852330us) log.cc:844] Serialized 14 byte log entry
0709 13:20:57.685485 (+541860us) log.cc:844] Serialized 14 byte log entry
0709 13:20:57.685691 (+   206us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:20:57.685735 (+    44us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:20:57.685779 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:57.685849 (+    70us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:57.685958 (+   109us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":346156,"prepare.queue_time_us":394343,"prepare.run_cpu_time_us":89,"prepare.run_wall_time_us":91,"raft.queue_time_us":346479,"raft.run_cpu_time_us":186,"raft.run_wall_time_us":186,"thread_start_us":1038184,"threads_started":2,"wal-append.queue_time_us":376315}
I20260709 13:20:57.755223 20618 raft_consensus.cc:493] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:57.744752 16819 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49896 (request call id 1166) took 2272 ms. Trace:
I20260709 13:20:57.754897 20691 raft_consensus.cc:493] T 84f2da98864b4425bf7c0055951fdd4b P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:20:57.762419 20578 raft_consensus.cc:493] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:20:57.783699 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 47653.4us
I20260709 13:20:57.772641 20673 raft_consensus.cc:493] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:57.772267 20685 raft_consensus.cc:493] T 5a959dbecc2648eb85d1cc550f5a3245 P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:57.767993 20684 raft_consensus.cc:493] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:57.788004 16266 leader_election.cc:304] T a378d7f796f94615b3b347119a9880fd P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860
I20260709 13:20:57.760727 20674 raft_consensus.cc:515] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:57.784337 16819 rpcz_store.cc:276] 0709 13:20:55.472564 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:55.472672 (+   108us) service_pool.cc:224] Handling call
0709 13:20:55.476427 (+  3755us) raft_consensus.cc:1471] Updating replica for 4 ops
0709 13:20:55.640002 (+163575us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:55.640036 (+    34us) raft_consensus.cc:1539] Triggering prepare for 4 ops
0709 13:20:55.736369 (+ 96333us) spinlock_profiling.cc:243] Waited 17 ms on lock 0x7b2c00047148. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69af458a11 00007f69af468372 00007f69b0b9aa01 00007f69b0b5f43a 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2
0709 13:20:55.743282 (+  6913us) log.cc:844] Serialized 104 byte log entry
0709 13:20:55.919539 (+176257us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:20:55.959935 (+ 40396us) log.cc:844] Serialized 14 byte log entry
0709 13:20:55.972865 (+ 12930us) log.cc:844] Serialized 14 byte log entry
0709 13:20:57.604131 (+1631266us) spinlock_profiling.cc:243] Waited 4.17 ms on lock 0x7b18002d7540. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b693ade1 00007f69b69475b1 00007f69ad9ecc0a 00007f69b712c219 00007f69ada74a40 00007f69ada6f4e4 00007f69ada6f455 00007f69b45573c7 00007f69b0b699a7 00007f69b0b790bf 00007f69b0b79038
0709 13:20:57.604340 (+   209us) log.cc:844] Serialized 14 byte log entry
0709 13:20:57.676720 (+ 72380us) spinlock_profiling.cc:243] Waited 4.64 ms on lock 0x7b18002d7540. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b693ade1 00007f69b69475b1 00007f69ada723f3 00007f69ada747c1 00007f69ada74779 00007f69ada74709 00007f69ada746d1 00007f69ada739cd 000000000038f354 000000000038f199 00007f69ada5583a
0709 13:20:57.744287 (+ 67567us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:57.744409 (+   122us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:20:57.744460 (+    51us) raft_consensus.cc:1685] finished
0709 13:20:57.744479 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:57.744681 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":287190,"prepare.queue_time_us":1762993,"prepare.run_cpu_time_us":101,"prepare.run_wall_time_us":105,"raft.queue_time_us":81476,"raft.run_cpu_time_us":615,"raft.run_wall_time_us":66992,"spinlock_wait_cycles":59443072,"thread_start_us":314829,"threads_started":2,"wal-append.queue_time_us":533080,"wal-append.run_cpu_time_us":4373,"wal-append.run_wall_time_us":1292461}
I20260709 13:20:57.786640 20594 raft_consensus.cc:2804] T 1df93779209748b9860f6d893dd3de53 P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:20:57.764039 20609 consensus_queue.cc:1059] T 23f82224f4564fea93572e8206bad1fe P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.025s
I20260709 13:20:57.800035 16192 leader_election.cc:400] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 12 election: Vote denied by peer cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923) with higher term. Message: Invalid argument: T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac for earlier term 12. Current term is 13.
I20260709 13:20:57.800452 16192 leader_election.cc:403] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 12 election: Cancelling election due to peer responding with higher term
I20260709 13:20:57.802350 16819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7ca61592e848ea9b0fb81a2ce33cdc" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:57.802987 16819 raft_consensus.cc:2468] T 3a7ca61592e848ea9b0fb81a2ce33cdc P cb82cc7bd6db46d980c75237a1402860 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 14.
I20260709 13:20:57.805215 16190 leader_election.cc:304] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860
I20260709 13:20:57.806649 16825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9944d5e4b8c4109af9955c4f08c9095" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:20:57.807195 16825 raft_consensus.cc:2376] T c9944d5e4b8c4109af9955c4f08c9095 P cb82cc7bd6db46d980c75237a1402860 [term 12 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 12. Re-sending same reply.
I20260709 13:20:57.808710 16825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c11074579b046d99e585c2818e18a2c" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 12 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:57.808969 16821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7ca61592e848ea9b0fb81a2ce33cdc" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:20:57.809705 16821 raft_consensus.cc:2410] T 3a7ca61592e848ea9b0fb81a2ce33cdc P cb82cc7bd6db46d980c75237a1402860 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 for term 14 because replica has last-logged OpId of term: 13 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 11.
I20260709 13:20:57.810463 16825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31969392ccdb43b491911a319e8bc7e1" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:57.810724 16818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:57.811816 16825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26354361e9b044cea1c2b86c4882fe92" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:57.788985 20576 consensus_queue.cc:260] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:57.815697 16824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2794af5b99984efebdee564035841cb0" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:57.815945 20576 raft_consensus.cc:3060] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:20:57.792030 20684 raft_consensus.cc:515] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:57.816944 16820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdbd78acb4cf46298e0de8ba0d130e19" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:57.817554 16820 raft_consensus.cc:2468] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 12.
I20260709 13:20:57.818562 16820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8638ee363273426db1393dd81e5c7b72" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:20:57.818742 20684 leader_election.cc:290] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:57.794042 20579 leader_election.cc:290] T e5ad4eb281564936a59f41e387a47156 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:57.819190 16823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:20:57.820016 16823 raft_consensus.cc:2393] T 19baa7bc367844c288bca8f81439c413 P cb82cc7bd6db46d980c75237a1402860 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 12: Already voted for candidate cb82cc7bd6db46d980c75237a1402860 in this term.
I20260709 13:20:57.820236 20646 raft_consensus.cc:2749] T ada5a004ba7c46819051e3ccec71ffdb P cb82cc7bd6db46d980c75237a1402860 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:20:57.820875 16823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a9de347c711477e87c582d003f36925" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:57.793342 20640 raft_consensus.cc:2749] T a378d7f796f94615b3b347119a9880fd P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:20:57.822064 16818 raft_consensus.cc:2468] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 11.
I20260709 13:20:57.816375 16824 raft_consensus.cc:2468] T 2794af5b99984efebdee564035841cb0 P cb82cc7bd6db46d980c75237a1402860 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 9.
W20260709 13:20:57.794802 20716 log.cc:927] Time spent T 3884f864a96d428785895e5463e5728f P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 0.086s	user 0.000s	sys 0.001s
I20260709 13:20:57.795750 20658 raft_consensus.cc:2749] T 434bff3082d141838068f323bdb0d1ed P cb82cc7bd6db46d980c75237a1402860 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:20:57.783751 20671 raft_consensus.cc:515] T b6f07842d3aa47858df5665874c86f3a P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:57.829986 16824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 14 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:57.832051 16825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e9b2bf7011f4ae688a79d7bb5a447b4" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:20:57.794834 20662 raft_consensus.cc:3055] T 26354361e9b044cea1c2b86c4882fe92 P cb82cc7bd6db46d980c75237a1402860 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:20:57.829277 16192 leader_election.cc:304] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:20:57.798321 20594 raft_consensus.cc:697] T 1df93779209748b9860f6d893dd3de53 P 90f3da57796b488c9087a2f523adf045 [term 12 LEADER]: Becoming Leader. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:20:57.838290 20594 consensus_queue.cc:237] T 1df93779209748b9860f6d893dd3de53 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:57.839520 20719 raft_consensus.cc:3055] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:20:57.839947 20719 raft_consensus.cc:740] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:20:57.792029 20685 raft_consensus.cc:515] T 5a959dbecc2648eb85d1cc550f5a3245 P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:57.840972 20601 raft_consensus.cc:3055] T e4dc6b0541ca431ebc757beac00521f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:20:57.841390 20601 raft_consensus.cc:740] T e4dc6b0541ca431ebc757beac00521f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:20:57.789512 20691 raft_consensus.cc:515] T 84f2da98864b4425bf7c0055951fdd4b P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:57.794230 20268 consensus_queue.cc:237] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:57.844847 20685 leader_election.cc:290] T 5a959dbecc2648eb85d1cc550f5a3245 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:57.845088 20601 consensus_queue.cc:260] T e4dc6b0541ca431ebc757beac00521f1 P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.5, Last appended by leader: 5, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:57.790058 20578 raft_consensus.cc:515] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:57.832666 16822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8638ee363273426db1393dd81e5c7b72" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
W20260709 13:20:57.846498 16218 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57600 (request call id 1480) took 1958 ms (client timeout 1731 ms). Trace:
I20260709 13:20:57.794185 20637 raft_consensus.cc:2749] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:20:57.832767 16824 raft_consensus.cc:2468] T 19baa7bc367844c288bca8f81439c413 P cb82cc7bd6db46d980c75237a1402860 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 12.
I20260709 13:20:57.821424 20672 raft_consensus.cc:3060] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:20:57.794068 20674 leader_election.cc:290] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:57.831624 16190 leader_election.cc:304] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:20:57.833508 20599 raft_consensus.cc:493] T c0eab25ce4c64e018a041beeda2867db P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:57.835572 20577 raft_consensus.cc:2804] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:20:57.833833 20662 raft_consensus.cc:740] T 26354361e9b044cea1c2b86c4882fe92 P cb82cc7bd6db46d980c75237a1402860 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:20:57.836719 20644 leader_election.cc:290] T e5ad4eb281564936a59f41e387a47156 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 13 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:20:57.833902 20669 raft_consensus.cc:2749] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:20:57.795208 20584 leader_election.cc:290] T 8638ee363273426db1393dd81e5c7b72 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 13 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:57.841605 16150 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40844 (request call id 2386) took 1661 ms. Trace:
W20260709 13:20:57.846884 16218 rpcz_store.cc:269] 0709 13:20:55.887970 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:55.908520 (+ 20550us) service_pool.cc:224] Handling call
0709 13:20:57.846310 (+1937790us) spinlock_profiling.cc:243] Waited 1.39 s on lock 0x7b54000154d4. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:20:57.846437 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3197921024}
I20260709 13:20:57.791348 20673 raft_consensus.cc:515] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:57.787582 20618 raft_consensus.cc:515] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:57.840126 20604 leader_election.cc:290] T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 13 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:57.854234 16298 raft_consensus.cc:1275] T cdbd78acb4cf46298e0de8ba0d130e19 P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:20:57.857302 16300 raft_consensus.cc:1240] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 11. Current term is 12. Ops: []
I20260709 13:20:57.855881 16292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ab0ac8347641e6955e9bce7d90c1a2" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:57.862848 16292 raft_consensus.cc:2393] T d2ab0ac8347641e6955e9bce7d90c1a2 P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 12: Already voted for candidate cb82cc7bd6db46d980c75237a1402860 in this term.
I20260709 13:20:57.857823 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a878999c0024d2abe517484180290fc" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:57.853351 16150 rpcz_store.cc:276] 0709 13:20:56.179901 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:56.238981 (+ 59080us) service_pool.cc:224] Handling call
0709 13:20:56.239423 (+   442us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:20:56.682515 (+443092us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:56.682601 (+    86us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:20:57.796856 (+1114255us) spinlock_profiling.cc:243] Waited 31.2 ms on lock 0x7b2c00000658. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69af458a11 00007f69af468372 00007f69b0b9aa01 00007f69b0b5f43a 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2
0709 13:20:57.797499 (+   643us) log.cc:844] Serialized 52 byte log entry
0709 13:20:57.827408 (+ 29909us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:20:57.827502 (+    94us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:57.827572 (+    70us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:20:57.841325 (+ 13753us) raft_consensus.cc:1685] finished
0709 13:20:57.841344 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:57.841548 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":9192,"raft.queue_time_us":212185,"raft.run_cpu_time_us":117,"raft.run_wall_time_us":119,"spinlock_wait_cycles":71871360,"thread_start_us":455777,"threads_started":2,"wal-append.queue_time_us":39097}
I20260709 13:20:57.863590 16297 raft_consensus.cc:2468] T 8a878999c0024d2abe517484180290fc P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 10.
I20260709 13:20:57.864884 16823 raft_consensus.cc:2468] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 13.
I20260709 13:20:57.865008 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5ad4eb281564936a59f41e387a47156" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:57.869086 16298 raft_consensus.cc:2393] T e5ad4eb281564936a59f41e387a47156 P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 13: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
I20260709 13:20:57.865895 20443 raft_consensus.cc:3055] T 5a96373bb0e14dce8a266b2fc6c58507 P 1ca2725d290e47d68b1a4b609c9816ac [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:20:57.870054 20443 raft_consensus.cc:740] T 5a96373bb0e14dce8a266b2fc6c58507 P 1ca2725d290e47d68b1a4b609c9816ac [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:20:57.854792 20577 raft_consensus.cc:493] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:57.870656 20577 raft_consensus.cc:3060] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:20:57.870970 20443 consensus_queue.cc:260] T 5a96373bb0e14dce8a266b2fc6c58507 P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.5, Last appended by leader: 5, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:57.872203 20443 raft_consensus.cc:3060] T 5a96373bb0e14dce8a266b2fc6c58507 P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:20:57.849756 20721 raft_consensus.cc:493] T 5a96373bb0e14dce8a266b2fc6c58507 P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:20:57.869904 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806d7690c2b644959152074bb2cab0a7" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:57.859622 20618 leader_election.cc:290] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:57.875450 20721 raft_consensus.cc:515] T 5a96373bb0e14dce8a266b2fc6c58507 P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:57.876212 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7ca61592e848ea9b0fb81a2ce33cdc" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
W20260709 13:20:57.880576 16263 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:57.851325 20671 leader_election.cc:290] T b6f07842d3aa47858df5665874c86f3a P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:57.852841 20685 raft_consensus.cc:2804] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:20:57.851886 20640 raft_consensus.cc:493] T fffab26427e24d398e2695aad40ca134 P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:20:57.884061 16297 raft_consensus.cc:2468] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 90f3da57796b488c9087a2f523adf045 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 14.
I20260709 13:20:57.856679 20662 consensus_queue.cc:260] T 26354361e9b044cea1c2b86c4882fe92 P cb82cc7bd6db46d980c75237a1402860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:57.852131 20601 raft_consensus.cc:3060] T e4dc6b0541ca431ebc757beac00521f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:20:57.885496 16819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74ca8140f35c4ae29e8a664f6bdfc829" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
W20260709 13:20:57.886231 20682 log.cc:927] Time spent T 746aabf856ab401c9ef5e859371e7b67 P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 0.286s	user 0.000s	sys 0.003s
I20260709 13:20:57.860796 20573 leader_election.cc:290] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 13 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:20:57.854965 20599 raft_consensus.cc:515] T c0eab25ce4c64e018a041beeda2867db P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:57.860675 20231 consensus_queue.cc:260] T 434bff3082d141838068f323bdb0d1ed P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:57.891177 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c61a39fd4f84424b129c1a6f9ba404d" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:57.916486 16299 raft_consensus.cc:2468] T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 12.
I20260709 13:20:57.916549 16823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62fe3346cd3c42c4a55e14af9073bca4" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:57.892400 16221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 14 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:57.885500 20721 leader_election.cc:290] T 5a96373bb0e14dce8a266b2fc6c58507 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:20:57.892902 20642 raft_consensus.cc:493] T 434bff3082d141838068f323bdb0d1ed P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:20:57.892903 20622 leader_election.cc:290] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:57.894750 16223 raft_consensus.cc:1275] T 5a959dbecc2648eb85d1cc550f5a3245 P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:20:57.895252 20644 raft_consensus.cc:493] T d2ab0ac8347641e6955e9bce7d90c1a2 P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:57.930395 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8638ee363273426db1393dd81e5c7b72" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:20:57.895330 20584 raft_consensus.cc:493] T 8d4ab870a05347868956aa8b841b2981 P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:20:57.896052 20657 raft_consensus.cc:493] T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:20:57.896456 20578 leader_election.cc:290] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:57.897324 16291 raft_consensus.cc:1240] T e5ad4eb281564936a59f41e387a47156 P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 12. Current term is 13. Ops: []
I20260709 13:20:57.900125 16294 raft_consensus.cc:1275] T b87147731a68400e8cce9b879daa7c5d P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 13:20:57.940011 16294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f82224f4564fea93572e8206bad1fe" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:57.941016 16294 raft_consensus.cc:2468] T 23f82224f4564fea93572e8206bad1fe P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 8.
I20260709 13:20:57.942233 16294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0eab25ce4c64e018a041beeda2867db" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:57.882401 16292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a959dbecc2648eb85d1cc550f5a3245" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:57.903906 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e65676b21b748728ad39668cfe722ec" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:20:57.898867 20592 raft_consensus.cc:3055] T 8c11074579b046d99e585c2818e18a2c P 1ca2725d290e47d68b1a4b609c9816ac [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:20:57.909626 16819 raft_consensus.cc:2410] T 74ca8140f35c4ae29e8a664f6bdfc829 P cb82cc7bd6db46d980c75237a1402860 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac for term 13 because replica has last-logged OpId of term: 11 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 9.
I20260709 13:20:57.910982 16825 raft_consensus.cc:1240] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860 [term 11 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 10. Current term is 11. Ops: []
I20260709 13:20:57.907651 20518 raft_consensus.cc:493] T 57a744f5aaf944678a63469cbf514015 P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:20:57.915367 20662 raft_consensus.cc:3060] T 26354361e9b044cea1c2b86c4882fe92 P cb82cc7bd6db46d980c75237a1402860 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:20:57.915696 20599 leader_election.cc:290] T c0eab25ce4c64e018a041beeda2867db P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:20:57.917598 16221 raft_consensus.cc:2468] T 19baa7bc367844c288bca8f81439c413 P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 13.
I20260709 13:20:57.860479 20673 leader_election.cc:290] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:57.868273 20691 leader_election.cc:290] T 84f2da98864b4425bf7c0055951fdd4b P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:57.921145 16219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c74490c40243a88e62d1760b5cd814" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:20:57.923677 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4dc6b0541ca431ebc757beac00521f1" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:57.923048 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fffab26427e24d398e2695aad40ca134" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:57.960641 16219 raft_consensus.cc:3060] T 08c74490c40243a88e62d1760b5cd814 P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:20:57.922147 20231 raft_consensus.cc:3060] T 434bff3082d141838068f323bdb0d1ed P 647a343a542d4a139fe106c19095129b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:20:57.944784 20592 raft_consensus.cc:740] T 8c11074579b046d99e585c2818e18a2c P 1ca2725d290e47d68b1a4b609c9816ac [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:20:57.918104 20685 raft_consensus.cc:697] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac [term 11 LEADER]: Becoming Leader. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:20:57.926752 16816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9944d5e4b8c4109af9955c4f08c9095" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:57.928210 16148 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40844 (request call id 2379) took 1754 ms. Trace:
I20260709 13:20:57.967587 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4f65cdf00b748bb963ee6e81ef43b79" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:57.968379 16297 raft_consensus.cc:2468] T c4f65cdf00b748bb963ee6e81ef43b79 P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 13.
I20260709 13:20:57.934930 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6f07842d3aa47858df5665874c86f3a" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:57.945487 16143 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40844 (request call id 2389) took 1750 ms. Trace:
I20260709 13:20:57.946158 16294 raft_consensus.cc:2410] T c0eab25ce4c64e018a041beeda2867db P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b for term 12 because replica has last-logged OpId of term: 10 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
I20260709 13:20:57.920236 20671 raft_consensus.cc:493] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:57.927997 20642 raft_consensus.cc:515] T 434bff3082d141838068f323bdb0d1ed P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:57.970755 16143 rpcz_store.cc:276] 0709 13:20:56.195227 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:56.270410 (+ 75183us) service_pool.cc:224] Handling call
0709 13:20:56.321717 (+ 51307us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:56.322006 (+   289us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:20:56.658716 (+336710us) log.cc:844] Serialized 14 byte log entry
0709 13:20:57.928452 (+1269736us) spinlock_profiling.cc:243] Waited 5.4 ms on lock 0x7b18003208c0. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b693ade1 00007f69b69475b1 00007f69b07074bc 00007f69ada5548b 00007f69b68afab9 00007f69ada7169b 00007f69ada6f93f 00007f69ada71c02 00007f69b06f920b 00007f69b06fed23 00007f69b06ff23a
0709 13:20:57.929307 (+   855us) log.cc:844] Serialized 14 byte log entry
0709 13:20:57.933457 (+  4150us) log.cc:844] Serialized 14 byte log entry
0709 13:20:57.933661 (+   204us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:20:57.933712 (+    51us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:20:57.933764 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:57.933829 (+    65us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:57.945424 (+ 11595us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1982,"prepare.queue_time_us":483244,"prepare.run_cpu_time_us":94,"prepare.run_wall_time_us":6324,"raft.queue_time_us":505,"raft.run_cpu_time_us":102,"raft.run_wall_time_us":15810,"spinlock_wait_cycles":57079424,"thread_start_us":517397,"threads_started":2,"wal-append.queue_time_us":635424}
I20260709 13:20:57.951563 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a96373bb0e14dce8a266b2fc6c58507" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:57.953918 16220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8158bbd17b994d2eb96f560bd0f1b88d" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:57.951830 16291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02838c86531d4f98bf58d1c22b8baf77" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
W20260709 13:20:57.956800 16117 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:57.957379 16223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a96373bb0e14dce8a266b2fc6c58507" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:57.980458 16298 raft_consensus.cc:2468] T 5a96373bb0e14dce8a266b2fc6c58507 P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 10.
I20260709 13:20:57.981117 16294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9944d5e4b8c4109af9955c4f08c9095" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:57.960651 16266 leader_election.cc:304] T c9944d5e4b8c4109af9955c4f08c9095 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac
I20260709 13:20:57.961505 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5ad4eb281564936a59f41e387a47156" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:20:57.956331 20602 raft_consensus.cc:493] T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:57.959910 20236 raft_consensus.cc:493] T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:57.984550 20699 raft_consensus.cc:493] T af57996a8be84d398f0f023e8d08ef77 P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:20:57.984897 20236 raft_consensus.cc:515] T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:57.963932 20679 raft_consensus.cc:493] T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:20:57.963932 16221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0eab25ce4c64e018a041beeda2867db" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:57.944878 20588 raft_consensus.cc:493] T b87147731a68400e8cce9b879daa7c5d P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:57.992463 16266 leader_election.cc:304] T 8c11074579b046d99e585c2818e18a2c P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860
I20260709 13:20:57.912463 20640 raft_consensus.cc:515] T fffab26427e24d398e2695aad40ca134 P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:57.933658 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87ac0d1f7e8a419f9bcbc04aee0e067e" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:20:57.985193 20699 raft_consensus.cc:515] T af57996a8be84d398f0f023e8d08ef77 P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:57.954596 20518 raft_consensus.cc:515] T 57a744f5aaf944678a63469cbf514015 P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:57.969403 20583 consensus_queue.cc:1048] T 5a959dbecc2648eb85d1cc550f5a3245 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.024s
I20260709 13:20:57.966917 20592 consensus_queue.cc:260] T 8c11074579b046d99e585c2818e18a2c P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:20:57.974871 16788 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:57.950553 16292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87ac0d1f7e8a419f9bcbc04aee0e067e" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:57.973821 20685 consensus_queue.cc:237] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:20:57.975200 16189 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:58.015367 20578 consensus_queue.cc:1059] T 8c11074579b046d99e585c2818e18a2c P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:20:57.981707 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f5f89186204242bf377f7c42b2e2da" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:57.967926 16148 rpcz_store.cc:276] 0709 13:20:56.174308 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:56.206232 (+ 31924us) service_pool.cc:224] Handling call
0709 13:20:56.231859 (+ 25627us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:20:56.258048 (+ 26189us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:56.258093 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:20:56.258706 (+   613us) log.cc:844] Serialized 26 byte log entry
0709 13:20:56.573804 (+315098us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:20:56.847600 (+273796us) log.cc:844] Serialized 14 byte log entry
0709 13:20:57.177053 (+329453us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:57.177142 (+    89us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:20:57.885498 (+708356us) spinlock_profiling.cc:243] Waited 5.69 ms on lock 0x7b4400022b4c. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69ad8fadb1 00007f69ad998c85 00007f69ad9996a2 00007f69b070b1db 00007f69b0706aaf 00007f69b06fa86e 00007f69b06f9bb6 00007f69b06f97d8 00007f69b0703221 00007f69b07031d9 00007f69b0703169
0709 13:20:57.927911 (+ 42413us) raft_consensus.cc:1685] finished
0709 13:20:57.928011 (+   100us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:57.928145 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":342364,"prepare.queue_time_us":329794,"prepare.run_cpu_time_us":48,"prepare.run_wall_time_us":51,"raft.queue_time_us":1022066,"raft.run_cpu_time_us":300,"raft.run_wall_time_us":30369,"spinlock_wait_cycles":13082368,"thread_start_us":551311,"threads_started":3,"wal-append.queue_time_us":1330224}
I20260709 13:20:57.982216 16225 raft_consensus.cc:3060] T e5ad4eb281564936a59f41e387a47156 P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:20:57.981781 16294 raft_consensus.cc:2468] T c9944d5e4b8c4109af9955c4f08c9095 P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 12.
I20260709 13:20:57.984239 20427 raft_consensus.cc:493] T 68a6838d82ef4cac83507f906139d90f P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:20:57.985956 16117 connection.cc:256] RPC call timeout handler was delayed by -1.4850940930000434s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1841s for a response
I20260709 13:20:57.937258 20644 raft_consensus.cc:515] T d2ab0ac8347641e6955e9bce7d90c1a2 P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:57.990607 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806d7690c2b644959152074bb2cab0a7" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:57.947887 20657 raft_consensus.cc:515] T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:58.026392 20721 raft_consensus.cc:2804] T c9944d5e4b8c4109af9955c4f08c9095 P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:20:58.028409 20721 raft_consensus.cc:697] T c9944d5e4b8c4109af9955c4f08c9095 P 90f3da57796b488c9087a2f523adf045 [term 12 LEADER]: Becoming Leader. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:20:58.029297 20657 leader_election.cc:290] T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:20:58.029516 20661 raft_consensus.cc:3060] T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:20:57.994553 16265 leader_election.cc:304] T 8158bbd17b994d2eb96f560bd0f1b88d P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:20:58.032001 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74ca8140f35c4ae29e8a664f6bdfc829" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:57.990839 20388 raft_consensus.cc:493] T 1df93779209748b9860f6d893dd3de53 P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:58.029335 20721 consensus_queue.cc:237] T c9944d5e4b8c4109af9955c4f08c9095 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:58.036089 20431 raft_consensus.cc:493] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:58.036616 20431 raft_consensus.cc:515] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:20:58.038101 16293 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48684 (request call id 2540) took 2740 ms (client timeout 1748 ms). Trace:
I20260709 13:20:58.038939 20663 raft_consensus.cc:2749] T 8158bbd17b994d2eb96f560bd0f1b88d P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:20:57.997109 16217 raft_consensus.cc:3060] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:20:57.931048 16226 raft_consensus.cc:3060] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:20:57.967706 16816 raft_consensus.cc:2468] T c9944d5e4b8c4109af9955c4f08c9095 P cb82cc7bd6db46d980c75237a1402860 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 12.
I20260709 13:20:57.998948 16266 leader_election.cc:304] T cdbd78acb4cf46298e0de8ba0d130e19 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac
W20260709 13:20:58.042496 16266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 43729.5us
I20260709 13:20:57.991087 20602 raft_consensus.cc:515] T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:57.998095 20588 raft_consensus.cc:515] T b87147731a68400e8cce9b879daa7c5d P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:57.984615 20691 raft_consensus.cc:493] T 829b4ad8c5a54eababeea1181b960d4b P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:58.000900 16300 raft_consensus.cc:2410] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac for term 12 because replica has last-logged OpId of term: 10 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 8.
I20260709 13:20:58.001739 20571 raft_consensus.cc:3055] T af57996a8be84d398f0f023e8d08ef77 P 1ca2725d290e47d68b1a4b609c9816ac [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:20:58.003420 20696 leader_election.cc:290] T 3884f864a96d428785895e5463e5728f P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:57.978345 20622 consensus_queue.cc:1059] T 746aabf856ab401c9ef5e859371e7b67 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.017s
I20260709 13:20:58.010488 16292 raft_consensus.cc:2393] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 13: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
I20260709 13:20:58.083865 16222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6f07842d3aa47858df5665874c86f3a" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:58.084028 16192 leader_election.cc:304] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 
W20260709 13:20:58.011956 20677 log.cc:927] Time spent T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 1.331s	user 0.001s	sys 0.001s
I20260709 13:20:58.013635 20699 leader_election.cc:290] T af57996a8be84d398f0f023e8d08ef77 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:57.996922 20679 raft_consensus.cc:515] T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:58.016438 16297 raft_consensus.cc:2393] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 12: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
I20260709 13:20:57.982916 16291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f2da98864b4425bf7c0055951fdd4b" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:58.022179 16294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7ca61592e848ea9b0fb81a2ce33cdc" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
W20260709 13:20:58.024695 16117 connection.cc:256] RPC call timeout handler was delayed by -1.0506779160000406s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20320000000000002s for a response
W20260709 13:20:58.157058 16117 connection.cc:256] RPC call timeout handler was delayed by -1.0321566249999705s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20320000000000002s for a response
W20260709 13:20:58.024910 16788 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:57.995600 20594 raft_consensus.cc:2749] T 8c11074579b046d99e585c2818e18a2c P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:20:57.978780 20671 raft_consensus.cc:515] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:57.945775 20584 raft_consensus.cc:515] T 8d4ab870a05347868956aa8b841b2981 P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:58.030057 16299 raft_consensus.cc:2468] T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 13.
I20260709 13:20:58.043852 16266 leader_election.cc:304] T 26354361e9b044cea1c2b86c4882fe92 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
I20260709 13:20:58.044149 16219 raft_consensus.cc:2468] T 08c74490c40243a88e62d1760b5cd814 P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 11.
W20260709 13:20:58.038554 16293 rpcz_store.cc:269] 0709 13:20:55.297600 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:55.365649 (+ 68049us) service_pool.cc:224] Handling call
0709 13:20:58.037862 (+2672213us) spinlock_profiling.cc:243] Waited 2.1 s on lock 0x7b540024c554. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:20:58.038025 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4833618688}
I20260709 13:20:58.038776 20431 leader_election.cc:290] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:58.091786 16226 raft_consensus.cc:2468] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 13.
I20260709 13:20:58.134336 16223 raft_consensus.cc:2468] T 5a96373bb0e14dce8a266b2fc6c58507 P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 11.
I20260709 13:20:58.136701 16819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5d59407eb545b895d8d922e0dd69fd" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 14 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:58.178501 16819 raft_consensus.cc:2468] T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 13.
I20260709 13:20:58.043833 20592 raft_consensus.cc:3060] T 8c11074579b046d99e585c2818e18a2c P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:20:58.084498 16222 raft_consensus.cc:2468] T b6f07842d3aa47858df5665874c86f3a P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 13.
I20260709 13:20:58.142671 16217 raft_consensus.cc:2468] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 13.
I20260709 13:20:58.159137 16117 leader_election.cc:304] T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860
I20260709 13:20:58.156510 16294 raft_consensus.cc:2468] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 90f3da57796b488c9087a2f523adf045 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 14.
W20260709 13:20:58.182350 16266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 138546us
I20260709 13:20:58.082990 20657 raft_consensus.cc:2804] T cdbd78acb4cf46298e0de8ba0d130e19 P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:20:58.030472 20427 raft_consensus.cc:515] T 68a6838d82ef4cac83507f906139d90f P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:58.183909 16266 leader_election.cc:304] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860
W20260709 13:20:58.166544 20730 log.cc:927] Time spent T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860: Append to log took a long time: real 0.246s	user 0.001s	sys 0.000s
I20260709 13:20:58.035977 20388 raft_consensus.cc:515] T 1df93779209748b9860f6d893dd3de53 P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:58.080194 20571 raft_consensus.cc:740] T af57996a8be84d398f0f023e8d08ef77 P 1ca2725d290e47d68b1a4b609c9816ac [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:20:58.190907 20699 raft_consensus.cc:2749] T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:20:58.199846 20594 raft_consensus.cc:2749] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 90f3da57796b488c9087a2f523adf045 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
W20260709 13:20:58.200172 20735 log.cc:927] Time spent T 02838c86531d4f98bf58d1c22b8baf77 P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 0.200s	user 0.001s	sys 0.000s
W20260709 13:20:58.201936 20734 log.cc:927] Time spent T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860: Append to log took a long time: real 0.109s	user 0.000s	sys 0.000s
I20260709 13:20:58.211539 16291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e65676b21b748728ad39668cfe722ec" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:20:58.218088 16224 raft_consensus.cc:2468] T e4dc6b0541ca431ebc757beac00521f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 11.
I20260709 13:20:58.221321 16298 raft_consensus.cc:2410] T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac for term 13 because replica has last-logged OpId of term: 11 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 9.
W20260709 13:20:58.223347 20726 log.cc:927] Time spent T 66f75124215f4ca6bef831e174a792d8 P cb82cc7bd6db46d980c75237a1402860: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
I20260709 13:20:58.082420 20691 raft_consensus.cc:515] T 829b4ad8c5a54eababeea1181b960d4b P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:58.232463 16115 leader_election.cc:304] T 23f82224f4564fea93572e8206bad1fe P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:20:58.233253 16148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d16dfad0c3b540e5b4b86492ddf87208" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:58.233843 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2794af5b99984efebdee564035841cb0" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:20:58.234577 16148 raft_consensus.cc:2393] T d16dfad0c3b540e5b4b86492ddf87208 P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 12: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
I20260709 13:20:58.234741 16150 raft_consensus.cc:3060] T 2794af5b99984efebdee564035841cb0 P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:20:58.236893 16144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1df93779209748b9860f6d893dd3de53" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:20:58.237619 16144 raft_consensus.cc:3060] T 1df93779209748b9860f6d893dd3de53 P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:20:58.239216 16146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
W20260709 13:20:58.257974 16266 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:20:58.271188 16192 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 187210us
I20260709 13:20:58.249136 16147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68a6838d82ef4cac83507f906139d90f" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:58.272497 16265 leader_election.cc:304] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:20:58.274608 16147 raft_consensus.cc:2393] T 68a6838d82ef4cac83507f906139d90f P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 12: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
I20260709 13:20:58.275127 16225 raft_consensus.cc:2468] T e5ad4eb281564936a59f41e387a47156 P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 13.
I20260709 13:20:58.275303 16265 leader_election.cc:304] T 8638ee363273426db1393dd81e5c7b72 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: cb82cc7bd6db46d980c75237a1402860
I20260709 13:20:58.279520 20699 consensus_queue.cc:1059] T e5ad4eb281564936a59f41e387a47156 P 647a343a542d4a139fe106c19095129b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.006s
W20260709 13:20:58.280441 20732 log.cc:927] Time spent T 1df93779209748b9860f6d893dd3de53 P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.213s	user 0.001s	sys 0.000s
I20260709 13:20:58.280771 16143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c61a39fd4f84424b129c1a6f9ba404d" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:20:58.281342 16143 raft_consensus.cc:3060] T 9c61a39fd4f84424b129c1a6f9ba404d P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:20:58.281852 20584 leader_election.cc:290] T 8d4ab870a05347868956aa8b841b2981 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:20:58.282488 20679 leader_election.cc:290] T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:58.282912 20518 leader_election.cc:290] T 57a744f5aaf944678a63469cbf514015 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:58.283511 16145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31969392ccdb43b491911a319e8bc7e1" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:58.284248 16145 raft_consensus.cc:2393] T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 11: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
I20260709 13:20:58.186661 20657 raft_consensus.cc:493] T cdbd78acb4cf46298e0de8ba0d130e19 P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:58.293331 16147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5ad4eb281564936a59f41e387a47156" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:20:58.285207 16145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c74490c40243a88e62d1760b5cd814" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:20:58.294476 16147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87ac0d1f7e8a419f9bcbc04aee0e067e" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:20:58.294628 16145 raft_consensus.cc:3060] T 08c74490c40243a88e62d1760b5cd814 P 647a343a542d4a139fe106c19095129b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:20:58.285828 16146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cc959fb7f704fba9009c0595467d074" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:58.295370 16146 raft_consensus.cc:2468] T 8cc959fb7f704fba9009c0595467d074 P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 11.
I20260709 13:20:58.295039 16147 raft_consensus.cc:3060] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:20:58.290895 16144 raft_consensus.cc:2468] T 1df93779209748b9860f6d893dd3de53 P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 12.
I20260709 13:20:58.280247 20475 consensus_queue.cc:1048] T 3884f864a96d428785895e5463e5728f P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:20:58.286453 16265 leader_election.cc:304] T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:20:58.296221 16146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e9b2bf7011f4ae688a79d7bb5a447b4" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:20:58.297399 16144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a96373bb0e14dce8a266b2fc6c58507" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:58.284185 20671 leader_election.cc:290] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:58.292809 16148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a9de347c711477e87c582d003f36925" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:58.307112 16151 raft_consensus.cc:1240] T 02838c86531d4f98bf58d1c22b8baf77 P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 11. Current term is 12. Ops: []
I20260709 13:20:58.332572 16148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 14 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:58.333246 16148 raft_consensus.cc:2468] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 13.
I20260709 13:20:58.307941 16142 raft_consensus.cc:1240] T 9e65676b21b748728ad39668cfe722ec P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 10. Current term is 11. Ops: []
I20260709 13:20:58.332602 16146 raft_consensus.cc:3060] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:20:58.335758 16144 raft_consensus.cc:2393] T 5a96373bb0e14dce8a266b2fc6c58507 P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 11: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
W20260709 13:20:58.343144 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 56802.2us
I20260709 13:20:58.351233 16147 raft_consensus.cc:2468] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 13.
I20260709 13:20:58.280803 20427 leader_election.cc:290] T 68a6838d82ef4cac83507f906139d90f P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
W20260709 13:20:58.352824 16788 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:58.359539 16144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fffab26427e24d398e2695aad40ca134" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:58.353224 16151 raft_consensus.cc:1275] T 18ad7e3af5504571a69d10c00750a01c P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 13:20:58.344084 16265 leader_election.cc:304] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:20:58.365466 16147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c61a39fd4f84424b129c1a6f9ba404d" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:20:58.354606 16142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0eab25ce4c64e018a041beeda2867db" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
W20260709 13:20:58.280295 20733 log.cc:927] Time spent T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860: Append to log took a long time: real 0.099s	user 0.000s	sys 0.001s
I20260709 13:20:58.309423 20657 raft_consensus.cc:3060] T cdbd78acb4cf46298e0de8ba0d130e19 P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:20:58.387598 20577 consensus_meta.cc:287] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 1ca2725d290e47d68b1a4b609c9816ac: Time spent flushing consensus metadata: real 0.516s	user 0.010s	sys 0.000s
I20260709 13:20:58.280491 20602 leader_election.cc:290] T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:58.309214 20594 raft_consensus.cc:2804] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:20:58.388092 20577 raft_consensus.cc:515] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:58.306649 20708 raft_consensus.cc:2804] T 8638ee363273426db1393dd81e5c7b72 P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:20:58.288852 20669 raft_consensus.cc:3055] T 746aabf856ab401c9ef5e859371e7b67 P 1ca2725d290e47d68b1a4b609c9816ac [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:20:58.389328 20669 raft_consensus.cc:740] T 746aabf856ab401c9ef5e859371e7b67 P 1ca2725d290e47d68b1a4b609c9816ac [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:20:58.334515 20679 raft_consensus.cc:2804] T 23f82224f4564fea93572e8206bad1fe P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:20:58.390272 20679 raft_consensus.cc:493] T 23f82224f4564fea93572e8206bad1fe P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:58.384654 16142 raft_consensus.cc:2468] T c0eab25ce4c64e018a041beeda2867db P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 11.
I20260709 13:20:58.390772 20679 raft_consensus.cc:3060] T 23f82224f4564fea93572e8206bad1fe P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:20:58.392390 20669 consensus_queue.cc:260] T 746aabf856ab401c9ef5e859371e7b67 P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:58.393568 20669 raft_consensus.cc:3060] T 746aabf856ab401c9ef5e859371e7b67 P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:20:58.393792 16151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1df93779209748b9860f6d893dd3de53" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:20:58.394480 16151 raft_consensus.cc:2376] T 1df93779209748b9860f6d893dd3de53 P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 12. Re-sending same reply.
I20260709 13:20:58.395385 16151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6f07842d3aa47858df5665874c86f3a" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:58.397334 20642 leader_election.cc:290] T 434bff3082d141838068f323bdb0d1ed P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:58.286190 20618 raft_consensus.cc:2804] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:20:58.399489 20618 raft_consensus.cc:493] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:58.400043 20618 raft_consensus.cc:3060] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:20:58.353010 20661 raft_consensus.cc:2804] T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:20:58.400719 20661 raft_consensus.cc:493] T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:20:58.401069 20661 raft_consensus.cc:3060] T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:20:58.393733 16150 raft_consensus.cc:2468] T 2794af5b99984efebdee564035841cb0 P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 9.
I20260709 13:20:58.332515 20583 raft_consensus.cc:2749] T 26354361e9b044cea1c2b86c4882fe92 P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:20:58.403613 20655 consensus_queue.cc:1048] T b87147731a68400e8cce9b879daa7c5d P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.119s
I20260709 13:20:58.406088 20708 raft_consensus.cc:697] T 8638ee363273426db1393dd81e5c7b72 P 90f3da57796b488c9087a2f523adf045 [term 13 LEADER]: Becoming Leader. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:20:58.332109 20584 raft_consensus.cc:493] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:58.388481 20594 raft_consensus.cc:697] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045 [term 11 LEADER]: Becoming Leader. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:20:58.406456 20644 leader_election.cc:290] T d2ab0ac8347641e6955e9bce7d90c1a2 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:58.385998 16144 raft_consensus.cc:2468] T fffab26427e24d398e2695aad40ca134 P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 9.
I20260709 13:20:58.386238 16146 raft_consensus.cc:2468] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 647a343a542d4a139fe106c19095129b [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 10.
I20260709 13:20:58.385852 16265 leader_election.cc:304] T e5ad4eb281564936a59f41e387a47156 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:20:58.424431 16265 leader_election.cc:304] T d16dfad0c3b540e5b4b86492ddf87208 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:20:58.384907 16148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:58.414551 20584 raft_consensus.cc:515] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:58.416044 20708 consensus_queue.cc:237] T 8638ee363273426db1393dd81e5c7b72 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:58.418993 20594 consensus_queue.cc:237] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:58.421732 16143 raft_consensus.cc:2410] T 9c61a39fd4f84424b129c1a6f9ba404d P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 for term 12 because replica has last-logged OpId of term: 11 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
I20260709 13:20:58.429471 16265 leader_election.cc:304] T 8cc959fb7f704fba9009c0595467d074 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac
I20260709 13:20:58.413157 20518 raft_consensus.cc:2804] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:20:58.413185 20577 leader_election.cc:290] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 15 election: Requested vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:58.432209 20644 raft_consensus.cc:2804] T e5ad4eb281564936a59f41e387a47156 P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:20:58.433218 20573 raft_consensus.cc:2804] T 8cc959fb7f704fba9009c0595467d074 P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:20:58.431918 20642 raft_consensus.cc:2749] T d16dfad0c3b540e5b4b86492ddf87208 P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:20:58.445894 16788 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:58.412731 20571 consensus_queue.cc:260] T af57996a8be84d398f0f023e8d08ef77 P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:58.455106 20678 raft_consensus.cc:493] T 18ad7e3af5504571a69d10c00750a01c P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:58.448751 20573 raft_consensus.cc:493] T 8cc959fb7f704fba9009c0595467d074 P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:20:58.470253 20679 raft_consensus.cc:515] T 23f82224f4564fea93572e8206bad1fe P 647a343a542d4a139fe106c19095129b [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:58.472517 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:58.473261 16299 raft_consensus.cc:2468] T ecc1e363230043fe9fef390f0b979f20 P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 11.
I20260709 13:20:58.474150 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31969392ccdb43b491911a319e8bc7e1" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:58.474447 20588 leader_election.cc:290] T b87147731a68400e8cce9b879daa7c5d P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:58.474735 16299 raft_consensus.cc:2468] T 31969392ccdb43b491911a319e8bc7e1 P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 10.
I20260709 13:20:58.461023 20475 raft_consensus.cc:493] T 5a959dbecc2648eb85d1cc550f5a3245 P 647a343a542d4a139fe106c19095129b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:20:58.484874 16115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 252473us
I20260709 13:20:58.466755 20573 raft_consensus.cc:3060] T 8cc959fb7f704fba9009c0595467d074 P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:20:58.486013 16115 leader_election.cc:304] T fffab26427e24d398e2695aad40ca134 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:20:58.492798 20236 leader_election.cc:290] T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:58.495529 20236 consensus_queue.cc:1048] T 7c2095a5f4ad49408dbc7f69c606120d P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.003s
I20260709 13:20:58.499959 20573 raft_consensus.cc:515] T 8cc959fb7f704fba9009c0595467d074 P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:58.502238 20573 leader_election.cc:290] T 8cc959fb7f704fba9009c0595467d074 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 12 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
W20260709 13:20:58.512249 16191 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:20:58.513216 16191 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:20:58.513693 16191 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:58.515563 15046 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:20:58.517943 15046 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:20:58.519075 16143 raft_consensus.cc:1240] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b [term 12 LEADER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 11. Current term is 12. Ops: []
I20260709 13:20:58.523571 16146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "829b4ad8c5a54eababeea1181b960d4b" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:58.524453 16146 raft_consensus.cc:2468] T 829b4ad8c5a54eababeea1181b960d4b P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 11.
I20260709 13:20:58.529336 20719 consensus_queue.cc:260] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:58.462301 20711 consensus_queue.cc:1059] T 02838c86531d4f98bf58d1c22b8baf77 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.021s
W20260709 13:20:58.536020 16190 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.603s	user 0.010s	sys 0.016s
W20260709 13:20:58.536423 16190 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.604s	user 0.010s	sys 0.016s
W20260709 13:20:58.536774 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 604464us
W20260709 13:20:58.538249 16190 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:20:58.539040 16190 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:20:58.542881 16115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 56931.8us
I20260709 13:20:58.465620 20678 raft_consensus.cc:515] T 18ad7e3af5504571a69d10c00750a01c P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } }
I20260709 13:20:58.563964 16115 leader_election.cc:304] T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:20:58.562086 16295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7ca61592e848ea9b0fb81a2ce33cdc" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:20:58.567435 16295 raft_consensus.cc:3060] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 90f3da57796b488c9087a2f523adf045 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:20:58.568642 16148 raft_consensus.cc:1275] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 13:20:58.531045 20719 raft_consensus.cc:3060] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:20:58.558681 20672 consensus_meta.cc:287] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac: Time spent flushing consensus metadata: real 0.707s	user 0.011s	sys 0.000s
I20260709 13:20:58.577939 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e65676b21b748728ad39668cfe722ec" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:20:58.579027 16149 raft_consensus.cc:2393] T 9e65676b21b748728ad39668cfe722ec P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 in current term 11: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
I20260709 13:20:58.581877 16300 raft_consensus.cc:1275] T 66ae176961b74813a58bc15c9c4c6f2a P 90f3da57796b488c9087a2f523adf045 [term 14 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:20:58.484087 20475 raft_consensus.cc:515] T 5a959dbecc2648eb85d1cc550f5a3245 P 647a343a542d4a139fe106c19095129b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:58.590231 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5d59407eb545b895d8d922e0dd69fd" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 14 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:58.489265 20657 raft_consensus.cc:515] T cdbd78acb4cf46298e0de8ba0d130e19 P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:20:58.606364 16190 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:58.611337 16300 raft_consensus.cc:1240] T 2794af5b99984efebdee564035841cb0 P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 8. Current term is 9. Ops: []
I20260709 13:20:58.616202 16295 raft_consensus.cc:2468] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 15.
I20260709 13:20:58.580981 20672 raft_consensus.cc:2749] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Leader election lost for term 12. Reason: Vote denied by peer cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923) with higher term. Message: Invalid argument: T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac for earlier term 12. Current term is 13.
I20260709 13:20:58.619938 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746aabf856ab401c9ef5e859371e7b67" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:20:58.630986 16191 leader_election.cc:304] T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:20:58.646639 16300 raft_consensus.cc:1240] T 26354361e9b044cea1c2b86c4882fe92 P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 9. Current term is 10. Ops: []
I20260709 13:20:58.647804 20679 leader_election.cc:290] T 23f82224f4564fea93572e8206bad1fe P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 10 election: Requested vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:58.605855 20669 raft_consensus.cc:493] T 829b4ad8c5a54eababeea1181b960d4b P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:58.606971 20293 raft_consensus.cc:493] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:58.606576 19861 raft_consensus.cc:493] T 7c2095a5f4ad49408dbc7f69c606120d P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:58.657438 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:58.658511 16300 raft_consensus.cc:1275] T ecc1e363230043fe9fef390f0b979f20 P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 13:20:58.629488 20685 raft_consensus.cc:493] T ecc1e363230043fe9fef390f0b979f20 P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:58.629233 20584 leader_election.cc:290] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:20:58.675432 20601 consensus_queue.cc:1048] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:20:58.662000 20293 raft_consensus.cc:515] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:58.705339 20685 raft_consensus.cc:515] T ecc1e363230043fe9fef390f0b979f20 P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:58.708166 20293 leader_election.cc:290] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:58.678776 20678 leader_election.cc:290] T 18ad7e3af5504571a69d10c00750a01c P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:20:58.665158 20431 consensus_queue.cc:1059] T 68a6838d82ef4cac83507f906139d90f P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.015s
I20260709 13:20:58.710426 20431 raft_consensus.cc:3055] T 68a6838d82ef4cac83507f906139d90f P 1ca2725d290e47d68b1a4b609c9816ac [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:20:58.711006 20431 raft_consensus.cc:740] T 68a6838d82ef4cac83507f906139d90f P 1ca2725d290e47d68b1a4b609c9816ac [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:20:58.712029 20431 consensus_queue.cc:260] T 68a6838d82ef4cac83507f906139d90f P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:58.713670 20431 raft_consensus.cc:3060] T 68a6838d82ef4cac83507f906139d90f P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:20:58.681294 16148 raft_consensus.cc:1240] T 23f82224f4564fea93572e8206bad1fe P 647a343a542d4a139fe106c19095129b [term 10 FOLLOWER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 8. Current term is 10. Ops: []
I20260709 13:20:58.683668 20657 leader_election.cc:290] T cdbd78acb4cf46298e0de8ba0d130e19 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 13 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:58.668089 19861 raft_consensus.cc:515] T 7c2095a5f4ad49408dbc7f69c606120d P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:58.717545 19861 leader_election.cc:290] T 7c2095a5f4ad49408dbc7f69c606120d P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:58.666639 16190 leader_election.cc:304] T 8a878999c0024d2abe517484180290fc P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
W20260709 13:20:58.718354 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 51818.8us
I20260709 13:20:58.664132 20669 raft_consensus.cc:515] T 829b4ad8c5a54eababeea1181b960d4b P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:58.687999 20679 raft_consensus.cc:2749] T fffab26427e24d398e2695aad40ca134 P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:20:58.719256 16190 leader_election.cc:304] T d2ab0ac8347641e6955e9bce7d90c1a2 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860
I20260709 13:20:58.662169 20671 consensus_queue.cc:1048] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.027s
I20260709 13:20:58.720938 20669 leader_election.cc:290] T 829b4ad8c5a54eababeea1181b960d4b P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:58.721210 20671 raft_consensus.cc:2804] T 8a878999c0024d2abe517484180290fc P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:20:58.721457 16190 leader_election.cc:304] T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:20:58.668116 16145 raft_consensus.cc:2468] T 08c74490c40243a88e62d1760b5cd814 P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 11.
W20260709 13:20:58.724007 16192 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:58.724433 16190 leader_election.cc:304] T b6f07842d3aa47858df5665874c86f3a P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:20:58.690011 20699 raft_consensus.cc:2804] T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:20:58.713342 20661 raft_consensus.cc:515] T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:58.660197 20577 raft_consensus.cc:2749] T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:20:58.704475 20475 leader_election.cc:290] T 5a959dbecc2648eb85d1cc550f5a3245 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:58.721652 20671 raft_consensus.cc:493] T 8a878999c0024d2abe517484180290fc P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Starting leader election (detected failure of leader 647a343a542d4a139fe106c19095129b)
W20260709 13:20:58.736646 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 273503us
I20260709 13:20:58.727240 16190 leader_election.cc:304] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
W20260709 13:20:58.729547 16117 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:58.730317 20293 raft_consensus.cc:2804] T b6f07842d3aa47858df5665874c86f3a P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:20:58.668011 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3884f864a96d428785895e5463e5728f" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:58.721513 20669 raft_consensus.cc:2749] T d2ab0ac8347641e6955e9bce7d90c1a2 P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:20:58.726495 20699 raft_consensus.cc:493] T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:58.741636 16295 raft_consensus.cc:1240] T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045 [term 14 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 13. Current term is 14. Ops: []
I20260709 13:20:58.728502 20601 raft_consensus.cc:2804] T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 13:20:58.744314 16788 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:58.734485 20671 raft_consensus.cc:3060] T 8a878999c0024d2abe517484180290fc P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:20:58.708791 20685 leader_election.cc:290] T ecc1e363230043fe9fef390f0b979f20 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
W20260709 13:20:58.752590 16115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 188710us
I20260709 13:20:58.746372 20577 raft_consensus.cc:2804] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:20:58.746429 20293 raft_consensus.cc:493] T b6f07842d3aa47858df5665874c86f3a P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Starting leader election (detected failure of leader 647a343a542d4a139fe106c19095129b)
W20260709 13:20:58.756307 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 255538us
I20260709 13:20:58.747938 20601 raft_consensus.cc:493] T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:58.748394 20699 raft_consensus.cc:3060] T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:20:58.775450 16266 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:58.758873 16115 leader_election.cc:304] T 5a96373bb0e14dce8a266b2fc6c58507 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
W20260709 13:20:58.803050 16788 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:20:58.823220 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 389716us
W20260709 13:20:58.760385 16790 leader_election.cc:336] T ee9ed67006144797b1de6ea8c2013a68 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 9 election: RPC error from VoteRequest() call to peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943): Timed out: RequestConsensusVote RPC to 127.14.177.130:35943 timed out after 2.176s (SENT)
I20260709 13:20:58.824486 16265 leader_election.cc:304] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
W20260709 13:20:58.824549 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 64210.6us
W20260709 13:20:58.827359 16115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 70367.5us
I20260709 13:20:58.829188 16115 leader_election.cc:304] T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:20:58.830570 16790 leader_election.cc:304] T 829b4ad8c5a54eababeea1181b960d4b P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:20:58.766196 20293 raft_consensus.cc:3060] T b6f07842d3aa47858df5665874c86f3a P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:20:58.822115 20601 raft_consensus.cc:3060] T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:20:58.835708 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18ad7e3af5504571a69d10c00750a01c" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:20:58.822130 20577 raft_consensus.cc:697] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 1ca2725d290e47d68b1a4b609c9816ac [term 15 LEADER]: Becoming Leader. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:20:58.840332 20696 consensus_queue.cc:1048] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.011s
I20260709 13:20:58.854907 20661 leader_election.cc:290] T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 14 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:20:58.831672 20518 raft_consensus.cc:697] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 90f3da57796b488c9087a2f523adf045 [term 13 LEADER]: Becoming Leader. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:20:58.832063 19861 raft_consensus.cc:493] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:20:58.834635 20267 raft_consensus.cc:2804] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:20:58.837059 20678 raft_consensus.cc:2804] T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:20:58.838207 20679 raft_consensus.cc:2804] T 5a96373bb0e14dce8a266b2fc6c58507 P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 13:20:58.853503 20737 log.cc:927] Time spent T d16dfad0c3b540e5b4b86492ddf87208 P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 0.507s	user 0.000s	sys 0.005s
I20260709 13:20:58.840477 20662 raft_consensus.cc:2804] T 829b4ad8c5a54eababeea1181b960d4b P cb82cc7bd6db46d980c75237a1402860 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:20:58.837472 20644 raft_consensus.cc:697] T e5ad4eb281564936a59f41e387a47156 P 90f3da57796b488c9087a2f523adf045 [term 13 LEADER]: Becoming Leader. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
W20260709 13:20:58.858562 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 190869us
I20260709 13:20:58.850945 16291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c2095a5f4ad49408dbc7f69c606120d" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:58.832927 20641 consensus_queue.cc:1059] T ee9ed67006144797b1de6ea8c2013a68 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.029s
I20260709 13:20:58.859433 20267 raft_consensus.cc:493] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:58.875355 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f82224f4564fea93572e8206bad1fe" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:20:58.860146 20678 raft_consensus.cc:493] T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:58.849823 20584 consensus_queue.cc:1059] T 9e65676b21b748728ad39668cfe722ec P 90f3da57796b488c9087a2f523adf045 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.019s
I20260709 13:20:58.865187 20679 raft_consensus.cc:493] T 5a96373bb0e14dce8a266b2fc6c58507 P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:58.889536 20679 raft_consensus.cc:3060] T 5a96373bb0e14dce8a266b2fc6c58507 P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:20:58.889940 20577 consensus_queue.cc:237] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:58.851728 20573 consensus_queue.cc:1048] T 18ad7e3af5504571a69d10c00750a01c P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.018s
W20260709 13:20:58.893142 16114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:58.893738 20518 consensus_queue.cc:237] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 90f3da57796b488c9087a2f523adf045 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:58.879096 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a959dbecc2648eb85d1cc550f5a3245" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:58.880192 20267 raft_consensus.cc:3060] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:20:58.900254 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68a6838d82ef4cac83507f906139d90f" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:58.884588 20443 raft_consensus.cc:493] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:58.888762 20678 raft_consensus.cc:3060] T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:20:58.869685 20661 raft_consensus.cc:493] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:58.873129 20662 raft_consensus.cc:493] T 829b4ad8c5a54eababeea1181b960d4b P cb82cc7bd6db46d980c75237a1402860 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:58.901510 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18ad7e3af5504571a69d10c00750a01c" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 12 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:58.936127 16218 raft_consensus.cc:1275] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 15 index: 11. (term mismatch)
I20260709 13:20:58.952036 16218 pending_rounds.cc:85] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac: Aborting all ops after (but not including) 10
I20260709 13:20:58.952399 16218 pending_rounds.cc:107] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac: Aborting uncommitted NO_OP operation due to leader change: 14.11
I20260709 13:20:58.949944 16297 raft_consensus.cc:1275] T c4f65cdf00b748bb963ee6e81ef43b79 P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 13:20:58.903473 16291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66ae176961b74813a58bc15c9c4c6f2a" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 15 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:58.905797 16265 leader_election.cc:304] T 5a96373bb0e14dce8a266b2fc6c58507 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 647a343a542d4a139fe106c19095129b
I20260709 13:20:58.954324 16291 raft_consensus.cc:2410] T 66ae176961b74813a58bc15c9c4c6f2a P 90f3da57796b488c9087a2f523adf045 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac for term 15 because replica has last-logged OpId of term: 12 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 9.
I20260709 13:20:58.889971 16296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 12 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:58.954802 16297 raft_consensus.cc:1275] T c0eab25ce4c64e018a041beeda2867db P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 11 index: 7. (term mismatch)
I20260709 13:20:58.955224 16297 pending_rounds.cc:85] T c0eab25ce4c64e018a041beeda2867db P 90f3da57796b488c9087a2f523adf045: Aborting all ops after (but not including) 6
I20260709 13:20:58.955392 16296 raft_consensus.cc:2410] T ecc1e363230043fe9fef390f0b979f20 P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac for term 12 because replica has last-logged OpId of term: 10 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 13:20:58.955255 16218 raft_consensus.cc:2889] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:20:58.911733 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4dc6b0541ca431ebc757beac00521f1" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:58.956698 16298 raft_consensus.cc:2468] T e4dc6b0541ca431ebc757beac00521f1 P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 10.
I20260709 13:20:58.956951 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5d59407eb545b895d8d922e0dd69fd" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 14 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:58.957229 16219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ab0ac8347641e6955e9bce7d90c1a2" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:58.957613 16226 raft_consensus.cc:2468] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 13.
I20260709 13:20:58.958439 16298 raft_consensus.cc:1240] T cdbd78acb4cf46298e0de8ba0d130e19 P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 12. Current term is 13. Ops: [12.11-12.11]
W20260709 13:20:58.913576 16189 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:20:58.959393 16189 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:58.918213 16223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af57996a8be84d398f0f023e8d08ef77" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:20:58.961714 16217 raft_consensus.cc:1240] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 12. Current term is 13. Ops: [12.9-12.9]
I20260709 13:20:58.955897 16297 pending_rounds.cc:107] T c0eab25ce4c64e018a041beeda2867db P 90f3da57796b488c9087a2f523adf045: Aborting uncommitted NO_OP operation due to leader change: 10.7
I20260709 13:20:58.962337 16297 raft_consensus.cc:2889] T c0eab25ce4c64e018a041beeda2867db P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:20:58.962257 16218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62fe3346cd3c42c4a55e14af9073bca4" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:20:58.924283 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "829b4ad8c5a54eababeea1181b960d4b" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:58.963619 16225 raft_consensus.cc:2468] T 829b4ad8c5a54eababeea1181b960d4b P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 11.
I20260709 13:20:58.863634 19861 raft_consensus.cc:515] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:20:58.983824 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 78117.6us
I20260709 13:20:58.994638 16221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a378d7f796f94615b3b347119a9880fd" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:20:58.995090 16220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7ca61592e848ea9b0fb81a2ce33cdc" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:58.999423 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:59.000126 20603 raft_consensus.cc:2804] T 5a96373bb0e14dce8a266b2fc6c58507 P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:20:59.000742 20603 raft_consensus.cc:493] T 5a96373bb0e14dce8a266b2fc6c58507 P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:20:59.001137 20603 raft_consensus.cc:3060] T 5a96373bb0e14dce8a266b2fc6c58507 P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:20:59.011377 20760 raft_consensus.cc:493] T c2f5f89186204242bf377f7c42b2e2da P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:59.011874 16294 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48684 (request call id 2559) took 1086 ms. Trace:
I20260709 13:20:59.012086 20760 raft_consensus.cc:515] T c2f5f89186204242bf377f7c42b2e2da P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:59.012328 16294 rpcz_store.cc:276] 0709 13:20:57.925319 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:58.182931 (+257612us) service_pool.cc:224] Handling call
0709 13:20:58.183224 (+   293us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:58.236040 (+ 52816us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:20:58.735783 (+499743us) log.cc:844] Serialized 14 byte log entry
0709 13:20:58.875364 (+139581us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:20:58.875419 (+    55us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:20:58.875472 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:59.011650 (+136178us) spinlock_profiling.cc:243] Waited 52.5 ms on lock 0x7b540032f854. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:20:59.011697 (+    47us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:59.011812 (+   115us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":101430,"raft.queue_time_us":89,"raft.run_cpu_time_us":166,"raft.run_wall_time_us":11093,"spinlock_wait_cycles":120837376,"thread_start_us":524989,"threads_started":2}
I20260709 13:20:59.000222 16226 raft_consensus.cc:2468] T ecc1e363230043fe9fef390f0b979f20 P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 11.
I20260709 13:20:58.999554 16220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4f65cdf00b748bb963ee6e81ef43b79" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:59.016330 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a878999c0024d2abe517484180290fc" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:59.017993 16218 raft_consensus.cc:2468] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 15.
I20260709 13:20:59.018946 16218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af57996a8be84d398f0f023e8d08ef77" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:59.019418 16146 raft_consensus.cc:1275] T d16dfad0c3b540e5b4b86492ddf87208 P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:20:59.020015 16218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "829b4ad8c5a54eababeea1181b960d4b" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:59.020601 16218 raft_consensus.cc:2468] T 829b4ad8c5a54eababeea1181b960d4b P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 11.
W20260709 13:20:58.903707 20744 log.cc:927] Time spent T 7c2095a5f4ad49408dbc7f69c606120d P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.156s	user 0.000s	sys 0.001s
I20260709 13:20:59.021551 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74ca8140f35c4ae29e8a664f6bdfc829" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:59.028127 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee9ed67006144797b1de6ea8c2013a68" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:20:59.029224 16225 raft_consensus.cc:2393] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 in current term 12: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
I20260709 13:20:59.028106 16116 leader_election.cc:304] T 829b4ad8c5a54eababeea1181b960d4b P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
I20260709 13:20:59.030661 16220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66f75124215f4ca6bef831e174a792d8" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:58.912740 20646 raft_consensus.cc:3055] T 23f82224f4564fea93572e8206bad1fe P cb82cc7bd6db46d980c75237a1402860 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:20:58.909555 20640 leader_election.cc:290] T fffab26427e24d398e2695aad40ca134 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:20:58.839125 20655 consensus_queue.cc:1048] T ecc1e363230043fe9fef390f0b979f20 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.122s
I20260709 13:20:59.023152 16146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806d7690c2b644959152074bb2cab0a7" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:59.038599 16146 raft_consensus.cc:2468] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 11.
I20260709 13:20:59.024537 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c61a39fd4f84424b129c1a6f9ba404d" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:59.025177 16144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5ad4eb281564936a59f41e387a47156" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:58.914366 20662 raft_consensus.cc:3060] T 829b4ad8c5a54eababeea1181b960d4b P cb82cc7bd6db46d980c75237a1402860 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:20:58.912840 20641 raft_consensus.cc:3055] T ee9ed67006144797b1de6ea8c2013a68 P cb82cc7bd6db46d980c75237a1402860 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:20:59.047338 20641 raft_consensus.cc:740] T ee9ed67006144797b1de6ea8c2013a68 P cb82cc7bd6db46d980c75237a1402860 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:20:59.048391 20641 consensus_queue.cc:260] T ee9ed67006144797b1de6ea8c2013a68 P cb82cc7bd6db46d980c75237a1402860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:59.049788 20641 raft_consensus.cc:3060] T ee9ed67006144797b1de6ea8c2013a68 P cb82cc7bd6db46d980c75237a1402860 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:20:59.025945 16143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2372981006d4b218215c28964dea623" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 13 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:20:59.052462 16143 raft_consensus.cc:3060] T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:20:59.026473 16145 raft_consensus.cc:1240] T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 10. Current term is 11. Ops: []
I20260709 13:20:59.036203 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f5f89186204242bf377f7c42b2e2da" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:59.054123 16226 raft_consensus.cc:2468] T c2f5f89186204242bf377f7c42b2e2da P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 12.
I20260709 13:20:58.879624 20755 raft_consensus.cc:493] T 9c61a39fd4f84424b129c1a6f9ba404d P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:20:59.055269 16788 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:59.055631 16116 leader_election.cc:304] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 90f3da57796b488c9087a2f523adf045
I20260709 13:20:59.055466 20755 raft_consensus.cc:515] T 9c61a39fd4f84424b129c1a6f9ba404d P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:59.056860 16149 raft_consensus.cc:2468] T 9c61a39fd4f84424b129c1a6f9ba404d P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 12.
I20260709 13:20:59.057739 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a959dbecc2648eb85d1cc550f5a3245" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:59.058569 16226 raft_consensus.cc:2410] T 5a959dbecc2648eb85d1cc550f5a3245 P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b for term 11 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 13:20:58.904202 20644 consensus_queue.cc:237] T e5ad4eb281564936a59f41e387a47156 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:20:58.914072 20618 consensus_meta.cc:287] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac: Time spent flushing consensus metadata: real 0.514s	user 0.012s	sys 0.000s
I20260709 13:20:59.045285 16221 raft_consensus.cc:2468] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 13.
I20260709 13:20:59.045909 20699 raft_consensus.cc:515] T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:59.059983 20750 raft_consensus.cc:2804] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:20:59.075611 20601 raft_consensus.cc:515] T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:59.076052 16221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68a6838d82ef4cac83507f906139d90f" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:59.076218 20750 raft_consensus.cc:493] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Starting leader election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:20:59.046767 16292 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48674 (request call id 2712) took 1140 ms. Trace:
I20260709 13:20:59.077502 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02838c86531d4f98bf58d1c22b8baf77" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:20:58.912324 20661 raft_consensus.cc:515] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:59.078860 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18ad7e3af5504571a69d10c00750a01c" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 12 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:59.079619 16296 raft_consensus.cc:1275] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:20:59.080013 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a96373bb0e14dce8a266b2fc6c58507" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:59.080320 16221 raft_consensus.cc:2468] T 68a6838d82ef4cac83507f906139d90f P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 12.
I20260709 13:20:59.076789 16292 rpcz_store.cc:276] 0709 13:20:57.906731 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:58.084079 (+177348us) service_pool.cc:224] Handling call
0709 13:20:58.084383 (+   304us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:20:58.084723 (+   340us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:58.084747 (+    24us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:20:58.166284 (+ 81537us) log.cc:844] Serialized 26 byte log entry
0709 13:20:58.566935 (+400651us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:20:58.566974 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:58.567046 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:20:59.046569 (+479523us) raft_consensus.cc:1685] finished
0709 13:20:59.046587 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:59.046712 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":10514,"raft.run_cpu_time_us":147,"raft.run_wall_time_us":19762,"thread_start_us":369003,"threads_started":1,"wal-append.queue_time_us":546139}
I20260709 13:20:59.081413 16116 leader_election.cc:304] T 02838c86531d4f98bf58d1c22b8baf77 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:20:59.059482 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2372981006d4b218215c28964dea623" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:59.082548 16217 raft_consensus.cc:2468] T 5a96373bb0e14dce8a266b2fc6c58507 P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 11.
I20260709 13:20:59.082811 16226 raft_consensus.cc:2393] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 13: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
I20260709 13:20:59.037467 20475 raft_consensus.cc:2804] T 829b4ad8c5a54eababeea1181b960d4b P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:20:59.083884 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746aabf856ab401c9ef5e859371e7b67" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:20:58.901957 20443 raft_consensus.cc:515] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:59.086905 20475 raft_consensus.cc:493] T 829b4ad8c5a54eababeea1181b960d4b P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:59.068781 20671 raft_consensus.cc:515] T 8a878999c0024d2abe517484180290fc P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:59.090955 20475 raft_consensus.cc:3060] T 829b4ad8c5a54eababeea1181b960d4b P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:20:59.059154 20678 raft_consensus.cc:515] T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:59.087641 20427 raft_consensus.cc:2749] T 02838c86531d4f98bf58d1c22b8baf77 P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:20:59.098243 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0eab25ce4c64e018a041beeda2867db" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:59.098615 16219 raft_consensus.cc:2376] T d2ab0ac8347641e6955e9bce7d90c1a2 P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 12. Re-sending same reply.
I20260709 13:20:59.099853 20755 leader_election.cc:290] T 9c61a39fd4f84424b129c1a6f9ba404d P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:59.038306 20646 raft_consensus.cc:740] T 23f82224f4564fea93572e8206bad1fe P cb82cc7bd6db46d980c75237a1402860 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:20:59.106829 16265 leader_election.cc:304] T c0eab25ce4c64e018a041beeda2867db P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac
I20260709 13:20:59.108389 16265 leader_election.cc:304] T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:20:59.076686 20750 raft_consensus.cc:3060] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:20:59.110615 16143 raft_consensus.cc:2410] T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac for term 13 because replica has last-logged OpId of term: 12 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 8.
I20260709 13:20:59.115607 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62fe3346cd3c42c4a55e14af9073bca4" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 16 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:59.124358 16217 raft_consensus.cc:2468] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 15.
I20260709 13:20:59.106458 20267 raft_consensus.cc:515] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:59.116715 20678 leader_election.cc:290] T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 12 election: Requested vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:59.128539 20751 raft_consensus.cc:2804] T c0eab25ce4c64e018a041beeda2867db P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:20:59.154104 20751 raft_consensus.cc:493] T c0eab25ce4c64e018a041beeda2867db P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:59.154313 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c61a39fd4f84424b129c1a6f9ba404d" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:59.154582 20751 raft_consensus.cc:3060] T c0eab25ce4c64e018a041beeda2867db P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:20:59.156508 20267 leader_election.cc:290] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 14 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:59.129902 16265 leader_election.cc:304] T fffab26427e24d398e2695aad40ca134 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:20:59.069924 20662 raft_consensus.cc:515] T 829b4ad8c5a54eababeea1181b960d4b P cb82cc7bd6db46d980c75237a1402860 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:59.161761 20662 leader_election.cc:290] T 829b4ad8c5a54eababeea1181b960d4b P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 12 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:20:59.131789 20640 raft_consensus.cc:2749] T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:20:59.087183 20618 raft_consensus.cc:515] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:59.164816 16265 leader_election.cc:304] T ecc1e363230043fe9fef390f0b979f20 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:20:59.166599 16217 raft_consensus.cc:2393] T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 13: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
I20260709 13:20:59.129045 20679 raft_consensus.cc:515] T 5a96373bb0e14dce8a266b2fc6c58507 P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:59.129336 20760 leader_election.cc:290] T c2f5f89186204242bf377f7c42b2e2da P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:59.149567 16225 raft_consensus.cc:2468] T 746aabf856ab401c9ef5e859371e7b67 P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 11.
I20260709 13:20:59.169055 16265 leader_election.cc:304] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: cb82cc7bd6db46d980c75237a1402860
W20260709 13:20:59.171375 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 322617us
I20260709 13:20:59.180943 20518 raft_consensus.cc:2804] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:20:59.189538 16218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f2da98864b4425bf7c0055951fdd4b" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:59.166491 20661 leader_election.cc:290] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:20:59.197012 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746aabf856ab401c9ef5e859371e7b67" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:59.178994 20267 raft_consensus.cc:2749] T ecc1e363230043fe9fef390f0b979f20 P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:20:59.198553 16143 raft_consensus.cc:1275] T c0eab25ce4c64e018a041beeda2867db P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:20:59.197845 16225 raft_consensus.cc:2376] T 746aabf856ab401c9ef5e859371e7b67 P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 11. Re-sending same reply.
I20260709 13:20:59.198365 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66f75124215f4ca6bef831e174a792d8" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:59.184969 16790 leader_election.cc:304] T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 647a343a542d4a139fe106c19095129b
I20260709 13:20:59.167961 20640 raft_consensus.cc:2804] T fffab26427e24d398e2695aad40ca134 P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:20:59.206959 20640 raft_consensus.cc:493] T fffab26427e24d398e2695aad40ca134 P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:20:59.207485 20640 raft_consensus.cc:3060] T fffab26427e24d398e2695aad40ca134 P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:20:59.211967 16191 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:59.187433 20518 raft_consensus.cc:697] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 90f3da57796b488c9087a2f523adf045 [term 10 LEADER]: Becoming Leader. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:20:59.167217 20293 raft_consensus.cc:515] T b6f07842d3aa47858df5665874c86f3a P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:59.164449 20646 consensus_queue.cc:260] T 23f82224f4564fea93572e8206bad1fe P cb82cc7bd6db46d980c75237a1402860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:59.217748 16142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6f07842d3aa47858df5665874c86f3a" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:59.217839 16145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87ac0d1f7e8a419f9bcbc04aee0e067e" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:59.218053 16143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a959dbecc2648eb85d1cc550f5a3245" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:59.242450 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4dc6b0541ca431ebc757beac00521f1" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:59.213198 16218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806d7690c2b644959152074bb2cab0a7" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:59.252202 16218 raft_consensus.cc:2410] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b for term 12 because replica has last-logged OpId of term: 10 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 7.
I20260709 13:20:59.252990 16220 raft_consensus.cc:1240] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 12. Current term is 13. Ops: []
I20260709 13:20:59.195528 20641 consensus_queue.cc:1048] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.023s
I20260709 13:20:59.254575 16220 raft_consensus.cc:1275] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 13:20:59.254565 16218 raft_consensus.cc:1275] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 12 index: 12. (index mismatch)
I20260709 13:20:59.256762 20573 raft_consensus.cc:3055] T 9e65676b21b748728ad39668cfe722ec P 90f3da57796b488c9087a2f523adf045 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:20:59.256853 16142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5d59407eb545b895d8d922e0dd69fd" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 14 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:59.257319 20573 raft_consensus.cc:740] T 9e65676b21b748728ad39668cfe722ec P 90f3da57796b488c9087a2f523adf045 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:20:59.257830 16147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 12 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:59.258363 16142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "829b4ad8c5a54eababeea1181b960d4b" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:59.258386 20573 consensus_queue.cc:260] T 9e65676b21b748728ad39668cfe722ec P 90f3da57796b488c9087a2f523adf045 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:59.259562 16144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5718c623ebee4d2bb770f0f9ddb2f164" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 13 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:59.260365 16147 raft_consensus.cc:1275] T 8158bbd17b994d2eb96f560bd0f1b88d P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 13:20:59.261309 16146 raft_consensus.cc:1275] T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b [term 10 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:20:59.262072 16218 raft_consensus.cc:1275] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:20:59.262746 16151 raft_consensus.cc:1240] T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b [term 14 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 13. Current term is 14. Ops: [13.9-13.9]
I20260709 13:20:59.263338 16218 raft_consensus.cc:1240] T 8a878999c0024d2abe517484180290fc P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 10. Current term is 11. Ops: []
I20260709 13:20:59.264568 16218 raft_consensus.cc:1240] T e5ad4eb281564936a59f41e387a47156 P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 12. Current term is 13. Ops: []
I20260709 13:20:59.240978 16143 raft_consensus.cc:2468] T 5a959dbecc2648eb85d1cc550f5a3245 P 647a343a542d4a139fe106c19095129b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 10.
I20260709 13:20:59.266255 20640 raft_consensus.cc:515] T fffab26427e24d398e2695aad40ca134 P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:59.267401 16116 leader_election.cc:304] T c0eab25ce4c64e018a041beeda2867db P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:20:59.267618 20679 leader_election.cc:290] T 5a96373bb0e14dce8a266b2fc6c58507 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 12 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:59.268678 16116 leader_election.cc:304] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
I20260709 13:20:59.269546 20678 raft_consensus.cc:2749] T c0eab25ce4c64e018a041beeda2867db P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:20:59.270253 20678 raft_consensus.cc:2804] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:20:59.270622 20678 raft_consensus.cc:493] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:59.270859 20573 raft_consensus.cc:3060] T 9e65676b21b748728ad39668cfe722ec P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:20:59.271021 20678 raft_consensus.cc:3060] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:20:59.272593 19861 leader_election.cc:290] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:59.239971 20662 consensus_queue.cc:1059] T 23f82224f4564fea93572e8206bad1fe P cb82cc7bd6db46d980c75237a1402860 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.036s
I20260709 13:20:59.279626 16190 leader_election.cc:304] T 18ad7e3af5504571a69d10c00750a01c P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:20:59.280290 20699 leader_election.cc:290] T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 14 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:59.281199 16190 leader_election.cc:304] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860
I20260709 13:20:59.282665 20577 raft_consensus.cc:2749] T 18ad7e3af5504571a69d10c00750a01c P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:20:59.283416 20577 raft_consensus.cc:2749] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:20:59.284034 16220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ab0ac8347641e6955e9bce7d90c1a2" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:59.284255 20388 leader_election.cc:290] T 1df93779209748b9860f6d893dd3de53 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:59.284032 20577 consensus_queue.cc:1048] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.002s
I20260709 13:20:59.286705 20601 leader_election.cc:290] T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 13 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:59.215771 20763 raft_consensus.cc:2804] T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:20:59.215423 20604 consensus_queue.cc:1048] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.026s
I20260709 13:20:59.288117 20763 raft_consensus.cc:493] T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:59.288655 20763 raft_consensus.cc:3060] T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:20:59.289407 16148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62fe3346cd3c42c4a55e14af9073bca4" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:59.261327 20518 consensus_queue.cc:237] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:20:59.291806 20764 log.cc:927] Time spent T 87ac0d1f7e8a419f9bcbc04aee0e067e P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.073s	user 0.000s	sys 0.000s
I20260709 13:20:59.296629 20293 leader_election.cc:290] T b6f07842d3aa47858df5665874c86f3a P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 14 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:59.298190 20763 raft_consensus.cc:515] T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:59.254693 20603 raft_consensus.cc:515] T 5a96373bb0e14dce8a266b2fc6c58507 P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:59.300374 20763 leader_election.cc:290] T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 14 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:20:59.296397 20640 leader_election.cc:290] T fffab26427e24d398e2695aad40ca134 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 10 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:20:59.301374 20603 leader_election.cc:290] T 5a96373bb0e14dce8a266b2fc6c58507 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 11 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
W20260709 13:20:59.301986 16114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:59.304705 16116 leader_election.cc:304] T 84f2da98864b4425bf7c0055951fdd4b P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:20:59.305919 16116 leader_election.cc:304] T 68a6838d82ef4cac83507f906139d90f P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
I20260709 13:20:59.298892 20618 leader_election.cc:290] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 13 election: Requested vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:59.308159 20750 raft_consensus.cc:515] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:59.309137 20760 consensus_queue.cc:1059] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:20:59.319797 20760 raft_consensus.cc:3055] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:20:59.320225 20760 raft_consensus.cc:740] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:20:59.321218 20760 consensus_queue.cc:260] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:59.322714 20760 raft_consensus.cc:3060] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:20:59.310570 20685 consensus_queue.cc:1048] T c0eab25ce4c64e018a041beeda2867db P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:20:59.312408 20640 raft_consensus.cc:493] T ada5a004ba7c46819051e3ccec71ffdb P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:59.323752 20640 raft_consensus.cc:515] T ada5a004ba7c46819051e3ccec71ffdb P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:59.325874 20640 leader_election.cc:290] T ada5a004ba7c46819051e3ccec71ffdb P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:59.296630 20475 raft_consensus.cc:515] T 829b4ad8c5a54eababeea1181b960d4b P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:59.327054 16142 raft_consensus.cc:2393] T 829b4ad8c5a54eababeea1181b960d4b P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 12: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
I20260709 13:20:59.328219 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdbd78acb4cf46298e0de8ba0d130e19" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:20:59.290395 20751 raft_consensus.cc:515] T c0eab25ce4c64e018a041beeda2867db P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:20:59.313347 16266 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:59.313746 20661 raft_consensus.cc:493] T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:59.329932 20661 raft_consensus.cc:515] T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:59.331463 16145 raft_consensus.cc:2393] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 13: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
I20260709 13:20:59.215979 16790 leader_election.cc:304] T 74ca8140f35c4ae29e8a664f6bdfc829 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:20:59.332829 16295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f5f89186204242bf377f7c42b2e2da" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:59.333705 16295 raft_consensus.cc:2468] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 12.
I20260709 13:20:59.314177 20573 raft_consensus.cc:493] T 66ae176961b74813a58bc15c9c4c6f2a P 90f3da57796b488c9087a2f523adf045 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:59.313180 16222 raft_consensus.cc:1240] T b6f07842d3aa47858df5665874c86f3a P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 13. Current term is 14. Ops: []
I20260709 13:20:59.332828 16116 leader_election.cc:304] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:20:59.337740 16298 raft_consensus.cc:1275] T 8158bbd17b994d2eb96f560bd0f1b88d P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 13:20:59.331488 16292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e4bfb9090c8460eb14fa7e0d818e7f1" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
W20260709 13:20:59.343509 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 127188us
I20260709 13:20:59.344705 16790 leader_election.cc:304] T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:20:59.216775 20607 consensus_queue.cc:1059] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.131s
I20260709 13:20:59.318462 20678 raft_consensus.cc:515] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:59.347652 16790 leader_election.cc:304] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 647a343a542d4a139fe106c19095129b
I20260709 13:20:59.352319 16151 raft_consensus.cc:1240] T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 12. Current term is 13. Ops: []
I20260709 13:20:59.353947 16147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5d59407eb545b895d8d922e0dd69fd" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 14 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:20:59.342082 16218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d4ab870a05347868956aa8b841b2981" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:59.348266 20573 raft_consensus.cc:515] T 66ae176961b74813a58bc15c9c4c6f2a P 90f3da57796b488c9087a2f523adf045 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:59.339254 20236 raft_consensus.cc:2749] T 84f2da98864b4425bf7c0055951fdd4b P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:20:59.362794 20604 raft_consensus.cc:2804] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:20:59.346189 20763 raft_consensus.cc:3060] T 74ca8140f35c4ae29e8a664f6bdfc829 P cb82cc7bd6db46d980c75237a1402860 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:20:59.364849 20751 leader_election.cc:290] T c0eab25ce4c64e018a041beeda2867db P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 12 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:20:59.365011 20678 leader_election.cc:290] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 16 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:59.365818 16148 raft_consensus.cc:2393] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 16: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
I20260709 13:20:59.368062 16145 raft_consensus.cc:1240] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 15. Current term is 16. Ops: [15.11-15.11]
I20260709 13:20:59.368906 16146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "829b4ad8c5a54eababeea1181b960d4b" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:20:59.369911 16146 raft_consensus.cc:2393] T 829b4ad8c5a54eababeea1181b960d4b P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 in current term 12: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
W20260709 13:20:59.370743 16788 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:59.347580 20629 raft_consensus.cc:2804] T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:20:59.374473 16146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f5f89186204242bf377f7c42b2e2da" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:59.346014 20388 raft_consensus.cc:2804] T 68a6838d82ef4cac83507f906139d90f P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 13:20:59.376235 16791 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:59.377660 16146 raft_consensus.cc:2468] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 12.
I20260709 13:20:59.379104 16192 leader_election.cc:304] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:20:59.370649 16218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cc959fb7f704fba9009c0595467d074" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:20:59.386530 16218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fffab26427e24d398e2695aad40ca134" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:59.348654 20699 raft_consensus.cc:2749] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:20:59.360404 20662 raft_consensus.cc:493] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:20:59.392901 16146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c61a39fd4f84424b129c1a6f9ba404d" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:20:59.395618 16218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66ae176961b74813a58bc15c9c4c6f2a" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:59.401548 16266 leader_election.cc:304] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 647a343a542d4a139fe106c19095129b
I20260709 13:20:59.398409 16218 raft_consensus.cc:2468] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 14.
I20260709 13:20:59.407488 16218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a96373bb0e14dce8a266b2fc6c58507" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:20:59.408058 16218 raft_consensus.cc:3060] T 5a96373bb0e14dce8a266b2fc6c58507 P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:20:59.408437 16146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a878999c0024d2abe517484180290fc" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:20:59.408617 16294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3884f864a96d428785895e5463e5728f" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:20:59.387337 20388 raft_consensus.cc:493] T 68a6838d82ef4cac83507f906139d90f P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:59.374892 20427 consensus_queue.cc:1059] T 8a878999c0024d2abe517484180290fc P 647a343a542d4a139fe106c19095129b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.035s
I20260709 13:20:59.373565 20755 consensus_queue.cc:1048] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.028s
I20260709 13:20:59.371997 20757 consensus_queue.cc:1048] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.048s
I20260709 13:20:59.389451 20618 raft_consensus.cc:2804] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:20:59.418171 16293 raft_consensus.cc:1275] T d2ab0ac8347641e6955e9bce7d90c1a2 P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:20:59.392372 20629 raft_consensus.cc:697] T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860 [term 13 LEADER]: Becoming Leader. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:20:59.386864 20762 consensus_queue.cc:1059] T e5ad4eb281564936a59f41e387a47156 P 647a343a542d4a139fe106c19095129b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.034s
W20260709 13:20:59.423715 16789 connection.cc:256] RPC call timeout handler was delayed by -1.5007107349999842s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1718s for a response
I20260709 13:20:59.419916 16265 leader_election.cc:304] T 66ae176961b74813a58bc15c9c4c6f2a P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:20:59.410782 20644 raft_consensus.cc:2804] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:20:59.390229 20607 consensus_queue.cc:1059] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: INVALID_TERM, Last received: 12.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.031s
I20260709 13:20:59.441030 16217 raft_consensus.cc:2410] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 for term 13 because replica has last-logged OpId of term: 12 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 10.
I20260709 13:20:59.416096 20388 raft_consensus.cc:3060] T 68a6838d82ef4cac83507f906139d90f P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:20:59.391861 20472 consensus_queue.cc:1048] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.039s
I20260709 13:20:59.449465 16294 raft_consensus.cc:1240] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 90f3da57796b488c9087a2f523adf045 [term 10 LEADER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 9. Current term is 10. Ops: []
I20260709 13:20:59.456321 16218 raft_consensus.cc:2468] T 5a96373bb0e14dce8a266b2fc6c58507 P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 12.
I20260709 13:20:59.422675 20618 raft_consensus.cc:493] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:59.459854 16115 leader_election.cc:304] T 5a959dbecc2648eb85d1cc550f5a3245 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:20:59.461491 16116 leader_election.cc:304] T 5a96373bb0e14dce8a266b2fc6c58507 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
I20260709 13:20:59.410130 20662 raft_consensus.cc:515] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:59.462594 16218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fffab26427e24d398e2695aad40ca134" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:20:59.466657 16293 raft_consensus.cc:1240] T e5ad4eb281564936a59f41e387a47156 P 90f3da57796b488c9087a2f523adf045 [term 13 LEADER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 12. Current term is 13. Ops: []
I20260709 13:20:59.468668 16218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0eab25ce4c64e018a041beeda2867db" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:20:59.468772 16115 leader_election.cc:304] T 18ad7e3af5504571a69d10c00750a01c P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:20:59.474260 16115 leader_election.cc:304] T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac
I20260709 13:20:59.479686 16218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1df93779209748b9860f6d893dd3de53" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:59.483660 16218 raft_consensus.cc:2393] T 1df93779209748b9860f6d893dd3de53 P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 12: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
I20260709 13:20:59.484863 20784 raft_consensus.cc:493] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:59.485181 16218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a96373bb0e14dce8a266b2fc6c58507" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:20:59.485759 20236 raft_consensus.cc:2749] T 5a959dbecc2648eb85d1cc550f5a3245 P 647a343a542d4a139fe106c19095129b [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:20:59.445051 20640 raft_consensus.cc:2804] T 66ae176961b74813a58bc15c9c4c6f2a P 90f3da57796b488c9087a2f523adf045 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:20:59.446509 20573 leader_election.cc:290] T 66ae176961b74813a58bc15c9c4c6f2a P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:59.492313 16218 raft_consensus.cc:2368] T 5a96373bb0e14dce8a266b2fc6c58507 P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 for earlier term 11. Current term is 12.
I20260709 13:20:59.457633 20760 raft_consensus.cc:493] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:59.498519 16218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada5a004ba7c46819051e3ccec71ffdb" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:59.498936 16265 leader_election.cc:400] T 5a96373bb0e14dce8a266b2fc6c58507 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 11 election: Vote denied by peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943) with higher term. Message: Invalid argument: T 5a96373bb0e14dce8a266b2fc6c58507 P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 for earlier term 11. Current term is 12.
I20260709 13:20:59.443917 20644 raft_consensus.cc:493] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 647a343a542d4a139fe106c19095129b)
W20260709 13:20:59.499028 16189 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:59.499377 16265 leader_election.cc:403] T 5a96373bb0e14dce8a266b2fc6c58507 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 11 election: Cancelling election due to peer responding with higher term
I20260709 13:20:59.460547 20762 raft_consensus.cc:493] T 434bff3082d141838068f323bdb0d1ed P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:59.467152 20618 raft_consensus.cc:3060] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:20:59.505177 20602 consensus_queue.cc:1059] T b6f07842d3aa47858df5665874c86f3a P 647a343a542d4a139fe106c19095129b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.007s
I20260709 13:20:59.501819 20784 raft_consensus.cc:515] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:59.512892 20602 raft_consensus.cc:3055] T b6f07842d3aa47858df5665874c86f3a P 647a343a542d4a139fe106c19095129b [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:20:59.513334 20602 raft_consensus.cc:740] T b6f07842d3aa47858df5665874c86f3a P 647a343a542d4a139fe106c19095129b [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:20:59.513969 16219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62fe3346cd3c42c4a55e14af9073bca4" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 16 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:20:59.514617 20784 leader_election.cc:290] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:59.508186 20583 raft_consensus.cc:3060] T 5a96373bb0e14dce8a266b2fc6c58507 P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:20:59.505340 20644 raft_consensus.cc:3060] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:20:59.515268 20691 leader_election.cc:290] T 829b4ad8c5a54eababeea1181b960d4b P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:59.514623 16219 raft_consensus.cc:3060] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:20:59.514211 20602 consensus_queue.cc:260] T b6f07842d3aa47858df5665874c86f3a P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:59.516374 16221 raft_consensus.cc:1275] T d2ab0ac8347641e6955e9bce7d90c1a2 P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:20:59.470935 20678 raft_consensus.cc:2804] T 5a96373bb0e14dce8a266b2fc6c58507 P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:20:59.487317 20607 raft_consensus.cc:493] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:59.492440 20746 raft_consensus.cc:2749] T 18ad7e3af5504571a69d10c00750a01c P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:20:59.522758 16818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434bff3082d141838068f323bdb0d1ed" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:59.523787 16818 raft_consensus.cc:2393] T 434bff3082d141838068f323bdb0d1ed P cb82cc7bd6db46d980c75237a1402860 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 11: Already voted for candidate cb82cc7bd6db46d980c75237a1402860 in this term.
I20260709 13:20:59.524544 16219 raft_consensus.cc:2468] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 16.
I20260709 13:20:59.524648 16822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57a744f5aaf944678a63469cbf514015" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:59.525568 16820 raft_consensus.cc:1240] T 9e65676b21b748728ad39668cfe722ec P cb82cc7bd6db46d980c75237a1402860 [term 11 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 10. Current term is 11. Ops: []
I20260709 13:20:59.498374 20640 raft_consensus.cc:493] T 66ae176961b74813a58bc15c9c4c6f2a P 90f3da57796b488c9087a2f523adf045 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:59.504370 20755 raft_consensus.cc:2804] T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:20:59.529318 16817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 14 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:20:59.525610 16219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6f07842d3aa47858df5665874c86f3a" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:20:59.530016 16116 leader_election.cc:304] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
I20260709 13:20:59.531661 16821 raft_consensus.cc:3055] T 8638ee363273426db1393dd81e5c7b72 P cb82cc7bd6db46d980c75237a1402860 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:20:59.534829 16817 raft_consensus.cc:3060] T 19baa7bc367844c288bca8f81439c413 P cb82cc7bd6db46d980c75237a1402860 [term 12 FOLLOWER]: Advancing to term 14
W20260709 13:20:59.536156 16264 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:59.536772 16818 raft_consensus.cc:1275] T c9944d5e4b8c4109af9955c4f08c9095 P cb82cc7bd6db46d980c75237a1402860 [term 12 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:20:59.535145 16821 raft_consensus.cc:740] T 8638ee363273426db1393dd81e5c7b72 P cb82cc7bd6db46d980c75237a1402860 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:20:59.539888 16818 raft_consensus.cc:1275] T b87147731a68400e8cce9b879daa7c5d P cb82cc7bd6db46d980c75237a1402860 [term 13 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 13:20:59.540952 16818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66f75124215f4ca6bef831e174a792d8" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:59.508333 20604 raft_consensus.cc:697] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860 [term 11 LEADER]: Becoming Leader. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:20:59.538458 16821 consensus_queue.cc:260] T 8638ee363273426db1393dd81e5c7b72 P cb82cc7bd6db46d980c75237a1402860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:59.545711 16821 raft_consensus.cc:3060] T 8638ee363273426db1393dd81e5c7b72 P cb82cc7bd6db46d980c75237a1402860 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:20:59.545953 16219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f5f89186204242bf377f7c42b2e2da" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:20:59.544411 16824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdbd78acb4cf46298e0de8ba0d130e19" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:20:59.520401 20784 raft_consensus.cc:493] T 806d7690c2b644959152074bb2cab0a7 P cb82cc7bd6db46d980c75237a1402860 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:20:59.516448 20475 leader_election.cc:290] T 829b4ad8c5a54eababeea1181b960d4b P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 12 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:59.509816 20760 raft_consensus.cc:515] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:20:59.547348 20771 log.cc:927] Time spent T e5ad4eb281564936a59f41e387a47156 P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.077s	user 0.001s	sys 0.000s
I20260709 13:20:59.548415 16816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 13 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:20:59.548756 16825 raft_consensus.cc:1240] T 434bff3082d141838068f323bdb0d1ed P cb82cc7bd6db46d980c75237a1402860 [term 11 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 10. Current term is 11. Ops: [10.10-10.10]
I20260709 13:20:59.550923 16819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada5a004ba7c46819051e3ccec71ffdb" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:59.551807 16820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ab0ac8347641e6955e9bce7d90c1a2" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:59.552619 16823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e65676b21b748728ad39668cfe722ec" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:20:59.554036 16822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87147731a68400e8cce9b879daa7c5d" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:59.554890 16817 raft_consensus.cc:2468] T 19baa7bc367844c288bca8f81439c413 P cb82cc7bd6db46d980c75237a1402860 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 14.
I20260709 13:20:59.556375 16824 raft_consensus.cc:3060] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:20:59.542151 16818 raft_consensus.cc:2410] T 66f75124215f4ca6bef831e174a792d8 P cb82cc7bd6db46d980c75237a1402860 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b for term 12 because replica has last-logged OpId of term: 11 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 9.
I20260709 13:20:59.534387 20755 raft_consensus.cc:493] T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:59.534094 20640 raft_consensus.cc:3060] T 66ae176961b74813a58bc15c9c4c6f2a P 90f3da57796b488c9087a2f523adf045 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:20:59.537340 20661 leader_election.cc:290] T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:20:59.517263 20762 raft_consensus.cc:515] T 434bff3082d141838068f323bdb0d1ed P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:59.558923 16146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31969392ccdb43b491911a319e8bc7e1" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:20:59.533701 20607 raft_consensus.cc:515] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:59.532809 20629 consensus_queue.cc:237] T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:20:59.786624 16791 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:20:59.948523 16791 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:59.953117 16822 raft_consensus.cc:2468] T b87147731a68400e8cce9b879daa7c5d P cb82cc7bd6db46d980c75237a1402860 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 13.
I20260709 13:20:59.946914 16219 raft_consensus.cc:2468] T c2f5f89186204242bf377f7c42b2e2da P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 12.
I20260709 13:20:59.954797 16219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4f65cdf00b748bb963ee6e81ef43b79" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:20:59.768613 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "829b4ad8c5a54eababeea1181b960d4b" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:20:59.962209 16217 raft_consensus.cc:3060] T 829b4ad8c5a54eababeea1181b960d4b P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:20:59.575861 16788 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:20:59.966959 16788 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:20:59.967653 16788 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:59.951429 16816 raft_consensus.cc:2368] T 19baa7bc367844c288bca8f81439c413 P cb82cc7bd6db46d980c75237a1402860 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b for earlier term 13. Current term is 14.
I20260709 13:20:59.680439 16821 raft_consensus.cc:1275] T 8638ee363273426db1393dd81e5c7b72 P cb82cc7bd6db46d980c75237a1402860 [term 13 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:20:59.955281 16265 leader_election.cc:304] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
W20260709 13:20:59.818642 16790 leader_election.cc:336] T d2ab0ac8347641e6955e9bce7d90c1a2 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943): Timed out: RequestConsensusVote RPC to 127.14.177.130:35943 timed out after 1.515s (SENT)
W20260709 13:20:59.970568 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 152043us
W20260709 13:20:59.771142 16224 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57600 (request call id 1496) took 1848 ms (client timeout 2178 ms). Trace:
I20260709 13:20:59.950765 16823 raft_consensus.cc:2393] T 9e65676b21b748728ad39668cfe722ec P cb82cc7bd6db46d980c75237a1402860 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 11: Already voted for candidate cb82cc7bd6db46d980c75237a1402860 in this term.
I20260709 13:20:59.965848 16219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 14 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:20:59.973851 16219 raft_consensus.cc:3060] T 19baa7bc367844c288bca8f81439c413 P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:21:00.018345 16189 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:20:59.972357 16224 rpcz_store.cc:269] 0709 13:20:57.922898 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:57.923318 (+   420us) service_pool.cc:224] Handling call
0709 13:20:59.770900 (+1847582us) spinlock_profiling.cc:243] Waited 257 ms on lock 0x7b54002986d4. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:20:59.771055 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":591782784}
I20260709 13:20:59.860384 20583 raft_consensus.cc:2749] T 5a96373bb0e14dce8a266b2fc6c58507 P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Leader election lost for term 11. Reason: Vote denied by peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943) with higher term. Message: Invalid argument: T 5a96373bb0e14dce8a266b2fc6c58507 P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 for earlier term 11. Current term is 12.
I20260709 13:21:00.035390 16817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee9ed67006144797b1de6ea8c2013a68" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:21:00.036638 16817 raft_consensus.cc:2410] T ee9ed67006144797b1de6ea8c2013a68 P cb82cc7bd6db46d980c75237a1402860 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b for term 10 because replica has last-logged OpId of term: 9 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 8.
I20260709 13:21:00.036531 16821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2372981006d4b218215c28964dea623" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
W20260709 13:21:00.035058 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 79937.7us
W20260709 13:21:00.038163 16265 leader_election.cc:336] T e4dc6b0541ca431ebc757beac00521f1 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943): Timed out: RequestConsensusVote RPC to 127.14.177.130:35943 timed out after 2.178s (SENT)
I20260709 13:21:00.038637 16265 leader_election.cc:304] T e4dc6b0541ca431ebc757beac00521f1 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:21:00.041231 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f5f89186204242bf377f7c42b2e2da" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:21:00.044025 16817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5718c623ebee4d2bb770f0f9ddb2f164" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 13 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:00.045284 16821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f82224f4564fea93572e8206bad1fe" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:59.637965 20750 leader_election.cc:290] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 12 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:59.672544 20783 raft_consensus.cc:493] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:59.599392 20518 raft_consensus.cc:493] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:00.055404 16816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f82224f4564fea93572e8206bad1fe" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:21:00.057415 16825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af57996a8be84d398f0f023e8d08ef77" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:00.063814 16825 raft_consensus.cc:2393] T af57996a8be84d398f0f023e8d08ef77 P cb82cc7bd6db46d980c75237a1402860 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 13: Already voted for candidate cb82cc7bd6db46d980c75237a1402860 in this term.
I20260709 13:21:00.043465 20603 raft_consensus.cc:2804] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:21:00.041301 16818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7ca61592e848ea9b0fb81a2ce33cdc" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
W20260709 13:21:00.072768 16114 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.501s	user 0.006s	sys 0.010s
W20260709 13:21:00.073217 16114 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.502s	user 0.006s	sys 0.010s
I20260709 13:21:00.073272 16818 raft_consensus.cc:3060] T 3a7ca61592e848ea9b0fb81a2ce33cdc P cb82cc7bd6db46d980c75237a1402860 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:20:59.769841 20780 raft_consensus.cc:493] T 2794af5b99984efebdee564035841cb0 P cb82cc7bd6db46d980c75237a1402860 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:00.019901 20755 raft_consensus.cc:3060] T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:21:00.058068 16823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31969392ccdb43b491911a319e8bc7e1" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:00.080904 16823 raft_consensus.cc:2468] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 11.
I20260709 13:21:00.058830 16822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "829b4ad8c5a54eababeea1181b960d4b" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:20:59.618350 20663 raft_consensus.cc:493] T a378d7f796f94615b3b347119a9880fd P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:59.824337 20599 raft_consensus.cc:493] T 7c2095a5f4ad49408dbc7f69c606120d P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:20:59.766039 20573 raft_consensus.cc:493] T 8c11074579b046d99e585c2818e18a2c P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:21:00.051551 20583 raft_consensus.cc:3060] T e4dc6b0541ca431ebc757beac00521f1 P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:20:59.896049 20434 raft_consensus.cc:493] T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:00.075933 16217 raft_consensus.cc:2468] T 829b4ad8c5a54eababeea1181b960d4b P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 12.
I20260709 13:21:00.052979 20782 raft_consensus.cc:493] T ada5a004ba7c46819051e3ccec71ffdb P cb82cc7bd6db46d980c75237a1402860 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:00.100317 16116 leader_election.cc:304] T 829b4ad8c5a54eababeea1181b960d4b P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
I20260709 13:21:00.102088 16816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66ae176961b74813a58bc15c9c4c6f2a" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 15 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:00.093225 20267 raft_consensus.cc:493] T 23f82224f4564fea93572e8206bad1fe P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
W20260709 13:21:00.090883 20763 consensus_meta.cc:287] T 74ca8140f35c4ae29e8a664f6bdfc829 P cb82cc7bd6db46d980c75237a1402860: Time spent flushing consensus metadata: real 0.727s	user 0.011s	sys 0.000s
I20260709 13:21:00.106719 16823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 12 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:00.108292 16823 raft_consensus.cc:2410] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac for term 12 because replica has last-logged OpId of term: 10 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 13:20:59.547451 20604 consensus_queue.cc:237] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:00.096225 16822 raft_consensus.cc:2393] T 829b4ad8c5a54eababeea1181b960d4b P cb82cc7bd6db46d980c75237a1402860 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 12: Already voted for candidate cb82cc7bd6db46d980c75237a1402860 in this term.
I20260709 13:21:00.072803 20603 raft_consensus.cc:493] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:00.111910 16825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62fe3346cd3c42c4a55e14af9073bca4" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 16 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:00.122802 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ad5dbfa99ee4b8d9e28be92b3f36db5" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:00.129606 16217 raft_consensus.cc:2468] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 10.
W20260709 13:21:00.122835 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 151289us
I20260709 13:21:00.125164 16299 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48674 (request call id 2723) took 2152 ms. Trace:
I20260709 13:21:00.109615 20750 raft_consensus.cc:2804] T 829b4ad8c5a54eababeea1181b960d4b P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:21:00.069507 20783 raft_consensus.cc:515] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:00.104753 20434 raft_consensus.cc:515] T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:00.114236 20763 raft_consensus.cc:2749] T 74ca8140f35c4ae29e8a664f6bdfc829 P cb82cc7bd6db46d980c75237a1402860 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:21:00.109256 20782 raft_consensus.cc:515] T ada5a004ba7c46819051e3ccec71ffdb P cb82cc7bd6db46d980c75237a1402860 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:21:00.144726 20776 log.cc:927] Time spent T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.576s	user 0.004s	sys 0.000s
I20260709 13:21:00.149518 16816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e4bfb9090c8460eb14fa7e0d818e7f1" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:00.071012 20518 raft_consensus.cc:515] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:20:59.777806 16264 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:00.153748 16217 raft_consensus.cc:1275] T 1df93779209748b9860f6d893dd3de53 P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:21:00.118469 20673 raft_consensus.cc:493] T e4dc6b0541ca431ebc757beac00521f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:00.108741 20573 raft_consensus.cc:515] T 8c11074579b046d99e585c2818e18a2c P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:20:59.821444 16789 leader_election.cc:304] T 3a7ca61592e848ea9b0fb81a2ce33cdc P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac
I20260709 13:21:00.104234 20599 raft_consensus.cc:515] T 7c2095a5f4ad49408dbc7f69c606120d P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:00.162051 16146 raft_consensus.cc:2393] T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 in current term 12: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
I20260709 13:21:00.177865 16217 raft_consensus.cc:1240] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 12. Current term is 13. Ops: []
I20260709 13:20:59.555657 20602 raft_consensus.cc:3060] T b6f07842d3aa47858df5665874c86f3a P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:21:00.095196 20780 raft_consensus.cc:515] T 2794af5b99984efebdee564035841cb0 P cb82cc7bd6db46d980c75237a1402860 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:21:00.161291 16264 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:00.160072 20655 raft_consensus.cc:493] T 8158bbd17b994d2eb96f560bd0f1b88d P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:00.163625 20657 raft_consensus.cc:493] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:21:00.113960 20267 raft_consensus.cc:515] T 23f82224f4564fea93572e8206bad1fe P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:00.175264 16817 raft_consensus.cc:2410] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac for term 13 because replica has last-logged OpId of term: 11 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 10.
I20260709 13:21:00.148285 20603 raft_consensus.cc:3060] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:21:00.177821 16789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 356476us
I20260709 13:21:00.203552 16823 raft_consensus.cc:1240] T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860 [term 14 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 13. Current term is 14. Ops: []
I20260709 13:21:00.203810 16789 leader_election.cc:304] T 9e65676b21b748728ad39668cfe722ec P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:21:00.162551 16822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "829b4ad8c5a54eababeea1181b960d4b" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:00.204897 16822 raft_consensus.cc:2393] T 829b4ad8c5a54eababeea1181b960d4b P cb82cc7bd6db46d980c75237a1402860 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 12: Already voted for candidate cb82cc7bd6db46d980c75237a1402860 in this term.
I20260709 13:21:00.205045 16789 leader_election.cc:304] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 647a343a542d4a139fe106c19095129b
W20260709 13:21:00.169570 16790 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:00.178825 16825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9944d5e4b8c4109af9955c4f08c9095" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:21:00.180081 16221 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33050 (request call id 781) took 1221 ms. Trace:
I20260709 13:20:59.659802 16222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74ca8140f35c4ae29e8a664f6bdfc829" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:00.181325 16218 raft_consensus.cc:2468] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 12.
I20260709 13:21:00.206386 16221 rpcz_store.cc:276] 0709 13:20:58.958716 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:59.242602 (+283886us) service_pool.cc:224] Handling call
0709 13:20:59.516032 (+273430us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:59.520732 (+  4700us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:00.179890 (+659158us) spinlock_profiling.cc:243] Waited 237 ms on lock 0x7b5400245750. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5e8ad 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:21:00.180020 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":545583360}
I20260709 13:21:00.207767 16222 raft_consensus.cc:2468] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 12.
I20260709 13:21:00.208387 16825 raft_consensus.cc:3060] T c9944d5e4b8c4109af9955c4f08c9095 P cb82cc7bd6db46d980c75237a1402860 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:21:00.182195 16820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31969392ccdb43b491911a319e8bc7e1" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:21:00.183007 16217 raft_consensus.cc:1275] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:20:59.801513 20662 leader_election.cc:290] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:20:59.556591 20678 raft_consensus.cc:697] T 5a96373bb0e14dce8a266b2fc6c58507 P 647a343a542d4a139fe106c19095129b [term 12 LEADER]: Becoming Leader. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:20:59.786123 20779 raft_consensus.cc:493] T 26354361e9b044cea1c2b86c4882fe92 P cb82cc7bd6db46d980c75237a1402860 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:59.564106 20236 raft_consensus.cc:2804] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:21:00.231575 20236 raft_consensus.cc:697] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [term 16 LEADER]: Becoming Leader. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:21:00.232810 20699 raft_consensus.cc:3055] T 8a878999c0024d2abe517484180290fc P 647a343a542d4a139fe106c19095129b [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:21:00.232553 20236 consensus_queue.cc:237] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:00.233270 20699 raft_consensus.cc:740] T 8a878999c0024d2abe517484180290fc P 647a343a542d4a139fe106c19095129b [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:21:00.233829 20267 leader_election.cc:290] T 23f82224f4564fea93572e8206bad1fe P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:00.104002 20663 raft_consensus.cc:515] T a378d7f796f94615b3b347119a9880fd P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:00.191869 20782 leader_election.cc:290] T ada5a004ba7c46819051e3ccec71ffdb P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:00.191726 16192 leader_election.cc:304] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860
I20260709 13:21:00.193617 20518 leader_election.cc:290] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:00.193758 20607 leader_election.cc:290] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:21:00.194144 20434 leader_election.cc:290] T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
W20260709 13:21:00.197540 20644 consensus_meta.cc:287] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045: Time spent flushing consensus metadata: real 0.682s	user 0.008s	sys 0.001s
I20260709 13:21:00.197993 20655 raft_consensus.cc:515] T 8158bbd17b994d2eb96f560bd0f1b88d P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:00.212374 20778 raft_consensus.cc:493] T 434bff3082d141838068f323bdb0d1ed P cb82cc7bd6db46d980c75237a1402860 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:00.191695 16265 leader_election.cc:304] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:21:00.180757 16819 raft_consensus.cc:2468] T ada5a004ba7c46819051e3ccec71ffdb P cb82cc7bd6db46d980c75237a1402860 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 12.
I20260709 13:21:00.209604 16820 raft_consensus.cc:3060] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:21:00.183418 16114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:00.174400 20750 raft_consensus.cc:697] T 829b4ad8c5a54eababeea1181b960d4b P 647a343a542d4a139fe106c19095129b [term 12 LEADER]: Becoming Leader. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:21:00.180124 20762 leader_election.cc:290] T 434bff3082d141838068f323bdb0d1ed P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
W20260709 13:21:00.210987 20427 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.567s	user 0.002s	sys 0.007s
I20260709 13:21:00.190392 20576 raft_consensus.cc:3055] T 02838c86531d4f98bf58d1c22b8baf77 P 90f3da57796b488c9087a2f523adf045 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:21:00.268673 20797 raft_consensus.cc:2749] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:21:00.271765 16824 raft_consensus.cc:2410] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 for term 13 because replica has last-logged OpId of term: 12 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 10.
I20260709 13:21:00.273527 16219 raft_consensus.cc:2468] T 19baa7bc367844c288bca8f81439c413 P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 14.
I20260709 13:21:00.274914 16818 raft_consensus.cc:2468] T 3a7ca61592e848ea9b0fb81a2ce33cdc P cb82cc7bd6db46d980c75237a1402860 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 15.
W20260709 13:21:00.311616 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 119994us
I20260709 13:21:00.279481 16224 raft_consensus.cc:2468] T c2f5f89186204242bf377f7c42b2e2da P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 12.
I20260709 13:21:00.212078 20763 raft_consensus.cc:2749] T 9e65676b21b748728ad39668cfe722ec P cb82cc7bd6db46d980c75237a1402860 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:21:00.234246 20699 consensus_queue.cc:260] T 8a878999c0024d2abe517484180290fc P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:00.177735 20673 raft_consensus.cc:515] T e4dc6b0541ca431ebc757beac00521f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:20:59.602820 20784 raft_consensus.cc:515] T 806d7690c2b644959152074bb2cab0a7 P cb82cc7bd6db46d980c75237a1402860 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:00.215519 20785 raft_consensus.cc:2804] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:21:00.289675 16145 raft_consensus.cc:1240] T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 12. Current term is 13. Ops: []
I20260709 13:21:00.212558 20657 raft_consensus.cc:515] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:21:00.307487 20427 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.666s	user 0.003s	sys 0.007s
W20260709 13:21:00.297487 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 86105.8us
I20260709 13:21:00.299417 16146 raft_consensus.cc:1240] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 647a343a542d4a139fe106c19095129b [term 10 FOLLOWER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 9. Current term is 10. Ops: []
I20260709 13:21:00.280669 20667 consensus_queue.cc:1059] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.028s
W20260709 13:21:00.201505 20777 log.cc:927] Time spent T 5a959dbecc2648eb85d1cc550f5a3245 P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 0.720s	user 0.000s	sys 0.009s
I20260709 13:21:00.311878 16300 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48684 (request call id 2563) took 1773 ms. Trace:
I20260709 13:21:00.314098 16116 leader_election.cc:304] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
I20260709 13:21:00.320355 20599 leader_election.cc:290] T 7c2095a5f4ad49408dbc7f69c606120d P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:21:00.306901 20576 raft_consensus.cc:740] T 02838c86531d4f98bf58d1c22b8baf77 P 90f3da57796b488c9087a2f523adf045 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:21:00.323247 20518 consensus_queue.cc:1048] T 1df93779209748b9860f6d893dd3de53 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:21:00.323835 20721 raft_consensus.cc:2804] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 13:21:00.325248 16789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 118920us
I20260709 13:21:00.260828 20678 consensus_queue.cc:237] T 5a96373bb0e14dce8a266b2fc6c58507 P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 10.5, Last appended by leader: 5, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:00.325995 16825 raft_consensus.cc:2468] T c9944d5e4b8c4109af9955c4f08c9095 P cb82cc7bd6db46d980c75237a1402860 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 13.
I20260709 13:21:00.298651 20434 raft_consensus.cc:493] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:21:00.300148 20388 consensus_meta.cc:287] T 68a6838d82ef4cac83507f906139d90f P 647a343a542d4a139fe106c19095129b: Time spent flushing consensus metadata: real 0.835s	user 0.008s	sys 0.007s
I20260709 13:21:00.345176 20576 consensus_queue.cc:260] T 02838c86531d4f98bf58d1c22b8baf77 P 90f3da57796b488c9087a2f523adf045 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:00.423635 20576 raft_consensus.cc:3060] T 02838c86531d4f98bf58d1c22b8baf77 P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:21:00.345947 20784 leader_election.cc:290] T 806d7690c2b644959152074bb2cab0a7 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:00.346166 20754 raft_consensus.cc:2804] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:21:00.427682 20754 raft_consensus.cc:697] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b [term 12 LEADER]: Becoming Leader. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:21:00.429067 20646 raft_consensus.cc:3060] T 23f82224f4564fea93572e8206bad1fe P cb82cc7bd6db46d980c75237a1402860 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:21:00.428784 20754 consensus_queue.cc:237] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:00.331743 16145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b7d1ab519044eda2d5e04a07242f25" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:00.431506 16145 raft_consensus.cc:2468] T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 10.
I20260709 13:21:00.341007 16300 rpcz_store.cc:276] 0709 13:20:58.538060 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:58.680054 (+141994us) service_pool.cc:224] Handling call
0709 13:20:58.680391 (+   337us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:58.683713 (+  3322us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:21:00.157560 (+1473847us) log.cc:844] Serialized 14 byte log entry
0709 13:21:00.311518 (+153958us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:21:00.311568 (+    50us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:21:00.311626 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:00.311695 (+    69us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:21:00.311819 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1110894,"prepare.queue_time_us":1129412,"prepare.run_cpu_time_us":33,"prepare.run_wall_time_us":33,"raft.queue_time_us":34005,"raft.run_cpu_time_us":227,"raft.run_wall_time_us":10825,"thread_start_us":447605,"threads_started":2,"wal-append.queue_time_us":117038}
I20260709 13:21:00.347868 20667 raft_consensus.cc:3055] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:21:00.349117 20755 raft_consensus.cc:515] T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:00.248934 20779 raft_consensus.cc:515] T 26354361e9b044cea1c2b86c4882fe92 P cb82cc7bd6db46d980c75237a1402860 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:00.349828 16146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806d7690c2b644959152074bb2cab0a7" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:00.349891 20763 consensus_queue.cc:1059] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: INVALID_TERM, Last received: 12.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:21:00.349879 20699 raft_consensus.cc:3060] T 8a878999c0024d2abe517484180290fc P 647a343a542d4a139fe106c19095129b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:21:00.350778 20676 consensus_queue.cc:1059] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:21:00.462199 20583 raft_consensus.cc:2749] T e4dc6b0541ca431ebc757beac00521f1 P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:21:00.313345 20787 raft_consensus.cc:2764] T 3a7ca61592e848ea9b0fb81a2ce33cdc P cb82cc7bd6db46d980c75237a1402860 [term 15 FOLLOWER]: Leader pre-election decision vote started in defunct term 14: won
I20260709 13:21:00.336073 20796 consensus_queue.cc:1059] T c9944d5e4b8c4109af9955c4f08c9095 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.037s
I20260709 13:21:00.437722 20667 raft_consensus.cc:740] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:21:00.463567 20796 raft_consensus.cc:3055] T c9944d5e4b8c4109af9955c4f08c9095 P 90f3da57796b488c9087a2f523adf045 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:21:00.463924 20796 raft_consensus.cc:740] T c9944d5e4b8c4109af9955c4f08c9095 P 90f3da57796b488c9087a2f523adf045 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:21:00.464198 20667 consensus_queue.cc:260] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.11, Last appended by leader: 11, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:00.464670 20796 consensus_queue.cc:260] T c9944d5e4b8c4109af9955c4f08c9095 P 90f3da57796b488c9087a2f523adf045 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:00.465873 20796 raft_consensus.cc:3060] T c9944d5e4b8c4109af9955c4f08c9095 P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:21:00.364135 16789 leader_election.cc:304] T 3884f864a96d428785895e5463e5728f P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:21:00.364418 16820 raft_consensus.cc:2468] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 12.
I20260709 13:21:00.285207 20644 raft_consensus.cc:515] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:00.326864 20673 leader_election.cc:290] T e4dc6b0541ca431ebc757beac00521f1 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:00.368098 16148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5718c623ebee4d2bb770f0f9ddb2f164" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:00.327191 20785 raft_consensus.cc:493] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:00.387744 20760 leader_election.cc:290] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:00.470153 16820 raft_consensus.cc:1275] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 [term 10 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:21:00.470095 20785 raft_consensus.cc:3060] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:21:00.471653 16820 raft_consensus.cc:1275] T 3a7ca61592e848ea9b0fb81a2ce33cdc P cb82cc7bd6db46d980c75237a1402860 [term 15 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 13:21:00.390246 16225 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57600 (request call id 1505) took 1116 ms. Trace:
I20260709 13:21:00.393805 20599 raft_consensus.cc:493] T 8158bbd17b994d2eb96f560bd0f1b88d P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:00.395535 20750 consensus_queue.cc:237] T 829b4ad8c5a54eababeea1181b960d4b P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:00.398245 20663 leader_election.cc:290] T a378d7f796f94615b3b347119a9880fd P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:00.404975 20657 leader_election.cc:290] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:00.301268 20778 raft_consensus.cc:515] T 434bff3082d141838068f323bdb0d1ed P cb82cc7bd6db46d980c75237a1402860 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:00.407276 20388 raft_consensus.cc:515] T 68a6838d82ef4cac83507f906139d90f P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:00.365871 20607 consensus_queue.cc:1048] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.043s
I20260709 13:21:00.410539 20655 leader_election.cc:290] T 8158bbd17b994d2eb96f560bd0f1b88d P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:00.359730 20721 raft_consensus.cc:493] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:00.416814 20780 leader_election.cc:290] T 2794af5b99984efebdee564035841cb0 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
W20260709 13:21:00.422144 20618 consensus_meta.cc:287] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac: Time spent flushing consensus metadata: real 0.900s	user 0.007s	sys 0.004s
I20260709 13:21:00.435731 16790 leader_election.cc:304] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860; no voters: 
W20260709 13:21:00.517735 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 82116.8us
W20260709 13:21:00.441341 16788 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:00.364920 20434 raft_consensus.cc:515] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:00.459275 20755 leader_election.cc:290] T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 12 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:00.468070 16146 raft_consensus.cc:2393] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 in current term 11: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
I20260709 13:21:00.390944 20573 leader_election.cc:290] T 8c11074579b046d99e585c2818e18a2c P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
W20260709 13:21:00.468317 16189 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:00.388970 20783 leader_election.cc:290] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:21:00.485031 20231 raft_consensus.cc:3055] T e5ad4eb281564936a59f41e387a47156 P 647a343a542d4a139fe106c19095129b [term 12 LEADER]: Stepping down as leader of term 12
W20260709 13:21:00.496150 16789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 132076us
I20260709 13:21:00.484440 16823 raft_consensus.cc:1275] T 26354361e9b044cea1c2b86c4882fe92 P cb82cc7bd6db46d980c75237a1402860 [term 10 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
W20260709 13:21:00.473518 16192 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 281862us
I20260709 13:21:00.513528 16148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2794af5b99984efebdee564035841cb0" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:00.473209 16225 rpcz_store.cc:276] 0709 13:20:59.273981 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:59.342843 (+ 68862us) service_pool.cc:224] Handling call
0709 13:20:59.374919 (+ 32076us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:20:59.375383 (+   464us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:59.375415 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:20:59.380407 (+  4992us) log.cc:844] Serialized 26 byte log entry
0709 13:20:59.474398 (+ 93991us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:20:59.474444 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:59.474517 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:21:00.200874 (+726357us) spinlock_profiling.cc:243] Waited 7.75 ms on lock 0x7b4400022b4c. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69ad8fadb1 00007f69ad998c85 00007f69ad9996a2 00007f69b070b1db 00007f69b0706aaf 00007f69b06fa86e 00007f69b06f9bb6 00007f69b06f97d8 00007f69b0703221 00007f69b07031d9 00007f69b0703169
0709 13:21:00.389842 (+188968us) raft_consensus.cc:1685] finished
0709 13:21:00.389875 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:21:00.390166 (+   291us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":3113,"raft.run_cpu_time_us":111,"raft.run_wall_time_us":5866,"spinlock_wait_cycles":18022016,"thread_start_us":93577,"threads_started":1,"wal-append.queue_time_us":100614}
I20260709 13:21:00.486796 16221 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57600 (request call id 1517) took 1173 ms. Trace:
I20260709 13:21:00.536998 16265 leader_election.cc:304] T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:21:00.537268 16221 rpcz_store.cc:276] 0709 13:20:59.313581 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:00.207975 (+894394us) service_pool.cc:224] Handling call
0709 13:21:00.231550 (+ 23575us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:21:00.231788 (+   238us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:21:00.260677 (+ 28889us) log.cc:844] Serialized 14 byte log entry
0709 13:21:00.486311 (+225634us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:21:00.486383 (+    72us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:21:00.486445 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:00.486535 (+    90us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:21:00.486718 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":28880,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":40,"raft.queue_time_us":1036,"raft.run_cpu_time_us":161,"raft.run_wall_time_us":15028,"thread_start_us":252474,"threads_started":2}
W20260709 13:21:00.541471 16789 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:00.543751 16146 raft_consensus.cc:1240] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [term 16 LEADER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 15. Current term is 16. Ops: [15.11-15.11]
I20260709 13:21:00.516897 20721 raft_consensus.cc:3060] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:21:00.527565 20800 raft_consensus.cc:2804] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:21:00.524868 20236 raft_consensus.cc:493] T 2794af5b99984efebdee564035841cb0 P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:00.555799 16820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada5a004ba7c46819051e3ccec71ffdb" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
W20260709 13:21:00.558532 16266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 286094us
I20260709 13:21:00.535894 20231 raft_consensus.cc:740] T e5ad4eb281564936a59f41e387a47156 P 647a343a542d4a139fe106c19095129b [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:21:00.580519 16145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ad5dbfa99ee4b8d9e28be92b3f36db5" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:00.570892 20434 leader_election.cc:290] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:00.583480 16147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d4ab870a05347868956aa8b841b2981" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:00.546072 20573 raft_consensus.cc:2804] T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:21:00.584781 20573 raft_consensus.cc:493] T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:00.585199 20573 raft_consensus.cc:3060] T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:21:00.585402 20231 consensus_queue.cc:260] T e5ad4eb281564936a59f41e387a47156 P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:00.586573 20231 raft_consensus.cc:3060] T e5ad4eb281564936a59f41e387a47156 P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:21:00.577654 20388 leader_election.cc:290] T 68a6838d82ef4cac83507f906139d90f P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 13 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:00.524142 20618 raft_consensus.cc:515] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:00.582679 16151 raft_consensus.cc:1240] T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 12. Current term is 13. Ops: []
I20260709 13:21:00.582883 16266 leader_election.cc:304] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:21:00.582901 16145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434bff3082d141838068f323bdb0d1ed" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:00.591326 16266 leader_election.cc:304] T b87147731a68400e8cce9b879daa7c5d P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:21:00.583372 16144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57a744f5aaf944678a63469cbf514015" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:00.591714 16145 raft_consensus.cc:2468] T 434bff3082d141838068f323bdb0d1ed P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 11.
I20260709 13:21:00.592618 16266 leader_election.cc:304] T d2ab0ac8347641e6955e9bce7d90c1a2 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860
I20260709 13:21:00.591771 16147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2794af5b99984efebdee564035841cb0" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:21:00.513164 20599 raft_consensus.cc:515] T 8158bbd17b994d2eb96f560bd0f1b88d P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:00.580765 16790 leader_election.cc:304] T 66f75124215f4ca6bef831e174a792d8 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:21:00.590389 16142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31969392ccdb43b491911a319e8bc7e1" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:00.600661 16142 raft_consensus.cc:2368] T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 for earlier term 11. Current term is 12.
I20260709 13:21:00.550590 20663 consensus_queue.cc:1048] T 8638ee363273426db1393dd81e5c7b72 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.037s
I20260709 13:21:00.557673 20800 raft_consensus.cc:493] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:00.588521 16146 raft_consensus.cc:1275] T 5a959dbecc2648eb85d1cc550f5a3245 P 647a343a542d4a139fe106c19095129b [term 10 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:21:00.591552 16151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68a6838d82ef4cac83507f906139d90f" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:00.607374 16147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a96373bb0e14dce8a266b2fc6c58507" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:21:00.608090 16151 raft_consensus.cc:2368] T 68a6838d82ef4cac83507f906139d90f P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 for earlier term 12. Current term is 13.
I20260709 13:21:00.608492 16147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0eab25ce4c64e018a041beeda2867db" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:21:00.609045 16147 raft_consensus.cc:3060] T c0eab25ce4c64e018a041beeda2867db P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:21:00.601094 16142 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40830 (request call id 1522) took 1313 ms. Trace:
I20260709 13:21:00.609453 16142 rpcz_store.cc:276] 0709 13:20:59.287839 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:00.587636 (+1299797us) service_pool.cc:224] Handling call
0709 13:21:00.601037 (+ 13401us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:21:00.592703 16145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cc959fb7f704fba9009c0595467d074" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:21:00.604063 16144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fffab26427e24d398e2695aad40ca134" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
W20260709 13:21:00.605580 20640 consensus_meta.cc:287] T 66ae176961b74813a58bc15c9c4c6f2a P 90f3da57796b488c9087a2f523adf045: Time spent flushing consensus metadata: real 0.561s	user 0.011s	sys 0.000s
I20260709 13:21:00.606129 20779 leader_election.cc:290] T 26354361e9b044cea1c2b86c4882fe92 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:00.598776 16143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6f07842d3aa47858df5665874c86f3a" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:21:00.609238 16265 leader_election.cc:304] T 8d4ab870a05347868956aa8b841b2981 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:21:00.601106 20267 raft_consensus.cc:2749] T d2ab0ac8347641e6955e9bce7d90c1a2 P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:21:00.612948 16145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87147731a68400e8cce9b879daa7c5d" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:00.609117 16151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4dc6b0541ca431ebc757beac00521f1" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:00.618131 16151 raft_consensus.cc:2393] T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 12: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
I20260709 13:21:00.568895 20236 raft_consensus.cc:515] T 2794af5b99984efebdee564035841cb0 P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:00.594177 20644 leader_election.cc:290] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 14 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:00.600337 20785 raft_consensus.cc:515] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:00.624976 16825 raft_consensus.cc:1240] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860 [term 12 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 11. Current term is 12. Ops: []
I20260709 13:21:00.611675 20800 raft_consensus.cc:3060] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:21:00.628844 20236 leader_election.cc:290] T 2794af5b99984efebdee564035841cb0 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:00.629375 20618 leader_election.cc:290] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 14 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:00.629570 16820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62fe3346cd3c42c4a55e14af9073bca4" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 16 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:21:00.630153 20618 consensus_queue.cc:1048] T d16dfad0c3b540e5b4b86492ddf87208 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:21:00.618621 20640 raft_consensus.cc:515] T 66ae176961b74813a58bc15c9c4c6f2a P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:00.607683 20663 consensus_queue.cc:1243] T 8638ee363273426db1393dd81e5c7b72 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Peer 1ca2725d290e47d68b1a4b609c9816ac log is divergent from this leader: its last log entry 9.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 13:21:00.620395 16142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 14 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:21:00.616288 16265 leader_election.cc:304] T 434bff3082d141838068f323bdb0d1ed P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:21:00.601966 20583 raft_consensus.cc:2804] T b87147731a68400e8cce9b879daa7c5d P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 13:21:00.646939 16114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:00.619518 16144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f5f89186204242bf377f7c42b2e2da" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:00.611479 20760 raft_consensus.cc:493] T 1df93779209748b9860f6d893dd3de53 P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:00.620363 16146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6f07842d3aa47858df5665874c86f3a" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:21:00.640395 16148 raft_consensus.cc:2468] T 2794af5b99984efebdee564035841cb0 P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 9.
I20260709 13:21:00.643534 16142 raft_consensus.cc:3060] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:21:00.644972 16264 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:00.645938 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87ac0d1f7e8a419f9bcbc04aee0e067e" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 14 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:00.601454 20657 raft_consensus.cc:2804] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:21:00.618979 20661 raft_consensus.cc:2749] T 8d4ab870a05347868956aa8b841b2981 P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:21:00.632459 16145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ad5dbfa99ee4b8d9e28be92b3f36db5" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:00.614117 20673 raft_consensus.cc:493] T d2ab0ac8347641e6955e9bce7d90c1a2 P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:00.614241 20797 raft_consensus.cc:493] T 08c74490c40243a88e62d1760b5cd814 P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:00.652228 16217 raft_consensus.cc:1275] T ecc1e363230043fe9fef390f0b979f20 P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 13:21:00.666046 16145 raft_consensus.cc:1275] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 13:21:00.650769 16219 raft_consensus.cc:1240] T e5ad4eb281564936a59f41e387a47156 P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 12. Current term is 13. Ops: []
I20260709 13:21:00.660526 16224 raft_consensus.cc:2410] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b for term 14 because replica has last-logged OpId of term: 11 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 8.
I20260709 13:21:00.648459 16144 raft_consensus.cc:1240] T 9e65676b21b748728ad39668cfe722ec P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 10. Current term is 11. Ops: []
I20260709 13:21:00.668136 16219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8158bbd17b994d2eb96f560bd0f1b88d" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 12 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:00.676358 16144 raft_consensus.cc:1275] T 1df93779209748b9860f6d893dd3de53 P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
W20260709 13:21:00.677426 16117 connection.cc:256] RPC call timeout handler was delayed by -1.1673467460000211s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16590000000000002s for a response
I20260709 13:21:00.677871 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68a6838d82ef4cac83507f906139d90f" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:21:00.678465 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4dc6b0541ca431ebc757beac00521f1" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:21:00.663080 16222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b7d1ab519044eda2d5e04a07242f25" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:00.679111 16224 raft_consensus.cc:3060] T e4dc6b0541ca431ebc757beac00521f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:21:00.680644 16265 leader_election.cc:304] T 2794af5b99984efebdee564035841cb0 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
I20260709 13:21:00.661583 16221 raft_consensus.cc:1240] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 13. Current term is 14. Ops: [11.11-13.12]
I20260709 13:21:00.682705 16143 raft_consensus.cc:2468] T b6f07842d3aa47858df5665874c86f3a P 647a343a542d4a139fe106c19095129b [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 14.
W20260709 13:21:00.662565 16192 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 124912us
I20260709 13:21:00.678609 16217 raft_consensus.cc:3060] T 68a6838d82ef4cac83507f906139d90f P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:21:00.662132 16147 raft_consensus.cc:2468] T c0eab25ce4c64e018a041beeda2867db P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 12.
I20260709 13:21:00.685214 16822 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49896 (request call id 1187) took 1029 ms. Trace:
I20260709 13:21:00.685546 16822 rpcz_store.cc:276] 0709 13:20:59.655383 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:00.224144 (+568761us) service_pool.cc:224] Handling call
0709 13:21:00.283100 (+ 58956us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:21:00.283341 (+   241us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:21:00.497127 (+213786us) log.cc:844] Serialized 14 byte log entry
0709 13:21:00.684928 (+187801us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:21:00.684977 (+    49us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:21:00.685017 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:00.685081 (+    64us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:21:00.685172 (+    91us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":296612,"prepare.run_cpu_time_us":73,"prepare.run_wall_time_us":5426,"thread_start_us":309530,"threads_started":2,"wal-append.queue_time_us":162757}
I20260709 13:21:00.660097 20785 leader_election.cc:290] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 12 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:00.687400 16224 raft_consensus.cc:2468] T e4dc6b0541ca431ebc757beac00521f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 12.
W20260709 13:21:00.689363 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 108665us
I20260709 13:21:00.691308 20607 raft_consensus.cc:2749] T 66f75124215f4ca6bef831e174a792d8 P cb82cc7bd6db46d980c75237a1402860 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:21:00.680277 20673 raft_consensus.cc:515] T d2ab0ac8347641e6955e9bce7d90c1a2 P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:00.692656 16293 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48674 (request call id 2738) took 1262 ms. Trace:
I20260709 13:21:00.677264 20797 raft_consensus.cc:515] T 08c74490c40243a88e62d1760b5cd814 P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:00.659795 20644 raft_consensus.cc:2804] T 434bff3082d141838068f323bdb0d1ed P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:21:00.687114 20267 raft_consensus.cc:2749] T 2794af5b99984efebdee564035841cb0 P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:21:00.700495 16144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434bff3082d141838068f323bdb0d1ed" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:00.701895 16144 raft_consensus.cc:2410] T 434bff3082d141838068f323bdb0d1ed P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 for term 12 because replica has last-logged OpId of term: 10 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 9.
W20260709 13:21:00.695137 16264 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:00.697417 16117 leader_election.cc:400] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 13 election: Vote denied by peer cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923) with higher term. Message: Invalid argument: T 19baa7bc367844c288bca8f81439c413 P cb82cc7bd6db46d980c75237a1402860 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b for earlier term 13. Current term is 14.
I20260709 13:21:00.698163 16147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26354361e9b044cea1c2b86c4882fe92" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:00.650461 20583 raft_consensus.cc:493] T b87147731a68400e8cce9b879daa7c5d P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:21:00.700661 16143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:21:00.693197 16293 rpcz_store.cc:276] 0709 13:20:59.430565 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:59.467731 (+ 37166us) service_pool.cc:224] Handling call
0709 13:20:59.494344 (+ 26613us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:20:59.494825 (+   481us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:20:59.494865 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:20:59.495446 (+   581us) log.cc:844] Serialized 26 byte log entry
0709 13:21:00.472433 (+976987us) spinlock_profiling.cc:243] Waited 6.23 ms on lock 0x7b180031cf00. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b693ade1 00007f69b69475b1 00007f69ada54e07 00007f69b68afab9 00007f69ada7169b 00007f69ada6f93f 00007f69ada71c02 00007f69b06f920b 00007f69b06fed23 00007f69b06ff0d1 00007f69b0b1aa0f
0709 13:21:00.588039 (+115606us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:21:00.588139 (+   100us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:00.588278 (+   139us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:21:00.692258 (+103980us) raft_consensus.cc:1685] finished
0709 13:21:00.692287 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:21:00.692575 (+   288us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4180,"raft.queue_time_us":110,"raft.run_cpu_time_us":148,"raft.run_wall_time_us":6242,"spinlock_wait_cycles":14340480,"thread_start_us":112113,"threads_started":1,"wal-append.queue_time_us":1145627}
I20260709 13:21:00.704452 16117 leader_election.cc:403] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 13 election: Cancelling election due to peer responding with higher term
I20260709 13:21:00.650050 20518 consensus_queue.cc:1048] T 8638ee363273426db1393dd81e5c7b72 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.047s
I20260709 13:21:00.712448 16142 raft_consensus.cc:2468] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 14.
I20260709 13:21:00.680094 20657 raft_consensus.cc:697] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045 [term 14 LEADER]: Becoming Leader. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:21:00.680037 20754 raft_consensus.cc:493] T 746aabf856ab401c9ef5e859371e7b67 P 647a343a542d4a139fe106c19095129b [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:21:00.700999 20644 raft_consensus.cc:493] T 434bff3082d141838068f323bdb0d1ed P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:21:00.722174 20644 raft_consensus.cc:3060] T 434bff3082d141838068f323bdb0d1ed P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:21:00.728019 16117 leader_election.cc:304] T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860
I20260709 13:21:00.729826 16117 leader_election.cc:304] T 9e65676b21b748728ad39668cfe722ec P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860
W20260709 13:21:00.733349 16114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:00.733801 16217 raft_consensus.cc:2468] T 68a6838d82ef4cac83507f906139d90f P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 13.
I20260709 13:21:00.664441 20760 raft_consensus.cc:515] T 1df93779209748b9860f6d893dd3de53 P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:00.693125 20799 raft_consensus.cc:2749] T 3884f864a96d428785895e5463e5728f P cb82cc7bd6db46d980c75237a1402860 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:21:00.696856 20803 raft_consensus.cc:493] T ee9ed67006144797b1de6ea8c2013a68 P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:21:00.736406 16117 leader_election.cc:304] T ee9ed67006144797b1de6ea8c2013a68 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860
I20260709 13:21:00.709725 20583 raft_consensus.cc:3060] T b87147731a68400e8cce9b879daa7c5d P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:21:00.740084 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 126462us
I20260709 13:21:00.681564 20602 consensus_queue.cc:1048] T b87147731a68400e8cce9b879daa7c5d P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.055s
I20260709 13:21:00.734659 20472 raft_consensus.cc:2749] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [term 14 FOLLOWER]: Leader election lost for term 13. Reason: Vote denied by peer cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923) with higher term. Message: Invalid argument: T 19baa7bc367844c288bca8f81439c413 P cb82cc7bd6db46d980c75237a1402860 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b for earlier term 13. Current term is 14.
I20260709 13:21:00.696750 20236 consensus_queue.cc:1048] T ecc1e363230043fe9fef390f0b979f20 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 3, Time since last communication: 0.042s
W20260709 13:21:00.745038 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 64502.6us
I20260709 13:21:00.742483 16117 leader_election.cc:304] T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860
I20260709 13:21:00.738205 20388 raft_consensus.cc:2749] T 9e65676b21b748728ad39668cfe722ec P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:21:00.746206 16191 leader_election.cc:304] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 90f3da57796b488c9087a2f523adf045
I20260709 13:21:00.703462 20607 consensus_queue.cc:1048] T d2ab0ac8347641e6955e9bce7d90c1a2 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.039s
I20260709 13:21:00.744117 20602 consensus_queue.cc:1243] T b87147731a68400e8cce9b879daa7c5d P 647a343a542d4a139fe106c19095129b [LEADER]: Peer cb82cc7bd6db46d980c75237a1402860 log is divergent from this leader: its last log entry 10.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 13:21:00.744335 20434 raft_consensus.cc:2749] T ee9ed67006144797b1de6ea8c2013a68 P 647a343a542d4a139fe106c19095129b [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:21:00.738129 20699 raft_consensus.cc:2749] T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:21:00.752657 16822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434bff3082d141838068f323bdb0d1ed" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:00.714319 20629 consensus_queue.cc:1059] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: INVALID_TERM, Last received: 12.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.032s
I20260709 13:21:00.741103 20599 leader_election.cc:290] T 8158bbd17b994d2eb96f560bd0f1b88d P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:21:00.753039 16117 leader_election.cc:304] T 23f82224f4564fea93572e8206bad1fe P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860
I20260709 13:21:00.756690 20640 leader_election.cc:290] T 66ae176961b74813a58bc15c9c4c6f2a P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 15 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:00.759395 16822 raft_consensus.cc:2468] T 434bff3082d141838068f323bdb0d1ed P cb82cc7bd6db46d980c75237a1402860 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 11.
I20260709 13:21:00.756042 16821 raft_consensus.cc:2468] T 23f82224f4564fea93572e8206bad1fe P cb82cc7bd6db46d980c75237a1402860 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 9.
W20260709 13:21:00.756953 20603 consensus_meta.cc:287] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045: Time spent flushing consensus metadata: real 0.522s	user 0.011s	sys 0.000s
I20260709 13:21:00.758183 20760 leader_election.cc:290] T 1df93779209748b9860f6d893dd3de53 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:00.791371 16117 leader_election.cc:304] T af57996a8be84d398f0f023e8d08ef77 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860
I20260709 13:21:00.738328 20754 raft_consensus.cc:515] T 746aabf856ab401c9ef5e859371e7b67 P 647a343a542d4a139fe106c19095129b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:21:00.753738 20817 log.cc:927] Time spent T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 0.123s	user 0.001s	sys 0.000s
I20260709 13:21:00.777509 20818 raft_consensus.cc:2804] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:21:00.801791 16117 leader_election.cc:304] T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:21:00.752197 20755 raft_consensus.cc:2749] T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:21:00.799317 20602 raft_consensus.cc:3060] T af57996a8be84d398f0f023e8d08ef77 P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:21:00.805450 20434 raft_consensus.cc:2804] T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:21:00.795791 20388 raft_consensus.cc:2749] T 23f82224f4564fea93572e8206bad1fe P 647a343a542d4a139fe106c19095129b [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
W20260709 13:21:00.806753 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 60601.7us
I20260709 13:21:00.807010 16265 leader_election.cc:400] T 68a6838d82ef4cac83507f906139d90f P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 12 pre-election: Vote denied by peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741) with higher term. Message: Invalid argument: T 68a6838d82ef4cac83507f906139d90f P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 for earlier term 12. Current term is 13.
I20260709 13:21:00.778291 20779 raft_consensus.cc:493] T ee9ed67006144797b1de6ea8c2013a68 P cb82cc7bd6db46d980c75237a1402860 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:00.801146 20760 raft_consensus.cc:493] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:21:00.819281 16114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:00.795935 20780 raft_consensus.cc:493] T 23f82224f4564fea93572e8206bad1fe P cb82cc7bd6db46d980c75237a1402860 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:00.761756 20778 leader_election.cc:290] T 434bff3082d141838068f323bdb0d1ed P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:00.842204 16222 raft_consensus.cc:1275] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 13:21:00.840716 16116 leader_election.cc:304] T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
I20260709 13:21:00.823998 16265 leader_election.cc:403] T 68a6838d82ef4cac83507f906139d90f P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 12 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:21:00.823825 16191 leader_election.cc:304] T e5ad4eb281564936a59f41e387a47156 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:21:00.826054 16117 leader_election.cc:304] T 434bff3082d141838068f323bdb0d1ed P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860; no voters: 
W20260709 13:21:00.805696 20809 log.cc:927] Time spent T 3884f864a96d428785895e5463e5728f P cb82cc7bd6db46d980c75237a1402860: Append to log took a long time: real 0.144s	user 0.000s	sys 0.000s
W20260709 13:21:00.831590 20814 log.cc:927] Time spent T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860: Append to log took a long time: real 0.130s	user 0.000s	sys 0.000s
I20260709 13:21:00.811887 20699 consensus_queue.cc:1059] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.027s
I20260709 13:21:00.806044 20818 raft_consensus.cc:493] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:00.834386 20779 raft_consensus.cc:515] T ee9ed67006144797b1de6ea8c2013a68 P cb82cc7bd6db46d980c75237a1402860 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:00.799834 20603 raft_consensus.cc:515] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:00.845690 16116 leader_election.cc:304] T 68a6838d82ef4cac83507f906139d90f P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
I20260709 13:21:00.837292 20573 raft_consensus.cc:515] T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:21:00.817575 20806 log.cc:927] Time spent T 18ad7e3af5504571a69d10c00750a01c P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 0.205s	user 0.000s	sys 0.001s
I20260709 13:21:00.847337 16265 leader_election.cc:304] T fffab26427e24d398e2695aad40ca134 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:21:00.823755 16822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7ca61592e848ea9b0fb81a2ce33cdc" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:21:00.826018 20434 raft_consensus.cc:697] T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b [term 12 LEADER]: Becoming Leader. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:21:00.744787 20803 raft_consensus.cc:515] T ee9ed67006144797b1de6ea8c2013a68 P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:00.884207 16821 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49896 (request call id 1179) took 1344 ms. Trace:
I20260709 13:21:00.841302 16296 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48674 (request call id 2731) took 2001 ms. Trace:
I20260709 13:21:00.885612 16265 leader_election.cc:400] T 31969392ccdb43b491911a319e8bc7e1 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 11 pre-election: Vote denied by peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741) with higher term. Message: Invalid argument: T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 for earlier term 11. Current term is 12.
I20260709 13:21:00.880446 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c2095a5f4ad49408dbc7f69c606120d" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:00.885475 16821 rpcz_store.cc:276] 0709 13:20:59.540254 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:00.044933 (+504679us) service_pool.cc:224] Handling call
0709 13:21:00.884022 (+839089us) spinlock_profiling.cc:243] Waited 710 ms on lock 0x7b54007d0354. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:21:00.884143 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1632442112}
I20260709 13:21:00.886332 16265 leader_election.cc:403] T 31969392ccdb43b491911a319e8bc7e1 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 11 pre-election: Cancelling election due to peer responding with higher term
W20260709 13:21:00.886862 16150 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40844 (request call id 2410) took 1629 ms (client timeout 1918 ms). Trace:
I20260709 13:21:00.887398 16821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2794af5b99984efebdee564035841cb0" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:00.831666 16225 raft_consensus.cc:2955] T ecc1e363230043fe9fef390f0b979f20 P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:21:00.844741 16191 leader_election.cc:304] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
I20260709 13:21:00.888142 16821 raft_consensus.cc:2468] T 2794af5b99984efebdee564035841cb0 P cb82cc7bd6db46d980c75237a1402860 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 9.
W20260709 13:21:00.889004 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 44354.9us
I20260709 13:21:00.889168 16821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5718c623ebee4d2bb770f0f9ddb2f164" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 13 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:00.877370 16822 raft_consensus.cc:2376] T 3a7ca61592e848ea9b0fb81a2ce33cdc P cb82cc7bd6db46d980c75237a1402860 [term 15 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 15. Re-sending same reply.
W20260709 13:21:00.887645 16150 rpcz_store.cc:269] 0709 13:20:59.257744 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:59.259287 (+  1543us) service_pool.cc:224] Handling call
0709 13:21:00.880321 (+1621034us) spinlock_profiling.cc:243] Waited 16.3 ms on lock 0x7b5800011198. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69ad8fadb1 00007f69ada3ab71 00007f69ada3aeee 0000000000375939 00007f69b5aae1e0 00007f69b5b770c4 00007f69b5b77131 00007f69b5b250a0 00007f69b5b24984 00007f69af09f6d4 00007f69af09f662
0709 13:21:00.886798 (+  6477us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":37474688}
W20260709 13:21:00.882287 16149 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40844 (request call id 2399) took 1858 ms (client timeout 2238 ms). Trace:
W20260709 13:21:00.891680 16149 rpcz_store.cc:269] 0709 13:20:59.024017 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:59.024255 (+   238us) service_pool.cc:224] Handling call
0709 13:21:00.882075 (+1857820us) spinlock_profiling.cc:243] Waited 15.4 ms on lock 0x7b54000d0f54. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:21:00.882188 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":35363584}
I20260709 13:21:00.885970 16296 rpcz_store.cc:276] 0709 13:20:58.840317 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:58.961124 (+120807us) service_pool.cc:224] Handling call
0709 13:20:58.961389 (+   265us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:59.079989 (+118600us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:00.841044 (+1761055us) spinlock_profiling.cc:243] Waited 118 ms on lock 0x7b54000be5d4. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:21:00.841239 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":270958848}
W20260709 13:21:00.898854 16117 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 72864.6us
I20260709 13:21:00.875443 20472 raft_consensus.cc:493] T 8cc959fb7f704fba9009c0595467d074 P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:21:00.902506 16818 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49896 (request call id 1188) took 1233 ms. Trace:
I20260709 13:21:00.899735 20673 leader_election.cc:290] T d2ab0ac8347641e6955e9bce7d90c1a2 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:00.900115 20618 consensus_queue.cc:1048] T c0eab25ce4c64e018a041beeda2867db P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:21:00.903581 20434 consensus_queue.cc:237] T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:00.901966 16822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "829b4ad8c5a54eababeea1181b960d4b" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:21:00.905584 16822 raft_consensus.cc:2393] T 829b4ad8c5a54eababeea1181b960d4b P cb82cc7bd6db46d980c75237a1402860 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 12: Already voted for candidate cb82cc7bd6db46d980c75237a1402860 in this term.
I20260709 13:21:00.906598 16822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ab0ac8347641e6955e9bce7d90c1a2" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:00.908640 16822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5d59407eb545b895d8d922e0dd69fd" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 14 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:21:00.909439 16822 raft_consensus.cc:2393] T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 14: Already voted for candidate cb82cc7bd6db46d980c75237a1402860 in this term.
I20260709 13:21:00.909976 20762 raft_consensus.cc:2804] T 434bff3082d141838068f323bdb0d1ed P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:21:00.910607 20602 raft_consensus.cc:2749] T af57996a8be84d398f0f023e8d08ef77 P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:21:00.910681 20762 raft_consensus.cc:493] T 434bff3082d141838068f323bdb0d1ed P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:00.902889 20472 raft_consensus.cc:515] T 8cc959fb7f704fba9009c0595467d074 P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:00.876154 20750 raft_consensus.cc:2804] T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:21:00.844671 20657 consensus_queue.cc:237] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.8, Last appended by leader: 8, Current term: 14, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:00.915464 20236 consensus_queue.cc:1059] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.006s
I20260709 13:21:00.844841 20760 raft_consensus.cc:515] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:00.923610 20754 leader_election.cc:290] T 746aabf856ab401c9ef5e859371e7b67 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:00.924152 20760 leader_election.cc:290] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:00.912842 16822 pending_rounds.cc:85] T b87147731a68400e8cce9b879daa7c5d P cb82cc7bd6db46d980c75237a1402860: Aborting all ops after (but not including) 8
I20260709 13:21:00.924888 16822 pending_rounds.cc:107] T b87147731a68400e8cce9b879daa7c5d P cb82cc7bd6db46d980c75237a1402860: Aborting uncommitted NO_OP operation due to leader change: 10.9
I20260709 13:21:00.848101 20711 raft_consensus.cc:3060] T 68a6838d82ef4cac83507f906139d90f P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Advancing to term 13
I20260709 13:21:00.916196 16265 leader_election.cc:304] T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:21:00.844301 20622 raft_consensus.cc:2749] T e5ad4eb281564936a59f41e387a47156 P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:21:00.903009 16818 rpcz_store.cc:276] 0709 13:20:59.668849 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:00.311380 (+642531us) service_pool.cc:224] Handling call
0709 13:21:00.311729 (+   349us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:21:00.312130 (+   401us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:21:00.312163 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:21:00.317468 (+  5305us) log.cc:844] Serialized 26 byte log entry
0709 13:21:00.529966 (+212498us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:21:00.633845 (+103879us) log.cc:844] Serialized 14 byte log entry
0709 13:21:00.688649 (+ 54804us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:00.688766 (+   117us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:21:00.902259 (+213493us) raft_consensus.cc:1685] finished
0709 13:21:00.902283 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:21:00.902442 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":73772,"prepare.queue_time_us":92466,"prepare.run_cpu_time_us":48,"prepare.run_wall_time_us":50,"raft.queue_time_us":57908,"raft.run_cpu_time_us":308,"raft.run_wall_time_us":16972,"thread_start_us":256131,"threads_started":2,"wal-append.queue_time_us":286218}
I20260709 13:21:00.936128 16265 leader_election.cc:304] T c0eab25ce4c64e018a041beeda2867db P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac
I20260709 13:21:00.902526 20576 raft_consensus.cc:3060] T 31969392ccdb43b491911a319e8bc7e1 P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Advancing to term 12
I20260709 13:21:00.891188 20599 raft_consensus.cc:2749] T fffab26427e24d398e2695aad40ca134 P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:21:00.913033 20583 raft_consensus.cc:515] T b87147731a68400e8cce9b879daa7c5d P 90f3da57796b488c9087a2f523adf045 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:00.925187 16822 raft_consensus.cc:2889] T b87147731a68400e8cce9b879daa7c5d P cb82cc7bd6db46d980c75237a1402860 [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:21:00.846179 20780 raft_consensus.cc:515] T 23f82224f4564fea93572e8206bad1fe P cb82cc7bd6db46d980c75237a1402860 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:00.924319 20820 consensus_queue.cc:1048] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.005s
I20260709 13:21:00.915151 20822 raft_consensus.cc:2749] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:21:00.848699 20644 raft_consensus.cc:515] T 434bff3082d141838068f323bdb0d1ed P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:00.929404 20778 raft_consensus.cc:493] T 8638ee363273426db1393dd81e5c7b72 P cb82cc7bd6db46d980c75237a1402860 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:00.963660 20778 raft_consensus.cc:515] T 8638ee363273426db1393dd81e5c7b72 P cb82cc7bd6db46d980c75237a1402860 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:00.930032 20797 leader_election.cc:290] T 08c74490c40243a88e62d1760b5cd814 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:00.875478 20818 raft_consensus.cc:3060] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:21:00.885251 20427 raft_consensus.cc:2804] T 68a6838d82ef4cac83507f906139d90f P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:21:00.914552 16821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b7d1ab519044eda2d5e04a07242f25" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:00.913921 20573 leader_election.cc:290] T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 13 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:00.946239 16149 raft_consensus.cc:3060] T 746aabf856ab401c9ef5e859371e7b67 P 647a343a542d4a139fe106c19095129b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:21:00.848506 20640 consensus_queue.cc:1048] T 5a959dbecc2648eb85d1cc550f5a3245 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.084s
I20260709 13:21:00.844655 20721 raft_consensus.cc:515] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:21:00.943674 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 52954.1us
I20260709 13:21:00.985308 16191 leader_election.cc:304] T ecc1e363230043fe9fef390f0b979f20 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:21:00.955956 20599 raft_consensus.cc:2804] T c0eab25ce4c64e018a041beeda2867db P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:21:00.925043 20762 raft_consensus.cc:3060] T 434bff3082d141838068f323bdb0d1ed P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:21:00.987545 16117 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 74198.7us
I20260709 13:21:00.988318 16117 leader_election.cc:304] T 2794af5b99984efebdee564035841cb0 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:21:00.989989 20472 leader_election.cc:290] T 8cc959fb7f704fba9009c0595467d074 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:00.951089 20603 leader_election.cc:290] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 13 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:21:00.954573 16222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746aabf856ab401c9ef5e859371e7b67" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:00.999846 16222 raft_consensus.cc:2393] T 746aabf856ab401c9ef5e859371e7b67 P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 11: Already voted for candidate cb82cc7bd6db46d980c75237a1402860 in this term.
I20260709 13:21:00.845484 20800 raft_consensus.cc:515] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:00.937269 20594 raft_consensus.cc:2749] T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:21:00.962841 20644 leader_election.cc:290] T 434bff3082d141838068f323bdb0d1ed P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 11 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:00.967943 16818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdbd78acb4cf46298e0de8ba0d130e19" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
W20260709 13:21:00.972858 16788 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:21:01.002328 20773 log.cc:927] Time spent T 7c2095a5f4ad49408dbc7f69c606120d P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 1.459s	user 0.000s	sys 0.010s
I20260709 13:21:00.951574 20750 raft_consensus.cc:697] T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b [term 12 LEADER]: Becoming Leader. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:21:01.003374 20827 consensus_queue.cc:1048] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:21:01.003640 20750 consensus_queue.cc:237] T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 10.5, Last appended by leader: 5, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:01.004154 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b7d1ab519044eda2d5e04a07242f25" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:21:01.004525 16220 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57600 (request call id 1508) took 1729 ms. Trace:
I20260709 13:21:01.004765 16150 raft_consensus.cc:3060] T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:21:01.004899 16220 rpcz_store.cc:276] 0709 13:20:59.274884 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:59.285035 (+ 10151us) service_pool.cc:224] Handling call
0709 13:20:59.285348 (+   313us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:59.285536 (+   188us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:20:59.423105 (+137569us) log.cc:844] Serialized 14 byte log entry
0709 13:21:01.004010 (+1580905us) spinlock_profiling.cc:243] Waited 6.7 ms on lock 0x7b18002c8d20. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b693ade1 00007f69b69475b1 00007f69b07074bc 00007f69ada5548b 00007f69b68afab9 00007f69ada7169b 00007f69ada6f93f 00007f69ada71c02 00007f69b06f920b 00007f69b06fed23 00007f69b06ff23a
0709 13:21:01.004203 (+   193us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:21:01.004254 (+    51us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:21:01.004292 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:01.004371 (+    79us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:21:01.004475 (+   104us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":220371,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":39,"raft.queue_time_us":769,"raft.run_cpu_time_us":83,"raft.run_wall_time_us":82,"spinlock_wait_cycles":203668096,"thread_start_us":240212,"threads_started":2,"wal-append.queue_time_us":124524}
I20260709 13:21:01.005820 20622 consensus_queue.cc:1048] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:21:01.007082 16226 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57604 (request call id 2639) took 1980 ms. Trace:
I20260709 13:21:00.949966 20757 raft_consensus.cc:493] T fffab26427e24d398e2695aad40ca134 P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:21:01.007429 20823 raft_consensus.cc:493] T 02838c86531d4f98bf58d1c22b8baf77 P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:21:01.007936 20757 raft_consensus.cc:515] T fffab26427e24d398e2695aad40ca134 P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:01.009554 20803 leader_election.cc:290] T ee9ed67006144797b1de6ea8c2013a68 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:01.010243 20757 leader_election.cc:290] T fffab26427e24d398e2695aad40ca134 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:00.956177 20754 consensus_queue.cc:1048] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.036s
I20260709 13:21:01.011886 20583 leader_election.cc:290] T b87147731a68400e8cce9b879daa7c5d P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 14 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:01.011977 20641 raft_consensus.cc:3055] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:21:00.954017 20820 consensus_queue.cc:1243] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 90f3da57796b488c9087a2f523adf045 [LEADER]: Peer 647a343a542d4a139fe106c19095129b log is divergent from this leader: its last log entry 10.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 13:21:01.012524 20641 raft_consensus.cc:740] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:21:00.947084 20661 consensus_queue.cc:1048] T 1df93779209748b9860f6d893dd3de53 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.029s
I20260709 13:21:00.991662 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cc959fb7f704fba9009c0595467d074" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:00.994387 20721 leader_election.cc:290] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 11 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:21:00.997714 20427 raft_consensus.cc:697] T 68a6838d82ef4cac83507f906139d90f P 647a343a542d4a139fe106c19095129b [term 13 LEADER]: Becoming Leader. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:21:00.971946 20778 leader_election.cc:290] T 8638ee363273426db1393dd81e5c7b72 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:00.999019 20443 leader_election.cc:290] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:01.001942 16821 raft_consensus.cc:2393] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 11: Already voted for candidate cb82cc7bd6db46d980c75237a1402860 in this term.
I20260709 13:21:01.007977 20823 raft_consensus.cc:515] T 02838c86531d4f98bf58d1c22b8baf77 P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:01.002354 16818 raft_consensus.cc:2468] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 13.
I20260709 13:21:00.978883 16190 leader_election.cc:304] T c2f5f89186204242bf377f7c42b2e2da P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:21:01.013394 16150 raft_consensus.cc:2468] T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 11.
I20260709 13:21:01.015533 16221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fffab26427e24d398e2695aad40ca134" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:01.007459 16226 rpcz_store.cc:276] 0709 13:20:59.026549 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:59.264009 (+237460us) service_pool.cc:224] Handling call
0709 13:20:59.264396 (+   387us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:20:59.264797 (+   401us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:59.264828 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:20:59.265307 (+   479us) log.cc:844] Serialized 26 byte log entry
0709 13:20:59.441622 (+176315us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:20:59.441690 (+    68us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:59.441806 (+   116us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:21:01.001814 (+1560008us) spinlock_profiling.cc:243] Waited 83.3 ms on lock 0x7b4400022b4c. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69ad8fadb1 00007f69ad998c85 00007f69ad9996a2 00007f69b070b1db 00007f69b0706aaf 00007f69b06fa86e 00007f69b06f9bb6 00007f69b06f97d8 00007f69b0703221 00007f69b07031d9 00007f69b0703169
0709 13:21:01.006814 (+  5000us) raft_consensus.cc:1685] finished
0709 13:21:01.006833 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:21:01.007028 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":119,"raft.run_cpu_time_us":105,"raft.run_wall_time_us":106,"spinlock_wait_cycles":191644672,"thread_start_us":175879,"threads_started":1,"wal-append.queue_time_us":233882}
I20260709 13:21:01.014637 16191 leader_election.cc:304] T 5a959dbecc2648eb85d1cc550f5a3245 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 90f3da57796b488c9087a2f523adf045
I20260709 13:21:01.020538 16149 raft_consensus.cc:1275] T 746aabf856ab401c9ef5e859371e7b67 P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
W20260709 13:21:01.023335 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 44519.2us
I20260709 13:21:01.025174 16790 leader_election.cc:304] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:21:01.030380 16191 leader_election.cc:304] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
W20260709 13:21:01.032965 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 96931.3us
I20260709 13:21:01.022644 20599 raft_consensus.cc:697] T c0eab25ce4c64e018a041beeda2867db P 90f3da57796b488c9087a2f523adf045 [term 12 LEADER]: Becoming Leader. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:21:01.035706 20472 raft_consensus.cc:2804] T 2794af5b99984efebdee564035841cb0 P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:21:01.036150 20472 raft_consensus.cc:493] T 2794af5b99984efebdee564035841cb0 P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:01.036478 20472 raft_consensus.cc:3060] T 2794af5b99984efebdee564035841cb0 P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:21:01.036375 20599 consensus_queue.cc:237] T c0eab25ce4c64e018a041beeda2867db P 90f3da57796b488c9087a2f523adf045 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:01.036768 20641 consensus_queue.cc:260] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:01.037907 20641 raft_consensus.cc:3060] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:21:01.038846 20576 raft_consensus.cc:2749] T 31969392ccdb43b491911a319e8bc7e1 P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Leader pre-election lost for term 11. Reason: Vote denied by peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741) with higher term. Message: Invalid argument: T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 for earlier term 11. Current term is 12.
I20260709 13:21:01.020758 20803 consensus_queue.cc:1059] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: INVALID_TERM, Last received: 11.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.019s
I20260709 13:21:01.039947 20604 raft_consensus.cc:2804] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:21:01.031523 20760 raft_consensus.cc:2804] T c2f5f89186204242bf377f7c42b2e2da P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:21:01.040560 20604 raft_consensus.cc:697] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 [term 11 LEADER]: Becoming Leader. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:21:01.036435 20443 consensus_queue.cc:1048] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.005s
I20260709 13:21:01.041617 20604 consensus_queue.cc:237] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:01.040416 20629 consensus_queue.cc:1048] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:21:01.048540 16217 raft_consensus.cc:1275] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
W20260709 13:21:01.050151 16788 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:01.053805 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c2095a5f4ad49408dbc7f69c606120d" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:01.057019 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c11074579b046d99e585c2818e18a2c" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:01.057705 16226 raft_consensus.cc:2468] T 8c11074579b046d99e585c2818e18a2c P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 12.
I20260709 13:21:01.057466 16818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdbd78acb4cf46298e0de8ba0d130e19" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:21:01.059036 16818 raft_consensus.cc:2410] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 for term 13 because replica has last-logged OpId of term: 12 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 10.
I20260709 13:21:01.050088 16222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806d7690c2b644959152074bb2cab0a7" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:01.061815 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8158bbd17b994d2eb96f560bd0f1b88d" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:01.063802 16142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806d7690c2b644959152074bb2cab0a7" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:01.064419 16142 raft_consensus.cc:2468] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 11.
I20260709 13:21:01.065038 16149 raft_consensus.cc:1240] T 02838c86531d4f98bf58d1c22b8baf77 P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 11. Current term is 12. Ops: [11.7-11.7]
I20260709 13:21:01.066249 16144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c2095a5f4ad49408dbc7f69c606120d" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:01.063155 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e4bfb9090c8460eb14fa7e0d818e7f1" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:01.066974 16217 raft_consensus.cc:2468] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 10.
I20260709 13:21:01.067281 16147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a9de347c711477e87c582d003f36925" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:21:01.067996 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74ca8140f35c4ae29e8a664f6bdfc829" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:21:01.068825 16146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8158bbd17b994d2eb96f560bd0f1b88d" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:01.069252 16145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cc959fb7f704fba9009c0595467d074" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:21:01.070267 16145 raft_consensus.cc:3060] T 8cc959fb7f704fba9009c0595467d074 P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:21:01.073163 16151 raft_consensus.cc:1275] T 08c74490c40243a88e62d1760b5cd814 P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 13:21:01.061573 20594 consensus_queue.cc:1048] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.012s
I20260709 13:21:01.074543 16143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f82224f4564fea93572e8206bad1fe" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:01.074756 16220 raft_consensus.cc:1275] T e5ad4eb281564936a59f41e387a47156 P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 13:21:01.069684 16146 raft_consensus.cc:2468] T 8158bbd17b994d2eb96f560bd0f1b88d P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 11.
I20260709 13:21:01.075332 16143 raft_consensus.cc:2368] T 23f82224f4564fea93572e8206bad1fe P 647a343a542d4a139fe106c19095129b [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 for earlier term 9. Current term is 10.
I20260709 13:21:01.073278 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a378d7f796f94615b3b347119a9880fd" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:01.075875 16142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87147731a68400e8cce9b879daa7c5d" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:21:01.076274 16226 raft_consensus.cc:2468] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 13.
I20260709 13:21:01.069157 16265 leader_election.cc:304] T 8c11074579b046d99e585c2818e18a2c P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:21:01.077049 16142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f5f89186204242bf377f7c42b2e2da" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
W20260709 13:21:01.077979 16790 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.14.177.130:35943 timed out after 1.727s (SENT) [suppressed 24 similar messages]
I20260709 13:21:01.078375 16265 leader_election.cc:400] T 23f82224f4564fea93572e8206bad1fe P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 9 pre-election: Vote denied by peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741) with higher term. Message: Invalid argument: T 23f82224f4564fea93572e8206bad1fe P 647a343a542d4a139fe106c19095129b [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 for earlier term 9. Current term is 10.
I20260709 13:21:01.078702 16265 leader_election.cc:403] T 23f82224f4564fea93572e8206bad1fe P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 9 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:21:01.040614 20760 raft_consensus.cc:493] T c2f5f89186204242bf377f7c42b2e2da P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:01.089421 16142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ad5dbfa99ee4b8d9e28be92b3f36db5" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:21:01.089797 16266 leader_election.cc:304] T cdbd78acb4cf46298e0de8ba0d130e19 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860
I20260709 13:21:01.085527 16143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434bff3082d141838068f323bdb0d1ed" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:21:01.086140 16150 raft_consensus.cc:1275] T e5ad4eb281564936a59f41e387a47156 P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 13:21:01.046254 20827 raft_consensus.cc:2804] T 5a959dbecc2648eb85d1cc550f5a3245 P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:21:01.086901 16265 leader_election.cc:304] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:21:01.087098 16217 raft_consensus.cc:3060] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:21:01.107656 16220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f5f89186204242bf377f7c42b2e2da" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:21:01.108148 16818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66ae176961b74813a58bc15c9c4c6f2a" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:01.113951 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66ae176961b74813a58bc15c9c4c6f2a" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:21:01.114653 16226 raft_consensus.cc:3060] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:21:01.109577 16265 leader_election.cc:304] T a378d7f796f94615b3b347119a9880fd P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:21:01.117767 16142 pending_rounds.cc:85] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 647a343a542d4a139fe106c19095129b: Aborting all ops after (but not including) 7
I20260709 13:21:01.118083 16142 pending_rounds.cc:107] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 647a343a542d4a139fe106c19095129b: Aborting uncommitted NO_OP operation due to leader change: 10.8
I20260709 13:21:01.118324 16142 raft_consensus.cc:2889] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:21:01.121578 16145 raft_consensus.cc:2468] T 8cc959fb7f704fba9009c0595467d074 P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 12.
W20260709 13:21:01.130298 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 66565.5us
I20260709 13:21:01.092687 20576 raft_consensus.cc:3060] T 23f82224f4564fea93572e8206bad1fe P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Advancing to term 10
W20260709 13:21:01.133148 16114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:01.073957 20427 consensus_queue.cc:237] T 68a6838d82ef4cac83507f906139d90f P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:01.141301 20576 raft_consensus.cc:2749] T 23f82224f4564fea93572e8206bad1fe P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Leader pre-election lost for term 9. Reason: Vote denied by peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741) with higher term. Message: Invalid argument: T 23f82224f4564fea93572e8206bad1fe P 647a343a542d4a139fe106c19095129b [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 for earlier term 9. Current term is 10.
I20260709 13:21:01.114094 20820 raft_consensus.cc:2804] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:21:01.143919 20820 raft_consensus.cc:493] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Starting leader election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:21:01.144389 20820 raft_consensus.cc:3060] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:21:01.147447 20822 raft_consensus.cc:2749] T ecc1e363230043fe9fef390f0b979f20 P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:21:01.088555 20751 raft_consensus.cc:2804] T 8c11074579b046d99e585c2818e18a2c P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:21:01.148730 20751 raft_consensus.cc:493] T 8c11074579b046d99e585c2818e18a2c P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:21:01.149183 20751 raft_consensus.cc:3060] T 8c11074579b046d99e585c2818e18a2c P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:21:01.156772 16789 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:01.131143 16191 leader_election.cc:304] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
I20260709 13:21:01.085588 20797 raft_consensus.cc:2749] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:21:01.088084 20800 leader_election.cc:290] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 11 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:21:01.094450 20644 raft_consensus.cc:2749] T cdbd78acb4cf46298e0de8ba0d130e19 P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:21:01.094069 20760 raft_consensus.cc:3060] T c2f5f89186204242bf377f7c42b2e2da P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:21:01.194306 16217 raft_consensus.cc:2468] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 13.
I20260709 13:21:01.174373 16818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e4bfb9090c8460eb14fa7e0d818e7f1" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:01.119321 20640 consensus_queue.cc:1048] T e5ad4eb281564936a59f41e387a47156 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.063s
I20260709 13:21:01.125164 20721 raft_consensus.cc:2804] T a378d7f796f94615b3b347119a9880fd P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 13:21:01.170011 20831 log.cc:927] Time spent T 5a959dbecc2648eb85d1cc550f5a3245 P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.061s	user 0.001s	sys 0.000s
I20260709 13:21:01.181306 16191 leader_election.cc:304] T b6f07842d3aa47858df5665874c86f3a P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
I20260709 13:21:01.191957 16226 raft_consensus.cc:2468] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 15.
I20260709 13:21:01.109755 20827 raft_consensus.cc:493] T 5a959dbecc2648eb85d1cc550f5a3245 P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Starting leader election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:21:01.197351 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ad5dbfa99ee4b8d9e28be92b3f36db5" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:21:01.206868 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ab0ac8347641e6955e9bce7d90c1a2" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:21:01.199656 16265 leader_election.cc:304] T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:21:01.207500 16226 raft_consensus.cc:2376] T d2ab0ac8347641e6955e9bce7d90c1a2 P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 12. Re-sending same reply.
I20260709 13:21:01.211900 16818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434bff3082d141838068f323bdb0d1ed" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:01.219477 16818 raft_consensus.cc:2393] T 434bff3082d141838068f323bdb0d1ed P cb82cc7bd6db46d980c75237a1402860 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 11: Already voted for candidate cb82cc7bd6db46d980c75237a1402860 in this term.
I20260709 13:21:01.236194 16818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74ca8140f35c4ae29e8a664f6bdfc829" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:21:01.236861 16818 raft_consensus.cc:3060] T 74ca8140f35c4ae29e8a664f6bdfc829 P cb82cc7bd6db46d980c75237a1402860 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:21:00.132498 16299 rpcz_store.cc:276] 0709 13:20:57.972853 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:58.489719 (+516866us) service_pool.cc:224] Handling call
0709 13:20:58.565334 (+ 75615us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:58.565582 (+   248us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:21:00.039815 (+1474233us) log.cc:844] Serialized 14 byte log entry
0709 13:21:00.124833 (+ 85018us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:21:00.124884 (+    51us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:21:00.124928 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:00.124991 (+    63us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:21:00.125112 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1102139,"prepare.queue_time_us":1247485,"prepare.run_cpu_time_us":33,"prepare.run_wall_time_us":33,"raft.queue_time_us":2395,"raft.run_cpu_time_us":157,"raft.run_wall_time_us":8035,"thread_start_us":358655,"threads_started":2}
I20260709 13:21:01.215070 16265 leader_election.cc:304] T 66ae176961b74813a58bc15c9c4c6f2a P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:21:01.242331 20671 leader_election.cc:290] T 8a878999c0024d2abe517484180290fc P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 11 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:01.242991 20640 raft_consensus.cc:2804] T 66ae176961b74813a58bc15c9c4c6f2a P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:21:01.243860 20640 raft_consensus.cc:697] T 66ae176961b74813a58bc15c9c4c6f2a P 90f3da57796b488c9087a2f523adf045 [term 15 LEADER]: Becoming Leader. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:21:01.244894 20640 consensus_queue.cc:237] T 66ae176961b74813a58bc15c9c4c6f2a P 90f3da57796b488c9087a2f523adf045 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.10, Last appended by leader: 10, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:21:01.245683 20804 log.cc:927] Time spent T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860: Append to log took a long time: real 0.667s	user 0.001s	sys 0.001s
I20260709 13:21:01.211860 20622 raft_consensus.cc:2804] T b6f07842d3aa47858df5665874c86f3a P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:21:01.248931 20622 raft_consensus.cc:697] T b6f07842d3aa47858df5665874c86f3a P 1ca2725d290e47d68b1a4b609c9816ac [term 14 LEADER]: Becoming Leader. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:21:01.250216 16291 raft_consensus.cc:1240] T 8cc959fb7f704fba9009c0595467d074 P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 11. Current term is 12. Ops: []
I20260709 13:21:01.249902 20622 consensus_queue.cc:237] T b6f07842d3aa47858df5665874c86f3a P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:01.250296 16295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c61a39fd4f84424b129c1a6f9ba404d" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:21:01.250979 16295 raft_consensus.cc:3060] T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:21:01.255836 16296 raft_consensus.cc:1240] T fffab26427e24d398e2695aad40ca134 P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 9. Current term is 10. Ops: []
I20260709 13:21:01.257257 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9944d5e4b8c4109af9955c4f08c9095" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:21:01.258284 16293 raft_consensus.cc:2410] T c9944d5e4b8c4109af9955c4f08c9095 P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac for term 13 because replica has last-logged OpId of term: 12 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 10.
I20260709 13:21:01.261103 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6f07842d3aa47858df5665874c86f3a" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:21:01.261999 16300 raft_consensus.cc:2393] T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 14: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
I20260709 13:21:01.268342 16296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada5a004ba7c46819051e3ccec71ffdb" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:01.270877 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4dc6b0541ca431ebc757beac00521f1" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:01.271723 16293 raft_consensus.cc:2468] T e4dc6b0541ca431ebc757beac00521f1 P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 11.
I20260709 13:21:01.273200 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1df93779209748b9860f6d893dd3de53" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:01.274391 16295 raft_consensus.cc:2468] T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 13.
I20260709 13:21:01.269079 16296 raft_consensus.cc:2468] T ada5a004ba7c46819051e3ccec71ffdb P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 12.
I20260709 13:21:01.275280 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ab0ac8347641e6955e9bce7d90c1a2" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:01.290746 16293 raft_consensus.cc:2468] T d2ab0ac8347641e6955e9bce7d90c1a2 P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 12.
I20260709 13:21:01.289831 16296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdbd78acb4cf46298e0de8ba0d130e19" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:01.292258 16296 raft_consensus.cc:2468] T cdbd78acb4cf46298e0de8ba0d130e19 P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 13.
I20260709 13:21:01.198756 20779 leader_election.cc:290] T ee9ed67006144797b1de6ea8c2013a68 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:21:01.205819 20721 raft_consensus.cc:493] T a378d7f796f94615b3b347119a9880fd P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:01.297804 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a878999c0024d2abe517484180290fc" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:21:01.213280 20827 raft_consensus.cc:3060] T 5a959dbecc2648eb85d1cc550f5a3245 P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:21:01.320662 16291 raft_consensus.cc:1275] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 13:21:01.324533 16299 raft_consensus.cc:3060] T 8a878999c0024d2abe517484180290fc P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:21:01.325883 20835 log.cc:927] Time spent T ecc1e363230043fe9fef390f0b979f20 P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 0.093s	user 0.001s	sys 0.000s
I20260709 13:21:01.234720 20644 raft_consensus.cc:2804] T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Leader election won for term 13
W20260709 13:21:01.345430 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 267455us
W20260709 13:21:01.328810 20831 log.cc:927] Time spent T 5a959dbecc2648eb85d1cc550f5a3245 P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.093s	user 0.000s	sys 0.000s
W20260709 13:21:01.333501 20819 log.cc:927] Time spent T 18ad7e3af5504571a69d10c00750a01c P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 0.093s	user 0.000s	sys 0.001s
I20260709 13:21:01.309677 20721 raft_consensus.cc:3060] T a378d7f796f94615b3b347119a9880fd P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:21:01.356477 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c74490c40243a88e62d1760b5cd814" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:01.366685 16291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ad5dbfa99ee4b8d9e28be92b3f36db5" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:01.349116 16818 raft_consensus.cc:2468] T 74ca8140f35c4ae29e8a664f6bdfc829 P cb82cc7bd6db46d980c75237a1402860 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 13.
I20260709 13:21:01.385375 16291 raft_consensus.cc:2393] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 11: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
I20260709 13:21:01.345701 20644 raft_consensus.cc:697] T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045 [term 13 LEADER]: Becoming Leader. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:21:01.402021 16299 raft_consensus.cc:2468] T 8a878999c0024d2abe517484180290fc P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 11.
I20260709 13:21:01.405966 20751 raft_consensus.cc:515] T 8c11074579b046d99e585c2818e18a2c P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:01.407627 20644 consensus_queue.cc:237] T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:01.409775 20443 consensus_queue.cc:1059] T 8cc959fb7f704fba9009c0595467d074 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: INVALID_TERM, Last received: 11.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:21:01.469937 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b7d1ab519044eda2d5e04a07242f25" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:01.418834 20818 raft_consensus.cc:515] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } }
I20260709 13:21:01.479067 16292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdbd78acb4cf46298e0de8ba0d130e19" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:01.484020 16294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada5a004ba7c46819051e3ccec71ffdb" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:01.602237 16294 raft_consensus.cc:2468] T ada5a004ba7c46819051e3ccec71ffdb P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 12.
I20260709 13:21:01.603675 16294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1df93779209748b9860f6d893dd3de53" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:01.524674 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26354361e9b044cea1c2b86c4882fe92" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
W20260709 13:21:01.526921 16822 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.514s	user 0.001s	sys 0.005s
I20260709 13:21:01.528760 16297 raft_consensus.cc:1240] T c0eab25ce4c64e018a041beeda2867db P 90f3da57796b488c9087a2f523adf045 [term 12 LEADER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 11. Current term is 12. Ops: [11.7-11.7]
I20260709 13:21:01.484202 20661 raft_consensus.cc:493] T 9e65676b21b748728ad39668cfe722ec P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:21:01.537443 16788 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:01.541462 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:21:01.606568 16298 raft_consensus.cc:3060] T ecc1e363230043fe9fef390f0b979f20 P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:21:01.572297 20594 raft_consensus.cc:493] T 26354361e9b044cea1c2b86c4882fe92 P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
W20260709 13:21:01.566226 20711 consensus_meta.cc:287] T 68a6838d82ef4cac83507f906139d90f P 90f3da57796b488c9087a2f523adf045: Time spent flushing consensus metadata: real 0.626s	user 0.009s	sys 0.000s
I20260709 13:21:01.616135 16292 raft_consensus.cc:2393] T cdbd78acb4cf46298e0de8ba0d130e19 P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 in current term 13: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
I20260709 13:21:01.566419 20667 raft_consensus.cc:3060] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:21:01.568133 16817 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49906 (request call id 920) took 1615 ms (client timeout 2064 ms). Trace:
W20260709 13:21:01.619686 16817 rpcz_store.cc:269] 0709 13:20:59.953052 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:00.043754 (+ 90702us) service_pool.cc:224] Handling call
0709 13:21:01.567887 (+1524133us) spinlock_profiling.cc:243] Waited 130 ms on lock 0x7b54005c35d4. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:21:01.568046 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":299452288}
I20260709 13:21:01.620853 16817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87147731a68400e8cce9b879daa7c5d" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:21:01.621515 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a96373bb0e14dce8a266b2fc6c58507" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:21:01.622107 16817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434bff3082d141838068f323bdb0d1ed" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:21:01.622874 16817 raft_consensus.cc:2393] T 434bff3082d141838068f323bdb0d1ed P cb82cc7bd6db46d980c75237a1402860 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 11: Already voted for candidate cb82cc7bd6db46d980c75237a1402860 in this term.
I20260709 13:21:01.568953 16824 raft_consensus.cc:1275] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 [term 12 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 12 index: 12. (index mismatch)
I20260709 13:21:01.528030 20803 raft_consensus.cc:2749] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:21:01.575547 16818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c11074579b046d99e585c2818e18a2c" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:01.525343 20671 consensus_queue.cc:1059] T fffab26427e24d398e2695aad40ca134 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.047s
I20260709 13:21:01.522975 20472 raft_consensus.cc:515] T 2794af5b99984efebdee564035841cb0 P 647a343a542d4a139fe106c19095129b [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:21:01.605844 16822 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.593s	user 0.001s	sys 0.005s
W20260709 13:21:01.544590 20762 consensus_meta.cc:287] T 434bff3082d141838068f323bdb0d1ed P 647a343a542d4a139fe106c19095129b: Time spent flushing consensus metadata: real 0.557s	user 0.007s	sys 0.003s
I20260709 13:21:01.611379 20470 raft_consensus.cc:3055] T 8cc959fb7f704fba9009c0595467d074 P 1ca2725d290e47d68b1a4b609c9816ac [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:21:01.626480 16818 raft_consensus.cc:2393] T 8c11074579b046d99e585c2818e18a2c P cb82cc7bd6db46d980c75237a1402860 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 12: Already voted for candidate cb82cc7bd6db46d980c75237a1402860 in this term.
I20260709 13:21:01.629076 16817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 12 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:01.623584 20711 raft_consensus.cc:2749] T 68a6838d82ef4cac83507f906139d90f P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Leader pre-election lost for term 12. Reason: Vote denied by peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741) with higher term. Message: Invalid argument: T 68a6838d82ef4cac83507f906139d90f P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 for earlier term 12. Current term is 13.
W20260709 13:21:01.645536 20781 log.cc:927] Time spent T 02838c86531d4f98bf58d1c22b8baf77 P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 2.097s	user 0.013s	sys 0.000s
I20260709 13:21:01.643110 20470 raft_consensus.cc:740] T 8cc959fb7f704fba9009c0595467d074 P 1ca2725d290e47d68b1a4b609c9816ac [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:21:01.618626 20661 raft_consensus.cc:515] T 9e65676b21b748728ad39668cfe722ec P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:01.619283 20594 raft_consensus.cc:515] T 26354361e9b044cea1c2b86c4882fe92 P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:01.645680 20762 raft_consensus.cc:515] T 434bff3082d141838068f323bdb0d1ed P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:01.675064 20751 leader_election.cc:290] T 8c11074579b046d99e585c2818e18a2c P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 12 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:01.676667 20818 leader_election.cc:290] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 12 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:01.677901 20661 leader_election.cc:290] T 9e65676b21b748728ad39668cfe722ec P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:01.678705 20470 consensus_queue.cc:260] T 8cc959fb7f704fba9009c0595467d074 P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:01.681514 20472 leader_election.cc:290] T 2794af5b99984efebdee564035841cb0 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 10 election: Requested vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:01.685396 20762 leader_election.cc:290] T 434bff3082d141838068f323bdb0d1ed P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 12 election: Requested vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
W20260709 13:21:01.687067 20820 consensus_meta.cc:287] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045: Time spent flushing consensus metadata: real 0.542s	user 0.012s	sys 0.000s
I20260709 13:21:01.687585 20820 raft_consensus.cc:515] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:01.689416 20820 leader_election.cc:290] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 11 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:01.692761 20594 leader_election.cc:290] T 26354361e9b044cea1c2b86c4882fe92 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:01.695387 20760 raft_consensus.cc:515] T c2f5f89186204242bf377f7c42b2e2da P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:01.698736 20827 raft_consensus.cc:515] T 5a959dbecc2648eb85d1cc550f5a3245 P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:01.701802 20851 raft_consensus.cc:3055] T fffab26427e24d398e2695aad40ca134 P 1ca2725d290e47d68b1a4b609c9816ac [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:21:01.702611 20827 leader_election.cc:290] T 5a959dbecc2648eb85d1cc550f5a3245 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 11 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:01.699769 20721 raft_consensus.cc:515] T a378d7f796f94615b3b347119a9880fd P 90f3da57796b488c9087a2f523adf045 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:01.705163 20721 leader_election.cc:290] T a378d7f796f94615b3b347119a9880fd P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 14 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:01.702312 20851 raft_consensus.cc:740] T fffab26427e24d398e2695aad40ca134 P 1ca2725d290e47d68b1a4b609c9816ac [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:21:01.707208 20851 consensus_queue.cc:260] T fffab26427e24d398e2695aad40ca134 P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:01.709108 20851 raft_consensus.cc:3060] T fffab26427e24d398e2695aad40ca134 P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:21:01.581409 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 210057us
I20260709 13:21:01.744679 16144 raft_consensus.cc:1275] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 647a343a542d4a139fe106c19095129b [term 10 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:21:01.746923 16143 raft_consensus.cc:2368] T 434bff3082d141838068f323bdb0d1ed P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 for earlier term 11. Current term is 12.
I20260709 13:21:01.749615 16142 raft_consensus.cc:1275] T c0eab25ce4c64e018a041beeda2867db P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:21:01.752207 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdbd78acb4cf46298e0de8ba0d130e19" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
W20260709 13:21:01.743386 16115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 165994us
I20260709 13:21:01.753822 16217 raft_consensus.cc:2468] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 13.
I20260709 13:21:01.754220 16190 leader_election.cc:304] T e4dc6b0541ca431ebc757beac00521f1 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 647a343a542d4a139fe106c19095129b
I20260709 13:21:01.756479 16220 raft_consensus.cc:2393] T c2f5f89186204242bf377f7c42b2e2da P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 13: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
I20260709 13:21:01.756474 16222 raft_consensus.cc:2393] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 12: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
I20260709 13:21:01.756618 16226 raft_consensus.cc:1240] T 5a959dbecc2648eb85d1cc550f5a3245 P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 10. Current term is 11. Ops: []
I20260709 13:21:01.758211 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:21:01.589362 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "829b4ad8c5a54eababeea1181b960d4b" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:21:01.786674 16225 raft_consensus.cc:2393] T 829b4ad8c5a54eababeea1181b960d4b P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 in current term 12: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
I20260709 13:21:01.617115 16292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8638ee363273426db1393dd81e5c7b72" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:01.789824 16292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434bff3082d141838068f323bdb0d1ed" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:01.765992 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a959dbecc2648eb85d1cc550f5a3245" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:21:01.792508 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434bff3082d141838068f323bdb0d1ed" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
W20260709 13:21:01.793278 16218 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57600 (request call id 1522) took 2426 ms (client timeout 1890 ms). Trace:
I20260709 13:21:01.478657 16295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31969392ccdb43b491911a319e8bc7e1" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:01.796299 16295 raft_consensus.cc:2468] T 31969392ccdb43b491911a319e8bc7e1 P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 12.
I20260709 13:21:01.796439 16219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af57996a8be84d398f0f023e8d08ef77" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:21:01.797847 16220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada5a004ba7c46819051e3ccec71ffdb" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:01.799623 16291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8158bbd17b994d2eb96f560bd0f1b88d" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 12 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:01.801017 16291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31969392ccdb43b491911a319e8bc7e1" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:21:01.768981 16296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806d7690c2b644959152074bb2cab0a7" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:01.803787 16296 raft_consensus.cc:2468] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 11.
I20260709 13:21:01.804867 16296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806d7690c2b644959152074bb2cab0a7" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:21:01.805028 16824 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49896 (request call id 1189) took 2038 ms. Trace:
I20260709 13:21:01.771023 20427 raft_consensus.cc:493] T d16dfad0c3b540e5b4b86492ddf87208 P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:01.805959 20427 raft_consensus.cc:515] T d16dfad0c3b540e5b4b86492ddf87208 P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:01.772557 16817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57a744f5aaf944678a63469cbf514015" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:01.773929 16818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746aabf856ab401c9ef5e859371e7b67" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:01.805455 16296 raft_consensus.cc:3060] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:21:01.775328 16819 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49892 (request call id 666) took 2245 ms (client timeout 1890 ms). Trace:
I20260709 13:21:01.775750 20571 raft_consensus.cc:3060] T af57996a8be84d398f0f023e8d08ef77 P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:21:01.775975 16821 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49896 (request call id 1197) took 1165 ms. Trace:
I20260709 13:21:01.614293 16294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2794af5b99984efebdee564035841cb0" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
W20260709 13:21:01.793934 16218 rpcz_store.cc:269] 0709 13:20:59.366173 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:59.498248 (+132075us) service_pool.cc:224] Handling call
0709 13:21:01.792741 (+2294493us) spinlock_profiling.cc:243] Waited 682 ms on lock 0x7b54001414d4. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:21:01.792994 (+   253us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1568321792}
I20260709 13:21:01.828547 16818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f82224f4564fea93572e8206bad1fe" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:01.828847 16218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8638ee363273426db1393dd81e5c7b72" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:01.829531 16818 raft_consensus.cc:2468] T 23f82224f4564fea93572e8206bad1fe P cb82cc7bd6db46d980c75237a1402860 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 9.
I20260709 13:21:01.830264 16218 raft_consensus.cc:2468] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 13.
I20260709 13:21:01.768290 16300 raft_consensus.cc:2410] T 26354361e9b044cea1c2b86c4882fe92 P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 for term 11 because replica has last-logged OpId of term: 10 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 9.
I20260709 13:21:01.830785 16818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee9ed67006144797b1de6ea8c2013a68" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:01.794528 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c61a39fd4f84424b129c1a6f9ba404d" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:01.797937 16295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4f65cdf00b748bb963ee6e81ef43b79" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:21:01.814455 16817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74ca8140f35c4ae29e8a664f6bdfc829" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:01.798976 16220 raft_consensus.cc:2468] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 12.
I20260709 13:21:01.805454 16824 rpcz_store.cc:276] 0709 13:20:59.766217 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:00.313602 (+547385us) service_pool.cc:224] Handling call
0709 13:21:01.568532 (+1254930us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:21:01.623863 (+ 55331us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:01.804550 (+180687us) spinlock_profiling.cc:243] Waited 1.25 s on lock 0x7b54005c35d0. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5e8ad 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:21:01.804804 (+   254us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2885318912}
I20260709 13:21:01.809413 16821 rpcz_store.cc:276] 0709 13:21:00.610902 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:00.914065 (+303163us) service_pool.cc:224] Handling call
0709 13:21:01.775480 (+861415us) spinlock_profiling.cc:243] Waited 22.1 ms on lock 0x7b54005032d4. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:21:01.775739 (+   259us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":50906496}
W20260709 13:21:01.807765 16819 rpcz_store.cc:269] 0709 13:20:59.530077 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:59.530813 (+   736us) service_pool.cc:224] Handling call
0709 13:21:01.774712 (+2243899us) spinlock_profiling.cc:243] Waited 17.9 ms on lock 0x7b54005519d4. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:21:01.774997 (+   285us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":41081216}
I20260709 13:21:01.822535 20427 leader_election.cc:290] T d16dfad0c3b540e5b4b86492ddf87208 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:01.823930 20470 raft_consensus.cc:3060] T 8cc959fb7f704fba9009c0595467d074 P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:21:01.790807 16292 raft_consensus.cc:2468] T 434bff3082d141838068f323bdb0d1ed P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 11.
I20260709 13:21:01.833168 16218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806d7690c2b644959152074bb2cab0a7" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:01.837963 16292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4dc6b0541ca431ebc757beac00521f1" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:21:01.837834 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5718c623ebee4d2bb770f0f9ddb2f164" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:01.843880 16821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806d7690c2b644959152074bb2cab0a7" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:01.847977 16218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c11074579b046d99e585c2818e18a2c" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:21:01.847867 16151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e65676b21b748728ad39668cfe722ec" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:01.851636 16818 raft_consensus.cc:2468] T ee9ed67006144797b1de6ea8c2013a68 P cb82cc7bd6db46d980c75237a1402860 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 10.
I20260709 13:21:01.853989 16819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e4bfb9090c8460eb14fa7e0d818e7f1" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:01.855890 16292 raft_consensus.cc:3060] T e4dc6b0541ca431ebc757beac00521f1 P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:21:01.856640 16825 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49896 (request call id 1192) took 2041 ms. Trace:
I20260709 13:21:01.859611 16151 raft_consensus.cc:2468] T 9e65676b21b748728ad39668cfe722ec P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 11.
I20260709 13:21:01.860080 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26354361e9b044cea1c2b86c4882fe92" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:01.861405 16821 raft_consensus.cc:2468] T 806d7690c2b644959152074bb2cab0a7 P cb82cc7bd6db46d980c75237a1402860 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 10.
I20260709 13:21:01.857055 16825 rpcz_store.cc:276] 0709 13:20:59.811722 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:00.360604 (+548882us) service_pool.cc:224] Handling call
0709 13:21:00.360995 (+   391us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:21:00.625496 (+264501us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:01.850390 (+1224894us) spinlock_profiling.cc:243] Waited 264 ms on lock 0x7b54007ffdd4. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:21:01.852642 (+  2252us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":606473600}
I20260709 13:21:01.862743 16821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "829b4ad8c5a54eababeea1181b960d4b" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:01.863348 16825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f82224f4564fea93572e8206bad1fe" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:01.863883 16821 raft_consensus.cc:2393] T 829b4ad8c5a54eababeea1181b960d4b P cb82cc7bd6db46d980c75237a1402860 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 12: Already voted for candidate cb82cc7bd6db46d980c75237a1402860 in this term.
I20260709 13:21:01.864071 16825 raft_consensus.cc:2468] T 23f82224f4564fea93572e8206bad1fe P cb82cc7bd6db46d980c75237a1402860 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 9.
I20260709 13:21:01.865272 16825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a378d7f796f94615b3b347119a9880fd" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:01.867034 16293 raft_consensus.cc:2393] T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 13: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
I20260709 13:21:01.851981 16817 raft_consensus.cc:2376] T 74ca8140f35c4ae29e8a664f6bdfc829 P cb82cc7bd6db46d980c75237a1402860 [term 13 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 13. Re-sending same reply.
I20260709 13:21:01.862959 16294 raft_consensus.cc:2468] T 2794af5b99984efebdee564035841cb0 P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 9.
I20260709 13:21:01.871663 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada5a004ba7c46819051e3ccec71ffdb" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:01.883332 16824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a9de347c711477e87c582d003f36925" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:21:01.884327 16824 raft_consensus.cc:3060] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:20:59.538726 15046 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:21:01.875666 20762 consensus_queue.cc:1048] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.015s
I20260709 13:21:01.876622 16224 raft_consensus.cc:2468] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 12.
W20260709 13:21:01.901081 20774 log.cc:927] Time spent T 00000000000000000000000000000000 P 667a34b27a39474ebefc1af3cf5b4271: Append to log took a long time: real 2.345s	user 0.003s	sys 0.010s
I20260709 13:21:01.908418 16295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f5f89186204242bf377f7c42b2e2da" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:21:01.911593 16295 raft_consensus.cc:2368] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b for earlier term 12. Current term is 13.
I20260709 13:21:01.912716 16295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2794af5b99984efebdee564035841cb0" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:01.915246 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e4bfb9090c8460eb14fa7e0d818e7f1" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
W20260709 13:21:01.924000 20795 log.cc:927] Time spent T 8a878999c0024d2abe517484180290fc P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 1.610s	user 0.003s	sys 0.003s
I20260709 13:21:01.924152 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5d59407eb545b895d8d922e0dd69fd" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 14 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:21:01.927037 16226 raft_consensus.cc:2393] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 in current term 14: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
W20260709 13:21:01.928036 16264 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:01.925724 16823 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49896 (request call id 1183) took 2290 ms. Trace:
I20260709 13:21:01.928596 16823 rpcz_store.cc:276] 0709 13:20:59.634868 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:00.221080 (+586212us) service_pool.cc:224] Handling call
0709 13:21:00.256167 (+ 35087us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:21:00.536098 (+279931us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:01.925094 (+1388996us) spinlock_profiling.cc:243] Waited 221 ms on lock 0x7b5400520dd4. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:21:01.925513 (+   419us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":507473664}
I20260709 13:21:01.930236 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b7d1ab519044eda2d5e04a07242f25" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:21:01.932794 16226 raft_consensus.cc:1275] T 08c74490c40243a88e62d1760b5cd814 P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 13:21:01.936830 16818 raft_consensus.cc:1240] T af57996a8be84d398f0f023e8d08ef77 P cb82cc7bd6db46d980c75237a1402860 [term 13 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 12. Current term is 13. Ops: []
W20260709 13:21:01.938414 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 184637us
I20260709 13:21:01.940330 16190 leader_election.cc:304] T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 647a343a542d4a139fe106c19095129b
I20260709 13:21:01.941965 20827 raft_consensus.cc:2804] T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:21:01.942055 16223 raft_consensus.cc:2468] T af57996a8be84d398f0f023e8d08ef77 P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 13.
I20260709 13:21:01.942966 20827 raft_consensus.cc:697] T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac [term 13 LEADER]: Becoming Leader. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:21:01.943323 20672 raft_consensus.cc:493] T af57996a8be84d398f0f023e8d08ef77 P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:01.944000 20672 raft_consensus.cc:515] T af57996a8be84d398f0f023e8d08ef77 P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:01.951601 16825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66ae176961b74813a58bc15c9c4c6f2a" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:21:01.953809 16298 raft_consensus.cc:2410] T ecc1e363230043fe9fef390f0b979f20 P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 for term 12 because replica has last-logged OpId of term: 11 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 7.
I20260709 13:21:01.953542 16151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c74490c40243a88e62d1760b5cd814" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:01.954630 16151 raft_consensus.cc:2468] T 08c74490c40243a88e62d1760b5cd814 P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 11.
I20260709 13:21:01.954898 20618 raft_consensus.cc:3055] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:21:01.955523 20618 raft_consensus.cc:740] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:21:01.955866 20594 raft_consensus.cc:493] T ecc1e363230043fe9fef390f0b979f20 P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:01.956223 16144 raft_consensus.cc:1240] T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 10. Current term is 11. Ops: [10.10-10.10]
I20260709 13:21:01.956673 20594 raft_consensus.cc:515] T ecc1e363230043fe9fef390f0b979f20 P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:01.957547 16146 raft_consensus.cc:1240] T c0eab25ce4c64e018a041beeda2867db P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 11. Current term is 12. Ops: [11.7-11.7]
I20260709 13:21:01.957018 20618 consensus_queue.cc:260] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:01.958599 20618 raft_consensus.cc:3060] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:21:01.959331 16191 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:01.960657 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1df93779209748b9860f6d893dd3de53" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:01.944629 20827 consensus_queue.cc:237] T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 11.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:01.962343 16143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5d59407eb545b895d8d922e0dd69fd" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 14 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:21:01.977656 20750 raft_consensus.cc:493] T 84f2da98864b4425bf7c0055951fdd4b P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:21:01.977735 20832 consensus_queue.cc:1048] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:21:01.963579 20592 raft_consensus.cc:493] T 18ad7e3af5504571a69d10c00750a01c P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:21:01.971731 16820 raft_consensus.cc:2410] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b for term 16 because replica has last-logged OpId of term: 15 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 10.
I20260709 13:21:01.997848 16226 raft_consensus.cc:1275] T 5a96373bb0e14dce8a266b2fc6c58507 P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 10 index: 5. Preceding OpId from leader: term: 12 index: 6. (index mismatch)
I20260709 13:21:01.976282 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68a6838d82ef4cac83507f906139d90f" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:21:01.978508 16224 raft_consensus.cc:3060] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:21:01.969864 16295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cc959fb7f704fba9009c0595467d074" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:01.937040 16225 raft_consensus.cc:1275] T 829b4ad8c5a54eababeea1181b960d4b P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:21:01.988983 16190 leader_election.cc:304] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:21:01.990520 16299 raft_consensus.cc:2468] T 5a96373bb0e14dce8a266b2fc6c58507 P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 12.
I20260709 13:21:01.993974 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87ac0d1f7e8a419f9bcbc04aee0e067e" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 14 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:01.973757 16294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fffab26427e24d398e2695aad40ca134" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:02.001471 20592 raft_consensus.cc:515] T 18ad7e3af5504571a69d10c00750a01c P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } }
I20260709 13:21:01.985455 20762 raft_consensus.cc:493] T 02838c86531d4f98bf58d1c22b8baf77 P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:02.002296 16293 raft_consensus.cc:1240] T 66ae176961b74813a58bc15c9c4c6f2a P 90f3da57796b488c9087a2f523adf045 [term 15 LEADER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 14. Current term is 15. Ops: [14.11-14.11]
I20260709 13:21:02.003077 20762 raft_consensus.cc:515] T 02838c86531d4f98bf58d1c22b8baf77 P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:02.004638 20592 leader_election.cc:290] T 18ad7e3af5504571a69d10c00750a01c P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:02.004760 16300 raft_consensus.cc:1240] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 10. Current term is 11. Ops: []
I20260709 13:21:02.001655 20750 raft_consensus.cc:515] T 84f2da98864b4425bf7c0055951fdd4b P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:01.977896 16265 leader_election.cc:304] T 8158bbd17b994d2eb96f560bd0f1b88d P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac
W20260709 13:21:02.010794 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 68368.5us
I20260709 13:21:02.011901 16291 raft_consensus.cc:2468] T 31969392ccdb43b491911a319e8bc7e1 P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 12.
I20260709 13:21:01.914197 20797 raft_consensus.cc:2764] T e4dc6b0541ca431ebc757beac00521f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Leader pre-election decision vote started in defunct term 11: won
I20260709 13:21:02.019372 16294 raft_consensus.cc:2393] T fffab26427e24d398e2695aad40ca134 P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 10: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
I20260709 13:21:02.021268 16296 raft_consensus.cc:2410] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac for term 12 because replica has last-logged OpId of term: 10 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 8.
I20260709 13:21:02.016162 20592 raft_consensus.cc:2804] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:21:02.023190 20592 raft_consensus.cc:493] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:02.023869 20592 raft_consensus.cc:3060] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:21:02.028294 20667 raft_consensus.cc:493] T 8c11074579b046d99e585c2818e18a2c P cb82cc7bd6db46d980c75237a1402860 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:02.028947 20667 raft_consensus.cc:515] T 8c11074579b046d99e585c2818e18a2c P cb82cc7bd6db46d980c75237a1402860 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:02.031752 20667 leader_election.cc:290] T 8c11074579b046d99e585c2818e18a2c P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:02.032387 16291 raft_consensus.cc:1240] T b87147731a68400e8cce9b879daa7c5d P 90f3da57796b488c9087a2f523adf045 [term 14 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 13. Current term is 14. Ops: []
I20260709 13:21:02.034256 16291 raft_consensus.cc:1275] T 5a96373bb0e14dce8a266b2fc6c58507 P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 10 index: 5. Preceding OpId from leader: term: 12 index: 6. (index mismatch)
I20260709 13:21:02.039464 20672 leader_election.cc:290] T af57996a8be84d398f0f023e8d08ef77 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:02.039566 20760 leader_election.cc:290] T c2f5f89186204242bf377f7c42b2e2da P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 13 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:02.043358 16293 raft_consensus.cc:1240] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [term 14 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 13. Current term is 14. Ops: []
I20260709 13:21:02.046079 16143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee9ed67006144797b1de6ea8c2013a68" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:02.046402 16222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02838c86531d4f98bf58d1c22b8baf77" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:02.046551 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee9ed67006144797b1de6ea8c2013a68" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:02.047220 16143 raft_consensus.cc:2393] T ee9ed67006144797b1de6ea8c2013a68 P 647a343a542d4a139fe106c19095129b [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 10: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
I20260709 13:21:02.047509 16222 raft_consensus.cc:2410] T 02838c86531d4f98bf58d1c22b8baf77 P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b for term 13 because replica has last-logged OpId of term: 11 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 6.
I20260709 13:21:02.048534 16222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d16dfad0c3b540e5b4b86492ddf87208" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 13 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:02.053789 16790 leader_election.cc:304] T af57996a8be84d398f0f023e8d08ef77 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
W20260709 13:21:02.060025 20834 log.cc:927] Time spent T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 0.725s	user 0.001s	sys 0.003s
I20260709 13:21:02.060582 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c61a39fd4f84424b129c1a6f9ba404d" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:02.061430 16149 raft_consensus.cc:2468] T 9c61a39fd4f84424b129c1a6f9ba404d P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 12.
I20260709 13:21:02.062474 16142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c2095a5f4ad49408dbc7f69c606120d" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:02.062896 16145 raft_consensus.cc:1240] T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b [term 14 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 13. Current term is 14. Ops: [13.9-13.9]
I20260709 13:21:02.064370 20602 consensus_queue.cc:1048] T 829b4ad8c5a54eababeea1181b960d4b P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:21:02.065052 16142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a959dbecc2648eb85d1cc550f5a3245" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:21:02.066082 16148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18ad7e3af5504571a69d10c00750a01c" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:02.066802 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af57996a8be84d398f0f023e8d08ef77" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:02.067405 20427 consensus_queue.cc:1048] T 5a96373bb0e14dce8a266b2fc6c58507 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:21:02.067740 16150 raft_consensus.cc:2468] T af57996a8be84d398f0f023e8d08ef77 P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 13.
I20260709 13:21:02.068908 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f5f89186204242bf377f7c42b2e2da" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:21:02.071216 16819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e4bfb9090c8460eb14fa7e0d818e7f1" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:21:02.071578 16823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c11074579b046d99e585c2818e18a2c" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:21:02.072561 16823 raft_consensus.cc:2393] T 8c11074579b046d99e585c2818e18a2c P cb82cc7bd6db46d980c75237a1402860 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 12: Already voted for candidate cb82cc7bd6db46d980c75237a1402860 in this term.
I20260709 13:21:02.072454 20696 raft_consensus.cc:2749] T af57996a8be84d398f0f023e8d08ef77 P cb82cc7bd6db46d980c75237a1402860 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:21:02.073819 16818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada5a004ba7c46819051e3ccec71ffdb" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:02.073997 16825 raft_consensus.cc:3055] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:21:02.074461 16825 raft_consensus.cc:740] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:21:02.074599 16818 raft_consensus.cc:2468] T ada5a004ba7c46819051e3ccec71ffdb P cb82cc7bd6db46d980c75237a1402860 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 12.
I20260709 13:21:02.074946 16145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806d7690c2b644959152074bb2cab0a7" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:21:02.075696 20236 raft_consensus.cc:493] T 8a878999c0024d2abe517484180290fc P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:02.075872 16145 raft_consensus.cc:3060] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:21:02.076236 16295 raft_consensus.cc:2393] T 8cc959fb7f704fba9009c0595467d074 P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 12: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
I20260709 13:21:02.076395 20236 raft_consensus.cc:515] T 8a878999c0024d2abe517484180290fc P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:02.072781 16819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26354361e9b044cea1c2b86c4882fe92" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:02.087517 16298 raft_consensus.cc:2468] T 68a6838d82ef4cac83507f906139d90f P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 13.
I20260709 13:21:02.089577 16292 raft_consensus.cc:2468] T e4dc6b0541ca431ebc757beac00521f1 P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 12.
I20260709 13:21:02.088524 20236 leader_election.cc:290] T 8a878999c0024d2abe517484180290fc P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:02.090221 20762 leader_election.cc:290] T 02838c86531d4f98bf58d1c22b8baf77 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:02.090703 16190 leader_election.cc:304] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:21:02.091507 20388 raft_consensus.cc:3055] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:21:02.091609 16825 consensus_queue.cc:260] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:02.092828 16819 raft_consensus.cc:2468] T 26354361e9b044cea1c2b86c4882fe92 P cb82cc7bd6db46d980c75237a1402860 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 10.
I20260709 13:21:02.093477 16822 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49896 (request call id 1199) took 1213 ms. Trace:
I20260709 13:21:02.095431 16790 leader_election.cc:304] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:21:02.110620 16818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a378d7f796f94615b3b347119a9880fd" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:21:02.050874 16222 raft_consensus.cc:1240] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 14. Current term is 15. Ops: [12.10-14.11]
I20260709 13:21:02.097391 16218 raft_consensus.cc:2468] T 8c11074579b046d99e585c2818e18a2c P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 12.
I20260709 13:21:02.075875 16226 raft_consensus.cc:1240] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 15. Current term is 16. Ops: [15.11-15.11]
I20260709 13:21:02.123447 20388 raft_consensus.cc:740] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:21:02.136669 16191 leader_election.cc:304] T 08c74490c40243a88e62d1760b5cd814 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
W20260709 13:21:02.138842 16223 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33050 (request call id 766) took 3221 ms (client timeout 1880 ms). Trace:
W20260709 13:21:02.139349 16223 rpcz_store.cc:269] 0709 13:20:58.917807 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:58.917947 (+   140us) service_pool.cc:224] Handling call
0709 13:21:02.138582 (+3220635us) spinlock_profiling.cc:243] Waited 2.97 s on lock 0x7b5400395654. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:21:02.138763 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6828654336}
I20260709 13:21:02.141185 16223 raft_consensus.cc:1275] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 13:21:02.143189 16825 raft_consensus.cc:3060] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:21:02.145061 20750 leader_election.cc:290] T 84f2da98864b4425bf7c0055951fdd4b P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:02.148587 20832 raft_consensus.cc:493] T 02838c86531d4f98bf58d1c22b8baf77 P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:02.149577 20832 raft_consensus.cc:515] T 02838c86531d4f98bf58d1c22b8baf77 P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:21:02.151523 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 173965us
W20260709 13:21:02.156019 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 60766.5us
I20260709 13:21:02.156931 16824 raft_consensus.cc:2468] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 14.
W20260709 13:21:02.157974 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 141850us
I20260709 13:21:02.164687 20661 raft_consensus.cc:2804] T 8158bbd17b994d2eb96f560bd0f1b88d P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:21:02.165352 20661 raft_consensus.cc:493] T 8158bbd17b994d2eb96f560bd0f1b88d P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:21:02.165911 20661 raft_consensus.cc:3060] T 8158bbd17b994d2eb96f560bd0f1b88d P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:21:02.167239 20667 raft_consensus.cc:2764] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 [term 13 FOLLOWER]: Leader pre-election decision vote started in defunct term 12: won
I20260709 13:21:02.168947 20388 consensus_queue.cc:260] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.12, Last appended by leader: 12, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:02.170827 20388 raft_consensus.cc:3060] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:21:02.161546 16824 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49892 (request call id 674) took 1176 ms. Trace:
I20260709 13:21:02.182248 16824 rpcz_store.cc:276] 0709 13:21:00.981457 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:01.869550 (+888093us) service_pool.cc:224] Handling call
0709 13:21:02.157567 (+288017us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:21:02.184026 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a378d7f796f94615b3b347119a9880fd" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:21:02.188228 16224 raft_consensus.cc:2468] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 11.
I20260709 13:21:02.190934 16190 leader_election.cc:304] T d2ab0ac8347641e6955e9bce7d90c1a2 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:21:02.195556 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a878999c0024d2abe517484180290fc" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:02.196077 16218 raft_consensus.cc:1240] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 12. Current term is 13. Ops: [12.9-12.9]
I20260709 13:21:02.196305 20832 leader_election.cc:290] T 02838c86531d4f98bf58d1c22b8baf77 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:21:02.196465 16224 raft_consensus.cc:2468] T 8a878999c0024d2abe517484180290fc P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 11.
I20260709 13:21:02.198158 16218 raft_consensus.cc:1275] T 746aabf856ab401c9ef5e859371e7b67 P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 13:21:02.110829 16822 rpcz_store.cc:276] 0709 13:21:00.825345 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:00.912267 (+ 86922us) service_pool.cc:224] Handling call
0709 13:21:00.912640 (+   373us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:21:00.959739 (+ 47099us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:21:00.959792 (+    53us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:21:00.972751 (+ 12959us) log.cc:844] Serialized 52 byte log entry
0709 13:21:01.627698 (+654947us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:21:01.878197 (+250499us) log.cc:844] Serialized 14 byte log entry
0709 13:21:02.038035 (+159838us) spinlock_profiling.cc:243] Waited 5.44 ms on lock 0x7b1800473c80. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b693ade1 00007f69b69475b1 00007f69ad9ecc0a 00007f69b712c219 00007f69ada74a40 00007f69ada6f4e4 00007f69ada6f455 00007f69b45573c7 00007f69b0b699a7 00007f69b0b790bf 00007f69b0b79038
0709 13:21:02.038310 (+   275us) log.cc:844] Serialized 14 byte log entry
0709 13:21:02.038618 (+   308us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:02.038731 (+   113us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:21:02.038890 (+   159us) raft_consensus.cc:1685] finished
0709 13:21:02.038906 (+    16us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:21:02.039105 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":142799,"prepare.queue_time_us":429858,"prepare.run_cpu_time_us":128,"prepare.run_wall_time_us":1323,"raft.queue_time_us":1839,"raft.run_cpu_time_us":661,"raft.run_wall_time_us":5657,"spinlock_wait_cycles":12521344,"thread_start_us":811471,"threads_started":2,"wal-append.queue_time_us":587812}
I20260709 13:21:02.171648 20797 raft_consensus.cc:2804] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:21:02.203549 20797 raft_consensus.cc:493] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:02.195590 20592 raft_consensus.cc:515] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:02.204093 20797 raft_consensus.cc:3060] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:21:02.205717 20592 leader_election.cc:290] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 13 election: Requested vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:02.208390 16218 raft_consensus.cc:1275] T e4dc6b0541ca431ebc757beac00521f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 10 index: 5. Preceding OpId from leader: term: 12 index: 6. (index mismatch)
W20260709 13:21:02.217921 16192 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:02.224025 16217 raft_consensus.cc:3060] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:21:02.236382 16226 raft_consensus.cc:1240] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 10. Current term is 11. Ops: []
I20260709 13:21:02.222040 20470 raft_consensus.cc:2804] T 08c74490c40243a88e62d1760b5cd814 P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:21:02.244825 16226 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33050 (request call id 790) took 1034 ms. Trace:
I20260709 13:21:02.248318 16822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af57996a8be84d398f0f023e8d08ef77" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:02.245436 16226 rpcz_store.cc:276] 0709 13:21:01.209964 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:02.135083 (+925119us) service_pool.cc:224] Handling call
0709 13:21:02.222928 (+ 87845us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:21:02.244457 (+ 21529us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:02.244740 (+   283us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:21:02.254498 16145 raft_consensus.cc:2468] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 12.
I20260709 13:21:02.249287 16822 raft_consensus.cc:2468] T af57996a8be84d398f0f023e8d08ef77 P cb82cc7bd6db46d980c75237a1402860 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 13.
I20260709 13:21:02.265087 16265 leader_election.cc:304] T 7c2095a5f4ad49408dbc7f69c606120d P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:21:02.247124 20470 raft_consensus.cc:493] T 08c74490c40243a88e62d1760b5cd814 P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:02.278126 16790 leader_election.cc:304] T 8638ee363273426db1393dd81e5c7b72 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 
W20260709 13:21:02.284752 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 82426.8us
I20260709 13:21:02.286461 16191 leader_election.cc:304] T 1df93779209748b9860f6d893dd3de53 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:21:02.287955 20822 consensus_queue.cc:1059] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 7, Time since last communication: 0.001s
W20260709 13:21:02.293437 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 46090.6us
W20260709 13:21:02.315320 16192 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 94530.8us
I20260709 13:21:02.287889 16116 leader_election.cc:304] T 8a878999c0024d2abe517484180290fc P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
I20260709 13:21:02.279542 20470 raft_consensus.cc:3060] T 08c74490c40243a88e62d1760b5cd814 P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:21:02.322345 20839 raft_consensus.cc:2804] T 8a878999c0024d2abe517484180290fc P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:21:02.322853 20839 raft_consensus.cc:493] T 8a878999c0024d2abe517484180290fc P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:02.331909 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f2da98864b4425bf7c0055951fdd4b" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:02.323309 20839 raft_consensus.cc:3060] T 8a878999c0024d2abe517484180290fc P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:21:02.318373 20851 consensus_meta.cc:287] T fffab26427e24d398e2695aad40ca134 P 1ca2725d290e47d68b1a4b609c9816ac: Time spent flushing consensus metadata: real 0.583s	user 0.011s	sys 0.006s
I20260709 13:21:02.339823 20780 leader_election.cc:290] T 23f82224f4564fea93572e8206bad1fe P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
W20260709 13:21:02.349529 16115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 163917us
I20260709 13:21:02.341408 20823 leader_election.cc:290] T 02838c86531d4f98bf58d1c22b8baf77 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
W20260709 13:21:02.341996 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 55739.6us
I20260709 13:21:02.344697 16817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e65676b21b748728ad39668cfe722ec" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:02.339582 16220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee9ed67006144797b1de6ea8c2013a68" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
W20260709 13:21:02.334599 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 162360us
I20260709 13:21:02.318344 20267 raft_consensus.cc:2749] T 7c2095a5f4ad49408dbc7f69c606120d P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:21:02.331123 20672 raft_consensus.cc:3055] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:21:02.323894 20822 consensus_queue.cc:1059] T c0eab25ce4c64e018a041beeda2867db P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.028s
W20260709 13:21:02.382967 16266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 224713us
W20260709 13:21:02.389362 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 158760us
I20260709 13:21:02.391626 16221 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57600 (request call id 1541) took 1241 ms. Trace:
I20260709 13:21:02.392802 16220 raft_consensus.cc:2468] T ee9ed67006144797b1de6ea8c2013a68 P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 9.
I20260709 13:21:02.393205 16191 leader_election.cc:304] T 7c2095a5f4ad49408dbc7f69c606120d P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:21:02.396622 20823 consensus_queue.cc:1048] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:21:02.397354 16190 leader_election.cc:304] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:21:02.398325 20836 consensus_queue.cc:1048] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:21:02.399353 16817 raft_consensus.cc:2468] T 9e65676b21b748728ad39668cfe722ec P cb82cc7bd6db46d980c75237a1402860 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 11.
I20260709 13:21:02.349153 20592 raft_consensus.cc:2749] T 1df93779209748b9860f6d893dd3de53 P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:21:02.363993 16790 leader_election.cc:304] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:21:02.408288 16225 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57604 (request call id 2669) took 1228 ms. Trace:
I20260709 13:21:02.359056 20641 raft_consensus.cc:2804] T 8638ee363273426db1393dd81e5c7b72 P cb82cc7bd6db46d980c75237a1402860 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:21:02.378077 20750 consensus_queue.cc:1048] T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.083s
W20260709 13:21:02.427755 16192 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 78387.1us
I20260709 13:21:02.406630 20679 raft_consensus.cc:493] T 57a744f5aaf944678a63469cbf514015 P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
W20260709 13:21:02.448210 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 127053us
I20260709 13:21:02.433621 16220 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33050 (request call id 802) took 1075 ms. Trace:
I20260709 13:21:02.359884 20672 raft_consensus.cc:740] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:21:02.382462 20780 raft_consensus.cc:493] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:21:02.476215 16820 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49896 (request call id 1194) took 1837 ms (client timeout 2142 ms). Trace:
W20260709 13:21:02.438611 16791 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:02.435803 16225 rpcz_store.cc:276] 0709 13:21:01.179839 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:02.009658 (+829819us) service_pool.cc:224] Handling call
0709 13:21:02.041080 (+ 31422us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:21:02.041599 (+   519us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:21:02.124114 (+ 82515us) log.cc:844] Serialized 14 byte log entry
0709 13:21:02.381928 (+257814us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:21:02.382013 (+    85us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:21:02.407138 (+ 25125us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:02.407701 (+   563us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:21:02.408215 (+   514us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10619,"prepare.queue_time_us":81238,"prepare.run_cpu_time_us":82,"prepare.run_wall_time_us":83,"raft.queue_time_us":19313,"raft.run_cpu_time_us":153,"raft.run_wall_time_us":6825,"thread_start_us":234580,"threads_started":2}
I20260709 13:21:02.463496 16265 leader_election.cc:304] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:21:02.422344 16221 rpcz_store.cc:276] 0709 13:21:01.149904 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:01.990054 (+840150us) service_pool.cc:224] Handling call
0709 13:21:01.990611 (+   557us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:21:01.991411 (+   800us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:21:01.991456 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:21:01.992043 (+   587us) log.cc:844] Serialized 26 byte log entry
0709 13:21:02.108039 (+115996us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:21:02.108099 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:02.108201 (+   102us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:21:02.391235 (+283034us) raft_consensus.cc:1685] finished
0709 13:21:02.391257 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:21:02.391418 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1542,"raft.run_cpu_time_us":133,"raft.run_wall_time_us":133,"thread_start_us":115459,"threads_started":1,"wal-append.queue_time_us":287530}
I20260709 13:21:02.484766 16220 rpcz_store.cc:276] 0709 13:21:01.347957 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:02.323833 (+975876us) service_pool.cc:224] Handling call
0709 13:21:02.422969 (+ 99136us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:21:02.497123 16217 raft_consensus.cc:2468] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 14.
I20260709 13:21:02.499155 16217 pending_rounds.cc:85] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac: Aborting all ops after (but not including) 8
I20260709 13:21:02.498788 20672 consensus_queue.cc:260] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:02.499584 16217 pending_rounds.cc:107] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac: Aborting uncommitted NO_OP operation due to leader change: 9.9
I20260709 13:21:02.500013 16217 raft_consensus.cc:2889] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:21:02.482762 20641 raft_consensus.cc:493] T 8638ee363273426db1393dd81e5c7b72 P cb82cc7bd6db46d980c75237a1402860 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
W20260709 13:21:02.466837 20618 consensus_meta.cc:287] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac: Time spent flushing consensus metadata: real 0.508s	user 0.008s	sys 0.004s
W20260709 13:21:02.492555 16820 rpcz_store.cc:269] 0709 13:21:00.609801 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:00.609962 (+   161us) service_pool.cc:224] Handling call
0709 13:21:02.409057 (+1799095us) spinlock_profiling.cc:243] Waited 1.34 s on lock 0x7b54006423d4. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:21:02.445978 (+ 36921us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3083824512}
I20260709 13:21:02.431351 20827 raft_consensus.cc:2749] T 7c2095a5f4ad49408dbc7f69c606120d P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:21:02.510821 16825 raft_consensus.cc:1275] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860 [term 10 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 9. (term mismatch)
I20260709 13:21:02.517409 16220 raft_consensus.cc:1275] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 13:21:02.472208 20679 raft_consensus.cc:515] T 57a744f5aaf944678a63469cbf514015 P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:02.448985 20667 raft_consensus.cc:2749] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
W20260709 13:21:02.525833 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 128651us
I20260709 13:21:02.446260 20851 raft_consensus.cc:3060] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:21:02.500355 20672 raft_consensus.cc:3060] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:21:02.521131 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 128082us
I20260709 13:21:02.515543 16825 pending_rounds.cc:85] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860: Aborting all ops after (but not including) 8
I20260709 13:21:02.507434 20641 raft_consensus.cc:3060] T 8638ee363273426db1393dd81e5c7b72 P cb82cc7bd6db46d980c75237a1402860 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:21:02.526863 16825 pending_rounds.cc:107] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860: Aborting uncommitted NO_OP operation due to leader change: 9.9
I20260709 13:21:02.532282 16819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2794af5b99984efebdee564035841cb0" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:21:02.524989 20760 consensus_queue.cc:1048] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:21:02.525514 20711 raft_consensus.cc:2749] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:21:02.530026 16822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434bff3082d141838068f323bdb0d1ed" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:21:02.530431 20679 leader_election.cc:290] T 57a744f5aaf944678a63469cbf514015 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:02.532744 16825 raft_consensus.cc:2889] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:21:02.530484 16817 raft_consensus.cc:1240] T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860 [term 14 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 13. Current term is 14. Ops: [11.11-13.12]
I20260709 13:21:02.533075 16224 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57600 (request call id 1542) took 1254 ms. Trace:
I20260709 13:21:02.473251 20267 consensus_queue.cc:1048] T e5ad4eb281564936a59f41e387a47156 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.047s
I20260709 13:21:02.513183 20661 raft_consensus.cc:515] T 8158bbd17b994d2eb96f560bd0f1b88d P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:02.539670 16115 leader_election.cc:304] T 8158bbd17b994d2eb96f560bd0f1b88d P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:21:02.540125 16790 leader_election.cc:304] T 806d7690c2b644959152074bb2cab0a7 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:21:02.540354 20673 raft_consensus.cc:2804] T d2ab0ac8347641e6955e9bce7d90c1a2 P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:21:02.541173 20661 leader_election.cc:290] T 8158bbd17b994d2eb96f560bd0f1b88d P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 12 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:21:02.542167 16819 raft_consensus.cc:3060] T 2794af5b99984efebdee564035841cb0 P cb82cc7bd6db46d980c75237a1402860 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:21:02.543293 16820 raft_consensus.cc:1275] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [term 16 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 16 index: 11. (term mismatch)
I20260709 13:21:02.543685 16820 pending_rounds.cc:85] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860: Aborting all ops after (but not including) 10
W20260709 13:21:02.543784 16191 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:02.543964 16820 pending_rounds.cc:107] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860: Aborting uncommitted NO_OP operation due to leader change: 15.11
I20260709 13:21:02.544286 16820 raft_consensus.cc:2889] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:21:02.534685 20823 raft_consensus.cc:493] T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:21:02.546813 20823 raft_consensus.cc:515] T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:02.545694 20231 raft_consensus.cc:493] T e5ad4eb281564936a59f41e387a47156 P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:02.516496 20780 raft_consensus.cc:515] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:02.548498 20754 raft_consensus.cc:493] T b6f07842d3aa47858df5665874c86f3a P 647a343a542d4a139fe106c19095129b [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:02.546451 16117 leader_election.cc:304] T 746aabf856ab401c9ef5e859371e7b67 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860
W20260709 13:21:02.549823 16116 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:02.537019 16224 rpcz_store.cc:276] 0709 13:21:01.278536 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:02.197523 (+918987us) service_pool.cc:224] Handling call
0709 13:21:02.198075 (+   552us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:21:02.281117 (+ 83042us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:21:02.283404 (+  2287us) log.cc:844] Serialized 14 byte log entry
0709 13:21:02.361981 (+ 78577us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:21:02.407886 (+ 45905us) log.cc:844] Serialized 26 byte log entry
0709 13:21:02.408690 (+   804us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:21:02.408735 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:02.408816 (+    81us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:21:02.532882 (+124066us) raft_consensus.cc:1685] finished
0709 13:21:02.532901 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:21:02.533019 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":110782,"prepare.queue_time_us":2843,"prepare.run_cpu_time_us":74,"prepare.run_wall_time_us":75,"raft.queue_time_us":138347,"raft.run_cpu_time_us":266,"raft.run_wall_time_us":266,"thread_start_us":49270,"threads_started":1,"wal-append.queue_time_us":78958}
I20260709 13:21:02.562120 16822 raft_consensus.cc:3060] T 434bff3082d141838068f323bdb0d1ed P cb82cc7bd6db46d980c75237a1402860 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:21:02.541203 16790 leader_election.cc:304] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:21:02.555300 20754 raft_consensus.cc:515] T b6f07842d3aa47858df5665874c86f3a P 647a343a542d4a139fe106c19095129b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:02.567003 20434 raft_consensus.cc:2749] T 746aabf856ab401c9ef5e859371e7b67 P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:21:02.568797 16117 leader_election.cc:304] T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860
I20260709 13:21:02.568682 20839 raft_consensus.cc:515] T 8a878999c0024d2abe517484180290fc P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:02.570801 16266 leader_election.cc:304] T b87147731a68400e8cce9b879daa7c5d P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
W20260709 13:21:02.576121 20782 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 10 current term: 10 is higher.
I20260709 13:21:02.576514 16192 leader_election.cc:304] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
I20260709 13:21:02.578399 16821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57a744f5aaf944678a63469cbf514015" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:02.559705 20667 raft_consensus.cc:3060] T 806d7690c2b644959152074bb2cab0a7 P cb82cc7bd6db46d980c75237a1402860 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:21:02.583514 20679 raft_consensus.cc:2749] T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:21:02.548067 20231 raft_consensus.cc:515] T e5ad4eb281564936a59f41e387a47156 P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:21:02.585749 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 122442us
I20260709 13:21:02.587255 16820 raft_consensus.cc:1240] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860 [term 14 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 13. Current term is 14. Ops: []
I20260709 13:21:02.602113 16817 raft_consensus.cc:1275] T 829b4ad8c5a54eababeea1181b960d4b P cb82cc7bd6db46d980c75237a1402860 [term 12 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:21:02.590531 20696 raft_consensus.cc:2749] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:21:02.606468 16265 leader_election.cc:304] T 8cc959fb7f704fba9009c0595467d074 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac
I20260709 13:21:02.605490 16190 leader_election.cc:304] T 8a878999c0024d2abe517484180290fc P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 647a343a542d4a139fe106c19095129b
I20260709 13:21:02.607549 16822 raft_consensus.cc:2468] T 434bff3082d141838068f323bdb0d1ed P cb82cc7bd6db46d980c75237a1402860 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 12.
I20260709 13:21:02.611518 20472 raft_consensus.cc:2749] T 8158bbd17b994d2eb96f560bd0f1b88d P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:21:02.611945 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 70860us
I20260709 13:21:02.612960 16790 leader_election.cc:304] T ada5a004ba7c46819051e3ccec71ffdb P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:21:02.614061 16790 leader_election.cc:304] T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:21:02.615131 20799 raft_consensus.cc:2804] T ada5a004ba7c46819051e3ccec71ffdb P cb82cc7bd6db46d980c75237a1402860 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:21:02.615535 20799 raft_consensus.cc:493] T ada5a004ba7c46819051e3ccec71ffdb P cb82cc7bd6db46d980c75237a1402860 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:02.615854 20799 raft_consensus.cc:3060] T ada5a004ba7c46819051e3ccec71ffdb P cb82cc7bd6db46d980c75237a1402860 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:21:02.591032 20673 raft_consensus.cc:493] T d2ab0ac8347641e6955e9bce7d90c1a2 P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Starting leader election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:21:02.616724 20673 raft_consensus.cc:3060] T d2ab0ac8347641e6955e9bce7d90c1a2 P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:21:02.603668 20711 raft_consensus.cc:2749] T b87147731a68400e8cce9b879daa7c5d P 90f3da57796b488c9087a2f523adf045 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:21:02.594811 20827 raft_consensus.cc:2749] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:21:02.622470 16192 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 46122.4us
I20260709 13:21:02.623500 20470 raft_consensus.cc:515] T 08c74490c40243a88e62d1760b5cd814 P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:02.623813 20594 leader_election.cc:290] T ecc1e363230043fe9fef390f0b979f20 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:02.596493 20797 raft_consensus.cc:515] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:02.626222 16819 raft_consensus.cc:2468] T 2794af5b99984efebdee564035841cb0 P cb82cc7bd6db46d980c75237a1402860 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 10.
W20260709 13:21:02.630777 16266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 120341us
I20260709 13:21:02.634568 20696 raft_consensus.cc:2749] T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:21:02.635367 16192 leader_election.cc:304] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
I20260709 13:21:02.635452 20696 consensus_queue.cc:1048] T d2ab0ac8347641e6955e9bce7d90c1a2 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:21:02.636418 20267 raft_consensus.cc:2804] T 8cc959fb7f704fba9009c0595467d074 P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:21:02.637962 20757 consensus_queue.cc:1048] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.013s
I20260709 13:21:02.642140 16218 raft_consensus.cc:1240] T c2f5f89186204242bf377f7c42b2e2da P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 12. Current term is 13. Ops: []
I20260709 13:21:02.642223 20231 leader_election.cc:290] T e5ad4eb281564936a59f41e387a47156 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:02.643780 20822 raft_consensus.cc:2749] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:21:02.633577 20760 raft_consensus.cc:2804] T 8a878999c0024d2abe517484180290fc P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:21:02.631984 20780 leader_election.cc:290] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:21:02.639834 20797 leader_election.cc:290] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 14 election: Requested vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:02.656423 20760 raft_consensus.cc:697] T 8a878999c0024d2abe517484180290fc P 1ca2725d290e47d68b1a4b609c9816ac [term 11 LEADER]: Becoming Leader. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:21:02.653265 20667 raft_consensus.cc:2749] T 806d7690c2b644959152074bb2cab0a7 P cb82cc7bd6db46d980c75237a1402860 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:21:02.658939 16115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 119497us
I20260709 13:21:02.661799 16192 leader_election.cc:304] T ee9ed67006144797b1de6ea8c2013a68 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 647a343a542d4a139fe106c19095129b
I20260709 13:21:02.639278 20388 raft_consensus.cc:493] T 18ad7e3af5504571a69d10c00750a01c P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:21:02.639494 20602 consensus_queue.cc:1059] T ecc1e363230043fe9fef390f0b979f20 P 647a343a542d4a139fe106c19095129b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.020s
I20260709 13:21:02.653546 20267 raft_consensus.cc:697] T 8cc959fb7f704fba9009c0595467d074 P 90f3da57796b488c9087a2f523adf045 [term 12 LEADER]: Becoming Leader. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:21:02.667325 20388 raft_consensus.cc:515] T 18ad7e3af5504571a69d10c00750a01c P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } }
I20260709 13:21:02.669257 20470 leader_election.cc:290] T 08c74490c40243a88e62d1760b5cd814 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 12 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:02.672835 20592 raft_consensus.cc:3055] T c0eab25ce4c64e018a041beeda2867db P 1ca2725d290e47d68b1a4b609c9816ac [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:21:02.678089 20592 raft_consensus.cc:740] T c0eab25ce4c64e018a041beeda2867db P 1ca2725d290e47d68b1a4b609c9816ac [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:21:02.681056 20592 consensus_queue.cc:260] T c0eab25ce4c64e018a041beeda2867db P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:02.682479 20592 raft_consensus.cc:3060] T c0eab25ce4c64e018a041beeda2867db P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:21:02.683314 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 69424.7us
I20260709 13:21:02.686326 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6f07842d3aa47858df5665874c86f3a" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
W20260709 13:21:02.688850 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 92868.1us
I20260709 13:21:02.695482 20780 consensus_queue.cc:1059] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: INVALID_TERM, Last received: 11.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:21:02.696485 20754 leader_election.cc:290] T b6f07842d3aa47858df5665874c86f3a P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:02.702845 16221 raft_consensus.cc:1275] T 68a6838d82ef4cac83507f906139d90f P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 13:21:02.705060 16190 leader_election.cc:304] T 5a959dbecc2648eb85d1cc550f5a3245 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
W20260709 13:21:02.707954 20875 log.cc:927] Time spent T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 0.141s	user 0.003s	sys 0.000s
I20260709 13:21:02.708262 16266 leader_election.cc:304] T 57a744f5aaf944678a63469cbf514015 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
I20260709 13:21:02.708709 20231 raft_consensus.cc:3055] T ecc1e363230043fe9fef390f0b979f20 P 647a343a542d4a139fe106c19095129b [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:21:02.709309 20231 raft_consensus.cc:740] T ecc1e363230043fe9fef390f0b979f20 P 647a343a542d4a139fe106c19095129b [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:21:02.711354 20832 raft_consensus.cc:2749] T 57a744f5aaf944678a63469cbf514015 P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:21:02.710775 20797 raft_consensus.cc:2749] T 5a959dbecc2648eb85d1cc550f5a3245 P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
W20260709 13:21:02.712893 16789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 96292.4us
I20260709 13:21:02.698498 20602 consensus_queue.cc:1048] T 26354361e9b044cea1c2b86c4882fe92 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.019s
W20260709 13:21:02.718454 16192 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 59509.3us
I20260709 13:21:02.710763 16266 leader_election.cc:304] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:21:02.710573 20231 consensus_queue.cc:260] T ecc1e363230043fe9fef390f0b979f20 P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:02.724210 16192 leader_election.cc:304] T 829b4ad8c5a54eababeea1181b960d4b P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
I20260709 13:21:02.725024 20231 raft_consensus.cc:3060] T ecc1e363230043fe9fef390f0b979f20 P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:21:02.725569 16192 leader_election.cc:304] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860
I20260709 13:21:02.678960 20839 leader_election.cc:290] T 8a878999c0024d2abe517484180290fc P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 12 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:02.714973 20851 raft_consensus.cc:2749] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:21:02.728772 16192 leader_election.cc:304] T af57996a8be84d398f0f023e8d08ef77 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:21:02.733203 16819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada5a004ba7c46819051e3ccec71ffdb" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:21:02.733628 16822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9944d5e4b8c4109af9955c4f08c9095" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:21:02.736249 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5ad4eb281564936a59f41e387a47156" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:02.738190 16821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdbd78acb4cf46298e0de8ba0d130e19" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:21:02.738867 16821 raft_consensus.cc:3060] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:21:02.738894 20592 raft_consensus.cc:493] T 746aabf856ab401c9ef5e859371e7b67 P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:02.739810 20592 raft_consensus.cc:515] T 746aabf856ab401c9ef5e859371e7b67 P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:02.743589 16221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a878999c0024d2abe517484180290fc" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:21:02.744876 16817 raft_consensus.cc:1240] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 [term 11 LEADER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 10. Current term is 11. Ops: [10.10-10.10]
I20260709 13:21:02.745721 20236 consensus_queue.cc:1059] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:21:02.745783 20472 consensus_queue.cc:1048] T 68a6838d82ef4cac83507f906139d90f P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:21:02.718854 20667 raft_consensus.cc:3055] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:21:02.696589 20780 consensus_queue.cc:1048] T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.038s
I20260709 13:21:02.754701 16218 raft_consensus.cc:1240] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 10. Current term is 11. Ops: []
I20260709 13:21:02.734650 20832 raft_consensus.cc:2764] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Leader pre-election decision vote started in defunct term 11: won
I20260709 13:21:02.757846 16822 raft_consensus.cc:2376] T c9944d5e4b8c4109af9955c4f08c9095 P cb82cc7bd6db46d980c75237a1402860 [term 13 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 13. Re-sending same reply.
I20260709 13:21:02.759245 16115 leader_election.cc:304] T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:21:02.760212 16789 leader_election.cc:304] T 434bff3082d141838068f323bdb0d1ed P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
W20260709 13:21:02.760039 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 131013us
I20260709 13:21:02.759152 16817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746aabf856ab401c9ef5e859371e7b67" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:02.764314 16821 raft_consensus.cc:2468] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 14.
I20260709 13:21:02.747567 20470 raft_consensus.cc:2804] T af57996a8be84d398f0f023e8d08ef77 P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:21:02.751864 20236 raft_consensus.cc:3055] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:21:02.752027 20571 raft_consensus.cc:493] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:02.749869 20667 raft_consensus.cc:740] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:21:02.767692 16192 leader_election.cc:304] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 90f3da57796b488c9087a2f523adf045
I20260709 13:21:02.765353 20470 raft_consensus.cc:493] T af57996a8be84d398f0f023e8d08ef77 P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:02.773638 20470 raft_consensus.cc:3060] T af57996a8be84d398f0f023e8d08ef77 P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:21:02.780280 20641 raft_consensus.cc:515] T 8638ee363273426db1393dd81e5c7b72 P cb82cc7bd6db46d980c75237a1402860 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:02.781733 20667 consensus_queue.cc:260] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:02.767104 20236 raft_consensus.cc:740] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:21:02.784261 20851 raft_consensus.cc:2804] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:21:02.784039 20236 consensus_queue.cc:260] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:02.784817 20851 raft_consensus.cc:697] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac [term 13 LEADER]: Becoming Leader. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:21:02.785211 20236 raft_consensus.cc:3060] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:21:02.785862 20851 consensus_queue.cc:237] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:02.772418 20571 raft_consensus.cc:515] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:02.776719 20267 consensus_queue.cc:237] T 8cc959fb7f704fba9009c0595467d074 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:02.790097 20470 raft_consensus.cc:515] T af57996a8be84d398f0f023e8d08ef77 P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:02.791208 20886 raft_consensus.cc:2804] T ee9ed67006144797b1de6ea8c2013a68 P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:21:02.791647 20886 raft_consensus.cc:493] T ee9ed67006144797b1de6ea8c2013a68 P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: Starting leader election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:21:02.792029 20886 raft_consensus.cc:3060] T ee9ed67006144797b1de6ea8c2013a68 P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:21:02.773766 20839 raft_consensus.cc:2749] T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:21:02.783046 20667 raft_consensus.cc:3060] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:21:02.800894 20641 leader_election.cc:290] T 8638ee363273426db1393dd81e5c7b72 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 14 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:02.819188 20760 consensus_queue.cc:237] T 8a878999c0024d2abe517484180290fc P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:02.820959 20388 leader_election.cc:290] T 18ad7e3af5504571a69d10c00750a01c P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
W20260709 13:21:02.826220 16266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 71096.8us
I20260709 13:21:02.787331 20892 consensus_queue.cc:1048] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.023s
I20260709 13:21:02.826740 20887 raft_consensus.cc:2749] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:21:02.837675 20571 leader_election.cc:290] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:02.855716 16221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18ad7e3af5504571a69d10c00750a01c" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
W20260709 13:21:02.855827 16115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 119845us
W20260709 13:21:02.852612 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 104638us
W20260709 13:21:02.853981 16789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 98567.8us
I20260709 13:21:02.859841 16822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2372981006d4b218215c28964dea623" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 14 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:02.860651 16822 raft_consensus.cc:2410] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac for term 14 because replica has last-logged OpId of term: 12 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 8.
I20260709 13:21:02.854104 20902 raft_consensus.cc:2749] T 434bff3082d141838068f323bdb0d1ed P cb82cc7bd6db46d980c75237a1402860 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:21:02.869902 16790 leader_election.cc:304] T ee9ed67006144797b1de6ea8c2013a68 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:21:02.872182 16265 leader_election.cc:304] T 9e65676b21b748728ad39668cfe722ec P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:21:02.875283 20894 consensus_queue.cc:1048] T c0eab25ce4c64e018a041beeda2867db P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.019s
I20260709 13:21:02.822764 20799 raft_consensus.cc:515] T ada5a004ba7c46819051e3ccec71ffdb P cb82cc7bd6db46d980c75237a1402860 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:02.879981 16266 leader_election.cc:304] T ada5a004ba7c46819051e3ccec71ffdb P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:21:02.878448 16789 leader_election.cc:304] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 647a343a542d4a139fe106c19095129b
I20260709 13:21:02.891199 16221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "829b4ad8c5a54eababeea1181b960d4b" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:21:02.851560 20673 raft_consensus.cc:515] T d2ab0ac8347641e6955e9bce7d90c1a2 P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:02.896363 16819 raft_consensus.cc:2393] T ada5a004ba7c46819051e3ccec71ffdb P cb82cc7bd6db46d980c75237a1402860 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 13: Already voted for candidate cb82cc7bd6db46d980c75237a1402860 in this term.
I20260709 13:21:02.900899 16151 raft_consensus.cc:1275] T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:21:02.901232 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c11074579b046d99e585c2818e18a2c" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 13 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:02.902494 16224 raft_consensus.cc:2410] T 8c11074579b046d99e585c2818e18a2c P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 for term 13 because replica has last-logged OpId of term: 11 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 10.
I20260709 13:21:02.904608 16219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2372981006d4b218215c28964dea623" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:02.875896 20782 raft_consensus.cc:2804] T ee9ed67006144797b1de6ea8c2013a68 P cb82cc7bd6db46d980c75237a1402860 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:21:02.853140 20903 consensus_queue.cc:1059] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.039s
I20260709 13:21:02.905228 16219 raft_consensus.cc:2468] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 13.
I20260709 13:21:02.909490 16115 leader_election.cc:400] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 12 election: Vote denied by peer 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389) with higher term. Message: Invalid argument: T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b for earlier term 12. Current term is 13.
I20260709 13:21:02.911667 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8638ee363273426db1393dd81e5c7b72" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:21:02.917309 20895 consensus_queue.cc:1048] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.039s
I20260709 13:21:02.918947 20894 raft_consensus.cc:2804] T 9e65676b21b748728ad39668cfe722ec P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:21:02.919405 20894 raft_consensus.cc:493] T 9e65676b21b748728ad39668cfe722ec P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:02.919746 20894 raft_consensus.cc:3060] T 9e65676b21b748728ad39668cfe722ec P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:21:02.920256 20799 leader_election.cc:290] T ada5a004ba7c46819051e3ccec71ffdb P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 13 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
W20260709 13:21:02.923453 16789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 45132.4us
I20260709 13:21:02.924762 16221 raft_consensus.cc:1240] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 13. Current term is 14. Ops: [13.10-13.10]
I20260709 13:21:02.924998 16789 leader_election.cc:304] T 2794af5b99984efebdee564035841cb0 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:21:02.925729 20890 raft_consensus.cc:2764] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860 [term 12 FOLLOWER]: Leader pre-election decision vote started in defunct term 11: won
I20260709 13:21:02.908104 20782 raft_consensus.cc:493] T ee9ed67006144797b1de6ea8c2013a68 P cb82cc7bd6db46d980c75237a1402860 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:02.927568 16143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f82224f4564fea93572e8206bad1fe" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:02.928609 16143 raft_consensus.cc:2393] T 23f82224f4564fea93572e8206bad1fe P 647a343a542d4a139fe106c19095129b [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 in current term 10: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
I20260709 13:21:02.930138 16151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2372981006d4b218215c28964dea623" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:02.931085 16151 raft_consensus.cc:2468] T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 13.
I20260709 13:21:02.926967 20782 raft_consensus.cc:3060] T ee9ed67006144797b1de6ea8c2013a68 P cb82cc7bd6db46d980c75237a1402860 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:21:02.941933 20884 log.cc:927] Time spent T d2ab0ac8347641e6955e9bce7d90c1a2 P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 0.411s	user 0.002s	sys 0.000s
I20260709 13:21:02.944387 20673 leader_election.cc:290] T d2ab0ac8347641e6955e9bce7d90c1a2 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 13 election: Requested vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:02.946414 16816 raft_consensus.cc:1275] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860 [term 12 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:21:02.949661 16223 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33050 (request call id 798) took 1736 ms. Trace:
I20260709 13:21:02.950191 16223 rpcz_store.cc:276] 0709 13:21:01.213220 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:02.148945 (+935725us) service_pool.cc:224] Handling call
0709 13:21:02.149480 (+   535us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:21:02.273963 (+124483us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:21:02.328258 (+ 54295us) log.cc:844] Serialized 14 byte log entry
0709 13:21:02.519597 (+191339us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:21:02.524917 (+  5320us) log.cc:844] Serialized 26 byte log entry
0709 13:21:02.525089 (+   172us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:21:02.525136 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:02.525215 (+    79us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:21:02.941069 (+415854us) spinlock_profiling.cc:243] Waited 63.2 ms on lock 0x7b4400022b4c. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69ad8fadb1 00007f69ad998c85 00007f69ad9996a2 00007f69b070b1db 00007f69b0706aaf 00007f69b06fa86e 00007f69b06f9bb6 00007f69b06f97d8 00007f69b0703221 00007f69b07031d9 00007f69b0703169
0709 13:21:02.949299 (+  8230us) raft_consensus.cc:1685] finished
0709 13:21:02.949440 (+   141us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:21:02.949591 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":224395,"prepare.queue_time_us":49757,"prepare.run_cpu_time_us":61,"prepare.run_wall_time_us":62,"raft.queue_time_us":177292,"raft.run_cpu_time_us":313,"raft.run_wall_time_us":41025,"spinlock_wait_cycles":145327872,"thread_start_us":107256,"threads_started":1,"wal-append.queue_time_us":170946}
I20260709 13:21:02.956063 20267 raft_consensus.cc:2804] T ada5a004ba7c46819051e3ccec71ffdb P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:21:02.956502 20267 raft_consensus.cc:493] T ada5a004ba7c46819051e3ccec71ffdb P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:02.956826 20267 raft_consensus.cc:3060] T ada5a004ba7c46819051e3ccec71ffdb P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:21:02.962527 16265 leader_election.cc:400] T 434bff3082d141838068f323bdb0d1ed P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 11 election: Vote denied by peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741) with higher term. Message: Invalid argument: T 434bff3082d141838068f323bdb0d1ed P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 for earlier term 11. Current term is 12.
I20260709 13:21:02.962885 16822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ab0ac8347641e6955e9bce7d90c1a2" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:21:02.962939 16265 leader_election.cc:403] T 434bff3082d141838068f323bdb0d1ed P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 11 election: Cancelling election due to peer responding with higher term
W20260709 13:21:02.965816 16115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 88674.2us
I20260709 13:21:02.967455 16115 leader_election.cc:304] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:21:02.970391 20231 raft_consensus.cc:2749] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:21:02.977527 16789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 52589.1us
I20260709 13:21:02.983919 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434bff3082d141838068f323bdb0d1ed" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:21:02.985388 16292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d16dfad0c3b540e5b4b86492ddf87208" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 13 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:02.990919 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2794af5b99984efebdee564035841cb0" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:21:02.991681 16300 raft_consensus.cc:3060] T 2794af5b99984efebdee564035841cb0 P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:21:02.993455 16143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02838c86531d4f98bf58d1c22b8baf77" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:02.993920 16144 raft_consensus.cc:1275] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [term 14 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 13:21:02.994098 16143 raft_consensus.cc:2468] T 02838c86531d4f98bf58d1c22b8baf77 P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 12.
I20260709 13:21:02.995398 20900 raft_consensus.cc:493] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:02.995615 16147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8158bbd17b994d2eb96f560bd0f1b88d" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:21:02.995942 20900 raft_consensus.cc:515] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:21:02.997699 20881 log.cc:927] Time spent T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 0.413s	user 0.002s	sys 0.000s
I20260709 13:21:02.998158 20900 leader_election.cc:290] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:21:02.997977 20894 raft_consensus.cc:515] T 9e65676b21b748728ad39668cfe722ec P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:03.001189 20594 raft_consensus.cc:493] T c4f65cdf00b748bb963ee6e81ef43b79 P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:21:03.001744 20594 raft_consensus.cc:515] T c4f65cdf00b748bb963ee6e81ef43b79 P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:02.974097 20754 consensus_queue.cc:1048] T 5a96373bb0e14dce8a266b2fc6c58507 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.011s
I20260709 13:21:03.003480 16294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18ad7e3af5504571a69d10c00750a01c" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:02.977125 20762 consensus_queue.cc:1059] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.028s
I20260709 13:21:03.004839 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f5f89186204242bf377f7c42b2e2da" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:21:03.005694 16299 raft_consensus.cc:2393] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 13: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
I20260709 13:21:03.005817 20762 raft_consensus.cc:3055] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:21:03.006345 20762 raft_consensus.cc:740] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:21:03.007085 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada5a004ba7c46819051e3ccec71ffdb" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:21:02.979768 20895 raft_consensus.cc:3060] T 434bff3082d141838068f323bdb0d1ed P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:21:03.012980 16790 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:02.982045 20890 raft_consensus.cc:2764] T 2794af5b99984efebdee564035841cb0 P cb82cc7bd6db46d980c75237a1402860 [term 10 FOLLOWER]: Leader pre-election decision vote started in defunct term 9: won
I20260709 13:21:03.017030 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02838c86531d4f98bf58d1c22b8baf77" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:03.017177 16115 leader_election.cc:304] T fffab26427e24d398e2695aad40ca134 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:21:03.017936 16299 raft_consensus.cc:2468] T 02838c86531d4f98bf58d1c22b8baf77 P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 12.
I20260709 13:21:03.016191 16144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:02.990669 20903 raft_consensus.cc:493] T b87147731a68400e8cce9b879daa7c5d P cb82cc7bd6db46d980c75237a1402860 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:21:03.021490 20903 raft_consensus.cc:515] T b87147731a68400e8cce9b879daa7c5d P cb82cc7bd6db46d980c75237a1402860 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:03.022464 20663 raft_consensus.cc:493] T 3884f864a96d428785895e5463e5728f P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:21:03.022957 20894 leader_election.cc:290] T 9e65676b21b748728ad39668cfe722ec P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 12 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:03.023408 16296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c11074579b046d99e585c2818e18a2c" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 13 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:03.023765 20903 leader_election.cc:290] T b87147731a68400e8cce9b879daa7c5d P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:03.024262 16296 raft_consensus.cc:2410] T 8c11074579b046d99e585c2818e18a2c P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 for term 13 because replica has last-logged OpId of term: 11 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 10.
I20260709 13:21:02.990998 20236 consensus_queue.cc:1059] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.035s
I20260709 13:21:03.023108 20663 raft_consensus.cc:515] T 3884f864a96d428785895e5463e5728f P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:02.979959 20679 consensus_queue.cc:1059] T b87147731a68400e8cce9b879daa7c5d P 647a343a542d4a139fe106c19095129b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: INVALID_TERM, Last received: 13.10, Next index: 11, Last known committed idx: 8, Time since last communication: 0.047s
I20260709 13:21:03.022957 16295 raft_consensus.cc:1240] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 10. Current term is 11. Ops: [10.7-10.7]
I20260709 13:21:03.027601 16294 raft_consensus.cc:1240] T a378d7f796f94615b3b347119a9880fd P 90f3da57796b488c9087a2f523adf045 [term 14 FOLLOWER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 13. Current term is 14. Ops: []
I20260709 13:21:03.029121 20594 leader_election.cc:290] T c4f65cdf00b748bb963ee6e81ef43b79 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:21:03.019929 16144 raft_consensus.cc:2468] T ecc1e363230043fe9fef390f0b979f20 P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 12.
I20260709 13:21:03.032949 16296 raft_consensus.cc:1240] T 8158bbd17b994d2eb96f560bd0f1b88d P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 11. Current term is 12. Ops: []
I20260709 13:21:03.035374 16292 raft_consensus.cc:2410] T d16dfad0c3b540e5b4b86492ddf87208 P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b for term 13 because replica has last-logged OpId of term: 12 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 13:21:03.035627 16294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdbd78acb4cf46298e0de8ba0d130e19" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:21:03.036276 16294 raft_consensus.cc:3060] T cdbd78acb4cf46298e0de8ba0d130e19 P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:21:03.036312 16265 leader_election.cc:304] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:21:03.036854 16292 raft_consensus.cc:1275] T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 13:21:02.998847 20782 raft_consensus.cc:515] T ee9ed67006144797b1de6ea8c2013a68 P cb82cc7bd6db46d980c75237a1402860 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:03.043107 20894 raft_consensus.cc:2749] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:21:03.043263 20594 raft_consensus.cc:493] T 84f2da98864b4425bf7c0055951fdd4b P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:21:03.044126 20594 raft_consensus.cc:515] T 84f2da98864b4425bf7c0055951fdd4b P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:03.046285 20594 leader_election.cc:290] T 84f2da98864b4425bf7c0055951fdd4b P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:21:03.046223 20894 consensus_queue.cc:1059] T 5a959dbecc2648eb85d1cc550f5a3245 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:21:03.047290 20894 raft_consensus.cc:3055] T 5a959dbecc2648eb85d1cc550f5a3245 P 90f3da57796b488c9087a2f523adf045 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:21:03.047847 20894 raft_consensus.cc:740] T 5a959dbecc2648eb85d1cc550f5a3245 P 90f3da57796b488c9087a2f523adf045 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:21:03.048808 20894 consensus_queue.cc:260] T 5a959dbecc2648eb85d1cc550f5a3245 P 90f3da57796b488c9087a2f523adf045 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:03.050053 20894 raft_consensus.cc:3060] T 5a959dbecc2648eb85d1cc550f5a3245 P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:21:03.050360 20470 leader_election.cc:290] T af57996a8be84d398f0f023e8d08ef77 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 14 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:03.019431 20832 raft_consensus.cc:493] T c9944d5e4b8c4109af9955c4f08c9095 P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:03.047339 20797 raft_consensus.cc:2749] T 829b4ad8c5a54eababeea1181b960d4b P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:21:03.047353 16116 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:03.055863 16265 leader_election.cc:304] T 8c11074579b046d99e585c2818e18a2c P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: cb82cc7bd6db46d980c75237a1402860
W20260709 13:21:03.047854 20874 log.cc:927] Time spent T 7c2095a5f4ad49408dbc7f69c606120d P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 0.450s	user 0.000s	sys 0.006s
I20260709 13:21:03.057862 20711 raft_consensus.cc:493] T 23f82224f4564fea93572e8206bad1fe P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:03.058506 20711 raft_consensus.cc:515] T 23f82224f4564fea93572e8206bad1fe P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:03.058946 16296 raft_consensus.cc:1275] T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045 [term 14 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 13:21:03.060010 20679 raft_consensus.cc:3055] T b87147731a68400e8cce9b879daa7c5d P 647a343a542d4a139fe106c19095129b [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:21:03.060403 20679 raft_consensus.cc:740] T b87147731a68400e8cce9b879daa7c5d P 647a343a542d4a139fe106c19095129b [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:21:03.060719 20711 leader_election.cc:290] T 23f82224f4564fea93572e8206bad1fe P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:03.061187 20679 consensus_queue.cc:260] T b87147731a68400e8cce9b879daa7c5d P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:03.062644 20679 raft_consensus.cc:3060] T b87147731a68400e8cce9b879daa7c5d P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:21:03.052217 20236 raft_consensus.cc:3055] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:21:03.063580 16295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c74490c40243a88e62d1760b5cd814" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:21:03.080986 16296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ab0ac8347641e6955e9bce7d90c1a2" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:21:03.047024 20594 consensus_queue.cc:1048] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.025s
I20260709 13:21:03.086987 16300 raft_consensus.cc:2468] T 2794af5b99984efebdee564035841cb0 P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 10.
W20260709 13:21:03.088205 16789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 100227us
I20260709 13:21:03.089807 16190 leader_election.cc:304] T 02838c86531d4f98bf58d1c22b8baf77 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:21:03.090951 16265 leader_election.cc:304] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: cb82cc7bd6db46d980c75237a1402860
I20260709 13:21:03.070056 20236 raft_consensus.cc:740] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:21:03.099128 16296 raft_consensus.cc:3060] T d2ab0ac8347641e6955e9bce7d90c1a2 P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:21:03.081107 20711 raft_consensus.cc:2804] T 8c11074579b046d99e585c2818e18a2c P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:21:03.084035 20754 raft_consensus.cc:3060] T fffab26427e24d398e2695aad40ca134 P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:21:03.086153 20890 raft_consensus.cc:493] T 66f75124215f4ca6bef831e174a792d8 P cb82cc7bd6db46d980c75237a1402860 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:21:03.109144 16790 leader_election.cc:304] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:21:03.079994 20832 raft_consensus.cc:515] T c9944d5e4b8c4109af9955c4f08c9095 P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:21:03.111703 16115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 129285us
I20260709 13:21:03.054207 20762 consensus_queue.cc:260] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:03.114264 16789 leader_election.cc:304] T 26354361e9b044cea1c2b86c4882fe92 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:21:03.103659 20470 raft_consensus.cc:2804] T 02838c86531d4f98bf58d1c22b8baf77 P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:21:03.088346 20760 consensus_queue.cc:1048] T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.023s
I20260709 13:21:03.088933 20797 consensus_queue.cc:1059] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: INVALID_TERM, Last received: 11.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.022s
I20260709 13:21:03.157136 16790 leader_election.cc:304] T 829b4ad8c5a54eababeea1181b960d4b P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:21:03.103505 20571 consensus_queue.cc:1048] T b6f07842d3aa47858df5665874c86f3a P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.012s
I20260709 13:21:03.116747 20902 raft_consensus.cc:3055] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P cb82cc7bd6db46d980c75237a1402860 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:21:03.164744 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87147731a68400e8cce9b879daa7c5d" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:03.108023 20890 raft_consensus.cc:515] T 66f75124215f4ca6bef831e174a792d8 P cb82cc7bd6db46d980c75237a1402860 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:03.165729 16115 leader_election.cc:304] T 8cc959fb7f704fba9009c0595467d074 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:21:03.108227 20903 consensus_queue.cc:1048] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.055s
I20260709 13:21:03.151837 20470 raft_consensus.cc:493] T 02838c86531d4f98bf58d1c22b8baf77 P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Starting leader election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:21:03.175356 20799 raft_consensus.cc:2749] T 26354361e9b044cea1c2b86c4882fe92 P cb82cc7bd6db46d980c75237a1402860 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:21:03.175552 20470 raft_consensus.cc:3060] T 02838c86531d4f98bf58d1c22b8baf77 P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:21:03.175983 20667 raft_consensus.cc:2749] T 829b4ad8c5a54eababeea1181b960d4b P cb82cc7bd6db46d980c75237a1402860 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:21:03.156668 20780 raft_consensus.cc:2804] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:21:03.176644 20780 raft_consensus.cc:493] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:03.177037 20780 raft_consensus.cc:3060] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:21:03.179306 20762 raft_consensus.cc:3060] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:21:03.181002 16294 raft_consensus.cc:2468] T cdbd78acb4cf46298e0de8ba0d130e19 P 90f3da57796b488c9087a2f523adf045 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 14.
I20260709 13:21:03.182271 20823 leader_election.cc:290] T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
W20260709 13:21:03.183744 20910 log.cc:927] Time spent T 8a878999c0024d2abe517484180290fc P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 0.086s	user 0.000s	sys 0.000s
I20260709 13:21:03.184569 16190 leader_election.cc:304] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:21:03.187580 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d4ab870a05347868956aa8b841b2981" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:03.188894 20757 raft_consensus.cc:2749] T 8cc959fb7f704fba9009c0595467d074 P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:21:03.190058 20895 raft_consensus.cc:2749] T 434bff3082d141838068f323bdb0d1ed P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Leader election lost for term 11. Reason: Vote denied by peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741) with higher term. Message: Invalid argument: T 434bff3082d141838068f323bdb0d1ed P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 for earlier term 11. Current term is 12.
I20260709 13:21:03.191445 16115 leader_election.cc:304] T d16dfad0c3b540e5b4b86492ddf87208 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:21:03.192162 20823 raft_consensus.cc:2804] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:21:03.192669 20823 raft_consensus.cc:697] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [term 14 LEADER]: Becoming Leader. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:21:03.193773 20823 consensus_queue.cc:237] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:03.196482 20832 leader_election.cc:290] T c9944d5e4b8c4109af9955c4f08c9095 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:03.196731 16115 leader_election.cc:304] T 2794af5b99984efebdee564035841cb0 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:21:03.196949 20388 raft_consensus.cc:2749] T d16dfad0c3b540e5b4b86492ddf87208 P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:21:03.197065 20832 raft_consensus.cc:493] T fffab26427e24d398e2695aad40ca134 P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:03.197539 20889 raft_consensus.cc:2804] T 2794af5b99984efebdee564035841cb0 P 647a343a542d4a139fe106c19095129b [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:21:03.197636 20832 raft_consensus.cc:515] T fffab26427e24d398e2695aad40ca134 P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:03.198089 20889 raft_consensus.cc:697] T 2794af5b99984efebdee564035841cb0 P 647a343a542d4a139fe106c19095129b [term 10 LEADER]: Becoming Leader. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:21:03.199110 20889 consensus_queue.cc:237] T 2794af5b99984efebdee564035841cb0 P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:03.201267 20663 leader_election.cc:290] T 3884f864a96d428785895e5463e5728f P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:03.170645 16148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62fe3346cd3c42c4a55e14af9073bca4" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 17 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:03.204823 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a878999c0024d2abe517484180290fc" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:03.206943 16292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f2da98864b4425bf7c0055951fdd4b" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
W20260709 13:21:03.164786 20905 log.cc:927] Time spent T 68a6838d82ef4cac83507f906139d90f P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 0.112s	user 0.000s	sys 0.001s
I20260709 13:21:03.158367 20911 raft_consensus.cc:2804] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:21:03.218521 16299 raft_consensus.cc:1240] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 12. Current term is 13. Ops: []
I20260709 13:21:03.219627 16295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02838c86531d4f98bf58d1c22b8baf77" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:03.220479 16295 raft_consensus.cc:2410] T 02838c86531d4f98bf58d1c22b8baf77 P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b for term 13 because replica has last-logged OpId of term: 11 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 6.
I20260709 13:21:03.165066 20902 raft_consensus.cc:740] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P cb82cc7bd6db46d980c75237a1402860 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:21:03.226414 16300 raft_consensus.cc:2468] T 8a878999c0024d2abe517484180290fc P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 11.
W20260709 13:21:03.237473 16115 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:03.240082 16292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a96373bb0e14dce8a266b2fc6c58507" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:21:03.240945 16292 raft_consensus.cc:2376] T 5a96373bb0e14dce8a266b2fc6c58507 P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 647a343a542d4a139fe106c19095129b in term 12. Re-sending same reply.
W20260709 13:21:03.231664 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 79457.1us
I20260709 13:21:03.204993 20711 raft_consensus.cc:697] T 8c11074579b046d99e585c2818e18a2c P 90f3da57796b488c9087a2f523adf045 [term 12 LEADER]: Becoming Leader. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
W20260709 13:21:03.247908 16789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 142286us
I20260709 13:21:03.249886 16265 leader_election.cc:304] T a378d7f796f94615b3b347119a9880fd P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:21:03.264050 16142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c74490c40243a88e62d1760b5cd814" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:21:03.160838 20236 consensus_queue.cc:260] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:03.258241 16146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02838c86531d4f98bf58d1c22b8baf77" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:03.272974 16148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746aabf856ab401c9ef5e859371e7b67" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:03.278169 16296 raft_consensus.cc:2468] T d2ab0ac8347641e6955e9bce7d90c1a2 P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 13.
I20260709 13:21:03.243026 20911 raft_consensus.cc:697] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045 [term 11 LEADER]: Becoming Leader. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:21:03.289500 16295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6f07842d3aa47858df5665874c86f3a" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:03.220443 20267 raft_consensus.cc:515] T ada5a004ba7c46819051e3ccec71ffdb P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:21:03.288339 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 168479us
I20260709 13:21:03.297497 16295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1df93779209748b9860f6d893dd3de53" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
W20260709 13:21:03.294245 16117 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:03.266268 20782 leader_election.cc:290] T ee9ed67006144797b1de6ea8c2013a68 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 11 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:21:03.299916 16296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5ad4eb281564936a59f41e387a47156" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:03.301586 16150 raft_consensus.cc:1275] T b6f07842d3aa47858df5665874c86f3a P 647a343a542d4a139fe106c19095129b [term 14 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 13:21:03.280246 16146 raft_consensus.cc:2393] T 02838c86531d4f98bf58d1c22b8baf77 P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 12: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
I20260709 13:21:03.302964 16142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2372981006d4b218215c28964dea623" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 14 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:03.303861 16142 raft_consensus.cc:2410] T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac for term 14 because replica has last-logged OpId of term: 12 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 8.
I20260709 13:21:03.309568 16298 raft_consensus.cc:2468] T 434bff3082d141838068f323bdb0d1ed P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 12.
I20260709 13:21:03.310436 16295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a878999c0024d2abe517484180290fc" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:21:03.310971 16295 raft_consensus.cc:3060] T 8a878999c0024d2abe517484180290fc P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:21:03.312209 16293 raft_consensus.cc:2393] T ada5a004ba7c46819051e3ccec71ffdb P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 13: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
I20260709 13:21:03.272996 20472 consensus_queue.cc:1059] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.031s
I20260709 13:21:03.317498 16296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57a744f5aaf944678a63469cbf514015" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:03.318943 16296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18ad7e3af5504571a69d10c00750a01c" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:03.320227 16296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c61a39fd4f84424b129c1a6f9ba404d" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:03.321074 16296 raft_consensus.cc:2393] T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 13: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
I20260709 13:21:03.322476 16296 raft_consensus.cc:1275] T e4dc6b0541ca431ebc757beac00521f1 P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 10 index: 5. Preceding OpId from leader: term: 12 index: 6. (index mismatch)
I20260709 13:21:03.323813 16142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee9ed67006144797b1de6ea8c2013a68" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:21:03.324533 16142 raft_consensus.cc:3060] T ee9ed67006144797b1de6ea8c2013a68 P 647a343a542d4a139fe106c19095129b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:21:03.324280 20911 consensus_queue.cc:237] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:03.328406 20780 raft_consensus.cc:515] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:03.330231 16191 leader_election.cc:304] T c2f5f89186204242bf377f7c42b2e2da P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:21:03.331465 20267 leader_election.cc:290] T ada5a004ba7c46819051e3ccec71ffdb P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 13 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:03.270421 20886 raft_consensus.cc:515] T ee9ed67006144797b1de6ea8c2013a68 P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:21:03.333412 16047 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:58400 (request call id 191) took 16680 ms (client timeout 15000 ms). Trace:
I20260709 13:21:03.336807 20760 raft_consensus.cc:2749] T c2f5f89186204242bf377f7c42b2e2da P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:21:03.337963 16191 leader_election.cc:304] T 18ad7e3af5504571a69d10c00750a01c P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:21:03.338788 20803 raft_consensus.cc:2749] T 18ad7e3af5504571a69d10c00750a01c P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:21:03.340009 16191 leader_election.cc:304] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 90f3da57796b488c9087a2f523adf045
I20260709 13:21:03.340749 20803 raft_consensus.cc:2804] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:21:03.334110 16295 raft_consensus.cc:2468] T 8a878999c0024d2abe517484180290fc P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 12.
I20260709 13:21:03.341523 16142 raft_consensus.cc:2468] T ee9ed67006144797b1de6ea8c2013a68 P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 11.
I20260709 13:21:03.344872 16295 raft_consensus.cc:1275] T 31969392ccdb43b491911a319e8bc7e1 P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:21:03.300930 20832 leader_election.cc:290] T fffab26427e24d398e2695aad40ca134 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:21:03.299832 20918 raft_consensus.cc:2804] T a378d7f796f94615b3b347119a9880fd P 90f3da57796b488c9087a2f523adf045 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:21:03.341251 20803 raft_consensus.cc:697] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac [term 12 LEADER]: Becoming Leader. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:21:03.351509 16819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee9ed67006144797b1de6ea8c2013a68" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:21:03.352257 16819 raft_consensus.cc:2368] T ee9ed67006144797b1de6ea8c2013a68 P cb82cc7bd6db46d980c75237a1402860 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac for earlier term 10. Current term is 11.
I20260709 13:21:03.354977 16221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8158bbd17b994d2eb96f560bd0f1b88d" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:21:03.355301 16295 raft_consensus.cc:1240] T ecc1e363230043fe9fef390f0b979f20 P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 11. Current term is 12. Ops: []
I20260709 13:21:03.355484 16224 raft_consensus.cc:1275] T c0eab25ce4c64e018a041beeda2867db P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 7. (term mismatch)
I20260709 13:21:03.355818 16224 pending_rounds.cc:85] T c0eab25ce4c64e018a041beeda2867db P 1ca2725d290e47d68b1a4b609c9816ac: Aborting all ops after (but not including) 6
I20260709 13:21:03.356056 16224 pending_rounds.cc:107] T c0eab25ce4c64e018a041beeda2867db P 1ca2725d290e47d68b1a4b609c9816ac: Aborting uncommitted NO_OP operation due to leader change: 11.7
I20260709 13:21:03.356295 16224 raft_consensus.cc:2889] T c0eab25ce4c64e018a041beeda2867db P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:21:03.356721 16295 raft_consensus.cc:1275] T 68a6838d82ef4cac83507f906139d90f P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 13:21:03.356889 16192 leader_election.cc:400] T ee9ed67006144797b1de6ea8c2013a68 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 10 election: Vote denied by peer cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923) with higher term. Message: Invalid argument: T ee9ed67006144797b1de6ea8c2013a68 P cb82cc7bd6db46d980c75237a1402860 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac for earlier term 10. Current term is 11.
I20260709 13:21:03.357221 16192 leader_election.cc:403] T ee9ed67006144797b1de6ea8c2013a68 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 10 election: Cancelling election due to peer responding with higher term
I20260709 13:21:03.357697 16223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:03.359558 20797 raft_consensus.cc:3060] T ee9ed67006144797b1de6ea8c2013a68 P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:21:03.360165 16222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee9ed67006144797b1de6ea8c2013a68" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:21:03.360473 16226 raft_consensus.cc:1275] T 19baa7bc367844c288bca8f81439c413 P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 13:21:03.360612 16218 raft_consensus.cc:1275] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 11 index: 10. (term mismatch)
I20260709 13:21:03.361233 16218 pending_rounds.cc:85] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac: Aborting all ops after (but not including) 9
I20260709 13:21:03.362875 16218 pending_rounds.cc:107] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac: Aborting uncommitted NO_OP operation due to leader change: 10.10
W20260709 13:21:03.333803 16047 rpcz_store.cc:269] 0709 13:20:46.672446 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:46.683285 (+ 10839us) service_pool.cc:224] Handling call
0709 13:20:46.936856 (+253571us) spinlock_profiling.cc:243] Waited 46.2 ms on lock 0x7b5400002490. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69ad8fadb1 00007f69ada3ab71 00007f69ada3b0b2 00007f69b69b4a89 00007f69b694cda6 00007f69b691e088 00007f69b6a1c37e 00007f69b4edd2a4 00007f69b4edd232 00007f69b4edd161 00007f69b4edd0dc
0709 13:20:46.958138 (+ 21282us) spinlock_profiling.cc:243] Waited 20.2 ms on lock 0x7b5400002490. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69ad8fadb1 00007f69ada3ae9c 00007f69ada3b0f6 00007f69b69b4a89 00007f69b694cda6 00007f69b691e088 00007f69b6a1c37e 00007f69b4edd2a4 00007f69b4edd232 00007f69b4edd161 00007f69b4edd0dc
0709 13:21:03.333148 (+16375010us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:20:56.437778 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:20:56.461246 (+ 23468us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:20:56.461310 (+    64us) write_op.cc:435] Acquired schema lock
0709 13:20:56.461330 (+    20us) tablet.cc:662] Decoding operations
0709 13:20:56.463135 (+  1805us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:20:56.585824 (+122689us) write_op.cc:641] Partition lock acquired for write op
0709 13:20:56.597337 (+ 11513us) tablet.cc:685] Acquiring locks for 20 operations
0709 13:20:57.491759 (+894422us) tablet.cc:701] Row locks acquired
0709 13:20:57.491813 (+    54us) write_op.cc:260] PREPARE: finished
0709 13:20:57.492009 (+   196us) write_op.cc:270] Start()
0709 13:20:57.492166 (+   157us) write_op.cc:276] Timestamp: P: 1783603257491940 usec, L: 0
0709 13:20:57.492213 (+    47us) op_driver.cc:348] REPLICATION: starting
0709 13:20:57.578219 (+ 86006us) log.cc:844] Serialized 7930 byte log entry
0709 13:20:57.839606 (+261387us) op_driver.cc:464] REPLICATION: finished
0709 13:20:57.999483 (+159877us) write_op.cc:301] APPLY: starting
0709 13:20:58.016410 (+ 16927us) tablet.cc:1367] starting BulkCheckPresence
0709 13:20:58.700927 (+684517us) tablet.cc:1370] finished BulkCheckPresence
0709 13:20:58.701076 (+   149us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:20:59.241356 (+540280us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:20:59.277414 (+ 36058us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=40,key_file_lookups=40,delta_file_lookups=0,mrs_lookups=0
0709 13:20:59.277478 (+    64us) write_op.cc:312] APPLY: finished
0709 13:20:59.277957 (+   479us) log.cc:844] Serialized 180 byte log entry
0709 13:20:59.480843 (+202886us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:20:59.481067 (+   224us) write_op.cc:454] Released schema lock
0709 13:20:59.481150 (+    83us) write_op.cc:341] FINISH: Updating metrics
0709 13:20:59.481574 (+   424us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:20:59.481601 (+    27us) write_op.cc:454] Released schema lock
0709 13:21:01.896806 (+2415205us) spinlock_profiling.cc:243] Waited 69.3 ms on lock 0x7b4400022b4c. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69ad8fadb1 00007f69ad998c85 00007f69ad9996a2 00007f69b070b1db 00007f69b0706aaf 00007f69b06fa86e 00007f69b06f9bb6 00007f69b06f97d8 00007f69b0703221 00007f69b07031d9 00007f69b0703169
Metrics: {"mutex_wait_us":861815,"reported_tablets":35,"spinlock_wait_cycles":152808448,"child_traces":[["op",{"apply.queue_time_us":132412,"apply.run_cpu_time_us":56798,"apply.run_wall_time_us":1494206,"cfile_cache_hit":82,"cfile_cache_hit_bytes":87647,"num_ops":20,"prepare.queue_time_us":665952,"prepare.run_cpu_time_us":20986,"prepare.run_wall_time_us":1301567,"raft.queue_time_us":64574,"raft.run_cpu_time_us":3852,"raft.run_wall_time_us":63762,"replication_time_us":332338,"spinlock_wait_cycles":159293184,"thread_start_us":398307,"threads_started":5,"wal-append.queue_time_us":343864,"wal-append.run_cpu_time_us":32254,"wal-append.run_wall_time_us":4501724}]]}
I20260709 13:21:03.363198 16218 raft_consensus.cc:2889] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:21:03.363570 16220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62fe3346cd3c42c4a55e14af9073bca4" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 17 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
W20260709 13:21:03.362387 20914 log.cc:927] Time spent T e5ad4eb281564936a59f41e387a47156 P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
I20260709 13:21:03.354696 20803 consensus_queue.cc:237] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.8, Last appended by leader: 8, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } }
I20260709 13:21:03.367591 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada5a004ba7c46819051e3ccec71ffdb" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:21:03.368348 16224 raft_consensus.cc:2393] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 in current term 13: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
I20260709 13:21:03.303390 20895 consensus_queue.cc:1048] T 66ae176961b74813a58bc15c9c4c6f2a P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.045s
W20260709 13:21:03.347632 20921 log.cc:927] Time spent T 1df93779209748b9860f6d893dd3de53 P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 0.100s	user 0.001s	sys 0.000s
I20260709 13:21:03.352316 16817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af57996a8be84d398f0f023e8d08ef77" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
W20260709 13:21:03.373399 16116 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:03.334126 20886 leader_election.cc:290] T ee9ed67006144797b1de6ea8c2013a68 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 10 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:03.383770 16817 raft_consensus.cc:3060] T af57996a8be84d398f0f023e8d08ef77 P cb82cc7bd6db46d980c75237a1402860 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:21:03.398326 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 116456us
W20260709 13:21:03.393690 16789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 48058.9us
I20260709 13:21:03.394613 20918 raft_consensus.cc:697] T a378d7f796f94615b3b347119a9880fd P 90f3da57796b488c9087a2f523adf045 [term 14 LEADER]: Becoming Leader. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:21:03.429486 20886 raft_consensus.cc:493] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:03.430140 20886 raft_consensus.cc:515] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:21:03.430670 20880 log.cc:927] Time spent T ecc1e363230043fe9fef390f0b979f20 P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 0.899s	user 0.003s	sys 0.005s
I20260709 13:21:03.430879 16790 leader_election.cc:304] T ee9ed67006144797b1de6ea8c2013a68 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:21:03.432303 20886 leader_election.cc:290] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:03.432988 16790 leader_election.cc:304] T 8c11074579b046d99e585c2818e18a2c P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:21:03.433928 20923 raft_consensus.cc:2804] T ee9ed67006144797b1de6ea8c2013a68 P cb82cc7bd6db46d980c75237a1402860 [term 11 FOLLOWER]: Leader election won for term 11
W20260709 13:21:03.436463 20913 log.cc:927] Time spent T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 0.131s	user 0.002s	sys 0.000s
W20260709 13:21:03.439368 20925 log.cc:927] Time spent T 2794af5b99984efebdee564035841cb0 P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 0.097s	user 0.001s	sys 0.000s
I20260709 13:21:03.439759 20470 raft_consensus.cc:515] T 02838c86531d4f98bf58d1c22b8baf77 P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:03.349931 20902 consensus_queue.cc:260] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P cb82cc7bd6db46d980c75237a1402860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:21:03.441608 20922 log.cc:927] Time spent T 87ac0d1f7e8a419f9bcbc04aee0e067e P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 0.070s	user 0.000s	sys 0.000s
I20260709 13:21:03.442101 20902 raft_consensus.cc:3060] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P cb82cc7bd6db46d980c75237a1402860 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:21:03.347033 20711 consensus_queue.cc:237] T 8c11074579b046d99e585c2818e18a2c P 90f3da57796b488c9087a2f523adf045 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:03.447602 16296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f82224f4564fea93572e8206bad1fe" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:03.448386 16296 raft_consensus.cc:2468] T 23f82224f4564fea93572e8206bad1fe P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 10.
I20260709 13:21:03.456261 16821 raft_consensus.cc:1275] T 19baa7bc367844c288bca8f81439c413 P cb82cc7bd6db46d980c75237a1402860 [term 14 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 13:21:03.456517 16149 raft_consensus.cc:2468] T b87147731a68400e8cce9b879daa7c5d P 647a343a542d4a139fe106c19095129b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 14.
I20260709 13:21:03.457165 16220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f2da98864b4425bf7c0055951fdd4b" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:03.463677 16296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8638ee363273426db1393dd81e5c7b72" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:21:03.464540 16819 raft_consensus.cc:1275] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [term 15 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 15 index: 11. (term mismatch)
I20260709 13:21:03.464927 16819 pending_rounds.cc:85] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860: Aborting all ops after (but not including) 10
I20260709 13:21:03.465130 16819 pending_rounds.cc:107] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860: Aborting uncommitted NO_OP operation due to leader change: 14.11
I20260709 13:21:03.465354 16819 raft_consensus.cc:2889] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:21:03.466261 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4f65cdf00b748bb963ee6e81ef43b79" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:03.466885 16224 raft_consensus.cc:2468] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 14.
I20260709 13:21:03.468267 16296 raft_consensus.cc:1240] T d2ab0ac8347641e6955e9bce7d90c1a2 P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 12. Current term is 13. Ops: [12.11-12.11]
I20260709 13:21:03.469278 16296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87147731a68400e8cce9b879daa7c5d" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:03.469942 16296 raft_consensus.cc:2393] T b87147731a68400e8cce9b879daa7c5d P 90f3da57796b488c9087a2f523adf045 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 in current term 14: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
W20260709 13:21:03.470162 20909 log.cc:927] Time spent T 5a96373bb0e14dce8a266b2fc6c58507 P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 13:21:03.470660 16296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada5a004ba7c46819051e3ccec71ffdb" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:21:03.471632 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fffab26427e24d398e2695aad40ca134" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:03.472332 16217 raft_consensus.cc:2468] T fffab26427e24d398e2695aad40ca134 P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 10.
I20260709 13:21:03.484545 16265 leader_election.cc:304] T 02838c86531d4f98bf58d1c22b8baf77 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:21:03.443372 20470 leader_election.cc:290] T 02838c86531d4f98bf58d1c22b8baf77 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 12 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:03.493212 16226 raft_consensus.cc:1240] T 08c74490c40243a88e62d1760b5cd814 P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 11. Current term is 12. Ops: [11.8-11.8]
I20260709 13:21:03.415409 20900 consensus_queue.cc:1059] T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.057s
I20260709 13:21:03.501034 16819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:03.501748 16819 raft_consensus.cc:2468] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 12.
I20260709 13:21:03.504057 16821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e65676b21b748728ad39668cfe722ec" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:21:03.504892 16821 raft_consensus.cc:3060] T 9e65676b21b748728ad39668cfe722ec P cb82cc7bd6db46d980c75237a1402860 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:21:03.506410 16817 raft_consensus.cc:2468] T af57996a8be84d398f0f023e8d08ef77 P cb82cc7bd6db46d980c75237a1402860 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 14.
I20260709 13:21:03.508054 16192 leader_election.cc:304] T af57996a8be84d398f0f023e8d08ef77 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:21:03.472697 20667 raft_consensus.cc:2749] T 8c11074579b046d99e585c2818e18a2c P cb82cc7bd6db46d980c75237a1402860 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:21:03.501811 16218 raft_consensus.cc:1275] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:21:03.476466 20923 raft_consensus.cc:697] T ee9ed67006144797b1de6ea8c2013a68 P cb82cc7bd6db46d980c75237a1402860 [term 11 LEADER]: Becoming Leader. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:21:03.518098 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9944d5e4b8c4109af9955c4f08c9095" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:03.484163 20890 leader_election.cc:290] T 66f75124215f4ca6bef831e174a792d8 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:21:03.518770 20886 raft_consensus.cc:2804] T af57996a8be84d398f0f023e8d08ef77 P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:21:03.528893 20886 raft_consensus.cc:697] T af57996a8be84d398f0f023e8d08ef77 P 1ca2725d290e47d68b1a4b609c9816ac [term 14 LEADER]: Becoming Leader. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:21:03.529920 20886 consensus_queue.cc:237] T af57996a8be84d398f0f023e8d08ef77 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:03.530259 20932 consensus_queue.cc:1048] T c0eab25ce4c64e018a041beeda2867db P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.045s
I20260709 13:21:03.534803 16822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3884f864a96d428785895e5463e5728f" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:03.535686 16819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f82224f4564fea93572e8206bad1fe" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:03.536166 16825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9944d5e4b8c4109af9955c4f08c9095" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:03.536515 16819 raft_consensus.cc:2468] T 23f82224f4564fea93572e8206bad1fe P cb82cc7bd6db46d980c75237a1402860 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 9.
I20260709 13:21:03.536882 16825 raft_consensus.cc:2468] T c9944d5e4b8c4109af9955c4f08c9095 P cb82cc7bd6db46d980c75237a1402860 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 13.
I20260709 13:21:03.539506 16823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62fe3346cd3c42c4a55e14af9073bca4" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 16 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:21:03.541677 20782 consensus_queue.cc:1059] T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: INVALID_TERM, Last received: 11.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:21:03.543059 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada5a004ba7c46819051e3ccec71ffdb" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:21:03.543808 16217 raft_consensus.cc:2393] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 13: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
I20260709 13:21:03.543812 16816 raft_consensus.cc:1275] T 74ca8140f35c4ae29e8a664f6bdfc829 P cb82cc7bd6db46d980c75237a1402860 [term 13 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
W20260709 13:21:03.510506 20880 log.cc:927] Time spent T ecc1e363230043fe9fef390f0b979f20 P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 0.052s	user 0.000s	sys 0.001s
I20260709 13:21:03.549546 16790 leader_election.cc:304] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
W20260709 13:21:03.556321 16266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 256566us
I20260709 13:21:03.559592 16266 leader_election.cc:304] T 26354361e9b044cea1c2b86c4882fe92 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 647a343a542d4a139fe106c19095129b
I20260709 13:21:03.486377 20832 consensus_queue.cc:1048] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.062s
I20260709 13:21:03.564992 16817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada5a004ba7c46819051e3ccec71ffdb" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:21:03.565757 16817 raft_consensus.cc:2393] T ada5a004ba7c46819051e3ccec71ffdb P cb82cc7bd6db46d980c75237a1402860 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 13: Already voted for candidate cb82cc7bd6db46d980c75237a1402860 in this term.
I20260709 13:21:03.563581 16821 raft_consensus.cc:2468] T 9e65676b21b748728ad39668cfe722ec P cb82cc7bd6db46d980c75237a1402860 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 12.
W20260709 13:21:03.563818 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 79354.7us
I20260709 13:21:03.573628 16226 raft_consensus.cc:1275] T 8cc959fb7f704fba9009c0595467d074 P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 13:21:03.588686 16265 leader_election.cc:304] T 8158bbd17b994d2eb96f560bd0f1b88d P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:21:03.589555 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66f75124215f4ca6bef831e174a792d8" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:03.576712 20931 raft_consensus.cc:2804] T 02838c86531d4f98bf58d1c22b8baf77 P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:21:03.592592 20931 raft_consensus.cc:493] T 02838c86531d4f98bf58d1c22b8baf77 P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:03.592935 20931 raft_consensus.cc:3060] T 02838c86531d4f98bf58d1c22b8baf77 P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:21:03.559551 20903 consensus_queue.cc:1048] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.026s
I20260709 13:21:03.602753 16823 raft_consensus.cc:2468] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 16.
I20260709 13:21:03.582885 20696 raft_consensus.cc:493] T 74ca8140f35c4ae29e8a664f6bdfc829 P cb82cc7bd6db46d980c75237a1402860 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:03.588894 20894 raft_consensus.cc:2804] T 26354361e9b044cea1c2b86c4882fe92 P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 13:21:03.622206 20754 consensus_meta.cc:287] T fffab26427e24d398e2695aad40ca134 P 647a343a542d4a139fe106c19095129b: Time spent flushing consensus metadata: real 0.508s	user 0.008s	sys 0.000s
I20260709 13:21:03.636005 20754 raft_consensus.cc:2749] T fffab26427e24d398e2695aad40ca134 P 647a343a542d4a139fe106c19095129b [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:21:03.648458 20923 consensus_queue.cc:237] T ee9ed67006144797b1de6ea8c2013a68 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:03.611760 20918 consensus_queue.cc:237] T a378d7f796f94615b3b347119a9880fd P 90f3da57796b488c9087a2f523adf045 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:03.657543 20936 consensus_queue.cc:1048] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
W20260709 13:21:03.627230 20935 log.cc:927] Time spent T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.089s	user 0.000s	sys 0.002s
I20260709 13:21:03.604156 20667 raft_consensus.cc:2749] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:21:03.627777 20919 raft_consensus.cc:2749] T 8158bbd17b994d2eb96f560bd0f1b88d P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
W20260709 13:21:03.628189 20920 log.cc:927] Time spent T 3a7ca61592e848ea9b0fb81a2ce33cdc P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.070s	user 0.000s	sys 0.000s
W20260709 13:21:03.457150 20892 consensus_queue.cc:1175] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P cb82cc7bd6db46d980c75237a1402860 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "90f3da57796b488c9087a2f523adf045" responder_term: 11 status { last_received { term: 10 index: 9 } last_committed_idx: 9 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 10. Current term is 11. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:21:03.611999 20797 raft_consensus.cc:2749] T ee9ed67006144797b1de6ea8c2013a68 P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Leader election lost for term 10. Reason: Vote denied by peer cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923) with higher term. Message: Invalid argument: T ee9ed67006144797b1de6ea8c2013a68 P cb82cc7bd6db46d980c75237a1402860 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac for earlier term 10. Current term is 11.
I20260709 13:21:03.602339 20932 consensus_queue.cc:1048] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.066s
I20260709 13:21:03.617168 20894 raft_consensus.cc:493] T 26354361e9b044cea1c2b86c4882fe92 P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:21:03.663946 20851 raft_consensus.cc:493] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:21:03.714895 20937 log.cc:927] Time spent T 746aabf856ab401c9ef5e859371e7b67 P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 0.123s	user 0.001s	sys 0.000s
I20260709 13:21:03.697435 20894 raft_consensus.cc:3060] T 26354361e9b044cea1c2b86c4882fe92 P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:21:03.719385 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 113434us
I20260709 13:21:03.616995 20696 raft_consensus.cc:515] T 74ca8140f35c4ae29e8a664f6bdfc829 P cb82cc7bd6db46d980c75237a1402860 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:03.702454 20931 raft_consensus.cc:515] T 02838c86531d4f98bf58d1c22b8baf77 P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:21:03.724944 16266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 100686us
I20260709 13:21:03.705505 20936 consensus_queue.cc:1048] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.027s
I20260709 13:21:03.724243 20641 raft_consensus.cc:493] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:03.752686 16265 leader_election.cc:304] T c4f65cdf00b748bb963ee6e81ef43b79 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:21:03.754559 16266 leader_election.cc:304] T ecc1e363230043fe9fef390f0b979f20 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac
I20260709 13:21:03.759608 20799 raft_consensus.cc:3055] T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:21:03.760046 20799 raft_consensus.cc:740] T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:21:03.760985 20799 consensus_queue.cc:260] T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:03.719939 20851 raft_consensus.cc:515] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:03.763471 16265 leader_election.cc:304] T fffab26427e24d398e2695aad40ca134 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:21:03.765739 20594 consensus_queue.cc:1048] T 66ae176961b74813a58bc15c9c4c6f2a P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.003s
I20260709 13:21:03.768482 20696 leader_election.cc:290] T 74ca8140f35c4ae29e8a664f6bdfc829 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:03.769552 20780 leader_election.cc:290] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 14 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
W20260709 13:21:03.770380 20908 log.cc:927] Time spent T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 0.600s	user 0.003s	sys 0.000s
I20260709 13:21:03.753839 20641 raft_consensus.cc:515] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:03.778051 20641 leader_election.cc:290] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:03.781849 16190 leader_election.cc:304] T d2ab0ac8347641e6955e9bce7d90c1a2 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: cb82cc7bd6db46d980c75237a1402860
I20260709 13:21:03.782316 20836 raft_consensus.cc:493] T 19baa7bc367844c288bca8f81439c413 P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:03.762439 20799 raft_consensus.cc:3060] T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:21:03.784493 20851 leader_election.cc:290] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:03.785000 20822 raft_consensus.cc:493] T 8c11074579b046d99e585c2818e18a2c P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:21:03.786882 16114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:03.787482 20672 raft_consensus.cc:3055] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac [term 11 LEADER]: Stepping down as leader of term 11
W20260709 13:21:03.787982 16789 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:03.788740 16296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e9b2bf7011f4ae688a79d7bb5a447b4" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:03.788847 20231 raft_consensus.cc:493] T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:21:03.788748 16294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74ca8140f35c4ae29e8a664f6bdfc829" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:03.791548 16222 raft_consensus.cc:2468] T ee9ed67006144797b1de6ea8c2013a68 P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 11.
I20260709 13:21:03.795307 20894 raft_consensus.cc:515] T 26354361e9b044cea1c2b86c4882fe92 P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:21:03.823045 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 59651.1us
W20260709 13:21:03.823971 16266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 69497us
I20260709 13:21:03.843362 16266 leader_election.cc:304] T 23f82224f4564fea93572e8206bad1fe P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 
W20260709 13:21:03.832262 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 50509.2us
I20260709 13:21:03.831857 20661 raft_consensus.cc:2804] T c4f65cdf00b748bb963ee6e81ef43b79 P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:21:03.841279 16818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b7d1ab519044eda2d5e04a07242f25" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 12 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:03.835203 20919 raft_consensus.cc:2804] T ecc1e363230043fe9fef390f0b979f20 P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:21:03.855103 16823 raft_consensus.cc:1275] T c9944d5e4b8c4109af9955c4f08c9095 P cb82cc7bd6db46d980c75237a1402860 [term 13 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:21:03.855417 16145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2372981006d4b218215c28964dea623" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:21:03.857086 16145 raft_consensus.cc:3060] T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:21:03.834227 20594 raft_consensus.cc:2804] T fffab26427e24d398e2695aad40ca134 P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:21:03.835686 20672 raft_consensus.cc:740] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:21:03.862993 16144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66f75124215f4ca6bef831e174a792d8" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:03.855994 20231 raft_consensus.cc:515] T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:03.851044 20661 raft_consensus.cc:493] T c4f65cdf00b748bb963ee6e81ef43b79 P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:21:03.842320 20803 raft_consensus.cc:2804] T d2ab0ac8347641e6955e9bce7d90c1a2 P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:21:03.845943 20839 raft_consensus.cc:493] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:03.887930 20839 raft_consensus.cc:515] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:03.888572 20894 leader_election.cc:290] T 26354361e9b044cea1c2b86c4882fe92 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 11 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:03.889287 20894 consensus_queue.cc:1048] T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:21:03.890053 20839 leader_election.cc:290] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:03.878530 20947 consensus_queue.cc:1059] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: INVALID_TERM, Last received: 9.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.045s
I20260709 13:21:03.841180 20836 raft_consensus.cc:515] T 19baa7bc367844c288bca8f81439c413 P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:03.891100 20947 raft_consensus.cc:2804] T 23f82224f4564fea93572e8206bad1fe P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:21:03.891556 20947 raft_consensus.cc:493] T 23f82224f4564fea93572e8206bad1fe P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:03.892127 20947 raft_consensus.cc:3060] T 23f82224f4564fea93572e8206bad1fe P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:21:03.893582 20836 leader_election.cc:290] T 19baa7bc367844c288bca8f81439c413 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:03.862675 20594 raft_consensus.cc:493] T fffab26427e24d398e2695aad40ca134 P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:03.883261 20661 raft_consensus.cc:3060] T c4f65cdf00b748bb963ee6e81ef43b79 P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:21:03.856712 20919 raft_consensus.cc:493] T ecc1e363230043fe9fef390f0b979f20 P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:03.908496 16146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e9b2bf7011f4ae688a79d7bb5a447b4" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:03.911334 16144 raft_consensus.cc:2468] T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 11.
I20260709 13:21:03.838016 20822 raft_consensus.cc:515] T 8c11074579b046d99e585c2818e18a2c P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:03.895507 20594 raft_consensus.cc:3060] T fffab26427e24d398e2695aad40ca134 P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:21:03.921152 16145 raft_consensus.cc:2468] T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 14.
I20260709 13:21:03.922994 16790 leader_election.cc:304] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860; no voters: 
W20260709 13:21:03.923036 16192 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 58319.4us
I20260709 13:21:03.910995 20919 raft_consensus.cc:3060] T ecc1e363230043fe9fef390f0b979f20 P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:21:03.929956 16818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 15 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:03.930794 16818 raft_consensus.cc:2468] T 19baa7bc367844c288bca8f81439c413 P cb82cc7bd6db46d980c75237a1402860 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 14.
I20260709 13:21:03.894325 20836 consensus_queue.cc:1048] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.035s
I20260709 13:21:03.925895 20711 raft_consensus.cc:3055] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:21:03.928267 20672 consensus_queue.cc:260] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:03.958806 20711 raft_consensus.cc:740] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:21:03.959103 20672 raft_consensus.cc:3060] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:21:03.959908 20711 consensus_queue.cc:260] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:03.961107 20711 raft_consensus.cc:3060] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:21:03.936657 20231 leader_election.cc:290] T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:03.972131 16818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c11074579b046d99e585c2818e18a2c" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:03.972981 16818 raft_consensus.cc:2468] T 8c11074579b046d99e585c2818e18a2c P cb82cc7bd6db46d980c75237a1402860 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 12.
I20260709 13:21:03.927217 20931 leader_election.cc:290] T 02838c86531d4f98bf58d1c22b8baf77 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 13 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
W20260709 13:21:03.943746 20949 log.cc:927] Time spent T d2ab0ac8347641e6955e9bce7d90c1a2 P cb82cc7bd6db46d980c75237a1402860: Append to log took a long time: real 0.062s	user 0.000s	sys 0.000s
I20260709 13:21:03.927080 20923 raft_consensus.cc:2804] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:21:03.948637 20822 leader_election.cc:290] T 8c11074579b046d99e585c2818e18a2c P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:03.949486 20947 raft_consensus.cc:515] T 23f82224f4564fea93572e8206bad1fe P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:03.951683 16790 leader_election.cc:304] T 66f75124215f4ca6bef831e174a792d8 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac
I20260709 13:21:03.979908 20799 raft_consensus.cc:2804] T 66f75124215f4ca6bef831e174a792d8 P cb82cc7bd6db46d980c75237a1402860 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:21:03.974704 16192 leader_election.cc:304] T 8c11074579b046d99e585c2818e18a2c P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 
W20260709 13:21:03.975023 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 132915us
I20260709 13:21:03.944945 20661 raft_consensus.cc:515] T c4f65cdf00b748bb963ee6e81ef43b79 P 90f3da57796b488c9087a2f523adf045 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:03.984608 20832 consensus_queue.cc:1048] T 8cc959fb7f704fba9009c0595467d074 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:21:03.980333 20799 raft_consensus.cc:493] T 66f75124215f4ca6bef831e174a792d8 P cb82cc7bd6db46d980c75237a1402860 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:21:03.981870 20923 raft_consensus.cc:697] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 [term 14 LEADER]: Becoming Leader. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:21:03.981894 20803 raft_consensus.cc:697] T d2ab0ac8347641e6955e9bce7d90c1a2 P 1ca2725d290e47d68b1a4b609c9816ac [term 13 LEADER]: Becoming Leader. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:21:03.993517 20851 raft_consensus.cc:2804] T 8c11074579b046d99e585c2818e18a2c P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:21:04.000824 20799 raft_consensus.cc:3060] T 66f75124215f4ca6bef831e174a792d8 P cb82cc7bd6db46d980c75237a1402860 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:21:04.014618 16265 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:21:04.015738 16266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 172434us
I20260709 13:21:04.014971 16220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02838c86531d4f98bf58d1c22b8baf77" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:21:04.016498 16266 leader_election.cc:304] T c9944d5e4b8c4109af9955c4f08c9095 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac
I20260709 13:21:04.016721 16220 raft_consensus.cc:3060] T 02838c86531d4f98bf58d1c22b8baf77 P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:21:04.014230 20851 raft_consensus.cc:493] T 8c11074579b046d99e585c2818e18a2c P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:04.026329 20592 leader_election.cc:290] T 746aabf856ab401c9ef5e859371e7b67 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:04.050002 20667 raft_consensus.cc:493] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:04.050619 20667 raft_consensus.cc:515] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:04.024937 20851 raft_consensus.cc:3060] T 8c11074579b046d99e585c2818e18a2c P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:21:04.052956 20667 leader_election.cc:290] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:04.054404 20890 raft_consensus.cc:493] T 3884f864a96d428785895e5463e5728f P cb82cc7bd6db46d980c75237a1402860 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:21:04.056018 16144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a9de347c711477e87c582d003f36925" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:04.056769 16144 raft_consensus.cc:2468] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 14.
I20260709 13:21:04.056576 20890 raft_consensus.cc:515] T 3884f864a96d428785895e5463e5728f P cb82cc7bd6db46d980c75237a1402860 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:04.056051 16294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a9de347c711477e87c582d003f36925" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:04.056243 20947 leader_election.cc:290] T 23f82224f4564fea93572e8206bad1fe P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 11 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:04.058310 16294 raft_consensus.cc:2468] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 14.
I20260709 13:21:04.058981 20890 leader_election.cc:290] T 3884f864a96d428785895e5463e5728f P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:04.059826 16790 leader_election.cc:304] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:21:04.055251 20836 raft_consensus.cc:493] T 5a96373bb0e14dce8a266b2fc6c58507 P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:21:04.060360 16144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3884f864a96d428785895e5463e5728f" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:04.060673 20836 raft_consensus.cc:515] T 5a96373bb0e14dce8a266b2fc6c58507 P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:04.062942 20836 leader_election.cc:290] T 5a96373bb0e14dce8a266b2fc6c58507 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:04.065204 16817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5d59407eb545b895d8d922e0dd69fd" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:04.065904 16817 raft_consensus.cc:2468] T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 14.
I20260709 13:21:04.066336 16220 raft_consensus.cc:2468] T 02838c86531d4f98bf58d1c22b8baf77 P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 13.
I20260709 13:21:04.066977 16821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66f75124215f4ca6bef831e174a792d8" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:04.067137 20923 consensus_queue.cc:237] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:04.068717 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3884f864a96d428785895e5463e5728f" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
W20260709 13:21:04.075001 16266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 58592.5us
I20260709 13:21:04.066921 20959 raft_consensus.cc:2804] T c9944d5e4b8c4109af9955c4f08c9095 P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:21:04.079149 16265 leader_election.cc:304] T 02838c86531d4f98bf58d1c22b8baf77 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:21:04.078657 16297 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48684 (request call id 2605) took 1044 ms. Trace:
I20260709 13:21:04.082600 16298 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48674 (request call id 2759) took 1099 ms. Trace:
I20260709 13:21:04.083329 16266 leader_election.cc:304] T 9e65676b21b748728ad39668cfe722ec P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:21:04.083137 16298 rpcz_store.cc:276] 0709 13:21:02.979359 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:02.983621 (+  4262us) service_pool.cc:224] Handling call
0709 13:21:04.078780 (+1095159us) spinlock_profiling.cc:243] Waited 238 ms on lock 0x7b54000a9954. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:21:04.078923 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":546425088}
I20260709 13:21:04.086730 16266 leader_election.cc:304] T ada5a004ba7c46819051e3ccec71ffdb P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860
I20260709 13:21:04.087126 16219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02838c86531d4f98bf58d1c22b8baf77" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:04.088828 16821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a9de347c711477e87c582d003f36925" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:21:04.088260 16219 raft_consensus.cc:2376] T 02838c86531d4f98bf58d1c22b8baf77 P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 13. Re-sending same reply.
I20260709 13:21:04.090912 16219 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57600 (request call id 1551) took 1121 ms. Trace:
I20260709 13:21:04.078169 20892 raft_consensus.cc:2804] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:21:04.091408 16219 rpcz_store.cc:276] 0709 13:21:02.969991 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:04.079315 (+1109324us) service_pool.cc:224] Handling call
0709 13:21:04.090841 (+ 11526us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:21:04.080729 16297 rpcz_store.cc:276] 0709 13:21:03.032336 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:03.034007 (+  1671us) service_pool.cc:224] Handling call
0709 13:21:03.037194 (+  3187us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:21:03.043451 (+  6257us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:21:03.043741 (+   290us) log.cc:844] Serialized 14 byte log entry
0709 13:21:03.373779 (+330038us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:21:03.373854 (+    75us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:21:03.373925 (+    71us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:03.374019 (+    94us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:21:03.947855 (+573836us) spinlock_profiling.cc:243] Waited 32.5 ms on lock 0x7b1800169fc0. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b693ade1 00007f69b69475b1 00007f69ada723f3 00007f69ada747c1 00007f69ada74779 00007f69ada74709 00007f69ada746d1 00007f69ada739cd 000000000038f354 000000000038f199 00007f69ada5583a
0709 13:21:04.076233 (+128378us) spinlock_profiling.cc:243] Waited 2.88 ms on lock 0x7b5400163850. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5e8ad 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:21:04.076347 (+   114us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":4692,"prepare.run_cpu_time_us":32,"prepare.run_wall_time_us":33,"raft.queue_time_us":63434,"raft.run_cpu_time_us":83,"raft.run_wall_time_us":83,"spinlock_wait_cycles":81476736,"thread_start_us":323753,"threads_started":2,"wal-append.queue_time_us":123825}
I20260709 13:21:04.080508 20959 raft_consensus.cc:493] T c9944d5e4b8c4109af9955c4f08c9095 P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:21:04.178768 16190 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:04.179571 16822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26354361e9b044cea1c2b86c4882fe92" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:21:04.180768 20966 raft_consensus.cc:2804] T 9e65676b21b748728ad39668cfe722ec P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:21:04.182606 16817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a378d7f796f94615b3b347119a9880fd" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:21:04.186050 16816 raft_consensus.cc:1240] T 8638ee363273426db1393dd81e5c7b72 P cb82cc7bd6db46d980c75237a1402860 [term 14 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 13. Current term is 14. Ops: []
I20260709 13:21:04.192533 20964 raft_consensus.cc:2804] T 02838c86531d4f98bf58d1c22b8baf77 P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:21:04.197031 20939 raft_consensus.cc:493] T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:04.197592 20939 raft_consensus.cc:515] T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:04.178617 20959 raft_consensus.cc:3060] T c9944d5e4b8c4109af9955c4f08c9095 P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:21:04.193212 16817 raft_consensus.cc:2410] T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 for term 14 because replica has last-logged OpId of term: 13 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 9.
I20260709 13:21:04.111809 20892 raft_consensus.cc:493] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:04.207717 20966 raft_consensus.cc:697] T 9e65676b21b748728ad39668cfe722ec P 90f3da57796b488c9087a2f523adf045 [term 12 LEADER]: Becoming Leader. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:21:04.208724 20966 consensus_queue.cc:237] T 9e65676b21b748728ad39668cfe722ec P 90f3da57796b488c9087a2f523adf045 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:04.178900 20961 raft_consensus.cc:493] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:21:04.087692 20947 consensus_queue.cc:1048] T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.106s
I20260709 13:21:04.209679 20964 raft_consensus.cc:697] T 02838c86531d4f98bf58d1c22b8baf77 P 90f3da57796b488c9087a2f523adf045 [term 13 LEADER]: Becoming Leader. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:21:04.193771 20963 raft_consensus.cc:2749] T ada5a004ba7c46819051e3ccec71ffdb P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:21:04.215898 16816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f82224f4564fea93572e8206bad1fe" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:21:04.224941 16816 raft_consensus.cc:3060] T 23f82224f4564fea93572e8206bad1fe P cb82cc7bd6db46d980c75237a1402860 [term 9 FOLLOWER]: Advancing to term 11
W20260709 13:21:04.225658 16117 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 163888us
I20260709 13:21:04.058941 20803 consensus_queue.cc:237] T d2ab0ac8347641e6955e9bce7d90c1a2 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.11, Last appended by leader: 11, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:04.221517 20961 raft_consensus.cc:515] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:04.206730 20892 raft_consensus.cc:3060] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:21:04.206024 20661 leader_election.cc:290] T c4f65cdf00b748bb963ee6e81ef43b79 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 14 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
W20260709 13:21:04.252800 16117 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:04.321211 16145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e65676b21b748728ad39668cfe722ec" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:21:04.322016 16145 raft_consensus.cc:3060] T 9e65676b21b748728ad39668cfe722ec P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:21:04.239724 20911 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.791s	user 0.007s	sys 0.014s
I20260709 13:21:04.324653 16147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f2da98864b4425bf7c0055951fdd4b" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:04.325762 16146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4f65cdf00b748bb963ee6e81ef43b79" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:04.329417 16142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f82224f4564fea93572e8206bad1fe" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:04.330188 16142 raft_consensus.cc:2468] T 23f82224f4564fea93572e8206bad1fe P 647a343a542d4a139fe106c19095129b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 10.
I20260709 13:21:04.332851 16146 raft_consensus.cc:2393] T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 14: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
I20260709 13:21:04.328044 16147 raft_consensus.cc:2468] T 84f2da98864b4425bf7c0055951fdd4b P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 9.
I20260709 13:21:04.336068 16221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74ca8140f35c4ae29e8a664f6bdfc829" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:04.339811 16143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3884f864a96d428785895e5463e5728f" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:04.343295 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fffab26427e24d398e2695aad40ca134" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:04.259351 20231 raft_consensus.cc:493] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:04.343977 16150 raft_consensus.cc:2468] T fffab26427e24d398e2695aad40ca134 P 647a343a542d4a139fe106c19095129b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 10.
I20260709 13:21:04.317062 20939 leader_election.cc:290] T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
W20260709 13:21:04.332078 20911 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.878s	user 0.007s	sys 0.015s
I20260709 13:21:04.349958 16144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02838c86531d4f98bf58d1c22b8baf77" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:21:04.350000 16142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26354361e9b044cea1c2b86c4882fe92" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:21:04.351876 16144 raft_consensus.cc:3060] T 02838c86531d4f98bf58d1c22b8baf77 P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:21:04.354367 20882 raft_consensus.cc:493] T ecc1e363230043fe9fef390f0b979f20 P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:21:04.355019 20882 raft_consensus.cc:515] T ecc1e363230043fe9fef390f0b979f20 P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:04.360847 16816 raft_consensus.cc:2468] T 23f82224f4564fea93572e8206bad1fe P cb82cc7bd6db46d980c75237a1402860 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 11.
W20260709 13:21:04.364107 16149 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46224 (request call id 706) took 1206 ms (client timeout 1541 ms). Trace:
W20260709 13:21:04.364709 16149 rpcz_store.cc:269] 0709 13:21:03.157357 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:03.157491 (+   134us) service_pool.cc:224] Handling call
0709 13:21:04.363841 (+1206350us) spinlock_profiling.cc:243] Waited 291 ms on lock 0x7b540041dcd4. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:21:04.363985 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":669362048}
I20260709 13:21:04.366626 16115 leader_election.cc:304] T 84f2da98864b4425bf7c0055951fdd4b P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:21:04.368814 16115 leader_election.cc:304] T 02838c86531d4f98bf58d1c22b8baf77 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:21:04.368906 20799 raft_consensus.cc:515] T 66f75124215f4ca6bef831e174a792d8 P cb82cc7bd6db46d980c75237a1402860 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:04.371371 20964 consensus_queue.cc:237] T 02838c86531d4f98bf58d1c22b8baf77 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:21:04.372860 20938 log.cc:927] Time spent T 5a96373bb0e14dce8a266b2fc6c58507 P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.799s	user 0.002s	sys 0.001s
I20260709 13:21:04.376405 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f82224f4564fea93572e8206bad1fe" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:21:04.377017 16150 raft_consensus.cc:3060] T 23f82224f4564fea93572e8206bad1fe P 647a343a542d4a139fe106c19095129b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:21:04.380537 16146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4f65cdf00b748bb963ee6e81ef43b79" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:21:04.381306 16146 raft_consensus.cc:2393] T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 14: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
I20260709 13:21:04.385932 16142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66f75124215f4ca6bef831e174a792d8" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:21:04.386629 16142 raft_consensus.cc:3060] T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:21:04.388569 16790 leader_election.cc:304] T b87147731a68400e8cce9b879daa7c5d P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:21:04.391433 16191 leader_election.cc:304] T 08c74490c40243a88e62d1760b5cd814 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:21:04.392169 16295 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48674 (request call id 2780) took 1158 ms. Trace:
I20260709 13:21:04.342555 20961 leader_election.cc:290] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:04.399942 16146 raft_consensus.cc:1275] T ee9ed67006144797b1de6ea8c2013a68 P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:21:04.350584 20641 raft_consensus.cc:493] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:04.406711 16221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2372981006d4b218215c28964dea623" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:21:04.392573 16295 rpcz_store.cc:276] 0709 13:21:03.244138 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:03.361851 (+117713us) service_pool.cc:224] Handling call
0709 13:21:03.362186 (+   335us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:21:03.365424 (+  3238us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:21:03.365462 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:21:03.366039 (+   577us) log.cc:844] Serialized 26 byte log entry
0709 13:21:03.422150 (+ 56111us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:21:03.720605 (+298455us) log.cc:844] Serialized 14 byte log entry
0709 13:21:03.720882 (+   277us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:03.723477 (+  2595us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:21:04.372125 (+648648us) spinlock_profiling.cc:243] Waited 4.17 ms on lock 0x7b4400022b4c. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69ad8fadb1 00007f69ad998c85 00007f69ad9996a2 00007f69b070b1db 00007f69b0706aaf 00007f69b06fa86e 00007f69b06f9bb6 00007f69b06f97d8 00007f69b0703221 00007f69b07031d9 00007f69b0703169
0709 13:21:04.387681 (+ 15556us) raft_consensus.cc:1685] finished
0709 13:21:04.391930 (+  4249us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:21:04.392101 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":22305,"prepare.queue_time_us":64475,"prepare.run_cpu_time_us":62,"prepare.run_wall_time_us":2586,"raft.queue_time_us":2692,"raft.run_cpu_time_us":220,"raft.run_wall_time_us":2107,"spinlock_wait_cycles":9601792,"thread_start_us":291962,"threads_started":2,"wal-append.queue_time_us":136078}
I20260709 13:21:04.409749 16191 leader_election.cc:304] T 746aabf856ab401c9ef5e859371e7b67 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
I20260709 13:21:04.417521 16221 raft_consensus.cc:3060] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:21:04.420945 16145 raft_consensus.cc:2468] T 9e65676b21b748728ad39668cfe722ec P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 12.
I20260709 13:21:04.421626 16191 leader_election.cc:304] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
I20260709 13:21:04.422466 16225 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57604 (request call id 2692) took 1457 ms. Trace:
I20260709 13:21:04.423020 16225 rpcz_store.cc:276] 0709 13:21:02.965393 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:04.086559 (+1121166us) service_pool.cc:224] Handling call
0709 13:21:04.112310 (+ 25751us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:21:04.112481 (+   171us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:21:04.185224 (+ 72743us) log.cc:844] Serialized 14 byte log entry
0709 13:21:04.422147 (+236923us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:21:04.422200 (+    53us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:21:04.422243 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:04.422304 (+    61us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:21:04.422410 (+   106us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":72562,"prepare.run_cpu_time_us":52,"prepare.run_wall_time_us":53,"thread_start_us":236543,"threads_started":1}
I20260709 13:21:04.400003 20931 raft_consensus.cc:493] T b87147731a68400e8cce9b879daa7c5d P 90f3da57796b488c9087a2f523adf045 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:04.435878 16298 raft_consensus.cc:1240] T 8158bbd17b994d2eb96f560bd0f1b88d P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 11. Current term is 12. Ops: []
I20260709 13:21:04.403469 20667 raft_consensus.cc:2804] T b87147731a68400e8cce9b879daa7c5d P cb82cc7bd6db46d980c75237a1402860 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:21:04.400812 20889 raft_consensus.cc:2749] T 84f2da98864b4425bf7c0055951fdd4b P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:21:04.351490 20231 raft_consensus.cc:515] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:04.432464 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4f65cdf00b748bb963ee6e81ef43b79" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:04.460752 16300 raft_consensus.cc:2468] T c4f65cdf00b748bb963ee6e81ef43b79 P 90f3da57796b488c9087a2f523adf045 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 14.
I20260709 13:21:04.463855 16300 raft_consensus.cc:1240] T 8a878999c0024d2abe517484180290fc P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 11. Current term is 12. Ops: []
I20260709 13:21:04.401901 20799 leader_election.cc:290] T 66f75124215f4ca6bef831e174a792d8 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 12 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:21:04.456236 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66f75124215f4ca6bef831e174a792d8" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:21:04.475790 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02838c86531d4f98bf58d1c22b8baf77" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:21:04.419612 20961 raft_consensus.cc:2749] T 08c74490c40243a88e62d1760b5cd814 P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:21:04.420524 20886 raft_consensus.cc:2749] T 746aabf856ab401c9ef5e859371e7b67 P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:21:04.399326 20472 consensus_queue.cc:1048] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.034s
I20260709 13:21:04.462038 20911 raft_consensus.cc:493] T 31969392ccdb43b491911a319e8bc7e1 P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:21:04.502319 16265 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:04.502967 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a96373bb0e14dce8a266b2fc6c58507" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
W20260709 13:21:04.483489 20938 log.cc:927] Time spent T 5a96373bb0e14dce8a266b2fc6c58507 P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.058s	user 0.001s	sys 0.000s
I20260709 13:21:04.420706 20641 raft_consensus.cc:515] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:21:04.501227 20960 log.cc:927] Time spent T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 0.061s	user 0.001s	sys 0.000s
I20260709 13:21:04.507507 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 15 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:04.518687 16299 raft_consensus.cc:1275] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 12 index: 9. (term mismatch)
I20260709 13:21:04.533264 16299 pending_rounds.cc:85] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045: Aborting all ops after (but not including) 8
I20260709 13:21:04.472982 20667 raft_consensus.cc:493] T b87147731a68400e8cce9b879daa7c5d P cb82cc7bd6db46d980c75237a1402860 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 647a343a542d4a139fe106c19095129b)
W20260709 13:21:04.505673 16266 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:04.467346 20931 raft_consensus.cc:515] T b87147731a68400e8cce9b879daa7c5d P 90f3da57796b488c9087a2f523adf045 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:21:04.494055 20594 consensus_meta.cc:287] T fffab26427e24d398e2695aad40ca134 P 90f3da57796b488c9087a2f523adf045: Time spent flushing consensus metadata: real 0.562s	user 0.008s	sys 0.002s
I20260709 13:21:04.525880 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c11074579b046d99e585c2818e18a2c" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:04.458701 20836 consensus_queue.cc:1048] T b6f07842d3aa47858df5665874c86f3a P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.065s
I20260709 13:21:04.533645 16299 pending_rounds.cc:107] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045: Aborting uncommitted CHANGE_CONFIG_OP operation due to leader change: 10.9
I20260709 13:21:04.541682 16299 raft_consensus.cc:2917] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Aborting config change with OpId 10.9: Aborted: Op aborted by new leader
I20260709 13:21:04.542400 16299 raft_consensus.cc:2889] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: CHANGE_CONFIG_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:21:04.500919 20780 consensus_queue.cc:1048] T ee9ed67006144797b1de6ea8c2013a68 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.038s
I20260709 13:21:04.511998 16142 raft_consensus.cc:2468] T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 12.
I20260709 13:21:04.549939 16293 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48684 (request call id 2599) took 1549 ms. Trace:
I20260709 13:21:04.504716 20889 consensus_queue.cc:1048] T 829b4ad8c5a54eababeea1181b960d4b P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.035s
I20260709 13:21:04.550505 16293 rpcz_store.cc:276] 0709 13:21:03.000173 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:03.006821 (+  6648us) service_pool.cc:224] Handling call
0709 13:21:04.549687 (+1542866us) spinlock_profiling.cc:243] Waited 304 ms on lock 0x7b540013ddd4. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:21:04.549864 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":700010880}
I20260709 13:21:04.537068 20667 raft_consensus.cc:3060] T b87147731a68400e8cce9b879daa7c5d P cb82cc7bd6db46d980c75237a1402860 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:21:04.527928 20236 raft_consensus.cc:3060] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:21:04.536654 20839 raft_consensus.cc:493] T 8158bbd17b994d2eb96f560bd0f1b88d P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
W20260709 13:21:04.554577 20919 consensus_meta.cc:287] T ecc1e363230043fe9fef390f0b979f20 P 90f3da57796b488c9087a2f523adf045: Time spent flushing consensus metadata: real 0.591s	user 0.010s	sys 0.002s
I20260709 13:21:04.519515 20911 raft_consensus.cc:515] T 31969392ccdb43b491911a319e8bc7e1 P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:04.571105 16150 raft_consensus.cc:2468] T 23f82224f4564fea93572e8206bad1fe P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 11.
I20260709 13:21:04.543537 20594 raft_consensus.cc:515] T fffab26427e24d398e2695aad40ca134 P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:04.559262 20973 raft_consensus.cc:493] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:04.600981 16816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62fe3346cd3c42c4a55e14af9073bca4" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:04.601724 16816 raft_consensus.cc:2468] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 16.
I20260709 13:21:04.578125 20961 raft_consensus.cc:493] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:21:04.602614 16117 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 218372us
I20260709 13:21:04.604928 16117 leader_election.cc:304] T 434bff3082d141838068f323bdb0d1ed P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:21:04.637893 16817 raft_consensus.cc:1275] T af57996a8be84d398f0f023e8d08ef77 P cb82cc7bd6db46d980c75237a1402860 [term 14 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 13:21:04.639693 20472 consensus_queue.cc:1048] T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:21:04.640447 16117 leader_election.cc:304] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:21:04.641831 20472 raft_consensus.cc:2804] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:21:04.642241 20472 raft_consensus.cc:493] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:04.642645 20472 raft_consensus.cc:3060] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:21:04.639680 20757 raft_consensus.cc:2804] T 434bff3082d141838068f323bdb0d1ed P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:21:04.644199 20757 raft_consensus.cc:697] T 434bff3082d141838068f323bdb0d1ed P 647a343a542d4a139fe106c19095129b [term 12 LEADER]: Becoming Leader. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:21:04.644996 16265 leader_election.cc:304] T 84f2da98864b4425bf7c0055951fdd4b P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac
I20260709 13:21:04.645218 20757 consensus_queue.cc:237] T 434bff3082d141838068f323bdb0d1ed P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:04.653131 16142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c61a39fd4f84424b129c1a6f9ba404d" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 14 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:04.654327 16142 raft_consensus.cc:2410] T 9c61a39fd4f84424b129c1a6f9ba404d P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 for term 14 because replica has last-logged OpId of term: 11 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
W20260709 13:21:04.601773 20851 consensus_meta.cc:287] T 8c11074579b046d99e585c2818e18a2c P 1ca2725d290e47d68b1a4b609c9816ac: Time spent flushing consensus metadata: real 0.547s	user 0.010s	sys 0.001s
I20260709 13:21:04.659499 16265 leader_election.cc:304] T 3884f864a96d428785895e5463e5728f P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
I20260709 13:21:04.578466 20886 raft_consensus.cc:493] T 68a6838d82ef4cac83507f906139d90f P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
W20260709 13:21:04.687638 16115 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:04.690286 16142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d4ab870a05347868956aa8b841b2981" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:04.690918 16265 leader_election.cc:304] T 23f82224f4564fea93572e8206bad1fe P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:21:04.691689 16150 raft_consensus.cc:3060] T 9c61a39fd4f84424b129c1a6f9ba404d P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:21:04.693379 16146 raft_consensus.cc:1275] T 8cc959fb7f704fba9009c0595467d074 P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 13:21:04.580505 20919 raft_consensus.cc:515] T ecc1e363230043fe9fef390f0b979f20 P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:04.694883 16146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02838c86531d4f98bf58d1c22b8baf77" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:21:04.695619 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af57996a8be84d398f0f023e8d08ef77" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:21:04.696137 16146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4f65cdf00b748bb963ee6e81ef43b79" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:04.696900 16146 raft_consensus.cc:2468] T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 14.
W20260709 13:21:04.698419 20975 log.cc:927] Time spent T 66ae176961b74813a58bc15c9c4c6f2a P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.098s	user 0.000s	sys 0.000s
W20260709 13:21:04.699316 20979 log.cc:927] Time spent T 8638ee363273426db1393dd81e5c7b72 P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.120s	user 0.001s	sys 0.000s
W20260709 13:21:04.701016 20974 log.cc:927] Time spent T 5a96373bb0e14dce8a266b2fc6c58507 P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 0.112s	user 0.000s	sys 0.000s
W20260709 13:21:04.702226 20976 log.cc:927] Time spent T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860: Append to log took a long time: real 0.164s	user 0.000s	sys 0.001s
I20260709 13:21:04.658416 20832 raft_consensus.cc:493] T 68a6838d82ef4cac83507f906139d90f P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:04.702981 20472 raft_consensus.cc:515] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:04.705183 16115 leader_election.cc:304] T b6f07842d3aa47858df5665874c86f3a P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:21:04.705892 20594 leader_election.cc:290] T fffab26427e24d398e2695aad40ca134 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 11 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:21:04.660806 20964 raft_consensus.cc:493] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:04.706921 20964 raft_consensus.cc:515] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:21:04.707757 20917 log.cc:927] Time spent T 3a7ca61592e848ea9b0fb81a2ce33cdc P cb82cc7bd6db46d980c75237a1402860: Append to log took a long time: real 1.403s	user 0.001s	sys 0.004s
I20260709 13:21:04.709096 20964 leader_election.cc:290] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:21:04.708772 20472 leader_election.cc:290] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 15 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:04.709976 20641 leader_election.cc:290] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:04.710659 20231 leader_election.cc:290] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:04.711176 20231 raft_consensus.cc:493] T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:04.711615 20231 raft_consensus.cc:515] T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:04.713531 20231 leader_election.cc:290] T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:04.713966 20231 raft_consensus.cc:493] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:21:04.714462 20231 raft_consensus.cc:515] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:04.590484 20839 raft_consensus.cc:515] T 8158bbd17b994d2eb96f560bd0f1b88d P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:04.719596 16820 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49906 (request call id 945) took 1982 ms. Trace:
W20260709 13:21:04.703691 20968 log.cc:927] Time spent T 8cc959fb7f704fba9009c0595467d074 P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 0.160s	user 0.001s	sys 0.000s
I20260709 13:21:04.703575 20832 raft_consensus.cc:515] T 68a6838d82ef4cac83507f906139d90f P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:04.704008 20886 raft_consensus.cc:515] T 68a6838d82ef4cac83507f906139d90f P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:04.602291 20973 raft_consensus.cc:515] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } }
I20260709 13:21:04.749374 16150 raft_consensus.cc:1275] T 9c61a39fd4f84424b129c1a6f9ba404d P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 13:21:04.696799 20919 leader_election.cc:290] T ecc1e363230043fe9fef390f0b979f20 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 13 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:04.696220 16149 raft_consensus.cc:3060] T af57996a8be84d398f0f023e8d08ef77 P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:21:04.668309 20895 raft_consensus.cc:2804] T 84f2da98864b4425bf7c0055951fdd4b P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:21:04.714579 20892 raft_consensus.cc:515] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:04.685859 20967 raft_consensus.cc:2749] T 3884f864a96d428785895e5463e5728f P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:21:04.754057 16142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee9ed67006144797b1de6ea8c2013a68" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:21:04.720052 16820 rpcz_store.cc:276] 0709 13:21:02.737332 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:02.737453 (+   121us) service_pool.cc:224] Handling call
0709 13:21:02.746138 (+  8685us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:21:02.895502 (+149364us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 13:21:03.079673 (+184171us) log.cc:844] Serialized 14 byte log entry
0709 13:21:03.162692 (+ 83019us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:21:03.204809 (+ 42117us) log.cc:844] Serialized 26 byte log entry
0709 13:21:03.205153 (+   344us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:21:03.205224 (+    71us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:03.205393 (+   169us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:21:03.719672 (+514279us) spinlock_profiling.cc:243] Waited 8.58 ms on lock 0x7b18004452a0. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b693ade1 00007f69b69475b1 00007f69ada72522 00007f69ada747c1 00007f69ada74779 00007f69ada74709 00007f69ada746d1 00007f69ada739cd 000000000038f354 000000000038f199 00007f69ada5583a
0709 13:21:04.635536 (+915864us) spinlock_profiling.cc:243] Waited 1.63 ms on lock 0x7b4400022b4c. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69ad8fadb1 00007f69ad998c85 00007f69ad9996a2 00007f69b070b1db 00007f69b0706aaf 00007f69b06fa86e 00007f69b06f9bb6 00007f69b06f97d8 00007f69b0703221 00007f69b07031d9 00007f69b0703169
0709 13:21:04.719341 (+ 83805us) raft_consensus.cc:1685] finished
0709 13:21:04.719364 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:21:04.719535 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2509,"prepare.queue_time_us":263211,"prepare.run_cpu_time_us":57,"prepare.run_wall_time_us":2841,"raft.queue_time_us":97018,"raft.run_cpu_time_us":243,"raft.run_wall_time_us":51921,"spinlock_wait_cycles":23474432,"thread_start_us":365448,"threads_started":3,"wal-append.queue_time_us":156473}
W20260709 13:21:04.697698 20984 log.cc:927] Time spent T 9e65676b21b748728ad39668cfe722ec P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.093s	user 0.000s	sys 0.000s
I20260709 13:21:04.757709 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62fe3346cd3c42c4a55e14af9073bca4" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:04.728545 20836 raft_consensus.cc:493] T 5a959dbecc2648eb85d1cc550f5a3245 P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:04.602128 20959 raft_consensus.cc:515] T c9944d5e4b8c4109af9955c4f08c9095 P 90f3da57796b488c9087a2f523adf045 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:04.735986 20939 raft_consensus.cc:2804] T 23f82224f4564fea93572e8206bad1fe P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:21:04.760984 16142 raft_consensus.cc:2368] T ee9ed67006144797b1de6ea8c2013a68 P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac for earlier term 10. Current term is 11.
I20260709 13:21:04.773229 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a96373bb0e14dce8a266b2fc6c58507" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:04.774879 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:21:04.777508 16150 raft_consensus.cc:3060] T ecc1e363230043fe9fef390f0b979f20 P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:21:04.778398 16144 raft_consensus.cc:2468] T 02838c86531d4f98bf58d1c22b8baf77 P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 13.
I20260709 13:21:04.660938 20851 raft_consensus.cc:515] T 8c11074579b046d99e585c2818e18a2c P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:04.783866 16821 raft_consensus.cc:2368] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 for earlier term 14. Current term is 15.
I20260709 13:21:04.683632 20961 raft_consensus.cc:515] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:04.786628 16148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b7d1ab519044eda2d5e04a07242f25" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 12 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:04.757736 20895 raft_consensus.cc:493] T 84f2da98864b4425bf7c0055951fdd4b P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
W20260709 13:21:04.789935 16115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 84816.4us
I20260709 13:21:04.796677 16115 leader_election.cc:304] T 1df93779209748b9860f6d893dd3de53 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:21:04.801182 16144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87147731a68400e8cce9b879daa7c5d" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:04.801931 16144 raft_consensus.cc:2468] T b87147731a68400e8cce9b879daa7c5d P 647a343a542d4a139fe106c19095129b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 14.
I20260709 13:21:04.770139 20672 raft_consensus.cc:493] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:04.803794 16265 leader_election.cc:304] T b87147731a68400e8cce9b879daa7c5d P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:21:04.839394 16224 raft_consensus.cc:1240] T 8c11074579b046d99e585c2818e18a2c P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 12. Current term is 13. Ops: []
I20260709 13:21:04.840426 16144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 15 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:04.842691 16142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fffab26427e24d398e2695aad40ca134" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:21:04.843796 16142 raft_consensus.cc:3060] T fffab26427e24d398e2695aad40ca134 P 647a343a542d4a139fe106c19095129b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:21:04.783766 20754 raft_consensus.cc:2749] T 02838c86531d4f98bf58d1c22b8baf77 P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:21:04.850415 16144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f5f89186204242bf377f7c42b2e2da" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:04.759130 20267 consensus_queue.cc:1048] T 8cc959fb7f704fba9009c0595467d074 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.045s
I20260709 13:21:04.851799 16144 raft_consensus.cc:2410] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 for term 14 because replica has last-logged OpId of term: 12 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 7.
I20260709 13:21:04.788437 20939 raft_consensus.cc:697] T 23f82224f4564fea93572e8206bad1fe P 90f3da57796b488c9087a2f523adf045 [term 11 LEADER]: Becoming Leader. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:21:04.797498 20895 raft_consensus.cc:3060] T 84f2da98864b4425bf7c0055951fdd4b P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:21:04.860458 16221 raft_consensus.cc:2468] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 14.
I20260709 13:21:04.869388 16150 raft_consensus.cc:2468] T ecc1e363230043fe9fef390f0b979f20 P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 13.
I20260709 13:21:04.834640 20931 leader_election.cc:290] T b87147731a68400e8cce9b879daa7c5d P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:04.775409 20836 raft_consensus.cc:515] T 5a959dbecc2648eb85d1cc550f5a3245 P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:04.855692 20964 raft_consensus.cc:2804] T b87147731a68400e8cce9b879daa7c5d P 90f3da57796b488c9087a2f523adf045 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:21:04.891227 16144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31969392ccdb43b491911a319e8bc7e1" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:04.892894 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4f65cdf00b748bb963ee6e81ef43b79" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:21:04.905853 20992 raft_consensus.cc:2749] T 1df93779209748b9860f6d893dd3de53 P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:21:04.895043 16221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fffab26427e24d398e2695aad40ca134" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:21:04.907330 16221 raft_consensus.cc:3060] T fffab26427e24d398e2695aad40ca134 P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:21:04.871480 20799 raft_consensus.cc:493] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P cb82cc7bd6db46d980c75237a1402860 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:04.870510 20470 raft_consensus.cc:2749] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:21:04.927832 16265 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:04.897356 16150 raft_consensus.cc:1240] T 8a878999c0024d2abe517484180290fc P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 11. Current term is 12. Ops: []
I20260709 13:21:04.929827 16144 raft_consensus.cc:1240] T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 11. Current term is 12. Ops: []
I20260709 13:21:04.918272 16149 raft_consensus.cc:2468] T af57996a8be84d398f0f023e8d08ef77 P 647a343a542d4a139fe106c19095129b [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 14.
W20260709 13:21:04.924445 20995 log.cc:927] Time spent T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.123s	user 0.001s	sys 0.000s
I20260709 13:21:04.924247 20939 consensus_queue.cc:237] T 23f82224f4564fea93572e8206bad1fe P 90f3da57796b488c9087a2f523adf045 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:04.925724 20991 raft_consensus.cc:2749] T b6f07842d3aa47858df5665874c86f3a P 647a343a542d4a139fe106c19095129b [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:21:04.933990 16142 raft_consensus.cc:2468] T fffab26427e24d398e2695aad40ca134 P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 11.
I20260709 13:21:04.849942 20672 raft_consensus.cc:515] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:04.938181 16147 raft_consensus.cc:1275] T af57996a8be84d398f0f023e8d08ef77 P 647a343a542d4a139fe106c19095129b [term 14 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 13:21:04.901273 20964 raft_consensus.cc:493] T b87147731a68400e8cce9b879daa7c5d P 90f3da57796b488c9087a2f523adf045 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:04.941174 20663 raft_consensus.cc:493] T 8d4ab870a05347868956aa8b841b2981 P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
W20260709 13:21:04.950659 20993 log.cc:927] Time spent T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 0.116s	user 0.001s	sys 0.000s
W20260709 13:21:04.951221 20978 log.cc:927] Time spent T c9944d5e4b8c4109af9955c4f08c9095 P cb82cc7bd6db46d980c75237a1402860: Append to log took a long time: real 0.190s	user 0.001s	sys 0.000s
W20260709 13:21:04.952116 20981 log.cc:927] Time spent T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860: Append to log took a long time: real 0.284s	user 0.000s	sys 0.001s
I20260709 13:21:04.970645 16218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f5f89186204242bf377f7c42b2e2da" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:04.974782 16218 raft_consensus.cc:2468] T c2f5f89186204242bf377f7c42b2e2da P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 13.
I20260709 13:21:04.953559 20911 leader_election.cc:290] T 31969392ccdb43b491911a319e8bc7e1 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:04.958321 20964 raft_consensus.cc:3060] T b87147731a68400e8cce9b879daa7c5d P 90f3da57796b488c9087a2f523adf045 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:21:04.988603 16217 raft_consensus.cc:2468] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 14.
I20260709 13:21:04.968691 20663 raft_consensus.cc:515] T 8d4ab870a05347868956aa8b841b2981 P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:21:04.989418 20916 log.cc:927] Time spent T 08c74490c40243a88e62d1760b5cd814 P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 1.719s	user 0.005s	sys 0.000s
I20260709 13:21:04.935860 20799 raft_consensus.cc:515] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P cb82cc7bd6db46d980c75237a1402860 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:04.967571 20839 leader_election.cc:290] T 8158bbd17b994d2eb96f560bd0f1b88d P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:04.997536 16218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66f75124215f4ca6bef831e174a792d8" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:05.000451 16151 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40830 (request call id 1563) took 2013 ms. Trace:
I20260709 13:21:05.009331 20961 leader_election.cc:290] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:05.015285 20231 leader_election.cc:290] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
W20260709 13:21:05.015874 20989 log.cc:927] Time spent T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.320s	user 0.001s	sys 0.000s
I20260709 13:21:05.028303 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5d59407eb545b895d8d922e0dd69fd" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:05.029089 16818 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49906 (request call id 958) took 1023 ms. Trace:
I20260709 13:21:05.029307 16218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5d59407eb545b895d8d922e0dd69fd" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:21:05.029018 16226 raft_consensus.cc:2468] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 14.
I20260709 13:21:05.032341 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:21:05.033097 16217 raft_consensus.cc:3060] T ecc1e363230043fe9fef390f0b979f20 P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Advancing to term 13
I20260709 13:21:05.017966 16151 rpcz_store.cc:276] 0709 13:21:02.987409 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:02.987621 (+   212us) service_pool.cc:224] Handling call
0709 13:21:03.008283 (+ 20662us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:21:03.042774 (+ 34491us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:21:03.043113 (+   339us) log.cc:844] Serialized 14 byte log entry
0709 13:21:03.130218 (+ 87105us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:21:03.165475 (+ 35257us) log.cc:844] Serialized 52 byte log entry
0709 13:21:03.165644 (+   169us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:21:03.168950 (+  3306us) log.cc:844] Serialized 14 byte log entry
0709 13:21:03.169101 (+   151us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:03.169149 (+    48us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:21:04.735449 (+1566300us) spinlock_profiling.cc:243] Waited 3.74 ms on lock 0x7b4400022b4c. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69ad8fadb1 00007f69ad998c85 00007f69ad9996a2 00007f69b070b1db 00007f69b0706aaf 00007f69b06fa86e 00007f69b06f9bb6 00007f69b06f97d8 00007f69b0703221 00007f69b07031d9 00007f69b0703169
0709 13:21:05.000167 (+264718us) raft_consensus.cc:1685] finished
0709 13:21:05.000193 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:21:05.000359 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":15450,"prepare.queue_time_us":4563,"prepare.run_cpu_time_us":96,"prepare.run_wall_time_us":3333,"raft.queue_time_us":22942,"raft.run_cpu_time_us":325,"raft.run_wall_time_us":22463,"spinlock_wait_cycles":8595328,"thread_start_us":49187,"threads_started":1,"wal-append.queue_time_us":128130}
I20260709 13:21:05.029614 16818 rpcz_store.cc:276] 0709 13:21:04.006080 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:04.006244 (+   164us) service_pool.cc:224] Handling call
0709 13:21:04.006612 (+   368us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:21:04.018801 (+ 12189us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:21:04.018820 (+    19us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:21:04.022873 (+  4053us) log.cc:844] Serialized 26 byte log entry
0709 13:21:04.518449 (+495576us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:21:04.518490 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:04.518565 (+    75us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:21:05.028719 (+510154us) raft_consensus.cc:1685] finished
0709 13:21:05.028751 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:21:05.028988 (+   237us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2919,"raft.run_cpu_time_us":197,"raft.run_wall_time_us":2228,"thread_start_us":443085,"threads_started":1,"wal-append.queue_time_us":573976}
W20260709 13:21:05.040095 16115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 243513us
I20260709 13:21:05.041004 16115 leader_election.cc:304] T e5ad4eb281564936a59f41e387a47156 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:21:05.042271 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f5f89186204242bf377f7c42b2e2da" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:05.042954 16226 raft_consensus.cc:2468] T c2f5f89186204242bf377f7c42b2e2da P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 13.
I20260709 13:21:05.043522 16115 leader_election.cc:304] T 57a744f5aaf944678a63469cbf514015 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860
I20260709 13:21:05.043846 20236 raft_consensus.cc:2749] T e5ad4eb281564936a59f41e387a47156 P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:21:05.023286 20832 leader_election.cc:290] T 68a6838d82ef4cac83507f906139d90f P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:21:05.049897 16115 leader_election.cc:304] T 18ad7e3af5504571a69d10c00750a01c P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:21:05.048641 20472 raft_consensus.cc:493] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:05.051925 20663 leader_election.cc:290] T 8d4ab870a05347868956aa8b841b2981 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:21:05.054387 16218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c61a39fd4f84424b129c1a6f9ba404d" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 14 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:05.054387 16219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4f65cdf00b748bb963ee6e81ef43b79" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 15 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:05.014979 20667 raft_consensus.cc:515] T b87147731a68400e8cce9b879daa7c5d P cb82cc7bd6db46d980c75237a1402860 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:05.058207 20882 leader_election.cc:290] T ecc1e363230043fe9fef390f0b979f20 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
W20260709 13:21:05.059779 21006 log.cc:927] Time spent T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 0.070s	user 0.000s	sys 0.000s
W20260709 13:21:05.061268 21004 log.cc:927] Time spent T e5ad4eb281564936a59f41e387a47156 P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 0.077s	user 0.000s	sys 0.000s
I20260709 13:21:05.064358 16219 raft_consensus.cc:2410] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b for term 15 because replica has last-logged OpId of term: 13 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 8.
I20260709 13:21:05.070377 20667 leader_election.cc:290] T b87147731a68400e8cce9b879daa7c5d P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 14 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:05.071331 20757 raft_consensus.cc:2749] T 18ad7e3af5504571a69d10c00750a01c P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:21:05.071712 20959 leader_election.cc:290] T c9944d5e4b8c4109af9955c4f08c9095 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 14 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:05.071925 20836 leader_election.cc:290] T 5a959dbecc2648eb85d1cc550f5a3245 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:05.058677 20472 raft_consensus.cc:515] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:05.072898 16115 leader_election.cc:304] T 9c61a39fd4f84424b129c1a6f9ba404d P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:21:05.074012 20236 raft_consensus.cc:2749] T 9c61a39fd4f84424b129c1a6f9ba404d P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:21:05.075387 20892 leader_election.cc:290] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 15 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:05.075593 16115 leader_election.cc:304] T 8a878999c0024d2abe517484180290fc P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac
I20260709 13:21:05.076332 20839 raft_consensus.cc:2804] T 8a878999c0024d2abe517484180290fc P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:21:05.076817 20839 raft_consensus.cc:697] T 8a878999c0024d2abe517484180290fc P 647a343a542d4a139fe106c19095129b [term 12 LEADER]: Becoming Leader. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:21:05.077718 20839 consensus_queue.cc:237] T 8a878999c0024d2abe517484180290fc P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:05.081729 20236 consensus_queue.cc:1048] T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:21:05.084093 20236 consensus_queue.cc:1048] T 68a6838d82ef4cac83507f906139d90f P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:21:05.085829 20236 consensus_queue.cc:1048] T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:21:05.055563 20990 raft_consensus.cc:2749] T 57a744f5aaf944678a63469cbf514015 P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:21:05.088253 16222 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33050 (request call id 812) took 1730 ms (client timeout 1937 ms). Trace:
I20260709 13:21:05.088575 20799 leader_election.cc:290] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
W20260709 13:21:05.088742 16222 rpcz_store.cc:269] 0709 13:21:03.358087 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:03.359914 (+  1827us) service_pool.cc:224] Handling call
0709 13:21:05.088027 (+1728113us) spinlock_profiling.cc:243] Waited 336 ms on lock 0x7b54002b4654. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:21:05.088162 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":772191488}
I20260709 13:21:05.089545 16225 raft_consensus.cc:1275] T ee9ed67006144797b1de6ea8c2013a68 P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:21:05.092527 20672 leader_election.cc:290] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:05.092648 16822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2794af5b99984efebdee564035841cb0" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:21:05.093317 16822 raft_consensus.cc:2376] T 2794af5b99984efebdee564035841cb0 P cb82cc7bd6db46d980c75237a1402860 [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 647a343a542d4a139fe106c19095129b in term 10. Re-sending same reply.
I20260709 13:21:05.094271 16266 leader_election.cc:304] T 26354361e9b044cea1c2b86c4882fe92 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
I20260709 13:21:05.096939 20972 raft_consensus.cc:2749] T 26354361e9b044cea1c2b86c4882fe92 P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:21:05.096864 16825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5d59407eb545b895d8d922e0dd69fd" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:21:05.097652 16825 raft_consensus.cc:3060] T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:21:05.100159 20892 raft_consensus.cc:493] T 806d7690c2b644959152074bb2cab0a7 P cb82cc7bd6db46d980c75237a1402860 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:05.100627 16819 raft_consensus.cc:1275] T 2794af5b99984efebdee564035841cb0 P cb82cc7bd6db46d980c75237a1402860 [term 10 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:21:05.100694 20892 raft_consensus.cc:515] T 806d7690c2b644959152074bb2cab0a7 P cb82cc7bd6db46d980c75237a1402860 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:05.053426 20998 raft_consensus.cc:493] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
W20260709 13:21:05.103865 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 175280us
I20260709 13:21:05.064397 16221 raft_consensus.cc:2468] T fffab26427e24d398e2695aad40ca134 P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 11.
I20260709 13:21:05.113288 16265 leader_election.cc:304] T c4f65cdf00b748bb963ee6e81ef43b79 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 647a343a542d4a139fe106c19095129b
I20260709 13:21:05.110577 16789 leader_election.cc:304] T 23f82224f4564fea93572e8206bad1fe P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 647a343a542d4a139fe106c19095129b
I20260709 13:21:05.126668 16265 leader_election.cc:304] T fffab26427e24d398e2695aad40ca134 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
W20260709 13:21:05.138803 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 340298us
I20260709 13:21:05.140089 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d4ab870a05347868956aa8b841b2981" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:05.148094 16820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:21:05.148762 16820 raft_consensus.cc:3060] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:21:05.149914 16819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31969392ccdb43b491911a319e8bc7e1" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:05.151492 16225 raft_consensus.cc:1275] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 13:21:05.153857 16816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87147731a68400e8cce9b879daa7c5d" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:05.154743 16816 raft_consensus.cc:2468] T b87147731a68400e8cce9b879daa7c5d P cb82cc7bd6db46d980c75237a1402860 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 14.
I20260709 13:21:05.096786 20663 consensus_queue.cc:1059] T 8638ee363273426db1393dd81e5c7b72 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 9, Time since last communication: 0.027s
I20260709 13:21:05.159142 16816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9944d5e4b8c4109af9955c4f08c9095" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:21:05.151723 16825 raft_consensus.cc:2468] T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 15.
I20260709 13:21:05.156774 16143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68a6838d82ef4cac83507f906139d90f" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
W20260709 13:21:05.158154 16115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 44508.4us
I20260709 13:21:05.174219 16217 raft_consensus.cc:2468] T ecc1e363230043fe9fef390f0b979f20 P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 13.
I20260709 13:21:05.177683 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9944d5e4b8c4109af9955c4f08c9095" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:21:05.125456 20977 raft_consensus.cc:493] T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:21:05.148988 20919 raft_consensus.cc:2804] T fffab26427e24d398e2695aad40ca134 P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:21:05.187752 16819 raft_consensus.cc:1275] T 9e65676b21b748728ad39668cfe722ec P cb82cc7bd6db46d980c75237a1402860 [term 12 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:21:05.139058 20959 raft_consensus.cc:2804] T c4f65cdf00b748bb963ee6e81ef43b79 P 90f3da57796b488c9087a2f523adf045 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:21:05.183183 16145 raft_consensus.cc:1275] T 02838c86531d4f98bf58d1c22b8baf77 P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
W20260709 13:21:05.184945 16789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 74506.8us
I20260709 13:21:05.189687 16789 leader_election.cc:304] T 8638ee363273426db1393dd81e5c7b72 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:21:05.185006 16221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68a6838d82ef4cac83507f906139d90f" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:05.191390 16221 raft_consensus.cc:2410] T 68a6838d82ef4cac83507f906139d90f P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 for term 14 because replica has last-logged OpId of term: 13 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 6.
I20260709 13:21:05.186283 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d4ab870a05347868956aa8b841b2981" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:05.192483 16225 raft_consensus.cc:2468] T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 11.
I20260709 13:21:05.207733 16789 leader_election.cc:304] T ada5a004ba7c46819051e3ccec71ffdb P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:21:05.209525 16217 raft_consensus.cc:1275] T 02838c86531d4f98bf58d1c22b8baf77 P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:21:05.209756 20696 consensus_queue.cc:1059] T d2ab0ac8347641e6955e9bce7d90c1a2 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:21:05.211210 20696 raft_consensus.cc:2749] T ada5a004ba7c46819051e3ccec71ffdb P cb82cc7bd6db46d980c75237a1402860 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:21:05.212108 20696 raft_consensus.cc:3055] T d2ab0ac8347641e6955e9bce7d90c1a2 P cb82cc7bd6db46d980c75237a1402860 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:21:05.212489 15046 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:21:05.209555 20641 raft_consensus.cc:2749] T 8638ee363273426db1393dd81e5c7b72 P cb82cc7bd6db46d980c75237a1402860 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:21:05.212520 20696 raft_consensus.cc:740] T d2ab0ac8347641e6955e9bce7d90c1a2 P cb82cc7bd6db46d980c75237a1402860 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:21:05.215413 20696 consensus_queue.cc:260] T d2ab0ac8347641e6955e9bce7d90c1a2 P cb82cc7bd6db46d980c75237a1402860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:05.187426 20985 raft_consensus.cc:2764] T 23f82224f4564fea93572e8206bad1fe P cb82cc7bd6db46d980c75237a1402860 [term 11 FOLLOWER]: Leader pre-election decision vote started in defunct term 9: won
I20260709 13:21:05.233268 20947 raft_consensus.cc:493] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:05.234045 20947 raft_consensus.cc:515] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:05.236850 20947 leader_election.cc:290] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:21:05.184995 20663 raft_consensus.cc:3055] T 8638ee363273426db1393dd81e5c7b72 P 90f3da57796b488c9087a2f523adf045 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:21:05.244624 20663 raft_consensus.cc:740] T 8638ee363273426db1393dd81e5c7b72 P 90f3da57796b488c9087a2f523adf045 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:21:05.246143 20663 consensus_queue.cc:260] T 8638ee363273426db1393dd81e5c7b72 P 90f3da57796b488c9087a2f523adf045 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:05.249945 16221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c74490c40243a88e62d1760b5cd814" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:05.281598 16221 raft_consensus.cc:2468] T 08c74490c40243a88e62d1760b5cd814 P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 12.
I20260709 13:21:05.250793 20919 raft_consensus.cc:697] T fffab26427e24d398e2695aad40ca134 P 90f3da57796b488c9087a2f523adf045 [term 11 LEADER]: Becoming Leader. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:21:05.252995 20892 leader_election.cc:290] T 806d7690c2b644959152074bb2cab0a7 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:05.140623 20998 raft_consensus.cc:515] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:05.280771 16116 leader_election.cc:304] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
I20260709 13:21:05.304275 20236 raft_consensus.cc:2804] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:21:05.304875 20236 raft_consensus.cc:493] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:05.305315 20236 raft_consensus.cc:3060] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:21:05.308254 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 346731us
I20260709 13:21:05.260207 20991 raft_consensus.cc:493] T 746aabf856ab401c9ef5e859371e7b67 P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:21:05.308353 16116 leader_election.cc:304] T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860
I20260709 13:21:05.260488 20959 raft_consensus.cc:697] T c4f65cdf00b748bb963ee6e81ef43b79 P 90f3da57796b488c9087a2f523adf045 [term 14 LEADER]: Becoming Leader. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:21:05.301563 20911 raft_consensus.cc:493] T e4dc6b0541ca431ebc757beac00521f1 P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:05.303328 20882 raft_consensus.cc:493] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:21:05.303272 20803 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.585s	user 0.009s	sys 0.023s
I20260709 13:21:05.203186 20977 raft_consensus.cc:515] T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:05.335559 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8158bbd17b994d2eb96f560bd0f1b88d" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:05.343436 16225 raft_consensus.cc:2468] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 12.
I20260709 13:21:05.339936 16221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87ac0d1f7e8a419f9bcbc04aee0e067e" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:05.259150 20964 raft_consensus.cc:515] T b87147731a68400e8cce9b879daa7c5d P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:05.345521 16221 raft_consensus.cc:2468] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 13.
I20260709 13:21:05.309593 20799 raft_consensus.cc:493] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:21:05.262400 20895 raft_consensus.cc:515] T 84f2da98864b4425bf7c0055951fdd4b P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:05.330814 20663 raft_consensus.cc:3060] T 8638ee363273426db1393dd81e5c7b72 P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:21:05.315649 20836 raft_consensus.cc:493] T 18ad7e3af5504571a69d10c00750a01c P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:21:05.361666 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68a6838d82ef4cac83507f906139d90f" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:05.363931 16116 leader_election.cc:304] T 8158bbd17b994d2eb96f560bd0f1b88d P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
I20260709 13:21:05.332652 21008 raft_consensus.cc:2749] T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:21:05.362582 16144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ad5dbfa99ee4b8d9e28be92b3f36db5" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:05.367856 16144 raft_consensus.cc:2468] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 11.
I20260709 13:21:05.362680 16148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
W20260709 13:21:05.340479 20803 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.622s	user 0.010s	sys 0.024s
I20260709 13:21:05.366739 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a959dbecc2648eb85d1cc550f5a3245" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
W20260709 13:21:05.372035 16788 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:05.374235 16148 raft_consensus.cc:2368] T ecc1e363230043fe9fef390f0b979f20 P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac for earlier term 12. Current term is 13.
I20260709 13:21:05.374433 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5d59407eb545b895d8d922e0dd69fd" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 15 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:05.342067 20911 raft_consensus.cc:515] T e4dc6b0541ca431ebc757beac00521f1 P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:05.385825 16149 raft_consensus.cc:2393] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 15: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
W20260709 13:21:05.386030 16789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 178411us
I20260709 13:21:05.387080 16789 leader_election.cc:304] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:21:05.388816 16789 leader_election.cc:304] T 74ca8140f35c4ae29e8a664f6bdfc829 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:21:05.390735 20236 raft_consensus.cc:515] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:05.387368 16191 leader_election.cc:304] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
I20260709 13:21:05.392278 16789 leader_election.cc:304] T 3884f864a96d428785895e5463e5728f P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:21:05.358708 21017 raft_consensus.cc:493] T 829b4ad8c5a54eababeea1181b960d4b P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:21:05.331203 20991 raft_consensus.cc:515] T 746aabf856ab401c9ef5e859371e7b67 P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:05.347088 20882 raft_consensus.cc:515] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:05.359344 20799 raft_consensus.cc:515] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:05.402884 16266 leader_election.cc:304] T 31969392ccdb43b491911a319e8bc7e1 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
I20260709 13:21:05.418773 16824 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49896 (request call id 1211) took 1886 ms. Trace:
W20260709 13:21:05.419457 21018 log.cc:927] Time spent T 8158bbd17b994d2eb96f560bd0f1b88d P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 0.052s	user 0.001s	sys 0.000s
I20260709 13:21:05.421602 16824 rpcz_store.cc:276] 0709 13:21:03.532788 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:03.537628 (+  4840us) service_pool.cc:224] Handling call
0709 13:21:03.537984 (+   356us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:21:03.560906 (+ 22922us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:21:03.560955 (+    49us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:21:03.569365 (+  8410us) log.cc:844] Serialized 26 byte log entry
0709 13:21:04.897543 (+1328178us) spinlock_profiling.cc:243] Waited 1.45 ms on lock 0x7b180038b8a0. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b693ade1 00007f69b69475b1 00007f69ada54e07 00007f69b68afab9 00007f69ada7169b 00007f69ada6f93f 00007f69ada71c02 00007f69b06f920b 00007f69b06fed23 00007f69b06ff0d1 00007f69b0b1aa0f
0709 13:21:05.283651 (+386108us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:21:05.307217 (+ 23566us) log.cc:844] Serialized 14 byte log entry
0709 13:21:05.307582 (+   365us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:05.307846 (+   264us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:21:05.418421 (+110575us) raft_consensus.cc:1685] finished
0709 13:21:05.418519 (+    98us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:21:05.418702 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":28662,"prepare.queue_time_us":23141,"prepare.run_cpu_time_us":57,"prepare.run_wall_time_us":2476,"raft.queue_time_us":24354,"raft.run_cpu_time_us":362,"raft.run_wall_time_us":16877,"spinlock_wait_cycles":3331968,"thread_start_us":385873,"threads_started":1,"wal-append.queue_time_us":1777657}
I20260709 13:21:05.411867 20641 raft_consensus.cc:2749] T 3884f864a96d428785895e5463e5728f P cb82cc7bd6db46d980c75237a1402860 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:21:05.424801 20895 leader_election.cc:290] T 84f2da98864b4425bf7c0055951fdd4b P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 10 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:21:05.397178 20992 raft_consensus.cc:2804] T 8158bbd17b994d2eb96f560bd0f1b88d P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:21:05.425869 20992 raft_consensus.cc:493] T 8158bbd17b994d2eb96f560bd0f1b88d P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Starting leader election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:21:05.426276 20992 raft_consensus.cc:3060] T 8158bbd17b994d2eb96f560bd0f1b88d P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:21:05.420011 21019 log.cc:927] Time spent T 829b4ad8c5a54eababeea1181b960d4b P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 13:21:05.395329 20890 raft_consensus.cc:2749] T 74ca8140f35c4ae29e8a664f6bdfc829 P cb82cc7bd6db46d980c75237a1402860 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:21:05.393213 20892 raft_consensus.cc:2749] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:21:05.369688 20836 raft_consensus.cc:515] T 18ad7e3af5504571a69d10c00750a01c P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } }
I20260709 13:21:05.432147 20836 leader_election.cc:290] T 18ad7e3af5504571a69d10c00750a01c P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:05.432768 20836 raft_consensus.cc:2804] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:21:05.433105 20836 raft_consensus.cc:493] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:05.433419 20836 raft_consensus.cc:3060] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:21:05.433743 20895 consensus_queue.cc:1048] T 9e65676b21b748728ad39668cfe722ec P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:21:05.419831 21014 raft_consensus.cc:493] T 57a744f5aaf944678a63469cbf514015 P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:21:05.416895 20803 raft_consensus.cc:493] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:21:05.472712 16820 raft_consensus.cc:2468] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 13.
I20260709 13:21:05.474792 16266 leader_election.cc:304] T ecc1e363230043fe9fef390f0b979f20 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:21:05.476475 20832 raft_consensus.cc:2804] T ecc1e363230043fe9fef390f0b979f20 P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Leader election won for term 13
W20260709 13:21:05.476790 21010 log.cc:927] Time spent T d16dfad0c3b540e5b4b86492ddf87208 P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 0.113s	user 0.000s	sys 0.000s
I20260709 13:21:05.459196 21014 raft_consensus.cc:515] T 57a744f5aaf944678a63469cbf514015 P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:05.490247 21014 leader_election.cc:290] T 57a744f5aaf944678a63469cbf514015 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:05.393785 20919 consensus_queue.cc:237] T fffab26427e24d398e2695aad40ca134 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:21:05.501610 20965 log.cc:927] Time spent T 19baa7bc367844c288bca8f81439c413 P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 1.432s	user 0.000s	sys 0.003s
I20260709 13:21:05.386523 20959 consensus_queue.cc:237] T c4f65cdf00b748bb963ee6e81ef43b79 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.9, Last appended by leader: 9, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:05.504381 16223 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57600 (request call id 1570) took 1739 ms. Trace:
I20260709 13:21:05.445618 20947 raft_consensus.cc:2749] T 31969392ccdb43b491911a319e8bc7e1 P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:21:05.504745 16223 rpcz_store.cc:276] 0709 13:21:03.765246 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:03.777087 (+ 11841us) service_pool.cc:224] Handling call
0709 13:21:03.826700 (+ 49613us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:21:03.894927 (+ 68227us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:21:03.894971 (+    44us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:21:03.895694 (+   723us) log.cc:844] Serialized 52 byte log entry
0709 13:21:03.985192 (+ 89498us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:21:03.985243 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:04.338689 (+353446us) spinlock_profiling.cc:243] Waited 67.8 ms on lock 0x7b5400246b54. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:21:04.338755 (+    66us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:21:05.500883 (+1162128us) spinlock_profiling.cc:243] Waited 5.68 ms on lock 0x7b4400022b4c. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69ad8fadb1 00007f69ad998c85 00007f69ad9996a2 00007f69b070b1db 00007f69b0706aaf 00007f69b06fa86e 00007f69b06f9bb6 00007f69b06f97d8 00007f69b0703221 00007f69b07031d9 00007f69b0703169
0709 13:21:05.504114 (+  3231us) raft_consensus.cc:1685] finished
0709 13:21:05.504134 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:21:05.504300 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":18771,"raft.run_cpu_time_us":46,"raft.run_wall_time_us":46,"spinlock_wait_cycles":168937600,"thread_start_us":89043,"threads_started":1,"wal-append.queue_time_us":117088}
I20260709 13:21:05.523703 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f2da98864b4425bf7c0055951fdd4b" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:21:05.528419 16820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57a744f5aaf944678a63469cbf514015" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:05.414673 21017 raft_consensus.cc:515] T 829b4ad8c5a54eababeea1181b960d4b P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:05.535746 20973 leader_election.cc:290] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
W20260709 13:21:05.540102 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 152792us
W20260709 13:21:05.540946 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 177086us
I20260709 13:21:05.542110 16116 leader_election.cc:304] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
I20260709 13:21:05.543458 21033 raft_consensus.cc:2804] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:21:05.546283 20919 raft_consensus.cc:493] T 8158bbd17b994d2eb96f560bd0f1b88d P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:05.547076 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f5f89186204242bf377f7c42b2e2da" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:21:05.547948 16217 raft_consensus.cc:3060] T c2f5f89186204242bf377f7c42b2e2da P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:21:05.547403 20919 raft_consensus.cc:515] T 8158bbd17b994d2eb96f560bd0f1b88d P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:05.551113 20919 leader_election.cc:290] T 8158bbd17b994d2eb96f560bd0f1b88d P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:21:05.508940 20803 raft_consensus.cc:515] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:05.562590 16217 raft_consensus.cc:2468] T c2f5f89186204242bf377f7c42b2e2da P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 14.
I20260709 13:21:05.568637 16116 leader_election.cc:304] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
I20260709 13:21:05.569844 21000 raft_consensus.cc:2804] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:21:05.570482 21000 raft_consensus.cc:697] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b [term 14 LEADER]: Becoming Leader. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:21:05.572082 16821 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49892 (request call id 699) took 1483 ms. Trace:
I20260709 13:21:05.571594 21000 consensus_queue.cc:237] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:05.572672 16821 rpcz_store.cc:276] 0709 13:21:04.088479 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:04.088605 (+   126us) service_pool.cc:224] Handling call
0709 13:21:05.571827 (+1483222us) spinlock_profiling.cc:243] Waited 694 ms on lock 0x7b5400787154. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:21:05.572005 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1596687744}
I20260709 13:21:05.576872 16266 leader_election.cc:400] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 14 election: Vote denied by peer cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923) with higher term. Message: Invalid argument: T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 for earlier term 14. Current term is 15.
I20260709 13:21:05.577232 16266 leader_election.cc:403] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 14 election: Cancelling election due to peer responding with higher term
I20260709 13:21:05.578379 16821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87147731a68400e8cce9b879daa7c5d" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:21:05.579221 16821 raft_consensus.cc:3060] T b87147731a68400e8cce9b879daa7c5d P cb82cc7bd6db46d980c75237a1402860 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:21:05.543993 21033 raft_consensus.cc:493] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Starting leader election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:21:05.544412 20998 leader_election.cc:290] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
W20260709 13:21:05.586237 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 45138.3us
I20260709 13:21:05.587946 16191 leader_election.cc:304] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
I20260709 13:21:05.578270 20964 leader_election.cc:290] T b87147731a68400e8cce9b879daa7c5d P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 15 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:05.579419 20832 raft_consensus.cc:697] T ecc1e363230043fe9fef390f0b979f20 P 90f3da57796b488c9087a2f523adf045 [term 13 LEADER]: Becoming Leader. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:21:05.582600 21033 raft_consensus.cc:3060] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:21:05.595372 20894 raft_consensus.cc:3060] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:21:05.607156 16142 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0a9de347c711477e87c582d003f36925"
dest_uuid: "647a343a542d4a139fe106c19095129b"
mode: GRACEFUL
new_leader_uuid: "90f3da57796b488c9087a2f523adf045"
 from {username='slave'} at 127.0.0.1:40290
W20260709 13:21:05.622215 20987 log.cc:927] Time spent T 3a7ca61592e848ea9b0fb81a2ce33cdc P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 0.918s	user 0.000s	sys 0.002s
I20260709 13:21:05.624004 20832 consensus_queue.cc:237] T ecc1e363230043fe9fef390f0b979f20 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.8, Last appended by leader: 8, Current term: 13, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:05.628651 21046 raft_consensus.cc:493] T 7c2095a5f4ad49408dbc7f69c606120d P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:21:05.629534 21046 raft_consensus.cc:515] T 7c2095a5f4ad49408dbc7f69c606120d P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:05.608451 20991 leader_election.cc:290] T 746aabf856ab401c9ef5e859371e7b67 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:05.631402 16142 raft_consensus.cc:606] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b [term 14 FOLLOWER]: Received request to transfer leadership to TS 90f3da57796b488c9087a2f523adf045
I20260709 13:21:05.632040 16142 raft_consensus.cc:614] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b [term 14 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:21:05.633828 20803 leader_election.cc:290] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:05.601207 21032 consensus_queue.cc:1048] T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.034s
I20260709 13:21:05.605476 20911 leader_election.cc:290] T e4dc6b0541ca431ebc757beac00521f1 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:21:05.636279 20992 raft_consensus.cc:515] T 8158bbd17b994d2eb96f560bd0f1b88d P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:05.638727 20836 raft_consensus.cc:515] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:05.641004 20836 leader_election.cc:290] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 15 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:05.609328 20882 leader_election.cc:290] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:05.603216 20998 raft_consensus.cc:2749] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:21:05.642055 20992 leader_election.cc:290] T 8158bbd17b994d2eb96f560bd0f1b88d P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 12 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:05.651885 21017 leader_election.cc:290] T 829b4ad8c5a54eababeea1181b960d4b P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:05.653095 16219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806d7690c2b644959152074bb2cab0a7" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:05.650523 16821 raft_consensus.cc:2468] T b87147731a68400e8cce9b879daa7c5d P cb82cc7bd6db46d980c75237a1402860 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 15.
I20260709 13:21:05.655704 16266 leader_election.cc:304] T b87147731a68400e8cce9b879daa7c5d P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:21:05.659518 20851 leader_election.cc:290] T 8c11074579b046d99e585c2818e18a2c P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 13 election: Requested vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:05.660748 16223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4dc6b0541ca431ebc757beac00521f1" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 13 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:05.661644 16223 raft_consensus.cc:2410] T e4dc6b0541ca431ebc757beac00521f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 for term 13 because replica has last-logged OpId of term: 12 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 5.
I20260709 13:21:05.662559 16219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8158bbd17b994d2eb96f560bd0f1b88d" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:05.663236 16219 raft_consensus.cc:2468] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 12.
I20260709 13:21:05.664024 20594 raft_consensus.cc:2804] T b87147731a68400e8cce9b879daa7c5d P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:21:05.664534 20594 raft_consensus.cc:697] T b87147731a68400e8cce9b879daa7c5d P 90f3da57796b488c9087a2f523adf045 [term 15 LEADER]: Becoming Leader. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:21:05.665241 16300 raft_consensus.cc:1240] T 84f2da98864b4425bf7c0055951fdd4b P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 9. Current term is 10. Ops: []
I20260709 13:21:05.665359 20594 consensus_queue.cc:237] T b87147731a68400e8cce9b879daa7c5d P 90f3da57796b488c9087a2f523adf045 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.10, Last appended by leader: 10, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:05.666411 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68a6838d82ef4cac83507f906139d90f" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:05.666994 16298 raft_consensus.cc:2468] T 68a6838d82ef4cac83507f906139d90f P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 13.
I20260709 13:21:05.670009 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746aabf856ab401c9ef5e859371e7b67" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:05.670626 16217 raft_consensus.cc:2468] T 746aabf856ab401c9ef5e859371e7b67 P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 11.
I20260709 13:21:05.671657 16295 raft_consensus.cc:1240] T c9944d5e4b8c4109af9955c4f08c9095 P 90f3da57796b488c9087a2f523adf045 [term 14 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 13. Current term is 14. Ops: []
I20260709 13:21:05.677145 16300 raft_consensus.cc:1275] T cdbd78acb4cf46298e0de8ba0d130e19 P 90f3da57796b488c9087a2f523adf045 [term 14 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:21:05.678237 16144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806d7690c2b644959152074bb2cab0a7" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:05.679736 16295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a959dbecc2648eb85d1cc550f5a3245" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:05.680656 16295 raft_consensus.cc:2468] T 5a959dbecc2648eb85d1cc550f5a3245 P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 11.
I20260709 13:21:05.683708 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ad5dbfa99ee4b8d9e28be92b3f36db5" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:05.683962 16144 raft_consensus.cc:2393] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 in current term 12: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
I20260709 13:21:05.684114 16151 raft_consensus.cc:1275] T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b [term 14 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 13:21:05.683732 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87147731a68400e8cce9b879daa7c5d" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
W20260709 13:21:05.687623 16190 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:05.684345 16297 raft_consensus.cc:2468] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 11.
I20260709 13:21:05.691140 16142 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1df93779209748b9860f6d893dd3de53"
dest_uuid: "647a343a542d4a139fe106c19095129b"
mode: GRACEFUL
new_leader_uuid: "90f3da57796b488c9087a2f523adf045"
 from {username='slave'} at 127.0.0.1:40290
I20260709 13:21:05.701285 16142 raft_consensus.cc:606] T 1df93779209748b9860f6d893dd3de53 P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Received request to transfer leadership to TS 90f3da57796b488c9087a2f523adf045
I20260709 13:21:05.701794 16142 raft_consensus.cc:614] T 1df93779209748b9860f6d893dd3de53 P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:21:05.698589 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87ac0d1f7e8a419f9bcbc04aee0e067e" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:05.699522 16148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a9de347c711477e87c582d003f36925" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:21:05.702996 16148 raft_consensus.cc:3060] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:21:05.704018 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:05.705955 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c11074579b046d99e585c2818e18a2c" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:21:05.709159 16295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18ad7e3af5504571a69d10c00750a01c" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
W20260709 13:21:05.708096 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 119366us
I20260709 13:21:05.722764 16295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8638ee363273426db1393dd81e5c7b72" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:05.723558 16295 raft_consensus.cc:2468] T 8638ee363273426db1393dd81e5c7b72 P 90f3da57796b488c9087a2f523adf045 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 14.
I20260709 13:21:05.725936 16149 raft_consensus.cc:2468] T b87147731a68400e8cce9b879daa7c5d P 647a343a542d4a139fe106c19095129b [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 14.
I20260709 13:21:05.692921 20851 raft_consensus.cc:493] T e4dc6b0541ca431ebc757beac00521f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:21:05.696411 20890 raft_consensus.cc:493] T 2794af5b99984efebdee564035841cb0 P cb82cc7bd6db46d980c75237a1402860 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:05.721521 16790 leader_election.cc:304] T 66f75124215f4ca6bef831e174a792d8 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac
I20260709 13:21:05.730340 16190 leader_election.cc:304] T 02838c86531d4f98bf58d1c22b8baf77 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:21:05.698318 20923 raft_consensus.cc:493] T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:05.705900 20470 raft_consensus.cc:493] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:05.739385 16190 leader_election.cc:304] T 5a96373bb0e14dce8a266b2fc6c58507 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:21:05.711189 20641 raft_consensus.cc:493] T 829b4ad8c5a54eababeea1181b960d4b P cb82cc7bd6db46d980c75237a1402860 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:05.729476 16191 leader_election.cc:304] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
I20260709 13:21:05.734040 16295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4f65cdf00b748bb963ee6e81ef43b79" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:21:05.750413 16191 leader_election.cc:400] T ecc1e363230043fe9fef390f0b979f20 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 12 pre-election: Vote denied by peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741) with higher term. Message: Invalid argument: T ecc1e363230043fe9fef390f0b979f20 P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac for earlier term 12. Current term is 13.
I20260709 13:21:05.750764 16191 leader_election.cc:403] T ecc1e363230043fe9fef390f0b979f20 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 12 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:21:05.740988 21032 consensus_queue.cc:1059] T 66f75124215f4ca6bef831e174a792d8 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.025s
I20260709 13:21:05.744426 20923 raft_consensus.cc:515] T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:05.724740 20900 raft_consensus.cc:493] T 3a7ca61592e848ea9b0fb81a2ce33cdc P cb82cc7bd6db46d980c75237a1402860 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:21:05.771075 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6f07842d3aa47858df5665874c86f3a" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:05.770013 16190 leader_election.cc:304] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860
I20260709 13:21:05.741005 20803 raft_consensus.cc:2749] T 02838c86531d4f98bf58d1c22b8baf77 P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:21:05.750741 20641 raft_consensus.cc:515] T 829b4ad8c5a54eababeea1181b960d4b P cb82cc7bd6db46d980c75237a1402860 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:05.799937 16149 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "62fe3346cd3c42c4a55e14af9073bca4"
dest_uuid: "647a343a542d4a139fe106c19095129b"
mode: GRACEFUL
new_leader_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
 from {username='slave'} at 127.0.0.1:40290
I20260709 13:21:05.800813 16149 raft_consensus.cc:606] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [term 16 LEADER]: Received request to transfer leadership to TS 1ca2725d290e47d68b1a4b609c9816ac
I20260709 13:21:05.800983 20641 leader_election.cc:290] T 829b4ad8c5a54eababeea1181b960d4b P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:21:05.801709 20641 raft_consensus.cc:2804] T 66f75124215f4ca6bef831e174a792d8 P cb82cc7bd6db46d980c75237a1402860 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:21:05.802249 20641 raft_consensus.cc:697] T 66f75124215f4ca6bef831e174a792d8 P cb82cc7bd6db46d980c75237a1402860 [term 12 LEADER]: Becoming Leader. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:21:05.803416 20696 raft_consensus.cc:3060] T d2ab0ac8347641e6955e9bce7d90c1a2 P cb82cc7bd6db46d980c75237a1402860 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:21:05.724340 20998 consensus_queue.cc:1059] T 8a878999c0024d2abe517484180290fc P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.044s
I20260709 13:21:05.810273 16224 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57600 (request call id 1572) took 1795 ms. Trace:
I20260709 13:21:05.812574 20236 leader_election.cc:290] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 14 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:05.739548 20977 leader_election.cc:290] T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:21:05.771955 21037 raft_consensus.cc:2749] T 5a96373bb0e14dce8a266b2fc6c58507 P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:21:05.815868 20663 consensus_queue.cc:1048] T 02838c86531d4f98bf58d1c22b8baf77 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:21:05.739248 20851 raft_consensus.cc:515] T e4dc6b0541ca431ebc757beac00521f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:05.819386 20851 leader_election.cc:290] T e4dc6b0541ca431ebc757beac00521f1 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:05.825618 16265 leader_election.cc:304] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 647a343a542d4a139fe106c19095129b
W20260709 13:21:05.826452 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 76160.3us
I20260709 13:21:05.810691 16224 rpcz_store.cc:276] 0709 13:21:04.014975 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:04.015099 (+   124us) service_pool.cc:224] Handling call
0709 13:21:04.018580 (+  3481us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:21:04.839872 (+821292us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:05.810017 (+970145us) spinlock_profiling.cc:243] Waited 820 ms on lock 0x7b5400367754. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:21:05.810209 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1886962560}
I20260709 13:21:05.727277 20985 consensus_queue.cc:1048] T ee9ed67006144797b1de6ea8c2013a68 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.094s
I20260709 13:21:05.832108 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87ac0d1f7e8a419f9bcbc04aee0e067e" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:05.830204 16265 leader_election.cc:304] T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:21:05.858565 16817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746aabf856ab401c9ef5e859371e7b67" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:05.830905 16148 raft_consensus.cc:2468] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 15.
W20260709 13:21:05.830754 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 60826.9us
I20260709 13:21:05.859387 16151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee9ed67006144797b1de6ea8c2013a68" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:21:05.853242 20236 consensus_queue.cc:1048] T 2794af5b99984efebdee564035841cb0 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.005s
I20260709 13:21:05.793325 20822 raft_consensus.cc:2804] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:21:05.833352 16142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d4ab870a05347868956aa8b841b2981" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:05.867363 20822 raft_consensus.cc:493] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:05.867863 20822 raft_consensus.cc:3060] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:21:05.868502 16142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4f65cdf00b748bb963ee6e81ef43b79" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:21:05.869321 16142 raft_consensus.cc:3060] T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:21:05.870832 20886 leader_election.cc:290] T 68a6838d82ef4cac83507f906139d90f P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:05.863369 16190 leader_election.cc:304] T 19baa7bc367844c288bca8f81439c413 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:21:05.871868 20967 raft_consensus.cc:2749] T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:21:05.872643 20886 raft_consensus.cc:2749] T 19baa7bc367844c288bca8f81439c413 P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:21:05.874619 20886 consensus_queue.cc:1048] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:21:05.864599 20998 raft_consensus.cc:3055] T 8a878999c0024d2abe517484180290fc P 1ca2725d290e47d68b1a4b609c9816ac [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:21:05.876173 20998 raft_consensus.cc:740] T 8a878999c0024d2abe517484180290fc P 1ca2725d290e47d68b1a4b609c9816ac [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:21:05.865406 21032 raft_consensus.cc:3055] T 66f75124215f4ca6bef831e174a792d8 P 1ca2725d290e47d68b1a4b609c9816ac [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:21:05.877156 21032 raft_consensus.cc:740] T 66f75124215f4ca6bef831e174a792d8 P 1ca2725d290e47d68b1a4b609c9816ac [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:21:05.878129 21032 consensus_queue.cc:260] T 66f75124215f4ca6bef831e174a792d8 P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:05.878839 16144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4dc6b0541ca431ebc757beac00521f1" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:05.881940 16117 leader_election.cc:304] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac
I20260709 13:21:05.883864 16818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5d59407eb545b895d8d922e0dd69fd" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 15 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:05.883958 16820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c11074579b046d99e585c2818e18a2c" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:21:05.884719 16820 raft_consensus.cc:3060] T 8c11074579b046d99e585c2818e18a2c P cb82cc7bd6db46d980c75237a1402860 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:21:05.885004 20472 leader_election.cc:290] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:05.885514 16816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a378d7f796f94615b3b347119a9880fd" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 15 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:05.886348 16816 raft_consensus.cc:2410] T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac for term 15 because replica has last-logged OpId of term: 13 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 9.
I20260709 13:21:05.887482 16822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:05.887964 20754 raft_consensus.cc:2804] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:21:05.888200 16822 raft_consensus.cc:2368] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac for earlier term 12. Current term is 13.
I20260709 13:21:05.888473 20754 raft_consensus.cc:697] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [term 15 LEADER]: Becoming Leader. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:21:05.889449 20754 consensus_queue.cc:237] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.12, Last appended by leader: 12, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:05.797652 20900 raft_consensus.cc:515] T 3a7ca61592e848ea9b0fb81a2ce33cdc P cb82cc7bd6db46d980c75237a1402860 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:05.891327 20959 raft_consensus.cc:493] T d16dfad0c3b540e5b4b86492ddf87208 P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:21:05.891920 20959 raft_consensus.cc:515] T d16dfad0c3b540e5b4b86492ddf87208 P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:05.892881 20900 leader_election.cc:290] T 3a7ca61592e848ea9b0fb81a2ce33cdc P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:05.893189 16819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "829b4ad8c5a54eababeea1181b960d4b" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:05.893853 16819 raft_consensus.cc:2468] T 829b4ad8c5a54eababeea1181b960d4b P cb82cc7bd6db46d980c75237a1402860 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 12.
I20260709 13:21:05.896653 20911 raft_consensus.cc:493] T 5a96373bb0e14dce8a266b2fc6c58507 P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:21:05.897213 20911 raft_consensus.cc:515] T 5a96373bb0e14dce8a266b2fc6c58507 P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:05.897753 16816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8638ee363273426db1393dd81e5c7b72" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:05.898458 20977 raft_consensus.cc:493] T 8a878999c0024d2abe517484180290fc P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:05.898479 16816 raft_consensus.cc:2393] T 8638ee363273426db1393dd81e5c7b72 P cb82cc7bd6db46d980c75237a1402860 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 14: Already voted for candidate cb82cc7bd6db46d980c75237a1402860 in this term.
I20260709 13:21:05.893934 20959 leader_election.cc:290] T d16dfad0c3b540e5b4b86492ddf87208 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:21:05.899011 20977 raft_consensus.cc:515] T 8a878999c0024d2abe517484180290fc P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:05.900098 16292 raft_consensus.cc:1275] T 2794af5b99984efebdee564035841cb0 P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:21:05.900482 16144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c74490c40243a88e62d1760b5cd814" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:05.884790 16818 raft_consensus.cc:2393] T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 15: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
I20260709 13:21:05.901618 20977 leader_election.cc:290] T 8a878999c0024d2abe517484180290fc P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:21:05.902081 20923 leader_election.cc:290] T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:05.901638 16144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6f07842d3aa47858df5665874c86f3a" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:05.902654 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f5f89186204242bf377f7c42b2e2da" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:05.904027 21046 leader_election.cc:290] T 7c2095a5f4ad49408dbc7f69c606120d P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:05.904280 16293 raft_consensus.cc:1240] T 26354361e9b044cea1c2b86c4882fe92 P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 10. Current term is 11. Ops: []
I20260709 13:21:05.904866 20799 leader_election.cc:290] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:21:05.905663 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8158bbd17b994d2eb96f560bd0f1b88d" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:05.906361 16293 raft_consensus.cc:2393] T 8158bbd17b994d2eb96f560bd0f1b88d P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 12: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
W20260709 13:21:05.906987 20980 log.cc:927] Time spent T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 1.327s	user 0.002s	sys 0.000s
W20260709 13:21:05.907760 21050 log.cc:927] Time spent T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 13:21:05.907919 16146 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40844 (request call id 2450) took 1216 ms. Trace:
I20260709 13:21:05.742304 20890 raft_consensus.cc:515] T 2794af5b99984efebdee564035841cb0 P cb82cc7bd6db46d980c75237a1402860 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:05.910063 16292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4f65cdf00b748bb963ee6e81ef43b79" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 15 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:05.924774 16816 raft_consensus.cc:1275] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P cb82cc7bd6db46d980c75237a1402860 [term 11 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:21:05.910173 16190 leader_election.cc:304] T 68a6838d82ef4cac83507f906139d90f P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 647a343a542d4a139fe106c19095129b
I20260709 13:21:05.864375 16143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18ad7e3af5504571a69d10c00750a01c" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:05.881999 16294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a378d7f796f94615b3b347119a9880fd" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 15 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
W20260709 13:21:05.867519 20988 log.cc:927] Time spent T 02838c86531d4f98bf58d1c22b8baf77 P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 1.163s	user 0.002s	sys 0.002s
I20260709 13:21:05.904069 16144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f2da98864b4425bf7c0055951fdd4b" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:21:05.914207 16142 raft_consensus.cc:2468] T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 15.
I20260709 13:21:05.911149 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87ac0d1f7e8a419f9bcbc04aee0e067e" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:05.748980 20470 raft_consensus.cc:515] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:05.929893 16220 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57600 (request call id 1574) took 1851 ms. Trace:
I20260709 13:21:05.929988 16192 leader_election.cc:304] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 647a343a542d4a139fe106c19095129b
I20260709 13:21:05.938668 16220 rpcz_store.cc:276] 0709 13:21:04.078281 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:04.078418 (+   137us) service_pool.cc:224] Handling call
0709 13:21:04.080024 (+  1606us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:21:04.087203 (+  7179us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:21:04.087235 (+    32us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:21:04.117041 (+ 29806us) log.cc:844] Serialized 52 byte log entry
0709 13:21:04.418914 (+301873us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:21:04.418950 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:04.418995 (+    45us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:21:05.906470 (+1487475us) spinlock_profiling.cc:243] Waited 11 ms on lock 0x7b4400022b4c. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69ad8fadb1 00007f69ad998c85 00007f69ad9996a2 00007f69b070b1db 00007f69b0706aaf 00007f69b06fa86e 00007f69b06f9bb6 00007f69b06f97d8 00007f69b0703221 00007f69b07031d9 00007f69b0703169
0709 13:21:05.910532 (+  4062us) raft_consensus.cc:1685] finished
0709 13:21:05.910555 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:21:05.929832 (+ 19277us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":17926,"raft.run_cpu_time_us":136,"raft.run_wall_time_us":5175,"spinlock_wait_cycles":25337600,"thread_start_us":203652,"threads_started":1,"wal-append.queue_time_us":341694}
I20260709 13:21:05.937748 16148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "829b4ad8c5a54eababeea1181b960d4b" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:05.860780 20661 raft_consensus.cc:2804] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:21:05.958777 16294 raft_consensus.cc:1275] T 434bff3082d141838068f323bdb0d1ed P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:21:05.862556 20962 raft_consensus.cc:2749] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:21:05.908319 16146 rpcz_store.cc:276] 0709 13:21:04.691628 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:04.700628 (+  9000us) service_pool.cc:224] Handling call
0709 13:21:04.700963 (+   335us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:21:04.757361 (+ 56398us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:21:05.026367 (+269006us) log.cc:844] Serialized 14 byte log entry
0709 13:21:05.160782 (+134415us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:21:05.160854 (+    72us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:21:05.160919 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:05.907700 (+746781us) spinlock_profiling.cc:243] Waited 56.1 ms on lock 0x7b54000e9f54. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:21:05.907754 (+    54us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:21:05.907867 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":57278,"prepare.queue_time_us":128889,"prepare.run_cpu_time_us":103,"prepare.run_wall_time_us":6959,"raft.queue_time_us":993,"raft.run_cpu_time_us":136,"raft.run_wall_time_us":4223,"spinlock_wait_cycles":129120640,"thread_start_us":402569,"threads_started":2,"wal-append.queue_time_us":309279}
I20260709 13:21:05.963580 16146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4dc6b0541ca431ebc757beac00521f1" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 13 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:05.865320 20836 consensus_queue.cc:1059] T 8a878999c0024d2abe517484180290fc P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.089s
I20260709 13:21:05.970052 16294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f5f89186204242bf377f7c42b2e2da" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:21:05.829499 21037 raft_consensus.cc:2749] T ecc1e363230043fe9fef390f0b979f20 P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Leader pre-election lost for term 12. Reason: Vote denied by peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741) with higher term. Message: Invalid argument: T ecc1e363230043fe9fef390f0b979f20 P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac for earlier term 12. Current term is 13.
W20260709 13:21:05.981601 16192 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 51709.2us
I20260709 13:21:05.980840 16143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57a744f5aaf944678a63469cbf514015" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
W20260709 13:21:05.982069 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 151942us
I20260709 13:21:05.982471 16143 raft_consensus.cc:2468] T 57a744f5aaf944678a63469cbf514015 P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 9.
I20260709 13:21:05.982654 16144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806d7690c2b644959152074bb2cab0a7" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:05.983361 16144 raft_consensus.cc:2468] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 12.
I20260709 13:21:05.979183 16146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87147731a68400e8cce9b879daa7c5d" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
W20260709 13:21:05.981201 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 259731us
I20260709 13:21:05.984149 16146 raft_consensus.cc:3060] T b87147731a68400e8cce9b879daa7c5d P 647a343a542d4a139fe106c19095129b [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:21:05.987375 16820 raft_consensus.cc:2468] T 8c11074579b046d99e585c2818e18a2c P cb82cc7bd6db46d980c75237a1402860 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 13.
I20260709 13:21:05.987978 16818 raft_consensus.cc:3060] T 806d7690c2b644959152074bb2cab0a7 P cb82cc7bd6db46d980c75237a1402860 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:21:05.987679 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 77598.5us
I20260709 13:21:05.959316 21034 raft_consensus.cc:2804] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:21:05.956305 20797 raft_consensus.cc:2804] T 68a6838d82ef4cac83507f906139d90f P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:21:05.988562 16823 raft_consensus.cc:1275] T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860 [term 14 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 14 index: 10. (term mismatch)
I20260709 13:21:06.022769 16823 pending_rounds.cc:85] T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860: Aborting all ops after (but not including) 9
I20260709 13:21:06.023087 16823 pending_rounds.cc:107] T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860: Aborting uncommitted NO_OP operation due to leader change: 13.10
I20260709 13:21:06.023377 16823 raft_consensus.cc:2889] T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:21:05.989347 16825 raft_consensus.cc:1275] T 434bff3082d141838068f323bdb0d1ed P cb82cc7bd6db46d980c75237a1402860 [term 12 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:21:05.954716 20911 leader_election.cc:290] T 5a96373bb0e14dce8a266b2fc6c58507 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:21:05.961915 20661 raft_consensus.cc:493] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:05.998024 16143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5718c623ebee4d2bb770f0f9ddb2f164" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:06.003038 16145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "829b4ad8c5a54eababeea1181b960d4b" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:05.938067 21046 consensus_queue.cc:1059] T 84f2da98864b4425bf7c0055951fdd4b P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.047s
I20260709 13:21:06.006381 16192 leader_election.cc:400] T ecc1e363230043fe9fef390f0b979f20 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 12 pre-election: Vote denied by peer cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923) with higher term. Message: Invalid argument: T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac for earlier term 12. Current term is 13.
I20260709 13:21:06.041266 16192 leader_election.cc:304] T 829b4ad8c5a54eababeea1181b960d4b P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:21:06.008949 16292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8158bbd17b994d2eb96f560bd0f1b88d" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:21:06.018052 16819 raft_consensus.cc:1240] T 8c11074579b046d99e585c2818e18a2c P cb82cc7bd6db46d980c75237a1402860 [term 13 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 12. Current term is 13. Ops: []
I20260709 13:21:05.828953 20641 consensus_queue.cc:237] T 66f75124215f4ca6bef831e174a792d8 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:05.985381 21032 raft_consensus.cc:3060] T 66f75124215f4ca6bef831e174a792d8 P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:21:05.982098 20959 raft_consensus.cc:493] T 7c2095a5f4ad49408dbc7f69c606120d P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:21:06.039439 16294 raft_consensus.cc:1275] T 8a878999c0024d2abe517484180290fc P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:21:06.048465 16292 raft_consensus.cc:2393] T 8158bbd17b994d2eb96f560bd0f1b88d P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 12: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
I20260709 13:21:06.036001 20797 raft_consensus.cc:493] T 68a6838d82ef4cac83507f906139d90f P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Starting leader election (detected failure of leader 647a343a542d4a139fe106c19095129b)
W20260709 13:21:06.067941 16191 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:06.069087 20797 raft_consensus.cc:3060] T 68a6838d82ef4cac83507f906139d90f P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:21:06.071468 16222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdbd78acb4cf46298e0de8ba0d130e19" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:06.072696 16218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e4bfb9090c8460eb14fa7e0d818e7f1" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:06.073307 16218 raft_consensus.cc:2468] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 11.
I20260709 13:21:06.074144 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806d7690c2b644959152074bb2cab0a7" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:06.075488 16226 raft_consensus.cc:1240] T d2ab0ac8347641e6955e9bce7d90c1a2 P 1ca2725d290e47d68b1a4b609c9816ac [term 13 LEADER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 12. Current term is 13. Ops: []
I20260709 13:21:06.076507 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "829b4ad8c5a54eababeea1181b960d4b" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:06.078135 16147 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40844 (request call id 2460) took 1375 ms. Trace:
I20260709 13:21:06.078485 16147 rpcz_store.cc:276] 0709 13:21:04.703221 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:04.924874 (+221653us) service_pool.cc:224] Handling call
0709 13:21:04.930959 (+  6085us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:21:04.945373 (+ 14414us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:06.077949 (+1132576us) spinlock_profiling.cc:243] Waited 5.79 ms on lock 0x7b54000e9a50. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5e8ad 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:21:06.078080 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":13315328}
I20260709 13:21:06.080813 16221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7ca61592e848ea9b0fb81a2ce33cdc" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:06.083971 16221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a378d7f796f94615b3b347119a9880fd" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:06.084508 16221 raft_consensus.cc:2468] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 14.
I20260709 13:21:06.086395 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5718c623ebee4d2bb770f0f9ddb2f164" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:06.088040 20592 raft_consensus.cc:2804] T 829b4ad8c5a54eababeea1181b960d4b P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:21:06.088414 20592 raft_consensus.cc:493] T 829b4ad8c5a54eababeea1181b960d4b P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Starting leader election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:21:06.088776 20592 raft_consensus.cc:3060] T 829b4ad8c5a54eababeea1181b960d4b P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:21:06.066684 20836 consensus_queue.cc:1059] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:21:06.094331 20836 consensus_queue.cc:1243] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Peer 90f3da57796b488c9087a2f523adf045 log is divergent from this leader: its last log entry 12.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 13:21:06.095675 16790 leader_election.cc:304] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:21:06.096498 20985 raft_consensus.cc:2804] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P cb82cc7bd6db46d980c75237a1402860 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:21:06.096908 20985 raft_consensus.cc:493] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P cb82cc7bd6db46d980c75237a1402860 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:06.097249 20985 raft_consensus.cc:3060] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P cb82cc7bd6db46d980c75237a1402860 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:21:06.099915 20797 raft_consensus.cc:515] T 68a6838d82ef4cac83507f906139d90f P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:06.100965 16790 leader_election.cc:304] T 806d7690c2b644959152074bb2cab0a7 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:21:06.102797 20797 leader_election.cc:290] T 68a6838d82ef4cac83507f906139d90f P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 14 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:06.107430 16790 leader_election.cc:304] T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:21:06.108269 20696 raft_consensus.cc:2804] T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:21:06.108659 20696 raft_consensus.cc:493] T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:06.108982 20696 raft_consensus.cc:3060] T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:21:06.100034 16147 raft_consensus.cc:1275] T 9e65676b21b748728ad39668cfe722ec P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:21:06.122655 16191 leader_election.cc:304] T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:21:06.022378 21034 raft_consensus.cc:493] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Starting leader election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:21:05.992506 20902 consensus_queue.cc:1048] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.056s
I20260709 13:21:06.142454 16143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8158bbd17b994d2eb96f560bd0f1b88d" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:06.040046 20661 raft_consensus.cc:3060] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:21:05.996630 21048 consensus_queue.cc:1048] T af57996a8be84d398f0f023e8d08ef77 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.137s
I20260709 13:21:06.150871 16291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ad5dbfa99ee4b8d9e28be92b3f36db5" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:06.154232 16291 raft_consensus.cc:2468] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 11.
I20260709 13:21:06.152099 16294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdbd78acb4cf46298e0de8ba0d130e19" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:06.155215 16294 raft_consensus.cc:2468] T cdbd78acb4cf46298e0de8ba0d130e19 P 90f3da57796b488c9087a2f523adf045 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 14.
I20260709 13:21:06.143484 21034 raft_consensus.cc:3060] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:21:06.182897 16294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87147731a68400e8cce9b879daa7c5d" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
W20260709 13:21:06.188937 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 166882us
W20260709 13:21:06.189481 16791 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:21:06.192909 16192 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 151724us
I20260709 13:21:06.121819 20959 raft_consensus.cc:515] T 7c2095a5f4ad49408dbc7f69c606120d P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:21:06.195230 21033 consensus_meta.cc:287] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 647a343a542d4a139fe106c19095129b: Time spent flushing consensus metadata: real 0.586s	user 0.014s	sys 0.003s
I20260709 13:21:06.204448 16294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a9de347c711477e87c582d003f36925" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:21:06.202389 16192 leader_election.cc:304] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
I20260709 13:21:06.148556 20696 raft_consensus.cc:515] T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:06.214735 16291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e4bfb9090c8460eb14fa7e0d818e7f1" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:06.216025 16295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806d7690c2b644959152074bb2cab0a7" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:06.216818 16295 raft_consensus.cc:2468] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 12.
I20260709 13:21:06.217904 16148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2794af5b99984efebdee564035841cb0" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
W20260709 13:21:06.216964 20894 consensus_meta.cc:287] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045: Time spent flushing consensus metadata: real 0.600s	user 0.007s	sys 0.012s
I20260709 13:21:06.212038 21033 raft_consensus.cc:515] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 647a343a542d4a139fe106c19095129b [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:21:06.276945 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 81342.4us
I20260709 13:21:06.291785 16190 leader_election.cc:304] T 5a959dbecc2648eb85d1cc550f5a3245 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 647a343a542d4a139fe106c19095129b
I20260709 13:21:06.240855 20894 raft_consensus.cc:2749] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Leader election lost for term 14. Reason: Vote denied by peer cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923) with higher term. Message: Invalid argument: T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 for earlier term 14. Current term is 15.
I20260709 13:21:06.243770 20470 leader_election.cc:290] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:06.239998 20890 leader_election.cc:290] T 2794af5b99984efebdee564035841cb0 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
W20260709 13:21:06.309355 21065 log.cc:927] Time spent T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 0.071s	user 0.000s	sys 0.000s
I20260709 13:21:06.312781 16149 raft_consensus.cc:2393] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 647a343a542d4a139fe106c19095129b [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 14: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
I20260709 13:21:06.316239 16190 leader_election.cc:304] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: cb82cc7bd6db46d980c75237a1402860
I20260709 13:21:06.317163 16291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7ca61592e848ea9b0fb81a2ce33cdc" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:06.323593 16295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a378d7f796f94615b3b347119a9880fd" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:06.337497 16789 leader_election.cc:304] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac
I20260709 13:21:06.341566 16190 leader_election.cc:304] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860
W20260709 13:21:06.335988 21063 log.cc:927] Time spent T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 0.113s	user 0.001s	sys 0.000s
W20260709 13:21:06.342875 16192 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 140597us
I20260709 13:21:06.336483 20470 raft_consensus.cc:2749] T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:21:06.352105 16192 leader_election.cc:304] T 8c11074579b046d99e585c2818e18a2c P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 90f3da57796b488c9087a2f523adf045
I20260709 13:21:06.346182 16225 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57604 (request call id 2704) took 1028 ms. Trace:
I20260709 13:21:06.358614 16220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8158bbd17b994d2eb96f560bd0f1b88d" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:21:06.360728 16146 raft_consensus.cc:2468] T b87147731a68400e8cce9b879daa7c5d P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 15.
W20260709 13:21:06.352788 16190 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:06.356532 16225 rpcz_store.cc:276] 0709 13:21:05.318038 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:05.362909 (+ 44871us) service_pool.cc:224] Handling call
0709 13:21:05.369379 (+  6470us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:21:05.563065 (+193686us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:21:05.563370 (+   305us) log.cc:844] Serialized 14 byte log entry
0709 13:21:05.608682 (+ 45312us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:21:05.608754 (+    72us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:21:05.608821 (+    67us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:06.345946 (+737125us) spinlock_profiling.cc:243] Waited 193 ms on lock 0x7b54001419d4. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:21:06.345992 (+    46us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:21:06.346127 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3089,"prepare.queue_time_us":112,"prepare.run_wall_time_us":35,"raft.queue_time_us":11551,"raft.run_cpu_time_us":146,"raft.run_wall_time_us":148,"spinlock_wait_cycles":449188352,"thread_start_us":39731,"threads_started":1,"wal-append.queue_time_us":77489}
W20260709 13:21:06.410022 16192 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 63480.2us
I20260709 13:21:06.411489 16821 raft_consensus.cc:1240] T 66f75124215f4ca6bef831e174a792d8 P cb82cc7bd6db46d980c75237a1402860 [term 12 LEADER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 11. Current term is 12. Ops: []
I20260709 13:21:06.345392 21060 raft_consensus.cc:2804] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:21:06.415038 16823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e4bfb9090c8460eb14fa7e0d818e7f1" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:06.337149 20592 raft_consensus.cc:515] T 829b4ad8c5a54eababeea1181b960d4b P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:06.424708 16294 raft_consensus.cc:2468] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 15.
I20260709 13:21:06.442618 16818 raft_consensus.cc:1275] T 806d7690c2b644959152074bb2cab0a7 P cb82cc7bd6db46d980c75237a1402860 [term 12 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 12 index: 9. (term mismatch)
I20260709 13:21:06.443408 16818 pending_rounds.cc:85] T 806d7690c2b644959152074bb2cab0a7 P cb82cc7bd6db46d980c75237a1402860: Aborting all ops after (but not including) 8
I20260709 13:21:06.460659 16824 raft_consensus.cc:1275] T d2ab0ac8347641e6955e9bce7d90c1a2 P cb82cc7bd6db46d980c75237a1402860 [term 13 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 13:21:06.460709 16818 pending_rounds.cc:107] T 806d7690c2b644959152074bb2cab0a7 P cb82cc7bd6db46d980c75237a1402860: Aborting uncommitted CHANGE_CONFIG_OP operation due to leader change: 10.9
I20260709 13:21:06.461318 16818 raft_consensus.cc:2917] T 806d7690c2b644959152074bb2cab0a7 P cb82cc7bd6db46d980c75237a1402860 [term 12 FOLLOWER]: Aborting config change with OpId 10.9: Aborted: Op aborted by new leader
W20260709 13:21:06.421329 21064 log.cc:927] Time spent T 31969392ccdb43b491911a319e8bc7e1 P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.179s	user 0.002s	sys 0.006s
I20260709 13:21:06.461819 16818 raft_consensus.cc:2889] T 806d7690c2b644959152074bb2cab0a7 P cb82cc7bd6db46d980c75237a1402860 [term 12 LEARNER]: CHANGE_CONFIG_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:21:06.463768 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ad5dbfa99ee4b8d9e28be92b3f36db5" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:06.417245 20641 raft_consensus.cc:493] T af57996a8be84d398f0f023e8d08ef77 P cb82cc7bd6db46d980c75237a1402860 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:06.478916 20641 raft_consensus.cc:515] T af57996a8be84d398f0f023e8d08ef77 P cb82cc7bd6db46d980c75237a1402860 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:21:06.461915 21073 log.cc:927] Time spent T e4dc6b0541ca431ebc757beac00521f1 P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.225s	user 0.000s	sys 0.002s
I20260709 13:21:06.424325 21060 raft_consensus.cc:493] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:06.470202 20959 leader_election.cc:290] T 7c2095a5f4ad49408dbc7f69c606120d P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
W20260709 13:21:06.487290 21071 log.cc:927] Time spent T 746aabf856ab401c9ef5e859371e7b67 P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 0.142s	user 0.001s	sys 0.000s
I20260709 13:21:06.488870 20641 leader_election.cc:290] T af57996a8be84d398f0f023e8d08ef77 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
W20260709 13:21:06.490257 20822 consensus_meta.cc:287] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 1ca2725d290e47d68b1a4b609c9816ac: Time spent flushing consensus metadata: real 0.622s	user 0.008s	sys 0.001s
I20260709 13:21:06.462595 20470 raft_consensus.cc:2804] T 5a959dbecc2648eb85d1cc550f5a3245 P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:21:06.491453 20470 raft_consensus.cc:493] T 5a959dbecc2648eb85d1cc550f5a3245 P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:06.491991 20470 raft_consensus.cc:3060] T 5a959dbecc2648eb85d1cc550f5a3245 P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:21:06.460091 16220 raft_consensus.cc:2468] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 12.
I20260709 13:21:06.473891 20923 raft_consensus.cc:2749] T 806d7690c2b644959152074bb2cab0a7 P cb82cc7bd6db46d980c75237a1402860 [term 12 LEARNER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:21:06.462481 21079 raft_consensus.cc:3060] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:21:06.480585 20696 leader_election.cc:290] T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 15 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:06.500053 21033 leader_election.cc:290] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 14 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:06.415822 20985 raft_consensus.cc:515] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P cb82cc7bd6db46d980c75237a1402860 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:06.414403 21068 consensus_queue.cc:1048] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 8, Time since last communication: 0.106s
I20260709 13:21:06.490772 20822 raft_consensus.cc:515] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:06.493992 21060 raft_consensus.cc:3060] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:21:06.508426 20962 raft_consensus.cc:2749] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:21:06.505872 21077 raft_consensus.cc:2804] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:21:06.523489 16823 raft_consensus.cc:2393] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P cb82cc7bd6db46d980c75237a1402860 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 12: Already voted for candidate cb82cc7bd6db46d980c75237a1402860 in this term.
I20260709 13:21:06.524992 16224 raft_consensus.cc:2393] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 12: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
I20260709 13:21:06.554889 20592 leader_election.cc:290] T 829b4ad8c5a54eababeea1181b960d4b P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 13 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:06.566059 16116 leader_election.cc:304] T 746aabf856ab401c9ef5e859371e7b67 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
W20260709 13:21:06.567704 21067 log.cc:927] Time spent T 829b4ad8c5a54eababeea1181b960d4b P cb82cc7bd6db46d980c75237a1402860: Append to log took a long time: real 0.084s	user 0.000s	sys 0.001s
I20260709 13:21:06.577908 16220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87ac0d1f7e8a419f9bcbc04aee0e067e" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:21:06.578840 16220 raft_consensus.cc:3060] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:21:06.527580 21077 raft_consensus.cc:493] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Starting leader election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:21:06.581382 21077 raft_consensus.cc:3060] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:21:06.583541 20985 leader_election.cc:290] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 12 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
W20260709 13:21:06.571949 16191 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:21:06.523612 21056 log.cc:927] Time spent T ecc1e363230043fe9fef390f0b979f20 P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.281s	user 0.003s	sys 0.000s
I20260709 13:21:06.526634 20822 leader_election.cc:290] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 12 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:06.522523 21081 raft_consensus.cc:2804] T 8c11074579b046d99e585c2818e18a2c P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:21:06.606352 16116 leader_election.cc:304] T 8158bbd17b994d2eb96f560bd0f1b88d P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
I20260709 13:21:06.587296 20754 raft_consensus.cc:493] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 647a343a542d4a139fe106c19095129b [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:21:06.588364 21007 raft_consensus.cc:493] T d16dfad0c3b540e5b4b86492ddf87208 P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:21:06.596706 20889 raft_consensus.cc:2804] T 746aabf856ab401c9ef5e859371e7b67 P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 13:21:06.608206 21032 consensus_meta.cc:287] T 66f75124215f4ca6bef831e174a792d8 P 1ca2725d290e47d68b1a4b609c9816ac: Time spent flushing consensus metadata: real 0.545s	user 0.005s	sys 0.005s
I20260709 13:21:06.623245 16825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "829b4ad8c5a54eababeea1181b960d4b" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:21:06.623572 16220 raft_consensus.cc:2468] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 14.
I20260709 13:21:06.603224 20985 raft_consensus.cc:493] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:21:06.611327 20472 raft_consensus.cc:2804] T 8158bbd17b994d2eb96f560bd0f1b88d P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:21:06.624692 16116 leader_election.cc:304] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
I20260709 13:21:06.613013 21081 raft_consensus.cc:697] T 8c11074579b046d99e585c2818e18a2c P 1ca2725d290e47d68b1a4b609c9816ac [term 13 LEADER]: Becoming Leader. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:21:06.618993 20889 raft_consensus.cc:493] T 746aabf856ab401c9ef5e859371e7b67 P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Starting leader election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:21:06.658995 21009 raft_consensus.cc:2804] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 647a343a542d4a139fe106c19095129b [term 14 FOLLOWER]: Leader election won for term 14
W20260709 13:21:06.678856 21083 log.cc:927] Time spent T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 0.104s	user 0.000s	sys 0.001s
I20260709 13:21:06.635694 20985 raft_consensus.cc:515] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:06.679396 21009 raft_consensus.cc:697] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 647a343a542d4a139fe106c19095129b [term 14 LEADER]: Becoming Leader. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:21:06.621778 21007 raft_consensus.cc:515] T d16dfad0c3b540e5b4b86492ddf87208 P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:06.621593 20754 raft_consensus.cc:515] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 647a343a542d4a139fe106c19095129b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:06.627138 20890 raft_consensus.cc:493] T 434bff3082d141838068f323bdb0d1ed P cb82cc7bd6db46d980c75237a1402860 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:06.739709 20236 raft_consensus.cc:493] T 5a959dbecc2648eb85d1cc550f5a3245 P 647a343a542d4a139fe106c19095129b [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:21:06.795924 20236 raft_consensus.cc:515] T 5a959dbecc2648eb85d1cc550f5a3245 P 647a343a542d4a139fe106c19095129b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:06.664520 20472 raft_consensus.cc:697] T 8158bbd17b994d2eb96f560bd0f1b88d P 647a343a542d4a139fe106c19095129b [term 12 LEADER]: Becoming Leader. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
W20260709 13:21:06.717374 21065 log.cc:927] Time spent T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 0.051s	user 0.001s	sys 0.000s
W20260709 13:21:06.811715 16190 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:06.810794 20236 leader_election.cc:290] T 5a959dbecc2648eb85d1cc550f5a3245 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:06.632087 20822 consensus_queue.cc:1048] T d2ab0ac8347641e6955e9bce7d90c1a2 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.174s
I20260709 13:21:06.628157 20902 raft_consensus.cc:493] T c9944d5e4b8c4109af9955c4f08c9095 P cb82cc7bd6db46d980c75237a1402860 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:21:06.709605 21033 raft_consensus.cc:493] T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:21:06.681043 20889 raft_consensus.cc:3060] T 746aabf856ab401c9ef5e859371e7b67 P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:21:06.729703 20991 raft_consensus.cc:493] T c0eab25ce4c64e018a041beeda2867db P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:21:06.709517 20231 raft_consensus.cc:493] T 08c74490c40243a88e62d1760b5cd814 P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:21:06.687847 16191 leader_election.cc:304] T 18ad7e3af5504571a69d10c00750a01c P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:21:06.805377 21090 raft_consensus.cc:493] T fffab26427e24d398e2695aad40ca134 P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:06.683912 21009 consensus_queue.cc:237] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:06.636301 21032 consensus_queue.cc:1048] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.099s
I20260709 13:21:06.705792 16789 leader_election.cc:304] T 3a7ca61592e848ea9b0fb81a2ce33cdc P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:21:06.697450 16217 raft_consensus.cc:1240] T 829b4ad8c5a54eababeea1181b960d4b P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 12. Current term is 13. Ops: []
W20260709 13:21:06.711176 16791 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:06.818408 20936 raft_consensus.cc:493] T 26354361e9b044cea1c2b86c4882fe92 P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:06.678928 20851 raft_consensus.cc:3055] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:21:06.804111 20964 raft_consensus.cc:493] T 434bff3082d141838068f323bdb0d1ed P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:21:06.680615 21078 log.cc:927] Time spent T 02838c86531d4f98bf58d1c22b8baf77 P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 13:21:06.845487 16218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a959dbecc2648eb85d1cc550f5a3245" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
W20260709 13:21:06.832269 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 144584us
W20260709 13:21:06.847360 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 863983us
I20260709 13:21:06.848557 16191 leader_election.cc:304] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 90f3da57796b488c9087a2f523adf045
I20260709 13:21:06.850965 16265 leader_election.cc:304] T c9944d5e4b8c4109af9955c4f08c9095 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860
I20260709 13:21:06.833549 16220 raft_consensus.cc:1240] T 68a6838d82ef4cac83507f906139d90f P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 13. Current term is 14. Ops: []
I20260709 13:21:06.809660 21039 raft_consensus.cc:493] T ada5a004ba7c46819051e3ccec71ffdb P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:06.708724 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87ac0d1f7e8a419f9bcbc04aee0e067e" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
W20260709 13:21:06.836987 16789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 131285us
I20260709 13:21:06.855737 16789 leader_election.cc:304] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:21:06.819380 21048 raft_consensus.cc:493] T 8cc959fb7f704fba9009c0595467d074 P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:21:06.733997 20998 consensus_queue.cc:260] T 8a878999c0024d2abe517484180290fc P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:06.825618 21017 raft_consensus.cc:493] T 1df93779209748b9860f6d893dd3de53 P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:21:06.798264 20890 raft_consensus.cc:515] T 434bff3082d141838068f323bdb0d1ed P cb82cc7bd6db46d980c75237a1402860 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:06.855518 20236 consensus_queue.cc:1059] T 829b4ad8c5a54eababeea1181b960d4b P 647a343a542d4a139fe106c19095129b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:21:06.835884 21092 raft_consensus.cc:2749] T 18ad7e3af5504571a69d10c00750a01c P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:21:06.907859 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4dc6b0541ca431ebc757beac00521f1" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:06.831760 21033 raft_consensus.cc:515] T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:06.840014 20851 raft_consensus.cc:740] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:21:06.859153 20961 raft_consensus.cc:2804] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:21:06.829216 20902 raft_consensus.cc:515] T c9944d5e4b8c4109af9955c4f08c9095 P cb82cc7bd6db46d980c75237a1402860 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:06.684149 21081 consensus_queue.cc:237] T 8c11074579b046d99e585c2818e18a2c P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.11, Last appended by leader: 11, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:06.927492 16291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e4bfb9090c8460eb14fa7e0d818e7f1" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:06.929468 16291 raft_consensus.cc:1275] T d2ab0ac8347641e6955e9bce7d90c1a2 P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 13:21:06.909711 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68a6838d82ef4cac83507f906139d90f" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
W20260709 13:21:06.930105 20661 consensus_meta.cc:287] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045: Time spent flushing consensus metadata: real 0.759s	user 0.006s	sys 0.003s
I20260709 13:21:06.930512 20661 raft_consensus.cc:515] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:06.931361 16292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ad5dbfa99ee4b8d9e28be92b3f36db5" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:21:06.932056 16292 raft_consensus.cc:3060] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:21:06.934571 20902 leader_election.cc:290] T c9944d5e4b8c4109af9955c4f08c9095 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:06.937306 20851 consensus_queue.cc:260] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:06.938529 20851 raft_consensus.cc:3060] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:21:06.838120 20991 raft_consensus.cc:515] T c0eab25ce4c64e018a041beeda2867db P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:06.837224 21090 raft_consensus.cc:515] T fffab26427e24d398e2695aad40ca134 P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:06.837791 20231 raft_consensus.cc:515] T 08c74490c40243a88e62d1760b5cd814 P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:06.910876 20594 raft_consensus.cc:493] T 3884f864a96d428785895e5463e5728f P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:21:06.940062 20594 raft_consensus.cc:515] T 3884f864a96d428785895e5463e5728f P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:06.940749 21090 leader_election.cc:290] T fffab26427e24d398e2695aad40ca134 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:06.940855 20991 leader_election.cc:290] T c0eab25ce4c64e018a041beeda2867db P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:06.941557 20991 consensus_queue.cc:1059] T 68a6838d82ef4cac83507f906139d90f P 647a343a542d4a139fe106c19095129b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: INVALID_TERM, Last received: 13.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:21:06.942394 20594 leader_election.cc:290] T 3884f864a96d428785895e5463e5728f P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:06.853331 20964 raft_consensus.cc:515] T 434bff3082d141838068f323bdb0d1ed P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:06.863523 20472 consensus_queue.cc:237] T 8158bbd17b994d2eb96f560bd0f1b88d P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:06.937594 21077 raft_consensus.cc:515] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:06.870735 20985 leader_election.cc:290] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:21:06.860340 20696 raft_consensus.cc:2804] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:21:06.842293 20936 raft_consensus.cc:515] T 26354361e9b044cea1c2b86c4882fe92 P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:06.943550 16220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c74490c40243a88e62d1760b5cd814" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:06.980273 16220 raft_consensus.cc:2393] T 08c74490c40243a88e62d1760b5cd814 P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 12: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
I20260709 13:21:06.943964 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0eab25ce4c64e018a041beeda2867db" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:06.944666 21033 leader_election.cc:290] T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
W20260709 13:21:06.908128 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 58585.4us
I20260709 13:21:06.996093 16265 leader_election.cc:304] T 68a6838d82ef4cac83507f906139d90f P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:21:06.924659 21076 raft_consensus.cc:493] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:21:06.926327 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c2095a5f4ad49408dbc7f69c606120d" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:06.858268 20894 consensus_queue.cc:1048] T a378d7f796f94615b3b347119a9880fd P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.058s
I20260709 13:21:06.963994 16222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b7d1ab519044eda2d5e04a07242f25" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:06.860417 21039 raft_consensus.cc:515] T ada5a004ba7c46819051e3ccec71ffdb P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:06.906833 20977 raft_consensus.cc:2749] T c9944d5e4b8c4109af9955c4f08c9095 P 90f3da57796b488c9087a2f523adf045 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:21:06.860891 21074 consensus_queue.cc:1048] T af57996a8be84d398f0f023e8d08ef77 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.068s
I20260709 13:21:06.975023 16292 raft_consensus.cc:2468] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 12.
I20260709 13:21:06.906772 21014 raft_consensus.cc:493] T 2794af5b99984efebdee564035841cb0 P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:07.047483 21097 raft_consensus.cc:3055] T 68a6838d82ef4cac83507f906139d90f P 647a343a542d4a139fe106c19095129b [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:21:07.047995 21097 raft_consensus.cc:740] T 68a6838d82ef4cac83507f906139d90f P 647a343a542d4a139fe106c19095129b [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:21:07.048879 21097 consensus_queue.cc:260] T 68a6838d82ef4cac83507f906139d90f P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:07.049641 20964 leader_election.cc:290] T 434bff3082d141838068f323bdb0d1ed P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:07.050128 21097 raft_consensus.cc:3060] T 68a6838d82ef4cac83507f906139d90f P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:21:07.051549 20890 leader_election.cc:290] T 434bff3082d141838068f323bdb0d1ed P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:06.973572 21090 raft_consensus.cc:493] T e5ad4eb281564936a59f41e387a47156 P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:21:06.983937 16300 raft_consensus.cc:2393] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 14: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
I20260709 13:21:06.965737 16297 raft_consensus.cc:1240] T 84f2da98864b4425bf7c0055951fdd4b P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 9. Current term is 10. Ops: []
I20260709 13:21:06.999517 16190 leader_election.cc:304] T e4dc6b0541ca431ebc757beac00521f1 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:21:06.940773 20231 leader_election.cc:290] T 08c74490c40243a88e62d1760b5cd814 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:06.911533 21017 raft_consensus.cc:515] T 1df93779209748b9860f6d893dd3de53 P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:06.964165 20961 raft_consensus.cc:697] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac [term 15 LEADER]: Becoming Leader. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:21:07.031044 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fffab26427e24d398e2695aad40ca134" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:06.974802 20991 raft_consensus.cc:493] T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:06.862694 21048 raft_consensus.cc:515] T 8cc959fb7f704fba9009c0595467d074 P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:07.032646 16265 leader_election.cc:304] T 8d4ab870a05347868956aa8b841b2981 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 647a343a542d4a139fe106c19095129b
I20260709 13:21:07.064970 16265 leader_election.cc:304] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:21:07.030905 21089 raft_consensus.cc:493] T 74ca8140f35c4ae29e8a664f6bdfc829 P cb82cc7bd6db46d980c75237a1402860 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:21:07.033418 21096 raft_consensus.cc:2749] T 68a6838d82ef4cac83507f906139d90f P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:21:06.926999 20470 raft_consensus.cc:515] T 5a959dbecc2648eb85d1cc550f5a3245 P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:06.980791 16217 raft_consensus.cc:2468] T c0eab25ce4c64e018a041beeda2867db P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 12.
W20260709 13:21:07.001960 21034 consensus_meta.cc:287] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac: Time spent flushing consensus metadata: real 0.805s	user 0.007s	sys 0.005s
W20260709 13:21:07.060289 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 60871us
I20260709 13:21:07.060310 16292 raft_consensus.cc:1240] T c9944d5e4b8c4109af9955c4f08c9095 P 90f3da57796b488c9087a2f523adf045 [term 14 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 13. Current term is 14. Ops: [13.11-13.11]
I20260709 13:21:07.000890 20696 raft_consensus.cc:697] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860 [term 15 LEADER]: Becoming Leader. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:21:06.822518 20918 raft_consensus.cc:2955] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045 [term 14 LEADER]: Committing config change with OpId 7.7: config changed from index 3 to 7, cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:21:07.107942 21039 leader_election.cc:290] T ada5a004ba7c46819051e3ccec71ffdb P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
W20260709 13:21:07.113458 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 45550.1us
I20260709 13:21:07.117184 16265 leader_election.cc:304] T 8158bbd17b994d2eb96f560bd0f1b88d P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:21:07.118924 20894 raft_consensus.cc:2804] T 8158bbd17b994d2eb96f560bd0f1b88d P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:21:07.032768 21076 raft_consensus.cc:515] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:07.121589 21076 leader_election.cc:290] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:07.121888 16219 raft_consensus.cc:1240] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 13. Current term is 14. Ops: []
I20260709 13:21:07.122413 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d16dfad0c3b540e5b4b86492ddf87208" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:07.123191 16223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a878999c0024d2abe517484180290fc" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:07.124256 20977 consensus_queue.cc:1059] T 8c11074579b046d99e585c2818e18a2c P 90f3da57796b488c9087a2f523adf045 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:21:07.124632 16224 raft_consensus.cc:1275] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:21:07.126533 20977 raft_consensus.cc:3055] T 8c11074579b046d99e585c2818e18a2c P 90f3da57796b488c9087a2f523adf045 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:21:07.000365 21032 consensus_queue.cc:1048] T d2ab0ac8347641e6955e9bce7d90c1a2 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 9, Time since last communication: 0.098s
I20260709 13:21:07.127069 20977 raft_consensus.cc:740] T 8c11074579b046d99e585c2818e18a2c P 90f3da57796b488c9087a2f523adf045 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:21:07.128337 20977 consensus_queue.cc:260] T 8c11074579b046d99e585c2818e18a2c P 90f3da57796b488c9087a2f523adf045 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.12, Last appended by leader: 12, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:07.129681 20977 raft_consensus.cc:3060] T 8c11074579b046d99e585c2818e18a2c P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:21:07.119408 20894 raft_consensus.cc:493] T 8158bbd17b994d2eb96f560bd0f1b88d P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:07.059937 20890 raft_consensus.cc:493] T 23f82224f4564fea93572e8206bad1fe P cb82cc7bd6db46d980c75237a1402860 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:07.155494 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a96373bb0e14dce8a266b2fc6c58507" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:07.132050 20661 leader_election.cc:290] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 14 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:21:07.063033 20231 raft_consensus.cc:493] T b6f07842d3aa47858df5665874c86f3a P 647a343a542d4a139fe106c19095129b [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
W20260709 13:21:07.150003 16191 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:07.062623 20822 raft_consensus.cc:2749] T e4dc6b0541ca431ebc757beac00521f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:21:07.048018 21014 raft_consensus.cc:515] T 2794af5b99984efebdee564035841cb0 P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:07.099530 20911 raft_consensus.cc:2804] T 8d4ab870a05347868956aa8b841b2981 P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:21:07.172654 16218 raft_consensus.cc:2368] T 5a959dbecc2648eb85d1cc550f5a3245 P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b for earlier term 11. Current term is 12.
I20260709 13:21:07.133176 20931 raft_consensus.cc:2804] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:21:07.097587 20641 raft_consensus.cc:2749] T 3a7ca61592e848ea9b0fb81a2ce33cdc P cb82cc7bd6db46d980c75237a1402860 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:21:07.178316 16223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c2095a5f4ad49408dbc7f69c606120d" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:07.062256 21090 raft_consensus.cc:515] T e5ad4eb281564936a59f41e387a47156 P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:07.183776 16824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3884f864a96d428785895e5463e5728f" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:07.191783 16824 raft_consensus.cc:2468] T 3884f864a96d428785895e5463e5728f P cb82cc7bd6db46d980c75237a1402860 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 8.
I20260709 13:21:07.187325 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada5a004ba7c46819051e3ccec71ffdb" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:07.153005 20754 leader_election.cc:290] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:07.171903 20894 raft_consensus.cc:3060] T 8158bbd17b994d2eb96f560bd0f1b88d P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:21:07.101140 20964 consensus_queue.cc:1048] T 02838c86531d4f98bf58d1c22b8baf77 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.079s
I20260709 13:21:07.193022 16224 raft_consensus.cc:2468] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 13.
I20260709 13:21:07.101298 20991 raft_consensus.cc:515] T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:07.135907 21034 raft_consensus.cc:515] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:07.199790 20236 consensus_queue.cc:1059] T 26354361e9b044cea1c2b86c4882fe92 P 647a343a542d4a139fe106c19095129b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:21:07.105330 21089 raft_consensus.cc:515] T 74ca8140f35c4ae29e8a664f6bdfc829 P cb82cc7bd6db46d980c75237a1402860 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:07.199241 16115 leader_election.cc:304] T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:21:07.186956 20911 raft_consensus.cc:493] T 8d4ab870a05347868956aa8b841b2981 P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:21:07.189612 20931 raft_consensus.cc:493] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:07.208163 16145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af57996a8be84d398f0f023e8d08ef77" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:07.229687 16145 raft_consensus.cc:2468] T af57996a8be84d398f0f023e8d08ef77 P 647a343a542d4a139fe106c19095129b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 14.
I20260709 13:21:07.211445 16146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62fe3346cd3c42c4a55e14af9073bca4" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:07.212131 16148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434bff3082d141838068f323bdb0d1ed" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 13 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:07.214459 20999 raft_consensus.cc:2749] T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:21:07.222592 16296 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48674 (request call id 2796) took 1229 ms. Trace:
I20260709 13:21:07.224002 16115 leader_election.cc:304] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac
I20260709 13:21:07.233660 20782 raft_consensus.cc:493] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:21:07.234257 20782 raft_consensus.cc:515] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:07.236224 21089 leader_election.cc:290] T 74ca8140f35c4ae29e8a664f6bdfc829 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:07.237071 21077 leader_election.cc:290] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 14 election: Requested vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:07.201346 20754 raft_consensus.cc:493] T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:07.238631 20754 raft_consensus.cc:515] T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:07.230595 20931 raft_consensus.cc:3060] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:21:07.240756 20754 leader_election.cc:290] T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:07.241747 21017 leader_election.cc:290] T 1df93779209748b9860f6d893dd3de53 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:07.245903 16820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74ca8140f35c4ae29e8a664f6bdfc829" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:07.247660 21090 leader_election.cc:290] T e5ad4eb281564936a59f41e387a47156 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:07.249205 21014 leader_election.cc:290] T 2794af5b99984efebdee564035841cb0 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:07.251389 21103 consensus_queue.cc:1048] T 434bff3082d141838068f323bdb0d1ed P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
W20260709 13:21:07.254318 21102 log.cc:927] Time spent T 3a7ca61592e848ea9b0fb81a2ce33cdc P cb82cc7bd6db46d980c75237a1402860: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
I20260709 13:21:07.255247 20999 consensus_queue.cc:1048] T 8a878999c0024d2abe517484180290fc P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:21:07.256062 20470 leader_election.cc:290] T 5a959dbecc2648eb85d1cc550f5a3245 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 12 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:07.256472 16818 raft_consensus.cc:1275] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860 [term 13 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 13:21:07.257921 16818 raft_consensus.cc:1275] T b87147731a68400e8cce9b879daa7c5d P cb82cc7bd6db46d980c75237a1402860 [term 15 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 13:21:07.240010 20782 leader_election.cc:290] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:07.260869 21048 leader_election.cc:290] T 8cc959fb7f704fba9009c0595467d074 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:07.179692 20890 raft_consensus.cc:515] T 23f82224f4564fea93572e8206bad1fe P cb82cc7bd6db46d980c75237a1402860 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:07.229370 20911 raft_consensus.cc:3060] T 8d4ab870a05347868956aa8b841b2981 P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:21:07.263019 20890 leader_election.cc:290] T 23f82224f4564fea93572e8206bad1fe P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
W20260709 13:21:07.264685 16300 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48674 (request call id 2785) took 1374 ms (client timeout 1823 ms). Trace:
I20260709 13:21:07.265393 21089 raft_consensus.cc:493] T 9e65676b21b748728ad39668cfe722ec P cb82cc7bd6db46d980c75237a1402860 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:21:07.265975 21089 raft_consensus.cc:515] T 9e65676b21b748728ad39668cfe722ec P cb82cc7bd6db46d980c75237a1402860 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:07.231992 16296 rpcz_store.cc:276] 0709 13:21:05.992885 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:05.993150 (+   265us) service_pool.cc:224] Handling call
0709 13:21:05.993461 (+   311us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:21:06.220041 (+226580us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:21:06.220083 (+    42us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:21:06.220124 (+    41us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:21:06.220155 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:07.222307 (+1002152us) spinlock_profiling.cc:243] Waited 226 ms on lock 0x7b54000cdd54. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:21:07.222358 (+    51us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:21:07.222500 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":520668544}
W20260709 13:21:07.265079 16300 rpcz_store.cc:269] 0709 13:21:05.890365 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:05.902411 (+ 12046us) service_pool.cc:224] Handling call
0709 13:21:07.264472 (+1362061us) spinlock_profiling.cc:243] Waited 1.08 s on lock 0x7b5400162e54. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:21:07.264629 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2484550528}
I20260709 13:21:07.270397 16265 leader_election.cc:304] T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:21:07.180950 20231 raft_consensus.cc:515] T b6f07842d3aa47858df5665874c86f3a P 647a343a542d4a139fe106c19095129b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:07.275640 16818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8638ee363273426db1393dd81e5c7b72" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:21:07.277400 16818 raft_consensus.cc:2393] T 8638ee363273426db1393dd81e5c7b72 P cb82cc7bd6db46d980c75237a1402860 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 14: Already voted for candidate cb82cc7bd6db46d980c75237a1402860 in this term.
I20260709 13:21:07.279381 16151 raft_consensus.cc:1275] T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:21:07.279381 16225 raft_consensus.cc:1240] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 16. Current term is 17. Ops: []
I20260709 13:21:07.289561 16265 leader_election.cc:304] T 84f2da98864b4425bf7c0055951fdd4b P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:21:07.254485 16046 catalog_manager.cc:5697] T 68a6838d82ef4cac83507f906139d90f P 90f3da57796b488c9087a2f523adf045 reported cstate change: term changed from 5 to 8, leader changed from 647a343a542d4a139fe106c19095129b (127.14.177.129) to 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130). New cstate: current_term: 8 leader_uuid: "1ca2725d290e47d68b1a4b609c9816ac" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } }
W20260709 13:21:07.304210 16115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 80313.7us
I20260709 13:21:07.259697 20936 leader_election.cc:290] T 26354361e9b044cea1c2b86c4882fe92 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:07.304796 16046 catalog_manager.cc:5697] T 8c11074579b046d99e585c2818e18a2c P 90f3da57796b488c9087a2f523adf045 reported cstate change: term changed from 6 to 7, leader changed from cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) to 90f3da57796b488c9087a2f523adf045 (127.14.177.131). New cstate: current_term: 7 leader_uuid: "90f3da57796b488c9087a2f523adf045" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:21:07.279469 21096 raft_consensus.cc:2749] T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:21:07.286091 21079 consensus_meta.cc:287] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac: Time spent flushing consensus metadata: real 0.774s	user 0.007s	sys 0.004s
I20260709 13:21:07.322364 16265 leader_election.cc:304] T e4dc6b0541ca431ebc757beac00521f1 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:21:07.194393 20961 consensus_queue.cc:237] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:07.345559 16148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c2095a5f4ad49408dbc7f69c606120d" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:07.309772 20782 raft_consensus.cc:493] T 8638ee363273426db1393dd81e5c7b72 P cb82cc7bd6db46d980c75237a1402860 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:07.351125 16145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68a6838d82ef4cac83507f906139d90f" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:21:07.336680 16142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "829b4ad8c5a54eababeea1181b960d4b" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:21:07.224335 21009 consensus_queue.cc:1048] T 2794af5b99984efebdee564035841cb0 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.062s
I20260709 13:21:07.335628 16146 raft_consensus.cc:1275] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 13:21:07.312090 20754 raft_consensus.cc:2804] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:21:07.340044 20966 raft_consensus.cc:2749] T 84f2da98864b4425bf7c0055951fdd4b P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:21:07.350122 16147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fffab26427e24d398e2695aad40ca134" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:07.343973 21079 raft_consensus.cc:2749] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:21:07.343370 20890 raft_consensus.cc:493] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:21:07.379429 16046 catalog_manager.cc:5697] T fffab26427e24d398e2695aad40ca134 P 90f3da57796b488c9087a2f523adf045 reported cstate change: term changed from 5 to 6, leader changed from 647a343a542d4a139fe106c19095129b (127.14.177.129) to 90f3da57796b488c9087a2f523adf045 (127.14.177.131). New cstate: current_term: 6 leader_uuid: "90f3da57796b488c9087a2f523adf045" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } health_report { overall_health: HEALTHY } } }
W20260709 13:21:07.357851 21105 log.cc:927] Time spent T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
W20260709 13:21:07.358305 16263 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:07.391530 16296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a959dbecc2648eb85d1cc550f5a3245" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:07.392292 16296 raft_consensus.cc:2468] T 5a959dbecc2648eb85d1cc550f5a3245 P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 11.
I20260709 13:21:07.231606 20696 consensus_queue.cc:237] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:07.353185 16143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ad5dbfa99ee4b8d9e28be92b3f36db5" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:21:07.449958 21034 leader_election.cc:290] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 17 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:07.367564 20754 raft_consensus.cc:493] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:07.451606 20754 raft_consensus.cc:3060] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:21:07.393558 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e9b2bf7011f4ae688a79d7bb5a447b4" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:07.366299 16147 raft_consensus.cc:2468] T fffab26427e24d398e2695aad40ca134 P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 11.
I20260709 13:21:07.309121 20889 raft_consensus.cc:515] T 746aabf856ab401c9ef5e859371e7b67 P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:21:07.368072 21099 log.cc:927] Time spent T 8c11074579b046d99e585c2818e18a2c P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
I20260709 13:21:07.384361 16221 raft_consensus.cc:1275] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 13:21:07.401391 16150 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "746aabf856ab401c9ef5e859371e7b67"
dest_uuid: "647a343a542d4a139fe106c19095129b"
mode: GRACEFUL
new_leader_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
 from {username='slave'} at 127.0.0.1:40290
I20260709 13:21:07.404356 20231 leader_election.cc:290] T b6f07842d3aa47858df5665874c86f3a P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:07.405112 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0eab25ce4c64e018a041beeda2867db" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:07.412700 16295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c74490c40243a88e62d1760b5cd814" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:07.413321 20890 raft_consensus.cc:515] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:07.417908 21033 raft_consensus.cc:3055] T 829b4ad8c5a54eababeea1181b960d4b P 647a343a542d4a139fe106c19095129b [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:21:07.420416 20923 raft_consensus.cc:493] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:07.433338 16818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b7d1ab519044eda2d5e04a07242f25" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:07.481052 16818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62fe3346cd3c42c4a55e14af9073bca4" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:21:07.355688 21115 raft_consensus.cc:493] T 19baa7bc367844c288bca8f81439c413 P cb82cc7bd6db46d980c75237a1402860 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:21:07.440047 21089 leader_election.cc:290] T 9e65676b21b748728ad39668cfe722ec P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:07.434332 20911 raft_consensus.cc:515] T 8d4ab870a05347868956aa8b841b2981 P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:21:07.444739 21060 consensus_meta.cc:287] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860: Time spent flushing consensus metadata: real 0.922s	user 0.011s	sys 0.000s
I20260709 13:21:07.474709 16145 raft_consensus.cc:2468] T 68a6838d82ef4cac83507f906139d90f P 647a343a542d4a139fe106c19095129b [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 14.
I20260709 13:21:07.481860 16818 raft_consensus.cc:3060] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:21:07.436036 20991 leader_election.cc:290] T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:07.484174 20889 leader_election.cc:290] T 746aabf856ab401c9ef5e859371e7b67 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 12 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:07.484616 16150 raft_consensus.cc:606] T 746aabf856ab401c9ef5e859371e7b67 P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Received request to transfer leadership to TS 1ca2725d290e47d68b1a4b609c9816ac
I20260709 13:21:07.497378 16150 raft_consensus.cc:614] T 746aabf856ab401c9ef5e859371e7b67 P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:21:07.484813 20923 raft_consensus.cc:515] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:07.485805 21007 leader_election.cc:290] T d16dfad0c3b540e5b4b86492ddf87208 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
W20260709 13:21:07.488126 16115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 93143.1us
I20260709 13:21:07.496865 16046 catalog_manager.cc:5697] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 90f3da57796b488c9087a2f523adf045 reported cstate change: term changed from 6 to 7, leader changed from 647a343a542d4a139fe106c19095129b (127.14.177.129) to 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130). New cstate: current_term: 7 leader_uuid: "1ca2725d290e47d68b1a4b609c9816ac" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } }
I20260709 13:21:07.479662 21033 raft_consensus.cc:740] T 829b4ad8c5a54eababeea1181b960d4b P 647a343a542d4a139fe106c19095129b [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:21:07.362577 20782 raft_consensus.cc:515] T 8638ee363273426db1393dd81e5c7b72 P cb82cc7bd6db46d980c75237a1402860 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:07.502929 16115 leader_election.cc:304] T 5a959dbecc2648eb85d1cc550f5a3245 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:21:07.508307 16817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66f75124215f4ca6bef831e174a792d8" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:07.514575 16818 raft_consensus.cc:2468] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 17.
I20260709 13:21:07.495025 21009 raft_consensus.cc:493] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:21:07.515831 16192 leader_election.cc:304] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:21:07.499956 21092 raft_consensus.cc:493] T 66f75124215f4ca6bef831e174a792d8 P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:07.533903 16818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e9b2bf7011f4ae688a79d7bb5a447b4" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:07.535930 16818 raft_consensus.cc:2468] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 10.
I20260709 13:21:07.535360 16046 catalog_manager.cc:5697] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045 reported cstate change: term changed from 5 to 6, leader changed from 647a343a542d4a139fe106c19095129b (127.14.177.129) to 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130). New cstate: current_term: 6 leader_uuid: "1ca2725d290e47d68b1a4b609c9816ac" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } } }
I20260709 13:21:07.537739 21075 raft_consensus.cc:2804] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:21:07.538004 21009 raft_consensus.cc:515] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:07.538849 20890 leader_election.cc:290] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:07.539484 20890 raft_consensus.cc:493] T 26354361e9b044cea1c2b86c4882fe92 P cb82cc7bd6db46d980c75237a1402860 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:21:07.540037 20890 raft_consensus.cc:515] T 26354361e9b044cea1c2b86c4882fe92 P cb82cc7bd6db46d980c75237a1402860 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:07.540695 21103 raft_consensus.cc:2804] T 5a959dbecc2648eb85d1cc550f5a3245 P 647a343a542d4a139fe106c19095129b [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:21:07.541152 21103 raft_consensus.cc:493] T 5a959dbecc2648eb85d1cc550f5a3245 P 647a343a542d4a139fe106c19095129b [term 10 FOLLOWER]: Starting leader election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:21:07.511400 20991 raft_consensus.cc:493] T 18ad7e3af5504571a69d10c00750a01c P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:21:07.541543 21103 raft_consensus.cc:3060] T 5a959dbecc2648eb85d1cc550f5a3245 P 647a343a542d4a139fe106c19095129b [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:21:07.556732 16149 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40844 (request call id 2469) took 1725 ms (client timeout 2042 ms). Trace:
W20260709 13:21:07.557196 16149 rpcz_store.cc:269] 0709 13:21:05.831523 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:05.831747 (+   224us) service_pool.cc:224] Handling call
0709 13:21:07.556487 (+1724740us) spinlock_profiling.cc:243] Waited 451 ms on lock 0x7b540028e454. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:21:07.556660 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1037306112}
I20260709 13:21:07.553805 16817 raft_consensus.cc:1275] T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860 [term 15 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 13:21:07.502157 21060 raft_consensus.cc:515] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:07.650863 16819 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49906 (request call id 969) took 1237 ms. Trace:
I20260709 13:21:07.541514 21009 leader_election.cc:290] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:07.653995 16819 rpcz_store.cc:276] 0709 13:21:06.413531 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:06.413699 (+   168us) service_pool.cc:224] Handling call
0709 13:21:06.445225 (+ 31526us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:21:06.489949 (+ 44724us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:21:06.616225 (+126276us) log.cc:844] Serialized 14 byte log entry
0709 13:21:07.429355 (+813130us) spinlock_profiling.cc:243] Waited 2.67 ms on lock 0x7b1800445240. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b693ade1 00007f69b69475b1 00007f69b07074bc 00007f69ada5548b 00007f69b68afab9 00007f69ada7169b 00007f69ada6f93f 00007f69ada71c02 00007f69b06f920b 00007f69b06fed23 00007f69b06ff23a
0709 13:21:07.429534 (+   179us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:21:07.430093 (+   559us) log.cc:844] Serialized 26 byte log entry
0709 13:21:07.430250 (+   157us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:21:07.430289 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:07.650329 (+220040us) spinlock_profiling.cc:243] Waited 44.3 ms on lock 0x7b54007c01d4. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:21:07.650417 (+    88us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:21:07.650501 (+    84us) raft_consensus.cc:1685] finished
0709 13:21:07.650523 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:21:07.650753 (+   230us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":321890,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":40,"raft.queue_time_us":73601,"raft.run_cpu_time_us":194,"raft.run_wall_time_us":4369,"spinlock_wait_cycles":107982976,"thread_start_us":315261,"threads_started":2,"wal-append.queue_time_us":326175}
I20260709 13:21:07.544782 21075 raft_consensus.cc:697] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [term 17 LEADER]: Becoming Leader. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:21:07.550995 20890 leader_election.cc:290] T 26354361e9b044cea1c2b86c4882fe92 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:07.499202 21115 raft_consensus.cc:515] T 19baa7bc367844c288bca8f81439c413 P cb82cc7bd6db46d980c75237a1402860 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:07.680173 16816 raft_consensus.cc:1240] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 [term 15 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 14. Current term is 15. Ops: []
I20260709 13:21:07.651412 21048 raft_consensus.cc:493] T 746aabf856ab401c9ef5e859371e7b67 P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:21:07.664915 21112 raft_consensus.cc:493] T 9e65676b21b748728ad39668cfe722ec P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:07.684387 16149 raft_consensus.cc:1240] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 647a343a542d4a139fe106c19095129b [term 14 LEADER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 13. Current term is 14. Ops: []
I20260709 13:21:07.686923 16147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a96373bb0e14dce8a266b2fc6c58507" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:07.542729 21033 consensus_queue.cc:260] T 829b4ad8c5a54eababeea1181b960d4b P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:07.545538 21092 raft_consensus.cc:515] T 66f75124215f4ca6bef831e174a792d8 P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:07.684909 16145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d16dfad0c3b540e5b4b86492ddf87208" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:07.699604 16145 raft_consensus.cc:2468] T d16dfad0c3b540e5b4b86492ddf87208 P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 12.
I20260709 13:21:07.568915 20991 raft_consensus.cc:515] T 18ad7e3af5504571a69d10c00750a01c P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } }
I20260709 13:21:07.677467 16046 catalog_manager.cc:5697] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045 reported cstate change: config changed from index 7 to 3, term changed from 11 to 7, leader changed from <none> to 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) changed from VOTER to NON_VOTER, now has a pending config: VOTER 647a343a542d4a139fe106c19095129b (127.14.177.129), VOTER 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130), VOTER 90f3da57796b488c9087a2f523adf045 (127.14.177.131), VOTER cb82cc7bd6db46d980c75237a1402860 (127.14.177.132). New cstate: current_term: 7 leader_uuid: "1ca2725d290e47d68b1a4b609c9816ac" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } } } pending_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:21:07.721942 16217 raft_consensus.cc:1240] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac [term 15 LEADER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 14. Current term is 15. Ops: []
I20260709 13:21:07.728302 16146 raft_consensus.cc:1240] T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 14. Current term is 15. Ops: []
I20260709 13:21:07.706306 21033 raft_consensus.cc:3060] T 829b4ad8c5a54eababeea1181b960d4b P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:21:07.731384 16146 raft_consensus.cc:1275] T fffab26427e24d398e2695aad40ca134 P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:21:07.690202 21048 raft_consensus.cc:515] T 746aabf856ab401c9ef5e859371e7b67 P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:07.733497 16146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3884f864a96d428785895e5463e5728f" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:07.693692 21112 raft_consensus.cc:515] T 9e65676b21b748728ad39668cfe722ec P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:07.738015 16145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a878999c0024d2abe517484180290fc" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:07.739916 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c2095a5f4ad49408dbc7f69c606120d" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:07.741268 16146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434bff3082d141838068f323bdb0d1ed" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:07.743397 16046 catalog_manager.cc:5697] T e4dc6b0541ca431ebc757beac00521f1 P 90f3da57796b488c9087a2f523adf045 reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "1ca2725d290e47d68b1a4b609c9816ac" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } }
W20260709 13:21:07.745369 20931 consensus_meta.cc:287] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045: Time spent flushing consensus metadata: real 0.504s	user 0.012s	sys 0.000s
I20260709 13:21:07.753645 20797 raft_consensus.cc:3055] T 84f2da98864b4425bf7c0055951fdd4b P 1ca2725d290e47d68b1a4b609c9816ac [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:21:07.754093 20797 raft_consensus.cc:740] T 84f2da98864b4425bf7c0055951fdd4b P 1ca2725d290e47d68b1a4b609c9816ac [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:21:07.754379 21092 leader_election.cc:290] T 66f75124215f4ca6bef831e174a792d8 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:07.754981 20797 consensus_queue.cc:260] T 84f2da98864b4425bf7c0055951fdd4b P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:07.756160 20797 raft_consensus.cc:3060] T 84f2da98864b4425bf7c0055951fdd4b P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:21:07.760239 20964 raft_consensus.cc:2749] T e4dc6b0541ca431ebc757beac00521f1 P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:21:07.737355 20911 leader_election.cc:290] T 8d4ab870a05347868956aa8b841b2981 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 12 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:21:07.767355 21034 raft_consensus.cc:493] T ee9ed67006144797b1de6ea8c2013a68 P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:21:07.767931 21034 raft_consensus.cc:515] T ee9ed67006144797b1de6ea8c2013a68 P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:07.769945 21034 leader_election.cc:290] T ee9ed67006144797b1de6ea8c2013a68 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:07.770645 20964 raft_consensus.cc:493] T 31969392ccdb43b491911a319e8bc7e1 P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:21:07.770751 20972 raft_consensus.cc:493] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:21:07.771252 20964 raft_consensus.cc:515] T 31969392ccdb43b491911a319e8bc7e1 P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:07.771535 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d16dfad0c3b540e5b4b86492ddf87208" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:07.771445 20972 raft_consensus.cc:515] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:07.772208 20961 raft_consensus.cc:493] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:21:07.772717 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:07.773247 20964 leader_election.cc:290] T 31969392ccdb43b491911a319e8bc7e1 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:07.773727 20964 raft_consensus.cc:493] T d2ab0ac8347641e6955e9bce7d90c1a2 P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:07.773842 20972 leader_election.cc:290] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
W20260709 13:21:07.776897 20894 consensus_meta.cc:287] T 8158bbd17b994d2eb96f560bd0f1b88d P 90f3da57796b488c9087a2f523adf045: Time spent flushing consensus metadata: real 0.576s	user 0.010s	sys 0.000s
I20260709 13:21:07.777058 16297 raft_consensus.cc:1240] T 26354361e9b044cea1c2b86c4882fe92 P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 10. Current term is 11. Ops: []
I20260709 13:21:07.777318 20894 raft_consensus.cc:515] T 8158bbd17b994d2eb96f560bd0f1b88d P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:07.778470 16297 raft_consensus.cc:1275] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045 [term 14 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 13:21:07.772787 20961 raft_consensus.cc:515] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:07.779497 16046 catalog_manager.cc:5697] T 8a878999c0024d2abe517484180290fc P 90f3da57796b488c9087a2f523adf045 reported cstate change: term changed from 3 to 6, leader changed from 647a343a542d4a139fe106c19095129b (127.14.177.129) to 90f3da57796b488c9087a2f523adf045 (127.14.177.131). New cstate: current_term: 6 leader_uuid: "90f3da57796b488c9087a2f523adf045" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } health_report { overall_health: HEALTHY } } }
I20260709 13:21:07.749876 20782 leader_election.cc:290] T 8638ee363273426db1393dd81e5c7b72 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:07.784884 16294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6f07842d3aa47858df5665874c86f3a" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:07.785290 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e4bfb9090c8460eb14fa7e0d818e7f1" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:21:07.792392 16294 raft_consensus.cc:2468] T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 14.
I20260709 13:21:07.793041 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f82224f4564fea93572e8206bad1fe" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 12 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:07.793355 16294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e9b2bf7011f4ae688a79d7bb5a447b4" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:07.794390 16294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e65676b21b748728ad39668cfe722ec" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:07.797180 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74ca8140f35c4ae29e8a664f6bdfc829" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:07.799155 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8638ee363273426db1393dd81e5c7b72" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:07.799685 16294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9944d5e4b8c4109af9955c4f08c9095" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:07.799878 16297 raft_consensus.cc:2468] T 8638ee363273426db1393dd81e5c7b72 P 90f3da57796b488c9087a2f523adf045 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 14.
I20260709 13:21:07.801060 20236 raft_consensus.cc:493] T 8cc959fb7f704fba9009c0595467d074 P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:21:07.801256 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a378d7f796f94615b3b347119a9880fd" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 15 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:21:07.803156 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31969392ccdb43b491911a319e8bc7e1" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:07.803874 16819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:07.804541 16819 raft_consensus.cc:2468] T 19baa7bc367844c288bca8f81439c413 P cb82cc7bd6db46d980c75237a1402860 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 14.
I20260709 13:21:07.804641 16294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26354361e9b044cea1c2b86c4882fe92" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:07.805583 16294 raft_consensus.cc:2393] T 26354361e9b044cea1c2b86c4882fe92 P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 in current term 11: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
I20260709 13:21:07.805622 16825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746aabf856ab401c9ef5e859371e7b67" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:21:07.806325 16294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2794af5b99984efebdee564035841cb0" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:07.807861 20236 raft_consensus.cc:515] T 8cc959fb7f704fba9009c0595467d074 P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:07.731463 21075 consensus_queue.cc:237] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:07.811944 16820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2372981006d4b218215c28964dea623" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:07.816953 16294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434bff3082d141838068f323bdb0d1ed" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 13 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:07.764725 20931 raft_consensus.cc:515] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:07.864013 16295 raft_consensus.cc:2368] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b for earlier term 12. Current term is 13.
I20260709 13:21:07.864354 16823 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49906 (request call id 970) took 1449 ms. Trace:
I20260709 13:21:07.864933 20931 leader_election.cc:290] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 13 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:21:07.809372 20999 consensus_queue.cc:1048] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.015s
I20260709 13:21:07.841051 16046 catalog_manager.cc:5697] T 18ad7e3af5504571a69d10c00750a01c P 90f3da57796b488c9087a2f523adf045 reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "1ca2725d290e47d68b1a4b609c9816ac" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } }
I20260709 13:21:07.870132 16046 catalog_manager.cc:5697] T c0eab25ce4c64e018a041beeda2867db P 90f3da57796b488c9087a2f523adf045 reported cstate change: term changed from 2 to 7, leader changed from 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130) to 90f3da57796b488c9087a2f523adf045 (127.14.177.131). New cstate: current_term: 7 leader_uuid: "90f3da57796b488c9087a2f523adf045" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } health_report { overall_health: HEALTHY } } }
I20260709 13:21:07.842917 20961 leader_election.cc:290] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:07.843531 21060 leader_election.cc:290] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 15 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:07.845249 21112 leader_election.cc:290] T 9e65676b21b748728ad39668cfe722ec P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:07.846396 20894 leader_election.cc:290] T 8158bbd17b994d2eb96f560bd0f1b88d P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 13 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:21:07.852643 21048 leader_election.cc:290] T 746aabf856ab401c9ef5e859371e7b67 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:07.856978 16293 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48674 (request call id 2792) took 1917 ms. Trace:
I20260709 13:21:07.774145 20964 raft_consensus.cc:515] T d2ab0ac8347641e6955e9bce7d90c1a2 P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:07.817569 21126 consensus_queue.cc:1059] T 26354361e9b044cea1c2b86c4882fe92 P 647a343a542d4a139fe106c19095129b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.055s
I20260709 13:21:07.858461 16821 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49906 (request call id 961) took 1976 ms. Trace:
I20260709 13:21:07.913434 16821 rpcz_store.cc:276] 0709 13:21:05.882249 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:05.904675 (+ 22426us) service_pool.cc:224] Handling call
0709 13:21:05.904929 (+   254us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:21:06.412076 (+507147us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:07.858250 (+1446174us) spinlock_profiling.cc:243] Waited 501 ms on lock 0x7b54005302d4. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:21:07.858407 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1151389056}
I20260709 13:21:07.906867 16293 rpcz_store.cc:276] 0709 13:21:05.939688 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:05.958297 (+ 18609us) service_pool.cc:224] Handling call
0709 13:21:05.958566 (+   269us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:21:05.977910 (+ 19344us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:21:05.977944 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:21:05.987595 (+  9651us) log.cc:844] Serialized 26 byte log entry
0709 13:21:06.082597 (+ 95002us) spinlock_profiling.cc:243] Waited 4.59 ms on lock 0x7b4800130158. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b5a88aa1 00007f69b5a7fb37 00007f69b5a7f9ca 00007f69b5b7726f 00007f69b5b76f37 00007f69b5b8b55c 00007f69b5b8b508 00007f69b5b8b488 00007f69b5b8b443 00007f69b5b8a6ec 00007f69b4ab4a96
0709 13:21:06.236708 (+154111us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:21:06.237034 (+   326us) log.cc:844] Serialized 14 byte log entry
0709 13:21:06.704409 (+467375us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:06.704511 (+   102us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:21:07.856023 (+1151512us) spinlock_profiling.cc:243] Waited 2.29 ms on lock 0x7b240024bb58. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69ad8fadb1 00007f69ad8f89c4 00007f69ad8f6acb 00007f69adab1dc9 00007f69adab1a55 00007f69adab0fb0 00007f69b0727488 00007f69b0727be8 00007f69b06ff398 00007f69b06fa76a 00007f69b06f9bb6
0709 13:21:07.856689 (+   666us) raft_consensus.cc:1685] finished
0709 13:21:07.856724 (+    35us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:21:07.856903 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1154963,"prepare.queue_time_us":164,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":38,"raft.queue_time_us":486985,"raft.run_cpu_time_us":2962,"raft.run_wall_time_us":119110,"spinlock_wait_cycles":15834240,"thread_start_us":174532,"threads_started":1,"wal-append.queue_time_us":201097}
I20260709 13:21:07.915485 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdbd78acb4cf46298e0de8ba0d130e19" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:21:07.916579 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e65676b21b748728ad39668cfe722ec" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:07.917551 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cc959fb7f704fba9009c0595467d074" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
W20260709 13:21:07.917816 16191 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:07.918046 21103 raft_consensus.cc:515] T 5a959dbecc2648eb85d1cc550f5a3245 P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:07.918560 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18ad7e3af5504571a69d10c00750a01c" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:07.922063 21048 raft_consensus.cc:493] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:21:07.922644 21048 raft_consensus.cc:515] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:07.924901 21048 leader_election.cc:290] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:07.926307 16191 leader_election.cc:304] T 829b4ad8c5a54eababeea1181b960d4b P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
I20260709 13:21:07.927707 21034 raft_consensus.cc:2749] T 829b4ad8c5a54eababeea1181b960d4b P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:21:07.872859 20236 leader_election.cc:290] T 8cc959fb7f704fba9009c0595467d074 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:07.876565 16297 raft_consensus.cc:3055] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 90f3da57796b488c9087a2f523adf045 [term 13 LEADER]: Stepping down as leader of term 13
W20260709 13:21:07.928350 20977 consensus_meta.cc:287] T 8c11074579b046d99e585c2818e18a2c P 90f3da57796b488c9087a2f523adf045: Time spent flushing consensus metadata: real 0.756s	user 0.009s	sys 0.002s
I20260709 13:21:07.928586 16297 raft_consensus.cc:740] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 90f3da57796b488c9087a2f523adf045 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:21:07.929405 16297 consensus_queue.cc:260] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 90f3da57796b488c9087a2f523adf045 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:07.929543 20754 raft_consensus.cc:515] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:07.930299 16297 raft_consensus.cc:3060] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:21:07.930624 16143 raft_consensus.cc:2393] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 12: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
I20260709 13:21:07.884249 16115 leader_election.cc:304] T b6f07842d3aa47858df5665874c86f3a P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:21:07.887120 16789 leader_election.cc:304] T 8638ee363273426db1393dd81e5c7b72 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 
W20260709 13:21:07.931515 16115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 47380.1us
W20260709 13:21:07.931645 16789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 44618.7us
I20260709 13:21:07.931945 21046 raft_consensus.cc:1064] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac: attempting to promote NON_VOTER cb82cc7bd6db46d980c75237a1402860 to VOTER
W20260709 13:21:07.932464 21046 raft_consensus.cc:1068] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac: Unable to promote non-voter cb82cc7bd6db46d980c75237a1402860: Illegal state: Leader has not yet committed an operation in its own term
I20260709 13:21:07.933490 16143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1df93779209748b9860f6d893dd3de53" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
W20260709 13:21:07.882194 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 406360us
I20260709 13:21:07.874970 20797 raft_consensus.cc:493] T 02838c86531d4f98bf58d1c22b8baf77 P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:07.911600 20991 leader_election.cc:290] T 18ad7e3af5504571a69d10c00750a01c P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:21:07.912451 20964 leader_election.cc:290] T d2ab0ac8347641e6955e9bce7d90c1a2 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:07.864748 16823 rpcz_store.cc:276] 0709 13:21:06.414625 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:06.414787 (+   162us) service_pool.cc:224] Handling call
0709 13:21:07.864174 (+1449387us) spinlock_profiling.cc:243] Waited 104 ms on lock 0x7b54007c3b54. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:21:07.864297 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":240264960}
I20260709 13:21:07.877290 16294 raft_consensus.cc:1240] T 8158bbd17b994d2eb96f560bd0f1b88d P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 12. Current term is 13. Ops: []
I20260709 13:21:07.935245 20236 raft_consensus.cc:2804] T b6f07842d3aa47858df5665874c86f3a P 647a343a542d4a139fe106c19095129b [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:21:07.940961 16191 leader_election.cc:304] T fffab26427e24d398e2695aad40ca134 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
I20260709 13:21:07.941315 16823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66ae176961b74813a58bc15c9c4c6f2a" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:07.941108 20236 raft_consensus.cc:493] T b6f07842d3aa47858df5665874c86f3a P 647a343a542d4a139fe106c19095129b [term 14 FOLLOWER]: Starting leader election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:21:07.941967 20236 raft_consensus.cc:3060] T b6f07842d3aa47858df5665874c86f3a P 647a343a542d4a139fe106c19095129b [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:21:07.938768 16818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee9ed67006144797b1de6ea8c2013a68" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:07.944262 16143 raft_consensus.cc:2468] T 1df93779209748b9860f6d893dd3de53 P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 12.
I20260709 13:21:07.945670 16143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee9ed67006144797b1de6ea8c2013a68" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:07.947624 16191 leader_election.cc:304] T 68a6838d82ef4cac83507f906139d90f P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 90f3da57796b488c9087a2f523adf045
I20260709 13:21:07.947788 16293 raft_consensus.cc:1240] T 8d4ab870a05347868956aa8b841b2981 P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 11. Current term is 12. Ops: []
I20260709 13:21:07.952601 16147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f82224f4564fea93572e8206bad1fe" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 12 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
W20260709 13:21:07.954663 16263 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:07.939344 16142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5ad4eb281564936a59f41e387a47156" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:07.956252 16142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e9b2bf7011f4ae688a79d7bb5a447b4" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:07.957108 16142 raft_consensus.cc:2468] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 647a343a542d4a139fe106c19095129b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 10.
I20260709 13:21:07.958148 16142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e65676b21b748728ad39668cfe722ec" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:07.963125 16817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66f75124215f4ca6bef831e174a792d8" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:07.967223 16823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746aabf856ab401c9ef5e859371e7b67" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:07.953439 16147 raft_consensus.cc:2468] T 23f82224f4564fea93572e8206bad1fe P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 11.
I20260709 13:21:07.969828 16147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 14 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:07.970850 16147 raft_consensus.cc:2410] T ecc1e363230043fe9fef390f0b979f20 P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 for term 14 because replica has last-logged OpId of term: 11 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 7.
I20260709 13:21:07.972213 16151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a959dbecc2648eb85d1cc550f5a3245" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:21:07.972878 16151 raft_consensus.cc:3060] T 5a959dbecc2648eb85d1cc550f5a3245 P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:21:07.882159 20999 raft_consensus.cc:493] T ee9ed67006144797b1de6ea8c2013a68 P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:21:07.978135 16148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cc959fb7f704fba9009c0595467d074" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:07.978318 16191 leader_election.cc:304] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:21:07.978951 16145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62fe3346cd3c42c4a55e14af9073bca4" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:21:07.980070 16148 raft_consensus.cc:2468] T 8cc959fb7f704fba9009c0595467d074 P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 12.
I20260709 13:21:07.986585 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66f75124215f4ca6bef831e174a792d8" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:07.987253 16149 raft_consensus.cc:2468] T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 12.
I20260709 13:21:07.943660 20797 raft_consensus.cc:515] T 02838c86531d4f98bf58d1c22b8baf77 P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:07.962833 21057 raft_consensus.cc:2804] T 8638ee363273426db1393dd81e5c7b72 P cb82cc7bd6db46d980c75237a1402860 [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 13:21:07.986011 21128 log.cc:927] Time spent T 9c61a39fd4f84424b129c1a6f9ba404d P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 0.075s	user 0.002s	sys 0.000s
W20260709 13:21:08.012120 16263 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:08.015507 16819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b7d1ab519044eda2d5e04a07242f25" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:07.994819 16817 raft_consensus.cc:1275] T 8c11074579b046d99e585c2818e18a2c P cb82cc7bd6db46d980c75237a1402860 [term 13 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 13:21:07.971899 21068 raft_consensus.cc:2804] T fffab26427e24d398e2695aad40ca134 P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:21:07.990813 21046 raft_consensus.cc:2804] T 68a6838d82ef4cac83507f906139d90f P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:21:07.974102 21034 consensus_queue.cc:1048] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.053s
I20260709 13:21:08.030956 16147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:07.961375 16790 leader_election.cc:304] T b87147731a68400e8cce9b879daa7c5d P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860; no voters: 90f3da57796b488c9087a2f523adf045
W20260709 13:21:07.892450 16117 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.14.177.132:38923 timed out after 1.777s (SENT) [suppressed 16 similar messages]
W20260709 13:21:08.032800 16116 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:07.958850 16142 raft_consensus.cc:2468] T 9e65676b21b748728ad39668cfe722ec P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 12.
I20260709 13:21:08.046128 16147 raft_consensus.cc:2468] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 14.
I20260709 13:21:08.013756 21057 raft_consensus.cc:493] T 8638ee363273426db1393dd81e5c7b72 P cb82cc7bd6db46d980c75237a1402860 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:07.995677 20991 consensus_queue.cc:1059] T 8158bbd17b994d2eb96f560bd0f1b88d P 647a343a542d4a139fe106c19095129b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.048s
I20260709 13:21:08.027840 21048 raft_consensus.cc:2749] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:21:08.022228 21068 raft_consensus.cc:493] T fffab26427e24d398e2695aad40ca134 P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:08.052636 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ad5dbfa99ee4b8d9e28be92b3f36db5" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:21:08.053623 21060 raft_consensus.cc:2764] T b87147731a68400e8cce9b879daa7c5d P cb82cc7bd6db46d980c75237a1402860 [term 15 FOLLOWER]: Leader election decision vote started in defunct term 14: won
I20260709 13:21:07.989055 20999 raft_consensus.cc:515] T ee9ed67006144797b1de6ea8c2013a68 P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:08.059664 16299 raft_consensus.cc:2393] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 12: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
I20260709 13:21:08.046623 21034 consensus_queue.cc:1243] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Peer 647a343a542d4a139fe106c19095129b log is divergent from this leader: its last log entry 9.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 13:21:08.030645 21046 raft_consensus.cc:697] T 68a6838d82ef4cac83507f906139d90f P 1ca2725d290e47d68b1a4b609c9816ac [term 14 LEADER]: Becoming Leader. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:21:08.017127 21112 consensus_queue.cc:1059] T 8d4ab870a05347868956aa8b841b2981 P 647a343a542d4a139fe106c19095129b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.038s
W20260709 13:21:08.049773 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 88508.8us
I20260709 13:21:08.060179 16142 raft_consensus.cc:1275] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:21:08.047341 20754 leader_election.cc:290] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 12 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:08.069044 16824 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49892 (request call id 725) took 1075 ms. Trace:
I20260709 13:21:08.069466 16824 rpcz_store.cc:276] 0709 13:21:06.993486 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:07.197533 (+204047us) service_pool.cc:224] Handling call
0709 13:21:07.231325 (+ 33792us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:21:07.267748 (+ 36423us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:21:07.267782 (+    34us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:21:07.267825 (+    43us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:21:07.267858 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:08.068805 (+800947us) spinlock_profiling.cc:243] Waited 36.2 ms on lock 0x7b54005a0d54. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:21:08.068849 (+    44us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:21:08.068979 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":83243264}
I20260709 13:21:08.069904 16790 leader_election.cc:304] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:21:08.054659 21057 raft_consensus.cc:3060] T 8638ee363273426db1393dd81e5c7b72 P cb82cc7bd6db46d980c75237a1402860 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:21:08.075618 20236 raft_consensus.cc:515] T b6f07842d3aa47858df5665874c86f3a P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:21:08.079340 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 757067us
I20260709 13:21:08.066614 21126 raft_consensus.cc:3055] T 8158bbd17b994d2eb96f560bd0f1b88d P 647a343a542d4a139fe106c19095129b [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:21:08.086596 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6f07842d3aa47858df5665874c86f3a" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:21:08.087426 16299 raft_consensus.cc:3060] T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:21:08.077991 21121 raft_consensus.cc:2749] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:21:08.059263 21068 raft_consensus.cc:3060] T fffab26427e24d398e2695aad40ca134 P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:21:08.076262 16143 raft_consensus.cc:2468] T ee9ed67006144797b1de6ea8c2013a68 P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 11.
I20260709 13:21:08.092974 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a959dbecc2648eb85d1cc550f5a3245" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:21:08.094506 20236 leader_election.cc:290] T b6f07842d3aa47858df5665874c86f3a P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 15 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:08.076368 16790 leader_election.cc:304] T 829b4ad8c5a54eababeea1181b960d4b P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:21:08.099416 16790 leader_election.cc:304] T 2794af5b99984efebdee564035841cb0 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:21:08.100258 21129 raft_consensus.cc:2749] T 2794af5b99984efebdee564035841cb0 P cb82cc7bd6db46d980c75237a1402860 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:21:08.106582 16151 raft_consensus.cc:2468] T 5a959dbecc2648eb85d1cc550f5a3245 P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 12.
I20260709 13:21:08.108538 20999 leader_election.cc:290] T ee9ed67006144797b1de6ea8c2013a68 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:08.110249 20797 leader_election.cc:290] T 02838c86531d4f98bf58d1c22b8baf77 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:08.094043 21103 leader_election.cc:290] T 5a959dbecc2648eb85d1cc550f5a3245 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 11 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:08.118958 16148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31969392ccdb43b491911a319e8bc7e1" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:08.120456 16294 raft_consensus.cc:1240] T 26354361e9b044cea1c2b86c4882fe92 P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 10. Current term is 11. Ops: []
W20260709 13:21:08.121956 21114 log.cc:927] Time spent T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 0.749s	user 0.002s	sys 0.000s
W20260709 13:21:08.122602 21123 log.cc:927] Time spent T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860: Append to log took a long time: real 0.312s	user 0.000s	sys 0.001s
I20260709 13:21:08.101027 20902 raft_consensus.cc:2749] T 829b4ad8c5a54eababeea1181b960d4b P cb82cc7bd6db46d980c75237a1402860 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:21:08.125066 16297 raft_consensus.cc:1275] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 90f3da57796b488c9087a2f523adf045 [term 14 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 13:21:08.083050 21126 raft_consensus.cc:740] T 8158bbd17b994d2eb96f560bd0f1b88d P 647a343a542d4a139fe106c19095129b [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:21:08.131098 16293 raft_consensus.cc:2468] T 5a959dbecc2648eb85d1cc550f5a3245 P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 11.
I20260709 13:21:08.135486 16115 leader_election.cc:304] T 5a959dbecc2648eb85d1cc550f5a3245 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:21:08.137459 16299 raft_consensus.cc:2468] T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 15.
W20260709 13:21:08.138366 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 160140us
I20260709 13:21:08.139488 16115 leader_election.cc:304] T b6f07842d3aa47858df5665874c86f3a P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:21:08.135757 16265 leader_election.cc:304] T ada5a004ba7c46819051e3ccec71ffdb P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
W20260709 13:21:08.140311 16117 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 247895us
I20260709 13:21:08.151253 16116 leader_election.cc:304] T c0eab25ce4c64e018a041beeda2867db P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 90f3da57796b488c9087a2f523adf045
I20260709 13:21:08.151141 16117 leader_election.cc:304] T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860
I20260709 13:21:08.150465 20236 consensus_queue.cc:1048] T 434bff3082d141838068f323bdb0d1ed P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.000s
W20260709 13:21:08.140035 21136 log.cc:927] Time spent T 2794af5b99984efebdee564035841cb0 P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.056s	user 0.003s	sys 0.004s
I20260709 13:21:08.140730 16790 leader_election.cc:304] T af57996a8be84d398f0f023e8d08ef77 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860; no voters: 
W20260709 13:21:08.142946 21130 log.cc:927] Time spent T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
W20260709 13:21:08.143814 21134 log.cc:927] Time spent T d2ab0ac8347641e6955e9bce7d90c1a2 P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 0.064s	user 0.000s	sys 0.000s
I20260709 13:21:08.138091 21093 consensus_queue.cc:1048] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.007s
I20260709 13:21:08.139699 21106 raft_consensus.cc:2764] T 5a959dbecc2648eb85d1cc550f5a3245 P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Leader election decision vote started in defunct term 11: won
W20260709 13:21:08.157012 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 56462.3us
W20260709 13:21:08.159415 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 48039.1us
I20260709 13:21:08.152453 21112 raft_consensus.cc:2804] T b6f07842d3aa47858df5665874c86f3a P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Leader election won for term 15
W20260709 13:21:08.161046 16192 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 178209us
I20260709 13:21:08.163353 16265 leader_election.cc:304] T 57a744f5aaf944678a63469cbf514015 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: cb82cc7bd6db46d980c75237a1402860
I20260709 13:21:08.175868 20236 raft_consensus.cc:2749] T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:21:08.133900 21009 consensus_queue.cc:1048] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.022s
I20260709 13:21:08.176137 16117 leader_election.cc:304] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860; no voters: 90f3da57796b488c9087a2f523adf045
I20260709 13:21:08.175235 16790 leader_election.cc:304] T 434bff3082d141838068f323bdb0d1ed P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:21:08.179684 16116 leader_election.cc:400] T 5a959dbecc2648eb85d1cc550f5a3245 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 11 pre-election: Vote denied by peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943) with higher term. Message: Invalid argument: T 5a959dbecc2648eb85d1cc550f5a3245 P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b for earlier term 11. Current term is 12.
I20260709 13:21:08.176122 16191 leader_election.cc:304] T 1df93779209748b9860f6d893dd3de53 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
I20260709 13:21:08.133838 21103 consensus_queue.cc:1059] T 26354361e9b044cea1c2b86c4882fe92 P 647a343a542d4a139fe106c19095129b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.023s
I20260709 13:21:08.180521 20757 raft_consensus.cc:2804] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 647a343a542d4a139fe106c19095129b [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:21:08.182300 20894 raft_consensus.cc:2804] T 57a744f5aaf944678a63469cbf514015 P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:21:08.184368 21034 raft_consensus.cc:2804] T 1df93779209748b9860f6d893dd3de53 P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:21:08.184890 21034 raft_consensus.cc:493] T 1df93779209748b9860f6d893dd3de53 P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Starting leader election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:21:08.188196 21034 raft_consensus.cc:3060] T 1df93779209748b9860f6d893dd3de53 P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:21:08.184386 16189 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:21:08.189749 16189 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:08.184614 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f5f89186204242bf377f7c42b2e2da" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:21:08.179699 16265 leader_election.cc:304] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac
I20260709 13:21:08.180603 21121 raft_consensus.cc:2749] T 434bff3082d141838068f323bdb0d1ed P cb82cc7bd6db46d980c75237a1402860 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:21:08.193753 16790 leader_election.cc:304] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860; no voters: 90f3da57796b488c9087a2f523adf045
I20260709 13:21:08.163398 20641 raft_consensus.cc:2804] T af57996a8be84d398f0f023e8d08ef77 P cb82cc7bd6db46d980c75237a1402860 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:21:08.196478 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6f07842d3aa47858df5665874c86f3a" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:08.197769 16817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e65676b21b748728ad39668cfe722ec" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:08.198120 16222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7ca61592e848ea9b0fb81a2ce33cdc" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:08.198684 16817 raft_consensus.cc:2410] T 9e65676b21b748728ad39668cfe722ec P cb82cc7bd6db46d980c75237a1402860 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b for term 13 because replica has last-logged OpId of term: 12 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 10.
I20260709 13:21:08.197726 16823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746aabf856ab401c9ef5e859371e7b67" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:08.199185 16222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8158bbd17b994d2eb96f560bd0f1b88d" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:21:08.194914 21084 raft_consensus.cc:493] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:08.201537 21096 raft_consensus.cc:2804] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:21:08.193989 16217 raft_consensus.cc:2393] T c2f5f89186204242bf377f7c42b2e2da P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 14: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
I20260709 13:21:08.201642 21084 raft_consensus.cc:515] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:08.202762 16223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66f75124215f4ca6bef831e174a792d8" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:08.202989 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d4ab870a05347868956aa8b841b2981" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:21:08.203428 16223 raft_consensus.cc:2468] T 66f75124215f4ca6bef831e174a792d8 P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 12.
I20260709 13:21:08.203588 16217 raft_consensus.cc:3060] T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:21:08.203701 20923 leader_election.cc:290] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:08.204334 16223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746aabf856ab401c9ef5e859371e7b67" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:21:08.182548 20797 raft_consensus.cc:493] T 5a96373bb0e14dce8a266b2fc6c58507 P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:21:08.204871 16223 raft_consensus.cc:3060] T 746aabf856ab401c9ef5e859371e7b67 P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:21:08.205343 16221 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57604 (request call id 2722) took 1375 ms. Trace:
I20260709 13:21:08.205715 16221 rpcz_store.cc:276] 0709 13:21:06.829627 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:06.829795 (+   168us) service_pool.cc:224] Handling call
0709 13:21:06.830277 (+   482us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:21:07.475833 (+645556us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:08.205140 (+729307us) spinlock_profiling.cc:243] Waited 554 ms on lock 0x7b54002c8b54. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:21:08.205291 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1273439104}
I20260709 13:21:08.184582 20894 raft_consensus.cc:493] T 57a744f5aaf944678a63469cbf514015 P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Starting leader election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:21:08.206614 20894 raft_consensus.cc:3060] T 57a744f5aaf944678a63469cbf514015 P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:21:08.192698 16191 leader_election.cc:304] T 8cc959fb7f704fba9009c0595467d074 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
I20260709 13:21:08.208285 16150 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7c2095a5f4ad49408dbc7f69c606120d"
dest_uuid: "647a343a542d4a139fe106c19095129b"
mode: GRACEFUL
new_leader_uuid: "90f3da57796b488c9087a2f523adf045"
 from {username='slave'} at 127.0.0.1:40290
I20260709 13:21:08.208918 16150 raft_consensus.cc:606] T 7c2095a5f4ad49408dbc7f69c606120d P 647a343a542d4a139fe106c19095129b [term 9 LEADER]: Received request to transfer leadership to TS 90f3da57796b488c9087a2f523adf045
I20260709 13:21:08.199134 16192 leader_election.cc:304] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860
I20260709 13:21:08.202092 21096 raft_consensus.cc:493] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:21:08.211594 21096 raft_consensus.cc:3060] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:21:08.184517 20757 raft_consensus.cc:493] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 647a343a542d4a139fe106c19095129b [term 10 FOLLOWER]: Starting leader election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:21:08.212242 20757 raft_consensus.cc:3060] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 647a343a542d4a139fe106c19095129b [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:21:08.214056 21139 log.cc:927] Time spent T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 0.064s	user 0.000s	sys 0.000s
I20260709 13:21:08.214411 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 14 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:08.215705 16296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:08.205011 20797 raft_consensus.cc:515] T 5a96373bb0e14dce8a266b2fc6c58507 P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:08.218418 20797 leader_election.cc:290] T 5a96373bb0e14dce8a266b2fc6c58507 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:08.200752 21103 consensus_queue.cc:1048] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 10, Time since last communication: 0.004s
I20260709 13:21:08.200028 16222 raft_consensus.cc:3060] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:21:08.163138 21133 raft_consensus.cc:2804] T ada5a004ba7c46819051e3ccec71ffdb P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:21:08.206890 16221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c74490c40243a88e62d1760b5cd814" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:21:08.232699 16221 raft_consensus.cc:3060] T 08c74490c40243a88e62d1760b5cd814 P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:21:08.200572 16790 leader_election.cc:304] T 23f82224f4564fea93572e8206bad1fe P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860; no voters: 90f3da57796b488c9087a2f523adf045
I20260709 13:21:08.182765 21033 raft_consensus.cc:2804] T c0eab25ce4c64e018a041beeda2867db P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:21:08.223631 16191 leader_election.cc:304] T 66f75124215f4ca6bef831e174a792d8 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
I20260709 13:21:08.225157 16223 raft_consensus.cc:2468] T 746aabf856ab401c9ef5e859371e7b67 P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 12.
I20260709 13:21:08.240407 16223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2372981006d4b218215c28964dea623" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:08.241009 16223 raft_consensus.cc:2468] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 14.
I20260709 13:21:08.242743 16790 leader_election.cc:304] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:21:08.243932 16790 leader_election.cc:304] T 9e65676b21b748728ad39668cfe722ec P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860; no voters: 90f3da57796b488c9087a2f523adf045
I20260709 13:21:08.228637 16147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2794af5b99984efebdee564035841cb0" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:08.230787 16142 raft_consensus.cc:1275] T ecc1e363230043fe9fef390f0b979f20 P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 13:21:08.250263 16142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8158bbd17b994d2eb96f560bd0f1b88d" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:21:08.234707 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f5f89186204242bf377f7c42b2e2da" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:21:08.237782 16217 raft_consensus.cc:2468] T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 12.
I20260709 13:21:08.222894 21112 raft_consensus.cc:697] T b6f07842d3aa47858df5665874c86f3a P 647a343a542d4a139fe106c19095129b [term 15 LEADER]: Becoming Leader. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:21:08.256458 16147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c74490c40243a88e62d1760b5cd814" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:21:08.257258 16147 raft_consensus.cc:3060] T 08c74490c40243a88e62d1760b5cd814 P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Advancing to term 13
I20260709 13:21:08.252460 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d4ab870a05347868956aa8b841b2981" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:21:08.254194 16218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d16dfad0c3b540e5b4b86492ddf87208" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:08.253886 16265 leader_election.cc:304] T 5a96373bb0e14dce8a266b2fc6c58507 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:21:08.162173 21057 raft_consensus.cc:515] T 8638ee363273426db1393dd81e5c7b72 P cb82cc7bd6db46d980c75237a1402860 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:08.178979 20967 consensus_queue.cc:1059] T c4f65cdf00b748bb963ee6e81ef43b79 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.046s
I20260709 13:21:08.261651 16191 leader_election.cc:304] T 5a959dbecc2648eb85d1cc550f5a3245 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
I20260709 13:21:08.219612 20782 raft_consensus.cc:2804] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:21:08.140120 21046 consensus_queue.cc:237] T 68a6838d82ef4cac83507f906139d90f P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:08.192134 21101 raft_consensus.cc:493] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:21:08.193037 20902 consensus_queue.cc:1048] T 66f75124215f4ca6bef831e174a792d8 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.070s
I20260709 13:21:08.264006 16820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee9ed67006144797b1de6ea8c2013a68" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:08.267355 16148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26354361e9b044cea1c2b86c4882fe92" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:08.263819 16265 leader_election.cc:304] T d16dfad0c3b540e5b4b86492ddf87208 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac
I20260709 13:21:08.290159 16221 raft_consensus.cc:2468] T 08c74490c40243a88e62d1760b5cd814 P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 13.
I20260709 13:21:08.266673 20797 raft_consensus.cc:2804] T 66f75124215f4ca6bef831e174a792d8 P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:21:08.263206 21087 raft_consensus.cc:3060] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:21:08.268956 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31969392ccdb43b491911a319e8bc7e1" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:08.201050 20641 raft_consensus.cc:493] T af57996a8be84d398f0f023e8d08ef77 P cb82cc7bd6db46d980c75237a1402860 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:21:08.221211 21075 raft_consensus.cc:493] T c2f5f89186204242bf377f7c42b2e2da P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:08.271353 16151 raft_consensus.cc:1275] T 23f82224f4564fea93572e8206bad1fe P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:21:08.272213 16223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ab0ac8347641e6955e9bce7d90c1a2" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 14 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:08.253700 21142 raft_consensus.cc:2804] T 8cc959fb7f704fba9009c0595467d074 P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:21:08.181077 21126 consensus_queue.cc:260] T 8158bbd17b994d2eb96f560bd0f1b88d P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.9, Last appended by leader: 9, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:08.273003 20782 raft_consensus.cc:493] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:21:08.265169 16790 leader_election.cc:304] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
W20260709 13:21:08.333930 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 68848.6us
I20260709 13:21:08.239535 21033 raft_consensus.cc:493] T c0eab25ce4c64e018a041beeda2867db P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Starting leader election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:21:08.336712 21033 raft_consensus.cc:3060] T c0eab25ce4c64e018a041beeda2867db P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:21:08.339370 21060 raft_consensus.cc:2749] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:21:08.253829 21069 raft_consensus.cc:493] T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:08.267390 20923 raft_consensus.cc:2749] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:21:08.224318 20911 consensus_queue.cc:1048] T 9e65676b21b748728ad39668cfe722ec P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.053s
I20260709 13:21:08.267208 20661 raft_consensus.cc:2749] T 5a96373bb0e14dce8a266b2fc6c58507 P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:21:08.346503 21141 raft_consensus.cc:2804] T 5a959dbecc2648eb85d1cc550f5a3245 P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:21:08.347011 21141 raft_consensus.cc:697] T 5a959dbecc2648eb85d1cc550f5a3245 P 1ca2725d290e47d68b1a4b609c9816ac [term 12 LEADER]: Becoming Leader. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:21:08.348121 16221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cc959fb7f704fba9009c0595467d074" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:08.347870 21141 consensus_queue.cc:237] T 5a959dbecc2648eb85d1cc550f5a3245 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:08.348328 16144 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46224 (request call id 731) took 1140 ms. Trace:
I20260709 13:21:08.348749 16144 rpcz_store.cc:276] 0709 13:21:07.208075 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:07.210763 (+  2688us) service_pool.cc:224] Handling call
0709 13:21:07.211013 (+   250us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:21:07.276778 (+ 65765us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:21:07.697679 (+420901us) log.cc:844] Serialized 14 byte log entry
0709 13:21:07.883117 (+185438us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:21:07.908444 (+ 25327us) log.cc:844] Serialized 26 byte log entry
0709 13:21:07.908652 (+   208us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:21:07.908701 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:08.124955 (+216254us) spinlock_profiling.cc:243] Waited 56.2 ms on lock 0x7b54000e9054. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:21:08.125022 (+    67us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:21:08.343276 (+218254us) raft_consensus.cc:1685] finished
0709 13:21:08.348147 (+  4871us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:21:08.348271 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":33048,"prepare.queue_time_us":449420,"prepare.run_cpu_time_us":83,"prepare.run_wall_time_us":2993,"raft.queue_time_us":9785,"raft.run_cpu_time_us":290,"raft.run_wall_time_us":38124,"spinlock_wait_cycles":129844736,"thread_start_us":578323,"threads_started":2,"wal-append.queue_time_us":314475}
I20260709 13:21:08.351364 16145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746aabf856ab401c9ef5e859371e7b67" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:08.352174 16145 raft_consensus.cc:2393] T 746aabf856ab401c9ef5e859371e7b67 P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 12: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
I20260709 13:21:08.265558 21096 raft_consensus.cc:515] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } }
W20260709 13:21:08.319406 16115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 78809.4us
I20260709 13:21:08.353852 21084 leader_election.cc:290] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:08.354707 21096 leader_election.cc:290] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 13 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:21:08.232434 21133 raft_consensus.cc:493] T ada5a004ba7c46819051e3ccec71ffdb P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:21:08.330295 16117 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 75862.3us
I20260709 13:21:08.360399 16117 leader_election.cc:304] T 9e65676b21b748728ad39668cfe722ec P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860
I20260709 13:21:08.360893 16116 leader_election.cc:304] T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: cb82cc7bd6db46d980c75237a1402860
I20260709 13:21:08.271268 21121 consensus_queue.cc:1048] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.063s
I20260709 13:21:08.277041 21001 raft_consensus.cc:493] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:21:08.365753 16817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada5a004ba7c46819051e3ccec71ffdb" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:08.344606 20782 raft_consensus.cc:3060] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:21:08.345120 16223 raft_consensus.cc:1240] T 829b4ad8c5a54eababeea1181b960d4b P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 12. Current term is 13. Ops: []
I20260709 13:21:08.305315 16222 raft_consensus.cc:2468] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 13.
I20260709 13:21:08.371961 16222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ad5dbfa99ee4b8d9e28be92b3f36db5" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:21:08.372896 16222 raft_consensus.cc:2393] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 12: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
I20260709 13:21:08.332929 21142 raft_consensus.cc:493] T 8cc959fb7f704fba9009c0595467d074 P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Starting leader election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:21:08.270417 20985 raft_consensus.cc:2804] T 9e65676b21b748728ad39668cfe722ec P cb82cc7bd6db46d980c75237a1402860 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:21:08.254405 21129 raft_consensus.cc:2804] T 23f82224f4564fea93572e8206bad1fe P cb82cc7bd6db46d980c75237a1402860 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:21:08.355372 21069 raft_consensus.cc:515] T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:08.331027 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:08.297148 20797 raft_consensus.cc:493] T 66f75124215f4ca6bef831e174a792d8 P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:08.363426 16818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31969392ccdb43b491911a319e8bc7e1" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:08.281872 20967 consensus_queue.cc:1243] T c4f65cdf00b748bb963ee6e81ef43b79 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Peer 1ca2725d290e47d68b1a4b609c9816ac log is divergent from this leader: its last log entry 15.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 13:21:08.330490 20641 raft_consensus.cc:3060] T af57996a8be84d398f0f023e8d08ef77 P cb82cc7bd6db46d980c75237a1402860 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:21:08.342679 21140 log.cc:927] Time spent T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
I20260709 13:21:08.364439 16823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434bff3082d141838068f323bdb0d1ed" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:08.366537 16817 raft_consensus.cc:2468] T ada5a004ba7c46819051e3ccec71ffdb P cb82cc7bd6db46d980c75237a1402860 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 13.
I20260709 13:21:08.368247 16820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2794af5b99984efebdee564035841cb0" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:08.376175 16819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26354361e9b044cea1c2b86c4882fe92" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:08.279903 21048 consensus_queue.cc:1048] T 8c11074579b046d99e585c2818e18a2c P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.078s
I20260709 13:21:08.358704 21009 raft_consensus.cc:493] T ecc1e363230043fe9fef390f0b979f20 P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:08.379899 16116 leader_election.cc:304] T 746aabf856ab401c9ef5e859371e7b67 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: cb82cc7bd6db46d980c75237a1402860
I20260709 13:21:08.380801 16222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6f07842d3aa47858df5665874c86f3a" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:21:08.390636 16825 raft_consensus.cc:1275] T 23f82224f4564fea93572e8206bad1fe P cb82cc7bd6db46d980c75237a1402860 [term 11 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
W20260709 13:21:08.392544 16190 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:08.362833 21133 raft_consensus.cc:3060] T ada5a004ba7c46819051e3ccec71ffdb P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:21:08.396946 16223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee9ed67006144797b1de6ea8c2013a68" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:08.378417 21142 raft_consensus.cc:3060] T 8cc959fb7f704fba9009c0595467d074 P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:21:08.500495 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18ad7e3af5504571a69d10c00750a01c" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:08.281900 21101 raft_consensus.cc:515] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:08.334120 21075 raft_consensus.cc:515] T c2f5f89186204242bf377f7c42b2e2da P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:08.403133 21149 raft_consensus.cc:2749] T 9e65676b21b748728ad39668cfe722ec P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:21:08.403831 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a959dbecc2648eb85d1cc550f5a3245" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
W20260709 13:21:08.540743 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 160934us
I20260709 13:21:08.398356 21001 raft_consensus.cc:515] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:08.378963 20985 raft_consensus.cc:493] T 9e65676b21b748728ad39668cfe722ec P cb82cc7bd6db46d980c75237a1402860 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:21:08.440694 16190 leader_election.cc:304] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 647a343a542d4a139fe106c19095129b
I20260709 13:21:08.473914 16223 raft_consensus.cc:2468] T ee9ed67006144797b1de6ea8c2013a68 P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 11.
I20260709 13:21:08.399870 16225 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57604 (request call id 2713) took 2022 ms. Trace:
I20260709 13:21:08.549723 16225 rpcz_store.cc:276] 0709 13:21:06.377664 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:06.428069 (+ 50405us) service_pool.cc:224] Handling call
0709 13:21:06.461435 (+ 33366us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:21:07.281157 (+819722us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:08.399567 (+1118410us) spinlock_profiling.cc:243] Waited 818 ms on lock 0x7b5400367c54. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:21:08.399788 (+   221us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1880646912}
I20260709 13:21:08.387430 21115 leader_election.cc:290] T 19baa7bc367844c288bca8f81439c413 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
W20260709 13:21:08.552621 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 112080us
I20260709 13:21:08.539052 16295 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48674 (request call id 2802) took 1149 ms. Trace:
I20260709 13:21:08.542502 16116 leader_election.cc:304] T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: cb82cc7bd6db46d980c75237a1402860
I20260709 13:21:08.544652 16222 raft_consensus.cc:1275] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 13:21:08.392357 20797 raft_consensus.cc:3060] T 66f75124215f4ca6bef831e174a792d8 P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:21:08.554575 16295 rpcz_store.cc:276] 0709 13:21:07.390027 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:07.393092 (+  3065us) service_pool.cc:224] Handling call
0709 13:21:08.538741 (+1145649us) spinlock_profiling.cc:243] Waited 387 ms on lock 0x7b5400240254. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:21:08.538967 (+   226us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":890349440}
I20260709 13:21:08.472992 20472 raft_consensus.cc:3055] T 26354361e9b044cea1c2b86c4882fe92 P 647a343a542d4a139fe106c19095129b [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:21:08.559019 16116 leader_election.cc:304] T d16dfad0c3b540e5b4b86492ddf87208 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:21:08.474278 21129 raft_consensus.cc:493] T 23f82224f4564fea93572e8206bad1fe P cb82cc7bd6db46d980c75237a1402860 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:08.560719 16218 raft_consensus.cc:1275] T c2f5f89186204242bf377f7c42b2e2da P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
W20260709 13:21:08.538118 21082 log.cc:927] Time spent T 2794af5b99984efebdee564035841cb0 P cb82cc7bd6db46d980c75237a1402860: Append to log took a long time: real 2.058s	user 0.004s	sys 0.014s
I20260709 13:21:08.554174 20985 raft_consensus.cc:3060] T 9e65676b21b748728ad39668cfe722ec P cb82cc7bd6db46d980c75237a1402860 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:21:08.572669 16819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7ca61592e848ea9b0fb81a2ce33cdc" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:08.541273 21150 raft_consensus.cc:2804] T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:21:08.579377 16217 raft_consensus.cc:1240] T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 11. Current term is 12. Ops: []
I20260709 13:21:08.370020 21112 consensus_queue.cc:237] T b6f07842d3aa47858df5665874c86f3a P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:08.563015 21129 raft_consensus.cc:3060] T 23f82224f4564fea93572e8206bad1fe P cb82cc7bd6db46d980c75237a1402860 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:21:08.560788 20472 raft_consensus.cc:740] T 26354361e9b044cea1c2b86c4882fe92 P 647a343a542d4a139fe106c19095129b [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:21:08.542297 21009 raft_consensus.cc:515] T ecc1e363230043fe9fef390f0b979f20 P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:08.569235 21034 raft_consensus.cc:515] T 1df93779209748b9860f6d893dd3de53 P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:08.670776 21057 leader_election.cc:290] T 8638ee363273426db1393dd81e5c7b72 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 15 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
W20260709 13:21:08.738714 16189 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:08.556668 20754 consensus_queue.cc:1048] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 10, Time since last communication: 0.027s
I20260709 13:21:08.746374 16147 raft_consensus.cc:2468] T 08c74490c40243a88e62d1760b5cd814 P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 13.
I20260709 13:21:08.583231 21150 raft_consensus.cc:493] T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:21:08.750011 20902 raft_consensus.cc:670] T 806d7690c2b644959152074bb2cab0a7 P cb82cc7bd6db46d980c75237a1402860: failed to trigger leader election: Illegal state: only voting members can start elections: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } }
W20260709 13:21:08.750404 20757 consensus_meta.cc:287] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 647a343a542d4a139fe106c19095129b: Time spent flushing consensus metadata: real 0.538s	user 0.007s	sys 0.003s
I20260709 13:21:08.751296 21048 raft_consensus.cc:2804] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Leader election won for term 12
W20260709 13:21:08.750969 20894 consensus_meta.cc:287] T 57a744f5aaf944678a63469cbf514015 P 90f3da57796b488c9087a2f523adf045: Time spent flushing consensus metadata: real 0.544s	user 0.007s	sys 0.003s
I20260709 13:21:08.751560 21087 raft_consensus.cc:2749] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:21:08.751775 21068 consensus_meta.cc:287] T fffab26427e24d398e2695aad40ca134 P 1ca2725d290e47d68b1a4b609c9816ac: Time spent flushing consensus metadata: real 0.639s	user 0.004s	sys 0.007s
I20260709 13:21:08.753674 21033 raft_consensus.cc:515] T c0eab25ce4c64e018a041beeda2867db P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:08.758621 21142 raft_consensus.cc:515] T 8cc959fb7f704fba9009c0595467d074 P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:08.760845 16822 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49896 (request call id 1224) took 2794 ms. Trace:
I20260709 13:21:08.774611 21103 raft_consensus.cc:2804] T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:21:08.738108 21159 raft_consensus.cc:493] T 02838c86531d4f98bf58d1c22b8baf77 P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:21:08.763720 21129 raft_consensus.cc:515] T 23f82224f4564fea93572e8206bad1fe P cb82cc7bd6db46d980c75237a1402860 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:21:08.798259 16191 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.531s	user 0.000s	sys 0.014s
W20260709 13:21:08.827598 16191 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.560s	user 0.000s	sys 0.014s
W20260709 13:21:08.828039 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 566437us
I20260709 13:21:08.829362 16219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a378d7f796f94615b3b347119a9880fd" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 15 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:21:08.764957 16819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ab0ac8347641e6955e9bce7d90c1a2" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 14 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:08.570545 21149 raft_consensus.cc:2804] T 746aabf856ab401c9ef5e859371e7b67 P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:21:08.791129 21048 raft_consensus.cc:697] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 1ca2725d290e47d68b1a4b609c9816ac [term 12 LEADER]: Becoming Leader. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
W20260709 13:21:08.781005 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 227297us
I20260709 13:21:08.790396 16822 rpcz_store.cc:276] 0709 13:21:05.966740 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:05.966939 (+   199us) service_pool.cc:224] Handling call
0709 13:21:05.967383 (+   444us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:21:05.967810 (+   427us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:21:05.967849 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:21:05.986029 (+ 18180us) log.cc:844] Serialized 26 byte log entry
0709 13:21:06.308425 (+322396us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:21:06.308471 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:06.311030 (+  2559us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:21:08.537335 (+2226305us) spinlock_profiling.cc:243] Waited 8.72 ms on lock 0x7b4400022b4c. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69ad8fadb1 00007f69ad998c85 00007f69ad9996a2 00007f69b070b1db 00007f69b0706aaf 00007f69b06fa86e 00007f69b06f9bb6 00007f69b06f97d8 00007f69b0703221 00007f69b07031d9 00007f69b0703169
0709 13:21:08.760462 (+223127us) raft_consensus.cc:1685] finished
0709 13:21:08.760486 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:21:08.760755 (+   269us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8981,"raft.queue_time_us":1286,"raft.run_cpu_time_us":179,"raft.run_wall_time_us":16055,"spinlock_wait_cycles":20066560,"thread_start_us":245764,"threads_started":1,"wal-append.queue_time_us":337483}
I20260709 13:21:08.762111 21075 leader_election.cc:290] T c2f5f89186204242bf377f7c42b2e2da P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
W20260709 13:21:08.793358 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 222422us
I20260709 13:21:08.754482 21133 raft_consensus.cc:515] T ada5a004ba7c46819051e3ccec71ffdb P 90f3da57796b488c9087a2f523adf045 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:08.803469 16221 raft_consensus.cc:2393] T 8cc959fb7f704fba9009c0595467d074 P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 13: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
I20260709 13:21:08.897708 16116 leader_election.cc:304] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:21:08.789160 21154 raft_consensus.cc:493] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:08.781826 21150 raft_consensus.cc:3060] T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:21:08.769500 16217 raft_consensus.cc:1240] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 12. Current term is 13. Ops: []
I20260709 13:21:08.896416 16219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af57996a8be84d398f0f023e8d08ef77" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:08.794835 21103 raft_consensus.cc:493] T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b [term 14 FOLLOWER]: Starting leader election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
W20260709 13:21:08.895007 16115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 517562us
I20260709 13:21:08.903201 21154 raft_consensus.cc:515] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:08.907035 20985 raft_consensus.cc:515] T 9e65676b21b748728ad39668cfe722ec P cb82cc7bd6db46d980c75237a1402860 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:08.802026 21153 raft_consensus.cc:2749] T d16dfad0c3b540e5b4b86492ddf87208 P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:21:08.908298 16819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2372981006d4b218215c28964dea623" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 15 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:08.909638 21097 raft_consensus.cc:2749] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
W20260709 13:21:08.902611 20999 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.504s	user 0.007s	sys 0.007s
I20260709 13:21:08.911265 16219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e4bfb9090c8460eb14fa7e0d818e7f1" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:21:08.917821 20797 raft_consensus.cc:515] T 66f75124215f4ca6bef831e174a792d8 P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:21:08.920321 20782 consensus_meta.cc:287] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860: Time spent flushing consensus metadata: real 0.528s	user 0.008s	sys 0.000s
I20260709 13:21:08.924070 21142 leader_election.cc:290] T 8cc959fb7f704fba9009c0595467d074 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 13 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:08.804740 20472 consensus_queue.cc:260] T 26354361e9b044cea1c2b86c4882fe92 P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:08.982371 16191 leader_election.cc:304] T 746aabf856ab401c9ef5e859371e7b67 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
I20260709 13:21:08.983613 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2372981006d4b218215c28964dea623" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 15 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
W20260709 13:21:08.927315 21148 log.cc:927] Time spent T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 0.597s	user 0.005s	sys 0.005s
I20260709 13:21:08.929014 21110 raft_consensus.cc:3055] T c4f65cdf00b748bb963ee6e81ef43b79 P 90f3da57796b488c9087a2f523adf045 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:21:08.937124 16223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdbd78acb4cf46298e0de8ba0d130e19" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:21:08.941725 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26354361e9b044cea1c2b86c4882fe92" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:08.790174 21068 raft_consensus.cc:515] T fffab26427e24d398e2695aad40ca134 P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:08.992909 16151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a96373bb0e14dce8a266b2fc6c58507" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:08.953042 21101 leader_election.cc:290] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:08.800971 21159 raft_consensus.cc:515] T 02838c86531d4f98bf58d1c22b8baf77 P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:08.955194 21034 leader_election.cc:290] T 1df93779209748b9860f6d893dd3de53 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 13 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:09.004590 16151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f5f89186204242bf377f7c42b2e2da" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 15 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:08.782379 20757 raft_consensus.cc:515] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:21:08.962368 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 60379.3us
I20260709 13:21:08.955294 21069 leader_election.cc:290] T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:08.969594 16147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b7d1ab519044eda2d5e04a07242f25" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:09.011245 16191 leader_election.cc:304] T ee9ed67006144797b1de6ea8c2013a68 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: cb82cc7bd6db46d980c75237a1402860
I20260709 13:21:09.011618 16147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5718c623ebee4d2bb770f0f9ddb2f164" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:08.972028 16146 pending_rounds.cc:85] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b: Aborting all ops after (but not including) 6
W20260709 13:21:08.973438 16192 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 418899us
I20260709 13:21:09.014458 16146 pending_rounds.cc:107] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b: Aborting uncommitted NO_OP operation due to leader change: 9.7
I20260709 13:21:09.014801 16146 raft_consensus.cc:2889] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:21:08.977308 16143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02838c86531d4f98bf58d1c22b8baf77" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:08.873934 21057 raft_consensus.cc:493] T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:08.978902 16217 raft_consensus.cc:1240] T 746aabf856ab401c9ef5e859371e7b67 P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 11. Current term is 12. Ops: []
W20260709 13:21:08.996518 16117 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:09.024230 16223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74ca8140f35c4ae29e8a664f6bdfc829" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:09.027585 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8638ee363273426db1393dd81e5c7b72" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:09.028472 16217 raft_consensus.cc:2468] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 14.
I20260709 13:21:09.028494 16147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cc959fb7f704fba9009c0595467d074" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:21:09.029495 16147 raft_consensus.cc:3060] T 8cc959fb7f704fba9009c0595467d074 P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:21:09.032325 16220 raft_consensus.cc:1240] T fffab26427e24d398e2695aad40ca134 P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 11. Current term is 12. Ops: []
I20260709 13:21:09.034526 16143 raft_consensus.cc:2410] T 02838c86531d4f98bf58d1c22b8baf77 P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac for term 14 because replica has last-logged OpId of term: 13 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 7.
I20260709 13:21:09.035851 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9944d5e4b8c4109af9955c4f08c9095" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:09.037181 16217 raft_consensus.cc:2468] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 14.
W20260709 13:21:08.990296 16114 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.515s	user 0.014s	sys 0.016s
I20260709 13:21:08.970803 21103 raft_consensus.cc:3060] T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:21:09.048489 16223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 14 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:08.786996 20894 raft_consensus.cc:515] T 57a744f5aaf944678a63469cbf514015 P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:09.053217 20894 leader_election.cc:290] T 57a744f5aaf944678a63469cbf514015 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 10 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:09.016224 16151 raft_consensus.cc:3055] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:21:09.053970 16151 raft_consensus.cc:740] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:21:09.030650 21057 raft_consensus.cc:515] T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:09.057948 21057 leader_election.cc:290] T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:21:09.030431 20472 raft_consensus.cc:3060] T 26354361e9b044cea1c2b86c4882fe92 P 647a343a542d4a139fe106c19095129b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:21:08.986910 20782 raft_consensus.cc:515] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:09.060956 20782 leader_election.cc:290] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 11 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:09.063983 21068 leader_election.cc:290] T fffab26427e24d398e2695aad40ca134 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 12 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:09.065840 21154 leader_election.cc:290] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:09.067004 20985 leader_election.cc:290] T 9e65676b21b748728ad39668cfe722ec P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 13 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:09.041388 21161 raft_consensus.cc:2804] T ee9ed67006144797b1de6ea8c2013a68 P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:21:09.067902 21161 raft_consensus.cc:493] T ee9ed67006144797b1de6ea8c2013a68 P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Starting leader election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:21:09.068359 21161 raft_consensus.cc:3060] T ee9ed67006144797b1de6ea8c2013a68 P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:21:09.069468 20797 leader_election.cc:290] T 66f75124215f4ca6bef831e174a792d8 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 13 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:09.072902 21048 consensus_queue.cc:237] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:09.078639 20991 raft_consensus.cc:3055] T 8d4ab870a05347868956aa8b841b2981 P 647a343a542d4a139fe106c19095129b [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:21:09.078877 20895 raft_consensus.cc:2804] T d16dfad0c3b540e5b4b86492ddf87208 P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:21:09.079495 20895 raft_consensus.cc:493] T d16dfad0c3b540e5b4b86492ddf87208 P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:21:09.079854 20895 raft_consensus.cc:3060] T d16dfad0c3b540e5b4b86492ddf87208 P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:21:09.079918 21033 leader_election.cc:290] T c0eab25ce4c64e018a041beeda2867db P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 13 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:09.030965 21001 leader_election.cc:290] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:09.030761 21142 raft_consensus.cc:2749] T 746aabf856ab401c9ef5e859371e7b67 P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:21:09.081167 20757 leader_election.cc:290] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 11 election: Requested vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
W20260709 13:21:09.078682 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 814886us
I20260709 13:21:09.083354 16825 raft_consensus.cc:1275] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [term 17 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 13:21:08.988891 21110 raft_consensus.cc:740] T c4f65cdf00b748bb963ee6e81ef43b79 P 90f3da57796b488c9087a2f523adf045 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:21:09.008452 20999 raft_consensus.cc:493] T b87147731a68400e8cce9b879daa7c5d P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:21:08.996721 20641 consensus_meta.cc:287] T af57996a8be84d398f0f023e8d08ef77 P cb82cc7bd6db46d980c75237a1402860: Time spent flushing consensus metadata: real 0.518s	user 0.008s	sys 0.002s
I20260709 13:21:09.045801 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62fe3346cd3c42c4a55e14af9073bca4" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:09.090268 21093 consensus_queue.cc:1059] T c4f65cdf00b748bb963ee6e81ef43b79 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:21:09.090943 20999 raft_consensus.cc:515] T b87147731a68400e8cce9b879daa7c5d P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:09.045939 21034 raft_consensus.cc:493] T ecc1e363230043fe9fef390f0b979f20 P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:21:09.093204 21148 log.cc:927] Time spent T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 0.053s	user 0.001s	sys 0.000s
I20260709 13:21:09.086061 20991 raft_consensus.cc:740] T 8d4ab870a05347868956aa8b841b2981 P 647a343a542d4a139fe106c19095129b [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:21:09.099864 16219 raft_consensus.cc:2468] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 12.
I20260709 13:21:09.086374 21101 raft_consensus.cc:493] T d2ab0ac8347641e6955e9bce7d90c1a2 P cb82cc7bd6db46d980c75237a1402860 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:21:09.102766 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:09.104327 16219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8638ee363273426db1393dd81e5c7b72" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:21:09.105176 16217 raft_consensus.cc:2468] T 19baa7bc367844c288bca8f81439c413 P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 14.
W20260709 13:21:09.107935 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 210287us
I20260709 13:21:09.109007 16116 leader_election.cc:304] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
W20260709 13:21:09.109892 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 190050us
I20260709 13:21:09.105201 16219 raft_consensus.cc:3060] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:21:09.103030 16147 raft_consensus.cc:2468] T 8cc959fb7f704fba9009c0595467d074 P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 13.
I20260709 13:21:09.115336 16147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1df93779209748b9860f6d893dd3de53" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:21:09.116096 16147 raft_consensus.cc:3060] T 1df93779209748b9860f6d893dd3de53 P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:21:09.047616 16114 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.572s	user 0.014s	sys 0.016s
W20260709 13:21:09.126228 16821 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.537s	user 0.003s	sys 0.017s
W20260709 13:21:09.126868 16821 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.538s	user 0.003s	sys 0.017s
I20260709 13:21:09.138056 16217 raft_consensus.cc:1240] T 66f75124215f4ca6bef831e174a792d8 P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 12. Current term is 13. Ops: []
I20260709 13:21:09.152575 16217 raft_consensus.cc:1275] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
W20260709 13:21:09.171854 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 65481.9us
I20260709 13:21:09.172865 16265 leader_election.cc:304] T 7c2095a5f4ad49408dbc7f69c606120d P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:21:09.103976 20931 raft_consensus.cc:493] T cdbd78acb4cf46298e0de8ba0d130e19 P 90f3da57796b488c9087a2f523adf045 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:21:09.173604 16790 leader_election.cc:304] T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:21:09.180402 16265 leader_election.cc:304] T 8a878999c0024d2abe517484180290fc P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:21:09.098491 20641 raft_consensus.cc:515] T af57996a8be84d398f0f023e8d08ef77 P cb82cc7bd6db46d980c75237a1402860 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:09.181867 20782 raft_consensus.cc:2749] T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:21:09.186391 16824 raft_consensus.cc:1240] T af57996a8be84d398f0f023e8d08ef77 P cb82cc7bd6db46d980c75237a1402860 [term 15 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 14. Current term is 15. Ops: []
I20260709 13:21:09.188740 20641 leader_election.cc:290] T af57996a8be84d398f0f023e8d08ef77 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 15 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:21:09.127013 20894 consensus_queue.cc:1048] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.056s
I20260709 13:21:09.180930 16790 leader_election.cc:304] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:21:09.192128 20236 raft_consensus.cc:493] T 829b4ad8c5a54eababeea1181b960d4b P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:09.194617 20236 raft_consensus.cc:515] T 829b4ad8c5a54eababeea1181b960d4b P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:21:09.191869 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 82953.9us
I20260709 13:21:09.197450 16116 leader_election.cc:304] T 18ad7e3af5504571a69d10c00750a01c P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:21:09.199901 20641 raft_consensus.cc:2749] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:21:09.167934 20757 raft_consensus.cc:493] T 9c61a39fd4f84424b129c1a6f9ba404d P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:21:09.201876 16265 leader_election.cc:304] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:21:09.208849 16816 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49906 (request call id 975) took 2507 ms. Trace:
I20260709 13:21:09.182854 20889 raft_consensus.cc:493] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:09.100579 21034 raft_consensus.cc:515] T ecc1e363230043fe9fef390f0b979f20 P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:09.209241 16816 rpcz_store.cc:276] 0709 13:21:06.701572 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:06.702561 (+   989us) service_pool.cc:224] Handling call
0709 13:21:06.702990 (+   429us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:21:07.681885 (+978895us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:09.208661 (+1526776us) spinlock_profiling.cc:243] Waited 973 ms on lock 0x7b54004b5d54. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:21:09.208804 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2237252864}
I20260709 13:21:09.170652 20661 consensus_queue.cc:1048] T fffab26427e24d398e2695aad40ca134 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.033s
I20260709 13:21:09.216149 20236 leader_election.cc:290] T 829b4ad8c5a54eababeea1181b960d4b P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
W20260709 13:21:09.217959 16266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 120517us
I20260709 13:21:09.218217 21188 raft_consensus.cc:2804] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:21:09.184191 20939 raft_consensus.cc:2749] T 7c2095a5f4ad49408dbc7f69c606120d P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:21:09.140894 21101 raft_consensus.cc:515] T d2ab0ac8347641e6955e9bce7d90c1a2 P cb82cc7bd6db46d980c75237a1402860 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:09.183513 21093 raft_consensus.cc:2749] T 8a878999c0024d2abe517484180290fc P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:21:09.224632 16219 raft_consensus.cc:2468] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 15.
I20260709 13:21:09.189141 21009 leader_election.cc:290] T ecc1e363230043fe9fef390f0b979f20 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
W20260709 13:21:09.215448 16263 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:09.200086 20832 consensus_queue.cc:1048] T ecc1e363230043fe9fef390f0b979f20 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.023s
I20260709 13:21:09.242134 16223 raft_consensus.cc:2410] T ecc1e363230043fe9fef390f0b979f20 P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 for term 14 because replica has last-logged OpId of term: 11 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 7.
I20260709 13:21:09.225741 16816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5718c623ebee4d2bb770f0f9ddb2f164" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:09.200460 21033 raft_consensus.cc:2749] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:21:09.230371 16147 raft_consensus.cc:2468] T 1df93779209748b9860f6d893dd3de53 P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 13.
I20260709 13:21:09.251861 16115 leader_election.cc:400] T 08c74490c40243a88e62d1760b5cd814 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 12 pre-election: Vote denied by peer 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389) with higher term. Message: Invalid argument: T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b for earlier term 12. Current term is 13.
I20260709 13:21:09.252254 16115 leader_election.cc:403] T 08c74490c40243a88e62d1760b5cd814 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 12 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:21:09.252321 16147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fffab26427e24d398e2695aad40ca134" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:21:09.252858 20236 raft_consensus.cc:2749] T 08c74490c40243a88e62d1760b5cd814 P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Leader pre-election lost for term 12. Reason: Vote denied by peer 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389) with higher term. Message: Invalid argument: T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b for earlier term 12. Current term is 13.
I20260709 13:21:09.253139 16147 raft_consensus.cc:3060] T fffab26427e24d398e2695aad40ca134 P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:21:09.230384 16790 leader_election.cc:304] T 74ca8140f35c4ae29e8a664f6bdfc829 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:21:09.256078 16790 leader_election.cc:304] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:21:09.203715 21174 raft_consensus.cc:493] T 84f2da98864b4425bf7c0055951fdd4b P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:21:09.191903 20991 consensus_queue.cc:260] T 8d4ab870a05347868956aa8b841b2981 P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:09.210023 21168 raft_consensus.cc:2749] T 18ad7e3af5504571a69d10c00750a01c P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:21:09.236521 16219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0eab25ce4c64e018a041beeda2867db" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:21:09.243517 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0eab25ce4c64e018a041beeda2867db" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:21:09.244567 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e9b2bf7011f4ae688a79d7bb5a447b4" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:21:09.246886 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:09.261485 16822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9944d5e4b8c4109af9955c4f08c9095" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
W20260709 13:21:09.262391 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 78213.8us
I20260709 13:21:09.281390 16822 raft_consensus.cc:2468] T c9944d5e4b8c4109af9955c4f08c9095 P cb82cc7bd6db46d980c75237a1402860 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 13.
I20260709 13:21:09.192373 20931 raft_consensus.cc:515] T cdbd78acb4cf46298e0de8ba0d130e19 P 90f3da57796b488c9087a2f523adf045 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:09.264308 16790 leader_election.cc:304] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 90f3da57796b488c9087a2f523adf045
I20260709 13:21:09.227909 21133 leader_election.cc:290] T ada5a004ba7c46819051e3ccec71ffdb P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 14 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:09.215008 20757 raft_consensus.cc:515] T 9c61a39fd4f84424b129c1a6f9ba404d P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:09.213056 20919 consensus_queue.cc:1048] T 23f82224f4564fea93572e8206bad1fe P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.040s
I20260709 13:21:09.282984 16219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66ae176961b74813a58bc15c9c4c6f2a" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:09.300336 16219 raft_consensus.cc:2468] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 15.
I20260709 13:21:09.301661 16219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806d7690c2b644959152074bb2cab0a7" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:21:09.288239 16292 raft_consensus.cc:1240] T 8d4ab870a05347868956aa8b841b2981 P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 11. Current term is 12. Ops: []
I20260709 13:21:09.288321 16144 raft_consensus.cc:1240] T b6f07842d3aa47858df5665874c86f3a P 647a343a542d4a139fe106c19095129b [term 15 LEADER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 14. Current term is 15. Ops: []
I20260709 13:21:09.274103 16147 raft_consensus.cc:2468] T fffab26427e24d398e2695aad40ca134 P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 12.
I20260709 13:21:09.304004 16147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57a744f5aaf944678a63469cbf514015" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:21:09.304980 16147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e9b2bf7011f4ae688a79d7bb5a447b4" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:21:09.305501 16147 raft_consensus.cc:2393] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 in current term 11: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
I20260709 13:21:09.293807 16790 leader_election.cc:304] T 19baa7bc367844c288bca8f81439c413 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860; no voters: 90f3da57796b488c9087a2f523adf045
I20260709 13:21:09.295202 16148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66f75124215f4ca6bef831e174a792d8" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:21:09.239338 20911 raft_consensus.cc:493] T c9944d5e4b8c4109af9955c4f08c9095 P 90f3da57796b488c9087a2f523adf045 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:09.297807 16293 raft_consensus.cc:1240] T 57a744f5aaf944678a63469cbf514015 P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 9. Current term is 10. Ops: []
I20260709 13:21:09.241329 21046 raft_consensus.cc:493] T 18ad7e3af5504571a69d10c00750a01c P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:21:09.313170 16825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66f75124215f4ca6bef831e174a792d8" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:21:09.277772 20902 raft_consensus.cc:2749] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:21:09.315544 21121 consensus_queue.cc:1059] T 66f75124215f4ca6bef831e174a792d8 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:21:09.316642 20667 raft_consensus.cc:2804] T 19baa7bc367844c288bca8f81439c413 P cb82cc7bd6db46d980c75237a1402860 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:21:09.317342 20667 raft_consensus.cc:493] T 19baa7bc367844c288bca8f81439c413 P cb82cc7bd6db46d980c75237a1402860 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:21:09.317871 20667 raft_consensus.cc:3060] T 19baa7bc367844c288bca8f81439c413 P cb82cc7bd6db46d980c75237a1402860 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:21:09.320727 20931 leader_election.cc:290] T cdbd78acb4cf46298e0de8ba0d130e19 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:09.320514 21046 raft_consensus.cc:515] T 18ad7e3af5504571a69d10c00750a01c P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } }
I20260709 13:21:09.234359 20641 consensus_queue.cc:1059] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: INVALID_TERM, Last received: 11.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.062s
I20260709 13:21:09.306349 20911 raft_consensus.cc:515] T c9944d5e4b8c4109af9955c4f08c9095 P 90f3da57796b488c9087a2f523adf045 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:09.227842 20889 raft_consensus.cc:515] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:09.275513 21115 raft_consensus.cc:2749] T 74ca8140f35c4ae29e8a664f6bdfc829 P cb82cc7bd6db46d980c75237a1402860 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:21:09.310384 16191 leader_election.cc:304] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
I20260709 13:21:09.277433 21149 raft_consensus.cc:697] T 746aabf856ab401c9ef5e859371e7b67 P 647a343a542d4a139fe106c19095129b [term 12 LEADER]: Becoming Leader. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:21:09.315754 21191 raft_consensus.cc:493] T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:21:09.316596 20902 consensus_queue.cc:1048] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:21:09.260480 21188 raft_consensus.cc:697] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045 [term 13 LEADER]: Becoming Leader. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:21:09.331296 21034 leader_election.cc:290] T ecc1e363230043fe9fef390f0b979f20 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:09.288774 20961 raft_consensus.cc:493] T 84f2da98864b4425bf7c0055951fdd4b P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:09.334564 16149 raft_consensus.cc:2468] T 26354361e9b044cea1c2b86c4882fe92 P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 11.
W20260709 13:21:09.336972 16266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 108622us
I20260709 13:21:09.287711 21175 raft_consensus.cc:493] T e5ad4eb281564936a59f41e387a47156 P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:21:09.311901 21057 raft_consensus.cc:2804] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P cb82cc7bd6db46d980c75237a1402860 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:21:09.278870 21174 raft_consensus.cc:515] T 84f2da98864b4425bf7c0055951fdd4b P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:09.314419 20991 raft_consensus.cc:3060] T 8d4ab870a05347868956aa8b841b2981 P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:21:09.350131 16191 leader_election.cc:304] T 8cc959fb7f704fba9009c0595467d074 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
I20260709 13:21:09.354115 16146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806d7690c2b644959152074bb2cab0a7" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:21:09.283766 21069 consensus_queue.cc:1059] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 8, Time since last communication: 0.053s
I20260709 13:21:09.273021 20797 consensus_queue.cc:1048] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.055s
I20260709 13:21:09.358420 16266 leader_election.cc:304] T 3884f864a96d428785895e5463e5728f P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 647a343a542d4a139fe106c19095129b
W20260709 13:21:09.554621 16266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 196271us
I20260709 13:21:09.308218 21093 consensus_queue.cc:1048] T a378d7f796f94615b3b347119a9880fd P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.048s
I20260709 13:21:09.557885 21093 raft_consensus.cc:493] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 90f3da57796b488c9087a2f523adf045 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:09.352862 21142 raft_consensus.cc:2749] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:21:09.587570 16115 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:09.356684 20923 raft_consensus.cc:3055] T 66f75124215f4ca6bef831e174a792d8 P cb82cc7bd6db46d980c75237a1402860 [term 12 LEADER]: Stepping down as leader of term 12
W20260709 13:21:09.606462 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 299614us
I20260709 13:21:09.558455 21093 raft_consensus.cc:515] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 90f3da57796b488c9087a2f523adf045 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:21:09.621657 21173 log.cc:927] Time spent T 00000000000000000000000000000000 P 667a34b27a39474ebefc1af3cf5b4271: Append to log took a long time: real 0.313s	user 0.001s	sys 0.000s
W20260709 13:21:09.656798 21103 consensus_meta.cc:287] T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b: Time spent flushing consensus metadata: real 0.608s	user 0.010s	sys 0.008s
I20260709 13:21:09.656816 16790 leader_election.cc:304] T 8638ee363273426db1393dd81e5c7b72 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:21:09.657440 21103 raft_consensus.cc:515] T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:09.658286 16790 leader_election.cc:304] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:21:09.658726 16150 raft_consensus.cc:2393] T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 15: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
I20260709 13:21:09.553031 21046 leader_election.cc:290] T 18ad7e3af5504571a69d10c00750a01c P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:09.585287 21034 raft_consensus.cc:2804] T 8cc959fb7f704fba9009c0595467d074 P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:21:09.672130 16825 raft_consensus.cc:1240] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [term 17 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 16. Current term is 17. Ops: []
I20260709 13:21:09.584532 21057 raft_consensus.cc:697] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P cb82cc7bd6db46d980c75237a1402860 [term 12 LEADER]: Becoming Leader. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:21:09.596180 20923 raft_consensus.cc:740] T 66f75124215f4ca6bef831e174a792d8 P cb82cc7bd6db46d980c75237a1402860 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:21:09.665385 20998 raft_consensus.cc:3060] T 8a878999c0024d2abe517484180290fc P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:21:09.369628 20961 raft_consensus.cc:515] T 84f2da98864b4425bf7c0055951fdd4b P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:09.810374 21149 consensus_queue.cc:237] T 746aabf856ab401c9ef5e859371e7b67 P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:09.845528 16825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:09.862236 16825 raft_consensus.cc:2468] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 13.
I20260709 13:21:09.863579 16825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "829b4ad8c5a54eababeea1181b960d4b" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:09.816846 20832 consensus_queue.cc:1048] T ecc1e363230043fe9fef390f0b979f20 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.035s
I20260709 13:21:09.848150 21093 leader_election.cc:290] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:21:09.810914 21195 raft_consensus.cc:2804] T 3884f864a96d428785895e5463e5728f P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:21:09.853039 21060 raft_consensus.cc:2804] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:21:09.858990 16219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af57996a8be84d398f0f023e8d08ef77" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:09.664960 21180 raft_consensus.cc:493] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:21:09.354813 21191 raft_consensus.cc:515] T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:09.871357 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66ae176961b74813a58bc15c9c4c6f2a" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:09.872258 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e9b2bf7011f4ae688a79d7bb5a447b4" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:21:09.873065 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e65676b21b748728ad39668cfe722ec" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:21:09.864651 16825 raft_consensus.cc:2468] T 829b4ad8c5a54eababeea1181b960d4b P cb82cc7bd6db46d980c75237a1402860 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 12.
I20260709 13:21:09.369125 21175 raft_consensus.cc:515] T e5ad4eb281564936a59f41e387a47156 P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:09.853641 21204 raft_consensus.cc:2804] T 8638ee363273426db1393dd81e5c7b72 P cb82cc7bd6db46d980c75237a1402860 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:21:09.873878 21195 raft_consensus.cc:493] T 3884f864a96d428785895e5463e5728f P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:21:09.954813 16292 raft_consensus.cc:1275] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:21:09.952950 16825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e9b2bf7011f4ae688a79d7bb5a447b4" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:21:09.932133 16296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8638ee363273426db1393dd81e5c7b72" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:21:09.972607 16825 raft_consensus.cc:2393] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 11: Already voted for candidate cb82cc7bd6db46d980c75237a1402860 in this term.
W20260709 13:21:09.972891 16266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 416675us
W20260709 13:21:09.919595 21048 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.554s	user 0.025s	sys 0.082s
I20260709 13:21:09.933516 21189 consensus_queue.cc:1048] T b87147731a68400e8cce9b879daa7c5d P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.106s
W20260709 13:21:09.976475 21048 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.620s	user 0.026s	sys 0.083s
I20260709 13:21:09.978880 16266 leader_election.cc:304] T 26354361e9b044cea1c2b86c4882fe92 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
W20260709 13:21:10.136883 16190 leader_election.cc:336] T 7c2095a5f4ad49408dbc7f69c606120d P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389): Timed out: RequestConsensusVote RPC to 127.14.177.131:41389 timed out after 1.756s (SENT)
I20260709 13:21:10.137598 16190 leader_election.cc:304] T 7c2095a5f4ad49408dbc7f69c606120d P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:21:09.921684 21060 raft_consensus.cc:493] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:21:09.968631 21101 leader_election.cc:290] T d2ab0ac8347641e6955e9bce7d90c1a2 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:09.962167 20923 consensus_queue.cc:260] T 66f75124215f4ca6bef831e174a792d8 P cb82cc7bd6db46d980c75237a1402860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:09.854594 21188 consensus_queue.cc:237] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:10.289422 16292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ab0ac8347641e6955e9bce7d90c1a2" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:10.577415 16292 raft_consensus.cc:2468] T d2ab0ac8347641e6955e9bce7d90c1a2 P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 13.
W20260709 13:21:10.359025 16266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 380233us
I20260709 13:21:10.589435 16266 leader_election.cc:304] T 434bff3082d141838068f323bdb0d1ed P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
W20260709 13:21:10.590175 16298 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.670s	user 0.024s	sys 0.026s
W20260709 13:21:10.589689 16264 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.716s	user 0.016s	sys 0.034s
W20260709 13:21:10.590555 16298 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.720s	user 0.024s	sys 0.027s
W20260709 13:21:10.590852 16291 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.916s	user 0.020s	sys 0.028s
W20260709 13:21:10.591364 16291 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.916s	user 0.020s	sys 0.028s
I20260709 13:21:10.591707 16298 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48674 (request call id 2838) took 1347 ms. Trace:
I20260709 13:21:10.181629 16219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
W20260709 13:21:10.593235 16265 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.923s	user 0.021s	sys 0.030s
W20260709 13:21:10.595695 16265 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.926s	user 0.021s	sys 0.030s
W20260709 13:21:10.596107 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 940016us
W20260709 13:21:10.597489 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 460642us
I20260709 13:21:09.964529 21194 raft_consensus.cc:493] T 8638ee363273426db1393dd81e5c7b72 P 90f3da57796b488c9087a2f523adf045 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:10.598970 16266 leader_election.cc:304] T 2794af5b99984efebdee564035841cb0 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
I20260709 13:21:09.963793 21198 raft_consensus.cc:493] T 829b4ad8c5a54eababeea1181b960d4b P cb82cc7bd6db46d980c75237a1402860 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
W20260709 13:21:10.290851 21190 log.cc:927] Time spent T 19baa7bc367844c288bca8f81439c413 P cb82cc7bd6db46d980c75237a1402860: Append to log took a long time: real 0.625s	user 0.000s	sys 0.000s
W20260709 13:21:10.346333 20667 consensus_meta.cc:287] T 19baa7bc367844c288bca8f81439c413 P cb82cc7bd6db46d980c75237a1402860: Time spent flushing consensus metadata: real 1.028s	user 0.010s	sys 0.000s
W20260709 13:21:10.482371 21150 consensus_meta.cc:287] T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b: Time spent flushing consensus metadata: real 1.506s	user 0.010s	sys 0.002s
I20260709 13:21:10.592193 16291 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48674 (request call id 2834) took 1351 ms. Trace:
I20260709 13:21:10.608174 21203 raft_consensus.cc:2749] T 434bff3082d141838068f323bdb0d1ed P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:21:10.608085 21209 consensus_queue.cc:1048] T 23f82224f4564fea93572e8206bad1fe P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:21:10.609359 21017 raft_consensus.cc:2749] T 7c2095a5f4ad49408dbc7f69c606120d P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:21:10.357583 20911 leader_election.cc:290] T c9944d5e4b8c4109af9955c4f08c9095 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:09.952864 20797 raft_consensus.cc:493] T e4dc6b0541ca431ebc757beac00521f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:21:09.932261 21046 raft_consensus.cc:493] T c0eab25ce4c64e018a041beeda2867db P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:21:10.592430 16298 rpcz_store.cc:276] 0709 13:21:09.244514 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:09.244637 (+   123us) service_pool.cc:224] Handling call
0709 13:21:09.244918 (+   281us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:21:09.245117 (+   199us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:21:09.341434 (+ 96317us) log.cc:844] Serialized 14 byte log entry
0709 13:21:10.591147 (+1249713us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:21:10.591272 (+   125us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:21:10.591339 (+    67us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:10.591438 (+    99us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:21:10.591628 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":184203,"prepare.queue_time_us":599363,"prepare.run_cpu_time_us":41,"prepare.run_wall_time_us":42,"raft.queue_time_us":431128,"raft.run_cpu_time_us":191,"raft.run_wall_time_us":112427,"thread_start_us":1155794,"threads_started":3,"wal-append.queue_time_us":508528}
I20260709 13:21:10.321894 21175 leader_election.cc:290] T e5ad4eb281564936a59f41e387a47156 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:10.601197 16219 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57604 (request call id 2750) took 1381 ms. Trace:
I20260709 13:21:10.626818 16219 rpcz_store.cc:276] 0709 13:21:09.220272 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:09.969762 (+749490us) service_pool.cc:224] Handling call
0709 13:21:10.601119 (+631357us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:21:09.980260 21195 raft_consensus.cc:3060] T 3884f864a96d428785895e5463e5728f P 90f3da57796b488c9087a2f523adf045 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:21:10.291563 21193 log.cc:927] Time spent T 8c11074579b046d99e585c2818e18a2c P cb82cc7bd6db46d980c75237a1402860: Append to log took a long time: real 0.424s	user 0.000s	sys 0.001s
W20260709 13:21:10.290105 21186 log.cc:927] Time spent T b6f07842d3aa47858df5665874c86f3a P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 0.609s	user 0.000s	sys 0.000s
I20260709 13:21:10.602656 16144 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40844 (request call id 2497) took 1624 ms. Trace:
W20260709 13:21:10.616397 16295 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.801s	user 0.000s	sys 0.011s
W20260709 13:21:10.638012 16295 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.958s	user 0.000s	sys 0.011s
I20260709 13:21:10.637857 16144 rpcz_store.cc:276] 0709 13:21:08.978440 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:08.979324 (+   884us) service_pool.cc:224] Handling call
0709 13:21:09.009317 (+ 29993us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:21:09.303457 (+294140us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:10.602295 (+1298838us) spinlock_profiling.cc:243] Waited 276 ms on lock 0x7b540012fa54. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:21:10.602570 (+   275us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":635495168}
I20260709 13:21:10.023420 21204 raft_consensus.cc:697] T 8638ee363273426db1393dd81e5c7b72 P cb82cc7bd6db46d980c75237a1402860 [term 15 LEADER]: Becoming Leader. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
W20260709 13:21:10.590885 16264 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.717s	user 0.017s	sys 0.034s
I20260709 13:21:10.326326 20961 leader_election.cc:290] T 84f2da98864b4425bf7c0055951fdd4b P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:10.615769 16265 leader_election.cc:304] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:21:10.318289 20661 raft_consensus.cc:493] T 5a959dbecc2648eb85d1cc550f5a3245 P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:21:10.546787 20895 consensus_meta.cc:287] T d16dfad0c3b540e5b4b86492ddf87208 P 90f3da57796b488c9087a2f523adf045: Time spent flushing consensus metadata: real 1.466s	user 0.012s	sys 0.000s
I20260709 13:21:10.320079 21206 raft_consensus.cc:2749] T 26354361e9b044cea1c2b86c4882fe92 P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:21:10.178457 21159 leader_election.cc:290] T 02838c86531d4f98bf58d1c22b8baf77 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:10.327951 21191 leader_election.cc:290] T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:21:10.586202 21060 raft_consensus.cc:3060] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:21:10.618391 16291 rpcz_store.cc:276] 0709 13:21:09.240796 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:09.240911 (+   115us) service_pool.cc:224] Handling call
0709 13:21:09.280589 (+ 39678us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:21:09.280867 (+   278us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:21:09.608604 (+327737us) log.cc:844] Serialized 14 byte log entry
0709 13:21:10.591778 (+983174us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:21:10.591849 (+    71us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:21:10.591906 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:10.591988 (+    82us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:21:10.592129 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":12968,"prepare.queue_time_us":563872,"prepare.run_cpu_time_us":31,"prepare.run_wall_time_us":31,"raft.queue_time_us":319060,"raft.run_cpu_time_us":87,"raft.run_wall_time_us":88,"thread_start_us":1273923,"threads_started":3,"wal-append.queue_time_us":238275}
I20260709 13:21:10.670109 16144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d4ab870a05347868956aa8b841b2981" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:10.328189 20757 leader_election.cc:290] T 9c61a39fd4f84424b129c1a6f9ba404d P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:10.291602 20889 leader_election.cc:290] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:10.336892 21189 raft_consensus.cc:493] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:21:10.691852 16790 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.878s	user 0.042s	sys 0.090s
W20260709 13:21:10.692387 16790 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.929s	user 0.042s	sys 0.090s
I20260709 13:21:10.308255 21174 leader_election.cc:290] T 84f2da98864b4425bf7c0055951fdd4b P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:10.611989 16266 leader_election.cc:304] T 31969392ccdb43b491911a319e8bc7e1 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
I20260709 13:21:09.937238 21180 raft_consensus.cc:515] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:10.613003 21211 raft_consensus.cc:2749] T 2794af5b99984efebdee564035841cb0 P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:21:10.622956 20923 raft_consensus.cc:3060] T 66f75124215f4ca6bef831e174a792d8 P cb82cc7bd6db46d980c75237a1402860 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:21:10.694379 16266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 82469.3us
I20260709 13:21:10.696070 16266 leader_election.cc:304] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860
I20260709 13:21:10.654954 16265 leader_election.cc:304] T 8d4ab870a05347868956aa8b841b2981 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 647a343a542d4a139fe106c19095129b
I20260709 13:21:10.623451 21150 raft_consensus.cc:515] T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:10.627008 21017 raft_consensus.cc:493] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:10.700477 21207 raft_consensus.cc:493] T ada5a004ba7c46819051e3ccec71ffdb P cb82cc7bd6db46d980c75237a1402860 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:10.701085 21017 raft_consensus.cc:515] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:10.701301 21207 raft_consensus.cc:515] T ada5a004ba7c46819051e3ccec71ffdb P cb82cc7bd6db46d980c75237a1402860 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:10.709659 21211 raft_consensus.cc:2804] T 8d4ab870a05347868956aa8b841b2981 P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:21:10.710372 21211 raft_consensus.cc:697] T 8d4ab870a05347868956aa8b841b2981 P 90f3da57796b488c9087a2f523adf045 [term 12 LEADER]: Becoming Leader. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:21:10.711410 21211 consensus_queue.cc:237] T 8d4ab870a05347868956aa8b841b2981 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:10.696017 21209 raft_consensus.cc:2749] T 31969392ccdb43b491911a319e8bc7e1 P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:21:10.660333 16219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9944d5e4b8c4109af9955c4f08c9095" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:10.717358 16219 raft_consensus.cc:2468] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 14.
W20260709 13:21:10.717859 16219 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33050 (request call id 849) took 1416 ms (client timeout 1791 ms). Trace:
W20260709 13:21:10.718218 16219 rpcz_store.cc:269] 0709 13:21:09.301663 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:10.633685 (+1332022us) service_pool.cc:224] Handling call
0709 13:21:10.717802 (+ 84117us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:21:10.147452 21057 consensus_queue.cc:237] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:10.621999 21198 raft_consensus.cc:515] T 829b4ad8c5a54eababeea1181b960d4b P cb82cc7bd6db46d980c75237a1402860 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:10.661880 20661 raft_consensus.cc:515] T 5a959dbecc2648eb85d1cc550f5a3245 P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:10.662665 20895 raft_consensus.cc:515] T d16dfad0c3b540e5b4b86492ddf87208 P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:10.624697 20667 raft_consensus.cc:515] T 19baa7bc367844c288bca8f81439c413 P cb82cc7bd6db46d980c75237a1402860 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:10.619412 21194 raft_consensus.cc:515] T 8638ee363273426db1393dd81e5c7b72 P 90f3da57796b488c9087a2f523adf045 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:10.630632 20797 raft_consensus.cc:515] T e4dc6b0541ca431ebc757beac00521f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:10.755574 16291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8638ee363273426db1393dd81e5c7b72" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:21:10.757035 21194 leader_election.cc:290] T 8638ee363273426db1393dd81e5c7b72 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:10.757520 21017 leader_election.cc:290] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:10.757968 16296 raft_consensus.cc:3060] T 8638ee363273426db1393dd81e5c7b72 P 90f3da57796b488c9087a2f523adf045 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:21:10.757105 21204 consensus_queue.cc:237] T 8638ee363273426db1393dd81e5c7b72 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:10.761693 21198 leader_election.cc:290] T 829b4ad8c5a54eababeea1181b960d4b P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
W20260709 13:21:10.669588 21161 consensus_meta.cc:287] T ee9ed67006144797b1de6ea8c2013a68 P 1ca2725d290e47d68b1a4b609c9816ac: Time spent flushing consensus metadata: real 1.601s	user 0.013s	sys 0.011s
I20260709 13:21:10.672228 21212 raft_consensus.cc:2749] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
W20260709 13:21:10.692570 20991 consensus_meta.cc:287] T 8d4ab870a05347868956aa8b841b2981 P 647a343a542d4a139fe106c19095129b: Time spent flushing consensus metadata: real 1.107s	user 0.009s	sys 0.003s
I20260709 13:21:10.699729 20641 raft_consensus.cc:493] T 2794af5b99984efebdee564035841cb0 P cb82cc7bd6db46d980c75237a1402860 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
W20260709 13:21:10.719291 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 64484.8us
I20260709 13:21:10.719628 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74ca8140f35c4ae29e8a664f6bdfc829" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:10.776535 16265 leader_election.cc:304] T d2ab0ac8347641e6955e9bce7d90c1a2 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860
W20260709 13:21:10.777707 20998 consensus_meta.cc:287] T 8a878999c0024d2abe517484180290fc P 1ca2725d290e47d68b1a4b609c9816ac: Time spent flushing consensus metadata: real 0.905s	user 0.010s	sys 0.001s
I20260709 13:21:10.777796 16265 leader_election.cc:304] T 8158bbd17b994d2eb96f560bd0f1b88d P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:21:10.778347 21032 raft_consensus.cc:493] T 8a878999c0024d2abe517484180290fc P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:10.778957 21032 raft_consensus.cc:515] T 8a878999c0024d2abe517484180290fc P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:10.779847 20895 leader_election.cc:290] T d16dfad0c3b540e5b4b86492ddf87208 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 13 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:21:10.780876 16226 raft_consensus.cc:1275] T 8a878999c0024d2abe517484180290fc P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 7. (term mismatch)
I20260709 13:21:10.781303 16226 pending_rounds.cc:85] T 8a878999c0024d2abe517484180290fc P 1ca2725d290e47d68b1a4b609c9816ac: Aborting all ops after (but not including) 6
I20260709 13:21:10.781553 16226 pending_rounds.cc:107] T 8a878999c0024d2abe517484180290fc P 1ca2725d290e47d68b1a4b609c9816ac: Aborting uncommitted NO_OP operation due to leader change: 11.7
I20260709 13:21:10.781814 16226 raft_consensus.cc:2889] T 8a878999c0024d2abe517484180290fc P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:21:10.721982 20832 raft_consensus.cc:2749] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:21:10.782966 20895 raft_consensus.cc:2749] T d2ab0ac8347641e6955e9bce7d90c1a2 P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:21:10.783339 21203 raft_consensus.cc:2804] T 8158bbd17b994d2eb96f560bd0f1b88d P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:21:10.786046 21203 raft_consensus.cc:697] T 8158bbd17b994d2eb96f560bd0f1b88d P 90f3da57796b488c9087a2f523adf045 [term 13 LEADER]: Becoming Leader. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:21:10.719828 21216 raft_consensus.cc:493] T 434bff3082d141838068f323bdb0d1ed P cb82cc7bd6db46d980c75237a1402860 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:21:10.783958 21032 leader_election.cc:290] T 8a878999c0024d2abe517484180290fc P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:10.734566 16819 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49906 (request call id 992) took 1711 ms. Trace:
I20260709 13:21:10.736567 16824 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49906 (request call id 989) took 2561 ms. Trace:
I20260709 13:21:10.745162 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1df93779209748b9860f6d893dd3de53" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:10.789319 16824 rpcz_store.cc:276] 0709 13:21:08.175501 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:08.175884 (+   383us) service_pool.cc:224] Handling call
0709 13:21:08.184911 (+  9027us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:21:09.186855 (+1001944us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:10.736286 (+1549431us) spinlock_profiling.cc:243] Waited 1 s on lock 0x7b54007c01d4. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:21:10.736490 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2302779264}
I20260709 13:21:10.746013 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5ad4eb281564936a59f41e387a47156" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:10.748441 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a959dbecc2648eb85d1cc550f5a3245" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:21:10.750820 16294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cc959fb7f704fba9009c0595467d074" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:10.757079 20797 leader_election.cc:290] T e4dc6b0541ca431ebc757beac00521f1 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:10.630365 21046 raft_consensus.cc:515] T c0eab25ce4c64e018a041beeda2867db P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:10.793021 16298 raft_consensus.cc:1275] T 8c11074579b046d99e585c2818e18a2c P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 12 index: 12. Preceding OpId from leader: term: 13 index: 12. (term mismatch)
I20260709 13:21:10.790961 16299 raft_consensus.cc:3060] T 5a959dbecc2648eb85d1cc550f5a3245 P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:21:10.795358 16298 pending_rounds.cc:85] T 8c11074579b046d99e585c2818e18a2c P 90f3da57796b488c9087a2f523adf045: Aborting all ops after (but not including) 11
I20260709 13:21:10.786939 21216 raft_consensus.cc:515] T 434bff3082d141838068f323bdb0d1ed P cb82cc7bd6db46d980c75237a1402860 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:10.773747 16148 raft_consensus.cc:2393] T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 13: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
I20260709 13:21:10.774523 16219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada5a004ba7c46819051e3ccec71ffdb" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:21:10.774574 16144 raft_consensus.cc:2468] T 8d4ab870a05347868956aa8b841b2981 P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 12.
I20260709 13:21:10.788609 16819 rpcz_store.cc:276] 0709 13:21:09.022880 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:09.026591 (+  3711us) service_pool.cc:224] Handling call
0709 13:21:09.041283 (+ 14692us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:21:09.041716 (+   433us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:21:09.041758 (+    42us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:21:09.044954 (+  3196us) log.cc:844] Serialized 52 byte log entry
0709 13:21:09.250548 (+205594us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:21:09.252223 (+  1675us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:09.252313 (+    90us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:21:10.734242 (+1481929us) raft_consensus.cc:1685] finished
0709 13:21:10.734279 (+    37us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:21:10.734485 (+   206us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":357,"raft.run_cpu_time_us":157,"raft.run_wall_time_us":34023,"thread_start_us":150350,"threads_started":1,"wal-append.queue_time_us":494682}
I20260709 13:21:10.799336 16219 raft_consensus.cc:3060] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:21:10.796471 16298 pending_rounds.cc:107] T 8c11074579b046d99e585c2818e18a2c P 90f3da57796b488c9087a2f523adf045: Aborting uncommitted NO_OP operation due to leader change: 12.12
I20260709 13:21:10.800319 16298 raft_consensus.cc:2889] T 8c11074579b046d99e585c2818e18a2c P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:21:10.778229 20661 leader_election.cc:290] T 5a959dbecc2648eb85d1cc550f5a3245 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:21:10.802373 16294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66ae176961b74813a58bc15c9c4c6f2a" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:10.790326 21155 raft_consensus.cc:493] T 3a7ca61592e848ea9b0fb81a2ce33cdc P cb82cc7bd6db46d980c75237a1402860 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:21:10.769569 16296 raft_consensus.cc:2468] T 8638ee363273426db1393dd81e5c7b72 P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 15.
I20260709 13:21:10.805660 16294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02838c86531d4f98bf58d1c22b8baf77" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:10.789145 21198 raft_consensus.cc:493] T b87147731a68400e8cce9b879daa7c5d P cb82cc7bd6db46d980c75237a1402860 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:21:10.806102 16265 connection.cc:256] RPC call timeout handler was delayed by -1.1489849079998748s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1723s for a response
I20260709 13:21:10.812361 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c2095a5f4ad49408dbc7f69c606120d" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:10.813098 16298 raft_consensus.cc:2468] T 7c2095a5f4ad49408dbc7f69c606120d P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 9.
I20260709 13:21:10.798508 21207 leader_election.cc:290] T ada5a004ba7c46819051e3ccec71ffdb P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:10.814410 16299 raft_consensus.cc:2468] T 5a959dbecc2648eb85d1cc550f5a3245 P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 12.
I20260709 13:21:10.795217 20667 leader_election.cc:290] T 19baa7bc367844c288bca8f81439c413 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 15 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:10.816797 16294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a96373bb0e14dce8a266b2fc6c58507" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:10.790625 20641 raft_consensus.cc:515] T 2794af5b99984efebdee564035841cb0 P cb82cc7bd6db46d980c75237a1402860 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:10.820027 21046 leader_election.cc:290] T c0eab25ce4c64e018a041beeda2867db P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:10.821542 20641 leader_election.cc:290] T 2794af5b99984efebdee564035841cb0 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:10.773142 21161 raft_consensus.cc:515] T ee9ed67006144797b1de6ea8c2013a68 P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:10.824062 16217 raft_consensus.cc:1240] T ee9ed67006144797b1de6ea8c2013a68 P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 11. Current term is 12. Ops: []
I20260709 13:21:10.827606 16219 raft_consensus.cc:2468] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 14.
W20260709 13:21:10.828372 16219 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57600 (request call id 1622) took 1594 ms (client timeout 2050 ms). Trace:
W20260709 13:21:10.828845 16219 rpcz_store.cc:269] 0709 13:21:09.233692 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:10.737731 (+1504039us) service_pool.cc:224] Handling call
0709 13:21:10.828295 (+ 90564us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:21:10.832932 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f5f89186204242bf377f7c42b2e2da" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 15 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:10.835878 21161 leader_election.cc:290] T ee9ed67006144797b1de6ea8c2013a68 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 12 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:10.838650 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cc959fb7f704fba9009c0595467d074" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:21:10.841347 16299 raft_consensus.cc:2410] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac for term 15 because replica has last-logged OpId of term: 14 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 7.
W20260709 13:21:10.841691 21147 log.cc:927] Time spent T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 2.289s	user 0.030s	sys 0.058s
I20260709 13:21:10.842191 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1df93779209748b9860f6d893dd3de53" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:21:10.787009 21203 consensus_queue.cc:237] T 8158bbd17b994d2eb96f560bd0f1b88d P 90f3da57796b488c9087a2f523adf045 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:10.696952 21189 raft_consensus.cc:515] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } }
I20260709 13:21:10.847868 16219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "829b4ad8c5a54eababeea1181b960d4b" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:10.849964 16219 raft_consensus.cc:2468] T 829b4ad8c5a54eababeea1181b960d4b P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 13.
W20260709 13:21:10.851557 16219 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57604 (request call id 2752) took 1629 ms (client timeout 2098 ms). Trace:
I20260709 13:21:10.816144 21211 raft_consensus.cc:493] T 5a96373bb0e14dce8a266b2fc6c58507 P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
W20260709 13:21:10.852314 16219 rpcz_store.cc:269] 0709 13:21:09.222286 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:10.847610 (+1625324us) service_pool.cc:224] Handling call
0709 13:21:10.851475 (+  3865us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:21:10.859942 16821 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49896 (request call id 1241) took 2662 ms. Trace:
I20260709 13:21:10.860579 16821 rpcz_store.cc:276] 0709 13:21:08.197309 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:08.199149 (+  1840us) service_pool.cc:224] Handling call
0709 13:21:08.199453 (+   304us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:21:08.263525 (+ 64072us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:21:08.263553 (+    28us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:21:08.270539 (+  6986us) log.cc:844] Serialized 52 byte log entry
0709 13:21:08.561786 (+291247us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:21:09.127471 (+565685us) log.cc:844] Serialized 14 byte log entry
0709 13:21:09.127848 (+   377us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:10.859394 (+1731546us) spinlock_profiling.cc:243] Waited 56.1 ms on lock 0x7b54007553d4. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:21:10.859468 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:21:10.859537 (+    69us) raft_consensus.cc:1685] finished
0709 13:21:10.859657 (+   120us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:21:10.859861 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7543,"prepare.queue_time_us":623744,"prepare.run_cpu_time_us":30,"prepare.run_wall_time_us":31,"raft.queue_time_us":21054,"raft.run_cpu_time_us":293,"raft.run_wall_time_us":33085,"spinlock_wait_cycles":129068800,"thread_start_us":744563,"threads_started":2,"wal-append.queue_time_us":471509,"wal-append.run_cpu_time_us":3182,"wal-append.run_wall_time_us":1415598}
I20260709 13:21:10.864167 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d4ab870a05347868956aa8b841b2981" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:10.814296 21198 raft_consensus.cc:515] T b87147731a68400e8cce9b879daa7c5d P cb82cc7bd6db46d980c75237a1402860 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:10.871050 16219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ab0ac8347641e6955e9bce7d90c1a2" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:10.871770 16219 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33050 (request call id 851) took 1196 ms. Trace:
I20260709 13:21:10.872021 16219 rpcz_store.cc:276] 0709 13:21:09.675687 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:10.853094 (+1177407us) service_pool.cc:224] Handling call
0709 13:21:10.871723 (+ 18629us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:21:10.811437 21155 raft_consensus.cc:515] T 3a7ca61592e848ea9b0fb81a2ce33cdc P cb82cc7bd6db46d980c75237a1402860 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:10.884977 21198 leader_election.cc:290] T b87147731a68400e8cce9b879daa7c5d P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:10.865299 21211 raft_consensus.cc:515] T 5a96373bb0e14dce8a266b2fc6c58507 P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:10.892889 16817 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49896 (request call id 1242) took 2514 ms. Trace:
I20260709 13:21:10.893373 16817 rpcz_store.cc:276] 0709 13:21:08.378321 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:08.946656 (+568335us) service_pool.cc:224] Handling call
0709 13:21:08.947148 (+   492us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 13:21:09.064109 (+116961us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:21:09.064150 (+    41us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 13:21:09.065015 (+   865us) log.cc:844] Serialized 78 byte log entry
0709 13:21:09.212493 (+147478us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:21:09.212526 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:10.884553 (+1672027us) spinlock_profiling.cc:243] Waited 16.4 ms on lock 0x7b48000cd758. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b5a88aa1 00007f69b5a7fb37 00007f69b5a7f9ca 00007f69b5b7726f 00007f69b5b76f37 00007f69b5b8b55c 00007f69b5b8b508 00007f69b5b8b488 00007f69b5b8b443 00007f69b5b8a6ec 00007f69b4ab4a96
0709 13:21:10.892545 (+  7992us) spinlock_profiling.cc:243] Waited 116 ms on lock 0x7b54005320d4. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:21:10.892600 (+    55us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:21:10.892651 (+    51us) raft_consensus.cc:1685] finished
0709 13:21:10.892663 (+    12us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:21:10.892821 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":3846,"raft.run_cpu_time_us":53657,"raft.run_wall_time_us":1816795,"spinlock_wait_cycles":305310848,"thread_start_us":147076,"threads_started":1,"wal-append.queue_time_us":265929,"wal-append.run_cpu_time_us":1951,"wal-append.run_wall_time_us":1001982}
I20260709 13:21:10.895567 16294 raft_consensus.cc:2468] T 5a96373bb0e14dce8a266b2fc6c58507 P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 12.
I20260709 13:21:10.895704 21129 leader_election.cc:290] T 23f82224f4564fea93572e8206bad1fe P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 12 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:10.900633 16219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af57996a8be84d398f0f023e8d08ef77" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:21:10.901749 16219 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33050 (request call id 850) took 1592 ms. Trace:
I20260709 13:21:10.902153 16219 rpcz_store.cc:276] 0709 13:21:09.309603 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:10.872523 (+1562920us) service_pool.cc:224] Handling call
0709 13:21:10.901669 (+ 29146us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:21:10.908449 16296 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40474 (request call id 817) took 1039 ms. Trace:
I20260709 13:21:10.909021 16296 rpcz_store.cc:276] 0709 13:21:09.869107 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:09.870949 (+  1842us) service_pool.cc:224] Handling call
0709 13:21:10.908204 (+1037255us) spinlock_profiling.cc:243] Waited 783 ms on lock 0x7b540010ed54. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:21:10.908369 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1801833216}
I20260709 13:21:10.912891 16146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e65676b21b748728ad39668cfe722ec" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:21:10.913748 16146 raft_consensus.cc:3060] T 9e65676b21b748728ad39668cfe722ec P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:21:10.919003 16147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af57996a8be84d398f0f023e8d08ef77" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:21:10.920002 16147 raft_consensus.cc:3060] T af57996a8be84d398f0f023e8d08ef77 P 647a343a542d4a139fe106c19095129b [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:21:10.922593 16217 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33050 (request call id 842) took 2008 ms. Trace:
I20260709 13:21:10.893913 21150 leader_election.cc:290] T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 13 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
W20260709 13:21:10.924410 16221 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57604 (request call id 2739) took 2577 ms (client timeout 2041 ms). Trace:
W20260709 13:21:10.932315 16221 rpcz_store.cc:269] 0709 13:21:08.347101 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:08.347889 (+   788us) service_pool.cc:224] Handling call
0709 13:21:10.924140 (+2576251us) spinlock_profiling.cc:243] Waited 455 ms on lock 0x7b5400245254. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:21:10.924323 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1045504896}
I20260709 13:21:10.934657 16221 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57604 (request call id 2751) took 1713 ms. Trace:
I20260709 13:21:10.920025 16218 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57604 (request call id 2733) took 2660 ms. Trace:
I20260709 13:21:10.930003 16300 raft_consensus.cc:1240] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 12. Current term is 13. Ops: []
I20260709 13:21:10.936215 16218 rpcz_store.cc:276] 0709 13:21:08.268593 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:08.268983 (+   390us) service_pool.cc:224] Handling call
0709 13:21:08.282575 (+ 13592us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:21:08.561365 (+278790us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:10.919658 (+2358293us) spinlock_profiling.cc:243] Waited 278 ms on lock 0x7b5400138654. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:21:10.919912 (+   254us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":638669696}
I20260709 13:21:10.936102 16221 rpcz_store.cc:276] 0709 13:21:09.221659 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:10.933634 (+1711975us) service_pool.cc:224] Handling call
0709 13:21:10.934000 (+   366us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:21:10.934237 (+   237us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:21:10.934278 (+    41us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:21:10.934334 (+    56us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:21:10.934381 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:10.934456 (+    75us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:21:10.934587 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:21:10.939302 16221 raft_consensus.cc:1240] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 11. Current term is 12. Ops: [11.10-11.10]
I20260709 13:21:10.935303 16296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fffab26427e24d398e2695aad40ca134" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:21:10.940192 16221 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57600 (request call id 1619) took 1707 ms. Trace:
I20260709 13:21:10.940938 16221 rpcz_store.cc:276] 0709 13:21:09.232686 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:10.938513 (+1705827us) service_pool.cc:224] Handling call
0709 13:21:10.938941 (+   428us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:21:10.939834 (+   893us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:10.940120 (+   286us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:21:10.943356 16221 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57600 (request call id 1623) took 1709 ms. Trace:
I20260709 13:21:10.923130 16217 rpcz_store.cc:276] 0709 13:21:08.914669 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:09.153524 (+238855us) service_pool.cc:224] Handling call
0709 13:21:09.166500 (+ 12976us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:21:10.824433 (+1657933us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:10.922271 (+ 97838us) spinlock_profiling.cc:243] Waited 1.66 s on lock 0x7b54002b4654. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:21:10.922519 (+   248us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3811668224}
I20260709 13:21:10.943842 16221 rpcz_store.cc:276] 0709 13:21:09.237138 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:10.942195 (+1705057us) service_pool.cc:224] Handling call
0709 13:21:10.942653 (+   458us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:21:10.942880 (+   227us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:21:10.942921 (+    41us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:21:10.942983 (+    62us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:21:10.943042 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:10.943119 (+    77us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:21:10.943287 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:21:10.951753 16219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5d59407eb545b895d8d922e0dd69fd" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 16 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:10.952993 16219 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33050 (request call id 847) took 1656 ms. Trace:
I20260709 13:21:10.953485 16219 rpcz_store.cc:276] 0709 13:21:09.297056 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:10.903067 (+1606011us) service_pool.cc:224] Handling call
0709 13:21:10.952910 (+ 49843us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:21:10.957744 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 77317.8us
I20260709 13:21:10.957978 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
W20260709 13:21:10.959951 16790 leader_election.cc:336] T af57996a8be84d398f0f023e8d08ef77 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943): Timed out: RequestConsensusVote RPC to 127.14.177.130:35943 timed out after 1.714s (SENT)
I20260709 13:21:10.966176 16146 raft_consensus.cc:2468] T 9e65676b21b748728ad39668cfe722ec P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 13.
W20260709 13:21:10.966706 16146 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46224 (request call id 752) took 1706 ms (client timeout 2120 ms). Trace:
W20260709 13:21:10.967010 16146 rpcz_store.cc:269] 0709 13:21:09.260129 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:10.912633 (+1652504us) service_pool.cc:224] Handling call
0709 13:21:10.966658 (+ 54025us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:21:10.972846 21060 raft_consensus.cc:515] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:10.973878 16817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:10.974535 16817 raft_consensus.cc:2468] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 13.
I20260709 13:21:10.977666 16147 raft_consensus.cc:2468] T af57996a8be84d398f0f023e8d08ef77 P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 15.
W20260709 13:21:10.980903 16147 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46224 (request call id 753) took 1691 ms (client timeout 2131 ms). Trace:
W20260709 13:21:10.981357 16147 rpcz_store.cc:269] 0709 13:21:09.289858 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:10.918583 (+1628725us) service_pool.cc:224] Handling call
0709 13:21:10.980677 (+ 62094us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:21:10.988124 21034 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.051s	user 0.015s	sys 0.025s
I20260709 13:21:10.989840 16146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18ad7e3af5504571a69d10c00750a01c" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:10.991021 16147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:10.991242 16146 raft_consensus.cc:2468] T 18ad7e3af5504571a69d10c00750a01c P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 11.
I20260709 13:21:10.991897 16147 raft_consensus.cc:2468] T ecc1e363230043fe9fef390f0b979f20 P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 13.
I20260709 13:21:10.992192 16146 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40844 (request call id 2512) took 1405 ms. Trace:
W20260709 13:21:10.992851 16790 leader_election.cc:336] T c9944d5e4b8c4109af9955c4f08c9095 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943): Timed out: RequestConsensusVote RPC to 127.14.177.130:35943 timed out after 1.791s (SENT)
I20260709 13:21:10.992925 16146 rpcz_store.cc:276] 0709 13:21:09.586850 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:10.989470 (+1402620us) service_pool.cc:224] Handling call
0709 13:21:10.992114 (+  2644us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:21:10.993757 16790 leader_election.cc:304] T c9944d5e4b8c4109af9955c4f08c9095 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:21:10.995513 16147 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40844 (request call id 2511) took 1442 ms. Trace:
I20260709 13:21:10.995740 16296 raft_consensus.cc:1240] T d16dfad0c3b540e5b4b86492ddf87208 P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 12. Current term is 13. Ops: []
I20260709 13:21:10.996138 16217 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33050 (request call id 846) took 1706 ms. Trace:
I20260709 13:21:10.996088 16147 rpcz_store.cc:276] 0709 13:21:09.553314 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:10.990733 (+1437419us) service_pool.cc:224] Handling call
0709 13:21:10.992560 (+  1827us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:21:10.928057 21180 leader_election.cc:290] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:10.997584 16217 rpcz_store.cc:276] 0709 13:21:09.289815 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:10.945369 (+1655554us) service_pool.cc:224] Handling call
0709 13:21:10.995644 (+ 50275us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:21:10.995830 (+   186us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:21:10.995858 (+    28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:21:10.995899 (+    41us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:21:10.995929 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:10.995985 (+    56us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:21:10.996089 (+   104us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:21:11.002071 16145 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46224 (request call id 744) took 2612 ms. Trace:
I20260709 13:21:11.003773 16296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18ad7e3af5504571a69d10c00750a01c" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
W20260709 13:21:10.988683 21034 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.055s	user 0.015s	sys 0.025s
I20260709 13:21:11.001405 16146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5d59407eb545b895d8d922e0dd69fd" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 16 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:11.008893 16146 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46224 (request call id 750) took 1792 ms. Trace:
I20260709 13:21:11.010031 16146 rpcz_store.cc:276] 0709 13:21:09.216956 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:11.001077 (+1784121us) service_pool.cc:224] Handling call
0709 13:21:11.008811 (+  7734us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:21:11.002849 16145 rpcz_store.cc:276] 0709 13:21:08.378321 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:08.380448 (+  2127us) service_pool.cc:224] Handling call
0709 13:21:08.406523 (+ 26075us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:21:08.406711 (+   188us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:21:08.407000 (+   289us) log.cc:844] Serialized 14 byte log entry
0709 13:21:10.724980 (+2317980us) spinlock_profiling.cc:243] Waited 4.12 ms on lock 0x7b1800055660. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b693ade1 00007f69b69475b1 00007f69b07074bc 00007f69ada5548b 00007f69b68afab9 00007f69ada7169b 00007f69ada6f93f 00007f69ada6f455 00007f69b0b57060 00007f69b0b69716 00007f69b0b7a287
0709 13:21:10.807907 (+ 82927us) log.cc:844] Serialized 14 byte log entry
0709 13:21:10.989390 (+181483us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:21:10.989469 (+    79us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:21:10.990153 (+   684us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:10.990253 (+   100us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:21:10.990396 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":111280,"prepare.run_cpu_time_us":95,"prepare.run_wall_time_us":97,"raft.queue_time_us":125192,"raft.run_cpu_time_us":241,"raft.run_wall_time_us":7438,"spinlock_wait_cycles":9469312,"thread_start_us":813058,"threads_started":4,"wal-append.queue_time_us":580602,"wal-append.run_cpu_time_us":1491,"wal-append.run_wall_time_us":1150622}
I20260709 13:21:11.012765 16146 raft_consensus.cc:1240] T 746aabf856ab401c9ef5e859371e7b67 P 647a343a542d4a139fe106c19095129b [term 12 LEADER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 11. Current term is 12. Ops: []
I20260709 13:21:11.013528 16146 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46224 (request call id 748) took 1820 ms. Trace:
I20260709 13:21:11.013887 16146 rpcz_store.cc:276] 0709 13:21:09.193285 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:11.012185 (+1818900us) service_pool.cc:224] Handling call
0709 13:21:11.012532 (+   347us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:21:11.013259 (+   727us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:11.013469 (+   210us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:21:11.017874 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74ca8140f35c4ae29e8a664f6bdfc829" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:11.020145 16296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8638ee363273426db1393dd81e5c7b72" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:21:11.021027 16296 raft_consensus.cc:2368] T 8638ee363273426db1393dd81e5c7b72 P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac for earlier term 14. Current term is 15.
I20260709 13:21:11.020557 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f2da98864b4425bf7c0055951fdd4b" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:11.021790 16300 raft_consensus.cc:2468] T 84f2da98864b4425bf7c0055951fdd4b P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 10.
I20260709 13:21:11.022087 16296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66ae176961b74813a58bc15c9c4c6f2a" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:11.020954 16292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cc959fb7f704fba9009c0595467d074" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:11.023258 16820 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49906 (request call id 996) took 1798 ms. Trace:
I20260709 13:21:11.023545 16292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5ad4eb281564936a59f41e387a47156" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:10.999291 21228 raft_consensus.cc:2749] T c9944d5e4b8c4109af9955c4f08c9095 P cb82cc7bd6db46d980c75237a1402860 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:21:11.035275 16292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a959dbecc2648eb85d1cc550f5a3245" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:21:11.036864 21211 leader_election.cc:290] T 5a96373bb0e14dce8a266b2fc6c58507 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:21:11.037117 16292 raft_consensus.cc:1240] T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 14. Current term is 15. Ops: []
I20260709 13:21:11.007889 21034 raft_consensus.cc:697] T 8cc959fb7f704fba9009c0595467d074 P 1ca2725d290e47d68b1a4b609c9816ac [term 13 LEADER]: Becoming Leader. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:21:11.024096 16299 raft_consensus.cc:1275] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:21:11.025647 16226 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57604 (request call id 2711) took 4666 ms. Trace:
I20260709 13:21:11.044504 16226 rpcz_store.cc:276] 0709 13:21:06.351884 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:06.351988 (+   104us) service_pool.cc:224] Handling call
0709 13:21:06.411321 (+ 59333us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:21:10.782709 (+4371388us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:11.018189 (+235480us) spinlock_profiling.cc:243] Waited 4.37 s on lock 0x7b5400394754. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:21:11.018425 (+   236us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":10048235904}
I20260709 13:21:11.055476 16291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a96373bb0e14dce8a266b2fc6c58507" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:11.063776 16819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee9ed67006144797b1de6ea8c2013a68" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:21:11.065268 16824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a378d7f796f94615b3b347119a9880fd" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:11.066407 16824 raft_consensus.cc:2393] T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 15: Already voted for candidate cb82cc7bd6db46d980c75237a1402860 in this term.
I20260709 13:21:11.067101 16296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4dc6b0541ca431ebc757beac00521f1" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:11.067999 16296 raft_consensus.cc:2468] T e4dc6b0541ca431ebc757beac00521f1 P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 12.
I20260709 13:21:11.069393 16143 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40844 (request call id 2496) took 2095 ms. Trace:
I20260709 13:21:11.023686 16820 rpcz_store.cc:276] 0709 13:21:09.224901 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:09.225030 (+   129us) service_pool.cc:224] Handling call
0709 13:21:09.237025 (+ 11995us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:21:09.274520 (+ 37495us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:21:09.274620 (+   100us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:21:09.274687 (+    67us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:21:09.274737 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:11.023016 (+1748279us) spinlock_profiling.cc:243] Waited 37.2 ms on lock 0x7b54003df254. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:21:11.023066 (+    50us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:21:11.023208 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":85592320}
I20260709 13:21:11.069801 16143 rpcz_store.cc:276] 0709 13:21:08.973638 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:08.976993 (+  3355us) service_pool.cc:224] Handling call
0709 13:21:11.063801 (+2086808us) spinlock_profiling.cc:243] Waited 18.3 ms on lock 0x7b5400427a54. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:21:11.069329 (+  5528us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":42118528}
I20260709 13:21:11.071456 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1df93779209748b9860f6d893dd3de53" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:11.075860 21195 raft_consensus.cc:515] T 3884f864a96d428785895e5463e5728f P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:11.078220 16145 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46224 (request call id 747) took 1884 ms. Trace:
I20260709 13:21:11.078600 16145 rpcz_store.cc:276] 0709 13:21:09.192971 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:11.011980 (+1819009us) service_pool.cc:224] Handling call
0709 13:21:11.015351 (+  3371us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:21:11.015999 (+   648us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:21:11.016034 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:21:11.016508 (+   474us) log.cc:844] Serialized 26 byte log entry
0709 13:21:11.075144 (+ 58636us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:21:11.075202 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:11.075263 (+    61us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:21:11.077398 (+  2135us) raft_consensus.cc:1685] finished
0709 13:21:11.077417 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:21:11.077560 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":129,"raft.run_cpu_time_us":75,"raft.run_wall_time_us":76,"thread_start_us":58250,"threads_started":1,"wal-append.queue_time_us":58755}
I20260709 13:21:11.081393 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8158bbd17b994d2eb96f560bd0f1b88d" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:11.081746 16219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87ac0d1f7e8a419f9bcbc04aee0e067e" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:11.064038 21075 raft_consensus.cc:493] T 19baa7bc367844c288bca8f81439c413 P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:21:11.082389 16219 raft_consensus.cc:2468] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 14.
I20260709 13:21:11.082639 21075 raft_consensus.cc:515] T 19baa7bc367844c288bca8f81439c413 P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:11.083961 16143 raft_consensus.cc:1275] T b87147731a68400e8cce9b879daa7c5d P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 13:21:11.084004 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdbd78acb4cf46298e0de8ba0d130e19" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:11.084502 16143 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40830 (request call id 1638) took 1870 ms. Trace:
I20260709 13:21:11.084738 16143 rpcz_store.cc:276] 0709 13:21:09.214404 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:11.077714 (+1863310us) service_pool.cc:224] Handling call
0709 13:21:11.078024 (+   310us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:21:11.084326 (+  6302us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:11.084462 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:21:11.085348 16226 raft_consensus.cc:1240] T 1df93779209748b9860f6d893dd3de53 P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 12. Current term is 13. Ops: []
I20260709 13:21:11.085764 16143 raft_consensus.cc:1240] T 9e65676b21b748728ad39668cfe722ec P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 12. Current term is 13. Ops: [12.11-12.11]
I20260709 13:21:11.086436 16143 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40830 (request call id 1639) took 1871 ms. Trace:
I20260709 13:21:11.086694 16143 rpcz_store.cc:276] 0709 13:21:09.214882 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:11.085306 (+1870424us) service_pool.cc:224] Handling call
0709 13:21:11.085531 (+   225us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:21:11.086201 (+   670us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:11.086392 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:21:11.063689 21180 raft_consensus.cc:493] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:11.087991 16143 raft_consensus.cc:1275] T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 13:21:11.083423 21060 leader_election.cc:290] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 16 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:11.088825 16143 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40844 (request call id 2510) took 1837 ms. Trace:
I20260709 13:21:11.092008 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9944d5e4b8c4109af9955c4f08c9095" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:11.092898 16299 raft_consensus.cc:2468] T c9944d5e4b8c4109af9955c4f08c9095 P 90f3da57796b488c9087a2f523adf045 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 14.
I20260709 13:21:11.093140 16790 leader_election.cc:304] T 9e65676b21b748728ad39668cfe722ec P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:21:11.092144 16143 rpcz_store.cc:276] 0709 13:21:09.251398 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:11.087469 (+1836071us) service_pool.cc:224] Handling call
0709 13:21:11.087754 (+   285us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:21:11.088447 (+   693us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:11.088633 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:21:11.094084 16790 leader_election.cc:304] T af57996a8be84d398f0f023e8d08ef77 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:21:11.102831 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9944d5e4b8c4109af9955c4f08c9095" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:11.104058 16226 raft_consensus.cc:2410] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 for term 15 because replica has last-logged OpId of term: 13 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 11.
I20260709 13:21:11.104836 16292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1df93779209748b9860f6d893dd3de53" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:21:11.105741 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a378d7f796f94615b3b347119a9880fd" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:11.108495 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0eab25ce4c64e018a041beeda2867db" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:11.110606 16291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a878999c0024d2abe517484180290fc" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:11.085906 21189 leader_election.cc:290] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:21:11.112264 16291 raft_consensus.cc:2410] T 8a878999c0024d2abe517484180290fc P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac for term 13 because replica has last-logged OpId of term: 12 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 7.
I20260709 13:21:11.115739 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8638ee363273426db1393dd81e5c7b72" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:11.119889 16217 raft_consensus.cc:2393] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 15: Already voted for candidate cb82cc7bd6db46d980c75237a1402860 in this term.
I20260709 13:21:11.121872 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fffab26427e24d398e2695aad40ca134" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:21:11.165707 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c61a39fd4f84424b129c1a6f9ba404d" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
W20260709 13:21:11.166071 16114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:11.107545 21089 raft_consensus.cc:2804] T af57996a8be84d398f0f023e8d08ef77 P cb82cc7bd6db46d980c75237a1402860 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:21:11.200737 16219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a959dbecc2648eb85d1cc550f5a3245" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:11.214973 16145 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40830 (request call id 1642) took 1998 ms. Trace:
I20260709 13:21:11.214913 20236 raft_consensus.cc:493] T 68a6838d82ef4cac83507f906139d90f P 647a343a542d4a139fe106c19095129b [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:11.215745 20236 raft_consensus.cc:515] T 68a6838d82ef4cac83507f906139d90f P 647a343a542d4a139fe106c19095129b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:11.217653 16143 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40844 (request call id 2513) took 1372 ms. Trace:
I20260709 13:21:11.172447 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02838c86531d4f98bf58d1c22b8baf77" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:11.106215 20902 raft_consensus.cc:2804] T 9e65676b21b748728ad39668cfe722ec P cb82cc7bd6db46d980c75237a1402860 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:21:11.106783 21228 raft_consensus.cc:493] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:11.167686 16296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f5f89186204242bf377f7c42b2e2da" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 15 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:11.230310 16296 raft_consensus.cc:2410] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac for term 15 because replica has last-logged OpId of term: 14 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 7.
I20260709 13:21:11.172561 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d16dfad0c3b540e5b4b86492ddf87208" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:21:11.167784 21106 raft_consensus.cc:493] T 5a959dbecc2648eb85d1cc550f5a3245 P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:11.218552 16143 rpcz_store.cc:276] 0709 13:21:09.845649 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:11.164825 (+1319176us) service_pool.cc:224] Handling call
0709 13:21:11.165106 (+   281us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:21:11.214494 (+ 49388us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:21:11.214529 (+    35us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:21:11.214579 (+    50us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:21:11.217289 (+  2710us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:11.217398 (+   109us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:21:11.217575 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":48959,"raft.queue_time_us":49356,"raft.run_cpu_time_us":95,"raft.run_wall_time_us":95}
I20260709 13:21:11.225672 16192 leader_election.cc:304] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:21:11.101019 21180 raft_consensus.cc:515] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:11.171051 21057 consensus_queue.cc:1059] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: INVALID_TERM, Last received: 11.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.058s
I20260709 13:21:11.120883 21195 leader_election.cc:290] T 3884f864a96d428785895e5463e5728f P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 9 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:11.290350 16224 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57604 (request call id 2738) took 2943 ms. Trace:
I20260709 13:21:11.348026 16296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d4ab870a05347868956aa8b841b2981" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:11.236577 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cc959fb7f704fba9009c0595467d074" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:21:11.234613 20236 leader_election.cc:290] T 68a6838d82ef4cac83507f906139d90f P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
W20260709 13:21:11.319646 16149 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46224 (request call id 740) took 3369 ms (client timeout 2096 ms). Trace:
W20260709 13:21:11.350063 16149 rpcz_store.cc:269] 0709 13:21:07.950453 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:07.987834 (+ 37381us) service_pool.cc:224] Handling call
0709 13:21:08.941500 (+953666us) spinlock_profiling.cc:243] Waited 11.9 ms on lock 0x7b5800011198. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69ad8fadb1 00007f69ada3ab71 00007f69ada3aa22 00007f69b6953c09 00007f69b5aadee0 00007f69b5b770ab 00007f69b5b77131 00007f69b5b250a0 00007f69b5b24984 00007f69af09f6d4 00007f69af09f662
0709 13:21:11.319447 (+2377947us) spinlock_profiling.cc:243] Waited 344 ms on lock 0x7b54003d9ad4. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:21:11.319555 (+   108us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":819152128}
I20260709 13:21:11.350433 16143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "829b4ad8c5a54eababeea1181b960d4b" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:11.168951 21211 raft_consensus.cc:493] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:11.191694 21087 raft_consensus.cc:3055] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:21:11.348542 16224 rpcz_store.cc:276] 0709 13:21:08.346789 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:08.551938 (+205149us) service_pool.cc:224] Handling call
0709 13:21:08.552296 (+   358us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:21:08.566176 (+ 13880us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:21:08.566220 (+    44us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:21:08.566282 (+    62us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:21:08.566325 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:11.289012 (+2722687us) spinlock_profiling.cc:243] Waited 13.6 ms on lock 0x7b54002c9054. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:21:11.290119 (+  1107us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:21:11.290272 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":31338880}
I20260709 13:21:11.354259 16295 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48674 (request call id 2830) took 2116 ms. Trace:
W20260709 13:21:11.360872 16192 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 135246us
I20260709 13:21:11.366796 16295 rpcz_store.cc:276] 0709 13:21:09.234589 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:09.235611 (+  1022us) service_pool.cc:224] Handling call
0709 13:21:09.591584 (+355973us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:21:09.679147 (+ 87563us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:21:09.679202 (+    55us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:21:09.679744 (+   542us) log.cc:844] Serialized 26 byte log entry
0709 13:21:10.638835 (+959091us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:21:10.638905 (+    70us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:11.327480 (+688575us) spinlock_profiling.cc:243] Waited 81.6 ms on lock 0x7b54002e0c54. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:21:11.327541 (+    61us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:21:11.327598 (+    57us) raft_consensus.cc:1685] finished
0709 13:21:11.327611 (+    13us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:21:11.351149 (+ 23538us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2827,"raft.queue_time_us":300101,"raft.run_cpu_time_us":55,"raft.run_wall_time_us":55,"spinlock_wait_cycles":187673984,"thread_start_us":958657,"threads_started":1,"wal-append.queue_time_us":1013736}
I20260709 13:21:11.368382 16192 leader_election.cc:304] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
I20260709 13:21:11.351558 16143 raft_consensus.cc:2468] T 829b4ad8c5a54eababeea1181b960d4b P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 13.
I20260709 13:21:11.370038 16143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f82224f4564fea93572e8206bad1fe" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 12 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:21:11.370764 16143 raft_consensus.cc:3060] T 23f82224f4564fea93572e8206bad1fe P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:21:11.219425 21007 raft_consensus.cc:493] T 57a744f5aaf944678a63469cbf514015 P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:21:11.326352 21126 raft_consensus.cc:3060] T 8158bbd17b994d2eb96f560bd0f1b88d P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:21:11.394932 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9944d5e4b8c4109af9955c4f08c9095" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:11.395507 21084 consensus_queue.cc:1048] T 8c11074579b046d99e585c2818e18a2c P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.024s
I20260709 13:21:11.397092 21084 consensus_queue.cc:1059] T af57996a8be84d398f0f023e8d08ef77 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: INVALID_TERM, Last received: 14.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:21:11.334666 21097 raft_consensus.cc:493] T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:11.398239 21097 raft_consensus.cc:515] T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:11.400477 21097 leader_election.cc:290] T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:11.401335 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87147731a68400e8cce9b879daa7c5d" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:11.402029 16819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada5a004ba7c46819051e3ccec71ffdb" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:21:11.402585 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f82224f4564fea93572e8206bad1fe" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 12 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:21:11.402828 16819 raft_consensus.cc:3060] T ada5a004ba7c46819051e3ccec71ffdb P cb82cc7bd6db46d980c75237a1402860 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:21:11.403489 16816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57a744f5aaf944678a63469cbf514015" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:21:11.381949 21069 consensus_queue.cc:1059] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: INVALID_TERM, Last received: 12.9, Next index: 10, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:21:11.383140 21075 leader_election.cc:290] T 19baa7bc367844c288bca8f81439c413 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:11.368728 21211 raft_consensus.cc:515] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:11.231032 21089 raft_consensus.cc:697] T af57996a8be84d398f0f023e8d08ef77 P cb82cc7bd6db46d980c75237a1402860 [term 15 LEADER]: Becoming Leader. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:21:11.149468 21034 consensus_queue.cc:237] T 8cc959fb7f704fba9009c0595467d074 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:11.284860 21163 raft_consensus.cc:493] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:21:11.385160 21007 raft_consensus.cc:515] T 57a744f5aaf944678a63469cbf514015 P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:11.273442 20902 raft_consensus.cc:697] T 9e65676b21b748728ad39668cfe722ec P cb82cc7bd6db46d980c75237a1402860 [term 13 LEADER]: Becoming Leader. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:21:11.386072 21180 leader_election.cc:290] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:11.394033 16143 raft_consensus.cc:2468] T 23f82224f4564fea93572e8206bad1fe P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 12.
I20260709 13:21:11.395761 16297 raft_consensus.cc:2468] T c9944d5e4b8c4109af9955c4f08c9095 P 90f3da57796b488c9087a2f523adf045 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 14.
I20260709 13:21:11.400406 16295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada5a004ba7c46819051e3ccec71ffdb" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:11.402143 21151 raft_consensus.cc:3055] T af57996a8be84d398f0f023e8d08ef77 P 1ca2725d290e47d68b1a4b609c9816ac [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:21:11.381738 21046 raft_consensus.cc:2749] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:21:11.293310 21121 raft_consensus.cc:3055] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:21:11.405840 16816 raft_consensus.cc:1240] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P cb82cc7bd6db46d980c75237a1402860 [term 12 LEADER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 11. Current term is 12. Ops: [11.10-11.10]
I20260709 13:21:11.405876 16818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdbd78acb4cf46298e0de8ba0d130e19" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:11.283747 20991 raft_consensus.cc:493] T fffab26427e24d398e2695aad40ca134 P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:11.357224 21106 raft_consensus.cc:515] T 5a959dbecc2648eb85d1cc550f5a3245 P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:11.359798 21057 consensus_queue.cc:1243] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Peer 647a343a542d4a139fe106c19095129b log is divergent from this leader: its last log entry 12.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 11
I20260709 13:21:11.406104 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:21:11.440900 16149 raft_consensus.cc:3060] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:21:11.416836 16819 raft_consensus.cc:2468] T ada5a004ba7c46819051e3ccec71ffdb P cb82cc7bd6db46d980c75237a1402860 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 14.
I20260709 13:21:11.407950 16790 leader_election.cc:304] T 829b4ad8c5a54eababeea1181b960d4b P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860; no voters: 
W20260709 13:21:11.383986 21239 log.cc:927] Time spent T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 0.068s	user 0.000s	sys 0.000s
I20260709 13:21:11.424763 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434bff3082d141838068f323bdb0d1ed" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:11.425109 16292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7ca61592e848ea9b0fb81a2ce33cdc" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:11.446918 16293 raft_consensus.cc:2468] T 434bff3082d141838068f323bdb0d1ed P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 12.
I20260709 13:21:11.425957 16296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:21:11.373446 21161 raft_consensus.cc:2804] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:21:11.366422 21087 raft_consensus.cc:740] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:21:11.435784 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66ae176961b74813a58bc15c9c4c6f2a" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:21:11.435931 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a9de347c711477e87c582d003f36925" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:11.420763 20967 raft_consensus.cc:493] T 8c11074579b046d99e585c2818e18a2c P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:11.408393 16821 raft_consensus.cc:1240] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860 [term 11 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 10. Current term is 11. Ops: []
I20260709 13:21:11.438212 16143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e9b2bf7011f4ae688a79d7bb5a447b4" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:21:11.419864 20918 raft_consensus.cc:493] T 68a6838d82ef4cac83507f906139d90f P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:21:11.456626 20918 raft_consensus.cc:515] T 68a6838d82ef4cac83507f906139d90f P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:11.439894 16816 raft_consensus.cc:1240] T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860 [term 15 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 14. Current term is 15. Ops: [14.10-14.10]
I20260709 13:21:11.445813 16295 raft_consensus.cc:2393] T ada5a004ba7c46819051e3ccec71ffdb P 90f3da57796b488c9087a2f523adf045 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 in current term 14: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
I20260709 13:21:11.459311 21007 leader_election.cc:290] T 57a744f5aaf944678a63469cbf514015 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:11.444701 16790 leader_election.cc:304] T 23f82224f4564fea93572e8206bad1fe P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:21:11.460502 16295 raft_consensus.cc:1240] T 3884f864a96d428785895e5463e5728f P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 8. Current term is 9. Ops: []
I20260709 13:21:11.291306 21216 leader_election.cc:290] T 434bff3082d141838068f323bdb0d1ed P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:11.449035 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87147731a68400e8cce9b879daa7c5d" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:11.461369 21216 raft_consensus.cc:2804] T 23f82224f4564fea93572e8206bad1fe P cb82cc7bd6db46d980c75237a1402860 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:21:11.461836 21216 raft_consensus.cc:697] T 23f82224f4564fea93572e8206bad1fe P cb82cc7bd6db46d980c75237a1402860 [term 12 LEADER]: Becoming Leader. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:21:11.264540 21228 raft_consensus.cc:515] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:11.452795 16297 raft_consensus.cc:2468] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 15.
I20260709 13:21:11.455538 20967 raft_consensus.cc:515] T 8c11074579b046d99e585c2818e18a2c P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:11.464344 16297 raft_consensus.cc:1275] T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 13:21:11.465787 16818 raft_consensus.cc:2393] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 15: Already voted for candidate cb82cc7bd6db46d980c75237a1402860 in this term.
I20260709 13:21:11.465696 20967 leader_election.cc:290] T 8c11074579b046d99e585c2818e18a2c P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:11.454128 16143 raft_consensus.cc:2393] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 in current term 11: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
I20260709 13:21:11.456233 20902 consensus_queue.cc:237] T 9e65676b21b748728ad39668cfe722ec P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:11.467968 16143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87147731a68400e8cce9b879daa7c5d" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:11.468523 21106 leader_election.cc:290] T 5a959dbecc2648eb85d1cc550f5a3245 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:11.468616 16143 raft_consensus.cc:2468] T b87147731a68400e8cce9b879daa7c5d P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 15.
I20260709 13:21:11.462674 21216 consensus_queue.cc:237] T 23f82224f4564fea93572e8206bad1fe P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:11.469857 16143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434bff3082d141838068f323bdb0d1ed" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:11.472182 21211 leader_election.cc:290] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:21:11.470306 16225 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57604 (request call id 2748) took 2253 ms. Trace:
I20260709 13:21:11.459302 20918 leader_election.cc:290] T 68a6838d82ef4cac83507f906139d90f P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:21:11.472997 16225 rpcz_store.cc:276] 0709 13:21:09.217341 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:10.927576 (+1710235us) service_pool.cc:224] Handling call
0709 13:21:10.928043 (+   467us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 13:21:10.928609 (+   566us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:21:10.928651 (+    42us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 13:21:10.929478 (+   827us) log.cc:844] Serialized 78 byte log entry
0709 13:21:11.086063 (+156585us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:21:11.086096 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:11.086152 (+    56us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:21:11.470090 (+383938us) raft_consensus.cc:1685] finished
0709 13:21:11.470109 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:21:11.470250 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2601,"raft.run_cpu_time_us":121,"raft.run_wall_time_us":1526,"thread_start_us":155975,"threads_started":1,"wal-append.queue_time_us":349982}
I20260709 13:21:11.470292 16790 leader_election.cc:304] T b87147731a68400e8cce9b879daa7c5d P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:21:11.476482 20782 raft_consensus.cc:2804] T b87147731a68400e8cce9b879daa7c5d P cb82cc7bd6db46d980c75237a1402860 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:21:11.476890 20782 raft_consensus.cc:493] T b87147731a68400e8cce9b879daa7c5d P cb82cc7bd6db46d980c75237a1402860 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:11.434660 21151 raft_consensus.cc:740] T af57996a8be84d398f0f023e8d08ef77 P 1ca2725d290e47d68b1a4b609c9816ac [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:21:11.477295 20782 raft_consensus.cc:3060] T b87147731a68400e8cce9b879daa7c5d P cb82cc7bd6db46d980c75237a1402860 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:21:11.477844 21151 consensus_queue.cc:260] T af57996a8be84d398f0f023e8d08ef77 P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:11.437513 21121 raft_consensus.cc:740] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:21:11.479815 21121 consensus_queue.cc:260] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:11.481079 21121 raft_consensus.cc:3060] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:21:11.483848 21087 consensus_queue.cc:260] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:11.485015 21087 raft_consensus.cc:3060] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:21:11.483909 16143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26354361e9b044cea1c2b86c4882fe92" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
W20260709 13:21:11.488451 16189 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:11.488068 16143 raft_consensus.cc:2468] T 26354361e9b044cea1c2b86c4882fe92 P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 11.
I20260709 13:21:11.492969 16220 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57600 (request call id 1610) took 3314 ms. Trace:
I20260709 13:21:11.493502 16220 rpcz_store.cc:276] 0709 13:21:08.178656 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:08.178762 (+   106us) service_pool.cc:224] Handling call
0709 13:21:08.192811 (+ 14049us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:21:09.032856 (+840045us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:11.492647 (+2459791us) spinlock_profiling.cc:243] Waited 839 ms on lock 0x7b5400140fd4. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:21:11.492892 (+   245us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1930307456}
I20260709 13:21:11.450678 20985 raft_consensus.cc:2804] T 829b4ad8c5a54eababeea1181b960d4b P cb82cc7bd6db46d980c75237a1402860 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:21:11.451319 21161 raft_consensus.cc:493] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:11.500459 16296 raft_consensus.cc:3055] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:21:11.503273 16296 raft_consensus.cc:740] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:21:11.495739 16292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02838c86531d4f98bf58d1c22b8baf77" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:11.501775 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f2da98864b4425bf7c0055951fdd4b" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:11.504643 16300 raft_consensus.cc:2393] T 84f2da98864b4425bf7c0055951fdd4b P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 10: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
I20260709 13:21:11.508216 16790 leader_election.cc:304] T 26354361e9b044cea1c2b86c4882fe92 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860; no voters: 90f3da57796b488c9087a2f523adf045
I20260709 13:21:11.509972 16149 raft_consensus.cc:2468] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 15.
I20260709 13:21:11.439949 20991 raft_consensus.cc:515] T fffab26427e24d398e2695aad40ca134 P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:11.501833 16295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5ad4eb281564936a59f41e387a47156" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:11.513818 16822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9944d5e4b8c4109af9955c4f08c9095" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:11.516134 16822 raft_consensus.cc:2410] T c9944d5e4b8c4109af9955c4f08c9095 P cb82cc7bd6db46d980c75237a1402860 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 for term 15 because replica has last-logged OpId of term: 13 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 11.
I20260709 13:21:11.486932 20782 raft_consensus.cc:515] T b87147731a68400e8cce9b879daa7c5d P cb82cc7bd6db46d980c75237a1402860 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:11.434338 21163 raft_consensus.cc:515] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } }
I20260709 13:21:11.523341 16143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2794af5b99984efebdee564035841cb0" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:11.523980 16822 raft_consensus.cc:1240] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [term 16 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 15. Current term is 16. Ops: []
I20260709 13:21:11.526115 16295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c61a39fd4f84424b129c1a6f9ba404d" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:11.502665 20985 raft_consensus.cc:493] T 829b4ad8c5a54eababeea1181b960d4b P cb82cc7bd6db46d980c75237a1402860 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:21:11.513218 21161 raft_consensus.cc:3060] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:21:11.528719 16822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3884f864a96d428785895e5463e5728f" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:21:11.524155 16296 consensus_queue.cc:260] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:11.526988 16295 raft_consensus.cc:2468] T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 13.
I20260709 13:21:11.532541 16296 raft_consensus.cc:3060] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:21:11.517716 21057 raft_consensus.cc:2804] T 26354361e9b044cea1c2b86c4882fe92 P cb82cc7bd6db46d980c75237a1402860 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:21:11.531407 20985 raft_consensus.cc:3060] T 829b4ad8c5a54eababeea1181b960d4b P cb82cc7bd6db46d980c75237a1402860 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:21:11.537716 16817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8638ee363273426db1393dd81e5c7b72" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:11.533686 16822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c11074579b046d99e585c2818e18a2c" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 14 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:11.534307 16821 raft_consensus.cc:1240] T b87147731a68400e8cce9b879daa7c5d P cb82cc7bd6db46d980c75237a1402860 [term 16 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 15. Current term is 16. Ops: [15.11-15.11]
I20260709 13:21:11.536481 16789 leader_election.cc:304] T d2ab0ac8347641e6955e9bce7d90c1a2 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:21:11.543359 16266 leader_election.cc:304] T 57a744f5aaf944678a63469cbf514015 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
I20260709 13:21:11.548596 16295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68a6838d82ef4cac83507f906139d90f" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:11.537431 21057 raft_consensus.cc:493] T 26354361e9b044cea1c2b86c4882fe92 P cb82cc7bd6db46d980c75237a1402860 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:21:11.551137 16266 leader_election.cc:304] T ada5a004ba7c46819051e3ccec71ffdb P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:21:11.553721 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57a744f5aaf944678a63469cbf514015" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:11.549451 16295 raft_consensus.cc:2468] T 68a6838d82ef4cac83507f906139d90f P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 13.
I20260709 13:21:11.556747 20967 raft_consensus.cc:2804] T ada5a004ba7c46819051e3ccec71ffdb P 90f3da57796b488c9087a2f523adf045 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:21:11.552384 20918 raft_consensus.cc:2749] T 57a744f5aaf944678a63469cbf514015 P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:21:11.547276 20999 leader_election.cc:290] T b87147731a68400e8cce9b879daa7c5d P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:11.550411 21223 raft_consensus.cc:2804] T d2ab0ac8347641e6955e9bce7d90c1a2 P cb82cc7bd6db46d980c75237a1402860 [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 13:21:11.567332 16263 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:11.567525 16295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a959dbecc2648eb85d1cc550f5a3245" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:11.568048 16115 leader_election.cc:304] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:21:11.571102 16297 raft_consensus.cc:2393] T 57a744f5aaf944678a63469cbf514015 P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 10: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
I20260709 13:21:11.563366 20964 consensus_queue.cc:1059] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.008s
I20260709 13:21:11.554668 21057 raft_consensus.cc:3060] T 26354361e9b044cea1c2b86c4882fe92 P cb82cc7bd6db46d980c75237a1402860 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:21:11.563951 20967 raft_consensus.cc:697] T ada5a004ba7c46819051e3ccec71ffdb P 90f3da57796b488c9087a2f523adf045 [term 14 LEADER]: Becoming Leader. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:21:11.550242 21115 consensus_queue.cc:1048] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.021s
I20260709 13:21:11.563650 21196 consensus_queue.cc:1059] T a378d7f796f94615b3b347119a9880fd P 90f3da57796b488c9087a2f523adf045 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: INVALID_TERM, Last received: 11.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:21:11.566304 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4f65cdf00b748bb963ee6e81ef43b79" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 16 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:11.526006 21089 consensus_queue.cc:237] T af57996a8be84d398f0f023e8d08ef77 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.10, Last appended by leader: 10, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:11.574010 16115 leader_election.cc:304] T 9c61a39fd4f84424b129c1a6f9ba404d P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:21:11.570375 16789 leader_election.cc:304] T 434bff3082d141838068f323bdb0d1ed P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 647a343a542d4a139fe106c19095129b
I20260709 13:21:11.546587 21129 consensus_queue.cc:1059] T 57a744f5aaf944678a63469cbf514015 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.022s
I20260709 13:21:11.578012 16295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66ae176961b74813a58bc15c9c4c6f2a" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:11.566457 16296 raft_consensus.cc:1275] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 10. (term mismatch)
I20260709 13:21:11.574374 21097 raft_consensus.cc:2804] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:21:11.580355 16115 leader_election.cc:304] T 68a6838d82ef4cac83507f906139d90f P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:21:11.584187 20236 raft_consensus.cc:2804] T 68a6838d82ef4cac83507f906139d90f P 647a343a542d4a139fe106c19095129b [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:21:11.584630 20236 raft_consensus.cc:493] T 68a6838d82ef4cac83507f906139d90f P 647a343a542d4a139fe106c19095129b [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:11.563970 21245 consensus_queue.cc:1059] T 3884f864a96d428785895e5463e5728f P 647a343a542d4a139fe106c19095129b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.014s
I20260709 13:21:11.571086 21223 raft_consensus.cc:493] T d2ab0ac8347641e6955e9bce7d90c1a2 P cb82cc7bd6db46d980c75237a1402860 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:21:11.585022 20236 raft_consensus.cc:3060] T 68a6838d82ef4cac83507f906139d90f P 647a343a542d4a139fe106c19095129b [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:21:11.594709 16144 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40844 (request call id 2507) took 2559 ms (client timeout 2101 ms). Trace:
W20260709 13:21:11.596875 16144 rpcz_store.cc:269] 0709 13:21:09.035623 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:10.648344 (+1612721us) service_pool.cc:224] Handling call
0709 13:21:11.594470 (+946126us) spinlock_profiling.cc:243] Waited 104 ms on lock 0x7b540041e6d4. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:21:11.594610 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":238178688}
I20260709 13:21:11.569868 20931 consensus_queue.cc:1059] T 66ae176961b74813a58bc15c9c4c6f2a P 90f3da57796b488c9087a2f523adf045 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: INVALID_TERM, Last received: 15.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.013s
I20260709 13:21:11.577634 20782 leader_election.cc:290] T b87147731a68400e8cce9b879daa7c5d P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 16 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:11.608596 16295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e9b2bf7011f4ae688a79d7bb5a447b4" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
W20260709 13:21:11.584197 16192 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 210340us
I20260709 13:21:11.582351 16296 pending_rounds.cc:85] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045: Aborting all ops after (but not including) 9
I20260709 13:21:11.610090 16296 pending_rounds.cc:107] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045: Aborting uncommitted NO_OP operation due to leader change: 11.10
I20260709 13:21:11.610499 16296 raft_consensus.cc:2889] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:21:11.582827 20999 raft_consensus.cc:2804] T 9c61a39fd4f84424b129c1a6f9ba404d P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:21:11.583782 21097 raft_consensus.cc:493] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Starting leader election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
W20260709 13:21:11.626288 16150 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40844 (request call id 2499) took 2645 ms (client timeout 3414 ms). Trace:
I20260709 13:21:11.624811 20236 raft_consensus.cc:515] T 68a6838d82ef4cac83507f906139d90f P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:11.631803 16825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e4bfb9090c8460eb14fa7e0d818e7f1" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:11.627007 16146 raft_consensus.cc:1240] T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 14. Current term is 15. Ops: []
I20260709 13:21:11.576098 21007 consensus_queue.cc:1048] T b6f07842d3aa47858df5665874c86f3a P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.022s
W20260709 13:21:11.630896 16150 rpcz_store.cc:269] 0709 13:21:08.982934 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:08.983339 (+   405us) service_pool.cc:224] Handling call
0709 13:21:11.626060 (+2642721us) spinlock_profiling.cc:243] Waited 674 ms on lock 0x7b54000e9054. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:21:11.626230 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1550257536}
I20260709 13:21:11.628360 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdbd78acb4cf46298e0de8ba0d130e19" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 16 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:11.629312 16822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:11.636204 16823 raft_consensus.cc:1240] T 23f82224f4564fea93572e8206bad1fe P cb82cc7bd6db46d980c75237a1402860 [term 12 LEADER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 11. Current term is 12. Ops: [11.7-11.7]
I20260709 13:21:11.636890 16822 raft_consensus.cc:2393] T 19baa7bc367844c288bca8f81439c413 P cb82cc7bd6db46d980c75237a1402860 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 15: Already voted for candidate cb82cc7bd6db46d980c75237a1402860 in this term.
I20260709 13:21:11.598480 21223 raft_consensus.cc:3060] T d2ab0ac8347641e6955e9bce7d90c1a2 P cb82cc7bd6db46d980c75237a1402860 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:21:11.623956 20999 raft_consensus.cc:493] T 9c61a39fd4f84424b129c1a6f9ba404d P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Starting leader election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:21:11.641254 16296 raft_consensus.cc:1275] T 8638ee363273426db1393dd81e5c7b72 P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
W20260709 13:21:11.639566 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 266193us
I20260709 13:21:11.643121 20236 leader_election.cc:290] T 68a6838d82ef4cac83507f906139d90f P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 15 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:11.642949 16144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87147731a68400e8cce9b879daa7c5d" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:21:11.656136 20999 raft_consensus.cc:3060] T 9c61a39fd4f84424b129c1a6f9ba404d P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:21:11.645015 21141 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 3.085s	user 0.033s	sys 0.031s
W20260709 13:21:11.657099 21141 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 3.101s	user 0.033s	sys 0.035s
W20260709 13:21:11.655686 16789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 85361.6us
I20260709 13:21:11.658628 16295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:11.655280 20967 consensus_queue.cc:237] T ada5a004ba7c46819051e3ccec71ffdb P 90f3da57796b488c9087a2f523adf045 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:11.659945 16295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e4bfb9090c8460eb14fa7e0d818e7f1" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
W20260709 13:21:11.660578 16190 leader_election.cc:336] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389): Timed out: RequestConsensusVote RPC to 127.14.177.131:41389 timed out after 1.645s (SENT)
I20260709 13:21:11.660791 16295 raft_consensus.cc:2468] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 12.
I20260709 13:21:11.661145 16190 leader_election.cc:304] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860
I20260709 13:21:11.663638 16789 leader_election.cc:304] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:21:11.635424 21097 raft_consensus.cc:3060] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:21:11.664201 16144 raft_consensus.cc:3060] T b87147731a68400e8cce9b879daa7c5d P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:21:11.667930 16822 raft_consensus.cc:1240] T 19baa7bc367844c288bca8f81439c413 P cb82cc7bd6db46d980c75237a1402860 [term 15 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 14. Current term is 15. Ops: []
I20260709 13:21:11.670383 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68a6838d82ef4cac83507f906139d90f" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:21:11.670938 16218 raft_consensus.cc:1240] T 8cc959fb7f704fba9009c0595467d074 P 1ca2725d290e47d68b1a4b609c9816ac [term 13 LEADER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 12. Current term is 13. Ops: []
I20260709 13:21:11.671028 16293 raft_consensus.cc:3060] T 68a6838d82ef4cac83507f906139d90f P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Advancing to term 15
I20260709 13:21:11.640282 20782 raft_consensus.cc:2804] T 434bff3082d141838068f323bdb0d1ed P cb82cc7bd6db46d980c75237a1402860 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:21:11.673873 16190 leader_election.cc:400] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 14 election: Vote denied by peer 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389) with higher term. Message: Invalid argument: T 8638ee363273426db1393dd81e5c7b72 P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac for earlier term 14. Current term is 15.
I20260709 13:21:11.676631 16190 leader_election.cc:403] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 14 election: Cancelling election due to peer responding with higher term
I20260709 13:21:11.677685 16291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2794af5b99984efebdee564035841cb0" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:11.678691 16291 raft_consensus.cc:2468] T 2794af5b99984efebdee564035841cb0 P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 10.
I20260709 13:21:11.680146 16291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87147731a68400e8cce9b879daa7c5d" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:21:11.689317 16790 leader_election.cc:304] T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:21:11.696914 16291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fffab26427e24d398e2695aad40ca134" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:11.699554 16291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806d7690c2b644959152074bb2cab0a7" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:11.701845 16144 raft_consensus.cc:2468] T b87147731a68400e8cce9b879daa7c5d P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 16.
I20260709 13:21:11.702646 16144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee9ed67006144797b1de6ea8c2013a68" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:21:11.703083 16144 raft_consensus.cc:3060] T ee9ed67006144797b1de6ea8c2013a68 P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:21:11.672104 21141 raft_consensus.cc:2749] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:21:11.716915 16191 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:11.676998 21060 raft_consensus.cc:2749] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:21:11.675930 20782 raft_consensus.cc:493] T 434bff3082d141838068f323bdb0d1ed P cb82cc7bd6db46d980c75237a1402860 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:21:11.697382 21163 leader_election.cc:290] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:11.676015 20991 leader_election.cc:290] T fffab26427e24d398e2695aad40ca134 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:11.709969 16291 raft_consensus.cc:2393] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 13: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
I20260709 13:21:11.742708 20782 raft_consensus.cc:3060] T 434bff3082d141838068f323bdb0d1ed P cb82cc7bd6db46d980c75237a1402860 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:21:11.742908 21129 consensus_queue.cc:1059] T ee9ed67006144797b1de6ea8c2013a68 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:21:11.752537 21129 raft_consensus.cc:2749] T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:21:11.755144 21262 raft_consensus.cc:3055] T ee9ed67006144797b1de6ea8c2013a68 P cb82cc7bd6db46d980c75237a1402860 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:21:11.755688 21262 raft_consensus.cc:740] T ee9ed67006144797b1de6ea8c2013a68 P cb82cc7bd6db46d980c75237a1402860 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:21:11.756712 21262 consensus_queue.cc:260] T ee9ed67006144797b1de6ea8c2013a68 P cb82cc7bd6db46d980c75237a1402860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:11.676205 21075 consensus_queue.cc:1059] T d16dfad0c3b540e5b4b86492ddf87208 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.048s
W20260709 13:21:11.761291 16223 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33050 (request call id 843) took 2846 ms (client timeout 1870 ms). Trace:
I20260709 13:21:11.739049 16144 raft_consensus.cc:2468] T ee9ed67006144797b1de6ea8c2013a68 P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 12.
W20260709 13:21:11.761806 16223 rpcz_store.cc:269] 0709 13:21:08.914990 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:09.038762 (+123772us) service_pool.cc:224] Handling call
0709 13:21:11.761088 (+2722326us) spinlock_profiling.cc:243] Waited 192 ms on lock 0x7b54002b4b54. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:21:11.761225 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":442498816}
I20260709 13:21:11.768059 16817 raft_consensus.cc:1240] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 [term 15 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 14. Current term is 15. Ops: [14.12-14.12]
I20260709 13:21:11.683390 21195 consensus_queue.cc:1059] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.070s
I20260709 13:21:11.780200 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5ad4eb281564936a59f41e387a47156" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
W20260709 13:21:11.783386 16114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:21:11.783922 16114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:11.668123 21115 consensus_queue.cc:1048] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.072s
I20260709 13:21:11.871850 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f2da98864b4425bf7c0055951fdd4b" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:11.872637 16225 raft_consensus.cc:2468] T 84f2da98864b4425bf7c0055951fdd4b P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 10.
I20260709 13:21:11.780889 16224 raft_consensus.cc:2468] T e5ad4eb281564936a59f41e387a47156 P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 13.
W20260709 13:21:11.875497 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 201749us
I20260709 13:21:11.876518 16221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2372981006d4b218215c28964dea623" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:21:11.876731 16190 leader_election.cc:304] T 84f2da98864b4425bf7c0055951fdd4b P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:21:11.877282 16221 raft_consensus.cc:3060] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:21:11.877378 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 188159us
W20260709 13:21:11.883437 21251 log.cc:927] Time spent T 87ac0d1f7e8a419f9bcbc04aee0e067e P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 0.169s	user 0.001s	sys 0.000s
I20260709 13:21:11.893505 16222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68a6838d82ef4cac83507f906139d90f" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:11.735059 16293 raft_consensus.cc:2468] T 68a6838d82ef4cac83507f906139d90f P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 15.
I20260709 13:21:11.890815 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4f65cdf00b748bb963ee6e81ef43b79" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 16 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
W20260709 13:21:11.909009 16189 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:11.754007 21141 raft_consensus.cc:2749] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: Leader election lost for term 14. Reason: Vote denied by peer 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389) with higher term. Message: Invalid argument: T 8638ee363273426db1393dd81e5c7b72 P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac for earlier term 14. Current term is 15.
W20260709 13:21:11.929713 16192 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 161444us
I20260709 13:21:11.930994 16223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02838c86531d4f98bf58d1c22b8baf77" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:11.966629 21260 raft_consensus.cc:493] T 3884f864a96d428785895e5463e5728f P cb82cc7bd6db46d980c75237a1402860 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:21:11.967432 21260 raft_consensus.cc:515] T 3884f864a96d428785895e5463e5728f P cb82cc7bd6db46d980c75237a1402860 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:11.969990 21260 leader_election.cc:290] T 3884f864a96d428785895e5463e5728f P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:11.972615 16144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8158bbd17b994d2eb96f560bd0f1b88d" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:11.973910 21264 raft_consensus.cc:493] T 746aabf856ab401c9ef5e859371e7b67 P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:11.974459 21264 raft_consensus.cc:515] T 746aabf856ab401c9ef5e859371e7b67 P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:11.975099 16144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f2da98864b4425bf7c0055951fdd4b" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:11.975800 16144 raft_consensus.cc:2468] T 84f2da98864b4425bf7c0055951fdd4b P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 9.
I20260709 13:21:11.976795 20894 raft_consensus.cc:3055] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 90f3da57796b488c9087a2f523adf045 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:21:11.977082 16144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a878999c0024d2abe517484180290fc" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:11.977470 20894 raft_consensus.cc:740] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 90f3da57796b488c9087a2f523adf045 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:21:11.978688 16144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4dc6b0541ca431ebc757beac00521f1" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:11.980191 16144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0eab25ce4c64e018a041beeda2867db" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:11.980921 16144 raft_consensus.cc:2393] T c0eab25ce4c64e018a041beeda2867db P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 13: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
I20260709 13:21:11.981987 16144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:11.982715 16144 raft_consensus.cc:2393] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 15: Already voted for candidate cb82cc7bd6db46d980c75237a1402860 in this term.
I20260709 13:21:11.984040 16144 raft_consensus.cc:1240] T ee9ed67006144797b1de6ea8c2013a68 P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 11. Current term is 12. Ops: []
I20260709 13:21:11.992151 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66f75124215f4ca6bef831e174a792d8" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:21:11.993206 16217 raft_consensus.cc:2393] T 66f75124215f4ca6bef831e174a792d8 P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 13: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
I20260709 13:21:11.994285 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68a6838d82ef4cac83507f906139d90f" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
W20260709 13:21:11.996083 21252 log.cc:927] Time spent T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 0.120s	user 0.000s	sys 0.000s
I20260709 13:21:11.915313 21060 raft_consensus.cc:493] T 8c11074579b046d99e585c2818e18a2c P cb82cc7bd6db46d980c75237a1402860 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:21:12.016682 21060 raft_consensus.cc:515] T 8c11074579b046d99e585c2818e18a2c P cb82cc7bd6db46d980c75237a1402860 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:21:12.064925 16790 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:21:12.098443 16116 connection.cc:256] RPC call timeout handler was delayed by -2.1070086679999349s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2041s for a response
W20260709 13:21:11.891515 21253 log.cc:927] Time spent T 02838c86531d4f98bf58d1c22b8baf77 P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 0.198s	user 0.001s	sys 0.000s
I20260709 13:21:11.930132 16220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c61a39fd4f84424b129c1a6f9ba404d" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:11.761027 21007 raft_consensus.cc:493] T d16dfad0c3b540e5b4b86492ddf87208 P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:21:12.119367 16222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a959dbecc2648eb85d1cc550f5a3245" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:12.134402 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806d7690c2b644959152074bb2cab0a7" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:12.123759 16825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a9de347c711477e87c582d003f36925" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
W20260709 13:21:12.195242 16192 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 259046us
I20260709 13:21:12.122037 16151 consensus_queue.cc:260] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:12.205401 16151 raft_consensus.cc:3060] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:21:12.224117 16223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fffab26427e24d398e2695aad40ca134" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:12.224949 16223 raft_consensus.cc:2468] T fffab26427e24d398e2695aad40ca134 P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 12.
W20260709 13:21:12.206087 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 329391us
I20260709 13:21:12.226835 16190 leader_election.cc:304] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860
I20260709 13:21:11.875960 21262 raft_consensus.cc:3060] T ee9ed67006144797b1de6ea8c2013a68 P cb82cc7bd6db46d980c75237a1402860 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:21:12.234571 16217 raft_consensus.cc:3055] T b6f07842d3aa47858df5665874c86f3a P 1ca2725d290e47d68b1a4b609c9816ac [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:21:12.235096 16217 raft_consensus.cc:740] T b6f07842d3aa47858df5665874c86f3a P 1ca2725d290e47d68b1a4b609c9816ac [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:21:12.246711 16221 raft_consensus.cc:2468] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 15.
I20260709 13:21:12.252365 16820 raft_consensus.cc:1240] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860 [term 11 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 10. Current term is 11. Ops: []
I20260709 13:21:12.236048 16217 consensus_queue.cc:260] T b6f07842d3aa47858df5665874c86f3a P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:12.254129 16217 raft_consensus.cc:3060] T b6f07842d3aa47858df5665874c86f3a P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:21:11.976902 21264 leader_election.cc:290] T 746aabf856ab401c9ef5e859371e7b67 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
W20260709 13:21:12.203364 21121 consensus_meta.cc:287] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860: Time spent flushing consensus metadata: real 0.722s	user 0.011s	sys 0.000s
I20260709 13:21:12.116516 21260 raft_consensus.cc:493] T 66f75124215f4ca6bef831e174a792d8 P cb82cc7bd6db46d980c75237a1402860 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:21:12.286269 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 59497us
I20260709 13:21:12.288218 16190 leader_election.cc:304] T e5ad4eb281564936a59f41e387a47156 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
W20260709 13:21:12.281615 16191 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.559s	user 0.010s	sys 0.022s
I20260709 13:21:12.283702 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3884f864a96d428785895e5463e5728f" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
W20260709 13:21:12.237289 21126 consensus_meta.cc:287] T 8158bbd17b994d2eb96f560bd0f1b88d P 647a343a542d4a139fe106c19095129b: Time spent flushing consensus metadata: real 0.842s	user 0.011s	sys 0.001s
I20260709 13:21:12.316927 16824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87147731a68400e8cce9b879daa7c5d" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
W20260709 13:21:12.304462 16191 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.582s	user 0.010s	sys 0.023s
I20260709 13:21:12.305166 16297 raft_consensus.cc:2393] T 3884f864a96d428785895e5463e5728f P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 in current term 9: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
I20260709 13:21:12.318720 16818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57a744f5aaf944678a63469cbf514015" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
W20260709 13:21:12.319406 16191 connection.cc:256] RPC call timeout handler was delayed by -1.0969990560001861s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1511s for a response
W20260709 13:21:12.319998 16191 connection.cc:256] RPC call timeout handler was delayed by -1.0708990560001439s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15400000000000003s for a response
I20260709 13:21:12.321035 16191 leader_election.cc:304] T 1df93779209748b9860f6d893dd3de53 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
I20260709 13:21:12.325523 16142 raft_consensus.cc:2410] T 8158bbd17b994d2eb96f560bd0f1b88d P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 for term 13 because replica has last-logged OpId of term: 12 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 8.
I20260709 13:21:12.252204 21141 raft_consensus.cc:2804] T 84f2da98864b4425bf7c0055951fdd4b P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:21:12.331753 16151 raft_consensus.cc:1275] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 13:21:12.369072 20236 raft_consensus.cc:493] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:12.369668 20236 raft_consensus.cc:515] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:12.366093 16820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66f75124215f4ca6bef831e174a792d8" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:21:12.251892 21075 raft_consensus.cc:493] T 08c74490c40243a88e62d1760b5cd814 P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:12.368631 16151 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40844 (request call id 2501) took 3384 ms. Trace:
I20260709 13:21:12.371294 16151 rpcz_store.cc:276] 0709 13:21:08.983795 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:09.006171 (+ 22376us) service_pool.cc:224] Handling call
0709 13:21:09.016071 (+  9900us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:21:12.121659 (+3105588us) spinlock_profiling.cc:243] Waited 28.8 ms on lock 0x7b4c0043fd80. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69aee2ba20 00007f69aee3ca29 00007f69aee02644 00007f69aee29f2d 00007f69aee29b93 00007f69b0b5a18b 00007f69b0b5acb5 00007f69b0b55f30 00007f69b0b577de 00007f69b0b61b49 00007f69b0b62416
0709 13:21:12.368325 (+246666us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:12.368566 (+   241us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":66219904}
I20260709 13:21:12.373443 16297 raft_consensus.cc:3055] T 9e65676b21b748728ad39668cfe722ec P 90f3da57796b488c9087a2f523adf045 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:21:12.373893 16297 raft_consensus.cc:740] T 9e65676b21b748728ad39668cfe722ec P 90f3da57796b488c9087a2f523adf045 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:21:12.374819 16297 consensus_queue.cc:260] T 9e65676b21b748728ad39668cfe722ec P 90f3da57796b488c9087a2f523adf045 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:12.379289 21249 raft_consensus.cc:2749] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:21:12.379704 16297 raft_consensus.cc:3060] T 9e65676b21b748728ad39668cfe722ec P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:21:12.381204 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 93050.8us
I20260709 13:21:12.384774 21110 consensus_queue.cc:260] T c4f65cdf00b748bb963ee6e81ef43b79 P 90f3da57796b488c9087a2f523adf045 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.10, Last appended by leader: 10, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:12.386029 21110 raft_consensus.cc:3060] T c4f65cdf00b748bb963ee6e81ef43b79 P 90f3da57796b488c9087a2f523adf045 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:21:12.386389 16151 raft_consensus.cc:1240] T 68a6838d82ef4cac83507f906139d90f P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 14. Current term is 15. Ops: []
W20260709 13:21:12.387595 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 66683.4us
I20260709 13:21:12.276490 21060 leader_election.cc:290] T 8c11074579b046d99e585c2818e18a2c P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:12.383004 16190 leader_election.cc:304] T 5a96373bb0e14dce8a266b2fc6c58507 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:21:12.393126 21180 raft_consensus.cc:2749] T e5ad4eb281564936a59f41e387a47156 P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:21:12.393923 21180 consensus_queue.cc:1048] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:21:12.395753 21264 raft_consensus.cc:2804] T 1df93779209748b9860f6d893dd3de53 P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:21:12.396318 21264 raft_consensus.cc:697] T 1df93779209748b9860f6d893dd3de53 P 1ca2725d290e47d68b1a4b609c9816ac [term 13 LEADER]: Becoming Leader. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:21:12.397483 21264 consensus_queue.cc:237] T 1df93779209748b9860f6d893dd3de53 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:21:12.400049 16788 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:12.401827 16824 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9e4bfb9090c8460eb14fa7e0d818e7f1"
dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
mode: GRACEFUL
new_leader_uuid: "90f3da57796b488c9087a2f523adf045"
 from {username='slave'} at 127.0.0.1:37818
I20260709 13:21:12.402462 16824 raft_consensus.cc:606] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P cb82cc7bd6db46d980c75237a1402860 [term 12 LEADER]: Received request to transfer leadership to TS 90f3da57796b488c9087a2f523adf045
I20260709 13:21:12.406644 16115 leader_election.cc:304] T 68a6838d82ef4cac83507f906139d90f P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:21:11.978688 20894 consensus_queue.cc:260] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 90f3da57796b488c9087a2f523adf045 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:12.363027 16217 raft_consensus.cc:1275] T b6f07842d3aa47858df5665874c86f3a P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 13:21:12.221841 21007 raft_consensus.cc:515] T d16dfad0c3b540e5b4b86492ddf87208 P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:12.411556 16190 leader_election.cc:304] T e4dc6b0541ca431ebc757beac00521f1 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:21:12.425311 16190 leader_election.cc:304] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860
W20260709 13:21:12.376487 20782 consensus_meta.cc:287] T 434bff3082d141838068f323bdb0d1ed P cb82cc7bd6db46d980c75237a1402860: Time spent flushing consensus metadata: real 0.621s	user 0.006s	sys 0.003s
I20260709 13:21:12.367123 21106 raft_consensus.cc:493] T 8158bbd17b994d2eb96f560bd0f1b88d P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:12.368270 21141 raft_consensus.cc:493] T 84f2da98864b4425bf7c0055951fdd4b P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:21:12.409746 21097 consensus_meta.cc:287] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b: Time spent flushing consensus metadata: real 0.715s	user 0.004s	sys 0.004s
W20260709 13:21:12.414275 21087 consensus_meta.cc:287] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac: Time spent flushing consensus metadata: real 0.906s	user 0.012s	sys 0.000s
I20260709 13:21:12.428646 20991 raft_consensus.cc:2804] T 68a6838d82ef4cac83507f906139d90f P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:21:12.464854 21155 leader_election.cc:290] T 3a7ca61592e848ea9b0fb81a2ce33cdc P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:12.320935 21260 raft_consensus.cc:515] T 66f75124215f4ca6bef831e174a792d8 P cb82cc7bd6db46d980c75237a1402860 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:12.415211 21265 raft_consensus.cc:493] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:21:12.430153 21060 raft_consensus.cc:493] T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:12.480471 21250 raft_consensus.cc:3055] T d16dfad0c3b540e5b4b86492ddf87208 P 1ca2725d290e47d68b1a4b609c9816ac [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:21:12.481174 21250 raft_consensus.cc:740] T d16dfad0c3b540e5b4b86492ddf87208 P 1ca2725d290e47d68b1a4b609c9816ac [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:21:12.486274 21103 leader_election.cc:290] T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 15 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:12.375802 21075 raft_consensus.cc:515] T 08c74490c40243a88e62d1760b5cd814 P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:12.450739 21141 raft_consensus.cc:3060] T 84f2da98864b4425bf7c0055951fdd4b P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:21:12.447269 20894 raft_consensus.cc:3060] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:21:12.511790 16820 raft_consensus.cc:2410] T 66f75124215f4ca6bef831e174a792d8 P cb82cc7bd6db46d980c75237a1402860 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b for term 13 because replica has last-logged OpId of term: 12 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 10.
W20260709 13:21:12.515210 16263 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:12.477361 20991 raft_consensus.cc:697] T 68a6838d82ef4cac83507f906139d90f P 647a343a542d4a139fe106c19095129b [term 15 LEADER]: Becoming Leader. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:21:12.448264 21106 raft_consensus.cc:515] T 8158bbd17b994d2eb96f560bd0f1b88d P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:12.482403 21250 consensus_queue.cc:260] T d16dfad0c3b540e5b4b86492ddf87208 P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:12.450019 20782 raft_consensus.cc:515] T 434bff3082d141838068f323bdb0d1ed P cb82cc7bd6db46d980c75237a1402860 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:21:12.526505 16791 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:12.463021 21097 raft_consensus.cc:515] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:12.509773 21267 raft_consensus.cc:2749] T 5a96373bb0e14dce8a266b2fc6c58507 P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:21:12.535920 21161 consensus_meta.cc:287] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac: Time spent flushing consensus metadata: real 0.995s	user 0.011s	sys 0.002s
I20260709 13:21:12.495436 21265 raft_consensus.cc:515] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:12.496953 21060 raft_consensus.cc:515] T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:12.570739 21228 leader_election.cc:290] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
W20260709 13:21:12.554350 20985 consensus_meta.cc:287] T 829b4ad8c5a54eababeea1181b960d4b P cb82cc7bd6db46d980c75237a1402860: Time spent flushing consensus metadata: real 0.997s	user 0.010s	sys 0.002s
I20260709 13:21:12.576591 21075 leader_election.cc:290] T 08c74490c40243a88e62d1760b5cd814 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
W20260709 13:21:12.558959 21057 consensus_meta.cc:287] T 26354361e9b044cea1c2b86c4882fe92 P cb82cc7bd6db46d980c75237a1402860: Time spent flushing consensus metadata: real 0.982s	user 0.009s	sys 0.004s
I20260709 13:21:12.595288 21216 raft_consensus.cc:3055] T 57a744f5aaf944678a63469cbf514015 P cb82cc7bd6db46d980c75237a1402860 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:21:12.595934 21216 raft_consensus.cc:740] T 57a744f5aaf944678a63469cbf514015 P cb82cc7bd6db46d980c75237a1402860 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:21:12.597213 21216 consensus_queue.cc:260] T 57a744f5aaf944678a63469cbf514015 P cb82cc7bd6db46d980c75237a1402860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:12.615464 20782 leader_election.cc:290] T 434bff3082d141838068f323bdb0d1ed P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 13 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
W20260709 13:21:12.618748 21263 log.cc:927] Time spent T 8c11074579b046d99e585c2818e18a2c P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.109s	user 0.001s	sys 0.000s
I20260709 13:21:12.662597 21007 leader_election.cc:290] T d16dfad0c3b540e5b4b86492ddf87208 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:12.672302 21260 leader_election.cc:290] T 66f75124215f4ca6bef831e174a792d8 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:21:12.633050 21060 leader_election.cc:290] T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:12.580758 20985 raft_consensus.cc:515] T 829b4ad8c5a54eababeea1181b960d4b P cb82cc7bd6db46d980c75237a1402860 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:12.675447 20985 leader_election.cc:290] T 829b4ad8c5a54eababeea1181b960d4b P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 13 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:21:12.559938 21161 raft_consensus.cc:515] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:12.678107 21161 leader_election.cc:290] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 15 election: Requested vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:12.587384 20991 consensus_queue.cc:237] T 68a6838d82ef4cac83507f906139d90f P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:12.658735 21097 leader_election.cc:290] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 16 election: Requested vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:12.660476 21265 leader_election.cc:290] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:12.675747 20964 raft_consensus.cc:3055] T a378d7f796f94615b3b347119a9880fd P 90f3da57796b488c9087a2f523adf045 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:21:12.522716 21180 raft_consensus.cc:2804] T e4dc6b0541ca431ebc757beac00521f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:21:12.692046 21191 raft_consensus.cc:493] T c4f65cdf00b748bb963ee6e81ef43b79 P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:12.717015 21191 raft_consensus.cc:515] T c4f65cdf00b748bb963ee6e81ef43b79 P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:12.658883 21170 raft_consensus.cc:3055] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:21:12.598917 21216 raft_consensus.cc:3060] T 57a744f5aaf944678a63469cbf514015 P cb82cc7bd6db46d980c75237a1402860 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:21:12.729758 21165 raft_consensus.cc:3055] T 3884f864a96d428785895e5463e5728f P 647a343a542d4a139fe106c19095129b [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:21:12.730423 21165 raft_consensus.cc:740] T 3884f864a96d428785895e5463e5728f P 647a343a542d4a139fe106c19095129b [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:21:12.731600 21165 consensus_queue.cc:260] T 3884f864a96d428785895e5463e5728f P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:12.699504 20961 raft_consensus.cc:3060] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:21:12.746757 21255 log.cc:927] Time spent T 23f82224f4564fea93572e8206bad1fe P cb82cc7bd6db46d980c75237a1402860: Append to log took a long time: real 0.871s	user 0.011s	sys 0.002s
I20260709 13:21:12.699254 21250 raft_consensus.cc:3060] T d16dfad0c3b540e5b4b86492ddf87208 P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:21:12.712551 20964 raft_consensus.cc:740] T a378d7f796f94615b3b347119a9880fd P 90f3da57796b488c9087a2f523adf045 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
W20260709 13:21:12.750788 21247 log.cc:927] Time spent T 806d7690c2b644959152074bb2cab0a7 P cb82cc7bd6db46d980c75237a1402860: Append to log took a long time: real 1.152s	user 0.002s	sys 0.012s
I20260709 13:21:12.725963 21170 raft_consensus.cc:740] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:21:12.715998 21180 raft_consensus.cc:493] T e4dc6b0541ca431ebc757beac00521f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Starting leader election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:21:12.698122 21057 raft_consensus.cc:515] T 26354361e9b044cea1c2b86c4882fe92 P cb82cc7bd6db46d980c75237a1402860 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:12.774119 21180 raft_consensus.cc:3060] T e4dc6b0541ca431ebc757beac00521f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:21:12.783185 21256 log.cc:927] Time spent T 8cc959fb7f704fba9009c0595467d074 P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 0.889s	user 0.004s	sys 0.014s
W20260709 13:21:12.799021 21223 consensus_meta.cc:287] T d2ab0ac8347641e6955e9bce7d90c1a2 P cb82cc7bd6db46d980c75237a1402860: Time spent flushing consensus metadata: real 1.145s	user 0.011s	sys 0.002s
W20260709 13:21:12.832653 16117 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.607s	user 0.001s	sys 0.014s
I20260709 13:21:12.837117 15046 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 13:21:12.838235 16150 raft_consensus.cc:1240] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 15. Current term is 16. Ops: []
W20260709 13:21:12.847397 16192 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.623s	user 0.019s	sys 0.027s
I20260709 13:21:12.918974 20918 raft_consensus.cc:493] T 84f2da98864b4425bf7c0055951fdd4b P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:21:12.919610 16192 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.695s	user 0.019s	sys 0.027s
I20260709 13:21:12.919953 16822 raft_consensus.cc:1240] T 9e65676b21b748728ad39668cfe722ec P cb82cc7bd6db46d980c75237a1402860 [term 13 LEADER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 12. Current term is 13. Ops: []
W20260709 13:21:12.856400 16263 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:21:12.856983 16266 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:21:12.857766 16293 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.945s	user 0.041s	sys 0.031s
I20260709 13:21:12.858556 16295 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48684 (request call id 2701) took 1199 ms. Trace:
I20260709 13:21:12.859576 16298 raft_consensus.cc:1275] T c4f65cdf00b748bb963ee6e81ef43b79 P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 15 index: 10. (term mismatch)
I20260709 13:21:12.860194 16291 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40474 (request call id 837) took 1191 ms. Trace:
W20260709 13:21:12.868252 16789 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.607s	user 0.006s	sys 0.031s
W20260709 13:21:12.922124 16789 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.661s	user 0.007s	sys 0.031s
I20260709 13:21:12.920282 20918 raft_consensus.cc:515] T 84f2da98864b4425bf7c0055951fdd4b P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:21:12.923377 16263 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:12.921505 16298 pending_rounds.cc:85] T c4f65cdf00b748bb963ee6e81ef43b79 P 90f3da57796b488c9087a2f523adf045: Aborting all ops after (but not including) 9
I20260709 13:21:12.924489 16298 pending_rounds.cc:107] T c4f65cdf00b748bb963ee6e81ef43b79 P 90f3da57796b488c9087a2f523adf045: Aborting uncommitted NO_OP operation due to leader change: 14.10
I20260709 13:21:12.924927 16298 raft_consensus.cc:2889] T c4f65cdf00b748bb963ee6e81ef43b79 P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:21:12.869246 16791 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:12.925648 20918 leader_election.cc:290] T 84f2da98864b4425bf7c0055951fdd4b P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:21:12.775794 21106 leader_election.cc:290] T 8158bbd17b994d2eb96f560bd0f1b88d P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
W20260709 13:21:12.774839 21089 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.787s	user 0.049s	sys 0.071s
W20260709 13:21:12.790395 21262 consensus_meta.cc:287] T ee9ed67006144797b1de6ea8c2013a68 P cb82cc7bd6db46d980c75237a1402860: Time spent flushing consensus metadata: real 0.528s	user 0.010s	sys 0.000s
W20260709 13:21:12.913800 16788 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:21:12.927408 21089 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.939s	user 0.050s	sys 0.071s
W20260709 13:21:12.919881 16117 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.694s	user 0.001s	sys 0.014s
W20260709 13:21:12.922314 16266 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:12.929783 21276 raft_consensus.cc:493] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:21:12.931098 20964 consensus_queue.cc:260] T a378d7f796f94615b3b347119a9880fd P 90f3da57796b488c9087a2f523adf045 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:12.933249 20964 raft_consensus.cc:3060] T a378d7f796f94615b3b347119a9880fd P 90f3da57796b488c9087a2f523adf045 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:21:12.934201 16117 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:12.934763 16824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746aabf856ab401c9ef5e859371e7b67" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:12.936076 16824 raft_consensus.cc:2468] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 12.
W20260709 13:21:12.939437 16293 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.027s	user 0.041s	sys 0.031s
I20260709 13:21:12.939666 16819 raft_consensus.cc:1240] T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860 [term 15 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 14. Current term is 15. Ops: [14.10-14.10]
I20260709 13:21:12.941015 16293 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48674 (request call id 2859) took 1186 ms. Trace:
W20260709 13:21:12.943835 16263 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:21:12.946288 16117 connection.cc:256] RPC call timeout handler was delayed by -1.2499851549998766s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18680000000000002s for a response
W20260709 13:21:12.946662 21046 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.531s	user 0.031s	sys 0.082s
W20260709 13:21:12.946892 16117 connection.cc:256] RPC call timeout handler was delayed by -1.0406482850000884s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21530000000000002s for a response
W20260709 13:21:12.947202 21046 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.575s	user 0.031s	sys 0.082s
W20260709 13:21:12.947438 16117 connection.cc:256] RPC call timeout handler was delayed by -1.0165580459997727s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2098s for a response
I20260709 13:21:12.947551 16789 leader_election.cc:304] T 2794af5b99984efebdee564035841cb0 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 647a343a542d4a139fe106c19095129b
I20260709 13:21:12.941515 16293 rpcz_store.cc:276] 0709 13:21:11.754620 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:11.898095 (+143475us) service_pool.cc:224] Handling call
0709 13:21:11.898448 (+   353us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:21:11.898810 (+   362us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:21:11.898849 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:21:11.909996 (+ 11147us) log.cc:844] Serialized 26 byte log entry
0709 13:21:12.940301 (+1030305us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:21:12.940422 (+   121us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:12.940571 (+   149us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:21:12.940631 (+    60us) raft_consensus.cc:1685] finished
0709 13:21:12.940649 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:21:12.940927 (+   278us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":74,"raft.run_cpu_time_us":177,"raft.run_wall_time_us":103171,"thread_start_us":1027067,"threads_started":1,"wal-append.queue_time_us":885893}
I20260709 13:21:12.951669 20902 consensus_queue.cc:1048] T 8638ee363273426db1393dd81e5c7b72 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:21:12.952039 16192 leader_election.cc:304] T 746aabf856ab401c9ef5e859371e7b67 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:21:12.951843 21089 raft_consensus.cc:2804] T 2794af5b99984efebdee564035841cb0 P cb82cc7bd6db46d980c75237a1402860 [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 13:21:12.955230 20999 consensus_meta.cc:287] T 9c61a39fd4f84424b129c1a6f9ba404d P 647a343a542d4a139fe106c19095129b: Time spent flushing consensus metadata: real 1.287s	user 0.009s	sys 0.005s
I20260709 13:21:12.955241 21089 raft_consensus.cc:493] T 2794af5b99984efebdee564035841cb0 P cb82cc7bd6db46d980c75237a1402860 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:21:12.956190 21089 raft_consensus.cc:3060] T 2794af5b99984efebdee564035841cb0 P cb82cc7bd6db46d980c75237a1402860 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:21:12.956396 20999 raft_consensus.cc:515] T 9c61a39fd4f84424b129c1a6f9ba404d P 647a343a542d4a139fe106c19095129b [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:12.958073 16144 raft_consensus.cc:1240] T 9c61a39fd4f84424b129c1a6f9ba404d P 647a343a542d4a139fe106c19095129b [term 14 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 13. Current term is 14. Ops: []
I20260709 13:21:12.960762 21170 consensus_queue.cc:260] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } }
W20260709 13:21:12.968730 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 543393us
I20260709 13:21:12.969322 16151 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40844 (request call id 2524) took 1243 ms. Trace:
I20260709 13:21:12.953575 21223 raft_consensus.cc:515] T d2ab0ac8347641e6955e9bce7d90c1a2 P cb82cc7bd6db46d980c75237a1402860 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:21:12.971961 16266 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:12.921439 16295 rpcz_store.cc:276] 0709 13:21:11.658511 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:11.661950 (+  3439us) service_pool.cc:224] Handling call
0709 13:21:11.662624 (+   674us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:21:11.663121 (+   497us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:21:11.663155 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:21:11.663664 (+   509us) log.cc:844] Serialized 26 byte log entry
0709 13:21:11.854663 (+190999us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:21:11.854727 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:11.854828 (+   101us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:21:12.857758 (+1002930us) raft_consensus.cc:1685] finished
0709 13:21:12.857806 (+    48us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:21:12.858210 (+   404us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":19966,"raft.queue_time_us":3197,"raft.run_cpu_time_us":116,"raft.run_wall_time_us":5269,"thread_start_us":190624,"threads_started":1,"wal-append.queue_time_us":795898}
W20260709 13:21:12.972832 16142 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40830 (request call id 1636) took 4742 ms (client timeout 1723 ms). Trace:
W20260709 13:21:12.963892 16191 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.511s	user 0.009s	sys 0.025s
I20260709 13:21:12.973942 16151 rpcz_store.cc:276] 0709 13:21:11.725639 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:12.387426 (+661787us) service_pool.cc:224] Handling call
0709 13:21:12.409587 (+ 22161us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:21:12.836040 (+426453us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:21:12.836122 (+    82us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:21:12.836209 (+    87us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:21:12.836290 (+    81us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:12.968717 (+132427us) spinlock_profiling.cc:243] Waited 426 ms on lock 0x7b540013f1d4. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:21:12.968786 (+    69us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:21:12.969043 (+   257us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":979082624}
I20260709 13:21:12.980048 21281 raft_consensus.cc:2804] T 746aabf856ab401c9ef5e859371e7b67 P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:21:12.978435 21223 leader_election.cc:290] T d2ab0ac8347641e6955e9bce7d90c1a2 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 14 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
W20260709 13:21:12.979465 16266 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:21:12.975102 16142 rpcz_store.cc:269] 0709 13:21:08.230004 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:08.249923 (+ 19919us) service_pool.cc:224] Handling call
0709 13:21:12.972400 (+4722477us) spinlock_profiling.cc:243] Waited 4.07 s on lock 0x7b54000e9f54. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:21:12.972630 (+   230us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":9369811712}
I20260709 13:21:12.980571 21281 raft_consensus.cc:493] T 746aabf856ab401c9ef5e859371e7b67 P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:21:12.982522 16191 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.594s	user 0.010s	sys 0.025s
I20260709 13:21:12.984127 16142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3884f864a96d428785895e5463e5728f" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:12.986342 16151 raft_consensus.cc:1275] T 23f82224f4564fea93572e8206bad1fe P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:21:12.980360 16218 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57600 (request call id 1621) took 3746 ms. Trace:
I20260709 13:21:12.931136 21276 raft_consensus.cc:515] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:21:12.990746 16264 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:12.980532 21057 leader_election.cc:290] T 26354361e9b044cea1c2b86c4882fe92 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 11 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
W20260709 13:21:12.994196 16115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 497672us
I20260709 13:21:13.001140 16823 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49892 (request call id 747) took 1483 ms. Trace:
I20260709 13:21:13.001672 16823 rpcz_store.cc:276] 0709 13:21:11.517633 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:11.517777 (+   144us) service_pool.cc:224] Handling call
0709 13:21:11.518139 (+   362us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:21:11.636825 (+118686us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:13.000417 (+1363592us) spinlock_profiling.cc:243] Waited 118 ms on lock 0x7b54007d0354. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:21:13.000889 (+   472us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":270636928}
I20260709 13:21:12.988210 16218 rpcz_store.cc:276] 0709 13:21:09.233345 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:10.938570 (+1705225us) service_pool.cc:224] Handling call
0709 13:21:10.959128 (+ 20558us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:21:11.671384 (+712256us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:12.979769 (+1308385us) spinlock_profiling.cc:243] Waited 712 ms on lock 0x7b5400245254. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:21:12.980111 (+   342us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1636575104}
I20260709 13:21:13.005141 20918 raft_consensus.cc:493] T 26354361e9b044cea1c2b86c4882fe92 P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:13.006327 20918 raft_consensus.cc:515] T 26354361e9b044cea1c2b86c4882fe92 P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:13.010475 20918 leader_election.cc:290] T 26354361e9b044cea1c2b86c4882fe92 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
W20260709 13:21:13.011310 16789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 64055.3us
I20260709 13:21:13.014290 16219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a96373bb0e14dce8a266b2fc6c58507" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:13.016819 16219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806d7690c2b644959152074bb2cab0a7" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:13.017885 21211 raft_consensus.cc:3055] T 66ae176961b74813a58bc15c9c4c6f2a P 90f3da57796b488c9087a2f523adf045 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:21:13.018359 21211 raft_consensus.cc:740] T 66ae176961b74813a58bc15c9c4c6f2a P 90f3da57796b488c9087a2f523adf045 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:21:12.993693 16819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9944d5e4b8c4109af9955c4f08c9095" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:21:13.023679 16819 raft_consensus.cc:3060] T c9944d5e4b8c4109af9955c4f08c9095 P cb82cc7bd6db46d980c75237a1402860 [term 13 FOLLOWER]: Advancing to term 15
I20260709 13:21:13.025559 20999 leader_election.cc:290] T 9c61a39fd4f84424b129c1a6f9ba404d P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 14 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
W20260709 13:21:13.025943 21266 log.cc:927] Time spent T 8c11074579b046d99e585c2818e18a2c P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 0.761s	user 0.000s	sys 0.001s
I20260709 13:21:12.991115 21276 leader_election.cc:290] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:21:12.988194 21281 raft_consensus.cc:3060] T 746aabf856ab401c9ef5e859371e7b67 P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:21:13.029371 16224 raft_consensus.cc:1275] T ecc1e363230043fe9fef390f0b979f20 P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 13:21:13.032145 21191 leader_election.cc:290] T c4f65cdf00b748bb963ee6e81ef43b79 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
W20260709 13:21:13.034061 16116 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.934s	user 0.003s	sys 0.015s
I20260709 13:21:13.033651 21211 consensus_queue.cc:260] T 66ae176961b74813a58bc15c9c4c6f2a P 90f3da57796b488c9087a2f523adf045 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:21:13.034457 16116 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.935s	user 0.003s	sys 0.015s
I20260709 13:21:12.993091 21170 raft_consensus.cc:3060] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:21:11.215559 16145 rpcz_store.cc:276] 0709 13:21:09.216860 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:11.085415 (+1868555us) service_pool.cc:224] Handling call
0709 13:21:11.100254 (+ 14839us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:21:11.100513 (+   259us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:21:11.100850 (+   337us) log.cc:844] Serialized 14 byte log entry
0709 13:21:11.214625 (+113775us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:21:11.214671 (+    46us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:21:11.214710 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:11.214794 (+    84us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:21:11.214918 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1947,"prepare.queue_time_us":3897,"prepare.run_cpu_time_us":30,"prepare.run_wall_time_us":31,"raft.queue_time_us":2680,"thread_start_us":110915,"threads_started":1}
I20260709 13:21:13.039386 20894 raft_consensus.cc:493] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:21:13.040347 20894 raft_consensus.cc:515] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:13.044430 16146 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46224 (request call id 749) took 3850 ms. Trace:
I20260709 13:21:13.044893 16146 rpcz_store.cc:276] 0709 13:21:09.193615 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:11.017617 (+1824002us) service_pool.cc:224] Handling call
0709 13:21:11.626759 (+609142us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:21:11.633184 (+  6425us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:13.043937 (+1410753us) spinlock_profiling.cc:243] Waited 609 ms on lock 0x7b54000e9050. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5e8ad 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:21:13.044173 (+   236us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1400178560}
I20260709 13:21:13.032248 16190 leader_election.cc:304] T c2f5f89186204242bf377f7c42b2e2da P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:21:13.046190 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d16dfad0c3b540e5b4b86492ddf87208" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:13.046635 21034 raft_consensus.cc:2749] T c2f5f89186204242bf377f7c42b2e2da P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:21:13.047235 16297 raft_consensus.cc:2393] T d16dfad0c3b540e5b4b86492ddf87208 P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 13: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
I20260709 13:21:13.046669 16146 raft_consensus.cc:1240] T 8cc959fb7f704fba9009c0595467d074 P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 12. Current term is 13. Ops: []
I20260709 13:21:13.034996 21211 raft_consensus.cc:3060] T 66ae176961b74813a58bc15c9c4c6f2a P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:21:13.048323 16149 raft_consensus.cc:1240] T 23f82224f4564fea93572e8206bad1fe P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 11. Current term is 12. Ops: [11.7-11.7]
I20260709 13:21:13.049275 16146 raft_consensus.cc:1240] T fffab26427e24d398e2695aad40ca134 P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 11. Current term is 12. Ops: [11.7-11.7]
I20260709 13:21:13.049762 16149 raft_consensus.cc:1240] T 1df93779209748b9860f6d893dd3de53 P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 12. Current term is 13. Ops: []
I20260709 13:21:13.049666 16295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a9de347c711477e87c582d003f36925" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:21:13.051153 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8158bbd17b994d2eb96f560bd0f1b88d" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
W20260709 13:21:13.051520 16148 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40844 (request call id 2509) took 3830 ms (client timeout 1540 ms). Trace:
I20260709 13:21:13.051827 16147 raft_consensus.cc:1240] T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 12. Current term is 13. Ops: [12.11-12.11]
W20260709 13:21:13.051971 16148 rpcz_store.cc:269] 0709 13:21:09.221250 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:09.262002 (+ 40752us) service_pool.cc:224] Handling call
0709 13:21:13.050984 (+3788982us) spinlock_profiling.cc:243] Waited 1.46 s on lock 0x7b540041ebd4. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:21:13.051281 (+   297us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3363176320}
I20260709 13:21:13.053471 16147 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46224 (request call id 746) took 3860 ms. Trace:
I20260709 13:21:13.053867 16147 rpcz_store.cc:276] 0709 13:21:09.192615 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:11.001457 (+1808842us) service_pool.cc:224] Handling call
0709 13:21:13.051246 (+2049789us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:21:13.052465 (+  1219us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:13.053254 (+   789us) spinlock_profiling.cc:243] Waited 2.05 s on lock 0x7b540041ebd0. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5e8ad 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:21:13.053399 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4712906624}
I20260709 13:21:13.054606 16190 leader_election.cc:304] T 19baa7bc367844c288bca8f81439c413 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860
W20260709 13:21:13.034963 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 935572us
I20260709 13:21:13.033443 16151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87ac0d1f7e8a419f9bcbc04aee0e067e" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
W20260709 13:21:13.059460 16294 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48684 (request call id 2660) took 2249 ms (client timeout 1554 ms). Trace:
I20260709 13:21:13.090175 16218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68a6838d82ef4cac83507f906139d90f" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
W20260709 13:21:13.090315 16294 rpcz_store.cc:269] 0709 13:21:10.810178 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:10.816436 (+  6258us) service_pool.cc:224] Handling call
0709 13:21:13.055018 (+2238582us) spinlock_profiling.cc:243] Waited 77.2 ms on lock 0x7b540032fd54. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:21:13.059237 (+  4219us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":177612672}
I20260709 13:21:13.059099 20236 leader_election.cc:290] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:13.092077 16218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c11074579b046d99e585c2818e18a2c" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 14 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:13.060488 16821 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49892 (request call id 744) took 1563 ms. Trace:
I20260709 13:21:13.092696 16821 rpcz_store.cc:276] 0709 13:21:11.496771 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:11.496939 (+   168us) service_pool.cc:224] Handling call
0709 13:21:11.497391 (+   452us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:21:11.540550 (+ 43159us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:13.059872 (+1519322us) spinlock_profiling.cc:243] Waited 36.6 ms on lock 0x7b54004e0cd4. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:21:13.060271 (+   399us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":84280576}
I20260709 13:21:13.093242 16298 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48684 (request call id 2666) took 2220 ms. Trace:
I20260709 13:21:13.093466 16226 raft_consensus.cc:1217] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac [term 15 LEADER]: Deduplicated request from leader. Original: 11.8->[13.9-14.10]   Dedup: 13.9->[14.10-14.10]
I20260709 13:21:13.094099 16226 raft_consensus.cc:1240] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac [term 15 LEADER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 14. Current term is 15. Ops: [13.9-14.10]
I20260709 13:21:13.068456 16145 raft_consensus.cc:1275] T 9e65676b21b748728ad39668cfe722ec P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:21:13.095413 16145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806d7690c2b644959152074bb2cab0a7" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:13.095852 21151 raft_consensus.cc:3060] T af57996a8be84d398f0f023e8d08ef77 P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:21:13.096200 16145 raft_consensus.cc:2468] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 12.
I20260709 13:21:13.097426 16145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8158bbd17b994d2eb96f560bd0f1b88d" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:21:13.098250 16145 raft_consensus.cc:2410] T 8158bbd17b994d2eb96f560bd0f1b88d P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 for term 13 because replica has last-logged OpId of term: 12 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 8.
W20260709 13:21:13.064978 16189 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:21:13.099612 16189 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:21:13.100165 16189 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:13.101127 21154 raft_consensus.cc:493] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:21:13.101894 21154 raft_consensus.cc:515] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:13.105115 21154 leader_election.cc:290] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:13.106745 16224 raft_consensus.cc:1275] T 08c74490c40243a88e62d1760b5cd814 P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 13:21:13.071066 20972 raft_consensus.cc:3055] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:21:13.108672 20972 raft_consensus.cc:740] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:21:13.108958 21165 raft_consensus.cc:3060] T 3884f864a96d428785895e5463e5728f P 647a343a542d4a139fe106c19095129b [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:21:13.108858 16292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c61a39fd4f84424b129c1a6f9ba404d" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
W20260709 13:21:13.016746 16790 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:13.075688 16150 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46224 (request call id 764) took 1668 ms. Trace:
I20260709 13:21:13.099023 16145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a96373bb0e14dce8a266b2fc6c58507" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:13.083127 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d16dfad0c3b540e5b4b86492ddf87208" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:21:13.113628 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3884f864a96d428785895e5463e5728f" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:21:13.082326 16148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a959dbecc2648eb85d1cc550f5a3245" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:13.084285 16146 raft_consensus.cc:1240] T c0eab25ce4c64e018a041beeda2867db P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 12. Current term is 13. Ops: []
I20260709 13:21:13.087742 16151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68a6838d82ef4cac83507f906139d90f" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
W20260709 13:21:13.088315 16789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 75111.8us
I20260709 13:21:13.088692 16190 leader_election.cc:304] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: cb82cc7bd6db46d980c75237a1402860
W20260709 13:21:13.086037 16116 connection.cc:256] RPC call timeout handler was delayed by -1.0721686190001947s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1643s for a response
I20260709 13:21:13.088758 16144 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40844 (request call id 2521) took 1392 ms. Trace:
W20260709 13:21:13.070448 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 681664us
I20260709 13:21:13.111621 16150 rpcz_store.cc:276] 0709 13:21:11.407017 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:11.635098 (+228081us) service_pool.cc:224] Handling call
0709 13:21:11.662603 (+ 27505us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:21:12.919558 (+1256955us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:13.075314 (+155756us) spinlock_profiling.cc:243] Waited 1.17 s on lock 0x7b5400073ad4. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:21:13.075607 (+   293us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2702223232}
I20260709 13:21:13.081655 16147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c61a39fd4f84424b129c1a6f9ba404d" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:13.093729 16298 rpcz_store.cc:276] 0709 13:21:10.873331 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:10.873442 (+   111us) service_pool.cc:224] Handling call
0709 13:21:10.873729 (+   287us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:21:12.925771 (+2052042us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:13.092960 (+167189us) spinlock_profiling.cc:243] Waited 1.98 s on lock 0x7b54002e1654. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:21:13.093160 (+   200us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4563831168}
I20260709 13:21:13.134982 15046 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:21:13.133373 16144 rpcz_store.cc:276] 0709 13:21:11.696383 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:11.984737 (+288354us) service_pool.cc:224] Handling call
0709 13:21:11.985006 (+   269us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:21:12.958541 (+973535us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:13.088454 (+129913us) spinlock_profiling.cc:243] Waited 973 ms on lock 0x7b54000d0f54. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:21:13.088694 (+   240us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2236936960}
I20260709 13:21:13.043009 20894 leader_election.cc:290] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
W20260709 13:21:13.141144 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 52640.4us
I20260709 13:21:13.142282 21281 raft_consensus.cc:515] T 746aabf856ab401c9ef5e859371e7b67 P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:13.181207 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434bff3082d141838068f323bdb0d1ed" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:21:13.193951 21281 leader_election.cc:290] T 746aabf856ab401c9ef5e859371e7b67 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 13 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:13.089133 21084 raft_consensus.cc:3060] T 19baa7bc367844c288bca8f81439c413 P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:21:13.159396 16265 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.14.177.129:36741 timed out after 1.788s (SENT) [suppressed 28 similar messages]
I20260709 13:21:13.170599 16147 raft_consensus.cc:2393] T 9c61a39fd4f84424b129c1a6f9ba404d P 647a343a542d4a139fe106c19095129b [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 14: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
I20260709 13:21:13.196766 16147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66f75124215f4ca6bef831e174a792d8" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:13.197333 16147 raft_consensus.cc:2468] T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 13.
I20260709 13:21:13.198415 16147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "829b4ad8c5a54eababeea1181b960d4b" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:21:13.174357 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c11074579b046d99e585c2818e18a2c" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:13.165820 21245 consensus_queue.cc:1048] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.034s
I20260709 13:21:13.200644 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ad5dbfa99ee4b8d9e28be92b3f36db5" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:13.201171 16218 raft_consensus.cc:1275] T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:21:13.200639 16226 raft_consensus.cc:1275] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 13:21:13.188624 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a378d7f796f94615b3b347119a9880fd" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 16 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:12.921633 16291 rpcz_store.cc:276] 0709 13:21:11.668541 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:11.873458 (+204917us) service_pool.cc:224] Handling call
0709 13:21:11.874867 (+  1409us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:21:12.136349 (+261482us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:21:12.136396 (+    47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:21:12.136930 (+   534us) log.cc:844] Serialized 26 byte log entry
0709 13:21:12.858275 (+721345us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:21:12.859209 (+   934us) log.cc:844] Serialized 14 byte log entry
0709 13:21:12.859588 (+   379us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:12.859752 (+   164us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:21:12.859812 (+    60us) raft_consensus.cc:1685] finished
0709 13:21:12.859826 (+    14us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:21:12.860034 (+   208us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":212514,"prepare.queue_time_us":39631,"prepare.run_cpu_time_us":89,"prepare.run_wall_time_us":3222,"raft.queue_time_us":255677,"raft.run_cpu_time_us":131,"raft.run_wall_time_us":88513,"spinlock_wait_cycles":22110976,"thread_start_us":488104,"threads_started":1,"wal-append.queue_time_us":497018}
I20260709 13:21:13.203456 16299 raft_consensus.cc:2410] T a378d7f796f94615b3b347119a9880fd P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 for term 16 because replica has last-logged OpId of term: 14 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 9.
I20260709 13:21:13.110180 20972 consensus_queue.cc:260] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:13.204108 16821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b7d1ab519044eda2d5e04a07242f25" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:13.204406 16820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746aabf856ab401c9ef5e859371e7b67" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:21:13.204696 20972 raft_consensus.cc:3060] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:21:13.205026 16820 raft_consensus.cc:3060] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:21:13.161046 16115 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:13.207046 21282 raft_consensus.cc:493] T 1df93779209748b9860f6d893dd3de53 P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:13.175233 16292 raft_consensus.cc:3055] T 23f82224f4564fea93572e8206bad1fe P 90f3da57796b488c9087a2f523adf045 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:21:13.207973 16292 raft_consensus.cc:740] T 23f82224f4564fea93572e8206bad1fe P 90f3da57796b488c9087a2f523adf045 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:21:13.207643 21282 raft_consensus.cc:515] T 1df93779209748b9860f6d893dd3de53 P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:13.209043 16292 consensus_queue.cc:260] T 23f82224f4564fea93572e8206bad1fe P 90f3da57796b488c9087a2f523adf045 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:13.210107 21282 leader_election.cc:290] T 1df93779209748b9860f6d893dd3de53 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:13.207461 21126 raft_consensus.cc:493] T 08c74490c40243a88e62d1760b5cd814 P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:13.086556 20991 raft_consensus.cc:493] T 8d4ab870a05347868956aa8b841b2981 P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:13.210140 16292 raft_consensus.cc:3060] T 23f82224f4564fea93572e8206bad1fe P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:21:13.149453 16822 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49892 (request call id 754) took 1428 ms. Trace:
I20260709 13:21:13.170063 16149 raft_consensus.cc:1240] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 14. Current term is 15. Ops: []
I20260709 13:21:13.163846 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31969392ccdb43b491911a319e8bc7e1" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:13.170395 16144 raft_consensus.cc:1240] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 10. Current term is 11. Ops: []
I20260709 13:21:13.152474 16151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f2da98864b4425bf7c0055951fdd4b" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:13.168699 16148 raft_consensus.cc:1275] T 8158bbd17b994d2eb96f560bd0f1b88d P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 13 index: 9. (term mismatch)
I20260709 13:21:13.164407 16145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434bff3082d141838068f323bdb0d1ed" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:21:13.203188 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f2da98864b4425bf7c0055951fdd4b" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:13.165442 16116 leader_election.cc:304] T c0eab25ce4c64e018a041beeda2867db P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:21:13.214721 16148 pending_rounds.cc:85] T 8158bbd17b994d2eb96f560bd0f1b88d P 647a343a542d4a139fe106c19095129b: Aborting all ops after (but not including) 8
I20260709 13:21:13.220875 16148 pending_rounds.cc:107] T 8158bbd17b994d2eb96f560bd0f1b88d P 647a343a542d4a139fe106c19095129b: Aborting uncommitted NO_OP operation due to leader change: 12.9
I20260709 13:21:13.221201 16148 raft_consensus.cc:2889] T 8158bbd17b994d2eb96f560bd0f1b88d P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:21:13.170960 16146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ad5dbfa99ee4b8d9e28be92b3f36db5" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:13.153177 16191 leader_election.cc:304] T fffab26427e24d398e2695aad40ca134 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 90f3da57796b488c9087a2f523adf045
I20260709 13:21:13.213338 16822 rpcz_store.cc:276] 0709 13:21:11.721380 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:11.721547 (+   167us) service_pool.cc:224] Handling call
0709 13:21:11.721842 (+   295us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:21:12.920673 (+1198831us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:13.137302 (+216629us) spinlock_profiling.cc:243] Waited 1.19 s on lock 0x7b540047fc54. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:21:13.149380 (+ 12078us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2742256384}
I20260709 13:21:13.160441 16117 leader_election.cc:304] T 829b4ad8c5a54eababeea1181b960d4b P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:21:13.086333 21106 raft_consensus.cc:493] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
W20260709 13:21:13.146739 21141 consensus_meta.cc:287] T 84f2da98864b4425bf7c0055951fdd4b P 1ca2725d290e47d68b1a4b609c9816ac: Time spent flushing consensus metadata: real 0.634s	user 0.007s	sys 0.011s
I20260709 13:21:13.151386 20961 raft_consensus.cc:2749] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:21:13.211325 21126 raft_consensus.cc:515] T 08c74490c40243a88e62d1760b5cd814 P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:13.238127 21245 raft_consensus.cc:2749] T c0eab25ce4c64e018a041beeda2867db P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:21:13.230970 16144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26354361e9b044cea1c2b86c4882fe92" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:21:13.239719 21126 leader_election.cc:290] T 08c74490c40243a88e62d1760b5cd814 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:13.239876 16292 raft_consensus.cc:1275] T 23f82224f4564fea93572e8206bad1fe P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 7. (term mismatch)
I20260709 13:21:13.224128 21089 raft_consensus.cc:515] T 2794af5b99984efebdee564035841cb0 P cb82cc7bd6db46d980c75237a1402860 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:13.234251 21250 raft_consensus.cc:493] T 7c2095a5f4ad49408dbc7f69c606120d P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:21:13.235422 21158 raft_consensus.cc:493] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:21:13.235445 16117 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 75116.3us
W20260709 13:21:13.236061 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 70762.7us
I20260709 13:21:13.166689 21180 raft_consensus.cc:515] T e4dc6b0541ca431ebc757beac00521f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:13.211436 20991 raft_consensus.cc:515] T 8d4ab870a05347868956aa8b841b2981 P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:13.232417 16148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62fe3346cd3c42c4a55e14af9073bca4" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:13.238145 16149 raft_consensus.cc:1240] T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 14. Current term is 15. Ops: [13.9-14.10]
I20260709 13:21:13.248492 16819 raft_consensus.cc:2468] T c9944d5e4b8c4109af9955c4f08c9095 P cb82cc7bd6db46d980c75237a1402860 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 15.
I20260709 13:21:13.240496 16292 pending_rounds.cc:85] T 23f82224f4564fea93572e8206bad1fe P 90f3da57796b488c9087a2f523adf045: Aborting all ops after (but not including) 6
I20260709 13:21:13.149051 21087 raft_consensus.cc:2804] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:21:13.166396 20894 raft_consensus.cc:493] T d2ab0ac8347641e6955e9bce7d90c1a2 P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:21:13.252593 16292 pending_rounds.cc:107] T 23f82224f4564fea93572e8206bad1fe P 90f3da57796b488c9087a2f523adf045: Aborting uncommitted NO_OP operation due to leader change: 11.7
I20260709 13:21:13.264796 16292 raft_consensus.cc:2889] T 23f82224f4564fea93572e8206bad1fe P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:21:13.243973 21084 raft_consensus.cc:2749] T 19baa7bc367844c288bca8f81439c413 P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:21:13.281702 16116 leader_election.cc:304] T ecc1e363230043fe9fef390f0b979f20 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:21:13.285235 21281 raft_consensus.cc:2804] T fffab26427e24d398e2695aad40ca134 P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:21:13.264061 21087 raft_consensus.cc:493] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:13.293999 16192 leader_election.cc:304] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:21:13.243773 21141 raft_consensus.cc:515] T 84f2da98864b4425bf7c0055951fdd4b P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:13.293670 16291 raft_consensus.cc:1240] T 3884f864a96d428785895e5463e5728f P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 8. Current term is 9. Ops: []
I20260709 13:21:13.293982 16151 raft_consensus.cc:2468] T 84f2da98864b4425bf7c0055951fdd4b P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 9.
I20260709 13:21:13.248319 20961 consensus_queue.cc:1048] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.047s
I20260709 13:21:13.306754 16148 raft_consensus.cc:2468] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 17.
W20260709 13:21:13.313773 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 160674us
I20260709 13:21:13.316316 16191 leader_election.cc:304] T 18ad7e3af5504571a69d10c00750a01c P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 90f3da57796b488c9087a2f523adf045
I20260709 13:21:13.318087 21180 leader_election.cc:290] T e4dc6b0541ca431ebc757beac00521f1 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 13 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:13.297762 21250 raft_consensus.cc:515] T 7c2095a5f4ad49408dbc7f69c606120d P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:13.321898 20991 leader_election.cc:290] T 8d4ab870a05347868956aa8b841b2981 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:13.323438 21250 leader_election.cc:290] T 7c2095a5f4ad49408dbc7f69c606120d P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:13.308950 16117 leader_election.cc:304] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860
I20260709 13:21:13.326246 20236 raft_consensus.cc:2749] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
W20260709 13:21:13.310915 16790 leader_election.cc:336] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943): Timed out: RequestConsensusVote RPC to 127.14.177.130:35943 timed out after 1.870s (SENT)
I20260709 13:21:13.327617 16147 raft_consensus.cc:2468] T 829b4ad8c5a54eababeea1181b960d4b P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 13.
I20260709 13:21:13.335459 21281 raft_consensus.cc:697] T fffab26427e24d398e2695aad40ca134 P 1ca2725d290e47d68b1a4b609c9816ac [term 12 LEADER]: Becoming Leader. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:21:13.336593 21281 consensus_queue.cc:237] T fffab26427e24d398e2695aad40ca134 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:13.337229 16819 raft_consensus.cc:1240] T 19baa7bc367844c288bca8f81439c413 P cb82cc7bd6db46d980c75237a1402860 [term 15 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 14. Current term is 15. Ops: []
I20260709 13:21:13.340063 16822 raft_consensus.cc:1240] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [term 16 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 15. Current term is 16. Ops: []
I20260709 13:21:13.294488 20894 raft_consensus.cc:515] T d2ab0ac8347641e6955e9bce7d90c1a2 P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:13.295562 21087 raft_consensus.cc:3060] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:21:13.345389 16192 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 67154.7us
I20260709 13:21:13.346349 16217 raft_consensus.cc:2393] T 84f2da98864b4425bf7c0055951fdd4b P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 11: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
I20260709 13:21:13.330634 16790 leader_election.cc:304] T b87147731a68400e8cce9b879daa7c5d P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860; no voters: 90f3da57796b488c9087a2f523adf045
I20260709 13:21:13.354000 16291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1df93779209748b9860f6d893dd3de53" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
W20260709 13:21:13.354835 16788 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:13.343974 21282 raft_consensus.cc:2804] T 829b4ad8c5a54eababeea1181b960d4b P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:21:13.355515 16821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26354361e9b044cea1c2b86c4882fe92" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:13.357836 16820 raft_consensus.cc:2468] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 13.
I20260709 13:21:13.357875 16821 raft_consensus.cc:2468] T 26354361e9b044cea1c2b86c4882fe92 P cb82cc7bd6db46d980c75237a1402860 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 11.
I20260709 13:21:13.291356 21106 raft_consensus.cc:515] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:21:13.364022 16192 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:21:13.364578 16192 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:13.360764 16144 raft_consensus.cc:2468] T 26354361e9b044cea1c2b86c4882fe92 P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 11.
W20260709 13:21:13.368228 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 122791us
I20260709 13:21:13.303191 21158 raft_consensus.cc:515] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:13.377398 21151 raft_consensus.cc:2804] T 18ad7e3af5504571a69d10c00750a01c P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:21:13.378079 21151 raft_consensus.cc:493] T 18ad7e3af5504571a69d10c00750a01c P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Starting leader election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:21:13.378598 21151 raft_consensus.cc:3060] T 18ad7e3af5504571a69d10c00750a01c P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:21:13.365636 21084 raft_consensus.cc:2804] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:21:13.389358 16146 raft_consensus.cc:2410] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 for term 13 because replica has last-logged OpId of term: 10 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
W20260709 13:21:13.388998 16117 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 62015.9us
W20260709 13:21:13.401140 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 84956.1us
W20260709 13:21:13.402134 16263 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:13.380679 20894 leader_election.cc:290] T d2ab0ac8347641e6955e9bce7d90c1a2 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:13.405393 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d4ab870a05347868956aa8b841b2981" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:13.406646 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31969392ccdb43b491911a319e8bc7e1" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
W20260709 13:21:13.389772 20967 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.577s	user 0.002s	sys 0.035s
I20260709 13:21:13.384057 21084 raft_consensus.cc:697] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac [term 15 LEADER]: Becoming Leader. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
W20260709 13:21:13.413722 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 83176.8us
W20260709 13:21:13.414028 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 278353us
I20260709 13:21:13.400055 21245 raft_consensus.cc:2749] T ecc1e363230043fe9fef390f0b979f20 P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:21:13.383822 21182 raft_consensus.cc:493] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
W20260709 13:21:13.409337 20967 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.689s	user 0.003s	sys 0.036s
W20260709 13:21:13.414911 16265 connection.cc:256] RPC call timeout handler was delayed by -1.9332717019999563s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.205s for a response
I20260709 13:21:13.416143 16117 leader_election.cc:304] T 57a744f5aaf944678a63469cbf514015 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860
W20260709 13:21:13.421133 16264 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:21:13.431660 16263 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:13.415254 21089 leader_election.cc:290] T 2794af5b99984efebdee564035841cb0 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 11 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
W20260709 13:21:13.444285 16265 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:13.447816 16192 leader_election.cc:304] T 746aabf856ab401c9ef5e859371e7b67 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:21:13.449714 16296 raft_consensus.cc:1240] T d16dfad0c3b540e5b4b86492ddf87208 P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 12. Current term is 13. Ops: []
W20260709 13:21:13.450188 16265 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:13.412161 21276 consensus_queue.cc:1048] T 23f82224f4564fea93572e8206bad1fe P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.025s
W20260709 13:21:13.454384 16117 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 44684us
I20260709 13:21:13.491279 16117 leader_election.cc:304] T b87147731a68400e8cce9b879daa7c5d P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860
I20260709 13:21:13.454821 21182 raft_consensus.cc:515] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:13.467752 21084 consensus_queue.cc:237] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:21:13.470117 16264 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:13.492678 16219 raft_consensus.cc:2468] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 13.
I20260709 13:21:13.472828 16295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ab0ac8347641e6955e9bce7d90c1a2" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:21:13.494488 16295 raft_consensus.cc:3060] T d2ab0ac8347641e6955e9bce7d90c1a2 P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:21:13.474061 16291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c74490c40243a88e62d1760b5cd814" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:13.477097 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26354361e9b044cea1c2b86c4882fe92" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:21:13.480123 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2794af5b99984efebdee564035841cb0" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:21:13.484082 21106 leader_election.cc:290] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
W20260709 13:21:13.484911 16189 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:13.485982 21151 raft_consensus.cc:515] T 18ad7e3af5504571a69d10c00750a01c P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } }
I20260709 13:21:13.487041 21141 leader_election.cc:290] T 84f2da98864b4425bf7c0055951fdd4b P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 11 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:13.488445 21282 raft_consensus.cc:493] T 829b4ad8c5a54eababeea1181b960d4b P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:21:13.490125 21259 log.cc:927] Time spent T 5a959dbecc2648eb85d1cc550f5a3245 P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 0.060s	user 0.000s	sys 0.001s
I20260709 13:21:13.490377 21198 raft_consensus.cc:2804] T b87147731a68400e8cce9b879daa7c5d P cb82cc7bd6db46d980c75237a1402860 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:21:13.450965 16191 leader_election.cc:304] T ecc1e363230043fe9fef390f0b979f20 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860; no voters: 90f3da57796b488c9087a2f523adf045
W20260709 13:21:13.493008 16791 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:13.472445 21158 leader_election.cc:290] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:13.448328 21126 raft_consensus.cc:3060] T 57a744f5aaf944678a63469cbf514015 P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:21:13.493712 20902 raft_consensus.cc:493] T ee9ed67006144797b1de6ea8c2013a68 P cb82cc7bd6db46d980c75237a1402860 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:13.498525 16292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ad5dbfa99ee4b8d9e28be92b3f36db5" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:13.500247 16298 raft_consensus.cc:2393] T 26354361e9b044cea1c2b86c4882fe92 P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 in current term 11: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
I20260709 13:21:13.501362 16116 leader_election.cc:304] T 8cc959fb7f704fba9009c0595467d074 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:21:13.504777 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c74490c40243a88e62d1760b5cd814" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 14 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:13.492030 21154 raft_consensus.cc:2804] T 746aabf856ab401c9ef5e859371e7b67 P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Leader election won for term 13
W20260709 13:21:13.497411 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 44090.3us
I20260709 13:21:13.502506 21282 raft_consensus.cc:3060] T 829b4ad8c5a54eababeea1181b960d4b P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:21:13.511622 16142 raft_consensus.cc:2468] T 3884f864a96d428785895e5463e5728f P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 9.
I20260709 13:21:13.513326 20902 raft_consensus.cc:515] T ee9ed67006144797b1de6ea8c2013a68 P cb82cc7bd6db46d980c75237a1402860 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:13.514606 21198 raft_consensus.cc:697] T b87147731a68400e8cce9b879daa7c5d P cb82cc7bd6db46d980c75237a1402860 [term 16 LEADER]: Becoming Leader. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
W20260709 13:21:13.516078 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 82947.4us
I20260709 13:21:13.517192 16191 leader_election.cc:304] T 02838c86531d4f98bf58d1c22b8baf77 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
W20260709 13:21:13.521195 16264 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:13.515744 21198 consensus_queue.cc:237] T b87147731a68400e8cce9b879daa7c5d P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:13.528136 16822 raft_consensus.cc:1240] T 2794af5b99984efebdee564035841cb0 P cb82cc7bd6db46d980c75237a1402860 [term 11 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 10. Current term is 11. Ops: []
I20260709 13:21:13.534770 21281 raft_consensus.cc:2749] T 02838c86531d4f98bf58d1c22b8baf77 P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:21:13.540047 16817 raft_consensus.cc:1240] T 434bff3082d141838068f323bdb0d1ed P cb82cc7bd6db46d980c75237a1402860 [term 13 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 12. Current term is 13. Ops: []
I20260709 13:21:13.542824 21182 leader_election.cc:290] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:13.543437 16819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdbd78acb4cf46298e0de8ba0d130e19" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:13.544116 16819 raft_consensus.cc:2468] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 15.
I20260709 13:21:13.545897 16820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a9de347c711477e87c582d003f36925" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:21:13.547047 16821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2372981006d4b218215c28964dea623" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:21:13.535490 21106 raft_consensus.cc:2749] T 8cc959fb7f704fba9009c0595467d074 P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:21:13.548200 21161 raft_consensus.cc:493] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:13.548756 21161 raft_consensus.cc:515] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:13.549125 21089 consensus_queue.cc:1048] T 23f82224f4564fea93572e8206bad1fe P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:21:13.553018 16192 leader_election.cc:304] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:21:13.531029 21154 raft_consensus.cc:697] T 746aabf856ab401c9ef5e859371e7b67 P 1ca2725d290e47d68b1a4b609c9816ac [term 13 LEADER]: Becoming Leader. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:21:13.554559 21154 consensus_queue.cc:237] T 746aabf856ab401c9ef5e859371e7b67 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:13.555228 21250 raft_consensus.cc:2804] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:21:13.555805 21250 raft_consensus.cc:493] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:13.556281 21250 raft_consensus.cc:3060] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:21:13.558987 21276 raft_consensus.cc:993] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P cb82cc7bd6db46d980c75237a1402860: : Instructing follower 90f3da57796b488c9087a2f523adf045 to start an election
I20260709 13:21:13.555821 21141 consensus_queue.cc:1048] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.004s
I20260709 13:21:13.528143 21170 raft_consensus.cc:2804] T ecc1e363230043fe9fef390f0b979f20 P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:21:13.560359 21170 raft_consensus.cc:493] T ecc1e363230043fe9fef390f0b979f20 P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:13.560731 21170 raft_consensus.cc:3060] T ecc1e363230043fe9fef390f0b979f20 P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:21:13.561736 16824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62fe3346cd3c42c4a55e14af9073bca4" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
W20260709 13:21:13.559695 16264 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:13.563426 20902 leader_election.cc:290] T ee9ed67006144797b1de6ea8c2013a68 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:21:13.564306 16299 raft_consensus.cc:1240] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 12. Current term is 13. Ops: []
I20260709 13:21:13.559585 16191 leader_election.cc:304] T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 90f3da57796b488c9087a2f523adf045
I20260709 13:21:13.566164 16191 leader_election.cc:304] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
W20260709 13:21:13.566601 16263 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:13.536515 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9944d5e4b8c4109af9955c4f08c9095" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:21:13.529657 21097 raft_consensus.cc:2749] T b87147731a68400e8cce9b879daa7c5d P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:21:13.577471 16117 leader_election.cc:304] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860
I20260709 13:21:13.586079 16821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8638ee363273426db1393dd81e5c7b72" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:13.559536 21276 raft_consensus.cc:1081] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P cb82cc7bd6db46d980c75237a1402860 [term 12 LEADER]: Signalling peer 90f3da57796b488c9087a2f523adf045 to start an election
I20260709 13:21:13.610042 16298 raft_consensus.cc:3060] T c9944d5e4b8c4109af9955c4f08c9095 P 90f3da57796b488c9087a2f523adf045 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:21:13.610217 16824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5718c623ebee4d2bb770f0f9ddb2f164" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:13.551023 21161 leader_election.cc:290] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
W20260709 13:21:13.623050 16265 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:13.589828 21106 consensus_queue.cc:1059] T 434bff3082d141838068f323bdb0d1ed P 647a343a542d4a139fe106c19095129b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.040s
W20260709 13:21:13.630645 21290 log.cc:927] Time spent T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860: Append to log took a long time: real 0.064s	user 0.000s	sys 0.000s
I20260709 13:21:13.630352 16292 raft_consensus.cc:1240] T 68a6838d82ef4cac83507f906139d90f P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 14. Current term is 15. Ops: []
I20260709 13:21:13.622220 21182 raft_consensus.cc:2749] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:21:13.635962 21276 raft_consensus.cc:493] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:21:13.630537 21106 raft_consensus.cc:3055] T 434bff3082d141838068f323bdb0d1ed P 647a343a542d4a139fe106c19095129b [term 12 LEADER]: Stepping down as leader of term 12
W20260709 13:21:13.637247 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 103556us
I20260709 13:21:13.650657 16291 raft_consensus.cc:3055] T 8cc959fb7f704fba9009c0595467d074 P 90f3da57796b488c9087a2f523adf045 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:21:13.651237 16291 raft_consensus.cc:740] T 8cc959fb7f704fba9009c0595467d074 P 90f3da57796b488c9087a2f523adf045 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:21:13.623975 21165 raft_consensus.cc:2749] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
W20260709 13:21:13.657487 16117 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 80082.6us
W20260709 13:21:13.664733 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 98679.3us
I20260709 13:21:13.644807 21106 raft_consensus.cc:740] T 434bff3082d141838068f323bdb0d1ed P 647a343a542d4a139fe106c19095129b [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
W20260709 13:21:13.670176 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 177474us
I20260709 13:21:13.664213 21283 raft_consensus.cc:2804] T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:21:13.647032 21276 raft_consensus.cc:515] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:13.711289 16142 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46224 (request call id 767) took 1290 ms. Trace:
I20260709 13:21:13.711867 16142 rpcz_store.cc:276] 0709 13:21:12.408484 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:12.983548 (+575064us) service_pool.cc:224] Handling call
0709 13:21:13.698225 (+714677us) spinlock_profiling.cc:243] Waited 457 ms on lock 0x7b54000730d4. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:21:13.698370 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1051020160}
I20260709 13:21:13.709379 16291 consensus_queue.cc:260] T 8cc959fb7f704fba9009c0595467d074 P 90f3da57796b488c9087a2f523adf045 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:13.699313 21283 raft_consensus.cc:493] T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:13.739144 16191 leader_election.cc:304] T ee9ed67006144797b1de6ea8c2013a68 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: cb82cc7bd6db46d980c75237a1402860
I20260709 13:21:13.746394 16291 raft_consensus.cc:3060] T 8cc959fb7f704fba9009c0595467d074 P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:21:13.747253 21288 log.cc:927] Time spent T 8c11074579b046d99e585c2818e18a2c P cb82cc7bd6db46d980c75237a1402860: Append to log took a long time: real 0.373s	user 0.007s	sys 0.000s
I20260709 13:21:13.748464 21216 consensus_queue.cc:1059] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: INVALID_TERM, Last received: 14.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.065s
I20260709 13:21:13.752519 21182 raft_consensus.cc:2804] T ee9ed67006144797b1de6ea8c2013a68 P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:21:13.753095 21182 raft_consensus.cc:697] T ee9ed67006144797b1de6ea8c2013a68 P 1ca2725d290e47d68b1a4b609c9816ac [term 12 LEADER]: Becoming Leader. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:21:13.753993 21182 consensus_queue.cc:237] T ee9ed67006144797b1de6ea8c2013a68 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:13.755264 21106 consensus_queue.cc:260] T 434bff3082d141838068f323bdb0d1ed P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:13.756937 21106 raft_consensus.cc:3060] T 434bff3082d141838068f323bdb0d1ed P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:21:13.758807 21126 raft_consensus.cc:2749] T 57a744f5aaf944678a63469cbf514015 P 647a343a542d4a139fe106c19095129b [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:21:13.737632 20972 consensus_queue.cc:1048] T b87147731a68400e8cce9b879daa7c5d P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.027s
I20260709 13:21:13.768204 21282 raft_consensus.cc:515] T 829b4ad8c5a54eababeea1181b960d4b P 647a343a542d4a139fe106c19095129b [term 14 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:13.760388 16790 leader_election.cc:304] T 66f75124215f4ca6bef831e174a792d8 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:21:13.769261 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4dc6b0541ca431ebc757beac00521f1" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:21:13.769984 16298 raft_consensus.cc:2468] T c9944d5e4b8c4109af9955c4f08c9095 P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 15.
I20260709 13:21:13.770536 21282 leader_election.cc:290] T 829b4ad8c5a54eababeea1181b960d4b P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 14 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:13.772260 21304 consensus_queue.cc:1048] T 9e65676b21b748728ad39668cfe722ec P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:21:13.773806 21304 consensus_queue.cc:1059] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:21:13.774600 21304 raft_consensus.cc:2804] T 66f75124215f4ca6bef831e174a792d8 P cb82cc7bd6db46d980c75237a1402860 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:21:13.742321 21087 raft_consensus.cc:515] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:13.781497 21087 leader_election.cc:290] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 13 election: Requested vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:13.781731 16294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdbd78acb4cf46298e0de8ba0d130e19" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:13.782490 16294 raft_consensus.cc:2468] T cdbd78acb4cf46298e0de8ba0d130e19 P 90f3da57796b488c9087a2f523adf045 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 14.
W20260709 13:21:13.782749 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 91834.4us
I20260709 13:21:13.783471 16294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18ad7e3af5504571a69d10c00750a01c" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:21:13.783676 16191 leader_election.cc:304] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:21:13.784562 16294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f2da98864b4425bf7c0055951fdd4b" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:21:13.785127 16294 raft_consensus.cc:3060] T 84f2da98864b4425bf7c0055951fdd4b P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:21:13.785758 16191 leader_election.cc:304] T 8a878999c0024d2abe517484180290fc P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:21:13.786602 21154 raft_consensus.cc:2749] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:21:13.741926 21151 leader_election.cc:290] T 18ad7e3af5504571a69d10c00750a01c P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 12 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:13.736961 21163 consensus_queue.cc:1059] T 2794af5b99984efebdee564035841cb0 P 647a343a542d4a139fe106c19095129b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.176s
W20260709 13:21:13.792569 16192 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:13.768659 21276 leader_election.cc:290] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:21:13.770364 16299 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "9e4bfb9090c8460eb14fa7e0d818e7f1"
dest_uuid: "90f3da57796b488c9087a2f523adf045"
 from {username='slave'} at 127.0.0.1:40474
I20260709 13:21:13.803311 16299 raft_consensus.cc:493] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:21:13.803761 16299 raft_consensus.cc:3060] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:21:13.805015 16295 raft_consensus.cc:2468] T d2ab0ac8347641e6955e9bce7d90c1a2 P 90f3da57796b488c9087a2f523adf045 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 14.
I20260709 13:21:13.742494 21283 raft_consensus.cc:3060] T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:21:13.774920 21304 raft_consensus.cc:493] T 66f75124215f4ca6bef831e174a792d8 P cb82cc7bd6db46d980c75237a1402860 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:13.808135 16824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e4bfb9090c8460eb14fa7e0d818e7f1" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:21:13.816941 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c2095a5f4ad49408dbc7f69c606120d" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
W20260709 13:21:13.820102 16789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 76853.7us
I20260709 13:21:13.823413 21126 consensus_queue.cc:1048] T 8a878999c0024d2abe517484180290fc P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:21:13.822851 20964 consensus_queue.cc:1059] T fffab26427e24d398e2695aad40ca134 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.002s
I20260709 13:21:13.825776 20964 raft_consensus.cc:3055] T fffab26427e24d398e2695aad40ca134 P 90f3da57796b488c9087a2f523adf045 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:21:13.826164 20964 raft_consensus.cc:740] T fffab26427e24d398e2695aad40ca134 P 90f3da57796b488c9087a2f523adf045 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:21:13.827152 20964 consensus_queue.cc:260] T fffab26427e24d398e2695aad40ca134 P 90f3da57796b488c9087a2f523adf045 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:13.828588 16266 leader_election.cc:304] T 26354361e9b044cea1c2b86c4882fe92 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:21:13.831889 21170 raft_consensus.cc:515] T ecc1e363230043fe9fef390f0b979f20 P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:13.832922 20972 raft_consensus.cc:2804] T 26354361e9b044cea1c2b86c4882fe92 P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:21:13.833380 20972 raft_consensus.cc:493] T 26354361e9b044cea1c2b86c4882fe92 P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:13.833750 20972 raft_consensus.cc:3060] T 26354361e9b044cea1c2b86c4882fe92 P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:21:13.833917 16266 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:13.836107 20964 raft_consensus.cc:3060] T fffab26427e24d398e2695aad40ca134 P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:21:13.806974 21276 raft_consensus.cc:3055] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:21:13.837000 21276 raft_consensus.cc:740] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:21:13.838014 21276 consensus_queue.cc:260] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:13.839936 21170 leader_election.cc:290] T ecc1e363230043fe9fef390f0b979f20 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 14 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:13.815964 21154 raft_consensus.cc:2749] T 8a878999c0024d2abe517484180290fc P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:21:13.842054 16824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:21:13.842818 16824 raft_consensus.cc:3060] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:21:13.856626 16821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ab0ac8347641e6955e9bce7d90c1a2" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:13.857543 16821 raft_consensus.cc:2393] T d2ab0ac8347641e6955e9bce7d90c1a2 P cb82cc7bd6db46d980c75237a1402860 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 14: Already voted for candidate cb82cc7bd6db46d980c75237a1402860 in this term.
I20260709 13:21:13.858362 16295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8638ee363273426db1393dd81e5c7b72" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
W20260709 13:21:13.856699 16266 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:21:13.859321 16266 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:13.859840 16295 raft_consensus.cc:3055] T 1df93779209748b9860f6d893dd3de53 P 90f3da57796b488c9087a2f523adf045 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:21:13.860244 16295 raft_consensus.cc:740] T 1df93779209748b9860f6d893dd3de53 P 90f3da57796b488c9087a2f523adf045 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:21:13.860375 21154 consensus_queue.cc:1059] T 68a6838d82ef4cac83507f906139d90f P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:21:13.861053 16295 consensus_queue.cc:260] T 1df93779209748b9860f6d893dd3de53 P 90f3da57796b488c9087a2f523adf045 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:13.861613 21170 raft_consensus.cc:3055] T 68a6838d82ef4cac83507f906139d90f P 1ca2725d290e47d68b1a4b609c9816ac [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:21:13.862043 16295 raft_consensus.cc:3060] T 1df93779209748b9860f6d893dd3de53 P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:21:13.862056 21170 raft_consensus.cc:740] T 68a6838d82ef4cac83507f906139d90f P 1ca2725d290e47d68b1a4b609c9816ac [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:21:13.863682 21198 raft_consensus.cc:3055] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:21:13.807891 20991 consensus_queue.cc:1048] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.040s
I20260709 13:21:13.863358 21170 consensus_queue.cc:260] T 68a6838d82ef4cac83507f906139d90f P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:13.864161 21198 raft_consensus.cc:740] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:21:13.865020 21170 raft_consensus.cc:3060] T 68a6838d82ef4cac83507f906139d90f P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:21:13.865121 21198 consensus_queue.cc:260] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:13.866638 21198 raft_consensus.cc:3060] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:21:13.816887 20894 consensus_queue.cc:1059] T b87147731a68400e8cce9b879daa7c5d P 90f3da57796b488c9087a2f523adf045 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.050s
I20260709 13:21:13.863641 21276 raft_consensus.cc:3060] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:21:13.867661 20894 raft_consensus.cc:3055] T b87147731a68400e8cce9b879daa7c5d P 90f3da57796b488c9087a2f523adf045 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:21:13.868098 20894 raft_consensus.cc:740] T b87147731a68400e8cce9b879daa7c5d P 90f3da57796b488c9087a2f523adf045 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
W20260709 13:21:13.870247 21297 log.cc:927] Time spent T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.166s	user 0.001s	sys 0.000s
W20260709 13:21:13.870864 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 83878us
I20260709 13:21:13.814462 21304 raft_consensus.cc:3060] T 66f75124215f4ca6bef831e174a792d8 P cb82cc7bd6db46d980c75237a1402860 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:21:13.871793 16191 leader_election.cc:304] T c0eab25ce4c64e018a041beeda2867db P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:21:13.872681 21151 raft_consensus.cc:3060] T c0eab25ce4c64e018a041beeda2867db P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:21:13.873519 16116 leader_election.cc:304] T 84f2da98864b4425bf7c0055951fdd4b P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 90f3da57796b488c9087a2f523adf045
I20260709 13:21:13.873522 16292 raft_consensus.cc:1240] T d2ab0ac8347641e6955e9bce7d90c1a2 P 90f3da57796b488c9087a2f523adf045 [term 14 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 13. Current term is 14. Ops: []
I20260709 13:21:13.875447 21277 raft_consensus.cc:2804] T 84f2da98864b4425bf7c0055951fdd4b P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:21:13.875905 21277 raft_consensus.cc:493] T 84f2da98864b4425bf7c0055951fdd4b P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Starting leader election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:21:13.876356 21277 raft_consensus.cc:3060] T 84f2da98864b4425bf7c0055951fdd4b P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:21:13.814743 21163 raft_consensus.cc:3055] T 2794af5b99984efebdee564035841cb0 P 647a343a542d4a139fe106c19095129b [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:21:13.877617 21163 raft_consensus.cc:740] T 2794af5b99984efebdee564035841cb0 P 647a343a542d4a139fe106c19095129b [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:21:13.878556 21163 consensus_queue.cc:260] T 2794af5b99984efebdee564035841cb0 P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:13.880086 21163 raft_consensus.cc:3060] T 2794af5b99984efebdee564035841cb0 P 647a343a542d4a139fe106c19095129b [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:21:13.943732 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 84057.3us
W20260709 13:21:13.860080 16266 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:21:13.949383 16264 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:13.951347 16292 raft_consensus.cc:1275] T 68a6838d82ef4cac83507f906139d90f P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
W20260709 13:21:13.950969 16264 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:21:13.948581 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 109861us
I20260709 13:21:13.965696 16821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "829b4ad8c5a54eababeea1181b960d4b" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:21:13.966617 16821 raft_consensus.cc:3060] T 829b4ad8c5a54eababeea1181b960d4b P cb82cc7bd6db46d980c75237a1402860 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:21:13.961483 21250 raft_consensus.cc:515] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:21:13.975144 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 214842us
I20260709 13:21:13.976315 16790 leader_election.cc:304] T 434bff3082d141838068f323bdb0d1ed P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
W20260709 13:21:13.968478 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 95100us
W20260709 13:21:13.969106 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 97426.3us
I20260709 13:21:13.973804 16789 leader_election.cc:304] T d2ab0ac8347641e6955e9bce7d90c1a2 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 
W20260709 13:21:13.972826 16263 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:13.906391 16044 catalog_manager.cc:5697] T af57996a8be84d398f0f023e8d08ef77 P 1ca2725d290e47d68b1a4b609c9816ac reported cstate change: term changed from 7 to 9. New cstate: current_term: 9 leader_uuid: "647a343a542d4a139fe106c19095129b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:21:13.995841 16296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e4bfb9090c8460eb14fa7e0d818e7f1" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:21:13.981977 21282 consensus_queue.cc:1048] T 68a6838d82ef4cac83507f906139d90f P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.017s
I20260709 13:21:13.988397 21211 raft_consensus.cc:493] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:21:13.983980 20918 raft_consensus.cc:493] T 8a878999c0024d2abe517484180290fc P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:21:14.007539 21087 raft_consensus.cc:493] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:13.997931 20894 consensus_queue.cc:260] T b87147731a68400e8cce9b879daa7c5d P 90f3da57796b488c9087a2f523adf045 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.11, Last appended by leader: 11, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:14.024232 21087 raft_consensus.cc:515] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:14.026473 21283 raft_consensus.cc:515] T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:14.008368 21211 raft_consensus.cc:515] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:14.028460 21283 leader_election.cc:290] T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 13 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:14.029911 21211 leader_election.cc:290] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:14.008754 20918 raft_consensus.cc:515] T 8a878999c0024d2abe517484180290fc P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:14.009109 21250 leader_election.cc:290] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 16 election: Requested vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:14.011528 16819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdbd78acb4cf46298e0de8ba0d130e19" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:21:14.024734 20894 raft_consensus.cc:3060] T b87147731a68400e8cce9b879daa7c5d P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:21:14.033135 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 56901.2us
I20260709 13:21:14.033402 16819 raft_consensus.cc:3060] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:21:14.033969 16790 leader_election.cc:304] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
W20260709 13:21:14.039407 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 80437.4us
I20260709 13:21:14.012610 20972 raft_consensus.cc:515] T 26354361e9b044cea1c2b86c4882fe92 P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:21:14.019863 16263 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:21:14.007510 16264 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:14.046106 16819 raft_consensus.cc:2468] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 16.
I20260709 13:21:14.041932 16265 leader_election.cc:304] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:21:14.048538 16294 raft_consensus.cc:2468] T 84f2da98864b4425bf7c0055951fdd4b P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 11.
I20260709 13:21:14.058578 16190 leader_election.cc:304] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860
I20260709 13:21:14.058923 21309 raft_consensus.cc:2804] T d2ab0ac8347641e6955e9bce7d90c1a2 P cb82cc7bd6db46d980c75237a1402860 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:21:14.059547 21309 raft_consensus.cc:697] T d2ab0ac8347641e6955e9bce7d90c1a2 P cb82cc7bd6db46d980c75237a1402860 [term 14 LEADER]: Becoming Leader. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
W20260709 13:21:14.059629 16046 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:49252 (request call id 288) took 51575 ms (client timeout 15000 ms). Trace:
I20260709 13:21:14.060781 21084 raft_consensus.cc:2749] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:21:14.060442 21309 consensus_queue.cc:237] T d2ab0ac8347641e6955e9bce7d90c1a2 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:14.061522 21084 consensus_queue.cc:1059] T d2ab0ac8347641e6955e9bce7d90c1a2 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: INVALID_TERM, Last received: 13.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:21:14.063084 21158 raft_consensus.cc:3055] T d2ab0ac8347641e6955e9bce7d90c1a2 P 1ca2725d290e47d68b1a4b609c9816ac [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:21:14.063239 20972 leader_election.cc:290] T 26354361e9b044cea1c2b86c4882fe92 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 12 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:14.063503 21158 raft_consensus.cc:740] T d2ab0ac8347641e6955e9bce7d90c1a2 P 1ca2725d290e47d68b1a4b609c9816ac [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:21:14.019610 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:21:14.064589 21151 raft_consensus.cc:2749] T c0eab25ce4c64e018a041beeda2867db P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:21:14.064518 21158 consensus_queue.cc:260] T d2ab0ac8347641e6955e9bce7d90c1a2 P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:14.066080 21158 raft_consensus.cc:3060] T d2ab0ac8347641e6955e9bce7d90c1a2 P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:21:14.066428 21089 raft_consensus.cc:2749] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:21:14.067528 21161 raft_consensus.cc:493] T 829b4ad8c5a54eababeea1181b960d4b P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:14.068166 21161 raft_consensus.cc:515] T 829b4ad8c5a54eababeea1181b960d4b P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:14.070356 21161 leader_election.cc:290] T 829b4ad8c5a54eababeea1181b960d4b P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:14.051379 21154 consensus_queue.cc:1048] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.020s
I20260709 13:21:14.072939 16817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "829b4ad8c5a54eababeea1181b960d4b" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:14.073139 21154 raft_consensus.cc:493] T e5ad4eb281564936a59f41e387a47156 P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:21:14.073747 21154 raft_consensus.cc:515] T e5ad4eb281564936a59f41e387a47156 P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:13.992969 21216 raft_consensus.cc:2749] T 434bff3082d141838068f323bdb0d1ed P cb82cc7bd6db46d980c75237a1402860 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:21:14.058962 16044 catalog_manager.cc:5697] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac reported cstate change: term changed from 6 to 8, leader changed from 90f3da57796b488c9087a2f523adf045 (127.14.177.131) to 647a343a542d4a139fe106c19095129b (127.14.177.129). New cstate: current_term: 8 leader_uuid: "647a343a542d4a139fe106c19095129b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } }
I20260709 13:21:14.076836 16190 leader_election.cc:304] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860
I20260709 13:21:14.078253 21180 raft_consensus.cc:2749] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
W20260709 13:21:14.081724 16189 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:14.082906 16292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87ac0d1f7e8a419f9bcbc04aee0e067e" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:14.087312 16821 raft_consensus.cc:2468] T 829b4ad8c5a54eababeea1181b960d4b P cb82cc7bd6db46d980c75237a1402860 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 14.
W20260709 13:21:14.090129 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 94702.8us
I20260709 13:21:14.090502 16295 raft_consensus.cc:1275] T 1df93779209748b9860f6d893dd3de53 P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:21:14.092209 16295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d4ab870a05347868956aa8b841b2981" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
W20260709 13:21:14.093050 16192 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:14.026980 21087 leader_election.cc:290] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:14.081633 16299 raft_consensus.cc:515] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:14.094477 16824 raft_consensus.cc:2410] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac for term 14 because replica has last-logged OpId of term: 13 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 8.
W20260709 13:21:14.060040 16046 rpcz_store.cc:269] 0709 13:20:22.483787 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:22.484047 (+   260us) service_pool.cc:224] Handling call
0709 13:21:14.059381 (+51575334us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:21:08.320387 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:21:08.408251 (+ 87864us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:21:08.408306 (+    55us) write_op.cc:435] Acquired schema lock
0709 13:21:08.408327 (+    21us) tablet.cc:662] Decoding operations
0709 13:21:08.409486 (+  1159us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:21:08.409615 (+   129us) write_op.cc:641] Partition lock acquired for write op
0709 13:21:08.409645 (+    30us) tablet.cc:685] Acquiring locks for 10 operations
0709 13:21:08.410693 (+  1048us) tablet.cc:701] Row locks acquired
0709 13:21:08.410747 (+    54us) write_op.cc:260] PREPARE: finished
0709 13:21:08.410951 (+   204us) write_op.cc:270] Start()
0709 13:21:08.443212 (+ 32261us) write_op.cc:276] Timestamp: P: 1783603268410882 usec, L: 0
0709 13:21:08.542815 (+ 99603us) op_driver.cc:348] REPLICATION: starting
0709 13:21:08.762669 (+219854us) log.cc:844] Serialized 4428 byte log entry
0709 13:21:10.798151 (+2035482us) op_driver.cc:464] REPLICATION: finished
0709 13:21:10.858258 (+ 60107us) spinlock_profiling.cc:243] Waited 55.4 ms on lock 0x7b54003cdf54. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5ae28 00007f69b4595581 00007f69b4594d58 00007f69b45999f1 00007f69b45999a9 00007f69b4599939 00007f69b4599901 00007f69b4598b2d 000000000038f354 000000000038f199
0709 13:21:11.003554 (+145296us) write_op.cc:301] APPLY: starting
0709 13:21:11.007285 (+  3731us) tablet.cc:1367] starting BulkCheckPresence
0709 13:21:11.115603 (+108318us) tablet.cc:1370] finished BulkCheckPresence
0709 13:21:11.115649 (+    46us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:21:11.372241 (+256592us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:21:11.382422 (+ 10181us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=20,key_file_lookups=20,delta_file_lookups=0,mrs_lookups=0
0709 13:21:11.382504 (+    82us) write_op.cc:312] APPLY: finished
0709 13:21:11.382915 (+   411us) log.cc:844] Serialized 97 byte log entry
0709 13:21:11.383127 (+   212us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:21:11.383298 (+   171us) write_op.cc:454] Released schema lock
0709 13:21:11.383380 (+    82us) write_op.cc:341] FINISH: Updating metrics
0709 13:21:11.383886 (+   506us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:21:11.383917 (+    31us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":579,"reported_tablets":46,"child_traces":[["op",{"apply.queue_time_us":158316,"apply.run_cpu_time_us":18209,"apply.run_wall_time_us":386783,"cfile_cache_hit":42,"cfile_cache_hit_bytes":45887,"num_ops":10,"prepare.queue_time_us":104147,"prepare.run_cpu_time_us":92502,"prepare.run_wall_time_us":2597108,"raft.queue_time_us":413284,"raft.run_cpu_time_us":53709,"raft.run_wall_time_us":140339,"replication_time_us":2245398,"spinlock_wait_cycles":127522560,"thread_start_us":814535,"threads_started":4,"wal-append.queue_time_us":460339,"wal-append.run_cpu_time_us":19069,"wal-append.run_wall_time_us":3162721}]]}
I20260709 13:21:14.096709 16044 catalog_manager.cc:5697] T c2f5f89186204242bf377f7c42b2e2da P 1ca2725d290e47d68b1a4b609c9816ac reported cstate change: term changed from 7 to 9. New cstate: current_term: 9 leader_uuid: "1ca2725d290e47d68b1a4b609c9816ac" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } health_report { overall_health: HEALTHY } } }
I20260709 13:21:14.099187 16192 leader_election.cc:304] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:21:14.100078 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdbd78acb4cf46298e0de8ba0d130e19" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:21:14.102998 16300 raft_consensus.cc:3060] T cdbd78acb4cf46298e0de8ba0d130e19 P 90f3da57796b488c9087a2f523adf045 [term 14 FOLLOWER]: Advancing to term 16
I20260709 13:21:14.100713 16293 raft_consensus.cc:1275] T c9944d5e4b8c4109af9955c4f08c9095 P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:21:14.104240 16190 leader_election.cc:304] T 84f2da98864b4425bf7c0055951fdd4b P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:21:14.100085 20964 raft_consensus.cc:2804] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:21:14.114854 21170 raft_consensus.cc:2804] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:21:14.115391 20964 raft_consensus.cc:493] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:14.115648 21170 raft_consensus.cc:697] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [term 16 LEADER]: Becoming Leader. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:21:14.088397 20991 raft_consensus.cc:493] T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:21:14.116835 20991 raft_consensus.cc:515] T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:14.116942 21170 consensus_queue.cc:237] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:14.118880 20991 leader_election.cc:290] T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:14.116478 16044 catalog_manager.cc:5697] T 02838c86531d4f98bf58d1c22b8baf77 P 1ca2725d290e47d68b1a4b609c9816ac reported cstate change: term changed from 6 to 8, leader changed from 90f3da57796b488c9087a2f523adf045 (127.14.177.131) to 647a343a542d4a139fe106c19095129b (127.14.177.129). New cstate: current_term: 8 leader_uuid: "647a343a542d4a139fe106c19095129b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } }
I20260709 13:21:14.115870 20964 raft_consensus.cc:3060] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:21:14.112390 21180 raft_consensus.cc:493] T d16dfad0c3b540e5b4b86492ddf87208 P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:14.111914 20918 leader_election.cc:290] T 8a878999c0024d2abe517484180290fc P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
W20260709 13:21:14.120256 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 77002.8us
I20260709 13:21:14.128887 16299 leader_election.cc:290] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 13 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:14.132285 16295 raft_consensus.cc:1275] T fffab26427e24d398e2695aad40ca134 P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 7. (term mismatch)
I20260709 13:21:14.136890 16295 pending_rounds.cc:85] T fffab26427e24d398e2695aad40ca134 P 90f3da57796b488c9087a2f523adf045: Aborting all ops after (but not including) 6
I20260709 13:21:14.137188 16295 pending_rounds.cc:107] T fffab26427e24d398e2695aad40ca134 P 90f3da57796b488c9087a2f523adf045: Aborting uncommitted NO_OP operation due to leader change: 11.7
I20260709 13:21:14.137456 16295 raft_consensus.cc:2889] T fffab26427e24d398e2695aad40ca134 P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:21:14.133342 16820 raft_consensus.cc:1240] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860 [term 14 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 13. Current term is 14. Ops: []
I20260709 13:21:14.134799 16300 raft_consensus.cc:2468] T cdbd78acb4cf46298e0de8ba0d130e19 P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 16.
I20260709 13:21:14.137218 16044 catalog_manager.cc:5697] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac reported cstate change: term changed from 6 to 8. New cstate: current_term: 8 leader_uuid: "1ca2725d290e47d68b1a4b609c9816ac" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:21:14.138031 21180 raft_consensus.cc:515] T d16dfad0c3b540e5b4b86492ddf87208 P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:14.162741 21154 leader_election.cc:290] T e5ad4eb281564936a59f41e387a47156 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:14.140671 21151 raft_consensus.cc:2804] T 84f2da98864b4425bf7c0055951fdd4b P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:21:14.167320 16116 leader_election.cc:304] T e5ad4eb281564936a59f41e387a47156 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 90f3da57796b488c9087a2f523adf045
W20260709 13:21:14.167380 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 68676.6us
I20260709 13:21:14.168497 20999 raft_consensus.cc:2804] T e5ad4eb281564936a59f41e387a47156 P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:21:14.168874 16116 leader_election.cc:304] T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:21:14.169600 21211 consensus_queue.cc:1059] T c0eab25ce4c64e018a041beeda2867db P 90f3da57796b488c9087a2f523adf045 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:21:14.170047 20991 raft_consensus.cc:2749] T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:21:14.170409 16116 leader_election.cc:304] T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860
I20260709 13:21:14.172719 16191 leader_election.cc:304] T 66f75124215f4ca6bef831e174a792d8 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
W20260709 13:21:14.173291 16264 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:14.171007 20972 raft_consensus.cc:3055] T c0eab25ce4c64e018a041beeda2867db P 90f3da57796b488c9087a2f523adf045 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:21:14.168946 20999 raft_consensus.cc:493] T e5ad4eb281564936a59f41e387a47156 P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Starting leader election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:21:14.178948 20999 raft_consensus.cc:3060] T e5ad4eb281564936a59f41e387a47156 P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:21:14.178867 16265 leader_election.cc:304] T 3884f864a96d428785895e5463e5728f P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
I20260709 13:21:14.177111 21304 raft_consensus.cc:515] T 66f75124215f4ca6bef831e174a792d8 P cb82cc7bd6db46d980c75237a1402860 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:14.180783 16219 raft_consensus.cc:1240] T 746aabf856ab401c9ef5e859371e7b67 P 1ca2725d290e47d68b1a4b609c9816ac [term 13 LEADER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 11. Current term is 13. Ops: []
I20260709 13:21:14.181777 21304 leader_election.cc:290] T 66f75124215f4ca6bef831e174a792d8 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 14 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:21:14.182004 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada5a004ba7c46819051e3ccec71ffdb" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:14.182868 16226 raft_consensus.cc:2393] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 in current term 14: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
I20260709 13:21:14.182713 16218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:21:14.183383 21154 raft_consensus.cc:2749] T 66f75124215f4ca6bef831e174a792d8 P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:21:14.183887 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "829b4ad8c5a54eababeea1181b960d4b" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:14.184173 16116 leader_election.cc:304] T 02838c86531d4f98bf58d1c22b8baf77 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:21:14.184599 16224 raft_consensus.cc:2393] T 829b4ad8c5a54eababeea1181b960d4b P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 in current term 13: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
I20260709 13:21:14.185016 16217 raft_consensus.cc:1240] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 14. Current term is 15. Ops: [12.9-14.10]
I20260709 13:21:14.185550 16219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7ca61592e848ea9b0fb81a2ce33cdc" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:14.186399 20991 raft_consensus.cc:2749] T 02838c86531d4f98bf58d1c22b8baf77 P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:21:14.187680 16221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66ae176961b74813a58bc15c9c4c6f2a" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:21:14.188323 16221 raft_consensus.cc:3060] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:21:14.189323 16116 leader_election.cc:304] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:21:14.191720 16223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 14 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:14.192448 16223 raft_consensus.cc:2393] T ecc1e363230043fe9fef390f0b979f20 P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 in current term 14: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
I20260709 13:21:14.193912 16222 raft_consensus.cc:1275] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:21:14.194300 20999 raft_consensus.cc:515] T e5ad4eb281564936a59f41e387a47156 P 647a343a542d4a139fe106c19095129b [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:14.194969 20972 raft_consensus.cc:740] T c0eab25ce4c64e018a041beeda2867db P 90f3da57796b488c9087a2f523adf045 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:21:14.195384 21165 raft_consensus.cc:3060] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:21:14.195904 20972 consensus_queue.cc:260] T c0eab25ce4c64e018a041beeda2867db P 90f3da57796b488c9087a2f523adf045 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:14.197185 20972 raft_consensus.cc:3060] T c0eab25ce4c64e018a041beeda2867db P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:21:14.174368 21151 raft_consensus.cc:697] T 84f2da98864b4425bf7c0055951fdd4b P 1ca2725d290e47d68b1a4b609c9816ac [term 11 LEADER]: Becoming Leader. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
W20260709 13:21:14.214785 16789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 241090us
I20260709 13:21:14.197885 21106 raft_consensus.cc:2749] T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:21:14.214962 16218 raft_consensus.cc:2468] T 19baa7bc367844c288bca8f81439c413 P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 15.
I20260709 13:21:14.219014 16291 raft_consensus.cc:1275] T 8cc959fb7f704fba9009c0595467d074 P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
W20260709 13:21:14.224045 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 119904us
I20260709 13:21:14.224973 16190 leader_election.cc:304] T ecc1e363230043fe9fef390f0b979f20 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860
I20260709 13:21:14.202018 20918 raft_consensus.cc:493] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:14.226828 16044 catalog_manager.cc:5697] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "1ca2725d290e47d68b1a4b609c9816ac" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:21:14.232453 16117 leader_election.cc:304] T 829b4ad8c5a54eababeea1181b960d4b P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860; no voters: 
W20260709 13:21:14.232920 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 43680us
I20260709 13:21:14.184137 21154 consensus_queue.cc:1059] T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: INVALID_TERM, Last received: 13.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.032s
I20260709 13:21:14.213693 21211 raft_consensus.cc:2749] T 3884f864a96d428785895e5463e5728f P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
W20260709 13:21:14.234982 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 201081us
I20260709 13:21:14.231244 16222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdbd78acb4cf46298e0de8ba0d130e19" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 16 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:14.236039 16790 leader_election.cc:304] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:21:14.243870 16221 raft_consensus.cc:2468] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 16.
I20260709 13:21:14.245818 16218 raft_consensus.cc:1240] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 12. Current term is 13. Ops: []
I20260709 13:21:14.247975 16822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a9de347c711477e87c582d003f36925" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:14.247841 21151 consensus_queue.cc:237] T 84f2da98864b4425bf7c0055951fdd4b P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:14.248838 16116 leader_election.cc:304] T 5a959dbecc2648eb85d1cc550f5a3245 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:21:14.250319 16116 leader_election.cc:304] T fffab26427e24d398e2695aad40ca134 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 90f3da57796b488c9087a2f523adf045
I20260709 13:21:14.251367 21126 raft_consensus.cc:2749] T 5a959dbecc2648eb85d1cc550f5a3245 P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:21:14.251788 21106 raft_consensus.cc:2804] T fffab26427e24d398e2695aad40ca134 P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:21:14.252099 21007 raft_consensus.cc:2804] T 829b4ad8c5a54eababeea1181b960d4b P 647a343a542d4a139fe106c19095129b [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:21:14.252261 21106 raft_consensus.cc:493] T fffab26427e24d398e2695aad40ca134 P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:14.252765 21106 raft_consensus.cc:3060] T fffab26427e24d398e2695aad40ca134 P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:21:14.255338 15046 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:21:14.255425 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af57996a8be84d398f0f023e8d08ef77" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
W20260709 13:21:14.248227 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 75658.8us
I20260709 13:21:14.258392 16820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26354361e9b044cea1c2b86c4882fe92" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:21:14.259249 16820 raft_consensus.cc:3060] T 26354361e9b044cea1c2b86c4882fe92 P cb82cc7bd6db46d980c75237a1402860 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:21:14.245991 16790 leader_election.cc:304] T 829b4ad8c5a54eababeea1181b960d4b P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:21:14.261981 16821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e4bfb9090c8460eb14fa7e0d818e7f1" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: true dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:21:14.262599 16821 raft_consensus.cc:3055] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P cb82cc7bd6db46d980c75237a1402860 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:21:14.262905 16821 raft_consensus.cc:740] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P cb82cc7bd6db46d980c75237a1402860 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:21:14.282763 16116 leader_election.cc:304] T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: cb82cc7bd6db46d980c75237a1402860
I20260709 13:21:14.283244 16044 catalog_manager.cc:5697] T 7c2095a5f4ad49408dbc7f69c606120d P 1ca2725d290e47d68b1a4b609c9816ac reported cstate change: term changed from 5 to 7. New cstate: current_term: 7 leader_uuid: "647a343a542d4a139fe106c19095129b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } }
I20260709 13:21:14.284878 16044 catalog_manager.cc:5697] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 1ca2725d290e47d68b1a4b609c9816ac reported cstate change: term changed from 5 to 7. New cstate: current_term: 7 leader_uuid: "647a343a542d4a139fe106c19095129b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } }
I20260709 13:21:14.286170 16044 catalog_manager.cc:5697] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac reported cstate change: term changed from 7 to 9. New cstate: current_term: 9 leader_uuid: "647a343a542d4a139fe106c19095129b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:21:14.288837 16221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d16dfad0c3b540e5b4b86492ddf87208" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:14.289481 16221 raft_consensus.cc:2468] T d16dfad0c3b540e5b4b86492ddf87208 P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 13.
I20260709 13:21:14.294953 16218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ab0ac8347641e6955e9bce7d90c1a2" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:14.295800 16218 raft_consensus.cc:2468] T d2ab0ac8347641e6955e9bce7d90c1a2 P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 14.
I20260709 13:21:14.287912 16821 consensus_queue.cc:260] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P cb82cc7bd6db46d980c75237a1402860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:14.252784 21007 raft_consensus.cc:697] T 829b4ad8c5a54eababeea1181b960d4b P 647a343a542d4a139fe106c19095129b [term 14 LEADER]: Becoming Leader. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
W20260709 13:21:14.294124 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 65275.3us
I20260709 13:21:14.305295 21153 raft_consensus.cc:2804] T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:21:14.305922 21153 raft_consensus.cc:697] T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b [term 15 LEADER]: Becoming Leader. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:21:14.306958 21153 consensus_queue.cc:237] T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.10, Last appended by leader: 10, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:14.297771 20967 raft_consensus.cc:493] T 66ae176961b74813a58bc15c9c4c6f2a P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:14.300065 16222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8158bbd17b994d2eb96f560bd0f1b88d" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:14.309755 21180 leader_election.cc:290] T d16dfad0c3b540e5b4b86492ddf87208 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:14.234433 20918 raft_consensus.cc:515] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:14.257290 21089 raft_consensus.cc:2804] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:21:14.244800 20894 consensus_queue.cc:1059] T ecc1e363230043fe9fef390f0b979f20 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.9, Next index: 10, Last known committed idx: 6, Time since last communication: 0.047s
W20260709 13:21:14.358916 16266 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:21:14.377780 16266 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:21:14.378362 16266 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:14.363626 16222 raft_consensus.cc:2468] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 13.
W20260709 13:21:14.366489 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 127191us
I20260709 13:21:14.380074 16790 leader_election.cc:304] T 26354361e9b044cea1c2b86c4882fe92 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860; no voters: 90f3da57796b488c9087a2f523adf045
I20260709 13:21:14.380133 16821 raft_consensus.cc:3060] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P cb82cc7bd6db46d980c75237a1402860 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:21:14.371886 16820 raft_consensus.cc:2468] T 26354361e9b044cea1c2b86c4882fe92 P cb82cc7bd6db46d980c75237a1402860 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 12.
I20260709 13:21:14.385113 16218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c61a39fd4f84424b129c1a6f9ba404d" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
W20260709 13:21:14.385239 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 81922.5us
I20260709 13:21:14.385327 16819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b7d1ab519044eda2d5e04a07242f25" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
W20260709 13:21:14.385987 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 88037.7us
I20260709 13:21:14.386827 16223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee9ed67006144797b1de6ea8c2013a68" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:14.388990 16219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c11074579b046d99e585c2818e18a2c" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
W20260709 13:21:14.397205 16263 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:14.403277 16221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62fe3346cd3c42c4a55e14af9073bca4" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:14.397461 16821 raft_consensus.cc:2468] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P cb82cc7bd6db46d980c75237a1402860 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 13.
W20260709 13:21:14.379312 21299 log.cc:927] Time spent T 9e65676b21b748728ad39668cfe722ec P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.087s	user 0.000s	sys 0.000s
I20260709 13:21:14.367786 21325 raft_consensus.cc:2749] T ecc1e363230043fe9fef390f0b979f20 P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:21:14.377095 21089 raft_consensus.cc:493] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:21:14.410190 21329 consensus_queue.cc:1048] T 8cc959fb7f704fba9009c0595467d074 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
W20260709 13:21:14.432750 21276 consensus_meta.cc:287] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860: Time spent flushing consensus metadata: real 0.565s	user 0.015s	sys 0.000s
I20260709 13:21:14.414316 21318 raft_consensus.cc:2764] T 26354361e9b044cea1c2b86c4882fe92 P cb82cc7bd6db46d980c75237a1402860 [term 12 FOLLOWER]: Leader election decision vote started in defunct term 11: won
I20260709 13:21:14.414515 16821 raft_consensus.cc:1275] T ee9ed67006144797b1de6ea8c2013a68 P cb82cc7bd6db46d980c75237a1402860 [term 12 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
W20260709 13:21:14.413435 16264 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:14.425544 16222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a378d7f796f94615b3b347119a9880fd" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 16 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:14.298121 21126 consensus_queue.cc:1048] T b6f07842d3aa47858df5665874c86f3a P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.101s
I20260709 13:21:14.391582 21158 raft_consensus.cc:3055] T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:21:14.438062 16218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1df93779209748b9860f6d893dd3de53" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:14.413446 16790 leader_election.cc:304] T 3884f864a96d428785895e5463e5728f P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860; no voters: 90f3da57796b488c9087a2f523adf045
I20260709 13:21:14.440627 16221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c74490c40243a88e62d1760b5cd814" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:14.426374 21320 raft_consensus.cc:2764] T 829b4ad8c5a54eababeea1181b960d4b P cb82cc7bd6db46d980c75237a1402860 [term 14 FOLLOWER]: Leader election decision vote started in defunct term 13: won
I20260709 13:21:14.359303 20967 raft_consensus.cc:515] T 66ae176961b74813a58bc15c9c4c6f2a P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:14.407482 21180 consensus_queue.cc:1048] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.038s
I20260709 13:21:14.379784 21161 consensus_queue.cc:1048] T 1df93779209748b9860f6d893dd3de53 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.055s
I20260709 13:21:14.448518 21163 raft_consensus.cc:493] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:14.494190 21158 raft_consensus.cc:740] T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:21:14.508754 21163 raft_consensus.cc:515] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:14.509546 21158 consensus_queue.cc:260] T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:14.510483 21165 raft_consensus.cc:2749] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:21:14.431964 21154 consensus_queue.cc:1048] T fffab26427e24d398e2695aad40ca134 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.017s
I20260709 13:21:14.447988 16044 catalog_manager.cc:5697] T c0eab25ce4c64e018a041beeda2867db P 1ca2725d290e47d68b1a4b609c9816ac reported cstate change: term changed from 7 to 8, leader changed from 90f3da57796b488c9087a2f523adf045 (127.14.177.131) to 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130). New cstate: current_term: 8 leader_uuid: "1ca2725d290e47d68b1a4b609c9816ac" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } health_report { overall_health: HEALTHY } } }
I20260709 13:21:14.512106 20894 consensus_queue.cc:1048] T 8158bbd17b994d2eb96f560bd0f1b88d P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:21:14.513530 16044 catalog_manager.cc:5697] T 19baa7bc367844c288bca8f81439c413 P 1ca2725d290e47d68b1a4b609c9816ac reported cstate change: config changed from index 3 to 7, term changed from 7 to 8, leader changed from 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130) to <none>, cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) changed from NON_VOTER to VOTER, no longer has a pending config: VOTER 647a343a542d4a139fe106c19095129b (127.14.177.129), VOTER 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130), VOTER 90f3da57796b488c9087a2f523adf045 (127.14.177.131), VOTER cb82cc7bd6db46d980c75237a1402860 (127.14.177.132). New cstate: current_term: 8 committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:21:14.431708 20999 leader_election.cc:290] T e5ad4eb281564936a59f41e387a47156 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 14 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:14.515640 20999 raft_consensus.cc:493] T 9e65676b21b748728ad39668cfe722ec P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:21:14.516021 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 102683us
I20260709 13:21:14.516191 20999 raft_consensus.cc:515] T 9e65676b21b748728ad39668cfe722ec P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:14.437074 21089 raft_consensus.cc:3060] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:21:14.450052 16219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66f75124215f4ca6bef831e174a792d8" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:14.523756 16219 raft_consensus.cc:2468] T 66f75124215f4ca6bef831e174a792d8 P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 13.
I20260709 13:21:14.524374 16220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "829b4ad8c5a54eababeea1181b960d4b" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:21:14.524804 16219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7ca61592e848ea9b0fb81a2ce33cdc" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:14.525113 16220 raft_consensus.cc:3060] T 829b4ad8c5a54eababeea1181b960d4b P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:21:14.526144 16219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ab0ac8347641e6955e9bce7d90c1a2" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:21:14.486233 16222 raft_consensus.cc:2410] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 for term 16 because replica has last-logged OpId of term: 14 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 9.
I20260709 13:21:14.528115 16222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62fe3346cd3c42c4a55e14af9073bca4" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:14.529316 16222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66ae176961b74813a58bc15c9c4c6f2a" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
W20260709 13:21:14.529377 16192 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 69025.8us
I20260709 13:21:14.529963 16222 raft_consensus.cc:2376] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 16. Re-sending same reply.
I20260709 13:21:14.496975 16221 raft_consensus.cc:2468] T 08c74490c40243a88e62d1760b5cd814 P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 13.
I20260709 13:21:14.530529 21250 consensus_queue.cc:1048] T ee9ed67006144797b1de6ea8c2013a68 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:21:14.531144 16221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b7d1ab519044eda2d5e04a07242f25" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:14.531847 16221 raft_consensus.cc:2468] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 11.
I20260709 13:21:14.532763 16221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "829b4ad8c5a54eababeea1181b960d4b" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
W20260709 13:21:14.533906 21277 consensus_meta.cc:287] T 84f2da98864b4425bf7c0055951fdd4b P 647a343a542d4a139fe106c19095129b: Time spent flushing consensus metadata: real 0.657s	user 0.005s	sys 0.009s
I20260709 13:21:14.534490 21277 raft_consensus.cc:515] T 84f2da98864b4425bf7c0055951fdd4b P 647a343a542d4a139fe106c19095129b [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:21:14.498893 16117 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 106608us
W20260709 13:21:14.500090 16266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 121041us
I20260709 13:21:14.445057 16223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d4ab870a05347868956aa8b841b2981" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:14.501776 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "829b4ad8c5a54eababeea1181b960d4b" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
W20260709 13:21:14.501865 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 114374us
W20260709 13:21:14.568372 16266 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:14.502878 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ad5dbfa99ee4b8d9e28be92b3f36db5" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:14.485955 16822 raft_consensus.cc:2468] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 16.
I20260709 13:21:14.533867 16221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:21:14.569702 16221 raft_consensus.cc:2376] T 19baa7bc367844c288bca8f81439c413 P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 15. Re-sending same reply.
W20260709 13:21:14.497648 16264 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:14.574458 16223 raft_consensus.cc:2393] T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 13: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
I20260709 13:21:14.576771 16820 raft_consensus.cc:1240] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860 [term 13 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 12. Current term is 13. Ops: []
I20260709 13:21:14.576382 16221 raft_consensus.cc:1275] T 68a6838d82ef4cac83507f906139d90f P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 15 index: 8. (term mismatch)
I20260709 13:21:14.577476 16221 pending_rounds.cc:85] T 68a6838d82ef4cac83507f906139d90f P 1ca2725d290e47d68b1a4b609c9816ac: Aborting all ops after (but not including) 7
I20260709 13:21:14.577723 16221 pending_rounds.cc:107] T 68a6838d82ef4cac83507f906139d90f P 1ca2725d290e47d68b1a4b609c9816ac: Aborting uncommitted NO_OP operation due to leader change: 14.8
I20260709 13:21:14.577957 16221 raft_consensus.cc:2889] T 68a6838d82ef4cac83507f906139d90f P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:21:14.579833 16266 leader_election.cc:304] T 26354361e9b044cea1c2b86c4882fe92 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:21:14.580919 16266 leader_election.cc:304] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:21:14.582789 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5ad4eb281564936a59f41e387a47156" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:14.584393 16291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d16dfad0c3b540e5b4b86492ddf87208" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:14.595557 20999 leader_election.cc:290] T 9e65676b21b748728ad39668cfe722ec P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:14.603147 16116 leader_election.cc:304] T d16dfad0c3b540e5b4b86492ddf87208 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 90f3da57796b488c9087a2f523adf045
I20260709 13:21:14.535889 21180 consensus_queue.cc:1243] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Peer 90f3da57796b488c9087a2f523adf045 log is divergent from this leader: its last log entry 12.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 13:21:14.604732 16116 leader_election.cc:304] T 8158bbd17b994d2eb96f560bd0f1b88d P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 90f3da57796b488c9087a2f523adf045
I20260709 13:21:14.607311 21323 raft_consensus.cc:2804] T 26354361e9b044cea1c2b86c4882fe92 P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:21:14.609740 16116 leader_election.cc:304] T ee9ed67006144797b1de6ea8c2013a68 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860
W20260709 13:21:14.563616 21327 log.cc:927] Time spent T d2ab0ac8347641e6955e9bce7d90c1a2 P cb82cc7bd6db46d980c75237a1402860: Append to log took a long time: real 0.076s	user 0.000s	sys 0.003s
I20260709 13:21:14.616071 16220 raft_consensus.cc:2468] T 829b4ad8c5a54eababeea1181b960d4b P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 14.
I20260709 13:21:14.560563 21318 raft_consensus.cc:2804] T 3884f864a96d428785895e5463e5728f P cb82cc7bd6db46d980c75237a1402860 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:21:14.607996 21323 raft_consensus.cc:697] T 26354361e9b044cea1c2b86c4882fe92 P 90f3da57796b488c9087a2f523adf045 [term 12 LEADER]: Becoming Leader. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
W20260709 13:21:14.618167 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 162122us
I20260709 13:21:14.623353 16219 raft_consensus.cc:2468] T d2ab0ac8347641e6955e9bce7d90c1a2 P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 14.
I20260709 13:21:14.616261 16291 raft_consensus.cc:2468] T d16dfad0c3b540e5b4b86492ddf87208 P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 13.
I20260709 13:21:14.604768 21338 raft_consensus.cc:2804] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:21:14.671411 16226 raft_consensus.cc:1240] T e4dc6b0541ca431ebc757beac00521f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 12. Current term is 13. Ops: []
I20260709 13:21:14.674651 16265 leader_election.cc:304] T 84f2da98864b4425bf7c0055951fdd4b P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:21:14.451016 21007 consensus_queue.cc:237] T 829b4ad8c5a54eababeea1181b960d4b P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:14.624433 21318 raft_consensus.cc:493] T 3884f864a96d428785895e5463e5728f P cb82cc7bd6db46d980c75237a1402860 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:21:14.697419 16044 catalog_manager.cc:5697] T 5a96373bb0e14dce8a266b2fc6c58507 P 1ca2725d290e47d68b1a4b609c9816ac reported cstate change: term changed from 5 to 7. New cstate: current_term: 7 leader_uuid: "647a343a542d4a139fe106c19095129b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } }
I20260709 13:21:14.704573 16225 raft_consensus.cc:2468] T af57996a8be84d398f0f023e8d08ef77 P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 15.
I20260709 13:21:14.705654 16221 raft_consensus.cc:1275] T af57996a8be84d398f0f023e8d08ef77 P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
W20260709 13:21:14.707289 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 98455.5us
W20260709 13:21:14.708053 16790 leader_election.cc:336] T d2ab0ac8347641e6955e9bce7d90c1a2 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943): Timed out: RequestConsensusVote RPC to 127.14.177.130:35943 timed out after 1.755s (SENT)
I20260709 13:21:14.719760 16148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5718c623ebee4d2bb770f0f9ddb2f164" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
W20260709 13:21:14.723579 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 117416us
I20260709 13:21:14.724960 16116 leader_election.cc:304] T 9c61a39fd4f84424b129c1a6f9ba404d P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:21:14.726202 16142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2794af5b99984efebdee564035841cb0" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:21:14.671428 21336 raft_consensus.cc:2804] T d16dfad0c3b540e5b4b86492ddf87208 P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:21:14.731240 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5ad4eb281564936a59f41e387a47156" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:21:14.673887 21342 raft_consensus.cc:2804] T 8158bbd17b994d2eb96f560bd0f1b88d P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:21:14.738924 16143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee9ed67006144797b1de6ea8c2013a68" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
W20260709 13:21:14.739514 16790 connection.cc:256] RPC call timeout handler was delayed by -1.1940194230000998s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21309999999999998s for a response
W20260709 13:21:14.740264 16790 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:21:14.740686 16790 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:21:14.742494 16790 leader_election.cc:336] T af57996a8be84d398f0f023e8d08ef77 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 15 election: RPC error from VoteRequest() call to peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943): Timed out: RequestConsensusVote RPC to 127.14.177.130:35943 timed out after 2.131s (SENT)
I20260709 13:21:14.740459 16145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c74490c40243a88e62d1760b5cd814" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 14 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:14.743332 21320 raft_consensus.cc:493] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:14.743562 16145 raft_consensus.cc:2410] T 08c74490c40243a88e62d1760b5cd814 P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac for term 14 because replica has last-logged OpId of term: 11 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 7.
I20260709 13:21:14.744187 16148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b7d1ab519044eda2d5e04a07242f25" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:14.743997 21320 raft_consensus.cc:515] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:14.744868 16148 raft_consensus.cc:2468] T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 11.
I20260709 13:21:14.745857 20918 leader_election.cc:290] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:14.754482 16151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66f75124215f4ca6bef831e174a792d8" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:21:14.755316 16151 raft_consensus.cc:3060] T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:21:14.691953 21153 raft_consensus.cc:2749] T ee9ed67006144797b1de6ea8c2013a68 P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:21:14.758585 16264 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:14.749188 16044 catalog_manager.cc:5697] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac reported cstate change: term changed from 8 to 11, leader changed from 647a343a542d4a139fe106c19095129b (127.14.177.129) to 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130). New cstate: current_term: 11 leader_uuid: "1ca2725d290e47d68b1a4b609c9816ac" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } health_report { overall_health: UNKNOWN } } }
I20260709 13:21:14.758314 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e65676b21b748728ad39668cfe722ec" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 14 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:14.761238 16149 raft_consensus.cc:1275] T b87147731a68400e8cce9b879daa7c5d P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 13:21:14.697322 21318 raft_consensus.cc:3060] T 3884f864a96d428785895e5463e5728f P cb82cc7bd6db46d980c75237a1402860 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:21:14.762635 16264 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:21:14.817437 20972 consensus_meta.cc:287] T c0eab25ce4c64e018a041beeda2867db P 90f3da57796b488c9087a2f523adf045: Time spent flushing consensus metadata: real 0.620s	user 0.013s	sys 0.004s
I20260709 13:21:14.735682 21126 raft_consensus.cc:493] T 18ad7e3af5504571a69d10c00750a01c P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:21:14.748864 21342 raft_consensus.cc:493] T 8158bbd17b994d2eb96f560bd0f1b88d P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:14.818548 16790 leader_election.cc:304] T ada5a004ba7c46819051e3ccec71ffdb P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:21:14.819628 16143 raft_consensus.cc:2468] T ee9ed67006144797b1de6ea8c2013a68 P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 12.
W20260709 13:21:14.821656 21106 consensus_meta.cc:287] T fffab26427e24d398e2695aad40ca134 P 647a343a542d4a139fe106c19095129b: Time spent flushing consensus metadata: real 0.568s	user 0.012s	sys 0.000s
I20260709 13:21:14.732831 21336 raft_consensus.cc:493] T d16dfad0c3b540e5b4b86492ddf87208 P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Starting leader election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:21:14.825199 16145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4dc6b0541ca431ebc757beac00521f1" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
W20260709 13:21:14.827651 16266 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:14.815515 21323 consensus_queue.cc:237] T 26354361e9b044cea1c2b86c4882fe92 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:14.692761 21245 consensus_queue.cc:1059] T 746aabf856ab401c9ef5e859371e7b67 P 647a343a542d4a139fe106c19095129b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.068s
I20260709 13:21:14.849820 16148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746aabf856ab401c9ef5e859371e7b67" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
W20260709 13:21:14.865463 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 190853us
I20260709 13:21:14.849785 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746aabf856ab401c9ef5e859371e7b67" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:14.757306 21276 raft_consensus.cc:493] T 57a744f5aaf944678a63469cbf514015 P cb82cc7bd6db46d980c75237a1402860 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:14.849401 20918 raft_consensus.cc:2764] T 84f2da98864b4425bf7c0055951fdd4b P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Leader pre-election decision vote started in defunct term 10: won
I20260709 13:21:14.862596 16044 catalog_manager.cc:5697] T d16dfad0c3b540e5b4b86492ddf87208 P 1ca2725d290e47d68b1a4b609c9816ac reported cstate change: term changed from 5 to 7. New cstate: current_term: 7 leader_uuid: "647a343a542d4a139fe106c19095129b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } }
I20260709 13:21:14.763114 21163 leader_election.cc:290] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:14.856732 20967 leader_election.cc:290] T 66ae176961b74813a58bc15c9c4c6f2a P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
W20260709 13:21:14.858062 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 133229us
I20260709 13:21:14.887799 16291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
W20260709 13:21:14.865881 21331 log.cc:927] Time spent T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 0.243s	user 0.001s	sys 0.000s
I20260709 13:21:14.888257 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c2095a5f4ad49408dbc7f69c606120d" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:14.859119 21342 raft_consensus.cc:3060] T 8158bbd17b994d2eb96f560bd0f1b88d P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:21:14.897115 16148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f2da98864b4425bf7c0055951fdd4b" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:21:14.919492 16148 raft_consensus.cc:3060] T 84f2da98864b4425bf7c0055951fdd4b P 647a343a542d4a139fe106c19095129b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:21:14.865958 21336 raft_consensus.cc:3060] T d16dfad0c3b540e5b4b86492ddf87208 P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:21:14.890107 16044 catalog_manager.cc:5697] T 8cc959fb7f704fba9009c0595467d074 P 1ca2725d290e47d68b1a4b609c9816ac reported cstate change: term changed from 6 to 8. New cstate: current_term: 8 leader_uuid: "647a343a542d4a139fe106c19095129b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } }
I20260709 13:21:14.917689 16291 raft_consensus.cc:2468] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 15.
I20260709 13:21:14.861779 20999 raft_consensus.cc:2749] T 9c61a39fd4f84424b129c1a6f9ba404d P 647a343a542d4a139fe106c19095129b [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:21:14.922015 16116 leader_election.cc:304] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860
I20260709 13:21:14.928890 16142 raft_consensus.cc:2468] T 2794af5b99984efebdee564035841cb0 P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 11.
W20260709 13:21:14.881618 16263 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:21:14.919636 16117 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 340624us
W20260709 13:21:14.933177 16789 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:14.933228 16145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5718c623ebee4d2bb770f0f9ddb2f164" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
W20260709 13:21:14.918483 20964 consensus_meta.cc:287] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045: Time spent flushing consensus metadata: real 0.776s	user 0.012s	sys 0.000s
W20260709 13:21:14.944080 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 125654us
I20260709 13:21:14.866442 21106 raft_consensus.cc:515] T fffab26427e24d398e2695aad40ca134 P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:14.947747 16116 leader_election.cc:304] T 1df93779209748b9860f6d893dd3de53 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:21:14.949990 16116 leader_election.cc:304] T 08c74490c40243a88e62d1760b5cd814 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 90f3da57796b488c9087a2f523adf045
I20260709 13:21:14.953018 16116 leader_election.cc:304] T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: cb82cc7bd6db46d980c75237a1402860
I20260709 13:21:14.884002 21126 raft_consensus.cc:515] T 18ad7e3af5504571a69d10c00750a01c P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } }
I20260709 13:21:14.883766 21089 raft_consensus.cc:515] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:14.970703 21089 leader_election.cc:290] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 18 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:21:14.971069 21126 leader_election.cc:290] T 18ad7e3af5504571a69d10c00750a01c P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:21:14.971405 16824 raft_consensus.cc:1240] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [term 18 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 17. Current term is 18. Ops: []
I20260709 13:21:14.954663 20964 raft_consensus.cc:515] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } }
I20260709 13:21:14.973894 20964 leader_election.cc:290] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 14 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:21:14.891083 21276 raft_consensus.cc:515] T 57a744f5aaf944678a63469cbf514015 P cb82cc7bd6db46d980c75237a1402860 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:14.953747 21335 raft_consensus.cc:2749] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:21:14.976367 21276 leader_election.cc:290] T 57a744f5aaf944678a63469cbf514015 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:14.956179 21165 raft_consensus.cc:2804] T 08c74490c40243a88e62d1760b5cd814 P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:21:14.978181 21165 raft_consensus.cc:493] T 08c74490c40243a88e62d1760b5cd814 P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:14.978581 21165 raft_consensus.cc:3060] T 08c74490c40243a88e62d1760b5cd814 P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:21:14.958293 21313 raft_consensus.cc:2749] T ada5a004ba7c46819051e3ccec71ffdb P cb82cc7bd6db46d980c75237a1402860 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:21:14.956907 21245 raft_consensus.cc:2749] T 1df93779209748b9860f6d893dd3de53 P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:21:14.968207 21106 leader_election.cc:290] T fffab26427e24d398e2695aad40ca134 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 13 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:14.989786 16151 raft_consensus.cc:2468] T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 14.
I20260709 13:21:14.986248 16148 raft_consensus.cc:2468] T 84f2da98864b4425bf7c0055951fdd4b P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 11.
W20260709 13:21:14.990962 16266 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:14.993150 16190 leader_election.cc:304] T d16dfad0c3b540e5b4b86492ddf87208 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:21:14.993458 21340 raft_consensus.cc:2804] T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:21:15.000018 21340 raft_consensus.cc:493] T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Starting leader election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:21:14.995633 21325 raft_consensus.cc:493] T 5a96373bb0e14dce8a266b2fc6c58507 P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:21:15.000545 21340 raft_consensus.cc:3060] T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:21:15.000845 21325 raft_consensus.cc:515] T 5a96373bb0e14dce8a266b2fc6c58507 P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:15.001328 21165 raft_consensus.cc:515] T 08c74490c40243a88e62d1760b5cd814 P 647a343a542d4a139fe106c19095129b [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:14.996968 21151 raft_consensus.cc:2804] T d16dfad0c3b540e5b4b86492ddf87208 P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:21:15.003767 21151 raft_consensus.cc:493] T d16dfad0c3b540e5b4b86492ddf87208 P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:15.003851 21165 leader_election.cc:290] T 08c74490c40243a88e62d1760b5cd814 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 14 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:15.004338 21151 raft_consensus.cc:3060] T d16dfad0c3b540e5b4b86492ddf87208 P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:21:15.008935 21250 raft_consensus.cc:493] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:15.009605 21250 raft_consensus.cc:515] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:15.011464 21325 leader_election.cc:290] T 5a96373bb0e14dce8a266b2fc6c58507 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
W20260709 13:21:15.015323 16264 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:14.993934 16148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18ad7e3af5504571a69d10c00750a01c" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:21:15.017520 16148 raft_consensus.cc:3060] T 18ad7e3af5504571a69d10c00750a01c P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:21:14.949889 21277 leader_election.cc:290] T 84f2da98864b4425bf7c0055951fdd4b P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 10 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:15.020063 16822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66ae176961b74813a58bc15c9c4c6f2a" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:15.023649 16822 raft_consensus.cc:2468] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 16.
W20260709 13:21:15.026168 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 110260us
I20260709 13:21:15.035126 16824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e9b2bf7011f4ae688a79d7bb5a447b4" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:15.038723 16299 raft_consensus.cc:1275] T d2ab0ac8347641e6955e9bce7d90c1a2 P 90f3da57796b488c9087a2f523adf045 [term 14 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 14 index: 13. (index mismatch)
W20260709 13:21:15.042331 16264 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:15.049049 16292 raft_consensus.cc:1275] T b87147731a68400e8cce9b879daa7c5d P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 16 index: 11. (term mismatch)
I20260709 13:21:15.049278 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:15.049572 16292 pending_rounds.cc:85] T b87147731a68400e8cce9b879daa7c5d P 90f3da57796b488c9087a2f523adf045: Aborting all ops after (but not including) 10
I20260709 13:21:15.050420 16292 pending_rounds.cc:107] T b87147731a68400e8cce9b879daa7c5d P 90f3da57796b488c9087a2f523adf045: Aborting uncommitted NO_OP operation due to leader change: 15.11
I20260709 13:21:15.050639 16292 raft_consensus.cc:2889] T b87147731a68400e8cce9b879daa7c5d P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:21:14.982266 21320 leader_election.cc:290] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:15.054011 16822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e65676b21b748728ad39668cfe722ec" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 14 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:14.981860 21161 consensus_queue.cc:1059] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: INVALID_TERM, Last received: 17.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.049s
I20260709 13:21:15.056811 16265 leader_election.cc:304] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 647a343a542d4a139fe106c19095129b
I20260709 13:21:15.060633 16824 raft_consensus.cc:2468] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 11.
I20260709 13:21:15.069730 16296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57a744f5aaf944678a63469cbf514015" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:15.070253 16148 raft_consensus.cc:2468] T 18ad7e3af5504571a69d10c00750a01c P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 12.
I20260709 13:21:15.071277 16145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:21:15.072028 16145 raft_consensus.cc:3060] T ecc1e363230043fe9fef390f0b979f20 P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:21:15.037966 21338 raft_consensus.cc:697] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045 [term 13 LEADER]: Becoming Leader. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:21:15.076404 16822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:15.070412 16296 raft_consensus.cc:2468] T 57a744f5aaf944678a63469cbf514015 P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 10.
I20260709 13:21:15.082110 16148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87ac0d1f7e8a419f9bcbc04aee0e067e" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:15.081261 16822 raft_consensus.cc:2468] T 19baa7bc367844c288bca8f81439c413 P cb82cc7bd6db46d980c75237a1402860 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 15.
I20260709 13:21:15.084630 16790 leader_election.cc:304] T 3a7ca61592e848ea9b0fb81a2ce33cdc P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
W20260709 13:21:15.152741 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 96160.9us
W20260709 13:21:15.154706 16789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 71264.3us
I20260709 13:21:15.164146 16148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d4ab870a05347868956aa8b841b2981" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:21:15.164837 16148 raft_consensus.cc:3060] T 8d4ab870a05347868956aa8b841b2981 P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:21:15.170436 16264 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:15.179883 21250 leader_election.cc:290] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:15.180614 21250 raft_consensus.cc:493] T 68a6838d82ef4cac83507f906139d90f P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:15.181133 21250 raft_consensus.cc:515] T 68a6838d82ef4cac83507f906139d90f P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:15.183233 21304 consensus_queue.cc:1048] T b87147731a68400e8cce9b879daa7c5d P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.022s
I20260709 13:21:15.184965 21361 consensus_queue.cc:1048] T d2ab0ac8347641e6955e9bce7d90c1a2 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.078s
I20260709 13:21:15.067263 21313 consensus_queue.cc:1048] T 8638ee363273426db1393dd81e5c7b72 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.085s
I20260709 13:21:15.163018 21089 raft_consensus.cc:2749] T 3a7ca61592e848ea9b0fb81a2ce33cdc P cb82cc7bd6db46d980c75237a1402860 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:21:15.213066 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 128574us
I20260709 13:21:15.221436 16790 leader_election.cc:304] T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:21:15.219718 16148 raft_consensus.cc:2468] T 8d4ab870a05347868956aa8b841b2981 P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 13.
I20260709 13:21:15.223889 21250 leader_election.cc:290] T 68a6838d82ef4cac83507f906139d90f P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:15.230453 21340 raft_consensus.cc:515] T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:15.231307 21354 raft_consensus.cc:2804] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 90f3da57796b488c9087a2f523adf045 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:21:15.226678 21342 raft_consensus.cc:515] T 8158bbd17b994d2eb96f560bd0f1b88d P 647a343a542d4a139fe106c19095129b [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:15.194610 21304 raft_consensus.cc:493] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:15.233747 16265 leader_election.cc:304] T cdbd78acb4cf46298e0de8ba0d130e19 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860
I20260709 13:21:15.234791 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66f75124215f4ca6bef831e174a792d8" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:21:15.268590 16225 raft_consensus.cc:3060] T 66f75124215f4ca6bef831e174a792d8 P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:21:15.193847 21165 raft_consensus.cc:493] T 8cc959fb7f704fba9009c0595467d074 P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:15.241039 16790 leader_election.cc:304] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 90f3da57796b488c9087a2f523adf045
W20260709 13:21:15.242357 21358 log.cc:927] Time spent T 829b4ad8c5a54eababeea1181b960d4b P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 0.080s	user 0.000s	sys 0.000s
I20260709 13:21:15.242926 16222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee9ed67006144797b1de6ea8c2013a68" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:15.243428 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5718c623ebee4d2bb770f0f9ddb2f164" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:15.279106 16226 raft_consensus.cc:2468] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 12.
I20260709 13:21:15.244591 21276 raft_consensus.cc:2749] T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:21:15.246572 16144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "829b4ad8c5a54eababeea1181b960d4b" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:15.247759 16149 raft_consensus.cc:1240] T fffab26427e24d398e2695aad40ca134 P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 12. Current term is 13. Ops: []
W20260709 13:21:15.250823 21332 log.cc:927] Time spent T 84f2da98864b4425bf7c0055951fdd4b P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 0.627s	user 0.002s	sys 0.000s
I20260709 13:21:15.252028 21338 consensus_queue.cc:237] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:15.261425 16142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d16dfad0c3b540e5b4b86492ddf87208" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:15.262506 16145 raft_consensus.cc:2410] T ecc1e363230043fe9fef390f0b979f20 P 647a343a542d4a139fe106c19095129b [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac for term 14 because replica has last-logged OpId of term: 13 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 8.
I20260709 13:21:15.286845 16145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a96373bb0e14dce8a266b2fc6c58507" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:15.287047 16149 raft_consensus.cc:3055] T 746aabf856ab401c9ef5e859371e7b67 P 647a343a542d4a139fe106c19095129b [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:21:15.275615 16148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5ad4eb281564936a59f41e387a47156" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:15.261813 21354 raft_consensus.cc:493] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 90f3da57796b488c9087a2f523adf045 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:21:15.277542 21211 raft_consensus.cc:3055] T ecc1e363230043fe9fef390f0b979f20 P 90f3da57796b488c9087a2f523adf045 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:21:15.230795 21318 raft_consensus.cc:515] T 3884f864a96d428785895e5463e5728f P cb82cc7bd6db46d980c75237a1402860 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:15.287493 16149 raft_consensus.cc:740] T 746aabf856ab401c9ef5e859371e7b67 P 647a343a542d4a139fe106c19095129b [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:21:15.287896 16115 leader_election.cc:304] T 9e65676b21b748728ad39668cfe722ec P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860
I20260709 13:21:15.288347 16148 raft_consensus.cc:2393] T e5ad4eb281564936a59f41e387a47156 P 647a343a542d4a139fe106c19095129b [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 14: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
I20260709 13:21:15.284410 21357 raft_consensus.cc:493] T c0eab25ce4c64e018a041beeda2867db P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:15.325191 16225 raft_consensus.cc:2468] T 66f75124215f4ca6bef831e174a792d8 P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 14.
I20260709 13:21:15.325695 16144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68a6838d82ef4cac83507f906139d90f" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:15.292707 21211 raft_consensus.cc:740] T ecc1e363230043fe9fef390f0b979f20 P 90f3da57796b488c9087a2f523adf045 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:21:15.329919 21352 raft_consensus.cc:2749] T cdbd78acb4cf46298e0de8ba0d130e19 P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:21:15.329718 21211 consensus_queue.cc:260] T ecc1e363230043fe9fef390f0b979f20 P 90f3da57796b488c9087a2f523adf045 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:15.330948 21211 raft_consensus.cc:3060] T ecc1e363230043fe9fef390f0b979f20 P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:21:15.328207 21276 raft_consensus.cc:493] T c9944d5e4b8c4109af9955c4f08c9095 P cb82cc7bd6db46d980c75237a1402860 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:15.331797 21276 raft_consensus.cc:515] T c9944d5e4b8c4109af9955c4f08c9095 P cb82cc7bd6db46d980c75237a1402860 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:21:15.333335 16264 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:15.291060 21361 raft_consensus.cc:2804] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [term 16 FOLLOWER]: Leader election won for term 16
W20260709 13:21:15.334867 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 93899.6us
I20260709 13:21:15.293052 21354 raft_consensus.cc:3060] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 90f3da57796b488c9087a2f523adf045 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:21:15.280385 21165 raft_consensus.cc:515] T 8cc959fb7f704fba9009c0595467d074 P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:15.278614 21304 raft_consensus.cc:515] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:15.333706 16149 consensus_queue.cc:260] T 746aabf856ab401c9ef5e859371e7b67 P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.12, Last appended by leader: 12, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:21:15.340041 16115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 58212.1us
I20260709 13:21:15.277812 21336 raft_consensus.cc:515] T d16dfad0c3b540e5b4b86492ddf87208 P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:15.342979 16790 leader_election.cc:304] T 19baa7bc367844c288bca8f81439c413 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860; no voters: 90f3da57796b488c9087a2f523adf045
I20260709 13:21:15.345024 16149 raft_consensus.cc:3060] T 746aabf856ab401c9ef5e859371e7b67 P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:21:15.343887 16115 leader_election.cc:304] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 
W20260709 13:21:15.354893 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 141800us
I20260709 13:21:15.357714 16191 leader_election.cc:304] T 08c74490c40243a88e62d1760b5cd814 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:21:15.357530 16144 raft_consensus.cc:1275] T 8cc959fb7f704fba9009c0595467d074 P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 13:21:15.360570 16824 raft_consensus.cc:1240] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860 [term 13 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 12. Current term is 13. Ops: [12.12-12.12]
I20260709 13:21:15.362689 16142 raft_consensus.cc:2468] T d16dfad0c3b540e5b4b86492ddf87208 P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 13.
I20260709 13:21:15.340792 21361 raft_consensus.cc:697] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [term 16 LEADER]: Becoming Leader. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:21:15.371619 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fffab26427e24d398e2695aad40ca134" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:21:15.380441 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f2da98864b4425bf7c0055951fdd4b" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:21:15.360772 21277 raft_consensus.cc:2804] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:21:15.381263 16300 raft_consensus.cc:2368] T 84f2da98864b4425bf7c0055951fdd4b P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b for earlier term 10. Current term is 11.
I20260709 13:21:15.383951 16265 leader_election.cc:304] T c9944d5e4b8c4109af9955c4f08c9095 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860
I20260709 13:21:15.384279 16191 leader_election.cc:304] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: cb82cc7bd6db46d980c75237a1402860
I20260709 13:21:15.387274 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18ad7e3af5504571a69d10c00750a01c" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
W20260709 13:21:15.389544 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 53980.2us
I20260709 13:21:15.372509 21089 raft_consensus.cc:2804] T 19baa7bc367844c288bca8f81439c413 P cb82cc7bd6db46d980c75237a1402860 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:21:15.372709 21337 raft_consensus.cc:2749] T 08c74490c40243a88e62d1760b5cd814 P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:21:15.394366 16191 leader_election.cc:304] T e4dc6b0541ca431ebc757beac00521f1 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:21:15.337980 21357 raft_consensus.cc:515] T c0eab25ce4c64e018a041beeda2867db P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:15.393651 16790 leader_election.cc:304] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:21:15.383176 21277 raft_consensus.cc:493] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:15.396432 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c74490c40243a88e62d1760b5cd814" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:21:15.399427 16191 leader_election.cc:304] T 7c2095a5f4ad49408dbc7f69c606120d P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:21:15.398093 16790 leader_election.cc:304] T 66f75124215f4ca6bef831e174a792d8 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:21:15.402045 16191 leader_election.cc:304] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
I20260709 13:21:15.403038 21366 raft_consensus.cc:2749] T 7c2095a5f4ad49408dbc7f69c606120d P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:21:15.403393 16265 leader_election.cc:304] T 8638ee363273426db1393dd81e5c7b72 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860
I20260709 13:21:15.403967 21154 raft_consensus.cc:2749] T e4dc6b0541ca431ebc757beac00521f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:21:15.401183 21277 raft_consensus.cc:3060] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:21:15.404920 16265 leader_election.cc:304] T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:21:15.405680 21318 leader_election.cc:290] T 3884f864a96d428785895e5463e5728f P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 9 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:15.406673 16265 leader_election.cc:304] T 5a959dbecc2648eb85d1cc550f5a3245 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:21:15.407346 20964 raft_consensus.cc:2749] T 8638ee363273426db1393dd81e5c7b72 P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:21:15.408046 21361 consensus_queue.cc:237] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:15.392668 21352 raft_consensus.cc:2749] T c9944d5e4b8c4109af9955c4f08c9095 P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:21:15.413592 16151 raft_consensus.cc:1275] T 1df93779209748b9860f6d893dd3de53 P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:21:15.418292 20236 raft_consensus.cc:493] T 434bff3082d141838068f323bdb0d1ed P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:15.409060 21343 raft_consensus.cc:3060] T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:21:15.417465 16191 leader_election.cc:304] T 18ad7e3af5504571a69d10c00750a01c P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 90f3da57796b488c9087a2f523adf045
I20260709 13:21:15.417979 21106 raft_consensus.cc:493] T b87147731a68400e8cce9b879daa7c5d P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:15.395568 21370 raft_consensus.cc:2804] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:21:15.427896 16191 leader_election.cc:304] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:21:15.401504 21216 raft_consensus.cc:2804] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:21:15.422245 20236 raft_consensus.cc:515] T 434bff3082d141838068f323bdb0d1ed P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:15.434826 16790 leader_election.cc:304] T 2794af5b99984efebdee564035841cb0 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860; no voters: 90f3da57796b488c9087a2f523adf045
I20260709 13:21:15.436627 21276 leader_election.cc:290] T c9944d5e4b8c4109af9955c4f08c9095 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:15.437877 21165 leader_election.cc:290] T 8cc959fb7f704fba9009c0595467d074 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:15.439476 21320 consensus_queue.cc:1048] T b87147731a68400e8cce9b879daa7c5d P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:21:15.439597 21354 raft_consensus.cc:515] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:15.415372 21089 raft_consensus.cc:697] T 19baa7bc367844c288bca8f81439c413 P cb82cc7bd6db46d980c75237a1402860 [term 15 LEADER]: Becoming Leader. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:21:15.433975 16300 pending_rounds.cc:85] T c9944d5e4b8c4109af9955c4f08c9095 P 90f3da57796b488c9087a2f523adf045: Aborting all ops after (but not including) 10
I20260709 13:21:15.444188 16191 leader_election.cc:304] T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 90f3da57796b488c9087a2f523adf045
I20260709 13:21:15.444502 16300 pending_rounds.cc:107] T c9944d5e4b8c4109af9955c4f08c9095 P 90f3da57796b488c9087a2f523adf045: Aborting uncommitted NO_OP operation due to leader change: 12.11
I20260709 13:21:15.444924 16300 raft_consensus.cc:2889] T c9944d5e4b8c4109af9955c4f08c9095 P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:21:15.446069 16191 leader_election.cc:304] T 829b4ad8c5a54eababeea1181b960d4b P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
I20260709 13:21:15.448539 20236 leader_election.cc:290] T 434bff3082d141838068f323bdb0d1ed P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:15.423599 21318 raft_consensus.cc:2804] T 66f75124215f4ca6bef831e174a792d8 P cb82cc7bd6db46d980c75237a1402860 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:21:15.421245 21340 leader_election.cc:290] T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 12 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:15.450726 16145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:15.452138 16145 raft_consensus.cc:2468] T ecc1e363230043fe9fef390f0b979f20 P 647a343a542d4a139fe106c19095129b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 14.
I20260709 13:21:15.453955 16191 leader_election.cc:304] T e5ad4eb281564936a59f41e387a47156 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:21:15.441099 21216 raft_consensus.cc:493] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:21:15.441601 21366 raft_consensus.cc:2804] T 18ad7e3af5504571a69d10c00750a01c P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:21:15.463260 16295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3884f864a96d428785895e5463e5728f" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:21:15.468554 16148 raft_consensus.cc:1275] T ee9ed67006144797b1de6ea8c2013a68 P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:21:15.434463 21370 raft_consensus.cc:493] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Starting leader election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:21:15.447023 21320 raft_consensus.cc:2804] T 2794af5b99984efebdee564035841cb0 P cb82cc7bd6db46d980c75237a1402860 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:21:15.473055 16294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9944d5e4b8c4109af9955c4f08c9095" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:15.447362 21337 raft_consensus.cc:2749] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:21:15.475271 16219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdbd78acb4cf46298e0de8ba0d130e19" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 16 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:15.476295 16220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c11074579b046d99e585c2818e18a2c" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:15.479130 16295 raft_consensus.cc:2393] T 3884f864a96d428785895e5463e5728f P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 in current term 9: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
I20260709 13:21:15.449674 21154 raft_consensus.cc:2804] T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:21:15.480865 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62fe3346cd3c42c4a55e14af9073bca4" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:21:15.481155 16145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62fe3346cd3c42c4a55e14af9073bca4" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:21:15.482697 16224 raft_consensus.cc:1275] T d2ab0ac8347641e6955e9bce7d90c1a2 P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 14 index: 13. (index mismatch)
I20260709 13:21:15.455482 21250 raft_consensus.cc:2749] T 829b4ad8c5a54eababeea1181b960d4b P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:21:15.459897 21158 raft_consensus.cc:3060] T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:21:15.485843 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:15.489144 16225 raft_consensus.cc:2468] T ecc1e363230043fe9fef390f0b979f20 P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 14.
I20260709 13:21:15.462566 21329 raft_consensus.cc:3060] T e5ad4eb281564936a59f41e387a47156 P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:21:15.479873 16790 leader_election.cc:304] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:21:15.460350 21304 leader_election.cc:290] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:21:15.448073 21106 raft_consensus.cc:515] T b87147731a68400e8cce9b879daa7c5d P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:15.491139 16220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746aabf856ab401c9ef5e859371e7b67" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 14 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:15.468591 21216 raft_consensus.cc:3060] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:21:15.465516 21318 raft_consensus.cc:697] T 66f75124215f4ca6bef831e174a792d8 P cb82cc7bd6db46d980c75237a1402860 [term 14 LEADER]: Becoming Leader. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:21:15.495961 16143 raft_consensus.cc:1275] T 84f2da98864b4425bf7c0055951fdd4b P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 13:21:15.475862 21370 raft_consensus.cc:3060] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:21:15.497146 16219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9944d5e4b8c4109af9955c4f08c9095" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:15.495878 16295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a96373bb0e14dce8a266b2fc6c58507" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:15.499358 16145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57a744f5aaf944678a63469cbf514015" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:15.502856 16822 raft_consensus.cc:1240] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860 [term 14 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 13. Current term is 14. Ops: []
I20260709 13:21:15.490947 21336 leader_election.cc:290] T d16dfad0c3b540e5b4b86492ddf87208 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 13 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:15.510653 21339 raft_consensus.cc:2749] T 5a959dbecc2648eb85d1cc550f5a3245 P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:21:15.505617 16145 raft_consensus.cc:2468] T 57a744f5aaf944678a63469cbf514015 P 647a343a542d4a139fe106c19095129b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 10.
I20260709 13:21:15.460500 21325 consensus_queue.cc:1059] T fffab26427e24d398e2695aad40ca134 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.032s
I20260709 13:21:15.488485 21320 raft_consensus.cc:697] T 2794af5b99984efebdee564035841cb0 P cb82cc7bd6db46d980c75237a1402860 [term 11 LEADER]: Becoming Leader. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
W20260709 13:21:15.512014 16266 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:15.519011 16143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3884f864a96d428785895e5463e5728f" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:21:15.516264 16266 leader_election.cc:304] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 
W20260709 13:21:15.546865 16264 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:15.566702 16145 raft_consensus.cc:1240] T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 11. Current term is 12. Ops: []
I20260709 13:21:15.506455 21304 raft_consensus.cc:2749] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:21:15.564766 16266 leader_election.cc:304] T 66ae176961b74813a58bc15c9c4c6f2a P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:21:15.568738 16266 leader_election.cc:304] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:21:15.511904 21353 raft_consensus.cc:493] T 2794af5b99984efebdee564035841cb0 P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:21:15.564875 16265 leader_election.cc:304] T d16dfad0c3b540e5b4b86492ddf87208 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:21:15.460446 21089 consensus_queue.cc:237] T 19baa7bc367844c288bca8f81439c413 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 15, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:21:15.570590 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 90797.7us
I20260709 13:21:15.576289 16790 leader_election.cc:304] T ee9ed67006144797b1de6ea8c2013a68 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac
I20260709 13:21:15.526430 21369 raft_consensus.cc:2749] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:21:15.577832 16790 leader_election.cc:304] T 57a744f5aaf944678a63469cbf514015 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:21:15.569190 21352 raft_consensus.cc:2804] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:21:15.572786 21211 raft_consensus.cc:2804] T 66ae176961b74813a58bc15c9c4c6f2a P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:21:15.515326 21277 raft_consensus.cc:515] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:15.593899 16145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746aabf856ab401c9ef5e859371e7b67" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 14 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:15.571326 21354 leader_election.cc:290] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 15 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:21:15.569907 21154 raft_consensus.cc:697] T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac [term 13 LEADER]: Becoming Leader. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:21:15.577697 21339 raft_consensus.cc:2804] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 13:21:15.611291 16264 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:15.585983 20902 raft_consensus.cc:2804] T ee9ed67006144797b1de6ea8c2013a68 P cb82cc7bd6db46d980c75237a1402860 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:21:15.623756 21366 raft_consensus.cc:697] T 18ad7e3af5504571a69d10c00750a01c P 1ca2725d290e47d68b1a4b609c9816ac [term 12 LEADER]: Becoming Leader. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
W20260709 13:21:15.631208 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 110419us
I20260709 13:21:15.603572 21211 raft_consensus.cc:493] T 66ae176961b74813a58bc15c9c4c6f2a P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:15.636096 16149 raft_consensus.cc:1275] T 746aabf856ab401c9ef5e859371e7b67 P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 12 index: 12. Preceding OpId from leader: term: 13 index: 12. (term mismatch)
I20260709 13:21:15.638950 16149 pending_rounds.cc:85] T 746aabf856ab401c9ef5e859371e7b67 P 647a343a542d4a139fe106c19095129b: Aborting all ops after (but not including) 11
I20260709 13:21:15.639344 16149 pending_rounds.cc:107] T 746aabf856ab401c9ef5e859371e7b67 P 647a343a542d4a139fe106c19095129b: Aborting uncommitted NO_OP operation due to leader change: 12.12
I20260709 13:21:15.603080 21304 raft_consensus.cc:2804] T 57a744f5aaf944678a63469cbf514015 P cb82cc7bd6db46d980c75237a1402860 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:21:15.639866 16149 raft_consensus.cc:2889] T 746aabf856ab401c9ef5e859371e7b67 P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:21:15.593164 21352 raft_consensus.cc:493] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Starting leader election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
W20260709 13:21:15.641573 21379 log.cc:927] Time spent T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.066s	user 0.000s	sys 0.001s
I20260709 13:21:15.619761 21357 leader_election.cc:290] T c0eab25ce4c64e018a041beeda2867db P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:15.647977 21304 raft_consensus.cc:493] T 57a744f5aaf944678a63469cbf514015 P cb82cc7bd6db46d980c75237a1402860 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:15.669100 21154 consensus_queue.cc:237] T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:21:15.672963 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 95197us
I20260709 13:21:15.626530 21339 raft_consensus.cc:493] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:15.629259 20902 raft_consensus.cc:493] T ee9ed67006144797b1de6ea8c2013a68 P cb82cc7bd6db46d980c75237a1402860 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:21:15.625312 21338 raft_consensus.cc:2749] T d16dfad0c3b540e5b4b86492ddf87208 P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:21:15.687275 16790 leader_election.cc:304] T 8c11074579b046d99e585c2818e18a2c P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:21:15.649109 21211 raft_consensus.cc:3060] T 66ae176961b74813a58bc15c9c4c6f2a P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:21:15.675962 21354 raft_consensus.cc:493] T c0eab25ce4c64e018a041beeda2867db P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:15.689993 21304 raft_consensus.cc:3060] T 57a744f5aaf944678a63469cbf514015 P cb82cc7bd6db46d980c75237a1402860 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:21:15.599577 21318 consensus_queue.cc:237] T 66f75124215f4ca6bef831e174a792d8 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:15.709908 21277 leader_election.cc:290] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 16 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:15.693813 20902 raft_consensus.cc:3060] T ee9ed67006144797b1de6ea8c2013a68 P cb82cc7bd6db46d980c75237a1402860 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:21:15.712435 21388 raft_consensus.cc:2749] T 8c11074579b046d99e585c2818e18a2c P cb82cc7bd6db46d980c75237a1402860 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:21:15.736629 16265 leader_election.cc:304] T 5a96373bb0e14dce8a266b2fc6c58507 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:21:15.664655 21352 raft_consensus.cc:3060] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:21:15.717753 21106 leader_election.cc:290] T b87147731a68400e8cce9b879daa7c5d P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
W20260709 13:21:15.732314 21383 log.cc:927] Time spent T 23f82224f4564fea93572e8206bad1fe P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 0.082s	user 0.000s	sys 0.000s
I20260709 13:21:15.690721 21339 raft_consensus.cc:3060] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:21:15.697666 20964 raft_consensus.cc:493] T 7c2095a5f4ad49408dbc7f69c606120d P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
W20260709 13:21:15.756896 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 69736.3us
I20260709 13:21:15.583679 21353 raft_consensus.cc:515] T 2794af5b99984efebdee564035841cb0 P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:15.634958 21320 consensus_queue.cc:237] T 2794af5b99984efebdee564035841cb0 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:15.761564 21388 consensus_queue.cc:1048] T d2ab0ac8347641e6955e9bce7d90c1a2 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.014s
I20260709 13:21:15.777037 21388 raft_consensus.cc:493] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:15.738895 21354 raft_consensus.cc:515] T c0eab25ce4c64e018a041beeda2867db P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:15.783756 21394 consensus_queue.cc:1059] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:21:15.786888 16146 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46224 (request call id 778) took 1100 ms. Trace:
I20260709 13:21:15.787340 16146 rpcz_store.cc:276] 0709 13:21:14.686473 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:14.697566 (+ 11093us) service_pool.cc:224] Handling call
0709 13:21:14.735483 (+ 37917us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:21:14.743357 (+  7874us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:21:14.743386 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:21:15.426546 (+683160us) spinlock_profiling.cc:243] Waited 5.37 ms on lock 0x7b2c00000658. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69af458a11 00007f69af468372 00007f69b0b9aa01 00007f69b0b5f43a 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2
0709 13:21:15.431605 (+  5059us) log.cc:844] Serialized 26 byte log entry
0709 13:21:15.543575 (+111970us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:21:15.543643 (+    68us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:15.543749 (+   106us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:21:15.786598 (+242849us) raft_consensus.cc:1685] finished
0709 13:21:15.786617 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:21:15.786828 (+   211us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":12069,"raft.run_cpu_time_us":81,"raft.run_wall_time_us":81,"spinlock_wait_cycles":12350848,"thread_start_us":88342,"threads_started":1,"wal-append.queue_time_us":137300}
I20260709 13:21:15.799605 16143 raft_consensus.cc:2468] T 3884f864a96d428785895e5463e5728f P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 9.
W20260709 13:21:15.782413 21151 consensus_meta.cc:287] T d16dfad0c3b540e5b4b86492ddf87208 P 1ca2725d290e47d68b1a4b609c9816ac: Time spent flushing consensus metadata: real 0.777s	user 0.010s	sys 0.009s
I20260709 13:21:15.801543 16265 leader_election.cc:304] T 68a6838d82ef4cac83507f906139d90f P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:21:15.803941 16265 leader_election.cc:304] T 8c11074579b046d99e585c2818e18a2c P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860
I20260709 13:21:15.760773 20964 raft_consensus.cc:515] T 7c2095a5f4ad49408dbc7f69c606120d P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:15.773504 21390 raft_consensus.cc:2749] T 5a96373bb0e14dce8a266b2fc6c58507 P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:21:15.806898 20964 leader_election.cc:290] T 7c2095a5f4ad49408dbc7f69c606120d P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:21:15.777637 21388 raft_consensus.cc:515] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:15.812266 21393 raft_consensus.cc:2749] T 68a6838d82ef4cac83507f906139d90f P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:21:15.813531 21390 raft_consensus.cc:2749] T 8c11074579b046d99e585c2818e18a2c P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:21:15.826658 16790 leader_election.cc:304] T 3884f864a96d428785895e5463e5728f P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860; no voters: 
W20260709 13:21:15.832201 21382 log.cc:927] Time spent T c4f65cdf00b748bb963ee6e81ef43b79 P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.127s	user 0.001s	sys 0.000s
I20260709 13:21:15.833159 21211 raft_consensus.cc:515] T 66ae176961b74813a58bc15c9c4c6f2a P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:15.836191 21343 raft_consensus.cc:2749] T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:21:15.771611 21366 consensus_queue.cc:237] T 18ad7e3af5504571a69d10c00750a01c P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } }
I20260709 13:21:15.839596 21353 leader_election.cc:290] T 2794af5b99984efebdee564035841cb0 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:15.843252 16292 raft_consensus.cc:1240] T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045 [term 14 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 13. Current term is 14. Ops: []
I20260709 13:21:15.847998 16822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2794af5b99984efebdee564035841cb0" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:15.848526 16820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66ae176961b74813a58bc15c9c4c6f2a" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:21:15.851971 21211 leader_election.cc:290] T 66ae176961b74813a58bc15c9c4c6f2a P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 17 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:15.856890 20902 raft_consensus.cc:515] T ee9ed67006144797b1de6ea8c2013a68 P cb82cc7bd6db46d980c75237a1402860 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:15.811605 21151 raft_consensus.cc:515] T d16dfad0c3b540e5b4b86492ddf87208 P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:15.859566 20902 leader_election.cc:290] T ee9ed67006144797b1de6ea8c2013a68 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 13 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:21:15.859719 21151 leader_election.cc:290] T d16dfad0c3b540e5b4b86492ddf87208 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 14 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:15.861311 20902 raft_consensus.cc:2804] T 3884f864a96d428785895e5463e5728f P cb82cc7bd6db46d980c75237a1402860 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:21:15.861832 20902 raft_consensus.cc:697] T 3884f864a96d428785895e5463e5728f P cb82cc7bd6db46d980c75237a1402860 [term 9 LEADER]: Becoming Leader. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:21:15.864370 16218 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57604 (request call id 2778) took 1344 ms. Trace:
I20260709 13:21:15.864733 16218 rpcz_store.cc:276] 0709 13:21:14.519507 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:14.559252 (+ 39745us) service_pool.cc:224] Handling call
0709 13:21:14.573088 (+ 13836us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:21:14.687463 (+114375us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:21:14.951798 (+264335us) log.cc:844] Serialized 14 byte log entry
0709 13:21:15.462189 (+510391us) spinlock_profiling.cc:243] Waited 1.2 ms on lock 0x7b18004d12e0. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b693ade1 00007f69b69475b1 00007f69ada54e07 00007f69b68afab9 00007f69ada7169b 00007f69ada6f93f 00007f69ada71c02 00007f69b06f920b 00007f69b06fed23 00007f69b06ff23a 00007f69b0b69a2a
0709 13:21:15.739362 (+277173us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:21:15.752864 (+ 13502us) log.cc:844] Serialized 26 byte log entry
0709 13:21:15.753101 (+   237us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:21:15.753388 (+   287us) log.cc:844] Serialized 14 byte log entry
0709 13:21:15.757413 (+  4025us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:15.757492 (+    79us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:21:15.864147 (+106655us) raft_consensus.cc:1685] finished
0709 13:21:15.864175 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:21:15.864311 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":39013,"prepare.queue_time_us":233385,"prepare.run_cpu_time_us":70,"prepare.run_wall_time_us":72,"raft.queue_time_us":55611,"raft.run_cpu_time_us":496,"raft.run_wall_time_us":46979,"spinlock_wait_cycles":2769280,"thread_start_us":489223,"threads_started":3,"wal-append.queue_time_us":499762}
I20260709 13:21:15.862447 21304 raft_consensus.cc:515] T 57a744f5aaf944678a63469cbf514015 P cb82cc7bd6db46d980c75237a1402860 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:21:15.873447 21350 log.cc:927] Time spent T ee9ed67006144797b1de6ea8c2013a68 P cb82cc7bd6db46d980c75237a1402860: Append to log took a long time: real 0.910s	user 0.005s	sys 0.000s
I20260709 13:21:15.874780 16821 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49906 (request call id 1026) took 1317 ms. Trace:
I20260709 13:21:15.875674 16221 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33050 (request call id 873) took 1350 ms. Trace:
W20260709 13:21:15.876482 16266 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:21:15.878371 21380 log.cc:927] Time spent T 8cc959fb7f704fba9009c0595467d074 P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.100s	user 0.001s	sys 0.000s
I20260709 13:21:15.887053 21342 leader_election.cc:290] T 8158bbd17b994d2eb96f560bd0f1b88d P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 14 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:15.875118 16821 rpcz_store.cc:276] 0709 13:21:14.557309 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:14.557451 (+   142us) service_pool.cc:224] Handling call
0709 13:21:14.557962 (+   511us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:21:14.558313 (+   351us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:21:14.558351 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:21:14.558887 (+   536us) log.cc:844] Serialized 26 byte log entry
0709 13:21:14.875571 (+316684us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:21:14.875642 (+    71us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:14.875751 (+   109us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:21:15.872879 (+997128us) spinlock_profiling.cc:243] Waited 18.7 ms on lock 0x7b4400022b4c. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69ad8fadb1 00007f69ad998c85 00007f69ad9996a2 00007f69b070b1db 00007f69b0706aaf 00007f69b06fa86e 00007f69b06f9bb6 00007f69b06f97d8 00007f69b0703221 00007f69b07031d9 00007f69b0703169
0709 13:21:15.874478 (+  1599us) raft_consensus.cc:1685] finished
0709 13:21:15.874499 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:21:15.874704 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7163,"raft.queue_time_us":782,"raft.run_cpu_time_us":120,"raft.run_wall_time_us":5016,"spinlock_wait_cycles":43072384,"thread_start_us":234786,"threads_started":1,"wal-append.queue_time_us":363319}
I20260709 13:21:15.876085 16221 rpcz_store.cc:276] 0709 13:21:14.525655 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:14.597921 (+ 72266us) service_pool.cc:224] Handling call
0709 13:21:14.705314 (+107393us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:21:14.706032 (+   718us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:15.875484 (+1169452us) spinlock_profiling.cc:243] Waited 107 ms on lock 0x7b5400395650. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5e8ad 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:21:15.875618 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":246222720}
W20260709 13:21:15.897946 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 71371.8us
I20260709 13:21:15.900419 16143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee9ed67006144797b1de6ea8c2013a68" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:21:15.901146 16143 raft_consensus.cc:3060] T ee9ed67006144797b1de6ea8c2013a68 P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:21:15.901371 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434bff3082d141838068f323bdb0d1ed" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:15.902063 16297 raft_consensus.cc:2468] T 434bff3082d141838068f323bdb0d1ed P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 12.
I20260709 13:21:15.902447 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8158bbd17b994d2eb96f560bd0f1b88d" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:21:15.902071 16294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d16dfad0c3b540e5b4b86492ddf87208" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:21:15.862833 20902 consensus_queue.cc:237] T 3884f864a96d428785895e5463e5728f P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:15.903465 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cc959fb7f704fba9009c0595467d074" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:15.903493 16294 raft_consensus.cc:2393] T d16dfad0c3b540e5b4b86492ddf87208 P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 13: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
I20260709 13:21:15.911479 21182 raft_consensus.cc:3055] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:21:15.911975 21182 raft_consensus.cc:740] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:21:15.915512 16295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d16dfad0c3b540e5b4b86492ddf87208" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:21:15.916447 16295 raft_consensus.cc:3060] T d16dfad0c3b540e5b4b86492ddf87208 P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:21:15.917695 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0eab25ce4c64e018a041beeda2867db" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:15.923990 16294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a378d7f796f94615b3b347119a9880fd" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:15.924844 16294 raft_consensus.cc:2468] T a378d7f796f94615b3b347119a9880fd P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 15.
I20260709 13:21:15.925647 16297 raft_consensus.cc:1275] T 84f2da98864b4425bf7c0055951fdd4b P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 13:21:15.925889 16294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87147731a68400e8cce9b879daa7c5d" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 17 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:15.926724 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:21:15.927327 16297 raft_consensus.cc:3060] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:21:15.927496 21392 raft_consensus.cc:493] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:15.928258 21392 raft_consensus.cc:515] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } }
I20260709 13:21:15.930480 21370 raft_consensus.cc:515] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:15.918493 16293 raft_consensus.cc:2468] T c0eab25ce4c64e018a041beeda2867db P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 13.
I20260709 13:21:15.941761 16297 raft_consensus.cc:2468] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 16.
I20260709 13:21:15.942651 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68a6838d82ef4cac83507f906139d90f" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:15.947422 16299 raft_consensus.cc:2410] T 68a6838d82ef4cac83507f906139d90f P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac for term 16 because replica has last-logged OpId of term: 15 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 7.
I20260709 13:21:15.930747 21392 leader_election.cc:290] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:15.903652 21354 leader_election.cc:290] T c0eab25ce4c64e018a041beeda2867db P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:21:15.959892 16190 leader_election.cc:304] T 5a96373bb0e14dce8a266b2fc6c58507 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:21:15.969780 16294 raft_consensus.cc:1240] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 14. Current term is 15. Ops: []
I20260709 13:21:15.973093 16190 leader_election.cc:304] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
W20260709 13:21:15.979853 16189 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:15.978088 16143 raft_consensus.cc:2468] T ee9ed67006144797b1de6ea8c2013a68 P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 13.
I20260709 13:21:15.982924 16295 raft_consensus.cc:2468] T d16dfad0c3b540e5b4b86492ddf87208 P 90f3da57796b488c9087a2f523adf045 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 14.
I20260709 13:21:15.981783 16790 leader_election.cc:304] T ee9ed67006144797b1de6ea8c2013a68 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:21:15.985801 16221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:15.987123 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fffab26427e24d398e2695aad40ca134" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:21:15.986624 16221 raft_consensus.cc:2468] T 19baa7bc367844c288bca8f81439c413 P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 15.
I20260709 13:21:15.981415 21216 raft_consensus.cc:515] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:21:15.991140 21395 log.cc:927] Time spent T b87147731a68400e8cce9b879daa7c5d P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 0.119s	user 0.001s	sys 0.000s
I20260709 13:21:15.992075 21216 leader_election.cc:290] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 13 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:21:15.995703 16222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5ad4eb281564936a59f41e387a47156" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:21:15.996145 16226 raft_consensus.cc:1240] T 746aabf856ab401c9ef5e859371e7b67 P 1ca2725d290e47d68b1a4b609c9816ac [term 13 LEADER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 12. Current term is 13. Ops: []
I20260709 13:21:15.975608 21182 consensus_queue.cc:260] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:15.997318 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18ad7e3af5504571a69d10c00750a01c" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:15.999740 16221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c74490c40243a88e62d1760b5cd814" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
W20260709 13:21:16.002844 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 199047us
I20260709 13:21:16.003578 16221 raft_consensus.cc:3060] T 08c74490c40243a88e62d1760b5cd814 P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:21:16.005005 21388 leader_election.cc:290] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:21:16.006901 21370 leader_election.cc:290] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 12 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:15.968878 21339 raft_consensus.cc:515] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:16.007861 21250 raft_consensus.cc:3055] T fffab26427e24d398e2695aad40ca134 P 1ca2725d290e47d68b1a4b609c9816ac [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:21:16.008279 21250 raft_consensus.cc:740] T fffab26427e24d398e2695aad40ca134 P 1ca2725d290e47d68b1a4b609c9816ac [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:21:16.002255 16220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f2da98864b4425bf7c0055951fdd4b" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:21:15.979915 21369 raft_consensus.cc:2804] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:21:16.009318 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d16dfad0c3b540e5b4b86492ddf87208" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:21:16.011212 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b7d1ab519044eda2d5e04a07242f25" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:21:16.012018 16224 raft_consensus.cc:2368] T d16dfad0c3b540e5b4b86492ddf87208 P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b for earlier term 13. Current term is 14.
I20260709 13:21:16.011831 16265 leader_election.cc:304] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:21:15.976008 21387 raft_consensus.cc:2749] T 5a96373bb0e14dce8a266b2fc6c58507 P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:21:16.013439 16145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5718c623ebee4d2bb770f0f9ddb2f164" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:21:16.017794 21410 raft_consensus.cc:2749] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:21:16.018149 16146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57a744f5aaf944678a63469cbf514015" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:21:16.018332 16220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8158bbd17b994d2eb96f560bd0f1b88d" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:21:16.018817 16146 raft_consensus.cc:3060] T 57a744f5aaf944678a63469cbf514015 P 647a343a542d4a139fe106c19095129b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:21:16.019205 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cc959fb7f704fba9009c0595467d074" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:16.020742 16226 raft_consensus.cc:1275] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 13:21:16.022020 16226 raft_consensus.cc:1275] T 829b4ad8c5a54eababeea1181b960d4b P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:21:16.018837 16220 raft_consensus.cc:3060] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:21:16.023492 21211 consensus_queue.cc:1048] T 8d4ab870a05347868956aa8b841b2981 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:21:16.025710 21353 consensus_queue.cc:1048] T 8158bbd17b994d2eb96f560bd0f1b88d P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:21:16.016362 16144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2372981006d4b218215c28964dea623" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:15.998843 21361 raft_consensus.cc:2804] T ee9ed67006144797b1de6ea8c2013a68 P cb82cc7bd6db46d980c75237a1402860 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:21:16.032308 21361 raft_consensus.cc:697] T ee9ed67006144797b1de6ea8c2013a68 P cb82cc7bd6db46d980c75237a1402860 [term 13 LEADER]: Becoming Leader. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:21:16.033252 21361 consensus_queue.cc:237] T ee9ed67006144797b1de6ea8c2013a68 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.11, Last appended by leader: 11, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:16.033838 16816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b7d1ab519044eda2d5e04a07242f25" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:21:16.016666 21354 consensus_queue.cc:1048] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.018s
I20260709 13:21:16.035099 16817 raft_consensus.cc:1275] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 [term 16 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:21:16.028302 16822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a378d7f796f94615b3b347119a9880fd" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:16.035707 16821 raft_consensus.cc:1275] T c9944d5e4b8c4109af9955c4f08c9095 P cb82cc7bd6db46d980c75237a1402860 [term 15 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:21:16.035974 16822 raft_consensus.cc:2468] T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 15.
I20260709 13:21:16.036522 21339 leader_election.cc:290] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 12 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:16.037169 21250 consensus_queue.cc:260] T fffab26427e24d398e2695aad40ca134 P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:16.038407 21250 raft_consensus.cc:3060] T fffab26427e24d398e2695aad40ca134 P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:21:16.013829 21352 raft_consensus.cc:515] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:16.040481 21352 leader_election.cc:290] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 16 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:15.982043 21325 consensus_queue.cc:1048] T 84f2da98864b4425bf7c0055951fdd4b P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.031s
I20260709 13:21:16.011586 21369 raft_consensus.cc:493] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:16.043992 21369 raft_consensus.cc:3060] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:21:16.045390 21368 log.cc:927] Time spent T fffab26427e24d398e2695aad40ca134 P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 0.712s	user 0.001s	sys 0.000s
I20260709 13:21:16.045508 16297 raft_consensus.cc:1275] T cdbd78acb4cf46298e0de8ba0d130e19 P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:21:16.048892 16821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a9de347c711477e87c582d003f36925" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:21:16.054008 16295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806d7690c2b644959152074bb2cab0a7" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:16.054831 16295 raft_consensus.cc:2393] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 14: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
I20260709 13:21:16.056322 16190 leader_election.cc:304] T 68a6838d82ef4cac83507f906139d90f P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:21:16.057935 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806d7690c2b644959152074bb2cab0a7" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:16.058306 16190 leader_election.cc:304] T d16dfad0c3b540e5b4b86492ddf87208 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:21:16.058583 16149 raft_consensus.cc:2468] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 13.
I20260709 13:21:16.059530 16145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d16dfad0c3b540e5b4b86492ddf87208" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:21:16.059675 21370 raft_consensus.cc:2804] T d16dfad0c3b540e5b4b86492ddf87208 P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:21:16.060242 21370 raft_consensus.cc:697] T d16dfad0c3b540e5b4b86492ddf87208 P 1ca2725d290e47d68b1a4b609c9816ac [term 14 LEADER]: Becoming Leader. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:21:16.059523 21087 raft_consensus.cc:2749] T 68a6838d82ef4cac83507f906139d90f P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:21:16.061102 21370 consensus_queue.cc:237] T d16dfad0c3b540e5b4b86492ddf87208 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:16.061882 16145 raft_consensus.cc:1240] T fffab26427e24d398e2695aad40ca134 P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 12. Current term is 13. Ops: [12.7-12.7]
W20260709 13:21:16.064882 21329 consensus_meta.cc:287] T e5ad4eb281564936a59f41e387a47156 P 1ca2725d290e47d68b1a4b609c9816ac: Time spent flushing consensus metadata: real 0.566s	user 0.000s	sys 0.010s
I20260709 13:21:16.065305 21329 raft_consensus.cc:2749] T e5ad4eb281564936a59f41e387a47156 P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:21:16.065748 16822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e9b2bf7011f4ae688a79d7bb5a447b4" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:21:16.066362 16822 raft_consensus.cc:3060] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:21:16.066298 21325 consensus_queue.cc:1048] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
W20260709 13:21:16.067541 21163 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 13 current term: 13 is higher.
I20260709 13:21:16.072914 16220 raft_consensus.cc:2468] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 14.
I20260709 13:21:16.074627 16821 raft_consensus.cc:2468] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 16.
I20260709 13:21:16.075471 16146 raft_consensus.cc:2468] T 57a744f5aaf944678a63469cbf514015 P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 11.
I20260709 13:21:16.073057 16222 raft_consensus.cc:2468] T e5ad4eb281564936a59f41e387a47156 P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 14.
W20260709 13:21:16.078850 16264 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:16.086318 16790 leader_election.cc:304] T 57a744f5aaf944678a63469cbf514015 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:21:16.099691 16266 leader_election.cc:304] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:21:16.099942 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b7d1ab519044eda2d5e04a07242f25" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:21:16.099810 16221 raft_consensus.cc:2468] T 08c74490c40243a88e62d1760b5cd814 P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 14.
I20260709 13:21:16.111881 16150 raft_consensus.cc:2393] T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 12: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
I20260709 13:21:16.116176 16790 leader_election.cc:304] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:21:16.117825 16790 leader_election.cc:304] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:21:16.119141 16790 leader_election.cc:304] T c9944d5e4b8c4109af9955c4f08c9095 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:21:16.124593 21398 raft_consensus.cc:2804] T 57a744f5aaf944678a63469cbf514015 P cb82cc7bd6db46d980c75237a1402860 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:21:16.126705 21401 raft_consensus.cc:3055] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:21:16.127409 21401 raft_consensus.cc:740] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:21:16.094305 21198 raft_consensus.cc:493] T 434bff3082d141838068f323bdb0d1ed P cb82cc7bd6db46d980c75237a1402860 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:16.126593 21369 raft_consensus.cc:515] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:16.119796 21352 raft_consensus.cc:2804] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:21:16.132365 21388 raft_consensus.cc:2749] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
W20260709 13:21:16.152129 21395 log.cc:927] Time spent T b87147731a68400e8cce9b879daa7c5d P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 0.053s	user 0.001s	sys 0.000s
I20260709 13:21:16.132850 21089 raft_consensus.cc:2749] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:21:16.132642 21398 raft_consensus.cc:697] T 57a744f5aaf944678a63469cbf514015 P cb82cc7bd6db46d980c75237a1402860 [term 11 LEADER]: Becoming Leader. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:21:16.181141 16221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0eab25ce4c64e018a041beeda2867db" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:16.181603 16220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:21:16.182070 16221 raft_consensus.cc:2468] T c0eab25ce4c64e018a041beeda2867db P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 13.
I20260709 13:21:16.182607 16220 raft_consensus.cc:3060] T 19baa7bc367844c288bca8f81439c413 P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:21:16.195989 16192 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 101864us
I20260709 13:21:16.135449 21329 consensus_queue.cc:1048] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.041s
I20260709 13:21:16.198144 16221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4f65cdf00b748bb963ee6e81ef43b79" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:16.202914 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f5f89186204242bf377f7c42b2e2da" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:16.203909 16225 raft_consensus.cc:2468] T c2f5f89186204242bf377f7c42b2e2da P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 14.
I20260709 13:21:16.205363 16221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e4bfb9090c8460eb14fa7e0d818e7f1" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: true dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:21:16.207960 16221 raft_consensus.cc:2393] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 13: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
I20260709 13:21:16.157997 21198 raft_consensus.cc:515] T 434bff3082d141838068f323bdb0d1ed P cb82cc7bd6db46d980c75237a1402860 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:16.175438 21180 raft_consensus.cc:493] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:21:16.213084 16219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ab0ac8347641e6955e9bce7d90c1a2" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:16.213308 16221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a878999c0024d2abe517484180290fc" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:16.219853 16219 raft_consensus.cc:2393] T d2ab0ac8347641e6955e9bce7d90c1a2 P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 14: Already voted for candidate cb82cc7bd6db46d980c75237a1402860 in this term.
I20260709 13:21:16.219762 16221 raft_consensus.cc:2468] T 8a878999c0024d2abe517484180290fc P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 12.
I20260709 13:21:16.188905 21352 raft_consensus.cc:697] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [term 16 LEADER]: Becoming Leader. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
W20260709 13:21:16.236814 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 117758us
I20260709 13:21:16.128671 21401 consensus_queue.cc:260] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:16.239387 16221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87ac0d1f7e8a419f9bcbc04aee0e067e" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:21:16.239899 16790 leader_election.cc:304] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860; no voters: 90f3da57796b488c9087a2f523adf045
I20260709 13:21:16.236228 21180 raft_consensus.cc:515] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:16.244346 21180 leader_election.cc:290] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:16.252135 16221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c2095a5f4ad49408dbc7f69c606120d" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:16.253003 16221 raft_consensus.cc:2468] T 7c2095a5f4ad49408dbc7f69c606120d P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 9.
I20260709 13:21:16.197115 21391 raft_consensus.cc:493] T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:16.278215 16822 raft_consensus.cc:2468] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 12.
I20260709 13:21:16.263861 16220 raft_consensus.cc:2468] T 19baa7bc367844c288bca8f81439c413 P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 16.
I20260709 13:21:16.280802 16220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2372981006d4b218215c28964dea623" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:16.263975 16219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806d7690c2b644959152074bb2cab0a7" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:21:16.281487 16220 raft_consensus.cc:2468] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 15.
I20260709 13:21:16.281737 16219 raft_consensus.cc:3060] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:21:16.267234 16221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66ae176961b74813a58bc15c9c4c6f2a" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:21:16.282538 16220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5718c623ebee4d2bb770f0f9ddb2f164" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:21:16.268119 21405 raft_consensus.cc:2749] T c9944d5e4b8c4109af9955c4f08c9095 P cb82cc7bd6db46d980c75237a1402860 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:21:16.268476 21425 consensus_queue.cc:1048] T 1df93779209748b9860f6d893dd3de53 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:21:16.269856 21422 consensus_queue.cc:1048] T 84f2da98864b4425bf7c0055951fdd4b P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:21:16.271342 21180 consensus_queue.cc:1048] T 746aabf856ab401c9ef5e859371e7b67 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:21:16.273872 21403 raft_consensus.cc:2804] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:21:16.299494 21401 raft_consensus.cc:3060] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:21:16.323086 16218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0eab25ce4c64e018a041beeda2867db" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:16.323573 21403 raft_consensus.cc:493] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:16.323879 16218 raft_consensus.cc:2468] T c0eab25ce4c64e018a041beeda2867db P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 13.
I20260709 13:21:16.324048 21403 raft_consensus.cc:3060] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:21:16.325177 16218 raft_consensus.cc:1240] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 11. Current term is 12. Ops: []
I20260709 13:21:16.296435 16221 raft_consensus.cc:3060] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:21:16.303710 16790 leader_election.cc:304] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 647a343a542d4a139fe106c19095129b
I20260709 13:21:16.304805 21425 consensus_queue.cc:1048] T ee9ed67006144797b1de6ea8c2013a68 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
W20260709 13:21:16.307773 21375 log.cc:927] Time spent T 87ac0d1f7e8a419f9bcbc04aee0e067e P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 0.828s	user 0.000s	sys 0.005s
I20260709 13:21:16.308061 21319 raft_consensus.cc:493] T 26354361e9b044cea1c2b86c4882fe92 P cb82cc7bd6db46d980c75237a1402860 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:16.309499 21198 leader_election.cc:290] T 434bff3082d141838068f323bdb0d1ed P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:16.310425 21304 leader_election.cc:290] T 57a744f5aaf944678a63469cbf514015 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 11 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:16.311990 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66ae176961b74813a58bc15c9c4c6f2a" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:16.312669 21329 consensus_queue.cc:1048] T 8cc959fb7f704fba9009c0595467d074 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.011s
I20260709 13:21:16.314034 21392 raft_consensus.cc:493] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:21:16.299708 21318 consensus_queue.cc:1048] T af57996a8be84d398f0f023e8d08ef77 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.016s
I20260709 13:21:16.315658 21352 consensus_queue.cc:237] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:16.302737 21398 consensus_queue.cc:237] T 57a744f5aaf944678a63469cbf514015 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:16.280853 21385 consensus_queue.cc:1048] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 8, Time since last communication: 0.219s
I20260709 13:21:16.260887 21324 raft_consensus.cc:493] T 02838c86531d4f98bf58d1c22b8baf77 P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:21:16.320432 21320 raft_consensus.cc:493] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P cb82cc7bd6db46d980c75237a1402860 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:16.285744 21391 raft_consensus.cc:515] T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:16.351683 16218 raft_consensus.cc:1275] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:21:16.352751 16225 raft_consensus.cc:1240] T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac [term 13 LEADER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 12. Current term is 13. Ops: [12.7-12.7]
I20260709 13:21:16.338398 16219 raft_consensus.cc:2468] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 14.
W20260709 13:21:16.315711 21406 log.cc:927] Time spent T 19baa7bc367844c288bca8f81439c413 P cb82cc7bd6db46d980c75237a1402860: Append to log took a long time: real 0.363s	user 0.002s	sys 0.000s
W20260709 13:21:16.312098 21413 log.cc:927] Time spent T 1df93779209748b9860f6d893dd3de53 P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
I20260709 13:21:16.339259 16220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee9ed67006144797b1de6ea8c2013a68" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
W20260709 13:21:16.347355 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 43744.2us
I20260709 13:21:16.344597 21319 raft_consensus.cc:515] T 26354361e9b044cea1c2b86c4882fe92 P cb82cc7bd6db46d980c75237a1402860 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:16.361652 16220 raft_consensus.cc:1240] T 08c74490c40243a88e62d1760b5cd814 P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 13. Current term is 14. Ops: [11.8-13.9]
I20260709 13:21:16.370949 16265 leader_election.cc:304] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:21:16.374615 16265 leader_election.cc:304] T 8a878999c0024d2abe517484180290fc P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:21:16.377413 16265 leader_election.cc:304] T d2ab0ac8347641e6955e9bce7d90c1a2 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860
I20260709 13:21:16.378297 16821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74ca8140f35c4ae29e8a664f6bdfc829" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:16.353091 21405 raft_consensus.cc:2804] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 13:21:16.399636 16189 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:16.406303 16151 raft_consensus.cc:1240] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 15. Current term is 16. Ops: []
I20260709 13:21:16.410009 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434bff3082d141838068f323bdb0d1ed" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:16.409277 21369 leader_election.cc:290] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 16 election: Requested vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:16.410984 16150 raft_consensus.cc:2468] T 434bff3082d141838068f323bdb0d1ed P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 13.
I20260709 13:21:16.376773 21283 raft_consensus.cc:493] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:16.381431 21338 raft_consensus.cc:2804] T 8a878999c0024d2abe517484180290fc P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:21:16.426259 16790 leader_election.cc:304] T 434bff3082d141838068f323bdb0d1ed P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:21:16.409658 21405 raft_consensus.cc:493] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:16.427330 21216 raft_consensus.cc:2804] T 434bff3082d141838068f323bdb0d1ed P cb82cc7bd6db46d980c75237a1402860 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:21:16.427153 21304 consensus_queue.cc:1059] T 19baa7bc367844c288bca8f81439c413 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:21:16.427456 21405 raft_consensus.cc:3060] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:21:16.427817 21216 raft_consensus.cc:493] T 434bff3082d141838068f323bdb0d1ed P cb82cc7bd6db46d980c75237a1402860 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:16.428288 21216 raft_consensus.cc:3060] T 434bff3082d141838068f323bdb0d1ed P cb82cc7bd6db46d980c75237a1402860 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:21:16.353408 21392 raft_consensus.cc:515] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:16.356969 21320 raft_consensus.cc:515] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P cb82cc7bd6db46d980c75237a1402860 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:16.439349 21320 leader_election.cc:290] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:16.405483 21324 raft_consensus.cc:515] T 02838c86531d4f98bf58d1c22b8baf77 P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:16.431733 21338 raft_consensus.cc:493] T 8a878999c0024d2abe517484180290fc P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:21:16.388408 16218 raft_consensus.cc:1240] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 13. Current term is 14. Ops: [13.9-13.9]
I20260709 13:21:16.461790 21392 leader_election.cc:290] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:16.461369 21216 raft_consensus.cc:515] T 434bff3082d141838068f323bdb0d1ed P cb82cc7bd6db46d980c75237a1402860 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:21:16.465128 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 87804us
I20260709 13:21:16.469117 16265 leader_election.cc:304] T 7c2095a5f4ad49408dbc7f69c606120d P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:21:16.471880 16295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74ca8140f35c4ae29e8a664f6bdfc829" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
W20260709 13:21:16.562160 21421 log.cc:927] Time spent T 84f2da98864b4425bf7c0055951fdd4b P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.260s	user 0.000s	sys 0.004s
I20260709 13:21:16.472628 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a378d7f796f94615b3b347119a9880fd" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:21:16.472826 21391 leader_election.cc:290] T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:16.474298 16298 raft_consensus.cc:3055] T 8d4ab870a05347868956aa8b841b2981 P 90f3da57796b488c9087a2f523adf045 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:21:16.565852 16297 raft_consensus.cc:3060] T a378d7f796f94615b3b347119a9880fd P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:21:16.565991 16298 raft_consensus.cc:740] T 8d4ab870a05347868956aa8b841b2981 P 90f3da57796b488c9087a2f523adf045 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:21:16.566992 16298 consensus_queue.cc:260] T 8d4ab870a05347868956aa8b841b2981 P 90f3da57796b488c9087a2f523adf045 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:16.568011 16298 raft_consensus.cc:3060] T 8d4ab870a05347868956aa8b841b2981 P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:21:16.474751 16189 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:16.570514 16190 leader_election.cc:304] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860
I20260709 13:21:16.464073 21338 raft_consensus.cc:3060] T 8a878999c0024d2abe517484180290fc P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:21:16.477313 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c61a39fd4f84424b129c1a6f9ba404d" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:16.472630 21403 raft_consensus.cc:515] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:16.479714 16217 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57604 (request call id 2772) took 2089 ms. Trace:
I20260709 13:21:16.467712 21354 raft_consensus.cc:2749] T d2ab0ac8347641e6955e9bce7d90c1a2 P 90f3da57796b488c9087a2f523adf045 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:21:16.485139 16816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434bff3082d141838068f323bdb0d1ed" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:16.485812 16817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b7d1ab519044eda2d5e04a07242f25" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:21:16.488711 16294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02838c86531d4f98bf58d1c22b8baf77" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:16.491822 21324 leader_election.cc:290] T 02838c86531d4f98bf58d1c22b8baf77 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:16.493113 21405 raft_consensus.cc:515] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:16.500278 16824 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49896 (request call id 1271) took 1237 ms. Trace:
I20260709 13:21:16.500545 16819 raft_consensus.cc:1275] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860 [term 13 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 13:21:16.507586 16823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:21:16.508850 16825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87147731a68400e8cce9b879daa7c5d" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 17 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:16.509531 16821 raft_consensus.cc:1275] T 829b4ad8c5a54eababeea1181b960d4b P cb82cc7bd6db46d980c75237a1402860 [term 14 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:21:16.510638 21319 leader_election.cc:290] T 26354361e9b044cea1c2b86c4882fe92 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:16.512068 16147 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46224 (request call id 782) took 1816 ms. Trace:
I20260709 13:21:16.512677 21245 raft_consensus.cc:2749] T 9e65676b21b748728ad39668cfe722ec P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:21:16.431421 21283 raft_consensus.cc:515] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:16.516981 21216 leader_election.cc:290] T 434bff3082d141838068f323bdb0d1ed P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 14 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:16.518263 16292 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48684 (request call id 2737) took 1107 ms. Trace:
W20260709 13:21:16.520478 16115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 447229us
I20260709 13:21:16.605496 16221 raft_consensus.cc:2468] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 17.
W20260709 13:21:16.523233 16116 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:16.523595 21336 raft_consensus.cc:493] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:16.523664 20991 raft_consensus.cc:493] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:16.577951 21391 raft_consensus.cc:2749] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:21:16.530767 16146 raft_consensus.cc:1240] T 08c74490c40243a88e62d1760b5cd814 P 647a343a542d4a139fe106c19095129b [term 14 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 13. Current term is 14. Ops: []
I20260709 13:21:16.532969 16143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4f65cdf00b748bb963ee6e81ef43b79" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:16.535315 16149 raft_consensus.cc:1217] T b87147731a68400e8cce9b879daa7c5d P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Deduplicated request from leader. Original: 13.10->[15.11-15.11]   Dedup: 15.11->[]
I20260709 13:21:16.625422 16149 raft_consensus.cc:1240] T b87147731a68400e8cce9b879daa7c5d P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 15. Current term is 16. Ops: [15.11-15.11]
I20260709 13:21:16.537606 16144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26354361e9b044cea1c2b86c4882fe92" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:16.626672 16116 leader_election.cc:304] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:21:16.627107 16144 raft_consensus.cc:2468] T 26354361e9b044cea1c2b86c4882fe92 P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 11.
I20260709 13:21:16.539990 16145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a9de347c711477e87c582d003f36925" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:16.541468 16148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f5f89186204242bf377f7c42b2e2da" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:16.542618 16150 raft_consensus.cc:1240] T 8d4ab870a05347868956aa8b841b2981 P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 12. Current term is 13. Ops: []
I20260709 13:21:16.543143 16151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806d7690c2b644959152074bb2cab0a7" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:21:16.551734 16142 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40844 (request call id 2543) took 1307 ms. Trace:
I20260709 13:21:16.630734 16142 rpcz_store.cc:276] 0709 13:21:15.244459 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:15.261158 (+ 16699us) service_pool.cc:224] Handling call
0709 13:21:16.551508 (+1290350us) spinlock_profiling.cc:243] Waited 76.3 ms on lock 0x7b540013f1d4. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:21:16.551640 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":175469824}
I20260709 13:21:16.554689 16224 raft_consensus.cc:1240] T e5ad4eb281564936a59f41e387a47156 P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 13. Current term is 14. Ops: []
I20260709 13:21:16.555625 16300 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48684 (request call id 2738) took 1144 ms. Trace:
I20260709 13:21:16.573817 16299 raft_consensus.cc:2468] T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 14.
I20260709 13:21:16.626745 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a878999c0024d2abe517484180290fc" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:16.382172 21390 raft_consensus.cc:2804] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:21:16.577258 16297 raft_consensus.cc:2468] T a378d7f796f94615b3b347119a9880fd P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 16.
I20260709 13:21:16.576741 16217 rpcz_store.cc:276] 0709 13:21:14.390809 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:14.390984 (+   175us) service_pool.cc:224] Handling call
0709 13:21:14.427636 (+ 36652us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:21:14.441088 (+ 13452us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:21:14.441135 (+    47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:21:14.450054 (+  8919us) log.cc:844] Serialized 26 byte log entry
0709 13:21:15.261360 (+811306us) spinlock_profiling.cc:243] Waited 6.17 ms on lock 0x7b18004d16a0. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b693ade1 00007f69b69475b1 00007f69ada54e07 00007f69b68afab9 00007f69ada7169b 00007f69ada6f93f 00007f69ada71c02 00007f69b06f920b 00007f69b06fed23 00007f69b06ff0d1 00007f69b0b1aa0f
0709 13:21:15.377924 (+116564us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:21:15.392793 (+ 14869us) log.cc:844] Serialized 14 byte log entry
0709 13:21:15.393018 (+   225us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:15.393082 (+    64us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:21:16.306959 (+913877us) spinlock_profiling.cc:243] Waited 6.15 ms on lock 0x7b4400022b4c. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69ad8fadb1 00007f69ad998c85 00007f69ad9996a2 00007f69b070b1db 00007f69b0706aaf 00007f69b06fa86e 00007f69b06f9bb6 00007f69b06f97d8 00007f69b0703221 00007f69b07031d9 00007f69b0703169
0709 13:21:16.479485 (+172526us) raft_consensus.cc:1685] finished
0709 13:21:16.479509 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:21:16.479650 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":50347,"prepare.queue_time_us":14406,"prepare.run_cpu_time_us":85,"prepare.run_wall_time_us":2458,"raft.queue_time_us":2342,"raft.run_cpu_time_us":226,"raft.run_wall_time_us":10781,"spinlock_wait_cycles":28339968,"thread_start_us":116358,"threads_started":1,"wal-append.queue_time_us":983261}
I20260709 13:21:16.585999 16816 raft_consensus.cc:2393] T 434bff3082d141838068f323bdb0d1ed P cb82cc7bd6db46d980c75237a1402860 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 14: Already voted for candidate cb82cc7bd6db46d980c75237a1402860 in this term.
I20260709 13:21:16.653656 16825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a378d7f796f94615b3b347119a9880fd" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:21:16.586367 16824 rpcz_store.cc:276] 0709 13:21:15.263406 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:15.263564 (+   158us) service_pool.cc:224] Handling call
0709 13:21:15.282540 (+ 18976us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:21:15.361180 (+ 78640us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:16.499905 (+1138725us) spinlock_profiling.cc:243] Waited 77.7 ms on lock 0x7b5400660b54. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:21:16.500213 (+   308us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":178665728}
I20260709 13:21:16.587901 16147 rpcz_store.cc:276] 0709 13:21:14.695812 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:14.719648 (+ 23836us) service_pool.cc:224] Handling call
0709 13:21:14.734785 (+ 15137us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:21:14.763219 (+ 28434us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:21:14.763262 (+    43us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:21:14.818728 (+ 55466us) log.cc:844] Serialized 52 byte log entry
0709 13:21:15.782734 (+964006us) spinlock_profiling.cc:243] Waited 6.1 ms on lock 0x7b1800072ac0. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b693ade1 00007f69b69475b1 00007f69ada54e07 00007f69b68afab9 00007f69ada7169b 00007f69ada6f93f 00007f69ada71c02 00007f69b06f920b 00007f69b06fed23 00007f69b06ff0d1 00007f69b0b1aa0f
0709 13:21:15.808181 (+ 25447us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:21:15.964188 (+156007us) log.cc:844] Serialized 14 byte log entry
0709 13:21:16.511670 (+547482us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:16.511761 (+    91us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:21:16.511798 (+    37us) raft_consensus.cc:1685] finished
0709 13:21:16.511881 (+    83us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:21:16.512012 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":594648,"prepare.queue_time_us":20911,"prepare.run_cpu_time_us":36,"prepare.run_wall_time_us":38,"raft.queue_time_us":580290,"raft.run_cpu_time_us":334,"raft.run_wall_time_us":37167,"spinlock_wait_cycles":14019328,"thread_start_us":166694,"threads_started":2,"wal-append.queue_time_us":1032829}
I20260709 13:21:16.654237 16825 raft_consensus.cc:3060] T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:21:16.604228 16292 rpcz_store.cc:276] 0709 13:21:15.410539 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:15.416640 (+  6101us) service_pool.cc:224] Handling call
0709 13:21:15.416921 (+   281us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:21:15.854029 (+437108us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:16.518031 (+664002us) spinlock_profiling.cc:243] Waited 426 ms on lock 0x7b5400347e54. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:21:16.518205 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":979738496}
I20260709 13:21:16.525547 16822 raft_consensus.cc:1240] T ee9ed67006144797b1de6ea8c2013a68 P cb82cc7bd6db46d980c75237a1402860 [term 13 LEADER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 12. Current term is 13. Ops: []
I20260709 13:21:16.638154 16115 leader_election.cc:400] T 84f2da98864b4425bf7c0055951fdd4b P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 10 election: Vote denied by peer 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389) with higher term. Message: Invalid argument: T 84f2da98864b4425bf7c0055951fdd4b P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b for earlier term 10. Current term is 11.
I20260709 13:21:16.638741 16821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5d59407eb545b895d8d922e0dd69fd" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:16.655494 16115 leader_election.cc:403] T 84f2da98864b4425bf7c0055951fdd4b P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 10 election: Cancelling election due to peer responding with higher term
I20260709 13:21:16.638831 16144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e9b2bf7011f4ae688a79d7bb5a447b4" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:16.640050 16142 raft_consensus.cc:1240] T e5ad4eb281564936a59f41e387a47156 P 647a343a542d4a139fe106c19095129b [term 14 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 13. Current term is 14. Ops: []
I20260709 13:21:16.596282 21410 raft_consensus.cc:493] T 3884f864a96d428785895e5463e5728f P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:16.632752 16300 rpcz_store.cc:276] 0709 13:21:15.410856 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:15.417468 (+  6612us) service_pool.cc:224] Handling call
0709 13:21:15.417882 (+   414us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:21:15.445785 (+ 27903us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:21:15.445833 (+    48us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:21:15.446549 (+   716us) log.cc:844] Serialized 52 byte log entry
0709 13:21:15.634088 (+187539us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:21:15.641490 (+  7402us) log.cc:844] Serialized 14 byte log entry
0709 13:21:15.641699 (+   209us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:16.555010 (+913311us) spinlock_profiling.cc:243] Waited 15.9 ms on lock 0x7b540010f254. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:21:16.555078 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:21:16.555148 (+    70us) raft_consensus.cc:1685] finished
0709 13:21:16.555298 (+   150us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:21:16.555534 (+   236us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8168,"prepare.queue_time_us":6883,"prepare.run_cpu_time_us":35,"prepare.run_wall_time_us":36,"raft.queue_time_us":32931,"raft.run_cpu_time_us":280,"raft.run_wall_time_us":6671,"spinlock_wait_cycles":36500224,"thread_start_us":162249,"threads_started":1,"wal-append.queue_time_us":305886}
I20260709 13:21:16.649971 16149 raft_consensus.cc:1275] T ee9ed67006144797b1de6ea8c2013a68 P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
W20260709 13:21:16.650918 16790 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:21:16.600064 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 131020us
I20260709 13:21:16.658221 16143 raft_consensus.cc:1240] T 8158bbd17b994d2eb96f560bd0f1b88d P 647a343a542d4a139fe106c19095129b [term 14 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 13. Current term is 14. Ops: [13.9-13.9]
I20260709 13:21:16.658591 16265 leader_election.cc:304] T c0eab25ce4c64e018a041beeda2867db P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:21:16.659408 16143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26354361e9b044cea1c2b86c4882fe92" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:16.660483 16148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26354361e9b044cea1c2b86c4882fe92" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:21:16.661756 16146 raft_consensus.cc:1240] T 18ad7e3af5504571a69d10c00750a01c P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 11. Current term is 12. Ops: []
I20260709 13:21:16.660765 16265 leader_election.cc:304] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:21:16.665426 16790 leader_election.cc:304] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:21:16.610112 21323 raft_consensus.cc:2804] T 7c2095a5f4ad49408dbc7f69c606120d P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:21:16.675231 16142 raft_consensus.cc:1275] T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b [term 14 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:21:16.676157 16298 raft_consensus.cc:1275] T 8d4ab870a05347868956aa8b841b2981 P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 7. (term mismatch)
I20260709 13:21:16.680436 16298 pending_rounds.cc:85] T 8d4ab870a05347868956aa8b841b2981 P 90f3da57796b488c9087a2f523adf045: Aborting all ops after (but not including) 6
I20260709 13:21:16.680728 16298 pending_rounds.cc:107] T 8d4ab870a05347868956aa8b841b2981 P 90f3da57796b488c9087a2f523adf045: Aborting uncommitted NO_OP operation due to leader change: 12.7
I20260709 13:21:16.680990 16298 raft_consensus.cc:2889] T 8d4ab870a05347868956aa8b841b2981 P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:21:16.681794 16816 raft_consensus.cc:1275] T ada5a004ba7c46819051e3ccec71ffdb P cb82cc7bd6db46d980c75237a1402860 [term 14 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:21:16.647827 21097 raft_consensus.cc:493] T 26354361e9b044cea1c2b86c4882fe92 P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:16.646790 21282 raft_consensus.cc:493] T 9c61a39fd4f84424b129c1a6f9ba404d P 647a343a542d4a139fe106c19095129b [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:16.646973 21163 raft_consensus.cc:493] T 02838c86531d4f98bf58d1c22b8baf77 P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:21:16.651893 21153 raft_consensus.cc:2749] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:21:16.700271 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806d7690c2b644959152074bb2cab0a7" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:21:16.702545 16147 raft_consensus.cc:1240] T ecc1e363230043fe9fef390f0b979f20 P 647a343a542d4a139fe106c19095129b [term 14 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 13. Current term is 14. Ops: []
W20260709 13:21:16.704061 16264 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:16.706888 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e4bfb9090c8460eb14fa7e0d818e7f1" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:16.706233 16292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434bff3082d141838068f323bdb0d1ed" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:16.727581 16292 raft_consensus.cc:2468] T 434bff3082d141838068f323bdb0d1ed P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 12.
I20260709 13:21:16.728693 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26354361e9b044cea1c2b86c4882fe92" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:16.712363 16291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434bff3082d141838068f323bdb0d1ed" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:21:16.730118 16291 raft_consensus.cc:3060] T 434bff3082d141838068f323bdb0d1ed P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Advancing to term 14
I20260709 13:21:16.733557 16151 raft_consensus.cc:2468] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 13.
I20260709 13:21:16.674645 21339 raft_consensus.cc:2804] T c0eab25ce4c64e018a041beeda2867db P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 13:21:16.718889 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 113713us
I20260709 13:21:16.718474 16142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87ac0d1f7e8a419f9bcbc04aee0e067e" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
W20260709 13:21:16.721724 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 93344.1us
I20260709 13:21:16.687520 21353 raft_consensus.cc:2804] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:21:16.739640 16116 leader_election.cc:304] T 8d4ab870a05347868956aa8b841b2981 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:21:16.638128 20991 raft_consensus.cc:515] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:16.752204 16142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c2095a5f4ad49408dbc7f69c606120d" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:16.679903 21323 raft_consensus.cc:493] T 7c2095a5f4ad49408dbc7f69c606120d P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:21:16.741812 16816 raft_consensus.cc:1275] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P cb82cc7bd6db46d980c75237a1402860 [term 13 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:21:16.742048 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57a744f5aaf944678a63469cbf514015" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:21:16.737295 16148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434bff3082d141838068f323bdb0d1ed" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:21:16.755976 16300 raft_consensus.cc:3060] T 57a744f5aaf944678a63469cbf514015 P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:21:16.717352 21097 raft_consensus.cc:515] T 26354361e9b044cea1c2b86c4882fe92 P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:16.745503 21250 raft_consensus.cc:493] T d2ab0ac8347641e6955e9bce7d90c1a2 P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:21:16.759267 21097 leader_election.cc:290] T 26354361e9b044cea1c2b86c4882fe92 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:16.759446 21250 raft_consensus.cc:515] T d2ab0ac8347641e6955e9bce7d90c1a2 P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:16.760718 21153 consensus_queue.cc:1048] T 68a6838d82ef4cac83507f906139d90f P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:21:16.761909 16818 raft_consensus.cc:1240] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 [term 16 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 15. Current term is 16. Ops: []
I20260709 13:21:16.762166 21153 raft_consensus.cc:2749] T 8d4ab870a05347868956aa8b841b2981 P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:21:16.698455 20894 consensus_queue.cc:1059] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: INVALID_TERM, Last received: 9.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.055s
I20260709 13:21:16.764979 20991 leader_election.cc:290] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:16.754180 21163 raft_consensus.cc:515] T 02838c86531d4f98bf58d1c22b8baf77 P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:16.768021 21163 leader_election.cc:290] T 02838c86531d4f98bf58d1c22b8baf77 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:16.723553 21282 raft_consensus.cc:515] T 9c61a39fd4f84424b129c1a6f9ba404d P 647a343a542d4a139fe106c19095129b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:16.768932 21403 leader_election.cc:290] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 15 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:16.770625 21282 leader_election.cc:290] T 9c61a39fd4f84424b129c1a6f9ba404d P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
W20260709 13:21:16.748733 16115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 142586us
I20260709 13:21:16.750797 16294 raft_consensus.cc:3060] T 2794af5b99984efebdee564035841cb0 P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:21:16.772192 16143 raft_consensus.cc:2468] T 26354361e9b044cea1c2b86c4882fe92 P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 11.
I20260709 13:21:16.773329 16145 raft_consensus.cc:2393] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 16: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
I20260709 13:21:16.652475 21390 raft_consensus.cc:493] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:21:16.624760 21336 raft_consensus.cc:515] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:16.753232 16291 raft_consensus.cc:2468] T 434bff3082d141838068f323bdb0d1ed P 90f3da57796b488c9087a2f523adf045 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 14.
I20260709 13:21:16.751398 16266 leader_election.cc:304] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:21:16.714825 21388 raft_consensus.cc:2749] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:21:16.656313 21337 consensus_queue.cc:1048] T 746aabf856ab401c9ef5e859371e7b67 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.086s
I20260709 13:21:16.756031 16148 raft_consensus.cc:3060] T 434bff3082d141838068f323bdb0d1ed P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:21:16.712158 16146 raft_consensus.cc:1275] T 2794af5b99984efebdee564035841cb0 P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:21:16.778103 16300 raft_consensus.cc:2468] T 57a744f5aaf944678a63469cbf514015 P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 11.
W20260709 13:21:16.751844 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 69472.2us
I20260709 13:21:16.778466 16151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2794af5b99984efebdee564035841cb0" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:16.824461 16151 raft_consensus.cc:2393] T 2794af5b99984efebdee564035841cb0 P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 11: Already voted for candidate cb82cc7bd6db46d980c75237a1402860 in this term.
I20260709 13:21:16.780241 16817 raft_consensus.cc:2468] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 12.
I20260709 13:21:16.792217 16221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e4bfb9090c8460eb14fa7e0d818e7f1" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:16.674618 21354 consensus_queue.cc:1048] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.064s
W20260709 13:21:16.794050 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 46640.9us
I20260709 13:21:16.858397 16190 leader_election.cc:304] T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:21:16.803341 16296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
W20260709 13:21:16.806562 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 67009.2us
I20260709 13:21:16.738317 21339 raft_consensus.cc:493] T c0eab25ce4c64e018a041beeda2867db P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:16.682879 21410 raft_consensus.cc:515] T 3884f864a96d428785895e5463e5728f P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:16.877691 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0eab25ce4c64e018a041beeda2867db" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:16.878433 16150 raft_consensus.cc:2468] T c0eab25ce4c64e018a041beeda2867db P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 13.
I20260709 13:21:16.805423 16115 leader_election.cc:304] T 434bff3082d141838068f323bdb0d1ed P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:21:16.826578 16221 raft_consensus.cc:2468] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 13.
I20260709 13:21:16.855639 16218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:21:16.856645 16222 raft_consensus.cc:1240] T 19baa7bc367844c288bca8f81439c413 P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 15. Current term is 16. Ops: []
I20260709 13:21:16.720247 21352 consensus_queue.cc:1059] T 8158bbd17b994d2eb96f560bd0f1b88d P 90f3da57796b488c9087a2f523adf045 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: INVALID_TERM, Last received: 11.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.088s
I20260709 13:21:16.773445 21323 raft_consensus.cc:3060] T 7c2095a5f4ad49408dbc7f69c606120d P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:21:16.856235 16117 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 57074.4us
I20260709 13:21:16.793316 21353 raft_consensus.cc:697] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045 [term 14 LEADER]: Becoming Leader. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:21:16.869582 16116 leader_election.cc:304] T 18ad7e3af5504571a69d10c00750a01c P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:21:16.876482 16225 raft_consensus.cc:1275] T 66f75124215f4ca6bef831e174a792d8 P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:21:16.880436 16144 raft_consensus.cc:2468] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 11.
I20260709 13:21:16.773010 21340 raft_consensus.cc:2749] T 84f2da98864b4425bf7c0055951fdd4b P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Leader election lost for term 10. Reason: Vote denied by peer 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389) with higher term. Message: Invalid argument: T 84f2da98864b4425bf7c0055951fdd4b P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b for earlier term 10. Current term is 11.
I20260709 13:21:16.880481 16218 raft_consensus.cc:3060] T ecc1e363230043fe9fef390f0b979f20 P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:21:16.861474 16296 raft_consensus.cc:3060] T ecc1e363230043fe9fef390f0b979f20 P 90f3da57796b488c9087a2f523adf045 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:21:16.803869 21390 raft_consensus.cc:3060] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:21:16.742148 21401 consensus_queue.cc:1048] T ee9ed67006144797b1de6ea8c2013a68 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.121s
W20260709 13:21:16.886109 16115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 80751.4us
I20260709 13:21:16.866195 16217 raft_consensus.cc:1240] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 16. Current term is 17. Ops: []
I20260709 13:21:16.890431 16146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e9b2bf7011f4ae688a79d7bb5a447b4" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:21:16.826896 21366 raft_consensus.cc:493] T af57996a8be84d398f0f023e8d08ef77 P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:16.877085 21339 raft_consensus.cc:3060] T c0eab25ce4c64e018a041beeda2867db P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:21:16.898665 16294 raft_consensus.cc:1275] T 2794af5b99984efebdee564035841cb0 P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:21:16.906172 16293 raft_consensus.cc:1275] T 3884f864a96d428785895e5463e5728f P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
I20260709 13:21:16.896616 16151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a9de347c711477e87c582d003f36925" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:21:16.904474 20894 raft_consensus.cc:2804] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:21:16.915513 20894 raft_consensus.cc:697] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 90f3da57796b488c9087a2f523adf045 [term 12 LEADER]: Becoming Leader. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:21:16.917080 21320 raft_consensus.cc:3055] T 19baa7bc367844c288bca8f81439c413 P cb82cc7bd6db46d980c75237a1402860 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:21:16.917726 21320 raft_consensus.cc:740] T 19baa7bc367844c288bca8f81439c413 P cb82cc7bd6db46d980c75237a1402860 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
W20260709 13:21:16.922886 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 64582.2us
I20260709 13:21:16.924041 16190 leader_election.cc:304] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:21:16.924443 16218 raft_consensus.cc:2468] T ecc1e363230043fe9fef390f0b979f20 P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 15.
I20260709 13:21:16.910878 16148 raft_consensus.cc:2468] T 434bff3082d141838068f323bdb0d1ed P 647a343a542d4a139fe106c19095129b [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 14.
I20260709 13:21:16.913576 16117 leader_election.cc:304] T b87147731a68400e8cce9b879daa7c5d P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860
I20260709 13:21:16.916700 20894 consensus_queue.cc:237] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:16.925882 16115 leader_election.cc:304] T c0eab25ce4c64e018a041beeda2867db P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:21:16.934415 16117 leader_election.cc:304] T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:21:16.936807 21163 raft_consensus.cc:2749] T b87147731a68400e8cce9b879daa7c5d P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:21:16.937656 21163 raft_consensus.cc:2804] T c0eab25ce4c64e018a041beeda2867db P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:21:16.938109 21163 raft_consensus.cc:493] T c0eab25ce4c64e018a041beeda2867db P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:16.938572 21163 raft_consensus.cc:3060] T c0eab25ce4c64e018a041beeda2867db P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:21:16.939671 16816 raft_consensus.cc:1240] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 [term 13 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 12. Current term is 13. Ops: []
I20260709 13:21:16.940181 21097 raft_consensus.cc:2749] T 18ad7e3af5504571a69d10c00750a01c P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:21:16.940933 21097 raft_consensus.cc:2804] T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:21:16.941423 21097 raft_consensus.cc:697] T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b [term 12 LEADER]: Becoming Leader. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:21:16.942142 21250 leader_election.cc:290] T d2ab0ac8347641e6955e9bce7d90c1a2 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:16.942637 21182 raft_consensus.cc:3060] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:21:16.943863 21437 log.cc:927] Time spent T af57996a8be84d398f0f023e8d08ef77 P cb82cc7bd6db46d980c75237a1402860: Append to log took a long time: real 0.058s	user 0.000s	sys 0.001s
I20260709 13:21:16.932881 21154 raft_consensus.cc:2804] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:21:16.883652 21153 consensus_queue.cc:1059] T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: INVALID_TERM, Last received: 12.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.046s
I20260709 13:21:16.947283 21153 raft_consensus.cc:3055] T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:21:16.947692 21153 raft_consensus.cc:740] T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:21:16.933130 21392 raft_consensus.cc:2804] T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:21:16.948650 21392 raft_consensus.cc:493] T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:16.949232 21392 raft_consensus.cc:3060] T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:21:16.948607 21153 consensus_queue.cc:260] T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.6, Last appended by leader: 6, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:16.950088 21153 raft_consensus.cc:3060] T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:21:16.927897 21447 raft_consensus.cc:2764] T 434bff3082d141838068f323bdb0d1ed P 647a343a542d4a139fe106c19095129b [term 14 FOLLOWER]: Leader pre-election decision vote started in defunct term 13: won
I20260709 13:21:16.977732 21336 leader_election.cc:290] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:16.921860 21366 raft_consensus.cc:515] T af57996a8be84d398f0f023e8d08ef77 P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:16.982111 21366 leader_election.cc:290] T af57996a8be84d398f0f023e8d08ef77 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:16.986304 16817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26354361e9b044cea1c2b86c4882fe92" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:16.987453 16817 raft_consensus.cc:2393] T 26354361e9b044cea1c2b86c4882fe92 P cb82cc7bd6db46d980c75237a1402860 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 12: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
I20260709 13:21:16.992898 16790 leader_election.cc:304] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 90f3da57796b488c9087a2f523adf045
W20260709 13:21:16.921396 21414 log.cc:927] Time spent T 3884f864a96d428785895e5463e5728f P cb82cc7bd6db46d980c75237a1402860: Append to log took a long time: real 0.619s	user 0.000s	sys 0.002s
I20260709 13:21:17.044819 16817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a9de347c711477e87c582d003f36925" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:17.045549 16817 raft_consensus.cc:2468] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 16.
I20260709 13:21:16.942364 21097 consensus_queue.cc:237] T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:17.055114 16789 leader_election.cc:304] T 434bff3082d141838068f323bdb0d1ed P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 
W20260709 13:21:17.058064 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 45744.4us
I20260709 13:21:17.079381 21154 raft_consensus.cc:697] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac [term 16 LEADER]: Becoming Leader. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:21:17.080726 21154 consensus_queue.cc:237] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:16.972793 21410 leader_election.cc:290] T 3884f864a96d428785895e5463e5728f P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:16.972884 21444 raft_consensus.cc:3055] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:21:17.091315 16825 raft_consensus.cc:2468] T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 16.
I20260709 13:21:17.108023 16294 raft_consensus.cc:1240] T 66ae176961b74813a58bc15c9c4c6f2a P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 16. Current term is 17. Ops: []
I20260709 13:21:17.048520 21403 raft_consensus.cc:2804] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P cb82cc7bd6db46d980c75237a1402860 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:21:16.928820 21245 consensus_queue.cc:1048] T 829b4ad8c5a54eababeea1181b960d4b P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.182s
I20260709 13:21:17.124362 16300 raft_consensus.cc:1275] T 57a744f5aaf944678a63469cbf514015 P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:21:17.110405 21444 raft_consensus.cc:740] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:21:17.112424 21283 leader_election.cc:290] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:16.919036 21320 consensus_queue.cc:260] T 19baa7bc367844c288bca8f81439c413 P cb82cc7bd6db46d980c75237a1402860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:17.113049 21084 consensus_queue.cc:1048] T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.024s
I20260709 13:21:17.118827 21410 raft_consensus.cc:493] T e4dc6b0541ca431ebc757beac00521f1 P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:21:17.129902 20991 raft_consensus.cc:493] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:21:17.123142 21319 raft_consensus.cc:2804] T 434bff3082d141838068f323bdb0d1ed P cb82cc7bd6db46d980c75237a1402860 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:21:17.126091 21403 raft_consensus.cc:493] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P cb82cc7bd6db46d980c75237a1402860 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:21:17.144941 16115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 219132us
I20260709 13:21:17.156322 16299 raft_consensus.cc:1240] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 15. Current term is 16. Ops: []
I20260709 13:21:17.129760 21282 raft_consensus.cc:493] T 746aabf856ab401c9ef5e859371e7b67 P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:21:17.148480 21440 log.cc:927] Time spent T c9944d5e4b8c4109af9955c4f08c9095 P cb82cc7bd6db46d980c75237a1402860: Append to log took a long time: real 0.107s	user 0.000s	sys 0.000s
I20260709 13:21:17.125290 20902 consensus_queue.cc:1048] T 2794af5b99984efebdee564035841cb0 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.024s
I20260709 13:21:17.143836 21398 consensus_queue.cc:1059] T 19baa7bc367844c288bca8f81439c413 P cb82cc7bd6db46d980c75237a1402860 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.026s
W20260709 13:21:17.136511 16266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 385157us
I20260709 13:21:17.136513 21388 consensus_queue.cc:1048] T 66f75124215f4ca6bef831e174a792d8 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.028s
I20260709 13:21:17.138365 21353 consensus_queue.cc:237] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.9, Last appended by leader: 9, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } }
I20260709 13:21:17.156014 21403 raft_consensus.cc:3060] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P cb82cc7bd6db46d980c75237a1402860 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:21:17.176234 21357 consensus_queue.cc:1059] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 647a343a542d4a139fe106c19095129b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: INVALID_TERM, Last received: 14.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.003s
I20260709 13:21:17.184151 16142 raft_consensus.cc:1240] T ee9ed67006144797b1de6ea8c2013a68 P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 12. Current term is 13. Ops: [12.11-12.11]
I20260709 13:21:17.184695 16151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5d59407eb545b895d8d922e0dd69fd" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:17.186790 21441 raft_consensus.cc:493] T 1df93779209748b9860f6d893dd3de53 P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:17.168452 21319 raft_consensus.cc:697] T 434bff3082d141838068f323bdb0d1ed P cb82cc7bd6db46d980c75237a1402860 [term 14 LEADER]: Becoming Leader. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:21:17.167955 21282 raft_consensus.cc:515] T 746aabf856ab401c9ef5e859371e7b67 P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:17.189517 21320 raft_consensus.cc:3060] T 19baa7bc367844c288bca8f81439c413 P cb82cc7bd6db46d980c75237a1402860 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:21:17.195341 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 371703us
I20260709 13:21:17.151345 21276 consensus_queue.cc:1059] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.026s
I20260709 13:21:17.164134 20991 raft_consensus.cc:515] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:17.202937 16265 leader_election.cc:304] T 66ae176961b74813a58bc15c9c4c6f2a P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: cb82cc7bd6db46d980c75237a1402860
I20260709 13:21:17.206511 16142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02838c86531d4f98bf58d1c22b8baf77" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:17.156201 21410 raft_consensus.cc:515] T e4dc6b0541ca431ebc757beac00521f1 P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:17.215461 16151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d16dfad0c3b540e5b4b86492ddf87208" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:17.217360 16151 raft_consensus.cc:2468] T d16dfad0c3b540e5b4b86492ddf87208 P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 13.
I20260709 13:21:17.192704 21352 raft_consensus.cc:493] T fffab26427e24d398e2695aad40ca134 P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:21:17.219318 16142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c61a39fd4f84424b129c1a6f9ba404d" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:17.177277 21318 consensus_queue.cc:1048] T 2794af5b99984efebdee564035841cb0 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.034s
I20260709 13:21:17.200114 21084 raft_consensus.cc:493] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:21:17.220757 16142 raft_consensus.cc:2468] T 9c61a39fd4f84424b129c1a6f9ba404d P 647a343a542d4a139fe106c19095129b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 14.
I20260709 13:21:17.198495 21339 raft_consensus.cc:515] T c0eab25ce4c64e018a041beeda2867db P 90f3da57796b488c9087a2f523adf045 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:17.240762 21339 leader_election.cc:290] T c0eab25ce4c64e018a041beeda2867db P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 14 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:21:17.214955 21447 raft_consensus.cc:3055] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 647a343a542d4a139fe106c19095129b [term 14 LEADER]: Stepping down as leader of term 14
W20260709 13:21:17.236840 21338 consensus_meta.cc:287] T 8a878999c0024d2abe517484180290fc P 90f3da57796b488c9087a2f523adf045: Time spent flushing consensus metadata: real 0.637s	user 0.011s	sys 0.000s
I20260709 13:21:17.225821 21405 leader_election.cc:290] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 16 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:21:17.250573 16151 raft_consensus.cc:3060] T d16dfad0c3b540e5b4b86492ddf87208 P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:21:17.220685 21401 raft_consensus.cc:3055] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:21:17.260952 16296 raft_consensus.cc:2468] T ecc1e363230043fe9fef390f0b979f20 P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 15.
I20260709 13:21:17.203495 21441 raft_consensus.cc:515] T 1df93779209748b9860f6d893dd3de53 P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:17.190438 21444 consensus_queue.cc:260] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:17.244673 21447 raft_consensus.cc:740] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 647a343a542d4a139fe106c19095129b [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:21:17.226250 21352 raft_consensus.cc:515] T fffab26427e24d398e2695aad40ca134 P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:21:17.288929 16791 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:17.293071 16820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af57996a8be84d398f0f023e8d08ef77" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:17.293543 16817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e4bfb9090c8460eb14fa7e0d818e7f1" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 14 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
W20260709 13:21:17.295343 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 425803us
W20260709 13:21:17.270223 21445 log.cc:927] Time spent T d2ab0ac8347641e6955e9bce7d90c1a2 P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 0.078s	user 0.000s	sys 0.001s
I20260709 13:21:17.296868 16151 raft_consensus.cc:1275] T d16dfad0c3b540e5b4b86492ddf87208 P 647a343a542d4a139fe106c19095129b [term 14 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:21:17.237264 21084 raft_consensus.cc:515] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:17.265147 21401 raft_consensus.cc:740] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
W20260709 13:21:17.303977 16117 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 372951us
W20260709 13:21:17.319674 16266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 142255us
I20260709 13:21:17.264878 21163 raft_consensus.cc:515] T c0eab25ce4c64e018a041beeda2867db P 647a343a542d4a139fe106c19095129b [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:17.311084 16823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ab0ac8347641e6955e9bce7d90c1a2" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:17.299446 21388 raft_consensus.cc:493] T 829b4ad8c5a54eababeea1181b960d4b P cb82cc7bd6db46d980c75237a1402860 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:17.266619 21338 raft_consensus.cc:515] T 8a878999c0024d2abe517484180290fc P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:21:17.337314 16789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 194501us
I20260709 13:21:17.308140 16116 leader_election.cc:304] T fffab26427e24d398e2695aad40ca134 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:21:17.273950 21451 consensus_queue.cc:1048] T ada5a004ba7c46819051e3ccec71ffdb P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.064s
I20260709 13:21:17.341578 16223 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33050 (request call id 888) took 1067 ms. Trace:
I20260709 13:21:17.303014 21366 consensus_queue.cc:1059] T ee9ed67006144797b1de6ea8c2013a68 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.032s
I20260709 13:21:17.268020 21319 consensus_queue.cc:237] T 434bff3082d141838068f323bdb0d1ed P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:21:17.352903 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 150032us
I20260709 13:21:17.334618 20991 leader_election.cc:290] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:17.344766 16223 rpcz_store.cc:276] 0709 13:21:16.274386 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:16.274558 (+   172us) service_pool.cc:224] Handling call
0709 13:21:16.274927 (+   369us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:21:16.598267 (+323340us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:21:16.598297 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:21:16.599634 (+  1337us) log.cc:844] Serialized 26 byte log entry
0709 13:21:16.745295 (+145661us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:21:16.745356 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:16.745952 (+   596us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:21:17.341347 (+595395us) raft_consensus.cc:1685] finished
0709 13:21:17.341368 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:21:17.341516 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":38550,"raft.run_cpu_time_us":66,"raft.run_wall_time_us":66,"thread_start_us":390612,"threads_started":2,"wal-append.queue_time_us":489194}
I20260709 13:21:17.357520 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e4bfb9090c8460eb14fa7e0d818e7f1" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 14 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:17.400592 16823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e9b2bf7011f4ae688a79d7bb5a447b4" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:17.401708 16823 raft_consensus.cc:2393] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 12: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
I20260709 13:21:17.403438 16295 raft_consensus.cc:1275] T d16dfad0c3b540e5b4b86492ddf87208 P 90f3da57796b488c9087a2f523adf045 [term 14 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:21:17.401265 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ab0ac8347641e6955e9bce7d90c1a2" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:17.332633 21390 raft_consensus.cc:515] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:17.331151 21323 raft_consensus.cc:515] T 7c2095a5f4ad49408dbc7f69c606120d P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:17.419697 16116 leader_election.cc:304] T 8cc959fb7f704fba9009c0595467d074 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:21:17.358273 21357 raft_consensus.cc:2749] T fffab26427e24d398e2695aad40ca134 P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:21:17.425621 21282 leader_election.cc:290] T 746aabf856ab401c9ef5e859371e7b67 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:17.335373 21388 raft_consensus.cc:515] T 829b4ad8c5a54eababeea1181b960d4b P cb82cc7bd6db46d980c75237a1402860 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:17.429090 16147 raft_consensus.cc:3060] T 26354361e9b044cea1c2b86c4882fe92 P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:21:17.420758 21454 log.cc:927] Time spent T 84f2da98864b4425bf7c0055951fdd4b P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 0.162s	user 0.000s	sys 0.001s
W20260709 13:21:17.423969 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 65726.9us
I20260709 13:21:17.349213 21455 consensus_queue.cc:1048] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.091s
I20260709 13:21:17.444557 21323 leader_election.cc:290] T 7c2095a5f4ad49408dbc7f69c606120d P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 10 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:21:17.445545 21323 consensus_queue.cc:1059] T e5ad4eb281564936a59f41e387a47156 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: INVALID_TERM, Last received: 13.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:21:17.362941 21456 raft_consensus.cc:2804] T 66ae176961b74813a58bc15c9c4c6f2a P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:21:17.349516 21405 consensus_queue.cc:1048] T 57a744f5aaf944678a63469cbf514015 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.080s
I20260709 13:21:17.427651 21366 consensus_queue.cc:1243] T ee9ed67006144797b1de6ea8c2013a68 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Peer cb82cc7bd6db46d980c75237a1402860 log is divergent from this leader: its last log entry 13.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 13:21:17.334640 21447 consensus_queue.cc:260] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:17.456300 16116 leader_election.cc:400] T d16dfad0c3b540e5b4b86492ddf87208 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 13 election: Vote denied by peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943) with higher term. Message: Invalid argument: T d16dfad0c3b540e5b4b86492ddf87208 P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b for earlier term 13. Current term is 14.
I20260709 13:21:17.468273 16116 leader_election.cc:403] T d16dfad0c3b540e5b4b86492ddf87208 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 13 election: Cancelling election due to peer responding with higher term
I20260709 13:21:17.437815 16818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3884f864a96d428785895e5463e5728f" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:17.433768 21425 raft_consensus.cc:493] T e4dc6b0541ca431ebc757beac00521f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:17.351555 21401 consensus_queue.cc:260] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.12, Last appended by leader: 12, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:17.450726 21325 raft_consensus.cc:493] T fffab26427e24d398e2695aad40ca134 P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:21:17.456373 16264 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:17.493405 16116 leader_election.cc:304] T 8158bbd17b994d2eb96f560bd0f1b88d P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 90f3da57796b488c9087a2f523adf045
I20260709 13:21:17.495146 16116 leader_election.cc:304] T 08c74490c40243a88e62d1760b5cd814 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 90f3da57796b488c9087a2f523adf045
I20260709 13:21:17.496450 21468 raft_consensus.cc:3055] T e5ad4eb281564936a59f41e387a47156 P 90f3da57796b488c9087a2f523adf045 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:21:17.496954 21468 raft_consensus.cc:740] T e5ad4eb281564936a59f41e387a47156 P 90f3da57796b488c9087a2f523adf045 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:21:17.459077 21410 leader_election.cc:290] T e4dc6b0541ca431ebc757beac00521f1 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:21:17.502274 16116 leader_election.cc:304] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: cb82cc7bd6db46d980c75237a1402860
I20260709 13:21:17.452395 21392 raft_consensus.cc:515] T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:21:17.522931 16790 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:17.469890 21097 consensus_queue.cc:1048] T 829b4ad8c5a54eababeea1181b960d4b P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.029s
I20260709 13:21:17.524150 21442 raft_consensus.cc:2804] T 08c74490c40243a88e62d1760b5cd814 P 647a343a542d4a139fe106c19095129b [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:21:17.544318 21388 leader_election.cc:290] T 829b4ad8c5a54eababeea1181b960d4b P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:21:17.458303 21357 consensus_queue.cc:1048] T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.048s
W20260709 13:21:17.524504 21449 log.cc:927] Time spent T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860: Append to log took a long time: real 0.250s	user 0.001s	sys 0.000s
I20260709 13:21:17.499215 21466 raft_consensus.cc:2749] T d16dfad0c3b540e5b4b86492ddf87208 P 647a343a542d4a139fe106c19095129b [term 14 FOLLOWER]: Leader election lost for term 13. Reason: Vote denied by peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943) with higher term. Message: Invalid argument: T d16dfad0c3b540e5b4b86492ddf87208 P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b for earlier term 13. Current term is 14.
I20260709 13:21:17.547957 16116 leader_election.cc:304] T e5ad4eb281564936a59f41e387a47156 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 90f3da57796b488c9087a2f523adf045
I20260709 13:21:17.489370 21456 raft_consensus.cc:697] T 66ae176961b74813a58bc15c9c4c6f2a P 90f3da57796b488c9087a2f523adf045 [term 17 LEADER]: Becoming Leader. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:21:17.503727 21282 raft_consensus.cc:2804] T 8158bbd17b994d2eb96f560bd0f1b88d P 647a343a542d4a139fe106c19095129b [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:21:17.553351 21447 raft_consensus.cc:3060] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 647a343a542d4a139fe106c19095129b [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:21:17.545616 21457 raft_consensus.cc:2804] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:21:17.493407 21425 raft_consensus.cc:515] T e4dc6b0541ca431ebc757beac00521f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:17.563961 21453 raft_consensus.cc:2804] T e5ad4eb281564936a59f41e387a47156 P 647a343a542d4a139fe106c19095129b [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:21:17.570159 21282 raft_consensus.cc:697] T 8158bbd17b994d2eb96f560bd0f1b88d P 647a343a542d4a139fe106c19095129b [term 14 LEADER]: Becoming Leader. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:21:17.502488 21325 raft_consensus.cc:515] T fffab26427e24d398e2695aad40ca134 P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:17.570787 21084 leader_election.cc:290] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:17.571959 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02838c86531d4f98bf58d1c22b8baf77" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:17.554162 21468 consensus_queue.cc:260] T e5ad4eb281564936a59f41e387a47156 P 90f3da57796b488c9087a2f523adf045 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:21:17.579725 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 118750us
I20260709 13:21:17.560040 21442 raft_consensus.cc:697] T 08c74490c40243a88e62d1760b5cd814 P 647a343a542d4a139fe106c19095129b [term 14 LEADER]: Becoming Leader. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:21:17.560880 21441 leader_election.cc:290] T 1df93779209748b9860f6d893dd3de53 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:17.602909 16190 leader_election.cc:304] T d2ab0ac8347641e6955e9bce7d90c1a2 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860
W20260709 13:21:17.607682 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 189364us
I20260709 13:21:17.577051 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c61a39fd4f84424b129c1a6f9ba404d" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:17.584860 16221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0eab25ce4c64e018a041beeda2867db" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:21:17.613180 16221 raft_consensus.cc:3060] T c0eab25ce4c64e018a041beeda2867db P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:21:17.593250 16291 raft_consensus.cc:1240] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 14. Current term is 15. Ops: []
I20260709 13:21:17.616398 16291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e9b2bf7011f4ae688a79d7bb5a447b4" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:17.595619 16298 raft_consensus.cc:1240] T 8a878999c0024d2abe517484180290fc P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 12. Current term is 13. Ops: []
I20260709 13:21:17.596567 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26354361e9b044cea1c2b86c4882fe92" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
W20260709 13:21:17.597275 16790 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:17.598603 16292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a9de347c711477e87c582d003f36925" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:17.622139 16292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c61a39fd4f84424b129c1a6f9ba404d" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:17.623015 16292 raft_consensus.cc:2468] T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 14.
I20260709 13:21:17.627440 16224 raft_consensus.cc:2393] T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 15: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
I20260709 13:21:17.639712 16222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1df93779209748b9860f6d893dd3de53" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:17.641559 16291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02838c86531d4f98bf58d1c22b8baf77" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:17.575920 21352 leader_election.cc:290] T fffab26427e24d398e2695aad40ca134 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:21:17.645653 16292 raft_consensus.cc:1240] T 7c2095a5f4ad49408dbc7f69c606120d P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 9. Current term is 10. Ops: []
I20260709 13:21:17.646776 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746aabf856ab401c9ef5e859371e7b67" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 14 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:17.647652 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62fe3346cd3c42c4a55e14af9073bca4" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:17.578011 21163 leader_election.cc:290] T c0eab25ce4c64e018a041beeda2867db P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 14 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
W20260709 13:21:17.638355 21449 log.cc:927] Time spent T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860: Append to log took a long time: real 0.064s	user 0.001s	sys 0.000s
I20260709 13:21:17.587363 21466 raft_consensus.cc:493] T 2794af5b99984efebdee564035841cb0 P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:17.586844 21338 leader_election.cc:290] T 8a878999c0024d2abe517484180290fc P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 13 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:21:17.640064 21469 raft_consensus.cc:493] T 31969392ccdb43b491911a319e8bc7e1 P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:21:17.607285 21457 raft_consensus.cc:697] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [term 16 LEADER]: Becoming Leader. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:21:17.659448 21468 raft_consensus.cc:3060] T e5ad4eb281564936a59f41e387a47156 P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:21:17.671579 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 87071.9us
I20260709 13:21:17.686358 21282 consensus_queue.cc:237] T 8158bbd17b994d2eb96f560bd0f1b88d P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 9, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:17.640326 21453 raft_consensus.cc:697] T e5ad4eb281564936a59f41e387a47156 P 647a343a542d4a139fe106c19095129b [term 14 LEADER]: Becoming Leader. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:21:17.672483 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8638ee363273426db1393dd81e5c7b72" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:17.658625 21391 raft_consensus.cc:2749] T d2ab0ac8347641e6955e9bce7d90c1a2 P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:21:17.679737 21456 consensus_queue.cc:237] T 66ae176961b74813a58bc15c9c4c6f2a P 90f3da57796b488c9087a2f523adf045 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:17.637285 21154 consensus_queue.cc:1048] T d16dfad0c3b540e5b4b86492ddf87208 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.043s
I20260709 13:21:17.699738 21163 raft_consensus.cc:493] T af57996a8be84d398f0f023e8d08ef77 P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
W20260709 13:21:17.703398 21467 log.cc:927] Time spent T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 0.128s	user 0.001s	sys 0.000s
I20260709 13:21:17.703214 21163 raft_consensus.cc:515] T af57996a8be84d398f0f023e8d08ef77 P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:17.704942 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1df93779209748b9860f6d893dd3de53" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:17.705439 21163 leader_election.cc:290] T af57996a8be84d398f0f023e8d08ef77 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:17.706290 16819 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49906 (request call id 1041) took 1177 ms. Trace:
I20260709 13:21:17.707104 21425 leader_election.cc:290] T e4dc6b0541ca431ebc757beac00521f1 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:17.704998 21392 leader_election.cc:290] T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 15 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:17.691754 21390 leader_election.cc:290] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 15 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:21:17.706482 21457 consensus_queue.cc:237] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.9, Last appended by leader: 9, Current term: 16, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:17.716878 16292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c61a39fd4f84424b129c1a6f9ba404d" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:21:17.706712 16819 rpcz_store.cc:276] 0709 13:21:16.529319 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:16.637672 (+108353us) service_pool.cc:224] Handling call
0709 13:21:16.647467 (+  9795us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:21:16.664671 (+ 17204us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:21:16.682907 (+ 18236us) log.cc:844] Serialized 14 byte log entry
0709 13:21:17.335367 (+652460us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:21:17.347608 (+ 12241us) log.cc:844] Serialized 52 byte log entry
0709 13:21:17.347823 (+   215us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:21:17.550482 (+202659us) spinlock_profiling.cc:243] Waited 7.04 ms on lock 0x7b18004c9600. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b693ade1 00007f69b69475b1 00007f69ada72522 00007f69ada747c1 00007f69ada74779 00007f69ada74709 00007f69ada746d1 00007f69ada739cd 000000000038f354 000000000038f199 00007f69ada5583a
0709 13:21:17.630277 (+ 79795us) log.cc:844] Serialized 14 byte log entry
0709 13:21:17.631667 (+  1390us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:17.631773 (+   106us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:21:17.706074 (+ 74301us) raft_consensus.cc:1685] finished
0709 13:21:17.706098 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:21:17.706230 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":16744,"prepare.queue_time_us":18036,"prepare.run_cpu_time_us":43,"prepare.run_wall_time_us":45,"raft.queue_time_us":41890,"raft.run_cpu_time_us":496,"raft.run_wall_time_us":44553,"spinlock_wait_cycles":16192256,"thread_start_us":877947,"threads_started":3,"wal-append.queue_time_us":531491}
W20260709 13:21:17.722081 16791 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:17.722645 16292 raft_consensus.cc:3060] T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:21:17.717792 16300 raft_consensus.cc:1275] T a378d7f796f94615b3b347119a9880fd P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 13:21:17.718628 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0eab25ce4c64e018a041beeda2867db" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:21:17.672051 16221 raft_consensus.cc:2468] T c0eab25ce4c64e018a041beeda2867db P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 14.
I20260709 13:21:17.728324 16297 raft_consensus.cc:2393] T c0eab25ce4c64e018a041beeda2867db P 90f3da57796b488c9087a2f523adf045 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 14: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
I20260709 13:21:17.729844 16291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4dc6b0541ca431ebc757beac00521f1" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:17.738345 16222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af57996a8be84d398f0f023e8d08ef77" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:17.763435 16292 raft_consensus.cc:2468] T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 15.
I20260709 13:21:17.679024 21466 raft_consensus.cc:515] T 2794af5b99984efebdee564035841cb0 P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:17.769537 21466 leader_election.cc:290] T 2794af5b99984efebdee564035841cb0 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:17.711803 21442 consensus_queue.cc:237] T 08c74490c40243a88e62d1760b5cd814 P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:17.738291 21455 raft_consensus.cc:493] T c9944d5e4b8c4109af9955c4f08c9095 P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:21:17.683213 21469 raft_consensus.cc:515] T 31969392ccdb43b491911a319e8bc7e1 P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:17.815085 21450 raft_consensus.cc:3055] T 8158bbd17b994d2eb96f560bd0f1b88d P 90f3da57796b488c9087a2f523adf045 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:21:17.815754 21450 raft_consensus.cc:740] T 8158bbd17b994d2eb96f560bd0f1b88d P 90f3da57796b488c9087a2f523adf045 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:21:17.816834 21450 consensus_queue.cc:260] T 8158bbd17b994d2eb96f560bd0f1b88d P 90f3da57796b488c9087a2f523adf045 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:17.818274 21450 raft_consensus.cc:3060] T 8158bbd17b994d2eb96f560bd0f1b88d P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:21:17.827283 21459 consensus_queue.cc:1059] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:21:17.828377 21153 raft_consensus.cc:3055] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:21:17.828238 21097 consensus_queue.cc:1059] T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:21:17.828845 21153 raft_consensus.cc:740] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:21:17.829291 21097 raft_consensus.cc:3055] T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:21:17.829676 21153 consensus_queue.cc:260] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 12.12, Last appended by leader: 12, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:17.829722 21097 raft_consensus.cc:740] T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:21:17.832293 21097 consensus_queue.cc:260] T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.11, Last appended by leader: 11, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:17.833461 21097 raft_consensus.cc:3060] T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:21:17.833561 16221 raft_consensus.cc:1240] T 18ad7e3af5504571a69d10c00750a01c P 1ca2725d290e47d68b1a4b609c9816ac [term 12 LEADER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 11. Current term is 12. Ops: []
I20260709 13:21:17.835088 16819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8638ee363273426db1393dd81e5c7b72" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:17.835839 16219 raft_consensus.cc:1275] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:21:17.838174 21391 raft_consensus.cc:493] T 19baa7bc367844c288bca8f81439c413 P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:17.838685 21391 raft_consensus.cc:515] T 19baa7bc367844c288bca8f81439c413 P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:17.779402 20991 raft_consensus.cc:2749] T 8cc959fb7f704fba9009c0595467d074 P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:21:17.750288 21357 consensus_queue.cc:1059] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.055s
I20260709 13:21:17.843222 16117 leader_election.cc:304] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:21:17.830926 21153 raft_consensus.cc:3060] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:21:17.856642 16819 raft_consensus.cc:1275] T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860 [term 16 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
W20260709 13:21:17.912367 16115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 172449us
I20260709 13:21:17.915799 16115 leader_election.cc:304] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860
I20260709 13:21:17.916682 16221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0eab25ce4c64e018a041beeda2867db" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:21:17.917945 16221 raft_consensus.cc:2393] T c0eab25ce4c64e018a041beeda2867db P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 14: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
I20260709 13:21:17.919977 16221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c2095a5f4ad49408dbc7f69c606120d" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:21:17.920608 16221 raft_consensus.cc:3060] T 7c2095a5f4ad49408dbc7f69c606120d P 1ca2725d290e47d68b1a4b609c9816ac [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:21:17.948050 16790 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:17.906932 21370 raft_consensus.cc:493] T e5ad4eb281564936a59f41e387a47156 P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:17.909423 21459 raft_consensus.cc:2804] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 13:21:18.006269 16264 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:17.905530 21325 leader_election.cc:290] T fffab26427e24d398e2695aad40ca134 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:17.905809 21198 raft_consensus.cc:493] T c9944d5e4b8c4109af9955c4f08c9095 P cb82cc7bd6db46d980c75237a1402860 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:21:17.906644 21323 raft_consensus.cc:493] T 5a96373bb0e14dce8a266b2fc6c58507 P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:21:17.803448 21455 raft_consensus.cc:515] T c9944d5e4b8c4109af9955c4f08c9095 P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:17.807569 21453 consensus_queue.cc:237] T e5ad4eb281564936a59f41e387a47156 P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:17.972018 21154 consensus_queue.cc:1048] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.243s
I20260709 13:21:17.973898 21338 raft_consensus.cc:493] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:21:18.670399 16192 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.717s	user 0.005s	sys 0.005s
W20260709 13:21:18.737865 16192 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.784s	user 0.005s	sys 0.007s
W20260709 13:21:18.738358 16192 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 784675us
I20260709 13:21:18.738763 16218 raft_consensus.cc:1275] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
W20260709 13:21:18.743695 16219 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.792s	user 0.007s	sys 0.003s
W20260709 13:21:18.744087 16219 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.793s	user 0.007s	sys 0.003s
I20260709 13:21:18.744113 16116 leader_election.cc:304] T c0eab25ce4c64e018a041beeda2867db P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
W20260709 13:21:18.732300 16264 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:21:18.079414 16790 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:18.191499 16221 raft_consensus.cc:2468] T 7c2095a5f4ad49408dbc7f69c606120d P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 10.
W20260709 13:21:17.980805 21182 consensus_meta.cc:287] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac: Time spent flushing consensus metadata: real 1.038s	user 0.010s	sys 0.005s
I20260709 13:21:18.732898 16818 raft_consensus.cc:1217] T ee9ed67006144797b1de6ea8c2013a68 P cb82cc7bd6db46d980c75237a1402860 [term 13 LEADER]: Deduplicated request from leader. Original: 11.10->[12.11-12.11]   Dedup: 12.11->[]
I20260709 13:21:18.739353 16292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fffab26427e24d398e2695aad40ca134" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:18.768455 16292 raft_consensus.cc:2468] T fffab26427e24d398e2695aad40ca134 P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 12.
W20260709 13:21:18.156636 16791 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:21:18.133597 16115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 220139us
I20260709 13:21:18.446388 16223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4dc6b0541ca431ebc757beac00521f1" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:18.769811 16223 raft_consensus.cc:2393] T e4dc6b0541ca431ebc757beac00521f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 13: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
W20260709 13:21:18.770056 16264 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:21:18.770470 16264 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:18.770071 16295 raft_consensus.cc:3055] T 18ad7e3af5504571a69d10c00750a01c P 90f3da57796b488c9087a2f523adf045 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:21:18.770843 16295 raft_consensus.cc:740] T 18ad7e3af5504571a69d10c00750a01c P 90f3da57796b488c9087a2f523adf045 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:21:18.771303 16115 leader_election.cc:304] T 9c61a39fd4f84424b129c1a6f9ba404d P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac
W20260709 13:21:18.471133 21457 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.704s	user 0.010s	sys 0.007s
I20260709 13:21:18.772617 16818 raft_consensus.cc:1240] T ee9ed67006144797b1de6ea8c2013a68 P cb82cc7bd6db46d980c75237a1402860 [term 13 LEADER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 12. Current term is 13. Ops: [12.11-12.11]
I20260709 13:21:18.775439 16217 raft_consensus.cc:2468] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 18.
W20260709 13:21:18.665802 21468 consensus_meta.cc:287] T e5ad4eb281564936a59f41e387a47156 P 90f3da57796b488c9087a2f523adf045: Time spent flushing consensus metadata: real 0.971s	user 0.011s	sys 0.001s
W20260709 13:21:18.471133 21447 consensus_meta.cc:287] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 647a343a542d4a139fe106c19095129b: Time spent flushing consensus metadata: real 0.892s	user 0.006s	sys 0.005s
I20260709 13:21:18.773358 16295 consensus_queue.cc:260] T 18ad7e3af5504571a69d10c00750a01c P 90f3da57796b488c9087a2f523adf045 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } }
I20260709 13:21:18.193863 21357 raft_consensus.cc:3060] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:21:18.670185 21442 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.538s	user 0.003s	sys 0.006s
W20260709 13:21:18.664916 21087 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.824s	user 0.006s	sys 0.010s
W20260709 13:21:18.129750 21474 log.cc:927] Time spent T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860: Append to log took a long time: real 0.116s	user 0.000s	sys 0.000s
I20260709 13:21:18.778891 16222 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57604 (request call id 2815) took 1034 ms. Trace:
W20260709 13:21:18.240687 21403 consensus_meta.cc:287] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P cb82cc7bd6db46d980c75237a1402860: Time spent flushing consensus metadata: real 1.038s	user 0.013s	sys 0.000s
I20260709 13:21:18.751324 21323 raft_consensus.cc:515] T 5a96373bb0e14dce8a266b2fc6c58507 P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:18.678606 21354 raft_consensus.cc:493] T 23f82224f4564fea93572e8206bad1fe P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
W20260709 13:21:18.789613 21087 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.949s	user 0.009s	sys 0.011s
I20260709 13:21:18.794250 21354 raft_consensus.cc:515] T 23f82224f4564fea93572e8206bad1fe P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:18.791780 21391 leader_election.cc:290] T 19baa7bc367844c288bca8f81439c413 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:18.787436 21497 raft_consensus.cc:493] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:18.797010 21354 leader_election.cc:290] T 23f82224f4564fea93572e8206bad1fe P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:18.796376 21497 raft_consensus.cc:515] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:18.225915 21469 leader_election.cc:290] T 31969392ccdb43b491911a319e8bc7e1 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:18.176304 21390 raft_consensus.cc:493] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:21:18.799289 21390 raft_consensus.cc:515] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:18.799981 21497 leader_election.cc:290] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:18.801419 21390 leader_election.cc:290] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:21:18.127754 21405 raft_consensus.cc:493] T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:21:18.802263 21405 raft_consensus.cc:515] T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:18.804768 21405 leader_election.cc:290] T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:21:18.805392 21405 raft_consensus.cc:493] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:21:18.805953 21405 raft_consensus.cc:515] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:18.734635 21459 raft_consensus.cc:493] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:18.807101 21459 raft_consensus.cc:3060] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:21:18.787506 21442 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.656s	user 0.003s	sys 0.007s
I20260709 13:21:18.793732 21403 raft_consensus.cc:515] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P cb82cc7bd6db46d980c75237a1402860 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:18.816218 21403 leader_election.cc:290] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 14 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:18.816763 16817 raft_consensus.cc:2393] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P cb82cc7bd6db46d980c75237a1402860 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 14: Already voted for candidate cb82cc7bd6db46d980c75237a1402860 in this term.
I20260709 13:21:18.816862 21403 raft_consensus.cc:493] T 3a7ca61592e848ea9b0fb81a2ce33cdc P cb82cc7bd6db46d980c75237a1402860 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:21:18.817718 21403 raft_consensus.cc:515] T 3a7ca61592e848ea9b0fb81a2ce33cdc P cb82cc7bd6db46d980c75237a1402860 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:18.827575 21459 raft_consensus.cc:515] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:18.832412 16295 raft_consensus.cc:3060] T 18ad7e3af5504571a69d10c00750a01c P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:21:18.855950 16220 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57604 (request call id 2813) took 1271 ms. Trace:
I20260709 13:21:18.856504 16220 rpcz_store.cc:276] 0709 13:21:17.584824 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:17.584945 (+   121us) service_pool.cc:224] Handling call
0709 13:21:17.637241 (+ 52296us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:21:17.637727 (+   486us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:21:17.637768 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:21:17.657491 (+ 19723us) log.cc:844] Serialized 26 byte log entry
0709 13:21:17.758398 (+100907us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:21:17.758439 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:17.758505 (+    66us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:21:18.855669 (+1097164us) raft_consensus.cc:1685] finished
0709 13:21:18.855702 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:21:18.855880 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":13563,"raft.queue_time_us":1125,"raft.run_cpu_time_us":126,"raft.run_wall_time_us":127,"thread_start_us":38528,"threads_started":1,"wal-append.queue_time_us":1183763}
I20260709 13:21:18.783138 21325 raft_consensus.cc:493] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:18.859615 21325 raft_consensus.cc:515] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:18.862334 21325 leader_election.cc:290] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:18.830695 21323 leader_election.cc:290] T 5a96373bb0e14dce8a266b2fc6c58507 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:21:18.779994 21480 raft_consensus.cc:493] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:18.730217 21370 raft_consensus.cc:515] T e5ad4eb281564936a59f41e387a47156 P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:18.844061 21163 raft_consensus.cc:493] T 5a959dbecc2648eb85d1cc550f5a3245 P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:21:18.764300 21198 raft_consensus.cc:515] T c9944d5e4b8c4109af9955c4f08c9095 P cb82cc7bd6db46d980c75237a1402860 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:18.771821 21338 raft_consensus.cc:515] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:18.838186 21447 raft_consensus.cc:493] T 23f82224f4564fea93572e8206bad1fe P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:21:18.877209 16295 raft_consensus.cc:1275] T 18ad7e3af5504571a69d10c00750a01c P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 13:21:18.847306 21442 raft_consensus.cc:2804] T c0eab25ce4c64e018a041beeda2867db P 647a343a542d4a139fe106c19095129b [term 14 FOLLOWER]: Leader election won for term 14
W20260709 13:21:18.871297 21097 consensus_meta.cc:287] T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b: Time spent flushing consensus metadata: real 1.037s	user 0.013s	sys 0.001s
I20260709 13:21:18.871793 21480 raft_consensus.cc:515] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:21:18.888760 21153 consensus_meta.cc:287] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b: Time spent flushing consensus metadata: real 0.938s	user 0.009s	sys 0.002s
W20260709 13:21:18.896178 21458 log.cc:927] Time spent T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 1.448s	user 0.010s	sys 0.011s
I20260709 13:21:18.896669 21370 leader_election.cc:290] T e5ad4eb281564936a59f41e387a47156 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:18.900398 21444 raft_consensus.cc:3060] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:21:18.902467 21198 leader_election.cc:290] T c9944d5e4b8c4109af9955c4f08c9095 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:18.873857 21163 raft_consensus.cc:515] T 5a959dbecc2648eb85d1cc550f5a3245 P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:18.909870 21163 leader_election.cc:290] T 5a959dbecc2648eb85d1cc550f5a3245 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:18.872287 21489 raft_consensus.cc:2804] T 9c61a39fd4f84424b129c1a6f9ba404d P 647a343a542d4a139fe106c19095129b [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:21:18.923996 21442 raft_consensus.cc:697] T c0eab25ce4c64e018a041beeda2867db P 647a343a542d4a139fe106c19095129b [term 14 LEADER]: Becoming Leader. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
W20260709 13:21:18.900549 21450 consensus_meta.cc:287] T 8158bbd17b994d2eb96f560bd0f1b88d P 90f3da57796b488c9087a2f523adf045: Time spent flushing consensus metadata: real 1.082s	user 0.010s	sys 0.004s
W20260709 13:21:18.938984 21494 log.cc:927] Time spent T 8158bbd17b994d2eb96f560bd0f1b88d P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
I20260709 13:21:18.943248 21498 raft_consensus.cc:493] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:21:18.944183 21403 leader_election.cc:290] T 3a7ca61592e848ea9b0fb81a2ce33cdc P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:18.915220 21198 raft_consensus.cc:493] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
W20260709 13:21:18.946960 16191 connection.cc:256] RPC call timeout handler was delayed by -1.4051113410000653s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15780000000000002s for a response
I20260709 13:21:18.947414 21325 raft_consensus.cc:493] T 66f75124215f4ca6bef831e174a792d8 P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:18.947826 21497 raft_consensus.cc:493] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:18.945658 21338 leader_election.cc:290] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
W20260709 13:21:18.960397 16191 connection.cc:256] RPC call timeout handler was delayed by -1.151393545000019s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 1.5s for a response
W20260709 13:21:18.960177 16265 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:18.957782 21459 leader_election.cc:290] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 17 election: Requested vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:18.963989 21370 raft_consensus.cc:493] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
W20260709 13:21:18.965534 16265 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:21:18.965993 16265 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:21:18.960754 16263 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:21:18.967784 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 223836us
I20260709 13:21:18.883106 21447 raft_consensus.cc:515] T 23f82224f4564fea93572e8206bad1fe P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:18.975061 16191 leader_election.cc:304] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 90f3da57796b488c9087a2f523adf045
I20260709 13:21:18.980670 16191 leader_election.cc:304] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
I20260709 13:21:18.954088 21489 raft_consensus.cc:493] T 9c61a39fd4f84424b129c1a6f9ba404d P 647a343a542d4a139fe106c19095129b [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:21:19.003834 16115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 232663us
I20260709 13:21:19.011809 16223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fffab26427e24d398e2695aad40ca134" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:19.012073 16221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a878999c0024d2abe517484180290fc" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:21:19.012524 16223 raft_consensus.cc:2468] T fffab26427e24d398e2695aad40ca134 P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 13.
I20260709 13:21:19.016790 16218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f5f89186204242bf377f7c42b2e2da" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:21:19.017374 16218 raft_consensus.cc:3060] T c2f5f89186204242bf377f7c42b2e2da P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:21:19.018853 16224 raft_consensus.cc:1275] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
W20260709 13:21:19.020052 16790 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:19.028210 16218 raft_consensus.cc:2468] T c2f5f89186204242bf377f7c42b2e2da P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 15.
I20260709 13:21:18.995694 21339 raft_consensus.cc:493] T 434bff3082d141838068f323bdb0d1ed P 90f3da57796b488c9087a2f523adf045 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:19.032516 16265 leader_election.cc:304] T c4f65cdf00b748bb963ee6e81ef43b79 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:21:19.032564 16221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a959dbecc2648eb85d1cc550f5a3245" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:19.033941 16223 raft_consensus.cc:1275] T 08c74490c40243a88e62d1760b5cd814 P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 13:21:18.994149 21489 raft_consensus.cc:3060] T 9c61a39fd4f84424b129c1a6f9ba404d P 647a343a542d4a139fe106c19095129b [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:21:19.036865 16220 raft_consensus.cc:1240] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 13. Current term is 14. Ops: [13.9-13.9]
I20260709 13:21:18.958470 21498 raft_consensus.cc:515] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:18.995469 21338 raft_consensus.cc:493] T 8cc959fb7f704fba9009c0595467d074 P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:21:19.038379 16115 leader_election.cc:304] T 26354361e9b044cea1c2b86c4882fe92 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860
I20260709 13:21:18.976676 21497 raft_consensus.cc:515] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:19.006610 21469 raft_consensus.cc:493] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:21:19.015254 20902 raft_consensus.cc:493] T 19baa7bc367844c288bca8f81439c413 P cb82cc7bd6db46d980c75237a1402860 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:18.965000 21198 raft_consensus.cc:515] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:19.005625 21390 raft_consensus.cc:493] T 5a959dbecc2648eb85d1cc550f5a3245 P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:21:19.015304 21403 raft_consensus.cc:493] T 74ca8140f35c4ae29e8a664f6bdfc829 P cb82cc7bd6db46d980c75237a1402860 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:21:18.978704 21325 raft_consensus.cc:515] T 66f75124215f4ca6bef831e174a792d8 P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:19.030124 21087 raft_consensus.cc:493] T c2f5f89186204242bf377f7c42b2e2da P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:19.050030 16115 leader_election.cc:304] T 02838c86531d4f98bf58d1c22b8baf77 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:21:18.992512 21370 raft_consensus.cc:515] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:19.037251 21468 raft_consensus.cc:493] T 84f2da98864b4425bf7c0055951fdd4b P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:21:19.042713 21444 raft_consensus.cc:493] T a378d7f796f94615b3b347119a9880fd P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:19.041167 21323 raft_consensus.cc:493] T 8638ee363273426db1393dd81e5c7b72 P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:21:19.055433 21450 raft_consensus.cc:2749] T c4f65cdf00b748bb963ee6e81ef43b79 P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:21:19.051757 21459 raft_consensus.cc:2749] T 26354361e9b044cea1c2b86c4882fe92 P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:21:19.051496 21504 consensus_queue.cc:1048] T 08c74490c40243a88e62d1760b5cd814 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.008s
I20260709 13:21:19.056169 21403 raft_consensus.cc:515] T 74ca8140f35c4ae29e8a664f6bdfc829 P cb82cc7bd6db46d980c75237a1402860 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:19.052700 20902 raft_consensus.cc:515] T 19baa7bc367844c288bca8f81439c413 P cb82cc7bd6db46d980c75237a1402860 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:19.039503 21339 raft_consensus.cc:515] T 434bff3082d141838068f323bdb0d1ed P 90f3da57796b488c9087a2f523adf045 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:19.063274 21497 leader_election.cc:290] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:19.063316 21457 raft_consensus.cc:2749] T 02838c86531d4f98bf58d1c22b8baf77 P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:21:19.048159 21154 raft_consensus.cc:2764] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Leader pre-election decision vote started in defunct term 13: won
I20260709 13:21:19.081629 16265 leader_election.cc:304] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:21:19.036641 21153 consensus_queue.cc:1048] T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.019s
W20260709 13:21:19.091929 16789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 81381.5us
I20260709 13:21:19.054353 21390 raft_consensus.cc:515] T 5a959dbecc2648eb85d1cc550f5a3245 P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:19.050530 21469 raft_consensus.cc:515] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:19.051638 21507 raft_consensus.cc:2749] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:21:19.048843 21338 raft_consensus.cc:515] T 8cc959fb7f704fba9009c0595467d074 P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:19.036935 21442 consensus_queue.cc:237] T c0eab25ce4c64e018a041beeda2867db P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:19.043869 21282 consensus_queue.cc:1048] T 8158bbd17b994d2eb96f560bd0f1b88d P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.036s
I20260709 13:21:19.047394 21490 consensus_queue.cc:1059] T 8a878999c0024d2abe517484180290fc P 647a343a542d4a139fe106c19095129b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.059s
I20260709 13:21:19.107084 16292 raft_consensus.cc:1275] T 434bff3082d141838068f323bdb0d1ed P 90f3da57796b488c9087a2f523adf045 [term 14 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:21:19.082731 21444 raft_consensus.cc:515] T a378d7f796f94615b3b347119a9880fd P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:19.109891 21510 raft_consensus.cc:493] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:19.110298 21339 leader_election.cc:290] T 434bff3082d141838068f323bdb0d1ed P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:19.110436 21510 raft_consensus.cc:515] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:19.057446 21087 raft_consensus.cc:515] T c2f5f89186204242bf377f7c42b2e2da P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:19.111096 21444 leader_election.cc:290] T a378d7f796f94615b3b347119a9880fd P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:19.112668 21510 leader_election.cc:290] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:19.112840 21087 leader_election.cc:290] T c2f5f89186204242bf377f7c42b2e2da P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:19.113278 20902 leader_election.cc:290] T 19baa7bc367844c288bca8f81439c413 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:19.113224 21510 consensus_queue.cc:1048] T 3884f864a96d428785895e5463e5728f P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:21:19.114310 21501 raft_consensus.cc:493] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:19.114965 21501 raft_consensus.cc:515] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:19.115641 21447 leader_election.cc:290] T 23f82224f4564fea93572e8206bad1fe P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:19.116441 16224 raft_consensus.cc:1240] T c2f5f89186204242bf377f7c42b2e2da P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 14. Current term is 15. Ops: []
I20260709 13:21:19.117108 16265 leader_election.cc:304] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac
I20260709 13:21:19.117393 21501 leader_election.cc:290] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:21:19.118031 21501 raft_consensus.cc:493] T ada5a004ba7c46819051e3ccec71ffdb P cb82cc7bd6db46d980c75237a1402860 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:19.118456 16265 leader_election.cc:304] T 2794af5b99984efebdee564035841cb0 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
I20260709 13:21:19.080361 21468 raft_consensus.cc:515] T 84f2da98864b4425bf7c0055951fdd4b P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:19.123430 21468 leader_election.cc:290] T 84f2da98864b4425bf7c0055951fdd4b P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:21:19.090215 21323 raft_consensus.cc:515] T 8638ee363273426db1393dd81e5c7b72 P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:19.114428 20902 raft_consensus.cc:493] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:19.130890 21506 raft_consensus.cc:2749] T 2794af5b99984efebdee564035841cb0 P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:21:19.132050 21509 raft_consensus.cc:493] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:19.132717 21403 leader_election.cc:290] T 74ca8140f35c4ae29e8a664f6bdfc829 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:19.132589 21509 raft_consensus.cc:515] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:19.106027 21450 raft_consensus.cc:2749] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:21:19.134238 21512 raft_consensus.cc:493] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:19.133920 21489 raft_consensus.cc:515] T 9c61a39fd4f84424b129c1a6f9ba404d P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:19.133019 21390 leader_election.cc:290] T 5a959dbecc2648eb85d1cc550f5a3245 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:21:19.144745 21509 leader_election.cc:290] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:19.145509 21323 leader_election.cc:290] T 8638ee363273426db1393dd81e5c7b72 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:19.135272 21444 raft_consensus.cc:493] T 9e65676b21b748728ad39668cfe722ec P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:21:19.159464 21198 leader_election.cc:290] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:19.160686 21450 raft_consensus.cc:493] T b87147731a68400e8cce9b879daa7c5d P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:21:19.118494 21501 raft_consensus.cc:515] T ada5a004ba7c46819051e3ccec71ffdb P cb82cc7bd6db46d980c75237a1402860 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:21:19.182694 16116 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.14.177.130:35943 timed out after 1.617s (SENT) [suppressed 6 similar messages]
I20260709 13:21:19.140995 20902 raft_consensus.cc:515] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:19.192245 16819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:19.193251 16819 raft_consensus.cc:2410] T 19baa7bc367844c288bca8f81439c413 P cb82cc7bd6db46d980c75237a1402860 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac for term 17 because replica has last-logged OpId of term: 15 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 9.
I20260709 13:21:19.193432 16823 raft_consensus.cc:1240] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P cb82cc7bd6db46d980c75237a1402860 [term 14 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 13. Current term is 14. Ops: [13.11-13.11]
W20260709 13:21:19.193652 16264 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:19.195997 16822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66ae176961b74813a58bc15c9c4c6f2a" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:19.196786 16817 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49906 (request call id 1048) took 1900 ms. Trace:
I20260709 13:21:19.197325 16817 rpcz_store.cc:276] 0709 13:21:17.293026 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:17.293189 (+   163us) service_pool.cc:224] Handling call
0709 13:21:19.193065 (+1899876us) spinlock_profiling.cc:243] Waited 1.52 s on lock 0x7b54007c3b54. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:21:19.193226 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3500121984}
I20260709 13:21:19.209867 21357 raft_consensus.cc:2749] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:21:19.209981 20902 leader_election.cc:290] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:21:19.166138 21450 raft_consensus.cc:515] T b87147731a68400e8cce9b879daa7c5d P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:21:19.209031 16115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 147452us
I20260709 13:21:19.212532 21450 leader_election.cc:290] T b87147731a68400e8cce9b879daa7c5d P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:19.166754 21444 raft_consensus.cc:515] T 9e65676b21b748728ad39668cfe722ec P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:19.213397 16115 leader_election.cc:304] T 1df93779209748b9860f6d893dd3de53 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:21:19.214885 21444 leader_election.cc:290] T 9e65676b21b748728ad39668cfe722ec P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:19.148299 21512 raft_consensus.cc:515] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:19.215269 21519 raft_consensus.cc:2749] T 1df93779209748b9860f6d893dd3de53 P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:21:19.212343 21357 consensus_queue.cc:1059] T 7c2095a5f4ad49408dbc7f69c606120d P 647a343a542d4a139fe106c19095129b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:21:19.221694 16294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a959dbecc2648eb85d1cc550f5a3245" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:19.232062 16294 raft_consensus.cc:2468] T 5a959dbecc2648eb85d1cc550f5a3245 P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 12.
I20260709 13:21:19.223363 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a9de347c711477e87c582d003f36925" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:21:19.225040 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f82224f4564fea93572e8206bad1fe" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
W20260709 13:21:19.231503 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 250930us
I20260709 13:21:19.242761 16295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2794af5b99984efebdee564035841cb0" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:19.243391 21357 raft_consensus.cc:3055] T 7c2095a5f4ad49408dbc7f69c606120d P 647a343a542d4a139fe106c19095129b [term 9 LEADER]: Stepping down as leader of term 9
W20260709 13:21:19.226451 16115 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:19.185607 21323 raft_consensus.cc:493] T cdbd78acb4cf46298e0de8ba0d130e19 P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
W20260709 13:21:19.321267 16192 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 46627.5us
W20260709 13:21:19.461637 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 205946us
W20260709 13:21:19.341032 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 158388us
W20260709 13:21:19.296403 16789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 191713us
I20260709 13:21:19.517886 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f2da98864b4425bf7c0055951fdd4b" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:19.617250 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:19.617410 16190 leader_election.cc:304] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860
I20260709 13:21:19.618131 16225 raft_consensus.cc:2468] T 19baa7bc367844c288bca8f81439c413 P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 16.
I20260709 13:21:19.252900 21338 leader_election.cc:290] T 8cc959fb7f704fba9009c0595467d074 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:21:19.518198 16151 raft_consensus.cc:1275] T 57a744f5aaf944678a63469cbf514015 P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
W20260709 13:21:19.524777 16265 connection.cc:256] RPC call timeout handler was delayed by -1.106260390999978s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22010000000000002s for a response
I20260709 13:21:19.620445 16820 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49896 (request call id 1284) took 2265 ms. Trace:
W20260709 13:21:19.620863 16217 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57604 (request call id 2808) took 2038 ms (client timeout 1617 ms). Trace:
I20260709 13:21:19.248992 16824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74ca8140f35c4ae29e8a664f6bdfc829" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:19.252161 21498 leader_election.cc:290] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:19.250130 16816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada5a004ba7c46819051e3ccec71ffdb" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 15 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:19.622068 16824 raft_consensus.cc:2468] T 74ca8140f35c4ae29e8a664f6bdfc829 P cb82cc7bd6db46d980c75237a1402860 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 13.
I20260709 13:21:19.250551 16821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66f75124215f4ca6bef831e174a792d8" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 15 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
W20260709 13:21:19.622411 21493 log.cc:927] Time spent T d16dfad0c3b540e5b4b86492ddf87208 P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.858s	user 0.010s	sys 0.000s
I20260709 13:21:19.620512 21452 raft_consensus.cc:493] T 8d4ab870a05347868956aa8b841b2981 P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:21:19.621181 21531 raft_consensus.cc:493] T e5ad4eb281564936a59f41e387a47156 P 90f3da57796b488c9087a2f523adf045 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:19.623319 21531 raft_consensus.cc:515] T e5ad4eb281564936a59f41e387a47156 P 90f3da57796b488c9087a2f523adf045 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:19.623788 16299 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48684 (request call id 2781) took 1907 ms. Trace:
W20260709 13:21:19.621486 16217 rpcz_store.cc:269] 0709 13:21:17.582870 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:17.583015 (+   145us) service_pool.cc:224] Handling call
0709 13:21:19.596264 (+2013249us) spinlock_profiling.cc:243] Waited 1.12 s on lock 0x7b5400367c54. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:21:19.620768 (+ 24504us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2582681856}
I20260709 13:21:19.625521 21531 leader_election.cc:290] T e5ad4eb281564936a59f41e387a47156 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:21:19.625545 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62fe3346cd3c42c4a55e14af9073bca4" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:19.626286 16217 raft_consensus.cc:2468] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 18.
I20260709 13:21:19.621513 16265 leader_election.cc:304] T 7c2095a5f4ad49408dbc7f69c606120d P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:21:19.624411 16299 rpcz_store.cc:276] 0709 13:21:17.716022 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:17.727371 (+ 11349us) service_pool.cc:224] Handling call
0709 13:21:17.744602 (+ 17231us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:21:17.744958 (+   356us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:21:17.744978 (+    20us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:21:17.748359 (+  3381us) log.cc:844] Serialized 26 byte log entry
0709 13:21:18.257418 (+509059us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:21:18.257482 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:18.257590 (+   108us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:21:19.519858 (+1262268us) spinlock_profiling.cc:243] Waited 6.81 ms on lock 0x7b4400022b4c. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69ad8fadb1 00007f69ad998c85 00007f69ad9996a2 00007f69b070b1db 00007f69b0706aaf 00007f69b06fa86e 00007f69b06f9bb6 00007f69b06f97d8 00007f69b0703221 00007f69b07031d9 00007f69b0703169
0709 13:21:19.623508 (+103650us) raft_consensus.cc:1685] finished
0709 13:21:19.623530 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:21:19.623730 (+   200us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":14546,"raft.queue_time_us":2218,"raft.run_cpu_time_us":136,"raft.run_wall_time_us":8268,"spinlock_wait_cycles":15671680,"thread_start_us":485379,"threads_started":1,"wal-append.queue_time_us":952737}
I20260709 13:21:19.627681 21151 raft_consensus.cc:2749] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:21:19.628261 21502 raft_consensus.cc:2804] T 7c2095a5f4ad49408dbc7f69c606120d P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:21:19.433919 16147 raft_consensus.cc:1275] T 3884f864a96d428785895e5463e5728f P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
I20260709 13:21:19.629051 21502 raft_consensus.cc:697] T 7c2095a5f4ad49408dbc7f69c606120d P 90f3da57796b488c9087a2f523adf045 [term 10 LEADER]: Becoming Leader. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:21:19.628604 21531 consensus_queue.cc:1048] T ada5a004ba7c46819051e3ccec71ffdb P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:21:19.629793 21151 consensus_queue.cc:1048] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:21:19.316603 16191 leader_election.cc:304] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
I20260709 13:21:19.631685 21151 raft_consensus.cc:2749] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:21:19.630757 21502 consensus_queue.cc:237] T 7c2095a5f4ad49408dbc7f69c606120d P 90f3da57796b488c9087a2f523adf045 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:21:19.251852 21471 log.cc:927] Time spent T 8cc959fb7f704fba9009c0595467d074 P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 1.589s	user 0.003s	sys 0.015s
I20260709 13:21:19.623054 21452 raft_consensus.cc:515] T 8d4ab870a05347868956aa8b841b2981 P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:19.456598 21370 leader_election.cc:290] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:19.629521 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ad5dbfa99ee4b8d9e28be92b3f36db5" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:19.542312 21245 raft_consensus.cc:493] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:21:19.633615 21245 raft_consensus.cc:515] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:19.634243 21452 leader_election.cc:290] T 8d4ab870a05347868956aa8b841b2981 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:21:19.620275 16823 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49892 (request call id 782) took 2160 ms. Trace:
W20260709 13:21:19.632005 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 315529us
I20260709 13:21:19.635681 16191 leader_election.cc:304] T 02838c86531d4f98bf58d1c22b8baf77 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
W20260709 13:21:19.405025 16263 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:19.635242 16823 rpcz_store.cc:276] 0709 13:21:17.461746 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:17.461901 (+   155us) service_pool.cc:224] Handling call
0709 13:21:19.193205 (+1731304us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:21:19.195297 (+  2092us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:19.494306 (+299009us) spinlock_profiling.cc:243] Waited 1.7 s on lock 0x7b54007c3b50. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5e8ad 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:21:19.620193 (+125887us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3918837120}
I20260709 13:21:19.637696 16190 leader_election.cc:304] T fffab26427e24d398e2695aad40ca134 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:21:19.638096 21324 raft_consensus.cc:2749] T 02838c86531d4f98bf58d1c22b8baf77 P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:21:19.638652 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a96373bb0e14dce8a266b2fc6c58507" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:19.638857 21324 raft_consensus.cc:2804] T fffab26427e24d398e2695aad40ca134 P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:21:19.639394 21324 raft_consensus.cc:493] T fffab26427e24d398e2695aad40ca134 P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:19.639827 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e4bfb9090c8460eb14fa7e0d818e7f1" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:21:19.639462 21283 consensus_queue.cc:1048] T d16dfad0c3b540e5b4b86492ddf87208 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.002s
I20260709 13:21:19.639917 21324 raft_consensus.cc:3060] T fffab26427e24d398e2695aad40ca134 P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:21:19.444451 21153 raft_consensus.cc:3055] T 8a878999c0024d2abe517484180290fc P 647a343a542d4a139fe106c19095129b [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:21:19.643383 21153 raft_consensus.cc:740] T 8a878999c0024d2abe517484180290fc P 647a343a542d4a139fe106c19095129b [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:21:19.644371 21153 consensus_queue.cc:260] T 8a878999c0024d2abe517484180290fc P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:19.645131 16150 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40844 (request call id 2563) took 2462 ms. Trace:
I20260709 13:21:19.645972 21153 raft_consensus.cc:3060] T 8a878999c0024d2abe517484180290fc P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:21:19.448092 16149 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46224 (request call id 807) took 2319 ms (client timeout 1819 ms). Trace:
I20260709 13:21:19.645604 16150 rpcz_store.cc:276] 0709 13:21:17.182705 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:17.184554 (+  1849us) service_pool.cc:224] Handling call
0709 13:21:17.190938 (+  6384us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:21:17.208974 (+ 18036us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:21:17.209018 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:21:17.225441 (+ 16423us) log.cc:844] Serialized 26 byte log entry
0709 13:21:17.489917 (+264476us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:21:17.775650 (+285733us) log.cc:844] Serialized 14 byte log entry
0709 13:21:17.776787 (+  1137us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:17.776878 (+    91us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:21:19.251079 (+1474201us) spinlock_profiling.cc:243] Waited 2.77 ms on lock 0x7b4400022b4c. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69ad8fadb1 00007f69ad998c85 00007f69ad9996a2 00007f69b070b1db 00007f69b0706aaf 00007f69b06fa86e 00007f69b06f9bb6 00007f69b06f97d8 00007f69b0703221 00007f69b07031d9 00007f69b0703169
0709 13:21:19.644777 (+393698us) raft_consensus.cc:1685] finished
0709 13:21:19.644892 (+   115us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:21:19.645039 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":28988,"prepare.queue_time_us":317029,"prepare.run_cpu_time_us":56,"prepare.run_wall_time_us":57,"raft.queue_time_us":19109,"raft.run_cpu_time_us":301,"raft.run_wall_time_us":28747,"spinlock_wait_cycles":6378752,"thread_start_us":488338,"threads_started":2,"wal-append.queue_time_us":321963}
W20260709 13:21:19.647456 16149 rpcz_store.cc:269] 0709 13:21:17.128736 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:19.447809 (+2319073us) service_pool.cc:211] Skipping call since client already timed out
0709 13:21:19.447995 (+   186us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260709 13:21:19.648806 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "829b4ad8c5a54eababeea1181b960d4b" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:19.502302 16143 raft_consensus.cc:1240] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [term 16 LEADER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 15. Current term is 16. Ops: [15.10-15.10]
W20260709 13:21:19.650498 16264 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:19.553471 21405 leader_election.cc:290] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:19.522018 21340 raft_consensus.cc:493] T 18ad7e3af5504571a69d10c00750a01c P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:19.652720 21340 raft_consensus.cc:515] T 18ad7e3af5504571a69d10c00750a01c P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } }
I20260709 13:21:19.450511 16144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2372981006d4b218215c28964dea623" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:21:19.657253 16144 raft_consensus.cc:2410] T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 for term 16 because replica has last-logged OpId of term: 15 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 10.
I20260709 13:21:19.657405 16142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5d59407eb545b895d8d922e0dd69fd" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:19.658172 21469 leader_election.cc:290] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:19.243625 16295 raft_consensus.cc:2468] T 2794af5b99984efebdee564035841cb0 P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 11.
I20260709 13:21:19.341765 21497 consensus_queue.cc:1059] T ee9ed67006144797b1de6ea8c2013a68 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.11, Next index: 11, Last known committed idx: 10, Time since last communication: 0.330s
I20260709 13:21:19.659799 21497 consensus_queue.cc:1243] T ee9ed67006144797b1de6ea8c2013a68 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Peer cb82cc7bd6db46d980c75237a1402860 log is divergent from this leader: its last log entry 13.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 13:21:19.234571 21444 raft_consensus.cc:493] T ecc1e363230043fe9fef390f0b979f20 P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:19.660815 21444 raft_consensus.cc:515] T ecc1e363230043fe9fef390f0b979f20 P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:19.556360 21455 leader_election.cc:290] T c9944d5e4b8c4109af9955c4f08c9095 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:19.663280 21087 consensus_queue.cc:1059] T ee9ed67006144797b1de6ea8c2013a68 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:21:19.664143 21480 leader_election.cc:290] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:19.664402 21444 leader_election.cc:290] T ecc1e363230043fe9fef390f0b979f20 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:19.664893 21512 leader_election.cc:290] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
W20260709 13:21:19.355916 21472 log.cc:927] Time spent T 1df93779209748b9860f6d893dd3de53 P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 1.649s	user 0.000s	sys 0.017s
I20260709 13:21:19.665169 21323 raft_consensus.cc:515] T cdbd78acb4cf46298e0de8ba0d130e19 P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:19.667052 16145 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40844 (request call id 2558) took 2485 ms. Trace:
I20260709 13:21:19.667418 16145 rpcz_store.cc:276] 0709 13:21:17.181334 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:17.191465 (+ 10131us) service_pool.cc:224] Handling call
0709 13:21:17.213087 (+ 21622us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:21:17.300035 (+ 86948us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:21:17.300066 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:21:17.317865 (+ 17799us) log.cc:844] Serialized 26 byte log entry
0709 13:21:17.564707 (+246842us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:21:17.727328 (+162621us) log.cc:844] Serialized 14 byte log entry
0709 13:21:17.727546 (+   218us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:17.837978 (+110432us) spinlock_profiling.cc:243] Waited 86.5 ms on lock 0x7b540011b2d4. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:21:17.838021 (+    43us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:21:19.352109 (+1514088us) spinlock_profiling.cc:243] Waited 69 ms on lock 0x7b4400022b4c. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69ad8fadb1 00007f69ad998c85 00007f69ad9996a2 00007f69b070b1db 00007f69b0706aaf 00007f69b06fa86e 00007f69b06f9bb6 00007f69b06f97d8 00007f69b0703221 00007f69b07031d9 00007f69b0703169
0709 13:21:19.666857 (+314748us) raft_consensus.cc:1685] finished
0709 13:21:19.666877 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:21:19.666998 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":258757,"prepare.run_cpu_time_us":32,"prepare.run_wall_time_us":32,"raft.queue_time_us":816,"raft.run_cpu_time_us":257,"raft.run_wall_time_us":15816,"spinlock_wait_cycles":357763712,"thread_start_us":370067,"threads_started":2,"wal-append.queue_time_us":354554}
I20260709 13:21:19.621135 16820 rpcz_store.cc:276] 0709 13:21:17.354830 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:17.356735 (+  1905us) service_pool.cc:224] Handling call
0709 13:21:17.357045 (+   310us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:21:17.467242 (+110197us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:21:17.467292 (+    50us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:21:17.472588 (+  5296us) log.cc:844] Serialized 26 byte log entry
0709 13:21:17.680163 (+207575us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:21:17.680220 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:19.458504 (+1778284us) spinlock_profiling.cc:243] Waited 104 ms on lock 0x7b54007c0e54. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:21:19.620069 (+161565us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:21:19.620135 (+    66us) raft_consensus.cc:1685] finished
0709 13:21:19.620147 (+    12us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:21:19.620387 (+   240us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":22819,"raft.queue_time_us":2373,"raft.run_cpu_time_us":209,"raft.run_wall_time_us":28247,"spinlock_wait_cycles":239596800,"thread_start_us":133523,"threads_started":1,"wal-append.queue_time_us":242351,"wal-append.run_cpu_time_us":2342,"wal-append.run_wall_time_us":1078034}
I20260709 13:21:19.407586 21515 raft_consensus.cc:493] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:21:19.671689 21515 raft_consensus.cc:515] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:19.612398 16192 leader_election.cc:304] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860
W20260709 13:21:19.673286 16192 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 60973.2us
I20260709 13:21:19.673800 21515 leader_election.cc:290] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:19.674212 21497 raft_consensus.cc:3060] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:21:19.352531 21489 leader_election.cc:290] T 9c61a39fd4f84424b129c1a6f9ba404d P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 15 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:19.358855 21441 raft_consensus.cc:3055] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:21:19.675477 21441 raft_consensus.cc:740] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:21:19.675690 16192 leader_election.cc:304] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:21:19.676335 21441 consensus_queue.cc:260] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:19.677652 21441 raft_consensus.cc:3060] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:21:19.245060 21501 leader_election.cc:290] T ada5a004ba7c46819051e3ccec71ffdb P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
W20260709 13:21:19.597892 21318 raft_consensus.cc:670] T 806d7690c2b644959152074bb2cab0a7 P cb82cc7bd6db46d980c75237a1402860: failed to trigger leader election: Illegal state: only voting members can start elections: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } } [suppressed 5 similar messages]
W20260709 13:21:19.679653 21505 log.cc:927] Time spent T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 0.776s	user 0.002s	sys 0.000s
W20260709 13:21:19.677749 21465 log.cc:927] Time spent T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 2.099s	user 0.016s	sys 0.032s
I20260709 13:21:19.682605 16144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62fe3346cd3c42c4a55e14af9073bca4" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:19.682715 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74ca8140f35c4ae29e8a664f6bdfc829" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
W20260709 13:21:19.681197 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 59753us
I20260709 13:21:19.662968 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "829b4ad8c5a54eababeea1181b960d4b" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:19.684549 16265 leader_election.cc:304] T fffab26427e24d398e2695aad40ca134 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:21:19.684960 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5d59407eb545b895d8d922e0dd69fd" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:19.683281 16144 raft_consensus.cc:2468] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 17.
I20260709 13:21:19.685774 16224 raft_consensus.cc:2468] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 15.
I20260709 13:21:19.639847 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c61a39fd4f84424b129c1a6f9ba404d" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:21:19.680570 16292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7ca61592e848ea9b0fb81a2ce33cdc" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:19.622406 16816 raft_consensus.cc:2468] T ada5a004ba7c46819051e3ccec71ffdb P cb82cc7bd6db46d980c75237a1402860 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 14.
I20260709 13:21:19.681634 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a9de347c711477e87c582d003f36925" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:19.687529 16292 raft_consensus.cc:2468] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 15.
I20260709 13:21:19.681998 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:19.688763 16297 raft_consensus.cc:2468] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 16.
I20260709 13:21:19.689297 16820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62fe3346cd3c42c4a55e14af9073bca4" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:19.683507 16294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e9b2bf7011f4ae688a79d7bb5a447b4" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:19.690258 16820 raft_consensus.cc:2468] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 18.
I20260709 13:21:19.683661 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a378d7f796f94615b3b347119a9880fd" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:19.686926 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7ca61592e848ea9b0fb81a2ce33cdc" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:19.694391 16821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746aabf856ab401c9ef5e859371e7b67" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 14 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:19.686482 16265 leader_election.cc:304] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:21:19.695602 16221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2372981006d4b218215c28964dea623" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:21:19.696252 16221 raft_consensus.cc:3060] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:21:19.695369 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 145183us
I20260709 13:21:19.698201 16821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af57996a8be84d398f0f023e8d08ef77" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:19.700852 16294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdbd78acb4cf46298e0de8ba0d130e19" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:19.701709 16294 raft_consensus.cc:2468] T cdbd78acb4cf46298e0de8ba0d130e19 P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 16.
I20260709 13:21:19.702770 16217 raft_consensus.cc:2393] T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 15: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
I20260709 13:21:19.707103 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62fe3346cd3c42c4a55e14af9073bca4" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:19.708074 16217 raft_consensus.cc:2468] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 18.
I20260709 13:21:19.709391 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:19.710235 16217 raft_consensus.cc:2468] T 19baa7bc367844c288bca8f81439c413 P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 16.
I20260709 13:21:19.699307 21444 raft_consensus.cc:2804] T fffab26427e24d398e2695aad40ca134 P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:21:19.715379 16824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2372981006d4b218215c28964dea623" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:19.649950 21357 raft_consensus.cc:740] T 7c2095a5f4ad49408dbc7f69c606120d P 647a343a542d4a139fe106c19095129b [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:21:19.712170 16295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9944d5e4b8c4109af9955c4f08c9095" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:19.723583 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e4bfb9090c8460eb14fa7e0d818e7f1" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:21:19.724480 16295 raft_consensus.cc:2468] T c9944d5e4b8c4109af9955c4f08c9095 P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 15.
W20260709 13:21:19.711486 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 72968.4us
I20260709 13:21:19.712980 16823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62fe3346cd3c42c4a55e14af9073bca4" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:19.716809 16818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2794af5b99984efebdee564035841cb0" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:19.717077 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada5a004ba7c46819051e3ccec71ffdb" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 15 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:19.711529 16116 leader_election.cc:304] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
I20260709 13:21:19.677003 21366 raft_consensus.cc:2804] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:21:19.733572 16221 raft_consensus.cc:2410] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 for term 16 because replica has last-logged OpId of term: 15 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 10.
I20260709 13:21:19.721001 16824 raft_consensus.cc:2393] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 16: Already voted for candidate cb82cc7bd6db46d980c75237a1402860 in this term.
I20260709 13:21:19.720278 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31969392ccdb43b491911a319e8bc7e1" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:19.796587 16297 raft_consensus.cc:2468] T 31969392ccdb43b491911a319e8bc7e1 P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 12.
W20260709 13:21:19.743481 16264 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:19.797880 16821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a9de347c711477e87c582d003f36925" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:21:19.755436 16823 raft_consensus.cc:2393] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 18: Already voted for candidate cb82cc7bd6db46d980c75237a1402860 in this term.
I20260709 13:21:19.732932 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9944d5e4b8c4109af9955c4f08c9095" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:19.698680 21323 leader_election.cc:290] T cdbd78acb4cf46298e0de8ba0d130e19 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:19.756703 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdbd78acb4cf46298e0de8ba0d130e19" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:19.798561 16821 raft_consensus.cc:3060] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:21:19.761108 16146 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40844 (request call id 2561) took 2579 ms. Trace:
I20260709 13:21:19.799199 16146 rpcz_store.cc:276] 0709 13:21:17.182171 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:17.185215 (+  3044us) service_pool.cc:224] Handling call
0709 13:21:17.185557 (+   342us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:21:17.217263 (+ 31706us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:21:17.217297 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:21:17.217787 (+   490us) log.cc:844] Serialized 26 byte log entry
0709 13:21:17.450180 (+232393us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:21:17.450218 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:19.752816 (+2302598us) spinlock_profiling.cc:243] Waited 31.3 ms on lock 0x7b540006dbd4. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:21:19.752870 (+    54us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:21:19.752919 (+    49us) raft_consensus.cc:1685] finished
0709 13:21:19.752931 (+    12us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:21:19.761016 (+  8085us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":676,"raft.run_cpu_time_us":205,"raft.run_wall_time_us":18168,"spinlock_wait_cycles":71884672,"thread_start_us":204571,"threads_started":1,"wal-append.queue_time_us":334536,"wal-append.run_cpu_time_us":1789,"wal-append.run_wall_time_us":1024826}
W20260709 13:21:19.804004 16790 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:21:19.804553 16790 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:19.726910 16816 raft_consensus.cc:1275] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 [term 12 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
W20260709 13:21:19.777359 21534 log.cc:927] Time spent T c0eab25ce4c64e018a041beeda2867db P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 0.083s	user 0.000s	sys 0.000s
I20260709 13:21:19.776947 21357 consensus_queue.cc:260] T 7c2095a5f4ad49408dbc7f69c606120d P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:19.723925 21444 raft_consensus.cc:493] T fffab26427e24d398e2695aad40ca134 P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:21:19.819442 21444 raft_consensus.cc:3060] T fffab26427e24d398e2695aad40ca134 P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:21:19.821602 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b7d1ab519044eda2d5e04a07242f25" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:19.778275 21530 consensus_queue.cc:1059] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.023s
I20260709 13:21:19.714673 21531 raft_consensus.cc:2804] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:21:19.824167 21531 raft_consensus.cc:697] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045 [term 15 LEADER]: Becoming Leader. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:21:19.825035 21497 raft_consensus.cc:2749] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:21:19.825152 21531 consensus_queue.cc:237] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:19.783748 21245 leader_election.cc:290] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:19.797997 21340 leader_election.cc:290] T 18ad7e3af5504571a69d10c00750a01c P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:21:18.793411 16222 rpcz_store.cc:276] 0709 13:21:17.744177 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:17.744363 (+   186us) service_pool.cc:224] Handling call
0709 13:21:17.756160 (+ 11797us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 13:21:17.756661 (+   501us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:21:17.756701 (+    40us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 13:21:17.761285 (+  4584us) log.cc:844] Serialized 78 byte log entry
0709 13:21:18.134036 (+372751us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:21:18.747740 (+613704us) log.cc:844] Serialized 14 byte log entry
0709 13:21:18.778444 (+ 30704us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:18.778546 (+   102us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:21:18.778603 (+    57us) raft_consensus.cc:1685] finished
0709 13:21:18.778623 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:21:18.778813 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":30339,"prepare.queue_time_us":603971,"prepare.run_cpu_time_us":69,"prepare.run_wall_time_us":3202,"raft.queue_time_us":738,"raft.run_cpu_time_us":156,"raft.run_wall_time_us":18858,"thread_start_us":470688,"threads_started":2,"wal-append.queue_time_us":963620}
I20260709 13:21:19.807291 21392 raft_consensus.cc:493] T d2ab0ac8347641e6955e9bce7d90c1a2 P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:21:19.827471 21441 raft_consensus.cc:493] T 57a744f5aaf944678a63469cbf514015 P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:19.827689 21392 raft_consensus.cc:515] T d2ab0ac8347641e6955e9bce7d90c1a2 P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:19.828063 21441 raft_consensus.cc:515] T 57a744f5aaf944678a63469cbf514015 P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:19.829811 16222 raft_consensus.cc:3055] T ee9ed67006144797b1de6ea8c2013a68 P 1ca2725d290e47d68b1a4b609c9816ac [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:21:19.830050 21392 leader_election.cc:290] T d2ab0ac8347641e6955e9bce7d90c1a2 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:19.830327 21441 leader_election.cc:290] T 57a744f5aaf944678a63469cbf514015 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:19.830228 16222 raft_consensus.cc:740] T ee9ed67006144797b1de6ea8c2013a68 P 1ca2725d290e47d68b1a4b609c9816ac [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:21:19.831646 16219 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57600 (request call id 1668) took 1988 ms. Trace:
I20260709 13:21:19.815848 21370 raft_consensus.cc:493] T 68a6838d82ef4cac83507f906139d90f P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:21:19.833274 21325 leader_election.cc:290] T 66f75124215f4ca6bef831e174a792d8 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:19.753722 21366 raft_consensus.cc:493] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Starting leader election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:21:19.788400 16817 raft_consensus.cc:1275] T 19baa7bc367844c288bca8f81439c413 P cb82cc7bd6db46d980c75237a1402860 [term 16 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 16 index: 10. (term mismatch)
I20260709 13:21:19.836591 16817 pending_rounds.cc:85] T 19baa7bc367844c288bca8f81439c413 P cb82cc7bd6db46d980c75237a1402860: Aborting all ops after (but not including) 9
I20260709 13:21:19.836943 16817 pending_rounds.cc:107] T 19baa7bc367844c288bca8f81439c413 P cb82cc7bd6db46d980c75237a1402860: Aborting uncommitted NO_OP operation due to leader change: 15.10
I20260709 13:21:19.837255 16817 raft_consensus.cc:2889] T 19baa7bc367844c288bca8f81439c413 P cb82cc7bd6db46d980c75237a1402860 [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:21:19.790349 16298 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48674 (request call id 2929) took 2189 ms. Trace:
I20260709 13:21:19.791431 16144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a959dbecc2648eb85d1cc550f5a3245" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:19.838711 16298 rpcz_store.cc:276] 0709 13:21:17.600778 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:17.643291 (+ 42513us) service_pool.cc:224] Handling call
0709 13:21:17.643683 (+   392us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:21:17.855679 (+211996us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:21:17.855729 (+    50us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:21:17.855794 (+    65us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:21:17.855858 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:19.789710 (+1933852us) spinlock_profiling.cc:243] Waited 212 ms on lock 0x7b54001eee54. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:21:19.790163 (+   453us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:21:19.790286 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":486787968}
I20260709 13:21:19.811673 16823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f82224f4564fea93572e8206bad1fe" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:19.775694 20991 raft_consensus.cc:493] T 8d4ab870a05347868956aa8b841b2981 P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:19.844585 20991 raft_consensus.cc:515] T 8d4ab870a05347868956aa8b841b2981 P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:19.845486 16823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31969392ccdb43b491911a319e8bc7e1" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:19.846691 20991 leader_election.cc:290] T 8d4ab870a05347868956aa8b841b2981 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:19.757185 21523 consensus_queue.cc:1048] T 66f75124215f4ca6bef831e174a792d8 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.102s
I20260709 13:21:19.847339 21461 consensus_queue.cc:1048] T 57a744f5aaf944678a63469cbf514015 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:21:19.815524 21519 raft_consensus.cc:493] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:19.848362 21519 raft_consensus.cc:515] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } }
I20260709 13:21:19.841563 21245 raft_consensus.cc:2804] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:21:19.849586 21245 raft_consensus.cc:493] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Starting leader election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:21:19.849992 21245 raft_consensus.cc:3060] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:21:19.850495 16825 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49906 (request call id 1049) took 2018 ms. Trace:
I20260709 13:21:19.850675 21519 leader_election.cc:290] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:19.818203 16146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a878999c0024d2abe517484180290fc" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:21:19.818440 21313 consensus_queue.cc:1048] T 3884f864a96d428785895e5463e5728f P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:21:19.832688 21370 raft_consensus.cc:515] T 68a6838d82ef4cac83507f906139d90f P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:19.850889 16825 rpcz_store.cc:276] 0709 13:21:17.831676 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:17.844825 (+ 13149us) service_pool.cc:224] Handling call
0709 13:21:17.920978 (+ 76153us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:21:18.901240 (+980262us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 13:21:19.058074 (+156834us) log.cc:844] Serialized 14 byte log entry
0709 13:21:19.168058 (+109984us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:21:19.168109 (+    51us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:21:19.168158 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:19.850279 (+682121us) spinlock_profiling.cc:243] Waited 980 ms on lock 0x7b54004b8cd4. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:21:19.850326 (+    47us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:21:19.850432 (+   106us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":11898,"prepare.queue_time_us":199718,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":40,"raft.queue_time_us":39144,"raft.run_cpu_time_us":128,"raft.run_wall_time_us":127,"spinlock_wait_cycles":2253837568,"thread_start_us":253956,"threads_started":3,"wal-append.queue_time_us":72566}
I20260709 13:21:19.832103 16219 rpcz_store.cc:276] 0709 13:21:17.843430 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:17.843547 (+   117us) service_pool.cc:224] Handling call
0709 13:21:17.950846 (+107299us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:21:18.744598 (+793752us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:21:18.744631 (+    33us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:21:18.745260 (+   629us) log.cc:844] Serialized 52 byte log entry
0709 13:21:18.749633 (+  4373us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:21:18.786333 (+ 36700us) log.cc:844] Serialized 14 byte log entry
0709 13:21:18.786640 (+   307us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:18.786740 (+   100us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:21:19.678898 (+892158us) spinlock_profiling.cc:243] Waited 9.77 ms on lock 0x7b4400022b4c. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69ad8fadb1 00007f69ad998c85 00007f69ad9996a2 00007f69b070b1db 00007f69b0706aaf 00007f69b06fa86e 00007f69b06f9bb6 00007f69b06f97d8 00007f69b0703221 00007f69b07031d9 00007f69b0703169
0709 13:21:19.831327 (+152429us) raft_consensus.cc:1685] finished
0709 13:21:19.831437 (+   110us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:21:19.831586 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":91469,"prepare.run_cpu_time_us":97,"prepare.run_wall_time_us":16595,"raft.queue_time_us":81768,"raft.run_cpu_time_us":251,"raft.run_wall_time_us":60847,"spinlock_wait_cycles":22479616,"thread_start_us":796989,"threads_started":2,"wal-append.queue_time_us":104972}
I20260709 13:21:19.725735 21515 consensus_queue.cc:1048] T 18ad7e3af5504571a69d10c00750a01c P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.063s
W20260709 13:21:19.835000 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 130382us
I20260709 13:21:19.853734 16817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f82224f4564fea93572e8206bad1fe" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:19.839547 16144 raft_consensus.cc:2468] T 5a959dbecc2648eb85d1cc550f5a3245 P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 12.
I20260709 13:21:19.876842 16144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f2da98864b4425bf7c0055951fdd4b" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:19.877982 16144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cc959fb7f704fba9009c0595467d074" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:19.879282 16144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:19.879916 16144 raft_consensus.cc:2468] T ecc1e363230043fe9fef390f0b979f20 P 647a343a542d4a139fe106c19095129b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 14.
I20260709 13:21:19.881120 16825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ab0ac8347641e6955e9bce7d90c1a2" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:19.808670 16221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5718c623ebee4d2bb770f0f9ddb2f164" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:19.884181 16221 raft_consensus.cc:2468] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 12.
I20260709 13:21:19.854245 16818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:19.838716 16821 raft_consensus.cc:2468] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 17.
I20260709 13:21:19.842425 16790 leader_election.cc:304] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:21:19.841326 21366 raft_consensus.cc:3060] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:21:19.858398 21354 raft_consensus.cc:493] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:19.867712 21370 leader_election.cc:290] T 68a6838d82ef4cac83507f906139d90f P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:19.843392 16222 consensus_queue.cc:260] T ee9ed67006144797b1de6ea8c2013a68 P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:19.870273 16192 leader_election.cc:304] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:21:19.874752 16146 raft_consensus.cc:2468] T 8a878999c0024d2abe517484180290fc P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 13.
I20260709 13:21:19.885686 16818 raft_consensus.cc:2468] T 19baa7bc367844c288bca8f81439c413 P cb82cc7bd6db46d980c75237a1402860 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 16.
I20260709 13:21:19.886566 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada5a004ba7c46819051e3ccec71ffdb" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 15 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:19.738029 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74ca8140f35c4ae29e8a664f6bdfc829" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:19.874296 21324 raft_consensus.cc:515] T fffab26427e24d398e2695aad40ca134 P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:19.887028 16221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ad5dbfa99ee4b8d9e28be92b3f36db5" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:19.926293 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87147731a68400e8cce9b879daa7c5d" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:19.898677 16220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18ad7e3af5504571a69d10c00750a01c" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:19.891237 16817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434bff3082d141838068f323bdb0d1ed" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:19.921231 16147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31969392ccdb43b491911a319e8bc7e1" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:20.048450 16144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c2095a5f4ad49408dbc7f69c606120d" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:21:20.028709 16142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a9de347c711477e87c582d003f36925" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:20.028399 16143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:19.902487 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5718c623ebee4d2bb770f0f9ddb2f164" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:19.901597 16145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e9b2bf7011f4ae688a79d7bb5a447b4" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:20.076934 16145 raft_consensus.cc:2468] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 12.
I20260709 13:21:19.909936 16820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:20.077996 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:20.078171 16820 raft_consensus.cc:2468] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 15.
I20260709 13:21:20.078668 16297 raft_consensus.cc:2468] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 16.
I20260709 13:21:19.904541 16218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806d7690c2b644959152074bb2cab0a7" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:20.079720 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66ae176961b74813a58bc15c9c4c6f2a" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:20.081305 16295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8158bbd17b994d2eb96f560bd0f1b88d" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 15 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:20.082173 16295 raft_consensus.cc:2410] T 8158bbd17b994d2eb96f560bd0f1b88d P 90f3da57796b488c9087a2f523adf045 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac for term 15 because replica has last-logged OpId of term: 13 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 8.
I20260709 13:21:20.083359 16292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5ad4eb281564936a59f41e387a47156" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:20.084203 16292 raft_consensus.cc:2468] T e5ad4eb281564936a59f41e387a47156 P 90f3da57796b488c9087a2f523adf045 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 14.
I20260709 13:21:20.076006 16149 raft_consensus.cc:2468] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 13.
I20260709 13:21:20.085019 16142 raft_consensus.cc:2393] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 in current term 17: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
I20260709 13:21:20.094800 16789 leader_election.cc:304] T 3a7ca61592e848ea9b0fb81a2ce33cdc P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:21:20.097891 16824 raft_consensus.cc:1240] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860 [term 17 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 16. Current term is 17. Ops: []
I20260709 13:21:19.914844 16825 raft_consensus.cc:1275] T 26354361e9b044cea1c2b86c4882fe92 P cb82cc7bd6db46d980c75237a1402860 [term 12 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:21:20.079762 16218 raft_consensus.cc:2393] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 14: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
I20260709 13:21:20.099650 16224 raft_consensus.cc:2468] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 14.
W20260709 13:21:20.104583 16192 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 234380us
I20260709 13:21:20.097371 16789 leader_election.cc:304] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:21:20.120465 16820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7ca61592e848ea9b0fb81a2ce33cdc" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:20.114560 16192 leader_election.cc:304] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:21:20.119783 16789 leader_election.cc:304] T c9944d5e4b8c4109af9955c4f08c9095 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:21:20.136555 16821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8638ee363273426db1393dd81e5c7b72" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:20.141471 16219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d4ab870a05347868956aa8b841b2981" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:20.143065 16291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada5a004ba7c46819051e3ccec71ffdb" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 15 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:20.146142 16789 leader_election.cc:304] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 
W20260709 13:21:20.147856 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 305511us
I20260709 13:21:20.149384 16790 leader_election.cc:304] T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 647a343a542d4a139fe106c19095129b
W20260709 13:21:19.898175 21462 log.cc:927] Time spent T af57996a8be84d398f0f023e8d08ef77 P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 2.456s	user 0.018s	sys 0.012s
W20260709 13:21:20.086011 21353 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 2.377s	user 0.018s	sys 0.014s
I20260709 13:21:20.069453 21401 raft_consensus.cc:3060] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:21:20.024715 21097 raft_consensus.cc:493] T 3884f864a96d428785895e5463e5728f P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:20.165019 16825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9944d5e4b8c4109af9955c4f08c9095" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:20.167574 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f5f89186204242bf377f7c42b2e2da" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:20.171352 16821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e65676b21b748728ad39668cfe722ec" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:20.171545 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5ad4eb281564936a59f41e387a47156" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:20.173664 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87ac0d1f7e8a419f9bcbc04aee0e067e" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:20.194605 16296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ab0ac8347641e6955e9bce7d90c1a2" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:20.176031 16818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdbd78acb4cf46298e0de8ba0d130e19" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:20.176185 16820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a378d7f796f94615b3b347119a9880fd" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:20.131253 21391 raft_consensus.cc:2804] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:21:20.176542 16790 leader_election.cc:304] T 19baa7bc367844c288bca8f81439c413 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:21:20.181496 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c61a39fd4f84424b129c1a6f9ba404d" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:21:20.194878 16300 raft_consensus.cc:2393] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 15: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
I20260709 13:21:20.174691 16291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74ca8140f35c4ae29e8a664f6bdfc829" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:20.085592 21151 raft_consensus.cc:493] T ecc1e363230043fe9fef390f0b979f20 P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:20.197059 16293 raft_consensus.cc:2376] T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 15. Re-sending same reply.
I20260709 13:21:20.141396 21434 raft_consensus.cc:2804] T 3a7ca61592e848ea9b0fb81a2ce33cdc P cb82cc7bd6db46d980c75237a1402860 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:21:20.163738 21313 raft_consensus.cc:2804] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 13:21:20.200742 16148 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.569s	user 0.040s	sys 0.048s
W20260709 13:21:20.186961 21353 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 2.468s	user 0.019s	sys 0.015s
I20260709 13:21:20.220585 16147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4dc6b0541ca431ebc757beac00521f1" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:20.227074 16147 raft_consensus.cc:2468] T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 13.
I20260709 13:21:20.222985 16190 leader_election.cc:304] T e5ad4eb281564936a59f41e387a47156 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:21:20.171612 21461 raft_consensus.cc:2804] T c9944d5e4b8c4109af9955c4f08c9095 P cb82cc7bd6db46d980c75237a1402860 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:21:20.232095 16146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fffab26427e24d398e2695aad40ca134" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
W20260709 13:21:20.225925 16148 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.594s	user 0.040s	sys 0.048s
I20260709 13:21:20.219969 16817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87147731a68400e8cce9b879daa7c5d" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:20.232988 16146 raft_consensus.cc:2393] T fffab26427e24d398e2695aad40ca134 P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 13: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
I20260709 13:21:20.171555 21319 raft_consensus.cc:2804] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:21:19.901948 21325 raft_consensus.cc:493] T 8a878999c0024d2abe517484180290fc P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:21:20.178472 21523 raft_consensus.cc:2804] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:21:19.916145 21444 raft_consensus.cc:515] T fffab26427e24d398e2695aad40ca134 P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:21:20.238289 16192 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 123876us
I20260709 13:21:20.225956 21434 raft_consensus.cc:493] T 3a7ca61592e848ea9b0fb81a2ce33cdc P cb82cc7bd6db46d980c75237a1402860 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:21:20.240276 21319 raft_consensus.cc:493] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:20.240703 16817 raft_consensus.cc:3060] T 806d7690c2b644959152074bb2cab0a7 P cb82cc7bd6db46d980c75237a1402860 [term 12 LEARNER]: Advancing to term 14
I20260709 13:21:20.240803 21337 raft_consensus.cc:2804] T e5ad4eb281564936a59f41e387a47156 P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:21:20.241312 21084 raft_consensus.cc:2804] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:21:20.199074 21151 raft_consensus.cc:515] T ecc1e363230043fe9fef390f0b979f20 P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:20.242043 16146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3884f864a96d428785895e5463e5728f" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:20.246641 16145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0eab25ce4c64e018a041beeda2867db" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:21:20.250226 21324 leader_election.cc:290] T fffab26427e24d398e2695aad40ca134 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 14 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:20.134560 21276 consensus_queue.cc:1048] T 434bff3082d141838068f323bdb0d1ed P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.067s
I20260709 13:21:20.253415 16218 raft_consensus.cc:1240] T 7c2095a5f4ad49408dbc7f69c606120d P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 9. Current term is 10. Ops: []
I20260709 13:21:20.253844 21357 raft_consensus.cc:3060] T 7c2095a5f4ad49408dbc7f69c606120d P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:21:20.287113 21337 raft_consensus.cc:493] T e5ad4eb281564936a59f41e387a47156 P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:20.305349 21337 raft_consensus.cc:3060] T e5ad4eb281564936a59f41e387a47156 P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:21:20.282950 21434 raft_consensus.cc:3060] T 3a7ca61592e848ea9b0fb81a2ce33cdc P cb82cc7bd6db46d980c75237a1402860 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:21:20.282277 21319 raft_consensus.cc:3060] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:21:20.255254 21461 raft_consensus.cc:493] T c9944d5e4b8c4109af9955c4f08c9095 P cb82cc7bd6db46d980c75237a1402860 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:21:20.312520 21461 raft_consensus.cc:3060] T c9944d5e4b8c4109af9955c4f08c9095 P cb82cc7bd6db46d980c75237a1402860 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:21:20.225804 21313 raft_consensus.cc:493] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:21:20.314603 21313 raft_consensus.cc:3060] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:21:20.318048 21151 leader_election.cc:290] T ecc1e363230043fe9fef390f0b979f20 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:20.202301 21391 raft_consensus.cc:493] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:20.319512 21391 raft_consensus.cc:3060] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:21:20.284291 21325 raft_consensus.cc:515] T 8a878999c0024d2abe517484180290fc P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:21:20.321234 21496 log.cc:927] Time spent T 08c74490c40243a88e62d1760b5cd814 P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 1.435s	user 0.006s	sys 0.000s
I20260709 13:21:20.321620 21325 leader_election.cc:290] T 8a878999c0024d2abe517484180290fc P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:20.135530 21354 raft_consensus.cc:515] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:20.237424 21442 raft_consensus.cc:493] T 84f2da98864b4425bf7c0055951fdd4b P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:21:20.257830 16296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68a6838d82ef4cac83507f906139d90f" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:20.323060 21442 raft_consensus.cc:515] T 84f2da98864b4425bf7c0055951fdd4b P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:20.259824 16222 raft_consensus.cc:3060] T ee9ed67006144797b1de6ea8c2013a68 P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:21:20.324039 16142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b7d1ab519044eda2d5e04a07242f25" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:20.325630 16142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ad5dbfa99ee4b8d9e28be92b3f36db5" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:20.326545 16142 raft_consensus.cc:2468] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 12.
I20260709 13:21:20.267984 21468 raft_consensus.cc:2764] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045 [term 14 LEADER]: Leader election decision vote started in defunct term 13: won
I20260709 13:21:20.269187 21501 raft_consensus.cc:2804] T 19baa7bc367844c288bca8f81439c413 P cb82cc7bd6db46d980c75237a1402860 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:21:20.269201 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
W20260709 13:21:20.273854 21526 log.cc:927] Time spent T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 0.096s	user 0.000s	sys 0.000s
I20260709 13:21:20.329027 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31969392ccdb43b491911a319e8bc7e1" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:20.237984 21507 raft_consensus.cc:493] T b6f07842d3aa47858df5665874c86f3a P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:21:20.274976 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d4ab870a05347868956aa8b841b2981" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:20.274835 16821 raft_consensus.cc:1275] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860 [term 12 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:21:20.276226 16147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e65676b21b748728ad39668cfe722ec" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:20.338433 16150 raft_consensus.cc:2468] T 8d4ab870a05347868956aa8b841b2981 P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 13.
I20260709 13:21:20.085135 21452 consensus_queue.cc:1059] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.116s
I20260709 13:21:20.194011 21530 raft_consensus.cc:2804] T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:21:20.289462 16143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f82224f4564fea93572e8206bad1fe" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:20.340054 16222 raft_consensus.cc:1275] T ee9ed67006144797b1de6ea8c2013a68 P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 13:21:20.343235 16143 raft_consensus.cc:1275] T 434bff3082d141838068f323bdb0d1ed P 647a343a542d4a139fe106c19095129b [term 14 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:21:20.344910 16143 raft_consensus.cc:1275] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:21:20.274483 21523 raft_consensus.cc:493] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:21:20.348836 16142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434bff3082d141838068f323bdb0d1ed" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:20.300766 16221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8638ee363273426db1393dd81e5c7b72" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:20.350492 16143 raft_consensus.cc:1240] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 16. Current term is 17. Ops: []
I20260709 13:21:20.286945 21084 raft_consensus.cc:493] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:20.302527 16226 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33050 (request call id 897) took 3498 ms. Trace:
I20260709 13:21:20.303052 16145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f5f89186204242bf377f7c42b2e2da" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:21:20.331288 16190 leader_election.cc:304] T d2ab0ac8347641e6955e9bce7d90c1a2 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860
I20260709 13:21:20.333525 16219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9944d5e4b8c4109af9955c4f08c9095" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:20.334630 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fffab26427e24d398e2695aad40ca134" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:21:20.354205 16300 raft_consensus.cc:3060] T fffab26427e24d398e2695aad40ca134 P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:21:20.335780 16291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 16 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:20.358198 16291 raft_consensus.cc:2410] T ecc1e363230043fe9fef390f0b979f20 P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac for term 16 because replica has last-logged OpId of term: 13 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 8.
I20260709 13:21:20.336493 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a878999c0024d2abe517484180290fc" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:20.327688 21501 raft_consensus.cc:493] T 19baa7bc367844c288bca8f81439c413 P cb82cc7bd6db46d980c75237a1402860 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:20.367842 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a96373bb0e14dce8a266b2fc6c58507" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:20.349710 16142 raft_consensus.cc:2468] T 434bff3082d141838068f323bdb0d1ed P 647a343a542d4a139fe106c19095129b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 14.
I20260709 13:21:20.207207 21097 raft_consensus.cc:515] T 3884f864a96d428785895e5463e5728f P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:20.299239 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a959dbecc2648eb85d1cc550f5a3245" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:20.345247 16149 raft_consensus.cc:3060] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:21:20.347939 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cc959fb7f704fba9009c0595467d074" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:20.324052 21354 leader_election.cc:290] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:21:20.371376 16190 leader_election.cc:304] T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:21:20.373883 21444 leader_election.cc:290] T fffab26427e24d398e2695aad40ca134 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 13 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:21:20.374802 21442 leader_election.cc:290] T 84f2da98864b4425bf7c0055951fdd4b P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:20.375979 16297 raft_consensus.cc:2393] T 8a878999c0024d2abe517484180290fc P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 13: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
I20260709 13:21:20.377162 16219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f2da98864b4425bf7c0055951fdd4b" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:20.351616 16226 rpcz_store.cc:276] 0709 13:21:16.790128 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:16.790233 (+   105us) service_pool.cc:224] Handling call
0709 13:21:16.864832 (+ 74599us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:21:17.110128 (+245296us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:21:17.110175 (+    47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:21:17.119981 (+  9806us) log.cc:844] Serialized 26 byte log entry
0709 13:21:17.333953 (+213972us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:21:17.347366 (+ 13413us) log.cc:844] Serialized 14 byte log entry
0709 13:21:17.362197 (+ 14831us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:19.897437 (+2535240us) spinlock_profiling.cc:243] Waited 12.4 ms on lock 0x7b4400022b4c. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69ad8fadb1 00007f69ad998c85 00007f69ad9996a2 00007f69b070b1db 00007f69b0706aaf 00007f69b06fa86e 00007f69b06f9bb6 00007f69b06f97d8 00007f69b0703221 00007f69b07031d9 00007f69b0703169
0709 13:21:19.905754 (+  8317us) spinlock_profiling.cc:243] Waited 119 ms on lock 0x7b5400395654. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:21:19.905810 (+    56us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:21:20.288040 (+382230us) raft_consensus.cc:1685] finished
0709 13:21:20.288063 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:21:20.288219 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":122241,"prepare.queue_time_us":14313,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":40,"raft.queue_time_us":185195,"raft.run_cpu_time_us":305,"raft.run_wall_time_us":14510,"spinlock_wait_cycles":302345344,"thread_start_us":183844,"threads_started":1,"wal-append.queue_time_us":303808}
I20260709 13:21:19.909147 15046 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:21:20.367899 21250 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 13 current term: 13 is higher.
I20260709 13:21:20.394774 16218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7ca61592e848ea9b0fb81a2ce33cdc" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:20.416817 16300 raft_consensus.cc:2468] T fffab26427e24d398e2695aad40ca134 P 90f3da57796b488c9087a2f523adf045 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 14.
I20260709 13:21:20.368178 21523 raft_consensus.cc:3060] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:21:20.346899 21530 raft_consensus.cc:493] T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:21:20.428387 16190 leader_election.cc:304] T fffab26427e24d398e2695aad40ca134 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:21:20.436733 16146 raft_consensus.cc:2468] T 3884f864a96d428785895e5463e5728f P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 9.
I20260709 13:21:20.369524 21084 raft_consensus.cc:3060] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:21:20.376283 21501 raft_consensus.cc:3060] T 19baa7bc367844c288bca8f81439c413 P cb82cc7bd6db46d980c75237a1402860 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:21:20.398629 21476 raft_consensus.cc:3055] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:21:20.398725 21497 raft_consensus.cc:2749] T d2ab0ac8347641e6955e9bce7d90c1a2 P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:21:20.411345 21324 raft_consensus.cc:2804] T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:21:20.346421 21507 raft_consensus.cc:515] T b6f07842d3aa47858df5665874c86f3a P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:20.345920 21353 consensus_queue.cc:1048] T 26354361e9b044cea1c2b86c4882fe92 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.094s
W20260709 13:21:20.485517 16117 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.674s	user 0.026s	sys 0.023s
W20260709 13:21:20.486071 16117 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.675s	user 0.026s	sys 0.023s
I20260709 13:21:20.467854 16145 raft_consensus.cc:2468] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 15.
I20260709 13:21:20.456713 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5ad4eb281564936a59f41e387a47156" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:20.501639 16222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d4ab870a05347868956aa8b841b2981" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:20.482129 21354 raft_consensus.cc:493] T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:21:20.507078 16044 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:49250 (request call id 321) took 46224 ms (client timeout 15000 ms). Trace:
I20260709 13:21:20.483397 21154 raft_consensus.cc:2804] T fffab26427e24d398e2695aad40ca134 P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:21:20.438651 21530 raft_consensus.cc:3060] T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:21:20.509136 16299 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.829s	user 0.008s	sys 0.047s
W20260709 13:21:20.509637 16299 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.830s	user 0.008s	sys 0.047s
W20260709 13:21:20.508834 16115 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.756s	user 0.013s	sys 0.028s
I20260709 13:21:20.512502 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
W20260709 13:21:20.512862 16115 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.760s	user 0.014s	sys 0.028s
I20260709 13:21:20.513249 16224 raft_consensus.cc:2468] T ecc1e363230043fe9fef390f0b979f20 P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 15.
I20260709 13:21:20.517107 16145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af57996a8be84d398f0f023e8d08ef77" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:20.518925 16820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 16 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:20.495023 21324 raft_consensus.cc:697] T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac [term 15 LEADER]: Becoming Leader. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:21:20.523475 16820 raft_consensus.cc:2410] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac for term 16 because replica has last-logged OpId of term: 13 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 8.
I20260709 13:21:20.444538 21476 raft_consensus.cc:740] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
W20260709 13:21:20.511416 21245 consensus_meta.cc:287] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b: Time spent flushing consensus metadata: real 0.661s	user 0.013s	sys 0.000s
I20260709 13:21:20.552846 16142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c61a39fd4f84424b129c1a6f9ba404d" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:21:20.553908 16150 raft_consensus.cc:1275] T 8d4ab870a05347868956aa8b841b2981 P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 13:21:20.505293 21508 raft_consensus.cc:493] T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
W20260709 13:21:20.511240 21531 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.652s	user 0.045s	sys 0.041s
W20260709 13:21:20.518702 21456 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 2.816s	user 0.025s	sys 0.046s
W20260709 13:21:20.507624 16044 rpcz_store.cc:269] 0709 13:20:34.294795 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:34.299821 (+  5026us) service_pool.cc:224] Handling call
0709 13:21:20.506872 (+46207051us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:21:15.247339 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:21:15.248006 (+   667us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:21:15.248060 (+    54us) write_op.cc:435] Acquired schema lock
0709 13:21:15.248072 (+    12us) tablet.cc:662] Decoding operations
0709 13:21:15.249482 (+  1410us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:21:15.249591 (+   109us) write_op.cc:641] Partition lock acquired for write op
0709 13:21:15.249628 (+    37us) tablet.cc:685] Acquiring locks for 15 operations
0709 13:21:15.250903 (+  1275us) tablet.cc:701] Row locks acquired
0709 13:21:15.250946 (+    43us) write_op.cc:260] PREPARE: finished
0709 13:21:15.262087 (+ 11141us) write_op.cc:270] Start()
0709 13:21:15.288523 (+ 26436us) write_op.cc:276] Timestamp: P: 1783603275251097 usec, L: 0
0709 13:21:15.296551 (+  8028us) op_driver.cc:348] REPLICATION: starting
0709 13:21:15.396509 (+ 99958us) log.cc:844] Serialized 6024 byte log entry
0709 13:21:15.900576 (+504067us) op_driver.cc:464] REPLICATION: finished
0709 13:21:15.955243 (+ 54667us) write_op.cc:301] APPLY: starting
0709 13:21:15.955392 (+   149us) tablet.cc:1367] starting BulkCheckPresence
0709 13:21:16.128885 (+173493us) tablet.cc:1370] finished BulkCheckPresence
0709 13:21:16.128996 (+   111us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:21:16.740991 (+611995us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:21:16.770514 (+ 29523us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=30,key_file_lookups=30,delta_file_lookups=0,mrs_lookups=0
0709 13:21:16.770567 (+    53us) write_op.cc:312] APPLY: finished
0709 13:21:16.771064 (+   497us) log.cc:844] Serialized 139 byte log entry
0709 13:21:16.771311 (+   247us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:21:16.771502 (+   191us) write_op.cc:454] Released schema lock
0709 13:21:16.771574 (+    72us) write_op.cc:341] FINISH: Updating metrics
0709 13:21:16.863027 (+ 91453us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:21:16.870272 (+  7245us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":1159468,"reported_tablets":48,"spinlock_wait_cycles":19200,"child_traces":[["op",{"apply.queue_time_us":54265,"apply.run_cpu_time_us":27270,"apply.run_wall_time_us":816882,"cfile_cache_hit":62,"cfile_cache_hit_bytes":66767,"num_ops":15,"prepare.queue_time_us":186166,"prepare.run_cpu_time_us":28588,"prepare.run_wall_time_us":341448,"raft.queue_time_us":106760,"raft.run_cpu_time_us":14715,"raft.run_wall_time_us":54589,"replication_time_us":602242,"thread_start_us":253260,"threads_started":4,"wal-append.queue_time_us":242373,"wal-append.run_cpu_time_us":32734,"wal-append.run_wall_time_us":2272804}]]}
I20260709 13:21:20.557536 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdbd78acb4cf46298e0de8ba0d130e19" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:20.558702 16142 raft_consensus.cc:2393] T 9c61a39fd4f84424b129c1a6f9ba404d P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 15: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
W20260709 13:21:20.563413 16223 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.711s	user 0.028s	sys 0.070s
W20260709 13:21:20.590678 16223 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.738s	user 0.029s	sys 0.075s
W20260709 13:21:20.556803 16294 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.749s	user 0.027s	sys 0.027s
W20260709 13:21:20.592743 16294 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.785s	user 0.028s	sys 0.027s
I20260709 13:21:20.594594 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:20.596565 16192 leader_election.cc:304] T ecc1e363230043fe9fef390f0b979f20 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860
I20260709 13:21:20.520879 21154 raft_consensus.cc:697] T fffab26427e24d398e2695aad40ca134 P 1ca2725d290e47d68b1a4b609c9816ac [term 14 LEADER]: Becoming Leader. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:21:20.572597 16145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fffab26427e24d398e2695aad40ca134" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:20.599866 16145 raft_consensus.cc:2468] T fffab26427e24d398e2695aad40ca134 P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 13.
I20260709 13:21:20.543963 21452 raft_consensus.cc:493] T 8158bbd17b994d2eb96f560bd0f1b88d P 90f3da57796b488c9087a2f523adf045 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:21:20.561015 21531 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.702s	user 0.046s	sys 0.041s
I20260709 13:21:20.514864 21354 raft_consensus.cc:515] T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:21:20.587692 21456 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 2.885s	user 0.025s	sys 0.050s
W20260709 13:21:20.628111 21182 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.745s	user 0.003s	sys 0.031s
I20260709 13:21:20.632123 16146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4dc6b0541ca431ebc757beac00521f1" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:20.638722 16146 raft_consensus.cc:2468] T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 13.
I20260709 13:21:20.621843 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5ad4eb281564936a59f41e387a47156" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:20.622874 16142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8158bbd17b994d2eb96f560bd0f1b88d" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 15 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:20.641858 16145 raft_consensus.cc:1275] T 18ad7e3af5504571a69d10c00750a01c P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 13:21:20.644632 16146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66f75124215f4ca6bef831e174a792d8" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 15 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
W20260709 13:21:20.650166 16151 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.774s	user 0.019s	sys 0.015s
I20260709 13:21:20.617158 21480 raft_consensus.cc:2749] T ecc1e363230043fe9fef390f0b979f20 P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:21:20.651823 16151 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.776s	user 0.020s	sys 0.015s
I20260709 13:21:20.653574 16142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2372981006d4b218215c28964dea623" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
W20260709 13:21:20.640228 21182 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.757s	user 0.003s	sys 0.031s
I20260709 13:21:20.655263 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f5f89186204242bf377f7c42b2e2da" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:20.654450 16142 raft_consensus.cc:2468] T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 16.
I20260709 13:21:20.658792 16150 raft_consensus.cc:2468] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 15.
I20260709 13:21:20.585851 21245 raft_consensus.cc:515] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:20.615423 21452 raft_consensus.cc:515] T 8158bbd17b994d2eb96f560bd0f1b88d P 90f3da57796b488c9087a2f523adf045 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:20.635847 21283 raft_consensus.cc:493] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:20.593122 21508 raft_consensus.cc:515] T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:20.663944 16145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68a6838d82ef4cac83507f906139d90f" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:20.652010 21357 raft_consensus.cc:493] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:21:20.656668 21539 log.cc:927] Time spent T 7c2095a5f4ad49408dbc7f69c606120d P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
W20260709 13:21:20.672395 16816 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.790s	user 0.016s	sys 0.020s
I20260709 13:21:20.673590 16146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62fe3346cd3c42c4a55e14af9073bca4" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:20.721195 16146 raft_consensus.cc:2468] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 18.
I20260709 13:21:20.674352 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87ac0d1f7e8a419f9bcbc04aee0e067e" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:20.675102 16142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fffab26427e24d398e2695aad40ca134" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:21:20.671618 21283 raft_consensus.cc:515] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } }
I20260709 13:21:20.681609 16148 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40844 (request call id 2560) took 3499 ms. Trace:
I20260709 13:21:20.695257 21154 consensus_queue.cc:237] T fffab26427e24d398e2695aad40ca134 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:21:20.700325 21538 log.cc:927] Time spent T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 0.050s	user 0.000s	sys 0.000s
I20260709 13:21:20.702767 21434 raft_consensus.cc:515] T 3a7ca61592e848ea9b0fb81a2ce33cdc P cb82cc7bd6db46d980c75237a1402860 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:20.703614 21507 leader_election.cc:290] T b6f07842d3aa47858df5665874c86f3a P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:20.660230 21476 consensus_queue.cc:260] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:20.706357 21245 leader_election.cc:290] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 18 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
W20260709 13:21:20.707136 21401 consensus_meta.cc:287] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860: Time spent flushing consensus metadata: real 0.511s	user 0.011s	sys 0.000s
I20260709 13:21:20.722592 16142 raft_consensus.cc:3060] T fffab26427e24d398e2695aad40ca134 P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:21:20.722908 16816 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.841s	user 0.016s	sys 0.020s
I20260709 13:21:20.726229 16148 rpcz_store.cc:276] 0709 13:21:17.181867 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:17.183308 (+  1441us) service_pool.cc:224] Handling call
0709 13:21:17.736316 (+553008us) spinlock_profiling.cc:243] Waited 7.27 ms on lock 0x7b5800011198. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69ad8fadb1 00007f69ada3ab71 00007f69ada3aeee 0000000000375939 00007f69b5aae1e0 00007f69b5b770c4 00007f69b5b77131 00007f69b5b38fc0 00007f69b5b23929 00007f69b5b233d2 00007f69af09abd4
0709 13:21:17.736541 (+   225us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:21:17.905363 (+168822us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:21:17.905409 (+    46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:21:19.555139 (+1649730us) spinlock_profiling.cc:243] Waited 4.44 ms on lock 0x7b2c00000658. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69af458a11 00007f69af468372 00007f69b0b9aa01 00007f69b0b5f43a 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2
0709 13:21:19.631328 (+ 76189us) log.cc:844] Serialized 26 byte log entry
0709 13:21:20.233040 (+601712us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:21:20.233081 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:20.233155 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:21:20.681396 (+448241us) raft_consensus.cc:1685] finished
0709 13:21:20.681416 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:21:20.681551 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":4396,"raft.run_cpu_time_us":127,"raft.run_wall_time_us":4907,"spinlock_wait_cycles":26923776,"thread_start_us":708642,"threads_started":2,"wal-append.queue_time_us":1010026}
I20260709 13:21:20.730662 16146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a878999c0024d2abe517484180290fc" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:20.736024 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c61a39fd4f84424b129c1a6f9ba404d" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:21:20.737169 16297 raft_consensus.cc:1275] T 8158bbd17b994d2eb96f560bd0f1b88d P 90f3da57796b488c9087a2f523adf045 [term 14 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 14 index: 9. (term mismatch)
I20260709 13:21:20.771623 16146 raft_consensus.cc:2393] T 8a878999c0024d2abe517484180290fc P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 13: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
I20260709 13:21:20.771763 16297 pending_rounds.cc:85] T 8158bbd17b994d2eb96f560bd0f1b88d P 90f3da57796b488c9087a2f523adf045: Aborting all ops after (but not including) 8
I20260709 13:21:20.772070 16297 pending_rounds.cc:107] T 8158bbd17b994d2eb96f560bd0f1b88d P 90f3da57796b488c9087a2f523adf045: Aborting uncommitted NO_OP operation due to leader change: 13.9
I20260709 13:21:20.772352 16297 raft_consensus.cc:2889] T 8158bbd17b994d2eb96f560bd0f1b88d P 90f3da57796b488c9087a2f523adf045 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:21:20.737905 16291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ad5dbfa99ee4b8d9e28be92b3f36db5" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:20.668758 21097 leader_election.cc:290] T 3884f864a96d428785895e5463e5728f P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:20.777616 16817 raft_consensus.cc:1240] T 3a7ca61592e848ea9b0fb81a2ce33cdc P cb82cc7bd6db46d980c75237a1402860 [term 16 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 15. Current term is 16. Ops: []
I20260709 13:21:20.778298 16822 raft_consensus.cc:2410] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac for term 18 because replica has last-logged OpId of term: 16 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 11.
I20260709 13:21:20.758802 16144 raft_consensus.cc:2468] T 7c2095a5f4ad49408dbc7f69c606120d P 647a343a542d4a139fe106c19095129b [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 10.
I20260709 13:21:20.653239 21324 consensus_queue.cc:237] T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:20.781185 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3884f864a96d428785895e5463e5728f" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
W20260709 13:21:20.764683 16265 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 1.053s	user 0.017s	sys 0.022s
I20260709 13:21:20.766743 16296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18ad7e3af5504571a69d10c00750a01c" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:20.771623 16300 raft_consensus.cc:2393] T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 15: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
I20260709 13:21:20.783423 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d4ab870a05347868956aa8b841b2981" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:20.784672 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806d7690c2b644959152074bb2cab0a7" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:20.785634 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57a744f5aaf944678a63469cbf514015" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:20.786195 16300 raft_consensus.cc:1275] T e5ad4eb281564936a59f41e387a47156 P 90f3da57796b488c9087a2f523adf045 [term 14 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 13:21:20.786654 16297 raft_consensus.cc:2410] T 57a744f5aaf944678a63469cbf514015 P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b for term 12 because replica has last-logged OpId of term: 11 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 9.
I20260709 13:21:20.775025 16115 leader_election.cc:304] T 5a959dbecc2648eb85d1cc550f5a3245 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac
I20260709 13:21:20.728440 21456 raft_consensus.cc:493] T 68a6838d82ef4cac83507f906139d90f P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:21:20.777196 16291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f2da98864b4425bf7c0055951fdd4b" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
W20260709 13:21:20.782402 16265 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 1.072s	user 0.018s	sys 0.022s
I20260709 13:21:20.789222 16297 raft_consensus.cc:1275] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045 [term 14 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 14 index: 9. (term mismatch)
I20260709 13:21:20.790169 16115 leader_election.cc:304] T 2794af5b99984efebdee564035841cb0 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:21:20.708801 21357 raft_consensus.cc:515] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:20.790467 16148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 16 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:20.794203 16148 raft_consensus.cc:2410] T ecc1e363230043fe9fef390f0b979f20 P 647a343a542d4a139fe106c19095129b [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac for term 16 because replica has last-logged OpId of term: 13 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 8.
I20260709 13:21:20.790606 16297 pending_rounds.cc:85] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045: Aborting all ops after (but not including) 8
I20260709 13:21:20.794842 16297 pending_rounds.cc:107] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045: Aborting uncommitted NO_OP operation due to leader change: 13.9
I20260709 13:21:20.795218 16297 raft_consensus.cc:2889] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:21:20.789902 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 1.07928e+06us
I20260709 13:21:20.801492 16296 raft_consensus.cc:1275] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 13:21:20.785302 21516 raft_consensus.cc:493] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:20.807786 16150 raft_consensus.cc:2468] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 15.
I20260709 13:21:20.818811 16142 raft_consensus.cc:2468] T fffab26427e24d398e2695aad40ca134 P 647a343a542d4a139fe106c19095129b [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 14.
I20260709 13:21:20.803404 21354 leader_election.cc:290] T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
W20260709 13:21:20.827142 16819 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.663s	user 0.018s	sys 0.018s
W20260709 13:21:20.838359 16819 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.674s	user 0.018s	sys 0.018s
I20260709 13:21:20.839664 16819 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49906 (request call id 1058) took 1641 ms. Trace:
I20260709 13:21:20.840186 16819 rpcz_store.cc:276] 0709 13:21:19.198752 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:19.198892 (+   140us) service_pool.cc:224] Handling call
0709 13:21:19.211075 (+ 12183us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:21:19.245351 (+ 34276us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:21:19.245397 (+    46us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:21:19.544472 (+299075us) log.cc:844] Serialized 52 byte log entry
0709 13:21:20.103573 (+559101us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:21:20.838888 (+735315us) log.cc:844] Serialized 14 byte log entry
0709 13:21:20.839229 (+   341us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:20.839324 (+    95us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:21:20.839386 (+    62us) raft_consensus.cc:1685] finished
0709 13:21:20.839398 (+    12us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:21:20.839582 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":33793,"raft.queue_time_us":34210,"raft.run_cpu_time_us":115,"raft.run_wall_time_us":116,"thread_start_us":1139796,"threads_started":2,"wal-append.queue_time_us":1036616}
I20260709 13:21:20.809732 21508 leader_election.cc:290] T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:20.820830 21452 leader_election.cc:290] T 8158bbd17b994d2eb96f560bd0f1b88d P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:21:20.794241 21456 raft_consensus.cc:515] T 68a6838d82ef4cac83507f906139d90f P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:21:20.844918 21337 consensus_meta.cc:287] T e5ad4eb281564936a59f41e387a47156 P 1ca2725d290e47d68b1a4b609c9816ac: Time spent flushing consensus metadata: real 0.539s	user 0.012s	sys 0.000s
W20260709 13:21:20.924726 21548 log.cc:927] Time spent T 66ae176961b74813a58bc15c9c4c6f2a P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.088s	user 0.001s	sys 0.000s
W20260709 13:21:20.925956 21549 log.cc:927] Time spent T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.090s	user 0.000s	sys 0.000s
I20260709 13:21:20.950171 16151 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46224 (request call id 815) took 1432 ms. Trace:
I20260709 13:21:20.950798 16151 rpcz_store.cc:276] 0709 13:21:19.518076 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:19.694206 (+176130us) service_pool.cc:224] Handling call
0709 13:21:19.719730 (+ 25524us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:21:19.720136 (+   406us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:21:19.720171 (+    35us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:21:19.874754 (+154583us) spinlock_profiling.cc:243] Waited 11.6 ms on lock 0x7b2c00000658. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69af458a11 00007f69af468372 00007f69b0b9aa01 00007f69b0b5f43a 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2
0709 13:21:19.875438 (+   684us) log.cc:844] Serialized 52 byte log entry
0709 13:21:20.655078 (+779640us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:21:20.655117 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:20.655202 (+    85us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:21:20.949716 (+294514us) raft_consensus.cc:1685] finished
0709 13:21:20.949747 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:21:20.950081 (+   334us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":70502,"raft.run_cpu_time_us":133,"raft.run_wall_time_us":6012,"spinlock_wait_cycles":26733440,"thread_start_us":779225,"threads_started":1,"wal-append.queue_time_us":1071735}
W20260709 13:21:20.963894 21547 log.cc:927] Time spent T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 13:21:20.819123 21516 raft_consensus.cc:515] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:21:20.967633 21496 log.cc:927] Time spent T 08c74490c40243a88e62d1760b5cd814 P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 0.097s	user 0.001s	sys 0.000s
I20260709 13:21:20.932902 21508 raft_consensus.cc:2804] T 5a959dbecc2648eb85d1cc550f5a3245 P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 13:21:20.978248 21313 consensus_meta.cc:287] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860: Time spent flushing consensus metadata: real 0.663s	user 0.009s	sys 0.005s
W20260709 13:21:20.990101 16144 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40830 (request call id 1693) took 1284 ms (client timeout 1642 ms). Trace:
W20260709 13:21:21.015560 16144 rpcz_store.cc:269] 0709 13:21:19.705408 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:20.003664 (+298256us) service_pool.cc:224] Handling call
0709 13:21:20.989854 (+986190us) spinlock_profiling.cc:243] Waited 573 ms on lock 0x7b5400072bd4. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:21:20.990012 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1317852416}
I20260709 13:21:20.987066 16223 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57604 (request call id 2827) took 1857 ms. Trace:
W20260709 13:21:21.002996 21501 consensus_meta.cc:287] T 19baa7bc367844c288bca8f81439c413 P cb82cc7bd6db46d980c75237a1402860: Time spent flushing consensus metadata: real 0.554s	user 0.006s	sys 0.004s
I20260709 13:21:21.017318 16223 rpcz_store.cc:276] 0709 13:21:19.129904 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:19.810617 (+680713us) service_pool.cc:224] Handling call
0709 13:21:19.811037 (+   420us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:21:19.851555 (+ 40518us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:21:19.851587 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:21:19.852146 (+   559us) log.cc:844] Serialized 26 byte log entry
0709 13:21:20.591180 (+739034us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:21:20.591228 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:20.986506 (+395278us) spinlock_profiling.cc:243] Waited 40.1 ms on lock 0x7b54002c9a54. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:21:20.986575 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:21:20.986646 (+    71us) raft_consensus.cc:1685] finished
0709 13:21:20.986668 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:21:20.986969 (+   301us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":25142,"raft.queue_time_us":1929,"raft.run_cpu_time_us":101,"raft.run_wall_time_us":100,"spinlock_wait_cycles":92275456,"thread_start_us":738590,"threads_started":1,"wal-append.queue_time_us":955634}
W20260709 13:21:21.014549 21391 consensus_meta.cc:287] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac: Time spent flushing consensus metadata: real 0.694s	user 0.011s	sys 0.002s
W20260709 13:21:20.976541 21540 log.cc:927] Time spent T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860: Append to log took a long time: real 0.056s	user 0.001s	sys 0.000s
I20260709 13:21:20.991557 21490 raft_consensus.cc:493] T 02838c86531d4f98bf58d1c22b8baf77 P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
W20260709 13:21:21.014302 21366 consensus_meta.cc:287] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac: Time spent flushing consensus metadata: real 0.878s	user 0.009s	sys 0.005s
W20260709 13:21:21.021112 21319 consensus_meta.cc:287] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860: Time spent flushing consensus metadata: real 0.703s	user 0.007s	sys 0.006s
I20260709 13:21:20.971566 21337 raft_consensus.cc:515] T e5ad4eb281564936a59f41e387a47156 P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:21.014940 21508 raft_consensus.cc:493] T 5a959dbecc2648eb85d1cc550f5a3245 P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Starting leader election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
W20260709 13:21:21.035148 16147 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.598s	user 0.026s	sys 0.128s
W20260709 13:21:21.038437 16147 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.601s	user 0.027s	sys 0.128s
I20260709 13:21:21.029443 21501 raft_consensus.cc:515] T 19baa7bc367844c288bca8f81439c413 P cb82cc7bd6db46d980c75237a1402860 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:21.039124 21319 raft_consensus.cc:515] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:21.044237 16816 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49892 (request call id 784) took 1212 ms. Trace:
I20260709 13:21:21.032996 21434 leader_election.cc:290] T 3a7ca61592e848ea9b0fb81a2ce33cdc P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 16 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:21.023455 21313 raft_consensus.cc:515] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:21.044625 16816 rpcz_store.cc:276] 0709 13:21:19.834551 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:19.854425 (+ 19874us) service_pool.cc:224] Handling call
0709 13:21:19.854886 (+   461us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:21:19.855613 (+   727us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:21:19.855657 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:21:19.882051 (+ 26394us) log.cc:844] Serialized 26 byte log entry
0709 13:21:20.770589 (+888538us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:21:20.770644 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:20.770747 (+   103us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:21:21.044003 (+273256us) raft_consensus.cc:1685] finished
0709 13:21:21.044023 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:21:21.044174 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":123,"raft.run_cpu_time_us":119,"raft.run_wall_time_us":120,"thread_start_us":888068,"threads_started":1,"wal-append.queue_time_us":1160117}
I20260709 13:21:21.047011 16226 raft_consensus.cc:2393] T e5ad4eb281564936a59f41e387a47156 P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 15: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
I20260709 13:21:21.051795 21319 leader_election.cc:290] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 19 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
W20260709 13:21:21.052253 21552 log.cc:927] Time spent T d16dfad0c3b540e5b4b86492ddf87208 P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 0.079s	user 0.000s	sys 0.000s
W20260709 13:21:21.007542 16292 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.768s	user 0.028s	sys 0.005s
I20260709 13:21:21.037688 21283 leader_election.cc:290] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
W20260709 13:21:21.058969 16790 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.837s	user 0.000s	sys 0.016s
W20260709 13:21:21.059598 16295 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.803s	user 0.013s	sys 0.003s
W20260709 13:21:21.069108 16295 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.812s	user 0.013s	sys 0.003s
W20260709 13:21:21.067296 16292 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.828s	user 0.028s	sys 0.005s
I20260709 13:21:21.040495 21508 raft_consensus.cc:3060] T 5a959dbecc2648eb85d1cc550f5a3245 P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:21:21.067080 16149 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.558s	user 0.033s	sys 0.084s
W20260709 13:21:21.069728 16149 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.561s	user 0.033s	sys 0.085s
W20260709 13:21:21.062144 16293 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.785s	user 0.000s	sys 0.015s
W20260709 13:21:20.919409 21403 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.661s	user 0.000s	sys 0.016s
W20260709 13:21:21.068732 16790 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.847s	user 0.000s	sys 0.016s
W20260709 13:21:21.070236 16293 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.819s	user 0.004s	sys 0.016s
W20260709 13:21:21.073304 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 897608us
I20260709 13:21:21.034158 21391 raft_consensus.cc:515] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:21.074400 16790 leader_election.cc:304] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:21:21.037482 21366 raft_consensus.cc:515] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:21.035899 21490 raft_consensus.cc:515] T 02838c86531d4f98bf58d1c22b8baf77 P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:21:21.072892 21403 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.821s	user 0.000s	sys 0.017s
W20260709 13:21:21.080382 16117 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.530s	user 0.004s	sys 0.011s
W20260709 13:21:21.061918 21424 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.788s	user 0.010s	sys 0.006s
I20260709 13:21:21.087708 16790 leader_election.cc:304] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 
W20260709 13:21:21.079879 16191 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.632s	user 0.006s	sys 0.011s
W20260709 13:21:21.088984 16191 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.641s	user 0.006s	sys 0.011s
I20260709 13:21:21.069252 21357 leader_election.cc:290] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
W20260709 13:21:21.084470 16117 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.534s	user 0.005s	sys 0.012s
W20260709 13:21:21.091100 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 643594us
I20260709 13:21:21.095028 16218 raft_consensus.cc:1240] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 14. Current term is 15. Ops: [14.11-14.11]
I20260709 13:21:21.096707 16225 raft_consensus.cc:1240] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 13. Current term is 14. Ops: []
I20260709 13:21:21.082381 21319 raft_consensus.cc:2749] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P cb82cc7bd6db46d980c75237a1402860 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:21:21.099627 16790 leader_election.cc:304] T ada5a004ba7c46819051e3ccec71ffdb P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 90f3da57796b488c9087a2f523adf045
W20260709 13:21:21.088265 21424 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.815s	user 0.010s	sys 0.006s
W20260709 13:21:21.103924 16115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 313791us
I20260709 13:21:21.101508 21512 raft_consensus.cc:2804] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 13:21:21.111217 16789 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.883s	user 0.100s	sys 0.105s
W20260709 13:21:21.111645 16789 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.890s	user 0.100s	sys 0.105s
I20260709 13:21:21.117566 16115 leader_election.cc:304] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:21:21.118404 16790 leader_election.cc:304] T 74ca8140f35c4ae29e8a664f6bdfc829 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:21:21.107502 21512 raft_consensus.cc:493] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:21.121606 16115 leader_election.cc:304] T 9c61a39fd4f84424b129c1a6f9ba404d P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
W20260709 13:21:21.122330 16143 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.677s	user 0.002s	sys 0.014s
W20260709 13:21:21.122803 16143 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.680s	user 0.002s	sys 0.014s
I20260709 13:21:21.110565 21319 raft_consensus.cc:2804] T ada5a004ba7c46819051e3ccec71ffdb P cb82cc7bd6db46d980c75237a1402860 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:21:21.114626 21337 leader_election.cc:290] T e5ad4eb281564936a59f41e387a47156 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 15 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:21.116858 21456 leader_election.cc:290] T 68a6838d82ef4cac83507f906139d90f P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:21:21.130955 16115 leader_election.cc:304] T 8d4ab870a05347868956aa8b841b2981 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:21:21.121680 21357 raft_consensus.cc:493] T 8cc959fb7f704fba9009c0595467d074 P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:21:21.124433 21097 raft_consensus.cc:2749] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:21:21.123725 21512 raft_consensus.cc:3060] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:21:21.146811 16219 raft_consensus.cc:1275] T 19baa7bc367844c288bca8f81439c413 P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 13:21:21.125852 21442 raft_consensus.cc:2804] T 2794af5b99984efebdee564035841cb0 P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:21:21.148732 16217 raft_consensus.cc:1240] T e5ad4eb281564936a59f41e387a47156 P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 14. Current term is 15. Ops: []
I20260709 13:21:21.150426 16115 leader_election.cc:304] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:21:21.151067 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62fe3346cd3c42c4a55e14af9073bca4" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:21:21.152263 16226 raft_consensus.cc:1275] T c0eab25ce4c64e018a041beeda2867db P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:21:21.124907 21319 raft_consensus.cc:493] T ada5a004ba7c46819051e3ccec71ffdb P cb82cc7bd6db46d980c75237a1402860 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:21:21.156260 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87ac0d1f7e8a419f9bcbc04aee0e067e" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:21.158805 16217 raft_consensus.cc:2468] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 14.
I20260709 13:21:21.157280 16222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4f65cdf00b748bb963ee6e81ef43b79" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:21.154354 21357 raft_consensus.cc:515] T 8cc959fb7f704fba9009c0595467d074 P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:21.145859 21468 consensus_queue.cc:1059] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.019s
I20260709 13:21:21.164378 16048 catalog_manager.cc:5697] T af57996a8be84d398f0f023e8d08ef77 P cb82cc7bd6db46d980c75237a1402860 reported cstate change: term changed from 9 to 15, leader changed from 647a343a542d4a139fe106c19095129b (127.14.177.129) to cb82cc7bd6db46d980c75237a1402860 (127.14.177.132). New cstate: current_term: 15 leader_uuid: "cb82cc7bd6db46d980c75237a1402860" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:21:21.168320 16048 catalog_manager.cc:5697] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 reported cstate change: term changed from 6 to 11, leader changed from 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130) to cb82cc7bd6db46d980c75237a1402860 (127.14.177.132). New cstate: current_term: 11 leader_uuid: "cb82cc7bd6db46d980c75237a1402860" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:21:21.174994 21357 leader_election.cc:290] T 8cc959fb7f704fba9009c0595467d074 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
W20260709 13:21:21.175561 21084 consensus_meta.cc:287] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac: Time spent flushing consensus metadata: real 0.730s	user 0.008s	sys 0.007s
I20260709 13:21:21.175725 21357 raft_consensus.cc:2749] T 9c61a39fd4f84424b129c1a6f9ba404d P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:21:21.176064 21084 raft_consensus.cc:515] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:21.176831 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7ca61592e848ea9b0fb81a2ce33cdc" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:21:21.166662 21543 consensus_queue.cc:1048] T 8158bbd17b994d2eb96f560bd0f1b88d P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.012s
W20260709 13:21:21.178493 21461 consensus_meta.cc:287] T c9944d5e4b8c4109af9955c4f08c9095 P cb82cc7bd6db46d980c75237a1402860: Time spent flushing consensus metadata: real 0.865s	user 0.006s	sys 0.006s
W20260709 13:21:21.179957 21545 log.cc:927] Time spent T 3884f864a96d428785895e5463e5728f P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.063s	user 0.000s	sys 0.000s
I20260709 13:21:21.179584 21461 raft_consensus.cc:515] T c9944d5e4b8c4109af9955c4f08c9095 P cb82cc7bd6db46d980c75237a1402860 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:21.178411 21556 raft_consensus.cc:2749] T 74ca8140f35c4ae29e8a664f6bdfc829 P cb82cc7bd6db46d980c75237a1402860 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:21:21.181056 16294 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40474 (request call id 903) took 1477 ms. Trace:
I20260709 13:21:21.181497 21391 leader_election.cc:290] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 15 election: Requested vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:21.183120 21461 leader_election.cc:290] T c9944d5e4b8c4109af9955c4f08c9095 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 16 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:21.178406 21084 leader_election.cc:290] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 19 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:21.181404 16294 rpcz_store.cc:276] 0709 13:21:19.704010 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:19.704122 (+   112us) service_pool.cc:224] Handling call
0709 13:21:19.704507 (+   385us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:21:19.704845 (+   338us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:21:19.704872 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:21:19.719542 (+ 14670us) log.cc:844] Serialized 26 byte log entry
0709 13:21:20.593322 (+873780us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:21:20.593381 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:20.593473 (+    92us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:21:21.180852 (+587379us) raft_consensus.cc:1685] finished
0709 13:21:21.180880 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:21:21.181000 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":604,"raft.run_cpu_time_us":132,"raft.run_wall_time_us":7511,"thread_start_us":785634,"threads_started":1,"wal-append.queue_time_us":1337958}
I20260709 13:21:21.186969 16147 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46224 (request call id 816) took 1668 ms. Trace:
I20260709 13:21:21.183337 21313 leader_election.cc:290] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 17 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:21.183386 16224 raft_consensus.cc:2368] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b for earlier term 18. Current term is 19.
I20260709 13:21:21.176450 21357 raft_consensus.cc:2749] T 8d4ab870a05347868956aa8b841b2981 P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:21:21.170269 16048 catalog_manager.cc:5697] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 reported cstate change: term changed from 5 to 12, leader changed from cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) to 647a343a542d4a139fe106c19095129b (127.14.177.129). New cstate: current_term: 12 leader_uuid: "647a343a542d4a139fe106c19095129b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:21:21.187604 16147 rpcz_store.cc:276] 0709 13:21:19.518361 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:20.342873 (+824512us) service_pool.cc:224] Handling call
0709 13:21:20.343332 (+   459us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:21:20.343830 (+   498us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:21:20.343882 (+    52us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:21:20.348400 (+  4518us) log.cc:844] Serialized 52 byte log entry
0709 13:21:21.040698 (+692298us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:21:21.044903 (+  4205us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:21.045011 (+   108us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:21:21.186735 (+141724us) raft_consensus.cc:1685] finished
0709 13:21:21.186756 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:21:21.186907 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8942,"raft.queue_time_us":134,"raft.run_cpu_time_us":159,"raft.run_wall_time_us":22888,"thread_start_us":603552,"threads_started":1,"wal-append.queue_time_us":757879}
I20260709 13:21:21.163803 21319 raft_consensus.cc:3060] T ada5a004ba7c46819051e3ccec71ffdb P cb82cc7bd6db46d980c75237a1402860 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:21:21.195865 16048 catalog_manager.cc:5697] T 8c11074579b046d99e585c2818e18a2c P cb82cc7bd6db46d980c75237a1402860 reported cstate change: term changed from 7 to 13, leader changed from 90f3da57796b488c9087a2f523adf045 (127.14.177.131) to 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130). New cstate: current_term: 13 leader_uuid: "1ca2725d290e47d68b1a4b609c9816ac" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:21:21.192739 16115 leader_election.cc:304] T 18ad7e3af5504571a69d10c00750a01c P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:21:21.203907 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02838c86531d4f98bf58d1c22b8baf77" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:21.204802 16217 raft_consensus.cc:2468] T 02838c86531d4f98bf58d1c22b8baf77 P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 13.
I20260709 13:21:21.165674 21456 consensus_queue.cc:1048] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.028s
I20260709 13:21:21.167841 21501 leader_election.cc:290] T 19baa7bc367844c288bca8f81439c413 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 17 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
W20260709 13:21:21.211066 16117 connection.cc:256] RPC call timeout handler was delayed by -1.343221819000064s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21330000000000002s for a response
I20260709 13:21:21.211486 16819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62fe3346cd3c42c4a55e14af9073bca4" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:21:21.228351 16816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada5a004ba7c46819051e3ccec71ffdb" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 15 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
W20260709 13:21:21.229035 16117 connection.cc:256] RPC call timeout handler was delayed by -1.3009218189999956s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21800000000000003s for a response
W20260709 13:21:21.229458 16117 connection.cc:256] RPC call timeout handler was delayed by -1.2775218189999578s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22060000000000002s for a response
I20260709 13:21:21.231338 16222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cc959fb7f704fba9009c0595467d074" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:21.193852 21468 raft_consensus.cc:3055] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [term 16 LEADER]: Stepping down as leader of term 16
W20260709 13:21:21.237146 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 432500us
I20260709 13:21:21.205154 21357 raft_consensus.cc:2749] T 18ad7e3af5504571a69d10c00750a01c P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:21:21.228713 16819 raft_consensus.cc:2393] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 19: Already voted for candidate cb82cc7bd6db46d980c75237a1402860 in this term.
I20260709 13:21:21.198460 16048 catalog_manager.cc:5697] T 8638ee363273426db1393dd81e5c7b72 P cb82cc7bd6db46d980c75237a1402860 reported cstate change: term changed from 5 to 15, leader changed from 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130) to cb82cc7bd6db46d980c75237a1402860 (127.14.177.132). New cstate: current_term: 15 leader_uuid: "cb82cc7bd6db46d980c75237a1402860" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:21:21.244731 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 126456us
I20260709 13:21:21.197469 21543 consensus_queue.cc:1048] T e5ad4eb281564936a59f41e387a47156 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.042s
I20260709 13:21:21.206125 21087 raft_consensus.cc:493] T c0eab25ce4c64e018a041beeda2867db P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:21.227492 16191 leader_election.cc:304] T af57996a8be84d398f0f023e8d08ef77 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
I20260709 13:21:21.242590 16820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66ae176961b74813a58bc15c9c4c6f2a" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:21.245908 16048 catalog_manager.cc:5697] T 9e65676b21b748728ad39668cfe722ec P cb82cc7bd6db46d980c75237a1402860 reported cstate change: term changed from 10 to 13, leader changed from 90f3da57796b488c9087a2f523adf045 (127.14.177.131) to cb82cc7bd6db46d980c75237a1402860 (127.14.177.132). New cstate: current_term: 13 leader_uuid: "cb82cc7bd6db46d980c75237a1402860" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:21:21.201380 16115 leader_election.cc:304] T 84f2da98864b4425bf7c0055951fdd4b P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:21:21.224997 21567 raft_consensus.cc:493] T 1df93779209748b9860f6d893dd3de53 P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:21:21.252045 16816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74ca8140f35c4ae29e8a664f6bdfc829" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
W20260709 13:21:21.252746 16298 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.049s	user 0.011s	sys 0.084s
I20260709 13:21:21.257977 16142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
W20260709 13:21:21.261586 16298 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.058s	user 0.011s	sys 0.084s
I20260709 13:21:21.261760 16142 raft_consensus.cc:3060] T ecc1e363230043fe9fef390f0b979f20 P 647a343a542d4a139fe106c19095129b [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:21:21.255456 16265 leader_election.cc:304] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:21:21.266415 16192 leader_election.cc:304] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860
I20260709 13:21:21.266825 16146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "829b4ad8c5a54eababeea1181b960d4b" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:21.268949 16292 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48684 (request call id 2803) took 1088 ms. Trace:
I20260709 13:21:21.236852 21468 raft_consensus.cc:740] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
W20260709 13:21:21.270645 16145 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.598s	user 0.020s	sys 0.020s
W20260709 13:21:21.271232 16145 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.599s	user 0.021s	sys 0.020s
W20260709 13:21:21.273841 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 46461.6us
W20260709 13:21:21.276907 16115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 75645.3us
W20260709 13:21:21.236449 21441 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.545s	user 0.016s	sys 0.009s
I20260709 13:21:21.269630 16292 rpcz_store.cc:276] 0709 13:21:20.180840 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:20.182911 (+  2071us) service_pool.cc:224] Handling call
0709 13:21:20.183239 (+   328us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:21:20.183454 (+   215us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:21:21.069945 (+886491us) log.cc:844] Serialized 14 byte log entry
0709 13:21:21.267910 (+197965us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:21:21.268444 (+   534us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:21:21.268542 (+    98us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:21.268645 (+   103us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:21:21.268865 (+   220us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":41979,"prepare.queue_time_us":980959,"prepare.run_cpu_time_us":34,"prepare.run_wall_time_us":35,"thread_start_us":1005913,"threads_started":2}
I20260709 13:21:21.261567 16048 catalog_manager.cc:5697] T 3a7ca61592e848ea9b0fb81a2ce33cdc P cb82cc7bd6db46d980c75237a1402860 reported cstate change: term changed from 9 to 15. New cstate: current_term: 15 leader_uuid: "1ca2725d290e47d68b1a4b609c9816ac" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:21:21.300321 16151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26354361e9b044cea1c2b86c4882fe92" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:21.299896 16148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2372981006d4b218215c28964dea623" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
W20260709 13:21:21.305958 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 75279.5us
I20260709 13:21:21.362715 16790 leader_election.cc:304] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:21:21.383375 16821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3884f864a96d428785895e5463e5728f" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:21.300971 21497 raft_consensus.cc:2749] T af57996a8be84d398f0f023e8d08ef77 P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:21:21.393649 16151 raft_consensus.cc:2468] T 26354361e9b044cea1c2b86c4882fe92 P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 12.
I20260709 13:21:21.265066 21401 consensus_queue.cc:1048] T ee9ed67006144797b1de6ea8c2013a68 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.119s
I20260709 13:21:21.338333 21182 raft_consensus.cc:2749] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:21:21.357029 16825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62fe3346cd3c42c4a55e14af9073bca4" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:21:21.355968 16819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57a744f5aaf944678a63469cbf514015" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:21.336901 21391 raft_consensus.cc:493] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:21:21.399114 16142 raft_consensus.cc:2468] T ecc1e363230043fe9fef390f0b979f20 P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 15.
I20260709 13:21:21.395690 16151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62fe3346cd3c42c4a55e14af9073bca4" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:21:21.401082 16151 raft_consensus.cc:3060] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:21:21.330788 16192 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 64457.5us
I20260709 13:21:21.402211 16148 raft_consensus.cc:2410] T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 for term 16 because replica has last-logged OpId of term: 15 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 10.
I20260709 13:21:21.402324 16825 raft_consensus.cc:2368] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b for earlier term 18. Current term is 19.
I20260709 13:21:21.330642 16146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:21:21.253022 21366 leader_election.cc:290] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 14 election: Requested vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:21.247726 21490 leader_election.cc:290] T 02838c86531d4f98bf58d1c22b8baf77 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:21.339948 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:21:21.414966 16297 raft_consensus.cc:3060] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:21:21.331946 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 55958us
I20260709 13:21:21.346594 16294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9944d5e4b8c4109af9955c4f08c9095" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:21:21.351630 16291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdbd78acb4cf46298e0de8ba0d130e19" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:21:21.338967 16816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87147731a68400e8cce9b879daa7c5d" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
W20260709 13:21:21.315884 16266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 144109us
W20260709 13:21:21.353688 16117 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 123517us
W20260709 13:21:21.291821 21441 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.621s	user 0.017s	sys 0.010s
I20260709 13:21:21.401695 16048 catalog_manager.cc:5697] T 806d7690c2b644959152074bb2cab0a7 P cb82cc7bd6db46d980c75237a1402860 reported cstate change: term changed from 6 to 12. New cstate: current_term: 12 leader_uuid: "1ca2725d290e47d68b1a4b609c9816ac" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } } }
I20260709 13:21:21.409387 16790 leader_election.cc:304] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860; no voters: 90f3da57796b488c9087a2f523adf045
I20260709 13:21:21.410037 16825 raft_consensus.cc:1240] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 [term 17 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 16. Current term is 17. Ops: []
I20260709 13:21:21.433441 16790 leader_election.cc:304] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:21:21.433058 16048 catalog_manager.cc:5697] T 74ca8140f35c4ae29e8a664f6bdfc829 P cb82cc7bd6db46d980c75237a1402860 reported cstate change: term changed from 10 to 13, leader changed from cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) to 90f3da57796b488c9087a2f523adf045 (127.14.177.131). New cstate: current_term: 13 leader_uuid: "90f3da57796b488c9087a2f523adf045" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:21:21.286644 21567 raft_consensus.cc:515] T 1df93779209748b9860f6d893dd3de53 P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:21.387295 21424 raft_consensus.cc:2749] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:21:21.271605 21087 raft_consensus.cc:515] T c0eab25ce4c64e018a041beeda2867db P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:21.417024 16191 leader_election.cc:304] T 19baa7bc367844c288bca8f81439c413 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
I20260709 13:21:21.445389 16117 leader_election.cc:304] T 746aabf856ab401c9ef5e859371e7b67 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860
I20260709 13:21:21.446074 16790 leader_election.cc:304] T 26354361e9b044cea1c2b86c4882fe92 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860; no voters: 90f3da57796b488c9087a2f523adf045
I20260709 13:21:21.443738 21366 consensus_queue.cc:1059] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: INVALID_TERM, Last received: 16.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.007s
I20260709 13:21:21.404783 21507 raft_consensus.cc:493] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:21:21.452507 16117 leader_election.cc:304] T af57996a8be84d398f0f023e8d08ef77 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860
I20260709 13:21:21.453452 16790 leader_election.cc:304] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:21:21.455334 16151 raft_consensus.cc:2468] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 19.
I20260709 13:21:21.338181 21084 consensus_queue.cc:1048] T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.106s
I20260709 13:21:21.434834 16048 catalog_manager.cc:5697] T 23f82224f4564fea93572e8206bad1fe P cb82cc7bd6db46d980c75237a1402860 reported cstate change: term changed from 8 to 12. New cstate: current_term: 12 leader_uuid: "cb82cc7bd6db46d980c75237a1402860" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:21:21.439679 21560 raft_consensus.cc:2749] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:21:21.392135 21571 consensus_queue.cc:1048] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.062s
I20260709 13:21:21.471307 16790 leader_election.cc:304] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:21:21.428351 21391 raft_consensus.cc:515] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:21.473644 16048 catalog_manager.cc:5697] T 3884f864a96d428785895e5463e5728f P cb82cc7bd6db46d980c75237a1402860 reported cstate change: term changed from 7 to 8, leader changed from cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) to 647a343a542d4a139fe106c19095129b (127.14.177.129). New cstate: current_term: 8 leader_uuid: "647a343a542d4a139fe106c19095129b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:21:21.482877 16297 raft_consensus.cc:2468] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 17.
I20260709 13:21:21.460642 21568 raft_consensus.cc:2749] T 84f2da98864b4425bf7c0055951fdd4b P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:21:21.451531 21500 raft_consensus.cc:493] T d16dfad0c3b540e5b4b86492ddf87208 P 647a343a542d4a139fe106c19095129b [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:21:21.491142 16048 catalog_manager.cc:5697] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 reported cstate change: term changed from 9 to 14, leader changed from 647a343a542d4a139fe106c19095129b (127.14.177.129) to cb82cc7bd6db46d980c75237a1402860 (127.14.177.132). New cstate: current_term: 14 leader_uuid: "cb82cc7bd6db46d980c75237a1402860" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:21:21.476191 21530 consensus_meta.cc:287] T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860: Time spent flushing consensus metadata: real 0.932s	user 0.015s	sys 0.000s
I20260709 13:21:21.462159 21456 raft_consensus.cc:493] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:21.442543 21403 consensus_queue.cc:1048] T 434bff3082d141838068f323bdb0d1ed P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.033s
I20260709 13:21:21.483361 16298 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48684 (request call id 2801) took 1303 ms. Trace:
I20260709 13:21:21.503991 16298 rpcz_store.cc:276] 0709 13:21:20.180260 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:20.181927 (+  1667us) service_pool.cc:224] Handling call
0709 13:21:20.182205 (+   278us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:21:20.182415 (+   210us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:21:21.262361 (+1079946us) log.cc:844] Serialized 14 byte log entry
0709 13:21:21.482983 (+220622us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:21:21.483038 (+    55us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:21:21.483090 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:21.483187 (+    97us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:21:21.483307 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":946754,"prepare.run_cpu_time_us":67,"prepare.run_wall_time_us":3532,"raft.queue_time_us":106244,"raft.run_cpu_time_us":152,"raft.run_wall_time_us":4101,"spinlock_wait_cycles":29613952,"thread_start_us":1277120,"threads_started":3,"wal-append.queue_time_us":182255}
I20260709 13:21:21.495147 16048 catalog_manager.cc:5697] T c9944d5e4b8c4109af9955c4f08c9095 P cb82cc7bd6db46d980c75237a1402860 reported cstate change: term changed from 7 to 13, leader changed from cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) to 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130). New cstate: current_term: 13 leader_uuid: "1ca2725d290e47d68b1a4b609c9816ac" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:21:21.448851 21543 consensus_queue.cc:1059] T e5ad4eb281564936a59f41e387a47156 P 647a343a542d4a139fe106c19095129b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.031s
I20260709 13:21:21.508392 16048 catalog_manager.cc:5697] T ee9ed67006144797b1de6ea8c2013a68 P cb82cc7bd6db46d980c75237a1402860 reported cstate change: term changed from 8 to 11, leader changed from 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130) to cb82cc7bd6db46d980c75237a1402860 (127.14.177.132). New cstate: current_term: 11 leader_uuid: "cb82cc7bd6db46d980c75237a1402860" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:21:21.434211 21468 consensus_queue.cc:260] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.13, Last appended by leader: 13, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:21.518718 16048 catalog_manager.cc:5697] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860 reported cstate change: term changed from 7 to 15, leader changed from 647a343a542d4a139fe106c19095129b (127.14.177.129) to cb82cc7bd6db46d980c75237a1402860 (127.14.177.132). New cstate: current_term: 15 leader_uuid: "cb82cc7bd6db46d980c75237a1402860" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:21:21.492663 21353 raft_consensus.cc:2764] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Leader pre-election decision vote started in defunct term 16: won
I20260709 13:21:21.467211 21507 raft_consensus.cc:515] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:21.526165 16048 catalog_manager.cc:5697] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P cb82cc7bd6db46d980c75237a1402860 reported cstate change: term changed from 10 to 12. New cstate: current_term: 12 leader_uuid: "cb82cc7bd6db46d980c75237a1402860" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:21:21.497110 21489 consensus_queue.cc:1048] T c0eab25ce4c64e018a041beeda2867db P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.047s
W20260709 13:21:21.520450 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 103697us
I20260709 13:21:21.496834 21401 raft_consensus.cc:2804] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:21:21.498270 21500 raft_consensus.cc:515] T d16dfad0c3b540e5b4b86492ddf87208 P 647a343a542d4a139fe106c19095129b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:21:21.532341 21575 log.cc:927] Time spent T fffab26427e24d398e2695aad40ca134 P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 0.097s	user 0.000s	sys 0.000s
W20260709 13:21:21.533900 21574 log.cc:927] Time spent T d16dfad0c3b540e5b4b86492ddf87208 P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.066s	user 0.000s	sys 0.000s
I20260709 13:21:21.576440 21501 raft_consensus.cc:2804] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:21:21.517422 21456 raft_consensus.cc:515] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:21.534046 21497 raft_consensus.cc:493] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:21:21.535044 21084 raft_consensus.cc:493] T ee9ed67006144797b1de6ea8c2013a68 P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:21:21.535557 21559 log.cc:927] Time spent T a378d7f796f94615b3b347119a9880fd P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.206s	user 0.000s	sys 0.001s
W20260709 13:21:21.528198 21523 consensus_meta.cc:287] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860: Time spent flushing consensus metadata: real 1.080s	user 0.013s	sys 0.003s
W20260709 13:21:21.576467 16117 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 114403us
I20260709 13:21:21.586139 21569 consensus_queue.cc:1048] T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.140s
I20260709 13:21:21.498448 21182 raft_consensus.cc:2749] T 19baa7bc367844c288bca8f81439c413 P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:21:21.502414 21530 raft_consensus.cc:515] T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:21.656647 21468 raft_consensus.cc:3060] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:21:21.661568 16048 catalog_manager.cc:5697] T 2794af5b99984efebdee564035841cb0 P cb82cc7bd6db46d980c75237a1402860 reported cstate change: term changed from 8 to 10, leader changed from cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) to 647a343a542d4a139fe106c19095129b (127.14.177.129). New cstate: current_term: 10 leader_uuid: "647a343a542d4a139fe106c19095129b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
W20260709 13:21:21.585861 21563 log.cc:927] Time spent T 2794af5b99984efebdee564035841cb0 P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 0.326s	user 0.000s	sys 0.001s
W20260709 13:21:21.643584 16266 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:21:21.751338 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 352536us
I20260709 13:21:21.752765 16291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6f07842d3aa47858df5665874c86f3a" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:21.753392 16116 leader_election.cc:304] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
I20260709 13:21:21.678412 16191 leader_election.cc:304] T e4dc6b0541ca431ebc757beac00521f1 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 90f3da57796b488c9087a2f523adf045
W20260709 13:21:21.754966 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 76651.2us
I20260709 13:21:21.755488 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5ad4eb281564936a59f41e387a47156" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:21:21.529670 21585 raft_consensus.cc:2749] T af57996a8be84d398f0f023e8d08ef77 P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:21:21.526772 21568 raft_consensus.cc:2749] T 746aabf856ab401c9ef5e859371e7b67 P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:21:21.480235 21441 consensus_queue.cc:1048] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.024s
I20260709 13:21:21.615360 16145 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "02838c86531d4f98bf58d1c22b8baf77"
dest_uuid: "647a343a542d4a139fe106c19095129b"
mode: GRACEFUL
new_leader_uuid: "90f3da57796b488c9087a2f523adf045"
 from {username='slave'} at 127.0.0.1:40290
I20260709 13:21:21.658996 21313 raft_consensus.cc:2804] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860 [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 13:21:21.725672 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 470275us
I20260709 13:21:21.780524 16292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada5a004ba7c46819051e3ccec71ffdb" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 15 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:21:21.780676 16265 leader_election.cc:304] T a378d7f796f94615b3b347119a9880fd P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860
I20260709 13:21:21.737108 16117 leader_election.cc:304] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860; no voters: 90f3da57796b488c9087a2f523adf045
W20260709 13:21:21.584074 16115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 304951us
I20260709 13:21:21.602255 21319 raft_consensus.cc:515] T ada5a004ba7c46819051e3ccec71ffdb P cb82cc7bd6db46d980c75237a1402860 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:21.780702 21182 raft_consensus.cc:2804] T e4dc6b0541ca431ebc757beac00521f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:21:21.794895 21182 raft_consensus.cc:493] T e4dc6b0541ca431ebc757beac00521f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:21.795293 21182 raft_consensus.cc:3060] T e4dc6b0541ca431ebc757beac00521f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:21:21.797997 21609 raft_consensus.cc:493] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:21.680028 21515 raft_consensus.cc:493] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:21.803555 21515 raft_consensus.cc:515] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:21.805800 21515 leader_election.cc:290] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:21.688685 21401 raft_consensus.cc:697] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860 [term 15 LEADER]: Becoming Leader. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:21:21.806752 16295 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48684 (request call id 2800) took 1626 ms. Trace:
I20260709 13:21:21.747411 16048 catalog_manager.cc:5697] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860 reported cstate change: term changed from 10 to 12, leader changed from cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) to 647a343a542d4a139fe106c19095129b (127.14.177.129). New cstate: current_term: 12 leader_uuid: "647a343a542d4a139fe106c19095129b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:21:21.756100 16293 raft_consensus.cc:3060] T e5ad4eb281564936a59f41e387a47156 P 90f3da57796b488c9087a2f523adf045 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:21:21.828504 16300 raft_consensus.cc:1275] T c0eab25ce4c64e018a041beeda2867db P 90f3da57796b488c9087a2f523adf045 [term 14 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:21:21.828154 16294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0eab25ce4c64e018a041beeda2867db" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:21.829715 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87ac0d1f7e8a419f9bcbc04aee0e067e" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:21.756134 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806d7690c2b644959152074bb2cab0a7" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:21.756289 16191 leader_election.cc:304] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:21:21.828423 16295 rpcz_store.cc:276] 0709 13:21:20.179956 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:20.180821 (+   865us) service_pool.cc:224] Handling call
0709 13:21:20.181126 (+   305us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:21:20.181516 (+   390us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:21:20.181553 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:21:20.192431 (+ 10878us) log.cc:844] Serialized 26 byte log entry
0709 13:21:21.070810 (+878379us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:21:21.070867 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:21.070959 (+    92us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:21:21.806535 (+735576us) raft_consensus.cc:1685] finished
0709 13:21:21.806557 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:21:21.806691 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1438,"raft.run_cpu_time_us":138,"raft.run_wall_time_us":10559,"thread_start_us":813663,"threads_started":1,"wal-append.queue_time_us":1003204}
I20260709 13:21:21.766018 16291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74ca8140f35c4ae29e8a664f6bdfc829" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:21:21.776058 16145 raft_consensus.cc:606] T 02838c86531d4f98bf58d1c22b8baf77 P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Received request to transfer leadership to TS 90f3da57796b488c9087a2f523adf045
I20260709 13:21:21.777546 16149 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40830 (request call id 1698) took 2070 ms. Trace:
I20260709 13:21:21.792562 21593 raft_consensus.cc:2749] T a378d7f796f94615b3b347119a9880fd P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:21:21.792927 21597 log.cc:927] Time spent T 3884f864a96d428785895e5463e5728f P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 0.107s	user 0.000s	sys 0.000s
I20260709 13:21:21.792896 16265 leader_election.cc:304] T 8638ee363273426db1393dd81e5c7b72 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860
W20260709 13:21:21.842868 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 50059.6us
I20260709 13:21:21.843201 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02838c86531d4f98bf58d1c22b8baf77" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:21.844283 16265 leader_election.cc:304] T c9944d5e4b8c4109af9955c4f08c9095 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860
I20260709 13:21:21.807024 21401 consensus_queue.cc:237] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.9, Last appended by leader: 9, Current term: 15, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:21.680096 21586 raft_consensus.cc:3055] T e5ad4eb281564936a59f41e387a47156 P 647a343a542d4a139fe106c19095129b [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:21:21.808384 16143 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46224 (request call id 828) took 2031 ms. Trace:
I20260709 13:21:21.726495 21497 raft_consensus.cc:515] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:21.735993 21084 raft_consensus.cc:515] T ee9ed67006144797b1de6ea8c2013a68 P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:21:21.827857 16117 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 90871.6us
W20260709 13:21:21.832368 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 76154.9us
I20260709 13:21:21.832407 16295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cc959fb7f704fba9009c0595467d074" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:21.862820 16117 leader_election.cc:304] T 3884f864a96d428785895e5463e5728f P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860
I20260709 13:21:21.863149 16191 leader_election.cc:304] T 66f75124215f4ca6bef831e174a792d8 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
I20260709 13:21:21.833283 16145 raft_consensus.cc:614] T 02838c86531d4f98bf58d1c22b8baf77 P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:21:21.833849 16300 raft_consensus.cc:2468] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 15.
I20260709 13:21:21.830127 16294 raft_consensus.cc:2468] T c0eab25ce4c64e018a041beeda2867db P 90f3da57796b488c9087a2f523adf045 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 14.
I20260709 13:21:21.838675 16296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4f65cdf00b748bb963ee6e81ef43b79" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:21.833397 16149 rpcz_store.cc:276] 0709 13:21:19.706680 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:20.344837 (+638157us) service_pool.cc:224] Handling call
0709 13:21:20.345135 (+   298us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:21:20.444942 (+ 99807us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:21:21.070038 (+625096us) log.cc:844] Serialized 14 byte log entry
0709 13:21:21.745926 (+675888us) log.cc:844] Serialized 14 byte log entry
0709 13:21:21.777068 (+ 31142us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:21:21.777155 (+    87us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:21:21.777232 (+    77us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:21.777325 (+    93us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:21:21.777474 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":97950,"prepare.queue_time_us":669930,"prepare.run_cpu_time_us":101,"prepare.run_wall_time_us":4375,"raft.queue_time_us":228977,"raft.run_cpu_time_us":262,"raft.run_wall_time_us":8106,"spinlock_wait_cycles":12747136,"thread_start_us":1204038,"threads_started":3,"wal-append.queue_time_us":395387}
W20260709 13:21:21.811869 21508 consensus_meta.cc:287] T 5a959dbecc2648eb85d1cc550f5a3245 P 647a343a542d4a139fe106c19095129b: Time spent flushing consensus metadata: real 0.739s	user 0.005s	sys 0.008s
I20260709 13:21:21.792749 21456 leader_election.cc:290] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:21:21.869148 16150 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40844 (request call id 2580) took 1254 ms. Trace:
I20260709 13:21:21.841296 16817 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49906 (request call id 1068) took 1323 ms. Trace:
I20260709 13:21:21.746906 21087 leader_election.cc:290] T c0eab25ce4c64e018a041beeda2867db P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:21.780015 21097 raft_consensus.cc:2749] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:21:21.748378 21424 raft_consensus.cc:2804] T 26354361e9b044cea1c2b86c4882fe92 P cb82cc7bd6db46d980c75237a1402860 [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 13:21:21.854844 16822 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49906 (request call id 1057) took 2642 ms (client timeout 2001 ms). Trace:
W20260709 13:21:21.888336 16822 rpcz_store.cc:269] 0709 13:21:19.195511 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:19.195672 (+   161us) service_pool.cc:224] Handling call
0709 13:21:21.837657 (+2641985us) spinlock_profiling.cc:243] Waited 1.58 s on lock 0x7b540054bad4. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:21:21.837812 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3636820352}
I20260709 13:21:21.836939 21515 raft_consensus.cc:2749] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:21:21.869561 16150 rpcz_store.cc:276] 0709 13:21:20.615239 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:20.659908 (+ 44669us) service_pool.cc:224] Handling call
0709 13:21:21.868964 (+1209056us) spinlock_profiling.cc:243] Waited 85.5 ms on lock 0x7b540028e454. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:21:21.869080 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":196575616}
I20260709 13:21:21.893136 16117 leader_election.cc:400] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 18 election: Vote denied by peer cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923) with higher term. Message: Invalid argument: T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b for earlier term 18. Current term is 19.
I20260709 13:21:21.869566 16191 leader_election.cc:304] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: cb82cc7bd6db46d980c75237a1402860
I20260709 13:21:21.869736 16817 rpcz_store.cc:276] 0709 13:21:20.517915 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:20.542197 (+ 24282us) service_pool.cc:224] Handling call
0709 13:21:20.542632 (+   435us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:21:20.778172 (+235540us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:21.839041 (+1060869us) spinlock_profiling.cc:243] Waited 235 ms on lock 0x7b54006441d4. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:21:21.841193 (+  2152us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":539799296}
I20260709 13:21:21.858228 16143 rpcz_store.cc:276] 0709 13:21:19.776743 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:20.351524 (+574781us) service_pool.cc:224] Handling call
0709 13:21:20.351836 (+   312us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:21:20.377926 (+ 26090us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:21:20.377957 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:21:20.378465 (+   508us) log.cc:844] Serialized 26 byte log entry
0709 13:21:21.123354 (+744889us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:21:21.123400 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:21.123478 (+    78us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:21:21.808168 (+684690us) raft_consensus.cc:1685] finished
0709 13:21:21.808190 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:21:21.808325 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":14948,"raft.queue_time_us":773,"raft.run_cpu_time_us":125,"raft.run_wall_time_us":12556,"thread_start_us":680610,"threads_started":1,"wal-append.queue_time_us":782960}
I20260709 13:21:21.893493 16117 leader_election.cc:403] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 18 election: Cancelling election due to peer responding with higher term
I20260709 13:21:21.847870 16265 leader_election.cc:304] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860
I20260709 13:21:21.815620 21391 leader_election.cc:290] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
W20260709 13:21:21.854667 21512 consensus_meta.cc:287] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860: Time spent flushing consensus metadata: real 0.681s	user 0.012s	sys 0.000s
I20260709 13:21:21.904276 21512 raft_consensus.cc:515] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:21.890674 21366 raft_consensus.cc:2749] T 66f75124215f4ca6bef831e174a792d8 P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:21:21.808602 21530 leader_election.cc:290] T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 16 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:21:21.915902 21543 raft_consensus.cc:2749] T 3884f864a96d428785895e5463e5728f P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:21:21.916728 21476 raft_consensus.cc:3060] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:21:21.922065 21442 raft_consensus.cc:493] T 2794af5b99984efebdee564035841cb0 P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Starting leader election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:21:21.922813 21442 raft_consensus.cc:3060] T 2794af5b99984efebdee564035841cb0 P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:21:21.925171 21324 raft_consensus.cc:3055] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:21:21.809458 21569 raft_consensus.cc:2804] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:21:21.798674 21609 raft_consensus.cc:515] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:21.741163 21523 raft_consensus.cc:515] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:21:21.886821 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 133501us
I20260709 13:21:21.845295 21501 raft_consensus.cc:697] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [term 19 LEADER]: Becoming Leader. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:21:21.947407 16190 leader_election.cc:304] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860
I20260709 13:21:21.814988 21182 raft_consensus.cc:515] T e4dc6b0541ca431ebc757beac00521f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:21.870165 16294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e4bfb9090c8460eb14fa7e0d818e7f1" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
W20260709 13:21:21.895480 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 47670us
I20260709 13:21:21.951364 16265 leader_election.cc:304] T ecc1e363230043fe9fef390f0b979f20 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:21:21.952610 16265 leader_election.cc:304] T cdbd78acb4cf46298e0de8ba0d130e19 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860
I20260709 13:21:21.859860 21595 raft_consensus.cc:2749] T 8638ee363273426db1393dd81e5c7b72 P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:21:21.897300 21608 raft_consensus.cc:2749] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:21:21.899365 21544 log.cc:927] Time spent T 2794af5b99984efebdee564035841cb0 P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 1.071s	user 0.000s	sys 0.007s
I20260709 13:21:21.903113 16823 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49892 (request call id 786) took 2056 ms. Trace:
I20260709 13:21:21.857565 21586 raft_consensus.cc:740] T e5ad4eb281564936a59f41e387a47156 P 647a343a542d4a139fe106c19095129b [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:21:21.936245 16145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806d7690c2b644959152074bb2cab0a7" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:21.934397 21391 raft_consensus.cc:2764] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Leader pre-election decision vote started in defunct term 15: won
I20260709 13:21:21.872360 21158 consensus_queue.cc:1048] T 18ad7e3af5504571a69d10c00750a01c P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.084s
W20260709 13:21:21.926604 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 57124.4us
I20260709 13:21:21.868171 21572 raft_consensus.cc:2749] T c9944d5e4b8c4109af9955c4f08c9095 P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:21:21.808902 21357 consensus_queue.cc:1048] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.081s
I20260709 13:21:21.946026 16116 leader_election.cc:304] T 02838c86531d4f98bf58d1c22b8baf77 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
I20260709 13:21:21.945353 21497 leader_election.cc:290] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:21.948886 21490 raft_consensus.cc:2749] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Leader election lost for term 18. Reason: Vote denied by peer cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923) with higher term. Message: Invalid argument: T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b for earlier term 18. Current term is 19.
W20260709 13:21:22.020284 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 52339.7us
I20260709 13:21:21.892155 21585 raft_consensus.cc:2804] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:21:22.022104 16145 raft_consensus.cc:2468] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 14.
W20260709 13:21:22.023347 16115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 92977.1us
I20260709 13:21:21.925671 21324 raft_consensus.cc:740] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:21:21.927790 16117 leader_election.cc:304] T 57a744f5aaf944678a63469cbf514015 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860
I20260709 13:21:21.960803 16823 rpcz_store.cc:276] 0709 13:21:19.846645 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:19.846759 (+   114us) service_pool.cc:224] Handling call
0709 13:21:21.837863 (+1991104us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:21:21.842603 (+  4740us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:21:21.842633 (+    30us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:21:21.842672 (+    39us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:21:21.842708 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:21.842767 (+    59us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:21:21.902872 (+ 60105us) spinlock_profiling.cc:243] Waited 1.99 s on lock 0x7b540054bad0. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5e8ad 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:21:21.903026 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":12979,"raft.queue_time_us":31200,"raft.run_cpu_time_us":145,"raft.run_wall_time_us":16877,"spinlock_wait_cycles":4578428160}
I20260709 13:21:21.956396 21601 raft_consensus.cc:2749] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:21:21.934659 21570 raft_consensus.cc:493] T ee9ed67006144797b1de6ea8c2013a68 P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
W20260709 13:21:21.957990 21561 log.cc:927] Time spent T 18ad7e3af5504571a69d10c00750a01c P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.072s	user 0.001s	sys 0.000s
I20260709 13:21:22.026254 16191 leader_election.cc:304] T c2f5f89186204242bf377f7c42b2e2da P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 90f3da57796b488c9087a2f523adf045
W20260709 13:21:22.019523 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 59857.3us
W20260709 13:21:22.209174 16115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 168002us
W20260709 13:21:22.209542 16117 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 281853us
I20260709 13:21:21.956773 16144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6f07842d3aa47858df5665874c86f3a" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
W20260709 13:21:22.042297 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 96351.1us
I20260709 13:21:22.041713 16190 leader_election.cc:304] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860
I20260709 13:21:21.886340 21508 raft_consensus.cc:515] T 5a959dbecc2648eb85d1cc550f5a3245 P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:22.019055 21593 raft_consensus.cc:2749] T cdbd78acb4cf46298e0de8ba0d130e19 P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:21:21.957305 21507 leader_election.cc:290] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:22.050699 21568 raft_consensus.cc:2804] T 02838c86531d4f98bf58d1c22b8baf77 P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 13:21:22.052989 16789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 127586us
I20260709 13:21:22.049319 21569 raft_consensus.cc:697] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b [term 17 LEADER]: Becoming Leader. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:21:22.048805 21585 raft_consensus.cc:493] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:22.050294 21319 leader_election.cc:290] T ada5a004ba7c46819051e3ccec71ffdb P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 15 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:22.041425 21324 consensus_queue.cc:260] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:22.020025 21456 raft_consensus.cc:2804] T ecc1e363230043fe9fef390f0b979f20 P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:21:22.025070 21582 consensus_queue.cc:1059] T ada5a004ba7c46819051e3ccec71ffdb P 90f3da57796b488c9087a2f523adf045 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.150s
W20260709 13:21:22.222674 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 181115us
I20260709 13:21:22.213630 16299 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40474 (request call id 896) took 2535 ms. Trace:
I20260709 13:21:22.317054 16299 rpcz_store.cc:276] 0709 13:21:19.677882 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:19.678019 (+   137us) service_pool.cc:224] Handling call
0709 13:21:19.678525 (+   506us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:21:19.678920 (+   395us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:21:19.678946 (+    26us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:21:19.679520 (+   574us) log.cc:844] Serialized 26 byte log entry
0709 13:21:20.510113 (+830593us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:21:20.510153 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:20.510226 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:21:21.898650 (+1388424us) spinlock_profiling.cc:243] Waited 68.7 ms on lock 0x7b4400022b4c. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69ad8fadb1 00007f69ad998c85 00007f69ad9996a2 00007f69b070b1db 00007f69b0706aaf 00007f69b06fa86e 00007f69b06f9bb6 00007f69b06f97d8 00007f69b0703221 00007f69b07031d9 00007f69b0703169
0709 13:21:22.191446 (+292796us) raft_consensus.cc:1685] finished
0709 13:21:22.191478 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:21:22.213535 (+ 22057us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":3581,"raft.run_cpu_time_us":158,"raft.run_wall_time_us":6970,"spinlock_wait_cycles":158103808,"thread_start_us":830188,"threads_started":1,"wal-append.queue_time_us":1108318}
I20260709 13:21:22.316748 16190 leader_election.cc:304] T c0eab25ce4c64e018a041beeda2867db P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:21:22.304953 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada5a004ba7c46819051e3ccec71ffdb" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:21:22.298115 21567 leader_election.cc:290] T 1df93779209748b9860f6d893dd3de53 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:22.351899 16817 raft_consensus.cc:1240] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860 [term 13 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 12. Current term is 13. Ops: []
I20260709 13:21:22.307754 21543 raft_consensus.cc:2749] T 57a744f5aaf944678a63469cbf514015 P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:21:22.295432 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 269303us
I20260709 13:21:22.295724 16225 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57600 (request call id 1677) took 3157 ms. Trace:
W20260709 13:21:22.309489 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 97080.2us
I20260709 13:21:22.366422 16191 leader_election.cc:304] T 68a6838d82ef4cac83507f906139d90f P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:21:22.295238 21601 raft_consensus.cc:2804] T c2f5f89186204242bf377f7c42b2e2da P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:21:22.293541 21497 raft_consensus.cc:2749] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:21:22.336422 16822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62fe3346cd3c42c4a55e14af9073bca4" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:22.336920 16816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f82224f4564fea93572e8206bad1fe" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:22.338161 16821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2794af5b99984efebdee564035841cb0" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:22.306200 21568 raft_consensus.cc:493] T 02838c86531d4f98bf58d1c22b8baf77 P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Starting leader election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:21:22.309294 21592 consensus_queue.cc:1048] T c0eab25ce4c64e018a041beeda2867db P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.012s
W20260709 13:21:22.295385 16115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 85397.5us
I20260709 13:21:22.370581 16218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1df93779209748b9860f6d893dd3de53" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
W20260709 13:21:22.364630 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 68897.7us
I20260709 13:21:22.083420 21570 raft_consensus.cc:515] T ee9ed67006144797b1de6ea8c2013a68 P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:22.312590 21456 raft_consensus.cc:493] T ecc1e363230043fe9fef390f0b979f20 P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:22.366175 16225 rpcz_store.cc:276] 0709 13:21:19.138453 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:19.822718 (+684265us) service_pool.cc:224] Handling call
0709 13:21:19.823046 (+   328us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:21:21.098492 (+1275446us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:22.294622 (+1196130us) spinlock_profiling.cc:243] Waited 1.27 s on lock 0x7b5400245c54. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:21:22.295611 (+   989us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2928559872}
I20260709 13:21:22.384617 16818 raft_consensus.cc:1240] T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860 [term 16 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 15. Current term is 16. Ops: []
I20260709 13:21:22.307380 21585 raft_consensus.cc:3060] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:21:22.295964 21598 consensus_queue.cc:1048] T 08c74490c40243a88e62d1760b5cd814 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.069s
I20260709 13:21:22.387061 16115 leader_election.cc:304] T 8cc959fb7f704fba9009c0595467d074 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
W20260709 13:21:22.392035 16293 consensus_meta.cc:287] T e5ad4eb281564936a59f41e387a47156 P 90f3da57796b488c9087a2f523adf045: Time spent flushing consensus metadata: real 0.563s	user 0.009s	sys 0.001s
I20260709 13:21:22.392688 16293 raft_consensus.cc:2468] T e5ad4eb281564936a59f41e387a47156 P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 15.
I20260709 13:21:22.393635 16291 raft_consensus.cc:1240] T e5ad4eb281564936a59f41e387a47156 P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 14. Current term is 15. Ops: [14.10-14.10]
I20260709 13:21:22.206261 21501 consensus_queue.cc:237] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:22.358357 21391 raft_consensus.cc:493] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:21:22.360849 21154 raft_consensus.cc:2804] T c0eab25ce4c64e018a041beeda2867db P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:21:22.372668 21601 raft_consensus.cc:493] T c2f5f89186204242bf377f7c42b2e2da P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:22.350760 21586 consensus_queue.cc:260] T e5ad4eb281564936a59f41e387a47156 P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.10, Last appended by leader: 10, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:21:22.404132 21614 log.cc:927] Time spent T 66f75124215f4ca6bef831e174a792d8 P cb82cc7bd6db46d980c75237a1402860: Append to log took a long time: real 0.222s	user 0.000s	sys 0.001s
W20260709 13:21:22.404412 21618 log.cc:927] Time spent T ee9ed67006144797b1de6ea8c2013a68 P cb82cc7bd6db46d980c75237a1402860: Append to log took a long time: real 0.188s	user 0.000s	sys 0.000s
W20260709 13:21:22.404572 21468 consensus_meta.cc:287] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045: Time spent flushing consensus metadata: real 0.646s	user 0.012s	sys 0.000s
I20260709 13:21:22.405226 21182 leader_election.cc:290] T e4dc6b0541ca431ebc757beac00521f1 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 14 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:22.405965 21569 consensus_queue.cc:237] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:21:22.408612 21606 log.cc:927] Time spent T 26354361e9b044cea1c2b86c4882fe92 P cb82cc7bd6db46d980c75237a1402860: Append to log took a long time: real 0.223s	user 0.001s	sys 0.000s
I20260709 13:21:22.364802 21594 raft_consensus.cc:493] T 8c11074579b046d99e585c2818e18a2c P cb82cc7bd6db46d980c75237a1402860 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:21:22.411180 16115 leader_election.cc:304] T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:21:22.380968 21568 raft_consensus.cc:3060] T 02838c86531d4f98bf58d1c22b8baf77 P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:21:22.388726 21543 raft_consensus.cc:493] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:22.387475 21456 raft_consensus.cc:3060] T ecc1e363230043fe9fef390f0b979f20 P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:21:22.419507 16191 leader_election.cc:304] T 8a878999c0024d2abe517484180290fc P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:21:22.424115 16143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fffab26427e24d398e2695aad40ca134" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:21:22.433440 16146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c61a39fd4f84424b129c1a6f9ba404d" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:22.502344 16143 raft_consensus.cc:2368] T fffab26427e24d398e2695aad40ca134 P 647a343a542d4a139fe106c19095129b [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 for earlier term 13. Current term is 14.
I20260709 13:21:22.478955 16144 raft_consensus.cc:1275] T 7c2095a5f4ad49408dbc7f69c606120d P 647a343a542d4a139fe106c19095129b [term 10 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
W20260709 13:21:22.419628 21596 log.cc:927] Time spent T 806d7690c2b644959152074bb2cab0a7 P cb82cc7bd6db46d980c75237a1402860: Append to log took a long time: real 0.246s	user 0.001s	sys 0.000s
W20260709 13:21:22.431044 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 59903.3us
I20260709 13:21:22.423273 16145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c74490c40243a88e62d1760b5cd814" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:22.414204 21567 raft_consensus.cc:2749] T 8cc959fb7f704fba9009c0595467d074 P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:21:22.435323 16142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68a6838d82ef4cac83507f906139d90f" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:22.437126 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8158bbd17b994d2eb96f560bd0f1b88d" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:22.485260 16190 leader_election.cc:304] T e5ad4eb281564936a59f41e387a47156 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:21:22.486905 16292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4dc6b0541ca431ebc757beac00521f1" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:21:22.514623 16292 raft_consensus.cc:3060] T e4dc6b0541ca431ebc757beac00521f1 P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Advancing to term 14
I20260709 13:21:22.487063 16293 raft_consensus.cc:1275] T fffab26427e24d398e2695aad40ca134 P 90f3da57796b488c9087a2f523adf045 [term 14 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:21:22.517437 16142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e9b2bf7011f4ae688a79d7bb5a447b4" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:22.414949 21154 raft_consensus.cc:493] T c0eab25ce4c64e018a041beeda2867db P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:22.523605 21154 raft_consensus.cc:3060] T c0eab25ce4c64e018a041beeda2867db P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:21:22.504194 21594 raft_consensus.cc:515] T 8c11074579b046d99e585c2818e18a2c P cb82cc7bd6db46d980c75237a1402860 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:22.525700 21594 leader_election.cc:290] T 8c11074579b046d99e585c2818e18a2c P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:22.526638 21626 raft_consensus.cc:493] T 9e65676b21b748728ad39668cfe722ec P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
W20260709 13:21:22.493584 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 70946.6us
W20260709 13:21:22.527657 16265 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:22.527240 21626 raft_consensus.cc:515] T 9e65676b21b748728ad39668cfe722ec P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:21:22.528085 16265 connection.cc:256] RPC call timeout handler was delayed by -1.6758148280000569s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1563s for a response
W20260709 13:21:22.528549 16265 connection.cc:256] RPC call timeout handler was delayed by -1.2150148279999939s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20750000000000002s for a response
W20260709 13:21:22.528873 16265 connection.cc:256] RPC call timeout handler was delayed by -1.1637148280001384s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21320000000000003s for a response
W20260709 13:21:22.529220 16265 connection.cc:256] RPC call timeout handler was delayed by -1.1424318400001994s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1512s for a response
W20260709 13:21:22.529521 16265 connection.cc:256] RPC call timeout handler was delayed by -1.1367148280000947s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2162s for a response
W20260709 13:21:22.529832 16265 connection.cc:256] RPC call timeout handler was delayed by -1.12141482800007s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21789999999999998s for a response
W20260709 13:21:22.529986 21602 log.cc:927] Time spent T 57a744f5aaf944678a63469cbf514015 P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 0.219s	user 0.000s	sys 0.000s
I20260709 13:21:22.412729 21084 leader_election.cc:290] T ee9ed67006144797b1de6ea8c2013a68 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:22.531057 21612 raft_consensus.cc:3060] T 8a878999c0024d2abe517484180290fc P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:21:22.535907 16144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a878999c0024d2abe517484180290fc" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:21:22.536674 21610 raft_consensus.cc:2749] T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:21:22.536757 16144 raft_consensus.cc:2376] T 8a878999c0024d2abe517484180290fc P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 13. Re-sending same reply.
W20260709 13:21:22.530198 16265 connection.cc:256] RPC call timeout handler was delayed by -1.0791148280000016s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22260000000000002s for a response
I20260709 13:21:22.420199 21601 raft_consensus.cc:3060] T c2f5f89186204242bf377f7c42b2e2da P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:21:22.539865 21609 leader_election.cc:290] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:22.502529 16146 raft_consensus.cc:2468] T 9c61a39fd4f84424b129c1a6f9ba404d P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 15.
W20260709 13:21:22.543943 16265 connection.cc:256] RPC call timeout handler was delayed by -1.0758318400000917s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15860000000000002s for a response
W20260709 13:21:22.546030 16265 connection.cc:256] RPC call timeout handler was delayed by -1.0758318400000917s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15860000000000002s for a response
W20260709 13:21:22.546455 16265 connection.cc:256] RPC call timeout handler was delayed by -1.0173318400002245s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16510000000000002s for a response
I20260709 13:21:22.547154 16265 leader_election.cc:304] T 5a959dbecc2648eb85d1cc550f5a3245 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac
I20260709 13:21:22.548573 21354 raft_consensus.cc:2804] T 5a959dbecc2648eb85d1cc550f5a3245 P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 13:21:22.501129 16115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 90061.3us
I20260709 13:21:22.548998 21354 raft_consensus.cc:493] T 5a959dbecc2648eb85d1cc550f5a3245 P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:21:22.549453 21354 raft_consensus.cc:3060] T 5a959dbecc2648eb85d1cc550f5a3245 P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:21:22.512601 16143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c2095a5f4ad49408dbc7f69c606120d" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:21:22.552073 16265 leader_election.cc:304] T 84f2da98864b4425bf7c0055951fdd4b P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:21:22.552481 16143 raft_consensus.cc:2376] T 7c2095a5f4ad49408dbc7f69c606120d P 647a343a542d4a139fe106c19095129b [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 10. Re-sending same reply.
I20260709 13:21:22.553092 16147 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46224 (request call id 829) took 1299 ms. Trace:
I20260709 13:21:22.553543 16147 rpcz_store.cc:276] 0709 13:21:21.253428 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:21.253653 (+   225us) service_pool.cc:224] Handling call
0709 13:21:21.254122 (+   469us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:21:21.444171 (+190049us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:21:21.449474 (+  5303us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:21:21.460537 (+ 11063us) log.cc:844] Serialized 26 byte log entry
0709 13:21:21.741805 (+281268us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:21:21.741863 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:21.741955 (+    92us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:21:22.552880 (+810925us) raft_consensus.cc:1685] finished
0709 13:21:22.552900 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:21:22.553034 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":15604,"raft.queue_time_us":193352,"raft.run_cpu_time_us":145,"raft.run_wall_time_us":10630,"thread_start_us":423650,"threads_started":2,"wal-append.queue_time_us":477401}
I20260709 13:21:22.557072 21516 leader_election.cc:290] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:22.418983 21391 raft_consensus.cc:515] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:22.560109 16145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87ac0d1f7e8a419f9bcbc04aee0e067e" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
W20260709 13:21:22.492551 21560 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.533s	user 0.000s	sys 0.012s
W20260709 13:21:22.513744 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 94359.4us
I20260709 13:21:22.412940 21611 raft_consensus.cc:493] T 7c2095a5f4ad49408dbc7f69c606120d P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:21:22.538884 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 53708us
I20260709 13:21:22.502009 21586 raft_consensus.cc:3060] T e5ad4eb281564936a59f41e387a47156 P 647a343a542d4a139fe106c19095129b [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:21:22.539554 21084 raft_consensus.cc:2804] T e5ad4eb281564936a59f41e387a47156 P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:21:22.547775 21508 leader_election.cc:290] T 5a959dbecc2648eb85d1cc550f5a3245 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 13 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:22.508603 16218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5718c623ebee4d2bb770f0f9ddb2f164" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:22.579140 16218 raft_consensus.cc:2468] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 12.
I20260709 13:21:22.421026 21607 raft_consensus.cc:2749] T 68a6838d82ef4cac83507f906139d90f P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:21:22.583282 16191 leader_election.cc:304] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 90f3da57796b488c9087a2f523adf045
I20260709 13:21:22.560894 16265 leader_election.cc:304] T 8cc959fb7f704fba9009c0595467d074 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:21:22.560418 16820 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49892 (request call id 802) took 1224 ms. Trace:
I20260709 13:21:22.585361 16820 rpcz_store.cc:276] 0709 13:21:21.337519 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:21.338070 (+   551us) service_pool.cc:224] Handling call
0709 13:21:21.338476 (+   406us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:21:21.389020 (+ 50544us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:21:21.389064 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:21:21.395900 (+  6836us) log.cc:844] Serialized 26 byte log entry
0709 13:21:21.747082 (+351182us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:21:21.747141 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:21.747308 (+   167us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:21:22.560069 (+812761us) raft_consensus.cc:1685] finished
0709 13:21:22.560100 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:21:22.560340 (+   240us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":44045,"raft.run_cpu_time_us":139,"raft.run_wall_time_us":3702,"thread_start_us":302364,"threads_started":1,"wal-append.queue_time_us":529592}
W20260709 13:21:22.539201 21614 log.cc:927] Time spent T 66f75124215f4ca6bef831e174a792d8 P cb82cc7bd6db46d980c75237a1402860: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
W20260709 13:21:22.567514 21560 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.605s	user 0.000s	sys 0.013s
I20260709 13:21:22.593214 16265 leader_election.cc:304] T b87147731a68400e8cce9b879daa7c5d P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
I20260709 13:21:22.596179 16117 leader_election.cc:304] T 23f82224f4564fea93572e8206bad1fe P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860
I20260709 13:21:22.595321 16819 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49892 (request call id 804) took 1176 ms. Trace:
I20260709 13:21:22.601649 16819 rpcz_store.cc:276] 0709 13:21:21.419211 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:21.419928 (+   717us) service_pool.cc:224] Handling call
0709 13:21:21.420325 (+   397us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:21:21.420602 (+   277us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:21:21.420640 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:21:21.421314 (+   674us) log.cc:844] Serialized 26 byte log entry
0709 13:21:21.489656 (+ 68342us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:21:21.489711 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:21.489784 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:21:22.595043 (+1105259us) raft_consensus.cc:1685] finished
0709 13:21:22.595067 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:21:22.595252 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":41239,"threads_started":1,"wal-append.queue_time_us":526961}
I20260709 13:21:22.602344 16265 leader_election.cc:304] T e5ad4eb281564936a59f41e387a47156 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:21:22.597088 21480 consensus_queue.cc:1048] T fffab26427e24d398e2695aad40ca134 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.019s
I20260709 13:21:22.616940 21630 raft_consensus.cc:2749] T b87147731a68400e8cce9b879daa7c5d P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:21:22.617868 21582 raft_consensus.cc:2749] T e5ad4eb281564936a59f41e387a47156 P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:21:22.567484 21626 leader_election.cc:290] T 9e65676b21b748728ad39668cfe722ec P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:22.621739 16265 leader_election.cc:304] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac
I20260709 13:21:22.567353 21633 raft_consensus.cc:2749] T 84f2da98864b4425bf7c0055951fdd4b P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:21:22.628698 16265 leader_election.cc:304] T 8d4ab870a05347868956aa8b841b2981 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac
I20260709 13:21:22.598935 21567 raft_consensus.cc:493] T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:21:22.689216 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 60604.3us
I20260709 13:21:22.693372 21354 raft_consensus.cc:515] T 5a959dbecc2648eb85d1cc550f5a3245 P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:22.694675 21633 raft_consensus.cc:2804] T 8d4ab870a05347868956aa8b841b2981 P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:21:22.690657 16265 leader_election.cc:304] T 9e65676b21b748728ad39668cfe722ec P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
I20260709 13:21:22.703277 16265 leader_election.cc:304] T 31969392ccdb43b491911a319e8bc7e1 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
I20260709 13:21:22.600288 21531 raft_consensus.cc:2749] T 8cc959fb7f704fba9009c0595467d074 P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:21:22.633019 21523 leader_election.cc:290] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 13 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:22.644593 21582 raft_consensus.cc:1064] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045: attempting to promote NON_VOTER cb82cc7bd6db46d980c75237a1402860 to VOTER
I20260709 13:21:22.509836 21543 raft_consensus.cc:515] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:22.652159 21626 consensus_queue.cc:1059] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: INVALID_TERM, Last received: 15.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:21:22.654069 16292 raft_consensus.cc:2468] T e4dc6b0541ca431ebc757beac00521f1 P 90f3da57796b488c9087a2f523adf045 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 14.
I20260709 13:21:22.632018 21515 raft_consensus.cc:2804] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:21:22.803857 21515 raft_consensus.cc:493] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Starting leader election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:21:22.804388 21515 raft_consensus.cc:3060] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:21:22.636816 21630 raft_consensus.cc:2804] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:21:22.807422 21630 raft_consensus.cc:493] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:21:22.808017 21630 raft_consensus.cc:3060] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:21:22.658986 16819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5718c623ebee4d2bb770f0f9ddb2f164" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:22.811627 16819 raft_consensus.cc:2393] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 14: Already voted for candidate cb82cc7bd6db46d980c75237a1402860 in this term.
W20260709 13:21:22.659659 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 153524us
I20260709 13:21:22.659988 16820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e65676b21b748728ad39668cfe722ec" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:22.606149 21508 consensus_queue.cc:1059] T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.044s
I20260709 13:21:22.675313 16147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5d59407eb545b895d8d922e0dd69fd" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:21:22.678956 21084 raft_consensus.cc:697] T e5ad4eb281564936a59f41e387a47156 P 1ca2725d290e47d68b1a4b609c9816ac [term 15 LEADER]: Becoming Leader. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:21:22.806504 21507 raft_consensus.cc:493] T 829b4ad8c5a54eababeea1181b960d4b P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:21:22.816192 21084 consensus_queue.cc:237] T e5ad4eb281564936a59f41e387a47156 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:22.816939 21507 raft_consensus.cc:515] T 829b4ad8c5a54eababeea1181b960d4b P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:22.576273 21611 raft_consensus.cc:515] T 7c2095a5f4ad49408dbc7f69c606120d P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:22.821231 21611 leader_election.cc:290] T 7c2095a5f4ad49408dbc7f69c606120d P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:22.822499 21612 raft_consensus.cc:2749] T 8a878999c0024d2abe517484180290fc P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:21:22.680974 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 97842.3us
W20260709 13:21:22.823825 21442 consensus_meta.cc:287] T 2794af5b99984efebdee564035841cb0 P 647a343a542d4a139fe106c19095129b: Time spent flushing consensus metadata: real 0.868s	user 0.010s	sys 0.000s
I20260709 13:21:22.651361 21635 raft_consensus.cc:2749] T 23f82224f4564fea93572e8206bad1fe P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:21:22.652719 21598 raft_consensus.cc:493] T 7c2095a5f4ad49408dbc7f69c606120d P 647a343a542d4a139fe106c19095129b [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:22.824285 21442 raft_consensus.cc:515] T 2794af5b99984efebdee564035841cb0 P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:22.824803 21598 raft_consensus.cc:515] T 7c2095a5f4ad49408dbc7f69c606120d P 647a343a542d4a139fe106c19095129b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:22.826490 21442 leader_election.cc:290] T 2794af5b99984efebdee564035841cb0 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 12 election: Requested vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:22.829866 21391 leader_election.cc:290] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
W20260709 13:21:22.687709 21476 consensus_meta.cc:287] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045: Time spent flushing consensus metadata: real 0.770s	user 0.013s	sys 0.002s
I20260709 13:21:22.713654 16265 leader_election.cc:304] T 23f82224f4564fea93572e8206bad1fe P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
W20260709 13:21:22.834229 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 120688us
I20260709 13:21:22.835361 16294 raft_consensus.cc:2468] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 14.
I20260709 13:21:22.835896 21468 raft_consensus.cc:493] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:22.779088 21354 leader_election.cc:290] T 5a959dbecc2648eb85d1cc550f5a3245 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 13 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:21:22.836704 21354 raft_consensus.cc:2749] T 31969392ccdb43b491911a319e8bc7e1 P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:21:22.837340 21354 raft_consensus.cc:2749] T 23f82224f4564fea93572e8206bad1fe P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:21:22.837967 21354 consensus_queue.cc:1048] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:21:22.839541 21585 raft_consensus.cc:515] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:22.840719 16145 raft_consensus.cc:2393] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 16: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
I20260709 13:21:22.841305 21628 raft_consensus.cc:493] T 8a878999c0024d2abe517484180290fc P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:22.841799 21628 raft_consensus.cc:515] T 8a878999c0024d2abe517484180290fc P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:22.694897 21567 raft_consensus.cc:515] T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:22.704120 21633 raft_consensus.cc:493] T 8d4ab870a05347868956aa8b841b2981 P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
W20260709 13:21:22.784245 21631 log.cc:927] Time spent T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 0.112s	user 0.000s	sys 0.000s
I20260709 13:21:22.823706 16191 leader_election.cc:304] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 90f3da57796b488c9087a2f523adf045
I20260709 13:21:22.836362 21468 raft_consensus.cc:515] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:22.830899 21476 raft_consensus.cc:2749] T 9e65676b21b748728ad39668cfe722ec P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:21:22.859817 16142 raft_consensus.cc:2468] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 12.
W20260709 13:21:22.875721 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 52131.7us
I20260709 13:21:22.904389 16191 leader_election.cc:304] T b6f07842d3aa47858df5665874c86f3a P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:21:22.875953 16820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2794af5b99984efebdee564035841cb0" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
W20260709 13:21:22.788348 21616 log.cc:927] Time spent T 434bff3082d141838068f323bdb0d1ed P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 0.478s	user 0.003s	sys 0.000s
W20260709 13:21:22.890154 16266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 254195us
I20260709 13:21:22.854635 16116 leader_election.cc:304] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
W20260709 13:21:22.916579 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 62002.5us
I20260709 13:21:22.876264 16819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e9b2bf7011f4ae688a79d7bb5a447b4" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:22.864693 16825 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3884f864a96d428785895e5463e5728f"
dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
mode: GRACEFUL
new_leader_uuid: "90f3da57796b488c9087a2f523adf045"
 from {username='slave'} at 127.0.0.1:37818
I20260709 13:21:22.923825 16147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5ad4eb281564936a59f41e387a47156" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:21:22.924160 16143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62fe3346cd3c42c4a55e14af9073bca4" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:21:22.925280 16143 raft_consensus.cc:2393] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 19: Already voted for candidate cb82cc7bd6db46d980c75237a1402860 in this term.
I20260709 13:21:22.926774 16146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c61a39fd4f84424b129c1a6f9ba404d" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:21:22.928424 16292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c2095a5f4ad49408dbc7f69c606120d" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:22.936764 16222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2372981006d4b218215c28964dea623" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:21:22.937148 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "829b4ad8c5a54eababeea1181b960d4b" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:22.937423 16219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7ca61592e848ea9b0fb81a2ce33cdc" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:21:22.938092 16222 raft_consensus.cc:2410] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 for term 16 because replica has last-logged OpId of term: 15 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 10.
I20260709 13:21:22.938092 16217 raft_consensus.cc:2410] T 829b4ad8c5a54eababeea1181b960d4b P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 for term 15 because replica has last-logged OpId of term: 14 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 11.
I20260709 13:21:22.939836 16219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:21:22.940480 16222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62fe3346cd3c42c4a55e14af9073bca4" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:21:22.940723 16219 raft_consensus.cc:3060] T 19baa7bc367844c288bca8f81439c413 P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:21:22.942549 16190 leader_election.cc:304] T e4dc6b0541ca431ebc757beac00521f1 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:21:22.887763 21366 raft_consensus.cc:2804] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 13:21:22.859314 21582 raft_consensus.cc:1068] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045: Unable to promote non-voter cb82cc7bd6db46d980c75237a1402860: Illegal state: Leader has not yet committed an operation in its own term
I20260709 13:21:22.985459 16144 raft_consensus.cc:1275] T fffab26427e24d398e2695aad40ca134 P 647a343a542d4a139fe106c19095129b [term 14 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:21:22.901784 21507 leader_election.cc:290] T 829b4ad8c5a54eababeea1181b960d4b P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:22.911800 21611 raft_consensus.cc:2749] T b6f07842d3aa47858df5665874c86f3a P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:21:22.989754 16116 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:22.957466 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee9ed67006144797b1de6ea8c2013a68" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:22.958570 16220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c2095a5f4ad49408dbc7f69c606120d" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:22.960707 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9944d5e4b8c4109af9955c4f08c9095" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:21:22.992758 16220 raft_consensus.cc:2468] T 7c2095a5f4ad49408dbc7f69c606120d P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 10.
I20260709 13:21:22.956632 16222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a959dbecc2648eb85d1cc550f5a3245" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:21:22.843106 21515 raft_consensus.cc:515] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:22.950868 16146 raft_consensus.cc:2393] T 9c61a39fd4f84424b129c1a6f9ba404d P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 15: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
I20260709 13:21:22.978993 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:21:22.855279 21508 raft_consensus.cc:3055] T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:21:22.854106 21626 raft_consensus.cc:3055] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:21:22.998025 16225 raft_consensus.cc:2376] T ecc1e363230043fe9fef390f0b979f20 P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 15. Re-sending same reply.
I20260709 13:21:22.986647 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8638ee363273426db1393dd81e5c7b72" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:22.853540 21567 leader_election.cc:290] T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:22.861207 21543 leader_election.cc:290] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:22.987936 16219 raft_consensus.cc:2468] T 19baa7bc367844c288bca8f81439c413 P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 17.
I20260709 13:21:22.952095 16143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c2095a5f4ad49408dbc7f69c606120d" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:23.052316 16143 raft_consensus.cc:2468] T 7c2095a5f4ad49408dbc7f69c606120d P 647a343a542d4a139fe106c19095129b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 10.
I20260709 13:21:23.052734 16219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdbd78acb4cf46298e0de8ba0d130e19" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
W20260709 13:21:22.995082 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 160084us
I20260709 13:21:22.942049 21442 raft_consensus.cc:2804] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:21:22.868798 21633 raft_consensus.cc:3060] T 8d4ab870a05347868956aa8b841b2981 P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:21:22.992317 16226 raft_consensus.cc:2468] T ee9ed67006144797b1de6ea8c2013a68 P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 13.
I20260709 13:21:22.980619 21366 raft_consensus.cc:493] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:22.998731 16299 raft_consensus.cc:2468] T 8638ee363273426db1393dd81e5c7b72 P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 15.
I20260709 13:21:22.849675 21154 raft_consensus.cc:515] T c0eab25ce4c64e018a041beeda2867db P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:23.046986 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4dc6b0541ca431ebc757beac00521f1" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:22.984037 21643 raft_consensus.cc:2804] T e4dc6b0541ca431ebc757beac00521f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:21:23.001006 21508 raft_consensus.cc:740] T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:21:23.049068 16116 leader_election.cc:304] T 7c2095a5f4ad49408dbc7f69c606120d P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
I20260709 13:21:23.049852 16144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee9ed67006144797b1de6ea8c2013a68" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:22.901428 21601 raft_consensus.cc:515] T c2f5f89186204242bf377f7c42b2e2da P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:23.063668 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5d59407eb545b895d8d922e0dd69fd" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:21:23.053241 16143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5d59407eb545b895d8d922e0dd69fd" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:23.055824 16265 leader_election.cc:304] T 434bff3082d141838068f323bdb0d1ed P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: cb82cc7bd6db46d980c75237a1402860
I20260709 13:21:23.056537 16220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a878999c0024d2abe517484180290fc" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:22.995419 21598 leader_election.cc:290] T 7c2095a5f4ad49408dbc7f69c606120d P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:23.067072 21154 leader_election.cc:290] T c0eab25ce4c64e018a041beeda2867db P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 15 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:23.067981 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b7d1ab519044eda2d5e04a07242f25" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:23.069556 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5718c623ebee4d2bb770f0f9ddb2f164" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:23.070252 21601 leader_election.cc:290] T c2f5f89186204242bf377f7c42b2e2da P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 16 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:23.065394 16220 raft_consensus.cc:2468] T 8a878999c0024d2abe517484180290fc P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 13.
I20260709 13:21:23.058079 16147 raft_consensus.cc:2410] T e5ad4eb281564936a59f41e387a47156 P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac for term 15 because replica has last-logged OpId of term: 14 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 9.
I20260709 13:21:23.060735 16151 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46224 (request call id 840) took 1447 ms. Trace:
I20260709 13:21:23.001232 21626 raft_consensus.cc:740] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:21:23.047909 16146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0eab25ce4c64e018a041beeda2867db" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:23.062758 16221 raft_consensus.cc:1240] T c0eab25ce4c64e018a041beeda2867db P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 14. Current term is 15. Ops: [14.8-14.8]
I20260709 13:21:22.998188 21647 raft_consensus.cc:493] T af57996a8be84d398f0f023e8d08ef77 P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:21:23.063257 16144 raft_consensus.cc:2410] T ee9ed67006144797b1de6ea8c2013a68 P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac for term 14 because replica has last-logged OpId of term: 13 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 11.
I20260709 13:21:23.073534 16151 rpcz_store.cc:276] 0709 13:21:21.613631 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:21.654046 (+ 40415us) service_pool.cc:224] Handling call
0709 13:21:21.654415 (+   369us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:21:21.723209 (+ 68794us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:21:21.723249 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:21:21.723789 (+   540us) log.cc:844] Serialized 26 byte log entry
0709 13:21:21.889270 (+165481us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:21:21.889337 (+    67us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:21.889454 (+   117us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:21:22.787566 (+898112us) spinlock_profiling.cc:243] Waited 2.92 ms on lock 0x7b4400022b4c. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69ad8fadb1 00007f69ad998c85 00007f69ad9996a2 00007f69b070b1db 00007f69b0706aaf 00007f69b06fa86e 00007f69b06f9bb6 00007f69b06f97d8 00007f69b0703221 00007f69b07031d9 00007f69b0703169
0709 13:21:23.056662 (+269096us) raft_consensus.cc:1685] finished
0709 13:21:23.056691 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:21:23.060670 (+  3979us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":88171,"raft.queue_time_us":85747,"raft.run_cpu_time_us":232,"raft.run_wall_time_us":17078,"spinlock_wait_cycles":6716288,"thread_start_us":139288,"threads_started":1,"wal-append.queue_time_us":311256}
I20260709 13:21:23.079505 16143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "829b4ad8c5a54eababeea1181b960d4b" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:23.145705 16151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4dc6b0541ca431ebc757beac00521f1" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:21:23.054102 21442 raft_consensus.cc:493] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:21:23.060632 21606 log.cc:927] Time spent T 26354361e9b044cea1c2b86c4882fe92 P cb82cc7bd6db46d980c75237a1402860: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
W20260709 13:21:23.143790 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 87784us
I20260709 13:21:23.059266 21366 raft_consensus.cc:3060] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:21:23.143527 16191 leader_election.cc:304] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
I20260709 13:21:23.146375 16151 raft_consensus.cc:3060] T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:21:23.062875 21628 leader_election.cc:290] T 8a878999c0024d2abe517484180290fc P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:23.150192 16265 leader_election.cc:304] T 5a96373bb0e14dce8a266b2fc6c58507 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
W20260709 13:21:23.152521 21649 log.cc:927] Time spent T 00000000000000000000000000000000 P 667a34b27a39474ebefc1af3cf5b4271: Append to log took a long time: real 0.071s	user 0.000s	sys 0.001s
W20260709 13:21:23.072225 21642 log.cc:927] Time spent T 26354361e9b044cea1c2b86c4882fe92 P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 0.184s	user 0.001s	sys 0.000s
I20260709 13:21:23.072501 21643 raft_consensus.cc:697] T e4dc6b0541ca431ebc757beac00521f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 14 LEADER]: Becoming Leader. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:21:23.157359 16191 leader_election.cc:304] T 7c2095a5f4ad49408dbc7f69c606120d P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
I20260709 13:21:23.082540 21598 raft_consensus.cc:2804] T 7c2095a5f4ad49408dbc7f69c606120d P 647a343a542d4a139fe106c19095129b [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:21:23.163708 16151 raft_consensus.cc:2468] T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 14.
I20260709 13:21:23.163861 16265 leader_election.cc:400] T fffab26427e24d398e2695aad40ca134 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 13 election: Vote denied by peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741) with higher term. Message: Invalid argument: T fffab26427e24d398e2695aad40ca134 P 647a343a542d4a139fe106c19095129b [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 for earlier term 13. Current term is 14.
I20260709 13:21:23.168183 16265 leader_election.cc:403] T fffab26427e24d398e2695aad40ca134 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 13 election: Cancelling election due to peer responding with higher term
I20260709 13:21:23.169976 16265 leader_election.cc:304] T 8a878999c0024d2abe517484180290fc P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac
I20260709 13:21:23.153966 21442 raft_consensus.cc:3060] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:21:23.157809 21337 raft_consensus.cc:2749] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:21:23.166380 21608 raft_consensus.cc:2749] T 5a96373bb0e14dce8a266b2fc6c58507 P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:21:23.159441 21612 consensus_queue.cc:1048] T fffab26427e24d398e2695aad40ca134 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.047s
W20260709 13:21:23.207960 21568 consensus_meta.cc:287] T 02838c86531d4f98bf58d1c22b8baf77 P 647a343a542d4a139fe106c19095129b: Time spent flushing consensus metadata: real 0.696s	user 0.008s	sys 0.002s
I20260709 13:21:23.178824 21087 raft_consensus.cc:2804] T 7c2095a5f4ad49408dbc7f69c606120d P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:21:23.209501 21087 raft_consensus.cc:493] T 7c2095a5f4ad49408dbc7f69c606120d P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:23.209923 21087 raft_consensus.cc:3060] T 7c2095a5f4ad49408dbc7f69c606120d P 1ca2725d290e47d68b1a4b609c9816ac [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:21:23.211485 21456 consensus_meta.cc:287] T ecc1e363230043fe9fef390f0b979f20 P 90f3da57796b488c9087a2f523adf045: Time spent flushing consensus metadata: real 0.651s	user 0.011s	sys 0.000s
I20260709 13:21:23.211918 21456 raft_consensus.cc:515] T ecc1e363230043fe9fef390f0b979f20 P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:23.214536 21456 leader_election.cc:290] T ecc1e363230043fe9fef390f0b979f20 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 16 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:23.171900 21598 raft_consensus.cc:493] T 7c2095a5f4ad49408dbc7f69c606120d P 647a343a542d4a139fe106c19095129b [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:23.180857 21582 consensus_queue.cc:1048] T 7c2095a5f4ad49408dbc7f69c606120d P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.035s
I20260709 13:21:23.215945 21598 raft_consensus.cc:3060] T 7c2095a5f4ad49408dbc7f69c606120d P 647a343a542d4a139fe106c19095129b [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:21:23.219054 21646 log.cc:927] Time spent T 26354361e9b044cea1c2b86c4882fe92 P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.062s	user 0.000s	sys 0.000s
I20260709 13:21:23.181579 21572 raft_consensus.cc:2749] T fffab26427e24d398e2695aad40ca134 P 90f3da57796b488c9087a2f523adf045 [term 14 FOLLOWER]: Leader election lost for term 13. Reason: Vote denied by peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741) with higher term. Message: Invalid argument: T fffab26427e24d398e2695aad40ca134 P 647a343a542d4a139fe106c19095129b [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 for earlier term 13. Current term is 14.
I20260709 13:21:23.153525 21645 raft_consensus.cc:2804] T 434bff3082d141838068f323bdb0d1ed P 90f3da57796b488c9087a2f523adf045 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:21:23.190675 21648 raft_consensus.cc:2804] T 8a878999c0024d2abe517484180290fc P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:21:23.220609 21645 raft_consensus.cc:493] T 434bff3082d141838068f323bdb0d1ed P 90f3da57796b488c9087a2f523adf045 [term 14 FOLLOWER]: Starting leader election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:21:23.220872 21648 raft_consensus.cc:697] T 8a878999c0024d2abe517484180290fc P 90f3da57796b488c9087a2f523adf045 [term 13 LEADER]: Becoming Leader. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:21:23.221043 21645 raft_consensus.cc:3060] T 434bff3082d141838068f323bdb0d1ed P 90f3da57796b488c9087a2f523adf045 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:21:23.230413 21442 raft_consensus.cc:515] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:23.231909 21087 raft_consensus.cc:515] T 7c2095a5f4ad49408dbc7f69c606120d P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:21:23.245143 21576 log.cc:927] Time spent T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 1.711s	user 0.007s	sys 0.001s
I20260709 13:21:23.260634 21087 leader_election.cc:290] T 7c2095a5f4ad49408dbc7f69c606120d P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 11 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:23.264743 21500 leader_election.cc:290] T d16dfad0c3b540e5b4b86492ddf87208 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:23.157248 21647 raft_consensus.cc:515] T af57996a8be84d398f0f023e8d08ef77 P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:23.208477 21568 raft_consensus.cc:515] T 02838c86531d4f98bf58d1c22b8baf77 P 647a343a542d4a139fe106c19095129b [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:23.265316 21598 raft_consensus.cc:515] T 7c2095a5f4ad49408dbc7f69c606120d P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:23.213562 21630 raft_consensus.cc:515] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:23.221810 21648 consensus_queue.cc:237] T 8a878999c0024d2abe517484180290fc P 90f3da57796b488c9087a2f523adf045 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:23.204389 21643 consensus_queue.cc:237] T e4dc6b0541ca431ebc757beac00521f1 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.6, Last appended by leader: 6, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:23.265339 21442 leader_election.cc:290] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 14 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:23.277794 21598 leader_election.cc:290] T 7c2095a5f4ad49408dbc7f69c606120d P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 11 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:23.491629 16146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0eab25ce4c64e018a041beeda2867db" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:21:23.420261 16150 raft_consensus.cc:2368] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac for earlier term 13. Current term is 14.
W20260709 13:21:23.465642 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 295579us
I20260709 13:21:23.413899 16148 raft_consensus.cc:1240] T 02838c86531d4f98bf58d1c22b8baf77 P 647a343a542d4a139fe106c19095129b [term 14 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 13. Current term is 14. Ops: []
I20260709 13:21:23.632718 16146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f5f89186204242bf377f7c42b2e2da" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:21:23.265763 21424 raft_consensus.cc:493] T 26354361e9b044cea1c2b86c4882fe92 P cb82cc7bd6db46d980c75237a1402860 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:21:23.634562 21354 raft_consensus.cc:493] T c4f65cdf00b748bb963ee6e81ef43b79 P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:21:23.635469 21572 raft_consensus.cc:493] T 8c11074579b046d99e585c2818e18a2c P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:21:23.636631 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1df93779209748b9860f6d893dd3de53" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:23.637622 21512 leader_election.cc:290] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 14 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
W20260709 13:21:23.434453 16191 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:21:23.417199 16116 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:23.556368 16149 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40830 (request call id 1726) took 1090 ms. Trace:
I20260709 13:21:23.586906 21353 raft_consensus.cc:493] T d16dfad0c3b540e5b4b86492ddf87208 P 90f3da57796b488c9087a2f523adf045 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:21:23.651767 16292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e65676b21b748728ad39668cfe722ec" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:23.652199 21353 raft_consensus.cc:515] T d16dfad0c3b540e5b4b86492ddf87208 P 90f3da57796b488c9087a2f523adf045 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:23.652783 16292 raft_consensus.cc:2468] T 9e65676b21b748728ad39668cfe722ec P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 13.
W20260709 13:21:23.588248 16117 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 713539us
W20260709 13:21:23.587460 16224 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57604 (request call id 2845) took 2458 ms (client timeout 1737 ms). Trace:
I20260709 13:21:23.585630 21401 raft_consensus.cc:493] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:23.271723 21568 leader_election.cc:290] T 02838c86531d4f98bf58d1c22b8baf77 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 14 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:23.671132 21572 raft_consensus.cc:515] T 8c11074579b046d99e585c2818e18a2c P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:23.672127 21353 leader_election.cc:290] T d16dfad0c3b540e5b4b86492ddf87208 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:21:23.673928 21354 raft_consensus.cc:515] T c4f65cdf00b748bb963ee6e81ef43b79 P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:23.541175 21515 leader_election.cc:290] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 15 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:23.287202 21508 consensus_queue.cc:260] T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:23.569852 21647 leader_election.cc:290] T af57996a8be84d398f0f023e8d08ef77 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:23.701906 21424 raft_consensus.cc:3060] T 26354361e9b044cea1c2b86c4882fe92 P cb82cc7bd6db46d980c75237a1402860 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:21:23.663056 16224 rpcz_store.cc:269] 0709 13:21:21.128902 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:21.129199 (+   297us) service_pool.cc:224] Handling call
0709 13:21:23.586755 (+2457556us) spinlock_profiling.cc:243] Waited 27.9 ms on lock 0x7b5400367c54. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:21:23.587085 (+   330us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":64066304}
I20260709 13:21:23.586244 21662 raft_consensus.cc:493] T 18ad7e3af5504571a69d10c00750a01c P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:21:23.649825 16149 rpcz_store.cc:276] 0709 13:21:22.436303 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:22.518023 (+ 81720us) service_pool.cc:224] Handling call
0709 13:21:22.518325 (+   302us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:21:22.518527 (+   202us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:21:22.518550 (+    23us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:21:22.567393 (+ 48843us) log.cc:844] Serialized 26 byte log entry
0709 13:21:22.718735 (+151342us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:21:22.718770 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:22.718843 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:21:23.419302 (+700459us) raft_consensus.cc:1685] finished
0709 13:21:23.419415 (+   113us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:21:23.526597 (+107182us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":99254,"threads_started":1,"wal-append.queue_time_us":269543}
I20260709 13:21:23.723933 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4dc6b0541ca431ebc757beac00521f1" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:23.725018 21507 raft_consensus.cc:493] T ecc1e363230043fe9fef390f0b979f20 P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:21:23.284071 21651 log.cc:927] Time spent T e5ad4eb281564936a59f41e387a47156 P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 0.233s	user 0.001s	sys 0.000s
I20260709 13:21:23.726845 16298 raft_consensus.cc:1240] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 16. Current term is 17. Ops: [16.10-16.10]
I20260709 13:21:23.564909 21671 raft_consensus.cc:493] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P cb82cc7bd6db46d980c75237a1402860 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:21:23.277104 21444 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.881s	user 0.009s	sys 0.011s
I20260709 13:21:23.559279 21543 raft_consensus.cc:493] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:21:23.282341 21630 leader_election.cc:290] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 13 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:21:23.810726 16146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31969392ccdb43b491911a319e8bc7e1" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:21:23.810976 16300 raft_consensus.cc:2393] T e4dc6b0541ca431ebc757beac00521f1 P 90f3da57796b488c9087a2f523adf045 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 14: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
W20260709 13:21:23.801493 16789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 83200.3us
I20260709 13:21:23.681190 21508 raft_consensus.cc:3060] T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:21:23.676908 21354 leader_election.cc:290] T c4f65cdf00b748bb963ee6e81ef43b79 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:21:23.674239 21572 leader_election.cc:290] T 8c11074579b046d99e585c2818e18a2c P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:23.800999 16265 leader_election.cc:304] T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
W20260709 13:21:23.827659 21661 log.cc:927] Time spent T ee9ed67006144797b1de6ea8c2013a68 P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 0.218s	user 0.001s	sys 0.000s
I20260709 13:21:23.830022 16291 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48674 (request call id 2958) took 1983 ms. Trace:
I20260709 13:21:23.830549 16291 rpcz_store.cc:276] 0709 13:21:21.846443 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:21.846670 (+   227us) service_pool.cc:224] Handling call
0709 13:21:22.393373 (+546703us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:21:22.394088 (+   715us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:23.829451 (+1435363us) spinlock_profiling.cc:243] Waited 546 ms on lock 0x7b54001ef350. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5e8ad 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:21:23.829803 (+   352us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1256516864}
I20260709 13:21:23.832656 16116 leader_election.cc:304] T ee9ed67006144797b1de6ea8c2013a68 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
W20260709 13:21:23.834102 16294 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48684 (request call id 2832) took 1965 ms (client timeout 1935 ms). Trace:
I20260709 13:21:23.834388 16116 leader_election.cc:304] T 8a878999c0024d2abe517484180290fc P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
I20260709 13:21:23.836905 21313 raft_consensus.cc:493] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:21:23.837574 21569 raft_consensus.cc:493] T 746aabf856ab401c9ef5e859371e7b67 P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:21:23.837850 21662 raft_consensus.cc:515] T 18ad7e3af5504571a69d10c00750a01c P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } }
I20260709 13:21:23.835263 21097 raft_consensus.cc:2804] T ee9ed67006144797b1de6ea8c2013a68 P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:21:23.368773 21645 raft_consensus.cc:515] T 434bff3082d141838068f323bdb0d1ed P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:21:23.834605 16294 rpcz_store.cc:269] 0709 13:21:21.868543 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:21.869904 (+  1361us) service_pool.cc:224] Handling call
0709 13:21:23.833628 (+1963724us) spinlock_profiling.cc:243] Waited 884 ms on lock 0x7b540030c0d4. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:21:23.833840 (+   212us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2034211712}
I20260709 13:21:23.850575 21585 leader_election.cc:290] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 16 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:23.855450 21543 raft_consensus.cc:515] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:23.856523 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a959dbecc2648eb85d1cc550f5a3245" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:21:23.858009 16144 raft_consensus.cc:1240] T 5a959dbecc2648eb85d1cc550f5a3245 P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 12. Current term is 13. Ops: []
I20260709 13:21:23.725029 16296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2794af5b99984efebdee564035841cb0" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
W20260709 13:21:23.868018 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 67282.5us
I20260709 13:21:23.879035 16297 raft_consensus.cc:2393] T 5a959dbecc2648eb85d1cc550f5a3245 P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 13: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
W20260709 13:21:23.837589 21366 consensus_meta.cc:287] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac: Time spent flushing consensus metadata: real 0.678s	user 0.010s	sys 0.018s
I20260709 13:21:23.777457 21401 raft_consensus.cc:515] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:23.880767 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e9b2bf7011f4ae688a79d7bb5a447b4" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:23.869812 21313 raft_consensus.cc:3060] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:21:23.869613 21097 raft_consensus.cc:493] T ee9ed67006144797b1de6ea8c2013a68 P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Starting leader election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:21:23.889622 21671 raft_consensus.cc:515] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P cb82cc7bd6db46d980c75237a1402860 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:23.890681 21097 raft_consensus.cc:3060] T ee9ed67006144797b1de6ea8c2013a68 P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:21:23.891506 21366 raft_consensus.cc:515] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } }
I20260709 13:21:23.893214 21671 leader_election.cc:290] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:23.894030 21635 raft_consensus.cc:2804] T 8a878999c0024d2abe517484180290fc P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:21:23.894520 21366 leader_election.cc:290] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 15 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:23.896414 21662 leader_election.cc:290] T 18ad7e3af5504571a69d10c00750a01c P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:21:23.898057 16291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c2095a5f4ad49408dbc7f69c606120d" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:23.899775 16143 raft_consensus.cc:1275] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 13:21:23.900462 16151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5718c623ebee4d2bb770f0f9ddb2f164" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
W20260709 13:21:23.901021 16189 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:23.901561 16151 raft_consensus.cc:2393] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 in current term 14: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
W20260709 13:21:23.902067 21444 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 2.428s	user 0.017s	sys 0.018s
W20260709 13:21:23.899806 16115 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:23.904613 21444 raft_consensus.cc:3055] T ada5a004ba7c46819051e3ccec71ffdb P 90f3da57796b488c9087a2f523adf045 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:21:23.904954 16291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a878999c0024d2abe517484180290fc" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:23.905886 21588 consensus_queue.cc:1059] T c0eab25ce4c64e018a041beeda2867db P 647a343a542d4a139fe106c19095129b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:21:23.906389 16291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02838c86531d4f98bf58d1c22b8baf77" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:21:23.907742 16291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c2095a5f4ad49408dbc7f69c606120d" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:21:23.908063 21592 raft_consensus.cc:3055] T c0eab25ce4c64e018a041beeda2867db P 647a343a542d4a139fe106c19095129b [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:21:23.908380 16300 raft_consensus.cc:1275] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 17 index: 13. (term mismatch)
I20260709 13:21:23.908567 21592 raft_consensus.cc:740] T c0eab25ce4c64e018a041beeda2867db P 647a343a542d4a139fe106c19095129b [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:21:23.908816 16300 pending_rounds.cc:85] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045: Aborting all ops after (but not including) 12
I20260709 13:21:23.909189 16300 pending_rounds.cc:107] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045: Aborting uncommitted NO_OP operation due to leader change: 16.13
I20260709 13:21:23.909518 16300 raft_consensus.cc:2889] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:21:23.909922 21592 consensus_queue.cc:260] T c0eab25ce4c64e018a041beeda2867db P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:23.911810 16143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c2095a5f4ad49408dbc7f69c606120d" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:21:23.913200 16295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d16dfad0c3b540e5b4b86492ddf87208" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:23.913492 21543 leader_election.cc:290] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:23.913924 16143 raft_consensus.cc:2393] T 7c2095a5f4ad49408dbc7f69c606120d P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 11: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
I20260709 13:21:23.913947 16295 raft_consensus.cc:2468] T d16dfad0c3b540e5b4b86492ddf87208 P 90f3da57796b488c9087a2f523adf045 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 14.
I20260709 13:21:23.925313 16143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87ac0d1f7e8a419f9bcbc04aee0e067e" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:21:23.926731 16143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af57996a8be84d398f0f023e8d08ef77" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:23.877537 21507 raft_consensus.cc:515] T ecc1e363230043fe9fef390f0b979f20 P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:23.931190 21570 leader_election.cc:290] T ee9ed67006144797b1de6ea8c2013a68 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:23.928862 21401 leader_election.cc:290] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:23.929214 21592 raft_consensus.cc:3060] T c0eab25ce4c64e018a041beeda2867db P 647a343a542d4a139fe106c19095129b [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:21:23.929760 21619 raft_consensus.cc:2804] T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:21:23.930277 21569 raft_consensus.cc:515] T 746aabf856ab401c9ef5e859371e7b67 P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:23.905061 21444 raft_consensus.cc:740] T ada5a004ba7c46819051e3ccec71ffdb P 90f3da57796b488c9087a2f523adf045 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:21:23.937868 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c11074579b046d99e585c2818e18a2c" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:23.965849 16293 raft_consensus.cc:2468] T 8c11074579b046d99e585c2818e18a2c P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 13.
I20260709 13:21:23.966784 21570 raft_consensus.cc:493] T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:23.966425 21444 consensus_queue.cc:260] T ada5a004ba7c46819051e3ccec71ffdb P 90f3da57796b488c9087a2f523adf045 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:23.967291 16296 raft_consensus.cc:3060] T 2794af5b99984efebdee564035841cb0 P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:21:23.967774 21444 raft_consensus.cc:3060] T ada5a004ba7c46819051e3ccec71ffdb P 90f3da57796b488c9087a2f523adf045 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:21:23.967610 21570 raft_consensus.cc:515] T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:23.968149 21313 raft_consensus.cc:515] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:23.947302 16291 raft_consensus.cc:1240] T 434bff3082d141838068f323bdb0d1ed P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 14. Current term is 15. Ops: []
I20260709 13:21:23.971776 16817 raft_consensus.cc:1240] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860 [term 13 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 12. Current term is 13. Ops: []
W20260709 13:21:23.965149 16117 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:23.894558 21635 raft_consensus.cc:493] T 8a878999c0024d2abe517484180290fc P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:23.978600 16295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31969392ccdb43b491911a319e8bc7e1" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:21:23.990267 16143 raft_consensus.cc:1275] T 9c61a39fd4f84424b129c1a6f9ba404d P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 13:21:23.990181 21424 raft_consensus.cc:515] T 26354361e9b044cea1c2b86c4882fe92 P cb82cc7bd6db46d980c75237a1402860 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:23.367754 21633 raft_consensus.cc:515] T 8d4ab870a05347868956aa8b841b2981 P 90f3da57796b488c9087a2f523adf045 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:23.965232 16221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c11074579b046d99e585c2818e18a2c" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:23.992257 16817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5718c623ebee4d2bb770f0f9ddb2f164" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
W20260709 13:21:23.993448 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 60352.1us
I20260709 13:21:23.948905 21619 raft_consensus.cc:493] T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:23.977130 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada5a004ba7c46819051e3ccec71ffdb" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:21:23.973258 21645 leader_election.cc:290] T 434bff3082d141838068f323bdb0d1ed P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 15 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:23.978315 21313 leader_election.cc:290] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 13 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:23.996522 21635 raft_consensus.cc:3060] T 8a878999c0024d2abe517484180290fc P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:21:24.001806 21619 raft_consensus.cc:3060] T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:21:24.007786 16817 raft_consensus.cc:2393] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 14: Already voted for candidate cb82cc7bd6db46d980c75237a1402860 in this term.
I20260709 13:21:23.997346 21507 leader_election.cc:290] T ecc1e363230043fe9fef390f0b979f20 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:24.012395 16295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66ae176961b74813a58bc15c9c4c6f2a" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:24.017374 16221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66ae176961b74813a58bc15c9c4c6f2a" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:24.018685 16221 raft_consensus.cc:2468] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 17.
I20260709 13:21:24.020030 16790 leader_election.cc:304] T 3a7ca61592e848ea9b0fb81a2ce33cdc P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:21:24.021893 16790 leader_election.cc:304] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:21:24.020085 16226 raft_consensus.cc:2393] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 in current term 15: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
I20260709 13:21:24.017388 21245 raft_consensus.cc:493] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:21:24.036150 21245 raft_consensus.cc:515] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:24.039336 16820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee9ed67006144797b1de6ea8c2013a68" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:24.041095 16294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a9de347c711477e87c582d003f36925" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:24.041803 16294 raft_consensus.cc:2468] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 17.
I20260709 13:21:24.041929 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e9b2bf7011f4ae688a79d7bb5a447b4" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:21:24.039561 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5718c623ebee4d2bb770f0f9ddb2f164" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:21:24.022809 21570 leader_election.cc:290] T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:24.022197 21675 raft_consensus.cc:493] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:21:24.061188 16298 raft_consensus.cc:1240] T ecc1e363230043fe9fef390f0b979f20 P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 15. Current term is 16. Ops: []
I20260709 13:21:24.025571 21313 raft_consensus.cc:493] T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:21:24.071628 16226 raft_consensus.cc:3060] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Advancing to term 14
I20260709 13:21:24.078570 16819 raft_consensus.cc:1275] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860 [term 17 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 13:21:24.061925 21245 leader_election.cc:290] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:24.095737 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87ac0d1f7e8a419f9bcbc04aee0e067e" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:21:24.070843 21523 raft_consensus.cc:2749] T 3a7ca61592e848ea9b0fb81a2ce33cdc P cb82cc7bd6db46d980c75237a1402860 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:21:24.071081 21671 raft_consensus.cc:2749] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:21:24.082990 21570 raft_consensus.cc:493] T 3884f864a96d428785895e5463e5728f P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:21:24.110126 21424 leader_election.cc:290] T 26354361e9b044cea1c2b86c4882fe92 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 13 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:24.119534 16115 leader_election.cc:304] T 1df93779209748b9860f6d893dd3de53 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:21:24.120857 16825 raft_consensus.cc:606] T 3884f864a96d428785895e5463e5728f P cb82cc7bd6db46d980c75237a1402860 [term 9 LEADER]: Received request to transfer leadership to TS 90f3da57796b488c9087a2f523adf045
I20260709 13:21:24.124064 16295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ad5dbfa99ee4b8d9e28be92b3f36db5" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:24.125377 16295 raft_consensus.cc:2393] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 13: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
W20260709 13:21:24.128851 16145 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40830 (request call id 1725) took 1692 ms (client timeout 2003 ms). Trace:
W20260709 13:21:24.129351 16145 rpcz_store.cc:269] 0709 13:21:22.436015 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:22.512908 (+ 76893us) service_pool.cc:224] Handling call
0709 13:21:24.128368 (+1615460us) spinlock_profiling.cc:243] Waited 280 ms on lock 0x7b540028e454. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:21:24.128659 (+   291us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":643214720}
I20260709 13:21:24.136113 16147 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40844 (request call id 2593) took 1212 ms. Trace:
I20260709 13:21:24.136684 16147 rpcz_store.cc:276] 0709 13:21:22.923298 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:22.923467 (+   169us) service_pool.cc:224] Handling call
0709 13:21:24.135627 (+1212160us) spinlock_profiling.cc:243] Waited 133 ms on lock 0x7b54000ea454. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:21:24.135900 (+   273us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":306231168}
I20260709 13:21:24.110814 21245 raft_consensus.cc:493] T 57a744f5aaf944678a63469cbf514015 P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:21:24.139905 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e4bfb9090c8460eb14fa7e0d818e7f1" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:24.139744 21245 raft_consensus.cc:515] T 57a744f5aaf944678a63469cbf514015 P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:24.098577 21675 raft_consensus.cc:515] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:21:24.141831 16142 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40830 (request call id 1724) took 1706 ms (client timeout 2201 ms). Trace:
I20260709 13:21:24.142410 21245 leader_election.cc:290] T 57a744f5aaf944678a63469cbf514015 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:24.116148 16298 raft_consensus.cc:3060] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:21:24.143467 16823 raft_consensus.cc:1240] T 19baa7bc367844c288bca8f81439c413 P cb82cc7bd6db46d980c75237a1402860 [term 17 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 16. Current term is 17. Ops: [16.10-16.10]
I20260709 13:21:24.140965 16300 raft_consensus.cc:2410] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 for term 15 because replica has last-logged OpId of term: 13 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 10.
I20260709 13:21:24.144830 16191 leader_election.cc:400] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 13 pre-election: Vote denied by peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741) with higher term. Message: Invalid argument: T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac for earlier term 13. Current term is 14.
I20260709 13:21:24.145128 16191 leader_election.cc:403] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 13 pre-election: Cancelling election due to peer responding with higher term
W20260709 13:21:24.148077 21676 log.cc:927] Time spent T c0eab25ce4c64e018a041beeda2867db P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.099s	user 0.001s	sys 0.000s
I20260709 13:21:24.148981 21182 consensus_queue.cc:1059] T 5a959dbecc2648eb85d1cc550f5a3245 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:21:24.150422 21182 raft_consensus.cc:3055] T 5a959dbecc2648eb85d1cc550f5a3245 P 1ca2725d290e47d68b1a4b609c9816ac [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:21:24.150844 21182 raft_consensus.cc:740] T 5a959dbecc2648eb85d1cc550f5a3245 P 1ca2725d290e47d68b1a4b609c9816ac [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:21:24.142787 21675 leader_election.cc:290] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:21:24.154088 21569 leader_election.cc:290] T 746aabf856ab401c9ef5e859371e7b67 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:24.156800 21468 leader_election.cc:290] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:24.157001 21613 raft_consensus.cc:493] T 26354361e9b044cea1c2b86c4882fe92 P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:21:24.105311 21632 raft_consensus.cc:493] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:21:24.158350 16298 raft_consensus.cc:2468] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 16.
I20260709 13:21:24.162087 21097 raft_consensus.cc:515] T ee9ed67006144797b1de6ea8c2013a68 P 647a343a542d4a139fe106c19095129b [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:24.191972 21097 leader_election.cc:290] T ee9ed67006144797b1de6ea8c2013a68 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 14 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:24.111460 21313 raft_consensus.cc:515] T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:24.195245 21313 leader_election.cc:290] T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
W20260709 13:21:24.167474 16114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:24.195642 16220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87ac0d1f7e8a419f9bcbc04aee0e067e" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
W20260709 13:21:24.142256 16142 rpcz_store.cc:269] 0709 13:21:22.435709 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:22.517211 (+ 81502us) service_pool.cc:224] Handling call
0709 13:21:24.141478 (+1624267us) spinlock_profiling.cc:243] Waited 340 ms on lock 0x7b54000d0554. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:21:24.141677 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":781477632}
I20260709 13:21:24.196504 16220 raft_consensus.cc:3060] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:21:24.197029 16150 raft_consensus.cc:1240] T ee9ed67006144797b1de6ea8c2013a68 P 647a343a542d4a139fe106c19095129b [term 14 FOLLOWER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 13. Current term is 14. Ops: []
I20260709 13:21:24.199599 16144 raft_consensus.cc:1275] T e5ad4eb281564936a59f41e387a47156 P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 15 index: 10. (term mismatch)
I20260709 13:21:24.200083 16144 pending_rounds.cc:85] T e5ad4eb281564936a59f41e387a47156 P 647a343a542d4a139fe106c19095129b: Aborting all ops after (but not including) 9
I20260709 13:21:24.200323 16144 pending_rounds.cc:107] T e5ad4eb281564936a59f41e387a47156 P 647a343a542d4a139fe106c19095129b: Aborting uncommitted NO_OP operation due to leader change: 14.10
I20260709 13:21:24.200603 16144 raft_consensus.cc:2889] T e5ad4eb281564936a59f41e387a47156 P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:21:24.176703 16146 raft_consensus.cc:2468] T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 13.
W20260709 13:21:24.186834 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 165096us
I20260709 13:21:24.201828 16790 leader_election.cc:304] T 829b4ad8c5a54eababeea1181b960d4b P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
W20260709 13:21:24.202732 16115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 107392us
I20260709 13:21:24.203640 21489 raft_consensus.cc:2749] T 1df93779209748b9860f6d893dd3de53 P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:21:24.203756 16115 leader_election.cc:304] T 9e65676b21b748728ad39668cfe722ec P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:21:24.205839 21580 raft_consensus.cc:2749] T 829b4ad8c5a54eababeea1181b960d4b P cb82cc7bd6db46d980c75237a1402860 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:21:24.202757 16148 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40830 (request call id 1720) took 1768 ms. Trace:
I20260709 13:21:24.206820 16148 rpcz_store.cc:276] 0709 13:21:22.434136 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:22.434280 (+   144us) service_pool.cc:224] Handling call
0709 13:21:22.498103 (+ 63823us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:21:23.694143 (+1196040us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:24.202415 (+508272us) spinlock_profiling.cc:243] Waited 915 ms on lock 0x7b5400427a54. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:21:24.202673 (+   258us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2104119168}
I20260709 13:21:24.209122 16825 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:37818 (request call id 1) took 1359 ms. Trace:
I20260709 13:21:24.209923 21569 raft_consensus.cc:2804] T 9e65676b21b748728ad39668cfe722ec P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:21:24.210388 21569 raft_consensus.cc:493] T 9e65676b21b748728ad39668cfe722ec P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Starting leader election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
W20260709 13:21:24.210707 16191 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:24.210779 21569 raft_consensus.cc:3060] T 9e65676b21b748728ad39668cfe722ec P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:21:24.213068 21324 raft_consensus.cc:3060] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:21:24.213820 21151 consensus_queue.cc:1048] T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:21:24.215505 21601 raft_consensus.cc:493] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:24.215906 21151 raft_consensus.cc:493] T 5a96373bb0e14dce8a266b2fc6c58507 P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:21:24.216423 21601 raft_consensus.cc:515] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:24.216724 21151 raft_consensus.cc:515] T 5a96373bb0e14dce8a266b2fc6c58507 P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:24.218822 21601 leader_election.cc:290] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:24.204931 16790 leader_election.cc:304] T c9944d5e4b8c4109af9955c4f08c9095 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:21:24.228341 21619 raft_consensus.cc:515] T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:24.231017 21543 raft_consensus.cc:493] T e5ad4eb281564936a59f41e387a47156 P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:24.231101 16790 leader_election.cc:304] T 19baa7bc367844c288bca8f81439c413 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 647a343a542d4a139fe106c19095129b
I20260709 13:21:24.231781 16820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e4bfb9090c8460eb14fa7e0d818e7f1" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:24.231931 21543 raft_consensus.cc:515] T e5ad4eb281564936a59f41e387a47156 P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:24.232775 16820 raft_consensus.cc:2468] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P cb82cc7bd6db46d980c75237a1402860 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 14.
I20260709 13:21:24.234231 21434 raft_consensus.cc:2749] T c9944d5e4b8c4109af9955c4f08c9095 P cb82cc7bd6db46d980c75237a1402860 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:21:24.234793 16790 leader_election.cc:304] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:21:24.224822 16217 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57604 (request call id 2865) took 1266 ms. Trace:
I20260709 13:21:24.224732 21097 raft_consensus.cc:493] T ecc1e363230043fe9fef390f0b979f20 P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:24.227362 21245 consensus_queue.cc:1059] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:21:24.209599 16825 rpcz_store.cc:276] 0709 13:21:22.849758 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:22.864547 (+ 14789us) service_pool.cc:224] Handling call
0709 13:21:24.208809 (+1344262us) spinlock_profiling.cc:243] Waited 1.18 s on lock 0x7b54006bf654. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b59853 00007f69b5b2760a 00007f69af0ad7d4 00007f69af0ad762 00007f69af0ad691 00007f69af0ad60c 00007f69af0ac8c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:21:24.208986 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2720113536}
I20260709 13:21:24.235657 16217 rpcz_store.cc:276] 0709 13:21:22.957979 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:23.008220 (+ 50241us) service_pool.cc:224] Handling call
0709 13:21:24.222966 (+1214746us) spinlock_profiling.cc:243] Waited 1.02 s on lock 0x7b54002986d4. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:21:24.223157 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2345346304}
I20260709 13:21:24.230904 16823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5d59407eb545b895d8d922e0dd69fd" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:24.235200 21434 raft_consensus.cc:2804] T 19baa7bc367844c288bca8f81439c413 P cb82cc7bd6db46d980c75237a1402860 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:21:24.249791 16790 leader_election.cc:304] T ada5a004ba7c46819051e3ccec71ffdb P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:21:24.264925 16823 raft_consensus.cc:2393] T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 16: Already voted for candidate cb82cc7bd6db46d980c75237a1402860 in this term.
I20260709 13:21:24.230933 21619 leader_election.cc:290] T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 16 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:21:24.272580 16296 raft_consensus.cc:2468] T 2794af5b99984efebdee564035841cb0 P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 12.
I20260709 13:21:24.261624 21543 leader_election.cc:290] T e5ad4eb281564936a59f41e387a47156 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:24.277490 16825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5718c623ebee4d2bb770f0f9ddb2f164" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:24.284474 16294 raft_consensus.cc:1240] T 2794af5b99984efebdee564035841cb0 P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 11. Current term is 12. Ops: []
I20260709 13:21:24.286259 16825 raft_consensus.cc:2368] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac for earlier term 13. Current term is 14.
W20260709 13:21:24.287297 16117 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 45960.5us
I20260709 13:21:24.284072 21648 raft_consensus.cc:493] T cdbd78acb4cf46298e0de8ba0d130e19 P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:21:24.261476 21434 raft_consensus.cc:697] T 19baa7bc367844c288bca8f81439c413 P cb82cc7bd6db46d980c75237a1402860 [term 17 LEADER]: Becoming Leader. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:21:24.250603 21632 raft_consensus.cc:515] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:24.292886 21623 raft_consensus.cc:2804] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:21:24.293462 16823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b7d1ab519044eda2d5e04a07242f25" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:24.270102 21245 consensus_queue.cc:1048] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.029s
I20260709 13:21:24.295861 21490 raft_consensus.cc:3055] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:21:24.296989 21434 consensus_queue.cc:237] T 19baa7bc367844c288bca8f81439c413 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 10, Current term: 17, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:24.248970 21182 consensus_queue.cc:260] T 5a959dbecc2648eb85d1cc550f5a3245 P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:24.289448 21635 raft_consensus.cc:515] T 8a878999c0024d2abe517484180290fc P 647a343a542d4a139fe106c19095129b [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:24.315104 16149 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40844 (request call id 2611) took 1134 ms. Trace:
I20260709 13:21:24.272540 21097 raft_consensus.cc:515] T ecc1e363230043fe9fef390f0b979f20 P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:24.321305 16825 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4e9b2bf7011f4ae688a79d7bb5a447b4"
dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
mode: GRACEFUL
new_leader_uuid: "90f3da57796b488c9087a2f523adf045"
 from {username='slave'} at 127.0.0.1:37818
I20260709 13:21:24.342693 16226 raft_consensus.cc:2468] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 14.
I20260709 13:21:24.280818 21523 raft_consensus.cc:2749] T ada5a004ba7c46819051e3ccec71ffdb P cb82cc7bd6db46d980c75237a1402860 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:21:24.275672 21468 raft_consensus.cc:493] T 1df93779209748b9860f6d893dd3de53 P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
W20260709 13:21:24.344556 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 338579us
W20260709 13:21:24.330298 16115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 49206.5us
I20260709 13:21:24.328500 16823 raft_consensus.cc:2468] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 12.
I20260709 13:21:24.267483 21570 raft_consensus.cc:515] T 3884f864a96d428785895e5463e5728f P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:24.335330 21480 raft_consensus.cc:493] T 08c74490c40243a88e62d1760b5cd814 P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:21:24.267199 21613 raft_consensus.cc:515] T 26354361e9b044cea1c2b86c4882fe92 P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:24.343495 16825 raft_consensus.cc:606] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860 [term 13 FOLLOWER]: Received request to transfer leadership to TS 90f3da57796b488c9087a2f523adf045
I20260709 13:21:24.328123 21623 raft_consensus.cc:493] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:21:24.350605 16825 raft_consensus.cc:614] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860 [term 13 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:21:24.334722 21619 raft_consensus.cc:493] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:24.336273 21490 raft_consensus.cc:740] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:21:24.354408 16115 leader_election.cc:304] T d16dfad0c3b540e5b4b86492ddf87208 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:21:24.356593 16115 leader_election.cc:304] T 5a959dbecc2648eb85d1cc550f5a3245 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:21:24.357751 16221 raft_consensus.cc:1240] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 14. Current term is 15. Ops: []
I20260709 13:21:24.345245 21468 raft_consensus.cc:515] T 1df93779209748b9860f6d893dd3de53 P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:21:24.360731 16789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 269169us
I20260709 13:21:24.339929 21604 raft_consensus.cc:493] T 02838c86531d4f98bf58d1c22b8baf77 P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:21:24.359006 16823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee9ed67006144797b1de6ea8c2013a68" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:24.362226 21588 raft_consensus.cc:2804] T d16dfad0c3b540e5b4b86492ddf87208 P 647a343a542d4a139fe106c19095129b [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:21:24.337857 21444 raft_consensus.cc:493] T c9944d5e4b8c4109af9955c4f08c9095 P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:21:24.355201 21623 raft_consensus.cc:3060] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:21:24.374850 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0eab25ce4c64e018a041beeda2867db" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:21:24.375794 16789 leader_election.cc:304] T 8c11074579b046d99e585c2818e18a2c P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac
I20260709 13:21:24.378053 21633 leader_election.cc:290] T 8d4ab870a05347868956aa8b841b2981 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 14 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:21:24.378350 21592 raft_consensus.cc:2749] T 5a959dbecc2648eb85d1cc550f5a3245 P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:21:24.378683 16222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62fe3346cd3c42c4a55e14af9073bca4" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:21:24.379613 16222 raft_consensus.cc:2368] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b for earlier term 18. Current term is 19.
I20260709 13:21:24.379515 16223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d16dfad0c3b540e5b4b86492ddf87208" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:24.380877 16225 raft_consensus.cc:1240] T 7c2095a5f4ad49408dbc7f69c606120d P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 10. Current term is 11. Ops: []
I20260709 13:21:24.381031 21151 leader_election.cc:290] T 5a96373bb0e14dce8a266b2fc6c58507 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:24.381712 21480 raft_consensus.cc:515] T 08c74490c40243a88e62d1760b5cd814 P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:24.382333 16296 raft_consensus.cc:1240] T 8d4ab870a05347868956aa8b841b2981 P 90f3da57796b488c9087a2f523adf045 [term 14 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 13. Current term is 14. Ops: []
I20260709 13:21:24.383595 21626 consensus_queue.cc:260] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:24.384336 21632 leader_election.cc:290] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:21:24.384848 21626 raft_consensus.cc:3060] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:21:24.386242 16221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5718c623ebee4d2bb770f0f9ddb2f164" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:21:24.341722 16149 rpcz_store.cc:276] 0709 13:21:23.180749 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:23.868140 (+687391us) service_pool.cc:224] Handling call
0709 13:21:23.931152 (+ 63012us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:21:23.970925 (+ 39773us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 13:21:23.971442 (+   517us) log.cc:844] Serialized 14 byte log entry
0709 13:21:24.260737 (+289295us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:21:24.260815 (+    78us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:21:24.260886 (+    71us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:24.314878 (+ 53992us) spinlock_profiling.cc:243] Waited 39.4 ms on lock 0x7b540006dbd4. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:21:24.314921 (+    43us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:21:24.315049 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3245,"prepare.queue_time_us":897,"prepare.run_cpu_time_us":50,"prepare.run_wall_time_us":51,"raft.queue_time_us":4430,"raft.run_cpu_time_us":161,"raft.run_wall_time_us":11626,"spinlock_wait_cycles":90641152,"thread_start_us":285415,"threads_started":1}
I20260709 13:21:24.386700 16819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "829b4ad8c5a54eababeea1181b960d4b" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:24.393571 21283 raft_consensus.cc:2749] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Leader pre-election lost for term 13. Reason: Vote denied by peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741) with higher term. Message: Invalid argument: T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac for earlier term 13. Current term is 14.
I20260709 13:21:24.369099 21588 raft_consensus.cc:493] T d16dfad0c3b540e5b4b86492ddf87208 P 647a343a542d4a139fe106c19095129b [term 14 FOLLOWER]: Starting leader election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:21:24.397967 21588 raft_consensus.cc:3060] T d16dfad0c3b540e5b4b86492ddf87208 P 647a343a542d4a139fe106c19095129b [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:21:24.397413 21490 consensus_queue.cc:260] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:24.398969 21490 raft_consensus.cc:3060] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:21:24.401818 21097 leader_election.cc:290] T ecc1e363230043fe9fef390f0b979f20 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:24.403057 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f5f89186204242bf377f7c42b2e2da" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:21:24.403769 16190 leader_election.cc:304] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:21:24.404666 21507 raft_consensus.cc:2804] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:21:24.405066 21507 raft_consensus.cc:493] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: Starting leader election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:21:24.376904 16117 leader_election.cc:304] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860
I20260709 13:21:24.407181 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c74490c40243a88e62d1760b5cd814" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:24.407980 16226 raft_consensus.cc:2410] T 08c74490c40243a88e62d1760b5cd814 P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 for term 15 because replica has last-logged OpId of term: 14 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 9.
I20260709 13:21:24.411374 16221 raft_consensus.cc:2393] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 14: Already voted for candidate cb82cc7bd6db46d980c75237a1402860 in this term.
I20260709 13:21:24.414016 16220 raft_consensus.cc:2468] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 16.
I20260709 13:21:24.416577 16220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02838c86531d4f98bf58d1c22b8baf77" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
W20260709 13:21:24.416700 16115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 60196.6us
I20260709 13:21:24.418316 16823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8638ee363273426db1393dd81e5c7b72" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:24.405642 21507 raft_consensus.cc:3060] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:21:24.361830 21648 raft_consensus.cc:515] T cdbd78acb4cf46298e0de8ba0d130e19 P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:24.367930 21619 raft_consensus.cc:515] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:24.413200 21580 raft_consensus.cc:2804] T 8c11074579b046d99e585c2818e18a2c P cb82cc7bd6db46d980c75237a1402860 [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 13:21:24.425804 16789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 54950.7us
I20260709 13:21:24.429347 21580 raft_consensus.cc:493] T 8c11074579b046d99e585c2818e18a2c P cb82cc7bd6db46d980c75237a1402860 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:21:24.429796 21580 raft_consensus.cc:3060] T 8c11074579b046d99e585c2818e18a2c P cb82cc7bd6db46d980c75237a1402860 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:21:24.431396 21097 raft_consensus.cc:2749] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:21:24.432052 21633 raft_consensus.cc:493] T a378d7f796f94615b3b347119a9880fd P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:21:24.432637 21633 raft_consensus.cc:515] T a378d7f796f94615b3b347119a9880fd P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:24.434803 21633 leader_election.cc:290] T a378d7f796f94615b3b347119a9880fd P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:24.436136 21480 leader_election.cc:290] T 08c74490c40243a88e62d1760b5cd814 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:24.436421 21468 leader_election.cc:290] T 1df93779209748b9860f6d893dd3de53 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:21:24.434477 21588 raft_consensus.cc:515] T d16dfad0c3b540e5b4b86492ddf87208 P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:24.388191 21604 raft_consensus.cc:515] T 02838c86531d4f98bf58d1c22b8baf77 P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:24.440891 21643 raft_consensus.cc:493] T d2ab0ac8347641e6955e9bce7d90c1a2 P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:21:24.447358 16825 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5718c623ebee4d2bb770f0f9ddb2f164"
dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
mode: GRACEFUL
new_leader_uuid: "647a343a542d4a139fe106c19095129b"
 from {username='slave'} at 127.0.0.1:37818
I20260709 13:21:24.443984 16294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c2095a5f4ad49408dbc7f69c606120d" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:21:24.449777 16825 raft_consensus.cc:606] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 [term 14 FOLLOWER]: Received request to transfer leadership to TS 647a343a542d4a139fe106c19095129b
I20260709 13:21:24.446020 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fffab26427e24d398e2695aad40ca134" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:21:24.447585 16823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af57996a8be84d398f0f023e8d08ef77" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:24.450122 16825 raft_consensus.cc:614] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 [term 14 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:21:24.448518 16817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 16 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:24.453867 16225 raft_consensus.cc:1240] T c2f5f89186204242bf377f7c42b2e2da P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 15. Current term is 16. Ops: []
W20260709 13:21:24.454073 21682 log.cc:927] Time spent T 8a878999c0024d2abe517484180290fc P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.172s	user 0.003s	sys 0.002s
I20260709 13:21:24.454319 16223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68a6838d82ef4cac83507f906139d90f" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:24.454973 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c2095a5f4ad49408dbc7f69c606120d" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:21:24.454080 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87ac0d1f7e8a419f9bcbc04aee0e067e" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:21:24.455693 16225 raft_consensus.cc:2393] T 7c2095a5f4ad49408dbc7f69c606120d P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 11: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
I20260709 13:21:24.455957 16298 raft_consensus.cc:2368] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac for earlier term 15. Current term is 16.
I20260709 13:21:24.456532 16222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87ac0d1f7e8a419f9bcbc04aee0e067e" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
W20260709 13:21:24.457008 16265 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:24.457263 16222 raft_consensus.cc:2393] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 16: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
W20260709 13:21:24.457895 16265 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:24.460327 16115 leader_election.cc:304] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:21:24.419647 21444 raft_consensus.cc:515] T c9944d5e4b8c4109af9955c4f08c9095 P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:24.457568 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c61a39fd4f84424b129c1a6f9ba404d" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:24.462304 16221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8158bbd17b994d2eb96f560bd0f1b88d" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:24.466818 16819 raft_consensus.cc:1240] T c9944d5e4b8c4109af9955c4f08c9095 P cb82cc7bd6db46d980c75237a1402860 [term 16 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 15. Current term is 16. Ops: []
I20260709 13:21:24.468801 16220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a959dbecc2648eb85d1cc550f5a3245" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
W20260709 13:21:24.472360 16117 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 111231us
I20260709 13:21:24.472999 16790 leader_election.cc:304] T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:21:24.477753 16298 raft_consensus.cc:1240] T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 15. Current term is 16. Ops: []
I20260709 13:21:24.461467 21569 raft_consensus.cc:515] T 9e65676b21b748728ad39668cfe722ec P 647a343a542d4a139fe106c19095129b [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:24.486060 16115 leader_election.cc:304] T 2794af5b99984efebdee564035841cb0 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: cb82cc7bd6db46d980c75237a1402860
I20260709 13:21:24.453111 21480 raft_consensus.cc:493] T 66f75124215f4ca6bef831e174a792d8 P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:21:24.492094 21468 raft_consensus.cc:476] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045 [term 14 LEADER]: Not starting pre-election -- already a leader
I20260709 13:21:24.505350 16219 raft_consensus.cc:2468] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 17.
I20260709 13:21:24.507269 21084 raft_consensus.cc:493] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:24.507735 21084 raft_consensus.cc:515] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:24.509151 21619 leader_election.cc:290] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:24.511898 21084 leader_election.cc:290] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:24.514870 21444 leader_election.cc:290] T c9944d5e4b8c4109af9955c4f08c9095 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
W20260709 13:21:24.515151 16219 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33050 (request call id 922) took 1573 ms (client timeout 1975 ms). Trace:
I20260709 13:21:24.515765 16144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806d7690c2b644959152074bb2cab0a7" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:21:24.516440 16144 raft_consensus.cc:3060] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:21:24.517446 16148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 16 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:24.518263 16148 raft_consensus.cc:2410] T ecc1e363230043fe9fef390f0b979f20 P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac for term 16 because replica has last-logged OpId of term: 13 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 8.
W20260709 13:21:24.515592 16219 rpcz_store.cc:269] 0709 13:21:22.958524 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:23.052505 (+ 93981us) service_pool.cc:224] Handling call
0709 13:21:24.514947 (+1462442us) spinlock_profiling.cc:243] Waited 1.36 s on lock 0x7b54001423d4. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:21:24.515082 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3134118144}
I20260709 13:21:24.519516 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2372981006d4b218215c28964dea623" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:24.520112 16150 raft_consensus.cc:2468] T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 16.
I20260709 13:21:24.520521 21543 raft_consensus.cc:2804] T 2794af5b99984efebdee564035841cb0 P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:21:24.520807 21632 raft_consensus.cc:2749] T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:21:24.521070 21543 raft_consensus.cc:697] T 2794af5b99984efebdee564035841cb0 P 647a343a542d4a139fe106c19095129b [term 12 LEADER]: Becoming Leader. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:21:24.522106 21543 consensus_queue.cc:237] T 2794af5b99984efebdee564035841cb0 P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:24.483888 21643 raft_consensus.cc:515] T d2ab0ac8347641e6955e9bce7d90c1a2 P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:24.525604 21643 leader_election.cc:290] T d2ab0ac8347641e6955e9bce7d90c1a2 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:24.530113 16149 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "62fe3346cd3c42c4a55e14af9073bca4"
dest_uuid: "647a343a542d4a139fe106c19095129b"
mode: GRACEFUL
new_leader_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
 from {username='slave'} at 127.0.0.1:40290
I20260709 13:21:24.530714 16149 raft_consensus.cc:606] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Received request to transfer leadership to TS 1ca2725d290e47d68b1a4b609c9816ac
I20260709 13:21:24.531131 16149 raft_consensus.cc:614] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:21:24.506445 16218 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33050 (request call id 915) took 1571 ms. Trace:
I20260709 13:21:24.535027 16789 leader_election.cc:304] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:21:24.486241 21097 raft_consensus.cc:2804] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:21:24.544737 16222 raft_consensus.cc:1240] T 19baa7bc367844c288bca8f81439c413 P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 16. Current term is 17. Ops: [16.10-16.10]
I20260709 13:21:24.545849 21613 leader_election.cc:290] T 26354361e9b044cea1c2b86c4882fe92 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
W20260709 13:21:24.548477 16116 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:24.547053 16145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a96373bb0e14dce8a266b2fc6c58507" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:24.558880 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:21:24.559733 16226 raft_consensus.cc:3060] T ecc1e363230043fe9fef390f0b979f20 P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:21:24.484512 21570 leader_election.cc:290] T 3884f864a96d428785895e5463e5728f P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:24.533334 16218 rpcz_store.cc:276] 0709 13:21:22.935309 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:22.935514 (+   205us) service_pool.cc:224] Handling call
0709 13:21:22.935910 (+   396us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:21:22.945565 (+  9655us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:21:22.945628 (+    63us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:21:24.131787 (+1186159us) spinlock_profiling.cc:243] Waited 2.17 ms on lock 0x7b18004cec40. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b693ade1 00007f69b69475b1 00007f69ada402a6 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69af458a11 00007f69af468372 00007f69b0b9aa01 00007f69b0b5f43a
0709 13:21:24.132829 (+  1042us) spinlock_profiling.cc:243] Waited 5.01 ms on lock 0x7b2c00115ac8. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69af458a11 00007f69af468372 00007f69b0b9aa01 00007f69b0b5f43a 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2
0709 13:21:24.133837 (+  1008us) log.cc:844] Serialized 52 byte log entry
0709 13:21:24.386428 (+252591us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:21:24.386463 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:24.391292 (+  4829us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:21:24.506004 (+114712us) raft_consensus.cc:1685] finished
0709 13:21:24.506026 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:21:24.506242 (+   216us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":9092,"raft.queue_time_us":9225,"raft.run_cpu_time_us":184,"raft.run_wall_time_us":37874,"spinlock_wait_cycles":16507392,"thread_start_us":239141,"threads_started":1,"wal-append.queue_time_us":350138}
I20260709 13:21:24.574829 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c74490c40243a88e62d1760b5cd814" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:24.575670 16142 raft_consensus.cc:1240] T d16dfad0c3b540e5b4b86492ddf87208 P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 14. Current term is 15. Ops: []
I20260709 13:21:24.553759 21632 raft_consensus.cc:2804] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:21:24.547250 21648 leader_election.cc:290] T cdbd78acb4cf46298e0de8ba0d130e19 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:24.563712 21635 leader_election.cc:290] T 8a878999c0024d2abe517484180290fc P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 14 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:24.565889 21097 raft_consensus.cc:697] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 647a343a542d4a139fe106c19095129b [term 16 LEADER]: Becoming Leader. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
W20260709 13:21:24.592094 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 123921us
I20260709 13:21:24.595014 16192 leader_election.cc:304] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 
W20260709 13:21:24.595078 21686 log.cc:927] Time spent T 746aabf856ab401c9ef5e859371e7b67 P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 0.076s	user 0.000s	sys 0.001s
I20260709 13:21:24.569521 21480 raft_consensus.cc:515] T 66f75124215f4ca6bef831e174a792d8 P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:24.603765 21097 consensus_queue.cc:237] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:24.604684 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ad5dbfa99ee4b8d9e28be92b3f36db5" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
W20260709 13:21:24.592355 16265 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.14.177.130:35943 timed out after 1.513s (SENT) [suppressed 24 similar messages]
I20260709 13:21:24.607785 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2372981006d4b218215c28964dea623" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:24.608462 16224 raft_consensus.cc:2468] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 16.
I20260709 13:21:24.608574 16218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a878999c0024d2abe517484180290fc" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:21:24.609264 16218 raft_consensus.cc:3060] T 8a878999c0024d2abe517484180290fc P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:21:24.612234 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57a744f5aaf944678a63469cbf514015" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:24.612990 16219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:24.613739 16292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f5f89186204242bf377f7c42b2e2da" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:24.609372 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5ad4eb281564936a59f41e387a47156" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:24.616307 16221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746aabf856ab401c9ef5e859371e7b67" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:24.616487 21507 raft_consensus.cc:515] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:24.618009 16222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d16dfad0c3b540e5b4b86492ddf87208" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:21:24.618446 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f5f89186204242bf377f7c42b2e2da" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:24.618644 21507 leader_election.cc:290] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 16 election: Requested vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:24.619123 16222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee9ed67006144797b1de6ea8c2013a68" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:21:24.583200 21632 raft_consensus.cc:493] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:24.627728 16144 raft_consensus.cc:2468] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 15.
I20260709 13:21:24.560763 21313 consensus_queue.cc:1059] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 6, Time since last communication: 0.032s
I20260709 13:21:24.555680 21643 consensus_queue.cc:1059] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: INVALID_TERM, Last received: 15.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.032s
I20260709 13:21:24.630615 16217 raft_consensus.cc:2393] T c2f5f89186204242bf377f7c42b2e2da P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 16: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
I20260709 13:21:24.637451 16790 leader_election.cc:304] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860; no voters: 90f3da57796b488c9087a2f523adf045
I20260709 13:21:24.631752 16148 raft_consensus.cc:1275] T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b [term 14 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 12 index: 6. Preceding OpId from leader: term: 14 index: 7. (index mismatch)
W20260709 13:21:24.646044 16192 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 98206.6us
I20260709 13:21:24.660368 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4dc6b0541ca431ebc757beac00521f1" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:24.661072 16150 raft_consensus.cc:2368] T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 for earlier term 13. Current term is 14.
I20260709 13:21:24.661525 16291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3884f864a96d428785895e5463e5728f" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:24.661402 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a878999c0024d2abe517484180290fc" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:21:24.662586 16144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fffab26427e24d398e2695aad40ca134" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:24.660588 16294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5ad4eb281564936a59f41e387a47156" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:24.662176 16295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26354361e9b044cea1c2b86c4882fe92" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:24.664685 16296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:24.667362 16296 raft_consensus.cc:2393] T ecc1e363230043fe9fef390f0b979f20 P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 16: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
I20260709 13:21:24.643407 21632 raft_consensus.cc:3060] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:21:24.675158 16189 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:24.676992 16298 raft_consensus.cc:1240] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 12. Current term is 13. Ops: []
I20260709 13:21:24.681221 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0eab25ce4c64e018a041beeda2867db" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:21:24.681380 16148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3884f864a96d428785895e5463e5728f" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:24.682289 16148 raft_consensus.cc:2410] T 3884f864a96d428785895e5463e5728f P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 for term 10 because replica has last-logged OpId of term: 9 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 9.
W20260709 13:21:24.683084 16115 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:24.683593 21313 raft_consensus.cc:3055] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:21:24.684044 21313 raft_consensus.cc:740] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:21:24.665170 21182 raft_consensus.cc:3060] T 5a959dbecc2648eb85d1cc550f5a3245 P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:21:24.687904 16150 raft_consensus.cc:2368] T c0eab25ce4c64e018a041beeda2867db P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 for earlier term 14. Current term is 15.
I20260709 13:21:24.678929 16192 leader_election.cc:304] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
I20260709 13:21:24.689240 15046 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:21:24.693763 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a378d7f796f94615b3b347119a9880fd" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
W20260709 13:21:24.694289 16115 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:24.694449 16293 raft_consensus.cc:2468] T a378d7f796f94615b3b347119a9880fd P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 16.
I20260709 13:21:24.637951 21579 consensus_queue.cc:1059] T ee9ed67006144797b1de6ea8c2013a68 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: INVALID_TERM, Last received: 13.12, Next index: 13, Last known committed idx: 10, Time since last communication: 0.033s
I20260709 13:21:24.671268 21635 raft_consensus.cc:493] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:24.691313 16294 raft_consensus.cc:2468] T e5ad4eb281564936a59f41e387a47156 P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 15.
I20260709 13:21:24.667490 21588 leader_election.cc:290] T d16dfad0c3b540e5b4b86492ddf87208 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 15 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:24.693120 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f5f89186204242bf377f7c42b2e2da" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:21:24.712900 16150 raft_consensus.cc:2376] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 15. Re-sending same reply.
I20260709 13:21:24.637977 21403 consensus_queue.cc:1048] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.042s
I20260709 13:21:24.664296 21626 raft_consensus.cc:493] T af57996a8be84d398f0f023e8d08ef77 P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:21:24.695490 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26354361e9b044cea1c2b86c4882fe92" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:21:24.709669 16819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746aabf856ab401c9ef5e859371e7b67" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
W20260709 13:21:24.721158 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 50859.1us
W20260709 13:21:24.722059 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 84699.5us
I20260709 13:21:24.729403 16819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2372981006d4b218215c28964dea623" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:24.676132 21084 raft_consensus.cc:2804] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:21:24.647826 21675 consensus_queue.cc:1059] T 434bff3082d141838068f323bdb0d1ed P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: INVALID_TERM, Last received: 14.12, Next index: 13, Last known committed idx: 10, Time since last communication: 0.045s
I20260709 13:21:24.740603 16819 raft_consensus.cc:2468] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 16.
I20260709 13:21:24.745258 16146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26354361e9b044cea1c2b86c4882fe92" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:21:24.750809 16146 raft_consensus.cc:3060] T 26354361e9b044cea1c2b86c4882fe92 P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:21:24.747949 16148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a959dbecc2648eb85d1cc550f5a3245" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:21:24.749056 16226 raft_consensus.cc:2468] T ecc1e363230043fe9fef390f0b979f20 P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 16.
I20260709 13:21:24.696763 21671 consensus_queue.cc:1059] T 2794af5b99984efebdee564035841cb0 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 8, Time since last communication: 0.048s
I20260709 13:21:24.751688 16115 leader_election.cc:304] T e5ad4eb281564936a59f41e387a47156 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:21:24.713405 21324 raft_consensus.cc:3060] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:21:24.719730 21569 leader_election.cc:290] T 9e65676b21b748728ad39668cfe722ec P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 14 election: Requested vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:24.761866 16148 raft_consensus.cc:2393] T 5a959dbecc2648eb85d1cc550f5a3245 P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 13: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
I20260709 13:21:24.762130 16145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e9b2bf7011f4ae688a79d7bb5a447b4" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:21:24.719321 21643 raft_consensus.cc:493] T 19baa7bc367844c288bca8f81439c413 P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:24.767553 16819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57a744f5aaf944678a63469cbf514015" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:24.761546 16147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b7d1ab519044eda2d5e04a07242f25" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:24.757287 16142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a9de347c711477e87c582d003f36925" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
W20260709 13:21:24.772168 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 160761us
I20260709 13:21:24.789373 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d16dfad0c3b540e5b4b86492ddf87208" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:21:24.790234 16293 raft_consensus.cc:3060] T d16dfad0c3b540e5b4b86492ddf87208 P 90f3da57796b488c9087a2f523adf045 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:21:24.737190 21084 raft_consensus.cc:493] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:24.772270 16296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e65676b21b748728ad39668cfe722ec" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:21:24.805647 16143 raft_consensus.cc:1240] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 14. Current term is 15. Ops: [14.10-14.10]
W20260709 13:21:24.808894 16192 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 143737us
I20260709 13:21:24.685149 21313 consensus_queue.cc:260] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:24.823486 16148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746aabf856ab401c9ef5e859371e7b67" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:24.824395 16820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee9ed67006144797b1de6ea8c2013a68" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:21:24.765283 21592 raft_consensus.cc:2804] T e5ad4eb281564936a59f41e387a47156 P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:21:24.758349 21604 leader_election.cc:290] T 02838c86531d4f98bf58d1c22b8baf77 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:24.835800 16145 raft_consensus.cc:3060] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:21:24.828022 16192 leader_election.cc:400] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 13 pre-election: Vote denied by peer cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923) with higher term. Message: Invalid argument: T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac for earlier term 13. Current term is 14.
I20260709 13:21:24.761090 21648 raft_consensus.cc:493] T 8cc959fb7f704fba9009c0595467d074 P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:21:24.829916 16116 leader_election.cc:400] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 18 election: Vote denied by peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943) with higher term. Message: Invalid argument: T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b for earlier term 18. Current term is 19.
I20260709 13:21:24.737360 21635 raft_consensus.cc:515] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:24.846971 16143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:21:24.806713 21084 raft_consensus.cc:3060] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:21:24.801466 21444 raft_consensus.cc:493] T 84f2da98864b4425bf7c0055951fdd4b P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:21:24.854951 16146 raft_consensus.cc:2468] T 26354361e9b044cea1c2b86c4882fe92 P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 13.
W20260709 13:21:24.856063 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 263724us
I20260709 13:21:24.861722 16820 raft_consensus.cc:1240] T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860 [term 16 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 15. Current term is 16. Ops: []
I20260709 13:21:24.880787 16819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:24.788515 21626 raft_consensus.cc:515] T af57996a8be84d398f0f023e8d08ef77 P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:24.879339 16296 raft_consensus.cc:1275] T e5ad4eb281564936a59f41e387a47156 P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 13:21:24.829217 21592 raft_consensus.cc:493] T e5ad4eb281564936a59f41e387a47156 P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:21:24.888002 16192 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 78185.6us
I20260709 13:21:24.857463 16143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02838c86531d4f98bf58d1c22b8baf77" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:24.895581 16143 raft_consensus.cc:2393] T 02838c86531d4f98bf58d1c22b8baf77 P 647a343a542d4a139fe106c19095129b [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 14: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
I20260709 13:21:24.877333 21619 raft_consensus.cc:493] T fffab26427e24d398e2695aad40ca134 P 90f3da57796b488c9087a2f523adf045 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:21:24.886677 21313 raft_consensus.cc:3060] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:21:24.908612 16147 raft_consensus.cc:1240] T 2794af5b99984efebdee564035841cb0 P 647a343a542d4a139fe106c19095129b [term 12 LEADER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 11. Current term is 12. Ops: []
I20260709 13:21:24.913309 16296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806d7690c2b644959152074bb2cab0a7" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:21:24.857678 21250 consensus_queue.cc:1048] T e5ad4eb281564936a59f41e387a47156 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.084s
I20260709 13:21:24.885978 21555 raft_consensus.cc:2804] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:21:24.914683 16820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3884f864a96d428785895e5463e5728f" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:24.889364 21592 raft_consensus.cc:3060] T e5ad4eb281564936a59f41e387a47156 P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:21:24.872915 21444 raft_consensus.cc:515] T 84f2da98864b4425bf7c0055951fdd4b P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:21:24.936605 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 76274.2us
W20260709 13:21:24.936866 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 132263us
I20260709 13:21:24.884045 21580 raft_consensus.cc:515] T 8c11074579b046d99e585c2818e18a2c P cb82cc7bd6db46d980c75237a1402860 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:24.878110 21648 raft_consensus.cc:515] T 8cc959fb7f704fba9009c0595467d074 P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:24.941905 16296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 16 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
W20260709 13:21:24.942116 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 186107us
I20260709 13:21:24.942871 16790 leader_election.cc:304] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 647a343a542d4a139fe106c19095129b
I20260709 13:21:24.859915 21480 leader_election.cc:290] T 66f75124215f4ca6bef831e174a792d8 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:24.944166 16147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ad5dbfa99ee4b8d9e28be92b3f36db5" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:21:24.948331 16296 raft_consensus.cc:2393] T ecc1e363230043fe9fef390f0b979f20 P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 16: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
I20260709 13:21:24.854230 21643 raft_consensus.cc:515] T 19baa7bc367844c288bca8f81439c413 P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:21:24.943609 16192 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 54774.9us
I20260709 13:21:24.954412 16820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26354361e9b044cea1c2b86c4882fe92" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:24.956409 16820 raft_consensus.cc:2393] T 26354361e9b044cea1c2b86c4882fe92 P cb82cc7bd6db46d980c75237a1402860 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 13: Already voted for candidate cb82cc7bd6db46d980c75237a1402860 in this term.
I20260709 13:21:24.957382 16191 leader_election.cc:304] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
I20260709 13:21:24.959290 16218 raft_consensus.cc:2468] T 8a878999c0024d2abe517484180290fc P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 14.
I20260709 13:21:24.967953 16116 leader_election.cc:304] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860
I20260709 13:21:24.970587 21507 raft_consensus.cc:3055] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 1ca2725d290e47d68b1a4b609c9816ac [term 15 LEADER]: Stepping down as leader of term 15
W20260709 13:21:24.970882 21695 log.cc:927] Time spent T 9c61a39fd4f84424b129c1a6f9ba404d P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 0.133s	user 0.001s	sys 0.000s
I20260709 13:21:24.971185 21507 raft_consensus.cc:740] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 1ca2725d290e47d68b1a4b609c9816ac [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:21:24.972286 21570 raft_consensus.cc:2749] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:21:24.972209 21507 consensus_queue.cc:260] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:24.973654 21507 raft_consensus.cc:3060] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:21:24.976895 15046 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:21:24.977567 16142 raft_consensus.cc:1240] T 7c2095a5f4ad49408dbc7f69c606120d P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 10. Current term is 11. Ops: [10.6-10.6]
W20260709 13:21:24.978935 16791 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:24.981905 21580 leader_election.cc:290] T 8c11074579b046d99e585c2818e18a2c P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 14 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:24.985257 21523 raft_consensus.cc:3055] T 434bff3082d141838068f323bdb0d1ed P cb82cc7bd6db46d980c75237a1402860 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:21:24.985759 21523 raft_consensus.cc:740] T 434bff3082d141838068f323bdb0d1ed P cb82cc7bd6db46d980c75237a1402860 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:21:24.991696 16147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d16dfad0c3b540e5b4b86492ddf87208" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:24.992887 16147 raft_consensus.cc:2393] T d16dfad0c3b540e5b4b86492ddf87208 P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 15: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
I20260709 13:21:24.980289 16116 leader_election.cc:304] T 02838c86531d4f98bf58d1c22b8baf77 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:21:24.994793 16116 leader_election.cc:304] T 7c2095a5f4ad49408dbc7f69c606120d P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:21:24.939087 21555 raft_consensus.cc:697] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860 [term 13 LEADER]: Becoming Leader. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:21:24.997473 21097 raft_consensus.cc:2749] T 02838c86531d4f98bf58d1c22b8baf77 P 647a343a542d4a139fe106c19095129b [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:21:24.998266 21097 raft_consensus.cc:2749] T 7c2095a5f4ad49408dbc7f69c606120d P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
W20260709 13:21:24.999598 21693 log.cc:927] Time spent T 3884f864a96d428785895e5463e5728f P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.178s	user 0.000s	sys 0.001s
I20260709 13:21:24.997125 21555 consensus_queue.cc:237] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:25.009536 16147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4f65cdf00b748bb963ee6e81ef43b79" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:25.010773 16147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434bff3082d141838068f323bdb0d1ed" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:21:25.008880 16192 leader_election.cc:304] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 647a343a542d4a139fe106c19095129b
I20260709 13:21:25.003937 16145 raft_consensus.cc:2468] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 13.
I20260709 13:21:25.005759 16116 leader_election.cc:304] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:21:25.012001 21635 leader_election.cc:290] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:25.011955 16147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1df93779209748b9860f6d893dd3de53" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:25.019786 21703 raft_consensus.cc:2804] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:21:24.950789 21619 raft_consensus.cc:515] T fffab26427e24d398e2695aad40ca134 P 90f3da57796b488c9087a2f523adf045 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:25.016638 16294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c74490c40243a88e62d1760b5cd814" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:25.026502 16144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d4ab870a05347868956aa8b841b2981" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:21:25.015816 16295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a96373bb0e14dce8a266b2fc6c58507" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:25.001971 21648 leader_election.cc:290] T 8cc959fb7f704fba9009c0595467d074 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:21:25.037586 16142 raft_consensus.cc:1240] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 12. Current term is 13. Ops: [12.10-12.10]
I20260709 13:21:25.041778 16293 raft_consensus.cc:2468] T d16dfad0c3b540e5b4b86492ddf87208 P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 15.
I20260709 13:21:25.049343 16296 raft_consensus.cc:1240] T d16dfad0c3b540e5b4b86492ddf87208 P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 14. Current term is 15. Ops: []
I20260709 13:21:25.039239 16116 leader_election.cc:304] T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
W20260709 13:21:25.043404 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 83122.3us
W20260709 13:21:25.019382 21692 log.cc:927] Time spent T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 0.158s	user 0.000s	sys 0.001s
I20260709 13:21:25.053386 16817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:21:25.055887 16145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18ad7e3af5504571a69d10c00750a01c" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
W20260709 13:21:25.050529 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 107738us
I20260709 13:21:25.037012 21283 raft_consensus.cc:2804] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:21:25.029151 21434 raft_consensus.cc:3055] T ee9ed67006144797b1de6ea8c2013a68 P cb82cc7bd6db46d980c75237a1402860 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:21:25.058257 16115 leader_election.cc:304] T d16dfad0c3b540e5b4b86492ddf87208 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:21:25.043367 21635 raft_consensus.cc:3060] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:21:25.065415 16147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c61a39fd4f84424b129c1a6f9ba404d" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:21:25.036517 21444 leader_election.cc:290] T 84f2da98864b4425bf7c0055951fdd4b P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:21:25.065564 21588 raft_consensus.cc:2804] T d16dfad0c3b540e5b4b86492ddf87208 P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:21:25.060985 21283 raft_consensus.cc:493] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:25.055012 21703 raft_consensus.cc:697] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 [term 14 LEADER]: Becoming Leader. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:21:24.986720 21523 consensus_queue.cc:260] T 434bff3082d141838068f323bdb0d1ed P cb82cc7bd6db46d980c75237a1402860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:25.081837 21708 raft_consensus.cc:2804] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:21:25.082459 21708 raft_consensus.cc:493] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Starting leader election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:21:25.073670 21283 raft_consensus.cc:3060] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:21:25.082947 21708 raft_consensus.cc:3060] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:21:25.081086 21097 consensus_queue.cc:1048] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.007s
I20260709 13:21:25.088368 16820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c11074579b046d99e585c2818e18a2c" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:25.091324 16823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434bff3082d141838068f323bdb0d1ed" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:21:25.092433 16825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e4bfb9090c8460eb14fa7e0d818e7f1" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:25.092659 21677 raft_consensus.cc:993] T 3884f864a96d428785895e5463e5728f P cb82cc7bd6db46d980c75237a1402860: : Instructing follower 90f3da57796b488c9087a2f523adf045 to start an election
I20260709 13:21:25.093154 16825 raft_consensus.cc:2468] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P cb82cc7bd6db46d980c75237a1402860 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 14.
I20260709 13:21:25.093355 21677 raft_consensus.cc:1081] T 3884f864a96d428785895e5463e5728f P cb82cc7bd6db46d980c75237a1402860 [term 9 LEADER]: Signalling peer 90f3da57796b488c9087a2f523adf045 to start an election
I20260709 13:21:25.061930 21434 raft_consensus.cc:740] T ee9ed67006144797b1de6ea8c2013a68 P cb82cc7bd6db46d980c75237a1402860 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:21:25.094664 21704 raft_consensus.cc:2804] T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:21:25.095211 21704 raft_consensus.cc:493] T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:25.095610 21704 raft_consensus.cc:3060] T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:21:25.095398 21434 consensus_queue.cc:260] T ee9ed67006144797b1de6ea8c2013a68 P cb82cc7bd6db46d980c75237a1402860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:25.096983 21434 raft_consensus.cc:3060] T ee9ed67006144797b1de6ea8c2013a68 P cb82cc7bd6db46d980c75237a1402860 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:21:25.099581 16192 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 139074us
I20260709 13:21:25.100418 16192 leader_election.cc:304] T ee9ed67006144797b1de6ea8c2013a68 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
I20260709 13:21:25.101806 16192 leader_election.cc:304] T 829b4ad8c5a54eababeea1181b960d4b P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
I20260709 13:21:25.103463 21626 leader_election.cc:290] T af57996a8be84d398f0f023e8d08ef77 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:25.103583 16116 leader_election.cc:304] T ecc1e363230043fe9fef390f0b979f20 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:21:25.104759 21626 raft_consensus.cc:3060] T ecc1e363230043fe9fef390f0b979f20 P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:21:25.109239 21643 leader_election.cc:290] T 19baa7bc367844c288bca8f81439c413 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
W20260709 13:21:25.082569 21705 log.cc:927] Time spent T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.069s	user 0.000s	sys 0.002s
I20260709 13:21:25.112805 21588 raft_consensus.cc:697] T d16dfad0c3b540e5b4b86492ddf87208 P 647a343a542d4a139fe106c19095129b [term 15 LEADER]: Becoming Leader. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:21:25.088274 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e4bfb9090c8460eb14fa7e0d818e7f1" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:25.113766 16299 raft_consensus.cc:2468] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 14.
I20260709 13:21:25.113909 21588 consensus_queue.cc:237] T d16dfad0c3b540e5b4b86492ddf87208 P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:25.080651 21703 consensus_queue.cc:237] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 12.12, Last appended by leader: 12, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:25.091068 16789 leader_election.cc:304] T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:21:25.124991 16819 raft_consensus.cc:2410] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b for term 16 because replica has last-logged OpId of term: 15 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 9.
I20260709 13:21:25.110694 16116 leader_election.cc:304] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:21:25.130761 16790 leader_election.cc:304] T 26354361e9b044cea1c2b86c4882fe92 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860; no voters: 
W20260709 13:21:25.138259 16789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 47285.3us
I20260709 13:21:25.140095 16117 leader_election.cc:304] T 746aabf856ab401c9ef5e859371e7b67 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860
I20260709 13:21:25.148927 16299 raft_consensus.cc:1275] T 2794af5b99984efebdee564035841cb0 P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:21:25.145947 21648 raft_consensus.cc:493] T 5a96373bb0e14dce8a266b2fc6c58507 P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:21:25.146175 21480 raft_consensus.cc:2749] T 829b4ad8c5a54eababeea1181b960d4b P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:21:25.145757 21604 raft_consensus.cc:2749] T ee9ed67006144797b1de6ea8c2013a68 P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:21:25.154055 16116 leader_election.cc:304] T 8a878999c0024d2abe517484180290fc P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 90f3da57796b488c9087a2f523adf045
W20260709 13:21:25.177515 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 46857.1us
I20260709 13:21:25.177925 16117 leader_election.cc:304] T 57a744f5aaf944678a63469cbf514015 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860
I20260709 13:21:25.178516 16790 leader_election.cc:304] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860; no voters: 90f3da57796b488c9087a2f523adf045
I20260709 13:21:25.182489 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c11074579b046d99e585c2818e18a2c" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:21:25.184615 16149 raft_consensus.cc:1275] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 17 index: 10. (term mismatch)
I20260709 13:21:25.185014 16149 pending_rounds.cc:85] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b: Aborting all ops after (but not including) 9
I20260709 13:21:25.185236 16149 pending_rounds.cc:107] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b: Aborting uncommitted NO_OP operation due to leader change: 16.10
I20260709 13:21:25.185449 16149 raft_consensus.cc:2889] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:21:25.187325 16294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdbd78acb4cf46298e0de8ba0d130e19" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
W20260709 13:21:25.189415 16189 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:25.190310 16820 raft_consensus.cc:1240] T ada5a004ba7c46819051e3ccec71ffdb P cb82cc7bd6db46d980c75237a1402860 [term 15 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 14. Current term is 15. Ops: []
W20260709 13:21:25.194828 16192 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 49749.8us
I20260709 13:21:25.195587 16192 leader_election.cc:304] T af57996a8be84d398f0f023e8d08ef77 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
I20260709 13:21:25.152395 21313 raft_consensus.cc:2804] T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:21:25.199898 21480 raft_consensus.cc:2749] T af57996a8be84d398f0f023e8d08ef77 P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:21:25.200289 16192 leader_election.cc:304] T ecc1e363230043fe9fef390f0b979f20 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
I20260709 13:21:25.200850 21480 consensus_queue.cc:1059] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:21:25.202773 21480 raft_consensus.cc:2749] T ecc1e363230043fe9fef390f0b979f20 P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:21:25.203234 21507 raft_consensus.cc:3055] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:21:25.203821 21507 raft_consensus.cc:740] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:21:25.204936 21507 consensus_queue.cc:260] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:25.206660 21507 raft_consensus.cc:3060] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:21:25.212316 16818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2372981006d4b218215c28964dea623" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:25.213261 16818 raft_consensus.cc:2468] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 16.
I20260709 13:21:25.217374 16818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdbd78acb4cf46298e0de8ba0d130e19" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:25.217980 16818 raft_consensus.cc:2468] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 17.
I20260709 13:21:25.151039 21648 raft_consensus.cc:515] T 5a96373bb0e14dce8a266b2fc6c58507 P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:25.189879 21580 raft_consensus.cc:2804] T 26354361e9b044cea1c2b86c4882fe92 P cb82cc7bd6db46d980c75237a1402860 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:21:25.193131 21677 raft_consensus.cc:2804] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:21:25.260795 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ab0ac8347641e6955e9bce7d90c1a2" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
W20260709 13:21:25.221499 21182 consensus_meta.cc:287] T 5a959dbecc2648eb85d1cc550f5a3245 P 1ca2725d290e47d68b1a4b609c9816ac: Time spent flushing consensus metadata: real 0.531s	user 0.010s	sys 0.000s
W20260709 13:21:25.263409 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 212217us
I20260709 13:21:25.266567 16821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8638ee363273426db1393dd81e5c7b72" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:21:25.267073 16294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8638ee363273426db1393dd81e5c7b72" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:21:25.267869 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02838c86531d4f98bf58d1c22b8baf77" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:25.264528 16818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ab0ac8347641e6955e9bce7d90c1a2" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:25.222316 21313 raft_consensus.cc:493] T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
W20260709 13:21:25.327589 16115 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:25.327716 16296 raft_consensus.cc:1275] T e4dc6b0541ca431ebc757beac00521f1 P 90f3da57796b488c9087a2f523adf045 [term 14 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 12 index: 6. Preceding OpId from leader: term: 14 index: 7. (index mismatch)
I20260709 13:21:25.265362 21613 raft_consensus.cc:3060] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:21:25.220499 21403 consensus_queue.cc:1059] T 2794af5b99984efebdee564035841cb0 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.077s
I20260709 13:21:25.197523 21084 raft_consensus.cc:515] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:25.357141 21245 raft_consensus.cc:2804] T 8a878999c0024d2abe517484180290fc P 647a343a542d4a139fe106c19095129b [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:21:25.358770 21245 raft_consensus.cc:697] T 8a878999c0024d2abe517484180290fc P 647a343a542d4a139fe106c19095129b [term 14 LEADER]: Becoming Leader. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:21:25.309717 21313 raft_consensus.cc:3060] T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:21:25.370026 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 216065us
W20260709 13:21:25.395216 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 47063.1us
I20260709 13:21:25.327625 21677 raft_consensus.cc:697] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860 [term 13 LEADER]: Becoming Leader. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
W20260709 13:21:25.353415 21712 log.cc:927] Time spent T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860: Append to log took a long time: real 0.172s	user 0.001s	sys 0.000s
I20260709 13:21:25.326666 21424 consensus_queue.cc:1048] T 19baa7bc367844c288bca8f81439c413 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.068s
I20260709 13:21:25.367735 21713 raft_consensus.cc:493] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:25.360110 21245 consensus_queue.cc:237] T 8a878999c0024d2abe517484180290fc P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:21:25.424505 21705 log.cc:927] Time spent T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.070s	user 0.000s	sys 0.001s
I20260709 13:21:25.403728 21403 consensus_queue.cc:1243] T 2794af5b99984efebdee564035841cb0 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Peer 647a343a542d4a139fe106c19095129b log is divergent from this leader: its last log entry 12.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
W20260709 13:21:25.425333 21623 consensus_meta.cc:287] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860: Time spent flushing consensus metadata: real 1.037s	user 0.004s	sys 0.009s
I20260709 13:21:25.451727 16220 raft_consensus.cc:2468] T 5a959dbecc2648eb85d1cc550f5a3245 P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 13.
I20260709 13:21:25.435518 21523 raft_consensus.cc:3060] T 434bff3082d141838068f323bdb0d1ed P cb82cc7bd6db46d980c75237a1402860 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:21:25.461076 16299 raft_consensus.cc:1275] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 13:21:25.429836 21580 raft_consensus.cc:697] T 26354361e9b044cea1c2b86c4882fe92 P cb82cc7bd6db46d980c75237a1402860 [term 13 LEADER]: Becoming Leader. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
W20260709 13:21:25.469444 16266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 138922us
I20260709 13:21:25.458875 21441 raft_consensus.cc:2749] T 746aabf856ab401c9ef5e859371e7b67 P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:21:25.475199 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 71526.3us
I20260709 13:21:25.449417 21713 raft_consensus.cc:515] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:25.474393 16266 leader_election.cc:304] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 
W20260709 13:21:25.475211 21324 consensus_meta.cc:287] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac: Time spent flushing consensus metadata: real 0.687s	user 0.011s	sys 0.000s
W20260709 13:21:25.514237 16117 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 336410us
I20260709 13:21:25.517660 21736 raft_consensus.cc:493] T 31969392ccdb43b491911a319e8bc7e1 P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:21:25.517555 21677 consensus_queue.cc:237] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:25.459180 21623 raft_consensus.cc:515] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:25.552526 21742 raft_consensus.cc:493] T c0eab25ce4c64e018a041beeda2867db P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:25.553263 21623 leader_election.cc:290] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 18 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:25.553884 21732 raft_consensus.cc:493] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:25.554366 21732 raft_consensus.cc:515] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:25.555546 16817 raft_consensus.cc:1240] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [term 18 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 17. Current term is 18. Ops: [17.12-17.12]
I20260709 13:21:25.556172 21604 raft_consensus.cc:493] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:25.556535 21732 leader_election.cc:290] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:25.477409 16293 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "3884f864a96d428785895e5463e5728f"
dest_uuid: "90f3da57796b488c9087a2f523adf045"
 from {username='slave'} at 127.0.0.1:40474
I20260709 13:21:25.556708 21604 raft_consensus.cc:515] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:25.535444 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c11074579b046d99e585c2818e18a2c" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:25.533731 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4f65cdf00b748bb963ee6e81ef43b79" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:25.557376 16293 raft_consensus.cc:493] T 3884f864a96d428785895e5463e5728f P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:21:25.558290 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a378d7f796f94615b3b347119a9880fd" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:25.558758 16293 raft_consensus.cc:3060] T 3884f864a96d428785895e5463e5728f P 90f3da57796b488c9087a2f523adf045 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:21:25.534101 16222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18ad7e3af5504571a69d10c00750a01c" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:25.484387 21580 consensus_queue.cc:237] T 26354361e9b044cea1c2b86c4882fe92 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:25.559780 21724 consensus_queue.cc:1059] T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:21:25.473381 21726 consensus_queue.cc:1048] T 2794af5b99984efebdee564035841cb0 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.063s
I20260709 13:21:25.532922 16223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ad5dbfa99ee4b8d9e28be92b3f36db5" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:21:25.531407 21444 consensus_queue.cc:1059] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.007s
I20260709 13:21:25.514309 21635 raft_consensus.cc:2749] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:21:25.538763 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e4bfb9090c8460eb14fa7e0d818e7f1" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:25.569110 16225 raft_consensus.cc:2393] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 15: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
I20260709 13:21:25.493065 21648 leader_election.cc:290] T 5a96373bb0e14dce8a266b2fc6c58507 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:21:25.543846 16117 leader_election.cc:304] T ee9ed67006144797b1de6ea8c2013a68 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860
I20260709 13:21:25.544859 21324 raft_consensus.cc:2749] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:21:25.547971 21720 log.cc:927] Time spent T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 0.089s	user 0.000s	sys 0.000s
I20260709 13:21:25.547699 21736 raft_consensus.cc:515] T 31969392ccdb43b491911a319e8bc7e1 P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:25.548491 21084 leader_election.cc:290] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 15 election: Requested vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:25.549046 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d4ab870a05347868956aa8b841b2981" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:21:25.551400 21719 raft_consensus.cc:2749] T 57a744f5aaf944678a63469cbf514015 P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:21:25.601967 21736 leader_election.cc:290] T 31969392ccdb43b491911a319e8bc7e1 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:25.602593 21736 consensus_queue.cc:1059] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:21:25.533335 16218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d16dfad0c3b540e5b4b86492ddf87208" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:25.534799 16221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c61a39fd4f84424b129c1a6f9ba404d" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:21:25.563055 16823 raft_consensus.cc:2468] T 434bff3082d141838068f323bdb0d1ed P cb82cc7bd6db46d980c75237a1402860 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 15.
I20260709 13:21:25.508280 21619 leader_election.cc:290] T fffab26427e24d398e2695aad40ca134 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:21:25.620841 21619 raft_consensus.cc:3055] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:21:25.621255 21619 raft_consensus.cc:740] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:21:25.579687 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1df93779209748b9860f6d893dd3de53" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:25.622219 21619 consensus_queue.cc:260] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:25.570794 16293 raft_consensus.cc:515] T 3884f864a96d428785895e5463e5728f P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:25.580909 21283 raft_consensus.cc:515] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:25.581664 16117 leader_election.cc:304] T 3884f864a96d428785895e5463e5728f P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860
I20260709 13:21:25.627887 16192 leader_election.cc:304] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 
W20260709 13:21:25.583262 21632 consensus_meta.cc:287] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860: Time spent flushing consensus metadata: real 0.892s	user 0.011s	sys 0.000s
W20260709 13:21:25.562295 21592 consensus_meta.cc:287] T e5ad4eb281564936a59f41e387a47156 P 647a343a542d4a139fe106c19095129b: Time spent flushing consensus metadata: real 0.618s	user 0.005s	sys 0.005s
I20260709 13:21:25.586125 21313 raft_consensus.cc:515] T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:25.587029 21722 consensus_queue.cc:1059] T 7c2095a5f4ad49408dbc7f69c606120d P 90f3da57796b488c9087a2f523adf045 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:21:25.653612 21722 raft_consensus.cc:3055] T 7c2095a5f4ad49408dbc7f69c606120d P 90f3da57796b488c9087a2f523adf045 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:21:25.653990 21722 raft_consensus.cc:740] T 7c2095a5f4ad49408dbc7f69c606120d P 90f3da57796b488c9087a2f523adf045 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:21:25.654948 21722 consensus_queue.cc:260] T 7c2095a5f4ad49408dbc7f69c606120d P 90f3da57796b488c9087a2f523adf045 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:25.656625 21722 raft_consensus.cc:3060] T 7c2095a5f4ad49408dbc7f69c606120d P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:21:25.657037 21739 raft_consensus.cc:2749] T 3884f864a96d428785895e5463e5728f P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:21:25.558971 21604 leader_election.cc:290] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:25.591356 21729 raft_consensus.cc:493] T ada5a004ba7c46819051e3ccec71ffdb P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:25.659840 21729 raft_consensus.cc:515] T ada5a004ba7c46819051e3ccec71ffdb P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:25.637600 21570 raft_consensus.cc:493] T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:21:25.492508 21708 raft_consensus.cc:515] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:25.661556 21570 raft_consensus.cc:515] T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:25.662474 16222 raft_consensus.cc:1240] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 12. Current term is 13. Ops: []
I20260709 13:21:25.662652 16221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5d59407eb545b895d8d922e0dd69fd" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:25.535784 21704 raft_consensus.cc:515] T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:25.604583 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af57996a8be84d398f0f023e8d08ef77" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:25.605659 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4dc6b0541ca431ebc757beac00521f1" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:25.606436 16223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62fe3346cd3c42c4a55e14af9073bca4" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:25.553037 21742 raft_consensus.cc:515] T c0eab25ce4c64e018a041beeda2867db P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:25.607656 21741 raft_consensus.cc:3055] T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:21:25.608583 21444 raft_consensus.cc:3055] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:21:25.609344 21713 leader_election.cc:290] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
W20260709 13:21:25.610455 21737 log.cc:927] Time spent T fffab26427e24d398e2695aad40ca134 P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.093s	user 0.000s	sys 0.001s
I20260709 13:21:25.612524 16219 raft_consensus.cc:1240] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 14. Current term is 15. Ops: [14.11-14.11]
I20260709 13:21:25.612684 21490 raft_consensus.cc:493] T fffab26427e24d398e2695aad40ca134 P 647a343a542d4a139fe106c19095129b [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
W20260709 13:21:25.615485 21626 consensus_meta.cc:287] T ecc1e363230043fe9fef390f0b979f20 P 647a343a542d4a139fe106c19095129b: Time spent flushing consensus metadata: real 0.510s	user 0.007s	sys 0.003s
W20260709 13:21:25.618209 16791 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:25.641352 21507 raft_consensus.cc:2804] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:21:25.627656 16819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e65676b21b748728ad39668cfe722ec" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:21:25.602906 21648 raft_consensus.cc:2804] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045 [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 13:21:25.638688 16117 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 57108.4us
I20260709 13:21:25.701068 16117 leader_election.cc:304] T 26354361e9b044cea1c2b86c4882fe92 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860
W20260709 13:21:25.638917 16192 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:25.647369 16820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62fe3346cd3c42c4a55e14af9073bca4" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:25.647598 16146 raft_consensus.cc:1240] T 9e65676b21b748728ad39668cfe722ec P 647a343a542d4a139fe106c19095129b [term 14 FOLLOWER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 13. Current term is 14. Ops: []
I20260709 13:21:25.650099 16822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af57996a8be84d398f0f023e8d08ef77" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
W20260709 13:21:25.652206 16191 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.686s	user 0.009s	sys 0.008s
I20260709 13:21:25.652791 21313 leader_election.cc:290] T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 17 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:25.662947 21708 leader_election.cc:290] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 13 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
W20260709 13:21:25.664753 16789 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:21:25.702463 16191 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.737s	user 0.009s	sys 0.008s
I20260709 13:21:25.666157 16293 leader_election.cc:290] T 3884f864a96d428785895e5463e5728f P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 10 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:25.668766 16296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66ae176961b74813a58bc15c9c4c6f2a" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:21:25.597714 21613 raft_consensus.cc:2749] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:21:25.676929 16223 raft_consensus.cc:2468] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 19.
I20260709 13:21:25.675961 16224 raft_consensus.cc:2468] T af57996a8be84d398f0f023e8d08ef77 P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 15.
I20260709 13:21:25.686743 16825 raft_consensus.cc:1240] T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860 [term 17 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 16. Current term is 17. Ops: []
I20260709 13:21:25.647733 21632 raft_consensus.cc:515] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:25.686758 21444 raft_consensus.cc:740] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:21:25.692675 21490 raft_consensus.cc:515] T fffab26427e24d398e2695aad40ca134 P 647a343a542d4a139fe106c19095129b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:25.579660 21569 raft_consensus.cc:2749] T ee9ed67006144797b1de6ea8c2013a68 P 647a343a542d4a139fe106c19095129b [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:21:25.697177 16294 raft_consensus.cc:1240] T 2794af5b99984efebdee564035841cb0 P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 11. Current term is 12. Ops: []
I20260709 13:21:25.697105 16299 raft_consensus.cc:1240] T 3884f864a96d428785895e5463e5728f P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 9. Current term is 10. Ops: []
W20260709 13:21:25.703018 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 745662us
I20260709 13:21:25.707118 16221 raft_consensus.cc:3055] T d16dfad0c3b540e5b4b86492ddf87208 P 1ca2725d290e47d68b1a4b609c9816ac [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:21:25.693624 21626 raft_consensus.cc:2749] T ecc1e363230043fe9fef390f0b979f20 P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:21:25.683396 21741 raft_consensus.cc:740] T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:21:25.712889 16820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 18 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:25.712749 16221 raft_consensus.cc:740] T d16dfad0c3b540e5b4b86492ddf87208 P 1ca2725d290e47d68b1a4b609c9816ac [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:21:25.712260 16821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66f75124215f4ca6bef831e174a792d8" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:25.700172 21507 raft_consensus.cc:493] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:25.699951 21648 raft_consensus.cc:493] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:25.663460 21592 raft_consensus.cc:515] T e5ad4eb281564936a59f41e387a47156 P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:25.724995 16191 leader_election.cc:304] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
W20260709 13:21:25.754423 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 49038.4us
I20260709 13:21:25.750937 16221 consensus_queue.cc:260] T d16dfad0c3b540e5b4b86492ddf87208 P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:25.763145 16822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5d59407eb545b895d8d922e0dd69fd" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
W20260709 13:21:25.765074 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 239880us
W20260709 13:21:25.765511 16220 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57600 (request call id 1705) took 1322 ms (client timeout 1594 ms). Trace:
I20260709 13:21:25.770485 16820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74ca8140f35c4ae29e8a664f6bdfc829" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:25.771832 16821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e4bfb9090c8460eb14fa7e0d818e7f1" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:21:25.773136 16821 raft_consensus.cc:3060] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P cb82cc7bd6db46d980c75237a1402860 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:21:25.764400 16822 raft_consensus.cc:2468] T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 16.
W20260709 13:21:25.765988 16220 rpcz_store.cc:269] 0709 13:21:24.443269 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:24.450420 (+  7151us) service_pool.cc:224] Handling call
0709 13:21:25.765356 (+1314936us) spinlock_profiling.cc:243] Waited 808 ms on lock 0x7b54002b3c54. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:21:25.765459 (+   103us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1857645824}
I20260709 13:21:25.793102 16292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a378d7f796f94615b3b347119a9880fd" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:21:25.751449 21507 raft_consensus.cc:3060] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:21:25.768172 21604 raft_consensus.cc:2804] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:21:25.753863 21726 raft_consensus.cc:2749] T 26354361e9b044cea1c2b86c4882fe92 P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:21:25.819788 16223 raft_consensus.cc:1275] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 13:21:25.825549 16822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada5a004ba7c46819051e3ccec71ffdb" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 16 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:25.826340 21740 raft_consensus.cc:493] T 23f82224f4564fea93572e8206bad1fe P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:21:25.825567 16816 raft_consensus.cc:1240] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860 [term 18 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 17. Current term is 18. Ops: [17.13-17.13]
I20260709 13:21:25.826673 16822 raft_consensus.cc:2410] T ada5a004ba7c46819051e3ccec71ffdb P cb82cc7bd6db46d980c75237a1402860 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac for term 16 because replica has last-logged OpId of term: 14 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 10.
I20260709 13:21:25.829231 16220 raft_consensus.cc:1240] T ecc1e363230043fe9fef390f0b979f20 P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 15. Current term is 16. Ops: []
I20260709 13:21:25.755133 21648 raft_consensus.cc:3060] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:21:25.858119 16820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2372981006d4b218215c28964dea623" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:21:25.858897 16820 raft_consensus.cc:3060] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:21:25.866402 16822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b7d1ab519044eda2d5e04a07242f25" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
W20260709 13:21:25.844902 16266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 370576us
I20260709 13:21:25.792198 21713 consensus_queue.cc:1059] T d16dfad0c3b540e5b4b86492ddf87208 P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.061s
I20260709 13:21:25.824364 21283 leader_election.cc:290] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 17 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:25.905396 16220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e4bfb9090c8460eb14fa7e0d818e7f1" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:25.907747 16220 raft_consensus.cc:2393] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 in current term 15: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
I20260709 13:21:25.768679 21480 consensus_queue.cc:1048] T e4dc6b0541ca431ebc757beac00521f1 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.086s
I20260709 13:21:25.833838 21604 raft_consensus.cc:697] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac [term 15 LEADER]: Becoming Leader. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
W20260709 13:21:25.914438 16789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 160853us
W20260709 13:21:25.908813 16117 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 151525us
I20260709 13:21:25.854364 21729 leader_election.cc:290] T ada5a004ba7c46819051e3ccec71ffdb P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:25.916447 16223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b7d1ab519044eda2d5e04a07242f25" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:25.833022 21424 consensus_queue.cc:1059] T 2794af5b99984efebdee564035841cb0 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 8, Time since last communication: 0.076s
W20260709 13:21:25.915343 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 159035us
I20260709 13:21:25.827047 21711 consensus_queue.cc:1048] T 19baa7bc367844c288bca8f81439c413 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.155s
I20260709 13:21:25.919129 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a378d7f796f94615b3b347119a9880fd" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:25.921340 16148 raft_consensus.cc:1275] T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:21:25.936153 16292 raft_consensus.cc:3060] T 31969392ccdb43b491911a319e8bc7e1 P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:21:25.939193 21665 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.507s	user 0.016s	sys 0.058s
W20260709 13:21:25.939604 21665 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.579s	user 0.016s	sys 0.058s
I20260709 13:21:25.940487 16218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c11074579b046d99e585c2818e18a2c" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:21:25.941885 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdbd78acb4cf46298e0de8ba0d130e19" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:21:25.928162 21570 leader_election.cc:290] T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:25.947281 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746aabf856ab401c9ef5e859371e7b67" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:25.947553 21742 leader_election.cc:290] T c0eab25ce4c64e018a041beeda2867db P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:25.948191 21704 leader_election.cc:290] T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 17 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:25.948823 16146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a9de347c711477e87c582d003f36925" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:21:25.950847 21507 raft_consensus.cc:515] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:25.953720 16226 raft_consensus.cc:2368] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 for earlier term 17. Current term is 18.
I20260709 13:21:25.954252 16818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdbd78acb4cf46298e0de8ba0d130e19" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:21:25.954803 16818 raft_consensus.cc:3060] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:21:25.955711 21490 leader_election.cc:290] T fffab26427e24d398e2695aad40ca134 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:25.956504 21632 leader_election.cc:290] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 18 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:25.956844 16820 raft_consensus.cc:2468] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 17.
I20260709 13:21:25.929469 16823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 18 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:25.929733 16819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9944d5e4b8c4109af9955c4f08c9095" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:25.936009 16822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a378d7f796f94615b3b347119a9880fd" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:25.826936 21740 raft_consensus.cc:515] T 23f82224f4564fea93572e8206bad1fe P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:25.953188 21507 leader_election.cc:290] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 18 election: Requested vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:25.926034 16117 leader_election.cc:304] T 9e65676b21b748728ad39668cfe722ec P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860
I20260709 13:21:25.929092 16116 leader_election.cc:304] T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:21:25.959664 16192 leader_election.cc:304] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:21:25.962765 16819 raft_consensus.cc:2393] T c9944d5e4b8c4109af9955c4f08c9095 P cb82cc7bd6db46d980c75237a1402860 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 16: Already voted for candidate cb82cc7bd6db46d980c75237a1402860 in this term.
I20260709 13:21:25.963904 16820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdbd78acb4cf46298e0de8ba0d130e19" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:25.965471 16220 raft_consensus.cc:1240] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 15. Current term is 16. Ops: []
I20260709 13:21:25.958740 16265 leader_election.cc:304] T 8c11074579b046d99e585c2818e18a2c P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860
I20260709 13:21:25.967092 16220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66ae176961b74813a58bc15c9c4c6f2a" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:21:25.971782 16220 raft_consensus.cc:3060] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:21:25.975013 16189 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:25.976233 21745 raft_consensus.cc:2749] T 8c11074579b046d99e585c2818e18a2c P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:21:25.978871 21752 raft_consensus.cc:2749] T 9e65676b21b748728ad39668cfe722ec P 647a343a542d4a139fe106c19095129b [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
W20260709 13:21:25.980183 16117 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 54221.5us
I20260709 13:21:25.981106 16117 leader_election.cc:304] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac
I20260709 13:21:25.981794 21752 raft_consensus.cc:2804] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:21:25.982129 21752 raft_consensus.cc:493] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:25.982456 21752 raft_consensus.cc:3060] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:21:25.983799 16220 raft_consensus.cc:2410] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 for term 18 because replica has last-logged OpId of term: 17 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 12.
I20260709 13:21:25.984251 16292 raft_consensus.cc:1275] T 31969392ccdb43b491911a319e8bc7e1 P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:21:25.855053 21444 consensus_queue.cc:260] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 8, Last appended: 14.10, Last appended by leader: 10, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } }
I20260709 13:21:25.989486 16265 leader_election.cc:304] T 5a959dbecc2648eb85d1cc550f5a3245 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:21:25.989717 16221 raft_consensus.cc:3060] T d16dfad0c3b540e5b4b86492ddf87208 P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:21:25.908792 21741 consensus_queue.cc:260] T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:25.992951 16818 raft_consensus.cc:2468] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 18.
I20260709 13:21:25.993974 16292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 18 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:25.995210 16292 raft_consensus.cc:2468] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 17.
I20260709 13:21:25.972501 21084 raft_consensus.cc:2804] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:21:25.994381 16192 leader_election.cc:304] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:21:25.996129 16822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31969392ccdb43b491911a319e8bc7e1" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
W20260709 13:21:25.998628 16191 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:26.000644 16790 leader_election.cc:304] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
W20260709 13:21:26.003857 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 74814.8us
I20260709 13:21:26.004509 16116 leader_election.cc:304] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: cb82cc7bd6db46d980c75237a1402860
I20260709 13:21:26.007961 16823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada5a004ba7c46819051e3ccec71ffdb" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:26.008702 16823 raft_consensus.cc:2468] T ada5a004ba7c46819051e3ccec71ffdb P cb82cc7bd6db46d980c75237a1402860 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 15.
I20260709 13:21:26.011294 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a378d7f796f94615b3b347119a9880fd" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:21:26.012969 16217 raft_consensus.cc:1275] T 19baa7bc367844c288bca8f81439c413 P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 13:21:26.016356 16818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3884f864a96d428785895e5463e5728f" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: true dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:21:26.008193 21745 raft_consensus.cc:2804] T 5a959dbecc2648eb85d1cc550f5a3245 P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:21:26.017019 16818 raft_consensus.cc:3055] T 3884f864a96d428785895e5463e5728f P cb82cc7bd6db46d980c75237a1402860 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:21:26.017472 16818 raft_consensus.cc:740] T 3884f864a96d428785895e5463e5728f P cb82cc7bd6db46d980c75237a1402860 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:21:26.018992 16295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a9de347c711477e87c582d003f36925" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:21:26.018785 16790 leader_election.cc:304] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:21:26.023667 16790 leader_election.cc:304] T 8c11074579b046d99e585c2818e18a2c P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:21:26.024881 21283 consensus_queue.cc:1059] T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: INVALID_TERM, Last received: 13.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:21:26.025522 16824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2372981006d4b218215c28964dea623" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:21:26.025696 21740 leader_election.cc:290] T 23f82224f4564fea93572e8206bad1fe P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:26.026062 21283 raft_consensus.cc:3055] T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:21:26.026283 16824 raft_consensus.cc:2393] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 17: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
I20260709 13:21:26.026446 21283 raft_consensus.cc:740] T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:21:26.021040 16266 leader_election.cc:304] T 434bff3082d141838068f323bdb0d1ed P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:21:26.028960 16790 leader_election.cc:304] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:21:26.030795 21741 raft_consensus.cc:3060] T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:21:26.031867 21750 raft_consensus.cc:2749] T 8c11074579b046d99e585c2818e18a2c P cb82cc7bd6db46d980c75237a1402860 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:21:25.999472 21490 raft_consensus.cc:493] T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:26.002254 21592 leader_election.cc:290] T e5ad4eb281564936a59f41e387a47156 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 16 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:26.047109 16790 leader_election.cc:304] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:21:26.026333 21461 raft_consensus.cc:2749] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:21:26.002954 21507 raft_consensus.cc:2804] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:21:26.018664 21732 raft_consensus.cc:2749] T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:21:26.013814 21604 consensus_queue.cc:237] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } }
I20260709 13:21:26.015412 21711 consensus_queue.cc:1048] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.035s
W20260709 13:21:26.051254 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 46802.5us
I20260709 13:21:26.089820 16116 leader_election.cc:304] T af57996a8be84d398f0f023e8d08ef77 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: cb82cc7bd6db46d980c75237a1402860
I20260709 13:21:26.049852 21704 raft_consensus.cc:2804] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Leader pre-election won for term 20
W20260709 13:21:26.067855 16189 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:26.080772 16821 raft_consensus.cc:2468] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P cb82cc7bd6db46d980c75237a1402860 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 15.
I20260709 13:21:26.061720 16266 leader_election.cc:304] T a378d7f796f94615b3b347119a9880fd P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860
I20260709 13:21:26.027746 21084 raft_consensus.cc:697] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [term 17 LEADER]: Becoming Leader. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
W20260709 13:21:26.094727 16192 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:26.098129 16192 leader_election.cc:304] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:21:26.071471 16818 consensus_queue.cc:260] T 3884f864a96d428785895e5463e5728f P cb82cc7bd6db46d980c75237a1402860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 9.10, Last appended by leader: 10, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:26.100909 16266 leader_election.cc:304] T ada5a004ba7c46819051e3ccec71ffdb P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:21:26.101894 16818 raft_consensus.cc:3060] T 3884f864a96d428785895e5463e5728f P cb82cc7bd6db46d980c75237a1402860 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:21:26.047685 21688 raft_consensus.cc:2749] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:21:26.098249 21084 consensus_queue.cc:237] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.11, Last appended by leader: 11, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:26.104921 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4f65cdf00b748bb963ee6e81ef43b79" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
W20260709 13:21:26.104527 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 115133us
I20260709 13:21:26.068706 21729 raft_consensus.cc:2804] T 434bff3082d141838068f323bdb0d1ed P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:21:26.047148 21745 raft_consensus.cc:697] T 5a959dbecc2648eb85d1cc550f5a3245 P 90f3da57796b488c9087a2f523adf045 [term 13 LEADER]: Becoming Leader. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:21:26.051466 21623 raft_consensus.cc:2749] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
W20260709 13:21:26.106523 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 78659.8us
I20260709 13:21:26.111838 16150 raft_consensus.cc:1275] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b [term 14 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 12 index: 12. Preceding OpId from leader: term: 14 index: 13. (index mismatch)
I20260709 13:21:26.080763 21461 raft_consensus.cc:493] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:26.087674 21742 raft_consensus.cc:493] T 8cc959fb7f704fba9009c0595467d074 P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:21:26.101902 21424 raft_consensus.cc:2749] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P cb82cc7bd6db46d980c75237a1402860 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:21:26.102645 21592 raft_consensus.cc:2804] T af57996a8be84d398f0f023e8d08ef77 P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:21:26.124969 16150 raft_consensus.cc:1240] T 26354361e9b044cea1c2b86c4882fe92 P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 12. Current term is 13. Ops: []
I20260709 13:21:26.096299 21704 raft_consensus.cc:493] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Starting leader election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:21:26.116343 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fffab26427e24d398e2695aad40ca134" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:26.123420 16295 raft_consensus.cc:3055] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 90f3da57796b488c9087a2f523adf045 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:21:26.132705 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5ad4eb281564936a59f41e387a47156" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:21:26.125412 21461 raft_consensus.cc:515] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:26.134138 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2372981006d4b218215c28964dea623" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:21:26.030036 21283 consensus_queue.cc:260] T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:26.136039 21283 raft_consensus.cc:3060] T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:21:26.125845 21732 raft_consensus.cc:493] T 9c61a39fd4f84424b129c1a6f9ba404d P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:21:26.138439 21732 raft_consensus.cc:515] T 9c61a39fd4f84424b129c1a6f9ba404d P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:26.050376 21579 consensus_queue.cc:1059] T 8638ee363273426db1393dd81e5c7b72 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.062s
I20260709 13:21:26.139973 21579 raft_consensus.cc:3055] T 8638ee363273426db1393dd81e5c7b72 P cb82cc7bd6db46d980c75237a1402860 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:21:26.140419 21579 raft_consensus.cc:740] T 8638ee363273426db1393dd81e5c7b72 P cb82cc7bd6db46d980c75237a1402860 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:21:26.140872 21732 leader_election.cc:290] T 9c61a39fd4f84424b129c1a6f9ba404d P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:26.141467 21579 consensus_queue.cc:260] T 8638ee363273426db1393dd81e5c7b72 P cb82cc7bd6db46d980c75237a1402860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:26.050473 21490 raft_consensus.cc:515] T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:26.101512 21444 raft_consensus.cc:3060] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:21:26.132056 16295 raft_consensus.cc:740] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 90f3da57796b488c9087a2f523adf045 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:21:26.145012 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0eab25ce4c64e018a041beeda2867db" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:26.145802 16818 raft_consensus.cc:2468] T 3884f864a96d428785895e5463e5728f P cb82cc7bd6db46d980c75237a1402860 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 10.
I20260709 13:21:26.146934 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 18 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:26.149255 16116 leader_election.cc:304] T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860
I20260709 13:21:26.168721 21759 raft_consensus.cc:2749] T a378d7f796f94615b3b347119a9880fd P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:21:26.169804 21759 raft_consensus.cc:2804] T ada5a004ba7c46819051e3ccec71ffdb P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:21:26.170255 21759 raft_consensus.cc:493] T ada5a004ba7c46819051e3ccec71ffdb P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:26.170681 21759 raft_consensus.cc:3060] T ada5a004ba7c46819051e3ccec71ffdb P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:21:26.175607 16192 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 77595.6us
I20260709 13:21:26.180142 16143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cc959fb7f704fba9009c0595467d074" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:26.180629 16146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f2da98864b4425bf7c0055951fdd4b" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:26.181020 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fffab26427e24d398e2695aad40ca134" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:26.181314 16146 raft_consensus.cc:2468] T 84f2da98864b4425bf7c0055951fdd4b P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 11.
I20260709 13:21:26.181602 16149 raft_consensus.cc:2468] T fffab26427e24d398e2695aad40ca134 P 647a343a542d4a139fe106c19095129b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 14.
I20260709 13:21:26.181856 16148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87ac0d1f7e8a419f9bcbc04aee0e067e" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:26.182585 16145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31969392ccdb43b491911a319e8bc7e1" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:26.183559 16142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3884f864a96d428785895e5463e5728f" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: true dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:21:26.184136 16142 raft_consensus.cc:3060] T 3884f864a96d428785895e5463e5728f P 647a343a542d4a139fe106c19095129b [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:21:26.177248 21677 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.594s	user 0.014s	sys 0.018s
W20260709 13:21:26.174489 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 51236.8us
I20260709 13:21:26.184295 16217 raft_consensus.cc:2468] T c0eab25ce4c64e018a041beeda2867db P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 15.
I20260709 13:21:26.132325 21592 raft_consensus.cc:493] T af57996a8be84d398f0f023e8d08ef77 P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Starting leader election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:21:26.192176 16145 raft_consensus.cc:2393] T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 13: Already voted for candidate cb82cc7bd6db46d980c75237a1402860 in this term.
I20260709 13:21:26.143601 21704 raft_consensus.cc:3060] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:21:26.188282 21677 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.605s	user 0.014s	sys 0.018s
I20260709 13:21:26.207206 16148 raft_consensus.cc:1240] T 8a878999c0024d2abe517484180290fc P 647a343a542d4a139fe106c19095129b [term 14 LEADER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 13. Current term is 14. Ops: []
I20260709 13:21:26.130586 21324 consensus_queue.cc:1059] T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.043s
I20260709 13:21:26.197134 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8158bbd17b994d2eb96f560bd0f1b88d" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:26.189567 16116 leader_election.cc:304] T c0eab25ce4c64e018a041beeda2867db P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
I20260709 13:21:26.173614 21757 raft_consensus.cc:2804] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:21:26.206760 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f82224f4564fea93572e8206bad1fe" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:26.196899 16147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a96373bb0e14dce8a266b2fc6c58507" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:26.174224 21623 raft_consensus.cc:493] T 3a7ca61592e848ea9b0fb81a2ce33cdc P cb82cc7bd6db46d980c75237a1402860 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:21:26.197027 21722 consensus_meta.cc:287] T 7c2095a5f4ad49408dbc7f69c606120d P 90f3da57796b488c9087a2f523adf045: Time spent flushing consensus metadata: real 0.540s	user 0.009s	sys 0.003s
W20260709 13:21:26.207006 16265 leader_election.cc:336] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Timed out: RequestConsensusVote RPC to 127.14.177.129:36741 timed out after 2.201s (SENT)
I20260709 13:21:26.218374 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c61a39fd4f84424b129c1a6f9ba404d" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:26.216027 16146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:26.214715 16223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fffab26427e24d398e2695aad40ca134" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:21:26.198899 21739 raft_consensus.cc:2749] T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:21:26.200469 21592 raft_consensus.cc:3060] T af57996a8be84d398f0f023e8d08ef77 P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:21:26.147043 21742 raft_consensus.cc:515] T 8cc959fb7f704fba9009c0595467d074 P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:26.221957 21729 raft_consensus.cc:697] T 434bff3082d141838068f323bdb0d1ed P 90f3da57796b488c9087a2f523adf045 [term 15 LEADER]: Becoming Leader. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:21:26.217860 16820 raft_consensus.cc:1240] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860 [term 13 LEADER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 12. Current term is 13. Ops: []
I20260709 13:21:26.230067 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68a6838d82ef4cac83507f906139d90f" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:26.217496 21588 raft_consensus.cc:2804] T c0eab25ce4c64e018a041beeda2867db P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:21:26.239399 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87ac0d1f7e8a419f9bcbc04aee0e067e" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:26.240195 16217 raft_consensus.cc:2393] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 16: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
I20260709 13:21:26.220561 21757 raft_consensus.cc:697] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 15 LEADER]: Becoming Leader. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:21:26.234776 21507 raft_consensus.cc:697] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [term 18 LEADER]: Becoming Leader. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:21:26.224833 16295 consensus_queue.cc:260] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 90f3da57796b488c9087a2f523adf045 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:26.248548 16223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 18 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:26.250705 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c74490c40243a88e62d1760b5cd814" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:26.249614 16143 raft_consensus.cc:2468] T 8cc959fb7f704fba9009c0595467d074 P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 13.
I20260709 13:21:26.224706 21745 consensus_queue.cc:237] T 5a959dbecc2648eb85d1cc550f5a3245 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:21:26.260130 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 60291.7us
I20260709 13:21:26.261193 21757 consensus_queue.cc:237] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:26.237632 21588 raft_consensus.cc:493] T c0eab25ce4c64e018a041beeda2867db P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:26.267424 16295 raft_consensus.cc:3060] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:21:26.288467 16265 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:26.267971 16223 raft_consensus.cc:2468] T 19baa7bc367844c288bca8f81439c413 P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 17.
I20260709 13:21:26.232916 21623 raft_consensus.cc:515] T 3a7ca61592e848ea9b0fb81a2ce33cdc P cb82cc7bd6db46d980c75237a1402860 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:26.291410 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f2da98864b4425bf7c0055951fdd4b" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:26.293205 16221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:26.294066 16220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9944d5e4b8c4109af9955c4f08c9095" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:26.300148 16221 raft_consensus.cc:2468] T ecc1e363230043fe9fef390f0b979f20 P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 16.
I20260709 13:21:26.300227 16220 raft_consensus.cc:2468] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 16.
W20260709 13:21:26.296808 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 97372.2us
W20260709 13:21:26.298036 16266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 197191us
I20260709 13:21:26.273133 16142 raft_consensus.cc:2468] T 3884f864a96d428785895e5463e5728f P 647a343a542d4a139fe106c19095129b [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 10.
W20260709 13:21:26.302846 16266 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:26.304363 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0eab25ce4c64e018a041beeda2867db" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:21:26.290936 21588 raft_consensus.cc:3060] T c0eab25ce4c64e018a041beeda2867db P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:21:26.315706 21742 leader_election.cc:290] T 8cc959fb7f704fba9009c0595467d074 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:26.291141 21762 consensus_queue.cc:1059] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.053s
I20260709 13:21:26.311754 21729 consensus_queue.cc:237] T 434bff3082d141838068f323bdb0d1ed P 90f3da57796b488c9087a2f523adf045 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.12, Last appended by leader: 12, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:26.345045 16223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a96373bb0e14dce8a266b2fc6c58507" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:26.345571 16293 raft_consensus.cc:3060] T c0eab25ce4c64e018a041beeda2867db P 90f3da57796b488c9087a2f523adf045 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:21:26.345638 16221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cc959fb7f704fba9009c0595467d074" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:26.346208 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cc959fb7f704fba9009c0595467d074" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
W20260709 13:21:26.346990 16192 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:26.349493 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e4bfb9090c8460eb14fa7e0d818e7f1" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:21:26.351886 16220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada5a004ba7c46819051e3ccec71ffdb" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:26.356248 16291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f5f89186204242bf377f7c42b2e2da" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:21:26.357339 16295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdbd78acb4cf46298e0de8ba0d130e19" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:21:26.360689 16295 raft_consensus.cc:3060] T cdbd78acb4cf46298e0de8ba0d130e19 P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Advancing to term 18
I20260709 13:21:26.347839 21711 raft_consensus.cc:493] T c9944d5e4b8c4109af9955c4f08c9095 P cb82cc7bd6db46d980c75237a1402860 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:26.367300 16265 leader_election.cc:400] T e4dc6b0541ca431ebc757beac00521f1 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 13 pre-election: Vote denied by peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741) with higher term. Message: Invalid argument: T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 for earlier term 13. Current term is 14.
I20260709 13:21:26.367712 16265 leader_election.cc:403] T e4dc6b0541ca431ebc757beac00521f1 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 13 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:21:26.369215 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fffab26427e24d398e2695aad40ca134" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:26.370316 21711 raft_consensus.cc:515] T c9944d5e4b8c4109af9955c4f08c9095 P cb82cc7bd6db46d980c75237a1402860 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:26.371490 21490 leader_election.cc:290] T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:26.372841 21711 leader_election.cc:290] T c9944d5e4b8c4109af9955c4f08c9095 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:26.373996 21752 raft_consensus.cc:515] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:26.357142 21648 raft_consensus.cc:515] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:26.378182 16300 raft_consensus.cc:2393] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 15: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
I20260709 13:21:26.391875 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdbd78acb4cf46298e0de8ba0d130e19" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:26.398368 16217 raft_consensus.cc:1240] T 8a878999c0024d2abe517484180290fc P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 13. Current term is 14. Ops: []
I20260709 13:21:26.402083 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74ca8140f35c4ae29e8a664f6bdfc829" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
W20260709 13:21:26.402360 21735 log.cc:927] Time spent T 00000000000000000000000000000000 P 667a34b27a39474ebefc1af3cf5b4271: Append to log took a long time: real 0.461s	user 0.004s	sys 0.000s
I20260709 13:21:26.404345 21648 leader_election.cc:290] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 15 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:26.405486 16298 raft_consensus.cc:2468] T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 14.
I20260709 13:21:26.410492 16150 raft_consensus.cc:1240] T e5ad4eb281564936a59f41e387a47156 P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 15. Current term is 16. Ops: [15.10-15.10]
I20260709 13:21:26.387897 21769 raft_consensus.cc:3055] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 1ca2725d290e47d68b1a4b609c9816ac [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:21:26.409147 21097 raft_consensus.cc:493] T 18ad7e3af5504571a69d10c00750a01c P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:21:26.405257 21507 consensus_queue.cc:237] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:26.396703 21746 raft_consensus.cc:2749] T e4dc6b0541ca431ebc757beac00521f1 P 90f3da57796b488c9087a2f523adf045 [term 14 FOLLOWER]: Leader pre-election lost for term 13. Reason: Vote denied by peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741) with higher term. Message: Invalid argument: T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 for earlier term 13. Current term is 14.
I20260709 13:21:26.415686 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 18 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:26.417002 16146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b7d1ab519044eda2d5e04a07242f25" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:21:26.420998 16293 raft_consensus.cc:2410] T c0eab25ce4c64e018a041beeda2867db P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac for term 15 because replica has last-logged OpId of term: 14 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 7.
I20260709 13:21:26.424795 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2372981006d4b218215c28964dea623" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:21:26.414040 16151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66f75124215f4ca6bef831e174a792d8" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:26.427726 16149 raft_consensus.cc:2393] T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 17: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
I20260709 13:21:26.428094 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fffab26427e24d398e2695aad40ca134" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:26.406507 21708 raft_consensus.cc:3055] T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:21:26.429678 21097 raft_consensus.cc:515] T 18ad7e3af5504571a69d10c00750a01c P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } }
I20260709 13:21:26.431501 16817 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49892 (request call id 813) took 1342 ms. Trace:
W20260709 13:21:26.434818 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 79329.4us
I20260709 13:21:26.445633 16817 rpcz_store.cc:276] 0709 13:21:25.089214 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:25.091300 (+  2086us) service_pool.cc:224] Handling call
0709 13:21:25.091729 (+   429us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:21:25.555949 (+464220us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:26.431239 (+875290us) spinlock_profiling.cc:243] Waited 463 ms on lock 0x7b540054bad4. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:21:26.431413 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1065941888}
I20260709 13:21:26.435875 21619 raft_consensus.cc:3060] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:21:26.452307 21752 leader_election.cc:290] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 17 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:26.454707 16265 leader_election.cc:304] T 3884f864a96d428785895e5463e5728f P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
I20260709 13:21:26.456324 16265 leader_election.cc:400] T c0eab25ce4c64e018a041beeda2867db P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 14 election: Vote denied by peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741) with higher term. Message: Invalid argument: T c0eab25ce4c64e018a041beeda2867db P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 for earlier term 14. Current term is 15.
I20260709 13:21:26.456629 16265 leader_election.cc:403] T c0eab25ce4c64e018a041beeda2867db P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 14 election: Cancelling election due to peer responding with higher term
I20260709 13:21:26.457455 21746 raft_consensus.cc:2749] T 3884f864a96d428785895e5463e5728f P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:21:26.458091 16790 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:26.458225 21746 raft_consensus.cc:2749] T c0eab25ce4c64e018a041beeda2867db P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Leader election lost for term 14. Reason: Vote denied by peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741) with higher term. Message: Invalid argument: T c0eab25ce4c64e018a041beeda2867db P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 for earlier term 14. Current term is 15.
I20260709 13:21:26.459697 21623 leader_election.cc:290] T 3a7ca61592e848ea9b0fb81a2ce33cdc P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:26.460309 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c61a39fd4f84424b129c1a6f9ba404d" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:26.460493 21592 raft_consensus.cc:515] T af57996a8be84d398f0f023e8d08ef77 P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:26.461194 16299 raft_consensus.cc:2393] T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 16: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
I20260709 13:21:26.462143 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5ad4eb281564936a59f41e387a47156" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:21:26.420032 21769 raft_consensus.cc:740] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 1ca2725d290e47d68b1a4b609c9816ac [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:21:26.464303 16220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5d59407eb545b895d8d922e0dd69fd" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:21:26.464154 21769 consensus_queue.cc:260] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:26.464834 16220 raft_consensus.cc:3060] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:21:26.465334 21769 raft_consensus.cc:3060] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:21:26.467219 21097 leader_election.cc:290] T 18ad7e3af5504571a69d10c00750a01c P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:21:26.468732 16825 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49906 (request call id 1092) took 1266 ms. Trace:
I20260709 13:21:26.468837 21759 raft_consensus.cc:515] T ada5a004ba7c46819051e3ccec71ffdb P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:26.469448 16222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af57996a8be84d398f0f023e8d08ef77" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:21:26.469201 16825 rpcz_store.cc:276] 0709 13:21:25.202067 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:25.202465 (+   398us) service_pool.cc:224] Handling call
0709 13:21:25.202862 (+   397us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:21:25.709487 (+506625us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:26.468455 (+758968us) spinlock_profiling.cc:243] Waited 461 ms on lock 0x7b54006f2054. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:21:26.468668 (+   213us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1060047104}
W20260709 13:21:26.469993 16789 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:26.471359 21759 leader_election.cc:290] T ada5a004ba7c46819051e3ccec71ffdb P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 16 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:26.472051 16266 leader_election.cc:304] T 3884f864a96d428785895e5463e5728f P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:21:26.472338 16294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4f65cdf00b748bb963ee6e81ef43b79" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:26.473096 16294 raft_consensus.cc:2468] T c4f65cdf00b748bb963ee6e81ef43b79 P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 15.
I20260709 13:21:26.474279 21592 leader_election.cc:290] T af57996a8be84d398f0f023e8d08ef77 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 16 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:26.462360 21773 consensus_queue.cc:1059] T 02838c86531d4f98bf58d1c22b8baf77 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.111s
I20260709 13:21:26.475625 21736 raft_consensus.cc:2804] T 3884f864a96d428785895e5463e5728f P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:21:26.476118 21736 raft_consensus.cc:697] T 3884f864a96d428785895e5463e5728f P 90f3da57796b488c9087a2f523adf045 [term 10 LEADER]: Becoming Leader. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:21:26.477017 21736 consensus_queue.cc:237] T 3884f864a96d428785895e5463e5728f P 90f3da57796b488c9087a2f523adf045 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 9.10, Last appended by leader: 10, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:26.477670 21592 raft_consensus.cc:493] T 434bff3082d141838068f323bdb0d1ed P 647a343a542d4a139fe106c19095129b [term 14 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:21:26.478183 21592 raft_consensus.cc:515] T 434bff3082d141838068f323bdb0d1ed P 647a343a542d4a139fe106c19095129b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:26.474011 16294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31969392ccdb43b491911a319e8bc7e1" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:26.480530 16294 raft_consensus.cc:2468] T 31969392ccdb43b491911a319e8bc7e1 P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 13.
I20260709 13:21:26.481762 16294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0eab25ce4c64e018a041beeda2867db" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:26.482300 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cc959fb7f704fba9009c0595467d074" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:26.482404 16294 raft_consensus.cc:2468] T c0eab25ce4c64e018a041beeda2867db P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 15.
I20260709 13:21:26.482952 21671 consensus_queue.cc:1048] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:21:26.485378 16820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31969392ccdb43b491911a319e8bc7e1" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:25.978503 15046 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:21:26.487493 16820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5d59407eb545b895d8d922e0dd69fd" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:21:26.488260 16820 raft_consensus.cc:3060] T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:21:26.489876 16825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af57996a8be84d398f0f023e8d08ef77" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:21:26.471045 16222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18ad7e3af5504571a69d10c00750a01c" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
W20260709 13:21:26.459007 16790 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:26.462761 16299 raft_consensus.cc:3060] T e5ad4eb281564936a59f41e387a47156 P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:21:26.491873 16294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada5a004ba7c46819051e3ccec71ffdb" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 16 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:26.429222 16295 raft_consensus.cc:2468] T cdbd78acb4cf46298e0de8ba0d130e19 P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 18.
I20260709 13:21:26.445374 21708 raft_consensus.cc:740] T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:21:26.504490 16294 raft_consensus.cc:2393] T ada5a004ba7c46819051e3ccec71ffdb P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 16: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
I20260709 13:21:26.563087 16115 leader_election.cc:304] T 9c61a39fd4f84424b129c1a6f9ba404d P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:21:26.532918 16821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f82224f4564fea93572e8206bad1fe" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
W20260709 13:21:26.517760 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 58350.8us
I20260709 13:21:26.541895 16143 raft_consensus.cc:1240] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 12. Current term is 13. Ops: []
I20260709 13:21:26.509771 16220 raft_consensus.cc:2468] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 17.
I20260709 13:21:26.542481 16825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada5a004ba7c46819051e3ccec71ffdb" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:21:26.567992 16825 raft_consensus.cc:3060] T ada5a004ba7c46819051e3ccec71ffdb P cb82cc7bd6db46d980c75237a1402860 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:21:26.547755 16790 leader_election.cc:304] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:21:26.523578 16817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e4bfb9090c8460eb14fa7e0d818e7f1" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:21:26.529253 16295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9944d5e4b8c4109af9955c4f08c9095" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:26.570627 16295 raft_consensus.cc:2468] T c9944d5e4b8c4109af9955c4f08c9095 P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 15.
I20260709 13:21:26.542043 16221 raft_consensus.cc:1275] T 8a878999c0024d2abe517484180290fc P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:21:26.577584 16817 raft_consensus.cc:2393] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P cb82cc7bd6db46d980c75237a1402860 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 15: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
I20260709 13:21:26.499432 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:26.578446 16115 leader_election.cc:304] T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac
I20260709 13:21:26.579919 16293 raft_consensus.cc:2468] T ecc1e363230043fe9fef390f0b979f20 P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 16.
I20260709 13:21:26.590510 16115 leader_election.cc:304] T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: cb82cc7bd6db46d980c75237a1402860
I20260709 13:21:26.592722 16820 raft_consensus.cc:2468] T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 17.
I20260709 13:21:26.598637 21704 raft_consensus.cc:515] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:26.554445 21403 consensus_queue.cc:1048] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.047s
I20260709 13:21:26.602393 16220 raft_consensus.cc:1275] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 13:21:26.587570 21729 consensus_queue.cc:1059] T 66ae176961b74813a58bc15c9c4c6f2a P 90f3da57796b488c9087a2f523adf045 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: INVALID_TERM, Last received: 15.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.026s
I20260709 13:21:26.603312 21729 consensus_queue.cc:1243] T 66ae176961b74813a58bc15c9c4c6f2a P 90f3da57796b488c9087a2f523adf045 [LEADER]: Peer cb82cc7bd6db46d980c75237a1402860 log is divergent from this leader: its last log entry 16.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 11
I20260709 13:21:26.518065 21708 consensus_queue.cc:260] T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:26.603833 21729 raft_consensus.cc:3055] T 66ae176961b74813a58bc15c9c4c6f2a P 90f3da57796b488c9087a2f523adf045 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:21:26.604224 21708 raft_consensus.cc:3060] T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:21:26.605753 21704 leader_election.cc:290] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 20 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:26.604270 21729 raft_consensus.cc:740] T 66ae176961b74813a58bc15c9c4c6f2a P 90f3da57796b488c9087a2f523adf045 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
W20260709 13:21:26.518594 21775 log.cc:927] Time spent T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.166s	user 0.000s	sys 0.001s
I20260709 13:21:26.607674 21729 consensus_queue.cc:260] T 66ae176961b74813a58bc15c9c4c6f2a P 90f3da57796b488c9087a2f523adf045 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:26.609026 21729 raft_consensus.cc:3060] T 66ae176961b74813a58bc15c9c4c6f2a P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:21:26.609728 21675 raft_consensus.cc:3055] T 2794af5b99984efebdee564035841cb0 P cb82cc7bd6db46d980c75237a1402860 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:21:26.610245 21675 raft_consensus.cc:740] T 2794af5b99984efebdee564035841cb0 P cb82cc7bd6db46d980c75237a1402860 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:21:26.610563 21719 raft_consensus.cc:493] T 23f82224f4564fea93572e8206bad1fe P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:21:26.611543 21543 raft_consensus.cc:2804] T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:21:26.612094 21543 raft_consensus.cc:493] T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:26.611213 21719 raft_consensus.cc:515] T 23f82224f4564fea93572e8206bad1fe P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:26.612527 21543 raft_consensus.cc:3060] T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:21:26.519111 21592 leader_election.cc:290] T 434bff3082d141838068f323bdb0d1ed P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:26.612108 21752 raft_consensus.cc:493] T 5a959dbecc2648eb85d1cc550f5a3245 P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:21:26.615741 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 51300.4us
I20260709 13:21:26.615775 21752 raft_consensus.cc:515] T 5a959dbecc2648eb85d1cc550f5a3245 P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:26.616465 21592 raft_consensus.cc:493] T 8d4ab870a05347868956aa8b841b2981 P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:21:26.616847 16265 leader_election.cc:304] T ecc1e363230043fe9fef390f0b979f20 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
I20260709 13:21:26.617063 21592 raft_consensus.cc:515] T 8d4ab870a05347868956aa8b841b2981 P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:26.618347 16265 leader_election.cc:304] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:21:26.619153 21592 leader_election.cc:290] T 8d4ab870a05347868956aa8b841b2981 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:26.544453 21749 consensus_queue.cc:1059] T 9e65676b21b748728ad39668cfe722ec P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: INVALID_TERM, Last received: 13.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.075s
I20260709 13:21:26.619633 21759 raft_consensus.cc:2749] T ecc1e363230043fe9fef390f0b979f20 P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:21:26.620303 21749 raft_consensus.cc:3055] T 9e65676b21b748728ad39668cfe722ec P cb82cc7bd6db46d980c75237a1402860 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:21:26.620373 21759 raft_consensus.cc:2749] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:21:26.621052 21749 raft_consensus.cc:740] T 9e65676b21b748728ad39668cfe722ec P cb82cc7bd6db46d980c75237a1402860 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:21:26.611548 21675 consensus_queue.cc:260] T 2794af5b99984efebdee564035841cb0 P cb82cc7bd6db46d980c75237a1402860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:26.622123 21749 consensus_queue.cc:260] T 9e65676b21b748728ad39668cfe722ec P cb82cc7bd6db46d980c75237a1402860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:26.623893 21434 raft_consensus.cc:2749] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:21:26.618348 21752 leader_election.cc:290] T 5a959dbecc2648eb85d1cc550f5a3245 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:26.622735 21675 raft_consensus.cc:3060] T 2794af5b99984efebdee564035841cb0 P cb82cc7bd6db46d980c75237a1402860 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:21:26.626546 21738 raft_consensus.cc:2804] T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:21:26.627216 21738 raft_consensus.cc:493] T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Starting leader election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:21:26.627640 21738 raft_consensus.cc:3060] T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:21:26.628360 21734 log.cc:927] Time spent T 3884f864a96d428785895e5463e5728f P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 0.803s	user 0.004s	sys 0.000s
I20260709 13:21:26.633250 16296 raft_consensus.cc:3055] T 26354361e9b044cea1c2b86c4882fe92 P 90f3da57796b488c9087a2f523adf045 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:21:26.633642 16296 raft_consensus.cc:740] T 26354361e9b044cea1c2b86c4882fe92 P 90f3da57796b488c9087a2f523adf045 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:21:26.634366 16296 consensus_queue.cc:260] T 26354361e9b044cea1c2b86c4882fe92 P 90f3da57796b488c9087a2f523adf045 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:21:26.635874 21777 log.cc:927] Time spent T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 0.104s	user 0.001s	sys 0.000s
W20260709 13:21:26.623796 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 76149.1us
I20260709 13:21:26.640646 21671 consensus_queue.cc:1048] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:21:26.608670 16115 leader_election.cc:304] T 8cc959fb7f704fba9009c0595467d074 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:21:26.599610 16294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7ca61592e848ea9b0fb81a2ce33cdc" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:26.643695 16790 leader_election.cc:304] T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:21:26.650014 16219 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57600 (request call id 1712) took 1116 ms. Trace:
I20260709 13:21:26.626315 16295 raft_consensus.cc:1275] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 13:21:26.650718 16219 rpcz_store.cc:276] 0709 13:21:25.533857 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:25.538126 (+  4269us) service_pool.cc:224] Handling call
0709 13:21:25.538440 (+   314us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:21:25.687225 (+148785us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:26.649671 (+962446us) spinlock_profiling.cc:243] Waited 73.7 ms on lock 0x7b54001414d4. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:21:26.649933 (+   262us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":169608960}
I20260709 13:21:26.657467 16296 raft_consensus.cc:3060] T 26354361e9b044cea1c2b86c4882fe92 P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:21:26.655274 16115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 46695.3us
I20260709 13:21:26.659940 16115 leader_election.cc:304] T fffab26427e24d398e2695aad40ca134 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:21:26.656278 16294 raft_consensus.cc:2468] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 15.
W20260709 13:21:26.670183 16192 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:26.683602 16790 leader_election.cc:400] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 17 election: Vote denied by peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943) with higher term. Message: Invalid argument: T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 for earlier term 17. Current term is 18.
I20260709 13:21:26.671837 16265 leader_election.cc:304] T d16dfad0c3b540e5b4b86492ddf87208 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:21:26.684223 16790 leader_election.cc:403] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 17 election: Cancelling election due to peer responding with higher term
W20260709 13:21:26.681311 16191 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:26.676115 16299 raft_consensus.cc:2468] T e5ad4eb281564936a59f41e387a47156 P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 16.
I20260709 13:21:26.681844 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62fe3346cd3c42c4a55e14af9073bca4" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
W20260709 13:21:26.685739 16191 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:26.687335 21759 raft_consensus.cc:2749] T d16dfad0c3b540e5b4b86492ddf87208 P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:21:26.673079 16820 raft_consensus.cc:1240] T 8c11074579b046d99e585c2818e18a2c P cb82cc7bd6db46d980c75237a1402860 [term 14 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 13. Current term is 14. Ops: []
I20260709 13:21:26.685127 16265 leader_election.cc:304] T c4f65cdf00b748bb963ee6e81ef43b79 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:21:26.700346 16191 leader_election.cc:304] T 19baa7bc367844c288bca8f81439c413 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
I20260709 13:21:26.651391 21523 raft_consensus.cc:2749] T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
W20260709 13:21:26.744841 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 72685.1us
I20260709 13:21:26.693985 16219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d4ab870a05347868956aa8b841b2981" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:26.781755 16292 raft_consensus.cc:1240] T 2794af5b99984efebdee564035841cb0 P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 11. Current term is 12. Ops: []
I20260709 13:21:26.707101 16220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a959dbecc2648eb85d1cc550f5a3245" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:26.795418 16220 raft_consensus.cc:2468] T 5a959dbecc2648eb85d1cc550f5a3245 P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 13.
W20260709 13:21:26.746956 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 61905.6us
I20260709 13:21:26.797433 16265 leader_election.cc:304] T 1df93779209748b9860f6d893dd3de53 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
W20260709 13:21:26.682451 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 333164us
W20260709 13:21:26.726146 16266 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:21:26.798553 16266 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:26.794986 16219 raft_consensus.cc:2468] T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 14.
I20260709 13:21:26.655252 21773 raft_consensus.cc:493] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:21:26.707762 21671 raft_consensus.cc:2749] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 [term 18 FOLLOWER]: Leader election lost for term 17. Reason: Vote denied by peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943) with higher term. Message: Invalid argument: T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 for earlier term 17. Current term is 18.
I20260709 13:21:26.801472 16265 leader_election.cc:304] T 8d4ab870a05347868956aa8b841b2981 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:21:26.696661 21507 consensus_queue.cc:1059] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: INVALID_TERM, Last received: 16.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.107s
I20260709 13:21:26.702899 21749 raft_consensus.cc:3060] T 9e65676b21b748728ad39668cfe722ec P cb82cc7bd6db46d980c75237a1402860 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:21:26.744256 21719 leader_election.cc:290] T 23f82224f4564fea93572e8206bad1fe P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
W20260709 13:21:26.813406 16263 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:26.747789 21736 raft_consensus.cc:2749] T c4f65cdf00b748bb963ee6e81ef43b79 P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:21:26.825024 16265 leader_election.cc:304] T 18ad7e3af5504571a69d10c00750a01c P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:21:26.690888 21713 raft_consensus.cc:493] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:26.804402 21773 raft_consensus.cc:515] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:26.828640 21755 raft_consensus.cc:493] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:26.805634 21745 raft_consensus.cc:2749] T 1df93779209748b9860f6d893dd3de53 P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:21:26.697305 21490 raft_consensus.cc:3060] T 9c61a39fd4f84424b129c1a6f9ba404d P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:21:26.667927 21245 raft_consensus.cc:2749] T 8cc959fb7f704fba9009c0595467d074 P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:21:26.845487 16265 leader_election.cc:304] T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:21:26.655450 21434 consensus_queue.cc:1048] T 19baa7bc367844c288bca8f81439c413 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.160s
I20260709 13:21:26.708128 21480 consensus_queue.cc:1059] T 8c11074579b046d99e585c2818e18a2c P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.139s
I20260709 13:21:26.858527 21751 raft_consensus.cc:493] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:26.858994 21774 consensus_queue.cc:1059] T e5ad4eb281564936a59f41e387a47156 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: INVALID_TERM, Last received: 13.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.127s
I20260709 13:21:26.863240 21743 raft_consensus.cc:493] T b87147731a68400e8cce9b879daa7c5d P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
W20260709 13:21:26.879921 21786 log.cc:927] Time spent T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 0.080s	user 0.001s	sys 0.000s
I20260709 13:21:26.845299 21759 raft_consensus.cc:2749] T 8d4ab870a05347868956aa8b841b2981 P 90f3da57796b488c9087a2f523adf045 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:21:26.848071 21648 raft_consensus.cc:2749] T 18ad7e3af5504571a69d10c00750a01c P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:21:26.877727 16788 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:21:26.878890 16192 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:26.899510 16115 leader_election.cc:304] T e5ad4eb281564936a59f41e387a47156 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac
I20260709 13:21:26.868013 16825 raft_consensus.cc:2468] T ada5a004ba7c46819051e3ccec71ffdb P cb82cc7bd6db46d980c75237a1402860 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 16.
I20260709 13:21:26.908892 16265 leader_election.cc:304] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
I20260709 13:21:26.881704 21444 raft_consensus.cc:2749] T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:21:26.909768 16265 leader_election.cc:304] T 84f2da98864b4425bf7c0055951fdd4b P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:21:26.852205 21713 raft_consensus.cc:515] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:26.911024 16265 leader_election.cc:304] T fffab26427e24d398e2695aad40ca134 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:21:26.873471 21755 raft_consensus.cc:515] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:26.915129 21245 raft_consensus.cc:2804] T e5ad4eb281564936a59f41e387a47156 P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:21:26.913401 21725 raft_consensus.cc:2749] T fffab26427e24d398e2695aad40ca134 P 647a343a542d4a139fe106c19095129b [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:21:26.911878 21736 raft_consensus.cc:2749] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:21:26.915217 21745 raft_consensus.cc:2804] T 84f2da98864b4425bf7c0055951fdd4b P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:21:26.937265 21745 raft_consensus.cc:493] T 84f2da98864b4425bf7c0055951fdd4b P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:21:26.937723 21745 raft_consensus.cc:3060] T 84f2da98864b4425bf7c0055951fdd4b P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:21:26.939010 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 134958us
I20260709 13:21:26.940660 16190 leader_election.cc:304] T 5a96373bb0e14dce8a266b2fc6c58507 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:21:26.912115 21743 raft_consensus.cc:515] T b87147731a68400e8cce9b879daa7c5d P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:26.941838 21604 raft_consensus.cc:2749] T 5a96373bb0e14dce8a266b2fc6c58507 P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:21:26.941931 16190 leader_election.cc:304] T 08c74490c40243a88e62d1760b5cd814 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:21:26.943038 21743 leader_election.cc:290] T b87147731a68400e8cce9b879daa7c5d P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
W20260709 13:21:26.920532 21733 log.cc:927] Time spent T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 1.375s	user 0.000s	sys 0.013s
I20260709 13:21:26.905224 21752 consensus_queue.cc:1048] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.019s
W20260709 13:21:26.926348 21788 log.cc:927] Time spent T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860: Append to log took a long time: real 0.075s	user 0.000s	sys 0.001s
I20260709 13:21:26.943477 21770 raft_consensus.cc:2749] T 08c74490c40243a88e62d1760b5cd814 P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:21:26.943572 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 243376us
I20260709 13:21:26.919423 21746 raft_consensus.cc:3055] T 02838c86531d4f98bf58d1c22b8baf77 P 90f3da57796b488c9087a2f523adf045 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:21:26.931737 21759 raft_consensus.cc:2804] T fffab26427e24d398e2695aad40ca134 P 90f3da57796b488c9087a2f523adf045 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:21:26.905822 21751 raft_consensus.cc:515] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:26.953215 16191 leader_election.cc:304] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
I20260709 13:21:26.940340 21738 raft_consensus.cc:515] T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:21:26.957432 16791 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:26.954825 21746 raft_consensus.cc:740] T 02838c86531d4f98bf58d1c22b8baf77 P 90f3da57796b488c9087a2f523adf045 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:21:26.956955 21759 raft_consensus.cc:493] T fffab26427e24d398e2695aad40ca134 P 90f3da57796b488c9087a2f523adf045 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:21:26.960553 21759 raft_consensus.cc:3060] T fffab26427e24d398e2695aad40ca134 P 90f3da57796b488c9087a2f523adf045 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:21:26.960475 21784 consensus_queue.cc:1048] T e5ad4eb281564936a59f41e387a47156 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.004s
I20260709 13:21:26.960952 21746 consensus_queue.cc:260] T 02838c86531d4f98bf58d1c22b8baf77 P 90f3da57796b488c9087a2f523adf045 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:26.962211 21746 raft_consensus.cc:3060] T 02838c86531d4f98bf58d1c22b8baf77 P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:21:26.958739 21769 raft_consensus.cc:3055] T e5ad4eb281564936a59f41e387a47156 P 1ca2725d290e47d68b1a4b609c9816ac [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:21:26.957118 21757 raft_consensus.cc:3055] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:21:26.963392 21769 raft_consensus.cc:740] T e5ad4eb281564936a59f41e387a47156 P 1ca2725d290e47d68b1a4b609c9816ac [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:21:26.963572 21757 raft_consensus.cc:740] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:21:26.964337 21769 consensus_queue.cc:260] T e5ad4eb281564936a59f41e387a47156 P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:26.964483 21757 consensus_queue.cc:260] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:26.965628 21769 raft_consensus.cc:3060] T e5ad4eb281564936a59f41e387a47156 P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:21:26.968433 21738 leader_election.cc:290] T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 16 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:26.953154 21507 raft_consensus.cc:2749] T 19baa7bc367844c288bca8f81439c413 P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:21:26.971822 16221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4f65cdf00b748bb963ee6e81ef43b79" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:21:26.945362 21245 raft_consensus.cc:697] T e5ad4eb281564936a59f41e387a47156 P 647a343a542d4a139fe106c19095129b [term 16 LEADER]: Becoming Leader. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:21:26.978013 21480 raft_consensus.cc:3055] T 8c11074579b046d99e585c2818e18a2c P 1ca2725d290e47d68b1a4b609c9816ac [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:21:26.978392 21751 leader_election.cc:290] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:26.978544 21480 raft_consensus.cc:740] T 8c11074579b046d99e585c2818e18a2c P 1ca2725d290e47d68b1a4b609c9816ac [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:21:26.979780 16789 leader_election.cc:304] T c9944d5e4b8c4109af9955c4f08c9095 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:21:26.979590 21480 consensus_queue.cc:260] T 8c11074579b046d99e585c2818e18a2c P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:26.980949 21480 raft_consensus.cc:3060] T 8c11074579b046d99e585c2818e18a2c P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:21:26.981302 16789 leader_election.cc:304] T 3a7ca61592e848ea9b0fb81a2ce33cdc P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:21:26.982239 16789 leader_election.cc:304] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 647a343a542d4a139fe106c19095129b
I20260709 13:21:26.983376 16297 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48684 (request call id 2865) took 1966 ms. Trace:
I20260709 13:21:26.983870 21677 raft_consensus.cc:2804] T c9944d5e4b8c4109af9955c4f08c9095 P cb82cc7bd6db46d980c75237a1402860 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:21:26.984447 21677 raft_consensus.cc:493] T c9944d5e4b8c4109af9955c4f08c9095 P cb82cc7bd6db46d980c75237a1402860 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:26.984858 21677 raft_consensus.cc:3060] T c9944d5e4b8c4109af9955c4f08c9095 P cb82cc7bd6db46d980c75237a1402860 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:21:26.986513 21762 consensus_queue.cc:1175] T 8c11074579b046d99e585c2818e18a2c P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "90f3da57796b488c9087a2f523adf045" responder_term: 13 status { last_received { term: 13 index: 12 } last_committed_idx: 12 last_received_current_leader { term: 13 index: 12 } }
I20260709 13:21:26.987362 21728 raft_consensus.cc:2804] T 3a7ca61592e848ea9b0fb81a2ce33cdc P cb82cc7bd6db46d980c75237a1402860 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:21:26.987785 21728 raft_consensus.cc:493] T 3a7ca61592e848ea9b0fb81a2ce33cdc P cb82cc7bd6db46d980c75237a1402860 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:26.985124 16190 leader_election.cc:304] T 02838c86531d4f98bf58d1c22b8baf77 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:21:26.988071 16150 raft_consensus.cc:1275] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:21:26.988293 21313 raft_consensus.cc:2804] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:21:26.988760 21313 raft_consensus.cc:493] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:26.989120 21313 raft_consensus.cc:3060] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:21:26.989326 16190 leader_election.cc:304] T d2ab0ac8347641e6955e9bce7d90c1a2 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860
I20260709 13:21:26.990238 21770 raft_consensus.cc:3060] T 02838c86531d4f98bf58d1c22b8baf77 P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:21:26.992592 16191 leader_election.cc:304] T 66f75124215f4ca6bef831e174a792d8 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
I20260709 13:21:26.995386 21507 consensus_queue.cc:1048] T e4dc6b0541ca431ebc757beac00521f1 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:21:26.996927 21507 raft_consensus.cc:2749] T d2ab0ac8347641e6955e9bce7d90c1a2 P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:21:26.997555 21507 raft_consensus.cc:2749] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:21:26.998135 21507 raft_consensus.cc:2749] T 66f75124215f4ca6bef831e174a792d8 P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:21:26.999634 21773 leader_election.cc:290] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:26.933614 21444 consensus_queue.cc:1059] T 8a878999c0024d2abe517484180290fc P 90f3da57796b488c9087a2f523adf045 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.028s
I20260709 13:21:27.000330 16190 leader_election.cc:304] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860
I20260709 13:21:27.000501 21444 consensus_queue.cc:1243] T 8a878999c0024d2abe517484180290fc P 90f3da57796b488c9087a2f523adf045 [LEADER]: Peer 647a343a542d4a139fe106c19095129b log is divergent from this leader: its last log entry 14.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 13:21:27.000598 21461 leader_election.cc:290] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:27.003561 16221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5718c623ebee4d2bb770f0f9ddb2f164" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 15 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:27.005475 16221 raft_consensus.cc:2468] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 14.
I20260709 13:21:27.007834 16190 leader_election.cc:304] T c0eab25ce4c64e018a041beeda2867db P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:21:27.008445 21324 raft_consensus.cc:2749] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:21:27.009989 16116 leader_election.cc:304] T af57996a8be84d398f0f023e8d08ef77 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860
I20260709 13:21:27.016724 21742 consensus_queue.cc:1059] T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:21:27.017621 16116 leader_election.cc:304] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
I20260709 13:21:27.017879 21742 raft_consensus.cc:2749] T af57996a8be84d398f0f023e8d08ef77 P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:21:27.013186 21245 consensus_queue.cc:237] T e5ad4eb281564936a59f41e387a47156 P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 10, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:27.019093 21441 raft_consensus.cc:3055] T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:21:27.019562 21441 raft_consensus.cc:740] T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:21:27.014276 21773 raft_consensus.cc:3055] T 8a878999c0024d2abe517484180290fc P 90f3da57796b488c9087a2f523adf045 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:21:27.020126 21773 raft_consensus.cc:740] T 8a878999c0024d2abe517484180290fc P 90f3da57796b488c9087a2f523adf045 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:21:27.020320 21736 raft_consensus.cc:493] T 8638ee363273426db1393dd81e5c7b72 P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:21:27.020540 21441 consensus_queue.cc:260] T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:27.020942 21736 raft_consensus.cc:515] T 8638ee363273426db1393dd81e5c7b72 P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:27.021046 21773 consensus_queue.cc:260] T 8a878999c0024d2abe517484180290fc P 90f3da57796b488c9087a2f523adf045 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:27.019322 21738 consensus_queue.cc:1048] T 8a878999c0024d2abe517484180290fc P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:21:27.022300 21441 raft_consensus.cc:3060] T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:21:27.006850 21579 raft_consensus.cc:3060] T 8638ee363273426db1393dd81e5c7b72 P cb82cc7bd6db46d980c75237a1402860 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:21:27.002544 21745 raft_consensus.cc:515] T 84f2da98864b4425bf7c0055951fdd4b P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:21:27.042073 16789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 58994us
I20260709 13:21:26.983771 16297 rpcz_store.cc:276] 0709 13:21:25.016637 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:25.017635 (+   998us) service_pool.cc:224] Handling call
0709 13:21:25.017999 (+   364us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:21:25.067427 (+ 49428us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:21:25.273548 (+206121us) log.cc:844] Serialized 14 byte log entry
0709 13:21:26.014570 (+741022us) spinlock_profiling.cc:243] Waited 2.01 ms on lock 0x7b18004ca680. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b693ade1 00007f69b69475b1 00007f69b07074bc 00007f69ada5548b 00007f69b68afab9 00007f69ada7169b 00007f69ada6f93f 00007f69ada71c02 00007f69b06f920b 00007f69b06fed23 00007f69b06ff23a
0709 13:21:26.014756 (+   186us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:21:26.014802 (+    46us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:21:26.014859 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:26.983128 (+968269us) spinlock_profiling.cc:243] Waited 46.7 ms on lock 0x7b54002cfbd4. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:21:26.983198 (+    70us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:21:26.983317 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":337013,"prepare.run_cpu_time_us":71,"prepare.run_wall_time_us":2617,"raft.queue_time_us":13625,"raft.run_cpu_time_us":144,"raft.run_wall_time_us":15646,"spinlock_wait_cycles":111968384,"thread_start_us":294902,"threads_started":2,"wal-append.queue_time_us":128829,"wal-append.run_cpu_time_us":9352,"wal-append.run_wall_time_us":1234489}
I20260709 13:21:27.089318 16817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434bff3082d141838068f323bdb0d1ed" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:27.090080 16817 raft_consensus.cc:2393] T 434bff3082d141838068f323bdb0d1ed P cb82cc7bd6db46d980c75237a1402860 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 15: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
I20260709 13:21:27.091765 16817 raft_consensus.cc:1275] T 2794af5b99984efebdee564035841cb0 P cb82cc7bd6db46d980c75237a1402860 [term 12 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
W20260709 13:21:27.093281 21733 log.cc:927] Time spent T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 0.131s	user 0.000s	sys 0.002s
I20260709 13:21:27.094822 16817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62fe3346cd3c42c4a55e14af9073bca4" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
W20260709 13:21:27.067641 21619 consensus_meta.cc:287] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045: Time spent flushing consensus metadata: real 0.572s	user 0.007s	sys 0.005s
I20260709 13:21:27.067756 21736 leader_election.cc:290] T 8638ee363273426db1393dd81e5c7b72 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:27.068003 21770 raft_consensus.cc:2749] T 02838c86531d4f98bf58d1c22b8baf77 P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:21:27.068317 21543 raft_consensus.cc:515] T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:27.069806 21755 leader_election.cc:290] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:27.070212 21523 raft_consensus.cc:493] T 74ca8140f35c4ae29e8a664f6bdfc829 P cb82cc7bd6db46d980c75237a1402860 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:21:27.023286 21757 raft_consensus.cc:3060] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:21:27.122289 16818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87147731a68400e8cce9b879daa7c5d" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:27.070958 21713 leader_election.cc:290] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
W20260709 13:21:27.067699 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 59947us
I20260709 13:21:27.071502 21604 raft_consensus.cc:2749] T c0eab25ce4c64e018a041beeda2867db P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:21:27.072145 21569 consensus_queue.cc:1048] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:21:27.074105 16148 raft_consensus.cc:1240] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 19. Current term is 20. Ops: []
I20260709 13:21:27.075371 21675 consensus_queue.cc:1048] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.000s
W20260709 13:21:27.076972 21588 consensus_meta.cc:287] T c0eab25ce4c64e018a041beeda2867db P 647a343a542d4a139fe106c19095129b: Time spent flushing consensus metadata: real 0.722s	user 0.010s	sys 0.005s
I20260709 13:21:27.078347 16116 leader_election.cc:304] T 5a959dbecc2648eb85d1cc550f5a3245 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
W20260709 13:21:27.082773 16189 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:26.988128 21728 raft_consensus.cc:3060] T 3a7ca61592e848ea9b0fb81a2ce33cdc P cb82cc7bd6db46d980c75237a1402860 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:21:27.063104 21742 raft_consensus.cc:2804] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Leader election won for term 17
W20260709 13:21:27.083858 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 88791.2us
W20260709 13:21:27.104897 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 191889us
I20260709 13:21:27.115195 16825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5718c623ebee4d2bb770f0f9ddb2f164" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 15 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
W20260709 13:21:27.151788 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 73511.9us
I20260709 13:21:27.115819 16820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66ae176961b74813a58bc15c9c4c6f2a" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:27.153059 16265 leader_election.cc:304] T 31969392ccdb43b491911a319e8bc7e1 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
I20260709 13:21:27.118995 16291 raft_consensus.cc:2410] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac for term 16 because replica has last-logged OpId of term: 15 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 9.
I20260709 13:21:27.125149 16190 leader_election.cc:304] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: cb82cc7bd6db46d980c75237a1402860
I20260709 13:21:27.134907 16266 leader_election.cc:304] T 23f82224f4564fea93572e8206bad1fe P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
I20260709 13:21:27.143896 16144 raft_consensus.cc:1240] T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b [term 14 FOLLOWER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 13. Current term is 14. Ops: [13.11-13.11]
I20260709 13:21:27.152853 16116 leader_election.cc:304] T 8d4ab870a05347868956aa8b841b2981 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
W20260709 13:21:27.148828 16189 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:27.150403 16818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5718c623ebee4d2bb770f0f9ddb2f164" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 15 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:27.152925 16820 raft_consensus.cc:2468] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 18.
I20260709 13:21:27.118002 16817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f82224f4564fea93572e8206bad1fe" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:27.157163 16192 leader_election.cc:304] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:21:27.158622 16265 leader_election.cc:304] T 8cc959fb7f704fba9009c0595467d074 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:21:27.158926 16117 leader_election.cc:304] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860
W20260709 13:21:27.160534 16265 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:21:27.160902 16265 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:27.161336 16265 leader_election.cc:304] T 8158bbd17b994d2eb96f560bd0f1b88d P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:21:27.123993 21523 raft_consensus.cc:515] T 74ca8140f35c4ae29e8a664f6bdfc829 P cb82cc7bd6db46d980c75237a1402860 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:21:27.170910 16266 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:27.169828 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a959dbecc2648eb85d1cc550f5a3245" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:27.164922 16190 leader_election.cc:304] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860
I20260709 13:21:27.165777 16294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434bff3082d141838068f323bdb0d1ed" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:27.166846 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d4ab870a05347868956aa8b841b2981" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:27.173183 16148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a9de347c711477e87c582d003f36925" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:27.173429 16300 raft_consensus.cc:2393] T 8d4ab870a05347868956aa8b841b2981 P 90f3da57796b488c9087a2f523adf045 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 14: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
I20260709 13:21:27.171866 16266 leader_election.cc:304] T ada5a004ba7c46819051e3ccec71ffdb P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:21:27.174489 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4f65cdf00b748bb963ee6e81ef43b79" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:21:27.178323 16300 raft_consensus.cc:3060] T c4f65cdf00b748bb963ee6e81ef43b79 P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:21:27.184567 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f82224f4564fea93572e8206bad1fe" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:27.185595 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18ad7e3af5504571a69d10c00750a01c" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:27.186220 16297 raft_consensus.cc:2468] T 18ad7e3af5504571a69d10c00750a01c P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 12.
I20260709 13:21:27.180639 21097 raft_consensus.cc:2749] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:21:27.189769 16147 raft_consensus.cc:1240] T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 15. Current term is 16. Ops: []
I20260709 13:21:27.190281 16296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87147731a68400e8cce9b879daa7c5d" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:27.190531 21084 raft_consensus.cc:2804] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:21:27.186164 21708 raft_consensus.cc:493] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:27.191896 16298 raft_consensus.cc:1275] T d16dfad0c3b540e5b4b86492ddf87208 P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:21:27.192233 21523 leader_election.cc:290] T 74ca8140f35c4ae29e8a664f6bdfc829 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:27.151386 21441 raft_consensus.cc:493] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:21:27.209895 21441 raft_consensus.cc:515] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:27.151598 21588 raft_consensus.cc:515] T c0eab25ce4c64e018a041beeda2867db P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:27.170853 21751 consensus_queue.cc:1048] T 2794af5b99984efebdee564035841cb0 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.051s
I20260709 13:21:27.214519 21588 leader_election.cc:290] T c0eab25ce4c64e018a041beeda2867db P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 16 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:27.183372 21729 raft_consensus.cc:2804] T 8cc959fb7f704fba9009c0595467d074 P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:21:27.215749 21729 raft_consensus.cc:493] T 8cc959fb7f704fba9009c0595467d074 P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:21:27.216248 21729 raft_consensus.cc:3060] T 8cc959fb7f704fba9009c0595467d074 P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:21:27.189639 16143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5718c623ebee4d2bb770f0f9ddb2f164" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 15 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:27.218194 16115 leader_election.cc:304] T 434bff3082d141838068f323bdb0d1ed P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860
I20260709 13:21:27.154655 21738 consensus_queue.cc:1059] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.045s
I20260709 13:21:27.204490 21768 raft_consensus.cc:3060] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:21:27.170017 21736 raft_consensus.cc:2749] T 31969392ccdb43b491911a319e8bc7e1 P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:21:27.223429 21648 raft_consensus.cc:2749] T 8158bbd17b994d2eb96f560bd0f1b88d P 90f3da57796b488c9087a2f523adf045 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:21:27.170217 21635 raft_consensus.cc:2804] T 8d4ab870a05347868956aa8b841b2981 P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:21:27.177867 21434 raft_consensus.cc:493] T ee9ed67006144797b1de6ea8c2013a68 P cb82cc7bd6db46d980c75237a1402860 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:27.156322 21759 raft_consensus.cc:515] T fffab26427e24d398e2695aad40ca134 P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:27.205626 21677 raft_consensus.cc:515] T c9944d5e4b8c4109af9955c4f08c9095 P cb82cc7bd6db46d980c75237a1402860 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:27.191605 21708 raft_consensus.cc:515] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:27.190958 21084 raft_consensus.cc:493] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:27.176419 21742 raft_consensus.cc:697] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [term 17 LEADER]: Becoming Leader. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:21:27.212700 21441 leader_election.cc:290] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:27.157836 21725 raft_consensus.cc:2804] T 5a959dbecc2648eb85d1cc550f5a3245 P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:21:27.149513 21555 consensus_queue.cc:1059] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: INVALID_TERM, Last received: 19.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.035s
W20260709 13:21:27.235690 16192 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 78599.3us
I20260709 13:21:27.237547 16221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9944d5e4b8c4109af9955c4f08c9095" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:27.310971 16221 raft_consensus.cc:2468] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 16.
I20260709 13:21:27.239048 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7ca61592e848ea9b0fb81a2ce33cdc" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:27.244194 16222 raft_consensus.cc:1275] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 12 index: 12. Preceding OpId from leader: term: 14 index: 13. (index mismatch)
I20260709 13:21:27.223588 21619 raft_consensus.cc:2749] T 23f82224f4564fea93572e8206bad1fe P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:21:27.176513 21324 raft_consensus.cc:2804] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:21:27.208774 21490 raft_consensus.cc:2749] T 9c61a39fd4f84424b129c1a6f9ba404d P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:21:27.297044 16820 raft_consensus.cc:1275] T 3884f864a96d428785895e5463e5728f P cb82cc7bd6db46d980c75237a1402860 [term 10 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 10 index: 11. (index mismatch)
I20260709 13:21:27.297691 16817 raft_consensus.cc:1275] T 434bff3082d141838068f323bdb0d1ed P cb82cc7bd6db46d980c75237a1402860 [term 15 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
W20260709 13:21:27.298902 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 137602us
W20260709 13:21:27.299140 16788 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:27.297374 16825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8638ee363273426db1393dd81e5c7b72" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:27.299577 16300 raft_consensus.cc:2468] T c4f65cdf00b748bb963ee6e81ef43b79 P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 16.
W20260709 13:21:27.307855 16115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 89751.8us
I20260709 13:21:27.333268 16819 raft_consensus.cc:1275] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 [term 18 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 13:21:27.338948 16115 leader_election.cc:304] T 23f82224f4564fea93572e8206bad1fe P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860
I20260709 13:21:27.315238 16221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e4bfb9090c8460eb14fa7e0d818e7f1" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:21:27.326395 16816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
W20260709 13:21:27.341274 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 176413us
I20260709 13:21:27.341820 16115 leader_election.cc:304] T 18ad7e3af5504571a69d10c00750a01c P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac
W20260709 13:21:27.309294 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 139821us
I20260709 13:21:27.318642 16291 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48684 (request call id 2874) took 1013 ms. Trace:
I20260709 13:21:27.325904 16818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a9de347c711477e87c582d003f36925" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:27.326104 16218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada5a004ba7c46819051e3ccec71ffdb" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:21:27.345827 16115 leader_election.cc:304] T b87147731a68400e8cce9b879daa7c5d P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860
I20260709 13:21:27.345714 16291 rpcz_store.cc:276] 0709 13:21:26.305002 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:26.348904 (+ 43902us) service_pool.cc:224] Handling call
0709 13:21:27.312984 (+964080us) spinlock_profiling.cc:243] Waited 762 ms on lock 0x7b5400162e54. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:21:27.318569 (+  5585us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1752254720}
I20260709 13:21:27.266039 21736 raft_consensus.cc:2804] T ada5a004ba7c46819051e3ccec71ffdb P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:21:27.336894 16825 raft_consensus.cc:2468] T 8638ee363273426db1393dd81e5c7b72 P cb82cc7bd6db46d980c75237a1402860 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 16.
I20260709 13:21:27.349409 16823 raft_consensus.cc:1275] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 [term 17 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 13:21:27.342911 16217 raft_consensus.cc:2468] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 16.
I20260709 13:21:27.350553 16818 raft_consensus.cc:2393] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 18: Already voted for candidate cb82cc7bd6db46d980c75237a1402860 in this term.
I20260709 13:21:27.267699 21759 leader_election.cc:290] T fffab26427e24d398e2695aad40ca134 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 15 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:21:27.356025 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8638ee363273426db1393dd81e5c7b72" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:27.316669 21738 raft_consensus.cc:3060] T 434bff3082d141838068f323bdb0d1ed P 647a343a542d4a139fe106c19095129b [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:21:27.298614 21635 raft_consensus.cc:493] T 8d4ab870a05347868956aa8b841b2981 P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Starting leader election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:21:27.302881 21084 raft_consensus.cc:3060] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:21:27.309751 21725 raft_consensus.cc:493] T 5a959dbecc2648eb85d1cc550f5a3245 P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:27.340596 16822 raft_consensus.cc:3060] T 806d7690c2b644959152074bb2cab0a7 P cb82cc7bd6db46d980c75237a1402860 [term 14 LEARNER]: Advancing to term 15
I20260709 13:21:27.357288 16116 leader_election.cc:304] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: cb82cc7bd6db46d980c75237a1402860
I20260709 13:21:27.325834 21324 raft_consensus.cc:493] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:27.372524 16296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74ca8140f35c4ae29e8a664f6bdfc829" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:27.374976 16224 raft_consensus.cc:2393] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 16: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
I20260709 13:21:27.386364 16296 raft_consensus.cc:2468] T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 14.
W20260709 13:21:27.372416 16791 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:27.334889 21749 raft_consensus.cc:3055] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:21:27.448721 21749 raft_consensus.cc:740] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:21:27.384701 16294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31969392ccdb43b491911a319e8bc7e1" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:21:27.363780 21736 raft_consensus.cc:697] T ada5a004ba7c46819051e3ccec71ffdb P 90f3da57796b488c9087a2f523adf045 [term 16 LEADER]: Becoming Leader. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:21:27.450776 21736 consensus_queue.cc:237] T ada5a004ba7c46819051e3ccec71ffdb P 90f3da57796b488c9087a2f523adf045 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:27.350772 21441 raft_consensus.cc:2749] T 23f82224f4564fea93572e8206bad1fe P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:21:27.453896 21441 consensus_queue.cc:1048] T d16dfad0c3b540e5b4b86492ddf87208 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:21:27.359079 21569 raft_consensus.cc:2749] T b87147731a68400e8cce9b879daa7c5d P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:21:27.458151 21569 raft_consensus.cc:2804] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:21:27.458693 21569 raft_consensus.cc:493] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b [term 14 FOLLOWER]: Starting leader election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:21:27.459197 21569 raft_consensus.cc:3060] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:21:27.422832 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:27.358856 21245 raft_consensus.cc:2804] T 18ad7e3af5504571a69d10c00750a01c P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:21:27.464449 21245 raft_consensus.cc:493] T 18ad7e3af5504571a69d10c00750a01c P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Starting leader election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:21:27.465004 21245 raft_consensus.cc:3060] T 18ad7e3af5504571a69d10c00750a01c P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:21:27.272717 21313 raft_consensus.cc:515] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:27.470026 21313 leader_election.cc:290] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 17 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:27.497403 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0eab25ce4c64e018a041beeda2867db" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:21:27.498174 16298 raft_consensus.cc:3060] T c0eab25ce4c64e018a041beeda2867db P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:21:27.499713 16789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 123291us
W20260709 13:21:27.497574 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 161129us
I20260709 13:21:27.501497 16265 leader_election.cc:304] T 68a6838d82ef4cac83507f906139d90f P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:21:27.501838 16789 leader_election.cc:304] T 74ca8140f35c4ae29e8a664f6bdfc829 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 
W20260709 13:21:27.513031 16192 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 164643us
I20260709 13:21:27.381443 21543 leader_election.cc:290] T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 14 election: Requested vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:27.379091 21635 raft_consensus.cc:3060] T 8d4ab870a05347868956aa8b841b2981 P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:21:27.379258 21725 raft_consensus.cc:3060] T 5a959dbecc2648eb85d1cc550f5a3245 P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:21:27.392421 21324 raft_consensus.cc:3060] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:21:27.552119 16296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:21:27.552996 16296 raft_consensus.cc:3060] T ecc1e363230043fe9fef390f0b979f20 P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:21:27.534253 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74ca8140f35c4ae29e8a664f6bdfc829" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
W20260709 13:21:27.561213 16115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 186162us
I20260709 13:21:27.534552 16191 leader_election.cc:304] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
W20260709 13:21:27.535475 16192 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:27.516947 21745 leader_election.cc:290] T 84f2da98864b4425bf7c0055951fdd4b P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 12 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:21:27.266245 21434 raft_consensus.cc:515] T ee9ed67006144797b1de6ea8c2013a68 P cb82cc7bd6db46d980c75237a1402860 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:21:27.632675 16789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 130944us
I20260709 13:21:27.669878 16115 leader_election.cc:304] T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac
W20260709 13:21:27.676610 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 319351us
W20260709 13:21:27.599728 21786 log.cc:927] Time spent T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 0.084s	user 0.001s	sys 0.000s
I20260709 13:21:27.510870 21555 consensus_queue.cc:1048] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.122s
I20260709 13:21:27.568811 21648 raft_consensus.cc:2749] T 68a6838d82ef4cac83507f906139d90f P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:21:27.713910 16298 raft_consensus.cc:2468] T c0eab25ce4c64e018a041beeda2867db P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 16.
I20260709 13:21:27.704051 16142 raft_consensus.cc:1217] T 2794af5b99984efebdee564035841cb0 P 647a343a542d4a139fe106c19095129b [term 12 LEADER]: Deduplicated request from leader. Original: 10.9->[11.10-11.10]   Dedup: 11.10->[]
I20260709 13:21:27.760582 16142 raft_consensus.cc:1240] T 2794af5b99984efebdee564035841cb0 P 647a343a542d4a139fe106c19095129b [term 12 LEADER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 11. Current term is 12. Ops: [11.10-11.10]
I20260709 13:21:27.655681 21784 consensus_queue.cc:1059] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: INVALID_TERM, Last received: 15.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.084s
W20260709 13:21:27.738408 21775 log.cc:927] Time spent T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.077s	user 0.000s	sys 0.001s
W20260709 13:21:27.722664 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 221252us
I20260709 13:21:27.655535 21743 raft_consensus.cc:493] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:21:27.736510 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 202078us
I20260709 13:21:27.515535 21742 consensus_queue.cc:237] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:27.701277 16296 raft_consensus.cc:2468] T ecc1e363230043fe9fef390f0b979f20 P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 17.
I20260709 13:21:27.717221 21719 raft_consensus.cc:493] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:27.775067 16265 leader_election.cc:304] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:21:27.794862 16265 leader_election.cc:304] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:21:27.751622 21802 raft_consensus.cc:2804] T 74ca8140f35c4ae29e8a664f6bdfc829 P cb82cc7bd6db46d980c75237a1402860 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:21:27.854049 16822 raft_consensus.cc:1275] T 806d7690c2b644959152074bb2cab0a7 P cb82cc7bd6db46d980c75237a1402860 [term 15 LEARNER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 15 index: 10. (term mismatch)
I20260709 13:21:27.868330 16296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9944d5e4b8c4109af9955c4f08c9095" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:21:27.810326 21648 raft_consensus.cc:2749] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:21:27.881780 16296 raft_consensus.cc:3060] T c9944d5e4b8c4109af9955c4f08c9095 P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Advancing to term 17
I20260709 13:21:27.864554 16822 pending_rounds.cc:85] T 806d7690c2b644959152074bb2cab0a7 P cb82cc7bd6db46d980c75237a1402860: Aborting all ops after (but not including) 9
I20260709 13:21:27.801290 21811 raft_consensus.cc:2749] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:21:27.900550 21828 raft_consensus.cc:493] T 8a878999c0024d2abe517484180290fc P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:27.901232 21828 raft_consensus.cc:515] T 8a878999c0024d2abe517484180290fc P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:27.804173 21743 raft_consensus.cc:515] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } }
W20260709 13:21:27.905239 21799 log.cc:927] Time spent T 19baa7bc367844c288bca8f81439c413 P cb82cc7bd6db46d980c75237a1402860: Append to log took a long time: real 0.479s	user 0.001s	sys 0.000s
W20260709 13:21:27.911859 21810 log.cc:927] Time spent T 2794af5b99984efebdee564035841cb0 P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 0.278s	user 0.001s	sys 0.000s
I20260709 13:21:27.912169 21821 raft_consensus.cc:2749] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:21:27.893942 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 99157.5us
I20260709 13:21:27.834450 21677 leader_election.cc:290] T c9944d5e4b8c4109af9955c4f08c9095 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 17 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:27.851768 21802 raft_consensus.cc:493] T 74ca8140f35c4ae29e8a664f6bdfc829 P cb82cc7bd6db46d980c75237a1402860 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:21:27.914008 21802 raft_consensus.cc:3060] T 74ca8140f35c4ae29e8a664f6bdfc829 P cb82cc7bd6db46d980c75237a1402860 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:21:27.781697 21803 consensus_queue.cc:1048] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.101s
W20260709 13:21:27.919828 21798 log.cc:927] Time spent T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 0.237s	user 0.000s	sys 0.001s
I20260709 13:21:27.920704 21803 consensus_queue.cc:1048] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:21:27.889515 21809 raft_consensus.cc:493] T 3884f864a96d428785895e5463e5728f P cb82cc7bd6db46d980c75237a1402860 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:27.905263 21743 leader_election.cc:290] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:27.923110 21809 raft_consensus.cc:515] T 3884f864a96d428785895e5463e5728f P cb82cc7bd6db46d980c75237a1402860 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:27.923959 21762 raft_consensus.cc:493] T af57996a8be84d398f0f023e8d08ef77 P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:21:27.924523 21762 raft_consensus.cc:515] T af57996a8be84d398f0f023e8d08ef77 P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:27.925663 21809 leader_election.cc:290] T 3884f864a96d428785895e5463e5728f P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:27.926518 21762 leader_election.cc:290] T af57996a8be84d398f0f023e8d08ef77 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
W20260709 13:21:27.927850 21795 log.cc:927] Time spent T 26354361e9b044cea1c2b86c4882fe92 P cb82cc7bd6db46d980c75237a1402860: Append to log took a long time: real 0.511s	user 0.001s	sys 0.000s
I20260709 13:21:27.929435 21749 consensus_queue.cc:260] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:27.930688 21749 raft_consensus.cc:3060] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:21:27.932302 21434 leader_election.cc:290] T ee9ed67006144797b1de6ea8c2013a68 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:21:27.853354 21708 leader_election.cc:290] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:27.811713 21719 raft_consensus.cc:515] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:27.936148 21719 leader_election.cc:290] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
W20260709 13:21:27.936874 21757 consensus_meta.cc:287] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac: Time spent flushing consensus metadata: real 0.810s	user 0.010s	sys 0.001s
W20260709 13:21:27.938973 21729 consensus_meta.cc:287] T 8cc959fb7f704fba9009c0595467d074 P 90f3da57796b488c9087a2f523adf045: Time spent flushing consensus metadata: real 0.722s	user 0.009s	sys 0.002s
I20260709 13:21:27.940663 16265 leader_election.cc:304] T c9944d5e4b8c4109af9955c4f08c9095 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: cb82cc7bd6db46d980c75237a1402860
I20260709 13:21:27.942766 16265 leader_election.cc:304] T 5a96373bb0e14dce8a266b2fc6c58507 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:21:27.888154 16822 pending_rounds.cc:107] T 806d7690c2b644959152074bb2cab0a7 P cb82cc7bd6db46d980c75237a1402860: Aborting uncommitted NO_OP operation due to leader change: 14.10
I20260709 13:21:27.945264 16822 raft_consensus.cc:2889] T 806d7690c2b644959152074bb2cab0a7 P cb82cc7bd6db46d980c75237a1402860 [term 15 LEARNER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:21:27.927845 21084 consensus_meta.cc:287] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac: Time spent flushing consensus metadata: real 0.525s	user 0.009s	sys 0.000s
I20260709 13:21:27.866200 21794 raft_consensus.cc:493] T 7c2095a5f4ad49408dbc7f69c606120d P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:27.970681 16265 leader_election.cc:304] T cdbd78acb4cf46298e0de8ba0d130e19 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860
I20260709 13:21:27.939605 21729 raft_consensus.cc:515] T 8cc959fb7f704fba9009c0595467d074 P 90f3da57796b488c9087a2f523adf045 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:27.980942 21097 raft_consensus.cc:2804] T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:21:27.995877 16816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7ca61592e848ea9b0fb81a2ce33cdc" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
W20260709 13:21:27.983788 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 208716us
W20260709 13:21:28.005218 16299 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.703s	user 0.008s	sys 0.001s
I20260709 13:21:28.009965 16825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af57996a8be84d398f0f023e8d08ef77" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:27.972061 21745 raft_consensus.cc:2749] T 5a96373bb0e14dce8a266b2fc6c58507 P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:21:27.976289 21769 raft_consensus.cc:493] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:21:28.009610 16299 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.754s	user 0.009s	sys 0.001s
W20260709 13:21:28.010409 16117 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.656s	user 0.008s	sys 0.002s
W20260709 13:21:28.023983 16117 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.670s	user 0.008s	sys 0.002s
W20260709 13:21:28.022542 16225 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.749s	user 0.005s	sys 0.018s
I20260709 13:21:28.014605 16296 raft_consensus.cc:2468] T c9944d5e4b8c4109af9955c4f08c9095 P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 17.
W20260709 13:21:27.997341 21768 consensus_meta.cc:287] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac: Time spent flushing consensus metadata: real 0.723s	user 0.011s	sys 0.000s
W20260709 13:21:28.008073 21738 consensus_meta.cc:287] T 434bff3082d141838068f323bdb0d1ed P 647a343a542d4a139fe106c19095129b: Time spent flushing consensus metadata: real 0.626s	user 0.003s	sys 0.007s
W20260709 13:21:28.015103 21245 consensus_meta.cc:287] T 18ad7e3af5504571a69d10c00750a01c P 647a343a542d4a139fe106c19095129b: Time spent flushing consensus metadata: real 0.550s	user 0.014s	sys 0.000s
I20260709 13:21:27.996905 21434 raft_consensus.cc:493] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:21:28.029295 16265 leader_election.cc:304] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860
W20260709 13:21:28.038214 16189 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.658s	user 0.058s	sys 0.077s
W20260709 13:21:28.038735 16189 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.659s	user 0.058s	sys 0.077s
I20260709 13:21:28.026638 21688 raft_consensus.cc:493] T 9e65676b21b748728ad39668cfe722ec P cb82cc7bd6db46d980c75237a1402860 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:28.026649 21097 raft_consensus.cc:697] T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b [term 16 LEADER]: Becoming Leader. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:21:28.029024 21743 raft_consensus.cc:493] T ee9ed67006144797b1de6ea8c2013a68 P 647a343a542d4a139fe106c19095129b [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:28.023344 21704 raft_consensus.cc:493] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:21:28.025156 16225 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.751s	user 0.005s	sys 0.019s
I20260709 13:21:28.001396 21084 raft_consensus.cc:515] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:21:28.048352 16265 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:28.054495 16223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fffab26427e24d398e2695aad40ca134" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:21:28.041805 16221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f2da98864b4425bf7c0055951fdd4b" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:21:28.061698 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:28.031253 21768 raft_consensus.cc:2749] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:21:28.005558 21811 raft_consensus.cc:2749] T cdbd78acb4cf46298e0de8ba0d130e19 P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:21:28.026933 21648 raft_consensus.cc:2764] T c9944d5e4b8c4109af9955c4f08c9095 P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Leader pre-election decision vote started in defunct term 15: won
I20260709 13:21:28.033864 21738 raft_consensus.cc:2749] T 434bff3082d141838068f323bdb0d1ed P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:21:28.004647 21794 raft_consensus.cc:515] T 7c2095a5f4ad49408dbc7f69c606120d P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:28.056735 16144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fffab26427e24d398e2695aad40ca134" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:21:28.063181 16221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806d7690c2b644959152074bb2cab0a7" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 16 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:28.029994 21769 raft_consensus.cc:515] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:28.090692 16147 raft_consensus.cc:1275] T 3884f864a96d428785895e5463e5728f P 647a343a542d4a139fe106c19095129b [term 10 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 10 index: 11. (index mismatch)
I20260709 13:21:28.091470 16222 raft_consensus.cc:1240] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 17. Current term is 19. Ops: []
I20260709 13:21:28.060078 21814 raft_consensus.cc:493] T ee9ed67006144797b1de6ea8c2013a68 P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:21:28.054175 21812 raft_consensus.cc:493] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:28.027532 21829 consensus_queue.cc:1048] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.041s
W20260709 13:21:28.074992 16791 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:28.107681 16150 raft_consensus.cc:1275] T 434bff3082d141838068f323bdb0d1ed P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 13:21:28.083626 16144 raft_consensus.cc:3060] T fffab26427e24d398e2695aad40ca134 P 647a343a542d4a139fe106c19095129b [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:21:28.048544 21828 leader_election.cc:290] T 8a878999c0024d2abe517484180290fc P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:28.084342 16116 leader_election.cc:304] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860
I20260709 13:21:28.048780 21745 raft_consensus.cc:2749] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:21:28.090261 16221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ad5dbfa99ee4b8d9e28be92b3f36db5" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:28.054874 21819 raft_consensus.cc:493] T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b [term 14 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:21:28.116830 21819 raft_consensus.cc:515] T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:28.071228 16143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f2da98864b4425bf7c0055951fdd4b" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:21:28.119535 16143 raft_consensus.cc:3060] T 84f2da98864b4425bf7c0055951fdd4b P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:21:28.120767 16218 raft_consensus.cc:2468] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 16.
I20260709 13:21:28.039270 21245 raft_consensus.cc:515] T 18ad7e3af5504571a69d10c00750a01c P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } }
I20260709 13:21:28.064321 21830 raft_consensus.cc:493] T 9e65676b21b748728ad39668cfe722ec P 647a343a542d4a139fe106c19095129b [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:28.131714 16221 raft_consensus.cc:2468] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 13.
W20260709 13:21:28.095268 21725 consensus_meta.cc:287] T 5a959dbecc2648eb85d1cc550f5a3245 P 647a343a542d4a139fe106c19095129b: Time spent flushing consensus metadata: real 0.537s	user 0.004s	sys 0.006s
I20260709 13:21:28.042001 21434 raft_consensus.cc:515] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:28.133071 16144 raft_consensus.cc:2468] T fffab26427e24d398e2695aad40ca134 P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 15.
I20260709 13:21:28.061300 21635 raft_consensus.cc:515] T 8d4ab870a05347868956aa8b841b2981 P 647a343a542d4a139fe106c19095129b [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:28.111258 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0eab25ce4c64e018a041beeda2867db" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
W20260709 13:21:28.114464 21728 consensus_meta.cc:287] T 3a7ca61592e848ea9b0fb81a2ce33cdc P cb82cc7bd6db46d980c75237a1402860: Time spent flushing consensus metadata: real 0.944s	user 0.009s	sys 0.003s
I20260709 13:21:28.065062 21704 raft_consensus.cc:515] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:28.114845 21812 raft_consensus.cc:515] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } }
W20260709 13:21:28.070379 16266 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.850s	user 0.017s	sys 0.024s
W20260709 13:21:28.160802 16266 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.941s	user 0.017s	sys 0.024s
W20260709 13:21:28.161172 16266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 989295us
W20260709 13:21:28.162137 16266 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:28.096442 21811 raft_consensus.cc:493] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:28.137439 16226 raft_consensus.cc:3060] T c0eab25ce4c64e018a041beeda2867db P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:21:28.054239 21688 raft_consensus.cc:515] T 9e65676b21b748728ad39668cfe722ec P cb82cc7bd6db46d980c75237a1402860 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:28.060760 21743 raft_consensus.cc:515] T ee9ed67006144797b1de6ea8c2013a68 P 647a343a542d4a139fe106c19095129b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:28.114481 21838 raft_consensus.cc:493] T 8c11074579b046d99e585c2818e18a2c P cb82cc7bd6db46d980c75237a1402860 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:28.132294 21738 raft_consensus.cc:2749] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:21:28.135933 21819 leader_election.cc:290] T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
W20260709 13:21:28.178776 16115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 508987us
I20260709 13:21:28.161376 21836 raft_consensus.cc:493] T 8c11074579b046d99e585c2818e18a2c P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:28.197570 16143 raft_consensus.cc:2468] T 84f2da98864b4425bf7c0055951fdd4b P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 12.
I20260709 13:21:28.110991 21814 raft_consensus.cc:515] T ee9ed67006144797b1de6ea8c2013a68 P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:28.139796 21725 raft_consensus.cc:515] T 5a959dbecc2648eb85d1cc550f5a3245 P 647a343a542d4a139fe106c19095129b [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:28.176191 21245 leader_election.cc:290] T 18ad7e3af5504571a69d10c00750a01c P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 13 election: Requested vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:21:28.185065 21729 leader_election.cc:290] T 8cc959fb7f704fba9009c0595467d074 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 14 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:21:28.197513 21543 raft_consensus.cc:493] T 02838c86531d4f98bf58d1c22b8baf77 P 647a343a542d4a139fe106c19095129b [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:28.136598 21830 raft_consensus.cc:515] T 9e65676b21b748728ad39668cfe722ec P 647a343a542d4a139fe106c19095129b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:28.161609 21728 raft_consensus.cc:515] T 3a7ca61592e848ea9b0fb81a2ce33cdc P cb82cc7bd6db46d980c75237a1402860 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:28.211697 16265 leader_election.cc:304] T fffab26427e24d398e2695aad40ca134 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:21:28.201922 21836 raft_consensus.cc:515] T 8c11074579b046d99e585c2818e18a2c P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:28.233855 16142 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46224 (request call id 881) took 1161 ms. Trace:
I20260709 13:21:28.235937 16190 leader_election.cc:304] T c2f5f89186204242bf377f7c42b2e2da P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:21:28.160928 21097 consensus_queue.cc:237] T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:28.241412 16265 leader_election.cc:304] T 84f2da98864b4425bf7c0055951fdd4b P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:21:28.243930 16142 rpcz_store.cc:276] 0709 13:21:27.072734 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:27.073266 (+   532us) service_pool.cc:224] Handling call
0709 13:21:27.073558 (+   292us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:21:27.761005 (+687447us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:28.233595 (+472590us) spinlock_profiling.cc:243] Waited 630 ms on lock 0x7b540041e1d4. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:21:28.233792 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1449667840}
I20260709 13:21:28.249482 16226 raft_consensus.cc:2468] T c0eab25ce4c64e018a041beeda2867db P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 16.
I20260709 13:21:28.219213 21704 leader_election.cc:290] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:28.251245 16116 leader_election.cc:304] T c0eab25ce4c64e018a041beeda2867db P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
I20260709 13:21:28.209484 21835 raft_consensus.cc:493] T 9e65676b21b748728ad39668cfe722ec P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
W20260709 13:21:28.245328 21324 consensus_meta.cc:287] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac: Time spent flushing consensus metadata: real 0.647s	user 0.011s	sys 0.000s
I20260709 13:21:28.222467 21814 leader_election.cc:290] T ee9ed67006144797b1de6ea8c2013a68 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:28.261919 16291 raft_consensus.cc:1275] T e5ad4eb281564936a59f41e387a47156 P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 13:21:28.262506 21847 raft_consensus.cc:493] T a378d7f796f94615b3b347119a9880fd P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:21:28.262782 21688 leader_election.cc:290] T 9e65676b21b748728ad39668cfe722ec P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:28.263317 21713 raft_consensus.cc:2749] T c2f5f89186204242bf377f7c42b2e2da P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:21:28.263526 21856 consensus_queue.cc:1048] T 3884f864a96d428785895e5463e5728f P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:21:28.283646 21856 raft_consensus.cc:2804] T 84f2da98864b4425bf7c0055951fdd4b P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:21:28.284195 21856 raft_consensus.cc:697] T 84f2da98864b4425bf7c0055951fdd4b P 90f3da57796b488c9087a2f523adf045 [term 12 LEADER]: Becoming Leader. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:21:28.285145 21856 consensus_queue.cc:237] T 84f2da98864b4425bf7c0055951fdd4b P 90f3da57796b488c9087a2f523adf045 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:28.286512 16295 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48674 (request call id 3019) took 1118 ms. Trace:
W20260709 13:21:28.286954 21834 log.cc:927] Time spent T 19baa7bc367844c288bca8f81439c413 P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 0.079s	user 0.000s	sys 0.000s
I20260709 13:21:28.217684 21843 raft_consensus.cc:493] T 66ae176961b74813a58bc15c9c4c6f2a P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:28.288187 16225 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33050 (request call id 955) took 1048 ms. Trace:
W20260709 13:21:28.288497 21806 log.cc:927] Time spent T e5ad4eb281564936a59f41e387a47156 P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 0.737s	user 0.002s	sys 0.000s
I20260709 13:21:28.265527 16223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66f75124215f4ca6bef831e174a792d8" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:28.288888 16296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ad5dbfa99ee4b8d9e28be92b3f36db5" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:28.289580 16296 raft_consensus.cc:2468] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 13.
I20260709 13:21:28.269968 16294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18ad7e3af5504571a69d10c00750a01c" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:21:28.291478 16294 raft_consensus.cc:3060] T 18ad7e3af5504571a69d10c00750a01c P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:21:28.292452 16223 raft_consensus.cc:1275] T e5ad4eb281564936a59f41e387a47156 P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 16 index: 10. (term mismatch)
I20260709 13:21:28.292870 16223 pending_rounds.cc:85] T e5ad4eb281564936a59f41e387a47156 P 1ca2725d290e47d68b1a4b609c9816ac: Aborting all ops after (but not including) 9
I20260709 13:21:28.293098 16223 pending_rounds.cc:107] T e5ad4eb281564936a59f41e387a47156 P 1ca2725d290e47d68b1a4b609c9816ac: Aborting uncommitted NO_OP operation due to leader change: 15.10
I20260709 13:21:28.293322 16223 raft_consensus.cc:2889] T e5ad4eb281564936a59f41e387a47156 P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:21:28.288563 16225 rpcz_store.cc:276] 0709 13:21:27.239628 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:27.239770 (+   142us) service_pool.cc:224] Handling call
0709 13:21:27.240141 (+   371us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:21:27.272864 (+ 32723us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:21:27.272895 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:21:27.273392 (+   497us) log.cc:844] Serialized 26 byte log entry
0709 13:21:28.057968 (+784576us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:21:28.058079 (+   111us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:28.058315 (+   236us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:21:28.287939 (+229624us) raft_consensus.cc:1685] finished
0709 13:21:28.287968 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:21:28.288132 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":32283,"raft.queue_time_us":61703,"raft.run_cpu_time_us":111,"raft.run_wall_time_us":4463,"thread_start_us":784187,"threads_started":1,"wal-append.queue_time_us":877910}
I20260709 13:21:28.295341 21729 raft_consensus.cc:2804] T fffab26427e24d398e2695aad40ca134 P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:21:28.295946 21729 raft_consensus.cc:697] T fffab26427e24d398e2695aad40ca134 P 90f3da57796b488c9087a2f523adf045 [term 15 LEADER]: Becoming Leader. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:21:28.296957 21729 consensus_queue.cc:237] T fffab26427e24d398e2695aad40ca134 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:28.300030 16824 raft_consensus.cc:1240] T 26354361e9b044cea1c2b86c4882fe92 P cb82cc7bd6db46d980c75237a1402860 [term 13 LEADER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 12. Current term is 13. Ops: []
W20260709 13:21:28.216473 21569 consensus_meta.cc:287] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b: Time spent flushing consensus metadata: real 0.757s	user 0.010s	sys 0.003s
I20260709 13:21:28.220669 21794 leader_election.cc:290] T 7c2095a5f4ad49408dbc7f69c606120d P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:21:28.219894 21769 leader_election.cc:290] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:28.278435 21324 raft_consensus.cc:515] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:28.320758 21743 leader_election.cc:290] T ee9ed67006144797b1de6ea8c2013a68 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:28.321022 21324 leader_election.cc:290] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 15 election: Requested vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:28.322023 16224 raft_consensus.cc:2368] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 for earlier term 14. Current term is 15.
I20260709 13:21:28.322445 21725 leader_election.cc:290] T 5a959dbecc2648eb85d1cc550f5a3245 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 14 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:28.323050 21084 leader_election.cc:290] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 19 election: Requested vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:28.325209 21815 consensus_queue.cc:1048] T e5ad4eb281564936a59f41e387a47156 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:21:28.325663 21762 raft_consensus.cc:3055] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:21:28.326118 21762 raft_consensus.cc:740] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:21:28.326984 21836 leader_election.cc:290] T 8c11074579b046d99e585c2818e18a2c P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:28.278344 21704 raft_consensus.cc:2804] T c0eab25ce4c64e018a041beeda2867db P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:21:28.326946 21762 consensus_queue.cc:260] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:28.327986 21704 raft_consensus.cc:697] T c0eab25ce4c64e018a041beeda2867db P 647a343a542d4a139fe106c19095129b [term 16 LEADER]: Becoming Leader. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:21:28.328365 21762 raft_consensus.cc:3060] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:21:28.328426 21728 leader_election.cc:290] T 3a7ca61592e848ea9b0fb81a2ce33cdc P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 17 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:28.283085 21635 leader_election.cc:290] T 8d4ab870a05347868956aa8b841b2981 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 14 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:28.328871 21704 consensus_queue.cc:237] T c0eab25ce4c64e018a041beeda2867db P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:28.217621 21773 raft_consensus.cc:3060] T 8a878999c0024d2abe517484180290fc P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:21:28.332886 21848 log.cc:927] Time spent T 87ac0d1f7e8a419f9bcbc04aee0e067e P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 0.065s	user 0.001s	sys 0.000s
I20260709 13:21:28.335201 21830 leader_election.cc:290] T 9e65676b21b748728ad39668cfe722ec P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:28.288340 21843 raft_consensus.cc:515] T 66ae176961b74813a58bc15c9c4c6f2a P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:21:28.336983 16191 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:28.336824 21802 raft_consensus.cc:515] T 74ca8140f35c4ae29e8a664f6bdfc829 P cb82cc7bd6db46d980c75237a1402860 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:28.338100 21847 raft_consensus.cc:515] T a378d7f796f94615b3b347119a9880fd P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:28.278932 16816 raft_consensus.cc:2393] T 3a7ca61592e848ea9b0fb81a2ce33cdc P cb82cc7bd6db46d980c75237a1402860 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 17: Already voted for candidate cb82cc7bd6db46d980c75237a1402860 in this term.
I20260709 13:21:28.280135 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806d7690c2b644959152074bb2cab0a7" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 16 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:28.339978 16299 raft_consensus.cc:2410] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b for term 16 because replica has last-logged OpId of term: 14 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 9.
I20260709 13:21:28.340293 21847 leader_election.cc:290] T a378d7f796f94615b3b347119a9880fd P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:28.340529 21769 raft_consensus.cc:493] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:21:28.246274 21543 raft_consensus.cc:515] T 02838c86531d4f98bf58d1c22b8baf77 P 647a343a542d4a139fe106c19095129b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:28.341066 21769 raft_consensus.cc:515] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:28.185734 21838 raft_consensus.cc:515] T 8c11074579b046d99e585c2818e18a2c P cb82cc7bd6db46d980c75237a1402860 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:28.342622 21543 leader_election.cc:290] T 02838c86531d4f98bf58d1c22b8baf77 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:28.225644 21812 leader_election.cc:290] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:21:28.169713 21811 raft_consensus.cc:515] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:28.343979 21838 leader_election.cc:290] T 8c11074579b046d99e585c2818e18a2c P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:28.279433 21835 raft_consensus.cc:515] T 9e65676b21b748728ad39668cfe722ec P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:28.344513 21569 raft_consensus.cc:515] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:28.344941 21811 leader_election.cc:290] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:21:28.345623 21769 leader_election.cc:290] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:28.346167 21769 raft_consensus.cc:493] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:21:28.346396 21835 leader_election.cc:290] T 9e65676b21b748728ad39668cfe722ec P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:28.346792 21569 leader_election.cc:290] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 15 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:28.346633 21769 raft_consensus.cc:515] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:28.281985 21434 leader_election.cc:290] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:21:28.311746 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18ad7e3af5504571a69d10c00750a01c" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:21:28.337698 21843 leader_election.cc:290] T 66ae176961b74813a58bc15c9c4c6f2a P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
W20260709 13:21:28.341361 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 105620us
I20260709 13:21:28.347676 16116 leader_election.cc:304] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
W20260709 13:21:28.185490 21774 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.826s	user 0.005s	sys 0.005s
I20260709 13:21:28.286969 16295 rpcz_store.cc:276] 0709 13:21:27.168534 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:27.168651 (+   117us) service_pool.cc:224] Handling call
0709 13:21:27.186038 (+ 17387us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:21:27.322821 (+136783us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:21:27.322868 (+    47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:21:27.334309 (+ 11441us) log.cc:844] Serialized 26 byte log entry
0709 13:21:27.789996 (+455687us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:21:27.790045 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:27.790123 (+    78us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:21:28.286190 (+496067us) raft_consensus.cc:1685] finished
0709 13:21:28.286215 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:21:28.286433 (+   218us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":79404,"raft.run_cpu_time_us":87,"raft.run_wall_time_us":88,"thread_start_us":469546,"threads_started":2,"wal-append.queue_time_us":904096}
W20260709 13:21:28.361266 21774 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.011s	user 0.008s	sys 0.008s
I20260709 13:21:28.378396 21752 raft_consensus.cc:3055] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:21:28.395596 16217 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33050 (request call id 952) took 1157 ms. Trace:
I20260709 13:21:28.384135 16220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:21:28.396723 16220 raft_consensus.cc:3060] T ecc1e363230043fe9fef390f0b979f20 P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:21:28.354761 21836 raft_consensus.cc:493] T 5a959dbecc2648eb85d1cc550f5a3245 P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:21:28.387552 16788 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:28.390363 16221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9944d5e4b8c4109af9955c4f08c9095" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:21:28.401968 16221 raft_consensus.cc:3060] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:21:28.393250 16294 raft_consensus.cc:2468] T 18ad7e3af5504571a69d10c00750a01c P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 13.
I20260709 13:21:28.362704 21815 raft_consensus.cc:2804] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:21:28.396271 16217 rpcz_store.cc:276] 0709 13:21:27.238500 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:27.238743 (+   243us) service_pool.cc:224] Handling call
0709 13:21:28.395363 (+1156620us) spinlock_profiling.cc:243] Waited 30.1 ms on lock 0x7b54002c8654. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:21:28.395516 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":69211904}
I20260709 13:21:28.393718 16223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee9ed67006144797b1de6ea8c2013a68" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:28.448005 16223 raft_consensus.cc:2468] T ee9ed67006144797b1de6ea8c2013a68 P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 13.
I20260709 13:21:28.408056 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b7d1ab519044eda2d5e04a07242f25" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:28.400452 21843 raft_consensus.cc:493] T 02838c86531d4f98bf58d1c22b8baf77 P 90f3da57796b488c9087a2f523adf045 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:28.416594 21836 raft_consensus.cc:515] T 5a959dbecc2648eb85d1cc550f5a3245 P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:28.417826 21850 raft_consensus.cc:493] T 7c2095a5f4ad49408dbc7f69c606120d P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:28.418946 21802 leader_election.cc:290] T 74ca8140f35c4ae29e8a664f6bdfc829 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 14 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:28.420387 21769 leader_election.cc:290] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
W20260709 13:21:28.426613 16791 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:28.439369 21830 raft_consensus.cc:493] T 26354361e9b044cea1c2b86c4882fe92 P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:21:28.439966 21592 raft_consensus.cc:493] T ecc1e363230043fe9fef390f0b979f20 P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:28.449400 16224 raft_consensus.cc:2393] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 in current term 13: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
I20260709 13:21:28.448501 21555 raft_consensus.cc:493] T 434bff3082d141838068f323bdb0d1ed P cb82cc7bd6db46d980c75237a1402860 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:28.451439 16145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:21:28.491849 16146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3884f864a96d428785895e5463e5728f" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:28.458062 16223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7ca61592e848ea9b0fb81a2ce33cdc" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:21:28.492553 16146 raft_consensus.cc:2468] T 3884f864a96d428785895e5463e5728f P 647a343a542d4a139fe106c19095129b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 10.
I20260709 13:21:28.492805 16223 raft_consensus.cc:3060] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:21:28.463647 16147 raft_consensus.cc:1240] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 19. Current term is 20. Ops: []
I20260709 13:21:28.464036 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c11074579b046d99e585c2818e18a2c" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:28.494029 16217 raft_consensus.cc:2468] T 8c11074579b046d99e585c2818e18a2c P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 14.
I20260709 13:21:28.457170 16221 raft_consensus.cc:2468] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 17.
I20260709 13:21:28.480382 16149 raft_consensus.cc:1240] T af57996a8be84d398f0f023e8d08ef77 P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 15. Current term is 16. Ops: []
I20260709 13:21:28.395900 21752 raft_consensus.cc:740] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:21:28.482309 16294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c2095a5f4ad49408dbc7f69c606120d" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:21:28.446342 21815 raft_consensus.cc:493] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:28.500813 16147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee9ed67006144797b1de6ea8c2013a68" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:28.498579 16294 raft_consensus.cc:2410] T 7c2095a5f4ad49408dbc7f69c606120d P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac for term 11 because replica has last-logged OpId of term: 10 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 5.
W20260709 13:21:28.497212 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 50048.2us
I20260709 13:21:28.501650 16147 raft_consensus.cc:2468] T ee9ed67006144797b1de6ea8c2013a68 P 647a343a542d4a139fe106c19095129b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 14.
I20260709 13:21:28.505738 16147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e65676b21b748728ad39668cfe722ec" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:28.506569 16147 raft_consensus.cc:2468] T 9e65676b21b748728ad39668cfe722ec P 647a343a542d4a139fe106c19095129b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 14.
I20260709 13:21:28.507241 16790 leader_election.cc:400] T 74ca8140f35c4ae29e8a664f6bdfc829 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 14 pre-election: Vote denied by peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943) with higher term. Message: Invalid argument: T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 for earlier term 14. Current term is 15.
I20260709 13:21:28.499895 21555 raft_consensus.cc:515] T 434bff3082d141838068f323bdb0d1ed P cb82cc7bd6db46d980c75237a1402860 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:28.488963 21847 raft_consensus.cc:493] T b87147731a68400e8cce9b879daa7c5d P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:21:28.521749 16790 leader_election.cc:304] T ee9ed67006144797b1de6ea8c2013a68 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:21:28.522217 16294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87ac0d1f7e8a419f9bcbc04aee0e067e" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:21:28.522766 16295 raft_consensus.cc:1240] T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 15. Current term is 16. Ops: [15.8-15.8]
I20260709 13:21:28.524744 16295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66ae176961b74813a58bc15c9c4c6f2a" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:28.525403 16295 raft_consensus.cc:2468] T 66ae176961b74813a58bc15c9c4c6f2a P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 18.
I20260709 13:21:28.527885 16291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7ca61592e848ea9b0fb81a2ce33cdc" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:28.528867 16300 raft_consensus.cc:1275] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:21:28.530267 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a878999c0024d2abe517484180290fc" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:28.531697 16220 raft_consensus.cc:2468] T ecc1e363230043fe9fef390f0b979f20 P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 17.
I20260709 13:21:28.532055 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c11074579b046d99e585c2818e18a2c" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:28.533423 16142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b7d1ab519044eda2d5e04a07242f25" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:28.533828 16299 raft_consensus.cc:1240] T 18ad7e3af5504571a69d10c00750a01c P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 12. Current term is 13. Ops: []
I20260709 13:21:28.534343 16142 raft_consensus.cc:2468] T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 13.
I20260709 13:21:28.500506 21769 raft_consensus.cc:493] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:28.528553 16291 raft_consensus.cc:2468] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 15.
I20260709 13:21:28.541826 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66ae176961b74813a58bc15c9c4c6f2a" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:21:28.542723 16299 raft_consensus.cc:3060] T 66ae176961b74813a58bc15c9c4c6f2a P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:21:28.503464 21815 raft_consensus.cc:3060] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:21:28.536322 16300 raft_consensus.cc:2468] T 8c11074579b046d99e585c2818e18a2c P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 13.
I20260709 13:21:28.538053 16790 leader_election.cc:304] T 8c11074579b046d99e585c2818e18a2c P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 
W20260709 13:21:28.540299 16791 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:28.480873 21843 raft_consensus.cc:515] T 02838c86531d4f98bf58d1c22b8baf77 P 90f3da57796b488c9087a2f523adf045 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:28.547916 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8158bbd17b994d2eb96f560bd0f1b88d" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:28.550392 16291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ad5dbfa99ee4b8d9e28be92b3f36db5" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:28.552546 16295 raft_consensus.cc:1240] T c4f65cdf00b748bb963ee6e81ef43b79 P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 15. Current term is 16. Ops: []
I20260709 13:21:28.553838 16790 leader_election.cc:304] T c9944d5e4b8c4109af9955c4f08c9095 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:21:28.557557 16291 raft_consensus.cc:2468] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 13.
I20260709 13:21:28.561312 16291 raft_consensus.cc:1240] T 8cc959fb7f704fba9009c0595467d074 P 90f3da57796b488c9087a2f523adf045 [term 14 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 13. Current term is 14. Ops: []
I20260709 13:21:28.563047 16291 raft_consensus.cc:1275] T cdbd78acb4cf46298e0de8ba0d130e19 P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 13:21:28.566064 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74ca8140f35c4ae29e8a664f6bdfc829" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:21:28.491593 21850 raft_consensus.cc:515] T 7c2095a5f4ad49408dbc7f69c606120d P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:28.491322 21830 raft_consensus.cc:515] T 26354361e9b044cea1c2b86c4882fe92 P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:28.566905 16300 raft_consensus.cc:3060] T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:21:28.567410 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 326092us
I20260709 13:21:28.580787 16295 raft_consensus.cc:1240] T fffab26427e24d398e2695aad40ca134 P 90f3da57796b488c9087a2f523adf045 [term 15 LEADER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 14. Current term is 15. Ops: []
I20260709 13:21:28.582312 16147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cc959fb7f704fba9009c0595467d074" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:21:28.583263 16147 raft_consensus.cc:3060] T 8cc959fb7f704fba9009c0595467d074 P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:21:28.584388 16299 raft_consensus.cc:2468] T 66ae176961b74813a58bc15c9c4c6f2a P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 19.
I20260709 13:21:28.584651 16146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806d7690c2b644959152074bb2cab0a7" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:28.579015 21825 raft_consensus.cc:2804] T c9944d5e4b8c4109af9955c4f08c9095 P cb82cc7bd6db46d980c75237a1402860 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:21:28.589723 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c2095a5f4ad49408dbc7f69c606120d" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:28.585354 16146 raft_consensus.cc:2468] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 15.
I20260709 13:21:28.593992 16146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f5f89186204242bf377f7c42b2e2da" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:28.594722 16146 raft_consensus.cc:2468] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 16.
I20260709 13:21:28.595824 21555 leader_election.cc:290] T 434bff3082d141838068f323bdb0d1ed P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:28.597193 16146 raft_consensus.cc:1240] T 5a959dbecc2648eb85d1cc550f5a3245 P 647a343a542d4a139fe106c19095129b [term 14 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 13. Current term is 14. Ops: []
I20260709 13:21:28.601058 16146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e65676b21b748728ad39668cfe722ec" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:28.602061 16146 raft_consensus.cc:2393] T 9e65676b21b748728ad39668cfe722ec P 647a343a542d4a139fe106c19095129b [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 14: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
I20260709 13:21:28.500041 21592 raft_consensus.cc:515] T ecc1e363230043fe9fef390f0b979f20 P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:28.606079 21592 leader_election.cc:290] T ecc1e363230043fe9fef390f0b979f20 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:28.544999 21832 raft_consensus.cc:2804] T ee9ed67006144797b1de6ea8c2013a68 P cb82cc7bd6db46d980c75237a1402860 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:21:28.607581 21832 raft_consensus.cc:493] T ee9ed67006144797b1de6ea8c2013a68 P cb82cc7bd6db46d980c75237a1402860 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:28.608033 21832 raft_consensus.cc:3060] T ee9ed67006144797b1de6ea8c2013a68 P cb82cc7bd6db46d980c75237a1402860 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:21:28.566298 21752 consensus_queue.cc:260] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:28.609632 21752 raft_consensus.cc:3060] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:21:28.616329 16115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 364120us
I20260709 13:21:28.618487 21635 consensus_queue.cc:1048] T e5ad4eb281564936a59f41e387a47156 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:21:28.620107 16145 raft_consensus.cc:3060] T ecc1e363230043fe9fef390f0b979f20 P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:21:28.548815 21623 raft_consensus.cc:493] T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:28.623663 21623 raft_consensus.cc:515] T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:28.626720 21623 leader_election.cc:290] T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:28.627818 16292 raft_consensus.cc:1275] T 8a878999c0024d2abe517484180290fc P 90f3da57796b488c9087a2f523adf045 [term 14 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 8. (term mismatch)
I20260709 13:21:28.639698 16292 pending_rounds.cc:85] T 8a878999c0024d2abe517484180290fc P 90f3da57796b488c9087a2f523adf045: Aborting all ops after (but not including) 7
I20260709 13:21:28.640071 16292 pending_rounds.cc:107] T 8a878999c0024d2abe517484180290fc P 90f3da57796b488c9087a2f523adf045: Aborting uncommitted NO_OP operation due to leader change: 13.8
I20260709 13:21:28.640417 16292 raft_consensus.cc:2889] T 8a878999c0024d2abe517484180290fc P 90f3da57796b488c9087a2f523adf045 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:21:28.627889 21793 log.cc:927] Time spent T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 1.559s	user 0.003s	sys 0.008s
I20260709 13:21:28.641579 21843 leader_election.cc:290] T 02838c86531d4f98bf58d1c22b8baf77 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:21:28.629299 21729 raft_consensus.cc:493] T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:28.629961 21745 raft_consensus.cc:493] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
W20260709 13:21:28.630565 16265 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:28.646147 16142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02838c86531d4f98bf58d1c22b8baf77" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:28.646600 16265 leader_election.cc:304] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:21:28.647082 16142 raft_consensus.cc:2468] T 02838c86531d4f98bf58d1c22b8baf77 P 647a343a542d4a139fe106c19095129b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 14.
I20260709 13:21:28.647977 16265 leader_election.cc:304] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:21:28.631361 21773 raft_consensus.cc:493] T 8d4ab870a05347868956aa8b841b2981 P 90f3da57796b488c9087a2f523adf045 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:28.549750 21769 raft_consensus.cc:515] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:21:28.649413 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 63297.1us
I20260709 13:21:28.618933 16115 leader_election.cc:304] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:21:28.650688 21825 raft_consensus.cc:697] T c9944d5e4b8c4109af9955c4f08c9095 P cb82cc7bd6db46d980c75237a1402860 [term 17 LEADER]: Becoming Leader. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:21:28.559795 21838 raft_consensus.cc:2804] T 8c11074579b046d99e585c2818e18a2c P cb82cc7bd6db46d980c75237a1402860 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:21:28.657177 16149 raft_consensus.cc:2468] T 7c2095a5f4ad49408dbc7f69c606120d P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 11.
I20260709 13:21:28.662919 16115 leader_election.cc:304] T 18ad7e3af5504571a69d10c00750a01c P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac
I20260709 13:21:28.663977 16300 raft_consensus.cc:2468] T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 15.
I20260709 13:21:28.667249 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a959dbecc2648eb85d1cc550f5a3245" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:21:28.667769 16147 raft_consensus.cc:2468] T 8cc959fb7f704fba9009c0595467d074 P 647a343a542d4a139fe106c19095129b [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 14.
I20260709 13:21:28.667959 16220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74ca8140f35c4ae29e8a664f6bdfc829" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:21:28.669579 16220 raft_consensus.cc:2368] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 for earlier term 14. Current term is 15.
I20260709 13:21:28.671412 16145 raft_consensus.cc:2468] T ecc1e363230043fe9fef390f0b979f20 P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 17.
I20260709 13:21:28.642925 21729 raft_consensus.cc:515] T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:28.674816 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e65676b21b748728ad39668cfe722ec" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:28.675786 16300 raft_consensus.cc:2468] T 9e65676b21b748728ad39668cfe722ec P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 13.
I20260709 13:21:28.655220 21843 raft_consensus.cc:2804] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:21:28.682145 16295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f5f89186204242bf377f7c42b2e2da" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:28.682250 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d4ab870a05347868956aa8b841b2981" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:21:28.546218 21847 raft_consensus.cc:515] T b87147731a68400e8cce9b879daa7c5d P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:28.683931 16265 leader_election.cc:304] T 02838c86531d4f98bf58d1c22b8baf77 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:21:28.671130 21588 raft_consensus.cc:2749] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:21:28.571235 21835 consensus_queue.cc:1048] T 3884f864a96d428785895e5463e5728f P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.090s
I20260709 13:21:28.639428 21728 consensus_queue.cc:1059] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.035s
I20260709 13:21:28.661171 21838 raft_consensus.cc:493] T 8c11074579b046d99e585c2818e18a2c P cb82cc7bd6db46d980c75237a1402860 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:28.690864 16296 raft_consensus.cc:1240] T 84f2da98864b4425bf7c0055951fdd4b P 90f3da57796b488c9087a2f523adf045 [term 12 LEADER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 11. Current term is 12. Ops: []
I20260709 13:21:28.673532 21704 raft_consensus.cc:2804] T 18ad7e3af5504571a69d10c00750a01c P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:21:28.656517 21745 raft_consensus.cc:515] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:28.679432 21648 raft_consensus.cc:2804] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 13:21:28.710765 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 51816.5us
I20260709 13:21:28.684561 16291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02838c86531d4f98bf58d1c22b8baf77" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:28.691057 16265 leader_election.cc:304] T 8cc959fb7f704fba9009c0595467d074 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:21:28.657631 21773 raft_consensus.cc:515] T 8d4ab870a05347868956aa8b841b2981 P 90f3da57796b488c9087a2f523adf045 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:28.687563 21843 raft_consensus.cc:493] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:21:28.713665 16266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 182962us
I20260709 13:21:28.716250 16221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d4ab870a05347868956aa8b841b2981" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:21:28.717000 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee9ed67006144797b1de6ea8c2013a68" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:28.721494 16217 raft_consensus.cc:2468] T ee9ed67006144797b1de6ea8c2013a68 P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 13.
I20260709 13:21:28.717723 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a959dbecc2648eb85d1cc550f5a3245" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:21:28.701043 21801 raft_consensus.cc:493] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:28.690297 21736 consensus_queue.cc:1059] T 5a959dbecc2648eb85d1cc550f5a3245 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.015s
I20260709 13:21:28.690265 21706 raft_consensus.cc:493] T 829b4ad8c5a54eababeea1181b960d4b P cb82cc7bd6db46d980c75237a1402860 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:21:28.722725 16224 raft_consensus.cc:3060] T 5a959dbecc2648eb85d1cc550f5a3245 P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:21:28.691754 21855 raft_consensus.cc:2804] T 02838c86531d4f98bf58d1c22b8baf77 P 90f3da57796b488c9087a2f523adf045 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:21:28.706291 21838 raft_consensus.cc:3060] T 8c11074579b046d99e585c2818e18a2c P cb82cc7bd6db46d980c75237a1402860 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:21:28.739470 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 17 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:28.740298 16298 raft_consensus.cc:2393] T ecc1e363230043fe9fef390f0b979f20 P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 17: Already voted for candidate cb82cc7bd6db46d980c75237a1402860 in this term.
I20260709 13:21:28.719099 21704 raft_consensus.cc:697] T 18ad7e3af5504571a69d10c00750a01c P 647a343a542d4a139fe106c19095129b [term 13 LEADER]: Becoming Leader. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:21:28.735548 16297 raft_consensus.cc:2393] T 8d4ab870a05347868956aa8b841b2981 P 90f3da57796b488c9087a2f523adf045 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 14: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
I20260709 13:21:28.735360 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5718c623ebee4d2bb770f0f9ddb2f164" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:21:28.709590 21648 raft_consensus.cc:493] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:28.737227 16221 raft_consensus.cc:2468] T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 14.
I20260709 13:21:28.737376 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f5f89186204242bf377f7c42b2e2da" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:28.785887 16226 raft_consensus.cc:2468] T c2f5f89186204242bf377f7c42b2e2da P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 16.
I20260709 13:21:28.786230 16221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02838c86531d4f98bf58d1c22b8baf77" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:28.703575 16223 raft_consensus.cc:2468] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 17.
I20260709 13:21:28.773307 16142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87147731a68400e8cce9b879daa7c5d" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:28.728178 21843 raft_consensus.cc:3060] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:21:28.731913 21867 raft_consensus.cc:2804] T 8cc959fb7f704fba9009c0595467d074 P 90f3da57796b488c9087a2f523adf045 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:21:28.755618 21729 leader_election.cc:290] T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:21:28.739212 21855 raft_consensus.cc:493] T 02838c86531d4f98bf58d1c22b8baf77 P 90f3da57796b488c9087a2f523adf045 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:28.758723 16147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c61a39fd4f84424b129c1a6f9ba404d" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 17 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:28.800771 16223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 17 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:28.801808 16223 raft_consensus.cc:2393] T ecc1e363230043fe9fef390f0b979f20 P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 17: Already voted for candidate cb82cc7bd6db46d980c75237a1402860 in this term.
I20260709 13:21:28.801913 16147 raft_consensus.cc:2410] T 9c61a39fd4f84424b129c1a6f9ba404d P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 for term 17 because replica has last-logged OpId of term: 15 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 7.
I20260709 13:21:28.734403 21836 leader_election.cc:290] T 5a959dbecc2648eb85d1cc550f5a3245 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:28.763854 16266 leader_election.cc:304] T 8638ee363273426db1393dd81e5c7b72 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac
I20260709 13:21:28.769601 21847 leader_election.cc:290] T b87147731a68400e8cce9b879daa7c5d P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:28.786948 16221 raft_consensus.cc:2468] T 02838c86531d4f98bf58d1c22b8baf77 P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 14.
I20260709 13:21:28.733516 21801 raft_consensus.cc:515] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:21:28.800060 16115 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:28.738862 21706 raft_consensus.cc:515] T 829b4ad8c5a54eababeea1181b960d4b P cb82cc7bd6db46d980c75237a1402860 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:21:28.796355 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 84052.4us
I20260709 13:21:28.777434 21588 raft_consensus.cc:493] T 746aabf856ab401c9ef5e859371e7b67 P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:21:28.728420 21825 consensus_queue.cc:237] T c9944d5e4b8c4109af9955c4f08c9095 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:28.793375 21648 raft_consensus.cc:3060] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:21:28.828333 16142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af57996a8be84d398f0f023e8d08ef77" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:28.849848 16142 raft_consensus.cc:2393] T af57996a8be84d398f0f023e8d08ef77 P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 16: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
I20260709 13:21:28.850106 16146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a878999c0024d2abe517484180290fc" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:28.851285 16142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee9ed67006144797b1de6ea8c2013a68" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:28.852402 16142 raft_consensus.cc:2393] T ee9ed67006144797b1de6ea8c2013a68 P 647a343a542d4a139fe106c19095129b [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 14: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
I20260709 13:21:28.852517 16221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c2095a5f4ad49408dbc7f69c606120d" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
W20260709 13:21:28.853477 16266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 89761.5us
I20260709 13:21:28.833984 21580 raft_consensus.cc:493] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P cb82cc7bd6db46d980c75237a1402860 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:28.873238 21580 raft_consensus.cc:515] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P cb82cc7bd6db46d980c75237a1402860 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:28.869360 21812 raft_consensus.cc:2804] T 8638ee363273426db1393dd81e5c7b72 P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:21:28.874785 21812 raft_consensus.cc:493] T 8638ee363273426db1393dd81e5c7b72 P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Starting leader election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:21:28.875270 21812 raft_consensus.cc:3060] T 8638ee363273426db1393dd81e5c7b72 P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:21:28.875741 21580 leader_election.cc:290] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:28.868538 16221 raft_consensus.cc:2468] T 7c2095a5f4ad49408dbc7f69c606120d P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 11.
I20260709 13:21:28.823035 16116 leader_election.cc:304] T ee9ed67006144797b1de6ea8c2013a68 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
W20260709 13:21:28.877003 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 54058.6us
I20260709 13:21:28.849749 21867 raft_consensus.cc:697] T 8cc959fb7f704fba9009c0595467d074 P 90f3da57796b488c9087a2f523adf045 [term 14 LEADER]: Becoming Leader. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:21:28.877738 16116 leader_election.cc:304] T 8d4ab870a05347868956aa8b841b2981 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
I20260709 13:21:28.868711 16142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ad5dbfa99ee4b8d9e28be92b3f36db5" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:28.878640 16116 leader_election.cc:304] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
I20260709 13:21:28.867658 16266 leader_election.cc:304] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
I20260709 13:21:28.879550 21847 raft_consensus.cc:3060] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:21:28.882414 16143 raft_consensus.cc:1275] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 13:21:28.833609 21569 raft_consensus.cc:493] T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:28.871320 16816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31969392ccdb43b491911a319e8bc7e1" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:21:28.871130 21704 consensus_queue.cc:237] T 18ad7e3af5504571a69d10c00750a01c P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } }
I20260709 13:21:28.883451 21569 raft_consensus.cc:515] T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:28.885215 16116 leader_election.cc:304] T 02838c86531d4f98bf58d1c22b8baf77 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
I20260709 13:21:28.885685 21569 leader_election.cc:290] T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:28.885980 21543 raft_consensus.cc:2804] T 02838c86531d4f98bf58d1c22b8baf77 P 647a343a542d4a139fe106c19095129b [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:21:28.886410 21543 raft_consensus.cc:493] T 02838c86531d4f98bf58d1c22b8baf77 P 647a343a542d4a139fe106c19095129b [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:28.886650 21830 leader_election.cc:290] T 26354361e9b044cea1c2b86c4882fe92 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:28.886741 21543 raft_consensus.cc:3060] T 02838c86531d4f98bf58d1c22b8baf77 P 647a343a542d4a139fe106c19095129b [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:21:28.853366 16147 raft_consensus.cc:1240] T 84f2da98864b4425bf7c0055951fdd4b P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 11. Current term is 12. Ops: []
I20260709 13:21:28.872619 16821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66f75124215f4ca6bef831e174a792d8" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:28.755677 21736 consensus_queue.cc:1048] T 434bff3082d141838068f323bdb0d1ed P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.044s
I20260709 13:21:28.890542 16190 leader_election.cc:304] T 7c2095a5f4ad49408dbc7f69c606120d P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:21:28.877084 21592 raft_consensus.cc:2804] T ee9ed67006144797b1de6ea8c2013a68 P 647a343a542d4a139fe106c19095129b [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:21:28.808198 21769 leader_election.cc:290] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:28.878144 21867 consensus_queue.cc:237] T 8cc959fb7f704fba9009c0595467d074 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:28.878501 21751 raft_consensus.cc:2804] T 8d4ab870a05347868956aa8b841b2981 P 647a343a542d4a139fe106c19095129b [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:21:28.879503 21490 raft_consensus.cc:2804] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:21:28.872264 16818 raft_consensus.cc:1240] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860 [term 18 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 17. Current term is 18. Ops: [17.13-17.13]
W20260709 13:21:28.895851 16117 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 68409.6us
I20260709 13:21:28.851342 21850 leader_election.cc:290] T 7c2095a5f4ad49408dbc7f69c606120d P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:28.806764 21855 raft_consensus.cc:3060] T 02838c86531d4f98bf58d1c22b8baf77 P 90f3da57796b488c9087a2f523adf045 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:21:28.902976 16147 raft_consensus.cc:1240] T fffab26427e24d398e2695aad40ca134 P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 14. Current term is 15. Ops: []
W20260709 13:21:28.899000 21848 log.cc:927] Time spent T 87ac0d1f7e8a419f9bcbc04aee0e067e P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
I20260709 13:21:28.904284 16825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e65676b21b748728ad39668cfe722ec" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:28.827739 16790 leader_election.cc:304] T 3884f864a96d428785895e5463e5728f P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:21:28.850021 21434 raft_consensus.cc:493] T 8638ee363273426db1393dd81e5c7b72 P cb82cc7bd6db46d980c75237a1402860 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:28.908254 16816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee9ed67006144797b1de6ea8c2013a68" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:28.830791 21836 raft_consensus.cc:493] T 19baa7bc367844c288bca8f81439c413 P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:21:28.909293 16821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5718c623ebee4d2bb770f0f9ddb2f164" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:21:28.910326 16819 raft_consensus.cc:1275] T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860 [term 17 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 13:21:28.834972 21588 raft_consensus.cc:515] T 746aabf856ab401c9ef5e859371e7b67 P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:28.920722 16146 raft_consensus.cc:1275] T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 13:21:28.922988 16224 raft_consensus.cc:2468] T 5a959dbecc2648eb85d1cc550f5a3245 P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 14.
W20260709 13:21:28.951782 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 124127us
I20260709 13:21:29.005231 16819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26354361e9b044cea1c2b86c4882fe92" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:28.952795 21749 raft_consensus.cc:2804] T 3884f864a96d428785895e5463e5728f P cb82cc7bd6db46d980c75237a1402860 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:21:28.876585 21801 leader_election.cc:290] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:28.953313 21434 raft_consensus.cc:515] T 8638ee363273426db1393dd81e5c7b72 P cb82cc7bd6db46d980c75237a1402860 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:28.955677 21706 leader_election.cc:290] T 829b4ad8c5a54eababeea1181b960d4b P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:21:28.956020 21773 leader_election.cc:290] T 8d4ab870a05347868956aa8b841b2981 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:21:29.068825 16146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8158bbd17b994d2eb96f560bd0f1b88d" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
W20260709 13:21:28.956840 16265 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:28.957615 21742 raft_consensus.cc:493] T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:21:28.951186 16825 raft_consensus.cc:2468] T 9e65676b21b748728ad39668cfe722ec P cb82cc7bd6db46d980c75237a1402860 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 14.
W20260709 13:21:28.958727 21860 log.cc:927] Time spent T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 0.089s	user 0.000s	sys 0.000s
I20260709 13:21:28.889869 21835 raft_consensus.cc:493] T 26354361e9b044cea1c2b86c4882fe92 P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
W20260709 13:21:28.967350 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 57006.3us
W20260709 13:21:28.971431 21861 log.cc:927] Time spent T ada5a004ba7c46819051e3ccec71ffdb P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.052s	user 0.000s	sys 0.001s
I20260709 13:21:28.972510 16821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2372981006d4b218215c28964dea623" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:28.973018 16789 leader_election.cc:304] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:21:28.977402 16294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a378d7f796f94615b3b347119a9880fd" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:28.978788 21836 raft_consensus.cc:515] T 19baa7bc367844c288bca8f81439c413 P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:28.981917 16151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a959dbecc2648eb85d1cc550f5a3245" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:29.087800 16294 raft_consensus.cc:2468] T a378d7f796f94615b3b347119a9880fd P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 16.
I20260709 13:21:29.088158 16151 raft_consensus.cc:2393] T 5a959dbecc2648eb85d1cc550f5a3245 P 647a343a542d4a139fe106c19095129b [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 14: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
I20260709 13:21:28.983736 16292 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48674 (request call id 3012) took 1818 ms. Trace:
I20260709 13:21:29.089613 16292 rpcz_store.cc:276] 0709 13:21:27.165686 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:27.166290 (+   604us) service_pool.cc:224] Handling call
0709 13:21:27.186588 (+ 20298us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:21:28.641043 (+1454455us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:28.983417 (+342374us) spinlock_profiling.cc:243] Waited 1.44 s on lock 0x7b540041c3d4. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:21:28.983654 (+   237us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3313340288}
I20260709 13:21:28.989940 21815 raft_consensus.cc:515] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:28.991216 16818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 17 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:29.091325 16818 raft_consensus.cc:2393] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 17: Already voted for candidate cb82cc7bd6db46d980c75237a1402860 in this term.
I20260709 13:21:29.091403 16142 raft_consensus.cc:2393] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 14: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
I20260709 13:21:28.901758 21755 raft_consensus.cc:2749] T 7c2095a5f4ad49408dbc7f69c606120d P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:21:29.092730 21815 leader_election.cc:290] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 14 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:29.094377 21812 raft_consensus.cc:515] T 8638ee363273426db1393dd81e5c7b72 P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:21:29.094902 16789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 121971us
I20260709 13:21:29.096603 21812 leader_election.cc:290] T 8638ee363273426db1393dd81e5c7b72 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 16 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:28.993781 21843 raft_consensus.cc:515] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } }
I20260709 13:21:28.996390 21832 raft_consensus.cc:515] T ee9ed67006144797b1de6ea8c2013a68 P cb82cc7bd6db46d980c75237a1402860 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:28.899039 21592 raft_consensus.cc:493] T ee9ed67006144797b1de6ea8c2013a68 P 647a343a542d4a139fe106c19095129b [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:29.105887 21592 raft_consensus.cc:3060] T ee9ed67006144797b1de6ea8c2013a68 P 647a343a542d4a139fe106c19095129b [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:21:29.106261 16816 raft_consensus.cc:2393] T ee9ed67006144797b1de6ea8c2013a68 P cb82cc7bd6db46d980c75237a1402860 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 15: Already voted for candidate cb82cc7bd6db46d980c75237a1402860 in this term.
I20260709 13:21:29.107410 21832 leader_election.cc:290] T ee9ed67006144797b1de6ea8c2013a68 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 15 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:21:29.108556 16298 raft_consensus.cc:1240] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 15. Current term is 16. Ops: []
I20260709 13:21:28.908720 21738 raft_consensus.cc:493] T 1df93779209748b9860f6d893dd3de53 P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:21:29.109323 21738 raft_consensus.cc:515] T 1df93779209748b9860f6d893dd3de53 P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:29.111207 21843 leader_election.cc:290] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 16 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:21:29.112289 21738 leader_election.cc:290] T 1df93779209748b9860f6d893dd3de53 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:28.905578 21490 raft_consensus.cc:493] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:29.116142 21490 raft_consensus.cc:3060] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:21:29.006152 16790 leader_election.cc:304] T 9e65676b21b748728ad39668cfe722ec P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860; no voters: 
W20260709 13:21:29.119637 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 113559us
I20260709 13:21:29.119900 21832 raft_consensus.cc:2804] T 9e65676b21b748728ad39668cfe722ec P cb82cc7bd6db46d980c75237a1402860 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:21:29.120373 16790 leader_election.cc:304] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac
I20260709 13:21:29.120780 21832 raft_consensus.cc:493] T 9e65676b21b748728ad39668cfe722ec P cb82cc7bd6db46d980c75237a1402860 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:29.120986 21434 leader_election.cc:290] T 8638ee363273426db1393dd81e5c7b72 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:29.121224 21832 raft_consensus.cc:3060] T 9e65676b21b748728ad39668cfe722ec P cb82cc7bd6db46d980c75237a1402860 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:21:29.121585 21434 raft_consensus.cc:2804] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:21:29.121961 21434 raft_consensus.cc:493] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:21:29.122323 21434 raft_consensus.cc:3060] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:21:29.057150 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a959dbecc2648eb85d1cc550f5a3245" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:28.971009 21847 raft_consensus.cc:2749] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:21:29.085127 16190 leader_election.cc:304] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:21:29.129078 16190 leader_election.cc:304] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:21:29.010840 21749 raft_consensus.cc:493] T 3884f864a96d428785895e5463e5728f P cb82cc7bd6db46d980c75237a1402860 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:29.130142 21749 raft_consensus.cc:3060] T 3884f864a96d428785895e5463e5728f P cb82cc7bd6db46d980c75237a1402860 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:21:29.130517 16825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e65676b21b748728ad39668cfe722ec" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:29.069340 21801 raft_consensus.cc:493] T d2ab0ac8347641e6955e9bce7d90c1a2 P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:21:29.131500 21801 raft_consensus.cc:515] T d2ab0ac8347641e6955e9bce7d90c1a2 P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:29.099761 16821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66ae176961b74813a58bc15c9c4c6f2a" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:29.133688 21801 leader_election.cc:290] T d2ab0ac8347641e6955e9bce7d90c1a2 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:29.134120 16821 raft_consensus.cc:2468] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 18.
I20260709 13:21:29.102195 16819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87147731a68400e8cce9b879daa7c5d" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:29.103700 16820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8638ee363273426db1393dd81e5c7b72" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:21:29.100978 16818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a378d7f796f94615b3b347119a9880fd" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:29.138088 16818 raft_consensus.cc:2393] T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 17: Already voted for candidate cb82cc7bd6db46d980c75237a1402860 in this term.
I20260709 13:21:29.140552 16266 leader_election.cc:304] T 66ae176961b74813a58bc15c9c4c6f2a P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:21:29.141748 16266 leader_election.cc:304] T b87147731a68400e8cce9b879daa7c5d P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
I20260709 13:21:29.043514 16147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62fe3346cd3c42c4a55e14af9073bca4" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:29.142890 16147 raft_consensus.cc:2393] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 20: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
I20260709 13:21:29.136479 16190 leader_election.cc:304] T 8c11074579b046d99e585c2818e18a2c P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:21:28.998920 21648 raft_consensus.cc:515] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:29.011006 21728 consensus_queue.cc:1059] T af57996a8be84d398f0f023e8d08ef77 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: INVALID_TERM, Last received: 15.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.092s
I20260709 13:21:29.285030 21728 raft_consensus.cc:3055] T af57996a8be84d398f0f023e8d08ef77 P cb82cc7bd6db46d980c75237a1402860 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:21:29.285410 21728 raft_consensus.cc:740] T af57996a8be84d398f0f023e8d08ef77 P cb82cc7bd6db46d980c75237a1402860 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:21:29.286275 21728 consensus_queue.cc:260] T af57996a8be84d398f0f023e8d08ef77 P cb82cc7bd6db46d980c75237a1402860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:29.287364 21801 raft_consensus.cc:2804] T 8c11074579b046d99e585c2818e18a2c P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:21:29.287907 21801 raft_consensus.cc:493] T 8c11074579b046d99e585c2818e18a2c P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:29.288316 21801 raft_consensus.cc:3060] T 8c11074579b046d99e585c2818e18a2c P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:21:29.277205 21832 raft_consensus.cc:515] T 9e65676b21b748728ad39668cfe722ec P cb82cc7bd6db46d980c75237a1402860 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:21:29.231472 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 95070.4us
I20260709 13:21:29.293010 16190 leader_election.cc:304] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:21:29.293543 21832 leader_election.cc:290] T 9e65676b21b748728ad39668cfe722ec P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 15 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:29.294775 16825 raft_consensus.cc:2368] T 9e65676b21b748728ad39668cfe722ec P cb82cc7bd6db46d980c75237a1402860 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 for earlier term 14. Current term is 15.
I20260709 13:21:29.295646 21769 raft_consensus.cc:2804] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:21:29.296137 21769 raft_consensus.cc:493] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:29.296550 21769 raft_consensus.cc:3060] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:21:29.298928 21444 raft_consensus.cc:493] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:21:29.299494 21444 raft_consensus.cc:515] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:29.010413 21838 raft_consensus.cc:515] T 8c11074579b046d99e585c2818e18a2c P cb82cc7bd6db46d980c75237a1402860 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:29.230949 21755 consensus_queue.cc:1048] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.026s
I20260709 13:21:29.231218 21812 raft_consensus.cc:2764] T 66ae176961b74813a58bc15c9c4c6f2a P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Leader pre-election decision vote started in defunct term 18: won
I20260709 13:21:29.257567 21804 consensus_queue.cc:1059] T 18ad7e3af5504571a69d10c00750a01c P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: INVALID_TERM, Last received: 12.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:21:29.258342 21588 leader_election.cc:290] T 746aabf856ab401c9ef5e859371e7b67 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:29.103693 21706 raft_consensus.cc:2804] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:21:29.151357 21847 raft_consensus.cc:493] T 2794af5b99984efebdee564035841cb0 P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:21:29.260741 21648 leader_election.cc:290] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 17 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:21:29.231506 21708 raft_consensus.cc:2764] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Leader pre-election decision vote started in defunct term 16: won
I20260709 13:21:29.261794 16295 raft_consensus.cc:2393] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 17: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
I20260709 13:21:29.262185 21729 raft_consensus.cc:493] T e5ad4eb281564936a59f41e387a47156 P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:29.265000 21543 raft_consensus.cc:515] T 02838c86531d4f98bf58d1c22b8baf77 P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:29.266657 16820 raft_consensus.cc:2468] T 8638ee363273426db1393dd81e5c7b72 P cb82cc7bd6db46d980c75237a1402860 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 16.
I20260709 13:21:29.252437 21490 raft_consensus.cc:515] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:29.283870 21843 raft_consensus.cc:2749] T b87147731a68400e8cce9b879daa7c5d P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:21:29.099573 21773 raft_consensus.cc:493] T d2ab0ac8347641e6955e9bce7d90c1a2 P 90f3da57796b488c9087a2f523adf045 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
W20260709 13:21:29.329026 16144 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.655s	user 0.011s	sys 0.042s
W20260709 13:21:29.351645 16144 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.678s	user 0.012s	sys 0.042s
I20260709 13:21:29.104475 21736 raft_consensus.cc:493] T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:21:29.146967 21480 raft_consensus.cc:2749] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:21:29.086233 21742 raft_consensus.cc:515] T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:29.340049 16266 leader_election.cc:304] T 8638ee363273426db1393dd81e5c7b72 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:21:29.319424 21811 raft_consensus.cc:493] T 8a878999c0024d2abe517484180290fc P 90f3da57796b488c9087a2f523adf045 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:29.099563 21835 raft_consensus.cc:515] T 26354361e9b044cea1c2b86c4882fe92 P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:29.381058 16824 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49892 (request call id 827) took 2890 ms. Trace:
I20260709 13:21:29.381454 16824 rpcz_store.cc:276] 0709 13:21:26.490752 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:26.490894 (+   142us) service_pool.cc:224] Handling call
0709 13:21:26.527555 (+ 36661us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:21:28.300591 (+1773036us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:29.380842 (+1080251us) spinlock_profiling.cc:243] Waited 1.77 s on lock 0x7b5400520dd4. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:21:29.381004 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4075816320}
I20260709 13:21:29.286813 21592 raft_consensus.cc:515] T ee9ed67006144797b1de6ea8c2013a68 P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:21:29.393203 16218 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57600 (request call id 1735) took 2077 ms (client timeout 1928 ms). Trace:
W20260709 13:21:29.393774 16218 rpcz_store.cc:269] 0709 13:21:27.315907 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:27.316036 (+   129us) service_pool.cc:224] Handling call
0709 13:21:29.392949 (+2076913us) spinlock_profiling.cc:243] Waited 745 ms on lock 0x7b54001414d4. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:21:29.393110 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1714364160}
W20260709 13:21:29.395181 21725 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.728s	user 0.026s	sys 0.080s
I20260709 13:21:29.349262 21706 raft_consensus.cc:697] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860 [term 17 LEADER]: Becoming Leader. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:21:29.340804 21855 raft_consensus.cc:515] T 02838c86531d4f98bf58d1c22b8baf77 P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:29.412160 21543 leader_election.cc:290] T 02838c86531d4f98bf58d1c22b8baf77 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 15 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:29.413584 21490 leader_election.cc:290] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 17 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
W20260709 13:21:29.419203 21725 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.743s	user 0.026s	sys 0.080s
I20260709 13:21:29.380548 21736 raft_consensus.cc:515] T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:29.427922 21736 leader_election.cc:290] T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:21:29.428005 21856 raft_consensus.cc:3055] T 5a959dbecc2648eb85d1cc550f5a3245 P 90f3da57796b488c9087a2f523adf045 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:21:29.428678 21856 raft_consensus.cc:740] T 5a959dbecc2648eb85d1cc550f5a3245 P 90f3da57796b488c9087a2f523adf045 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:21:29.442663 16824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:29.349004 21729 raft_consensus.cc:515] T e5ad4eb281564936a59f41e387a47156 P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:29.395438 21812 raft_consensus.cc:2804] T 8638ee363273426db1393dd81e5c7b72 P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:21:29.346885 21847 raft_consensus.cc:515] T 2794af5b99984efebdee564035841cb0 P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:29.395751 21811 raft_consensus.cc:515] T 8a878999c0024d2abe517484180290fc P 90f3da57796b488c9087a2f523adf045 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:29.397186 21480 raft_consensus.cc:493] T b6f07842d3aa47858df5665874c86f3a P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:21:29.455793 21480 raft_consensus.cc:515] T b6f07842d3aa47858df5665874c86f3a P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:29.456909 21811 leader_election.cc:290] T 8a878999c0024d2abe517484180290fc P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:21:29.449023 21855 leader_election.cc:290] T 02838c86531d4f98bf58d1c22b8baf77 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 15 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:21:29.449710 16291 raft_consensus.cc:2393] T 02838c86531d4f98bf58d1c22b8baf77 P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 15: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
I20260709 13:21:29.450605 21729 leader_election.cc:290] T e5ad4eb281564936a59f41e387a47156 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:21:29.378685 21773 raft_consensus.cc:515] T d2ab0ac8347641e6955e9bce7d90c1a2 P 90f3da57796b488c9087a2f523adf045 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:29.391158 21708 consensus_queue.cc:1059] T 8cc959fb7f704fba9009c0595467d074 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: INVALID_TERM, Last received: 13.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.034s
I20260709 13:21:29.422634 21444 leader_election.cc:290] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:29.421001 21706 consensus_queue.cc:237] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 15.10, Last appended by leader: 10, Current term: 17, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:29.429878 21856 consensus_queue.cc:260] T 5a959dbecc2648eb85d1cc550f5a3245 P 90f3da57796b488c9087a2f523adf045 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:21:29.454519 21863 log.cc:927] Time spent T c0eab25ce4c64e018a041beeda2867db P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 0.691s	user 0.013s	sys 0.002s
I20260709 13:21:29.454289 21854 raft_consensus.cc:493] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:21:29.483117 16824 raft_consensus.cc:1275] T ada5a004ba7c46819051e3ccec71ffdb P cb82cc7bd6db46d980c75237a1402860 [term 16 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
W20260709 13:21:29.478221 16116 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.571s	user 0.021s	sys 0.038s
I20260709 13:21:29.453325 21812 raft_consensus.cc:697] T 8638ee363273426db1393dd81e5c7b72 P 90f3da57796b488c9087a2f523adf045 [term 16 LEADER]: Becoming Leader. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
W20260709 13:21:29.483800 16116 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.577s	user 0.021s	sys 0.038s
W20260709 13:21:29.466156 21704 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.580s	user 0.021s	sys 0.009s
W20260709 13:21:29.471661 16226 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.604s	user 0.010s	sys 0.020s
W20260709 13:21:29.491494 16226 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.624s	user 0.010s	sys 0.020s
I20260709 13:21:29.502105 16145 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46224 (request call id 891) took 1059 ms. Trace:
I20260709 13:21:29.502547 16145 rpcz_store.cc:276] 0709 13:21:28.442555 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:28.444458 (+  1903us) service_pool.cc:224] Handling call
0709 13:21:29.501978 (+1057520us) spinlock_profiling.cc:243] Waited 127 ms on lock 0x7b54003d9fd4. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:21:29.502059 (+    81us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":291447040}
W20260709 13:21:29.490840 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 600493us
W20260709 13:21:29.480535 21825 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.591s	user 0.029s	sys 0.043s
W20260709 13:21:29.657630 21825 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.768s	user 0.029s	sys 0.043s
W20260709 13:21:29.668288 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 131284us
I20260709 13:21:29.673089 16221 raft_consensus.cc:1275] T c0eab25ce4c64e018a041beeda2867db P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 13:21:29.675313 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ad5dbfa99ee4b8d9e28be92b3f36db5" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
W20260709 13:21:29.494313 21704 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.608s	user 0.022s	sys 0.009s
I20260709 13:21:29.676393 16218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746aabf856ab401c9ef5e859371e7b67" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:29.677981 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f5f89186204242bf377f7c42b2e2da" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:21:29.679347 16224 raft_consensus.cc:3060] T c2f5f89186204242bf377f7c42b2e2da P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:21:29.471784 21555 raft_consensus.cc:493] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:21:29.679147 16222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2372981006d4b218215c28964dea623" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:29.501317 21856 raft_consensus.cc:3060] T 5a959dbecc2648eb85d1cc550f5a3245 P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:21:29.688159 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02838c86531d4f98bf58d1c22b8baf77" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:21:29.689002 16217 raft_consensus.cc:3060] T 02838c86531d4f98bf58d1c22b8baf77 P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:21:29.690114 16220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1df93779209748b9860f6d893dd3de53" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:29.695744 16146 raft_consensus.cc:1240] T 18ad7e3af5504571a69d10c00750a01c P 647a343a542d4a139fe106c19095129b [term 13 LEADER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 12. Current term is 13. Ops: []
I20260709 13:21:29.669844 21480 leader_election.cc:290] T b6f07842d3aa47858df5665874c86f3a P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:29.703347 21480 raft_consensus.cc:3055] T 8cc959fb7f704fba9009c0595467d074 P 1ca2725d290e47d68b1a4b609c9816ac [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:21:29.703761 21480 raft_consensus.cc:740] T 8cc959fb7f704fba9009c0595467d074 P 1ca2725d290e47d68b1a4b609c9816ac [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:21:29.704603 21480 consensus_queue.cc:260] T 8cc959fb7f704fba9009c0595467d074 P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:21:29.706449 21434 consensus_meta.cc:287] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860: Time spent flushing consensus metadata: real 0.584s	user 0.010s	sys 0.001s
I20260709 13:21:29.706956 21434 raft_consensus.cc:515] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:29.709487 21434 leader_election.cc:290] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 13 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:21:29.676472 21801 raft_consensus.cc:515] T 8c11074579b046d99e585c2818e18a2c P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:29.717381 21801 leader_election.cc:290] T 8c11074579b046d99e585c2818e18a2c P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 15 election: Requested vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:29.494095 21854 raft_consensus.cc:515] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:29.719127 21736 raft_consensus.cc:493] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:29.699975 21592 leader_election.cc:290] T ee9ed67006144797b1de6ea8c2013a68 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 15 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
W20260709 13:21:29.701097 16115 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:29.719928 21736 raft_consensus.cc:515] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:29.721481 16115 leader_election.cc:304] T 9e65676b21b748728ad39668cfe722ec P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:21:29.722718 16115 leader_election.cc:304] T ecc1e363230043fe9fef390f0b979f20 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:21:29.692870 21555 raft_consensus.cc:515] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:29.681872 21729 raft_consensus.cc:493] T cdbd78acb4cf46298e0de8ba0d130e19 P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:21:29.740152 16114 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.802s	user 0.018s	sys 0.051s
W20260709 13:21:29.751543 16114 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.814s	user 0.018s	sys 0.051s
I20260709 13:21:29.719813 21854 leader_election.cc:290] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:21:29.757614 16224 raft_consensus.cc:2468] T c2f5f89186204242bf377f7c42b2e2da P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 17.
I20260709 13:21:29.758997 16145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434bff3082d141838068f323bdb0d1ed" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:29.760187 16217 raft_consensus.cc:2468] T 02838c86531d4f98bf58d1c22b8baf77 P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 15.
I20260709 13:21:29.764070 16142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "829b4ad8c5a54eababeea1181b960d4b" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:29.765277 16151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee9ed67006144797b1de6ea8c2013a68" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:21:29.790910 16145 raft_consensus.cc:1240] T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b [term 14 FOLLOWER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 13. Current term is 14. Ops: [13.11-13.11]
I20260709 13:21:29.791625 16151 raft_consensus.cc:2393] T ee9ed67006144797b1de6ea8c2013a68 P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 in current term 15: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
W20260709 13:21:29.739804 21751 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.820s	user 0.033s	sys 0.054s
I20260709 13:21:29.728188 21592 raft_consensus.cc:2804] T 9e65676b21b748728ad39668cfe722ec P 647a343a542d4a139fe106c19095129b [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:21:29.728467 21588 raft_consensus.cc:2749] T ecc1e363230043fe9fef390f0b979f20 P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:21:29.726280 21867 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.764s	user 0.021s	sys 0.049s
I20260709 13:21:29.799383 16150 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40830 (request call id 1758) took 1745 ms. Trace:
I20260709 13:21:29.727640 21838 leader_election.cc:290] T 8c11074579b046d99e585c2818e18a2c P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 15 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
W20260709 13:21:29.800668 16817 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.695s	user 0.000s	sys 0.079s
I20260709 13:21:29.801129 16150 rpcz_store.cc:276] 0709 13:21:28.054010 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:28.054170 (+   160us) service_pool.cc:224] Handling call
0709 13:21:28.054548 (+   378us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:21:28.108173 (+ 53625us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:29.799115 (+1690942us) spinlock_profiling.cc:243] Waited 42.3 ms on lock 0x7b540020ba54. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:21:29.799326 (+   211us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":97290624}
I20260709 13:21:29.800766 16151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e65676b21b748728ad39668cfe722ec" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
W20260709 13:21:29.803494 16817 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.699s	user 0.000s	sys 0.079s
I20260709 13:21:29.806463 21555 leader_election.cc:290] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
W20260709 13:21:29.795686 21751 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.876s	user 0.037s	sys 0.054s
I20260709 13:21:29.807428 21751 raft_consensus.cc:697] T 8d4ab870a05347868956aa8b841b2981 P 647a343a542d4a139fe106c19095129b [term 14 LEADER]: Becoming Leader. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:21:29.808732 21751 consensus_queue.cc:237] T 8d4ab870a05347868956aa8b841b2981 P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:29.813805 21579 raft_consensus.cc:3055] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:21:29.814391 21579 raft_consensus.cc:740] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:21:29.815552 21579 consensus_queue.cc:260] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:29.816932 21579 raft_consensus.cc:3060] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:21:29.818123 21728 raft_consensus.cc:3060] T af57996a8be84d398f0f023e8d08ef77 P cb82cc7bd6db46d980c75237a1402860 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:21:29.795758 21854 raft_consensus.cc:493] T 5a96373bb0e14dce8a266b2fc6c58507 P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:21:29.796501 21835 leader_election.cc:290] T 26354361e9b044cea1c2b86c4882fe92 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:29.801695 21592 raft_consensus.cc:493] T 9e65676b21b748728ad39668cfe722ec P 647a343a542d4a139fe106c19095129b [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:21:29.801064 21867 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.839s	user 0.021s	sys 0.049s
I20260709 13:21:29.743703 21729 raft_consensus.cc:515] T cdbd78acb4cf46298e0de8ba0d130e19 P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:29.801440 21742 leader_election.cc:290] T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
W20260709 13:21:29.830137 21749 consensus_meta.cc:287] T 3884f864a96d428785895e5463e5728f P cb82cc7bd6db46d980c75237a1402860: Time spent flushing consensus metadata: real 0.699s	user 0.007s	sys 0.005s
I20260709 13:21:29.798022 21635 consensus_queue.cc:1048] T 8a878999c0024d2abe517484180290fc P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.035s
I20260709 13:21:29.805153 21836 leader_election.cc:290] T 19baa7bc367844c288bca8f81439c413 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:29.751403 21812 consensus_queue.cc:237] T 8638ee363273426db1393dd81e5c7b72 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:29.838526 21592 raft_consensus.cc:3060] T 9e65676b21b748728ad39668cfe722ec P 647a343a542d4a139fe106c19095129b [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:21:29.855309 16791 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.682s	user 0.011s	sys 0.050s
W20260709 13:21:29.855669 16791 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.683s	user 0.011s	sys 0.050s
I20260709 13:21:29.840281 21847 leader_election.cc:290] T 2794af5b99984efebdee564035841cb0 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
W20260709 13:21:29.857427 16117 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.950s	user 0.056s	sys 0.081s
W20260709 13:21:29.868600 16117 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.961s	user 0.056s	sys 0.081s
I20260709 13:21:29.852135 21773 leader_election.cc:290] T d2ab0ac8347641e6955e9bce7d90c1a2 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
W20260709 13:21:29.871541 16190 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.577s	user 0.014s	sys 0.057s
I20260709 13:21:29.839524 21854 raft_consensus.cc:515] T 5a96373bb0e14dce8a266b2fc6c58507 P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:21:29.861778 21876 log.cc:927] Time spent T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 0.179s	user 0.001s	sys 0.000s
W20260709 13:21:29.875195 16190 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.581s	user 0.014s	sys 0.057s
W20260709 13:21:29.883882 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 589792us
I20260709 13:21:29.884608 16824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee9ed67006144797b1de6ea8c2013a68" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
W20260709 13:21:29.880692 16265 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.777s	user 0.019s	sys 0.030s
I20260709 13:21:29.880555 16820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66ae176961b74813a58bc15c9c4c6f2a" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
W20260709 13:21:29.885268 16265 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.782s	user 0.019s	sys 0.030s
I20260709 13:21:29.885562 16820 raft_consensus.cc:3060] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:21:29.885607 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 782084us
I20260709 13:21:29.847944 21749 raft_consensus.cc:515] T 3884f864a96d428785895e5463e5728f P cb82cc7bd6db46d980c75237a1402860 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:29.881222 16143 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40844 (request call id 2660) took 1051 ms. Trace:
I20260709 13:21:29.891479 16190 leader_election.cc:304] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:21:29.861599 21555 raft_consensus.cc:493] T 2794af5b99984efebdee564035841cb0 P cb82cc7bd6db46d980c75237a1402860 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
W20260709 13:21:29.880463 21623 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.798s	user 0.017s	sys 0.032s
I20260709 13:21:29.899771 16150 raft_consensus.cc:1275] T 84f2da98864b4425bf7c0055951fdd4b P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:21:29.890686 16143 rpcz_store.cc:276] 0709 13:21:28.829653 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:28.883178 (+ 53525us) service_pool.cc:224] Handling call
0709 13:21:28.883479 (+   301us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:21:29.081240 (+197761us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:21:29.081284 (+    44us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:21:29.081341 (+    57us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:21:29.081394 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:29.880885 (+799491us) spinlock_profiling.cc:243] Waited 198 ms on lock 0x7b540006dbd4. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:21:29.880954 (+    69us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:21:29.881142 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":454315904}
I20260709 13:21:29.901490 16142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d4ab870a05347868956aa8b841b2981" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:29.902138 16145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e9b2bf7011f4ae688a79d7bb5a447b4" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:29.903028 16147 raft_consensus.cc:1275] T fffab26427e24d398e2695aad40ca134 P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:21:29.914309 16190 leader_election.cc:304] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:21:29.914402 16818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c11074579b046d99e585c2818e18a2c" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:29.915462 16818 raft_consensus.cc:2393] T 8c11074579b046d99e585c2818e18a2c P cb82cc7bd6db46d980c75237a1402860 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 15: Already voted for candidate cb82cc7bd6db46d980c75237a1402860 in this term.
I20260709 13:21:29.920930 16818 raft_consensus.cc:1275] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P cb82cc7bd6db46d980c75237a1402860 [term 15 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:21:29.925138 16823 raft_consensus.cc:1240] T 3884f864a96d428785895e5463e5728f P cb82cc7bd6db46d980c75237a1402860 [term 11 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 10. Current term is 11. Ops: [10.11-10.11]
I20260709 13:21:29.921118 16820 raft_consensus.cc:2468] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 19.
I20260709 13:21:29.924114 16824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74ca8140f35c4ae29e8a664f6bdfc829" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:21:29.926442 16824 raft_consensus.cc:3060] T 74ca8140f35c4ae29e8a664f6bdfc829 P cb82cc7bd6db46d980c75237a1402860 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:21:29.927250 16263 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.539s	user 0.001s	sys 0.019s
W20260709 13:21:29.927623 16263 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.540s	user 0.001s	sys 0.019s
I20260709 13:21:29.930004 16291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3884f864a96d428785895e5463e5728f" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:29.906335 21736 leader_election.cc:290] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
W20260709 13:21:29.907420 21623 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.825s	user 0.017s	sys 0.033s
I20260709 13:21:29.956707 16820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62fe3346cd3c42c4a55e14af9073bca4" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:29.957600 16820 raft_consensus.cc:2468] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 20.
I20260709 13:21:29.971320 16143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806d7690c2b644959152074bb2cab0a7" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:21:30.029088 16143 raft_consensus.cc:3060] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:21:29.997792 16789 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.524s	user 0.006s	sys 0.018s
W20260709 13:21:30.030818 16789 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.557s	user 0.007s	sys 0.018s
W20260709 13:21:30.033460 21859 log.cc:927] Time spent T 84f2da98864b4425bf7c0055951fdd4b P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 1.247s	user 0.002s	sys 0.001s
I20260709 13:21:30.034236 16146 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40844 (request call id 2662) took 1180 ms. Trace:
I20260709 13:21:30.034696 16146 rpcz_store.cc:276] 0709 13:21:28.853453 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:29.070000 (+216547us) service_pool.cc:224] Handling call
0709 13:21:29.103949 (+ 33949us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:21:29.710072 (+606123us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:30.033993 (+323921us) spinlock_profiling.cc:243] Waited 589 ms on lock 0x7b540028d554. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:21:30.034170 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1354675456}
I20260709 13:21:30.036044 16816 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49896 (request call id 1347) took 1165 ms. Trace:
I20260709 13:21:30.036419 21736 raft_consensus.cc:493] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:21:30.036998 21736 raft_consensus.cc:515] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:30.038152 21480 raft_consensus.cc:3060] T 8cc959fb7f704fba9009c0595467d074 P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:21:30.039099 21736 leader_election.cc:290] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:21:29.929097 21773 raft_consensus.cc:493] T 23f82224f4564fea93572e8206bad1fe P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
W20260709 13:21:30.050227 16192 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.749s	user 0.016s	sys 0.015s
W20260709 13:21:30.050680 16192 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.749s	user 0.016s	sys 0.015s
I20260709 13:21:29.930728 21745 leader_election.cc:290] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:29.916653 21555 raft_consensus.cc:515] T 2794af5b99984efebdee564035841cb0 P cb82cc7bd6db46d980c75237a1402860 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:29.906471 21836 consensus_queue.cc:1048] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.036s
W20260709 13:21:29.950369 21874 log.cc:927] Time spent T 26354361e9b044cea1c2b86c4882fe92 P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 0.194s	user 0.000s	sys 0.001s
I20260709 13:21:30.019439 16820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b7d1ab519044eda2d5e04a07242f25" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:30.041186 21801 raft_consensus.cc:2804] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:21:30.021319 16142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f5f89186204242bf377f7c42b2e2da" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:21:30.165151 16142 raft_consensus.cc:2393] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 17: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
I20260709 13:21:30.022140 16147 raft_consensus.cc:1240] T 5a959dbecc2648eb85d1cc550f5a3245 P 647a343a542d4a139fe106c19095129b [term 14 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 13. Current term is 14. Ops: [13.9-13.9]
I20260709 13:21:30.166440 16147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5ad4eb281564936a59f41e387a47156" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 17 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
W20260709 13:21:30.027915 21879 log.cc:927] Time spent T 18ad7e3af5504571a69d10c00750a01c P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 0.173s	user 0.000s	sys 0.000s
W20260709 13:21:29.948742 21313 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.824s	user 0.006s	sys 0.053s
I20260709 13:21:30.183794 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a378d7f796f94615b3b347119a9880fd" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:30.184542 16293 raft_consensus.cc:2468] T a378d7f796f94615b3b347119a9880fd P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 16.
I20260709 13:21:30.036433 16816 rpcz_store.cc:276] 0709 13:21:28.895197 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:28.895695 (+   498us) service_pool.cc:224] Handling call
0709 13:21:30.035855 (+1140160us) spinlock_profiling.cc:243] Waited 134 ms on lock 0x7b5400721854. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:21:30.035982 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":308668800}
I20260709 13:21:30.043277 16818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a378d7f796f94615b3b347119a9880fd" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:30.185608 16818 raft_consensus.cc:2468] T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 17.
I20260709 13:21:30.186597 16818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ab0ac8347641e6955e9bce7d90c1a2" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:30.188033 16818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2794af5b99984efebdee564035841cb0" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:30.188735 16818 raft_consensus.cc:2468] T 2794af5b99984efebdee564035841cb0 P cb82cc7bd6db46d980c75237a1402860 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 12.
I20260709 13:21:30.189700 16818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e4bfb9090c8460eb14fa7e0d818e7f1" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:30.049173 16146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:30.071830 16824 raft_consensus.cc:2468] T 74ca8140f35c4ae29e8a664f6bdfc829 P cb82cc7bd6db46d980c75237a1402860 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 15.
W20260709 13:21:29.930042 21804 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.555s	user 0.027s	sys 0.035s
I20260709 13:21:30.117744 16145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6f07842d3aa47858df5665874c86f3a" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:30.127630 16819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ab0ac8347641e6955e9bce7d90c1a2" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:30.018292 16291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e65676b21b748728ad39668cfe722ec" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:30.138754 16821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26354361e9b044cea1c2b86c4882fe92" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:30.147863 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02838c86531d4f98bf58d1c22b8baf77" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
W20260709 13:21:30.151647 16191 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:21:30.048974 21873 log.cc:927] Time spent T 87ac0d1f7e8a419f9bcbc04aee0e067e P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 0.246s	user 0.000s	sys 0.001s
I20260709 13:21:30.153539 16296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c11074579b046d99e585c2818e18a2c" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:30.155018 16820 raft_consensus.cc:2468] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 13.
I20260709 13:21:30.020891 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7ca61592e848ea9b0fb81a2ce33cdc" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:21:30.159857 16143 raft_consensus.cc:2468] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 16.
W20260709 13:21:30.161520 16115 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:30.162843 16148 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46224 (request call id 890) took 1720 ms. Trace:
I20260709 13:21:30.163650 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434bff3082d141838068f323bdb0d1ed" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:30.047477 21847 raft_consensus.cc:493] T 31969392ccdb43b491911a319e8bc7e1 P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:21:30.169219 16147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a878999c0024d2abe517484180290fc" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:30.163354 21888 raft_consensus.cc:493] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:30.155856 21490 raft_consensus.cc:493] T 84f2da98864b4425bf7c0055951fdd4b P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:30.114073 21736 raft_consensus.cc:493] T 8158bbd17b994d2eb96f560bd0f1b88d P 90f3da57796b488c9087a2f523adf045 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:21:30.242184 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e4bfb9090c8460eb14fa7e0d818e7f1" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 16 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:30.139684 21835 raft_consensus.cc:493] T d16dfad0c3b540e5b4b86492ddf87208 P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:21:30.139909 21867 raft_consensus.cc:493] T e4dc6b0541ca431ebc757beac00521f1 P 90f3da57796b488c9087a2f523adf045 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:21:30.224695 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e4bfb9090c8460eb14fa7e0d818e7f1" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 16 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:30.226523 16299 raft_consensus.cc:3060] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Advancing to term 17
I20260709 13:21:30.248502 16293 raft_consensus.cc:2410] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 for term 16 because replica has last-logged OpId of term: 13 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 10.
I20260709 13:21:30.226985 16296 raft_consensus.cc:2468] T 8c11074579b046d99e585c2818e18a2c P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 13.
I20260709 13:21:30.190362 16818 raft_consensus.cc:2468] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P cb82cc7bd6db46d980c75237a1402860 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 15.
I20260709 13:21:30.249508 16117 leader_election.cc:304] T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860
I20260709 13:21:30.228466 16143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2794af5b99984efebdee564035841cb0" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:30.228716 16146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26354361e9b044cea1c2b86c4882fe92" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:30.229712 16147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c74490c40243a88e62d1760b5cd814" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:30.232740 16150 raft_consensus.cc:2393] T 02838c86531d4f98bf58d1c22b8baf77 P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 15: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
I20260709 13:21:30.250670 16117 leader_election.cc:304] T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860
I20260709 13:21:30.236253 16820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee9ed67006144797b1de6ea8c2013a68" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
W20260709 13:21:30.251294 16226 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.759s	user 0.013s	sys 0.036s
I20260709 13:21:30.155659 21745 raft_consensus.cc:493] T 57a744f5aaf944678a63469cbf514015 P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:21:30.239758 16223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a378d7f796f94615b3b347119a9880fd" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
W20260709 13:21:30.227911 21804 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.876s	user 0.029s	sys 0.039s
I20260709 13:21:30.254653 16223 raft_consensus.cc:2468] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 17.
I20260709 13:21:30.229130 16265 leader_election.cc:304] T 7c2095a5f4ad49408dbc7f69c606120d P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:21:30.238145 16824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746aabf856ab401c9ef5e859371e7b67" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
W20260709 13:21:30.184185 21313 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.076s	user 0.006s	sys 0.053s
I20260709 13:21:30.177580 21623 raft_consensus.cc:493] T ada5a004ba7c46819051e3ccec71ffdb P cb82cc7bd6db46d980c75237a1402860 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:30.245697 16222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee9ed67006144797b1de6ea8c2013a68" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:21:30.266533 16222 raft_consensus.cc:3060] T ee9ed67006144797b1de6ea8c2013a68 P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Advancing to term 15
I20260709 13:21:30.227510 16148 rpcz_store.cc:276] 0709 13:21:28.442029 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:28.442287 (+   258us) service_pool.cc:224] Handling call
0709 13:21:28.442701 (+   414us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:21:28.723024 (+280323us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:21:28.723319 (+   295us) log.cc:844] Serialized 14 byte log entry
0709 13:21:29.008602 (+285283us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:21:29.008661 (+    59us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:21:29.008730 (+    69us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:30.162620 (+1153890us) spinlock_profiling.cc:243] Waited 280 ms on lock 0x7b540028f354. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:21:30.162665 (+    45us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:21:30.162787 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":6291,"prepare.run_cpu_time_us":49,"prepare.run_wall_time_us":51,"raft.queue_time_us":347,"raft.run_cpu_time_us":190,"raft.run_wall_time_us":47821,"spinlock_wait_cycles":644154240,"thread_start_us":283941,"threads_started":1,"wal-append.queue_time_us":759995}
I20260709 13:21:30.268162 16148 raft_consensus.cc:1240] T 8cc959fb7f704fba9009c0595467d074 P 647a343a542d4a139fe106c19095129b [term 14 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 13. Current term is 14. Ops: []
I20260709 13:21:30.194545 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74ca8140f35c4ae29e8a664f6bdfc829" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:21:30.232581 16142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87ac0d1f7e8a419f9bcbc04aee0e067e" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:30.229948 16291 raft_consensus.cc:2468] T 9e65676b21b748728ad39668cfe722ec P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 13.
I20260709 13:21:30.251605 16820 raft_consensus.cc:2393] T ee9ed67006144797b1de6ea8c2013a68 P cb82cc7bd6db46d980c75237a1402860 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 15: Already voted for candidate cb82cc7bd6db46d980c75237a1402860 in this term.
I20260709 13:21:30.252418 16218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8638ee363273426db1393dd81e5c7b72" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
W20260709 13:21:30.246455 16790 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.720s	user 0.028s	sys 0.080s
W20260709 13:21:30.257107 16226 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.765s	user 0.013s	sys 0.037s
I20260709 13:21:30.262660 16143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b7d1ab519044eda2d5e04a07242f25" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:30.270107 16218 raft_consensus.cc:2468] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 16.
I20260709 13:21:30.253055 16147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:30.270591 16143 raft_consensus.cc:2468] T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 13.
I20260709 13:21:30.192802 21801 raft_consensus.cc:697] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [term 19 LEADER]: Becoming Leader. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:21:30.223932 21543 raft_consensus.cc:493] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:30.140085 21773 raft_consensus.cc:515] T 23f82224f4564fea93572e8206bad1fe P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:30.273037 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6f07842d3aa47858df5665874c86f3a" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:30.280691 16220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "829b4ad8c5a54eababeea1181b960d4b" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:30.284303 16117 leader_election.cc:304] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860
I20260709 13:21:30.268038 16223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c11074579b046d99e585c2818e18a2c" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
W20260709 13:21:30.274565 16790 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.748s	user 0.029s	sys 0.080s
I20260709 13:21:30.285637 16223 raft_consensus.cc:2393] T 8c11074579b046d99e585c2818e18a2c P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 in current term 15: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
I20260709 13:21:30.269587 16298 raft_consensus.cc:2368] T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 for earlier term 14. Current term is 15.
I20260709 13:21:30.194974 21097 raft_consensus.cc:493] T 57a744f5aaf944678a63469cbf514015 P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
W20260709 13:21:30.248347 21706 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.535s	user 0.008s	sys 0.038s
I20260709 13:21:30.264154 16292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8638ee363273426db1393dd81e5c7b72" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:30.270980 16147 raft_consensus.cc:2468] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 17.
I20260709 13:21:30.280683 16825 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49892 (request call id 836) took 1179 ms. Trace:
I20260709 13:21:30.233990 21729 leader_election.cc:290] T cdbd78acb4cf46298e0de8ba0d130e19 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:30.272450 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e65676b21b748728ad39668cfe722ec" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:21:30.315297 16293 raft_consensus.cc:3060] T 9e65676b21b748728ad39668cfe722ec P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Advancing to term 15
I20260709 13:21:30.269119 16191 leader_election.cc:304] T af57996a8be84d398f0f023e8d08ef77 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
I20260709 13:21:30.288430 16148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b7d1ab519044eda2d5e04a07242f25" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:21:30.316860 16146 raft_consensus.cc:1240] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 14. Current term is 15. Ops: []
I20260709 13:21:30.178916 21084 consensus_queue.cc:1059] T fffab26427e24d398e2695aad40ca134 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.088s
I20260709 13:21:30.271860 21635 raft_consensus.cc:2749] T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:21:30.261430 21569 raft_consensus.cc:2749] T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:21:30.334235 16294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c11074579b046d99e585c2818e18a2c" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
W20260709 13:21:30.319931 21706 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.601s	user 0.015s	sys 0.039s
W20260709 13:21:30.343362 16149 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.545s	user 0.017s	sys 0.036s
I20260709 13:21:30.291584 16144 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46224 (request call id 888) took 1850 ms. Trace:
W20260709 13:21:30.390316 16149 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.592s	user 0.017s	sys 0.036s
I20260709 13:21:30.338069 16825 rpcz_store.cc:276] 0709 13:21:29.101581 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:29.101719 (+   138us) service_pool.cc:224] Handling call
0709 13:21:30.280459 (+1178740us) spinlock_profiling.cc:243] Waited 163 ms on lock 0x7b540047fc54. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:21:30.280614 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":375630208}
I20260709 13:21:30.343977 16294 raft_consensus.cc:3060] T 8c11074579b046d99e585c2818e18a2c P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Advancing to term 15
I20260709 13:21:30.313547 21891 raft_consensus.cc:2804] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:21:30.352809 21895 raft_consensus.cc:493] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:21:30.344148 21804 raft_consensus.cc:493] T 66f75124215f4ca6bef831e174a792d8 P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:21:30.354638 21773 leader_election.cc:290] T 23f82224f4564fea93572e8206bad1fe P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:30.244772 21888 raft_consensus.cc:515] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:30.355412 21801 consensus_queue.cc:237] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.12, Last appended by leader: 12, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:30.250422 21490 raft_consensus.cc:515] T 84f2da98864b4425bf7c0055951fdd4b P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:30.333899 21749 leader_election.cc:290] T 3884f864a96d428785895e5463e5728f P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 11 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
W20260709 13:21:30.358294 21769 consensus_meta.cc:287] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 1ca2725d290e47d68b1a4b609c9816ac: Time spent flushing consensus metadata: real 1.061s	user 0.006s	sys 0.006s
I20260709 13:21:30.359334 21084 consensus_queue.cc:1243] T fffab26427e24d398e2695aad40ca134 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Peer 90f3da57796b488c9087a2f523adf045 log is divergent from this leader: its last log entry 15.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
W20260709 13:21:30.360483 21880 log.cc:927] Time spent T c9944d5e4b8c4109af9955c4f08c9095 P cb82cc7bd6db46d980c75237a1402860: Append to log took a long time: real 0.342s	user 0.000s	sys 0.000s
W20260709 13:21:30.361238 21886 log.cc:927] Time spent T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860: Append to log took a long time: real 0.111s	user 0.000s	sys 0.000s
I20260709 13:21:30.261641 21867 raft_consensus.cc:515] T e4dc6b0541ca431ebc757beac00521f1 P 90f3da57796b488c9087a2f523adf045 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:30.361470 21755 raft_consensus.cc:3055] T 18ad7e3af5504571a69d10c00750a01c P 1ca2725d290e47d68b1a4b609c9816ac [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:21:30.361914 21894 raft_consensus.cc:493] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:30.410321 21891 raft_consensus.cc:697] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [term 15 LEADER]: Becoming Leader. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:21:30.431377 21894 raft_consensus.cc:515] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:30.432029 21891 consensus_queue.cc:237] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:30.432873 21867 leader_election.cc:290] T e4dc6b0541ca431ebc757beac00521f1 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:21:30.433444 21867 raft_consensus.cc:493] T c0eab25ce4c64e018a041beeda2867db P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:30.433590 21894 leader_election.cc:290] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:30.355284 21843 raft_consensus.cc:2804] T 7c2095a5f4ad49408dbc7f69c606120d P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:21:30.433928 21867 raft_consensus.cc:515] T c0eab25ce4c64e018a041beeda2867db P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:30.434361 21843 raft_consensus.cc:493] T 7c2095a5f4ad49408dbc7f69c606120d P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:30.434909 21843 raft_consensus.cc:3060] T 7c2095a5f4ad49408dbc7f69c606120d P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:21:30.316555 21543 raft_consensus.cc:515] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:21:30.439788 21856 consensus_meta.cc:287] T 5a959dbecc2648eb85d1cc550f5a3245 P 90f3da57796b488c9087a2f523adf045: Time spent flushing consensus metadata: real 0.746s	user 0.011s	sys 0.000s
I20260709 13:21:30.441287 21543 leader_election.cc:290] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:30.366562 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26354361e9b044cea1c2b86c4882fe92" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:30.424203 21480 raft_consensus.cc:3055] T fffab26427e24d398e2695aad40ca134 P 1ca2725d290e47d68b1a4b609c9816ac [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:21:30.443446 21480 raft_consensus.cc:740] T fffab26427e24d398e2695aad40ca134 P 1ca2725d290e47d68b1a4b609c9816ac [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:21:30.443673 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1df93779209748b9860f6d893dd3de53" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:30.444546 16300 raft_consensus.cc:2468] T 1df93779209748b9860f6d893dd3de53 P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 13.
I20260709 13:21:30.444408 21480 consensus_queue.cc:260] T fffab26427e24d398e2695aad40ca134 P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:30.445712 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4dc6b0541ca431ebc757beac00521f1" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:30.445945 21480 raft_consensus.cc:3060] T fffab26427e24d398e2695aad40ca134 P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:21:30.446486 16300 raft_consensus.cc:2468] T e4dc6b0541ca431ebc757beac00521f1 P 90f3da57796b488c9087a2f523adf045 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 14.
I20260709 13:21:30.322091 21751 raft_consensus.cc:2749] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:21:30.280279 21745 raft_consensus.cc:515] T 57a744f5aaf944678a63469cbf514015 P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:30.448266 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ad5dbfa99ee4b8d9e28be92b3f36db5" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:21:30.448923 16300 raft_consensus.cc:3060] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:21:30.449225 21745 leader_election.cc:290] T 57a744f5aaf944678a63469cbf514015 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:30.449862 21745 consensus_queue.cc:1048] T ada5a004ba7c46819051e3ccec71ffdb P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:21:30.271602 21884 raft_consensus.cc:493] T b87147731a68400e8cce9b879daa7c5d P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:21:30.450871 21884 raft_consensus.cc:515] T b87147731a68400e8cce9b879daa7c5d P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:30.369199 16292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdbd78acb4cf46298e0de8ba0d130e19" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:30.454288 16292 raft_consensus.cc:2468] T cdbd78acb4cf46298e0de8ba0d130e19 P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 18.
I20260709 13:21:30.455376 16292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3884f864a96d428785895e5463e5728f" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:21:30.292485 21623 raft_consensus.cc:515] T ada5a004ba7c46819051e3ccec71ffdb P cb82cc7bd6db46d980c75237a1402860 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:30.347502 16147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3884f864a96d428785895e5463e5728f" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:21:30.346048 16299 raft_consensus.cc:2468] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 17.
I20260709 13:21:30.458603 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c2095a5f4ad49408dbc7f69c606120d" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:30.371142 16790 leader_election.cc:400] T 74ca8140f35c4ae29e8a664f6bdfc829 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 14 election: Vote denied by peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943) with higher term. Message: Invalid argument: T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 for earlier term 14. Current term is 15.
I20260709 13:21:30.459484 16790 leader_election.cc:403] T 74ca8140f35c4ae29e8a664f6bdfc829 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 14 election: Cancelling election due to peer responding with higher term
W20260709 13:21:30.460211 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 89234.9us
I20260709 13:21:30.461145 16790 leader_election.cc:304] T 3a7ca61592e848ea9b0fb81a2ce33cdc P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:21:30.374660 21424 raft_consensus.cc:493] T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:30.375525 21836 raft_consensus.cc:493] T e5ad4eb281564936a59f41e387a47156 P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:21:30.323050 21097 raft_consensus.cc:515] T 57a744f5aaf944678a63469cbf514015 P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:30.376307 21708 raft_consensus.cc:3060] T af57996a8be84d398f0f023e8d08ef77 P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:21:30.377151 21854 leader_election.cc:290] T 5a96373bb0e14dce8a266b2fc6c58507 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:21:30.379837 16823 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49892 (request call id 838) took 1277 ms. Trace:
I20260709 13:21:30.239537 21847 raft_consensus.cc:515] T 31969392ccdb43b491911a319e8bc7e1 P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:30.387727 16116 leader_election.cc:304] T 7c2095a5f4ad49408dbc7f69c606120d P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
I20260709 13:21:30.388319 16293 raft_consensus.cc:2468] T 9e65676b21b748728ad39668cfe722ec P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 15.
I20260709 13:21:30.272138 21883 raft_consensus.cc:493] T af57996a8be84d398f0f023e8d08ef77 P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:30.393697 16148 raft_consensus.cc:2468] T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 13.
I20260709 13:21:30.390497 16144 rpcz_store.cc:276] 0709 13:21:28.441252 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:28.443663 (+  2411us) service_pool.cc:224] Handling call
0709 13:21:28.450190 (+  6527us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:21:28.661551 (+211361us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:21:28.661598 (+    47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:21:28.673634 (+ 12036us) log.cc:844] Serialized 26 byte log entry
0709 13:21:29.352367 (+678733us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:21:29.352425 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:30.047994 (+695569us) spinlock_profiling.cc:243] Waited 211 ms on lock 0x7b54003d9ad4. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:21:30.048046 (+    52us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:21:30.291296 (+243250us) raft_consensus.cc:1685] finished
0709 13:21:30.291315 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:21:30.291519 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":485353344,"thread_start_us":678347,"threads_started":1,"wal-append.queue_time_us":1050853}
I20260709 13:21:30.253479 21736 raft_consensus.cc:515] T 8158bbd17b994d2eb96f560bd0f1b88d P 90f3da57796b488c9087a2f523adf045 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:21:30.401412 16791 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:30.376547 21555 leader_election.cc:290] T 2794af5b99984efebdee564035841cb0 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:30.312549 21881 consensus_queue.cc:1048] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.084s
I20260709 13:21:30.408074 16298 raft_consensus.cc:1275] T c9944d5e4b8c4109af9955c4f08c9095 P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
W20260709 13:21:30.408277 16789 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:30.403234 21895 raft_consensus.cc:515] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:30.396847 21569 raft_consensus.cc:493] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:30.422580 16294 raft_consensus.cc:2468] T 8c11074579b046d99e585c2818e18a2c P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 15.
I20260709 13:21:30.395459 21743 raft_consensus.cc:493] T 434bff3082d141838068f323bdb0d1ed P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:30.395805 21635 raft_consensus.cc:493] T 9c61a39fd4f84424b129c1a6f9ba404d P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:30.397073 21882 raft_consensus.cc:493] T 3884f864a96d428785895e5463e5728f P 647a343a542d4a139fe106c19095129b [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:21:30.423321 21769 raft_consensus.cc:515] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:30.407517 21773 raft_consensus.cc:493] T c4f65cdf00b748bb963ee6e81ef43b79 P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:21:30.475936 16822 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.561s	user 0.024s	sys 0.019s
W20260709 13:21:30.476758 16190 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.561s	user 0.024s	sys 0.020s
W20260709 13:21:30.563297 16190 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.648s	user 0.024s	sys 0.020s
I20260709 13:21:30.483780 16292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2794af5b99984efebdee564035841cb0" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:30.485219 16300 raft_consensus.cc:2468] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 90f3da57796b488c9087a2f523adf045 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 14.
I20260709 13:21:30.470892 16115 leader_election.cc:304] T 1df93779209748b9860f6d893dd3de53 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
W20260709 13:21:30.564631 16115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 93805.9us
I20260709 13:21:30.564860 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02838c86531d4f98bf58d1c22b8baf77" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
W20260709 13:21:30.429458 21592 consensus_meta.cc:287] T 9e65676b21b748728ad39668cfe722ec P 647a343a542d4a139fe106c19095129b: Time spent flushing consensus metadata: real 0.568s	user 0.009s	sys 0.001s
I20260709 13:21:30.499550 16823 rpcz_store.cc:276] 0709 13:21:29.102122 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:29.102234 (+   112us) service_pool.cc:224] Handling call
0709 13:21:29.103112 (+   878us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:21:29.925736 (+822624us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:30.379589 (+453853us) spinlock_profiling.cc:243] Waited 822 ms on lock 0x7b54006bf654. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:21:30.379778 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1889695616}
W20260709 13:21:30.502102 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 114461us
I20260709 13:21:30.405295 21804 raft_consensus.cc:515] T 66f75124215f4ca6bef831e174a792d8 P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:30.430081 21755 raft_consensus.cc:740] T 18ad7e3af5504571a69d10c00750a01c P 1ca2725d290e47d68b1a4b609c9816ac [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:21:30.572007 21755 consensus_queue.cc:260] T 18ad7e3af5504571a69d10c00750a01c P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } }
I20260709 13:21:30.573980 21755 raft_consensus.cc:3060] T 18ad7e3af5504571a69d10c00750a01c P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:21:30.521014 16144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2794af5b99984efebdee564035841cb0" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:30.527259 21856 raft_consensus.cc:493] T c9944d5e4b8c4109af9955c4f08c9095 P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:30.575608 16817 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49892 (request call id 839) took 1473 ms. Trace:
I20260709 13:21:30.527930 16296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87147731a68400e8cce9b879daa7c5d" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:30.272594 21835 raft_consensus.cc:515] T d16dfad0c3b540e5b4b86492ddf87208 P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:30.528993 16818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f82224f4564fea93572e8206bad1fe" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:30.529356 16790 leader_election.cc:304] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:21:30.532266 16825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdbd78acb4cf46298e0de8ba0d130e19" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:30.580384 16825 raft_consensus.cc:2410] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 for term 19 because replica has last-logged OpId of term: 18 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 13.
I20260709 13:21:30.498924 21749 raft_consensus.cc:2804] T 3a7ca61592e848ea9b0fb81a2ce33cdc P cb82cc7bd6db46d980c75237a1402860 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:21:30.496979 21424 raft_consensus.cc:515] T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:30.568773 16116 leader_election.cc:304] T 5a959dbecc2648eb85d1cc550f5a3245 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 90f3da57796b488c9087a2f523adf045
I20260709 13:21:30.476441 21580 raft_consensus.cc:2749] T 74ca8140f35c4ae29e8a664f6bdfc829 P cb82cc7bd6db46d980c75237a1402860 [term 15 FOLLOWER]: Leader election lost for term 14. Reason: Vote denied by peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943) with higher term. Message: Invalid argument: T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 for earlier term 14. Current term is 15.
I20260709 13:21:30.576128 16817 rpcz_store.cc:276] 0709 13:21:29.102500 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:29.102621 (+   121us) service_pool.cc:224] Handling call
0709 13:21:29.102919 (+   298us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:21:29.811341 (+708422us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:21:29.811394 (+    53us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:21:29.812752 (+  1358us) log.cc:844] Serialized 26 byte log entry
0709 13:21:30.259698 (+446946us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:21:30.259754 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:30.259844 (+    90us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:21:30.575344 (+315500us) raft_consensus.cc:1685] finished
0709 13:21:30.575371 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:21:30.575541 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1295,"raft.queue_time_us":3184,"raft.run_cpu_time_us":85,"raft.run_wall_time_us":86,"thread_start_us":1145875,"threads_started":2,"wal-append.queue_time_us":651076}
I20260709 13:21:30.497409 21884 leader_election.cc:290] T b87147731a68400e8cce9b879daa7c5d P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:30.556509 16824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e9b2bf7011f4ae688a79d7bb5a447b4" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:30.508826 21588 raft_consensus.cc:2804] T 7c2095a5f4ad49408dbc7f69c606120d P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:21:30.565587 16300 raft_consensus.cc:2393] T 02838c86531d4f98bf58d1c22b8baf77 P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 15: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
I20260709 13:21:30.564186 16292 raft_consensus.cc:2468] T 2794af5b99984efebdee564035841cb0 P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 12.
W20260709 13:21:30.563050 16822 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.648s	user 0.025s	sys 0.020s
W20260709 13:21:30.563639 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 648135us
I20260709 13:21:30.565265 16115 leader_election.cc:304] T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:21:30.499750 21836 raft_consensus.cc:515] T e5ad4eb281564936a59f41e387a47156 P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:30.592581 16294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f5f89186204242bf377f7c42b2e2da" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:21:30.600651 16294 raft_consensus.cc:2393] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 17: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
I20260709 13:21:30.598683 16190 leader_election.cc:304] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:21:30.599959 16225 raft_consensus.cc:2393] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 14: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
I20260709 13:21:30.594476 16116 leader_election.cc:304] T 02838c86531d4f98bf58d1c22b8baf77 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
I20260709 13:21:30.575099 21245 raft_consensus.cc:2804] T 1df93779209748b9860f6d893dd3de53 P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:21:30.616392 16116 leader_election.cc:304] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
W20260709 13:21:30.623553 16192 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.567s	user 0.007s	sys 0.035s
W20260709 13:21:30.633458 16192 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.577s	user 0.008s	sys 0.035s
I20260709 13:21:30.595371 21543 raft_consensus.cc:2804] T 5a959dbecc2648eb85d1cc550f5a3245 P 647a343a542d4a139fe106c19095129b [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:21:30.525272 21883 raft_consensus.cc:515] T af57996a8be84d398f0f023e8d08ef77 P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:30.614459 21890 raft_consensus.cc:2804] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:21:30.556593 21569 raft_consensus.cc:515] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:30.557431 21635 raft_consensus.cc:515] T 9c61a39fd4f84424b129c1a6f9ba404d P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:30.623483 21084 raft_consensus.cc:2804] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Leader pre-election won for term 18
W20260709 13:21:30.650897 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 421820us
I20260709 13:21:30.600558 21749 raft_consensus.cc:697] T 3a7ca61592e848ea9b0fb81a2ce33cdc P cb82cc7bd6db46d980c75237a1402860 [term 17 LEADER]: Becoming Leader. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:21:30.584384 21592 raft_consensus.cc:515] T 9e65676b21b748728ad39668cfe722ec P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:30.681705 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee9ed67006144797b1de6ea8c2013a68" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:21:30.603423 21623 leader_election.cc:290] T ada5a004ba7c46819051e3ccec71ffdb P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:30.562995 21743 raft_consensus.cc:515] T 434bff3082d141838068f323bdb0d1ed P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:30.655786 16291 raft_consensus.cc:1275] T c0eab25ce4c64e018a041beeda2867db P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 13:21:30.576066 21773 raft_consensus.cc:515] T c4f65cdf00b748bb963ee6e81ef43b79 P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:30.604140 21588 raft_consensus.cc:493] T 7c2095a5f4ad49408dbc7f69c606120d P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:21:30.642253 16048 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:58400 (request call id 192) took 16123 ms (client timeout 15000 ms). Trace:
I20260709 13:21:30.658638 16226 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57604 (request call id 2949) took 1938 ms. Trace:
I20260709 13:21:30.562534 21882 raft_consensus.cc:515] T 3884f864a96d428785895e5463e5728f P 647a343a542d4a139fe106c19095129b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:30.633448 21245 raft_consensus.cc:493] T 1df93779209748b9860f6d893dd3de53 P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Starting leader election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:21:30.630846 21751 raft_consensus.cc:2804] T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:21:30.598657 21856 raft_consensus.cc:515] T c9944d5e4b8c4109af9955c4f08c9095 P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:30.600418 21742 consensus_queue.cc:1048] T c0eab25ce4c64e018a041beeda2867db P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.055s
W20260709 13:21:30.641244 16192 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:30.683929 16217 raft_consensus.cc:1275] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 13:21:30.689267 16220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4dc6b0541ca431ebc757beac00521f1" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:30.628126 21894 consensus_queue.cc:1059] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.038s
I20260709 13:21:30.701651 16218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62fe3346cd3c42c4a55e14af9073bca4" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
W20260709 13:21:30.701473 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 432458us
I20260709 13:21:30.661633 21084 raft_consensus.cc:493] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:30.663195 21704 raft_consensus.cc:2804] T 02838c86531d4f98bf58d1c22b8baf77 P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Leader election won for term 15
W20260709 13:21:30.711424 16266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 304529us
I20260709 13:21:30.680088 21884 raft_consensus.cc:2804] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:21:30.663471 21890 raft_consensus.cc:697] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 [term 13 LEADER]: Becoming Leader. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:21:30.665341 21867 leader_election.cc:290] T c0eab25ce4c64e018a041beeda2867db P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:21:30.725195 16151 raft_consensus.cc:2393] T 9e65676b21b748728ad39668cfe722ec P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 in current term 15: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
I20260709 13:21:30.657318 21838 consensus_queue.cc:1059] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: INVALID_TERM, Last received: 14.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.046s
W20260709 13:21:30.766340 16117 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 482108us
I20260709 13:21:30.892087 16117 leader_election.cc:304] T 26354361e9b044cea1c2b86c4882fe92 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860
W20260709 13:21:30.777860 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 254403us
I20260709 13:21:30.778102 16266 leader_election.cc:304] T 2794af5b99984efebdee564035841cb0 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:21:30.768445 16298 raft_consensus.cc:1275] T ecc1e363230043fe9fef390f0b979f20 P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 13:21:30.700589 16226 rpcz_store.cc:276] 0709 13:21:28.720408 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:28.786537 (+ 66129us) service_pool.cc:224] Handling call
0709 13:21:28.803611 (+ 17074us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:21:28.803879 (+   268us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:21:29.491996 (+688117us) log.cc:844] Serialized 14 byte log entry
0709 13:21:30.658330 (+1166334us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:21:30.658378 (+    48us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:21:30.658425 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:30.658484 (+    59us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:21:30.658586 (+   102us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7889,"prepare.queue_time_us":63589,"prepare.run_cpu_time_us":41,"prepare.run_wall_time_us":43,"raft.queue_time_us":101551,"raft.run_cpu_time_us":86,"raft.run_wall_time_us":86,"thread_start_us":1782244,"threads_started":3,"wal-append.queue_time_us":1040755}
I20260709 13:21:30.771750 16219 raft_consensus.cc:1275] T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:21:30.772081 16191 leader_election.cc:304] T 8a878999c0024d2abe517484180290fc P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:21:30.770896 16218 raft_consensus.cc:2468] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 19.
W20260709 13:21:30.775658 16222 consensus_meta.cc:287] T ee9ed67006144797b1de6ea8c2013a68 P 1ca2725d290e47d68b1a4b609c9816ac: Time spent flushing consensus metadata: real 0.502s	user 0.009s	sys 0.001s
I20260709 13:21:30.897569 16222 raft_consensus.cc:2468] T ee9ed67006144797b1de6ea8c2013a68 P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 15.
I20260709 13:21:30.894702 16117 leader_election.cc:304] T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860
I20260709 13:21:30.690718 21854 raft_consensus.cc:493] T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:21:30.783824 16265 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:30.708132 21751 raft_consensus.cc:493] T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b [term 14 FOLLOWER]: Starting leader election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:21:30.707726 21888 leader_election.cc:290] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:30.775725 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f2da98864b4425bf7c0055951fdd4b" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
W20260709 13:21:30.899921 16265 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.14.177.130:35943 timed out after 1.560s (SENT) [suppressed 12 similar messages]
I20260709 13:21:30.707906 21245 raft_consensus.cc:3060] T 1df93779209748b9860f6d893dd3de53 P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:21:30.777158 16294 raft_consensus.cc:1275] T c4f65cdf00b748bb963ee6e81ef43b79 P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 13:21:30.703212 21588 raft_consensus.cc:3060] T 7c2095a5f4ad49408dbc7f69c606120d P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:21:30.667833 21903 consensus_queue.cc:1059] T 84f2da98864b4425bf7c0055951fdd4b P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.230s
W20260709 13:21:30.941663 16789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 413462us
I20260709 13:21:30.778756 16147 raft_consensus.cc:3060] T 3884f864a96d428785895e5463e5728f P 647a343a542d4a139fe106c19095129b [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:21:30.952355 16789 leader_election.cc:304] T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 
W20260709 13:21:30.903726 16266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 125694us
I20260709 13:21:30.955551 16266 leader_election.cc:304] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:21:30.775484 21490 leader_election.cc:290] T 84f2da98864b4425bf7c0055951fdd4b P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:30.967415 16266 leader_election.cc:400] T 9e65676b21b748728ad39668cfe722ec P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 14 pre-election: Vote denied by peer cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923) with higher term. Message: Invalid argument: T 9e65676b21b748728ad39668cfe722ec P cb82cc7bd6db46d980c75237a1402860 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 for earlier term 14. Current term is 15.
I20260709 13:21:30.968158 16789 leader_election.cc:304] T 434bff3082d141838068f323bdb0d1ed P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:21:30.968171 16266 leader_election.cc:403] T 9e65676b21b748728ad39668cfe722ec P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 14 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:21:30.972723 16291 raft_consensus.cc:1275] T 18ad7e3af5504571a69d10c00750a01c P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 13:21:30.977944 16789 leader_election.cc:400] T 74ca8140f35c4ae29e8a664f6bdfc829 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 14 election: Vote denied by peer 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389) with higher term. Message: Invalid argument: T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 for earlier term 14. Current term is 15.
I20260709 13:21:30.931490 21444 raft_consensus.cc:493] T 5a959dbecc2648eb85d1cc550f5a3245 P 90f3da57796b488c9087a2f523adf045 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:21:30.982934 16115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 417684us
I20260709 13:21:30.709336 21893 consensus_queue.cc:1048] T 434bff3082d141838068f323bdb0d1ed P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.266s
W20260709 13:21:30.975756 21898 log.cc:927] Time spent T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860: Append to log took a long time: real 0.420s	user 0.001s	sys 0.000s
I20260709 13:21:30.946904 21555 raft_consensus.cc:3055] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:21:30.966568 21751 raft_consensus.cc:3060] T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:21:31.020115 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cc959fb7f704fba9009c0595467d074" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:21:31.021270 16222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c2095a5f4ad49408dbc7f69c606120d" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:31.021456 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ab0ac8347641e6955e9bce7d90c1a2" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:31.021862 16222 raft_consensus.cc:2468] T 7c2095a5f4ad49408dbc7f69c606120d P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 11.
I20260709 13:21:31.022668 16222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02838c86531d4f98bf58d1c22b8baf77" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:31.023324 16222 raft_consensus.cc:2393] T 02838c86531d4f98bf58d1c22b8baf77 P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 15: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
I20260709 13:21:30.918934 21543 raft_consensus.cc:697] T 5a959dbecc2648eb85d1cc550f5a3245 P 647a343a542d4a139fe106c19095129b [term 14 LEADER]: Becoming Leader. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:21:31.035672 16220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f5f89186204242bf377f7c42b2e2da" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:31.036619 16218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806d7690c2b644959152074bb2cab0a7" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:31.001958 16115 leader_election.cc:304] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac
W20260709 13:21:31.012221 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 294872us
I20260709 13:21:30.762730 21804 leader_election.cc:290] T 66f75124215f4ca6bef831e174a792d8 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:30.966437 21891 raft_consensus.cc:493] T 08c74490c40243a88e62d1760b5cd814 P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:21:30.851392 21812 consensus_queue.cc:1048] T fffab26427e24d398e2695aad40ca134 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.156s
I20260709 13:21:31.024839 16222 raft_consensus.cc:1275] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:21:30.984051 16789 leader_election.cc:304] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:21:31.031656 16147 raft_consensus.cc:2410] T 3884f864a96d428785895e5463e5728f P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 for term 11 because replica has last-logged OpId of term: 10 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 10.
I20260709 13:21:30.972539 21847 leader_election.cc:290] T 31969392ccdb43b491911a319e8bc7e1 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:30.788446 21084 raft_consensus.cc:3060] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:21:30.976440 21736 leader_election.cc:290] T 8158bbd17b994d2eb96f560bd0f1b88d P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:21:31.046626 16220 raft_consensus.cc:2393] T c2f5f89186204242bf377f7c42b2e2da P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 17: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
I20260709 13:21:31.047233 16218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c61a39fd4f84424b129c1a6f9ba404d" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 17 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:30.788942 21843 raft_consensus.cc:515] T 7c2095a5f4ad49408dbc7f69c606120d P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:21:31.015739 21901 log.cc:927] Time spent T 8d4ab870a05347868956aa8b841b2981 P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 0.438s	user 0.001s	sys 0.001s
I20260709 13:21:31.019572 21555 raft_consensus.cc:740] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:21:31.053524 16224 raft_consensus.cc:2468] T 8cc959fb7f704fba9009c0595467d074 P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 14.
I20260709 13:21:30.892453 21908 consensus_queue.cc:1048] T 84f2da98864b4425bf7c0055951fdd4b P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.192s
I20260709 13:21:30.939564 21854 raft_consensus.cc:515] T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:30.996594 21579 raft_consensus.cc:2804] T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860 [term 17 FOLLOWER]: Leader pre-election won for term 18
W20260709 13:21:31.055085 16789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 71112.9us
I20260709 13:21:31.055583 16219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66ae176961b74813a58bc15c9c4c6f2a" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:31.055047 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a378d7f796f94615b3b347119a9880fd" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:31.056087 16225 raft_consensus.cc:1240] T 5a959dbecc2648eb85d1cc550f5a3245 P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 13. Current term is 14. Ops: []
I20260709 13:21:31.055546 16222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d4ab870a05347868956aa8b841b2981" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:30.996838 21897 raft_consensus.cc:2749] T 434bff3082d141838068f323bdb0d1ed P cb82cc7bd6db46d980c75237a1402860 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:21:30.999197 21903 consensus_queue.cc:1243] T 84f2da98864b4425bf7c0055951fdd4b P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Peer 90f3da57796b488c9087a2f523adf045 log is divergent from this leader: its last log entry 12.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 13:21:31.057024 16149 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46224 (request call id 904) took 1291 ms. Trace:
I20260709 13:21:31.015407 21769 leader_election.cc:290] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 14 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:31.018808 21895 leader_election.cc:290] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:21:31.089181 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:31.062628 21891 raft_consensus.cc:515] T 08c74490c40243a88e62d1760b5cd814 P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:31.099901 16222 raft_consensus.cc:2468] T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 14.
I20260709 13:21:31.008483 21444 raft_consensus.cc:515] T 5a959dbecc2648eb85d1cc550f5a3245 P 90f3da57796b488c9087a2f523adf045 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:31.062057 21804 raft_consensus.cc:493] T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:31.062381 21424 leader_election.cc:290] T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:21:30.979027 21749 consensus_queue.cc:237] T 3a7ca61592e848ea9b0fb81a2ce33cdc P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:21:31.224962 21728 consensus_meta.cc:287] T af57996a8be84d398f0f023e8d08ef77 P cb82cc7bd6db46d980c75237a1402860: Time spent flushing consensus metadata: real 1.406s	user 0.013s	sys 0.001s
I20260709 13:21:31.160511 21930 raft_consensus.cc:493] T c0eab25ce4c64e018a041beeda2867db P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:21:30.696305 16048 rpcz_store.cc:269] 0709 13:21:14.518889 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:14.519200 (+   311us) service_pool.cc:224] Handling call
0709 13:21:30.641967 (+16122767us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:21:22.398380 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:21:22.407577 (+  9197us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:21:22.407613 (+    36us) write_op.cc:435] Acquired schema lock
0709 13:21:22.407632 (+    19us) tablet.cc:662] Decoding operations
0709 13:21:22.581593 (+173961us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:21:22.584358 (+  2765us) write_op.cc:641] Partition lock acquired for write op
0709 13:21:22.593760 (+  9402us) tablet.cc:685] Acquiring locks for 18 operations
0709 13:21:22.692928 (+ 99168us) tablet.cc:701] Row locks acquired
0709 13:21:22.692951 (+    23us) write_op.cc:260] PREPARE: finished
0709 13:21:22.693164 (+   213us) write_op.cc:270] Start()
0709 13:21:22.693303 (+   139us) write_op.cc:276] Timestamp: P: 1783603282693088 usec, L: 0
0709 13:21:22.693334 (+    31us) op_driver.cc:348] REPLICATION: starting
0709 13:21:22.694214 (+   880us) log.cc:844] Serialized 7162 byte log entry
0709 13:21:23.267230 (+573016us) op_driver.cc:464] REPLICATION: finished
0709 13:21:23.595906 (+328676us) write_op.cc:301] APPLY: starting
0709 13:21:23.600930 (+  5024us) tablet.cc:1367] starting BulkCheckPresence
0709 13:21:24.482866 (+881936us) tablet.cc:1370] finished BulkCheckPresence
0709 13:21:24.495320 (+ 12454us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:21:25.358964 (+863644us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:21:25.359419 (+   455us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=36,key_file_lookups=36,delta_file_lookups=0,mrs_lookups=0
0709 13:21:25.359477 (+    58us) write_op.cc:312] APPLY: finished
0709 13:21:25.360008 (+   531us) log.cc:844] Serialized 164 byte log entry
0709 13:21:25.939991 (+579983us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:21:25.940220 (+   229us) write_op.cc:454] Released schema lock
0709 13:21:25.940309 (+    89us) write_op.cc:341] FINISH: Updating metrics
0709 13:21:25.940834 (+   525us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:21:25.940861 (+    27us) write_op.cc:454] Released schema lock
0709 13:21:26.351502 (+410641us) spinlock_profiling.cc:243] Waited 17.5 ms on lock 0x7b4400022b4c. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69ad8fadb1 00007f69ad998c85 00007f69ad9996a2 00007f69b070b1db 00007f69b0706aaf 00007f69b06fa86e 00007f69b06f9bb6 00007f69b06f97d8 00007f69b0703221 00007f69b07031d9 00007f69b0703169
Metrics: {"mutex_wait_us":41174,"reported_tablets":35,"child_traces":[["op",{"apply.queue_time_us":308528,"apply.run_cpu_time_us":113997,"apply.run_wall_time_us":2364596,"cfile_cache_hit":74,"cfile_cache_hit_bytes":79295,"num_ops":18,"prepare.queue_time_us":419418,"prepare.run_cpu_time_us":18967,"prepare.run_wall_time_us":568541,"raft.queue_time_us":57902,"raft.run_cpu_time_us":52665,"raft.run_wall_time_us":109348,"replication_time_us":573728,"spinlock_wait_cycles":40150784,"thread_start_us":1070908,"threads_started":5,"wal-append.queue_time_us":910831,"wal-append.run_cpu_time_us":17441,"wal-append.run_wall_time_us":2742818}]]}
I20260709 13:21:31.593947 21888 raft_consensus.cc:493] T af57996a8be84d398f0f023e8d08ef77 P cb82cc7bd6db46d980c75237a1402860 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:31.434834 21751 raft_consensus.cc:515] T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:31.585119 21084 raft_consensus.cc:515] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:31.594544 21888 raft_consensus.cc:515] T af57996a8be84d398f0f023e8d08ef77 P cb82cc7bd6db46d980c75237a1402860 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:31.595155 16226 raft_consensus.cc:2368] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 for earlier term 17. Current term is 18.
I20260709 13:21:31.593026 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f5f89186204242bf377f7c42b2e2da" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:21:31.534838 15046 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:21:31.122833 16294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6f07842d3aa47858df5665874c86f3a" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
W20260709 13:21:31.159897 16151 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46224 (request call id 907) took 1364 ms (client timeout 1754 ms). Trace:
W20260709 13:21:31.596452 16189 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.538s	user 0.012s	sys 0.003s
W20260709 13:21:31.589888 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 817834us
W20260709 13:21:31.431573 16263 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.613s	user 0.004s	sys 0.006s
I20260709 13:21:31.596300 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c11074579b046d99e585c2818e18a2c" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:21:31.118325 16299 raft_consensus.cc:2393] T 7c2095a5f4ad49408dbc7f69c606120d P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 12: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
I20260709 13:21:31.230264 16145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a9de347c711477e87c582d003f36925" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 19 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
W20260709 13:21:31.437347 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 537424us
W20260709 13:21:31.431926 16115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 391829us
I20260709 13:21:31.270437 16144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2372981006d4b218215c28964dea623" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
W20260709 13:21:31.159914 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 53501.5us
I20260709 13:21:31.074191 21847 raft_consensus.cc:493] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:31.293352 16148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5d59407eb545b895d8d922e0dd69fd" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:31.625715 16298 raft_consensus.cc:2393] T 8c11074579b046d99e585c2818e18a2c P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 15: Already voted for candidate cb82cc7bd6db46d980c75237a1402860 in this term.
W20260709 13:21:31.690518 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 89322.9us
I20260709 13:21:31.620754 16294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8638ee363273426db1393dd81e5c7b72" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
W20260709 13:21:31.601552 16263 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.783s	user 0.004s	sys 0.011s
W20260709 13:21:31.430727 16117 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.532s	user 0.000s	sys 0.010s
W20260709 13:21:31.648545 16115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 44591.6us
W20260709 13:21:31.792454 16117 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.893s	user 0.002s	sys 0.015s
W20260709 13:21:31.592664 16293 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.612s	user 0.006s	sys 0.010s
W20260709 13:21:31.793254 16117 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 898588us
W20260709 13:21:31.793521 16293 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.813s	user 0.006s	sys 0.020s
W20260709 13:21:31.602692 16789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 506461us
W20260709 13:21:31.771574 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 170207us
W20260709 13:21:31.600244 16189 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.542s	user 0.012s	sys 0.003s
I20260709 13:21:31.598616 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5ad4eb281564936a59f41e387a47156" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:31.597426 16149 rpcz_store.cc:276] 0709 13:21:29.765227 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:29.766791 (+  1564us) service_pool.cc:224] Handling call
0709 13:21:29.798228 (+ 31437us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:21:29.798453 (+   225us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:21:30.390750 (+592297us) log.cc:844] Serialized 14 byte log entry
0709 13:21:31.056690 (+665940us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:21:31.056746 (+    56us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:21:31.056800 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:31.056865 (+    65us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:21:31.056972 (+   107us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":40307,"prepare.queue_time_us":708274,"prepare.run_cpu_time_us":45,"prepare.run_wall_time_us":46,"raft.queue_time_us":346022,"raft.run_cpu_time_us":124,"raft.run_wall_time_us":17746,"thread_start_us":1256329,"threads_started":3,"wal-append.queue_time_us":785635}
I20260709 13:21:31.737218 16296 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48674 (request call id 3044) took 1383 ms. Trace:
I20260709 13:21:31.799248 16789 leader_election.cc:304] T 9e65676b21b748728ad39668cfe722ec P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:21:31.746618 16292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ad5dbfa99ee4b8d9e28be92b3f36db5" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:21:31.749666 16219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806d7690c2b644959152074bb2cab0a7" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:21:31.603235 16145 raft_consensus.cc:2410] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 for term 19 because replica has last-logged OpId of term: 17 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 12.
I20260709 13:21:31.598290 16224 raft_consensus.cc:2393] T c2f5f89186204242bf377f7c42b2e2da P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 17: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
I20260709 13:21:31.148628 21854 leader_election.cc:290] T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:31.149518 21909 raft_consensus.cc:493] T fffab26427e24d398e2695aad40ca134 P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:31.797329 21769 consensus_queue.cc:1059] T 84f2da98864b4425bf7c0055951fdd4b P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.009s
W20260709 13:21:31.336186 21708 consensus_meta.cc:287] T af57996a8be84d398f0f023e8d08ef77 P 1ca2725d290e47d68b1a4b609c9816ac: Time spent flushing consensus metadata: real 0.829s	user 0.005s	sys 0.006s
I20260709 13:21:31.799803 16296 rpcz_store.cc:276] 0709 13:21:30.353623 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:30.577902 (+224279us) service_pool.cc:224] Handling call
0709 13:21:30.578242 (+   340us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:21:30.589031 (+ 10789us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:21:30.589071 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:21:30.589657 (+   586us) log.cc:844] Serialized 26 byte log entry
0709 13:21:30.851097 (+261440us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:21:30.854353 (+  3256us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:30.854473 (+   120us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:21:31.736919 (+882446us) raft_consensus.cc:1685] finished
0709 13:21:31.736950 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:21:31.737137 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":290,"raft.run_cpu_time_us":143,"raft.run_wall_time_us":14633,"thread_start_us":205972,"threads_started":1,"wal-append.queue_time_us":533947}
I20260709 13:21:31.808372 16789 leader_election.cc:304] T 8c11074579b046d99e585c2818e18a2c P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:21:31.093550 21736 raft_consensus.cc:2804] T 2794af5b99984efebdee564035841cb0 P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:21:31.124889 21812 raft_consensus.cc:2749] T 9e65676b21b748728ad39668cfe722ec P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Leader pre-election lost for term 14. Reason: Vote denied by peer cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923) with higher term. Message: Invalid argument: T 9e65676b21b748728ad39668cfe722ec P cb82cc7bd6db46d980c75237a1402860 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 for earlier term 14. Current term is 15.
W20260709 13:21:31.599296 16151 rpcz_store.cc:269] 0709 13:21:29.795804 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:29.800492 (+  4688us) service_pool.cc:224] Handling call
0709 13:21:31.159655 (+1359163us) spinlock_profiling.cc:243] Waited 921 ms on lock 0x7b54003f0f54. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:21:31.159833 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2117892480}
I20260709 13:21:31.094704 21579 raft_consensus.cc:493] T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:31.803434 16292 raft_consensus.cc:2393] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 90f3da57796b488c9087a2f523adf045 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 14: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
W20260709 13:21:31.165750 21480 consensus_meta.cc:287] T fffab26427e24d398e2695aad40ca134 P 1ca2725d290e47d68b1a4b609c9816ac: Time spent flushing consensus metadata: real 0.719s	user 0.011s	sys 0.000s
W20260709 13:21:31.136346 21902 log.cc:927] Time spent T 806d7690c2b644959152074bb2cab0a7 P cb82cc7bd6db46d980c75237a1402860: Append to log took a long time: real 0.375s	user 0.001s	sys 0.000s
I20260709 13:21:31.088474 21890 consensus_queue.cc:237] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:21:31.127967 21912 log.cc:927] Time spent T 26354361e9b044cea1c2b86c4882fe92 P cb82cc7bd6db46d980c75237a1402860: Append to log took a long time: real 0.109s	user 0.000s	sys 0.001s
I20260709 13:21:31.094933 21835 leader_election.cc:290] T d16dfad0c3b540e5b4b86492ddf87208 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:21:31.136754 21635 leader_election.cc:290] T 9c61a39fd4f84424b129c1a6f9ba404d P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:31.136437 21913 raft_consensus.cc:493] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:31.112139 21836 leader_election.cc:290] T e5ad4eb281564936a59f41e387a47156 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:31.123618 21904 raft_consensus.cc:493] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
W20260709 13:21:31.854661 16115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 61378.6us
I20260709 13:21:31.139566 21592 leader_election.cc:290] T 9e65676b21b748728ad39668cfe722ec P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 15 election: Requested vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:31.595978 21751 leader_election.cc:290] T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 15 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:31.143663 21743 leader_election.cc:290] T 434bff3082d141838068f323bdb0d1ed P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:31.151767 21623 raft_consensus.cc:2804] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:21:31.812886 16293 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48674 (request call id 3048) took 1458 ms. Trace:
I20260709 13:21:31.862847 21951 consensus_queue.cc:1048] T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:21:31.867818 21743 consensus_queue.cc:1048] T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:21:31.869505 21635 consensus_queue.cc:1048] T 18ad7e3af5504571a69d10c00750a01c P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:21:31.844421 16294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a378d7f796f94615b3b347119a9880fd" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:21:31.872102 16294 raft_consensus.cc:3060] T a378d7f796f94615b3b347119a9880fd P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Advancing to term 18
I20260709 13:21:31.875314 21623 raft_consensus.cc:493] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:21:31.875882 21623 raft_consensus.cc:3060] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:21:31.684073 21804 raft_consensus.cc:515] T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:31.847745 21920 raft_consensus.cc:2804] T 8c11074579b046d99e585c2818e18a2c P cb82cc7bd6db46d980c75237a1402860 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:21:31.847678 21580 raft_consensus.cc:493] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:31.667409 21930 raft_consensus.cc:515] T c0eab25ce4c64e018a041beeda2867db P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:21:31.450873 21704 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.510s	user 0.001s	sys 0.009s
I20260709 13:21:31.821892 16191 leader_election.cc:304] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:21:31.806528 21909 raft_consensus.cc:515] T fffab26427e24d398e2695aad40ca134 P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:21:31.845882 21755 consensus_meta.cc:287] T 18ad7e3af5504571a69d10c00750a01c P 1ca2725d290e47d68b1a4b609c9816ac: Time spent flushing consensus metadata: real 1.256s	user 0.009s	sys 0.003s
I20260709 13:21:31.149700 21555 consensus_queue.cc:260] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 14.13, Last appended by leader: 13, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:31.811834 21736 raft_consensus.cc:493] T 2794af5b99984efebdee564035841cb0 P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:21:31.830680 21854 raft_consensus.cc:493] T 18ad7e3af5504571a69d10c00750a01c P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:31.811976 21708 raft_consensus.cc:2749] T af57996a8be84d398f0f023e8d08ef77 P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:21:31.144519 21931 raft_consensus.cc:2804] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:21:31.599740 21906 raft_consensus.cc:493] T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:31.728084 21084 leader_election.cc:290] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 18 election: Requested vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:31.731339 21908 raft_consensus.cc:2804] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:21:31.144049 21725 raft_consensus.cc:2749] T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:21:31.158141 21934 raft_consensus.cc:3055] T 84f2da98864b4425bf7c0055951fdd4b P 1ca2725d290e47d68b1a4b609c9816ac [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:21:31.156567 21910 raft_consensus.cc:2749] T 8a878999c0024d2abe517484180290fc P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:21:31.140201 21882 leader_election.cc:290] T 3884f864a96d428785895e5463e5728f P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:31.597059 21937 raft_consensus.cc:493] T ecc1e363230043fe9fef390f0b979f20 P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:31.810873 16789 leader_election.cc:304] T 2794af5b99984efebdee564035841cb0 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 647a343a542d4a139fe106c19095129b
I20260709 13:21:32.007814 21892 raft_consensus.cc:2749] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:21:31.153522 21843 leader_election.cc:290] T 7c2095a5f4ad49408dbc7f69c606120d P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 12 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:21:31.852643 16222 raft_consensus.cc:1275] T fffab26427e24d398e2695aad40ca134 P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:21:31.356742 21884 raft_consensus.cc:697] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b [term 17 LEADER]: Becoming Leader. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:21:31.807186 21847 raft_consensus.cc:515] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:21:31.854773 16117 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 60869.6us
W20260709 13:21:31.856662 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 252293us
I20260709 13:21:31.812099 21897 raft_consensus.cc:2804] T 9e65676b21b748728ad39668cfe722ec P cb82cc7bd6db46d980c75237a1402860 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:21:31.859380 16292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f2da98864b4425bf7c0055951fdd4b" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:31.809247 21838 consensus_queue.cc:1048] T c9944d5e4b8c4109af9955c4f08c9095 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.048s
I20260709 13:21:32.061921 21897 raft_consensus.cc:697] T 9e65676b21b748728ad39668cfe722ec P cb82cc7bd6db46d980c75237a1402860 [term 15 LEADER]: Becoming Leader. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
W20260709 13:21:32.062551 21939 log.cc:927] Time spent T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
I20260709 13:21:31.357937 21894 consensus_queue.cc:1243] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Peer 90f3da57796b488c9087a2f523adf045 log is divergent from this leader: its last log entry 14.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 13:21:31.850353 21579 raft_consensus.cc:3060] T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:21:32.063732 16221 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57604 (request call id 2974) took 1402 ms. Trace:
I20260709 13:21:32.063123 21897 consensus_queue.cc:237] T 9e65676b21b748728ad39668cfe722ec P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:32.064138 16221 rpcz_store.cc:276] 0709 13:21:30.661762 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:30.661865 (+   103us) service_pool.cc:224] Handling call
0709 13:21:30.662154 (+   289us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:21:30.662404 (+   250us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:21:30.662442 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:21:30.684971 (+ 22529us) log.cc:844] Serialized 26 byte log entry
0709 13:21:31.097579 (+412608us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:21:31.097629 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:31.097718 (+    89us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:21:32.063532 (+965814us) raft_consensus.cc:1685] finished
0709 13:21:32.063552 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:21:32.063674 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":205757,"threads_started":1,"wal-append.queue_time_us":1079027}
I20260709 13:21:31.850098 21706 raft_consensus.cc:493] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:21:31.850244 21949 raft_consensus.cc:493] T 66ae176961b74813a58bc15c9c4c6f2a P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:32.067001 21706 raft_consensus.cc:515] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:32.067410 21941 consensus_queue.cc:1048] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:21:31.809685 21801 consensus_queue.cc:1048] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 9, Time since last communication: 0.048s
I20260709 13:21:32.063256 21944 consensus_queue.cc:1048] T 8d4ab870a05347868956aa8b841b2981 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.005s
I20260709 13:21:31.887916 21580 raft_consensus.cc:515] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:31.862001 16293 rpcz_store.cc:276] 0709 13:21:30.354665 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:30.636693 (+282028us) service_pool.cc:224] Handling call
0709 13:21:30.637070 (+   377us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:21:30.637333 (+   263us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:21:30.980237 (+342904us) log.cc:844] Serialized 14 byte log entry
0709 13:21:31.812416 (+832179us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:21:31.812609 (+   193us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:21:31.812656 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:31.812718 (+    62us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:21:31.812821 (+   103us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":18306,"raft.queue_time_us":52728,"raft.run_cpu_time_us":105,"raft.run_wall_time_us":107,"thread_start_us":1155856,"threads_started":2,"wal-append.queue_time_us":610267}
I20260709 13:21:32.069504 21706 leader_election.cc:290] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:21:32.069563 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c61a39fd4f84424b129c1a6f9ba404d" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:32.070005 21580 leader_election.cc:290] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:31.141103 21543 consensus_queue.cc:237] T 5a959dbecc2648eb85d1cc550f5a3245 P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:32.070353 16293 raft_consensus.cc:2468] T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 16.
I20260709 13:21:32.071334 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c74490c40243a88e62d1760b5cd814" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:31.847146 21954 consensus_queue.cc:1048] T c0eab25ce4c64e018a041beeda2867db P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.075s
I20260709 13:21:31.944804 16294 raft_consensus.cc:2468] T a378d7f796f94615b3b347119a9880fd P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 18.
I20260709 13:21:31.945502 21946 raft_consensus.cc:493] T d16dfad0c3b540e5b4b86492ddf87208 P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:21:31.955897 16822 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49906 (request call id 1118) took 2089 ms. Trace:
W20260709 13:21:31.959964 21245 consensus_meta.cc:287] T 1df93779209748b9860f6d893dd3de53 P 647a343a542d4a139fe106c19095129b: Time spent flushing consensus metadata: real 0.985s	user 0.010s	sys 0.000s
I20260709 13:21:32.078730 21245 raft_consensus.cc:515] T 1df93779209748b9860f6d893dd3de53 P 647a343a542d4a139fe106c19095129b [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:32.079901 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e65676b21b748728ad39668cfe722ec" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:21:31.961674 21804 leader_election.cc:290] T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:31.962386 16218 raft_consensus.cc:2410] T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 for term 17 because replica has last-logged OpId of term: 15 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 7.
W20260709 13:21:31.887394 21704 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.957s	user 0.004s	sys 0.012s
I20260709 13:21:31.964906 21773 leader_election.cc:290] T c4f65cdf00b748bb963ee6e81ef43b79 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:21:31.973001 21930 leader_election.cc:290] T c0eab25ce4c64e018a041beeda2867db P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:31.974591 16297 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40474 (request call id 986) took 1737 ms. Trace:
I20260709 13:21:31.977800 16299 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48674 (request call id 3047) took 1623 ms. Trace:
I20260709 13:21:31.859748 21904 raft_consensus.cc:515] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:21:32.000072 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 194298us
I20260709 13:21:32.008881 16223 raft_consensus.cc:1275] T 18ad7e3af5504571a69d10c00750a01c P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 13:21:31.858460 21913 raft_consensus.cc:515] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:31.999699 21920 raft_consensus.cc:697] T 8c11074579b046d99e585c2818e18a2c P cb82cc7bd6db46d980c75237a1402860 [term 15 LEADER]: Becoming Leader. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:21:32.088109 16191 leader_election.cc:304] T 5a959dbecc2648eb85d1cc550f5a3245 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:21:32.006975 21736 raft_consensus.cc:3060] T 2794af5b99984efebdee564035841cb0 P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:21:32.052282 16789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 241469us
I20260709 13:21:31.147769 21569 leader_election.cc:290] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
W20260709 13:21:32.048311 21881 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.071s	user 0.014s	sys 0.031s
I20260709 13:21:32.056975 16144 raft_consensus.cc:2468] T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 17.
I20260709 13:21:32.057447 21937 raft_consensus.cc:515] T ecc1e363230043fe9fef390f0b979f20 P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:32.057462 16291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4dc6b0541ca431ebc757beac00521f1" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:21:32.048821 21931 raft_consensus.cc:697] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b [term 14 LEADER]: Becoming Leader. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:21:32.059651 16117 leader_election.cc:304] T 746aabf856ab401c9ef5e859371e7b67 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860
I20260709 13:21:32.061162 16116 leader_election.cc:304] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
I20260709 13:21:32.053165 21555 raft_consensus.cc:3060] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:21:32.067123 21949 raft_consensus.cc:515] T 66ae176961b74813a58bc15c9c4c6f2a P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:32.067544 16292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0eab25ce4c64e018a041beeda2867db" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 17 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:32.055270 21749 raft_consensus.cc:2804] T 2794af5b99984efebdee564035841cb0 P cb82cc7bd6db46d980c75237a1402860 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:21:31.144052 21927 raft_consensus.cc:2749] T 26354361e9b044cea1c2b86c4882fe92 P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:21:32.061928 21097 leader_election.cc:290] T 57a744f5aaf944678a63469cbf514015 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
W20260709 13:21:31.942842 21588 consensus_meta.cc:287] T 7c2095a5f4ad49408dbc7f69c606120d P 647a343a542d4a139fe106c19095129b: Time spent flushing consensus metadata: real 0.968s	user 0.013s	sys 0.006s
I20260709 13:21:32.050547 21934 raft_consensus.cc:740] T 84f2da98864b4425bf7c0055951fdd4b P 1ca2725d290e47d68b1a4b609c9816ac [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:21:32.008052 21854 raft_consensus.cc:515] T 18ad7e3af5504571a69d10c00750a01c P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } }
I20260709 13:21:32.074589 21944 raft_consensus.cc:493] T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b [term 14 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:21:32.077109 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57a744f5aaf944678a63469cbf514015" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:32.078068 16115 leader_election.cc:304] T 9c61a39fd4f84424b129c1a6f9ba404d P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:21:32.118239 16293 raft_consensus.cc:2468] T 57a744f5aaf944678a63469cbf514015 P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 11.
I20260709 13:21:32.053902 21891 leader_election.cc:290] T 08c74490c40243a88e62d1760b5cd814 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:32.080698 16298 raft_consensus.cc:2393] T 9e65676b21b748728ad39668cfe722ec P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 15: Already voted for candidate cb82cc7bd6db46d980c75237a1402860 in this term.
I20260709 13:21:32.076761 16822 rpcz_store.cc:276] 0709 13:21:29.866287 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:29.867347 (+  1060us) service_pool.cc:224] Handling call
0709 13:21:29.867801 (+   454us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:21:29.868282 (+   481us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:21:29.868326 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:21:29.879650 (+ 11324us) log.cc:844] Serialized 26 byte log entry
0709 13:21:30.598093 (+718443us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:21:30.598135 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:30.598212 (+    77us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:21:31.955625 (+1357413us) raft_consensus.cc:1685] finished
0709 13:21:31.955652 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:21:31.955827 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2456,"raft.run_cpu_time_us":150,"raft.run_wall_time_us":8568,"thread_start_us":683350,"threads_started":1,"wal-append.queue_time_us":774857}
I20260709 13:21:32.081636 21704 raft_consensus.cc:697] T 02838c86531d4f98bf58d1c22b8baf77 P 647a343a542d4a139fe106c19095129b [term 15 LEADER]: Becoming Leader. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:21:32.130085 21884 consensus_queue.cc:237] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.10, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:32.130609 21704 consensus_queue.cc:237] T 02838c86531d4f98bf58d1c22b8baf77 P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:31.881667 21742 raft_consensus.cc:493] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:21:32.108793 16292 raft_consensus.cc:2410] T c0eab25ce4c64e018a041beeda2867db P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac for term 17 because replica has last-logged OpId of term: 14 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 7.
I20260709 13:21:32.133955 16292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3884f864a96d428785895e5463e5728f" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:32.135030 16292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c61a39fd4f84424b129c1a6f9ba404d" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
W20260709 13:21:32.108960 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 47866.6us
I20260709 13:21:32.100529 16191 leader_election.cc:304] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
I20260709 13:21:32.095971 16296 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48684 (request call id 2907) took 1109 ms. Trace:
I20260709 13:21:32.137609 16296 rpcz_store.cc:276] 0709 13:21:30.986518 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:31.830543 (+844025us) service_pool.cc:224] Handling call
0709 13:21:31.830918 (+   375us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:21:31.847461 (+ 16543us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:21:31.847491 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:21:31.854587 (+  7096us) log.cc:844] Serialized 26 byte log entry
0709 13:21:32.048802 (+194215us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:21:32.048885 (+    83us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:32.048967 (+    82us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:21:32.095743 (+ 46776us) raft_consensus.cc:1685] finished
0709 13:21:32.095764 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:21:32.095911 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":16094,"raft.queue_time_us":22195,"raft.run_cpu_time_us":150,"raft.run_wall_time_us":22354,"thread_start_us":150760,"threads_started":1,"wal-append.queue_time_us":184447}
I20260709 13:21:32.139439 16296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada5a004ba7c46819051e3ccec71ffdb" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:32.140450 16296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434bff3082d141838068f323bdb0d1ed" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:32.102437 16294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9944d5e4b8c4109af9955c4f08c9095" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:32.141424 16294 raft_consensus.cc:2468] T c9944d5e4b8c4109af9955c4f08c9095 P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 17.
I20260709 13:21:32.085377 16297 rpcz_store.cc:276] 0709 13:21:30.237035 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:30.237181 (+   146us) service_pool.cc:224] Handling call
0709 13:21:30.253312 (+ 16131us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:21:30.261311 (+  7999us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 13:21:30.669162 (+407851us) log.cc:844] Serialized 14 byte log entry
0709 13:21:31.974002 (+1304840us) spinlock_profiling.cc:243] Waited 8.58 ms on lock 0x7b18000a6fc0. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b693ade1 00007f69b69475b1 00007f69b07074bc 00007f69ada5548b 00007f69b68afab9 00007f69ada7169b 00007f69ada6f93f 00007f69ada6f455 00007f69b0b57060 00007f69b0b69716 00007f69b0b7a287
0709 13:21:31.974121 (+   119us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:21:31.974187 (+    66us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:21:31.974260 (+    73us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:31.974352 (+    92us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:21:31.974515 (+   163us) inbound_call.cc:177] Queueing success response
0709 13:21:32.078728 (+104213us) spinlock_profiling.cc:243] Waited 5.35 ms on lock 0x7b18000a6fc0. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b693ade1 00007f69b69475b1 00007f69ada72522 00007f69ada747c1 00007f69ada74779 00007f69ada74709 00007f69ada746d1 00007f69ada739cd 000000000038f354 000000000038f199 00007f69ada5583a
Metrics: {"mutex_wait_us":879265,"prepare.queue_time_us":757696,"prepare.run_cpu_time_us":62,"prepare.run_wall_time_us":10031,"raft.queue_time_us":238945,"raft.run_cpu_time_us":361,"raft.run_wall_time_us":17327,"spinlock_wait_cycles":32046592,"thread_start_us":734761,"threads_started":3,"wal-append.queue_time_us":489148}
I20260709 13:21:32.087507 16299 rpcz_store.cc:276] 0709 13:21:30.354426 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:30.458325 (+103899us) service_pool.cc:224] Handling call
0709 13:21:31.977593 (+1519268us) spinlock_profiling.cc:243] Waited 659 ms on lock 0x7b54000cdd54. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:21:31.977734 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1515152768}
I20260709 13:21:32.096246 21245 leader_election.cc:290] T 1df93779209748b9860f6d893dd3de53 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 14 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:32.136358 16116 leader_election.cc:304] T 84f2da98864b4425bf7c0055951fdd4b P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:21:32.144471 16291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a9de347c711477e87c582d003f36925" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:32.144836 16294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a9de347c711477e87c582d003f36925" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 19 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:32.145138 16291 raft_consensus.cc:2468] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 18.
I20260709 13:21:32.073076 21444 leader_election.cc:290] T 5a959dbecc2648eb85d1cc550f5a3245 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
W20260709 13:21:32.110610 21881 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.138s	user 0.014s	sys 0.035s
I20260709 13:21:32.147671 16822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57a744f5aaf944678a63469cbf514015" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:32.127512 16221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6f07842d3aa47858df5665874c86f3a" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:32.127633 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1df93779209748b9860f6d893dd3de53" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:21:32.148593 16817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31969392ccdb43b491911a319e8bc7e1" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:32.149057 16293 raft_consensus.cc:3060] T 1df93779209748b9860f6d893dd3de53 P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:21:32.149187 16817 raft_consensus.cc:2468] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 14.
I20260709 13:21:32.128337 16219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02838c86531d4f98bf58d1c22b8baf77" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:21:32.150404 16219 raft_consensus.cc:2393] T 02838c86531d4f98bf58d1c22b8baf77 P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 15: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
I20260709 13:21:32.129407 21909 leader_election.cc:290] T fffab26427e24d398e2695aad40ca134 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
W20260709 13:21:32.116479 16117 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 56894.3us
I20260709 13:21:32.153226 16117 leader_election.cc:304] T ee9ed67006144797b1de6ea8c2013a68 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860
W20260709 13:21:32.154912 16147 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46224 (request call id 911) took 1808 ms (client timeout 1976 ms). Trace:
W20260709 13:21:32.155467 16147 rpcz_store.cc:269] 0709 13:21:30.346964 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:30.347189 (+   225us) service_pool.cc:224] Handling call
0709 13:21:32.147881 (+1800692us) spinlock_profiling.cc:243] Waited 321 ms on lock 0x7b54000730d4. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:21:32.154848 (+  6967us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":737858944}
I20260709 13:21:32.135639 16292 raft_consensus.cc:2468] T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 16.
I20260709 13:21:32.047039 21906 raft_consensus.cc:515] T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:32.094866 21804 raft_consensus.cc:2749] T 5a959dbecc2648eb85d1cc550f5a3245 P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:21:32.102144 21904 leader_election.cc:290] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:32.147902 16191 leader_election.cc:304] T b6f07842d3aa47858df5665874c86f3a P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:21:32.108253 21749 raft_consensus.cc:493] T 2794af5b99984efebdee564035841cb0 P cb82cc7bd6db46d980c75237a1402860 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:21:32.174868 16822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62fe3346cd3c42c4a55e14af9073bca4" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:32.175179 16817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87147731a68400e8cce9b879daa7c5d" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:32.103933 16047 catalog_manager.cc:5697] T 829b4ad8c5a54eababeea1181b960d4b P 1ca2725d290e47d68b1a4b609c9816ac reported cstate change: term changed from 10 to 12, leader changed from cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) to 647a343a542d4a139fe106c19095129b (127.14.177.129). New cstate: current_term: 12 leader_uuid: "647a343a542d4a139fe106c19095129b" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:21:32.161363 16221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ab0ac8347641e6955e9bce7d90c1a2" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:32.176858 16219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a96373bb0e14dce8a266b2fc6c58507" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:32.177271 16221 raft_consensus.cc:2468] T d2ab0ac8347641e6955e9bce7d90c1a2 P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 14.
I20260709 13:21:32.150892 16266 leader_election.cc:304] T 31969392ccdb43b491911a319e8bc7e1 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:21:32.107754 21588 raft_consensus.cc:515] T 7c2095a5f4ad49408dbc7f69c606120d P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:32.120141 21569 raft_consensus.cc:2749] T 746aabf856ab401c9ef5e859371e7b67 P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:21:32.119990 21097 raft_consensus.cc:2804] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:21:32.158481 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31969392ccdb43b491911a319e8bc7e1" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:32.168430 16824 raft_consensus.cc:1275] T 8638ee363273426db1393dd81e5c7b72 P cb82cc7bd6db46d980c75237a1402860 [term 16 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:21:32.182309 16819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a9de347c711477e87c582d003f36925" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:32.134459 21927 raft_consensus.cc:2804] T 9c61a39fd4f84424b129c1a6f9ba404d P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:21:32.117650 21773 consensus_queue.cc:1059] T 3884f864a96d428785895e5463e5728f P 90f3da57796b488c9087a2f523adf045 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.056s
I20260709 13:21:32.118925 21623 raft_consensus.cc:515] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:32.172150 16825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74ca8140f35c4ae29e8a664f6bdfc829" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:32.081642 21946 raft_consensus.cc:515] T d16dfad0c3b540e5b4b86492ddf87208 P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:32.178149 16221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1df93779209748b9860f6d893dd3de53" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:21:32.178509 16297 raft_consensus.cc:1275] T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:21:32.179641 16823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e65676b21b748728ad39668cfe722ec" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:21:32.147029 21891 raft_consensus.cc:2804] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:21:32.180336 16821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3884f864a96d428785895e5463e5728f" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:32.215085 16821 raft_consensus.cc:2393] T 3884f864a96d428785895e5463e5728f P cb82cc7bd6db46d980c75237a1402860 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 11: Already voted for candidate cb82cc7bd6db46d980c75237a1402860 in this term.
I20260709 13:21:32.190035 16219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4dc6b0541ca431ebc757beac00521f1" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:21:32.157785 21424 raft_consensus.cc:493] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:32.215457 16221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b7d1ab519044eda2d5e04a07242f25" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:21:32.184826 16820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57a744f5aaf944678a63469cbf514015" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:32.098007 21913 leader_election.cc:290] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:32.176827 21749 raft_consensus.cc:3060] T 2794af5b99984efebdee564035841cb0 P cb82cc7bd6db46d980c75237a1402860 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:21:32.185257 16819 raft_consensus.cc:2468] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 18.
I20260709 13:21:32.179224 21897 raft_consensus.cc:493] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:32.233821 21897 raft_consensus.cc:515] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:32.235340 21867 raft_consensus.cc:3055] T 3884f864a96d428785895e5463e5728f P 90f3da57796b488c9087a2f523adf045 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:21:32.235800 21867 raft_consensus.cc:740] T 3884f864a96d428785895e5463e5728f P 90f3da57796b488c9087a2f523adf045 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:21:32.236173 21897 leader_election.cc:290] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:32.181874 16817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434bff3082d141838068f323bdb0d1ed" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:32.237154 16817 raft_consensus.cc:2410] T 434bff3082d141838068f323bdb0d1ed P cb82cc7bd6db46d980c75237a1402860 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b for term 16 because replica has last-logged OpId of term: 15 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 12.
I20260709 13:21:32.236696 21867 consensus_queue.cc:260] T 3884f864a96d428785895e5463e5728f P 90f3da57796b488c9087a2f523adf045 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 10.11, Last appended by leader: 11, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:32.237929 21867 raft_consensus.cc:3060] T 3884f864a96d428785895e5463e5728f P 90f3da57796b488c9087a2f523adf045 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:21:32.240386 16791 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:32.184857 21097 raft_consensus.cc:493] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:32.241168 21097 raft_consensus.cc:3060] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:21:32.173514 21909 raft_consensus.cc:2749] T ee9ed67006144797b1de6ea8c2013a68 P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:21:32.171923 21245 raft_consensus.cc:2749] T 84f2da98864b4425bf7c0055951fdd4b P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:21:32.191152 16115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 71967.4us
I20260709 13:21:32.217505 16825 raft_consensus.cc:2468] T 74ca8140f35c4ae29e8a664f6bdfc829 P cb82cc7bd6db46d980c75237a1402860 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 15.
I20260709 13:21:32.212026 16293 raft_consensus.cc:2468] T 1df93779209748b9860f6d893dd3de53 P 90f3da57796b488c9087a2f523adf045 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 14.
I20260709 13:21:32.134151 21944 raft_consensus.cc:515] T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:32.256412 16295 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40474 (request call id 978) took 2318 ms. Trace:
I20260709 13:21:32.259064 16292 raft_consensus.cc:1240] T 1df93779209748b9860f6d893dd3de53 P 90f3da57796b488c9087a2f523adf045 [term 14 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 13. Current term is 14. Ops: []
I20260709 13:21:32.118057 21580 consensus_queue.cc:1048] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 6, Time since last communication: 0.053s
I20260709 13:21:32.220577 16221 raft_consensus.cc:2393] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 in current term 13: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
I20260709 13:21:32.220553 21424 raft_consensus.cc:515] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:32.181073 21843 raft_consensus.cc:2804] T 31969392ccdb43b491911a319e8bc7e1 P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:21:32.259402 16295 rpcz_store.cc:276] 0709 13:21:29.929906 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:29.930075 (+   169us) service_pool.cc:224] Handling call
0709 13:21:29.957253 (+ 27178us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:21:29.957534 (+   281us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:21:30.612143 (+654609us) log.cc:844] Serialized 14 byte log entry
0709 13:21:31.972094 (+1359951us) spinlock_profiling.cc:243] Waited 6.29 ms on lock 0x7b180027a660. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b693ade1 00007f69b69475b1 00007f69ad9ecc0a 00007f69b712c219 00007f69b4a3358f 00007f69ada72571 00007f69ada747c1 00007f69ada74779 00007f69ada74709 00007f69ada746d1 00007f69ada739cd
0709 13:21:31.979062 (+  6968us) spinlock_profiling.cc:243] Waited 8.16 ms on lock 0x7b180027a660. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b693ade1 00007f69b69475b1 00007f69ada72522 00007f69ada747c1 00007f69ada74779 00007f69ada74709 00007f69ada746d1 00007f69ada739cd 000000000038f354 000000000038f199 00007f69ada5583a
0709 13:21:32.238891 (+259829us) spinlock_profiling.cc:243] Waited 6.85 ms on lock 0x7b180027a660. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b693ade1 00007f69b69475b1 00007f69ada723f3 00007f69ada747c1 00007f69ada74779 00007f69ada74709 00007f69ada746d1 00007f69ada739cd 000000000038f354 000000000038f199 00007f69ada5583a
0709 13:21:32.248089 (+  9198us) spinlock_profiling.cc:243] Waited 8.49 ms on lock 0x7b180027a660. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b693ade1 00007f69b69475b1 00007f69b07074bc 00007f69ada5548b 00007f69b68afab9 00007f69ada7169b 00007f69ada6f93f 00007f69ada6f455 00007f69b0b57060 00007f69b0b69716 00007f69b0b7a287
0709 13:21:32.248177 (+    88us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:21:32.248223 (+    46us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:21:32.248263 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:32.248325 (+    62us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:21:32.248422 (+    97us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":49219,"prepare.queue_time_us":1038183,"prepare.run_cpu_time_us":45,"prepare.run_wall_time_us":46,"raft.queue_time_us":239753,"raft.run_cpu_time_us":177,"raft.run_wall_time_us":13649,"spinlock_wait_cycles":68524928,"thread_start_us":868090,"threads_started":3,"wal-append.queue_time_us":436576}
I20260709 13:21:32.231986 21847 leader_election.cc:290] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:32.218345 16192 leader_election.cc:304] T ee9ed67006144797b1de6ea8c2013a68 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
I20260709 13:21:32.151716 21742 raft_consensus.cc:515] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:32.169839 21888 leader_election.cc:290] T af57996a8be84d398f0f023e8d08ef77 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:21:32.247469 16223 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57604 (request call id 2970) took 1588 ms. Trace:
I20260709 13:21:32.180064 21904 raft_consensus.cc:2749] T b6f07842d3aa47858df5665874c86f3a P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:21:32.295753 16223 rpcz_store.cc:276] 0709 13:21:30.659424 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:30.659707 (+   283us) service_pool.cc:224] Handling call
0709 13:21:30.660069 (+   362us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:21:32.087843 (+1427774us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:32.247246 (+159403us) spinlock_profiling.cc:243] Waited 1.35 s on lock 0x7b5400367254. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:21:32.247412 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3101107584}
I20260709 13:21:32.214268 21927 raft_consensus.cc:493] T 9c61a39fd4f84424b129c1a6f9ba404d P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:32.183317 21954 raft_consensus.cc:493] T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:32.095825 21920 consensus_queue.cc:237] T 8c11074579b046d99e585c2818e18a2c P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:21:32.271714 16192 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 53416.4us
I20260709 13:21:32.266572 16047 catalog_manager.cc:5697] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac reported cstate change: term changed from 12 to 16. New cstate: current_term: 16 leader_uuid: "647a343a542d4a139fe106c19095129b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:21:32.270985 16219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0eab25ce4c64e018a041beeda2867db" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
W20260709 13:21:32.274061 21962 log.cc:927] Time spent T 9e65676b21b748728ad39668cfe722ec P cb82cc7bd6db46d980c75237a1402860: Append to log took a long time: real 0.089s	user 0.000s	sys 0.000s
I20260709 13:21:32.277427 21903 raft_consensus.cc:3055] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:21:32.278041 21946 leader_election.cc:290] T d16dfad0c3b540e5b4b86492ddf87208 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:32.255317 21934 consensus_queue.cc:260] T 84f2da98864b4425bf7c0055951fdd4b P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:32.281670 16220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8638ee363273426db1393dd81e5c7b72" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:21:32.283855 16151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ad5dbfa99ee4b8d9e28be92b3f36db5" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:21:32.284551 21424 leader_election.cc:290] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:21:32.285167 16147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66f75124215f4ca6bef831e174a792d8" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:32.285722 16143 raft_consensus.cc:1240] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 15. Current term is 16. Ops: [15.10-15.10]
I20260709 13:21:32.285873 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c74490c40243a88e62d1760b5cd814" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:32.286139 16115 leader_election.cc:304] T 57a744f5aaf944678a63469cbf514015 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:21:32.286594 16146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0eab25ce4c64e018a041beeda2867db" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 17 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:32.287130 16822 raft_consensus.cc:2468] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 20.
I20260709 13:21:32.290591 16145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5ad4eb281564936a59f41e387a47156" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:32.291342 16148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c61a39fd4f84424b129c1a6f9ba404d" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:32.268707 21929 raft_consensus.cc:493] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:32.183570 21579 raft_consensus.cc:515] T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:32.266892 21773 consensus_queue.cc:1048] T 8638ee363273426db1393dd81e5c7b72 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.155s
I20260709 13:21:32.341526 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d16dfad0c3b540e5b4b86492ddf87208" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:32.660147 16223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af57996a8be84d398f0f023e8d08ef77" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:32.689311 16300 raft_consensus.cc:2468] T d16dfad0c3b540e5b4b86492ddf87208 P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 15.
W20260709 13:21:32.634508 16791 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:21:32.691140 16115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 405080us
I20260709 13:21:32.692787 16115 leader_election.cc:304] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:21:32.693589 16220 raft_consensus.cc:2393] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 16: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
W20260709 13:21:32.694065 16220 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57600 (request call id 1751) took 1631 ms (client timeout 1519 ms). Trace:
W20260709 13:21:32.694449 16220 rpcz_store.cc:269] 0709 13:21:31.063428 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:31.091534 (+ 28106us) service_pool.cc:224] Handling call
0709 13:21:32.281535 (+1190001us) spinlock_profiling.cc:243] Waited 16.4 ms on lock 0x7b5800010b98. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69ad8fadb1 00007f69ada3ab71 00007f69ada3aa22 00007f69b6953c09 00007f69b5aadee0 00007f69b5b770ab 00007f69b5b77131 00007f69b5b250a0 00007f69b5b24984 00007f69af09f6d4 00007f69af09f662
0709 13:21:32.693980 (+412445us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":37616384}
I20260709 13:21:32.695477 16147 raft_consensus.cc:2468] T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 14.
I20260709 13:21:32.299300 16221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a878999c0024d2abe517484180290fc" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:32.696622 16221 raft_consensus.cc:2410] T 8a878999c0024d2abe517484180290fc P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 for term 15 because replica has last-logged OpId of term: 14 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 7.
I20260709 13:21:32.347237 16222 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57600 (request call id 1749) took 1306 ms. Trace:
I20260709 13:21:32.337929 16226 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57600 (request call id 1744) took 1317 ms. Trace:
I20260709 13:21:32.698764 16226 rpcz_store.cc:276] 0709 13:21:31.020650 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:31.020772 (+   122us) service_pool.cc:224] Handling call
0709 13:21:32.337652 (+1316880us) spinlock_profiling.cc:243] Waited 498 ms on lock 0x7b5400138b54. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:21:32.337848 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1145191680}
I20260709 13:21:32.670089 16190 leader_election.cc:304] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:21:32.697713 16222 rpcz_store.cc:276] 0709 13:21:31.042485 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:31.667466 (+624981us) service_pool.cc:224] Handling call
0709 13:21:31.667872 (+   406us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:21:32.055732 (+387860us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:32.346906 (+291174us) spinlock_profiling.cc:243] Waited 184 ms on lock 0x7b5400140fd4. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:21:32.347145 (+   239us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":424134912}
I20260709 13:21:32.702324 16222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada5a004ba7c46819051e3ccec71ffdb" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:21:32.610616 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:32.703260 16222 raft_consensus.cc:2376] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 16. Re-sending same reply.
I20260709 13:21:32.689693 16223 raft_consensus.cc:2468] T af57996a8be84d398f0f023e8d08ef77 P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 16.
I20260709 13:21:32.705192 16143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5718c623ebee4d2bb770f0f9ddb2f164" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:32.703812 16225 raft_consensus.cc:2468] T 19baa7bc367844c288bca8f81439c413 P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 17.
I20260709 13:21:32.706880 16219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c61a39fd4f84424b129c1a6f9ba404d" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
W20260709 13:21:32.707609 16225 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57600 (request call id 1754) took 1612 ms (client timeout 1771 ms). Trace:
W20260709 13:21:32.314950 21963 log.cc:927] Time spent T 5a959dbecc2648eb85d1cc550f5a3245 P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 0.231s	user 0.001s	sys 0.000s
W20260709 13:21:32.708253 16225 rpcz_store.cc:269] 0709 13:21:31.095581 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:31.105677 (+ 10096us) service_pool.cc:224] Handling call
0709 13:21:32.610445 (+1504768us) spinlock_profiling.cc:243] Waited 11.2 ms on lock 0x7b5800010b98. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69ad8fadb1 00007f69ada3ab71 00007f69ada3aa22 00007f69b6953c09 00007f69b5aadee0 00007f69b5b770ab 00007f69b5b77131 00007f69b5b250a0 00007f69b5b24984 00007f69af09f6d4 00007f69af09f662
0709 13:21:32.707516 (+ 97071us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":25665280}
I20260709 13:21:32.710901 16145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d16dfad0c3b540e5b4b86492ddf87208" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:32.778548 16819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e9b2bf7011f4ae688a79d7bb5a447b4" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:32.271782 21843 raft_consensus.cc:493] T 31969392ccdb43b491911a319e8bc7e1 P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Starting leader election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:21:32.837085 21843 raft_consensus.cc:3060] T 31969392ccdb43b491911a319e8bc7e1 P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:21:32.838634 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8638ee363273426db1393dd81e5c7b72" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:32.839354 16226 raft_consensus.cc:2468] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 16.
I20260709 13:21:32.839357 21883 leader_election.cc:290] T af57996a8be84d398f0f023e8d08ef77 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:32.840339 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c11074579b046d99e585c2818e18a2c" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:21:32.841150 16226 raft_consensus.cc:2393] T 8c11074579b046d99e585c2818e18a2c P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 in current term 15: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
I20260709 13:21:32.841658 16149 raft_consensus.cc:1240] T 1df93779209748b9860f6d893dd3de53 P 647a343a542d4a139fe106c19095129b [term 14 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 13. Current term is 14. Ops: []
I20260709 13:21:32.841964 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee9ed67006144797b1de6ea8c2013a68" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:21:32.259687 21931 consensus_queue.cc:237] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:32.834888 16047 catalog_manager.cc:5697] T e5ad4eb281564936a59f41e387a47156 P 1ca2725d290e47d68b1a4b609c9816ac reported cstate change: term changed from 4 to 13, leader changed from 647a343a542d4a139fe106c19095129b (127.14.177.129) to 90f3da57796b488c9087a2f523adf045 (127.14.177.131). New cstate: current_term: 13 leader_uuid: "90f3da57796b488c9087a2f523adf045" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } }
I20260709 13:21:32.682667 16266 leader_election.cc:304] T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:21:32.834053 16219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746aabf856ab401c9ef5e859371e7b67" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:32.843230 21946 consensus_queue.cc:1048] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.070s
I20260709 13:21:32.851445 16221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af57996a8be84d398f0f023e8d08ef77" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:32.852173 16221 raft_consensus.cc:2468] T af57996a8be84d398f0f023e8d08ef77 P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 16.
I20260709 13:21:32.855921 16219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5d59407eb545b895d8d922e0dd69fd" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
W20260709 13:21:32.857649 16266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 175077us
I20260709 13:21:32.858904 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d4ab870a05347868956aa8b841b2981" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:32.294943 21941 raft_consensus.cc:2749] T ee9ed67006144797b1de6ea8c2013a68 P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:21:32.271034 21891 raft_consensus.cc:493] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:32.708963 16192 leader_election.cc:304] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 647a343a542d4a139fe106c19095129b
I20260709 13:21:32.777014 16220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "829b4ad8c5a54eababeea1181b960d4b" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:32.845139 16823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a9de347c711477e87c582d003f36925" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:32.866707 16823 raft_consensus.cc:2468] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 18.
I20260709 13:21:32.846850 16819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66ae176961b74813a58bc15c9c4c6f2a" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:32.715541 16222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdbd78acb4cf46298e0de8ba0d130e19" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:32.867569 16819 raft_consensus.cc:2468] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 19.
I20260709 13:21:32.868525 16220 raft_consensus.cc:2468] T 829b4ad8c5a54eababeea1181b960d4b P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 14.
W20260709 13:21:32.631215 21555 consensus_meta.cc:287] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860: Time spent flushing consensus metadata: real 0.515s	user 0.008s	sys 0.001s
I20260709 13:21:32.859578 16300 raft_consensus.cc:2468] T 8d4ab870a05347868956aa8b841b2981 P 90f3da57796b488c9087a2f523adf045 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 14.
I20260709 13:21:32.654798 21742 leader_election.cc:290] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
W20260709 13:21:32.711861 21736 consensus_meta.cc:287] T 2794af5b99984efebdee564035841cb0 P 90f3da57796b488c9087a2f523adf045: Time spent flushing consensus metadata: real 0.589s	user 0.010s	sys 0.000s
I20260709 13:21:32.705031 21569 raft_consensus.cc:2804] T 57a744f5aaf944678a63469cbf514015 P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:21:32.711627 21929 raft_consensus.cc:515] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:32.864019 16221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada5a004ba7c46819051e3ccec71ffdb" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:32.868441 16222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2372981006d4b218215c28964dea623" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:32.877588 16226 raft_consensus.cc:2376] T ee9ed67006144797b1de6ea8c2013a68 P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 15. Re-sending same reply.
I20260709 13:21:32.877940 16221 raft_consensus.cc:2468] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 16.
I20260709 13:21:32.878618 16222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c74490c40243a88e62d1760b5cd814" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:32.878885 16221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdbd78acb4cf46298e0de8ba0d130e19" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:32.879503 16222 raft_consensus.cc:2468] T 08c74490c40243a88e62d1760b5cd814 P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 14.
I20260709 13:21:32.880065 16221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fffab26427e24d398e2695aad40ca134" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:32.880705 16221 raft_consensus.cc:2468] T fffab26427e24d398e2695aad40ca134 P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 15.
I20260709 13:21:32.848371 16223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4dc6b0541ca431ebc757beac00521f1" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:32.882892 16825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9944d5e4b8c4109af9955c4f08c9095" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:32.781315 21913 raft_consensus.cc:2804] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Leader election won for term 18
W20260709 13:21:32.875737 16192 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 166894us
I20260709 13:21:32.888041 16192 leader_election.cc:304] T d2ab0ac8347641e6955e9bce7d90c1a2 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860
I20260709 13:21:32.891774 16220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87ac0d1f7e8a419f9bcbc04aee0e067e" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:32.346283 21908 raft_consensus.cc:493] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:21:32.691772 21903 raft_consensus.cc:740] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:21:32.340658 21949 leader_election.cc:290] T 66ae176961b74813a58bc15c9c4c6f2a P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:32.318010 21927 raft_consensus.cc:3060] T 9c61a39fd4f84424b129c1a6f9ba404d P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:21:32.348242 21854 leader_election.cc:290] T 18ad7e3af5504571a69d10c00750a01c P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:21:32.876631 16219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e4bfb9090c8460eb14fa7e0d818e7f1" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:32.624331 21906 leader_election.cc:290] T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:32.865043 16047 catalog_manager.cc:5697] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac reported cstate change: term changed from 8 to 11, leader changed from 647a343a542d4a139fe106c19095129b (127.14.177.129) to 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130). New cstate: current_term: 11 leader_uuid: "1ca2725d290e47d68b1a4b609c9816ac" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } health_report { overall_health: UNKNOWN } } }
I20260709 13:21:32.664011 21856 leader_election.cc:290] T c9944d5e4b8c4109af9955c4f08c9095 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:32.626590 21097 raft_consensus.cc:515] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:32.705538 21245 raft_consensus.cc:2804] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:21:32.902693 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5ad4eb281564936a59f41e387a47156" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 17 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:32.860570 16266 leader_election.cc:304] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
I20260709 13:21:32.354082 21749 raft_consensus.cc:515] T 2794af5b99984efebdee564035841cb0 P cb82cc7bd6db46d980c75237a1402860 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:32.870911 21847 raft_consensus.cc:2804] T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 13:21:32.999068 16266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 138578us
I20260709 13:21:32.935626 16146 raft_consensus.cc:1275] T ecc1e363230043fe9fef390f0b979f20 P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 13:21:32.885265 21736 raft_consensus.cc:515] T 2794af5b99984efebdee564035841cb0 P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:32.709466 21954 raft_consensus.cc:515] T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:32.896701 21569 raft_consensus.cc:493] T 57a744f5aaf944678a63469cbf514015 P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Starting leader election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:21:32.957909 21929 leader_election.cc:290] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:32.958756 16143 raft_consensus.cc:2393] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 15: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
I20260709 13:21:32.937461 21742 raft_consensus.cc:493] T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:32.871814 21891 raft_consensus.cc:3060] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:21:32.986577 21725 raft_consensus.cc:493] T 23f82224f4564fea93572e8206bad1fe P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:21:33.015537 21736 leader_election.cc:290] T 2794af5b99984efebdee564035841cb0 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 13 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:33.000413 21704 raft_consensus.cc:493] T ecc1e363230043fe9fef390f0b979f20 P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:33.017563 21704 raft_consensus.cc:515] T ecc1e363230043fe9fef390f0b979f20 P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:21:32.893934 21884 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.626s	user 0.016s	sys 0.026s
W20260709 13:21:33.019367 21884 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.763s	user 0.017s	sys 0.027s
I20260709 13:21:33.014997 21742 raft_consensus.cc:515] T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:33.020736 21569 raft_consensus.cc:3060] T 57a744f5aaf944678a63469cbf514015 P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:21:32.964246 21579 leader_election.cc:290] T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 18 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:33.022802 16297 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48674 (request call id 3071) took 1142 ms. Trace:
I20260709 13:21:33.023224 16297 rpcz_store.cc:276] 0709 13:21:31.880807 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:32.227405 (+346598us) service_pool.cc:224] Handling call
0709 13:21:32.227734 (+   329us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:21:32.255421 (+ 27687us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:21:32.255453 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:21:32.259966 (+  4513us) log.cc:844] Serialized 26 byte log entry
0709 13:21:32.660354 (+400388us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:21:32.660415 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:32.660513 (+    98us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:21:33.022590 (+362077us) raft_consensus.cc:1685] finished
0709 13:21:33.022617 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:21:33.022748 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":27305,"raft.queue_time_us":28805,"raft.run_cpu_time_us":113,"raft.run_wall_time_us":11205,"thread_start_us":246276,"threads_started":1,"wal-append.queue_time_us":685300}
I20260709 13:21:32.966356 21904 raft_consensus.cc:2804] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:21:32.935364 16145 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "02838c86531d4f98bf58d1c22b8baf77"
dest_uuid: "647a343a542d4a139fe106c19095129b"
mode: GRACEFUL
new_leader_uuid: "90f3da57796b488c9087a2f523adf045"
 from {username='slave'} at 127.0.0.1:40290
I20260709 13:21:32.937781 16151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31969392ccdb43b491911a319e8bc7e1" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:33.027390 16146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af57996a8be84d398f0f023e8d08ef77" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:33.027011 16145 raft_consensus.cc:606] T 02838c86531d4f98bf58d1c22b8baf77 P 647a343a542d4a139fe106c19095129b [term 15 LEADER]: Received request to transfer leadership to TS 90f3da57796b488c9087a2f523adf045
I20260709 13:21:33.027998 16146 raft_consensus.cc:2468] T af57996a8be84d398f0f023e8d08ef77 P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 16.
I20260709 13:21:32.910815 21908 raft_consensus.cc:3060] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:21:33.002175 16219 raft_consensus.cc:1275] T ecc1e363230043fe9fef390f0b979f20 P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 13:21:33.000751 16266 leader_election.cc:304] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:21:32.992913 21245 raft_consensus.cc:493] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:33.009073 21856 raft_consensus.cc:2749] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:21:33.020262 21749 leader_election.cc:290] T 2794af5b99984efebdee564035841cb0 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 13 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
W20260709 13:21:32.965837 21867 consensus_meta.cc:287] T 3884f864a96d428785895e5463e5728f P 90f3da57796b488c9087a2f523adf045: Time spent flushing consensus metadata: real 0.727s	user 0.011s	sys 0.000s
I20260709 13:21:32.936014 16147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746aabf856ab401c9ef5e859371e7b67" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:33.027586 16151 raft_consensus.cc:2468] T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 14.
I20260709 13:21:33.029690 21906 raft_consensus.cc:2749] T d2ab0ac8347641e6955e9bce7d90c1a2 P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:21:33.034111 16150 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.926s	user 0.030s	sys 0.055s
W20260709 13:21:33.410996 16150 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.303s	user 0.033s	sys 0.072s
I20260709 13:21:33.033609 16047 catalog_manager.cc:5697] T 66f75124215f4ca6bef831e174a792d8 P 1ca2725d290e47d68b1a4b609c9816ac reported cstate change: term changed from 7 to 11, leader changed from cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) to 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130). New cstate: current_term: 11 leader_uuid: "1ca2725d290e47d68b1a4b609c9816ac" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:21:33.412626 16224 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57604 (request call id 2982) took 1614 ms. Trace:
I20260709 13:21:33.414129 16219 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33050 (request call id 981) took 1227 ms. Trace:
I20260709 13:21:33.414610 16219 rpcz_store.cc:276] 0709 13:21:32.187074 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:33.001456 (+814382us) service_pool.cc:224] Handling call
0709 13:21:33.001774 (+   318us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:21:33.413833 (+412059us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:33.414077 (+   244us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:21:33.082772 16144 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46224 (request call id 915) took 1813 ms (client timeout 2176 ms). Trace:
I20260709 13:21:33.059706 16146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e65676b21b748728ad39668cfe722ec" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
W20260709 13:21:33.416714 16266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 416040us
I20260709 13:21:33.413009 16224 rpcz_store.cc:276] 0709 13:21:31.798088 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:31.803329 (+  5241us) service_pool.cc:224] Handling call
0709 13:21:31.803766 (+   437us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:21:31.969375 (+165609us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:21:31.969422 (+    47us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:21:31.970248 (+   826us) log.cc:844] Serialized 52 byte log entry
0709 13:21:32.110755 (+140507us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:21:32.110793 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:33.353796 (+1243003us) spinlock_profiling.cc:243] Waited 165 ms on lock 0x7b54002981d4. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:21:33.353872 (+    76us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:21:33.354036 (+   164us) raft_consensus.cc:1685] finished
0709 13:21:33.354058 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:21:33.412542 (+ 58484us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1560,"raft.run_cpu_time_us":150,"raft.run_wall_time_us":152,"spinlock_wait_cycles":379632896,"thread_start_us":117772,"threads_started":1,"wal-append.queue_time_us":153370,"wal-append.run_cpu_time_us":1820,"wal-append.run_wall_time_us":1103520}
I20260709 13:21:33.318467 16818 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49896 (request call id 1359) took 1143 ms. Trace:
I20260709 13:21:33.423230 16146 raft_consensus.cc:2393] T 9e65676b21b748728ad39668cfe722ec P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 in current term 15: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
W20260709 13:21:33.415809 16144 rpcz_store.cc:269] 0709 13:21:31.269936 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:31.270109 (+   173us) service_pool.cc:224] Handling call
0709 13:21:33.082505 (+1812396us) spinlock_profiling.cc:243] Waited 453 ms on lock 0x7b54000e9054. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:21:33.082688 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1041844480}
I20260709 13:21:33.417994 16266 leader_election.cc:304] T 66ae176961b74813a58bc15c9c4c6f2a P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:21:32.915637 21623 leader_election.cc:290] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 19 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:33.439720 16147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62fe3346cd3c42c4a55e14af9073bca4" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:33.440842 16147 raft_consensus.cc:2393] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 in current term 21: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
I20260709 13:21:33.007474 21847 raft_consensus.cc:493] T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:21:33.465396 16265 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.709s	user 0.015s	sys 0.029s
W20260709 13:21:33.467022 16265 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.710s	user 0.016s	sys 0.030s
W20260709 13:21:33.467419 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 761335us
I20260709 13:21:33.420847 16818 rpcz_store.cc:276] 0709 13:21:32.175115 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:32.175316 (+   201us) service_pool.cc:224] Handling call
0709 13:21:32.175587 (+   271us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:21:32.176453 (+   866us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:21:32.176480 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:21:32.176930 (+   450us) log.cc:844] Serialized 26 byte log entry
0709 13:21:32.656566 (+479636us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:21:32.656626 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:32.656821 (+   195us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:21:33.318165 (+661344us) raft_consensus.cc:1685] finished
0709 13:21:33.318194 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:21:33.318392 (+   198us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":520,"raft.queue_time_us":3268,"raft.run_cpu_time_us":122,"raft.run_wall_time_us":2242,"thread_start_us":423010,"threads_started":1,"wal-append.queue_time_us":845308}
I20260709 13:21:33.471508 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66ae176961b74813a58bc15c9c4c6f2a" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 20 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
W20260709 13:21:33.536906 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 68707.2us
I20260709 13:21:33.540664 16823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2794af5b99984efebdee564035841cb0" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
W20260709 13:21:33.276629 21444 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.709s	user 0.023s	sys 0.042s
I20260709 13:21:33.545058 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62fe3346cd3c42c4a55e14af9073bca4" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:33.033202 21904 raft_consensus.cc:493] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Starting leader election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:21:33.550046 16147 raft_consensus.cc:1240] T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 14. Current term is 15. Ops: []
I20260709 13:21:33.571457 16147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f82224f4564fea93572e8206bad1fe" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:33.543332 16823 raft_consensus.cc:2393] T 2794af5b99984efebdee564035841cb0 P cb82cc7bd6db46d980c75237a1402860 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 13: Already voted for candidate cb82cc7bd6db46d980c75237a1402860 in this term.
W20260709 13:21:33.555393 16225 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.538s	user 0.049s	sys 0.036s
W20260709 13:21:33.572463 16225 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.555s	user 0.049s	sys 0.036s
I20260709 13:21:33.569342 16824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 18 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:33.569604 16146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d4ab870a05347868956aa8b841b2981" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:33.277544 21555 raft_consensus.cc:493] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:33.572608 16151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a96373bb0e14dce8a266b2fc6c58507" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:33.677875 16151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8158bbd17b994d2eb96f560bd0f1b88d" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:33.457445 21245 raft_consensus.cc:3060] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:21:33.668049 16146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31969392ccdb43b491911a319e8bc7e1" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:33.697117 16146 raft_consensus.cc:2393] T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 14: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
I20260709 13:21:33.670831 16147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0eab25ce4c64e018a041beeda2867db" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:33.699438 16147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57a744f5aaf944678a63469cbf514015" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:33.684438 16151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d16dfad0c3b540e5b4b86492ddf87208" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:33.317807 21937 leader_election.cc:290] T ecc1e363230043fe9fef390f0b979f20 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:33.463217 21736 raft_consensus.cc:2804] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:21:33.536809 21854 raft_consensus.cc:2804] T 66ae176961b74813a58bc15c9c4c6f2a P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Leader pre-election won for term 20
W20260709 13:21:33.544265 21444 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.982s	user 0.023s	sys 0.042s
I20260709 13:21:33.467697 21847 raft_consensus.cc:3060] T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:21:33.709002 16151 raft_consensus.cc:1240] T 3884f864a96d428785895e5463e5728f P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 10. Current term is 11. Ops: []
I20260709 13:21:33.536770 21949 raft_consensus.cc:493] T 8c11074579b046d99e585c2818e18a2c P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:21:33.543558 21825 raft_consensus.cc:670] T 806d7690c2b644959152074bb2cab0a7 P cb82cc7bd6db46d980c75237a1402860: failed to trigger leader election: Illegal state: only voting members can start elections: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } } [suppressed 3 similar messages]
I20260709 13:21:33.412308 21891 raft_consensus.cc:515] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:33.669198 21904 raft_consensus.cc:3060] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:21:33.541414 21867 raft_consensus.cc:493] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 90f3da57796b488c9087a2f523adf045 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:33.015563 21725 raft_consensus.cc:515] T 23f82224f4564fea93572e8206bad1fe P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:33.706456 16047 catalog_manager.cc:5697] T b6f07842d3aa47858df5665874c86f3a P 1ca2725d290e47d68b1a4b609c9816ac reported cstate change: term changed from 4 to 14, leader changed from 90f3da57796b488c9087a2f523adf045 (127.14.177.131) to 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130). New cstate: current_term: 14 leader_uuid: "1ca2725d290e47d68b1a4b609c9816ac" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } health_report { overall_health: UNKNOWN } } }
I20260709 13:21:33.740018 16147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4dc6b0541ca431ebc757beac00521f1" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:33.705207 21736 raft_consensus.cc:493] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:21:33.755615 16817 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.894s	user 0.001s	sys 0.018s
W20260709 13:21:33.756074 16817 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.894s	user 0.002s	sys 0.018s
W20260709 13:21:33.747908 16820 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.887s	user 0.034s	sys 0.042s
W20260709 13:21:33.744509 16116 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.167s	user 0.004s	sys 0.016s
I20260709 13:21:33.754317 16147 raft_consensus.cc:2393] T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 15: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
W20260709 13:21:33.756811 16820 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.895s	user 0.034s	sys 0.042s
I20260709 13:21:33.674705 21856 raft_consensus.cc:493] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:33.735617 21854 raft_consensus.cc:493] T 66ae176961b74813a58bc15c9c4c6f2a P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:21:33.757872 16116 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.180s	user 0.004s	sys 0.016s
W20260709 13:21:33.740900 21892 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.808s	user 0.006s	sys 0.013s
W20260709 13:21:33.739117 21913 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.748s	user 0.010s	sys 0.008s
W20260709 13:21:33.787676 21913 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.804s	user 0.011s	sys 0.008s
I20260709 13:21:33.788002 21913 raft_consensus.cc:697] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac [term 18 LEADER]: Becoming Leader. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:21:33.788664 21725 leader_election.cc:290] T 23f82224f4564fea93572e8206bad1fe P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:33.788914 21913 consensus_queue.cc:237] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:33.790691 21725 raft_consensus.cc:493] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
W20260709 13:21:33.758211 16221 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.848s	user 0.003s	sys 0.016s
I20260709 13:21:33.767304 16218 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57600 (request call id 1748) took 2745 ms. Trace:
W20260709 13:21:33.813496 16221 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.903s	user 0.003s	sys 0.016s
I20260709 13:21:33.813580 16218 rpcz_store.cc:276] 0709 13:21:31.038663 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:31.040900 (+  2237us) service_pool.cc:224] Handling call
0709 13:21:33.767006 (+2726106us) spinlock_profiling.cc:243] Waited 893 ms on lock 0x7b54000154d4. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:21:33.767230 (+   224us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2054399360}
W20260709 13:21:33.773496 16192 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.882s	user 0.024s	sys 0.099s
I20260709 13:21:33.766610 21854 raft_consensus.cc:3060] T 66ae176961b74813a58bc15c9c4c6f2a P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:21:33.763674 21097 leader_election.cc:290] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 21 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
W20260709 13:21:33.763622 21801 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.825s	user 0.007s	sys 0.012s
I20260709 13:21:33.748991 21949 raft_consensus.cc:515] T 8c11074579b046d99e585c2818e18a2c P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:33.786343 21954 leader_election.cc:290] T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:33.706032 21555 raft_consensus.cc:515] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:21:33.749782 21930 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.875s	user 0.008s	sys 0.011s
I20260709 13:21:33.802893 21937 raft_consensus.cc:493] T 9e65676b21b748728ad39668cfe722ec P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:21:33.839392 21930 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.969s	user 0.009s	sys 0.011s
I20260709 13:21:33.839586 21937 raft_consensus.cc:515] T 9e65676b21b748728ad39668cfe722ec P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:33.840564 21930 raft_consensus.cc:493] T 02838c86531d4f98bf58d1c22b8baf77 P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:33.841034 21930 raft_consensus.cc:515] T 02838c86531d4f98bf58d1c22b8baf77 P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:33.841734 21937 leader_election.cc:290] T 9e65676b21b748728ad39668cfe722ec P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:33.769855 21856 raft_consensus.cc:515] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:33.843734 21856 leader_election.cc:290] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:33.760284 21736 raft_consensus.cc:3060] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:21:33.769490 21892 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.836s	user 0.006s	sys 0.013s
I20260709 13:21:33.845932 21856 raft_consensus.cc:493] T 68a6838d82ef4cac83507f906139d90f P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:21:33.846423 21856 raft_consensus.cc:515] T 68a6838d82ef4cac83507f906139d90f P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:33.756584 21867 raft_consensus.cc:515] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 90f3da57796b488c9087a2f523adf045 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:21:33.799516 21927 consensus_meta.cc:287] T 9c61a39fd4f84424b129c1a6f9ba404d P 647a343a542d4a139fe106c19095129b: Time spent flushing consensus metadata: real 0.845s	user 0.007s	sys 0.002s
I20260709 13:21:33.847725 21927 raft_consensus.cc:515] T 9c61a39fd4f84424b129c1a6f9ba404d P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:33.848562 16291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdbd78acb4cf46298e0de8ba0d130e19" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:21:33.848969 21867 leader_election.cc:290] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:21:33.849247 16291 raft_consensus.cc:3060] T cdbd78acb4cf46298e0de8ba0d130e19 P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:21:33.849861 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66ae176961b74813a58bc15c9c4c6f2a" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 20 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:33.850080 21927 leader_election.cc:290] T 9c61a39fd4f84424b129c1a6f9ba404d P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 17 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:33.851783 21588 leader_election.cc:290] T 7c2095a5f4ad49408dbc7f69c606120d P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 12 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:33.668161 21903 consensus_queue.cc:260] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } }
W20260709 13:21:33.818444 16192 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.927s	user 0.024s	sys 0.099s
W20260709 13:21:33.831979 16222 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.938s	user 0.012s	sys 0.012s
I20260709 13:21:33.817380 21944 leader_election.cc:290] T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
W20260709 13:21:33.853159 16222 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.964s	user 0.013s	sys 0.012s
W20260709 13:21:33.853196 16192 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 965282us
I20260709 13:21:33.854099 16222 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57600 (request call id 1762) took 1728 ms. Trace:
I20260709 13:21:33.854418 16222 rpcz_store.cc:276] 0709 13:21:32.126107 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:32.880359 (+754252us) service_pool.cc:224] Handling call
0709 13:21:32.880709 (+   350us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:21:33.853772 (+973063us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:21:33.853800 (+    28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:21:33.853841 (+    41us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:21:33.853879 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:33.853940 (+    61us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:21:33.854048 (+   108us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":109077,"raft.run_cpu_time_us":124,"raft.run_wall_time_us":8463,"thread_start_us":964417,"threads_started":1}
I20260709 13:21:33.843106 21930 leader_election.cc:290] T 02838c86531d4f98bf58d1c22b8baf77 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:33.851841 16296 raft_consensus.cc:1275] T 9e65676b21b748728ad39668cfe722ec P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 13:21:33.806983 16790 leader_election.cc:304] T af57996a8be84d398f0f023e8d08ef77 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:21:33.848603 21856 leader_election.cc:290] T 68a6838d82ef4cac83507f906139d90f P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:21:33.794652 21946 raft_consensus.cc:493] T c2f5f89186204242bf377f7c42b2e2da P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:33.859539 16291 raft_consensus.cc:2468] T cdbd78acb4cf46298e0de8ba0d130e19 P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 19.
I20260709 13:21:33.859829 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a378d7f796f94615b3b347119a9880fd" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:21:33.867640 16300 raft_consensus.cc:2393] T a378d7f796f94615b3b347119a9880fd P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 in current term 18: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
W20260709 13:21:33.833328 16821 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.984s	user 0.018s	sys 0.076s
I20260709 13:21:33.860854 16296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2794af5b99984efebdee564035841cb0" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
W20260709 13:21:33.834571 21801 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.902s	user 0.008s	sys 0.012s
I20260709 13:21:33.871268 16294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fffab26427e24d398e2695aad40ca134" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:33.873034 16296 raft_consensus.cc:2393] T 2794af5b99984efebdee564035841cb0 P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 in current term 13: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
I20260709 13:21:33.866753 16292 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48684 (request call id 2912) took 2254 ms. Trace:
W20260709 13:21:33.873526 16295 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.207s	user 0.037s	sys 0.093s
I20260709 13:21:33.868441 16822 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49896 (request call id 1357) took 1694 ms. Trace:
I20260709 13:21:33.874416 16822 rpcz_store.cc:276] 0709 13:21:32.174475 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:32.174629 (+   154us) service_pool.cc:224] Handling call
0709 13:21:33.868252 (+1693623us) spinlock_profiling.cc:243] Waited 112 ms on lock 0x7b54006423d4. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:21:33.868377 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":256601344}
W20260709 13:21:33.874150 16295 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.208s	user 0.037s	sys 0.093s
I20260709 13:21:33.873688 16292 rpcz_store.cc:276] 0709 13:21:31.612499 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:32.167666 (+555167us) service_pool.cc:224] Handling call
0709 13:21:32.258845 (+ 91179us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:21:32.270104 (+ 11259us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:33.866485 (+1596381us) spinlock_profiling.cc:243] Waited 90.9 ms on lock 0x7b540024d950. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5e8ad 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:21:33.866678 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":209037568}
W20260709 13:21:33.868557 16821 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.019s	user 0.019s	sys 0.077s
I20260709 13:21:33.882300 16300 raft_consensus.cc:1240] T 2794af5b99984efebdee564035841cb0 P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 12. Current term is 13. Ops: []
W20260709 13:21:33.882632 16293 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.989s	user 0.004s	sys 0.030s
W20260709 13:21:33.882912 16293 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.989s	user 0.004s	sys 0.030s
I20260709 13:21:33.877247 16295 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48674 (request call id 3073) took 1996 ms. Trace:
I20260709 13:21:33.890481 16295 rpcz_store.cc:276] 0709 13:21:31.881113 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:32.272291 (+391178us) service_pool.cc:224] Handling call
0709 13:21:32.316578 (+ 44287us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:21:32.317050 (+   472us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:21:32.317091 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:21:32.661095 (+344004us) log.cc:844] Serialized 26 byte log entry
0709 13:21:33.876706 (+1215611us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:21:33.876772 (+    66us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:33.876890 (+   118us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:21:33.876968 (+    78us) raft_consensus.cc:1685] finished
0709 13:21:33.876990 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:21:33.877168 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":295,"raft.run_cpu_time_us":106,"raft.run_wall_time_us":21745,"thread_start_us":1209959,"threads_started":1,"wal-append.queue_time_us":1149844}
I20260709 13:21:33.866647 21946 raft_consensus.cc:515] T c2f5f89186204242bf377f7c42b2e2da P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:33.887256 21623 raft_consensus.cc:2804] T af57996a8be84d398f0f023e8d08ef77 P cb82cc7bd6db46d980c75237a1402860 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:21:33.829916 21725 raft_consensus.cc:515] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:21:33.901438 21569 consensus_meta.cc:287] T 57a744f5aaf944678a63469cbf514015 P 647a343a542d4a139fe106c19095129b: Time spent flushing consensus metadata: real 0.880s	user 0.012s	sys 0.000s
I20260709 13:21:33.898110 21867 raft_consensus.cc:493] T 3884f864a96d428785895e5463e5728f P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:33.878479 21930 consensus_queue.cc:1059] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: INVALID_TERM, Last received: 12.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.023s
I20260709 13:21:33.907457 21623 raft_consensus.cc:493] T af57996a8be84d398f0f023e8d08ef77 P cb82cc7bd6db46d980c75237a1402860 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:33.919948 21555 leader_election.cc:290] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:21:33.931981 21623 raft_consensus.cc:3060] T af57996a8be84d398f0f023e8d08ef77 P cb82cc7bd6db46d980c75237a1402860 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:21:33.918856 21569 raft_consensus.cc:515] T 57a744f5aaf944678a63469cbf514015 P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:33.922994 21867 raft_consensus.cc:515] T 3884f864a96d428785895e5463e5728f P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:33.964859 21891 leader_election.cc:290] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 14 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:33.950022 21245 raft_consensus.cc:515] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:21:33.971112 21843 consensus_meta.cc:287] T 31969392ccdb43b491911a319e8bc7e1 P 90f3da57796b488c9087a2f523adf045: Time spent flushing consensus metadata: real 1.133s	user 0.009s	sys 0.000s
I20260709 13:21:33.999655 21843 raft_consensus.cc:515] T 31969392ccdb43b491911a319e8bc7e1 P 90f3da57796b488c9087a2f523adf045 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:33.976665 21742 leader_election.cc:290] T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:33.979460 21725 leader_election.cc:290] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:33.980301 21946 leader_election.cc:290] T c2f5f89186204242bf377f7c42b2e2da P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:33.980827 21949 leader_election.cc:290] T 8c11074579b046d99e585c2818e18a2c P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:33.989149 21569 leader_election.cc:290] T 57a744f5aaf944678a63469cbf514015 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 12 election: Requested vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:33.985700 21867 leader_election.cc:290] T 3884f864a96d428785895e5463e5728f P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:34.007930 21736 raft_consensus.cc:515] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:33.992002 21245 leader_election.cc:290] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 19 election: Requested vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:33.996031 21934 raft_consensus.cc:3060] T 84f2da98864b4425bf7c0055951fdd4b P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:21:34.012578 21904 raft_consensus.cc:515] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:33.989966 21847 raft_consensus.cc:515] T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:33.991478 21704 leader_election.cc:290] T ecc1e363230043fe9fef390f0b979f20 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
W20260709 13:21:33.983912 21979 log.cc:927] Time spent T 8638ee363273426db1393dd81e5c7b72 P cb82cc7bd6db46d980c75237a1402860: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
I20260709 13:21:34.016217 21847 leader_election.cc:290] T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 16 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:33.998803 21854 raft_consensus.cc:515] T 66ae176961b74813a58bc15c9c4c6f2a P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:34.024567 21904 leader_election.cc:290] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 20 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
W20260709 13:21:34.005851 21908 consensus_meta.cc:287] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045: Time spent flushing consensus metadata: real 0.920s	user 0.009s	sys 0.000s
I20260709 13:21:33.800099 16047 catalog_manager.cc:5697] T 02838c86531d4f98bf58d1c22b8baf77 P 1ca2725d290e47d68b1a4b609c9816ac reported cstate change: term changed from 8 to 11, leader changed from 647a343a542d4a139fe106c19095129b (127.14.177.129) to 90f3da57796b488c9087a2f523adf045 (127.14.177.131). New cstate: current_term: 11 leader_uuid: "90f3da57796b488c9087a2f523adf045" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } }
I20260709 13:21:34.131951 16047 catalog_manager.cc:5697] T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac reported cstate change: term changed from 1 to 11, leader changed from 90f3da57796b488c9087a2f523adf045 (127.14.177.131) to 647a343a542d4a139fe106c19095129b (127.14.177.129). New cstate: current_term: 11 leader_uuid: "647a343a542d4a139fe106c19095129b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } }
I20260709 13:21:34.138541 16047 catalog_manager.cc:5697] T 18ad7e3af5504571a69d10c00750a01c P 1ca2725d290e47d68b1a4b609c9816ac reported cstate change: term changed from 6 to 11, leader changed from 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130) to 90f3da57796b488c9087a2f523adf045 (127.14.177.131). New cstate: current_term: 11 leader_uuid: "90f3da57796b488c9087a2f523adf045" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } }
I20260709 13:21:34.141470 16047 catalog_manager.cc:5697] T 7c2095a5f4ad49408dbc7f69c606120d P 1ca2725d290e47d68b1a4b609c9816ac reported cstate change: term changed from 7 to 9. New cstate: current_term: 9 leader_uuid: "647a343a542d4a139fe106c19095129b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } }
W20260709 13:21:34.146605 16142 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.062s	user 0.006s	sys 0.019s
W20260709 13:21:34.147261 16142 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.063s	user 0.007s	sys 0.019s
I20260709 13:21:34.150309 16148 raft_consensus.cc:2393] T 9c61a39fd4f84424b129c1a6f9ba404d P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 17: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
I20260709 13:21:34.150751 16149 raft_consensus.cc:1240] T 57a744f5aaf944678a63469cbf514015 P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 11. Current term is 12. Ops: []
I20260709 13:21:34.156656 16144 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46224 (request call id 927) took 1220 ms. Trace:
I20260709 13:21:34.156275 16047 catalog_manager.cc:5697] T fffab26427e24d398e2695aad40ca134 P 1ca2725d290e47d68b1a4b609c9816ac reported cstate change: term changed from 6 to 9, leader changed from 90f3da57796b488c9087a2f523adf045 (127.14.177.131) to 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130). New cstate: current_term: 9 leader_uuid: "1ca2725d290e47d68b1a4b609c9816ac" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } health_report { overall_health: HEALTHY } } }
I20260709 13:21:34.157208 16144 rpcz_store.cc:276] 0709 13:21:32.936454 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:33.462977 (+526523us) service_pool.cc:224] Handling call
0709 13:21:33.463460 (+   483us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:21:33.807652 (+344192us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:21:33.807686 (+    34us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:21:33.807732 (+    46us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:21:33.807766 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:34.155934 (+348168us) spinlock_profiling.cc:243] Waited 344 ms on lock 0x7b54000749d4. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:21:34.156037 (+   103us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:21:34.156392 (+   355us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":791076352}
I20260709 13:21:34.163748 16145 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:40290 (request call id 33) took 1229 ms. Trace:
I20260709 13:21:34.163323 16047 catalog_manager.cc:5697] T 84f2da98864b4425bf7c0055951fdd4b P 1ca2725d290e47d68b1a4b609c9816ac reported cstate change: term changed from 1 to 9. New cstate: current_term: 9 leader_uuid: "1ca2725d290e47d68b1a4b609c9816ac" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } health_report { overall_health: HEALTHY } } }
I20260709 13:21:34.164327 16145 rpcz_store.cc:276] 0709 13:21:32.933698 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:32.935246 (+  1548us) service_pool.cc:224] Handling call
0709 13:21:34.163209 (+1227963us) spinlock_profiling.cc:243] Waited 18.1 ms on lock 0x7b4c0043fd80. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69aee2ba20 00007f69aee3ca29 00007f69aee02644 00007f69aee29f2d 00007f69aee29b93 00007f69b0b59c92 00007f69b0b5984a 00007f69b5b2760a 00007f69af0ad7d4 00007f69af0ad762 00007f69af0ad691
0709 13:21:34.163441 (+   232us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":41671040}
I20260709 13:21:34.164690 16147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c2095a5f4ad49408dbc7f69c606120d" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:21:34.166262 16147 raft_consensus.cc:2393] T 7c2095a5f4ad49408dbc7f69c606120d P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 12: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
I20260709 13:21:34.166419 16145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a959dbecc2648eb85d1cc550f5a3245" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:34.179054 16147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4f65cdf00b748bb963ee6e81ef43b79" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:34.241321 16299 raft_consensus.cc:2393] T 66ae176961b74813a58bc15c9c4c6f2a P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 in current term 20: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
I20260709 13:21:34.175926 16144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18ad7e3af5504571a69d10c00750a01c" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
W20260709 13:21:34.211336 16265 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.640s	user 0.002s	sys 0.002s
I20260709 13:21:34.170070 16142 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40844 (request call id 2678) took 1885 ms. Trace:
I20260709 13:21:34.171018 16149 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46224 (request call id 920) took 1237 ms. Trace:
W20260709 13:21:34.196775 16143 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40844 (request call id 2680) took 1911 ms (client timeout 1775 ms). Trace:
I20260709 13:21:34.212844 16293 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40474 (request call id 1002) took 2097 ms. Trace:
I20260709 13:21:34.292663 16823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74ca8140f35c4ae29e8a664f6bdfc829" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
W20260709 13:21:34.184633 16223 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.161s	user 0.000s	sys 0.027s
W20260709 13:21:34.181264 16190 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.309s	user 0.006s	sys 0.022s
W20260709 13:21:34.183354 16226 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33050 (request call id 990) took 1888 ms (client timeout 2000 ms). Trace:
I20260709 13:21:34.286966 16817 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49892 (request call id 860) took 1427 ms. Trace:
W20260709 13:21:34.184118 16220 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.285s	user 0.020s	sys 0.010s
W20260709 13:21:34.261754 16791 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:34.399048 16145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a9de347c711477e87c582d003f36925" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
W20260709 13:21:34.400614 16265 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.829s	user 0.008s	sys 0.005s
W20260709 13:21:34.287329 16820 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.526s	user 0.000s	sys 0.008s
W20260709 13:21:34.398415 16143 rpcz_store.cc:269] 0709 13:21:32.285404 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:32.704904 (+419500us) service_pool.cc:224] Handling call
0709 13:21:34.196245 (+1491341us) spinlock_profiling.cc:243] Waited 253 ms on lock 0x7b540028e954. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:21:34.196560 (+   315us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":580923264}
W20260709 13:21:34.168787 16148 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40844 (request call id 2676) took 1884 ms (client timeout 2100 ms). Trace:
I20260709 13:21:34.303393 16825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e65676b21b748728ad39668cfe722ec" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:34.291888 16824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7ca61592e848ea9b0fb81a2ce33cdc" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:34.285940 16816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31969392ccdb43b491911a319e8bc7e1" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:21:34.396032 16149 rpcz_store.cc:276] 0709 13:21:32.934005 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:32.934120 (+   115us) service_pool.cc:224] Handling call
0709 13:21:32.934457 (+   337us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:21:34.151533 (+1217076us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:34.170721 (+ 19188us) spinlock_profiling.cc:243] Waited 1.22 s on lock 0x7b54001f1dd4. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:21:34.170954 (+   233us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2795764608}
I20260709 13:21:34.212888 16295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e9b2bf7011f4ae688a79d7bb5a447b4" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
W20260709 13:21:34.422626 16190 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.551s	user 0.013s	sys 0.041s
I20260709 13:21:34.286001 16818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3884f864a96d428785895e5463e5728f" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:34.286859 16819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c11074579b046d99e585c2818e18a2c" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:34.185498 16224 raft_consensus.cc:2468] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 20.
I20260709 13:21:34.223796 16298 raft_consensus.cc:2468] T 31969392ccdb43b491911a319e8bc7e1 P 90f3da57796b488c9087a2f523adf045 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 14.
W20260709 13:21:34.443773 16220 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.545s	user 0.020s	sys 0.027s
W20260709 13:21:34.428398 16223 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.405s	user 0.000s	sys 0.044s
W20260709 13:21:34.416693 16148 rpcz_store.cc:269] 0709 13:21:32.284318 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:32.291117 (+  6799us) service_pool.cc:224] Handling call
0709 13:21:34.168481 (+1877364us) spinlock_profiling.cc:243] Waited 1.45 s on lock 0x7b54000d0f54. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:21:34.168694 (+   213us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3333249408}
W20260709 13:21:34.428748 16820 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.667s	user 0.005s	sys 0.012s
W20260709 13:21:34.283416 16115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 134503us
W20260709 13:21:34.259063 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 452004us
I20260709 13:21:34.350502 21854 leader_election.cc:290] T 66ae176961b74813a58bc15c9c4c6f2a P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 20 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
W20260709 13:21:34.402832 16226 rpcz_store.cc:269] 0709 13:21:32.295446 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:32.841708 (+546262us) service_pool.cc:224] Handling call
0709 13:21:34.182741 (+1341033us) spinlock_profiling.cc:243] Waited 35 ms on lock 0x7b54002b4654. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:21:34.182998 (+   257us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":80428928}
I20260709 13:21:34.399031 16293 rpcz_store.cc:276] 0709 13:21:32.115103 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:32.662953 (+547850us) service_pool.cc:224] Handling call
0709 13:21:32.698737 (+ 35784us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:21:32.703764 (+  5027us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:21:32.703796 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:21:32.774182 (+ 70386us) log.cc:844] Serialized 26 byte log entry
0709 13:21:33.883286 (+1109104us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:21:33.883320 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:33.883377 (+    57us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:21:34.212083 (+328706us) raft_consensus.cc:1685] finished
0709 13:21:34.212136 (+    53us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:21:34.212545 (+   409us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":610,"raft.queue_time_us":2106,"raft.run_cpu_time_us":106,"raft.run_wall_time_us":68686,"thread_start_us":989373,"threads_started":1,"wal-append.queue_time_us":1084332}
W20260709 13:21:34.452414 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 1.70349e+06us
I20260709 13:21:34.465296 16823 raft_consensus.cc:3060] T 74ca8140f35c4ae29e8a664f6bdfc829 P cb82cc7bd6db46d980c75237a1402860 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:21:34.465868 16145 raft_consensus.cc:2393] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 19: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
W20260709 13:21:34.471359 16225 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.876s	user 0.010s	sys 0.004s
W20260709 13:21:34.471928 16225 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.899s	user 0.010s	sys 0.004s
I20260709 13:21:34.318490 21623 raft_consensus.cc:515] T af57996a8be84d398f0f023e8d08ef77 P cb82cc7bd6db46d980c75237a1402860 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:34.348698 21908 raft_consensus.cc:515] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:34.471387 16295 raft_consensus.cc:2468] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 13.
I20260709 13:21:34.319538 21736 leader_election.cc:290] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 19 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:34.395771 16142 rpcz_store.cc:276] 0709 13:21:32.284865 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:32.284963 (+    98us) service_pool.cc:224] Handling call
0709 13:21:33.082679 (+797716us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:21:33.083101 (+   422us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:21:33.083134 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:21:33.083654 (+   520us) log.cc:844] Serialized 26 byte log entry
0709 13:21:34.148372 (+1064718us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:21:34.148519 (+   147us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:34.148715 (+   196us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:21:34.148825 (+   110us) raft_consensus.cc:1685] finished
0709 13:21:34.148861 (+    36us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:21:34.169718 (+ 20857us) spinlock_profiling.cc:243] Waited 797 ms on lock 0x7b54000e9050. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5e8ad 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:21:34.169982 (+   264us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1133,"raft.run_cpu_time_us":169,"raft.run_wall_time_us":51204,"spinlock_wait_cycles":1833799552,"thread_start_us":1063745,"threads_started":1,"wal-append.queue_time_us":848548}
W20260709 13:21:34.481546 16291 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.522s	user 0.014s	sys 0.011s
W20260709 13:21:34.482110 16291 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.562s	user 0.014s	sys 0.011s
I20260709 13:21:34.483218 16225 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33050 (request call id 985) took 2262 ms. Trace:
W20260709 13:21:34.483376 21931 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.614s	user 0.004s	sys 0.008s
I20260709 13:21:34.002945 21843 leader_election.cc:290] T 31969392ccdb43b491911a319e8bc7e1 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 14 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:34.483831 16825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5718c623ebee4d2bb770f0f9ddb2f164" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:34.494379 16825 raft_consensus.cc:2468] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 15.
W20260709 13:21:34.495118 16298 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40474 (request call id 1003) took 2378 ms (client timeout 1881 ms). Trace:
I20260709 13:21:34.495774 16825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9944d5e4b8c4109af9955c4f08c9095" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
W20260709 13:21:34.495684 16298 rpcz_store.cc:269] 0709 13:21:32.117016 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:32.133575 (+ 16559us) service_pool.cc:224] Handling call
0709 13:21:34.494869 (+2361294us) spinlock_profiling.cc:243] Waited 2.04 s on lock 0x7b54001eee54. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:21:34.495015 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4687406976}
I20260709 13:21:34.482841 16190 leader_election.cc:304] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
W20260709 13:21:34.504415 16191 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.753s	user 0.003s	sys 0.008s
W20260709 13:21:34.504853 16116 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.689s	user 0.007s	sys 0.004s
W20260709 13:21:34.505273 16116 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.742s	user 0.007s	sys 0.004s
I20260709 13:21:34.505155 16294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c2095a5f4ad49408dbc7f69c606120d" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:34.506333 16294 raft_consensus.cc:2393] T 7c2095a5f4ad49408dbc7f69c606120d P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 12: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
I20260709 13:21:34.507083 21946 raft_consensus.cc:2804] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:21:34.507504 21245 raft_consensus.cc:493] T ee9ed67006144797b1de6ea8c2013a68 P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:34.507730 21946 raft_consensus.cc:493] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Starting leader election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:21:34.508325 21946 raft_consensus.cc:3060] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:21:34.508517 21245 raft_consensus.cc:515] T ee9ed67006144797b1de6ea8c2013a68 P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:21:34.507268 16146 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.704s	user 0.001s	sys 0.018s
W20260709 13:21:34.511663 16146 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.765s	user 0.001s	sys 0.018s
I20260709 13:21:34.513902 16790 leader_election.cc:304] T 8638ee363273426db1393dd81e5c7b72 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 90f3da57796b488c9087a2f523adf045
I20260709 13:21:34.515386 21920 raft_consensus.cc:2804] T 8638ee363273426db1393dd81e5c7b72 P cb82cc7bd6db46d980c75237a1402860 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:21:34.515904 21920 raft_consensus.cc:493] T 8638ee363273426db1393dd81e5c7b72 P cb82cc7bd6db46d980c75237a1402860 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:21:34.516434 21920 raft_consensus.cc:3060] T 8638ee363273426db1393dd81e5c7b72 P cb82cc7bd6db46d980c75237a1402860 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:21:34.506171 16190 leader_election.cc:304] T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:21:34.520212 21904 raft_consensus.cc:2804] T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:21:34.520795 21904 raft_consensus.cc:493] T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:34.521201 21904 raft_consensus.cc:3060] T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:21:34.498579 21931 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.706s	user 0.004s	sys 0.009s
I20260709 13:21:34.170612 16047 catalog_manager.cc:5697] T d16dfad0c3b540e5b4b86492ddf87208 P 1ca2725d290e47d68b1a4b609c9816ac reported cstate change: term changed from 7 to 12, leader changed from 647a343a542d4a139fe106c19095129b (127.14.177.129) to 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130). New cstate: current_term: 12 leader_uuid: "1ca2725d290e47d68b1a4b609c9816ac" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } health_report { overall_health: UNKNOWN } } }
I20260709 13:21:34.529096 16047 catalog_manager.cc:5697] T 8cc959fb7f704fba9009c0595467d074 P 1ca2725d290e47d68b1a4b609c9816ac reported cstate change: term changed from 8 to 11, leader changed from 647a343a542d4a139fe106c19095129b (127.14.177.129) to 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130). New cstate: current_term: 11 leader_uuid: "1ca2725d290e47d68b1a4b609c9816ac" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } health_report { overall_health: HEALTHY } } }
I20260709 13:21:34.532073 21579 raft_consensus.cc:493] T 434bff3082d141838068f323bdb0d1ed P cb82cc7bd6db46d980c75237a1402860 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:21:34.291796 16822 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0a9de347c711477e87c582d003f36925"
dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
mode: GRACEFUL
new_leader_uuid: "90f3da57796b488c9087a2f523adf045"
 from {username='slave'} at 127.0.0.1:37818
I20260709 13:21:34.533601 21996 consensus_queue.cc:1048] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:21:34.535492 16190 leader_election.cc:304] T d16dfad0c3b540e5b4b86492ddf87208 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
W20260709 13:21:34.538715 22009 log.cc:927] Time spent T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 0.475s	user 0.011s	sys 0.003s
I20260709 13:21:34.403347 16817 rpcz_store.cc:276] 0709 13:21:32.859580 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:32.859728 (+   148us) service_pool.cc:224] Handling call
0709 13:21:32.860171 (+   443us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:21:32.860646 (+   475us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:21:32.860688 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:21:32.861240 (+   552us) log.cc:844] Serialized 26 byte log entry
0709 13:21:33.756553 (+895313us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:21:33.756602 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:33.756680 (+    78us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:21:34.286310 (+529630us) raft_consensus.cc:1685] finished
0709 13:21:34.286357 (+    47us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:21:34.286692 (+   335us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1237,"raft.run_cpu_time_us":136,"raft.run_wall_time_us":12068,"thread_start_us":894887,"threads_started":1,"wal-append.queue_time_us":1033535}
I20260709 13:21:34.544021 16221 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57604 (request call id 2984) took 2377 ms. Trace:
I20260709 13:21:34.548137 16221 rpcz_store.cc:276] 0709 13:21:32.166141 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:32.881455 (+715314us) service_pool.cc:224] Handling call
0709 13:21:32.881789 (+   334us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:21:32.882024 (+   235us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:21:33.816223 (+934199us) log.cc:844] Serialized 14 byte log entry
0709 13:21:34.427843 (+611620us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:21:34.455732 (+ 27889us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:21:34.486046 (+ 30314us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:34.543647 (+ 57601us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:21:34.543818 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10565,"prepare.queue_time_us":148008,"prepare.run_cpu_time_us":37,"prepare.run_wall_time_us":37,"raft.queue_time_us":211789,"raft.run_cpu_time_us":460,"raft.run_wall_time_us":28402,"thread_start_us":1269697,"threads_started":2,"wal-append.queue_time_us":135379}
W20260709 13:21:34.504904 16191 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.811s	user 0.003s	sys 0.008s
W20260709 13:21:34.550729 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 857351us
W20260709 13:21:34.551314 16150 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.046s	user 0.013s	sys 0.010s
W20260709 13:21:34.551806 16150 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.140s	user 0.013s	sys 0.011s
I20260709 13:21:34.551892 16191 leader_election.cc:304] T c0eab25ce4c64e018a041beeda2867db P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:21:34.553409 21996 raft_consensus.cc:2749] T c0eab25ce4c64e018a041beeda2867db P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:21:34.555601 21245 leader_election.cc:290] T ee9ed67006144797b1de6ea8c2013a68 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:34.483774 16225 rpcz_store.cc:276] 0709 13:21:32.220400 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:33.003200 (+782800us) service_pool.cc:224] Handling call
0709 13:21:33.016463 (+ 13263us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:21:33.016713 (+   250us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 13:21:33.572913 (+556200us) log.cc:844] Serialized 14 byte log entry
0709 13:21:34.482504 (+909591us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:21:34.482645 (+   141us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:21:34.482763 (+   118us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:34.482906 (+   143us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:21:34.483098 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":9787,"prepare.queue_time_us":13823,"prepare.run_cpu_time_us":28,"prepare.run_wall_time_us":26,"thread_start_us":1454987,"threads_started":2}
I20260709 13:21:34.545410 21941 raft_consensus.cc:2804] T d16dfad0c3b540e5b4b86492ddf87208 P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:21:34.556587 21941 raft_consensus.cc:493] T d16dfad0c3b540e5b4b86492ddf87208 P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: Starting leader election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:21:34.557160 21941 raft_consensus.cc:3060] T d16dfad0c3b540e5b4b86492ddf87208 P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:21:34.532845 21579 raft_consensus.cc:515] T 434bff3082d141838068f323bdb0d1ed P cb82cc7bd6db46d980c75237a1402860 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:34.461051 21704 raft_consensus.cc:493] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:34.559724 21704 raft_consensus.cc:515] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } }
I20260709 13:21:34.562004 21946 raft_consensus.cc:515] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:34.533998 16820 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49892 (request call id 857) took 1757 ms. Trace:
I20260709 13:21:34.554697 16191 leader_election.cc:304] T 08c74490c40243a88e62d1760b5cd814 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:21:34.478144 21908 leader_election.cc:290] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 16 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:34.557217 16819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af57996a8be84d398f0f023e8d08ef77" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:34.518769 16790 leader_election.cc:304] T 829b4ad8c5a54eababeea1181b960d4b P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 647a343a542d4a139fe106c19095129b
I20260709 13:21:34.585980 16819 raft_consensus.cc:2393] T af57996a8be84d398f0f023e8d08ef77 P cb82cc7bd6db46d980c75237a1402860 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 17: Already voted for candidate cb82cc7bd6db46d980c75237a1402860 in this term.
I20260709 13:21:34.587244 16819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:34.588855 16819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5718c623ebee4d2bb770f0f9ddb2f164" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:34.589551 16819 raft_consensus.cc:2468] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 15.
I20260709 13:21:34.590444 16819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62fe3346cd3c42c4a55e14af9073bca4" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
W20260709 13:21:34.591301 16151 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.755s	user 0.011s	sys 0.019s
W20260709 13:21:34.591770 16151 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.849s	user 0.011s	sys 0.019s
I20260709 13:21:34.591006 16819 raft_consensus.cc:3060] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:21:34.598776 16818 raft_consensus.cc:2468] T 3884f864a96d428785895e5463e5728f P cb82cc7bd6db46d980c75237a1402860 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 11.
I20260709 13:21:34.552280 21904 raft_consensus.cc:515] T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:34.604370 16223 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57604 (request call id 2989) took 2436 ms. Trace:
I20260709 13:21:34.568287 16820 rpcz_store.cc:276] 0709 13:21:32.776391 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:32.778800 (+  2409us) service_pool.cc:224] Handling call
0709 13:21:32.779076 (+   276us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:21:32.779297 (+   221us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 13:21:33.761294 (+981997us) log.cc:844] Serialized 14 byte log entry
0709 13:21:34.531606 (+770312us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:21:34.531719 (+   113us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:21:34.531823 (+   104us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:34.533740 (+  1917us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:21:34.533924 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":69842,"prepare.queue_time_us":1128352,"prepare.run_cpu_time_us":42,"prepare.run_wall_time_us":44,"raft.queue_time_us":14373,"raft.run_cpu_time_us":178,"raft.run_wall_time_us":178,"thread_start_us":1590816,"threads_started":2,"wal-append.queue_time_us":798032}
I20260709 13:21:34.643988 21891 raft_consensus.cc:2749] T 08c74490c40243a88e62d1760b5cd814 P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:21:34.644546 16150 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46224 (request call id 919) took 3187 ms. Trace:
I20260709 13:21:34.645061 16150 rpcz_store.cc:276] 0709 13:21:31.456992 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:31.457153 (+   161us) service_pool.cc:224] Handling call
0709 13:21:31.457532 (+   379us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:21:32.107661 (+650129us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 13:21:33.411552 (+1303891us) log.cc:844] Serialized 14 byte log entry
0709 13:21:34.552995 (+1141443us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:21:34.553125 (+   130us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:21:34.553226 (+   101us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:34.644160 (+ 90934us) spinlock_profiling.cc:243] Waited 650 ms on lock 0x7b54003d9ad4. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:21:34.644223 (+    63us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:21:34.644442 (+   219us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":1632694,"prepare.run_cpu_time_us":63,"prepare.run_wall_time_us":2841,"raft.queue_time_us":140,"raft.run_cpu_time_us":183,"raft.run_wall_time_us":182,"spinlock_wait_cycles":1494632192,"thread_start_us":2443971,"threads_started":2,"wal-append.queue_time_us":1146052}
W20260709 13:21:34.653611 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 99217.9us
I20260709 13:21:34.658450 16820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c11074579b046d99e585c2818e18a2c" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:21:34.606520 16223 rpcz_store.cc:276] 0709 13:21:32.167456 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:32.893531 (+726075us) service_pool.cc:224] Handling call
0709 13:21:32.893899 (+   368us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:21:32.896832 (+  2933us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:21:32.896873 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:21:32.907747 (+ 10874us) log.cc:844] Serialized 26 byte log entry
0709 13:21:34.597720 (+1689973us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:21:34.597786 (+    66us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:34.597903 (+   117us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:21:34.600597 (+  2694us) raft_consensus.cc:1685] finished
0709 13:21:34.600610 (+    13us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:21:34.604297 (+  3687us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2493,"raft.queue_time_us":457637,"raft.run_cpu_time_us":148,"raft.run_wall_time_us":14733,"thread_start_us":1424393,"threads_started":1,"wal-append.queue_time_us":957573}
I20260709 13:21:34.691301 16825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8638ee363273426db1393dd81e5c7b72" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:34.648494 21623 leader_election.cc:290] T af57996a8be84d398f0f023e8d08ef77 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 17 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:21:34.692682 16818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a378d7f796f94615b3b347119a9880fd" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:21:34.687918 22011 raft_consensus.cc:2804] T 829b4ad8c5a54eababeea1181b960d4b P cb82cc7bd6db46d980c75237a1402860 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:21:34.655985 21908 raft_consensus.cc:493] T 1df93779209748b9860f6d893dd3de53 P 90f3da57796b488c9087a2f523adf045 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:34.657749 21704 leader_election.cc:290] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:34.699450 16821 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49892 (request call id 854) took 1881 ms. Trace:
I20260709 13:21:34.718437 16824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e9b2bf7011f4ae688a79d7bb5a447b4" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
W20260709 13:21:34.734628 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 199447us
I20260709 13:21:34.699995 22011 raft_consensus.cc:493] T 829b4ad8c5a54eababeea1181b960d4b P cb82cc7bd6db46d980c75237a1402860 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 647a343a542d4a139fe106c19095129b)
W20260709 13:21:34.694185 16117 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.717s	user 0.015s	sys 0.034s
I20260709 13:21:34.746510 16823 raft_consensus.cc:2468] T 74ca8140f35c4ae29e8a664f6bdfc829 P cb82cc7bd6db46d980c75237a1402860 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 16.
I20260709 13:21:34.748409 16824 raft_consensus.cc:1240] T 2794af5b99984efebdee564035841cb0 P cb82cc7bd6db46d980c75237a1402860 [term 13 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 12. Current term is 13. Ops: []
W20260709 13:21:34.749079 16263 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:34.749440 16191 leader_election.cc:304] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:21:34.768178 21930 raft_consensus.cc:2749] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
W20260709 13:21:34.768546 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 67708.9us
I20260709 13:21:34.769955 16191 leader_election.cc:304] T 66f75124215f4ca6bef831e174a792d8 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
I20260709 13:21:34.771420 21930 raft_consensus.cc:2804] T 66f75124215f4ca6bef831e174a792d8 P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:21:34.772033 21930 raft_consensus.cc:493] T 66f75124215f4ca6bef831e174a792d8 P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Starting leader election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:21:34.772534 21930 raft_consensus.cc:3060] T 66f75124215f4ca6bef831e174a792d8 P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:21:34.775620 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f82224f4564fea93572e8206bad1fe" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:34.776602 16293 raft_consensus.cc:2468] T 23f82224f4564fea93572e8206bad1fe P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 12.
W20260709 13:21:34.756909 22016 log.cc:927] Time spent T ecc1e363230043fe9fef390f0b979f20 P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.221s	user 0.002s	sys 0.000s
I20260709 13:21:34.760787 21903 raft_consensus.cc:3060] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:21:34.761711 16817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66f75124215f4ca6bef831e174a792d8" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:34.764575 16822 raft_consensus.cc:606] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860 [term 18 FOLLOWER]: Received request to transfer leadership to TS 90f3da57796b488c9087a2f523adf045
W20260709 13:21:34.765923 21998 log.cc:927] Time spent T 84f2da98864b4425bf7c0055951fdd4b P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 0.245s	user 0.001s	sys 0.005s
W20260709 13:21:34.768369 22003 log.cc:927] Time spent T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 0.245s	user 0.000s	sys 0.005s
I20260709 13:21:34.780471 16819 raft_consensus.cc:2468] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 21.
I20260709 13:21:34.801028 16818 raft_consensus.cc:2393] T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 18: Already voted for candidate cb82cc7bd6db46d980c75237a1402860 in this term.
I20260709 13:21:34.754443 21579 leader_election.cc:290] T 434bff3082d141838068f323bdb0d1ed P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
W20260709 13:21:34.693600 21769 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.531s	user 0.023s	sys 0.014s
I20260709 13:21:34.774767 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31969392ccdb43b491911a319e8bc7e1" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:34.723127 16821 rpcz_store.cc:276] 0709 13:21:32.775483 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:32.777419 (+  1936us) service_pool.cc:224] Handling call
0709 13:21:32.777738 (+   319us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:21:32.778118 (+   380us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:21:33.877824 (+1099706us) log.cc:844] Serialized 14 byte log entry
0709 13:21:34.354651 (+476827us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:21:34.471527 (+116876us) log.cc:844] Serialized 26 byte log entry
0709 13:21:34.471815 (+   288us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:21:34.471862 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:34.471994 (+   132us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:21:34.640469 (+168475us) raft_consensus.cc:1685] finished
0709 13:21:34.640490 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:21:34.656487 (+ 15997us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1771,"prepare.queue_time_us":1118984,"prepare.run_cpu_time_us":47,"prepare.run_wall_time_us":48,"raft.queue_time_us":61452,"raft.run_cpu_time_us":497,"raft.run_wall_time_us":500,"thread_start_us":1428550,"threads_started":2,"wal-append.queue_time_us":592278}
W20260709 13:21:34.779219 22013 log.cc:927] Time spent T c0eab25ce4c64e018a041beeda2867db P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.267s	user 0.003s	sys 0.002s
W20260709 13:21:34.795979 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 277370us
I20260709 13:21:34.784487 16822 raft_consensus.cc:614] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860 [term 18 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 13:21:34.817327 16117 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.036s	user 0.016s	sys 0.035s
I20260709 13:21:34.746184 22011 raft_consensus.cc:3060] T 829b4ad8c5a54eababeea1181b960d4b P cb82cc7bd6db46d980c75237a1402860 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:21:34.790647 21941 raft_consensus.cc:515] T d16dfad0c3b540e5b4b86492ddf87208 P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:34.831153 16298 raft_consensus.cc:2410] T 31969392ccdb43b491911a319e8bc7e1 P 90f3da57796b488c9087a2f523adf045 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b for term 15 because replica has last-logged OpId of term: 13 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 10.
I20260709 13:21:34.832252 16295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c61a39fd4f84424b129c1a6f9ba404d" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:21:34.835645 16821 raft_consensus.cc:1240] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 [term 19 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 18. Current term is 19. Ops: []
I20260709 13:21:34.837800 16292 raft_consensus.cc:1275] T 8d4ab870a05347868956aa8b841b2981 P 90f3da57796b488c9087a2f523adf045 [term 14 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:21:34.838828 16292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57a744f5aaf944678a63469cbf514015" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:21:34.839476 16292 raft_consensus.cc:3060] T 57a744f5aaf944678a63469cbf514015 P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:21:34.840689 16790 leader_election.cc:304] T ada5a004ba7c46819051e3ccec71ffdb P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 90f3da57796b488c9087a2f523adf045
I20260709 13:21:34.843842 21904 leader_election.cc:290] T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 17 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:34.847940 16299 raft_consensus.cc:1275] T 5a959dbecc2648eb85d1cc550f5a3245 P 90f3da57796b488c9087a2f523adf045 [term 14 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 14 index: 9. (term mismatch)
I20260709 13:21:34.848353 16299 pending_rounds.cc:85] T 5a959dbecc2648eb85d1cc550f5a3245 P 90f3da57796b488c9087a2f523adf045: Aborting all ops after (but not including) 8
I20260709 13:21:34.848615 16299 pending_rounds.cc:107] T 5a959dbecc2648eb85d1cc550f5a3245 P 90f3da57796b488c9087a2f523adf045: Aborting uncommitted NO_OP operation due to leader change: 13.9
I20260709 13:21:34.848874 16299 raft_consensus.cc:2889] T 5a959dbecc2648eb85d1cc550f5a3245 P 90f3da57796b488c9087a2f523adf045 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:21:34.856510 16191 leader_election.cc:304] T e5ad4eb281564936a59f41e387a47156 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:21:34.856868 21904 raft_consensus.cc:493] T 5a96373bb0e14dce8a266b2fc6c58507 P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:21:34.857409 21891 raft_consensus.cc:493] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:21:34.857501 21904 raft_consensus.cc:515] T 5a96373bb0e14dce8a266b2fc6c58507 P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:34.858075 16223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8158bbd17b994d2eb96f560bd0f1b88d" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:34.858151 21891 raft_consensus.cc:515] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:34.858803 16221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d16dfad0c3b540e5b4b86492ddf87208" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:34.859189 16223 raft_consensus.cc:2468] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 14.
I20260709 13:21:34.859653 16221 raft_consensus.cc:2393] T d16dfad0c3b540e5b4b86492ddf87208 P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 16: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
I20260709 13:21:34.859952 16116 leader_election.cc:304] T af57996a8be84d398f0f023e8d08ef77 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
I20260709 13:21:34.860636 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c2095a5f4ad49408dbc7f69c606120d" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:21:34.861168 16116 leader_election.cc:304] T fffab26427e24d398e2695aad40ca134 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
I20260709 13:21:34.861423 16224 raft_consensus.cc:3060] T 7c2095a5f4ad49408dbc7f69c606120d P 1ca2725d290e47d68b1a4b609c9816ac [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:21:34.863004 21245 consensus_queue.cc:1048] T 18ad7e3af5504571a69d10c00750a01c P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:21:34.863451 21997 raft_consensus.cc:2804] T af57996a8be84d398f0f023e8d08ef77 P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 13:21:34.863816 21934 consensus_meta.cc:287] T 84f2da98864b4425bf7c0055951fdd4b P 1ca2725d290e47d68b1a4b609c9816ac: Time spent flushing consensus metadata: real 0.853s	user 0.016s	sys 0.001s
I20260709 13:21:34.864069 21997 raft_consensus.cc:493] T af57996a8be84d398f0f023e8d08ef77 P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:34.864576 21997 raft_consensus.cc:3060] T af57996a8be84d398f0f023e8d08ef77 P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:21:34.864639 21934 raft_consensus.cc:2749] T e5ad4eb281564936a59f41e387a47156 P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:21:34.866490 22002 log.cc:927] Time spent T 8c11074579b046d99e585c2818e18a2c P cb82cc7bd6db46d980c75237a1402860: Append to log took a long time: real 0.354s	user 0.004s	sys 0.002s
I20260709 13:21:34.868691 16293 raft_consensus.cc:1275] T 02838c86531d4f98bf58d1c22b8baf77 P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:21:34.869629 16217 raft_consensus.cc:1275] T 84f2da98864b4425bf7c0055951fdd4b P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:21:34.871366 16217 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57600 (request call id 1755) took 3745 ms. Trace:
I20260709 13:21:34.871810 16217 rpcz_store.cc:276] 0709 13:21:31.125575 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:31.982269 (+856694us) service_pool.cc:224] Handling call
0709 13:21:32.007348 (+ 25079us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:21:34.870123 (+2862775us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:34.870996 (+   873us) spinlock_profiling.cc:243] Waited 2.86 s on lock 0x7b5400396554. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:21:34.871275 (+   279us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6581289728}
I20260709 13:21:34.875473 21904 leader_election.cc:290] T 5a96373bb0e14dce8a266b2fc6c58507 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:34.876142 21891 leader_election.cc:290] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:34.876639 16291 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48674 (request call id 3078) took 1017 ms. Trace:
I20260709 13:21:34.877398 21941 leader_election.cc:290] T d16dfad0c3b540e5b4b86492ddf87208 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 16 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:34.878273 21996 consensus_queue.cc:1059] T 1df93779209748b9860f6d893dd3de53 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:21:34.877121 16291 rpcz_store.cc:276] 0709 13:21:33.858830 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:33.883278 (+ 24448us) service_pool.cc:224] Handling call
0709 13:21:33.883658 (+   380us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:21:33.884176 (+   518us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:21:33.884224 (+    48us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:21:33.920267 (+ 36043us) log.cc:844] Serialized 26 byte log entry
0709 13:21:34.482986 (+562719us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:21:34.483057 (+    71us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:34.483198 (+   141us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:21:34.778170 (+294972us) spinlock_profiling.cc:243] Waited 44.5 ms on lock 0x7b4400022b4c. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69ad8fadb1 00007f69ad998c85 00007f69ad9996a2 00007f69b070b1db 00007f69b0706aaf 00007f69b06fa86e 00007f69b06f9bb6 00007f69b06f97d8 00007f69b0703221 00007f69b07031d9 00007f69b0703169
0709 13:21:34.876230 (+ 98060us) raft_consensus.cc:1685] finished
0709 13:21:34.876267 (+    37us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:21:34.876549 (+   282us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":106,"raft.run_cpu_time_us":230,"raft.run_wall_time_us":4990,"spinlock_wait_cycles":102310272,"thread_start_us":562056,"threads_started":1,"wal-append.queue_time_us":591300}
I20260709 13:21:34.879356 21941 consensus_queue.cc:1059] T e4dc6b0541ca431ebc757beac00521f1 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: INVALID_TERM, Last received: 14.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:21:34.879716 21996 raft_consensus.cc:3055] T 1df93779209748b9860f6d893dd3de53 P 1ca2725d290e47d68b1a4b609c9816ac [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:21:34.880241 21996 raft_consensus.cc:740] T 1df93779209748b9860f6d893dd3de53 P 1ca2725d290e47d68b1a4b609c9816ac [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:21:34.880584 21941 raft_consensus.cc:3055] T e4dc6b0541ca431ebc757beac00521f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:21:34.881007 21941 raft_consensus.cc:740] T e4dc6b0541ca431ebc757beac00521f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:21:34.881304 21996 consensus_queue.cc:260] T 1df93779209748b9860f6d893dd3de53 P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:34.882026 21941 consensus_queue.cc:260] T e4dc6b0541ca431ebc757beac00521f1 P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.7, Last appended by leader: 7, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:34.882789 21996 raft_consensus.cc:3060] T 1df93779209748b9860f6d893dd3de53 P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:21:34.884145 16225 raft_consensus.cc:1275] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:21:34.884948 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74ca8140f35c4ae29e8a664f6bdfc829" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:34.885289 16224 raft_consensus.cc:2468] T 7c2095a5f4ad49408dbc7f69c606120d P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 12.
I20260709 13:21:34.826902 16296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c2095a5f4ad49408dbc7f69c606120d" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:21:34.846701 16818 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "23f82224f4564fea93572e8206bad1fe"
dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
mode: GRACEFUL
new_leader_uuid: "90f3da57796b488c9087a2f523adf045"
 from {username='slave'} at 127.0.0.1:37818
I20260709 13:21:34.889528 16292 raft_consensus.cc:2468] T 57a744f5aaf944678a63469cbf514015 P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 12.
W20260709 13:21:34.891911 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 49212.5us
I20260709 13:21:34.902926 16296 raft_consensus.cc:2393] T 7c2095a5f4ad49408dbc7f69c606120d P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 12: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
I20260709 13:21:34.903141 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:34.904017 16300 raft_consensus.cc:2468] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 17.
I20260709 13:21:34.908939 16819 raft_consensus.cc:1275] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [term 19 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 13:21:34.919831 21946 leader_election.cc:290] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 18 election: Requested vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:34.920212 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4f65cdf00b748bb963ee6e81ef43b79" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:34.921103 16226 raft_consensus.cc:2410] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 for term 17 because replica has last-logged OpId of term: 16 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 10.
I20260709 13:21:34.922333 16295 raft_consensus.cc:3060] T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:21:34.912628 16818 raft_consensus.cc:606] T 23f82224f4564fea93572e8206bad1fe P cb82cc7bd6db46d980c75237a1402860 [term 12 LEADER]: Received request to transfer leadership to TS 90f3da57796b488c9087a2f523adf045
I20260709 13:21:34.913631 16190 leader_election.cc:304] T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:21:34.927299 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a9de347c711477e87c582d003f36925" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
W20260709 13:21:34.911764 16265 connection.cc:256] RPC call timeout handler was delayed by -2.8246750289999909s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1966s for a response
W20260709 13:21:34.928679 16265 connection.cc:256] RPC call timeout handler was delayed by -2.7166750289998163s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2086s for a response
W20260709 13:21:34.950294 16789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 393223us
I20260709 13:21:34.956091 16291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 18 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:34.925618 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c2095a5f4ad49408dbc7f69c606120d" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:34.959203 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66ae176961b74813a58bc15c9c4c6f2a" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
W20260709 13:21:34.966020 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 219519us
I20260709 13:21:34.966118 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1df93779209748b9860f6d893dd3de53" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:21:34.958037 16217 raft_consensus.cc:2376] T 7c2095a5f4ad49408dbc7f69c606120d P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 12. Re-sending same reply.
I20260709 13:21:34.963480 21579 raft_consensus.cc:2804] T ada5a004ba7c46819051e3ccec71ffdb P cb82cc7bd6db46d980c75237a1402860 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:21:34.968740 21579 raft_consensus.cc:493] T ada5a004ba7c46819051e3ccec71ffdb P cb82cc7bd6db46d980c75237a1402860 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:21:34.974920 21579 raft_consensus.cc:3060] T ada5a004ba7c46819051e3ccec71ffdb P cb82cc7bd6db46d980c75237a1402860 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:21:34.977501 16150 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5a96373bb0e14dce8a266b2fc6c58507"
dest_uuid: "647a343a542d4a139fe106c19095129b"
mode: GRACEFUL
new_leader_uuid: "90f3da57796b488c9087a2f523adf045"
 from {username='slave'} at 127.0.0.1:40290
I20260709 13:21:34.978132 16150 raft_consensus.cc:606] T 5a96373bb0e14dce8a266b2fc6c58507 P 647a343a542d4a139fe106c19095129b [term 12 LEADER]: Received request to transfer leadership to TS 90f3da57796b488c9087a2f523adf045
I20260709 13:21:34.981040 21930 raft_consensus.cc:515] T 66f75124215f4ca6bef831e174a792d8 P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:34.974913 21946 raft_consensus.cc:2804] T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:21:34.983688 21946 raft_consensus.cc:493] T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Starting leader election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:21:34.983845 21930 leader_election.cc:290] T 66f75124215f4ca6bef831e174a792d8 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 15 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:34.984130 21946 raft_consensus.cc:3060] T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:21:34.984668 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a959dbecc2648eb85d1cc550f5a3245" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:34.985306 21903 raft_consensus.cc:493] T ecc1e363230043fe9fef390f0b979f20 P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:34.986348 21903 raft_consensus.cc:515] T ecc1e363230043fe9fef390f0b979f20 P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:34.989429 21903 leader_election.cc:290] T ecc1e363230043fe9fef390f0b979f20 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:34.990033 21903 raft_consensus.cc:493] T 8a878999c0024d2abe517484180290fc P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:21:34.991320 16297 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40474 (request call id 1008) took 1109 ms. Trace:
I20260709 13:21:34.991788 16297 rpcz_store.cc:276] 0709 13:21:33.846182 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:33.850555 (+  4373us) service_pool.cc:224] Handling call
0709 13:21:33.850892 (+   337us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:21:33.851356 (+   464us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:21:34.213732 (+362376us) log.cc:844] Serialized 14 byte log entry
0709 13:21:34.280978 (+ 67246us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:21:34.356383 (+ 75405us) log.cc:844] Serialized 52 byte log entry
0709 13:21:34.356607 (+   224us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:21:34.356650 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:34.356756 (+   106us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:21:34.755778 (+399022us) spinlock_profiling.cc:243] Waited 22.3 ms on lock 0x7b4400022b4c. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69ad8fadb1 00007f69ad998c85 00007f69ad9996a2 00007f69b070b1db 00007f69b0706aaf 00007f69b06fa86e 00007f69b06f9bb6 00007f69b06f97d8 00007f69b0703221 00007f69b07031d9 00007f69b0703169
0709 13:21:34.921799 (+166021us) raft_consensus.cc:1685] finished
0709 13:21:34.927267 (+  5468us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:21:34.955654 (+ 28387us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":136022,"prepare.run_cpu_time_us":53,"prepare.run_wall_time_us":54,"raft.queue_time_us":74542,"raft.run_cpu_time_us":387,"raft.run_wall_time_us":6717,"spinlock_wait_cycles":51315072,"thread_start_us":428199,"threads_started":2,"wal-append.queue_time_us":321449}
I20260709 13:21:34.991305 16225 raft_consensus.cc:2468] T 5a959dbecc2648eb85d1cc550f5a3245 P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 14.
I20260709 13:21:34.990765 21903 raft_consensus.cc:515] T 8a878999c0024d2abe517484180290fc P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:34.996033 16146 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40830 (request call id 1798) took 1390 ms. Trace:
I20260709 13:21:34.996225 21934 raft_consensus.cc:493] T 5a959dbecc2648eb85d1cc550f5a3245 P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:35.000986 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66ae176961b74813a58bc15c9c4c6f2a" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:34.996445 16146 rpcz_store.cc:276] 0709 13:21:33.605745 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:33.740875 (+135130us) service_pool.cc:224] Handling call
0709 13:21:33.741192 (+   317us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:21:33.741493 (+   301us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:21:33.741517 (+    24us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:21:33.746192 (+  4675us) log.cc:844] Serialized 26 byte log entry
0709 13:21:34.512530 (+766338us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:21:34.512601 (+    71us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:34.512715 (+   114us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:21:34.762868 (+250153us) spinlock_profiling.cc:243] Waited 35.2 ms on lock 0x7b4400022b4c. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69ad8fadb1 00007f69ad998c85 00007f69ad9996a2 00007f69b070b1db 00007f69b0706aaf 00007f69b06fa86e 00007f69b06f9bb6 00007f69b06f97d8 00007f69b0703221 00007f69b07031d9 00007f69b0703169
0709 13:21:34.995677 (+232809us) raft_consensus.cc:1685] finished
0709 13:21:34.995697 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:21:34.995968 (+   271us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":13078,"raft.run_cpu_time_us":120,"raft.run_wall_time_us":7978,"spinlock_wait_cycles":80920448,"thread_start_us":765679,"threads_started":1,"wal-append.queue_time_us":773929}
I20260709 13:21:35.005959 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c2095a5f4ad49408dbc7f69c606120d" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:21:35.006505 16298 raft_consensus.cc:1275] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 13:21:35.008162 16225 raft_consensus.cc:2393] T 7c2095a5f4ad49408dbc7f69c606120d P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 12: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
I20260709 13:21:35.008251 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18ad7e3af5504571a69d10c00750a01c" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
W20260709 13:21:35.008961 16816 consensus_meta.cc:287] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860: Time spent flushing consensus metadata: real 0.542s	user 0.006s	sys 0.006s
I20260709 13:21:35.009719 16150 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "746aabf856ab401c9ef5e859371e7b67"
dest_uuid: "647a343a542d4a139fe106c19095129b"
mode: GRACEFUL
new_leader_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
 from {username='slave'} at 127.0.0.1:40290
I20260709 13:21:34.951185 21908 raft_consensus.cc:515] T 1df93779209748b9860f6d893dd3de53 P 90f3da57796b488c9087a2f523adf045 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:35.009323 16816 raft_consensus.cc:2468] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 14.
I20260709 13:21:35.012394 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66f75124215f4ca6bef831e174a792d8" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:35.016536 16217 raft_consensus.cc:2393] T 66f75124215f4ca6bef831e174a792d8 P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 15: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
I20260709 13:21:35.022342 16221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9944d5e4b8c4109af9955c4f08c9095" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:35.000113 21623 consensus_queue.cc:1048] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.015s
I20260709 13:21:35.024241 16221 raft_consensus.cc:2393] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 18: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
I20260709 13:21:35.028437 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b7d1ab519044eda2d5e04a07242f25" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:35.029425 16217 raft_consensus.cc:2393] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 14: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
I20260709 13:21:35.030328 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 18 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:35.032234 16219 raft_consensus.cc:1275] T 02838c86531d4f98bf58d1c22b8baf77 P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:21:35.033429 16219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62fe3346cd3c42c4a55e14af9073bca4" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:21:35.034065 16219 raft_consensus.cc:3060] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:21:35.030934 16217 raft_consensus.cc:2468] T ecc1e363230043fe9fef390f0b979f20 P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 17.
I20260709 13:21:35.035964 16222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 18 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:35.036761 16222 raft_consensus.cc:2468] T ecc1e363230043fe9fef390f0b979f20 P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 17.
I20260709 13:21:35.037776 16222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c61a39fd4f84424b129c1a6f9ba404d" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:21:35.038753 16222 raft_consensus.cc:2393] T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 17: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
I20260709 13:21:35.037443 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5718c623ebee4d2bb770f0f9ddb2f164" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:35.040091 16217 raft_consensus.cc:2468] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 14.
I20260709 13:21:35.043040 16291 raft_consensus.cc:1240] T 66ae176961b74813a58bc15c9c4c6f2a P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 19. Current term is 20. Ops: []
I20260709 13:21:35.020867 16150 raft_consensus.cc:606] T 746aabf856ab401c9ef5e859371e7b67 P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Received request to transfer leadership to TS 1ca2725d290e47d68b1a4b609c9816ac
I20260709 13:21:35.040267 16224 raft_consensus.cc:2468] T 18ad7e3af5504571a69d10c00750a01c P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 13.
I20260709 13:21:35.045305 16300 raft_consensus.cc:2376] T 1df93779209748b9860f6d893dd3de53 P 90f3da57796b488c9087a2f523adf045 [term 14 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 647a343a542d4a139fe106c19095129b in term 14. Re-sending same reply.
I20260709 13:21:35.047962 16150 raft_consensus.cc:614] T 746aabf856ab401c9ef5e859371e7b67 P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:21:35.050339 16292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02838c86531d4f98bf58d1c22b8baf77" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:35.047040 16295 raft_consensus.cc:2468] T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 17.
I20260709 13:21:35.052459 16292 raft_consensus.cc:2468] T 02838c86531d4f98bf58d1c22b8baf77 P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 15.
I20260709 13:21:35.052958 16299 raft_consensus.cc:1240] T cdbd78acb4cf46298e0de8ba0d130e19 P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 18. Current term is 19. Ops: []
I20260709 13:21:35.054407 16297 raft_consensus.cc:1240] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 15. Current term is 16. Ops: []
I20260709 13:21:35.056034 16296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f5f89186204242bf377f7c42b2e2da" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 18 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:35.016819 21941 raft_consensus.cc:3060] T e4dc6b0541ca431ebc757beac00521f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:21:35.005722 21934 raft_consensus.cc:515] T 5a959dbecc2648eb85d1cc550f5a3245 P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:35.065166 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02838c86531d4f98bf58d1c22b8baf77" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:35.071089 16224 raft_consensus.cc:2393] T 02838c86531d4f98bf58d1c22b8baf77 P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 15: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
I20260709 13:21:35.073335 16293 raft_consensus.cc:2393] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 19: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
I20260709 13:21:35.079545 16218 raft_consensus.cc:1275] T 5a959dbecc2648eb85d1cc550f5a3245 P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 13:21:35.081223 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee9ed67006144797b1de6ea8c2013a68" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:35.080487 16222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:35.042222 21579 raft_consensus.cc:515] T ada5a004ba7c46819051e3ccec71ffdb P cb82cc7bd6db46d980c75237a1402860 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:35.086118 22018 raft_consensus.cc:493] T 84f2da98864b4425bf7c0055951fdd4b P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:35.086360 16296 raft_consensus.cc:1240] T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 15. Current term is 16. Ops: [15.12-15.12]
I20260709 13:21:35.086580 15046 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:21:35.087050 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c61a39fd4f84424b129c1a6f9ba404d" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
W20260709 13:21:35.087440 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 208413us
W20260709 13:21:35.093538 21920 consensus_meta.cc:287] T 8638ee363273426db1393dd81e5c7b72 P cb82cc7bd6db46d980c75237a1402860: Time spent flushing consensus metadata: real 0.576s	user 0.005s	sys 0.007s
I20260709 13:21:35.095337 16222 raft_consensus.cc:2468] T 19baa7bc367844c288bca8f81439c413 P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 17.
I20260709 13:21:35.110342 16217 raft_consensus.cc:1275] T c2f5f89186204242bf377f7c42b2e2da P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 13:21:35.112764 22018 raft_consensus.cc:515] T 84f2da98864b4425bf7c0055951fdd4b P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:35.118039 21903 leader_election.cc:290] T 8a878999c0024d2abe517484180290fc P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:35.121085 21908 leader_election.cc:290] T 1df93779209748b9860f6d893dd3de53 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
W20260709 13:21:35.122610 22019 log.cc:927] Time spent T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 0.106s	user 0.000s	sys 0.000s
I20260709 13:21:35.123871 21946 raft_consensus.cc:515] T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:35.096957 16223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806d7690c2b644959152074bb2cab0a7" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:35.130764 16300 raft_consensus.cc:2393] T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 17: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
I20260709 13:21:35.130409 16224 raft_consensus.cc:2468] T ee9ed67006144797b1de6ea8c2013a68 P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 15.
I20260709 13:21:35.133780 16219 raft_consensus.cc:2468] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 21.
I20260709 13:21:35.135582 16223 raft_consensus.cc:2410] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 for term 16 because replica has last-logged OpId of term: 15 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 10.
W20260709 13:21:35.130422 16151 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.537s	user 0.009s	sys 0.018s
W20260709 13:21:35.137282 16151 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.544s	user 0.009s	sys 0.018s
I20260709 13:21:35.143889 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806d7690c2b644959152074bb2cab0a7" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 17 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:35.145201 16224 raft_consensus.cc:2410] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b for term 17 because replica has last-logged OpId of term: 15 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 9.
I20260709 13:21:35.126889 21946 leader_election.cc:290] T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 15 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:35.150427 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a96373bb0e14dce8a266b2fc6c58507" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:35.150879 22018 leader_election.cc:290] T 84f2da98864b4425bf7c0055951fdd4b P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:35.143653 16218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cc959fb7f704fba9009c0595467d074" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:21:35.155264 21245 raft_consensus.cc:2804] T fffab26427e24d398e2695aad40ca134 P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:21:35.136142 21920 raft_consensus.cc:515] T 8638ee363273426db1393dd81e5c7b72 P cb82cc7bd6db46d980c75237a1402860 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:35.166215 21579 leader_election.cc:290] T ada5a004ba7c46819051e3ccec71ffdb P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 17 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:35.166517 15046 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:21:35.169970 16223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f5f89186204242bf377f7c42b2e2da" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:35.172035 16296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4f65cdf00b748bb963ee6e81ef43b79" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:35.172920 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4dc6b0541ca431ebc757beac00521f1" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:21:35.179281 21245 raft_consensus.cc:493] T fffab26427e24d398e2695aad40ca134 P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Starting leader election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:21:35.181516 16296 raft_consensus.cc:2468] T c4f65cdf00b748bb963ee6e81ef43b79 P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 16.
I20260709 13:21:35.185784 16223 raft_consensus.cc:2393] T c2f5f89186204242bf377f7c42b2e2da P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 17: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
I20260709 13:21:35.185595 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d16dfad0c3b540e5b4b86492ddf87208" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:21:35.194657 16220 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57604 (request call id 2990) took 3027 ms. Trace:
I20260709 13:21:35.183146 21245 raft_consensus.cc:3060] T fffab26427e24d398e2695aad40ca134 P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:21:35.195554 16218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c61a39fd4f84424b129c1a6f9ba404d" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:35.196621 16218 raft_consensus.cc:2393] T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 17: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
I20260709 13:21:35.199143 16825 raft_consensus.cc:2393] T 8638ee363273426db1393dd81e5c7b72 P cb82cc7bd6db46d980c75237a1402860 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 17: Already voted for candidate cb82cc7bd6db46d980c75237a1402860 in this term.
W20260709 13:21:35.215893 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 251353us
W20260709 13:21:35.221161 16789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 225149us
W20260709 13:21:35.227111 16791 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:35.195255 16220 rpcz_store.cc:276] 0709 13:21:32.167698 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:32.897896 (+730198us) service_pool.cc:224] Handling call
0709 13:21:32.898147 (+   251us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:21:34.544886 (+1646739us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:21:34.544961 (+    75us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:21:34.545591 (+   630us) log.cc:844] Serialized 26 byte log entry
0709 13:21:34.656715 (+111124us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:21:34.656750 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:34.690520 (+ 33770us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:21:35.187456 (+496936us) raft_consensus.cc:1685] finished
0709 13:21:35.187486 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:21:35.194574 (+  7088us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":27483,"raft.queue_time_us":565477,"raft.run_cpu_time_us":219,"raft.run_wall_time_us":34026,"thread_start_us":1665591,"threads_started":2,"wal-append.queue_time_us":424509}
I20260709 13:21:35.179313 21997 raft_consensus.cc:515] T af57996a8be84d398f0f023e8d08ef77 P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:35.238471 16218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7ca61592e848ea9b0fb81a2ce33cdc" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:35.236550 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9944d5e4b8c4109af9955c4f08c9095" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:21:35.240854 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a378d7f796f94615b3b347119a9880fd" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:35.239254 16218 raft_consensus.cc:2468] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 17.
I20260709 13:21:35.263425 16300 raft_consensus.cc:3060] T c9944d5e4b8c4109af9955c4f08c9095 P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:21:35.267076 16296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 18 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:35.280655 16220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68a6838d82ef4cac83507f906139d90f" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
W20260709 13:21:35.285106 16789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 50304.8us
I20260709 13:21:35.292793 16219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c11074579b046d99e585c2818e18a2c" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:35.294142 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ad5dbfa99ee4b8d9e28be92b3f36db5" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
W20260709 13:21:35.295492 16263 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:35.284680 21934 leader_election.cc:290] T 5a959dbecc2648eb85d1cc550f5a3245 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
W20260709 13:21:35.299989 22022 log.cc:927] Time spent T fffab26427e24d398e2695aad40ca134 P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 0.075s	user 0.000s	sys 0.000s
W20260709 13:21:35.300743 22021 log.cc:927] Time spent T 434bff3082d141838068f323bdb0d1ed P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 0.120s	user 0.000s	sys 0.000s
I20260709 13:21:35.302806 16825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b7d1ab519044eda2d5e04a07242f25" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:21:35.303509 16219 raft_consensus.cc:2468] T 8c11074579b046d99e585c2818e18a2c P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 15.
I20260709 13:21:35.303604 16816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62fe3346cd3c42c4a55e14af9073bca4" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:21:35.308503 16816 raft_consensus.cc:2368] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac for earlier term 20. Current term is 21.
I20260709 13:21:35.308540 16217 raft_consensus.cc:2468] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 14.
I20260709 13:21:35.293097 16220 raft_consensus.cc:2468] T 68a6838d82ef4cac83507f906139d90f P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 15.
I20260709 13:21:35.303579 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66ae176961b74813a58bc15c9c4c6f2a" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
W20260709 13:21:35.309623 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 228876us
I20260709 13:21:35.310072 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e4bfb9090c8460eb14fa7e0d818e7f1" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:21:35.312458 16295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a878999c0024d2abe517484180290fc" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:35.317801 16222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74ca8140f35c4ae29e8a664f6bdfc829" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
W20260709 13:21:35.318667 16115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 433478us
I20260709 13:21:35.323843 16818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9944d5e4b8c4109af9955c4f08c9095" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:21:35.325541 16300 raft_consensus.cc:2410] T c9944d5e4b8c4109af9955c4f08c9095 P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac for term 18 because replica has last-logged OpId of term: 17 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 12.
I20260709 13:21:35.338385 16296 raft_consensus.cc:1275] T a378d7f796f94615b3b347119a9880fd P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 13:21:35.339085 16819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66f75124215f4ca6bef831e174a792d8" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:21:35.291411 21245 raft_consensus.cc:515] T fffab26427e24d398e2695aad40ca134 P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:21:35.343926 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 438040us
I20260709 13:21:35.341900 16824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 18 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:35.368623 16295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d4ab870a05347868956aa8b841b2981" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:21:35.369554 16295 raft_consensus.cc:3060] T 8d4ab870a05347868956aa8b841b2981 P 90f3da57796b488c9087a2f523adf045 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:21:35.379086 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a959dbecc2648eb85d1cc550f5a3245" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:35.380123 16300 raft_consensus.cc:2468] T 5a959dbecc2648eb85d1cc550f5a3245 P 90f3da57796b488c9087a2f523adf045 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 14.
I20260709 13:21:35.382122 16147 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40830 (request call id 1800) took 1660 ms. Trace:
I20260709 13:21:35.382587 16147 rpcz_store.cc:276] 0709 13:21:33.722302 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:34.458679 (+736377us) service_pool.cc:224] Handling call
0709 13:21:34.513710 (+ 55031us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:21:34.514388 (+   678us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:21:34.514420 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:21:34.566645 (+ 52225us) log.cc:844] Serialized 26 byte log entry
0709 13:21:34.956926 (+390281us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:21:34.956966 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:34.957033 (+    67us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:21:35.381846 (+424813us) raft_consensus.cc:1685] finished
0709 13:21:35.381872 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:21:35.382049 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":7301,"raft.run_cpu_time_us":270,"raft.run_wall_time_us":270,"thread_start_us":370807,"threads_started":1,"wal-append.queue_time_us":572339}
W20260709 13:21:35.385106 16265 connection.cc:256] RPC call timeout handler was delayed by -2.6050750289998632s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.221s for a response
W20260709 13:21:35.385717 16265 connection.cc:256] RPC call timeout handler was delayed by -2.6050750289998632s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.221s for a response
W20260709 13:21:35.386142 16265 connection.cc:256] RPC call timeout handler was delayed by -2.5807750289998239s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2237s for a response
W20260709 13:21:35.386539 16265 connection.cc:256] RPC call timeout handler was delayed by -2.5195750289999523s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.23050000000000004s for a response
I20260709 13:21:35.388139 21997 leader_election.cc:290] T af57996a8be84d398f0f023e8d08ef77 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 17 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:35.389014 21997 raft_consensus.cc:493] T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:21:35.389846 21997 raft_consensus.cc:515] T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:21:35.387060 16265 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:21:35.412732 16265 connection.cc:256] RPC call timeout handler was delayed by -2.3890750289999687s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.24500000000000002s for a response
I20260709 13:21:35.414520 16296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f2da98864b4425bf7c0055951fdd4b" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:35.360713 21920 leader_election.cc:290] T 8638ee363273426db1393dd81e5c7b72 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 17 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
W20260709 13:21:35.417209 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 45242.9us
I20260709 13:21:35.462756 16790 leader_election.cc:304] T ee9ed67006144797b1de6ea8c2013a68 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 647a343a542d4a139fe106c19095129b
I20260709 13:21:35.320894 22011 raft_consensus.cc:515] T 829b4ad8c5a54eababeea1181b960d4b P cb82cc7bd6db46d980c75237a1402860 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:35.465580 22011 leader_election.cc:290] T 829b4ad8c5a54eababeea1181b960d4b P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 15 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:21:35.466365 16820 raft_consensus.cc:1240] T 829b4ad8c5a54eababeea1181b960d4b P cb82cc7bd6db46d980c75237a1402860 [term 15 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 14. Current term is 15. Ops: []
I20260709 13:21:35.467514 21704 raft_consensus.cc:493] T 26354361e9b044cea1c2b86c4882fe92 P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:21:35.430683 16190 leader_election.cc:304] T 02838c86531d4f98bf58d1c22b8baf77 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:21:35.419703 21769 raft_consensus.cc:493] T 8cc959fb7f704fba9009c0595467d074 P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:21:35.458783 21996 raft_consensus.cc:493] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:21:35.478292 21996 raft_consensus.cc:515] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:35.480867 22027 raft_consensus.cc:493] T 8cc959fb7f704fba9009c0595467d074 P 647a343a542d4a139fe106c19095129b [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:21:35.481770 16151 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40830 (request call id 1799) took 1875 ms. Trace:
I20260709 13:21:35.481639 22027 raft_consensus.cc:515] T 8cc959fb7f704fba9009c0595467d074 P 647a343a542d4a139fe106c19095129b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:35.482206 16151 rpcz_store.cc:276] 0709 13:21:33.606028 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:33.742309 (+136281us) service_pool.cc:224] Handling call
0709 13:21:33.742593 (+   284us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:21:34.592328 (+849735us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:21:34.592385 (+    57us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:21:34.593224 (+   839us) log.cc:844] Serialized 26 byte log entry
0709 13:21:35.143245 (+550021us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:21:35.143294 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:35.148497 (+  5203us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:21:35.479066 (+330569us) spinlock_profiling.cc:243] Waited 57.9 ms on lock 0x7b24001206c8. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69ad8fadb1 00007f69ad8f89c4 00007f69ad8f6acb 00007f69adab1dc9 00007f69adab1a55 00007f69adab0fb0 00007f69b0727488 00007f69b0727be8 00007f69b06ff398 00007f69b06fa76a 00007f69b06f9bb6
0709 13:21:35.481428 (+  2362us) raft_consensus.cc:1685] finished
0709 13:21:35.481453 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:21:35.481706 (+   253us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":19665,"raft.run_cpu_time_us":131,"raft.run_wall_time_us":27011,"spinlock_wait_cycles":133166976,"thread_start_us":1393178,"threads_started":2,"wal-append.queue_time_us":571096}
I20260709 13:21:35.480967 21996 leader_election.cc:290] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:35.490352 21904 raft_consensus.cc:493] T 829b4ad8c5a54eababeea1181b960d4b P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
W20260709 13:21:35.460903 16265 connection.cc:256] RPC call timeout handler was delayed by -2.3890750289999687s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.24500000000000002s for a response
I20260709 13:21:35.491137 21904 raft_consensus.cc:515] T 829b4ad8c5a54eababeea1181b960d4b P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:35.468305 21704 raft_consensus.cc:515] T 26354361e9b044cea1c2b86c4882fe92 P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:35.468327 16117 leader_election.cc:304] T b87147731a68400e8cce9b879daa7c5d P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860
W20260709 13:21:35.492581 16265 connection.cc:256] RPC call timeout handler was delayed by -2.3406785199999831s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16010000000000002s for a response
W20260709 13:21:35.504065 16265 connection.cc:256] RPC call timeout handler was delayed by -2.3406785199999831s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16010000000000002s for a response
W20260709 13:21:35.504539 16265 connection.cc:256] RPC call timeout handler was delayed by -2.3208785199999511s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1623s for a response
W20260709 13:21:35.505030 16265 connection.cc:256] RPC call timeout handler was delayed by -2.3208785199999511s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1623s for a response
W20260709 13:21:35.505525 16265 connection.cc:256] RPC call timeout handler was delayed by -2.046378519999962s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1928s for a response
W20260709 13:21:35.507925 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 102079us
W20260709 13:21:35.517745 16265 connection.cc:256] RPC call timeout handler was delayed by -1.9887785200000963s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19920000000000002s for a response
I20260709 13:21:35.517760 16224 raft_consensus.cc:2468] T e4dc6b0541ca431ebc757beac00521f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 15.
I20260709 13:21:35.518419 21904 leader_election.cc:290] T 829b4ad8c5a54eababeea1181b960d4b P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
W20260709 13:21:35.519914 16265 connection.cc:256] RPC call timeout handler was delayed by -1.9518785200000366s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2033s for a response
I20260709 13:21:35.522861 16820 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49896 (request call id 1366) took 1248 ms. Trace:
I20260709 13:21:35.524562 16820 rpcz_store.cc:276] 0709 13:21:34.274191 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:34.698989 (+424798us) service_pool.cc:224] Handling call
0709 13:21:34.738178 (+ 39189us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:21:35.466842 (+728664us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:35.522427 (+ 55585us) spinlock_profiling.cc:243] Waited 728 ms on lock 0x7b54007c0e54. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:21:35.522752 (+   325us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1674117888}
W20260709 13:21:35.526099 16265 connection.cc:256] RPC call timeout handler was delayed by -1.9518785200000366s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2033s for a response
I20260709 13:21:35.511066 21930 raft_consensus.cc:2804] T 02838c86531d4f98bf58d1c22b8baf77 P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 13:21:35.526788 16265 connection.cc:256] RPC call timeout handler was delayed by -1.77007851999997s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2235s for a response
W20260709 13:21:35.529801 16265 connection.cc:256] RPC call timeout handler was delayed by -1.6405573489998915s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.45380000000000004s for a response
W20260709 13:21:35.530303 16265 connection.cc:256] RPC call timeout handler was delayed by -1.6328676289997475s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.30350000000000005s for a response
W20260709 13:21:35.531778 16265 connection.cc:256] RPC call timeout handler was delayed by -1.6328676289997475s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.30350000000000005s for a response
W20260709 13:21:35.532253 16265 connection.cc:256] RPC call timeout handler was delayed by -1.4443872199999532s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15600000000000003s for a response
W20260709 13:21:35.532724 16265 connection.cc:256] RPC call timeout handler was delayed by -1.4074872199998936s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16010000000000002s for a response
W20260709 13:21:35.533159 16265 connection.cc:256] RPC call timeout handler was delayed by -1.397067628999821s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.32970000000000005s for a response
W20260709 13:21:35.533638 16265 connection.cc:256] RPC call timeout handler was delayed by -1.3057872199999565s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1714s for a response
W20260709 13:21:35.534101 16265 connection.cc:256] RPC call timeout handler was delayed by -1.3057872199999565s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1714s for a response
W20260709 13:21:35.534579 16265 connection.cc:256] RPC call timeout handler was delayed by -1.2760872199999085s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17470000000000002s for a response
W20260709 13:21:35.535053 16265 connection.cc:256] RPC call timeout handler was delayed by -1.2760872199999085s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17470000000000002s for a response
W20260709 13:21:35.535579 16265 connection.cc:256] RPC call timeout handler was delayed by -1.2679872199998954s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1756s for a response
W20260709 13:21:35.536077 16265 connection.cc:256] RPC call timeout handler was delayed by -1.2400872199998503s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1787s for a response
W20260709 13:21:35.536559 16265 connection.cc:256] RPC call timeout handler was delayed by -1.2400872199998503s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1787s for a response
W20260709 13:21:35.537009 16265 connection.cc:256] RPC call timeout handler was delayed by -1.183387219999986s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18500000000000003s for a response
W20260709 13:21:35.537434 16265 connection.cc:256] RPC call timeout handler was delayed by -1.1050872199998594s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1937s for a response
W20260709 13:21:35.537878 16265 connection.cc:256] RPC call timeout handler was delayed by -1.102387219999855s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.194s for a response
W20260709 13:21:35.538302 16265 connection.cc:256] RPC call timeout handler was delayed by -1.102387219999855s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.194s for a response
I20260709 13:21:35.521687 22029 raft_consensus.cc:2804] T ee9ed67006144797b1de6ea8c2013a68 P cb82cc7bd6db46d980c75237a1402860 [term 15 FOLLOWER]: Leader election won for term 15
W20260709 13:21:35.551800 16789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 247304us
I20260709 13:21:35.551450 16295 raft_consensus.cc:2468] T 8d4ab870a05347868956aa8b841b2981 P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 15.
I20260709 13:21:35.530247 21930 raft_consensus.cc:493] T 02838c86531d4f98bf58d1c22b8baf77 P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:21:35.558060 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 95484.5us
I20260709 13:21:35.562084 21931 raft_consensus.cc:2749] T b87147731a68400e8cce9b879daa7c5d P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:21:35.562402 22000 consensus_queue.cc:1048] T 02838c86531d4f98bf58d1c22b8baf77 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:21:35.562847 21931 raft_consensus.cc:493] T 84f2da98864b4425bf7c0055951fdd4b P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:21:35.563506 21908 consensus_queue.cc:1048] T ada5a004ba7c46819051e3ccec71ffdb P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:21:35.564066 16115 leader_election.cc:304] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:21:35.526023 22018 consensus_queue.cc:1059] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: INVALID_TERM, Last received: 18.14, Next index: 15, Last known committed idx: 13, Time since last communication: 0.032s
I20260709 13:21:35.526407 21769 raft_consensus.cc:515] T 8cc959fb7f704fba9009c0595467d074 P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:35.565274 16116 leader_election.cc:304] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
I20260709 13:21:35.557742 21930 raft_consensus.cc:3060] T 02838c86531d4f98bf58d1c22b8baf77 P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:21:35.557827 21997 leader_election.cc:290] T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:35.563444 21931 raft_consensus.cc:515] T 84f2da98864b4425bf7c0055951fdd4b P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:35.559518 21245 leader_election.cc:290] T fffab26427e24d398e2695aad40ca134 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 16 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:35.556923 21904 consensus_queue.cc:1059] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 10, Time since last communication: 0.016s
I20260709 13:21:35.575256 22018 consensus_queue.cc:1059] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.005s
I20260709 13:21:35.583504 22018 raft_consensus.cc:3055] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:21:35.584033 22036 raft_consensus.cc:2804] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:21:35.584098 22018 raft_consensus.cc:740] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:21:35.584671 22036 raft_consensus.cc:493] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:35.579682 21996 consensus_queue.cc:1059] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.006s
I20260709 13:21:35.585181 22036 raft_consensus.cc:3060] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:21:35.585752 21996 raft_consensus.cc:3055] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:21:35.586360 21996 raft_consensus.cc:740] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:21:35.585659 22018 consensus_queue.cc:260] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.13, Last appended by leader: 13, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:35.568218 22027 leader_election.cc:290] T 8cc959fb7f704fba9009c0595467d074 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:35.587600 21904 raft_consensus.cc:3055] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:21:35.587885 16790 leader_election.cc:304] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:21:35.580489 16192 leader_election.cc:304] T 19baa7bc367844c288bca8f81439c413 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860
W20260709 13:21:35.583078 16117 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 114983us
I20260709 13:21:35.589478 21946 raft_consensus.cc:2749] T 19baa7bc367844c288bca8f81439c413 P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:21:35.573648 21941 raft_consensus.cc:3055] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:21:35.590519 16117 leader_election.cc:304] T 9e65676b21b748728ad39668cfe722ec P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860
I20260709 13:21:35.591027 21941 raft_consensus.cc:740] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:21:35.592156 21941 consensus_queue.cc:260] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:35.593366 21941 raft_consensus.cc:3060] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:21:35.588038 21904 raft_consensus.cc:740] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:21:35.580864 16190 leader_election.cc:304] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
W20260709 13:21:35.598471 16192 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 92341.3us
I20260709 13:21:35.598810 21946 raft_consensus.cc:2804] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:21:35.600145 21946 raft_consensus.cc:493] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Starting leader election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:21:35.600553 21946 raft_consensus.cc:3060] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:21:35.601900 21769 leader_election.cc:290] T 8cc959fb7f704fba9009c0595467d074 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:35.595588 21904 consensus_queue.cc:260] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:35.607724 22029 raft_consensus.cc:697] T ee9ed67006144797b1de6ea8c2013a68 P cb82cc7bd6db46d980c75237a1402860 [term 15 LEADER]: Becoming Leader. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:21:35.608402 16823 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49906 (request call id 1137) took 1138 ms. Trace:
I20260709 13:21:35.608919 22031 raft_consensus.cc:993] T 23f82224f4564fea93572e8206bad1fe P cb82cc7bd6db46d980c75237a1402860: : Instructing follower 90f3da57796b488c9087a2f523adf045 to start an election
I20260709 13:21:35.609678 22031 raft_consensus.cc:1081] T 23f82224f4564fea93572e8206bad1fe P cb82cc7bd6db46d980c75237a1402860 [term 12 LEADER]: Signalling peer 90f3da57796b488c9087a2f523adf045 to start an election
I20260709 13:21:35.608978 22029 consensus_queue.cc:237] T ee9ed67006144797b1de6ea8c2013a68 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:35.611873 21904 raft_consensus.cc:3060] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:21:35.615430 16191 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:21:35.616959 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 46183.7us
I20260709 13:21:35.584743 22025 consensus_queue.cc:1059] T 2794af5b99984efebdee564035841cb0 P 647a343a542d4a139fe106c19095129b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.050s
I20260709 13:21:35.619851 22026 raft_consensus.cc:2764] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [term 21 FOLLOWER]: Leader pre-election decision vote started in defunct term 20: won
I20260709 13:21:35.598323 21704 leader_election.cc:290] T 26354361e9b044cea1c2b86c4882fe92 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:35.621496 22024 raft_consensus.cc:993] T 5a96373bb0e14dce8a266b2fc6c58507 P 647a343a542d4a139fe106c19095129b: : Instructing follower 90f3da57796b488c9087a2f523adf045 to start an election
I20260709 13:21:35.621999 22024 raft_consensus.cc:1081] T 5a96373bb0e14dce8a266b2fc6c58507 P 647a343a542d4a139fe106c19095129b [term 12 LEADER]: Signalling peer 90f3da57796b488c9087a2f523adf045 to start an election
I20260709 13:21:35.608814 16823 rpcz_store.cc:276] 0709 13:21:34.470139 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:34.824090 (+353951us) service_pool.cc:224] Handling call
0709 13:21:34.828916 (+  4826us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:21:35.013023 (+184107us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 13:21:35.359659 (+346636us) log.cc:844] Serialized 14 byte log entry
0709 13:21:35.607849 (+248190us) log.cc:844] Serialized 14 byte log entry
0709 13:21:35.608015 (+   166us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:21:35.608056 (+    41us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:21:35.608134 (+    78us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:35.608214 (+    80us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:21:35.608340 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":128081,"prepare.queue_time_us":454884,"prepare.run_cpu_time_us":99,"prepare.run_wall_time_us":4888,"raft.queue_time_us":120247,"raft.run_cpu_time_us":124,"raft.run_wall_time_us":125,"thread_start_us":347774,"threads_started":2,"wal-append.queue_time_us":122315}
I20260709 13:21:35.623522 22025 raft_consensus.cc:3055] T 2794af5b99984efebdee564035841cb0 P 647a343a542d4a139fe106c19095129b [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:21:35.623899 22025 raft_consensus.cc:740] T 2794af5b99984efebdee564035841cb0 P 647a343a542d4a139fe106c19095129b [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:21:35.624783 22025 consensus_queue.cc:260] T 2794af5b99984efebdee564035841cb0 P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:35.626233 22025 raft_consensus.cc:3060] T 2794af5b99984efebdee564035841cb0 P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:21:35.624424 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a378d7f796f94615b3b347119a9880fd" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:35.632157 16192 leader_election.cc:304] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 647a343a542d4a139fe106c19095129b
I20260709 13:21:35.634357 16192 leader_election.cc:304] T 8c11074579b046d99e585c2818e18a2c P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860
W20260709 13:21:35.635195 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 53473.7us
I20260709 13:21:35.635520 16151 raft_consensus.cc:1275] T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 13:21:35.635888 21934 raft_consensus.cc:2804] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:21:35.636338 21934 raft_consensus.cc:493] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Starting leader election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:21:35.636719 21934 raft_consensus.cc:3060] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:21:35.636992 16189 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:35.639361 21903 raft_consensus.cc:2749] T 8c11074579b046d99e585c2818e18a2c P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:21:35.639916 16151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2794af5b99984efebdee564035841cb0" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:21:35.642760 16147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5718c623ebee4d2bb770f0f9ddb2f164" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:35.643330 21996 consensus_queue.cc:260] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:35.644601 21996 raft_consensus.cc:3060] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:21:35.645308 16221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a378d7f796f94615b3b347119a9880fd" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
W20260709 13:21:35.647854 16117 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 57455.8us
I20260709 13:21:35.648751 16117 leader_election.cc:304] T 3884f864a96d428785895e5463e5728f P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860
I20260709 13:21:35.649518 21704 raft_consensus.cc:2749] T 9e65676b21b748728ad39668cfe722ec P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:21:35.650209 21704 raft_consensus.cc:2749] T 3884f864a96d428785895e5463e5728f P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:21:35.651512 22018 raft_consensus.cc:3060] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:21:35.655290 22001 raft_consensus.cc:2804] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:21:35.655747 22001 raft_consensus.cc:493] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Starting leader election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:21:35.656123 22001 raft_consensus.cc:3060] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:21:35.644560 21946 raft_consensus.cc:515] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:35.660290 21946 leader_election.cc:290] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 17 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:35.668887 16116 leader_election.cc:304] T ee9ed67006144797b1de6ea8c2013a68 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
I20260709 13:21:35.669253 21704 consensus_queue.cc:1048] T 5a959dbecc2648eb85d1cc550f5a3245 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:21:35.669832 16116 leader_election.cc:304] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
I20260709 13:21:35.670708 21704 raft_consensus.cc:2804] T ee9ed67006144797b1de6ea8c2013a68 P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:21:35.662261 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdbd78acb4cf46298e0de8ba0d130e19" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:21:35.671386 22000 raft_consensus.cc:2804] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:21:35.671965 22000 raft_consensus.cc:697] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [term 21 LEADER]: Becoming Leader. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:21:35.672964 22000 consensus_queue.cc:237] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:35.671061 21704 raft_consensus.cc:493] T ee9ed67006144797b1de6ea8c2013a68 P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:35.675526 22044 consensus_queue.cc:1048] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:21:35.676055 21704 raft_consensus.cc:3060] T ee9ed67006144797b1de6ea8c2013a68 P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:21:35.679860 16217 raft_consensus.cc:1240] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 13. Current term is 14. Ops: []
I20260709 13:21:35.681365 16223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5718c623ebee4d2bb770f0f9ddb2f164" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:35.682966 16116 leader_election.cc:304] T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 90f3da57796b488c9087a2f523adf045
I20260709 13:21:35.683683 16217 raft_consensus.cc:1275] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 13:21:35.683941 22037 raft_consensus.cc:2804] T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:21:35.684536 22037 raft_consensus.cc:697] T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b [term 15 LEADER]: Becoming Leader. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:21:35.685456 22037 consensus_queue.cc:237] T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.7, Last appended by leader: 7, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:35.687119 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af57996a8be84d398f0f023e8d08ef77" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:21:35.687777 16217 raft_consensus.cc:3060] T af57996a8be84d398f0f023e8d08ef77 P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:21:35.650915 16117 leader_election.cc:304] T 434bff3082d141838068f323bdb0d1ed P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860
W20260709 13:21:35.691155 16192 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 48944.3us
I20260709 13:21:35.635349 21931 leader_election.cc:290] T 84f2da98864b4425bf7c0055951fdd4b P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:35.697727 16151 raft_consensus.cc:2468] T 2794af5b99984efebdee564035841cb0 P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 13.
I20260709 13:21:35.596810 16265 leader_election.cc:304] T 8d4ab870a05347868956aa8b841b2981 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 647a343a542d4a139fe106c19095129b
I20260709 13:21:35.698482 16226 raft_consensus.cc:1240] T d16dfad0c3b540e5b4b86492ddf87208 P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 15. Current term is 16. Ops: []
W20260709 13:21:35.698529 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 55300.4us
W20260709 13:21:35.702006 16789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 145615us
I20260709 13:21:35.706974 16190 leader_election.cc:304] T 5a959dbecc2648eb85d1cc550f5a3245 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:21:35.717072 16222 raft_consensus.cc:1240] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 16. Current term is 17. Ops: []
I20260709 13:21:35.706599 21997 raft_consensus.cc:2749] T 434bff3082d141838068f323bdb0d1ed P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:21:35.730376 16117 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 79628.8us
I20260709 13:21:35.731649 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f2da98864b4425bf7c0055951fdd4b" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:35.732204 16226 raft_consensus.cc:2468] T 84f2da98864b4425bf7c0055951fdd4b P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 12.
I20260709 13:21:35.707460 21769 consensus_queue.cc:1048] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.022s
W20260709 13:21:35.736129 16115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 172151us
I20260709 13:21:35.733709 16150 raft_consensus.cc:1275] T 9e65676b21b748728ad39668cfe722ec P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 13:21:35.735242 16218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cc959fb7f704fba9009c0595467d074" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:35.738251 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2372981006d4b218215c28964dea623" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:35.723839 21908 raft_consensus.cc:2764] T 8d4ab870a05347868956aa8b841b2981 P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Leader pre-election decision vote started in defunct term 14: won
I20260709 13:21:35.737548 16218 raft_consensus.cc:2468] T 8cc959fb7f704fba9009c0595467d074 P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 14.
I20260709 13:21:35.742957 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fffab26427e24d398e2695aad40ca134" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:21:35.744321 21930 raft_consensus.cc:515] T 02838c86531d4f98bf58d1c22b8baf77 P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:35.745112 16789 leader_election.cc:304] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 
W20260709 13:21:35.747453 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 174171us
I20260709 13:21:35.747917 22049 consensus_queue.cc:1048] T 9e65676b21b748728ad39668cfe722ec P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.015s
I20260709 13:21:35.780638 21930 leader_election.cc:290] T 02838c86531d4f98bf58d1c22b8baf77 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 16 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:35.782545 16219 raft_consensus.cc:1240] T 02838c86531d4f98bf58d1c22b8baf77 P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 15. Current term is 16. Ops: [15.9-15.9]
I20260709 13:21:35.752220 21997 consensus_queue.cc:1059] T d16dfad0c3b540e5b4b86492ddf87208 P 647a343a542d4a139fe106c19095129b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:21:35.753674 21934 raft_consensus.cc:515] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:35.757349 16192 leader_election.cc:304] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860
I20260709 13:21:35.758332 16115 leader_election.cc:304] T 23f82224f4564fea93572e8206bad1fe P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:21:35.727293 21946 raft_consensus.cc:2804] T 5a959dbecc2648eb85d1cc550f5a3245 P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:21:35.807053 21946 raft_consensus.cc:493] T 5a959dbecc2648eb85d1cc550f5a3245 P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:35.807623 21946 raft_consensus.cc:3060] T 5a959dbecc2648eb85d1cc550f5a3245 P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:21:35.813225 21930 raft_consensus.cc:3060] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:21:35.813617 16790 connection.cc:256] RPC call timeout handler was delayed by -2.7407542140001624s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2142s for a response
W20260709 13:21:35.814047 16790 connection.cc:256] RPC call timeout handler was delayed by -2.7101542140001129s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21760000000000002s for a response
W20260709 13:21:35.814392 16790 connection.cc:256] RPC call timeout handler was delayed by -2.1092339080000784s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1582s for a response
W20260709 13:21:35.814711 16790 connection.cc:256] RPC call timeout handler was delayed by -1.8493655480001507s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18810000000000002s for a response
W20260709 13:21:35.815004 16790 connection.cc:256] RPC call timeout handler was delayed by -1.7782655480002632s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.196s for a response
W20260709 13:21:35.815326 16790 connection.cc:256] RPC call timeout handler was delayed by -1.7514978589999828s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1754s for a response
W20260709 13:21:35.815629 16790 connection.cc:256] RPC call timeout handler was delayed by -1.7073978590001389s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18030000000000002s for a response
W20260709 13:21:35.815917 16790 connection.cc:256] RPC call timeout handler was delayed by -1.5923542140001246s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.34180000000000005s for a response
I20260709 13:21:35.797402 16220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e4bfb9090c8460eb14fa7e0d818e7f1" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 16 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:35.818343 16151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af57996a8be84d398f0f023e8d08ef77" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:21:35.819324 16151 raft_consensus.cc:2393] T af57996a8be84d398f0f023e8d08ef77 P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 in current term 17: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
W20260709 13:21:35.819360 16263 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:35.766745 16224 raft_consensus.cc:2468] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 19.
I20260709 13:21:35.767496 16217 raft_consensus.cc:2468] T af57996a8be84d398f0f023e8d08ef77 P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 17.
I20260709 13:21:35.768237 22036 raft_consensus.cc:515] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:35.771000 16789 leader_election.cc:304] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:21:35.778857 22041 raft_consensus.cc:2804] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:21:35.779887 16218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b7d1ab519044eda2d5e04a07242f25" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:21:35.801025 21934 leader_election.cc:290] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 15 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:35.802382 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3884f864a96d428785895e5463e5728f" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:21:35.806218 16223 raft_consensus.cc:2468] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 15.
I20260709 13:21:35.823431 16225 raft_consensus.cc:3060] T fffab26427e24d398e2695aad40ca134 P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:21:35.822902 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 88404.2us
I20260709 13:21:35.824939 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdbd78acb4cf46298e0de8ba0d130e19" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:35.826489 16790 leader_election.cc:304] T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:21:35.827134 22041 raft_consensus.cc:697] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 [term 19 LEADER]: Becoming Leader. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:21:35.867539 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "829b4ad8c5a54eababeea1181b960d4b" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:21:35.867931 22041 consensus_queue.cc:237] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.14, Last appended by leader: 14, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:35.868754 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada5a004ba7c46819051e3ccec71ffdb" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:21:35.869364 16217 raft_consensus.cc:3060] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:21:35.870150 16116 leader_election.cc:304] T 84f2da98864b4425bf7c0055951fdd4b P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
I20260709 13:21:35.870417 16790 leader_election.cc:304] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:21:35.870565 22025 consensus_queue.cc:1059] T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: INVALID_TERM, Last received: 16.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:21:35.871348 21579 raft_consensus.cc:2749] T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:21:35.871507 22025 raft_consensus.cc:2804] T 84f2da98864b4425bf7c0055951fdd4b P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:21:35.872041 22025 raft_consensus.cc:493] T 84f2da98864b4425bf7c0055951fdd4b P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Starting leader election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:21:35.872403 22025 raft_consensus.cc:3060] T 84f2da98864b4425bf7c0055951fdd4b P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:21:35.872680 21579 raft_consensus.cc:2749] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:21:35.828464 22036 leader_election.cc:290] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 16 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:35.829316 16218 raft_consensus.cc:2368] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 for earlier term 13. Current term is 14.
I20260709 13:21:35.874379 16218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8638ee363273426db1393dd81e5c7b72" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:21:35.829661 22000 raft_consensus.cc:3055] T d16dfad0c3b540e5b4b86492ddf87208 P 647a343a542d4a139fe106c19095129b [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:21:35.829526 16146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "829b4ad8c5a54eababeea1181b960d4b" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:21:35.875100 22000 raft_consensus.cc:740] T d16dfad0c3b540e5b4b86492ddf87208 P 647a343a542d4a139fe106c19095129b [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:21:35.871522 22044 raft_consensus.cc:3055] T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:21:35.875741 22044 raft_consensus.cc:740] T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:21:35.876142 22000 consensus_queue.cc:260] T d16dfad0c3b540e5b4b86492ddf87208 P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:35.876576 22044 consensus_queue.cc:260] T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:35.877787 22000 raft_consensus.cc:3060] T d16dfad0c3b540e5b4b86492ddf87208 P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:21:35.877884 22044 raft_consensus.cc:3060] T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:21:35.830374 22049 raft_consensus.cc:3060] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:21:35.830554 16151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434bff3082d141838068f323bdb0d1ed" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:35.835978 21704 raft_consensus.cc:515] T ee9ed67006144797b1de6ea8c2013a68 P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:35.881669 21704 leader_election.cc:290] T ee9ed67006144797b1de6ea8c2013a68 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 16 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
W20260709 13:21:35.838802 16192 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 81594.6us
I20260709 13:21:35.843569 16147 raft_consensus.cc:2393] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 in current term 16: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
W20260709 13:21:35.846040 16117 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 63629.8us
I20260709 13:21:35.882892 16192 leader_election.cc:400] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 20 election: Vote denied by peer cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923) with higher term. Message: Invalid argument: T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac for earlier term 20. Current term is 21.
I20260709 13:21:35.883277 16192 leader_election.cc:403] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 20 election: Cancelling election due to peer responding with higher term
I20260709 13:21:35.884383 16192 leader_election.cc:304] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860
I20260709 13:21:35.885176 22018 raft_consensus.cc:2749] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Leader election lost for term 20. Reason: Vote denied by peer cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923) with higher term. Message: Invalid argument: T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac for earlier term 20. Current term is 21.
I20260709 13:21:35.885845 22018 raft_consensus.cc:2749] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:21:35.854251 16265 leader_election.cc:304] T cdbd78acb4cf46298e0de8ba0d130e19 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860
I20260709 13:21:35.854898 21908 consensus_queue.cc:1048] T fffab26427e24d398e2695aad40ca134 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:21:35.881676 16790 leader_election.cc:304] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:21:35.861356 16220 raft_consensus.cc:2410] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 for term 16 because replica has last-logged OpId of term: 15 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 10.
I20260709 13:21:35.881852 21623 consensus_queue.cc:1048] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 7, Time since last communication: 0.000s
W20260709 13:21:35.848085 16115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 89885.8us
I20260709 13:21:35.887734 22048 raft_consensus.cc:2749] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:21:35.889176 16790 leader_election.cc:304] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:21:35.889647 16115 leader_election.cc:304] T 7c2095a5f4ad49408dbc7f69c606120d P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:21:35.848599 22024 raft_consensus.cc:2804] T 23f82224f4564fea93572e8206bad1fe P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:21:35.889655 16265 leader_election.cc:304] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:21:35.890347 22024 raft_consensus.cc:493] T 23f82224f4564fea93572e8206bad1fe P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Starting leader election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:21:35.889822 21908 raft_consensus.cc:2749] T cdbd78acb4cf46298e0de8ba0d130e19 P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:21:35.830641 16789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 59764.8us
I20260709 13:21:35.885609 16192 leader_election.cc:304] T ecc1e363230043fe9fef390f0b979f20 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860
I20260709 13:21:35.892168 16150 raft_consensus.cc:2410] T 3884f864a96d428785895e5463e5728f P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 for term 11 because replica has last-logged OpId of term: 10 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 10.
I20260709 13:21:35.894063 16225 raft_consensus.cc:2468] T fffab26427e24d398e2695aad40ca134 P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 16.
I20260709 13:21:35.890895 16115 leader_election.cc:304] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:21:35.899346 21931 consensus_queue.cc:1048] T 02838c86531d4f98bf58d1c22b8baf77 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:21:35.890864 22024 raft_consensus.cc:3060] T 23f82224f4564fea93572e8206bad1fe P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:21:35.913137 16225 raft_consensus.cc:1240] T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 14. Current term is 15. Ops: []
W20260709 13:21:35.913295 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 206395us
I20260709 13:21:35.865867 22001 raft_consensus.cc:515] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 647a343a542d4a139fe106c19095129b [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:35.890069 21623 raft_consensus.cc:2804] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:21:35.916040 16790 leader_election.cc:304] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac
I20260709 13:21:35.901916 21904 raft_consensus.cc:2749] T ecc1e363230043fe9fef390f0b979f20 P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:21:35.905396 22027 raft_consensus.cc:2749] T 7c2095a5f4ad49408dbc7f69c606120d P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:21:35.930533 16222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af57996a8be84d398f0f023e8d08ef77" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:21:35.932570 16222 raft_consensus.cc:2393] T af57996a8be84d398f0f023e8d08ef77 P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 in current term 17: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
I20260709 13:21:35.932435 16115 leader_election.cc:304] T 57a744f5aaf944678a63469cbf514015 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:21:35.928845 16190 leader_election.cc:304] T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:21:35.900664 21997 consensus_queue.cc:1048] T 5a959dbecc2648eb85d1cc550f5a3245 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.028s
I20260709 13:21:35.919144 21623 raft_consensus.cc:493] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:35.941586 16819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "829b4ad8c5a54eababeea1181b960d4b" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:35.942587 16819 raft_consensus.cc:2393] T 829b4ad8c5a54eababeea1181b960d4b P cb82cc7bd6db46d980c75237a1402860 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 15: Already voted for candidate cb82cc7bd6db46d980c75237a1402860 in this term.
I20260709 13:21:35.920897 22026 raft_consensus.cc:2804] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:21:35.919819 21908 raft_consensus.cc:2804] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:21:35.890885 22000 consensus_queue.cc:1048] T 8d4ab870a05347868956aa8b841b2981 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.046s
I20260709 13:21:35.952459 16824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5718c623ebee4d2bb770f0f9ddb2f164" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
W20260709 13:21:35.951644 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 62098.2us
I20260709 13:21:35.953389 16217 raft_consensus.cc:2468] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 17.
I20260709 13:21:35.980358 16818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66ae176961b74813a58bc15c9c4c6f2a" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:21:35.941576 21623 raft_consensus.cc:3060] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:21:35.977494 16817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a9de347c711477e87c582d003f36925" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:21:35.951336 22037 raft_consensus.cc:2804] T 57a744f5aaf944678a63469cbf514015 P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:21:35.953933 16221 raft_consensus.cc:1240] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 17. Current term is 18. Ops: [17.13-17.13]
I20260709 13:21:35.955376 16820 raft_consensus.cc:1275] T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860 [term 18 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 13:21:35.978375 16823 raft_consensus.cc:1240] T 74ca8140f35c4ae29e8a664f6bdfc829 P cb82cc7bd6db46d980c75237a1402860 [term 16 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 15. Current term is 16. Ops: []
I20260709 13:21:35.937639 21931 raft_consensus.cc:2804] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:21:35.981782 16825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e4bfb9090c8460eb14fa7e0d818e7f1" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:21:35.982753 16115 leader_election.cc:304] T 9c61a39fd4f84424b129c1a6f9ba404d P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac
I20260709 13:21:35.953164 21908 raft_consensus.cc:493] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:21:35.948318 22026 raft_consensus.cc:493] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:21:35.987637 22037 raft_consensus.cc:697] T 57a744f5aaf944678a63469cbf514015 P 647a343a542d4a139fe106c19095129b [term 12 LEADER]: Becoming Leader. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:21:36.008677 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada5a004ba7c46819051e3ccec71ffdb" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:21:35.951642 22057 raft_consensus.cc:2804] T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:21:35.990743 21946 raft_consensus.cc:515] T 5a959dbecc2648eb85d1cc550f5a3245 P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:35.991894 16296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31969392ccdb43b491911a319e8bc7e1" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:36.004321 21908 raft_consensus.cc:3060] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:21:36.011641 16296 raft_consensus.cc:2468] T 31969392ccdb43b491911a319e8bc7e1 P 90f3da57796b488c9087a2f523adf045 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 14.
I20260709 13:21:35.995492 21623 raft_consensus.cc:515] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:35.995481 22025 raft_consensus.cc:515] T 84f2da98864b4425bf7c0055951fdd4b P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:35.998489 22011 consensus_queue.cc:1059] T 57a744f5aaf944678a63469cbf514015 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
W20260709 13:21:35.999425 22040 log.cc:927] Time spent T 18ad7e3af5504571a69d10c00750a01c P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 13:21:35.938930 21920 consensus_queue.cc:1048] T 3a7ca61592e848ea9b0fb81a2ce33cdc P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.061s
I20260709 13:21:36.004843 16790 leader_election.cc:304] T 2794af5b99984efebdee564035841cb0 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860; no voters: 90f3da57796b488c9087a2f523adf045
I20260709 13:21:36.007723 22026 raft_consensus.cc:3060] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:21:36.025976 22011 raft_consensus.cc:3055] T 57a744f5aaf944678a63469cbf514015 P cb82cc7bd6db46d980c75237a1402860 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:21:36.026453 22011 raft_consensus.cc:740] T 57a744f5aaf944678a63469cbf514015 P cb82cc7bd6db46d980c75237a1402860 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:21:36.027330 22011 consensus_queue.cc:260] T 57a744f5aaf944678a63469cbf514015 P cb82cc7bd6db46d980c75237a1402860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:36.008060 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8638ee363273426db1393dd81e5c7b72" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:21:36.028681 22011 raft_consensus.cc:3060] T 57a744f5aaf944678a63469cbf514015 P cb82cc7bd6db46d980c75237a1402860 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:21:36.008646 22001 leader_election.cc:290] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 14 election: Requested vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:36.008759 16291 raft_consensus.cc:1275] T 8c11074579b046d99e585c2818e18a2c P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 13:21:35.990743 16295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434bff3082d141838068f323bdb0d1ed" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:36.020653 16298 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "23f82224f4564fea93572e8206bad1fe"
dest_uuid: "90f3da57796b488c9087a2f523adf045"
 from {username='slave'} at 127.0.0.1:40474
I20260709 13:21:36.031133 16298 raft_consensus.cc:493] T 23f82224f4564fea93572e8206bad1fe P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:21:36.011283 22048 consensus_queue.cc:1048] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.020s
I20260709 13:21:36.031587 16298 raft_consensus.cc:3060] T 23f82224f4564fea93572e8206bad1fe P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:21:36.033370 16790 leader_election.cc:304] T 3884f864a96d428785895e5463e5728f P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:21:36.033560 16226 raft_consensus.cc:1240] T 5a959dbecc2648eb85d1cc550f5a3245 P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 14. Current term is 15. Ops: [14.9-14.9]
I20260709 13:21:36.037470 21802 raft_consensus.cc:2804] T 2794af5b99984efebdee564035841cb0 P cb82cc7bd6db46d980c75237a1402860 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:21:36.021554 22057 raft_consensus.cc:697] T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac [term 15 LEADER]: Becoming Leader. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:21:36.039311 22057 consensus_queue.cc:237] T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:36.039585 22037 consensus_queue.cc:237] T 57a744f5aaf944678a63469cbf514015 P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:36.040539 21930 raft_consensus.cc:2749] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:21:36.042234 16298 raft_consensus.cc:515] T 23f82224f4564fea93572e8206bad1fe P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:36.043470 16116 leader_election.cc:304] T 8cc959fb7f704fba9009c0595467d074 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
I20260709 13:21:36.045626 22025 leader_election.cc:290] T 84f2da98864b4425bf7c0055951fdd4b P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 13 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:36.000958 21931 raft_consensus.cc:493] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Starting leader election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:21:36.053645 16225 raft_consensus.cc:1240] T 66f75124215f4ca6bef831e174a792d8 P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 14. Current term is 15. Ops: []
I20260709 13:21:36.078006 16218 raft_consensus.cc:2468] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 17.
I20260709 13:21:36.074596 21931 raft_consensus.cc:3060] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:21:36.087116 16115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 108219us
I20260709 13:21:36.087885 16265 leader_election.cc:304] T e5ad4eb281564936a59f41e387a47156 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:21:36.088238 16115 leader_election.cc:304] T 1df93779209748b9860f6d893dd3de53 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac
I20260709 13:21:36.089241 16265 leader_election.cc:304] T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:21:36.015707 22027 raft_consensus.cc:2804] T 9c61a39fd4f84424b129c1a6f9ba404d P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:21:36.092320 16116 leader_election.cc:304] T af57996a8be84d398f0f023e8d08ef77 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
I20260709 13:21:36.034690 21946 leader_election.cc:290] T 5a959dbecc2648eb85d1cc550f5a3245 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 15 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:36.097215 16825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b7d1ab519044eda2d5e04a07242f25" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:36.097471 16820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f82224f4564fea93572e8206bad1fe" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:36.098981 16820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66f75124215f4ca6bef831e174a792d8" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:36.100925 16820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31969392ccdb43b491911a319e8bc7e1" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:36.101897 16820 raft_consensus.cc:2393] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 15: Already voted for candidate cb82cc7bd6db46d980c75237a1402860 in this term.
I20260709 13:21:36.103494 16298 leader_election.cc:290] T 23f82224f4564fea93572e8206bad1fe P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 13 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:36.107106 22026 raft_consensus.cc:515] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:36.108227 16222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee9ed67006144797b1de6ea8c2013a68" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:21:36.108817 16222 raft_consensus.cc:3060] T ee9ed67006144797b1de6ea8c2013a68 P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:21:36.109450 22026 leader_election.cc:290] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 18 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:21:36.110508 16824 raft_consensus.cc:2468] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 15.
W20260709 13:21:36.111317 16114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:36.111521 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f2da98864b4425bf7c0055951fdd4b" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:21:36.118718 16225 raft_consensus.cc:3060] T 84f2da98864b4425bf7c0055951fdd4b P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:21:36.119367 16192 leader_election.cc:304] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:21:36.094779 22058 raft_consensus.cc:2804] T 8cc959fb7f704fba9009c0595467d074 P 647a343a542d4a139fe106c19095129b [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:21:36.115567 16144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a9de347c711477e87c582d003f36925" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:21:36.115739 16295 raft_consensus.cc:1240] T c9944d5e4b8c4109af9955c4f08c9095 P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 17. Current term is 18. Ops: []
I20260709 13:21:36.116052 22064 raft_consensus.cc:2749] T e5ad4eb281564936a59f41e387a47156 P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:21:36.116611 21623 leader_election.cc:290] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 15 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:36.116678 22043 raft_consensus.cc:493] T 746aabf856ab401c9ef5e859371e7b67 P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:21:36.116966 16790 leader_election.cc:304] T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:21:36.117445 16292 raft_consensus.cc:1240] T 57a744f5aaf944678a63469cbf514015 P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 11. Current term is 12. Ops: []
I20260709 13:21:36.127950 16820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:36.111095 22001 raft_consensus.cc:2804] T 1df93779209748b9860f6d893dd3de53 P 647a343a542d4a139fe106c19095129b [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:21:36.090281 22048 raft_consensus.cc:2749] T 3884f864a96d428785895e5463e5728f P cb82cc7bd6db46d980c75237a1402860 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:21:36.122787 16144 raft_consensus.cc:2393] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 19: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
I20260709 13:21:36.126538 16151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e65676b21b748728ad39668cfe722ec" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:36.149974 16151 raft_consensus.cc:2468] T 9e65676b21b748728ad39668cfe722ec P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 15.
I20260709 13:21:36.135036 16825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57a744f5aaf944678a63469cbf514015" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:21:36.145520 16147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 18 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:36.145651 22049 raft_consensus.cc:2749] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:21:36.146070 16299 raft_consensus.cc:1240] T 23f82224f4564fea93572e8206bad1fe P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 12. Current term is 13. Ops: []
I20260709 13:21:36.146183 16146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68a6838d82ef4cac83507f906139d90f" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:36.079779 21920 consensus_queue.cc:1048] T 9e65676b21b748728ad39668cfe722ec P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.047s
I20260709 13:21:36.127326 21704 raft_consensus.cc:2804] T af57996a8be84d398f0f023e8d08ef77 P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:21:36.147123 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5718c623ebee4d2bb770f0f9ddb2f164" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:21:36.147513 16824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 18 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:36.093281 21930 consensus_queue.cc:1048] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.057s
I20260709 13:21:36.153934 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2794af5b99984efebdee564035841cb0" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:21:36.152796 16145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31969392ccdb43b491911a319e8bc7e1" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:21:36.163321 16150 raft_consensus.cc:2393] T 2794af5b99984efebdee564035841cb0 P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 13: Already voted for candidate cb82cc7bd6db46d980c75237a1402860 in this term.
I20260709 13:21:36.163334 16145 raft_consensus.cc:2393] T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 14: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
I20260709 13:21:36.156273 16142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ad5dbfa99ee4b8d9e28be92b3f36db5" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:36.157301 16823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a9de347c711477e87c582d003f36925" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:21:36.122462 22058 raft_consensus.cc:493] T 8cc959fb7f704fba9009c0595467d074 P 647a343a542d4a139fe106c19095129b [term 14 FOLLOWER]: Starting leader election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:21:36.147564 22025 consensus_queue.cc:1059] T 02838c86531d4f98bf58d1c22b8baf77 P 647a343a542d4a139fe106c19095129b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.016s
I20260709 13:21:36.153671 16148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3884f864a96d428785895e5463e5728f" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:36.150595 21802 raft_consensus.cc:697] T 2794af5b99984efebdee564035841cb0 P cb82cc7bd6db46d980c75237a1402860 [term 13 LEADER]: Becoming Leader. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
W20260709 13:21:36.171734 16115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 83552.8us
I20260709 13:21:36.166471 16148 raft_consensus.cc:2468] T 3884f864a96d428785895e5463e5728f P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 11.
I20260709 13:21:36.155190 22029 raft_consensus.cc:2749] T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:21:36.132638 22027 raft_consensus.cc:697] T 9c61a39fd4f84424b129c1a6f9ba404d P 647a343a542d4a139fe106c19095129b [term 17 LEADER]: Becoming Leader. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:21:36.170908 16225 raft_consensus.cc:2468] T 84f2da98864b4425bf7c0055951fdd4b P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 13.
I20260709 13:21:36.128723 22059 raft_consensus.cc:2804] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:21:36.161641 16816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee9ed67006144797b1de6ea8c2013a68" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:36.177170 16819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af57996a8be84d398f0f023e8d08ef77" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:21:36.168496 16147 raft_consensus.cc:2468] T ecc1e363230043fe9fef390f0b979f20 P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 17.
W20260709 13:21:36.183507 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 94339.2us
I20260709 13:21:36.184064 16790 leader_election.cc:304] T 829b4ad8c5a54eababeea1181b960d4b P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
W20260709 13:21:36.180061 16117 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 61915.6us
I20260709 13:21:36.181607 16824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2372981006d4b218215c28964dea623" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:36.181726 16820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26354361e9b044cea1c2b86c4882fe92" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:36.160445 22001 raft_consensus.cc:697] T 1df93779209748b9860f6d893dd3de53 P 647a343a542d4a139fe106c19095129b [term 14 LEADER]: Becoming Leader. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:21:36.185395 16790 leader_election.cc:400] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 13 election: Vote denied by peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943) with higher term. Message: Invalid argument: T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 for earlier term 13. Current term is 14.
W20260709 13:21:36.178411 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 60913.7us
I20260709 13:21:36.186115 16116 leader_election.cc:304] T fffab26427e24d398e2695aad40ca134 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
I20260709 13:21:36.185961 22001 consensus_queue.cc:237] T 1df93779209748b9860f6d893dd3de53 P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:36.184078 16819 raft_consensus.cc:2393] T af57996a8be84d398f0f023e8d08ef77 P cb82cc7bd6db46d980c75237a1402860 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 17: Already voted for candidate cb82cc7bd6db46d980c75237a1402860 in this term.
I20260709 13:21:36.159364 22041 consensus_queue.cc:1059] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.024s
I20260709 13:21:36.168058 22058 raft_consensus.cc:3060] T 8cc959fb7f704fba9009c0595467d074 P 647a343a542d4a139fe106c19095129b [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:21:36.199311 22073 raft_consensus.cc:2749] T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:21:36.199673 21802 consensus_queue.cc:237] T 2794af5b99984efebdee564035841cb0 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:36.148708 22043 raft_consensus.cc:515] T 746aabf856ab401c9ef5e859371e7b67 P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:36.202526 15046 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:21:36.180487 22057 raft_consensus.cc:493] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:36.201109 16790 leader_election.cc:304] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
W20260709 13:21:36.200726 16266 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.14.177.132:38923 timed out after 1.669s (SENT) [suppressed 35 similar messages]
I20260709 13:21:36.184335 21704 raft_consensus.cc:697] T af57996a8be84d398f0f023e8d08ef77 P 647a343a542d4a139fe106c19095129b [term 17 LEADER]: Becoming Leader. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:21:36.205245 16816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5718c623ebee4d2bb770f0f9ddb2f164" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:21:36.189795 22031 raft_consensus.cc:2749] T 829b4ad8c5a54eababeea1181b960d4b P cb82cc7bd6db46d980c75237a1402860 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:21:36.214346 16816 raft_consensus.cc:3060] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:21:36.207015 16265 leader_election.cc:304] T 8158bbd17b994d2eb96f560bd0f1b88d P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:21:36.207484 21997 raft_consensus.cc:3055] T 02838c86531d4f98bf58d1c22b8baf77 P 647a343a542d4a139fe106c19095129b [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:21:36.217885 22029 raft_consensus.cc:3055] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:21:36.190099 22059 raft_consensus.cc:697] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac [term 15 LEADER]: Becoming Leader. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
W20260709 13:21:36.224715 16266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 201771us
I20260709 13:21:36.226145 16222 raft_consensus.cc:2468] T ee9ed67006144797b1de6ea8c2013a68 P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 16.
I20260709 13:21:36.226879 16818 raft_consensus.cc:2468] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 20.
I20260709 13:21:36.200829 22036 raft_consensus.cc:2804] T fffab26427e24d398e2695aad40ca134 P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:21:36.232661 16300 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "5a96373bb0e14dce8a266b2fc6c58507"
dest_uuid: "90f3da57796b488c9087a2f523adf045"
 from {username='slave'} at 127.0.0.1:48674
I20260709 13:21:36.233299 16300 raft_consensus.cc:493] T 5a96373bb0e14dce8a266b2fc6c58507 P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:21:36.233722 16300 raft_consensus.cc:3060] T 5a96373bb0e14dce8a266b2fc6c58507 P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:21:36.234728 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26354361e9b044cea1c2b86c4882fe92" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:36.236445 16295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cc959fb7f704fba9009c0595467d074" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:36.239996 16292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fffab26427e24d398e2695aad40ca134" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:21:36.209491 21931 raft_consensus.cc:515] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:36.242905 22067 consensus_queue.cc:1048] T 3a7ca61592e848ea9b0fb81a2ce33cdc P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.079s
I20260709 13:21:36.239526 16819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee9ed67006144797b1de6ea8c2013a68" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:21:36.244082 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806d7690c2b644959152074bb2cab0a7" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 17 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:36.252348 16293 raft_consensus.cc:2410] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b for term 17 because replica has last-logged OpId of term: 14 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 9.
I20260709 13:21:36.244720 16294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f2da98864b4425bf7c0055951fdd4b" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:36.228230 22026 raft_consensus.cc:3060] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P cb82cc7bd6db46d980c75237a1402860 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:21:36.245337 21931 leader_election.cc:290] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 18 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:36.257786 16117 leader_election.cc:304] T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860
I20260709 13:21:36.223894 21997 raft_consensus.cc:740] T 02838c86531d4f98bf58d1c22b8baf77 P 647a343a542d4a139fe106c19095129b [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:21:36.228109 22029 raft_consensus.cc:740] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
W20260709 13:21:36.271667 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 82747us
I20260709 13:21:36.272739 16790 leader_election.cc:304] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 647a343a542d4a139fe106c19095129b
I20260709 13:21:36.279908 16819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e9b2bf7011f4ae688a79d7bb5a447b4" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:21:36.284948 16789 leader_election.cc:304] T 434bff3082d141838068f323bdb0d1ed P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:21:36.267721 22036 raft_consensus.cc:697] T fffab26427e24d398e2695aad40ca134 P 647a343a542d4a139fe106c19095129b [term 16 LEADER]: Becoming Leader. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:21:36.301766 16816 raft_consensus.cc:2468] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 16.
I20260709 13:21:36.303519 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e9b2bf7011f4ae688a79d7bb5a447b4" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:21:36.310487 16790 leader_election.cc:304] T af57996a8be84d398f0f023e8d08ef77 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:21:36.242372 22000 consensus_queue.cc:1059] T 8d4ab870a05347868956aa8b841b2981 P 647a343a542d4a139fe106c19095129b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.050s
I20260709 13:21:36.241613 22044 consensus_queue.cc:1059] T 829b4ad8c5a54eababeea1181b960d4b P 647a343a542d4a139fe106c19095129b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: INVALID_TERM, Last received: 14.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.051s
W20260709 13:21:36.314747 16117 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 57103.5us
I20260709 13:21:36.227820 22057 raft_consensus.cc:515] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } }
I20260709 13:21:36.247148 22024 raft_consensus.cc:515] T 23f82224f4564fea93572e8206bad1fe P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:36.324159 16790 leader_election.cc:304] T ada5a004ba7c46819051e3ccec71ffdb P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 90f3da57796b488c9087a2f523adf045
I20260709 13:21:36.303088 22067 raft_consensus.cc:2764] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 [term 16 FOLLOWER]: Leader pre-election decision vote started in defunct term 15: won
I20260709 13:21:36.327383 22069 raft_consensus.cc:2749] T af57996a8be84d398f0f023e8d08ef77 P cb82cc7bd6db46d980c75237a1402860 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:21:36.343145 22043 leader_election.cc:290] T 746aabf856ab401c9ef5e859371e7b67 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:36.344436 16292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f2da98864b4425bf7c0055951fdd4b" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:21:36.250229 22027 consensus_queue.cc:237] T 9c61a39fd4f84424b129c1a6f9ba404d P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:36.298528 21920 raft_consensus.cc:2749] T 434bff3082d141838068f323bdb0d1ed P cb82cc7bd6db46d980c75237a1402860 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:21:36.410079 16263 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:36.341238 16300 raft_consensus.cc:515] T 5a96373bb0e14dce8a266b2fc6c58507 P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:21:36.414784 22070 log.cc:927] Time spent T 57a744f5aaf944678a63469cbf514015 P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 0.163s	user 0.001s	sys 0.000s
I20260709 13:21:36.346809 22059 consensus_queue.cc:237] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 14.13, Last appended by leader: 13, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:36.416353 21997 consensus_queue.cc:260] T 02838c86531d4f98bf58d1c22b8baf77 P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:36.416965 16117 leader_election.cc:304] T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860
I20260709 13:21:36.292894 21704 consensus_queue.cc:237] T af57996a8be84d398f0f023e8d08ef77 P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:36.419098 22069 consensus_queue.cc:1048] T 8c11074579b046d99e585c2818e18a2c P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:21:36.422042 16817 raft_consensus.cc:3060] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:21:36.425102 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31969392ccdb43b491911a319e8bc7e1" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:21:36.483368 16816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e9b2bf7011f4ae688a79d7bb5a447b4" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:36.483912 16293 raft_consensus.cc:3060] T 31969392ccdb43b491911a319e8bc7e1 P 90f3da57796b488c9087a2f523adf045 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:21:36.431753 22064 raft_consensus.cc:2804] T 8158bbd17b994d2eb96f560bd0f1b88d P 90f3da57796b488c9087a2f523adf045 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:21:36.462795 16294 raft_consensus.cc:1240] T 5a96373bb0e14dce8a266b2fc6c58507 P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 12. Current term is 13. Ops: []
W20260709 13:21:36.463929 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 51414.1us
W20260709 13:21:36.464471 16266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 222160us
I20260709 13:21:36.474812 16818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31969392ccdb43b491911a319e8bc7e1" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:21:36.487820 16818 raft_consensus.cc:2368] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 for earlier term 14. Current term is 15.
W20260709 13:21:36.476750 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 190566us
I20260709 13:21:36.485664 21931 raft_consensus.cc:3055] T 8d4ab870a05347868956aa8b841b2981 P 647a343a542d4a139fe106c19095129b [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:21:36.410439 22041 raft_consensus.cc:2804] T ada5a004ba7c46819051e3ccec71ffdb P cb82cc7bd6db46d980c75237a1402860 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:21:36.408869 22057 leader_election.cc:290] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:36.489349 16300 leader_election.cc:290] T 5a96373bb0e14dce8a266b2fc6c58507 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 13 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:21:36.492264 16820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62fe3346cd3c42c4a55e14af9073bca4" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:21:36.508041 16821 raft_consensus.cc:1275] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [term 21 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
I20260709 13:21:36.508165 16819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af57996a8be84d398f0f023e8d08ef77" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:36.464318 22018 raft_consensus.cc:493] T 1df93779209748b9860f6d893dd3de53 P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:36.511678 16819 raft_consensus.cc:2393] T af57996a8be84d398f0f023e8d08ef77 P cb82cc7bd6db46d980c75237a1402860 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 17: Already voted for candidate cb82cc7bd6db46d980c75237a1402860 in this term.
I20260709 13:21:36.519066 16265 leader_election.cc:304] T 7c2095a5f4ad49408dbc7f69c606120d P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:21:36.519716 16823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
W20260709 13:21:36.521255 16115 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:36.518599 16266 leader_election.cc:304] T 3884f864a96d428785895e5463e5728f P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 
W20260709 13:21:36.530650 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 253650us
W20260709 13:21:36.543900 16117 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 127005us
I20260709 13:21:36.493196 22000 raft_consensus.cc:3060] T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:21:36.490761 22064 raft_consensus.cc:493] T 8158bbd17b994d2eb96f560bd0f1b88d P 90f3da57796b488c9087a2f523adf045 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:21:36.473968 22031 consensus_queue.cc:1059] T c9944d5e4b8c4109af9955c4f08c9095 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: INVALID_TERM, Last received: 17.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.064s
W20260709 13:21:36.546943 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 57790us
I20260709 13:21:36.491807 21931 raft_consensus.cc:740] T 8d4ab870a05347868956aa8b841b2981 P 647a343a542d4a139fe106c19095129b [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:21:36.555327 16117 leader_election.cc:304] T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860
I20260709 13:21:36.561425 16300 raft_consensus.cc:1240] T 8d4ab870a05347868956aa8b841b2981 P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 14. Current term is 15. Ops: [14.8-14.8]
I20260709 13:21:36.508090 22074 raft_consensus.cc:3060] T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:21:36.482928 22067 consensus_queue.cc:1059] T 23f82224f4564fea93572e8206bad1fe P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.072s
I20260709 13:21:36.519398 22041 raft_consensus.cc:697] T ada5a004ba7c46819051e3ccec71ffdb P cb82cc7bd6db46d980c75237a1402860 [term 17 LEADER]: Becoming Leader. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:21:36.548382 21997 raft_consensus.cc:3060] T 02838c86531d4f98bf58d1c22b8baf77 P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:21:36.570326 22064 raft_consensus.cc:3060] T 8158bbd17b994d2eb96f560bd0f1b88d P 90f3da57796b488c9087a2f523adf045 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:21:36.464329 22029 consensus_queue.cc:260] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.12, Last appended by leader: 12, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:36.568372 22030 consensus_queue.cc:1059] T 66f75124215f4ca6bef831e174a792d8 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: INVALID_TERM, Last received: 14.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.046s
I20260709 13:21:36.533507 22018 raft_consensus.cc:515] T 1df93779209748b9860f6d893dd3de53 P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:21:36.616607 16117 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 70670.2us
I20260709 13:21:36.528152 22036 consensus_queue.cc:237] T fffab26427e24d398e2695aad40ca134 P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:36.583640 21579 consensus_queue.cc:1059] T c9944d5e4b8c4109af9955c4f08c9095 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.044s
I20260709 13:21:36.651523 16823 raft_consensus.cc:1240] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 [term 18 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 17. Current term is 18. Ops: []
I20260709 13:21:36.623955 21245 raft_consensus.cc:3055] T 829b4ad8c5a54eababeea1181b960d4b P 647a343a542d4a139fe106c19095129b [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:21:36.614815 22058 raft_consensus.cc:515] T 8cc959fb7f704fba9009c0595467d074 P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:21:36.669406 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 82260.3us
I20260709 13:21:36.653384 22031 raft_consensus.cc:3055] T 23f82224f4564fea93572e8206bad1fe P cb82cc7bd6db46d980c75237a1402860 [term 12 LEADER]: Stepping down as leader of term 12
W20260709 13:21:36.678021 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 159090us
W20260709 13:21:36.679307 16117 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 58356us
I20260709 13:21:36.658403 22029 raft_consensus.cc:3060] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:21:36.670353 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 117704us
I20260709 13:21:36.683120 16117 leader_election.cc:304] T ecc1e363230043fe9fef390f0b979f20 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860
I20260709 13:21:36.663774 21245 raft_consensus.cc:740] T 829b4ad8c5a54eababeea1181b960d4b P 647a343a542d4a139fe106c19095129b [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:21:36.659147 22088 raft_consensus.cc:3060] T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:21:36.670243 22085 raft_consensus.cc:2804] T 7c2095a5f4ad49408dbc7f69c606120d P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:21:36.621469 22082 consensus_queue.cc:1048] T 84f2da98864b4425bf7c0055951fdd4b P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.062s
I20260709 13:21:36.688277 16790 leader_election.cc:304] T 8638ee363273426db1393dd81e5c7b72 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 90f3da57796b488c9087a2f523adf045
I20260709 13:21:36.676087 22031 raft_consensus.cc:740] T 23f82224f4564fea93572e8206bad1fe P cb82cc7bd6db46d980c75237a1402860 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:21:36.734733 16817 raft_consensus.cc:2468] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 19.
W20260709 13:21:36.756562 16117 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 73526.5us
I20260709 13:21:36.712966 21245 consensus_queue.cc:260] T 829b4ad8c5a54eababeea1181b960d4b P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:36.674611 22024 leader_election.cc:290] T 23f82224f4564fea93572e8206bad1fe P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 13 election: Requested vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:36.633090 21931 consensus_queue.cc:260] T 8d4ab870a05347868956aa8b841b2981 P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:36.749804 16116 leader_election.cc:304] T 84f2da98864b4425bf7c0055951fdd4b P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
I20260709 13:21:36.711560 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02838c86531d4f98bf58d1c22b8baf77" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:36.755651 16147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b7d1ab519044eda2d5e04a07242f25" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:21:36.764683 16142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f5f89186204242bf377f7c42b2e2da" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 18 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:36.757385 16117 leader_election.cc:304] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860
I20260709 13:21:36.674551 21946 consensus_queue.cc:1059] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: INVALID_TERM, Last received: 17.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.079s
W20260709 13:21:36.765902 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 77708.9us
I20260709 13:21:36.764904 16116 leader_election.cc:304] T ee9ed67006144797b1de6ea8c2013a68 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
I20260709 13:21:36.750867 22044 raft_consensus.cc:2749] T ecc1e363230043fe9fef390f0b979f20 P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:21:36.798635 16117 leader_election.cc:304] T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860
I20260709 13:21:36.799748 16142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62fe3346cd3c42c4a55e14af9073bca4" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:21:36.800207 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2372981006d4b218215c28964dea623" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:21:36.788425 22067 raft_consensus.cc:2804] T 8638ee363273426db1393dd81e5c7b72 P cb82cc7bd6db46d980c75237a1402860 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:21:36.789870 22024 raft_consensus.cc:2804] T 84f2da98864b4425bf7c0055951fdd4b P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:21:36.821120 16226 raft_consensus.cc:1275] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
W20260709 13:21:36.818675 16266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 310044us
I20260709 13:21:36.793186 21931 raft_consensus.cc:3060] T 8d4ab870a05347868956aa8b841b2981 P 647a343a542d4a139fe106c19095129b [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:21:36.824819 22024 raft_consensus.cc:697] T 84f2da98864b4425bf7c0055951fdd4b P 647a343a542d4a139fe106c19095129b [term 13 LEADER]: Becoming Leader. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:21:36.826814 22083 raft_consensus.cc:2804] T 3884f864a96d428785895e5463e5728f P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 13:21:36.827222 22086 log.cc:927] Time spent T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.105s	user 0.000s	sys 0.001s
I20260709 13:21:36.814759 22027 raft_consensus.cc:2804] T ee9ed67006144797b1de6ea8c2013a68 P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:21:36.689378 22041 consensus_queue.cc:237] T ada5a004ba7c46819051e3ccec71ffdb P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:36.833539 16142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c61a39fd4f84424b129c1a6f9ba404d" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:21:36.855365 16296 raft_consensus.cc:1275] T cdbd78acb4cf46298e0de8ba0d130e19 P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 13:21:36.858912 16147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a96373bb0e14dce8a266b2fc6c58507" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:36.836937 16117 leader_election.cc:304] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:21:36.800966 21930 raft_consensus.cc:493] T c0eab25ce4c64e018a041beeda2867db P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:21:36.849830 16293 raft_consensus.cc:2468] T 31969392ccdb43b491911a319e8bc7e1 P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 15.
W20260709 13:21:36.831744 22078 log.cc:927] Time spent T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 0.156s	user 0.000s	sys 0.000s
I20260709 13:21:36.803714 22097 raft_consensus.cc:2749] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:21:36.825354 22044 raft_consensus.cc:2749] T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:21:36.859911 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 58547.4us
I20260709 13:21:36.869900 16265 leader_election.cc:304] T 5a959dbecc2648eb85d1cc550f5a3245 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:21:36.870734 16148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66f75124215f4ca6bef831e174a792d8" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:21:36.871335 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a878999c0024d2abe517484180290fc" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
W20260709 13:21:36.872596 22095 log.cc:927] Time spent T 18ad7e3af5504571a69d10c00750a01c P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.080s	user 0.000s	sys 0.000s
I20260709 13:21:36.872732 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a959dbecc2648eb85d1cc550f5a3245" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:36.873791 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87ac0d1f7e8a419f9bcbc04aee0e067e" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:36.874797 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cc959fb7f704fba9009c0595467d074" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:36.875499 16150 raft_consensus.cc:2393] T 8cc959fb7f704fba9009c0595467d074 P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 15: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
I20260709 13:21:36.876538 16151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f2da98864b4425bf7c0055951fdd4b" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:36.841789 22083 raft_consensus.cc:493] T 3884f864a96d428785895e5463e5728f P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:36.851274 22067 raft_consensus.cc:697] T 8638ee363273426db1393dd81e5c7b72 P cb82cc7bd6db46d980c75237a1402860 [term 17 LEADER]: Becoming Leader. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:21:36.882893 16151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5718c623ebee4d2bb770f0f9ddb2f164" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:21:36.854303 22018 leader_election.cc:290] T 1df93779209748b9860f6d893dd3de53 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:36.893487 16145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 18 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:36.923715 16144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d4ab870a05347868956aa8b841b2981" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:21:36.865511 22027 raft_consensus.cc:697] T ee9ed67006144797b1de6ea8c2013a68 P 647a343a542d4a139fe106c19095129b [term 16 LEADER]: Becoming Leader. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:21:36.927147 16147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4f65cdf00b748bb963ee6e81ef43b79" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:36.877792 21704 raft_consensus.cc:2804] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:21:36.939576 21704 raft_consensus.cc:697] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b [term 16 LEADER]: Becoming Leader. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:21:36.939361 22024 consensus_queue.cc:237] T 84f2da98864b4425bf7c0055951fdd4b P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:36.940547 21704 consensus_queue.cc:237] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.13, Last appended by leader: 13, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:36.934026 16147 raft_consensus.cc:2468] T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 17.
W20260709 13:21:36.881937 22026 consensus_meta.cc:287] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P cb82cc7bd6db46d980c75237a1402860: Time spent flushing consensus metadata: real 0.614s	user 0.009s	sys 0.002s
I20260709 13:21:36.924477 16266 leader_election.cc:304] T 66ae176961b74813a58bc15c9c4c6f2a P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:21:36.933547 16142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d16dfad0c3b540e5b4b86492ddf87208" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
W20260709 13:21:36.937134 21908 consensus_meta.cc:287] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045: Time spent flushing consensus metadata: real 0.844s	user 0.014s	sys 0.010s
I20260709 13:21:36.937361 16145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "829b4ad8c5a54eababeea1181b960d4b" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:36.938284 21802 raft_consensus.cc:3055] T c9944d5e4b8c4109af9955c4f08c9095 P cb82cc7bd6db46d980c75237a1402860 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:21:36.886461 22085 raft_consensus.cc:697] T 7c2095a5f4ad49408dbc7f69c606120d P 90f3da57796b488c9087a2f523adf045 [term 12 LEADER]: Becoming Leader. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:21:36.882409 22083 raft_consensus.cc:3060] T 3884f864a96d428785895e5463e5728f P 90f3da57796b488c9087a2f523adf045 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:21:36.937297 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4f65cdf00b748bb963ee6e81ef43b79" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:21:36.823752 22031 consensus_queue.cc:260] T 23f82224f4564fea93572e8206bad1fe P cb82cc7bd6db46d980c75237a1402860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:36.948444 16266 leader_election.cc:400] T 31969392ccdb43b491911a319e8bc7e1 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 14 election: Vote denied by peer cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923) with higher term. Message: Invalid argument: T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 for earlier term 14. Current term is 15.
I20260709 13:21:36.959426 16266 leader_election.cc:403] T 31969392ccdb43b491911a319e8bc7e1 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 14 election: Cancelling election due to peer responding with higher term
W20260709 13:21:36.960201 16788 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:36.921876 21930 raft_consensus.cc:515] T c0eab25ce4c64e018a041beeda2867db P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:36.965137 16147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1df93779209748b9860f6d893dd3de53" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:36.946480 22058 leader_election.cc:290] T 8cc959fb7f704fba9009c0595467d074 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 15 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:36.888748 22001 consensus_queue.cc:1048] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.073s
I20260709 13:21:36.947817 21802 raft_consensus.cc:740] T c9944d5e4b8c4109af9955c4f08c9095 P cb82cc7bd6db46d980c75237a1402860 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
W20260709 13:21:36.969210 16117 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 86724us
I20260709 13:21:36.969466 22073 raft_consensus.cc:2749] T 31969392ccdb43b491911a319e8bc7e1 P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Leader election lost for term 14. Reason: Vote denied by peer cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923) with higher term. Message: Invalid argument: T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 for earlier term 14. Current term is 15.
I20260709 13:21:36.969369 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4f65cdf00b748bb963ee6e81ef43b79" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
W20260709 13:21:36.969345 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 204508us
I20260709 13:21:36.980819 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87ac0d1f7e8a419f9bcbc04aee0e067e" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
W20260709 13:21:36.987236 16117 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:36.947584 22026 raft_consensus.cc:2749] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P cb82cc7bd6db46d980c75237a1402860 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:21:36.986023 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 62985us
I20260709 13:21:36.981675 16296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cc959fb7f704fba9009c0595467d074" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
W20260709 13:21:36.980192 22011 consensus_meta.cc:287] T 57a744f5aaf944678a63469cbf514015 P cb82cc7bd6db46d980c75237a1402860: Time spent flushing consensus metadata: real 0.951s	user 0.013s	sys 0.001s
I20260709 13:21:36.955935 21908 raft_consensus.cc:515] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:36.989969 16147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806d7690c2b644959152074bb2cab0a7" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:37.025776 16147 raft_consensus.cc:2468] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 16.
I20260709 13:21:37.023602 16143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02838c86531d4f98bf58d1c22b8baf77" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:21:36.990087 16820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f82224f4564fea93572e8206bad1fe" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:21:37.026276 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:21:36.990720 16818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746aabf856ab401c9ef5e859371e7b67" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:36.991313 16821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a9de347c711477e87c582d003f36925" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:21:36.993832 16294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806d7690c2b644959152074bb2cab0a7" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:37.027588 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cc959fb7f704fba9009c0595467d074" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:21:37.028262 16225 raft_consensus.cc:3060] T 8cc959fb7f704fba9009c0595467d074 P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:21:36.995311 22093 log.cc:927] Time spent T 1df93779209748b9860f6d893dd3de53 P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 0.230s	user 0.001s	sys 0.001s
I20260709 13:21:36.995379 22067 consensus_queue.cc:237] T 8638ee363273426db1393dd81e5c7b72 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.13, Last appended by leader: 13, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:36.998134 22018 raft_consensus.cc:493] T b6f07842d3aa47858df5665874c86f3a P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:21:37.003322 22104 raft_consensus.cc:2804] T 66ae176961b74813a58bc15c9c4c6f2a P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:21:36.983129 22090 consensus_queue.cc:1048] T 8638ee363273426db1393dd81e5c7b72 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.181s
I20260709 13:21:36.938330 22027 consensus_queue.cc:237] T ee9ed67006144797b1de6ea8c2013a68 P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:37.007263 22106 raft_consensus.cc:2804] T 5a959dbecc2648eb85d1cc550f5a3245 P 90f3da57796b488c9087a2f523adf045 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:21:37.009018 16824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f82224f4564fea93572e8206bad1fe" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: true dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:21:37.009503 16297 raft_consensus.cc:2468] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 16.
I20260709 13:21:37.008431 16823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74ca8140f35c4ae29e8a664f6bdfc829" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
W20260709 13:21:37.030344 22101 log.cc:927] Time spent T 5a959dbecc2648eb85d1cc550f5a3245 P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.149s	user 0.001s	sys 0.000s
I20260709 13:21:37.032249 22000 raft_consensus.cc:2749] T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:21:37.033705 16221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746aabf856ab401c9ef5e859371e7b67" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:37.035470 16224 raft_consensus.cc:1275] T 1df93779209748b9860f6d893dd3de53 P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 13:21:37.037015 16222 raft_consensus.cc:1275] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
I20260709 13:21:37.032298 16226 raft_consensus.cc:1275] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 13:21:37.028095 16294 raft_consensus.cc:2468] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 16.
I20260709 13:21:37.039073 16150 raft_consensus.cc:2468] T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 17.
W20260709 13:21:37.043247 16116 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:37.049384 16292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a959dbecc2648eb85d1cc550f5a3245" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:21:37.052003 16823 raft_consensus.cc:2376] T 74ca8140f35c4ae29e8a664f6bdfc829 P cb82cc7bd6db46d980c75237a1402860 [term 16 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 16. Re-sending same reply.
I20260709 13:21:37.055932 16265 leader_election.cc:304] T c9944d5e4b8c4109af9955c4f08c9095 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860
I20260709 13:21:37.057809 16147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a959dbecc2648eb85d1cc550f5a3245" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:21:37.062740 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02838c86531d4f98bf58d1c22b8baf77" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:21:37.064952 16291 raft_consensus.cc:1240] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 14. Current term is 15. Ops: []
I20260709 13:21:37.069638 16225 raft_consensus.cc:2468] T 8cc959fb7f704fba9009c0595467d074 P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 15.
I20260709 13:21:37.076269 16292 raft_consensus.cc:1275] T 8d4ab870a05347868956aa8b841b2981 P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:21:37.036026 22001 consensus_queue.cc:1059] T 5a959dbecc2648eb85d1cc550f5a3245 P 647a343a542d4a139fe106c19095129b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.047s
I20260709 13:21:37.085402 16224 raft_consensus.cc:1240] T ee9ed67006144797b1de6ea8c2013a68 P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 15. Current term is 16. Ops: []
I20260709 13:21:37.058712 22106 raft_consensus.cc:493] T 5a959dbecc2648eb85d1cc550f5a3245 P 90f3da57796b488c9087a2f523adf045 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:21:37.061925 22104 raft_consensus.cc:697] T 66ae176961b74813a58bc15c9c4c6f2a P 90f3da57796b488c9087a2f523adf045 [term 20 LEADER]: Becoming Leader. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:21:37.100283 21908 leader_election.cc:290] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 15 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:21:37.100566 22104 consensus_queue.cc:237] T 66ae176961b74813a58bc15c9c4c6f2a P 90f3da57796b488c9087a2f523adf045 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:37.102937 22094 raft_consensus.cc:3055] T 66f75124215f4ca6bef831e174a792d8 P cb82cc7bd6db46d980c75237a1402860 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:21:37.103408 22094 raft_consensus.cc:740] T 66f75124215f4ca6bef831e174a792d8 P cb82cc7bd6db46d980c75237a1402860 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:21:37.104305 22094 consensus_queue.cc:260] T 66f75124215f4ca6bef831e174a792d8 P cb82cc7bd6db46d980c75237a1402860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:37.105192 22030 consensus_queue.cc:1048] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:21:37.035638 21802 consensus_queue.cc:260] T c9944d5e4b8c4109af9955c4f08c9095 P cb82cc7bd6db46d980c75237a1402860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:21:37.095945 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 55294.4us
I20260709 13:21:37.093122 22106 raft_consensus.cc:3060] T 5a959dbecc2648eb85d1cc550f5a3245 P 90f3da57796b488c9087a2f523adf045 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:21:37.062182 22085 consensus_queue.cc:237] T 7c2095a5f4ad49408dbc7f69c606120d P 90f3da57796b488c9087a2f523adf045 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:37.098055 22001 raft_consensus.cc:3055] T 5a959dbecc2648eb85d1cc550f5a3245 P 647a343a542d4a139fe106c19095129b [term 14 LEADER]: Stepping down as leader of term 14
W20260709 13:21:37.098713 22092 log.cc:927] Time spent T 9c61a39fd4f84424b129c1a6f9ba404d P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 0.281s	user 0.002s	sys 0.001s
I20260709 13:21:37.099332 22073 raft_consensus.cc:2749] T c9944d5e4b8c4109af9955c4f08c9095 P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:21:37.087713 22088 raft_consensus.cc:2749] T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:21:37.112464 16825 raft_consensus.cc:2468] T 57a744f5aaf944678a63469cbf514015 P cb82cc7bd6db46d980c75237a1402860 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 12.
W20260709 13:21:37.145944 22112 log.cc:927] Time spent T 3a7ca61592e848ea9b0fb81a2ce33cdc P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 0.069s	user 0.000s	sys 0.000s
I20260709 13:21:37.147755 22100 consensus_queue.cc:1059] T ee9ed67006144797b1de6ea8c2013a68 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:21:37.148798 22100 raft_consensus.cc:3055] T ee9ed67006144797b1de6ea8c2013a68 P cb82cc7bd6db46d980c75237a1402860 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:21:37.149147 22100 raft_consensus.cc:740] T ee9ed67006144797b1de6ea8c2013a68 P cb82cc7bd6db46d980c75237a1402860 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
W20260709 13:21:37.149893 22103 log.cc:927] Time spent T 02838c86531d4f98bf58d1c22b8baf77 P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.202s	user 0.001s	sys 0.000s
I20260709 13:21:37.123323 22001 raft_consensus.cc:740] T 5a959dbecc2648eb85d1cc550f5a3245 P 647a343a542d4a139fe106c19095129b [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:21:37.152482 22001 consensus_queue.cc:260] T 5a959dbecc2648eb85d1cc550f5a3245 P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:37.154276 22001 raft_consensus.cc:3060] T 5a959dbecc2648eb85d1cc550f5a3245 P 647a343a542d4a139fe106c19095129b [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:21:37.052047 22018 raft_consensus.cc:515] T b6f07842d3aa47858df5665874c86f3a P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:37.159947 16142 raft_consensus.cc:2468] T d16dfad0c3b540e5b4b86492ddf87208 P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 16.
I20260709 13:21:37.165686 16265 leader_election.cc:304] T 18ad7e3af5504571a69d10c00750a01c P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
W20260709 13:21:37.185776 16116 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:21:37.187892 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 79785.9us
I20260709 13:21:37.192595 16116 leader_election.cc:304] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860
W20260709 13:21:37.183624 21997 consensus_meta.cc:287] T 02838c86531d4f98bf58d1c22b8baf77 P 647a343a542d4a139fe106c19095129b: Time spent flushing consensus metadata: real 0.571s	user 0.010s	sys 0.003s
I20260709 13:21:37.203701 16149 raft_consensus.cc:2410] T 02838c86531d4f98bf58d1c22b8baf77 P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac for term 16 because replica has last-logged OpId of term: 15 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 8.
W20260709 13:21:37.263067 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 73394.4us
I20260709 13:21:37.191191 22094 raft_consensus.cc:3060] T 66f75124215f4ca6bef831e174a792d8 P cb82cc7bd6db46d980c75237a1402860 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:21:37.186883 22090 raft_consensus.cc:2804] T 18ad7e3af5504571a69d10c00750a01c P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:21:37.221062 21997 raft_consensus.cc:2749] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:21:37.150266 22100 consensus_queue.cc:260] T ee9ed67006144797b1de6ea8c2013a68 P cb82cc7bd6db46d980c75237a1402860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.13, Last appended by leader: 13, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:37.271955 22090 raft_consensus.cc:493] T 18ad7e3af5504571a69d10c00750a01c P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:21:37.296018 16148 raft_consensus.cc:2468] T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 15.
I20260709 13:21:37.268939 21704 consensus_queue.cc:1048] T 1df93779209748b9860f6d893dd3de53 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.018s
I20260709 13:21:37.286048 22119 raft_consensus.cc:493] T b87147731a68400e8cce9b879daa7c5d P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:21:37.296604 22090 raft_consensus.cc:3060] T 18ad7e3af5504571a69d10c00750a01c P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:21:37.297677 22059 raft_consensus.cc:493] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:21:37.357602 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 68794.8us
I20260709 13:21:37.300767 22100 raft_consensus.cc:3060] T ee9ed67006144797b1de6ea8c2013a68 P cb82cc7bd6db46d980c75237a1402860 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:21:37.295506 22124 consensus_queue.cc:1048] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.071s
W20260709 13:21:37.378545 16788 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:21:37.397861 22121 log.cc:927] Time spent T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860: Append to log took a long time: real 0.112s	user 0.000s	sys 0.001s
W20260709 13:21:37.401381 22123 log.cc:927] Time spent T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 0.119s	user 0.000s	sys 0.000s
I20260709 13:21:37.406096 21245 raft_consensus.cc:3060] T 829b4ad8c5a54eababeea1181b960d4b P 647a343a542d4a139fe106c19095129b [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:21:37.418793 22116 log.cc:927] Time spent T 8c11074579b046d99e585c2818e18a2c P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 0.234s	user 0.000s	sys 0.001s
W20260709 13:21:37.420680 22117 log.cc:927] Time spent T 8638ee363273426db1393dd81e5c7b72 P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.136s	user 0.001s	sys 0.000s
W20260709 13:21:37.430621 22029 consensus_meta.cc:287] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860: Time spent flushing consensus metadata: real 0.745s	user 0.008s	sys 0.000s
I20260709 13:21:37.358546 22059 raft_consensus.cc:515] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:37.469197 22106 raft_consensus.cc:515] T 5a959dbecc2648eb85d1cc550f5a3245 P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:37.469941 16291 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48674 (request call id 3102) took 1248 ms. Trace:
W20260709 13:21:37.441129 22074 consensus_meta.cc:287] T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b: Time spent flushing consensus metadata: real 0.850s	user 0.022s	sys 0.000s
I20260709 13:21:37.482249 16291 rpcz_store.cc:276] 0709 13:21:36.221470 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:36.224168 (+  2698us) service_pool.cc:224] Handling call
0709 13:21:36.227983 (+  3815us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:21:37.065370 (+837387us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:37.469668 (+404298us) spinlock_profiling.cc:243] Waited 828 ms on lock 0x7b5400240254. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:21:37.469874 (+   206us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1905099520}
I20260709 13:21:37.401232 22083 raft_consensus.cc:515] T 3884f864a96d428785895e5463e5728f P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:21:37.492614 22129 log.cc:927] Time spent T 8638ee363273426db1393dd81e5c7b72 P cb82cc7bd6db46d980c75237a1402860: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
I20260709 13:21:37.497820 22059 leader_election.cc:290] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:37.500181 16821 raft_consensus.cc:1240] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 [term 18 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 17. Current term is 18. Ops: []
I20260709 13:21:37.500370 22106 leader_election.cc:290] T 5a959dbecc2648eb85d1cc550f5a3245 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 15 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:21:37.501528 16825 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49896 (request call id 1376) took 1397 ms. Trace:
I20260709 13:21:37.501688 22018 leader_election.cc:290] T b6f07842d3aa47858df5665874c86f3a P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:37.501988 16825 rpcz_store.cc:276] 0709 13:21:36.104241 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:36.104351 (+   110us) service_pool.cc:224] Handling call
0709 13:21:37.501338 (+1396987us) spinlock_profiling.cc:243] Waited 886 ms on lock 0x7b54006dbfd4. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:21:37.501451 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2038308608}
W20260709 13:21:37.503000 22064 consensus_meta.cc:287] T 8158bbd17b994d2eb96f560bd0f1b88d P 90f3da57796b488c9087a2f523adf045: Time spent flushing consensus metadata: real 0.871s	user 0.011s	sys 0.000s
I20260709 13:21:37.372375 22119 raft_consensus.cc:515] T b87147731a68400e8cce9b879daa7c5d P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:37.505525 16816 raft_consensus.cc:1275] T 57a744f5aaf944678a63469cbf514015 P cb82cc7bd6db46d980c75237a1402860 [term 12 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
W20260709 13:21:37.508852 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 150143us
I20260709 13:21:37.507059 16144 raft_consensus.cc:2468] T 8d4ab870a05347868956aa8b841b2981 P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 15.
I20260709 13:21:37.510571 16265 leader_election.cc:304] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: cb82cc7bd6db46d980c75237a1402860
I20260709 13:21:37.515409 16825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e4bfb9090c8460eb14fa7e0d818e7f1" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:37.516472 16825 raft_consensus.cc:2468] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P cb82cc7bd6db46d980c75237a1402860 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 16.
I20260709 13:21:37.521414 16291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1df93779209748b9860f6d893dd3de53" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:37.522272 16291 raft_consensus.cc:2468] T 1df93779209748b9860f6d893dd3de53 P 90f3da57796b488c9087a2f523adf045 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 14.
I20260709 13:21:37.487385 22074 raft_consensus.cc:2749] T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:21:37.503641 22064 raft_consensus.cc:515] T 8158bbd17b994d2eb96f560bd0f1b88d P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:37.526070 16190 leader_election.cc:304] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:21:37.527801 16192 leader_election.cc:304] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:21:37.498823 22090 raft_consensus.cc:515] T 18ad7e3af5504571a69d10c00750a01c P 90f3da57796b488c9087a2f523adf045 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } }
I20260709 13:21:37.520807 21946 raft_consensus.cc:493] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:37.520283 22106 raft_consensus.cc:2804] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:21:37.520473 22059 raft_consensus.cc:493] T e4dc6b0541ca431ebc757beac00521f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:37.532956 16265 leader_election.cc:304] T a378d7f796f94615b3b347119a9880fd P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860
I20260709 13:21:37.537405 22059 raft_consensus.cc:515] T e4dc6b0541ca431ebc757beac00521f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:37.540104 22059 leader_election.cc:290] T e4dc6b0541ca431ebc757beac00521f1 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
W20260709 13:21:37.542462 22081 log.cc:927] Time spent T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860: Append to log took a long time: real 0.616s	user 0.000s	sys 0.001s
I20260709 13:21:37.543422 22133 consensus_queue.cc:1059] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: INVALID_TERM, Last received: 17.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.010s
I20260709 13:21:37.524585 22018 raft_consensus.cc:493] T d2ab0ac8347641e6955e9bce7d90c1a2 P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:21:37.545388 22018 raft_consensus.cc:515] T d2ab0ac8347641e6955e9bce7d90c1a2 P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:37.546033 21930 leader_election.cc:290] T c0eab25ce4c64e018a041beeda2867db P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
W20260709 13:21:37.548074 22126 log.cc:927] Time spent T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 0.252s	user 0.001s	sys 0.000s
W20260709 13:21:37.549368 16817 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49892 (request call id 869) took 1612 ms (client timeout 1617 ms). Trace:
W20260709 13:21:37.549957 16817 rpcz_store.cc:269] 0709 13:21:35.937377 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:35.941353 (+  3976us) service_pool.cc:224] Handling call
0709 13:21:37.549123 (+1607770us) spinlock_profiling.cc:243] Waited 441 ms on lock 0x7b5400787154. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:21:37.549287 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1013734656}
I20260709 13:21:37.551775 16190 leader_election.cc:304] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:21:37.565006 16190 leader_election.cc:304] T 1df93779209748b9860f6d893dd3de53 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:21:37.534958 22106 raft_consensus.cc:493] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Starting leader election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:21:37.568065 22119 leader_election.cc:290] T b87147731a68400e8cce9b879daa7c5d P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:37.568544 22106 raft_consensus.cc:3060] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:21:37.563122 22085 raft_consensus.cc:2749] T a378d7f796f94615b3b347119a9880fd P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:21:37.571475 22018 leader_election.cc:290] T d2ab0ac8347641e6955e9bce7d90c1a2 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
W20260709 13:21:37.572566 16790 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:37.561949 22041 raft_consensus.cc:493] T 74ca8140f35c4ae29e8a664f6bdfc829 P cb82cc7bd6db46d980c75237a1402860 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:37.572039 21930 consensus_queue.cc:1048] T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.008s
I20260709 13:21:37.572119 22059 raft_consensus.cc:2804] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:21:37.556963 21946 raft_consensus.cc:515] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:37.579896 16817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ab0ac8347641e6955e9bce7d90c1a2" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
W20260709 13:21:37.580885 22132 log.cc:927] Time spent T 66ae176961b74813a58bc15c9c4c6f2a P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.098s	user 0.000s	sys 0.000s
W20260709 13:21:37.585642 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 54965.8us
I20260709 13:21:37.589569 22133 raft_consensus.cc:2804] T 1df93779209748b9860f6d893dd3de53 P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:21:37.589845 22085 raft_consensus.cc:493] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:21:37.590622 16296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4dc6b0541ca431ebc757beac00521f1" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:37.592887 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0eab25ce4c64e018a041beeda2867db" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:37.595302 16295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ab0ac8347641e6955e9bce7d90c1a2" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:37.596081 16291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6f07842d3aa47858df5665874c86f3a" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:37.594084 16294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e4bfb9090c8460eb14fa7e0d818e7f1" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:37.599785 16294 raft_consensus.cc:2468] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 16.
I20260709 13:21:37.598614 16296 raft_consensus.cc:2468] T e4dc6b0541ca431ebc757beac00521f1 P 90f3da57796b488c9087a2f523adf045 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 14.
I20260709 13:21:37.601612 22064 leader_election.cc:290] T 8158bbd17b994d2eb96f560bd0f1b88d P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 15 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:21:37.602242 22140 raft_consensus.cc:2804] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:21:37.602914 22140 raft_consensus.cc:493] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Starting leader election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:21:37.603307 22140 raft_consensus.cc:3060] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:21:37.602181 21946 leader_election.cc:290] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:37.599659 16291 raft_consensus.cc:2468] T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 15.
I20260709 13:21:37.602408 22083 leader_election.cc:290] T 3884f864a96d428785895e5463e5728f P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 12 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:37.599822 16265 leader_election.cc:304] T 8c11074579b046d99e585c2818e18a2c P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: cb82cc7bd6db46d980c75237a1402860
I20260709 13:21:37.610832 16190 leader_election.cc:304] T e4dc6b0541ca431ebc757beac00521f1 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:21:37.592842 22059 raft_consensus.cc:493] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:37.598158 22133 raft_consensus.cc:493] T 1df93779209748b9860f6d893dd3de53 P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:37.601629 22085 raft_consensus.cc:515] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:37.614723 22133 raft_consensus.cc:3060] T 1df93779209748b9860f6d893dd3de53 P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:21:37.615774 22064 raft_consensus.cc:2804] T 8c11074579b046d99e585c2818e18a2c P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:21:37.616212 22064 raft_consensus.cc:493] T 8c11074579b046d99e585c2818e18a2c P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Starting leader election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:21:37.616633 22064 raft_consensus.cc:3060] T 8c11074579b046d99e585c2818e18a2c P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:21:37.618489 22059 raft_consensus.cc:3060] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:21:37.620357 16116 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:37.594076 22041 raft_consensus.cc:515] T 74ca8140f35c4ae29e8a664f6bdfc829 P cb82cc7bd6db46d980c75237a1402860 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:37.628382 16265 leader_election.cc:304] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:21:37.628646 22057 raft_consensus.cc:3055] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:21:37.629122 22057 raft_consensus.cc:740] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:21:37.629492 22115 raft_consensus.cc:2804] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 90f3da57796b488c9087a2f523adf045 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:21:37.629931 22115 raft_consensus.cc:493] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 90f3da57796b488c9087a2f523adf045 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:37.629822 16817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3884f864a96d428785895e5463e5728f" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:21:37.630409 22115 raft_consensus.cc:3060] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 90f3da57796b488c9087a2f523adf045 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:21:37.630716 16817 raft_consensus.cc:3060] T 3884f864a96d428785895e5463e5728f P cb82cc7bd6db46d980c75237a1402860 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:21:37.632295 16145 raft_consensus.cc:2468] T 829b4ad8c5a54eababeea1181b960d4b P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 15.
I20260709 13:21:37.629542 16265 leader_election.cc:304] T 68a6838d82ef4cac83507f906139d90f P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:21:37.633859 21930 raft_consensus.cc:493] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:21:37.635921 16265 leader_election.cc:304] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860
I20260709 13:21:37.637955 16816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66ae176961b74813a58bc15c9c4c6f2a" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:37.647966 16816 raft_consensus.cc:2468] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 20.
I20260709 13:21:37.626709 22064 raft_consensus.cc:515] T 8c11074579b046d99e585c2818e18a2c P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:21:37.649487 22108 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.645s	user 0.004s	sys 0.008s
I20260709 13:21:37.649547 22104 raft_consensus.cc:2804] T 68a6838d82ef4cac83507f906139d90f P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:21:37.660897 16817 raft_consensus.cc:2468] T 3884f864a96d428785895e5463e5728f P cb82cc7bd6db46d980c75237a1402860 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 12.
I20260709 13:21:37.666481 16265 leader_election.cc:304] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac
W20260709 13:21:37.655215 22108 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.651s	user 0.004s	sys 0.008s
I20260709 13:21:37.652895 22083 raft_consensus.cc:2749] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:21:37.663131 22104 raft_consensus.cc:493] T 68a6838d82ef4cac83507f906139d90f P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:21:37.649426 21930 raft_consensus.cc:515] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:37.662295 22106 raft_consensus.cc:515] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:37.686807 22106 leader_election.cc:290] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 18 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:37.630388 22140 raft_consensus.cc:515] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:37.674840 22102 raft_consensus.cc:2804] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:21:37.630017 22057 consensus_queue.cc:260] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:37.696619 16217 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57604 (request call id 3037) took 1590 ms. Trace:
I20260709 13:21:37.701676 16217 rpcz_store.cc:276] 0709 13:21:36.106140 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:36.127613 (+ 21473us) service_pool.cc:224] Handling call
0709 13:21:37.696409 (+1568796us) spinlock_profiling.cc:243] Waited 1.05 s on lock 0x7b54002c9054. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:21:37.696558 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2405226368}
I20260709 13:21:37.688527 22104 raft_consensus.cc:3060] T 68a6838d82ef4cac83507f906139d90f P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:21:37.697302 22134 raft_consensus.cc:2804] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:21:37.687121 22133 raft_consensus.cc:515] T 1df93779209748b9860f6d893dd3de53 P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:37.701261 22090 leader_election.cc:290] T 18ad7e3af5504571a69d10c00750a01c P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 14 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:21:37.703964 22145 raft_consensus.cc:2804] T e4dc6b0541ca431ebc757beac00521f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:21:37.739787 22145 raft_consensus.cc:493] T e4dc6b0541ca431ebc757beac00521f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:37.740224 22145 raft_consensus.cc:3060] T e4dc6b0541ca431ebc757beac00521f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:21:37.740980 22140 leader_election.cc:290] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 17 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
W20260709 13:21:37.743948 22141 log.cc:927] Time spent T ecc1e363230043fe9fef390f0b979f20 P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 0.078s	user 0.001s	sys 0.000s
I20260709 13:21:37.745311 22059 raft_consensus.cc:515] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } }
I20260709 13:21:37.745869 21930 leader_election.cc:290] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
W20260709 13:21:37.749549 22105 log.cc:927] Time spent T 3a7ca61592e848ea9b0fb81a2ce33cdc P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.801s	user 0.003s	sys 0.000s
I20260709 13:21:37.745748 22133 leader_election.cc:290] T 1df93779209748b9860f6d893dd3de53 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 15 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
W20260709 13:21:37.753278 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 111465us
I20260709 13:21:37.705219 22102 raft_consensus.cc:697] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045 [term 16 LEADER]: Becoming Leader. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:21:37.755630 22059 leader_election.cc:290] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 17 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
W20260709 13:21:37.745844 22144 log.cc:927] Time spent T ada5a004ba7c46819051e3ccec71ffdb P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
I20260709 13:21:37.756214 16150 raft_consensus.cc:1275] T 8d4ab870a05347868956aa8b841b2981 P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:21:37.732816 22041 leader_election.cc:290] T 74ca8140f35c4ae29e8a664f6bdfc829 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:37.753779 22115 raft_consensus.cc:515] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:21:37.772122 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 68933.2us
I20260709 13:21:37.737499 22134 raft_consensus.cc:493] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:37.737987 22057 raft_consensus.cc:3060] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:21:37.784611 16223 raft_consensus.cc:1275] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 13:21:37.807713 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74ca8140f35c4ae29e8a664f6bdfc829" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 17 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:37.783854 21930 raft_consensus.cc:493] T af57996a8be84d398f0f023e8d08ef77 P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:37.809753 16226 raft_consensus.cc:2410] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 for term 17 because replica has last-logged OpId of term: 15 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 11.
I20260709 13:21:37.810153 22115 leader_election.cc:290] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 15 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:21:37.810853 22149 raft_consensus.cc:493] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:37.806135 22134 raft_consensus.cc:3060] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:21:37.811460 22149 raft_consensus.cc:515] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:37.809139 21930 raft_consensus.cc:515] T af57996a8be84d398f0f023e8d08ef77 P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:37.821163 22064 leader_election.cc:290] T 8c11074579b046d99e585c2818e18a2c P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 16 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:37.822065 22148 raft_consensus.cc:493] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:21:37.822686 22148 raft_consensus.cc:515] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:37.824965 22148 leader_election.cc:290] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:37.825448 22088 raft_consensus.cc:493] T 7c2095a5f4ad49408dbc7f69c606120d P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:37.825786 22147 raft_consensus.cc:493] T 08c74490c40243a88e62d1760b5cd814 P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:21:37.826004 22088 raft_consensus.cc:515] T 7c2095a5f4ad49408dbc7f69c606120d P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:37.826308 22147 raft_consensus.cc:515] T 08c74490c40243a88e62d1760b5cd814 P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:37.827073 16265 leader_election.cc:304] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:21:37.828509 22147 leader_election.cc:290] T 08c74490c40243a88e62d1760b5cd814 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:37.828632 16265 leader_election.cc:304] T 23f82224f4564fea93572e8206bad1fe P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
I20260709 13:21:37.829671 22102 consensus_queue.cc:237] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.10, Last appended by leader: 10, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } }
I20260709 13:21:37.830298 16265 leader_election.cc:304] T 5a96373bb0e14dce8a266b2fc6c58507 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:21:37.832036 22079 raft_consensus.cc:2749] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:21:37.838929 22079 raft_consensus.cc:2749] T 5a96373bb0e14dce8a266b2fc6c58507 P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:21:37.832719 22115 raft_consensus.cc:2749] T 23f82224f4564fea93572e8206bad1fe P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:21:37.844189 16117 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:37.844537 16300 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40474 (request call id 1028) took 1497 ms. Trace:
I20260709 13:21:37.846866 16817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87147731a68400e8cce9b879daa7c5d" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:37.847303 16300 rpcz_store.cc:276] 0709 13:21:36.347043 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:36.592770 (+245727us) service_pool.cc:224] Handling call
0709 13:21:36.593127 (+   357us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:21:36.674764 (+ 81637us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:21:36.674794 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:21:36.689056 (+ 14262us) log.cc:844] Serialized 26 byte log entry
0709 13:21:36.766833 (+ 77777us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:21:36.766867 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:36.766931 (+    64us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:21:37.748841 (+981910us) spinlock_profiling.cc:243] Waited 2.07 ms on lock 0x7b4400022b4c. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69ad8fadb1 00007f69ad998c85 00007f69ad9996a2 00007f69b070b1db 00007f69b0706aaf 00007f69b06fa86e 00007f69b06f9bb6 00007f69b06f97d8 00007f69b0703221 00007f69b07031d9 00007f69b0703169
0709 13:21:37.843715 (+ 94874us) raft_consensus.cc:1685] finished
0709 13:21:37.843737 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:21:37.844445 (+   708us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4999,"raft.queue_time_us":172648,"raft.run_cpu_time_us":116,"raft.run_wall_time_us":1538,"spinlock_wait_cycles":4752128,"thread_start_us":99579,"threads_started":2,"wal-append.queue_time_us":178917}
W20260709 13:21:37.851894 16117 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:21:37.854192 16117 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:37.844498 22024 raft_consensus.cc:493] T 02838c86531d4f98bf58d1c22b8baf77 P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:37.857414 16144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2372981006d4b218215c28964dea623" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:21:37.860567 16299 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48674 (request call id 3103) took 1638 ms. Trace:
I20260709 13:21:37.862046 16265 leader_election.cc:304] T c0eab25ce4c64e018a041beeda2867db P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:21:37.876967 16144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31969392ccdb43b491911a319e8bc7e1" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:21:37.885200 16223 raft_consensus.cc:1275] T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
I20260709 13:21:37.863375 16299 rpcz_store.cc:276] 0709 13:21:36.221803 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:36.221940 (+   137us) service_pool.cc:224] Handling call
0709 13:21:36.222338 (+   398us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:21:36.222629 (+   291us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:21:36.488545 (+265916us) log.cc:844] Serialized 14 byte log entry
0709 13:21:37.788369 (+1299824us) spinlock_profiling.cc:243] Waited 7.1 ms on lock 0x7b180036bb60. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b693ade1 00007f69b69475b1 00007f69ada54e07 00007f69b68afab9 00007f69ada7169b 00007f69ada6f93f 00007f69ada71c02 00007f69b06f920b 00007f69b06fed23 00007f69b06ff23a 00007f69b0b69a2a
0709 13:21:37.860215 (+ 71846us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:21:37.860277 (+    62us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:21:37.860320 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:37.860385 (+    65us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:21:37.860512 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":338095,"prepare.run_cpu_time_us":59,"prepare.run_wall_time_us":5178,"raft.queue_time_us":3363,"raft.run_cpu_time_us":154,"raft.run_wall_time_us":4152,"spinlock_wait_cycles":16320768,"thread_start_us":333678,"threads_started":2}
I20260709 13:21:37.888737 16219 raft_consensus.cc:1240] T 1df93779209748b9860f6d893dd3de53 P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 14. Current term is 15. Ops: [14.9-14.9]
I20260709 13:21:37.889278 16223 raft_consensus.cc:1240] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 16. Current term is 17. Ops: []
I20260709 13:21:37.890012 16265 leader_election.cc:304] T 57a744f5aaf944678a63469cbf514015 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
I20260709 13:21:37.873560 22079 raft_consensus.cc:2749] T c0eab25ce4c64e018a041beeda2867db P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:21:37.904584 16150 raft_consensus.cc:1240] T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 14. Current term is 15. Ops: []
I20260709 13:21:37.903285 16116 leader_election.cc:304] T 8cc959fb7f704fba9009c0595467d074 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
I20260709 13:21:37.906502 16148 raft_consensus.cc:1240] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 17. Current term is 18. Ops: []
I20260709 13:21:37.882092 22100 raft_consensus.cc:493] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:37.859942 22024 raft_consensus.cc:515] T 02838c86531d4f98bf58d1c22b8baf77 P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:37.898110 16144 raft_consensus.cc:2468] T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 15.
I20260709 13:21:37.886209 22147 raft_consensus.cc:493] T ee9ed67006144797b1de6ea8c2013a68 P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:37.905635 22115 raft_consensus.cc:2749] T 57a744f5aaf944678a63469cbf514015 P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:21:37.908516 22027 raft_consensus.cc:2804] T 8cc959fb7f704fba9009c0595467d074 P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Leader election won for term 15
W20260709 13:21:37.920682 16788 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:37.923610 22151 raft_consensus.cc:493] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:37.926348 16219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c2095a5f4ad49408dbc7f69c606120d" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 13 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:37.929872 16219 raft_consensus.cc:2468] T 7c2095a5f4ad49408dbc7f69c606120d P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 12.
I20260709 13:21:37.905319 22043 consensus_queue.cc:1048] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.014s
I20260709 13:21:37.951507 16192 leader_election.cc:304] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:21:37.952267 22145 raft_consensus.cc:515] T e4dc6b0541ca431ebc757beac00521f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:37.915135 22147 raft_consensus.cc:515] T ee9ed67006144797b1de6ea8c2013a68 P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:37.954003 22049 raft_consensus.cc:493] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:37.954579 22049 raft_consensus.cc:515] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:37.956512 22049 leader_election.cc:290] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:21:37.957271 16817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5d59407eb545b895d8d922e0dd69fd" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:37.957872 16817 raft_consensus.cc:2468] T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 17.
I20260709 13:21:37.960096 16265 leader_election.cc:304] T d16dfad0c3b540e5b4b86492ddf87208 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:21:37.960781 16150 raft_consensus.cc:1240] T ee9ed67006144797b1de6ea8c2013a68 P 647a343a542d4a139fe106c19095129b [term 16 LEADER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 15. Current term is 16. Ops: []
I20260709 13:21:37.911443 22100 raft_consensus.cc:515] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:37.927932 22143 raft_consensus.cc:493] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:37.961828 22147 leader_election.cc:290] T ee9ed67006144797b1de6ea8c2013a68 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:37.927990 22088 leader_election.cc:290] T 7c2095a5f4ad49408dbc7f69c606120d P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:37.961714 22115 raft_consensus.cc:3060] T d16dfad0c3b540e5b4b86492ddf87208 P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:21:37.967234 22155 log.cc:927] Time spent T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860: Append to log took a long time: real 0.087s	user 0.001s	sys 0.000s
I20260709 13:21:37.933879 22027 raft_consensus.cc:697] T 8cc959fb7f704fba9009c0595467d074 P 647a343a542d4a139fe106c19095129b [term 15 LEADER]: Becoming Leader. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:21:37.968061 22153 raft_consensus.cc:2804] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:21:37.968780 22143 raft_consensus.cc:515] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:37.964906 16192 leader_election.cc:304] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:21:37.977846 16816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee9ed67006144797b1de6ea8c2013a68" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 17 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:37.979122 16816 raft_consensus.cc:2410] T ee9ed67006144797b1de6ea8c2013a68 P cb82cc7bd6db46d980c75237a1402860 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac for term 17 because replica has last-logged OpId of term: 15 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 12.
W20260709 13:21:37.981459 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 71404.6us
I20260709 13:21:37.986287 22043 consensus_queue.cc:1048] T 9c61a39fd4f84424b129c1a6f9ba404d P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:21:37.986613 22088 consensus_queue.cc:1059] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 647a343a542d4a139fe106c19095129b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: INVALID_TERM, Last received: 16.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:21:37.987644 22001 consensus_queue.cc:1059] T 1df93779209748b9860f6d893dd3de53 P 647a343a542d4a139fe106c19095129b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:21:37.988078 22088 raft_consensus.cc:3055] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 647a343a542d4a139fe106c19095129b [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:21:37.988755 22088 raft_consensus.cc:740] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 647a343a542d4a139fe106c19095129b [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:21:37.988826 22001 raft_consensus.cc:3055] T 1df93779209748b9860f6d893dd3de53 P 647a343a542d4a139fe106c19095129b [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:21:37.989560 22001 raft_consensus.cc:740] T 1df93779209748b9860f6d893dd3de53 P 647a343a542d4a139fe106c19095129b [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:21:37.990268 22088 consensus_queue.cc:260] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:37.990849 22001 consensus_queue.cc:260] T 1df93779209748b9860f6d893dd3de53 P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:37.992839 22088 raft_consensus.cc:3060] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:21:37.993453 16116 leader_election.cc:304] T 7c2095a5f4ad49408dbc7f69c606120d P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
I20260709 13:21:37.994393 21704 raft_consensus.cc:2804] T 7c2095a5f4ad49408dbc7f69c606120d P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:21:37.994858 21704 raft_consensus.cc:493] T 7c2095a5f4ad49408dbc7f69c606120d P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:37.995276 21704 raft_consensus.cc:3060] T 7c2095a5f4ad49408dbc7f69c606120d P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:21:37.998291 16150 raft_consensus.cc:1275] T 2794af5b99984efebdee564035841cb0 P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 13:21:37.998677 16148 raft_consensus.cc:1240] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 647a343a542d4a139fe106c19095129b [term 14 FOLLOWER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 13. Current term is 14. Ops: []
I20260709 13:21:38.000342 16265 leader_election.cc:304] T e4dc6b0541ca431ebc757beac00521f1 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
W20260709 13:21:38.001453 16789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 86435.9us
I20260709 13:21:37.974988 22153 raft_consensus.cc:493] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:38.002489 22153 raft_consensus.cc:3060] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:21:38.004473 21930 leader_election.cc:290] T af57996a8be84d398f0f023e8d08ef77 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:38.005086 22059 raft_consensus.cc:493] T fffab26427e24d398e2695aad40ca134 P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:38.005646 22059 raft_consensus.cc:515] T fffab26427e24d398e2695aad40ca134 P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:38.007836 16148 raft_consensus.cc:1240] T 23f82224f4564fea93572e8206bad1fe P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 12. Current term is 13. Ops: []
I20260709 13:21:38.008353 22059 leader_election.cc:290] T fffab26427e24d398e2695aad40ca134 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:38.010829 16220 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33050 (request call id 1015) took 1209 ms. Trace:
I20260709 13:21:37.934098 22151 raft_consensus.cc:515] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:38.011298 16220 rpcz_store.cc:276] 0709 13:21:36.801231 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:36.801349 (+   118us) service_pool.cc:224] Handling call
0709 13:21:36.817948 (+ 16599us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:21:36.832219 (+ 14271us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:21:36.832249 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:21:36.832757 (+   508us) log.cc:844] Serialized 26 byte log entry
0709 13:21:38.010212 (+1177455us) spinlock_profiling.cc:243] Waited 1.13 ms on lock 0x7b18003ba340. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b693ade1 00007f69b69475b1 00007f69b07074bc 00007f69ada5548b 00007f69b68afab9 00007f69ada7169b 00007f69ada6f93f 00007f69ada71c02 00007f69b06f920b 00007f69b06fed23 00007f69b06ff0d1
0709 13:21:38.010387 (+   175us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:21:38.010427 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:38.010501 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:21:38.010548 (+    47us) raft_consensus.cc:1685] finished
0709 13:21:38.010560 (+    12us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:21:38.010765 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":9226,"raft.run_cpu_time_us":143,"raft.run_wall_time_us":9011,"spinlock_wait_cycles":2607360,"thread_start_us":25245,"threads_started":1,"wal-append.queue_time_us":176159}
I20260709 13:21:38.013350 22151 leader_election.cc:290] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:37.999390 22147 raft_consensus.cc:2804] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:21:38.015303 22100 leader_election.cc:290] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:21:38.015604 22147 raft_consensus.cc:493] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Starting leader election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:21:37.955015 22145 leader_election.cc:290] T e4dc6b0541ca431ebc757beac00521f1 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 16 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:38.016139 22147 raft_consensus.cc:3060] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:21:38.018563 16265 leader_election.cc:304] T c4f65cdf00b748bb963ee6e81ef43b79 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:21:38.019418 22079 raft_consensus.cc:3060] T e4dc6b0541ca431ebc757beac00521f1 P 90f3da57796b488c9087a2f523adf045 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:21:38.019917 22157 log.cc:927] Time spent T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.074s	user 0.000s	sys 0.000s
I20260709 13:21:38.027608 22064 raft_consensus.cc:2749] T c4f65cdf00b748bb963ee6e81ef43b79 P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:21:38.028113 22085 leader_election.cc:290] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:38.028656 16816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af57996a8be84d398f0f023e8d08ef77" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:38.029453 16816 raft_consensus.cc:2468] T af57996a8be84d398f0f023e8d08ef77 P cb82cc7bd6db46d980c75237a1402860 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 17.
I20260709 13:21:38.031438 16819 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49906 (request call id 1151) took 1407 ms. Trace:
I20260709 13:21:38.031867 16819 rpcz_store.cc:276] 0709 13:21:36.624441 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:36.624600 (+   159us) service_pool.cc:224] Handling call
0709 13:21:36.650885 (+ 26285us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:21:36.665693 (+ 14808us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:21:36.665726 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:21:36.685079 (+ 19353us) log.cc:844] Serialized 26 byte log entry
0709 13:21:37.634227 (+949148us) spinlock_profiling.cc:243] Waited 3.52 ms on lock 0x7b180040aec0. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b693ade1 00007f69b69475b1 00007f69ada54e07 00007f69b68afab9 00007f69ada7169b 00007f69ada6f93f 00007f69ada71c02 00007f69b06f920b 00007f69b06fed23 00007f69b06ff0d1 00007f69b0b1aa0f
0709 13:21:37.784332 (+150105us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:21:37.784380 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:37.784447 (+    67us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:21:38.031053 (+246606us) raft_consensus.cc:1685] finished
0709 13:21:38.031074 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:21:38.031373 (+   299us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":9818,"raft.queue_time_us":97882,"raft.run_cpu_time_us":168,"raft.run_wall_time_us":4717,"spinlock_wait_cycles":8098688,"thread_start_us":149265,"threads_started":1,"wal-append.queue_time_us":1060856}
I20260709 13:21:38.033373 22024 leader_election.cc:290] T 02838c86531d4f98bf58d1c22b8baf77 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
W20260709 13:21:38.034307 22120 log.cc:927] Time spent T fffab26427e24d398e2695aad40ca134 P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 0.749s	user 0.003s	sys 0.000s
I20260709 13:21:38.034729 16192 leader_election.cc:304] T af57996a8be84d398f0f023e8d08ef77 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:21:38.035692 22152 raft_consensus.cc:2804] T af57996a8be84d398f0f023e8d08ef77 P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:21:38.036154 22152 raft_consensus.cc:493] T af57996a8be84d398f0f023e8d08ef77 P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:38.044492 16117 leader_election.cc:304] T 746aabf856ab401c9ef5e859371e7b67 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860
I20260709 13:21:38.062196 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cc959fb7f704fba9009c0595467d074" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
W20260709 13:21:38.028698 22156 log.cc:927] Time spent T c4f65cdf00b748bb963ee6e81ef43b79 P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.057s	user 0.001s	sys 0.000s
I20260709 13:21:38.074753 16292 raft_consensus.cc:1275] T 2794af5b99984efebdee564035841cb0 P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 13:21:38.062330 16294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f82224f4564fea93572e8206bad1fe" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:21:38.070386 16191 leader_election.cc:304] T 02838c86531d4f98bf58d1c22b8baf77 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:21:38.038462 22001 raft_consensus.cc:3060] T 1df93779209748b9860f6d893dd3de53 P 647a343a542d4a139fe106c19095129b [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:21:38.081453 16291 raft_consensus.cc:1240] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 14. Current term is 15. Ops: []
I20260709 13:21:38.036517 22152 raft_consensus.cc:3060] T af57996a8be84d398f0f023e8d08ef77 P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:21:38.080390 16191 leader_election.cc:304] T 5a959dbecc2648eb85d1cc550f5a3245 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:21:38.080113 16296 raft_consensus.cc:1275] T 57a744f5aaf944678a63469cbf514015 P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:21:38.082139 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:21:38.086910 16299 raft_consensus.cc:3060] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:21:38.001648 22025 raft_consensus.cc:493] T 434bff3082d141838068f323bdb0d1ed P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:21:38.055516 22024 raft_consensus.cc:2749] T 746aabf856ab401c9ef5e859371e7b67 P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:21:38.084146 16819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62fe3346cd3c42c4a55e14af9073bca4" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 22 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:38.083564 16294 raft_consensus.cc:2393] T 23f82224f4564fea93572e8206bad1fe P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 13: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
I20260709 13:21:38.080561 22147 raft_consensus.cc:515] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:38.107565 16300 raft_consensus.cc:1275] T 1df93779209748b9860f6d893dd3de53 P 90f3da57796b488c9087a2f523adf045 [term 14 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 13:21:38.109504 16300 raft_consensus.cc:1240] T 8158bbd17b994d2eb96f560bd0f1b88d P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 14. Current term is 15. Ops: []
I20260709 13:21:38.116044 16296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74ca8140f35c4ae29e8a664f6bdfc829" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 17 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:38.094192 22158 consensus_queue.cc:1048] T 57a744f5aaf944678a63469cbf514015 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.024s
I20260709 13:21:38.101514 22025 raft_consensus.cc:515] T 434bff3082d141838068f323bdb0d1ed P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:38.120322 22025 leader_election.cc:290] T 434bff3082d141838068f323bdb0d1ed P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:38.103225 22134 raft_consensus.cc:515] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:38.126015 22134 leader_election.cc:290] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 17 election: Requested vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:38.134867 22134 consensus_queue.cc:1048] T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:21:38.085115 22059 raft_consensus.cc:2749] T 02838c86531d4f98bf58d1c22b8baf77 P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:21:38.116830 16296 raft_consensus.cc:2468] T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 16.
I20260709 13:21:38.076246 22098 consensus_queue.cc:1048] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.039s
I20260709 13:21:38.145005 16291 raft_consensus.cc:3055] T 8638ee363273426db1393dd81e5c7b72 P 90f3da57796b488c9087a2f523adf045 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:21:38.099623 22143 leader_election.cc:290] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:38.101377 22145 raft_consensus.cc:2749] T 5a959dbecc2648eb85d1cc550f5a3245 P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
W20260709 13:21:38.155790 16117 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 92388.3us
I20260709 13:21:38.158109 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87147731a68400e8cce9b879daa7c5d" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:38.149062 16294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31969392ccdb43b491911a319e8bc7e1" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:21:38.166072 16291 raft_consensus.cc:740] T 8638ee363273426db1393dd81e5c7b72 P 90f3da57796b488c9087a2f523adf045 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:21:38.173607 16291 consensus_queue.cc:260] T 8638ee363273426db1393dd81e5c7b72 P 90f3da57796b488c9087a2f523adf045 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:38.172260 16294 raft_consensus.cc:2376] T 31969392ccdb43b491911a319e8bc7e1 P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 15. Re-sending same reply.
I20260709 13:21:38.172094 16300 raft_consensus.cc:2468] T b87147731a68400e8cce9b879daa7c5d P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 16.
I20260709 13:21:38.177309 16294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c2095a5f4ad49408dbc7f69c606120d" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 13 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:38.177568 16300 raft_consensus.cc:3055] T fffab26427e24d398e2695aad40ca134 P 90f3da57796b488c9087a2f523adf045 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:21:38.178071 16300 raft_consensus.cc:740] T fffab26427e24d398e2695aad40ca134 P 90f3da57796b488c9087a2f523adf045 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:21:38.179340 16294 raft_consensus.cc:1275] T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
I20260709 13:21:38.179958 16291 raft_consensus.cc:3060] T 8638ee363273426db1393dd81e5c7b72 P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:21:38.178885 16300 consensus_queue.cc:260] T fffab26427e24d398e2695aad40ca134 P 90f3da57796b488c9087a2f523adf045 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:38.100705 22031 raft_consensus.cc:3060] T 23f82224f4564fea93572e8206bad1fe P cb82cc7bd6db46d980c75237a1402860 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:21:38.190781 16789 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:38.192024 16300 raft_consensus.cc:3060] T fffab26427e24d398e2695aad40ca134 P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:21:38.195715 16115 leader_election.cc:304] T 26354361e9b044cea1c2b86c4882fe92 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860
W20260709 13:21:38.199003 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 119493us
I20260709 13:21:38.203415 16789 leader_election.cc:304] T 74ca8140f35c4ae29e8a664f6bdfc829 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:21:38.205469 16789 leader_election.cc:304] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:21:38.199424 22079 raft_consensus.cc:2749] T e4dc6b0541ca431ebc757beac00521f1 P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:21:38.214924 16296 raft_consensus.cc:1240] T 8c11074579b046d99e585c2818e18a2c P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 15. Current term is 16. Ops: []
I20260709 13:21:38.220367 16115 leader_election.cc:304] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:21:38.210548 22029 raft_consensus.cc:2804] T 74ca8140f35c4ae29e8a664f6bdfc829 P cb82cc7bd6db46d980c75237a1402860 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:21:38.149338 22027 consensus_queue.cc:237] T 8cc959fb7f704fba9009c0595467d074 P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:38.213336 21579 raft_consensus.cc:2804] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:21:38.233112 16819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434bff3082d141838068f323bdb0d1ed" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:38.238902 16298 raft_consensus.cc:1275] T e4dc6b0541ca431ebc757beac00521f1 P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 14 index: 7. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 13:21:38.233479 16218 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57604 (request call id 3049) took 1216 ms. Trace:
I20260709 13:21:38.236511 22153 raft_consensus.cc:515] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:38.202677 22048 consensus_queue.cc:1048] T 2794af5b99984efebdee564035841cb0 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.016s
I20260709 13:21:38.243125 16218 rpcz_store.cc:276] 0709 13:21:37.017397 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:37.033120 (+ 15723us) service_pool.cc:224] Handling call
0709 13:21:37.033362 (+   242us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:21:37.189100 (+155738us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:21:37.189134 (+    34us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:21:37.189180 (+    46us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:21:37.189215 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:38.233224 (+1044009us) spinlock_profiling.cc:243] Waited 156 ms on lock 0x7b5400014fd4. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:21:38.233272 (+    48us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:21:38.233416 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":357672832}
I20260709 13:21:38.221637 16115 leader_election.cc:304] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860
I20260709 13:21:38.275949 16297 raft_consensus.cc:1240] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 17. Current term is 18. Ops: []
W20260709 13:21:38.279904 16115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 58351.5us
I20260709 13:21:38.233323 22029 raft_consensus.cc:493] T 74ca8140f35c4ae29e8a664f6bdfc829 P cb82cc7bd6db46d980c75237a1402860 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:21:38.291781 16264 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:21:38.292685 16264 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:38.293064 16265 leader_election.cc:304] T 9e65676b21b748728ad39668cfe722ec P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: cb82cc7bd6db46d980c75237a1402860
I20260709 13:21:38.293644 21579 raft_consensus.cc:697] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860 [term 15 LEADER]: Becoming Leader. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:21:38.245790 21623 raft_consensus.cc:493] T 829b4ad8c5a54eababeea1181b960d4b P cb82cc7bd6db46d980c75237a1402860 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:38.296103 21623 raft_consensus.cc:515] T 829b4ad8c5a54eababeea1181b960d4b P cb82cc7bd6db46d980c75237a1402860 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:21:38.282477 16263 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:38.295397 22115 raft_consensus.cc:2749] T d16dfad0c3b540e5b4b86492ddf87208 P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:21:38.291688 22029 raft_consensus.cc:3060] T 74ca8140f35c4ae29e8a664f6bdfc829 P cb82cc7bd6db46d980c75237a1402860 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:21:38.295028 21579 consensus_queue.cc:237] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:38.280982 22164 raft_consensus.cc:2749] T 26354361e9b044cea1c2b86c4882fe92 P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:21:38.280821 22043 raft_consensus.cc:2749] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:21:38.402804 16115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 67767.7us
I20260709 13:21:38.338213 16265 leader_election.cc:304] T 2794af5b99984efebdee564035841cb0 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
I20260709 13:21:38.340374 16291 raft_consensus.cc:1275] T 8638ee363273426db1393dd81e5c7b72 P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 13:21:38.405193 16300 raft_consensus.cc:1275] T fffab26427e24d398e2695aad40ca134 P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 13:21:38.353946 16299 raft_consensus.cc:2468] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 18.
I20260709 13:21:38.414414 16293 raft_consensus.cc:1240] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 17. Current term is 18. Ops: []
W20260709 13:21:38.296094 22104 consensus_meta.cc:287] T 68a6838d82ef4cac83507f906139d90f P 90f3da57796b488c9087a2f523adf045: Time spent flushing consensus metadata: real 0.557s	user 0.011s	sys 0.001s
I20260709 13:21:38.436689 16115 leader_election.cc:304] T b87147731a68400e8cce9b879daa7c5d P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: cb82cc7bd6db46d980c75237a1402860
I20260709 13:21:38.303275 22153 leader_election.cc:290] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 21 election: Requested vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:38.307524 22147 leader_election.cc:290] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 18 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:38.316962 22119 raft_consensus.cc:2749] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 647a343a542d4a139fe106c19095129b [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:21:38.299113 21623 leader_election.cc:290] T 829b4ad8c5a54eababeea1181b960d4b P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
W20260709 13:21:38.448379 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 110260us
I20260709 13:21:38.425537 22057 raft_consensus.cc:493] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:21:38.451637 22158 raft_consensus.cc:2804] T b87147731a68400e8cce9b879daa7c5d P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:21:38.447609 22164 consensus_queue.cc:1059] T 8158bbd17b994d2eb96f560bd0f1b88d P 647a343a542d4a139fe106c19095129b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.009s
W20260709 13:21:38.461850 16788 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:38.477150 16265 leader_election.cc:304] T ecc1e363230043fe9fef390f0b979f20 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: cb82cc7bd6db46d980c75237a1402860
I20260709 13:21:38.442725 22115 raft_consensus.cc:2749] T 2794af5b99984efebdee564035841cb0 P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:21:38.426192 22043 consensus_queue.cc:1059] T 08c74490c40243a88e62d1760b5cd814 P 647a343a542d4a139fe106c19095129b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.030s
I20260709 13:21:38.313534 22169 consensus_queue.cc:1059] T 5a96373bb0e14dce8a266b2fc6c58507 P 647a343a542d4a139fe106c19095129b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: INVALID_TERM, Last received: 12.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.140s
W20260709 13:21:38.501219 22161 log.cc:927] Time spent T 9e65676b21b748728ad39668cfe722ec P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 0.304s	user 0.005s	sys 0.003s
I20260709 13:21:38.502789 16819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9944d5e4b8c4109af9955c4f08c9095" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 19 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
W20260709 13:21:38.506317 16151 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40844 (request call id 2707) took 1632 ms (client timeout 1710 ms). Trace:
I20260709 13:21:38.436107 22167 consensus_queue.cc:1048] T 57a744f5aaf944678a63469cbf514015 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.042s
W20260709 13:21:38.487118 16145 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40844 (request call id 2703) took 1610 ms (client timeout 1870 ms). Trace:
W20260709 13:21:38.506776 16151 rpcz_store.cc:269] 0709 13:21:36.874310 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:36.882676 (+  8366us) service_pool.cc:224] Handling call
0709 13:21:38.506054 (+1623378us) spinlock_profiling.cc:243] Waited 209 ms on lock 0x7b540028e954. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:21:38.506248 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":481175424}
W20260709 13:21:38.507534 16145 rpcz_store.cc:269] 0709 13:21:36.873178 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:36.937133 (+ 63955us) service_pool.cc:224] Handling call
0709 13:21:38.483008 (+1545875us) spinlock_profiling.cc:243] Waited 687 ms on lock 0x7b540006d6d4. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:21:38.483129 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1579035520}
I20260709 13:21:38.474421 22158 raft_consensus.cc:493] T b87147731a68400e8cce9b879daa7c5d P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Starting leader election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:21:38.443046 22168 consensus_queue.cc:1059] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.071s
I20260709 13:21:38.514336 16816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e4bfb9090c8460eb14fa7e0d818e7f1" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:21:38.515305 16816 raft_consensus.cc:3060] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P cb82cc7bd6db46d980c75237a1402860 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:21:38.445334 22104 raft_consensus.cc:515] T 68a6838d82ef4cac83507f906139d90f P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:21:38.505616 21704 consensus_meta.cc:287] T 7c2095a5f4ad49408dbc7f69c606120d P 647a343a542d4a139fe106c19095129b: Time spent flushing consensus metadata: real 0.510s	user 0.011s	sys 0.000s
I20260709 13:21:38.529656 16825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66ae176961b74813a58bc15c9c4c6f2a" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:21:38.530412 16825 raft_consensus.cc:3060] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:21:38.537547 22152 raft_consensus.cc:515] T af57996a8be84d398f0f023e8d08ef77 P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:38.514279 22158 raft_consensus.cc:3060] T b87147731a68400e8cce9b879daa7c5d P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:21:38.478713 22057 raft_consensus.cc:515] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:21:38.545094 16791 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:38.516350 22024 raft_consensus.cc:3055] T 8158bbd17b994d2eb96f560bd0f1b88d P 647a343a542d4a139fe106c19095129b [term 14 LEADER]: Stepping down as leader of term 14
W20260709 13:21:38.551908 16791 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:38.560710 16816 raft_consensus.cc:2468] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P cb82cc7bd6db46d980c75237a1402860 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 17.
I20260709 13:21:38.562709 16192 leader_election.cc:304] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:21:38.540571 22147 raft_consensus.cc:493] T 18ad7e3af5504571a69d10c00750a01c P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:21:38.577382 22147 raft_consensus.cc:515] T 18ad7e3af5504571a69d10c00750a01c P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } }
I20260709 13:21:38.575520 16190 leader_election.cc:304] T b6f07842d3aa47858df5665874c86f3a P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:21:38.578833 16190 leader_election.cc:304] T d2ab0ac8347641e6955e9bce7d90c1a2 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860
I20260709 13:21:38.579540 22147 leader_election.cc:290] T 18ad7e3af5504571a69d10c00750a01c P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:38.579622 22134 raft_consensus.cc:2804] T b6f07842d3aa47858df5665874c86f3a P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:21:38.580180 22147 raft_consensus.cc:2749] T d2ab0ac8347641e6955e9bce7d90c1a2 P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:21:38.581700 22059 raft_consensus.cc:2804] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:21:38.582264 22059 raft_consensus.cc:697] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 17 LEADER]: Becoming Leader. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:21:38.583122 22059 consensus_queue.cc:237] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:38.553251 22024 raft_consensus.cc:740] T 8158bbd17b994d2eb96f560bd0f1b88d P 647a343a542d4a139fe106c19095129b [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:21:38.592905 22024 consensus_queue.cc:260] T 8158bbd17b994d2eb96f560bd0f1b88d P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:38.596017 21802 raft_consensus.cc:3060] T c9944d5e4b8c4109af9955c4f08c9095 P cb82cc7bd6db46d980c75237a1402860 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:21:38.547210 22043 raft_consensus.cc:3055] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:21:38.566480 16817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5d59407eb545b895d8d922e0dd69fd" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
W20260709 13:21:38.570426 22172 log.cc:927] Time spent T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 0.072s	user 0.001s	sys 0.000s
I20260709 13:21:38.574690 16220 raft_consensus.cc:1240] T af57996a8be84d398f0f023e8d08ef77 P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 17. Current term is 18. Ops: []
I20260709 13:21:38.575219 16224 raft_consensus.cc:1275] T 84f2da98864b4425bf7c0055951fdd4b P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:21:38.575456 16222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02838c86531d4f98bf58d1c22b8baf77" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:38.642174 16222 raft_consensus.cc:2468] T 02838c86531d4f98bf58d1c22b8baf77 P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 16.
I20260709 13:21:38.573843 16266 leader_election.cc:304] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
I20260709 13:21:38.576119 16226 raft_consensus.cc:1240] T e4dc6b0541ca431ebc757beac00521f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 15. Current term is 16. Ops: []
I20260709 13:21:38.613886 16820 raft_consensus.cc:2468] T 23f82224f4564fea93572e8206bad1fe P cb82cc7bd6db46d980c75237a1402860 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 13.
W20260709 13:21:38.621031 16263 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:38.628340 16817 raft_consensus.cc:3060] T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:21:38.645072 16825 raft_consensus.cc:2468] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 21.
I20260709 13:21:38.540205 21704 raft_consensus.cc:515] T 7c2095a5f4ad49408dbc7f69c606120d P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:38.647622 16192 leader_election.cc:304] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:21:38.654688 16116 leader_election.cc:304] T 02838c86531d4f98bf58d1c22b8baf77 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
I20260709 13:21:38.658993 21245 consensus_queue.cc:1048] T 84f2da98864b4425bf7c0055951fdd4b P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
W20260709 13:21:38.639513 22069 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.876s	user 0.018s	sys 0.016s
I20260709 13:21:38.661789 16816 raft_consensus.cc:1240] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 [term 16 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 15. Current term is 16. Ops: []
I20260709 13:21:38.580184 22134 raft_consensus.cc:493] T b6f07842d3aa47858df5665874c86f3a P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: Starting leader election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:21:38.676960 21245 consensus_queue.cc:1059] T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
W20260709 13:21:38.642254 22155 log.cc:927] Time spent T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860: Append to log took a long time: real 0.083s	user 0.000s	sys 0.000s
I20260709 13:21:38.678895 16226 raft_consensus.cc:1240] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 17. Current term is 18. Ops: []
I20260709 13:21:38.623896 22043 raft_consensus.cc:740] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:21:38.648022 21930 raft_consensus.cc:493] T ecc1e363230043fe9fef390f0b979f20 P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
W20260709 13:21:38.677029 22069 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.918s	user 0.019s	sys 0.017s
I20260709 13:21:38.661109 22147 raft_consensus.cc:2804] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:21:38.694679 21245 raft_consensus.cc:493] T d16dfad0c3b540e5b4b86492ddf87208 P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:38.695232 21245 raft_consensus.cc:515] T d16dfad0c3b540e5b4b86492ddf87208 P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:38.680574 22167 raft_consensus.cc:2804] T 02838c86531d4f98bf58d1c22b8baf77 P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:21:38.654008 22168 raft_consensus.cc:493] T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
W20260709 13:21:38.697371 16115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 260810us
I20260709 13:21:38.682308 22134 raft_consensus.cc:3060] T b6f07842d3aa47858df5665874c86f3a P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:21:38.692734 16817 raft_consensus.cc:2468] T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 18.
I20260709 13:21:38.702557 16192 leader_election.cc:304] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:21:38.716526 21245 leader_election.cc:290] T d16dfad0c3b540e5b4b86492ddf87208 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:38.717006 21245 consensus_queue.cc:1048] T 9c61a39fd4f84424b129c1a6f9ba404d P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:21:38.715847 22027 consensus_queue.cc:1059] T af57996a8be84d398f0f023e8d08ef77 P 647a343a542d4a139fe106c19095129b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.005s
I20260709 13:21:38.702610 22167 raft_consensus.cc:493] T 02838c86531d4f98bf58d1c22b8baf77 P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:38.726584 22167 raft_consensus.cc:3060] T 02838c86531d4f98bf58d1c22b8baf77 P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:21:38.702472 22147 raft_consensus.cc:697] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [term 21 LEADER]: Becoming Leader. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:21:38.732126 22147 consensus_queue.cc:237] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.13, Last appended by leader: 13, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:38.702314 22183 raft_consensus.cc:3055] T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:21:38.735469 22183 raft_consensus.cc:740] T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
W20260709 13:21:38.742412 16264 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:38.713548 22140 raft_consensus.cc:2804] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:21:38.698340 22153 consensus_queue.cc:1059] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.048s
W20260709 13:21:38.771715 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 74623.8us
I20260709 13:21:38.736339 22037 raft_consensus.cc:493] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:38.698115 21930 raft_consensus.cc:515] T ecc1e363230043fe9fef390f0b979f20 P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:38.798424 16219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d16dfad0c3b540e5b4b86492ddf87208" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
W20260709 13:21:38.800421 16789 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:38.804812 16821 raft_consensus.cc:1240] T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860 [term 18 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 17. Current term is 18. Ops: []
I20260709 13:21:38.772773 22104 leader_election.cc:290] T 68a6838d82ef4cac83507f906139d90f P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 16 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:21:38.776185 22140 raft_consensus.cc:697] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac [term 18 LEADER]: Becoming Leader. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:21:38.799194 16219 raft_consensus.cc:2468] T d16dfad0c3b540e5b4b86492ddf87208 P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 16.
I20260709 13:21:38.821553 16821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c11074579b046d99e585c2818e18a2c" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
W20260709 13:21:38.823498 16115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 121123us
I20260709 13:21:38.830572 16818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e9b2bf7011f4ae688a79d7bb5a447b4" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
W20260709 13:21:38.830749 16117 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 87144.6us
I20260709 13:21:38.840041 16817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af57996a8be84d398f0f023e8d08ef77" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:21:38.840665 16817 raft_consensus.cc:3060] T af57996a8be84d398f0f023e8d08ef77 P cb82cc7bd6db46d980c75237a1402860 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:21:38.842271 16824 raft_consensus.cc:1240] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [term 21 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 20. Current term is 21. Ops: []
I20260709 13:21:38.837364 16822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7ca61592e848ea9b0fb81a2ce33cdc" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:21:38.746122 22043 consensus_queue.cc:260] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:38.805527 22152 leader_election.cc:290] T af57996a8be84d398f0f023e8d08ef77 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 18 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:38.852281 16219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c2095a5f4ad49408dbc7f69c606120d" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 13 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:21:38.852890 16219 raft_consensus.cc:3060] T 7c2095a5f4ad49408dbc7f69c606120d P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:21:38.855113 16821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7ca61592e848ea9b0fb81a2ce33cdc" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:38.835970 22057 leader_election.cc:290] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:38.795886 22037 raft_consensus.cc:515] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:38.835933 22026 raft_consensus.cc:493] T ee9ed67006144797b1de6ea8c2013a68 P cb82cc7bd6db46d980c75237a1402860 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:38.836133 22031 raft_consensus.cc:493] T 66f75124215f4ca6bef831e174a792d8 P cb82cc7bd6db46d980c75237a1402860 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:38.822322 22098 consensus_queue.cc:1059] T 8c11074579b046d99e585c2818e18a2c P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: INVALID_TERM, Last received: 15.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.035s
I20260709 13:21:38.831817 22069 consensus_queue.cc:1059] T 3a7ca61592e848ea9b0fb81a2ce33cdc P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: INVALID_TERM, Last received: 17.14, Next index: 15, Last known committed idx: 13, Time since last communication: 0.029s
I20260709 13:21:38.857124 22027 raft_consensus.cc:493] T 2794af5b99984efebdee564035841cb0 P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:38.838346 21623 consensus_queue.cc:1048] T 8638ee363273426db1393dd81e5c7b72 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.025s
I20260709 13:21:38.863915 22043 raft_consensus.cc:3060] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:21:38.851086 22097 consensus_queue.cc:1048] T fffab26427e24d398e2695aad40ca134 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.025s
W20260709 13:21:38.888868 16790 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:38.874960 22031 raft_consensus.cc:515] T 66f75124215f4ca6bef831e174a792d8 P cb82cc7bd6db46d980c75237a1402860 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:38.874446 21704 leader_election.cc:290] T 7c2095a5f4ad49408dbc7f69c606120d P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 13 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:38.710439 22168 raft_consensus.cc:515] T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:38.908468 16790 leader_election.cc:304] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:21:38.910328 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b7d1ab519044eda2d5e04a07242f25" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:38.911278 16217 raft_consensus.cc:2468] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 14.
I20260709 13:21:38.798789 22158 raft_consensus.cc:515] T b87147731a68400e8cce9b879daa7c5d P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:38.882308 22041 raft_consensus.cc:3055] T 8c11074579b046d99e585c2818e18a2c P cb82cc7bd6db46d980c75237a1402860 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:21:38.875424 22026 raft_consensus.cc:515] T ee9ed67006144797b1de6ea8c2013a68 P cb82cc7bd6db46d980c75237a1402860 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:38.919006 16220 raft_consensus.cc:1275] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 13:21:38.923622 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "829b4ad8c5a54eababeea1181b960d4b" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:38.917032 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5718c623ebee4d2bb770f0f9ddb2f164" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 17 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:38.771628 22183 consensus_queue.cc:260] T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 15.8, Last appended by leader: 8, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:38.896299 16817 raft_consensus.cc:2468] T af57996a8be84d398f0f023e8d08ef77 P cb82cc7bd6db46d980c75237a1402860 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 18.
I20260709 13:21:38.883337 22098 consensus_queue.cc:1059] T 19baa7bc367844c288bca8f81439c413 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: INVALID_TERM, Last received: 17.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.033s
I20260709 13:21:38.932909 16192 leader_election.cc:304] T af57996a8be84d398f0f023e8d08ef77 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 
W20260709 13:21:38.936290 16789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 90298.7us
I20260709 13:21:38.903923 22153 raft_consensus.cc:493] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:38.945382 22183 raft_consensus.cc:3060] T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:21:38.932905 22184 log.cc:927] Time spent T 9e65676b21b748728ad39668cfe722ec P cb82cc7bd6db46d980c75237a1402860: Append to log took a long time: real 0.159s	user 0.001s	sys 0.001s
I20260709 13:21:38.881299 22027 raft_consensus.cc:515] T 2794af5b99984efebdee564035841cb0 P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:38.920688 22041 raft_consensus.cc:740] T 8c11074579b046d99e585c2818e18a2c P cb82cc7bd6db46d980c75237a1402860 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:21:38.956976 16219 raft_consensus.cc:2468] T 7c2095a5f4ad49408dbc7f69c606120d P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 13.
W20260709 13:21:38.957859 16791 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:38.947260 22057 raft_consensus.cc:2804] T af57996a8be84d398f0f023e8d08ef77 P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:21:38.897410 22140 consensus_queue.cc:237] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:38.987116 16817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:38.990913 22149 leader_election.cc:290] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:38.995647 22041 consensus_queue.cc:260] T 8c11074579b046d99e585c2818e18a2c P cb82cc7bd6db46d980c75237a1402860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:38.997026 22041 raft_consensus.cc:3060] T 8c11074579b046d99e585c2818e18a2c P cb82cc7bd6db46d980c75237a1402860 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:21:38.942103 22025 consensus_queue.cc:1175] T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "90f3da57796b488c9087a2f523adf045" responder_term: 15 status { last_received { term: 14 index: 7 } last_committed_idx: 7 error { code: PRECEDING_ENTRY_DIDNT_MATCH status { code: ILLEGAL_STATE message: "Log matching property violated. Preceding OpId in replica: term: 14 index: 7. Preceding OpId from leader: term: 15 index: 8. (index mismatch)" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:21:38.999758 22152 raft_consensus.cc:493] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:21:39.000522 22152 raft_consensus.cc:515] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:39.002094 22025 raft_consensus.cc:493] T 3884f864a96d428785895e5463e5728f P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:39.003813 22031 leader_election.cc:290] T 66f75124215f4ca6bef831e174a792d8 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:21:39.004436 22195 raft_consensus.cc:2749] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:21:39.006909 22158 leader_election.cc:290] T b87147731a68400e8cce9b879daa7c5d P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 17 election: Requested vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:38.983991 22057 raft_consensus.cc:697] T af57996a8be84d398f0f023e8d08ef77 P 1ca2725d290e47d68b1a4b609c9816ac [term 18 LEADER]: Becoming Leader. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:21:39.008102 22057 consensus_queue.cc:237] T af57996a8be84d398f0f023e8d08ef77 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:39.010210 22026 leader_election.cc:290] T ee9ed67006144797b1de6ea8c2013a68 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:21:38.985301 21930 leader_election.cc:290] T ecc1e363230043fe9fef390f0b979f20 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:39.016099 22037 leader_election.cc:290] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:39.017519 22027 leader_election.cc:290] T 2794af5b99984efebdee564035841cb0 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:38.953997 22153 raft_consensus.cc:515] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:39.022106 22153 leader_election.cc:290] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:39.024350 22167 raft_consensus.cc:515] T 02838c86531d4f98bf58d1c22b8baf77 P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:39.026224 16821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74ca8140f35c4ae29e8a664f6bdfc829" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:39.026557 22167 leader_election.cc:290] T 02838c86531d4f98bf58d1c22b8baf77 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 17 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:39.028221 22143 raft_consensus.cc:3055] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:21:39.028807 22143 raft_consensus.cc:740] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
W20260709 13:21:39.030345 22174 log.cc:927] Time spent T ecc1e363230043fe9fef390f0b979f20 P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.516s	user 0.003s	sys 0.000s
I20260709 13:21:39.029983 22143 consensus_queue.cc:260] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.14, Last appended by leader: 14, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:39.002666 22025 raft_consensus.cc:515] T 3884f864a96d428785895e5463e5728f P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:21:39.039779 16789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 102422us
W20260709 13:21:39.042440 16263 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:21:39.043524 16263 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:21:39.040829 16266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 467043us
I20260709 13:21:39.048091 16266 leader_election.cc:304] T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac
I20260709 13:21:39.048305 16824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f82224f4564fea93572e8206bad1fe" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:21:39.049713 16819 raft_consensus.cc:2410] T c9944d5e4b8c4109af9955c4f08c9095 P cb82cc7bd6db46d980c75237a1402860 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac for term 19 because replica has last-logged OpId of term: 17 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 12.
I20260709 13:21:39.056475 16292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02838c86531d4f98bf58d1c22b8baf77" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:39.057487 16292 raft_consensus.cc:2468] T 02838c86531d4f98bf58d1c22b8baf77 P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 15.
I20260709 13:21:39.060752 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02838c86531d4f98bf58d1c22b8baf77" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:21:39.061434 16225 raft_consensus.cc:3060] T 02838c86531d4f98bf58d1c22b8baf77 P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:21:39.038468 22029 consensus_meta.cc:287] T 74ca8140f35c4ae29e8a664f6bdfc829 P cb82cc7bd6db46d980c75237a1402860: Time spent flushing consensus metadata: real 0.738s	user 0.012s	sys 0.001s
I20260709 13:21:39.033473 22064 raft_consensus.cc:2804] T 9e65676b21b748728ad39668cfe722ec P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:21:39.068002 16822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdbd78acb4cf46298e0de8ba0d130e19" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:39.071959 16225 raft_consensus.cc:2468] T 02838c86531d4f98bf58d1c22b8baf77 P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 17.
I20260709 13:21:39.075730 16818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87147731a68400e8cce9b879daa7c5d" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:21:39.076599 16294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434bff3082d141838068f323bdb0d1ed" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:39.075730 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2372981006d4b218215c28964dea623" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:39.079577 16217 raft_consensus.cc:2468] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 18.
I20260709 13:21:39.081431 16220 raft_consensus.cc:1275] T 8cc959fb7f704fba9009c0595467d074 P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 13:21:39.078605 16816 raft_consensus.cc:1275] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 [term 16 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 14 index: 13. Preceding OpId from leader: term: 16 index: 14. (index mismatch)
I20260709 13:21:39.078584 16266 leader_election.cc:304] T 3884f864a96d428785895e5463e5728f P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:21:39.048941 22106 raft_consensus.cc:2749] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
W20260709 13:21:39.087261 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 610138us
I20260709 13:21:39.061745 21579 raft_consensus.cc:493] T c9944d5e4b8c4109af9955c4f08c9095 P cb82cc7bd6db46d980c75237a1402860 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:39.062120 22104 raft_consensus.cc:493] T 8d4ab870a05347868956aa8b841b2981 P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:39.089267 16822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a9de347c711477e87c582d003f36925" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:39.088056 16293 raft_consensus.cc:1240] T 5a96373bb0e14dce8a266b2fc6c58507 P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 12. Current term is 13. Ops: []
I20260709 13:21:39.055371 22085 raft_consensus.cc:2804] T ecc1e363230043fe9fef390f0b979f20 P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:21:39.090237 16822 raft_consensus.cc:2468] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 19.
I20260709 13:21:39.094885 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d16dfad0c3b540e5b4b86492ddf87208" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:39.095688 16298 raft_consensus.cc:2468] T d16dfad0c3b540e5b4b86492ddf87208 P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 16.
I20260709 13:21:39.014235 22152 leader_election.cc:290] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:39.067234 16292 raft_consensus.cc:1240] T 68a6838d82ef4cac83507f906139d90f P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 15. Current term is 16. Ops: []
I20260709 13:21:39.100075 16824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2794af5b99984efebdee564035841cb0" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:39.102497 16293 raft_consensus.cc:1240] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 14. Current term is 15. Ops: []
I20260709 13:21:39.089761 22079 raft_consensus.cc:2804] T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:21:39.111088 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1df93779209748b9860f6d893dd3de53" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:39.109865 16297 raft_consensus.cc:3055] T ada5a004ba7c46819051e3ccec71ffdb P 90f3da57796b488c9087a2f523adf045 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:21:39.111933 16297 raft_consensus.cc:740] T ada5a004ba7c46819051e3ccec71ffdb P 90f3da57796b488c9087a2f523adf045 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:21:39.107180 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8638ee363273426db1393dd81e5c7b72" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:21:39.113127 16225 raft_consensus.cc:2368] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 for earlier term 16. Current term is 17.
I20260709 13:21:39.114483 16220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ab0ac8347641e6955e9bce7d90c1a2" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:39.117044 16294 raft_consensus.cc:1240] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 14. Current term is 15. Ops: [14.9-14.9]
I20260709 13:21:39.078696 22029 raft_consensus.cc:515] T 74ca8140f35c4ae29e8a664f6bdfc829 P cb82cc7bd6db46d980c75237a1402860 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:39.119112 16299 raft_consensus.cc:1240] T 18ad7e3af5504571a69d10c00750a01c P 90f3da57796b488c9087a2f523adf045 [term 14 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 13. Current term is 14. Ops: []
I20260709 13:21:39.120872 16822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2372981006d4b218215c28964dea623" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:39.120384 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f5f89186204242bf377f7c42b2e2da" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:21:39.121801 16822 raft_consensus.cc:2468] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 18.
I20260709 13:21:39.122143 16217 raft_consensus.cc:2393] T c2f5f89186204242bf377f7c42b2e2da P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 17: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
I20260709 13:21:39.122660 16226 raft_consensus.cc:2393] T 1df93779209748b9860f6d893dd3de53 P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 15: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
W20260709 13:21:39.123178 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 214815us
I20260709 13:21:39.092751 22085 raft_consensus.cc:493] T ecc1e363230043fe9fef390f0b979f20 P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Starting leader election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:21:39.128539 22025 leader_election.cc:290] T 3884f864a96d428785895e5463e5728f P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:39.102218 22082 raft_consensus.cc:2804] T 3884f864a96d428785895e5463e5728f P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:21:39.134841 16824 raft_consensus.cc:1240] T af57996a8be84d398f0f023e8d08ef77 P cb82cc7bd6db46d980c75237a1402860 [term 18 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 17. Current term is 18. Ops: []
I20260709 13:21:39.135236 16821 raft_consensus.cc:2393] T 74ca8140f35c4ae29e8a664f6bdfc829 P cb82cc7bd6db46d980c75237a1402860 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 17: Already voted for candidate cb82cc7bd6db46d980c75237a1402860 in this term.
I20260709 13:21:39.131307 16297 consensus_queue.cc:260] T ada5a004ba7c46819051e3ccec71ffdb P 90f3da57796b488c9087a2f523adf045 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:39.137759 16297 raft_consensus.cc:3060] T ada5a004ba7c46819051e3ccec71ffdb P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:21:39.139932 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02838c86531d4f98bf58d1c22b8baf77" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:21:39.144361 16225 raft_consensus.cc:2368] T 02838c86531d4f98bf58d1c22b8baf77 P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 for earlier term 15. Current term is 17.
I20260709 13:21:39.091785 22134 raft_consensus.cc:515] T b6f07842d3aa47858df5665874c86f3a P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:39.145262 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:39.093640 22104 raft_consensus.cc:515] T 8d4ab870a05347868956aa8b841b2981 P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:39.148823 16117 leader_election.cc:304] T 23f82224f4564fea93572e8206bad1fe P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860
I20260709 13:21:39.149895 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6f07842d3aa47858df5665874c86f3a" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:39.094645 21579 raft_consensus.cc:515] T c9944d5e4b8c4109af9955c4f08c9095 P cb82cc7bd6db46d980c75237a1402860 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:39.130272 22085 raft_consensus.cc:3060] T ecc1e363230043fe9fef390f0b979f20 P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:21:39.130605 22079 raft_consensus.cc:697] T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045 [term 16 LEADER]: Becoming Leader. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:21:39.106318 22106 consensus_queue.cc:1059] T 66ae176961b74813a58bc15c9c4c6f2a P 90f3da57796b488c9087a2f523adf045 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.034s
I20260709 13:21:39.166051 16144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26354361e9b044cea1c2b86c4882fe92" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:39.176594 16144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e9b2bf7011f4ae688a79d7bb5a447b4" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:39.173229 16147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a959dbecc2648eb85d1cc550f5a3245" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:21:39.175793 16151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1df93779209748b9860f6d893dd3de53" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:39.179500 16151 raft_consensus.cc:2410] T 1df93779209748b9860f6d893dd3de53 P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 for term 15 because replica has last-logged OpId of term: 14 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 8.
I20260709 13:21:39.172706 16142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c74490c40243a88e62d1760b5cd814" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
W20260709 13:21:39.169725 16191 leader_election.cc:336] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Timed out: RequestConsensusVote RPC to 127.14.177.129:36741 timed out after 1.536s (SENT)
W20260709 13:21:39.181559 16191 leader_election.cc:336] T 8cc959fb7f704fba9009c0595467d074 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Timed out: RequestConsensusVote RPC to 127.14.177.129:36741 timed out after 1.549s (SENT)
I20260709 13:21:39.182051 16191 leader_election.cc:304] T 8cc959fb7f704fba9009c0595467d074 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:21:39.170255 16221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a878999c0024d2abe517484180290fc" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:39.168445 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6f07842d3aa47858df5665874c86f3a" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:39.184614 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a96373bb0e14dce8a266b2fc6c58507" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: true dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:21:39.185168 16223 raft_consensus.cc:1240] T b6f07842d3aa47858df5665874c86f3a P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 15. Current term is 16. Ops: []
I20260709 13:21:39.185253 16225 raft_consensus.cc:3060] T 5a96373bb0e14dce8a266b2fc6c58507 P 1ca2725d290e47d68b1a4b609c9816ac [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:21:39.185745 16144 raft_consensus.cc:2393] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 647a343a542d4a139fe106c19095129b [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 14: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
I20260709 13:21:39.186942 16146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f82224f4564fea93572e8206bad1fe" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: true dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:21:39.187105 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8158bbd17b994d2eb96f560bd0f1b88d" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:21:39.187566 16148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:39.188143 16265 leader_election.cc:400] T 8638ee363273426db1393dd81e5c7b72 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 16 election: Vote denied by peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943) with higher term. Message: Invalid argument: T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 for earlier term 16. Current term is 17.
I20260709 13:21:39.188372 16148 raft_consensus.cc:2393] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 18: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
I20260709 13:21:39.189199 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a878999c0024d2abe517484180290fc" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:39.144580 22082 raft_consensus.cc:697] T 3884f864a96d428785895e5463e5728f P 90f3da57796b488c9087a2f523adf045 [term 12 LEADER]: Becoming Leader. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:21:39.194871 16145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02838c86531d4f98bf58d1c22b8baf77" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:21:39.195323 16151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3884f864a96d428785895e5463e5728f" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:21:39.195453 16143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a96373bb0e14dce8a266b2fc6c58507" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: true dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:21:39.200508 16144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b7d1ab519044eda2d5e04a07242f25" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:39.205824 16151 raft_consensus.cc:3060] T 3884f864a96d428785895e5463e5728f P 647a343a542d4a139fe106c19095129b [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:21:39.206130 16144 raft_consensus.cc:2468] T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 14.
I20260709 13:21:39.206463 16142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18ad7e3af5504571a69d10c00750a01c" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:21:39.205849 16143 raft_consensus.cc:3055] T 5a96373bb0e14dce8a266b2fc6c58507 P 647a343a542d4a139fe106c19095129b [term 12 LEADER]: Stepping down as leader of term 12
W20260709 13:21:39.208475 16789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 54393.5us
W20260709 13:21:39.208709 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 81418.4us
I20260709 13:21:39.201263 16116 leader_election.cc:304] T d16dfad0c3b540e5b4b86492ddf87208 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
I20260709 13:21:39.207808 16143 raft_consensus.cc:740] T 5a96373bb0e14dce8a266b2fc6c58507 P 647a343a542d4a139fe106c19095129b [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:21:39.215062 16221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a959dbecc2648eb85d1cc550f5a3245" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:21:39.217193 16144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "829b4ad8c5a54eababeea1181b960d4b" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:39.217810 16144 raft_consensus.cc:2468] T 829b4ad8c5a54eababeea1181b960d4b P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 15.
I20260709 13:21:39.202442 16219 raft_consensus.cc:1240] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 16. Current term is 17. Ops: [14.11-16.12]
I20260709 13:21:39.200874 16297 raft_consensus.cc:1275] T ada5a004ba7c46819051e3ccec71ffdb P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 13:21:39.212443 16148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5718c623ebee4d2bb770f0f9ddb2f164" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 17 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:39.214798 16145 raft_consensus.cc:2368] T 02838c86531d4f98bf58d1c22b8baf77 P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 for earlier term 15. Current term is 17.
W20260709 13:21:39.200606 22193 log.cc:927] Time spent T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 0.076s	user 0.000s	sys 0.000s
I20260709 13:21:39.139891 22197 consensus_queue.cc:1059] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: INVALID_TERM, Last received: 17.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.066s
I20260709 13:21:39.187762 22057 raft_consensus.cc:2749] T 8cc959fb7f704fba9009c0595467d074 P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:21:39.225597 16143 consensus_queue.cc:260] T 5a96373bb0e14dce8a266b2fc6c58507 P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.6, Last appended by leader: 6, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:39.226725 16143 raft_consensus.cc:3060] T 5a96373bb0e14dce8a266b2fc6c58507 P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:21:39.170310 22204 raft_consensus.cc:2749] T 23f82224f4564fea93572e8206bad1fe P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:21:39.223284 16221 raft_consensus.cc:2393] T 5a959dbecc2648eb85d1cc550f5a3245 P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 15: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
W20260709 13:21:39.228996 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 59220.3us
I20260709 13:21:39.229511 16151 raft_consensus.cc:2468] T 3884f864a96d428785895e5463e5728f P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 12.
I20260709 13:21:39.221308 16225 raft_consensus.cc:2468] T 5a96373bb0e14dce8a266b2fc6c58507 P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 13.
I20260709 13:21:39.232087 16824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3884f864a96d428785895e5463e5728f" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:39.236434 16824 raft_consensus.cc:2393] T 3884f864a96d428785895e5463e5728f P cb82cc7bd6db46d980c75237a1402860 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 12: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
I20260709 13:21:39.236914 16142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ad5dbfa99ee4b8d9e28be92b3f36db5" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:21:39.205144 22106 consensus_queue.cc:1243] T 66ae176961b74813a58bc15c9c4c6f2a P 90f3da57796b488c9087a2f523adf045 [LEADER]: Peer cb82cc7bd6db46d980c75237a1402860 log is divergent from this leader: its last log entry 16.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 11
I20260709 13:21:39.238935 16219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7ca61592e848ea9b0fb81a2ce33cdc" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:21:39.245213 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c74490c40243a88e62d1760b5cd814" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:21:39.244865 16146 raft_consensus.cc:2393] T 23f82224f4564fea93572e8206bad1fe P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 13: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
I20260709 13:21:39.245760 16219 raft_consensus.cc:3060] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:21:39.247962 16266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 155750us
I20260709 13:21:39.250471 16292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87ac0d1f7e8a419f9bcbc04aee0e067e" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:21:39.255509 16145 raft_consensus.cc:1240] T 7c2095a5f4ad49408dbc7f69c606120d P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 12. Current term is 13. Ops: []
I20260709 13:21:39.259938 16292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8158bbd17b994d2eb96f560bd0f1b88d" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:39.260804 16292 raft_consensus.cc:2393] T 8158bbd17b994d2eb96f560bd0f1b88d P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 15: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
I20260709 13:21:39.228341 22202 raft_consensus.cc:493] T 434bff3082d141838068f323bdb0d1ed P cb82cc7bd6db46d980c75237a1402860 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:21:39.253679 16221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18ad7e3af5504571a69d10c00750a01c" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:21:39.263358 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66ae176961b74813a58bc15c9c4c6f2a" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:39.263872 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8158bbd17b994d2eb96f560bd0f1b88d" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:21:39.267477 16225 raft_consensus.cc:3060] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:21:39.258309 16151 raft_consensus.cc:1240] T 8cc959fb7f704fba9009c0595467d074 P 647a343a542d4a139fe106c19095129b [term 15 LEADER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 14. Current term is 15. Ops: [14.11-14.11]
I20260709 13:21:39.266351 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806d7690c2b644959152074bb2cab0a7" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:21:39.237025 22025 raft_consensus.cc:2804] T d16dfad0c3b540e5b4b86492ddf87208 P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:21:39.269788 16294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1df93779209748b9860f6d893dd3de53" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:21:39.272759 16294 raft_consensus.cc:3060] T 1df93779209748b9860f6d893dd3de53 P 90f3da57796b488c9087a2f523adf045 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:21:39.288828 16292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9944d5e4b8c4109af9955c4f08c9095" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 19 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:39.289328 16225 raft_consensus.cc:2468] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 15.
I20260709 13:21:39.228547 22026 consensus_queue.cc:1059] T 19baa7bc367844c288bca8f81439c413 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true }, Status: INVALID_TERM, Last received: 17.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.035s
I20260709 13:21:39.303902 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ad5dbfa99ee4b8d9e28be92b3f36db5" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:21:39.295508 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4dc6b0541ca431ebc757beac00521f1" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:21:39.305076 16298 raft_consensus.cc:3060] T e4dc6b0541ca431ebc757beac00521f1 P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:21:39.296093 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fffab26427e24d398e2695aad40ca134" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:39.299436 16292 raft_consensus.cc:2410] T c9944d5e4b8c4109af9955c4f08c9095 P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac for term 19 because replica has last-logged OpId of term: 17 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 12.
I20260709 13:21:39.302155 16219 raft_consensus.cc:2468] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 18.
I20260709 13:21:39.306605 16293 raft_consensus.cc:2410] T fffab26427e24d398e2695aad40ca134 P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac for term 17 because replica has last-logged OpId of term: 15 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 8.
I20260709 13:21:39.306977 16219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c11074579b046d99e585c2818e18a2c" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:21:39.309477 16219 raft_consensus.cc:1240] T fffab26427e24d398e2695aad40ca134 P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 15. Current term is 16. Ops: [15.9-15.9]
I20260709 13:21:39.311640 16219 raft_consensus.cc:1240] T 8cc959fb7f704fba9009c0595467d074 P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 14. Current term is 15. Ops: [14.11-14.11]
I20260709 13:21:39.237155 21802 consensus_queue.cc:1048] T 2794af5b99984efebdee564035841cb0 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.035s
I20260709 13:21:39.313588 16151 raft_consensus.cc:1240] T 84f2da98864b4425bf7c0055951fdd4b P 647a343a542d4a139fe106c19095129b [term 13 LEADER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 12. Current term is 13. Ops: []
I20260709 13:21:39.313611 16221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68a6838d82ef4cac83507f906139d90f" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
W20260709 13:21:39.315404 16191 leader_election.cc:336] T 5a959dbecc2648eb85d1cc550f5a3245 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Timed out: RequestConsensusVote RPC to 127.14.177.129:36741 timed out after 1.690s (SENT)
I20260709 13:21:39.316453 16292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66ae176961b74813a58bc15c9c4c6f2a" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:21:39.288008 22029 leader_election.cc:290] T 74ca8140f35c4ae29e8a664f6bdfc829 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 17 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:39.332530 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e4bfb9090c8460eb14fa7e0d818e7f1" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:21:39.339571 16293 raft_consensus.cc:3060] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:21:39.335119 16225 raft_consensus.cc:1240] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 16. Current term is 17. Ops: [16.13-16.13]
I20260709 13:21:39.271831 22025 raft_consensus.cc:493] T d16dfad0c3b540e5b4b86492ddf87208 P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:39.345201 16292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c74490c40243a88e62d1760b5cd814" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:39.347838 16225 raft_consensus.cc:1240] T 7c2095a5f4ad49408dbc7f69c606120d P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 12. Current term is 13. Ops: []
W20260709 13:21:39.350613 16789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 129657us
W20260709 13:21:39.349663 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 124444us
I20260709 13:21:39.352581 16116 leader_election.cc:304] T 7c2095a5f4ad49408dbc7f69c606120d P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
I20260709 13:21:39.355020 16142 raft_consensus.cc:2410] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 for term 15 because replica has last-logged OpId of term: 14 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 8.
I20260709 13:21:39.354880 16116 leader_election.cc:304] T 02838c86531d4f98bf58d1c22b8baf77 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
I20260709 13:21:39.295787 22079 consensus_queue.cc:237] T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:39.303505 22082 consensus_queue.cc:237] T 3884f864a96d428785895e5463e5728f P 90f3da57796b488c9087a2f523adf045 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 10.11, Last appended by leader: 11, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:21:39.365136 16820 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49896 (request call id 1394) took 2364 ms (client timeout 1531 ms). Trace:
W20260709 13:21:39.365614 16820 rpcz_store.cc:269] 0709 13:21:36.985485 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:36.985627 (+   142us) service_pool.cc:224] Handling call
0709 13:21:39.349200 (+2363573us) spinlock_profiling.cc:243] Waited 1.41 s on lock 0x7b54007d0354. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:21:39.349361 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3241953664}
I20260709 13:21:39.295648 22059 raft_consensus.cc:493] T 19baa7bc367844c288bca8f81439c413 P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:21:39.366887 16292 raft_consensus.cc:2393] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 15: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
I20260709 13:21:39.367949 21704 raft_consensus.cc:2804] T 02838c86531d4f98bf58d1c22b8baf77 P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Leader election won for term 17
W20260709 13:21:39.368824 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 116282us
I20260709 13:21:39.371119 16144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4f65cdf00b748bb963ee6e81ef43b79" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:39.368054 16116 leader_election.cc:304] T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
I20260709 13:21:39.376426 16292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7ca61592e848ea9b0fb81a2ce33cdc" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
W20260709 13:21:39.372915 16191 leader_election.cc:336] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Timed out: RequestConsensusVote RPC to 127.14.177.129:36741 timed out after 1.746s (SENT)
I20260709 13:21:39.377305 16144 raft_consensus.cc:2376] T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 17. Re-sending same reply.
I20260709 13:21:39.378216 16144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87ac0d1f7e8a419f9bcbc04aee0e067e" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:39.378921 16144 raft_consensus.cc:2468] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 17.
I20260709 13:21:39.380179 16790 leader_election.cc:304] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:21:39.382467 16148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a959dbecc2648eb85d1cc550f5a3245" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:39.451284 16144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cc959fb7f704fba9009c0595467d074" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:39.460469 16145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0eab25ce4c64e018a041beeda2867db" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:39.397815 16292 raft_consensus.cc:2393] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 18: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
I20260709 13:21:39.453883 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f2da98864b4425bf7c0055951fdd4b" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
W20260709 13:21:39.455696 16191 leader_election.cc:336] T 84f2da98864b4425bf7c0055951fdd4b P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Timed out: RequestConsensusVote RPC to 127.14.177.129:36741 timed out after 1.776s (SENT)
I20260709 13:21:39.354080 22025 raft_consensus.cc:3060] T d16dfad0c3b540e5b4b86492ddf87208 P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:21:39.381306 16790 leader_election.cc:304] T 829b4ad8c5a54eababeea1181b960d4b P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:21:39.455098 16265 leader_election.cc:304] T 26354361e9b044cea1c2b86c4882fe92 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
I20260709 13:21:39.379531 22043 raft_consensus.cc:2804] T 7c2095a5f4ad49408dbc7f69c606120d P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:21:39.464275 16292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:39.340678 22069 consensus_queue.cc:1059] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.052s
I20260709 13:21:39.467391 16144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1df93779209748b9860f6d893dd3de53" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:21:39.269577 22202 raft_consensus.cc:515] T 434bff3082d141838068f323bdb0d1ed P cb82cc7bd6db46d980c75237a1402860 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:39.464268 16191 leader_election.cc:304] T 84f2da98864b4425bf7c0055951fdd4b P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:21:39.468251 16148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5d59407eb545b895d8d922e0dd69fd" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:39.471895 16292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74ca8140f35c4ae29e8a664f6bdfc829" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
W20260709 13:21:39.475198 16191 leader_election.cc:336] T 5a96373bb0e14dce8a266b2fc6c58507 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Timed out: RequestConsensusVote RPC to 127.14.177.129:36741 timed out after 1.817s (SENT)
I20260709 13:21:39.475651 16298 raft_consensus.cc:2468] T e4dc6b0541ca431ebc757beac00521f1 P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 16.
I20260709 13:21:39.476068 16191 leader_election.cc:304] T 5a96373bb0e14dce8a266b2fc6c58507 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:21:39.476986 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18ad7e3af5504571a69d10c00750a01c" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:39.479528 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4dc6b0541ca431ebc757beac00521f1" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:39.482662 16149 raft_consensus.cc:2410] T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac for term 16 because replica has last-logged OpId of term: 15 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 7.
I20260709 13:21:39.462568 22201 raft_consensus.cc:2804] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:21:39.481127 22140 raft_consensus.cc:2749] T 5a96373bb0e14dce8a266b2fc6c58507 P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:21:39.481685 16148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806d7690c2b644959152074bb2cab0a7" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:21:39.485950 16148 raft_consensus.cc:3060] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:21:39.481204 16293 raft_consensus.cc:2468] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 17.
I20260709 13:21:39.383122 22200 consensus_queue.cc:1048] T ada5a004ba7c46819051e3ccec71ffdb P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.096s
W20260709 13:21:39.480646 16263 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:39.484506 16292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdbd78acb4cf46298e0de8ba0d130e19" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:39.489900 16145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4f65cdf00b748bb963ee6e81ef43b79" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:21:39.490517 16145 raft_consensus.cc:2376] T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 17. Re-sending same reply.
W20260709 13:21:39.484898 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 116947us
I20260709 13:21:39.486783 16144 raft_consensus.cc:2410] T 1df93779209748b9860f6d893dd3de53 P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac for term 15 because replica has last-logged OpId of term: 14 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 8.
I20260709 13:21:39.490057 16823 raft_consensus.cc:1240] T 8638ee363273426db1393dd81e5c7b72 P cb82cc7bd6db46d980c75237a1402860 [term 17 LEADER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 16. Current term is 17. Ops: []
I20260709 13:21:39.490885 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8158bbd17b994d2eb96f560bd0f1b88d" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
W20260709 13:21:39.492056 16191 leader_election.cc:336] T 02838c86531d4f98bf58d1c22b8baf77 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Timed out: RequestConsensusVote RPC to 127.14.177.129:36741 timed out after 1.864s (SENT)
I20260709 13:21:39.484014 16298 raft_consensus.cc:2393] T 18ad7e3af5504571a69d10c00750a01c P 90f3da57796b488c9087a2f523adf045 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 14: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
I20260709 13:21:39.473649 22134 leader_election.cc:290] T b6f07842d3aa47858df5665874c86f3a P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 16 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:39.495218 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02838c86531d4f98bf58d1c22b8baf77" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:39.480420 22210 raft_consensus.cc:2749] T 26354361e9b044cea1c2b86c4882fe92 P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:21:39.496496 22221 raft_consensus.cc:2804] T 829b4ad8c5a54eababeea1181b960d4b P cb82cc7bd6db46d980c75237a1402860 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:21:39.496773 22218 raft_consensus.cc:2804] T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:21:39.496948 22221 raft_consensus.cc:493] T 829b4ad8c5a54eababeea1181b960d4b P cb82cc7bd6db46d980c75237a1402860 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:39.497277 22218 raft_consensus.cc:493] T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Starting leader election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:21:39.497756 22218 raft_consensus.cc:3060] T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:21:39.497394 22221 raft_consensus.cc:3060] T 829b4ad8c5a54eababeea1181b960d4b P cb82cc7bd6db46d980c75237a1402860 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:21:39.500454 22209 log.cc:927] Time spent T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 0.124s	user 0.000s	sys 0.000s
I20260709 13:21:39.504268 16819 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49906 (request call id 1163) took 1019 ms. Trace:
I20260709 13:21:39.504716 16819 rpcz_store.cc:276] 0709 13:21:38.484692 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:38.485401 (+   709us) service_pool.cc:224] Handling call
0709 13:21:39.504070 (+1018669us) spinlock_profiling.cc:243] Waited 542 ms on lock 0x7b54004b8cd4. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:21:39.504205 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1245660928}
W20260709 13:21:39.505837 22170 log.cc:927] Time spent T 8c11074579b046d99e585c2818e18a2c P cb82cc7bd6db46d980c75237a1402860: Append to log took a long time: real 1.027s	user 0.003s	sys 0.000s
W20260709 13:21:39.506438 22130 log.cc:927] Time spent T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 0.523s	user 0.000s	sys 0.001s
I20260709 13:21:39.387624 22059 raft_consensus.cc:515] T 19baa7bc367844c288bca8f81439c413 P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:39.508314 16295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87147731a68400e8cce9b879daa7c5d" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
W20260709 13:21:39.516592 22192 log.cc:927] Time spent T 57a744f5aaf944678a63469cbf514015 P cb82cc7bd6db46d980c75237a1402860: Append to log took a long time: real 0.504s	user 0.007s	sys 0.003s
W20260709 13:21:39.518590 22199 log.cc:927] Time spent T 8c11074579b046d99e585c2818e18a2c P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 0.349s	user 0.000s	sys 0.000s
W20260709 13:21:39.519500 16263 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:39.521845 16142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5d59407eb545b895d8d922e0dd69fd" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:21:39.478932 22152 raft_consensus.cc:2749] T 84f2da98864b4425bf7c0055951fdd4b P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:21:39.526755 16292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c2095a5f4ad49408dbc7f69c606120d" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 13 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:21:39.508231 22064 raft_consensus.cc:493] T 9e65676b21b748728ad39668cfe722ec P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Starting leader election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
W20260709 13:21:39.536397 16191 leader_election.cc:336] T 829b4ad8c5a54eababeea1181b960d4b P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Timed out: RequestConsensusVote RPC to 127.14.177.129:36741 timed out after 1.870s (SENT)
I20260709 13:21:39.547375 16191 leader_election.cc:304] T 829b4ad8c5a54eababeea1181b960d4b P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
I20260709 13:21:39.539582 16292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2794af5b99984efebdee564035841cb0" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
W20260709 13:21:39.540421 16263 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:39.544560 16144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "829b4ad8c5a54eababeea1181b960d4b" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:39.549567 16144 raft_consensus.cc:2468] T 829b4ad8c5a54eababeea1181b960d4b P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 15.
W20260709 13:21:39.550253 16191 leader_election.cc:336] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 17 election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Timed out: RequestConsensusVote RPC to 127.14.177.129:36741 timed out after 1.765s (SENT)
I20260709 13:21:39.550854 16191 leader_election.cc:304] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:21:39.486830 22201 raft_consensus.cc:493] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:39.569989 16144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4dc6b0541ca431ebc757beac00521f1" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:21:39.571319 16144 raft_consensus.cc:2410] T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac for term 16 because replica has last-logged OpId of term: 15 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 7.
I20260709 13:21:39.575553 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a9de347c711477e87c582d003f36925" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:39.576493 16298 raft_consensus.cc:2468] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 19.
I20260709 13:21:39.580164 16148 raft_consensus.cc:2468] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 17.
I20260709 13:21:39.581525 16148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68a6838d82ef4cac83507f906139d90f" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:21:39.588752 16145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a96373bb0e14dce8a266b2fc6c58507" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
W20260709 13:21:39.589790 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 208544us
I20260709 13:21:39.591430 16144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66f75124215f4ca6bef831e174a792d8" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:21:39.594412 16144 raft_consensus.cc:2376] T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 15. Re-sending same reply.
I20260709 13:21:39.595333 16142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:39.596134 16144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fffab26427e24d398e2695aad40ca134" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
W20260709 13:21:39.592633 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 137628us
I20260709 13:21:39.493670 22104 leader_election.cc:290] T 8d4ab870a05347868956aa8b841b2981 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:21:39.591325 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02838c86531d4f98bf58d1c22b8baf77" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:21:39.566350 22210 raft_consensus.cc:493] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:21:39.569504 22147 raft_consensus.cc:3060] T 829b4ad8c5a54eababeea1181b960d4b P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:21:39.599933 16295 raft_consensus.cc:1275] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 13:21:39.585966 22201 raft_consensus.cc:3060] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:21:39.592407 16191 leader_election.cc:336] T d16dfad0c3b540e5b4b86492ddf87208 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 16 election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Timed out: RequestConsensusVote RPC to 127.14.177.129:36741 timed out after 1.921s (SENT)
I20260709 13:21:39.617849 16299 raft_consensus.cc:3060] T 02838c86531d4f98bf58d1c22b8baf77 P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Advancing to term 17
I20260709 13:21:39.560186 22064 raft_consensus.cc:3060] T 9e65676b21b748728ad39668cfe722ec P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:21:39.596153 16142 raft_consensus.cc:2468] T ecc1e363230043fe9fef390f0b979f20 P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 17.
W20260709 13:21:39.621819 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 130045us
I20260709 13:21:39.601742 16265 leader_election.cc:304] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
I20260709 13:21:39.638032 16148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d16dfad0c3b540e5b4b86492ddf87208" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:21:39.638795 16191 leader_election.cc:304] T d16dfad0c3b540e5b4b86492ddf87208 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
W20260709 13:21:39.640215 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 47912.3us
I20260709 13:21:39.641248 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3884f864a96d428785895e5463e5728f" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:39.641351 16145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c61a39fd4f84424b129c1a6f9ba404d" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:21:39.642766 16144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6f07842d3aa47858df5665874c86f3a" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:39.642685 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d4ab870a05347868956aa8b841b2981" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:39.645170 16265 leader_election.cc:304] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
W20260709 13:21:39.620960 22130 log.cc:927] Time spent T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 0.076s	user 0.001s	sys 0.000s
I20260709 13:21:39.619563 22134 raft_consensus.cc:2749] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:21:39.656003 16144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87ac0d1f7e8a419f9bcbc04aee0e067e" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:21:39.656507 16295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6f07842d3aa47858df5665874c86f3a" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
W20260709 13:21:39.656366 16789 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:21:39.658855 16263 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:39.668231 16147 raft_consensus.cc:2468] T 5a959dbecc2648eb85d1cc550f5a3245 P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 15.
I20260709 13:21:39.668503 16142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af57996a8be84d398f0f023e8d08ef77" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:39.669289 16147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18ad7e3af5504571a69d10c00750a01c" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:39.666441 22221 raft_consensus.cc:515] T 829b4ad8c5a54eababeea1181b960d4b P cb82cc7bd6db46d980c75237a1402860 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:39.674266 16145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee9ed67006144797b1de6ea8c2013a68" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 17 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
W20260709 13:21:39.651484 22214 log.cc:927] Time spent T 2794af5b99984efebdee564035841cb0 P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.388s	user 0.001s	sys 0.000s
W20260709 13:21:39.680593 16191 leader_election.cc:336] T 66f75124215f4ca6bef831e174a792d8 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 15 election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Timed out: RequestConsensusVote RPC to 127.14.177.129:36741 timed out after 2.048s (SENT)
I20260709 13:21:39.686246 16191 leader_election.cc:304] T 66f75124215f4ca6bef831e174a792d8 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
I20260709 13:21:39.686982 16294 raft_consensus.cc:2468] T 1df93779209748b9860f6d893dd3de53 P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 15.
I20260709 13:21:39.647799 21704 raft_consensus.cc:697] T 02838c86531d4f98bf58d1c22b8baf77 P 647a343a542d4a139fe106c19095129b [term 17 LEADER]: Becoming Leader. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:21:39.690639 21704 consensus_queue.cc:237] T 02838c86531d4f98bf58d1c22b8baf77 P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:39.695241 22134 raft_consensus.cc:2749] T 66f75124215f4ca6bef831e174a792d8 P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:21:39.696334 22059 leader_election.cc:290] T 19baa7bc367844c288bca8f81439c413 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:39.685755 22082 raft_consensus.cc:3060] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:21:39.702143 22041 consensus_meta.cc:287] T 8c11074579b046d99e585c2818e18a2c P cb82cc7bd6db46d980c75237a1402860: Time spent flushing consensus metadata: real 0.705s	user 0.005s	sys 0.005s
I20260709 13:21:39.644536 22152 raft_consensus.cc:2749] T d16dfad0c3b540e5b4b86492ddf87208 P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:21:39.708088 16144 raft_consensus.cc:2468] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 17.
I20260709 13:21:39.637987 22210 raft_consensus.cc:515] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:39.709487 16144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62fe3346cd3c42c4a55e14af9073bca4" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 22 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:39.710520 22210 leader_election.cc:290] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:21:39.655331 21802 raft_consensus.cc:3055] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:21:39.712127 21802 raft_consensus.cc:740] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:21:39.712392 22082 raft_consensus.cc:2749] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 90f3da57796b488c9087a2f523adf045 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:21:39.713182 16144 raft_consensus.cc:1240] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b [term 16 LEADER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 15. Current term is 16. Ops: []
I20260709 13:21:39.712952 21802 consensus_queue.cc:260] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:39.714041 21802 raft_consensus.cc:3060] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:21:39.722956 22152 raft_consensus.cc:493] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:39.723657 22152 raft_consensus.cc:515] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:39.726622 22152 leader_election.cc:290] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:39.681111 22037 consensus_queue.cc:1059] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: INVALID_TERM, Last received: 17.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.051s
I20260709 13:21:39.657310 22215 raft_consensus.cc:2749] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:21:39.733590 16821 raft_consensus.cc:1275] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [term 21 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
I20260709 13:21:39.735462 16142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c74490c40243a88e62d1760b5cd814" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:39.736775 16145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af57996a8be84d398f0f023e8d08ef77" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
W20260709 13:21:39.746361 16191 leader_election.cc:336] T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 17 election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Timed out: RequestConsensusVote RPC to 127.14.177.129:36741 timed out after 2.102s (SENT)
I20260709 13:21:39.746937 16191 leader_election.cc:304] T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:21:39.691717 16825 raft_consensus.cc:1240] T ada5a004ba7c46819051e3ccec71ffdb P cb82cc7bd6db46d980c75237a1402860 [term 17 LEADER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 16. Current term is 17. Ops: []
I20260709 13:21:39.748601 22152 raft_consensus.cc:2749] T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:21:39.749881 16825 raft_consensus.cc:1275] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P cb82cc7bd6db46d980c75237a1402860 [term 17 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 13:21:39.753417 16825 raft_consensus.cc:1275] T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860 [term 18 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 18 index: 15. (index mismatch)
W20260709 13:21:39.758631 16143 consensus_meta.cc:287] T 5a96373bb0e14dce8a266b2fc6c58507 P 647a343a542d4a139fe106c19095129b: Time spent flushing consensus metadata: real 0.531s	user 0.008s	sys 0.002s
I20260709 13:21:39.759455 16143 raft_consensus.cc:2468] T 5a96373bb0e14dce8a266b2fc6c58507 P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 13.
I20260709 13:21:39.694886 16299 raft_consensus.cc:2468] T 02838c86531d4f98bf58d1c22b8baf77 P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 17.
I20260709 13:21:39.760901 16147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d4ab870a05347868956aa8b841b2981" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:21:39.795586 16821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
W20260709 13:21:39.801983 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 198453us
I20260709 13:21:39.708221 22202 leader_election.cc:290] T 434bff3082d141838068f323bdb0d1ed P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:39.834201 16825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8638ee363273426db1393dd81e5c7b72" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:39.834789 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1df93779209748b9860f6d893dd3de53" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:21:39.835485 16299 raft_consensus.cc:2376] T 1df93779209748b9860f6d893dd3de53 P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 15. Re-sending same reply.
W20260709 13:21:39.838971 16117 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:39.825323 22043 raft_consensus.cc:697] T 7c2095a5f4ad49408dbc7f69c606120d P 647a343a542d4a139fe106c19095129b [term 13 LEADER]: Becoming Leader. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:21:39.843855 22082 raft_consensus.cc:493] T 5a96373bb0e14dce8a266b2fc6c58507 P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:39.854696 16825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9944d5e4b8c4109af9955c4f08c9095" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 19 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:39.855990 16825 raft_consensus.cc:2410] T c9944d5e4b8c4109af9955c4f08c9095 P cb82cc7bd6db46d980c75237a1402860 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac for term 19 because replica has last-logged OpId of term: 17 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 12.
I20260709 13:21:39.844374 22210 raft_consensus.cc:493] T c4f65cdf00b748bb963ee6e81ef43b79 P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
W20260709 13:21:39.859705 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 214597us
I20260709 13:21:39.861207 16265 leader_election.cc:400] T 02838c86531d4f98bf58d1c22b8baf77 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 15 election: Vote denied by peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741) with higher term. Message: Invalid argument: T 02838c86531d4f98bf58d1c22b8baf77 P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 for earlier term 15. Current term is 17.
I20260709 13:21:39.861630 16265 leader_election.cc:403] T 02838c86531d4f98bf58d1c22b8baf77 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 15 election: Cancelling election due to peer responding with higher term
I20260709 13:21:39.861142 22082 raft_consensus.cc:515] T 5a96373bb0e14dce8a266b2fc6c58507 P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:39.864070 16790 leader_election.cc:304] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:21:39.864038 22073 consensus_queue.cc:1059] T 7c2095a5f4ad49408dbc7f69c606120d P 90f3da57796b488c9087a2f523adf045 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:21:39.864948 22082 leader_election.cc:290] T 5a96373bb0e14dce8a266b2fc6c58507 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:21:39.865387 22073 raft_consensus.cc:3055] T 7c2095a5f4ad49408dbc7f69c606120d P 90f3da57796b488c9087a2f523adf045 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:21:39.865891 22073 raft_consensus.cc:740] T 7c2095a5f4ad49408dbc7f69c606120d P 90f3da57796b488c9087a2f523adf045 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:21:39.866789 22200 consensus_queue.cc:1048] T ada5a004ba7c46819051e3ccec71ffdb P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:21:39.867892 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee9ed67006144797b1de6ea8c2013a68" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:39.868331 22200 raft_consensus.cc:2749] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:21:39.868748 16217 raft_consensus.cc:2468] T ee9ed67006144797b1de6ea8c2013a68 P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 16.
I20260709 13:21:39.870209 16220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66f75124215f4ca6bef831e174a792d8" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:39.871054 16220 raft_consensus.cc:2468] T 66f75124215f4ca6bef831e174a792d8 P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 15.
W20260709 13:21:39.878087 16789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 220139us
I20260709 13:21:39.880165 16821 raft_consensus.cc:1240] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [term 21 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 20. Current term is 21. Ops: [17.12-20.13]
I20260709 13:21:39.882385 16825 raft_consensus.cc:1275] T af57996a8be84d398f0f023e8d08ef77 P cb82cc7bd6db46d980c75237a1402860 [term 18 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 13:21:39.887480 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74ca8140f35c4ae29e8a664f6bdfc829" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 17 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:21:39.888185 16226 raft_consensus.cc:3060] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:21:39.866982 22073 consensus_queue.cc:260] T 7c2095a5f4ad49408dbc7f69c606120d P 90f3da57796b488c9087a2f523adf045 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:39.892279 22073 raft_consensus.cc:3060] T 7c2095a5f4ad49408dbc7f69c606120d P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:21:39.895902 22221 leader_election.cc:290] T 829b4ad8c5a54eababeea1181b960d4b P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 16 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:21:39.905879 16217 raft_consensus.cc:1240] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 17. Current term is 18. Ops: []
I20260709 13:21:39.910380 22079 consensus_queue.cc:1059] T 84f2da98864b4425bf7c0055951fdd4b P 90f3da57796b488c9087a2f523adf045 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:21:39.911517 22079 consensus_queue.cc:1243] T 84f2da98864b4425bf7c0055951fdd4b P 90f3da57796b488c9087a2f523adf045 [LEADER]: Peer 647a343a542d4a139fe106c19095129b log is divergent from this leader: its last log entry 13.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 13:21:39.912075 22082 raft_consensus.cc:3055] T 84f2da98864b4425bf7c0055951fdd4b P 90f3da57796b488c9087a2f523adf045 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:21:39.911777 22043 consensus_queue.cc:237] T 7c2095a5f4ad49408dbc7f69c606120d P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:39.912483 22082 raft_consensus.cc:740] T 84f2da98864b4425bf7c0055951fdd4b P 90f3da57796b488c9087a2f523adf045 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:21:39.913158 22119 raft_consensus.cc:3055] T af57996a8be84d398f0f023e8d08ef77 P 647a343a542d4a139fe106c19095129b [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:21:39.913384 22082 consensus_queue.cc:260] T 84f2da98864b4425bf7c0055951fdd4b P 90f3da57796b488c9087a2f523adf045 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:39.914717 22082 raft_consensus.cc:3060] T 84f2da98864b4425bf7c0055951fdd4b P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:21:39.913769 22119 raft_consensus.cc:740] T af57996a8be84d398f0f023e8d08ef77 P 647a343a542d4a139fe106c19095129b [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:21:39.854869 22202 consensus_queue.cc:1048] T 8638ee363273426db1393dd81e5c7b72 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.027s
I20260709 13:21:39.915897 22119 consensus_queue.cc:260] T af57996a8be84d398f0f023e8d08ef77 P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.12, Last appended by leader: 12, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:39.916944 22119 raft_consensus.cc:3060] T af57996a8be84d398f0f023e8d08ef77 P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:21:39.917590 22164 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 13 current term: 13 is higher.
W20260709 13:21:39.917851 22085 consensus_meta.cc:287] T ecc1e363230043fe9fef390f0b979f20 P 90f3da57796b488c9087a2f523adf045: Time spent flushing consensus metadata: real 0.712s	user 0.004s	sys 0.010s
I20260709 13:21:39.918354 22085 raft_consensus.cc:515] T ecc1e363230043fe9fef390f0b979f20 P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:39.925419 22085 leader_election.cc:290] T ecc1e363230043fe9fef390f0b979f20 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 18 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
W20260709 13:21:39.931304 16114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:39.933386 16192 leader_election.cc:304] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:21:39.934216 16143 raft_consensus.cc:1275] T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:21:39.935036 16151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66f75124215f4ca6bef831e174a792d8" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:39.935923 16151 raft_consensus.cc:2468] T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 15.
I20260709 13:21:39.987018 16147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee9ed67006144797b1de6ea8c2013a68" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:39.988673 16147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "829b4ad8c5a54eababeea1181b960d4b" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:21:39.871909 22215 raft_consensus.cc:2749] T 02838c86531d4f98bf58d1c22b8baf77 P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Leader election lost for term 15. Reason: Vote denied by peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741) with higher term. Message: Invalid argument: T 02838c86531d4f98bf58d1c22b8baf77 P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 for earlier term 15. Current term is 17.
I20260709 13:21:39.989444 16147 raft_consensus.cc:3060] T 829b4ad8c5a54eababeea1181b960d4b P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:21:39.860917 22218 raft_consensus.cc:515] T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:39.861052 22210 raft_consensus.cc:515] T c4f65cdf00b748bb963ee6e81ef43b79 P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:21:40.042027 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 400198us
I20260709 13:21:39.991802 22152 raft_consensus.cc:2804] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:21:40.032152 16817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:21:40.059743 22152 raft_consensus.cc:493] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:40.060227 22152 raft_consensus.cc:3060] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:21:39.898330 22201 raft_consensus.cc:515] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:40.051940 16226 raft_consensus.cc:2410] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 for term 17 because replica has last-logged OpId of term: 15 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 11.
I20260709 13:21:40.061035 22218 leader_election.cc:290] T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 18 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:39.990523 22242 consensus_queue.cc:1059] T 8cc959fb7f704fba9009c0595467d074 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: INVALID_TERM, Last received: 13.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.127s
I20260709 13:21:40.061098 16265 leader_election.cc:304] T d2ab0ac8347641e6955e9bce7d90c1a2 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860
I20260709 13:21:40.029716 16142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434bff3082d141838068f323bdb0d1ed" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:40.055589 16115 leader_election.cc:304] T 434bff3082d141838068f323bdb0d1ed P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860
I20260709 13:21:40.063841 22210 leader_election.cc:290] T c4f65cdf00b748bb963ee6e81ef43b79 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:21:40.034770 22147 raft_consensus.cc:2749] T 829b4ad8c5a54eababeea1181b960d4b P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:21:40.070753 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:21:40.061990 22201 leader_election.cc:290] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 15 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
W20260709 13:21:40.074981 16117 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:40.079496 16147 raft_consensus.cc:2468] T 829b4ad8c5a54eababeea1181b960d4b P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 16.
I20260709 13:21:40.080152 16790 leader_election.cc:304] T 66f75124215f4ca6bef831e174a792d8 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:21:40.071599 16226 raft_consensus.cc:3060] T ecc1e363230043fe9fef390f0b979f20 P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:21:40.072336 16266 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:21:40.030292 22025 consensus_meta.cc:287] T d16dfad0c3b540e5b4b86492ddf87208 P 647a343a542d4a139fe106c19095129b: Time spent flushing consensus metadata: real 0.556s	user 0.005s	sys 0.008s
I20260709 13:21:40.086205 16790 leader_election.cc:304] T ee9ed67006144797b1de6ea8c2013a68 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 647a343a542d4a139fe106c19095129b
I20260709 13:21:40.091477 22031 raft_consensus.cc:2804] T 66f75124215f4ca6bef831e174a792d8 P cb82cc7bd6db46d980c75237a1402860 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:21:40.091635 22085 raft_consensus.cc:2749] T d2ab0ac8347641e6955e9bce7d90c1a2 P 90f3da57796b488c9087a2f523adf045 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:21:40.092098 22031 raft_consensus.cc:493] T 66f75124215f4ca6bef831e174a792d8 P cb82cc7bd6db46d980c75237a1402860 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:40.092671 22031 raft_consensus.cc:3060] T 66f75124215f4ca6bef831e174a792d8 P cb82cc7bd6db46d980c75237a1402860 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:21:40.094508 16265 leader_election.cc:304] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:21:40.095484 22085 raft_consensus.cc:2749] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:21:40.097678 22202 raft_consensus.cc:2804] T ee9ed67006144797b1de6ea8c2013a68 P cb82cc7bd6db46d980c75237a1402860 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:21:40.101718 22202 raft_consensus.cc:493] T ee9ed67006144797b1de6ea8c2013a68 P cb82cc7bd6db46d980c75237a1402860 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:40.102149 22202 raft_consensus.cc:3060] T ee9ed67006144797b1de6ea8c2013a68 P cb82cc7bd6db46d980c75237a1402860 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:21:40.095511 16142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a96373bb0e14dce8a266b2fc6c58507" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:40.103471 16142 raft_consensus.cc:2468] T 5a96373bb0e14dce8a266b2fc6c58507 P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 13.
I20260709 13:21:40.096179 16143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d4ab870a05347868956aa8b841b2981" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:40.105453 16143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b7d1ab519044eda2d5e04a07242f25" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:21:40.106061 16143 raft_consensus.cc:3060] T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:21:40.074445 22242 consensus_queue.cc:1243] T 8cc959fb7f704fba9009c0595467d074 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Peer 647a343a542d4a139fe106c19095129b log is divergent from this leader: its last log entry 15.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 13:21:40.118665 16226 raft_consensus.cc:2468] T ecc1e363230043fe9fef390f0b979f20 P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 18.
I20260709 13:21:40.109858 21579 leader_election.cc:290] T c9944d5e4b8c4109af9955c4f08c9095 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:40.076203 21704 raft_consensus.cc:493] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:21:40.121104 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 56651.9us
I20260709 13:21:40.129999 16300 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40474 (request call id 1045) took 1032 ms. Trace:
I20260709 13:21:40.067266 22059 consensus_queue.cc:1048] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 11, Time since last communication: 0.038s
I20260709 13:21:40.134565 22059 consensus_queue.cc:1243] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Peer cb82cc7bd6db46d980c75237a1402860 log is divergent from this leader: its last log entry 16.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 11
W20260709 13:21:40.136386 16115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 80936.3us
I20260709 13:21:40.136660 22064 raft_consensus.cc:515] T 9e65676b21b748728ad39668cfe722ec P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:40.126586 22243 consensus_queue.cc:1048] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.042s
I20260709 13:21:40.128590 21704 raft_consensus.cc:515] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:40.138666 21245 raft_consensus.cc:2749] T 434bff3082d141838068f323bdb0d1ed P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:21:40.130410 16300 rpcz_store.cc:276] 0709 13:21:39.097893 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:39.098021 (+   128us) service_pool.cc:224] Handling call
0709 13:21:39.098305 (+   284us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:21:39.100749 (+  2444us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 13:21:39.100779 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:21:39.106524 (+  5745us) log.cc:844] Serialized 26 byte log entry
0709 13:21:39.926947 (+820423us) spinlock_profiling.cc:243] Waited 4.45 ms on lock 0x7b180044a5e0. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b693ade1 00007f69b69475b1 00007f69ada54e07 00007f69b68afab9 00007f69ada7169b 00007f69ada6f93f 00007f69ada71c02 00007f69b06f920b 00007f69b06fed23 00007f69b06ff0d1 00007f69b0b1aa0f
0709 13:21:39.993706 (+ 66759us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:21:39.993769 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:39.993866 (+    97us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:21:40.129702 (+135836us) raft_consensus.cc:1685] finished
0709 13:21:40.129723 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:21:40.129937 (+   214us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2065,"raft.queue_time_us":27297,"raft.run_cpu_time_us":139,"raft.run_wall_time_us":3641,"spinlock_wait_cycles":10243840,"thread_start_us":66506,"threads_started":1,"wal-append.queue_time_us":1003656}
I20260709 13:21:40.140205 16143 raft_consensus.cc:2468] T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 15.
W20260709 13:21:40.140599 16192 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 55504.4us
I20260709 13:21:40.074749 22134 consensus_queue.cc:1048] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.048s
I20260709 13:21:40.147742 16142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87ac0d1f7e8a419f9bcbc04aee0e067e" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:40.148819 16142 raft_consensus.cc:2393] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 17: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
I20260709 13:21:40.146936 16825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2372981006d4b218215c28964dea623" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:21:40.150538 16825 raft_consensus.cc:2393] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 18: Already voted for candidate cb82cc7bd6db46d980c75237a1402860 in this term.
I20260709 13:21:40.151770 16143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:21:40.152232 16143 raft_consensus.cc:3060] T ecc1e363230043fe9fef390f0b979f20 P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:21:40.154481 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 60062.3us
I20260709 13:21:40.150765 21245 consensus_queue.cc:1059] T 68a6838d82ef4cac83507f906139d90f P 647a343a542d4a139fe106c19095129b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: INVALID_TERM, Last received: 15.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:21:40.155375 16265 leader_election.cc:304] T 1df93779209748b9860f6d893dd3de53 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:21:40.147907 16821 pending_rounds.cc:85] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860: Aborting all ops after (but not including) 11
I20260709 13:21:40.156307 16821 pending_rounds.cc:107] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860: Aborting uncommitted NO_OP operation due to leader change: 16.12
I20260709 13:21:40.156662 16821 raft_consensus.cc:2889] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [term 21 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:21:40.158782 16143 raft_consensus.cc:2468] T ecc1e363230043fe9fef390f0b979f20 P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 18.
I20260709 13:21:40.089967 22025 raft_consensus.cc:515] T d16dfad0c3b540e5b4b86492ddf87208 P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:40.104624 22147 consensus_queue.cc:1048] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.057s
I20260709 13:21:40.164790 16265 leader_election.cc:400] T 02838c86531d4f98bf58d1c22b8baf77 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 15 election: Vote denied by peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943) with higher term. Message: Invalid argument: T 02838c86531d4f98bf58d1c22b8baf77 P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 for earlier term 15. Current term is 17.
I20260709 13:21:40.169339 22064 leader_election.cc:290] T 9e65676b21b748728ad39668cfe722ec P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 16 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:40.169572 22218 raft_consensus.cc:493] T 1df93779209748b9860f6d893dd3de53 P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:40.170197 22218 raft_consensus.cc:515] T 1df93779209748b9860f6d893dd3de53 P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:40.173228 16819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:40.173558 22246 raft_consensus.cc:493] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 647a343a542d4a139fe106c19095129b [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:40.174134 22246 raft_consensus.cc:515] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 647a343a542d4a139fe106c19095129b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:21:40.175818 22251 log.cc:927] Time spent T 7c2095a5f4ad49408dbc7f69c606120d P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 0.102s	user 0.000s	sys 0.001s
I20260709 13:21:40.176790 16115 leader_election.cc:304] T b87147731a68400e8cce9b879daa7c5d P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860
I20260709 13:21:40.163415 22082 raft_consensus.cc:2749] T 1df93779209748b9860f6d893dd3de53 P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:21:40.167431 22231 consensus_queue.cc:1059] T 08c74490c40243a88e62d1760b5cd814 P 647a343a542d4a139fe106c19095129b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.025s
I20260709 13:21:40.142072 22219 consensus_queue.cc:1048] T 8cc959fb7f704fba9009c0595467d074 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.125s
I20260709 13:21:40.182225 22231 raft_consensus.cc:2749] T b87147731a68400e8cce9b879daa7c5d P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:21:40.184705 22202 raft_consensus.cc:515] T ee9ed67006144797b1de6ea8c2013a68 P cb82cc7bd6db46d980c75237a1402860 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:40.185195 21623 raft_consensus.cc:3055] T 3a7ca61592e848ea9b0fb81a2ce33cdc P cb82cc7bd6db46d980c75237a1402860 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:21:40.185779 21623 raft_consensus.cc:740] T 3a7ca61592e848ea9b0fb81a2ce33cdc P cb82cc7bd6db46d980c75237a1402860 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:21:40.163388 22088 raft_consensus.cc:493] T 8d4ab870a05347868956aa8b841b2981 P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:21:40.187791 22088 raft_consensus.cc:515] T 8d4ab870a05347868956aa8b841b2981 P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:40.189224 16148 raft_consensus.cc:2368] T d16dfad0c3b540e5b4b86492ddf87208 P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac for earlier term 16. Current term is 17.
I20260709 13:21:40.182337 22219 raft_consensus.cc:3055] T 68a6838d82ef4cac83507f906139d90f P 647a343a542d4a139fe106c19095129b [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:21:40.191876 16817 raft_consensus.cc:1240] T ee9ed67006144797b1de6ea8c2013a68 P cb82cc7bd6db46d980c75237a1402860 [term 17 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 16. Current term is 17. Ops: []
I20260709 13:21:40.192075 16142 raft_consensus.cc:1240] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 16. Current term is 17. Ops: []
I20260709 13:21:40.170576 21704 leader_election.cc:290] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:40.178846 16115 leader_election.cc:304] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:21:40.190171 22025 leader_election.cc:290] T d16dfad0c3b540e5b4b86492ddf87208 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 17 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:40.193246 16143 raft_consensus.cc:1240] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 17. Current term is 18. Ops: []
I20260709 13:21:40.190634 22219 raft_consensus.cc:740] T 68a6838d82ef4cac83507f906139d90f P 647a343a542d4a139fe106c19095129b [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:21:40.207105 16223 raft_consensus.cc:1240] T 5a96373bb0e14dce8a266b2fc6c58507 P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 12. Current term is 13. Ops: []
I20260709 13:21:40.208318 16143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee9ed67006144797b1de6ea8c2013a68" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
W20260709 13:21:40.215495 16115 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:40.217497 16293 raft_consensus.cc:1275] T 84f2da98864b4425bf7c0055951fdd4b P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:21:40.217408 16296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:40.218175 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d16dfad0c3b540e5b4b86492ddf87208" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:21:40.218977 16297 raft_consensus.cc:3060] T d16dfad0c3b540e5b4b86492ddf87208 P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:21:40.218885 16296 raft_consensus.cc:2468] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 18.
I20260709 13:21:40.225908 22119 raft_consensus.cc:493] T 746aabf856ab401c9ef5e859371e7b67 P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:21:40.226876 22119 raft_consensus.cc:515] T 746aabf856ab401c9ef5e859371e7b67 P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:40.230099 22119 leader_election.cc:290] T 746aabf856ab401c9ef5e859371e7b67 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:40.231726 16223 raft_consensus.cc:1275] T ee9ed67006144797b1de6ea8c2013a68 P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:21:40.232930 16297 raft_consensus.cc:2468] T d16dfad0c3b540e5b4b86492ddf87208 P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 17.
I20260709 13:21:40.234622 16825 raft_consensus.cc:1240] T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860 [term 18 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 17. Current term is 18. Ops: []
I20260709 13:21:40.139889 22258 consensus_queue.cc:1048] T af57996a8be84d398f0f023e8d08ef77 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.077s
W20260709 13:21:40.241349 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 74794.9us
I20260709 13:21:40.242235 16265 leader_election.cc:304] T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:21:40.243119 16223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2372981006d4b218215c28964dea623" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:21:40.207585 22225 consensus_queue.cc:1059] T 18ad7e3af5504571a69d10c00750a01c P 647a343a542d4a139fe106c19095129b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: INVALID_TERM, Last received: 13.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.048s
I20260709 13:21:40.205641 21704 raft_consensus.cc:493] T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:40.237370 22097 raft_consensus.cc:2804] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:21:40.243482 22031 raft_consensus.cc:515] T 66f75124215f4ca6bef831e174a792d8 P cb82cc7bd6db46d980c75237a1402860 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:40.244117 16143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62fe3346cd3c42c4a55e14af9073bca4" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:21:40.245162 16265 leader_election.cc:304] T 8a878999c0024d2abe517484180290fc P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:21:40.261901 22102 raft_consensus.cc:2749] T 8a878999c0024d2abe517484180290fc P 90f3da57796b488c9087a2f523adf045 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:21:40.245450 21704 raft_consensus.cc:515] T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:40.265524 21704 leader_election.cc:290] T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:40.245360 22242 raft_consensus.cc:2749] T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:21:40.246172 22097 raft_consensus.cc:493] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:40.268421 22097 raft_consensus.cc:3060] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:21:40.268622 22210 raft_consensus.cc:3055] T 8cc959fb7f704fba9009c0595467d074 P 90f3da57796b488c9087a2f523adf045 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:21:40.269194 22210 raft_consensus.cc:740] T 8cc959fb7f704fba9009c0595467d074 P 90f3da57796b488c9087a2f523adf045 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:21:40.270190 22210 consensus_queue.cc:260] T 8cc959fb7f704fba9009c0595467d074 P 90f3da57796b488c9087a2f523adf045 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:40.271791 16265 leader_election.cc:304] T 5a96373bb0e14dce8a266b2fc6c58507 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:21:40.273638 16144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6f07842d3aa47858df5665874c86f3a" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
W20260709 13:21:40.274516 22189 log.cc:927] Time spent T 84f2da98864b4425bf7c0055951fdd4b P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 1.255s	user 0.001s	sys 0.009s
I20260709 13:21:40.274838 16144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5718c623ebee4d2bb770f0f9ddb2f164" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:21:40.275964 16222 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57604 (request call id 3078) took 1615 ms. Trace:
I20260709 13:21:40.246594 16825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746aabf856ab401c9ef5e859371e7b67" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:40.276895 22027 raft_consensus.cc:3055] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:21:40.277426 22027 raft_consensus.cc:740] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:21:40.276343 16222 rpcz_store.cc:276] 0709 13:21:38.661029 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:38.661313 (+   284us) service_pool.cc:224] Handling call
0709 13:21:38.661637 (+   324us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:21:38.661989 (+   352us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:21:38.662017 (+    28us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:21:38.678810 (+ 16793us) log.cc:844] Serialized 52 byte log entry
0709 13:21:38.865215 (+186405us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:21:38.865255 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:38.865324 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:21:40.273947 (+1408623us) spinlock_profiling.cc:243] Waited 13.3 ms on lock 0x7b4400022b4c. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69ad8fadb1 00007f69ad998c85 00007f69ad9996a2 00007f69b070b1db 00007f69b0706aaf 00007f69b06fa86e 00007f69b06f9bb6 00007f69b06f97d8 00007f69b0703221 00007f69b07031d9 00007f69b0703169
0709 13:21:40.275654 (+  1707us) raft_consensus.cc:1685] finished
0709 13:21:40.275679 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:21:40.275905 (+   226us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":15759,"raft.run_cpu_time_us":119,"raft.run_wall_time_us":5074,"spinlock_wait_cycles":30701312,"thread_start_us":149347,"threads_started":1,"wal-append.queue_time_us":264339}
I20260709 13:21:40.278234 22027 consensus_queue.cc:260] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:40.275893 16144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5d59407eb545b895d8d922e0dd69fd" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:40.279357 16219 raft_consensus.cc:1217] T 84f2da98864b4425bf7c0055951fdd4b P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Deduplicated request from leader. Original: 11.6->[12.7-12.7]   Dedup: 12.7->[]
I20260709 13:21:40.278640 22246 leader_election.cc:290] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:40.279937 16219 raft_consensus.cc:1240] T 84f2da98864b4425bf7c0055951fdd4b P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 12. Current term is 13. Ops: [12.7-12.7]
I20260709 13:21:40.246778 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d16dfad0c3b540e5b4b86492ddf87208" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:21:40.280598 16226 raft_consensus.cc:3060] T d16dfad0c3b540e5b4b86492ddf87208 P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:21:40.278770 16222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:40.281272 16222 raft_consensus.cc:2468] T 19baa7bc367844c288bca8f81439c413 P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 17.
I20260709 13:21:40.282183 16222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746aabf856ab401c9ef5e859371e7b67" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:40.255874 16221 raft_consensus.cc:1240] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 16. Current term is 17. Ops: []
I20260709 13:21:40.258318 16220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87ac0d1f7e8a419f9bcbc04aee0e067e" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:40.284495 16220 raft_consensus.cc:2393] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 17: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
W20260709 13:21:40.284659 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 46682.6us
I20260709 13:21:40.285607 16143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4dc6b0541ca431ebc757beac00521f1" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:40.291492 16143 raft_consensus.cc:2410] T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac for term 16 because replica has last-logged OpId of term: 15 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 7.
I20260709 13:21:40.289994 16265 leader_election.cc:304] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:21:40.299209 16825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e9b2bf7011f4ae688a79d7bb5a447b4" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:40.300005 16825 raft_consensus.cc:2468] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 14.
I20260709 13:21:40.300293 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74ca8140f35c4ae29e8a664f6bdfc829" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 17 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:21:40.254132 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a96373bb0e14dce8a266b2fc6c58507" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:40.302484 16225 raft_consensus.cc:2468] T 5a96373bb0e14dce8a266b2fc6c58507 P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 13.
I20260709 13:21:40.304510 16143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1df93779209748b9860f6d893dd3de53" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:21:40.207448 22202 leader_election.cc:290] T ee9ed67006144797b1de6ea8c2013a68 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 17 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:21:40.305197 16222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1df93779209748b9860f6d893dd3de53" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:40.309684 16265 leader_election.cc:304] T 18ad7e3af5504571a69d10c00750a01c P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:21:40.305380 16143 raft_consensus.cc:2410] T 1df93779209748b9860f6d893dd3de53 P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac for term 15 because replica has last-logged OpId of term: 14 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 8.
I20260709 13:21:40.313331 16222 raft_consensus.cc:2468] T 1df93779209748b9860f6d893dd3de53 P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 15.
I20260709 13:21:40.314725 16222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4f65cdf00b748bb963ee6e81ef43b79" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:40.316428 16222 raft_consensus.cc:2468] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 17.
I20260709 13:21:40.255241 21623 consensus_queue.cc:260] T 3a7ca61592e848ea9b0fb81a2ce33cdc P cb82cc7bd6db46d980c75237a1402860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:40.318549 21623 raft_consensus.cc:3060] T 3a7ca61592e848ea9b0fb81a2ce33cdc P cb82cc7bd6db46d980c75237a1402860 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:21:40.249852 22031 leader_election.cc:290] T 66f75124215f4ca6bef831e174a792d8 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 16 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:21:40.254307 22218 leader_election.cc:290] T 1df93779209748b9860f6d893dd3de53 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:40.326174 16116 leader_election.cc:304] T 1df93779209748b9860f6d893dd3de53 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
I20260709 13:21:40.308781 22082 raft_consensus.cc:2749] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:21:40.328579 16143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806d7690c2b644959152074bb2cab0a7" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:21:40.332824 16143 raft_consensus.cc:2376] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 17. Re-sending same reply.
W20260709 13:21:40.333179 16266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 49570us
I20260709 13:21:40.333770 16143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0eab25ce4c64e018a041beeda2867db" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:40.311075 22106 raft_consensus.cc:2804] T 5a96373bb0e14dce8a266b2fc6c58507 P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:21:40.331982 16265 leader_election.cc:304] T 8158bbd17b994d2eb96f560bd0f1b88d P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:21:40.341478 16226 raft_consensus.cc:2468] T d16dfad0c3b540e5b4b86492ddf87208 P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 17.
I20260709 13:21:40.344453 16143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4dc6b0541ca431ebc757beac00521f1" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:21:40.345968 16143 raft_consensus.cc:2410] T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac for term 16 because replica has last-logged OpId of term: 15 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 7.
I20260709 13:21:40.348421 16265 leader_election.cc:304] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: cb82cc7bd6db46d980c75237a1402860
I20260709 13:21:40.302425 22195 raft_consensus.cc:3055] T 19baa7bc367844c288bca8f81439c413 P cb82cc7bd6db46d980c75237a1402860 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:21:40.354164 16116 leader_election.cc:304] T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
I20260709 13:21:40.355100 16265 leader_election.cc:304] T 8c11074579b046d99e585c2818e18a2c P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860
I20260709 13:21:40.355350 16143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66f75124215f4ca6bef831e174a792d8" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:21:40.357897 16143 raft_consensus.cc:3060] T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:21:40.356398 16790 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:40.207315 22207 consensus_queue.cc:1059] T b6f07842d3aa47858df5665874c86f3a P 647a343a542d4a139fe106c19095129b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: INVALID_TERM, Last received: 15.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.135s
I20260709 13:21:40.308483 22210 raft_consensus.cc:3060] T 8cc959fb7f704fba9009c0595467d074 P 90f3da57796b488c9087a2f523adf045 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:21:40.360083 16296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434bff3082d141838068f323bdb0d1ed" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:40.334317 22027 raft_consensus.cc:3060] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:21:40.343600 22187 consensus_queue.cc:1048] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.017s
I20260709 13:21:40.363988 16117 leader_election.cc:304] T 746aabf856ab401c9ef5e859371e7b67 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860
I20260709 13:21:40.342224 22242 raft_consensus.cc:2749] T 18ad7e3af5504571a69d10c00750a01c P 90f3da57796b488c9087a2f523adf045 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:21:40.368355 16296 raft_consensus.cc:1275] T 31969392ccdb43b491911a319e8bc7e1 P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:21:40.361114 22241 raft_consensus.cc:2804] T 8158bbd17b994d2eb96f560bd0f1b88d P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:21:40.304697 22043 consensus_queue.cc:1048] T ee9ed67006144797b1de6ea8c2013a68 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.058s
I20260709 13:21:40.359651 21704 raft_consensus.cc:2804] T 1df93779209748b9860f6d893dd3de53 P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:21:40.373323 16296 raft_consensus.cc:1240] T ecc1e363230043fe9fef390f0b979f20 P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 17. Current term is 18. Ops: []
I20260709 13:21:40.358129 22106 raft_consensus.cc:697] T 5a96373bb0e14dce8a266b2fc6c58507 P 90f3da57796b488c9087a2f523adf045 [term 13 LEADER]: Becoming Leader. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:21:40.357336 22195 raft_consensus.cc:740] T 19baa7bc367844c288bca8f81439c413 P cb82cc7bd6db46d980c75237a1402860 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:21:40.343695 22102 consensus_queue.cc:1059] T 66ae176961b74813a58bc15c9c4c6f2a P 90f3da57796b488c9087a2f523adf045 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.027s
W20260709 13:21:40.421622 22229 log.cc:927] Time spent T 3884f864a96d428785895e5463e5728f P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.569s	user 0.001s	sys 0.001s
I20260709 13:21:40.421917 16145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af57996a8be84d398f0f023e8d08ef77" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:40.422786 16145 raft_consensus.cc:2410] T af57996a8be84d398f0f023e8d08ef77 P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac for term 18 because replica has last-logged OpId of term: 17 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 11.
I20260709 13:21:40.423393 22158 raft_consensus.cc:3055] T 18ad7e3af5504571a69d10c00750a01c P 647a343a542d4a139fe106c19095129b [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:21:40.423679 16145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6f07842d3aa47858df5665874c86f3a" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:40.424450 22152 raft_consensus.cc:515] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:40.424867 16145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2372981006d4b218215c28964dea623" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:40.426712 22152 leader_election.cc:290] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 19 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
W20260709 13:21:40.427358 16264 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:40.379153 16117 leader_election.cc:304] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:21:40.384423 16291 raft_consensus.cc:1240] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 17. Current term is 18. Ops: []
I20260709 13:21:40.428730 16223 raft_consensus.cc:2368] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b for earlier term 18. Current term is 19.
I20260709 13:21:40.390789 22246 raft_consensus.cc:2804] T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:21:40.342000 22219 consensus_queue.cc:260] T 68a6838d82ef4cac83507f906139d90f P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:40.366688 22215 raft_consensus.cc:2749] T 8c11074579b046d99e585c2818e18a2c P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:21:40.433584 16145 raft_consensus.cc:2468] T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 18.
I20260709 13:21:40.397017 22177 raft_consensus.cc:3055] T 08c74490c40243a88e62d1760b5cd814 P 647a343a542d4a139fe106c19095129b [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:21:40.400552 22088 leader_election.cc:290] T 8d4ab870a05347868956aa8b841b2981 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:40.393167 22196 raft_consensus.cc:2749] T 746aabf856ab401c9ef5e859371e7b67 P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:21:40.375581 21704 raft_consensus.cc:493] T 1df93779209748b9860f6d893dd3de53 P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:40.450316 21704 raft_consensus.cc:3060] T 1df93779209748b9860f6d893dd3de53 P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:21:40.384631 22241 raft_consensus.cc:697] T 8158bbd17b994d2eb96f560bd0f1b88d P 90f3da57796b488c9087a2f523adf045 [term 15 LEADER]: Becoming Leader. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:21:40.415994 16144 raft_consensus.cc:2468] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 18.
I20260709 13:21:40.416599 16292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1df93779209748b9860f6d893dd3de53" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:40.455847 16292 raft_consensus.cc:2468] T 1df93779209748b9860f6d893dd3de53 P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 15.
I20260709 13:21:40.423816 22158 raft_consensus.cc:740] T 18ad7e3af5504571a69d10c00750a01c P 647a343a542d4a139fe106c19095129b [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:21:40.378744 16116 leader_election.cc:304] T d16dfad0c3b540e5b4b86492ddf87208 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
I20260709 13:21:40.430431 16291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4f65cdf00b748bb963ee6e81ef43b79" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:40.432040 16142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
W20260709 13:21:40.432926 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 77905.7us
I20260709 13:21:40.433055 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e9b2bf7011f4ae688a79d7bb5a447b4" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:40.361239 22079 raft_consensus.cc:2804] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:21:40.457419 16142 raft_consensus.cc:2393] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 18: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
I20260709 13:21:40.434366 22246 raft_consensus.cc:493] T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:40.461050 16145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b7d1ab519044eda2d5e04a07242f25" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:21:40.461843 16145 raft_consensus.cc:2368] T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac for earlier term 14. Current term is 15.
W20260709 13:21:40.435899 16117 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 56831.9us
I20260709 13:21:40.434366 22102 consensus_queue.cc:1243] T 66ae176961b74813a58bc15c9c4c6f2a P 90f3da57796b488c9087a2f523adf045 [LEADER]: Peer cb82cc7bd6db46d980c75237a1402860 log is divergent from this leader: its last log entry 16.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 11
I20260709 13:21:40.441515 22168 leader_election.cc:290] T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:40.442543 16149 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40844 (request call id 2718) took 1070 ms. Trace:
I20260709 13:21:40.464725 16149 rpcz_store.cc:276] 0709 13:21:39.371950 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:39.494855 (+122905us) service_pool.cc:224] Handling call
0709 13:21:40.442325 (+947470us) spinlock_profiling.cc:243] Waited 381 ms on lock 0x7b5400427a54. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:21:40.442476 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":875471232}
I20260709 13:21:40.412285 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9944d5e4b8c4109af9955c4f08c9095" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:40.435297 22177 raft_consensus.cc:740] T 08c74490c40243a88e62d1760b5cd814 P 647a343a542d4a139fe106c19095129b [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:21:40.466073 16293 raft_consensus.cc:2468] T c9944d5e4b8c4109af9955c4f08c9095 P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 18.
I20260709 13:21:40.451963 16115 leader_election.cc:304] T 2794af5b99984efebdee564035841cb0 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860
I20260709 13:21:40.437084 16296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d4ab870a05347868956aa8b841b2981" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:40.457264 16291 raft_consensus.cc:2468] T c4f65cdf00b748bb963ee6e81ef43b79 P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 16.
I20260709 13:21:40.435225 16143 raft_consensus.cc:2468] T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 16.
I20260709 13:21:40.434016 22242 raft_consensus.cc:493] T 23f82224f4564fea93572e8206bad1fe P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:40.472146 22242 raft_consensus.cc:515] T 23f82224f4564fea93572e8206bad1fe P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:40.474370 22242 leader_election.cc:290] T 23f82224f4564fea93572e8206bad1fe P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:40.446508 16789 leader_election.cc:304] T 74ca8140f35c4ae29e8a664f6bdfc829 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:21:40.475443 22106 consensus_queue.cc:237] T 5a96373bb0e14dce8a266b2fc6c58507 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:40.478125 16789 leader_election.cc:304] T 434bff3082d141838068f323bdb0d1ed P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:21:40.477854 22158 consensus_queue.cc:260] T 18ad7e3af5504571a69d10c00750a01c P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } }
I20260709 13:21:40.479000 16818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2372981006d4b218215c28964dea623" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:40.479730 16818 raft_consensus.cc:2393] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 18: Already voted for candidate cb82cc7bd6db46d980c75237a1402860 in this term.
I20260709 13:21:40.480652 22241 consensus_queue.cc:237] T 8158bbd17b994d2eb96f560bd0f1b88d P 90f3da57796b488c9087a2f523adf045 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:40.481459 22205 raft_consensus.cc:2749] T 74ca8140f35c4ae29e8a664f6bdfc829 P cb82cc7bd6db46d980c75237a1402860 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:21:40.482187 22205 raft_consensus.cc:2749] T 434bff3082d141838068f323bdb0d1ed P cb82cc7bd6db46d980c75237a1402860 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:21:40.482910 22205 consensus_queue.cc:1048] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
W20260709 13:21:40.469715 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 91080.6us
I20260709 13:21:40.484373 22177 consensus_queue.cc:260] T 08c74490c40243a88e62d1760b5cd814 P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:40.484241 22195 consensus_queue.cc:260] T 19baa7bc367844c288bca8f81439c413 P cb82cc7bd6db46d980c75237a1402860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:40.485788 22195 raft_consensus.cc:3060] T 19baa7bc367844c288bca8f81439c413 P cb82cc7bd6db46d980c75237a1402860 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:21:40.465144 22246 raft_consensus.cc:3060] T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:21:40.486907 16821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f82224f4564fea93572e8206bad1fe" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:40.487582 16821 raft_consensus.cc:2468] T 23f82224f4564fea93572e8206bad1fe P cb82cc7bd6db46d980c75237a1402860 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 13.
I20260709 13:21:40.488432 16151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a878999c0024d2abe517484180290fc" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:40.469741 22088 raft_consensus.cc:2804] T d16dfad0c3b540e5b4b86492ddf87208 P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:21:40.489300 22088 raft_consensus.cc:697] T d16dfad0c3b540e5b4b86492ddf87208 P 647a343a542d4a139fe106c19095129b [term 17 LEADER]: Becoming Leader. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:21:40.489370 21623 consensus_queue.cc:1059] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true }, Status: INVALID_TERM, Last received: 17.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:21:40.490442 22235 raft_consensus.cc:3055] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:21:40.490239 22088 consensus_queue.cc:237] T d16dfad0c3b540e5b4b86492ddf87208 P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:40.490847 22235 raft_consensus.cc:740] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:21:40.470256 22037 raft_consensus.cc:2749] T 2794af5b99984efebdee564035841cb0 P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:21:40.470621 16115 leader_election.cc:304] T 3884f864a96d428785895e5463e5728f P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860
I20260709 13:21:40.491758 22235 consensus_queue.cc:260] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:40.492651 16266 leader_election.cc:304] T 23f82224f4564fea93572e8206bad1fe P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:21:40.492941 22235 raft_consensus.cc:3060] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:21:40.494064 22027 raft_consensus.cc:2749] T 3884f864a96d428785895e5463e5728f P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:21:40.471115 16142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f5f89186204242bf377f7c42b2e2da" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 18 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:40.471565 22079 raft_consensus.cc:697] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 90f3da57796b488c9087a2f523adf045 [term 18 LEADER]: Becoming Leader. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:21:40.497233 22027 consensus_queue.cc:1048] T 84f2da98864b4425bf7c0055951fdd4b P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:21:40.497354 22205 raft_consensus.cc:493] T 8c11074579b046d99e585c2818e18a2c P cb82cc7bd6db46d980c75237a1402860 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:40.503739 22242 raft_consensus.cc:2804] T 23f82224f4564fea93572e8206bad1fe P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:21:40.503618 22205 raft_consensus.cc:515] T 8c11074579b046d99e585c2818e18a2c P cb82cc7bd6db46d980c75237a1402860 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:40.504298 22242 raft_consensus.cc:493] T 23f82224f4564fea93572e8206bad1fe P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:40.504838 22242 raft_consensus.cc:3060] T 23f82224f4564fea93572e8206bad1fe P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:21:40.467922 16143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 18 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:40.450774 22234 raft_consensus.cc:2804] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 647a343a542d4a139fe106c19095129b [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:21:40.506179 22234 raft_consensus.cc:493] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 647a343a542d4a139fe106c19095129b [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:40.506559 22234 raft_consensus.cc:3060] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 647a343a542d4a139fe106c19095129b [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:21:40.508947 16148 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40844 (request call id 2720) took 1136 ms. Trace:
I20260709 13:21:40.509308 16148 rpcz_store.cc:276] 0709 13:21:39.372466 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:39.595302 (+222836us) service_pool.cc:224] Handling call
0709 13:21:40.508761 (+913459us) spinlock_profiling.cc:243] Waited 549 ms on lock 0x7b540013f1d4. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:21:40.508890 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1263091456}
I20260709 13:21:40.510355 16823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e65676b21b748728ad39668cfe722ec" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:21:40.501998 16151 raft_consensus.cc:1240] T ecc1e363230043fe9fef390f0b979f20 P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 17. Current term is 18. Ops: []
W20260709 13:21:40.502372 16263 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:21:40.512209 22194 log.cc:927] Time spent T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 1.493s	user 0.007s	sys 0.009s
I20260709 13:21:40.513010 16143 raft_consensus.cc:2393] T ecc1e363230043fe9fef390f0b979f20 P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 18: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
W20260709 13:21:40.517141 16047 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:49250 (request call id 322) took 36159 ms (client timeout 15000 ms). Trace:
W20260709 13:21:40.525441 16191 leader_election.cc:336] T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 15 election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Timed out: RequestConsensusVote RPC to 127.14.177.129:36741 timed out after 2.235s (SENT)
I20260709 13:21:40.485684 22177 raft_consensus.cc:3060] T 08c74490c40243a88e62d1760b5cd814 P 647a343a542d4a139fe106c19095129b [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:21:40.527588 16790 leader_election.cc:304] T 829b4ad8c5a54eababeea1181b960d4b P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:21:40.528378 22235 raft_consensus.cc:2804] T 829b4ad8c5a54eababeea1181b960d4b P cb82cc7bd6db46d980c75237a1402860 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:21:40.528507 16790 leader_election.cc:304] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:21:40.497051 22079 consensus_queue.cc:237] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 90f3da57796b488c9087a2f523adf045 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.14, Last appended by leader: 14, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:40.517506 22234 raft_consensus.cc:515] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:40.532167 22225 raft_consensus.cc:493] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:40.532272 16223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d4ab870a05347868956aa8b841b2981" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:40.532718 22234 leader_election.cc:290] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 15 election: Requested vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:40.518678 22205 leader_election.cc:290] T 8c11074579b046d99e585c2818e18a2c P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:40.532805 22225 raft_consensus.cc:515] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:40.533456 16223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b7d1ab519044eda2d5e04a07242f25" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:21:40.534039 16223 raft_consensus.cc:3060] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:21:40.502848 22158 raft_consensus.cc:3060] T 18ad7e3af5504571a69d10c00750a01c P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:21:40.535683 16116 leader_election.cc:400] T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 18 election: Vote denied by peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943) with higher term. Message: Invalid argument: T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b for earlier term 18. Current term is 19.
I20260709 13:21:40.536016 16116 leader_election.cc:403] T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 18 election: Cancelling election due to peer responding with higher term
I20260709 13:21:40.502300 16115 leader_election.cc:304] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: cb82cc7bd6db46d980c75237a1402860
I20260709 13:21:39.618922 15046 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:21:40.538812 16117 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:40.540716 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "829b4ad8c5a54eababeea1181b960d4b" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:21:40.541286 16226 raft_consensus.cc:3060] T 829b4ad8c5a54eababeea1181b960d4b P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:21:40.547545 16147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4f65cdf00b748bb963ee6e81ef43b79" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:40.550179 16143 raft_consensus.cc:1275] T 3884f864a96d428785895e5463e5728f P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 10 index: 11. Preceding OpId from leader: term: 12 index: 12. (index mismatch)
I20260709 13:21:40.552910 16143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e65676b21b748728ad39668cfe722ec" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:21:40.555274 16143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f82224f4564fea93572e8206bad1fe" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:40.557802 16265 leader_election.cc:304] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:21:40.585060 16265 leader_election.cc:304] T 68a6838d82ef4cac83507f906139d90f P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:21:40.593282 16265 leader_election.cc:304] T 5a959dbecc2648eb85d1cc550f5a3245 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac
W20260709 13:21:40.594149 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 65714.2us
I20260709 13:21:40.601833 22073 raft_consensus.cc:2804] T 5a959dbecc2648eb85d1cc550f5a3245 P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:21:40.602451 22073 raft_consensus.cc:697] T 5a959dbecc2648eb85d1cc550f5a3245 P 90f3da57796b488c9087a2f523adf045 [term 15 LEADER]: Becoming Leader. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:21:40.603365 22073 consensus_queue.cc:237] T 5a959dbecc2648eb85d1cc550f5a3245 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:40.610173 16823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e9b2bf7011f4ae688a79d7bb5a447b4" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:21:40.611052 16823 raft_consensus.cc:3060] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:21:40.545698 22235 raft_consensus.cc:697] T 829b4ad8c5a54eababeea1181b960d4b P cb82cc7bd6db46d980c75237a1402860 [term 16 LEADER]: Becoming Leader. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:21:40.591924 22234 raft_consensus.cc:2804] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:21:40.584287 22102 raft_consensus.cc:493] T 31969392ccdb43b491911a319e8bc7e1 P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:40.619656 16790 leader_election.cc:304] T 66f75124215f4ca6bef831e174a792d8 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:21:40.626715 16265 leader_election.cc:304] T 5a96373bb0e14dce8a266b2fc6c58507 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
W20260709 13:21:40.629357 16114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:40.599274 22215 raft_consensus.cc:2749] T 68a6838d82ef4cac83507f906139d90f P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:21:40.632583 16226 raft_consensus.cc:2468] T 829b4ad8c5a54eababeea1181b960d4b P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 16.
I20260709 13:21:40.634025 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9944d5e4b8c4109af9955c4f08c9095" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:40.634783 16226 raft_consensus.cc:2468] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 18.
I20260709 13:21:40.607376 22029 raft_consensus.cc:2804] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:21:40.584162 22167 consensus_queue.cc:1059] T ee9ed67006144797b1de6ea8c2013a68 P 647a343a542d4a139fe106c19095129b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.038s
I20260709 13:21:40.639886 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66f75124215f4ca6bef831e174a792d8" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:21:40.644775 16226 raft_consensus.cc:3060] T 66f75124215f4ca6bef831e174a792d8 P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:21:40.520798 16047 rpcz_store.cc:269] 0709 13:21:04.357562 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:04.357842 (+   280us) service_pool.cc:224] Handling call
0709 13:21:40.517032 (+36159190us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:21:35.209792 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:21:35.210578 (+   786us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:21:35.210646 (+    68us) write_op.cc:435] Acquired schema lock
0709 13:21:35.210693 (+    47us) tablet.cc:662] Decoding operations
0709 13:21:35.212238 (+  1545us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:21:35.212336 (+    98us) write_op.cc:641] Partition lock acquired for write op
0709 13:21:35.212368 (+    32us) tablet.cc:685] Acquiring locks for 14 operations
0709 13:21:35.213640 (+  1272us) tablet.cc:701] Row locks acquired
0709 13:21:35.213686 (+    46us) write_op.cc:260] PREPARE: finished
0709 13:21:35.213890 (+   204us) write_op.cc:270] Start()
0709 13:21:35.214042 (+   152us) write_op.cc:276] Timestamp: P: 1783603295213817 usec, L: 0
0709 13:21:35.214060 (+    18us) op_driver.cc:348] REPLICATION: starting
0709 13:21:35.214901 (+   841us) log.cc:844] Serialized 5609 byte log entry
0709 13:21:35.595928 (+381027us) spinlock_profiling.cc:243] Waited 4.16 ms on lock 0x7b4400022b4c. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69ad8fadb1 00007f69ad998c85 00007f69ad9996a2 00007f69b070b1db 00007f69b0706aaf 00007f69ada550ac 00007f69b68afab9 00007f69ada7169b 00007f69ada6f93f 00007f69ada71c02 00007f69b06f920b
0709 13:21:35.946028 (+350100us) op_driver.cc:464] REPLICATION: finished
0709 13:21:36.027720 (+ 81692us) write_op.cc:301] APPLY: starting
0709 13:21:36.027815 (+    95us) tablet.cc:1367] starting BulkCheckPresence
0709 13:21:36.109872 (+ 82057us) tablet.cc:1370] finished BulkCheckPresence
0709 13:21:36.109917 (+    45us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:21:36.417981 (+308064us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:21:36.418343 (+   362us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=28,key_file_lookups=28,delta_file_lookups=0,mrs_lookups=0
0709 13:21:36.418399 (+    56us) write_op.cc:312] APPLY: finished
0709 13:21:36.431555 (+ 13156us) log.cc:844] Serialized 129 byte log entry
0709 13:21:36.432474 (+   919us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:21:36.434526 (+  2052us) write_op.cc:454] Released schema lock
0709 13:21:36.479330 (+ 44804us) write_op.cc:341] FINISH: Updating metrics
0709 13:21:36.567184 (+ 87854us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:21:36.575041 (+  7857us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":10840,"reported_tablets":48,"child_traces":[["op",{"apply.queue_time_us":81432,"apply.run_cpu_time_us":22143,"apply.run_wall_time_us":492683,"cfile_cache_hit":58,"cfile_cache_hit_bytes":62591,"num_ops":14,"prepare.queue_time_us":564463,"prepare.run_cpu_time_us":29499,"prepare.run_wall_time_us":515646,"raft.queue_time_us":170815,"raft.run_cpu_time_us":7172,"raft.run_wall_time_us":33120,"replication_time_us":731803,"spinlock_wait_cycles":9565440,"thread_start_us":1147076,"threads_started":4,"wal-append.queue_time_us":547631,"wal-append.run_cpu_time_us":30883,"wal-append.run_wall_time_us":1660774}]]}
I20260709 13:21:40.624917 22234 raft_consensus.cc:493] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:40.657481 16148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2372981006d4b218215c28964dea623" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:21:40.658252 16143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87ac0d1f7e8a419f9bcbc04aee0e067e" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:21:40.658865 16143 raft_consensus.cc:2376] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 17. Re-sending same reply.
I20260709 13:21:40.659710 16143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fffab26427e24d398e2695aad40ca134" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:40.660995 16143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee9ed67006144797b1de6ea8c2013a68" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 17 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
W20260709 13:21:40.665022 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 45451.2us
I20260709 13:21:40.668583 16191 leader_election.cc:304] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
I20260709 13:21:40.693186 16299 raft_consensus.cc:1240] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 90f3da57796b488c9087a2f523adf045 [term 18 LEADER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 17. Current term is 18. Ops: []
I20260709 13:21:40.734755 16143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c74490c40243a88e62d1760b5cd814" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:40.636173 22201 raft_consensus.cc:493] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:40.636240 22057 raft_consensus.cc:493] T e5ad4eb281564936a59f41e387a47156 P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:21:40.741370 16295 raft_consensus.cc:1275] T 8cc959fb7f704fba9009c0595467d074 P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 15 index: 11. (term mismatch)
I20260709 13:21:40.741869 16295 pending_rounds.cc:85] T 8cc959fb7f704fba9009c0595467d074 P 90f3da57796b488c9087a2f523adf045: Aborting all ops after (but not including) 10
I20260709 13:21:40.742208 16295 pending_rounds.cc:107] T 8cc959fb7f704fba9009c0595467d074 P 90f3da57796b488c9087a2f523adf045: Aborting uncommitted NO_OP operation due to leader change: 14.11
I20260709 13:21:40.742602 16295 raft_consensus.cc:2889] T 8cc959fb7f704fba9009c0595467d074 P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:21:40.734819 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 66342.9us
I20260709 13:21:40.739326 22188 raft_consensus.cc:3060] T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:21:40.739805 16226 raft_consensus.cc:2468] T 66f75124215f4ca6bef831e174a792d8 P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 16.
I20260709 13:21:40.740233 21623 raft_consensus.cc:493] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
W20260709 13:21:40.740530 16789 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:21:40.748664 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 82916us
I20260709 13:21:40.694361 22234 raft_consensus.cc:3060] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:21:40.745460 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 118801us
I20260709 13:21:40.692284 22167 consensus_queue.cc:1243] T ee9ed67006144797b1de6ea8c2013a68 P 647a343a542d4a139fe106c19095129b [LEADER]: Peer cb82cc7bd6db46d980c75237a1402860 log is divergent from this leader: its last log entry 15.13 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 12
I20260709 13:21:40.753026 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c11074579b046d99e585c2818e18a2c" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:40.753949 16226 raft_consensus.cc:2468] T 8c11074579b046d99e585c2818e18a2c P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 15.
I20260709 13:21:40.754024 16823 raft_consensus.cc:2468] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 15.
I20260709 13:21:40.754242 16265 leader_election.cc:304] T 8d4ab870a05347868956aa8b841b2981 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:21:40.734037 22041 raft_consensus.cc:2804] T 66f75124215f4ca6bef831e174a792d8 P cb82cc7bd6db46d980c75237a1402860 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:21:40.752731 16191 leader_election.cc:304] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
I20260709 13:21:40.635176 22102 raft_consensus.cc:515] T 31969392ccdb43b491911a319e8bc7e1 P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:40.777467 16191 leader_election.cc:304] T c0eab25ce4c64e018a041beeda2867db P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:21:40.692528 21930 raft_consensus.cc:493] T d2ab0ac8347641e6955e9bce7d90c1a2 P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:21:40.757288 22115 raft_consensus.cc:2749] T 8d4ab870a05347868956aa8b841b2981 P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:21:40.757733 16115 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:40.784068 16265 leader_election.cc:304] T ecc1e363230043fe9fef390f0b979f20 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: cb82cc7bd6db46d980c75237a1402860
I20260709 13:21:40.760362 16224 raft_consensus.cc:1240] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [term 21 LEADER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 20. Current term is 21. Ops: []
I20260709 13:21:40.761283 22211 raft_consensus.cc:3055] T 66ae176961b74813a58bc15c9c4c6f2a P 90f3da57796b488c9087a2f523adf045 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:21:40.763257 22267 raft_consensus.cc:2749] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
W20260709 13:21:40.778650 16790 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:40.780248 16191 leader_election.cc:304] T 19baa7bc367844c288bca8f81439c413 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
I20260709 13:21:40.754868 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ad5dbfa99ee4b8d9e28be92b3f36db5" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:40.741026 22153 raft_consensus.cc:2749] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:21:40.744338 22029 raft_consensus.cc:697] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 [term 15 LEADER]: Becoming Leader. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
W20260709 13:21:40.819694 16115 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:40.791993 16226 raft_consensus.cc:2468] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 14.
I20260709 13:21:40.829116 16265 leader_election.cc:304] T 9e65676b21b748728ad39668cfe722ec P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
I20260709 13:21:40.835630 16116 leader_election.cc:304] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
I20260709 13:21:40.646811 22097 raft_consensus.cc:515] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:40.840384 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee9ed67006144797b1de6ea8c2013a68" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:21:40.841248 16224 raft_consensus.cc:3060] T ee9ed67006144797b1de6ea8c2013a68 P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:21:40.845736 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 53961.2us
I20260709 13:21:40.846964 16790 leader_election.cc:304] T c9944d5e4b8c4109af9955c4f08c9095 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:21:40.789095 22147 raft_consensus.cc:2804] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:21:40.864534 22147 raft_consensus.cc:697] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac [term 17 LEADER]: Becoming Leader. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:21:40.786424 22134 raft_consensus.cc:2749] T c0eab25ce4c64e018a041beeda2867db P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:21:40.746577 22201 raft_consensus.cc:515] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:40.783794 22057 raft_consensus.cc:515] T e5ad4eb281564936a59f41e387a47156 P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:40.834892 22041 raft_consensus.cc:697] T 66f75124215f4ca6bef831e174a792d8 P cb82cc7bd6db46d980c75237a1402860 [term 16 LEADER]: Becoming Leader. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:21:40.755770 21623 raft_consensus.cc:515] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:40.789486 21930 raft_consensus.cc:515] T d2ab0ac8347641e6955e9bce7d90c1a2 P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:40.854346 22153 raft_consensus.cc:3060] T 19baa7bc367844c288bca8f81439c413 P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:21:40.747418 22152 consensus_queue.cc:1059] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.089s
I20260709 13:21:40.787963 22211 raft_consensus.cc:740] T 66ae176961b74813a58bc15c9c4c6f2a P 90f3da57796b488c9087a2f523adf045 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
W20260709 13:21:40.869975 16791 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:40.836992 22241 raft_consensus.cc:2804] T ecc1e363230043fe9fef390f0b979f20 P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:21:40.845168 22206 raft_consensus.cc:2804] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 13:21:40.884155 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 103978us
I20260709 13:21:40.900750 16191 leader_election.cc:400] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 14 election: Vote denied by peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741) with higher term. Message: Invalid argument: T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac for earlier term 14. Current term is 15.
I20260709 13:21:40.901055 16191 leader_election.cc:403] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 14 election: Cancelling election due to peer responding with higher term
I20260709 13:21:40.887665 16224 raft_consensus.cc:2468] T ee9ed67006144797b1de6ea8c2013a68 P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 17.
I20260709 13:21:40.864686 22098 raft_consensus.cc:2804] T c9944d5e4b8c4109af9955c4f08c9095 P cb82cc7bd6db46d980c75237a1402860 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:21:40.852990 22225 leader_election.cc:290] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:40.854075 22115 raft_consensus.cc:2749] T 9e65676b21b748728ad39668cfe722ec P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:21:40.838523 22210 consensus_queue.cc:1048] T 3884f864a96d428785895e5463e5728f P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.040s
I20260709 13:21:40.778671 22235 consensus_queue.cc:237] T 829b4ad8c5a54eababeea1181b960d4b P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:40.836032 22215 consensus_queue.cc:1059] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.046s
I20260709 13:21:40.905448 22098 raft_consensus.cc:493] T c9944d5e4b8c4109af9955c4f08c9095 P cb82cc7bd6db46d980c75237a1402860 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:21:40.905752 22253 log.cc:927] Time spent T cdbd78acb4cf46298e0de8ba0d130e19 P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.065s	user 0.000s	sys 0.000s
I20260709 13:21:40.847090 22242 raft_consensus.cc:515] T 23f82224f4564fea93572e8206bad1fe P 90f3da57796b488c9087a2f523adf045 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:40.912559 16191 leader_election.cc:304] T 8a878999c0024d2abe517484180290fc P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:21:40.920528 21930 leader_election.cc:290] T d2ab0ac8347641e6955e9bce7d90c1a2 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
W20260709 13:21:40.947571 22277 log.cc:927] Time spent T 3a7ca61592e848ea9b0fb81a2ce33cdc P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.097s	user 0.001s	sys 0.000s
I20260709 13:21:40.947780 16219 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57600 (request call id 1807) took 1791 ms. Trace:
W20260709 13:21:40.921058 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 1.3802e+06us
I20260709 13:21:40.948136 16219 rpcz_store.cc:276] 0709 13:21:39.156186 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:39.331556 (+175370us) service_pool.cc:224] Handling call
0709 13:21:40.279227 (+947671us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:21:40.280336 (+  1109us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:40.947586 (+667250us) spinlock_profiling.cc:243] Waited 947 ms on lock 0x7b5400396550. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5e8ad 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:21:40.947725 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2178507520}
W20260709 13:21:40.950471 16190 connection.cc:256] RPC call timeout handler was delayed by -1.3864010800000415s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1623s for a response
W20260709 13:21:40.951663 22269 log.cc:927] Time spent T 2794af5b99984efebdee564035841cb0 P cb82cc7bd6db46d980c75237a1402860: Append to log took a long time: real 0.210s	user 0.000s	sys 0.000s
W20260709 13:21:40.952553 22271 log.cc:927] Time spent T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 0.211s	user 0.001s	sys 0.000s
I20260709 13:21:40.953426 22211 consensus_queue.cc:260] T 66ae176961b74813a58bc15c9c4c6f2a P 90f3da57796b488c9087a2f523adf045 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.13, Last appended by leader: 13, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:21:40.954586 22272 log.cc:927] Time spent T 5a96373bb0e14dce8a266b2fc6c58507 P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.170s	user 0.000s	sys 0.000s
I20260709 13:21:40.954813 22211 raft_consensus.cc:3060] T 66ae176961b74813a58bc15c9c4c6f2a P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:21:40.959352 16143 raft_consensus.cc:2468] T 08c74490c40243a88e62d1760b5cd814 P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 15.
I20260709 13:21:40.907418 16224 raft_consensus.cc:1275] T fffab26427e24d398e2695aad40ca134 P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 13:21:40.934604 16223 raft_consensus.cc:2468] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 15.
I20260709 13:21:40.901427 22152 consensus_queue.cc:1243] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Peer 647a343a542d4a139fe106c19095129b log is divergent from this leader: its last log entry 16.14 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 12
W20260709 13:21:40.950837 16190 connection.cc:256] RPC call timeout handler was delayed by -1.3818335240000579s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1791s for a response
I20260709 13:21:40.904291 22206 raft_consensus.cc:493] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:40.950861 22119 raft_consensus.cc:3055] T b6f07842d3aa47858df5665874c86f3a P 647a343a542d4a139fe106c19095129b [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:21:40.944250 22201 leader_election.cc:290] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:21:40.946081 22188 raft_consensus.cc:2749] T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Leader election lost for term 18. Reason: Vote denied by peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943) with higher term. Message: Invalid argument: T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b for earlier term 18. Current term is 19.
W20260709 13:21:40.936708 16117 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:40.940336 21623 leader_election.cc:290] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:21:40.924214 22187 raft_consensus.cc:493] T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:40.902011 22246 raft_consensus.cc:515] T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:40.932500 22106 raft_consensus.cc:493] T b87147731a68400e8cce9b879daa7c5d P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:21:40.931154 22241 raft_consensus.cc:697] T ecc1e363230043fe9fef390f0b979f20 P 90f3da57796b488c9087a2f523adf045 [term 18 LEADER]: Becoming Leader. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:21:40.935056 22098 raft_consensus.cc:3060] T c9944d5e4b8c4109af9955c4f08c9095 P cb82cc7bd6db46d980c75237a1402860 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:21:40.938988 22215 raft_consensus.cc:3055] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:21:40.983981 16117 leader_election.cc:304] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860; no voters: 
W20260709 13:21:41.014871 16117 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:41.003144 16224 raft_consensus.cc:1240] T ee9ed67006144797b1de6ea8c2013a68 P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 16. Current term is 17. Ops: [16.13-16.13]
I20260709 13:21:40.964720 22206 raft_consensus.cc:3060] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:21:41.016757 22273 log.cc:927] Time spent T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860: Append to log took a long time: real 0.150s	user 0.000s	sys 0.000s
I20260709 13:21:40.969573 22119 raft_consensus.cc:740] T b6f07842d3aa47858df5665874c86f3a P 647a343a542d4a139fe106c19095129b [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
W20260709 13:21:41.029448 16265 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.14.177.129:36741 timed out after 1.809s (SENT) [suppressed 42 similar messages]
I20260709 13:21:41.031544 16147 raft_consensus.cc:2368] T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 for earlier term 17. Current term is 18.
I20260709 13:21:41.010149 22258 raft_consensus.cc:2749] T 8a878999c0024d2abe517484180290fc P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:21:40.929025 22029 consensus_queue.cc:237] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.12, Last appended by leader: 12, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:40.961627 21704 raft_consensus.cc:515] T 1df93779209748b9860f6d893dd3de53 P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:21:41.047207 16190 connection.cc:256] RPC call timeout handler was delayed by -1.2174013559999821s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2209s for a response
W20260709 13:21:41.048724 16190 connection.cc:256] RPC call timeout handler was delayed by -1.1826907810000193s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1612s for a response
W20260709 13:21:41.049211 16190 connection.cc:256] RPC call timeout handler was delayed by -1.1741077730002871s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1962s for a response
W20260709 13:21:41.049605 16190 connection.cc:256] RPC call timeout handler was delayed by -1.110615252000116s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.223s for a response
I20260709 13:21:41.045023 16148 raft_consensus.cc:2468] T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 19.
I20260709 13:21:41.011049 21802 raft_consensus.cc:493] T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:41.004943 22167 consensus_queue.cc:1048] T fffab26427e24d398e2695aad40ca134 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.049s
I20260709 13:21:41.014031 22215 raft_consensus.cc:740] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:21:41.027534 22225 raft_consensus.cc:2804] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Leader election won for term 15
W20260709 13:21:41.050927 16190 connection.cc:256] RPC call timeout handler was delayed by -1.0961979510000219s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19190000000000002s for a response
I20260709 13:21:41.005335 22187 raft_consensus.cc:515] T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:41.012017 22106 raft_consensus.cc:515] T b87147731a68400e8cce9b879daa7c5d P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:41.112419 16190 leader_election.cc:304] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860
W20260709 13:21:41.074075 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 161615us
I20260709 13:21:41.083877 22285 raft_consensus.cc:2749] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: Leader election lost for term 14. Reason: Vote denied by peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741) with higher term. Message: Invalid argument: T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac for earlier term 14. Current term is 15.
I20260709 13:21:40.943033 22147 consensus_queue.cc:237] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.10, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } }
I20260709 13:21:40.990728 22041 consensus_queue.cc:237] T 66f75124215f4ca6bef831e174a792d8 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:41.122592 22206 raft_consensus.cc:515] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:41.072378 21802 raft_consensus.cc:515] T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:41.133241 16190 leader_election.cc:304] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860
I20260709 13:21:41.125240 21802 leader_election.cc:290] T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
W20260709 13:21:41.161360 22288 log.cc:927] Time spent T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 0.085s	user 0.000s	sys 0.000s
W20260709 13:21:41.162720 22283 log.cc:927] Time spent T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 0.088s	user 0.000s	sys 0.000s
I20260709 13:21:41.167652 22281 raft_consensus.cc:2749] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:21:41.075295 22102 leader_election.cc:290] T 31969392ccdb43b491911a319e8bc7e1 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:41.111187 22225 raft_consensus.cc:697] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 647a343a542d4a139fe106c19095129b [term 15 LEADER]: Becoming Leader. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:21:41.114003 16191 leader_election.cc:304] T c2f5f89186204242bf377f7c42b2e2da P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:21:41.117574 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31969392ccdb43b491911a319e8bc7e1" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:41.118268 22057 leader_election.cc:290] T e5ad4eb281564936a59f41e387a47156 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
W20260709 13:21:41.122723 22158 consensus_meta.cc:287] T 18ad7e3af5504571a69d10c00750a01c P 647a343a542d4a139fe106c19095129b: Time spent flushing consensus metadata: real 0.537s	user 0.011s	sys 0.006s
W20260709 13:21:41.181131 16146 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40830 (request call id 1820) took 2012 ms (client timeout 1809 ms). Trace:
I20260709 13:21:41.182734 16142 raft_consensus.cc:1275] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 18 index: 15. (index mismatch)
W20260709 13:21:41.187321 16146 rpcz_store.cc:269] 0709 13:21:39.168362 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:39.168516 (+   154us) service_pool.cc:224] Handling call
0709 13:21:41.180908 (+2012392us) spinlock_profiling.cc:243] Waited 53.2 ms on lock 0x7b54000749d4. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:21:41.181056 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":122290560}
I20260709 13:21:41.183192 16144 raft_consensus.cc:1275] T 5a96373bb0e14dce8a266b2fc6c58507 P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 12 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
W20260709 13:21:41.188824 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 55654.1us
I20260709 13:21:41.190902 16190 leader_election.cc:304] T ecc1e363230043fe9fef390f0b979f20 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860
I20260709 13:21:41.187577 22291 raft_consensus.cc:2749] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:21:41.074712 22234 raft_consensus.cc:515] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:21:41.196758 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 82826us
W20260709 13:21:41.203336 16114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:41.203136 16191 leader_election.cc:304] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 90f3da57796b488c9087a2f523adf045
I20260709 13:21:41.198231 16190 leader_election.cc:304] T e4dc6b0541ca431ebc757beac00521f1 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 647a343a542d4a139fe106c19095129b
I20260709 13:21:41.217664 22187 leader_election.cc:290] T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:41.218536 21704 leader_election.cc:290] T 1df93779209748b9860f6d893dd3de53 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 16 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:41.200094 22057 raft_consensus.cc:2749] T ecc1e363230043fe9fef390f0b979f20 P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:21:41.211556 22242 leader_election.cc:290] T 23f82224f4564fea93572e8206bad1fe P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 14 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:41.221108 22143 raft_consensus.cc:3060] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:21:41.221601 22281 raft_consensus.cc:493] T 68a6838d82ef4cac83507f906139d90f P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:21:41.225183 22246 leader_election.cc:290] T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 18 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:41.225659 16225 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57604 (request call id 3101) took 1010 ms. Trace:
I20260709 13:21:41.225821 16821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2372981006d4b218215c28964dea623" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:21:41.258589 16821 raft_consensus.cc:3060] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:21:41.229182 22276 log.cc:927] Time spent T d16dfad0c3b540e5b4b86492ddf87208 P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 0.461s	user 0.001s	sys 0.000s
I20260709 13:21:41.230051 16300 raft_consensus.cc:1240] T 9e65676b21b748728ad39668cfe722ec P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 15. Current term is 16. Ops: []
I20260709 13:21:41.231719 22106 leader_election.cc:290] T b87147731a68400e8cce9b879daa7c5d P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:41.232342 22225 consensus_queue.cc:237] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:41.225267 16818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ab0ac8347641e6955e9bce7d90c1a2" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:41.238111 16190 leader_election.cc:304] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860
I20260709 13:21:41.250093 22258 raft_consensus.cc:2804] T e4dc6b0541ca431ebc757beac00521f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:21:41.242075 16298 raft_consensus.cc:1240] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 14. Current term is 15. Ops: []
W20260709 13:21:41.283996 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 45981.2us
W20260709 13:21:41.246383 16117 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:41.246515 16296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ad5dbfa99ee4b8d9e28be92b3f36db5" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:41.285280 16190 leader_election.cc:304] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860
I20260709 13:21:41.285722 16296 raft_consensus.cc:2468] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 15.
I20260709 13:21:41.231731 22098 raft_consensus.cc:515] T c9944d5e4b8c4109af9955c4f08c9095 P cb82cc7bd6db46d980c75237a1402860 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:41.136487 22241 consensus_queue.cc:237] T ecc1e363230043fe9fef390f0b979f20 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 18, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:41.255033 16292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c11074579b046d99e585c2818e18a2c" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:41.289852 16292 raft_consensus.cc:2468] T 8c11074579b046d99e585c2818e18a2c P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 16.
I20260709 13:21:41.196753 22296 raft_consensus.cc:2749] T c2f5f89186204242bf377f7c42b2e2da P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:21:41.258256 16225 rpcz_store.cc:276] 0709 13:21:40.215505 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:40.956655 (+741150us) service_pool.cc:224] Handling call
0709 13:21:40.956968 (+   313us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:21:40.957304 (+   336us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:21:40.957329 (+    25us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:21:40.957811 (+   482us) log.cc:844] Serialized 26 byte log entry
0709 13:21:41.072075 (+114264us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:21:41.072119 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:41.072190 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:21:41.225450 (+153260us) raft_consensus.cc:1685] finished
0709 13:21:41.225470 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:21:41.225604 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":6523,"raft.run_cpu_time_us":164,"raft.run_wall_time_us":12337,"spinlock_wait_cycles":81792,"thread_start_us":113863,"threads_started":1,"wal-append.queue_time_us":216327}
I20260709 13:21:41.291920 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4f65cdf00b748bb963ee6e81ef43b79" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:41.292920 16225 raft_consensus.cc:2393] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 17: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
I20260709 13:21:41.244801 22285 raft_consensus.cc:2804] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:21:41.241266 16191 leader_election.cc:304] T fffab26427e24d398e2695aad40ca134 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:21:41.295653 16190 leader_election.cc:304] T 18ad7e3af5504571a69d10c00750a01c P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
W20260709 13:21:41.298600 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 59106.7us
I20260709 13:21:41.299403 16191 leader_election.cc:304] T ee9ed67006144797b1de6ea8c2013a68 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
I20260709 13:21:41.302990 22291 raft_consensus.cc:2749] T fffab26427e24d398e2695aad40ca134 P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:21:41.308252 16789 leader_election.cc:304] T 8c11074579b046d99e585c2818e18a2c P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:21:41.264881 22281 raft_consensus.cc:515] T 68a6838d82ef4cac83507f906139d90f P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:41.316560 16294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e9b2bf7011f4ae688a79d7bb5a447b4" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:21:41.293880 22134 raft_consensus.cc:2749] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:21:41.299877 22057 raft_consensus.cc:2749] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:21:41.321152 16294 raft_consensus.cc:3060] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 90f3da57796b488c9087a2f523adf045 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:21:41.318635 22031 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.557s	user 0.000s	sys 0.020s
I20260709 13:21:41.315271 22285 raft_consensus.cc:697] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 1ca2725d290e47d68b1a4b609c9816ac [term 17 LEADER]: Becoming Leader. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
W20260709 13:21:41.348347 16117 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:41.314239 22291 raft_consensus.cc:2749] T ee9ed67006144797b1de6ea8c2013a68 P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:21:41.334704 22031 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.577s	user 0.001s	sys 0.020s
I20260709 13:21:41.319722 22029 raft_consensus.cc:2804] T 8c11074579b046d99e585c2818e18a2c P cb82cc7bd6db46d980c75237a1402860 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:21:41.332633 22296 raft_consensus.cc:3060] T 18ad7e3af5504571a69d10c00750a01c P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:21:41.376124 22098 leader_election.cc:290] T c9944d5e4b8c4109af9955c4f08c9095 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 19 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:41.319620 22286 consensus_queue.cc:1059] T 9e65676b21b748728ad39668cfe722ec P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: INVALID_TERM, Last received: 15.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.059s
I20260709 13:21:41.364867 22029 raft_consensus.cc:493] T 8c11074579b046d99e585c2818e18a2c P cb82cc7bd6db46d980c75237a1402860 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:21:41.379088 22208 log.cc:927] Time spent T af57996a8be84d398f0f023e8d08ef77 P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 2.138s	user 0.003s	sys 0.004s
I20260709 13:21:41.379527 22029 raft_consensus.cc:3060] T 8c11074579b046d99e585c2818e18a2c P cb82cc7bd6db46d980c75237a1402860 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:21:41.390383 16297 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40474 (request call id 1055) took 1035 ms. Trace:
I20260709 13:21:41.391023 16297 rpcz_store.cc:276] 0709 13:21:40.355369 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:40.355492 (+   123us) service_pool.cc:224] Handling call
0709 13:21:40.355822 (+   330us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:21:40.359990 (+  4168us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:21:40.360019 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:21:40.360490 (+   471us) log.cc:844] Serialized 26 byte log entry
0709 13:21:41.184333 (+823843us) spinlock_profiling.cc:243] Waited 4.36 ms on lock 0x7b1800355120. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b693ade1 00007f69b69475b1 00007f69ada54e07 00007f69b68afab9 00007f69ada7169b 00007f69ada6f93f 00007f69ada71c02 00007f69b06f920b 00007f69b06fed23 00007f69b06ff0d1 00007f69b0b1aa0f
0709 13:21:41.265283 (+ 80950us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:21:41.265334 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:41.265424 (+    90us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:21:41.383985 (+118561us) raft_consensus.cc:1685] finished
0709 13:21:41.384017 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:21:41.390273 (+  6256us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3764,"raft.queue_time_us":4972,"raft.run_cpu_time_us":103,"raft.run_wall_time_us":2447,"spinlock_wait_cycles":10037120,"thread_start_us":80747,"threads_started":1,"wal-append.queue_time_us":948339}
I20260709 13:21:41.393505 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9944d5e4b8c4109af9955c4f08c9095" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:21:41.394127 16297 raft_consensus.cc:3060] T c9944d5e4b8c4109af9955c4f08c9095 P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:21:41.393967 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 94615.8us
I20260709 13:21:41.401844 16294 raft_consensus.cc:2468] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 15.
I20260709 13:21:41.403625 16821 raft_consensus.cc:2468] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 19.
I20260709 13:21:41.355000 22285 consensus_queue.cc:237] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:41.414659 22230 raft_consensus.cc:3055] T 9e65676b21b748728ad39668cfe722ec P cb82cc7bd6db46d980c75237a1402860 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:21:41.416507 22230 raft_consensus.cc:740] T 9e65676b21b748728ad39668cfe722ec P cb82cc7bd6db46d980c75237a1402860 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:21:41.389611 22258 raft_consensus.cc:697] T e4dc6b0541ca431ebc757beac00521f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 16 LEADER]: Becoming Leader. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:21:41.440675 16297 raft_consensus.cc:2468] T c9944d5e4b8c4109af9955c4f08c9095 P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 19.
I20260709 13:21:41.417500 22230 consensus_queue.cc:260] T 9e65676b21b748728ad39668cfe722ec P cb82cc7bd6db46d980c75237a1402860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:41.444110 22029 raft_consensus.cc:515] T 8c11074579b046d99e585c2818e18a2c P cb82cc7bd6db46d980c75237a1402860 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:41.448653 22296 raft_consensus.cc:2749] T 18ad7e3af5504571a69d10c00750a01c P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:21:41.451980 22281 leader_election.cc:290] T 68a6838d82ef4cac83507f906139d90f P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:41.453162 22219 raft_consensus.cc:3060] T 68a6838d82ef4cac83507f906139d90f P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:21:41.453547 16218 raft_consensus.cc:1275] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 15 index: 14. Preceding OpId from leader: term: 16 index: 14. (term mismatch)
W20260709 13:21:41.453833 22300 log.cc:927] Time spent T 66f75124215f4ca6bef831e174a792d8 P cb82cc7bd6db46d980c75237a1402860: Append to log took a long time: real 0.065s	user 0.000s	sys 0.000s
W20260709 13:21:41.456538 22311 log.cc:927] Time spent T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 0.067s	user 0.000s	sys 0.000s
W20260709 13:21:41.458254 22153 consensus_meta.cc:287] T 19baa7bc367844c288bca8f81439c413 P 1ca2725d290e47d68b1a4b609c9816ac: Time spent flushing consensus metadata: real 0.557s	user 0.011s	sys 0.000s
I20260709 13:21:41.486132 16218 pending_rounds.cc:85] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac: Aborting all ops after (but not including) 13
I20260709 13:21:41.487190 16299 raft_consensus.cc:1275] T 02838c86531d4f98bf58d1c22b8baf77 P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 13:21:41.495791 16218 pending_rounds.cc:107] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac: Aborting uncommitted NO_OP operation due to leader change: 15.14
W20260709 13:21:41.487752 22297 log.cc:927] Time spent T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860: Append to log took a long time: real 0.084s	user 0.000s	sys 0.000s
I20260709 13:21:41.496232 16218 raft_consensus.cc:2889] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:21:41.464032 22234 leader_election.cc:290] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 19 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
W20260709 13:21:41.489511 16265 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:41.491338 22230 raft_consensus.cc:3060] T 9e65676b21b748728ad39668cfe722ec P cb82cc7bd6db46d980c75237a1402860 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:21:41.492501 22097 leader_election.cc:290] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 20 election: Requested vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
W20260709 13:21:41.500420 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 204894us
I20260709 13:21:41.517781 16190 leader_election.cc:304] T b6f07842d3aa47858df5665874c86f3a P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:21:41.520805 16190 leader_election.cc:304] T 1df93779209748b9860f6d893dd3de53 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 647a343a542d4a139fe106c19095129b
I20260709 13:21:41.523363 16265 leader_election.cc:304] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:21:41.511689 22153 raft_consensus.cc:2749] T 19baa7bc367844c288bca8f81439c413 P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:21:41.521646 22059 raft_consensus.cc:2804] T 1df93779209748b9860f6d893dd3de53 P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:21:41.498705 22145 raft_consensus.cc:493] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:41.521910 22281 raft_consensus.cc:2749] T b6f07842d3aa47858df5665874c86f3a P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:21:41.528666 22106 raft_consensus.cc:3060] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:21:41.528847 22104 raft_consensus.cc:493] T c0eab25ce4c64e018a041beeda2867db P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:21:41.542376 22104 raft_consensus.cc:515] T c0eab25ce4c64e018a041beeda2867db P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:41.535344 22258 consensus_queue.cc:237] T e4dc6b0541ca431ebc757beac00521f1 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.7, Last appended by leader: 7, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:41.548103 22104 leader_election.cc:290] T c0eab25ce4c64e018a041beeda2867db P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:21:41.536435 22145 raft_consensus.cc:515] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:41.549856 16048 catalog_manager.cc:5697] T 434bff3082d141838068f323bdb0d1ed P cb82cc7bd6db46d980c75237a1402860 reported cstate change: term changed from 9 to 15, leader changed from cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) to 90f3da57796b488c9087a2f523adf045 (127.14.177.131). New cstate: current_term: 15 leader_uuid: "90f3da57796b488c9087a2f523adf045" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:21:41.535749 22059 raft_consensus.cc:697] T 1df93779209748b9860f6d893dd3de53 P 1ca2725d290e47d68b1a4b609c9816ac [term 15 LEADER]: Becoming Leader. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:21:41.530840 22242 consensus_queue.cc:1059] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.024s
I20260709 13:21:41.552232 22059 consensus_queue.cc:237] T 1df93779209748b9860f6d893dd3de53 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:41.553023 22242 consensus_queue.cc:1243] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Peer 1ca2725d290e47d68b1a4b609c9816ac log is divergent from this leader: its last log entry 15.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260709 13:21:41.556533 16048 catalog_manager.cc:5697] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 reported cstate change: term changed from 11 to 13. New cstate: current_term: 13 leader_uuid: "cb82cc7bd6db46d980c75237a1402860" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:21:41.558615 16048 catalog_manager.cc:5697] T 8c11074579b046d99e585c2818e18a2c P cb82cc7bd6db46d980c75237a1402860 reported cstate change: term changed from 13 to 15, leader changed from 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130) to cb82cc7bd6db46d980c75237a1402860 (127.14.177.132). New cstate: current_term: 15 leader_uuid: "cb82cc7bd6db46d980c75237a1402860" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:21:41.560592 16048 catalog_manager.cc:5697] T 9e65676b21b748728ad39668cfe722ec P cb82cc7bd6db46d980c75237a1402860 reported cstate change: term changed from 13 to 15. New cstate: current_term: 15 leader_uuid: "cb82cc7bd6db46d980c75237a1402860" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:21:41.561339 22119 consensus_queue.cc:260] T b6f07842d3aa47858df5665874c86f3a P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:41.580463 16223 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33050 (request call id 1035) took 1053 ms. Trace:
I20260709 13:21:41.566978 16048 catalog_manager.cc:5697] T 3a7ca61592e848ea9b0fb81a2ce33cdc P cb82cc7bd6db46d980c75237a1402860 reported cstate change: term changed from 15 to 17, leader changed from 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130) to cb82cc7bd6db46d980c75237a1402860 (127.14.177.132). New cstate: current_term: 17 leader_uuid: "cb82cc7bd6db46d980c75237a1402860" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:21:41.522089 22296 consensus_queue.cc:1048] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 10, Time since last communication: 0.018s
I20260709 13:21:41.581136 16223 rpcz_store.cc:276] 0709 13:21:40.526742 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:41.116533 (+589791us) service_pool.cc:224] Handling call
0709 13:21:41.116852 (+   319us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:21:41.137710 (+ 20858us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:21:41.137758 (+    48us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:21:41.155488 (+ 17730us) log.cc:844] Serialized 52 byte log entry
0709 13:21:41.235660 (+ 80172us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:21:41.314417 (+ 78757us) log.cc:844] Serialized 14 byte log entry
0709 13:21:41.325951 (+ 11534us) log.cc:844] Serialized 14 byte log entry
0709 13:21:41.565769 (+239818us) spinlock_profiling.cc:243] Waited 18.7 ms on lock 0x7b1800406ae0. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b693ade1 00007f69b69475b1 00007f69ad9ecc0a 00007f69b712c219 00007f69b0715d70 00007f69b0709293 00007f69b06fecea 00007f69b06ff23a 00007f69b0b69a2a 00007f69b0b790bf 00007f69b0b79038
0709 13:21:41.566024 (+   255us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:41.566106 (+    82us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:21:41.580088 (+ 13982us) raft_consensus.cc:1685] finished
0709 13:21:41.580133 (+    45us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:21:41.580357 (+   224us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":212001,"prepare.queue_time_us":147632,"prepare.run_cpu_time_us":126,"prepare.run_wall_time_us":13829,"raft.queue_time_us":78668,"raft.run_cpu_time_us":121,"raft.run_wall_time_us":2977,"spinlock_wait_cycles":53563520,"thread_start_us":129830,"threads_started":2,"wal-append.queue_time_us":162781}
I20260709 13:21:41.582459 16048 catalog_manager.cc:5697] T 19baa7bc367844c288bca8f81439c413 P cb82cc7bd6db46d980c75237a1402860 reported cstate change: term changed from 8 to 17, leader changed from <none> to cb82cc7bd6db46d980c75237a1402860 (127.14.177.132). New cstate: current_term: 17 leader_uuid: "cb82cc7bd6db46d980c75237a1402860" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:21:41.589004 22119 raft_consensus.cc:3060] T b6f07842d3aa47858df5665874c86f3a P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:21:41.597256 16223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0eab25ce4c64e018a041beeda2867db" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:41.594477 16048 catalog_manager.cc:5697] T 806d7690c2b644959152074bb2cab0a7 P cb82cc7bd6db46d980c75237a1402860 reported cstate change: term changed from 12 to 15. New cstate: current_term: 15 leader_uuid: "1ca2725d290e47d68b1a4b609c9816ac" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } } }
I20260709 13:21:41.601428 16048 catalog_manager.cc:5697] T d2ab0ac8347641e6955e9bce7d90c1a2 P cb82cc7bd6db46d980c75237a1402860 reported cstate change: term changed from 10 to 14. New cstate: current_term: 14 leader_uuid: "cb82cc7bd6db46d980c75237a1402860" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:21:41.604715 22279 log.cc:927] Time spent T 8158bbd17b994d2eb96f560bd0f1b88d P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.638s	user 0.000s	sys 0.002s
I20260709 13:21:41.604139 16048 catalog_manager.cc:5697] T b87147731a68400e8cce9b879daa7c5d P cb82cc7bd6db46d980c75237a1402860 reported cstate change: term changed from 3 to 16, leader changed from 90f3da57796b488c9087a2f523adf045 (127.14.177.131) to cb82cc7bd6db46d980c75237a1402860 (127.14.177.132). New cstate: current_term: 16 leader_uuid: "cb82cc7bd6db46d980c75237a1402860" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:21:41.607026 16191 leader_election.cc:336] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Timed out: RequestConsensusVote RPC to 127.14.177.129:36741 timed out after 1.705s (SENT)
I20260709 13:21:41.608294 22024 raft_consensus.cc:3060] T 8158bbd17b994d2eb96f560bd0f1b88d P 647a343a542d4a139fe106c19095129b [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:21:41.612843 22029 leader_election.cc:290] T 8c11074579b046d99e585c2818e18a2c P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 17 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:41.612993 16225 raft_consensus.cc:1275] T 02838c86531d4f98bf58d1c22b8baf77 P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
W20260709 13:21:41.604832 22314 log.cc:927] Time spent T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860: Append to log took a long time: real 0.117s	user 0.001s	sys 0.000s
I20260709 13:21:41.613773 16225 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57604 (request call id 3111) took 1078 ms. Trace:
I20260709 13:21:41.617327 16295 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48674 (request call id 3166) took 1179 ms. Trace:
I20260709 13:21:41.617141 16048 catalog_manager.cc:5697] T 66f75124215f4ca6bef831e174a792d8 P cb82cc7bd6db46d980c75237a1402860 reported cstate change: term changed from 11 to 14, leader changed from 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130) to cb82cc7bd6db46d980c75237a1402860 (127.14.177.132). New cstate: current_term: 14 leader_uuid: "cb82cc7bd6db46d980c75237a1402860" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:21:41.619114 16048 catalog_manager.cc:5697] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 reported cstate change: term changed from 14 to 17, leader changed from cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) to 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130). New cstate: current_term: 17 leader_uuid: "1ca2725d290e47d68b1a4b609c9816ac" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:21:41.620232 16220 raft_consensus.cc:1240] T 19baa7bc367844c288bca8f81439c413 P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 17. Current term is 18. Ops: []
I20260709 13:21:41.620926 16220 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33050 (request call id 1041) took 1006 ms. Trace:
I20260709 13:21:41.621248 16220 rpcz_store.cc:276] 0709 13:21:40.614469 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:41.614873 (+1000404us) service_pool.cc:224] Handling call
0709 13:21:41.620047 (+  5174us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:21:41.620685 (+   638us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:41.620861 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:21:41.622606 16220 raft_consensus.cc:1240] T ecc1e363230043fe9fef390f0b979f20 P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 17. Current term is 18. Ops: []
I20260709 13:21:41.617712 16295 rpcz_store.cc:276] 0709 13:21:40.437867 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:40.438028 (+   161us) service_pool.cc:224] Handling call
0709 13:21:40.438332 (+   304us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:21:40.743430 (+305098us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:41.617082 (+873652us) spinlock_profiling.cc:243] Waited 303 ms on lock 0x7b5400178754. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:21:41.617266 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":695977344}
I20260709 13:21:41.623962 16220 raft_consensus.cc:1275] T 7c2095a5f4ad49408dbc7f69c606120d P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 13 index: 6. (index mismatch)
I20260709 13:21:41.614169 16225 rpcz_store.cc:276] 0709 13:21:40.535193 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:41.612332 (+1077139us) service_pool.cc:224] Handling call
0709 13:21:41.612644 (+   312us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:21:41.613498 (+   854us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:41.613715 (+   217us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:21:41.626722 16225 raft_consensus.cc:1240] T b6f07842d3aa47858df5665874c86f3a P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 15. Current term is 16. Ops: []
I20260709 13:21:41.627198 16220 raft_consensus.cc:1240] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 16. Current term is 17. Ops: []
I20260709 13:21:41.628844 22241 consensus_queue.cc:1059] T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:21:41.629662 22241 consensus_queue.cc:1243] T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Peer 1ca2725d290e47d68b1a4b609c9816ac log is divergent from this leader: its last log entry 15.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 11
I20260709 13:21:41.630944 16220 raft_consensus.cc:1275] T 5a96373bb0e14dce8a266b2fc6c58507 P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 12 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 13:21:41.630097 22104 raft_consensus.cc:3055] T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:21:41.632118 22104 raft_consensus.cc:740] T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:21:41.630321 16048 catalog_manager.cc:5697] T c9944d5e4b8c4109af9955c4f08c9095 P cb82cc7bd6db46d980c75237a1402860 reported cstate change: term changed from 13 to 17, leader changed from 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130) to cb82cc7bd6db46d980c75237a1402860 (127.14.177.132). New cstate: current_term: 17 leader_uuid: "cb82cc7bd6db46d980c75237a1402860" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:21:41.636829 16048 catalog_manager.cc:5697] T ee9ed67006144797b1de6ea8c2013a68 P cb82cc7bd6db46d980c75237a1402860 reported cstate change: term changed from 11 to 15. New cstate: current_term: 15 leader_uuid: "cb82cc7bd6db46d980c75237a1402860" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:21:41.644737 22145 leader_election.cc:290] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:41.649685 16146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87147731a68400e8cce9b879daa7c5d" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:41.669804 16146 raft_consensus.cc:2393] T b87147731a68400e8cce9b879daa7c5d P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 17: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
I20260709 13:21:41.651834 16142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f82224f4564fea93572e8206bad1fe" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: true dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:21:41.656903 16048 catalog_manager.cc:5697] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P cb82cc7bd6db46d980c75237a1402860 reported cstate change: term changed from 12 to 15, leader changed from cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) to 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130). New cstate: current_term: 15 leader_uuid: "1ca2725d290e47d68b1a4b609c9816ac" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:21:41.672307 16142 raft_consensus.cc:2393] T 23f82224f4564fea93572e8206bad1fe P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 13: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
I20260709 13:21:41.672369 16048 catalog_manager.cc:5697] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860 reported cstate change: term changed from 5 to 13. New cstate: current_term: 13 leader_uuid: "cb82cc7bd6db46d980c75237a1402860" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:21:41.674962 16048 catalog_manager.cc:5697] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860 reported cstate change: term changed from 9 to 17, leader changed from <none> to cb82cc7bd6db46d980c75237a1402860 (127.14.177.132). New cstate: current_term: 17 leader_uuid: "cb82cc7bd6db46d980c75237a1402860" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:21:41.678113 16146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8158bbd17b994d2eb96f560bd0f1b88d" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:21:41.633960 22241 consensus_queue.cc:1048] T 5a96373bb0e14dce8a266b2fc6c58507 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.021s
I20260709 13:21:41.667747 22225 raft_consensus.cc:493] T 68a6838d82ef4cac83507f906139d90f P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:21:41.678221 22315 log.cc:927] Time spent T ecc1e363230043fe9fef390f0b979f20 P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.061s	user 0.001s	sys 0.000s
I20260709 13:21:41.654358 22196 raft_consensus.cc:493] T 26354361e9b044cea1c2b86c4882fe92 P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:21:41.686167 16142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0eab25ce4c64e018a041beeda2867db" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:41.687842 16146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f82224f4564fea93572e8206bad1fe" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:21:41.680796 16048 catalog_manager.cc:5697] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860 reported cstate change: term changed from 9 to 13, leader changed from 647a343a542d4a139fe106c19095129b (127.14.177.129) to 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130). New cstate: current_term: 13 leader_uuid: "1ca2725d290e47d68b1a4b609c9816ac" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:21:41.689666 16146 raft_consensus.cc:3060] T 23f82224f4564fea93572e8206bad1fe P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:21:41.693392 16048 catalog_manager.cc:5697] T 57a744f5aaf944678a63469cbf514015 P cb82cc7bd6db46d980c75237a1402860 reported cstate change: term changed from 9 to 11. New cstate: current_term: 11 leader_uuid: "cb82cc7bd6db46d980c75237a1402860" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:21:41.696527 16143 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40844 (request call id 2768) took 1001 ms. Trace:
I20260709 13:21:41.696945 16143 rpcz_store.cc:276] 0709 13:21:40.695203 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:40.695487 (+   284us) service_pool.cc:224] Handling call
0709 13:21:41.696398 (+1000911us) spinlock_profiling.cc:243] Waited 223 ms on lock 0x7b540006e0d4. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:21:41.696483 (+    85us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":512902912}
I20260709 13:21:41.698176 16819 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "434bff3082d141838068f323bdb0d1ed"
dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
mode: GRACEFUL
new_leader_uuid: "90f3da57796b488c9087a2f523adf045"
 from {username='slave'} at 127.0.0.1:37818
I20260709 13:21:41.698755 16819 raft_consensus.cc:606] T 434bff3082d141838068f323bdb0d1ed P cb82cc7bd6db46d980c75237a1402860 [term 15 FOLLOWER]: Received request to transfer leadership to TS 90f3da57796b488c9087a2f523adf045
I20260709 13:21:41.699146 16819 raft_consensus.cc:614] T 434bff3082d141838068f323bdb0d1ed P cb82cc7bd6db46d980c75237a1402860 [term 15 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:21:41.698800 16048 catalog_manager.cc:5697] T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860 reported cstate change: term changed from 8 to 17, leader changed from cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) to 647a343a542d4a139fe106c19095129b (127.14.177.129). New cstate: current_term: 17 leader_uuid: "647a343a542d4a139fe106c19095129b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:21:41.700407 16048 catalog_manager.cc:5697] T 26354361e9b044cea1c2b86c4882fe92 P cb82cc7bd6db46d980c75237a1402860 reported cstate change: term changed from 8 to 13, leader changed from 647a343a542d4a139fe106c19095129b (127.14.177.129) to cb82cc7bd6db46d980c75237a1402860 (127.14.177.132). New cstate: current_term: 13 leader_uuid: "cb82cc7bd6db46d980c75237a1402860" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:21:41.708559 22168 raft_consensus.cc:3055] T ee9ed67006144797b1de6ea8c2013a68 P 647a343a542d4a139fe106c19095129b [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:21:41.709131 22168 raft_consensus.cc:740] T ee9ed67006144797b1de6ea8c2013a68 P 647a343a542d4a139fe106c19095129b [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:21:41.710096 22168 consensus_queue.cc:260] T ee9ed67006144797b1de6ea8c2013a68 P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.13, Last appended by leader: 13, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:41.711648 22168 raft_consensus.cc:3060] T ee9ed67006144797b1de6ea8c2013a68 P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:21:41.687227 22225 raft_consensus.cc:515] T 68a6838d82ef4cac83507f906139d90f P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:41.717037 16143 raft_consensus.cc:1275] T 5a959dbecc2648eb85d1cc550f5a3245 P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 13:21:41.691681 22196 raft_consensus.cc:515] T 26354361e9b044cea1c2b86c4882fe92 P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:41.678411 22104 consensus_queue.cc:260] T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.12, Last appended by leader: 12, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:41.722508 22196 leader_election.cc:290] T 26354361e9b044cea1c2b86c4882fe92 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
W20260709 13:21:41.739480 22268 log.cc:927] Time spent T 84f2da98864b4425bf7c0055951fdd4b P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 1.148s	user 0.003s	sys 0.005s
I20260709 13:21:41.733745 16150 raft_consensus.cc:2468] T 8158bbd17b994d2eb96f560bd0f1b88d P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 15.
I20260709 13:21:41.741765 22164 raft_consensus.cc:493] T 8158bbd17b994d2eb96f560bd0f1b88d P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:41.745012 22106 raft_consensus.cc:2749] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:21:41.747963 22215 consensus_queue.cc:260] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.11, Last appended by leader: 11, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } }
I20260709 13:21:41.748729 16219 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "66f75124215f4ca6bef831e174a792d8"
dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
mode: GRACEFUL
new_leader_uuid: "647a343a542d4a139fe106c19095129b"
 from {username='slave'} at 127.0.0.1:39144
I20260709 13:21:41.749398 16219 raft_consensus.cc:606] T 66f75124215f4ca6bef831e174a792d8 P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Received request to transfer leadership to TS 647a343a542d4a139fe106c19095129b
I20260709 13:21:41.749778 16219 raft_consensus.cc:614] T 66f75124215f4ca6bef831e174a792d8 P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 13:21:41.751274 22305 log.cc:927] Time spent T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860: Append to log took a long time: real 0.335s	user 0.003s	sys 0.000s
W20260709 13:21:41.752496 22275 log.cc:927] Time spent T fffab26427e24d398e2695aad40ca134 P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 1.164s	user 0.000s	sys 0.012s
I20260709 13:21:41.752671 16148 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40844 (request call id 2764) took 1095 ms. Trace:
I20260709 13:21:41.753100 16148 rpcz_store.cc:276] 0709 13:21:40.657033 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:40.657164 (+   131us) service_pool.cc:224] Handling call
0709 13:21:41.752495 (+1095331us) spinlock_profiling.cc:243] Waited 337 ms on lock 0x7b54000e9054. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:21:41.752605 (+   110us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":774332160}
I20260709 13:21:41.734609 22104 raft_consensus.cc:3060] T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:21:41.755278 16219 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "84f2da98864b4425bf7c0055951fdd4b"
dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
mode: GRACEFUL
new_leader_uuid: "90f3da57796b488c9087a2f523adf045"
 from {username='slave'} at 127.0.0.1:39144
I20260709 13:21:41.756742 16219 raft_consensus.cc:606] T 84f2da98864b4425bf7c0055951fdd4b P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Received request to transfer leadership to TS 90f3da57796b488c9087a2f523adf045
I20260709 13:21:41.757279 16219 raft_consensus.cc:614] T 84f2da98864b4425bf7c0055951fdd4b P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:21:41.746606 22164 raft_consensus.cc:515] T 8158bbd17b994d2eb96f560bd0f1b88d P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:41.764734 16146 raft_consensus.cc:2468] T 23f82224f4564fea93572e8206bad1fe P 647a343a542d4a139fe106c19095129b [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 14.
W20260709 13:21:41.768213 16264 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:41.769531 22106 raft_consensus.cc:493] T 02838c86531d4f98bf58d1c22b8baf77 P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:41.771584 22106 raft_consensus.cc:515] T 02838c86531d4f98bf58d1c22b8baf77 P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:41.772468 16222 raft_consensus.cc:1240] T af57996a8be84d398f0f023e8d08ef77 P 1ca2725d290e47d68b1a4b609c9816ac [term 18 LEADER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 17. Current term is 18. Ops: [17.12-17.12]
I20260709 13:21:41.773268 16222 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57604 (request call id 3092) took 1568 ms. Trace:
I20260709 13:21:41.773655 16222 rpcz_store.cc:276] 0709 13:21:40.204955 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:40.340430 (+135475us) service_pool.cc:224] Handling call
0709 13:21:40.340802 (+   372us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:21:41.772175 (+1431373us) spinlock_profiling.cc:243] Waited 1.51 ms on lock 0x7b5400374720. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0ad6dc0 00007f69b0b611c0 00007f69b0b62231 00007f69b0b5ed8f 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c
0709 13:21:41.772954 (+   779us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:41.773205 (+   251us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3483520}
I20260709 13:21:41.779371 22241 consensus_queue.cc:1048] T 5a96373bb0e14dce8a266b2fc6c58507 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:21:41.785684 22206 leader_election.cc:290] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 16 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:41.788002 22225 leader_election.cc:290] T 68a6838d82ef4cac83507f906139d90f P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:41.791991 16265 leader_election.cc:304] T 23f82224f4564fea93572e8206bad1fe P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
I20260709 13:21:41.793824 16265 leader_election.cc:304] T c0eab25ce4c64e018a041beeda2867db P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:21:41.794387 16144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02838c86531d4f98bf58d1c22b8baf77" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:41.796729 16265 leader_election.cc:304] T 23f82224f4564fea93572e8206bad1fe P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:21:41.773804 22106 leader_election.cc:290] T 02838c86531d4f98bf58d1c22b8baf77 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
W20260709 13:21:41.805794 22320 log.cc:927] Time spent T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 0.089s	user 0.001s	sys 0.000s
I20260709 13:21:41.794173 22241 raft_consensus.cc:2749] T 23f82224f4564fea93572e8206bad1fe P 90f3da57796b488c9087a2f523adf045 [term 14 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:21:41.806526 22211 consensus_queue.cc:1048] T 5a959dbecc2648eb85d1cc550f5a3245 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.003s
I20260709 13:21:41.811034 16226 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57604 (request call id 3095) took 1597 ms. Trace:
I20260709 13:21:41.811666 16226 rpcz_store.cc:276] 0709 13:21:40.213584 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:40.844713 (+631129us) service_pool.cc:224] Handling call
0709 13:21:40.845087 (+   374us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:21:40.847177 (+  2090us) raft_consensus.cc:1534] Early marking committed up to index 14
0709 13:21:40.853517 (+  6340us) log.cc:844] Serialized 14 byte log entry
0709 13:21:41.150950 (+297433us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:21:41.151013 (+    63us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:21:41.151093 (+    80us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:41.810671 (+659578us) spinlock_profiling.cc:243] Waited 1.8 ms on lock 0x7b5400367c54. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:21:41.810751 (+    80us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:21:41.810948 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":28958,"prepare.queue_time_us":6613,"prepare.run_cpu_time_us":32,"prepare.run_wall_time_us":33,"raft.queue_time_us":39548,"raft.run_cpu_time_us":142,"raft.run_wall_time_us":18432,"spinlock_wait_cycles":4132608,"thread_start_us":268039,"threads_started":2,"wal-append.queue_time_us":162078}
I20260709 13:21:41.815419 22106 raft_consensus.cc:2804] T 23f82224f4564fea93572e8206bad1fe P 90f3da57796b488c9087a2f523adf045 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:21:41.816074 22106 raft_consensus.cc:697] T 23f82224f4564fea93572e8206bad1fe P 90f3da57796b488c9087a2f523adf045 [term 14 LEADER]: Becoming Leader. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:21:41.816685 22202 raft_consensus.cc:493] T 3a7ca61592e848ea9b0fb81a2ce33cdc P cb82cc7bd6db46d980c75237a1402860 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:41.817188 22202 raft_consensus.cc:515] T 3a7ca61592e848ea9b0fb81a2ce33cdc P cb82cc7bd6db46d980c75237a1402860 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:41.795034 22111 raft_consensus.cc:2749] T c0eab25ce4c64e018a041beeda2867db P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:21:41.817045 22106 consensus_queue.cc:237] T 23f82224f4564fea93572e8206bad1fe P 90f3da57796b488c9087a2f523adf045 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:41.822615 16218 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57604 (request call id 3094) took 1609 ms. Trace:
I20260709 13:21:41.822824 22202 leader_election.cc:290] T 3a7ca61592e848ea9b0fb81a2ce33cdc P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:41.823045 16218 rpcz_store.cc:276] 0709 13:21:40.213254 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:40.434980 (+221726us) service_pool.cc:224] Handling call
0709 13:21:40.435338 (+   358us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:21:41.496844 (+1061506us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:41.822276 (+325432us) spinlock_profiling.cc:243] Waited 1.02 s on lock 0x7b54002c9054. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:21:41.822541 (+   265us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2340907264}
I20260709 13:21:41.826253 16819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5718c623ebee4d2bb770f0f9ddb2f164" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 17 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:41.827013 16143 raft_consensus.cc:1240] T b87147731a68400e8cce9b879daa7c5d P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 16. Current term is 17. Ops: []
I20260709 13:21:41.827632 16148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2372981006d4b218215c28964dea623" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:41.828368 16148 raft_consensus.cc:2393] T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 in current term 19: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
I20260709 13:21:41.826319 16146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746aabf856ab401c9ef5e859371e7b67" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
W20260709 13:21:41.835680 16117 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:41.859651 16295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7ca61592e848ea9b0fb81a2ce33cdc" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:41.861620 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c11074579b046d99e585c2818e18a2c" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:21:41.866247 16293 raft_consensus.cc:3060] T 8c11074579b046d99e585c2818e18a2c P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:21:41.885015 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 88321.2us
I20260709 13:21:41.830006 22029 raft_consensus.cc:493] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
W20260709 13:21:41.886035 16790 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:41.887336 16143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5d59407eb545b895d8d922e0dd69fd" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:41.888326 16143 raft_consensus.cc:2468] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 18.
W20260709 13:21:41.890110 16789 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:41.892978 16819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:21:41.893798 16819 raft_consensus.cc:3060] T 19baa7bc367844c288bca8f81439c413 P cb82cc7bd6db46d980c75237a1402860 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:21:41.896518 16790 leader_election.cc:304] T ee9ed67006144797b1de6ea8c2013a68 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 647a343a542d4a139fe106c19095129b
I20260709 13:21:41.896852 16825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26354361e9b044cea1c2b86c4882fe92" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:41.897305 16817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a9de347c711477e87c582d003f36925" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:21:41.898084 16817 raft_consensus.cc:3060] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:21:41.864187 22211 raft_consensus.cc:493] T ada5a004ba7c46819051e3ccec71ffdb P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:21:41.908738 16293 raft_consensus.cc:2468] T 8c11074579b046d99e585c2818e18a2c P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 17.
I20260709 13:21:41.892084 22243 raft_consensus.cc:493] T 3884f864a96d428785895e5463e5728f P cb82cc7bd6db46d980c75237a1402860 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:41.905459 21802 raft_consensus.cc:2804] T ee9ed67006144797b1de6ea8c2013a68 P cb82cc7bd6db46d980c75237a1402860 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:21:41.900362 22104 raft_consensus.cc:493] T 7c2095a5f4ad49408dbc7f69c606120d P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:41.897209 22230 raft_consensus.cc:493] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:41.914897 16817 raft_consensus.cc:2468] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 20.
I20260709 13:21:41.912724 22104 raft_consensus.cc:515] T 7c2095a5f4ad49408dbc7f69c606120d P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:41.920817 16819 raft_consensus.cc:2468] T 19baa7bc367844c288bca8f81439c413 P cb82cc7bd6db46d980c75237a1402860 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 19.
I20260709 13:21:41.916247 21623 consensus_queue.cc:1059] T b87147731a68400e8cce9b879daa7c5d P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: INVALID_TERM, Last received: 16.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.008s
I20260709 13:21:41.925602 21623 raft_consensus.cc:3055] T b87147731a68400e8cce9b879daa7c5d P cb82cc7bd6db46d980c75237a1402860 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:21:41.926119 21623 raft_consensus.cc:740] T b87147731a68400e8cce9b879daa7c5d P cb82cc7bd6db46d980c75237a1402860 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:21:41.928349 16151 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40844 (request call id 2763) took 1271 ms. Trace:
I20260709 13:21:41.928951 16151 rpcz_store.cc:276] 0709 13:21:40.656723 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:40.661795 (+  5072us) service_pool.cc:224] Handling call
0709 13:21:40.691283 (+ 29488us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:21:41.627659 (+936376us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:21:41.627687 (+    28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:21:41.627725 (+    38us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:21:41.627759 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:41.928007 (+300248us) spinlock_profiling.cc:243] Waited 936 ms on lock 0x7b540006dbd4. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:21:41.928078 (+    71us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:21:41.928271 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2153041792}
I20260709 13:21:41.932163 22104 leader_election.cc:290] T 7c2095a5f4ad49408dbc7f69c606120d P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:21:41.932710 16147 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40830 (request call id 1839) took 1386 ms. Trace:
I20260709 13:21:41.933100 16147 rpcz_store.cc:276] 0709 13:21:40.546708 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:40.547199 (+   491us) service_pool.cc:224] Handling call
0709 13:21:41.932506 (+1385307us) spinlock_profiling.cc:243] Waited 483 ms on lock 0x7b54000d1454. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:21:41.932652 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1110977280}
W20260709 13:21:41.941721 16266 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:21:41.943778 16114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:41.917906 16790 leader_election.cc:304] T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:21:41.894148 22029 raft_consensus.cc:515] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:41.954681 16147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c2095a5f4ad49408dbc7f69c606120d" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:41.954542 16825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31969392ccdb43b491911a319e8bc7e1" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:41.956266 16821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f82224f4564fea93572e8206bad1fe" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:21:41.957885 16821 raft_consensus.cc:3060] T 23f82224f4564fea93572e8206bad1fe P cb82cc7bd6db46d980c75237a1402860 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:21:41.959741 16820 raft_consensus.cc:3060] T 806d7690c2b644959152074bb2cab0a7 P cb82cc7bd6db46d980c75237a1402860 [term 15 LEARNER]: Advancing to term 16
I20260709 13:21:41.915719 21802 raft_consensus.cc:697] T ee9ed67006144797b1de6ea8c2013a68 P cb82cc7bd6db46d980c75237a1402860 [term 17 LEADER]: Becoming Leader. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:21:41.965665 16151 raft_consensus.cc:1275] T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 13:21:41.967119 16819 raft_consensus.cc:1240] T 74ca8140f35c4ae29e8a664f6bdfc829 P cb82cc7bd6db46d980c75237a1402860 [term 17 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 16. Current term is 17. Ops: []
I20260709 13:21:41.967892 16295 raft_consensus.cc:1275] T 66ae176961b74813a58bc15c9c4c6f2a P 90f3da57796b488c9087a2f523adf045 [term 21 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 20 index: 13. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
I20260709 13:21:41.968477 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5ad4eb281564936a59f41e387a47156" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:41.969261 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ab0ac8347641e6955e9bce7d90c1a2" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:41.969583 16292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8158bbd17b994d2eb96f560bd0f1b88d" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:41.970345 16294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66ae176961b74813a58bc15c9c4c6f2a" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:41.969529 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:41.971131 16294 raft_consensus.cc:2410] T 66ae176961b74813a58bc15c9c4c6f2a P 90f3da57796b488c9087a2f523adf045 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac for term 21 because replica has last-logged OpId of term: 20 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 13.
I20260709 13:21:41.967227 16291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8638ee363273426db1393dd81e5c7b72" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:41.971452 22111 raft_consensus.cc:493] T e4dc6b0541ca431ebc757beac00521f1 P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:41.971959 22111 raft_consensus.cc:515] T e4dc6b0541ca431ebc757beac00521f1 P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:41.970829 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68a6838d82ef4cac83507f906139d90f" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:41.973325 16297 raft_consensus.cc:2393] T 68a6838d82ef4cac83507f906139d90f P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 16: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
I20260709 13:21:41.973841 22111 leader_election.cc:290] T e4dc6b0541ca431ebc757beac00521f1 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:21:41.971253 16293 raft_consensus.cc:2393] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 18: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
I20260709 13:21:41.980619 22031 consensus_queue.cc:1048] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.001s
W20260709 13:21:41.983666 16265 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:41.984622 16265 leader_election.cc:400] T c4f65cdf00b748bb963ee6e81ef43b79 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 17 pre-election: Vote denied by peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741) with higher term. Message: Invalid argument: T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 for earlier term 17. Current term is 18.
I20260709 13:21:41.984999 16265 leader_election.cc:403] T c4f65cdf00b748bb963ee6e81ef43b79 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 17 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:21:41.985687 22106 raft_consensus.cc:3060] T c4f65cdf00b748bb963ee6e81ef43b79 P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Advancing to term 18
I20260709 13:21:41.989774 16817 raft_consensus.cc:1275] T 3a7ca61592e848ea9b0fb81a2ce33cdc P cb82cc7bd6db46d980c75237a1402860 [term 18 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 18 index: 15. (index mismatch)
I20260709 13:21:41.909526 22211 raft_consensus.cc:515] T ada5a004ba7c46819051e3ccec71ffdb P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:41.943461 22242 raft_consensus.cc:493] T a378d7f796f94615b3b347119a9880fd P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:21:42.008029 16296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806d7690c2b644959152074bb2cab0a7" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:21:42.013252 22211 leader_election.cc:290] T ada5a004ba7c46819051e3ccec71ffdb P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:42.013842 22211 raft_consensus.cc:493] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:42.014298 22211 raft_consensus.cc:515] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:41.915908 22243 raft_consensus.cc:515] T 3884f864a96d428785895e5463e5728f P cb82cc7bd6db46d980c75237a1402860 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:42.016428 16818 raft_consensus.cc:1275] T 3884f864a96d428785895e5463e5728f P cb82cc7bd6db46d980c75237a1402860 [term 12 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 12 index: 12. (index mismatch)
I20260709 13:21:42.017477 22243 leader_election.cc:290] T 3884f864a96d428785895e5463e5728f P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:41.957991 21623 consensus_queue.cc:260] T b87147731a68400e8cce9b879daa7c5d P cb82cc7bd6db46d980c75237a1402860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:41.960225 22104 raft_consensus.cc:493] T 8cc959fb7f704fba9009c0595467d074 P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:41.990734 16816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87147731a68400e8cce9b879daa7c5d" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:41.991086 16789 leader_election.cc:304] T c9944d5e4b8c4109af9955c4f08c9095 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:21:41.996150 16151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4dc6b0541ca431ebc757beac00521f1" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 17 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:42.032393 16151 raft_consensus.cc:2410] T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 for term 17 because replica has last-logged OpId of term: 15 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 7.
I20260709 13:21:42.030826 16148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3884f864a96d428785895e5463e5728f" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 13 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:41.999308 16147 raft_consensus.cc:1275] T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:21:41.966223 22201 raft_consensus.cc:2804] T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860 [term 18 FOLLOWER]: Leader pre-election won for term 19
W20260709 13:21:42.035748 16789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 44930.1us
I20260709 13:21:41.951390 22230 raft_consensus.cc:515] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:42.043311 16142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62fe3346cd3c42c4a55e14af9073bca4" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:42.010212 22111 raft_consensus.cc:493] T 18ad7e3af5504571a69d10c00750a01c P 90f3da57796b488c9087a2f523adf045 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:42.048341 16821 raft_consensus.cc:2468] T 23f82224f4564fea93572e8206bad1fe P cb82cc7bd6db46d980c75237a1402860 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 14.
I20260709 13:21:42.051664 16820 raft_consensus.cc:1275] T 806d7690c2b644959152074bb2cab0a7 P cb82cc7bd6db46d980c75237a1402860 [term 16 LEARNER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 13:21:42.040252 22243 raft_consensus.cc:2804] T c9944d5e4b8c4109af9955c4f08c9095 P cb82cc7bd6db46d980c75237a1402860 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:21:42.039125 22201 raft_consensus.cc:493] T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
W20260709 13:21:42.057580 16116 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:42.027441 22242 raft_consensus.cc:515] T a378d7f796f94615b3b347119a9880fd P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:42.035583 22104 raft_consensus.cc:515] T 8cc959fb7f704fba9009c0595467d074 P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:42.040491 22205 consensus_queue.cc:1048] T 66f75124215f4ca6bef831e174a792d8 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.019s
I20260709 13:21:42.087743 22074 consensus_queue.cc:1048] T 7c2095a5f4ad49408dbc7f69c606120d P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:21:42.088785 22196 consensus_queue.cc:1048] T 02838c86531d4f98bf58d1c22b8baf77 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:21:42.108589 22211 leader_election.cc:290] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:42.110693 22230 leader_election.cc:290] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:42.058926 22029 leader_election.cc:290] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:21:42.114045 22242 leader_election.cc:290] T a378d7f796f94615b3b347119a9880fd P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:42.069854 22201 raft_consensus.cc:3060] T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:21:42.051141 22111 raft_consensus.cc:515] T 18ad7e3af5504571a69d10c00750a01c P 90f3da57796b488c9087a2f523adf045 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } }
I20260709 13:21:42.073580 22182 raft_consensus.cc:493] T ecc1e363230043fe9fef390f0b979f20 P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:42.118249 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a9de347c711477e87c582d003f36925" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:21:42.158473 16293 raft_consensus.cc:3060] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:21:42.118542 16292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1df93779209748b9860f6d893dd3de53" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:21:42.159696 16292 raft_consensus.cc:3060] T 1df93779209748b9860f6d893dd3de53 P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:21:42.118918 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ad5dbfa99ee4b8d9e28be92b3f36db5" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:21:42.119226 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4f65cdf00b748bb963ee6e81ef43b79" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:21:42.119545 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4dc6b0541ca431ebc757beac00521f1" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:42.161077 16297 raft_consensus.cc:3060] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:21:42.161550 16299 raft_consensus.cc:2468] T e4dc6b0541ca431ebc757beac00521f1 P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 16.
I20260709 13:21:42.124800 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02838c86531d4f98bf58d1c22b8baf77" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:42.162257 16222 raft_consensus.cc:1275] T d16dfad0c3b540e5b4b86492ddf87208 P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 13:21:42.126111 16192 leader_election.cc:304] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:21:42.126724 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4dc6b0541ca431ebc757beac00521f1" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 17 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:42.127307 16223 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33050 (request call id 1037) took 1600 ms. Trace:
I20260709 13:21:42.137383 16298 raft_consensus.cc:1275] T 7c2095a5f4ad49408dbc7f69c606120d P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 6. (term mismatch)
I20260709 13:21:42.163810 16298 pending_rounds.cc:85] T 7c2095a5f4ad49408dbc7f69c606120d P 90f3da57796b488c9087a2f523adf045: Aborting all ops after (but not including) 5
I20260709 13:21:42.164112 16298 pending_rounds.cc:107] T 7c2095a5f4ad49408dbc7f69c606120d P 90f3da57796b488c9087a2f523adf045: Aborting uncommitted NO_OP operation due to leader change: 10.6
I20260709 13:21:42.164405 16298 raft_consensus.cc:2889] T 7c2095a5f4ad49408dbc7f69c606120d P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:21:42.164785 16298 pending_rounds.cc:107] T 7c2095a5f4ad49408dbc7f69c606120d P 90f3da57796b488c9087a2f523adf045: Aborting uncommitted NO_OP operation due to leader change: 12.7
I20260709 13:21:42.163594 16222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1df93779209748b9860f6d893dd3de53" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:21:42.165189 16298 raft_consensus.cc:2889] T 7c2095a5f4ad49408dbc7f69c606120d P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:21:42.115111 22310 log.cc:927] Time spent T af57996a8be84d398f0f023e8d08ef77 P cb82cc7bd6db46d980c75237a1402860: Append to log took a long time: real 0.620s	user 0.003s	sys 0.001s
I20260709 13:21:42.128309 16191 leader_election.cc:304] T 08c74490c40243a88e62d1760b5cd814 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 90f3da57796b488c9087a2f523adf045
I20260709 13:21:42.117106 16294 raft_consensus.cc:1275] T d16dfad0c3b540e5b4b86492ddf87208 P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 13:21:42.128113 16225 raft_consensus.cc:1275] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 13:21:42.167960 16222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4dc6b0541ca431ebc757beac00521f1" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:42.168424 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68a6838d82ef4cac83507f906139d90f" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
W20260709 13:21:42.130383 16115 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:21:42.169628 16115 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:42.173856 16143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5d59407eb545b895d8d922e0dd69fd" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:21:42.175490 16147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5ad4eb281564936a59f41e387a47156" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:42.176543 16147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5718c623ebee4d2bb770f0f9ddb2f164" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 17 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:42.177816 16151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68a6838d82ef4cac83507f906139d90f" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:42.178429 16151 raft_consensus.cc:2468] T 68a6838d82ef4cac83507f906139d90f P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 16.
I20260709 13:21:42.139868 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c2095a5f4ad49408dbc7f69c606120d" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:42.150157 22292 consensus_queue.cc:1048] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:21:42.151129 22164 leader_election.cc:290] T 8158bbd17b994d2eb96f560bd0f1b88d P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:42.135598 22291 raft_consensus.cc:493] T 02838c86531d4f98bf58d1c22b8baf77 P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:21:42.152851 16150 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40830 (request call id 1825) took 2979 ms (client timeout 2159 ms). Trace:
W20260709 13:21:42.191076 16150 rpcz_store.cc:269] 0709 13:21:39.173471 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:39.186871 (+ 13400us) service_pool.cc:224] Handling call
0709 13:21:42.152634 (+2965763us) spinlock_profiling.cc:243] Waited 2.53 s on lock 0x7b54000e9f54. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:21:42.152751 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5812458880}
I20260709 13:21:42.153894 22104 leader_election.cc:290] T 8cc959fb7f704fba9009c0595467d074 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:21:42.154165 16145 raft_consensus.cc:1275] T 8158bbd17b994d2eb96f560bd0f1b88d P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 13:21:42.137877 16142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:42.163312 16223 rpcz_store.cc:276] 0709 13:21:40.544233 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:41.602065 (+1057832us) service_pool.cc:224] Handling call
0709 13:21:41.611747 (+  9682us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:21:41.611900 (+   153us) raft_consensus.cc:1534] Early marking committed up to index 15
0709 13:21:41.623950 (+ 12050us) log.cc:844] Serialized 14 byte log entry
0709 13:21:42.123122 (+499172us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:21:42.126910 (+  3788us) raft_consensus.cc:1645] Marking committed up to 15
0709 13:21:42.126975 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:42.127060 (+    85us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:21:42.127242 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":429529,"prepare.queue_time_us":13492,"prepare.run_cpu_time_us":34,"prepare.run_wall_time_us":35,"raft.queue_time_us":431068,"raft.run_cpu_time_us":74,"raft.run_wall_time_us":74,"thread_start_us":69283,"threads_started":1,"wal-append.queue_time_us":106884}
I20260709 13:21:42.173317 16115 leader_election.cc:304] T 8d4ab870a05347868956aa8b841b2981 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
W20260709 13:21:42.148049 16264 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:42.194514 16221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4f65cdf00b748bb963ee6e81ef43b79" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:21:42.166483 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68a6838d82ef4cac83507f906139d90f" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:42.195142 16221 raft_consensus.cc:3060] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:21:42.181556 16219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ad5dbfa99ee4b8d9e28be92b3f36db5" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:21:42.195894 16219 raft_consensus.cc:3060] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Advancing to term 16
I20260709 13:21:42.182250 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26354361e9b044cea1c2b86c4882fe92" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:42.055495 21802 consensus_queue.cc:237] T ee9ed67006144797b1de6ea8c2013a68 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.13, Last appended by leader: 13, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:21:42.182749 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 54526.4us
I20260709 13:21:42.179890 16224 raft_consensus.cc:2468] T 7c2095a5f4ad49408dbc7f69c606120d P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 13.
I20260709 13:21:42.185803 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:21:42.199594 16225 raft_consensus.cc:3060] T 19baa7bc367844c288bca8f81439c413 P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:21:42.192914 16143 raft_consensus.cc:2468] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 18.
W20260709 13:21:42.243856 16115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 70603.3us
I20260709 13:21:42.222219 16222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746aabf856ab401c9ef5e859371e7b67" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:42.249960 16150 raft_consensus.cc:1240] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 16. Current term is 17. Ops: [14.10-16.11]
I20260709 13:21:42.182003 22236 raft_consensus.cc:493] T af57996a8be84d398f0f023e8d08ef77 P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:42.212540 16219 raft_consensus.cc:2468] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 16.
I20260709 13:21:42.251127 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cc959fb7f704fba9009c0595467d074" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:42.249815 16147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a9de347c711477e87c582d003f36925" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:42.270535 16147 raft_consensus.cc:2393] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 20: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
I20260709 13:21:42.195369 16298 raft_consensus.cc:2468] T 68a6838d82ef4cac83507f906139d90f P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 16.
I20260709 13:21:42.242973 16217 raft_consensus.cc:1240] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 14. Current term is 15. Ops: []
I20260709 13:21:42.250967 16294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:21:42.263351 16297 raft_consensus.cc:2468] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 16.
I20260709 13:21:42.274626 16293 raft_consensus.cc:2468] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 20.
I20260709 13:21:42.275590 16222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c11074579b046d99e585c2818e18a2c" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:21:42.279330 16222 raft_consensus.cc:3060] T 8c11074579b046d99e585c2818e18a2c P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: Advancing to term 17
I20260709 13:21:42.183296 22197 raft_consensus.cc:493] T 18ad7e3af5504571a69d10c00750a01c P 647a343a542d4a139fe106c19095129b [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:21:42.293375 16789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 252624us
W20260709 13:21:42.294536 16192 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 168495us
I20260709 13:21:42.307981 16789 leader_election.cc:304] T 8c11074579b046d99e585c2818e18a2c P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:21:42.312304 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3884f864a96d428785895e5463e5728f" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 13 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:42.312469 16266 leader_election.cc:304] T 31969392ccdb43b491911a319e8bc7e1 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
I20260709 13:21:42.315714 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:42.311471 16117 leader_election.cc:304] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:21:42.194005 22149 raft_consensus.cc:2804] T 08c74490c40243a88e62d1760b5cd814 P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:21:42.191061 22241 raft_consensus.cc:493] T d16dfad0c3b540e5b4b86492ddf87208 P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:42.248814 22104 raft_consensus.cc:493] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:42.322088 16292 raft_consensus.cc:2468] T 1df93779209748b9860f6d893dd3de53 P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 16.
I20260709 13:21:42.326905 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5d59407eb545b895d8d922e0dd69fd" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:42.267055 22211 raft_consensus.cc:493] T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:21:42.344369 16823 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49906 (request call id 1190) took 1120 ms. Trace:
I20260709 13:21:42.344836 16823 rpcz_store.cc:276] 0709 13:21:41.224177 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:41.224923 (+   746us) service_pool.cc:224] Handling call
0709 13:21:41.225293 (+   370us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:21:41.248802 (+ 23509us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:21:41.248850 (+    48us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:21:41.254904 (+  6054us) log.cc:844] Serialized 26 byte log entry
0709 13:21:41.368799 (+113895us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:21:41.368838 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:41.368907 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:21:41.543989 (+175082us) spinlock_profiling.cc:243] Waited 2.67 ms on lock 0x7b48000cd758. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b5a88aa1 00007f69b5a7fb37 00007f69b5a7f9ca 00007f69b5b7726f 00007f69b5b76f37 00007f69b5b8b55c 00007f69b5b8b508 00007f69b5b8b488 00007f69b5b8b443 00007f69b5b8a6ec 00007f69b4ab4a96
0709 13:21:42.114448 (+570459us) spinlock_profiling.cc:243] Waited 6.5 ms on lock 0x7b4400022b4c. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69ad8fadb1 00007f69ad998c85 00007f69ad9996a2 00007f69b070b1db 00007f69b0706aaf 00007f69b06fa86e 00007f69b06f9bb6 00007f69b06f97d8 00007f69b0703221 00007f69b07031d9 00007f69b0703169
0709 13:21:42.344143 (+229695us) raft_consensus.cc:1685] finished
0709 13:21:42.344169 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:21:42.344311 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":12003,"raft.queue_time_us":26831,"raft.run_cpu_time_us":2714,"raft.run_wall_time_us":287210,"spinlock_wait_cycles":21141120,"thread_start_us":60184,"threads_started":1,"wal-append.queue_time_us":119722}
I20260709 13:21:42.346752 16820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada5a004ba7c46819051e3ccec71ffdb" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:42.347505 16821 raft_consensus.cc:1275] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860 [term 18 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 13:21:42.344086 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7ca61592e848ea9b0fb81a2ce33cdc" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:42.348419 16224 raft_consensus.cc:2468] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 18.
I20260709 13:21:42.349949 16225 raft_consensus.cc:2468] T 19baa7bc367844c288bca8f81439c413 P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 19.
I20260709 13:21:42.350018 16820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a378d7f796f94615b3b347119a9880fd" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:42.351339 16223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9944d5e4b8c4109af9955c4f08c9095" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:21:42.351895 16223 raft_consensus.cc:3060] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:21:42.351651 16821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a9de347c711477e87c582d003f36925" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:42.352834 16821 raft_consensus.cc:2393] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 20: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
I20260709 13:21:42.352823 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62fe3346cd3c42c4a55e14af9073bca4" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:42.353930 16225 raft_consensus.cc:2468] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 21.
I20260709 13:21:42.354723 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2372981006d4b218215c28964dea623" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:42.350553 16789 leader_election.cc:304] T 3884f864a96d428785895e5463e5728f P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:21:42.316090 22106 raft_consensus.cc:2749] T c4f65cdf00b748bb963ee6e81ef43b79 P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Leader pre-election lost for term 17. Reason: Vote denied by peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741) with higher term. Message: Invalid argument: T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 for earlier term 17. Current term is 18.
I20260709 13:21:42.314882 21930 raft_consensus.cc:493] T 7c2095a5f4ad49408dbc7f69c606120d P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:42.360801 16222 raft_consensus.cc:2468] T 8c11074579b046d99e585c2818e18a2c P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 17.
I20260709 13:21:42.362169 16222 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c2f5f89186204242bf377f7c42b2e2da"
dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
mode: GRACEFUL
new_leader_uuid: "90f3da57796b488c9087a2f523adf045"
 from {username='slave'} at 127.0.0.1:39144
W20260709 13:21:42.362360 16114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:42.202950 22291 raft_consensus.cc:515] T 02838c86531d4f98bf58d1c22b8baf77 P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:42.361760 16790 leader_election.cc:304] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:21:42.364570 16222 raft_consensus.cc:606] T c2f5f89186204242bf377f7c42b2e2da P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Received request to transfer leadership to TS 90f3da57796b488c9087a2f523adf045
I20260709 13:21:42.326866 22152 raft_consensus.cc:2804] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:21:42.374006 16790 leader_election.cc:304] T 3a7ca61592e848ea9b0fb81a2ce33cdc P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 90f3da57796b488c9087a2f523adf045
I20260709 13:21:42.328253 22149 raft_consensus.cc:493] T 08c74490c40243a88e62d1760b5cd814 P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Starting leader election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:21:42.375977 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:42.386569 16224 raft_consensus.cc:2468] T ecc1e363230043fe9fef390f0b979f20 P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 18.
I20260709 13:21:42.328591 22220 raft_consensus.cc:2804] T 8c11074579b046d99e585c2818e18a2c P cb82cc7bd6db46d980c75237a1402860 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:21:42.334331 22246 raft_consensus.cc:2804] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:21:42.410058 16226 raft_consensus.cc:2468] T 02838c86531d4f98bf58d1c22b8baf77 P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 17.
I20260709 13:21:42.351109 22145 raft_consensus.cc:493] T 5a959dbecc2648eb85d1cc550f5a3245 P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:42.333300 22119 consensus_queue.cc:1048] T 02838c86531d4f98bf58d1c22b8baf77 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.050s
I20260709 13:21:42.370253 22325 raft_consensus.cc:2749] T 31969392ccdb43b491911a319e8bc7e1 P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:21:42.317204 22242 consensus_queue.cc:1048] T 8158bbd17b994d2eb96f560bd0f1b88d P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.049s
W20260709 13:21:42.367208 22322 log.cc:927] Time spent T 3884f864a96d428785895e5463e5728f P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 0.051s	user 0.001s	sys 0.000s
I20260709 13:21:42.285454 22236 raft_consensus.cc:515] T af57996a8be84d398f0f023e8d08ef77 P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:42.400264 22149 raft_consensus.cc:3060] T 08c74490c40243a88e62d1760b5cd814 P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:21:42.369150 16222 raft_consensus.cc:614] T c2f5f89186204242bf377f7c42b2e2da P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 13:21:42.448386 22272 log.cc:927] Time spent T 5a96373bb0e14dce8a266b2fc6c58507 P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.065s	user 0.001s	sys 0.000s
I20260709 13:21:42.359376 22211 raft_consensus.cc:515] T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:42.376946 21579 raft_consensus.cc:2749] T 3884f864a96d428785895e5463e5728f P cb82cc7bd6db46d980c75237a1402860 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:21:42.370725 21930 raft_consensus.cc:515] T 7c2095a5f4ad49408dbc7f69c606120d P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:42.467914 22220 raft_consensus.cc:697] T 8c11074579b046d99e585c2818e18a2c P cb82cc7bd6db46d980c75237a1402860 [term 17 LEADER]: Becoming Leader. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:21:42.400472 22198 raft_consensus.cc:2749] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:21:42.186046 22182 raft_consensus.cc:515] T ecc1e363230043fe9fef390f0b979f20 P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:42.399891 22152 raft_consensus.cc:697] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [term 19 LEADER]: Becoming Leader. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:21:42.366552 22074 consensus_queue.cc:1048] T 8cc959fb7f704fba9009c0595467d074 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.090s
I20260709 13:21:42.419466 16223 raft_consensus.cc:2468] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 19.
I20260709 13:21:42.409053 21704 raft_consensus.cc:493] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:42.344699 22241 raft_consensus.cc:515] T d16dfad0c3b540e5b4b86492ddf87208 P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:42.416551 22195 raft_consensus.cc:2804] T 3a7ca61592e848ea9b0fb81a2ce33cdc P cb82cc7bd6db46d980c75237a1402860 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:21:42.522773 16223 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e4dc6b0541ca431ebc757beac00521f1"
dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
mode: GRACEFUL
new_leader_uuid: "90f3da57796b488c9087a2f523adf045"
 from {username='slave'} at 127.0.0.1:39144
I20260709 13:21:42.523337 22195 raft_consensus.cc:493] T 3a7ca61592e848ea9b0fb81a2ce33cdc P cb82cc7bd6db46d980c75237a1402860 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:42.523686 16223 raft_consensus.cc:606] T e4dc6b0541ca431ebc757beac00521f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 16 LEADER]: Received request to transfer leadership to TS 90f3da57796b488c9087a2f523adf045
I20260709 13:21:42.523839 22195 raft_consensus.cc:3060] T 3a7ca61592e848ea9b0fb81a2ce33cdc P cb82cc7bd6db46d980c75237a1402860 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:21:42.525444 15046 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
W20260709 13:21:42.526677 16265 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:42.527644 15046 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:21:42.529209 21930 leader_election.cc:290] T 7c2095a5f4ad49408dbc7f69c606120d P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:42.467068 16224 raft_consensus.cc:1275] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 13:21:42.536278 16145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18ad7e3af5504571a69d10c00750a01c" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:42.536717 16143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d16dfad0c3b540e5b4b86492ddf87208" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:42.336483 22197 raft_consensus.cc:515] T 18ad7e3af5504571a69d10c00750a01c P 647a343a542d4a139fe106c19095129b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } }
W20260709 13:21:42.519972 22329 log.cc:927] Time spent T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 0.333s	user 0.001s	sys 0.000s
I20260709 13:21:42.522893 22241 leader_election.cc:290] T d16dfad0c3b540e5b4b86492ddf87208 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
W20260709 13:21:42.557075 16265 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:42.485740 22243 raft_consensus.cc:697] T c9944d5e4b8c4109af9955c4f08c9095 P cb82cc7bd6db46d980c75237a1402860 [term 19 LEADER]: Becoming Leader. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:21:42.363399 22104 raft_consensus.cc:515] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:42.465423 22246 raft_consensus.cc:697] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b [term 20 LEADER]: Becoming Leader. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:21:42.505110 16148 raft_consensus.cc:1275] T af57996a8be84d398f0f023e8d08ef77 P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 18 index: 12. (term mismatch)
W20260709 13:21:42.490600 22299 log.cc:927] Time spent T ada5a004ba7c46819051e3ccec71ffdb P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.080s	user 0.001s	sys 0.000s
I20260709 13:21:42.511392 16222 raft_consensus.cc:1275] T 66f75124215f4ca6bef831e174a792d8 P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:21:42.470516 22145 raft_consensus.cc:515] T 5a959dbecc2648eb85d1cc550f5a3245 P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:42.521024 22111 leader_election.cc:290] T 18ad7e3af5504571a69d10c00750a01c P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:21:42.479239 16300 raft_consensus.cc:2468] T c4f65cdf00b748bb963ee6e81ef43b79 P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 18.
I20260709 13:21:42.547639 16142 raft_consensus.cc:2468] T ecc1e363230043fe9fef390f0b979f20 P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 18.
I20260709 13:21:42.548890 16147 raft_consensus.cc:1275] T 23f82224f4564fea93572e8206bad1fe P 647a343a542d4a139fe106c19095129b [term 14 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:21:42.580420 16294 raft_consensus.cc:3060] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:21:42.594220 16116 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.517s	user 0.008s	sys 0.029s
W20260709 13:21:42.636373 16116 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.559s	user 0.009s	sys 0.030s
I20260709 13:21:42.620256 22236 leader_election.cc:290] T af57996a8be84d398f0f023e8d08ef77 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
W20260709 13:21:42.636786 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 559920us
I20260709 13:21:42.533973 21704 raft_consensus.cc:515] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:21:42.640511 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 266581us
I20260709 13:21:42.641497 16790 leader_election.cc:304] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 647a343a542d4a139fe106c19095129b
W20260709 13:21:42.609573 16192 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 301499us
I20260709 13:21:42.615512 16148 pending_rounds.cc:85] T af57996a8be84d398f0f023e8d08ef77 P 647a343a542d4a139fe106c19095129b: Aborting all ops after (but not including) 11
W20260709 13:21:42.616320 16266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 303930us
I20260709 13:21:42.653586 16148 pending_rounds.cc:107] T af57996a8be84d398f0f023e8d08ef77 P 647a343a542d4a139fe106c19095129b: Aborting uncommitted NO_OP operation due to leader change: 17.12
I20260709 13:21:42.654037 16148 raft_consensus.cc:2889] T af57996a8be84d398f0f023e8d08ef77 P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:21:42.619375 16145 raft_consensus.cc:2468] T 18ad7e3af5504571a69d10c00750a01c P 647a343a542d4a139fe106c19095129b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 14.
I20260709 13:21:42.603449 22164 raft_consensus.cc:493] T 2794af5b99984efebdee564035841cb0 P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:21:42.602710 16221 raft_consensus.cc:2468] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 18.
I20260709 13:21:42.584812 22220 consensus_queue.cc:237] T 8c11074579b046d99e585c2818e18a2c P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:42.640040 22206 raft_consensus.cc:2749] T 8d4ab870a05347868956aa8b841b2981 P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:21:42.640865 22291 leader_election.cc:290] T 02838c86531d4f98bf58d1c22b8baf77 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:42.605755 22152 consensus_queue.cc:237] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:42.681092 16147 raft_consensus.cc:1275] T 829b4ad8c5a54eababeea1181b960d4b P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:21:42.686036 22029 consensus_queue.cc:1048] T 66f75124215f4ca6bef831e174a792d8 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.010s
W20260709 13:21:42.695870 22337 log.cc:927] Time spent T 8d4ab870a05347868956aa8b841b2981 P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.088s	user 0.001s	sys 0.000s
I20260709 13:21:42.697167 22031 consensus_queue.cc:1048] T 829b4ad8c5a54eababeea1181b960d4b P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:21:42.697953 22236 raft_consensus.cc:493] T b6f07842d3aa47858df5665874c86f3a P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:42.699347 22243 consensus_queue.cc:237] T c9944d5e4b8c4109af9955c4f08c9095 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:42.680256 16116 leader_election.cc:304] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
I20260709 13:21:42.685472 16226 raft_consensus.cc:1275] T 829b4ad8c5a54eababeea1181b960d4b P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:21:42.710268 21704 leader_election.cc:290] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:42.678725 22336 consensus_queue.cc:1048] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.033s
I20260709 13:21:42.711886 22197 leader_election.cc:290] T 18ad7e3af5504571a69d10c00750a01c P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:21:42.677649 22037 consensus_queue.cc:1048] T d16dfad0c3b540e5b4b86492ddf87208 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.009s
I20260709 13:21:42.679471 22339 raft_consensus.cc:2804] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:21:42.728109 22195 raft_consensus.cc:515] T 3a7ca61592e848ea9b0fb81a2ce33cdc P cb82cc7bd6db46d980c75237a1402860 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:42.678259 22225 consensus_queue.cc:1048] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.048s
I20260709 13:21:42.733570 16150 raft_consensus.cc:1275] T ecc1e363230043fe9fef390f0b979f20 P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 13:21:42.714751 22291 raft_consensus.cc:493] T 84f2da98864b4425bf7c0055951fdd4b P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:21:42.736737 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7ca61592e848ea9b0fb81a2ce33cdc" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:21:42.698733 22236 raft_consensus.cc:515] T b6f07842d3aa47858df5665874c86f3a P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:42.742237 16224 raft_consensus.cc:3060] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:21:42.739375 16790 leader_election.cc:304] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860; no voters: 90f3da57796b488c9087a2f523adf045
I20260709 13:21:42.661970 22164 raft_consensus.cc:515] T 2794af5b99984efebdee564035841cb0 P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:42.738442 16116 leader_election.cc:304] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:21:42.752194 22284 raft_consensus.cc:2804] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:21:42.752729 22284 raft_consensus.cc:493] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:42.753113 22284 raft_consensus.cc:3060] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:21:42.752210 22205 consensus_queue.cc:1048] T 829b4ad8c5a54eababeea1181b960d4b P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.002s
I20260709 13:21:42.721947 22339 raft_consensus.cc:493] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:21:42.758682 16142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c61a39fd4f84424b129c1a6f9ba404d" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
W20260709 13:21:42.760005 16789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 386387us
I20260709 13:21:42.733840 22195 leader_election.cc:290] T 3a7ca61592e848ea9b0fb81a2ce33cdc P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 19 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:42.741042 22197 raft_consensus.cc:2804] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:21:42.772523 16294 raft_consensus.cc:2468] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 19.
I20260709 13:21:42.777855 22029 raft_consensus.cc:493] T 434bff3082d141838068f323bdb0d1ed P cb82cc7bd6db46d980c75237a1402860 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:21:42.777628 22149 raft_consensus.cc:515] T 08c74490c40243a88e62d1760b5cd814 P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:42.778445 22029 raft_consensus.cc:515] T 434bff3082d141838068f323bdb0d1ed P cb82cc7bd6db46d980c75237a1402860 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:42.744251 22211 leader_election.cc:290] T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
W20260709 13:21:42.757426 16115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 422726us
I20260709 13:21:42.751139 16116 leader_election.cc:304] T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
I20260709 13:21:42.787938 16224 raft_consensus.cc:2468] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 19.
W20260709 13:21:42.792697 16789 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:42.757728 21704 raft_consensus.cc:2804] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Leader election won for term 19
W20260709 13:21:42.796191 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 45264.8us
W20260709 13:21:42.759153 22201 consensus_meta.cc:287] T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860: Time spent flushing consensus metadata: real 0.592s	user 0.006s	sys 0.005s
I20260709 13:21:42.763504 22339 raft_consensus.cc:3060] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:21:42.745947 22291 raft_consensus.cc:515] T 84f2da98864b4425bf7c0055951fdd4b P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:42.746112 22281 consensus_queue.cc:1048] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.047s
I20260709 13:21:42.793906 22197 raft_consensus.cc:697] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b [term 16 LEADER]: Becoming Leader. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:21:42.803592 22027 raft_consensus.cc:2804] T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:21:42.795846 22195 raft_consensus.cc:493] T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:21:42.806291 22215 raft_consensus.cc:3060] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:21:42.849571 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 213710us
I20260709 13:21:42.739820 22246 consensus_queue.cc:237] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:42.862593 16190 leader_election.cc:304] T d2ab0ac8347641e6955e9bce7d90c1a2 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860
W20260709 13:21:42.851104 16115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 45258.9us
I20260709 13:21:42.872325 16115 leader_election.cc:304] T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac
I20260709 13:21:42.825620 22281 consensus_queue.cc:1243] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Peer 90f3da57796b488c9087a2f523adf045 log is divergent from this leader: its last log entry 20.13 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 12
I20260709 13:21:42.876222 16190 leader_election.cc:304] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:21:42.822990 21704 raft_consensus.cc:697] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [term 19 LEADER]: Becoming Leader. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:21:42.890323 16190 leader_election.cc:304] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860
I20260709 13:21:42.882494 22027 raft_consensus.cc:697] T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b [term 18 LEADER]: Becoming Leader. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:21:42.859061 22145 leader_election.cc:290] T 5a959dbecc2648eb85d1cc550f5a3245 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:42.883745 22338 raft_consensus.cc:2804] T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:21:42.884938 22119 consensus_queue.cc:1048] T 7c2095a5f4ad49408dbc7f69c606120d P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:21:42.885349 22029 leader_election.cc:290] T 434bff3082d141838068f323bdb0d1ed P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:42.885385 16115 leader_election.cc:304] T 26354361e9b044cea1c2b86c4882fe92 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860
I20260709 13:21:42.886073 22074 consensus_queue.cc:1048] T d16dfad0c3b540e5b4b86492ddf87208 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:21:42.823863 22235 raft_consensus.cc:493] T 9e65676b21b748728ad39668cfe722ec P cb82cc7bd6db46d980c75237a1402860 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:21:42.895624 16192 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 243778us
I20260709 13:21:42.902698 16223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8158bbd17b994d2eb96f560bd0f1b88d" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:42.852058 22351 raft_consensus.cc:493] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
W20260709 13:21:42.949971 16115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 64675.3us
I20260709 13:21:42.951390 16115 leader_election.cc:304] T 68a6838d82ef4cac83507f906139d90f P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac
I20260709 13:21:42.954043 16115 leader_election.cc:304] T 1df93779209748b9860f6d893dd3de53 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac
I20260709 13:21:42.908100 22347 raft_consensus.cc:2749] T d2ab0ac8347641e6955e9bce7d90c1a2 P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:21:42.932420 22338 raft_consensus.cc:493] T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:42.869958 22195 raft_consensus.cc:515] T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:42.931964 22149 leader_election.cc:290] T 08c74490c40243a88e62d1760b5cd814 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 15 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:42.821050 22201 raft_consensus.cc:515] T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:42.980479 22356 raft_consensus.cc:2749] T 26354361e9b044cea1c2b86c4882fe92 P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:21:42.936220 22145 raft_consensus.cc:2749] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:21:42.941984 22029 raft_consensus.cc:493] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P cb82cc7bd6db46d980c75237a1402860 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
W20260709 13:21:42.981416 22343 log.cc:927] Time spent T 00000000000000000000000000000000 P 667a34b27a39474ebefc1af3cf5b4271: Append to log took a long time: real 0.152s	user 0.003s	sys 0.000s
I20260709 13:21:42.983911 22037 raft_consensus.cc:2804] T 1df93779209748b9860f6d893dd3de53 P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:21:42.981041 22088 raft_consensus.cc:2804] T 68a6838d82ef4cac83507f906139d90f P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:21:42.983861 22359 raft_consensus.cc:2749] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:21:42.986897 22338 raft_consensus.cc:3060] T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:21:42.960083 22235 raft_consensus.cc:515] T 9e65676b21b748728ad39668cfe722ec P cb82cc7bd6db46d980c75237a1402860 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:43.037468 16821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af57996a8be84d398f0f023e8d08ef77" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:43.037729 16142 raft_consensus.cc:1275] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 13:21:42.995422 22164 leader_election.cc:290] T 2794af5b99984efebdee564035841cb0 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:43.015045 21623 raft_consensus.cc:3060] T b87147731a68400e8cce9b879daa7c5d P cb82cc7bd6db46d980c75237a1402860 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:21:43.022599 22088 raft_consensus.cc:493] T 68a6838d82ef4cac83507f906139d90f P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:21:43.048899 16192 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 67209.4us
I20260709 13:21:43.052174 16223 raft_consensus.cc:2468] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 15.
I20260709 13:21:43.053191 16818 raft_consensus.cc:3060] T 806d7690c2b644959152074bb2cab0a7 P cb82cc7bd6db46d980c75237a1402860 [term 16 LEARNER]: Advancing to term 17
I20260709 13:21:43.030993 22037 raft_consensus.cc:697] T 1df93779209748b9860f6d893dd3de53 P 647a343a542d4a139fe106c19095129b [term 16 LEADER]: Becoming Leader. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:21:42.983433 22197 consensus_queue.cc:237] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:21:43.067938 16190 leader_election.cc:336] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 17 election: RPC error from VoteRequest() call to peer 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389): Timed out: RequestConsensusVote RPC to 127.14.177.131:41389 timed out after 1.739s (SENT)
I20260709 13:21:43.023293 22029 raft_consensus.cc:515] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P cb82cc7bd6db46d980c75237a1402860 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:43.054589 22088 raft_consensus.cc:3060] T 68a6838d82ef4cac83507f906139d90f P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:21:42.980561 22351 raft_consensus.cc:515] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:43.022073 22027 consensus_queue.cc:237] T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:43.080788 22195 leader_election.cc:290] T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:43.054564 22236 leader_election.cc:290] T b6f07842d3aa47858df5665874c86f3a P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:43.060866 22225 raft_consensus.cc:493] T b87147731a68400e8cce9b879daa7c5d P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:21:43.099478 22344 log.cc:927] Time spent T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 0.253s	user 0.001s	sys 0.001s
W20260709 13:21:43.106189 22360 log.cc:927] Time spent T ee9ed67006144797b1de6ea8c2013a68 P cb82cc7bd6db46d980c75237a1402860: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
W20260709 13:21:43.106915 22354 log.cc:927] Time spent T 8cc959fb7f704fba9009c0595467d074 P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 0.115s	user 0.000s	sys 0.001s
I20260709 13:21:43.107131 22235 leader_election.cc:290] T 9e65676b21b748728ad39668cfe722ec P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
W20260709 13:21:43.108495 22335 log.cc:927] Time spent T 3884f864a96d428785895e5463e5728f P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.694s	user 0.005s	sys 0.000s
I20260709 13:21:43.108392 22201 leader_election.cc:290] T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 19 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:21:43.109110 22225 raft_consensus.cc:515] T b87147731a68400e8cce9b879daa7c5d P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:43.111363 22336 raft_consensus.cc:493] T 74ca8140f35c4ae29e8a664f6bdfc829 P cb82cc7bd6db46d980c75237a1402860 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:43.070665 22145 raft_consensus.cc:493] T ee9ed67006144797b1de6ea8c2013a68 P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:21:43.111881 22342 log.cc:927] Time spent T 8d4ab870a05347868956aa8b841b2981 P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 0.220s	user 0.000s	sys 0.001s
I20260709 13:21:43.112782 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02838c86531d4f98bf58d1c22b8baf77" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 18 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:43.117060 16818 raft_consensus.cc:1275] T 806d7690c2b644959152074bb2cab0a7 P cb82cc7bd6db46d980c75237a1402860 [term 17 LEARNER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 13:21:43.121687 16147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c2095a5f4ad49408dbc7f69c606120d" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 14 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:43.084079 22152 raft_consensus.cc:493] T d16dfad0c3b540e5b4b86492ddf87208 P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:43.044986 21704 consensus_queue.cc:237] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 19, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:43.093580 22291 leader_election.cc:290] T 84f2da98864b4425bf7c0055951fdd4b P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:43.172200 22281 raft_consensus.cc:493] T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:21:43.172814 22281 raft_consensus.cc:515] T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:21:43.176532 22358 log.cc:927] Time spent T 8158bbd17b994d2eb96f560bd0f1b88d P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
I20260709 13:21:43.176780 16300 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48674 (request call id 3181) took 1059 ms. Trace:
I20260709 13:21:43.177153 16300 rpcz_store.cc:276] 0709 13:21:42.117367 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:42.118993 (+  1626us) service_pool.cc:224] Handling call
0709 13:21:43.176598 (+1057605us) spinlock_profiling.cc:243] Waited 208 ms on lock 0x7b54002e1654. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:21:43.176704 (+   106us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":478096256}
W20260709 13:21:43.178371 22366 log.cc:927] Time spent T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
I20260709 13:21:43.178555 16143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5d59407eb545b895d8d922e0dd69fd" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:21:43.157399 22145 raft_consensus.cc:515] T ee9ed67006144797b1de6ea8c2013a68 P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:43.180269 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d16dfad0c3b540e5b4b86492ddf87208" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:43.194342 16818 raft_consensus.cc:1240] T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860 [term 19 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 18. Current term is 19. Ops: []
I20260709 13:21:43.184811 16816 raft_consensus.cc:2468] T b87147731a68400e8cce9b879daa7c5d P cb82cc7bd6db46d980c75237a1402860 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 17.
I20260709 13:21:43.190371 16147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e65676b21b748728ad39668cfe722ec" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:43.180100 16143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434bff3082d141838068f323bdb0d1ed" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:43.183008 16220 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57604 (request call id 3121) took 1023 ms. Trace:
I20260709 13:21:43.198477 16143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c74490c40243a88e62d1760b5cd814" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:21:43.198598 16147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f2da98864b4425bf7c0055951fdd4b" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:43.162354 22152 raft_consensus.cc:515] T d16dfad0c3b540e5b4b86492ddf87208 P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:21:43.203882 22363 log.cc:927] Time spent T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 0.054s	user 0.000s	sys 0.001s
I20260709 13:21:43.205353 16221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87ac0d1f7e8a419f9bcbc04aee0e067e" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:43.205552 16225 raft_consensus.cc:2468] T d16dfad0c3b540e5b4b86492ddf87208 P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 17.
I20260709 13:21:43.206794 16221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6f07842d3aa47858df5665874c86f3a" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:43.207427 22057 raft_consensus.cc:493] T c2f5f89186204242bf377f7c42b2e2da P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:21:43.207958 22057 raft_consensus.cc:515] T c2f5f89186204242bf377f7c42b2e2da P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:43.208503 16219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:43.157692 22037 consensus_queue.cc:237] T 1df93779209748b9860f6d893dd3de53 P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:43.209131 16219 raft_consensus.cc:2468] T ecc1e363230043fe9fef390f0b979f20 P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 18.
I20260709 13:21:43.209977 22057 leader_election.cc:290] T c2f5f89186204242bf377f7c42b2e2da P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:43.210050 16219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af57996a8be84d398f0f023e8d08ef77" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:43.157855 22336 raft_consensus.cc:515] T 74ca8140f35c4ae29e8a664f6bdfc829 P cb82cc7bd6db46d980c75237a1402860 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:43.213528 22336 leader_election.cc:290] T 74ca8140f35c4ae29e8a664f6bdfc829 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:43.191036 16144 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46224 (request call id 983) took 1034 ms. Trace:
I20260709 13:21:43.214154 16144 rpcz_store.cc:276] 0709 13:21:42.156720 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:42.156917 (+   197us) service_pool.cc:224] Handling call
0709 13:21:42.157318 (+   401us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:21:42.157720 (+   402us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:21:42.157761 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:21:42.158304 (+   543us) log.cc:844] Serialized 26 byte log entry
0709 13:21:42.659799 (+501495us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:21:42.659886 (+    87us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:42.659960 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:21:43.190757 (+530797us) raft_consensus.cc:1685] finished
0709 13:21:43.190786 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:21:43.190966 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":9266,"raft.run_cpu_time_us":115,"raft.run_wall_time_us":116,"thread_start_us":382968,"threads_started":1,"wal-append.queue_time_us":578501}
I20260709 13:21:43.216005 16144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a959dbecc2648eb85d1cc550f5a3245" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 16 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:43.215504 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8158bbd17b994d2eb96f560bd0f1b88d" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:43.216482 16219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18ad7e3af5504571a69d10c00750a01c" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:43.217197 16219 raft_consensus.cc:2468] T 18ad7e3af5504571a69d10c00750a01c P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 14.
I20260709 13:21:43.219761 16219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a378d7f796f94615b3b347119a9880fd" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:43.219986 16222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada5a004ba7c46819051e3ccec71ffdb" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:43.220806 16222 raft_consensus.cc:2468] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 17.
I20260709 13:21:43.220830 16219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cc959fb7f704fba9009c0595467d074" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:43.222033 16222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18ad7e3af5504571a69d10c00750a01c" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:43.222795 16222 raft_consensus.cc:2468] T 18ad7e3af5504571a69d10c00750a01c P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 14.
I20260709 13:21:43.207472 16221 raft_consensus.cc:2468] T b6f07842d3aa47858df5665874c86f3a P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 16.
I20260709 13:21:43.206804 16151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c61a39fd4f84424b129c1a6f9ba404d" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
W20260709 13:21:43.357620 16266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 704211us
I20260709 13:21:43.197939 16220 rpcz_store.cc:276] 0709 13:21:42.159706 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:42.159839 (+   133us) service_pool.cc:224] Handling call
0709 13:21:42.160253 (+   414us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:21:42.293934 (+133681us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:21:42.293978 (+    44us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:21:42.358738 (+ 64760us) log.cc:844] Serialized 52 byte log entry
0709 13:21:42.808988 (+450250us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:21:42.945990 (+137002us) log.cc:844] Serialized 14 byte log entry
0709 13:21:43.025194 (+ 79204us) log.cc:844] Serialized 14 byte log entry
0709 13:21:43.182452 (+157258us) spinlock_profiling.cc:243] Waited 2.95 ms on lock 0x7b1800354100. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b693ade1 00007f69b69475b1 00007f69b07074bc 00007f69ada5548b 00007f69b68afab9 00007f69ada7169b 00007f69ada6f93f 00007f69ada6f455 00007f69b0b57060 00007f69b0b69716 00007f69b0b7a287
0709 13:21:43.182585 (+   133us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:43.182660 (+    75us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:21:43.182784 (+   124us) raft_consensus.cc:1685] finished
0709 13:21:43.182800 (+    16us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:21:43.182947 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":205890,"prepare.queue_time_us":56661,"prepare.run_cpu_time_us":74,"prepare.run_wall_time_us":75,"raft.queue_time_us":250949,"raft.run_cpu_time_us":364,"raft.run_wall_time_us":21320,"spinlock_wait_cycles":6776704,"thread_start_us":490739,"threads_started":3,"wal-append.queue_time_us":572689}
I20260709 13:21:43.174986 22281 leader_election.cc:290] T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
W20260709 13:21:43.184214 22352 log.cc:927] Time spent T 5a96373bb0e14dce8a266b2fc6c58507 P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 0.408s	user 0.001s	sys 0.000s
I20260709 13:21:43.358757 16116 leader_election.cc:304] T 18ad7e3af5504571a69d10c00750a01c P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
I20260709 13:21:43.370469 16191 leader_election.cc:304] T e5ad4eb281564936a59f41e387a47156 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:21:43.379949 22346 raft_consensus.cc:493] T 9e65676b21b748728ad39668cfe722ec P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:43.380451 22140 raft_consensus.cc:493] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:21:43.391546 16143 raft_consensus.cc:2468] T 08c74490c40243a88e62d1760b5cd814 P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 15.
I20260709 13:21:43.385794 16116 leader_election.cc:304] T b6f07842d3aa47858df5665874c86f3a P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
I20260709 13:21:43.395056 16191 leader_election.cc:304] T 68a6838d82ef4cac83507f906139d90f P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 90f3da57796b488c9087a2f523adf045
I20260709 13:21:43.399412 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87ac0d1f7e8a419f9bcbc04aee0e067e" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:43.400238 16299 raft_consensus.cc:2468] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 17.
I20260709 13:21:43.179940 22351 leader_election.cc:290] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:43.191193 22225 leader_election.cc:290] T b87147731a68400e8cce9b879daa7c5d P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:43.409504 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18ad7e3af5504571a69d10c00750a01c" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:43.425639 16299 raft_consensus.cc:2468] T 18ad7e3af5504571a69d10c00750a01c P 90f3da57796b488c9087a2f523adf045 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 14.
W20260709 13:21:43.398453 22215 consensus_meta.cc:287] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045: Time spent flushing consensus metadata: real 0.529s	user 0.008s	sys 0.003s
I20260709 13:21:43.185870 22029 leader_election.cc:290] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:43.356949 22321 raft_consensus.cc:493] T 66ae176961b74813a58bc15c9c4c6f2a P 90f3da57796b488c9087a2f523adf045 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:43.394188 22211 raft_consensus.cc:493] T 8a878999c0024d2abe517484180290fc P 90f3da57796b488c9087a2f523adf045 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:21:43.205384 22152 leader_election.cc:290] T d16dfad0c3b540e5b4b86492ddf87208 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:43.185715 22182 leader_election.cc:290] T ecc1e363230043fe9fef390f0b979f20 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
W20260709 13:21:43.420608 16266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 60740us
I20260709 13:21:43.444947 16266 leader_election.cc:304] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
I20260709 13:21:43.412317 16191 leader_election.cc:304] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
I20260709 13:21:43.427275 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:43.447022 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6f07842d3aa47858df5665874c86f3a" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:43.448374 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2794af5b99984efebdee564035841cb0" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:43.449070 16293 raft_consensus.cc:2468] T 2794af5b99984efebdee564035841cb0 P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 13.
I20260709 13:21:43.181387 22291 consensus_queue.cc:1048] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.229s
I20260709 13:21:43.399027 22356 raft_consensus.cc:2804] T 18ad7e3af5504571a69d10c00750a01c P 647a343a542d4a139fe106c19095129b [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:21:43.232715 22350 raft_consensus.cc:493] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:43.406520 22380 raft_consensus.cc:2749] T e5ad4eb281564936a59f41e387a47156 P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:21:43.431399 22197 raft_consensus.cc:2804] T b6f07842d3aa47858df5665874c86f3a P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:21:43.443761 16296 raft_consensus.cc:2410] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac for term 17 because replica has last-logged OpId of term: 16 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 10.
I20260709 13:21:43.162689 22284 raft_consensus.cc:515] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:43.455986 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5d59407eb545b895d8d922e0dd69fd" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:21:43.464069 16819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2794af5b99984efebdee564035841cb0" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:43.462805 16820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:43.431667 22347 raft_consensus.cc:2804] T 68a6838d82ef4cac83507f906139d90f P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:21:43.441847 16292 raft_consensus.cc:1275] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:21:43.475113 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a378d7f796f94615b3b347119a9880fd" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:43.482373 16818 raft_consensus.cc:1275] T 23f82224f4564fea93572e8206bad1fe P cb82cc7bd6db46d980c75237a1402860 [term 14 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:21:43.471139 16117 leader_election.cc:304] T af57996a8be84d398f0f023e8d08ef77 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860
I20260709 13:21:43.483852 16150 raft_consensus.cc:1240] T 1df93779209748b9860f6d893dd3de53 P 647a343a542d4a139fe106c19095129b [term 16 LEADER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 15. Current term is 16. Ops: []
I20260709 13:21:43.484226 16824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87147731a68400e8cce9b879daa7c5d" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:43.480460 16218 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57600 (request call id 1829) took 1356 ms. Trace:
I20260709 13:21:43.485350 16218 rpcz_store.cc:276] 0709 13:21:42.124056 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:42.126266 (+  2210us) service_pool.cc:224] Handling call
0709 13:21:42.137361 (+ 11095us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:21:42.293311 (+155950us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:21:42.293344 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:21:42.319691 (+ 26347us) log.cc:844] Serialized 26 byte log entry
0709 13:21:42.655766 (+336075us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:21:42.655812 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:42.655903 (+    91us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:21:43.183569 (+527666us) spinlock_profiling.cc:243] Waited 5.76 ms on lock 0x7b4400022b4c. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69ad8fadb1 00007f69ad998c85 00007f69ad9996a2 00007f69b070b1db 00007f69b0706aaf 00007f69b06fa86e 00007f69b06f9bb6 00007f69b06f97d8 00007f69b0703221 00007f69b07031d9 00007f69b0703169
0709 13:21:43.469444 (+285875us) raft_consensus.cc:1685] finished
0709 13:21:43.469464 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:21:43.480390 (+ 10926us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":155571,"raft.queue_time_us":156147,"spinlock_wait_cycles":29210752,"thread_start_us":272105,"threads_started":1,"wal-append.queue_time_us":416182}
I20260709 13:21:43.453866 22356 raft_consensus.cc:493] T 18ad7e3af5504571a69d10c00750a01c P 647a343a542d4a139fe106c19095129b [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:43.459815 22215 raft_consensus.cc:493] T e5ad4eb281564936a59f41e387a47156 P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:21:43.413523 22140 raft_consensus.cc:515] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:43.435495 22106 consensus_queue.cc:1048] T 3884f864a96d428785895e5463e5728f P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.046s
I20260709 13:21:43.465538 22377 raft_consensus.cc:2749] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:21:43.499315 16820 raft_consensus.cc:2393] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 19: Already voted for candidate cb82cc7bd6db46d980c75237a1402860 in this term.
I20260709 13:21:43.432013 22064 consensus_queue.cc:1048] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.047s
I20260709 13:21:43.478185 22197 raft_consensus.cc:493] T b6f07842d3aa47858df5665874c86f3a P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:43.452620 22321 raft_consensus.cc:515] T 66ae176961b74813a58bc15c9c4c6f2a P 90f3da57796b488c9087a2f523adf045 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:43.442166 22242 consensus_queue.cc:1048] T ecc1e363230043fe9fef390f0b979f20 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.051s
I20260709 13:21:43.476492 22079 raft_consensus.cc:2749] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:21:43.510094 16218 raft_consensus.cc:1275] T 5a959dbecc2648eb85d1cc550f5a3245 P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 13:21:43.402784 22346 raft_consensus.cc:515] T 9e65676b21b748728ad39668cfe722ec P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:43.493220 22236 raft_consensus.cc:2749] T af57996a8be84d398f0f023e8d08ef77 P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:21:43.481765 22347 raft_consensus.cc:493] T 68a6838d82ef4cac83507f906139d90f P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: Starting leader election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:21:43.513537 16292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87147731a68400e8cce9b879daa7c5d" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:43.462590 22211 raft_consensus.cc:515] T 8a878999c0024d2abe517484180290fc P 90f3da57796b488c9087a2f523adf045 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:43.479452 22152 consensus_queue.cc:1048] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.036s
I20260709 13:21:43.457458 22334 consensus_queue.cc:1048] T 23f82224f4564fea93572e8206bad1fe P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.064s
I20260709 13:21:43.500063 22356 raft_consensus.cc:3060] T 18ad7e3af5504571a69d10c00750a01c P 647a343a542d4a139fe106c19095129b [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:21:43.515558 22197 raft_consensus.cc:3060] T b6f07842d3aa47858df5665874c86f3a P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:21:43.531347 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 71806.5us
I20260709 13:21:43.532027 16218 raft_consensus.cc:1240] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 18. Current term is 19. Ops: []
W20260709 13:21:43.531401 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 798676us
I20260709 13:21:43.468663 22350 raft_consensus.cc:515] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:43.549698 16191 leader_election.cc:304] T 08c74490c40243a88e62d1760b5cd814 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
I20260709 13:21:43.530185 22347 raft_consensus.cc:3060] T 68a6838d82ef4cac83507f906139d90f P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:21:43.508327 22215 raft_consensus.cc:515] T e5ad4eb281564936a59f41e387a47156 P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:21:43.551532 22339 consensus_meta.cc:287] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860: Time spent flushing consensus metadata: real 0.723s	user 0.012s	sys 0.006s
I20260709 13:21:43.563335 22336 raft_consensus.cc:493] T 19baa7bc367844c288bca8f81439c413 P cb82cc7bd6db46d980c75237a1402860 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:43.569149 22111 raft_consensus.cc:493] T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:43.506392 22338 raft_consensus.cc:515] T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:43.567011 22361 raft_consensus.cc:493] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:43.568032 22377 raft_consensus.cc:2804] T 08c74490c40243a88e62d1760b5cd814 P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:21:43.652770 16817 raft_consensus.cc:1275] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860 [term 15 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:21:43.566807 22079 consensus_queue.cc:1048] T ecc1e363230043fe9fef390f0b979f20 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.101s
I20260709 13:21:43.659636 22145 leader_election.cc:290] T ee9ed67006144797b1de6ea8c2013a68 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:43.702626 16142 raft_consensus.cc:1240] T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 16. Current term is 17. Ops: []
I20260709 13:21:43.661728 22339 raft_consensus.cc:515] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:21:43.695530 22374 log.cc:927] Time spent T c9944d5e4b8c4109af9955c4f08c9095 P cb82cc7bd6db46d980c75237a1402860: Append to log took a long time: real 0.263s	user 0.001s	sys 0.000s
I20260709 13:21:43.694965 22377 raft_consensus.cc:697] T 08c74490c40243a88e62d1760b5cd814 P 1ca2725d290e47d68b1a4b609c9816ac [term 15 LEADER]: Becoming Leader. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:21:43.683856 22111 raft_consensus.cc:515] T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:43.668447 22336 raft_consensus.cc:515] T 19baa7bc367844c288bca8f81439c413 P cb82cc7bd6db46d980c75237a1402860 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:43.700376 22392 raft_consensus.cc:493] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:43.689378 22361 raft_consensus.cc:515] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:43.734617 16825 raft_consensus.cc:1240] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [term 22 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 21. Current term is 22. Ops: []
W20260709 13:21:43.737337 22387 log.cc:927] Time spent T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 0.115s	user 0.000s	sys 0.001s
W20260709 13:21:43.739627 22394 log.cc:927] Time spent T af57996a8be84d398f0f023e8d08ef77 P cb82cc7bd6db46d980c75237a1402860: Append to log took a long time: real 0.155s	user 0.000s	sys 0.001s
W20260709 13:21:43.743500 22381 log.cc:927] Time spent T af57996a8be84d398f0f023e8d08ef77 P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 0.221s	user 0.000s	sys 0.001s
I20260709 13:21:43.744568 22140 leader_election.cc:290] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:43.746387 22215 leader_election.cc:290] T e5ad4eb281564936a59f41e387a47156 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:21:43.748003 22377 consensus_queue.cc:237] T 08c74490c40243a88e62d1760b5cd814 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:43.751379 22399 raft_consensus.cc:493] T d2ab0ac8347641e6955e9bce7d90c1a2 P 90f3da57796b488c9087a2f523adf045 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:21:43.752110 22399 raft_consensus.cc:515] T d2ab0ac8347641e6955e9bce7d90c1a2 P 90f3da57796b488c9087a2f523adf045 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:43.754977 22321 leader_election.cc:290] T 66ae176961b74813a58bc15c9c4c6f2a P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:43.714824 22334 raft_consensus.cc:493] T cdbd78acb4cf46298e0de8ba0d130e19 P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:21:43.759357 22140 raft_consensus.cc:493] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:43.763696 22140 raft_consensus.cc:515] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:21:43.724287 22379 log.cc:927] Time spent T 8cc959fb7f704fba9009c0595467d074 P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 0.239s	user 0.000s	sys 0.001s
I20260709 13:21:43.766656 22140 leader_election.cc:290] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:43.757827 22361 leader_election.cc:290] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:43.760613 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434bff3082d141838068f323bdb0d1ed" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:43.768421 22284 leader_election.cc:290] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 19 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:43.728811 22395 raft_consensus.cc:493] T c9944d5e4b8c4109af9955c4f08c9095 P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:21:43.769309 22088 consensus_meta.cc:287] T 68a6838d82ef4cac83507f906139d90f P 647a343a542d4a139fe106c19095129b: Time spent flushing consensus metadata: real 0.667s	user 0.005s	sys 0.004s
I20260709 13:21:43.769408 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e4bfb9090c8460eb14fa7e0d818e7f1" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:43.769846 22088 raft_consensus.cc:515] T 68a6838d82ef4cac83507f906139d90f P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:43.769876 22392 raft_consensus.cc:515] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } }
I20260709 13:21:43.771129 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e65676b21b748728ad39668cfe722ec" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:43.771871 16298 raft_consensus.cc:2468] T 9e65676b21b748728ad39668cfe722ec P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 16.
I20260709 13:21:43.772542 22392 leader_election.cc:290] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:21:43.772827 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66ae176961b74813a58bc15c9c4c6f2a" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:43.773622 16298 raft_consensus.cc:2468] T 66ae176961b74813a58bc15c9c4c6f2a P 90f3da57796b488c9087a2f523adf045 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 21.
I20260709 13:21:43.774576 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a378d7f796f94615b3b347119a9880fd" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:43.763585 22334 raft_consensus.cc:515] T cdbd78acb4cf46298e0de8ba0d130e19 P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:43.755991 22399 leader_election.cc:290] T d2ab0ac8347641e6955e9bce7d90c1a2 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:43.776322 16298 raft_consensus.cc:2468] T a378d7f796f94615b3b347119a9880fd P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 18.
I20260709 13:21:43.779088 22334 leader_election.cc:290] T cdbd78acb4cf46298e0de8ba0d130e19 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:43.779209 22338 leader_election.cc:290] T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 17 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:43.767760 22346 leader_election.cc:290] T 9e65676b21b748728ad39668cfe722ec P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
W20260709 13:21:43.780189 22345 log.cc:927] Time spent T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.921s	user 0.000s	sys 0.002s
I20260709 13:21:43.780547 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7ca61592e848ea9b0fb81a2ce33cdc" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:21:43.780534 22399 raft_consensus.cc:493] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:21:43.782367 16291 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48684 (request call id 2983) took 1810 ms. Trace:
I20260709 13:21:43.785138 16143 raft_consensus.cc:1275] T ee9ed67006144797b1de6ea8c2013a68 P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 13:21:43.785710 16221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66ae176961b74813a58bc15c9c4c6f2a" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:43.786152 16147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:21:43.787766 16790 leader_election.cc:304] T 3a7ca61592e848ea9b0fb81a2ce33cdc P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:21:43.787074 16789 leader_election.cc:304] T 9e65676b21b748728ad39668cfe722ec P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:21:43.820456 16151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e9b2bf7011f4ae688a79d7bb5a447b4" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:43.821589 16221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e4bfb9090c8460eb14fa7e0d818e7f1" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:43.819345 16147 raft_consensus.cc:3060] T ecc1e363230043fe9fef390f0b979f20 P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:21:43.823495 16221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74ca8140f35c4ae29e8a664f6bdfc829" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 18 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:43.824295 16221 raft_consensus.cc:2410] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 for term 18 because replica has last-logged OpId of term: 15 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 11.
I20260709 13:21:43.825129 16790 leader_election.cc:304] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
W20260709 13:21:43.826547 16816 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49892 (request call id 894) took 1869 ms (client timeout 1740 ms). Trace:
I20260709 13:21:43.819550 16291 rpcz_store.cc:276] 0709 13:21:41.972105 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:41.975830 (+  3725us) service_pool.cc:224] Handling call
0709 13:21:41.976252 (+   422us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:21:41.976747 (+   495us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 13:21:42.182115 (+205368us) log.cc:844] Serialized 14 byte log entry
0709 13:21:42.640800 (+458685us) log.cc:844] Serialized 14 byte log entry
0709 13:21:43.357108 (+716308us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:21:43.379386 (+ 22278us) log.cc:844] Serialized 26 byte log entry
0709 13:21:43.379537 (+   151us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:21:43.379574 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:43.379649 (+    75us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:21:43.779539 (+399890us) spinlock_profiling.cc:243] Waited 8.94 ms on lock 0x7b4400022b4c. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69ad8fadb1 00007f69ad998c85 00007f69ad9996a2 00007f69b070b1db 00007f69b0706aaf 00007f69b06fa86e 00007f69b06f9bb6 00007f69b06f97d8 00007f69b0703221 00007f69b07031d9 00007f69b0703169
0709 13:21:43.782104 (+  2565us) raft_consensus.cc:1685] finished
0709 13:21:43.782133 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:21:43.782298 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":716075,"prepare.queue_time_us":35127,"prepare.run_cpu_time_us":91,"prepare.run_wall_time_us":90,"raft.queue_time_us":779368,"raft.run_cpu_time_us":288,"raft.run_wall_time_us":18721,"spinlock_wait_cycles":20566400,"thread_start_us":552747,"threads_started":2,"wal-append.queue_time_us":546147}
W20260709 13:21:43.827039 16816 rpcz_store.cc:269] 0709 13:21:41.956893 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:41.957016 (+   123us) service_pool.cc:224] Handling call
0709 13:21:43.826373 (+1869357us) spinlock_profiling.cc:243] Waited 1.15 s on lock 0x7b54004e0cd4. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:21:43.826466 (+    93us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2650788224}
I20260709 13:21:43.848133 16221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:21:43.861510 16291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e9b2bf7011f4ae688a79d7bb5a447b4" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:43.862284 16291 raft_consensus.cc:2468] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 15.
I20260709 13:21:43.863500 16291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74ca8140f35c4ae29e8a664f6bdfc829" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 18 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:43.864396 16291 raft_consensus.cc:2410] T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 for term 18 because replica has last-logged OpId of term: 16 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 11.
I20260709 13:21:43.767786 22350 leader_election.cc:290] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:43.848860 16221 raft_consensus.cc:3060] T ecc1e363230043fe9fef390f0b979f20 P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:21:43.794822 22284 raft_consensus.cc:493] T 23f82224f4564fea93572e8206bad1fe P cb82cc7bd6db46d980c75237a1402860 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:43.746428 22111 leader_election.cc:290] T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:43.876894 16265 leader_election.cc:304] T e4dc6b0541ca431ebc757beac00521f1 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:21:43.809245 22334 raft_consensus.cc:493] T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:21:43.881429 16291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c74490c40243a88e62d1760b5cd814" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:21:43.823705 22182 raft_consensus.cc:493] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:43.833032 21623 raft_consensus.cc:2804] T 3a7ca61592e848ea9b0fb81a2ce33cdc P cb82cc7bd6db46d980c75237a1402860 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:21:43.906595 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:21:43.862061 22031 raft_consensus.cc:2749] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:21:43.920357 16147 raft_consensus.cc:2468] T ecc1e363230043fe9fef390f0b979f20 P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 19.
I20260709 13:21:43.879459 22346 raft_consensus.cc:493] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:43.941797 16217 raft_consensus.cc:1275] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 13:21:43.943890 16221 raft_consensus.cc:2468] T ecc1e363230043fe9fef390f0b979f20 P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 19.
I20260709 13:21:43.879684 22321 raft_consensus.cc:493] T 8638ee363273426db1393dd81e5c7b72 P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:21:43.907260 16265 leader_election.cc:304] T 7c2095a5f4ad49408dbc7f69c606120d P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 647a343a542d4a139fe106c19095129b
I20260709 13:21:43.939847 22361 raft_consensus.cc:493] T 8c11074579b046d99e585c2818e18a2c P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:43.984284 16145 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40830 (request call id 1865) took 1448 ms. Trace:
I20260709 13:21:43.983349 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f5f89186204242bf377f7c42b2e2da" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:43.985491 16145 rpcz_store.cc:276] 0709 13:21:42.535655 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:42.536018 (+   363us) service_pool.cc:224] Handling call
0709 13:21:43.984025 (+1448007us) spinlock_profiling.cc:243] Waited 26 ms on lock 0x7b540028d554. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:21:43.984201 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":59775744}
I20260709 13:21:43.964085 21802 raft_consensus.cc:2804] T 9e65676b21b748728ad39668cfe722ec P cb82cc7bd6db46d980c75237a1402860 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:21:43.959906 22111 raft_consensus.cc:493] T 26354361e9b044cea1c2b86c4882fe92 P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:21:43.904633 22284 raft_consensus.cc:515] T 23f82224f4564fea93572e8206bad1fe P cb82cc7bd6db46d980c75237a1402860 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:43.824728 22399 raft_consensus.cc:515] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:21:43.994978 16189 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.599s	user 0.015s	sys 0.015s
I20260709 13:21:43.994092 21802 raft_consensus.cc:493] T 9e65676b21b748728ad39668cfe722ec P cb82cc7bd6db46d980c75237a1402860 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:43.974807 22339 leader_election.cc:290] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 22 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
W20260709 13:21:44.065729 16295 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.852s	user 0.004s	sys 0.019s
W20260709 13:21:44.138644 16295 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.925s	user 0.004s	sys 0.019s
I20260709 13:21:43.918936 22334 raft_consensus.cc:515] T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:43.943451 22346 raft_consensus.cc:515] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:21:44.138185 22400 log.cc:927] Time spent T 8cc959fb7f704fba9009c0595467d074 P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.215s	user 0.001s	sys 0.000s
I20260709 13:21:43.924784 22182 raft_consensus.cc:515] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } }
W20260709 13:21:43.997560 16189 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.602s	user 0.015s	sys 0.015s
I20260709 13:21:44.171039 16298 raft_consensus.cc:2468] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 17.
I20260709 13:21:43.974239 22140 consensus_queue.cc:1059] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: INVALID_TERM, Last received: 18.15, Next index: 16, Last known committed idx: 14, Time since last communication: 0.032s
I20260709 13:21:43.994827 22404 raft_consensus.cc:493] T 84f2da98864b4425bf7c0055951fdd4b P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:21:43.971542 22321 raft_consensus.cc:515] T 8638ee363273426db1393dd81e5c7b72 P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:44.141739 21802 raft_consensus.cc:3060] T 9e65676b21b748728ad39668cfe722ec P cb82cc7bd6db46d980c75237a1402860 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:21:43.991029 22361 raft_consensus.cc:515] T 8c11074579b046d99e585c2818e18a2c P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:44.140187 22390 raft_consensus.cc:493] T c0eab25ce4c64e018a041beeda2867db P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:21:44.237196 16291 raft_consensus.cc:2393] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 15: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
W20260709 13:21:44.174599 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 348163us
I20260709 13:21:44.170559 21930 raft_consensus.cc:493] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:21:44.262442 16790 leader_election.cc:304] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:21:43.996876 22111 raft_consensus.cc:515] T 26354361e9b044cea1c2b86c4882fe92 P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:43.769506 22395 raft_consensus.cc:515] T c9944d5e4b8c4109af9955c4f08c9095 P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:43.979382 22347 raft_consensus.cc:515] T 68a6838d82ef4cac83507f906139d90f P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:44.366202 16217 raft_consensus.cc:1275] T 8c11074579b046d99e585c2818e18a2c P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
W20260709 13:21:44.367273 16224 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.585s	user 0.001s	sys 0.019s
W20260709 13:21:44.367728 16224 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.585s	user 0.001s	sys 0.019s
I20260709 13:21:44.229123 22088 leader_election.cc:290] T 68a6838d82ef4cac83507f906139d90f P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 17 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:44.230695 22145 raft_consensus.cc:493] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:21:44.175205 22405 consensus_queue.cc:1048] T 8158bbd17b994d2eb96f560bd0f1b88d P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.185s
I20260709 13:21:44.273296 22236 raft_consensus.cc:493] T 8d4ab870a05347868956aa8b841b2981 P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:21:44.253274 22410 raft_consensus.cc:493] T fffab26427e24d398e2695aad40ca134 P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
W20260709 13:21:44.434697 22400 log.cc:927] Time spent T 8cc959fb7f704fba9009c0595467d074 P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.101s	user 0.000s	sys 0.000s
W20260709 13:21:44.434461 22377 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.658s	user 0.012s	sys 0.020s
I20260709 13:21:44.253298 22406 raft_consensus.cc:2749] T e4dc6b0541ca431ebc757beac00521f1 P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:21:44.195638 22404 raft_consensus.cc:515] T 84f2da98864b4425bf7c0055951fdd4b P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:44.367216 21623 raft_consensus.cc:697] T 3a7ca61592e848ea9b0fb81a2ce33cdc P cb82cc7bd6db46d980c75237a1402860 [term 19 LEADER]: Becoming Leader. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:21:44.372774 22152 raft_consensus.cc:493] T 8cc959fb7f704fba9009c0595467d074 P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:21:44.480350 16148 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40844 (request call id 2772) took 2308 ms. Trace:
I20260709 13:21:44.488722 16148 rpcz_store.cc:276] 0709 13:21:42.172016 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:42.177170 (+  5154us) service_pool.cc:224] Handling call
0709 13:21:42.177428 (+   258us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:21:42.654590 (+477162us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:44.480026 (+1825436us) spinlock_profiling.cc:243] Waited 320 ms on lock 0x7b54000e9a54. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:21:44.480273 (+   247us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":735416576}
I20260709 13:21:44.320057 22390 raft_consensus.cc:515] T c0eab25ce4c64e018a041beeda2867db P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:21:44.474525 22377 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.698s	user 0.016s	sys 0.020s
I20260709 13:21:44.350536 22353 consensus_queue.cc:1048] T 23f82224f4564fea93572e8206bad1fe P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.127s
W20260709 13:21:44.502812 16265 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.518s	user 0.006s	sys 0.055s
I20260709 13:21:44.351315 21930 raft_consensus.cc:515] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:21:44.511837 16265 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.527s	user 0.006s	sys 0.055s
W20260709 13:21:44.512470 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 605307us
I20260709 13:21:44.512966 16300 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48674 (request call id 3191) took 1280 ms. Trace:
I20260709 13:21:44.513186 16265 leader_election.cc:304] T 02838c86531d4f98bf58d1c22b8baf77 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 647a343a542d4a139fe106c19095129b
I20260709 13:21:44.513341 16300 rpcz_store.cc:276] 0709 13:21:43.233127 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:43.233268 (+   141us) service_pool.cc:224] Handling call
0709 13:21:43.233576 (+   308us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:21:43.440894 (+207318us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:21:43.440926 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:21:43.441475 (+   549us) log.cc:844] Serialized 26 byte log entry
0709 13:21:43.668551 (+227076us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:21:43.826618 (+158067us) log.cc:844] Serialized 14 byte log entry
0709 13:21:43.827823 (+  1205us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:43.827939 (+   116us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:21:44.512749 (+684810us) raft_consensus.cc:1685] finished
0709 13:21:44.512769 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:21:44.512909 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":206927,"prepare.queue_time_us":826448,"raft.queue_time_us":229904,"raft.run_cpu_time_us":98,"raft.run_wall_time_us":2762,"spinlock_wait_cycles":37147520,"thread_start_us":319552,"threads_started":2,"wal-append.queue_time_us":364941}
W20260709 13:21:44.486446 22402 log.cc:927] Time spent T 7c2095a5f4ad49408dbc7f69c606120d P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.523s	user 0.000s	sys 0.001s
I20260709 13:21:44.471627 22140 raft_consensus.cc:493] T 66f75124215f4ca6bef831e174a792d8 P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:21:44.521802 16265 leader_election.cc:304] T a378d7f796f94615b3b347119a9880fd P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860
W20260709 13:21:44.551506 16789 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.720s	user 0.008s	sys 0.060s
W20260709 13:21:44.553491 16789 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.722s	user 0.008s	sys 0.060s
W20260709 13:21:44.523707 16263 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.532s	user 0.012s	sys 0.054s
W20260709 13:21:44.556057 16263 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.564s	user 0.016s	sys 0.056s
W20260709 13:21:44.555503 16789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 768449us
W20260709 13:21:44.504293 22356 consensus_meta.cc:287] T 18ad7e3af5504571a69d10c00750a01c P 647a343a542d4a139fe106c19095129b: Time spent flushing consensus metadata: real 0.930s	user 0.012s	sys 0.000s
I20260709 13:21:44.531920 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f2da98864b4425bf7c0055951fdd4b" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:44.466150 22236 raft_consensus.cc:515] T 8d4ab870a05347868956aa8b841b2981 P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:44.463078 22145 raft_consensus.cc:515] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:21:44.561765 16266 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.504s	user 0.028s	sys 0.062s
W20260709 13:21:44.568553 16266 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.511s	user 0.028s	sys 0.062s
I20260709 13:21:44.568274 16300 raft_consensus.cc:2468] T 84f2da98864b4425bf7c0055951fdd4b P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 13.
W20260709 13:21:44.568928 16266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 511244us
I20260709 13:21:44.484557 22411 raft_consensus.cc:2804] T 7c2095a5f4ad49408dbc7f69c606120d P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:21:44.505537 22406 raft_consensus.cc:493] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:21:44.573833 22406 raft_consensus.cc:515] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:44.576027 22406 leader_election.cc:290] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:44.577841 22411 raft_consensus.cc:493] T 7c2095a5f4ad49408dbc7f69c606120d P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:21:44.578267 22411 raft_consensus.cc:3060] T 7c2095a5f4ad49408dbc7f69c606120d P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:21:44.511229 22104 leader_election.cc:290] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
W20260709 13:21:44.558656 22401 log.cc:927] Time spent T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 0.580s	user 0.000s	sys 0.001s
W20260709 13:21:44.559949 22197 consensus_meta.cc:287] T b6f07842d3aa47858df5665874c86f3a P 647a343a542d4a139fe106c19095129b: Time spent flushing consensus metadata: real 0.992s	user 0.006s	sys 0.006s
I20260709 13:21:44.580442 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c61a39fd4f84424b129c1a6f9ba404d" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:44.471980 22410 raft_consensus.cc:515] T fffab26427e24d398e2695aad40ca134 P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:44.495823 22152 raft_consensus.cc:515] T 8cc959fb7f704fba9009c0595467d074 P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:44.595398 16292 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48674 (request call id 3203) took 1127 ms. Trace:
I20260709 13:21:44.565059 22336 leader_election.cc:290] T 19baa7bc367844c288bca8f81439c413 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:44.598483 16292 rpcz_store.cc:276] 0709 13:21:43.467836 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:43.528000 (+ 60164us) service_pool.cc:224] Handling call
0709 13:21:43.564699 (+ 36699us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:21:43.567327 (+  2628us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:21:43.567356 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:21:43.567932 (+   576us) log.cc:844] Serialized 26 byte log entry
0709 13:21:43.697436 (+129504us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:21:43.697475 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:43.697546 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:21:44.595121 (+897575us) raft_consensus.cc:1685] finished
0709 13:21:44.595147 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:21:44.595331 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2026,"raft.queue_time_us":137198,"raft.run_cpu_time_us":171,"raft.run_wall_time_us":19588,"thread_start_us":129218,"threads_started":1,"wal-append.queue_time_us":293272}
I20260709 13:21:44.595674 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ad5dbfa99ee4b8d9e28be92b3f36db5" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:44.571918 22339 raft_consensus.cc:2749] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P cb82cc7bd6db46d980c75237a1402860 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:21:44.579733 22399 leader_election.cc:290] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:21:44.566901 22140 raft_consensus.cc:515] T 66f75124215f4ca6bef831e174a792d8 P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:44.571379 22356 raft_consensus.cc:515] T 18ad7e3af5504571a69d10c00750a01c P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } }
I20260709 13:21:44.571264 22421 raft_consensus.cc:493] T 3884f864a96d428785895e5463e5728f P cb82cc7bd6db46d980c75237a1402860 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:21:45.435784 16223 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57604 (request call id 3129) took 2573 ms (client timeout 1533 ms). Trace:
I20260709 13:21:44.627404 16292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdbd78acb4cf46298e0de8ba0d130e19" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
W20260709 13:21:45.436892 16223 rpcz_store.cc:269] 0709 13:21:42.861852 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:42.862029 (+   177us) service_pool.cc:224] Handling call
0709 13:21:45.435209 (+2573180us) spinlock_profiling.cc:243] Waited 111 ms on lock 0x7b54002c9554. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:21:45.435495 (+   286us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":256363904}
I20260709 13:21:45.437996 16292 raft_consensus.cc:2468] T cdbd78acb4cf46298e0de8ba0d130e19 P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 19.
W20260709 13:21:45.416517 16190 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.936s	user 0.006s	sys 0.010s
W20260709 13:21:45.439949 16190 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.960s	user 0.007s	sys 0.010s
I20260709 13:21:44.628579 16299 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48674 (request call id 3199) took 1204 ms. Trace:
I20260709 13:21:44.755700 16300 raft_consensus.cc:2468] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 16.
I20260709 13:21:44.617828 16291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d16dfad0c3b540e5b4b86492ddf87208" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
W20260709 13:21:45.442764 16300 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48684 (request call id 2999) took 1559 ms (client timeout 1710 ms). Trace:
I20260709 13:21:45.443181 16291 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48684 (request call id 2994) took 1561 ms. Trace:
W20260709 13:21:45.443361 16300 rpcz_store.cc:269] 0709 13:21:43.883157 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:44.594876 (+711719us) service_pool.cc:224] Handling call
0709 13:21:45.442438 (+847562us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:21:44.626770 16297 raft_consensus.cc:1275] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 17 index: 11. (term mismatch)
I20260709 13:21:45.425053 16224 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57604 (request call id 3138) took 2547 ms. Trace:
I20260709 13:21:45.444902 16297 pending_rounds.cc:85] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045: Aborting all ops after (but not including) 10
W20260709 13:21:44.635644 16294 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48674 (request call id 3185) took 2498 ms (client timeout 2289 ms). Trace:
I20260709 13:21:45.446285 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74ca8140f35c4ae29e8a664f6bdfc829" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
W20260709 13:21:44.626458 16296 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48684 (request call id 2973) took 2660 ms (client timeout 1739 ms). Trace:
W20260709 13:21:45.412578 16189 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.186s	user 0.001s	sys 0.034s
I20260709 13:21:45.414047 16149 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46224 (request call id 987) took 2717 ms. Trace:
W20260709 13:21:45.447423 16296 rpcz_store.cc:269] 0709 13:21:41.966140 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:41.966277 (+   137us) service_pool.cc:224] Handling call
0709 13:21:44.626227 (+2659950us) spinlock_profiling.cc:243] Waited 1.43 s on lock 0x7b54000cec54. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:21:44.626386 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3293330688}
I20260709 13:21:45.448321 16217 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33050 (request call id 1071) took 1625 ms. Trace:
I20260709 13:21:45.449337 16296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02838c86531d4f98bf58d1c22b8baf77" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 18 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
W20260709 13:21:45.448531 16189 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.222s	user 0.001s	sys 0.037s
I20260709 13:21:45.447819 16300 raft_consensus.cc:2410] T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac for term 18 because replica has last-logged OpId of term: 16 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 12.
I20260709 13:21:45.445580 16297 pending_rounds.cc:107] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045: Aborting uncommitted NO_OP operation due to leader change: 16.11
W20260709 13:21:45.446310 16294 rpcz_store.cc:269] 0709 13:21:42.136905 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:42.186449 (+ 49544us) service_pool.cc:224] Handling call
0709 13:21:44.635451 (+2449002us) spinlock_profiling.cc:243] Waited 303 ms on lock 0x7b5400399ed4. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:21:44.635579 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":696178944}
I20260709 13:21:45.450986 16300 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48684 (request call id 2998) took 1568 ms. Trace:
I20260709 13:21:45.451310 16297 raft_consensus.cc:2889] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:21:45.451687 16300 rpcz_store.cc:276] 0709 13:21:43.882866 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:45.445501 (+1562635us) service_pool.cc:224] Handling call
0709 13:21:45.450863 (+  5362us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:21:45.452219 16294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a959dbecc2648eb85d1cc550f5a3245" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 16 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:45.453088 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c2095a5f4ad49408dbc7f69c606120d" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 14 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:45.453284 16294 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48684 (request call id 2988) took 1576 ms. Trace:
I20260709 13:21:45.415786 16150 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40844 (request call id 2781) took 2358 ms. Trace:
I20260709 13:21:45.453933 16300 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48684 (request call id 2985) took 1578 ms. Trace:
I20260709 13:21:45.454352 16300 rpcz_store.cc:276] 0709 13:21:43.876015 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:45.452662 (+1576647us) service_pool.cc:224] Handling call
0709 13:21:45.453871 (+  1209us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:21:45.442134 16299 rpcz_store.cc:276] 0709 13:21:43.424650 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:43.426389 (+  1739us) service_pool.cc:224] Handling call
0709 13:21:43.426689 (+   300us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:21:43.565237 (+138548us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:21:43.565282 (+    45us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:21:43.565342 (+    60us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:21:43.565398 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:44.628297 (+1062899us) spinlock_profiling.cc:243] Waited 138 ms on lock 0x7b540041c3d4. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:21:44.628352 (+    55us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:21:44.628507 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":318070144}
W20260709 13:21:45.457079 16790 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.137s	user 0.009s	sys 0.021s
W20260709 13:21:45.457603 16790 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.138s	user 0.009s	sys 0.021s
W20260709 13:21:45.443641 16298 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48684 (request call id 2995) took 1561 ms (client timeout 1577 ms). Trace:
W20260709 13:21:45.459529 16298 rpcz_store.cc:269] 0709 13:21:43.881933 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:43.965394 (+ 83461us) service_pool.cc:224] Handling call
0709 13:21:45.443247 (+1477853us) spinlock_profiling.cc:243] Waited 185 ms on lock 0x7b5400162e54. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:21:45.443462 (+   215us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":425987840}
W20260709 13:21:45.460222 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 1.19779e+06us
I20260709 13:21:44.793627 16789 leader_election.cc:304] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac
I20260709 13:21:45.454237 16150 rpcz_store.cc:276] 0709 13:21:43.057426 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:43.155366 (+ 97940us) service_pool.cc:224] Handling call
0709 13:21:43.155680 (+   314us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:21:43.484263 (+328583us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:45.415062 (+1930799us) spinlock_profiling.cc:243] Waited 328 ms on lock 0x7b540011b2d4. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:21:45.415544 (+   482us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":754328448}
W20260709 13:21:45.440704 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 972672us
I20260709 13:21:45.467259 16291 rpcz_store.cc:276] 0709 13:21:43.881658 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:44.617501 (+735843us) service_pool.cc:224] Handling call
0709 13:21:45.442956 (+825455us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:21:45.467914 16789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 674251us
I20260709 13:21:45.449128 16217 rpcz_store.cc:276] 0709 13:21:43.823270 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:43.942576 (+119306us) service_pool.cc:224] Handling call
0709 13:21:43.942813 (+   237us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:21:44.366621 (+423808us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:45.447557 (+1080936us) spinlock_profiling.cc:243] Waited 423 ms on lock 0x7b5400367754. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:21:45.448113 (+   556us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":972757760}
I20260709 13:21:45.469544 16789 leader_election.cc:304] T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac
I20260709 13:21:45.425757 16218 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33050 (request call id 1061) took 2056 ms. Trace:
I20260709 13:21:45.471903 16218 rpcz_store.cc:276] 0709 13:21:43.368875 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:43.711721 (+342846us) service_pool.cc:224] Handling call
0709 13:21:43.733345 (+ 21624us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:21:45.414013 (+1680668us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:21:45.414139 (+   126us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:21:45.414241 (+   102us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:21:45.414323 (+    82us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:45.425139 (+ 10816us) spinlock_profiling.cc:243] Waited 1.68 s on lock 0x7b5400141ed4. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:21:45.425204 (+    65us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:21:45.425534 (+   330us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3862983424}
I20260709 13:21:45.412652 16142 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40844 (request call id 2778) took 2362 ms. Trace:
I20260709 13:21:45.465348 16790 leader_election.cc:304] T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:21:45.474380 16142 rpcz_store.cc:276] 0709 13:21:43.051041 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:43.051896 (+   855us) service_pool.cc:224] Handling call
0709 13:21:43.052192 (+   296us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:21:43.704350 (+652158us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:45.411493 (+1707143us) spinlock_profiling.cc:243] Waited 650 ms on lock 0x7b54001f22d4. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:21:45.412294 (+   801us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1495429376}
W20260709 13:21:45.428342 16225 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57600 (request call id 1839) took 2264 ms (client timeout 1570 ms). Trace:
W20260709 13:21:45.476500 16225 rpcz_store.cc:269] 0709 13:21:43.163571 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:43.180002 (+ 16431us) service_pool.cc:224] Handling call
0709 13:21:45.427824 (+2247822us) spinlock_profiling.cc:243] Waited 8.57 ms on lock 0x7b5400139054. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:21:45.428078 (+   254us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":19709312}
I20260709 13:21:45.448426 16149 rpcz_store.cc:276] 0709 13:21:42.696061 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:42.696241 (+   180us) service_pool.cc:224] Handling call
0709 13:21:42.696657 (+   416us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:21:42.766443 (+ 69786us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:21:42.766483 (+    40us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:21:42.766541 (+    58us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:21:42.766587 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:45.413211 (+2646624us) spinlock_profiling.cc:243] Waited 69.5 ms on lock 0x7b540041e1d4. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:21:45.413383 (+   172us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:21:45.413787 (+   404us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":159917312}
I20260709 13:21:45.477402 16790 leader_election.cc:304] T 434bff3082d141838068f323bdb0d1ed P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:21:44.634904 16821 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49906 (request call id 1198) took 1578 ms. Trace:
W20260709 13:21:45.484364 16820 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49896 (request call id 1429) took 2023 ms (client timeout 1603 ms). Trace:
W20260709 13:21:45.484920 16820 rpcz_store.cc:269] 0709 13:21:43.461092 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:43.462527 (+  1435us) service_pool.cc:224] Handling call
0709 13:21:45.483778 (+2021251us) spinlock_profiling.cc:243] Waited 22.6 ms on lock 0x7b54007ffdd4. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:21:45.484140 (+   362us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":52011264}
I20260709 13:21:44.713114 16819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74ca8140f35c4ae29e8a664f6bdfc829" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:45.486850 16819 raft_consensus.cc:2468] T 74ca8140f35c4ae29e8a664f6bdfc829 P cb82cc7bd6db46d980c75237a1402860 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 17.
I20260709 13:21:44.618991 16825 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49896 (request call id 1432) took 1157 ms. Trace:
I20260709 13:21:45.482175 16822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ab0ac8347641e6955e9bce7d90c1a2" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:45.487812 16825 rpcz_store.cc:276] 0709 13:21:43.462007 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:43.464634 (+  2627us) service_pool.cc:224] Handling call
0709 13:21:43.464953 (+   319us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:21:43.735293 (+270340us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:44.618683 (+883390us) spinlock_profiling.cc:243] Waited 269 ms on lock 0x7b54006423d4. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:21:44.618914 (+   231us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":619717376}
I20260709 13:21:45.488761 16819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e4bfb9090c8460eb14fa7e0d818e7f1" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:44.608259 16824 raft_consensus.cc:1275] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860 [term 20 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 13:21:45.489748 16822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:45.490252 16825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e65676b21b748728ad39668cfe722ec" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:45.483995 16818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66ae176961b74813a58bc15c9c4c6f2a" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:45.487038 16823 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49906 (request call id 1201) took 2279 ms. Trace:
I20260709 13:21:45.492190 16818 raft_consensus.cc:2410] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 for term 22 because replica has last-logged OpId of term: 21 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 13.
I20260709 13:21:45.482650 16817 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49896 (request call id 1430) took 2021 ms. Trace:
I20260709 13:21:45.493278 16817 rpcz_store.cc:276] 0709 13:21:43.461420 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:43.463261 (+  1841us) service_pool.cc:224] Handling call
0709 13:21:43.463582 (+   321us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:21:43.685926 (+222344us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:45.481798 (+1795872us) spinlock_profiling.cc:243] Waited 127 ms on lock 0x7b5400771354. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:21:45.482289 (+   491us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":291057408}
I20260709 13:21:45.491706 16823 rpcz_store.cc:276] 0709 13:21:43.207940 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:43.210768 (+  2828us) service_pool.cc:224] Handling call
0709 13:21:43.211096 (+   328us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:21:43.211336 (+   240us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 13:21:43.394181 (+182845us) log.cc:844] Serialized 14 byte log entry
0709 13:21:45.485639 (+2091458us) spinlock_profiling.cc:243] Waited 5.18 ms on lock 0x7b1800207ee0. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b693ade1 00007f69b69475b1 00007f69b07074bc 00007f69ada5548b 00007f69b68afab9 00007f69ada7169b 00007f69ada6f93f 00007f69ada71c02 00007f69b06f920b 00007f69b06fed23 00007f69b06ff23a
0709 13:21:45.486124 (+   485us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:21:45.486231 (+   107us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:21:45.486375 (+   144us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:45.486537 (+   162us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:21:45.486831 (+   294us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":242081,"prepare.run_cpu_time_us":61,"prepare.run_wall_time_us":6225,"spinlock_wait_cycles":11918080,"thread_start_us":242365,"threads_started":2,"wal-append.queue_time_us":130364,"wal-append.run_cpu_time_us":1826,"wal-append.run_wall_time_us":1262887}
I20260709 13:21:45.483520 16821 rpcz_store.cc:276] 0709 13:21:43.056594 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:43.056768 (+   174us) service_pool.cc:224] Handling call
0709 13:21:43.071447 (+ 14679us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:21:43.113438 (+ 41991us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 13:21:43.217383 (+103945us) log.cc:844] Serialized 14 byte log entry
0709 13:21:43.427538 (+210155us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:21:43.427585 (+    47us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:21:43.427634 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:44.622910 (+1195276us) spinlock_profiling.cc:243] Waited 4.53 ms on lock 0x7b1800208cc0. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b693ade1 00007f69b69475b1 00007f69ada723f3 00007f69ada747c1 00007f69ada74779 00007f69ada74709 00007f69ada746d1 00007f69ada739cd 000000000038f354 000000000038f199 00007f69ada5583a
0709 13:21:44.634621 (+ 11711us) spinlock_profiling.cc:243] Waited 41.8 ms on lock 0x7b54007c3b54. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:21:44.634694 (+    73us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:21:44.634837 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":318017,"prepare.run_cpu_time_us":45,"prepare.run_wall_time_us":45,"raft.queue_time_us":7254,"raft.run_cpu_time_us":110,"raft.run_wall_time_us":112,"spinlock_wait_cycles":106465792,"thread_start_us":313249,"threads_started":2,"wal-append.queue_time_us":449115,"wal-append.run_cpu_time_us":1371,"wal-append.run_wall_time_us":1050821}
I20260709 13:21:44.617183 22145 leader_election.cc:290] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:45.510758 16822 raft_consensus.cc:2393] T 19baa7bc367844c288bca8f81439c413 P cb82cc7bd6db46d980c75237a1402860 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 19: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
I20260709 13:21:44.600656 21623 consensus_queue.cc:237] T 3a7ca61592e848ea9b0fb81a2ce33cdc P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:45.453708 16294 rpcz_store.cc:276] 0709 13:21:43.876805 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:45.451764 (+1574959us) service_pool.cc:224] Handling call
0709 13:21:45.453203 (+  1439us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:21:44.637036 22152 leader_election.cc:290] T 8cc959fb7f704fba9009c0595467d074 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:45.399017 15046 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:21:45.454627 16297 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48684 (request call id 2984) took 3167 ms. Trace:
W20260709 13:21:45.518606 16791 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.242s	user 0.000s	sys 0.013s
I20260709 13:21:45.525010 16297 rpcz_store.cc:276] 0709 13:21:42.286924 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:42.287085 (+   161us) service_pool.cc:224] Handling call
0709 13:21:44.626384 (+2339299us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:21:45.452458 (+826074us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:45.454268 (+  1810us) spinlock_profiling.cc:243] Waited 2.34 s on lock 0x7b54000cec50. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5e8ad 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:21:45.454517 (+   249us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5378713984}
W20260709 13:21:45.519984 16292 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48684 (request call id 2997) took 1637 ms (client timeout 2043 ms). Trace:
I20260709 13:21:45.532464 16820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdbd78acb4cf46298e0de8ba0d130e19" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
W20260709 13:21:45.533126 16292 rpcz_store.cc:269] 0709 13:21:43.882555 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:44.627108 (+744553us) service_pool.cc:224] Handling call
0709 13:21:45.510573 (+883465us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:21:45.587750 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 110535us
W20260709 13:21:45.592125 16791 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.318s	user 0.000s	sys 0.015s
W20260709 13:21:44.943226 22416 log.cc:927] Time spent T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 0.439s	user 0.002s	sys 0.000s
W20260709 13:21:44.945011 22393 log.cc:927] Time spent T 1df93779209748b9860f6d893dd3de53 P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 1.280s	user 0.010s	sys 0.017s
W20260709 13:21:44.619920 22415 log.cc:927] Time spent T 829b4ad8c5a54eababeea1181b960d4b P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 0.135s	user 0.000s	sys 0.001s
I20260709 13:21:45.629238 22082 raft_consensus.cc:493] T 68a6838d82ef4cac83507f906139d90f P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:45.630124 22201 consensus_queue.cc:1048] T c9944d5e4b8c4109af9955c4f08c9095 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:21:44.611927 22361 leader_election.cc:290] T 8c11074579b046d99e585c2818e18a2c P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:44.628925 22182 leader_election.cc:290] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
W20260709 13:21:44.892426 22385 log.cc:927] Time spent T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 1.329s	user 0.004s	sys 0.008s
I20260709 13:21:44.597652 22104 raft_consensus.cc:2804] T 02838c86531d4f98bf58d1c22b8baf77 P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:21:45.639971 22024 raft_consensus.cc:493] T 26354361e9b044cea1c2b86c4882fe92 P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:21:45.640159 22145 raft_consensus.cc:493] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
W20260709 13:21:44.889966 22427 log.cc:927] Time spent T 5a959dbecc2648eb85d1cc550f5a3245 P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.184s	user 0.001s	sys 0.000s
I20260709 13:21:44.946971 22292 raft_consensus.cc:3055] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:21:44.632212 22284 leader_election.cc:290] T 23f82224f4564fea93572e8206bad1fe P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
W20260709 13:21:45.626704 22031 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.917s	user 0.007s	sys 0.002s
W20260709 13:21:45.648886 22079 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.205s	user 0.007s	sys 0.023s
I20260709 13:21:44.601390 22197 raft_consensus.cc:515] T b6f07842d3aa47858df5665874c86f3a P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:44.635236 22346 leader_election.cc:290] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
W20260709 13:21:45.317682 22411 consensus_meta.cc:287] T 7c2095a5f4ad49408dbc7f69c606120d P 90f3da57796b488c9087a2f523adf045: Time spent flushing consensus metadata: real 0.739s	user 0.008s	sys 0.003s
I20260709 13:21:44.625514 22241 raft_consensus.cc:493] T 31969392ccdb43b491911a319e8bc7e1 P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:21:44.723498 22430 raft_consensus.cc:493] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:21:44.621411 22111 leader_election.cc:290] T 26354361e9b044cea1c2b86c4882fe92 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:44.619997 22353 raft_consensus.cc:2749] T a378d7f796f94615b3b347119a9880fd P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:21:44.628988 22205 raft_consensus.cc:493] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
W20260709 13:21:45.673058 16295 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.029s	user 0.009s	sys 0.004s
I20260709 13:21:45.672817 22088 raft_consensus.cc:493] T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:21:44.608235 22230 raft_consensus.cc:493] T 57a744f5aaf944678a63469cbf514015 P cb82cc7bd6db46d980c75237a1402860 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:21:45.673214 16296 raft_consensus.cc:2410] T 02838c86531d4f98bf58d1c22b8baf77 P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac for term 18 because replica has last-logged OpId of term: 17 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 8.
I20260709 13:21:44.631611 22404 leader_election.cc:290] T 84f2da98864b4425bf7c0055951fdd4b P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:21:45.673173 22285 raft_consensus.cc:493] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:21:44.620357 22334 leader_election.cc:290] T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:21:45.677495 22372 raft_consensus.cc:2804] T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:21:45.678148 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4dc6b0541ca431ebc757beac00521f1" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:21:44.614194 22321 leader_election.cc:290] T 8638ee363273426db1393dd81e5c7b72 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:44.936539 22211 leader_election.cc:290] T 8a878999c0024d2abe517484180290fc P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:21:44.611114 21930 leader_election.cc:290] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:44.621600 22347 leader_election.cc:290] T 68a6838d82ef4cac83507f906139d90f P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 16 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:44.613485 22236 leader_election.cc:290] T 8d4ab870a05347868956aa8b841b2981 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:45.666817 22029 raft_consensus.cc:2804] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:21:44.634829 22395 leader_election.cc:290] T c9944d5e4b8c4109af9955c4f08c9095 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:45.667181 22196 raft_consensus.cc:493] T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:21:45.666887 22149 raft_consensus.cc:493] T 5a96373bb0e14dce8a266b2fc6c58507 P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:21:45.692723 22378 raft_consensus.cc:493] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:45.693476 22149 raft_consensus.cc:515] T 5a96373bb0e14dce8a266b2fc6c58507 P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:45.670991 22348 raft_consensus.cc:493] T c0eab25ce4c64e018a041beeda2867db P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
W20260709 13:21:45.684823 21802 consensus_meta.cc:287] T 9e65676b21b748728ad39668cfe722ec P cb82cc7bd6db46d980c75237a1402860: Time spent flushing consensus metadata: real 1.396s	user 0.008s	sys 0.004s
W20260709 13:21:45.681352 22079 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.401s	user 0.007s	sys 0.023s
W20260709 13:21:45.685622 16295 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.122s	user 0.010s	sys 0.004s
I20260709 13:21:44.628470 22390 leader_election.cc:290] T c0eab25ce4c64e018a041beeda2867db P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:21:44.634029 22410 leader_election.cc:290] T fffab26427e24d398e2695aad40ca134 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
W20260709 13:21:45.676422 22031 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.060s	user 0.007s	sys 0.002s
I20260709 13:21:45.693276 22196 raft_consensus.cc:515] T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:21:45.676815 16265 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.957s	user 0.007s	sys 0.006s
I20260709 13:21:45.664384 22258 raft_consensus.cc:493] T 8a878999c0024d2abe517484180290fc P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
W20260709 13:21:45.713125 16265 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.156s	user 0.008s	sys 0.006s
I20260709 13:21:45.698993 22024 raft_consensus.cc:515] T 26354361e9b044cea1c2b86c4882fe92 P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:45.705161 16819 raft_consensus.cc:2468] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P cb82cc7bd6db46d980c75237a1402860 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 17.
I20260709 13:21:45.672900 22399 raft_consensus.cc:493] T 2794af5b99984efebdee564035841cb0 P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:21:45.706938 22348 raft_consensus.cc:515] T c0eab25ce4c64e018a041beeda2867db P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:45.702255 22378 raft_consensus.cc:515] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:45.708536 22417 raft_consensus.cc:2749] T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:21:45.709007 22140 leader_election.cc:290] T 66f75124215f4ca6bef831e174a792d8 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:45.445211 16224 rpcz_store.cc:276] 0709 13:21:42.877253 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:43.482772 (+605519us) service_pool.cc:224] Handling call
0709 13:21:43.483078 (+   306us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:21:43.523063 (+ 39985us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:21:43.523103 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:21:43.523712 (+   609us) log.cc:844] Serialized 26 byte log entry
0709 13:21:43.715932 (+192220us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:21:44.368131 (+652199us) log.cc:844] Serialized 14 byte log entry
0709 13:21:44.424301 (+ 56170us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:45.424088 (+999787us) spinlock_profiling.cc:243] Waited 35.5 ms on lock 0x7b54002c9054. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:21:45.424177 (+    89us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:21:45.424303 (+   126us) raft_consensus.cc:1685] finished
0709 13:21:45.424472 (+   169us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:21:45.424763 (+   291us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":59847,"prepare.queue_time_us":883077,"prepare.run_cpu_time_us":53,"prepare.run_wall_time_us":55,"raft.queue_time_us":71933,"raft.run_cpu_time_us":299,"raft.run_wall_time_us":30404,"spinlock_wait_cycles":81675008,"thread_start_us":771281,"threads_started":2,"wal-append.queue_time_us":362649}
W20260709 13:21:45.710379 16789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 215146us
W20260709 13:21:45.695461 16263 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.037s	user 0.002s	sys 0.009s
I20260709 13:21:45.713012 22339 raft_consensus.cc:2749] T 434bff3082d141838068f323bdb0d1ed P cb82cc7bd6db46d980c75237a1402860 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:21:45.687238 22372 raft_consensus.cc:493] T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
W20260709 13:21:45.715378 16266 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.064s	user 0.036s	sys 0.059s
I20260709 13:21:45.701050 22149 leader_election.cc:290] T 5a96373bb0e14dce8a266b2fc6c58507 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:45.717085 16297 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48674 (request call id 3206) took 1932 ms. Trace:
I20260709 13:21:45.720335 16297 rpcz_store.cc:276] 0709 13:21:43.784225 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:45.624992 (+1840767us) service_pool.cc:224] Handling call
0709 13:21:45.669309 (+ 44317us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:21:45.715763 (+ 46454us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:21:45.715814 (+    51us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:21:45.715868 (+    54us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:21:45.715924 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:45.716005 (+    81us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:21:45.716164 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:21:45.691876 22029 raft_consensus.cc:493] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
W20260709 13:21:45.723863 16263 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.163s	user 0.003s	sys 0.010s
I20260709 13:21:45.719718 22372 raft_consensus.cc:3060] T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860 [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:21:45.724501 16266 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.141s	user 0.037s	sys 0.059s
I20260709 13:21:45.666769 22421 raft_consensus.cc:515] T 3884f864a96d428785895e5463e5728f P cb82cc7bd6db46d980c75237a1402860 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:45.700946 22152 raft_consensus.cc:493] T 19baa7bc367844c288bca8f81439c413 P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:45.724928 22029 raft_consensus.cc:3060] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:21:45.736325 16292 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40474 (request call id 1096) took 2007 ms. Trace:
I20260709 13:21:45.726522 22430 raft_consensus.cc:515] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:45.743126 16292 rpcz_store.cc:276] 0709 13:21:43.728849 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:45.638474 (+1909625us) service_pool.cc:224] Handling call
0709 13:21:45.684356 (+ 45882us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:21:45.735722 (+ 51366us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:21:45.735778 (+    56us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:21:45.735879 (+   101us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:21:45.735958 (+    79us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:45.736064 (+   106us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:21:45.736237 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:21:45.695699 22082 raft_consensus.cc:515] T 68a6838d82ef4cac83507f906139d90f P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:45.701655 22145 raft_consensus.cc:515] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:45.713491 21802 raft_consensus.cc:515] T 9e65676b21b748728ad39668cfe722ec P cb82cc7bd6db46d980c75237a1402860 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:45.700894 22241 raft_consensus.cc:515] T 31969392ccdb43b491911a319e8bc7e1 P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:45.750635 22361 raft_consensus.cc:493] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:45.734360 22292 raft_consensus.cc:740] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:21:45.751740 16190 leader_election.cc:304] T 84f2da98864b4425bf7c0055951fdd4b P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 647a343a542d4a139fe106c19095129b
I20260709 13:21:45.772154 16219 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33050 (request call id 1064) took 2402 ms. Trace:
I20260709 13:21:45.771620 22285 raft_consensus.cc:515] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:45.763877 22429 raft_consensus.cc:493] T d2ab0ac8347641e6955e9bce7d90c1a2 P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:21:45.778585 16296 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48684 (request call id 2987) took 1902 ms. Trace:
I20260709 13:21:45.778839 22429 raft_consensus.cc:515] T d2ab0ac8347641e6955e9bce7d90c1a2 P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:45.779038 16296 rpcz_store.cc:276] 0709 13:21:43.876538 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:45.448911 (+1572373us) service_pool.cc:224] Handling call
0709 13:21:45.778342 (+329431us) spinlock_profiling.cc:243] Waited 207 ms on lock 0x7b540035e8d4. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:21:45.778496 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":476974464}
I20260709 13:21:45.780680 16296 raft_consensus.cc:1240] T 1df93779209748b9860f6d893dd3de53 P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 15. Current term is 16. Ops: []
I20260709 13:21:45.781301 16296 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48684 (request call id 2992) took 1900 ms. Trace:
I20260709 13:21:45.781358 22429 leader_election.cc:290] T d2ab0ac8347641e6955e9bce7d90c1a2 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:45.752916 22104 raft_consensus.cc:493] T 02838c86531d4f98bf58d1c22b8baf77 P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:21:45.775009 22064 raft_consensus.cc:493] T 8d4ab870a05347868956aa8b841b2981 P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:21:45.777962 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68a6838d82ef4cac83507f906139d90f" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:21:45.778173 16825 raft_consensus.cc:2393] T 9e65676b21b748728ad39668cfe722ec P cb82cc7bd6db46d980c75237a1402860 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 17: Already voted for candidate cb82cc7bd6db46d980c75237a1402860 in this term.
I20260709 13:21:45.772572 16219 rpcz_store.cc:276] 0709 13:21:43.369608 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:43.765097 (+395489us) service_pool.cc:224] Handling call
0709 13:21:43.765360 (+   263us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:21:43.765749 (+   389us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:21:43.765775 (+    26us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:21:43.766202 (+   427us) log.cc:844] Serialized 26 byte log entry
0709 13:21:43.997472 (+231270us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:21:43.997518 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:43.997610 (+    92us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:21:45.771648 (+1774038us) raft_consensus.cc:1685] finished
0709 13:21:45.771705 (+    57us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:21:45.771976 (+   271us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1560,"raft.run_cpu_time_us":125,"raft.run_wall_time_us":125,"thread_start_us":230886,"threads_started":1,"wal-append.queue_time_us":597199}
I20260709 13:21:45.781767 16296 rpcz_store.cc:276] 0709 13:21:43.881101 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:45.779915 (+1898814us) service_pool.cc:224] Handling call
0709 13:21:45.780342 (+   427us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:21:45.781064 (+   722us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:45.781231 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:21:45.782004 16298 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40474 (request call id 1087) took 2234 ms. Trace:
I20260709 13:21:45.785183 16298 rpcz_store.cc:276] 0709 13:21:43.547468 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:45.780814 (+2233346us) service_pool.cc:224] Handling call
0709 13:21:45.781287 (+   473us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:21:45.781672 (+   385us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:21:45.781701 (+    29us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:21:45.781753 (+    52us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:21:45.781786 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:45.781849 (+    63us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:21:45.781952 (+   103us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:21:45.783955 16295 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48684 (request call id 2982) took 3815 ms. Trace:
I20260709 13:21:45.737178 22399 raft_consensus.cc:515] T 2794af5b99984efebdee564035841cb0 P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:45.735982 22258 raft_consensus.cc:515] T 8a878999c0024d2abe517484180290fc P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:45.791965 22064 raft_consensus.cc:515] T 8d4ab870a05347868956aa8b841b2981 P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:45.794925 22031 raft_consensus.cc:493] T b87147731a68400e8cce9b879daa7c5d P cb82cc7bd6db46d980c75237a1402860 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:45.797983 22197 leader_election.cc:290] T b6f07842d3aa47858df5665874c86f3a P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 17 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:45.797397 22031 raft_consensus.cc:515] T b87147731a68400e8cce9b879daa7c5d P cb82cc7bd6db46d980c75237a1402860 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:45.799540 16225 raft_consensus.cc:3060] T 68a6838d82ef4cac83507f906139d90f P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:21:45.801131 22024 leader_election.cc:290] T 26354361e9b044cea1c2b86c4882fe92 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:45.801578 22285 leader_election.cc:290] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:45.763547 22411 raft_consensus.cc:515] T 7c2095a5f4ad49408dbc7f69c606120d P 90f3da57796b488c9087a2f523adf045 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:45.767537 22088 raft_consensus.cc:515] T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:45.809131 16218 raft_consensus.cc:1275] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 13:21:45.789062 22104 raft_consensus.cc:3060] T 02838c86531d4f98bf58d1c22b8baf77 P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:21:45.801651 22082 leader_election.cc:290] T 68a6838d82ef4cac83507f906139d90f P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:21:45.790915 22361 raft_consensus.cc:515] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:21:45.828723 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 167403us
I20260709 13:21:45.828892 22356 leader_election.cc:290] T 18ad7e3af5504571a69d10c00750a01c P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 15 election: Requested vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
W20260709 13:21:45.831211 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 43977.6us
W20260709 13:21:45.832105 16117 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 177251us
I20260709 13:21:45.832185 21802 leader_election.cc:290] T 9e65676b21b748728ad39668cfe722ec P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 17 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:45.833163 22292 consensus_queue.cc:260] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 18.15, Last appended by leader: 15, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:45.834750 22292 raft_consensus.cc:3060] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:21:45.819268 22230 raft_consensus.cc:515] T 57a744f5aaf944678a63469cbf514015 P cb82cc7bd6db46d980c75237a1402860 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:45.827711 16219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806d7690c2b644959152074bb2cab0a7" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 18 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:45.822041 16789 leader_election.cc:304] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:21:45.850378 16296 raft_consensus.cc:1275] T e4dc6b0541ca431ebc757beac00521f1 P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 14 index: 7. Preceding OpId from leader: term: 16 index: 8. (index mismatch)
I20260709 13:21:45.852407 16825 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49892 (request call id 907) took 1060 ms. Trace:
I20260709 13:21:45.857496 16224 raft_consensus.cc:1275] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 13:21:45.859735 16821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee9ed67006144797b1de6ea8c2013a68" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 18 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:45.869700 16825 rpcz_store.cc:276] 0709 13:21:44.792431 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:45.489818 (+697387us) service_pool.cc:224] Handling call
0709 13:21:45.852168 (+362350us) spinlock_profiling.cc:243] Waited 284 ms on lock 0x7b540047fc54. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:21:45.852320 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":654064896}
I20260709 13:21:45.863286 16817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdbd78acb4cf46298e0de8ba0d130e19" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:45.921958 16296 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48684 (request call id 2996) took 2039 ms. Trace:
I20260709 13:21:45.922704 16296 rpcz_store.cc:276] 0709 13:21:43.882258 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:45.796684 (+1914426us) service_pool.cc:224] Handling call
0709 13:21:45.849722 (+ 53038us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:21:45.869267 (+ 19545us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:45.921882 (+ 52615us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:21:45.854166 16292 pending_rounds.cc:85] T 66ae176961b74813a58bc15c9c4c6f2a P 90f3da57796b488c9087a2f523adf045: Aborting all ops after (but not including) 12
I20260709 13:21:45.924158 16292 pending_rounds.cc:107] T 66ae176961b74813a58bc15c9c4c6f2a P 90f3da57796b488c9087a2f523adf045: Aborting uncommitted NO_OP operation due to leader change: 20.13
I20260709 13:21:45.841526 16146 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40844 (request call id 2777) took 2795 ms. Trace:
I20260709 13:21:45.851256 16299 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40474 (request call id 1085) took 2319 ms. Trace:
I20260709 13:21:45.849328 16291 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40474 (request call id 1086) took 2304 ms. Trace:
I20260709 13:21:45.850031 22236 raft_consensus.cc:493] T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:21:45.931537 16789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 165439us
I20260709 13:21:45.932073 16297 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48684 (request call id 2986) took 2055 ms. Trace:
I20260709 13:21:45.932466 16789 leader_election.cc:304] T 74ca8140f35c4ae29e8a664f6bdfc829 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:21:45.933212 16294 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40474 (request call id 1090) took 2348 ms. Trace:
W20260709 13:21:45.927312 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 96209.9us
I20260709 13:21:45.933822 22411 leader_election.cc:290] T 7c2095a5f4ad49408dbc7f69c606120d P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 14 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:21:45.935506 16790 leader_election.cc:304] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:21:45.936564 22246 raft_consensus.cc:493] T 5a96373bb0e14dce8a266b2fc6c58507 P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:21:45.937367 22246 raft_consensus.cc:515] T 5a96373bb0e14dce8a266b2fc6c58507 P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:21:45.838745 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 1.31695e+06us
I20260709 13:21:45.924415 16292 raft_consensus.cc:2889] T 66ae176961b74813a58bc15c9c4c6f2a P 90f3da57796b488c9087a2f523adf045 [term 21 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:21:45.928721 16291 rpcz_store.cc:276] 0709 13:21:43.544585 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:45.848006 (+2303421us) service_pool.cc:224] Handling call
0709 13:21:45.848548 (+   542us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:21:45.848913 (+   365us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:21:45.848954 (+    41us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:21:45.849027 (+    73us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:21:45.849070 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:45.849139 (+    69us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:21:45.849262 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:21:45.840444 22196 leader_election.cc:290] T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:45.932121 16824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62fe3346cd3c42c4a55e14af9073bca4" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:45.823925 22205 raft_consensus.cc:515] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:45.831025 22399 leader_election.cc:290] T 2794af5b99984efebdee564035841cb0 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:45.834966 22421 leader_election.cc:290] T 3884f864a96d428785895e5463e5728f P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:45.843835 22378 leader_election.cc:290] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:45.787067 16295 rpcz_store.cc:276] 0709 13:21:41.968954 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:41.974286 (+  5332us) service_pool.cc:224] Handling call
0709 13:21:41.974704 (+   418us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:21:41.974993 (+   289us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:21:42.055824 (+ 80831us) log.cc:844] Serialized 14 byte log entry
0709 13:21:44.139130 (+2083306us) log.cc:844] Serialized 14 byte log entry
0709 13:21:45.746514 (+1607384us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:21:45.754085 (+  7571us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:21:45.763267 (+  9182us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:45.770607 (+  7340us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:21:45.783899 (+ 13292us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":842435,"prepare.queue_time_us":1298576,"prepare.run_cpu_time_us":90,"prepare.run_wall_time_us":94,"raft.queue_time_us":987710,"raft.run_cpu_time_us":238,"raft.run_wall_time_us":25419,"thread_start_us":2959051,"threads_started":5,"wal-append.queue_time_us":1102271,"wal-append.run_cpu_time_us":5470,"wal-append.run_wall_time_us":2145464}
I20260709 13:21:45.947037 22031 leader_election.cc:290] T b87147731a68400e8cce9b879daa7c5d P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:45.950975 22202 raft_consensus.cc:2804] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:21:45.987474 22285 raft_consensus.cc:2804] T 84f2da98864b4425bf7c0055951fdd4b P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 13:21:45.928138 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 75870.9us
I20260709 13:21:45.955688 16218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d4ab870a05347868956aa8b841b2981" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:46.022877 16295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fffab26427e24d398e2695aad40ca134" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:46.027314 22205 leader_election.cc:290] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:21:45.842228 22348 leader_election.cc:290] T c0eab25ce4c64e018a041beeda2867db P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:45.843079 22258 leader_election.cc:290] T 8a878999c0024d2abe517484180290fc P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:45.845085 22182 raft_consensus.cc:493] T 3884f864a96d428785895e5463e5728f P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:21:45.840005 22430 leader_election.cc:290] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
W20260709 13:21:45.952602 16116 connection.cc:256] RPC call timeout handler was delayed by -1.2402085010000974s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1533s for a response
I20260709 13:21:45.947918 16142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62fe3346cd3c42c4a55e14af9073bca4" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
W20260709 13:21:46.060348 21579 raft_consensus.cc:670] T 806d7690c2b644959152074bb2cab0a7 P cb82cc7bd6db46d980c75237a1402860: failed to trigger leader election: Illegal state: only voting members can start elections: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } } [suppressed 6 similar messages]
I20260709 13:21:46.061146 22202 raft_consensus.cc:697] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860 [term 19 LEADER]: Becoming Leader. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:21:46.048013 22285 raft_consensus.cc:493] T 84f2da98864b4425bf7c0055951fdd4b P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Starting leader election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:21:46.063983 22285 raft_consensus.cc:3060] T 84f2da98864b4425bf7c0055951fdd4b P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:21:45.953044 16818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74ca8140f35c4ae29e8a664f6bdfc829" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:45.948930 16296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c11074579b046d99e585c2818e18a2c" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:45.931087 22236 raft_consensus.cc:515] T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:46.070361 22236 leader_election.cc:290] T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:45.932504 16297 rpcz_store.cc:276] 0709 13:21:43.876308 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:45.725662 (+1849354us) service_pool.cc:224] Handling call
0709 13:21:45.771231 (+ 45569us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:21:45.931666 (+160435us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:21:45.931695 (+    29us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:21:45.931762 (+    67us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:21:45.931805 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:45.931869 (+    64us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:21:45.932008 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":31298,"raft.queue_time_us":86234,"raft.run_cpu_time_us":149,"raft.run_wall_time_us":149}
I20260709 13:21:46.071429 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68a6838d82ef4cac83507f906139d90f" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:21:46.072252 16297 raft_consensus.cc:2393] T 68a6838d82ef4cac83507f906139d90f P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 16: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
I20260709 13:21:45.966145 22230 leader_election.cc:290] T 57a744f5aaf944678a63469cbf514015 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:45.952699 22074 raft_consensus.cc:493] T 829b4ad8c5a54eababeea1181b960d4b P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:21:45.969491 16819 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49892 (request call id 910) took 1176 ms. Trace:
I20260709 13:21:46.073954 16819 rpcz_store.cc:276] 0709 13:21:44.793316 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:45.488358 (+695042us) service_pool.cc:224] Handling call
0709 13:21:45.969309 (+480951us) spinlock_profiling.cc:243] Waited 182 ms on lock 0x7b54007c3b54. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:21:45.969425 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":419241728}
I20260709 13:21:46.074013 22074 raft_consensus.cc:515] T 829b4ad8c5a54eababeea1181b960d4b P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:46.075505 16819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada5a004ba7c46819051e3ccec71ffdb" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:46.076871 16819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5718c623ebee4d2bb770f0f9ddb2f164" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:45.949542 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:45.950467 22198 raft_consensus.cc:2749] T 74ca8140f35c4ae29e8a664f6bdfc829 P cb82cc7bd6db46d980c75237a1402860 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:21:45.931550 16293 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48674 (request call id 3202) took 2478 ms. Trace:
I20260709 13:21:45.950044 22201 raft_consensus.cc:2804] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:21:46.023600 16824 raft_consensus.cc:2393] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 22: Already voted for candidate cb82cc7bd6db46d980c75237a1402860 in this term.
I20260709 13:21:45.960446 16220 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57604 (request call id 3137) took 3083 ms. Trace:
W20260709 13:21:45.966976 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 250514us
W20260709 13:21:45.985955 16189 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:45.933717 16294 rpcz_store.cc:276] 0709 13:21:43.584554 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:45.601348 (+2016794us) service_pool.cc:224] Handling call
0709 13:21:45.676127 (+ 74779us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:21:45.802424 (+126297us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:21:45.802465 (+    41us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:21:45.809292 (+  6827us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:21:45.809340 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:45.932926 (+123586us) spinlock_profiling.cc:243] Waited 107 ms on lock 0x7b54001eee54. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:21:45.932981 (+    55us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:21:45.933119 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":245769472}
I20260709 13:21:45.928279 16299 rpcz_store.cc:276] 0709 13:21:43.531882 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:45.849962 (+2318080us) service_pool.cc:224] Handling call
0709 13:21:45.850457 (+   495us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:21:45.850847 (+   390us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:21:45.850882 (+    35us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:21:45.850947 (+    65us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:21:45.850988 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:45.851053 (+    65us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:21:45.851191 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:21:45.968091 16300 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40474 (request call id 1084) took 2440 ms. Trace:
I20260709 13:21:45.837653 22436 raft_consensus.cc:493] T 8c11074579b046d99e585c2818e18a2c P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:46.042106 16225 raft_consensus.cc:2468] T 68a6838d82ef4cac83507f906139d90f P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 17.
I20260709 13:21:45.969977 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f82224f4564fea93572e8206bad1fe" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:45.826113 22024 raft_consensus.cc:493] T 434bff3082d141838068f323bdb0d1ed P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:21:45.861379 22145 leader_election.cc:290] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:45.986949 16265 leader_election.cc:304] T ada5a004ba7c46819051e3ccec71ffdb P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: cb82cc7bd6db46d980c75237a1402860
I20260709 13:21:45.960239 16817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c11074579b046d99e585c2818e18a2c" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:45.847374 21704 raft_consensus.cc:493] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:21:46.050793 16295 raft_consensus.cc:2468] T fffab26427e24d398e2695aad40ca134 P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 16.
I20260709 13:21:45.931067 22440 consensus_queue.cc:1048] T ee9ed67006144797b1de6ea8c2013a68 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 10, Time since last communication: 0.159s
I20260709 13:21:46.077425 16819 raft_consensus.cc:2468] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 16.
I20260709 13:21:46.053602 16291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cc959fb7f704fba9009c0595467d074" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:45.925923 16146 rpcz_store.cc:276] 0709 13:21:43.045522 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:43.045633 (+   111us) service_pool.cc:224] Handling call
0709 13:21:43.061061 (+ 15428us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:21:43.061421 (+   360us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:21:43.061451 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:21:43.061911 (+   460us) log.cc:844] Serialized 26 byte log entry
0709 13:21:45.422899 (+2360988us) spinlock_profiling.cc:243] Waited 8.47 ms on lock 0x7b180033f220. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b693ade1 00007f69b69475b1 00007f69ada54e07 00007f69b68afab9 00007f69ada7169b 00007f69ada6f93f 00007f69ada71c02 00007f69b06f920b 00007f69b06fed23 00007f69b06ff0d1 00007f69b0b1aa0f
0709 13:21:45.840781 (+417882us) spinlock_profiling.cc:243] Waited 13.6 ms on lock 0x7b5400313920. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac9090 00007f69b0ad66b8 00007f69b0b5f92b 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c
0709 13:21:45.840863 (+    82us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:21:45.840911 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:45.841045 (+   134us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:21:45.841110 (+    65us) raft_consensus.cc:1685] finished
0709 13:21:45.841128 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:21:45.841332 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":10217,"raft.run_cpu_time_us":129,"raft.run_wall_time_us":9026,"spinlock_wait_cycles":50836608,"thread_start_us":163624,"threads_started":1,"wal-append.queue_time_us":2558192}
I20260709 13:21:45.959429 22361 leader_election.cc:290] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:45.957058 22088 leader_election.cc:290] T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:45.859632 22064 leader_election.cc:290] T 8d4ab870a05347868956aa8b841b2981 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
W20260709 13:21:46.082208 16190 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.14.177.131:41389 timed out after 1.577s (SENT) [suppressed 7 similar messages]
I20260709 13:21:46.086941 16817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7ca61592e848ea9b0fb81a2ce33cdc" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
W20260709 13:21:46.085729 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 98890.8us
I20260709 13:21:46.083498 16300 rpcz_store.cc:276] 0709 13:21:43.528224 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:45.512851 (+1984627us) service_pool.cc:224] Handling call
0709 13:21:45.674987 (+162136us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:21:45.919515 (+244528us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:21:45.919549 (+    34us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:21:45.925895 (+  6346us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:21:45.925965 (+    70us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:45.967858 (+ 41893us) spinlock_profiling.cc:243] Waited 145 ms on lock 0x7b54000aa854. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:21:45.967909 (+    51us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:21:45.968035 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":334619520}
I20260709 13:21:46.086997 16818 raft_consensus.cc:2468] T 74ca8140f35c4ae29e8a664f6bdfc829 P cb82cc7bd6db46d980c75237a1402860 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 17.
I20260709 13:21:46.079852 16824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ab0ac8347641e6955e9bce7d90c1a2" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:46.084947 22383 raft_consensus.cc:493] T ee9ed67006144797b1de6ea8c2013a68 P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:46.117674 22236 raft_consensus.cc:493] T 9e65676b21b748728ad39668cfe722ec P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:21:46.084293 16149 raft_consensus.cc:2468] T 23f82224f4564fea93572e8206bad1fe P 647a343a542d4a139fe106c19095129b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 14.
I20260709 13:21:46.079115 22201 raft_consensus.cc:493] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:46.085216 22168 raft_consensus.cc:493] T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:21:46.128347 22201 raft_consensus.cc:3060] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:21:46.128640 22168 raft_consensus.cc:515] T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:46.131109 22168 leader_election.cc:290] T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:46.124611 16821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66f75124215f4ca6bef831e174a792d8" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
W20260709 13:21:46.131307 16192 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 43538.7us
W20260709 13:21:46.118984 16115 connection.cc:256] RPC call timeout handler was delayed by -1.2613850110001295s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22890000000000002s for a response
W20260709 13:21:46.132293 16115 connection.cc:256] RPC call timeout handler was delayed by -1.0904450480002197s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1516s for a response
W20260709 13:21:46.132659 16115 connection.cc:256] RPC call timeout handler was delayed by -1.0751450480001949s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1533s for a response
I20260709 13:21:46.087179 22399 raft_consensus.cc:2804] T ada5a004ba7c46819051e3ccec71ffdb P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Leader pre-election won for term 18
W20260709 13:21:46.132987 16115 connection.cc:256] RPC call timeout handler was delayed by -1.0121450480000931s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1603s for a response
I20260709 13:21:46.133203 22399 raft_consensus.cc:493] T ada5a004ba7c46819051e3ccec71ffdb P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Starting leader election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:21:46.133448 22145 raft_consensus.cc:1064] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac: attempting to promote NON_VOTER cb82cc7bd6db46d980c75237a1402860 to VOTER
I20260709 13:21:46.133736 22399 raft_consensus.cc:3060] T ada5a004ba7c46819051e3ccec71ffdb P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:21:46.133988 22145 raft_consensus.cc:1068] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac: Unable to promote non-voter cb82cc7bd6db46d980c75237a1402860: Illegal state: Leader has not yet committed an operation in its own term
I20260709 13:21:46.087617 21704 raft_consensus.cc:515] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:46.085498 22119 raft_consensus.cc:493] T 746aabf856ab401c9ef5e859371e7b67 P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:21:46.077554 22074 leader_election.cc:290] T 829b4ad8c5a54eababeea1181b960d4b P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:46.140422 22168 raft_consensus.cc:493] T 08c74490c40243a88e62d1760b5cd814 P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:46.141212 22168 raft_consensus.cc:515] T 08c74490c40243a88e62d1760b5cd814 P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:46.116596 22082 raft_consensus.cc:493] T 57a744f5aaf944678a63469cbf514015 P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:21:46.143294 22445 consensus_queue.cc:1048] T af57996a8be84d398f0f023e8d08ef77 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.105s
I20260709 13:21:46.143656 22082 raft_consensus.cc:515] T 57a744f5aaf944678a63469cbf514015 P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:46.144018 22168 leader_election.cc:290] T 08c74490c40243a88e62d1760b5cd814 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:46.145846 16816 raft_consensus.cc:1240] T 3a7ca61592e848ea9b0fb81a2ce33cdc P cb82cc7bd6db46d980c75237a1402860 [term 19 LEADER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 18. Current term is 19. Ops: [18.15-18.15]
I20260709 13:21:46.145979 22082 leader_election.cc:290] T 57a744f5aaf944678a63469cbf514015 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:45.927832 22152 raft_consensus.cc:515] T 19baa7bc367844c288bca8f81439c413 P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:46.149173 22215 raft_consensus.cc:493] T e4dc6b0541ca431ebc757beac00521f1 P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:21:46.149993 16789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 217667us
I20260709 13:21:46.123116 16825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b7d1ab519044eda2d5e04a07242f25" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:46.137768 22119 raft_consensus.cc:515] T 746aabf856ab401c9ef5e859371e7b67 P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:46.084681 22024 raft_consensus.cc:515] T 434bff3082d141838068f323bdb0d1ed P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:21:46.153038 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 92815.7us
I20260709 13:21:46.089998 22440 consensus_queue.cc:1243] T ee9ed67006144797b1de6ea8c2013a68 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Peer 647a343a542d4a139fe106c19095129b log is divergent from this leader: its last log entry 16.13 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 13:21:46.159220 22246 leader_election.cc:290] T 5a96373bb0e14dce8a266b2fc6c58507 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:46.083881 22436 raft_consensus.cc:515] T 8c11074579b046d99e585c2818e18a2c P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:46.161229 22119 leader_election.cc:290] T 746aabf856ab401c9ef5e859371e7b67 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:46.162564 21704 leader_election.cc:290] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:46.086858 22241 leader_election.cc:290] T 31969392ccdb43b491911a319e8bc7e1 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:46.163362 16291 raft_consensus.cc:2468] T 8cc959fb7f704fba9009c0595467d074 P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 15.
W20260709 13:21:46.159369 22427 log.cc:927] Time spent T 5a959dbecc2648eb85d1cc550f5a3245 P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.353s	user 0.000s	sys 0.001s
I20260709 13:21:46.151602 22348 raft_consensus.cc:493] T b6f07842d3aa47858df5665874c86f3a P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:46.120549 22383 raft_consensus.cc:515] T ee9ed67006144797b1de6ea8c2013a68 P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:46.164103 16265 leader_election.cc:304] T 8cc959fb7f704fba9009c0595467d074 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:21:46.164896 16819 raft_consensus.cc:1275] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 [term 19 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 13:21:46.165438 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e65676b21b748728ad39668cfe722ec" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:21:46.079299 16220 rpcz_store.cc:276] 0709 13:21:42.876915 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:43.377797 (+500882us) service_pool.cc:224] Handling call
0709 13:21:45.431420 (+2053623us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:21:45.432595 (+  1175us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:21:45.432648 (+    53us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:21:45.433920 (+  1272us) log.cc:844] Serialized 26 byte log entry
0709 13:21:45.531629 (+ 97709us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:21:45.531668 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:45.587529 (+ 55861us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:21:45.958828 (+371299us) raft_consensus.cc:1685] finished
0709 13:21:45.958858 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:21:45.960055 (+  1197us) spinlock_profiling.cc:243] Waited 2.05 s on lock 0x7b5400139050. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5e8ad 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:21:45.960231 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3837,"raft.queue_time_us":166458,"raft.run_cpu_time_us":395,"raft.run_wall_time_us":38928,"spinlock_wait_cycles":4712942464,"thread_start_us":92073,"threads_started":1,"wal-append.queue_time_us":380433}
I20260709 13:21:46.124361 22197 raft_consensus.cc:493] T 23f82224f4564fea93572e8206bad1fe P 647a343a542d4a139fe106c19095129b [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:46.078917 16293 rpcz_store.cc:276] 0709 13:21:43.457815 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:43.462445 (+  4630us) service_pool.cc:224] Handling call
0709 13:21:43.462768 (+   323us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:21:43.963443 (+500675us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:21:43.963477 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:21:45.440728 (+1477251us) spinlock_profiling.cc:243] Waited 2.64 ms on lock 0x7b2c00132978. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69af458a11 00007f69af468372 00007f69b0b9aa01 00007f69b0b5f43a 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2
0709 13:21:45.449909 (+  9181us) log.cc:844] Serialized 26 byte log entry
0709 13:21:45.647111 (+197202us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:21:45.647372 (+   261us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:45.658295 (+ 10923us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:21:45.874295 (+216000us) raft_consensus.cc:1685] finished
0709 13:21:45.874324 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:21:45.931375 (+ 57051us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3376,"raft.queue_time_us":34762,"raft.run_cpu_time_us":204,"raft.run_wall_time_us":15797,"spinlock_wait_cycles":6082816,"thread_start_us":687304,"threads_started":2,"wal-append.queue_time_us":314367}
I20260709 13:21:46.062592 22202 consensus_queue.cc:237] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 19, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:46.090811 22029 raft_consensus.cc:515] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:46.123253 22088 raft_consensus.cc:493] T 5a959dbecc2648eb85d1cc550f5a3245 P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:21:46.191730 16296 raft_consensus.cc:2468] T 8c11074579b046d99e585c2818e18a2c P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 17.
I20260709 13:21:46.183844 22348 raft_consensus.cc:515] T b6f07842d3aa47858df5665874c86f3a P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:46.186247 16116 leader_election.cc:304] T 8158bbd17b994d2eb96f560bd0f1b88d P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 90f3da57796b488c9087a2f523adf045
I20260709 13:21:46.186182 22197 raft_consensus.cc:515] T 23f82224f4564fea93572e8206bad1fe P 647a343a542d4a139fe106c19095129b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:46.189736 16265 leader_election.cc:304] T 18ad7e3af5504571a69d10c00750a01c P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:21:46.190872 22064 raft_consensus.cc:2749] T 8cc959fb7f704fba9009c0595467d074 P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:21:46.086089 22182 raft_consensus.cc:515] T 3884f864a96d428785895e5463e5728f P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:46.172922 22082 raft_consensus.cc:493] T 1df93779209748b9860f6d893dd3de53 P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:46.201496 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:46.208587 16292 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48684 (request call id 2990) took 2331 ms. Trace:
I20260709 13:21:46.209043 16292 rpcz_store.cc:276] 0709 13:21:43.877369 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:45.752815 (+1875446us) service_pool.cc:224] Handling call
0709 13:21:45.820024 (+ 67209us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:21:45.991394 (+171370us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:21:45.991428 (+    34us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:21:45.992193 (+   765us) log.cc:844] Serialized 52 byte log entry
0709 13:21:46.113047 (+120854us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:21:46.201137 (+ 88090us) log.cc:844] Serialized 14 byte log entry
0709 13:21:46.201607 (+   470us) log.cc:844] Serialized 14 byte log entry
0709 13:21:46.208262 (+  6655us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:46.208334 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:21:46.208368 (+    34us) raft_consensus.cc:1685] finished
0709 13:21:46.208382 (+    14us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:21:46.208531 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6410,"prepare.queue_time_us":80410,"raft.queue_time_us":187,"raft.run_cpu_time_us":223,"raft.run_wall_time_us":223,"thread_start_us":149682,"threads_started":2,"wal-append.queue_time_us":134071}
I20260709 13:21:46.212226 22082 raft_consensus.cc:515] T 1df93779209748b9860f6d893dd3de53 P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:46.212879 16265 leader_election.cc:304] T d16dfad0c3b540e5b4b86492ddf87208 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 647a343a542d4a139fe106c19095129b
I20260709 13:21:46.213724 22404 raft_consensus.cc:2804] T 18ad7e3af5504571a69d10c00750a01c P 90f3da57796b488c9087a2f523adf045 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:21:46.214141 22404 raft_consensus.cc:493] T 18ad7e3af5504571a69d10c00750a01c P 90f3da57796b488c9087a2f523adf045 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:46.214507 22404 raft_consensus.cc:3060] T 18ad7e3af5504571a69d10c00750a01c P 90f3da57796b488c9087a2f523adf045 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:21:46.217379 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:46.218108 16823 raft_consensus.cc:1240] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [term 22 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 21. Current term is 22. Ops: []
I20260709 13:21:46.202878 16224 raft_consensus.cc:2468] T 19baa7bc367844c288bca8f81439c413 P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 19.
I20260709 13:21:46.217715 22246 raft_consensus.cc:2804] T 8158bbd17b994d2eb96f560bd0f1b88d P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:21:46.220682 16293 raft_consensus.cc:2468] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 19.
I20260709 13:21:46.220793 22246 raft_consensus.cc:493] T 8158bbd17b994d2eb96f560bd0f1b88d P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Starting leader election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:21:46.221364 22246 raft_consensus.cc:3060] T 8158bbd17b994d2eb96f560bd0f1b88d P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:21:46.223223 16147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746aabf856ab401c9ef5e859371e7b67" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:46.223358 22236 raft_consensus.cc:515] T 9e65676b21b748728ad39668cfe722ec P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:46.223994 16147 raft_consensus.cc:2468] T 746aabf856ab401c9ef5e859371e7b67 P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 13.
I20260709 13:21:46.225256 16299 raft_consensus.cc:1275] T c9944d5e4b8c4109af9955c4f08c9095 P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 13:21:46.225528 22119 consensus_queue.cc:1048] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:21:46.226295 16148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3884f864a96d428785895e5463e5728f" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 13 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:46.227720 16150 raft_consensus.cc:3060] T 9e65676b21b748728ad39668cfe722ec P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Advancing to term 17
I20260709 13:21:46.228909 16148 raft_consensus.cc:2410] T 3884f864a96d428785895e5463e5728f P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 for term 13 because replica has last-logged OpId of term: 12 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 10.
I20260709 13:21:46.174158 22104 raft_consensus.cc:515] T 02838c86531d4f98bf58d1c22b8baf77 P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:46.217229 22082 leader_election.cc:290] T 1df93779209748b9860f6d893dd3de53 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:21:46.238168 22439 consensus_queue.cc:1059] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 90f3da57796b488c9087a2f523adf045 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.004s
I20260709 13:21:46.255501 22201 raft_consensus.cc:515] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:46.258880 22106 raft_consensus.cc:2804] T d16dfad0c3b540e5b4b86492ddf87208 P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Leader pre-election won for term 18
W20260709 13:21:46.259644 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 47010.8us
I20260709 13:21:46.260219 22408 consensus_queue.cc:1048] T 5a959dbecc2648eb85d1cc550f5a3245 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:21:46.260703 22201 leader_election.cc:290] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 16 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:46.229866 21704 consensus_queue.cc:1048] T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:21:46.267912 16151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:46.273679 22088 raft_consensus.cc:515] T 5a959dbecc2648eb85d1cc550f5a3245 P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:46.277257 16295 raft_consensus.cc:1275] T 8c11074579b046d99e585c2818e18a2c P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
W20260709 13:21:46.278314 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 61662.9us
I20260709 13:21:46.180743 22372 raft_consensus.cc:515] T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:46.175747 22215 raft_consensus.cc:515] T e4dc6b0541ca431ebc757beac00521f1 P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:21:46.283561 16266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 134332us
I20260709 13:21:46.285678 16816 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49892 (request call id 902) took 1580 ms. Trace:
I20260709 13:21:46.200632 22205 consensus_queue.cc:1048] T 8c11074579b046d99e585c2818e18a2c P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.042s
W20260709 13:21:46.290839 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 80969.4us
I20260709 13:21:46.286213 16816 rpcz_store.cc:276] 0709 13:21:44.706002 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:44.706161 (+   159us) service_pool.cc:224] Handling call
0709 13:21:44.706534 (+   373us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:21:46.146520 (+1439986us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:46.228676 (+ 82156us) spinlock_profiling.cc:243] Waited 1.41 s on lock 0x7b54006441d4. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:21:46.285586 (+ 56910us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3252498816}
W20260709 13:21:46.296284 16789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 129516us
I20260709 13:21:46.303390 16820 raft_consensus.cc:1240] T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860 [term 19 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 18. Current term is 19. Ops: []
I20260709 13:21:46.259608 22106 raft_consensus.cc:493] T d16dfad0c3b540e5b4b86492ddf87208 P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:21:46.310703 16217 raft_consensus.cc:1275] T ee9ed67006144797b1de6ea8c2013a68 P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 13:21:46.312167 16217 raft_consensus.cc:1275] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 13:21:46.306842 22106 raft_consensus.cc:3060] T d16dfad0c3b540e5b4b86492ddf87208 P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:21:46.317930 22383 leader_election.cc:290] T ee9ed67006144797b1de6ea8c2013a68 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:46.320248 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746aabf856ab401c9ef5e859371e7b67" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:46.321557 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62fe3346cd3c42c4a55e14af9073bca4" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:21:46.322950 16192 leader_election.cc:304] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:21:46.323513 22439 raft_consensus.cc:3055] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 90f3da57796b488c9087a2f523adf045 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:21:46.323884 22439 raft_consensus.cc:740] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 90f3da57796b488c9087a2f523adf045 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:21:46.325377 16294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68a6838d82ef4cac83507f906139d90f" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:21:46.325691 16293 raft_consensus.cc:1275] T c4f65cdf00b748bb963ee6e81ef43b79 P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 13:21:46.325958 16294 raft_consensus.cc:3060] T 68a6838d82ef4cac83507f906139d90f P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:21:46.326953 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d4ab870a05347868956aa8b841b2981" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:46.327627 16293 raft_consensus.cc:2468] T 8d4ab870a05347868956aa8b841b2981 P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 15.
I20260709 13:21:46.328692 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6f07842d3aa47858df5665874c86f3a" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:21:46.329277 16293 raft_consensus.cc:3060] T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Advancing to term 17
I20260709 13:21:46.330641 22292 raft_consensus.cc:2804] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:21:46.331051 22292 raft_consensus.cc:493] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Starting leader election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:21:46.331444 22292 raft_consensus.cc:3060] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:21:46.331930 22029 leader_election.cc:290] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 22 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:46.333004 16221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5718c623ebee4d2bb770f0f9ddb2f164" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:46.324731 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4dc6b0541ca431ebc757beac00521f1" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:21:46.335872 16299 raft_consensus.cc:3060] T e4dc6b0541ca431ebc757beac00521f1 P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:21:46.324712 16192 leader_election.cc:304] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:21:46.330508 22285 raft_consensus.cc:515] T 84f2da98864b4425bf7c0055951fdd4b P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:46.340293 16116 leader_election.cc:304] T 68a6838d82ef4cac83507f906139d90f P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
I20260709 13:21:46.371050 16293 raft_consensus.cc:2468] T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 17.
I20260709 13:21:46.374567 22197 leader_election.cc:290] T 23f82224f4564fea93572e8206bad1fe P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
W20260709 13:21:46.391180 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 83972.8us
I20260709 13:21:46.391412 22196 raft_consensus.cc:2804] T 68a6838d82ef4cac83507f906139d90f P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:21:46.392180 22196 raft_consensus.cc:697] T 68a6838d82ef4cac83507f906139d90f P 647a343a542d4a139fe106c19095129b [term 17 LEADER]: Becoming Leader. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:21:46.393689 22196 consensus_queue.cc:237] T 68a6838d82ef4cac83507f906139d90f P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:21:46.395351 16192 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 70745.9us
I20260709 13:21:46.396494 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87147731a68400e8cce9b879daa7c5d" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:46.397346 16293 raft_consensus.cc:2468] T b87147731a68400e8cce9b879daa7c5d P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 16.
I20260709 13:21:46.397866 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c61a39fd4f84424b129c1a6f9ba404d" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:46.398381 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a96373bb0e14dce8a266b2fc6c58507" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:46.398599 16224 raft_consensus.cc:2468] T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 17.
I20260709 13:21:46.399575 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66ae176961b74813a58bc15c9c4c6f2a" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:46.400228 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5ad4eb281564936a59f41e387a47156" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:46.400995 16217 raft_consensus.cc:2468] T e5ad4eb281564936a59f41e387a47156 P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 16.
I20260709 13:21:46.401407 16223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74ca8140f35c4ae29e8a664f6bdfc829" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:46.401842 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ab0ac8347641e6955e9bce7d90c1a2" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:46.402609 16824 raft_consensus.cc:1240] T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860 [term 19 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 18. Current term is 19. Ops: []
I20260709 13:21:46.403820 16297 raft_consensus.cc:1275] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 13:21:46.400991 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806d7690c2b644959152074bb2cab0a7" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:46.397272 16291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57a744f5aaf944678a63469cbf514015" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:46.423995 16300 raft_consensus.cc:1275] T 1df93779209748b9860f6d893dd3de53 P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 13:21:46.424693 16291 raft_consensus.cc:2468] T 57a744f5aaf944678a63469cbf514015 P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 12.
W20260709 13:21:46.438817 16115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 304248us
I20260709 13:21:46.447342 16219 raft_consensus.cc:1240] T 84f2da98864b4425bf7c0055951fdd4b P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 13. Current term is 14. Ops: []
I20260709 13:21:46.399487 22445 raft_consensus.cc:2804] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:21:46.446805 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18ad7e3af5504571a69d10c00750a01c" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:21:46.480443 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18ad7e3af5504571a69d10c00750a01c" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:21:46.481185 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e9b2bf7011f4ae688a79d7bb5a447b4" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:21:46.492420 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3884f864a96d428785895e5463e5728f" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 13 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:46.567766 16825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2372981006d4b218215c28964dea623" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:46.567610 16297 raft_consensus.cc:3060] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:21:46.509881 16820 raft_consensus.cc:1275] T 19baa7bc367844c288bca8f81439c413 P cb82cc7bd6db46d980c75237a1402860 [term 19 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 13:21:46.764233 16296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26354361e9b044cea1c2b86c4882fe92" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:46.549392 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a96373bb0e14dce8a266b2fc6c58507" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:46.769119 16224 raft_consensus.cc:2468] T 5a96373bb0e14dce8a266b2fc6c58507 P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 13.
I20260709 13:21:46.769802 16296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c74490c40243a88e62d1760b5cd814" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:21:46.770171 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "829b4ad8c5a54eababeea1181b960d4b" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:46.771135 16296 raft_consensus.cc:2393] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 15: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
I20260709 13:21:46.771567 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdbd78acb4cf46298e0de8ba0d130e19" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:46.600694 16822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "829b4ad8c5a54eababeea1181b960d4b" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:46.776597 22457 consensus_queue.cc:1048] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:21:46.777673 16822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee9ed67006144797b1de6ea8c2013a68" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:46.778960 16822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5d59407eb545b895d8d922e0dd69fd" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:46.779758 16822 raft_consensus.cc:2393] T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 19: Already voted for candidate cb82cc7bd6db46d980c75237a1402860 in this term.
I20260709 13:21:46.548970 16824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26354361e9b044cea1c2b86c4882fe92" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:46.537755 16291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ab0ac8347641e6955e9bce7d90c1a2" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:46.785940 16291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0eab25ce4c64e018a041beeda2867db" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:46.786778 16291 raft_consensus.cc:2468] T c0eab25ce4c64e018a041beeda2867db P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 16.
I20260709 13:21:46.798640 16291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e65676b21b748728ad39668cfe722ec" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:21:46.572245 16821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31969392ccdb43b491911a319e8bc7e1" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:46.560454 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6f07842d3aa47858df5665874c86f3a" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:21:46.839583 16225 raft_consensus.cc:3060] T b6f07842d3aa47858df5665874c86f3a P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:21:46.773736 22168 consensus_queue.cc:1059] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: INVALID_TERM, Last received: 21.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:21:46.775205 22197 consensus_queue.cc:1048] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:21:46.842244 22168 raft_consensus.cc:3055] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:21:46.560411 22408 raft_consensus.cc:493] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:46.458834 22088 leader_election.cc:290] T 5a959dbecc2648eb85d1cc550f5a3245 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:46.560011 16115 leader_election.cc:304] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac
I20260709 13:21:46.763233 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada5a004ba7c46819051e3ccec71ffdb" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:46.763636 16823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66f75124215f4ca6bef831e174a792d8" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:46.767387 16825 raft_consensus.cc:2468] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 19.
I20260709 13:21:46.635196 16818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b7d1ab519044eda2d5e04a07242f25" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:46.660122 16817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746aabf856ab401c9ef5e859371e7b67" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:46.807628 16291 raft_consensus.cc:3060] T 9e65676b21b748728ad39668cfe722ec P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:21:46.821163 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66f75124215f4ca6bef831e174a792d8" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:46.772251 16296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f82224f4564fea93572e8206bad1fe" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:46.830643 16819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:46.834065 16292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a878999c0024d2abe517484180290fc" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:46.613380 16217 raft_consensus.cc:3060] T 18ad7e3af5504571a69d10c00750a01c P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:21:46.842715 22168 raft_consensus.cc:740] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:21:46.492192 22445 raft_consensus.cc:493] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:46.860677 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b7d1ab519044eda2d5e04a07242f25" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
W20260709 13:21:46.853677 22431 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.678s	user 0.024s	sys 0.010s
I20260709 13:21:46.880162 16292 raft_consensus.cc:2468] T 8a878999c0024d2abe517484180290fc P 90f3da57796b488c9087a2f523adf045 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 14.
W20260709 13:21:46.884097 16299 consensus_meta.cc:287] T e4dc6b0541ca431ebc757beac00521f1 P 90f3da57796b488c9087a2f523adf045: Time spent flushing consensus metadata: real 0.523s	user 0.009s	sys 0.001s
I20260709 13:21:46.884606 16299 raft_consensus.cc:2468] T e4dc6b0541ca431ebc757beac00521f1 P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 17.
I20260709 13:21:46.837597 22372 leader_election.cc:290] T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 19 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:46.885456 16148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6f07842d3aa47858df5665874c86f3a" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:46.890903 16821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3884f864a96d428785895e5463e5728f" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:46.892243 16821 raft_consensus.cc:2468] T 3884f864a96d428785895e5463e5728f P cb82cc7bd6db46d980c75237a1402860 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 12.
W20260709 13:21:46.873172 22431 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.701s	user 0.024s	sys 0.010s
W20260709 13:21:46.879261 22106 consensus_meta.cc:287] T d16dfad0c3b540e5b4b86492ddf87208 P 90f3da57796b488c9087a2f523adf045: Time spent flushing consensus metadata: real 0.557s	user 0.009s	sys 0.002s
I20260709 13:21:46.871963 22445 raft_consensus.cc:3060] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:21:46.915937 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4f65cdf00b748bb963ee6e81ef43b79" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:46.843449 22408 raft_consensus.cc:515] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:46.916663 16295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7ca61592e848ea9b0fb81a2ce33cdc" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:46.916700 16299 raft_consensus.cc:2468] T c4f65cdf00b748bb963ee6e81ef43b79 P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 18.
I20260709 13:21:46.909155 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c74490c40243a88e62d1760b5cd814" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:46.929796 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806d7690c2b644959152074bb2cab0a7" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 18 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:46.890501 22197 raft_consensus.cc:2804] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:21:46.937525 16821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434bff3082d141838068f323bdb0d1ed" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
W20260709 13:21:46.938519 16115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 454434us
I20260709 13:21:46.926757 16816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f82224f4564fea93572e8206bad1fe" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:46.937734 16145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5ad4eb281564936a59f41e387a47156" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:46.950865 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746aabf856ab401c9ef5e859371e7b67" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:46.952497 16143 raft_consensus.cc:1240] T ecc1e363230043fe9fef390f0b979f20 P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 18. Current term is 19. Ops: [18.12-18.12]
I20260709 13:21:46.952630 22349 consensus_queue.cc:1048] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:21:46.954267 16143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806d7690c2b644959152074bb2cab0a7" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:46.948411 16821 raft_consensus.cc:2468] T 434bff3082d141838068f323bdb0d1ed P cb82cc7bd6db46d980c75237a1402860 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 15.
I20260709 13:21:46.955317 16151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a878999c0024d2abe517484180290fc" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:46.958241 16300 raft_consensus.cc:2410] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b for term 18 because replica has last-logged OpId of term: 14 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 9.
I20260709 13:21:46.957013 16143 raft_consensus.cc:2468] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 17.
I20260709 13:21:46.966435 16148 raft_consensus.cc:2368] T b6f07842d3aa47858df5665874c86f3a P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 for earlier term 16. Current term is 17.
I20260709 13:21:46.968158 16147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f5f89186204242bf377f7c42b2e2da" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:46.969396 16147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26354361e9b044cea1c2b86c4882fe92" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:46.969954 16147 raft_consensus.cc:2468] T 26354361e9b044cea1c2b86c4882fe92 P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 13.
I20260709 13:21:46.970750 16147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0eab25ce4c64e018a041beeda2867db" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:46.963236 16816 raft_consensus.cc:2468] T 23f82224f4564fea93572e8206bad1fe P cb82cc7bd6db46d980c75237a1402860 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 14.
I20260709 13:21:47.002240 16818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e65676b21b748728ad39668cfe722ec" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:47.002439 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee9ed67006144797b1de6ea8c2013a68" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:47.004254 16224 raft_consensus.cc:2468] T ee9ed67006144797b1de6ea8c2013a68 P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 17.
I20260709 13:21:46.934350 22197 raft_consensus.cc:493] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Starting leader election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:21:47.005977 16818 raft_consensus.cc:2368] T 9e65676b21b748728ad39668cfe722ec P cb82cc7bd6db46d980c75237a1402860 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b for earlier term 16. Current term is 17.
W20260709 13:21:47.015734 16150 consensus_meta.cc:287] T 9e65676b21b748728ad39668cfe722ec P 647a343a542d4a139fe106c19095129b: Time spent flushing consensus metadata: real 0.733s	user 0.010s	sys 0.000s
I20260709 13:21:47.016202 16150 raft_consensus.cc:2468] T 9e65676b21b748728ad39668cfe722ec P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 17.
I20260709 13:21:46.917497 22106 raft_consensus.cc:515] T d16dfad0c3b540e5b4b86492ddf87208 P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:47.020345 16295 raft_consensus.cc:1275] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 13:21:46.977620 22215 leader_election.cc:290] T e4dc6b0541ca431ebc757beac00521f1 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:21:47.009639 22197 raft_consensus.cc:3060] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:21:47.031893 16816 raft_consensus.cc:1240] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860 [term 16 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 15. Current term is 16. Ops: [15.12-15.12]
W20260709 13:21:47.052536 16192 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 111897us
I20260709 13:21:47.019143 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f5f89186204242bf377f7c42b2e2da" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
W20260709 13:21:47.055668 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 147553us
I20260709 13:21:47.009410 22285 leader_election.cc:290] T 84f2da98864b4425bf7c0055951fdd4b P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 14 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:47.058087 16143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c74490c40243a88e62d1760b5cd814" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:47.065346 16296 raft_consensus.cc:1275] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 13:21:47.064913 16143 raft_consensus.cc:2468] T 08c74490c40243a88e62d1760b5cd814 P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 15.
W20260709 13:21:47.090101 22468 log.cc:927] Time spent T 00000000000000000000000000000000 P 667a34b27a39474ebefc1af3cf5b4271: Append to log took a long time: real 0.119s	user 0.001s	sys 0.001s
W20260709 13:21:47.094444 16189 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:47.099567 16816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8638ee363273426db1393dd81e5c7b72" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:47.101112 16822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2794af5b99984efebdee564035841cb0" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:47.102597 16822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57a744f5aaf944678a63469cbf514015" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:47.106078 16820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9944d5e4b8c4109af9955c4f08c9095" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
W20260709 13:21:47.110049 22467 log.cc:927] Time spent T 829b4ad8c5a54eababeea1181b960d4b P cb82cc7bd6db46d980c75237a1402860: Append to log took a long time: real 0.181s	user 0.001s	sys 0.000s
I20260709 13:21:47.111791 22182 leader_election.cc:290] T 3884f864a96d428785895e5463e5728f P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:47.112718 22236 leader_election.cc:290] T 9e65676b21b748728ad39668cfe722ec P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:47.114646 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5d59407eb545b895d8d922e0dd69fd" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:47.118760 16222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:47.174618 16291 raft_consensus.cc:2468] T 9e65676b21b748728ad39668cfe722ec P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 17.
W20260709 13:21:47.121189 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 815367us
I20260709 13:21:47.083545 22168 consensus_queue.cc:260] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 21.14, Last appended by leader: 14, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:47.122658 22436 leader_election.cc:290] T 8c11074579b046d99e585c2818e18a2c P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:47.190289 16816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c11074579b046d99e585c2818e18a2c" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:47.128159 22152 leader_election.cc:290] T 19baa7bc367844c288bca8f81439c413 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:47.191555 22152 consensus_queue.cc:1059] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: INVALID_TERM, Last received: 21.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:21:47.192503 22152 consensus_queue.cc:1059] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: INVALID_TERM, Last received: 18.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:21:47.194154 22152 raft_consensus.cc:3055] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:21:47.194538 22152 raft_consensus.cc:740] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:21:47.088454 16144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f2da98864b4425bf7c0055951fdd4b" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:47.196355 16822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:47.196745 16222 raft_consensus.cc:2393] T 19baa7bc367844c288bca8f81439c413 P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 19: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
I20260709 13:21:47.173091 16116 leader_election.cc:304] T 5a96373bb0e14dce8a266b2fc6c58507 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
W20260709 13:21:47.129565 22246 consensus_meta.cc:287] T 8158bbd17b994d2eb96f560bd0f1b88d P 647a343a542d4a139fe106c19095129b: Time spent flushing consensus metadata: real 0.908s	user 0.010s	sys 0.002s
W20260709 13:21:47.131271 22465 log.cc:927] Time spent T 5a96373bb0e14dce8a266b2fc6c58507 P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 0.111s	user 0.001s	sys 0.000s
I20260709 13:21:47.006312 22349 consensus_queue.cc:1059] T e4dc6b0541ca431ebc757beac00521f1 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.062s
I20260709 13:21:47.132364 16147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68a6838d82ef4cac83507f906139d90f" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:47.135738 22104 leader_election.cc:290] T 02838c86531d4f98bf58d1c22b8baf77 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 18 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:21:47.137470 22106 leader_election.cc:290] T d16dfad0c3b540e5b4b86492ddf87208 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 18 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:21:47.146023 16819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26354361e9b044cea1c2b86c4882fe92" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:47.146907 16817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87147731a68400e8cce9b879daa7c5d" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:47.209425 16817 raft_consensus.cc:2468] T b87147731a68400e8cce9b879daa7c5d P cb82cc7bd6db46d980c75237a1402860 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 17.
I20260709 13:21:47.148285 16824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31969392ccdb43b491911a319e8bc7e1" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
W20260709 13:21:47.149880 16115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 109927us
W20260709 13:21:47.155489 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 71036.4us
W20260709 13:21:47.038162 22404 consensus_meta.cc:287] T 18ad7e3af5504571a69d10c00750a01c P 90f3da57796b488c9087a2f523adf045: Time spent flushing consensus metadata: real 0.821s	user 0.012s	sys 0.000s
I20260709 13:21:47.090955 16224 raft_consensus.cc:2468] T c2f5f89186204242bf377f7c42b2e2da P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 17.
I20260709 13:21:47.130812 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d4ab870a05347868956aa8b841b2981" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:47.206509 16146 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46224 (request call id 1000) took 1043 ms. Trace:
I20260709 13:21:47.159369 16225 raft_consensus.cc:2468] T b6f07842d3aa47858df5665874c86f3a P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 17.
W20260709 13:21:47.164614 16294 consensus_meta.cc:287] T 68a6838d82ef4cac83507f906139d90f P 90f3da57796b488c9087a2f523adf045: Time spent flushing consensus metadata: real 0.838s	user 0.010s	sys 0.000s
W20260709 13:21:47.172895 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 304480us
I20260709 13:21:47.215369 16294 raft_consensus.cc:2468] T 68a6838d82ef4cac83507f906139d90f P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 17.
I20260709 13:21:47.180114 16297 raft_consensus.cc:2468] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 16.
I20260709 13:21:47.174489 16226 raft_consensus.cc:2468] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 19.
I20260709 13:21:47.183640 16143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2794af5b99984efebdee564035841cb0" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:47.183743 16820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdbd78acb4cf46298e0de8ba0d130e19" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:47.187470 22348 leader_election.cc:290] T b6f07842d3aa47858df5665874c86f3a P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:47.188478 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a378d7f796f94615b3b347119a9880fd" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:21:47.189767 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66ae176961b74813a58bc15c9c4c6f2a" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:21:47.225327 16222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9944d5e4b8c4109af9955c4f08c9095" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:47.199935 16116 leader_election.cc:304] T 829b4ad8c5a54eababeea1181b960d4b P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860
I20260709 13:21:47.226621 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66ae176961b74813a58bc15c9c4c6f2a" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:21:47.201802 16217 raft_consensus.cc:2468] T 18ad7e3af5504571a69d10c00750a01c P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 15.
I20260709 13:21:47.197118 16822 raft_consensus.cc:2468] T 19baa7bc367844c288bca8f81439c413 P cb82cc7bd6db46d980c75237a1402860 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 19.
W20260709 13:21:47.159730 22292 consensus_meta.cc:287] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac: Time spent flushing consensus metadata: real 0.828s	user 0.011s	sys 0.001s
I20260709 13:21:47.214336 16146 rpcz_store.cc:276] 0709 13:21:46.163206 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:46.173867 (+ 10661us) service_pool.cc:224] Handling call
0709 13:21:46.198747 (+ 24880us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:21:46.218298 (+ 19551us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 13:21:46.568439 (+350141us) log.cc:844] Serialized 14 byte log entry
0709 13:21:47.205857 (+637418us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:21:47.205986 (+   129us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:21:47.206066 (+    80us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:47.206192 (+   126us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:21:47.206399 (+   207us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3445,"prepare.queue_time_us":46703,"prepare.run_cpu_time_us":30,"prepare.run_wall_time_us":31,"raft.queue_time_us":5858,"spinlock_wait_cycles":1048960,"thread_start_us":653954,"threads_started":2}
I20260709 13:21:47.203222 22236 raft_consensus.cc:2804] T 5a96373bb0e14dce8a266b2fc6c58507 P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 13:21:47.206600 16265 connection.cc:256] RPC call timeout handler was delayed by -1.1295686979999573s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.30800000000000005s for a response
I20260709 13:21:47.214918 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6f07842d3aa47858df5665874c86f3a" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:47.261232 22348 raft_consensus.cc:3055] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac [term 18 LEADER]: Stepping down as leader of term 18
W20260709 13:21:47.258603 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 58752.4us
I20260709 13:21:47.261538 16225 raft_consensus.cc:2368] T b6f07842d3aa47858df5665874c86f3a P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 for earlier term 16. Current term is 17.
I20260709 13:21:47.261701 22348 raft_consensus.cc:740] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:21:47.262629 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2372981006d4b218215c28964dea623" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:47.263106 16116 leader_election.cc:304] T ee9ed67006144797b1de6ea8c2013a68 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: cb82cc7bd6db46d980c75237a1402860
I20260709 13:21:47.262833 22348 consensus_queue.cc:260] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:47.263953 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8638ee363273426db1393dd81e5c7b72" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:47.264320 22348 raft_consensus.cc:3060] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:21:47.264691 16225 raft_consensus.cc:2468] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 17.
I20260709 13:21:47.265362 21704 raft_consensus.cc:2804] T ee9ed67006144797b1de6ea8c2013a68 P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:21:47.265755 21704 raft_consensus.cc:493] T ee9ed67006144797b1de6ea8c2013a68 P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:47.259893 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e4bfb9090c8460eb14fa7e0d818e7f1" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:47.201124 16117 leader_election.cc:304] T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:21:47.265724 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a378d7f796f94615b3b347119a9880fd" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:21:47.277745 16190 leader_election.cc:304] T 7c2095a5f4ad49408dbc7f69c606120d P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
W20260709 13:21:47.278625 16117 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 77573.2us
I20260709 13:21:47.258327 16144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e9b2bf7011f4ae688a79d7bb5a447b4" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:21:47.279592 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a959dbecc2648eb85d1cc550f5a3245" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:47.279820 16150 raft_consensus.cc:2468] T 8d4ab870a05347868956aa8b841b2981 P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 15.
I20260709 13:21:47.281297 16117 leader_election.cc:304] T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860
I20260709 13:21:47.282299 16790 leader_election.cc:304] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
W20260709 13:21:47.283429 16266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 229636us
I20260709 13:21:47.284013 16147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87147731a68400e8cce9b879daa7c5d" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:47.284242 16823 raft_consensus.cc:1240] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860 [term 19 LEADER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 18. Current term is 19. Ops: [18.12-18.12]
I20260709 13:21:47.284369 16146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57a744f5aaf944678a63469cbf514015" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:47.217430 22082 raft_consensus.cc:493] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:47.325560 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a878999c0024d2abe517484180290fc" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:47.232064 22349 consensus_queue.cc:1243] T e4dc6b0541ca431ebc757beac00521f1 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Peer 647a343a542d4a139fe106c19095129b log is divergent from this leader: its last log entry 15.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
W20260709 13:21:47.332080 16117 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 50889.2us
I20260709 13:21:47.276978 21704 raft_consensus.cc:3060] T ee9ed67006144797b1de6ea8c2013a68 P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:21:47.335350 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57a744f5aaf944678a63469cbf514015" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:47.204037 22285 consensus_queue.cc:1059] T 1df93779209748b9860f6d893dd3de53 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.081s
I20260709 13:21:47.328006 16226 raft_consensus.cc:2468] T 8a878999c0024d2abe517484180290fc P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 14.
W20260709 13:21:47.330583 16218 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.679s	user 0.008s	sys 0.009s
I20260709 13:21:47.204272 22246 raft_consensus.cc:515] T 8158bbd17b994d2eb96f560bd0f1b88d P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:47.333030 16117 leader_election.cc:304] T 746aabf856ab401c9ef5e859371e7b67 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860
I20260709 13:21:47.333222 16143 raft_consensus.cc:2468] T 2794af5b99984efebdee564035841cb0 P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 13.
W20260709 13:21:47.339279 16218 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.691s	user 0.009s	sys 0.010s
I20260709 13:21:47.334743 16217 raft_consensus.cc:2468] T 5a959dbecc2648eb85d1cc550f5a3245 P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 15.
I20260709 13:21:47.260756 22236 raft_consensus.cc:493] T 5a96373bb0e14dce8a266b2fc6c58507 P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Starting leader election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:21:47.277621 22182 raft_consensus.cc:2749] T 829b4ad8c5a54eababeea1181b960d4b P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:21:47.340343 16266 leader_election.cc:304] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:21:47.226085 22404 raft_consensus.cc:515] T 18ad7e3af5504571a69d10c00750a01c P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } }
I20260709 13:21:47.348598 16143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5718c623ebee4d2bb770f0f9ddb2f164" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:47.232328 22372 consensus_queue.cc:1048] T 3a7ca61592e848ea9b0fb81a2ce33cdc P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.100s
I20260709 13:21:47.327682 16220 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33050 (request call id 1076) took 1107 ms. Trace:
I20260709 13:21:47.256911 22431 consensus_queue.cc:1048] T ee9ed67006144797b1de6ea8c2013a68 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.082s
I20260709 13:21:47.339538 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c74490c40243a88e62d1760b5cd814" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:47.349804 16222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0eab25ce4c64e018a041beeda2867db" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:47.350790 16147 raft_consensus.cc:2468] T b87147731a68400e8cce9b879daa7c5d P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 17.
I20260709 13:21:47.259634 22292 raft_consensus.cc:515] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:21:47.345866 16115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 61926.2us
I20260709 13:21:47.333845 22196 raft_consensus.cc:2749] T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:21:47.361461 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 78536.6us
I20260709 13:21:47.355058 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f2da98864b4425bf7c0055951fdd4b" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:47.355945 16146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e65676b21b748728ad39668cfe722ec" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:47.356740 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c2095a5f4ad49408dbc7f69c606120d" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 14 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:21:47.336319 22202 raft_consensus.cc:2749] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:21:47.377517 16146 raft_consensus.cc:2393] T 9e65676b21b748728ad39668cfe722ec P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 17: Already voted for candidate cb82cc7bd6db46d980c75237a1402860 in this term.
I20260709 13:21:47.354571 16222 raft_consensus.cc:2468] T c0eab25ce4c64e018a041beeda2867db P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 16.
I20260709 13:21:47.358410 16143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31969392ccdb43b491911a319e8bc7e1" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
W20260709 13:21:47.359306 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 101549us
I20260709 13:21:47.351588 16220 rpcz_store.cc:276] 0709 13:21:46.219954 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:46.220106 (+   152us) service_pool.cc:224] Handling call
0709 13:21:46.239419 (+ 19313us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:21:46.371072 (+131653us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 13:21:46.824272 (+453200us) log.cc:844] Serialized 14 byte log entry
0709 13:21:47.010659 (+186387us) log.cc:844] Serialized 14 byte log entry
0709 13:21:47.327264 (+316605us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:21:47.327321 (+    57us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:21:47.327379 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:47.327450 (+    71us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:21:47.327607 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":792068,"prepare.run_cpu_time_us":368,"prepare.run_wall_time_us":9111,"raft.queue_time_us":290211,"raft.run_cpu_time_us":186,"raft.run_wall_time_us":186,"spinlock_wait_cycles":91023744,"thread_start_us":836110,"threads_started":3,"wal-append.queue_time_us":378323}
I20260709 13:21:47.365752 22439 consensus_queue.cc:260] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 90f3da57796b488c9087a2f523adf045 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.15, Last appended by leader: 15, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:47.367483 22481 raft_consensus.cc:493] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:21:47.353533 22455 log.cc:927] Time spent T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 1.150s	user 0.008s	sys 0.019s
I20260709 13:21:47.353199 22106 raft_consensus.cc:2804] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Leader pre-election won for term 18
W20260709 13:21:47.369043 22475 log.cc:927] Time spent T d16dfad0c3b540e5b4b86492ddf87208 P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 0.173s	user 0.000s	sys 0.000s
W20260709 13:21:47.370158 22399 consensus_meta.cc:287] T ada5a004ba7c46819051e3ccec71ffdb P 90f3da57796b488c9087a2f523adf045: Time spent flushing consensus metadata: real 1.236s	user 0.007s	sys 0.004s
I20260709 13:21:47.325109 22152 consensus_queue.cc:260] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 21.14, Last appended by leader: 14, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:47.343603 22349 raft_consensus.cc:493] T ecc1e363230043fe9fef390f0b979f20 P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:47.372537 16115 leader_election.cc:304] T 2794af5b99984efebdee564035841cb0 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: cb82cc7bd6db46d980c75237a1402860
I20260709 13:21:47.373005 22292 leader_election.cc:290] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 17 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:47.373620 16147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1df93779209748b9860f6d893dd3de53" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 17 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:47.332266 22082 raft_consensus.cc:515] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:47.375633 16221 raft_consensus.cc:2393] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 in current term 17: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
W20260709 13:21:47.397176 16266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 43895.5us
W20260709 13:21:47.396826 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 179164us
I20260709 13:21:47.378945 16265 leader_election.cc:304] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
I20260709 13:21:47.348284 22236 raft_consensus.cc:3060] T 5a96373bb0e14dce8a266b2fc6c58507 P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:21:47.388985 16142 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.818s	user 0.002s	sys 0.023s
W20260709 13:21:47.475509 16142 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.914s	user 0.002s	sys 0.023s
I20260709 13:21:47.470252 16790 leader_election.cc:304] T 23f82224f4564fea93572e8206bad1fe P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:21:47.431193 22106 raft_consensus.cc:493] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:21:47.430235 16298 raft_consensus.cc:2393] T 18ad7e3af5504571a69d10c00750a01c P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 15: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
I20260709 13:21:47.437258 16225 raft_consensus.cc:2468] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 19.
I20260709 13:21:47.438606 16146 raft_consensus.cc:1240] T 8158bbd17b994d2eb96f560bd0f1b88d P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 15. Current term is 16. Ops: []
I20260709 13:21:47.478644 16790 leader_election.cc:304] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac
W20260709 13:21:47.396086 22460 log.cc:927] Time spent T 02838c86531d4f98bf58d1c22b8baf77 P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 13:21:47.339196 22383 raft_consensus.cc:2804] T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:21:47.403965 22024 leader_election.cc:290] T 434bff3082d141838068f323bdb0d1ed P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:47.442965 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68a6838d82ef4cac83507f906139d90f" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
W20260709 13:21:47.444377 16117 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 111482us
I20260709 13:21:47.376209 22285 consensus_queue.cc:1243] T 1df93779209748b9860f6d893dd3de53 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Peer 647a343a542d4a139fe106c19095129b log is divergent from this leader: its last log entry 16.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 13:21:47.396921 22182 raft_consensus.cc:2749] T 746aabf856ab401c9ef5e859371e7b67 P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:21:47.449043 16222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c2095a5f4ad49408dbc7f69c606120d" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 14 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:21:47.396543 22480 raft_consensus.cc:3055] T e4dc6b0541ca431ebc757beac00521f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:21:47.473691 16817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5718c623ebee4d2bb770f0f9ddb2f164" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:21:47.477563 22088 raft_consensus.cc:2804] T 2794af5b99984efebdee564035841cb0 P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:21:47.420069 16143 raft_consensus.cc:2468] T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 15.
I20260709 13:21:47.381744 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d4ab870a05347868956aa8b841b2981" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:47.434391 22436 raft_consensus.cc:493] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:21:47.496261 16220 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57604 (request call id 3154) took 1088 ms. Trace:
I20260709 13:21:47.497970 16220 rpcz_store.cc:276] 0709 13:21:46.403618 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:47.440435 (+1036817us) service_pool.cc:224] Handling call
0709 13:21:47.468227 (+ 27792us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:21:47.480872 (+ 12645us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:21:47.480900 (+    28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:21:47.488077 (+  7177us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:21:47.492063 (+  3986us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:47.492130 (+    67us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:21:47.492238 (+   108us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:21:47.447832 22399 raft_consensus.cc:515] T ada5a004ba7c46819051e3ccec71ffdb P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:21:47.480950 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 102074us
I20260709 13:21:47.405478 22197 raft_consensus.cc:515] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:47.449790 22348 raft_consensus.cc:2749] T 7c2095a5f4ad49408dbc7f69c606120d P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:21:47.488391 16217 raft_consensus.cc:2393] T 68a6838d82ef4cac83507f906139d90f P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 17: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
I20260709 13:21:47.488088 16790 leader_election.cc:304] T 9e65676b21b748728ad39668cfe722ec P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:21:47.489019 16117 leader_election.cc:304] T ecc1e363230043fe9fef390f0b979f20 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860
W20260709 13:21:47.484890 16789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 267907us
I20260709 13:21:47.440806 22104 consensus_queue.cc:1059] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 90f3da57796b488c9087a2f523adf045 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: INVALID_TERM, Last received: 17.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.045s
I20260709 13:21:47.541172 22104 consensus_queue.cc:1243] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 90f3da57796b488c9087a2f523adf045 [NON_LEADER]: Peer cb82cc7bd6db46d980c75237a1402860 log is divergent from this leader: its last log entry 19.15 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 14
I20260709 13:21:47.472754 16190 leader_election.cc:304] T 5a959dbecc2648eb85d1cc550f5a3245 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:21:47.481848 22106 raft_consensus.cc:3060] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:21:47.487461 22445 consensus_meta.cc:287] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac: Time spent flushing consensus metadata: real 0.545s	user 0.009s	sys 0.002s
I20260709 13:21:47.543869 22445 raft_consensus.cc:515] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:47.544682 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c11074579b046d99e585c2818e18a2c" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:47.545184 16223 raft_consensus.cc:2393] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 18: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
I20260709 13:21:47.545301 16217 raft_consensus.cc:2468] T 8c11074579b046d99e585c2818e18a2c P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 17.
I20260709 13:21:47.545993 22445 leader_election.cc:290] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 18 election: Requested vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:47.546581 22445 raft_consensus.cc:2749] T 5a959dbecc2648eb85d1cc550f5a3245 P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:21:47.478268 22481 raft_consensus.cc:515] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:21:47.547694 22460 log.cc:927] Time spent T 02838c86531d4f98bf58d1c22b8baf77 P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
I20260709 13:21:47.548484 22436 raft_consensus.cc:515] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:47.549321 22182 raft_consensus.cc:2749] T ecc1e363230043fe9fef390f0b979f20 P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:21:47.495100 22474 consensus_queue.cc:1048] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.057s
I20260709 13:21:47.492597 22202 raft_consensus.cc:2804] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:21:47.550940 22202 raft_consensus.cc:493] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:21:47.551357 22202 raft_consensus.cc:3060] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:21:47.493083 22480 raft_consensus.cc:740] T e4dc6b0541ca431ebc757beac00521f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:21:47.552639 22197 leader_election.cc:290] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 18 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:47.553357 22480 consensus_queue.cc:260] T e4dc6b0541ca431ebc757beac00521f1 P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 16.8, Last appended by leader: 8, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:47.482686 22383 raft_consensus.cc:493] T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:47.557336 22383 raft_consensus.cc:3060] T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:21:47.503017 21623 raft_consensus.cc:2804] T 23f82224f4564fea93572e8206bad1fe P cb82cc7bd6db46d980c75237a1402860 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:21:47.508221 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1df93779209748b9860f6d893dd3de53" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 17 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:47.520759 16149 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40830 (request call id 1873) took 1223 ms. Trace:
I20260709 13:21:47.495465 22088 raft_consensus.cc:493] T 2794af5b99984efebdee564035841cb0 P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Starting leader election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
W20260709 13:21:47.666390 16264 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:47.487176 22152 raft_consensus.cc:3060] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:21:47.539561 22482 raft_consensus.cc:2749] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:21:47.539754 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 51740.3us
I20260709 13:21:47.541213 22404 leader_election.cc:290] T 18ad7e3af5504571a69d10c00750a01c P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 15 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:21:47.504349 22285 raft_consensus.cc:1064] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac: attempting to promote NON_VOTER cb82cc7bd6db46d980c75237a1402860 to VOTER
I20260709 13:21:47.546739 16217 raft_consensus.cc:1275] T 19baa7bc367844c288bca8f81439c413 P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 13:21:47.554890 22480 raft_consensus.cc:3060] T e4dc6b0541ca431ebc757beac00521f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:21:47.658501 16149 rpcz_store.cc:276] 0709 13:21:46.298496 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:46.310148 (+ 11652us) service_pool.cc:224] Handling call
0709 13:21:46.310508 (+   360us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:21:46.310879 (+   371us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:21:46.310910 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:21:46.391481 (+ 80571us) log.cc:844] Serialized 26 byte log entry
0709 13:21:46.859534 (+468053us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:21:46.859651 (+   117us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:46.859799 (+   148us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:21:47.520490 (+660691us) raft_consensus.cc:1685] finished
0709 13:21:47.520515 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:21:47.520673 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":7742,"raft.run_cpu_time_us":123,"raft.run_wall_time_us":120,"thread_start_us":467599,"threads_started":1,"wal-append.queue_time_us":867840}
I20260709 13:21:47.666380 22088 raft_consensus.cc:3060] T 2794af5b99984efebdee564035841cb0 P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:21:47.670120 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 105586us
I20260709 13:21:47.702965 16217 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57604 (request call id 3155) took 1283 ms. Trace:
I20260709 13:21:47.500363 16218 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33050 (request call id 1074) took 1316 ms. Trace:
I20260709 13:21:47.656391 21623 raft_consensus.cc:493] T 23f82224f4564fea93572e8206bad1fe P cb82cc7bd6db46d980c75237a1402860 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:21:47.710260 16220 raft_consensus.cc:3055] T 1df93779209748b9860f6d893dd3de53 P 1ca2725d290e47d68b1a4b609c9816ac [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:21:47.768601 16220 raft_consensus.cc:740] T 1df93779209748b9860f6d893dd3de53 P 1ca2725d290e47d68b1a4b609c9816ac [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:21:47.710527 16217 rpcz_store.cc:276] 0709 13:21:46.403913 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:47.546140 (+1142227us) service_pool.cc:224] Handling call
0709 13:21:47.546436 (+   296us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:21:47.687017 (+140581us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:47.687310 (+   293us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:21:47.788914 22431 raft_consensus.cc:2804] T 9e65676b21b748728ad39668cfe722ec P cb82cc7bd6db46d980c75237a1402860 [term 17 FOLLOWER]: Leader election won for term 17
W20260709 13:21:47.789340 16117 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 300377us
W20260709 13:21:47.789922 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 317329us
I20260709 13:21:47.493031 22349 raft_consensus.cc:515] T ecc1e363230043fe9fef390f0b979f20 P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:47.759516 16265 leader_election.cc:304] T 26354361e9b044cea1c2b86c4882fe92 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
W20260709 13:21:47.797287 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 128263us
W20260709 13:21:47.797746 16192 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 352104us
I20260709 13:21:47.801195 22399 leader_election.cc:290] T ada5a004ba7c46819051e3ccec71ffdb P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 18 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:47.716826 22483 consensus_queue.cc:1059] T ecc1e363230043fe9fef390f0b979f20 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: INVALID_TERM, Last received: 17.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.101s
I20260709 13:21:47.788997 22481 leader_election.cc:290] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:47.806535 22504 raft_consensus.cc:493] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P cb82cc7bd6db46d980c75237a1402860 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
W20260709 13:21:47.807399 16115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 434949us
I20260709 13:21:47.813235 22349 leader_election.cc:290] T ecc1e363230043fe9fef390f0b979f20 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:47.789655 22431 raft_consensus.cc:697] T 9e65676b21b748728ad39668cfe722ec P cb82cc7bd6db46d980c75237a1402860 [term 17 LEADER]: Becoming Leader. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:21:47.824084 22483 raft_consensus.cc:3055] T ecc1e363230043fe9fef390f0b979f20 P 90f3da57796b488c9087a2f523adf045 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:21:47.824807 22483 raft_consensus.cc:740] T ecc1e363230043fe9fef390f0b979f20 P 90f3da57796b488c9087a2f523adf045 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:21:47.824977 22431 consensus_queue.cc:237] T 9e65676b21b748728ad39668cfe722ec P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:47.825846 22483 consensus_queue.cc:260] T ecc1e363230043fe9fef390f0b979f20 P 90f3da57796b488c9087a2f523adf045 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.12, Last appended by leader: 12, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:21:47.815474 16192 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:21:47.815907 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 83471.1us
I20260709 13:21:47.791335 21623 raft_consensus.cc:3060] T 23f82224f4564fea93572e8206bad1fe P cb82cc7bd6db46d980c75237a1402860 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:21:47.731328 22489 log.cc:927] Time spent T 3884f864a96d428785895e5463e5728f P cb82cc7bd6db46d980c75237a1402860: Append to log took a long time: real 0.063s	user 0.000s	sys 0.000s
I20260709 13:21:47.790793 16117 leader_election.cc:304] T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860
W20260709 13:21:47.823724 16266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 197793us
I20260709 13:21:47.764659 16218 rpcz_store.cc:276] 0709 13:21:46.183113 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:46.183329 (+   216us) service_pool.cc:224] Handling call
0709 13:21:46.195360 (+ 12031us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:21:46.280296 (+ 84936us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 13:21:46.535785 (+255489us) log.cc:844] Serialized 14 byte log entry
0709 13:21:47.499493 (+963708us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:21:47.499556 (+    63us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:21:47.499609 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:47.499676 (+    67us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:21:47.499800 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8058,"prepare.queue_time_us":669629,"prepare.run_cpu_time_us":7906,"prepare.run_wall_time_us":26970,"raft.queue_time_us":139877,"raft.run_cpu_time_us":130,"raft.run_wall_time_us":130,"thread_start_us":1046619,"threads_started":3,"wal-append.queue_time_us":895010}
W20260709 13:21:47.710194 22285 raft_consensus.cc:1068] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac: Unable to promote non-voter cb82cc7bd6db46d980c75237a1402860: Illegal state: Leader has not yet committed an operation in its own term
W20260709 13:21:47.832413 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 338131us
I20260709 13:21:47.833665 16265 leader_election.cc:304] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
W20260709 13:21:47.836442 16117 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 45734.9us
I20260709 13:21:47.837173 16117 leader_election.cc:304] T 3884f864a96d428785895e5463e5728f P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:21:47.838410 16190 leader_election.cc:304] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: cb82cc7bd6db46d980c75237a1402860
I20260709 13:21:47.823002 22399 raft_consensus.cc:2804] T 26354361e9b044cea1c2b86c4882fe92 P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 13:21:47.848506 16266 leader_election.cc:336] T b87147731a68400e8cce9b879daa7c5d P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923): Timed out: RequestConsensusVote RPC to 127.14.177.132:38923 timed out after 1.740s (SENT)
I20260709 13:21:47.856346 16219 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57604 (request call id 3152) took 1872 ms. Trace:
I20260709 13:21:47.856758 16219 rpcz_store.cc:276] 0709 13:21:45.983772 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:45.983975 (+   203us) service_pool.cc:224] Handling call
0709 13:21:45.984397 (+   422us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:21:46.474894 (+490497us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:47.856100 (+1381206us) spinlock_profiling.cc:243] Waited 449 ms on lock 0x7b5400396554. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:21:47.856286 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1031918848}
I20260709 13:21:47.833986 22483 raft_consensus.cc:3060] T ecc1e363230043fe9fef390f0b979f20 P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:21:47.854736 16266 leader_election.cc:304] T b87147731a68400e8cce9b879daa7c5d P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
I20260709 13:21:47.842118 16117 leader_election.cc:304] T 434bff3082d141838068f323bdb0d1ed P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:21:47.860014 22474 raft_consensus.cc:2804] T 434bff3082d141838068f323bdb0d1ed P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:21:47.860440 22474 raft_consensus.cc:493] T 434bff3082d141838068f323bdb0d1ed P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Starting leader election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:21:47.860837 22474 raft_consensus.cc:3060] T 434bff3082d141838068f323bdb0d1ed P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:21:47.863457 16817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:47.862986 16816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74ca8140f35c4ae29e8a664f6bdfc829" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:21:47.871376 22104 raft_consensus.cc:2804] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:21:47.871750 16816 raft_consensus.cc:3060] T 74ca8140f35c4ae29e8a664f6bdfc829 P cb82cc7bd6db46d980c75237a1402860 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:21:47.871879 22104 raft_consensus.cc:493] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:47.872332 22104 raft_consensus.cc:3060] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:21:47.874243 22436 leader_election.cc:290] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:21:47.876751 22474 raft_consensus.cc:515] T 434bff3082d141838068f323bdb0d1ed P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:21:47.862027 16191 leader_election.cc:336] T c2f5f89186204242bf377f7c42b2e2da P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Timed out: RequestConsensusVote RPC to 127.14.177.129:36741 timed out after 1.577s (SENT)
I20260709 13:21:47.879142 22474 leader_election.cc:290] T 434bff3082d141838068f323bdb0d1ed P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 16 election: Requested vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:47.883535 16224 raft_consensus.cc:2468] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 22.
I20260709 13:21:47.885638 16117 leader_election.cc:304] T 23f82224f4564fea93572e8206bad1fe P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:21:47.891089 22182 raft_consensus.cc:2804] T 23f82224f4564fea93572e8206bad1fe P 647a343a542d4a139fe106c19095129b [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:21:47.891537 22182 raft_consensus.cc:493] T 23f82224f4564fea93572e8206bad1fe P 647a343a542d4a139fe106c19095129b [term 14 FOLLOWER]: Starting leader election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:21:47.891893 22182 raft_consensus.cc:3060] T 23f82224f4564fea93572e8206bad1fe P 647a343a542d4a139fe106c19095129b [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:21:47.893224 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 54895.6us
I20260709 13:21:47.894032 16265 leader_election.cc:400] T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 16 pre-election: Vote denied by peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741) with higher term. Message: Invalid argument: T b6f07842d3aa47858df5665874c86f3a P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 for earlier term 16. Current term is 17.
I20260709 13:21:47.894528 16265 leader_election.cc:403] T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 16 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:21:47.894519 16117 leader_election.cc:400] T 9e65676b21b748728ad39668cfe722ec P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 16 pre-election: Vote denied by peer cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923) with higher term. Message: Invalid argument: T 9e65676b21b748728ad39668cfe722ec P cb82cc7bd6db46d980c75237a1402860 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b for earlier term 16. Current term is 17.
I20260709 13:21:47.842932 22197 raft_consensus.cc:2804] T 3884f864a96d428785895e5463e5728f P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:21:47.895210 16117 leader_election.cc:403] T 9e65676b21b748728ad39668cfe722ec P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 16 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:21:47.898566 22474 raft_consensus.cc:2749] T 9e65676b21b748728ad39668cfe722ec P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Leader pre-election lost for term 16. Reason: Vote denied by peer cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923) with higher term. Message: Invalid argument: T 9e65676b21b748728ad39668cfe722ec P cb82cc7bd6db46d980c75237a1402860 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b for earlier term 16. Current term is 17.
W20260709 13:21:47.899480 16223 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57600 (request call id 1842) took 1495 ms (client timeout 1845 ms). Trace:
W20260709 13:21:47.899932 16223 rpcz_store.cc:269] 0709 13:21:46.397974 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:46.401101 (+  3127us) service_pool.cc:224] Handling call
0709 13:21:47.893417 (+1492316us) spinlock_profiling.cc:243] Waited 1.14 s on lock 0x7b5400245c54. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:21:47.893566 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2628290304}
I20260709 13:21:47.900439 16115 leader_election.cc:304] T b87147731a68400e8cce9b879daa7c5d P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860
I20260709 13:21:47.847918 22399 raft_consensus.cc:493] T 26354361e9b044cea1c2b86c4882fe92 P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Starting leader election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:21:47.834088 22349 consensus_queue.cc:1048] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.055s
I20260709 13:21:47.839255 22504 raft_consensus.cc:515] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P cb82cc7bd6db46d980c75237a1402860 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:47.849779 22074 consensus_queue.cc:1048] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.063s
I20260709 13:21:47.902743 22436 raft_consensus.cc:3060] T b87147731a68400e8cce9b879daa7c5d P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:21:47.910347 22201 consensus_queue.cc:1048] T c9944d5e4b8c4109af9955c4f08c9095 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.103s
W20260709 13:21:47.999058 16264 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:47.976572 16190 leader_election.cc:304] T 02838c86531d4f98bf58d1c22b8baf77 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:21:47.928403 16789 leader_election.cc:304] T b87147731a68400e8cce9b879daa7c5d P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 
W20260709 13:21:48.020356 16789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 92059.4us
I20260709 13:21:47.956064 16226 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57604 (request call id 3160) took 1534 ms. Trace:
I20260709 13:21:48.021404 16789 leader_election.cc:304] T 57a744f5aaf944678a63469cbf514015 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:21:48.029247 16789 leader_election.cc:304] T 3884f864a96d428785895e5463e5728f P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:21:47.885125 22285 raft_consensus.cc:2804] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Leader pre-election won for term 20
W20260709 13:21:47.921679 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 73964.6us
I20260709 13:21:48.022789 16226 rpcz_store.cc:276] 0709 13:21:46.421874 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:47.756470 (+1334596us) service_pool.cc:224] Handling call
0709 13:21:47.756843 (+   373us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:21:47.757109 (+   266us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:21:47.757540 (+   431us) log.cc:844] Serialized 14 byte log entry
0709 13:21:47.888382 (+130842us) log.cc:844] Serialized 14 byte log entry
0709 13:21:47.955540 (+ 67158us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:21:47.955616 (+    76us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:21:47.955688 (+    72us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:47.955794 (+   106us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:21:47.955982 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":71813,"prepare.queue_time_us":3268,"prepare.run_cpu_time_us":57,"prepare.run_wall_time_us":104,"raft.queue_time_us":97853,"raft.run_cpu_time_us":135,"raft.run_wall_time_us":23466,"spinlock_wait_cycles":10330368,"thread_start_us":96221,"threads_started":1,"wal-append.queue_time_us":97955}
W20260709 13:21:47.958390 16298 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48674 (request call id 3217) took 1631 ms (client timeout 1876 ms). Trace:
W20260709 13:21:48.037426 16298 rpcz_store.cc:269] 0709 13:21:46.326534 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:46.371339 (+ 44805us) service_pool.cc:224] Handling call
0709 13:21:47.958107 (+1586768us) spinlock_profiling.cc:243] Waited 945 ms on lock 0x7b540024c554. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:21:47.958306 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2172611968}
W20260709 13:21:48.044389 16115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 156799us
I20260709 13:21:48.027305 22478 raft_consensus.cc:2804] T b87147731a68400e8cce9b879daa7c5d P cb82cc7bd6db46d980c75237a1402860 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:21:47.897732 22197 raft_consensus.cc:493] T 3884f864a96d428785895e5463e5728f P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Starting leader election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:21:48.036155 22430 raft_consensus.cc:2804] T 57a744f5aaf944678a63469cbf514015 P cb82cc7bd6db46d980c75237a1402860 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:21:47.974553 22494 raft_consensus.cc:2749] T b87147731a68400e8cce9b879daa7c5d P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:21:48.054018 21623 raft_consensus.cc:515] T 23f82224f4564fea93572e8206bad1fe P cb82cc7bd6db46d980c75237a1402860 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:21:48.054611 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 131518us
I20260709 13:21:48.057060 16789 leader_election.cc:304] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 
W20260709 13:21:48.057912 16264 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:21:48.036893 16191 leader_election.cc:336] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Timed out: RequestConsensusVote RPC to 127.14.177.129:36741 timed out after 1.710s (SENT)
I20260709 13:21:48.028012 22480 raft_consensus.cc:2749] T 02838c86531d4f98bf58d1c22b8baf77 P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:21:48.034587 22399 raft_consensus.cc:3060] T 26354361e9b044cea1c2b86c4882fe92 P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:21:48.060930 16265 leader_election.cc:304] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:21:48.068084 16816 raft_consensus.cc:2468] T 74ca8140f35c4ae29e8a664f6bdfc829 P cb82cc7bd6db46d980c75237a1402860 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 18.
I20260709 13:21:47.959381 22404 raft_consensus.cc:2749] T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Leader pre-election lost for term 16. Reason: Vote denied by peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741) with higher term. Message: Invalid argument: T b6f07842d3aa47858df5665874c86f3a P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 for earlier term 16. Current term is 17.
W20260709 13:21:48.049110 21704 consensus_meta.cc:287] T ee9ed67006144797b1de6ea8c2013a68 P 647a343a542d4a139fe106c19095129b: Time spent flushing consensus metadata: real 0.693s	user 0.011s	sys 0.008s
I20260709 13:21:48.034520 22349 consensus_queue.cc:1243] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Peer 90f3da57796b488c9087a2f523adf045 log is divergent from this leader: its last log entry 14.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
W20260709 13:21:48.069108 16266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 233723us
I20260709 13:21:48.010960 22088 raft_consensus.cc:515] T 2794af5b99984efebdee564035841cb0 P 647a343a542d4a139fe106c19095129b [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:48.035439 22431 raft_consensus.cc:2749] T 3884f864a96d428785895e5463e5728f P cb82cc7bd6db46d980c75237a1402860 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:21:48.104470 16789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 47494us
I20260709 13:21:48.024366 22474 consensus_queue.cc:1059] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 647a343a542d4a139fe106c19095129b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.038s
W20260709 13:21:48.066978 22383 consensus_meta.cc:287] T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b: Time spent flushing consensus metadata: real 0.509s	user 0.013s	sys 0.005s
I20260709 13:21:48.034952 22285 raft_consensus.cc:493] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Starting leader election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:21:48.124949 21623 leader_election.cc:290] T 23f82224f4564fea93572e8206bad1fe P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 15 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:48.125196 22285 raft_consensus.cc:3060] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:21:48.128065 22088 leader_election.cc:290] T 2794af5b99984efebdee564035841cb0 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 14 election: Requested vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:48.128728 22088 raft_consensus.cc:3055] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 647a343a542d4a139fe106c19095129b [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:21:48.129077 22088 raft_consensus.cc:740] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 647a343a542d4a139fe106c19095129b [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:21:48.109069 16192 leader_election.cc:304] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:21:48.129899 22088 consensus_queue.cc:260] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:48.067154 22478 raft_consensus.cc:493] T b87147731a68400e8cce9b879daa7c5d P cb82cc7bd6db46d980c75237a1402860 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:48.104415 16226 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57604 (request call id 3173) took 1012 ms. Trace:
I20260709 13:21:48.131312 16226 rpcz_store.cc:276] 0709 13:21:47.091840 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:48.049294 (+957454us) service_pool.cc:224] Handling call
0709 13:21:48.049666 (+   372us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:21:48.070161 (+ 20495us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:21:48.070191 (+    30us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:21:48.104131 (+ 33940us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:21:48.104179 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:48.104240 (+    61us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:21:48.104364 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:21:48.102626 22106 consensus_meta.cc:287] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045: Time spent flushing consensus metadata: real 0.560s	user 0.010s	sys 0.007s
I20260709 13:21:48.118021 16150 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40830 (request call id 1878) took 1314 ms. Trace:
W20260709 13:21:48.107218 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 70359.2us
I20260709 13:21:48.070192 22430 raft_consensus.cc:493] T 57a744f5aaf944678a63469cbf514015 P cb82cc7bd6db46d980c75237a1402860 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:21:48.130370 16143 raft_consensus.cc:1240] T 2794af5b99984efebdee564035841cb0 P 647a343a542d4a139fe106c19095129b [term 14 FOLLOWER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 13. Current term is 14. Ops: []
I20260709 13:21:47.996835 22499 consensus_queue.cc:1048] T 8c11074579b046d99e585c2818e18a2c P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.079s
I20260709 13:21:48.130951 22478 raft_consensus.cc:3060] T b87147731a68400e8cce9b879daa7c5d P cb82cc7bd6db46d980c75237a1402860 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:21:48.070240 22197 raft_consensus.cc:3060] T 3884f864a96d428785895e5463e5728f P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:21:48.134322 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 73451.6us
W20260709 13:21:48.138744 16115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 67743.6us
I20260709 13:21:48.134449 16150 rpcz_store.cc:276] 0709 13:21:46.803132 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:47.380928 (+577796us) service_pool.cc:224] Handling call
0709 13:21:47.437869 (+ 56941us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:21:47.445137 (+  7268us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:21:47.482381 (+ 37244us) log.cc:844] Serialized 14 byte log entry
0709 13:21:48.023759 (+541378us) log.cc:844] Serialized 14 byte log entry
0709 13:21:48.117650 (+ 93891us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:21:48.117704 (+    54us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:21:48.117758 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:48.117825 (+    67us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:21:48.117944 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":649955,"prepare.queue_time_us":240562,"prepare.run_cpu_time_us":91,"prepare.run_wall_time_us":5266,"raft.queue_time_us":100664,"raft.run_cpu_time_us":101,"raft.run_wall_time_us":100,"thread_start_us":442779,"threads_started":2,"wal-append.queue_time_us":381208}
I20260709 13:21:48.107056 22506 raft_consensus.cc:2804] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:21:48.151424 16822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66ae176961b74813a58bc15c9c4c6f2a" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:48.153033 16819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9944d5e4b8c4109af9955c4f08c9095" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:48.148262 16226 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57604 (request call id 3174) took 1049 ms. Trace:
I20260709 13:21:48.158628 16822 raft_consensus.cc:2393] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 22: Already voted for candidate cb82cc7bd6db46d980c75237a1402860 in this term.
I20260709 13:21:48.161338 16817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ab0ac8347641e6955e9bce7d90c1a2" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:48.161533 16820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada5a004ba7c46819051e3ccec71ffdb" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:21:48.162611 16821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74ca8140f35c4ae29e8a664f6bdfc829" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:48.145840 22106 raft_consensus.cc:515] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:48.147279 22430 raft_consensus.cc:3060] T 57a744f5aaf944678a63469cbf514015 P cb82cc7bd6db46d980c75237a1402860 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:21:48.183334 22506 raft_consensus.cc:697] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860 [term 16 LEADER]: Becoming Leader. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:21:48.124990 22383 raft_consensus.cc:515] T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:48.184693 22506 consensus_queue.cc:237] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:48.187209 22246 leader_election.cc:290] T 8158bbd17b994d2eb96f560bd0f1b88d P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 16 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:48.187973 16218 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57604 (request call id 3168) took 1218 ms. Trace:
I20260709 13:21:48.188757 22504 leader_election.cc:290] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:48.189169 22436 raft_consensus.cc:2749] T b87147731a68400e8cce9b879daa7c5d P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:21:48.155779 22201 raft_consensus.cc:493] T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:48.191831 22201 raft_consensus.cc:515] T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:48.193970 22201 leader_election.cc:290] T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:21:48.155937 22483 raft_consensus.cc:2804] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:21:48.194917 22483 raft_consensus.cc:493] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:48.195338 22483 raft_consensus.cc:3060] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:21:48.162206 16825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434bff3082d141838068f323bdb0d1ed" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:21:48.199815 16825 raft_consensus.cc:3060] T 434bff3082d141838068f323bdb0d1ed P cb82cc7bd6db46d980c75237a1402860 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:21:48.167351 22502 raft_consensus.cc:2804] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:21:48.168927 22106 leader_election.cc:290] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 18 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:48.169693 16265 leader_election.cc:304] T 8d4ab870a05347868956aa8b841b2981 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
W20260709 13:21:48.174000 22202 consensus_meta.cc:287] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860: Time spent flushing consensus metadata: real 0.622s	user 0.008s	sys 0.007s
I20260709 13:21:48.217388 22202 raft_consensus.cc:515] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:21:48.176837 22236 consensus_meta.cc:287] T 5a96373bb0e14dce8a266b2fc6c58507 P 647a343a542d4a139fe106c19095129b: Time spent flushing consensus metadata: real 0.696s	user 0.011s	sys 0.012s
W20260709 13:21:48.218708 22473 log.cc:927] Time spent T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 0.938s	user 0.000s	sys 0.008s
I20260709 13:21:48.218502 22236 raft_consensus.cc:515] T 5a96373bb0e14dce8a266b2fc6c58507 P 647a343a542d4a139fe106c19095129b [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:48.220587 22236 leader_election.cc:290] T 5a96373bb0e14dce8a266b2fc6c58507 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 14 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:48.220745 22202 leader_election.cc:290] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 14 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:21:48.145929 22480 consensus_queue.cc:1059] T 1df93779209748b9860f6d893dd3de53 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.031s
I20260709 13:21:48.202270 22399 raft_consensus.cc:515] T 26354361e9b044cea1c2b86c4882fe92 P 90f3da57796b488c9087a2f523adf045 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:48.222486 16818 raft_consensus.cc:1240] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860 [term 14 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 13. Current term is 14. Ops: []
W20260709 13:21:48.223398 16116 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:48.223658 22399 leader_election.cc:290] T 26354361e9b044cea1c2b86c4882fe92 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 14 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:48.224596 22430 raft_consensus.cc:515] T 57a744f5aaf944678a63469cbf514015 P cb82cc7bd6db46d980c75237a1402860 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:48.177210 22182 raft_consensus.cc:515] T 23f82224f4564fea93572e8206bad1fe P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:48.119438 21704 raft_consensus.cc:515] T ee9ed67006144797b1de6ea8c2013a68 P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:48.226289 22496 consensus_queue.cc:1059] T 746aabf856ab401c9ef5e859371e7b67 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:21:48.227262 16147 raft_consensus.cc:1217] T ee9ed67006144797b1de6ea8c2013a68 P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Deduplicated request from leader. Original: 11.10->[12.11-17.14]   Dedup: 13.12->[15.13-17.14]
I20260709 13:21:48.227660 22496 raft_consensus.cc:3055] T 746aabf856ab401c9ef5e859371e7b67 P 1ca2725d290e47d68b1a4b609c9816ac [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:21:48.188383 16218 rpcz_store.cc:276] 0709 13:21:46.969711 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:47.838239 (+868528us) service_pool.cc:224] Handling call
0709 13:21:47.849147 (+ 10908us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:21:47.900022 (+ 50875us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:21:47.900054 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:21:47.912170 (+ 12116us) log.cc:844] Serialized 26 byte log entry
0709 13:21:48.115925 (+203755us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:21:48.115982 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:48.116061 (+    79us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:21:48.187743 (+ 71682us) raft_consensus.cc:1685] finished
0709 13:21:48.187766 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:21:48.187911 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7857,"raft.queue_time_us":15285,"raft.run_cpu_time_us":90,"raft.run_wall_time_us":89,"thread_start_us":203366,"threads_started":1,"wal-append.queue_time_us":248806}
I20260709 13:21:48.228178 22496 raft_consensus.cc:740] T 746aabf856ab401c9ef5e859371e7b67 P 1ca2725d290e47d68b1a4b609c9816ac [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:21:48.228739 16218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66ae176961b74813a58bc15c9c4c6f2a" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:21:48.229302 22496 consensus_queue.cc:260] T 746aabf856ab401c9ef5e859371e7b67 P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:48.230118 16218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e4bfb9090c8460eb14fa7e0d818e7f1" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:48.202569 22502 raft_consensus.cc:697] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [term 18 LEADER]: Becoming Leader. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:21:48.166519 16821 raft_consensus.cc:2393] T 74ca8140f35c4ae29e8a664f6bdfc829 P cb82cc7bd6db46d980c75237a1402860 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 18: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
I20260709 13:21:48.204609 22285 raft_consensus.cc:515] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:48.204311 16115 leader_election.cc:304] T 8d4ab870a05347868956aa8b841b2981 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac
I20260709 13:21:48.206086 22168 raft_consensus.cc:3060] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:21:48.205957 22196 consensus_queue.cc:1048] T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:21:48.174088 22104 raft_consensus.cc:515] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } }
I20260709 13:21:48.216647 22408 leader_election.cc:290] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:21:48.227787 16147 raft_consensus.cc:1240] T ee9ed67006144797b1de6ea8c2013a68 P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 17. Current term is 18. Ops: [12.11-17.14]
I20260709 13:21:48.154757 16226 rpcz_store.cc:276] 0709 13:21:47.092173 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:48.132233 (+1040060us) service_pool.cc:224] Handling call
0709 13:21:48.132605 (+   372us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:21:48.140711 (+  8106us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:21:48.140744 (+    33us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:21:48.140792 (+    48us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:21:48.140822 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:48.140883 (+    61us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:21:48.141053 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:21:48.232110 22502 consensus_queue.cc:237] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.12, Last appended by leader: 12, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:48.232903 16218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746aabf856ab401c9ef5e859371e7b67" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:21:48.233073 16825 raft_consensus.cc:2468] T 434bff3082d141838068f323bdb0d1ed P cb82cc7bd6db46d980c75237a1402860 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 16.
I20260709 13:21:48.233850 16790 leader_election.cc:304] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:21:48.234266 16115 leader_election.cc:304] T b6f07842d3aa47858df5665874c86f3a P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:21:48.235973 16222 raft_consensus.cc:3060] T 7c2095a5f4ad49408dbc7f69c606120d P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:21:48.239806 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5d59407eb545b895d8d922e0dd69fd" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
W20260709 13:21:48.256036 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 86423.9us
I20260709 13:21:48.255832 16226 raft_consensus.cc:2468] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 19.
I20260709 13:21:48.259991 16790 leader_election.cc:304] T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:21:48.261814 16265 leader_election.cc:304] T 2794af5b99984efebdee564035841cb0 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:21:48.309864 16142 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46224 (request call id 1002) took 2146 ms. Trace:
W20260709 13:21:48.309944 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 50028.4us
W20260709 13:21:48.311640 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 49929.7us
I20260709 13:21:48.312343 16825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdbd78acb4cf46298e0de8ba0d130e19" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:21:48.234493 22518 raft_consensus.cc:2804] T 8d4ab870a05347868956aa8b841b2981 P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:21:48.232288 22496 raft_consensus.cc:3060] T 746aabf856ab401c9ef5e859371e7b67 P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:21:48.315054 16265 leader_election.cc:304] T 9e65676b21b748728ad39668cfe722ec P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
I20260709 13:21:48.236212 22104 leader_election.cc:290] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 18 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:21:48.309206 22236 raft_consensus.cc:2804] T b6f07842d3aa47858df5665874c86f3a P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:21:48.316589 16115 leader_election.cc:304] T 26354361e9b044cea1c2b86c4882fe92 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860
I20260709 13:21:48.317598 22196 consensus_queue.cc:1048] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:21:48.317890 22082 leader_election.cc:290] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:48.318244 22074 consensus_queue.cc:1048] T 1df93779209748b9860f6d893dd3de53 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:21:48.320626 22500 raft_consensus.cc:2749] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:21:48.233946 22285 leader_election.cc:290] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 20 election: Requested vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:48.321216 22430 leader_election.cc:290] T 57a744f5aaf944678a63469cbf514015 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 13 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:48.324456 22383 leader_election.cc:290] T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 20 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
W20260709 13:21:48.362428 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 47452.4us
I20260709 13:21:48.327601 22518 raft_consensus.cc:493] T 8d4ab870a05347868956aa8b841b2981 P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Starting leader election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:21:48.328598 16225 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33050 (request call id 1088) took 1116 ms. Trace:
I20260709 13:21:48.367609 16225 rpcz_store.cc:276] 0709 13:21:47.211871 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:47.265392 (+ 53521us) service_pool.cc:224] Handling call
0709 13:21:48.328353 (+1062961us) spinlock_profiling.cc:243] Waited 126 ms on lock 0x7b5400138b54. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:21:48.328516 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":288706304}
I20260709 13:21:48.233352 22197 raft_consensus.cc:515] T 3884f864a96d428785895e5463e5728f P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:48.311496 22356 raft_consensus.cc:2749] T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:21:48.349180 16266 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:48.309506 22399 raft_consensus.cc:2804] T 8d4ab870a05347868956aa8b841b2981 P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:21:48.311384 16142 rpcz_store.cc:276] 0709 13:21:46.163842 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:46.174490 (+ 10648us) service_pool.cc:224] Handling call
0709 13:21:46.180679 (+  6189us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:21:46.192686 (+ 12007us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 13:21:46.371360 (+178674us) log.cc:844] Serialized 14 byte log entry
0709 13:21:48.013923 (+1642563us) spinlock_profiling.cc:243] Waited 4.06 ms on lock 0x7b180040b760. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b693ade1 00007f69b69475b1 00007f69ad9ecc0a 00007f69b712c219 00007f69ada74a40 00007f69ada6f4e4 00007f69ada6f455 00007f69b45573c7 00007f69b0b699a7 00007f69b0b790bf 00007f69b0b79038
0709 13:21:48.070622 (+ 56699us) log.cc:844] Serialized 14 byte log entry
0709 13:21:48.226052 (+155430us) spinlock_profiling.cc:243] Waited 4.57 ms on lock 0x7b180040b760. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b693ade1 00007f69b69475b1 00007f69ada723f3 00007f69ada747c1 00007f69ada74779 00007f69ada74709 00007f69ada746d1 00007f69ada739cd 000000000038f354 000000000038f199 00007f69ada5583a
0709 13:21:48.309477 (+ 83425us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:21:48.309551 (+    74us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:21:48.309605 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:48.309680 (+    75us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:21:48.309809 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":11203,"prepare.queue_time_us":615530,"prepare.run_cpu_time_us":66,"prepare.run_wall_time_us":69,"raft.queue_time_us":54487,"raft.run_cpu_time_us":178,"raft.run_wall_time_us":4107,"spinlock_wait_cycles":19833344,"thread_start_us":1340375,"threads_started":4,"wal-append.queue_time_us":992217}
I20260709 13:21:48.239686 22478 raft_consensus.cc:515] T b87147731a68400e8cce9b879daa7c5d P cb82cc7bd6db46d980c75237a1402860 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:48.357331 16115 leader_election.cc:304] T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac
I20260709 13:21:48.358434 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:48.314368 22508 raft_consensus.cc:2804] T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:21:48.330890 22404 raft_consensus.cc:2804] T 2794af5b99984efebdee564035841cb0 P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:21:48.360594 16823 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49892 (request call id 916) took 1263 ms. Trace:
I20260709 13:21:48.361783 16298 raft_consensus.cc:1240] T ada5a004ba7c46819051e3ccec71ffdb P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 17. Current term is 18. Ops: []
I20260709 13:21:48.365810 16265 leader_election.cc:304] T 31969392ccdb43b491911a319e8bc7e1 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
W20260709 13:21:48.329494 22473 log.cc:927] Time spent T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 0.069s	user 0.000s	sys 0.000s
I20260709 13:21:48.231132 22483 raft_consensus.cc:515] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:48.369578 16790 leader_election.cc:304] T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 90f3da57796b488c9087a2f523adf045
I20260709 13:21:48.356091 22408 raft_consensus.cc:2749] T 9e65676b21b748728ad39668cfe722ec P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:21:48.447300 16825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e4bfb9090c8460eb14fa7e0d818e7f1" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:21:48.459801 16825 raft_consensus.cc:3060] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P cb82cc7bd6db46d980c75237a1402860 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:21:48.463899 16189 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:48.467445 22517 raft_consensus.cc:2804] T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:21:48.468109 22517 raft_consensus.cc:697] T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b [term 17 LEADER]: Becoming Leader. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
W20260709 13:21:48.468650 16221 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33050 (request call id 1083) took 2164 ms (client timeout 2232 ms). Trace:
W20260709 13:21:48.469121 16221 rpcz_store.cc:269] 0709 13:21:46.304721 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:46.332612 (+ 27891us) service_pool.cc:224] Handling call
0709 13:21:48.468430 (+2135818us) spinlock_profiling.cc:243] Waited 1.04 s on lock 0x7b54002c9054. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:21:48.468578 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2395859200}
I20260709 13:21:48.470453 16221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2372981006d4b218215c28964dea623" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:21:48.374724 16292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f82224f4564fea93572e8206bad1fe" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:21:48.474577 16821 raft_consensus.cc:1240] T 434bff3082d141838068f323bdb0d1ed P cb82cc7bd6db46d980c75237a1402860 [term 16 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 15. Current term is 16. Ops: []
I20260709 13:21:48.475766 22246 raft_consensus.cc:493] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:48.476401 22246 raft_consensus.cc:515] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:48.374397 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e4bfb9090c8460eb14fa7e0d818e7f1" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:48.478766 16297 raft_consensus.cc:2393] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 in current term 18: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
I20260709 13:21:48.469028 22517 consensus_queue.cc:237] T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 15.8, Last appended by leader: 8, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:48.480126 22106 raft_consensus.cc:493] T fffab26427e24d398e2695aad40ca134 P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:21:48.481045 16293 raft_consensus.cc:2468] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 19.
I20260709 13:21:48.480870 22106 raft_consensus.cc:515] T fffab26427e24d398e2695aad40ca134 P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:48.476943 16821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e9b2bf7011f4ae688a79d7bb5a447b4" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 17 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:48.448004 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57a744f5aaf944678a63469cbf514015" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:21:48.483525 16299 raft_consensus.cc:3060] T 57a744f5aaf944678a63469cbf514015 P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:21:48.483944 22478 leader_election.cc:290] T b87147731a68400e8cce9b879daa7c5d P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 18 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:48.458576 16823 rpcz_store.cc:276] 0709 13:21:47.097240 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:47.147472 (+ 50232us) service_pool.cc:224] Handling call
0709 13:21:47.147821 (+   349us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:21:47.326940 (+179119us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:48.360363 (+1033423us) spinlock_profiling.cc:243] Waited 133 ms on lock 0x7b54007ffdd4. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:21:48.360542 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":304771328}
I20260709 13:21:48.458425 16817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26354361e9b044cea1c2b86c4882fe92" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
W20260709 13:21:48.449476 16192 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:21:48.485345 16115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 128080us
I20260709 13:21:48.458787 22508 raft_consensus.cc:493] T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:21:48.423213 16264 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:48.492321 16818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57a744f5aaf944678a63469cbf514015" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:48.428546 16225 raft_consensus.cc:1240] T ecc1e363230043fe9fef390f0b979f20 P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 18. Current term is 19. Ops: []
I20260709 13:21:48.495904 16225 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57600 (request call id 1854) took 1297 ms. Trace:
W20260709 13:21:48.496078 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 305936us
I20260709 13:21:48.423079 16217 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33050 (request call id 1085) took 1999 ms. Trace:
I20260709 13:21:48.500975 16217 rpcz_store.cc:276] 0709 13:21:46.438034 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:47.819106 (+1381072us) service_pool.cc:224] Handling call
0709 13:21:47.832354 (+ 13248us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:21:47.858859 (+ 26505us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 13:21:48.025868 (+167009us) log.cc:844] Serialized 14 byte log entry
0709 13:21:48.422580 (+396712us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:21:48.422659 (+    79us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:21:48.422720 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:48.422812 (+    92us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:21:48.423000 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":130512,"prepare.run_cpu_time_us":30,"prepare.run_wall_time_us":30,"raft.queue_time_us":56808,"raft.run_cpu_time_us":151,"raft.run_wall_time_us":49217,"thread_start_us":449328,"threads_started":2}
W20260709 13:21:48.492569 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 123112us
I20260709 13:21:48.496305 16225 rpcz_store.cc:276] 0709 13:21:47.198697 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:48.368758 (+1170061us) service_pool.cc:224] Handling call
0709 13:21:48.428302 (+ 59544us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:21:48.495685 (+ 67383us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:48.495856 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:21:48.511035 16818 raft_consensus.cc:2393] T 57a744f5aaf944678a63469cbf514015 P cb82cc7bd6db46d980c75237a1402860 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 13: Already voted for candidate cb82cc7bd6db46d980c75237a1402860 in this term.
I20260709 13:21:48.511270 16222 raft_consensus.cc:2468] T 7c2095a5f4ad49408dbc7f69c606120d P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 14.
I20260709 13:21:48.511339 16115 leader_election.cc:304] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
W20260709 13:21:48.512171 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 146438us
I20260709 13:21:48.504760 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a96373bb0e14dce8a266b2fc6c58507" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:21:48.514612 16217 raft_consensus.cc:3060] T 5a96373bb0e14dce8a266b2fc6c58507 P 1ca2725d290e47d68b1a4b609c9816ac [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:21:48.513661 16190 leader_election.cc:304] T fffab26427e24d398e2695aad40ca134 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:21:48.359063 22236 raft_consensus.cc:697] T b6f07842d3aa47858df5665874c86f3a P 647a343a542d4a139fe106c19095129b [term 17 LEADER]: Becoming Leader. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
W20260709 13:21:48.520200 16264 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:48.362073 22461 raft_consensus.cc:2749] T 26354361e9b044cea1c2b86c4882fe92 P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:21:48.530354 16789 leader_election.cc:304] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:21:48.525108 16825 raft_consensus.cc:2468] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P cb82cc7bd6db46d980c75237a1402860 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 18.
I20260709 13:21:48.532081 16190 leader_election.cc:304] T 8cc959fb7f704fba9009c0595467d074 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:21:48.370651 22518 raft_consensus.cc:3060] T 8d4ab870a05347868956aa8b841b2981 P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:21:48.486624 22404 raft_consensus.cc:493] T 2794af5b99984efebdee564035841cb0 P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Starting leader election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:21:48.540591 16299 raft_consensus.cc:2468] T 57a744f5aaf944678a63469cbf514015 P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 13.
I20260709 13:21:48.492499 22508 raft_consensus.cc:3060] T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:21:48.543136 22404 raft_consensus.cc:3060] T 2794af5b99984efebdee564035841cb0 P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:21:48.544109 22196 raft_consensus.cc:2749] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:21:48.541169 22482 raft_consensus.cc:493] T a378d7f796f94615b3b347119a9880fd P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:21:48.546274 22482 raft_consensus.cc:515] T a378d7f796f94615b3b347119a9880fd P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:48.552049 16297 raft_consensus.cc:1275] T 9e65676b21b748728ad39668cfe722ec P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 13:21:48.553539 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87147731a68400e8cce9b879daa7c5d" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:21:48.554199 16300 raft_consensus.cc:3060] T b87147731a68400e8cce9b879daa7c5d P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:21:48.531677 22436 raft_consensus.cc:2804] T 31969392ccdb43b491911a319e8bc7e1 P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:21:48.554700 22404 raft_consensus.cc:515] T 2794af5b99984efebdee564035841cb0 P 90f3da57796b488c9087a2f523adf045 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:48.555325 22436 raft_consensus.cc:493] T 31969392ccdb43b491911a319e8bc7e1 P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Starting leader election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:21:48.555776 22436 raft_consensus.cc:3060] T 31969392ccdb43b491911a319e8bc7e1 P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:21:48.510931 22031 raft_consensus.cc:2804] T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:21:48.563851 22031 raft_consensus.cc:697] T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860 [term 19 LEADER]: Becoming Leader. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:21:48.564827 22031 consensus_queue.cc:237] T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:48.565042 22236 consensus_queue.cc:237] T b6f07842d3aa47858df5665874c86f3a P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:48.568322 22106 leader_election.cc:290] T fffab26427e24d398e2695aad40ca134 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:21:48.568384 16190 leader_election.cc:304] T 8c11074579b046d99e585c2818e18a2c P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: cb82cc7bd6db46d980c75237a1402860
I20260709 13:21:48.571138 16223 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57604 (request call id 3172) took 1482 ms. Trace:
I20260709 13:21:48.571333 22285 raft_consensus.cc:2804] T fffab26427e24d398e2695aad40ca134 P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:21:48.572191 22526 raft_consensus.cc:2804] T 8cc959fb7f704fba9009c0595467d074 P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:21:48.572686 22526 raft_consensus.cc:493] T 8cc959fb7f704fba9009c0595467d074 P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: Starting leader election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:21:48.573125 22526 raft_consensus.cc:3060] T 8cc959fb7f704fba9009c0595467d074 P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:21:48.542217 16789 leader_election.cc:304] T 19baa7bc367844c288bca8f81439c413 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 647a343a542d4a139fe106c19095129b
I20260709 13:21:48.574123 16221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8158bbd17b994d2eb96f560bd0f1b88d" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:21:48.574731 16221 raft_consensus.cc:3060] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:21:48.505669 22399 raft_consensus.cc:493] T 8d4ab870a05347868956aa8b841b2981 P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:21:48.582278 22152 raft_consensus.cc:2804] T 8c11074579b046d99e585c2818e18a2c P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:21:48.583580 22404 leader_election.cc:290] T 2794af5b99984efebdee564035841cb0 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 14 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:48.584834 22197 leader_election.cc:290] T 3884f864a96d428785895e5463e5728f P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 13 election: Requested vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:48.536450 22505 raft_consensus.cc:2749] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P cb82cc7bd6db46d980c75237a1402860 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:21:48.585232 22483 leader_election.cc:290] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 16 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
W20260709 13:21:48.585496 16115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 80549us
I20260709 13:21:48.600791 22474 consensus_queue.cc:1048] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:21:48.601529 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f5f89186204242bf377f7c42b2e2da" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:48.602656 16220 consensus_queue.cc:260] T 1df93779209748b9860f6d893dd3de53 P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:48.603765 16144 raft_consensus.cc:1240] T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 19. Current term is 20. Ops: []
W20260709 13:21:48.587949 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 335154us
I20260709 13:21:48.607086 16142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d16dfad0c3b540e5b4b86492ddf87208" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:48.607692 16116 leader_election.cc:304] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: cb82cc7bd6db46d980c75237a1402860
I20260709 13:21:48.588873 16147 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46224 (request call id 1010) took 1405 ms. Trace:
I20260709 13:21:48.608443 16147 rpcz_store.cc:276] 0709 13:21:47.183157 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:47.483392 (+300235us) service_pool.cc:224] Handling call
0709 13:21:47.496712 (+ 13320us) raft_consensus.cc:1471] Updating replica for 4 ops
0709 13:21:48.235983 (+739271us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:48.588611 (+352628us) spinlock_profiling.cc:243] Waited 730 ms on lock 0x7b54002d0d54. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:21:48.588814 (+   203us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1679501696}
I20260709 13:21:48.609627 16222 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57600 (request call id 1861) took 1255 ms. Trace:
I20260709 13:21:48.611635 16219 raft_consensus.cc:1240] T 7c2095a5f4ad49408dbc7f69c606120d P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 13. Current term is 14. Ops: []
I20260709 13:21:48.612046 16226 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57604 (request call id 3175) took 1519 ms. Trace:
I20260709 13:21:48.612920 16217 raft_consensus.cc:2468] T 5a96373bb0e14dce8a266b2fc6c58507 P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 14.
I20260709 13:21:48.370865 22104 consensus_queue.cc:1059] T 8158bbd17b994d2eb96f560bd0f1b88d P 90f3da57796b488c9087a2f523adf045 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: INVALID_TERM, Last received: 15.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.205s
I20260709 13:21:48.603832 16220 raft_consensus.cc:3060] T 1df93779209748b9860f6d893dd3de53 P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:21:48.614151 16300 raft_consensus.cc:2468] T b87147731a68400e8cce9b879daa7c5d P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 18.
I20260709 13:21:48.585790 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87ac0d1f7e8a419f9bcbc04aee0e067e" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:21:48.571753 22285 raft_consensus.cc:493] T fffab26427e24d398e2695aad40ca134 P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Starting leader election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:21:48.571566 16223 rpcz_store.cc:276] 0709 13:21:47.091401 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:47.903606 (+812205us) service_pool.cc:224] Handling call
0709 13:21:47.903934 (+   328us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:21:47.904357 (+   423us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:21:48.108417 (+204060us) log.cc:844] Serialized 14 byte log entry
0709 13:21:48.352430 (+244013us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:21:48.369045 (+ 16615us) log.cc:844] Serialized 26 byte log entry
0709 13:21:48.369227 (+   182us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:21:48.369274 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:48.369362 (+    88us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:21:48.570920 (+201558us) raft_consensus.cc:1685] finished
0709 13:21:48.570943 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:21:48.571080 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":16601,"prepare.queue_time_us":121805,"prepare.run_cpu_time_us":22,"prepare.run_wall_time_us":23,"raft.queue_time_us":3688,"raft.run_cpu_time_us":167,"raft.run_wall_time_us":3425,"thread_start_us":352524,"threads_started":2,"wal-append.queue_time_us":410218}
I20260709 13:21:48.610251 16222 rpcz_store.cc:276] 0709 13:21:47.353763 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:47.421053 (+ 67290us) service_pool.cc:224] Handling call
0709 13:21:48.609379 (+1188326us) spinlock_profiling.cc:243] Waited 745 ms on lock 0x7b54002c8154. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:21:48.609567 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1712822016}
I20260709 13:21:48.621424 16147 raft_consensus.cc:1275] T ecc1e363230043fe9fef390f0b979f20 P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 13:21:48.621567 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87ac0d1f7e8a419f9bcbc04aee0e067e" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
W20260709 13:21:48.626453 16789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 92334.3us
I20260709 13:21:48.626720 16116 leader_election.cc:304] T 18ad7e3af5504571a69d10c00750a01c P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 90f3da57796b488c9087a2f523adf045
I20260709 13:21:48.628862 16147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e9b2bf7011f4ae688a79d7bb5a447b4" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:21:48.589347 22439 raft_consensus.cc:3060] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:21:48.592451 22399 raft_consensus.cc:3060] T 8d4ab870a05347868956aa8b841b2981 P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:21:48.636209 16300 raft_consensus.cc:1240] T 26354361e9b044cea1c2b86c4882fe92 P 90f3da57796b488c9087a2f523adf045 [term 14 FOLLOWER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 13. Current term is 14. Ops: []
I20260709 13:21:48.640020 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18ad7e3af5504571a69d10c00750a01c" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:21:48.641120 16217 raft_consensus.cc:2393] T 18ad7e3af5504571a69d10c00750a01c P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 15: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
I20260709 13:21:48.594555 22152 raft_consensus.cc:493] T 8c11074579b046d99e585c2818e18a2c P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:48.642211 16265 leader_election.cc:304] T e5ad4eb281564936a59f41e387a47156 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 647a343a542d4a139fe106c19095129b
I20260709 13:21:48.594236 22506 raft_consensus.cc:2804] T 19baa7bc367844c288bca8f81439c413 P cb82cc7bd6db46d980c75237a1402860 [term 19 FOLLOWER]: Leader pre-election won for term 20
W20260709 13:21:48.644704 16266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 171890us
W20260709 13:21:48.650549 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 80694.4us
I20260709 13:21:48.651551 16190 leader_election.cc:304] T d2ab0ac8347641e6955e9bce7d90c1a2 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860
I20260709 13:21:48.600661 22482 leader_election.cc:290] T a378d7f796f94615b3b347119a9880fd P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:48.645028 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada5a004ba7c46819051e3ccec71ffdb" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:21:48.658613 16217 raft_consensus.cc:3060] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:21:48.661948 16223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d16dfad0c3b540e5b4b86492ddf87208" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:21:48.621214 22285 raft_consensus.cc:3060] T fffab26427e24d398e2695aad40ca134 P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:21:48.653661 16151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee9ed67006144797b1de6ea8c2013a68" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 18 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:48.666045 16151 raft_consensus.cc:2393] T ee9ed67006144797b1de6ea8c2013a68 P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 18: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
I20260709 13:21:48.649134 16222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806d7690c2b644959152074bb2cab0a7" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 18 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:21:48.649997 22506 raft_consensus.cc:493] T 19baa7bc367844c288bca8f81439c413 P cb82cc7bd6db46d980c75237a1402860 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:21:48.690519 16144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87147731a68400e8cce9b879daa7c5d" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:48.690976 22506 raft_consensus.cc:3060] T 19baa7bc367844c288bca8f81439c413 P cb82cc7bd6db46d980c75237a1402860 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:21:48.691260 16144 raft_consensus.cc:2468] T b87147731a68400e8cce9b879daa7c5d P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 17.
I20260709 13:21:48.692071 16298 raft_consensus.cc:1240] T 2794af5b99984efebdee564035841cb0 P 90f3da57796b488c9087a2f523adf045 [term 14 FOLLOWER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 13. Current term is 14. Ops: []
I20260709 13:21:48.692366 16142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f82224f4564fea93572e8206bad1fe" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
W20260709 13:21:48.693996 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 67413us
I20260709 13:21:48.695436 16116 leader_election.cc:304] T 5a959dbecc2648eb85d1cc550f5a3245 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
I20260709 13:21:48.658964 16190 leader_election.cc:304] T c0eab25ce4c64e018a041beeda2867db P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:21:48.697579 16190 leader_election.cc:304] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860
I20260709 13:21:48.641202 22196 raft_consensus.cc:2804] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:21:48.633283 22483 raft_consensus.cc:493] T 8cc959fb7f704fba9009c0595467d074 P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:21:48.645875 22522 raft_consensus.cc:493] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:48.654534 22152 raft_consensus.cc:3060] T 8c11074579b046d99e585c2818e18a2c P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:21:48.701216 22520 raft_consensus.cc:2804] T 5a959dbecc2648eb85d1cc550f5a3245 P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:21:48.715720 22520 raft_consensus.cc:493] T 5a959dbecc2648eb85d1cc550f5a3245 P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Starting leader election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:21:48.716159 22520 raft_consensus.cc:3060] T 5a959dbecc2648eb85d1cc550f5a3245 P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:21:48.717263 16266 leader_election.cc:304] T 66ae176961b74813a58bc15c9c4c6f2a P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860
I20260709 13:21:48.702075 22404 raft_consensus.cc:2804] T e5ad4eb281564936a59f41e387a47156 P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:21:48.722245 16223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02838c86531d4f98bf58d1c22b8baf77" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:21:48.724046 16223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e4bfb9090c8460eb14fa7e0d818e7f1" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:21:48.725471 16223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5718c623ebee4d2bb770f0f9ddb2f164" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:48.726332 16223 raft_consensus.cc:2393] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 in current term 17: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
W20260709 13:21:48.707398 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 74259.4us
I20260709 13:21:48.728799 16265 leader_election.cc:304] T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 647a343a542d4a139fe106c19095129b
I20260709 13:21:48.722625 22404 raft_consensus.cc:493] T e5ad4eb281564936a59f41e387a47156 P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:21:48.730139 22404 raft_consensus.cc:3060] T e5ad4eb281564936a59f41e387a47156 P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:21:48.704221 16190 leader_election.cc:304] T 8a878999c0024d2abe517484180290fc P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:21:48.738740 16190 leader_election.cc:304] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:21:48.740259 16266 leader_election.cc:304] T d2ab0ac8347641e6955e9bce7d90c1a2 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860
I20260709 13:21:48.739008 16221 raft_consensus.cc:2468] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 16.
I20260709 13:21:48.693504 22502 raft_consensus.cc:2749] T d2ab0ac8347641e6955e9bce7d90c1a2 P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:21:48.743646 16146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5718c623ebee4d2bb770f0f9ddb2f164" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:48.741382 16223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4dc6b0541ca431ebc757beac00521f1" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 17 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:48.740777 16190 leader_election.cc:304] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860
I20260709 13:21:48.742303 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87147731a68400e8cce9b879daa7c5d" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:21:48.748027 16142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5d59407eb545b895d8d922e0dd69fd" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:48.748462 16223 raft_consensus.cc:2410] T e4dc6b0541ca431ebc757beac00521f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 for term 17 because replica has last-logged OpId of term: 16 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 7.
I20260709 13:21:48.748530 16149 raft_consensus.cc:3060] T b87147731a68400e8cce9b879daa7c5d P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:21:48.746716 16217 raft_consensus.cc:2468] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 18.
I20260709 13:21:48.751969 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ad5dbfa99ee4b8d9e28be92b3f36db5" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 17 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:48.641958 22202 consensus_queue.cc:1059] T ada5a004ba7c46819051e3ccec71ffdb P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: INVALID_TERM, Last received: 17.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.064s
I20260709 13:21:48.706503 22496 raft_consensus.cc:2804] T c0eab25ce4c64e018a041beeda2867db P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:21:48.748858 16151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57a744f5aaf944678a63469cbf514015" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:48.737399 22168 consensus_queue.cc:1059] T 84f2da98864b4425bf7c0055951fdd4b P 647a343a542d4a139fe106c19095129b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.017s
I20260709 13:21:48.754462 22430 consensus_queue.cc:1048] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:21:48.756592 16145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57a744f5aaf944678a63469cbf514015" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:21:48.706691 22196 raft_consensus.cc:493] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Starting leader election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:21:48.758998 16144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746aabf856ab401c9ef5e859371e7b67" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:21:48.736848 22517 raft_consensus.cc:2804] T 18ad7e3af5504571a69d10c00750a01c P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:21:48.737255 22524 raft_consensus.cc:3060] T 66ae176961b74813a58bc15c9c4c6f2a P 90f3da57796b488c9087a2f523adf045 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:21:48.743760 22348 raft_consensus.cc:2804] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:21:48.705754 22457 consensus_queue.cc:1048] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.056s
W20260709 13:21:48.781522 16190 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:48.743911 22481 raft_consensus.cc:2804] T 8a878999c0024d2abe517484180290fc P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:21:48.752038 22484 raft_consensus.cc:2749] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:21:48.770797 16217 raft_consensus.cc:1240] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 15. Current term is 16. Ops: [15.10-15.10]
I20260709 13:21:48.803551 16217 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57600 (request call id 1857) took 1567 ms. Trace:
I20260709 13:21:48.803920 16217 rpcz_store.cc:276] 0709 13:21:47.236145 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:48.770097 (+1533952us) service_pool.cc:224] Handling call
0709 13:21:48.770499 (+   402us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:21:48.803250 (+ 32751us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:48.803487 (+   237us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:21:48.762962 22408 raft_consensus.cc:2804] T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:21:48.805451 16217 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57604 (request call id 3176) took 1368 ms. Trace:
W20260709 13:21:48.782387 16190 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:48.766077 22196 raft_consensus.cc:3060] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:21:48.808555 16146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ad5dbfa99ee4b8d9e28be92b3f36db5" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:48.744377 22478 consensus_queue.cc:1059] T ee9ed67006144797b1de6ea8c2013a68 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.033s
I20260709 13:21:48.811180 16217 rpcz_store.cc:276] 0709 13:21:47.436831 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:48.804649 (+1367818us) service_pool.cc:224] Handling call
0709 13:21:48.804955 (+   306us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:21:48.805154 (+   199us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:21:48.805178 (+    24us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:21:48.805219 (+    41us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:21:48.805248 (+    29us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:48.805300 (+    52us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:21:48.805400 (+   100us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:21:48.762822 22349 raft_consensus.cc:2749] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:21:48.763453 22496 raft_consensus.cc:493] T c0eab25ce4c64e018a041beeda2867db P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Starting leader election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:21:48.828516 22088 raft_consensus.cc:3060] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:21:48.823021 22502 raft_consensus.cc:493] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:21:48.831748 22517 raft_consensus.cc:697] T 18ad7e3af5504571a69d10c00750a01c P 647a343a542d4a139fe106c19095129b [term 15 LEADER]: Becoming Leader. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:21:48.833390 16143 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46224 (request call id 1011) took 1575 ms. Trace:
I20260709 13:21:48.838042 22399 raft_consensus.cc:515] T 8d4ab870a05347868956aa8b841b2981 P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:48.753266 22483 raft_consensus.cc:515] T 8cc959fb7f704fba9009c0595467d074 P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:21:48.847402 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 118708us
I20260709 13:21:48.775040 22348 raft_consensus.cc:493] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Starting leader election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:21:48.760205 22480 consensus_queue.cc:1048] T 08c74490c40243a88e62d1760b5cd814 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.060s
I20260709 13:21:48.800354 22104 raft_consensus.cc:2749] T d2ab0ac8347641e6955e9bce7d90c1a2 P 90f3da57796b488c9087a2f523adf045 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:21:48.795645 22481 raft_consensus.cc:493] T 8a878999c0024d2abe517484180290fc P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Starting leader election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:21:48.850957 22496 raft_consensus.cc:3060] T c0eab25ce4c64e018a041beeda2867db P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:21:48.853808 16143 rpcz_store.cc:276] 0709 13:21:47.277291 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:47.500852 (+223561us) service_pool.cc:224] Handling call
0709 13:21:47.525575 (+ 24723us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:21:48.148196 (+622621us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:48.833126 (+684930us) spinlock_profiling.cc:243] Waited 605 ms on lock 0x7b540041e1d4. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:21:48.833333 (+   207us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1390417280}
I20260709 13:21:48.811134 22408 raft_consensus.cc:493] T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 647a343a542d4a139fe106c19095129b)
W20260709 13:21:48.862432 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 110468us
I20260709 13:21:48.874897 16265 leader_election.cc:304] T cdbd78acb4cf46298e0de8ba0d130e19 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860
I20260709 13:21:48.742187 22522 raft_consensus.cc:515] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:48.858239 22348 raft_consensus.cc:3060] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:21:48.883720 16825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f82224f4564fea93572e8206bad1fe" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:21:48.884720 16825 raft_consensus.cc:2393] T 23f82224f4564fea93572e8206bad1fe P cb82cc7bd6db46d980c75237a1402860 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 15: Already voted for candidate cb82cc7bd6db46d980c75237a1402860 in this term.
I20260709 13:21:48.872189 22481 raft_consensus.cc:3060] T 8a878999c0024d2abe517484180290fc P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:21:48.848009 16117 leader_election.cc:304] T 434bff3082d141838068f323bdb0d1ed P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:21:48.852833 22478 consensus_queue.cc:1243] T ee9ed67006144797b1de6ea8c2013a68 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Peer 647a343a542d4a139fe106c19095129b log is divergent from this leader: its last log entry 16.13 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 13:21:48.872565 16218 raft_consensus.cc:2468] T 746aabf856ab401c9ef5e859371e7b67 P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 14.
I20260709 13:21:48.876448 16143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62fe3346cd3c42c4a55e14af9073bca4" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:48.892287 16143 raft_consensus.cc:2468] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 22.
I20260709 13:21:48.893270 16143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b7d1ab519044eda2d5e04a07242f25" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:48.893862 16143 raft_consensus.cc:2468] T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 15.
W20260709 13:21:48.894037 16264 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:48.894699 16143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68a6838d82ef4cac83507f906139d90f" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:21:48.863396 16146 raft_consensus.cc:1240] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 17. Current term is 18. Ops: [17.12-17.12]
I20260709 13:21:48.894904 16816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2794af5b99984efebdee564035841cb0" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
W20260709 13:21:48.895823 16117 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 47915.8us
I20260709 13:21:48.888012 16822 raft_consensus.cc:1240] T 57a744f5aaf944678a63469cbf514015 P cb82cc7bd6db46d980c75237a1402860 [term 13 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 12. Current term is 13. Ops: []
I20260709 13:21:48.876108 22408 raft_consensus.cc:3060] T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:21:48.875907 16790 leader_election.cc:304] T 23f82224f4564fea93572e8206bad1fe P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:21:48.899344 16818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee9ed67006144797b1de6ea8c2013a68" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
W20260709 13:21:48.895838 16266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 155631us
W20260709 13:21:48.867375 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 123549us
I20260709 13:21:48.897523 16817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2372981006d4b218215c28964dea623" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:21:48.898746 16821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3884f864a96d428785895e5463e5728f" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:21:48.901554 16821 raft_consensus.cc:3060] T 3884f864a96d428785895e5463e5728f P cb82cc7bd6db46d980c75237a1402860 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:21:48.901109 16116 leader_election.cc:304] T 5a96373bb0e14dce8a266b2fc6c58507 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
I20260709 13:21:48.890405 22482 raft_consensus.cc:2749] T cdbd78acb4cf46298e0de8ba0d130e19 P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:21:48.909026 16790 leader_election.cc:304] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:21:48.872687 22502 raft_consensus.cc:515] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:48.909771 16265 leader_election.cc:304] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 647a343a542d4a139fe106c19095129b
I20260709 13:21:48.915966 16143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fffab26427e24d398e2695aad40ca134" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:48.904410 22197 raft_consensus.cc:2804] T 434bff3082d141838068f323bdb0d1ed P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Leader election won for term 16
W20260709 13:21:48.923804 22530 log.cc:927] Time spent T d16dfad0c3b540e5b4b86492ddf87208 P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 0.084s	user 0.000s	sys 0.000s
I20260709 13:21:48.929001 22182 leader_election.cc:290] T 23f82224f4564fea93572e8206bad1fe P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 15 election: Requested vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:48.908100 22478 raft_consensus.cc:3055] T ee9ed67006144797b1de6ea8c2013a68 P cb82cc7bd6db46d980c75237a1402860 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:21:48.932960 22537 raft_consensus.cc:2804] T 5a96373bb0e14dce8a266b2fc6c58507 P 647a343a542d4a139fe106c19095129b [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:21:48.950901 16295 raft_consensus.cc:1275] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 18 index: 15. Preceding OpId from leader: term: 19 index: 15. (term mismatch)
I20260709 13:21:48.986194 16295 pending_rounds.cc:85] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 90f3da57796b488c9087a2f523adf045: Aborting all ops after (but not including) 14
I20260709 13:21:48.986555 16295 pending_rounds.cc:107] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 90f3da57796b488c9087a2f523adf045: Aborting uncommitted NO_OP operation due to leader change: 18.15
I20260709 13:21:48.986912 16295 raft_consensus.cc:2889] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:21:48.932659 22399 leader_election.cc:290] T 8d4ab870a05347868956aa8b841b2981 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 16 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
W20260709 13:21:48.951360 22535 log.cc:927] Time spent T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860: Append to log took a long time: real 0.062s	user 0.001s	sys 0.000s
I20260709 13:21:48.612464 16226 rpcz_store.cc:276] 0709 13:21:47.092528 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:48.258382 (+1165854us) service_pool.cc:224] Handling call
0709 13:21:48.258661 (+   279us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:21:48.258843 (+   182us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:21:48.337991 (+ 79148us) log.cc:844] Serialized 14 byte log entry
0709 13:21:48.611715 (+273724us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:21:48.611771 (+    56us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:21:48.611824 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:48.611889 (+    65us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:21:48.611994 (+   105us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":28634,"prepare.run_cpu_time_us":67,"prepare.run_wall_time_us":2916,"raft.queue_time_us":15500,"thread_start_us":257754,"threads_started":1}
W20260709 13:21:49.001690 16266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 94396us
I20260709 13:21:48.998364 22482 consensus_queue.cc:1059] T 434bff3082d141838068f323bdb0d1ed P 90f3da57796b488c9087a2f523adf045 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: INVALID_TERM, Last received: 15.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.043s
I20260709 13:21:49.042356 22483 leader_election.cc:290] T 8cc959fb7f704fba9009c0595467d074 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:21:48.997787 22517 consensus_queue.cc:237] T 18ad7e3af5504571a69d10c00750a01c P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } }
I20260709 13:21:48.985946 22478 raft_consensus.cc:740] T ee9ed67006144797b1de6ea8c2013a68 P cb82cc7bd6db46d980c75237a1402860 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:21:49.051896 22478 consensus_queue.cc:260] T ee9ed67006144797b1de6ea8c2013a68 P cb82cc7bd6db46d980c75237a1402860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.14, Last appended by leader: 14, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:49.053169 22478 raft_consensus.cc:3060] T ee9ed67006144797b1de6ea8c2013a68 P cb82cc7bd6db46d980c75237a1402860 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:21:49.056005 22522 leader_election.cc:290] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:49.056280 22526 raft_consensus.cc:515] T 8cc959fb7f704fba9009c0595467d074 P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:49.060025 22518 raft_consensus.cc:515] T 8d4ab870a05347868956aa8b841b2981 P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:21:49.061022 22508 consensus_meta.cc:287] T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860: Time spent flushing consensus metadata: real 0.509s	user 0.010s	sys 0.000s
I20260709 13:21:49.061535 22508 raft_consensus.cc:515] T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:49.062506 22518 leader_election.cc:290] T 8d4ab870a05347868956aa8b841b2981 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 16 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:49.008535 22537 raft_consensus.cc:697] T 5a96373bb0e14dce8a266b2fc6c58507 P 647a343a542d4a139fe106c19095129b [term 14 LEADER]: Becoming Leader. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:21:49.065876 22537 consensus_queue.cc:237] T 5a96373bb0e14dce8a266b2fc6c58507 P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:49.066169 22506 raft_consensus.cc:515] T 19baa7bc367844c288bca8f81439c413 P cb82cc7bd6db46d980c75237a1402860 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:49.011276 22436 raft_consensus.cc:515] T 31969392ccdb43b491911a319e8bc7e1 P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:49.069849 22436 leader_election.cc:290] T 31969392ccdb43b491911a319e8bc7e1 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 16 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:49.016662 22474 raft_consensus.cc:3055] T 84f2da98864b4425bf7c0055951fdd4b P 647a343a542d4a139fe106c19095129b [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:21:49.070907 22474 raft_consensus.cc:740] T 84f2da98864b4425bf7c0055951fdd4b P 647a343a542d4a139fe106c19095129b [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:21:48.936069 22545 raft_consensus.cc:2749] T 23f82224f4564fea93572e8206bad1fe P cb82cc7bd6db46d980c75237a1402860 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:21:49.071885 22474 consensus_queue.cc:260] T 84f2da98864b4425bf7c0055951fdd4b P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:49.073256 22474 raft_consensus.cc:3060] T 84f2da98864b4425bf7c0055951fdd4b P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:21:48.996603 22505 raft_consensus.cc:3060] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:21:49.017971 16148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cc959fb7f704fba9009c0595467d074" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:49.076064 16145 raft_consensus.cc:1240] T 8d4ab870a05347868956aa8b841b2981 P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 15. Current term is 16. Ops: []
I20260709 13:21:49.022328 16220 raft_consensus.cc:1275] T 1df93779209748b9860f6d893dd3de53 P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 13:21:49.035219 16226 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33050 (request call id 1089) took 1186 ms. Trace:
I20260709 13:21:49.026177 16147 raft_consensus.cc:2410] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 for term 16 because replica has last-logged OpId of term: 15 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 11.
I20260709 13:21:49.078917 16220 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57604 (request call id 3156) took 2674 ms. Trace:
I20260709 13:21:49.079627 16220 rpcz_store.cc:276] 0709 13:21:46.404161 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:47.674302 (+1270141us) service_pool.cc:224] Handling call
0709 13:21:47.710132 (+ 35830us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:21:48.602272 (+892140us) spinlock_profiling.cc:243] Waited 23.3 ms on lock 0x7b4c00070fc0. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69aee2ba20 00007f69aee3ca29 00007f69aee02644 00007f69aee29f2d 00007f69aee29b93 00007f69b0b5a18b 00007f69b0b5acb5 00007f69b0b55f30 00007f69b0b577de 00007f69b0b61b49 00007f69b0b62416
0709 13:21:49.078644 (+476372us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:49.078865 (+   221us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":53482368}
I20260709 13:21:49.078715 16226 rpcz_store.cc:276] 0709 13:21:47.848897 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:49.014613 (+1165716us) service_pool.cc:224] Handling call
0709 13:21:49.014931 (+   318us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:21:49.015116 (+   185us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:21:49.015141 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:21:49.034908 (+ 19767us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:21:49.034955 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:49.035011 (+    56us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:21:49.035125 (+   114us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:21:49.086809 16226 raft_consensus.cc:1240] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 17. Current term is 18. Ops: []
W20260709 13:21:49.088882 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 179171us
I20260709 13:21:49.090121 16265 leader_election.cc:400] T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 16 pre-election: Vote denied by peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943) with higher term. Message: Invalid argument: T b6f07842d3aa47858df5665874c86f3a P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 for earlier term 16. Current term is 17.
I20260709 13:21:49.093411 16821 raft_consensus.cc:2468] T 3884f864a96d428785895e5463e5728f P cb82cc7bd6db46d980c75237a1402860 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 13.
I20260709 13:21:49.098102 16226 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33050 (request call id 1092) took 1237 ms. Trace:
I20260709 13:21:49.099072 16226 rpcz_store.cc:276] 0709 13:21:47.860412 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:49.082445 (+1222033us) service_pool.cc:224] Handling call
0709 13:21:49.082709 (+   264us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:21:49.097693 (+ 14984us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:49.098016 (+   323us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:21:49.026773 22285 raft_consensus.cc:515] T fffab26427e24d398e2695aad40ca134 P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:49.103868 22285 leader_election.cc:290] T fffab26427e24d398e2695aad40ca134 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 17 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:49.002895 22439 raft_consensus.cc:2804] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:21:49.106797 22439 raft_consensus.cc:493] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:21:49.107254 22439 raft_consensus.cc:3060] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:21:49.114583 16226 raft_consensus.cc:1240] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [term 20 FOLLOWER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 19. Current term is 20. Ops: []
I20260709 13:21:49.093279 16266 leader_election.cc:304] T 57a744f5aaf944678a63469cbf514015 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
I20260709 13:21:49.096591 16149 raft_consensus.cc:2468] T b87147731a68400e8cce9b879daa7c5d P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 18.
I20260709 13:21:48.996737 22197 raft_consensus.cc:697] T 434bff3082d141838068f323bdb0d1ed P 647a343a542d4a139fe106c19095129b [term 16 LEADER]: Becoming Leader. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
W20260709 13:21:49.086192 22292 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 16 current term: 16 is higher.
I20260709 13:21:49.122563 16226 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33050 (request call id 1093) took 1261 ms. Trace:
I20260709 13:21:49.122951 16226 rpcz_store.cc:276] 0709 13:21:47.860753 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:49.100318 (+1239565us) service_pool.cc:224] Handling call
0709 13:21:49.114426 (+ 14108us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:21:49.122320 (+  7894us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:49.122509 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:21:49.124434 16226 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33050 (request call id 1094) took 1164 ms. Trace:
I20260709 13:21:49.124727 16226 rpcz_store.cc:276] 0709 13:21:47.959887 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:49.123674 (+1163787us) service_pool.cc:224] Handling call
0709 13:21:49.123938 (+   264us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:21:49.124150 (+   212us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:21:49.124174 (+    24us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:21:49.124213 (+    39us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:21:49.124249 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:49.124299 (+    50us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:21:49.124387 (+    88us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:21:49.054498 22496 raft_consensus.cc:515] T c0eab25ce4c64e018a041beeda2867db P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:49.127988 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f5f89186204242bf377f7c42b2e2da" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:49.126339 16226 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33050 (request call id 1095) took 1134 ms. Trace:
I20260709 13:21:49.130501 16226 rpcz_store.cc:276] 0709 13:21:47.992219 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:49.125522 (+1133303us) service_pool.cc:224] Handling call
0709 13:21:49.125766 (+   244us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:21:49.125935 (+   169us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:21:49.125972 (+    37us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:21:49.126028 (+    56us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:21:49.126094 (+    66us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:49.126163 (+    69us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:21:49.126279 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:21:49.109930 22404 raft_consensus.cc:515] T e5ad4eb281564936a59f41e387a47156 P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:49.018477 22524 raft_consensus.cc:2749] T 66ae176961b74813a58bc15c9c4c6f2a P 90f3da57796b488c9087a2f523adf045 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:21:49.059060 22526 leader_election.cc:290] T 8cc959fb7f704fba9009c0595467d074 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 16 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:49.132567 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f2da98864b4425bf7c0055951fdd4b" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:21:49.129885 16265 leader_election.cc:304] T 8638ee363273426db1393dd81e5c7b72 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:21:49.026814 22502 leader_election.cc:290] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:49.138450 16148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fffab26427e24d398e2695aad40ca134" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:49.148089 22408 raft_consensus.cc:515] T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:49.150193 22408 leader_election.cc:290] T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 18 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:21:49.151968 16295 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40474 (request call id 1111) took 1989 ms. Trace:
W20260709 13:21:49.155925 16266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 62979.3us
I20260709 13:21:49.157274 22399 raft_consensus.cc:2749] T 57a744f5aaf944678a63469cbf514015 P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:21:49.158681 22496 leader_election.cc:290] T c0eab25ce4c64e018a041beeda2867db P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 17 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:49.159394 16217 raft_consensus.cc:1240] T c0eab25ce4c64e018a041beeda2867db P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 16. Current term is 17. Ops: []
I20260709 13:21:49.160784 16217 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57604 (request call id 3177) took 1723 ms. Trace:
I20260709 13:21:49.161180 16217 rpcz_store.cc:276] 0709 13:21:47.437167 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:48.841192 (+1404025us) service_pool.cc:224] Handling call
0709 13:21:48.841419 (+   227us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:21:49.159859 (+318440us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:49.160523 (+   664us) spinlock_profiling.cc:243] Waited 318 ms on lock 0x7b54002981d4. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:21:49.160715 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":730801920}
I20260709 13:21:49.164662 22246 leader_election.cc:290] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:49.113988 22520 raft_consensus.cc:515] T 5a959dbecc2648eb85d1cc550f5a3245 P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:49.158727 16266 leader_election.cc:304] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:21:49.174201 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62fe3346cd3c42c4a55e14af9073bca4" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:49.175118 16151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8158bbd17b994d2eb96f560bd0f1b88d" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:49.193079 16150 raft_consensus.cc:2468] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 22.
I20260709 13:21:49.193413 16151 raft_consensus.cc:2393] T 8158bbd17b994d2eb96f560bd0f1b88d P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 16: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
I20260709 13:21:49.152371 16295 rpcz_store.cc:276] 0709 13:21:47.162362 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:47.162517 (+   155us) service_pool.cc:224] Handling call
0709 13:21:47.183618 (+ 21101us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:21:49.002521 (+1818903us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:49.151715 (+149194us) spinlock_profiling.cc:243] Waited 1.77 s on lock 0x7b5400163354. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:21:49.151888 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4063546752}
W20260709 13:21:49.175906 16791 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:49.194896 16821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a378d7f796f94615b3b347119a9880fd" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:49.151875 16148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a96373bb0e14dce8a266b2fc6c58507" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:49.196283 16819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8638ee363273426db1393dd81e5c7b72" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:49.174140 16221 raft_consensus.cc:1275] T 68a6838d82ef4cac83507f906139d90f P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
I20260709 13:21:49.194172 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66f75124215f4ca6bef831e174a792d8" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
W20260709 13:21:49.175475 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 45700.2us
W20260709 13:21:49.197242 16266 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:21:49.184183 16192 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:21:49.197672 16266 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:49.172721 16225 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33050 (request call id 1098) took 1004 ms. Trace:
I20260709 13:21:49.203531 16225 rpcz_store.cc:276] 0709 13:21:48.167916 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:49.171814 (+1003898us) service_pool.cc:224] Handling call
0709 13:21:49.172180 (+   366us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:21:49.172367 (+   187us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:21:49.172396 (+    29us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:21:49.172444 (+    48us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:21:49.172482 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:49.172538 (+    56us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:21:49.172658 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:21:49.026125 22152 raft_consensus.cc:515] T 8c11074579b046d99e585c2818e18a2c P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:49.208832 16226 raft_consensus.cc:1275] T ecc1e363230043fe9fef390f0b979f20 P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
W20260709 13:21:49.206774 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 305705us
I20260709 13:21:49.173910 22508 leader_election.cc:290] T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 20 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:21:49.216832 16819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9944d5e4b8c4109af9955c4f08c9095" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:49.220115 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cc959fb7f704fba9009c0595467d074" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:49.226850 16823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2794af5b99984efebdee564035841cb0" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:21:49.219245 16226 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33050 (request call id 1096) took 1202 ms. Trace:
I20260709 13:21:49.228454 16226 rpcz_store.cc:276] 0709 13:21:48.017144 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:49.131372 (+1114228us) service_pool.cc:224] Handling call
0709 13:21:49.180864 (+ 49492us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:21:49.213608 (+ 32744us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:49.219191 (+  5583us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:21:49.229745 16148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ad5dbfa99ee4b8d9e28be92b3f36db5" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:49.230898 16225 raft_consensus.cc:1240] T fffab26427e24d398e2695aad40ca134 P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 16. Current term is 17. Ops: []
I20260709 13:21:49.174862 22399 raft_consensus.cc:2804] T 8638ee363273426db1393dd81e5c7b72 P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:21:49.232923 16265 leader_election.cc:304] T c9944d5e4b8c4109af9955c4f08c9095 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860
I20260709 13:21:49.248139 16824 raft_consensus.cc:1240] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P cb82cc7bd6db46d980c75237a1402860 [term 18 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 17. Current term is 18. Ops: []
I20260709 13:21:49.249027 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:49.249681 16293 raft_consensus.cc:2468] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 19.
I20260709 13:21:49.176322 22506 leader_election.cc:290] T 19baa7bc367844c288bca8f81439c413 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 20 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:49.236101 16296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31969392ccdb43b491911a319e8bc7e1" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:49.240228 16116 leader_election.cc:304] T 8158bbd17b994d2eb96f560bd0f1b88d P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
I20260709 13:21:49.208666 22482 raft_consensus.cc:2804] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:21:49.252588 16294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c61a39fd4f84424b129c1a6f9ba404d" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:49.253208 16223 raft_consensus.cc:1240] T 8c11074579b046d99e585c2818e18a2c P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 17. Current term is 18. Ops: [17.14-17.14]
I20260709 13:21:49.261545 16291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f5f89186204242bf377f7c42b2e2da" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:49.253407 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66f75124215f4ca6bef831e174a792d8" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:49.255573 16819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26354361e9b044cea1c2b86c4882fe92" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:49.258178 16295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6f07842d3aa47858df5665874c86f3a" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:49.259534 16823 raft_consensus.cc:1240] T 23f82224f4564fea93572e8206bad1fe P cb82cc7bd6db46d980c75237a1402860 [term 15 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 14. Current term is 15. Ops: []
I20260709 13:21:49.261562 16294 raft_consensus.cc:2393] T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 18: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
I20260709 13:21:49.232964 16825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2794af5b99984efebdee564035841cb0" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:49.263641 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ad5dbfa99ee4b8d9e28be92b3f36db5" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:49.264526 16297 raft_consensus.cc:2468] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 16.
I20260709 13:21:49.265836 16226 raft_consensus.cc:1240] T 5a96373bb0e14dce8a266b2fc6c58507 P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 13. Current term is 14. Ops: []
I20260709 13:21:49.212373 22492 raft_consensus.cc:493] T 18ad7e3af5504571a69d10c00750a01c P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:49.171015 22196 raft_consensus.cc:515] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:49.212807 22545 raft_consensus.cc:493] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:21:49.273766 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d16dfad0c3b540e5b4b86492ddf87208" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:49.274514 16293 raft_consensus.cc:2393] T d16dfad0c3b540e5b4b86492ddf87208 P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 18: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
I20260709 13:21:49.259253 22246 raft_consensus.cc:2804] T 8158bbd17b994d2eb96f560bd0f1b88d P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:21:49.269459 16148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b7d1ab519044eda2d5e04a07242f25" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:49.270738 16265 leader_election.cc:304] T 8a878999c0024d2abe517484180290fc P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 647a343a542d4a139fe106c19095129b
I20260709 13:21:49.268263 16295 raft_consensus.cc:2393] T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 17: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
I20260709 13:21:49.167093 22481 raft_consensus.cc:515] T 8a878999c0024d2abe517484180290fc P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:49.218020 22502 raft_consensus.cc:493] T e4dc6b0541ca431ebc757beac00521f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:49.275555 16296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f2da98864b4425bf7c0055951fdd4b" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:21:49.269948 16151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a96373bb0e14dce8a266b2fc6c58507" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:49.280898 16296 raft_consensus.cc:3060] T 84f2da98864b4425bf7c0055951fdd4b P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:21:49.252904 22399 raft_consensus.cc:493] T 8638ee363273426db1393dd81e5c7b72 P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Starting leader election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:21:49.300397 16294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74ca8140f35c4ae29e8a664f6bdfc829" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:21:49.309727 16294 raft_consensus.cc:3060] T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:21:49.260439 22404 leader_election.cc:290] T e5ad4eb281564936a59f41e387a47156 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 17 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:21:49.287658 16819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31969392ccdb43b491911a319e8bc7e1" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:21:49.270411 22524 raft_consensus.cc:2749] T c9944d5e4b8c4109af9955c4f08c9095 P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:21:49.319927 16819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62fe3346cd3c42c4a55e14af9073bca4" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:49.320973 16819 raft_consensus.cc:2468] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 22.
I20260709 13:21:49.311009 16824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a9de347c711477e87c582d003f36925" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 21 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:49.311417 16142 raft_consensus.cc:2468] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 19.
I20260709 13:21:49.313129 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c74490c40243a88e62d1760b5cd814" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:49.323407 16297 raft_consensus.cc:2393] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 16: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
I20260709 13:21:49.323918 16151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87ac0d1f7e8a419f9bcbc04aee0e067e" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:49.275288 16148 raft_consensus.cc:2468] T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 15.
W20260709 13:21:49.318558 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 47904.4us
I20260709 13:21:49.329205 16151 raft_consensus.cc:2393] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 18: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
I20260709 13:21:49.300441 22399 raft_consensus.cc:3060] T 8638ee363273426db1393dd81e5c7b72 P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:21:49.334002 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a96373bb0e14dce8a266b2fc6c58507" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:49.330909 16295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8158bbd17b994d2eb96f560bd0f1b88d" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:49.335616 16819 raft_consensus.cc:1240] T 19baa7bc367844c288bca8f81439c413 P cb82cc7bd6db46d980c75237a1402860 [term 20 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 19. Current term is 20. Ops: []
I20260709 13:21:49.273200 22482 raft_consensus.cc:697] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045 [term 18 LEADER]: Becoming Leader. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:21:49.346315 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:49.318240 22553 raft_consensus.cc:2804] T 8a878999c0024d2abe517484180290fc P 90f3da57796b488c9087a2f523adf045 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:21:49.337555 22548 consensus_queue.cc:1059] T c0eab25ce4c64e018a041beeda2867db P 647a343a542d4a139fe106c19095129b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: INVALID_TERM, Last received: 16.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.034s
I20260709 13:21:49.376047 16295 raft_consensus.cc:1217] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Deduplicated request from leader. Original: 10.8->[12.9-17.11]   Dedup: 12.9->[15.10-17.11]
I20260709 13:21:49.328264 22505 raft_consensus.cc:2749] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:21:49.376981 16295 raft_consensus.cc:1240] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 17. Current term is 18. Ops: [12.9-17.11]
I20260709 13:21:49.328883 22517 raft_consensus.cc:493] T af57996a8be84d398f0f023e8d08ef77 P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:49.280794 22480 consensus_queue.cc:1059] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: INVALID_TERM, Last received: 17.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.090s
I20260709 13:21:49.380851 16148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806d7690c2b644959152074bb2cab0a7" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 18 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:21:49.390532 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3884f864a96d428785895e5463e5728f" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:49.301254 22502 raft_consensus.cc:515] T e4dc6b0541ca431ebc757beac00521f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:49.413625 22502 leader_election.cc:290] T e4dc6b0541ca431ebc757beac00521f1 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:49.285566 22182 consensus_queue.cc:1048] T 1df93779209748b9860f6d893dd3de53 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.049s
W20260709 13:21:49.414201 16189 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:49.415576 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee9ed67006144797b1de6ea8c2013a68" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
W20260709 13:21:49.416942 16224 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33050 (request call id 1086) took 2206 ms (client timeout 1545 ms). Trace:
W20260709 13:21:49.417364 16224 rpcz_store.cc:269] 0709 13:21:47.211035 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:47.226309 (+ 15274us) service_pool.cc:224] Handling call
0709 13:21:49.416783 (+2190474us) spinlock_profiling.cc:243] Waited 640 ms on lock 0x7b5400395154. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:21:49.416880 (+    97us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1472108800}
I20260709 13:21:49.424588 22106 raft_consensus.cc:3055] T 8158bbd17b994d2eb96f560bd0f1b88d P 90f3da57796b488c9087a2f523adf045 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:21:49.425081 22106 raft_consensus.cc:740] T 8158bbd17b994d2eb96f560bd0f1b88d P 90f3da57796b488c9087a2f523adf045 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:21:49.425980 22106 consensus_queue.cc:260] T 8158bbd17b994d2eb96f560bd0f1b88d P 90f3da57796b488c9087a2f523adf045 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:21:49.427575 22556 log.cc:927] Time spent T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 0.059s	user 0.000s	sys 0.001s
I20260709 13:21:49.429131 16296 raft_consensus.cc:2468] T 84f2da98864b4425bf7c0055951fdd4b P 90f3da57796b488c9087a2f523adf045 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 14.
I20260709 13:21:49.431046 16220 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33050 (request call id 1091) took 1573 ms. Trace:
I20260709 13:21:49.431449 16220 rpcz_store.cc:276] 0709 13:21:47.857512 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:49.080820 (+1223308us) service_pool.cc:224] Handling call
0709 13:21:49.081274 (+   454us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:21:49.128798 (+ 47524us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:21:49.128847 (+    49us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:21:49.135894 (+  7047us) log.cc:844] Serialized 52 byte log entry
0709 13:21:49.319240 (+183346us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:21:49.319286 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:49.319358 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:21:49.430728 (+111370us) raft_consensus.cc:1685] finished
0709 13:21:49.430752 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:21:49.430985 (+   233us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":22007,"raft.run_cpu_time_us":158,"raft.run_wall_time_us":15224,"thread_start_us":102525,"threads_started":1,"wal-append.queue_time_us":246672}
I20260709 13:21:49.276363 22197 consensus_queue.cc:237] T 434bff3082d141838068f323bdb0d1ed P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:49.437058 16147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d16dfad0c3b540e5b4b86492ddf87208" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:21:49.438443 16147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fffab26427e24d398e2695aad40ca134" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:21:49.381856 22246 raft_consensus.cc:697] T 8158bbd17b994d2eb96f560bd0f1b88d P 647a343a542d4a139fe106c19095129b [term 16 LEADER]: Becoming Leader. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
W20260709 13:21:49.397671 22551 log.cc:927] Time spent T 23f82224f4564fea93572e8206bad1fe P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.078s	user 0.000s	sys 0.000s
I20260709 13:21:49.276044 22492 raft_consensus.cc:515] T 18ad7e3af5504571a69d10c00750a01c P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } }
I20260709 13:21:49.440315 22480 raft_consensus.cc:3055] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:21:49.440701 22480 raft_consensus.cc:740] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:21:49.441244 22492 leader_election.cc:290] T 18ad7e3af5504571a69d10c00750a01c P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:49.402501 21704 leader_election.cc:290] T ee9ed67006144797b1de6ea8c2013a68 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 18 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:49.407445 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdbd78acb4cf46298e0de8ba0d130e19" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:21:49.441684 16145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5718c623ebee4d2bb770f0f9ddb2f164" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:21:49.441440 22480 consensus_queue.cc:260] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:49.408886 22481 leader_election.cc:290] T 8a878999c0024d2abe517484180290fc P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 15 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:49.410130 22196 leader_election.cc:290] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 19 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:49.411412 22520 leader_election.cc:290] T 5a959dbecc2648eb85d1cc550f5a3245 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 16 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:49.442766 16145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:21:49.438844 22517 raft_consensus.cc:515] T af57996a8be84d398f0f023e8d08ef77 P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:49.382169 16265 leader_election.cc:304] T 84f2da98864b4425bf7c0055951fdd4b P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:21:49.450122 16295 raft_consensus.cc:1240] T 1df93779209748b9860f6d893dd3de53 P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 15. Current term is 16. Ops: [15.9-15.9]
I20260709 13:21:49.444296 16143 raft_consensus.cc:1275] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 16 index: 12. (term mismatch)
I20260709 13:21:49.453886 16143 pending_rounds.cc:85] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 647a343a542d4a139fe106c19095129b: Aborting all ops after (but not including) 11
I20260709 13:21:49.454216 16143 pending_rounds.cc:107] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 647a343a542d4a139fe106c19095129b: Aborting uncommitted NO_OP operation due to leader change: 15.12
I20260709 13:21:49.454525 16143 raft_consensus.cc:2889] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:21:49.455262 16223 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33050 (request call id 1087) took 2243 ms. Trace:
I20260709 13:21:49.455650 16223 rpcz_store.cc:276] 0709 13:21:47.211589 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:48.765898 (+1554309us) service_pool.cc:224] Handling call
0709 13:21:48.796052 (+ 30154us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:21:49.264250 (+468198us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:49.455000 (+190750us) spinlock_profiling.cc:243] Waited 457 ms on lock 0x7b5400367754. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:21:49.455202 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1050753920}
I20260709 13:21:49.280968 22545 raft_consensus.cc:515] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:49.446828 16149 raft_consensus.cc:2468] T 84f2da98864b4425bf7c0055951fdd4b P 647a343a542d4a139fe106c19095129b [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 14.
I20260709 13:21:49.391201 16151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18ad7e3af5504571a69d10c00750a01c" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:21:49.414215 22182 consensus_queue.cc:1243] T 1df93779209748b9860f6d893dd3de53 P 647a343a542d4a139fe106c19095129b [LEADER]: Peer 1ca2725d290e47d68b1a4b609c9816ac log is divergent from this leader: its last log entry 15.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 13:21:49.376044 22553 raft_consensus.cc:493] T 8a878999c0024d2abe517484180290fc P 90f3da57796b488c9087a2f523adf045 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:21:49.464320 16146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0eab25ce4c64e018a041beeda2867db" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
W20260709 13:21:49.567641 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 185716us
I20260709 13:21:49.483716 16296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e65676b21b748728ad39668cfe722ec" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:49.575791 22518 consensus_queue.cc:1048] T 68a6838d82ef4cac83507f906139d90f P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:21:49.567378 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f82224f4564fea93572e8206bad1fe" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:49.576611 22548 raft_consensus.cc:3055] T c0eab25ce4c64e018a041beeda2867db P 647a343a542d4a139fe106c19095129b [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:21:49.577098 22548 raft_consensus.cc:740] T c0eab25ce4c64e018a041beeda2867db P 647a343a542d4a139fe106c19095129b [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:21:49.567778 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434bff3082d141838068f323bdb0d1ed" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:49.568506 16217 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33050 (request call id 1097) took 1532 ms. Trace:
I20260709 13:21:49.578059 22548 consensus_queue.cc:260] T c0eab25ce4c64e018a041beeda2867db P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:49.578295 16217 rpcz_store.cc:276] 0709 13:21:48.036516 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:49.162207 (+1125691us) service_pool.cc:224] Handling call
0709 13:21:49.162560 (+   353us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:21:49.162782 (+   222us) raft_consensus.cc:1534] Early marking committed up to index 14
0709 13:21:49.277695 (+114913us) log.cc:844] Serialized 14 byte log entry
0709 13:21:49.567909 (+290214us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:21:49.567971 (+    62us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:21:49.568043 (+    72us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:49.568147 (+   104us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:21:49.568432 (+   285us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4074,"prepare.queue_time_us":247733,"prepare.run_cpu_time_us":36,"prepare.run_wall_time_us":37,"thread_start_us":400244,"threads_started":2}
I20260709 13:21:49.579643 22548 raft_consensus.cc:3060] T c0eab25ce4c64e018a041beeda2867db P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:21:49.580250 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62fe3346cd3c42c4a55e14af9073bca4" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:49.580965 16217 raft_consensus.cc:2468] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 21.
I20260709 13:21:49.582176 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a959dbecc2648eb85d1cc550f5a3245" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
W20260709 13:21:49.578059 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 149114us
I20260709 13:21:49.583359 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d4ab870a05347868956aa8b841b2981" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:21:49.584715 16217 raft_consensus.cc:1240] T 84f2da98864b4425bf7c0055951fdd4b P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 13. Current term is 14. Ops: []
I20260709 13:21:49.585430 22197 consensus_queue.cc:1059] T fffab26427e24d398e2695aad40ca134 P 647a343a542d4a139fe106c19095129b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: INVALID_TERM, Last received: 16.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:21:49.586818 22518 raft_consensus.cc:3055] T fffab26427e24d398e2695aad40ca134 P 647a343a542d4a139fe106c19095129b [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:21:49.587045 16217 raft_consensus.cc:1275] T e4dc6b0541ca431ebc757beac00521f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 16 index: 8. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
I20260709 13:21:49.587405 22518 raft_consensus.cc:740] T fffab26427e24d398e2695aad40ca134 P 647a343a542d4a139fe106c19095129b [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:21:49.588482 22518 consensus_queue.cc:260] T fffab26427e24d398e2695aad40ca134 P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:49.506254 16293 raft_consensus.cc:1240] T 8d4ab870a05347868956aa8b841b2981 P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 15. Current term is 16. Ops: []
W20260709 13:21:49.556115 16189 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:49.590088 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a959dbecc2648eb85d1cc550f5a3245" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:49.590103 22518 raft_consensus.cc:3060] T fffab26427e24d398e2695aad40ca134 P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:21:49.465847 16824 raft_consensus.cc:2410] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 for term 21 because replica has last-logged OpId of term: 20 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 13.
I20260709 13:21:49.442786 22480 raft_consensus.cc:3060] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:21:49.570242 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 50622.7us
I20260709 13:21:49.486848 16224 raft_consensus.cc:1240] T 8cc959fb7f704fba9009c0595467d074 P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 15. Current term is 16. Ops: []
I20260709 13:21:49.427094 22106 raft_consensus.cc:3060] T 8158bbd17b994d2eb96f560bd0f1b88d P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:21:49.520020 16223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5d59407eb545b895d8d922e0dd69fd" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:21:49.574573 22553 raft_consensus.cc:3060] T 8a878999c0024d2abe517484180290fc P 90f3da57796b488c9087a2f523adf045 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:21:49.483072 16294 raft_consensus.cc:2410] T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac for term 18 because replica has last-logged OpId of term: 16 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 12.
I20260709 13:21:49.575397 16147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5718c623ebee4d2bb770f0f9ddb2f164" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:49.576247 16296 raft_consensus.cc:2368] T 9e65676b21b748728ad39668cfe722ec P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b for earlier term 16. Current term is 17.
I20260709 13:21:49.571556 16265 leader_election.cc:304] T c0eab25ce4c64e018a041beeda2867db P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 647a343a542d4a139fe106c19095129b
I20260709 13:21:49.549000 22545 leader_election.cc:290] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:49.423141 22439 raft_consensus.cc:515] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:49.595583 16224 raft_consensus.cc:1240] T 8a878999c0024d2abe517484180290fc P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 14. Current term is 15. Ops: []
I20260709 13:21:49.608511 16146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fffab26427e24d398e2695aad40ca134" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:49.592283 16116 leader_election.cc:304] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
I20260709 13:21:49.604817 16148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f2da98864b4425bf7c0055951fdd4b" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:21:49.548679 22492 raft_consensus.cc:1064] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac: attempting to promote NON_VOTER cb82cc7bd6db46d980c75237a1402860 to VOTER
I20260709 13:21:49.482983 22524 raft_consensus.cc:2749] T 84f2da98864b4425bf7c0055951fdd4b P 90f3da57796b488c9087a2f523adf045 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:21:49.505843 22502 raft_consensus.cc:493] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:49.620247 16147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
W20260709 13:21:49.617707 16790 leader_election.cc:336] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 16 election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Timed out: RequestConsensusVote RPC to 127.14.177.129:36741 timed out after 1.504s (SENT)
W20260709 13:21:49.623438 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 51968us
I20260709 13:21:49.624015 16816 raft_consensus.cc:1240] T 434bff3082d141838068f323bdb0d1ed P cb82cc7bd6db46d980c75237a1402860 [term 16 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 15. Current term is 16. Ops: []
I20260709 13:21:49.625478 16265 leader_election.cc:304] T 68a6838d82ef4cac83507f906139d90f P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:21:49.627424 16265 leader_election.cc:304] T 8c11074579b046d99e585c2818e18a2c P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: cb82cc7bd6db46d980c75237a1402860
I20260709 13:21:49.596717 22285 raft_consensus.cc:493] T e5ad4eb281564936a59f41e387a47156 P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:21:49.657526 16265 leader_election.cc:304] T 1df93779209748b9860f6d893dd3de53 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:21:49.661732 16291 raft_consensus.cc:2393] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 18: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
I20260709 13:21:49.655530 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af57996a8be84d398f0f023e8d08ef77" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 19 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:49.596356 22543 consensus_queue.cc:1059] T ada5a004ba7c46819051e3ccec71ffdb P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: INVALID_TERM, Last received: 17.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.056s
I20260709 13:21:49.599701 22504 consensus_queue.cc:1059] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: INVALID_TERM, Last received: 19.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.060s
I20260709 13:21:49.603928 22482 consensus_queue.cc:237] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:49.783926 16219 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57604 (request call id 3169) took 2813 ms. Trace:
I20260709 13:21:49.616990 22168 raft_consensus.cc:2804] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:21:49.803268 16219 rpcz_store.cc:276] 0709 13:21:46.970132 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:47.857666 (+887534us) service_pool.cc:224] Handling call
0709 13:21:48.611371 (+753705us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:21:48.612271 (+   900us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:49.783651 (+1171380us) spinlock_profiling.cc:243] Waited 716 ms on lock 0x7b54002c8150. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5e8ad 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:21:49.783851 (+   200us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1647536000}
W20260709 13:21:49.753880 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 96485.9us
I20260709 13:21:49.804140 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fffab26427e24d398e2695aad40ca134" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:21:49.664853 16296 raft_consensus.cc:1240] T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 17. Current term is 18. Ops: []
I20260709 13:21:49.779814 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2794af5b99984efebdee564035841cb0" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:21:49.604403 22152 leader_election.cc:290] T 8c11074579b046d99e585c2818e18a2c P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 18 election: Requested vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:49.804483 22504 raft_consensus.cc:3055] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:21:49.617561 22481 raft_consensus.cc:493] T 1df93779209748b9860f6d893dd3de53 P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:49.665971 22517 leader_election.cc:290] T af57996a8be84d398f0f023e8d08ef77 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:49.658875 22292 consensus_queue.cc:1059] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.147s
I20260709 13:21:49.652537 22502 raft_consensus.cc:515] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:49.722195 16293 raft_consensus.cc:1240] T 57a744f5aaf944678a63469cbf514015 P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 12. Current term is 13. Ops: []
W20260709 13:21:49.691283 22348 consensus_meta.cc:287] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac: Time spent flushing consensus metadata: real 0.810s	user 0.011s	sys 0.004s
I20260709 13:21:49.740968 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434bff3082d141838068f323bdb0d1ed" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:21:49.813171 16190 leader_election.cc:304] T 19baa7bc367844c288bca8f81439c413 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:21:49.654078 22508 consensus_queue.cc:1048] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.152s
I20260709 13:21:49.759434 22506 raft_consensus.cc:493] T af57996a8be84d398f0f023e8d08ef77 P cb82cc7bd6db46d980c75237a1402860 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:21:49.690567 16148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
W20260709 13:21:49.790053 16791 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:21:49.691772 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 57528.4us
I20260709 13:21:49.831350 16265 leader_election.cc:304] T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860
I20260709 13:21:49.668735 22285 raft_consensus.cc:515] T e5ad4eb281564936a59f41e387a47156 P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:21:49.757501 16264 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:49.819392 16300 raft_consensus.cc:2393] T 2794af5b99984efebdee564035841cb0 P 90f3da57796b488c9087a2f523adf045 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 14: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
I20260709 13:21:49.749624 22524 raft_consensus.cc:2749] T 1df93779209748b9860f6d893dd3de53 P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:21:49.810285 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 180083us
I20260709 13:21:49.729301 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87ac0d1f7e8a419f9bcbc04aee0e067e" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
W20260709 13:21:49.781036 22532 log.cc:927] Time spent T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 1.091s	user 0.007s	sys 0.001s
I20260709 13:21:49.651963 22408 raft_consensus.cc:2749] T 68a6838d82ef4cac83507f906139d90f P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:21:49.625281 22554 raft_consensus.cc:2804] T c0eab25ce4c64e018a041beeda2867db P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 13:21:49.779403 22563 log.cc:927] Time spent T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 0.063s	user 0.001s	sys 0.000s
W20260709 13:21:49.626976 22492 raft_consensus.cc:1068] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac: Unable to promote non-voter cb82cc7bd6db46d980c75237a1402860: Illegal state: Leader has not yet committed an operation in its own term
I20260709 13:21:49.905088 22492 consensus_queue.cc:1059] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: INVALID_TERM, Last received: 16.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:21:49.906405 22492 raft_consensus.cc:3055] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 1ca2725d290e47d68b1a4b609c9816ac [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:21:49.825646 16296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0eab25ce4c64e018a041beeda2867db" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:21:49.666510 22404 raft_consensus.cc:2804] T 8c11074579b046d99e585c2818e18a2c P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:21:49.907546 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a96373bb0e14dce8a266b2fc6c58507" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:21:49.907707 22404 raft_consensus.cc:493] T 8c11074579b046d99e585c2818e18a2c P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Starting leader election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:21:49.908330 22404 raft_consensus.cc:3060] T 8c11074579b046d99e585c2818e18a2c P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:21:49.909085 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9944d5e4b8c4109af9955c4f08c9095" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:49.910589 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3884f864a96d428785895e5463e5728f" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:21:49.829682 16190 leader_election.cc:304] T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:21:49.833803 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a878999c0024d2abe517484180290fc" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:21:49.834061 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8158bbd17b994d2eb96f560bd0f1b88d" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:21:49.821502 22504 raft_consensus.cc:740] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:21:49.817480 22569 raft_consensus.cc:493] T 829b4ad8c5a54eababeea1181b960d4b P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:21:49.743667 22399 raft_consensus.cc:515] T 8638ee363273426db1393dd81e5c7b72 P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:49.811971 22168 raft_consensus.cc:493] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:49.921710 22168 raft_consensus.cc:3060] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:21:49.921303 22504 consensus_queue.cc:260] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:49.831112 22348 raft_consensus.cc:515] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:49.923035 22504 raft_consensus.cc:3060] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:21:49.924755 22348 leader_election.cc:290] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 19 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:49.925376 16217 raft_consensus.cc:1240] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 18. Current term is 19. Ops: []
I20260709 13:21:49.900856 22292 raft_consensus.cc:2804] T 19baa7bc367844c288bca8f81439c413 P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:21:49.824273 22481 raft_consensus.cc:515] T 1df93779209748b9860f6d893dd3de53 P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:49.859970 16148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68a6838d82ef4cac83507f906139d90f" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:21:49.879527 16145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0eab25ce4c64e018a041beeda2867db" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
W20260709 13:21:49.879565 16264 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:49.881613 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a878999c0024d2abe517484180290fc" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:49.883023 16143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26354361e9b044cea1c2b86c4882fe92" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:21:49.916435 22285 leader_election.cc:290] T e5ad4eb281564936a59f41e387a47156 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
W20260709 13:21:49.885398 16266 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:49.929169 22548 consensus_queue.cc:1059] T 8cc959fb7f704fba9009c0595467d074 P 647a343a542d4a139fe106c19095129b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: INVALID_TERM, Last received: 15.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.025s
I20260709 13:21:49.888358 16223 raft_consensus.cc:2468] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 19.
I20260709 13:21:49.927245 22399 leader_election.cc:290] T 8638ee363273426db1393dd81e5c7b72 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 18 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:49.932674 22564 consensus_queue.cc:1059] T 8a878999c0024d2abe517484180290fc P 647a343a542d4a139fe106c19095129b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.024s
I20260709 13:21:49.957966 22564 raft_consensus.cc:3055] T 8a878999c0024d2abe517484180290fc P 647a343a542d4a139fe106c19095129b [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:21:49.958531 22564 raft_consensus.cc:740] T 8a878999c0024d2abe517484180290fc P 647a343a542d4a139fe106c19095129b [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:21:49.854928 16146 raft_consensus.cc:2468] T fffab26427e24d398e2695aad40ca134 P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 17.
I20260709 13:21:49.935537 22031 raft_consensus.cc:3055] T ada5a004ba7c46819051e3ccec71ffdb P cb82cc7bd6db46d980c75237a1402860 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:21:49.961592 22031 raft_consensus.cc:740] T ada5a004ba7c46819051e3ccec71ffdb P cb82cc7bd6db46d980c75237a1402860 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:21:49.962430 22031 consensus_queue.cc:260] T ada5a004ba7c46819051e3ccec71ffdb P cb82cc7bd6db46d980c75237a1402860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:49.965147 22522 raft_consensus.cc:3055] T 434bff3082d141838068f323bdb0d1ed P 90f3da57796b488c9087a2f523adf045 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:21:49.965572 22522 raft_consensus.cc:740] T 434bff3082d141838068f323bdb0d1ed P 90f3da57796b488c9087a2f523adf045 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:21:49.967598 16295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a959dbecc2648eb85d1cc550f5a3245" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:21:49.968765 16295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d4ab870a05347868956aa8b841b2981" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:21:49.969408 16295 raft_consensus.cc:2393] T 8d4ab870a05347868956aa8b841b2981 P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 16: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
I20260709 13:21:49.970139 16295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18ad7e3af5504571a69d10c00750a01c" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:49.970675 16295 raft_consensus.cc:2468] T 18ad7e3af5504571a69d10c00750a01c P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 15.
I20260709 13:21:49.971807 16295 raft_consensus.cc:1240] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 15. Current term is 16. Ops: [15.10-15.10]
I20260709 13:21:49.973840 16151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0eab25ce4c64e018a041beeda2867db" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:49.975903 16295 raft_consensus.cc:1240] T 7c2095a5f4ad49408dbc7f69c606120d P 90f3da57796b488c9087a2f523adf045 [term 14 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 13. Current term is 14. Ops: []
I20260709 13:21:49.903837 22408 raft_consensus.cc:2749] T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:21:49.828883 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cc959fb7f704fba9009c0595467d074" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:21:49.979995 16293 raft_consensus.cc:2468] T 8158bbd17b994d2eb96f560bd0f1b88d P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 16.
I20260709 13:21:49.951014 21704 raft_consensus.cc:493] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:49.980870 21704 raft_consensus.cc:515] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:49.834527 22439 leader_election.cc:290] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 18 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:21:49.833685 22506 raft_consensus.cc:515] T af57996a8be84d398f0f023e8d08ef77 P cb82cc7bd6db46d980c75237a1402860 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:49.626477 22246 consensus_queue.cc:237] T 8158bbd17b994d2eb96f560bd0f1b88d P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.10, Last appended by leader: 10, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:49.906847 22492 raft_consensus.cc:740] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 1ca2725d290e47d68b1a4b609c9816ac [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:21:49.919790 22518 consensus_queue.cc:1048] T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.002s
I20260709 13:21:49.919674 22569 raft_consensus.cc:515] T 829b4ad8c5a54eababeea1181b960d4b P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:49.991645 22031 raft_consensus.cc:3060] T ada5a004ba7c46819051e3ccec71ffdb P cb82cc7bd6db46d980c75237a1402860 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:21:49.966573 22522 consensus_queue.cc:260] T 434bff3082d141838068f323bdb0d1ed P 90f3da57796b488c9087a2f523adf045 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:49.992722 22522 raft_consensus.cc:3060] T 434bff3082d141838068f323bdb0d1ed P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:21:49.993228 22569 leader_election.cc:290] T 829b4ad8c5a54eababeea1181b960d4b P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:49.993039 22506 leader_election.cc:290] T af57996a8be84d398f0f023e8d08ef77 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:21:49.976408 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5718c623ebee4d2bb770f0f9ddb2f164" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:49.994801 16146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a878999c0024d2abe517484180290fc" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:21:49.936507 16294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4dc6b0541ca431ebc757beac00521f1" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:49.995625 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a9de347c711477e87c582d003f36925" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:49.995857 16294 raft_consensus.cc:2468] T e4dc6b0541ca431ebc757beac00521f1 P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 17.
I20260709 13:21:49.975019 16151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02838c86531d4f98bf58d1c22b8baf77" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:21:49.901062 16221 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57600 (request call id 1864) took 1351 ms. Trace:
W20260709 13:21:49.937290 16266 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:49.997076 21704 leader_election.cc:290] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
W20260709 13:21:49.997494 16266 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:49.939982 16224 pending_rounds.cc:85] T 1df93779209748b9860f6d893dd3de53 P 1ca2725d290e47d68b1a4b609c9816ac: Aborting all ops after (but not including) 8
I20260709 13:21:49.999840 16224 pending_rounds.cc:107] T 1df93779209748b9860f6d893dd3de53 P 1ca2725d290e47d68b1a4b609c9816ac: Aborting uncommitted NO_OP operation due to leader change: 15.9
I20260709 13:21:49.999972 16820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "829b4ad8c5a54eababeea1181b960d4b" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:49.979415 16143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a878999c0024d2abe517484180290fc" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
W20260709 13:21:49.950356 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 120727us
I20260709 13:21:49.980340 16299 raft_consensus.cc:3060] T 8cc959fb7f704fba9009c0595467d074 P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:21:49.951501 16218 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33050 (request call id 1102) took 1722 ms (client timeout 2083 ms). Trace:
I20260709 13:21:50.001541 16190 leader_election.cc:304] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 647a343a542d4a139fe106c19095129b
I20260709 13:21:49.911634 22554 raft_consensus.cc:493] T c0eab25ce4c64e018a041beeda2867db P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:21:49.950826 16148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee9ed67006144797b1de6ea8c2013a68" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 18 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:49.953523 16191 leader_election.cc:304] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: cb82cc7bd6db46d980c75237a1402860
I20260709 13:21:49.982515 16145 raft_consensus.cc:2468] T c0eab25ce4c64e018a041beeda2867db P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 17.
I20260709 13:21:49.985589 16115 leader_election.cc:304] T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860
I20260709 13:21:49.991427 22518 consensus_queue.cc:1243] T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b [LEADER]: Peer 1ca2725d290e47d68b1a4b609c9816ac log is divergent from this leader: its last log entry 16.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 13:21:49.994252 16816 raft_consensus.cc:1275] T 74ca8140f35c4ae29e8a664f6bdfc829 P cb82cc7bd6db46d980c75237a1402860 [term 18 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 13:21:50.007417 22404 raft_consensus.cc:515] T 8c11074579b046d99e585c2818e18a2c P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:49.977063 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c11074579b046d99e585c2818e18a2c" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:21:49.937016 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cc959fb7f704fba9009c0595467d074" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:21:50.008219 22554 raft_consensus.cc:3060] T c0eab25ce4c64e018a041beeda2867db P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:21:50.009403 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:50.010325 16190 leader_election.cc:304] T ecc1e363230043fe9fef390f0b979f20 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860
I20260709 13:21:50.010584 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5d59407eb545b895d8d922e0dd69fd" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:21:50.000185 16224 raft_consensus.cc:2889] T 1df93779209748b9860f6d893dd3de53 P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:21:49.939360 16789 leader_election.cc:304] T 57a744f5aaf944678a63469cbf514015 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 647a343a542d4a139fe106c19095129b
I20260709 13:21:50.012463 16149 raft_consensus.cc:3060] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:21:49.944242 22553 raft_consensus.cc:515] T 8a878999c0024d2abe517484180290fc P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:49.953331 22517 raft_consensus.cc:493] T 9e65676b21b748728ad39668cfe722ec P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:50.013443 22517 raft_consensus.cc:515] T 9e65676b21b748728ad39668cfe722ec P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:50.015317 16298 raft_consensus.cc:2393] T 8a878999c0024d2abe517484180290fc P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 15: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
I20260709 13:21:49.954568 22548 raft_consensus.cc:3055] T 8cc959fb7f704fba9009c0595467d074 P 647a343a542d4a139fe106c19095129b [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:21:50.016729 22548 raft_consensus.cc:740] T 8cc959fb7f704fba9009c0595467d074 P 647a343a542d4a139fe106c19095129b [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:21:50.019578 22502 leader_election.cc:290] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:50.020316 16148 raft_consensus.cc:2393] T ee9ed67006144797b1de6ea8c2013a68 P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 18: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
I20260709 13:21:50.020916 16297 raft_consensus.cc:2393] T 8c11074579b046d99e585c2818e18a2c P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 18: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
W20260709 13:21:50.004320 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 173030us
I20260709 13:21:49.936618 22348 raft_consensus.cc:3060] T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:21:50.021488 16148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6f07842d3aa47858df5665874c86f3a" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
W20260709 13:21:50.001713 16218 rpcz_store.cc:269] 0709 13:21:48.229565 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:48.231091 (+  1526us) service_pool.cc:224] Handling call
0709 13:21:49.951300 (+1720209us) spinlock_profiling.cc:243] Waited 445 ms on lock 0x7b5400139554. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:21:49.951430 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1022795136}
I20260709 13:21:50.005916 22492 consensus_queue.cc:260] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.12, Last appended by leader: 12, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:50.007700 16151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4dc6b0541ca431ebc757beac00521f1" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:49.877034 16142 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46224 (request call id 1020) took 1149 ms. Trace:
I20260709 13:21:50.002969 16221 rpcz_store.cc:276] 0709 13:21:48.550107 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:49.219787 (+669680us) service_pool.cc:224] Handling call
0709 13:21:49.220105 (+   318us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:21:49.220803 (+   698us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:21:49.220834 (+    31us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:21:49.419211 (+198377us) spinlock_profiling.cc:243] Waited 11 ms on lock 0x7b2c00115ac8. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69af458a11 00007f69af468372 00007f69b0b9aa01 00007f69b0b5f43a 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2
0709 13:21:49.419798 (+   587us) log.cc:844] Serialized 52 byte log entry
0709 13:21:49.574682 (+154884us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:21:49.575007 (+   325us) log.cc:844] Serialized 14 byte log entry
0709 13:21:49.831296 (+256289us) log.cc:844] Serialized 14 byte log entry
0709 13:21:49.900353 (+ 69057us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:49.900507 (+   154us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:21:49.900806 (+   299us) raft_consensus.cc:1685] finished
0709 13:21:49.900828 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:21:49.900997 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":20781,"prepare.queue_time_us":39214,"prepare.run_cpu_time_us":80,"prepare.run_wall_time_us":75,"raft.queue_time_us":99046,"raft.run_cpu_time_us":461,"raft.run_wall_time_us":34553,"spinlock_wait_cycles":25305216,"thread_start_us":460585,"threads_started":3,"wal-append.queue_time_us":306741}
I20260709 13:21:50.006194 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e9b2bf7011f4ae688a79d7bb5a447b4" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 17 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:50.022951 16148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4f65cdf00b748bb963ee6e81ef43b79" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:50.023823 16265 leader_election.cc:304] T 7c2095a5f4ad49408dbc7f69c606120d P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 647a343a542d4a139fe106c19095129b
I20260709 13:21:50.026767 16145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d16dfad0c3b540e5b4b86492ddf87208" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:50.007781 22518 raft_consensus.cc:2749] T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:21:50.026417 16150 raft_consensus.cc:2468] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 16.
W20260709 13:21:50.028716 16263 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:21:50.026036 16789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 86786.5us
I20260709 13:21:50.025857 16142 rpcz_store.cc:276] 0709 13:21:48.727498 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:48.741100 (+ 13602us) service_pool.cc:224] Handling call
0709 13:21:49.876798 (+1135698us) spinlock_profiling.cc:243] Waited 552 ms on lock 0x7b540012f7d4. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:21:49.876931 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1270085504}
I20260709 13:21:50.004904 16115 leader_election.cc:304] T 08c74490c40243a88e62d1760b5cd814 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:21:50.035216 16299 raft_consensus.cc:2468] T 8cc959fb7f704fba9009c0595467d074 P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 16.
I20260709 13:21:49.953863 22168 raft_consensus.cc:515] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:49.929595 22292 raft_consensus.cc:493] T 19baa7bc367844c288bca8f81439c413 P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Starting leader election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:21:50.037673 16820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5d59407eb545b895d8d922e0dd69fd" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:21:50.038158 16819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e9b2bf7011f4ae688a79d7bb5a447b4" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 17 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:50.041661 16294 raft_consensus.cc:1240] T d16dfad0c3b540e5b4b86492ddf87208 P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 17. Current term is 18. Ops: []
I20260709 13:21:50.049726 16143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af57996a8be84d398f0f023e8d08ef77" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:50.043263 16265 leader_election.cc:304] T ada5a004ba7c46819051e3ccec71ffdb P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: cb82cc7bd6db46d980c75237a1402860
W20260709 13:21:50.050247 16115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 45444.2us
I20260709 13:21:50.046104 16821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af57996a8be84d398f0f023e8d08ef77" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 19 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:50.045218 16823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8638ee363273426db1393dd81e5c7b72" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:21:49.935957 22481 leader_election.cc:290] T 1df93779209748b9860f6d893dd3de53 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:50.045171 22504 consensus_queue.cc:1048] T 9e65676b21b748728ad39668cfe722ec P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:21:50.053454 16115 leader_election.cc:400] T 9e65676b21b748728ad39668cfe722ec P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 16 pre-election: Vote denied by peer 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389) with higher term. Message: Invalid argument: T 9e65676b21b748728ad39668cfe722ec P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b for earlier term 16. Current term is 17.
I20260709 13:21:50.017946 22548 consensus_queue.cc:260] T 8cc959fb7f704fba9009c0595467d074 P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:21:50.062496 16189 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:50.014894 22553 leader_election.cc:290] T 8a878999c0024d2abe517484180290fc P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 15 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:21:50.055418 16145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ad5dbfa99ee4b8d9e28be92b3f36db5" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 17 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:50.055709 16151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4dc6b0541ca431ebc757beac00521f1" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 17 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:50.065682 22285 raft_consensus.cc:493] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:50.066257 22285 raft_consensus.cc:515] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:50.072532 16142 raft_consensus.cc:1240] T 5a96373bb0e14dce8a266b2fc6c58507 P 647a343a542d4a139fe106c19095129b [term 14 LEADER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 13. Current term is 14. Ops: []
I20260709 13:21:50.056371 22292 raft_consensus.cc:3060] T 19baa7bc367844c288bca8f81439c413 P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:21:50.073788 16294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a9de347c711477e87c582d003f36925" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:50.074455 16294 raft_consensus.cc:2468] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 20.
I20260709 13:21:50.075255 22569 raft_consensus.cc:2804] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:21:50.075346 16294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:21:50.076120 16294 raft_consensus.cc:3060] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:21:50.076957 16150 raft_consensus.cc:1275] T 9e65676b21b748728ad39668cfe722ec P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 13:21:50.077077 22517 leader_election.cc:290] T 9e65676b21b748728ad39668cfe722ec P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:50.061615 22576 raft_consensus.cc:2804] T 7c2095a5f4ad49408dbc7f69c606120d P 90f3da57796b488c9087a2f523adf045 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:21:50.068900 22285 leader_election.cc:290] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:50.075922 22496 raft_consensus.cc:3055] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:21:50.060269 16825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a378d7f796f94615b3b347119a9880fd" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
W20260709 13:21:50.060837 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 107414us
I20260709 13:21:50.060806 16149 raft_consensus.cc:2468] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 20.
I20260709 13:21:50.061319 16820 raft_consensus.cc:2368] T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b for earlier term 19. Current term is 20.
W20260709 13:21:50.063059 16189 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:50.043062 16789 leader_election.cc:304] T b87147731a68400e8cce9b879daa7c5d P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:21:50.064754 16143 raft_consensus.cc:1240] T 434bff3082d141838068f323bdb0d1ed P 647a343a542d4a139fe106c19095129b [term 16 LEADER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 15. Current term is 16. Ops: []
W20260709 13:21:50.090154 16192 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 49793.1us
W20260709 13:21:50.056797 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 46546.6us
I20260709 13:21:50.097249 21704 raft_consensus.cc:3060] T 08c74490c40243a88e62d1760b5cd814 P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:21:50.134404 16294 raft_consensus.cc:2468] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 20.
W20260709 13:21:50.139130 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 57782.2us
I20260709 13:21:50.139868 16149 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46224 (request call id 1029) took 1002 ms. Trace:
I20260709 13:21:50.139933 16116 leader_election.cc:304] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
I20260709 13:21:50.046327 22478 raft_consensus.cc:2804] T 57a744f5aaf944678a63469cbf514015 P cb82cc7bd6db46d980c75237a1402860 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:21:50.100817 16295 raft_consensus.cc:1240] T 8a878999c0024d2abe517484180290fc P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 14. Current term is 15. Ops: []
W20260709 13:21:50.100247 16266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 51456.2us
I20260709 13:21:50.154062 16191 leader_election.cc:304] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
W20260709 13:21:50.138809 16189 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:21:50.154096 16192 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:21:50.155021 16789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 112026us
I20260709 13:21:50.159332 16293 raft_consensus.cc:1240] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 19. Current term is 20. Ops: [19.11-19.11]
I20260709 13:21:50.160125 16221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fffab26427e24d398e2695aad40ca134" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
W20260709 13:21:50.160347 16115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 43783us
I20260709 13:21:50.160908 16221 raft_consensus.cc:2393] T fffab26427e24d398e2695aad40ca134 P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 17: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
I20260709 13:21:50.140426 16149 rpcz_store.cc:276] 0709 13:21:49.137864 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:50.010336 (+872472us) service_pool.cc:224] Handling call
0709 13:21:50.139808 (+129472us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:21:50.155331 16190 leader_election.cc:304] T 84f2da98864b4425bf7c0055951fdd4b P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:21:50.158727 16144 raft_consensus.cc:1240] T 1df93779209748b9860f6d893dd3de53 P 647a343a542d4a139fe106c19095129b [term 16 LEADER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 15. Current term is 16. Ops: [15.9-15.9]
I20260709 13:21:50.161464 16223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cc959fb7f704fba9009c0595467d074" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:50.161998 16144 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40844 (request call id 2828) took 1007 ms. Trace:
I20260709 13:21:50.162128 16190 leader_election.cc:304] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860
I20260709 13:21:50.162189 16223 raft_consensus.cc:2393] T 8cc959fb7f704fba9009c0595467d074 P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 16: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
I20260709 13:21:50.162354 16144 rpcz_store.cc:276] 0709 13:21:49.154281 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:50.034673 (+880392us) service_pool.cc:224] Handling call
0709 13:21:50.062027 (+ 27354us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:21:50.161764 (+ 99737us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:50.161949 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:21:50.173880 16218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c74490c40243a88e62d1760b5cd814" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:21:50.174429 16818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c11074579b046d99e585c2818e18a2c" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:21:50.193907 16115 leader_election.cc:304] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:21:50.098531 22439 raft_consensus.cc:2804] T ada5a004ba7c46819051e3ccec71ffdb P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Leader election won for term 18
W20260709 13:21:50.194759 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 54903us
I20260709 13:21:50.061578 22574 raft_consensus.cc:2804] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:21:50.167785 16292 raft_consensus.cc:1240] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045 [term 18 LEADER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 17. Current term is 18. Ops: []
I20260709 13:21:50.066272 22404 leader_election.cc:290] T 8c11074579b046d99e585c2818e18a2c P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 18 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:50.098408 22152 raft_consensus.cc:2749] T ecc1e363230043fe9fef390f0b979f20 P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:21:50.201521 16221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f5f89186204242bf377f7c42b2e2da" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:21:50.075945 22569 raft_consensus.cc:493] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Starting leader election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:21:50.206269 16824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66ae176961b74813a58bc15c9c4c6f2a" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:50.206374 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f82224f4564fea93572e8206bad1fe" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:21:50.215452 16789 leader_election.cc:304] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:21:50.208400 16218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a878999c0024d2abe517484180290fc" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:21:50.220250 16218 raft_consensus.cc:2393] T 8a878999c0024d2abe517484180290fc P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 15: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
I20260709 13:21:50.097816 22496 raft_consensus.cc:740] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:21:50.223738 16293 raft_consensus.cc:1275] T 68a6838d82ef4cac83507f906139d90f P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
I20260709 13:21:50.223711 21704 raft_consensus.cc:2749] T 08c74490c40243a88e62d1760b5cd814 P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:21:50.224669 16115 leader_election.cc:304] T 5a959dbecc2648eb85d1cc550f5a3245 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:21:50.063526 22506 consensus_queue.cc:1059] T 2794af5b99984efebdee564035841cb0 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: INVALID_TERM, Last received: 13.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.149s
I20260709 13:21:50.138790 22576 raft_consensus.cc:697] T 7c2095a5f4ad49408dbc7f69c606120d P 90f3da57796b488c9087a2f523adf045 [term 14 LEADER]: Becoming Leader. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:21:50.221869 16294 raft_consensus.cc:1275] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 13:21:50.222821 16220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5ad4eb281564936a59f41e387a47156" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:21:50.228077 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a378d7f796f94615b3b347119a9880fd" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:50.228468 16220 raft_consensus.cc:3060] T e5ad4eb281564936a59f41e387a47156 P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:21:50.228762 16217 raft_consensus.cc:2393] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 19: Already voted for candidate cb82cc7bd6db46d980c75237a1402860 in this term.
I20260709 13:21:50.221700 16219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c61a39fd4f84424b129c1a6f9ba404d" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:21:50.226811 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d4ab870a05347868956aa8b841b2981" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:21:50.229676 16265 leader_election.cc:304] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:21:50.227198 16218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8638ee363273426db1393dd81e5c7b72" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:21:50.097062 22106 consensus_queue.cc:1059] T 5a96373bb0e14dce8a266b2fc6c58507 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: INVALID_TERM, Last received: 13.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.110s
I20260709 13:21:50.173484 21623 consensus_queue.cc:1059] T 26354361e9b044cea1c2b86c4882fe92 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: INVALID_TERM, Last received: 13.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.062s
I20260709 13:21:50.215121 16824 raft_consensus.cc:2468] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 22.
I20260709 13:21:50.165200 22372 raft_consensus.cc:2804] T b87147731a68400e8cce9b879daa7c5d P cb82cc7bd6db46d980c75237a1402860 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:21:50.061120 22408 consensus_queue.cc:1059] T 23f82224f4564fea93572e8206bad1fe P 90f3da57796b488c9087a2f523adf045 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.115s
I20260709 13:21:50.199635 22399 raft_consensus.cc:493] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:50.200462 22480 raft_consensus.cc:2749] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:21:50.241349 21704 raft_consensus.cc:2749] T 5a959dbecc2648eb85d1cc550f5a3245 P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:21:50.207852 22548 raft_consensus.cc:3060] T 8cc959fb7f704fba9009c0595467d074 P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:21:50.200122 22481 raft_consensus.cc:2804] T 84f2da98864b4425bf7c0055951fdd4b P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:21:50.238754 16265 leader_election.cc:304] T d16dfad0c3b540e5b4b86492ddf87208 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:21:50.240242 16300 raft_consensus.cc:1240] T e5ad4eb281564936a59f41e387a47156 P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 16. Current term is 17. Ops: []
I20260709 13:21:50.246742 16294 raft_consensus.cc:1240] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 17. Current term is 18. Ops: []
I20260709 13:21:50.234763 16148 raft_consensus.cc:1240] T 08c74490c40243a88e62d1760b5cd814 P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 15. Current term is 16. Ops: []
I20260709 13:21:50.200911 22478 raft_consensus.cc:697] T 57a744f5aaf944678a63469cbf514015 P cb82cc7bd6db46d980c75237a1402860 [term 13 LEADER]: Becoming Leader. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:21:50.243520 16223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c11074579b046d99e585c2818e18a2c" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:21:50.256228 16223 raft_consensus.cc:2393] T 8c11074579b046d99e585c2818e18a2c P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 18: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
I20260709 13:21:50.217046 22517 raft_consensus.cc:2764] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Leader election decision vote started in defunct term 19: won
I20260709 13:21:50.209842 22574 raft_consensus.cc:493] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Starting leader election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:21:50.158713 22474 consensus_queue.cc:1059] T 7c2095a5f4ad49408dbc7f69c606120d P 647a343a542d4a139fe106c19095129b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: INVALID_TERM, Last received: 13.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.075s
I20260709 13:21:50.199905 22356 consensus_queue.cc:1059] T 9c61a39fd4f84424b129c1a6f9ba404d P 647a343a542d4a139fe106c19095129b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: INVALID_TERM, Last received: 17.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.064s
I20260709 13:21:50.224400 22569 raft_consensus.cc:3060] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:21:50.174568 22502 consensus_queue.cc:1059] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.076s
I20260709 13:21:50.239624 22573 raft_consensus.cc:2749] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:21:50.229115 22492 raft_consensus.cc:3060] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:21:50.221700 22439 raft_consensus.cc:697] T ada5a004ba7c46819051e3ccec71ffdb P 90f3da57796b488c9087a2f523adf045 [term 18 LEADER]: Becoming Leader. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:21:50.161260 22568 consensus_queue.cc:1048] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.102s
I20260709 13:21:50.232821 22197 raft_consensus.cc:2749] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:21:50.248492 16293 raft_consensus.cc:1275] T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 13:21:50.049747 22564 consensus_queue.cc:260] T 8a878999c0024d2abe517484180290fc P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:50.294339 22564 raft_consensus.cc:3060] T 8a878999c0024d2abe517484180290fc P 647a343a542d4a139fe106c19095129b [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:21:50.295819 16294 raft_consensus.cc:1275] T 18ad7e3af5504571a69d10c00750a01c P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 13:21:50.257584 22494 consensus_queue.cc:1059] T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: INVALID_TERM, Last received: 18.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.007s
I20260709 13:21:50.243434 22524 raft_consensus.cc:3060] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:21:50.274953 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 226746us
I20260709 13:21:50.275498 16226 raft_consensus.cc:1240] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 16. Current term is 17. Ops: []
I20260709 13:21:50.270342 16265 leader_election.cc:304] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:21:50.207617 22461 consensus_queue.cc:1059] T 57a744f5aaf944678a63469cbf514015 P 647a343a542d4a139fe106c19095129b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.051s
I20260709 13:21:50.243826 22246 raft_consensus.cc:493] T 746aabf856ab401c9ef5e859371e7b67 P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:21:50.254153 22404 raft_consensus.cc:3055] T 23f82224f4564fea93572e8206bad1fe P 90f3da57796b488c9087a2f523adf045 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:21:50.278954 16192 leader_election.cc:304] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:21:50.261823 22574 raft_consensus.cc:3060] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:21:50.233731 22505 consensus_queue.cc:1048] T 3a7ca61592e848ea9b0fb81a2ce33cdc P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.035s
W20260709 13:21:50.284739 16115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 60127.2us
I20260709 13:21:50.333431 16115 leader_election.cc:304] T 8d4ab870a05347868956aa8b841b2981 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:21:50.278069 22502 consensus_queue.cc:1243] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Peer 90f3da57796b488c9087a2f523adf045 log is divergent from this leader: its last log entry 14.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 13:21:50.254206 22152 consensus_queue.cc:1059] T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.039s
W20260709 13:21:50.293535 16114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:21:50.297432 16789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 82045.4us
I20260709 13:21:50.254343 22168 leader_election.cc:290] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 23 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:50.299945 16292 raft_consensus.cc:1240] T 8cc959fb7f704fba9009c0595467d074 P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 15. Current term is 16. Ops: []
I20260709 13:21:50.341252 16293 raft_consensus.cc:1240] T 02838c86531d4f98bf58d1c22b8baf77 P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 17. Current term is 18. Ops: []
I20260709 13:21:50.343037 16217 raft_consensus.cc:1275] T b6f07842d3aa47858df5665874c86f3a P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 13:21:50.299794 16220 raft_consensus.cc:2468] T e5ad4eb281564936a59f41e387a47156 P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 17.
I20260709 13:21:50.302604 16265 leader_election.cc:304] T 18ad7e3af5504571a69d10c00750a01c P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
W20260709 13:21:50.302609 16117 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 62515.9us
I20260709 13:21:50.277808 22500 consensus_queue.cc:1059] T 2794af5b99984efebdee564035841cb0 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: INVALID_TERM, Last received: 13.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.038s
I20260709 13:21:50.254477 22399 raft_consensus.cc:515] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:50.279587 22522 raft_consensus.cc:2749] T d16dfad0c3b540e5b4b86492ddf87208 P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
W20260709 13:21:50.311214 16264 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:50.319017 16148 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40844 (request call id 2808) took 1215 ms. Trace:
I20260709 13:21:50.351936 16143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c74490c40243a88e62d1760b5cd814" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:21:50.262382 22481 raft_consensus.cc:697] T 84f2da98864b4425bf7c0055951fdd4b P 1ca2725d290e47d68b1a4b609c9816ac [term 14 LEADER]: Becoming Leader. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:21:50.365347 22586 consensus_queue.cc:1048] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:21:50.366221 22461 raft_consensus.cc:3055] T 57a744f5aaf944678a63469cbf514015 P 647a343a542d4a139fe106c19095129b [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:21:50.365880 22481 consensus_queue.cc:237] T 84f2da98864b4425bf7c0055951fdd4b P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:50.366693 22461 raft_consensus.cc:740] T 57a744f5aaf944678a63469cbf514015 P 647a343a542d4a139fe106c19095129b [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:21:50.309741 22408 raft_consensus.cc:2749] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
W20260709 13:21:50.368103 22579 log.cc:927] Time spent T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 0.125s	user 0.000s	sys 0.000s
I20260709 13:21:50.367673 22461 consensus_queue.cc:260] T 57a744f5aaf944678a63469cbf514015 P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:50.369112 22461 raft_consensus.cc:3060] T 57a744f5aaf944678a63469cbf514015 P 647a343a542d4a139fe106c19095129b [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:21:50.353870 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 51373.4us
I20260709 13:21:50.309039 22404 raft_consensus.cc:740] T 23f82224f4564fea93572e8206bad1fe P 90f3da57796b488c9087a2f523adf045 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:21:50.360739 16226 raft_consensus.cc:1240] T 8a878999c0024d2abe517484180290fc P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 14. Current term is 15. Ops: []
I20260709 13:21:50.316116 22196 raft_consensus.cc:3055] T 7c2095a5f4ad49408dbc7f69c606120d P 647a343a542d4a139fe106c19095129b [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:21:50.361819 16218 raft_consensus.cc:1275] T 18ad7e3af5504571a69d10c00750a01c P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 13:21:50.363763 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2794af5b99984efebdee564035841cb0" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:21:50.405896 16142 raft_consensus.cc:1240] T 8158bbd17b994d2eb96f560bd0f1b88d P 647a343a542d4a139fe106c19095129b [term 16 LEADER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 15. Current term is 16. Ops: []
I20260709 13:21:50.406443 16149 raft_consensus.cc:2393] T 2794af5b99984efebdee564035841cb0 P 647a343a542d4a139fe106c19095129b [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 14: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
I20260709 13:21:50.364182 22520 raft_consensus.cc:3055] T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:21:50.411613 22520 raft_consensus.cc:740] T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:21:50.412674 22520 consensus_queue.cc:260] T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:50.413815 22520 raft_consensus.cc:3060] T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:21:50.364624 16151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cc959fb7f704fba9009c0595467d074" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:50.370111 16265 leader_election.cc:304] T 26354361e9b044cea1c2b86c4882fe92 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
I20260709 13:21:50.394301 22457 raft_consensus.cc:2749] T 8d4ab870a05347868956aa8b841b2981 P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
W20260709 13:21:50.416607 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 46580us
I20260709 13:21:50.417450 16265 leader_election.cc:304] T fffab26427e24d398e2695aad40ca134 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:21:50.418298 22408 raft_consensus.cc:2749] T 26354361e9b044cea1c2b86c4882fe92 P 90f3da57796b488c9087a2f523adf045 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:21:50.418457 16265 leader_election.cc:304] T 02838c86531d4f98bf58d1c22b8baf77 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:21:50.370646 16217 pending_rounds.cc:85] T e4dc6b0541ca431ebc757beac00521f1 P 1ca2725d290e47d68b1a4b609c9816ac: Aborting all ops after (but not including) 7
I20260709 13:21:50.419134 22408 raft_consensus.cc:2804] T fffab26427e24d398e2695aad40ca134 P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:21:50.394867 22478 consensus_queue.cc:237] T 57a744f5aaf944678a63469cbf514015 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:50.419931 22408 raft_consensus.cc:493] T fffab26427e24d398e2695aad40ca134 P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:21:50.420332 22408 raft_consensus.cc:3060] T fffab26427e24d398e2695aad40ca134 P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:21:50.371311 16295 raft_consensus.cc:3055] T 5a96373bb0e14dce8a266b2fc6c58507 P 90f3da57796b488c9087a2f523adf045 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:21:50.421316 22522 raft_consensus.cc:2749] T 02838c86531d4f98bf58d1c22b8baf77 P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:21:50.421602 16295 raft_consensus.cc:740] T 5a96373bb0e14dce8a266b2fc6c58507 P 90f3da57796b488c9087a2f523adf045 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:21:50.353757 16223 raft_consensus.cc:1240] T e5ad4eb281564936a59f41e387a47156 P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 16. Current term is 17. Ops: []
I20260709 13:21:50.359409 16148 rpcz_store.cc:276] 0709 13:21:49.103424 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:50.030504 (+927080us) service_pool.cc:224] Handling call
0709 13:21:50.064401 (+ 33897us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:21:50.257006 (+192605us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:50.318713 (+ 61707us) spinlock_profiling.cc:243] Waited 170 ms on lock 0x7b540006e0d4. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:21:50.318934 (+   221us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":391360384}
I20260709 13:21:50.394722 16144 raft_consensus.cc:1240] T 23f82224f4564fea93572e8206bad1fe P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 14. Current term is 15. Ops: []
I20260709 13:21:50.336565 22568 raft_consensus.cc:2804] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:21:50.370813 16145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a9de347c711477e87c582d003f36925" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 21 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:50.404378 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d4ab870a05347868956aa8b841b2981" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:21:50.411060 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c61a39fd4f84424b129c1a6f9ba404d" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
W20260709 13:21:50.389459 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 162251us
I20260709 13:21:50.419430 16217 pending_rounds.cc:107] T e4dc6b0541ca431ebc757beac00521f1 P 1ca2725d290e47d68b1a4b609c9816ac: Aborting uncommitted NO_OP operation due to leader change: 16.8
I20260709 13:21:50.424597 16217 raft_consensus.cc:2889] T e4dc6b0541ca431ebc757beac00521f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:21:50.422998 16293 raft_consensus.cc:1240] T 8638ee363273426db1393dd81e5c7b72 P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 17. Current term is 18. Ops: []
W20260709 13:21:50.426532 16115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 55958.6us
I20260709 13:21:50.426721 16145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f5f89186204242bf377f7c42b2e2da" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:21:50.426999 16144 raft_consensus.cc:1240] T 3884f864a96d428785895e5463e5728f P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 12. Current term is 13. Ops: []
W20260709 13:21:50.428131 16114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:50.425485 16265 leader_election.cc:304] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: cb82cc7bd6db46d980c75237a1402860
I20260709 13:21:50.430045 16150 raft_consensus.cc:2393] T 8d4ab870a05347868956aa8b841b2981 P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 16: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
I20260709 13:21:50.432623 16115 leader_election.cc:304] T 23f82224f4564fea93572e8206bad1fe P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860
I20260709 13:21:50.301081 22496 consensus_queue.cc:260] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:50.372920 22482 raft_consensus.cc:2749] T 18ad7e3af5504571a69d10c00750a01c P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:21:50.438043 16117 leader_election.cc:304] T 3884f864a96d428785895e5463e5728f P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860; no voters: 90f3da57796b488c9087a2f523adf045
I20260709 13:21:50.438717 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a878999c0024d2abe517484180290fc" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:21:50.332365 22246 raft_consensus.cc:515] T 746aabf856ab401c9ef5e859371e7b67 P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:50.394407 22591 raft_consensus.cc:493] T 434bff3082d141838068f323bdb0d1ed P cb82cc7bd6db46d980c75237a1402860 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:21:50.439802 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 285834us
I20260709 13:21:50.442412 16142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31969392ccdb43b491911a319e8bc7e1" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
W20260709 13:21:50.444730 16116 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:50.448716 16142 raft_consensus.cc:3060] T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:21:50.440640 16265 leader_election.cc:304] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:21:50.447613 16148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5ad4eb281564936a59f41e387a47156" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:21:50.449792 16293 raft_consensus.cc:3060] T a378d7f796f94615b3b347119a9880fd P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:21:50.451864 16265 leader_election.cc:304] T e4dc6b0541ca431ebc757beac00521f1 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:21:50.425740 22196 raft_consensus.cc:740] T 7c2095a5f4ad49408dbc7f69c606120d P 647a343a542d4a139fe106c19095129b [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:21:50.427464 22568 raft_consensus.cc:493] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:50.399333 22197 consensus_queue.cc:1059] T 7c2095a5f4ad49408dbc7f69c606120d P 647a343a542d4a139fe106c19095129b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: INVALID_TERM, Last received: 13.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.046s
W20260709 13:21:50.454699 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 107069us
I20260709 13:21:50.467296 16790 leader_election.cc:304] T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:21:50.439770 22356 raft_consensus.cc:493] T 26354361e9b044cea1c2b86c4882fe92 P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:21:50.441147 22372 raft_consensus.cc:697] T b87147731a68400e8cce9b879daa7c5d P cb82cc7bd6db46d980c75237a1402860 [term 18 LEADER]: Becoming Leader. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:21:50.451557 22474 raft_consensus.cc:2804] T 3884f864a96d428785895e5463e5728f P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:21:50.480849 16819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c11074579b046d99e585c2818e18a2c" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:21:50.478228 16191 leader_election.cc:304] T 66f75124215f4ca6bef831e174a792d8 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
I20260709 13:21:50.490217 16293 raft_consensus.cc:1275] T a378d7f796f94615b3b347119a9880fd P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 13:21:50.456341 22106 raft_consensus.cc:2749] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:21:50.437872 22236 consensus_queue.cc:1059] T d16dfad0c3b540e5b4b86492ddf87208 P 647a343a542d4a139fe106c19095129b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: INVALID_TERM, Last received: 17.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.031s
I20260709 13:21:50.501803 22348 raft_consensus.cc:2749] T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:21:50.484077 22356 raft_consensus.cc:515] T 26354361e9b044cea1c2b86c4882fe92 P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:50.505333 22356 leader_election.cc:290] T 26354361e9b044cea1c2b86c4882fe92 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:50.466111 22568 raft_consensus.cc:3060] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:21:50.507794 22372 consensus_queue.cc:237] T b87147731a68400e8cce9b879daa7c5d P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:50.339315 22576 consensus_queue.cc:237] T 7c2095a5f4ad49408dbc7f69c606120d P 90f3da57796b488c9087a2f523adf045 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 13.6, Last appended by leader: 6, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:50.511472 16824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e65676b21b748728ad39668cfe722ec" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
W20260709 13:21:50.512673 22581 log.cc:927] Time spent T 8158bbd17b994d2eb96f560bd0f1b88d P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 0.118s	user 0.000s	sys 0.000s
I20260709 13:21:50.512987 22106 consensus_queue.cc:1059] T 5a96373bb0e14dce8a266b2fc6c58507 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: INVALID_TERM, Last received: 13.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:21:50.513863 22106 consensus_queue.cc:1243] T 5a96373bb0e14dce8a266b2fc6c58507 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Peer 647a343a542d4a139fe106c19095129b log is divergent from this leader: its last log entry 14.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 13:21:50.495332 22480 raft_consensus.cc:2749] T 66f75124215f4ca6bef831e174a792d8 P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:21:50.495114 22545 raft_consensus.cc:2804] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:21:50.471589 22496 raft_consensus.cc:3060] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:21:50.518023 16788 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:50.491367 16191 leader_election.cc:304] T 5a96373bb0e14dce8a266b2fc6c58507 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:21:50.497913 22585 raft_consensus.cc:2749] T e4dc6b0541ca431ebc757beac00521f1 P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:21:50.499913 22589 raft_consensus.cc:2749] T 23f82224f4564fea93572e8206bad1fe P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:21:50.503342 22595 consensus_queue.cc:1048] T 68a6838d82ef4cac83507f906139d90f P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.012s
I20260709 13:21:50.457957 22591 raft_consensus.cc:515] T 434bff3082d141838068f323bdb0d1ed P cb82cc7bd6db46d980c75237a1402860 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:21:50.495673 16791 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:50.493264 16293 raft_consensus.cc:1240] T 31969392ccdb43b491911a319e8bc7e1 P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 15. Current term is 16. Ops: []
I20260709 13:21:50.549849 22591 leader_election.cc:290] T 434bff3082d141838068f323bdb0d1ed P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
W20260709 13:21:50.552814 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 61624.3us
I20260709 13:21:50.507715 16818 raft_consensus.cc:1240] T 3884f864a96d428785895e5463e5728f P cb82cc7bd6db46d980c75237a1402860 [term 13 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 12. Current term is 13. Ops: []
I20260709 13:21:50.458455 22593 raft_consensus.cc:2804] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:21:50.424446 22439 consensus_queue.cc:237] T ada5a004ba7c46819051e3ccec71ffdb P 90f3da57796b488c9087a2f523adf045 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:50.447463 21623 consensus_queue.cc:1048] T 9e65676b21b748728ad39668cfe722ec P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.038s
I20260709 13:21:50.456111 22457 consensus_queue.cc:1059] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: INVALID_TERM, Last received: 17.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.040s
W20260709 13:21:50.515743 22582 log.cc:927] Time spent T af57996a8be84d398f0f023e8d08ef77 P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 0.220s	user 0.000s	sys 0.001s
I20260709 13:21:50.543375 22246 leader_election.cc:290] T 746aabf856ab401c9ef5e859371e7b67 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:50.516711 22197 consensus_queue.cc:1048] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.030s
I20260709 13:21:50.516260 22031 raft_consensus.cc:2804] T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:21:50.578601 21704 raft_consensus.cc:3055] T 9c61a39fd4f84424b129c1a6f9ba404d P 647a343a542d4a139fe106c19095129b [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:21:50.661290 21704 raft_consensus.cc:740] T 9c61a39fd4f84424b129c1a6f9ba404d P 647a343a542d4a139fe106c19095129b [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:21:50.592701 16191 leader_election.cc:304] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: cb82cc7bd6db46d980c75237a1402860
W20260709 13:21:50.596889 16266 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:50.516785 22474 raft_consensus.cc:697] T 3884f864a96d428785895e5463e5728f P 647a343a542d4a139fe106c19095129b [term 13 LEADER]: Becoming Leader. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:21:50.605707 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434bff3082d141838068f323bdb0d1ed" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
W20260709 13:21:50.577988 22292 consensus_meta.cc:287] T 19baa7bc367844c288bca8f81439c413 P 1ca2725d290e47d68b1a4b609c9816ac: Time spent flushing consensus metadata: real 0.505s	user 0.009s	sys 0.000s
W20260709 13:21:50.691073 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 133354us
I20260709 13:21:50.546265 22568 raft_consensus.cc:515] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:50.595008 22348 raft_consensus.cc:2749] T 5a96373bb0e14dce8a266b2fc6c58507 P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:21:50.705812 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 152066us
I20260709 13:21:50.702507 22480 raft_consensus.cc:2804] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:21:50.769232 22480 raft_consensus.cc:493] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: Starting leader election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:21:50.769878 22480 raft_consensus.cc:3060] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:21:50.708631 22554 consensus_meta.cc:287] T c0eab25ce4c64e018a041beeda2867db P 90f3da57796b488c9087a2f523adf045: Time spent flushing consensus metadata: real 0.700s	user 0.011s	sys 0.000s
I20260709 13:21:50.708190 22474 consensus_queue.cc:237] T 3884f864a96d428785895e5463e5728f P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 12.12, Last appended by leader: 12, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:50.709115 22031 raft_consensus.cc:697] T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860 [term 20 LEADER]: Becoming Leader. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:21:50.710036 22524 raft_consensus.cc:2749] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:21:50.703492 21704 consensus_queue.cc:260] T 9c61a39fd4f84424b129c1a6f9ba404d P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.9, Last appended by leader: 9, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:50.578022 22593 raft_consensus.cc:493] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:50.520514 22404 consensus_queue.cc:260] T 23f82224f4564fea93572e8206bad1fe P 90f3da57796b488c9087a2f523adf045 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:50.787086 16191 leader_election.cc:304] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
W20260709 13:21:50.787755 16222 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.633s	user 0.011s	sys 0.014s
W20260709 13:21:50.790814 16115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 307594us
W20260709 13:21:50.797118 16222 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.644s	user 0.011s	sys 0.014s
I20260709 13:21:50.778505 16265 leader_election.cc:304] T 8638ee363273426db1393dd81e5c7b72 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860
W20260709 13:21:50.799425 16190 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.585s	user 0.007s	sys 0.035s
W20260709 13:21:50.801124 16190 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.586s	user 0.008s	sys 0.035s
W20260709 13:21:50.801640 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 586898us
I20260709 13:21:50.799474 16265 leader_election.cc:304] T a378d7f796f94615b3b347119a9880fd P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860
I20260709 13:21:50.808156 21704 raft_consensus.cc:3060] T 9c61a39fd4f84424b129c1a6f9ba404d P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:21:50.800113 22106 raft_consensus.cc:2749] T 8638ee363273426db1393dd81e5c7b72 P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:21:50.809127 22564 consensus_queue.cc:1059] T e5ad4eb281564936a59f41e387a47156 P 647a343a542d4a139fe106c19095129b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: INVALID_TERM, Last received: 16.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.003s
I20260709 13:21:50.809202 22595 consensus_queue.cc:1048] T 18ad7e3af5504571a69d10c00750a01c P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.003s
I20260709 13:21:50.812004 22602 consensus_queue.cc:1059] T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.019s
I20260709 13:21:50.702916 22074 consensus_queue.cc:1059] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: INVALID_TERM, Last received: 19.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.102s
I20260709 13:21:50.779799 22505 raft_consensus.cc:493] T ee9ed67006144797b1de6ea8c2013a68 P cb82cc7bd6db46d980c75237a1402860 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:50.790740 22586 raft_consensus.cc:493] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P cb82cc7bd6db46d980c75237a1402860 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:21:50.844106 16117 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 399577us
I20260709 13:21:50.844962 16117 leader_election.cc:304] T af57996a8be84d398f0f023e8d08ef77 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860
I20260709 13:21:50.715952 22292 raft_consensus.cc:515] T 19baa7bc367844c288bca8f81439c413 P 1ca2725d290e47d68b1a4b609c9816ac [term 20 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:50.828034 16191 leader_election.cc:304] T fffab26427e24d398e2695aad40ca134 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:21:50.848220 16218 raft_consensus.cc:1240] T 5a96373bb0e14dce8a266b2fc6c58507 P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 13. Current term is 14. Ops: []
I20260709 13:21:50.774142 22569 raft_consensus.cc:515] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:50.858920 16147 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40844 (request call id 2809) took 1755 ms. Trace:
I20260709 13:21:50.772230 22601 consensus_queue.cc:1059] T 3884f864a96d428785895e5463e5728f P 90f3da57796b488c9087a2f523adf045 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.141s
I20260709 13:21:50.781531 22554 raft_consensus.cc:515] T c0eab25ce4c64e018a041beeda2867db P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:50.866955 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7ca61592e848ea9b0fb81a2ce33cdc" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:50.868782 21704 raft_consensus.cc:493] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:21:50.818958 22106 raft_consensus.cc:2749] T a378d7f796f94615b3b347119a9880fd P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:21:50.887992 22106 raft_consensus.cc:3055] T 3884f864a96d428785895e5463e5728f P 90f3da57796b488c9087a2f523adf045 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:21:50.888391 22106 raft_consensus.cc:740] T 3884f864a96d428785895e5463e5728f P 90f3da57796b488c9087a2f523adf045 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
W20260709 13:21:50.888633 22594 log.cc:927] Time spent T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860: Append to log took a long time: real 0.108s	user 0.000s	sys 0.001s
W20260709 13:21:50.889364 22596 log.cc:927] Time spent T 1df93779209748b9860f6d893dd3de53 P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 0.187s	user 0.000s	sys 0.001s
I20260709 13:21:50.889169 22106 consensus_queue.cc:260] T 3884f864a96d428785895e5463e5728f P 90f3da57796b488c9087a2f523adf045 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 12.12, Last appended by leader: 12, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:50.890416 22106 raft_consensus.cc:3060] T 3884f864a96d428785895e5463e5728f P 90f3da57796b488c9087a2f523adf045 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:21:50.849401 22481 raft_consensus.cc:2749] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:21:50.859508 16147 rpcz_store.cc:276] 0709 13:21:49.103680 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:50.033453 (+929773us) service_pool.cc:224] Handling call
0709 13:21:50.033913 (+   460us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:21:50.075235 (+ 41322us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:21:50.075264 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:21:50.075713 (+   449us) log.cc:844] Serialized 26 byte log entry
0709 13:21:50.228236 (+152523us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:21:50.235934 (+  7698us) log.cc:844] Serialized 14 byte log entry
0709 13:21:50.241397 (+  5463us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:50.241465 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:21:50.858641 (+617176us) raft_consensus.cc:1685] finished
0709 13:21:50.858668 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:21:50.858845 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":21305,"prepare.queue_time_us":7721,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":41,"raft.queue_time_us":43302,"raft.run_cpu_time_us":149,"raft.run_wall_time_us":150,"thread_start_us":152182,"threads_started":1,"wal-append.queue_time_us":206282}
I20260709 13:21:50.829023 22404 raft_consensus.cc:3060] T 23f82224f4564fea93572e8206bad1fe P 90f3da57796b488c9087a2f523adf045 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:21:50.885664 22461 consensus_meta.cc:287] T 57a744f5aaf944678a63469cbf514015 P 647a343a542d4a139fe106c19095129b: Time spent flushing consensus metadata: real 0.516s	user 0.006s	sys 0.002s
W20260709 13:21:50.925694 16143 consensus_meta.cc:287] T 08c74490c40243a88e62d1760b5cd814 P 647a343a542d4a139fe106c19095129b: Time spent flushing consensus metadata: real 0.554s	user 0.008s	sys 0.003s
I20260709 13:21:50.878334 21704 raft_consensus.cc:515] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } }
I20260709 13:21:50.793763 22593 raft_consensus.cc:3060] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:21:50.897289 16296 raft_consensus.cc:2393] T c0eab25ce4c64e018a041beeda2867db P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 17: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
I20260709 13:21:50.898861 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0eab25ce4c64e018a041beeda2867db" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
W20260709 13:21:50.937678 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 169076us
I20260709 13:21:50.915390 16219 raft_consensus.cc:2468] T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 18.
I20260709 13:21:50.891127 16142 raft_consensus.cc:2468] T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 16.
W20260709 13:21:50.915731 16266 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:50.818871 22356 consensus_queue.cc:1059] T 02838c86531d4f98bf58d1c22b8baf77 P 647a343a542d4a139fe106c19095129b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: INVALID_TERM, Last received: 17.10, Next index: 11, Last known committed idx: 8, Time since last communication: 0.056s
I20260709 13:21:50.809291 22246 consensus_queue.cc:1048] T b6f07842d3aa47858df5665874c86f3a P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.066s
I20260709 13:21:50.923236 16818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7ca61592e848ea9b0fb81a2ce33cdc" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:50.805087 22589 consensus_queue.cc:1059] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: INVALID_TERM, Last received: 17.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.098s
I20260709 13:21:50.917162 22602 raft_consensus.cc:2749] T fffab26427e24d398e2695aad40ca134 P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:21:50.930097 16143 raft_consensus.cc:2468] T 08c74490c40243a88e62d1760b5cd814 P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 16.
I20260709 13:21:50.931389 21704 leader_election.cc:290] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:50.827152 16190 leader_election.cc:304] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860
I20260709 13:21:50.935025 16291 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48674 (request call id 3244) took 1072 ms. Trace:
I20260709 13:21:50.948928 16291 rpcz_store.cc:276] 0709 13:21:49.862545 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:50.099454 (+236909us) service_pool.cc:224] Handling call
0709 13:21:50.165727 (+ 66273us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:21:50.216170 (+ 50443us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:21:50.216202 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:21:50.240748 (+ 24546us) log.cc:844] Serialized 26 byte log entry
0709 13:21:50.774770 (+534022us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:21:50.774815 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:50.774889 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:21:50.934768 (+159879us) raft_consensus.cc:1685] finished
0709 13:21:50.934789 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:21:50.934930 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":12948,"raft.run_cpu_time_us":126,"raft.run_wall_time_us":127,"thread_start_us":482193,"threads_started":1,"wal-append.queue_time_us":632989}
I20260709 13:21:50.950058 22356 raft_consensus.cc:3055] T 02838c86531d4f98bf58d1c22b8baf77 P 647a343a542d4a139fe106c19095129b [term 17 LEADER]: Stepping down as leader of term 17
W20260709 13:21:50.891317 22605 log.cc:927] Time spent T c4f65cdf00b748bb963ee6e81ef43b79 P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.075s	user 0.000s	sys 0.000s
I20260709 13:21:50.950541 22356 raft_consensus.cc:740] T 02838c86531d4f98bf58d1c22b8baf77 P 647a343a542d4a139fe106c19095129b [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:21:50.951900 16299 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48674 (request call id 3239) took 1091 ms. Trace:
I20260709 13:21:50.952265 16299 rpcz_store.cc:276] 0709 13:21:49.860939 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:50.035951 (+175012us) service_pool.cc:224] Handling call
0709 13:21:50.057138 (+ 21187us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:21:50.308547 (+251409us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:21:50.308591 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:21:50.338855 (+ 30264us) log.cc:844] Serialized 26 byte log entry
0709 13:21:50.578710 (+239855us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:21:50.578741 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:50.578798 (+    57us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:21:50.951689 (+372891us) raft_consensus.cc:1685] finished
0709 13:21:50.951712 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:21:50.951842 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":61382,"raft.queue_time_us":103797,"raft.run_cpu_time_us":120,"raft.run_wall_time_us":16011,"thread_start_us":288377,"threads_started":2,"wal-append.queue_time_us":401045}
I20260709 13:21:50.896988 22554 leader_election.cc:290] T c0eab25ce4c64e018a041beeda2867db P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 17 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:21:50.939510 16149 raft_consensus.cc:1240] T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 15. Current term is 16. Ops: []
W20260709 13:21:50.902031 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 74105.7us
W20260709 13:21:50.955474 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 147738us
I20260709 13:21:50.957466 16191 leader_election.cc:304] T 68a6838d82ef4cac83507f906139d90f P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:21:50.958173 22568 leader_election.cc:290] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 23 election: Requested vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:50.958879 22568 raft_consensus.cc:2749] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:21:50.959571 22568 raft_consensus.cc:2749] T 68a6838d82ef4cac83507f906139d90f P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:21:50.940222 16226 raft_consensus.cc:2393] T c0eab25ce4c64e018a041beeda2867db P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 17: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
I20260709 13:21:50.934926 22292 leader_election.cc:290] T 19baa7bc367844c288bca8f81439c413 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 20 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:50.894560 22564 raft_consensus.cc:3055] T e5ad4eb281564936a59f41e387a47156 P 647a343a542d4a139fe106c19095129b [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:21:50.960695 22589 raft_consensus.cc:3055] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:21:50.960825 22564 raft_consensus.cc:740] T e5ad4eb281564936a59f41e387a47156 P 647a343a542d4a139fe106c19095129b [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:21:50.961274 22589 raft_consensus.cc:740] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:21:50.961742 22285 raft_consensus.cc:3055] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:21:50.961858 22564 consensus_queue.cc:260] T e5ad4eb281564936a59f41e387a47156 P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:50.956720 16190 leader_election.cc:304] T 18ad7e3af5504571a69d10c00750a01c P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:21:50.964087 22292 raft_consensus.cc:2804] T 18ad7e3af5504571a69d10c00750a01c P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:21:50.964488 22537 raft_consensus.cc:3055] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:21:50.964502 22292 raft_consensus.cc:493] T 18ad7e3af5504571a69d10c00750a01c P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:50.965030 22537 raft_consensus.cc:740] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:21:50.965126 22292 raft_consensus.cc:3060] T 18ad7e3af5504571a69d10c00750a01c P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:21:50.856189 22505 raft_consensus.cc:515] T ee9ed67006144797b1de6ea8c2013a68 P cb82cc7bd6db46d980c75237a1402860 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:50.918023 22399 leader_election.cc:290] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:50.968878 22505 leader_election.cc:290] T ee9ed67006144797b1de6ea8c2013a68 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:21:50.912431 22595 raft_consensus.cc:2749] T af57996a8be84d398f0f023e8d08ef77 P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:21:50.900243 22569 leader_election.cc:290] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 22 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:50.962284 22285 raft_consensus.cc:740] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:21:50.956759 16265 leader_election.cc:304] T 8c11074579b046d99e585c2818e18a2c P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860
I20260709 13:21:50.962239 22589 consensus_queue.cc:260] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:50.865955 22408 raft_consensus.cc:515] T fffab26427e24d398e2695aad40ca134 P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:21:50.977653 22548 consensus_meta.cc:287] T 8cc959fb7f704fba9009c0595467d074 P 647a343a542d4a139fe106c19095129b: Time spent flushing consensus metadata: real 0.730s	user 0.007s	sys 0.003s
I20260709 13:21:50.971704 16191 leader_election.cc:304] T c0eab25ce4c64e018a041beeda2867db P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
I20260709 13:21:50.852739 22586 raft_consensus.cc:515] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P cb82cc7bd6db46d980c75237a1402860 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:50.982594 16265 leader_election.cc:304] T e5ad4eb281564936a59f41e387a47156 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:21:50.989005 16265 leader_election.cc:304] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860
I20260709 13:21:50.970486 16190 leader_election.cc:304] T e4dc6b0541ca431ebc757beac00521f1 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:21:50.993170 16191 leader_election.cc:304] T ee9ed67006144797b1de6ea8c2013a68 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
I20260709 13:21:50.995853 16190 leader_election.cc:304] T 8cc959fb7f704fba9009c0595467d074 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 647a343a542d4a139fe106c19095129b
I20260709 13:21:50.996939 16151 raft_consensus.cc:2468] T 8cc959fb7f704fba9009c0595467d074 P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 16.
I20260709 13:21:50.980597 22564 raft_consensus.cc:3060] T e5ad4eb281564936a59f41e387a47156 P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:21:51.000190 16190 leader_election.cc:304] T 8c11074579b046d99e585c2818e18a2c P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860
I20260709 13:21:50.966017 22537 consensus_queue.cc:260] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:50.896346 22031 consensus_queue.cc:237] T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.15, Last appended by leader: 15, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:50.983727 22399 raft_consensus.cc:2749] T 8c11074579b046d99e585c2818e18a2c P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:21:51.004472 16191 leader_election.cc:304] T b6f07842d3aa47858df5665874c86f3a P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:21:50.978238 22152 consensus_queue.cc:1059] T 08c74490c40243a88e62d1760b5cd814 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.025s
I20260709 13:21:51.002919 16190 leader_election.cc:304] T c2f5f89186204242bf377f7c42b2e2da P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:21:51.017015 16191 leader_election.cc:304] T d16dfad0c3b540e5b4b86492ddf87208 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:21:50.998116 22569 raft_consensus.cc:2804] T c0eab25ce4c64e018a041beeda2867db P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:21:51.002697 22526 raft_consensus.cc:3060] T ee9ed67006144797b1de6ea8c2013a68 P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:21:51.014873 16146 raft_consensus.cc:2468] T 8a878999c0024d2abe517484180290fc P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 15.
I20260709 13:21:51.001340 22584 raft_consensus.cc:2804] T e5ad4eb281564936a59f41e387a47156 P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:21:51.001195 22602 raft_consensus.cc:2804] T e4dc6b0541ca431ebc757beac00521f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:21:51.002499 22568 raft_consensus.cc:2804] T 8cc959fb7f704fba9009c0595467d074 P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:21:51.004033 22482 raft_consensus.cc:2749] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:21:51.013471 22481 raft_consensus.cc:2749] T 8c11074579b046d99e585c2818e18a2c P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:21:51.037259 16819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:21:51.041769 16819 raft_consensus.cc:2393] T 19baa7bc367844c288bca8f81439c413 P cb82cc7bd6db46d980c75237a1402860 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 20: Already voted for candidate cb82cc7bd6db46d980c75237a1402860 in this term.
I20260709 13:21:50.994450 22356 consensus_queue.cc:260] T 02838c86531d4f98bf58d1c22b8baf77 P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:51.037266 22348 raft_consensus.cc:3055] T 08c74490c40243a88e62d1760b5cd814 P 1ca2725d290e47d68b1a4b609c9816ac [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:21:51.030267 22589 raft_consensus.cc:3060] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:21:51.049953 16824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62fe3346cd3c42c4a55e14af9073bca4" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:21:51.058194 16824 raft_consensus.cc:2393] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 22: Already voted for candidate cb82cc7bd6db46d980c75237a1402860 in this term.
I20260709 13:21:51.026230 22611 raft_consensus.cc:2749] T b6f07842d3aa47858df5665874c86f3a P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:21:51.055956 16823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66ae176961b74813a58bc15c9c4c6f2a" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:21:51.059417 16823 raft_consensus.cc:3060] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:21:51.013917 22615 consensus_queue.cc:1059] T 5a96373bb0e14dce8a266b2fc6c58507 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: INVALID_TERM, Last received: 13.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.047s
W20260709 13:21:51.061008 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 44060.9us
I20260709 13:21:51.062740 22152 raft_consensus.cc:3060] T d16dfad0c3b540e5b4b86492ddf87208 P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:21:51.065696 22614 log.cc:927] Time spent T 7c2095a5f4ad49408dbc7f69c606120d P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
I20260709 13:21:51.072044 22481 raft_consensus.cc:493] T 746aabf856ab401c9ef5e859371e7b67 P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:51.072847 22481 raft_consensus.cc:515] T 746aabf856ab401c9ef5e859371e7b67 P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:51.074087 16297 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48674 (request call id 3241) took 1212 ms. Trace:
I20260709 13:21:51.074519 16297 rpcz_store.cc:276] 0709 13:21:49.861624 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:50.080897 (+219273us) service_pool.cc:224] Handling call
0709 13:21:50.081283 (+   386us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:21:50.447735 (+366452us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:21:50.447775 (+    40us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:21:50.477025 (+ 29250us) log.cc:844] Serialized 52 byte log entry
0709 13:21:50.871914 (+394889us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:21:50.872045 (+   131us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:50.872549 (+   504us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:21:51.073821 (+201272us) raft_consensus.cc:1685] finished
0709 13:21:51.073843 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:21:51.074022 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":57342,"raft.queue_time_us":93542,"raft.run_cpu_time_us":130,"raft.run_wall_time_us":9608,"thread_start_us":638088,"threads_started":2,"wal-append.queue_time_us":463504}
W20260709 13:21:51.076445 16788 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:21:51.061889 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 59046.8us
I20260709 13:21:51.050854 22537 raft_consensus.cc:3060] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:21:51.081663 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5d59407eb545b895d8d922e0dd69fd" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:21:51.082361 16217 raft_consensus.cc:3060] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:21:51.054787 22502 raft_consensus.cc:3060] T c2f5f89186204242bf377f7c42b2e2da P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:21:51.035902 22602 raft_consensus.cc:493] T e4dc6b0541ca431ebc757beac00521f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Starting leader election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:21:51.042698 22584 raft_consensus.cc:697] T e5ad4eb281564936a59f41e387a47156 P 90f3da57796b488c9087a2f523adf045 [term 17 LEADER]: Becoming Leader. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:21:51.055090 22348 raft_consensus.cc:740] T 08c74490c40243a88e62d1760b5cd814 P 1ca2725d290e47d68b1a4b609c9816ac [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:21:51.099125 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4f65cdf00b748bb963ee6e81ef43b79" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:21:51.077121 22356 raft_consensus.cc:3060] T 02838c86531d4f98bf58d1c22b8baf77 P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:21:51.055956 22568 raft_consensus.cc:697] T 8cc959fb7f704fba9009c0595467d074 P 1ca2725d290e47d68b1a4b609c9816ac [term 16 LEADER]: Becoming Leader. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:21:51.099193 16218 raft_consensus.cc:1275] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 13:21:51.081620 22611 consensus_queue.cc:1059] T 08c74490c40243a88e62d1760b5cd814 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.019s
I20260709 13:21:51.101621 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
W20260709 13:21:51.122956 22574 consensus_meta.cc:287] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 1ca2725d290e47d68b1a4b609c9816ac: Time spent flushing consensus metadata: real 0.772s	user 0.005s	sys 0.002s
I20260709 13:21:51.104216 22584 consensus_queue.cc:237] T e5ad4eb281564936a59f41e387a47156 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:21:51.164585 16117 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 319693us
I20260709 13:21:51.108938 22408 leader_election.cc:290] T fffab26427e24d398e2695aad40ca134 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 17 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
W20260709 13:21:51.111688 22620 log.cc:927] Time spent T 8c11074579b046d99e585c2818e18a2c P cb82cc7bd6db46d980c75237a1402860: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 13:21:51.112118 22481 leader_election.cc:290] T 746aabf856ab401c9ef5e859371e7b67 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:51.114033 16217 raft_consensus.cc:2468] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 20.
I20260709 13:21:51.168673 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fffab26427e24d398e2695aad40ca134" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:21:51.169557 16217 raft_consensus.cc:2393] T fffab26427e24d398e2695aad40ca134 P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 17: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
I20260709 13:21:51.170554 16148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18ad7e3af5504571a69d10c00750a01c" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:51.038569 22285 consensus_queue.cc:260] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } }
W20260709 13:21:51.117764 22520 consensus_meta.cc:287] T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b: Time spent flushing consensus metadata: real 0.703s	user 0.009s	sys 0.002s
I20260709 13:21:51.174363 16294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66ae176961b74813a58bc15c9c4c6f2a" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:21:51.175009 16294 raft_consensus.cc:3060] T 66ae176961b74813a58bc15c9c4c6f2a P 90f3da57796b488c9087a2f523adf045 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:21:51.110972 22480 raft_consensus.cc:515] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:51.125651 16291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1df93779209748b9860f6d893dd3de53" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:51.127741 16226 raft_consensus.cc:2393] T 19baa7bc367844c288bca8f81439c413 P 1ca2725d290e47d68b1a4b609c9816ac [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 in current term 20: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
I20260709 13:21:51.096244 22602 raft_consensus.cc:3060] T e4dc6b0541ca431ebc757beac00521f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:21:51.129777 16790 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.14.177.130:35943 timed out after 1.827s (SENT) [suppressed 37 similar messages]
I20260709 13:21:51.132303 22152 raft_consensus.cc:2749] T d16dfad0c3b540e5b4b86492ddf87208 P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:21:51.132776 22586 leader_election.cc:290] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:51.139683 16222 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33050 (request call id 1090) took 3290 ms. Trace:
I20260709 13:21:51.141286 22526 raft_consensus.cc:2749] T ee9ed67006144797b1de6ea8c2013a68 P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:21:51.141472 16292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a378d7f796f94615b3b347119a9880fd" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:51.143553 16225 raft_consensus.cc:1240] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 18. Current term is 19. Ops: [18.15-18.15]
I20260709 13:21:51.216404 16292 raft_consensus.cc:2468] T a378d7f796f94615b3b347119a9880fd P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 19.
I20260709 13:21:51.217089 16220 raft_consensus.cc:1240] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 15. Current term is 16. Ops: []
I20260709 13:21:51.000972 15046 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:21:51.147346 16295 consensus_queue.cc:260] T 5a96373bb0e14dce8a266b2fc6c58507 P 90f3da57796b488c9087a2f523adf045 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:51.148622 22502 raft_consensus.cc:2749] T c2f5f89186204242bf377f7c42b2e2da P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:21:51.155872 16218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee9ed67006144797b1de6ea8c2013a68" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:51.155710 22568 consensus_queue.cc:237] T 8cc959fb7f704fba9009c0595467d074 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:21:51.157706 16266 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:51.128947 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5ad4eb281564936a59f41e387a47156" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:51.106297 22629 raft_consensus.cc:493] T 02838c86531d4f98bf58d1c22b8baf77 P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:21:51.223199 22543 raft_consensus.cc:3055] T 2794af5b99984efebdee564035841cb0 P cb82cc7bd6db46d980c75237a1402860 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:21:51.223429 22629 raft_consensus.cc:515] T 02838c86531d4f98bf58d1c22b8baf77 P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:51.170615 16143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5ad4eb281564936a59f41e387a47156" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:51.225688 22518 raft_consensus.cc:3055] T d16dfad0c3b540e5b4b86492ddf87208 P 647a343a542d4a139fe106c19095129b [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:21:51.226258 16143 raft_consensus.cc:2468] T e5ad4eb281564936a59f41e387a47156 P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 17.
I20260709 13:21:51.227521 16143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746aabf856ab401c9ef5e859371e7b67" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:51.227979 16218 raft_consensus.cc:2468] T ee9ed67006144797b1de6ea8c2013a68 P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 18.
I20260709 13:21:51.173069 16823 raft_consensus.cc:2468] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 23.
I20260709 13:21:51.172385 16142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4f65cdf00b748bb963ee6e81ef43b79" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:21:51.124112 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66ae176961b74813a58bc15c9c4c6f2a" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:51.179767 16147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1df93779209748b9860f6d893dd3de53" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:51.229074 22285 raft_consensus.cc:3060] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:21:51.230129 16147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:21:51.173816 16117 leader_election.cc:400] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 19 election: Vote denied by peer cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923) with higher term. Message: Invalid argument: T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b for earlier term 19. Current term is 20.
I20260709 13:21:51.201671 16145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62fe3346cd3c42c4a55e14af9073bca4" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:21:51.231673 16145 raft_consensus.cc:2368] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac for earlier term 22. Current term is 23.
I20260709 13:21:51.202474 22504 raft_consensus.cc:493] T ada5a004ba7c46819051e3ccec71ffdb P cb82cc7bd6db46d980c75237a1402860 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:51.232632 22504 raft_consensus.cc:515] T ada5a004ba7c46819051e3ccec71ffdb P cb82cc7bd6db46d980c75237a1402860 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:51.202696 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "829b4ad8c5a54eababeea1181b960d4b" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:51.226274 22518 raft_consensus.cc:740] T d16dfad0c3b540e5b4b86492ddf87208 P 647a343a542d4a139fe106c19095129b [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:21:51.236024 16291 raft_consensus.cc:1275] T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 13:21:51.236342 22518 consensus_queue.cc:260] T d16dfad0c3b540e5b4b86492ddf87208 P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:21:51.236970 16117 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 68464.8us
I20260709 13:21:51.237705 22518 raft_consensus.cc:3060] T d16dfad0c3b540e5b4b86492ddf87208 P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:21:51.237013 22590 raft_consensus.cc:493] T 23f82224f4564fea93572e8206bad1fe P cb82cc7bd6db46d980c75237a1402860 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:21:51.209580 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 79825.8us
I20260709 13:21:51.240432 22031 consensus_queue.cc:1059] T 8c11074579b046d99e585c2818e18a2c P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: INVALID_TERM, Last received: 15.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:21:51.241675 16142 raft_consensus.cc:2468] T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 19.
I20260709 13:21:51.243475 22505 raft_consensus.cc:3055] T 8c11074579b046d99e585c2818e18a2c P cb82cc7bd6db46d980c75237a1402860 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:21:51.243971 22505 raft_consensus.cc:740] T 8c11074579b046d99e585c2818e18a2c P cb82cc7bd6db46d980c75237a1402860 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:21:51.245074 22505 consensus_queue.cc:260] T 8c11074579b046d99e585c2818e18a2c P cb82cc7bd6db46d980c75237a1402860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:51.247458 22500 consensus_queue.cc:1048] T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
W20260709 13:21:51.250049 22496 consensus_meta.cc:287] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac: Time spent flushing consensus metadata: real 0.727s	user 0.009s	sys 0.002s
I20260709 13:21:51.239078 22590 raft_consensus.cc:515] T 23f82224f4564fea93572e8206bad1fe P cb82cc7bd6db46d980c75237a1402860 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:51.191223 16144 raft_consensus.cc:1240] T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 19. Current term is 20. Ops: [19.13-19.13]
I20260709 13:21:51.212901 16222 rpcz_store.cc:276] 0709 13:21:47.849189 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:49.033575 (+1184386us) service_pool.cc:224] Handling call
0709 13:21:49.097959 (+ 64384us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:21:49.114468 (+ 16509us) raft_consensus.cc:1534] Early marking committed up to index 14
0709 13:21:49.954077 (+839609us) spinlock_profiling.cc:243] Waited 3.38 ms on lock 0x7b1800418ea0. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b693ade1 00007f69b69475b1 00007f69ada54e07 00007f69b68afab9 00007f69ada7169b 00007f69ada6f93f 00007f69ada6f455 00007f69b45573c7 00007f69b0b699a7 00007f69b0b790bf 00007f69b0b79038
0709 13:21:50.024879 (+ 70802us) log.cc:844] Serialized 14 byte log entry
0709 13:21:50.823367 (+798488us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:21:50.843573 (+ 20206us) log.cc:844] Serialized 26 byte log entry
0709 13:21:50.843734 (+   161us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:21:50.843767 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:50.843835 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:21:51.128413 (+284578us) raft_consensus.cc:1685] finished
0709 13:21:51.139428 (+ 11015us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:21:51.139609 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":16084,"prepare.queue_time_us":257018,"prepare.run_cpu_time_us":32,"prepare.run_wall_time_us":33,"raft.queue_time_us":42051,"raft.run_cpu_time_us":309,"raft.run_wall_time_us":28819,"spinlock_wait_cycles":7780096,"thread_start_us":714838,"threads_started":2,"wal-append.queue_time_us":987237}
I20260709 13:21:51.223934 22543 raft_consensus.cc:740] T 2794af5b99984efebdee564035841cb0 P cb82cc7bd6db46d980c75237a1402860 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:21:51.228154 16143 raft_consensus.cc:2468] T 746aabf856ab401c9ef5e859371e7b67 P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 13.
I20260709 13:21:51.103502 22569 raft_consensus.cc:697] T c0eab25ce4c64e018a041beeda2867db P 1ca2725d290e47d68b1a4b609c9816ac [term 17 LEADER]: Becoming Leader. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:21:51.255939 16823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62fe3346cd3c42c4a55e14af9073bca4" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:21:51.188158 16294 raft_consensus.cc:2468] T 66ae176961b74813a58bc15c9c4c6f2a P 90f3da57796b488c9087a2f523adf045 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 23.
I20260709 13:21:51.256230 16816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2794af5b99984efebdee564035841cb0" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:21:51.260627 16823 raft_consensus.cc:3060] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:21:51.256593 16819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2372981006d4b218215c28964dea623" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
W20260709 13:21:51.261348 16114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:21:51.261890 16114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:51.267278 16142 raft_consensus.cc:1240] T 2794af5b99984efebdee564035841cb0 P 647a343a542d4a139fe106c19095129b [term 14 FOLLOWER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 13. Current term is 14. Ops: []
I20260709 13:21:51.165055 22574 raft_consensus.cc:515] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:51.270306 16295 raft_consensus.cc:3060] T 5a96373bb0e14dce8a266b2fc6c58507 P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:21:51.270569 16819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee9ed67006144797b1de6ea8c2013a68" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:21:51.272184 16816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746aabf856ab401c9ef5e859371e7b67" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:51.273218 16265 leader_election.cc:304] T 2794af5b99984efebdee564035841cb0 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
I20260709 13:21:51.273905 16819 raft_consensus.cc:2410] T ee9ed67006144797b1de6ea8c2013a68 P cb82cc7bd6db46d980c75237a1402860 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b for term 18 because replica has last-logged OpId of term: 17 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 13.
I20260709 13:21:51.274025 16816 raft_consensus.cc:2393] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 14: Already voted for candidate cb82cc7bd6db46d980c75237a1402860 in this term.
I20260709 13:21:51.274850 16265 leader_election.cc:304] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
W20260709 13:21:51.277612 16117 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:21:51.278076 16117 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:51.277949 16265 leader_election.cc:304] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:21:51.280467 16820 raft_consensus.cc:1240] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 [term 17 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 16. Current term is 17. Ops: []
I20260709 13:21:51.280637 16825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26354361e9b044cea1c2b86c4882fe92" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:51.282410 16265 leader_election.cc:304] T 8d4ab870a05347868956aa8b841b2981 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:21:51.290321 22569 consensus_queue.cc:237] T c0eab25ce4c64e018a041beeda2867db P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:51.288406 16790 leader_election.cc:304] T ee9ed67006144797b1de6ea8c2013a68 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 
W20260709 13:21:51.269549 22492 consensus_meta.cc:287] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 1ca2725d290e47d68b1a4b609c9816ac: Time spent flushing consensus metadata: real 0.990s	user 0.009s	sys 0.005s
I20260709 13:21:51.303380 16143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee9ed67006144797b1de6ea8c2013a68" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:51.295740 16823 raft_consensus.cc:2468] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 23.
I20260709 13:21:51.306396 16143 raft_consensus.cc:2468] T ee9ed67006144797b1de6ea8c2013a68 P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 18.
I20260709 13:21:51.289420 16191 leader_election.cc:304] T e5ad4eb281564936a59f41e387a47156 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
I20260709 13:21:51.308189 16265 leader_election.cc:304] T 8a878999c0024d2abe517484180290fc P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:21:51.295028 16117 leader_election.cc:304] T 2794af5b99984efebdee564035841cb0 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860
I20260709 13:21:51.309340 16191 leader_election.cc:400] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 22 election: Vote denied by peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741) with higher term. Message: Invalid argument: T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac for earlier term 22. Current term is 23.
I20260709 13:21:51.311717 16191 leader_election.cc:403] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 22 election: Cancelling election due to peer responding with higher term
I20260709 13:21:51.313408 16265 leader_election.cc:304] T 31969392ccdb43b491911a319e8bc7e1 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: cb82cc7bd6db46d980c75237a1402860
I20260709 13:21:51.314800 16265 leader_election.cc:304] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac
I20260709 13:21:51.316308 16191 leader_election.cc:304] T 829b4ad8c5a54eababeea1181b960d4b P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
I20260709 13:21:51.318967 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fffab26427e24d398e2695aad40ca134" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:21:51.319453 16142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0eab25ce4c64e018a041beeda2867db" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:21:51.320384 16142 raft_consensus.cc:2393] T c0eab25ce4c64e018a041beeda2867db P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 17: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
I20260709 13:21:51.286273 22408 raft_consensus.cc:2749] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:21:51.294214 22404 raft_consensus.cc:2749] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:21:51.281973 22106 raft_consensus.cc:2749] T 2794af5b99984efebdee564035841cb0 P 90f3da57796b488c9087a2f523adf045 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:21:51.321039 16191 leader_election.cc:304] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
I20260709 13:21:51.322790 16117 leader_election.cc:304] T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860
I20260709 13:21:51.327559 16143 raft_consensus.cc:1275] T b87147731a68400e8cce9b879daa7c5d P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 13:21:51.267383 22292 raft_consensus.cc:515] T 18ad7e3af5504571a69d10c00750a01c P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } }
I20260709 13:21:51.298051 22615 raft_consensus.cc:2749] T 8d4ab870a05347868956aa8b841b2981 P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:21:51.322757 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:21:51.330484 16117 leader_election.cc:304] T ee9ed67006144797b1de6ea8c2013a68 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860
I20260709 13:21:51.331461 16300 raft_consensus.cc:2393] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 20: Already voted for candidate cb82cc7bd6db46d980c75237a1402860 in this term.
I20260709 13:21:51.338042 16150 raft_consensus.cc:2468] T fffab26427e24d398e2695aad40ca134 P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 17.
I20260709 13:21:51.315626 22152 raft_consensus.cc:2764] T e5ad4eb281564936a59f41e387a47156 P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Leader pre-election decision vote started in defunct term 16: won
I20260709 13:21:51.341717 16117 leader_election.cc:304] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:21:51.319664 22482 raft_consensus.cc:2749] T 8a878999c0024d2abe517484180290fc P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:21:51.345413 16191 leader_election.cc:304] T 746aabf856ab401c9ef5e859371e7b67 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
I20260709 13:21:51.318871 22554 raft_consensus.cc:2804] T 31969392ccdb43b491911a319e8bc7e1 P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:21:51.321727 22439 raft_consensus.cc:2804] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:21:51.322695 22496 raft_consensus.cc:3060] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:21:51.323333 22545 raft_consensus.cc:493] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:21:51.347652 16265 leader_election.cc:304] T c0eab25ce4c64e018a041beeda2867db P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:21:51.323809 22492 raft_consensus.cc:2749] T 829b4ad8c5a54eababeea1181b960d4b P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:21:51.343114 22494 raft_consensus.cc:2749] T 2794af5b99984efebdee564035841cb0 P 647a343a542d4a139fe106c19095129b [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:21:51.350812 22520 raft_consensus.cc:2804] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:21:51.340034 22504 leader_election.cc:290] T ada5a004ba7c46819051e3ccec71ffdb P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:51.344808 22600 raft_consensus.cc:493] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:51.353224 22526 raft_consensus.cc:2804] T 746aabf856ab401c9ef5e859371e7b67 P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:21:51.350956 22285 raft_consensus.cc:2804] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Leader election won for term 19
W20260709 13:21:51.372757 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 44542.2us
I20260709 13:21:51.380527 22574 leader_election.cc:290] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 17 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:51.380587 22480 leader_election.cc:290] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 16 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:51.381914 22622 raft_consensus.cc:2804] T ee9ed67006144797b1de6ea8c2013a68 P cb82cc7bd6db46d980c75237a1402860 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:21:51.382421 22622 raft_consensus.cc:493] T ee9ed67006144797b1de6ea8c2013a68 P cb82cc7bd6db46d980c75237a1402860 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:51.382810 22622 raft_consensus.cc:3060] T ee9ed67006144797b1de6ea8c2013a68 P cb82cc7bd6db46d980c75237a1402860 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:21:51.361650 22545 raft_consensus.cc:3060] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:21:51.387727 16147 raft_consensus.cc:2468] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 20.
I20260709 13:21:51.372355 22482 raft_consensus.cc:2749] T c0eab25ce4c64e018a041beeda2867db P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:21:51.383723 22600 raft_consensus.cc:515] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:51.371712 22554 raft_consensus.cc:697] T 31969392ccdb43b491911a319e8bc7e1 P 90f3da57796b488c9087a2f523adf045 [term 16 LEADER]: Becoming Leader. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:21:51.373443 22439 raft_consensus.cc:697] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045 [term 16 LEADER]: Becoming Leader. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:21:51.377897 22520 raft_consensus.cc:697] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [term 23 LEADER]: Becoming Leader. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:21:51.404194 16295 raft_consensus.cc:1275] T 5a96373bb0e14dce8a266b2fc6c58507 P 90f3da57796b488c9087a2f523adf045 [term 14 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:21:51.410969 16295 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48674 (request call id 3256) took 1428 ms. Trace:
I20260709 13:21:51.384533 22526 raft_consensus.cc:493] T 746aabf856ab401c9ef5e859371e7b67 P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:51.411420 16295 rpcz_store.cc:276] 0709 13:21:49.982921 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:50.353780 (+370859us) service_pool.cc:224] Handling call
0709 13:21:50.354101 (+   321us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:21:51.146910 (+792809us) spinlock_profiling.cc:243] Waited 1.61 ms on lock 0x7b4c0007bd00. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69aee2ba20 00007f69aee3ca29 00007f69aee02644 00007f69aee29f2d 00007f69aee29b93 00007f69b0b5a18b 00007f69b0b5acb5 00007f69b0b55f30 00007f69b0b577de 00007f69b0b61b49 00007f69b0b62416
0709 13:21:51.410624 (+263714us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:51.410899 (+   275us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3695360}
I20260709 13:21:51.412310 16225 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57600 (request call id 1869) took 2840 ms. Trace:
I20260709 13:21:51.416128 22610 raft_consensus.cc:2749] T ee9ed67006144797b1de6ea8c2013a68 P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:21:51.416913 22630 raft_consensus.cc:2749] T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:21:51.417266 22600 leader_election.cc:290] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
W20260709 13:21:51.418490 22632 log.cc:927] Time spent T 74ca8140f35c4ae29e8a664f6bdfc829 P cb82cc7bd6db46d980c75237a1402860: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
I20260709 13:21:51.420019 22588 consensus_queue.cc:1059] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: INVALID_TERM, Last received: 16.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.075s
I20260709 13:21:51.421203 22588 raft_consensus.cc:3055] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b [term 16 LEADER]: Stepping down as leader of term 16
W20260709 13:21:51.421547 16146 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40844 (request call id 2834) took 1988 ms (client timeout 1963 ms). Trace:
W20260709 13:21:51.421973 16146 rpcz_store.cc:269] 0709 13:21:49.432861 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:49.961132 (+528271us) service_pool.cc:224] Handling call
0709 13:21:51.421367 (+1460235us) spinlock_profiling.cc:243] Waited 601 ms on lock 0x7b540006ccd4. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:21:51.421484 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1383382784}
I20260709 13:21:51.422266 22348 consensus_queue.cc:260] T 08c74490c40243a88e62d1760b5cd814 P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:51.423928 22348 raft_consensus.cc:3060] T 08c74490c40243a88e62d1760b5cd814 P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:21:51.415721 16225 rpcz_store.cc:276] 0709 13:21:48.572029 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:49.231778 (+659749us) service_pool.cc:224] Handling call
0709 13:21:51.143269 (+1911491us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:21:51.216342 (+ 73073us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:51.412102 (+195760us) spinlock_profiling.cc:243] Waited 1.89 s on lock 0x7b54002c8650. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5e8ad 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:21:51.412247 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4348943744}
W20260709 13:21:51.413017 22580 log.cc:927] Time spent T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 1.155s	user 0.000s	sys 0.007s
I20260709 13:21:51.424714 16223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af57996a8be84d398f0f023e8d08ef77" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:51.350322 22543 consensus_queue.cc:260] T 2794af5b99984efebdee564035841cb0 P cb82cc7bd6db46d980c75237a1402860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:51.405659 22285 raft_consensus.cc:697] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac [term 19 LEADER]: Becoming Leader. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:21:51.431949 22526 raft_consensus.cc:3060] T 746aabf856ab401c9ef5e859371e7b67 P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:21:51.411433 22590 leader_election.cc:290] T 23f82224f4564fea93572e8206bad1fe P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:51.447849 22598 consensus_queue.cc:1059] T 2794af5b99984efebdee564035841cb0 P cb82cc7bd6db46d980c75237a1402860 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: INVALID_TERM, Last received: 13.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.015s
I20260709 13:21:51.473196 22554 consensus_queue.cc:237] T 31969392ccdb43b491911a319e8bc7e1 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:21:51.476126 22553 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 14 current term: 14 is higher.
I20260709 13:21:51.476205 22285 consensus_queue.cc:237] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.12, Last appended by leader: 12, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:51.421561 22588 raft_consensus.cc:740] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:21:51.455410 22520 consensus_queue.cc:237] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 21.14, Last appended by leader: 14, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:51.497319 22399 raft_consensus.cc:493] T ecc1e363230043fe9fef390f0b979f20 P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:21:51.457454 22292 leader_election.cc:290] T 18ad7e3af5504571a69d10c00750a01c P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 16 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:51.458081 22439 consensus_queue.cc:237] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:21:51.441867 22628 log.cc:927] Time spent T 806d7690c2b644959152074bb2cab0a7 P cb82cc7bd6db46d980c75237a1402860: Append to log took a long time: real 0.091s	user 0.001s	sys 0.000s
I20260709 13:21:51.499238 16292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e9b2bf7011f4ae688a79d7bb5a447b4" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 17 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:51.500599 16292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806d7690c2b644959152074bb2cab0a7" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:51.501343 16292 raft_consensus.cc:2393] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 18: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
I20260709 13:21:51.502421 16292 raft_consensus.cc:1240] T 7c2095a5f4ad49408dbc7f69c606120d P 90f3da57796b488c9087a2f523adf045 [term 14 LEADER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 13. Current term is 14. Ops: []
I20260709 13:21:51.481407 22482 raft_consensus.cc:493] T c9944d5e4b8c4109af9955c4f08c9095 P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:21:51.503203 22482 raft_consensus.cc:515] T c9944d5e4b8c4109af9955c4f08c9095 P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:51.503537 16295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e65676b21b748728ad39668cfe722ec" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:51.505475 22482 leader_election.cc:290] T c9944d5e4b8c4109af9955c4f08c9095 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:51.505620 16294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26354361e9b044cea1c2b86c4882fe92" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:51.506054 22482 raft_consensus.cc:493] T d2ab0ac8347641e6955e9bce7d90c1a2 P 90f3da57796b488c9087a2f523adf045 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:21:51.506399 16294 raft_consensus.cc:2393] T 26354361e9b044cea1c2b86c4882fe92 P 90f3da57796b488c9087a2f523adf045 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 14: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
W20260709 13:21:51.481838 16116 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:21:51.509191 16116 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:21:51.509630 16116 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:21:51.510161 16116 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:51.434208 22408 raft_consensus.cc:493] T 84f2da98864b4425bf7c0055951fdd4b P 90f3da57796b488c9087a2f523adf045 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:51.489989 22106 raft_consensus.cc:493] T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:21:51.491896 16114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:51.492842 22629 leader_election.cc:290] T 02838c86531d4f98bf58d1c22b8baf77 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:51.494493 16300 raft_consensus.cc:1275] T e4dc6b0541ca431ebc757beac00521f1 P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 14 index: 7. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
I20260709 13:21:51.519084 22492 raft_consensus.cc:493] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:51.519294 16220 raft_consensus.cc:1275] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 13:21:51.519745 22492 raft_consensus.cc:515] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:51.521337 16148 raft_consensus.cc:1240] T 5a959dbecc2648eb85d1cc550f5a3245 P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 15. Current term is 16. Ops: []
I20260709 13:21:51.522027 22615 consensus_queue.cc:1048] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:21:51.522269 22492 leader_election.cc:290] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:51.516795 22537 raft_consensus.cc:493] T 84f2da98864b4425bf7c0055951fdd4b P 647a343a542d4a139fe106c19095129b [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:51.498072 22399 raft_consensus.cc:515] T ecc1e363230043fe9fef390f0b979f20 P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:51.523676 22537 raft_consensus.cc:515] T 84f2da98864b4425bf7c0055951fdd4b P 647a343a542d4a139fe106c19095129b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:51.525938 22537 leader_election.cc:290] T 84f2da98864b4425bf7c0055951fdd4b P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:51.526206 22399 leader_election.cc:290] T ecc1e363230043fe9fef390f0b979f20 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:51.511790 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9944d5e4b8c4109af9955c4f08c9095" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:51.526605 16298 raft_consensus.cc:1240] T c0eab25ce4c64e018a041beeda2867db P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 16. Current term is 17. Ops: []
I20260709 13:21:51.527146 16217 raft_consensus.cc:2468] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 19.
I20260709 13:21:51.529853 16265 leader_election.cc:304] T c9944d5e4b8c4109af9955c4f08c9095 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
W20260709 13:21:51.515563 16296 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48684 (request call id 3035) took 1722 ms (client timeout 1636 ms). Trace:
I20260709 13:21:51.530877 22106 raft_consensus.cc:515] T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:21:51.531589 16296 rpcz_store.cc:269] 0709 13:21:49.792799 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:49.810294 (+ 17495us) service_pool.cc:224] Handling call
0709 13:21:51.515352 (+1705058us) spinlock_profiling.cc:243] Waited 990 ms on lock 0x7b54000aa354. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:21:51.515495 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2275947904}
I20260709 13:21:51.533365 22106 leader_election.cc:290] T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:21:51.530207 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
W20260709 13:21:51.514587 16189 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:51.518828 22404 raft_consensus.cc:493] T 57a744f5aaf944678a63469cbf514015 P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:51.506577 22482 raft_consensus.cc:515] T d2ab0ac8347641e6955e9bce7d90c1a2 P 90f3da57796b488c9087a2f523adf045 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:51.539130 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62fe3346cd3c42c4a55e14af9073bca4" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:21:51.518751 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e4bfb9090c8460eb14fa7e0d818e7f1" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:51.538511 16220 raft_consensus.cc:1240] T c2f5f89186204242bf377f7c42b2e2da P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 17. Current term is 18. Ops: []
I20260709 13:21:51.437533 22593 raft_consensus.cc:515] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:51.544896 22572 raft_consensus.cc:3055] T 26354361e9b044cea1c2b86c4882fe92 P cb82cc7bd6db46d980c75237a1402860 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:21:51.558115 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806d7690c2b644959152074bb2cab0a7" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:51.561816 16226 raft_consensus.cc:2468] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 18.
I20260709 13:21:51.563043 16218 raft_consensus.cc:1275] T 5a96373bb0e14dce8a266b2fc6c58507 P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:21:51.565025 16223 raft_consensus.cc:1240] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 16. Current term is 17. Ops: []
I20260709 13:21:51.522924 22492 raft_consensus.cc:493] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:21:51.418485 22196 consensus_queue.cc:260] T 7c2095a5f4ad49408dbc7f69c606120d P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 13.6, Last appended by leader: 6, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:51.571729 16148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:51.537952 22629 raft_consensus.cc:493] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:51.577486 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada5a004ba7c46819051e3ccec71ffdb" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:51.577018 16220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746aabf856ab401c9ef5e859371e7b67" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:51.606410 16223 raft_consensus.cc:1240] T 19baa7bc367844c288bca8f81439c413 P 1ca2725d290e47d68b1a4b609c9816ac [term 20 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 19. Current term is 20. Ops: []
I20260709 13:21:51.607734 16293 raft_consensus.cc:1240] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 16. Current term is 17. Ops: []
I20260709 13:21:51.612916 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6f07842d3aa47858df5665874c86f3a" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:51.613598 16226 raft_consensus.cc:2468] T b6f07842d3aa47858df5665874c86f3a P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 17.
I20260709 13:21:51.535398 22408 raft_consensus.cc:515] T 84f2da98864b4425bf7c0055951fdd4b P 90f3da57796b488c9087a2f523adf045 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:51.566569 22537 consensus_queue.cc:1048] T b6f07842d3aa47858df5665874c86f3a P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.049s
I20260709 13:21:51.561391 22572 raft_consensus.cc:740] T 26354361e9b044cea1c2b86c4882fe92 P cb82cc7bd6db46d980c75237a1402860 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:21:51.623238 16223 raft_consensus.cc:1240] T d16dfad0c3b540e5b4b86492ddf87208 P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 17. Current term is 18. Ops: []
I20260709 13:21:51.577651 22573 raft_consensus.cc:3055] T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:21:51.612800 22106 raft_consensus.cc:493] T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:51.546803 22404 raft_consensus.cc:515] T 57a744f5aaf944678a63469cbf514015 P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:51.625658 22482 leader_election.cc:290] T d2ab0ac8347641e6955e9bce7d90c1a2 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:51.574088 22518 consensus_queue.cc:1048] T 18ad7e3af5504571a69d10c00750a01c P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.052s
I20260709 13:21:51.626760 22572 consensus_queue.cc:260] T 26354361e9b044cea1c2b86c4882fe92 P cb82cc7bd6db46d980c75237a1402860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:21:51.636072 16788 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:51.601418 22494 raft_consensus.cc:493] T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:21:51.641266 16265 leader_election.cc:304] T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:21:51.610132 22196 raft_consensus.cc:3060] T 7c2095a5f4ad49408dbc7f69c606120d P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:21:51.646741 16265 leader_election.cc:304] T fffab26427e24d398e2695aad40ca134 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac
I20260709 13:21:51.582079 22492 raft_consensus.cc:515] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:51.648052 16300 raft_consensus.cc:1275] T 8158bbd17b994d2eb96f560bd0f1b88d P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 13:21:51.546098 22553 raft_consensus.cc:2804] T c9944d5e4b8c4109af9955c4f08c9095 P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:21:51.650204 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f2da98864b4425bf7c0055951fdd4b" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:51.610502 22629 raft_consensus.cc:515] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:51.655018 22572 raft_consensus.cc:3060] T 26354361e9b044cea1c2b86c4882fe92 P cb82cc7bd6db46d980c75237a1402860 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:21:51.655793 22482 raft_consensus.cc:2804] T fffab26427e24d398e2695aad40ca134 P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:21:51.656564 16265 leader_election.cc:304] T ecc1e363230043fe9fef390f0b979f20 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:21:51.600625 22588 consensus_queue.cc:260] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 16.14, Last appended by leader: 14, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:51.638422 22573 raft_consensus.cc:740] T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:21:51.642335 22496 raft_consensus.cc:2749] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [term 23 FOLLOWER]: Leader election lost for term 22. Reason: Vote denied by peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741) with higher term. Message: Invalid argument: T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac for earlier term 22. Current term is 23.
I20260709 13:21:51.660857 16223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ab0ac8347641e6955e9bce7d90c1a2" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:51.660451 16144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6f07842d3aa47858df5665874c86f3a" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:51.666775 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f82224f4564fea93572e8206bad1fe" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:51.667529 16150 raft_consensus.cc:2468] T 23f82224f4564fea93572e8206bad1fe P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 15.
W20260709 13:21:51.670284 16790 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:51.643150 22106 raft_consensus.cc:515] T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:51.671849 16790 leader_election.cc:304] T af57996a8be84d398f0f023e8d08ef77 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:21:51.677888 22591 raft_consensus.cc:2749] T af57996a8be84d398f0f023e8d08ef77 P cb82cc7bd6db46d980c75237a1402860 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:21:51.649451 22500 raft_consensus.cc:493] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
W20260709 13:21:51.688774 16788 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:21:51.708025 16114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:21:51.711710 16114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:51.652622 22553 raft_consensus.cc:493] T c9944d5e4b8c4109af9955c4f08c9095 P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Starting leader election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:21:51.727484 16300 pending_rounds.cc:85] T ecc1e363230043fe9fef390f0b979f20 P 90f3da57796b488c9087a2f523adf045: Aborting all ops after (but not including) 11
I20260709 13:21:51.730711 16300 pending_rounds.cc:107] T ecc1e363230043fe9fef390f0b979f20 P 90f3da57796b488c9087a2f523adf045: Aborting uncommitted NO_OP operation due to leader change: 18.12
I20260709 13:21:51.731191 16300 raft_consensus.cc:2889] T ecc1e363230043fe9fef390f0b979f20 P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:21:51.657254 22399 consensus_queue.cc:1059] T 5a959dbecc2648eb85d1cc550f5a3245 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: INVALID_TERM, Last received: 15.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.017s
I20260709 13:21:51.736129 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada5a004ba7c46819051e3ccec71ffdb" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:51.737016 16226 raft_consensus.cc:2468] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 18.
I20260709 13:21:51.743722 16224 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57604 (request call id 3199) took 2101 ms. Trace:
I20260709 13:21:51.652396 22494 raft_consensus.cc:515] T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:51.733973 16116 leader_election.cc:304] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
I20260709 13:21:51.674410 22482 raft_consensus.cc:697] T fffab26427e24d398e2695aad40ca134 P 90f3da57796b488c9087a2f523adf045 [term 17 LEADER]: Becoming Leader. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:21:51.746217 16223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdbd78acb4cf46298e0de8ba0d130e19" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:51.744163 16224 rpcz_store.cc:276] 0709 13:21:49.642352 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:49.696355 (+ 54003us) service_pool.cc:224] Handling call
0709 13:21:49.712988 (+ 16633us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:21:50.058015 (+345027us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:21:50.058048 (+    33us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:21:50.058665 (+   617us) log.cc:844] Serialized 52 byte log entry
0709 13:21:50.403727 (+345062us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:21:50.403781 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:51.726589 (+1322808us) spinlock_profiling.cc:243] Waited 227 ms on lock 0x7b5400246654. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:21:51.726641 (+    52us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:21:51.726688 (+    47us) raft_consensus.cc:1685] finished
0709 13:21:51.726702 (+    14us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:21:51.743648 (+ 16946us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":45901,"raft.queue_time_us":49017,"raft.run_cpu_time_us":123,"raft.run_wall_time_us":56949,"spinlock_wait_cycles":521384704,"thread_start_us":199583,"threads_started":1,"wal-append.queue_time_us":362575}
I20260709 13:21:51.755841 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746aabf856ab401c9ef5e859371e7b67" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:21:51.756582 16223 raft_consensus.cc:2468] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 20.
I20260709 13:21:51.740401 22553 raft_consensus.cc:3060] T c9944d5e4b8c4109af9955c4f08c9095 P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:21:51.672816 22630 consensus_queue.cc:1059] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: INVALID_TERM, Last received: 16.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.074s
W20260709 13:21:51.751632 22589 consensus_meta.cc:287] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b: Time spent flushing consensus metadata: real 0.687s	user 0.010s	sys 0.004s
I20260709 13:21:51.780937 16217 raft_consensus.cc:2468] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 23.
W20260709 13:21:51.782600 16117 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:51.717499 22586 consensus_queue.cc:1059] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: INVALID_TERM, Last received: 15.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.041s
I20260709 13:21:51.782862 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:21:51.784880 16226 raft_consensus.cc:2393] T 19baa7bc367844c288bca8f81439c413 P 1ca2725d290e47d68b1a4b609c9816ac [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 in current term 20: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
W20260709 13:21:51.805413 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 48254.5us
I20260709 13:21:51.729620 22504 consensus_queue.cc:1059] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.033s
I20260709 13:21:51.704327 22500 raft_consensus.cc:515] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:21:51.784897 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 57998.6us
I20260709 13:21:51.762809 22197 raft_consensus.cc:2804] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:21:51.735555 22508 consensus_queue.cc:1048] T b87147731a68400e8cce9b879daa7c5d P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.033s
I20260709 13:21:51.826716 16790 leader_election.cc:304] T 23f82224f4564fea93572e8206bad1fe P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860; no voters: 
W20260709 13:21:51.777668 22602 consensus_meta.cc:287] T e4dc6b0541ca431ebc757beac00521f1 P 1ca2725d290e47d68b1a4b609c9816ac: Time spent flushing consensus metadata: real 0.567s	user 0.011s	sys 0.000s
I20260709 13:21:51.851585 16223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e4bfb9090c8460eb14fa7e0d818e7f1" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:51.783782 22439 raft_consensus.cc:2804] T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:21:51.762858 22474 consensus_queue.cc:1048] T 5a96373bb0e14dce8a266b2fc6c58507 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.065s
I20260709 13:21:51.867018 16218 raft_consensus.cc:1240] T 7c2095a5f4ad49408dbc7f69c606120d P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 13. Current term is 14. Ops: []
I20260709 13:21:51.819669 22649 raft_consensus.cc:2749] T ecc1e363230043fe9fef390f0b979f20 P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:21:51.829505 22554 raft_consensus.cc:493] T 434bff3082d141838068f323bdb0d1ed P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:51.827907 22197 raft_consensus.cc:493] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Starting leader election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:21:51.889083 16223 raft_consensus.cc:1275] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 13:21:51.853838 22593 leader_election.cc:290] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 17 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:51.820405 22589 consensus_queue.cc:1059] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: INVALID_TERM, Last received: 16.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.050s
W20260709 13:21:51.881577 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 54969.4us
I20260709 13:21:51.877468 22591 raft_consensus.cc:3055] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 [term 15 LEADER]: Stepping down as leader of term 15
W20260709 13:21:51.934139 16219 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57600 (request call id 1881) took 1773 ms (client timeout 1778 ms). Trace:
W20260709 13:21:51.934567 16219 rpcz_store.cc:269] 0709 13:21:50.161161 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:50.174007 (+ 12846us) service_pool.cc:224] Handling call
0709 13:21:51.933934 (+1759927us) spinlock_profiling.cc:243] Waited 273 ms on lock 0x7b54000154d4. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:21:51.934076 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":628363520}
W20260709 13:21:51.963585 16114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:51.935649 16221 raft_consensus.cc:1240] T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 17. Current term is 18. Ops: []
I20260709 13:21:51.963905 16790 leader_election.cc:304] T ada5a004ba7c46819051e3ccec71ffdb P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:21:51.892282 22197 raft_consensus.cc:3060] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:21:51.857875 22602 raft_consensus.cc:515] T e4dc6b0541ca431ebc757beac00521f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:51.897760 22404 leader_election.cc:290] T 57a744f5aaf944678a63469cbf514015 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:51.999375 16144 raft_consensus.cc:1275] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 18 index: 15. Preceding OpId from leader: term: 20 index: 16. (index mismatch)
I20260709 13:21:51.928166 22408 leader_election.cc:290] T 84f2da98864b4425bf7c0055951fdd4b P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:21:51.967595 22662 raft_consensus.cc:493] T 9c61a39fd4f84424b129c1a6f9ba404d P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:51.934051 22591 raft_consensus.cc:740] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
W20260709 13:21:52.048395 16116 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:21:52.048067 16189 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:52.067751 16218 raft_consensus.cc:1240] T e4dc6b0541ca431ebc757beac00521f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 17. Current term is 18. Ops: []
I20260709 13:21:51.935945 22492 leader_election.cc:290] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
W20260709 13:21:52.014691 16114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:51.894876 22554 raft_consensus.cc:515] T 434bff3082d141838068f323bdb0d1ed P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:52.042848 22399 raft_consensus.cc:493] T c4f65cdf00b748bb963ee6e81ef43b79 P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
W20260709 13:21:52.119150 16115 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:21:52.104647 16114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:52.046947 22600 raft_consensus.cc:2804] T 23f82224f4564fea93572e8206bad1fe P cb82cc7bd6db46d980c75237a1402860 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:21:52.120807 16144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02838c86531d4f98bf58d1c22b8baf77" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:52.122053 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ad5dbfa99ee4b8d9e28be92b3f36db5" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:21:52.122618 16143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a878999c0024d2abe517484180290fc" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:21:52.123230 16143 raft_consensus.cc:2376] T 8a878999c0024d2abe517484180290fc P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 15. Re-sending same reply.
I20260709 13:21:52.127818 16150 raft_consensus.cc:1275] T 84f2da98864b4425bf7c0055951fdd4b P 647a343a542d4a139fe106c19095129b [term 14 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 13:21:52.121524 16144 raft_consensus.cc:2468] T 02838c86531d4f98bf58d1c22b8baf77 P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 18.
I20260709 13:21:52.121084 16148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b7d1ab519044eda2d5e04a07242f25" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:21:52.052040 22670 raft_consensus.cc:3055] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:21:52.137007 22106 leader_election.cc:290] T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:52.138616 16151 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40830 (request call id 1912) took 1786 ms. Trace:
W20260709 13:21:52.139331 22665 log.cc:927] Time spent T ecc1e363230043fe9fef390f0b979f20 P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.149s	user 0.000s	sys 0.001s
I20260709 13:21:51.925165 22482 consensus_queue.cc:237] T fffab26427e24d398e2695aad40ca134 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:52.068651 22408 raft_consensus.cc:493] T cdbd78acb4cf46298e0de8ba0d130e19 P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
W20260709 13:21:52.143811 22648 log.cc:927] Time spent T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860: Append to log took a long time: real 0.092s	user 0.000s	sys 0.000s
I20260709 13:21:52.066542 22356 raft_consensus.cc:493] T 5a959dbecc2648eb85d1cc550f5a3245 P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:52.148138 16143 raft_consensus.cc:1240] T 8d4ab870a05347868956aa8b841b2981 P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 15. Current term is 16. Ops: []
W20260709 13:21:52.150517 16788 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:21:52.199783 16788 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:21:52.157469 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 193681us
I20260709 13:21:52.162698 16142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18ad7e3af5504571a69d10c00750a01c" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:21:52.157524 16151 rpcz_store.cc:276] 0709 13:21:50.352565 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:50.364326 (+ 11761us) service_pool.cc:224] Handling call
0709 13:21:52.138410 (+1774084us) spinlock_profiling.cc:243] Waited 581 ms on lock 0x7b540028da54. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:21:52.138555 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1337069312}
I20260709 13:21:52.133628 22600 raft_consensus.cc:493] T 23f82224f4564fea93572e8206bad1fe P cb82cc7bd6db46d980c75237a1402860 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:52.200809 16790 leader_election.cc:304] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:21:52.131879 22494 leader_election.cc:290] T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:52.148944 22670 raft_consensus.cc:740] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:21:52.169090 22589 raft_consensus.cc:493] T c0eab25ce4c64e018a041beeda2867db P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:52.161929 22664 raft_consensus.cc:2804] T ada5a004ba7c46819051e3ccec71ffdb P cb82cc7bd6db46d980c75237a1402860 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:21:52.071334 22662 raft_consensus.cc:515] T 9c61a39fd4f84424b129c1a6f9ba404d P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:52.221864 16151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2372981006d4b218215c28964dea623" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:52.231894 16151 raft_consensus.cc:2393] T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 in current term 20: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
W20260709 13:21:52.203431 22545 consensus_meta.cc:287] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860: Time spent flushing consensus metadata: real 0.801s	user 0.013s	sys 0.000s
I20260709 13:21:52.183504 22629 leader_election.cc:290] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:52.207468 22600 raft_consensus.cc:3060] T 23f82224f4564fea93572e8206bad1fe P cb82cc7bd6db46d980c75237a1402860 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:21:52.147836 22399 raft_consensus.cc:515] T c4f65cdf00b748bb963ee6e81ef43b79 P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:21:52.214897 22348 consensus_meta.cc:287] T 08c74490c40243a88e62d1760b5cd814 P 1ca2725d290e47d68b1a4b609c9816ac: Time spent flushing consensus metadata: real 0.790s	user 0.013s	sys 0.000s
I20260709 13:21:52.228760 22496 raft_consensus.cc:493] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:52.236572 22664 raft_consensus.cc:493] T ada5a004ba7c46819051e3ccec71ffdb P cb82cc7bd6db46d980c75237a1402860 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:52.203001 22408 raft_consensus.cc:515] T cdbd78acb4cf46298e0de8ba0d130e19 P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:52.238468 22589 raft_consensus.cc:515] T c0eab25ce4c64e018a041beeda2867db P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:21:52.281148 22526 consensus_meta.cc:287] T 746aabf856ab401c9ef5e859371e7b67 P 1ca2725d290e47d68b1a4b609c9816ac: Time spent flushing consensus metadata: real 0.805s	user 0.012s	sys 0.001s
I20260709 13:21:52.281736 22408 leader_election.cc:290] T cdbd78acb4cf46298e0de8ba0d130e19 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:52.237854 22590 raft_consensus.cc:2804] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:21:52.281672 22526 raft_consensus.cc:515] T 746aabf856ab401c9ef5e859371e7b67 P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:52.282289 22589 leader_election.cc:290] T c0eab25ce4c64e018a041beeda2867db P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:52.210366 22356 raft_consensus.cc:515] T 5a959dbecc2648eb85d1cc550f5a3245 P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:52.282405 22590 raft_consensus.cc:493] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:52.283972 22590 raft_consensus.cc:3060] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:21:52.285370 22356 leader_election.cc:290] T 5a959dbecc2648eb85d1cc550f5a3245 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:52.254055 22545 raft_consensus.cc:515] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:52.288151 22545 leader_election.cc:290] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 21 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:52.212482 22591 consensus_queue.cc:260] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:52.290087 16220 raft_consensus.cc:2368] T 746aabf856ab401c9ef5e859371e7b67 P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b for earlier term 14. Current term is 15.
I20260709 13:21:52.291517 22526 leader_election.cc:290] T 746aabf856ab401c9ef5e859371e7b67 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 15 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:52.294369 22676 consensus_queue.cc:1048] T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.000s
W20260709 13:21:52.295352 22646 log.cc:927] Time spent T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 0.083s	user 0.000s	sys 0.001s
I20260709 13:21:52.263444 22500 leader_election.cc:290] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
W20260709 13:21:52.297284 22622 consensus_meta.cc:287] T ee9ed67006144797b1de6ea8c2013a68 P cb82cc7bd6db46d980c75237a1402860: Time spent flushing consensus metadata: real 0.914s	user 0.009s	sys 0.001s
I20260709 13:21:52.297894 22622 raft_consensus.cc:515] T ee9ed67006144797b1de6ea8c2013a68 P cb82cc7bd6db46d980c75237a1402860 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:52.299973 22662 leader_election.cc:290] T 9c61a39fd4f84424b129c1a6f9ba404d P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:52.301045 16291 raft_consensus.cc:1240] T 26354361e9b044cea1c2b86c4882fe92 P 90f3da57796b488c9087a2f523adf045 [term 14 FOLLOWER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 13. Current term is 14. Ops: []
I20260709 13:21:52.301146 22664 raft_consensus.cc:3060] T ada5a004ba7c46819051e3ccec71ffdb P cb82cc7bd6db46d980c75237a1402860 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:21:52.302735 16291 raft_consensus.cc:1275] T 57a744f5aaf944678a63469cbf514015 P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 13:21:52.305105 16218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66f75124215f4ca6bef831e174a792d8" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:52.308801 22399 leader_election.cc:290] T c4f65cdf00b748bb963ee6e81ef43b79 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:21:52.297185 22670 consensus_queue.cc:260] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:52.312047 22670 raft_consensus.cc:3060] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:21:52.319940 16219 raft_consensus.cc:1240] T 18ad7e3af5504571a69d10c00750a01c P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 15. Current term is 16. Ops: [15.11-15.11]
W20260709 13:21:52.321179 22656 log.cc:927] Time spent T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.252s	user 0.000s	sys 0.001s
W20260709 13:21:52.320124 22667 log.cc:927] Time spent T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 0.289s	user 0.001s	sys 0.000s
W20260709 13:21:52.259351 22647 log.cc:927] Time spent T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.406s	user 0.000s	sys 0.002s
W20260709 13:21:52.324501 22671 log.cc:927] Time spent T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
W20260709 13:21:52.331739 22658 log.cc:927] Time spent T 18ad7e3af5504571a69d10c00750a01c P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.289s	user 0.001s	sys 0.000s
I20260709 13:21:52.332333 22661 raft_consensus.cc:493] T 26354361e9b044cea1c2b86c4882fe92 P 90f3da57796b488c9087a2f523adf045 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:52.332887 22661 raft_consensus.cc:515] T 26354361e9b044cea1c2b86c4882fe92 P 90f3da57796b488c9087a2f523adf045 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:52.336861 16295 raft_consensus.cc:1275] T b87147731a68400e8cce9b879daa7c5d P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 13:21:52.360389 16218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f2da98864b4425bf7c0055951fdd4b" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:52.363317 16218 raft_consensus.cc:1275] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
W20260709 13:21:52.376652 16115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 251278us
I20260709 13:21:52.380784 16218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4f65cdf00b748bb963ee6e81ef43b79" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:52.386258 16218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdbd78acb4cf46298e0de8ba0d130e19" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:52.387259 16218 raft_consensus.cc:2393] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 20: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
I20260709 13:21:52.335439 22661 leader_election.cc:290] T 26354361e9b044cea1c2b86c4882fe92 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
W20260709 13:21:52.388229 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 169697us
I20260709 13:21:52.393563 16218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2372981006d4b218215c28964dea623" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
W20260709 13:21:52.395008 16790 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:52.395557 16218 raft_consensus.cc:2468] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 20.
I20260709 13:21:52.399595 16218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74ca8140f35c4ae29e8a664f6bdfc829" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:52.402318 16115 leader_election.cc:304] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860
I20260709 13:21:52.383479 22348 raft_consensus.cc:493] T 7c2095a5f4ad49408dbc7f69c606120d P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:52.406172 16115 leader_election.cc:304] T 9e65676b21b748728ad39668cfe722ec P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860
I20260709 13:21:52.408656 16115 leader_election.cc:304] T 26354361e9b044cea1c2b86c4882fe92 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860
I20260709 13:21:52.335731 22496 raft_consensus.cc:515] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:52.412472 16151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a9de347c711477e87c582d003f36925" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:21:52.428709 16143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee9ed67006144797b1de6ea8c2013a68" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:21:52.429531 16143 raft_consensus.cc:3060] T ee9ed67006144797b1de6ea8c2013a68 P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:21:52.433311 22679 raft_consensus.cc:493] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:52.434175 22679 raft_consensus.cc:515] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:21:52.433490 16048 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:58400 (request call id 193) took 16317 ms (client timeout 15000 ms). Trace:
I20260709 13:21:52.407567 22673 raft_consensus.cc:493] T 3884f864a96d428785895e5463e5728f P cb82cc7bd6db46d980c75237a1402860 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:52.407714 22662 raft_consensus.cc:2749] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:21:52.412267 22474 raft_consensus.cc:2749] T 9e65676b21b748728ad39668cfe722ec P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:21:52.457135 22553 consensus_meta.cc:287] T c9944d5e4b8c4109af9955c4f08c9095 P 90f3da57796b488c9087a2f523adf045: Time spent flushing consensus metadata: real 0.674s	user 0.006s	sys 0.006s
W20260709 13:21:52.449690 22572 consensus_meta.cc:287] T 26354361e9b044cea1c2b86c4882fe92 P cb82cc7bd6db46d980c75237a1402860: Time spent flushing consensus metadata: real 0.787s	user 0.006s	sys 0.007s
I20260709 13:21:52.421254 22602 leader_election.cc:290] T e4dc6b0541ca431ebc757beac00521f1 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 18 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:52.407347 22610 consensus_queue.cc:1048] T 5a96373bb0e14dce8a266b2fc6c58507 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.043s
I20260709 13:21:52.421211 22348 raft_consensus.cc:515] T 7c2095a5f4ad49408dbc7f69c606120d P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:52.457582 22474 consensus_queue.cc:1048] T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.004s
I20260709 13:21:52.457605 22553 raft_consensus.cc:515] T c9944d5e4b8c4109af9955c4f08c9095 P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:52.469236 22679 leader_election.cc:290] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
W20260709 13:21:52.483444 16263 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.559s	user 0.006s	sys 0.017s
I20260709 13:21:52.452922 16143 raft_consensus.cc:2468] T ee9ed67006144797b1de6ea8c2013a68 P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 19.
I20260709 13:21:52.452111 22554 leader_election.cc:290] T 434bff3082d141838068f323bdb0d1ed P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:52.509254 16790 leader_election.cc:304] T ee9ed67006144797b1de6ea8c2013a68 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:21:52.512532 16218 raft_consensus.cc:1275] T 7c2095a5f4ad49408dbc7f69c606120d P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 13 index: 6. Preceding OpId from leader: term: 14 index: 7. (index mismatch)
W20260709 13:21:52.501643 16223 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.540s	user 0.009s	sys 0.017s
W20260709 13:21:52.508420 16263 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.584s	user 0.007s	sys 0.017s
I20260709 13:21:52.483470 22622 leader_election.cc:290] T ee9ed67006144797b1de6ea8c2013a68 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 19 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:21:52.516865 16291 raft_consensus.cc:1240] T c9944d5e4b8c4109af9955c4f08c9095 P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 19. Current term is 20. Ops: []
W20260709 13:21:52.516861 16115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 94460.8us
W20260709 13:21:52.519294 16114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:52.460453 22673 raft_consensus.cc:515] T 3884f864a96d428785895e5463e5728f P cb82cc7bd6db46d980c75237a1402860 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:21:52.519361 16223 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.584s	user 0.010s	sys 0.018s
I20260709 13:21:52.521239 16191 leader_election.cc:304] T 8a878999c0024d2abe517484180290fc P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 90f3da57796b488c9087a2f523adf045
I20260709 13:21:52.523521 16191 leader_election.cc:304] T 02838c86531d4f98bf58d1c22b8baf77 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
I20260709 13:21:52.528190 16294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f2da98864b4425bf7c0055951fdd4b" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:52.528926 16294 raft_consensus.cc:2468] T 84f2da98864b4425bf7c0055951fdd4b P 90f3da57796b488c9087a2f523adf045 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 14.
I20260709 13:21:52.501327 22662 consensus_queue.cc:1048] T 8158bbd17b994d2eb96f560bd0f1b88d P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.019s
I20260709 13:21:52.529511 16295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0eab25ce4c64e018a041beeda2867db" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:52.515816 22681 raft_consensus.cc:2804] T ee9ed67006144797b1de6ea8c2013a68 P cb82cc7bd6db46d980c75237a1402860 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:21:52.520136 22679 raft_consensus.cc:493] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:52.532087 16294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a959dbecc2648eb85d1cc550f5a3245" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
W20260709 13:21:52.530490 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 163545us
I20260709 13:21:52.531085 16295 raft_consensus.cc:2468] T c0eab25ce4c64e018a041beeda2867db P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 17.
W20260709 13:21:52.551561 16114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:21:52.590276 16788 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:52.640429 16300 raft_consensus.cc:1275] T 3884f864a96d428785895e5463e5728f P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 12 index: 12. Preceding OpId from leader: term: 13 index: 13. (index mismatch)
I20260709 13:21:52.526571 22526 raft_consensus.cc:2804] T 8a878999c0024d2abe517484180290fc P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:21:52.648633 16222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee9ed67006144797b1de6ea8c2013a68" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:21:52.530565 22677 raft_consensus.cc:2804] T 02838c86531d4f98bf58d1c22b8baf77 P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:21:52.650789 16265 leader_election.cc:304] T 8cc959fb7f704fba9009c0595467d074 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac
I20260709 13:21:52.847033 22677 raft_consensus.cc:493] T 02838c86531d4f98bf58d1c22b8baf77 P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Starting leader election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:21:52.640714 22285 raft_consensus.cc:493] T 8c11074579b046d99e585c2818e18a2c P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:52.543890 22600 raft_consensus.cc:515] T 23f82224f4564fea93572e8206bad1fe P cb82cc7bd6db46d980c75237a1402860 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:52.530130 22602 consensus_queue.cc:1048] T 84f2da98864b4425bf7c0055951fdd4b P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.311s
I20260709 13:21:52.871268 16293 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40474 (request call id 1139) took 2813 ms. Trace:
I20260709 13:21:52.872475 16217 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57604 (request call id 3207) took 1336 ms. Trace:
I20260709 13:21:52.872099 16293 rpcz_store.cc:276] 0709 13:21:50.057641 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:50.549737 (+492096us) service_pool.cc:224] Handling call
0709 13:21:50.550074 (+   337us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:21:51.608188 (+1058114us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:52.870916 (+1262728us) spinlock_profiling.cc:243] Waited 1.06 s on lock 0x7b540024ca54. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:21:52.871183 (+   267us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2432025856}
I20260709 13:21:52.873255 16217 rpcz_store.cc:276] 0709 13:21:51.536033 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:51.538797 (+  2764us) service_pool.cc:224] Handling call
0709 13:21:52.872240 (+1333443us) spinlock_profiling.cc:243] Waited 143 ms on lock 0x7b5400367c54. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:21:52.872402 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":328447232}
W20260709 13:21:52.436021 16048 rpcz_store.cc:269] 0709 13:21:36.115770 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:36.116080 (+   310us) service_pool.cc:224] Handling call
0709 13:21:52.433243 (+16317163us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:21:41.990719 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:21:42.015976 (+ 25257us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:21:42.016031 (+    55us) write_op.cc:435] Acquired schema lock
0709 13:21:42.016059 (+    28us) tablet.cc:662] Decoding operations
0709 13:21:42.018466 (+  2407us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:21:42.018579 (+   113us) write_op.cc:641] Partition lock acquired for write op
0709 13:21:42.018633 (+    54us) tablet.cc:685] Acquiring locks for 20 operations
0709 13:21:42.154048 (+135415us) tablet.cc:701] Row locks acquired
0709 13:21:42.154069 (+    21us) write_op.cc:260] PREPARE: finished
0709 13:21:42.154239 (+   170us) write_op.cc:270] Start()
0709 13:21:42.154365 (+   126us) write_op.cc:276] Timestamp: P: 1783603302154183 usec, L: 0
0709 13:21:42.154384 (+    19us) op_driver.cc:348] REPLICATION: starting
0709 13:21:42.157766 (+  3382us) log.cc:844] Serialized 8066 byte log entry
0709 13:21:43.514240 (+1356474us) op_driver.cc:464] REPLICATION: finished
0709 13:21:43.995097 (+480857us) write_op.cc:301] APPLY: starting
0709 13:21:43.999130 (+  4033us) tablet.cc:1367] starting BulkCheckPresence
0709 13:21:45.229885 (+1230755us) tablet.cc:1370] finished BulkCheckPresence
0709 13:21:45.230131 (+   246us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:21:46.194732 (+964601us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:21:46.334438 (+139706us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=40,key_file_lookups=40,delta_file_lookups=0,mrs_lookups=0
0709 13:21:46.334518 (+    80us) write_op.cc:312] APPLY: finished
0709 13:21:46.335204 (+   686us) log.cc:844] Serialized 180 byte log entry
0709 13:21:46.862838 (+527634us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:21:47.141807 (+278969us) write_op.cc:454] Released schema lock
0709 13:21:47.141887 (+    80us) write_op.cc:341] FINISH: Updating metrics
0709 13:21:47.142522 (+   635us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:21:47.142554 (+    32us) write_op.cc:454] Released schema lock
Metrics: {"dns_us":217,"mutex_wait_us":9874,"reported_tablets":35,"child_traces":[["op",{"apply.queue_time_us":457330,"apply.run_cpu_time_us":58064,"apply.run_wall_time_us":3157558,"cfile_cache_hit":82,"cfile_cache_hit_bytes":87647,"num_ops":20,"prepare.queue_time_us":178001,"prepare.run_cpu_time_us":21039,"prepare.run_wall_time_us":698149,"raft.queue_time_us":394947,"raft.run_cpu_time_us":35511,"raft.run_wall_time_us":364955,"replication_time_us":1325514,"thread_start_us":1186401,"threads_started":5,"wal-append.queue_time_us":1274234,"wal-append.run_cpu_time_us":13421,"wal-append.run_wall_time_us":2478656}]]}
I20260709 13:21:52.657707 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c61a39fd4f84424b129c1a6f9ba404d" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:52.879608 16297 raft_consensus.cc:2468] T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 18.
I20260709 13:21:52.628371 22492 consensus_queue.cc:1059] T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.310s
I20260709 13:21:52.648839 22399 raft_consensus.cc:493] T d16dfad0c3b540e5b4b86492ddf87208 P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:52.893231 22674 raft_consensus.cc:493] T 8158bbd17b994d2eb96f560bd0f1b88d P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:52.893836 22674 raft_consensus.cc:515] T 8158bbd17b994d2eb96f560bd0f1b88d P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:52.896307 22674 leader_election.cc:290] T 8158bbd17b994d2eb96f560bd0f1b88d P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:21:52.896903 22674 raft_consensus.cc:2764] T 8cc959fb7f704fba9009c0595467d074 P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Leader pre-election decision vote started in defunct term 15: won
I20260709 13:21:52.891793 22610 raft_consensus.cc:493] T 8a878999c0024d2abe517484180290fc P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:52.906186 22610 raft_consensus.cc:515] T 8a878999c0024d2abe517484180290fc P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:52.849602 22677 raft_consensus.cc:3060] T 02838c86531d4f98bf58d1c22b8baf77 P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:21:52.909036 22610 leader_election.cc:290] T 8a878999c0024d2abe517484180290fc P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:52.912047 16223 raft_consensus.cc:1275] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac [term 20 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 18 index: 15. Preceding OpId from leader: term: 20 index: 16. (index mismatch)
I20260709 13:21:52.915920 22600 leader_election.cc:290] T 23f82224f4564fea93572e8206bad1fe P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 16 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:52.917701 22663 raft_consensus.cc:493] T 02838c86531d4f98bf58d1c22b8baf77 P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:52.919530 22659 raft_consensus.cc:493] T 74ca8140f35c4ae29e8a664f6bdfc829 P cb82cc7bd6db46d980c75237a1402860 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:21:52.920282 22659 raft_consensus.cc:515] T 74ca8140f35c4ae29e8a664f6bdfc829 P cb82cc7bd6db46d980c75237a1402860 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:21:52.659543 22652 log.cc:927] Time spent T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 0.492s	user 0.000s	sys 0.002s
I20260709 13:21:52.923532 22645 raft_consensus.cc:493] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:52.923943 16225 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57600 (request call id 1895) took 1399 ms. Trace:
I20260709 13:21:52.924320 22645 raft_consensus.cc:515] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:52.924371 16225 rpcz_store.cc:276] 0709 13:21:51.524974 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:51.526925 (+  1951us) service_pool.cc:224] Handling call
0709 13:21:51.527262 (+   337us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:21:51.527793 (+   531us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:21:51.527824 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:21:51.528292 (+   468us) log.cc:844] Serialized 26 byte log entry
0709 13:21:51.752512 (+224220us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:21:51.752556 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:51.752627 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:21:52.923718 (+1171091us) raft_consensus.cc:1685] finished
0709 13:21:52.923737 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:21:52.923868 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":550,"raft.run_cpu_time_us":92,"raft.run_wall_time_us":93,"thread_start_us":223689,"threads_started":1,"wal-append.queue_time_us":320398}
W20260709 13:21:52.530961 22506 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.527s	user 0.001s	sys 0.012s
I20260709 13:21:52.644610 22682 raft_consensus.cc:493] T 5a96373bb0e14dce8a266b2fc6c58507 P 90f3da57796b488c9087a2f523adf045 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:52.927575 22682 raft_consensus.cc:515] T 5a96373bb0e14dce8a266b2fc6c58507 P 90f3da57796b488c9087a2f523adf045 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:52.930176 22682 leader_election.cc:290] T 5a96373bb0e14dce8a266b2fc6c58507 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:21:52.645705 22106 raft_consensus.cc:493] T 3884f864a96d428785895e5463e5728f P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:52.931149 22106 raft_consensus.cc:515] T 3884f864a96d428785895e5463e5728f P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:52.934402 22106 leader_election.cc:290] T 3884f864a96d428785895e5463e5728f P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:52.645866 22664 raft_consensus.cc:515] T ada5a004ba7c46819051e3ccec71ffdb P cb82cc7bd6db46d980c75237a1402860 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:52.650473 16222 raft_consensus.cc:3060] T ee9ed67006144797b1de6ea8c2013a68 P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:21:52.646548 16294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f5f89186204242bf377f7c42b2e2da" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:52.937896 16294 raft_consensus.cc:2468] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 18.
I20260709 13:21:52.653313 16823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b7d1ab519044eda2d5e04a07242f25" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:21:52.653816 16825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada5a004ba7c46819051e3ccec71ffdb" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:52.939723 16825 raft_consensus.cc:2393] T ada5a004ba7c46819051e3ccec71ffdb P cb82cc7bd6db46d980c75237a1402860 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 19: Already voted for candidate cb82cc7bd6db46d980c75237a1402860 in this term.
I20260709 13:21:52.652215 16818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746aabf856ab401c9ef5e859371e7b67" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:52.649031 22669 raft_consensus.cc:3060] T 26354361e9b044cea1c2b86c4882fe92 P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:21:52.651134 16115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 57370.3us
I20260709 13:21:52.645220 22615 raft_consensus.cc:493] T 23f82224f4564fea93572e8206bad1fe P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:52.840317 22681 raft_consensus.cc:697] T ee9ed67006144797b1de6ea8c2013a68 P cb82cc7bd6db46d980c75237a1402860 [term 19 LEADER]: Becoming Leader. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:21:52.906641 16226 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57604 (request call id 3221) took 1231 ms. Trace:
I20260709 13:21:52.871151 22554 raft_consensus.cc:493] T e4dc6b0541ca431ebc757beac00521f1 P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:52.949091 16115 leader_election.cc:304] T 84f2da98864b4425bf7c0055951fdd4b P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac
I20260709 13:21:52.940418 16818 raft_consensus.cc:2468] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 14.
I20260709 13:21:52.844347 22675 raft_consensus.cc:493] T 1df93779209748b9860f6d893dd3de53 P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:21:52.885401 22526 raft_consensus.cc:697] T 8a878999c0024d2abe517484180290fc P 1ca2725d290e47d68b1a4b609c9816ac [term 15 LEADER]: Becoming Leader. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:21:52.918447 22663 raft_consensus.cc:515] T 02838c86531d4f98bf58d1c22b8baf77 P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:52.847543 22404 raft_consensus.cc:493] T 9e65676b21b748728ad39668cfe722ec P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:21:52.949757 16226 rpcz_store.cc:276] 0709 13:21:51.674884 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:52.309029 (+634145us) service_pool.cc:224] Handling call
0709 13:21:52.309401 (+   372us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:21:52.376000 (+ 66599us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:21:52.376059 (+    59us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:21:52.380346 (+  4287us) log.cc:844] Serialized 26 byte log entry
0709 13:21:52.526878 (+146532us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:21:52.526930 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:52.527000 (+    70us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:21:52.906418 (+379418us) raft_consensus.cc:1685] finished
0709 13:21:52.906437 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:21:52.906580 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":66141,"raft.queue_time_us":68661,"raft.run_cpu_time_us":183,"raft.run_wall_time_us":9128,"thread_start_us":105900,"threads_started":1,"wal-append.queue_time_us":502487}
I20260709 13:21:52.969422 16115 leader_election.cc:304] T c0eab25ce4c64e018a041beeda2867db P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:21:52.646045 22679 raft_consensus.cc:515] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:52.941133 22659 leader_election.cc:290] T 74ca8140f35c4ae29e8a664f6bdfc829 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:52.978078 16222 raft_consensus.cc:2468] T ee9ed67006144797b1de6ea8c2013a68 P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 19.
I20260709 13:21:52.946022 22645 leader_election.cc:290] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:21:52.878417 22285 raft_consensus.cc:515] T 8c11074579b046d99e585c2818e18a2c P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:52.953420 22693 raft_consensus.cc:493] T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:21:52.973042 22348 leader_election.cc:290] T 7c2095a5f4ad49408dbc7f69c606120d P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:52.946420 22399 raft_consensus.cc:515] T d16dfad0c3b540e5b4b86492ddf87208 P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:52.967355 22496 leader_election.cc:290] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:52.979324 22610 raft_consensus.cc:2804] T 84f2da98864b4425bf7c0055951fdd4b P 647a343a542d4a139fe106c19095129b [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:21:52.986058 22662 raft_consensus.cc:2804] T c0eab25ce4c64e018a041beeda2867db P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:21:52.989861 22664 leader_election.cc:290] T ada5a004ba7c46819051e3ccec71ffdb P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 19 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:52.950893 22615 raft_consensus.cc:515] T 23f82224f4564fea93572e8206bad1fe P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:52.975584 22404 raft_consensus.cc:515] T 9e65676b21b748728ad39668cfe722ec P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:52.955142 22554 raft_consensus.cc:515] T e4dc6b0541ca431ebc757beac00521f1 P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:21:53.023485 16788 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:53.011054 22662 raft_consensus.cc:493] T c0eab25ce4c64e018a041beeda2867db P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:53.007155 22610 raft_consensus.cc:493] T 84f2da98864b4425bf7c0055951fdd4b P 647a343a542d4a139fe106c19095129b [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:53.031401 16115 leader_election.cc:304] T 9c61a39fd4f84424b129c1a6f9ba404d P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:21:53.032524 16115 leader_election.cc:304] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:21:53.044472 22699 consensus_queue.cc:1048] T 3884f864a96d428785895e5463e5728f P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:21:53.045971 22699 raft_consensus.cc:2804] T 9c61a39fd4f84424b129c1a6f9ba404d P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:21:53.046331 22699 raft_consensus.cc:493] T 9c61a39fd4f84424b129c1a6f9ba404d P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:21:53.046501 22590 consensus_meta.cc:287] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860: Time spent flushing consensus metadata: real 0.762s	user 0.007s	sys 0.003s
I20260709 13:21:53.047129 22590 raft_consensus.cc:515] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:53.049340 22590 leader_election.cc:290] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 21 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
W20260709 13:21:53.050756 22696 log.cc:927] Time spent T ecc1e363230043fe9fef390f0b979f20 P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 0.054s	user 0.001s	sys 0.000s
I20260709 13:21:53.052243 22663 leader_election.cc:290] T 02838c86531d4f98bf58d1c22b8baf77 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:21:53.017261 22681 consensus_queue.cc:237] T ee9ed67006144797b1de6ea8c2013a68 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.14, Last appended by leader: 14, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:21:53.053517 22196 consensus_meta.cc:287] T 7c2095a5f4ad49408dbc7f69c606120d P 647a343a542d4a139fe106c19095129b: Time spent flushing consensus metadata: real 1.401s	user 0.010s	sys 0.002s
I20260709 13:21:53.054381 22196 raft_consensus.cc:2804] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:21:53.046777 22699 raft_consensus.cc:3060] T 9c61a39fd4f84424b129c1a6f9ba404d P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:21:53.032333 22610 raft_consensus.cc:3060] T 84f2da98864b4425bf7c0055951fdd4b P 647a343a542d4a139fe106c19095129b [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:21:53.057420 16114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:21:53.022817 22670 consensus_meta.cc:287] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b: Time spent flushing consensus metadata: real 0.710s	user 0.014s	sys 0.000s
W20260709 13:21:53.061892 16192 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 185423us
I20260709 13:21:53.015069 22553 leader_election.cc:290] T c9944d5e4b8c4109af9955c4f08c9095 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 20 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:52.969560 22675 raft_consensus.cc:515] T 1df93779209748b9860f6d893dd3de53 P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:21:53.092206 16147 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40844 (request call id 2844) took 1881 ms (client timeout 2137 ms). Trace:
W20260709 13:21:53.106410 16147 rpcz_store.cc:269] 0709 13:21:51.211088 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:51.229829 (+ 18741us) service_pool.cc:224] Handling call
0709 13:21:53.091979 (+1862150us) spinlock_profiling.cc:243] Waited 145 ms on lock 0x7b540028f354. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:21:53.092125 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":333044992}
W20260709 13:21:53.063177 16220 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57604 (request call id 3214) took 1518 ms (client timeout 1837 ms). Trace:
W20260709 13:21:53.104725 16114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:53.054909 22196 raft_consensus.cc:493] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:21:53.100696 16790 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:53.108688 22196 raft_consensus.cc:3060] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:21:53.110567 16147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f82224f4564fea93572e8206bad1fe" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:21:53.111438 16147 raft_consensus.cc:3060] T 23f82224f4564fea93572e8206bad1fe P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:21:53.025921 22564 raft_consensus.cc:493] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:21:53.115512 16220 rpcz_store.cc:269] 0709 13:21:51.545225 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:51.559588 (+ 14363us) service_pool.cc:224] Handling call
0709 13:21:53.062944 (+1503356us) spinlock_profiling.cc:243] Waited 686 ms on lock 0x7b5400139554. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:21:53.063055 (+   111us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1578486656}
I20260709 13:21:53.033345 22662 raft_consensus.cc:3060] T c0eab25ce4c64e018a041beeda2867db P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:21:53.117944 16115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 85491us
I20260709 13:21:53.001365 22693 raft_consensus.cc:515] T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:53.128646 22399 leader_election.cc:290] T d16dfad0c3b540e5b4b86492ddf87208 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:21:53.129076 22404 leader_election.cc:290] T 9e65676b21b748728ad39668cfe722ec P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:53.094050 22584 raft_consensus.cc:3055] T 5a959dbecc2648eb85d1cc550f5a3245 P 90f3da57796b488c9087a2f523adf045 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:21:53.129835 22584 raft_consensus.cc:740] T 5a959dbecc2648eb85d1cc550f5a3245 P 90f3da57796b488c9087a2f523adf045 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:21:53.130455 22693 leader_election.cc:290] T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:53.130779 22584 consensus_queue.cc:260] T 5a959dbecc2648eb85d1cc550f5a3245 P 90f3da57796b488c9087a2f523adf045 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:53.131927 22584 raft_consensus.cc:3060] T 5a959dbecc2648eb85d1cc550f5a3245 P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:21:53.132108 22285 leader_election.cc:290] T 8c11074579b046d99e585c2818e18a2c P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:53.127839 22564 raft_consensus.cc:515] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:53.128373 22673 leader_election.cc:290] T 3884f864a96d428785895e5463e5728f P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:53.061206 22520 raft_consensus.cc:493] T 7c2095a5f4ad49408dbc7f69c606120d P 647a343a542d4a139fe106c19095129b [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:53.138250 16143 raft_consensus.cc:1275] T 57a744f5aaf944678a63469cbf514015 P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 13:21:53.168366 16220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74ca8140f35c4ae29e8a664f6bdfc829" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
W20260709 13:21:53.143397 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 492711us
I20260709 13:21:53.142500 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a878999c0024d2abe517484180290fc" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:53.169605 16299 raft_consensus.cc:2468] T 8a878999c0024d2abe517484180290fc P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 15.
I20260709 13:21:53.145731 16147 raft_consensus.cc:2468] T 23f82224f4564fea93572e8206bad1fe P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 16.
I20260709 13:21:53.149091 22709 raft_consensus.cc:493] T 68a6838d82ef4cac83507f906139d90f P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:21:53.149827 16295 raft_consensus.cc:1275] T 434bff3082d141838068f323bdb0d1ed P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 16 index: 14. (index mismatch)
I20260709 13:21:53.166628 16222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada5a004ba7c46819051e3ccec71ffdb" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:21:53.170892 16115 leader_election.cc:304] T 8a878999c0024d2abe517484180290fc P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:21:53.172658 16147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3884f864a96d428785895e5463e5728f" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:53.206300 16222 raft_consensus.cc:3060] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:21:53.184975 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c61a39fd4f84424b129c1a6f9ba404d" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:21:53.211275 16148 raft_consensus.cc:1275] T e5ad4eb281564936a59f41e387a47156 P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 13:21:53.214365 16148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57a744f5aaf944678a63469cbf514015" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:53.215205 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdbd78acb4cf46298e0de8ba0d130e19" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:21:53.215358 16148 raft_consensus.cc:2468] T 57a744f5aaf944678a63469cbf514015 P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 13.
I20260709 13:21:53.215790 16225 raft_consensus.cc:3060] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:21:53.217306 16148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f2da98864b4425bf7c0055951fdd4b" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:53.224840 16220 raft_consensus.cc:1275] T 08c74490c40243a88e62d1760b5cd814 P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 16 index: 10. (term mismatch)
I20260709 13:21:53.225274 16220 pending_rounds.cc:85] T 08c74490c40243a88e62d1760b5cd814 P 1ca2725d290e47d68b1a4b609c9816ac: Aborting all ops after (but not including) 9
I20260709 13:21:53.225602 16220 pending_rounds.cc:107] T 08c74490c40243a88e62d1760b5cd814 P 1ca2725d290e47d68b1a4b609c9816ac: Aborting uncommitted NO_OP operation due to leader change: 15.10
I20260709 13:21:53.226132 16220 raft_consensus.cc:2889] T 08c74490c40243a88e62d1760b5cd814 P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:21:53.211608 16226 raft_consensus.cc:2376] T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 18. Re-sending same reply.
I20260709 13:21:53.228335 16147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e9b2bf7011f4ae688a79d7bb5a447b4" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 17 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:21:53.229346 16147 raft_consensus.cc:3060] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:21:53.219029 16790 leader_election.cc:304] T 23f82224f4564fea93572e8206bad1fe P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860; no voters: 
W20260709 13:21:53.231498 16788 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:21:53.205170 22197 consensus_meta.cc:287] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b: Time spent flushing consensus metadata: real 1.148s	user 0.013s	sys 0.000s
I20260709 13:21:53.237360 16220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a96373bb0e14dce8a266b2fc6c58507" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:53.242291 16220 raft_consensus.cc:2468] T 5a96373bb0e14dce8a266b2fc6c58507 P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 14.
I20260709 13:21:53.241339 16149 raft_consensus.cc:1240] T 5a959dbecc2648eb85d1cc550f5a3245 P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 15. Current term is 16. Ops: []
I20260709 13:21:53.169046 22615 leader_election.cc:290] T 23f82224f4564fea93572e8206bad1fe P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:53.234463 22695 raft_consensus.cc:493] T 57a744f5aaf944678a63469cbf514015 P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:53.063426 22526 consensus_queue.cc:237] T 8a878999c0024d2abe517484180290fc P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:53.185914 22404 raft_consensus.cc:493] T 8638ee363273426db1393dd81e5c7b72 P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:53.224864 22517 raft_consensus.cc:2804] T 8a878999c0024d2abe517484180290fc P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:21:53.228701 22645 raft_consensus.cc:493] T b87147731a68400e8cce9b879daa7c5d P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:53.263451 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8158bbd17b994d2eb96f560bd0f1b88d" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:53.284039 16226 raft_consensus.cc:2468] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 16.
I20260709 13:21:53.265930 16220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87ac0d1f7e8a419f9bcbc04aee0e067e" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:53.285511 16220 raft_consensus.cc:2468] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 18.
W20260709 13:21:53.267894 16264 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:53.286010 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4f65cdf00b748bb963ee6e81ef43b79" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:53.268289 16223 raft_consensus.cc:1275] T e5ad4eb281564936a59f41e387a47156 P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 13:21:53.286780 16149 raft_consensus.cc:2393] T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 19: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
W20260709 13:21:53.271721 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 52764us
I20260709 13:21:53.247574 16217 raft_consensus.cc:1275] T fffab26427e24d398e2695aad40ca134 P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 13:21:53.209342 22399 raft_consensus.cc:493] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:21:53.290437 16222 raft_consensus.cc:2468] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 19.
I20260709 13:21:53.286414 16147 raft_consensus.cc:2410] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 for term 17 because replica has last-logged OpId of term: 16 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 11.
I20260709 13:21:53.209862 22663 raft_consensus.cc:493] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:53.186487 22675 leader_election.cc:290] T 1df93779209748b9860f6d893dd3de53 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:21:53.168892 22677 raft_consensus.cc:515] T 02838c86531d4f98bf58d1c22b8baf77 P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:53.295042 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26354361e9b044cea1c2b86c4882fe92" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:53.174250 22520 raft_consensus.cc:515] T 7c2095a5f4ad49408dbc7f69c606120d P 647a343a542d4a139fe106c19095129b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:53.262727 22615 raft_consensus.cc:493] T 2794af5b99984efebdee564035841cb0 P 90f3da57796b488c9087a2f523adf045 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:53.341782 22663 raft_consensus.cc:515] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } }
I20260709 13:21:53.342708 22615 raft_consensus.cc:515] T 2794af5b99984efebdee564035841cb0 P 90f3da57796b488c9087a2f523adf045 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:53.320734 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9944d5e4b8c4109af9955c4f08c9095" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:21:53.319517 16222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02838c86531d4f98bf58d1c22b8baf77" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:53.344098 16225 raft_consensus.cc:2468] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 21.
I20260709 13:21:53.349063 16147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434bff3082d141838068f323bdb0d1ed" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:53.274456 22517 raft_consensus.cc:493] T 8a878999c0024d2abe517484180290fc P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:53.351133 16222 raft_consensus.cc:2468] T 02838c86531d4f98bf58d1c22b8baf77 P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 18.
I20260709 13:21:53.175144 22709 raft_consensus.cc:515] T 68a6838d82ef4cac83507f906139d90f P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:53.350292 22706 raft_consensus.cc:493] T 8c11074579b046d99e585c2818e18a2c P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:53.352542 22541 consensus_queue.cc:1048] T 57a744f5aaf944678a63469cbf514015 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.126s
I20260709 13:21:53.352761 22706 raft_consensus.cc:515] T 8c11074579b046d99e585c2818e18a2c P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:53.353377 22677 leader_election.cc:290] T 02838c86531d4f98bf58d1c22b8baf77 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 18 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:53.354203 16292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434bff3082d141838068f323bdb0d1ed" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:53.354526 22702 raft_consensus.cc:493] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:21:53.354724 22703 raft_consensus.cc:2804] T 23f82224f4564fea93572e8206bad1fe P cb82cc7bd6db46d980c75237a1402860 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:21:53.350219 22708 consensus_queue.cc:1048] T 434bff3082d141838068f323bdb0d1ed P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.118s
I20260709 13:21:53.354924 16292 raft_consensus.cc:2468] T 434bff3082d141838068f323bdb0d1ed P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 16.
I20260709 13:21:53.355137 22710 raft_consensus.cc:493] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:53.355581 16292 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40474 (request call id 1143) took 1841 ms. Trace:
I20260709 13:21:53.355973 16292 rpcz_store.cc:276] 0709 13:21:51.514290 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:53.347800 (+1833510us) service_pool.cc:224] Handling call
0709 13:21:53.355520 (+  7720us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:21:53.355769 22710 raft_consensus.cc:515] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:53.356375 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d16dfad0c3b540e5b4b86492ddf87208" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:53.260545 22197 raft_consensus.cc:515] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } }
I20260709 13:21:53.291816 22645 raft_consensus.cc:515] T b87147731a68400e8cce9b879daa7c5d P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:53.357288 16292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7ca61592e848ea9b0fb81a2ce33cdc" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:53.357522 16225 raft_consensus.cc:2468] T d16dfad0c3b540e5b4b86492ddf87208 P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 18.
I20260709 13:21:53.359200 22197 leader_election.cc:290] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 18 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:53.269093 22695 raft_consensus.cc:515] T 57a744f5aaf944678a63469cbf514015 P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:53.368686 16292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ad5dbfa99ee4b8d9e28be92b3f36db5" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:21:53.369935 22695 leader_election.cc:290] T 57a744f5aaf944678a63469cbf514015 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:53.317233 22399 raft_consensus.cc:515] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:53.371727 16292 raft_consensus.cc:3060] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:21:53.372496 22399 leader_election.cc:290] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:21:53.264994 22699 raft_consensus.cc:515] T 9c61a39fd4f84424b129c1a6f9ba404d P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:53.373374 22564 leader_election.cc:290] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:53.345679 22663 leader_election.cc:290] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:21:53.377650 16144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8158bbd17b994d2eb96f560bd0f1b88d" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:53.319289 22679 leader_election.cc:290] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:53.276989 22404 raft_consensus.cc:515] T 8638ee363273426db1393dd81e5c7b72 P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:53.425355 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1df93779209748b9860f6d893dd3de53" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:53.454002 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806d7690c2b644959152074bb2cab0a7" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 19 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:53.454182 16225 raft_consensus.cc:2468] T 1df93779209748b9860f6d893dd3de53 P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 16.
W20260709 13:21:53.437979 16114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:53.454372 16144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a96373bb0e14dce8a266b2fc6c58507" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:53.454911 16226 raft_consensus.cc:2410] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 for term 19 because replica has last-logged OpId of term: 17 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 10.
I20260709 13:21:53.420584 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87ac0d1f7e8a419f9bcbc04aee0e067e" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:53.464984 16292 raft_consensus.cc:2468] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 17.
I20260709 13:21:53.459972 16150 raft_consensus.cc:2468] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 18.
I20260709 13:21:53.351976 22517 raft_consensus.cc:3060] T 8a878999c0024d2abe517484180290fc P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:21:53.355273 22711 raft_consensus.cc:493] T ecc1e363230043fe9fef390f0b979f20 P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
W20260709 13:21:53.494397 22669 consensus_meta.cc:287] T 26354361e9b044cea1c2b86c4882fe92 P 647a343a542d4a139fe106c19095129b: Time spent flushing consensus metadata: real 0.540s	user 0.009s	sys 0.000s
I20260709 13:21:53.499871 22662 raft_consensus.cc:515] T c0eab25ce4c64e018a041beeda2867db P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:21:53.502573 22644 log.cc:927] Time spent T c0eab25ce4c64e018a041beeda2867db P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 1.634s	user 0.004s	sys 0.000s
I20260709 13:21:53.505195 22404 leader_election.cc:290] T 8638ee363273426db1393dd81e5c7b72 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:53.366967 22610 raft_consensus.cc:515] T 84f2da98864b4425bf7c0055951fdd4b P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:53.417657 22710 leader_election.cc:290] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:53.416293 22702 raft_consensus.cc:515] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:53.512765 22573 consensus_queue.cc:260] T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:21:53.513899 16114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:53.515940 16150 raft_consensus.cc:1275] T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:21:53.550209 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f82224f4564fea93572e8206bad1fe" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:53.551003 16150 raft_consensus.cc:2393] T 23f82224f4564fea93572e8206bad1fe P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 16: Already voted for candidate cb82cc7bd6db46d980c75237a1402860 in this term.
I20260709 13:21:53.517166 22588 raft_consensus.cc:3060] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:21:53.518812 22689 log.cc:927] Time spent T b87147731a68400e8cce9b879daa7c5d P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 0.639s	user 0.000s	sys 0.000s
I20260709 13:21:53.518919 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0eab25ce4c64e018a041beeda2867db" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:21:53.553187 16297 raft_consensus.cc:2393] T c0eab25ce4c64e018a041beeda2867db P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 17: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
I20260709 13:21:53.519492 16145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02838c86531d4f98bf58d1c22b8baf77" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:53.553449 16148 raft_consensus.cc:2393] T 84f2da98864b4425bf7c0055951fdd4b P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 15: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
I20260709 13:21:53.553709 16145 raft_consensus.cc:2468] T 02838c86531d4f98bf58d1c22b8baf77 P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 18.
I20260709 13:21:53.554416 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e65676b21b748728ad39668cfe722ec" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:53.554572 16145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4dc6b0541ca431ebc757beac00521f1" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 18 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:53.522903 16818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7ca61592e848ea9b0fb81a2ce33cdc" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:53.523589 16824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746aabf856ab401c9ef5e859371e7b67" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:21:53.523717 16825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2372981006d4b218215c28964dea623" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:53.524451 16816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c11074579b046d99e585c2818e18a2c" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 19 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:53.525163 16820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746aabf856ab401c9ef5e859371e7b67" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:53.525846 16821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66ae176961b74813a58bc15c9c4c6f2a" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:21:53.487664 22682 raft_consensus.cc:493] T 8d4ab870a05347868956aa8b841b2981 P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:53.529270 22615 leader_election.cc:290] T 2794af5b99984efebdee564035841cb0 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:53.531258 16822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b7d1ab519044eda2d5e04a07242f25" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
W20260709 13:21:53.532202 16790 leader_election.cc:336] T 19baa7bc367844c288bca8f81439c413 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 20 election: RPC error from VoteRequest() call to peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943): Timed out: RequestConsensusVote RPC to 127.14.177.130:35943 timed out after 1.827s (SENT)
I20260709 13:21:53.458745 22543 raft_consensus.cc:3060] T 2794af5b99984efebdee564035841cb0 P cb82cc7bd6db46d980c75237a1402860 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:21:53.467666 16144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3884f864a96d428785895e5463e5728f" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:53.563709 16790 leader_election.cc:304] T 19baa7bc367844c288bca8f81439c413 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:21:53.382251 22554 leader_election.cc:290] T e4dc6b0541ca431ebc757beac00521f1 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:21:53.403596 22680 raft_consensus.cc:493] T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:53.381402 22703 raft_consensus.cc:697] T 23f82224f4564fea93572e8206bad1fe P cb82cc7bd6db46d980c75237a1402860 [term 16 LEADER]: Becoming Leader. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:21:53.544775 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:53.468511 22699 leader_election.cc:290] T 9c61a39fd4f84424b129c1a6f9ba404d P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 19 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:53.542804 22706 leader_election.cc:290] T 8c11074579b046d99e585c2818e18a2c P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:53.555900 16824 raft_consensus.cc:3060] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:21:53.556143 16825 raft_consensus.cc:2468] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 19.
I20260709 13:21:53.555708 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1df93779209748b9860f6d893dd3de53" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:53.556800 16821 raft_consensus.cc:2376] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [term 23 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 23. Re-sending same reply.
I20260709 13:21:53.561661 16145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806d7690c2b644959152074bb2cab0a7" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 19 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:53.655526 16145 raft_consensus.cc:2410] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 for term 19 because replica has last-logged OpId of term: 17 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 10.
I20260709 13:21:53.543792 22669 raft_consensus.cc:2749] T 26354361e9b044cea1c2b86c4882fe92 P 647a343a542d4a139fe106c19095129b [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:21:53.589268 16192 leader_election.cc:304] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
W20260709 13:21:53.593194 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 61009.3us
W20260709 13:21:53.712693 16790 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:21:53.712944 16192 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 123747us
I20260709 13:21:53.713615 16790 leader_election.cc:304] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:21:53.714202 16192 leader_election.cc:304] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:21:53.549798 22710 raft_consensus.cc:493] T e5ad4eb281564936a59f41e387a47156 P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:53.596879 16144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d16dfad0c3b540e5b4b86492ddf87208" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
W20260709 13:21:53.611181 16788 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:53.637036 16299 raft_consensus.cc:2468] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 20.
I20260709 13:21:53.728225 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada5a004ba7c46819051e3ccec71ffdb" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:53.729768 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806d7690c2b644959152074bb2cab0a7" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
W20260709 13:21:53.639386 16264 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:53.715127 16192 leader_election.cc:304] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 
W20260709 13:21:53.732312 16788 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:53.707034 22677 raft_consensus.cc:493] T d2ab0ac8347641e6955e9bce7d90c1a2 P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:21:53.736342 16299 raft_consensus.cc:2393] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 18: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
I20260709 13:21:53.743289 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57a744f5aaf944678a63469cbf514015" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:53.743992 16299 raft_consensus.cc:2468] T 57a744f5aaf944678a63469cbf514015 P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 13.
I20260709 13:21:53.745180 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f82224f4564fea93572e8206bad1fe" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:53.745950 16299 raft_consensus.cc:2468] T 23f82224f4564fea93572e8206bad1fe P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 15.
I20260709 13:21:53.739414 16142 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46224 (request call id 1051) took 1518 ms. Trace:
I20260709 13:21:53.748584 16142 rpcz_store.cc:276] 0709 13:21:52.221092 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:52.227599 (+  6507us) service_pool.cc:224] Handling call
0709 13:21:52.256020 (+ 28421us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:21:52.262042 (+  6022us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:21:52.262080 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:21:52.264602 (+  2522us) log.cc:844] Serialized 26 byte log entry
0709 13:21:52.499857 (+235255us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:21:52.499963 (+   106us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:52.500329 (+   366us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:21:53.518259 (+1017930us) spinlock_profiling.cc:243] Waited 37.2 ms on lock 0x7b4400022b4c. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69ad8fadb1 00007f69ad998c85 00007f69ad9996a2 00007f69b070b1db 00007f69b0706aaf 00007f69b06fa86e 00007f69b06f9bb6 00007f69b06f97d8 00007f69b0703221 00007f69b07031d9 00007f69b0703169
0709 13:21:53.738944 (+220685us) raft_consensus.cc:1685] finished
0709 13:21:53.738980 (+    36us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:21:53.739312 (+   332us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":29612,"spinlock_wait_cycles":99583104,"thread_start_us":174996,"threads_started":1,"wal-append.queue_time_us":206166}
I20260709 13:21:53.726761 16144 raft_consensus.cc:2468] T d16dfad0c3b540e5b4b86492ddf87208 P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 18.
I20260709 13:21:53.755695 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18ad7e3af5504571a69d10c00750a01c" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:21:53.649876 22504 raft_consensus.cc:2749] T 19baa7bc367844c288bca8f81439c413 P cb82cc7bd6db46d980c75237a1402860 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
W20260709 13:21:53.763075 16264 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:53.767956 16149 raft_consensus.cc:2468] T 26354361e9b044cea1c2b86c4882fe92 P 647a343a542d4a139fe106c19095129b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 14.
I20260709 13:21:53.761236 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02838c86531d4f98bf58d1c22b8baf77" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:53.769110 16297 raft_consensus.cc:2393] T 02838c86531d4f98bf58d1c22b8baf77 P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 18: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
I20260709 13:21:53.770097 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a9de347c711477e87c582d003f36925" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:21:53.770443 22707 raft_consensus.cc:493] T 08c74490c40243a88e62d1760b5cd814 P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:53.770663 16297 raft_consensus.cc:3060] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:21:53.774586 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdbd78acb4cf46298e0de8ba0d130e19" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:53.775837 22669 raft_consensus.cc:493] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:53.779685 16790 leader_election.cc:304] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 647a343a542d4a139fe106c19095129b
I20260709 13:21:53.694063 22573 raft_consensus.cc:3060] T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:21:53.707247 22699 raft_consensus.cc:493] T 8d4ab870a05347868956aa8b841b2981 P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:53.738317 22348 raft_consensus.cc:2749] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
W20260709 13:21:53.767618 22584 consensus_meta.cc:287] T 5a959dbecc2648eb85d1cc550f5a3245 P 90f3da57796b488c9087a2f523adf045: Time spent flushing consensus metadata: real 0.635s	user 0.004s	sys 0.008s
I20260709 13:21:53.565819 22682 raft_consensus.cc:515] T 8d4ab870a05347868956aa8b841b2981 P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:53.747892 22496 raft_consensus.cc:2804] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:21:53.784508 16299 raft_consensus.cc:2468] T cdbd78acb4cf46298e0de8ba0d130e19 P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 19.
W20260709 13:21:53.787408 22196 consensus_meta.cc:287] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b: Time spent flushing consensus metadata: real 0.657s	user 0.009s	sys 0.001s
I20260709 13:21:53.808733 16824 raft_consensus.cc:2468] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 15.
W20260709 13:21:53.811751 16791 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:53.811365 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87ac0d1f7e8a419f9bcbc04aee0e067e" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:53.820825 16298 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48674 (request call id 3245) took 3958 ms. Trace:
I20260709 13:21:53.826498 16297 raft_consensus.cc:2468] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 21.
I20260709 13:21:53.751858 22710 raft_consensus.cc:515] T e5ad4eb281564936a59f41e387a47156 P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:53.648593 22711 raft_consensus.cc:515] T ecc1e363230043fe9fef390f0b979f20 P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:53.652172 22680 raft_consensus.cc:515] T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:53.807586 22504 raft_consensus.cc:2804] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:21:53.807729 22496 raft_consensus.cc:493] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:53.768383 22677 raft_consensus.cc:515] T d2ab0ac8347641e6955e9bce7d90c1a2 P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:53.816555 22568 raft_consensus.cc:2804] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:21:53.776469 22669 raft_consensus.cc:515] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:53.771004 22707 raft_consensus.cc:515] T 08c74490c40243a88e62d1760b5cd814 P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:53.815600 22597 raft_consensus.cc:2749] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860 [term 15 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:21:53.815326 22662 leader_election.cc:290] T c0eab25ce4c64e018a041beeda2867db P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 18 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:53.804369 22699 raft_consensus.cc:515] T 8d4ab870a05347868956aa8b841b2981 P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:53.828329 22584 raft_consensus.cc:493] T ecc1e363230043fe9fef390f0b979f20 P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:21:53.845067 22439 raft_consensus.cc:493] T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:21:53.848395 22682 leader_election.cc:290] T 8d4ab870a05347868956aa8b841b2981 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:21:53.816409 16299 raft_consensus.cc:2410] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b for term 19 because replica has last-logged OpId of term: 17 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 11.
I20260709 13:21:53.867376 16298 rpcz_store.cc:276] 0709 13:21:49.862812 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:50.099979 (+237167us) service_pool.cc:224] Handling call
0709 13:21:51.515935 (+1415956us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:21:51.527099 (+ 11164us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:53.818275 (+2291176us) spinlock_profiling.cc:243] Waited 1.42 s on lock 0x7b54000aa350. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5e8ad 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:21:53.820743 (+  2468us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3255520640}
I20260709 13:21:53.819490 22196 raft_consensus.cc:515] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:53.875190 22496 raft_consensus.cc:3060] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:21:53.875715 16266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 312073us
W20260709 13:21:53.887503 16114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:53.875121 22504 raft_consensus.cc:493] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:21:53.880239 22610 leader_election.cc:290] T 84f2da98864b4425bf7c0055951fdd4b P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 15 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:53.880534 22568 raft_consensus.cc:697] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [term 23 LEADER]: Becoming Leader. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:21:53.889982 22439 raft_consensus.cc:3060] T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:21:53.895027 16299 raft_consensus.cc:1240] T 8d4ab870a05347868956aa8b841b2981 P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 15. Current term is 16. Ops: []
I20260709 13:21:53.839372 22703 consensus_queue.cc:237] T 23f82224f4564fea93572e8206bad1fe P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:21:53.906698 16191 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:21:53.906934 22724 log.cc:927] Time spent T 57a744f5aaf944678a63469cbf514015 P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 0.064s	user 0.001s	sys 0.000s
I20260709 13:21:53.912235 22710 leader_election.cc:290] T e5ad4eb281564936a59f41e387a47156 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:53.911725 22568 consensus_queue.cc:237] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 21.14, Last appended by leader: 14, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:53.914027 22680 leader_election.cc:290] T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:53.890673 22584 raft_consensus.cc:515] T ecc1e363230043fe9fef390f0b979f20 P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:53.915388 22702 leader_election.cc:290] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
W20260709 13:21:53.924155 16192 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 207795us
W20260709 13:21:53.920578 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 141849us
I20260709 13:21:53.893637 22504 raft_consensus.cc:3060] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:21:53.922576 16299 raft_consensus.cc:1275] T 84f2da98864b4425bf7c0055951fdd4b P 90f3da57796b488c9087a2f523adf045 [term 14 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 13:21:53.927731 16299 raft_consensus.cc:3060] T c4f65cdf00b748bb963ee6e81ef43b79 P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:21:53.925642 16192 leader_election.cc:304] T 746aabf856ab401c9ef5e859371e7b67 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:21:53.923101 22593 raft_consensus.cc:493] T a378d7f796f94615b3b347119a9880fd P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:53.929551 22537 raft_consensus.cc:493] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:53.930943 22688 consensus_queue.cc:1048] T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.020s
I20260709 13:21:53.942375 22731 raft_consensus.cc:2804] T 746aabf856ab401c9ef5e859371e7b67 P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:21:53.947906 22669 leader_election.cc:290] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:53.944043 22593 raft_consensus.cc:515] T a378d7f796f94615b3b347119a9880fd P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:53.990864 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada5a004ba7c46819051e3ccec71ffdb" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:21:53.993786 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdbd78acb4cf46298e0de8ba0d130e19" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:21:53.994905 22707 leader_election.cc:290] T 08c74490c40243a88e62d1760b5cd814 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:53.950851 22537 raft_consensus.cc:515] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:53.997107 16219 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33050 (request call id 1123) took 1774 ms. Trace:
W20260709 13:21:53.997272 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 66780.3us
I20260709 13:21:54.001717 16821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:54.001266 16219 rpcz_store.cc:276] 0709 13:21:52.222392 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:52.321305 (+ 98913us) service_pool.cc:224] Handling call
0709 13:21:52.321701 (+   396us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:21:52.321946 (+   245us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 13:21:52.509982 (+188036us) log.cc:844] Serialized 14 byte log entry
0709 13:21:53.996527 (+1486545us) spinlock_profiling.cc:243] Waited 2.25 ms on lock 0x7b18002aa840. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b693ade1 00007f69b69475b1 00007f69b07074bc 00007f69ada5548b 00007f69b68afab9 00007f69ada7169b 00007f69ada6f93f 00007f69ada71c02 00007f69b06f920b 00007f69b06fed23 00007f69b06ff23a
0709 13:21:53.996727 (+   200us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:21:53.996778 (+    51us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:21:53.996835 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:53.996922 (+    87us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:21:53.997040 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":517088,"prepare.run_cpu_time_us":50,"prepare.run_wall_time_us":50,"raft.queue_time_us":94,"spinlock_wait_cycles":5166336,"thread_start_us":519572,"threads_started":2,"wal-append.queue_time_us":462264}
I20260709 13:21:54.002866 16825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ab0ac8347641e6955e9bce7d90c1a2" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:54.003021 16819 raft_consensus.cc:1240] T ada5a004ba7c46819051e3ccec71ffdb P cb82cc7bd6db46d980c75237a1402860 [term 19 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 18. Current term is 19. Ops: []
I20260709 13:21:53.998625 16298 raft_consensus.cc:3060] T cdbd78acb4cf46298e0de8ba0d130e19 P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Advancing to term 21
I20260709 13:21:54.005462 16825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdbd78acb4cf46298e0de8ba0d130e19" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:53.988348 22731 raft_consensus.cc:697] T 746aabf856ab401c9ef5e859371e7b67 P 1ca2725d290e47d68b1a4b609c9816ac [term 15 LEADER]: Becoming Leader. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:21:54.006213 16825 raft_consensus.cc:2368] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 for earlier term 20. Current term is 21.
I20260709 13:21:53.990334 22733 raft_consensus.cc:493] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:54.004681 16819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74ca8140f35c4ae29e8a664f6bdfc829" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:54.007565 16822 raft_consensus.cc:1275] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P cb82cc7bd6db46d980c75237a1402860 [term 18 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 13:21:54.011261 16817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57a744f5aaf944678a63469cbf514015" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:54.011396 16819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434bff3082d141838068f323bdb0d1ed" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:54.012318 16819 raft_consensus.cc:2468] T 434bff3082d141838068f323bdb0d1ed P cb82cc7bd6db46d980c75237a1402860 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 16.
I20260709 13:21:54.007562 16299 raft_consensus.cc:1275] T c4f65cdf00b748bb963ee6e81ef43b79 P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 13:21:54.012941 16818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e9b2bf7011f4ae688a79d7bb5a447b4" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 17 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:21:54.013983 16820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3884f864a96d428785895e5463e5728f" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:54.016443 16824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9944d5e4b8c4109af9955c4f08c9095" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:54.014464 16820 raft_consensus.cc:2468] T 3884f864a96d428785895e5463e5728f P cb82cc7bd6db46d980c75237a1402860 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 13.
I20260709 13:21:54.021063 16820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e65676b21b748728ad39668cfe722ec" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:54.024488 16821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26354361e9b044cea1c2b86c4882fe92" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:54.025036 16821 raft_consensus.cc:2468] T 26354361e9b044cea1c2b86c4882fe92 P cb82cc7bd6db46d980c75237a1402860 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 14.
I20260709 13:21:54.001053 22677 leader_election.cc:290] T d2ab0ac8347641e6955e9bce7d90c1a2 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:54.027022 16817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9944d5e4b8c4109af9955c4f08c9095" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:21:54.018533 22699 leader_election.cc:290] T 8d4ab870a05347868956aa8b841b2981 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
W20260709 13:21:54.038758 16190 connection.cc:256] RPC call timeout handler was delayed by -1.4097607630001221s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17020000000000002s for a response
W20260709 13:21:54.041329 16190 connection.cc:256] RPC call timeout handler was delayed by -1.3998607630001061s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1713s for a response
W20260709 13:21:54.041713 16190 connection.cc:256] RPC call timeout handler was delayed by -1.3368607630000042s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17830000000000001s for a response
W20260709 13:21:54.042043 16190 connection.cc:256] RPC call timeout handler was delayed by -1.3170607630001996s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1805s for a response
W20260709 13:21:54.042495 16190 connection.cc:256] RPC call timeout handler was delayed by -1.2045390400000997s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16010000000000002s for a response
W20260709 13:21:54.042814 16190 connection.cc:256] RPC call timeout handler was delayed by -1.0774391759998707s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1636s for a response
W20260709 13:21:54.043129 16190 connection.cc:256] RPC call timeout handler was delayed by -1.0254607630001829s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2129s for a response
W20260709 13:21:54.043454 16190 connection.cc:256] RPC call timeout handler was delayed by -1.0182607630001712s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2137s for a response
I20260709 13:21:54.044071 16190 leader_election.cc:304] T 1df93779209748b9860f6d893dd3de53 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
W20260709 13:21:54.045928 22726 log.cc:927] Time spent T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 0.119s	user 0.001s	sys 0.001s
I20260709 13:21:54.023957 22196 leader_election.cc:290] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 19 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:54.050946 16821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f82224f4564fea93572e8206bad1fe" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:54.009473 22733 raft_consensus.cc:515] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:54.032533 22348 raft_consensus.cc:493] T b6f07842d3aa47858df5665874c86f3a P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:21:54.067417 22699 raft_consensus.cc:493] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:54.040103 22526 raft_consensus.cc:493] T 08c74490c40243a88e62d1760b5cd814 P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:54.080897 16821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8638ee363273426db1393dd81e5c7b72" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
W20260709 13:21:54.082634 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 81109.4us
I20260709 13:21:54.084635 16817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2794af5b99984efebdee564035841cb0" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:54.084263 16820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c11074579b046d99e585c2818e18a2c" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:54.086431 16790 leader_election.cc:304] T ada5a004ba7c46819051e3ccec71ffdb P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:21:54.087394 22738 raft_consensus.cc:2804] T ada5a004ba7c46819051e3ccec71ffdb P cb82cc7bd6db46d980c75237a1402860 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:21:54.087538 16790 leader_election.cc:304] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:21:54.088171 22738 raft_consensus.cc:697] T ada5a004ba7c46819051e3ccec71ffdb P cb82cc7bd6db46d980c75237a1402860 [term 19 LEADER]: Becoming Leader. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:21:54.089920 16822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a9de347c711477e87c582d003f36925" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:54.092130 16822 raft_consensus.cc:2393] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 21: Already voted for candidate cb82cc7bd6db46d980c75237a1402860 in this term.
I20260709 13:21:54.094282 16822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f82224f4564fea93572e8206bad1fe" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:54.097661 16822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e9b2bf7011f4ae688a79d7bb5a447b4" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 17 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:21:54.102034 16821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
W20260709 13:21:54.101577 22543 consensus_meta.cc:287] T 2794af5b99984efebdee564035841cb0 P cb82cc7bd6db46d980c75237a1402860: Time spent flushing consensus metadata: real 0.501s	user 0.012s	sys 0.001s
I20260709 13:21:54.104784 22622 raft_consensus.cc:493] T 2794af5b99984efebdee564035841cb0 P cb82cc7bd6db46d980c75237a1402860 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:54.105266 22622 raft_consensus.cc:515] T 2794af5b99984efebdee564035841cb0 P cb82cc7bd6db46d980c75237a1402860 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:21:54.108238 22690 log.cc:927] Time spent T fffab26427e24d398e2695aad40ca134 P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 1.258s	user 0.004s	sys 0.000s
W20260709 13:21:54.110204 22683 log.cc:927] Time spent T 3a7ca61592e848ea9b0fb81a2ce33cdc P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 1.580s	user 0.002s	sys 0.000s
I20260709 13:21:54.084471 22731 consensus_queue.cc:237] T 746aabf856ab401c9ef5e859371e7b67 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:54.110836 22738 consensus_queue.cc:237] T ada5a004ba7c46819051e3ccec71ffdb P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:54.119232 22568 raft_consensus.cc:2749] T 1df93779209748b9860f6d893dd3de53 P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:21:54.122568 16115 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:54.088806 22699 raft_consensus.cc:515] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:54.125385 22699 leader_election.cc:290] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:54.126822 16817 raft_consensus.cc:2468] T 2794af5b99984efebdee564035841cb0 P cb82cc7bd6db46d980c75237a1402860 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 14.
I20260709 13:21:54.091439 22526 raft_consensus.cc:515] T 08c74490c40243a88e62d1760b5cd814 P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:54.134369 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2372981006d4b218215c28964dea623" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:54.135152 16149 raft_consensus.cc:2468] T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 20.
I20260709 13:21:54.135540 16143 raft_consensus.cc:1240] T c0eab25ce4c64e018a041beeda2867db P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 17. Current term is 18. Ops: []
I20260709 13:21:54.139145 16148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4dc6b0541ca431ebc757beac00521f1" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:21:54.139508 16146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87ac0d1f7e8a419f9bcbc04aee0e067e" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:54.139915 16144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c2095a5f4ad49408dbc7f69c606120d" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 15 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:54.140112 16146 raft_consensus.cc:2468] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 18.
I20260709 13:21:54.138779 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746aabf856ab401c9ef5e859371e7b67" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:21:54.140869 16149 raft_consensus.cc:3060] T 746aabf856ab401c9ef5e859371e7b67 P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Advancing to term 15
I20260709 13:21:54.142186 16148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02838c86531d4f98bf58d1c22b8baf77" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:21:54.145293 16145 raft_consensus.cc:1275] T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 13:21:54.140722 16144 raft_consensus.cc:2468] T 7c2095a5f4ad49408dbc7f69c606120d P 647a343a542d4a139fe106c19095129b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 14.
W20260709 13:21:54.103529 22728 log.cc:927] Time spent T 8cc959fb7f704fba9009c0595467d074 P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 0.171s	user 0.000s	sys 0.001s
I20260709 13:21:54.159044 16143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c74490c40243a88e62d1760b5cd814" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:54.169350 16143 raft_consensus.cc:2468] T 08c74490c40243a88e62d1760b5cd814 P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 16.
I20260709 13:21:54.160322 16115 leader_election.cc:304] T 57a744f5aaf944678a63469cbf514015 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
W20260709 13:21:54.165885 16264 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:54.168298 16146 raft_consensus.cc:1240] T 84f2da98864b4425bf7c0055951fdd4b P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 14. Current term is 15. Ops: [14.9-14.9]
I20260709 13:21:54.131909 22688 raft_consensus.cc:2804] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:21:54.095446 22711 leader_election.cc:290] T ecc1e363230043fe9fef390f0b979f20 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:54.085292 22348 raft_consensus.cc:515] T b6f07842d3aa47858df5665874c86f3a P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:54.178762 16817 raft_consensus.cc:1275] T 3884f864a96d428785895e5463e5728f P cb82cc7bd6db46d980c75237a1402860 [term 13 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 12 index: 12. Preceding OpId from leader: term: 13 index: 13. (index mismatch)
I20260709 13:21:54.179772 16149 raft_consensus.cc:2468] T 746aabf856ab401c9ef5e859371e7b67 P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 15.
I20260709 13:21:54.148837 22699 raft_consensus.cc:493] T fffab26427e24d398e2695aad40ca134 P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:21:54.188957 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 144974us
I20260709 13:21:54.190236 16190 leader_election.cc:304] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: cb82cc7bd6db46d980c75237a1402860
I20260709 13:21:54.193754 16296 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40474 (request call id 1142) took 2679 ms. Trace:
I20260709 13:21:54.194285 16296 rpcz_store.cc:276] 0709 13:21:51.514023 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:52.161009 (+646986us) service_pool.cc:224] Handling call
0709 13:21:52.187554 (+ 26545us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:21:52.187969 (+   415us) raft_consensus.cc:1534] Early marking committed up to index 14
0709 13:21:52.211850 (+ 23881us) log.cc:844] Serialized 14 byte log entry
0709 13:21:52.476890 (+265040us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:21:52.477746 (+   856us) log.cc:844] Serialized 26 byte log entry
0709 13:21:52.477949 (+   203us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:21:52.477981 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:52.478064 (+    83us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:21:54.109613 (+1631549us) spinlock_profiling.cc:243] Waited 404 ms on lock 0x7b4400022b4c. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69ad8fadb1 00007f69ad998c85 00007f69ad9996a2 00007f69b070b1db 00007f69b0706aaf 00007f69b06fa86e 00007f69b06f9bb6 00007f69b06f97d8 00007f69b0703221 00007f69b07031d9 00007f69b0703169
0709 13:21:54.188558 (+ 78945us) raft_consensus.cc:1685] finished
0709 13:21:54.193478 (+  4920us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:21:54.193680 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":78963,"prepare.run_cpu_time_us":48,"prepare.run_wall_time_us":52,"raft.queue_time_us":9434,"raft.run_cpu_time_us":274,"raft.run_wall_time_us":12026,"spinlock_wait_cycles":928508160,"thread_start_us":212467,"threads_started":1,"wal-append.queue_time_us":261920}
I20260709 13:21:54.200755 16821 raft_consensus.cc:1240] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860 [term 21 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 20. Current term is 21. Ops: []
I20260709 13:21:54.186071 22584 leader_election.cc:290] T ecc1e363230043fe9fef390f0b979f20 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
W20260709 13:21:54.224269 22517 consensus_meta.cc:287] T 8a878999c0024d2abe517484180290fc P 647a343a542d4a139fe106c19095129b: Time spent flushing consensus metadata: real 0.686s	user 0.007s	sys 0.003s
I20260709 13:21:54.229121 22520 leader_election.cc:290] T 7c2095a5f4ad49408dbc7f69c606120d P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:54.229307 22622 leader_election.cc:290] T 2794af5b99984efebdee564035841cb0 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:54.229321 22348 leader_election.cc:290] T b6f07842d3aa47858df5665874c86f3a P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:54.230296 16221 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57604 (request call id 3203) took 4065 ms. Trace:
I20260709 13:21:54.230859 16292 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48684 (request call id 3049) took 1006 ms. Trace:
I20260709 13:21:54.180675 22735 raft_consensus.cc:2804] T 57a744f5aaf944678a63469cbf514015 P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:21:54.188269 22526 leader_election.cc:290] T 08c74490c40243a88e62d1760b5cd814 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:54.180516 22645 leader_election.cc:290] T b87147731a68400e8cce9b879daa7c5d P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:54.350376 16819 raft_consensus.cc:1275] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [term 23 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 21 index: 14. Preceding OpId from leader: term: 23 index: 15. (index mismatch)
I20260709 13:21:54.459862 16148 raft_consensus.cc:2468] T 02838c86531d4f98bf58d1c22b8baf77 P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 18.
W20260709 13:21:54.342888 16264 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:54.487666 16821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66f75124215f4ca6bef831e174a792d8" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:54.232394 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4dc6b0541ca431ebc757beac00521f1" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:21:54.316864 16824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:54.488857 16221 rpcz_store.cc:276] 0709 13:21:50.165230 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:50.317688 (+152458us) service_pool.cc:224] Handling call
0709 13:21:51.935421 (+1617733us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:21:51.989538 (+ 54117us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:54.230098 (+2240560us) spinlock_profiling.cc:243] Waited 1.62 s on lock 0x7b54000154d0. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5e8ad 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:21:54.230235 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3719365376}
I20260709 13:21:54.285532 16820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5d59407eb545b895d8d922e0dd69fd" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 21 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
W20260709 13:21:54.449357 16789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 81230.1us
W20260709 13:21:54.264573 16116 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:54.251349 16298 raft_consensus.cc:2468] T cdbd78acb4cf46298e0de8ba0d130e19 P 90f3da57796b488c9087a2f523adf045 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 21.
I20260709 13:21:54.311820 16296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3884f864a96d428785895e5463e5728f" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:54.493871 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f5f89186204242bf377f7c42b2e2da" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:21:54.265182 16822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57a744f5aaf944678a63469cbf514015" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
W20260709 13:21:54.494603 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 407133us
I20260709 13:21:54.489410 16824 raft_consensus.cc:2468] T 19baa7bc367844c288bca8f81439c413 P cb82cc7bd6db46d980c75237a1402860 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 20.
I20260709 13:21:54.243953 16817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b7d1ab519044eda2d5e04a07242f25" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:54.497196 16825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e9b2bf7011f4ae688a79d7bb5a447b4" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
W20260709 13:21:54.497429 16115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 307350us
I20260709 13:21:54.187865 22677 raft_consensus.cc:493] T fffab26427e24d398e2695aad40ca134 P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:54.241458 16295 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48674 (request call id 3281) took 1093 ms. Trace:
I20260709 13:21:54.500387 16295 rpcz_store.cc:276] 0709 13:21:53.148191 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:53.172783 (+ 24592us) service_pool.cc:224] Handling call
0709 13:21:53.173292 (+   509us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:21:53.353929 (+180637us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:21:53.353982 (+    53us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:21:53.354026 (+    44us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:21:53.354075 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:54.241154 (+887079us) spinlock_profiling.cc:243] Waited 180 ms on lock 0x7b540010de54. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:21:54.241217 (+    63us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:21:54.241382 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":414913792}
I20260709 13:21:54.501047 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c61a39fd4f84424b129c1a6f9ba404d" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:21:54.501701 16299 raft_consensus.cc:3060] T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:21:54.502027 16295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74ca8140f35c4ae29e8a664f6bdfc829" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
W20260709 13:21:54.502790 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 312614us
I20260709 13:21:54.502852 16295 raft_consensus.cc:2468] T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 18.
I20260709 13:21:54.504535 16295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f82224f4564fea93572e8206bad1fe" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:21:54.330513 16818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:54.494469 16292 rpcz_store.cc:276] 0709 13:21:53.224552 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:53.367782 (+143230us) service_pool.cc:224] Handling call
0709 13:21:54.230655 (+862873us) spinlock_profiling.cc:243] Waited 1.34 ms on lock 0x7b54000be5d4. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:21:54.230784 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3082112}
I20260709 13:21:54.505223 16295 raft_consensus.cc:3060] T 23f82224f4564fea93572e8206bad1fe P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:21:54.505507 16298 raft_consensus.cc:3060] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:21:54.197037 22688 raft_consensus.cc:697] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 [term 21 LEADER]: Becoming Leader. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:21:54.197276 22593 leader_election.cc:290] T a378d7f796f94615b3b347119a9880fd P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:54.503185 16148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6f07842d3aa47858df5665874c86f3a" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
W20260709 13:21:54.236145 22716 log.cc:927] Time spent T 8158bbd17b994d2eb96f560bd0f1b88d P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.864s	user 0.001s	sys 0.006s
W20260709 13:21:54.381042 22718 log.cc:927] Time spent T 5a96373bb0e14dce8a266b2fc6c58507 P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 1.009s	user 0.006s	sys 0.003s
W20260709 13:21:54.450907 22496 consensus_meta.cc:287] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac: Time spent flushing consensus metadata: real 0.554s	user 0.006s	sys 0.005s
I20260709 13:21:54.519058 16296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02838c86531d4f98bf58d1c22b8baf77" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
W20260709 13:21:54.471693 22439 consensus_meta.cc:287] T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045: Time spent flushing consensus metadata: real 0.548s	user 0.009s	sys 0.001s
I20260709 13:21:54.521418 16299 raft_consensus.cc:2468] T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 19.
I20260709 13:21:54.488016 22591 raft_consensus.cc:3060] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:21:54.528610 16116 connection.cc:256] RPC call timeout handler was delayed by -2.4116014020000875s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18400000000000002s for a response
I20260709 13:21:54.524546 16298 raft_consensus.cc:2468] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 19.
I20260709 13:21:54.525873 16292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ab0ac8347641e6955e9bce7d90c1a2" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:54.531493 16292 raft_consensus.cc:2468] T d2ab0ac8347641e6955e9bce7d90c1a2 P 90f3da57796b488c9087a2f523adf045 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 14.
I20260709 13:21:54.527325 16296 raft_consensus.cc:2393] T 02838c86531d4f98bf58d1c22b8baf77 P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 18: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
I20260709 13:21:54.527876 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5ad4eb281564936a59f41e387a47156" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:54.528072 16789 leader_election.cc:304] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:21:54.198037 22699 raft_consensus.cc:515] T fffab26427e24d398e2695aad40ca134 P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:54.531785 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f2da98864b4425bf7c0055951fdd4b" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:21:54.310393 22586 raft_consensus.cc:3055] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:21:54.533703 16298 raft_consensus.cc:3060] T 84f2da98864b4425bf7c0055951fdd4b P 90f3da57796b488c9087a2f523adf045 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:21:54.534408 16148 raft_consensus.cc:1240] T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 15. Current term is 16. Ops: []
I20260709 13:21:54.535055 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5718c623ebee4d2bb770f0f9ddb2f164" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:54.535738 16149 raft_consensus.cc:2468] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 17.
I20260709 13:21:54.533631 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:54.517068 22568 raft_consensus.cc:2804] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:21:54.510672 22677 raft_consensus.cc:515] T fffab26427e24d398e2695aad40ca134 P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:54.498224 22735 raft_consensus.cc:493] T 57a744f5aaf944678a63469cbf514015 P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Starting leader election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:21:54.543642 16295 raft_consensus.cc:2468] T 23f82224f4564fea93572e8206bad1fe P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 16.
I20260709 13:21:54.538357 16299 raft_consensus.cc:2468] T ecc1e363230043fe9fef390f0b979f20 P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 19.
I20260709 13:21:54.549158 16292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c2095a5f4ad49408dbc7f69c606120d" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 15 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:54.549324 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4f65cdf00b748bb963ee6e81ef43b79" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:54.549911 16299 raft_consensus.cc:2468] T c4f65cdf00b748bb963ee6e81ef43b79 P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 19.
I20260709 13:21:54.240687 22504 raft_consensus.cc:515] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:54.524194 22348 raft_consensus.cc:493] T c2f5f89186204242bf377f7c42b2e2da P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:54.546098 16149 raft_consensus.cc:1240] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 16. Current term is 17. Ops: []
I20260709 13:21:54.550709 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ad5dbfa99ee4b8d9e28be92b3f36db5" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:54.527626 22703 raft_consensus.cc:493] T af57996a8be84d398f0f023e8d08ef77 P cb82cc7bd6db46d980c75237a1402860 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:21:54.552796 16299 raft_consensus.cc:2468] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 17.
I20260709 13:21:54.511582 22517 raft_consensus.cc:515] T 8a878999c0024d2abe517484180290fc P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:54.542443 22568 raft_consensus.cc:493] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Starting leader election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:21:54.542075 22742 raft_consensus.cc:493] T e5ad4eb281564936a59f41e387a47156 P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:54.555342 16148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2794af5b99984efebdee564035841cb0" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:54.551388 16298 raft_consensus.cc:2468] T 84f2da98864b4425bf7c0055951fdd4b P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 15.
I20260709 13:21:54.541507 22737 raft_consensus.cc:2749] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P cb82cc7bd6db46d980c75237a1402860 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:21:54.547624 22709 leader_election.cc:290] T 68a6838d82ef4cac83507f906139d90f P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:21:54.526106 22478 consensus_queue.cc:1059] T 8638ee363273426db1393dd81e5c7b72 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: INVALID_TERM, Last received: 17.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.026s
I20260709 13:21:54.538447 22586 raft_consensus.cc:740] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:21:54.544764 22610 raft_consensus.cc:493] T 2794af5b99984efebdee564035841cb0 P 647a343a542d4a139fe106c19095129b [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:54.550969 22735 raft_consensus.cc:3060] T 57a744f5aaf944678a63469cbf514015 P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:21:54.560019 22568 raft_consensus.cc:3060] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:21:54.575002 22508 consensus_queue.cc:1048] T b87147731a68400e8cce9b879daa7c5d P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.003s
I20260709 13:21:54.575508 22743 consensus_queue.cc:1048] T 57a744f5aaf944678a63469cbf514015 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.010s
I20260709 13:21:54.525054 22572 consensus_queue.cc:1048] T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.026s
I20260709 13:21:54.564599 16294 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48674 (request call id 3274) took 1586 ms. Trace:
I20260709 13:21:54.582707 16294 rpcz_store.cc:276] 0709 13:21:53.107932 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:53.108138 (+   206us) service_pool.cc:224] Handling call
0709 13:21:53.108563 (+   425us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:21:53.108994 (+   431us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:21:53.109022 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:21:53.138363 (+ 29341us) log.cc:844] Serialized 26 byte log entry
0709 13:21:53.234062 (+ 95699us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:21:53.234127 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:53.234214 (+    87us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:21:54.380327 (+1146113us) spinlock_profiling.cc:243] Waited 6.77 ms on lock 0x7b4400022b4c. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69ad8fadb1 00007f69ad998c85 00007f69ad9996a2 00007f69b070b1db 00007f69b0706aaf 00007f69b06fa86e 00007f69b06f9bb6 00007f69b06f97d8 00007f69b0703221 00007f69b07031d9 00007f69b0703169
0709 13:21:54.560753 (+180426us) raft_consensus.cc:1685] finished
0709 13:21:54.560776 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:21:54.564530 (+  3754us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":402,"raft.run_cpu_time_us":90,"raft.run_wall_time_us":89,"spinlock_wait_cycles":15560960,"thread_start_us":49453,"threads_started":1,"wal-append.queue_time_us":103436}
I20260709 13:21:54.585367 16294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c11074579b046d99e585c2818e18a2c" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 19 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:54.586459 16294 raft_consensus.cc:2410] T 8c11074579b046d99e585c2818e18a2c P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac for term 19 because replica has last-logged OpId of term: 17 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 13.
I20260709 13:21:54.554569 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87ac0d1f7e8a419f9bcbc04aee0e067e" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:54.533378 22701 consensus_queue.cc:1059] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.023s
I20260709 13:21:54.540211 22573 raft_consensus.cc:493] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:21:54.570631 16142 raft_consensus.cc:1240] T 8a878999c0024d2abe517484180290fc P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 15. Current term is 16. Ops: []
I20260709 13:21:54.559965 22196 raft_consensus.cc:493] T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:54.572645 16293 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48674 (request call id 3276) took 1432 ms. Trace:
I20260709 13:21:54.533793 22439 raft_consensus.cc:515] T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:54.560676 22703 raft_consensus.cc:515] T af57996a8be84d398f0f023e8d08ef77 P cb82cc7bd6db46d980c75237a1402860 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:54.616966 16222 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57600 (request call id 1914) took 1303 ms. Trace:
I20260709 13:21:54.623844 16222 rpcz_store.cc:276] 0709 13:21:53.313902 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:53.317630 (+  3728us) service_pool.cc:224] Handling call
0709 13:21:54.616805 (+1299175us) spinlock_profiling.cc:243] Waited 4.1 ms on lock 0x7b54002b3754. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:21:54.616908 (+   103us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":9433344}
I20260709 13:21:54.625209 16147 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46224 (request call id 1064) took 1270 ms. Trace:
I20260709 13:21:54.625614 16147 rpcz_store.cc:276] 0709 13:21:53.355034 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:53.357637 (+  2603us) service_pool.cc:224] Handling call
0709 13:21:53.358185 (+   548us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:21:53.368879 (+ 10694us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:21:53.368915 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:21:53.369444 (+   529us) log.cc:844] Serialized 26 byte log entry
0709 13:21:53.546085 (+176641us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:21:53.546126 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:54.615691 (+1069565us) spinlock_profiling.cc:243] Waited 10.3 ms on lock 0x7b54001f1dd4. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:21:54.615740 (+    49us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:21:54.615785 (+    45us) raft_consensus.cc:1685] finished
0709 13:21:54.615799 (+    14us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:21:54.625144 (+  9345us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1208,"raft.run_cpu_time_us":80,"raft.run_wall_time_us":81,"spinlock_wait_cycles":23616384,"thread_start_us":176239,"threads_started":1,"wal-append.queue_time_us":437876}
W20260709 13:21:54.627113 16265 connection.cc:256] RPC call timeout handler was delayed by -1.2311938539999119s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1714s for a response
I20260709 13:21:54.546466 22731 consensus_queue.cc:1048] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.023s
I20260709 13:21:54.591077 16299 raft_consensus.cc:2468] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 18.
W20260709 13:21:54.632301 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 78378.5us
I20260709 13:21:54.633147 16790 leader_election.cc:304] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:21:54.637374 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a959dbecc2648eb85d1cc550f5a3245" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:54.637389 16219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0eab25ce4c64e018a041beeda2867db" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:54.563647 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c74490c40243a88e62d1760b5cd814" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:54.639191 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c61a39fd4f84424b129c1a6f9ba404d" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:54.639806 16226 raft_consensus.cc:2468] T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 18.
I20260709 13:21:54.640801 16298 raft_consensus.cc:1240] T 8c11074579b046d99e585c2818e18a2c P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 17. Current term is 18. Ops: []
I20260709 13:21:54.555531 22348 raft_consensus.cc:515] T c2f5f89186204242bf377f7c42b2e2da P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:54.642064 22752 consensus_queue.cc:1059] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: INVALID_TERM, Last received: 16.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:21:54.642892 22478 raft_consensus.cc:3055] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:21:54.643288 22478 raft_consensus.cc:740] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
W20260709 13:21:54.593397 16116 connection.cc:256] RPC call timeout handler was delayed by -1.3687883800000691s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1837s for a response
I20260709 13:21:54.643926 16219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a878999c0024d2abe517484180290fc" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:54.532408 22496 raft_consensus.cc:515] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:54.638105 16224 raft_consensus.cc:2468] T 5a959dbecc2648eb85d1cc550f5a3245 P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 15.
I20260709 13:21:54.644443 22478 consensus_queue.cc:260] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:54.640887 22701 raft_consensus.cc:2804] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:21:54.645371 22701 raft_consensus.cc:493] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:54.645654 22478 raft_consensus.cc:3060] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:21:54.601540 16294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d4ab870a05347868956aa8b841b2981" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:54.645733 22701 raft_consensus.cc:3060] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:21:54.646359 16294 raft_consensus.cc:2468] T 8d4ab870a05347868956aa8b841b2981 P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 16.
I20260709 13:21:54.647693 16294 raft_consensus.cc:1275] T c0eab25ce4c64e018a041beeda2867db P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
W20260709 13:21:54.628084 16265 connection.cc:256] RPC call timeout handler was delayed by -1.1825938539998333s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1768s for a response
I20260709 13:21:54.648445 22457 consensus_queue.cc:1048] T 8158bbd17b994d2eb96f560bd0f1b88d P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:21:54.648973 16294 raft_consensus.cc:1275] T 8cc959fb7f704fba9009c0595467d074 P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 13:21:54.606032 16293 rpcz_store.cc:276] 0709 13:21:53.140258 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:53.140386 (+   128us) service_pool.cc:224] Handling call
0709 13:21:53.149589 (+  9203us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:21:53.151269 (+  1680us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:21:53.151301 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:21:53.152045 (+   744us) log.cc:844] Serialized 26 byte log entry
0709 13:21:53.262226 (+110181us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:21:53.262266 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:53.285894 (+ 23628us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:21:54.235517 (+949623us) spinlock_profiling.cc:243] Waited 6.39 ms on lock 0x7b4400022b4c. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69ad8fadb1 00007f69ad998c85 00007f69ad9996a2 00007f69b070b1db 00007f69b0706aaf 00007f69b06fa86e 00007f69b06f9bb6 00007f69b06f97d8 00007f69b0703221 00007f69b07031d9 00007f69b0703169
0709 13:21:54.568968 (+333451us) raft_consensus.cc:1685] finished
0709 13:21:54.568991 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:21:54.572563 (+  3572us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":209,"raft.run_cpu_time_us":141,"raft.run_wall_time_us":417,"spinlock_wait_cycles":16291456,"thread_start_us":109813,"threads_started":1,"wal-append.queue_time_us":219382}
I20260709 13:21:54.643222 22348 leader_election.cc:290] T c2f5f89186204242bf377f7c42b2e2da P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:54.650112 16294 raft_consensus.cc:1240] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [term 21 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 20. Current term is 21. Ops: []
I20260709 13:21:54.643288 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f5f89186204242bf377f7c42b2e2da" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:54.651593 16226 raft_consensus.cc:2468] T c2f5f89186204242bf377f7c42b2e2da P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 18.
W20260709 13:21:54.651795 16191 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:21:54.652058 16325 heartbeater.cc:646] Failed to heartbeat to 127.14.177.190:44079 (4 consecutive failures): Timed out: Failed to send heartbeat to master: TSHeartbeat RPC to 127.14.177.190:44079 timed out after 15.000s (SENT)
I20260709 13:21:54.652640 22748 raft_consensus.cc:493] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:54.658207 22356 consensus_queue.cc:1059] T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: INVALID_TERM, Last received: 17.9, Next index: 10, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:21:54.659114 22457 raft_consensus.cc:3055] T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:21:54.659518 22457 raft_consensus.cc:740] T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:21:54.666003 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b7d1ab519044eda2d5e04a07242f25" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:54.666894 16220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806d7690c2b644959152074bb2cab0a7" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:21:54.667151 16225 raft_consensus.cc:2393] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 16: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
I20260709 13:21:54.668731 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c61a39fd4f84424b129c1a6f9ba404d" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:21:54.669420 16225 raft_consensus.cc:3060] T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:21:54.579198 22610 raft_consensus.cc:515] T 2794af5b99984efebdee564035841cb0 P 647a343a542d4a139fe106c19095129b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:21:54.677047 22745 log.cc:927] Time spent T 23f82224f4564fea93572e8206bad1fe P cb82cc7bd6db46d980c75237a1402860: Append to log took a long time: real 0.108s	user 0.001s	sys 0.000s
I20260709 13:21:54.670024 22731 consensus_queue.cc:1243] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Peer 90f3da57796b488c9087a2f523adf045 log is divergent from this leader: its last log entry 16.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 11
I20260709 13:21:54.677608 22709 raft_consensus.cc:493] T c0eab25ce4c64e018a041beeda2867db P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:54.606662 22573 raft_consensus.cc:515] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:54.610785 22196 raft_consensus.cc:515] T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:54.629496 22676 consensus_queue.cc:1059] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.071s
I20260709 13:21:54.696789 22747 raft_consensus.cc:493] T 68a6838d82ef4cac83507f906139d90f P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:21:54.717859 22537 leader_election.cc:290] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
W20260709 13:21:54.730950 22744 log.cc:927] Time spent T 746aabf856ab401c9ef5e859371e7b67 P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 0.163s	user 0.000s	sys 0.001s
I20260709 13:21:54.732436 22610 leader_election.cc:290] T 2794af5b99984efebdee564035841cb0 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:54.685328 22645 consensus_queue.cc:1048] T ada5a004ba7c46819051e3ccec71ffdb P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.035s
I20260709 13:21:54.716652 22709 raft_consensus.cc:515] T c0eab25ce4c64e018a041beeda2867db P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:54.569527 22742 raft_consensus.cc:515] T e5ad4eb281564936a59f41e387a47156 P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:54.653190 22748 raft_consensus.cc:515] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:54.660585 22457 consensus_queue.cc:260] T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.9, Last appended by leader: 9, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:54.578783 22688 consensus_queue.cc:237] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:54.751549 22699 leader_election.cc:290] T fffab26427e24d398e2695aad40ca134 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:54.731841 22747 raft_consensus.cc:515] T 68a6838d82ef4cac83507f906139d90f P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:54.660254 22701 raft_consensus.cc:515] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:54.751634 22677 leader_election.cc:290] T fffab26427e24d398e2695aad40ca134 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:54.761735 22196 leader_election.cc:290] T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:54.764638 22747 leader_election.cc:290] T 68a6838d82ef4cac83507f906139d90f P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:54.765408 22457 raft_consensus.cc:3060] T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:21:54.765605 22709 leader_election.cc:290] T c0eab25ce4c64e018a041beeda2867db P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:21:54.767772 22701 leader_election.cc:290] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 18 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:21:54.757365 22504 leader_election.cc:290] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 20 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:21:54.775233 22703 leader_election.cc:290] T af57996a8be84d398f0f023e8d08ef77 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:21:54.772912 22659 raft_consensus.cc:493] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:54.776430 22659 raft_consensus.cc:515] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:54.759541 22517 leader_election.cc:290] T 8a878999c0024d2abe517484180290fc P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 16 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:54.788673 22439 leader_election.cc:290] T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 18 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:21:54.794859 22735 raft_consensus.cc:515] T 57a744f5aaf944678a63469cbf514015 P 647a343a542d4a139fe106c19095129b [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:54.799741 22735 leader_election.cc:290] T 57a744f5aaf944678a63469cbf514015 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 14 election: Requested vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:54.800066 22505 raft_consensus.cc:3060] T 8c11074579b046d99e585c2818e18a2c P cb82cc7bd6db46d980c75237a1402860 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:21:54.766553 22568 raft_consensus.cc:515] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:54.761765 22742 leader_election.cc:290] T e5ad4eb281564936a59f41e387a47156 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:54.837754 22681 raft_consensus.cc:493] T 8c11074579b046d99e585c2818e18a2c P cb82cc7bd6db46d980c75237a1402860 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:54.838320 22681 raft_consensus.cc:515] T 8c11074579b046d99e585c2818e18a2c P cb82cc7bd6db46d980c75237a1402860 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:54.858374 16222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5d59407eb545b895d8d922e0dd69fd" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 21 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
W20260709 13:21:54.862409 22713 log.cc:927] Time spent T b6f07842d3aa47858df5665874c86f3a P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 1.495s	user 0.001s	sys 0.011s
I20260709 13:21:54.866660 22749 raft_consensus.cc:493] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac [term 20 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:21:54.867417 22749 raft_consensus.cc:515] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:21:54.882144 16791 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:54.889412 16043 catalog_manager.cc:5697] T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045 reported cstate change: term changed from 7 to 13, leader changed from 647a343a542d4a139fe106c19095129b (127.14.177.129) to 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130). New cstate: current_term: 13 leader_uuid: "1ca2725d290e47d68b1a4b609c9816ac" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } }
I20260709 13:21:54.892191 16043 catalog_manager.cc:5697] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 90f3da57796b488c9087a2f523adf045 reported cstate change: term changed from 7 to 12, leader changed from 647a343a542d4a139fe106c19095129b (127.14.177.129) to 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130). New cstate: current_term: 12 leader_uuid: "1ca2725d290e47d68b1a4b609c9816ac" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } }
I20260709 13:21:54.894671 16043 catalog_manager.cc:5697] T 8158bbd17b994d2eb96f560bd0f1b88d P 90f3da57796b488c9087a2f523adf045 reported cstate change: term changed from 11 to 13, leader changed from 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130) to 90f3da57796b488c9087a2f523adf045 (127.14.177.131). New cstate: current_term: 13 leader_uuid: "90f3da57796b488c9087a2f523adf045" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } health_report { overall_health: HEALTHY } } }
I20260709 13:21:54.897466 16043 catalog_manager.cc:5697] T 68a6838d82ef4cac83507f906139d90f P 90f3da57796b488c9087a2f523adf045 reported cstate change: term changed from 8 to 15, leader changed from 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130) to 647a343a542d4a139fe106c19095129b (127.14.177.129). New cstate: current_term: 15 leader_uuid: "647a343a542d4a139fe106c19095129b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } }
I20260709 13:21:54.899293 16043 catalog_manager.cc:5697] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 90f3da57796b488c9087a2f523adf045 reported cstate change: term changed from 7 to 14, leader changed from 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130) to 647a343a542d4a139fe106c19095129b (127.14.177.129). New cstate: current_term: 14 leader_uuid: "647a343a542d4a139fe106c19095129b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } }
I20260709 13:21:54.902261 16043 catalog_manager.cc:5697] T 5a959dbecc2648eb85d1cc550f5a3245 P 90f3da57796b488c9087a2f523adf045 reported cstate change: term changed from 5 to 12, leader changed from 647a343a542d4a139fe106c19095129b (127.14.177.129) to 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130). New cstate: current_term: 12 leader_uuid: "1ca2725d290e47d68b1a4b609c9816ac" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } }
I20260709 13:21:54.904521 16043 catalog_manager.cc:5697] T 8d4ab870a05347868956aa8b841b2981 P 90f3da57796b488c9087a2f523adf045 reported cstate change: term changed from 11 to 12, leader changed from 647a343a542d4a139fe106c19095129b (127.14.177.129) to 90f3da57796b488c9087a2f523adf045 (127.14.177.131). New cstate: current_term: 12 leader_uuid: "90f3da57796b488c9087a2f523adf045" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } health_report { overall_health: HEALTHY } } }
I20260709 13:21:54.907531 16043 catalog_manager.cc:5697] T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045 reported cstate change: term changed from 14 to 15, leader changed from 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130) to 647a343a542d4a139fe106c19095129b (127.14.177.129). New cstate: current_term: 15 leader_uuid: "647a343a542d4a139fe106c19095129b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } }
I20260709 13:21:54.911511 16043 catalog_manager.cc:5697] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045 reported cstate change: term changed from 9 to 14, leader changed from 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130) to 647a343a542d4a139fe106c19095129b (127.14.177.129). New cstate: current_term: 14 leader_uuid: "647a343a542d4a139fe106c19095129b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } }
I20260709 13:21:54.914860 16043 catalog_manager.cc:5697] T 8a878999c0024d2abe517484180290fc P 90f3da57796b488c9087a2f523adf045 reported cstate change: term changed from 6 to 12, leader changed from 90f3da57796b488c9087a2f523adf045 (127.14.177.131) to 647a343a542d4a139fe106c19095129b (127.14.177.129). New cstate: current_term: 12 leader_uuid: "647a343a542d4a139fe106c19095129b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } }
I20260709 13:21:54.916926 16043 catalog_manager.cc:5697] T ada5a004ba7c46819051e3ccec71ffdb P 90f3da57796b488c9087a2f523adf045 reported cstate change: term changed from 8 to 14, leader changed from 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130) to 90f3da57796b488c9087a2f523adf045 (127.14.177.131). New cstate: current_term: 14 leader_uuid: "90f3da57796b488c9087a2f523adf045" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:21:54.920110 16043 catalog_manager.cc:5697] T 5a96373bb0e14dce8a266b2fc6c58507 P 90f3da57796b488c9087a2f523adf045 reported cstate change: term changed from 7 to 12. New cstate: current_term: 12 leader_uuid: "647a343a542d4a139fe106c19095129b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } }
I20260709 13:21:54.922106 16043 catalog_manager.cc:5697] T c4f65cdf00b748bb963ee6e81ef43b79 P 90f3da57796b488c9087a2f523adf045 reported cstate change: term changed from 11 to 15. New cstate: current_term: 15 leader_uuid: "1ca2725d290e47d68b1a4b609c9816ac" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } }
I20260709 13:21:54.924361 16043 catalog_manager.cc:5697] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045 reported cstate change: term changed from 6 to 13, leader changed from 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130) to 90f3da57796b488c9087a2f523adf045 (127.14.177.131). New cstate: current_term: 13 leader_uuid: "90f3da57796b488c9087a2f523adf045" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } health_report { overall_health: HEALTHY } } }
I20260709 13:21:54.926855 16043 catalog_manager.cc:5697] T 02838c86531d4f98bf58d1c22b8baf77 P 90f3da57796b488c9087a2f523adf045 reported cstate change: term changed from 11 to 13. New cstate: current_term: 13 leader_uuid: "90f3da57796b488c9087a2f523adf045" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } health_report { overall_health: HEALTHY } } }
I20260709 13:21:54.929044 16043 catalog_manager.cc:5697] T e4dc6b0541ca431ebc757beac00521f1 P 90f3da57796b488c9087a2f523adf045 reported cstate change: term changed from 6 to 12, leader changed from 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130) to 647a343a542d4a139fe106c19095129b (127.14.177.129). New cstate: current_term: 12 leader_uuid: "647a343a542d4a139fe106c19095129b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } }
I20260709 13:21:54.932101 16043 catalog_manager.cc:5697] T c0eab25ce4c64e018a041beeda2867db P 90f3da57796b488c9087a2f523adf045 reported cstate change: term changed from 8 to 12, leader changed from 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130) to 90f3da57796b488c9087a2f523adf045 (127.14.177.131). New cstate: current_term: 12 leader_uuid: "90f3da57796b488c9087a2f523adf045" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } health_report { overall_health: HEALTHY } } }
I20260709 13:21:54.960662 22500 raft_consensus.cc:3055] T 8638ee363273426db1393dd81e5c7b72 P cb82cc7bd6db46d980c75237a1402860 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:21:54.961148 22500 raft_consensus.cc:740] T 8638ee363273426db1393dd81e5c7b72 P cb82cc7bd6db46d980c75237a1402860 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:21:54.962031 22500 consensus_queue.cc:260] T 8638ee363273426db1393dd81e5c7b72 P cb82cc7bd6db46d980c75237a1402860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:54.968576 22500 raft_consensus.cc:3060] T 8638ee363273426db1393dd81e5c7b72 P cb82cc7bd6db46d980c75237a1402860 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:21:54.998438 22733 leader_election.cc:290] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:21:55.001550 16147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:55.004685 22496 leader_election.cc:290] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 21 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:55.006268 22573 leader_election.cc:290] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:55.006457 16147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a9de347c711477e87c582d003f36925" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:55.008411 16147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d4ab870a05347868956aa8b841b2981" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:55.008926 22568 leader_election.cc:290] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 20 election: Requested vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:55.009150 16147 raft_consensus.cc:2468] T 8d4ab870a05347868956aa8b841b2981 P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 16.
I20260709 13:21:55.010368 16147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2794af5b99984efebdee564035841cb0" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:55.012023 16147 raft_consensus.cc:1275] T 7c2095a5f4ad49408dbc7f69c606120d P 647a343a542d4a139fe106c19095129b [term 14 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 13 index: 6. Preceding OpId from leader: term: 14 index: 7. (index mismatch)
W20260709 13:21:55.012431 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 336147us
I20260709 13:21:55.013526 16265 leader_election.cc:304] T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 647a343a542d4a139fe106c19095129b
I20260709 13:21:55.014314 22593 raft_consensus.cc:2764] T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Leader election decision vote started in defunct term 18: won
I20260709 13:21:55.015007 22645 consensus_queue.cc:1048] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:21:55.024817 16220 raft_consensus.cc:2468] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 18.
W20260709 13:21:55.027688 22712 log.cc:927] Time spent T 1df93779209748b9860f6d893dd3de53 P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 1.656s	user 0.006s	sys 0.003s
I20260709 13:21:55.031080 16218 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57600 (request call id 1902) took 2666 ms. Trace:
I20260709 13:21:55.031533 16823 raft_consensus.cc:2468] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 16.
I20260709 13:21:55.031569 16218 rpcz_store.cc:276] 0709 13:21:52.365121 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:52.402460 (+ 37339us) service_pool.cc:224] Handling call
0709 13:21:52.403339 (+   879us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:21:52.513086 (+109747us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:55.030799 (+2517713us) spinlock_profiling.cc:243] Waited 109 ms on lock 0x7b54002c8154. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:21:55.031010 (+   211us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":250030848}
I20260709 13:21:55.015081 16265 leader_election.cc:304] T 5a96373bb0e14dce8a266b2fc6c58507 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:21:55.034564 16265 leader_election.cc:304] T 8158bbd17b994d2eb96f560bd0f1b88d P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:21:55.035437 22709 raft_consensus.cc:2804] T 5a96373bb0e14dce8a266b2fc6c58507 P 90f3da57796b488c9087a2f523adf045 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:21:55.035533 22645 raft_consensus.cc:2804] T 8158bbd17b994d2eb96f560bd0f1b88d P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:21:55.035446 16265 leader_election.cc:304] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:21:55.036001 22709 raft_consensus.cc:493] T 5a96373bb0e14dce8a266b2fc6c58507 P 90f3da57796b488c9087a2f523adf045 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:21:55.036294 22645 raft_consensus.cc:493] T 8158bbd17b994d2eb96f560bd0f1b88d P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:21:55.036533 22709 raft_consensus.cc:3060] T 5a96373bb0e14dce8a266b2fc6c58507 P 90f3da57796b488c9087a2f523adf045 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:21:55.036851 22645 raft_consensus.cc:3060] T 8158bbd17b994d2eb96f560bd0f1b88d P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:21:55.037007 22439 raft_consensus.cc:2804] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:21:55.037493 22439 raft_consensus.cc:493] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:55.037812 22439 raft_consensus.cc:3060] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:21:55.039382 22593 consensus_queue.cc:1048] T e5ad4eb281564936a59f41e387a47156 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:21:55.044313 22709 raft_consensus.cc:515] T 5a96373bb0e14dce8a266b2fc6c58507 P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:55.046846 22645 raft_consensus.cc:515] T 8158bbd17b994d2eb96f560bd0f1b88d P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:55.045157 22439 raft_consensus.cc:515] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:21:55.047370 16190 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.534s	user 0.007s	sys 0.019s
I20260709 13:21:55.047742 22709 leader_election.cc:290] T 5a96373bb0e14dce8a266b2fc6c58507 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 15 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:21:55.050665 22709 consensus_queue.cc:1048] T fffab26427e24d398e2695aad40ca134 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
W20260709 13:21:55.049599 16190 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.536s	user 0.007s	sys 0.019s
W20260709 13:21:55.052152 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 548208us
I20260709 13:21:55.049844 22439 leader_election.cc:290] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 19 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:21:55.049371 22645 leader_election.cc:290] T 8158bbd17b994d2eb96f560bd0f1b88d P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 17 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:21:55.054636 16190 leader_election.cc:304] T 19baa7bc367844c288bca8f81439c413 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860
I20260709 13:21:55.056313 16190 leader_election.cc:304] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860
I20260709 13:21:55.057608 22348 raft_consensus.cc:2749] T 19baa7bc367844c288bca8f81439c413 P 1ca2725d290e47d68b1a4b609c9816ac [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:21:55.058287 22348 raft_consensus.cc:2749] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:21:55.058243 16190 leader_election.cc:304] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860
I20260709 13:21:55.064543 16190 leader_election.cc:304] T 18ad7e3af5504571a69d10c00750a01c P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:21:55.065299 16265 leader_election.cc:304] T d16dfad0c3b540e5b4b86492ddf87208 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:21:55.066264 16265 leader_election.cc:304] T 1df93779209748b9860f6d893dd3de53 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:21:55.067312 16220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c11074579b046d99e585c2818e18a2c" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:55.067425 16190 leader_election.cc:304] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 647a343a542d4a139fe106c19095129b
I20260709 13:21:55.068089 16220 raft_consensus.cc:2468] T 8c11074579b046d99e585c2818e18a2c P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 18.
I20260709 13:21:55.068620 16190 leader_election.cc:304] T d2ab0ac8347641e6955e9bce7d90c1a2 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:21:55.069429 22756 raft_consensus.cc:2749] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:21:55.069541 22772 raft_consensus.cc:2804] T d16dfad0c3b540e5b4b86492ddf87208 P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:21:55.070152 22772 raft_consensus.cc:493] T d16dfad0c3b540e5b4b86492ddf87208 P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:55.070176 22756 raft_consensus.cc:2749] T 18ad7e3af5504571a69d10c00750a01c P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:21:55.070713 22772 raft_consensus.cc:3060] T d16dfad0c3b540e5b4b86492ddf87208 P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:21:55.071024 22756 consensus_queue.cc:1048] T 84f2da98864b4425bf7c0055951fdd4b P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:21:55.072522 22756 consensus_queue.cc:1048] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:21:55.073912 22756 raft_consensus.cc:2804] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:21:55.074352 22756 raft_consensus.cc:697] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 1ca2725d290e47d68b1a4b609c9816ac [term 17 LEADER]: Becoming Leader. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:21:55.075301 22756 consensus_queue.cc:237] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:55.075730 16218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ad5dbfa99ee4b8d9e28be92b3f36db5" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:55.079278 22772 raft_consensus.cc:515] T d16dfad0c3b540e5b4b86492ddf87208 P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:55.081635 22772 leader_election.cc:290] T d16dfad0c3b540e5b4b86492ddf87208 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 19 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:21:55.082275 22772 raft_consensus.cc:2804] T 1df93779209748b9860f6d893dd3de53 P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:21:55.083279 16265 leader_election.cc:304] T 57a744f5aaf944678a63469cbf514015 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:21:55.084672 16265 leader_election.cc:304] T c4f65cdf00b748bb963ee6e81ef43b79 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:21:55.085582 22754 raft_consensus.cc:2804] T d2ab0ac8347641e6955e9bce7d90c1a2 P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:21:55.082634 22772 raft_consensus.cc:493] T 1df93779209748b9860f6d893dd3de53 P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:21:55.086097 22754 raft_consensus.cc:493] T d2ab0ac8347641e6955e9bce7d90c1a2 P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Starting leader election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:21:55.086220 22772 raft_consensus.cc:3060] T 1df93779209748b9860f6d893dd3de53 P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:21:55.086549 22754 raft_consensus.cc:3060] T d2ab0ac8347641e6955e9bce7d90c1a2 P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:21:55.089226 16220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8638ee363273426db1393dd81e5c7b72" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:55.089926 16220 raft_consensus.cc:2468] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 17.
I20260709 13:21:55.090940 16218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4dc6b0541ca431ebc757beac00521f1" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 18 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:55.091245 16146 raft_consensus.cc:1275] T 08c74490c40243a88e62d1760b5cd814 P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 13:21:55.092053 16218 raft_consensus.cc:2393] T e4dc6b0541ca431ebc757beac00521f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 18: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
I20260709 13:21:55.093385 16145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ad5dbfa99ee4b8d9e28be92b3f36db5" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:55.094053 16145 raft_consensus.cc:2468] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 17.
I20260709 13:21:55.094434 16143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68a6838d82ef4cac83507f906139d90f" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
W20260709 13:21:55.096637 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 436698us
I20260709 13:21:55.098156 16116 leader_election.cc:400] T 746aabf856ab401c9ef5e859371e7b67 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 14 pre-election: Vote denied by peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943) with higher term. Message: Invalid argument: T 746aabf856ab401c9ef5e859371e7b67 P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b for earlier term 14. Current term is 15.
I20260709 13:21:55.098537 16116 leader_election.cc:403] T 746aabf856ab401c9ef5e859371e7b67 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 14 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:21:55.098790 22774 consensus_queue.cc:1048] T e5ad4eb281564936a59f41e387a47156 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:21:55.100324 22774 raft_consensus.cc:2804] T 57a744f5aaf944678a63469cbf514015 P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:21:55.100709 22774 raft_consensus.cc:493] T 57a744f5aaf944678a63469cbf514015 P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:55.101073 22774 raft_consensus.cc:3060] T 57a744f5aaf944678a63469cbf514015 P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:21:55.101294 22754 raft_consensus.cc:515] T d2ab0ac8347641e6955e9bce7d90c1a2 P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:55.103739 22754 leader_election.cc:290] T d2ab0ac8347641e6955e9bce7d90c1a2 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 15 election: Requested vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:55.109541 22772 raft_consensus.cc:515] T 1df93779209748b9860f6d893dd3de53 P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:55.129983 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68a6838d82ef4cac83507f906139d90f" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:55.129771 22774 raft_consensus.cc:515] T 57a744f5aaf944678a63469cbf514015 P 90f3da57796b488c9087a2f523adf045 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:55.130678 16224 raft_consensus.cc:2468] T 68a6838d82ef4cac83507f906139d90f P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 17.
I20260709 13:21:55.130929 22772 leader_election.cc:290] T 1df93779209748b9860f6d893dd3de53 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 17 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:21:55.131707 22772 raft_consensus.cc:2749] T c4f65cdf00b748bb963ee6e81ef43b79 P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:21:55.132396 22772 consensus_queue.cc:1048] T 31969392ccdb43b491911a319e8bc7e1 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:21:55.119786 22756 raft_consensus.cc:493] T 66f75124215f4ca6bef831e174a792d8 P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:21:55.123612 16221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d4ab870a05347868956aa8b841b2981" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:55.123720 16265 leader_election.cc:304] T 02838c86531d4f98bf58d1c22b8baf77 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:21:55.124013 16218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:55.125022 16220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a378d7f796f94615b3b347119a9880fd" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:55.142540 16221 raft_consensus.cc:2468] T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 16.
I20260709 13:21:55.142728 16218 raft_consensus.cc:2468] T ecc1e363230043fe9fef390f0b979f20 P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 19.
I20260709 13:21:55.133846 22774 leader_election.cc:290] T 57a744f5aaf944678a63469cbf514015 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 14 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:55.144826 16265 leader_election.cc:304] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:21:55.147547 16265 leader_election.cc:304] T 84f2da98864b4425bf7c0055951fdd4b P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:21:55.148270 22772 raft_consensus.cc:2749] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:21:55.148675 16265 leader_election.cc:304] T 26354361e9b044cea1c2b86c4882fe92 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:21:55.149062 22772 raft_consensus.cc:2749] T 84f2da98864b4425bf7c0055951fdd4b P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:21:55.149917 22772 raft_consensus.cc:2804] T 26354361e9b044cea1c2b86c4882fe92 P 90f3da57796b488c9087a2f523adf045 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:21:55.150126 16265 leader_election.cc:304] T 8d4ab870a05347868956aa8b841b2981 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:21:55.150374 22772 raft_consensus.cc:493] T 26354361e9b044cea1c2b86c4882fe92 P 90f3da57796b488c9087a2f523adf045 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:55.150920 22772 raft_consensus.cc:3060] T 26354361e9b044cea1c2b86c4882fe92 P 90f3da57796b488c9087a2f523adf045 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:21:55.150146 16114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:55.142468 22756 raft_consensus.cc:515] T 66f75124215f4ca6bef831e174a792d8 P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:55.153318 22756 leader_election.cc:290] T 66f75124215f4ca6bef831e174a792d8 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:55.153903 22756 raft_consensus.cc:493] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:21:55.154389 22756 raft_consensus.cc:515] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:55.155195 22775 raft_consensus.cc:2804] T 02838c86531d4f98bf58d1c22b8baf77 P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:21:55.155678 22775 raft_consensus.cc:493] T 02838c86531d4f98bf58d1c22b8baf77 P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:55.156045 22775 raft_consensus.cc:3060] T 02838c86531d4f98bf58d1c22b8baf77 P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:21:55.143625 22754 raft_consensus.cc:493] T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:55.159664 22754 raft_consensus.cc:515] T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:55.160746 22772 raft_consensus.cc:515] T 26354361e9b044cea1c2b86c4882fe92 P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:55.161902 22774 raft_consensus.cc:2804] T 8d4ab870a05347868956aa8b841b2981 P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:21:55.164036 22774 raft_consensus.cc:493] T 8d4ab870a05347868956aa8b841b2981 P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:55.164469 22774 raft_consensus.cc:3060] T 8d4ab870a05347868956aa8b841b2981 P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:21:55.165408 22772 leader_election.cc:290] T 26354361e9b044cea1c2b86c4882fe92 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 15 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:55.165572 16144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87147731a68400e8cce9b879daa7c5d" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:55.166561 16144 raft_consensus.cc:2410] T b87147731a68400e8cce9b879daa7c5d P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 for term 19 because replica has last-logged OpId of term: 18 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 11.
I20260709 13:21:55.168480 16150 raft_consensus.cc:1275] T fffab26427e24d398e2695aad40ca134 P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 13:21:55.169837 16265 leader_election.cc:304] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:21:55.171283 22457 consensus_queue.cc:1059] T 18ad7e3af5504571a69d10c00750a01c P 647a343a542d4a139fe106c19095129b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: INVALID_TERM, Last received: 13.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:21:55.171348 22772 consensus_queue.cc:1048] T 7c2095a5f4ad49408dbc7f69c606120d P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:21:55.171833 22774 raft_consensus.cc:515] T 8d4ab870a05347868956aa8b841b2981 P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:55.173560 22457 raft_consensus.cc:2749] T 746aabf856ab401c9ef5e859371e7b67 P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Leader pre-election lost for term 14. Reason: Vote denied by peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943) with higher term. Message: Invalid argument: T 746aabf856ab401c9ef5e859371e7b67 P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b for earlier term 14. Current term is 15.
I20260709 13:21:55.174036 22774 leader_election.cc:290] T 8d4ab870a05347868956aa8b841b2981 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 17 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:21:55.174682 22774 raft_consensus.cc:2764] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Leader pre-election decision vote started in defunct term 16: won
I20260709 13:21:55.175341 22756 leader_election.cc:290] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:55.175918 22756 raft_consensus.cc:493] T ecc1e363230043fe9fef390f0b979f20 P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:21:55.176407 22756 raft_consensus.cc:515] T ecc1e363230043fe9fef390f0b979f20 P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:55.173038 22772 consensus_queue.cc:1048] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.005s
I20260709 13:21:55.180347 16219 raft_consensus.cc:1275] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [term 23 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 21 index: 14. Preceding OpId from leader: term: 23 index: 15. (index mismatch)
I20260709 13:21:55.182376 22196 raft_consensus.cc:493] T b87147731a68400e8cce9b879daa7c5d P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:21:55.182924 22196 raft_consensus.cc:515] T b87147731a68400e8cce9b879daa7c5d P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:55.186260 16222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a96373bb0e14dce8a266b2fc6c58507" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:21:55.186864 22756 leader_election.cc:290] T ecc1e363230043fe9fef390f0b979f20 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:55.187582 16222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8158bbd17b994d2eb96f560bd0f1b88d" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:21:55.188143 22772 consensus_queue.cc:1048] T 7c2095a5f4ad49408dbc7f69c606120d P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:21:55.188165 16222 raft_consensus.cc:3060] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:21:55.190330 16265 leader_election.cc:304] T 8c11074579b046d99e585c2818e18a2c P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:21:55.191308 22772 raft_consensus.cc:2804] T 8c11074579b046d99e585c2818e18a2c P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:21:55.191697 22772 raft_consensus.cc:493] T 8c11074579b046d99e585c2818e18a2c P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:55.192054 22772 raft_consensus.cc:3060] T 8c11074579b046d99e585c2818e18a2c P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:21:55.194839 16143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0eab25ce4c64e018a041beeda2867db" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:55.195693 16143 raft_consensus.cc:2393] T c0eab25ce4c64e018a041beeda2867db P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 18: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
I20260709 13:21:55.196530 16143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1df93779209748b9860f6d893dd3de53" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:21:55.197551 16143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26354361e9b044cea1c2b86c4882fe92" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:21:55.198040 16143 raft_consensus.cc:3060] T 26354361e9b044cea1c2b86c4882fe92 P 647a343a542d4a139fe106c19095129b [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:21:55.199486 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a96373bb0e14dce8a266b2fc6c58507" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:21:55.199959 16147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d16dfad0c3b540e5b4b86492ddf87208" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:21:55.200505 16147 raft_consensus.cc:3060] T d16dfad0c3b540e5b4b86492ddf87208 P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:21:55.200575 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57a744f5aaf944678a63469cbf514015" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:21:55.201413 16149 raft_consensus.cc:2393] T 57a744f5aaf944678a63469cbf514015 P 647a343a542d4a139fe106c19095129b [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 14: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
I20260709 13:21:55.201475 16265 leader_election.cc:304] T 8638ee363273426db1393dd81e5c7b72 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:21:55.202358 22774 raft_consensus.cc:2804] T 8638ee363273426db1393dd81e5c7b72 P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:21:55.202741 16145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8158bbd17b994d2eb96f560bd0f1b88d" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:21:55.202786 22774 raft_consensus.cc:493] T 8638ee363273426db1393dd81e5c7b72 P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:55.203907 22774 raft_consensus.cc:3060] T 8638ee363273426db1393dd81e5c7b72 P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:21:55.204468 16146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87ac0d1f7e8a419f9bcbc04aee0e067e" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:21:55.205020 16146 raft_consensus.cc:3060] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:21:55.206030 22586 consensus_queue.cc:260] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:55.208714 22586 raft_consensus.cc:3060] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:21:55.209004 22196 leader_election.cc:290] T b87147731a68400e8cce9b879daa7c5d P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:55.209944 16147 raft_consensus.cc:2468] T d16dfad0c3b540e5b4b86492ddf87208 P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 19.
I20260709 13:21:55.210219 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6f07842d3aa47858df5665874c86f3a" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:21:55.211889 16825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2372981006d4b218215c28964dea623" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:21:55.212519 16825 raft_consensus.cc:3060] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:21:55.213425 16291 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40474 (request call id 1147) took 3697 ms. Trace:
I20260709 13:21:55.213850 16291 rpcz_store.cc:276] 0709 13:21:51.515791 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:52.303446 (+787655us) service_pool.cc:224] Handling call
0709 13:21:52.303715 (+   269us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:21:52.517479 (+213764us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:55.213179 (+2695700us) spinlock_profiling.cc:243] Waited 213 ms on lock 0x7b540010f254. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:21:55.213365 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":489645312}
I20260709 13:21:55.211057 16816 raft_consensus.cc:2410] T 8c11074579b046d99e585c2818e18a2c P cb82cc7bd6db46d980c75237a1402860 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac for term 19 because replica has last-logged OpId of term: 17 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 13.
I20260709 13:21:55.216351 16219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6f07842d3aa47858df5665874c86f3a" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:21:55.221122 16219 raft_consensus.cc:3060] T b6f07842d3aa47858df5665874c86f3a P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:21:55.221443 22774 raft_consensus.cc:515] T 8638ee363273426db1393dd81e5c7b72 P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:55.223567 16265 leader_election.cc:304] T e4dc6b0541ca431ebc757beac00521f1 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:21:55.225934 16265 leader_election.cc:304] T 68a6838d82ef4cac83507f906139d90f P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 647a343a542d4a139fe106c19095129b
I20260709 13:21:55.226732 16818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66f75124215f4ca6bef831e174a792d8" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:55.232998 16818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9944d5e4b8c4109af9955c4f08c9095" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:55.234195 16818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:55.236482 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2372981006d4b218215c28964dea623" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:21:55.237218 16150 raft_consensus.cc:2393] T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 in current term 20: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
I20260709 13:21:55.237906 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d4ab870a05347868956aa8b841b2981" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:21:55.238411 16150 raft_consensus.cc:3060] T 8d4ab870a05347868956aa8b841b2981 P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:21:55.242542 16825 raft_consensus.cc:2468] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 21.
I20260709 13:21:55.230702 22774 leader_election.cc:290] T 8638ee363273426db1393dd81e5c7b72 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 19 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:55.253858 22774 raft_consensus.cc:2804] T 68a6838d82ef4cac83507f906139d90f P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:21:55.254318 22774 raft_consensus.cc:493] T 68a6838d82ef4cac83507f906139d90f P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:21:55.246863 22782 raft_consensus.cc:3060] T e4dc6b0541ca431ebc757beac00521f1 P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:21:55.255470 16291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e9b2bf7011f4ae688a79d7bb5a447b4" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:55.256186 16291 raft_consensus.cc:2468] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 17.
I20260709 13:21:55.256484 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2794af5b99984efebdee564035841cb0" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:55.257117 16298 raft_consensus.cc:2468] T 2794af5b99984efebdee564035841cb0 P 90f3da57796b488c9087a2f523adf045 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 14.
I20260709 13:21:55.257174 16291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31969392ccdb43b491911a319e8bc7e1" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:55.258924 16296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c2095a5f4ad49408dbc7f69c606120d" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 15 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:55.261037 16294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fffab26427e24d398e2695aad40ca134" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:55.230203 16220 raft_consensus.cc:1240] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 18. Current term is 19. Ops: [18.14-18.14]
I20260709 13:21:55.263123 16220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f5f89186204242bf377f7c42b2e2da" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:21:55.263821 16220 raft_consensus.cc:3060] T c2f5f89186204242bf377f7c42b2e2da P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:21:55.231268 16218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0eab25ce4c64e018a041beeda2867db" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:55.266628 16218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a878999c0024d2abe517484180290fc" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:21:55.222901 16223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:55.250967 22496 raft_consensus.cc:493] T 829b4ad8c5a54eababeea1181b960d4b P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:21:55.273113 16223 raft_consensus.cc:2468] T 19baa7bc367844c288bca8f81439c413 P 1ca2725d290e47d68b1a4b609c9816ac [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 20.
I20260709 13:21:55.273195 22496 raft_consensus.cc:515] T 829b4ad8c5a54eababeea1181b960d4b P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:55.274233 16223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5ad4eb281564936a59f41e387a47156" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:55.254233 16192 leader_election.cc:304] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:21:55.278964 22568 raft_consensus.cc:2804] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:21:55.279727 22568 raft_consensus.cc:697] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [term 21 LEADER]: Becoming Leader. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:21:55.280915 22568 consensus_queue.cc:237] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:55.274861 16223 raft_consensus.cc:2468] T e5ad4eb281564936a59f41e387a47156 P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 17.
W20260709 13:21:55.283818 16789 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.720s	user 0.020s	sys 0.016s
I20260709 13:21:55.283931 16223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c2095a5f4ad49408dbc7f69c606120d" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 15 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:55.285043 16223 raft_consensus.cc:2468] T 7c2095a5f4ad49408dbc7f69c606120d P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 14.
I20260709 13:21:55.286303 16223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f2da98864b4425bf7c0055951fdd4b" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
W20260709 13:21:55.284533 16789 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.721s	user 0.020s	sys 0.016s
W20260709 13:21:55.288314 16789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 734244us
I20260709 13:21:55.289743 16789 leader_election.cc:304] T 434bff3082d141838068f323bdb0d1ed P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 647a343a542d4a139fe106c19095129b
I20260709 13:21:55.290475 22500 raft_consensus.cc:2804] T 434bff3082d141838068f323bdb0d1ed P cb82cc7bd6db46d980c75237a1402860 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:21:55.290968 22500 raft_consensus.cc:493] T 434bff3082d141838068f323bdb0d1ed P cb82cc7bd6db46d980c75237a1402860 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:55.291412 22500 raft_consensus.cc:3060] T 434bff3082d141838068f323bdb0d1ed P cb82cc7bd6db46d980c75237a1402860 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:21:55.295790 22526 raft_consensus.cc:493] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:55.296358 22526 raft_consensus.cc:515] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:55.303054 16820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ab0ac8347641e6955e9bce7d90c1a2" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:21:55.315455 16822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e4bfb9090c8460eb14fa7e0d818e7f1" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:55.316329 16822 raft_consensus.cc:2468] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P cb82cc7bd6db46d980c75237a1402860 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 18.
I20260709 13:21:55.306882 16295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0eab25ce4c64e018a041beeda2867db" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:21:55.179313 22775 raft_consensus.cc:515] T 02838c86531d4f98bf58d1c22b8baf77 P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:55.200585 22772 raft_consensus.cc:515] T 8c11074579b046d99e585c2818e18a2c P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:21:55.330021 16823 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49906 (request call id 1251) took 2677 ms (client timeout 1692 ms). Trace:
I20260709 13:21:55.340960 16294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57a744f5aaf944678a63469cbf514015" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:21:55.345372 16295 raft_consensus.cc:3060] T c0eab25ce4c64e018a041beeda2867db P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:21:55.338312 16789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 48635.1us
I20260709 13:21:55.319885 16192 leader_election.cc:304] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:21:55.327195 16222 raft_consensus.cc:2468] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 17.
W20260709 13:21:55.348146 16823 rpcz_store.cc:269] 0709 13:21:52.652790 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:52.653004 (+   214us) service_pool.cc:224] Handling call
0709 13:21:55.329788 (+2676784us) spinlock_profiling.cc:243] Waited 2.08 s on lock 0x7b54005032d4. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:21:55.329915 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4795003648}
I20260709 13:21:55.319065 16146 raft_consensus.cc:2468] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 19.
I20260709 13:21:55.349215 16222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0eab25ce4c64e018a041beeda2867db" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:21:55.348798 16789 leader_election.cc:304] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 647a343a542d4a139fe106c19095129b
I20260709 13:21:55.350026 16146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af57996a8be84d398f0f023e8d08ef77" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
W20260709 13:21:55.320778 16788 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:55.350662 16222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4f65cdf00b748bb963ee6e81ef43b79" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:55.324075 16223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:55.268178 16218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1df93779209748b9860f6d893dd3de53" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:21:55.342902 16143 raft_consensus.cc:2468] T 26354361e9b044cea1c2b86c4882fe92 P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 15.
I20260709 13:21:55.351137 16146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5718c623ebee4d2bb770f0f9ddb2f164" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:21:55.362418 16146 raft_consensus.cc:3060] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:21:55.352080 16294 raft_consensus.cc:2393] T 57a744f5aaf944678a63469cbf514015 P 90f3da57796b488c9087a2f523adf045 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 14: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
I20260709 13:21:55.342937 16150 raft_consensus.cc:2468] T 8d4ab870a05347868956aa8b841b2981 P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 17.
I20260709 13:21:55.356321 22747 raft_consensus.cc:2804] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:21:55.354467 22372 raft_consensus.cc:2804] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:21:55.364107 22747 raft_consensus.cc:493] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Starting leader election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:21:55.364624 22747 raft_consensus.cc:3060] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:21:55.367600 22372 raft_consensus.cc:697] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860 [term 21 LEADER]: Becoming Leader. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:21:55.359619 16291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87147731a68400e8cce9b879daa7c5d" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:55.370321 16291 raft_consensus.cc:2468] T b87147731a68400e8cce9b879daa7c5d P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 18.
I20260709 13:21:55.372761 16295 raft_consensus.cc:2468] T c0eab25ce4c64e018a041beeda2867db P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 18.
I20260709 13:21:55.373322 16295 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48674 (request call id 3291) took 1246 ms. Trace:
I20260709 13:21:55.375746 16295 rpcz_store.cc:276] 0709 13:21:54.127205 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:54.548680 (+421475us) service_pool.cc:224] Handling call
0709 13:21:55.306690 (+758010us) spinlock_profiling.cc:243] Waited 4.34 ms on lock 0x7b5800090098. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69ad8fadb1 00007f69ada3ab71 00007f69ada3aa22 00007f69b6953c09 00007f69b5aadee0 00007f69b5b770ab 00007f69b5b77131 00007f69b5b250a0 00007f69b5b24984 00007f69af09f6d4 00007f69af09f662
0709 13:21:55.373266 (+ 66576us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":9974016}
I20260709 13:21:55.376713 22747 raft_consensus.cc:515] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:55.356870 16218 raft_consensus.cc:3060] T 1df93779209748b9860f6d893dd3de53 P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:21:55.353691 16822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "829b4ad8c5a54eababeea1181b960d4b" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:55.352257 16223 raft_consensus.cc:2468] T ecc1e363230043fe9fef390f0b979f20 P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 19.
I20260709 13:21:55.381896 16223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fffab26427e24d398e2695aad40ca134" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:55.382604 16223 raft_consensus.cc:2468] T fffab26427e24d398e2695aad40ca134 P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 17.
I20260709 13:21:55.369350 16150 raft_consensus.cc:1275] T 23f82224f4564fea93572e8206bad1fe P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 13:21:55.354924 16296 raft_consensus.cc:1240] T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 17. Current term is 18. Ops: []
I20260709 13:21:55.351824 16222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ad5dbfa99ee4b8d9e28be92b3f36db5" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:55.384613 16822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62fe3346cd3c42c4a55e14af9073bca4" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 24 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:55.385336 16822 raft_consensus.cc:2468] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 23.
I20260709 13:21:55.361799 16219 raft_consensus.cc:2468] T b6f07842d3aa47858df5665874c86f3a P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 18.
I20260709 13:21:55.369390 22372 consensus_queue.cc:237] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.14, Last appended by leader: 14, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:55.386861 16192 leader_election.cc:304] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:21:55.347925 22774 raft_consensus.cc:3060] T 68a6838d82ef4cac83507f906139d90f P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:21:55.386494 16219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5718c623ebee4d2bb770f0f9ddb2f164" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
W20260709 13:21:55.354221 16788 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:21:55.391767 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 162060us
I20260709 13:21:55.390333 16789 leader_election.cc:304] T 3884f864a96d428785895e5463e5728f P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:21:55.389838 16148 raft_consensus.cc:2468] T 2794af5b99984efebdee564035841cb0 P 647a343a542d4a139fe106c19095129b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 14.
I20260709 13:21:55.325035 22496 leader_election.cc:290] T 829b4ad8c5a54eababeea1181b960d4b P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:55.393410 16265 leader_election.cc:304] T 5a96373bb0e14dce8a266b2fc6c58507 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:21:55.393695 16219 raft_consensus.cc:2468] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 17.
I20260709 13:21:55.393797 16789 leader_election.cc:304] T 74ca8140f35c4ae29e8a664f6bdfc829 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac
I20260709 13:21:55.396899 16150 raft_consensus.cc:1240] T 57a744f5aaf944678a63469cbf514015 P 647a343a542d4a139fe106c19095129b [term 14 FOLLOWER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 13. Current term is 14. Ops: []
I20260709 13:21:55.396991 16146 raft_consensus.cc:2468] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 18.
I20260709 13:21:55.400177 16790 leader_election.cc:304] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:21:55.388265 22526 leader_election.cc:290] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:55.392822 22573 raft_consensus.cc:2804] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:21:55.410006 16223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c74490c40243a88e62d1760b5cd814" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:55.413081 16822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e4bfb9090c8460eb14fa7e0d818e7f1" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:21:55.413663 16822 raft_consensus.cc:3060] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P cb82cc7bd6db46d980c75237a1402860 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:21:55.411125 16265 leader_election.cc:304] T d16dfad0c3b540e5b4b86492ddf87208 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:21:55.415457 16294 raft_consensus.cc:1240] T 57a744f5aaf944678a63469cbf514015 P 90f3da57796b488c9087a2f523adf045 [term 14 FOLLOWER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 13. Current term is 14. Ops: [13.12-13.12]
I20260709 13:21:55.419034 16220 raft_consensus.cc:2468] T c2f5f89186204242bf377f7c42b2e2da P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 19.
I20260709 13:21:55.419529 16291 raft_consensus.cc:1240] T 31969392ccdb43b491911a319e8bc7e1 P 90f3da57796b488c9087a2f523adf045 [term 16 LEADER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 15. Current term is 16. Ops: []
I20260709 13:21:55.419255 16296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2794af5b99984efebdee564035841cb0" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:55.420941 16296 raft_consensus.cc:2468] T 2794af5b99984efebdee564035841cb0 P 90f3da57796b488c9087a2f523adf045 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 14.
I20260709 13:21:55.410616 22573 raft_consensus.cc:493] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:55.423347 16218 raft_consensus.cc:2468] T 1df93779209748b9860f6d893dd3de53 P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 17.
I20260709 13:21:55.427510 16218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87ac0d1f7e8a419f9bcbc04aee0e067e" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:21:55.428721 16218 raft_consensus.cc:3060] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:21:55.432471 16223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d16dfad0c3b540e5b4b86492ddf87208" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:21:55.433362 16823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434bff3082d141838068f323bdb0d1ed" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:55.433398 16223 raft_consensus.cc:3060] T d16dfad0c3b540e5b4b86492ddf87208 P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:21:55.435014 16820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3884f864a96d428785895e5463e5728f" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:55.435681 16820 raft_consensus.cc:2468] T 3884f864a96d428785895e5463e5728f P cb82cc7bd6db46d980c75237a1402860 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 13.
I20260709 13:21:55.437058 16825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdbd78acb4cf46298e0de8ba0d130e19" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:55.438215 16820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e65676b21b748728ad39668cfe722ec" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:55.415187 22747 leader_election.cc:290] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 19 election: Requested vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
W20260709 13:21:55.439991 16791 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:55.442337 16820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:55.444600 16825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87147731a68400e8cce9b879daa7c5d" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:55.431902 22788 raft_consensus.cc:2749] T 5a96373bb0e14dce8a266b2fc6c58507 P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:21:55.448279 22786 raft_consensus.cc:2749] T 3884f864a96d428785895e5463e5728f P cb82cc7bd6db46d980c75237a1402860 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:21:55.449290 22786 raft_consensus.cc:2804] T 74ca8140f35c4ae29e8a664f6bdfc829 P cb82cc7bd6db46d980c75237a1402860 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:21:55.449724 22786 raft_consensus.cc:493] T 74ca8140f35c4ae29e8a664f6bdfc829 P cb82cc7bd6db46d980c75237a1402860 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:21:55.449909 16218 raft_consensus.cc:2468] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 19.
I20260709 13:21:55.450295 22786 raft_consensus.cc:3060] T 74ca8140f35c4ae29e8a664f6bdfc829 P cb82cc7bd6db46d980c75237a1402860 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:21:55.448396 22788 raft_consensus.cc:2804] T d16dfad0c3b540e5b4b86492ddf87208 P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:21:55.451112 22788 raft_consensus.cc:697] T d16dfad0c3b540e5b4b86492ddf87208 P 90f3da57796b488c9087a2f523adf045 [term 19 LEADER]: Becoming Leader. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:21:55.452291 16218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d4ab870a05347868956aa8b841b2981" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:55.453280 16218 raft_consensus.cc:2468] T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 16.
I20260709 13:21:55.452201 22788 consensus_queue.cc:237] T d16dfad0c3b540e5b4b86492ddf87208 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.10, Last appended by leader: 10, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:21:55.454886 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 43798.8us
I20260709 13:21:55.455889 16265 leader_election.cc:304] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:21:55.459832 22787 raft_consensus.cc:2804] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:21:55.460471 22787 raft_consensus.cc:697] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 90f3da57796b488c9087a2f523adf045 [term 19 LEADER]: Becoming Leader. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:21:55.461459 22787 consensus_queue.cc:237] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 90f3da57796b488c9087a2f523adf045 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.12, Last appended by leader: 12, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:55.468405 16820 raft_consensus.cc:1275] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860 [term 15 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 13:21:55.432994 22573 raft_consensus.cc:3060] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:21:55.410259 22785 consensus_queue.cc:1048] T fffab26427e24d398e2695aad40ca134 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.028s
W20260709 13:21:55.466513 22783 log.cc:927] Time spent T 3884f864a96d428785895e5463e5728f P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
I20260709 13:21:55.480820 22747 consensus_queue.cc:1048] T 746aabf856ab401c9ef5e859371e7b67 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:21:55.439723 22779 raft_consensus.cc:493] T 66ae176961b74813a58bc15c9c4c6f2a P 90f3da57796b488c9087a2f523adf045 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:55.482173 22779 raft_consensus.cc:515] T 66ae176961b74813a58bc15c9c4c6f2a P 90f3da57796b488c9087a2f523adf045 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:55.484635 22779 leader_election.cc:290] T 66ae176961b74813a58bc15c9c4c6f2a P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:55.422860 22505 consensus_queue.cc:1048] T 23f82224f4564fea93572e8206bad1fe P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.044s
I20260709 13:21:55.480932 16820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a378d7f796f94615b3b347119a9880fd" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:55.489578 16820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ab0ac8347641e6955e9bce7d90c1a2" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:55.495378 16223 raft_consensus.cc:2468] T d16dfad0c3b540e5b4b86492ddf87208 P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 19.
I20260709 13:21:55.499703 16299 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48674 (request call id 3285) took 2124 ms. Trace:
I20260709 13:21:55.500138 16299 rpcz_store.cc:276] 0709 13:21:53.375312 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:54.673016 (+1297704us) service_pool.cc:224] Handling call
0709 13:21:54.673348 (+   332us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:21:54.673821 (+   473us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:21:54.673865 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:21:54.674425 (+   560us) log.cc:844] Serialized 26 byte log entry
0709 13:21:55.416428 (+742003us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:21:55.416466 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:55.416532 (+    66us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:21:55.499478 (+ 82946us) raft_consensus.cc:1685] finished
0709 13:21:55.499500 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:21:55.499641 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":894,"raft.run_cpu_time_us":220,"raft.run_wall_time_us":17655,"thread_start_us":196146,"threads_started":1,"wal-append.queue_time_us":216139}
I20260709 13:21:55.505517 16297 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48674 (request call id 3282) took 2357 ms. Trace:
I20260709 13:21:55.506157 16297 rpcz_store.cc:276] 0709 13:21:53.148457 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:53.871771 (+723314us) service_pool.cc:224] Handling call
0709 13:21:53.883569 (+ 11798us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:21:53.884103 (+   534us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:21:53.884149 (+    46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:21:54.926645 (+1042496us) spinlock_profiling.cc:243] Waited 1.79 ms on lock 0x7b1800324a60. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b693ade1 00007f69b69475b1 00007f69ada402a6 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69af458a11 00007f69af468372 00007f69b0b9aa01 00007f69b0b5f43a
0709 13:21:55.264968 (+338323us) spinlock_profiling.cc:243] Waited 893 us on lock 0x7b2c00132978. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69af458a11 00007f69af468372 00007f69b0b9aa01 00007f69b0b5f43a 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2
0709 13:21:55.265629 (+   661us) log.cc:844] Serialized 26 byte log entry
0709 13:21:55.390846 (+125217us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:21:55.390955 (+   109us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:55.391107 (+   152us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:21:55.505236 (+114129us) raft_consensus.cc:1685] finished
0709 13:21:55.505266 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:21:55.505443 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":868,"raft.run_cpu_time_us":136,"raft.run_wall_time_us":4075,"spinlock_wait_cycles":6181504,"thread_start_us":124796,"threads_started":1,"wal-append.queue_time_us":143306}
I20260709 13:21:55.509732 16820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31969392ccdb43b491911a319e8bc7e1" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:55.519933 22568 raft_consensus.cc:493] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:55.520730 22568 raft_consensus.cc:515] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:55.524415 22568 leader_election.cc:290] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:55.527855 16820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:55.529196 22748 leader_election.cc:290] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:55.542904 16820 raft_consensus.cc:2468] T 19baa7bc367844c288bca8f81439c413 P cb82cc7bd6db46d980c75237a1402860 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 20.
I20260709 13:21:55.531817 22500 raft_consensus.cc:515] T 434bff3082d141838068f323bdb0d1ed P cb82cc7bd6db46d980c75237a1402860 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:55.534822 16822 raft_consensus.cc:2468] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P cb82cc7bd6db46d980c75237a1402860 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 19.
I20260709 13:21:55.512032 16151 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46224 (request call id 1061) took 2307 ms. Trace:
I20260709 13:21:55.545006 16151 rpcz_store.cc:276] 0709 13:21:53.205111 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:53.205270 (+   159us) service_pool.cc:224] Handling call
0709 13:21:53.235093 (+ 29823us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:21:53.886435 (+651342us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:21:53.886468 (+    33us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:21:53.886513 (+    45us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:21:53.886550 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:55.511758 (+1625208us) spinlock_profiling.cc:243] Waited 651 ms on lock 0x7b54003d9fd4. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:21:55.511804 (+    46us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:21:55.511960 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1497433472}
I20260709 13:21:55.546232 16192 leader_election.cc:304] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:21:55.536890 16817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ab0ac8347641e6955e9bce7d90c1a2" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
W20260709 13:21:55.547148 16114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:55.548121 16822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5d59407eb545b895d8d922e0dd69fd" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 21 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:55.548669 16817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66f75124215f4ca6bef831e174a792d8" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:55.550137 16822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57a744f5aaf944678a63469cbf514015" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:21:55.552495 16822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87147731a68400e8cce9b879daa7c5d" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:55.553107 16817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9944d5e4b8c4109af9955c4f08c9095" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:21:55.553977 16822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5718c623ebee4d2bb770f0f9ddb2f164" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:55.554474 16817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9944d5e4b8c4109af9955c4f08c9095" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:55.555709 16820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b7d1ab519044eda2d5e04a07242f25" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:55.556391 16223 raft_consensus.cc:1240] T 1df93779209748b9860f6d893dd3de53 P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 16. Current term is 17. Ops: []
I20260709 13:21:55.556624 16820 raft_consensus.cc:2393] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 16: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
I20260709 13:21:55.558509 16820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2794af5b99984efebdee564035841cb0" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:55.559556 16820 raft_consensus.cc:2468] T 2794af5b99984efebdee564035841cb0 P cb82cc7bd6db46d980c75237a1402860 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 14.
I20260709 13:21:55.558936 16789 leader_election.cc:304] T 2794af5b99984efebdee564035841cb0 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:21:55.561385 16822 raft_consensus.cc:2393] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 18: Already voted for candidate cb82cc7bd6db46d980c75237a1402860 in this term.
I20260709 13:21:55.562389 16823 raft_consensus.cc:2393] T 434bff3082d141838068f323bdb0d1ed P cb82cc7bd6db46d980c75237a1402860 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 17: Already voted for candidate cb82cc7bd6db46d980c75237a1402860 in this term.
I20260709 13:21:55.558549 16817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b7d1ab519044eda2d5e04a07242f25" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:55.494849 22786 raft_consensus.cc:515] T 74ca8140f35c4ae29e8a664f6bdfc829 P cb82cc7bd6db46d980c75237a1402860 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:55.563218 16817 raft_consensus.cc:2468] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 16.
I20260709 13:21:55.508980 22506 consensus_queue.cc:1059] T 57a744f5aaf944678a63469cbf514015 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: INVALID_TERM, Last received: 13.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.050s
I20260709 13:21:55.555552 22747 raft_consensus.cc:2804] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:21:55.544618 22775 leader_election.cc:290] T 02838c86531d4f98bf58d1c22b8baf77 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 19 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
W20260709 13:21:55.566046 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 110221us
I20260709 13:21:55.565923 16192 leader_election.cc:304] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:21:55.566924 16265 leader_election.cc:304] T 26354361e9b044cea1c2b86c4882fe92 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:21:55.558281 22772 leader_election.cc:290] T 8c11074579b046d99e585c2818e18a2c P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 19 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:55.568059 16265 leader_election.cc:304] T 8d4ab870a05347868956aa8b841b2981 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:21:55.569813 22797 raft_consensus.cc:2804] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:21:55.571131 22748 raft_consensus.cc:2804] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:21:55.571599 22593 raft_consensus.cc:2804] T 26354361e9b044cea1c2b86c4882fe92 P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:21:55.559820 22545 consensus_queue.cc:1059] T c9944d5e4b8c4109af9955c4f08c9095 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: INVALID_TERM, Last received: 19.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.009s
I20260709 13:21:55.576072 16825 raft_consensus.cc:1240] T 74ca8140f35c4ae29e8a664f6bdfc829 P cb82cc7bd6db46d980c75237a1402860 [term 19 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 18. Current term is 19. Ops: []
I20260709 13:21:55.583981 16223 raft_consensus.cc:1240] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 16. Current term is 17. Ops: [15.10-16.11]
I20260709 13:21:55.572446 22500 leader_election.cc:290] T 434bff3082d141838068f323bdb0d1ed P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 17 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:55.607759 22797 raft_consensus.cc:697] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 [term 18 LEADER]: Becoming Leader. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:21:55.609675 22797 consensus_queue.cc:237] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 16.14, Last appended by leader: 14, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:55.613860 22754 leader_election.cc:290] T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:55.616879 22786 leader_election.cc:290] T 74ca8140f35c4ae29e8a664f6bdfc829 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 19 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:55.572803 22747 raft_consensus.cc:697] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 19 LEADER]: Becoming Leader. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:21:55.604655 22782 raft_consensus.cc:2749] T e4dc6b0541ca431ebc757beac00521f1 P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:21:55.621270 16818 raft_consensus.cc:1275] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860 [term 16 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:21:55.620637 16292 raft_consensus.cc:1240] T e4dc6b0541ca431ebc757beac00521f1 P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 17. Current term is 18. Ops: []
I20260709 13:21:55.583469 22748 raft_consensus.cc:493] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:55.639376 22748 raft_consensus.cc:3060] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:21:55.584582 22506 raft_consensus.cc:2804] T 2794af5b99984efebdee564035841cb0 P cb82cc7bd6db46d980c75237a1402860 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:21:55.636234 16221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d4ab870a05347868956aa8b841b2981" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:21:55.649231 16116 leader_election.cc:304] T 5a959dbecc2648eb85d1cc550f5a3245 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 90f3da57796b488c9087a2f523adf045
I20260709 13:21:55.649490 16223 raft_consensus.cc:1240] T b6f07842d3aa47858df5665874c86f3a P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 17. Current term is 18. Ops: []
I20260709 13:21:55.656736 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a878999c0024d2abe517484180290fc" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:21:55.657636 16293 raft_consensus.cc:3060] T 8a878999c0024d2abe517484180290fc P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:21:55.577039 22372 consensus_queue.cc:1059] T 57a744f5aaf944678a63469cbf514015 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.053s
W20260709 13:21:55.661630 16115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 414180us
I20260709 13:21:55.662639 16115 leader_election.cc:304] T 9c61a39fd4f84424b129c1a6f9ba404d P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:21:55.666360 16221 raft_consensus.cc:3060] T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:21:55.663810 16115 leader_election.cc:304] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:21:55.678349 16293 raft_consensus.cc:2468] T 8a878999c0024d2abe517484180290fc P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 16.
W20260709 13:21:55.680143 16789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 115867us
I20260709 13:21:55.620615 22747 consensus_queue.cc:237] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:55.686249 16221 raft_consensus.cc:2468] T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 17.
I20260709 13:21:55.691062 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434bff3082d141838068f323bdb0d1ed" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:21:55.640825 22506 raft_consensus.cc:493] T 2794af5b99984efebdee564035841cb0 P cb82cc7bd6db46d980c75237a1402860 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:21:55.713037 22688 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.950s	user 0.005s	sys 0.008s
W20260709 13:21:55.715658 22688 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.953s	user 0.005s	sys 0.009s
I20260709 13:21:55.670751 22699 raft_consensus.cc:2804] T 9c61a39fd4f84424b129c1a6f9ba404d P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:21:55.716627 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74ca8140f35c4ae29e8a664f6bdfc829" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:21:55.720320 16297 raft_consensus.cc:3060] T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:21:55.697916 22797 raft_consensus.cc:493] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:55.699815 22630 raft_consensus.cc:2804] T 5a959dbecc2648eb85d1cc550f5a3245 P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:21:55.700572 22506 raft_consensus.cc:3060] T 2794af5b99984efebdee564035841cb0 P cb82cc7bd6db46d980c75237a1402860 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:21:55.720883 16036 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 19baa7bc367844c288bca8f81439c413 with cas_config_opid_index 7: ChangeConfig:REMOVE_PEER failed with leader cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica cb82cc7bd6db46d980c75237a1402860 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 20 committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
W20260709 13:21:55.832064 16036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.ChangeConfig -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 120367us
I20260709 13:21:55.759502 16816 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49906 (request call id 1257) took 2237 ms. Trace:
I20260709 13:21:55.734586 16297 raft_consensus.cc:2468] T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 19.
I20260709 13:21:55.839269 16816 rpcz_store.cc:276] 0709 13:21:53.521774 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:53.524225 (+  2451us) service_pool.cc:224] Handling call
0709 13:21:55.759303 (+2235078us) spinlock_profiling.cc:243] Waited 1.65 s on lock 0x7b540051b154. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:21:55.759441 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3804788480}
I20260709 13:21:55.840648 16291 raft_consensus.cc:3055] T ada5a004ba7c46819051e3ccec71ffdb P 90f3da57796b488c9087a2f523adf045 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:21:55.841190 16291 raft_consensus.cc:740] T ada5a004ba7c46819051e3ccec71ffdb P 90f3da57796b488c9087a2f523adf045 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:21:55.842379 16299 raft_consensus.cc:1275] T 23f82224f4564fea93572e8206bad1fe P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 13:21:55.843128 16291 consensus_queue.cc:260] T ada5a004ba7c46819051e3ccec71ffdb P 90f3da57796b488c9087a2f523adf045 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.14, Last appended by leader: 14, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:55.746505 22748 raft_consensus.cc:515] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:55.847667 16291 raft_consensus.cc:3060] T ada5a004ba7c46819051e3ccec71ffdb P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:21:55.786860 16788 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:55.726917 22630 raft_consensus.cc:493] T 5a959dbecc2648eb85d1cc550f5a3245 P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:55.725271 22573 raft_consensus.cc:515] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:55.761613 22749 leader_election.cc:290] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:55.850683 22630 raft_consensus.cc:3060] T 5a959dbecc2648eb85d1cc550f5a3245 P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:21:55.746721 22774 raft_consensus.cc:515] T 68a6838d82ef4cac83507f906139d90f P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:55.763433 16226 raft_consensus.cc:1240] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac [term 20 FOLLOWER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 19. Current term is 20. Ops: []
I20260709 13:21:55.726725 22797 raft_consensus.cc:515] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:55.861150 16821 raft_consensus.cc:1275] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [term 23 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 21 index: 14. Preceding OpId from leader: term: 23 index: 15. (index mismatch)
I20260709 13:21:55.863765 16293 raft_consensus.cc:1240] T 57a744f5aaf944678a63469cbf514015 P 90f3da57796b488c9087a2f523adf045 [term 14 FOLLOWER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 13. Current term is 14. Ops: [13.12-13.12]
W20260709 13:21:55.865271 16788 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:55.869848 16817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b7d1ab519044eda2d5e04a07242f25" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:21:55.870932 16817 raft_consensus.cc:3060] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:21:55.871220 22774 leader_election.cc:290] T 68a6838d82ef4cac83507f906139d90f P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 18 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:21:55.873682 16816 raft_consensus.cc:1275] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 [term 21 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
I20260709 13:21:55.876296 16789 leader_election.cc:304] T 74ca8140f35c4ae29e8a664f6bdfc829 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:21:55.877890 22733 raft_consensus.cc:2804] T 74ca8140f35c4ae29e8a664f6bdfc829 P cb82cc7bd6db46d980c75237a1402860 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:21:55.878537 22733 raft_consensus.cc:697] T 74ca8140f35c4ae29e8a664f6bdfc829 P cb82cc7bd6db46d980c75237a1402860 [term 19 LEADER]: Becoming Leader. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:21:55.878393 22630 raft_consensus.cc:515] T 5a959dbecc2648eb85d1cc550f5a3245 P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:55.879462 16293 raft_consensus.cc:1240] T c9944d5e4b8c4109af9955c4f08c9095 P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 19. Current term is 20. Ops: []
I20260709 13:21:55.880633 22733 consensus_queue.cc:237] T 74ca8140f35c4ae29e8a664f6bdfc829 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:55.881428 22630 leader_election.cc:290] T 5a959dbecc2648eb85d1cc550f5a3245 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 17 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:55.884733 22797 leader_election.cc:290] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:55.886065 22478 consensus_queue.cc:1048] T 23f82224f4564fea93572e8206bad1fe P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:21:55.886304 22688 consensus_queue.cc:1059] T 57a744f5aaf944678a63469cbf514015 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:21:55.768421 22506 raft_consensus.cc:515] T 2794af5b99984efebdee564035841cb0 P cb82cc7bd6db46d980c75237a1402860 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:55.891501 22506 leader_election.cc:290] T 2794af5b99984efebdee564035841cb0 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 15 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:55.892776 22348 consensus_queue.cc:1048] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:21:55.894623 16817 raft_consensus.cc:2468] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 17.
I20260709 13:21:55.881364 22573 leader_election.cc:290] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 24 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:55.899967 16816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62fe3346cd3c42c4a55e14af9073bca4" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 24 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:21:55.906517 16224 raft_consensus.cc:1240] T d2ab0ac8347641e6955e9bce7d90c1a2 P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 14. Current term is 15. Ops: []
I20260709 13:21:55.908833 16223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2372981006d4b218215c28964dea623" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:21:55.910251 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66ae176961b74813a58bc15c9c4c6f2a" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 24 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:55.911302 16293 raft_consensus.cc:2468] T 66ae176961b74813a58bc15c9c4c6f2a P 90f3da57796b488c9087a2f523adf045 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 23.
I20260709 13:21:55.843743 16822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5d59407eb545b895d8d922e0dd69fd" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:55.910772 16223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5718c623ebee4d2bb770f0f9ddb2f164" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:55.915472 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2794af5b99984efebdee564035841cb0" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:21:55.915791 22545 consensus_queue.cc:1059] T c9944d5e4b8c4109af9955c4f08c9095 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: INVALID_TERM, Last received: 19.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:21:55.916471 16297 raft_consensus.cc:3060] T 2794af5b99984efebdee564035841cb0 P 90f3da57796b488c9087a2f523adf045 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:21:55.919377 16789 leader_election.cc:304] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:21:55.921384 22737 raft_consensus.cc:2804] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:21:55.921870 22737 raft_consensus.cc:493] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:55.922283 22737 raft_consensus.cc:3060] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:21:55.922539 16221 raft_consensus.cc:1275] T ee9ed67006144797b1de6ea8c2013a68 P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 13:21:55.922705 16816 raft_consensus.cc:3060] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:21:55.923941 16221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74ca8140f35c4ae29e8a664f6bdfc829" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:21:55.925413 16192 leader_election.cc:304] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:21:55.925618 16219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66ae176961b74813a58bc15c9c4c6f2a" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 24 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:55.927399 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af57996a8be84d398f0f023e8d08ef77" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:55.928396 22573 raft_consensus.cc:2804] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:21:55.929011 16222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5718c623ebee4d2bb770f0f9ddb2f164" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:21:55.929478 22750 consensus_queue.cc:1059] T c0eab25ce4c64e018a041beeda2867db P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:21:55.929903 16222 raft_consensus.cc:3060] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:21:55.931226 22749 raft_consensus.cc:3055] T c0eab25ce4c64e018a041beeda2867db P 1ca2725d290e47d68b1a4b609c9816ac [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:21:55.931704 22749 raft_consensus.cc:740] T c0eab25ce4c64e018a041beeda2867db P 1ca2725d290e47d68b1a4b609c9816ac [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:21:55.932889 16223 raft_consensus.cc:1275] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 13:21:55.933312 22749 consensus_queue.cc:260] T c0eab25ce4c64e018a041beeda2867db P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:55.935117 22573 raft_consensus.cc:697] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac [term 17 LEADER]: Becoming Leader. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:21:55.940934 16291 raft_consensus.cc:1275] T ada5a004ba7c46819051e3ccec71ffdb P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 19 index: 14. (term mismatch)
I20260709 13:21:55.919100 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87ac0d1f7e8a419f9bcbc04aee0e067e" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:55.936784 22573 consensus_queue.cc:237] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:55.942214 16217 raft_consensus.cc:2393] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 19: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
I20260709 13:21:55.942971 16217 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57604 (request call id 3232) took 1277 ms. Trace:
I20260709 13:21:55.943368 16217 rpcz_store.cc:276] 0709 13:21:54.665911 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:54.666042 (+   131us) service_pool.cc:224] Handling call
0709 13:21:55.942852 (+1276810us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:21:55.944682 16191 leader_election.cc:304] T e4dc6b0541ca431ebc757beac00521f1 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:21:55.946794 16191 leader_election.cc:304] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
I20260709 13:21:55.947230 16300 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48674 (request call id 3278) took 2806 ms. Trace:
I20260709 13:21:55.948830 16824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5718c623ebee4d2bb770f0f9ddb2f164" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:55.950008 22737 raft_consensus.cc:515] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:55.941437 16291 pending_rounds.cc:85] T ada5a004ba7c46819051e3ccec71ffdb P 90f3da57796b488c9087a2f523adf045: Aborting all ops after (but not including) 13
I20260709 13:21:55.954672 22348 raft_consensus.cc:2749] T e4dc6b0541ca431ebc757beac00521f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:21:55.956424 22500 raft_consensus.cc:3055] T 57a744f5aaf944678a63469cbf514015 P cb82cc7bd6db46d980c75237a1402860 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:21:55.963899 16291 pending_rounds.cc:107] T ada5a004ba7c46819051e3ccec71ffdb P 90f3da57796b488c9087a2f523adf045: Aborting uncommitted NO_OP operation due to leader change: 18.14
I20260709 13:21:55.965692 22500 raft_consensus.cc:740] T 57a744f5aaf944678a63469cbf514015 P cb82cc7bd6db46d980c75237a1402860 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:21:55.965865 16291 raft_consensus.cc:2889] T ada5a004ba7c46819051e3ccec71ffdb P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:21:55.966725 16222 raft_consensus.cc:2468] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 18.
I20260709 13:21:55.967190 22500 consensus_queue.cc:260] T 57a744f5aaf944678a63469cbf514015 P cb82cc7bd6db46d980c75237a1402860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:55.959070 16294 raft_consensus.cc:1275] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [term 21 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 20 index: 14. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
I20260709 13:21:55.969877 16297 raft_consensus.cc:2468] T 2794af5b99984efebdee564035841cb0 P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 15.
I20260709 13:21:55.973243 22500 raft_consensus.cc:3060] T 57a744f5aaf944678a63469cbf514015 P cb82cc7bd6db46d980c75237a1402860 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:21:55.973619 22677 raft_consensus.cc:2764] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Leader pre-election decision vote started in defunct term 18: won
I20260709 13:21:55.974108 22797 consensus_queue.cc:1048] T ada5a004ba7c46819051e3ccec71ffdb P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:21:55.973270 22737 leader_election.cc:290] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 24 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:55.960659 16300 rpcz_store.cc:276] 0709 13:21:53.140889 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:53.145986 (+  5097us) service_pool.cc:224] Handling call
0709 13:21:53.146365 (+   379us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:21:53.166985 (+ 20620us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:21:53.291816 (+124831us) log.cc:844] Serialized 14 byte log entry
0709 13:21:53.447996 (+156180us) log.cc:844] Serialized 14 byte log entry
0709 13:21:53.464775 (+ 16779us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:21:53.464830 (+    55us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:21:53.464884 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:55.290743 (+1825859us) spinlock_profiling.cc:243] Waited 2.66 ms on lock 0x7b18000dfae0. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b693ade1 00007f69b69475b1 00007f69ada402a6 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419
0709 13:21:55.946711 (+655968us) spinlock_profiling.cc:243] Waited 20.4 ms on lock 0x7b540024d954. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:21:55.946802 (+    91us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:21:55.947043 (+   241us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":16464,"prepare.queue_time_us":401232,"prepare.run_cpu_time_us":52,"prepare.run_wall_time_us":61,"raft.queue_time_us":17529,"raft.run_cpu_time_us":245,"raft.run_wall_time_us":21913,"spinlock_wait_cycles":52954752,"thread_start_us":213192,"threads_started":2,"wal-append.queue_time_us":494901,"wal-append.run_cpu_time_us":1858,"wal-append.run_wall_time_us":1050837}
I20260709 13:21:55.976151 22731 consensus_queue.cc:1048] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:21:55.978675 16191 leader_election.cc:304] T 7c2095a5f4ad49408dbc7f69c606120d P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
I20260709 13:21:55.980698 16816 raft_consensus.cc:2468] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 24.
I20260709 13:21:55.981760 22731 raft_consensus.cc:2804] T 7c2095a5f4ad49408dbc7f69c606120d P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:21:55.982316 22731 raft_consensus.cc:493] T 7c2095a5f4ad49408dbc7f69c606120d P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Starting leader election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:21:55.982736 22731 raft_consensus.cc:3060] T 7c2095a5f4ad49408dbc7f69c606120d P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:21:55.986760 16190 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:55.992722 16192 leader_election.cc:304] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:21:55.995929 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66ae176961b74813a58bc15c9c4c6f2a" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 24 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:21:55.996622 16300 raft_consensus.cc:3060] T 66ae176961b74813a58bc15c9c4c6f2a P 90f3da57796b488c9087a2f523adf045 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:21:56.000609 16789 leader_election.cc:304] T 2794af5b99984efebdee564035841cb0 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:21:56.003954 22573 raft_consensus.cc:2804] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [term 24 FOLLOWER]: Leader election won for term 24
I20260709 13:21:56.004928 22573 raft_consensus.cc:697] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [term 24 LEADER]: Becoming Leader. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:21:56.006709 22797 raft_consensus.cc:2804] T 2794af5b99984efebdee564035841cb0 P cb82cc7bd6db46d980c75237a1402860 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:21:56.007427 22797 raft_consensus.cc:697] T 2794af5b99984efebdee564035841cb0 P cb82cc7bd6db46d980c75237a1402860 [term 15 LEADER]: Becoming Leader. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:21:56.008929 16191 leader_election.cc:304] T 08c74490c40243a88e62d1760b5cd814 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
I20260709 13:21:56.008793 22797 consensus_queue.cc:237] T 2794af5b99984efebdee564035841cb0 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:56.012804 22701 consensus_queue.cc:1048] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:21:56.013893 22748 leader_election.cc:290] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 17 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:56.015393 22348 consensus_queue.cc:1048] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.026s
I20260709 13:21:56.045027 16821 raft_consensus.cc:1240] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [term 24 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 23. Current term is 24. Ops: [23.15-23.15]
I20260709 13:21:56.048405 22593 raft_consensus.cc:697] T 26354361e9b044cea1c2b86c4882fe92 P 90f3da57796b488c9087a2f523adf045 [term 15 LEADER]: Becoming Leader. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
W20260709 13:21:56.048924 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 480760us
I20260709 13:21:56.049434 22782 raft_consensus.cc:2804] T 8d4ab870a05347868956aa8b841b2981 P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:21:56.050176 22782 raft_consensus.cc:697] T 8d4ab870a05347868956aa8b841b2981 P 90f3da57796b488c9087a2f523adf045 [term 17 LEADER]: Becoming Leader. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:21:56.022825 22754 raft_consensus.cc:2804] T 08c74490c40243a88e62d1760b5cd814 P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:21:56.050856 22754 raft_consensus.cc:493] T 08c74490c40243a88e62d1760b5cd814 P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Starting leader election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
W20260709 13:21:56.026121 16148 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46224 (request call id 1072) took 1478 ms (client timeout 1781 ms). Trace:
I20260709 13:21:56.051450 22754 raft_consensus.cc:3060] T 08c74490c40243a88e62d1760b5cd814 P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:21:56.051751 16148 rpcz_store.cc:269] 0709 13:21:54.547776 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:54.555085 (+  7309us) service_pool.cc:224] Handling call
0709 13:21:56.025503 (+1470418us) spinlock_profiling.cc:243] Waited 827 ms on lock 0x7b540041e1d4. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:21:56.025816 (+   313us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1902275968}
I20260709 13:21:56.052209 16819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a378d7f796f94615b3b347119a9880fd" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:21:56.026906 22747 consensus_queue.cc:1059] T 84f2da98864b4425bf7c0055951fdd4b P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.027s
I20260709 13:21:56.054049 22747 raft_consensus.cc:3055] T 84f2da98864b4425bf7c0055951fdd4b P 1ca2725d290e47d68b1a4b609c9816ac [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:21:56.054431 22747 raft_consensus.cc:740] T 84f2da98864b4425bf7c0055951fdd4b P 1ca2725d290e47d68b1a4b609c9816ac [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:21:56.055603 22747 consensus_queue.cc:260] T 84f2da98864b4425bf7c0055951fdd4b P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:56.057534 22747 raft_consensus.cc:3060] T 84f2da98864b4425bf7c0055951fdd4b P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:21:56.058897 22792 raft_consensus.cc:493] T 8cc959fb7f704fba9009c0595467d074 P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:56.035537 16145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434bff3082d141838068f323bdb0d1ed" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:21:56.059782 22792 raft_consensus.cc:515] T 8cc959fb7f704fba9009c0595467d074 P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:56.061422 16820 raft_consensus.cc:1240] T 434bff3082d141838068f323bdb0d1ed P cb82cc7bd6db46d980c75237a1402860 [term 17 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 16. Current term is 17. Ops: []
I20260709 13:21:56.014520 22573 consensus_queue.cc:237] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 21.14, Last appended by leader: 14, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:56.050177 22593 consensus_queue.cc:237] T 26354361e9b044cea1c2b86c4882fe92 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:56.064674 16147 raft_consensus.cc:1240] T 57a744f5aaf944678a63469cbf514015 P 647a343a542d4a139fe106c19095129b [term 14 FOLLOWER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 13. Current term is 14. Ops: []
I20260709 13:21:56.065932 22782 consensus_queue.cc:237] T 8d4ab870a05347868956aa8b841b2981 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:21:56.067987 16266 connection.cc:256] RPC call timeout handler was delayed by -1.1433002059998216s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.3347s for a response
I20260709 13:21:56.069044 22788 raft_consensus.cc:493] T 5a959dbecc2648eb85d1cc550f5a3245 P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:56.070076 22707 raft_consensus.cc:3055] T 18ad7e3af5504571a69d10c00750a01c P 647a343a542d4a139fe106c19095129b [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:21:56.069792 22788 raft_consensus.cc:515] T 5a959dbecc2648eb85d1cc550f5a3245 P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:56.070510 22707 raft_consensus.cc:740] T 18ad7e3af5504571a69d10c00750a01c P 647a343a542d4a139fe106c19095129b [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:21:56.071354 22792 leader_election.cc:290] T 8cc959fb7f704fba9009c0595467d074 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:21:56.072240 22788 leader_election.cc:290] T 5a959dbecc2648eb85d1cc550f5a3245 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:21:56.071946 22707 consensus_queue.cc:260] T 18ad7e3af5504571a69d10c00750a01c P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } }
I20260709 13:21:56.073472 16148 raft_consensus.cc:3055] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:21:56.073956 16148 raft_consensus.cc:740] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
W20260709 13:21:56.031198 16263 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:56.070397 22709 raft_consensus.cc:493] T 18ad7e3af5504571a69d10c00750a01c P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:21:56.075052 16148 consensus_queue.cc:260] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.14, Last appended by leader: 14, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:56.079188 16148 raft_consensus.cc:3060] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:21:56.088640 22802 log.cc:927] Time spent T 7c2095a5f4ad49408dbc7f69c606120d P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 0.378s	user 0.001s	sys 0.002s
I20260709 13:21:56.088752 16790 leader_election.cc:304] T 434bff3082d141838068f323bdb0d1ed P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:21:56.070071 16266 leader_election.cc:304] T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860
W20260709 13:21:56.091275 22804 log.cc:927] Time spent T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860: Append to log took a long time: real 0.377s	user 0.002s	sys 0.000s
I20260709 13:21:56.091709 22504 raft_consensus.cc:2749] T 434bff3082d141838068f323bdb0d1ed P cb82cc7bd6db46d980c75237a1402860 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:21:56.091981 22439 raft_consensus.cc:2749] T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:21:56.095319 22659 leader_election.cc:290] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:21:56.097426 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c11074579b046d99e585c2818e18a2c" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:56.098469 16298 raft_consensus.cc:2393] T 8c11074579b046d99e585c2818e18a2c P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 in current term 19: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
I20260709 13:21:56.094827 16226 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33050 (request call id 1133) took 1441 ms. Trace:
I20260709 13:21:56.100322 16142 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40844 (request call id 2863) took 1956 ms. Trace:
I20260709 13:21:56.100868 16142 rpcz_store.cc:276] 0709 13:21:54.143370 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:54.143488 (+   118us) service_pool.cc:224] Handling call
0709 13:21:54.143814 (+   326us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:21:54.602767 (+458953us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:56.099843 (+1497076us) spinlock_profiling.cc:243] Waited 427 ms on lock 0x7b540006ccd4. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:21:56.100137 (+   294us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":981183744}
I20260709 13:21:56.080982 16144 raft_consensus.cc:1275] T ee9ed67006144797b1de6ea8c2013a68 P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
W20260709 13:21:56.058918 16265 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:56.100201 16226 rpcz_store.cc:276] 0709 13:21:54.653236 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:54.658025 (+  4789us) service_pool.cc:224] Handling call
0709 13:21:54.658369 (+   344us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:21:55.856080 (+1197711us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:56.094353 (+238273us) spinlock_profiling.cc:243] Waited 1.1 s on lock 0x7b5400138b54. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:21:56.094626 (+   273us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2540846848}
I20260709 13:21:56.071991 22792 raft_consensus.cc:493] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:56.031778 16191 leader_election.cc:304] T 02838c86531d4f98bf58d1c22b8baf77 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 90f3da57796b488c9087a2f523adf045
I20260709 13:21:56.075575 22709 raft_consensus.cc:515] T 18ad7e3af5504571a69d10c00750a01c P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } }
I20260709 13:21:56.104943 22792 raft_consensus.cc:515] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:56.107885 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8638ee363273426db1393dd81e5c7b72" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:21:56.081961 16147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2794af5b99984efebdee564035841cb0" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:21:56.075089 22707 raft_consensus.cc:3060] T 18ad7e3af5504571a69d10c00750a01c P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:21:56.108987 16147 raft_consensus.cc:3060] T 2794af5b99984efebdee564035841cb0 P 647a343a542d4a139fe106c19095129b [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:21:56.085192 22681 leader_election.cc:290] T 8c11074579b046d99e585c2818e18a2c P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:56.110093 22749 raft_consensus.cc:3060] T c0eab25ce4c64e018a041beeda2867db P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:21:56.114917 16226 raft_consensus.cc:3060] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Advancing to term 19
I20260709 13:21:56.126964 22677 raft_consensus.cc:2804] T 02838c86531d4f98bf58d1c22b8baf77 P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:21:56.127779 22677 raft_consensus.cc:697] T 02838c86531d4f98bf58d1c22b8baf77 P 1ca2725d290e47d68b1a4b609c9816ac [term 18 LEADER]: Becoming Leader. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
W20260709 13:21:56.128286 16846 debug-util.cc:398] Leaking SignalData structure 0x7b080028f080 after lost signal to thread 15049
W20260709 13:21:56.130090 16846 debug-util.cc:398] Leaking SignalData structure 0x7b0800348a80 after lost signal to thread 16083
I20260709 13:21:56.131068 22677 consensus_queue.cc:237] T 02838c86531d4f98bf58d1c22b8baf77 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:56.151515 22586 raft_consensus.cc:3055] T c9944d5e4b8c4109af9955c4f08c9095 P cb82cc7bd6db46d980c75237a1402860 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:21:56.152225 22586 raft_consensus.cc:740] T c9944d5e4b8c4109af9955c4f08c9095 P cb82cc7bd6db46d980c75237a1402860 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:21:56.153633 22586 consensus_queue.cc:260] T c9944d5e4b8c4109af9955c4f08c9095 P cb82cc7bd6db46d980c75237a1402860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:56.174103 16151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b7d1ab519044eda2d5e04a07242f25" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:56.174643 16220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c11074579b046d99e585c2818e18a2c" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:21:56.175307 16151 raft_consensus.cc:2468] T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 15.
I20260709 13:21:56.175580 16220 raft_consensus.cc:3060] T 8c11074579b046d99e585c2818e18a2c P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:21:56.178752 16115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 514887us
I20260709 13:21:56.180897 16115 leader_election.cc:304] T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:21:56.181365 22196 consensus_queue.cc:1048] T 3884f864a96d428785895e5463e5728f P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:21:56.182093 16115 leader_election.cc:304] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:21:56.182824 22517 raft_consensus.cc:2804] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:21:56.183481 22517 raft_consensus.cc:697] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b [term 19 LEADER]: Becoming Leader. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:21:56.184937 22517 consensus_queue.cc:237] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:21:56.186211 22799 log.cc:927] Time spent T 9e65676b21b748728ad39668cfe722ec P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.470s	user 0.001s	sys 0.002s
W20260709 13:21:56.187573 16117 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 778194us
I20260709 13:21:56.189189 16222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66ae176961b74813a58bc15c9c4c6f2a" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 24 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:56.191618 22610 raft_consensus.cc:2804] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:21:56.192155 22610 raft_consensus.cc:493] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:56.192399 16115 leader_election.cc:304] T 84f2da98864b4425bf7c0055951fdd4b P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:21:56.192637 22610 raft_consensus.cc:3060] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:21:56.193498 22699 raft_consensus.cc:697] T 9c61a39fd4f84424b129c1a6f9ba404d P 647a343a542d4a139fe106c19095129b [term 19 LEADER]: Becoming Leader. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:21:56.022377 16300 raft_consensus.cc:2468] T 66ae176961b74813a58bc15c9c4c6f2a P 90f3da57796b488c9087a2f523adf045 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 24.
I20260709 13:21:56.216300 22709 leader_election.cc:290] T 18ad7e3af5504571a69d10c00750a01c P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
W20260709 13:21:56.218896 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 180776us
W20260709 13:21:56.178859 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 489806us
W20260709 13:21:56.224802 16846 debug-util.cc:398] Leaking SignalData structure 0x7b0800351aa0 after lost signal to thread 16175
W20260709 13:21:56.226162 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 194580us
W20260709 13:21:56.238546 16266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 168625us
I20260709 13:21:56.245513 16222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02838c86531d4f98bf58d1c22b8baf77" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:21:56.244632 16789 leader_election.cc:304] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 
W20260709 13:21:56.250553 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 162138us
W20260709 13:21:56.252085 16846 debug-util.cc:398] Leaking SignalData structure 0x7b08002949a0 after lost signal to thread 16250
I20260709 13:21:56.253369 22196 raft_consensus.cc:2804] T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Leader pre-election won for term 20
W20260709 13:21:56.253712 16846 debug-util.cc:398] Leaking SignalData structure 0x7b080039e1e0 after lost signal to thread 16324
I20260709 13:21:56.253906 22196 raft_consensus.cc:493] T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:56.254383 22196 raft_consensus.cc:3060] T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:21:56.256027 16225 consensus_meta.cc:287] T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac: Time spent flushing consensus metadata: real 1.586s	user 0.012s	sys 0.007s
I20260709 13:21:56.256435 16225 raft_consensus.cc:2468] T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 19.
W20260709 13:21:56.257023 16225 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57604 (request call id 3236) took 1590 ms (client timeout 1707 ms). Trace:
W20260709 13:21:56.257357 16225 rpcz_store.cc:269] 0709 13:21:54.667076 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:54.668443 (+  1367us) service_pool.cc:224] Handling call
0709 13:21:56.256941 (+1588498us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:21:56.263767 22709 consensus_queue.cc:1048] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:21:56.195106 22699 consensus_queue.cc:237] T 9c61a39fd4f84424b129c1a6f9ba404d P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.9, Last appended by leader: 9, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:56.271332 16265 leader_election.cc:304] T c0eab25ce4c64e018a041beeda2867db P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:21:56.273687 16265 leader_election.cc:304] T 8158bbd17b994d2eb96f560bd0f1b88d P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 647a343a542d4a139fe106c19095129b
W20260709 13:21:56.274897 16117 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:21:56.275417 16115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 92269.1us
I20260709 13:21:56.276181 22457 raft_consensus.cc:493] T 02838c86531d4f98bf58d1c22b8baf77 P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:56.274694 16223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68a6838d82ef4cac83507f906139d90f" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:21:56.264031 16116 leader_election.cc:304] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 90f3da57796b488c9087a2f523adf045
W20260709 13:21:56.275645 16117 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:56.293987 16223 raft_consensus.cc:3060] T 68a6838d82ef4cac83507f906139d90f P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:21:56.277067 16115 leader_election.cc:304] T 8d4ab870a05347868956aa8b841b2981 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:21:56.243927 22586 raft_consensus.cc:3060] T c9944d5e4b8c4109af9955c4f08c9095 P cb82cc7bd6db46d980c75237a1402860 [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:21:56.296620 16846 debug-util.cc:398] Leaking SignalData structure 0x7b080040ab00 after lost signal to thread 16849
I20260709 13:21:56.297030 16147 raft_consensus.cc:2468] T 2794af5b99984efebdee564035841cb0 P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 15.
I20260709 13:21:56.297844 16226 raft_consensus.cc:2468] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 19.
I20260709 13:21:56.301218 22573 raft_consensus.cc:493] T ee9ed67006144797b1de6ea8c2013a68 P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:56.251492 22797 raft_consensus.cc:2804] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 13:21:56.276820 22457 raft_consensus.cc:515] T 02838c86531d4f98bf58d1c22b8baf77 P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:56.305135 16790 leader_election.cc:304] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:21:56.316642 16266 leader_election.cc:304] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
I20260709 13:21:56.311614 22750 raft_consensus.cc:493] T 5a959dbecc2648eb85d1cc550f5a3245 P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:21:56.333829 16151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02838c86531d4f98bf58d1c22b8baf77" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:21:56.311028 22494 raft_consensus.cc:2804] T 84f2da98864b4425bf7c0055951fdd4b P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:21:56.336740 16151 raft_consensus.cc:3060] T 02838c86531d4f98bf58d1c22b8baf77 P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:21:56.322350 22709 raft_consensus.cc:2804] T 8158bbd17b994d2eb96f560bd0f1b88d P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:21:56.324218 22439 raft_consensus.cc:2749] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:21:56.321242 22797 raft_consensus.cc:697] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [term 24 LEADER]: Becoming Leader. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
W20260709 13:21:56.344873 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 71337.3us
I20260709 13:21:56.316733 22348 consensus_queue.cc:1059] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: INVALID_TERM, Last received: 21.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.024s
W20260709 13:21:56.349188 16115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 72282us
W20260709 13:21:56.356508 16791 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:21:56.357232 16266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 54652.8us
I20260709 13:21:56.357314 22645 raft_consensus.cc:2749] T c0eab25ce4c64e018a041beeda2867db P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:21:56.359153 16266 leader_election.cc:304] T 23f82224f4564fea93572e8206bad1fe P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
I20260709 13:21:56.362617 16220 raft_consensus.cc:2468] T 8c11074579b046d99e585c2818e18a2c P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 19.
I20260709 13:21:56.363055 22543 raft_consensus.cc:493] T 26354361e9b044cea1c2b86c4882fe92 P cb82cc7bd6db46d980c75237a1402860 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:56.363461 22537 raft_consensus.cc:493] T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:56.363901 22673 consensus_queue.cc:1048] T ee9ed67006144797b1de6ea8c2013a68 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 10, Time since last communication: 0.053s
I20260709 13:21:56.366379 22541 raft_consensus.cc:2764] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 [term 17 FOLLOWER]: Leader pre-election decision vote started in defunct term 16: won
I20260709 13:21:56.313225 22573 raft_consensus.cc:515] T ee9ed67006144797b1de6ea8c2013a68 P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:56.367477 22709 raft_consensus.cc:697] T 8158bbd17b994d2eb96f560bd0f1b88d P 90f3da57796b488c9087a2f523adf045 [term 17 LEADER]: Becoming Leader. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:21:56.398352 22830 raft_consensus.cc:2804] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:21:56.398869 22709 consensus_queue.cc:237] T 8158bbd17b994d2eb96f560bd0f1b88d P 90f3da57796b488c9087a2f523adf045 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.11, Last appended by leader: 11, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:56.402871 16265 leader_election.cc:304] T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 647a343a542d4a139fe106c19095129b
I20260709 13:21:56.403920 22731 raft_consensus.cc:515] T 7c2095a5f4ad49408dbc7f69c606120d P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:56.404740 16265 leader_election.cc:304] T 1df93779209748b9860f6d893dd3de53 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 647a343a542d4a139fe106c19095129b
W20260709 13:21:56.406172 16114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:56.406539 22731 leader_election.cc:290] T 7c2095a5f4ad49408dbc7f69c606120d P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 15 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:56.406658 22543 raft_consensus.cc:515] T 26354361e9b044cea1c2b86c4882fe92 P cb82cc7bd6db46d980c75237a1402860 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:56.411422 22673 consensus_queue.cc:1243] T ee9ed67006144797b1de6ea8c2013a68 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Peer 647a343a542d4a139fe106c19095129b log is divergent from this leader: its last log entry 16.13 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 13:21:56.413414 16148 raft_consensus.cc:1275] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b [term 21 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 20 index: 14. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
I20260709 13:21:56.381654 16147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68a6838d82ef4cac83507f906139d90f" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:21:56.404343 22787 raft_consensus.cc:2804] T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:21:56.415761 22537 raft_consensus.cc:515] T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:56.416468 22787 raft_consensus.cc:697] T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045 [term 18 LEADER]: Becoming Leader. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:21:56.416867 22196 raft_consensus.cc:515] T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:56.417557 22787 consensus_queue.cc:237] T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:56.418378 22537 leader_election.cc:290] T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
W20260709 13:21:56.419824 16192 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 62874.5us
I20260709 13:21:56.418975 22537 raft_consensus.cc:2764] T 8d4ab870a05347868956aa8b841b2981 P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Leader pre-election decision vote started in defunct term 16: won
I20260709 13:21:56.353159 22630 raft_consensus.cc:493] T 23f82224f4564fea93572e8206bad1fe P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:21:56.452605 22830 raft_consensus.cc:697] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b [term 18 LEADER]: Becoming Leader. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:21:56.388305 22782 raft_consensus.cc:2749] T 23f82224f4564fea93572e8206bad1fe P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:21:56.366864 22610 raft_consensus.cc:515] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:21:56.468943 16117 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 104941us
I20260709 13:21:56.419150 22196 leader_election.cc:290] T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 20 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:56.435071 22494 raft_consensus.cc:697] T 84f2da98864b4425bf7c0055951fdd4b P 647a343a542d4a139fe106c19095129b [term 15 LEADER]: Becoming Leader. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:21:56.476465 16142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26354361e9b044cea1c2b86c4882fe92" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
W20260709 13:21:56.482656 16266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 123574us
I20260709 13:21:56.471832 22673 raft_consensus.cc:493] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:56.479007 22593 raft_consensus.cc:2804] T 1df93779209748b9860f6d893dd3de53 P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:21:56.493425 16148 raft_consensus.cc:1275] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 16 index: 14. Preceding OpId from leader: term: 18 index: 15. (index mismatch)
I20260709 13:21:56.504324 22494 consensus_queue.cc:237] T 84f2da98864b4425bf7c0055951fdd4b P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:56.442610 22750 raft_consensus.cc:515] T 5a959dbecc2648eb85d1cc550f5a3245 P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:21:56.492951 16115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 78110us
W20260709 13:21:56.514042 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 74977.4us
I20260709 13:21:56.517835 16142 raft_consensus.cc:2393] T 26354361e9b044cea1c2b86c4882fe92 P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 in current term 15: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
I20260709 13:21:56.458675 22748 consensus_queue.cc:1059] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: INVALID_TERM, Last received: 18.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.039s
I20260709 13:21:56.485478 22630 raft_consensus.cc:515] T 23f82224f4564fea93572e8206bad1fe P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:56.509889 22593 raft_consensus.cc:697] T 1df93779209748b9860f6d893dd3de53 P 90f3da57796b488c9087a2f523adf045 [term 17 LEADER]: Becoming Leader. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:21:56.524190 16266 leader_election.cc:304] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
I20260709 13:21:56.510385 22749 raft_consensus.cc:493] T 5a96373bb0e14dce8a266b2fc6c58507 P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
W20260709 13:21:56.549145 22819 log.cc:927] Time spent T 8d4ab870a05347868956aa8b841b2981 P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.198s	user 0.001s	sys 0.000s
I20260709 13:21:56.550240 16115 leader_election.cc:304] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:21:56.519951 22792 leader_election.cc:290] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:56.551522 22740 raft_consensus.cc:493] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:56.552981 22610 leader_election.cc:290] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 18 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:56.497006 22673 raft_consensus.cc:515] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:56.378868 16223 raft_consensus.cc:2468] T 68a6838d82ef4cac83507f906139d90f P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 18.
I20260709 13:21:56.553676 16150 raft_consensus.cc:1217] T ee9ed67006144797b1de6ea8c2013a68 P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Deduplicated request from leader. Original: 11.10->[12.11-19.15]   Dedup: 13.12->[15.13-19.15]
I20260709 13:21:56.521823 22738 consensus_queue.cc:1048] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.062s
I20260709 13:21:56.579320 22740 raft_consensus.cc:515] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:56.490973 22838 consensus_queue.cc:1048] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.059s
I20260709 13:21:56.578976 22749 raft_consensus.cc:515] T 5a96373bb0e14dce8a266b2fc6c58507 P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:56.578410 22750 leader_election.cc:290] T 5a959dbecc2648eb85d1cc550f5a3245 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:56.578773 22673 leader_election.cc:290] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
W20260709 13:21:56.583748 22813 log.cc:927] Time spent T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860: Append to log took a long time: real 0.088s	user 0.000s	sys 0.000s
I20260709 13:21:56.574841 16150 pending_rounds.cc:85] T ee9ed67006144797b1de6ea8c2013a68 P 647a343a542d4a139fe106c19095129b: Aborting all ops after (but not including) 12
I20260709 13:21:56.584288 16150 pending_rounds.cc:107] T ee9ed67006144797b1de6ea8c2013a68 P 647a343a542d4a139fe106c19095129b: Aborting uncommitted NO_OP operation due to leader change: 16.13
I20260709 13:21:56.584350 16115 leader_election.cc:304] T 2794af5b99984efebdee564035841cb0 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:21:56.584632 16150 raft_consensus.cc:2889] T ee9ed67006144797b1de6ea8c2013a68 P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:21:56.522333 22797 consensus_queue.cc:237] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 21.14, Last appended by leader: 14, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:21:56.586125 22824 log.cc:927] Time spent T b87147731a68400e8cce9b879daa7c5d P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.096s	user 0.000s	sys 0.000s
I20260709 13:21:56.586519 16115 leader_election.cc:304] T b87147731a68400e8cce9b879daa7c5d P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:21:56.566887 16151 raft_consensus.cc:2468] T 02838c86531d4f98bf58d1c22b8baf77 P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 19.
I20260709 13:21:56.595907 16294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26354361e9b044cea1c2b86c4882fe92" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:56.583679 22749 leader_election.cc:290] T 5a96373bb0e14dce8a266b2fc6c58507 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:56.578933 22457 leader_election.cc:290] T 02838c86531d4f98bf58d1c22b8baf77 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:56.624322 16299 raft_consensus.cc:1275] T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 13:21:56.596505 22573 leader_election.cc:290] T ee9ed67006144797b1de6ea8c2013a68 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:56.612031 22707 raft_consensus.cc:2764] T 2794af5b99984efebdee564035841cb0 P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Leader pre-election decision vote started in defunct term 14: won
W20260709 13:21:56.623730 16190 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.14.177.131:41389 timed out after 2.230s (SENT) [suppressed 23 similar messages]
I20260709 13:21:56.621162 22792 raft_consensus.cc:2749] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
W20260709 13:21:56.643764 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 379931us
I20260709 13:21:56.644642 16115 leader_election.cc:304] T c0eab25ce4c64e018a041beeda2867db P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:21:56.537513 22830 consensus_queue.cc:237] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } }
I20260709 13:21:56.589018 22593 consensus_queue.cc:237] T 1df93779209748b9860f6d893dd3de53 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:21:56.650215 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 104315us
W20260709 13:21:56.656809 16190 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:56.633618 22833 raft_consensus.cc:2804] T b87147731a68400e8cce9b879daa7c5d P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:21:56.669582 22735 raft_consensus.cc:2804] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:21:56.670066 22735 raft_consensus.cc:493] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:56.670406 22735 raft_consensus.cc:3060] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:21:56.672317 22754 consensus_meta.cc:287] T 08c74490c40243a88e62d1760b5cd814 P 1ca2725d290e47d68b1a4b609c9816ac: Time spent flushing consensus metadata: real 0.620s	user 0.013s	sys 0.004s
I20260709 13:21:56.672780 22754 raft_consensus.cc:515] T 08c74490c40243a88e62d1760b5cd814 P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:56.661785 22833 raft_consensus.cc:493] T b87147731a68400e8cce9b879daa7c5d P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Starting leader election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:21:56.673638 22833 raft_consensus.cc:3060] T b87147731a68400e8cce9b879daa7c5d P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:21:56.675952 22828 log.cc:927] Time spent T e5ad4eb281564936a59f41e387a47156 P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 0.186s	user 0.001s	sys 0.000s
I20260709 13:21:56.676559 16300 raft_consensus.cc:1275] T 2794af5b99984efebdee564035841cb0 P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 13:21:56.677685 16294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31969392ccdb43b491911a319e8bc7e1" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
W20260709 13:21:56.678766 16117 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 63326.6us
I20260709 13:21:56.680431 22676 consensus_queue.cc:1048] T 74ca8140f35c4ae29e8a664f6bdfc829 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:21:56.681176 22676 consensus_queue.cc:1243] T 74ca8140f35c4ae29e8a664f6bdfc829 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Peer 90f3da57796b488c9087a2f523adf045 log is divergent from this leader: its last log entry 16.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 11
W20260709 13:21:56.682524 16115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 61502us
I20260709 13:21:56.685030 16115 leader_election.cc:304] T 8a878999c0024d2abe517484180290fc P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
W20260709 13:21:56.694785 16266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 179315us
I20260709 13:21:56.697619 16266 leader_election.cc:304] T 2794af5b99984efebdee564035841cb0 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 647a343a542d4a139fe106c19095129b
I20260709 13:21:56.680955 16117 leader_election.cc:304] T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860
I20260709 13:21:56.706316 16148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31969392ccdb43b491911a319e8bc7e1" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:56.707015 16819 raft_consensus.cc:1275] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P cb82cc7bd6db46d980c75237a1402860 [term 19 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 13:21:56.690752 22677 raft_consensus.cc:3055] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [term 23 LEADER]: Stepping down as leader of term 23
I20260709 13:21:56.706212 16823 raft_consensus.cc:1240] T 74ca8140f35c4ae29e8a664f6bdfc829 P cb82cc7bd6db46d980c75237a1402860 [term 19 LEADER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 18. Current term is 19. Ops: []
I20260709 13:21:56.715698 16820 raft_consensus.cc:1275] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [term 24 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 21 index: 14. Preceding OpId from leader: term: 24 index: 15. (index mismatch)
I20260709 13:21:56.722469 16790 leader_election.cc:304] T 26354361e9b044cea1c2b86c4882fe92 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:21:56.728483 22610 raft_consensus.cc:2804] T 8a878999c0024d2abe517484180290fc P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:21:56.734807 16148 raft_consensus.cc:1275] T 2794af5b99984efebdee564035841cb0 P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 13:21:56.715466 22677 raft_consensus.cc:740] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:21:56.729234 16295 pending_rounds.cc:85] T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045: Aborting all ops after (but not including) 11
I20260709 13:21:56.737645 16295 pending_rounds.cc:107] T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045: Aborting uncommitted NO_OP operation due to leader change: 16.12
I20260709 13:21:56.737936 16295 raft_consensus.cc:2889] T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:21:56.735294 22837 consensus_queue.cc:1048] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.013s
I20260709 13:21:56.740463 16825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee9ed67006144797b1de6ea8c2013a68" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
W20260709 13:21:56.742453 16117 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 61575.5us
I20260709 13:21:56.743124 22508 consensus_queue.cc:1048] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.016s
I20260709 13:21:56.743533 16117 leader_election.cc:304] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:21:56.744621 16117 leader_election.cc:304] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860
I20260709 13:21:56.745143 22356 raft_consensus.cc:2749] T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:21:56.742590 16819 raft_consensus.cc:1275] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 [term 17 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
W20260709 13:21:56.749610 22827 log.cc:927] Time spent T 23f82224f4564fea93572e8206bad1fe P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 0.080s	user 0.002s	sys 0.000s
I20260709 13:21:56.751649 16143 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40830 (request call id 1968) took 1515 ms. Trace:
I20260709 13:21:56.752259 22543 leader_election.cc:290] T 26354361e9b044cea1c2b86c4882fe92 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:56.723794 22792 raft_consensus.cc:2764] T 2794af5b99984efebdee564035841cb0 P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Leader pre-election decision vote started in defunct term 14: won
I20260709 13:21:56.753521 22457 raft_consensus.cc:2804] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:21:56.754032 22457 raft_consensus.cc:493] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:56.754438 22457 raft_consensus.cc:3060] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:21:56.755451 22610 raft_consensus.cc:697] T 8a878999c0024d2abe517484180290fc P 647a343a542d4a139fe106c19095129b [term 16 LEADER]: Becoming Leader. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:21:56.756726 22610 consensus_queue.cc:237] T 8a878999c0024d2abe517484180290fc P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:56.758249 16117 leader_election.cc:304] T 57a744f5aaf944678a63469cbf514015 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860
I20260709 13:21:56.726183 22196 raft_consensus.cc:2804] T c0eab25ce4c64e018a041beeda2867db P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:21:56.739482 22659 raft_consensus.cc:3060] T 26354361e9b044cea1c2b86c4882fe92 P cb82cc7bd6db46d980c75237a1402860 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:21:56.739217 22754 leader_election.cc:290] T 08c74490c40243a88e62d1760b5cd814 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 17 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:56.752090 16143 rpcz_store.cc:276] 0709 13:21:55.236155 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:55.360611 (+124456us) service_pool.cc:224] Handling call
0709 13:21:55.360898 (+   287us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:21:56.179657 (+818759us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:21:56.179704 (+    47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:21:56.180345 (+   641us) log.cc:844] Serialized 26 byte log entry
0709 13:21:56.220382 (+ 40037us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:21:56.220427 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:56.244392 (+ 23965us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:21:56.751257 (+506865us) raft_consensus.cc:1685] finished
0709 13:21:56.751289 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:21:56.751459 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":818140,"raft.queue_time_us":829615,"raft.run_cpu_time_us":180,"raft.run_wall_time_us":180,"thread_start_us":39514,"threads_started":1,"wal-append.queue_time_us":468577}
I20260709 13:21:56.721186 22537 consensus_queue.cc:1059] T b6f07842d3aa47858df5665874c86f3a P 647a343a542d4a139fe106c19095129b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: INVALID_TERM, Last received: 17.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.044s
I20260709 13:21:56.775480 16223 raft_consensus.cc:1275] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 16 index: 14. Preceding OpId from leader: term: 18 index: 15. (index mismatch)
I20260709 13:21:56.761871 22707 raft_consensus.cc:2749] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:21:56.778895 16266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 81379.8us
I20260709 13:21:56.782528 16220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c11074579b046d99e585c2818e18a2c" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:56.783655 16220 raft_consensus.cc:2393] T 8c11074579b046d99e585c2818e18a2c P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 in current term 19: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
I20260709 13:21:56.754912 22833 raft_consensus.cc:515] T b87147731a68400e8cce9b879daa7c5d P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:56.796721 16223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b7d1ab519044eda2d5e04a07242f25" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:56.767402 22348 consensus_queue.cc:1048] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.022s
I20260709 13:21:56.765079 22750 consensus_queue.cc:1048] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.024s
I20260709 13:21:56.770761 22747 consensus_queue.cc:1048] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.029s
I20260709 13:21:56.768107 22840 consensus_queue.cc:1059] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: INVALID_TERM, Last received: 18.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.036s
I20260709 13:21:56.792747 22196 raft_consensus.cc:697] T c0eab25ce4c64e018a041beeda2867db P 647a343a542d4a139fe106c19095129b [term 18 LEADER]: Becoming Leader. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:21:56.805907 16790 leader_election.cc:304] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:21:56.796015 22500 raft_consensus.cc:493] T 8638ee363273426db1393dd81e5c7b72 P cb82cc7bd6db46d980c75237a1402860 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:56.842008 16218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66ae176961b74813a58bc15c9c4c6f2a" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 24 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:21:56.861858 16266 leader_election.cc:304] T 3884f864a96d428785895e5463e5728f P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 647a343a542d4a139fe106c19095129b
I20260709 13:21:56.847394 16142 raft_consensus.cc:1240] T b87147731a68400e8cce9b879daa7c5d P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 18. Current term is 19. Ops: []
I20260709 13:21:56.799829 22837 raft_consensus.cc:2749] T 57a744f5aaf944678a63469cbf514015 P 647a343a542d4a139fe106c19095129b [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:21:56.790668 22543 consensus_queue.cc:1059] T d2ab0ac8347641e6955e9bce7d90c1a2 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: INVALID_TERM, Last received: 14.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.055s
I20260709 13:21:56.881084 16265 leader_election.cc:304] T 8638ee363273426db1393dd81e5c7b72 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:21:56.841825 22537 raft_consensus.cc:3055] T b6f07842d3aa47858df5665874c86f3a P 647a343a542d4a139fe106c19095129b [term 17 LEADER]: Stepping down as leader of term 17
W20260709 13:21:56.860204 22813 log.cc:927] Time spent T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
W20260709 13:21:56.900650 16117 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 149896us
W20260709 13:21:56.902871 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 120343us
W20260709 13:21:56.909834 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 74449.5us
I20260709 13:21:56.793550 22735 raft_consensus.cc:515] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:56.868307 22508 raft_consensus.cc:2749] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 [term 21 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:21:56.884749 22840 consensus_queue.cc:1243] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Peer cb82cc7bd6db46d980c75237a1402860 log is divergent from this leader: its last log entry 19.14 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 13
I20260709 13:21:56.921195 16116 leader_election.cc:304] T e5ad4eb281564936a59f41e387a47156 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 90f3da57796b488c9087a2f523adf045
I20260709 13:21:56.919890 16265 leader_election.cc:304] T 8c11074579b046d99e585c2818e18a2c P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:21:56.795980 22677 consensus_queue.cc:260] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 23.15, Last appended by leader: 15, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:56.904062 22537 raft_consensus.cc:740] T b6f07842d3aa47858df5665874c86f3a P 647a343a542d4a139fe106c19095129b [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:21:56.925679 16117 leader_election.cc:304] T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860
W20260709 13:21:56.929898 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 150214us
I20260709 13:21:56.919296 22782 raft_consensus.cc:2804] T 8638ee363273426db1393dd81e5c7b72 P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:21:56.885274 22500 raft_consensus.cc:515] T 8638ee363273426db1393dd81e5c7b72 P cb82cc7bd6db46d980c75237a1402860 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:21:56.934099 16115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 211046us
I20260709 13:21:56.916471 22645 raft_consensus.cc:2804] T 3884f864a96d428785895e5463e5728f P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:21:56.934939 16265 leader_election.cc:304] T 68a6838d82ef4cac83507f906139d90f P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
W20260709 13:21:56.970626 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 54198.3us
I20260709 13:21:56.955142 22787 raft_consensus.cc:2804] T 8c11074579b046d99e585c2818e18a2c P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:21:56.955657 22645 raft_consensus.cc:493] T 3884f864a96d428785895e5463e5728f P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 647a343a542d4a139fe106c19095129b)
W20260709 13:21:56.979296 16266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 144178us
I20260709 13:21:56.934532 22672 consensus_queue.cc:1048] T 2794af5b99984efebdee564035841cb0 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.189s
I20260709 13:21:56.969683 22837 raft_consensus.cc:2804] T e5ad4eb281564936a59f41e387a47156 P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:21:56.972762 22593 raft_consensus.cc:2804] T 68a6838d82ef4cac83507f906139d90f P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:21:56.999259 16790 leader_election.cc:304] T af57996a8be84d398f0f023e8d08ef77 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:21:57.004487 16266 leader_election.cc:304] T cdbd78acb4cf46298e0de8ba0d130e19 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860
I20260709 13:21:56.989286 22645 raft_consensus.cc:3060] T 3884f864a96d428785895e5463e5728f P 90f3da57796b488c9087a2f523adf045 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:21:57.020115 22586 consensus_meta.cc:287] T c9944d5e4b8c4109af9955c4f08c9095 P cb82cc7bd6db46d980c75237a1402860: Time spent flushing consensus metadata: real 0.710s	user 0.003s	sys 0.009s
I20260709 13:21:56.981863 22196 consensus_queue.cc:237] T c0eab25ce4c64e018a041beeda2867db P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:21:57.024255 16115 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:57.026049 16143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2372981006d4b218215c28964dea623" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:21:57.026721 16143 raft_consensus.cc:3060] T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:21:57.027302 16116 leader_election.cc:304] T 7c2095a5f4ad49408dbc7f69c606120d P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 90f3da57796b488c9087a2f523adf045
I20260709 13:21:57.028157 22500 leader_election.cc:290] T 8638ee363273426db1393dd81e5c7b72 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
W20260709 13:21:57.029109 16036 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 19baa7bc367844c288bca8f81439c413 with cas_config_opid_index 7: ChangeConfig:REMOVE_PEER failed with leader cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica cb82cc7bd6db46d980c75237a1402860 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 20 committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:21:56.972142 22508 consensus_queue.cc:1048] T ee9ed67006144797b1de6ea8c2013a68 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 12, Time since last communication: 0.033s
I20260709 13:21:57.030649 16116 leader_election.cc:304] T ecc1e363230043fe9fef390f0b979f20 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: cb82cc7bd6db46d980c75237a1402860
W20260709 13:21:57.010183 16117 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 84635.8us
I20260709 13:21:57.034267 16117 leader_election.cc:304] T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860
I20260709 13:21:57.034410 22833 leader_election.cc:290] T b87147731a68400e8cce9b879daa7c5d P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 19 election: Requested vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:57.035331 16142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f5f89186204242bf377f7c42b2e2da" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
W20260709 13:21:57.036958 16191 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:57.039849 22749 consensus_queue.cc:1059] T 8a878999c0024d2abe517484180290fc P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:21:57.040828 22749 raft_consensus.cc:3055] T 8a878999c0024d2abe517484180290fc P 1ca2725d290e47d68b1a4b609c9816ac [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:21:57.040683 22834 consensus_queue.cc:1059] T 434bff3082d141838068f323bdb0d1ed P 647a343a542d4a139fe106c19095129b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:21:57.041311 22749 raft_consensus.cc:740] T 8a878999c0024d2abe517484180290fc P 1ca2725d290e47d68b1a4b609c9816ac [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:21:57.042347 22749 consensus_queue.cc:260] T 8a878999c0024d2abe517484180290fc P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:21:57.010934 22825 log.cc:927] Time spent T 02838c86531d4f98bf58d1c22b8baf77 P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 0.344s	user 0.005s	sys 0.000s
I20260709 13:21:56.978477 22537 consensus_queue.cc:260] T b6f07842d3aa47858df5665874c86f3a P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:57.069931 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fffab26427e24d398e2695aad40ca134" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
W20260709 13:21:57.072135 22848 log.cc:927] Time spent T fffab26427e24d398e2695aad40ca134 P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 0.104s	user 0.001s	sys 0.000s
I20260709 13:21:57.072631 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4dc6b0541ca431ebc757beac00521f1" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:57.073284 16300 raft_consensus.cc:2468] T e4dc6b0541ca431ebc757beac00521f1 P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 18.
I20260709 13:21:57.075269 16294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a959dbecc2648eb85d1cc550f5a3245" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:21:57.075922 16294 raft_consensus.cc:3060] T 5a959dbecc2648eb85d1cc550f5a3245 P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:21:57.075871 16144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cc959fb7f704fba9009c0595467d074" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:57.076902 16144 raft_consensus.cc:2468] T 8cc959fb7f704fba9009c0595467d074 P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 16.
I20260709 13:21:57.040092 22829 raft_consensus.cc:3060] T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:21:57.032404 22855 raft_consensus.cc:2804] T 7c2095a5f4ad49408dbc7f69c606120d P 647a343a542d4a139fe106c19095129b [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:21:57.079489 22855 raft_consensus.cc:493] T 7c2095a5f4ad49408dbc7f69c606120d P 647a343a542d4a139fe106c19095129b [term 14 FOLLOWER]: Starting leader election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:21:57.079937 22855 raft_consensus.cc:3060] T 7c2095a5f4ad49408dbc7f69c606120d P 647a343a542d4a139fe106c19095129b [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:21:57.008754 22857 raft_consensus.cc:2749] T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:21:57.011760 22842 log.cc:927] Time spent T fffab26427e24d398e2695aad40ca134 P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 0.151s	user 0.001s	sys 0.000s
W20260709 13:21:57.084088 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 104689us
I20260709 13:21:57.014564 16266 leader_election.cc:304] T 9e65676b21b748728ad39668cfe722ec P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
I20260709 13:21:57.048341 16143 raft_consensus.cc:2468] T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 21.
I20260709 13:21:57.069885 22622 raft_consensus.cc:2749] T af57996a8be84d398f0f023e8d08ef77 P cb82cc7bd6db46d980c75237a1402860 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:21:57.070621 22735 leader_election.cc:290] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 18 election: Requested vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:57.076268 22707 raft_consensus.cc:2804] T ecc1e363230043fe9fef390f0b979f20 P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:21:57.068050 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4f65cdf00b748bb963ee6e81ef43b79" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:21:57.089612 16819 raft_consensus.cc:1240] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [term 24 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 23. Current term is 24. Ops: [23.15-23.15]
I20260709 13:21:56.998008 22837 raft_consensus.cc:493] T e5ad4eb281564936a59f41e387a47156 P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Starting leader election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:21:57.095134 16298 raft_consensus.cc:3060] T c4f65cdf00b748bb963ee6e81ef43b79 P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:21:57.095479 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 64904.4us
I20260709 13:21:57.026916 22740 leader_election.cc:290] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:57.040782 22593 raft_consensus.cc:697] T 68a6838d82ef4cac83507f906139d90f P 90f3da57796b488c9087a2f523adf045 [term 18 LEADER]: Becoming Leader. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:21:57.102506 22829 raft_consensus.cc:2749] T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:21:57.095396 22787 raft_consensus.cc:697] T 8c11074579b046d99e585c2818e18a2c P 90f3da57796b488c9087a2f523adf045 [term 19 LEADER]: Becoming Leader. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:21:57.105681 22749 raft_consensus.cc:3060] T 8a878999c0024d2abe517484180290fc P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:21:57.117173 16151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a959dbecc2648eb85d1cc550f5a3245" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:57.107591 22457 raft_consensus.cc:515] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [term 21 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:57.110119 22659 raft_consensus.cc:2749] T 26354361e9b044cea1c2b86c4882fe92 P cb82cc7bd6db46d980c75237a1402860 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:21:57.093531 22707 raft_consensus.cc:493] T ecc1e363230043fe9fef390f0b979f20 P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Starting leader election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:21:57.123733 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ad5dbfa99ee4b8d9e28be92b3f36db5" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:21:57.113193 16299 raft_consensus.cc:1240] T 5a96373bb0e14dce8a266b2fc6c58507 P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 14. Current term is 15. Ops: []
I20260709 13:21:57.113306 16143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18ad7e3af5504571a69d10c00750a01c" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:57.124961 16143 raft_consensus.cc:2468] T 18ad7e3af5504571a69d10c00750a01c P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 16.
I20260709 13:21:57.118325 16151 raft_consensus.cc:2393] T 5a959dbecc2648eb85d1cc550f5a3245 P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 17: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
W20260709 13:21:57.117717 16789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 51597.6us
I20260709 13:21:57.121923 22782 raft_consensus.cc:697] T 8638ee363273426db1393dd81e5c7b72 P 90f3da57796b488c9087a2f523adf045 [term 19 LEADER]: Becoming Leader. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:21:57.096375 22837 raft_consensus.cc:3060] T e5ad4eb281564936a59f41e387a47156 P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:21:57.127434 16116 leader_election.cc:304] T fffab26427e24d398e2695aad40ca134 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 90f3da57796b488c9087a2f523adf045
I20260709 13:21:57.127779 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 21 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:57.128257 16144 raft_consensus.cc:1275] T 746aabf856ab401c9ef5e859371e7b67 P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 13:21:57.136473 16149 raft_consensus.cc:2393] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 21: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
I20260709 13:21:57.128604 22707 raft_consensus.cc:3060] T ecc1e363230043fe9fef390f0b979f20 P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:21:57.121269 22792 raft_consensus.cc:2749] T cdbd78acb4cf46298e0de8ba0d130e19 P 90f3da57796b488c9087a2f523adf045 [term 21 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:21:57.139642 16300 raft_consensus.cc:3060] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:21:57.147527 16294 raft_consensus.cc:2468] T 5a959dbecc2648eb85d1cc550f5a3245 P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 17.
I20260709 13:21:57.147331 22356 raft_consensus.cc:3055] T 434bff3082d141838068f323bdb0d1ed P 647a343a542d4a139fe106c19095129b [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:21:57.153057 16144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5ad4eb281564936a59f41e387a47156" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:57.155665 16298 raft_consensus.cc:2468] T c4f65cdf00b748bb963ee6e81ef43b79 P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 20.
I20260709 13:21:57.150732 22356 raft_consensus.cc:740] T 434bff3082d141838068f323bdb0d1ed P 647a343a542d4a139fe106c19095129b [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:21:57.149959 22735 raft_consensus.cc:2804] T fffab26427e24d398e2695aad40ca134 P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:21:57.160531 22792 raft_consensus.cc:2749] T 9e65676b21b748728ad39668cfe722ec P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:21:57.179898 22735 raft_consensus.cc:493] T fffab26427e24d398e2695aad40ca134 P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Starting leader election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:21:57.180464 22735 raft_consensus.cc:3060] T fffab26427e24d398e2695aad40ca134 P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:21:57.183400 22645 raft_consensus.cc:515] T 3884f864a96d428785895e5463e5728f P 90f3da57796b488c9087a2f523adf045 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:57.184830 16291 raft_consensus.cc:1240] T 3884f864a96d428785895e5463e5728f P 90f3da57796b488c9087a2f523adf045 [term 14 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 13. Current term is 14. Ops: []
I20260709 13:21:57.160392 16142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5718c623ebee4d2bb770f0f9ddb2f164" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:57.160235 22593 consensus_queue.cc:237] T 68a6838d82ef4cac83507f906139d90f P 90f3da57796b488c9087a2f523adf045 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.9, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:57.186765 16142 raft_consensus.cc:2393] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 18: Already voted for candidate cb82cc7bd6db46d980c75237a1402860 in this term.
I20260709 13:21:57.160792 16292 raft_consensus.cc:1240] T 8158bbd17b994d2eb96f560bd0f1b88d P 90f3da57796b488c9087a2f523adf045 [term 17 LEADER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 16. Current term is 17. Ops: []
I20260709 13:21:57.161217 16296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02838c86531d4f98bf58d1c22b8baf77" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:57.190935 16296 raft_consensus.cc:2393] T 02838c86531d4f98bf58d1c22b8baf77 P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 19: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
I20260709 13:21:57.161532 22865 raft_consensus.cc:493] T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:57.162969 22787 consensus_queue.cc:237] T 8c11074579b046d99e585c2818e18a2c P 90f3da57796b488c9087a2f523adf045 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.14, Last appended by leader: 14, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:57.151654 22709 raft_consensus.cc:493] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:21:57.167757 22622 raft_consensus.cc:670] T 806d7690c2b644959152074bb2cab0a7 P cb82cc7bd6db46d980c75237a1402860: failed to trigger leader election: Illegal state: only voting members can start elections: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } } [suppressed 4 similar messages]
I20260709 13:21:57.139278 22500 consensus_queue.cc:1048] T ada5a004ba7c46819051e3ccec71ffdb P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.018s
I20260709 13:21:57.199708 16151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68a6838d82ef4cac83507f906139d90f" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:57.169070 16116 leader_election.cc:304] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: cb82cc7bd6db46d980c75237a1402860
I20260709 13:21:57.169754 16265 leader_election.cc:304] T 02838c86531d4f98bf58d1c22b8baf77 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac
I20260709 13:21:57.173342 22457 leader_election.cc:290] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 21 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:57.176652 16226 raft_consensus.cc:3055] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [term 18 LEADER]: Stepping down as leader of term 18
W20260709 13:21:57.160619 16266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 146126us
I20260709 13:21:57.203703 16226 raft_consensus.cc:740] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:21:57.194893 22645 leader_election.cc:290] T 3884f864a96d428785895e5463e5728f P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 14 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:57.212860 16266 leader_election.cc:304] T b87147731a68400e8cce9b879daa7c5d P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
I20260709 13:21:57.207933 22838 raft_consensus.cc:2764] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Leader pre-election decision vote started in defunct term 17: won
I20260709 13:21:57.223096 16151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66f75124215f4ca6bef831e174a792d8" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:57.219535 16226 consensus_queue.cc:260] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:21:57.224473 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 55523.6us
I20260709 13:21:57.229550 16300 raft_consensus.cc:2468] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 18.
I20260709 13:21:57.209972 22865 raft_consensus.cc:515] T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:57.232959 22865 leader_election.cc:290] T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:21:57.235296 16142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:57.221415 22873 raft_consensus.cc:2804] T 02838c86531d4f98bf58d1c22b8baf77 P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:21:57.236127 22873 raft_consensus.cc:697] T 02838c86531d4f98bf58d1c22b8baf77 P 90f3da57796b488c9087a2f523adf045 [term 19 LEADER]: Becoming Leader. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:21:57.237155 22873 consensus_queue.cc:237] T 02838c86531d4f98bf58d1c22b8baf77 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 17.10, Last appended by leader: 10, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:57.239363 22872 raft_consensus.cc:493] T 5a96373bb0e14dce8a266b2fc6c58507 P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:57.239976 22872 raft_consensus.cc:515] T 5a96373bb0e14dce8a266b2fc6c58507 P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:21:57.241603 22831 log.cc:927] Time spent T 9c61a39fd4f84424b129c1a6f9ba404d P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 0.573s	user 0.002s	sys 0.000s
I20260709 13:21:57.242391 22872 leader_election.cc:290] T 5a96373bb0e14dce8a266b2fc6c58507 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:21:57.207834 22782 consensus_queue.cc:237] T 8638ee363273426db1393dd81e5c7b72 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:57.230072 22645 raft_consensus.cc:2749] T b87147731a68400e8cce9b879daa7c5d P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:21:57.200459 22494 raft_consensus.cc:493] T 746aabf856ab401c9ef5e859371e7b67 P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:57.208499 22709 raft_consensus.cc:515] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:57.251637 16226 raft_consensus.cc:3060] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:21:57.253762 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 84087.2us
I20260709 13:21:57.256387 22709 leader_election.cc:290] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:21:57.257877 16147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "829b4ad8c5a54eababeea1181b960d4b" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:57.258145 22630 leader_election.cc:290] T 23f82224f4564fea93572e8206bad1fe P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:57.255275 16266 leader_election.cc:304] T ecc1e363230043fe9fef390f0b979f20 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
I20260709 13:21:57.257069 22855 raft_consensus.cc:515] T 7c2095a5f4ad49408dbc7f69c606120d P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:57.258162 16142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62fe3346cd3c42c4a55e14af9073bca4" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 24 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:57.261034 16151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b7d1ab519044eda2d5e04a07242f25" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:57.261387 22855 leader_election.cc:290] T 7c2095a5f4ad49408dbc7f69c606120d P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 15 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:57.262306 16142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d4ab870a05347868956aa8b841b2981" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:57.261744 16151 raft_consensus.cc:2468] T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 16.
I20260709 13:21:57.264874 16142 raft_consensus.cc:2393] T 8d4ab870a05347868956aa8b841b2981 P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 17: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
I20260709 13:21:57.265296 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5d59407eb545b895d8d922e0dd69fd" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:57.266069 16147 raft_consensus.cc:1240] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 17. Current term is 18. Ops: []
I20260709 13:21:57.266009 22500 consensus_queue.cc:1059] T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: INVALID_TERM, Last received: 19.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:21:57.267225 22591 raft_consensus.cc:3055] T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:21:57.267554 16790 leader_election.cc:304] T 8c11074579b046d99e585c2818e18a2c P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:21:57.268631 22673 consensus_queue.cc:1048] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:21:57.267616 22591 raft_consensus.cc:740] T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:21:57.270789 22591 consensus_queue.cc:260] T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:21:57.271968 22818 log.cc:927] Time spent T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.639s	user 0.005s	sys 0.001s
I20260709 13:21:57.273592 22673 raft_consensus.cc:3060] T 8c11074579b046d99e585c2818e18a2c P cb82cc7bd6db46d980c75237a1402860 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:21:57.273821 16297 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40474 (request call id 1185) took 1245 ms. Trace:
I20260709 13:21:57.248992 22356 consensus_queue.cc:260] T 434bff3082d141838068f323bdb0d1ed P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 16.14, Last appended by leader: 14, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:57.274245 16297 rpcz_store.cc:276] 0709 13:21:56.028696 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:56.028915 (+   219us) service_pool.cc:224] Handling call
0709 13:21:56.029419 (+   504us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:21:56.065915 (+ 36496us) raft_consensus.cc:1534] Early marking committed up to index 14
0709 13:21:56.066323 (+   408us) log.cc:844] Serialized 14 byte log entry
0709 13:21:56.303278 (+236955us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:21:56.318184 (+ 14906us) log.cc:844] Serialized 26 byte log entry
0709 13:21:56.318392 (+   208us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:21:56.318426 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:56.318487 (+    61us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:21:57.271655 (+953168us) spinlock_profiling.cc:243] Waited 4.84 ms on lock 0x7b240024bb58. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69ad8fadb1 00007f69ad8f89c4 00007f69ad8f6acb 00007f69adab1dc9 00007f69adab1a55 00007f69adab109d 00007f69b06e310e 00007f69b06fa88c 00007f69b06f9bb6 00007f69b06f97d8 00007f69b0703221
0709 13:21:57.273504 (+  1849us) raft_consensus.cc:1685] finished
0709 13:21:57.273615 (+   111us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:21:57.273743 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":37552,"prepare.queue_time_us":3427,"prepare.run_cpu_time_us":52,"prepare.run_wall_time_us":53,"raft.queue_time_us":38873,"raft.run_cpu_time_us":239,"raft.run_wall_time_us":11560,"spinlock_wait_cycles":11133696,"thread_start_us":234418,"threads_started":1,"wal-append.queue_time_us":565596}
I20260709 13:21:57.275995 22356 raft_consensus.cc:3060] T 434bff3082d141838068f323bdb0d1ed P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:21:57.277319 16820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e9b2bf7011f4ae688a79d7bb5a447b4" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:21:57.277930 16820 raft_consensus.cc:3060] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:21:57.279227 22838 consensus_queue.cc:1059] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: INVALID_TERM, Last received: 21.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:21:57.279697 16819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87147731a68400e8cce9b879daa7c5d" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:21:57.281179 22838 raft_consensus.cc:3055] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [term 23 LEADER]: Stepping down as leader of term 23
I20260709 13:21:57.281569 22838 raft_consensus.cc:740] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:21:57.282380 22838 consensus_queue.cc:260] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 23.15, Last appended by leader: 15, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:57.284209 22735 raft_consensus.cc:515] T fffab26427e24d398e2695aad40ca134 P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:57.286448 22735 leader_election.cc:290] T fffab26427e24d398e2695aad40ca134 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 18 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:57.287295 22838 raft_consensus.cc:3060] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:21:57.289616 16266 leader_election.cc:304] T a378d7f796f94615b3b347119a9880fd P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860
I20260709 13:21:57.290510 22709 raft_consensus.cc:2749] T ecc1e363230043fe9fef390f0b979f20 P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:21:57.292018 16116 leader_election.cc:304] T 08c74490c40243a88e62d1760b5cd814 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:21:57.292559 22709 raft_consensus.cc:2749] T a378d7f796f94615b3b347119a9880fd P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:21:57.291571 16265 leader_election.cc:304] T 8cc959fb7f704fba9009c0595467d074 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:21:57.291090 16266 leader_election.cc:304] T d2ab0ac8347641e6955e9bce7d90c1a2 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860
I20260709 13:21:57.297073 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e9b2bf7011f4ae688a79d7bb5a447b4" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:57.297724 16149 raft_consensus.cc:2393] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 in current term 18: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
I20260709 13:21:57.308811 16226 raft_consensus.cc:1275] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 13:21:57.310281 16820 raft_consensus.cc:2468] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 18.
I20260709 13:21:57.294493 16265 leader_election.cc:304] T 18ad7e3af5504571a69d10c00750a01c P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:21:57.272123 22591 raft_consensus.cc:3060] T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:21:57.347721 16790 leader_election.cc:304] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
W20260709 13:21:57.349640 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 56352.5us
I20260709 13:21:57.348719 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8638ee363273426db1393dd81e5c7b72" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:57.358925 16149 raft_consensus.cc:1240] T 7c2095a5f4ad49408dbc7f69c606120d P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 14. Current term is 15. Ops: []
I20260709 13:21:57.359341 16225 raft_consensus.cc:2393] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 in current term 19: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
I20260709 13:21:57.356539 22735 consensus_queue.cc:1059] T 1df93779209748b9860f6d893dd3de53 P 647a343a542d4a139fe106c19095129b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: INVALID_TERM, Last received: 16.10, Next index: 11, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:21:57.357062 22630 consensus_queue.cc:1059] T 8158bbd17b994d2eb96f560bd0f1b88d P 647a343a542d4a139fe106c19095129b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.007s
I20260709 13:21:57.367066 16823 raft_consensus.cc:1240] T 434bff3082d141838068f323bdb0d1ed P cb82cc7bd6db46d980c75237a1402860 [term 17 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 16. Current term is 17. Ops: [16.14-16.14]
W20260709 13:21:57.359649 22731 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 19 current term: 19 is higher.
I20260709 13:21:57.371084 16823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:21:57.367488 22740 raft_consensus.cc:2749] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:21:57.374437 16820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f82224f4564fea93572e8206bad1fe" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:57.375662 16145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c61a39fd4f84424b129c1a6f9ba404d" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 20 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:57.382946 16823 raft_consensus.cc:3060] T 19baa7bc367844c288bca8f81439c413 P cb82cc7bd6db46d980c75237a1402860 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:21:57.376840 16151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3884f864a96d428785895e5463e5728f" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:21:57.329885 22865 raft_consensus.cc:2804] T 8cc959fb7f704fba9009c0595467d074 P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:21:57.346904 22593 raft_consensus.cc:2804] T 18ad7e3af5504571a69d10c00750a01c P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:21:57.310032 22855 raft_consensus.cc:2749] T 08c74490c40243a88e62d1760b5cd814 P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:21:57.401533 16150 raft_consensus.cc:1275] T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b [term 21 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
W20260709 13:21:57.413933 16191 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:57.418519 16148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a96373bb0e14dce8a266b2fc6c58507" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:57.419008 16117 leader_election.cc:304] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:21:57.304937 22494 raft_consensus.cc:515] T 746aabf856ab401c9ef5e859371e7b67 P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:57.396867 22865 raft_consensus.cc:493] T 8cc959fb7f704fba9009c0595467d074 P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:57.413383 22593 raft_consensus.cc:493] T 18ad7e3af5504571a69d10c00750a01c P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 647a343a542d4a139fe106c19095129b)
W20260709 13:21:57.443751 16266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 152774us
I20260709 13:21:57.442322 16116 leader_election.cc:304] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:21:57.431205 22610 raft_consensus.cc:2804] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:21:57.464223 22864 raft_consensus.cc:2749] T d2ab0ac8347641e6955e9bce7d90c1a2 P 90f3da57796b488c9087a2f523adf045 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:21:57.464392 16146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f5f89186204242bf377f7c42b2e2da" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:57.471500 16266 leader_election.cc:304] T c9944d5e4b8c4109af9955c4f08c9095 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860
I20260709 13:21:57.471956 22707 raft_consensus.cc:515] T ecc1e363230043fe9fef390f0b979f20 P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:57.479839 22855 raft_consensus.cc:2749] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:21:57.424069 22457 consensus_queue.cc:1059] T b6f07842d3aa47858df5665874c86f3a P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: INVALID_TERM, Last received: 17.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.032s
I20260709 13:21:57.486606 22855 raft_consensus.cc:493] T d16dfad0c3b540e5b4b86492ddf87208 P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:57.437315 22865 raft_consensus.cc:3060] T 8cc959fb7f704fba9009c0595467d074 P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:21:57.447402 22593 raft_consensus.cc:3060] T 18ad7e3af5504571a69d10c00750a01c P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:21:57.411746 22508 consensus_queue.cc:1048] T 2794af5b99984efebdee564035841cb0 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.030s
I20260709 13:21:57.439129 22792 consensus_queue.cc:1059] T 7c2095a5f4ad49408dbc7f69c606120d P 90f3da57796b488c9087a2f523adf045 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: INVALID_TERM, Last received: 14.7, Next index: 8, Last known committed idx: 5, Time since last communication: 0.060s
I20260709 13:21:57.505110 16225 raft_consensus.cc:1240] T 68a6838d82ef4cac83507f906139d90f P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 17. Current term is 18. Ops: []
I20260709 13:21:57.505365 16223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a959dbecc2648eb85d1cc550f5a3245" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:21:57.506059 16223 raft_consensus.cc:3060] T 5a959dbecc2648eb85d1cc550f5a3245 P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: Advancing to term 17
I20260709 13:21:57.506426 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ad5dbfa99ee4b8d9e28be92b3f36db5" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:21:57.507145 22494 leader_election.cc:290] T 746aabf856ab401c9ef5e859371e7b67 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:57.507426 16221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4dc6b0541ca431ebc757beac00521f1" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:57.508116 16221 raft_consensus.cc:2468] T e4dc6b0541ca431ebc757beac00521f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 18.
I20260709 13:21:57.509048 16221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:21:57.509554 16221 raft_consensus.cc:3060] T 19baa7bc367844c288bca8f81439c413 P 1ca2725d290e47d68b1a4b609c9816ac [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:21:57.445560 22754 raft_consensus.cc:493] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:21:57.511451 22754 raft_consensus.cc:515] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:57.512179 16219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4f65cdf00b748bb963ee6e81ef43b79" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:21:57.525538 22754 leader_election.cc:290] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:57.526400 22676 raft_consensus.cc:3055] T d2ab0ac8347641e6955e9bce7d90c1a2 P cb82cc7bd6db46d980c75237a1402860 [term 14 LEADER]: Stepping down as leader of term 14
W20260709 13:21:57.528106 16191 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:57.522195 22855 raft_consensus.cc:515] T d16dfad0c3b540e5b4b86492ddf87208 P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:57.531767 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b7d1ab519044eda2d5e04a07242f25" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:21:57.532493 16150 raft_consensus.cc:3060] T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:21:57.450529 22543 consensus_queue.cc:1059] T b87147731a68400e8cce9b879daa7c5d P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: INVALID_TERM, Last received: 18.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.065s
I20260709 13:21:57.464282 22610 raft_consensus.cc:697] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 647a343a542d4a139fe106c19095129b [term 18 LEADER]: Becoming Leader. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:21:57.526928 22676 raft_consensus.cc:740] T d2ab0ac8347641e6955e9bce7d90c1a2 P cb82cc7bd6db46d980c75237a1402860 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
W20260709 13:21:57.537434 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 97277.9us
I20260709 13:21:57.545439 16223 raft_consensus.cc:2468] T 5a959dbecc2648eb85d1cc550f5a3245 P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 17.
I20260709 13:21:57.538187 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fffab26427e24d398e2695aad40ca134" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:21:57.521698 22782 raft_consensus.cc:2749] T c9944d5e4b8c4109af9955c4f08c9095 P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:21:57.567152 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02838c86531d4f98bf58d1c22b8baf77" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:57.532514 22855 leader_election.cc:290] T d16dfad0c3b540e5b4b86492ddf87208 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:57.568468 16116 leader_election.cc:304] T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
W20260709 13:21:57.568837 16266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 97471.5us
I20260709 13:21:57.535071 22673 raft_consensus.cc:2749] T 8c11074579b046d99e585c2818e18a2c P cb82cc7bd6db46d980c75237a1402860 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:21:57.534633 22754 raft_consensus.cc:493] T 84f2da98864b4425bf7c0055951fdd4b P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:57.538973 22707 leader_election.cc:290] T ecc1e363230043fe9fef390f0b979f20 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 20 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:57.574357 16116 leader_election.cc:304] T 5a959dbecc2648eb85d1cc550f5a3245 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
I20260709 13:21:57.551668 22735 raft_consensus.cc:493] T af57996a8be84d398f0f023e8d08ef77 P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:21:57.568670 22672 raft_consensus.cc:3055] T b87147731a68400e8cce9b879daa7c5d P cb82cc7bd6db46d980c75237a1402860 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:21:57.579020 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:21:57.579624 16225 raft_consensus.cc:3060] T ecc1e363230043fe9fef390f0b979f20 P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:21:57.577589 16223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c2095a5f4ad49408dbc7f69c606120d" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:21:57.572419 22740 raft_consensus.cc:493] T c9944d5e4b8c4109af9955c4f08c9095 P cb82cc7bd6db46d980c75237a1402860 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:57.585976 16223 raft_consensus.cc:2393] T 7c2095a5f4ad49408dbc7f69c606120d P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 15: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
I20260709 13:21:57.539080 22837 raft_consensus.cc:515] T e5ad4eb281564936a59f41e387a47156 P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:57.586521 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746aabf856ab401c9ef5e859371e7b67" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
W20260709 13:21:57.587653 16788 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:57.580577 22672 raft_consensus.cc:740] T b87147731a68400e8cce9b879daa7c5d P cb82cc7bd6db46d980c75237a1402860 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:21:57.570990 22356 consensus_queue.cc:1059] T 68a6838d82ef4cac83507f906139d90f P 647a343a542d4a139fe106c19095129b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: INVALID_TERM, Last received: 17.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.014s
I20260709 13:21:57.577436 22855 raft_consensus.cc:2804] T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:21:57.573246 22508 consensus_queue.cc:1048] T 74ca8140f35c4ae29e8a664f6bdfc829 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.020s
I20260709 13:21:57.586825 22782 raft_consensus.cc:493] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:57.606168 22855 raft_consensus.cc:493] T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:57.640775 22855 raft_consensus.cc:3060] T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:21:57.640733 22782 raft_consensus.cc:515] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } }
I20260709 13:21:57.643447 22348 raft_consensus.cc:493] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:57.644078 22348 raft_consensus.cc:515] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } }
I20260709 13:21:57.646610 22348 leader_election.cc:290] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:57.612408 22672 consensus_queue.cc:260] T b87147731a68400e8cce9b879daa7c5d P cb82cc7bd6db46d980c75237a1402860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:57.664215 22672 raft_consensus.cc:3060] T b87147731a68400e8cce9b879daa7c5d P cb82cc7bd6db46d980c75237a1402860 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:21:57.600986 22887 raft_consensus.cc:2804] T 5a959dbecc2648eb85d1cc550f5a3245 P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:21:57.667599 22887 raft_consensus.cc:697] T 5a959dbecc2648eb85d1cc550f5a3245 P 647a343a542d4a139fe106c19095129b [term 17 LEADER]: Becoming Leader. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:21:57.668712 22887 consensus_queue.cc:237] T 5a959dbecc2648eb85d1cc550f5a3245 P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:57.615521 16219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d16dfad0c3b540e5b4b86492ddf87208" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:57.670197 16219 raft_consensus.cc:2468] T d16dfad0c3b540e5b4b86492ddf87208 P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 19.
I20260709 13:21:57.671205 16219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c61a39fd4f84424b129c1a6f9ba404d" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 20 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:57.590420 22735 raft_consensus.cc:515] T af57996a8be84d398f0f023e8d08ef77 P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:57.671835 16219 raft_consensus.cc:2468] T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 19.
I20260709 13:21:57.672797 16219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a959dbecc2648eb85d1cc550f5a3245" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:57.673606 22735 leader_election.cc:290] T af57996a8be84d398f0f023e8d08ef77 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:57.563375 22610 consensus_queue.cc:237] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:57.674077 16219 raft_consensus.cc:2393] T 5a959dbecc2648eb85d1cc550f5a3245 P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 17: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
W20260709 13:21:57.674357 16190 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:21:57.674824 16190 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:57.625633 22356 raft_consensus.cc:3055] T 68a6838d82ef4cac83507f906139d90f P 647a343a542d4a139fe106c19095129b [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:21:57.631449 16266 leader_election.cc:304] T 434bff3082d141838068f323bdb0d1ed P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
I20260709 13:21:57.632196 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8638ee363273426db1393dd81e5c7b72" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:57.633555 22867 consensus_queue.cc:1048] T 31969392ccdb43b491911a319e8bc7e1 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:21:57.634022 22677 raft_consensus.cc:3060] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:21:57.634490 16292 raft_consensus.cc:1275] T 66ae176961b74813a58bc15c9c4c6f2a P 90f3da57796b488c9087a2f523adf045 [term 24 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 21 index: 14. Preceding OpId from leader: term: 24 index: 15. (index mismatch)
I20260709 13:21:57.635110 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e9b2bf7011f4ae688a79d7bb5a447b4" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:57.577332 22754 raft_consensus.cc:515] T 84f2da98864b4425bf7c0055951fdd4b P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:57.651006 16225 raft_consensus.cc:2468] T ecc1e363230043fe9fef390f0b979f20 P 1ca2725d290e47d68b1a4b609c9816ac [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 20.
W20260709 13:21:57.652006 22823 log.cc:927] Time spent T 08c74490c40243a88e62d1760b5cd814 P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 0.141s	user 0.002s	sys 0.000s
I20260709 13:21:57.654808 16144 raft_consensus.cc:2393] T e5ad4eb281564936a59f41e387a47156 P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 18: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
I20260709 13:21:57.655653 22837 leader_election.cc:290] T e5ad4eb281564936a59f41e387a47156 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 18 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:57.657546 22855 raft_consensus.cc:515] T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:57.658514 16823 raft_consensus.cc:2468] T 19baa7bc367844c288bca8f81439c413 P cb82cc7bd6db46d980c75237a1402860 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 21.
I20260709 13:21:57.658700 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18ad7e3af5504571a69d10c00750a01c" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:57.660225 22865 raft_consensus.cc:515] T 8cc959fb7f704fba9009c0595467d074 P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:57.593997 22740 raft_consensus.cc:515] T c9944d5e4b8c4109af9955c4f08c9095 P cb82cc7bd6db46d980c75237a1402860 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:21:57.676787 16117 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:21:57.787649 16117 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:57.681102 16265 leader_election.cc:304] T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
W20260709 13:21:57.791124 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 110113us
I20260709 13:21:57.793851 16265 leader_election.cc:304] T 5a959dbecc2648eb85d1cc550f5a3245 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:21:57.684826 16116 leader_election.cc:304] T d16dfad0c3b540e5b4b86492ddf87208 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
W20260709 13:21:57.794838 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 110334us
I20260709 13:21:57.675138 16219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5ad4eb281564936a59f41e387a47156" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:21:57.795789 16219 raft_consensus.cc:3060] T e5ad4eb281564936a59f41e387a47156 P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:21:57.716804 16220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af57996a8be84d398f0f023e8d08ef77" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:57.713877 16299 raft_consensus.cc:2468] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 17.
I20260709 13:21:57.715878 22348 raft_consensus.cc:493] T 1df93779209748b9860f6d893dd3de53 P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:57.773999 22792 raft_consensus.cc:3060] T 434bff3082d141838068f323bdb0d1ed P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:21:57.713392 22782 leader_election.cc:290] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:21:57.712363 22543 raft_consensus.cc:493] T 57a744f5aaf944678a63469cbf514015 P cb82cc7bd6db46d980c75237a1402860 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:57.773999 16222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a96373bb0e14dce8a266b2fc6c58507" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:57.798758 16789 leader_election.cc:304] T 8638ee363273426db1393dd81e5c7b72 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:21:57.683959 22356 raft_consensus.cc:740] T 68a6838d82ef4cac83507f906139d90f P 647a343a542d4a139fe106c19095129b [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:21:57.807682 22754 leader_election.cc:290] T 84f2da98864b4425bf7c0055951fdd4b P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:57.675824 22593 raft_consensus.cc:515] T 18ad7e3af5504571a69d10c00750a01c P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } }
I20260709 13:21:57.812655 16221 raft_consensus.cc:2468] T 19baa7bc367844c288bca8f81439c413 P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 21.
I20260709 13:21:57.800156 22731 raft_consensus.cc:493] T c0eab25ce4c64e018a041beeda2867db P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:57.810149 22864 raft_consensus.cc:2804] T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:21:57.839573 22731 raft_consensus.cc:515] T c0eab25ce4c64e018a041beeda2867db P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:57.842514 22731 leader_election.cc:290] T c0eab25ce4c64e018a041beeda2867db P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
W20260709 13:21:57.814430 16266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 183106us
I20260709 13:21:57.809657 22709 raft_consensus.cc:2749] T 5a959dbecc2648eb85d1cc550f5a3245 P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:21:57.820804 22888 raft_consensus.cc:3060] T 8638ee363273426db1393dd81e5c7b72 P cb82cc7bd6db46d980c75237a1402860 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:21:57.821375 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cc959fb7f704fba9009c0595467d074" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:57.822381 22591 raft_consensus.cc:493] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:57.823066 16820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:21:57.823520 16822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af57996a8be84d398f0f023e8d08ef77" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:57.824005 16222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 21 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:57.850095 16221 raft_consensus.cc:1240] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [term 24 LEADER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 23. Current term is 24. Ops: [23.15-23.15]
I20260709 13:21:57.850590 16222 raft_consensus.cc:2393] T 19baa7bc367844c288bca8f81439c413 P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 21: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
I20260709 13:21:57.824186 16823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746aabf856ab401c9ef5e859371e7b67" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:57.825444 16143 raft_consensus.cc:1275] T 8d4ab870a05347868956aa8b841b2981 P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 13:21:57.804404 22707 raft_consensus.cc:2804] T d16dfad0c3b540e5b4b86492ddf87208 P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:21:57.827310 16142 raft_consensus.cc:1240] T e5ad4eb281564936a59f41e387a47156 P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 17. Current term is 18. Ops: []
I20260709 13:21:57.807992 16042 catalog_manager.cc:5697] T ee9ed67006144797b1de6ea8c2013a68 P 647a343a542d4a139fe106c19095129b reported cstate change: term changed from 15 to 16, leader changed from cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) to 647a343a542d4a139fe106c19095129b (127.14.177.129). New cstate: current_term: 16 leader_uuid: "647a343a542d4a139fe106c19095129b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:21:57.833166 16150 raft_consensus.cc:2468] T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 17.
I20260709 13:21:57.835033 16226 raft_consensus.cc:2393] T 18ad7e3af5504571a69d10c00750a01c P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 16: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
I20260709 13:21:57.811271 22348 raft_consensus.cc:515] T 1df93779209748b9860f6d893dd3de53 P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:57.836149 22740 leader_election.cc:290] T c9944d5e4b8c4109af9955c4f08c9095 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:57.838176 16145 raft_consensus.cc:3055] T 8158bbd17b994d2eb96f560bd0f1b88d P 647a343a542d4a139fe106c19095129b [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:21:57.887663 16145 raft_consensus.cc:740] T 8158bbd17b994d2eb96f560bd0f1b88d P 647a343a542d4a139fe106c19095129b [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:21:57.888501 16295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c74490c40243a88e62d1760b5cd814" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:57.888643 16145 consensus_queue.cc:260] T 8158bbd17b994d2eb96f560bd0f1b88d P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:57.890005 16145 raft_consensus.cc:3060] T 8158bbd17b994d2eb96f560bd0f1b88d P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:21:57.847714 16220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f5f89186204242bf377f7c42b2e2da" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:57.826740 16151 raft_consensus.cc:1240] T fffab26427e24d398e2695aad40ca134 P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 17. Current term is 18. Ops: []
I20260709 13:21:57.891534 16220 raft_consensus.cc:2468] T c2f5f89186204242bf377f7c42b2e2da P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 19.
I20260709 13:21:57.871034 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9944d5e4b8c4109af9955c4f08c9095" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:57.895828 16297 raft_consensus.cc:2468] T c9944d5e4b8c4109af9955c4f08c9095 P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 20.
I20260709 13:21:57.895992 16222 raft_consensus.cc:1275] T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 17 index: 9. Preceding OpId from leader: term: 19 index: 10. (index mismatch)
I20260709 13:21:57.897409 16222 raft_consensus.cc:1275] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 13:21:57.904968 16219 raft_consensus.cc:2468] T e5ad4eb281564936a59f41e387a47156 P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 18.
I20260709 13:21:57.905558 16822 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "08b7d1ab519044eda2d5e04a07242f25"
dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
mode: GRACEFUL
new_leader_uuid: "647a343a542d4a139fe106c19095129b"
 from {username='slave'} at 127.0.0.1:37818
I20260709 13:21:57.906442 16822 raft_consensus.cc:606] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 [term 17 FOLLOWER]: Received request to transfer leadership to TS 647a343a542d4a139fe106c19095129b
I20260709 13:21:57.906790 16822 raft_consensus.cc:614] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 [term 17 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:21:57.910506 16822 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4e9b2bf7011f4ae688a79d7bb5a447b4"
dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
mode: GRACEFUL
new_leader_uuid: "90f3da57796b488c9087a2f523adf045"
 from {username='slave'} at 127.0.0.1:37818
I20260709 13:21:57.911191 16822 raft_consensus.cc:606] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860 [term 18 FOLLOWER]: Received request to transfer leadership to TS 90f3da57796b488c9087a2f523adf045
I20260709 13:21:57.911784 16145 raft_consensus.cc:1275] T 8158bbd17b994d2eb96f560bd0f1b88d P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 13:21:57.911906 16822 raft_consensus.cc:614] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860 [term 18 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 13:21:57.912487 16191 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:57.915512 16822 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "66ae176961b74813a58bc15c9c4c6f2a"
dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
mode: GRACEFUL
new_leader_uuid: "90f3da57796b488c9087a2f523adf045"
 from {username='slave'} at 127.0.0.1:37818
I20260709 13:21:57.916162 16822 raft_consensus.cc:606] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [term 24 LEADER]: Received request to transfer leadership to TS 90f3da57796b488c9087a2f523adf045
I20260709 13:21:57.839576 22864 raft_consensus.cc:493] T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:57.834000 16265 leader_election.cc:304] T 5a96373bb0e14dce8a266b2fc6c58507 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:21:57.871583 22591 raft_consensus.cc:515] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:57.850587 22681 consensus_queue.cc:1048] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.106s
I20260709 13:21:57.874948 22707 raft_consensus.cc:493] T d16dfad0c3b540e5b4b86492ddf87208 P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:57.846692 22543 raft_consensus.cc:515] T 57a744f5aaf944678a63469cbf514015 P cb82cc7bd6db46d980c75237a1402860 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:57.965327 16222 raft_consensus.cc:1240] T d2ab0ac8347641e6955e9bce7d90c1a2 P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 14. Current term is 15. Ops: []
I20260709 13:21:57.945719 16789 leader_election.cc:304] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 647a343a542d4a139fe106c19095129b
I20260709 13:21:57.945611 22348 leader_election.cc:290] T 1df93779209748b9860f6d893dd3de53 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:57.957593 22707 raft_consensus.cc:3060] T d16dfad0c3b540e5b4b86492ddf87208 P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:21:57.937567 22676 consensus_queue.cc:260] T d2ab0ac8347641e6955e9bce7d90c1a2 P cb82cc7bd6db46d980c75237a1402860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 14.13, Last appended by leader: 13, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:57.973613 16295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5ad4eb281564936a59f41e387a47156" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
W20260709 13:21:57.922425 22857 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 17 current term: 17 is higher.
I20260709 13:21:57.973021 22676 raft_consensus.cc:3060] T d2ab0ac8347641e6955e9bce7d90c1a2 P cb82cc7bd6db46d980c75237a1402860 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:21:57.975768 16116 leader_election.cc:304] T af57996a8be84d398f0f023e8d08ef77 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860
I20260709 13:21:57.994231 16291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6f07842d3aa47858df5665874c86f3a" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
W20260709 13:21:57.975785 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 141871us
I20260709 13:21:58.001289 16116 leader_election.cc:304] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:21:57.975103 16299 raft_consensus.cc:1240] T 66ae176961b74813a58bc15c9c4c6f2a P 90f3da57796b488c9087a2f523adf045 [term 24 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 23. Current term is 24. Ops: []
I20260709 13:21:57.944411 22591 leader_election.cc:290] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:21:58.015472 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a378d7f796f94615b3b347119a9880fd" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:21:58.016547 16294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68a6838d82ef4cac83507f906139d90f" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:57.976668 16789 leader_election.cc:304] T c9944d5e4b8c4109af9955c4f08c9095 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:21:58.017997 16296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e4bfb9090c8460eb14fa7e0d818e7f1" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:58.018922 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fffab26427e24d398e2695aad40ca134" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:57.997906 16222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4dc6b0541ca431ebc757beac00521f1" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:21:57.925804 22864 raft_consensus.cc:3060] T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:21:58.022084 16117 leader_election.cc:304] T 746aabf856ab401c9ef5e859371e7b67 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860
I20260709 13:21:57.992504 22586 raft_consensus.cc:2764] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860 [term 18 FOLLOWER]: Leader pre-election decision vote started in defunct term 17: won
I20260709 13:21:57.941627 22356 consensus_queue.cc:260] T 68a6838d82ef4cac83507f906139d90f P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.9, Last appended by leader: 9, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:57.936077 22517 raft_consensus.cc:3055] T 1df93779209748b9860f6d893dd3de53 P 647a343a542d4a139fe106c19095129b [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:21:57.919070 22537 raft_consensus.cc:3060] T b6f07842d3aa47858df5665874c86f3a P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:21:58.020493 16292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f5f89186204242bf377f7c42b2e2da" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:57.925494 22865 leader_election.cc:290] T 8cc959fb7f704fba9009c0595467d074 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 17 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:21:58.024662 16222 raft_consensus.cc:3060] T e4dc6b0541ca431ebc757beac00521f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:21:57.959458 22543 leader_election.cc:290] T 57a744f5aaf944678a63469cbf514015 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:57.941375 22593 leader_election.cc:290] T 18ad7e3af5504571a69d10c00750a01c P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 16 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:21:58.029215 22787 raft_consensus.cc:493] T 84f2da98864b4425bf7c0055951fdd4b P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:58.029742 22872 consensus_queue.cc:1048] T 8d4ab870a05347868956aa8b841b2981 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:21:58.110064 16816 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20"
dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
mode: GRACEFUL
new_leader_uuid: "90f3da57796b488c9087a2f523adf045"
 from {username='slave'} at 127.0.0.1:37818
I20260709 13:21:58.110782 16816 raft_consensus.cc:606] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860 [term 19 LEADER]: Received request to transfer leadership to TS 90f3da57796b488c9087a2f523adf045
I20260709 13:21:58.030010 16819 raft_consensus.cc:3060] T 806d7690c2b644959152074bb2cab0a7 P cb82cc7bd6db46d980c75237a1402860 [term 17 LEARNER]: Advancing to term 18
I20260709 13:21:58.112399 16298 raft_consensus.cc:1240] T c4f65cdf00b748bb963ee6e81ef43b79 P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 19. Current term is 20. Ops: [19.13-19.13]
I20260709 13:21:58.113747 16151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c2095a5f4ad49408dbc7f69c606120d" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:21:58.114498 16151 raft_consensus.cc:2393] T 7c2095a5f4ad49408dbc7f69c606120d P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 15: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
I20260709 13:21:58.026800 22622 raft_consensus.cc:2804] T c9944d5e4b8c4109af9955c4f08c9095 P cb82cc7bd6db46d980c75237a1402860 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:21:58.115406 22622 raft_consensus.cc:493] T c9944d5e4b8c4109af9955c4f08c9095 P cb82cc7bd6db46d980c75237a1402860 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:58.115963 22622 raft_consensus.cc:3060] T c9944d5e4b8c4109af9955c4f08c9095 P cb82cc7bd6db46d980c75237a1402860 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:21:58.033283 16142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62fe3346cd3c42c4a55e14af9073bca4" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 24 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:21:58.077126 22754 raft_consensus.cc:493] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:58.117858 16142 raft_consensus.cc:2410] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac for term 24 because replica has last-logged OpId of term: 23 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 14.
I20260709 13:21:58.038156 16147 raft_consensus.cc:1240] T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 19. Current term is 20. Ops: [19.13-19.13]
W20260709 13:21:58.119778 16191 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:58.041256 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9944d5e4b8c4109af9955c4f08c9095" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:58.120880 16225 raft_consensus.cc:2468] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 19.
I20260709 13:21:58.045321 22886 consensus_queue.cc:1048] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:21:58.046264 22837 raft_consensus.cc:2749] T 746aabf856ab401c9ef5e859371e7b67 P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:21:58.047191 16116 leader_election.cc:304] T e5ad4eb281564936a59f41e387a47156 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
I20260709 13:21:58.047880 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57a744f5aaf944678a63469cbf514015" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:58.048478 16218 raft_consensus.cc:2410] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 for term 24 because replica has last-logged OpId of term: 23 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 14.
I20260709 13:21:58.053354 16226 raft_consensus.cc:1240] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 19 LEADER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 18. Current term is 19. Ops: []
W20260709 13:21:58.122236 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 75156.3us
I20260709 13:21:58.054567 16219 raft_consensus.cc:1240] T ecc1e363230043fe9fef390f0b979f20 P 1ca2725d290e47d68b1a4b609c9816ac [term 20 FOLLOWER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 19. Current term is 20. Ops: []
I20260709 13:21:57.956393 22855 leader_election.cc:290] T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 19 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
W20260709 13:21:58.061782 22897 log.cc:927] Time spent T 2794af5b99984efebdee564035841cb0 P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 0.280s	user 0.004s	sys 0.000s
I20260709 13:21:58.122237 16299 raft_consensus.cc:2468] T 57a744f5aaf944678a63469cbf514015 P 90f3da57796b488c9087a2f523adf045 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 14.
I20260709 13:21:57.948283 16042 catalog_manager.cc:5697] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b reported cstate change: term changed from 16 to 21. New cstate: current_term: 21 leader_uuid: "647a343a542d4a139fe106c19095129b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:21:58.026880 22196 consensus_queue.cc:1048] T 9c61a39fd4f84424b129c1a6f9ba404d P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.040s
I20260709 13:21:58.130919 16151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a959dbecc2648eb85d1cc550f5a3245" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:58.131466 22196 raft_consensus.cc:2804] T e5ad4eb281564936a59f41e387a47156 P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Leader election won for term 18
W20260709 13:21:58.068814 16789 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:58.132016 16298 raft_consensus.cc:1240] T 8a878999c0024d2abe517484180290fc P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 15. Current term is 16. Ops: []
I20260709 13:21:58.132135 16151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1df93779209748b9860f6d893dd3de53" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:58.133131 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d4ab870a05347868956aa8b841b2981" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:58.133244 16221 raft_consensus.cc:1240] T e5ad4eb281564936a59f41e387a47156 P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 17. Current term is 18. Ops: []
I20260709 13:21:58.134063 16146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806d7690c2b644959152074bb2cab0a7" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:58.135249 16146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f2da98864b4425bf7c0055951fdd4b" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:58.085091 22645 consensus_queue.cc:1048] T 8158bbd17b994d2eb96f560bd0f1b88d P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.051s
I20260709 13:21:58.136691 16146 raft_consensus.cc:1275] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [term 24 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 23 index: 15. Preceding OpId from leader: term: 24 index: 15. (term mismatch)
I20260709 13:21:58.104590 22782 consensus_queue.cc:1059] T e5ad4eb281564936a59f41e387a47156 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: INVALID_TERM, Last received: 17.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.040s
I20260709 13:21:58.137563 22782 raft_consensus.cc:3055] T e5ad4eb281564936a59f41e387a47156 P 90f3da57796b488c9087a2f523adf045 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:21:58.137849 16150 raft_consensus.cc:1275] T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 13:21:58.137948 22782 raft_consensus.cc:740] T e5ad4eb281564936a59f41e387a47156 P 90f3da57796b488c9087a2f523adf045 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:21:57.993914 22874 raft_consensus.cc:2749] T 5a96373bb0e14dce8a266b2fc6c58507 P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:21:58.080523 16265 leader_election.cc:304] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:21:58.027220 22830 raft_consensus.cc:2749] T af57996a8be84d398f0f023e8d08ef77 P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:21:58.025563 22854 raft_consensus.cc:2804] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:21:58.100987 16292 raft_consensus.cc:2393] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 19: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
I20260709 13:21:58.127640 16148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a96373bb0e14dce8a266b2fc6c58507" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:58.128790 16226 raft_consensus.cc:1240] T 7c2095a5f4ad49408dbc7f69c606120d P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 14. Current term is 15. Ops: []
I20260709 13:21:58.148526 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e9b2bf7011f4ae688a79d7bb5a447b4" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:21:58.134176 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c2095a5f4ad49408dbc7f69c606120d" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:21:58.137055 16146 pending_rounds.cc:85] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b: Aborting all ops after (but not including) 14
I20260709 13:21:58.139441 16143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c74490c40243a88e62d1760b5cd814" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:21:58.140136 16291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9944d5e4b8c4109af9955c4f08c9095" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:58.149230 16299 raft_consensus.cc:3060] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:21:58.149636 16146 pending_rounds.cc:107] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b: Aborting uncommitted NO_OP operation due to leader change: 23.15
I20260709 13:21:58.140985 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e4bfb9090c8460eb14fa7e0d818e7f1" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:21:58.131507 16296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ab0ac8347641e6955e9bce7d90c1a2" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:21:58.140705 16149 raft_consensus.cc:1240] T 02838c86531d4f98bf58d1c22b8baf77 P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 18. Current term is 19. Ops: []
I20260709 13:21:58.100250 16820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7ca61592e848ea9b0fb81a2ce33cdc" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:58.071995 16223 raft_consensus.cc:1240] T 08c74490c40243a88e62d1760b5cd814 P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 16. Current term is 17. Ops: []
I20260709 13:21:58.142551 16147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee9ed67006144797b1de6ea8c2013a68" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
W20260709 13:21:58.146292 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 65852.1us
I20260709 13:21:58.152879 16265 leader_election.cc:304] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 647a343a542d4a139fe106c19095129b
I20260709 13:21:58.133265 16151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0eab25ce4c64e018a041beeda2867db" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:58.149812 16291 raft_consensus.cc:2393] T c9944d5e4b8c4109af9955c4f08c9095 P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 20: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
I20260709 13:21:58.082381 22517 raft_consensus.cc:740] T 1df93779209748b9860f6d893dd3de53 P 647a343a542d4a139fe106c19095129b [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:21:58.152040 16789 leader_election.cc:304] T 57a744f5aaf944678a63469cbf514015 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:21:58.085808 22709 consensus_queue.cc:1059] T fffab26427e24d398e2695aad40ca134 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: INVALID_TERM, Last received: 17.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.063s
I20260709 13:21:58.150457 16146 raft_consensus.cc:2889] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [term 24 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:21:58.171962 16295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
W20260709 13:21:58.163635 16117 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:58.174135 16291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a378d7f796f94615b3b347119a9880fd" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
W20260709 13:21:58.155364 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 480057us
I20260709 13:21:58.192180 16291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d16dfad0c3b540e5b4b86492ddf87208" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:58.155650 16191 leader_election.cc:304] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
I20260709 13:21:58.132058 22196 raft_consensus.cc:697] T e5ad4eb281564936a59f41e387a47156 P 647a343a542d4a139fe106c19095129b [term 18 LEADER]: Becoming Leader. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
W20260709 13:21:58.197923 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 351733us
W20260709 13:21:58.180754 16190 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:58.200557 16291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:21:58.211894 16225 raft_consensus.cc:1240] T 1df93779209748b9860f6d893dd3de53 P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 16. Current term is 17. Ops: []
I20260709 13:21:58.214303 16292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fffab26427e24d398e2695aad40ca134" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:21:58.216814 16226 raft_consensus.cc:1275] T 8a878999c0024d2abe517484180290fc P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 16 index: 9. (term mismatch)
I20260709 13:21:58.217363 16226 pending_rounds.cc:85] T 8a878999c0024d2abe517484180290fc P 1ca2725d290e47d68b1a4b609c9816ac: Aborting all ops after (but not including) 8
I20260709 13:21:58.217653 16226 pending_rounds.cc:107] T 8a878999c0024d2abe517484180290fc P 1ca2725d290e47d68b1a4b609c9816ac: Aborting uncommitted NO_OP operation due to leader change: 15.9
I20260709 13:21:58.217957 16226 raft_consensus.cc:2889] T 8a878999c0024d2abe517484180290fc P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:21:58.171509 22645 raft_consensus.cc:2804] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:21:58.140085 22754 raft_consensus.cc:515] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:58.219972 22865 consensus_queue.cc:1059] T 7c2095a5f4ad49408dbc7f69c606120d P 90f3da57796b488c9087a2f523adf045 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: INVALID_TERM, Last received: 14.7, Next index: 8, Last known committed idx: 5, Time since last communication: 0.037s
W20260709 13:21:58.238843 16117 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:58.257157 16820 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ee9ed67006144797b1de6ea8c2013a68"
dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
mode: GRACEFUL
new_leader_uuid: "647a343a542d4a139fe106c19095129b"
 from {username='slave'} at 127.0.0.1:37818
I20260709 13:21:58.262682 16820 raft_consensus.cc:606] T ee9ed67006144797b1de6ea8c2013a68 P cb82cc7bd6db46d980c75237a1402860 [term 19 LEADER]: Received request to transfer leadership to TS 647a343a542d4a139fe106c19095129b
I20260709 13:21:58.224716 16115 leader_election.cc:304] T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac
W20260709 13:21:58.259562 16218 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33050 (request call id 1150) took 1493 ms (client timeout 1863 ms). Trace:
I20260709 13:21:58.266553 16819 raft_consensus.cc:1275] T 806d7690c2b644959152074bb2cab0a7 P cb82cc7bd6db46d980c75237a1402860 [term 18 LEARNER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 13:21:58.278767 16295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f82224f4564fea93572e8206bad1fe" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:58.278074 16220 raft_consensus.cc:1275] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [term 24 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 23 index: 15. Preceding OpId from leader: term: 24 index: 15. (term mismatch)
I20260709 13:21:58.280378 16220 pending_rounds.cc:85] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac: Aborting all ops after (but not including) 14
I20260709 13:21:58.280596 16220 pending_rounds.cc:107] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac: Aborting uncommitted NO_OP operation due to leader change: 23.15
I20260709 13:21:58.280807 16220 raft_consensus.cc:2889] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [term 24 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:21:58.265899 16221 raft_consensus.cc:1275] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
W20260709 13:21:58.283629 16115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 58974.8us
W20260709 13:21:58.284245 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 68464.7us
I20260709 13:21:58.272634 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cc959fb7f704fba9009c0595467d074" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:21:58.196442 22797 raft_consensus.cc:2804] T 57a744f5aaf944678a63469cbf514015 P cb82cc7bd6db46d980c75237a1402860 [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 13:21:58.266760 16218 rpcz_store.cc:269] 0709 13:21:56.766242 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:56.766427 (+   185us) service_pool.cc:224] Handling call
0709 13:21:58.259238 (+1492811us) spinlock_profiling.cc:243] Waited 1.19 s on lock 0x7b5400395154. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:21:58.259478 (+   240us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2727851392}
I20260709 13:21:58.196095 22593 raft_consensus.cc:3060] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:21:58.290263 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18ad7e3af5504571a69d10c00750a01c" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:21:58.178838 22888 raft_consensus.cc:2749] T 8638ee363273426db1393dd81e5c7b72 P cb82cc7bd6db46d980c75237a1402860 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:21:58.155460 22748 consensus_queue.cc:1048] T 746aabf856ab401c9ef5e859371e7b67 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.116s
I20260709 13:21:58.175182 22854 raft_consensus.cc:697] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [term 21 LEADER]: Becoming Leader. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:21:58.298754 16292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ab0ac8347641e6955e9bce7d90c1a2" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:21:58.139045 22782 consensus_queue.cc:260] T e5ad4eb281564936a59f41e387a47156 P 90f3da57796b488c9087a2f523adf045 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:58.299703 16148 raft_consensus.cc:1275] T 02838c86531d4f98bf58d1c22b8baf77 P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 13:21:58.303682 16115 leader_election.cc:304] T 02838c86531d4f98bf58d1c22b8baf77 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:21:58.305243 16822 raft_consensus.cc:1275] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860 [term 18 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 13:21:58.306780 16151 raft_consensus.cc:1240] T 7c2095a5f4ad49408dbc7f69c606120d P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 14. Current term is 15. Ops: []
I20260709 13:21:58.124531 22787 raft_consensus.cc:515] T 84f2da98864b4425bf7c0055951fdd4b P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:58.305856 16224 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33050 (request call id 1159) took 1006 ms. Trace:
I20260709 13:21:58.252440 22731 raft_consensus.cc:2749] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:21:58.312886 16143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806d7690c2b644959152074bb2cab0a7" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 19 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:58.227054 22645 raft_consensus.cc:493] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:58.322467 15046 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:21:58.322909 22645 raft_consensus.cc:3060] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:21:58.325372 22731 raft_consensus.cc:493] T 18ad7e3af5504571a69d10c00750a01c P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:58.326040 22731 raft_consensus.cc:515] T 18ad7e3af5504571a69d10c00750a01c P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } }
I20260709 13:21:58.327730 16144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57a744f5aaf944678a63469cbf514015" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:58.328464 16144 raft_consensus.cc:2468] T 57a744f5aaf944678a63469cbf514015 P 647a343a542d4a139fe106c19095129b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 14.
I20260709 13:21:58.328699 22731 leader_election.cc:290] T 18ad7e3af5504571a69d10c00750a01c P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:58.330108 22866 raft_consensus.cc:3055] T fffab26427e24d398e2695aad40ca134 P 90f3da57796b488c9087a2f523adf045 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:21:58.330852 22866 raft_consensus.cc:740] T fffab26427e24d398e2695aad40ca134 P 90f3da57796b488c9087a2f523adf045 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:21:58.297279 22797 raft_consensus.cc:493] T 57a744f5aaf944678a63469cbf514015 P cb82cc7bd6db46d980c75237a1402860 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:58.332193 22797 raft_consensus.cc:3060] T 57a744f5aaf944678a63469cbf514015 P cb82cc7bd6db46d980c75237a1402860 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:21:58.331857 22866 consensus_queue.cc:260] T fffab26427e24d398e2695aad40ca134 P 90f3da57796b488c9087a2f523adf045 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:58.315152 16224 rpcz_store.cc:276] 0709 13:21:57.299888 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:57.300010 (+   122us) service_pool.cc:224] Handling call
0709 13:21:57.347650 (+ 47640us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:21:57.347859 (+   209us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:21:57.347885 (+    26us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:21:57.380328 (+ 32443us) log.cc:844] Serialized 26 byte log entry
0709 13:21:57.703621 (+323293us) raft_consensus.cc:1645] Marking committed up to 16
0709 13:21:57.770312 (+ 66691us) log.cc:844] Serialized 14 byte log entry
0709 13:21:57.974615 (+204303us) spinlock_profiling.cc:243] Waited 8.69 ms on lock 0x7b18004c9720. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b693ade1 00007f69b69475b1 00007f69ada54e07 00007f69b68afab9 00007f69ada7169b 00007f69ada6f93f 00007f69ada6f455 00007f69b0b57060 00007f69b0b69716 00007f69b0b7a287 00007f69b0b7a208
0709 13:21:58.281746 (+307131us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:58.281812 (+    66us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:21:58.281850 (+    38us) raft_consensus.cc:1685] finished
0709 13:21:58.305672 (+ 23822us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:21:58.305803 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":66372,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":41,"raft.queue_time_us":8980,"raft.run_cpu_time_us":91,"raft.run_wall_time_us":91,"spinlock_wait_cycles":19988352,"thread_start_us":564186,"threads_started":2,"wal-append.queue_time_us":466913}
I20260709 13:21:58.337059 22866 raft_consensus.cc:3060] T fffab26427e24d398e2695aad40ca134 P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:21:58.209594 22874 consensus_queue.cc:1059] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: INVALID_TERM, Last received: 18.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.101s
I20260709 13:21:58.340498 22874 consensus_queue.cc:1243] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Peer 1ca2725d290e47d68b1a4b609c9816ac log is divergent from this leader: its last log entry 19.15 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 14
I20260709 13:21:58.341037 22874 raft_consensus.cc:3055] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:21:58.341380 22874 raft_consensus.cc:740] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:21:58.342199 22874 consensus_queue.cc:260] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:58.317775 16142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62fe3346cd3c42c4a55e14af9073bca4" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 25 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:58.347546 16142 raft_consensus.cc:2468] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 24.
I20260709 13:21:58.227198 22834 consensus_queue.cc:1059] T 5a96373bb0e14dce8a266b2fc6c58507 P 647a343a542d4a139fe106c19095129b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.089s
I20260709 13:21:58.348665 22834 raft_consensus.cc:3055] T 5a96373bb0e14dce8a266b2fc6c58507 P 647a343a542d4a139fe106c19095129b [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:21:58.349045 22834 raft_consensus.cc:740] T 5a96373bb0e14dce8a266b2fc6c58507 P 647a343a542d4a139fe106c19095129b [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:21:58.220991 22872 consensus_queue.cc:1059] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: INVALID_TERM, Last received: 16.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.090s
I20260709 13:21:58.301396 22830 raft_consensus.cc:2804] T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:21:58.352492 16295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a959dbecc2648eb85d1cc550f5a3245" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:58.297515 16222 raft_consensus.cc:2468] T e4dc6b0541ca431ebc757beac00521f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 19.
I20260709 13:21:58.358312 16149 raft_consensus.cc:2468] T 18ad7e3af5504571a69d10c00750a01c P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 16.
I20260709 13:21:58.363938 16116 leader_election.cc:304] T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
I20260709 13:21:58.354568 16219 pending_rounds.cc:85] T 84f2da98864b4425bf7c0055951fdd4b P 1ca2725d290e47d68b1a4b609c9816ac: Aborting all ops after (but not including) 8
I20260709 13:21:58.301065 22754 leader_election.cc:290] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:58.365623 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4dc6b0541ca431ebc757beac00521f1" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:21:58.366484 16298 raft_consensus.cc:3060] T e4dc6b0541ca431ebc757beac00521f1 P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:21:58.366855 16295 raft_consensus.cc:2368] T 5a959dbecc2648eb85d1cc550f5a3245 P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac for earlier term 16. Current term is 17.
I20260709 13:21:58.365707 16219 pending_rounds.cc:107] T 84f2da98864b4425bf7c0055951fdd4b P 1ca2725d290e47d68b1a4b609c9816ac: Aborting uncommitted NO_OP operation due to leader change: 14.9
I20260709 13:21:58.367580 16219 raft_consensus.cc:2889] T 84f2da98864b4425bf7c0055951fdd4b P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:21:58.237003 22707 raft_consensus.cc:515] T d16dfad0c3b540e5b4b86492ddf87208 P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:21:58.370704 16115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 67131us
I20260709 13:21:58.372025 16115 leader_election.cc:304] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac
I20260709 13:21:58.372510 16294 raft_consensus.cc:1240] T 84f2da98864b4425bf7c0055951fdd4b P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 14. Current term is 15. Ops: [14.9-14.9]
I20260709 13:21:58.287324 22630 consensus_queue.cc:1048] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.065s
I20260709 13:21:58.376044 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806d7690c2b644959152074bb2cab0a7" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:58.380366 16300 raft_consensus.cc:2468] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 18.
I20260709 13:21:58.377041 16825 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49896 (request call id 1497) took 1321 ms. Trace:
I20260709 13:21:58.380992 16225 raft_consensus.cc:1275] T c2f5f89186204242bf377f7c42b2e2da P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 13:21:58.379400 16299 raft_consensus.cc:2468] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 18.
I20260709 13:21:58.309221 22792 raft_consensus.cc:2749] T 434bff3082d141838068f323bdb0d1ed P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:21:58.386996 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:21:58.387890 16115 leader_election.cc:304] T fffab26427e24d398e2695aad40ca134 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:21:58.388444 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0eab25ce4c64e018a041beeda2867db" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:58.390661 16299 raft_consensus.cc:2468] T c0eab25ce4c64e018a041beeda2867db P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 18.
I20260709 13:21:58.391728 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1df93779209748b9860f6d893dd3de53" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:58.373366 22782 raft_consensus.cc:3060] T e5ad4eb281564936a59f41e387a47156 P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:21:58.255081 22196 consensus_queue.cc:237] T e5ad4eb281564936a59f41e387a47156 P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.11, Last appended by leader: 11, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:58.247963 22517 consensus_queue.cc:260] T 1df93779209748b9860f6d893dd3de53 P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:58.419338 16298 raft_consensus.cc:2468] T e4dc6b0541ca431ebc757beac00521f1 P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 19.
I20260709 13:21:58.393826 22457 raft_consensus.cc:2804] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:21:58.423902 16295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c2095a5f4ad49408dbc7f69c606120d" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 15 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:58.424044 22833 consensus_queue.cc:1048] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:21:58.424758 22707 leader_election.cc:290] T d16dfad0c3b540e5b4b86492ddf87208 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 20 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:58.424861 16822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26354361e9b044cea1c2b86c4882fe92" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:58.360900 22886 consensus_queue.cc:1048] T 8a878999c0024d2abe517484180290fc P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.026s
I20260709 13:21:58.381151 16825 rpcz_store.cc:276] 0709 13:21:57.046601 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:57.046781 (+   180us) service_pool.cc:224] Handling call
0709 13:21:57.070272 (+ 23491us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:21:57.123465 (+ 53193us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:21:57.123497 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:21:57.132520 (+  9023us) log.cc:844] Serialized 26 byte log entry
0709 13:21:57.784669 (+652149us) spinlock_profiling.cc:243] Waited 6.97 ms on lock 0x7b1800353ec0. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b693ade1 00007f69b69475b1 00007f69ada54e07 00007f69b68afab9 00007f69ada7169b 00007f69ada6f93f 00007f69ada71c02 00007f69b06f920b 00007f69b06fed23 00007f69b06ff0d1 00007f69b0b1aa0f
0709 13:21:58.159183 (+374514us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:21:58.159236 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:58.160183 (+   947us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:21:58.356346 (+196163us) raft_consensus.cc:1685] finished
0709 13:21:58.356376 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:21:58.367994 (+ 11618us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":52106,"raft.queue_time_us":59756,"raft.run_cpu_time_us":104,"raft.run_wall_time_us":1890,"spinlock_wait_cycles":16022272,"thread_start_us":374262,"threads_started":1,"wal-append.queue_time_us":1079962}
I20260709 13:21:58.378791 22699 raft_consensus.cc:2749] T 02838c86531d4f98bf58d1c22b8baf77 P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:21:58.384122 22610 raft_consensus.cc:2804] T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:21:58.428484 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c74490c40243a88e62d1760b5cd814" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:21:58.430969 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5ad4eb281564936a59f41e387a47156" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:21:58.375144 22709 consensus_queue.cc:1048] T 02838c86531d4f98bf58d1c22b8baf77 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.054s
I20260709 13:21:58.384356 22830 raft_consensus.cc:697] T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b [term 20 LEADER]: Becoming Leader. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:21:58.457552 16295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87ac0d1f7e8a419f9bcbc04aee0e067e" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
W20260709 13:21:58.459738 22904 log.cc:927] Time spent T 74ca8140f35c4ae29e8a664f6bdfc829 P cb82cc7bd6db46d980c75237a1402860: Append to log took a long time: real 0.064s	user 0.000s	sys 0.000s
W20260709 13:21:58.356546 22873 consensus_queue.cc:1175] T e5ad4eb281564936a59f41e387a47156 P 90f3da57796b488c9087a2f523adf045 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "1ca2725d290e47d68b1a4b609c9816ac" responder_term: 18 status { last_received { term: 17 index: 11 } last_committed_idx: 10 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 17. Current term is 18. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:21:58.437901 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a96373bb0e14dce8a266b2fc6c58507" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:58.385739 22874 raft_consensus.cc:3060] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:21:58.378695 22439 raft_consensus.cc:3055] T 7c2095a5f4ad49408dbc7f69c606120d P 90f3da57796b488c9087a2f523adf045 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:21:58.374061 22864 raft_consensus.cc:515] T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:58.450402 22356 raft_consensus.cc:3060] T 68a6838d82ef4cac83507f906139d90f P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:21:58.452881 16296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c74490c40243a88e62d1760b5cd814" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:58.441148 16822 raft_consensus.cc:2468] T 26354361e9b044cea1c2b86c4882fe92 P cb82cc7bd6db46d980c75237a1402860 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 15.
I20260709 13:21:58.444643 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d16dfad0c3b540e5b4b86492ddf87208" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:21:58.395044 22837 raft_consensus.cc:2749] T fffab26427e24d398e2695aad40ca134 P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:21:58.468125 16296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f2da98864b4425bf7c0055951fdd4b" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:58.468817 16225 raft_consensus.cc:3060] T d16dfad0c3b540e5b4b86492ddf87208 P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:21:58.472214 16295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7ca61592e848ea9b0fb81a2ce33cdc" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:58.443861 22457 raft_consensus.cc:697] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b [term 18 LEADER]: Becoming Leader. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:21:58.463961 22439 raft_consensus.cc:740] T 7c2095a5f4ad49408dbc7f69c606120d P 90f3da57796b488c9087a2f523adf045 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:21:58.465514 22864 leader_election.cc:290] T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 20 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:21:58.480113 22457 consensus_queue.cc:237] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:58.481045 22676 raft_consensus.cc:493] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:21:58.481587 22676 raft_consensus.cc:515] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:58.482085 16296 raft_consensus.cc:1217] T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Deduplicated request from leader. Original: 13.11->[15.12-18.13]   Dedup: 18.13->[]
I20260709 13:21:58.482650 16296 raft_consensus.cc:1240] T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 18. Current term is 19. Ops: [15.12-18.13]
I20260709 13:21:58.483848 16296 raft_consensus.cc:1240] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 17. Current term is 18. Ops: []
I20260709 13:21:58.489151 22645 raft_consensus.cc:515] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:58.491472 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c2095a5f4ad49408dbc7f69c606120d" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
W20260709 13:21:58.479281 16264 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:58.480134 22439 consensus_queue.cc:260] T 7c2095a5f4ad49408dbc7f69c606120d P 90f3da57796b488c9087a2f523adf045 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.7, Last appended by leader: 7, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:58.494805 22439 raft_consensus.cc:3060] T 7c2095a5f4ad49408dbc7f69c606120d P 90f3da57796b488c9087a2f523adf045 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:21:58.495913 22676 leader_election.cc:290] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:21:58.497200 16146 raft_consensus.cc:1275] T 68a6838d82ef4cac83507f906139d90f P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 17 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 13:21:58.498322 22645 leader_election.cc:290] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 20 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:21:58.498848 16225 raft_consensus.cc:2468] T d16dfad0c3b540e5b4b86492ddf87208 P 1ca2725d290e47d68b1a4b609c9816ac [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 20.
I20260709 13:21:58.500875 16116 leader_election.cc:304] T d16dfad0c3b540e5b4b86492ddf87208 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
I20260709 13:21:58.505002 22787 leader_election.cc:290] T 84f2da98864b4425bf7c0055951fdd4b P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
W20260709 13:21:58.505491 16115 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:58.505919 22630 raft_consensus.cc:2804] T d16dfad0c3b540e5b4b86492ddf87208 P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:21:58.506651 22630 raft_consensus.cc:697] T d16dfad0c3b540e5b4b86492ddf87208 P 647a343a542d4a139fe106c19095129b [term 20 LEADER]: Becoming Leader. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:21:58.507603 22630 consensus_queue.cc:237] T d16dfad0c3b540e5b4b86492ddf87208 P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:58.511124 16147 raft_consensus.cc:1275] T b6f07842d3aa47858df5665874c86f3a P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 13:21:58.515902 16217 raft_consensus.cc:1275] T c0eab25ce4c64e018a041beeda2867db P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 18 index: 10. (term mismatch)
I20260709 13:21:58.516294 16217 pending_rounds.cc:85] T c0eab25ce4c64e018a041beeda2867db P 1ca2725d290e47d68b1a4b609c9816ac: Aborting all ops after (but not including) 9
I20260709 13:21:58.516530 16217 pending_rounds.cc:107] T c0eab25ce4c64e018a041beeda2867db P 1ca2725d290e47d68b1a4b609c9816ac: Aborting uncommitted NO_OP operation due to leader change: 17.10
I20260709 13:21:58.516753 16217 raft_consensus.cc:2889] T c0eab25ce4c64e018a041beeda2867db P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:21:58.516851 16298 raft_consensus.cc:2393] T 5a96373bb0e14dce8a266b2fc6c58507 P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 15: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
I20260709 13:21:58.520032 22356 consensus_queue.cc:1048] T c0eab25ce4c64e018a041beeda2867db P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.002s
I20260709 13:21:58.509603 22622 raft_consensus.cc:515] T c9944d5e4b8c4109af9955c4f08c9095 P cb82cc7bd6db46d980c75237a1402860 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:58.525305 16265 leader_election.cc:304] T 18ad7e3af5504571a69d10c00750a01c P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:21:58.526250 22837 consensus_queue.cc:1048] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:21:58.527513 22709 raft_consensus.cc:2804] T 18ad7e3af5504571a69d10c00750a01c P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:21:58.528090 22709 raft_consensus.cc:697] T 18ad7e3af5504571a69d10c00750a01c P 90f3da57796b488c9087a2f523adf045 [term 16 LEADER]: Becoming Leader. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:21:58.428113 22494 consensus_queue.cc:1059] T 1df93779209748b9860f6d893dd3de53 P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: INVALID_TERM, Last received: 16.10, Next index: 11, Last known committed idx: 8, Time since last communication: 0.044s
I20260709 13:21:58.461187 22610 raft_consensus.cc:697] T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b [term 19 LEADER]: Becoming Leader. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:21:58.532169 16295 raft_consensus.cc:1240] T 68a6838d82ef4cac83507f906139d90f P 90f3da57796b488c9087a2f523adf045 [term 18 LEADER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 17. Current term is 18. Ops: []
I20260709 13:21:58.528791 15046 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:21:58.394026 22857 consensus_queue.cc:1059] T 3884f864a96d428785895e5463e5728f P 647a343a542d4a139fe106c19095129b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: INVALID_TERM, Last received: 13.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.069s
I20260709 13:21:58.534127 16225 raft_consensus.cc:1275] T 5a959dbecc2648eb85d1cc550f5a3245 P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 13:21:58.535692 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c61a39fd4f84424b129c1a6f9ba404d" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 20 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:21:58.538954 16300 raft_consensus.cc:2468] T 7c2095a5f4ad49408dbc7f69c606120d P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 15.
I20260709 13:21:58.540072 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f5f89186204242bf377f7c42b2e2da" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
W20260709 13:21:58.545483 16114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:58.551587 16144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f2da98864b4425bf7c0055951fdd4b" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:58.542923 16298 raft_consensus.cc:1240] T 434bff3082d141838068f323bdb0d1ed P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 16. Current term is 17. Ops: []
I20260709 13:21:58.558477 16144 raft_consensus.cc:1240] T ecc1e363230043fe9fef390f0b979f20 P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 19. Current term is 20. Ops: []
I20260709 13:21:58.566309 16298 raft_consensus.cc:1240] T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 19. Current term is 20. Ops: []
I20260709 13:21:58.533641 22787 consensus_queue.cc:1048] T 68a6838d82ef4cac83507f906139d90f P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.026s
I20260709 13:21:58.558794 22748 raft_consensus.cc:493] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:58.537962 22797 raft_consensus.cc:515] T 57a744f5aaf944678a63469cbf514015 P cb82cc7bd6db46d980c75237a1402860 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:58.590250 16222 raft_consensus.cc:1275] T 1df93779209748b9860f6d893dd3de53 P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 13:21:58.591652 16222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f5f89186204242bf377f7c42b2e2da" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:21:58.592325 16222 raft_consensus.cc:3060] T c2f5f89186204242bf377f7c42b2e2da P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:21:58.456934 22854 consensus_queue.cc:237] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 21, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:58.611912 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cc959fb7f704fba9009c0595467d074" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
W20260709 13:21:58.611788 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 464350us
I20260709 13:21:58.610122 16220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806d7690c2b644959152074bb2cab0a7" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 19 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:58.614882 16144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18ad7e3af5504571a69d10c00750a01c" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 17 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:58.615767 16144 raft_consensus.cc:2410] T 18ad7e3af5504571a69d10c00750a01c P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac for term 17 because replica has last-logged OpId of term: 15 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 10.
I20260709 13:21:58.618817 16221 raft_consensus.cc:1275] T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
W20260709 13:21:58.623051 16191 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:21:58.623755 16191 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:58.612000 22886 raft_consensus.cc:493] T 26354361e9b044cea1c2b86c4882fe92 P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:58.636368 16222 raft_consensus.cc:2468] T c2f5f89186204242bf377f7c42b2e2da P 1ca2725d290e47d68b1a4b609c9816ac [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 20.
I20260709 13:21:58.640245 16191 leader_election.cc:304] T ecc1e363230043fe9fef390f0b979f20 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
I20260709 13:21:58.641535 22677 raft_consensus.cc:2749] T ecc1e363230043fe9fef390f0b979f20 P 1ca2725d290e47d68b1a4b609c9816ac [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:21:58.641813 16191 leader_election.cc:304] T 829b4ad8c5a54eababeea1181b960d4b P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
I20260709 13:21:58.643018 22677 raft_consensus.cc:2749] T 829b4ad8c5a54eababeea1181b960d4b P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:21:58.646569 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87ac0d1f7e8a419f9bcbc04aee0e067e" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:58.573872 22830 consensus_queue.cc:237] T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:58.583841 22748 raft_consensus.cc:515] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:58.655490 16149 raft_consensus.cc:2468] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 19.
I20260709 13:21:58.657989 16115 leader_election.cc:304] T 23f82224f4564fea93572e8206bad1fe P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860
I20260709 13:21:58.569309 22709 consensus_queue.cc:237] T 18ad7e3af5504571a69d10c00750a01c P 90f3da57796b488c9087a2f523adf045 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } }
I20260709 13:21:58.663981 16221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f2da98864b4425bf7c0055951fdd4b" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:58.632162 22738 raft_consensus.cc:493] T 3884f864a96d428785895e5463e5728f P cb82cc7bd6db46d980c75237a1402860 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
W20260709 13:21:58.669649 16263 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:58.670360 16191 leader_election.cc:304] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
I20260709 13:21:58.544890 16042 catalog_manager.cc:5697] T e5ad4eb281564936a59f41e387a47156 P 647a343a542d4a139fe106c19095129b reported cstate change: term changed from 13 to 16, leader changed from 90f3da57796b488c9087a2f523adf045 (127.14.177.131) to 647a343a542d4a139fe106c19095129b (127.14.177.129). New cstate: current_term: 16 leader_uuid: "647a343a542d4a139fe106c19095129b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } health_report { overall_health: HEALTHY } } }
I20260709 13:21:58.673141 16144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2372981006d4b218215c28964dea623" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:21:58.677999 16144 raft_consensus.cc:2376] T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b [term 21 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 21. Re-sending same reply.
I20260709 13:21:58.679488 16221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66ae176961b74813a58bc15c9c4c6f2a" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 24 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:21:58.625840 22645 consensus_queue.cc:1048] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.041s
I20260709 13:21:58.658450 22622 leader_election.cc:290] T c9944d5e4b8c4109af9955c4f08c9095 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 21 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:58.689229 16296 raft_consensus.cc:1275] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 13:21:58.668059 22196 raft_consensus.cc:2749] T 23f82224f4564fea93572e8206bad1fe P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:21:58.667438 22699 raft_consensus.cc:493] T 434bff3082d141838068f323bdb0d1ed P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:58.696781 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d16dfad0c3b540e5b4b86492ddf87208" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:21:58.612092 22610 consensus_queue.cc:237] T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.9, Last appended by leader: 9, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:58.649854 22439 consensus_queue.cc:1048] T 8d4ab870a05347868956aa8b841b2981 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.031s
W20260709 13:21:58.694742 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 216833us
I20260709 13:21:58.699647 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d4ab870a05347868956aa8b841b2981" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:58.705089 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9944d5e4b8c4109af9955c4f08c9095" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:21:58.705775 16293 raft_consensus.cc:3060] T c9944d5e4b8c4109af9955c4f08c9095 P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:21:58.701941 16790 leader_election.cc:304] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860; no voters: 
W20260709 13:21:58.707777 16264 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:58.652204 22886 raft_consensus.cc:515] T 26354361e9b044cea1c2b86c4882fe92 P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:21:58.712034 16115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 46229.7us
I20260709 13:21:58.715404 16115 leader_election.cc:304] T ecc1e363230043fe9fef390f0b979f20 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860
I20260709 13:21:58.727382 22672 consensus_queue.cc:1059] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: INVALID_TERM, Last received: 19.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:21:58.729307 22672 raft_consensus.cc:3055] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:21:58.729849 22672 raft_consensus.cc:740] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:21:58.637979 22914 consensus_queue.cc:1048] T 1df93779209748b9860f6d893dd3de53 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.050s
W20260709 13:21:58.708987 22907 log.cc:927] Time spent T fffab26427e24d398e2695aad40ca134 P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.175s	user 0.000s	sys 0.001s
I20260709 13:21:58.685052 22916 raft_consensus.cc:2749] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:21:58.732616 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c61a39fd4f84424b129c1a6f9ba404d" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 20 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:21:58.744397 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18ad7e3af5504571a69d10c00750a01c" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:21:58.677237 22738 raft_consensus.cc:515] T 3884f864a96d428785895e5463e5728f P cb82cc7bd6db46d980c75237a1402860 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:58.744145 16191 leader_election.cc:304] T ee9ed67006144797b1de6ea8c2013a68 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
I20260709 13:21:58.753196 16226 raft_consensus.cc:2393] T 18ad7e3af5504571a69d10c00750a01c P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 16: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
I20260709 13:21:58.714336 22622 raft_consensus.cc:2804] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:21:58.764307 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62fe3346cd3c42c4a55e14af9073bca4" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 25 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
W20260709 13:21:58.739300 22911 log.cc:927] Time spent T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860: Append to log took a long time: real 0.196s	user 0.000s	sys 0.001s
I20260709 13:21:58.769286 16221 raft_consensus.cc:2468] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 24.
I20260709 13:21:58.770671 16295 raft_consensus.cc:1275] T 8a878999c0024d2abe517484180290fc P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 13:21:58.711329 22677 consensus_queue.cc:1048] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.055s
I20260709 13:21:58.703929 22699 raft_consensus.cc:515] T 434bff3082d141838068f323bdb0d1ed P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:58.715746 22908 consensus_queue.cc:1059] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.057s
I20260709 13:21:58.757512 22857 raft_consensus.cc:2749] T ecc1e363230043fe9fef390f0b979f20 P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:21:58.785262 16295 raft_consensus.cc:1240] T 02838c86531d4f98bf58d1c22b8baf77 P 90f3da57796b488c9087a2f523adf045 [term 19 LEADER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 18. Current term is 19. Ops: []
I20260709 13:21:58.766901 22833 raft_consensus.cc:493] T b6f07842d3aa47858df5665874c86f3a P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:21:58.777797 22909 log.cc:927] Time spent T 66ae176961b74813a58bc15c9c4c6f2a P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.204s	user 0.000s	sys 0.001s
I20260709 13:21:58.771098 22622 raft_consensus.cc:493] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [term 24 FOLLOWER]: Starting leader election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:21:58.775641 22916 raft_consensus.cc:2749] T ee9ed67006144797b1de6ea8c2013a68 P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:21:58.753859 22754 consensus_queue.cc:1048] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.030s
I20260709 13:21:58.754070 22731 consensus_queue.cc:1059] T 02838c86531d4f98bf58d1c22b8baf77 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.029s
I20260709 13:21:58.796613 16295 raft_consensus.cc:1240] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 19. Current term is 20. Ops: []
I20260709 13:21:58.821031 16226 raft_consensus.cc:1275] T 8c11074579b046d99e585c2818e18a2c P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 13:21:58.769035 22196 consensus_queue.cc:1059] T 9c61a39fd4f84424b129c1a6f9ba404d P 647a343a542d4a139fe106c19095129b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.022s
I20260709 13:21:58.823411 22622 raft_consensus.cc:3060] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:21:58.758759 22917 consensus_queue.cc:1059] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: INVALID_TERM, Last received: 18.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.075s
I20260709 13:21:58.780735 22676 consensus_queue.cc:1048] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.046s
I20260709 13:21:58.843626 16148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57a744f5aaf944678a63469cbf514015" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:21:58.844326 16148 raft_consensus.cc:3060] T 57a744f5aaf944678a63469cbf514015 P 647a343a542d4a139fe106c19095129b [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:21:58.842675 16295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57a744f5aaf944678a63469cbf514015" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:21:58.845667 16295 raft_consensus.cc:3060] T 57a744f5aaf944678a63469cbf514015 P 90f3da57796b488c9087a2f523adf045 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:21:58.840131 16821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57a744f5aaf944678a63469cbf514015" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
W20260709 13:21:58.849459 22912 log.cc:927] Time spent T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 0.169s	user 0.001s	sys 0.000s
W20260709 13:21:58.850199 22913 log.cc:927] Time spent T 8cc959fb7f704fba9009c0595467d074 P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 0.213s	user 0.000s	sys 0.001s
W20260709 13:21:58.851945 22928 log.cc:927] Time spent T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860: Append to log took a long time: real 0.068s	user 0.000s	sys 0.000s
W20260709 13:21:58.854423 22923 log.cc:927] Time spent T e5ad4eb281564936a59f41e387a47156 P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 0.107s	user 0.000s	sys 0.001s
I20260709 13:21:58.784469 22457 consensus_queue.cc:1048] T 8a878999c0024d2abe517484180290fc P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.055s
I20260709 13:21:58.856367 16143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2372981006d4b218215c28964dea623" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:58.875033 16143 raft_consensus.cc:2468] T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 21.
I20260709 13:21:58.829382 22908 raft_consensus.cc:3055] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 1ca2725d290e47d68b1a4b609c9816ac [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:21:58.867408 22748 leader_election.cc:290] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:58.868924 16820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26354361e9b044cea1c2b86c4882fe92" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:21:58.869674 22857 consensus_queue.cc:1059] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
W20260709 13:21:58.870265 22593 consensus_meta.cc:287] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045: Time spent flushing consensus metadata: real 0.568s	user 0.006s	sys 0.006s
I20260709 13:21:58.820225 22833 raft_consensus.cc:515] T b6f07842d3aa47858df5665874c86f3a P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:58.870958 22886 leader_election.cc:290] T 26354361e9b044cea1c2b86c4882fe92 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
W20260709 13:21:58.850069 22926 log.cc:927] Time spent T 8158bbd17b994d2eb96f560bd0f1b88d P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 0.096s	user 0.000s	sys 0.001s
I20260709 13:21:58.871610 16821 raft_consensus.cc:2368] T 57a744f5aaf944678a63469cbf514015 P cb82cc7bd6db46d980c75237a1402860 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 for earlier term 14. Current term is 15.
I20260709 13:21:58.873554 16265 leader_election.cc:304] T 8cc959fb7f704fba9009c0595467d074 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:21:58.733826 22672 consensus_queue.cc:260] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:21:58.879789 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 135782us
W20260709 13:21:58.882047 16190 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:21:58.874848 22915 log.cc:927] Time spent T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860: Append to log took a long time: real 0.197s	user 0.000s	sys 0.001s
I20260709 13:21:58.890760 16825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8638ee363273426db1393dd81e5c7b72" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
W20260709 13:21:58.933115 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 59661.9us
W20260709 13:21:58.896183 16264 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:58.876488 22908 raft_consensus.cc:740] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 1ca2725d290e47d68b1a4b609c9816ac [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:21:58.940404 16816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c11074579b046d99e585c2818e18a2c" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
W20260709 13:21:58.941092 22932 log.cc:927] Time spent T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
I20260709 13:21:58.943246 22748 consensus_queue.cc:1048] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
W20260709 13:21:58.943971 22930 log.cc:927] Time spent T 02838c86531d4f98bf58d1c22b8baf77 P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 0.064s	user 0.000s	sys 0.001s
W20260709 13:21:58.946345 22934 log.cc:927] Time spent T 746aabf856ab401c9ef5e859371e7b67 P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
I20260709 13:21:58.883610 22916 raft_consensus.cc:3055] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac [term 19 LEADER]: Stepping down as leader of term 19
W20260709 13:21:58.948884 16190 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:21:58.949365 16190 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:58.937932 16191 leader_election.cc:304] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
I20260709 13:21:58.901482 16148 raft_consensus.cc:2468] T 57a744f5aaf944678a63469cbf514015 P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 15.
I20260709 13:21:58.937449 16265 leader_election.cc:304] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:21:58.949220 16819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66ae176961b74813a58bc15c9c4c6f2a" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 24 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:58.950614 16820 raft_consensus.cc:2468] T 26354361e9b044cea1c2b86c4882fe92 P cb82cc7bd6db46d980c75237a1402860 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 15.
I20260709 13:21:58.951195 16816 raft_consensus.cc:2468] T 8c11074579b046d99e585c2818e18a2c P cb82cc7bd6db46d980c75237a1402860 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 19.
I20260709 13:21:58.890473 22593 raft_consensus.cc:2749] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:21:58.954612 16265 leader_election.cc:304] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 647a343a542d4a139fe106c19095129b
I20260709 13:21:58.958873 16820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57a744f5aaf944678a63469cbf514015" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:58.959308 16265 leader_election.cc:304] T 84f2da98864b4425bf7c0055951fdd4b P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:21:58.960348 16820 raft_consensus.cc:2368] T 57a744f5aaf944678a63469cbf514015 P cb82cc7bd6db46d980c75237a1402860 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 for earlier term 14. Current term is 15.
I20260709 13:21:58.960896 16265 leader_election.cc:304] T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:21:58.939344 22914 raft_consensus.cc:2749] T 8cc959fb7f704fba9009c0595467d074 P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
W20260709 13:21:58.978142 16190 leader_election.cc:336] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 20 election: RPC error from VoteRequest() call to peer 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389): Timed out: RequestConsensusVote RPC to 127.14.177.131:41389 timed out after 1.554s (SENT)
I20260709 13:21:58.980986 22865 raft_consensus.cc:3055] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:21:58.981654 22865 raft_consensus.cc:740] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:21:58.985821 22622 raft_consensus.cc:515] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:58.986259 16816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8638ee363273426db1393dd81e5c7b72" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:58.990089 16825 raft_consensus.cc:2468] T 8638ee363273426db1393dd81e5c7b72 P cb82cc7bd6db46d980c75237a1402860 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 19.
I20260709 13:21:58.953363 22916 raft_consensus.cc:740] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:21:58.957396 22866 raft_consensus.cc:2749] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:21:58.996220 16114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:58.965406 22864 raft_consensus.cc:2804] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:21:58.962604 22537 raft_consensus.cc:493] T 08c74490c40243a88e62d1760b5cd814 P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:21:58.979218 16190 leader_election.cc:304] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860
I20260709 13:21:58.972035 22787 raft_consensus.cc:2749] T 84f2da98864b4425bf7c0055951fdd4b P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:21:59.015292 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 64381.7us
W20260709 13:21:59.023857 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 45781.5us
W20260709 13:21:59.024931 16190 leader_election.cc:336] T 7c2095a5f4ad49408dbc7f69c606120d P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389): Timed out: RequestConsensusVote RPC to 127.14.177.131:41389 timed out after 2.005s (SENT)
I20260709 13:21:58.991380 22935 raft_consensus.cc:2804] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:21:59.033334 16819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 21 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:59.034610 16819 raft_consensus.cc:2393] T 19baa7bc367844c288bca8f81439c413 P cb82cc7bd6db46d980c75237a1402860 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 21: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
I20260709 13:21:59.036713 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9944d5e4b8c4109af9955c4f08c9095" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:21:59.037590 16225 raft_consensus.cc:3060] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Advancing to term 21
I20260709 13:21:59.001183 22593 raft_consensus.cc:2749] T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:21:59.024992 22864 raft_consensus.cc:697] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045 [term 20 LEADER]: Becoming Leader. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:21:59.035625 22935 raft_consensus.cc:493] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:59.045975 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2372981006d4b218215c28964dea623" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:59.035856 22938 raft_consensus.cc:2749] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:21:59.051338 22885 raft_consensus.cc:493] T 434bff3082d141838068f323bdb0d1ed P cb82cc7bd6db46d980c75237a1402860 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:21:59.052210 22937 log.cc:927] Time spent T d16dfad0c3b540e5b4b86492ddf87208 P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 0.096s	user 0.001s	sys 0.000s
I20260709 13:21:59.012850 22738 leader_election.cc:290] T 3884f864a96d428785895e5463e5728f P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:59.045980 22908 consensus_queue.cc:260] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.11, Last appended by leader: 11, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:59.053449 16293 raft_consensus.cc:2468] T c9944d5e4b8c4109af9955c4f08c9095 P 90f3da57796b488c9087a2f523adf045 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 21.
I20260709 13:21:59.053951 16822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3884f864a96d428785895e5463e5728f" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:21:59.069581 16822 raft_consensus.cc:3060] T 3884f864a96d428785895e5463e5728f P cb82cc7bd6db46d980c75237a1402860 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:21:59.046339 22864 consensus_queue.cc:237] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:59.062855 22348 raft_consensus.cc:493] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:21:59.064718 16297 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40474 (request call id 1221) took 1039 ms. Trace:
I20260709 13:21:59.065807 22196 raft_consensus.cc:3055] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:21:59.068024 16820 raft_consensus.cc:1217] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P cb82cc7bd6db46d980c75237a1402860 [term 19 FOLLOWER]: Deduplicated request from leader. Original: 17.13->[18.14-18.14]   Dedup: 18.14->[]
I20260709 13:21:59.086946 16820 raft_consensus.cc:1240] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P cb82cc7bd6db46d980c75237a1402860 [term 19 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 18. Current term is 19. Ops: [18.14-18.14]
I20260709 13:21:58.983009 22865 consensus_queue.cc:260] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:59.087453 16295 raft_consensus.cc:2468] T 57a744f5aaf944678a63469cbf514015 P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 15.
I20260709 13:21:59.073284 16291 raft_consensus.cc:2468] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 21.
I20260709 13:21:59.076094 16297 rpcz_store.cc:276] 0709 13:21:58.025789 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:58.025965 (+   176us) service_pool.cc:224] Handling call
0709 13:21:58.030848 (+  4883us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:21:58.031273 (+   425us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:21:58.031304 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:21:58.031796 (+   492us) log.cc:844] Serialized 26 byte log entry
0709 13:21:58.354159 (+322363us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:21:58.354200 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:58.354282 (+    82us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:21:59.061202 (+706920us) spinlock_profiling.cc:243] Waited 2.61 ms on lock 0x7b240024bb58. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69ad8fadb1 00007f69ad8f89c4 00007f69ad8f6acb 00007f69adab1dc9 00007f69adab1a55 00007f69adab0fb0 00007f69b0727488 00007f69b0727be8 00007f69b06ff398 00007f69b06fa76a 00007f69b06f9bb6
0709 13:21:59.064430 (+  3228us) raft_consensus.cc:1685] finished
0709 13:21:59.064449 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:21:59.064657 (+   208us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1151,"raft.run_cpu_time_us":82,"raft.run_wall_time_us":83,"spinlock_wait_cycles":5998336,"thread_start_us":321971,"threads_started":1,"wal-append.queue_time_us":508261}
W20260709 13:21:59.088599 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 127804us
I20260709 13:21:59.092397 16789 leader_election.cc:304] T c9944d5e4b8c4109af9955c4f08c9095 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:21:59.093665 16789 leader_election.cc:304] T 57a744f5aaf944678a63469cbf514015 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:21:59.053205 22699 leader_election.cc:290] T 434bff3082d141838068f323bdb0d1ed P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:59.021323 22537 raft_consensus.cc:515] T 08c74490c40243a88e62d1760b5cd814 P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:59.099706 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26354361e9b044cea1c2b86c4882fe92" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:59.106704 16225 raft_consensus.cc:2468] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 21.
I20260709 13:21:59.108388 16295 raft_consensus.cc:1275] T c0eab25ce4c64e018a041beeda2867db P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 13:21:59.046372 22935 raft_consensus.cc:3060] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:21:59.055302 22938 raft_consensus.cc:493] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:21:59.051990 22885 raft_consensus.cc:515] T 434bff3082d141838068f323bdb0d1ed P cb82cc7bd6db46d980c75237a1402860 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:21:59.125480 16036 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 19baa7bc367844c288bca8f81439c413 with cas_config_opid_index 7: ChangeConfig:REMOVE_PEER failed with leader cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica cb82cc7bd6db46d980c75237a1402860 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 21 committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:21:59.097162 22348 raft_consensus.cc:515] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:59.133234 16818 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49906 (request call id 1299) took 1017 ms. Trace:
W20260709 13:21:59.133423 22936 log.cc:927] Time spent T 68a6838d82ef4cac83507f906139d90f P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 0.141s	user 0.001s	sys 0.000s
I20260709 13:21:59.134644 22508 raft_consensus.cc:493] T d2ab0ac8347641e6955e9bce7d90c1a2 P cb82cc7bd6db46d980c75237a1402860 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:21:59.135290 22925 log.cc:927] Time spent T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 0.208s	user 0.000s	sys 0.001s
I20260709 13:21:59.136466 22673 raft_consensus.cc:993] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860: : Instructing follower 90f3da57796b488c9087a2f523adf045 to start an election
W20260709 13:21:59.137079 22941 log.cc:927] Time spent T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 0.069s	user 0.000s	sys 0.000s
W20260709 13:21:59.126451 16263 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:59.096972 22196 raft_consensus.cc:740] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
W20260709 13:21:59.156334 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 48462.7us
I20260709 13:21:59.108812 22676 raft_consensus.cc:2804] T c9944d5e4b8c4109af9955c4f08c9095 P cb82cc7bd6db46d980c75237a1402860 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:21:59.170183 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3884f864a96d428785895e5463e5728f" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:59.171128 16297 raft_consensus.cc:2393] T 3884f864a96d428785895e5463e5728f P 90f3da57796b488c9087a2f523adf045 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 in current term 14: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
I20260709 13:21:59.168289 22677 consensus_queue.cc:1059] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: INVALID_TERM, Last received: 18.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:21:59.168910 22954 raft_consensus.cc:493] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:59.114568 22738 raft_consensus.cc:2804] T 57a744f5aaf944678a63469cbf514015 P cb82cc7bd6db46d980c75237a1402860 [term 15 FOLLOWER]: Leader election won for term 15
W20260709 13:21:59.131458 22939 log.cc:927] Time spent T 9e4bfb9090c8460eb14fa7e0d818e7f1 P cb82cc7bd6db46d980c75237a1402860: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
I20260709 13:21:59.133697 16818 rpcz_store.cc:276] 0709 13:21:58.050913 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:58.077191 (+ 26278us) service_pool.cc:224] Handling call
0709 13:21:58.097715 (+ 20524us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:21:58.108810 (+ 11095us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:21:58.108862 (+    52us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:21:58.109700 (+   838us) log.cc:844] Serialized 52 byte log entry
0709 13:21:58.822191 (+712491us) spinlock_profiling.cc:243] Waited 13.8 ms on lock 0x7b18004714c0. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b693ade1 00007f69b69475b1 00007f69ada54e07 00007f69b68afab9 00007f69ada7169b 00007f69ada6f93f 00007f69ada71c02 00007f69b06f920b 00007f69b06fed23 00007f69b06ff0d1 00007f69b0b1aa0f
0709 13:21:58.962512 (+140321us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:21:59.053932 (+ 91420us) log.cc:844] Serialized 14 byte log entry
0709 13:21:59.054228 (+   296us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:59.054334 (+   106us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:21:59.054368 (+    34us) raft_consensus.cc:1685] finished
0709 13:21:59.057563 (+  3195us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:21:59.068458 (+ 10895us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":149416,"prepare.run_cpu_time_us":47,"prepare.run_wall_time_us":47,"raft.queue_time_us":8607,"raft.run_cpu_time_us":251,"raft.run_wall_time_us":10604,"spinlock_wait_cycles":31711488,"thread_start_us":217839,"threads_started":2,"wal-append.queue_time_us":907448}
I20260709 13:21:59.137071 22673 raft_consensus.cc:1081] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [term 24 LEADER]: Signalling peer 90f3da57796b488c9087a2f523adf045 to start an election
I20260709 13:21:59.112023 22946 consensus_queue.cc:1048] T 8cc959fb7f704fba9009c0595467d074 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.064s
I20260709 13:21:59.140205 22865 raft_consensus.cc:3060] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:21:59.162564 22908 raft_consensus.cc:3060] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:21:59.134665 22348 leader_election.cc:290] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:59.201171 16142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66f75124215f4ca6bef831e174a792d8" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:59.173939 22645 consensus_queue.cc:1048] T 8c11074579b046d99e585c2818e18a2c P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.019s
I20260709 13:21:59.185111 22833 leader_election.cc:290] T b6f07842d3aa47858df5665874c86f3a P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:59.209271 16191 leader_election.cc:304] T 66f75124215f4ca6bef831e174a792d8 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
I20260709 13:21:59.184860 22622 leader_election.cc:290] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 25 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
W20260709 13:21:59.203279 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 96816.8us
I20260709 13:21:59.209867 16297 raft_consensus.cc:1275] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 13:21:59.130584 22938 raft_consensus.cc:515] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:59.206297 22738 raft_consensus.cc:697] T 57a744f5aaf944678a63469cbf514015 P cb82cc7bd6db46d980c75237a1402860 [term 15 LEADER]: Becoming Leader. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:21:59.163199 22508 raft_consensus.cc:515] T d2ab0ac8347641e6955e9bce7d90c1a2 P cb82cc7bd6db46d980c75237a1402860 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:21:59.223680 16263 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:59.207574 22348 raft_consensus.cc:493] T 8a878999c0024d2abe517484180290fc P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:59.227587 22622 raft_consensus.cc:493] T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:59.228525 22885 leader_election.cc:290] T 434bff3082d141838068f323bdb0d1ed P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:59.228819 16190 leader_election.cc:304] T d2ab0ac8347641e6955e9bce7d90c1a2 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860
W20260709 13:21:59.230278 22922 log.cc:927] Time spent T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 0.471s	user 0.002s	sys 0.000s
W20260709 13:21:59.273154 22949 log.cc:927] Time spent T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 13:21:59.230890 22943 raft_consensus.cc:493] T b87147731a68400e8cce9b879daa7c5d P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
W20260709 13:21:59.232218 22950 log.cc:927] Time spent T 18ad7e3af5504571a69d10c00750a01c P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.050s	user 0.000s	sys 0.000s
I20260709 13:21:59.233435 22917 raft_consensus.cc:2749] T 66f75124215f4ca6bef831e174a792d8 P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:21:59.181957 22954 raft_consensus.cc:515] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:59.235664 22508 leader_election.cc:290] T d2ab0ac8347641e6955e9bce7d90c1a2 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:59.235927 22837 raft_consensus.cc:3055] T 3884f864a96d428785895e5463e5728f P 647a343a542d4a139fe106c19095129b [term 13 LEADER]: Stepping down as leader of term 13
W20260709 13:21:59.237629 22924 log.cc:927] Time spent T 84f2da98864b4425bf7c0055951fdd4b P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 0.418s	user 0.001s	sys 0.001s
W20260709 13:21:59.244006 22782 consensus_meta.cc:287] T e5ad4eb281564936a59f41e387a47156 P 90f3da57796b488c9087a2f523adf045: Time spent flushing consensus metadata: real 0.810s	user 0.011s	sys 0.001s
I20260709 13:21:59.286648 22787 raft_consensus.cc:493] T d2ab0ac8347641e6955e9bce7d90c1a2 P 90f3da57796b488c9087a2f523adf045 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:21:59.316599 22787 raft_consensus.cc:515] T d2ab0ac8347641e6955e9bce7d90c1a2 P 90f3da57796b488c9087a2f523adf045 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:59.250945 16816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26354361e9b044cea1c2b86c4882fe92" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:59.274341 22943 raft_consensus.cc:515] T b87147731a68400e8cce9b879daa7c5d P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:59.321687 22943 leader_election.cc:290] T b87147731a68400e8cce9b879daa7c5d P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
W20260709 13:21:59.272495 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 43760.6us
I20260709 13:21:59.322751 16190 leader_election.cc:304] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 647a343a542d4a139fe106c19095129b
I20260709 13:21:59.324045 16190 leader_election.cc:304] T c0eab25ce4c64e018a041beeda2867db P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 647a343a542d4a139fe106c19095129b
I20260709 13:21:59.323875 22738 consensus_queue.cc:237] T 57a744f5aaf944678a63469cbf514015 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:59.324815 22954 leader_election.cc:290] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:59.325313 16190 leader_election.cc:304] T 1df93779209748b9860f6d893dd3de53 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:21:59.284890 22938 leader_election.cc:290] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:59.285957 22537 leader_election.cc:290] T 08c74490c40243a88e62d1760b5cd814 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:59.286633 22754 raft_consensus.cc:2749] T d2ab0ac8347641e6955e9bce7d90c1a2 P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:21:59.327970 22754 raft_consensus.cc:2749] T 1df93779209748b9860f6d893dd3de53 P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:21:59.277316 22348 raft_consensus.cc:515] T 8a878999c0024d2abe517484180290fc P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:59.330675 22348 leader_election.cc:290] T 8a878999c0024d2abe517484180290fc P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:59.291504 16825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434bff3082d141838068f323bdb0d1ed" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:59.328958 16190 leader_election.cc:304] T 84f2da98864b4425bf7c0055951fdd4b P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:21:59.332376 16825 raft_consensus.cc:2468] T 434bff3082d141838068f323bdb0d1ed P cb82cc7bd6db46d980c75237a1402860 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 17.
I20260709 13:21:59.216046 22914 raft_consensus.cc:493] T a378d7f796f94615b3b347119a9880fd P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:21:59.333631 22914 raft_consensus.cc:515] T a378d7f796f94615b3b347119a9880fd P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:59.334178 16190 leader_election.cc:304] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860
I20260709 13:21:59.319024 22787 leader_election.cc:290] T d2ab0ac8347641e6955e9bce7d90c1a2 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:59.336426 22914 leader_election.cc:290] T a378d7f796f94615b3b347119a9880fd P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:59.311014 16822 raft_consensus.cc:2468] T 3884f864a96d428785895e5463e5728f P cb82cc7bd6db46d980c75237a1402860 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 14.
I20260709 13:21:59.210788 22857 consensus_queue.cc:1048] T c0eab25ce4c64e018a041beeda2867db P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.068s
I20260709 13:21:59.286661 22622 raft_consensus.cc:515] T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:59.340728 16818 raft_consensus.cc:1240] T 3884f864a96d428785895e5463e5728f P cb82cc7bd6db46d980c75237a1402860 [term 14 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 13. Current term is 14. Ops: []
I20260709 13:21:59.251884 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6f07842d3aa47858df5665874c86f3a" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:59.341583 22622 leader_election.cc:290] T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:59.342736 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434bff3082d141838068f323bdb0d1ed" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:59.343561 16297 raft_consensus.cc:2468] T 434bff3082d141838068f323bdb0d1ed P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 17.
I20260709 13:21:59.344904 16297 raft_consensus.cc:1240] T 1df93779209748b9860f6d893dd3de53 P 90f3da57796b488c9087a2f523adf045 [term 17 LEADER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 16. Current term is 17. Ops: []
I20260709 13:21:59.345589 16297 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48674 (request call id 3337) took 1073 ms. Trace:
I20260709 13:21:59.345956 16297 rpcz_store.cc:276] 0709 13:21:58.272471 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:59.344316 (+1071845us) service_pool.cc:224] Handling call
0709 13:21:59.344686 (+   370us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:21:59.345303 (+   617us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:59.345522 (+   219us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:21:59.347368 22754 raft_consensus.cc:2749] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:21:59.347612 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 138423us
I20260709 13:21:59.325428 22954 raft_consensus.cc:2804] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:21:59.348322 22954 raft_consensus.cc:493] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Starting leader election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:21:59.348264 22754 consensus_queue.cc:1059] T 02838c86531d4f98bf58d1c22b8baf77 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:21:59.348901 22954 raft_consensus.cc:3060] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:21:59.349098 22754 consensus_queue.cc:1243] T 02838c86531d4f98bf58d1c22b8baf77 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Peer 90f3da57796b488c9087a2f523adf045 log is divergent from this leader: its last log entry 19.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 13:21:59.198284 22196 consensus_queue.cc:260] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.12, Last appended by leader: 12, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:59.326349 22946 raft_consensus.cc:2804] T c0eab25ce4c64e018a041beeda2867db P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:21:59.349797 22946 raft_consensus.cc:493] T c0eab25ce4c64e018a041beeda2867db P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Starting leader election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:21:59.349897 22196 raft_consensus.cc:3060] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:21:59.350286 22946 raft_consensus.cc:3060] T c0eab25ce4c64e018a041beeda2867db P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:21:59.339552 22938 raft_consensus.cc:2749] T 84f2da98864b4425bf7c0055951fdd4b P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:21:59.319541 16816 raft_consensus.cc:2468] T 26354361e9b044cea1c2b86c4882fe92 P cb82cc7bd6db46d980c75237a1402860 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 15.
I20260709 13:21:59.314626 16218 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57604 (request call id 3276) took 1173 ms. Trace:
I20260709 13:21:59.357713 16299 raft_consensus.cc:2468] T e5ad4eb281564936a59f41e387a47156 P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 18.
I20260709 13:21:59.359474 16297 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40474 (request call id 1225) took 1086 ms. Trace:
I20260709 13:21:59.360190 16297 rpcz_store.cc:276] 0709 13:21:58.273423 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:59.346911 (+1073488us) service_pool.cc:224] Handling call
0709 13:21:59.358742 (+ 11831us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:21:59.358982 (+   240us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:21:59.359024 (+    42us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:21:59.359097 (+    73us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:21:59.359147 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:59.359247 (+   100us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:21:59.359400 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:21:59.365355 16294 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48684 (request call id 3074) took 1296 ms. Trace:
I20260709 13:21:59.365614 16298 raft_consensus.cc:1275] T 84f2da98864b4425bf7c0055951fdd4b P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:21:59.365770 16294 rpcz_store.cc:276] 0709 13:21:58.069116 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:58.069251 (+   135us) service_pool.cc:224] Handling call
0709 13:21:58.069597 (+   346us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:21:58.373139 (+303542us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:59.365020 (+991881us) spinlock_profiling.cc:243] Waited 303 ms on lock 0x7b540024de54. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:21:59.365295 (+   275us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":695881472}
I20260709 13:21:59.353719 16218 rpcz_store.cc:276] 0709 13:21:58.141551 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:58.309952 (+168401us) service_pool.cc:224] Handling call
0709 13:21:58.353053 (+ 43101us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:21:58.390601 (+ 37548us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:21:58.390637 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:21:58.433591 (+ 42954us) log.cc:844] Serialized 26 byte log entry
0709 13:21:58.622838 (+189247us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:21:58.622876 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:58.622941 (+    65us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:21:59.229516 (+606575us) spinlock_profiling.cc:243] Waited 2.54 ms on lock 0x7b4400022b4c. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69ad8fadb1 00007f69ad998c85 00007f69ad9996a2 00007f69b070b1db 00007f69b0706aaf 00007f69b06fa86e 00007f69b06f9bb6 00007f69b06f97d8 00007f69b0703221 00007f69b07031d9 00007f69b0703169
0709 13:21:59.274496 (+ 44980us) raft_consensus.cc:1685] finished
0709 13:21:59.274517 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:21:59.314507 (+ 39990us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":14237,"raft.run_cpu_time_us":123,"raft.run_wall_time_us":31192,"spinlock_wait_cycles":5851904,"thread_start_us":149406,"threads_started":1,"wal-append.queue_time_us":246230}
I20260709 13:21:59.374101 16294 raft_consensus.cc:1275] T 5a959dbecc2648eb85d1cc550f5a3245 P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 13:21:59.378405 16294 raft_consensus.cc:1275] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 13:21:59.315373 22837 raft_consensus.cc:740] T 3884f864a96d428785895e5463e5728f P 647a343a542d4a139fe106c19095129b [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:21:59.346449 22916 consensus_queue.cc:260] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.13, Last appended by leader: 13, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:59.390357 16225 raft_consensus.cc:1275] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
W20260709 13:21:59.394378 16264 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:59.398764 16218 raft_consensus.cc:1240] T 08c74490c40243a88e62d1760b5cd814 P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 16. Current term is 17. Ops: []
I20260709 13:21:59.382696 22610 raft_consensus.cc:493] T ee9ed67006144797b1de6ea8c2013a68 P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:21:59.368721 22676 raft_consensus.cc:697] T c9944d5e4b8c4109af9955c4f08c9095 P cb82cc7bd6db46d980c75237a1402860 [term 21 LEADER]: Becoming Leader. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:21:59.409147 16816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada5a004ba7c46819051e3ccec71ffdb" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:59.435667 16142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8158bbd17b994d2eb96f560bd0f1b88d" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 18 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:59.436515 16822 raft_consensus.cc:1240] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [term 25 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 24. Current term is 25. Ops: []
I20260709 13:21:59.439522 22958 raft_consensus.cc:493] T b87147731a68400e8cce9b879daa7c5d P cb82cc7bd6db46d980c75237a1402860 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:59.440375 22958 raft_consensus.cc:515] T b87147731a68400e8cce9b879daa7c5d P cb82cc7bd6db46d980c75237a1402860 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:59.444293 22958 leader_election.cc:290] T b87147731a68400e8cce9b879daa7c5d P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:21:59.451057 16822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdbd78acb4cf46298e0de8ba0d130e19" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:21:59.463995 16146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a878999c0024d2abe517484180290fc" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
W20260709 13:21:59.481702 16191 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:59.486778 16150 raft_consensus.cc:1240] T 02838c86531d4f98bf58d1c22b8baf77 P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 18. Current term is 19. Ops: [18.11-18.11]
I20260709 13:21:59.479417 16294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e4bfb9090c8460eb14fa7e0d818e7f1" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:21:59.452903 22833 raft_consensus.cc:493] T 9e65676b21b748728ad39668cfe722ec P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:21:59.467926 22439 raft_consensus.cc:493] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:59.495471 16294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d4ab870a05347868956aa8b841b2981" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:59.504528 16294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87ac0d1f7e8a419f9bcbc04aee0e067e" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:59.506060 16294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9944d5e4b8c4109af9955c4f08c9095" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:59.506860 16294 raft_consensus.cc:2368] T c9944d5e4b8c4109af9955c4f08c9095 P 90f3da57796b488c9087a2f523adf045 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac for earlier term 20. Current term is 21.
I20260709 13:21:59.507990 16294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e4bfb9090c8460eb14fa7e0d818e7f1" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:59.509599 16294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5ad4eb281564936a59f41e387a47156" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:59.513046 22709 consensus_queue.cc:1048] T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:21:59.499401 22833 raft_consensus.cc:515] T 9e65676b21b748728ad39668cfe722ec P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:59.517441 16221 raft_consensus.cc:3055] T 02838c86531d4f98bf58d1c22b8baf77 P 1ca2725d290e47d68b1a4b609c9816ac [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:21:59.517850 16221 raft_consensus.cc:740] T 02838c86531d4f98bf58d1c22b8baf77 P 1ca2725d290e47d68b1a4b609c9816ac [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:21:59.518519 22833 leader_election.cc:290] T 9e65676b21b748728ad39668cfe722ec P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:59.520673 16299 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48674 (request call id 3332) took 1328 ms. Trace:
I20260709 13:21:59.521111 16299 rpcz_store.cc:276] 0709 13:21:58.192637 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:58.430622 (+237985us) service_pool.cc:224] Handling call
0709 13:21:59.520479 (+1089857us) spinlock_profiling.cc:243] Waited 871 ms on lock 0x7b54001ef354. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:21:59.520588 (+   109us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2004013440}
I20260709 13:21:59.522373 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c2095a5f4ad49408dbc7f69c606120d" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:21:59.523928 22914 raft_consensus.cc:493] T 5a959dbecc2648eb85d1cc550f5a3245 P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:59.523736 22834 consensus_queue.cc:260] T 5a96373bb0e14dce8a266b2fc6c58507 P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:59.524483 22914 raft_consensus.cc:515] T 5a959dbecc2648eb85d1cc550f5a3245 P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:59.525104 22834 raft_consensus.cc:3060] T 5a96373bb0e14dce8a266b2fc6c58507 P 647a343a542d4a139fe106c19095129b [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:21:59.525169 22866 consensus_queue.cc:1048] T 8638ee363273426db1393dd81e5c7b72 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:21:59.471385 22610 raft_consensus.cc:515] T ee9ed67006144797b1de6ea8c2013a68 P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:59.529345 16296 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48684 (request call id 3085) took 1399 ms. Trace:
I20260709 13:21:59.528457 22676 consensus_queue.cc:237] T c9944d5e4b8c4109af9955c4f08c9095 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:59.529489 22610 leader_election.cc:290] T ee9ed67006144797b1de6ea8c2013a68 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:59.526746 22914 leader_election.cc:290] T 5a959dbecc2648eb85d1cc550f5a3245 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:21:59.536484 16294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a959dbecc2648eb85d1cc550f5a3245" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:59.518678 16221 consensus_queue.cc:260] T 02838c86531d4f98bf58d1c22b8baf77 P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.11, Last appended by leader: 11, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:59.536113 16296 rpcz_store.cc:276] 0709 13:21:58.129883 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:58.484515 (+354632us) service_pool.cc:224] Handling call
0709 13:21:58.493187 (+  8672us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:21:58.690849 (+197662us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:59.529079 (+838230us) spinlock_profiling.cc:243] Waited 194 ms on lock 0x7b540030c0d4. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:21:59.529269 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":445112064}
I20260709 13:21:59.547120 16221 raft_consensus.cc:3060] T 02838c86531d4f98bf58d1c22b8baf77 P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:21:59.538182 16790 leader_election.cc:304] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:21:59.544673 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a878999c0024d2abe517484180290fc" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:59.548504 16299 raft_consensus.cc:2468] T 8a878999c0024d2abe517484180290fc P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 16.
I20260709 13:21:59.552357 16294 raft_consensus.cc:2368] T 5a959dbecc2648eb85d1cc550f5a3245 P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac for earlier term 16. Current term is 17.
I20260709 13:21:59.554229 16145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3884f864a96d428785895e5463e5728f" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:59.558947 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a96373bb0e14dce8a266b2fc6c58507" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:59.560009 16299 raft_consensus.cc:2393] T 5a96373bb0e14dce8a266b2fc6c58507 P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 15: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
I20260709 13:21:59.560200 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ab0ac8347641e6955e9bce7d90c1a2" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:59.560799 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8638ee363273426db1393dd81e5c7b72" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:59.560886 16226 raft_consensus.cc:2468] T d2ab0ac8347641e6955e9bce7d90c1a2 P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 15.
I20260709 13:21:59.558947 16223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62fe3346cd3c42c4a55e14af9073bca4" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:21:59.563611 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada5a004ba7c46819051e3ccec71ffdb" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:59.564945 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6f07842d3aa47858df5665874c86f3a" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:59.566152 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68a6838d82ef4cac83507f906139d90f" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
W20260709 13:21:59.566999 16192 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 162989us
I20260709 13:21:59.567878 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:59.568452 16299 raft_consensus.cc:2468] T ecc1e363230043fe9fef390f0b979f20 P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 19.
I20260709 13:21:59.569455 16143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62fe3346cd3c42c4a55e14af9073bca4" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:21:59.571403 16292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87147731a68400e8cce9b879daa7c5d" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:21:59.598623 16292 raft_consensus.cc:3060] T b87147731a68400e8cce9b879daa7c5d P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:21:59.596577 16114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:59.571554 16294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdbd78acb4cf46298e0de8ba0d130e19" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:59.580293 16223 raft_consensus.cc:1275] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 13:21:59.600314 16223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a378d7f796f94615b3b347119a9880fd" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:59.600934 16223 raft_consensus.cc:2468] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 20.
I20260709 13:21:59.587956 16226 raft_consensus.cc:1275] T b6f07842d3aa47858df5665874c86f3a P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 13:21:59.602718 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f5f89186204242bf377f7c42b2e2da" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:59.504355 22439 raft_consensus.cc:515] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:59.606665 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18ad7e3af5504571a69d10c00750a01c" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 17 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:59.596547 16296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c74490c40243a88e62d1760b5cd814" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:21:59.608958 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434bff3082d141838068f323bdb0d1ed" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:59.609946 16149 raft_consensus.cc:2410] T 434bff3082d141838068f323bdb0d1ed P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 for term 18 because replica has last-logged OpId of term: 16 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 13.
I20260709 13:21:59.609831 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fffab26427e24d398e2695aad40ca134" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:59.611009 16297 raft_consensus.cc:2410] T fffab26427e24d398e2695aad40ca134 P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac for term 18 because replica has last-logged OpId of term: 17 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 10.
I20260709 13:21:59.558467 22622 raft_consensus.cc:2804] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 [term 21 FOLLOWER]: Leader pre-election won for term 22
W20260709 13:21:59.618479 16266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 84207.6us
I20260709 13:21:59.617338 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c11074579b046d99e585c2818e18a2c" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 19 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:59.618409 16143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87147731a68400e8cce9b879daa7c5d" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:59.618639 22955 raft_consensus.cc:493] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P cb82cc7bd6db46d980c75237a1402860 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:21:59.498008 22837 consensus_queue.cc:260] T 3884f864a96d428785895e5463e5728f P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 13.13, Last appended by leader: 13, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:59.622066 22914 consensus_queue.cc:1048] T 8158bbd17b994d2eb96f560bd0f1b88d P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:21:59.622710 16219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ab0ac8347641e6955e9bce7d90c1a2" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:59.647761 16294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8158bbd17b994d2eb96f560bd0f1b88d" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 18 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:59.629717 22935 raft_consensus.cc:515] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 1ca2725d290e47d68b1a4b609c9816ac [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:21:59.636739 16143 raft_consensus.cc:2468] T b87147731a68400e8cce9b879daa7c5d P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 19.
I20260709 13:21:59.643312 16221 raft_consensus.cc:1275] T 02838c86531d4f98bf58d1c22b8baf77 P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 19 index: 11. (term mismatch)
W20260709 13:21:59.644827 16117 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:59.642740 16226 raft_consensus.cc:1275] T 68a6838d82ef4cac83507f906139d90f P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 17 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 13:21:59.644938 16222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a378d7f796f94615b3b347119a9880fd" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:59.696223 16222 raft_consensus.cc:2393] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 20: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
I20260709 13:21:59.622874 22943 consensus_queue.cc:1048] T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:21:59.649362 16219 raft_consensus.cc:2393] T d2ab0ac8347641e6955e9bce7d90c1a2 P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 15: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
I20260709 13:21:59.597640 22917 raft_consensus.cc:493] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:21:59.698781 16149 raft_consensus.cc:1240] T d16dfad0c3b540e5b4b86492ddf87208 P 647a343a542d4a139fe106c19095129b [term 20 LEADER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 19. Current term is 20. Ops: []
I20260709 13:21:59.699007 16226 raft_consensus.cc:1240] T d16dfad0c3b540e5b4b86492ddf87208 P 1ca2725d290e47d68b1a4b609c9816ac [term 20 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 19. Current term is 20. Ops: []
W20260709 13:21:59.700181 16189 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:59.646245 22622 raft_consensus.cc:493] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:21:59.674983 16266 leader_election.cc:400] T 57a744f5aaf944678a63469cbf514015 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 14 election: Vote denied by peer cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923) with higher term. Message: Invalid argument: T 57a744f5aaf944678a63469cbf514015 P cb82cc7bd6db46d980c75237a1402860 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 for earlier term 14. Current term is 15.
I20260709 13:21:59.695466 16221 pending_rounds.cc:85] T 02838c86531d4f98bf58d1c22b8baf77 P 1ca2725d290e47d68b1a4b609c9816ac: Aborting all ops after (but not including) 10
I20260709 13:21:59.709447 16266 leader_election.cc:403] T 57a744f5aaf944678a63469cbf514015 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 14 election: Cancelling election due to peer responding with higher term
I20260709 13:21:59.709537 16221 pending_rounds.cc:107] T 02838c86531d4f98bf58d1c22b8baf77 P 1ca2725d290e47d68b1a4b609c9816ac: Aborting uncommitted NO_OP operation due to leader change: 18.11
I20260709 13:21:59.709910 16221 raft_consensus.cc:2889] T 02838c86531d4f98bf58d1c22b8baf77 P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:21:59.710619 16221 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57600 (request call id 1962) took 1123 ms. Trace:
I20260709 13:21:59.696599 16224 raft_consensus.cc:1240] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 1ca2725d290e47d68b1a4b609c9816ac [term 20 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 19. Current term is 20. Ops: [19.13-19.13]
I20260709 13:21:59.688182 16299 raft_consensus.cc:1240] T 8cc959fb7f704fba9009c0595467d074 P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 16. Current term is 17. Ops: [16.12-16.12]
I20260709 13:21:59.702136 16222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a959dbecc2648eb85d1cc550f5a3245" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:21:59.712133 16222 raft_consensus.cc:2468] T 5a959dbecc2648eb85d1cc550f5a3245 P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 17.
W20260709 13:21:59.717757 16291 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48674 (request call id 3322) took 1572 ms (client timeout 1841 ms). Trace:
W20260709 13:21:59.718364 16291 rpcz_store.cc:269] 0709 13:21:58.145765 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:58.200212 (+ 54447us) service_pool.cc:224] Handling call
0709 13:21:59.717536 (+1517324us) spinlock_profiling.cc:243] Waited 782 ms on lock 0x7b5400399ed4. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:21:59.717689 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1799135488}
I20260709 13:21:59.711009 16221 rpcz_store.cc:276] 0709 13:21:58.587466 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:58.789728 (+202262us) service_pool.cc:224] Handling call
0709 13:21:58.790068 (+   340us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:21:59.517345 (+727277us) spinlock_profiling.cc:243] Waited 56.8 ms on lock 0x7b540038fa20. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0ad6dc0 00007f69b0b611c0 00007f69b0b62231 00007f69b0b5ed8f 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c
0709 13:21:59.710356 (+193011us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:59.710562 (+   206us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":130528512}
I20260709 13:21:59.736349 16222 raft_consensus.cc:1275] T 18ad7e3af5504571a69d10c00750a01c P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
W20260709 13:21:59.737212 16192 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 63268.6us
I20260709 13:21:59.697397 22837 raft_consensus.cc:3060] T 3884f864a96d428785895e5463e5728f P 647a343a542d4a139fe106c19095129b [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:21:59.740317 16143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87147731a68400e8cce9b879daa7c5d" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:59.743762 16790 leader_election.cc:304] T d2ab0ac8347641e6955e9bce7d90c1a2 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:21:59.760574 16790 leader_election.cc:304] T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 
W20260709 13:21:59.762101 16266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 125857us
I20260709 13:21:59.764187 16266 leader_election.cc:304] T 66ae176961b74813a58bc15c9c4c6f2a P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860
I20260709 13:21:59.765120 16146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a959dbecc2648eb85d1cc550f5a3245" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:21:59.765452 22782 raft_consensus.cc:2749] T 66ae176961b74813a58bc15c9c4c6f2a P 90f3da57796b488c9087a2f523adf045 [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:21:59.767284 16266 leader_election.cc:400] T 57a744f5aaf944678a63469cbf514015 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 14 pre-election: Vote denied by peer cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923) with higher term. Message: Invalid argument: T 57a744f5aaf944678a63469cbf514015 P cb82cc7bd6db46d980c75237a1402860 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 for earlier term 14. Current term is 15.
I20260709 13:21:59.767328 22982 raft_consensus.cc:493] T 19baa7bc367844c288bca8f81439c413 P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:21:59.736020 22731 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 19 current term: 19 is higher.
I20260709 13:21:59.768136 22982 raft_consensus.cc:515] T 19baa7bc367844c288bca8f81439c413 P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:59.768991 16142 raft_consensus.cc:1275] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 13:21:59.770614 16266 leader_election.cc:304] T 3884f864a96d428785895e5463e5728f P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 647a343a542d4a139fe106c19095129b
I20260709 13:21:59.774787 16143 raft_consensus.cc:2393] T b87147731a68400e8cce9b879daa7c5d P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 19: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
I20260709 13:21:59.710232 22622 raft_consensus.cc:3060] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:21:59.816479 16192 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 64369.6us
I20260709 13:21:59.818838 16147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
W20260709 13:21:59.816181 16266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 45966.6us
I20260709 13:21:59.836632 22864 raft_consensus.cc:493] T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:21:59.837544 22864 raft_consensus.cc:515] T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:59.841133 22864 leader_election.cc:290] T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:21:59.843292 22864 raft_consensus.cc:493] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:21:59.844098 22864 raft_consensus.cc:515] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:21:59.851985 22864 leader_election.cc:290] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
W20260709 13:21:59.853255 16192 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:59.881688 22439 leader_election.cc:290] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
W20260709 13:21:59.931342 16264 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:59.933573 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:21:59.934484 16190 leader_election.cc:304] T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:21:59.935277 22959 raft_consensus.cc:2804] T d2ab0ac8347641e6955e9bce7d90c1a2 P cb82cc7bd6db46d980c75237a1402860 [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 13:22:00.026831 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 92423.1us
I20260709 13:22:00.028259 16190 leader_election.cc:400] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 20 pre-election: Vote denied by peer 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389) with higher term. Message: Invalid argument: T c9944d5e4b8c4109af9955c4f08c9095 P 90f3da57796b488c9087a2f523adf045 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac for earlier term 20. Current term is 21.
I20260709 13:22:00.028563 16190 leader_election.cc:403] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 20 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:21:59.962183 22738 raft_consensus.cc:2804] T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:22:00.029058 22738 raft_consensus.cc:493] T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:00.029440 22738 raft_consensus.cc:3060] T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860 [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:22:00.031332 16117 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 49519.4us
I20260709 13:22:00.032012 16117 leader_election.cc:304] T 434bff3082d141838068f323bdb0d1ed P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:22:00.032826 16117 leader_election.cc:304] T 26354361e9b044cea1c2b86c4882fe92 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860; no voters: 90f3da57796b488c9087a2f523adf045
I20260709 13:22:00.036571 22977 raft_consensus.cc:2804] T 434bff3082d141838068f323bdb0d1ed P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:22:00.037016 22977 raft_consensus.cc:493] T 434bff3082d141838068f323bdb0d1ed P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:00.037365 22977 raft_consensus.cc:3060] T 434bff3082d141838068f323bdb0d1ed P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:22:00.039376 22854 raft_consensus.cc:2804] T 26354361e9b044cea1c2b86c4882fe92 P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:22:00.039839 22854 raft_consensus.cc:493] T 26354361e9b044cea1c2b86c4882fe92 P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:00.040201 22854 raft_consensus.cc:3060] T 26354361e9b044cea1c2b86c4882fe92 P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:21:59.972141 22982 leader_election.cc:290] T 19baa7bc367844c288bca8f81439c413 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
W20260709 13:21:59.991324 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 170049us
W20260709 13:21:59.995970 16114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:59.754765 22873 raft_consensus.cc:2749] T 57a744f5aaf944678a63469cbf514015 P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Leader election lost for term 14. Reason: Vote denied by peer cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923) with higher term. Message: Invalid argument: T 57a744f5aaf944678a63469cbf514015 P cb82cc7bd6db46d980c75237a1402860 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 for earlier term 14. Current term is 15.
W20260709 13:22:00.009372 16115 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:59.649528 22955 raft_consensus.cc:515] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P cb82cc7bd6db46d980c75237a1402860 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:00.022199 16299 raft_consensus.cc:2468] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 21.
I20260709 13:22:00.046322 22967 raft_consensus.cc:493] T 829b4ad8c5a54eababeea1181b960d4b P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:22:00.046890 22967 raft_consensus.cc:515] T 829b4ad8c5a54eababeea1181b960d4b P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:00.026171 16265 leader_election.cc:304] T 5a959dbecc2648eb85d1cc550f5a3245 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:22:00.026307 22959 raft_consensus.cc:493] T d2ab0ac8347641e6955e9bce7d90c1a2 P cb82cc7bd6db46d980c75237a1402860 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:59.989609 22782 raft_consensus.cc:2804] T 3884f864a96d428785895e5463e5728f P 90f3da57796b488c9087a2f523adf045 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:21:59.698796 22917 raft_consensus.cc:515] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:22:00.021222 22908 consensus_meta.cc:287] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 1ca2725d290e47d68b1a4b609c9816ac: Time spent flushing consensus metadata: real 0.810s	user 0.011s	sys 0.000s
W20260709 13:22:00.063206 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 302672us
I20260709 13:22:00.062830 22959 raft_consensus.cc:3060] T d2ab0ac8347641e6955e9bce7d90c1a2 P cb82cc7bd6db46d980c75237a1402860 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:22:00.081005 22972 log.cc:927] Time spent T 57a744f5aaf944678a63469cbf514015 P cb82cc7bd6db46d980c75237a1402860: Append to log took a long time: real 0.372s	user 0.001s	sys 0.000s
I20260709 13:22:00.061518 22517 raft_consensus.cc:3060] T 1df93779209748b9860f6d893dd3de53 P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:22:00.168773 16189 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:22:00.217805 16151 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.514s	user 0.016s	sys 0.037s
I20260709 13:22:00.079551 22953 consensus_queue.cc:1059] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: INVALID_TERM, Last received: 24.15, Next index: 16, Last known committed idx: 14, Time since last communication: 0.307s
W20260709 13:22:00.327387 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 266068us
I20260709 13:22:00.053491 22943 consensus_queue.cc:1048] T 68a6838d82ef4cac83507f906139d90f P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.046s
I20260709 13:22:00.057135 22439 consensus_queue.cc:1059] T d16dfad0c3b540e5b4b86492ddf87208 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.085s
W20260709 13:22:00.333829 16151 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.630s	user 0.016s	sys 0.037s
I20260709 13:22:00.334374 16147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a9de347c711477e87c582d003f36925" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
W20260709 13:22:00.336045 16142 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.519s	user 0.004s	sys 0.011s
W20260709 13:22:00.332258 16297 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.561s	user 0.004s	sys 0.015s
I20260709 13:22:00.359719 22873 consensus_queue.cc:1048] T 18ad7e3af5504571a69d10c00750a01c P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.000s
W20260709 13:22:00.366073 16297 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.595s	user 0.004s	sys 0.015s
I20260709 13:22:00.364883 16147 raft_consensus.cc:2468] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 21.
W20260709 13:22:00.361275 22978 log.cc:927] Time spent T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 0.340s	user 0.003s	sys 0.010s
I20260709 13:22:00.383952 16147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ad5dbfa99ee4b8d9e28be92b3f36db5" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:22:00.348593 22864 raft_consensus.cc:2804] T 5a959dbecc2648eb85d1cc550f5a3245 P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:22:00.358166 22917 leader_election.cc:290] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
W20260709 13:22:00.405179 16264 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:22:00.374243 22874 raft_consensus.cc:493] T c0eab25ce4c64e018a041beeda2867db P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:00.360812 22914 consensus_queue.cc:1048] T 02838c86531d4f98bf58d1c22b8baf77 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.050s
I20260709 13:22:00.393893 22864 raft_consensus.cc:493] T 5a959dbecc2648eb85d1cc550f5a3245 P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:00.412883 22935 leader_election.cc:290] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 20 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
W20260709 13:22:00.456077 16300 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48674 (request call id 3326) took 2304 ms (client timeout 2166 ms). Trace:
I20260709 13:22:00.454209 16144 raft_consensus.cc:1275] T 18ad7e3af5504571a69d10c00750a01c P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
W20260709 13:22:00.456584 16300 rpcz_store.cc:269] 0709 13:21:58.151577 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:58.456614 (+305037us) service_pool.cc:224] Handling call
0709 13:22:00.455881 (+1999267us) spinlock_profiling.cc:243] Waited 33.3 ms on lock 0x7b54000cdd54. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:22:00.456007 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":76628864}
I20260709 13:22:00.464485 16222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ad5dbfa99ee4b8d9e28be92b3f36db5" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:00.461071 16822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:00.465284 16822 raft_consensus.cc:2468] T 19baa7bc367844c288bca8f81439c413 P cb82cc7bd6db46d980c75237a1402860 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 21.
W20260709 13:22:00.458550 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 127493us
I20260709 13:22:00.461457 16816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74ca8140f35c4ae29e8a664f6bdfc829" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:00.465422 16222 raft_consensus.cc:2410] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 for term 19 because replica has last-logged OpId of term: 17 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 10.
I20260709 13:22:00.469434 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74ca8140f35c4ae29e8a664f6bdfc829" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:00.443302 22864 raft_consensus.cc:3060] T 5a959dbecc2648eb85d1cc550f5a3245 P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:22:00.467700 16790 leader_election.cc:304] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:22:00.470636 16217 raft_consensus.cc:2468] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 19.
W20260709 13:22:00.485361 16292 consensus_meta.cc:287] T b87147731a68400e8cce9b879daa7c5d P 90f3da57796b488c9087a2f523adf045: Time spent flushing consensus metadata: real 0.883s	user 0.009s	sys 0.005s
I20260709 13:22:00.485778 16292 raft_consensus.cc:2468] T b87147731a68400e8cce9b879daa7c5d P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 19.
I20260709 13:22:00.486935 16790 leader_election.cc:304] T b87147731a68400e8cce9b879daa7c5d P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:22:00.451881 22955 leader_election.cc:290] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:00.490485 22955 raft_consensus.cc:2804] T b87147731a68400e8cce9b879daa7c5d P cb82cc7bd6db46d980c75237a1402860 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:22:00.490885 22955 raft_consensus.cc:493] T b87147731a68400e8cce9b879daa7c5d P cb82cc7bd6db46d980c75237a1402860 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:00.491267 22955 raft_consensus.cc:3060] T b87147731a68400e8cce9b879daa7c5d P cb82cc7bd6db46d980c75237a1402860 [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:22:00.491437 22980 log.cc:927] Time spent T c9944d5e4b8c4109af9955c4f08c9095 P cb82cc7bd6db46d980c75237a1402860: Append to log took a long time: real 0.367s	user 0.009s	sys 0.000s
I20260709 13:22:00.457110 22967 leader_election.cc:290] T 829b4ad8c5a54eababeea1181b960d4b P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
W20260709 13:22:00.513489 16292 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48674 (request call id 3320) took 2344 ms (client timeout 1849 ms). Trace:
I20260709 13:22:00.452157 22943 raft_consensus.cc:493] T 5a96373bb0e14dce8a266b2fc6c58507 P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:22:00.578608 16296 consensus_meta.cc:287] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045: Time spent flushing consensus metadata: real 0.967s	user 0.008s	sys 0.004s
I20260709 13:22:00.579005 16296 raft_consensus.cc:2468] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 17.
I20260709 13:22:00.579543 16296 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48684 (request call id 3117) took 1057 ms. Trace:
I20260709 13:22:00.580159 16296 rpcz_store.cc:276] 0709 13:21:59.522098 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:59.547343 (+ 25245us) service_pool.cc:224] Handling call
0709 13:22:00.579475 (+1032132us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:22:00.441205 22874 raft_consensus.cc:515] T c0eab25ce4c64e018a041beeda2867db P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:22:00.578034 16292 rpcz_store.cc:269] 0709 13:21:58.142187 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:59.545619 (+1403432us) service_pool.cc:224] Handling call
0709 13:22:00.486198 (+940579us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:22:00.588693 23005 raft_consensus.cc:493] T 2794af5b99984efebdee564035841cb0 P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
W20260709 13:22:00.608011 22989 log.cc:927] Time spent T 8c11074579b046d99e585c2818e18a2c P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 0.143s	user 0.000s	sys 0.001s
W20260709 13:22:00.614897 22968 log.cc:927] Time spent T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 0.689s	user 0.000s	sys 0.009s
I20260709 13:22:00.632771 16151 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40844 (request call id 2908) took 1428 ms. Trace:
I20260709 13:22:00.598204 22943 raft_consensus.cc:515] T 5a96373bb0e14dce8a266b2fc6c58507 P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:00.622599 22967 raft_consensus.cc:493] T 18ad7e3af5504571a69d10c00750a01c P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:00.634303 16151 rpcz_store.cc:276] 0709 13:21:59.204369 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:59.204500 (+   131us) service_pool.cc:224] Handling call
0709 13:21:59.214717 (+ 10217us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:21:59.369035 (+154318us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:21:59.369071 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:21:59.369608 (+   537us) log.cc:844] Serialized 26 byte log entry
0709 13:21:59.689717 (+320109us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:22:00.364827 (+675110us) log.cc:844] Serialized 14 byte log entry
0709 13:22:00.628281 (+263454us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:22:00.628852 (+   571us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:22:00.629077 (+   225us) raft_consensus.cc:1685] finished
0709 13:22:00.629098 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:22:00.632703 (+  3605us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":173908,"prepare.queue_time_us":52919,"prepare.run_cpu_time_us":3506,"prepare.run_wall_time_us":182832,"raft.queue_time_us":372668,"raft.run_cpu_time_us":287,"raft.run_wall_time_us":18371,"thread_start_us":1212429,"threads_started":3,"wal-append.queue_time_us":369929}
W20260709 13:22:00.626995 22984 log.cc:927] Time spent T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 0.335s	user 0.001s	sys 0.015s
I20260709 13:22:00.686188 16298 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48674 (request call id 3327) took 2534 ms. Trace:
I20260709 13:22:00.690503 16298 rpcz_store.cc:276] 0709 13:21:58.152190 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:58.569489 (+417299us) service_pool.cc:224] Handling call
0709 13:21:59.365255 (+795766us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:21:59.366015 (+   760us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:22:00.685846 (+1319831us) spinlock_profiling.cc:243] Waited 790 ms on lock 0x7b540024de50. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5e8ad 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:22:00.686089 (+   243us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1817725824}
I20260709 13:22:00.707582 22943 leader_election.cc:290] T 5a96373bb0e14dce8a266b2fc6c58507 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:22:00.713234 22996 raft_consensus.cc:493] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:22:00.715044 22992 raft_consensus.cc:2749] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
W20260709 13:22:00.676527 22985 log.cc:927] Time spent T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.315s	user 0.001s	sys 0.001s
I20260709 13:22:00.714012 22996 raft_consensus.cc:515] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:00.640633 23005 raft_consensus.cc:515] T 2794af5b99984efebdee564035841cb0 P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:00.734144 16148 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40844 (request call id 2909) took 1529 ms. Trace:
I20260709 13:22:00.780279 16223 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57600 (request call id 1974) took 1191 ms. Trace:
W20260709 13:22:00.630270 22946 consensus_meta.cc:287] T c0eab25ce4c64e018a041beeda2867db P 1ca2725d290e47d68b1a4b609c9816ac: Time spent flushing consensus metadata: real 1.279s	user 0.015s	sys 0.012s
I20260709 13:22:00.788654 16223 rpcz_store.cc:276] 0709 13:21:59.588963 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:59.611626 (+ 22663us) service_pool.cc:224] Handling call
0709 13:21:59.612121 (+   495us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:21:59.671937 (+ 59816us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:21:59.673490 (+  1553us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:21:59.694019 (+ 20529us) log.cc:844] Serialized 52 byte log entry
0709 13:21:59.872601 (+178582us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:21:59.872661 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:59.872764 (+   103us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:22:00.779936 (+907172us) raft_consensus.cc:1685] finished
0709 13:22:00.779963 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:22:00.780194 (+   231us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":41318,"raft.queue_time_us":63216,"raft.run_cpu_time_us":106,"raft.run_wall_time_us":3289,"thread_start_us":63285,"threads_started":1,"wal-append.queue_time_us":336475}
I20260709 13:22:00.788508 16148 rpcz_store.cc:276] 0709 13:21:59.204693 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:59.204826 (+   133us) service_pool.cc:224] Handling call
0709 13:21:59.205167 (+   341us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:21:59.351587 (+146420us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:21:59.351619 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:21:59.352085 (+   466us) log.cc:844] Serialized 26 byte log entry
0709 13:21:59.465514 (+113429us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:21:59.465558 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:59.465629 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:22:00.733906 (+1268277us) raft_consensus.cc:1685] finished
0709 13:22:00.733930 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:22:00.734077 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":165280,"raft.queue_time_us":147149,"raft.run_cpu_time_us":127,"raft.run_wall_time_us":9080,"spinlock_wait_cycles":2304,"thread_start_us":113030,"threads_started":1,"wal-append.queue_time_us":248019}
I20260709 13:22:00.705108 22699 raft_consensus.cc:1064] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b: attempting to promote NON_VOTER cb82cc7bd6db46d980c75237a1402860 to VOTER
I20260709 13:22:00.711105 23003 raft_consensus.cc:493] T 23f82224f4564fea93572e8206bad1fe P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
W20260709 13:22:00.805650 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 429347us
I20260709 13:22:00.807420 22946 raft_consensus.cc:515] T c0eab25ce4c64e018a041beeda2867db P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:00.700486 22735 raft_consensus.cc:3055] T 9c61a39fd4f84424b129c1a6f9ba404d P 647a343a542d4a139fe106c19095129b [term 19 LEADER]: Stepping down as leader of term 19
W20260709 13:22:00.764333 22999 log.cc:927] Time spent T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 0.107s	user 0.000s	sys 0.000s
I20260709 13:22:00.828799 16149 raft_consensus.cc:1275] T 1df93779209748b9860f6d893dd3de53 P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 13:22:00.851683 16225 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33050 (request call id 1176) took 1296 ms. Trace:
I20260709 13:22:00.832273 16265 leader_election.cc:304] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:22:00.680433 22967 raft_consensus.cc:515] T 18ad7e3af5504571a69d10c00750a01c P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } }
W20260709 13:22:00.852969 16115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 398184us
I20260709 13:22:00.780745 22874 leader_election.cc:290] T c0eab25ce4c64e018a041beeda2867db P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:22:00.852959 16225 rpcz_store.cc:276] 0709 13:21:59.554873 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:59.555007 (+   134us) service_pool.cc:224] Handling call
0709 13:21:59.555332 (+   325us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:21:59.580885 (+ 25553us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:21:59.580914 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:21:59.581476 (+   562us) log.cc:844] Serialized 26 byte log entry
0709 13:21:59.689194 (+107718us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:21:59.689239 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:59.689308 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:22:00.851366 (+1162058us) raft_consensus.cc:1685] finished
0709 13:22:00.851395 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:22:00.851594 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":72273,"raft.queue_time_us":42946,"raft.run_cpu_time_us":178,"raft.run_wall_time_us":34620,"thread_start_us":44003,"threads_started":1,"wal-append.queue_time_us":126872}
I20260709 13:22:00.831403 22659 raft_consensus.cc:493] T 8c11074579b046d99e585c2818e18a2c P cb82cc7bd6db46d980c75237a1402860 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:00.710788 22955 raft_consensus.cc:515] T b87147731a68400e8cce9b879daa7c5d P cb82cc7bd6db46d980c75237a1402860 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:22:00.829396 22699 raft_consensus.cc:1068] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b: Unable to promote non-voter cb82cc7bd6db46d980c75237a1402860: Illegal state: Leader has not yet committed an operation in its own term
W20260709 13:22:00.869242 16115 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:22:00.855027 23015 raft_consensus.cc:493] T 1df93779209748b9860f6d893dd3de53 P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:00.831511 22735 raft_consensus.cc:740] T 9c61a39fd4f84424b129c1a6f9ba404d P 647a343a542d4a139fe106c19095129b [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:22:00.846952 22943 consensus_queue.cc:1048] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.030s
I20260709 13:22:00.853551 22914 consensus_queue.cc:1059] T d16dfad0c3b540e5b4b86492ddf87208 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.037s
W20260709 13:22:00.890071 22834 consensus_meta.cc:287] T 5a96373bb0e14dce8a266b2fc6c58507 P 647a343a542d4a139fe106c19095129b: Time spent flushing consensus metadata: real 1.364s	user 0.015s	sys 0.008s
I20260709 13:22:00.880215 22874 raft_consensus.cc:2804] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:22:00.907065 16219 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57600 (request call id 1981) took 1286 ms. Trace:
I20260709 13:22:00.879966 22659 raft_consensus.cc:515] T 8c11074579b046d99e585c2818e18a2c P cb82cc7bd6db46d980c75237a1402860 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:22:00.902413 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 46356.7us
I20260709 13:22:00.905514 22537 raft_consensus.cc:493] T 5a96373bb0e14dce8a266b2fc6c58507 P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:00.900777 23016 raft_consensus.cc:493] T c4f65cdf00b748bb963ee6e81ef43b79 P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:00.874351 22782 raft_consensus.cc:697] T 3884f864a96d428785895e5463e5728f P 90f3da57796b488c9087a2f523adf045 [term 14 LEADER]: Becoming Leader. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:22:00.907027 22699 raft_consensus.cc:493] T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:22:00.907331 22874 raft_consensus.cc:493] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [term 21 FOLLOWER]: Starting leader election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:22:00.912313 23020 raft_consensus.cc:493] T 7c2095a5f4ad49408dbc7f69c606120d P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:00.894157 23015 raft_consensus.cc:515] T 1df93779209748b9860f6d893dd3de53 P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:00.923429 23020 raft_consensus.cc:515] T 7c2095a5f4ad49408dbc7f69c606120d P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:00.918279 23019 raft_consensus.cc:493] T ada5a004ba7c46819051e3ccec71ffdb P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:22:00.832829 23003 raft_consensus.cc:515] T 23f82224f4564fea93572e8206bad1fe P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:00.926870 23019 raft_consensus.cc:515] T ada5a004ba7c46819051e3ccec71ffdb P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:00.860061 23008 consensus_queue.cc:1048] T 5a959dbecc2648eb85d1cc550f5a3245 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.053s
I20260709 13:22:00.918715 22699 raft_consensus.cc:515] T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:00.932156 23005 leader_election.cc:290] T 2794af5b99984efebdee564035841cb0 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:00.932657 22699 leader_election.cc:290] T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:00.909688 22914 consensus_queue.cc:1243] T d16dfad0c3b540e5b4b86492ddf87208 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Peer 647a343a542d4a139fe106c19095129b log is divergent from this leader: its last log entry 20.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 13:22:00.933532 22946 leader_election.cc:290] T c0eab25ce4c64e018a041beeda2867db P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 19 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:00.934247 22537 raft_consensus.cc:515] T 5a96373bb0e14dce8a266b2fc6c58507 P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:00.922431 22874 raft_consensus.cc:3060] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:22:00.937132 22738 consensus_meta.cc:287] T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860: Time spent flushing consensus metadata: real 0.907s	user 0.012s	sys 0.000s
I20260709 13:22:00.936973 23016 raft_consensus.cc:515] T c4f65cdf00b748bb963ee6e81ef43b79 P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:00.937770 22738 raft_consensus.cc:515] T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:00.939961 22967 leader_election.cc:290] T 18ad7e3af5504571a69d10c00750a01c P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:22:00.910441 22834 raft_consensus.cc:493] T fffab26427e24d398e2695aad40ca134 P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:00.944928 22834 raft_consensus.cc:515] T fffab26427e24d398e2695aad40ca134 P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:00.907629 16219 rpcz_store.cc:276] 0709 13:21:59.620202 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:59.701098 (+ 80896us) service_pool.cc:224] Handling call
0709 13:21:59.701391 (+   293us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:21:59.724411 (+ 23020us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:21:59.724441 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:21:59.752878 (+ 28437us) log.cc:844] Serialized 26 byte log entry
0709 13:22:00.405380 (+652502us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:22:00.405448 (+    68us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:22:00.405566 (+   118us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:22:00.906826 (+501260us) raft_consensus.cc:1685] finished
0709 13:22:00.906852 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:22:00.907003 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8369,"raft.queue_time_us":7700,"raft.run_cpu_time_us":166,"raft.run_wall_time_us":29105,"thread_start_us":379977,"threads_started":1,"wal-append.queue_time_us":591957}
I20260709 13:22:00.947028 22834 leader_election.cc:290] T fffab26427e24d398e2695aad40ca134 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:00.947844 22797 leader_election.cc:290] T 57a744f5aaf944678a63469cbf514015 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 15 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:00.912115 22735 consensus_queue.cc:260] T 9c61a39fd4f84424b129c1a6f9ba404d P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 19.10, Last appended by leader: 10, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:00.949533 22659 leader_election.cc:290] T 8c11074579b046d99e585c2818e18a2c P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:00.949844 23016 leader_election.cc:290] T c4f65cdf00b748bb963ee6e81ef43b79 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:22:00.950449 23016 raft_consensus.cc:493] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:22:00.951359 23016 raft_consensus.cc:515] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:00.929894 23003 leader_election.cc:290] T 23f82224f4564fea93572e8206bad1fe P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
W20260709 13:22:00.960707 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 51804.6us
W20260709 13:22:00.926983 22954 consensus_meta.cc:287] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac: Time spent flushing consensus metadata: real 1.578s	user 0.006s	sys 0.006s
I20260709 13:22:00.925740 23020 leader_election.cc:290] T 7c2095a5f4ad49408dbc7f69c606120d P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:22:00.976361 23013 raft_consensus.cc:493] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:22:00.929946 23019 leader_election.cc:290] T ada5a004ba7c46819051e3ccec71ffdb P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
W20260709 13:22:00.973023 22977 consensus_meta.cc:287] T 434bff3082d141838068f323bdb0d1ed P 647a343a542d4a139fe106c19095129b: Time spent flushing consensus metadata: real 0.931s	user 0.010s	sys 0.004s
I20260709 13:22:00.947090 22943 raft_consensus.cc:493] T 2794af5b99984efebdee564035841cb0 P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:21:59.570439 15046 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:22:01.005574 16264 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:22:00.991649 23020 raft_consensus.cc:493] T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:22:00.995944 22854 consensus_meta.cc:287] T 26354361e9b044cea1c2b86c4882fe92 P 647a343a542d4a139fe106c19095129b: Time spent flushing consensus metadata: real 0.955s	user 0.010s	sys 0.005s
I20260709 13:22:00.992321 22591 raft_consensus.cc:493] T af57996a8be84d398f0f023e8d08ef77 P cb82cc7bd6db46d980c75237a1402860 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:22:00.989709 23013 raft_consensus.cc:515] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:01.018575 23019 raft_consensus.cc:493] T 23f82224f4564fea93572e8206bad1fe P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:22:01.019124 22591 raft_consensus.cc:515] T af57996a8be84d398f0f023e8d08ef77 P cb82cc7bd6db46d980c75237a1402860 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:01.019359 23019 raft_consensus.cc:515] T 23f82224f4564fea93572e8206bad1fe P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:01.021004 23013 leader_election.cc:290] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:22:01.014494 22738 leader_election.cc:290] T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 21 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:01.021942 23008 consensus_queue.cc:1048] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.000s
W20260709 13:22:01.016208 22959 consensus_meta.cc:287] T d2ab0ac8347641e6955e9bce7d90c1a2 P cb82cc7bd6db46d980c75237a1402860: Time spent flushing consensus metadata: real 0.639s	user 0.007s	sys 0.004s
I20260709 13:22:01.015913 23014 consensus_queue.cc:1048] T 5a959dbecc2648eb85d1cc550f5a3245 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
W20260709 13:22:01.017242 22622 consensus_meta.cc:287] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860: Time spent flushing consensus metadata: real 1.022s	user 0.010s	sys 0.001s
I20260709 13:22:01.024564 22622 raft_consensus.cc:515] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:22:01.018607 22987 log.cc:927] Time spent T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 0.141s	user 0.000s	sys 0.000s
I20260709 13:22:01.026422 16226 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57600 (request call id 1979) took 1406 ms. Trace:
I20260709 13:22:01.026775 16226 rpcz_store.cc:276] 0709 13:21:59.619579 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:59.699913 (+ 80334us) service_pool.cc:224] Handling call
0709 13:21:59.700194 (+   281us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:21:59.700628 (+   434us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:21:59.700661 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:21:59.701129 (+   468us) log.cc:844] Serialized 26 byte log entry
0709 13:22:00.054620 (+353491us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:22:00.054666 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:22:00.054756 (+    90us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:22:01.026212 (+971456us) raft_consensus.cc:1685] finished
0709 13:22:01.026232 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:22:01.026362 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":11,"raft.queue_time_us":164,"raft.run_cpu_time_us":96,"raft.run_wall_time_us":96,"thread_start_us":353042,"threads_started":1,"wal-append.queue_time_us":1104327}
I20260709 13:22:01.030085 22622 leader_election.cc:290] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 22 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:22:01.022228 23020 raft_consensus.cc:515] T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:01.032866 23020 leader_election.cc:290] T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:22:00.996431 23018 raft_consensus.cc:493] T e5ad4eb281564936a59f41e387a47156 P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:00.996047 23016 leader_election.cc:290] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:01.049780 22782 consensus_queue.cc:237] T 3884f864a96d428785895e5463e5728f P 90f3da57796b488c9087a2f523adf045 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.13, Last appended by leader: 13, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:00.992841 23003 consensus_queue.cc:1048] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 8, Time since last communication: 0.043s
I20260709 13:22:00.978583 22954 raft_consensus.cc:515] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } }
I20260709 13:22:01.021767 23019 leader_election.cc:290] T 23f82224f4564fea93572e8206bad1fe P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:01.021265 22591 leader_election.cc:290] T af57996a8be84d398f0f023e8d08ef77 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:22:01.030782 22959 raft_consensus.cc:515] T d2ab0ac8347641e6955e9bce7d90c1a2 P cb82cc7bd6db46d980c75237a1402860 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:00.997915 22977 raft_consensus.cc:515] T 434bff3082d141838068f323bdb0d1ed P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:01.003588 22943 raft_consensus.cc:515] T 2794af5b99984efebdee564035841cb0 P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:00.947000 22864 raft_consensus.cc:515] T 5a959dbecc2648eb85d1cc550f5a3245 P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:22:01.179888 16115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 180293us
I20260709 13:22:01.057821 22996 leader_election.cc:290] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:01.168730 23003 consensus_queue.cc:1243] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b [LEADER]: Peer 90f3da57796b488c9087a2f523adf045 log is divergent from this leader: its last log entry 14.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 13:22:01.189427 16115 leader_election.cc:304] T 7c2095a5f4ad49408dbc7f69c606120d P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac
I20260709 13:22:01.196614 16820 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49892 (request call id 986) took 2221 ms. Trace:
I20260709 13:22:01.197149 22955 leader_election.cc:290] T b87147731a68400e8cce9b879daa7c5d P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 20 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
W20260709 13:22:01.202592 23001 log.cc:927] Time spent T 8a878999c0024d2abe517484180290fc P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.764s	user 0.001s	sys 0.005s
I20260709 13:22:01.197047 16820 rpcz_store.cc:276] 0709 13:21:58.970405 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:59.011500 (+ 41095us) service_pool.cc:224] Handling call
0709 13:21:59.067896 (+ 56396us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:21:59.087410 (+ 19514us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:22:01.191012 (+2103602us) spinlock_profiling.cc:243] Waited 23.3 ms on lock 0x7b54007c3b50. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5e8ad 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:22:01.191202 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":53605248}
I20260709 13:22:01.210980 16297 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48674 (request call id 3343) took 2000 ms. Trace:
I20260709 13:22:01.211462 16297 rpcz_store.cc:276] 0709 13:21:59.210409 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:59.648745 (+438336us) service_pool.cc:224] Handling call
0709 13:21:59.649091 (+   346us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:21:59.692068 (+ 42977us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:21:59.692099 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:21:59.697876 (+  5777us) log.cc:844] Serialized 26 byte log entry
0709 13:22:00.374221 (+676345us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:22:00.374302 (+    81us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:22:00.374478 (+   176us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:22:01.201782 (+827304us) spinlock_profiling.cc:243] Waited 174 ms on lock 0x7b4400022b4c. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69ad8fadb1 00007f69ad998c85 00007f69ad9996a2 00007f69b070b1db 00007f69b0706aaf 00007f69b06fa86e 00007f69b06f9bb6 00007f69b06f97d8 00007f69b0703221 00007f69b07031d9 00007f69b0703169
0709 13:22:01.210678 (+  8896us) raft_consensus.cc:1685] finished
0709 13:22:01.210701 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:22:01.210916 (+   215us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":19015,"raft.run_cpu_time_us":106,"raft.run_wall_time_us":4438,"spinlock_wait_cycles":399533824,"thread_start_us":602609,"threads_started":1,"wal-append.queue_time_us":656234}
I20260709 13:22:01.215271 22921 consensus_queue.cc:1059] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: INVALID_TERM, Last received: 19.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:22:01.216573 16818 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49896 (request call id 1512) took 1959 ms. Trace:
I20260709 13:22:01.217103 16818 rpcz_store.cc:276] 0709 13:21:59.257685 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:59.257869 (+   184us) service_pool.cc:224] Handling call
0709 13:21:59.340469 (+ 82600us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:21:59.341231 (+   762us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:22:01.216369 (+1875138us) spinlock_profiling.cc:243] Waited 68.4 ms on lock 0x7b54006bf650. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5e8ad 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:22:01.216514 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":157325696}
I20260709 13:22:01.037565 22854 raft_consensus.cc:515] T 26354361e9b044cea1c2b86c4882fe92 P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:01.178489 23019 raft_consensus.cc:493] T 434bff3082d141838068f323bdb0d1ed P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:01.230878 16300 raft_consensus.cc:1240] T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 19. Current term is 20. Ops: [19.10-19.10]
I20260709 13:22:01.226111 22457 raft_consensus.cc:2804] T 7c2095a5f4ad49408dbc7f69c606120d P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:22:01.232265 22956 raft_consensus.cc:493] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:01.233114 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c74490c40243a88e62d1760b5cd814" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:01.233800 23024 raft_consensus.cc:493] T cdbd78acb4cf46298e0de8ba0d130e19 P 90f3da57796b488c9087a2f523adf045 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:22:01.232861 16296 raft_consensus.cc:1275] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045 [term 21 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260709 13:22:01.234414 23024 raft_consensus.cc:515] T cdbd78acb4cf46298e0de8ba0d130e19 P 90f3da57796b488c9087a2f523adf045 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:01.233945 16298 raft_consensus.cc:2393] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 17: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
I20260709 13:22:01.232849 22956 raft_consensus.cc:515] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:01.236243 16296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e65676b21b748728ad39668cfe722ec" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:01.236971 22977 leader_election.cc:290] T 434bff3082d141838068f323bdb0d1ed P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 18 election: Requested vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:01.237712 16300 raft_consensus.cc:1275] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 13:22:01.238583 22959 leader_election.cc:290] T d2ab0ac8347641e6955e9bce7d90c1a2 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 16 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:01.238647 16295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f82224f4564fea93572e8206bad1fe" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:01.240449 22864 leader_election.cc:290] T 5a959dbecc2648eb85d1cc550f5a3245 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 18 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
W20260709 13:22:01.240972 23010 log.cc:927] Time spent T 02838c86531d4f98bf58d1c22b8baf77 P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 0.384s	user 0.001s	sys 0.000s
I20260709 13:22:01.190598 23020 raft_consensus.cc:493] T 9e65676b21b748728ad39668cfe722ec P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:22:01.241516 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2794af5b99984efebdee564035841cb0" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:01.242043 23016 raft_consensus.cc:493] T 8cc959fb7f704fba9009c0595467d074 P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:01.187623 22699 raft_consensus.cc:493] T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:22:01.244150 16291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:22:01.244612 22943 leader_election.cc:290] T 2794af5b99984efebdee564035841cb0 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:01.043346 23018 raft_consensus.cc:515] T e5ad4eb281564936a59f41e387a47156 P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:01.247256 16294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87147731a68400e8cce9b879daa7c5d" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:22:01.294946 16294 raft_consensus.cc:2376] T b87147731a68400e8cce9b879daa7c5d P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 647a343a542d4a139fe106c19095129b in term 19. Re-sending same reply.
I20260709 13:22:01.247310 22916 raft_consensus.cc:3060] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:22:01.249987 23019 raft_consensus.cc:515] T 434bff3082d141838068f323bdb0d1ed P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:01.252646 22537 leader_election.cc:290] T 5a96373bb0e14dce8a266b2fc6c58507 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
W20260709 13:22:01.253536 23002 log.cc:927] Time spent T b6f07842d3aa47858df5665874c86f3a P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 0.788s	user 0.011s	sys 0.000s
W20260709 13:22:01.253715 22837 consensus_meta.cc:287] T 3884f864a96d428785895e5463e5728f P 647a343a542d4a139fe106c19095129b: Time spent flushing consensus metadata: real 1.444s	user 0.013s	sys 0.000s
I20260709 13:22:01.255354 22854 leader_election.cc:290] T 26354361e9b044cea1c2b86c4882fe92 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 16 election: Requested vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:01.267976 23025 raft_consensus.cc:493] T af57996a8be84d398f0f023e8d08ef77 P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:22:01.301108 16145 raft_consensus.cc:2468] T 3884f864a96d428785895e5463e5728f P 647a343a542d4a139fe106c19095129b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 14.
I20260709 13:22:01.301173 23025 raft_consensus.cc:515] T af57996a8be84d398f0f023e8d08ef77 P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:01.268513 22834 raft_consensus.cc:493] T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:22:01.269014 23014 raft_consensus.cc:493] T 8cc959fb7f704fba9009c0595467d074 P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:22:01.269551 22996 consensus_queue.cc:1048] T 84f2da98864b4425bf7c0055951fdd4b P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:22:01.271982 23026 raft_consensus.cc:493] T fffab26427e24d398e2695aad40ca134 P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:01.273645 23015 leader_election.cc:290] T 1df93779209748b9860f6d893dd3de53 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:01.274155 22954 leader_election.cc:290] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 19 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:01.283183 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87147731a68400e8cce9b879daa7c5d" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:01.175436 22874 raft_consensus.cc:515] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:01.285080 16295 raft_consensus.cc:2468] T 23f82224f4564fea93572e8206bad1fe P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 16.
I20260709 13:22:01.294386 23016 raft_consensus.cc:515] T 8cc959fb7f704fba9009c0595467d074 P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:01.289135 16299 raft_consensus.cc:2468] T 2794af5b99984efebdee564035841cb0 P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 15.
I20260709 13:22:01.325601 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5ad4eb281564936a59f41e387a47156" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:22:01.290863 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18ad7e3af5504571a69d10c00750a01c" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:01.292757 16291 raft_consensus.cc:2376] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045 [term 21 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 647a343a542d4a139fe106c19095129b in term 21. Re-sending same reply.
I20260709 13:22:01.284549 22676 raft_consensus.cc:493] T 8638ee363273426db1393dd81e5c7b72 P cb82cc7bd6db46d980c75237a1402860 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:01.296597 16294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a378d7f796f94615b3b347119a9880fd" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:01.334686 16294 raft_consensus.cc:2468] T a378d7f796f94615b3b347119a9880fd P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 19.
I20260709 13:22:01.298645 16297 raft_consensus.cc:1275] T e5ad4eb281564936a59f41e387a47156 P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 13:22:01.299614 23018 leader_election.cc:290] T e5ad4eb281564936a59f41e387a47156 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:22:01.301262 16146 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40830 (request call id 2008) took 1588 ms. Trace:
I20260709 13:22:01.336252 16146 rpcz_store.cc:276] 0709 13:21:59.713252 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:59.766263 (+ 53011us) service_pool.cc:224] Handling call
0709 13:21:59.766681 (+   418us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:21:59.990591 (+223910us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:21:59.990631 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:22:00.020118 (+ 29487us) log.cc:844] Serialized 26 byte log entry
0709 13:22:00.329304 (+309186us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:22:00.329361 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:22:00.329457 (+    96us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:22:01.249012 (+919555us) spinlock_profiling.cc:243] Waited 143 ms on lock 0x7b4400022b4c. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69ad8fadb1 00007f69ad998c85 00007f69ad9996a2 00007f69b070b1db 00007f69b0706aaf 00007f69b06fa86e 00007f69b06f9bb6 00007f69b06f97d8 00007f69b0703221 00007f69b07031d9 00007f69b0703169
0709 13:22:01.300970 (+ 51958us) raft_consensus.cc:1685] finished
0709 13:22:01.300989 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:22:01.301200 (+   211us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":223406,"raft.queue_time_us":225362,"raft.run_cpu_time_us":139,"raft.run_wall_time_us":29958,"spinlock_wait_cycles":328008576,"thread_start_us":261433,"threads_started":1,"wal-append.queue_time_us":358133}
I20260709 13:22:01.328445 16291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434bff3082d141838068f323bdb0d1ed" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:22:01.337623 22457 raft_consensus.cc:697] T 7c2095a5f4ad49408dbc7f69c606120d P 647a343a542d4a139fe106c19095129b [term 15 LEADER]: Becoming Leader. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:22:01.338758 16291 raft_consensus.cc:3060] T 434bff3082d141838068f323bdb0d1ed P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:22:01.312619 16298 raft_consensus.cc:2468] T b87147731a68400e8cce9b879daa7c5d P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 19.
I20260709 13:22:01.342871 16295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ab0ac8347641e6955e9bce7d90c1a2" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:01.343672 16295 raft_consensus.cc:2468] T d2ab0ac8347641e6955e9bce7d90c1a2 P 90f3da57796b488c9087a2f523adf045 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 14.
I20260709 13:22:01.347929 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26354361e9b044cea1c2b86c4882fe92" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:22:01.351035 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fffab26427e24d398e2695aad40ca134" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:01.354035 16299 raft_consensus.cc:1240] T 5a959dbecc2648eb85d1cc550f5a3245 P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 17. Current term is 18. Ops: [17.11-17.11]
I20260709 13:22:01.356186 16299 raft_consensus.cc:1275] T e4dc6b0541ca431ebc757beac00521f1 P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 17 index: 9. Preceding OpId from leader: term: 19 index: 10. (index mismatch)
I20260709 13:22:01.357520 16299 raft_consensus.cc:3055] T d16dfad0c3b540e5b4b86492ddf87208 P 90f3da57796b488c9087a2f523adf045 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:22:01.357936 16299 raft_consensus.cc:740] T d16dfad0c3b540e5b4b86492ddf87208 P 90f3da57796b488c9087a2f523adf045 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
W20260709 13:22:01.349681 16115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 79729.3us
I20260709 13:22:01.374693 16291 raft_consensus.cc:2468] T 434bff3082d141838068f323bdb0d1ed P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 18.
I20260709 13:22:01.299625 23019 leader_election.cc:290] T 434bff3082d141838068f323bdb0d1ed P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:01.295519 22699 raft_consensus.cc:515] T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:01.414304 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434bff3082d141838068f323bdb0d1ed" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:01.295324 23020 raft_consensus.cc:515] T 9e65676b21b748728ad39668cfe722ec P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:01.413703 16294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a96373bb0e14dce8a266b2fc6c58507" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:01.437933 16294 raft_consensus.cc:2468] T 5a96373bb0e14dce8a266b2fc6c58507 P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 15.
I20260709 13:22:01.419617 16299 consensus_queue.cc:260] T d16dfad0c3b540e5b4b86492ddf87208 P 90f3da57796b488c9087a2f523adf045 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 19.11, Last appended by leader: 11, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:01.399013 16291 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "66ae176961b74813a58bc15c9c4c6f2a"
dest_uuid: "90f3da57796b488c9087a2f523adf045"
 from {username='slave'} at 127.0.0.1:40474
I20260709 13:22:01.436798 16298 raft_consensus.cc:2393] T 434bff3082d141838068f323bdb0d1ed P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 in current term 18: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
I20260709 13:22:01.332332 23015 raft_consensus.cc:493] T 746aabf856ab401c9ef5e859371e7b67 P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:22:01.439062 16291 raft_consensus.cc:493] T 66ae176961b74813a58bc15c9c4c6f2a P 90f3da57796b488c9087a2f523adf045 [term 24 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:22:01.439518 16291 raft_consensus.cc:3060] T 66ae176961b74813a58bc15c9c4c6f2a P 90f3da57796b488c9087a2f523adf045 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:22:01.324047 22834 raft_consensus.cc:515] T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:01.449031 16296 raft_consensus.cc:2468] T 9e65676b21b748728ad39668cfe722ec P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 17.
I20260709 13:22:01.448827 16291 raft_consensus.cc:515] T 66ae176961b74813a58bc15c9c4c6f2a P 90f3da57796b488c9087a2f523adf045 [term 25 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:01.433887 22943 raft_consensus.cc:493] T e4dc6b0541ca431ebc757beac00521f1 P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:01.310542 23014 raft_consensus.cc:515] T 8cc959fb7f704fba9009c0595467d074 P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:01.347851 22676 raft_consensus.cc:515] T 8638ee363273426db1393dd81e5c7b72 P cb82cc7bd6db46d980c75237a1402860 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:01.455022 22956 leader_election.cc:290] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:01.459426 22457 consensus_queue.cc:237] T 7c2095a5f4ad49408dbc7f69c606120d P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 14.7, Last appended by leader: 7, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:01.457382 16294 raft_consensus.cc:1217] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Deduplicated request from leader. Original: 10.8->[12.9-18.12]   Dedup: 12.9->[15.10-18.12]
I20260709 13:22:01.461838 16294 pending_rounds.cc:85] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045: Aborting all ops after (but not including) 9
I20260709 13:22:01.462231 16294 pending_rounds.cc:107] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045: Aborting uncommitted NO_OP operation due to leader change: 14.10
I20260709 13:22:01.462590 16294 raft_consensus.cc:2889] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:22:01.454748 16291 leader_election.cc:290] T 66ae176961b74813a58bc15c9c4c6f2a P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 25 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:01.449893 22854 consensus_queue.cc:1048] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.035s
I20260709 13:22:01.484997 16115 leader_election.cc:304] T b87147731a68400e8cce9b879daa7c5d P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: cb82cc7bd6db46d980c75237a1402860
I20260709 13:22:01.485689 23037 raft_consensus.cc:493] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:22:01.485857 22854 consensus_queue.cc:1048] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:22:01.486862 22537 raft_consensus.cc:493] T 02838c86531d4f98bf58d1c22b8baf77 P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:22:01.486629 23037 raft_consensus.cc:515] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:01.487396 22837 raft_consensus.cc:2804] T b87147731a68400e8cce9b879daa7c5d P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:22:01.487442 22537 raft_consensus.cc:515] T 02838c86531d4f98bf58d1c22b8baf77 P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:01.488117 22837 raft_consensus.cc:697] T b87147731a68400e8cce9b879daa7c5d P 647a343a542d4a139fe106c19095129b [term 19 LEADER]: Becoming Leader. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:22:01.489027 22837 consensus_queue.cc:237] T b87147731a68400e8cce9b879daa7c5d P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:01.489828 22537 leader_election.cc:290] T 02838c86531d4f98bf58d1c22b8baf77 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:01.486148 16115 leader_election.cc:304] T 23f82224f4564fea93572e8206bad1fe P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:22:01.493852 16115 leader_election.cc:304] T 2794af5b99984efebdee564035841cb0 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:22:01.500327 16115 leader_election.cc:304] T 434bff3082d141838068f323bdb0d1ed P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:22:01.347584 23026 raft_consensus.cc:515] T fffab26427e24d398e2695aad40ca134 P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:01.517768 16115 leader_election.cc:304] T 5a96373bb0e14dce8a266b2fc6c58507 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:22:01.491039 23037 leader_election.cc:290] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:01.448554 23015 raft_consensus.cc:515] T 746aabf856ab401c9ef5e859371e7b67 P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:22:01.524325 16115 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:22:01.497954 22517 raft_consensus.cc:2804] T 23f82224f4564fea93572e8206bad1fe P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:22:01.523105 22854 raft_consensus.cc:2804] T 434bff3082d141838068f323bdb0d1ed P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:22:01.497326 23036 raft_consensus.cc:2804] T 2794af5b99984efebdee564035841cb0 P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:22:01.531215 16300 raft_consensus.cc:2468] T fffab26427e24d398e2695aad40ca134 P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 18.
W20260709 13:22:01.525888 23038 log.cc:927] Time spent T 8a878999c0024d2abe517484180290fc P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 0.183s	user 0.001s	sys 0.000s
I20260709 13:22:01.454324 22943 raft_consensus.cc:515] T e4dc6b0541ca431ebc757beac00521f1 P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:01.523030 22996 raft_consensus.cc:2804] T 5a96373bb0e14dce8a266b2fc6c58507 P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:22:01.516637 23008 consensus_queue.cc:1059] T 5a959dbecc2648eb85d1cc550f5a3245 P 647a343a542d4a139fe106c19095129b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: INVALID_TERM, Last received: 15.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.031s
W20260709 13:22:01.549661 16790 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:22:01.516986 22977 consensus_queue.cc:1048] T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.019s
I20260709 13:22:01.536028 23036 raft_consensus.cc:493] T 2794af5b99984efebdee564035841cb0 P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Starting leader election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:22:01.540344 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02838c86531d4f98bf58d1c22b8baf77" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:01.531819 22517 raft_consensus.cc:493] T 23f82224f4564fea93572e8206bad1fe P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Starting leader election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:22:01.532480 22537 raft_consensus.cc:493] T ecc1e363230043fe9fef390f0b979f20 P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:01.572568 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1df93779209748b9860f6d893dd3de53" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:01.549530 22996 raft_consensus.cc:493] T 5a96373bb0e14dce8a266b2fc6c58507 P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:01.565508 22457 raft_consensus.cc:493] T 68a6838d82ef4cac83507f906139d90f P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:22:01.567893 22543 raft_consensus.cc:493] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:22:01.572278 22517 raft_consensus.cc:3060] T 23f82224f4564fea93572e8206bad1fe P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:22:01.569988 23036 raft_consensus.cc:3060] T 2794af5b99984efebdee564035841cb0 P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:22:01.583865 22996 raft_consensus.cc:3060] T 5a96373bb0e14dce8a266b2fc6c58507 P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:22:01.595232 16264 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:22:01.576927 23025 leader_election.cc:290] T af57996a8be84d398f0f023e8d08ef77 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:01.590217 23008 raft_consensus.cc:3055] T 5a959dbecc2648eb85d1cc550f5a3245 P 647a343a542d4a139fe106c19095129b [term 17 LEADER]: Stepping down as leader of term 17
W20260709 13:22:01.568871 16266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 71665.6us
I20260709 13:22:01.600214 22854 raft_consensus.cc:697] T 434bff3082d141838068f323bdb0d1ed P 647a343a542d4a139fe106c19095129b [term 18 LEADER]: Becoming Leader. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
W20260709 13:22:01.607710 23034 log.cc:927] Time spent T 68a6838d82ef4cac83507f906139d90f P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.073s	user 0.000s	sys 0.001s
I20260709 13:22:01.611150 23014 leader_election.cc:290] T 8cc959fb7f704fba9009c0595467d074 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:01.612149 22676 leader_election.cc:290] T 8638ee363273426db1393dd81e5c7b72 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:01.613377 23015 leader_election.cc:290] T 746aabf856ab401c9ef5e859371e7b67 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:01.615448 22854 consensus_queue.cc:237] T 434bff3082d141838068f323bdb0d1ed P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 16.14, Last appended by leader: 14, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:01.596239 22543 raft_consensus.cc:515] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:01.621469 22543 leader_election.cc:290] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:01.579036 22537 raft_consensus.cc:515] T ecc1e363230043fe9fef390f0b979f20 P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:01.625021 22537 leader_election.cc:290] T ecc1e363230043fe9fef390f0b979f20 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:01.625300 23026 leader_election.cc:290] T fffab26427e24d398e2695aad40ca134 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:22:01.626466 22699 leader_election.cc:290] T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:01.631659 23020 leader_election.cc:290] T 9e65676b21b748728ad39668cfe722ec P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:01.636070 22943 leader_election.cc:290] T e4dc6b0541ca431ebc757beac00521f1 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:22:01.599761 23008 raft_consensus.cc:740] T 5a959dbecc2648eb85d1cc550f5a3245 P 647a343a542d4a139fe106c19095129b [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:22:01.637334 23008 consensus_queue.cc:260] T 5a959dbecc2648eb85d1cc550f5a3245 P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.11, Last appended by leader: 11, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:01.638549 23008 raft_consensus.cc:3060] T 5a959dbecc2648eb85d1cc550f5a3245 P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:22:01.645248 22969 log.cc:927] Time spent T 2794af5b99984efebdee564035841cb0 P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 1.273s	user 0.002s	sys 0.000s
I20260709 13:22:01.648097 16295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87147731a68400e8cce9b879daa7c5d" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:22:01.648810 16295 raft_consensus.cc:3060] T b87147731a68400e8cce9b879daa7c5d P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:22:01.654696 16114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:22:01.656322 16114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:22:01.664090 16143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87147731a68400e8cce9b879daa7c5d" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:22:01.672552 16143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af57996a8be84d398f0f023e8d08ef77" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:22:01.673070 16143 raft_consensus.cc:2468] T af57996a8be84d398f0f023e8d08ef77 P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 18.
I20260709 13:22:01.677416 16143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2372981006d4b218215c28964dea623" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:22:01.679483 16143 raft_consensus.cc:3060] T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:22:01.684125 16115 leader_election.cc:304] T 9e65676b21b748728ad39668cfe722ec P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:22:01.698966 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746aabf856ab401c9ef5e859371e7b67" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:22:01.699674 16150 raft_consensus.cc:2468] T 746aabf856ab401c9ef5e859371e7b67 P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 15.
W20260709 13:22:01.702781 16115 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:22:01.704737 16295 raft_consensus.cc:2468] T b87147731a68400e8cce9b879daa7c5d P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 20.
I20260709 13:22:01.663069 22874 leader_election.cc:290] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 22 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:01.670831 22940 raft_consensus.cc:493] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:22:01.670985 22196 raft_consensus.cc:493] T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:22:01.595512 22457 raft_consensus.cc:515] T 68a6838d82ef4cac83507f906139d90f P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:01.675408 22991 raft_consensus.cc:493] T 26354361e9b044cea1c2b86c4882fe92 P cb82cc7bd6db46d980c75237a1402860 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:01.715688 16295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a378d7f796f94615b3b347119a9880fd" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:22:01.677922 22963 raft_consensus.cc:493] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:22:01.721825 16295 raft_consensus.cc:3060] T a378d7f796f94615b3b347119a9880fd P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Advancing to term 21
I20260709 13:22:01.712889 16790 leader_election.cc:304] T af57996a8be84d398f0f023e8d08ef77 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:22:01.721781 16146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e9b2bf7011f4ae688a79d7bb5a447b4" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:22:01.709545 23015 raft_consensus.cc:2804] T 9e65676b21b748728ad39668cfe722ec P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:22:01.739214 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e4bfb9090c8460eb14fa7e0d818e7f1" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:01.740008 16298 raft_consensus.cc:2468] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 19.
I20260709 13:22:01.700542 23036 raft_consensus.cc:515] T 2794af5b99984efebdee564035841cb0 P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:01.728569 16790 leader_election.cc:304] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:22:01.745918 16295 raft_consensus.cc:2468] T a378d7f796f94615b3b347119a9880fd P 90f3da57796b488c9087a2f523adf045 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 21.
W20260709 13:22:01.751868 16790 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.14.177.129:36741 timed out after 1.887s (SENT) [suppressed 27 similar messages]
I20260709 13:22:01.736529 22681 raft_consensus.cc:2804] T af57996a8be84d398f0f023e8d08ef77 P cb82cc7bd6db46d980c75237a1402860 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:22:01.755020 16143 raft_consensus.cc:2468] T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 22.
I20260709 13:22:01.737071 23015 raft_consensus.cc:493] T 9e65676b21b748728ad39668cfe722ec P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Starting leader election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:22:01.725654 22196 raft_consensus.cc:515] T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:01.764846 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e9b2bf7011f4ae688a79d7bb5a447b4" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:01.734102 22963 raft_consensus.cc:515] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:01.750515 22591 raft_consensus.cc:2804] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:22:01.723493 22940 raft_consensus.cc:515] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:01.727783 22991 raft_consensus.cc:515] T 26354361e9b044cea1c2b86c4882fe92 P cb82cc7bd6db46d980c75237a1402860 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:01.759903 22681 raft_consensus.cc:493] T af57996a8be84d398f0f023e8d08ef77 P cb82cc7bd6db46d980c75237a1402860 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:22:01.777705 22681 raft_consensus.cc:3060] T af57996a8be84d398f0f023e8d08ef77 P cb82cc7bd6db46d980c75237a1402860 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:22:01.761085 23015 raft_consensus.cc:3060] T 9e65676b21b748728ad39668cfe722ec P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:22:01.772491 22591 raft_consensus.cc:493] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:22:01.789218 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cc959fb7f704fba9009c0595467d074" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:01.790318 16298 raft_consensus.cc:2393] T 8cc959fb7f704fba9009c0595467d074 P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 17: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
W20260709 13:22:01.797721 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 45891.5us
I20260709 13:22:01.791733 22591 raft_consensus.cc:3060] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:22:01.803697 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ab0ac8347641e6955e9bce7d90c1a2" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:22:01.805020 16298 raft_consensus.cc:3060] T d2ab0ac8347641e6955e9bce7d90c1a2 P 90f3da57796b488c9087a2f523adf045 [term 14 FOLLOWER]: Advancing to term 16
I20260709 13:22:01.810236 16143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b7d1ab519044eda2d5e04a07242f25" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:22:01.810966 16143 raft_consensus.cc:2468] T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 17.
W20260709 13:22:01.810557 16789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 93016.2us
I20260709 13:22:01.816756 16789 leader_election.cc:304] T 434bff3082d141838068f323bdb0d1ed P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
W20260709 13:22:01.827284 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 1.78479e+06us
I20260709 13:22:01.842782 16298 raft_consensus.cc:2468] T d2ab0ac8347641e6955e9bce7d90c1a2 P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 16.
I20260709 13:22:01.829206 22982 raft_consensus.cc:2749] T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:22:01.844770 16143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26354361e9b044cea1c2b86c4882fe92" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:22:01.830194 22979 raft_consensus.cc:2749] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Leader pre-election lost for term 20. Reason: Vote denied by peer 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389) with higher term. Message: Invalid argument: T c9944d5e4b8c4109af9955c4f08c9095 P 90f3da57796b488c9087a2f523adf045 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac for earlier term 20. Current term is 21.
I20260709 13:22:01.847394 16789 leader_election.cc:304] T b87147731a68400e8cce9b879daa7c5d P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 647a343a542d4a139fe106c19095129b
I20260709 13:22:01.851465 16224 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57600 (request call id 1968) took 2460 ms. Trace:
I20260709 13:22:01.852620 16143 raft_consensus.cc:2393] T 26354361e9b044cea1c2b86c4882fe92 P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 in current term 16: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
I20260709 13:22:01.846197 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c11074579b046d99e585c2818e18a2c" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:01.857777 16224 rpcz_store.cc:276] 0709 13:21:59.382063 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:59.382203 (+   140us) service_pool.cc:224] Handling call
0709 13:21:59.393216 (+ 11013us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:21:59.724201 (+330985us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:22:01.839823 (+2115622us) spinlock_profiling.cc:243] Waited 301 ms on lock 0x7b54003d2054. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:22:01.841940 (+  2117us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":693152512}
I20260709 13:22:01.859241 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:01.859937 16298 raft_consensus.cc:2468] T ecc1e363230043fe9fef390f0b979f20 P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 19.
I20260709 13:22:01.861045 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8638ee363273426db1393dd81e5c7b72" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:01.862572 16298 raft_consensus.cc:1240] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [term 22 FOLLOWER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 21. Current term is 22. Ops: []
W20260709 13:22:01.867725 16117 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:22:01.867834 16143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31969392ccdb43b491911a319e8bc7e1" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:22:01.868662 16143 raft_consensus.cc:2468] T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 16.
I20260709 13:22:01.868842 16149 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40830 (request call id 2006) took 2170 ms. Trace:
I20260709 13:22:01.844533 22917 raft_consensus.cc:3055] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [term 24 LEADER]: Stepping down as leader of term 24
I20260709 13:22:01.870743 22917 raft_consensus.cc:740] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:22:01.869203 16149 rpcz_store.cc:276] 0709 13:21:59.696459 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:59.699629 (+  3170us) service_pool.cc:224] Handling call
0709 13:21:59.699907 (+   278us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:22:00.854859 (+1154952us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:22:01.866943 (+1012084us) spinlock_profiling.cc:243] Waited 1.11 s on lock 0x7b540011b2d4. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:22:01.867096 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2544122112}
I20260709 13:22:01.867946 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3884f864a96d428785895e5463e5728f" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:22:01.840019 22754 raft_consensus.cc:493] T 829b4ad8c5a54eababeea1181b960d4b P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
W20260709 13:22:01.860977 23043 log.cc:927] Time spent T c0eab25ce4c64e018a041beeda2867db P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.099s	user 0.000s	sys 0.001s
I20260709 13:22:01.851713 22888 raft_consensus.cc:3060] T 434bff3082d141838068f323bdb0d1ed P cb82cc7bd6db46d980c75237a1402860 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:22:01.862865 22963 leader_election.cc:290] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
W20260709 13:22:01.870980 23048 log.cc:927] Time spent T 3884f864a96d428785895e5463e5728f P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.094s	user 0.000s	sys 0.001s
I20260709 13:22:01.861559 22940 leader_election.cc:290] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:22:01.881908 16789 leader_election.cc:304] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:22:01.860077 22457 leader_election.cc:290] T 68a6838d82ef4cac83507f906139d90f P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:01.888947 16789 leader_election.cc:304] T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:22:01.886330 22908 raft_consensus.cc:493] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:01.892005 22908 raft_consensus.cc:515] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:01.892647 16816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87147731a68400e8cce9b879daa7c5d" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:01.893802 16818 raft_consensus.cc:1275] T 8c11074579b046d99e585c2818e18a2c P cb82cc7bd6db46d980c75237a1402860 [term 19 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 13:22:01.894399 16817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7ca61592e848ea9b0fb81a2ce33cdc" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:01.894651 16822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a378d7f796f94615b3b347119a9880fd" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:01.895632 16822 raft_consensus.cc:2368] T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 for earlier term 20. Current term is 21.
I20260709 13:22:01.896591 16825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74ca8140f35c4ae29e8a664f6bdfc829" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:01.907639 16823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a9de347c711477e87c582d003f36925" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
W20260709 13:22:01.907693 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 66019.3us
I20260709 13:22:01.957580 16820 raft_consensus.cc:1275] T 8638ee363273426db1393dd81e5c7b72 P cb82cc7bd6db46d980c75237a1402860 [term 19 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
W20260709 13:22:01.958150 16790 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:22:01.959285 16821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada5a004ba7c46819051e3ccec71ffdb" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:01.959411 16819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ab0ac8347641e6955e9bce7d90c1a2" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:01.884336 22991 leader_election.cc:290] T 26354361e9b044cea1c2b86c4882fe92 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:01.960297 16819 raft_consensus.cc:2368] T d2ab0ac8347641e6955e9bce7d90c1a2 P cb82cc7bd6db46d980c75237a1402860 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 for earlier term 15. Current term is 16.
W20260709 13:22:01.961923 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 1.93363e+06us
I20260709 13:22:01.888304 22348 raft_consensus.cc:493] T 66f75124215f4ca6bef831e174a792d8 P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
W20260709 13:22:01.957441 23050 log.cc:927] Time spent T 434bff3082d141838068f323bdb0d1ed P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 0.165s	user 0.001s	sys 0.000s
I20260709 13:22:01.898813 22938 raft_consensus.cc:493] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:22:01.912050 22963 raft_consensus.cc:2804] T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:22:01.900473 22940 raft_consensus.cc:2804] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P cb82cc7bd6db46d980c75237a1402860 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:22:01.894076 22908 leader_election.cc:290] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:01.971560 16823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f82224f4564fea93572e8206bad1fe" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:01.881353 22754 raft_consensus.cc:515] T 829b4ad8c5a54eababeea1181b960d4b P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:01.900672 23053 raft_consensus.cc:2804] T b87147731a68400e8cce9b879daa7c5d P cb82cc7bd6db46d980c75237a1402860 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:22:01.974690 22754 leader_election.cc:290] T 829b4ad8c5a54eababeea1181b960d4b P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:01.975524 22996 raft_consensus.cc:515] T 5a96373bb0e14dce8a266b2fc6c58507 P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:01.975894 16823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434bff3082d141838068f323bdb0d1ed" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:01.976992 16190 leader_election.cc:304] T e5ad4eb281564936a59f41e387a47156 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:22:01.978062 16824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ab0ac8347641e6955e9bce7d90c1a2" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:01.978060 16817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a9de347c711477e87c582d003f36925" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:22:01.878420 22917 consensus_queue.cc:260] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 24.15, Last appended by leader: 15, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:01.984364 16824 raft_consensus.cc:2368] T d2ab0ac8347641e6955e9bce7d90c1a2 P cb82cc7bd6db46d980c75237a1402860 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 for earlier term 15. Current term is 16.
I20260709 13:22:01.984786 16825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87147731a68400e8cce9b879daa7c5d" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:01.985870 16822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e65676b21b748728ad39668cfe722ec" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:01.979301 16820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2794af5b99984efebdee564035841cb0" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:01.987224 16817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f82224f4564fea93572e8206bad1fe" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:01.989426 16220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0eab25ce4c64e018a041beeda2867db" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:01.990165 16220 raft_consensus.cc:2393] T c0eab25ce4c64e018a041beeda2867db P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 19: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
I20260709 13:22:01.990792 16817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2794af5b99984efebdee564035841cb0" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:01.992690 16223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a96373bb0e14dce8a266b2fc6c58507" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:01.993371 16223 raft_consensus.cc:2468] T 5a96373bb0e14dce8a266b2fc6c58507 P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 14.
I20260709 13:22:01.995548 16825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e65676b21b748728ad39668cfe722ec" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:01.997767 16821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e4bfb9090c8460eb14fa7e0d818e7f1" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:01.997990 16818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a9de347c711477e87c582d003f36925" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:01.999356 16819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66ae176961b74813a58bc15c9c4c6f2a" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: true dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:22:02.000133 16819 raft_consensus.cc:3055] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [term 24 LEADER]: Stepping down as leader of term 24
I20260709 13:22:02.000550 16819 raft_consensus.cc:740] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:22:02.003983 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c2095a5f4ad49408dbc7f69c606120d" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:02.004767 16217 raft_consensus.cc:2468] T 7c2095a5f4ad49408dbc7f69c606120d P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 15.
I20260709 13:22:02.006776 16820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "829b4ad8c5a54eababeea1181b960d4b" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:01.972196 22940 raft_consensus.cc:493] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P cb82cc7bd6db46d980c75237a1402860 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:22:02.001529 16819 consensus_queue.cc:260] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 24.15, Last appended by leader: 15, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:02.008347 16825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b7d1ab519044eda2d5e04a07242f25" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:02.008733 16819 raft_consensus.cc:3060] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:22:02.008924 16825 raft_consensus.cc:2468] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 17.
I20260709 13:22:02.011443 16223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7ca61592e848ea9b0fb81a2ce33cdc" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:02.009977 16824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee9ed67006144797b1de6ea8c2013a68" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:01.989611 22954 raft_consensus.cc:493] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac [term 20 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:22:02.012990 16220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada5a004ba7c46819051e3ccec71ffdb" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:02.014302 16220 raft_consensus.cc:2468] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 19.
I20260709 13:22:01.984277 22517 raft_consensus.cc:515] T 23f82224f4564fea93572e8206bad1fe P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:01.990841 22963 raft_consensus.cc:697] T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860 [term 21 LEADER]: Becoming Leader. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:22:01.972429 22348 raft_consensus.cc:515] T 66f75124215f4ca6bef831e174a792d8 P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:01.994064 23053 raft_consensus.cc:697] T b87147731a68400e8cce9b879daa7c5d P cb82cc7bd6db46d980c75237a1402860 [term 20 LEADER]: Becoming Leader. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
W20260709 13:22:02.007717 23046 log.cc:927] Time spent T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.194s	user 0.001s	sys 0.000s
W20260709 13:22:02.014964 16790 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:22:01.971809 22938 raft_consensus.cc:515] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:02.019675 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4f65cdf00b748bb963ee6e81ef43b79" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
W20260709 13:22:02.027737 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 50890.4us
I20260709 13:22:02.028344 16825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "829b4ad8c5a54eababeea1181b960d4b" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:02.028528 16190 leader_election.cc:304] T 7c2095a5f4ad49408dbc7f69c606120d P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:22:02.018608 22962 raft_consensus.cc:493] T e4dc6b0541ca431ebc757beac00521f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:02.016862 22954 raft_consensus.cc:515] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:02.021690 22996 leader_election.cc:290] T 5a96373bb0e14dce8a266b2fc6c58507 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 16 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:02.021270 22963 consensus_queue.cc:237] T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:02.022431 22748 raft_consensus.cc:2749] T e5ad4eb281564936a59f41e387a47156 P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:22:02.044960 22348 leader_election.cc:290] T 66f75124215f4ca6bef831e174a792d8 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:02.047605 16824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434bff3082d141838068f323bdb0d1ed" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:22:02.048514 22938 leader_election.cc:290] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:02.049968 16819 raft_consensus.cc:2468] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 25.
I20260709 13:22:02.011132 22940 raft_consensus.cc:3060] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P cb82cc7bd6db46d980c75237a1402860 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:22:02.024849 16820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5718c623ebee4d2bb770f0f9ddb2f164" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:02.050813 16223 raft_consensus.cc:2468] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 19.
I20260709 13:22:02.032289 22982 raft_consensus.cc:493] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:02.032948 22908 raft_consensus.cc:493] T 84f2da98864b4425bf7c0055951fdd4b P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:22:02.035349 16217 raft_consensus.cc:2410] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 for term 21 because replica has last-logged OpId of term: 19 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 12.
I20260709 13:22:02.002727 23015 raft_consensus.cc:515] T 9e65676b21b748728ad39668cfe722ec P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:02.039994 16818 raft_consensus.cc:1240] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 [term 22 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 21. Current term is 22. Ops: []
I20260709 13:22:02.044849 22517 leader_election.cc:290] T 23f82224f4564fea93572e8206bad1fe P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 17 election: Requested vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:02.025488 16220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a959dbecc2648eb85d1cc550f5a3245" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:22:02.058207 16220 raft_consensus.cc:3060] T 5a959dbecc2648eb85d1cc550f5a3245 P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:22:02.060035 16218 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33050 (request call id 1175) took 2509 ms. Trace:
I20260709 13:22:02.060563 16190 leader_election.cc:304] T 8a878999c0024d2abe517484180290fc P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 647a343a542d4a139fe106c19095129b
I20260709 13:22:02.051277 16819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66f75124215f4ca6bef831e174a792d8" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:02.064424 16825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26354361e9b044cea1c2b86c4882fe92" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:22:02.065141 16825 raft_consensus.cc:3060] T 26354361e9b044cea1c2b86c4882fe92 P cb82cc7bd6db46d980c75237a1402860 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:22:02.051635 23060 log.cc:927] Time spent T 84f2da98864b4425bf7c0055951fdd4b P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.087s	user 0.000s	sys 0.001s
I20260709 13:22:02.073534 22748 raft_consensus.cc:2749] T 7c2095a5f4ad49408dbc7f69c606120d P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:22:02.042589 22962 raft_consensus.cc:515] T e4dc6b0541ca431ebc757beac00521f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:02.095361 16222 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57600 (request call id 1986) took 1630 ms. Trace:
I20260709 13:22:02.074833 16824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7ca61592e848ea9b0fb81a2ce33cdc" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:02.076205 23015 leader_election.cc:290] T 9e65676b21b748728ad39668cfe722ec P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 18 election: Requested vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:02.077771 22681 raft_consensus.cc:515] T af57996a8be84d398f0f023e8d08ef77 P cb82cc7bd6db46d980c75237a1402860 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:02.064893 22908 raft_consensus.cc:515] T 84f2da98864b4425bf7c0055951fdd4b P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:02.095763 16222 rpcz_store.cc:276] 0709 13:22:00.465015 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:22:00.480091 (+ 15076us) service_pool.cc:224] Handling call
0709 13:22:00.480486 (+   395us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 13:22:01.868552 (+1388066us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:22:01.868583 (+    31us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 13:22:01.875682 (+  7099us) log.cc:844] Serialized 78 byte log entry
0709 13:22:01.979117 (+103435us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:22:02.015251 (+ 36134us) log.cc:844] Serialized 14 byte log entry
0709 13:22:02.036817 (+ 21566us) log.cc:844] Serialized 14 byte log entry
0709 13:22:02.060146 (+ 23329us) log.cc:844] Serialized 14 byte log entry
0709 13:22:02.095033 (+ 34887us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:22:02.095115 (+    82us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:22:02.095151 (+    36us) raft_consensus.cc:1685] finished
0709 13:22:02.095179 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:22:02.095307 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1572650,"prepare.queue_time_us":114972,"prepare.run_cpu_time_us":78,"prepare.run_wall_time_us":83,"raft.queue_time_us":1435361,"raft.run_cpu_time_us":194,"raft.run_wall_time_us":6017,"spinlock_wait_cycles":28422016,"thread_start_us":117858,"threads_started":2,"wal-append.queue_time_us":146869}
I20260709 13:22:02.107868 16222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18ad7e3af5504571a69d10c00750a01c" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:02.065671 22982 raft_consensus.cc:515] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:02.116019 16292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26354361e9b044cea1c2b86c4882fe92" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:02.116250 16222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee9ed67006144797b1de6ea8c2013a68" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:02.083467 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6f07842d3aa47858df5665874c86f3a" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:02.084184 22888 raft_consensus.cc:2749] T 434bff3082d141838068f323bdb0d1ed P cb82cc7bd6db46d980c75237a1402860 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:22:02.084107 23061 consensus_queue.cc:1048] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:22:02.085019 16820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af57996a8be84d398f0f023e8d08ef77" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:02.087452 16149 raft_consensus.cc:1240] T 9e65676b21b748728ad39668cfe722ec P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 17. Current term is 18. Ops: []
W20260709 13:22:02.088335 16036 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 19baa7bc367844c288bca8f81439c413 with cas_config_opid_index 7: ChangeConfig:REMOVE_PEER failed with leader cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica cb82cc7bd6db46d980c75237a1402860 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 21 committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:22:02.122365 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2372981006d4b218215c28964dea623" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:02.120096 16265 leader_election.cc:304] T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:22:02.090713 16291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31969392ccdb43b491911a319e8bc7e1" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:02.093995 16150 raft_consensus.cc:1240] T 23f82224f4564fea93572e8206bad1fe P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 16. Current term is 17. Ops: []
I20260709 13:22:02.085039 22591 raft_consensus.cc:515] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:02.059124 22960 raft_consensus.cc:493] T c9944d5e4b8c4109af9955c4f08c9095 P 90f3da57796b488c9087a2f523adf045 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:02.125726 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5718c623ebee4d2bb770f0f9ddb2f164" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:02.087518 22834 leader_election.cc:290] T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:02.106758 16190 leader_election.cc:400] T 5a959dbecc2648eb85d1cc550f5a3245 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 16 pre-election: Vote denied by peer 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389) with higher term. Message: Invalid argument: T 5a959dbecc2648eb85d1cc550f5a3245 P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac for earlier term 16. Current term is 17.
I20260709 13:22:02.066027 22953 raft_consensus.cc:493] T ee9ed67006144797b1de6ea8c2013a68 P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:22:02.120081 16825 raft_consensus.cc:2468] T 26354361e9b044cea1c2b86c4882fe92 P cb82cc7bd6db46d980c75237a1402860 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 16.
I20260709 13:22:02.119122 16220 raft_consensus.cc:2468] T 5a959dbecc2648eb85d1cc550f5a3245 P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 18.
W20260709 13:22:02.117051 23065 log.cc:927] Time spent T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
I20260709 13:22:02.126328 16225 raft_consensus.cc:2468] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 18.
I20260709 13:22:02.060417 16218 rpcz_store.cc:276] 0709 13:21:59.550819 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:59.550967 (+   148us) service_pool.cc:224] Handling call
0709 13:21:59.595845 (+ 44878us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:21:59.596132 (+   287us) raft_consensus.cc:1534] Early marking committed up to index 15
0709 13:21:59.645643 (+ 49511us) log.cc:844] Serialized 14 byte log entry
0709 13:22:00.022852 (+377209us) log.cc:844] Serialized 14 byte log entry
0709 13:22:01.636438 (+1613586us) spinlock_profiling.cc:243] Waited 30.3 ms on lock 0x7b1800416320. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b693ade1 00007f69b69475b1 00007f69ada723f3 00007f69ada747c1 00007f69ada74779 00007f69ada74709 00007f69ada746d1 00007f69ada739cd 000000000038f354 000000000038f199 00007f69ada5583a
0709 13:22:01.812623 (+176185us) spinlock_profiling.cc:243] Waited 25.5 ms on lock 0x7b1800416320. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b693ade1 00007f69b69475b1 00007f69ad9ecc0a 00007f69b712c219 00007f69ada74a40 00007f69ada6f4e4 00007f69ada6f455 00007f69b0b57060 00007f69b0b69716 00007f69b0b7a287 00007f69b0b7a208
0709 13:22:01.960378 (+147755us) log.cc:844] Serialized 14 byte log entry
0709 13:22:02.059712 (+ 99334us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:22:02.059762 (+    50us) raft_consensus.cc:1645] Marking committed up to 15
0709 13:22:02.059811 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:22:02.059882 (+    71us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:22:02.059984 (+   102us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":451516,"prepare.queue_time_us":183726,"prepare.run_cpu_time_us":82,"prepare.run_wall_time_us":7649,"raft.queue_time_us":1809465,"raft.run_cpu_time_us":218,"raft.run_wall_time_us":3879,"spinlock_wait_cycles":128292096,"thread_start_us":569065,"threads_started":4,"wal-append.queue_time_us":1076666,"wal-append.run_cpu_time_us":2297,"wal-append.run_wall_time_us":1037494}
I20260709 13:22:02.123973 16819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5d59407eb545b895d8d922e0dd69fd" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:02.127462 16817 raft_consensus.cc:1240] T af57996a8be84d398f0f023e8d08ef77 P cb82cc7bd6db46d980c75237a1402860 [term 19 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 18. Current term is 19. Ops: []
I20260709 13:22:02.129719 16190 leader_election.cc:403] T 5a959dbecc2648eb85d1cc550f5a3245 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 16 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:22:02.130389 16823 raft_consensus.cc:2468] T 434bff3082d141838068f323bdb0d1ed P cb82cc7bd6db46d980c75237a1402860 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 18.
I20260709 13:22:02.131251 16820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746aabf856ab401c9ef5e859371e7b67" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:02.115439 22916 raft_consensus.cc:2804] T 8a878999c0024d2abe517484180290fc P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:22:02.133031 16824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2372981006d4b218215c28964dea623" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:02.131963 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a96373bb0e14dce8a266b2fc6c58507" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:02.135037 16225 raft_consensus.cc:2468] T 5a96373bb0e14dce8a266b2fc6c58507 P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 14.
I20260709 13:22:02.136888 16824 raft_consensus.cc:2393] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 22: Already voted for candidate cb82cc7bd6db46d980c75237a1402860 in this term.
I20260709 13:22:02.136338 16190 leader_election.cc:304] T 5a96373bb0e14dce8a266b2fc6c58507 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:22:02.138496 16820 raft_consensus.cc:2393] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 16: Already voted for candidate cb82cc7bd6db46d980c75237a1402860 in this term.
I20260709 13:22:02.141337 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af57996a8be84d398f0f023e8d08ef77" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:02.142961 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e4bfb9090c8460eb14fa7e0d818e7f1" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:02.143569 16220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a378d7f796f94615b3b347119a9880fd" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:22:02.143945 16218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af57996a8be84d398f0f023e8d08ef77" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:02.144212 16220 raft_consensus.cc:3060] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:22:02.130316 22864 raft_consensus.cc:2804] T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:22:02.148371 16819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5d59407eb545b895d8d922e0dd69fd" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:02.116850 22938 consensus_queue.cc:1059] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: INVALID_TERM, Last received: 21.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.022s
I20260709 13:22:02.149811 16190 leader_election.cc:304] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860
I20260709 13:22:02.151232 16265 leader_election.cc:304] T 5a96373bb0e14dce8a266b2fc6c58507 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:22:02.135329 22916 raft_consensus.cc:493] T 8a878999c0024d2abe517484180290fc P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:02.152561 22916 raft_consensus.cc:3060] T 8a878999c0024d2abe517484180290fc P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:22:02.154685 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c74490c40243a88e62d1760b5cd814" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:02.156494 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b7d1ab519044eda2d5e04a07242f25" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:02.156778 16265 leader_election.cc:304] T 7c2095a5f4ad49408dbc7f69c606120d P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:22:02.137957 22748 raft_consensus.cc:2749] T 5a959dbecc2648eb85d1cc550f5a3245 P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Leader pre-election lost for term 16. Reason: Vote denied by peer 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389) with higher term. Message: Invalid argument: T 5a959dbecc2648eb85d1cc550f5a3245 P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac for earlier term 16. Current term is 17.
I20260709 13:22:02.162154 16225 raft_consensus.cc:2393] T 08c74490c40243a88e62d1760b5cd814 P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 17: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
I20260709 13:22:02.170147 16824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:02.175911 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fffab26427e24d398e2695aad40ca134" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:02.176558 16225 raft_consensus.cc:2468] T fffab26427e24d398e2695aad40ca134 P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 17.
I20260709 13:22:02.177425 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746aabf856ab401c9ef5e859371e7b67" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:02.153380 23059 raft_consensus.cc:3060] T 5a96373bb0e14dce8a266b2fc6c58507 P 1ca2725d290e47d68b1a4b609c9816ac [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:22:02.130723 22960 raft_consensus.cc:515] T c9944d5e4b8c4109af9955c4f08c9095 P 90f3da57796b488c9087a2f523adf045 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:02.180238 16190 leader_election.cc:304] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860
I20260709 13:22:02.153687 22864 raft_consensus.cc:493] T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Starting leader election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:22:02.190984 22864 raft_consensus.cc:3060] T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:22:02.155551 22874 consensus_queue.cc:1059] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 90f3da57796b488c9087a2f523adf045 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: INVALID_TERM, Last received: 17.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.036s
I20260709 13:22:02.192123 22591 leader_election.cc:290] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 16 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
W20260709 13:22:02.192795 23062 log.cc:927] Time spent T e4dc6b0541ca431ebc757beac00521f1 P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.098s	user 0.000s	sys 0.001s
I20260709 13:22:02.186077 16220 raft_consensus.cc:2468] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 21.
I20260709 13:22:02.186527 16190 leader_election.cc:304] T b6f07842d3aa47858df5665874c86f3a P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:22:02.186408 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5d59407eb545b895d8d922e0dd69fd" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:02.189694 22754 raft_consensus.cc:2749] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:22:02.190248 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e4bfb9090c8460eb14fa7e0d818e7f1" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:02.190487 23061 raft_consensus.cc:2749] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:22:02.201061 16218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02838c86531d4f98bf58d1c22b8baf77" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:02.201275 16190 leader_election.cc:304] T 68a6838d82ef4cac83507f906139d90f P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:22:02.204432 22864 raft_consensus.cc:515] T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:02.209295 22681 leader_election.cc:290] T af57996a8be84d398f0f023e8d08ef77 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 19 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:22:02.209839 22864 leader_election.cc:290] T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 20 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:02.161604 23020 raft_consensus.cc:2804] T 5a96373bb0e14dce8a266b2fc6c58507 P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:22:02.211643 23020 raft_consensus.cc:493] T 5a96373bb0e14dce8a266b2fc6c58507 P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:02.211863 16143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746aabf856ab401c9ef5e859371e7b67" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:22:02.212282 23020 raft_consensus.cc:3060] T 5a96373bb0e14dce8a266b2fc6c58507 P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:22:02.212471 16143 raft_consensus.cc:3060] T 746aabf856ab401c9ef5e859371e7b67 P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:22:02.215689 23036 leader_election.cc:290] T 2794af5b99984efebdee564035841cb0 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 16 election: Requested vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:02.215870 16298 raft_consensus.cc:1275] T 57a744f5aaf944678a63469cbf514015 P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 13:22:02.133522 23053 consensus_queue.cc:237] T b87147731a68400e8cce9b879daa7c5d P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:22:02.218483 23049 log.cc:927] Time spent T b87147731a68400e8cce9b879daa7c5d P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 0.426s	user 0.001s	sys 0.001s
I20260709 13:22:02.178740 22938 raft_consensus.cc:3055] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:22:02.220528 23024 leader_election.cc:290] T cdbd78acb4cf46298e0de8ba0d130e19 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:02.223264 16143 raft_consensus.cc:2468] T 746aabf856ab401c9ef5e859371e7b67 P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 16.
I20260709 13:22:02.199726 16225 raft_consensus.cc:2468] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 20.
I20260709 13:22:02.227274 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746aabf856ab401c9ef5e859371e7b67" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:02.136044 22953 raft_consensus.cc:515] T ee9ed67006144797b1de6ea8c2013a68 P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:02.210459 22864 raft_consensus.cc:2804] T 7c2095a5f4ad49408dbc7f69c606120d P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:22:02.261734 16819 raft_consensus.cc:1275] T 19baa7bc367844c288bca8f81439c413 P cb82cc7bd6db46d980c75237a1402860 [term 21 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260709 13:22:02.263325 16218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cc959fb7f704fba9009c0595467d074" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:02.263793 16143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af57996a8be84d398f0f023e8d08ef77" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:22:02.264519 16143 raft_consensus.cc:3060] T af57996a8be84d398f0f023e8d08ef77 P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:22:02.265174 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 63982.5us
I20260709 13:22:02.267603 16291 raft_consensus.cc:1275] T c9944d5e4b8c4109af9955c4f08c9095 P 90f3da57796b488c9087a2f523adf045 [term 21 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
I20260709 13:22:02.271005 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2372981006d4b218215c28964dea623" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:22:02.273921 16222 raft_consensus.cc:2468] T ee9ed67006144797b1de6ea8c2013a68 P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 19.
I20260709 13:22:02.231333 22938 raft_consensus.cc:740] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:22:02.279305 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1df93779209748b9860f6d893dd3de53" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:02.258873 22748 raft_consensus.cc:2749] T 68a6838d82ef4cac83507f906139d90f P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:22:02.285215 16219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746aabf856ab401c9ef5e859371e7b67" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:22:02.296427 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5ad4eb281564936a59f41e387a47156" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:02.297169 16224 raft_consensus.cc:2468] T e5ad4eb281564936a59f41e387a47156 P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 18.
I20260709 13:22:02.285131 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b7d1ab519044eda2d5e04a07242f25" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:02.298705 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ab0ac8347641e6955e9bce7d90c1a2" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
W20260709 13:22:02.291843 22939 log.cc:927] Time spent T 9e4bfb9090c8460eb14fa7e0d818e7f1 P cb82cc7bd6db46d980c75237a1402860: Append to log took a long time: real 1.884s	user 0.013s	sys 0.000s
I20260709 13:22:02.259171 22864 raft_consensus.cc:493] T 7c2095a5f4ad49408dbc7f69c606120d P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:22:02.290302 23051 log.cc:927] Time spent T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.461s	user 0.000s	sys 0.001s
I20260709 13:22:02.299582 16224 raft_consensus.cc:3060] T d2ab0ac8347641e6955e9bce7d90c1a2 P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:22:02.308019 16789 leader_election.cc:304] T d2ab0ac8347641e6955e9bce7d90c1a2 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 
W20260709 13:22:02.365891 16789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 57959us
I20260709 13:22:02.328322 16143 raft_consensus.cc:2468] T af57996a8be84d398f0f023e8d08ef77 P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 19.
I20260709 13:22:02.361503 16219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66f75124215f4ca6bef831e174a792d8" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:02.363265 16218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66ae176961b74813a58bc15c9c4c6f2a" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: true dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:22:02.364259 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fffab26427e24d398e2695aad40ca134" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:02.369030 16219 raft_consensus.cc:2468] T 66f75124215f4ca6bef831e174a792d8 P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 16.
I20260709 13:22:02.369267 16225 raft_consensus.cc:2468] T fffab26427e24d398e2695aad40ca134 P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 17.
I20260709 13:22:02.369925 16219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "829b4ad8c5a54eababeea1181b960d4b" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:02.370044 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdbd78acb4cf46298e0de8ba0d130e19" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:02.370544 16225 raft_consensus.cc:2468] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 21.
I20260709 13:22:02.369035 16218 raft_consensus.cc:3060] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:22:02.370517 16219 raft_consensus.cc:2468] T 829b4ad8c5a54eababeea1181b960d4b P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 16.
I20260709 13:22:02.372726 16219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c61a39fd4f84424b129c1a6f9ba404d" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 21 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:02.373521 16219 raft_consensus.cc:2410] T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 for term 21 because replica has last-logged OpId of term: 19 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 9.
W20260709 13:22:02.381568 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 115538us
I20260709 13:22:02.382699 16190 leader_election.cc:304] T 18ad7e3af5504571a69d10c00750a01c P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:22:02.379274 22508 raft_consensus.cc:2804] T d2ab0ac8347641e6955e9bce7d90c1a2 P cb82cc7bd6db46d980c75237a1402860 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:22:02.389689 22754 raft_consensus.cc:2749] T 18ad7e3af5504571a69d10c00750a01c P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:22:02.395910 22508 raft_consensus.cc:697] T d2ab0ac8347641e6955e9bce7d90c1a2 P cb82cc7bd6db46d980c75237a1402860 [term 16 LEADER]: Becoming Leader. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:22:02.396917 22508 consensus_queue.cc:237] T d2ab0ac8347641e6955e9bce7d90c1a2 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 14.13, Last appended by leader: 13, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:02.385331 16190 leader_election.cc:304] T c2f5f89186204242bf377f7c42b2e2da P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:22:02.399370 22591 consensus_queue.cc:1048] T 57a744f5aaf944678a63469cbf514015 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.003s
I20260709 13:22:02.366922 22864 raft_consensus.cc:3060] T 7c2095a5f4ad49408dbc7f69c606120d P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:22:02.373610 22874 raft_consensus.cc:3055] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 90f3da57796b488c9087a2f523adf045 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:22:02.414098 16190 leader_election.cc:304] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860
W20260709 13:22:02.415050 16790 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:22:02.410735 22874 raft_consensus.cc:740] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 90f3da57796b488c9087a2f523adf045 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:22:02.381213 22676 consensus_queue.cc:1059] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: INVALID_TERM, Last received: 21.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.033s
I20260709 13:22:02.411247 22754 raft_consensus.cc:2749] T c2f5f89186204242bf377f7c42b2e2da P 1ca2725d290e47d68b1a4b609c9816ac [term 20 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:22:02.400139 22940 raft_consensus.cc:515] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P cb82cc7bd6db46d980c75237a1402860 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:02.428275 22874 consensus_queue.cc:260] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 90f3da57796b488c9087a2f523adf045 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.13, Last appended by leader: 13, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:22:02.436998 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 280318us
I20260709 13:22:02.421015 22748 raft_consensus.cc:2749] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:22:02.439921 16265 leader_election.cc:304] T ada5a004ba7c46819051e3ccec71ffdb P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:22:02.441715 16218 raft_consensus.cc:2468] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 25.
I20260709 13:22:02.443717 16218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af57996a8be84d398f0f023e8d08ef77" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:22:02.444458 16821 raft_consensus.cc:2393] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P cb82cc7bd6db46d980c75237a1402860 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 20: Already voted for candidate cb82cc7bd6db46d980c75237a1402860 in this term.
I20260709 13:22:02.416818 16190 leader_election.cc:304] T fffab26427e24d398e2695aad40ca134 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:22:02.437731 22735 raft_consensus.cc:3060] T 9c61a39fd4f84424b129c1a6f9ba404d P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:22:02.449254 16265 leader_election.cc:304] T 5a959dbecc2648eb85d1cc550f5a3245 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:22:02.451828 16218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4dc6b0541ca431ebc757beac00521f1" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 20 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:02.442265 23077 raft_consensus.cc:2804] T ada5a004ba7c46819051e3ccec71ffdb P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:22:02.455415 22754 raft_consensus.cc:3060] T fffab26427e24d398e2695aad40ca134 P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:22:02.456657 22672 raft_consensus.cc:3060] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:22:02.442977 23003 raft_consensus.cc:493] T 8d4ab870a05347868956aa8b841b2981 P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:22:02.459239 16218 raft_consensus.cc:2468] T e4dc6b0541ca431ebc757beac00521f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 19.
I20260709 13:22:02.458392 16265 leader_election.cc:304] T fffab26427e24d398e2695aad40ca134 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:22:02.459105 16190 leader_election.cc:304] T 8c11074579b046d99e585c2818e18a2c P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860
I20260709 13:22:02.422380 22938 consensus_queue.cc:260] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 21.14, Last appended by leader: 14, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:02.454180 22874 raft_consensus.cc:3060] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:22:02.475342 16190 leader_election.cc:304] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:22:02.460000 23077 raft_consensus.cc:493] T ada5a004ba7c46819051e3ccec71ffdb P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Starting leader election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:22:02.465831 22709 raft_consensus.cc:2804] T 5a959dbecc2648eb85d1cc550f5a3245 P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:22:02.465574 22960 leader_election.cc:290] T c9944d5e4b8c4109af9955c4f08c9095 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:02.482973 16190 leader_election.cc:304] T 19baa7bc367844c288bca8f81439c413 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 647a343a542d4a139fe106c19095129b
I20260709 13:22:02.472965 22748 raft_consensus.cc:2749] T 8c11074579b046d99e585c2818e18a2c P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:22:02.489602 23000 raft_consensus.cc:2804] T 19baa7bc367844c288bca8f81439c413 P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:22:02.491884 23000 raft_consensus.cc:493] T 19baa7bc367844c288bca8f81439c413 P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:02.492216 23000 raft_consensus.cc:3060] T 19baa7bc367844c288bca8f81439c413 P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:22:02.487237 22979 consensus_queue.cc:1059] T 8cc959fb7f704fba9009c0595467d074 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: INVALID_TERM, Last received: 15.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.006s
I20260709 13:22:02.493567 22979 raft_consensus.cc:3055] T 8cc959fb7f704fba9009c0595467d074 P 1ca2725d290e47d68b1a4b609c9816ac [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:22:02.493886 22979 raft_consensus.cc:740] T 8cc959fb7f704fba9009c0595467d074 P 1ca2725d290e47d68b1a4b609c9816ac [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:22:02.476961 23059 raft_consensus.cc:2749] T 5a96373bb0e14dce8a266b2fc6c58507 P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:22:02.496677 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
W20260709 13:22:02.489972 16145 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46224 (request call id 1128) took 2931 ms (client timeout 1887 ms). Trace:
I20260709 13:22:02.498775 22494 raft_consensus.cc:493] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
W20260709 13:22:02.498893 16145 rpcz_store.cc:269] 0709 13:21:59.553749 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:59.553909 (+   160us) service_pool.cc:224] Handling call
0709 13:22:02.484870 (+2930961us) spinlock_profiling.cc:243] Waited 1.75 s on lock 0x7b54000730d4. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:22:02.485020 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4015292160}
I20260709 13:22:02.499384 22494 raft_consensus.cc:515] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:02.501379 22494 leader_election.cc:290] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:02.504036 22954 leader_election.cc:290] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:02.507437 22938 raft_consensus.cc:3060] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:22:02.512066 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2794af5b99984efebdee564035841cb0" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:22:02.512703 16293 raft_consensus.cc:3060] T 2794af5b99984efebdee564035841cb0 P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:22:02.497370 16217 raft_consensus.cc:2468] T ecc1e363230043fe9fef390f0b979f20 P 1ca2725d290e47d68b1a4b609c9816ac [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 20.
I20260709 13:22:02.480871 22971 raft_consensus.cc:2749] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:22:02.518252 16191 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:22:02.488662 16218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c11074579b046d99e585c2818e18a2c" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:02.514447 16190 leader_election.cc:304] T 08c74490c40243a88e62d1760b5cd814 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 647a343a542d4a139fe106c19095129b
I20260709 13:22:02.469240 23003 raft_consensus.cc:515] T 8d4ab870a05347868956aa8b841b2981 P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:02.520540 16296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f82224f4564fea93572e8206bad1fe" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:22:02.522572 16296 raft_consensus.cc:3060] T 23f82224f4564fea93572e8206bad1fe P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:22:02.522874 16116 leader_election.cc:304] T 18ad7e3af5504571a69d10c00750a01c P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:22:02.524650 16116 leader_election.cc:304] T b6f07842d3aa47858df5665874c86f3a P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:22:02.526491 16116 leader_election.cc:304] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
I20260709 13:22:02.479360 22782 raft_consensus.cc:2804] T fffab26427e24d398e2695aad40ca134 P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:22:02.553745 22782 raft_consensus.cc:493] T fffab26427e24d398e2695aad40ca134 P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:02.554226 22782 raft_consensus.cc:3060] T fffab26427e24d398e2695aad40ca134 P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:22:02.554762 22935 raft_consensus.cc:2804] T 08c74490c40243a88e62d1760b5cd814 P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:22:02.555339 22935 raft_consensus.cc:697] T 08c74490c40243a88e62d1760b5cd814 P 1ca2725d290e47d68b1a4b609c9816ac [term 17 LEADER]: Becoming Leader. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:22:02.555565 22677 raft_consensus.cc:493] T d16dfad0c3b540e5b4b86492ddf87208 P 1ca2725d290e47d68b1a4b609c9816ac [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:02.556981 16218 raft_consensus.cc:2410] T 8c11074579b046d99e585c2818e18a2c P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 for term 20 because replica has last-logged OpId of term: 19 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 14.
I20260709 13:22:02.534950 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68a6838d82ef4cac83507f906139d90f" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:02.535238 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31969392ccdb43b491911a319e8bc7e1" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:02.482347 23077 raft_consensus.cc:3060] T ada5a004ba7c46819051e3ccec71ffdb P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:22:02.534669 16292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a96373bb0e14dce8a266b2fc6c58507" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:22:02.535856 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68a6838d82ef4cac83507f906139d90f" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:02.534296 16294 raft_consensus.cc:1275] T c4f65cdf00b748bb963ee6e81ef43b79 P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
W20260709 13:22:02.560636 16115 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:22:02.560807 16296 raft_consensus.cc:2468] T 23f82224f4564fea93572e8206bad1fe P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 17.
I20260709 13:22:02.561378 16224 raft_consensus.cc:2468] T d2ab0ac8347641e6955e9bce7d90c1a2 P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 16.
I20260709 13:22:02.566717 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a96373bb0e14dce8a266b2fc6c58507" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:22:02.567320 16224 raft_consensus.cc:3060] T 5a96373bb0e14dce8a266b2fc6c58507 P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:22:02.531903 22917 raft_consensus.cc:3060] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [term 24 FOLLOWER]: Advancing to term 25
W20260709 13:22:02.515027 23075 log.cc:927] Time spent T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860: Append to log took a long time: real 0.060s	user 0.001s	sys 0.000s
I20260709 13:22:02.546307 16824 raft_consensus.cc:2468] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 20.
I20260709 13:22:02.556209 22731 raft_consensus.cc:493] T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
W20260709 13:22:02.583359 16115 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:22:02.573716 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 47335.5us
I20260709 13:22:02.584960 16115 leader_election.cc:304] T fffab26427e24d398e2695aad40ca134 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:22:02.578900 23015 raft_consensus.cc:2804] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:22:02.587018 23015 raft_consensus.cc:493] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Starting leader election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:22:02.587589 23015 raft_consensus.cc:3060] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:22:02.589579 22916 raft_consensus.cc:515] T 8a878999c0024d2abe517484180290fc P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:02.592145 23037 raft_consensus.cc:2804] T fffab26427e24d398e2695aad40ca134 P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:22:02.592741 23037 raft_consensus.cc:493] T fffab26427e24d398e2695aad40ca134 P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:02.593155 23037 raft_consensus.cc:3060] T fffab26427e24d398e2695aad40ca134 P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:22:02.593704 16297 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48674 (request call id 3345) took 1365 ms. Trace:
I20260709 13:22:02.594092 16297 rpcz_store.cc:276] 0709 13:22:01.228217 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:22:01.228349 (+   132us) service_pool.cc:224] Handling call
0709 13:22:01.228670 (+   321us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:22:01.336029 (+107359us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:22:02.593460 (+1257431us) spinlock_profiling.cc:243] Waited 69.6 ms on lock 0x7b54001ef354. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:22:02.593649 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":160060160}
I20260709 13:22:02.556233 22935 consensus_queue.cc:237] T 08c74490c40243a88e62d1760b5cd814 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.10, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:02.537942 22754 raft_consensus.cc:2749] T fffab26427e24d398e2695aad40ca134 P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:22:02.490012 22709 raft_consensus.cc:697] T 5a959dbecc2648eb85d1cc550f5a3245 P 90f3da57796b488c9087a2f523adf045 [term 18 LEADER]: Becoming Leader. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:22:02.556134 22677 raft_consensus.cc:515] T d16dfad0c3b540e5b4b86492ddf87208 P 1ca2725d290e47d68b1a4b609c9816ac [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:02.535362 23036 raft_consensus.cc:2749] T 18ad7e3af5504571a69d10c00750a01c P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:22:02.585956 16116 leader_election.cc:304] T 08c74490c40243a88e62d1760b5cd814 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:22:02.547868 23003 leader_election.cc:290] T 8d4ab870a05347868956aa8b841b2981 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:02.611768 16116 leader_election.cc:304] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
I20260709 13:22:02.606922 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e65676b21b748728ad39668cfe722ec" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:22:02.576195 16293 raft_consensus.cc:2468] T 2794af5b99984efebdee564035841cb0 P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 16.
I20260709 13:22:02.565505 22494 raft_consensus.cc:2749] T b6f07842d3aa47858df5665874c86f3a P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:22:02.569828 22996 raft_consensus.cc:493] T 57a744f5aaf944678a63469cbf514015 P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:02.562426 23023 raft_consensus.cc:493] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:22:02.578117 22921 raft_consensus.cc:493] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:02.609072 16224 raft_consensus.cc:2468] T 5a96373bb0e14dce8a266b2fc6c58507 P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 16.
I20260709 13:22:02.551020 22864 raft_consensus.cc:515] T 7c2095a5f4ad49408dbc7f69c606120d P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:02.620469 22996 raft_consensus.cc:515] T 57a744f5aaf944678a63469cbf514015 P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:02.622282 23036 raft_consensus.cc:2804] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:22:02.629980 23036 raft_consensus.cc:493] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Starting leader election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:22:02.630438 23036 raft_consensus.cc:3060] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:22:02.631263 23023 raft_consensus.cc:515] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:02.634263 23023 leader_election.cc:290] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:02.620448 22940 leader_election.cc:290] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 20 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:02.584520 22731 raft_consensus.cc:515] T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:02.547708 22979 consensus_queue.cc:260] T 8cc959fb7f704fba9009c0595467d074 P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:02.636483 22979 raft_consensus.cc:3060] T 8cc959fb7f704fba9009c0595467d074 P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:22:02.637079 22982 leader_election.cc:290] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
W20260709 13:22:02.638582 23081 log.cc:927] Time spent T 8638ee363273426db1393dd81e5c7b72 P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 13:22:02.617987 23003 raft_consensus.cc:3060] T 08c74490c40243a88e62d1760b5cd814 P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:22:02.624944 22921 raft_consensus.cc:515] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:02.646559 23000 raft_consensus.cc:515] T 19baa7bc367844c288bca8f81439c413 P 1ca2725d290e47d68b1a4b609c9816ac [term 22 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:02.646662 22709 consensus_queue.cc:237] T 5a959dbecc2648eb85d1cc550f5a3245 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:02.648552 22921 leader_election.cc:290] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:02.629392 22864 leader_election.cc:290] T 7c2095a5f4ad49408dbc7f69c606120d P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 16 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:22:02.657717 22782 raft_consensus.cc:515] T fffab26427e24d398e2695aad40ca134 P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:02.663552 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8638ee363273426db1393dd81e5c7b72" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:02.666128 16224 raft_consensus.cc:2410] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 for term 20 because replica has last-logged OpId of term: 19 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 14.
I20260709 13:22:02.662983 23003 raft_consensus.cc:2749] T 08c74490c40243a88e62d1760b5cd814 P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:22:02.671615 16224 raft_consensus.cc:1240] T c2f5f89186204242bf377f7c42b2e2da P 1ca2725d290e47d68b1a4b609c9816ac [term 20 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 19. Current term is 20. Ops: [19.12-19.12]
I20260709 13:22:02.681411 22962 leader_election.cc:290] T e4dc6b0541ca431ebc757beac00521f1 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:02.680703 16042 catalog_manager.cc:5697] T 57a744f5aaf944678a63469cbf514015 P 647a343a542d4a139fe106c19095129b reported cstate change: term changed from 11 to 12, leader changed from cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) to 647a343a542d4a139fe106c19095129b (127.14.177.129). New cstate: current_term: 12 leader_uuid: "647a343a542d4a139fe106c19095129b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:02.682880 16042 catalog_manager.cc:5697] T 8158bbd17b994d2eb96f560bd0f1b88d P 647a343a542d4a139fe106c19095129b reported cstate change: term changed from 13 to 14, leader changed from 90f3da57796b488c9087a2f523adf045 (127.14.177.131) to 647a343a542d4a139fe106c19095129b (127.14.177.129). New cstate: current_term: 14 leader_uuid: "647a343a542d4a139fe106c19095129b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } health_report { overall_health: HEALTHY } } }
I20260709 13:22:02.685607 16222 raft_consensus.cc:1275] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
I20260709 13:22:02.685123 16042 catalog_manager.cc:5697] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b reported cstate change: term changed from 14 to 17. New cstate: current_term: 17 leader_uuid: "647a343a542d4a139fe106c19095129b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } health_report { overall_health: HEALTHY } } }
I20260709 13:22:02.687359 16042 catalog_manager.cc:5697] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 647a343a542d4a139fe106c19095129b reported cstate change: term changed from 14 to 16. New cstate: current_term: 16 leader_uuid: "647a343a542d4a139fe106c19095129b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } health_report { overall_health: HEALTHY } } }
I20260709 13:22:02.690268 16224 raft_consensus.cc:1240] T 8a878999c0024d2abe517484180290fc P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 16. Current term is 17. Ops: [16.9-16.9]
I20260709 13:22:02.657758 23077 raft_consensus.cc:515] T ada5a004ba7c46819051e3ccec71ffdb P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:02.691922 16224 raft_consensus.cc:1275] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 18 index: 11. (term mismatch)
I20260709 13:22:02.692301 16224 pending_rounds.cc:85] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 1ca2725d290e47d68b1a4b609c9816ac: Aborting all ops after (but not including) 10
I20260709 13:22:02.692580 16224 pending_rounds.cc:107] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 1ca2725d290e47d68b1a4b609c9816ac: Aborting uncommitted NO_OP operation due to leader change: 17.11
I20260709 13:22:02.692900 16224 raft_consensus.cc:2889] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:22:02.674685 23000 leader_election.cc:290] T 19baa7bc367844c288bca8f81439c413 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 22 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:02.697328 16117 leader_election.cc:304] T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:22:02.699024 16224 raft_consensus.cc:1275] T e5ad4eb281564936a59f41e387a47156 P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 13:22:02.694181 23003 raft_consensus.cc:493] T 8158bbd17b994d2eb96f560bd0f1b88d P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:22:02.703024 16224 raft_consensus.cc:1275] T c2f5f89186204242bf377f7c42b2e2da P 1ca2725d290e47d68b1a4b609c9816ac [term 20 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 13:22:02.695386 22916 leader_election.cc:290] T 8a878999c0024d2abe517484180290fc P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 17 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:02.699766 23020 raft_consensus.cc:515] T 5a96373bb0e14dce8a266b2fc6c58507 P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:02.705266 16224 raft_consensus.cc:1240] T 5a959dbecc2648eb85d1cc550f5a3245 P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 17. Current term is 18. Ops: [17.11-17.11]
I20260709 13:22:02.706056 23020 leader_election.cc:290] T 5a96373bb0e14dce8a266b2fc6c58507 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 16 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
W20260709 13:22:02.702558 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 188032us
I20260709 13:22:02.707037 16292 raft_consensus.cc:2393] T 5a96373bb0e14dce8a266b2fc6c58507 P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 16: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
W20260709 13:22:02.711225 16115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 126386us
I20260709 13:22:02.697299 22996 leader_election.cc:290] T 57a744f5aaf944678a63469cbf514015 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:02.721577 16117 leader_election.cc:304] T af57996a8be84d398f0f023e8d08ef77 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860
I20260709 13:22:02.719537 16222 raft_consensus.cc:1240] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 18. Current term is 19. Ops: []
I20260709 13:22:02.708781 22782 leader_election.cc:290] T fffab26427e24d398e2695aad40ca134 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 19 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:22:02.725004 16145 raft_consensus.cc:1240] T 2794af5b99984efebdee564035841cb0 P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 15. Current term is 16. Ops: []
I20260709 13:22:02.725519 16143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:22:02.726374 16143 raft_consensus.cc:2468] T ecc1e363230043fe9fef390f0b979f20 P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 20.
I20260709 13:22:02.702015 22494 raft_consensus.cc:2804] T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:22:02.732488 16222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e4bfb9090c8460eb14fa7e0d818e7f1" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:22:02.732928 16143 raft_consensus.cc:1275] T 57a744f5aaf944678a63469cbf514015 P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 13:22:02.735101 16222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:02.736176 16222 raft_consensus.cc:2468] T ecc1e363230043fe9fef390f0b979f20 P 1ca2725d290e47d68b1a4b609c9816ac [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 20.
W20260709 13:22:02.729880 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 260481us
I20260709 13:22:02.716501 22740 raft_consensus.cc:493] T 19baa7bc367844c288bca8f81439c413 P cb82cc7bd6db46d980c75237a1402860 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:02.723917 22196 leader_election.cc:290] T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:02.745232 16222 raft_consensus.cc:1240] T c0eab25ce4c64e018a041beeda2867db P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 18. Current term is 19. Ops: []
I20260709 13:22:02.750437 16222 raft_consensus.cc:1240] T 19baa7bc367844c288bca8f81439c413 P 1ca2725d290e47d68b1a4b609c9816ac [term 22 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 21. Current term is 22. Ops: []
I20260709 13:22:02.755890 16790 leader_election.cc:304] T 8c11074579b046d99e585c2818e18a2c P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:22:02.748946 23023 raft_consensus.cc:2749] T af57996a8be84d398f0f023e8d08ef77 P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:22:02.732232 22494 raft_consensus.cc:493] T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Starting leader election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:22:02.758352 16790 leader_election.cc:304] T 8638ee363273426db1393dd81e5c7b72 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:22:02.710517 23003 raft_consensus.cc:515] T 8158bbd17b994d2eb96f560bd0f1b88d P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:02.758749 22508 raft_consensus.cc:2749] T 8c11074579b046d99e585c2818e18a2c P cb82cc7bd6db46d980c75237a1402860 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:22:02.765723 16115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 45244.4us
W20260709 13:22:02.767884 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 156187us
I20260709 13:22:02.768661 16116 leader_election.cc:304] T 02838c86531d4f98bf58d1c22b8baf77 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:22:02.756075 22782 raft_consensus.cc:493] T 57a744f5aaf944678a63469cbf514015 P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:22:02.777211 16191 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:22:02.777704 22672 consensus_queue.cc:1048] T c9944d5e4b8c4109af9955c4f08c9095 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.002s
W20260709 13:22:02.781208 16043 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:49252 (request call id 290) took 46567 ms (client timeout 15000 ms). Trace:
I20260709 13:22:02.785923 16042 catalog_manager.cc:5697] T 8a878999c0024d2abe517484180290fc P 647a343a542d4a139fe106c19095129b reported cstate change: term changed from 12 to 14. New cstate: current_term: 14 leader_uuid: "647a343a542d4a139fe106c19095129b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } health_report { overall_health: HEALTHY } } }
I20260709 13:22:02.787999 16042 catalog_manager.cc:5697] T 18ad7e3af5504571a69d10c00750a01c P 647a343a542d4a139fe106c19095129b reported cstate change: term changed from 11 to 13, leader changed from 90f3da57796b488c9087a2f523adf045 (127.14.177.131) to 647a343a542d4a139fe106c19095129b (127.14.177.129). New cstate: current_term: 13 leader_uuid: "647a343a542d4a139fe106c19095129b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: HEALTHY } } }
I20260709 13:22:02.791710 23016 leader_election.cc:290] T 8cc959fb7f704fba9009c0595467d074 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
W20260709 13:22:02.781795 16043 rpcz_store.cc:269] 0709 13:21:16.214104 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:16.271641 (+ 57537us) service_pool.cc:224] Handling call
0709 13:21:16.979238 (+707597us) spinlock_profiling.cc:243] Waited 21.9 ms on lock 0x7b54003cdf54. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac9090 00007f69b0b67f3a 00007f69b691e1cc 00007f69b6a1c37e 00007f69b4edd2a4 00007f69b4edd232 00007f69b4edd161 00007f69b4edd0dc 00007f69b4edc392 00007f69aee9a48c 00007f69aee998c6
0709 13:22:02.780940 (+45801702us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:21:54.978508 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:21:54.979454 (+   946us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:21:54.979513 (+    59us) write_op.cc:435] Acquired schema lock
0709 13:21:54.979535 (+    22us) tablet.cc:662] Decoding operations
0709 13:21:54.981108 (+  1573us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:21:54.981212 (+   104us) write_op.cc:641] Partition lock acquired for write op
0709 13:21:54.981231 (+    19us) tablet.cc:685] Acquiring locks for 17 operations
0709 13:21:54.982151 (+   920us) tablet.cc:701] Row locks acquired
0709 13:21:54.982193 (+    42us) write_op.cc:260] PREPARE: finished
0709 13:21:54.982341 (+   148us) write_op.cc:270] Start()
0709 13:21:54.982460 (+   119us) write_op.cc:276] Timestamp: P: 1783603314982291 usec, L: 0
0709 13:21:54.982474 (+    14us) op_driver.cc:348] REPLICATION: starting
0709 13:21:54.983199 (+   725us) log.cc:844] Serialized 6796 byte log entry
0709 13:21:54.992824 (+  9625us) op_driver.cc:464] REPLICATION: finished
0709 13:21:54.994419 (+  1595us) write_op.cc:301] APPLY: starting
0709 13:21:54.994521 (+   102us) tablet.cc:1367] starting BulkCheckPresence
0709 13:21:55.004391 (+  9870us) tablet.cc:1370] finished BulkCheckPresence
0709 13:21:55.004523 (+   132us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:21:55.028429 (+ 23906us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:21:55.028876 (+   447us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=34,key_file_lookups=34,delta_file_lookups=0,mrs_lookups=0
0709 13:21:55.028944 (+    68us) write_op.cc:312] APPLY: finished
0709 13:21:55.029476 (+   532us) log.cc:844] Serialized 156 byte log entry
0709 13:21:55.029671 (+   195us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:21:55.029832 (+   161us) write_op.cc:454] Released schema lock
0709 13:21:55.029882 (+    50us) write_op.cc:341] FINISH: Updating metrics
0709 13:21:55.030323 (+   441us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:21:55.030369 (+    46us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":1150761,"reported_tablets":49,"spinlock_wait_cycles":50375424,"child_traces":[["op",{"apply.queue_time_us":1302,"apply.run_cpu_time_us":23832,"apply.run_wall_time_us":35883,"cfile_cache_hit":70,"cfile_cache_hit_bytes":75119,"num_ops":17,"prepare.queue_time_us":35694,"prepare.run_cpu_time_us":8732,"prepare.run_wall_time_us":8892,"raft.queue_time_us":2126,"raft.run_cpu_time_us":1626,"raft.run_wall_time_us":1626,"replication_time_us":10177,"spinlock_wait_cycles":20352,"thread_start_us":41094,"threads_started":4,"wal-append.queue_time_us":3982,"wal-append.run_cpu_time_us":5678,"wal-append.run_wall_time_us":1043025}]]}
I20260709 13:22:02.800078 23077 leader_election.cc:290] T ada5a004ba7c46819051e3ccec71ffdb P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 20 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:02.790202 16042 catalog_manager.cc:5697] T c0eab25ce4c64e018a041beeda2867db P 647a343a542d4a139fe106c19095129b reported cstate change: term changed from 12 to 16, leader changed from 90f3da57796b488c9087a2f523adf045 (127.14.177.131) to 647a343a542d4a139fe106c19095129b (127.14.177.129). New cstate: current_term: 16 leader_uuid: "647a343a542d4a139fe106c19095129b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:02.802716 23086 consensus_queue.cc:1048] T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:22:02.805361 23086 consensus_queue.cc:1048] T e5ad4eb281564936a59f41e387a47156 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:22:02.806295 23089 raft_consensus.cc:2749] T 02838c86531d4f98bf58d1c22b8baf77 P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:22:02.806630 16116 leader_election.cc:304] T 8cc959fb7f704fba9009c0595467d074 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:22:02.808117 16116 leader_election.cc:304] T 1df93779209748b9860f6d893dd3de53 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:22:02.764840 22494 raft_consensus.cc:3060] T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:22:02.782996 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 44320us
I20260709 13:22:02.833758 16143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0eab25ce4c64e018a041beeda2867db" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:22:02.834872 16148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "829b4ad8c5a54eababeea1181b960d4b" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:22:02.835078 16145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806d7690c2b644959152074bb2cab0a7" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
W20260709 13:22:02.815997 16266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 53220.6us
I20260709 13:22:02.804531 16042 catalog_manager.cc:5697] T 84f2da98864b4425bf7c0055951fdd4b P 647a343a542d4a139fe106c19095129b reported cstate change: term changed from 9 to 12, leader changed from 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130) to 90f3da57796b488c9087a2f523adf045 (127.14.177.131). New cstate: current_term: 12 leader_uuid: "90f3da57796b488c9087a2f523adf045" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } }
I20260709 13:22:02.837915 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4f65cdf00b748bb963ee6e81ef43b79" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:22:02.838232 23089 raft_consensus.cc:2749] T 1df93779209748b9860f6d893dd3de53 P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:22:02.823709 16116 leader_election.cc:304] T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
I20260709 13:22:02.839416 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87ac0d1f7e8a419f9bcbc04aee0e067e" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:22:02.838734 16042 catalog_manager.cc:5697] T d16dfad0c3b540e5b4b86492ddf87208 P 647a343a542d4a139fe106c19095129b reported cstate change: term changed from 12 to 15, leader changed from 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130) to 647a343a542d4a139fe106c19095129b (127.14.177.129). New cstate: current_term: 15 leader_uuid: "647a343a542d4a139fe106c19095129b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } health_report { overall_health: HEALTHY } } }
I20260709 13:22:02.840060 16149 raft_consensus.cc:3060] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:22:02.827971 16224 raft_consensus.cc:1275] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac [term 20 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 20 index: 13. (term mismatch)
I20260709 13:22:02.840934 16224 pending_rounds.cc:85] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac: Aborting all ops after (but not including) 12
I20260709 13:22:02.841229 16224 pending_rounds.cc:107] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac: Aborting uncommitted NO_OP operation due to leader change: 19.13
I20260709 13:22:02.841698 16224 raft_consensus.cc:2889] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:22:02.842033 23003 leader_election.cc:290] T 8158bbd17b994d2eb96f560bd0f1b88d P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:02.836263 16142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66f75124215f4ca6bef831e174a792d8" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:22:02.842693 23003 raft_consensus.cc:2804] T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:22:02.841177 16042 catalog_manager.cc:5697] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b reported cstate change: term changed from 12 to 14, leader changed from 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130) to 647a343a542d4a139fe106c19095129b (127.14.177.129). New cstate: current_term: 14 leader_uuid: "647a343a542d4a139fe106c19095129b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:02.838368 16151 raft_consensus.cc:1240] T 746aabf856ab401c9ef5e859371e7b67 P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 15. Current term is 16. Ops: []
I20260709 13:22:02.822041 23087 raft_consensus.cc:3060] T 8cc959fb7f704fba9009c0595467d074 P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:22:02.836704 16148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5d59407eb545b895d8d922e0dd69fd" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:22:02.758404 23036 raft_consensus.cc:515] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:02.844918 16265 leader_election.cc:304] T e5ad4eb281564936a59f41e387a47156 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
W20260709 13:22:02.845265 16117 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 123752us
I20260709 13:22:02.766037 22885 raft_consensus.cc:2749] T 8638ee363273426db1393dd81e5c7b72 P cb82cc7bd6db46d980c75237a1402860 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:22:02.840060 16116 leader_election.cc:304] T 829b4ad8c5a54eababeea1181b960d4b P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: cb82cc7bd6db46d980c75237a1402860
I20260709 13:22:02.850777 16117 leader_election.cc:304] T 26354361e9b044cea1c2b86c4882fe92 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860; no voters: 90f3da57796b488c9087a2f523adf045
I20260709 13:22:02.845808 16142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ad5dbfa99ee4b8d9e28be92b3f36db5" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
W20260709 13:22:02.846383 16266 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:22:02.746826 22740 raft_consensus.cc:515] T 19baa7bc367844c288bca8f81439c413 P cb82cc7bd6db46d980c75237a1402860 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:02.784667 22782 raft_consensus.cc:515] T 57a744f5aaf944678a63469cbf514015 P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:02.856079 16142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f2da98864b4425bf7c0055951fdd4b" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:22:02.857064 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4dc6b0541ca431ebc757beac00521f1" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 20 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:22:02.857419 22864 raft_consensus.cc:2804] T e5ad4eb281564936a59f41e387a47156 P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:22:02.857913 22864 raft_consensus.cc:493] T e5ad4eb281564936a59f41e387a47156 P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:02.857945 16294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806d7690c2b644959152074bb2cab0a7" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:22:02.858402 22864 raft_consensus.cc:3060] T e5ad4eb281564936a59f41e387a47156 P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:22:02.859581 16294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4f65cdf00b748bb963ee6e81ef43b79" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:02.860285 16294 raft_consensus.cc:2468] T c4f65cdf00b748bb963ee6e81ef43b79 P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 20.
I20260709 13:22:02.861238 16294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7ca61592e848ea9b0fb81a2ce33cdc" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:02.850950 16265 leader_election.cc:304] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:22:02.863509 16148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c61a39fd4f84424b129c1a6f9ba404d" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:22:02.864162 16148 raft_consensus.cc:2468] T 9c61a39fd4f84424b129c1a6f9ba404d P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 20.
I20260709 13:22:02.865139 16145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:22:02.865393 23015 raft_consensus.cc:515] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:02.867460 22439 raft_consensus.cc:2749] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:22:02.868105 16151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d16dfad0c3b540e5b4b86492ddf87208" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 21 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:22:02.869108 22439 consensus_queue.cc:1048] T 18ad7e3af5504571a69d10c00750a01c P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:22:02.869745 22885 raft_consensus.cc:493] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:02.870281 22885 raft_consensus.cc:515] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:02.871961 16191 leader_election.cc:304] T 829b4ad8c5a54eababeea1181b960d4b P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
I20260709 13:22:02.872016 16116 leader_election.cc:304] T ecc1e363230043fe9fef390f0b979f20 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:22:02.873131 16191 leader_election.cc:304] T 66f75124215f4ca6bef831e174a792d8 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
I20260709 13:22:02.843123 23003 raft_consensus.cc:493] T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Starting leader election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:22:02.876736 16291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74ca8140f35c4ae29e8a664f6bdfc829" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:02.877382 16294 raft_consensus.cc:2468] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 19.
I20260709 13:22:02.853401 16149 raft_consensus.cc:2468] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 20.
I20260709 13:22:02.853935 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87ac0d1f7e8a419f9bcbc04aee0e067e" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:22:02.878582 16293 raft_consensus.cc:2410] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac for term 20 because replica has last-logged OpId of term: 19 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 12.
I20260709 13:22:02.854285 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0eab25ce4c64e018a041beeda2867db" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:22:02.875667 16191 leader_election.cc:304] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
I20260709 13:22:02.848721 22645 raft_consensus.cc:493] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:02.880664 16117 leader_election.cc:304] T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860
I20260709 13:22:02.880793 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4dc6b0541ca431ebc757beac00521f1" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 20 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:02.881378 16191 leader_election.cc:304] T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
I20260709 13:22:02.883858 16191 leader_election.cc:304] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
I20260709 13:22:02.885798 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ad5dbfa99ee4b8d9e28be92b3f36db5" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:02.848979 16224 raft_consensus.cc:1275] T d16dfad0c3b540e5b4b86492ddf87208 P 1ca2725d290e47d68b1a4b609c9816ac [term 20 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 13:22:02.889297 16291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d4ab870a05347868956aa8b841b2981" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:02.890060 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f5f89186204242bf377f7c42b2e2da" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:02.891149 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57a744f5aaf944678a63469cbf514015" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:02.891803 16293 raft_consensus.cc:2468] T 57a744f5aaf944678a63469cbf514015 P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 15.
I20260709 13:22:02.892473 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:22:02.858867 23037 raft_consensus.cc:515] T fffab26427e24d398e2695aad40ca134 P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:02.893623 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a878999c0024d2abe517484180290fc" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:22:02.895292 16293 raft_consensus.cc:1275] T 434bff3082d141838068f323bdb0d1ed P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 16 index: 14. Preceding OpId from leader: term: 18 index: 15. (index mismatch)
I20260709 13:22:02.899783 23015 leader_election.cc:290] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 19 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:02.886860 22610 raft_consensus.cc:2804] T 26354361e9b044cea1c2b86c4882fe92 P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:22:02.900707 22610 raft_consensus.cc:697] T 26354361e9b044cea1c2b86c4882fe92 P 647a343a542d4a139fe106c19095129b [term 16 LEADER]: Becoming Leader. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:22:02.901795 22610 consensus_queue.cc:237] T 26354361e9b044cea1c2b86c4882fe92 P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:02.902561 23083 consensus_queue.cc:1048] T 8638ee363273426db1393dd81e5c7b72 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:22:02.907593 16294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8158bbd17b994d2eb96f560bd0f1b88d" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:02.880004 22979 raft_consensus.cc:2749] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:22:02.876652 23000 raft_consensus.cc:2749] T 829b4ad8c5a54eababeea1181b960d4b P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:22:02.917174 16147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f82224f4564fea93572e8206bad1fe" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:22:02.904785 16266 leader_election.cc:400] T d2ab0ac8347641e6955e9bce7d90c1a2 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 15 pre-election: Vote denied by peer cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923) with higher term. Message: Invalid argument: T d2ab0ac8347641e6955e9bce7d90c1a2 P cb82cc7bd6db46d980c75237a1402860 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 for earlier term 15. Current term is 16.
I20260709 13:22:02.904641 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a878999c0024d2abe517484180290fc" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:22:02.886814 23096 raft_consensus.cc:2804] T 829b4ad8c5a54eababeea1181b960d4b P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:22:02.927819 23096 raft_consensus.cc:493] T 829b4ad8c5a54eababeea1181b960d4b P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Starting leader election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:22:02.919139 16042 catalog_manager.cc:5697] T 08c74490c40243a88e62d1760b5cd814 P 647a343a542d4a139fe106c19095129b reported cstate change: term changed from 13 to 14, leader changed from 90f3da57796b488c9087a2f523adf045 (127.14.177.131) to 647a343a542d4a139fe106c19095129b (127.14.177.129). New cstate: current_term: 14 leader_uuid: "647a343a542d4a139fe106c19095129b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } health_report { overall_health: HEALTHY } } }
I20260709 13:22:02.928428 22885 leader_election.cc:290] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:22:02.887292 22645 raft_consensus.cc:515] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:02.919456 16147 raft_consensus.cc:2393] T 23f82224f4564fea93572e8206bad1fe P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 17: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
I20260709 13:22:02.920725 23087 raft_consensus.cc:2749] T 8cc959fb7f704fba9009c0595467d074 P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:22:02.930728 16147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a959dbecc2648eb85d1cc550f5a3245" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:22:02.930999 16143 raft_consensus.cc:1240] T 8cc959fb7f704fba9009c0595467d074 P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 16. Current term is 17. Ops: []
I20260709 13:22:02.931156 22645 leader_election.cc:290] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:22:02.931563 16147 raft_consensus.cc:2410] T 5a959dbecc2648eb85d1cc550f5a3245 P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 for term 18 because replica has last-logged OpId of term: 17 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 10.
I20260709 13:22:02.880261 22962 consensus_queue.cc:1059] T 746aabf856ab401c9ef5e859371e7b67 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: INVALID_TERM, Last received: 15.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.023s
I20260709 13:22:02.932587 22962 raft_consensus.cc:3055] T 746aabf856ab401c9ef5e859371e7b67 P 1ca2725d290e47d68b1a4b609c9816ac [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:22:02.932933 22962 raft_consensus.cc:740] T 746aabf856ab401c9ef5e859371e7b67 P 1ca2725d290e47d68b1a4b609c9816ac [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:22:02.907593 16220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5718c623ebee4d2bb770f0f9ddb2f164" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:22:02.933619 23037 leader_election.cc:290] T fffab26427e24d398e2695aad40ca134 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 19 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:02.933802 22962 consensus_queue.cc:260] T 746aabf856ab401c9ef5e859371e7b67 P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:02.934424 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0eab25ce4c64e018a041beeda2867db" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:22:02.910648 16151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a96373bb0e14dce8a266b2fc6c58507" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:22:02.935652 16151 raft_consensus.cc:2393] T 5a96373bb0e14dce8a266b2fc6c58507 P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 16: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
I20260709 13:22:02.936657 16151 raft_consensus.cc:1240] T af57996a8be84d398f0f023e8d08ef77 P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 18. Current term is 19. Ops: []
I20260709 13:22:02.911767 16145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c2095a5f4ad49408dbc7f69c606120d" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:22:02.912307 16148 raft_consensus.cc:1240] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 19. Current term is 20. Ops: [17.12-19.13]
I20260709 13:22:02.887287 22917 raft_consensus.cc:2804] T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:22:02.889168 23061 raft_consensus.cc:2804] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 1ca2725d290e47d68b1a4b609c9816ac [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:22:02.916544 22782 leader_election.cc:290] T 57a744f5aaf944678a63469cbf514015 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:02.917166 16144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4f65cdf00b748bb963ee6e81ef43b79" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:22:02.903795 23077 consensus_queue.cc:1048] T 8c11074579b046d99e585c2818e18a2c P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.014s
I20260709 13:22:02.917671 23090 consensus_queue.cc:1048] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:22:02.928272 23096 raft_consensus.cc:3060] T 829b4ad8c5a54eababeea1181b960d4b P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:22:02.935287 16266 leader_election.cc:403] T d2ab0ac8347641e6955e9bce7d90c1a2 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 15 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:22:02.881871 22935 raft_consensus.cc:2749] T 66f75124215f4ca6bef831e174a792d8 P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:22:02.938247 16220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8158bbd17b994d2eb96f560bd0f1b88d" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:02.970391 16294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fffab26427e24d398e2695aad40ca134" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:22:02.935685 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c61a39fd4f84424b129c1a6f9ba404d" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 21 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:22:02.939746 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d4ab870a05347868956aa8b841b2981" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:02.971894 16224 raft_consensus.cc:2468] T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 17.
I20260709 13:22:02.888727 22735 raft_consensus.cc:2804] T ecc1e363230043fe9fef390f0b979f20 P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:22:02.945207 16148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2794af5b99984efebdee564035841cb0" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:22:02.946492 16145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434bff3082d141838068f323bdb0d1ed" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
W20260709 13:22:02.973384 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 88529.1us
I20260709 13:22:02.935071 22962 raft_consensus.cc:3060] T 746aabf856ab401c9ef5e859371e7b67 P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:22:02.969116 16266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 64468.3us
I20260709 13:22:02.914709 23089 raft_consensus.cc:2749] T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:22:02.972134 16149 raft_consensus.cc:2410] T 9c61a39fd4f84424b129c1a6f9ba404d P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 for term 21 because replica has last-logged OpId of term: 19 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 9.
I20260709 13:22:02.880059 23003 raft_consensus.cc:3060] T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:22:02.973001 16220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada5a004ba7c46819051e3ccec71ffdb" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:22:02.989058 16218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f5f89186204242bf377f7c42b2e2da" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:02.989696 16220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a96373bb0e14dce8a266b2fc6c58507" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:22:02.990537 16220 raft_consensus.cc:2393] T 5a96373bb0e14dce8a266b2fc6c58507 P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 16: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
I20260709 13:22:02.990960 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74ca8140f35c4ae29e8a664f6bdfc829" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:22:02.991329 16220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fffab26427e24d398e2695aad40ca134" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:22:02.991981 16220 raft_consensus.cc:3060] T fffab26427e24d398e2695aad40ca134 P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:22:02.971415 23061 raft_consensus.cc:697] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 1ca2725d290e47d68b1a4b609c9816ac [term 20 LEADER]: Becoming Leader. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:22:02.974016 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c2095a5f4ad49408dbc7f69c606120d" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:22:02.951628 16151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5ad4eb281564936a59f41e387a47156" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:22:02.996055 16224 raft_consensus.cc:3060] T 7c2095a5f4ad49408dbc7f69c606120d P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:22:02.975934 16144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a9de347c711477e87c582d003f36925" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:22:02.976854 16147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cc959fb7f704fba9009c0595467d074" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:22:02.950867 22917 raft_consensus.cc:493] T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Starting leader election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:22:02.979533 16148 raft_consensus.cc:2393] T 2794af5b99984efebdee564035841cb0 P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 16: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
I20260709 13:22:02.978888 22735 raft_consensus.cc:493] T ecc1e363230043fe9fef390f0b979f20 P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:02.985477 16145 raft_consensus.cc:1240] T 26354361e9b044cea1c2b86c4882fe92 P 647a343a542d4a139fe106c19095129b [term 16 LEADER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 15. Current term is 16. Ops: []
I20260709 13:22:02.973201 16294 raft_consensus.cc:2393] T fffab26427e24d398e2695aad40ca134 P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 19: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
I20260709 13:22:02.989753 16218 raft_consensus.cc:2468] T c2f5f89186204242bf377f7c42b2e2da P 1ca2725d290e47d68b1a4b609c9816ac [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 20.
I20260709 13:22:03.005956 16265 leader_election.cc:304] T cdbd78acb4cf46298e0de8ba0d130e19 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:22:03.006420 23036 leader_election.cc:290] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 21 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:03.007263 22740 leader_election.cc:290] T 19baa7bc367844c288bca8f81439c413 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:03.004691 22917 raft_consensus.cc:3060] T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:22:02.994270 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62fe3346cd3c42c4a55e14af9073bca4" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:02.929137 16042 catalog_manager.cc:5697] T 5a959dbecc2648eb85d1cc550f5a3245 P 647a343a542d4a139fe106c19095129b reported cstate change: term changed from 12 to 14, leader changed from 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130) to 647a343a542d4a139fe106c19095129b (127.14.177.129). New cstate: current_term: 14 leader_uuid: "647a343a542d4a139fe106c19095129b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:02.971614 22782 raft_consensus.cc:2749] T d2ab0ac8347641e6955e9bce7d90c1a2 P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Leader pre-election lost for term 15. Reason: Vote denied by peer cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923) with higher term. Message: Invalid argument: T d2ab0ac8347641e6955e9bce7d90c1a2 P cb82cc7bd6db46d980c75237a1402860 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 for earlier term 15. Current term is 16.
I20260709 13:22:02.997547 16151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fffab26427e24d398e2695aad40ca134" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:22:03.014564 16151 raft_consensus.cc:2393] T fffab26427e24d398e2695aad40ca134 P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 19: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
W20260709 13:22:02.998363 23094 log.cc:927] Time spent T 18ad7e3af5504571a69d10c00750a01c P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.087s	user 0.000s	sys 0.000s
I20260709 13:22:02.953194 22979 consensus_queue.cc:1059] T af57996a8be84d398f0f023e8d08ef77 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: INVALID_TERM, Last received: 18.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.033s
I20260709 13:22:03.022174 22979 raft_consensus.cc:3055] T af57996a8be84d398f0f023e8d08ef77 P 1ca2725d290e47d68b1a4b609c9816ac [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:22:03.022562 22979 raft_consensus.cc:740] T af57996a8be84d398f0f023e8d08ef77 P 1ca2725d290e47d68b1a4b609c9816ac [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:22:03.023531 22979 consensus_queue.cc:260] T af57996a8be84d398f0f023e8d08ef77 P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:03.024816 22979 raft_consensus.cc:3060] T af57996a8be84d398f0f023e8d08ef77 P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:22:02.983880 16266 leader_election.cc:304] T b87147731a68400e8cce9b879daa7c5d P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
I20260709 13:22:03.027572 16144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c2095a5f4ad49408dbc7f69c606120d" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
W20260709 13:22:03.027846 16266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 44046.1us
I20260709 13:22:03.028332 16148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a96373bb0e14dce8a266b2fc6c58507" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:22:03.029095 16148 raft_consensus.cc:2393] T 5a96373bb0e14dce8a266b2fc6c58507 P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 16: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
I20260709 13:22:03.029397 16145 raft_consensus.cc:1275] T 3884f864a96d428785895e5463e5728f P 647a343a542d4a139fe106c19095129b [term 14 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 13 index: 13. Preceding OpId from leader: term: 14 index: 14. (index mismatch)
I20260709 13:22:03.004518 16146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e65676b21b748728ad39668cfe722ec" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:22:03.030124 23077 raft_consensus.cc:2749] T b87147731a68400e8cce9b879daa7c5d P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:22:03.030661 16146 raft_consensus.cc:2393] T 9e65676b21b748728ad39668cfe722ec P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 18: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
I20260709 13:22:03.030833 23077 raft_consensus.cc:493] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
W20260709 13:22:02.992071 23088 log.cc:927] Time spent T 08c74490c40243a88e62d1760b5cd814 P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 0.104s	user 0.000s	sys 0.000s
I20260709 13:22:03.031235 16219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fffab26427e24d398e2695aad40ca134" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:22:03.031479 23077 raft_consensus.cc:515] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:03.006328 16218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c74490c40243a88e62d1760b5cd814" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:03.032601 16219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cc959fb7f704fba9009c0595467d074" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:03.033277 16218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9944d5e4b8c4109af9955c4f08c9095" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 22 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:03.033782 23077 leader_election.cc:290] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:03.012740 16265 leader_election.cc:304] T 66ae176961b74813a58bc15c9c4c6f2a P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:22:03.012028 16225 raft_consensus.cc:2468] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 25.
I20260709 13:22:03.034956 16265 leader_election.cc:304] T e4dc6b0541ca431ebc757beac00521f1 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:22:03.036013 16115 leader_election.cc:304] T 2794af5b99984efebdee564035841cb0 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:22:03.036945 16115 leader_election.cc:304] T 23f82224f4564fea93572e8206bad1fe P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:22:03.037652 23016 raft_consensus.cc:2804] T 66ae176961b74813a58bc15c9c4c6f2a P 90f3da57796b488c9087a2f523adf045 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 13:22:03.016196 16822 raft_consensus.cc:1240] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860 [term 16 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 15. Current term is 16. Ops: []
I20260709 13:22:03.038388 23016 raft_consensus.cc:697] T 66ae176961b74813a58bc15c9c4c6f2a P 90f3da57796b488c9087a2f523adf045 [term 25 LEADER]: Becoming Leader. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:22:03.011617 22735 raft_consensus.cc:3060] T ecc1e363230043fe9fef390f0b979f20 P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:22:03.039619 23016 consensus_queue.cc:237] T 66ae176961b74813a58bc15c9c4c6f2a P 90f3da57796b488c9087a2f523adf045 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 24.15, Last appended by leader: 15, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:03.040621 22782 consensus_queue.cc:1048] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:22:03.037681 22645 raft_consensus.cc:2804] T e4dc6b0541ca431ebc757beac00521f1 P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:22:03.042623 23005 raft_consensus.cc:2804] T 23f82224f4564fea93572e8206bad1fe P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:22:03.042977 22645 raft_consensus.cc:493] T e4dc6b0541ca431ebc757beac00521f1 P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:22:03.017223 16821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:22:03.043409 23005 raft_consensus.cc:697] T 23f82224f4564fea93572e8206bad1fe P 647a343a542d4a139fe106c19095129b [term 17 LEADER]: Becoming Leader. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:22:03.043583 22645 raft_consensus.cc:3060] T e4dc6b0541ca431ebc757beac00521f1 P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:22:03.019656 16224 raft_consensus.cc:2468] T 7c2095a5f4ad49408dbc7f69c606120d P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 16.
I20260709 13:22:03.044643 23005 consensus_queue.cc:237] T 23f82224f4564fea93572e8206bad1fe P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:03.045534 16224 raft_consensus.cc:1240] T 7c2095a5f4ad49408dbc7f69c606120d P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 15. Current term is 16. Ops: []
I20260709 13:22:03.045884 22908 leader_election.cc:290] T 84f2da98864b4425bf7c0055951fdd4b P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:03.036891 23096 raft_consensus.cc:515] T 829b4ad8c5a54eababeea1181b960d4b P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:03.013276 23083 raft_consensus.cc:2804] T cdbd78acb4cf46298e0de8ba0d130e19 P 90f3da57796b488c9087a2f523adf045 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:22:03.047549 23083 raft_consensus.cc:493] T cdbd78acb4cf46298e0de8ba0d130e19 P 90f3da57796b488c9087a2f523adf045 [term 21 FOLLOWER]: Starting leader election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:22:03.048020 23083 raft_consensus.cc:3060] T cdbd78acb4cf46298e0de8ba0d130e19 P 90f3da57796b488c9087a2f523adf045 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:22:03.048439 16299 raft_consensus.cc:3060] T d16dfad0c3b540e5b4b86492ddf87208 P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:22:03.051399 16151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4dc6b0541ca431ebc757beac00521f1" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 20 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:22:03.052268 23096 leader_election.cc:290] T 829b4ad8c5a54eababeea1181b960d4b P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 17 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:03.036952 22953 leader_election.cc:290] T ee9ed67006144797b1de6ea8c2013a68 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:03.042415 23086 raft_consensus.cc:2804] T 2794af5b99984efebdee564035841cb0 P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:22:03.053408 23086 raft_consensus.cc:697] T 2794af5b99984efebdee564035841cb0 P 647a343a542d4a139fe106c19095129b [term 16 LEADER]: Becoming Leader. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:22:03.054363 23086 consensus_queue.cc:237] T 2794af5b99984efebdee564035841cb0 P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:03.007200 23061 consensus_queue.cc:237] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.12, Last appended by leader: 12, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:03.033938 16218 raft_consensus.cc:2468] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 21.
I20260709 13:22:03.066715 16822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee9ed67006144797b1de6ea8c2013a68" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
W20260709 13:22:03.069103 16190 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:22:03.019958 22888 raft_consensus.cc:3055] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:22:03.043761 16821 raft_consensus.cc:3060] T 19baa7bc367844c288bca8f81439c413 P cb82cc7bd6db46d980c75237a1402860 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:22:03.058085 16225 raft_consensus.cc:1275] T e4dc6b0541ca431ebc757beac00521f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 17 index: 9. Preceding OpId from leader: term: 19 index: 10. (index mismatch)
I20260709 13:22:03.033504 16219 raft_consensus.cc:2410] T 8cc959fb7f704fba9009c0595467d074 P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 for term 18 because replica has last-logged OpId of term: 16 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 11.
I20260709 13:22:03.061600 16115 leader_election.cc:304] T 57a744f5aaf944678a63469cbf514015 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:22:03.069618 16265 leader_election.cc:304] T 7c2095a5f4ad49408dbc7f69c606120d P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:22:03.076077 16220 raft_consensus.cc:2468] T fffab26427e24d398e2695aad40ca134 P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 19.
I20260709 13:22:03.078008 16225 raft_consensus.cc:1240] T d16dfad0c3b540e5b4b86492ddf87208 P 1ca2725d290e47d68b1a4b609c9816ac [term 20 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 19. Current term is 20. Ops: [19.11-19.11]
I20260709 13:22:03.077980 16822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a378d7f796f94615b3b347119a9880fd" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:03.080199 16147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fffab26427e24d398e2695aad40ca134" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:22:03.081049 16147 raft_consensus.cc:2393] T fffab26427e24d398e2695aad40ca134 P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 19: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
I20260709 13:22:03.081079 16822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74ca8140f35c4ae29e8a664f6bdfc829" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:22:03.082270 16146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee9ed67006144797b1de6ea8c2013a68" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:22:03.082789 16825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdbd78acb4cf46298e0de8ba0d130e19" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:03.086160 16818 raft_consensus.cc:1275] T 3884f864a96d428785895e5463e5728f P cb82cc7bd6db46d980c75237a1402860 [term 14 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 13 index: 13. Preceding OpId from leader: term: 14 index: 14. (index mismatch)
I20260709 13:22:03.085263 16822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9944d5e4b8c4109af9955c4f08c9095" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 22 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:03.076882 23096 raft_consensus.cc:2804] T 57a744f5aaf944678a63469cbf514015 P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:22:03.089749 16820 raft_consensus.cc:1240] T 26354361e9b044cea1c2b86c4882fe92 P cb82cc7bd6db46d980c75237a1402860 [term 16 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 15. Current term is 16. Ops: []
I20260709 13:22:03.072043 22888 raft_consensus.cc:740] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:22:03.093293 16191 leader_election.cc:304] T ee9ed67006144797b1de6ea8c2013a68 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
I20260709 13:22:03.090013 23096 raft_consensus.cc:493] T 57a744f5aaf944678a63469cbf514015 P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:03.077004 23077 raft_consensus.cc:2804] T 7c2095a5f4ad49408dbc7f69c606120d P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:22:03.080045 22701 consensus_queue.cc:1048] T c9944d5e4b8c4109af9955c4f08c9095 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.015s
I20260709 13:22:03.102159 23096 raft_consensus.cc:3060] T 57a744f5aaf944678a63469cbf514015 P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:22:03.104506 23073 log.cc:927] Time spent T 1df93779209748b9860f6d893dd3de53 P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 0.663s	user 0.003s	sys 0.000s
W20260709 13:22:03.106033 23100 log.cc:927] Time spent T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 0.114s	user 0.000s	sys 0.000s
I20260709 13:22:03.083911 23003 raft_consensus.cc:515] T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:03.109519 16145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c74490c40243a88e62d1760b5cd814" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:22:03.109746 16821 raft_consensus.cc:2468] T 19baa7bc367844c288bca8f81439c413 P cb82cc7bd6db46d980c75237a1402860 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 22.
I20260709 13:22:03.134651 16145 raft_consensus.cc:2468] T 08c74490c40243a88e62d1760b5cd814 P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 17.
I20260709 13:22:03.081952 23089 consensus_queue.cc:1048] T 434bff3082d141838068f323bdb0d1ed P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.035s
I20260709 13:22:03.136162 16822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2794af5b99984efebdee564035841cb0" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:22:03.112988 16148 pending_rounds.cc:85] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b: Aborting all ops after (but not including) 11
I20260709 13:22:03.138648 16148 pending_rounds.cc:107] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b: Aborting uncommitted NO_OP operation due to leader change: 19.12
I20260709 13:22:03.139060 16148 raft_consensus.cc:2889] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:22:03.114923 22953 raft_consensus.cc:2749] T ee9ed67006144797b1de6ea8c2013a68 P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:22:03.116380 16219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "829b4ad8c5a54eababeea1181b960d4b" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:22:03.118101 16299 raft_consensus.cc:1275] T d16dfad0c3b540e5b4b86492ddf87208 P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 20 index: 11. (term mismatch)
I20260709 13:22:03.142197 16299 pending_rounds.cc:85] T d16dfad0c3b540e5b4b86492ddf87208 P 90f3da57796b488c9087a2f523adf045: Aborting all ops after (but not including) 10
I20260709 13:22:03.142535 16299 pending_rounds.cc:107] T d16dfad0c3b540e5b4b86492ddf87208 P 90f3da57796b488c9087a2f523adf045: Aborting uncommitted NO_OP operation due to leader change: 19.11
W20260709 13:22:03.118831 16192 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:22:03.142781 16819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62fe3346cd3c42c4a55e14af9073bca4" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:03.143625 16819 raft_consensus.cc:2393] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 25: Already voted for candidate cb82cc7bd6db46d980c75237a1402860 in this term.
I20260709 13:22:03.118706 16820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31969392ccdb43b491911a319e8bc7e1" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:03.142783 16299 raft_consensus.cc:2889] T d16dfad0c3b540e5b4b86492ddf87208 P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:22:03.120191 16823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66f75124215f4ca6bef831e174a792d8" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:03.122007 16825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e65676b21b748728ad39668cfe722ec" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:22:03.144704 16265 leader_election.cc:304] T c0eab25ce4c64e018a041beeda2867db P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:22:03.106499 22917 raft_consensus.cc:515] T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:03.127362 16151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57a744f5aaf944678a63469cbf514015" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 16 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:22:03.075217 22864 raft_consensus.cc:515] T e5ad4eb281564936a59f41e387a47156 P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:03.110304 16144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e9b2bf7011f4ae688a79d7bb5a447b4" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:22:03.141652 16818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f82224f4564fea93572e8206bad1fe" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:22:03.144605 16820 raft_consensus.cc:2468] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 16.
I20260709 13:22:03.150069 16820 raft_consensus.cc:1240] T 19baa7bc367844c288bca8f81439c413 P cb82cc7bd6db46d980c75237a1402860 [term 22 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 21. Current term is 22. Ops: [21.12-21.12]
I20260709 13:22:03.152505 16299 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48674 (request call id 3361) took 1902 ms. Trace:
I20260709 13:22:03.125582 23083 raft_consensus.cc:515] T cdbd78acb4cf46298e0de8ba0d130e19 P 90f3da57796b488c9087a2f523adf045 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:03.155416 16219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee9ed67006144797b1de6ea8c2013a68" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:03.153012 16299 rpcz_store.cc:276] 0709 13:22:01.250209 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:22:01.357043 (+106834us) service_pool.cc:224] Handling call
0709 13:22:01.357360 (+   317us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:22:03.047970 (+1690610us) spinlock_profiling.cc:243] Waited 1.02 s on lock 0x7b58007f9ad0. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0abb0aa 00007f69b0b3d447 00007f69b0b55fe7 00007f69b0b577de 00007f69b0b61b49 00007f69b0b62416 00007f69b0b5ed8f 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4
0709 13:22:03.152213 (+104243us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:22:03.152440 (+   227us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2357341056}
I20260709 13:22:03.157006 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5d59407eb545b895d8d922e0dd69fd" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:22:03.151903 16821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5718c623ebee4d2bb770f0f9ddb2f164" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:22:03.157027 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:03.159492 16820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5d59407eb545b895d8d922e0dd69fd" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:22:03.160439 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:03.163491 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e4bfb9090c8460eb14fa7e0d818e7f1" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:22:03.106714 22645 raft_consensus.cc:515] T e4dc6b0541ca431ebc757beac00521f1 P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:03.165470 16825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57a744f5aaf944678a63469cbf514015" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:03.131443 23077 raft_consensus.cc:697] T 7c2095a5f4ad49408dbc7f69c606120d P 90f3da57796b488c9087a2f523adf045 [term 16 LEADER]: Becoming Leader. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:22:03.176658 16299 raft_consensus.cc:1240] T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 19. Current term is 20. Ops: []
I20260709 13:22:03.177026 16295 raft_consensus.cc:1240] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [term 22 FOLLOWER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 21. Current term is 22. Ops: []
I20260709 13:22:03.174475 16789 leader_election.cc:304] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:22:03.174772 22888 consensus_queue.cc:260] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.15, Last appended by leader: 15, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:03.136749 23037 raft_consensus.cc:493] T 5a959dbecc2648eb85d1cc550f5a3245 P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:03.168565 16822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "829b4ad8c5a54eababeea1181b960d4b" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:22:03.176587 16300 raft_consensus.cc:1240] T e5ad4eb281564936a59f41e387a47156 P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 18. Current term is 19. Ops: [18.12-18.12]
I20260709 13:22:03.178891 16295 raft_consensus.cc:1275] T a378d7f796f94615b3b347119a9880fd P 90f3da57796b488c9087a2f523adf045 [term 21 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
I20260709 13:22:03.185103 16295 raft_consensus.cc:1275] T d2ab0ac8347641e6955e9bce7d90c1a2 P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 14 index: 13. Preceding OpId from leader: term: 16 index: 14. (index mismatch)
I20260709 13:22:03.188483 16295 raft_consensus.cc:1275] T b87147731a68400e8cce9b879daa7c5d P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 13:22:03.190552 16295 raft_consensus.cc:1240] T 23f82224f4564fea93572e8206bad1fe P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 16. Current term is 17. Ops: []
W20260709 13:22:03.167506 22873 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 20 current term: 20 is higher.
I20260709 13:22:03.195689 16299 raft_consensus.cc:1240] T cdbd78acb4cf46298e0de8ba0d130e19 P 90f3da57796b488c9087a2f523adf045 [term 22 FOLLOWER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 21. Current term is 22. Ops: []
I20260709 13:22:03.197997 16818 raft_consensus.cc:1275] T 434bff3082d141838068f323bdb0d1ed P cb82cc7bd6db46d980c75237a1402860 [term 18 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 18 index: 15. (index mismatch)
I20260709 13:22:03.143366 22494 raft_consensus.cc:515] T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:03.181574 22543 raft_consensus.cc:2749] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P cb82cc7bd6db46d980c75237a1402860 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:22:03.151881 23076 consensus_queue.cc:1048] T 1df93779209748b9860f6d893dd3de53 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.045s
I20260709 13:22:03.197677 22782 raft_consensus.cc:3060] T c0eab25ce4c64e018a041beeda2867db P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:22:03.168730 23036 consensus_queue.cc:1048] T d16dfad0c3b540e5b4b86492ddf87208 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.033s
I20260709 13:22:03.213932 16293 raft_consensus.cc:1240] T b87147731a68400e8cce9b879daa7c5d P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 19. Current term is 20. Ops: []
I20260709 13:22:03.154536 16265 leader_election.cc:304] T c4f65cdf00b748bb963ee6e81ef43b79 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:22:03.201508 22888 raft_consensus.cc:3060] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:22:03.193871 23037 raft_consensus.cc:515] T 5a959dbecc2648eb85d1cc550f5a3245 P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:03.180230 16117 leader_election.cc:304] T 746aabf856ab401c9ef5e859371e7b67 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860
I20260709 13:22:03.198431 22940 consensus_queue.cc:1048] T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.016s
I20260709 13:22:03.198603 22676 consensus_queue.cc:1048] T d2ab0ac8347641e6955e9bce7d90c1a2 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.017s
W20260709 13:22:03.237715 16115 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:22:03.198208 22508 consensus_queue.cc:1059] T 74ca8140f35c4ae29e8a664f6bdfc829 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: INVALID_TERM, Last received: 19.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.016s
I20260709 13:22:03.243372 22885 consensus_queue.cc:1059] T 23f82224f4564fea93572e8206bad1fe P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: INVALID_TERM, Last received: 16.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
W20260709 13:22:03.257236 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 102777us
I20260709 13:22:03.204931 22991 consensus_queue.cc:1048] T b87147731a68400e8cce9b879daa7c5d P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.025s
I20260709 13:22:03.262081 16142 raft_consensus.cc:1240] T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 17. Current term is 18. Ops: []
I20260709 13:22:03.301374 16265 leader_election.cc:304] T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
W20260709 13:22:03.265532 16115 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:22:03.152931 16042 catalog_manager.cc:5697] T 8cc959fb7f704fba9009c0595467d074 P 647a343a542d4a139fe106c19095129b reported cstate change: term changed from 11 to 14, leader changed from 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130) to 90f3da57796b488c9087a2f523adf045 (127.14.177.131). New cstate: current_term: 14 leader_uuid: "90f3da57796b488c9087a2f523adf045" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } }
I20260709 13:22:03.289744 16223 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57600 (request call id 1996) took 1278 ms. Trace:
I20260709 13:22:03.292383 16817 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49906 (request call id 1319) took 1256 ms. Trace:
W20260709 13:22:03.300681 16117 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 120537us
I20260709 13:22:03.264400 23024 raft_consensus.cc:2749] T c4f65cdf00b748bb963ee6e81ef43b79 P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:22:03.307693 16817 rpcz_store.cc:276] 0709 13:22:02.036181 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:22:02.036305 (+   124us) service_pool.cc:224] Handling call
0709 13:22:02.036633 (+   328us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:22:02.131997 (+ 95364us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:22:03.292045 (+1160048us) spinlock_profiling.cc:243] Waited 90.6 ms on lock 0x7b54007c01d4. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:22:03.292298 (+   253us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":208441600}
I20260709 13:22:03.308957 16117 leader_election.cc:304] T 9e65676b21b748728ad39668cfe722ec P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860
I20260709 13:22:03.307359 16223 rpcz_store.cc:276] 0709 13:22:02.010973 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:22:02.011111 (+   138us) service_pool.cc:224] Handling call
0709 13:22:03.289494 (+1278383us) spinlock_profiling.cc:243] Waited 38.6 ms on lock 0x7b54002c8654. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:22:03.289666 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":88858880}
I20260709 13:22:03.268151 22946 raft_consensus.cc:2749] T b6f07842d3aa47858df5665874c86f3a P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:22:03.320631 16223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66f75124215f4ca6bef831e174a792d8" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:22:03.313261 16265 leader_election.cc:304] T 2794af5b99984efebdee564035841cb0 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
I20260709 13:22:03.302309 23016 raft_consensus.cc:2749] T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:22:03.323632 16144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c61a39fd4f84424b129c1a6f9ba404d" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:22:03.328506 23096 raft_consensus.cc:515] T 57a744f5aaf944678a63469cbf514015 P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:03.330477 22645 leader_election.cc:290] T e4dc6b0541ca431ebc757beac00521f1 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 20 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:22:03.331769 16151 raft_consensus.cc:2393] T 57a744f5aaf944678a63469cbf514015 P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 16: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
I20260709 13:22:03.321538 23090 consensus_queue.cc:1059] T b87147731a68400e8cce9b879daa7c5d P 647a343a542d4a139fe106c19095129b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.014s
I20260709 13:22:03.337637 16117 leader_election.cc:304] T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860; no voters: 90f3da57796b488c9087a2f523adf045
I20260709 13:22:03.339982 16265 leader_election.cc:304] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
I20260709 13:22:03.305492 22196 raft_consensus.cc:2749] T 746aabf856ab401c9ef5e859371e7b67 P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:22:03.233294 23077 consensus_queue.cc:237] T 7c2095a5f4ad49408dbc7f69c606120d P 90f3da57796b488c9087a2f523adf045 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.7, Last appended by leader: 7, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:03.335404 22543 raft_consensus.cc:3055] T 23f82224f4564fea93572e8206bad1fe P cb82cc7bd6db46d980c75237a1402860 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:22:03.355965 16265 leader_election.cc:304] T 5a96373bb0e14dce8a266b2fc6c58507 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:22:03.388774 16265 leader_election.cc:304] T 9e65676b21b748728ad39668cfe722ec P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
I20260709 13:22:03.334415 23096 leader_election.cc:290] T 57a744f5aaf944678a63469cbf514015 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 16 election: Requested vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:03.353133 22645 raft_consensus.cc:2749] T 2794af5b99984efebdee564035841cb0 P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:22:03.355973 22439 raft_consensus.cc:2749] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
W20260709 13:22:03.446713 16117 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 109190us
I20260709 13:22:03.382318 23005 raft_consensus.cc:2749] T 9e65676b21b748728ad39668cfe722ec P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
W20260709 13:22:03.454386 23110 log.cc:927] Time spent T 57a744f5aaf944678a63469cbf514015 P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
W20260709 13:22:03.454547 23105 log.cc:927] Time spent T c4f65cdf00b748bb963ee6e81ef43b79 P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.136s	user 0.000s	sys 0.000s
I20260709 13:22:03.441962 22960 raft_consensus.cc:2749] T 5a96373bb0e14dce8a266b2fc6c58507 P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
W20260709 13:22:03.455991 23109 log.cc:927] Time spent T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
I20260709 13:22:03.457190 16223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b7d1ab519044eda2d5e04a07242f25" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:22:03.458212 22494 leader_election.cc:290] T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 18 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:03.458508 16265 leader_election.cc:304] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac
I20260709 13:22:03.464496 16221 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57600 (request call id 1985) took 3000 ms. Trace:
I20260709 13:22:03.396209 22917 leader_election.cc:290] T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 20 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:03.386518 22543 raft_consensus.cc:740] T 23f82224f4564fea93572e8206bad1fe P cb82cc7bd6db46d980c75237a1402860 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:22:03.396404 22864 leader_election.cc:290] T e5ad4eb281564936a59f41e387a47156 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 19 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:22:03.469866 16823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57a744f5aaf944678a63469cbf514015" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:22:03.444645 23015 raft_consensus.cc:2804] T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:22:03.436669 23083 leader_election.cc:290] T cdbd78acb4cf46298e0de8ba0d130e19 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 22 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:03.440339 23003 leader_election.cc:290] T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 17 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
W20260709 13:22:03.459074 23104 log.cc:927] Time spent T c0eab25ce4c64e018a041beeda2867db P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.107s	user 0.000s	sys 0.000s
I20260709 13:22:03.444516 23090 raft_consensus.cc:3055] T b87147731a68400e8cce9b879daa7c5d P 647a343a542d4a139fe106c19095129b [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:22:03.479274 16817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66f75124215f4ca6bef831e174a792d8" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
W20260709 13:22:03.479701 23106 log.cc:927] Time spent T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 0.227s	user 0.000s	sys 0.001s
I20260709 13:22:03.480911 16148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c2095a5f4ad49408dbc7f69c606120d" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:22:03.481215 22960 raft_consensus.cc:2804] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:22:03.480950 16265 leader_election.cc:304] T c9944d5e4b8c4109af9955c4f08c9095 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:22:03.482635 16145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f82224f4564fea93572e8206bad1fe" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:22:03.484338 16143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a96373bb0e14dce8a266b2fc6c58507" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:22:03.512267 15046 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:22:03.489130 16816 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49892 (request call id 993) took 1597 ms (client timeout 1750 ms). Trace:
I20260709 13:22:03.512632 16143 raft_consensus.cc:2393] T 5a96373bb0e14dce8a266b2fc6c58507 P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 16: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
W20260709 13:22:03.512840 16816 rpcz_store.cc:269] 0709 13:22:01.891344 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:22:01.892395 (+  1051us) service_pool.cc:224] Handling call
0709 13:22:03.488873 (+1596478us) spinlock_profiling.cc:243] Waited 575 ms on lock 0x7b54004e0cd4. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:22:03.489034 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1322673408}
I20260709 13:22:03.512527 16265 leader_election.cc:304] T 8cc959fb7f704fba9009c0595467d074 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:22:03.490062 16296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57a744f5aaf944678a63469cbf514015" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:22:03.491549 16819 raft_consensus.cc:1240] T b87147731a68400e8cce9b879daa7c5d P cb82cc7bd6db46d980c75237a1402860 [term 20 LEADER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 19. Current term is 20. Ops: []
I20260709 13:22:03.492781 22677 leader_election.cc:290] T d16dfad0c3b540e5b4b86492ddf87208 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:03.493708 22834 raft_consensus.cc:1064] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b: attempting to promote NON_VOTER cb82cc7bd6db46d980c75237a1402860 to VOTER
I20260709 13:22:03.500041 16116 leader_election.cc:304] T 68a6838d82ef4cac83507f906139d90f P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:22:03.536141 23123 consensus_queue.cc:1059] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: INVALID_TERM, Last received: 19.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.021s
W20260709 13:22:03.554001 23105 log.cc:927] Time spent T c4f65cdf00b748bb963ee6e81ef43b79 P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.052s	user 0.000s	sys 0.001s
W20260709 13:22:03.553854 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 53900.6us
I20260709 13:22:03.555317 16116 leader_election.cc:304] T 5a96373bb0e14dce8a266b2fc6c58507 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
I20260709 13:22:03.556531 23036 raft_consensus.cc:2749] T 68a6838d82ef4cac83507f906139d90f P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:22:03.556767 22782 raft_consensus.cc:2749] T c0eab25ce4c64e018a041beeda2867db P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:22:03.557687 23036 raft_consensus.cc:2804] T 5a96373bb0e14dce8a266b2fc6c58507 P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:22:03.558420 23036 raft_consensus.cc:697] T 5a96373bb0e14dce8a266b2fc6c58507 P 647a343a542d4a139fe106c19095129b [term 16 LEADER]: Becoming Leader. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:22:03.561033 23036 consensus_queue.cc:237] T 5a96373bb0e14dce8a266b2fc6c58507 P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:03.378803 23079 consensus_queue.cc:1059] T 26354361e9b044cea1c2b86c4882fe92 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.073s
I20260709 13:22:03.532121 23083 raft_consensus.cc:2749] T 8cc959fb7f704fba9009c0595467d074 P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:22:03.566648 23079 consensus_queue.cc:1243] T 26354361e9b044cea1c2b86c4882fe92 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Peer 647a343a542d4a139fe106c19095129b log is divergent from this leader: its last log entry 16.13 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 12
I20260709 13:22:03.567481 23125 consensus_queue.cc:1048] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:22:03.444869 23076 raft_consensus.cc:493] T ecc1e363230043fe9fef390f0b979f20 P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:22:03.476241 16221 rpcz_store.cc:276] 0709 13:22:00.464511 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:22:00.470048 (+  5537us) service_pool.cc:224] Handling call
0709 13:22:00.470336 (+   288us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:22:01.853495 (+1383159us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:22:02.000125 (+146630us) log.cc:844] Serialized 14 byte log entry
0709 13:22:03.112054 (+1111929us) spinlock_profiling.cc:243] Waited 10.5 ms on lock 0x7b1800390700. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b693ade1 00007f69b69475b1 00007f69ada54e07 00007f69b68afab9 00007f69ada7169b 00007f69ada6f93f 00007f69ada71c02 00007f69b06f920b 00007f69b06fed23 00007f69b06ff23a 00007f69b0b69a2a
0709 13:22:03.264224 (+152170us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:22:03.264834 (+   610us) log.cc:844] Serialized 26 byte log entry
0709 13:22:03.264987 (+   153us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:22:03.265024 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:22:03.265102 (+    78us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:22:03.464075 (+198973us) raft_consensus.cc:1685] finished
0709 13:22:03.464218 (+   143us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:22:03.464425 (+   207us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1400519,"prepare.queue_time_us":160162,"prepare.run_cpu_time_us":53,"prepare.run_wall_time_us":3403,"raft.queue_time_us":1390011,"raft.run_cpu_time_us":184,"raft.run_wall_time_us":4539,"spinlock_wait_cycles":24261504,"thread_start_us":272640,"threads_started":2,"wal-append.queue_time_us":1364746}
I20260709 13:22:03.438235 23024 consensus_queue.cc:1048] T 3884f864a96d428785895e5463e5728f P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.064s
W20260709 13:22:03.472513 23108 log.cc:927] Time spent T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.169s	user 0.001s	sys 0.000s
W20260709 13:22:03.529074 23112 log.cc:927] Time spent T 87ac0d1f7e8a419f9bcbc04aee0e067e P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 0.083s	user 0.000s	sys 0.001s
I20260709 13:22:03.532189 16150 raft_consensus.cc:1275] T 08c74490c40243a88e62d1760b5cd814 P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 13:22:03.511622 22960 raft_consensus.cc:493] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:03.534575 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdbd78acb4cf46298e0de8ba0d130e19" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:22:03.565086 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4dc6b0541ca431ebc757beac00521f1" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:22:03.551882 16218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7ca61592e848ea9b0fb81a2ce33cdc" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:03.573347 16217 raft_consensus.cc:3060] T e4dc6b0541ca431ebc757beac00521f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:22:03.565716 22967 consensus_queue.cc:1059] T 8a878999c0024d2abe517484180290fc P 647a343a542d4a139fe106c19095129b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:22:03.567618 23083 raft_consensus.cc:3055] T 26354361e9b044cea1c2b86c4882fe92 P 90f3da57796b488c9087a2f523adf045 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:22:03.503397 23015 raft_consensus.cc:493] T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Starting leader election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:22:03.439404 23078 raft_consensus.cc:3060] T 9e65676b21b748728ad39668cfe722ec P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:22:03.568442 23076 raft_consensus.cc:515] T ecc1e363230043fe9fef390f0b979f20 P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:03.537366 16223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5ad4eb281564936a59f41e387a47156" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:22:03.578066 16223 raft_consensus.cc:3060] T e5ad4eb281564936a59f41e387a47156 P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:22:03.505316 23090 raft_consensus.cc:740] T b87147731a68400e8cce9b879daa7c5d P 647a343a542d4a139fe106c19095129b [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
W20260709 13:22:03.574689 22979 consensus_meta.cc:287] T af57996a8be84d398f0f023e8d08ef77 P 1ca2725d290e47d68b1a4b609c9816ac: Time spent flushing consensus metadata: real 0.549s	user 0.014s	sys 0.000s
I20260709 13:22:03.574889 22960 raft_consensus.cc:3060] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:22:03.576541 23083 raft_consensus.cc:740] T 26354361e9b044cea1c2b86c4882fe92 P 90f3da57796b488c9087a2f523adf045 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:22:03.577378 23015 raft_consensus.cc:3060] T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:22:03.530933 22864 raft_consensus.cc:2804] T c9944d5e4b8c4109af9955c4f08c9095 P 90f3da57796b488c9087a2f523adf045 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:22:03.568945 23125 consensus_queue.cc:1048] T e5ad4eb281564936a59f41e387a47156 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.012s
I20260709 13:22:03.591903 16222 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33050 (request call id 1195) took 1469 ms. Trace:
I20260709 13:22:03.572903 23089 consensus_queue.cc:1059] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true }, Status: INVALID_TERM, Last received: 18.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.014s
I20260709 13:22:03.580930 23122 consensus_queue.cc:1059] T c0eab25ce4c64e018a041beeda2867db P 647a343a542d4a139fe106c19095129b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: INVALID_TERM, Last received: 18.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.017s
I20260709 13:22:03.596617 23083 consensus_queue.cc:260] T 26354361e9b044cea1c2b86c4882fe92 P 90f3da57796b488c9087a2f523adf045 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.13, Last appended by leader: 13, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:03.597865 22681 raft_consensus.cc:3055] T 74ca8140f35c4ae29e8a664f6bdfc829 P cb82cc7bd6db46d980c75237a1402860 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:22:03.599265 16824 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49896 (request call id 1529) took 1517 ms. Trace:
I20260709 13:22:03.601179 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:03.602088 16217 raft_consensus.cc:2468] T e4dc6b0541ca431ebc757beac00521f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 20.
I20260709 13:22:03.583681 23037 leader_election.cc:290] T 5a959dbecc2648eb85d1cc550f5a3245 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:03.590875 23076 leader_election.cc:290] T ecc1e363230043fe9fef390f0b979f20 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:03.637878 16222 rpcz_store.cc:276] 0709 13:22:02.122223 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:22:02.751543 (+629320us) service_pool.cc:224] Handling call
0709 13:22:03.291605 (+540062us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:22:03.291844 (+   239us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:22:03.291897 (+    53us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:22:03.291957 (+    60us) raft_consensus.cc:1645] Marking committed up to 15
0709 13:22:03.292004 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:22:03.292082 (+    78us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:22:03.591712 (+299630us) spinlock_profiling.cc:243] Waited 540 ms on lock 0x7b54002c8650. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5e8ad 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:22:03.591834 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1240994176}
I20260709 13:22:03.611819 16116 leader_election.cc:304] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860
I20260709 13:22:03.638558 16824 rpcz_store.cc:276] 0709 13:22:02.081574 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:22:02.166555 (+ 84981us) service_pool.cc:224] Handling call
0709 13:22:03.599064 (+1432509us) spinlock_profiling.cc:243] Waited 372 ms on lock 0x7b54007ffdd4. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:22:03.599201 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":854746496}
I20260709 13:22:03.643770 16224 raft_consensus.cc:2393] T ecc1e363230043fe9fef390f0b979f20 P 1ca2725d290e47d68b1a4b609c9816ac [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 20: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
W20260709 13:22:03.628547 23116 log.cc:927] Time spent T b6f07842d3aa47858df5665874c86f3a P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 0.108s	user 0.001s	sys 0.001s
I20260709 13:22:03.625355 23128 consensus_queue.cc:1048] T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.021s
I20260709 13:22:03.640977 22681 raft_consensus.cc:740] T 74ca8140f35c4ae29e8a664f6bdfc829 P cb82cc7bd6db46d980c75237a1402860 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:22:03.583804 22543 consensus_queue.cc:260] T 23f82224f4564fea93572e8206bad1fe P cb82cc7bd6db46d980c75237a1402860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:03.607955 23121 consensus_queue.cc:1059] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.077s
I20260709 13:22:03.647006 23037 consensus_queue.cc:1048] T d16dfad0c3b540e5b4b86492ddf87208 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.024s
I20260709 13:22:03.571569 16042 catalog_manager.cc:5697] T 8d4ab870a05347868956aa8b841b2981 P 647a343a542d4a139fe106c19095129b reported cstate change: term changed from 12 to 14, leader changed from 90f3da57796b488c9087a2f523adf045 (127.14.177.131) to 647a343a542d4a139fe106c19095129b (127.14.177.129). New cstate: current_term: 14 leader_uuid: "647a343a542d4a139fe106c19095129b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:03.677465 22873 raft_consensus.cc:493] T 84f2da98864b4425bf7c0055951fdd4b P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:22:03.677095 16042 catalog_manager.cc:5697] T 2794af5b99984efebdee564035841cb0 P 647a343a542d4a139fe106c19095129b reported cstate change: term changed from 10 to 12. New cstate: current_term: 12 leader_uuid: "647a343a542d4a139fe106c19095129b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:22:03.678004 22873 raft_consensus.cc:515] T 84f2da98864b4425bf7c0055951fdd4b P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:03.680341 22873 leader_election.cc:290] T 84f2da98864b4425bf7c0055951fdd4b P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:22:03.654454 23083 raft_consensus.cc:3060] T 26354361e9b044cea1c2b86c4882fe92 P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:22:03.683580 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 71840.3us
I20260709 13:22:03.684508 16116 leader_election.cc:304] T 8d4ab870a05347868956aa8b841b2981 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 90f3da57796b488c9087a2f523adf045
W20260709 13:22:03.674608 22735 consensus_meta.cc:287] T ecc1e363230043fe9fef390f0b979f20 P 647a343a542d4a139fe106c19095129b: Time spent flushing consensus metadata: real 0.635s	user 0.011s	sys 0.000s
I20260709 13:22:03.674654 16292 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48674 (request call id 3380) took 1155 ms. Trace:
I20260709 13:22:03.675309 23037 raft_consensus.cc:2749] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:22:03.686015 16223 raft_consensus.cc:2468] T e5ad4eb281564936a59f41e387a47156 P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 19.
I20260709 13:22:03.685856 16292 rpcz_store.cc:276] 0709 13:22:02.518835 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:22:02.520152 (+  1317us) service_pool.cc:224] Handling call
0709 13:22:03.674431 (+1154279us) spinlock_profiling.cc:243] Waited 143 ms on lock 0x7b540032fd54. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:22:03.674574 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":328558464}
I20260709 13:22:03.686910 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f2da98864b4425bf7c0055951fdd4b" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:03.686808 16116 leader_election.cc:304] T 8158bbd17b994d2eb96f560bd0f1b88d P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:22:03.691102 16222 raft_consensus.cc:1275] T 5a959dbecc2648eb85d1cc550f5a3245 P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 13:22:03.693538 16116 leader_election.cc:304] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 90f3da57796b488c9087a2f523adf045
I20260709 13:22:03.693459 16144 raft_consensus.cc:2468] T 9c61a39fd4f84424b129c1a6f9ba404d P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 20.
I20260709 13:22:03.689651 23089 raft_consensus.cc:2804] T 8d4ab870a05347868956aa8b841b2981 P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:22:03.688004 22979 raft_consensus.cc:493] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:03.698046 16116 leader_election.cc:304] T fffab26427e24d398e2695aad40ca134 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:22:03.702821 16265 leader_election.cc:304] T fffab26427e24d398e2695aad40ca134 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 647a343a542d4a139fe106c19095129b
I20260709 13:22:03.644317 23090 consensus_queue.cc:260] T b87147731a68400e8cce9b879daa7c5d P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.13, Last appended by leader: 13, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:03.690949 22543 raft_consensus.cc:3060] T 23f82224f4564fea93572e8206bad1fe P cb82cc7bd6db46d980c75237a1402860 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:22:03.709074 23015 raft_consensus.cc:515] T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:03.694126 23128 raft_consensus.cc:2749] T 8158bbd17b994d2eb96f560bd0f1b88d P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:22:03.710319 22961 raft_consensus.cc:2804] T fffab26427e24d398e2695aad40ca134 P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Leader election won for term 19
W20260709 13:22:03.711592 16266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 681987us
I20260709 13:22:03.715354 23078 raft_consensus.cc:2749] T 9e65676b21b748728ad39668cfe722ec P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:22:03.690111 22735 raft_consensus.cc:515] T ecc1e363230043fe9fef390f0b979f20 P 647a343a542d4a139fe106c19095129b [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:03.718020 16116 leader_election.cc:304] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: cb82cc7bd6db46d980c75237a1402860
I20260709 13:22:03.693022 22864 raft_consensus.cc:493] T c9944d5e4b8c4109af9955c4f08c9095 P 90f3da57796b488c9087a2f523adf045 [term 21 FOLLOWER]: Starting leader election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:22:03.720659 16144 raft_consensus.cc:1275] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 13:22:03.722368 16294 raft_consensus.cc:1240] T ada5a004ba7c46819051e3ccec71ffdb P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 19. Current term is 20. Ops: []
I20260709 13:22:03.722465 16266 leader_election.cc:304] T 434bff3082d141838068f323bdb0d1ed P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 647a343a542d4a139fe106c19095129b
I20260709 13:22:03.724754 16818 raft_consensus.cc:1240] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P cb82cc7bd6db46d980c75237a1402860 [term 20 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 19. Current term is 20. Ops: []
I20260709 13:22:03.697180 23089 raft_consensus.cc:493] T 8d4ab870a05347868956aa8b841b2981 P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Starting leader election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:22:03.705253 23132 raft_consensus.cc:2804] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:22:03.735445 23132 raft_consensus.cc:493] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Starting leader election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:22:03.736007 23132 raft_consensus.cc:3060] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:22:03.740607 23134 consensus_queue.cc:1059] T 7c2095a5f4ad49408dbc7f69c606120d P 647a343a542d4a139fe106c19095129b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:22:03.740957 16116 leader_election.cc:304] T 829b4ad8c5a54eababeea1181b960d4b P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860
I20260709 13:22:03.741607 23134 consensus_queue.cc:1048] T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:22:03.742137 16116 leader_election.cc:304] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860
I20260709 13:22:03.742614 22735 leader_election.cc:290] T ecc1e363230043fe9fef390f0b979f20 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 21 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:03.742918 23134 raft_consensus.cc:3055] T 7c2095a5f4ad49408dbc7f69c606120d P 647a343a542d4a139fe106c19095129b [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:22:03.743376 23128 raft_consensus.cc:2749] T 829b4ad8c5a54eababeea1181b960d4b P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:22:03.743451 23131 raft_consensus.cc:2749] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:22:03.744115 23122 raft_consensus.cc:2804] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:22:03.746194 16298 raft_consensus.cc:1240] T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 19. Current term is 20. Ops: []
I20260709 13:22:03.747961 16266 leader_election.cc:304] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860
I20260709 13:22:03.748703 22961 raft_consensus.cc:697] T fffab26427e24d398e2695aad40ca134 P 90f3da57796b488c9087a2f523adf045 [term 19 LEADER]: Becoming Leader. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:22:03.748703 23078 raft_consensus.cc:2764] T 434bff3082d141838068f323bdb0d1ed P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Leader pre-election decision vote started in defunct term 17: won
I20260709 13:22:03.750587 16265 leader_election.cc:304] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860
I20260709 13:22:03.726362 23133 raft_consensus.cc:2749] T fffab26427e24d398e2695aad40ca134 P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:22:03.788986 16265 leader_election.cc:304] T e4dc6b0541ca431ebc757beac00521f1 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:22:03.716604 22979 raft_consensus.cc:515] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:03.695443 22681 consensus_queue.cc:260] T 74ca8140f35c4ae29e8a664f6bdfc829 P cb82cc7bd6db46d980c75237a1402860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:03.760959 22731 leader_election.cc:290] T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:03.763350 23015 leader_election.cc:290] T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 17 election: Requested vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:03.771153 22834 consensus_queue.cc:237] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 11, Current term: 18, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:03.781387 16293 raft_consensus.cc:1240] T 7c2095a5f4ad49408dbc7f69c606120d P 90f3da57796b488c9087a2f523adf045 [term 16 LEADER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 15. Current term is 16. Ops: []
I20260709 13:22:03.779592 23134 raft_consensus.cc:740] T 7c2095a5f4ad49408dbc7f69c606120d P 647a343a542d4a139fe106c19095129b [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:22:03.796306 23134 consensus_queue.cc:260] T 7c2095a5f4ad49408dbc7f69c606120d P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 15.8, Last appended by leader: 8, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:03.797556 23134 raft_consensus.cc:3060] T 7c2095a5f4ad49408dbc7f69c606120d P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:22:03.782729 23132 raft_consensus.cc:515] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:03.785815 16116 leader_election.cc:304] T ee9ed67006144797b1de6ea8c2013a68 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860
I20260709 13:22:03.729276 22864 raft_consensus.cc:3060] T c9944d5e4b8c4109af9955c4f08c9095 P 90f3da57796b488c9087a2f523adf045 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:22:03.731607 23090 raft_consensus.cc:3060] T b87147731a68400e8cce9b879daa7c5d P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:22:03.789389 16266 leader_election.cc:304] T a378d7f796f94615b3b347119a9880fd P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860
I20260709 13:22:03.748091 23089 raft_consensus.cc:3060] T 8d4ab870a05347868956aa8b841b2981 P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:22:03.804558 16266 leader_election.cc:304] T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860
I20260709 13:22:03.792289 16265 leader_election.cc:304] T e5ad4eb281564936a59f41e387a47156 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:22:03.784005 23122 raft_consensus.cc:493] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [term 24 FOLLOWER]: Starting leader election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:22:03.793637 23077 raft_consensus.cc:2749] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:22:03.810230 22979 leader_election.cc:290] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
W20260709 13:22:03.814810 16789 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:22:03.816264 16224 raft_consensus.cc:1240] T e4dc6b0541ca431ebc757beac00521f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 20 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 19. Current term is 20. Ops: [19.10-19.10]
I20260709 13:22:03.815341 16226 raft_consensus.cc:1240] T e5ad4eb281564936a59f41e387a47156 P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 18. Current term is 19. Ops: [18.12-18.12]
I20260709 13:22:03.815477 16223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:22:03.817312 16223 raft_consensus.cc:3060] T ecc1e363230043fe9fef390f0b979f20 P 1ca2725d290e47d68b1a4b609c9816ac [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:22:03.818914 16190 leader_election.cc:304] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:22:03.800498 23076 raft_consensus.cc:2804] T e4dc6b0541ca431ebc757beac00521f1 P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:22:03.821111 16190 leader_election.cc:304] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 647a343a542d4a139fe106c19095129b
I20260709 13:22:03.822615 16190 leader_election.cc:304] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860
I20260709 13:22:03.827842 16219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a959dbecc2648eb85d1cc550f5a3245" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:03.828002 16144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2372981006d4b218215c28964dea623" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:22:03.828689 16219 raft_consensus.cc:2468] T 5a959dbecc2648eb85d1cc550f5a3245 P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 18.
I20260709 13:22:03.828832 16144 raft_consensus.cc:2468] T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 22.
I20260709 13:22:03.829732 22782 consensus_queue.cc:1048] T 3884f864a96d428785895e5463e5728f P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:22:03.805413 22681 raft_consensus.cc:3060] T 74ca8140f35c4ae29e8a664f6bdfc829 P cb82cc7bd6db46d980c75237a1402860 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:22:03.811602 22961 consensus_queue.cc:237] T fffab26427e24d398e2695aad40ca134 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:03.834501 16223 raft_consensus.cc:2468] T ecc1e363230043fe9fef390f0b979f20 P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 21.
I20260709 13:22:03.812538 23122 raft_consensus.cc:3060] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:22:03.835307 22709 raft_consensus.cc:2749] T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:22:03.836531 23076 raft_consensus.cc:697] T e4dc6b0541ca431ebc757beac00521f1 P 90f3da57796b488c9087a2f523adf045 [term 20 LEADER]: Becoming Leader. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
W20260709 13:22:03.838769 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 46533.4us
I20260709 13:22:03.840202 16190 leader_election.cc:304] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: cb82cc7bd6db46d980c75237a1402860
I20260709 13:22:03.840701 16116 leader_election.cc:304] T 5a959dbecc2648eb85d1cc550f5a3245 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
I20260709 13:22:03.843266 16190 leader_election.cc:304] T c0eab25ce4c64e018a041beeda2867db P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:22:03.845834 16190 leader_election.cc:304] T e4dc6b0541ca431ebc757beac00521f1 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:22:03.847522 16190 leader_election.cc:304] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:22:03.853188 16223 raft_consensus.cc:1240] T 19baa7bc367844c288bca8f81439c413 P 1ca2725d290e47d68b1a4b609c9816ac [term 22 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 21. Current term is 22. Ops: [21.12-21.12]
I20260709 13:22:03.848536 22748 raft_consensus.cc:2749] T e4dc6b0541ca431ebc757beac00521f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:22:03.853801 23083 consensus_queue.cc:1048] T 5a959dbecc2648eb85d1cc550f5a3245 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:22:03.855397 23076 consensus_queue.cc:237] T e4dc6b0541ca431ebc757beac00521f1 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.10, Last appended by leader: 10, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:03.848702 22917 raft_consensus.cc:2749] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:22:03.817170 22996 raft_consensus.cc:2749] T ee9ed67006144797b1de6ea8c2013a68 P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:22:03.842988 22979 raft_consensus.cc:2749] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:22:03.852263 23061 raft_consensus.cc:2804] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:22:03.850063 23078 raft_consensus.cc:2804] T e5ad4eb281564936a59f41e387a47156 P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:22:03.861588 23003 raft_consensus.cc:2804] T 5a959dbecc2648eb85d1cc550f5a3245 P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:22:03.849606 22971 raft_consensus.cc:2749] T c0eab25ce4c64e018a041beeda2867db P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:22:03.877966 22960 raft_consensus.cc:515] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:03.878624 23078 raft_consensus.cc:697] T e5ad4eb281564936a59f41e387a47156 P 90f3da57796b488c9087a2f523adf045 [term 19 LEADER]: Becoming Leader. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:22:03.879710 23078 consensus_queue.cc:237] T e5ad4eb281564936a59f41e387a47156 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:03.850171 23123 consensus_queue.cc:1059] T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: INVALID_TERM, Last received: 17.9, Next index: 10, Last known committed idx: 7, Time since last communication: 0.019s
I20260709 13:22:03.849821 23133 consensus_queue.cc:1059] T e5ad4eb281564936a59f41e387a47156 P 647a343a542d4a139fe106c19095129b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: INVALID_TERM, Last received: 17.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.023s
I20260709 13:22:03.888075 16300 raft_consensus.cc:1240] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 17. Current term is 18. Ops: []
I20260709 13:22:03.889145 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d16dfad0c3b540e5b4b86492ddf87208" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 21 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:03.891551 16294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c61a39fd4f84424b129c1a6f9ba404d" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:22:03.889084 16190 leader_election.cc:304] T 19baa7bc367844c288bca8f81439c413 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:22:03.912963 16300 raft_consensus.cc:1275] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 20 index: 13. (term mismatch)
I20260709 13:22:03.913457 16300 pending_rounds.cc:85] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 90f3da57796b488c9087a2f523adf045: Aborting all ops after (but not including) 12
I20260709 13:22:03.913684 16300 pending_rounds.cc:107] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 90f3da57796b488c9087a2f523adf045: Aborting uncommitted NO_OP operation due to leader change: 19.13
I20260709 13:22:03.913941 16300 raft_consensus.cc:2889] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:22:03.914342 22960 leader_election.cc:290] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 18 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:22:03.914577 16190 leader_election.cc:304] T 8a878999c0024d2abe517484180290fc P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:22:03.864341 23015 consensus_queue.cc:1059] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: INVALID_TERM, Last received: 19.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.020s
I20260709 13:22:03.872905 23061 raft_consensus.cc:493] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Starting leader election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:22:03.875851 23003 raft_consensus.cc:493] T 5a959dbecc2648eb85d1cc550f5a3245 P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:03.919755 16223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c74490c40243a88e62d1760b5cd814" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
W20260709 13:22:03.913690 23141 log.cc:927] Time spent T d2ab0ac8347641e6955e9bce7d90c1a2 P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.121s	user 0.001s	sys 0.000s
W20260709 13:22:03.918812 23136 log.cc:927] Time spent T 7c2095a5f4ad49408dbc7f69c606120d P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
I20260709 13:22:03.885648 22748 raft_consensus.cc:493] T 8cc959fb7f704fba9009c0595467d074 P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:03.918465 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f2da98864b4425bf7c0055951fdd4b" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:03.924957 16298 raft_consensus.cc:2468] T 84f2da98864b4425bf7c0055951fdd4b P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 15.
I20260709 13:22:03.918582 22908 raft_consensus.cc:2749] T 8a878999c0024d2abe517484180290fc P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:22:03.923967 16190 leader_election.cc:304] T d16dfad0c3b540e5b4b86492ddf87208 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:22:03.917389 22979 raft_consensus.cc:2749] T 19baa7bc367844c288bca8f81439c413 P 1ca2725d290e47d68b1a4b609c9816ac [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:22:03.916385 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c61a39fd4f84424b129c1a6f9ba404d" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:03.920481 23061 raft_consensus.cc:3060] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:22:03.922226 22996 raft_consensus.cc:3055] T e5ad4eb281564936a59f41e387a47156 P 647a343a542d4a139fe106c19095129b [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:22:03.944042 22996 raft_consensus.cc:740] T e5ad4eb281564936a59f41e387a47156 P 647a343a542d4a139fe106c19095129b [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:22:03.944964 22996 consensus_queue.cc:260] T e5ad4eb281564936a59f41e387a47156 P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.12, Last appended by leader: 12, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:03.922883 22731 raft_consensus.cc:2804] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:22:03.946310 22996 raft_consensus.cc:3060] T e5ad4eb281564936a59f41e387a47156 P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:22:03.948140 16223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f5f89186204242bf377f7c42b2e2da" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:22:03.921752 23003 raft_consensus.cc:3060] T 5a959dbecc2648eb85d1cc550f5a3245 P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:22:03.920310 23128 raft_consensus.cc:3055] T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:22:03.895223 22864 raft_consensus.cc:515] T c9944d5e4b8c4109af9955c4f08c9095 P 90f3da57796b488c9087a2f523adf045 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:22:03.965907 16117 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:22:03.982776 16117 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:22:03.972009 16291 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48684 (request call id 3130) took 1117 ms. Trace:
I20260709 13:22:03.985493 16117 leader_election.cc:304] T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860
I20260709 13:22:03.927644 22971 consensus_queue.cc:1059] T 08c74490c40243a88e62d1760b5cd814 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.056s
I20260709 13:22:03.927410 22748 raft_consensus.cc:515] T 8cc959fb7f704fba9009c0595467d074 P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:03.967492 23128 raft_consensus.cc:740] T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:22:03.983147 22873 raft_consensus.cc:3060] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:22:03.983749 23121 raft_consensus.cc:3055] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:22:03.994745 22834 raft_consensus.cc:2749] T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
W20260709 13:22:03.996879 23120 log.cc:927] Time spent T 806d7690c2b644959152074bb2cab0a7 P cb82cc7bd6db46d980c75237a1402860: Append to log took a long time: real 0.426s	user 0.003s	sys 0.000s
I20260709 13:22:04.000546 16142 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40844 (request call id 2933) took 1114 ms. Trace:
I20260709 13:22:03.988596 22962 raft_consensus.cc:2749] T d16dfad0c3b540e5b4b86492ddf87208 P 1ca2725d290e47d68b1a4b609c9816ac [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:22:04.009941 16142 rpcz_store.cc:276] 0709 13:22:02.885722 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:22:02.885871 (+   149us) service_pool.cc:224] Handling call
0709 13:22:02.886167 (+   296us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:22:03.301780 (+415613us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:22:04.000292 (+698512us) spinlock_profiling.cc:243] Waited 376 ms on lock 0x7b5400073fd4. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:22:04.000484 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":864061696}
I20260709 13:22:03.988771 23133 raft_consensus.cc:493] T 3884f864a96d428785895e5463e5728f P 647a343a542d4a139fe106c19095129b [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:04.010797 16220 raft_consensus.cc:1240] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 18. Current term is 19. Ops: []
I20260709 13:22:04.002295 22731 raft_consensus.cc:493] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac [term 20 FOLLOWER]: Starting leader election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:22:03.991961 22735 raft_consensus.cc:493] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b [term 21 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
W20260709 13:22:04.016489 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 69228.1us
I20260709 13:22:04.019073 16190 leader_election.cc:304] T 84f2da98864b4425bf7c0055951fdd4b P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 647a343a542d4a139fe106c19095129b
I20260709 13:22:04.019276 16151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5ad4eb281564936a59f41e387a47156" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:22:03.993804 23132 leader_election.cc:290] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 21 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:04.020450 16146 raft_consensus.cc:1240] T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 16. Current term is 17. Ops: []
I20260709 13:22:03.988935 22908 consensus_queue.cc:1048] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.024s
I20260709 13:22:04.022521 16143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:22:04.005518 23121 raft_consensus.cc:740] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:22:04.011889 23087 raft_consensus.cc:3055] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:22:04.011374 22979 raft_consensus.cc:3055] T 08c74490c40243a88e62d1760b5cd814 P 1ca2725d290e47d68b1a4b609c9816ac [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:22:04.024837 16143 raft_consensus.cc:2368] T ecc1e363230043fe9fef390f0b979f20 P 647a343a542d4a139fe106c19095129b [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 for earlier term 20. Current term is 21.
I20260709 13:22:04.018014 22731 raft_consensus.cc:3060] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:22:03.992118 23123 consensus_queue.cc:1048] T 434bff3082d141838068f323bdb0d1ed P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.026s
I20260709 13:22:04.025952 16144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4dc6b0541ca431ebc757beac00521f1" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:22:04.023527 16148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f2da98864b4425bf7c0055951fdd4b" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:22:04.020153 22917 raft_consensus.cc:2749] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:22:04.034868 16148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c74490c40243a88e62d1760b5cd814" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:22:04.035478 16148 raft_consensus.cc:3060] T 08c74490c40243a88e62d1760b5cd814 P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:22:04.019933 23133 raft_consensus.cc:515] T 3884f864a96d428785895e5463e5728f P 647a343a542d4a139fe106c19095129b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:04.041462 23133 leader_election.cc:290] T 3884f864a96d428785895e5463e5728f P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
W20260709 13:22:04.048532 23137 log.cc:927] Time spent T b87147731a68400e8cce9b879daa7c5d P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.099s	user 0.000s	sys 0.000s
I20260709 13:22:04.028173 23087 raft_consensus.cc:740] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:22:04.062053 22864 leader_election.cc:290] T c9944d5e4b8c4109af9955c4f08c9095 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 22 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:04.063525 23125 raft_consensus.cc:3055] T c0eab25ce4c64e018a041beeda2867db P 647a343a542d4a139fe106c19095129b [term 18 LEADER]: Stepping down as leader of term 18
W20260709 13:22:04.066870 16791 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:22:04.067739 23152 consensus_queue.cc:1059] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: INVALID_TERM, Last received: 19.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 13:22:04.029726 22979 raft_consensus.cc:740] T 08c74490c40243a88e62d1760b5cd814 P 1ca2725d290e47d68b1a4b609c9816ac [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:22:04.032582 22971 raft_consensus.cc:2804] T 84f2da98864b4425bf7c0055951fdd4b P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:22:04.023301 22735 raft_consensus.cc:515] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:04.099902 22735 leader_election.cc:290] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:04.099449 22979 consensus_queue.cc:260] T 08c74490c40243a88e62d1760b5cd814 P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.11, Last appended by leader: 11, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:04.101048 22979 raft_consensus.cc:3060] T 08c74490c40243a88e62d1760b5cd814 P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:22:04.102763 23142 log.cc:927] Time spent T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 0.152s	user 0.002s	sys 0.000s
I20260709 13:22:04.074781 16820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2372981006d4b218215c28964dea623" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:04.027947 23128 consensus_queue.cc:260] T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.10, Last appended by leader: 10, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:22:04.073668 23145 log.cc:927] Time spent T fffab26427e24d398e2695aad40ca134 P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.067s	user 0.001s	sys 0.000s
I20260709 13:22:04.096359 22748 leader_election.cc:290] T 8cc959fb7f704fba9009c0595467d074 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:04.135766 23128 raft_consensus.cc:3060] T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:22:04.096623 23087 consensus_queue.cc:260] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 21.12, Last appended by leader: 12, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:04.107399 23121 consensus_queue.cc:260] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.13, Last appended by leader: 13, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:04.108449 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9944d5e4b8c4109af9955c4f08c9095" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:22:04.109087 23086 raft_consensus.cc:3055] T 8a878999c0024d2abe517484180290fc P 647a343a542d4a139fe106c19095129b [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:22:04.120065 23061 raft_consensus.cc:515] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 1ca2725d290e47d68b1a4b609c9816ac [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:04.121825 16820 raft_consensus.cc:2468] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 22.
I20260709 13:22:04.157636 16265 leader_election.cc:304] T 23f82224f4564fea93572e8206bad1fe P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
I20260709 13:22:04.069922 16148 raft_consensus.cc:2468] T 08c74490c40243a88e62d1760b5cd814 P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 18.
I20260709 13:22:04.123903 16294 raft_consensus.cc:2393] T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 20: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
I20260709 13:22:04.074625 16222 raft_consensus.cc:1240] T 8a878999c0024d2abe517484180290fc P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 16. Current term is 17. Ops: [16.9-16.9]
I20260709 13:22:04.166776 16192 leader_election.cc:304] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:22:04.161378 16226 raft_consensus.cc:3060] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:22:04.173516 16145 raft_consensus.cc:1275] T 7c2095a5f4ad49408dbc7f69c606120d P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 16 index: 8. (term mismatch)
I20260709 13:22:04.174053 16145 pending_rounds.cc:85] T 7c2095a5f4ad49408dbc7f69c606120d P 647a343a542d4a139fe106c19095129b: Aborting all ops after (but not including) 7
I20260709 13:22:04.174335 16145 pending_rounds.cc:107] T 7c2095a5f4ad49408dbc7f69c606120d P 647a343a542d4a139fe106c19095129b: Aborting uncommitted NO_OP operation due to leader change: 15.8
I20260709 13:22:04.174654 16145 raft_consensus.cc:2889] T 7c2095a5f4ad49408dbc7f69c606120d P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:22:04.176074 23121 raft_consensus.cc:3060] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:22:04.176128 16115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 148997us
I20260709 13:22:04.177266 16115 leader_election.cc:304] T 57a744f5aaf944678a63469cbf514015 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860
W20260709 13:22:04.189529 16790 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:22:04.192077 16790 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:22:04.140434 22971 raft_consensus.cc:493] T 84f2da98864b4425bf7c0055951fdd4b P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: Starting leader election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:22:04.196671 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e4bfb9090c8460eb14fa7e0d818e7f1" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:04.198803 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62fe3346cd3c42c4a55e14af9073bca4" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 26 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:04.203464 16224 raft_consensus.cc:2468] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 25.
I20260709 13:22:04.200259 16292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cc959fb7f704fba9009c0595467d074" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:04.073889 23125 raft_consensus.cc:740] T c0eab25ce4c64e018a041beeda2867db P 647a343a542d4a139fe106c19095129b [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:22:04.204317 16292 raft_consensus.cc:2468] T 8cc959fb7f704fba9009c0595467d074 P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 17.
I20260709 13:22:04.164196 23086 raft_consensus.cc:740] T 8a878999c0024d2abe517484180290fc P 647a343a542d4a139fe106c19095129b [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:22:04.207556 16217 raft_consensus.cc:1275] T d2ab0ac8347641e6955e9bce7d90c1a2 P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 14 index: 13. Preceding OpId from leader: term: 16 index: 14. (index mismatch)
I20260709 13:22:04.207898 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:04.171656 22782 raft_consensus.cc:2749] T 23f82224f4564fea93572e8206bad1fe P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:22:04.214699 16820 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "26354361e9b044cea1c2b86c4882fe92"
dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
mode: GRACEFUL
new_leader_uuid: "90f3da57796b488c9087a2f523adf045"
 from {username='slave'} at 127.0.0.1:37818
I20260709 13:22:04.215603 16820 raft_consensus.cc:606] T 26354361e9b044cea1c2b86c4882fe92 P cb82cc7bd6db46d980c75237a1402860 [term 16 FOLLOWER]: Received request to transfer leadership to TS 90f3da57796b488c9087a2f523adf045
I20260709 13:22:04.216048 16820 raft_consensus.cc:614] T 26354361e9b044cea1c2b86c4882fe92 P cb82cc7bd6db46d980c75237a1402860 [term 16 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:22:04.215884 16222 raft_consensus.cc:1275] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
I20260709 13:22:04.212344 16224 raft_consensus.cc:2393] T 19baa7bc367844c288bca8f81439c413 P 1ca2725d290e47d68b1a4b609c9816ac [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 in current term 22: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
I20260709 13:22:04.182252 22748 raft_consensus.cc:2804] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:22:04.211376 16790 leader_election.cc:304] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:22:04.202168 23087 raft_consensus.cc:3060] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:22:04.210871 23153 raft_consensus.cc:493] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:22:04.206063 22971 raft_consensus.cc:3060] T 84f2da98864b4425bf7c0055951fdd4b P 1ca2725d290e47d68b1a4b609c9816ac [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:22:04.241166 16191 leader_election.cc:304] T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
I20260709 13:22:04.260082 16226 raft_consensus.cc:2468] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 22.
I20260709 13:22:04.260977 16297 raft_consensus.cc:1240] T e4dc6b0541ca431ebc757beac00521f1 P 90f3da57796b488c9087a2f523adf045 [term 20 LEADER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 19. Current term is 20. Ops: []
I20260709 13:22:04.263700 16219 raft_consensus.cc:1240] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac [term 22 FOLLOWER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 21. Current term is 22. Ops: [21.15-21.15]
I20260709 13:22:04.266695 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a959dbecc2648eb85d1cc550f5a3245" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:04.268594 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:22:04.269315 16300 raft_consensus.cc:3060] T ecc1e363230043fe9fef390f0b979f20 P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Advancing to term 21
I20260709 13:22:04.219704 22748 raft_consensus.cc:493] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:04.222966 23023 raft_consensus.cc:2749] T 57a744f5aaf944678a63469cbf514015 P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:22:04.274690 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31969392ccdb43b491911a319e8bc7e1" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
W20260709 13:22:04.276696 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 72031.6us
I20260709 13:22:04.280506 16298 raft_consensus.cc:1275] T 26354361e9b044cea1c2b86c4882fe92 P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 16 index: 13. (term mismatch)
I20260709 13:22:04.304556 16298 pending_rounds.cc:85] T 26354361e9b044cea1c2b86c4882fe92 P 90f3da57796b488c9087a2f523adf045: Aborting all ops after (but not including) 12
I20260709 13:22:04.304903 16298 pending_rounds.cc:107] T 26354361e9b044cea1c2b86c4882fe92 P 90f3da57796b488c9087a2f523adf045: Aborting uncommitted NO_OP operation due to leader change: 15.13
I20260709 13:22:04.305200 16298 raft_consensus.cc:2889] T 26354361e9b044cea1c2b86c4882fe92 P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:22:04.306160 16297 raft_consensus.cc:1275] T 23f82224f4564fea93572e8206bad1fe P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 13:22:04.307297 22508 raft_consensus.cc:2749] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:22:04.308382 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 67301.6us
I20260709 13:22:04.316043 23152 consensus_queue.cc:1048] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:22:04.318176 23152 consensus_queue.cc:1059] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: INVALID_TERM, Last received: 17.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
W20260709 13:22:04.319294 16115 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:22:04.302268 22748 raft_consensus.cc:3060] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:22:04.232758 23130 raft_consensus.cc:2749] T a378d7f796f94615b3b347119a9880fd P 90f3da57796b488c9087a2f523adf045 [term 21 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:22:04.280869 16147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cc959fb7f704fba9009c0595467d074" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:22:04.312400 16820 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9e65676b21b748728ad39668cfe722ec"
dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
mode: GRACEFUL
new_leader_uuid: "90f3da57796b488c9087a2f523adf045"
 from {username='slave'} at 127.0.0.1:37818
I20260709 13:22:04.313046 23160 raft_consensus.cc:493] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
W20260709 13:22:04.313542 16116 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:22:04.313611 23159 raft_consensus.cc:2804] T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:22:04.322853 16300 raft_consensus.cc:2468] T ecc1e363230043fe9fef390f0b979f20 P 90f3da57796b488c9087a2f523adf045 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 21.
I20260709 13:22:04.199674 23089 raft_consensus.cc:515] T 8d4ab870a05347868956aa8b841b2981 P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:04.328074 16265 leader_election.cc:400] T ecc1e363230043fe9fef390f0b979f20 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 20 pre-election: Vote denied by peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741) with higher term. Message: Invalid argument: T ecc1e363230043fe9fef390f0b979f20 P 647a343a542d4a139fe106c19095129b [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 for earlier term 20. Current term is 21.
I20260709 13:22:04.330552 16820 raft_consensus.cc:606] T 9e65676b21b748728ad39668cfe722ec P cb82cc7bd6db46d980c75237a1402860 [term 17 LEADER]: Received request to transfer leadership to TS 90f3da57796b488c9087a2f523adf045
W20260709 13:22:04.278576 23149 log.cc:927] Time spent T d16dfad0c3b540e5b4b86492ddf87208 P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
W20260709 13:22:04.309897 23090 consensus_meta.cc:287] T b87147731a68400e8cce9b879daa7c5d P 647a343a542d4a139fe106c19095129b: Time spent flushing consensus metadata: real 0.504s	user 0.007s	sys 0.005s
I20260709 13:22:04.211935 23122 raft_consensus.cc:515] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:04.273501 22709 raft_consensus.cc:493] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
W20260709 13:22:04.177210 23138 consensus_queue.cc:1175] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "1ca2725d290e47d68b1a4b609c9816ac" responder_term: 19 status { last_received { term: 18 index: 12 } last_committed_idx: 11 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 18. Current term is 19. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:22:04.210278 23131 consensus_queue.cc:1059] T 8a878999c0024d2abe517484180290fc P 647a343a542d4a139fe106c19095129b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.119s
I20260709 13:22:04.264724 22961 consensus_queue.cc:1059] T 31969392ccdb43b491911a319e8bc7e1 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: INVALID_TERM, Last received: 16.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.068s
I20260709 13:22:04.350620 16150 raft_consensus.cc:1240] T 8d4ab870a05347868956aa8b841b2981 P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 17. Current term is 18. Ops: []
I20260709 13:22:04.331069 16147 raft_consensus.cc:2468] T 8cc959fb7f704fba9009c0595467d074 P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 17.
I20260709 13:22:04.331647 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f5f89186204242bf377f7c42b2e2da" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:22:04.344732 16217 raft_consensus.cc:1275] T 5a96373bb0e14dce8a266b2fc6c58507 P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 13:22:04.334539 16265 leader_election.cc:403] T ecc1e363230043fe9fef390f0b979f20 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 20 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:22:04.335814 16300 raft_consensus.cc:1240] T c0eab25ce4c64e018a041beeda2867db P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 18. Current term is 19. Ops: []
I20260709 13:22:03.987181 16291 rpcz_store.cc:276] 0709 13:22:02.854945 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:22:02.893584 (+ 38639us) service_pool.cc:224] Handling call
0709 13:22:02.913830 (+ 20246us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:22:02.914211 (+   381us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:22:02.914254 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:22:02.914819 (+   565us) log.cc:844] Serialized 26 byte log entry
0709 13:22:03.893603 (+978784us) spinlock_profiling.cc:243] Waited 1.94 ms on lock 0x7b1800051b80. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b693ade1 00007f69b69475b1 00007f69ada54e07 00007f69b68afab9 00007f69ada7169b 00007f69ada6f93f 00007f69ada71c02 00007f69b06f920b 00007f69b06fed23 00007f69b06ff0d1 00007f69b0b1aa0f
0709 13:22:03.968790 (+ 75187us) raft_consensus.cc:1645] Marking committed up to 15
0709 13:22:03.969169 (+   379us) log.cc:844] Serialized 14 byte log entry
0709 13:22:03.970599 (+  1430us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:22:03.970683 (+    84us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:22:03.971707 (+  1024us) raft_consensus.cc:1685] finished
0709 13:22:03.971804 (+    97us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:22:03.971943 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":14570,"prepare.queue_time_us":13122,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":40,"raft.queue_time_us":963,"raft.run_cpu_time_us":200,"raft.run_wall_time_us":200,"spinlock_wait_cycles":4465024,"thread_start_us":74954,"threads_started":1,"wal-append.queue_time_us":1053202}
I20260709 13:22:04.349308 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3884f864a96d428785895e5463e5728f" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 15 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:04.365741 16295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a9de347c711477e87c582d003f36925" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
W20260709 13:22:04.401571 16117 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:22:04.378734 16116 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:22:04.349309 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 138053us
I20260709 13:22:04.406328 16790 leader_election.cc:304] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac
I20260709 13:22:04.406774 16298 raft_consensus.cc:1275] T 2794af5b99984efebdee564035841cb0 P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 16 index: 14. (index mismatch)
I20260709 13:22:04.407078 16115 leader_election.cc:304] T ecc1e363230043fe9fef390f0b979f20 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:22:04.407577 16790 leader_election.cc:304] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac
I20260709 13:22:04.337704 16190 leader_election.cc:304] T 8cc959fb7f704fba9009c0595467d074 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:22:04.404194 16151 raft_consensus.cc:2410] T e5ad4eb281564936a59f41e387a47156 P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 for term 19 because replica has last-logged OpId of term: 18 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 11.
W20260709 13:22:04.408501 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 70915.8us
I20260709 13:22:04.264341 23153 raft_consensus.cc:515] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:04.350180 23167 raft_consensus.cc:3055] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:22:04.344496 22681 raft_consensus.cc:493] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:04.413538 16295 raft_consensus.cc:2393] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 22: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
I20260709 13:22:04.413540 16297 raft_consensus.cc:1275] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 13:22:04.415028 22677 raft_consensus.cc:2804] T 8cc959fb7f704fba9009c0595467d074 P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:22:04.415222 23166 raft_consensus.cc:2804] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:22:04.415761 22543 raft_consensus.cc:2804] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:22:04.417177 22996 raft_consensus.cc:2804] T ecc1e363230043fe9fef390f0b979f20 P 647a343a542d4a139fe106c19095129b [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:22:04.391716 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a96373bb0e14dce8a266b2fc6c58507" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:22:04.429042 16300 raft_consensus.cc:2393] T 5a96373bb0e14dce8a266b2fc6c58507 P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 16: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
I20260709 13:22:04.302757 23125 consensus_queue.cc:260] T c0eab25ce4c64e018a041beeda2867db P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:04.419400 16790 leader_election.cc:304] T 26354361e9b044cea1c2b86c4882fe92 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:22:04.429633 16291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7ca61592e848ea9b0fb81a2ce33cdc" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
W20260709 13:22:04.431088 16117 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:22:04.359234 22967 consensus_queue.cc:1048] T 26354361e9b044cea1c2b86c4882fe92 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.066s
I20260709 13:22:04.368619 23090 consensus_queue.cc:1048] T 23f82224f4564fea93572e8206bad1fe P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.057s
I20260709 13:22:04.421336 23167 raft_consensus.cc:740] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:22:04.475479 23061 leader_election.cc:290] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 20 election: Requested vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:04.366200 22731 raft_consensus.cc:515] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:04.423813 22677 raft_consensus.cc:493] T 8cc959fb7f704fba9009c0595467d074 P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:22:04.476290 16791 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:22:04.428610 22543 raft_consensus.cc:493] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
W20260709 13:22:04.480257 16117 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:22:04.484719 16226 raft_consensus.cc:1240] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 20. Current term is 21. Ops: []
I20260709 13:22:04.350970 22709 raft_consensus.cc:515] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:04.424015 22494 consensus_queue.cc:1048] T 5a96373bb0e14dce8a266b2fc6c58507 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.061s
I20260709 13:22:04.494539 16819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7ca61592e848ea9b0fb81a2ce33cdc" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:22:04.428802 22681 raft_consensus.cc:515] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:04.336474 23160 raft_consensus.cc:515] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:04.477814 22996 raft_consensus.cc:697] T ecc1e363230043fe9fef390f0b979f20 P 647a343a542d4a139fe106c19095129b [term 21 LEADER]: Becoming Leader. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:22:04.489108 22677 raft_consensus.cc:3060] T 8cc959fb7f704fba9009c0595467d074 P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:22:04.483124 23167 consensus_queue.cc:260] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:04.497980 22508 raft_consensus.cc:2749] T 26354361e9b044cea1c2b86c4882fe92 P cb82cc7bd6db46d980c75237a1402860 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:22:04.616532 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4f65cdf00b748bb963ee6e81ef43b79" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:22:04.621833 22979 raft_consensus.cc:493] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:04.598421 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62fe3346cd3c42c4a55e14af9073bca4" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:22:04.623812 16293 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48674 (request call id 3397) took 1394 ms. Trace:
I20260709 13:22:04.493396 22543 raft_consensus.cc:3060] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:22:04.608580 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d4ab870a05347868956aa8b841b2981" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
W20260709 13:22:04.582300 22873 consensus_meta.cc:287] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045: Time spent flushing consensus metadata: real 0.568s	user 0.012s	sys 0.000s
I20260709 13:22:04.563817 23061 raft_consensus.cc:493] T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:22:04.646999 16293 rpcz_store.cc:276] 0709 13:22:03.229265 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:22:03.254435 (+ 25170us) service_pool.cc:224] Handling call
0709 13:22:03.254777 (+   342us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:22:03.795296 (+540519us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:22:04.623588 (+828292us) spinlock_profiling.cc:243] Waited 526 ms on lock 0x7b54000cdd54. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:22:04.623755 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1210427264}
I20260709 13:22:04.572928 22701 raft_consensus.cc:493] T 434bff3082d141838068f323bdb0d1ed P cb82cc7bd6db46d980c75237a1402860 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:22:04.623232 23165 log.cc:927] Time spent T 00000000000000000000000000000000 P 667a34b27a39474ebefc1af3cf5b4271: Append to log took a long time: real 0.194s	user 0.000s	sys 0.002s
I20260709 13:22:04.605651 22740 raft_consensus.cc:493] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:22:04.641855 16789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 145573us
I20260709 13:22:04.596956 23122 leader_election.cc:290] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 25 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:04.607805 23089 leader_election.cc:290] T 8d4ab870a05347868956aa8b841b2981 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 18 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:04.650818 16144 raft_consensus.cc:2468] T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 20.
I20260709 13:22:04.615052 22731 leader_election.cc:290] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 21 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:04.616443 22681 leader_election.cc:290] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:04.621129 23160 leader_election.cc:290] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:04.613224 22709 leader_election.cc:290] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:04.698910 22681 consensus_queue.cc:1059] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: INVALID_TERM, Last received: 21.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 13:22:04.700641 22681 consensus_queue.cc:1059] T ada5a004ba7c46819051e3ccec71ffdb P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: INVALID_TERM, Last received: 19.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:22:04.678189 22740 raft_consensus.cc:515] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:04.704548 22740 leader_election.cc:290] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:04.705117 22740 raft_consensus.cc:3055] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:22:04.705538 22740 raft_consensus.cc:740] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:22:04.659693 22735 raft_consensus.cc:493] T b6f07842d3aa47858df5665874c86f3a P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:22:04.706507 22735 raft_consensus.cc:515] T b6f07842d3aa47858df5665874c86f3a P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:22:04.707613 23107 log.cc:927] Time spent T 66ae176961b74813a58bc15c9c4c6f2a P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 1.205s	user 0.022s	sys 0.014s
I20260709 13:22:04.708006 22971 raft_consensus.cc:515] T 84f2da98864b4425bf7c0055951fdd4b P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:04.708724 22735 leader_election.cc:290] T b6f07842d3aa47858df5665874c86f3a P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:04.659746 22979 raft_consensus.cc:515] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:04.712527 22979 leader_election.cc:290] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:04.713914 23175 raft_consensus.cc:3055] T ada5a004ba7c46819051e3ccec71ffdb P cb82cc7bd6db46d980c75237a1402860 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:22:04.713625 22748 raft_consensus.cc:515] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:04.714401 23175 raft_consensus.cc:740] T ada5a004ba7c46819051e3ccec71ffdb P cb82cc7bd6db46d980c75237a1402860 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:22:04.715349 23175 consensus_queue.cc:260] T ada5a004ba7c46819051e3ccec71ffdb P cb82cc7bd6db46d980c75237a1402860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:04.715498 22996 consensus_queue.cc:237] T ecc1e363230043fe9fef390f0b979f20 P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 21, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:04.716797 23175 raft_consensus.cc:3060] T ada5a004ba7c46819051e3ccec71ffdb P cb82cc7bd6db46d980c75237a1402860 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:22:04.650897 23167 raft_consensus.cc:3060] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:22:04.669483 22873 raft_consensus.cc:2749] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:22:04.719797 22740 consensus_queue.cc:260] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 21.16, Last appended by leader: 16, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:04.720790 16291 raft_consensus.cc:1240] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 19. Current term is 20. Ops: []
I20260709 13:22:04.676676 22967 consensus_queue.cc:1059] T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: INVALID_TERM, Last received: 20.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.047s
I20260709 13:22:04.723891 22967 raft_consensus.cc:3055] T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:22:04.724236 22967 raft_consensus.cc:740] T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:22:04.675981 22962 raft_consensus.cc:493] T af57996a8be84d398f0f023e8d08ef77 P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:22:04.725315 16114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:22:04.725850 23015 raft_consensus.cc:493] T 18ad7e3af5504571a69d10c00750a01c P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:22:04.655117 23159 raft_consensus.cc:697] T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac [term 20 LEADER]: Becoming Leader. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:22:04.726765 16223 raft_consensus.cc:1240] T 84f2da98864b4425bf7c0055951fdd4b P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 15. Current term is 16. Ops: []
W20260709 13:22:04.726948 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 399038us
I20260709 13:22:04.728933 16219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6f07842d3aa47858df5665874c86f3a" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:04.730247 16226 raft_consensus.cc:1240] T 8a878999c0024d2abe517484180290fc P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 16. Current term is 17. Ops: [16.9-16.9]
I20260709 13:22:04.752820 16265 leader_election.cc:304] T 84f2da98864b4425bf7c0055951fdd4b P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:22:04.718647 22971 leader_election.cc:290] T 84f2da98864b4425bf7c0055951fdd4b P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 16 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:04.725857 22748 leader_election.cc:290] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 23 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:04.736318 16217 raft_consensus.cc:2468] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 25.
I20260709 13:22:04.655330 23166 raft_consensus.cc:697] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 [term 22 LEADER]: Becoming Leader. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:22:04.667760 22917 raft_consensus.cc:493] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:04.749750 16150 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b6f07842d3aa47858df5665874c86f3a"
dest_uuid: "647a343a542d4a139fe106c19095129b"
mode: GRACEFUL
new_leader_uuid: "90f3da57796b488c9087a2f523adf045"
 from {username='slave'} at 127.0.0.1:40290
I20260709 13:22:04.750669 16151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62fe3346cd3c42c4a55e14af9073bca4" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 26 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:22:04.751190 16143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:22:04.767030 16150 raft_consensus.cc:606] T b6f07842d3aa47858df5665874c86f3a P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Received request to transfer leadership to TS 90f3da57796b488c9087a2f523adf045
I20260709 13:22:04.768066 16150 raft_consensus.cc:614] T b6f07842d3aa47858df5665874c86f3a P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:22:04.767198 16151 raft_consensus.cc:2468] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 25.
W20260709 13:22:04.771188 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 351853us
I20260709 13:22:04.774410 16265 leader_election.cc:304] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac
I20260709 13:22:04.775990 16790 leader_election.cc:304] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860; no voters: 90f3da57796b488c9087a2f523adf045
I20260709 13:22:04.787851 16115 leader_election.cc:304] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:22:04.723285 22740 raft_consensus.cc:3060] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:22:04.778729 16146 raft_consensus.cc:1275] T b87147731a68400e8cce9b879daa7c5d P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 20 index: 13. (term mismatch)
I20260709 13:22:04.750456 22962 raft_consensus.cc:515] T af57996a8be84d398f0f023e8d08ef77 P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:22:04.782409 16116 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:22:04.674052 23061 raft_consensus.cc:515] T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:04.790127 16146 pending_rounds.cc:85] T b87147731a68400e8cce9b879daa7c5d P 647a343a542d4a139fe106c19095129b: Aborting all ops after (but not including) 12
I20260709 13:22:04.802505 16146 pending_rounds.cc:107] T b87147731a68400e8cce9b879daa7c5d P 647a343a542d4a139fe106c19095129b: Aborting uncommitted NO_OP operation due to leader change: 19.13
I20260709 13:22:04.676769 22701 raft_consensus.cc:515] T 434bff3082d141838068f323bdb0d1ed P cb82cc7bd6db46d980c75237a1402860 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:04.805063 22701 leader_election.cc:290] T 434bff3082d141838068f323bdb0d1ed P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
W20260709 13:22:04.805445 16189 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:22:04.805706 22701 raft_consensus.cc:2804] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:22:04.806187 22701 raft_consensus.cc:493] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:22:04.806663 22701 raft_consensus.cc:3060] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:22:04.807659 16789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 125428us
I20260709 13:22:04.793125 16790 leader_election.cc:304] T 3884f864a96d428785895e5463e5728f P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:22:04.756167 23168 raft_consensus.cc:2749] T ecc1e363230043fe9fef390f0b979f20 P 90f3da57796b488c9087a2f523adf045 [term 21 FOLLOWER]: Leader pre-election lost for term 20. Reason: Vote denied by peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741) with higher term. Message: Invalid argument: T ecc1e363230043fe9fef390f0b979f20 P 647a343a542d4a139fe106c19095129b [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 for earlier term 20. Current term is 21.
I20260709 13:22:04.768261 22946 raft_consensus.cc:493] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac [term 20 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:22:04.802881 16146 raft_consensus.cc:2889] T b87147731a68400e8cce9b879daa7c5d P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:22:04.777256 23077 raft_consensus.cc:2749] T 84f2da98864b4425bf7c0055951fdd4b P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:22:04.753559 23015 raft_consensus.cc:515] T 18ad7e3af5504571a69d10c00750a01c P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } }
I20260709 13:22:04.789669 22967 consensus_queue.cc:260] T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.13, Last appended by leader: 13, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:04.784336 22873 raft_consensus.cc:2804] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:22:04.832540 16225 raft_consensus.cc:3060] T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:22:04.790714 22610 raft_consensus.cc:2749] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:22:04.768674 23138 consensus_queue.cc:1059] T 8a878999c0024d2abe517484180290fc P 647a343a542d4a139fe106c19095129b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.049s
W20260709 13:22:04.827998 23087 consensus_meta.cc:287] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b: Time spent flushing consensus metadata: real 0.566s	user 0.011s	sys 0.000s
I20260709 13:22:04.836582 23159 consensus_queue.cc:237] T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 19.10, Last appended by leader: 10, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:04.827543 22748 raft_consensus.cc:493] T 68a6838d82ef4cac83507f906139d90f P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:22:04.845918 16143 raft_consensus.cc:2410] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 for term 22 because replica has last-logged OpId of term: 21 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 11.
I20260709 13:22:04.772094 22917 raft_consensus.cc:515] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:04.801221 23089 consensus_queue.cc:1048] T 2794af5b99984efebdee564035841cb0 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.041s
W20260709 13:22:04.844413 23170 log.cc:927] Time spent T 5a959dbecc2648eb85d1cc550f5a3245 P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.094s	user 0.000s	sys 0.001s
I20260709 13:22:04.840761 23153 leader_election.cc:290] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:04.839656 22873 raft_consensus.cc:697] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045 [term 18 LEADER]: Becoming Leader. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:22:04.801350 23162 consensus_queue.cc:1048] T 7c2095a5f4ad49408dbc7f69c606120d P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.068s
W20260709 13:22:04.875386 16116 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:22:04.844161 23090 consensus_queue.cc:1175] T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "90f3da57796b488c9087a2f523adf045" responder_term: 20 status { last_received { term: 20 index: 13 } last_committed_idx: 13 last_received_current_leader { term: 20 index: 13 } }
I20260709 13:22:04.861778 22967 raft_consensus.cc:3060] T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:22:04.830991 22946 raft_consensus.cc:515] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:04.887702 22917 leader_election.cc:290] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:04.888350 23015 leader_election.cc:290] T 18ad7e3af5504571a69d10c00750a01c P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:22:04.890667 22873 consensus_queue.cc:237] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:04.882375 23181 raft_consensus.cc:2749] T 3884f864a96d428785895e5463e5728f P cb82cc7bd6db46d980c75237a1402860 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:22:04.896014 23061 leader_election.cc:290] T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:04.899559 22946 leader_election.cc:290] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:04.899609 22962 leader_election.cc:290] T af57996a8be84d398f0f023e8d08ef77 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:04.861265 22748 raft_consensus.cc:515] T 68a6838d82ef4cac83507f906139d90f P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:04.904291 22748 leader_election.cc:290] T 68a6838d82ef4cac83507f906139d90f P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:04.905396 22543 raft_consensus.cc:515] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:04.906525 16820 raft_consensus.cc:1240] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 [term 18 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 17. Current term is 18. Ops: []
I20260709 13:22:04.908335 22543 leader_election.cc:290] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 18 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
W20260709 13:22:04.912725 23003 consensus_meta.cc:287] T 5a959dbecc2648eb85d1cc550f5a3245 P 647a343a542d4a139fe106c19095129b: Time spent flushing consensus metadata: real 0.929s	user 0.010s	sys 0.002s
I20260709 13:22:04.912720 16225 raft_consensus.cc:2468] T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 18.
I20260709 13:22:04.920822 23003 raft_consensus.cc:515] T 5a959dbecc2648eb85d1cc550f5a3245 P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:04.922063 16149 raft_consensus.cc:1240] T 5a959dbecc2648eb85d1cc550f5a3245 P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 18. Current term is 19. Ops: []
I20260709 13:22:04.845850 23166 consensus_queue.cc:237] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 21.14, Last appended by leader: 14, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:04.918637 23167 raft_consensus.cc:493] T ee9ed67006144797b1de6ea8c2013a68 P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:22:04.915635 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18ad7e3af5504571a69d10c00750a01c" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:04.925581 16226 raft_consensus.cc:2468] T 18ad7e3af5504571a69d10c00750a01c P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 16.
I20260709 13:22:04.935423 16192 leader_election.cc:304] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860
W20260709 13:22:04.940186 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 147127us
I20260709 13:22:04.943780 16819 raft_consensus.cc:1240] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P cb82cc7bd6db46d980c75237a1402860 [term 20 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 19. Current term is 20. Ops: []
I20260709 13:22:04.945925 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a959dbecc2648eb85d1cc550f5a3245" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:22:04.956743 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f2da98864b4425bf7c0055951fdd4b" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:22:04.959028 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada5a004ba7c46819051e3ccec71ffdb" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:04.962247 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d4ab870a05347868956aa8b841b2981" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
W20260709 13:22:04.964673 16190 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:22:04.961052 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66ae176961b74813a58bc15c9c4c6f2a" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 26 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:04.968222 16299 raft_consensus.cc:2393] T ada5a004ba7c46819051e3ccec71ffdb P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 20: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
W20260709 13:22:04.993139 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 216345us
I20260709 13:22:04.997646 16825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a378d7f796f94615b3b347119a9880fd" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 22 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:05.000393 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68a6838d82ef4cac83507f906139d90f" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:04.943403 22962 consensus_queue.cc:1059] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: INVALID_TERM, Last received: 19.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.012s
I20260709 13:22:04.938426 22748 raft_consensus.cc:2749] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 1ca2725d290e47d68b1a4b609c9816ac [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:22:05.009542 16818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2372981006d4b218215c28964dea623" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:22:05.010039 16291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a378d7f796f94615b3b347119a9880fd" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 22 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
W20260709 13:22:05.011747 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 70041.6us
I20260709 13:22:05.012574 16819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5718c623ebee4d2bb770f0f9ddb2f164" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:05.012854 16292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18ad7e3af5504571a69d10c00750a01c" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:05.014206 16295 raft_consensus.cc:1275] T 5a96373bb0e14dce8a266b2fc6c58507 P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 13:22:05.015578 16220 raft_consensus.cc:1275] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [term 25 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 24 index: 15. Preceding OpId from leader: term: 25 index: 16. (index mismatch)
I20260709 13:22:05.016281 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6f07842d3aa47858df5665874c86f3a" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
W20260709 13:22:05.017771 16296 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.541s	user 0.032s	sys 0.028s
I20260709 13:22:05.018540 16295 raft_consensus.cc:1240] T 9e65676b21b748728ad39668cfe722ec P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 17. Current term is 18. Ops: []
W20260709 13:22:05.018978 16296 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.542s	user 0.033s	sys 0.028s
I20260709 13:22:05.029116 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d4ab870a05347868956aa8b841b2981" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:22:05.029453 16294 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48684 (request call id 3137) took 1141 ms. Trace:
I20260709 13:22:05.029899 16294 rpcz_store.cc:276] 0709 13:22:03.887840 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:22:03.891323 (+  3483us) service_pool.cc:224] Handling call
0709 13:22:05.029260 (+1137937us) spinlock_profiling.cc:243] Waited 231 ms on lock 0x7b5400347e54. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:22:05.029398 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":532257024}
I20260709 13:22:05.035317 16818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada5a004ba7c46819051e3ccec71ffdb" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:05.036026 16818 raft_consensus.cc:2468] T ada5a004ba7c46819051e3ccec71ffdb P cb82cc7bd6db46d980c75237a1402860 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 20.
I20260709 13:22:04.937492 23167 raft_consensus.cc:515] T ee9ed67006144797b1de6ea8c2013a68 P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:05.004916 22946 raft_consensus.cc:493] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:22:05.042011 16221 raft_consensus.cc:1275] T 7c2095a5f4ad49408dbc7f69c606120d P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 14 index: 7. Preceding OpId from leader: term: 16 index: 8. (index mismatch)
I20260709 13:22:05.039475 16818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5d59407eb545b895d8d922e0dd69fd" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:05.043448 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:05.044443 16224 raft_consensus.cc:2393] T 19baa7bc367844c288bca8f81439c413 P 1ca2725d290e47d68b1a4b609c9816ac [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 22: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
I20260709 13:22:05.050153 16825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af57996a8be84d398f0f023e8d08ef77" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:05.051047 16825 raft_consensus.cc:2468] T af57996a8be84d398f0f023e8d08ef77 P cb82cc7bd6db46d980c75237a1402860 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 19.
I20260709 13:22:05.049945 16192 leader_election.cc:304] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860
I20260709 13:22:05.067126 16192 leader_election.cc:304] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 90f3da57796b488c9087a2f523adf045
I20260709 13:22:05.019623 23016 raft_consensus.cc:493] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:22:05.071000 16819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66ae176961b74813a58bc15c9c4c6f2a" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 26 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:05.075507 16819 raft_consensus.cc:2468] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 25.
I20260709 13:22:05.072634 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a959dbecc2648eb85d1cc550f5a3245" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:22:05.038352 22962 raft_consensus.cc:493] T 1df93779209748b9860f6d893dd3de53 P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
W20260709 13:22:05.074133 16789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 257516us
I20260709 13:22:05.072978 23061 raft_consensus.cc:2804] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:22:05.084604 23061 raft_consensus.cc:493] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Starting leader election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:22:05.085111 23061 raft_consensus.cc:3060] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:22:05.070385 23193 raft_consensus.cc:493] T 8638ee363273426db1393dd81e5c7b72 P cb82cc7bd6db46d980c75237a1402860 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:05.097489 16825 raft_consensus.cc:1275] T 23f82224f4564fea93572e8206bad1fe P cb82cc7bd6db46d980c75237a1402860 [term 17 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 13:22:05.070894 22917 raft_consensus.cc:2749] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:22:05.101814 16817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31969392ccdb43b491911a319e8bc7e1" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:22:05.102888 16823 raft_consensus.cc:1240] T 19baa7bc367844c288bca8f81439c413 P cb82cc7bd6db46d980c75237a1402860 [term 22 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 21. Current term is 22. Ops: [21.12-21.12]
I20260709 13:22:05.073086 23003 leader_election.cc:290] T 5a959dbecc2648eb85d1cc550f5a3245 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 19 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:05.100957 16816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
W20260709 13:22:05.107124 16788 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:22:05.117631 16816 raft_consensus.cc:3060] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:22:05.119875 16819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b7d1ab519044eda2d5e04a07242f25" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:22:05.122707 16823 raft_consensus.cc:1275] T 806d7690c2b644959152074bb2cab0a7 P cb82cc7bd6db46d980c75237a1402860 [term 18 LEARNER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 13:22:05.125281 16823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3884f864a96d428785895e5463e5728f" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 15 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:05.129042 16821 raft_consensus.cc:1275] T 26354361e9b044cea1c2b86c4882fe92 P cb82cc7bd6db46d980c75237a1402860 [term 16 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:22:05.131742 16825 raft_consensus.cc:3055] T 2794af5b99984efebdee564035841cb0 P cb82cc7bd6db46d980c75237a1402860 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:22:05.132278 16825 raft_consensus.cc:740] T 2794af5b99984efebdee564035841cb0 P cb82cc7bd6db46d980c75237a1402860 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:22:05.126083 16823 raft_consensus.cc:2468] T 3884f864a96d428785895e5463e5728f P cb82cc7bd6db46d980c75237a1402860 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 14.
I20260709 13:22:05.063460 22946 raft_consensus.cc:515] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:05.063185 23191 consensus_queue.cc:1059] T 8d4ab870a05347868956aa8b841b2981 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: INVALID_TERM, Last received: 17.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.063s
I20260709 13:22:05.072029 22610 consensus_queue.cc:1048] T 5a96373bb0e14dce8a266b2fc6c58507 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.035s
I20260709 13:22:05.073256 22748 consensus_queue.cc:1059] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: INVALID_TERM, Last received: 19.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.045s
I20260709 13:22:05.078466 23016 raft_consensus.cc:515] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:05.084043 22962 raft_consensus.cc:515] T 1df93779209748b9860f6d893dd3de53 P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:05.148192 23184 raft_consensus.cc:493] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:05.145027 22677 raft_consensus.cc:515] T 8cc959fb7f704fba9009c0595467d074 P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:05.147653 16823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62fe3346cd3c42c4a55e14af9073bca4" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:22:05.154215 16822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74ca8140f35c4ae29e8a664f6bdfc829" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:05.170482 16822 raft_consensus.cc:2468] T 74ca8140f35c4ae29e8a664f6bdfc829 P cb82cc7bd6db46d980c75237a1402860 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 20.
I20260709 13:22:05.138662 16825 consensus_queue.cc:260] T 2794af5b99984efebdee564035841cb0 P cb82cc7bd6db46d980c75237a1402860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:05.172360 16825 raft_consensus.cc:3060] T 2794af5b99984efebdee564035841cb0 P cb82cc7bd6db46d980c75237a1402860 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:22:05.171886 16822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57a744f5aaf944678a63469cbf514015" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 16 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:05.162410 16819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a9de347c711477e87c582d003f36925" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:05.174650 16819 raft_consensus.cc:2468] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 22.
I20260709 13:22:05.175859 16819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e9b2bf7011f4ae688a79d7bb5a447b4" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:05.161031 16192 leader_election.cc:304] T af57996a8be84d398f0f023e8d08ef77 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:22:05.177659 16192 leader_election.cc:304] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 90f3da57796b488c9087a2f523adf045
I20260709 13:22:05.166021 16816 raft_consensus.cc:2468] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 21.
I20260709 13:22:05.167302 16824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada5a004ba7c46819051e3ccec71ffdb" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:22:05.180110 16816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:05.181592 16822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdbd78acb4cf46298e0de8ba0d130e19" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
W20260709 13:22:05.171891 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 159443us
I20260709 13:22:05.183509 16816 raft_consensus.cc:2368] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 for earlier term 20. Current term is 21.
I20260709 13:22:05.183737 16265 leader_election.cc:304] T c9944d5e4b8c4109af9955c4f08c9095 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:22:05.186239 16790 leader_election.cc:304] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac
I20260709 13:22:05.189606 16823 raft_consensus.cc:2393] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 25: Already voted for candidate cb82cc7bd6db46d980c75237a1402860 in this term.
I20260709 13:22:05.192888 16819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9944d5e4b8c4109af9955c4f08c9095" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:22:05.195962 16790 leader_election.cc:304] T af57996a8be84d398f0f023e8d08ef77 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac
I20260709 13:22:05.197831 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a9de347c711477e87c582d003f36925" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:05.199856 16292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434bff3082d141838068f323bdb0d1ed" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 19 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:05.200620 16292 raft_consensus.cc:2410] T 434bff3082d141838068f323bdb0d1ed P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 for term 19 because replica has last-logged OpId of term: 18 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 13.
I20260709 13:22:05.204028 16266 leader_election.cc:304] T 57a744f5aaf944678a63469cbf514015 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
I20260709 13:22:05.198460 16294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66ae176961b74813a58bc15c9c4c6f2a" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 26 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:05.205852 16294 raft_consensus.cc:1240] T c9944d5e4b8c4109af9955c4f08c9095 P 90f3da57796b488c9087a2f523adf045 [term 22 FOLLOWER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 21. Current term is 22. Ops: []
I20260709 13:22:05.206529 16819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:05.207499 16819 raft_consensus.cc:2393] T 19baa7bc367844c288bca8f81439c413 P cb82cc7bd6db46d980c75237a1402860 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 22: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
I20260709 13:22:05.121013 23193 raft_consensus.cc:515] T 8638ee363273426db1393dd81e5c7b72 P cb82cc7bd6db46d980c75237a1402860 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:05.188298 22748 raft_consensus.cc:2804] T af57996a8be84d398f0f023e8d08ef77 P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:22:05.188380 22439 raft_consensus.cc:493] T 2794af5b99984efebdee564035841cb0 P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:22:05.192371 23076 raft_consensus.cc:3055] T 8d4ab870a05347868956aa8b841b2981 P 90f3da57796b488c9087a2f523adf045 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:22:05.193112 22917 raft_consensus.cc:2804] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 13:22:05.208374 23153 raft_consensus.cc:493] T d2ab0ac8347641e6955e9bce7d90c1a2 P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:05.209188 22740 raft_consensus.cc:2804] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:22:05.192194 23191 raft_consensus.cc:493] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:22:05.233927 22917 raft_consensus.cc:493] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:05.227651 16825 raft_consensus.cc:1275] T 2794af5b99984efebdee564035841cb0 P cb82cc7bd6db46d980c75237a1402860 [term 16 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 16 index: 14. (index mismatch)
I20260709 13:22:05.234804 16822 raft_consensus.cc:2468] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 22.
I20260709 13:22:05.182395 23184 raft_consensus.cc:515] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:22:05.256985 16116 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:22:05.225930 22748 raft_consensus.cc:493] T af57996a8be84d398f0f023e8d08ef77 P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:05.232034 23076 raft_consensus.cc:740] T 8d4ab870a05347868956aa8b841b2981 P 90f3da57796b488c9087a2f523adf045 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:22:05.265051 16822 raft_consensus.cc:1275] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [term 25 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 24 index: 15. Preceding OpId from leader: term: 25 index: 16. (index mismatch)
I20260709 13:22:05.255508 22917 raft_consensus.cc:3060] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:22:05.287278 16224 raft_consensus.cc:1275] T fffab26427e24d398e2695aad40ca134 P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 13:22:05.284370 16297 raft_consensus.cc:2468] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 22.
I20260709 13:22:05.236443 22439 raft_consensus.cc:515] T 2794af5b99984efebdee564035841cb0 P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:05.265743 22748 raft_consensus.cc:3060] T af57996a8be84d398f0f023e8d08ef77 P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:22:05.247862 23153 raft_consensus.cc:515] T d2ab0ac8347641e6955e9bce7d90c1a2 P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:05.271173 23167 leader_election.cc:290] T ee9ed67006144797b1de6ea8c2013a68 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
W20260709 13:22:05.318557 23201 log.cc:927] Time spent T 26354361e9b044cea1c2b86c4882fe92 P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.070s	user 0.000s	sys 0.000s
I20260709 13:22:05.319289 22740 raft_consensus.cc:697] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860 [term 16 LEADER]: Becoming Leader. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:22:05.320488 22740 consensus_queue.cc:237] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:05.323469 16299 raft_consensus.cc:1240] T 2794af5b99984efebdee564035841cb0 P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 15. Current term is 16. Ops: []
I20260709 13:22:05.323725 23173 raft_consensus.cc:493] T e5ad4eb281564936a59f41e387a47156 P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:05.324330 23173 raft_consensus.cc:515] T e5ad4eb281564936a59f41e387a47156 P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:05.325855 22908 raft_consensus.cc:3055] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:22:05.326336 22908 raft_consensus.cc:740] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:22:05.337898 22908 consensus_queue.cc:260] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:05.340996 23179 raft_consensus.cc:493] T 23f82224f4564fea93572e8206bad1fe P cb82cc7bd6db46d980c75237a1402860 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:05.341619 23179 raft_consensus.cc:515] T 23f82224f4564fea93572e8206bad1fe P cb82cc7bd6db46d980c75237a1402860 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:05.344408 23210 raft_consensus.cc:493] T d2ab0ac8347641e6955e9bce7d90c1a2 P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:22:05.345002 23210 raft_consensus.cc:515] T d2ab0ac8347641e6955e9bce7d90c1a2 P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:05.287952 23203 raft_consensus.cc:2804] T c9944d5e4b8c4109af9955c4f08c9095 P 90f3da57796b488c9087a2f523adf045 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:22:05.347644 23203 raft_consensus.cc:697] T c9944d5e4b8c4109af9955c4f08c9095 P 90f3da57796b488c9087a2f523adf045 [term 22 LEADER]: Becoming Leader. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
W20260709 13:22:05.348825 23190 log.cc:927] Time spent T 5a96373bb0e14dce8a266b2fc6c58507 P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 0.200s	user 0.002s	sys 0.000s
I20260709 13:22:05.348599 23203 consensus_queue.cc:237] T c9944d5e4b8c4109af9955c4f08c9095 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 21.15, Last appended by leader: 15, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:05.256145 23191 raft_consensus.cc:515] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } }
W20260709 13:22:05.378422 16190 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:22:05.364356 23196 log.cc:927] Time spent T 3884f864a96d428785895e5463e5728f P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.106s	user 0.000s	sys 0.001s
I20260709 13:22:05.304134 23199 raft_consensus.cc:2804] T af57996a8be84d398f0f023e8d08ef77 P cb82cc7bd6db46d980c75237a1402860 [term 19 FOLLOWER]: Leader election won for term 19
W20260709 13:22:05.389472 16266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 185527us
I20260709 13:22:05.390496 16266 leader_election.cc:400] T ecc1e363230043fe9fef390f0b979f20 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 20 pre-election: Vote denied by peer cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923) with higher term. Message: Invalid argument: T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 for earlier term 20. Current term is 21.
I20260709 13:22:05.391564 16266 leader_election.cc:304] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
W20260709 13:22:05.406541 16116 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:22:05.407039 22946 leader_election.cc:290] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
W20260709 13:22:05.422883 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 227608us
I20260709 13:22:05.356199 23076 consensus_queue.cc:260] T 8d4ab870a05347868956aa8b841b2981 P 90f3da57796b488c9087a2f523adf045 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:05.410806 23199 raft_consensus.cc:697] T af57996a8be84d398f0f023e8d08ef77 P cb82cc7bd6db46d980c75237a1402860 [term 19 LEADER]: Becoming Leader. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:22:05.426594 16790 leader_election.cc:304] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860; no voters: 90f3da57796b488c9087a2f523adf045
I20260709 13:22:05.424878 23202 raft_consensus.cc:2749] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:22:05.427703 22677 leader_election.cc:290] T 8cc959fb7f704fba9009c0595467d074 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 18 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:05.448568 22946 raft_consensus.cc:493] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:05.480432 16824 raft_consensus.cc:2468] T ada5a004ba7c46819051e3ccec71ffdb P cb82cc7bd6db46d980c75237a1402860 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 20.
I20260709 13:22:05.504856 16790 leader_election.cc:304] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:22:05.450830 23015 raft_consensus.cc:493] T b87147731a68400e8cce9b879daa7c5d P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:05.451643 22962 leader_election.cc:290] T 1df93779209748b9860f6d893dd3de53 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:05.455470 23211 raft_consensus.cc:493] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P cb82cc7bd6db46d980c75237a1402860 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:05.430917 23210 leader_election.cc:290] T d2ab0ac8347641e6955e9bce7d90c1a2 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:05.453248 23179 leader_election.cc:290] T 23f82224f4564fea93572e8206bad1fe P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:05.456457 23175 raft_consensus.cc:2764] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860 [term 21 FOLLOWER]: Leader pre-election decision vote started in defunct term 20: won
I20260709 13:22:05.505203 22946 raft_consensus.cc:515] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } }
I20260709 13:22:05.449127 23193 leader_election.cc:290] T 8638ee363273426db1393dd81e5c7b72 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
W20260709 13:22:05.524581 23218 log.cc:927] Time spent T 23f82224f4564fea93572e8206bad1fe P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.133s	user 0.000s	sys 0.001s
I20260709 13:22:05.522229 23211 raft_consensus.cc:515] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P cb82cc7bd6db46d980c75237a1402860 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:05.527676 23125 raft_consensus.cc:3060] T c0eab25ce4c64e018a041beeda2867db P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:22:05.529827 22701 consensus_meta.cc:287] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860: Time spent flushing consensus metadata: real 0.711s	user 0.016s	sys 0.002s
I20260709 13:22:05.522258 23089 raft_consensus.cc:493] T 02838c86531d4f98bf58d1c22b8baf77 P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:22:05.530658 23188 consensus_queue.cc:1048] T d2ab0ac8347641e6955e9bce7d90c1a2 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:22:05.533861 16143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a9de347c711477e87c582d003f36925" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:22:05.535866 16146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434bff3082d141838068f323bdb0d1ed" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 19 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:22:05.537739 16142 raft_consensus.cc:1240] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [term 21 FOLLOWER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 20. Current term is 21. Ops: []
I20260709 13:22:05.538398 16151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b7d1ab519044eda2d5e04a07242f25" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:22:05.538470 16147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f82224f4564fea93572e8206bad1fe" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
W20260709 13:22:05.539534 23220 log.cc:927] Time spent T 434bff3082d141838068f323bdb0d1ed P cb82cc7bd6db46d980c75237a1402860: Append to log took a long time: real 0.111s	user 0.000s	sys 0.001s
I20260709 13:22:05.539395 16116 leader_election.cc:304] T b6f07842d3aa47858df5665874c86f3a P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:22:05.541625 16145 raft_consensus.cc:1240] T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 16. Current term is 17. Ops: []
I20260709 13:22:05.506894 23174 consensus_queue.cc:1048] T 57a744f5aaf944678a63469cbf514015 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.006s
I20260709 13:22:05.552003 23003 raft_consensus.cc:493] T 68a6838d82ef4cac83507f906139d90f P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
W20260709 13:22:05.552279 23209 log.cc:927] Time spent T b87147731a68400e8cce9b879daa7c5d P cb82cc7bd6db46d980c75237a1402860: Append to log took a long time: real 0.099s	user 0.000s	sys 0.000s
I20260709 13:22:05.513834 23187 consensus_queue.cc:1048] T b87147731a68400e8cce9b879daa7c5d P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.042s
W20260709 13:22:05.556037 23189 log.cc:927] Time spent T 87ac0d1f7e8a419f9bcbc04aee0e067e P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.398s	user 0.000s	sys 0.002s
I20260709 13:22:05.556841 23191 leader_election.cc:290] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
W20260709 13:22:05.557196 23197 log.cc:927] Time spent T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 0.347s	user 0.000s	sys 0.001s
I20260709 13:22:05.513768 23204 raft_consensus.cc:2804] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [term 25 FOLLOWER]: Leader pre-election won for term 26
W20260709 13:22:05.512176 23196 log.cc:927] Time spent T 3884f864a96d428785895e5463e5728f P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
I20260709 13:22:05.511873 23219 raft_consensus.cc:493] T d16dfad0c3b540e5b4b86492ddf87208 P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:22:05.557691 23161 raft_consensus.cc:3055] T 31969392ccdb43b491911a319e8bc7e1 P 90f3da57796b488c9087a2f523adf045 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:22:05.512423 22543 consensus_queue.cc:1059] T 9e65676b21b748728ad39668cfe722ec P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: INVALID_TERM, Last received: 17.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.048s
I20260709 13:22:05.515070 23015 raft_consensus.cc:515] T b87147731a68400e8cce9b879daa7c5d P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:05.560763 22439 leader_election.cc:290] T 2794af5b99984efebdee564035841cb0 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:05.518267 23184 leader_election.cc:290] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:05.562271 23153 leader_election.cc:290] T d2ab0ac8347641e6955e9bce7d90c1a2 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
W20260709 13:22:05.581503 16144 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40830 (request call id 2054) took 1562 ms (client timeout 1576 ms). Trace:
I20260709 13:22:05.599785 16223 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57604 (request call id 3340) took 1474 ms. Trace:
W20260709 13:22:05.600942 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 1.19713e+06us
I20260709 13:22:05.625486 16820 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49906 (request call id 1331) took 1206 ms. Trace:
I20260709 13:22:05.692606 16143 raft_consensus.cc:2468] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 21.
I20260709 13:22:05.566337 16226 raft_consensus.cc:1240] T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 17. Current term is 18. Ops: []
I20260709 13:22:05.632314 16825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ab0ac8347641e6955e9bce7d90c1a2" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 17 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:05.690685 16151 raft_consensus.cc:2393] T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 in current term 18: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
W20260709 13:22:05.689785 16144 rpcz_store.cc:269] 0709 13:22:04.018897 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:22:04.019016 (+   119us) service_pool.cc:224] Handling call
0709 13:22:05.581296 (+1562280us) spinlock_profiling.cc:243] Waited 532 ms on lock 0x7b54001f22d4. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:22:05.581435 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1224689408}
I20260709 13:22:05.690862 16223 rpcz_store.cc:276] 0709 13:22:04.125802 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:22:04.126193 (+   391us) service_pool.cc:224] Handling call
0709 13:22:04.126598 (+   405us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:22:04.752445 (+625847us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:22:05.599459 (+847014us) spinlock_profiling.cc:243] Waited 600 ms on lock 0x7b5400396554. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:22:05.599706 (+   247us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1379827072}
I20260709 13:22:05.692631 16820 rpcz_store.cc:276] 0709 13:22:04.420694 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:22:04.426132 (+  5438us) service_pool.cc:224] Handling call
0709 13:22:04.433941 (+  7809us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:22:04.906915 (+472974us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:22:05.625080 (+718165us) spinlock_profiling.cc:243] Waited 472 ms on lock 0x7b54005032d4. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:22:05.625363 (+   283us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1086443776}
I20260709 13:22:05.632844 16818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee9ed67006144797b1de6ea8c2013a68" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:05.456823 23076 raft_consensus.cc:3060] T 8d4ab870a05347868956aa8b841b2981 P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:22:05.564348 22946 leader_election.cc:290] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:05.513629 23173 leader_election.cc:290] T e5ad4eb281564936a59f41e387a47156 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:05.611953 23133 raft_consensus.cc:493] T 746aabf856ab401c9ef5e859371e7b67 P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:05.660447 22962 raft_consensus.cc:493] T c2f5f89186204242bf377f7c42b2e2da P 1ca2725d290e47d68b1a4b609c9816ac [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:22:05.692816 23204 raft_consensus.cc:493] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:05.660728 23159 raft_consensus.cc:493] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:22:05.513898 23202 raft_consensus.cc:493] T c0eab25ce4c64e018a041beeda2867db P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:05.666216 22677 raft_consensus.cc:493] T 02838c86531d4f98bf58d1c22b8baf77 P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:22:05.715489 16824 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "cdbd78acb4cf46298e0de8ba0d130e19"
dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
mode: GRACEFUL
new_leader_uuid: "90f3da57796b488c9087a2f523adf045"
 from {username='slave'} at 127.0.0.1:37818
I20260709 13:22:05.764267 16824 raft_consensus.cc:606] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 [term 22 FOLLOWER]: Received request to transfer leadership to TS 90f3da57796b488c9087a2f523adf045
I20260709 13:22:05.717435 23184 raft_consensus.cc:493] T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:05.775849 16294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1df93779209748b9860f6d893dd3de53" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:05.616074 23090 raft_consensus.cc:493] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:05.777128 16789 leader_election.cc:304] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:22:05.777390 23090 raft_consensus.cc:515] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:05.612370 22610 raft_consensus.cc:493] T 9e65676b21b748728ad39668cfe722ec P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:05.623231 22967 raft_consensus.cc:493] T 8158bbd17b994d2eb96f560bd0f1b88d P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:22:05.787194 16221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8638ee363273426db1393dd81e5c7b72" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:05.611678 23096 raft_consensus.cc:493] T 7c2095a5f4ad49408dbc7f69c606120d P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:05.621943 23121 raft_consensus.cc:493] T 1df93779209748b9860f6d893dd3de53 P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:22:05.642446 23138 raft_consensus.cc:493] T e5ad4eb281564936a59f41e387a47156 P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:05.705940 22701 raft_consensus.cc:515] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:05.569985 16149 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40830 (request call id 2062) took 1545 ms. Trace:
I20260709 13:22:05.795696 16149 rpcz_store.cc:276] 0709 13:22:04.025150 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:22:04.025264 (+   114us) service_pool.cc:224] Handling call
0709 13:22:04.032763 (+  7499us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:22:04.922548 (+889785us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:22:05.569759 (+647211us) spinlock_profiling.cc:243] Waited 889 ms on lock 0x7b54002dafd4. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:22:05.569927 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2044713856}
I20260709 13:22:05.567749 23086 consensus_queue.cc:260] T 8a878999c0024d2abe517484180290fc P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:05.606479 16790 leader_election.cc:304] T 19baa7bc367844c288bca8f81439c413 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:22:05.692045 23161 raft_consensus.cc:740] T 31969392ccdb43b491911a319e8bc7e1 P 90f3da57796b488c9087a2f523adf045 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:22:05.579824 23016 leader_election.cc:290] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:22:05.604542 23152 raft_consensus.cc:493] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:22:05.756420 16220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66ae176961b74813a58bc15c9c4c6f2a" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 26 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:05.764940 16824 raft_consensus.cc:614] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 [term 22 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:22:05.639384 22908 raft_consensus.cc:3060] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:22:05.768155 23191 raft_consensus.cc:493] T 8a878999c0024d2abe517484180290fc P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:22:05.768936 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cc959fb7f704fba9009c0595467d074" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:22:05.816903 16297 raft_consensus.cc:3060] T 8cc959fb7f704fba9009c0595467d074 P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:22:05.752823 22962 raft_consensus.cc:515] T c2f5f89186204242bf377f7c42b2e2da P 1ca2725d290e47d68b1a4b609c9816ac [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:05.781759 23184 raft_consensus.cc:515] T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:05.763631 23167 raft_consensus.cc:493] T fffab26427e24d398e2695aad40ca134 P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:05.723845 23203 raft_consensus.cc:493] T 9e65676b21b748728ad39668cfe722ec P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:05.575917 23061 raft_consensus.cc:515] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:05.783996 16296 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48684 (request call id 3142) took 1604 ms. Trace:
I20260709 13:22:05.749310 23204 raft_consensus.cc:3060] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:22:05.639416 22494 raft_consensus.cc:493] T 829b4ad8c5a54eababeea1181b960d4b P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:05.723062 23153 raft_consensus.cc:493] T 829b4ad8c5a54eababeea1181b960d4b P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:22:05.787251 16218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b7d1ab519044eda2d5e04a07242f25" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:22:05.702353 23089 raft_consensus.cc:515] T 02838c86531d4f98bf58d1c22b8baf77 P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:05.699625 23003 raft_consensus.cc:515] T 68a6838d82ef4cac83507f906139d90f P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:05.577215 22748 raft_consensus.cc:515] T af57996a8be84d398f0f023e8d08ef77 P 1ca2725d290e47d68b1a4b609c9816ac [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:05.571216 22917 raft_consensus.cc:515] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [term 26 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:05.791599 16818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ab0ac8347641e6955e9bce7d90c1a2" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:05.816054 23090 leader_election.cc:290] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:05.753062 23173 raft_consensus.cc:493] T b6f07842d3aa47858df5665874c86f3a P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:22:05.753531 22946 raft_consensus.cc:493] T 746aabf856ab401c9ef5e859371e7b67 P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:05.699358 23219 raft_consensus.cc:515] T d16dfad0c3b540e5b4b86492ddf87208 P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:05.804462 16825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2794af5b99984efebdee564035841cb0" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:05.804512 16820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a9de347c711477e87c582d003f36925" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
W20260709 13:22:05.803771 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 404618us
I20260709 13:22:05.857846 16825 raft_consensus.cc:2468] T 2794af5b99984efebdee564035841cb0 P cb82cc7bd6db46d980c75237a1402860 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 16.
I20260709 13:22:05.773854 23202 raft_consensus.cc:515] T c0eab25ce4c64e018a041beeda2867db P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:05.792528 23166 raft_consensus.cc:2804] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860 [term 22 FOLLOWER]: Leader pre-election won for term 23
W20260709 13:22:05.812021 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 205619us
I20260709 13:22:05.752377 23223 consensus_queue.cc:1059] T 9e65676b21b748728ad39668cfe722ec P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: INVALID_TERM, Last received: 17.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.045s
W20260709 13:22:05.813942 16298 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.607s	user 0.016s	sys 0.019s
W20260709 13:22:05.871582 16298 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.672s	user 0.017s	sys 0.020s
I20260709 13:22:05.873260 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5ad4eb281564936a59f41e387a47156" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:05.863472 16222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806d7690c2b644959152074bb2cab0a7" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:05.752612 23177 consensus_queue.cc:1059] T c9944d5e4b8c4109af9955c4f08c9095 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: INVALID_TERM, Last received: 21.15, Next index: 16, Last known committed idx: 14, Time since last communication: 0.045s
I20260709 13:22:05.816283 16817 raft_consensus.cc:2393] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 17: Already voted for candidate cb82cc7bd6db46d980c75237a1402860 in this term.
I20260709 13:22:05.769315 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8158bbd17b994d2eb96f560bd0f1b88d" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:05.764369 23133 raft_consensus.cc:515] T 746aabf856ab401c9ef5e859371e7b67 P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:05.781008 22677 raft_consensus.cc:515] T 02838c86531d4f98bf58d1c22b8baf77 P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:05.765318 23159 raft_consensus.cc:515] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:05.822499 16296 rpcz_store.cc:276] 0709 13:22:04.179940 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:22:04.180133 (+   193us) service_pool.cc:224] Handling call
0709 13:22:04.199637 (+ 19504us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:22:04.424467 (+224830us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:22:04.424513 (+    46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:22:04.425734 (+  1221us) log.cc:844] Serialized 26 byte log entry
0709 13:22:05.019425 (+593691us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:22:05.019478 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:22:05.019587 (+   109us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:22:05.555474 (+535887us) spinlock_profiling.cc:243] Waited 72 ms on lock 0x7b4400022b4c. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69ad8fadb1 00007f69ad998c85 00007f69ad9996a2 00007f69b070b1db 00007f69b0706aaf 00007f69b06fa86e 00007f69b06f9bb6 00007f69b06f97d8 00007f69b0703221 00007f69b07031d9 00007f69b0703169
0709 13:22:05.775672 (+220198us) raft_consensus.cc:1685] finished
0709 13:22:05.775704 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:22:05.783928 (+  8224us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":21587,"raft.queue_time_us":25065,"raft.run_cpu_time_us":131,"raft.run_wall_time_us":11226,"spinlock_wait_cycles":165548288,"thread_start_us":752359,"threads_started":2,"wal-append.queue_time_us":727085}
I20260709 13:22:05.830783 16266 leader_election.cc:304] T cdbd78acb4cf46298e0de8ba0d130e19 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac
I20260709 13:22:05.834254 16297 raft_consensus.cc:2468] T 8cc959fb7f704fba9009c0595467d074 P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 18.
I20260709 13:22:05.814704 23187 raft_consensus.cc:2749] T 19baa7bc367844c288bca8f81439c413 P cb82cc7bd6db46d980c75237a1402860 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:22:05.594076 23199 consensus_queue.cc:237] T af57996a8be84d398f0f023e8d08ef77 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:05.841095 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806d7690c2b644959152074bb2cab0a7" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:05.794114 22967 raft_consensus.cc:515] T 8158bbd17b994d2eb96f560bd0f1b88d P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:05.796705 23096 raft_consensus.cc:515] T 7c2095a5f4ad49408dbc7f69c606120d P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:05.610244 23087 raft_consensus.cc:493] T af57996a8be84d398f0f023e8d08ef77 P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:05.791086 22610 raft_consensus.cc:515] T 9e65676b21b748728ad39668cfe722ec P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:05.842878 16295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f82224f4564fea93572e8206bad1fe" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
W20260709 13:22:05.848902 16115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 210561us
I20260709 13:22:05.798231 23138 raft_consensus.cc:515] T e5ad4eb281564936a59f41e387a47156 P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:05.798395 23121 raft_consensus.cc:515] T 1df93779209748b9860f6d893dd3de53 P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:05.861197 16220 raft_consensus.cc:2393] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 in current term 26: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
I20260709 13:22:05.835707 23191 raft_consensus.cc:515] T 8a878999c0024d2abe517484180290fc P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:05.862727 16223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ab0ac8347641e6955e9bce7d90c1a2" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
W20260709 13:22:05.867208 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 327887us
I20260709 13:22:05.923882 16223 raft_consensus.cc:2468] T d2ab0ac8347641e6955e9bce7d90c1a2 P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 16.
I20260709 13:22:05.924101 16116 leader_election.cc:304] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
I20260709 13:22:05.857298 23173 raft_consensus.cc:515] T b6f07842d3aa47858df5665874c86f3a P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:05.867229 16219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87ac0d1f7e8a419f9bcbc04aee0e067e" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:05.861865 22946 raft_consensus.cc:515] T 746aabf856ab401c9ef5e859371e7b67 P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:22:05.881448 16265 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:22:05.881999 16222 raft_consensus.cc:2393] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 19: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
I20260709 13:22:05.822435 23152 raft_consensus.cc:515] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:05.835523 23167 raft_consensus.cc:515] T fffab26427e24d398e2695aad40ca134 P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:05.893563 16218 raft_consensus.cc:2468] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 18.
I20260709 13:22:05.895126 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ab0ac8347641e6955e9bce7d90c1a2" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 17 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:05.857759 22439 raft_consensus.cc:493] T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:05.897195 16226 raft_consensus.cc:1275] T e4dc6b0541ca431ebc757beac00521f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 20 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 17 index: 9. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 13:22:05.900429 16293 raft_consensus.cc:2468] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 18.
I20260709 13:22:05.908666 23202 leader_election.cc:290] T c0eab25ce4c64e018a041beeda2867db P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:22:05.835243 23203 raft_consensus.cc:515] T 9e65676b21b748728ad39668cfe722ec P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:05.900731 16217 raft_consensus.cc:1275] T e5ad4eb281564936a59f41e387a47156 P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
W20260709 13:22:05.903343 16266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 72668.7us
I20260709 13:22:05.868273 23166 raft_consensus.cc:493] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:05.908607 23240 raft_consensus.cc:493] T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:05.909147 22967 leader_election.cc:290] T 8158bbd17b994d2eb96f560bd0f1b88d P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:05.910521 23159 leader_election.cc:290] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:05.916317 16221 raft_consensus.cc:2410] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 for term 20 because replica has last-logged OpId of term: 19 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 14.
I20260709 13:22:05.917171 16296 raft_consensus.cc:1240] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 19. Current term is 20. Ops: []
I20260709 13:22:05.917555 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e4bfb9090c8460eb14fa7e0d818e7f1" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:05.898756 23174 raft_consensus.cc:3055] T c9944d5e4b8c4109af9955c4f08c9095 P cb82cc7bd6db46d980c75237a1402860 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:22:05.874919 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8638ee363273426db1393dd81e5c7b72" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:05.925045 16223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e4bfb9090c8460eb14fa7e0d818e7f1" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:05.884402 23216 raft_consensus.cc:3060] T 57a744f5aaf944678a63469cbf514015 P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:22:05.842028 23153 raft_consensus.cc:515] T 829b4ad8c5a54eababeea1181b960d4b P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:05.933553 16265 leader_election.cc:304] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860
I20260709 13:22:05.918432 23210 raft_consensus.cc:2804] T cdbd78acb4cf46298e0de8ba0d130e19 P 90f3da57796b488c9087a2f523adf045 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:22:05.841760 22494 raft_consensus.cc:515] T 829b4ad8c5a54eababeea1181b960d4b P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:05.918184 23015 leader_election.cc:290] T b87147731a68400e8cce9b879daa7c5d P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
W20260709 13:22:05.944782 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 74213.7us
I20260709 13:22:05.922160 23211 leader_election.cc:290] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:05.990093 16825 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d2ab0ac8347641e6955e9bce7d90c1a2"
dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
mode: GRACEFUL
new_leader_uuid: "90f3da57796b488c9087a2f523adf045"
 from {username='slave'} at 127.0.0.1:37818
I20260709 13:22:05.993402 16297 raft_consensus.cc:2468] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 20.
I20260709 13:22:06.009114 16825 raft_consensus.cc:606] T d2ab0ac8347641e6955e9bce7d90c1a2 P cb82cc7bd6db46d980c75237a1402860 [term 16 LEADER]: Received request to transfer leadership to TS 90f3da57796b488c9087a2f523adf045
I20260709 13:22:05.990972 23089 leader_election.cc:290] T 02838c86531d4f98bf58d1c22b8baf77 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:05.924439 23087 raft_consensus.cc:515] T af57996a8be84d398f0f023e8d08ef77 P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:05.984769 23166 raft_consensus.cc:3060] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:22:06.010545 16789 leader_election.cc:304] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 
W20260709 13:22:06.013202 23241 log.cc:927] Time spent T 434bff3082d141838068f323bdb0d1ed P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.086s	user 0.000s	sys 0.000s
W20260709 13:22:06.013975 23237 log.cc:927] Time spent T 57a744f5aaf944678a63469cbf514015 P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.077s	user 0.000s	sys 0.001s
I20260709 13:22:06.031279 22701 leader_election.cc:290] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 17 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:06.034307 23086 raft_consensus.cc:3060] T 8a878999c0024d2abe517484180290fc P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:22:06.014417 23177 consensus_queue.cc:1048] T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:22:06.014452 23211 consensus_queue.cc:1059] T c9944d5e4b8c4109af9955c4f08c9095 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: INVALID_TERM, Last received: 19.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:22:06.015617 16790 leader_election.cc:304] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:22:06.017316 16825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8638ee363273426db1393dd81e5c7b72" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:05.996215 23174 raft_consensus.cc:740] T c9944d5e4b8c4109af9955c4f08c9095 P cb82cc7bd6db46d980c75237a1402860 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:22:05.975549 23183 consensus_queue.cc:1048] T 7c2095a5f4ad49408dbc7f69c606120d P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.030s
I20260709 13:22:06.012974 23161 consensus_queue.cc:260] T 31969392ccdb43b491911a319e8bc7e1 P 90f3da57796b488c9087a2f523adf045 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:06.038539 23161 raft_consensus.cc:3060] T 31969392ccdb43b491911a319e8bc7e1 P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:22:06.022105 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 88627.5us
I20260709 13:22:05.999579 23159 raft_consensus.cc:493] T 08c74490c40243a88e62d1760b5cd814 P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:06.040153 23159 raft_consensus.cc:515] T 08c74490c40243a88e62d1760b5cd814 P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:06.042689 23159 leader_election.cc:290] T 08c74490c40243a88e62d1760b5cd814 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:06.043699 23191 leader_election.cc:290] T 8a878999c0024d2abe517484180290fc P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:22:05.968510 22439 raft_consensus.cc:515] T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:06.027838 23015 raft_consensus.cc:493] T 9c61a39fd4f84424b129c1a6f9ba404d P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:06.057757 23015 raft_consensus.cc:515] T 9c61a39fd4f84424b129c1a6f9ba404d P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:06.058808 22439 leader_election.cc:290] T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:06.028599 23090 raft_consensus.cc:2804] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [term 25 FOLLOWER]: Leader election won for term 25
I20260709 13:22:06.044073 23167 leader_election.cc:290] T fffab26427e24d398e2695aad40ca134 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:06.045118 22610 leader_election.cc:290] T 9e65676b21b748728ad39668cfe722ec P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:06.025822 22740 raft_consensus.cc:2804] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P cb82cc7bd6db46d980c75237a1402860 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:22:06.000741 23239 raft_consensus.cc:2749] T b6f07842d3aa47858df5665874c86f3a P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:22:06.049248 23184 leader_election.cc:290] T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:22:05.975731 23233 consensus_queue.cc:1048] T 66ae176961b74813a58bc15c9c4c6f2a P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.078s
I20260709 13:22:06.004334 23242 raft_consensus.cc:493] T 08c74490c40243a88e62d1760b5cd814 P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:06.051440 23159 raft_consensus.cc:493] T c0eab25ce4c64e018a041beeda2867db P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:06.051980 23023 raft_consensus.cc:493] T 8a878999c0024d2abe517484180290fc P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:06.052533 23121 leader_election.cc:290] T 1df93779209748b9860f6d893dd3de53 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:06.052796 23177 raft_consensus.cc:2804] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:22:06.112520 23216 raft_consensus.cc:2749] T 57a744f5aaf944678a63469cbf514015 P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:22:06.055435 23211 consensus_queue.cc:1059] T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: INVALID_TERM, Last received: 20.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 13:22:06.042997 23219 leader_election.cc:290] T d16dfad0c3b540e5b4b86492ddf87208 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:22:06.114873 23199 raft_consensus.cc:3055] T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:22:06.026278 23210 raft_consensus.cc:697] T cdbd78acb4cf46298e0de8ba0d130e19 P 90f3da57796b488c9087a2f523adf045 [term 22 LEADER]: Becoming Leader. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:22:06.115321 23204 raft_consensus.cc:515] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [term 26 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:06.115938 23210 consensus_queue.cc:237] T cdbd78acb4cf46298e0de8ba0d130e19 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 21.16, Last appended by leader: 16, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:06.099491 23193 raft_consensus.cc:3055] T 9e65676b21b748728ad39668cfe722ec P cb82cc7bd6db46d980c75237a1402860 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:22:05.994432 23240 raft_consensus.cc:515] T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:06.117514 23193 raft_consensus.cc:740] T 9e65676b21b748728ad39668cfe722ec P cb82cc7bd6db46d980c75237a1402860 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:22:06.118419 23193 consensus_queue.cc:260] T 9e65676b21b748728ad39668cfe722ec P cb82cc7bd6db46d980c75237a1402860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:06.119191 23240 leader_election.cc:290] T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:06.099987 23202 consensus_queue.cc:1048] T 66ae176961b74813a58bc15c9c4c6f2a P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.004s
I20260709 13:22:06.100833 23242 raft_consensus.cc:515] T 08c74490c40243a88e62d1760b5cd814 P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:06.115296 23199 raft_consensus.cc:740] T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:22:06.121148 23199 consensus_queue.cc:260] T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 20.16, Last appended by leader: 16, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:06.121587 23242 leader_election.cc:290] T 08c74490c40243a88e62d1760b5cd814 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:06.122540 23199 raft_consensus.cc:3060] T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:22:06.119781 23193 raft_consensus.cc:3060] T 9e65676b21b748728ad39668cfe722ec P cb82cc7bd6db46d980c75237a1402860 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:22:06.123353 23203 leader_election.cc:290] T 9e65676b21b748728ad39668cfe722ec P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:06.125690 23204 leader_election.cc:290] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 26 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
W20260709 13:22:06.164233 16115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 241601us
W20260709 13:22:05.972509 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 48478.3us
I20260709 13:22:06.166738 16117 leader_election.cc:304] T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860
I20260709 13:22:06.168576 16116 leader_election.cc:304] T 18ad7e3af5504571a69d10c00750a01c P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 90f3da57796b488c9087a2f523adf045
I20260709 13:22:06.170243 16117 leader_election.cc:304] T 3884f864a96d428785895e5463e5728f P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860; no voters: 90f3da57796b488c9087a2f523adf045
I20260709 13:22:06.173521 16143 raft_consensus.cc:1275] T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b [term 22 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 21 index: 14. Preceding OpId from leader: term: 22 index: 15. (index mismatch)
W20260709 13:22:06.177989 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 105392us
I20260709 13:22:06.182081 16265 leader_election.cc:304] T d2ab0ac8347641e6955e9bce7d90c1a2 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: cb82cc7bd6db46d980c75237a1402860
W20260709 13:22:06.183946 16190 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:22:06.188118 15046 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:22:06.069180 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31969392ccdb43b491911a319e8bc7e1" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:22:06.191612 16149 raft_consensus.cc:1240] T 57a744f5aaf944678a63469cbf514015 P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 15. Current term is 16. Ops: [15.13-15.13]
I20260709 13:22:06.191756 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0eab25ce4c64e018a041beeda2867db" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:06.022365 16266 leader_election.cc:304] T 2794af5b99984efebdee564035841cb0 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 
W20260709 13:22:06.194047 16266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 171601us
I20260709 13:22:06.053676 16790 leader_election.cc:304] T 434bff3082d141838068f323bdb0d1ed P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:22:06.064738 16816 raft_consensus.cc:1240] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860 [term 17 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 16. Current term is 17. Ops: []
I20260709 13:22:06.055227 22494 leader_election.cc:290] T 829b4ad8c5a54eababeea1181b960d4b P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:06.053203 22701 consensus_queue.cc:1059] T 66f75124215f4ca6bef831e174a792d8 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: INVALID_TERM, Last received: 16.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:22:06.210093 22439 raft_consensus.cc:493] T c4f65cdf00b748bb963ee6e81ef43b79 P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:22:06.078548 22740 raft_consensus.cc:493] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P cb82cc7bd6db46d980c75237a1402860 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:06.211841 22740 raft_consensus.cc:3060] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P cb82cc7bd6db46d980c75237a1402860 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:22:06.211283 22439 raft_consensus.cc:515] T c4f65cdf00b748bb963ee6e81ef43b79 P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:06.026008 22960 raft_consensus.cc:3060] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:22:06.215574 23167 raft_consensus.cc:493] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:22:06.216811 23188 raft_consensus.cc:3055] T 66f75124215f4ca6bef831e174a792d8 P cb82cc7bd6db46d980c75237a1402860 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:22:06.216647 23167 raft_consensus.cc:515] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:06.217491 23188 raft_consensus.cc:740] T 66f75124215f4ca6bef831e174a792d8 P cb82cc7bd6db46d980c75237a1402860 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:22:06.027379 22962 leader_election.cc:290] T c2f5f89186204242bf377f7c42b2e2da P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:06.218292 22709 consensus_queue.cc:1048] T fffab26427e24d398e2695aad40ca134 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:22:06.084931 23003 leader_election.cc:290] T 68a6838d82ef4cac83507f906139d90f P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:06.081743 23090 raft_consensus.cc:697] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [term 25 LEADER]: Becoming Leader. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:22:06.099347 23133 leader_election.cc:290] T 746aabf856ab401c9ef5e859371e7b67 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:06.072332 23138 leader_election.cc:290] T e5ad4eb281564936a59f41e387a47156 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:06.098412 23159 raft_consensus.cc:515] T c0eab25ce4c64e018a041beeda2867db P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:06.272493 23159 leader_election.cc:290] T c0eab25ce4c64e018a041beeda2867db P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:06.053663 23173 leader_election.cc:290] T b6f07842d3aa47858df5665874c86f3a P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:06.062227 23174 consensus_queue.cc:260] T c9944d5e4b8c4109af9955c4f08c9095 P cb82cc7bd6db46d980c75237a1402860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.15, Last appended by leader: 15, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:06.275556 22677 leader_election.cc:290] T 02838c86531d4f98bf58d1c22b8baf77 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:06.276856 22701 raft_consensus.cc:2749] T 434bff3082d141838068f323bdb0d1ed P cb82cc7bd6db46d980c75237a1402860 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:22:06.277225 23174 raft_consensus.cc:3060] T c9944d5e4b8c4109af9955c4f08c9095 P cb82cc7bd6db46d980c75237a1402860 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:22:06.103216 23177 raft_consensus.cc:697] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 [term 18 LEADER]: Becoming Leader. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:22:06.276894 23086 consensus_queue.cc:1048] T 26354361e9b044cea1c2b86c4882fe92 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.002s
I20260709 13:22:06.279788 23024 raft_consensus.cc:2804] T 2794af5b99984efebdee564035841cb0 P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:22:06.201323 23121 raft_consensus.cc:493] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:06.071605 23061 leader_election.cc:290] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 20 election: Requested vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:06.280512 23086 raft_consensus.cc:2804] T 18ad7e3af5504571a69d10c00750a01c P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:22:06.221956 22610 consensus_queue.cc:1048] T 23f82224f4564fea93572e8206bad1fe P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:22:06.281504 23086 raft_consensus.cc:493] T 18ad7e3af5504571a69d10c00750a01c P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Starting leader election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:22:06.283267 22610 raft_consensus.cc:2804] T 3884f864a96d428785895e5463e5728f P 647a343a542d4a139fe106c19095129b [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:22:06.228979 23087 leader_election.cc:290] T af57996a8be84d398f0f023e8d08ef77 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:06.102047 23023 raft_consensus.cc:515] T 8a878999c0024d2abe517484180290fc P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:06.282956 23086 raft_consensus.cc:3060] T 18ad7e3af5504571a69d10c00750a01c P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:22:06.287523 16224 raft_consensus.cc:2468] T c0eab25ce4c64e018a041beeda2867db P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 19.
I20260709 13:22:06.288874 23121 raft_consensus.cc:515] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:06.280355 23177 consensus_queue.cc:237] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:06.266317 22782 consensus_queue.cc:1048] T e4dc6b0541ca431ebc757beac00521f1 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:22:06.264971 22864 consensus_queue.cc:1048] T e5ad4eb281564936a59f41e387a47156 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:22:06.277282 23240 raft_consensus.cc:2749] T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:22:06.283913 22610 raft_consensus.cc:493] T 3884f864a96d428785895e5463e5728f P 647a343a542d4a139fe106c19095129b [term 14 FOLLOWER]: Starting leader election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:22:06.297101 23023 leader_election.cc:290] T 8a878999c0024d2abe517484180290fc P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:06.280308 23024 raft_consensus.cc:493] T 2794af5b99984efebdee564035841cb0 P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:22:06.297915 15046 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:22:06.267273 16190 leader_election.cc:304] T 8cc959fb7f704fba9009c0595467d074 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
W20260709 13:22:06.299744 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 131742us
W20260709 13:22:06.305238 16115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 52329.1us
I20260709 13:22:06.307081 23245 raft_consensus.cc:493] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:22:06.307525 22439 leader_election.cc:290] T c4f65cdf00b748bb963ee6e81ef43b79 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:22:06.308871 22917 leader_election.cc:290] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 26 election: Requested vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:06.314010 22610 raft_consensus.cc:3060] T 3884f864a96d428785895e5463e5728f P 647a343a542d4a139fe106c19095129b [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:22:06.336369 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 69339.5us
I20260709 13:22:06.337949 16190 leader_election.cc:304] T d2ab0ac8347641e6955e9bce7d90c1a2 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860
I20260709 13:22:06.339092 23173 raft_consensus.cc:493] T 5a96373bb0e14dce8a266b2fc6c58507 P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:22:06.339768 23173 raft_consensus.cc:515] T 5a96373bb0e14dce8a266b2fc6c58507 P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:06.340034 23090 consensus_queue.cc:237] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 24.15, Last appended by leader: 15, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:06.342046 23173 leader_election.cc:290] T 5a96373bb0e14dce8a266b2fc6c58507 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:06.342727 23173 raft_consensus.cc:2804] T 8cc959fb7f704fba9009c0595467d074 P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Leader election won for term 18
W20260709 13:22:06.344597 16117 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 174655us
I20260709 13:22:06.345825 16117 leader_election.cc:304] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860; no voters: 90f3da57796b488c9087a2f523adf045
I20260709 13:22:06.347369 22873 raft_consensus.cc:2804] T d2ab0ac8347641e6955e9bce7d90c1a2 P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 13:22:06.347693 23076 consensus_meta.cc:287] T 8d4ab870a05347868956aa8b841b2981 P 90f3da57796b488c9087a2f523adf045: Time spent flushing consensus metadata: real 0.638s	user 0.000s	sys 0.018s
I20260709 13:22:06.348081 23256 raft_consensus.cc:493] T 7c2095a5f4ad49408dbc7f69c606120d P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:06.348776 23256 raft_consensus.cc:515] T 7c2095a5f4ad49408dbc7f69c606120d P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:06.325199 23152 leader_election.cc:290] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:06.326815 23153 leader_election.cc:290] T 829b4ad8c5a54eababeea1181b960d4b P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:06.317492 23024 raft_consensus.cc:3060] T 2794af5b99984efebdee564035841cb0 P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:22:06.328886 23245 raft_consensus.cc:515] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:06.330169 16116 leader_election.cc:304] T 5a959dbecc2648eb85d1cc550f5a3245 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:22:06.330492 23257 raft_consensus.cc:493] T 66f75124215f4ca6bef831e174a792d8 P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:22:06.355369 16817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada5a004ba7c46819051e3ccec71ffdb" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:22:06.372164 16817 raft_consensus.cc:2393] T ada5a004ba7c46819051e3ccec71ffdb P cb82cc7bd6db46d980c75237a1402860 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 20: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
I20260709 13:22:06.371806 23257 raft_consensus.cc:515] T 66f75124215f4ca6bef831e174a792d8 P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:06.363241 16821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "829b4ad8c5a54eababeea1181b960d4b" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:06.363463 16825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746aabf856ab401c9ef5e859371e7b67" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:06.241914 23188 consensus_queue.cc:260] T 66f75124215f4ca6bef831e174a792d8 P cb82cc7bd6db46d980c75237a1402860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:06.374864 23188 raft_consensus.cc:3060] T 66f75124215f4ca6bef831e174a792d8 P cb82cc7bd6db46d980c75237a1402860 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:22:06.374804 23257 leader_election.cc:290] T 66f75124215f4ca6bef831e174a792d8 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:06.379527 23121 leader_election.cc:290] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
W20260709 13:22:06.364828 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 311138us
I20260709 13:22:06.383276 16818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e65676b21b748728ad39668cfe722ec" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
W20260709 13:22:06.398824 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 68881.5us
I20260709 13:22:06.400537 23240 raft_consensus.cc:2749] T 5a959dbecc2648eb85d1cc550f5a3245 P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:22:06.413131 22946 leader_election.cc:290] T 746aabf856ab401c9ef5e859371e7b67 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:06.408018 16819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74ca8140f35c4ae29e8a664f6bdfc829" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:06.414374 16819 raft_consensus.cc:2468] T 74ca8140f35c4ae29e8a664f6bdfc829 P cb82cc7bd6db46d980c75237a1402860 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 20.
I20260709 13:22:06.404557 23167 leader_election.cc:290] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:06.418079 22748 leader_election.cc:290] T af57996a8be84d398f0f023e8d08ef77 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 20 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:06.419761 23096 leader_election.cc:290] T 7c2095a5f4ad49408dbc7f69c606120d P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:06.414954 23015 leader_election.cc:290] T 9c61a39fd4f84424b129c1a6f9ba404d P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:06.424563 23245 leader_election.cc:290] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:06.432152 22873 raft_consensus.cc:493] T d2ab0ac8347641e6955e9bce7d90c1a2 P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Starting leader election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:22:06.432922 22873 raft_consensus.cc:3060] T d2ab0ac8347641e6955e9bce7d90c1a2 P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:22:06.435868 16816 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49892 (request call id 1017) took 1277 ms. Trace:
I20260709 13:22:06.370386 23256 leader_election.cc:290] T 7c2095a5f4ad49408dbc7f69c606120d P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:06.436343 16816 rpcz_store.cc:276] 0709 13:22:05.158221 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:22:05.184920 (+ 26699us) service_pool.cc:224] Handling call
0709 13:22:06.064551 (+879631us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:22:06.204785 (+140234us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:22:06.435479 (+230694us) spinlock_profiling.cc:243] Waited 879 ms on lock 0x7b5400625550. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5e8ad 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:22:06.435671 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2022133504}
I20260709 13:22:06.437340 16819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af57996a8be84d398f0f023e8d08ef77" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:22:06.408185 16817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74ca8140f35c4ae29e8a664f6bdfc829" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:06.439052 16817 raft_consensus.cc:2468] T 74ca8140f35c4ae29e8a664f6bdfc829 P cb82cc7bd6db46d980c75237a1402860 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 20.
W20260709 13:22:06.441781 23247 log.cc:927] Time spent T a378d7f796f94615b3b347119a9880fd P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.328s	user 0.000s	sys 0.002s
I20260709 13:22:06.450670 23257 raft_consensus.cc:493] T e4dc6b0541ca431ebc757beac00521f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:06.450989 16824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66ae176961b74813a58bc15c9c4c6f2a" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 26 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:22:06.451882 16824 raft_consensus.cc:3060] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:22:06.451726 23257 raft_consensus.cc:515] T e4dc6b0541ca431ebc757beac00521f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:06.401382 16825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada5a004ba7c46819051e3ccec71ffdb" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:22:06.454453 16825 raft_consensus.cc:2376] T ada5a004ba7c46819051e3ccec71ffdb P cb82cc7bd6db46d980c75237a1402860 [term 20 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 20. Re-sending same reply.
I20260709 13:22:06.418646 16116 leader_election.cc:304] T 8d4ab870a05347868956aa8b841b2981 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 90f3da57796b488c9087a2f523adf045
I20260709 13:22:06.460268 23252 raft_consensus.cc:2804] T 8d4ab870a05347868956aa8b841b2981 P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:22:06.460929 23252 raft_consensus.cc:697] T 8d4ab870a05347868956aa8b841b2981 P 647a343a542d4a139fe106c19095129b [term 18 LEADER]: Becoming Leader. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:22:06.367933 23023 raft_consensus.cc:2804] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:22:06.461171 16294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87147731a68400e8cce9b879daa7c5d" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:06.461670 23023 raft_consensus.cc:493] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b [term 21 FOLLOWER]: Starting leader election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:22:06.462289 23023 raft_consensus.cc:3060] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:22:06.462124 23252 consensus_queue.cc:237] T 8d4ab870a05347868956aa8b841b2981 P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:06.463968 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02838c86531d4f98bf58d1c22b8baf77" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:06.464250 16294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8158bbd17b994d2eb96f560bd0f1b88d" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:06.467973 16294 raft_consensus.cc:1275] T ecc1e363230043fe9fef390f0b979f20 P 90f3da57796b488c9087a2f523adf045 [term 21 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
I20260709 13:22:06.400813 16790 leader_election.cc:304] T 23f82224f4564fea93572e8206bad1fe P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:22:06.470434 22701 raft_consensus.cc:2749] T 23f82224f4564fea93572e8206bad1fe P cb82cc7bd6db46d980c75237a1402860 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:22:06.356501 23173 raft_consensus.cc:697] T 8cc959fb7f704fba9009c0595467d074 P 1ca2725d290e47d68b1a4b609c9816ac [term 18 LEADER]: Becoming Leader. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:22:06.474866 23173 consensus_queue.cc:237] T 8cc959fb7f704fba9009c0595467d074 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:22:06.486810 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 68355.5us
W20260709 13:22:06.501573 16117 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 155969us
I20260709 13:22:06.504976 16294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada5a004ba7c46819051e3ccec71ffdb" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:22:06.510713 16296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f5f89186204242bf377f7c42b2e2da" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:06.511622 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e65676b21b748728ad39668cfe722ec" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:06.512671 16298 raft_consensus.cc:2468] T 9e65676b21b748728ad39668cfe722ec P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 18.
I20260709 13:22:06.513170 16296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74ca8140f35c4ae29e8a664f6bdfc829" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:06.513882 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66ae176961b74813a58bc15c9c4c6f2a" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 26 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:22:06.514127 16296 raft_consensus.cc:2393] T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 20: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
I20260709 13:22:06.516476 23257 leader_election.cc:290] T e4dc6b0541ca431ebc757beac00521f1 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:06.517792 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c74490c40243a88e62d1760b5cd814" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:06.508100 22677 raft_consensus.cc:2749] T d2ab0ac8347641e6955e9bce7d90c1a2 P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:22:06.527796 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fffab26427e24d398e2695aad40ca134" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:06.534951 16824 raft_consensus.cc:2468] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 26.
W20260709 13:22:06.543506 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 142907us
I20260709 13:22:06.546252 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02838c86531d4f98bf58d1c22b8baf77" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:06.553347 16296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6f07842d3aa47858df5665874c86f3a" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
W20260709 13:22:06.564889 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 227141us
I20260709 13:22:06.568802 16144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d4ab870a05347868956aa8b841b2981" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:22:06.570679 16142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2372981006d4b218215c28964dea623" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:22:06.571202 16148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68a6838d82ef4cac83507f906139d90f" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:22:06.573127 16142 raft_consensus.cc:3060] T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:22:06.573287 16148 raft_consensus.cc:2468] T 68a6838d82ef4cac83507f906139d90f P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 18.
I20260709 13:22:06.574373 16148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1df93779209748b9860f6d893dd3de53" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:22:06.575094 16148 raft_consensus.cc:2468] T 1df93779209748b9860f6d893dd3de53 P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 17.
I20260709 13:22:06.575898 16148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806d7690c2b644959152074bb2cab0a7" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:22:06.571707 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5d59407eb545b895d8d922e0dd69fd" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:22:06.576604 22610 raft_consensus.cc:515] T 3884f864a96d428785895e5463e5728f P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:06.577315 16150 raft_consensus.cc:2393] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 21: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
I20260709 13:22:06.578173 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5ad4eb281564936a59f41e387a47156" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:22:06.578785 22610 leader_election.cc:290] T 3884f864a96d428785895e5463e5728f P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 15 election: Requested vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:06.570683 16144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cc959fb7f704fba9009c0595467d074" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:22:06.579787 16144 raft_consensus.cc:3060] T 8cc959fb7f704fba9009c0595467d074 P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:22:06.579625 23125 consensus_queue.cc:1059] T 84f2da98864b4425bf7c0055951fdd4b P 647a343a542d4a139fe106c19095129b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:22:06.581421 23125 raft_consensus.cc:3055] T 84f2da98864b4425bf7c0055951fdd4b P 647a343a542d4a139fe106c19095129b [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:22:06.581873 23125 raft_consensus.cc:740] T 84f2da98864b4425bf7c0055951fdd4b P 647a343a542d4a139fe106c19095129b [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:22:06.578914 16150 raft_consensus.cc:2410] T e5ad4eb281564936a59f41e387a47156 P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac for term 20 because replica has last-logged OpId of term: 18 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 11.
I20260709 13:22:06.583738 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4f65cdf00b748bb963ee6e81ef43b79" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:22:06.584417 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8158bbd17b994d2eb96f560bd0f1b88d" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:22:06.585407 16150 raft_consensus.cc:2468] T 8158bbd17b994d2eb96f560bd0f1b88d P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 17.
I20260709 13:22:06.586819 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee9ed67006144797b1de6ea8c2013a68" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:22:06.587762 16147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5718c623ebee4d2bb770f0f9ddb2f164" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:22:06.588956 16147 raft_consensus.cc:2393] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 19: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
I20260709 13:22:06.589687 16145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af57996a8be84d398f0f023e8d08ef77" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:22:06.590494 16145 raft_consensus.cc:2468] T af57996a8be84d398f0f023e8d08ef77 P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 19.
I20260709 13:22:06.590883 16818 raft_consensus.cc:2468] T 9e65676b21b748728ad39668cfe722ec P cb82cc7bd6db46d980c75237a1402860 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 18.
I20260709 13:22:06.591307 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1df93779209748b9860f6d893dd3de53" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:06.588794 16151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f2da98864b4425bf7c0055951fdd4b" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:22:06.592525 16294 raft_consensus.cc:2393] T ada5a004ba7c46819051e3ccec71ffdb P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 20: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
I20260709 13:22:06.588809 16293 raft_consensus.cc:1275] T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 17 index: 9. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
W20260709 13:22:06.593617 16192 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 53465.6us
I20260709 13:22:06.595060 16192 leader_election.cc:304] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 
W20260709 13:22:06.596359 22908 consensus_meta.cc:287] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 1ca2725d290e47d68b1a4b609c9816ac: Time spent flushing consensus metadata: real 0.776s	user 0.014s	sys 0.005s
I20260709 13:22:06.596467 16292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8638ee363273426db1393dd81e5c7b72" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:06.596377 22740 raft_consensus.cc:515] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P cb82cc7bd6db46d980c75237a1402860 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:06.597409 16223 raft_consensus.cc:2468] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 20.
I20260709 13:22:06.599787 22740 leader_election.cc:290] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 21 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:06.576610 16148 raft_consensus.cc:2410] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac for term 20 because replica has last-logged OpId of term: 18 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 12.
W20260709 13:22:06.605584 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 54356.7us
I20260709 13:22:06.607959 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdbd78acb4cf46298e0de8ba0d130e19" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:06.551299 23256 raft_consensus.cc:493] T 8c11074579b046d99e585c2818e18a2c P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
W20260709 13:22:06.611753 16265 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:22:06.615058 16292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5ad4eb281564936a59f41e387a47156" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:06.617523 16142 raft_consensus.cc:2468] T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 23.
I20260709 13:22:06.617123 16144 raft_consensus.cc:2468] T 8cc959fb7f704fba9009c0595467d074 P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 18.
I20260709 13:22:06.622359 16294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0eab25ce4c64e018a041beeda2867db" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:06.582968 23125 consensus_queue.cc:260] T 84f2da98864b4425bf7c0055951fdd4b P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:06.630319 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c61a39fd4f84424b129c1a6f9ba404d" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:06.574371 23024 raft_consensus.cc:515] T 2794af5b99984efebdee564035841cb0 P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:06.636485 16293 raft_consensus.cc:2468] T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 20.
I20260709 13:22:06.635474 16294 raft_consensus.cc:2468] T c0eab25ce4c64e018a041beeda2867db P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 19.
I20260709 13:22:06.634719 16190 leader_election.cc:304] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:22:06.624543 22677 raft_consensus.cc:493] T 5a959dbecc2648eb85d1cc550f5a3245 P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:22:06.654556 16142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af57996a8be84d398f0f023e8d08ef77" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:22:06.654735 16144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f5f89186204242bf377f7c42b2e2da" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:22:06.655390 16142 raft_consensus.cc:3060] T af57996a8be84d398f0f023e8d08ef77 P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:22:06.657985 16223 raft_consensus.cc:1275] T 08c74490c40243a88e62d1760b5cd814 P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 18 index: 11. (term mismatch)
I20260709 13:22:06.658404 16223 pending_rounds.cc:85] T 08c74490c40243a88e62d1760b5cd814 P 1ca2725d290e47d68b1a4b609c9816ac: Aborting all ops after (but not including) 10
I20260709 13:22:06.658663 16223 pending_rounds.cc:107] T 08c74490c40243a88e62d1760b5cd814 P 1ca2725d290e47d68b1a4b609c9816ac: Aborting uncommitted NO_OP operation due to leader change: 17.11
I20260709 13:22:06.658937 16223 raft_consensus.cc:2889] T 08c74490c40243a88e62d1760b5cd814 P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:22:06.659214 16791 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:22:06.661285 16818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66f75124215f4ca6bef831e174a792d8" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:06.662269 16824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdbd78acb4cf46298e0de8ba0d130e19" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:06.663105 16824 raft_consensus.cc:2393] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 22: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
I20260709 13:22:06.663203 22960 raft_consensus.cc:2749] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:22:06.664377 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a878999c0024d2abe517484180290fc" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:06.665339 16224 raft_consensus.cc:2393] T 8a878999c0024d2abe517484180290fc P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 17: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
I20260709 13:22:06.643182 23125 raft_consensus.cc:3060] T 84f2da98864b4425bf7c0055951fdd4b P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:22:06.641439 22908 raft_consensus.cc:2804] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:22:06.672091 16149 raft_consensus.cc:2468] T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 21.
I20260709 13:22:06.678874 16144 raft_consensus.cc:2393] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 21: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
I20260709 13:22:06.679461 16221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74ca8140f35c4ae29e8a664f6bdfc829" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:06.686302 16291 raft_consensus.cc:1240] T 2794af5b99984efebdee564035841cb0 P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 16. Current term is 17. Ops: []
I20260709 13:22:06.685708 16297 raft_consensus.cc:2468] T 31969392ccdb43b491911a319e8bc7e1 P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 17.
I20260709 13:22:06.673930 22908 raft_consensus.cc:493] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Starting leader election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:22:06.692567 16294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c2095a5f4ad49408dbc7f69c606120d" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 17 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:06.648332 23256 raft_consensus.cc:515] T 8c11074579b046d99e585c2818e18a2c P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:06.686002 23090 raft_consensus.cc:493] T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:06.667922 22677 raft_consensus.cc:515] T 5a959dbecc2648eb85d1cc550f5a3245 P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:06.700901 16220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d16dfad0c3b540e5b4b86492ddf87208" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:06.704968 23193 consensus_queue.cc:1059] T 57a744f5aaf944678a63469cbf514015 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: INVALID_TERM, Last received: 13.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.028s
I20260709 13:22:06.706480 16144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746aabf856ab401c9ef5e859371e7b67" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:22:06.709105 16220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5718c623ebee4d2bb770f0f9ddb2f164" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:06.710839 16218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c61a39fd4f84424b129c1a6f9ba404d" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 21 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:06.695704 22908 raft_consensus.cc:3060] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:22:06.722693 23269 raft_consensus.cc:2804] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:22:06.723317 23269 raft_consensus.cc:493] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:06.724740 16294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68a6838d82ef4cac83507f906139d90f" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
W20260709 13:22:06.727043 16117 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 177689us
I20260709 13:22:06.728097 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c74490c40243a88e62d1760b5cd814" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
W20260709 13:22:06.737708 23274 log.cc:927] Time spent T 8d4ab870a05347868956aa8b841b2981 P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 0.159s	user 0.002s	sys 0.000s
W20260709 13:22:06.739099 23166 consensus_meta.cc:287] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860: Time spent flushing consensus metadata: real 0.666s	user 0.013s	sys 0.004s
I20260709 13:22:06.739660 23166 raft_consensus.cc:515] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:06.741225 16820 raft_consensus.cc:2393] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 23: Already voted for candidate cb82cc7bd6db46d980c75237a1402860 in this term.
I20260709 13:22:06.741973 16220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4f65cdf00b748bb963ee6e81ef43b79" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:06.742149 23166 leader_election.cc:290] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 23 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:06.742864 16220 raft_consensus.cc:2393] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 21: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
I20260709 13:22:06.701076 23211 consensus_queue.cc:1048] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.035s
I20260709 13:22:06.753136 16266 leader_election.cc:304] T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:22:06.755599 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8158bbd17b994d2eb96f560bd0f1b88d" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:06.756358 16217 raft_consensus.cc:2468] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 17.
I20260709 13:22:06.756481 16294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c74490c40243a88e62d1760b5cd814" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:06.723742 23269 raft_consensus.cc:3060] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:22:06.760138 16144 raft_consensus.cc:2468] T 746aabf856ab401c9ef5e859371e7b67 P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 16.
I20260709 13:22:06.764314 16149 raft_consensus.cc:2468] T 08c74490c40243a88e62d1760b5cd814 P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 18.
I20260709 13:22:06.765836 16220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02838c86531d4f98bf58d1c22b8baf77" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:06.766942 16294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a878999c0024d2abe517484180290fc" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:06.768378 16148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fffab26427e24d398e2695aad40ca134" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:22:06.755904 23061 raft_consensus.cc:493] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:06.775599 16266 leader_election.cc:304] T ada5a004ba7c46819051e3ccec71ffdb P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac
W20260709 13:22:06.734946 16036 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 19baa7bc367844c288bca8f81439c413 with cas_config_opid_index 7: ChangeConfig:REMOVE_PEER failed with leader cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica cb82cc7bd6db46d980c75237a1402860 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 22 committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
W20260709 13:22:06.784065 16036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.ChangeConfig -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 50364.6us
I20260709 13:22:06.787635 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "829b4ad8c5a54eababeea1181b960d4b" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:22:06.792593 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68a6838d82ef4cac83507f906139d90f" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:06.795562 16294 raft_consensus.cc:2468] T 8a878999c0024d2abe517484180290fc P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 16.
I20260709 13:22:06.780738 22960 raft_consensus.cc:2804] T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:22:06.806417 16223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746aabf856ab401c9ef5e859371e7b67" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:06.806700 23202 raft_consensus.cc:493] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:22:06.802006 16265 leader_election.cc:304] T c0eab25ce4c64e018a041beeda2867db P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
W20260709 13:22:06.807819 23174 consensus_meta.cc:287] T c9944d5e4b8c4109af9955c4f08c9095 P cb82cc7bd6db46d980c75237a1402860: Time spent flushing consensus metadata: real 0.530s	user 0.008s	sys 0.005s
I20260709 13:22:06.808321 16144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02838c86531d4f98bf58d1c22b8baf77" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:22:06.811646 16146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6f07842d3aa47858df5665874c86f3a" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:22:06.813318 23077 raft_consensus.cc:2804] T ada5a004ba7c46819051e3ccec71ffdb P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:22:06.812888 16145 raft_consensus.cc:1275] T 9c61a39fd4f84424b129c1a6f9ba404d P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 19 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
W20260709 13:22:06.812230 16791 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:22:06.819921 16147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a96373bb0e14dce8a266b2fc6c58507" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
W20260709 13:22:06.807948 23276 log.cc:927] Time spent T 87ac0d1f7e8a419f9bcbc04aee0e067e P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 0.071s	user 0.000s	sys 0.000s
W20260709 13:22:06.828918 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 49412.7us
I20260709 13:22:06.829401 16148 raft_consensus.cc:2468] T fffab26427e24d398e2695aad40ca134 P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 19.
I20260709 13:22:06.827543 16146 raft_consensus.cc:2468] T b6f07842d3aa47858df5665874c86f3a P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 18.
I20260709 13:22:06.836990 16149 raft_consensus.cc:2393] T 829b4ad8c5a54eababeea1181b960d4b P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 17: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
I20260709 13:22:06.837343 16218 raft_consensus.cc:1240] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [term 26 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 25. Current term is 26. Ops: [25.16-25.16]
I20260709 13:22:06.830506 16142 raft_consensus.cc:2468] T af57996a8be84d398f0f023e8d08ef77 P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 20.
I20260709 13:22:06.801271 23024 leader_election.cc:290] T 2794af5b99984efebdee564035841cb0 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 17 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:06.838680 16146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c2095a5f4ad49408dbc7f69c606120d" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 17 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:22:06.796067 23177 raft_consensus.cc:493] T 26354361e9b044cea1c2b86c4882fe92 P cb82cc7bd6db46d980c75237a1402860 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:06.773705 23090 raft_consensus.cc:515] T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:06.823645 22960 raft_consensus.cc:493] T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:06.840950 16145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66f75124215f4ca6bef831e174a792d8" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:22:06.846743 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1df93779209748b9860f6d893dd3de53" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:06.847836 16148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31969392ccdb43b491911a319e8bc7e1" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:22:06.850756 16821 raft_consensus.cc:1275] T c9944d5e4b8c4109af9955c4f08c9095 P cb82cc7bd6db46d980c75237a1402860 [term 22 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 21 index: 15. Preceding OpId from leader: term: 22 index: 16. (index mismatch)
W20260709 13:22:06.852531 16117 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 92907.9us
I20260709 13:22:06.854494 16144 raft_consensus.cc:2468] T 02838c86531d4f98bf58d1c22b8baf77 P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 19.
I20260709 13:22:06.847330 23077 raft_consensus.cc:697] T ada5a004ba7c46819051e3ccec71ffdb P 90f3da57796b488c9087a2f523adf045 [term 20 LEADER]: Becoming Leader. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:22:06.856627 22645 raft_consensus.cc:2804] T c0eab25ce4c64e018a041beeda2867db P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:22:06.857124 22645 raft_consensus.cc:493] T c0eab25ce4c64e018a041beeda2867db P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:06.857499 22645 raft_consensus.cc:3060] T c0eab25ce4c64e018a041beeda2867db P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:22:06.857508 23077 consensus_queue.cc:237] T ada5a004ba7c46819051e3ccec71ffdb P 90f3da57796b488c9087a2f523adf045 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:06.860250 23256 leader_election.cc:290] T 8c11074579b046d99e585c2818e18a2c P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:06.866405 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4dc6b0541ca431ebc757beac00521f1" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 21 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:22:06.867281 16789 leader_election.cc:304] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:22:06.868180 16144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ad5dbfa99ee4b8d9e28be92b3f36db5" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:22:06.832368 23173 raft_consensus.cc:493] T 19baa7bc367844c288bca8f81439c413 P 1ca2725d290e47d68b1a4b609c9816ac [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:06.868623 23202 raft_consensus.cc:515] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:06.869334 16218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:06.850317 22960 raft_consensus.cc:3060] T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:22:06.871254 16148 raft_consensus.cc:2393] T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 in current term 17: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
I20260709 13:22:06.871491 23177 raft_consensus.cc:515] T 26354361e9b044cea1c2b86c4882fe92 P cb82cc7bd6db46d980c75237a1402860 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:22:06.872035 23086 consensus_meta.cc:287] T 18ad7e3af5504571a69d10c00750a01c P 647a343a542d4a139fe106c19095129b: Time spent flushing consensus metadata: real 0.583s	user 0.012s	sys 0.001s
I20260709 13:22:06.873267 22740 raft_consensus.cc:993] T d2ab0ac8347641e6955e9bce7d90c1a2 P cb82cc7bd6db46d980c75237a1402860: : Instructing follower 90f3da57796b488c9087a2f523adf045 to start an election
I20260709 13:22:06.862937 16146 raft_consensus.cc:2468] T 7c2095a5f4ad49408dbc7f69c606120d P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 16.
I20260709 13:22:06.867508 16149 raft_consensus.cc:2410] T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac for term 21 because replica has last-logged OpId of term: 19 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 9.
I20260709 13:22:06.868777 23256 raft_consensus.cc:493] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:06.874511 16223 raft_consensus.cc:2468] T 746aabf856ab401c9ef5e859371e7b67 P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 16.
I20260709 13:22:06.836529 23087 raft_consensus.cc:493] T fffab26427e24d398e2695aad40ca134 P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:06.853261 16226 raft_consensus.cc:2468] T 1df93779209748b9860f6d893dd3de53 P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 17.
I20260709 13:22:06.875602 16291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806d7690c2b644959152074bb2cab0a7" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:06.884832 16191 leader_election.cc:304] T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:22:06.869783 16143 raft_consensus.cc:1275] T 746aabf856ab401c9ef5e859371e7b67 P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 16 index: 14. (index mismatch)
I20260709 13:22:06.887455 16820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c11074579b046d99e585c2818e18a2c" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:06.889760 22917 raft_consensus.cc:493] T d16dfad0c3b540e5b4b86492ddf87208 P 1ca2725d290e47d68b1a4b609c9816ac [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:22:06.892515 22864 consensus_queue.cc:1048] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:22:06.893689 23233 consensus_queue.cc:1059] T 66ae176961b74813a58bc15c9c4c6f2a P 90f3da57796b488c9087a2f523adf045 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: INVALID_TERM, Last received: 24.15, Next index: 16, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:22:06.895426 16145 raft_consensus.cc:2393] T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 17: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
I20260709 13:22:06.826606 23061 raft_consensus.cc:515] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:06.888017 16220 raft_consensus.cc:1240] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 1ca2725d290e47d68b1a4b609c9816ac [term 20 FOLLOWER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 19. Current term is 20. Ops: []
I20260709 13:22:06.903332 16144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62fe3346cd3c42c4a55e14af9073bca4" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 26 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:22:06.907563 16192 leader_election.cc:304] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:22:06.908811 16151 raft_consensus.cc:2468] T 84f2da98864b4425bf7c0055951fdd4b P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 16.
I20260709 13:22:06.906324 23216 raft_consensus.cc:493] T ecc1e363230043fe9fef390f0b979f20 P 90f3da57796b488c9087a2f523adf045 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:06.902650 22748 raft_consensus.cc:493] T 8a878999c0024d2abe517484180290fc P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:06.916729 22748 raft_consensus.cc:515] T 8a878999c0024d2abe517484180290fc P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:06.916973 23216 raft_consensus.cc:515] T ecc1e363230043fe9fef390f0b979f20 P 90f3da57796b488c9087a2f523adf045 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:06.919618 16223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62fe3346cd3c42c4a55e14af9073bca4" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 26 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:22:06.920014 23216 leader_election.cc:290] T ecc1e363230043fe9fef390f0b979f20 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:06.920465 22917 raft_consensus.cc:515] T d16dfad0c3b540e5b4b86492ddf87208 P 1ca2725d290e47d68b1a4b609c9816ac [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:06.922848 22917 leader_election.cc:290] T d16dfad0c3b540e5b4b86492ddf87208 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:06.924328 16146 raft_consensus.cc:1240] T 9e65676b21b748728ad39668cfe722ec P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 17. Current term is 18. Ops: []
I20260709 13:22:06.914929 23256 raft_consensus.cc:515] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:06.914706 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e4bfb9090c8460eb14fa7e0d818e7f1" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:22:06.915577 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5ad4eb281564936a59f41e387a47156" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:06.917111 23090 leader_election.cc:290] T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:06.898518 23125 raft_consensus.cc:493] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:06.915834 16820 raft_consensus.cc:2468] T 8c11074579b046d99e585c2818e18a2c P cb82cc7bd6db46d980c75237a1402860 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 19.
I20260709 13:22:06.937479 16294 raft_consensus.cc:1240] T 57a744f5aaf944678a63469cbf514015 P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 15. Current term is 16. Ops: []
I20260709 13:22:06.935484 16149 raft_consensus.cc:1240] T af57996a8be84d398f0f023e8d08ef77 P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 19. Current term is 20. Ops: []
I20260709 13:22:06.940542 23152 raft_consensus.cc:2749] T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:22:06.932703 22946 raft_consensus.cc:2804] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [term 26 FOLLOWER]: Leader election won for term 26
I20260709 13:22:06.942960 22946 raft_consensus.cc:697] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [term 26 LEADER]: Becoming Leader. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:22:06.943324 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3884f864a96d428785895e5463e5728f" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 15 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:22:06.943958 22946 consensus_queue.cc:237] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 24.15, Last appended by leader: 15, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:06.941865 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0eab25ce4c64e018a041beeda2867db" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
W20260709 13:22:06.947052 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 83935.7us
I20260709 13:22:06.955302 16296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a9de347c711477e87c582d003f36925" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:22:06.938603 16291 raft_consensus.cc:2468] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 18.
W20260709 13:22:06.956516 16266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 192599us
I20260709 13:22:06.932317 23177 leader_election.cc:290] T 26354361e9b044cea1c2b86c4882fe92 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:06.929113 23167 raft_consensus.cc:493] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 1ca2725d290e47d68b1a4b609c9816ac [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:06.919100 22748 leader_election.cc:290] T 8a878999c0024d2abe517484180290fc P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:06.960307 16223 raft_consensus.cc:3060] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:22:06.962788 16298 raft_consensus.cc:3060] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:22:06.966498 16144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a9de347c711477e87c582d003f36925" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:22:06.916222 23173 raft_consensus.cc:515] T 19baa7bc367844c288bca8f81439c413 P 1ca2725d290e47d68b1a4b609c9816ac [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:06.901219 22873 raft_consensus.cc:515] T d2ab0ac8347641e6955e9bce7d90c1a2 P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:06.969062 16220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:06.902882 23086 raft_consensus.cc:515] T 18ad7e3af5504571a69d10c00750a01c P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } }
I20260709 13:22:06.996176 16149 raft_consensus.cc:1240] T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b [term 23 FOLLOWER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 22. Current term is 23. Ops: [22.15-22.15]
I20260709 13:22:06.999809 16192 leader_election.cc:304] T 8c11074579b046d99e585c2818e18a2c P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:22:07.004452 16191 leader_election.cc:304] T 68a6838d82ef4cac83507f906139d90f P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 90f3da57796b488c9087a2f523adf045
I20260709 13:22:06.999033 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c2095a5f4ad49408dbc7f69c606120d" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 17 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:07.011554 16218 raft_consensus.cc:2393] T 19baa7bc367844c288bca8f81439c413 P 1ca2725d290e47d68b1a4b609c9816ac [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 in current term 22: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
I20260709 13:22:06.930059 23087 raft_consensus.cc:515] T fffab26427e24d398e2695aad40ca134 P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:07.012835 16220 raft_consensus.cc:2468] T ecc1e363230043fe9fef390f0b979f20 P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 21.
I20260709 13:22:07.008090 16117 leader_election.cc:304] T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860
I20260709 13:22:07.011238 16150 raft_consensus.cc:2468] T c0eab25ce4c64e018a041beeda2867db P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 19.
I20260709 13:22:06.993184 16266 leader_election.cc:304] T 9e65676b21b748728ad39668cfe722ec P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:22:07.021035 16790 leader_election.cc:304] T 8638ee363273426db1393dd81e5c7b72 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:22:06.993708 23166 raft_consensus.cc:2804] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:22:06.991863 22677 leader_election.cc:290] T 5a959dbecc2648eb85d1cc550f5a3245 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
W20260709 13:22:07.026666 16263 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:22:07.027360 23280 raft_consensus.cc:493] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:22:07.027601 16220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "829b4ad8c5a54eababeea1181b960d4b" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:07.028347 16220 raft_consensus.cc:2468] T 829b4ad8c5a54eababeea1181b960d4b P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 16.
I20260709 13:22:07.029326 16222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c61a39fd4f84424b129c1a6f9ba404d" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:07.029624 16220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2372981006d4b218215c28964dea623" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:07.030011 23024 raft_consensus.cc:2804] T 9e65676b21b748728ad39668cfe722ec P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:22:07.030426 23024 raft_consensus.cc:493] T 9e65676b21b748728ad39668cfe722ec P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:07.030586 16222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c74490c40243a88e62d1760b5cd814" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:07.030865 23024 raft_consensus.cc:3060] T 9e65676b21b748728ad39668cfe722ec P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:22:07.031217 23199 consensus_meta.cc:287] T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860: Time spent flushing consensus metadata: real 0.783s	user 0.008s	sys 0.012s
I20260709 13:22:07.032941 23090 raft_consensus.cc:2749] T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:22:07.028030 23280 raft_consensus.cc:515] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:06.957361 22610 consensus_queue.cc:1048] T 2794af5b99984efebdee564035841cb0 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.052s
I20260709 13:22:07.030441 16220 raft_consensus.cc:2393] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 23: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
I20260709 13:22:07.009167 23216 raft_consensus.cc:493] T 8d4ab870a05347868956aa8b841b2981 P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:07.040903 16222 raft_consensus.cc:2468] T 08c74490c40243a88e62d1760b5cd814 P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 18.
W20260709 13:22:07.025004 16266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 67705.4us
I20260709 13:22:06.969156 23125 raft_consensus.cc:515] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:07.041134 16296 raft_consensus.cc:3060] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:22:07.048486 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:22:07.016021 22917 raft_consensus.cc:2804] T 8c11074579b046d99e585c2818e18a2c P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:22:07.020603 22701 raft_consensus.cc:1081] T d2ab0ac8347641e6955e9bce7d90c1a2 P cb82cc7bd6db46d980c75237a1402860 [term 16 LEADER]: Signalling peer 90f3da57796b488c9087a2f523adf045 to start an election
I20260709 13:22:07.055348 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e4bfb9090c8460eb14fa7e0d818e7f1" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:22:07.061153 16225 raft_consensus.cc:3060] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:22:07.035332 22748 raft_consensus.cc:2804] T 68a6838d82ef4cac83507f906139d90f P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:22:07.035363 23199 raft_consensus.cc:493] T ada5a004ba7c46819051e3ccec71ffdb P cb82cc7bd6db46d980c75237a1402860 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:07.064226 16149 raft_consensus.cc:1275] T fffab26427e24d398e2695aad40ca134 P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 13:22:07.065075 16222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af57996a8be84d398f0f023e8d08ef77" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:07.069267 16291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26354361e9b044cea1c2b86c4882fe92" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 17 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:07.004060 23167 raft_consensus.cc:515] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 1ca2725d290e47d68b1a4b609c9816ac [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:07.017638 23177 consensus_queue.cc:1059] T 57a744f5aaf944678a63469cbf514015 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: INVALID_TERM, Last received: 15.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.040s
I20260709 13:22:07.078025 16150 raft_consensus.cc:2468] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 22.
I20260709 13:22:07.056382 22917 raft_consensus.cc:493] T 8c11074579b046d99e585c2818e18a2c P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Starting leader election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
W20260709 13:22:07.090476 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 99381.4us
I20260709 13:22:07.091560 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87ac0d1f7e8a419f9bcbc04aee0e067e" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:22:07.065102 23253 raft_consensus.cc:493] T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:22:07.068675 16265 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.14.177.129:36741 timed out after 1.915s (SENT) [suppressed 13 similar messages]
I20260709 13:22:07.096562 16222 raft_consensus.cc:2393] T af57996a8be84d398f0f023e8d08ef77 P 1ca2725d290e47d68b1a4b609c9816ac [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 20: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
I20260709 13:22:07.098820 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a9de347c711477e87c582d003f36925" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:22:07.067770 22748 raft_consensus.cc:493] T 68a6838d82ef4cac83507f906139d90f P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Starting leader election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:22:07.095993 16143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2794af5b99984efebdee564035841cb0" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:22:07.106951 22645 raft_consensus.cc:515] T c0eab25ce4c64e018a041beeda2867db P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:07.108974 22645 leader_election.cc:290] T c0eab25ce4c64e018a041beeda2867db P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 20 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:22:07.110697 16191 leader_election.cc:304] T 1df93779209748b9860f6d893dd3de53 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 90f3da57796b488c9087a2f523adf045
I20260709 13:22:07.111661 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806d7690c2b644959152074bb2cab0a7" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:22:07.112551 16149 raft_consensus.cc:2410] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 for term 19 because replica has last-logged OpId of term: 18 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 12.
I20260709 13:22:07.113668 16149 raft_consensus.cc:1240] T 18ad7e3af5504571a69d10c00750a01c P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 16. Current term is 17. Ops: []
I20260709 13:22:07.114782 16149 raft_consensus.cc:1240] T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 16. Current term is 17. Ops: []
I20260709 13:22:07.119843 23061 leader_election.cc:290] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:07.120723 23061 raft_consensus.cc:2804] T 1df93779209748b9860f6d893dd3de53 P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:22:07.121093 23061 raft_consensus.cc:493] T 1df93779209748b9860f6d893dd3de53 P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Starting leader election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:22:07.121438 23061 raft_consensus.cc:3060] T 1df93779209748b9860f6d893dd3de53 P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:22:07.124368 23204 raft_consensus.cc:3055] T 57a744f5aaf944678a63469cbf514015 P cb82cc7bd6db46d980c75237a1402860 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:22:07.124781 23204 raft_consensus.cc:740] T 57a744f5aaf944678a63469cbf514015 P cb82cc7bd6db46d980c75237a1402860 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:22:07.125597 23204 consensus_queue.cc:260] T 57a744f5aaf944678a63469cbf514015 P cb82cc7bd6db46d980c75237a1402860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:22:07.126398 23023 consensus_meta.cc:287] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b: Time spent flushing consensus metadata: real 0.664s	user 0.004s	sys 0.008s
I20260709 13:22:07.130363 23087 leader_election.cc:290] T fffab26427e24d398e2695aad40ca134 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:07.132364 16220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a878999c0024d2abe517484180290fc" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:07.133041 16220 raft_consensus.cc:2468] T 8a878999c0024d2abe517484180290fc P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 17.
I20260709 13:22:07.135702 16145 raft_consensus.cc:1240] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b [term 21 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 20. Current term is 21. Ops: []
I20260709 13:22:07.138847 16145 raft_consensus.cc:1240] T 3884f864a96d428785895e5463e5728f P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 14. Current term is 15. Ops: []
I20260709 13:22:07.139751 23256 leader_election.cc:290] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
W20260709 13:22:07.093143 23188 consensus_meta.cc:287] T 66f75124215f4ca6bef831e174a792d8 P cb82cc7bd6db46d980c75237a1402860: Time spent flushing consensus metadata: real 0.718s	user 0.012s	sys 0.001s
I20260709 13:22:07.097049 22917 raft_consensus.cc:3060] T 8c11074579b046d99e585c2818e18a2c P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:22:07.155651 16791 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:22:07.063589 23216 raft_consensus.cc:515] T 8d4ab870a05347868956aa8b841b2981 P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:07.154664 16818 raft_consensus.cc:2468] T 66f75124215f4ca6bef831e174a792d8 P cb82cc7bd6db46d980c75237a1402860 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 17.
I20260709 13:22:07.156651 16223 raft_consensus.cc:2468] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 26.
I20260709 13:22:07.153218 16298 raft_consensus.cc:2468] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 21.
W20260709 13:22:07.161676 16114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:22:07.104648 22748 raft_consensus.cc:3060] T 68a6838d82ef4cac83507f906139d90f P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:22:07.165172 16266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 90037.1us
I20260709 13:22:07.166194 16115 leader_election.cc:304] T 8158bbd17b994d2eb96f560bd0f1b88d P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 90f3da57796b488c9087a2f523adf045
I20260709 13:22:07.167397 23187 raft_consensus.cc:2749] T 8638ee363273426db1393dd81e5c7b72 P cb82cc7bd6db46d980c75237a1402860 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:22:07.162525 16218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806d7690c2b644959152074bb2cab0a7" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:07.126849 23023 raft_consensus.cc:515] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b [term 22 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:07.170096 16222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66f75124215f4ca6bef831e174a792d8" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
W20260709 13:22:07.179255 16115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 99758.3us
I20260709 13:22:07.173246 16149 raft_consensus.cc:1275] T e5ad4eb281564936a59f41e387a47156 P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 19 index: 12. (term mismatch)
I20260709 13:22:07.084720 23199 raft_consensus.cc:515] T ada5a004ba7c46819051e3ccec71ffdb P cb82cc7bd6db46d980c75237a1402860 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:07.158061 22864 raft_consensus.cc:493] T 434bff3082d141838068f323bdb0d1ed P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
W20260709 13:22:07.225319 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 50216.7us
W20260709 13:22:07.227396 16115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 46721.5us
W20260709 13:22:07.235299 16791 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:22:07.236395 16296 raft_consensus.cc:2468] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 23.
I20260709 13:22:07.160040 23202 leader_election.cc:290] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:22:07.186276 22610 consensus_queue.cc:1048] T ecc1e363230043fe9fef390f0b979f20 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:22:07.187260 23125 leader_election.cc:290] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:07.188485 16144 raft_consensus.cc:3060] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:22:07.189797 23023 leader_election.cc:290] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 22 election: Requested vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:07.191918 23280 leader_election.cc:290] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:07.195550 23086 leader_election.cc:290] T 18ad7e3af5504571a69d10c00750a01c P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 17 election: Requested vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:22:07.155479 23253 raft_consensus.cc:515] T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:07.201085 23199 leader_election.cc:290] T ada5a004ba7c46819051e3ccec71ffdb P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:07.268957 23253 leader_election.cc:290] T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:07.159861 23188 raft_consensus.cc:493] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:07.271574 23188 raft_consensus.cc:515] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:07.276271 16142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0eab25ce4c64e018a041beeda2867db" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:22:07.277124 16142 raft_consensus.cc:2468] T c0eab25ce4c64e018a041beeda2867db P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 19.
I20260709 13:22:07.278216 16142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a878999c0024d2abe517484180290fc" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:22:07.278878 16142 raft_consensus.cc:2468] T 8a878999c0024d2abe517484180290fc P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 17.
I20260709 13:22:07.210875 16220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fffab26427e24d398e2695aad40ca134" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:07.279804 16142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4f65cdf00b748bb963ee6e81ef43b79" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:22:07.280827 16142 raft_consensus.cc:2393] T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 21: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
I20260709 13:22:07.280627 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c61a39fd4f84424b129c1a6f9ba404d" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 21 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:22:07.281647 16142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2794af5b99984efebdee564035841cb0" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:22:07.281905 16150 raft_consensus.cc:2410] T 9c61a39fd4f84424b129c1a6f9ba404d P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 for term 21 because replica has last-logged OpId of term: 19 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 9.
I20260709 13:22:07.214529 16821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62fe3346cd3c42c4a55e14af9073bca4" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 26 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:07.282931 16821 raft_consensus.cc:2393] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 26: Already voted for candidate cb82cc7bd6db46d980c75237a1402860 in this term.
I20260709 13:22:07.215731 16820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a378d7f796f94615b3b347119a9880fd" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 22 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:07.283627 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4dc6b0541ca431ebc757beac00521f1" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 21 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:07.217855 16222 raft_consensus.cc:1275] T 746aabf856ab401c9ef5e859371e7b67 P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 16 index: 14. (index mismatch)
I20260709 13:22:07.285413 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c11074579b046d99e585c2818e18a2c" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:07.219599 16226 raft_consensus.cc:1240] T af57996a8be84d398f0f023e8d08ef77 P 1ca2725d290e47d68b1a4b609c9816ac [term 20 FOLLOWER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 19. Current term is 20. Ops: []
I20260709 13:22:07.282848 16142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e65676b21b748728ad39668cfe722ec" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:22:07.183967 23087 raft_consensus.cc:2804] T 8158bbd17b994d2eb96f560bd0f1b88d P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:22:07.181303 16149 pending_rounds.cc:85] T e5ad4eb281564936a59f41e387a47156 P 647a343a542d4a139fe106c19095129b: Aborting all ops after (but not including) 11
I20260709 13:22:07.295862 16149 pending_rounds.cc:107] T e5ad4eb281564936a59f41e387a47156 P 647a343a542d4a139fe106c19095129b: Aborting uncommitted NO_OP operation due to leader change: 18.12
I20260709 13:22:07.296314 16149 raft_consensus.cc:2889] T e5ad4eb281564936a59f41e387a47156 P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:22:07.286998 16190 leader_election.cc:304] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860
I20260709 13:22:07.254109 16115 leader_election.cc:304] T 9e65676b21b748728ad39668cfe722ec P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:22:07.256445 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a96373bb0e14dce8a266b2fc6c58507" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:07.272168 16291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ad5dbfa99ee4b8d9e28be92b3f36db5" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:07.297283 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c2095a5f4ad49408dbc7f69c606120d" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 17 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:07.205374 23216 leader_election.cc:290] T 8d4ab870a05347868956aa8b841b2981 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:22:07.279721 16143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4dc6b0541ca431ebc757beac00521f1" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:22:07.280006 16147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d16dfad0c3b540e5b4b86492ddf87208" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:22:07.245016 16225 raft_consensus.cc:2468] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 21.
I20260709 13:22:07.208382 23024 raft_consensus.cc:515] T 9e65676b21b748728ad39668cfe722ec P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:07.297760 16291 raft_consensus.cc:2468] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 18.
I20260709 13:22:07.313405 16146 raft_consensus.cc:1275] T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 19 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 13:22:07.164870 22908 raft_consensus.cc:515] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:07.315240 16142 raft_consensus.cc:2468] T 9e65676b21b748728ad39668cfe722ec P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 18.
W20260709 13:22:07.322028 16115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 93721.3us
I20260709 13:22:07.328909 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d16dfad0c3b540e5b4b86492ddf87208" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:07.295800 23087 raft_consensus.cc:493] T 8158bbd17b994d2eb96f560bd0f1b88d P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Starting leader election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
W20260709 13:22:07.334882 16789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 340572us
I20260709 13:22:07.299922 23188 leader_election.cc:290] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:07.336665 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a959dbecc2648eb85d1cc550f5a3245" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:07.339100 16291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a378d7f796f94615b3b347119a9880fd" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 22 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
W20260709 13:22:07.340477 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 53568us
I20260709 13:22:07.334614 16221 raft_consensus.cc:2468] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 20.
I20260709 13:22:07.323693 22873 leader_election.cc:290] T d2ab0ac8347641e6955e9bce7d90c1a2 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 17 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:07.322837 22677 raft_consensus.cc:2749] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:22:07.347712 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:22:07.349853 16789 leader_election.cc:304] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:22:07.264874 22864 raft_consensus.cc:515] T 434bff3082d141838068f323bdb0d1ed P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:07.354425 16226 raft_consensus.cc:1275] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac [term 22 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 22 index: 16. (index mismatch)
I20260709 13:22:07.356524 23188 raft_consensus.cc:2804] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P cb82cc7bd6db46d980c75237a1402860 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:22:07.356889 16789 leader_election.cc:304] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:22:07.357316 23188 raft_consensus.cc:697] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P cb82cc7bd6db46d980c75237a1402860 [term 21 LEADER]: Becoming Leader. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:22:07.358179 23211 raft_consensus.cc:2804] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:22:07.363819 16816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2794af5b99984efebdee564035841cb0" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:22:07.364748 16816 raft_consensus.cc:3060] T 2794af5b99984efebdee564035841cb0 P cb82cc7bd6db46d980c75237a1402860 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:22:07.340276 23087 raft_consensus.cc:3060] T 8158bbd17b994d2eb96f560bd0f1b88d P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:22:07.360100 16115 leader_election.cc:304] T 9c61a39fd4f84424b129c1a6f9ba404d P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:22:07.359184 23216 consensus_queue.cc:1048] T c9944d5e4b8c4109af9955c4f08c9095 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.009s
I20260709 13:22:07.355545 23166 raft_consensus.cc:697] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860 [term 17 LEADER]: Becoming Leader. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
W20260709 13:22:07.393175 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 324521us
I20260709 13:22:07.383678 16190 leader_election.cc:304] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860
I20260709 13:22:07.358758 23211 raft_consensus.cc:697] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860 [term 23 LEADER]: Becoming Leader. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:22:07.387827 16821 raft_consensus.cc:1240] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [term 26 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 25. Current term is 26. Ops: [25.16-25.16]
I20260709 13:22:07.395748 16824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:07.358731 23188 consensus_queue.cc:237] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:22:07.404248 16789 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:22:07.405325 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 397496us
I20260709 13:22:07.405618 16824 raft_consensus.cc:2468] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 21.
I20260709 13:22:07.405130 16144 raft_consensus.cc:2468] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 23.
I20260709 13:22:07.352984 23257 consensus_queue.cc:1048] T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.037s
W20260709 13:22:07.411823 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 301238us
I20260709 13:22:07.412439 15046 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:22:07.420622 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e9b2bf7011f4ae688a79d7bb5a447b4" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:07.421802 16299 raft_consensus.cc:1275] T 8cc959fb7f704fba9009c0595467d074 P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 13:22:07.422343 16298 raft_consensus.cc:1240] T d2ab0ac8347641e6955e9bce7d90c1a2 P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 16. Current term is 17. Ops: []
I20260709 13:22:07.392333 23086 raft_consensus.cc:2804] T 9c61a39fd4f84424b129c1a6f9ba404d P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:22:07.425298 16825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ab0ac8347641e6955e9bce7d90c1a2" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:22:07.428220 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdbd78acb4cf46298e0de8ba0d130e19" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:07.430822 16266 leader_election.cc:304] T ecc1e363230043fe9fef390f0b979f20 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:22:07.421584 23280 raft_consensus.cc:493] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:07.435299 16225 raft_consensus.cc:1275] T ecc1e363230043fe9fef390f0b979f20 P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
I20260709 13:22:07.437618 16790 leader_election.cc:304] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:22:07.420168 22946 raft_consensus.cc:2749] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:22:07.451057 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 81340.6us
I20260709 13:22:07.455606 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada5a004ba7c46819051e3ccec71ffdb" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:07.428675 23086 raft_consensus.cc:493] T 9c61a39fd4f84424b129c1a6f9ba404d P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:07.457873 16191 leader_election.cc:304] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
I20260709 13:22:07.460606 16190 leader_election.cc:304] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860
I20260709 13:22:07.440827 23248 raft_consensus.cc:2804] T 9e65676b21b748728ad39668cfe722ec P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:22:07.469991 16115 leader_election.cc:304] T 8a878999c0024d2abe517484180290fc P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
W20260709 13:22:07.471985 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 48572.8us
I20260709 13:22:07.431917 23291 consensus_queue.cc:1059] T d2ab0ac8347641e6955e9bce7d90c1a2 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: INVALID_TERM, Last received: 16.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.031s
W20260709 13:22:07.481091 16266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 50377.4us
I20260709 13:22:07.473766 16190 leader_election.cc:304] T c0eab25ce4c64e018a041beeda2867db P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:22:07.480154 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ad5dbfa99ee4b8d9e28be92b3f36db5" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:07.459734 23086 raft_consensus.cc:3060] T 9c61a39fd4f84424b129c1a6f9ba404d P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:22:07.487221 16221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ab0ac8347641e6955e9bce7d90c1a2" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:22:07.472229 16295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a878999c0024d2abe517484180290fc" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 18 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:07.495725 16295 raft_consensus.cc:2410] T 8a878999c0024d2abe517484180290fc P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac for term 18 because replica has last-logged OpId of term: 16 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 8.
I20260709 13:22:07.432049 23280 raft_consensus.cc:515] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:07.469135 23248 raft_consensus.cc:493] T 9e65676b21b748728ad39668cfe722ec P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:07.502712 16190 leader_election.cc:304] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:22:07.504046 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d4ab870a05347868956aa8b841b2981" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:07.505200 16221 raft_consensus.cc:3060] T d2ab0ac8347641e6955e9bce7d90c1a2 P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:22:07.427631 16293 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d2ab0ac8347641e6955e9bce7d90c1a2"
dest_uuid: "90f3da57796b488c9087a2f523adf045"
 from {username='slave'} at 127.0.0.1:40474
I20260709 13:22:07.508308 16293 raft_consensus.cc:493] T d2ab0ac8347641e6955e9bce7d90c1a2 P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:22:07.508626 16293 raft_consensus.cc:3060] T d2ab0ac8347641e6955e9bce7d90c1a2 P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:22:07.512883 16190 leader_election.cc:304] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860
I20260709 13:22:07.495879 23286 raft_consensus.cc:2804] T 8a878999c0024d2abe517484180290fc P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:22:07.500962 23024 leader_election.cc:290] T 9e65676b21b748728ad39668cfe722ec P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 19 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
W20260709 13:22:07.525940 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 101590us
I20260709 13:22:07.503741 23248 raft_consensus.cc:3060] T 9e65676b21b748728ad39668cfe722ec P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:22:07.500573 22677 raft_consensus.cc:3060] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:22:07.509323 23287 raft_consensus.cc:2804] T ecc1e363230043fe9fef390f0b979f20 P 90f3da57796b488c9087a2f523adf045 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:22:07.505028 23256 raft_consensus.cc:2804] T c0eab25ce4c64e018a041beeda2867db P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:22:07.532939 16816 raft_consensus.cc:2468] T 2794af5b99984efebdee564035841cb0 P cb82cc7bd6db46d980c75237a1402860 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 17.
I20260709 13:22:07.504077 23233 raft_consensus.cc:493] T 26354361e9b044cea1c2b86c4882fe92 P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:22:07.536175 16820 raft_consensus.cc:1275] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860 [term 21 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
W20260709 13:22:07.534209 22960 consensus_meta.cc:287] T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045: Time spent flushing consensus metadata: real 0.627s	user 0.009s	sys 0.004s
I20260709 13:22:07.550241 23292 raft_consensus.cc:2749] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [term 26 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:22:07.550590 22960 raft_consensus.cc:515] T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:22:07.552534 23293 log.cc:927] Time spent T 23f82224f4564fea93572e8206bad1fe P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
W20260709 13:22:07.548592 23271 log.cc:927] Time spent T 5a96373bb0e14dce8a266b2fc6c58507 P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 0.812s	user 0.000s	sys 0.006s
W20260709 13:22:07.548856 23283 log.cc:927] Time spent T 00000000000000000000000000000000 P 667a34b27a39474ebefc1af3cf5b4271: Append to log took a long time: real 0.162s	user 0.000s	sys 0.001s
I20260709 13:22:07.549618 23294 raft_consensus.cc:3060] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:22:07.506927 23023 consensus_queue.cc:1059] T 2794af5b99984efebdee564035841cb0 P 647a343a542d4a139fe106c19095129b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: INVALID_TERM, Last received: 16.14, Next index: 15, Last known committed idx: 13, Time since last communication: 0.029s
I20260709 13:22:07.555702 23023 raft_consensus.cc:3055] T 2794af5b99984efebdee564035841cb0 P 647a343a542d4a139fe106c19095129b [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:22:07.556245 23023 raft_consensus.cc:740] T 2794af5b99984efebdee564035841cb0 P 647a343a542d4a139fe106c19095129b [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:22:07.557237 23023 consensus_queue.cc:260] T 2794af5b99984efebdee564035841cb0 P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 16.14, Last appended by leader: 14, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:07.558437 23023 raft_consensus.cc:3060] T 2794af5b99984efebdee564035841cb0 P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:22:07.538064 23256 raft_consensus.cc:493] T c0eab25ce4c64e018a041beeda2867db P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:07.559530 23256 raft_consensus.cc:3060] T c0eab25ce4c64e018a041beeda2867db P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:22:07.559573 22864 leader_election.cc:290] T 434bff3082d141838068f323bdb0d1ed P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:07.536301 23287 raft_consensus.cc:493] T ecc1e363230043fe9fef390f0b979f20 P 90f3da57796b488c9087a2f523adf045 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:07.526170 23286 raft_consensus.cc:493] T 8a878999c0024d2abe517484180290fc P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:07.563241 23287 raft_consensus.cc:3060] T ecc1e363230043fe9fef390f0b979f20 P 90f3da57796b488c9087a2f523adf045 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:22:07.553083 22960 leader_election.cc:290] T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 21 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:07.563387 23286 raft_consensus.cc:3060] T 8a878999c0024d2abe517484180290fc P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:22:07.563078 23294 raft_consensus.cc:2749] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:22:07.527218 22908 leader_election.cc:290] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 20 election: Requested vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
W20260709 13:22:07.575832 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 135277us
I20260709 13:22:07.568419 16293 raft_consensus.cc:515] T d2ab0ac8347641e6955e9bce7d90c1a2 P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:07.586638 23087 raft_consensus.cc:515] T 8158bbd17b994d2eb96f560bd0f1b88d P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:07.589036 16145 raft_consensus.cc:1240] T 8158bbd17b994d2eb96f560bd0f1b88d P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 17. Current term is 18. Ops: []
I20260709 13:22:07.591089 23087 leader_election.cc:290] T 8158bbd17b994d2eb96f560bd0f1b88d P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 18 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:07.498041 23166 consensus_queue.cc:237] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:07.602614 16293 leader_election.cc:290] T d2ab0ac8347641e6955e9bce7d90c1a2 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 18 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
W20260709 13:22:07.611755 16266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 119901us
I20260709 13:22:07.503371 23211 consensus_queue.cc:237] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.15, Last appended by leader: 15, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:22:07.623999 16791 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:22:07.543874 23061 raft_consensus.cc:515] T 1df93779209748b9860f6d893dd3de53 P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:07.635282 16191 leader_election.cc:304] T e5ad4eb281564936a59f41e387a47156 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
W20260709 13:22:07.635690 16115 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:22:07.636336 16263 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:22:07.611444 23294 raft_consensus.cc:2804] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:22:07.641657 23294 raft_consensus.cc:493] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Starting leader election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:22:07.642185 23294 raft_consensus.cc:3060] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:22:07.614679 22960 consensus_queue.cc:1059] T 66ae176961b74813a58bc15c9c4c6f2a P 90f3da57796b488c9087a2f523adf045 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: INVALID_TERM, Last received: 24.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.029s
I20260709 13:22:07.546721 23233 raft_consensus.cc:515] T 26354361e9b044cea1c2b86c4882fe92 P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:07.658787 16222 raft_consensus.cc:1240] T 1df93779209748b9860f6d893dd3de53 P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 17. Current term is 18. Ops: []
I20260709 13:22:07.635201 23296 raft_consensus.cc:2749] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:22:07.683579 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fffab26427e24d398e2695aad40ca134" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:07.695814 16295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18ad7e3af5504571a69d10c00750a01c" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:22:07.689591 22864 raft_consensus.cc:493] T 57a744f5aaf944678a63469cbf514015 P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:07.696451 23210 raft_consensus.cc:3055] T 66ae176961b74813a58bc15c9c4c6f2a P 90f3da57796b488c9087a2f523adf045 [term 25 LEADER]: Stepping down as leader of term 25
I20260709 13:22:07.696971 23210 raft_consensus.cc:740] T 66ae176961b74813a58bc15c9c4c6f2a P 90f3da57796b488c9087a2f523adf045 [term 25 LEADER]: Becoming Follower/Learner. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:22:07.696820 22864 raft_consensus.cc:515] T 57a744f5aaf944678a63469cbf514015 P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:07.698105 23210 consensus_queue.cc:260] T 66ae176961b74813a58bc15c9c4c6f2a P 90f3da57796b488c9087a2f523adf045 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 25.16, Last appended by leader: 16, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:07.699216 16296 raft_consensus.cc:1275] T 8d4ab870a05347868956aa8b841b2981 P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 13:22:07.663102 22908 raft_consensus.cc:2749] T e5ad4eb281564936a59f41e387a47156 P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:22:07.659664 23161 raft_consensus.cc:493] T 23f82224f4564fea93572e8206bad1fe P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:22:07.706348 16225 raft_consensus.cc:2468] T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 18.
I20260709 13:22:07.706784 23161 raft_consensus.cc:515] T 23f82224f4564fea93572e8206bad1fe P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:07.727783 16151 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40844 (request call id 2944) took 1160 ms. Trace:
I20260709 13:22:07.729271 16151 rpcz_store.cc:276] 0709 13:22:06.566915 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:22:06.588379 (+ 21464us) service_pool.cc:224] Handling call
0709 13:22:07.727509 (+1139130us) spinlock_profiling.cc:243] Waited 298 ms on lock 0x7b54000d1954. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:22:07.727671 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":685784448}
I20260709 13:22:07.733542 23204 raft_consensus.cc:3060] T 57a744f5aaf944678a63469cbf514015 P cb82cc7bd6db46d980c75237a1402860 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:22:07.741925 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0eab25ce4c64e018a041beeda2867db" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
W20260709 13:22:07.772863 16791 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:22:07.795732 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 282906us
W20260709 13:22:07.766211 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 256097us
I20260709 13:22:07.804641 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a9de347c711477e87c582d003f36925" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:22:07.805410 16298 raft_consensus.cc:2368] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b for earlier term 22. Current term is 23.
I20260709 13:22:07.802387 23291 raft_consensus.cc:493] T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:07.810724 16296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4dc6b0541ca431ebc757beac00521f1" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
W20260709 13:22:07.811378 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 208431us
I20260709 13:22:07.796600 23210 raft_consensus.cc:3060] T 66ae176961b74813a58bc15c9c4c6f2a P 90f3da57796b488c9087a2f523adf045 [term 25 FOLLOWER]: Advancing to term 26
W20260709 13:22:07.815424 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 200532us
I20260709 13:22:07.817066 23024 consensus_queue.cc:1048] T c9944d5e4b8c4109af9955c4f08c9095 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.002s
I20260709 13:22:07.818547 23311 raft_consensus.cc:493] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:07.819116 23311 raft_consensus.cc:515] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:07.819698 22908 raft_consensus.cc:493] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:07.820251 22908 raft_consensus.cc:515] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:07.821192 23311 leader_election.cc:290] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:07.821811 23291 raft_consensus.cc:515] T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:07.822330 22908 leader_election.cc:290] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:07.822621 23309 consensus_queue.cc:1059] T 5a959dbecc2648eb85d1cc550f5a3245 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:22:07.823412 23309 consensus_queue.cc:1243] T 5a959dbecc2648eb85d1cc550f5a3245 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Peer 647a343a542d4a139fe106c19095129b log is divergent from this leader: its last log entry 17.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 13:22:07.823863 23307 raft_consensus.cc:3055] T 5a959dbecc2648eb85d1cc550f5a3245 P 90f3da57796b488c9087a2f523adf045 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:22:07.824222 23307 raft_consensus.cc:740] T 5a959dbecc2648eb85d1cc550f5a3245 P 90f3da57796b488c9087a2f523adf045 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:22:07.825068 23307 consensus_queue.cc:260] T 5a959dbecc2648eb85d1cc550f5a3245 P 90f3da57796b488c9087a2f523adf045 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:07.825865 16265 leader_election.cc:304] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:22:07.826508 23307 raft_consensus.cc:3060] T 5a959dbecc2648eb85d1cc550f5a3245 P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:22:07.826665 23309 consensus_queue.cc:1048] T fffab26427e24d398e2695aad40ca134 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:22:07.835773 16148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26354361e9b044cea1c2b86c4882fe92" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 17 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:22:07.836236 16291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8158bbd17b994d2eb96f560bd0f1b88d" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:22:07.836452 16151 raft_consensus.cc:1240] T 829b4ad8c5a54eababeea1181b960d4b P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 16. Current term is 17. Ops: []
I20260709 13:22:07.840934 16191 leader_election.cc:304] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 90f3da57796b488c9087a2f523adf045
I20260709 13:22:07.844194 16223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ab0ac8347641e6955e9bce7d90c1a2" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: true dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:22:07.845101 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e9b2bf7011f4ae688a79d7bb5a447b4" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:22:07.832373 23024 raft_consensus.cc:2749] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:22:07.847081 16151 raft_consensus.cc:1275] T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 18 index: 15. (index mismatch)
I20260709 13:22:07.850948 16191 leader_election.cc:304] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
I20260709 13:22:07.851605 23173 leader_election.cc:290] T 19baa7bc367844c288bca8f81439c413 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:07.840430 23087 raft_consensus.cc:493] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:22:07.852756 22748 consensus_meta.cc:287] T 68a6838d82ef4cac83507f906139d90f P 1ca2725d290e47d68b1a4b609c9816ac: Time spent flushing consensus metadata: real 0.666s	user 0.017s	sys 0.003s
I20260709 13:22:07.853196 22748 raft_consensus.cc:515] T 68a6838d82ef4cac83507f906139d90f P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:07.854032 16142 raft_consensus.cc:1240] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 18. Current term is 19. Ops: []
I20260709 13:22:07.852414 23087 raft_consensus.cc:515] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:07.855129 16220 raft_consensus.cc:1240] T 68a6838d82ef4cac83507f906139d90f P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 18. Current term is 19. Ops: []
I20260709 13:22:07.855949 23167 leader_election.cc:290] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:07.856868 23087 leader_election.cc:290] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:07.857215 16220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74ca8140f35c4ae29e8a664f6bdfc829" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:22:07.857810 16220 raft_consensus.cc:3060] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:22:07.860796 23291 leader_election.cc:290] T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
W20260709 13:22:07.862036 23282 log.cc:927] Time spent T e5ad4eb281564936a59f41e387a47156 P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 0.473s	user 0.001s	sys 0.002s
I20260709 13:22:07.862098 23311 raft_consensus.cc:2804] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:22:07.862601 22864 leader_election.cc:290] T 57a744f5aaf944678a63469cbf514015 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:07.862658 23311 raft_consensus.cc:493] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Starting leader election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:22:07.863303 23311 raft_consensus.cc:3060] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:22:07.863991 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 730205us
I20260709 13:22:07.864727 16116 leader_election.cc:304] T 02838c86531d4f98bf58d1c22b8baf77 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:22:07.867756 22748 leader_election.cc:290] T 68a6838d82ef4cac83507f906139d90f P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 19 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:07.868572 23252 raft_consensus.cc:2749] T 02838c86531d4f98bf58d1c22b8baf77 P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:22:07.886627 16223 raft_consensus.cc:1275] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [term 22 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 21 index: 16. Preceding OpId from leader: term: 22 index: 17. (index mismatch)
W20260709 13:22:07.873010 22917 consensus_meta.cc:287] T 8c11074579b046d99e585c2818e18a2c P 1ca2725d290e47d68b1a4b609c9816ac: Time spent flushing consensus metadata: real 0.697s	user 0.016s	sys 0.004s
I20260709 13:22:07.874765 23161 leader_election.cc:290] T 23f82224f4564fea93572e8206bad1fe P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
W20260709 13:22:07.879598 16818 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49906 (request call id 1350) took 1219 ms (client timeout 1611 ms). Trace:
I20260709 13:22:07.880491 22677 raft_consensus.cc:2749] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:22:07.880934 16116 leader_election.cc:304] T 68a6838d82ef4cac83507f906139d90f P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:22:07.881624 23233 leader_election.cc:290] T 26354361e9b044cea1c2b86c4882fe92 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:07.882150 23248 raft_consensus.cc:515] T 9e65676b21b748728ad39668cfe722ec P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:07.834276 22908 consensus_queue.cc:1048] T 8cc959fb7f704fba9009c0595467d074 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.035s
I20260709 13:22:07.894184 23252 raft_consensus.cc:2749] T 68a6838d82ef4cac83507f906139d90f P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:22:07.894454 16226 raft_consensus.cc:1275] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac [term 20 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 13:22:07.883585 23173 raft_consensus.cc:3060] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:22:07.884128 16191 leader_election.cc:304] T ee9ed67006144797b1de6ea8c2013a68 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
W20260709 13:22:07.884403 23304 log.cc:927] Time spent T e4dc6b0541ca431ebc757beac00521f1 P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 0.240s	user 0.002s	sys 0.000s
I20260709 13:22:07.869315 23061 leader_election.cc:290] T 1df93779209748b9860f6d893dd3de53 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 18 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
W20260709 13:22:07.888862 16818 rpcz_store.cc:269] 0709 13:22:06.660332 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:22:06.660954 (+   622us) service_pool.cc:224] Handling call
0709 13:22:07.879388 (+1218434us) spinlock_profiling.cc:243] Waited 493 ms on lock 0x7b54005302d4. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:22:07.879518 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1133029120}
I20260709 13:22:07.892289 16265 leader_election.cc:304] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:22:07.897152 16294 raft_consensus.cc:1240] T d2ab0ac8347641e6955e9bce7d90c1a2 P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 16. Current term is 18. Ops: []
I20260709 13:22:07.899457 16821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e65676b21b748728ad39668cfe722ec" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:22:07.884229 23256 raft_consensus.cc:515] T c0eab25ce4c64e018a041beeda2867db P 1ca2725d290e47d68b1a4b609c9816ac [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:07.900002 16825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434bff3082d141838068f323bdb0d1ed" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:07.900101 16816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74ca8140f35c4ae29e8a664f6bdfc829" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:22:07.907814 16816 raft_consensus.cc:3060] T 74ca8140f35c4ae29e8a664f6bdfc829 P cb82cc7bd6db46d980c75237a1402860 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:22:07.903123 16821 raft_consensus.cc:3060] T 9e65676b21b748728ad39668cfe722ec P cb82cc7bd6db46d980c75237a1402860 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:22:07.899708 23289 raft_consensus.cc:3055] T d2ab0ac8347641e6955e9bce7d90c1a2 P cb82cc7bd6db46d980c75237a1402860 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:22:07.910080 23289 raft_consensus.cc:740] T d2ab0ac8347641e6955e9bce7d90c1a2 P cb82cc7bd6db46d980c75237a1402860 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:22:07.904465 16192 leader_election.cc:304] T 66f75124215f4ca6bef831e174a792d8 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:22:07.904623 16819 raft_consensus.cc:1275] T ada5a004ba7c46819051e3ccec71ffdb P cb82cc7bd6db46d980c75237a1402860 [term 20 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 13:22:07.905313 16824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ab0ac8347641e6955e9bce7d90c1a2" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: true dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:22:07.909508 23061 raft_consensus.cc:2749] T ee9ed67006144797b1de6ea8c2013a68 P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:22:07.909396 16818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5718c623ebee4d2bb770f0f9ddb2f164" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:07.914896 16220 raft_consensus.cc:2468] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 21.
I20260709 13:22:07.915664 16825 raft_consensus.cc:2468] T 434bff3082d141838068f323bdb0d1ed P cb82cc7bd6db46d980c75237a1402860 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 18.
W20260709 13:22:07.918824 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 49712.4us
I20260709 13:22:07.921095 16817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87147731a68400e8cce9b879daa7c5d" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:07.921469 16225 raft_consensus.cc:2393] T c0eab25ce4c64e018a041beeda2867db P 1ca2725d290e47d68b1a4b609c9816ac [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 20: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
I20260709 13:22:07.914083 22908 raft_consensus.cc:2804] T 66f75124215f4ca6bef831e174a792d8 P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:22:07.923939 22908 raft_consensus.cc:493] T 66f75124215f4ca6bef831e174a792d8 P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Starting leader election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:22:07.924360 22908 raft_consensus.cc:3060] T 66f75124215f4ca6bef831e174a792d8 P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:22:07.929446 16818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e65676b21b748728ad39668cfe722ec" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:07.927755 16116 leader_election.cc:304] T 746aabf856ab401c9ef5e859371e7b67 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
I20260709 13:22:07.932765 23188 consensus_queue.cc:1048] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:22:07.931008 16224 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57600 (request call id 2071) took 1079 ms. Trace:
I20260709 13:22:07.937233 16821 raft_consensus.cc:2468] T 9e65676b21b748728ad39668cfe722ec P cb82cc7bd6db46d980c75237a1402860 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 19.
I20260709 13:22:07.938512 16116 leader_election.cc:304] T 1df93779209748b9860f6d893dd3de53 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 90f3da57796b488c9087a2f523adf045
I20260709 13:22:07.941244 16816 raft_consensus.cc:2468] T 74ca8140f35c4ae29e8a664f6bdfc829 P cb82cc7bd6db46d980c75237a1402860 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 21.
I20260709 13:22:07.891084 22917 raft_consensus.cc:515] T 8c11074579b046d99e585c2818e18a2c P 1ca2725d290e47d68b1a4b609c9816ac [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:07.942701 16219 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57600 (request call id 2070) took 1092 ms. Trace:
I20260709 13:22:07.923975 23161 raft_consensus.cc:3060] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:22:07.944384 23252 raft_consensus.cc:2804] T 746aabf856ab401c9ef5e859371e7b67 P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 13:22:07.945150 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 52956.7us
I20260709 13:22:07.952075 16818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "829b4ad8c5a54eababeea1181b960d4b" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:07.953949 23087 raft_consensus.cc:2804] T 1df93779209748b9860f6d893dd3de53 P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:22:07.954293 16116 leader_election.cc:304] T e5ad4eb281564936a59f41e387a47156 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:22:07.954440 23087 raft_consensus.cc:493] T 1df93779209748b9860f6d893dd3de53 P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Starting leader election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:22:07.945071 16219 rpcz_store.cc:276] 0709 13:22:06.824256 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:22:06.824387 (+   131us) service_pool.cc:224] Handling call
0709 13:22:06.836245 (+ 11858us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:22:06.857948 (+ 21703us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:22:06.858259 (+   311us) log.cc:844] Serialized 14 byte log entry
0709 13:22:07.169382 (+311123us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:22:07.296482 (+127100us) log.cc:844] Serialized 26 byte log entry
0709 13:22:07.296638 (+   156us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:22:07.296677 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:22:07.296743 (+    66us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:22:07.861446 (+564703us) spinlock_profiling.cc:243] Waited 4.89 ms on lock 0x7b4400022b4c. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69ad8fadb1 00007f69ad998c85 00007f69ad9996a2 00007f69b070b1db 00007f69b0706aaf 00007f69b06fa86e 00007f69b06f9bb6 00007f69b06f97d8 00007f69b0703221 00007f69b07031d9 00007f69b0703169
0709 13:22:07.912045 (+ 50599us) raft_consensus.cc:1685] finished
0709 13:22:07.916666 (+  4621us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:22:07.916805 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":932,"prepare.run_cpu_time_us":33,"prepare.run_wall_time_us":34,"raft.queue_time_us":1738,"raft.run_cpu_time_us":224,"raft.run_wall_time_us":4410,"spinlock_wait_cycles":11257088,"thread_start_us":309849,"threads_started":1,"wal-append.queue_time_us":476140}
W20260709 13:22:07.968184 23299 log.cc:927] Time spent T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 0.310s	user 0.001s	sys 0.001s
I20260709 13:22:07.955040 23087 raft_consensus.cc:3060] T 1df93779209748b9860f6d893dd3de53 P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:22:07.910971 23289 consensus_queue.cc:260] T d2ab0ac8347641e6955e9bce7d90c1a2 P cb82cc7bd6db46d980c75237a1402860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 16.14, Last appended by leader: 14, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:07.986449 16217 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33050 (request call id 1223) took 1149 ms. Trace:
I20260709 13:22:07.986800 16116 leader_election.cc:304] T 829b4ad8c5a54eababeea1181b960d4b P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
I20260709 13:22:07.950707 23252 raft_consensus.cc:493] T 746aabf856ab401c9ef5e859371e7b67 P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:22:07.950568 23269 consensus_meta.cc:287] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac: Time spent flushing consensus metadata: real 1.178s	user 0.014s	sys 0.013s
I20260709 13:22:07.985533 16818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746aabf856ab401c9ef5e859371e7b67" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:07.986737 16790 leader_election.cc:304] T 26354361e9b044cea1c2b86c4882fe92 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:22:07.935446 16224 rpcz_store.cc:276] 0709 13:22:06.851785 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:22:07.047219 (+195434us) service_pool.cc:224] Handling call
0709 13:22:07.057228 (+ 10009us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:22:07.336711 (+279483us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:22:07.368184 (+ 31473us) log.cc:844] Serialized 14 byte log entry
0709 13:22:07.501974 (+133790us) log.cc:844] Serialized 14 byte log entry
0709 13:22:07.749025 (+247051us) spinlock_profiling.cc:243] Waited 3.95 ms on lock 0x7b18003347e0. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b693ade1 00007f69b69475b1 00007f69ada72522 00007f69ada747c1 00007f69ada74779 00007f69ada74709 00007f69ada746d1 00007f69ada739cd 000000000038f354 000000000038f199 00007f69ada5583a
0709 13:22:07.831349 (+ 82324us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:22:07.843129 (+ 11780us) log.cc:844] Serialized 52 byte log entry
0709 13:22:07.844908 (+  1779us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:22:07.848248 (+  3340us) log.cc:844] Serialized 14 byte log entry
0709 13:22:07.848445 (+   197us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:22:07.868645 (+ 20200us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:22:07.883672 (+ 15027us) spinlock_profiling.cc:243] Waited 18.7 ms on lock 0x7b4400022b4c. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69ad8fadb1 00007f69ad998c85 00007f69ad9996a2 00007f69b070b1db 00007f69b0706aaf 00007f69b06fa86e 00007f69b06f9bb6 00007f69b06f97d8 00007f69b0703221 00007f69b07031d9 00007f69b0703169
0709 13:22:07.927427 (+ 43755us) raft_consensus.cc:1685] finished
0709 13:22:07.927452 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:22:07.930938 (+  3486us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4547,"prepare.queue_time_us":46013,"prepare.run_cpu_time_us":106,"prepare.run_wall_time_us":110,"raft.queue_time_us":121087,"raft.run_cpu_time_us":1348,"raft.run_wall_time_us":41033,"spinlock_wait_cycles":52051584,"thread_start_us":659738,"threads_started":3,"wal-append.queue_time_us":249212}
I20260709 13:22:07.998008 23311 raft_consensus.cc:515] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:22:08.002611 16115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 156053us
I20260709 13:22:08.006268 16818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2372981006d4b218215c28964dea623" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:07.991219 16217 rpcz_store.cc:276] 0709 13:22:06.836732 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:22:06.851723 (+ 14991us) service_pool.cc:224] Handling call
0709 13:22:06.877394 (+ 25671us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:22:06.995806 (+118412us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 13:22:07.031936 (+ 36130us) log.cc:844] Serialized 14 byte log entry
0709 13:22:07.386686 (+354750us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:22:07.400313 (+ 13627us) log.cc:844] Serialized 26 byte log entry
0709 13:22:07.400465 (+   152us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:22:07.405284 (+  4819us) log.cc:844] Serialized 14 byte log entry
0709 13:22:07.417197 (+ 11913us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:22:07.700860 (+283663us) spinlock_profiling.cc:243] Waited 4.53 ms on lock 0x7b18003e0f20. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b693ade1 00007f69b69475b1 00007f69ada723f3 00007f69ada747c1 00007f69ada74779 00007f69ada74709 00007f69ada746d1 00007f69ada739cd 000000000038f354 000000000038f199 00007f69ada5583a
0709 13:22:07.986128 (+285268us) spinlock_profiling.cc:243] Waited 92.2 ms on lock 0x7b5400138b54. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:22:07.986185 (+    57us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:22:07.986232 (+    47us) raft_consensus.cc:1685] finished
0709 13:22:07.986244 (+    12us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:22:07.986389 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":11622,"prepare.queue_time_us":40153,"prepare.run_cpu_time_us":159,"prepare.run_wall_time_us":6259,"raft.queue_time_us":17347,"raft.run_cpu_time_us":363,"raft.run_wall_time_us":17826,"spinlock_wait_cycles":222584448,"thread_start_us":349852,"threads_started":1,"wal-append.queue_time_us":504452}
I20260709 13:22:08.017247 16818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af57996a8be84d398f0f023e8d08ef77" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:08.000077 23252 raft_consensus.cc:3060] T 746aabf856ab401c9ef5e859371e7b67 P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:22:07.989442 23280 leader_election.cc:290] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:08.028867 16191 leader_election.cc:304] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: cb82cc7bd6db46d980c75237a1402860
I20260709 13:22:08.007622 23211 raft_consensus.cc:2749] T 26354361e9b044cea1c2b86c4882fe92 P cb82cc7bd6db46d980c75237a1402860 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:22:08.038036 16221 consensus_meta.cc:287] T d2ab0ac8347641e6955e9bce7d90c1a2 P 1ca2725d290e47d68b1a4b609c9816ac: Time spent flushing consensus metadata: real 0.528s	user 0.012s	sys 0.006s
I20260709 13:22:08.038477 16221 raft_consensus.cc:2468] T d2ab0ac8347641e6955e9bce7d90c1a2 P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 17.
I20260709 13:22:08.000636 23177 consensus_queue.cc:1059] T af57996a8be84d398f0f023e8d08ef77 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.029s
I20260709 13:22:08.000427 23188 consensus_queue.cc:1059] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: INVALID_TERM, Last received: 21.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.029s
W20260709 13:22:08.050201 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 63496.6us
I20260709 13:22:08.001884 23235 consensus_queue.cc:1059] T 18ad7e3af5504571a69d10c00750a01c P 90f3da57796b488c9087a2f523adf045 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: INVALID_TERM, Last received: 16.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.037s
I20260709 13:22:08.054450 16144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d4ab870a05347868956aa8b841b2981" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:22:08.054666 16142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e65676b21b748728ad39668cfe722ec" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:22:08.059396 16142 raft_consensus.cc:2393] T 9e65676b21b748728ad39668cfe722ec P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 19: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
I20260709 13:22:08.055516 16147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434bff3082d141838068f323bdb0d1ed" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:22:08.056452 16145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:22:08.064596 16145 raft_consensus.cc:2468] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 22.
I20260709 13:22:08.057360 16148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57a744f5aaf944678a63469cbf514015" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:22:08.013590 23292 consensus_queue.cc:1059] T 829b4ad8c5a54eababeea1181b960d4b P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: INVALID_TERM, Last received: 16.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.037s
I20260709 13:22:08.013444 23199 consensus_queue.cc:1048] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.033s
I20260709 13:22:08.007926 23269 raft_consensus.cc:515] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac [term 20 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } }
I20260709 13:22:08.060003 16265 leader_election.cc:304] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
I20260709 13:22:08.065761 16148 raft_consensus.cc:2468] T 57a744f5aaf944678a63469cbf514015 P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 16.
I20260709 13:22:08.045421 23289 raft_consensus.cc:3060] T d2ab0ac8347641e6955e9bce7d90c1a2 P cb82cc7bd6db46d980c75237a1402860 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:22:08.082597 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0eab25ce4c64e018a041beeda2867db" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:22:08.083355 16149 raft_consensus.cc:3060] T c0eab25ce4c64e018a041beeda2867db P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:22:08.084560 16146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ad5dbfa99ee4b8d9e28be92b3f36db5" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:22:08.091478 16266 leader_election.cc:304] T 2794af5b99984efebdee564035841cb0 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 
W20260709 13:22:08.073794 23319 log.cc:927] Time spent T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860: Append to log took a long time: real 0.183s	user 0.001s	sys 0.000s
W20260709 13:22:08.093804 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 85050.5us
I20260709 13:22:08.074656 23061 raft_consensus.cc:2804] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [term 23 FOLLOWER]: Leader election won for term 23
W20260709 13:22:08.085529 23287 consensus_meta.cc:287] T ecc1e363230043fe9fef390f0b979f20 P 90f3da57796b488c9087a2f523adf045: Time spent flushing consensus metadata: real 0.522s	user 0.008s	sys 0.012s
W20260709 13:22:08.103011 23303 log.cc:927] Time spent T 26354361e9b044cea1c2b86c4882fe92 P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.461s	user 0.003s	sys 0.000s
I20260709 13:22:08.103235 23308 consensus_queue.cc:1059] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: INVALID_TERM, Last received: 20.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.094s
I20260709 13:22:08.107496 22917 leader_election.cc:290] T 8c11074579b046d99e585c2818e18a2c P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 20 election: Requested vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:08.107887 23292 raft_consensus.cc:3055] T 829b4ad8c5a54eababeea1181b960d4b P cb82cc7bd6db46d980c75237a1402860 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:22:08.114717 16823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 23 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:08.115177 23248 leader_election.cc:290] T 9e65676b21b748728ad39668cfe722ec P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 19 election: Requested vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:08.115428 16821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74ca8140f35c4ae29e8a664f6bdfc829" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:22:08.115655 16819 raft_consensus.cc:1275] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [term 26 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 24 index: 15. Preceding OpId from leader: term: 26 index: 16. (index mismatch)
I20260709 13:22:08.116019 16817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9944d5e4b8c4109af9955c4f08c9095" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 23 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:08.116418 16822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7ca61592e848ea9b0fb81a2ce33cdc" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:08.116708 16825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b7d1ab519044eda2d5e04a07242f25" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
W20260709 13:22:08.119867 16117 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 63614.1us
I20260709 13:22:08.083441 23280 raft_consensus.cc:3055] T af57996a8be84d398f0f023e8d08ef77 P cb82cc7bd6db46d980c75237a1402860 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:22:08.121035 16218 raft_consensus.cc:2393] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 20: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
I20260709 13:22:08.149773 16821 raft_consensus.cc:2368] T 74ca8140f35c4ae29e8a664f6bdfc829 P cb82cc7bd6db46d980c75237a1402860 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac for earlier term 20. Current term is 21.
W20260709 13:22:08.121871 16115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 73578.3us
W20260709 13:22:08.125968 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 117767us
W20260709 13:22:08.128234 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 51807.4us
I20260709 13:22:08.102912 23321 raft_consensus.cc:2749] T e5ad4eb281564936a59f41e387a47156 P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:22:08.121532 23291 raft_consensus.cc:3055] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:22:08.128942 16823 raft_consensus.cc:2468] T 19baa7bc367844c288bca8f81439c413 P cb82cc7bd6db46d980c75237a1402860 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 22.
I20260709 13:22:08.159225 16116 leader_election.cc:304] T 7c2095a5f4ad49408dbc7f69c606120d P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:22:08.095629 23256 leader_election.cc:290] T c0eab25ce4c64e018a041beeda2867db P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 20 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:08.165997 16146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5d59407eb545b895d8d922e0dd69fd" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
W20260709 13:22:08.166558 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 117413us
I20260709 13:22:08.128619 23292 raft_consensus.cc:740] T 829b4ad8c5a54eababeea1181b960d4b P cb82cc7bd6db46d980c75237a1402860 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:22:08.151424 23280 raft_consensus.cc:740] T af57996a8be84d398f0f023e8d08ef77 P cb82cc7bd6db46d980c75237a1402860 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:22:08.154493 23329 raft_consensus.cc:2804] T 829b4ad8c5a54eababeea1181b960d4b P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:22:08.196887 23329 raft_consensus.cc:493] T 829b4ad8c5a54eababeea1181b960d4b P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:08.197306 23329 raft_consensus.cc:3060] T 829b4ad8c5a54eababeea1181b960d4b P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:22:08.198196 23294 consensus_meta.cc:287] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 1ca2725d290e47d68b1a4b609c9816ac: Time spent flushing consensus metadata: real 0.556s	user 0.013s	sys 0.002s
I20260709 13:22:08.169628 23291 raft_consensus.cc:740] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:22:08.198815 23294 raft_consensus.cc:515] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:08.199860 23291 consensus_queue.cc:260] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 22.15, Last appended by leader: 15, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:08.201122 23291 raft_consensus.cc:3060] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 [term 22 FOLLOWER]: Advancing to term 23
W20260709 13:22:08.202096 23023 consensus_meta.cc:287] T 2794af5b99984efebdee564035841cb0 P 647a343a542d4a139fe106c19095129b: Time spent flushing consensus metadata: real 0.643s	user 0.021s	sys 0.000s
W20260709 13:22:08.203487 23325 log.cc:927] Time spent T c9944d5e4b8c4109af9955c4f08c9095 P cb82cc7bd6db46d980c75237a1402860: Append to log took a long time: real 0.056s	user 0.000s	sys 0.001s
I20260709 13:22:08.203685 16821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a9de347c711477e87c582d003f36925" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
W20260709 13:22:08.204566 16263 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:22:08.204839 16821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5d59407eb545b895d8d922e0dd69fd" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:08.205389 16821 raft_consensus.cc:2468] T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 21.
I20260709 13:22:08.206130 16821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66f75124215f4ca6bef831e174a792d8" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:08.206632 16821 raft_consensus.cc:2468] T 66f75124215f4ca6bef831e174a792d8 P cb82cc7bd6db46d980c75237a1402860 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 17.
W20260709 13:22:08.210335 16790 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:22:08.211449 16825 raft_consensus.cc:1240] T 2794af5b99984efebdee564035841cb0 P cb82cc7bd6db46d980c75237a1402860 [term 17 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 16. Current term is 17. Ops: [16.14-16.14]
I20260709 13:22:08.153172 23211 consensus_queue.cc:1059] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: INVALID_TERM, Last received: 18.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.062s
I20260709 13:22:08.215864 23199 raft_consensus.cc:3055] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:22:08.216377 23199 raft_consensus.cc:740] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:22:08.215535 23294 leader_election.cc:290] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 19 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:08.166293 22873 raft_consensus.cc:2804] T 2794af5b99984efebdee564035841cb0 P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:22:08.220140 22873 raft_consensus.cc:697] T 2794af5b99984efebdee564035841cb0 P 90f3da57796b488c9087a2f523adf045 [term 17 LEADER]: Becoming Leader. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:22:08.220996 22873 consensus_queue.cc:237] T 2794af5b99984efebdee564035841cb0 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.14, Last appended by leader: 14, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:08.213505 16825 raft_consensus.cc:1240] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [term 26 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 25. Current term is 26. Ops: []
I20260709 13:22:08.176261 23269 leader_election.cc:290] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 20 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:08.152283 23061 raft_consensus.cc:697] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [term 23 LEADER]: Becoming Leader. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:22:08.176651 23161 raft_consensus.cc:2749] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:22:08.174062 16817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c11074579b046d99e585c2818e18a2c" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:22:08.228919 16817 raft_consensus.cc:3060] T 8c11074579b046d99e585c2818e18a2c P cb82cc7bd6db46d980c75237a1402860 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:22:08.174603 16822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806d7690c2b644959152074bb2cab0a7" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:08.158254 23173 raft_consensus.cc:2749] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:22:08.173111 16789 leader_election.cc:304] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:22:08.173797 23252 raft_consensus.cc:515] T 746aabf856ab401c9ef5e859371e7b67 P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:22:08.231971 16789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 58972.8us
I20260709 13:22:08.229771 16822 raft_consensus.cc:2468] T 806d7690c2b644959152074bb2cab0a7 P cb82cc7bd6db46d980c75237a1402860 [term 18 LEARNER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 18.
I20260709 13:22:08.175617 16823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3884f864a96d428785895e5463e5728f" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 15 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
W20260709 13:22:08.170344 16266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 78882.1us
I20260709 13:22:08.178871 16149 raft_consensus.cc:2468] T c0eab25ce4c64e018a041beeda2867db P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 20.
I20260709 13:22:08.182871 23248 consensus_queue.cc:1048] T 8d4ab870a05347868956aa8b841b2981 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.057s
W20260709 13:22:08.241190 23086 consensus_meta.cc:287] T 9c61a39fd4f84424b129c1a6f9ba404d P 647a343a542d4a139fe106c19095129b: Time spent flushing consensus metadata: real 0.734s	user 0.016s	sys 0.003s
I20260709 13:22:08.241643 23086 raft_consensus.cc:515] T 9c61a39fd4f84424b129c1a6f9ba404d P 647a343a542d4a139fe106c19095129b [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:08.128911 23287 raft_consensus.cc:515] T ecc1e363230043fe9fef390f0b979f20 P 90f3da57796b488c9087a2f523adf045 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:08.244764 16297 raft_consensus.cc:1240] T ecc1e363230043fe9fef390f0b979f20 P 90f3da57796b488c9087a2f523adf045 [term 22 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 21. Current term is 22. Ops: [21.13-21.13]
I20260709 13:22:08.244864 16151 raft_consensus.cc:1240] T 746aabf856ab401c9ef5e859371e7b67 P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 16. Current term is 17. Ops: [16.14-16.14]
I20260709 13:22:08.246942 16294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7ca61592e848ea9b0fb81a2ce33cdc" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:08.247665 23287 leader_election.cc:290] T ecc1e363230043fe9fef390f0b979f20 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 22 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:08.179641 16146 raft_consensus.cc:2468] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 21.
I20260709 13:22:08.187371 23216 raft_consensus.cc:3055] T 18ad7e3af5504571a69d10c00750a01c P 90f3da57796b488c9087a2f523adf045 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:22:08.248987 23216 raft_consensus.cc:740] T 18ad7e3af5504571a69d10c00750a01c P 90f3da57796b488c9087a2f523adf045 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:22:08.248594 23292 consensus_queue.cc:260] T 829b4ad8c5a54eababeea1181b960d4b P cb82cc7bd6db46d980c75237a1402860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:08.249738 23292 raft_consensus.cc:3060] T 829b4ad8c5a54eababeea1181b960d4b P cb82cc7bd6db46d980c75237a1402860 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:22:08.249902 23216 consensus_queue.cc:260] T 18ad7e3af5504571a69d10c00750a01c P 90f3da57796b488c9087a2f523adf045 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } }
I20260709 13:22:08.188273 23326 raft_consensus.cc:3055] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:22:08.251412 23326 raft_consensus.cc:740] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:22:08.251480 23280 consensus_queue.cc:260] T af57996a8be84d398f0f023e8d08ef77 P cb82cc7bd6db46d980c75237a1402860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.13, Last appended by leader: 13, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:08.252341 23326 consensus_queue.cc:260] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.12, Last appended by leader: 12, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:08.253605 23326 raft_consensus.cc:3060] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:22:08.191427 23328 consensus_queue.cc:1048] T ecc1e363230043fe9fef390f0b979f20 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:22:08.255609 16265 leader_election.cc:304] T 8a878999c0024d2abe517484180290fc P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac
I20260709 13:22:08.192739 16116 leader_election.cc:304] T 08c74490c40243a88e62d1760b5cd814 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 90f3da57796b488c9087a2f523adf045
W20260709 13:22:08.256569 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 63912.8us
I20260709 13:22:08.256801 16265 leader_election.cc:304] T c4f65cdf00b748bb963ee6e81ef43b79 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:22:08.257651 23332 raft_consensus.cc:2804] T 08c74490c40243a88e62d1760b5cd814 P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:22:08.258087 23332 raft_consensus.cc:493] T 08c74490c40243a88e62d1760b5cd814 P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Starting leader election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:22:08.193390 23322 raft_consensus.cc:2749] T 7c2095a5f4ad49408dbc7f69c606120d P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:22:08.258430 23332 raft_consensus.cc:3060] T 08c74490c40243a88e62d1760b5cd814 P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:22:08.193818 23311 leader_election.cc:290] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 18 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:08.188827 16144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:22:08.258458 16116 leader_election.cc:304] T fffab26427e24d398e2695aad40ca134 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:22:08.217284 23199 consensus_queue.cc:260] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 18.15, Last appended by leader: 15, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:08.156205 23235 raft_consensus.cc:2749] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:22:08.176158 16191 leader_election.cc:304] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 90f3da57796b488c9087a2f523adf045
I20260709 13:22:08.269977 16295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68a6838d82ef4cac83507f906139d90f" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:22:08.234146 16821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e65676b21b748728ad39668cfe722ec" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:22:08.228143 23061 consensus_queue.cc:237] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 21.14, Last appended by leader: 14, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:08.232537 23166 raft_consensus.cc:2749] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:22:08.277655 16219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4dc6b0541ca431ebc757beac00521f1" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:08.278239 16821 raft_consensus.cc:2393] T 9e65676b21b748728ad39668cfe722ec P cb82cc7bd6db46d980c75237a1402860 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 19: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
I20260709 13:22:08.275784 16222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5d59407eb545b895d8d922e0dd69fd" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:08.196074 16218 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57604 (request call id 3369) took 1134 ms. Trace:
I20260709 13:22:08.263019 16817 raft_consensus.cc:2468] T 8c11074579b046d99e585c2818e18a2c P cb82cc7bd6db46d980c75237a1402860 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 20.
I20260709 13:22:08.182184 16115 leader_election.cc:400] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 22 election: Vote denied by peer 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389) with higher term. Message: Invalid argument: T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b for earlier term 22. Current term is 23.
I20260709 13:22:08.279508 16115 leader_election.cc:403] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 22 election: Cancelling election due to peer responding with higher term
W20260709 13:22:08.280138 16115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 122700us
I20260709 13:22:08.264353 16265 leader_election.cc:304] T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:22:08.265005 16790 leader_election.cc:304] T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:22:08.190888 22908 raft_consensus.cc:515] T 66f75124215f4ca6bef831e174a792d8 P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:22:08.210017 23286 consensus_meta.cc:287] T 8a878999c0024d2abe517484180290fc P 647a343a542d4a139fe106c19095129b: Time spent flushing consensus metadata: real 0.646s	user 0.013s	sys 0.005s
I20260709 13:22:08.252769 23280 raft_consensus.cc:3060] T af57996a8be84d398f0f023e8d08ef77 P cb82cc7bd6db46d980c75237a1402860 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:22:08.286576 23286 raft_consensus.cc:515] T 8a878999c0024d2abe517484180290fc P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:08.295895 16217 raft_consensus.cc:1240] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [term 26 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 25. Current term is 26. Ops: []
I20260709 13:22:08.297025 22908 leader_election.cc:290] T 66f75124215f4ca6bef831e174a792d8 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 17 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
W20260709 13:22:08.247010 23204 consensus_meta.cc:287] T 57a744f5aaf944678a63469cbf514015 P cb82cc7bd6db46d980c75237a1402860: Time spent flushing consensus metadata: real 0.513s	user 0.010s	sys 0.009s
I20260709 13:22:08.275592 16221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18ad7e3af5504571a69d10c00750a01c" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:22:08.281152 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9944d5e4b8c4109af9955c4f08c9095" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 23 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
W20260709 13:22:08.304492 16790 leader_election.cc:336] T 19baa7bc367844c288bca8f81439c413 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943): Timed out: RequestConsensusVote RPC to 127.14.177.130:35943 timed out after 1.537s (SENT)
I20260709 13:22:08.278906 16222 raft_consensus.cc:2468] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 21.
I20260709 13:22:08.279332 16218 rpcz_store.cc:276] 0709 13:22:07.061555 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:22:07.142682 (+ 81127us) service_pool.cc:224] Handling call
0709 13:22:08.195853 (+1053171us) spinlock_profiling.cc:243] Waited 952 ms on lock 0x7b54000159d4. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:22:08.195988 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2189230464}
I20260709 13:22:08.281749 16291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74ca8140f35c4ae29e8a664f6bdfc829" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:22:08.280838 16292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1df93779209748b9860f6d893dd3de53" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:22:08.284119 16294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 23 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:08.267096 23216 raft_consensus.cc:3060] T 18ad7e3af5504571a69d10c00750a01c P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:22:08.300819 16296 raft_consensus.cc:1275] T 66ae176961b74813a58bc15c9c4c6f2a P 90f3da57796b488c9087a2f523adf045 [term 26 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 25 index: 16. Preceding OpId from leader: term: 26 index: 16. (term mismatch)
I20260709 13:22:08.311125 16296 pending_rounds.cc:85] T 66ae176961b74813a58bc15c9c4c6f2a P 90f3da57796b488c9087a2f523adf045: Aborting all ops after (but not including) 15
I20260709 13:22:08.311441 16296 pending_rounds.cc:107] T 66ae176961b74813a58bc15c9c4c6f2a P 90f3da57796b488c9087a2f523adf045: Aborting uncommitted NO_OP operation due to leader change: 25.16
I20260709 13:22:08.311661 16296 raft_consensus.cc:2889] T 66ae176961b74813a58bc15c9c4c6f2a P 90f3da57796b488c9087a2f523adf045 [term 26 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:22:08.312278 16294 raft_consensus.cc:2410] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac for term 23 because replica has last-logged OpId of term: 21 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 11.
I20260709 13:22:08.309252 16222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8158bbd17b994d2eb96f560bd0f1b88d" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:22:08.242728 23329 raft_consensus.cc:515] T 829b4ad8c5a54eababeea1181b960d4b P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:08.310552 16291 raft_consensus.cc:2368] T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac for earlier term 20. Current term is 21.
I20260709 13:22:08.276414 23210 raft_consensus.cc:2804] T 8a878999c0024d2abe517484180290fc P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:22:08.318223 16217 raft_consensus.cc:1240] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 18. Current term is 19. Ops: []
I20260709 13:22:08.280093 23287 raft_consensus.cc:3060] T c4f65cdf00b748bb963ee6e81ef43b79 P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:22:08.280588 23322 raft_consensus.cc:2749] T fffab26427e24d398e2695aad40ca134 P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:22:08.325094 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 149032us
I20260709 13:22:08.298053 23211 raft_consensus.cc:2804] T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:22:08.270934 23233 consensus_queue.cc:1059] T 3884f864a96d428785895e5463e5728f P 90f3da57796b488c9087a2f523adf045 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: INVALID_TERM, Last received: 14.14, Next index: 15, Last known committed idx: 12, Time since last communication: 0.044s
I20260709 13:22:08.330555 16150 raft_consensus.cc:1240] T 829b4ad8c5a54eababeea1181b960d4b P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 16. Current term is 18. Ops: []
I20260709 13:22:08.327313 16191 leader_election.cc:304] T c2f5f89186204242bf377f7c42b2e2da P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:22:08.299949 23173 raft_consensus.cc:2804] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:22:08.347448 16294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c11074579b046d99e585c2818e18a2c" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:22:08.360103 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0eab25ce4c64e018a041beeda2867db" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:22:08.353271 16192 leader_election.cc:400] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 20 election: Vote denied by peer cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923) with higher term. Message: Invalid argument: T 74ca8140f35c4ae29e8a664f6bdfc829 P cb82cc7bd6db46d980c75237a1402860 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac for earlier term 20. Current term is 21.
I20260709 13:22:08.361204 16192 leader_election.cc:403] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 20 election: Cancelling election due to peer responding with higher term
I20260709 13:22:08.354909 16820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66f75124215f4ca6bef831e174a792d8" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:22:08.367772 16117 leader_election.cc:304] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860
I20260709 13:22:08.285277 23328 consensus_queue.cc:1048] T ecc1e363230043fe9fef390f0b979f20 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.041s
I20260709 13:22:08.369072 16117 leader_election.cc:304] T b87147731a68400e8cce9b879daa7c5d P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860
I20260709 13:22:08.369688 23328 raft_consensus.cc:2749] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:22:08.369865 16294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8158bbd17b994d2eb96f560bd0f1b88d" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:22:08.370435 23328 raft_consensus.cc:2749] T b87147731a68400e8cce9b879daa7c5d P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:22:08.372180 16294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ad5dbfa99ee4b8d9e28be92b3f36db5" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:22:08.355489 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f5f89186204242bf377f7c42b2e2da" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:08.301819 23321 raft_consensus.cc:2749] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b [term 23 FOLLOWER]: Leader election lost for term 22. Reason: Vote denied by peer 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389) with higher term. Message: Invalid argument: T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b for earlier term 22. Current term is 23.
I20260709 13:22:08.356631 16222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746aabf856ab401c9ef5e859371e7b67" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:22:08.375666 16222 raft_consensus.cc:3060] T 746aabf856ab401c9ef5e859371e7b67 P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:22:08.376451 16117 leader_election.cc:304] T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860
I20260709 13:22:08.377655 16117 leader_election.cc:304] T af57996a8be84d398f0f023e8d08ef77 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860
I20260709 13:22:08.378368 23328 raft_consensus.cc:2749] T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:22:08.357123 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c61a39fd4f84424b129c1a6f9ba404d" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:22:08.357227 16221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a878999c0024d2abe517484180290fc" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:22:08.379602 16221 raft_consensus.cc:3060] T 8a878999c0024d2abe517484180290fc P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:22:08.379756 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 75300.1us
I20260709 13:22:08.381932 23233 raft_consensus.cc:3055] T 3884f864a96d428785895e5463e5728f P 90f3da57796b488c9087a2f523adf045 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:22:08.382419 23233 raft_consensus.cc:740] T 3884f864a96d428785895e5463e5728f P 90f3da57796b488c9087a2f523adf045 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:22:08.383154 23204 consensus_queue.cc:1059] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: INVALID_TERM, Last received: 15.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:22:08.383401 23233 consensus_queue.cc:260] T 3884f864a96d428785895e5463e5728f P 90f3da57796b488c9087a2f523adf045 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 14.14, Last appended by leader: 14, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:08.384227 23204 raft_consensus.cc:3055] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:22:08.384414 23286 leader_election.cc:290] T 8a878999c0024d2abe517484180290fc P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 18 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:08.384884 23233 raft_consensus.cc:3060] T 3884f864a96d428785895e5463e5728f P 90f3da57796b488c9087a2f523adf045 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:22:08.357916 16116 leader_election.cc:304] T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:22:08.360960 16293 raft_consensus.cc:2393] T c0eab25ce4c64e018a041beeda2867db P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 20: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
I20260709 13:22:08.368325 16296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806d7690c2b644959152074bb2cab0a7" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:22:08.386360 16296 raft_consensus.cc:3060] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:22:08.379014 23328 raft_consensus.cc:2749] T af57996a8be84d398f0f023e8d08ef77 P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:22:08.318342 23210 raft_consensus.cc:493] T 8a878999c0024d2abe517484180290fc P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:22:08.387657 23210 raft_consensus.cc:3060] T 8a878999c0024d2abe517484180290fc P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:22:08.316844 23023 raft_consensus.cc:493] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:22:08.390547 23023 raft_consensus.cc:515] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:22:08.391151 16791 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:22:08.381942 16218 raft_consensus.cc:1275] T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 13:22:08.392848 16291 raft_consensus.cc:1275] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [term 23 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 21 index: 15. Preceding OpId from leader: term: 23 index: 16. (index mismatch)
I20260709 13:22:08.355248 23086 leader_election.cc:290] T 9c61a39fd4f84424b129c1a6f9ba404d P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 21 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
W20260709 13:22:08.408034 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 50225.7us
I20260709 13:22:08.397603 16117 leader_election.cc:304] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:22:08.409269 16117 leader_election.cc:304] T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac
I20260709 13:22:08.398852 16825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26354361e9b044cea1c2b86c4882fe92" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:08.405943 16190 leader_election.cc:304] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860
I20260709 13:22:08.408198 16816 raft_consensus.cc:1240] T 8c11074579b046d99e585c2818e18a2c P cb82cc7bd6db46d980c75237a1402860 [term 20 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 19. Current term is 20. Ops: []
I20260709 13:22:08.396242 16192 leader_election.cc:304] T 8c11074579b046d99e585c2818e18a2c P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:22:08.396004 16221 raft_consensus.cc:2468] T 8a878999c0024d2abe517484180290fc P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 18.
I20260709 13:22:08.417147 16116 leader_election.cc:304] T 18ad7e3af5504571a69d10c00750a01c P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:22:08.417739 16117 leader_election.cc:304] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:22:08.418764 16294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:08.419986 16294 raft_consensus.cc:2393] T ecc1e363230043fe9fef390f0b979f20 P 90f3da57796b488c9087a2f523adf045 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 in current term 22: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
I20260709 13:22:08.420821 16820 raft_consensus.cc:2468] T 66f75124215f4ca6bef831e174a792d8 P cb82cc7bd6db46d980c75237a1402860 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 17.
W20260709 13:22:08.424302 16115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 122672us
I20260709 13:22:08.435372 16817 raft_consensus.cc:1275] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 [term 22 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 21 index: 16. Preceding OpId from leader: term: 22 index: 17. (index mismatch)
W20260709 13:22:08.437808 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 173514us
I20260709 13:22:08.439105 16222 raft_consensus.cc:2468] T 746aabf856ab401c9ef5e859371e7b67 P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 17.
I20260709 13:22:08.447105 16822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f82224f4564fea93572e8206bad1fe" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:08.446908 16821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57a744f5aaf944678a63469cbf514015" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:08.449108 16821 raft_consensus.cc:2468] T 57a744f5aaf944678a63469cbf514015 P cb82cc7bd6db46d980c75237a1402860 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 16.
I20260709 13:22:08.329285 23211 raft_consensus.cc:493] T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:08.444944 16266 leader_election.cc:304] T 434bff3082d141838068f323bdb0d1ed P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:22:08.338296 23199 raft_consensus.cc:3060] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:22:08.456494 16266 leader_election.cc:304] T 9e65676b21b748728ad39668cfe722ec P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:22:08.339581 23252 leader_election.cc:290] T 746aabf856ab401c9ef5e859371e7b67 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 17 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:08.448673 16816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:22:08.464732 16816 raft_consensus.cc:3060] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:22:08.459056 16266 leader_election.cc:304] T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:22:08.464069 16296 raft_consensus.cc:2468] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 20.
I20260709 13:22:08.468022 16150 raft_consensus.cc:1275] T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 13:22:08.385269 23204 raft_consensus.cc:740] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
W20260709 13:22:08.469194 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 141989us
I20260709 13:22:08.407737 22908 raft_consensus.cc:3060] T c2f5f89186204242bf377f7c42b2e2da P 1ca2725d290e47d68b1a4b609c9816ac [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:22:08.469977 16192 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 73786.8us
W20260709 13:22:08.473707 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 67883.3us
I20260709 13:22:08.435497 23321 raft_consensus.cc:2804] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:22:08.481295 16820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746aabf856ab401c9ef5e859371e7b67" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:22:08.447701 23328 raft_consensus.cc:2749] T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:22:08.466939 23211 raft_consensus.cc:3060] T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:22:08.407754 23269 raft_consensus.cc:2749] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Leader election lost for term 20. Reason: Vote denied by peer cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923) with higher term. Message: Invalid argument: T 74ca8140f35c4ae29e8a664f6bdfc829 P cb82cc7bd6db46d980c75237a1402860 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac for earlier term 20. Current term is 21.
I20260709 13:22:08.470168 23173 raft_consensus.cc:697] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac [term 21 LEADER]: Becoming Leader. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:22:08.447926 23286 raft_consensus.cc:2804] T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:22:08.480275 16191 leader_election.cc:304] T 746aabf856ab401c9ef5e859371e7b67 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: cb82cc7bd6db46d980c75237a1402860
I20260709 13:22:08.362561 23332 raft_consensus.cc:515] T 08c74490c40243a88e62d1760b5cd814 P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:08.476639 16192 leader_election.cc:304] T 66f75124215f4ca6bef831e174a792d8 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:22:08.476079 22677 raft_consensus.cc:2804] T 8c11074579b046d99e585c2818e18a2c P 1ca2725d290e47d68b1a4b609c9816ac [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:22:08.477681 16190 leader_election.cc:400] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 20 election: Vote denied by peer 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389) with higher term. Message: Invalid argument: T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac for earlier term 20. Current term is 21.
I20260709 13:22:08.520911 16190 leader_election.cc:304] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860
I20260709 13:22:08.481350 23321 raft_consensus.cc:493] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:08.472877 23316 raft_consensus.cc:2804] T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:22:08.415788 23257 consensus_queue.cc:1048] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.066s
I20260709 13:22:08.472659 23235 raft_consensus.cc:2804] T 9e65676b21b748728ad39668cfe722ec P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:22:08.435546 22645 raft_consensus.cc:2749] T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:22:08.498716 16150 raft_consensus.cc:1275] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b [term 23 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 21 index: 15. Preceding OpId from leader: term: 23 index: 16. (index mismatch)
I20260709 13:22:08.447762 22610 consensus_queue.cc:1059] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: INVALID_TERM, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.039s
I20260709 13:22:08.511703 16816 raft_consensus.cc:2468] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 22.
I20260709 13:22:08.517987 23286 raft_consensus.cc:493] T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:22:08.556285 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 76104.2us
I20260709 13:22:08.471082 23086 raft_consensus.cc:2804] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:22:08.482918 23256 raft_consensus.cc:2749] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 1ca2725d290e47d68b1a4b609c9816ac [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:22:08.487214 23252 raft_consensus.cc:2749] T 18ad7e3af5504571a69d10c00750a01c P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:22:08.456766 23314 consensus_queue.cc:1048] T ada5a004ba7c46819051e3ccec71ffdb P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.045s
W20260709 13:22:08.525938 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 87073.3us
W20260709 13:22:08.563575 16117 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 145884us
W20260709 13:22:08.567037 16263 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:22:08.512871 22873 consensus_queue.cc:1059] T 8c11074579b046d99e585c2818e18a2c P 90f3da57796b488c9087a2f523adf045 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: INVALID_TERM, Last received: 19.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.049s
I20260709 13:22:08.441550 23087 raft_consensus.cc:515] T 1df93779209748b9860f6d893dd3de53 P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:08.560714 23286 raft_consensus.cc:3060] T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:22:08.557322 16191 leader_election.cc:304] T 08c74490c40243a88e62d1760b5cd814 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 90f3da57796b488c9087a2f523adf045
I20260709 13:22:08.525673 23311 raft_consensus.cc:2804] T 66f75124215f4ca6bef831e174a792d8 P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:22:08.525596 22946 raft_consensus.cc:2749] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:22:08.500685 23177 consensus_queue.cc:1048] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.056s
W20260709 13:22:08.562990 16115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 115904us
I20260709 13:22:08.560115 23294 consensus_queue.cc:1048] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.024s
W20260709 13:22:08.553053 16788 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:22:08.527412 23321 raft_consensus.cc:3060] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:22:08.450990 23307 raft_consensus.cc:493] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:08.549934 23269 raft_consensus.cc:2764] T 746aabf856ab401c9ef5e859371e7b67 P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Leader pre-election decision vote started in defunct term 16: won
I20260709 13:22:08.571885 16117 leader_election.cc:304] T 3884f864a96d428785895e5463e5728f P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860
I20260709 13:22:08.550249 23316 raft_consensus.cc:697] T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045 [term 21 LEADER]: Becoming Leader. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:22:08.554106 16817 raft_consensus.cc:1240] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860 [term 22 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 21. Current term is 22. Ops: [21.13-21.13]
W20260709 13:22:08.609412 16266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 138592us
I20260709 13:22:08.595410 23306 raft_consensus.cc:2804] T 434bff3082d141838068f323bdb0d1ed P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Leader pre-election won for term 19
W20260709 13:22:08.557307 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 107584us
I20260709 13:22:08.563390 23086 raft_consensus.cc:493] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:08.617655 16116 leader_election.cc:304] T 8158bbd17b994d2eb96f560bd0f1b88d P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:22:08.555920 22677 raft_consensus.cc:697] T 8c11074579b046d99e585c2818e18a2c P 1ca2725d290e47d68b1a4b609c9816ac [term 20 LEADER]: Becoming Leader. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:22:08.581154 16294 raft_consensus.cc:1275] T 31969392ccdb43b491911a319e8bc7e1 P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 13:22:08.523736 23322 consensus_queue.cc:1059] T ecc1e363230043fe9fef390f0b979f20 P 647a343a542d4a139fe106c19095129b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true }, Status: INVALID_TERM, Last received: 19.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.040s
I20260709 13:22:08.587860 16265 leader_election.cc:304] T d16dfad0c3b540e5b4b86492ddf87208 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:22:08.579309 23248 consensus_queue.cc:1059] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.019s
I20260709 13:22:08.563084 23280 consensus_queue.cc:1048] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.022s
I20260709 13:22:08.568578 16789 leader_election.cc:304] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:22:08.595202 23061 raft_consensus.cc:2804] T 08c74490c40243a88e62d1760b5cd814 P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Leader pre-election won for term 19
W20260709 13:22:08.600910 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 43716.8us
I20260709 13:22:08.602790 16296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f5f89186204242bf377f7c42b2e2da" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
W20260709 13:22:08.605279 16790 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:22:08.564277 23204 consensus_queue.cc:260] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.14, Last appended by leader: 14, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:08.602312 23023 leader_election.cc:290] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
W20260709 13:22:08.610227 23289 consensus_meta.cc:287] T d2ab0ac8347641e6955e9bce7d90c1a2 P cb82cc7bd6db46d980c75237a1402860: Time spent flushing consensus metadata: real 0.521s	user 0.010s	sys 0.000s
I20260709 13:22:08.637904 23061 raft_consensus.cc:493] T 08c74490c40243a88e62d1760b5cd814 P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Starting leader election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:22:08.642446 16824 raft_consensus.cc:3060] T d2ab0ac8347641e6955e9bce7d90c1a2 P cb82cc7bd6db46d980c75237a1402860 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:22:08.642717 23061 raft_consensus.cc:3060] T 08c74490c40243a88e62d1760b5cd814 P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:22:08.643076 23280 raft_consensus.cc:2749] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:22:08.644138 23347 raft_consensus.cc:2749] T 8158bbd17b994d2eb96f560bd0f1b88d P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:22:08.610508 23329 leader_election.cc:290] T 829b4ad8c5a54eababeea1181b960d4b P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 18 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:08.630604 16291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e65676b21b748728ad39668cfe722ec" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:22:08.595513 23311 raft_consensus.cc:697] T 66f75124215f4ca6bef831e174a792d8 P 1ca2725d290e47d68b1a4b609c9816ac [term 17 LEADER]: Becoming Leader. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:22:08.635133 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a878999c0024d2abe517484180290fc" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:22:08.638571 16296 raft_consensus.cc:2468] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 21.
I20260709 13:22:08.613785 23306 raft_consensus.cc:493] T 434bff3082d141838068f323bdb0d1ed P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:22:08.638424 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c61a39fd4f84424b129c1a6f9ba404d" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:22:08.649154 16191 leader_election.cc:304] T fffab26427e24d398e2695aad40ca134 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 90f3da57796b488c9087a2f523adf045
W20260709 13:22:08.651088 16117 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 79296.1us
I20260709 13:22:08.654946 16824 raft_consensus.cc:2468] T d2ab0ac8347641e6955e9bce7d90c1a2 P cb82cc7bd6db46d980c75237a1402860 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 18.
W20260709 13:22:08.655542 16790 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:22:08.621511 23086 raft_consensus.cc:3060] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:22:08.603448 23307 raft_consensus.cc:515] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:22:08.667279 16789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 98775.6us
I20260709 13:22:08.669516 16191 leader_election.cc:304] T b6f07842d3aa47858df5665874c86f3a P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 90f3da57796b488c9087a2f523adf045
I20260709 13:22:08.606617 23173 consensus_queue.cc:237] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.13, Last appended by leader: 13, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:08.673851 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87ac0d1f7e8a419f9bcbc04aee0e067e" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:08.699822 16116 leader_election.cc:304] T 8a878999c0024d2abe517484180290fc P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
I20260709 13:22:08.700933 16116 leader_election.cc:304] T 746aabf856ab401c9ef5e859371e7b67 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
I20260709 13:22:08.657794 23306 raft_consensus.cc:3060] T 434bff3082d141838068f323bdb0d1ed P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:22:08.692749 23341 log.cc:927] Time spent T 7c2095a5f4ad49408dbc7f69c606120d P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.076s	user 0.000s	sys 0.000s
W20260709 13:22:08.693606 23345 log.cc:927] Time spent T 434bff3082d141838068f323bdb0d1ed P cb82cc7bd6db46d980c75237a1402860: Append to log took a long time: real 0.062s	user 0.000s	sys 0.000s
I20260709 13:22:08.694262 23311 consensus_queue.cc:237] T 66f75124215f4ca6bef831e174a792d8 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:22:08.695559 23338 log.cc:927] Time spent T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 0.107s	user 0.001s	sys 0.000s
W20260709 13:22:08.696271 23342 log.cc:927] Time spent T 2794af5b99984efebdee564035841cb0 P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.084s	user 0.000s	sys 0.000s
I20260709 13:22:08.698369 23321 raft_consensus.cc:515] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:22:08.699479 23333 log.cc:927] Time spent T 26354361e9b044cea1c2b86c4882fe92 P cb82cc7bd6db46d980c75237a1402860: Append to log took a long time: real 0.180s	user 0.001s	sys 0.000s
I20260709 13:22:08.658187 23328 raft_consensus.cc:2749] T 3884f864a96d428785895e5463e5728f P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:22:08.660840 22946 raft_consensus.cc:2804] T fffab26427e24d398e2695aad40ca134 P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:22:08.711966 23356 consensus_queue.cc:1048] T 8d4ab870a05347868956aa8b841b2981 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
W20260709 13:22:08.712600 16117 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 57648.7us
I20260709 13:22:08.652120 23292 consensus_queue.cc:1048] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.053s
I20260709 13:22:08.723634 23256 raft_consensus.cc:2804] T b6f07842d3aa47858df5665874c86f3a P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:22:08.729811 23256 raft_consensus.cc:493] T b6f07842d3aa47858df5665874c86f3a P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Starting leader election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:22:08.730077 23287 raft_consensus.cc:2749] T c4f65cdf00b748bb963ee6e81ef43b79 P 90f3da57796b488c9087a2f523adf045 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:22:08.730373 23256 raft_consensus.cc:3060] T b6f07842d3aa47858df5665874c86f3a P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:22:08.732939 23321 leader_election.cc:290] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 22 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:08.718561 23280 consensus_queue.cc:1048] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.010s
I20260709 13:22:08.718732 23355 raft_consensus.cc:2804] T 8a878999c0024d2abe517484180290fc P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:22:08.737388 22946 raft_consensus.cc:493] T fffab26427e24d398e2695aad40ca134 P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Starting leader election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:22:08.737962 22946 raft_consensus.cc:3060] T fffab26427e24d398e2695aad40ca134 P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:22:08.720729 22677 consensus_queue.cc:237] T 8c11074579b046d99e585c2818e18a2c P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:08.741357 23256 raft_consensus.cc:515] T b6f07842d3aa47858df5665874c86f3a P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:08.745283 23256 leader_election.cc:290] T b6f07842d3aa47858df5665874c86f3a P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 19 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:08.748345 22610 raft_consensus.cc:2804] T 746aabf856ab401c9ef5e859371e7b67 P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:22:08.661084 23269 consensus_queue.cc:1048] T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.059s
I20260709 13:22:08.673321 23340 consensus_queue.cc:1059] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: INVALID_TERM, Last received: 18.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.042s
I20260709 13:22:08.753856 16817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "829b4ad8c5a54eababeea1181b960d4b" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:22:08.753902 16818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5d59407eb545b895d8d922e0dd69fd" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
W20260709 13:22:08.752167 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 82783.2us
I20260709 13:22:08.754750 16817 raft_consensus.cc:3060] T 829b4ad8c5a54eababeea1181b960d4b P cb82cc7bd6db46d980c75237a1402860 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:22:08.755362 16191 leader_election.cc:304] T 829b4ad8c5a54eababeea1181b960d4b P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
W20260709 13:22:08.755875 16788 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:22:08.757682 16115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 97942.7us
W20260709 13:22:08.756389 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 55550.8us
I20260709 13:22:08.766434 16224 raft_consensus.cc:1240] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [term 26 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 25. Current term is 26. Ops: [25.16-25.16]
I20260709 13:22:08.766427 16221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5d59407eb545b895d8d922e0dd69fd" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:22:08.767563 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c74490c40243a88e62d1760b5cd814" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:22:08.767767 16221 raft_consensus.cc:3060] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:22:08.769024 16219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1df93779209748b9860f6d893dd3de53" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:22:08.769917 16219 raft_consensus.cc:2393] T 1df93779209748b9860f6d893dd3de53 P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 18: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
I20260709 13:22:08.771016 16218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87ac0d1f7e8a419f9bcbc04aee0e067e" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
W20260709 13:22:08.771955 23344 log.cc:927] Time spent T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 0.123s	user 0.001s	sys 0.000s
W20260709 13:22:08.772455 16264 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:22:08.753952 23329 raft_consensus.cc:493] T b6f07842d3aa47858df5665874c86f3a P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:22:08.761705 22610 raft_consensus.cc:697] T 746aabf856ab401c9ef5e859371e7b67 P 647a343a542d4a139fe106c19095129b [term 17 LEADER]: Becoming Leader. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:22:08.773360 23329 raft_consensus.cc:515] T b6f07842d3aa47858df5665874c86f3a P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:08.774024 22610 consensus_queue.cc:237] T 746aabf856ab401c9ef5e859371e7b67 P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:08.775086 16221 raft_consensus.cc:2468] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 22.
I20260709 13:22:08.766821 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "829b4ad8c5a54eababeea1181b960d4b" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:22:08.776080 16115 leader_election.cc:304] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac
I20260709 13:22:08.776827 23321 raft_consensus.cc:3055] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:22:08.777283 23321 raft_consensus.cc:740] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:22:08.777376 23357 raft_consensus.cc:2804] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:22:08.777946 23357 raft_consensus.cc:493] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:08.778296 23357 raft_consensus.cc:3060] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:22:08.779389 23329 leader_election.cc:290] T b6f07842d3aa47858df5665874c86f3a P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:08.660437 23316 consensus_queue.cc:237] T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:08.789767 16221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6f07842d3aa47858df5665874c86f3a" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:08.790635 16221 raft_consensus.cc:2393] T b6f07842d3aa47858df5665874c86f3a P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 19: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
I20260709 13:22:08.761813 23328 raft_consensus.cc:493] T 5a959dbecc2648eb85d1cc550f5a3245 P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:08.791260 23357 raft_consensus.cc:515] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:08.792944 23307 leader_election.cc:290] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:08.794588 23328 raft_consensus.cc:515] T 5a959dbecc2648eb85d1cc550f5a3245 P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:08.793241 16115 leader_election.cc:304] T 9c61a39fd4f84424b129c1a6f9ba404d P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:22:08.797058 23340 consensus_queue.cc:1059] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: INVALID_TERM, Last received: 24.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 13:22:08.797222 23329 raft_consensus.cc:2749] T 9c61a39fd4f84424b129c1a6f9ba404d P 647a343a542d4a139fe106c19095129b [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:22:08.797120 23328 leader_election.cc:290] T 5a959dbecc2648eb85d1cc550f5a3245 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:08.798774 16116 leader_election.cc:304] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:22:08.799687 23357 leader_election.cc:290] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 22 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:08.760480 23355 raft_consensus.cc:697] T 8a878999c0024d2abe517484180290fc P 647a343a542d4a139fe106c19095129b [term 18 LEADER]: Becoming Leader. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:22:08.806511 23355 consensus_queue.cc:237] T 8a878999c0024d2abe517484180290fc P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:22:08.820021 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 64759.3us
I20260709 13:22:08.803987 16116 leader_election.cc:304] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
I20260709 13:22:08.827230 16817 raft_consensus.cc:2468] T 829b4ad8c5a54eababeea1181b960d4b P cb82cc7bd6db46d980c75237a1402860 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 18.
I20260709 13:22:08.803020 23329 raft_consensus.cc:2749] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:22:08.830765 23210 raft_consensus.cc:515] T 8a878999c0024d2abe517484180290fc P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:08.834100 23210 leader_election.cc:290] T 8a878999c0024d2abe517484180290fc P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 17 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:22:08.834941 16299 raft_consensus.cc:3060] T 8a878999c0024d2abe517484180290fc P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:22:08.830785 16221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f5f89186204242bf377f7c42b2e2da" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:22:08.778317 23321 consensus_queue.cc:260] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:08.848546 16191 leader_election.cc:304] T af57996a8be84d398f0f023e8d08ef77 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: cb82cc7bd6db46d980c75237a1402860
I20260709 13:22:08.812922 23256 raft_consensus.cc:3060] T 829b4ad8c5a54eababeea1181b960d4b P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:22:08.854305 23023 raft_consensus.cc:493] T 2794af5b99984efebdee564035841cb0 P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:08.854966 23023 raft_consensus.cc:515] T 2794af5b99984efebdee564035841cb0 P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:08.857583 23023 leader_election.cc:290] T 2794af5b99984efebdee564035841cb0 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:08.860283 16117 leader_election.cc:304] T 829b4ad8c5a54eababeea1181b960d4b P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:22:08.760984 23211 raft_consensus.cc:515] T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:08.863430 23340 raft_consensus.cc:2804] T 829b4ad8c5a54eababeea1181b960d4b P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:22:08.864060 23340 raft_consensus.cc:697] T 829b4ad8c5a54eababeea1181b960d4b P 647a343a542d4a139fe106c19095129b [term 18 LEADER]: Becoming Leader. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:22:08.864109 16822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2794af5b99984efebdee564035841cb0" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:08.864951 16822 raft_consensus.cc:2468] T 2794af5b99984efebdee564035841cb0 P cb82cc7bd6db46d980c75237a1402860 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 17.
I20260709 13:22:08.866472 16818 raft_consensus.cc:2393] T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 22: Already voted for candidate cb82cc7bd6db46d980c75237a1402860 in this term.
I20260709 13:22:08.875047 16299 raft_consensus.cc:2468] T 8a878999c0024d2abe517484180290fc P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 18.
I20260709 13:22:08.868392 16117 leader_election.cc:304] T 2794af5b99984efebdee564035841cb0 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:22:08.836457 23327 raft_consensus.cc:2804] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:22:08.865186 23340 consensus_queue.cc:237] T 829b4ad8c5a54eababeea1181b960d4b P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:08.865288 23211 leader_election.cc:290] T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 22 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
W20260709 13:22:08.893193 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 64620.1us
I20260709 13:22:08.900964 23357 raft_consensus.cc:493] T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:08.905594 16191 leader_election.cc:304] T 5a96373bb0e14dce8a266b2fc6c58507 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:22:08.906656 16221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a959dbecc2648eb85d1cc550f5a3245" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:08.907922 16221 raft_consensus.cc:2410] T 5a959dbecc2648eb85d1cc550f5a3245 P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b for term 20 because replica has last-logged OpId of term: 18 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 11.
I20260709 13:22:08.911993 16191 leader_election.cc:304] T 02838c86531d4f98bf58d1c22b8baf77 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 90f3da57796b488c9087a2f523adf045
I20260709 13:22:08.901901 23269 raft_consensus.cc:2804] T af57996a8be84d398f0f023e8d08ef77 P 1ca2725d290e47d68b1a4b609c9816ac [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:22:08.892755 23327 raft_consensus.cc:697] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [term 22 LEADER]: Becoming Leader. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:22:08.911669 23357 raft_consensus.cc:515] T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:08.922952 23286 raft_consensus.cc:515] T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:08.911556 22917 raft_consensus.cc:2749] T 5a96373bb0e14dce8a266b2fc6c58507 P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:22:08.929992 23235 raft_consensus.cc:697] T 9e65676b21b748728ad39668cfe722ec P 90f3da57796b488c9087a2f523adf045 [term 19 LEADER]: Becoming Leader. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
W20260709 13:22:08.930941 23353 log.cc:927] Time spent T c9944d5e4b8c4109af9955c4f08c9095 P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.179s	user 0.001s	sys 0.000s
W20260709 13:22:08.931636 16266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 318066us
I20260709 13:22:08.980679 16266 leader_election.cc:304] T 57a744f5aaf944678a63469cbf514015 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:22:08.931782 22873 raft_consensus.cc:3055] T 8c11074579b046d99e585c2818e18a2c P 90f3da57796b488c9087a2f523adf045 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:22:08.932231 23326 consensus_queue.cc:1048] T cdbd78acb4cf46298e0de8ba0d130e19 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 13:22:08.934626 16824 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49892 (request call id 1047) took 1032 ms. Trace:
W20260709 13:22:08.935025 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 354166us
I20260709 13:22:08.935791 23349 raft_consensus.cc:2749] T d16dfad0c3b540e5b4b86492ddf87208 P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:22:08.939280 23314 raft_consensus.cc:493] T 5a96373bb0e14dce8a266b2fc6c58507 P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:22:08.941461 16298 raft_consensus.cc:1275] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045 [term 21 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 21 index: 16. (index mismatch)
I20260709 13:22:08.941898 23332 leader_election.cc:290] T 08c74490c40243a88e62d1760b5cd814 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 19 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:08.992184 16190 leader_election.cc:304] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:22:08.943310 16295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c74490c40243a88e62d1760b5cd814" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
W20260709 13:22:08.952332 16789 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:22:08.955384 16791 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:22:08.881964 22946 raft_consensus.cc:515] T fffab26427e24d398e2695aad40ca134 P 1ca2725d290e47d68b1a4b609c9816ac [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:08.923660 23354 raft_consensus.cc:2804] T 2794af5b99984efebdee564035841cb0 P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:22:08.928802 23256 raft_consensus.cc:2749] T 829b4ad8c5a54eababeea1181b960d4b P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:22:08.983096 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 71193.5us
I20260709 13:22:08.913134 23321 raft_consensus.cc:3060] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:22:08.974464 23252 raft_consensus.cc:493] T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:08.988056 16824 rpcz_store.cc:276] 0709 13:22:07.902447 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:22:07.902587 (+   140us) service_pool.cc:224] Handling call
0709 13:22:08.934403 (+1031816us) spinlock_profiling.cc:243] Waited 730 ms on lock 0x7b54003df254. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:22:08.934541 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1678904064}
I20260709 13:22:08.988219 22873 raft_consensus.cc:740] T 8c11074579b046d99e585c2818e18a2c P 90f3da57796b488c9087a2f523adf045 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
W20260709 13:22:09.008088 16788 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:22:09.008759 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6f07842d3aa47858df5665874c86f3a" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:22:09.011230 16820 raft_consensus.cc:1275] T 2794af5b99984efebdee564035841cb0 P cb82cc7bd6db46d980c75237a1402860 [term 17 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 17 index: 15. (index mismatch)
W20260709 13:22:08.989591 22908 consensus_meta.cc:287] T c2f5f89186204242bf377f7c42b2e2da P 1ca2725d290e47d68b1a4b609c9816ac: Time spent flushing consensus metadata: real 0.501s	user 0.009s	sys 0.002s
I20260709 13:22:09.024639 16225 raft_consensus.cc:3060] T 829b4ad8c5a54eababeea1181b960d4b P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:22:09.025451 16191 leader_election.cc:304] T 7c2095a5f4ad49408dbc7f69c606120d P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 90f3da57796b488c9087a2f523adf045
I20260709 13:22:09.012483 16825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:08.988005 23077 raft_consensus.cc:2804] T 57a744f5aaf944678a63469cbf514015 P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:22:09.005812 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6f07842d3aa47858df5665874c86f3a" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:09.020167 16266 leader_election.cc:304] T d2ab0ac8347641e6955e9bce7d90c1a2 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:22:09.034643 23235 consensus_queue.cc:237] T 9e65676b21b748728ad39668cfe722ec P 90f3da57796b488c9087a2f523adf045 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:09.027767 16825 raft_consensus.cc:2468] T 19baa7bc367844c288bca8f81439c413 P cb82cc7bd6db46d980c75237a1402860 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 22.
I20260709 13:22:09.006731 23354 raft_consensus.cc:493] T 2794af5b99984efebdee564035841cb0 P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:09.005782 23314 raft_consensus.cc:515] T 5a96373bb0e14dce8a266b2fc6c58507 P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:09.068887 23314 leader_election.cc:290] T 5a96373bb0e14dce8a266b2fc6c58507 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:22:09.070641 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a959dbecc2648eb85d1cc550f5a3245" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:09.071609 16297 raft_consensus.cc:2410] T 5a959dbecc2648eb85d1cc550f5a3245 P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b for term 20 because replica has last-logged OpId of term: 18 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 11.
I20260709 13:22:09.023571 22917 raft_consensus.cc:2804] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac [term 20 FOLLOWER]: Leader election won for term 20
W20260709 13:22:09.036162 23233 consensus_meta.cc:287] T 3884f864a96d428785895e5463e5728f P 90f3da57796b488c9087a2f523adf045: Time spent flushing consensus metadata: real 0.651s	user 0.013s	sys 0.002s
I20260709 13:22:09.014240 23366 raft_consensus.cc:2804] T 02838c86531d4f98bf58d1c22b8baf77 P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:22:09.087999 23321 raft_consensus.cc:493] T 84f2da98864b4425bf7c0055951fdd4b P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:09.035866 23077 raft_consensus.cc:493] T 57a744f5aaf944678a63469cbf514015 P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:09.076416 23286 leader_election.cc:290] T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 18 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:09.102310 16825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66f75124215f4ca6bef831e174a792d8" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:22:09.103178 16825 raft_consensus.cc:3060] T 66f75124215f4ca6bef831e174a792d8 P cb82cc7bd6db46d980c75237a1402860 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:22:09.066357 23354 raft_consensus.cc:3060] T 2794af5b99984efebdee564035841cb0 P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:22:09.096299 23321 raft_consensus.cc:515] T 84f2da98864b4425bf7c0055951fdd4b P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:09.048883 22908 raft_consensus.cc:2749] T c2f5f89186204242bf377f7c42b2e2da P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:22:09.124856 16296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f5f89186204242bf377f7c42b2e2da" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:22:09.125131 16225 raft_consensus.cc:2468] T 829b4ad8c5a54eababeea1181b960d4b P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 18.
I20260709 13:22:09.125550 16296 raft_consensus.cc:3060] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:22:09.131788 23087 leader_election.cc:290] T 1df93779209748b9860f6d893dd3de53 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 18 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:09.132385 23269 raft_consensus.cc:697] T af57996a8be84d398f0f023e8d08ef77 P 1ca2725d290e47d68b1a4b609c9816ac [term 20 LEADER]: Becoming Leader. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:22:09.134065 23269 consensus_queue.cc:237] T af57996a8be84d398f0f023e8d08ef77 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:09.139863 23331 raft_consensus.cc:3055] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [term 25 LEADER]: Stepping down as leader of term 25
I20260709 13:22:09.140362 23331 raft_consensus.cc:740] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [term 25 LEADER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:22:09.141240 23331 consensus_queue.cc:260] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 25.16, Last appended by leader: 16, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:09.024129 23252 raft_consensus.cc:515] T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:09.097214 23366 raft_consensus.cc:493] T 02838c86531d4f98bf58d1c22b8baf77 P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Starting leader election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:22:09.145937 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2794af5b99984efebdee564035841cb0" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
W20260709 13:22:09.146052 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 120673us
I20260709 13:22:09.104722 23077 raft_consensus.cc:3060] T 57a744f5aaf944678a63469cbf514015 P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:22:09.061604 23349 consensus_queue.cc:1059] T 8158bbd17b994d2eb96f560bd0f1b88d P 90f3da57796b488c9087a2f523adf045 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: INVALID_TERM, Last received: 17.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.067s
I20260709 13:22:09.018581 23316 consensus_queue.cc:1048] T e5ad4eb281564936a59f41e387a47156 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.093s
I20260709 13:22:09.064636 23024 consensus_queue.cc:1048] T 2794af5b99984efebdee564035841cb0 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.047s
I20260709 13:22:09.060631 23210 consensus_queue.cc:1048] T e4dc6b0541ca431ebc757beac00521f1 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.051s
I20260709 13:22:09.101311 22917 raft_consensus.cc:697] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac [term 20 LEADER]: Becoming Leader. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:22:09.079595 23086 raft_consensus.cc:515] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Starting leader election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:09.047745 23327 consensus_queue.cc:237] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 20.16, Last appended by leader: 16, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:09.157048 23366 raft_consensus.cc:3060] T 02838c86531d4f98bf58d1c22b8baf77 P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:22:09.177836 16296 raft_consensus.cc:2468] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 22.
I20260709 13:22:09.158378 16145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5d59407eb545b895d8d922e0dd69fd" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:22:09.155933 16191 leader_election.cc:304] T e4dc6b0541ca431ebc757beac00521f1 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:22:09.178655 16219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66f75124215f4ca6bef831e174a792d8" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
W20260709 13:22:09.200582 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 183212us
I20260709 13:22:09.201314 16217 raft_consensus.cc:1240] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [term 26 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 25. Current term is 26. Ops: [25.16-25.16]
I20260709 13:22:09.206219 16219 raft_consensus.cc:1275] T 746aabf856ab401c9ef5e859371e7b67 P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 13:22:09.206755 23371 raft_consensus.cc:2804] T 7c2095a5f4ad49408dbc7f69c606120d P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:22:09.207293 23371 raft_consensus.cc:493] T 7c2095a5f4ad49408dbc7f69c606120d P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Starting leader election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:22:09.207603 22946 leader_election.cc:290] T fffab26427e24d398e2695aad40ca134 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 20 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
W20260709 13:22:09.208339 23360 log.cc:927] Time spent T ecc1e363230043fe9fef390f0b979f20 P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 0.161s	user 0.001s	sys 0.000s
I20260709 13:22:09.208451 22946 raft_consensus.cc:493] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:09.208715 16296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fffab26427e24d398e2695aad40ca134" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:22:09.209021 22946 raft_consensus.cc:515] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:09.206241 23351 raft_consensus.cc:2804] T d2ab0ac8347641e6955e9bce7d90c1a2 P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:22:09.210402 23351 raft_consensus.cc:697] T d2ab0ac8347641e6955e9bce7d90c1a2 P 90f3da57796b488c9087a2f523adf045 [term 18 LEADER]: Becoming Leader. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:22:09.161613 23328 consensus_queue.cc:1059] T ecc1e363230043fe9fef390f0b979f20 P 647a343a542d4a139fe106c19095129b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: INVALID_TERM, Last received: 19.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.036s
I20260709 13:22:09.168911 23306 raft_consensus.cc:515] T 434bff3082d141838068f323bdb0d1ed P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:22:09.222563 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 177152us
I20260709 13:22:09.242650 16265 leader_election.cc:304] T c0eab25ce4c64e018a041beeda2867db P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:22:09.248124 16296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada5a004ba7c46819051e3ccec71ffdb" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:09.255981 16115 leader_election.cc:304] T 08c74490c40243a88e62d1760b5cd814 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:22:09.257984 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1df93779209748b9860f6d893dd3de53" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:22:09.261646 16115 leader_election.cc:304] T b6f07842d3aa47858df5665874c86f3a P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:22:09.263775 16296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31969392ccdb43b491911a319e8bc7e1" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:09.308586 16293 raft_consensus.cc:1240] T 434bff3082d141838068f323bdb0d1ed P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 18. Current term is 19. Ops: []
I20260709 13:22:09.163455 22873 consensus_queue.cc:260] T 8c11074579b046d99e585c2818e18a2c P 90f3da57796b488c9087a2f523adf045 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:09.221422 23375 consensus_queue.cc:1048] T 746aabf856ab401c9ef5e859371e7b67 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.071s
I20260709 13:22:09.207649 23371 raft_consensus.cc:3060] T 7c2095a5f4ad49408dbc7f69c606120d P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:22:09.281706 22946 leader_election.cc:290] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:09.322372 16224 raft_consensus.cc:1240] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 17. Current term is 18. Ops: []
W20260709 13:22:09.304909 23343 log.cc:927] Time spent T 8cc959fb7f704fba9009c0595467d074 P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.066s	user 0.000s	sys 0.001s
I20260709 13:22:09.290659 16115 leader_election.cc:304] T 5a959dbecc2648eb85d1cc550f5a3245 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
W20260709 13:22:09.291697 16788 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:22:09.292513 16220 raft_consensus.cc:1240] T 8c11074579b046d99e585c2818e18a2c P 1ca2725d290e47d68b1a4b609c9816ac [term 20 LEADER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 19. Current term is 20. Ops: []
I20260709 13:22:09.266286 23349 raft_consensus.cc:2804] T c0eab25ce4c64e018a041beeda2867db P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:22:09.393527 23349 raft_consensus.cc:697] T c0eab25ce4c64e018a041beeda2867db P 90f3da57796b488c9087a2f523adf045 [term 20 LEADER]: Becoming Leader. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:22:09.393847 16220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:22:09.284937 16191 leader_election.cc:304] T 84f2da98864b4425bf7c0055951fdd4b P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 90f3da57796b488c9087a2f523adf045
W20260709 13:22:09.396245 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 177380us
I20260709 13:22:09.286068 23087 raft_consensus.cc:3060] T b6f07842d3aa47858df5665874c86f3a P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:22:09.399291 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 244132us
W20260709 13:22:09.400035 16790 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:22:09.314682 16822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada5a004ba7c46819051e3ccec71ffdb" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:09.315282 16223 raft_consensus.cc:1240] T b6f07842d3aa47858df5665874c86f3a P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 18. Current term is 19. Ops: []
I20260709 13:22:09.282110 23355 raft_consensus.cc:2749] T 08c74490c40243a88e62d1760b5cd814 P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
W20260709 13:22:09.337419 23368 log.cc:927] Time spent T 8d4ab870a05347868956aa8b841b2981 P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.168s	user 0.001s	sys 0.000s
I20260709 13:22:09.293162 23373 raft_consensus.cc:2749] T e4dc6b0541ca431ebc757beac00521f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:22:09.339444 23061 consensus_meta.cc:287] T 08c74490c40243a88e62d1760b5cd814 P 1ca2725d290e47d68b1a4b609c9816ac: Time spent flushing consensus metadata: real 0.696s	user 0.008s	sys 0.005s
I20260709 13:22:09.328150 22917 consensus_queue.cc:237] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.12, Last appended by leader: 12, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } }
W20260709 13:22:09.376298 16789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 83566.1us
I20260709 13:22:09.346020 22873 raft_consensus.cc:3060] T 8c11074579b046d99e585c2818e18a2c P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:22:09.345531 23286 raft_consensus.cc:493] T 8cc959fb7f704fba9009c0595467d074 P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:09.374877 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f2da98864b4425bf7c0055951fdd4b" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:09.239665 23351 consensus_queue.cc:237] T d2ab0ac8347641e6955e9bce7d90c1a2 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 16.14, Last appended by leader: 14, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:09.388958 16296 raft_consensus.cc:1275] T c4f65cdf00b748bb963ee6e81ef43b79 P 90f3da57796b488c9087a2f523adf045 [term 21 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 20 index: 13. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
I20260709 13:22:09.390161 23306 leader_election.cc:290] T 434bff3082d141838068f323bdb0d1ed P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 19 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:09.390686 23077 raft_consensus.cc:515] T 57a744f5aaf944678a63469cbf514015 P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:09.425608 16223 raft_consensus.cc:1240] T fffab26427e24d398e2695aad40ca134 P 1ca2725d290e47d68b1a4b609c9816ac [term 20 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 19. Current term is 20. Ops: []
I20260709 13:22:09.373872 23321 leader_election.cc:290] T 84f2da98864b4425bf7c0055951fdd4b P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:09.421013 16226 raft_consensus.cc:2468] T 84f2da98864b4425bf7c0055951fdd4b P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 16.
I20260709 13:22:09.406283 23252 leader_election.cc:290] T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
W20260709 13:22:09.438345 16115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 147813us
I20260709 13:22:09.454746 16825 raft_consensus.cc:2468] T 66f75124215f4ca6bef831e174a792d8 P cb82cc7bd6db46d980c75237a1402860 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 18.
I20260709 13:22:09.457020 16223 raft_consensus.cc:1240] T c2f5f89186204242bf377f7c42b2e2da P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 20. Current term is 21. Ops: []
I20260709 13:22:09.394912 23349 consensus_queue.cc:237] T c0eab25ce4c64e018a041beeda2867db P 90f3da57796b488c9087a2f523adf045 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:09.429592 23286 raft_consensus.cc:515] T 8cc959fb7f704fba9009c0595467d074 P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:09.471128 23354 raft_consensus.cc:515] T 2794af5b99984efebdee564035841cb0 P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:09.473206 23086 leader_election.cc:290] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 20 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:09.473476 23286 leader_election.cc:290] T 8cc959fb7f704fba9009c0595467d074 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:09.474058 23354 leader_election.cc:290] T 2794af5b99984efebdee564035841cb0 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 18 election: Requested vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:09.417256 23061 raft_consensus.cc:515] T 08c74490c40243a88e62d1760b5cd814 P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:09.475438 23077 leader_election.cc:290] T 57a744f5aaf944678a63469cbf514015 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 17 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:09.449754 22610 raft_consensus.cc:2749] T 5a959dbecc2648eb85d1cc550f5a3245 P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:22:09.457096 23377 raft_consensus.cc:2804] T 84f2da98864b4425bf7c0055951fdd4b P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:22:09.477289 23061 leader_election.cc:290] T 08c74490c40243a88e62d1760b5cd814 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 19 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:09.478123 23077 raft_consensus.cc:493] T 84f2da98864b4425bf7c0055951fdd4b P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:22:09.478708 23077 raft_consensus.cc:515] T 84f2da98864b4425bf7c0055951fdd4b P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:09.481329 23077 leader_election.cc:290] T 84f2da98864b4425bf7c0055951fdd4b P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:22:09.483652 16223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4f65cdf00b748bb963ee6e81ef43b79" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
W20260709 13:22:09.484602 16265 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:22:09.486480 16265 leader_election.cc:304] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:22:09.489928 23077 raft_consensus.cc:2749] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:22:09.491508 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdbd78acb4cf46298e0de8ba0d130e19" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:09.492173 16226 raft_consensus.cc:2468] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 22.
I20260709 13:22:09.493229 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a96373bb0e14dce8a266b2fc6c58507" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:09.491333 16265 leader_election.cc:304] T 8d4ab870a05347868956aa8b841b2981 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 647a343a542d4a139fe106c19095129b
I20260709 13:22:09.496512 23087 raft_consensus.cc:2749] T b6f07842d3aa47858df5665874c86f3a P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:22:09.485186 16223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada5a004ba7c46819051e3ccec71ffdb" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:09.497484 16223 raft_consensus.cc:2468] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 20.
I20260709 13:22:09.498581 16223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5d59407eb545b895d8d922e0dd69fd" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:09.499508 16223 raft_consensus.cc:2393] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 in current term 22: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
I20260709 13:22:09.499526 23306 consensus_queue.cc:1059] T 68a6838d82ef4cac83507f906139d90f P 90f3da57796b488c9087a2f523adf045 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: INVALID_TERM, Last received: 18.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:22:09.500629 16223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:09.500825 23210 raft_consensus.cc:3055] T 68a6838d82ef4cac83507f906139d90f P 90f3da57796b488c9087a2f523adf045 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:22:09.501271 23210 raft_consensus.cc:740] T 68a6838d82ef4cac83507f906139d90f P 90f3da57796b488c9087a2f523adf045 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:22:09.501298 16223 raft_consensus.cc:2468] T 19baa7bc367844c288bca8f81439c413 P 1ca2725d290e47d68b1a4b609c9816ac [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 22.
I20260709 13:22:09.502571 16223 raft_consensus.cc:1240] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [term 23 LEADER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 22. Current term is 23. Ops: []
I20260709 13:22:09.502264 23210 consensus_queue.cc:260] T 68a6838d82ef4cac83507f906139d90f P 90f3da57796b488c9087a2f523adf045 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:22:09.459448 23381 log.cc:927] Time spent T ada5a004ba7c46819051e3ccec71ffdb P cb82cc7bd6db46d980c75237a1402860: Append to log took a long time: real 0.073s	user 0.001s	sys 0.000s
I20260709 13:22:09.496584 23077 raft_consensus.cc:2804] T 8d4ab870a05347868956aa8b841b2981 P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:22:09.494863 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a878999c0024d2abe517484180290fc" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:22:09.503815 23077 raft_consensus.cc:493] T 8d4ab870a05347868956aa8b841b2981 P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:22:09.504465 16226 raft_consensus.cc:2368] T 8a878999c0024d2abe517484180290fc P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 for earlier term 17. Current term is 18.
I20260709 13:22:09.504408 23077 raft_consensus.cc:3060] T 8d4ab870a05347868956aa8b841b2981 P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:22:09.505663 16226 raft_consensus.cc:1240] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 18. Current term is 19. Ops: []
I20260709 13:22:09.509696 16296 raft_consensus.cc:1275] T 8c11074579b046d99e585c2818e18a2c P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 20 index: 16. (index mismatch)
W20260709 13:22:09.514457 16115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 67754.1us
W20260709 13:22:09.517508 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 68709.7us
I20260709 13:22:09.518049 16115 leader_election.cc:304] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac
I20260709 13:22:09.526345 16292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c74490c40243a88e62d1760b5cd814" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
W20260709 13:22:09.529609 16790 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:22:09.532119 16145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a959dbecc2648eb85d1cc550f5a3245" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:22:09.533138 16145 raft_consensus.cc:2393] T 5a959dbecc2648eb85d1cc550f5a3245 P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 19: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
I20260709 13:22:09.533952 16147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d16dfad0c3b540e5b4b86492ddf87208" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
W20260709 13:22:09.539649 16788 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:22:09.527755 16265 leader_election.cc:304] T d2ab0ac8347641e6955e9bce7d90c1a2 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: cb82cc7bd6db46d980c75237a1402860
I20260709 13:22:09.556130 16147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 23 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:22:09.557111 16147 raft_consensus.cc:2410] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac for term 23 because replica has last-logged OpId of term: 21 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 11.
I20260709 13:22:09.558408 16151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1df93779209748b9860f6d893dd3de53" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:22:09.559581 16151 raft_consensus.cc:2393] T 1df93779209748b9860f6d893dd3de53 P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 18: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
I20260709 13:22:09.560282 16144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68a6838d82ef4cac83507f906139d90f" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:22:09.560479 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8158bbd17b994d2eb96f560bd0f1b88d" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:22:09.560951 16144 raft_consensus.cc:3060] T 68a6838d82ef4cac83507f906139d90f P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:22:09.562252 16789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 148244us
W20260709 13:22:09.564560 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 53132.3us
I20260709 13:22:09.564577 16148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a878999c0024d2abe517484180290fc" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 18 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:22:09.513407 23311 raft_consensus.cc:493] T 18ad7e3af5504571a69d10c00750a01c P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:22:09.566627 16145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62fe3346cd3c42c4a55e14af9073bca4" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 26 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:22:09.526512 22610 raft_consensus.cc:2804] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:22:09.571089 16190 leader_election.cc:304] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860
I20260709 13:22:09.577900 16146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0eab25ce4c64e018a041beeda2867db" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:22:09.574180 16143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806d7690c2b644959152074bb2cab0a7" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:22:09.574244 16142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b7d1ab519044eda2d5e04a07242f25" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:22:09.485057 23211 consensus_queue.cc:1048] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.041s
I20260709 13:22:09.583199 16825 raft_consensus.cc:1240] T 8c11074579b046d99e585c2818e18a2c P cb82cc7bd6db46d980c75237a1402860 [term 20 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 19. Current term is 20. Ops: []
I20260709 13:22:09.575680 16147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ad5dbfa99ee4b8d9e28be92b3f36db5" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:22:09.576172 16226 raft_consensus.cc:1275] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 18 index: 15. (index mismatch)
I20260709 13:22:09.580704 16146 raft_consensus.cc:2393] T c0eab25ce4c64e018a041beeda2867db P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 20: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
I20260709 13:22:09.580518 16223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806d7690c2b644959152074bb2cab0a7" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:22:09.581772 16143 raft_consensus.cc:2393] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 20: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
I20260709 13:22:09.526111 23326 consensus_queue.cc:1048] T cdbd78acb4cf46298e0de8ba0d130e19 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.049s
I20260709 13:22:09.512745 23366 raft_consensus.cc:515] T 02838c86531d4f98bf58d1c22b8baf77 P 1ca2725d290e47d68b1a4b609c9816ac [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:09.515513 22873 consensus_queue.cc:1059] T 1df93779209748b9860f6d893dd3de53 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: INVALID_TERM, Last received: 17.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.059s
I20260709 13:22:09.526391 23316 consensus_queue.cc:1048] T ada5a004ba7c46819051e3ccec71ffdb P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.055s
I20260709 13:22:09.588979 16822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434bff3082d141838068f323bdb0d1ed" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:22:09.569691 23377 raft_consensus.cc:697] T 84f2da98864b4425bf7c0055951fdd4b P 1ca2725d290e47d68b1a4b609c9816ac [term 16 LEADER]: Becoming Leader. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:22:09.607241 16143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66f75124215f4ca6bef831e174a792d8" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:22:09.607882 16824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57a744f5aaf944678a63469cbf514015" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:22:09.611011 16824 raft_consensus.cc:3060] T 57a744f5aaf944678a63469cbf514015 P cb82cc7bd6db46d980c75237a1402860 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:22:09.612097 16223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:09.612932 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f2da98864b4425bf7c0055951fdd4b" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:09.613247 16143 raft_consensus.cc:2368] T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac for earlier term 17. Current term is 18.
I20260709 13:22:09.621490 16218 raft_consensus.cc:1240] T 02838c86531d4f98bf58d1c22b8baf77 P 1ca2725d290e47d68b1a4b609c9816ac [term 20 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 19. Current term is 20. Ops: []
I20260709 13:22:09.622072 16144 raft_consensus.cc:2468] T 68a6838d82ef4cac83507f906139d90f P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 19.
W20260709 13:22:09.622159 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 48375.9us
I20260709 13:22:09.587230 22610 raft_consensus.cc:697] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b [term 22 LEADER]: Becoming Leader. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:22:09.625204 16144 raft_consensus.cc:1240] T 9c61a39fd4f84424b129c1a6f9ba404d P 647a343a542d4a139fe106c19095129b [term 21 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 20. Current term is 21. Ops: [20.11-20.11]
I20260709 13:22:09.590633 23061 raft_consensus.cc:2749] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:22:09.624233 16146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6f07842d3aa47858df5665874c86f3a" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:22:09.633309 16223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5d59407eb545b895d8d922e0dd69fd" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:22:09.648698 16223 raft_consensus.cc:2393] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 in current term 22: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
I20260709 13:22:09.602811 23256 consensus_queue.cc:1048] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.028s
I20260709 13:22:09.657119 16824 raft_consensus.cc:2468] T 57a744f5aaf944678a63469cbf514015 P cb82cc7bd6db46d980c75237a1402860 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 17.
I20260709 13:22:09.602350 23311 raft_consensus.cc:515] T 18ad7e3af5504571a69d10c00750a01c P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } }
I20260709 13:22:09.674306 16143 raft_consensus.cc:1275] T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b [term 23 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 21 index: 14. Preceding OpId from leader: term: 23 index: 15. (index mismatch)
I20260709 13:22:09.677686 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62fe3346cd3c42c4a55e14af9073bca4" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 26 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:22:09.678627 16226 raft_consensus.cc:2376] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [term 26 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 26. Re-sending same reply.
I20260709 13:22:09.635565 23235 raft_consensus.cc:493] T 5a959dbecc2648eb85d1cc550f5a3245 P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:09.682902 16223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cc959fb7f704fba9009c0595467d074" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:09.686148 16226 raft_consensus.cc:1275] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 21 index: 16. (index mismatch)
I20260709 13:22:09.657475 23388 raft_consensus.cc:493] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:09.688344 16144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fffab26427e24d398e2695aad40ca134" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:22:09.696545 16144 raft_consensus.cc:3060] T fffab26427e24d398e2695aad40ca134 P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:22:09.700170 16790 leader_election.cc:304] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 90f3da57796b488c9087a2f523adf045
I20260709 13:22:09.710340 16148 raft_consensus.cc:1275] T 8cc959fb7f704fba9009c0595467d074 P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 13:22:09.711532 16191 leader_election.cc:304] T d16dfad0c3b540e5b4b86492ddf87208 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:22:09.712549 23385 raft_consensus.cc:2749] T d16dfad0c3b540e5b4b86492ddf87208 P 1ca2725d290e47d68b1a4b609c9816ac [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:22:09.712895 16191 leader_election.cc:304] T 5a959dbecc2648eb85d1cc550f5a3245 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:22:09.713948 23385 raft_consensus.cc:3060] T 5a959dbecc2648eb85d1cc550f5a3245 P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:22:09.714329 16191 leader_election.cc:304] T 1df93779209748b9860f6d893dd3de53 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:22:09.715376 23370 raft_consensus.cc:2749] T 1df93779209748b9860f6d893dd3de53 P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:22:09.715853 16191 leader_election.cc:304] T 19baa7bc367844c288bca8f81439c413 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:22:09.716837 23373 raft_consensus.cc:2749] T 19baa7bc367844c288bca8f81439c413 P 1ca2725d290e47d68b1a4b609c9816ac [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:22:09.717626 16142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c74490c40243a88e62d1760b5cd814" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:22:09.718890 16191 leader_election.cc:304] T 8a878999c0024d2abe517484180290fc P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:22:09.719573 16142 raft_consensus.cc:2393] T 08c74490c40243a88e62d1760b5cd814 P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 19: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
I20260709 13:22:09.720203 16144 raft_consensus.cc:2468] T fffab26427e24d398e2695aad40ca134 P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 20.
I20260709 13:22:09.721115 16191 leader_election.cc:304] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
I20260709 13:22:09.723532 16191 leader_election.cc:304] T c0eab25ce4c64e018a041beeda2867db P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:22:09.731411 23061 raft_consensus.cc:2749] T 8a878999c0024d2abe517484180290fc P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:22:09.688325 23235 raft_consensus.cc:515] T 5a959dbecc2648eb85d1cc550f5a3245 P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:09.735523 23383 raft_consensus.cc:493] T 9e65676b21b748728ad39668cfe722ec P cb82cc7bd6db46d980c75237a1402860 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:09.728669 23308 raft_consensus.cc:2764] T d2ab0ac8347641e6955e9bce7d90c1a2 P 90f3da57796b488c9087a2f523adf045 [term 18 LEADER]: Leader election decision vote started in defunct term 17: won
I20260709 13:22:09.694701 23388 raft_consensus.cc:515] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:09.703305 23395 consensus_queue.cc:1059] T 3a7ca61592e848ea9b0fb81a2ce33cdc P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: INVALID_TERM, Last received: 19.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.044s
I20260709 13:22:09.737835 23370 raft_consensus.cc:2749] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:22:09.736946 23256 raft_consensus.cc:2749] T c0eab25ce4c64e018a041beeda2867db P 1ca2725d290e47d68b1a4b609c9816ac [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:22:09.738545 23396 raft_consensus.cc:493] T 26354361e9b044cea1c2b86c4882fe92 P cb82cc7bd6db46d980c75237a1402860 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:22:09.709977 22610 consensus_queue.cc:237] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.12, Last appended by leader: 12, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:09.692502 23377 consensus_queue.cc:237] T 84f2da98864b4425bf7c0055951fdd4b P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:09.736310 23371 raft_consensus.cc:515] T 7c2095a5f4ad49408dbc7f69c606120d P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:09.841912 16220 raft_consensus.cc:1240] T 7c2095a5f4ad49408dbc7f69c606120d P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 16. Current term is 17. Ops: []
I20260709 13:22:09.843037 23371 leader_election.cc:290] T 7c2095a5f4ad49408dbc7f69c606120d P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 17 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:09.843690 23371 raft_consensus.cc:493] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:22:09.844179 23371 raft_consensus.cc:515] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:09.845187 16225 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57604 (request call id 3389) took 1080 ms. Trace:
I20260709 13:22:09.845242 16821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806d7690c2b644959152074bb2cab0a7" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:22:09.846019 16821 raft_consensus.cc:3060] T 806d7690c2b644959152074bb2cab0a7 P cb82cc7bd6db46d980c75237a1402860 [term 18 LEARNER]: Advancing to term 20
I20260709 13:22:09.845750 16225 rpcz_store.cc:276] 0709 13:22:08.764666 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:22:08.766568 (+  1902us) service_pool.cc:224] Handling call
0709 13:22:09.845022 (+1078454us) spinlock_profiling.cc:243] Waited 249 ms on lock 0x7b54001419d4. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:22:09.845133 (+   111us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":571689344}
I20260709 13:22:09.846352 23371 leader_election.cc:290] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:09.846983 16822 raft_consensus.cc:1240] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860 [term 22 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 21. Current term is 22. Ops: [21.13-21.13]
I20260709 13:22:09.848278 16819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31969392ccdb43b491911a319e8bc7e1" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:09.849679 16823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2794af5b99984efebdee564035841cb0" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:22:09.801925 23366 leader_election.cc:290] T 02838c86531d4f98bf58d1c22b8baf77 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 20 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:09.816681 23204 raft_consensus.cc:3060] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:22:09.821053 23331 raft_consensus.cc:3060] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:22:09.821861 23077 raft_consensus.cc:515] T 8d4ab870a05347868956aa8b841b2981 P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:09.823585 23311 leader_election.cc:290] T 18ad7e3af5504571a69d10c00750a01c P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
W20260709 13:22:09.824784 23393 log.cc:927] Time spent T c0eab25ce4c64e018a041beeda2867db P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.130s	user 0.000s	sys 0.000s
I20260709 13:22:09.831292 23235 leader_election.cc:290] T 5a959dbecc2648eb85d1cc550f5a3245 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:22:09.882582 16223 raft_consensus.cc:1275] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
I20260709 13:22:09.882601 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a959dbecc2648eb85d1cc550f5a3245" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:09.885334 16293 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48674 (request call id 3473) took 1221 ms. Trace:
I20260709 13:22:09.885850 16293 rpcz_store.cc:276] 0709 13:22:08.663750 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:22:08.663879 (+   129us) service_pool.cc:224] Handling call
0709 13:22:08.664369 (+   490us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:22:09.309213 (+644844us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:22:09.885008 (+575795us) spinlock_profiling.cc:243] Waited 644 ms on lock 0x7b54000a9954. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:22:09.885236 (+   228us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1480988928}
I20260709 13:22:09.888955 23077 leader_election.cc:290] T 8d4ab870a05347868956aa8b841b2981 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 19 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:22:09.889858 23375 raft_consensus.cc:493] T e5ad4eb281564936a59f41e387a47156 P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:09.890395 23375 raft_consensus.cc:515] T e5ad4eb281564936a59f41e387a47156 P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:09.891289 23347 raft_consensus.cc:493] T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:22:09.891794 23347 raft_consensus.cc:515] T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:09.892381 23385 raft_consensus.cc:2749] T 5a959dbecc2648eb85d1cc550f5a3245 P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:22:09.893090 16226 raft_consensus.cc:2468] T 5a959dbecc2648eb85d1cc550f5a3245 P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 19.
I20260709 13:22:09.894471 23210 raft_consensus.cc:3060] T 68a6838d82ef4cac83507f906139d90f P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:22:09.801198 23396 raft_consensus.cc:515] T 26354361e9b044cea1c2b86c4882fe92 P cb82cc7bd6db46d980c75237a1402860 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:09.897068 23396 leader_election.cc:290] T 26354361e9b044cea1c2b86c4882fe92 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:09.881203 23398 raft_consensus.cc:493] T 74ca8140f35c4ae29e8a664f6bdfc829 P cb82cc7bd6db46d980c75237a1402860 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:09.898037 23398 raft_consensus.cc:515] T 74ca8140f35c4ae29e8a664f6bdfc829 P cb82cc7bd6db46d980c75237a1402860 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:09.900205 23398 leader_election.cc:290] T 74ca8140f35c4ae29e8a664f6bdfc829 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:09.835209 23348 raft_consensus.cc:3055] T ecc1e363230043fe9fef390f0b979f20 P 647a343a542d4a139fe106c19095129b [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:22:09.903393 16217 raft_consensus.cc:1275] T 829b4ad8c5a54eababeea1181b960d4b P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 13:22:09.837878 23357 leader_election.cc:290] T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:09.839385 23388 leader_election.cc:290] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:22:09.839910 16149 raft_consensus.cc:2393] T 8158bbd17b994d2eb96f560bd0f1b88d P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 18: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
I20260709 13:22:09.840981 23086 raft_consensus.cc:493] T 02838c86531d4f98bf58d1c22b8baf77 P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:22:09.854224 16148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f82224f4564fea93572e8206bad1fe" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
W20260709 13:22:09.795199 16266 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:22:09.854216 16821 raft_consensus.cc:2468] T 806d7690c2b644959152074bb2cab0a7 P cb82cc7bd6db46d980c75237a1402860 [term 20 LEARNER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 20.
I20260709 13:22:09.854526 16151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26354361e9b044cea1c2b86c4882fe92" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
W20260709 13:22:09.908418 16266 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:22:09.870456 16147 raft_consensus.cc:2468] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 19.
I20260709 13:22:09.758864 23383 raft_consensus.cc:515] T 9e65676b21b748728ad39668cfe722ec P cb82cc7bd6db46d980c75237a1402860 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:09.880285 16219 raft_consensus.cc:1240] T 08c74490c40243a88e62d1760b5cd814 P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 18. Current term is 19. Ops: []
I20260709 13:22:09.912694 16151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434bff3082d141838068f323bdb0d1ed" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:22:09.912789 16223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d4ab870a05347868956aa8b841b2981" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:22:09.910567 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18ad7e3af5504571a69d10c00750a01c" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:09.913697 16298 raft_consensus.cc:2468] T 18ad7e3af5504571a69d10c00750a01c P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 17.
I20260709 13:22:09.916543 16151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a878999c0024d2abe517484180290fc" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:22:09.863787 23292 raft_consensus.cc:493] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:09.880286 16142 raft_consensus.cc:1240] T fffab26427e24d398e2695aad40ca134 P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 19. Current term is 20. Ops: [19.12-19.12]
I20260709 13:22:09.918547 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02838c86531d4f98bf58d1c22b8baf77" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:22:09.916538 16148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a96373bb0e14dce8a266b2fc6c58507" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:22:09.919754 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c2095a5f4ad49408dbc7f69c606120d" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:22:09.920826 16151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:22:09.921837 23292 raft_consensus.cc:515] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:09.922253 23347 leader_election.cc:290] T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
W20260709 13:22:09.923893 23353 log.cc:927] Time spent T c9944d5e4b8c4109af9955c4f08c9095 P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.908s	user 0.001s	sys 0.002s
I20260709 13:22:09.924008 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a378d7f796f94615b3b347119a9880fd" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:09.924943 16142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:22:09.903074 23348 raft_consensus.cc:740] T ecc1e363230043fe9fef390f0b979f20 P 647a343a542d4a139fe106c19095129b [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:22:09.927387 16824 raft_consensus.cc:1275] T 9e65676b21b748728ad39668cfe722ec P cb82cc7bd6db46d980c75237a1402860 [term 19 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 13:22:09.928153 16151 raft_consensus.cc:1240] T 68a6838d82ef4cac83507f906139d90f P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 18. Current term is 19. Ops: []
I20260709 13:22:09.924254 23292 leader_election.cc:290] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:22:09.925572 16142 raft_consensus.cc:2468] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 22.
I20260709 13:22:09.929576 23388 raft_consensus.cc:493] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:09.930117 23388 raft_consensus.cc:515] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:09.930354 23348 consensus_queue.cc:260] T ecc1e363230043fe9fef390f0b979f20 P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.13, Last appended by leader: 13, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:09.931578 23348 raft_consensus.cc:3060] T ecc1e363230043fe9fef390f0b979f20 P 647a343a542d4a139fe106c19095129b [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:22:09.932276 23388 leader_election.cc:290] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
W20260709 13:22:09.933298 16114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:22:09.934352 16151 raft_consensus.cc:1240] T 08c74490c40243a88e62d1760b5cd814 P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 18. Current term is 19. Ops: [18.11-18.11]
I20260709 13:22:09.934505 16190 leader_election.cc:304] T 18ad7e3af5504571a69d10c00750a01c P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:22:09.935510 23311 raft_consensus.cc:2804] T 18ad7e3af5504571a69d10c00750a01c P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:22:09.936008 23311 raft_consensus.cc:493] T 18ad7e3af5504571a69d10c00750a01c P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Starting leader election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:22:09.936462 23311 raft_consensus.cc:3060] T 18ad7e3af5504571a69d10c00750a01c P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:22:09.948565 23375 leader_election.cc:290] T e5ad4eb281564936a59f41e387a47156 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:09.918686 23086 raft_consensus.cc:515] T 02838c86531d4f98bf58d1c22b8baf77 P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:09.953516 23086 leader_election.cc:290] T 02838c86531d4f98bf58d1c22b8baf77 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:09.954187 16142 raft_consensus.cc:1240] T 2794af5b99984efebdee564035841cb0 P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 17. Current term is 18. Ops: []
I20260709 13:22:09.954807 23086 raft_consensus.cc:493] T b87147731a68400e8cce9b879daa7c5d P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:22:09.968958 23394 raft_consensus.cc:3055] T 3a7ca61592e848ea9b0fb81a2ce33cdc P cb82cc7bd6db46d980c75237a1402860 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:22:09.969405 23394 raft_consensus.cc:740] T 3a7ca61592e848ea9b0fb81a2ce33cdc P cb82cc7bd6db46d980c75237a1402860 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:22:09.970239 23394 consensus_queue.cc:260] T 3a7ca61592e848ea9b0fb81a2ce33cdc P cb82cc7bd6db46d980c75237a1402860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:22:09.973721 23376 log.cc:927] Time spent T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 0.636s	user 0.003s	sys 0.000s
I20260709 13:22:09.979252 16219 raft_consensus.cc:3060] T d2ab0ac8347641e6955e9bce7d90c1a2 P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:22:09.953554 23383 leader_election.cc:290] T 9e65676b21b748728ad39668cfe722ec P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:09.984547 16223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b7d1ab519044eda2d5e04a07242f25" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:09.985231 16223 raft_consensus.cc:2468] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 18.
I20260709 13:22:09.985735 16265 leader_election.cc:304] T 26354361e9b044cea1c2b86c4882fe92 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
I20260709 13:22:09.982718 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f2da98864b4425bf7c0055951fdd4b" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:22:09.986136 16223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02838c86531d4f98bf58d1c22b8baf77" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:09.986140 16221 raft_consensus.cc:1275] T c0eab25ce4c64e018a041beeda2867db P 1ca2725d290e47d68b1a4b609c9816ac [term 20 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 13:22:09.987308 16265 leader_election.cc:304] T 23f82224f4564fea93572e8206bad1fe P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
I20260709 13:22:09.989660 16222 raft_consensus.cc:1275] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac [term 22 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 20 index: 16. Preceding OpId from leader: term: 22 index: 17. (index mismatch)
I20260709 13:22:09.989789 16221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5ad4eb281564936a59f41e387a47156" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:09.989125 16223 raft_consensus.cc:1275] T 8a878999c0024d2abe517484180290fc P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 13:22:09.992703 16146 raft_consensus.cc:2468] T b6f07842d3aa47858df5665874c86f3a P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 19.
I20260709 13:22:09.993636 16146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a959dbecc2648eb85d1cc550f5a3245" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:22:09.994021 16150 raft_consensus.cc:2393] T 84f2da98864b4425bf7c0055951fdd4b P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 16: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
I20260709 13:22:09.994204 16146 raft_consensus.cc:2468] T 5a959dbecc2648eb85d1cc550f5a3245 P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 19.
I20260709 13:22:09.987717 23347 raft_consensus.cc:493] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:09.910745 16147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57a744f5aaf944678a63469cbf514015" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:22:09.999155 16147 raft_consensus.cc:3060] T 57a744f5aaf944678a63469cbf514015 P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:22:10.002345 23369 log.cc:927] Time spent T 31969392ccdb43b491911a319e8bc7e1 P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.903s	user 0.001s	sys 0.000s
W20260709 13:22:10.005431 16263 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:22:09.991442 22873 raft_consensus.cc:2749] T 26354361e9b044cea1c2b86c4882fe92 P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:22:10.008339 22873 raft_consensus.cc:493] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:10.008752 16821 raft_consensus.cc:1275] T d2ab0ac8347641e6955e9bce7d90c1a2 P cb82cc7bd6db46d980c75237a1402860 [term 18 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 16 index: 14. Preceding OpId from leader: term: 18 index: 15. (index mismatch)
I20260709 13:22:10.008900 22873 raft_consensus.cc:515] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:10.011001 16115 leader_election.cc:304] T 1df93779209748b9860f6d893dd3de53 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:22:10.011976 16294 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40474 (request call id 1315) took 1070 ms. Trace:
W20260709 13:22:10.028240 16263 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:22:10.029922 16292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f2da98864b4425bf7c0055951fdd4b" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:10.030717 16292 raft_consensus.cc:2468] T 84f2da98864b4425bf7c0055951fdd4b P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 15.
I20260709 13:22:10.032328 16292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4f65cdf00b748bb963ee6e81ef43b79" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:10.033921 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806d7690c2b644959152074bb2cab0a7" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:22:10.034459 16265 leader_election.cc:304] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:22:10.037147 16297 raft_consensus.cc:1275] T 8a878999c0024d2abe517484180290fc P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 13:22:10.043088 16298 raft_consensus.cc:2393] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 20: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
W20260709 13:22:10.052750 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 352672us
I20260709 13:22:10.053556 16790 leader_election.cc:304] T ada5a004ba7c46819051e3ccec71ffdb P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 90f3da57796b488c9087a2f523adf045
W20260709 13:22:10.055016 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 331602us
I20260709 13:22:10.047857 16265 leader_election.cc:304] T 5a959dbecc2648eb85d1cc550f5a3245 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:22:10.059227 16116 leader_election.cc:304] T 84f2da98864b4425bf7c0055951fdd4b P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
I20260709 13:22:10.023705 16294 rpcz_store.cc:276] 0709 13:22:08.941599 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:22:08.941742 (+   143us) service_pool.cc:224] Handling call
0709 13:22:08.942106 (+   364us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:22:08.942490 (+   384us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:22:08.942523 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:22:08.943021 (+   498us) log.cc:844] Serialized 26 byte log entry
0709 13:22:08.987849 (+ 44828us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:22:08.987895 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:22:08.988069 (+   174us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:22:10.001566 (+1013497us) spinlock_profiling.cc:243] Waited 6.83 ms on lock 0x7b4400022b4c. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69ad8fadb1 00007f69ad998c85 00007f69ad9996a2 00007f69b070b1db 00007f69b0706aaf 00007f69b06fa86e 00007f69b06f9bb6 00007f69b06f97d8 00007f69b0703221 00007f69b07031d9 00007f69b0703169
0709 13:22:10.007304 (+  5738us) raft_consensus.cc:1685] finished
0709 13:22:10.007339 (+    35us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:22:10.011884 (+  4545us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":908,"raft.run_cpu_time_us":89,"raft.run_wall_time_us":89,"spinlock_wait_cycles":15713536,"thread_start_us":44141,"threads_started":1,"wal-append.queue_time_us":71976}
I20260709 13:22:10.061122 16821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b7d1ab519044eda2d5e04a07242f25" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:10.061218 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cc959fb7f704fba9009c0595467d074" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:10.061985 16146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d4ab870a05347868956aa8b841b2981" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:22:10.062867 16299 raft_consensus.cc:1240] T ecc1e363230043fe9fef390f0b979f20 P 90f3da57796b488c9087a2f523adf045 [term 22 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 21. Current term is 22. Ops: [21.13-21.13]
I20260709 13:22:10.063552 16191 leader_election.cc:400] T 66f75124215f4ca6bef831e174a792d8 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 17 election: Vote denied by peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741) with higher term. Message: Invalid argument: T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac for earlier term 17. Current term is 18.
I20260709 13:22:10.065383 16191 leader_election.cc:304] T 68a6838d82ef4cac83507f906139d90f P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 90f3da57796b488c9087a2f523adf045
W20260709 13:22:10.065868 16115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 54973us
I20260709 13:22:10.069868 16147 raft_consensus.cc:2468] T 57a744f5aaf944678a63469cbf514015 P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 17.
W20260709 13:22:10.072194 16115 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:22:10.054497 23389 raft_consensus.cc:2804] T ada5a004ba7c46819051e3ccec71ffdb P cb82cc7bd6db46d980c75237a1402860 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:22:10.070958 23161 consensus_queue.cc:1059] T 2794af5b99984efebdee564035841cb0 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.012s
I20260709 13:22:10.074232 23389 raft_consensus.cc:493] T ada5a004ba7c46819051e3ccec71ffdb P cb82cc7bd6db46d980c75237a1402860 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:22:10.074649 23161 raft_consensus.cc:3055] T 2794af5b99984efebdee564035841cb0 P 90f3da57796b488c9087a2f523adf045 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:22:10.075253 23161 raft_consensus.cc:740] T 2794af5b99984efebdee564035841cb0 P 90f3da57796b488c9087a2f523adf045 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:22:10.076237 23161 consensus_queue.cc:260] T 2794af5b99984efebdee564035841cb0 P 90f3da57796b488c9087a2f523adf045 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.15, Last appended by leader: 15, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:10.077512 23233 raft_consensus.cc:3055] T 8158bbd17b994d2eb96f560bd0f1b88d P 90f3da57796b488c9087a2f523adf045 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:22:10.077590 23161 raft_consensus.cc:3060] T 2794af5b99984efebdee564035841cb0 P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:22:10.078104 23233 raft_consensus.cc:740] T 8158bbd17b994d2eb96f560bd0f1b88d P 90f3da57796b488c9087a2f523adf045 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:22:10.079051 23233 consensus_queue.cc:260] T 8158bbd17b994d2eb96f560bd0f1b88d P 90f3da57796b488c9087a2f523adf045 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:10.081383 22917 raft_consensus.cc:2804] T 68a6838d82ef4cac83507f906139d90f P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:22:10.081997 22917 raft_consensus.cc:697] T 68a6838d82ef4cac83507f906139d90f P 1ca2725d290e47d68b1a4b609c9816ac [term 19 LEADER]: Becoming Leader. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:22:10.058677 23322 raft_consensus.cc:493] T c0eab25ce4c64e018a041beeda2867db P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:10.082980 22917 consensus_queue.cc:237] T 68a6838d82ef4cac83507f906139d90f P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:10.083504 16295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2794af5b99984efebdee564035841cb0" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:22:10.084216 23407 raft_consensus.cc:2804] T 5a959dbecc2648eb85d1cc550f5a3245 P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:22:10.086062 16790 leader_election.cc:304] T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:22:10.010636 23416 raft_consensus.cc:2749] T 23f82224f4564fea93572e8206bad1fe P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:22:09.966970 23086 raft_consensus.cc:515] T b87147731a68400e8cce9b879daa7c5d P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:09.996171 23347 raft_consensus.cc:515] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:10.074735 23389 raft_consensus.cc:3060] T ada5a004ba7c46819051e3ccec71ffdb P cb82cc7bd6db46d980c75237a1402860 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:22:10.098814 16224 raft_consensus.cc:1240] T 1df93779209748b9860f6d893dd3de53 P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 17. Current term is 18. Ops: []
I20260709 13:22:10.084679 23407 raft_consensus.cc:493] T 5a959dbecc2648eb85d1cc550f5a3245 P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:10.056584 23077 raft_consensus.cc:2804] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:22:10.115257 16818 raft_consensus.cc:1240] T 66f75124215f4ca6bef831e174a792d8 P cb82cc7bd6db46d980c75237a1402860 [term 18 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 17. Current term is 18. Ops: []
W20260709 13:22:10.133069 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 47260.4us
I20260709 13:22:10.149385 16191 leader_election.cc:304] T 08c74490c40243a88e62d1760b5cd814 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:22:10.133798 16265 leader_election.cc:304] T ecc1e363230043fe9fef390f0b979f20 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:22:10.136353 16825 raft_consensus.cc:1275] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860 [term 17 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 16 index: 14. Preceding OpId from leader: term: 17 index: 14. (term mismatch)
I20260709 13:22:10.150082 16266 leader_election.cc:304] T 434bff3082d141838068f323bdb0d1ed P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
I20260709 13:22:10.150481 16825 pending_rounds.cc:85] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860: Aborting all ops after (but not including) 13
I20260709 13:22:10.150869 16825 pending_rounds.cc:107] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860: Aborting uncommitted NO_OP operation due to leader change: 16.14
I20260709 13:22:10.151153 16825 raft_consensus.cc:2889] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860 [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:22:10.099265 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26354361e9b044cea1c2b86c4882fe92" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:10.071812 23252 raft_consensus.cc:2749] T 1df93779209748b9860f6d893dd3de53 P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:22:10.145814 16821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a378d7f796f94615b3b347119a9880fd" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:10.158645 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66ae176961b74813a58bc15c9c4c6f2a" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 27 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:10.095290 23233 raft_consensus.cc:3060] T 8158bbd17b994d2eb96f560bd0f1b88d P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:22:10.153795 16149 raft_consensus.cc:1240] T 8158bbd17b994d2eb96f560bd0f1b88d P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 17. Current term is 18. Ops: []
W20260709 13:22:10.159834 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 73836us
I20260709 13:22:10.154624 16145 raft_consensus.cc:2410] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac for term 26 because replica has last-logged OpId of term: 25 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 24 index: 15.
I20260709 13:22:10.155712 16146 raft_consensus.cc:1240] T b6f07842d3aa47858df5665874c86f3a P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 18. Current term is 19. Ops: []
W20260709 13:22:10.163203 16788 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:22:10.162660 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 103568us
I20260709 13:22:10.047595 23287 consensus_queue.cc:1059] T fffab26427e24d398e2695aad40ca134 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: INVALID_TERM, Last received: 17.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.100s
I20260709 13:22:10.054956 23306 consensus_queue.cc:1059] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: INVALID_TERM, Last received: 16.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.097s
I20260709 13:22:10.192457 16192 leader_election.cc:304] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860
I20260709 13:22:10.193573 16790 leader_election.cc:304] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 647a343a542d4a139fe106c19095129b
I20260709 13:22:10.192334 16266 leader_election.cc:304] T 57a744f5aaf944678a63469cbf514015 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:22:10.151211 23407 raft_consensus.cc:3060] T 5a959dbecc2648eb85d1cc550f5a3245 P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:22:10.061712 23311 raft_consensus.cc:515] T 18ad7e3af5504571a69d10c00750a01c P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } }
I20260709 13:22:10.151808 23077 raft_consensus.cc:493] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:10.142706 23257 raft_consensus.cc:493] T ee9ed67006144797b1de6ea8c2013a68 P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:22:10.143857 23204 consensus_queue.cc:1048] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.057s
I20260709 13:22:10.146502 23354 consensus_queue.cc:1059] T 434bff3082d141838068f323bdb0d1ed P 647a343a542d4a139fe106c19095129b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: INVALID_TERM, Last received: 18.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.054s
I20260709 13:22:10.146668 23328 consensus_queue.cc:1048] T 8a878999c0024d2abe517484180290fc P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.053s
I20260709 13:22:10.083949 23322 raft_consensus.cc:515] T c0eab25ce4c64e018a041beeda2867db P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:10.186903 23330 raft_consensus.cc:493] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:10.215952 16219 raft_consensus.cc:1275] T d2ab0ac8347641e6955e9bce7d90c1a2 P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 16 index: 14. Preceding OpId from leader: term: 18 index: 15. (index mismatch)
I20260709 13:22:10.201623 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74ca8140f35c4ae29e8a664f6bdfc829" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 22 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:10.202198 16149 raft_consensus.cc:1275] T 9e65676b21b748728ad39668cfe722ec P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 13:22:10.207142 16295 raft_consensus.cc:2410] T 2794af5b99984efebdee564035841cb0 P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b for term 18 because replica has last-logged OpId of term: 17 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 14.
W20260709 13:22:10.197809 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 64101us
I20260709 13:22:10.160652 23331 raft_consensus.cc:2804] T 84f2da98864b4425bf7c0055951fdd4b P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:22:10.191126 23173 raft_consensus.cc:2749] T 08c74490c40243a88e62d1760b5cd814 P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:22:10.220649 16265 leader_election.cc:304] T 5a96373bb0e14dce8a266b2fc6c58507 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:22:10.191533 23388 raft_consensus.cc:2749] T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:22:10.190186 22677 consensus_queue.cc:1048] T 8cc959fb7f704fba9009c0595467d074 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.045s
I20260709 13:22:10.224545 16218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2372981006d4b218215c28964dea623" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:10.235414 23351 raft_consensus.cc:2749] T 5a96373bb0e14dce8a266b2fc6c58507 P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:22:10.225153 16219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e4bfb9090c8460eb14fa7e0d818e7f1" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:22:10.227586 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74ca8140f35c4ae29e8a664f6bdfc829" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 22 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:10.236418 16224 raft_consensus.cc:2468] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 21.
I20260709 13:22:10.237588 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c2095a5f4ad49408dbc7f69c606120d" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
W20260709 13:22:10.238183 23425 log.cc:927] Time spent T 746aabf856ab401c9ef5e859371e7b67 P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 0.192s	user 0.001s	sys 0.000s
I20260709 13:22:10.238912 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02838c86531d4f98bf58d1c22b8baf77" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:22:10.219277 23292 raft_consensus.cc:2804] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [term 26 FOLLOWER]: Leader election won for term 26
I20260709 13:22:10.240003 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18ad7e3af5504571a69d10c00750a01c" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:22:10.228353 16221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66ae176961b74813a58bc15c9c4c6f2a" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 27 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:10.206666 23403 raft_consensus.cc:2749] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:22:10.206404 23382 consensus_queue.cc:1059] T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: INVALID_TERM, Last received: 19.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.084s
I20260709 13:22:10.230033 16223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5718c623ebee4d2bb770f0f9ddb2f164" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:10.230144 23287 consensus_queue.cc:1059] T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: INVALID_TERM, Last received: 18.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:22:10.160099 23385 consensus_queue.cc:1048] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.058s
I20260709 13:22:10.204375 23418 raft_consensus.cc:2749] T 434bff3082d141838068f323bdb0d1ed P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:22:10.231030 16825 raft_consensus.cc:1275] T 74ca8140f35c4ae29e8a664f6bdfc829 P cb82cc7bd6db46d980c75237a1402860 [term 21 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
I20260709 13:22:10.219408 23257 raft_consensus.cc:515] T ee9ed67006144797b1de6ea8c2013a68 P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:10.219421 23331 raft_consensus.cc:493] T 84f2da98864b4425bf7c0055951fdd4b P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:10.231452 23210 raft_consensus.cc:2804] T 57a744f5aaf944678a63469cbf514015 P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:22:10.203379 23415 raft_consensus.cc:2749] T ecc1e363230043fe9fef390f0b979f20 P 90f3da57796b488c9087a2f523adf045 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:22:10.231938 23417 consensus_queue.cc:1048] T d2ab0ac8347641e6955e9bce7d90c1a2 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:22:10.216506 23077 raft_consensus.cc:3060] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:22:10.241422 16219 raft_consensus.cc:2376] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 21. Re-sending same reply.
I20260709 13:22:10.235435 16265 leader_election.cc:400] T 8a878999c0024d2abe517484180290fc P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 17 election: Vote denied by peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943) with higher term. Message: Invalid argument: T 8a878999c0024d2abe517484180290fc P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 for earlier term 17. Current term is 18.
I20260709 13:22:10.265370 16265 leader_election.cc:403] T 8a878999c0024d2abe517484180290fc P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 17 election: Cancelling election due to peer responding with higher term
I20260709 13:22:10.243662 16149 raft_consensus.cc:2393] T 18ad7e3af5504571a69d10c00750a01c P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 17: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
I20260709 13:22:10.246999 16146 raft_consensus.cc:1275] T c0eab25ce4c64e018a041beeda2867db P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 13:22:10.246033 16223 raft_consensus.cc:2468] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 19.
I20260709 13:22:10.251394 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e65676b21b748728ad39668cfe722ec" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
W20260709 13:22:10.250537 16117 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 130540us
I20260709 13:22:10.269958 16143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26354361e9b044cea1c2b86c4882fe92" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:22:10.270421 16117 leader_election.cc:304] T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac
I20260709 13:22:10.271041 16143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2372981006d4b218215c28964dea623" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:22:10.271603 16143 raft_consensus.cc:2468] T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 23.
I20260709 13:22:10.272158 16117 leader_election.cc:304] T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860
I20260709 13:22:10.254261 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18ad7e3af5504571a69d10c00750a01c" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:22:10.197079 23371 consensus_queue.cc:1059] T 66f75124215f4ca6bef831e174a792d8 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.051s
I20260709 13:22:10.257977 16151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18ad7e3af5504571a69d10c00750a01c" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:22:10.232314 23311 leader_election.cc:290] T 18ad7e3af5504571a69d10c00750a01c P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 17 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:10.284216 16790 leader_election.cc:304] T 74ca8140f35c4ae29e8a664f6bdfc829 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 90f3da57796b488c9087a2f523adf045
W20260709 13:22:10.248294 22908 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.036s	user 0.002s	sys 0.021s
W20260709 13:22:10.290346 22908 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.098s	user 0.003s	sys 0.021s
I20260709 13:22:10.292548 16790 leader_election.cc:304] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:22:10.250006 23395 raft_consensus.cc:493] T 8c11074579b046d99e585c2818e18a2c P cb82cc7bd6db46d980c75237a1402860 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:10.298425 16299 raft_consensus.cc:2468] T 18ad7e3af5504571a69d10c00750a01c P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 17.
I20260709 13:22:10.262437 23331 raft_consensus.cc:3060] T 84f2da98864b4425bf7c0055951fdd4b P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:22:10.243446 23349 consensus_queue.cc:1059] T fffab26427e24d398e2695aad40ca134 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: INVALID_TERM, Last received: 19.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.045s
I20260709 13:22:10.263980 23292 raft_consensus.cc:697] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [term 26 LEADER]: Becoming Leader. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:22:10.289095 23403 raft_consensus.cc:3055] T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:22:10.243711 23166 consensus_queue.cc:1048] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.048s
I20260709 13:22:10.282537 23351 raft_consensus.cc:493] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:22:10.315356 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5718c623ebee4d2bb770f0f9ddb2f164" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:22:10.318086 16149 raft_consensus.cc:2468] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 19.
I20260709 13:22:10.220192 23330 raft_consensus.cc:515] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:10.294209 22873 leader_election.cc:290] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:10.301962 23287 raft_consensus.cc:2749] T 8a878999c0024d2abe517484180290fc P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Leader election lost for term 17. Reason: Vote denied by peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943) with higher term. Message: Invalid argument: T 8a878999c0024d2abe517484180290fc P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 for earlier term 17. Current term is 18.
I20260709 13:22:10.326646 23433 raft_consensus.cc:493] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b [term 23 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:22:10.334687 16825 raft_consensus.cc:1275] T 829b4ad8c5a54eababeea1181b960d4b P cb82cc7bd6db46d980c75237a1402860 [term 18 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
W20260709 13:22:10.335924 23431 log.cc:927] Time spent T 87ac0d1f7e8a419f9bcbc04aee0e067e P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 0.089s	user 0.000s	sys 0.000s
I20260709 13:22:10.336081 16115 leader_election.cc:304] T 2794af5b99984efebdee564035841cb0 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860
I20260709 13:22:10.313757 23403 raft_consensus.cc:740] T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:22:10.349596 16147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e65676b21b748728ad39668cfe722ec" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:22:10.377812 16147 raft_consensus.cc:2468] T 9e65676b21b748728ad39668cfe722ec P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 19.
I20260709 13:22:10.303596 23396 raft_consensus.cc:2749] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 13:22:10.297996 23388 raft_consensus.cc:493] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:22:10.298183 23383 raft_consensus.cc:2804] T 74ca8140f35c4ae29e8a664f6bdfc829 P cb82cc7bd6db46d980c75237a1402860 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:22:10.291329 23316 consensus_queue.cc:1048] T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.031s
W20260709 13:22:10.339731 23430 log.cc:927] Time spent T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860: Append to log took a long time: real 0.295s	user 0.003s	sys 0.007s
W20260709 13:22:10.383381 23369 log.cc:927] Time spent T 31969392ccdb43b491911a319e8bc7e1 P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.057s	user 0.001s	sys 0.000s
I20260709 13:22:10.310575 23437 raft_consensus.cc:2749] T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:22:10.304463 23395 raft_consensus.cc:515] T 8c11074579b046d99e585c2818e18a2c P cb82cc7bd6db46d980c75237a1402860 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:10.383203 23328 raft_consensus.cc:2749] T 2794af5b99984efebdee564035841cb0 P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:22:10.297818 22610 raft_consensus.cc:2804] T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:22:10.325006 23351 raft_consensus.cc:515] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:22:10.379051 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 213832us
I20260709 13:22:10.395613 23383 raft_consensus.cc:493] T 74ca8140f35c4ae29e8a664f6bdfc829 P cb82cc7bd6db46d980c75237a1402860 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:22:10.410212 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 174892us
W20260709 13:22:10.416002 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 123580us
I20260709 13:22:10.417900 16825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87147731a68400e8cce9b879daa7c5d" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:10.368142 23087 consensus_queue.cc:1048] T 829b4ad8c5a54eababeea1181b960d4b P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.043s
I20260709 13:22:10.416303 16116 leader_election.cc:304] T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:22:10.409673 23269 raft_consensus.cc:493] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:10.421993 16265 leader_election.cc:304] T 84f2da98864b4425bf7c0055951fdd4b P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:22:10.420660 16790 leader_election.cc:304] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:22:10.431691 16790 leader_election.cc:304] T 26354361e9b044cea1c2b86c4882fe92 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:22:10.419896 23383 raft_consensus.cc:3060] T 74ca8140f35c4ae29e8a664f6bdfc829 P cb82cc7bd6db46d980c75237a1402860 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:22:10.400265 23388 raft_consensus.cc:515] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:10.369011 23433 raft_consensus.cc:515] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:10.434391 16825 raft_consensus.cc:1240] T 66f75124215f4ca6bef831e174a792d8 P cb82cc7bd6db46d980c75237a1402860 [term 18 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 17. Current term is 18. Ops: [17.14-17.14]
I20260709 13:22:10.419286 23086 leader_election.cc:290] T b87147731a68400e8cce9b879daa7c5d P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:10.435617 16116 leader_election.cc:304] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:22:10.420133 22610 raft_consensus.cc:697] T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b [term 18 LEADER]: Becoming Leader. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:22:10.439749 16790 leader_election.cc:304] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac
I20260709 13:22:10.429086 23287 raft_consensus.cc:3060] T 84f2da98864b4425bf7c0055951fdd4b P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:22:10.430899 23166 raft_consensus.cc:2804] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:22:10.411759 23407 raft_consensus.cc:515] T 5a959dbecc2648eb85d1cc550f5a3245 P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:10.429229 23210 raft_consensus.cc:697] T 57a744f5aaf944678a63469cbf514015 P 90f3da57796b488c9087a2f523adf045 [term 17 LEADER]: Becoming Leader. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:22:10.472589 23371 consensus_queue.cc:1059] T 66f75124215f4ca6bef831e174a792d8 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: INVALID_TERM, Last received: 16.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:22:10.465157 23166 raft_consensus.cc:493] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:10.459738 23322 leader_election.cc:290] T c0eab25ce4c64e018a041beeda2867db P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:10.489193 23407 leader_election.cc:290] T 5a959dbecc2648eb85d1cc550f5a3245 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 20 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
W20260709 13:22:10.490309 23434 log.cc:927] Time spent T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860: Append to log took a long time: real 0.068s	user 0.000s	sys 0.000s
I20260709 13:22:10.490121 23441 raft_consensus.cc:2749] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:22:10.521699 16151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a959dbecc2648eb85d1cc550f5a3245" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
W20260709 13:22:10.491385 23428 log.cc:927] Time spent T 7c2095a5f4ad49408dbc7f69c606120d P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 0.441s	user 0.004s	sys 0.002s
I20260709 13:22:10.467439 23440 raft_consensus.cc:2749] T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:22:10.492125 23388 leader_election.cc:290] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:10.536782 23388 raft_consensus.cc:493] T 829b4ad8c5a54eababeea1181b960d4b P cb82cc7bd6db46d980c75237a1402860 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:10.537379 23388 raft_consensus.cc:515] T 829b4ad8c5a54eababeea1181b960d4b P cb82cc7bd6db46d980c75237a1402860 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:10.468529 23400 raft_consensus.cc:2749] T 26354361e9b044cea1c2b86c4882fe92 P cb82cc7bd6db46d980c75237a1402860 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:22:10.541651 23351 leader_election.cc:290] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:22:10.415045 23292 consensus_queue.cc:237] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 24.15, Last appended by leader: 15, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:10.472000 22610 consensus_queue.cc:237] T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:10.468737 23419 consensus_queue.cc:1048] T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.049s
I20260709 13:22:10.468894 23391 raft_consensus.cc:2804] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:22:10.432631 23396 raft_consensus.cc:493] T 3884f864a96d428785895e5463e5728f P cb82cc7bd6db46d980c75237a1402860 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:22:10.462683 23435 consensus_queue.cc:1048] T 746aabf856ab401c9ef5e859371e7b67 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.149s
I20260709 13:22:10.495527 16191 leader_election.cc:304] T fffab26427e24d398e2695aad40ca134 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 90f3da57796b488c9087a2f523adf045
I20260709 13:22:10.498649 16116 leader_election.cc:304] T 8cc959fb7f704fba9009c0595467d074 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
W20260709 13:22:10.441150 23384 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.735s	user 0.015s	sys 0.000s
I20260709 13:22:10.518208 16222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e4bfb9090c8460eb14fa7e0d818e7f1" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 22 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:10.522321 16151 raft_consensus.cc:3060] T 5a959dbecc2648eb85d1cc550f5a3245 P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:22:10.436870 23161 consensus_queue.cc:1059] T 7c2095a5f4ad49408dbc7f69c606120d P 90f3da57796b488c9087a2f523adf045 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: INVALID_TERM, Last received: 16.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.071s
I20260709 13:22:10.430694 23269 raft_consensus.cc:515] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:10.586129 23399 raft_consensus.cc:2804] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:22:10.587322 23441 raft_consensus.cc:2749] T 8cc959fb7f704fba9009c0595467d074 P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:22:10.588030 23269 leader_election.cc:290] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:10.588701 23269 raft_consensus.cc:2804] T fffab26427e24d398e2695aad40ca134 P 1ca2725d290e47d68b1a4b609c9816ac [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:22:10.589227 23269 raft_consensus.cc:697] T fffab26427e24d398e2695aad40ca134 P 1ca2725d290e47d68b1a4b609c9816ac [term 20 LEADER]: Becoming Leader. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:22:10.589530 16825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5d59407eb545b895d8d922e0dd69fd" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 23 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:10.590183 16825 raft_consensus.cc:2468] T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 22.
I20260709 13:22:10.519690 23415 consensus_queue.cc:1059] T 1df93779209748b9860f6d893dd3de53 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: INVALID_TERM, Last received: 17.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.010s
I20260709 13:22:10.591368 23382 raft_consensus.cc:3055] T 66f75124215f4ca6bef831e174a792d8 P 1ca2725d290e47d68b1a4b609c9816ac [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:22:10.591773 16192 leader_election.cc:304] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:22:10.592644 16291 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48674 (request call id 3465) took 1985 ms. Trace:
I20260709 13:22:10.519820 23349 consensus_queue.cc:1048] T d2ab0ac8347641e6955e9bce7d90c1a2 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.011s
I20260709 13:22:10.593014 23371 raft_consensus.cc:2804] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:22:10.593530 23371 raft_consensus.cc:493] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:10.593940 23371 raft_consensus.cc:3060] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:22:10.547814 16265 leader_election.cc:304] T 8d4ab870a05347868956aa8b841b2981 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:22:10.593012 16291 rpcz_store.cc:276] 0709 13:22:08.607467 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:22:08.611148 (+  3681us) service_pool.cc:224] Handling call
0709 13:22:10.592442 (+1981294us) spinlock_profiling.cc:243] Waited 511 ms on lock 0x7b54000cd854. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:22:10.592564 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1174727552}
W20260709 13:22:10.596025 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 48340.4us
I20260709 13:22:10.591776 23382 raft_consensus.cc:740] T 66f75124215f4ca6bef831e174a792d8 P 1ca2725d290e47d68b1a4b609c9816ac [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:22:10.597559 23382 consensus_queue.cc:260] T 66f75124215f4ca6bef831e174a792d8 P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.14, Last appended by leader: 14, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:10.599432 23395 leader_election.cc:290] T 8c11074579b046d99e585c2818e18a2c P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:10.600013 22645 raft_consensus.cc:3055] T 1df93779209748b9860f6d893dd3de53 P 90f3da57796b488c9087a2f523adf045 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:22:10.600417 22645 raft_consensus.cc:740] T 1df93779209748b9860f6d893dd3de53 P 90f3da57796b488c9087a2f523adf045 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:22:10.584115 23396 raft_consensus.cc:515] T 3884f864a96d428785895e5463e5728f P cb82cc7bd6db46d980c75237a1402860 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:10.601625 23161 raft_consensus.cc:3055] T 7c2095a5f4ad49408dbc7f69c606120d P 90f3da57796b488c9087a2f523adf045 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:22:10.602133 23161 raft_consensus.cc:740] T 7c2095a5f4ad49408dbc7f69c606120d P 90f3da57796b488c9087a2f523adf045 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:22:10.603075 23161 consensus_queue.cc:260] T 7c2095a5f4ad49408dbc7f69c606120d P 90f3da57796b488c9087a2f523adf045 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.8, Last appended by leader: 8, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:10.603749 23024 raft_consensus.cc:3055] T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:22:10.604154 23024 raft_consensus.cc:740] T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:22:10.604403 23161 raft_consensus.cc:3060] T 7c2095a5f4ad49408dbc7f69c606120d P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:22:10.604894 16825 raft_consensus.cc:1275] T 8c11074579b046d99e585c2818e18a2c P cb82cc7bd6db46d980c75237a1402860 [term 20 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 20 index: 16. (index mismatch)
I20260709 13:22:10.604987 23024 consensus_queue.cc:260] T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:10.606539 23024 raft_consensus.cc:3060] T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:22:10.517879 23166 raft_consensus.cc:3060] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:22:10.607703 16147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "829b4ad8c5a54eababeea1181b960d4b" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:22:10.609341 23410 consensus_queue.cc:1048] T 8c11074579b046d99e585c2818e18a2c P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 13:22:10.557803 16144 raft_consensus.cc:1240] T 1df93779209748b9860f6d893dd3de53 P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 17. Current term is 18. Ops: []
I20260709 13:22:10.579818 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a959dbecc2648eb85d1cc550f5a3245" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:22:10.612890 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e9b2bf7011f4ae688a79d7bb5a447b4" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:22:10.616297 16190 leader_election.cc:304] T 18ad7e3af5504571a69d10c00750a01c P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:22:10.517176 23233 consensus_queue.cc:1048] T c0eab25ce4c64e018a041beeda2867db P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.052s
W20260709 13:22:10.582592 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 87151.4us
I20260709 13:22:10.623100 16191 leader_election.cc:304] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 90f3da57796b488c9087a2f523adf045
W20260709 13:22:10.583611 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 85077.4us
I20260709 13:22:10.564221 23351 raft_consensus.cc:493] T 3884f864a96d428785895e5463e5728f P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:10.601290 22645 consensus_queue.cc:260] T 1df93779209748b9860f6d893dd3de53 P 90f3da57796b488c9087a2f523adf045 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:10.449084 23235 consensus_queue.cc:1059] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: INVALID_TERM, Last received: 18.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.066s
I20260709 13:22:10.618722 16790 leader_election.cc:304] T 9e65676b21b748728ad39668cfe722ec P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860; no voters: 90f3da57796b488c9087a2f523adf045
W20260709 13:22:10.584935 23443 log.cc:927] Time spent T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860: Append to log took a long time: real 0.074s	user 0.001s	sys 0.000s
I20260709 13:22:10.520170 23316 consensus_queue.cc:1059] T 02838c86531d4f98bf58d1c22b8baf77 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: INVALID_TERM, Last received: 19.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.058s
I20260709 13:22:10.564787 23391 raft_consensus.cc:493] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:10.590301 23269 consensus_queue.cc:237] T fffab26427e24d398e2695aad40ca134 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.12, Last appended by leader: 12, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:10.638453 16116 leader_election.cc:304] T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: cb82cc7bd6db46d980c75237a1402860
W20260709 13:22:10.622617 23384 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.923s	user 0.015s	sys 0.000s
I20260709 13:22:10.659243 23385 raft_consensus.cc:2804] T 18ad7e3af5504571a69d10c00750a01c P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:22:10.659689 23441 raft_consensus.cc:2804] T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:22:10.659910 23385 raft_consensus.cc:697] T 18ad7e3af5504571a69d10c00750a01c P 1ca2725d290e47d68b1a4b609c9816ac [term 17 LEADER]: Becoming Leader. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:22:10.661528 23435 consensus_queue.cc:1048] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 13:22:10.658764 23371 raft_consensus.cc:515] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:10.662847 23435 consensus_queue.cc:1048] T 8a878999c0024d2abe517484180290fc P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:22:10.660137 23441 raft_consensus.cc:493] T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Starting leader election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:22:10.664390 23441 raft_consensus.cc:3060] T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:22:10.668190 23173 raft_consensus.cc:493] T d2ab0ac8347641e6955e9bce7d90c1a2 P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:10.668735 23173 raft_consensus.cc:515] T d2ab0ac8347641e6955e9bce7d90c1a2 P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:10.669423 16816 raft_consensus.cc:1275] T af57996a8be84d398f0f023e8d08ef77 P cb82cc7bd6db46d980c75237a1402860 [term 20 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 20 index: 13. (term mismatch)
I20260709 13:22:10.670244 16816 pending_rounds.cc:85] T af57996a8be84d398f0f023e8d08ef77 P cb82cc7bd6db46d980c75237a1402860: Aborting all ops after (but not including) 12
I20260709 13:22:10.615852 23311 raft_consensus.cc:493] T ecc1e363230043fe9fef390f0b979f20 P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:22:10.670496 16816 pending_rounds.cc:107] T af57996a8be84d398f0f023e8d08ef77 P cb82cc7bd6db46d980c75237a1402860: Aborting uncommitted NO_OP operation due to leader change: 19.13
I20260709 13:22:10.647529 23433 leader_election.cc:290] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:10.670787 16816 raft_consensus.cc:2889] T af57996a8be84d398f0f023e8d08ef77 P cb82cc7bd6db46d980c75237a1402860 [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:22:10.672528 23173 leader_election.cc:290] T d2ab0ac8347641e6955e9bce7d90c1a2 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:10.671864 16821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ab0ac8347641e6955e9bce7d90c1a2" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 19 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:10.680855 16821 raft_consensus.cc:2468] T d2ab0ac8347641e6955e9bce7d90c1a2 P cb82cc7bd6db46d980c75237a1402860 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 18.
I20260709 13:22:10.672356 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0eab25ce4c64e018a041beeda2867db" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:10.682072 16192 leader_election.cc:304] T d2ab0ac8347641e6955e9bce7d90c1a2 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 
W20260709 13:22:10.671147 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 48139.6us
I20260709 13:22:10.683089 16217 raft_consensus.cc:2468] T c0eab25ce4c64e018a041beeda2867db P 1ca2725d290e47d68b1a4b609c9816ac [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 20.
I20260709 13:22:10.643362 23410 consensus_queue.cc:1048] T 8c11074579b046d99e585c2818e18a2c P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.040s
I20260709 13:22:10.683667 16191 leader_election.cc:304] T b6f07842d3aa47858df5665874c86f3a P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 90f3da57796b488c9087a2f523adf045
I20260709 13:22:10.684813 16191 leader_election.cc:304] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:22:10.684893 23410 raft_consensus.cc:2804] T d2ab0ac8347641e6955e9bce7d90c1a2 P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:22:10.685439 23410 raft_consensus.cc:493] T d2ab0ac8347641e6955e9bce7d90c1a2 P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:10.685652 23460 raft_consensus.cc:2804] T b6f07842d3aa47858df5665874c86f3a P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:22:10.686115 23173 raft_consensus.cc:2749] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:22:10.685954 16821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5d59407eb545b895d8d922e0dd69fd" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 23 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:22:10.671414 22908 raft_consensus.cc:2804] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:22:10.641348 23448 consensus_queue.cc:1048] T 9e65676b21b748728ad39668cfe722ec P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.046s
I20260709 13:22:10.687532 16821 raft_consensus.cc:3060] T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:22:10.687649 22908 raft_consensus.cc:697] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 1ca2725d290e47d68b1a4b609c9816ac [term 19 LEADER]: Becoming Leader. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:22:10.686645 23371 leader_election.cc:290] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 23 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:10.648382 23437 consensus_queue.cc:1048] T 829b4ad8c5a54eababeea1181b960d4b P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.006s
I20260709 13:22:10.688630 22908 consensus_queue.cc:237] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:10.690164 23316 raft_consensus.cc:3055] T 02838c86531d4f98bf58d1c22b8baf77 P 90f3da57796b488c9087a2f523adf045 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:22:10.690603 23316 raft_consensus.cc:740] T 02838c86531d4f98bf58d1c22b8baf77 P 90f3da57796b488c9087a2f523adf045 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:22:10.685973 16191 leader_election.cc:304] T 7c2095a5f4ad49408dbc7f69c606120d P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
W20260709 13:22:10.691460 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 63549.3us
I20260709 13:22:10.661165 23385 consensus_queue.cc:237] T 18ad7e3af5504571a69d10c00750a01c P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } }
I20260709 13:22:10.691607 23349 consensus_queue.cc:1059] T 1df93779209748b9860f6d893dd3de53 P 90f3da57796b488c9087a2f523adf045 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: INVALID_TERM, Last received: 17.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:22:10.674360 16221 raft_consensus.cc:1240] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac [term 23 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 22. Current term is 23. Ops: [22.17-22.17]
I20260709 13:22:10.682031 23210 consensus_queue.cc:237] T 57a744f5aaf944678a63469cbf514015 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:10.698745 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87ac0d1f7e8a419f9bcbc04aee0e067e" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
W20260709 13:22:10.701548 23458 log.cc:927] Time spent T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 0.076s	user 0.000s	sys 0.000s
I20260709 13:22:10.702100 22645 raft_consensus.cc:3060] T 1df93779209748b9860f6d893dd3de53 P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:22:10.704767 23372 raft_consensus.cc:2749] T 7c2095a5f4ad49408dbc7f69c606120d P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:22:10.704921 16147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87ac0d1f7e8a419f9bcbc04aee0e067e" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:22:10.705430 16191 leader_election.cc:304] T 02838c86531d4f98bf58d1c22b8baf77 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:22:10.706329 23388 leader_election.cc:290] T 829b4ad8c5a54eababeea1181b960d4b P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:22:10.707114 23366 raft_consensus.cc:2749] T 02838c86531d4f98bf58d1c22b8baf77 P 1ca2725d290e47d68b1a4b609c9816ac [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:22:10.697912 23349 raft_consensus.cc:493] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:22:10.709451 23349 raft_consensus.cc:515] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:10.711360 23199 raft_consensus.cc:2804] T 9e65676b21b748728ad39668cfe722ec P cb82cc7bd6db46d980c75237a1402860 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:22:10.694777 23391 raft_consensus.cc:3060] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:22:10.711952 23199 raft_consensus.cc:493] T 9e65676b21b748728ad39668cfe722ec P cb82cc7bd6db46d980c75237a1402860 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:10.712272 16191 leader_election.cc:304] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
I20260709 13:22:10.712456 23199 raft_consensus.cc:3060] T 9e65676b21b748728ad39668cfe722ec P cb82cc7bd6db46d980c75237a1402860 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:22:10.711490 23349 leader_election.cc:290] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:10.714473 16150 raft_consensus.cc:1240] T fffab26427e24d398e2695aad40ca134 P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 19. Current term is 20. Ops: [19.12-19.12]
I20260709 13:22:10.708774 22946 consensus_queue.cc:1048] T af57996a8be84d398f0f023e8d08ef77 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:22:10.732887 16147 raft_consensus.cc:1275] T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b [term 21 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 20 index: 13. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
I20260709 13:22:10.694614 16821 raft_consensus.cc:2468] T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 23.
I20260709 13:22:10.655038 16151 raft_consensus.cc:2468] T 5a959dbecc2648eb85d1cc550f5a3245 P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 20.
I20260709 13:22:10.725057 16265 leader_election.cc:304] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:22:10.734803 16192 leader_election.cc:304] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:22:10.742367 16298 raft_consensus.cc:1275] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 13:22:10.747395 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ab0ac8347641e6955e9bce7d90c1a2" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 19 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:10.738418 23460 raft_consensus.cc:697] T b6f07842d3aa47858df5665874c86f3a P 1ca2725d290e47d68b1a4b609c9816ac [term 19 LEADER]: Becoming Leader. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:22:10.749202 23460 consensus_queue.cc:237] T b6f07842d3aa47858df5665874c86f3a P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:10.750380 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee9ed67006144797b1de6ea8c2013a68" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:22:10.669353 23396 leader_election.cc:290] T 3884f864a96d428785895e5463e5728f P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:10.753796 23417 raft_consensus.cc:493] T 8c11074579b046d99e585c2818e18a2c P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:10.754436 23417 raft_consensus.cc:515] T 8c11074579b046d99e585c2818e18a2c P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:10.685869 23410 raft_consensus.cc:3060] T d2ab0ac8347641e6955e9bce7d90c1a2 P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:22:10.743680 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 97758.5us
I20260709 13:22:10.622874 22873 raft_consensus.cc:2749] T 8d4ab870a05347868956aa8b841b2981 P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:22:10.749835 16144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5d59407eb545b895d8d922e0dd69fd" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 23 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:22:10.757786 16147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e9b2bf7011f4ae688a79d7bb5a447b4" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
W20260709 13:22:10.796319 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 71317.7us
I20260709 13:22:10.810734 16265 leader_election.cc:304] T 5a959dbecc2648eb85d1cc550f5a3245 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac
W20260709 13:22:10.756363 23389 consensus_meta.cc:287] T ada5a004ba7c46819051e3ccec71ffdb P cb82cc7bd6db46d980c75237a1402860: Time spent flushing consensus metadata: real 0.603s	user 0.011s	sys 0.000s
W20260709 13:22:10.850306 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 55217.3us
I20260709 13:22:10.739354 23384 raft_consensus.cc:2749] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [term 26 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:22:10.850826 16147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3884f864a96d428785895e5463e5728f" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 15 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:22:10.867666 16116 leader_election.cc:304] T c0eab25ce4c64e018a041beeda2867db P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
I20260709 13:22:10.758960 23366 raft_consensus.cc:493] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 1ca2725d290e47d68b1a4b609c9816ac [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:10.885423 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5d59407eb545b895d8d922e0dd69fd" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 23 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:22:10.845952 23087 raft_consensus.cc:493] T 8158bbd17b994d2eb96f560bd0f1b88d P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:10.906369 23087 raft_consensus.cc:515] T 8158bbd17b994d2eb96f560bd0f1b88d P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:10.907838 23453 raft_consensus.cc:493] T 9c61a39fd4f84424b129c1a6f9ba404d P 647a343a542d4a139fe106c19095129b [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:10.846853 23437 raft_consensus.cc:493] T 18ad7e3af5504571a69d10c00750a01c P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:10.866513 23086 raft_consensus.cc:493] T 3884f864a96d428785895e5463e5728f P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:10.846400 23349 raft_consensus.cc:2749] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:22:10.930004 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 119386us
I20260709 13:22:10.670907 23311 raft_consensus.cc:515] T ecc1e363230043fe9fef390f0b979f20 P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:10.958889 16144 raft_consensus.cc:1275] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 20 index: 13. (term mismatch)
I20260709 13:22:10.966415 16144 pending_rounds.cc:85] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b: Aborting all ops after (but not including) 12
I20260709 13:22:10.845903 23448 raft_consensus.cc:493] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:22:10.868599 22917 raft_consensus.cc:2804] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:22:10.876016 23389 raft_consensus.cc:515] T ada5a004ba7c46819051e3ccec71ffdb P cb82cc7bd6db46d980c75237a1402860 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:10.720507 23351 raft_consensus.cc:515] T 3884f864a96d428785895e5463e5728f P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:10.908838 23435 raft_consensus.cc:2804] T c0eab25ce4c64e018a041beeda2867db P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:22:10.966813 16144 pending_rounds.cc:107] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b: Aborting uncommitted CHANGE_CONFIG_OP operation due to leader change: 18.13
I20260709 13:22:10.928155 23377 consensus_queue.cc:1048] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.105s
W20260709 13:22:11.092895 16189 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:22:11.094388 23409 raft_consensus.cc:493] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:11.094973 23409 raft_consensus.cc:515] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:11.088618 16144 raft_consensus.cc:2917] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Aborting config change with OpId 18.13: Aborted: Op aborted by new leader
I20260709 13:22:11.212003 16144 raft_consensus.cc:2889] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: CHANGE_CONFIG_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:22:10.999657 23435 raft_consensus.cc:493] T c0eab25ce4c64e018a041beeda2867db P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:10.927191 23366 raft_consensus.cc:515] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 1ca2725d290e47d68b1a4b609c9816ac [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:10.885792 23166 raft_consensus.cc:515] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:22:11.141475 23461 log.cc:927] Time spent T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860: Append to log took a long time: real 0.142s	user 0.000s	sys 0.000s
I20260709 13:22:11.141839 23449 consensus_queue.cc:1059] T fffab26427e24d398e2695aad40ca134 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: INVALID_TERM, Last received: 17.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.159s
I20260709 13:22:11.163255 23322 raft_consensus.cc:493] T af57996a8be84d398f0f023e8d08ef77 P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:11.164858 23087 leader_election.cc:290] T 8158bbd17b994d2eb96f560bd0f1b88d P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
W20260709 13:22:11.165868 23459 log.cc:927] Time spent T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.106s	user 0.000s	sys 0.000s
W20260709 13:22:11.165979 23456 log.cc:927] Time spent T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 0.193s	user 0.001s	sys 0.000s
I20260709 13:22:11.166752 23409 leader_election.cc:290] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
W20260709 13:22:11.169584 23441 consensus_meta.cc:287] T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b: Time spent flushing consensus metadata: real 0.505s	user 0.009s	sys 0.000s
I20260709 13:22:10.973320 23437 raft_consensus.cc:515] T 18ad7e3af5504571a69d10c00750a01c P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } }
I20260709 13:22:11.040284 23448 raft_consensus.cc:515] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:22:11.174202 23161 consensus_meta.cc:287] T 7c2095a5f4ad49408dbc7f69c606120d P 90f3da57796b488c9087a2f523adf045: Time spent flushing consensus metadata: real 0.569s	user 0.012s	sys 0.000s
I20260709 13:22:10.973973 23086 raft_consensus.cc:515] T 3884f864a96d428785895e5463e5728f P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:10.960484 23453 raft_consensus.cc:515] T 9c61a39fd4f84424b129c1a6f9ba404d P 647a343a542d4a139fe106c19095129b [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:11.175325 23417 leader_election.cc:290] T 8c11074579b046d99e585c2818e18a2c P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:11.301343 23366 leader_election.cc:290] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:11.303586 23448 leader_election.cc:290] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:11.305685 23437 leader_election.cc:290] T 18ad7e3af5504571a69d10c00750a01c P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:22:11.307511 23166 leader_election.cc:290] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 20 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:22:11.178231 23391 raft_consensus.cc:515] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:11.140334 23404 raft_consensus.cc:2804] T 5a959dbecc2648eb85d1cc550f5a3245 P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:22:11.308966 23404 raft_consensus.cc:697] T 5a959dbecc2648eb85d1cc550f5a3245 P 90f3da57796b488c9087a2f523adf045 [term 20 LEADER]: Becoming Leader. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:22:11.309831 23404 consensus_queue.cc:237] T 5a959dbecc2648eb85d1cc550f5a3245 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:11.310537 23354 raft_consensus.cc:3055] T 434bff3082d141838068f323bdb0d1ed P 647a343a542d4a139fe106c19095129b [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:22:11.310917 23354 raft_consensus.cc:740] T 434bff3082d141838068f323bdb0d1ed P 647a343a542d4a139fe106c19095129b [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:22:11.311734 23354 consensus_queue.cc:260] T 434bff3082d141838068f323bdb0d1ed P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 18.15, Last appended by leader: 15, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:11.191895 23257 leader_election.cc:290] T ee9ed67006144797b1de6ea8c2013a68 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:11.287079 23441 raft_consensus.cc:515] T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:22:11.317164 23411 log.cc:927] Time spent T e4dc6b0541ca431ebc757beac00521f1 P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 1.110s	user 0.002s	sys 0.010s
I20260709 13:22:11.317591 23441 leader_election.cc:290] T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 19 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:11.192587 23400 raft_consensus.cc:493] T 2794af5b99984efebdee564035841cb0 P cb82cc7bd6db46d980c75237a1402860 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:22:11.193013 22610 raft_consensus.cc:493] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:22:10.975791 23287 raft_consensus.cc:2749] T 84f2da98864b4425bf7c0055951fdd4b P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:22:10.867331 23316 consensus_queue.cc:260] T 02838c86531d4f98bf58d1c22b8baf77 P 90f3da57796b488c9087a2f523adf045 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:22:11.214761 23024 consensus_meta.cc:287] T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045: Time spent flushing consensus metadata: real 0.608s	user 0.005s	sys 0.005s
W20260709 13:22:11.214972 23199 consensus_meta.cc:287] T 9e65676b21b748728ad39668cfe722ec P cb82cc7bd6db46d980c75237a1402860: Time spent flushing consensus metadata: real 0.502s	user 0.008s	sys 0.000s
I20260709 13:22:11.185066 23351 leader_election.cc:290] T 3884f864a96d428785895e5463e5728f P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:11.182580 23292 raft_consensus.cc:493] T 8638ee363273426db1393dd81e5c7b72 P cb82cc7bd6db46d980c75237a1402860 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
W20260709 13:22:11.219703 23077 consensus_meta.cc:287] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045: Time spent flushing consensus metadata: real 0.924s	user 0.008s	sys 0.003s
I20260709 13:22:11.220000 23435 raft_consensus.cc:3060] T c0eab25ce4c64e018a041beeda2867db P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:22:11.230135 23329 raft_consensus.cc:493] T ecc1e363230043fe9fef390f0b979f20 P 647a343a542d4a139fe106c19095129b [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:11.291966 16147 raft_consensus.cc:2393] T 3884f864a96d428785895e5463e5728f P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 in current term 15: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
I20260709 13:22:11.296329 16149 raft_consensus.cc:1240] T 9c61a39fd4f84424b129c1a6f9ba404d P 647a343a542d4a139fe106c19095129b [term 21 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 20. Current term is 21. Ops: [20.11-20.11]
I20260709 13:22:11.297399 23453 leader_election.cc:290] T 9c61a39fd4f84424b129c1a6f9ba404d P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
W20260709 13:22:11.129252 23383 consensus_meta.cc:287] T 74ca8140f35c4ae29e8a664f6bdfc829 P cb82cc7bd6db46d980c75237a1402860: Time spent flushing consensus metadata: real 0.664s	user 0.010s	sys 0.000s
W20260709 13:22:11.293949 23331 consensus_meta.cc:287] T 84f2da98864b4425bf7c0055951fdd4b P 647a343a542d4a139fe106c19095129b: Time spent flushing consensus metadata: real 0.981s	user 0.011s	sys 0.000s
I20260709 13:22:11.314508 23391 leader_election.cc:290] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 24 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
W20260709 13:22:11.336490 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 468881us
I20260709 13:22:11.342485 16291 raft_consensus.cc:1275] T 84f2da98864b4425bf7c0055951fdd4b P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
W20260709 13:22:11.333990 22645 consensus_meta.cc:287] T 1df93779209748b9860f6d893dd3de53 P 90f3da57796b488c9087a2f523adf045: Time spent flushing consensus metadata: real 0.631s	user 0.008s	sys 0.003s
I20260709 13:22:11.274067 23322 raft_consensus.cc:515] T af57996a8be84d398f0f023e8d08ef77 P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:11.558033 16144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 23 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:22:11.560468 16142 raft_consensus.cc:1240] T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 18. Current term is 19. Ops: []
I20260709 13:22:11.562805 16817 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49892 (request call id 1060) took 1999 ms. Trace:
I20260709 13:22:11.574234 16817 rpcz_store.cc:276] 0709 13:22:09.563871 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:22:09.564031 (+   160us) service_pool.cc:224] Handling call
0709 13:22:09.580300 (+ 16269us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:22:09.647936 (+ 67636us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:22:09.647968 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:22:09.660524 (+ 12556us) log.cc:844] Serialized 26 byte log entry
0709 13:22:10.492669 (+832145us) spinlock_profiling.cc:243] Waited 1.2 ms on lock 0x7b180017c680. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b693ade1 00007f69b69475b1 00007f69ada54e07 00007f69b68afab9 00007f69ada7169b 00007f69ada6f93f 00007f69ada71c02 00007f69b06f920b 00007f69b06fed23 00007f69b06ff0d1 00007f69b0b1aa0f
0709 13:22:10.526440 (+ 33771us) raft_consensus.cc:1645] Marking committed up to 16
0709 13:22:10.526482 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:22:10.526549 (+    67us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:22:11.463706 (+937157us) raft_consensus.cc:1685] finished
0709 13:22:11.463737 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:22:11.562726 (+ 98989us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":643,"raft.queue_time_us":111345,"raft.run_cpu_time_us":132,"raft.run_wall_time_us":5597,"spinlock_wait_cycles":2750848,"thread_start_us":91259,"threads_started":2,"wal-append.queue_time_us":1101283}
I20260709 13:22:11.461133 23166 raft_consensus.cc:493] T 23f82224f4564fea93572e8206bad1fe P cb82cc7bd6db46d980c75237a1402860 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:22:11.568533 23354 raft_consensus.cc:3060] T 434bff3082d141838068f323bdb0d1ed P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:22:11.559003 23441 raft_consensus.cc:493] T 7c2095a5f4ad49408dbc7f69c606120d P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:22:11.572235 23383 raft_consensus.cc:515] T 74ca8140f35c4ae29e8a664f6bdfc829 P cb82cc7bd6db46d980c75237a1402860 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:11.333079 22610 raft_consensus.cc:515] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:11.559887 23311 leader_election.cc:290] T ecc1e363230043fe9fef390f0b979f20 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:11.573454 23316 raft_consensus.cc:3060] T 02838c86531d4f98bf58d1c22b8baf77 P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:22:11.604504 16189 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:22:11.468950 23371 raft_consensus.cc:493] T 1df93779209748b9860f6d893dd3de53 P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:11.341603 23199 raft_consensus.cc:515] T 9e65676b21b748728ad39668cfe722ec P cb82cc7bd6db46d980c75237a1402860 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:11.333223 23400 raft_consensus.cc:515] T 2794af5b99984efebdee564035841cb0 P cb82cc7bd6db46d980c75237a1402860 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:11.616667 16142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2372981006d4b218215c28964dea623" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:22:11.595659 23389 leader_election.cc:290] T ada5a004ba7c46819051e3ccec71ffdb P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 21 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
W20260709 13:22:11.620570 16192 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.838s	user 0.012s	sys 0.056s
I20260709 13:22:11.606153 23086 leader_election.cc:290] T 3884f864a96d428785895e5463e5728f P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:11.561080 23292 raft_consensus.cc:515] T 8638ee363273426db1393dd81e5c7b72 P cb82cc7bd6db46d980c75237a1402860 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:22:11.624346 16192 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.842s	user 0.012s	sys 0.056s
W20260709 13:22:11.625924 16192 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 891159us
I20260709 13:22:11.628017 16142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5718c623ebee4d2bb770f0f9ddb2f164" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
W20260709 13:22:11.618412 23269 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.836s	user 0.020s	sys 0.048s
I20260709 13:22:11.628598 16142 raft_consensus.cc:3060] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:22:11.578279 23329 raft_consensus.cc:515] T ecc1e363230043fe9fef390f0b979f20 P 647a343a542d4a139fe106c19095129b [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:11.633224 23292 leader_election.cc:290] T 8638ee363273426db1393dd81e5c7b72 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:11.634707 23403 consensus_queue.cc:260] T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.11, Last appended by leader: 11, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:11.600685 23166 raft_consensus.cc:515] T 23f82224f4564fea93572e8206bad1fe P cb82cc7bd6db46d980c75237a1402860 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:11.606786 23441 raft_consensus.cc:515] T 7c2095a5f4ad49408dbc7f69c606120d P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:11.566071 23077 raft_consensus.cc:515] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:11.579725 23331 raft_consensus.cc:515] T 84f2da98864b4425bf7c0055951fdd4b P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:22:11.629706 23269 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.890s	user 0.020s	sys 0.048s
I20260709 13:22:11.625895 23211 raft_consensus.cc:493] T 66f75124215f4ca6bef831e174a792d8 P cb82cc7bd6db46d980c75237a1402860 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:11.624496 23409 raft_consensus.cc:493] T 434bff3082d141838068f323bdb0d1ed P cb82cc7bd6db46d980c75237a1402860 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:22:11.669414 16148 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40830 (request call id 2103) took 1791 ms. Trace:
I20260709 13:22:11.618463 23371 raft_consensus.cc:515] T 1df93779209748b9860f6d893dd3de53 P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:11.673396 16144 raft_consensus.cc:2410] T ecc1e363230043fe9fef390f0b979f20 P 647a343a542d4a139fe106c19095129b [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 for term 23 because replica has last-logged OpId of term: 21 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 12.
I20260709 13:22:11.677428 16142 raft_consensus.cc:2468] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 20.
I20260709 13:22:11.670346 16148 rpcz_store.cc:276] 0709 13:22:09.854478 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:22:09.920047 (+ 65569us) service_pool.cc:224] Handling call
0709 13:22:09.920301 (+   254us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:22:09.955592 (+ 35291us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:22:09.955621 (+    29us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:22:09.955659 (+    38us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:22:09.955696 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:22:11.646054 (+1690358us) spinlock_profiling.cc:243] Waited 35.1 ms on lock 0x7b5400427a54. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:22:11.646114 (+    60us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:22:11.646279 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":80727296}
I20260709 13:22:11.679757 23409 raft_consensus.cc:515] T 434bff3082d141838068f323bdb0d1ed P cb82cc7bd6db46d980c75237a1402860 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:11.701172 23385 raft_consensus.cc:493] T 19baa7bc367844c288bca8f81439c413 P 1ca2725d290e47d68b1a4b609c9816ac [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:11.701747 23269 raft_consensus.cc:493] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:22:11.701867 23385 raft_consensus.cc:515] T 19baa7bc367844c288bca8f81439c413 P 1ca2725d290e47d68b1a4b609c9816ac [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:11.702481 23269 raft_consensus.cc:515] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:22:11.706552 23395 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.672s	user 0.016s	sys 0.050s
W20260709 13:22:11.707345 23395 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.707s	user 0.016s	sys 0.050s
W20260709 13:22:11.681319 16265 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.674s	user 0.059s	sys 0.064s
W20260709 13:22:11.708712 16150 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.734s	user 0.020s	sys 0.044s
W20260709 13:22:11.709331 16265 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.702s	user 0.059s	sys 0.064s
W20260709 13:22:11.709568 16150 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.750s	user 0.020s	sys 0.044s
W20260709 13:22:11.709795 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 736091us
W20260709 13:22:11.710645 16191 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.631s	user 0.007s	sys 0.015s
I20260709 13:22:11.703429 23472 raft_consensus.cc:493] T 5a959dbecc2648eb85d1cc550f5a3245 P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:22:11.711282 16191 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.632s	user 0.008s	sys 0.015s
I20260709 13:22:11.711796 23472 raft_consensus.cc:515] T 5a959dbecc2648eb85d1cc550f5a3245 P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:22:11.679862 16225 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.727s	user 0.017s	sys 0.046s
W20260709 13:22:11.715152 16225 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.762s	user 0.017s	sys 0.046s
I20260709 13:22:11.705809 23409 leader_election.cc:290] T 434bff3082d141838068f323bdb0d1ed P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:11.722327 16148 raft_consensus.cc:1240] T 08c74490c40243a88e62d1760b5cd814 P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 18. Current term is 19. Ops: [18.11-18.11]
I20260709 13:22:11.723784 16142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3884f864a96d428785895e5463e5728f" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 16 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:22:11.728201 16222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c11074579b046d99e585c2818e18a2c" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:11.730422 16222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7ca61592e848ea9b0fb81a2ce33cdc" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:11.731256 16222 raft_consensus.cc:2393] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 1ca2725d290e47d68b1a4b609c9816ac [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 20: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
I20260709 13:22:11.705158 23385 leader_election.cc:290] T 19baa7bc367844c288bca8f81439c413 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:11.714799 23472 leader_election.cc:290] T 5a959dbecc2648eb85d1cc550f5a3245 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:11.733143 16221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c11074579b046d99e585c2818e18a2c" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:11.734375 16221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 23 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:11.735509 16221 raft_consensus.cc:2410] T ecc1e363230043fe9fef390f0b979f20 P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 for term 23 because replica has last-logged OpId of term: 21 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 12.
I20260709 13:22:11.736282 16221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5718c623ebee4d2bb770f0f9ddb2f164" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:22:11.736809 16221 raft_consensus.cc:3060] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:22:11.679589 23211 raft_consensus.cc:515] T 66f75124215f4ca6bef831e174a792d8 P cb82cc7bd6db46d980c75237a1402860 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:11.747786 16225 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57600 (request call id 2112) took 1799 ms. Trace:
I20260709 13:22:11.748867 16225 rpcz_store.cc:276] 0709 13:22:09.948077 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:22:09.955324 (+  7247us) service_pool.cc:224] Handling call
0709 13:22:09.955603 (+   279us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:22:09.955774 (+   171us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:22:09.956167 (+   393us) log.cc:844] Serialized 14 byte log entry
0709 13:22:10.952523 (+996356us) spinlock_profiling.cc:243] Waited 5.12 ms on lock 0x7b18004a9fe0. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b693ade1 00007f69b69475b1 00007f69ada54e07 00007f69b68afab9 00007f69ada7169b 00007f69ada6f93f 00007f69ada71c02 00007f69b06f920b 00007f69b06fed23 00007f69b06ff23a 00007f69b0b69a2a
0709 13:22:11.747294 (+794771us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:22:11.747372 (+    78us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:22:11.747491 (+   119us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:22:11.747602 (+   111us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:22:11.747727 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":26168,"prepare.queue_time_us":455,"prepare.run_cpu_time_us":49,"prepare.run_wall_time_us":51,"raft.queue_time_us":32818,"spinlock_wait_cycles":11771648,"thread_start_us":763049,"threads_started":1}
W20260709 13:22:11.742470 23410 consensus_meta.cc:287] T d2ab0ac8347641e6955e9bce7d90c1a2 P 1ca2725d290e47d68b1a4b609c9816ac: Time spent flushing consensus metadata: real 0.876s	user 0.011s	sys 0.000s
I20260709 13:22:11.751973 16222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "829b4ad8c5a54eababeea1181b960d4b" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:11.708889 23269 leader_election.cc:290] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:11.756345 16225 raft_consensus.cc:1275] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [term 26 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 24 index: 15. Preceding OpId from leader: term: 26 index: 16. (index mismatch)
I20260709 13:22:11.761508 23410 raft_consensus.cc:515] T d2ab0ac8347641e6955e9bce7d90c1a2 P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:11.762538 23306 raft_consensus.cc:3055] T fffab26427e24d398e2695aad40ca134 P 90f3da57796b488c9087a2f523adf045 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:22:11.763010 23306 raft_consensus.cc:740] T fffab26427e24d398e2695aad40ca134 P 90f3da57796b488c9087a2f523adf045 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:22:11.750846 23024 consensus_queue.cc:1059] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: INVALID_TERM, Last received: 16.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.015s
I20260709 13:22:11.764192 23410 leader_election.cc:290] T d2ab0ac8347641e6955e9bce7d90c1a2 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 19 election: Requested vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:11.764992 23199 leader_election.cc:290] T 9e65676b21b748728ad39668cfe722ec P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 20 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:11.765759 16224 raft_consensus.cc:1240] T d2ab0ac8347641e6955e9bce7d90c1a2 P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 18. Current term is 19. Ops: [18.15-18.15]
I20260709 13:22:11.766247 16291 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48684 (request call id 3213) took 1025 ms. Trace:
I20260709 13:22:11.766661 16291 rpcz_store.cc:276] 0709 13:22:10.741177 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:22:10.741335 (+   158us) service_pool.cc:224] Handling call
0709 13:22:10.846304 (+104969us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:22:11.569880 (+723576us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:22:11.765991 (+196111us) spinlock_profiling.cc:243] Waited 496 ms on lock 0x7b540024de54. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:22:11.766183 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1140272896}
I20260709 13:22:11.767505 16224 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57600 (request call id 2119) took 1072 ms. Trace:
I20260709 13:22:11.768358 16224 rpcz_store.cc:276] 0709 13:22:10.695257 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:22:10.695982 (+   725us) service_pool.cc:224] Handling call
0709 13:22:10.696313 (+   331us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:22:11.766202 (+1069889us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:22:11.766899 (+   697us) spinlock_profiling.cc:243] Waited 1.07 s on lock 0x7b5400245754. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:22:11.767099 (+   200us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2458905344}
W20260709 13:22:11.767815 23414 log.cc:927] Time spent T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 1.721s	user 0.004s	sys 0.015s
I20260709 13:22:11.763954 23306 consensus_queue.cc:260] T fffab26427e24d398e2695aad40ca134 P 90f3da57796b488c9087a2f523adf045 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:11.781478 16222 raft_consensus.cc:1240] T 08c74490c40243a88e62d1760b5cd814 P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 18. Current term is 19. Ops: []
I20260709 13:22:11.776491 23416 consensus_queue.cc:1059] T d2ab0ac8347641e6955e9bce7d90c1a2 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: INVALID_TERM, Last received: 16.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.006s
I20260709 13:22:11.776762 23448 raft_consensus.cc:493] T 31969392ccdb43b491911a319e8bc7e1 P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:22:11.797569 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2372981006d4b218215c28964dea623" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:22:11.792476 23306 raft_consensus.cc:3060] T fffab26427e24d398e2695aad40ca134 P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:22:11.791667 23383 leader_election.cc:290] T 74ca8140f35c4ae29e8a664f6bdfc829 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 22 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
W20260709 13:22:11.794219 22908 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.731s	user 0.004s	sys 0.012s
W20260709 13:22:11.796334 23464 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.633s	user 0.025s	sys 0.024s
I20260709 13:22:11.800891 23024 raft_consensus.cc:493] T b87147731a68400e8cce9b879daa7c5d P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:22:11.796684 23322 leader_election.cc:290] T af57996a8be84d398f0f023e8d08ef77 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:11.805437 16221 raft_consensus.cc:2468] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 20.
I20260709 13:22:11.813324 16222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada5a004ba7c46819051e3ccec71ffdb" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
W20260709 13:22:11.843429 16219 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.520s	user 0.005s	sys 0.009s
I20260709 13:22:11.862613 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8638ee363273426db1393dd81e5c7b72" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:11.796597 23448 raft_consensus.cc:515] T 31969392ccdb43b491911a319e8bc7e1 P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:11.812358 22610 leader_election.cc:290] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
W20260709 13:22:11.816927 23464 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.654s	user 0.026s	sys 0.024s
W20260709 13:22:11.895634 16219 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.577s	user 0.008s	sys 0.009s
W20260709 13:22:11.815389 22908 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.789s	user 0.004s	sys 0.012s
W20260709 13:22:11.843895 22917 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.785s	user 0.003s	sys 0.020s
I20260709 13:22:11.906868 16221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74ca8140f35c4ae29e8a664f6bdfc829" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 22 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
W20260709 13:22:11.871557 23460 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.692s	user 0.017s	sys 0.041s
I20260709 13:22:11.844656 23400 leader_election.cc:290] T 2794af5b99984efebdee564035841cb0 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:11.911806 16790 leader_election.cc:304] T 829b4ad8c5a54eababeea1181b960d4b P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:22:11.828998 23473 raft_consensus.cc:493] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:22:11.876309 23166 leader_election.cc:290] T 23f82224f4564fea93572e8206bad1fe P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
W20260709 13:22:11.947973 23479 log.cc:927] Time spent T 18ad7e3af5504571a69d10c00750a01c P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
I20260709 13:22:11.895646 23383 raft_consensus.cc:493] T 19baa7bc367844c288bca8f81439c413 P cb82cc7bd6db46d980c75237a1402860 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:11.950961 23383 raft_consensus.cc:515] T 19baa7bc367844c288bca8f81439c413 P cb82cc7bd6db46d980c75237a1402860 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:11.898921 23371 leader_election.cc:290] T 1df93779209748b9860f6d893dd3de53 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:11.961040 23448 leader_election.cc:290] T 31969392ccdb43b491911a319e8bc7e1 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:11.962908 23329 leader_election.cc:290] T ecc1e363230043fe9fef390f0b979f20 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:11.870172 23211 leader_election.cc:290] T 66f75124215f4ca6bef831e174a792d8 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:22:11.809100 23417 consensus_queue.cc:1059] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: INVALID_TERM, Last received: 18.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.097s
W20260709 13:22:11.921312 23468 log.cc:927] Time spent T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 0.232s	user 0.000s	sys 0.001s
I20260709 13:22:11.869184 23024 raft_consensus.cc:515] T b87147731a68400e8cce9b879daa7c5d P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:22:11.913416 23460 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.734s	user 0.018s	sys 0.043s
I20260709 13:22:11.939327 23473 raft_consensus.cc:515] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:11.939864 16150 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40844 (request call id 2997) took 1201 ms. Trace:
W20260709 13:22:11.912978 22917 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.875s	user 0.003s	sys 0.020s
I20260709 13:22:11.897346 23486 raft_consensus.cc:493] T e4dc6b0541ca431ebc757beac00521f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:22:11.961985 23408 raft_consensus.cc:493] T c9944d5e4b8c4109af9955c4f08c9095 P cb82cc7bd6db46d980c75237a1402860 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
W20260709 13:22:11.997298 16226 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57600 (request call id 2110) took 2109 ms (client timeout 1746 ms). Trace:
I20260709 13:22:11.999632 16040 catalog_manager.cc:5697] T af57996a8be84d398f0f023e8d08ef77 P cb82cc7bd6db46d980c75237a1402860 reported cstate change: term changed from 15 to 18, leader changed from cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) to 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130). New cstate: current_term: 18 leader_uuid: "1ca2725d290e47d68b1a4b609c9816ac" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:22:12.008468 23287 raft_consensus.cc:3055] T d2ab0ac8347641e6955e9bce7d90c1a2 P 90f3da57796b488c9087a2f523adf045 [term 18 LEADER]: Stepping down as leader of term 18
W20260709 13:22:12.011742 16226 rpcz_store.cc:269] 0709 13:22:09.876039 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:22:09.880675 (+  4636us) service_pool.cc:224] Handling call
0709 13:22:11.985043 (+2104368us) spinlock_profiling.cc:243] Waited 9.43 ms on lock 0x7b54002b3c54. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:22:11.985159 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":21679872}
W20260709 13:22:12.020267 16218 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.747s	user 0.023s	sys 0.022s
W20260709 13:22:12.020444 16789 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.505s	user 0.015s	sys 0.035s
I20260709 13:22:11.994160 16150 rpcz_store.cc:276] 0709 13:22:10.738542 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:22:10.760939 (+ 22397us) service_pool.cc:224] Handling call
0709 13:22:10.761455 (+   516us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:22:10.783523 (+ 22068us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:22:10.783555 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:22:10.849642 (+ 66087us) log.cc:844] Serialized 26 byte log entry
0709 13:22:11.710520 (+860878us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:22:11.735142 (+ 24622us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:22:11.735284 (+   142us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:22:11.939594 (+204310us) raft_consensus.cc:1685] finished
0709 13:22:11.939616 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:22:11.939798 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":22212,"raft.queue_time_us":32626,"raft.run_cpu_time_us":188,"raft.run_wall_time_us":25549,"thread_start_us":750705,"threads_started":1,"wal-append.queue_time_us":1026141}
W20260709 13:22:12.022584 16789 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.507s	user 0.015s	sys 0.035s
W20260709 13:22:12.022281 16218 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.792s	user 0.023s	sys 0.022s
W20260709 13:22:12.013988 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 102282us
I20260709 13:22:11.998147 23480 raft_consensus.cc:2749] T 829b4ad8c5a54eababeea1181b960d4b P cb82cc7bd6db46d980c75237a1402860 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:22:11.998679 22917 raft_consensus.cc:697] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac [term 23 LEADER]: Becoming Leader. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:22:12.039342 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66f75124215f4ca6bef831e174a792d8" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:11.989041 23371 consensus_queue.cc:1048] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.039s
I20260709 13:22:12.018173 23287 raft_consensus.cc:740] T d2ab0ac8347641e6955e9bce7d90c1a2 P 90f3da57796b488c9087a2f523adf045 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:22:11.988150 22908 consensus_queue.cc:1048] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.044s
I20260709 13:22:12.070693 16821 raft_consensus.cc:1275] T 806d7690c2b644959152074bb2cab0a7 P cb82cc7bd6db46d980c75237a1402860 [term 20 LEARNER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 13:22:12.034613 16790 leader_election.cc:304] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:22:12.085443 16816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee9ed67006144797b1de6ea8c2013a68" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
W20260709 13:22:12.090459 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 55962us
I20260709 13:22:12.093144 16822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3884f864a96d428785895e5463e5728f" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 16 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:12.093822 16822 raft_consensus.cc:2468] T 3884f864a96d428785895e5463e5728f P cb82cc7bd6db46d980c75237a1402860 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 14.
I20260709 13:22:12.091125 16818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a9de347c711477e87c582d003f36925" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:12.094821 16822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e4bfb9090c8460eb14fa7e0d818e7f1" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 22 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:12.096758 16822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af57996a8be84d398f0f023e8d08ef77" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:12.099321 16818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e9b2bf7011f4ae688a79d7bb5a447b4" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:12.021631 23486 raft_consensus.cc:515] T e4dc6b0541ca431ebc757beac00521f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:12.100155 16818 raft_consensus.cc:2468] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 18.
I20260709 13:22:12.101657 16819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b7d1ab519044eda2d5e04a07242f25" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
W20260709 13:22:12.103654 23204 raft_consensus.cc:670] T 806d7690c2b644959152074bb2cab0a7 P cb82cc7bd6db46d980c75237a1402860: failed to trigger leader election: Illegal state: only voting members can start elections: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } } [suppressed 3 similar messages]
I20260709 13:22:12.098498 23389 raft_consensus.cc:2804] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 [term 20 FOLLOWER]: Leader election won for term 20
W20260709 13:22:12.106014 16220 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.524s	user 0.044s	sys 0.070s
I20260709 13:22:12.040381 23166 consensus_queue.cc:1048] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.055s
W20260709 13:22:12.112583 16220 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.530s	user 0.044s	sys 0.070s
I20260709 13:22:12.079756 23077 leader_election.cc:290] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 23 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:12.114389 16816 raft_consensus.cc:1240] T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860 [term 23 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 22. Current term is 23. Ops: []
I20260709 13:22:12.111610 16822 raft_consensus.cc:1240] T ada5a004ba7c46819051e3ccec71ffdb P cb82cc7bd6db46d980c75237a1402860 [term 21 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 20. Current term is 21. Ops: []
I20260709 13:22:12.113220 16221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 23 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:12.114588 16818 raft_consensus.cc:1240] T 74ca8140f35c4ae29e8a664f6bdfc829 P cb82cc7bd6db46d980c75237a1402860 [term 22 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 21. Current term is 22. Ops: [21.15-21.15]
I20260709 13:22:12.131274 16221 raft_consensus.cc:2468] T 19baa7bc367844c288bca8f81439c413 P 1ca2725d290e47d68b1a4b609c9816ac [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 22.
W20260709 13:22:12.113292 23321 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.882s	user 0.025s	sys 0.059s
I20260709 13:22:12.133224 23389 raft_consensus.cc:697] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 [term 20 LEADER]: Becoming Leader. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:22:12.037869 23408 raft_consensus.cc:515] T c9944d5e4b8c4109af9955c4f08c9095 P cb82cc7bd6db46d980c75237a1402860 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:22:12.189320 16189 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:22:12.095366 23480 raft_consensus.cc:493] T 57a744f5aaf944678a63469cbf514015 P cb82cc7bd6db46d980c75237a1402860 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:12.130007 16819 raft_consensus.cc:1240] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 [term 24 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 23. Current term is 24. Ops: []
I20260709 13:22:12.255368 23373 raft_consensus.cc:493] T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:22:12.198907 23488 raft_consensus.cc:493] T c2f5f89186204242bf377f7c42b2e2da P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:12.286620 16816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7ca61592e848ea9b0fb81a2ce33cdc" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:12.275363 23373 raft_consensus.cc:515] T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:12.193441 23061 raft_consensus.cc:493] T 08c74490c40243a88e62d1760b5cd814 P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:22:12.191114 23321 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.961s	user 0.025s	sys 0.060s
W20260709 13:22:12.209592 23307 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 2.379s	user 0.052s	sys 0.076s
I20260709 13:22:12.197502 16822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:12.111666 23287 consensus_queue.cc:260] T d2ab0ac8347641e6955e9bce7d90c1a2 P 90f3da57796b488c9087a2f523adf045 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.15, Last appended by leader: 15, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:12.325913 16822 raft_consensus.cc:2393] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 22: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
W20260709 13:22:12.297004 23354 consensus_meta.cc:287] T 434bff3082d141838068f323bdb0d1ed P 647a343a542d4a139fe106c19095129b: Time spent flushing consensus metadata: real 0.693s	user 0.008s	sys 0.005s
I20260709 13:22:12.332937 16822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 23 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:12.336077 16822 raft_consensus.cc:2468] T 19baa7bc367844c288bca8f81439c413 P cb82cc7bd6db46d980c75237a1402860 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 22.
W20260709 13:22:12.324772 23307 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 2.494s	user 0.054s	sys 0.086s
W20260709 13:22:12.333534 23404 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.692s	user 0.002s	sys 0.032s
I20260709 13:22:12.323385 23473 leader_election.cc:290] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:12.275782 23488 raft_consensus.cc:515] T c2f5f89186204242bf377f7c42b2e2da P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:12.347602 16822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8638ee363273426db1393dd81e5c7b72" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:12.351614 16822 raft_consensus.cc:2468] T 8638ee363273426db1393dd81e5c7b72 P cb82cc7bd6db46d980c75237a1402860 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 19.
I20260709 13:22:12.335445 23024 leader_election.cc:290] T b87147731a68400e8cce9b879daa7c5d P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:12.199322 22917 consensus_queue.cc:237] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 20.16, Last appended by leader: 16, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:22:12.372797 16790 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.14.177.129:36741 timed out after 1.588s (SENT) [suppressed 2 similar messages]
W20260709 13:22:12.356724 23404 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.716s	user 0.003s	sys 0.032s
I20260709 13:22:12.326750 23061 raft_consensus.cc:515] T 08c74490c40243a88e62d1760b5cd814 P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:22:12.378501 23490 log.cc:927] Time spent T b6f07842d3aa47858df5665874c86f3a P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 0.187s	user 0.000s	sys 0.001s
I20260709 13:22:12.403939 16192 leader_election.cc:304] T ee9ed67006144797b1de6ea8c2013a68 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
I20260709 13:22:12.405977 23473 consensus_queue.cc:1048] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:22:12.406500 16192 leader_election.cc:304] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:22:12.387880 23486 leader_election.cc:290] T e4dc6b0541ca431ebc757beac00521f1 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:12.289642 23480 raft_consensus.cc:515] T 57a744f5aaf944678a63469cbf514015 P cb82cc7bd6db46d980c75237a1402860 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:22:12.389007 23306 consensus_meta.cc:287] T fffab26427e24d398e2695aad40ca134 P 90f3da57796b488c9087a2f523adf045: Time spent flushing consensus metadata: real 0.583s	user 0.011s	sys 0.000s
I20260709 13:22:12.350396 23287 raft_consensus.cc:3060] T d2ab0ac8347641e6955e9bce7d90c1a2 P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:22:12.396471 23408 leader_election.cc:290] T c9944d5e4b8c4109af9955c4f08c9095 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:12.401374 16149 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40844 (request call id 3000) took 1653 ms. Trace:
W20260709 13:22:12.379484 23492 log.cc:927] Time spent T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 0.124s	user 0.001s	sys 0.000s
W20260709 13:22:12.409678 16791 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:22:12.378485 23493 log.cc:927] Time spent T 8a878999c0024d2abe517484180290fc P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 0.278s	user 0.000s	sys 0.000s
W20260709 13:22:12.387015 23435 consensus_meta.cc:287] T c0eab25ce4c64e018a041beeda2867db P 647a343a542d4a139fe106c19095129b: Time spent flushing consensus metadata: real 0.814s	user 0.008s	sys 0.005s
I20260709 13:22:12.408032 23473 raft_consensus.cc:2749] T ee9ed67006144797b1de6ea8c2013a68 P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:22:12.336935 23389 consensus_queue.cc:237] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 18.15, Last appended by leader: 15, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:12.420070 16822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ab0ac8347641e6955e9bce7d90c1a2" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 19 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:22:12.426046 16149 rpcz_store.cc:276] 0709 13:22:10.748421 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:22:10.929468 (+181047us) service_pool.cc:224] Handling call
0709 13:22:10.997882 (+ 68414us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:22:11.564172 (+566290us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:22:12.401016 (+836844us) spinlock_profiling.cc:243] Waited 298 ms on lock 0x7b54000d0f54. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:22:12.401296 (+   280us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":685835008}
I20260709 13:22:12.420559 23311 raft_consensus.cc:2804] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:22:12.408882 23460 consensus_queue.cc:1059] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.015s
I20260709 13:22:12.437950 23373 leader_election.cc:290] T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:12.439580 23061 leader_election.cc:290] T 08c74490c40243a88e62d1760b5cd814 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:12.436826 23311 raft_consensus.cc:493] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Starting leader election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:22:12.440708 23311 raft_consensus.cc:3060] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:22:12.441129 23488 leader_election.cc:290] T c2f5f89186204242bf377f7c42b2e2da P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:12.448822 16817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b7d1ab519044eda2d5e04a07242f25" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:12.450388 16219 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57604 (request call id 3420) took 1603 ms. Trace:
I20260709 13:22:12.451610 16219 rpcz_store.cc:276] 0709 13:22:10.847515 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:22:10.848247 (+   732us) service_pool.cc:224] Handling call
0709 13:22:10.848659 (+   412us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:22:11.222409 (+373750us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:22:11.222444 (+    35us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:22:11.268105 (+ 45661us) log.cc:844] Serialized 52 byte log entry
0709 13:22:11.909274 (+641169us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:22:12.328688 (+419414us) log.cc:844] Serialized 14 byte log entry
0709 13:22:12.336564 (+  7876us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:22:12.336702 (+   138us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:22:12.449605 (+112903us) raft_consensus.cc:1685] finished
0709 13:22:12.449726 (+   121us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:22:12.450310 (+   584us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":284425,"prepare.run_cpu_time_us":28,"prepare.run_wall_time_us":29,"raft.queue_time_us":192779,"raft.run_cpu_time_us":281,"raft.run_wall_time_us":43210,"thread_start_us":1297983,"threads_started":3,"wal-append.queue_time_us":755549}
I20260709 13:22:12.456892 16218 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57600 (request call id 2122) took 1747 ms. Trace:
I20260709 13:22:12.457432 16218 rpcz_store.cc:276] 0709 13:22:10.712371 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:22:10.715149 (+  2778us) service_pool.cc:224] Handling call
0709 13:22:10.715505 (+   356us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:22:11.219554 (+504049us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:22:11.219599 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:22:11.220188 (+   589us) log.cc:844] Serialized 26 byte log entry
0709 13:22:12.024818 (+804630us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:22:12.024862 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:22:12.024940 (+    78us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:22:12.456644 (+431704us) raft_consensus.cc:1685] finished
0709 13:22:12.456671 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:22:12.456827 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":40509,"raft.queue_time_us":66796,"raft.run_cpu_time_us":196,"raft.run_wall_time_us":81432,"thread_start_us":1257602,"threads_started":2,"wal-append.queue_time_us":961296}
I20260709 13:22:12.458678 16143 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46224 (request call id 1202) took 2213 ms. Trace:
I20260709 13:22:12.459802 23480 leader_election.cc:290] T 57a744f5aaf944678a63469cbf514015 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:12.459944 23326 raft_consensus.cc:3055] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:22:12.461829 23326 raft_consensus.cc:740] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
W20260709 13:22:12.462132 16264 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:22:12.452661 23460 consensus_queue.cc:1243] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Peer cb82cc7bd6db46d980c75237a1402860 log is divergent from this leader: its last log entry 22.15 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 14
I20260709 13:22:12.432821 23435 raft_consensus.cc:515] T c0eab25ce4c64e018a041beeda2867db P 647a343a542d4a139fe106c19095129b [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:12.459131 16143 rpcz_store.cc:276] 0709 13:22:10.245754 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:22:10.272118 (+ 26364us) service_pool.cc:224] Handling call
0709 13:22:10.279972 (+  7854us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:22:10.493378 (+213406us) raft_consensus.cc:1534] Early marking committed up to index 16
0709 13:22:10.493668 (+   290us) log.cc:844] Serialized 14 byte log entry
0709 13:22:10.884466 (+390798us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:22:10.884523 (+    57us) raft_consensus.cc:1645] Marking committed up to 16
0709 13:22:10.884575 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:22:12.458408 (+1573833us) spinlock_profiling.cc:243] Waited 213 ms on lock 0x7b5400073ad4. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:22:12.458452 (+    44us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:22:12.458596 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":64853,"prepare.queue_time_us":517,"prepare.run_cpu_time_us":41,"prepare.run_wall_time_us":41,"raft.queue_time_us":11172,"raft.run_cpu_time_us":94,"raft.run_wall_time_us":94,"spinlock_wait_cycles":490270592,"thread_start_us":382527,"threads_started":1,"wal-append.queue_time_us":559530,"wal-append.run_cpu_time_us":10084,"wal-append.run_wall_time_us":1109668}
I20260709 13:22:12.463719 23404 raft_consensus.cc:493] T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:22:12.464804 23306 raft_consensus.cc:493] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:22:12.478986 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 106215us
I20260709 13:22:12.472272 23024 raft_consensus.cc:493] T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:12.467350 23307 raft_consensus.cc:493] T 26354361e9b044cea1c2b86c4882fe92 P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:22:12.471334 23351 raft_consensus.cc:493] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:22:12.465375 23326 consensus_queue.cc:260] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:22:12.487845 23397 log.cc:927] Time spent T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 2.786s	user 0.000s	sys 0.017s
W20260709 13:22:12.488444 23429 log.cc:927] Time spent T c4f65cdf00b748bb963ee6e81ef43b79 P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 2.436s	user 0.015s	sys 0.012s
I20260709 13:22:12.498430 16220 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57604 (request call id 3421) took 1650 ms. Trace:
I20260709 13:22:12.490396 23024 raft_consensus.cc:515] T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:12.498929 16220 rpcz_store.cc:276] 0709 13:22:10.847877 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:22:10.848150 (+   273us) service_pool.cc:224] Handling call
0709 13:22:10.887574 (+ 39424us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:22:11.334827 (+447253us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:22:11.334874 (+    47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:22:11.387514 (+ 52640us) log.cc:844] Serialized 26 byte log entry
0709 13:22:12.150697 (+763183us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:22:12.150764 (+    67us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:22:12.150871 (+   107us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:22:12.498165 (+347294us) raft_consensus.cc:1685] finished
0709 13:22:12.498189 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:22:12.498345 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":198291,"raft.run_cpu_time_us":135,"raft.run_wall_time_us":11918,"thread_start_us":1014924,"threads_started":2,"wal-append.queue_time_us":889950}
I20260709 13:22:12.501327 23024 leader_election.cc:290] T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
W20260709 13:22:12.501716 16191 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:22:12.492722 23351 raft_consensus.cc:515] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:12.504729 23399 raft_consensus.cc:493] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 [term 22 FOLLOWER]: Starting leader election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:22:12.505362 23351 leader_election.cc:290] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:12.483410 23404 raft_consensus.cc:515] T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:12.506546 23373 raft_consensus.cc:3055] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [term 23 LEADER]: Stepping down as leader of term 23
I20260709 13:22:12.506997 23373 raft_consensus.cc:740] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:22:12.507937 23404 leader_election.cc:290] T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:22:12.507903 23373 consensus_queue.cc:260] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 23.15, Last appended by leader: 15, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:12.486748 23306 raft_consensus.cc:515] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } }
W20260709 13:22:12.509385 16147 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46224 (request call id 1213) took 1724 ms (client timeout 1588 ms). Trace:
W20260709 13:22:12.509811 16147 rpcz_store.cc:269] 0709 13:22:10.784872 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:22:10.850516 (+ 65644us) service_pool.cc:224] Handling call
0709 13:22:12.509167 (+1658651us) spinlock_profiling.cc:243] Waited 406 ms on lock 0x7b54000730d4. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:22:12.509313 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":934392576}
I20260709 13:22:12.511389 16146 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40830 (request call id 2122) took 2526 ms. Trace:
I20260709 13:22:12.492040 23307 raft_consensus.cc:515] T 26354361e9b044cea1c2b86c4882fe92 P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:12.511622 23373 raft_consensus.cc:3060] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:22:12.523540 16790 leader_election.cc:304] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:22:12.511837 16146 rpcz_store.cc:276] 0709 13:22:09.985284 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:22:10.191525 (+206241us) service_pool.cc:224] Handling call
0709 13:22:10.201518 (+  9993us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:22:10.266095 (+ 64577us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:22:12.511078 (+2244983us) spinlock_profiling.cc:243] Waited 45.1 ms on lock 0x7b54000735d4. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:22:12.511329 (+   251us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":103678848}
I20260709 13:22:12.510859 23306 leader_election.cc:290] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:22:12.523836 16218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9944d5e4b8c4109af9955c4f08c9095" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:12.505297 23399 raft_consensus.cc:3060] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:22:12.534591 23404 raft_consensus.cc:493] T ecc1e363230043fe9fef390f0b979f20 P 90f3da57796b488c9087a2f523adf045 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:12.534580 23460 raft_consensus.cc:493] T e5ad4eb281564936a59f41e387a47156 P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:22:12.542523 16151 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40844 (request call id 2996) took 1811 ms. Trace:
I20260709 13:22:12.544517 23166 raft_consensus.cc:2749] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 13:22:12.551127 16151 rpcz_store.cc:276] 0709 13:22:10.731457 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:22:10.737115 (+  5658us) service_pool.cc:224] Handling call
0709 13:22:10.737524 (+   409us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:22:10.866748 (+129224us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:22:10.866794 (+    46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:22:10.887905 (+ 21111us) log.cc:844] Serialized 26 byte log entry
0709 13:22:11.342313 (+454408us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:22:11.563664 (+221351us) log.cc:844] Serialized 14 byte log entry
0709 13:22:12.514902 (+951238us) spinlock_profiling.cc:243] Waited 3.31 ms on lock 0x7b18002c5480. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b693ade1 00007f69b69475b1 00007f69ad9ecc0a 00007f69b712c219 00007f69ada74a40 00007f69ada6f4e4 00007f69ada6f455 00007f69b0b57060 00007f69b0b69716 00007f69b0b7a287 00007f69b0b7a208
0709 13:22:12.515132 (+   230us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:22:12.525758 (+ 10626us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:22:12.525829 (+    71us) raft_consensus.cc:1685] finished
0709 13:22:12.541368 (+ 15539us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:22:12.542452 (+  1084us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":61607,"prepare.queue_time_us":226274,"prepare.run_cpu_time_us":47,"prepare.run_wall_time_us":50,"raft.queue_time_us":1007961,"raft.run_cpu_time_us":227,"raft.run_wall_time_us":25241,"spinlock_wait_cycles":7612672,"thread_start_us":341840,"threads_started":1,"wal-append.queue_time_us":780523}
I20260709 13:22:12.559312 22610 raft_consensus.cc:493] T 08c74490c40243a88e62d1760b5cd814 P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:12.559917 22610 raft_consensus.cc:515] T 08c74490c40243a88e62d1760b5cd814 P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:12.562462 22610 leader_election.cc:290] T 08c74490c40243a88e62d1760b5cd814 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:12.563144 23476 raft_consensus.cc:493] T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:12.563604 23322 raft_consensus.cc:493] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:12.563771 23476 raft_consensus.cc:515] T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:12.564674 23322 raft_consensus.cc:515] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:12.566257 23476 leader_election.cc:290] T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:12.568121 23460 raft_consensus.cc:515] T e5ad4eb281564936a59f41e387a47156 P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:12.568599 23322 leader_election.cc:290] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:12.570379 23460 leader_election.cc:290] T e5ad4eb281564936a59f41e387a47156 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:12.503044 23435 leader_election.cc:290] T c0eab25ce4c64e018a041beeda2867db P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 21 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:12.571893 23404 raft_consensus.cc:515] T ecc1e363230043fe9fef390f0b979f20 P 90f3da57796b488c9087a2f523adf045 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:12.574776 23404 leader_election.cc:290] T ecc1e363230043fe9fef390f0b979f20 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:12.544656 23480 consensus_queue.cc:1059] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: INVALID_TERM, Last received: 18.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.031s
I20260709 13:22:12.575891 23199 raft_consensus.cc:3055] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:22:12.576258 23199 raft_consensus.cc:740] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:22:12.577178 23307 leader_election.cc:290] T 26354361e9b044cea1c2b86c4882fe92 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:12.577091 23199 consensus_queue.cc:260] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 18.15, Last appended by leader: 15, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:12.541251 22917 consensus_queue.cc:1059] T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: INVALID_TERM, Last received: 19.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.037s
I20260709 13:22:12.578269 23199 raft_consensus.cc:3060] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:22:12.534112 23370 raft_consensus.cc:493] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:22:12.579487 23370 raft_consensus.cc:515] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:12.581501 23370 leader_election.cc:290] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:12.586990 23441 leader_election.cc:290] T 7c2095a5f4ad49408dbc7f69c606120d P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
W20260709 13:22:12.587479 23464 consensus_queue.cc:1175] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "1ca2725d290e47d68b1a4b609c9816ac" responder_term: 18 status { last_received { term: 18 index: 15 } last_committed_idx: 15 last_received_current_leader { term: 18 index: 15 } }
I20260709 13:22:12.595889 23347 leader_election.cc:290] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:12.583066 16825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada5a004ba7c46819051e3ccec71ffdb" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:12.601230 16825 raft_consensus.cc:2393] T ada5a004ba7c46819051e3ccec71ffdb P cb82cc7bd6db46d980c75237a1402860 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 21: Already voted for candidate cb82cc7bd6db46d980c75237a1402860 in this term.
I20260709 13:22:12.604281 16293 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48684 (request call id 3205) took 2690 ms. Trace:
I20260709 13:22:12.545202 23437 raft_consensus.cc:493] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:12.604774 16293 rpcz_store.cc:276] 0709 13:22:09.909346 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:22:09.909517 (+   171us) service_pool.cc:224] Handling call
0709 13:22:09.909900 (+   383us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:22:09.959883 (+ 49983us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:22:09.959927 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:22:09.960509 (+   582us) log.cc:844] Serialized 26 byte log entry
0709 13:22:10.051447 (+ 90938us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:22:10.051485 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:22:10.051553 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:22:12.487899 (+2436346us) spinlock_profiling.cc:243] Waited 60.7 ms on lock 0x7b4400022b4c. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69ad8fadb1 00007f69ad998c85 00007f69ad9996a2 00007f69b070b1db 00007f69b0706aaf 00007f69b06fa86e 00007f69b06f9bb6 00007f69b06f97d8 00007f69b0703221 00007f69b07031d9 00007f69b0703169
0709 13:22:12.599025 (+111126us) raft_consensus.cc:1685] finished
0709 13:22:12.599055 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:22:12.599323 (+   268us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":57076,"raft.queue_time_us":50916,"raft.run_cpu_time_us":150,"raft.run_wall_time_us":6404,"spinlock_wait_cycles":139522944,"thread_start_us":84520,"threads_started":1,"wal-append.queue_time_us":84919}
I20260709 13:22:12.557960 23326 raft_consensus.cc:3060] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:22:12.618139 16824 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49892 (request call id 1066) took 2819 ms. Trace:
I20260709 13:22:12.618685 16824 rpcz_store.cc:276] 0709 13:22:09.792653 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:22:09.792915 (+   262us) service_pool.cc:224] Handling call
0709 13:22:09.793367 (+   452us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:22:09.927897 (+134530us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:22:12.611689 (+2683792us) spinlock_profiling.cc:243] Waited 133 ms on lock 0x7b540047fc54. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:22:12.611948 (+   259us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":304968960}
I20260709 13:22:12.602795 22677 raft_consensus.cc:493] T 02838c86531d4f98bf58d1c22b8baf77 P 1ca2725d290e47d68b1a4b609c9816ac [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:12.563009 22610 raft_consensus.cc:493] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:22:12.641299 23519 log.cc:927] Time spent T d2ab0ac8347641e6955e9bce7d90c1a2 P cb82cc7bd6db46d980c75237a1402860: Append to log took a long time: real 0.159s	user 0.001s	sys 0.000s
I20260709 13:22:12.645345 16219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6f07842d3aa47858df5665874c86f3a" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:12.646664 16222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c61a39fd4f84424b129c1a6f9ba404d" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:12.648313 16218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:22:12.649035 16218 raft_consensus.cc:3060] T 19baa7bc367844c288bca8f81439c413 P 1ca2725d290e47d68b1a4b609c9816ac [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:22:12.653163 23307 raft_consensus.cc:493] T a378d7f796f94615b3b347119a9880fd P 90f3da57796b488c9087a2f523adf045 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:22:12.653898 23307 raft_consensus.cc:515] T a378d7f796f94615b3b347119a9880fd P 90f3da57796b488c9087a2f523adf045 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:12.654362 16219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806d7690c2b644959152074bb2cab0a7" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:12.655316 23404 raft_consensus.cc:493] T 66ae176961b74813a58bc15c9c4c6f2a P 90f3da57796b488c9087a2f523adf045 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:22:12.656342 16219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 23 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:12.656875 23306 raft_consensus.cc:493] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:22:12.655993 23404 raft_consensus.cc:515] T 66ae176961b74813a58bc15c9c4c6f2a P 90f3da57796b488c9087a2f523adf045 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:12.657554 23306 raft_consensus.cc:515] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:12.617115 23407 raft_consensus.cc:493] T d2ab0ac8347641e6955e9bce7d90c1a2 P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:12.657147 16219 raft_consensus.cc:2410] T ecc1e363230043fe9fef390f0b979f20 P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 for term 23 because replica has last-logged OpId of term: 21 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 12.
I20260709 13:22:12.668771 23491 raft_consensus.cc:493] T 26354361e9b044cea1c2b86c4882fe92 P cb82cc7bd6db46d980c75237a1402860 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:22:12.620678 23437 raft_consensus.cc:515] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:12.669498 23404 leader_election.cc:290] T 66ae176961b74813a58bc15c9c4c6f2a P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:12.660082 22677 raft_consensus.cc:515] T 02838c86531d4f98bf58d1c22b8baf77 P 1ca2725d290e47d68b1a4b609c9816ac [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:12.672137 23371 raft_consensus.cc:493] T c0eab25ce4c64e018a041beeda2867db P 1ca2725d290e47d68b1a4b609c9816ac [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:22:12.640615 22610 raft_consensus.cc:515] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:22:12.687471 16115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 168826us
W20260709 13:22:12.701095 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 49955.1us
I20260709 13:22:12.703150 16818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:12.704000 16818 raft_consensus.cc:2468] T 19baa7bc367844c288bca8f81439c413 P cb82cc7bd6db46d980c75237a1402860 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 22.
I20260709 13:22:12.708333 16818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5718c623ebee4d2bb770f0f9ddb2f164" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:12.712023 16823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:12.712800 16823 raft_consensus.cc:2468] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 22.
I20260709 13:22:12.678771 23407 raft_consensus.cc:515] T d2ab0ac8347641e6955e9bce7d90c1a2 P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:12.725209 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66ae176961b74813a58bc15c9c4c6f2a" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 27 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:12.728842 23478 raft_consensus.cc:493] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:12.735458 23210 consensus_queue.cc:1048] T c0eab25ce4c64e018a041beeda2867db P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:22:12.689527 23491 raft_consensus.cc:515] T 26354361e9b044cea1c2b86c4882fe92 P cb82cc7bd6db46d980c75237a1402860 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:12.738269 23437 leader_election.cc:290] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
W20260709 13:22:12.738893 16189 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:22:12.739707 23491 leader_election.cc:290] T 26354361e9b044cea1c2b86c4882fe92 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:12.731227 23311 raft_consensus.cc:515] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:12.747910 23306 leader_election.cc:290] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:22:12.748544 16292 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48684 (request call id 3218) took 1985 ms. Trace:
I20260709 13:22:12.749004 16292 rpcz_store.cc:276] 0709 13:22:10.763091 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:22:10.763884 (+   793us) service_pool.cc:224] Handling call
0709 13:22:10.764344 (+   460us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:22:10.909120 (+144776us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:22:10.909158 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:22:12.592131 (+1682973us) spinlock_profiling.cc:243] Waited 18 ms on lock 0x7b180048b1a0. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b693ade1 00007f69b69475b1 00007f69ada402a6 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69af458a11 00007f69af468372 00007f69b0b9aa01 00007f69b0b5f43a
0709 13:22:12.592705 (+   574us) spinlock_profiling.cc:243] Waited 2.03 ms on lock 0x7b2c00132978. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69af458a11 00007f69af468372 00007f69b0b9aa01 00007f69b0b5f43a 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2
0709 13:22:12.593219 (+   514us) log.cc:844] Serialized 26 byte log entry
0709 13:22:12.694610 (+101391us) raft_consensus.cc:1645] Marking committed up to 15
0709 13:22:12.694680 (+    70us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:22:12.694795 (+   115us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:22:12.748295 (+ 53500us) raft_consensus.cc:1685] finished
0709 13:22:12.748328 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:22:12.748483 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":203613,"raft.queue_time_us":144799,"raft.run_cpu_time_us":169,"raft.run_wall_time_us":39747,"spinlock_wait_cycles":45956736,"thread_start_us":100880,"threads_started":1,"wal-append.queue_time_us":150795}
I20260709 13:22:12.706123 23371 raft_consensus.cc:515] T c0eab25ce4c64e018a041beeda2867db P 1ca2725d290e47d68b1a4b609c9816ac [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:22:12.751832 23518 log.cc:927] Time spent T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860: Append to log took a long time: real 0.066s	user 0.000s	sys 0.000s
I20260709 13:22:12.757134 16218 raft_consensus.cc:2468] T 19baa7bc367844c288bca8f81439c413 P 1ca2725d290e47d68b1a4b609c9816ac [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 23.
I20260709 13:22:12.761380 23307 leader_election.cc:290] T a378d7f796f94615b3b347119a9880fd P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:12.767688 23382 raft_consensus.cc:3060] T 66f75124215f4ca6bef831e174a792d8 P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:22:12.776350 23403 raft_consensus.cc:3060] T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:22:12.729547 23478 raft_consensus.cc:515] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:12.788679 16219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a959dbecc2648eb85d1cc550f5a3245" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:12.789451 16219 raft_consensus.cc:2468] T 5a959dbecc2648eb85d1cc550f5a3245 P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 19.
I20260709 13:22:12.774441 23371 leader_election.cc:290] T c0eab25ce4c64e018a041beeda2867db P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:12.795835 16219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a378d7f796f94615b3b347119a9880fd" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:12.791749 22610 leader_election.cc:290] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:12.804316 23478 leader_election.cc:290] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:12.805438 16220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ad5dbfa99ee4b8d9e28be92b3f36db5" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:12.808156 16223 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57600 (request call id 2123) took 2095 ms. Trace:
I20260709 13:22:12.808619 16223 rpcz_store.cc:276] 0709 13:22:10.712727 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:22:10.712950 (+   223us) service_pool.cc:224] Handling call
0709 13:22:10.713333 (+   383us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:22:10.713739 (+   406us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:22:10.714070 (+   331us) log.cc:844] Serialized 14 byte log entry
0709 13:22:11.073302 (+359232us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:22:12.293637 (+1220335us) spinlock_profiling.cc:243] Waited 22.8 ms on lock 0x7b18003a9840. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b693ade1 00007f69b69475b1 00007f69ada402a6 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b693ade1 00007f69b69475b1 00007f69ada723f3 00007f69ada747c1
0709 13:22:12.493269 (+199632us) spinlock_profiling.cc:243] Waited 7.18 ms on lock 0x7b2c00115ac8. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69af458a11 00007f69af468372 00007f69b0b9aa01 00007f69b0b5f43a 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2
0709 13:22:12.515470 (+ 22201us) log.cc:844] Serialized 26 byte log entry
0709 13:22:12.671087 (+155617us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:22:12.671132 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:22:12.672285 (+  1153us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:22:12.807068 (+134783us) raft_consensus.cc:1685] finished
0709 13:22:12.807085 (+    17us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:22:12.808093 (+  1008us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":17502,"raft.queue_time_us":94761,"raft.run_cpu_time_us":191,"raft.run_wall_time_us":25217,"spinlock_wait_cycles":85786368,"thread_start_us":513944,"threads_started":2,"wal-append.queue_time_us":767073,"wal-append.run_cpu_time_us":11328,"wal-append.run_wall_time_us":1009704}
I20260709 13:22:12.826083 22908 raft_consensus.cc:493] T 8a878999c0024d2abe517484180290fc P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:22:12.826650 22908 raft_consensus.cc:515] T 8a878999c0024d2abe517484180290fc P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:12.826947 22677 leader_election.cc:290] T 02838c86531d4f98bf58d1c22b8baf77 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:12.828914 22908 leader_election.cc:290] T 8a878999c0024d2abe517484180290fc P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:12.846225 23256 raft_consensus.cc:493] T 829b4ad8c5a54eababeea1181b960d4b P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:22:12.846999 23256 raft_consensus.cc:515] T 829b4ad8c5a54eababeea1181b960d4b P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:12.850544 23256 leader_election.cc:290] T 829b4ad8c5a54eababeea1181b960d4b P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:12.862396 22677 raft_consensus.cc:493] T 7c2095a5f4ad49408dbc7f69c606120d P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:12.877977 16292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e9b2bf7011f4ae688a79d7bb5a447b4" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:12.878266 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c11074579b046d99e585c2818e18a2c" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:12.878679 16292 raft_consensus.cc:2468] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 18.
I20260709 13:22:12.879045 16293 raft_consensus.cc:2410] T 8c11074579b046d99e585c2818e18a2c P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 for term 21 because replica has last-logged OpId of term: 20 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 15.
I20260709 13:22:12.880018 16292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3884f864a96d428785895e5463e5728f" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 15 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:12.880635 16292 raft_consensus.cc:2468] T 3884f864a96d428785895e5463e5728f P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 15.
I20260709 13:22:12.881214 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8638ee363273426db1393dd81e5c7b72" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:12.882622 16294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada5a004ba7c46819051e3ccec71ffdb" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:22:12.873629 23024 raft_consensus.cc:493] T 68a6838d82ef4cac83507f906139d90f P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:12.888243 16291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 23 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:12.889676 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434bff3082d141838068f323bdb0d1ed" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:12.889046 16291 raft_consensus.cc:2468] T ecc1e363230043fe9fef390f0b979f20 P 90f3da57796b488c9087a2f523adf045 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 22.
I20260709 13:22:12.875962 22677 raft_consensus.cc:515] T 7c2095a5f4ad49408dbc7f69c606120d P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:12.894191 16291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e65676b21b748728ad39668cfe722ec" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:22:12.895133 22677 leader_election.cc:290] T 7c2095a5f4ad49408dbc7f69c606120d P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:12.895895 16291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74ca8140f35c4ae29e8a664f6bdfc829" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 22 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:22:12.883561 23235 raft_consensus.cc:493] T 5a96373bb0e14dce8a266b2fc6c58507 P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:22:12.892791 16298 raft_consensus.cc:2393] T 434bff3082d141838068f323bdb0d1ed P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 in current term 19: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
I20260709 13:22:12.897348 16291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2794af5b99984efebdee564035841cb0" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:12.883396 23404 raft_consensus.cc:493] T 23f82224f4564fea93572e8206bad1fe P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:22:12.898568 16291 raft_consensus.cc:2468] T 2794af5b99984efebdee564035841cb0 P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 18.
I20260709 13:22:12.901358 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f82224f4564fea93572e8206bad1fe" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:12.905107 22677 raft_consensus.cc:493] T d16dfad0c3b540e5b4b86492ddf87208 P 1ca2725d290e47d68b1a4b609c9816ac [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:22:12.904126 23256 raft_consensus.cc:493] T 5a96373bb0e14dce8a266b2fc6c58507 P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:22:12.905695 22677 raft_consensus.cc:515] T d16dfad0c3b540e5b4b86492ddf87208 P 1ca2725d290e47d68b1a4b609c9816ac [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:12.905797 23256 raft_consensus.cc:515] T 5a96373bb0e14dce8a266b2fc6c58507 P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:12.906792 23331 leader_election.cc:290] T 84f2da98864b4425bf7c0055951fdd4b P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 17 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:12.907899 22677 leader_election.cc:290] T d16dfad0c3b540e5b4b86492ddf87208 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:12.907998 23256 leader_election.cc:290] T 5a96373bb0e14dce8a266b2fc6c58507 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
W20260709 13:22:12.910095 23511 log.cc:927] Time spent T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860: Append to log took a long time: real 0.477s	user 0.003s	sys 0.003s
I20260709 13:22:12.911487 16294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02838c86531d4f98bf58d1c22b8baf77" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:12.912156 16294 raft_consensus.cc:2468] T 02838c86531d4f98bf58d1c22b8baf77 P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 20.
I20260709 13:22:12.912717 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5ad4eb281564936a59f41e387a47156" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:12.892041 23024 raft_consensus.cc:515] T 68a6838d82ef4cac83507f906139d90f P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:12.920976 16291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87147731a68400e8cce9b879daa7c5d" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:12.899711 23235 raft_consensus.cc:515] T 5a96373bb0e14dce8a266b2fc6c58507 P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:12.925086 16144 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46224 (request call id 1215) took 1585 ms. Trace:
I20260709 13:22:12.926962 16820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c11074579b046d99e585c2818e18a2c" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:12.927502 16819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3884f864a96d428785895e5463e5728f" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 16 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:12.936784 16817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7ca61592e848ea9b0fb81a2ce33cdc" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:12.937988 16824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "829b4ad8c5a54eababeea1181b960d4b" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:12.938345 23384 raft_consensus.cc:1064] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac: attempting to promote NON_VOTER cb82cc7bd6db46d980c75237a1402860 to VOTER
W20260709 13:22:12.938783 23384 raft_consensus.cc:1068] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac: Unable to promote non-voter cb82cc7bd6db46d980c75237a1402860: Illegal state: Leader has not yet committed an operation in its own term
I20260709 13:22:12.939225 16823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8638ee363273426db1393dd81e5c7b72" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:22:12.933838 16819 raft_consensus.cc:2468] T 3884f864a96d428785895e5463e5728f P cb82cc7bd6db46d980c75237a1402860 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 14.
I20260709 13:22:12.932853 16144 rpcz_store.cc:276] 0709 13:22:11.340071 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:22:11.340274 (+   203us) service_pool.cc:224] Handling call
0709 13:22:12.924879 (+1584605us) spinlock_profiling.cc:243] Waited 98.9 ms on lock 0x7b54003d9fd4. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:22:12.925030 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":227566976}
I20260709 13:22:12.941962 16819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31969392ccdb43b491911a319e8bc7e1" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:12.900947 23404 raft_consensus.cc:515] T 23f82224f4564fea93572e8206bad1fe P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:12.947574 22677 consensus_queue.cc:1048] T 84f2da98864b4425bf7c0055951fdd4b P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.009s
I20260709 13:22:12.957713 16819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87147731a68400e8cce9b879daa7c5d" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:12.958942 16820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26354361e9b044cea1c2b86c4882fe92" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:12.959090 16291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0eab25ce4c64e018a041beeda2867db" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:12.959589 16820 raft_consensus.cc:2468] T 26354361e9b044cea1c2b86c4882fe92 P cb82cc7bd6db46d980c75237a1402860 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 16.
I20260709 13:22:12.962584 16298 raft_consensus.cc:2468] T 23f82224f4564fea93572e8206bad1fe P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 17.
I20260709 13:22:12.963333 16823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:22:12.963774 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ab0ac8347641e6955e9bce7d90c1a2" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:12.964391 16217 raft_consensus.cc:2468] T d2ab0ac8347641e6955e9bce7d90c1a2 P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 19.
I20260709 13:22:12.964712 16824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a9de347c711477e87c582d003f36925" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:12.966125 23326 raft_consensus.cc:493] T 8cc959fb7f704fba9009c0595467d074 P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:22:12.966827 16817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 23 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:12.967440 16817 raft_consensus.cc:2468] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 22.
W20260709 13:22:12.968365 16191 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:22:12.968554 16265 leader_election.cc:304] T d2ab0ac8347641e6955e9bce7d90c1a2 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:22:12.963930 16823 raft_consensus.cc:3060] T 19baa7bc367844c288bca8f81439c413 P cb82cc7bd6db46d980c75237a1402860 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:22:12.976012 16291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a9de347c711477e87c582d003f36925" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:12.976581 16291 raft_consensus.cc:2468] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 23.
I20260709 13:22:12.977854 16819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a378d7f796f94615b3b347119a9880fd" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:12.980787 16219 raft_consensus.cc:1240] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [term 20 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 19. Current term is 20. Ops: []
I20260709 13:22:12.982465 23235 leader_election.cc:290] T 5a96373bb0e14dce8a266b2fc6c58507 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:22:12.985880 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a96373bb0e14dce8a266b2fc6c58507" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:12.962234 23528 raft_consensus.cc:493] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:12.988099 16817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66ae176961b74813a58bc15c9c4c6f2a" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 27 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
W20260709 13:22:12.989040 16791 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:22:12.988989 16217 raft_consensus.cc:2468] T 5a96373bb0e14dce8a266b2fc6c58507 P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 16.
I20260709 13:22:12.966722 23351 raft_consensus.cc:493] T 8158bbd17b994d2eb96f560bd0f1b88d P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:12.978885 23210 raft_consensus.cc:2804] T d2ab0ac8347641e6955e9bce7d90c1a2 P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:22:12.998157 16265 leader_election.cc:304] T 5a96373bb0e14dce8a266b2fc6c58507 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:22:13.002526 16291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8158bbd17b994d2eb96f560bd0f1b88d" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:12.982649 23399 raft_consensus.cc:515] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:12.998929 22677 raft_consensus.cc:493] T 746aabf856ab401c9ef5e859371e7b67 P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:22:13.004701 22677 raft_consensus.cc:515] T 746aabf856ab401c9ef5e859371e7b67 P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:13.002857 23235 consensus_queue.cc:1059] T 8638ee363273426db1393dd81e5c7b72 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: INVALID_TERM, Last received: 19.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.003s
I20260709 13:22:13.007340 23404 leader_election.cc:290] T 23f82224f4564fea93572e8206bad1fe P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
W20260709 13:22:13.008206 16264 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:22:13.008297 23407 leader_election.cc:290] T d2ab0ac8347641e6955e9bce7d90c1a2 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:13.009691 23235 raft_consensus.cc:3055] T 8638ee363273426db1393dd81e5c7b72 P 90f3da57796b488c9087a2f523adf045 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:22:13.010092 23235 raft_consensus.cc:740] T 8638ee363273426db1393dd81e5c7b72 P 90f3da57796b488c9087a2f523adf045 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:22:13.010947 23235 consensus_queue.cc:260] T 8638ee363273426db1393dd81e5c7b72 P 90f3da57796b488c9087a2f523adf045 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:13.011358 23307 raft_consensus.cc:2804] T 5a96373bb0e14dce8a266b2fc6c58507 P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:22:13.013213 23307 raft_consensus.cc:493] T 5a96373bb0e14dce8a266b2fc6c58507 P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:22:13.013592 23307 raft_consensus.cc:3060] T 5a96373bb0e14dce8a266b2fc6c58507 P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:22:13.018270 23373 raft_consensus.cc:493] T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:13.018850 23373 raft_consensus.cc:515] T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:13.020351 16222 raft_consensus.cc:2468] T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 21.
I20260709 13:22:12.996961 23528 raft_consensus.cc:515] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:13.022807 23528 leader_election.cc:290] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:22:13.003671 23210 raft_consensus.cc:493] T d2ab0ac8347641e6955e9bce7d90c1a2 P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:13.026618 23399 leader_election.cc:290] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 23 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:13.029047 16219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f5f89186204242bf377f7c42b2e2da" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 23 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:13.024060 15046 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:22:13.036600 16219 raft_consensus.cc:2468] T c2f5f89186204242bf377f7c42b2e2da P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 21.
I20260709 13:22:13.037863 16265 leader_election.cc:304] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:22:12.970629 23326 raft_consensus.cc:515] T 8cc959fb7f704fba9009c0595467d074 P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:22:13.044524 16191 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:22:13.027194 23210 raft_consensus.cc:3060] T d2ab0ac8347641e6955e9bce7d90c1a2 P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:22:13.012264 23235 raft_consensus.cc:3060] T 8638ee363273426db1393dd81e5c7b72 P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:22:13.072081 16817 raft_consensus.cc:1275] T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860 [term 23 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 20 index: 16. Preceding OpId from leader: term: 23 index: 17. (index mismatch)
I20260709 13:22:13.073756 23024 leader_election.cc:290] T 68a6838d82ef4cac83507f906139d90f P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:22:13.075155 23061 consensus_queue.cc:1048] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 13:22:13.074821 16219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68a6838d82ef4cac83507f906139d90f" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
W20260709 13:22:13.085920 23497 log.cc:927] Time spent T 8d4ab870a05347868956aa8b841b2981 P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 0.830s	user 0.004s	sys 0.000s
I20260709 13:22:13.087110 16295 raft_consensus.cc:1240] T 8d4ab870a05347868956aa8b841b2981 P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 18. Current term is 19. Ops: []
I20260709 13:22:13.052320 23404 raft_consensus.cc:2804] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:22:13.089651 16295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:13.092558 16295 raft_consensus.cc:2393] T ecc1e363230043fe9fef390f0b979f20 P 90f3da57796b488c9087a2f523adf045 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 22: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
I20260709 13:22:13.093932 16294 raft_consensus.cc:1275] T 68a6838d82ef4cac83507f906139d90f P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 13:22:13.027442 23351 raft_consensus.cc:515] T 8158bbd17b994d2eb96f560bd0f1b88d P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:13.102082 16296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7ca61592e848ea9b0fb81a2ce33cdc" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:13.093884 23404 raft_consensus.cc:493] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:13.104072 23404 raft_consensus.cc:3060] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:22:13.104231 16296 raft_consensus.cc:3060] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:22:13.112133 16295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a96373bb0e14dce8a266b2fc6c58507" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:13.113664 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c61a39fd4f84424b129c1a6f9ba404d" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 22 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:13.116341 16299 raft_consensus.cc:2410] T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b for term 22 because replica has last-logged OpId of term: 20 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 10.
I20260709 13:22:13.114188 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a878999c0024d2abe517484180290fc" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:13.118453 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d16dfad0c3b540e5b4b86492ddf87208" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:13.124863 16296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8638ee363273426db1393dd81e5c7b72" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:22:13.128157 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4dc6b0541ca431ebc757beac00521f1" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:13.090817 23024 raft_consensus.cc:493] T 434bff3082d141838068f323bdb0d1ed P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:13.129912 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8638ee363273426db1393dd81e5c7b72" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:13.114154 23474 raft_consensus.cc:493] T 66f75124215f4ca6bef831e174a792d8 P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:13.131312 16291 raft_consensus.cc:2468] T 8158bbd17b994d2eb96f560bd0f1b88d P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 18.
I20260709 13:22:13.115839 23326 leader_election.cc:290] T 8cc959fb7f704fba9009c0595467d074 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
W20260709 13:22:13.135584 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 1.56579e+06us
I20260709 13:22:13.135711 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ab0ac8347641e6955e9bce7d90c1a2" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 19 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:22:13.138166 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0eab25ce4c64e018a041beeda2867db" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:13.143729 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c74490c40243a88e62d1760b5cd814" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:13.142979 16266 leader_election.cc:304] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860
I20260709 13:22:13.144711 16293 raft_consensus.cc:2468] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 19.
I20260709 13:22:13.150985 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada5a004ba7c46819051e3ccec71ffdb" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:13.153015 16266 leader_election.cc:304] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 647a343a542d4a139fe106c19095129b
I20260709 13:22:13.153752 16148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:22:13.154443 16148 raft_consensus.cc:2468] T ecc1e363230043fe9fef390f0b979f20 P 647a343a542d4a139fe106c19095129b [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 22.
I20260709 13:22:13.154783 16151 raft_consensus.cc:1275] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [term 26 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 25 index: 16. Preceding OpId from leader: term: 26 index: 16. (term mismatch)
I20260709 13:22:13.155162 16151 pending_rounds.cc:85] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b: Aborting all ops after (but not including) 15
I20260709 13:22:13.155395 16151 pending_rounds.cc:107] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b: Aborting uncommitted NO_OP operation due to leader change: 25.16
W20260709 13:22:13.138590 23534 log.cc:927] Time spent T 8a878999c0024d2abe517484180290fc P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.063s	user 0.001s	sys 0.001s
I20260709 13:22:13.155628 16147 raft_consensus.cc:1240] T 84f2da98864b4425bf7c0055951fdd4b P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 16. Current term is 17. Ops: []
I20260709 13:22:13.156231 16145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434bff3082d141838068f323bdb0d1ed" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:22:13.155609 16151 raft_consensus.cc:2889] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [term 26 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:22:13.156561 16143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e65676b21b748728ad39668cfe722ec" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:22:13.159548 16789 leader_election.cc:304] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 647a343a542d4a139fe106c19095129b
I20260709 13:22:13.160760 23306 raft_consensus.cc:2749] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:22:13.162025 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a959dbecc2648eb85d1cc550f5a3245" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:22:13.163061 16150 raft_consensus.cc:2393] T 5a959dbecc2648eb85d1cc550f5a3245 P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 20: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
I20260709 13:22:13.163326 23347 raft_consensus.cc:493] T 1df93779209748b9860f6d893dd3de53 P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:13.163465 23524 raft_consensus.cc:493] T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:22:13.157838 16145 raft_consensus.cc:2468] T 434bff3082d141838068f323bdb0d1ed P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 19.
I20260709 13:22:13.163755 16144 raft_consensus.cc:1275] T 68a6838d82ef4cac83507f906139d90f P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 13:22:13.172015 16146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2794af5b99984efebdee564035841cb0" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:22:13.172787 16146 raft_consensus.cc:2393] T 2794af5b99984efebdee564035841cb0 P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 in current term 18: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
I20260709 13:22:13.172724 16790 leader_election.cc:304] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:22:13.158210 16147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b7d1ab519044eda2d5e04a07242f25" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:22:13.174000 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0eab25ce4c64e018a041beeda2867db" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:22:13.174755 16147 raft_consensus.cc:2393] T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 19: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
I20260709 13:22:13.167001 16789 leader_election.cc:304] T 3884f864a96d428785895e5463e5728f P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:22:13.163897 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3884f864a96d428785895e5463e5728f" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 15 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:22:13.177779 16150 raft_consensus.cc:2393] T 3884f864a96d428785895e5463e5728f P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 in current term 15: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
I20260709 13:22:13.179208 16789 leader_election.cc:304] T 8c11074579b046d99e585c2818e18a2c P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:22:13.180667 16789 leader_election.cc:304] T ada5a004ba7c46819051e3ccec71ffdb P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:22:13.180428 16147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4dc6b0541ca431ebc757beac00521f1" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:22:13.164065 23524 raft_consensus.cc:515] T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:13.195129 16789 leader_election.cc:304] T 8638ee363273426db1393dd81e5c7b72 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:22:13.195254 16144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57a744f5aaf944678a63469cbf514015" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:22:13.182224 16147 raft_consensus.cc:2468] T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 20.
I20260709 13:22:13.182298 16142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1df93779209748b9860f6d893dd3de53" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:22:13.196061 16144 raft_consensus.cc:2468] T 57a744f5aaf944678a63469cbf514015 P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 17.
I20260709 13:22:13.180558 16790 leader_election.cc:304] T 434bff3082d141838068f323bdb0d1ed P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:22:13.163859 23347 raft_consensus.cc:515] T 1df93779209748b9860f6d893dd3de53 P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:13.183593 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 23 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:22:13.198215 16149 raft_consensus.cc:2410] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac for term 23 because replica has last-logged OpId of term: 21 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 11.
I20260709 13:22:13.184175 23396 consensus_queue.cc:1048] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:22:13.184789 23408 raft_consensus.cc:2749] T 8c11074579b046d99e585c2818e18a2c P cb82cc7bd6db46d980c75237a1402860 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:22:13.184923 23311 leader_election.cc:290] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 20 election: Requested vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:13.185423 23407 consensus_queue.cc:1059] T ada5a004ba7c46819051e3ccec71ffdb P 90f3da57796b488c9087a2f523adf045 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: INVALID_TERM, Last received: 20.15, Next index: 16, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:22:13.187634 16146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d4ab870a05347868956aa8b841b2981" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
W20260709 13:22:13.202451 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 64474.1us
I20260709 13:22:13.164354 23399 raft_consensus.cc:493] T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:13.188503 23533 consensus_queue.cc:1059] T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: INVALID_TERM, Last received: 19.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:22:13.184958 23483 raft_consensus.cc:2804] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:22:13.181739 23398 raft_consensus.cc:2804] T 3884f864a96d428785895e5463e5728f P cb82cc7bd6db46d980c75237a1402860 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:22:13.227939 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:13.228610 16293 raft_consensus.cc:2468] T ecc1e363230043fe9fef390f0b979f20 P 90f3da57796b488c9087a2f523adf045 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 22.
I20260709 13:22:13.140027 23474 raft_consensus.cc:515] T 66f75124215f4ca6bef831e174a792d8 P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:13.196223 16823 raft_consensus.cc:2468] T 19baa7bc367844c288bca8f81439c413 P cb82cc7bd6db46d980c75237a1402860 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 23.
I20260709 13:22:13.193833 23394 raft_consensus.cc:3060] T 3a7ca61592e848ea9b0fb81a2ce33cdc P cb82cc7bd6db46d980c75237a1402860 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:22:13.230892 16221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8158bbd17b994d2eb96f560bd0f1b88d" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:13.231649 16221 raft_consensus.cc:2468] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 18.
I20260709 13:22:13.233736 16221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c61a39fd4f84424b129c1a6f9ba404d" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 22 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:13.197252 16147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c74490c40243a88e62d1760b5cd814" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:22:13.136463 23024 raft_consensus.cc:515] T 434bff3082d141838068f323bdb0d1ed P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:13.198690 16789 leader_election.cc:304] T 74ca8140f35c4ae29e8a664f6bdfc829 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:22:13.200652 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 23 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:22:13.240221 16149 raft_consensus.cc:2410] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 for term 23 because replica has last-logged OpId of term: 21 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 11.
I20260709 13:22:13.179050 23199 raft_consensus.cc:2804] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:22:13.205852 16144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66f75124215f4ca6bef831e174a792d8" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:22:13.200297 16190 leader_election.cc:304] T ecc1e363230043fe9fef390f0b979f20 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860
I20260709 13:22:13.242931 16226 raft_consensus.cc:2410] T 66f75124215f4ca6bef831e174a792d8 P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 for term 19 because replica has last-logged OpId of term: 17 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 13.
I20260709 13:22:13.194603 23391 raft_consensus.cc:2749] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:22:13.142119 23321 consensus_queue.cc:1059] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: INVALID_TERM, Last received: 20.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.042s
I20260709 13:22:13.229682 16142 raft_consensus.cc:2468] T 1df93779209748b9860f6d893dd3de53 P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 18.
I20260709 13:22:13.234098 16146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f82224f4564fea93572e8206bad1fe" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:22:13.234741 16147 raft_consensus.cc:2468] T 08c74490c40243a88e62d1760b5cd814 P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 19.
I20260709 13:22:13.198885 23495 raft_consensus.cc:2749] T 8638ee363273426db1393dd81e5c7b72 P cb82cc7bd6db46d980c75237a1402860 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:22:13.197607 23450 raft_consensus.cc:2749] T ada5a004ba7c46819051e3ccec71ffdb P cb82cc7bd6db46d980c75237a1402860 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:22:13.240687 16789 leader_election.cc:304] T 2794af5b99984efebdee564035841cb0 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 647a343a542d4a139fe106c19095129b
I20260709 13:22:13.203979 23491 raft_consensus.cc:2804] T 434bff3082d141838068f323bdb0d1ed P cb82cc7bd6db46d980c75237a1402860 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:22:13.245733 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31969392ccdb43b491911a319e8bc7e1" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:22:13.258700 23491 raft_consensus.cc:493] T 434bff3082d141838068f323bdb0d1ed P cb82cc7bd6db46d980c75237a1402860 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:22:13.259186 23491 raft_consensus.cc:3060] T 434bff3082d141838068f323bdb0d1ed P cb82cc7bd6db46d980c75237a1402860 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:22:13.259467 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26354361e9b044cea1c2b86c4882fe92" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:22:13.205322 23061 consensus_queue.cc:1048] T 68a6838d82ef4cac83507f906139d90f P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.055s
I20260709 13:22:13.260578 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 23 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:22:13.261350 16149 raft_consensus.cc:2410] T ecc1e363230043fe9fef390f0b979f20 P 647a343a542d4a139fe106c19095129b [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 for term 23 because replica has last-logged OpId of term: 21 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 12.
I20260709 13:22:13.262072 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806d7690c2b644959152074bb2cab0a7" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:22:13.247946 16190 leader_election.cc:304] T 08c74490c40243a88e62d1760b5cd814 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:22:13.261698 23450 raft_consensus.cc:2804] T 2794af5b99984efebdee564035841cb0 P cb82cc7bd6db46d980c75237a1402860 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:22:13.263424 16190 leader_election.cc:304] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860
I20260709 13:22:13.263549 23311 raft_consensus.cc:2804] T 08c74490c40243a88e62d1760b5cd814 P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:22:13.264024 23311 raft_consensus.cc:493] T 08c74490c40243a88e62d1760b5cd814 P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:13.264567 23311 raft_consensus.cc:3060] T 08c74490c40243a88e62d1760b5cd814 P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:22:13.263430 23450 raft_consensus.cc:493] T 2794af5b99984efebdee564035841cb0 P cb82cc7bd6db46d980c75237a1402860 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:22:13.265534 23450 raft_consensus.cc:3060] T 2794af5b99984efebdee564035841cb0 P cb82cc7bd6db46d980c75237a1402860 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:22:13.265610 23173 raft_consensus.cc:3060] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:22:13.246147 23199 raft_consensus.cc:493] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:22:13.267683 23199 raft_consensus.cc:3060] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:22:13.247397 23256 raft_consensus.cc:3060] T ecc1e363230043fe9fef390f0b979f20 P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:22:13.256794 16144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a9de347c711477e87c582d003f36925" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:22:13.227555 23331 raft_consensus.cc:493] T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:22:13.230314 23087 raft_consensus.cc:493] T 68a6838d82ef4cac83507f906139d90f P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:13.263870 16220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f2da98864b4425bf7c0055951fdd4b" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:22:13.250871 16147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0eab25ce4c64e018a041beeda2867db" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:22:13.287577 16218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af57996a8be84d398f0f023e8d08ef77" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:13.288115 16147 raft_consensus.cc:2393] T c0eab25ce4c64e018a041beeda2867db P 647a343a542d4a139fe106c19095129b [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 21: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
I20260709 13:22:13.242065 23408 raft_consensus.cc:2749] T 74ca8140f35c4ae29e8a664f6bdfc829 P cb82cc7bd6db46d980c75237a1402860 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:22:13.276624 16223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c2095a5f4ad49408dbc7f69c606120d" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:13.292250 16223 raft_consensus.cc:2393] T 7c2095a5f4ad49408dbc7f69c606120d P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 17: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
I20260709 13:22:13.274606 16221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b7d1ab519044eda2d5e04a07242f25" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:22:13.273999 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c61a39fd4f84424b129c1a6f9ba404d" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:22:13.293002 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f2da98864b4425bf7c0055951fdd4b" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:22:13.272450 23330 leader_election.cc:290] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:22:13.280148 16296 raft_consensus.cc:2468] T 8638ee363273426db1393dd81e5c7b72 P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 20.
I20260709 13:22:13.227375 23483 raft_consensus.cc:493] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:22:13.281064 16144 raft_consensus.cc:2468] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 23.
I20260709 13:22:13.257211 23347 leader_election.cc:290] T 1df93779209748b9860f6d893dd3de53 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:13.262121 23321 raft_consensus.cc:3055] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:22:13.282428 16219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18ad7e3af5504571a69d10c00750a01c" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:13.288862 16218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdbd78acb4cf46298e0de8ba0d130e19" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:22:13.303478 16218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62fe3346cd3c42c4a55e14af9073bca4" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 27 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:13.290880 16146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6f07842d3aa47858df5665874c86f3a" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:22:13.291041 16220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0eab25ce4c64e018a041beeda2867db" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:22:13.293799 16293 raft_consensus.cc:3060] T 84f2da98864b4425bf7c0055951fdd4b P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:22:13.291236 16147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a878999c0024d2abe517484180290fc" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:22:13.305667 16147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "829b4ad8c5a54eababeea1181b960d4b" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:22:13.306655 16147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a96373bb0e14dce8a266b2fc6c58507" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:22:13.227216 23398 raft_consensus.cc:493] T 3884f864a96d428785895e5463e5728f P cb82cc7bd6db46d980c75237a1402860 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:22:13.301992 16149 raft_consensus.cc:2393] T 9c61a39fd4f84424b129c1a6f9ba404d P 647a343a542d4a139fe106c19095129b [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 21: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
I20260709 13:22:13.233189 23307 raft_consensus.cc:515] T 5a96373bb0e14dce8a266b2fc6c58507 P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:13.234795 23399 raft_consensus.cc:515] T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:13.308529 16147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f5f89186204242bf377f7c42b2e2da" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 23 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:22:13.291106 23354 raft_consensus.cc:493] T b6f07842d3aa47858df5665874c86f3a P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:13.309258 16116 leader_election.cc:304] T 8158bbd17b994d2eb96f560bd0f1b88d P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
I20260709 13:22:13.313301 16220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:13.302469 23441 raft_consensus.cc:493] T ee9ed67006144797b1de6ea8c2013a68 P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:22:13.314778 16221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:13.302717 23437 raft_consensus.cc:493] T 2794af5b99984efebdee564035841cb0 P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:13.299589 23453 raft_consensus.cc:493] T 8cc959fb7f704fba9009c0595467d074 P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:22:13.302230 23483 raft_consensus.cc:3060] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:22:13.302407 23321 raft_consensus.cc:740] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:22:13.331504 16295 raft_consensus.cc:2393] T 5a96373bb0e14dce8a266b2fc6c58507 P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 17: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
I20260709 13:22:13.317526 16218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ad5dbfa99ee4b8d9e28be92b3f36db5" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:13.316000 16144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746aabf856ab401c9ef5e859371e7b67" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:22:13.332501 16817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66f75124215f4ca6bef831e174a792d8" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:13.310178 23398 raft_consensus.cc:3060] T 3884f864a96d428785895e5463e5728f P cb82cc7bd6db46d980c75237a1402860 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:22:13.319658 16220 raft_consensus.cc:2393] T 19baa7bc367844c288bca8f81439c413 P 1ca2725d290e47d68b1a4b609c9816ac [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 23: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
I20260709 13:22:13.331774 16117 leader_election.cc:304] T 3884f864a96d428785895e5463e5728f P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:22:13.336493 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:22:13.337059 16149 raft_consensus.cc:3060] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:22:13.319870 16219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746aabf856ab401c9ef5e859371e7b67" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:13.321081 23354 raft_consensus.cc:515] T b6f07842d3aa47858df5665874c86f3a P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:13.333132 16817 raft_consensus.cc:2468] T 66f75124215f4ca6bef831e174a792d8 P cb82cc7bd6db46d980c75237a1402860 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 18.
I20260709 13:22:13.296968 23331 raft_consensus.cc:515] T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:13.339030 16147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5ad4eb281564936a59f41e387a47156" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
W20260709 13:22:13.339512 16789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 79588.5us
I20260709 13:22:13.337531 16117 leader_election.cc:304] T af57996a8be84d398f0f023e8d08ef77 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860
I20260709 13:22:13.341825 16192 leader_election.cc:304] T 66f75124215f4ca6bef831e174a792d8 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:22:13.300261 23087 raft_consensus.cc:515] T 68a6838d82ef4cac83507f906139d90f P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:13.344879 16220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5718c623ebee4d2bb770f0f9ddb2f164" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:13.323956 23347 raft_consensus.cc:2804] T 8158bbd17b994d2eb96f560bd0f1b88d P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:22:13.343420 16147 raft_consensus.cc:2468] T e5ad4eb281564936a59f41e387a47156 P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 19.
I20260709 13:22:13.319000 23372 raft_consensus.cc:493] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:22:13.348224 16144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d16dfad0c3b540e5b4b86492ddf87208" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:22:13.327829 23474 leader_election.cc:290] T 66f75124215f4ca6bef831e174a792d8 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:13.348469 16147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a96373bb0e14dce8a266b2fc6c58507" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:22:13.349045 16220 raft_consensus.cc:2393] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 20: Already voted for candidate cb82cc7bd6db46d980c75237a1402860 in this term.
I20260709 13:22:13.348842 16218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4f65cdf00b748bb963ee6e81ef43b79" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:13.353794 16146 raft_consensus.cc:2468] T b6f07842d3aa47858df5665874c86f3a P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 19.
I20260709 13:22:13.352174 16220 raft_consensus.cc:1275] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac [term 20 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 18 index: 15. Preceding OpId from leader: term: 20 index: 16. (index mismatch)
I20260709 13:22:13.353564 16147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02838c86531d4f98bf58d1c22b8baf77" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:22:13.352561 23372 raft_consensus.cc:515] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:13.360294 23399 leader_election.cc:290] T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:22:13.361351 23372 leader_election.cc:290] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:13.361933 23354 leader_election.cc:290] T b6f07842d3aa47858df5665874c86f3a P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:13.350088 23327 raft_consensus.cc:2749] T af57996a8be84d398f0f023e8d08ef77 P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:22:13.329141 23441 raft_consensus.cc:515] T ee9ed67006144797b1de6ea8c2013a68 P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:13.364856 23087 leader_election.cc:290] T 68a6838d82ef4cac83507f906139d90f P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:13.350260 23347 raft_consensus.cc:493] T 8158bbd17b994d2eb96f560bd0f1b88d P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:13.366021 23347 raft_consensus.cc:3060] T 8158bbd17b994d2eb96f560bd0f1b88d P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:22:13.366189 23441 leader_election.cc:290] T ee9ed67006144797b1de6ea8c2013a68 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:13.342757 23322 raft_consensus.cc:2804] T 3884f864a96d428785895e5463e5728f P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:22:13.367064 23322 raft_consensus.cc:493] T 3884f864a96d428785895e5463e5728f P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:13.335757 23351 leader_election.cc:290] T 8158bbd17b994d2eb96f560bd0f1b88d P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:22:13.377378 16293 raft_consensus.cc:2468] T 84f2da98864b4425bf7c0055951fdd4b P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 17.
I20260709 13:22:13.368214 16218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5d59407eb545b895d8d922e0dd69fd" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 24 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:13.369685 16817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9944d5e4b8c4109af9955c4f08c9095" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:13.370222 16219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c74490c40243a88e62d1760b5cd814" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 20 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:13.370898 16819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ab0ac8347641e6955e9bce7d90c1a2" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:13.357769 16147 raft_consensus.cc:2468] T 02838c86531d4f98bf58d1c22b8baf77 P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 19.
I20260709 13:22:13.373503 16825 raft_consensus.cc:1240] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 [term 23 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 22. Current term is 23. Ops: []
I20260709 13:22:13.381031 16147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87147731a68400e8cce9b879daa7c5d" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:22:13.381603 16147 raft_consensus.cc:2468] T b87147731a68400e8cce9b879daa7c5d P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 20.
I20260709 13:22:13.382292 16147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cc959fb7f704fba9009c0595467d074" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:22:13.352116 23173 raft_consensus.cc:2749] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:22:13.392318 16144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ad5dbfa99ee4b8d9e28be92b3f36db5" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:22:13.393164 16144 raft_consensus.cc:2393] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 19: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
I20260709 13:22:13.388338 16820 raft_consensus.cc:1275] T 57a744f5aaf944678a63469cbf514015 P cb82cc7bd6db46d980c75237a1402860 [term 17 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 13:22:13.391574 16824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f82224f4564fea93572e8206bad1fe" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:13.331450 23453 raft_consensus.cc:515] T 8cc959fb7f704fba9009c0595467d074 P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:13.353523 23403 raft_consensus.cc:2804] T 66f75124215f4ca6bef831e174a792d8 P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:22:13.367460 23322 raft_consensus.cc:3060] T 3884f864a96d428785895e5463e5728f P 647a343a542d4a139fe106c19095129b [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:22:13.331303 23437 raft_consensus.cc:515] T 2794af5b99984efebdee564035841cb0 P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:13.409864 23403 raft_consensus.cc:493] T 66f75124215f4ca6bef831e174a792d8 P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:13.439754 23453 leader_election.cc:290] T 8cc959fb7f704fba9009c0595467d074 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:13.420094 23307 leader_election.cc:290] T 5a96373bb0e14dce8a266b2fc6c58507 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 17 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:22:13.440237 23347 raft_consensus.cc:515] T 8158bbd17b994d2eb96f560bd0f1b88d P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:13.425390 16816 raft_consensus.cc:2468] T 3a7ca61592e848ea9b0fb81a2ce33cdc P cb82cc7bd6db46d980c75237a1402860 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 20.
I20260709 13:22:13.448261 23409 raft_consensus.cc:493] T 3a7ca61592e848ea9b0fb81a2ce33cdc P cb82cc7bd6db46d980c75237a1402860 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:13.448873 23409 raft_consensus.cc:515] T 3a7ca61592e848ea9b0fb81a2ce33cdc P cb82cc7bd6db46d980c75237a1402860 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:13.453732 16144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68a6838d82ef4cac83507f906139d90f" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:22:13.454519 16144 raft_consensus.cc:2468] T 68a6838d82ef4cac83507f906139d90f P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 19.
I20260709 13:22:13.427768 23199 raft_consensus.cc:515] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:13.472400 16190 leader_election.cc:304] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: cb82cc7bd6db46d980c75237a1402860
I20260709 13:22:13.473482 16296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9944d5e4b8c4109af9955c4f08c9095" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:13.474640 16296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57a744f5aaf944678a63469cbf514015" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
W20260709 13:22:13.475337 16117 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 126265us
I20260709 13:22:13.428093 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a959dbecc2648eb85d1cc550f5a3245" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:13.392016 23321 consensus_queue.cc:260] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 22.17, Last appended by leader: 17, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:13.431512 23331 leader_election.cc:290] T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:13.437357 16148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5d59407eb545b895d8d922e0dd69fd" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 24 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:22:13.439853 23403 raft_consensus.cc:3060] T 66f75124215f4ca6bef831e174a792d8 P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:22:13.442426 16147 raft_consensus.cc:2468] T 8cc959fb7f704fba9009c0595467d074 P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 18.
I20260709 13:22:13.498672 16190 leader_election.cc:304] T 5a959dbecc2648eb85d1cc550f5a3245 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:22:13.499111 16144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f5f89186204242bf377f7c42b2e2da" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:22:13.497730 16220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cc959fb7f704fba9009c0595467d074" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:13.427208 16149 raft_consensus.cc:2468] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 23.
I20260709 13:22:13.502311 16294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 23 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:13.482075 16296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c2095a5f4ad49408dbc7f69c606120d" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:13.503372 16296 raft_consensus.cc:2468] T 7c2095a5f4ad49408dbc7f69c606120d P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 17.
I20260709 13:22:13.504840 16266 leader_election.cc:304] T 8c11074579b046d99e585c2818e18a2c P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860
W20260709 13:22:13.485059 16117 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:22:13.492406 16191 leader_election.cc:304] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
I20260709 13:22:13.506261 16294 raft_consensus.cc:2393] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 23: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
I20260709 13:22:13.506541 16191 leader_election.cc:304] T e4dc6b0541ca431ebc757beac00521f1 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 90f3da57796b488c9087a2f523adf045
I20260709 13:22:13.507256 16266 leader_election.cc:304] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860
I20260709 13:22:13.509233 16191 leader_election.cc:304] T c0eab25ce4c64e018a041beeda2867db P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:22:13.471343 23383 leader_election.cc:290] T 19baa7bc367844c288bca8f81439c413 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:13.510505 16296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5ad4eb281564936a59f41e387a47156" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:13.514313 16144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8158bbd17b994d2eb96f560bd0f1b88d" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:22:13.515062 16144 raft_consensus.cc:2393] T 8158bbd17b994d2eb96f560bd0f1b88d P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 19: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
I20260709 13:22:13.516698 16220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8158bbd17b994d2eb96f560bd0f1b88d" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:13.479465 23433 raft_consensus.cc:493] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:22:13.517445 16220 raft_consensus.cc:2468] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 18.
I20260709 13:22:13.508293 23372 raft_consensus.cc:3060] T 5a959dbecc2648eb85d1cc550f5a3245 P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:22:13.523814 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f82224f4564fea93572e8206bad1fe" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:22:13.499881 23347 leader_election.cc:290] T 8158bbd17b994d2eb96f560bd0f1b88d P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 19 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:13.511792 16266 leader_election.cc:304] T 3884f864a96d428785895e5463e5728f P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 647a343a542d4a139fe106c19095129b
I20260709 13:22:13.521071 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d4ab870a05347868956aa8b841b2981" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:13.504498 22946 raft_consensus.cc:2804] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:22:13.477438 23407 consensus_queue.cc:1048] T 9e65676b21b748728ad39668cfe722ec P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.041s
I20260709 13:22:13.512213 23543 raft_consensus.cc:3060] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:22:13.504727 23173 consensus_queue.cc:1048] T 68a6838d82ef4cac83507f906139d90f P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.029s
I20260709 13:22:13.495016 23382 consensus_queue.cc:1059] T 84f2da98864b4425bf7c0055951fdd4b P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.030s
I20260709 13:22:13.493721 23404 raft_consensus.cc:515] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:13.512279 23061 raft_consensus.cc:3060] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:22:13.512898 23321 raft_consensus.cc:3060] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:22:13.538478 23210 raft_consensus.cc:515] T d2ab0ac8347641e6955e9bce7d90c1a2 P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:22:13.500826 22610 consensus_queue.cc:1175] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "cb82cc7bd6db46d980c75237a1402860" responder_term: 23 status { last_received { term: 20 index: 16 } last_committed_idx: 16 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 22. Current term is 23. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:22:13.530797 16293 raft_consensus.cc:2393] T 8d4ab870a05347868956aa8b841b2981 P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 19: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
I20260709 13:22:13.543936 16820 raft_consensus.cc:1275] T 66f75124215f4ca6bef831e174a792d8 P cb82cc7bd6db46d980c75237a1402860 [term 18 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 13:22:13.623908 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a96373bb0e14dce8a266b2fc6c58507" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:22:13.508394 23077 raft_consensus.cc:2749] T 8c11074579b046d99e585c2818e18a2c P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:22:13.545632 16817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee9ed67006144797b1de6ea8c2013a68" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:13.547370 16819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31969392ccdb43b491911a319e8bc7e1" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:13.548076 16789 leader_election.cc:304] T c9944d5e4b8c4109af9955c4f08c9095 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
W20260709 13:22:13.634238 16789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 86262us
I20260709 13:22:13.516240 23474 raft_consensus.cc:2804] T e4dc6b0541ca431ebc757beac00521f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:22:13.586819 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c2095a5f4ad49408dbc7f69c606120d" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:22:13.497704 23483 raft_consensus.cc:515] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:22:13.614837 16266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 103165us
I20260709 13:22:13.615291 16144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26354361e9b044cea1c2b86c4882fe92" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:22:13.619721 16824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62fe3346cd3c42c4a55e14af9073bca4" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 27 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:13.648488 16819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746aabf856ab401c9ef5e859371e7b67" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
W20260709 13:22:13.620473 16117 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 87661us
I20260709 13:22:13.614897 23311 raft_consensus.cc:515] T 08c74490c40243a88e62d1760b5cd814 P 1ca2725d290e47d68b1a4b609c9816ac [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:13.622287 23533 raft_consensus.cc:2804] T 3884f864a96d428785895e5463e5728f P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:22:13.534423 23024 leader_election.cc:290] T 434bff3082d141838068f323bdb0d1ed P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:13.617197 16151 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "08c74490c40243a88e62d1760b5cd814"
dest_uuid: "647a343a542d4a139fe106c19095129b"
mode: GRACEFUL
new_leader_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
 from {username='slave'} at 127.0.0.1:40290
I20260709 13:22:13.652302 16151 raft_consensus.cc:606] T 08c74490c40243a88e62d1760b5cd814 P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Received request to transfer leadership to TS 1ca2725d290e47d68b1a4b609c9816ac
I20260709 13:22:13.614085 23383 raft_consensus.cc:493] T 829b4ad8c5a54eababeea1181b960d4b P cb82cc7bd6db46d980c75237a1402860 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:22:13.652704 16151 raft_consensus.cc:614] T 08c74490c40243a88e62d1760b5cd814 P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:22:13.643221 16149 raft_consensus.cc:2468] T 7c2095a5f4ad49408dbc7f69c606120d P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 16.
I20260709 13:22:13.653337 16299 raft_consensus.cc:2368] T d2ab0ac8347641e6955e9bce7d90c1a2 P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac for earlier term 19. Current term is 20.
I20260709 13:22:13.646883 16266 leader_election.cc:304] T 26354361e9b044cea1c2b86c4882fe92 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:22:13.656476 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdbd78acb4cf46298e0de8ba0d130e19" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:22:13.665094 16266 leader_election.cc:304] T a378d7f796f94615b3b347119a9880fd P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860
I20260709 13:22:13.666291 16266 leader_election.cc:304] T 66ae176961b74813a58bc15c9c4c6f2a P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860
I20260709 13:22:13.666529 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8158bbd17b994d2eb96f560bd0f1b88d" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:22:13.667413 16150 raft_consensus.cc:2393] T 8158bbd17b994d2eb96f560bd0f1b88d P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 19: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
I20260709 13:22:13.667866 23542 raft_consensus.cc:2749] T a378d7f796f94615b3b347119a9880fd P 90f3da57796b488c9087a2f523adf045 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:22:13.668231 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a96373bb0e14dce8a266b2fc6c58507" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:22:13.668627 23542 raft_consensus.cc:2749] T 66ae176961b74813a58bc15c9c4c6f2a P 90f3da57796b488c9087a2f523adf045 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 13:22:13.670166 16266 leader_election.cc:304] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:22:13.528815 23433 raft_consensus.cc:515] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } }
I20260709 13:22:13.634774 23326 raft_consensus.cc:493] T fffab26427e24d398e2695aad40ca134 P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:13.674799 16150 raft_consensus.cc:1240] T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b [term 18 LEADER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 17. Current term is 18. Ops: []
I20260709 13:22:13.640350 23408 raft_consensus.cc:2749] T c9944d5e4b8c4109af9955c4f08c9095 P cb82cc7bd6db46d980c75237a1402860 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:22:13.727882 23450 raft_consensus.cc:515] T 2794af5b99984efebdee564035841cb0 P cb82cc7bd6db46d980c75237a1402860 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:13.729882 23287 consensus_queue.cc:1048] T 57a744f5aaf944678a63469cbf514015 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:22:13.730546 23450 leader_election.cc:290] T 2794af5b99984efebdee564035841cb0 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 18 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:13.731038 16142 raft_consensus.cc:1275] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 13:22:13.732271 23256 raft_consensus.cc:2749] T ecc1e363230043fe9fef390f0b979f20 P 1ca2725d290e47d68b1a4b609c9816ac [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:22:13.732513 16142 raft_consensus.cc:1275] T fffab26427e24d398e2695aad40ca134 P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 13:22:13.681856 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434bff3082d141838068f323bdb0d1ed" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:22:13.733700 16221 raft_consensus.cc:2468] T ecc1e363230043fe9fef390f0b979f20 P 1ca2725d290e47d68b1a4b609c9816ac [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 22.
I20260709 13:22:13.734899 16149 raft_consensus.cc:2468] T 434bff3082d141838068f323bdb0d1ed P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 19.
I20260709 13:22:13.640671 23474 raft_consensus.cc:493] T e4dc6b0541ca431ebc757beac00521f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 20 FOLLOWER]: Starting leader election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:22:13.677644 16265 leader_election.cc:304] T 31969392ccdb43b491911a319e8bc7e1 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
I20260709 13:22:13.741842 23474 raft_consensus.cc:3060] T e4dc6b0541ca431ebc757beac00521f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:22:13.742301 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 65287.9us
I20260709 13:22:13.733736 23306 raft_consensus.cc:3055] T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:22:13.743544 23306 raft_consensus.cc:740] T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:22:13.743659 16265 leader_election.cc:304] T ecc1e363230043fe9fef390f0b979f20 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:22:13.744382 23306 consensus_queue.cc:260] T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.15, Last appended by leader: 15, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:13.744933 16265 leader_election.cc:304] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:22:13.745393 23306 raft_consensus.cc:3060] T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:22:13.645247 23351 raft_consensus.cc:493] T 8d4ab870a05347868956aa8b841b2981 P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:13.746022 23351 raft_consensus.cc:515] T 8d4ab870a05347868956aa8b841b2981 P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:13.688458 23543 raft_consensus.cc:2749] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:22:13.692850 23235 raft_consensus.cc:3055] T ada5a004ba7c46819051e3ccec71ffdb P 90f3da57796b488c9087a2f523adf045 [term 20 LEADER]: Stepping down as leader of term 20
W20260709 13:22:13.697665 16226 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33050 (request call id 1260) took 1662 ms (client timeout 2149 ms). Trace:
W20260709 13:22:13.751549 16226 rpcz_store.cc:269] 0709 13:22:12.035614 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:22:12.038993 (+  3379us) service_pool.cc:224] Handling call
0709 13:22:13.697380 (+1658387us) spinlock_profiling.cc:243] Waited 1.17 s on lock 0x7b54001428d4. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:22:13.697586 (+   206us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2683404672}
I20260709 13:22:13.701026 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87ac0d1f7e8a419f9bcbc04aee0e067e" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:13.703450 16146 raft_consensus.cc:1275] T 5a959dbecc2648eb85d1cc550f5a3245 P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 13:22:13.704926 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87ac0d1f7e8a419f9bcbc04aee0e067e" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:13.754448 16300 raft_consensus.cc:2468] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 20.
I20260709 13:22:13.705379 16151 raft_consensus.cc:1275] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 18 index: 15. Preceding OpId from leader: term: 20 index: 16. (index mismatch)
I20260709 13:22:13.677475 23077 raft_consensus.cc:2804] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:22:13.653414 22677 leader_election.cc:290] T 746aabf856ab401c9ef5e859371e7b67 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:13.707271 16219 raft_consensus.cc:2393] T 08c74490c40243a88e62d1760b5cd814 P 1ca2725d290e47d68b1a4b609c9816ac [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 20: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
I20260709 13:22:13.710410 23199 leader_election.cc:290] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 19 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:13.713198 23404 leader_election.cc:290] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 23 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:22:13.713578 23546 consensus_queue.cc:1048] T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.031s
I20260709 13:22:13.720911 16816 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49906 (request call id 1379) took 1531 ms. Trace:
I20260709 13:22:13.653687 23533 raft_consensus.cc:493] T 3884f864a96d428785895e5463e5728f P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:13.723460 23173 raft_consensus.cc:3060] T c0eab25ce4c64e018a041beeda2867db P 1ca2725d290e47d68b1a4b609c9816ac [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:22:13.724489 23437 leader_election.cc:290] T 2794af5b99984efebdee564035841cb0 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:13.725193 23407 consensus_queue.cc:1059] T cdbd78acb4cf46298e0de8ba0d130e19 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: INVALID_TERM, Last received: 22.17, Next index: 18, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 13:22:13.734253 16790 leader_election.cc:304] T 9e65676b21b748728ad39668cfe722ec P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:22:13.723492 16145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e9b2bf7011f4ae688a79d7bb5a447b4" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:22:13.736342 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02838c86531d4f98bf58d1c22b8baf77" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:13.776058 16293 raft_consensus.cc:2468] T 02838c86531d4f98bf58d1c22b8baf77 P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 20.
I20260709 13:22:13.776749 16790 leader_election.cc:304] T 57a744f5aaf944678a63469cbf514015 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860; no voters: 90f3da57796b488c9087a2f523adf045
I20260709 13:22:13.777128 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e9b2bf7011f4ae688a79d7bb5a447b4" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:22:13.778596 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9944d5e4b8c4109af9955c4f08c9095" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:13.737126 16817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2794af5b99984efebdee564035841cb0" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 19 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:13.779982 16817 raft_consensus.cc:2410] T 2794af5b99984efebdee564035841cb0 P cb82cc7bd6db46d980c75237a1402860 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b for term 19 because replica has last-logged OpId of term: 17 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 14.
I20260709 13:22:13.738526 16144 raft_consensus.cc:1240] T c0eab25ce4c64e018a041beeda2867db P 647a343a542d4a139fe106c19095129b [term 21 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 20. Current term is 21. Ops: [20.11-20.11]
I20260709 13:22:13.782114 16144 raft_consensus.cc:1275] T af57996a8be84d398f0f023e8d08ef77 P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 13:22:13.783303 16144 raft_consensus.cc:1275] T 18ad7e3af5504571a69d10c00750a01c P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
W20260709 13:22:13.789098 23537 log.cc:927] Time spent T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.285s	user 0.002s	sys 0.000s
I20260709 13:22:13.790747 16149 raft_consensus.cc:1275] T b6f07842d3aa47858df5665874c86f3a P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 13:22:13.791599 23199 raft_consensus.cc:2749] T 9e65676b21b748728ad39668cfe722ec P cb82cc7bd6db46d980c75237a1402860 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:22:13.668238 23403 raft_consensus.cc:515] T 66f75124215f4ca6bef831e174a792d8 P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:13.795990 23210 leader_election.cc:290] T d2ab0ac8347641e6955e9bce7d90c1a2 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 20 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
W20260709 13:22:13.796581 23491 consensus_meta.cc:287] T 434bff3082d141838068f323bdb0d1ed P cb82cc7bd6db46d980c75237a1402860: Time spent flushing consensus metadata: real 0.537s	user 0.008s	sys 0.003s
I20260709 13:22:13.796957 23403 leader_election.cc:290] T 66f75124215f4ca6bef831e174a792d8 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 19 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:13.751317 23235 raft_consensus.cc:740] T ada5a004ba7c46819051e3ccec71ffdb P 90f3da57796b488c9087a2f523adf045 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:22:13.797029 23491 raft_consensus.cc:515] T 434bff3082d141838068f323bdb0d1ed P cb82cc7bd6db46d980c75237a1402860 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:13.798185 23235 consensus_queue.cc:260] T ada5a004ba7c46819051e3ccec71ffdb P 90f3da57796b488c9087a2f523adf045 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 15, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:13.772938 23533 raft_consensus.cc:3060] T 3884f864a96d428785895e5463e5728f P 90f3da57796b488c9087a2f523adf045 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:22:13.790975 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3884f864a96d428785895e5463e5728f" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 16 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:13.781658 23543 raft_consensus.cc:2749] T 31969392ccdb43b491911a319e8bc7e1 P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:22:13.805787 23404 raft_consensus.cc:2749] T ecc1e363230043fe9fef390f0b979f20 P 90f3da57796b488c9087a2f523adf045 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:22:13.806525 23404 raft_consensus.cc:2749] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:22:13.772374 22677 raft_consensus.cc:493] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:13.807133 23404 raft_consensus.cc:493] T 1df93779209748b9860f6d893dd3de53 P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:13.807694 23404 raft_consensus.cc:515] T 1df93779209748b9860f6d893dd3de53 P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:13.810000 23404 leader_election.cc:290] T 1df93779209748b9860f6d893dd3de53 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:22:13.810708 23404 raft_consensus.cc:3055] T cdbd78acb4cf46298e0de8ba0d130e19 P 90f3da57796b488c9087a2f523adf045 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:22:13.735785 23326 raft_consensus.cc:515] T fffab26427e24d398e2695aad40ca134 P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:13.811297 23495 raft_consensus.cc:2804] T 57a744f5aaf944678a63469cbf514015 P cb82cc7bd6db46d980c75237a1402860 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:22:13.811750 23495 raft_consensus.cc:493] T 57a744f5aaf944678a63469cbf514015 P cb82cc7bd6db46d980c75237a1402860 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:13.812067 23173 raft_consensus.cc:2749] T c0eab25ce4c64e018a041beeda2867db P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:22:13.812914 23326 leader_election.cc:290] T fffab26427e24d398e2695aad40ca134 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:22:13.813863 16296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:13.814208 23524 leader_election.cc:290] T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:13.814925 16217 raft_consensus.cc:1240] T 66f75124215f4ca6bef831e174a792d8 P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 18. Current term is 19. Ops: []
W20260709 13:22:13.748412 16115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 258064us
I20260709 13:22:13.654453 22946 raft_consensus.cc:697] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [term 20 LEADER]: Becoming Leader. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:22:13.817483 23474 raft_consensus.cc:515] T e4dc6b0541ca431ebc757beac00521f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:13.818033 22946 consensus_queue.cc:237] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:13.674063 23024 raft_consensus.cc:2804] T 26354361e9b044cea1c2b86c4882fe92 P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:22:13.820220 23024 raft_consensus.cc:493] T 26354361e9b044cea1c2b86c4882fe92 P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:22:13.820664 23024 raft_consensus.cc:3060] T 26354361e9b044cea1c2b86c4882fe92 P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:22:13.671451 23383 raft_consensus.cc:515] T 829b4ad8c5a54eababeea1181b960d4b P cb82cc7bd6db46d980c75237a1402860 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:13.824077 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2794af5b99984efebdee564035841cb0" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:22:13.824685 23286 raft_consensus.cc:493] T fffab26427e24d398e2695aad40ca134 P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:13.825457 23383 leader_election.cc:290] T 829b4ad8c5a54eababeea1181b960d4b P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:22:13.825213 23286 raft_consensus.cc:515] T fffab26427e24d398e2695aad40ca134 P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:13.784765 23372 raft_consensus.cc:2749] T 5a959dbecc2648eb85d1cc550f5a3245 P 1ca2725d290e47d68b1a4b609c9816ac [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:22:13.722749 23398 raft_consensus.cc:515] T 3884f864a96d428785895e5463e5728f P cb82cc7bd6db46d980c75237a1402860 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:13.827937 23286 leader_election.cc:290] T fffab26427e24d398e2695aad40ca134 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:13.828723 23398 leader_election.cc:290] T 3884f864a96d428785895e5463e5728f P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 15 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:13.832209 16144 raft_consensus.cc:1275] T 57a744f5aaf944678a63469cbf514015 P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 13:22:13.834118 16222 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57600 (request call id 2132) took 1189 ms. Trace:
I20260709 13:22:13.836611 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ad5dbfa99ee4b8d9e28be92b3f36db5" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:13.834533 16222 rpcz_store.cc:276] 0709 13:22:12.644965 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:22:12.646406 (+  1441us) service_pool.cc:224] Handling call
0709 13:22:13.833953 (+1187547us) spinlock_profiling.cc:243] Waited 373 ms on lock 0x7b54000154d4. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:22:13.834063 (+   110us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":857910144}
W20260709 13:22:13.756687 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 247528us
I20260709 13:22:13.766955 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18ad7e3af5504571a69d10c00750a01c" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:13.768139 16142 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1df93779209748b9860f6d893dd3de53"
dest_uuid: "647a343a542d4a139fe106c19095129b"
mode: GRACEFUL
new_leader_uuid: "90f3da57796b488c9087a2f523adf045"
 from {username='slave'} at 127.0.0.1:40290
I20260709 13:22:13.839079 16115 leader_election.cc:304] T e5ad4eb281564936a59f41e387a47156 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:22:13.839365 16142 raft_consensus.cc:606] T 1df93779209748b9860f6d893dd3de53 P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Received request to transfer leadership to TS 90f3da57796b488c9087a2f523adf045
I20260709 13:22:13.839720 16142 raft_consensus.cc:614] T 1df93779209748b9860f6d893dd3de53 P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:22:13.770066 16816 rpcz_store.cc:276] 0709 13:22:12.189938 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:22:12.255524 (+ 65586us) service_pool.cc:224] Handling call
0709 13:22:13.720757 (+1465233us) spinlock_profiling.cc:243] Waited 1.14 s on lock 0x7b54006441d4. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:22:13.720856 (+    99us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2617201024}
I20260709 13:22:13.840062 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 23 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:13.841202 16191 leader_election.cc:304] T 8a878999c0024d2abe517484180290fc P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:22:13.842968 16148 raft_consensus.cc:2410] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 for term 24 because replica has last-logged OpId of term: 22 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 16.
I20260709 13:22:13.844538 16144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2794af5b99984efebdee564035841cb0" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:22:13.847147 16145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3884f864a96d428785895e5463e5728f" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 15 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:22:13.849152 16144 raft_consensus.cc:2393] T 2794af5b99984efebdee564035841cb0 P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 in current term 18: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
I20260709 13:22:13.852311 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1df93779209748b9860f6d893dd3de53" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:13.859061 16300 raft_consensus.cc:2468] T 1df93779209748b9860f6d893dd3de53 P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 18.
I20260709 13:22:13.860214 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1df93779209748b9860f6d893dd3de53" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:13.860495 16191 leader_election.cc:304] T 829b4ad8c5a54eababeea1181b960d4b P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
I20260709 13:22:13.861019 16300 raft_consensus.cc:2468] T 1df93779209748b9860f6d893dd3de53 P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 18.
I20260709 13:22:13.812126 23495 raft_consensus.cc:3060] T 57a744f5aaf944678a63469cbf514015 P cb82cc7bd6db46d980c75237a1402860 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:22:13.862108 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f5f89186204242bf377f7c42b2e2da" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:13.834264 23491 leader_election.cc:290] T 434bff3082d141838068f323bdb0d1ed P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 19 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:13.868338 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "829b4ad8c5a54eababeea1181b960d4b" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:22:13.867930 16151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434bff3082d141838068f323bdb0d1ed" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:22:13.869647 16299 raft_consensus.cc:2468] T 2794af5b99984efebdee564035841cb0 P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 18.
I20260709 13:22:13.867506 16300 raft_consensus.cc:2368] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac for earlier term 22. Current term is 23.
I20260709 13:22:13.849386 23384 raft_consensus.cc:2749] T 8a878999c0024d2abe517484180290fc P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:22:13.873064 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6f07842d3aa47858df5665874c86f3a" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:13.862325 16191 leader_election.cc:304] T e5ad4eb281564936a59f41e387a47156 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
I20260709 13:22:13.871304 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8158bbd17b994d2eb96f560bd0f1b88d" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:13.874218 16299 raft_consensus.cc:2468] T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 19.
I20260709 13:22:13.848474 23023 raft_consensus.cc:2749] T e5ad4eb281564936a59f41e387a47156 P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:22:13.850459 23399 raft_consensus.cc:493] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:13.807631 22677 raft_consensus.cc:515] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:13.875901 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4f65cdf00b748bb963ee6e81ef43b79" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:13.877100 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c2095a5f4ad49408dbc7f69c606120d" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:13.877473 16191 leader_election.cc:304] T d16dfad0c3b540e5b4b86492ddf87208 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:22:13.863727 23372 raft_consensus.cc:2749] T 829b4ad8c5a54eababeea1181b960d4b P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:22:13.887455 16299 raft_consensus.cc:2468] T 7c2095a5f4ad49408dbc7f69c606120d P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 17.
I20260709 13:22:13.892231 16191 leader_election.cc:304] T 02838c86531d4f98bf58d1c22b8baf77 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
I20260709 13:22:13.888068 16115 leader_election.cc:304] T 9c61a39fd4f84424b129c1a6f9ba404d P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:22:13.811097 23404 raft_consensus.cc:740] T cdbd78acb4cf46298e0de8ba0d130e19 P 90f3da57796b488c9087a2f523adf045 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:22:13.866611 23394 raft_consensus.cc:493] T d2ab0ac8347641e6955e9bce7d90c1a2 P cb82cc7bd6db46d980c75237a1402860 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:22:13.899314 16192 leader_election.cc:304] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 90f3da57796b488c9087a2f523adf045
I20260709 13:22:13.900501 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68a6838d82ef4cac83507f906139d90f" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:13.901191 16293 raft_consensus.cc:2468] T 68a6838d82ef4cac83507f906139d90f P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 19.
I20260709 13:22:13.849845 23398 consensus_queue.cc:1048] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.024s
I20260709 13:22:13.905578 16115 leader_election.cc:304] T ecc1e363230043fe9fef390f0b979f20 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:22:13.895237 23384 raft_consensus.cc:2749] T d16dfad0c3b540e5b4b86492ddf87208 P 1ca2725d290e47d68b1a4b609c9816ac [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:22:13.892889 23173 raft_consensus.cc:2804] T e5ad4eb281564936a59f41e387a47156 P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:22:13.906883 23474 leader_election.cc:290] T e4dc6b0541ca431ebc757beac00521f1 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 21 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:13.901019 22610 raft_consensus.cc:2749] T 9c61a39fd4f84424b129c1a6f9ba404d P 647a343a542d4a139fe106c19095129b [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:22:13.901232 23403 raft_consensus.cc:2804] T 02838c86531d4f98bf58d1c22b8baf77 P 1ca2725d290e47d68b1a4b609c9816ac [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:22:13.895519 23523 consensus_queue.cc:1059] T 8d4ab870a05347868956aa8b841b2981 P 647a343a542d4a139fe106c19095129b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: INVALID_TERM, Last received: 18.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.024s
I20260709 13:22:13.904214 23394 raft_consensus.cc:515] T d2ab0ac8347641e6955e9bce7d90c1a2 P cb82cc7bd6db46d980c75237a1402860 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:13.898797 23351 leader_election.cc:290] T 8d4ab870a05347868956aa8b841b2981 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:22:13.895677 16819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66f75124215f4ca6bef831e174a792d8" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:22:13.895834 16151 raft_consensus.cc:2468] T 434bff3082d141838068f323bdb0d1ed P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 19.
I20260709 13:22:13.896291 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c74490c40243a88e62d1760b5cd814" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 20 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:13.896792 16790 leader_election.cc:304] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:22:13.897790 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26354361e9b044cea1c2b86c4882fe92" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:13.911125 16221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "829b4ad8c5a54eababeea1181b960d4b" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:13.935742 16816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5718c623ebee4d2bb770f0f9ddb2f164" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 21 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:13.923939 16298 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40474 (request call id 1337) took 1023 ms. Trace:
I20260709 13:22:13.925388 16300 raft_consensus.cc:2410] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b for term 20 because replica has last-logged OpId of term: 18 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 10.
I20260709 13:22:13.926465 16790 leader_election.cc:304] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:22:13.924360 16819 raft_consensus.cc:3060] T 66f75124215f4ca6bef831e174a792d8 P cb82cc7bd6db46d980c75237a1402860 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:22:13.953629 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e65676b21b748728ad39668cfe722ec" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:22:13.946969 16298 rpcz_store.cc:276] 0709 13:22:12.900340 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:22:12.901182 (+   842us) service_pool.cc:224] Handling call
0709 13:22:13.923758 (+1022576us) spinlock_profiling.cc:243] Waited 60.7 ms on lock 0x7b540035fcd4. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:22:13.923882 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":139558784}
I20260709 13:22:13.961105 16300 raft_consensus.cc:1275] T fffab26427e24d398e2695aad40ca134 P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 13:22:13.966818 16291 raft_consensus.cc:1240] T 84f2da98864b4425bf7c0055951fdd4b P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 16. Current term is 17. Ops: [16.10-16.10]
I20260709 13:22:13.971024 16300 raft_consensus.cc:1275] T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
W20260709 13:22:14.039958 16266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 288851us
W20260709 13:22:14.041945 16115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 136490us
I20260709 13:22:14.042785 16115 leader_election.cc:304] T 84f2da98864b4425bf7c0055951fdd4b P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac
I20260709 13:22:14.043152 16300 raft_consensus.cc:1240] T 5a96373bb0e14dce8a266b2fc6c58507 P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 16. Current term is 17. Ops: []
I20260709 13:22:14.043800 16817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434bff3082d141838068f323bdb0d1ed" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
W20260709 13:22:14.045414 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 82326.8us
I20260709 13:22:14.046466 16790 leader_election.cc:304] T 66f75124215f4ca6bef831e174a792d8 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:22:14.047912 16825 raft_consensus.cc:1240] T 9e65676b21b748728ad39668cfe722ec P cb82cc7bd6db46d980c75237a1402860 [term 20 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 19. Current term is 20. Ops: [19.15-19.15]
I20260709 13:22:14.048851 16190 leader_election.cc:304] T 19baa7bc367844c288bca8f81439c413 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:22:14.044790 16817 raft_consensus.cc:2468] T 434bff3082d141838068f323bdb0d1ed P cb82cc7bd6db46d980c75237a1402860 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 19.
I20260709 13:22:14.054143 16190 leader_election.cc:304] T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:22:14.054700 16820 raft_consensus.cc:1240] T 74ca8140f35c4ae29e8a664f6bdfc829 P cb82cc7bd6db46d980c75237a1402860 [term 22 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 21. Current term is 22. Ops: [21.15-21.15]
I20260709 13:22:14.043869 16115 leader_election.cc:304] T 7c2095a5f4ad49408dbc7f69c606120d P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
W20260709 13:22:14.061785 16191 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:22:14.074751 16190 leader_election.cc:400] T d2ab0ac8347641e6955e9bce7d90c1a2 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 19 election: Vote denied by peer 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389) with higher term. Message: Invalid argument: T d2ab0ac8347641e6955e9bce7d90c1a2 P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac for earlier term 19. Current term is 20.
I20260709 13:22:14.074818 16151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746aabf856ab401c9ef5e859371e7b67" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:22:14.075277 16190 leader_election.cc:403] T d2ab0ac8347641e6955e9bce7d90c1a2 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 19 election: Cancelling election due to peer responding with higher term
I20260709 13:22:13.917018 22677 leader_election.cc:290] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:13.918655 23403 raft_consensus.cc:493] T 02838c86531d4f98bf58d1c22b8baf77 P 1ca2725d290e47d68b1a4b609c9816ac [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:13.921926 23394 leader_election.cc:290] T d2ab0ac8347641e6955e9bce7d90c1a2 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:13.895872 23399 raft_consensus.cc:515] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:14.082837 16823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e9b2bf7011f4ae688a79d7bb5a447b4" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:22:14.080567 16816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ab0ac8347641e6955e9bce7d90c1a2" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:22:13.958413 23450 raft_consensus.cc:2749] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 13:22:14.083947 16816 raft_consensus.cc:3060] T d2ab0ac8347641e6955e9bce7d90c1a2 P cb82cc7bd6db46d980c75237a1402860 [term 18 FOLLOWER]: Advancing to term 20
I20260709 13:22:13.912395 23173 raft_consensus.cc:493] T e5ad4eb281564936a59f41e387a47156 P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Starting leader election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:22:13.959574 23408 raft_consensus.cc:2749] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:22:13.960340 23384 raft_consensus.cc:2804] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 1ca2725d290e47d68b1a4b609c9816ac [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:22:14.088845 16142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66f75124215f4ca6bef831e174a792d8" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:22:14.072753 23061 raft_consensus.cc:2749] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:22:14.078430 23086 raft_consensus.cc:2804] T 7c2095a5f4ad49408dbc7f69c606120d P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:22:14.074111 23023 raft_consensus.cc:2804] T ecc1e363230043fe9fef390f0b979f20 P 647a343a542d4a139fe106c19095129b [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:22:14.078126 23491 consensus_queue.cc:1048] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.025s
I20260709 13:22:14.093333 16266 leader_election.cc:304] T 434bff3082d141838068f323bdb0d1ed P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:22:14.073400 23453 raft_consensus.cc:2804] T 84f2da98864b4425bf7c0055951fdd4b P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:22:14.078768 23552 raft_consensus.cc:2749] T 19baa7bc367844c288bca8f81439c413 P 1ca2725d290e47d68b1a4b609c9816ac [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:22:14.106997 16142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5718c623ebee4d2bb770f0f9ddb2f164" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 21 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:22:14.108088 16142 raft_consensus.cc:2468] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 20.
W20260709 13:22:14.103312 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 358429us
I20260709 13:22:14.103488 16151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4dc6b0541ca431ebc757beac00521f1" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:22:14.079871 23357 raft_consensus.cc:493] T 5a959dbecc2648eb85d1cc550f5a3245 P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:14.110937 16265 leader_election.cc:304] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: cb82cc7bd6db46d980c75237a1402860
I20260709 13:22:14.082964 23403 raft_consensus.cc:3060] T 02838c86531d4f98bf58d1c22b8baf77 P 1ca2725d290e47d68b1a4b609c9816ac [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:22:14.090554 23173 raft_consensus.cc:3060] T e5ad4eb281564936a59f41e387a47156 P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:22:14.106434 23322 consensus_meta.cc:287] T 3884f864a96d428785895e5463e5728f P 647a343a542d4a139fe106c19095129b: Time spent flushing consensus metadata: real 0.660s	user 0.007s	sys 0.003s
I20260709 13:22:14.104607 23023 raft_consensus.cc:493] T ecc1e363230043fe9fef390f0b979f20 P 647a343a542d4a139fe106c19095129b [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:14.093999 23384 raft_consensus.cc:493] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 1ca2725d290e47d68b1a4b609c9816ac [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:14.096789 22677 raft_consensus.cc:3060] T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:22:14.125254 16142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66f75124215f4ca6bef831e174a792d8" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:22:14.122792 16151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c61a39fd4f84424b129c1a6f9ba404d" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:22:14.133940 16151 raft_consensus.cc:2468] T 9c61a39fd4f84424b129c1a6f9ba404d P 647a343a542d4a139fe106c19095129b [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 21.
I20260709 13:22:14.135318 16151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c74490c40243a88e62d1760b5cd814" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:22:14.136103 16151 raft_consensus.cc:3060] T 08c74490c40243a88e62d1760b5cd814 P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:22:14.105765 23086 raft_consensus.cc:493] T 7c2095a5f4ad49408dbc7f69c606120d P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Starting leader election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:22:14.132493 16265 leader_election.cc:304] T b87147731a68400e8cce9b879daa7c5d P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: cb82cc7bd6db46d980c75237a1402860
I20260709 13:22:14.141336 16816 raft_consensus.cc:2468] T d2ab0ac8347641e6955e9bce7d90c1a2 P cb82cc7bd6db46d980c75237a1402860 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 20.
I20260709 13:22:14.122892 23453 raft_consensus.cc:697] T 84f2da98864b4425bf7c0055951fdd4b P 647a343a542d4a139fe106c19095129b [term 17 LEADER]: Becoming Leader. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:22:14.122174 23553 raft_consensus.cc:2804] T 434bff3082d141838068f323bdb0d1ed P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:22:14.129212 23210 raft_consensus.cc:2804] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:22:14.152586 16265 leader_election.cc:304] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:22:14.157922 16151 raft_consensus.cc:2468] T 08c74490c40243a88e62d1760b5cd814 P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 20.
I20260709 13:22:14.089761 23404 consensus_queue.cc:260] T cdbd78acb4cf46298e0de8ba0d130e19 P 90f3da57796b488c9087a2f523adf045 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 22.17, Last appended by leader: 17, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:14.136628 23023 raft_consensus.cc:3060] T ecc1e363230043fe9fef390f0b979f20 P 647a343a542d4a139fe106c19095129b [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:22:14.138473 23384 raft_consensus.cc:3060] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 1ca2725d290e47d68b1a4b609c9816ac [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:22:14.126332 23407 consensus_queue.cc:1059] T 9e65676b21b748728ad39668cfe722ec P 90f3da57796b488c9087a2f523adf045 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: INVALID_TERM, Last received: 17.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.024s
I20260709 13:22:14.134397 23554 raft_consensus.cc:3060] T d2ab0ac8347641e6955e9bce7d90c1a2 P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:22:14.139458 23086 raft_consensus.cc:3060] T 7c2095a5f4ad49408dbc7f69c606120d P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:22:14.118697 23357 raft_consensus.cc:515] T 5a959dbecc2648eb85d1cc550f5a3245 P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:14.153730 16265 leader_election.cc:304] T 68a6838d82ef4cac83507f906139d90f P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:22:14.130066 23322 raft_consensus.cc:515] T 3884f864a96d428785895e5463e5728f P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:14.162170 23326 raft_consensus.cc:2804] T b87147731a68400e8cce9b879daa7c5d P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:22:14.187244 23326 raft_consensus.cc:493] T b87147731a68400e8cce9b879daa7c5d P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Starting leader election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:22:14.187773 23326 raft_consensus.cc:3060] T b87147731a68400e8cce9b879daa7c5d P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:22:14.152206 23553 raft_consensus.cc:493] T 434bff3082d141838068f323bdb0d1ed P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:14.189828 23399 leader_election.cc:290] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:22:14.152869 23210 raft_consensus.cc:493] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [term 23 FOLLOWER]: Starting leader election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:22:14.191808 23210 raft_consensus.cc:3060] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:22:14.196233 16145 raft_consensus.cc:2368] T 3884f864a96d428785895e5463e5728f P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 for earlier term 15. Current term is 16.
I20260709 13:22:14.197233 23322 leader_election.cc:290] T 3884f864a96d428785895e5463e5728f P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 16 election: Requested vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:14.185012 16265 leader_election.cc:304] T 23f82224f4564fea93572e8206bad1fe P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
I20260709 13:22:14.185616 23559 raft_consensus.cc:2804] T 68a6838d82ef4cac83507f906139d90f P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:22:14.198870 23559 raft_consensus.cc:493] T 68a6838d82ef4cac83507f906139d90f P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:22:14.199379 23559 raft_consensus.cc:3060] T 68a6838d82ef4cac83507f906139d90f P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:22:14.203399 23558 raft_consensus.cc:2749] T 23f82224f4564fea93572e8206bad1fe P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:22:14.189944 23553 raft_consensus.cc:3060] T 434bff3082d141838068f323bdb0d1ed P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:22:14.208391 16149 raft_consensus.cc:1275] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [term 23 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 22 index: 17. Preceding OpId from leader: term: 23 index: 17. (term mismatch)
I20260709 13:22:14.208837 16149 pending_rounds.cc:85] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b: Aborting all ops after (but not including) 16
I20260709 13:22:14.209084 16149 pending_rounds.cc:107] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b: Aborting uncommitted NO_OP operation due to leader change: 22.17
I20260709 13:22:14.209360 16149 raft_consensus.cc:2889] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [term 23 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:22:14.207206 16265 leader_election.cc:304] T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac
I20260709 13:22:14.183574 23407 raft_consensus.cc:3055] T 9e65676b21b748728ad39668cfe722ec P 90f3da57796b488c9087a2f523adf045 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:22:14.211104 23287 raft_consensus.cc:2804] T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:22:14.211375 23407 raft_consensus.cc:740] T 9e65676b21b748728ad39668cfe722ec P 90f3da57796b488c9087a2f523adf045 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:22:14.211604 23287 raft_consensus.cc:493] T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:14.212119 23287 raft_consensus.cc:3060] T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:22:14.212632 23407 consensus_queue.cc:260] T 9e65676b21b748728ad39668cfe722ec P 90f3da57796b488c9087a2f523adf045 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:14.214339 23407 raft_consensus.cc:3060] T 9e65676b21b748728ad39668cfe722ec P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:22:14.217419 22610 raft_consensus.cc:3055] T 8d4ab870a05347868956aa8b841b2981 P 647a343a542d4a139fe106c19095129b [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:22:14.217868 22610 raft_consensus.cc:740] T 8d4ab870a05347868956aa8b841b2981 P 647a343a542d4a139fe106c19095129b [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:22:14.205729 23483 leader_election.cc:290] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 19 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:14.174959 16151 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9ad5dbfa99ee4b8d9e28be92b3f36db5"
dest_uuid: "647a343a542d4a139fe106c19095129b"
mode: GRACEFUL
new_leader_uuid: "90f3da57796b488c9087a2f523adf045"
 from {username='slave'} at 127.0.0.1:40290
I20260709 13:22:14.220994 16151 raft_consensus.cc:606] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Received request to transfer leadership to TS 90f3da57796b488c9087a2f523adf045
I20260709 13:22:14.221343 16151 raft_consensus.cc:614] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:22:14.221406 23210 raft_consensus.cc:515] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:14.228233 23554 raft_consensus.cc:2749] T d2ab0ac8347641e6955e9bce7d90c1a2 P 1ca2725d290e47d68b1a4b609c9816ac [term 20 FOLLOWER]: Leader election lost for term 19. Reason: Vote denied by peer 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389) with higher term. Message: Invalid argument: T d2ab0ac8347641e6955e9bce7d90c1a2 P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac for earlier term 19. Current term is 20.
I20260709 13:22:14.229667 16266 leader_election.cc:304] T d2ab0ac8347641e6955e9bce7d90c1a2 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:22:14.232249 23538 raft_consensus.cc:2804] T d2ab0ac8347641e6955e9bce7d90c1a2 P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:22:14.232924 23538 raft_consensus.cc:697] T d2ab0ac8347641e6955e9bce7d90c1a2 P 90f3da57796b488c9087a2f523adf045 [term 20 LEADER]: Becoming Leader. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:22:14.234064 23538 consensus_queue.cc:237] T d2ab0ac8347641e6955e9bce7d90c1a2 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.15, Last appended by leader: 15, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:14.237296 16816 raft_consensus.cc:1240] T ada5a004ba7c46819051e3ccec71ffdb P cb82cc7bd6db46d980c75237a1402860 [term 21 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 20. Current term is 21. Ops: []
I20260709 13:22:14.240312 16265 leader_election.cc:304] T 8158bbd17b994d2eb96f560bd0f1b88d P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 647a343a542d4a139fe106c19095129b
I20260709 13:22:14.246126 23357 leader_election.cc:290] T 5a959dbecc2648eb85d1cc550f5a3245 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:14.249609 23559 raft_consensus.cc:515] T 68a6838d82ef4cac83507f906139d90f P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:14.251461 23433 leader_election.cc:290] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:14.252252 23559 leader_election.cc:290] T 68a6838d82ef4cac83507f906139d90f P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 20 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:22:14.253283 23373 leader_election.cc:290] T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:14.254096 23403 raft_consensus.cc:515] T 02838c86531d4f98bf58d1c22b8baf77 P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:14.255311 16292 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48674 (request call id 3495) took 1345 ms. Trace:
I20260709 13:22:14.255681 16292 rpcz_store.cc:276] 0709 13:22:12.910011 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:22:12.910121 (+   110us) service_pool.cc:224] Handling call
0709 13:22:13.924864 (+1014743us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:22:13.925105 (+   241us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:22:13.925129 (+    24us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:22:13.925169 (+    40us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:22:13.925214 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:22:13.925273 (+    59us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:22:14.255122 (+329849us) spinlock_profiling.cc:243] Waited 1.01 s on lock 0x7b540035fcd0. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5e8ad 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:22:14.255255 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2333063552}
I20260709 13:22:14.256676 23403 leader_election.cc:290] T 02838c86531d4f98bf58d1c22b8baf77 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 21 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:14.257064 16292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806d7690c2b644959152074bb2cab0a7" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:14.237468 23086 raft_consensus.cc:515] T 7c2095a5f4ad49408dbc7f69c606120d P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:14.259771 23384 raft_consensus.cc:515] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:14.261402 16150 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40844 (request call id 3016) took 1061 ms. Trace:
I20260709 13:22:14.198114 23023 raft_consensus.cc:515] T ecc1e363230043fe9fef390f0b979f20 P 647a343a542d4a139fe106c19095129b [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:14.261643 16221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1df93779209748b9860f6d893dd3de53" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:14.262367 16221 raft_consensus.cc:2468] T 1df93779209748b9860f6d893dd3de53 P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 18.
I20260709 13:22:14.261888 16150 rpcz_store.cc:276] 0709 13:22:13.199885 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:22:13.669179 (+469294us) service_pool.cc:224] Handling call
0709 13:22:13.669469 (+   290us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:22:13.682534 (+ 13065us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:22:14.261124 (+578590us) spinlock_profiling.cc:243] Waited 91.1 ms on lock 0x7b54004a31a0. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0ad6dc0 00007f69b0b62a02 00007f69b0b5edd6 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2
0709 13:22:14.261341 (+   217us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":209569536}
I20260709 13:22:14.263562 16223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8158bbd17b994d2eb96f560bd0f1b88d" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:22:14.263659 16222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6f07842d3aa47858df5665874c86f3a" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:14.264253 16223 raft_consensus.cc:3060] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:22:14.264802 23023 leader_election.cc:290] T ecc1e363230043fe9fef390f0b979f20 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 23 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:14.265101 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68a6838d82ef4cac83507f906139d90f" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:14.266111 23553 raft_consensus.cc:515] T 434bff3082d141838068f323bdb0d1ed P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:14.266546 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee9ed67006144797b1de6ea8c2013a68" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:14.267765 23086 leader_election.cc:290] T 7c2095a5f4ad49408dbc7f69c606120d P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 17 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:14.269147 16820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7ca61592e848ea9b0fb81a2ce33cdc" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:22:14.269799 16820 raft_consensus.cc:3060] T 3a7ca61592e848ea9b0fb81a2ce33cdc P cb82cc7bd6db46d980c75237a1402860 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:22:14.269928 16224 raft_consensus.cc:3060] T c2f5f89186204242bf377f7c42b2e2da P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:22:14.267714 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cc959fb7f704fba9009c0595467d074" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:14.285841 16219 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57604 (request call id 3436) took 1021 ms. Trace:
I20260709 13:22:14.286520 16219 rpcz_store.cc:276] 0709 13:22:13.264343 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:22:13.343564 (+ 79221us) service_pool.cc:224] Handling call
0709 13:22:14.285484 (+941920us) spinlock_profiling.cc:243] Waited 299 ms on lock 0x7b5400368154. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:22:14.285711 (+   227us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":687041792}
I20260709 13:22:14.308377 16223 raft_consensus.cc:2468] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 19.
I20260709 13:22:14.279767 23384 leader_election.cc:290] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 21 election: Requested vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:14.316038 23453 consensus_queue.cc:237] T 84f2da98864b4425bf7c0055951fdd4b P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:14.322046 23495 raft_consensus.cc:515] T 57a744f5aaf944678a63469cbf514015 P cb82cc7bd6db46d980c75237a1402860 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:14.337482 23409 leader_election.cc:290] T 3a7ca61592e848ea9b0fb81a2ce33cdc P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:14.385938 16292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4dc6b0541ca431ebc757beac00521f1" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:22:14.218753 22610 consensus_queue.cc:260] T 8d4ab870a05347868956aa8b841b2981 P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:14.401505 16823 raft_consensus.cc:1240] T 57a744f5aaf944678a63469cbf514015 P cb82cc7bd6db46d980c75237a1402860 [term 18 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 17. Current term is 18. Ops: [17.14-17.14]
W20260709 13:22:14.404908 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 325464us
I20260709 13:22:14.369823 16819 raft_consensus.cc:2468] T 66f75124215f4ca6bef831e174a792d8 P cb82cc7bd6db46d980c75237a1402860 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 19.
I20260709 13:22:14.408409 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fffab26427e24d398e2695aad40ca134" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 21 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
W20260709 13:22:14.384881 23024 consensus_meta.cc:287] T 26354361e9b044cea1c2b86c4882fe92 P 90f3da57796b488c9087a2f523adf045: Time spent flushing consensus metadata: real 0.561s	user 0.008s	sys 0.003s
I20260709 13:22:14.379923 16223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806d7690c2b644959152074bb2cab0a7" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:14.243446 23558 raft_consensus.cc:2804] T 8158bbd17b994d2eb96f560bd0f1b88d P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:22:14.405826 16219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2372981006d4b218215c28964dea623" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:14.414778 16219 raft_consensus.cc:2468] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 24.
W20260709 13:22:14.387707 23533 consensus_meta.cc:287] T 3884f864a96d428785895e5463e5728f P 90f3da57796b488c9087a2f523adf045: Time spent flushing consensus metadata: real 0.587s	user 0.010s	sys 0.001s
I20260709 13:22:14.409118 16192 leader_election.cc:304] T 66f75124215f4ca6bef831e174a792d8 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:22:14.370502 16146 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e4dc6b0541ca431ebc757beac00521f1"
dest_uuid: "647a343a542d4a139fe106c19095129b"
mode: GRACEFUL
new_leader_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
 from {username='slave'} at 127.0.0.1:40290
I20260709 13:22:14.410449 16191 leader_election.cc:304] T 7c2095a5f4ad49408dbc7f69c606120d P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
I20260709 13:22:14.419287 16146 raft_consensus.cc:606] T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Received request to transfer leadership to TS 1ca2725d290e47d68b1a4b609c9816ac
I20260709 13:22:14.419750 16146 raft_consensus.cc:614] T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:22:14.420913 23061 raft_consensus.cc:2804] T 7c2095a5f4ad49408dbc7f69c606120d P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:22:14.421027 23554 raft_consensus.cc:2804] T 66f75124215f4ca6bef831e174a792d8 P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:22:14.421504 23061 raft_consensus.cc:493] T 7c2095a5f4ad49408dbc7f69c606120d P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:14.421761 23554 raft_consensus.cc:697] T 66f75124215f4ca6bef831e174a792d8 P 1ca2725d290e47d68b1a4b609c9816ac [term 19 LEADER]: Becoming Leader. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:22:14.422025 23061 raft_consensus.cc:3060] T 7c2095a5f4ad49408dbc7f69c606120d P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:22:14.420236 16191 leader_election.cc:304] T 1df93779209748b9860f6d893dd3de53 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
W20260709 13:22:14.424999 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 377444us
I20260709 13:22:14.368219 16820 raft_consensus.cc:2468] T 3a7ca61592e848ea9b0fb81a2ce33cdc P cb82cc7bd6db46d980c75237a1402860 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 21.
W20260709 13:22:14.420210 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 345553us
I20260709 13:22:14.417716 23558 raft_consensus.cc:493] T 8158bbd17b994d2eb96f560bd0f1b88d P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:14.435557 23558 raft_consensus.cc:3060] T 8158bbd17b994d2eb96f560bd0f1b88d P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:22:14.435454 23373 consensus_queue.cc:1048] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:22:14.436858 23553 leader_election.cc:290] T 434bff3082d141838068f323bdb0d1ed P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 20 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:14.437685 23373 consensus_queue.cc:1048] T fffab26427e24d398e2695aad40ca134 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:22:14.438647 23495 leader_election.cc:290] T 57a744f5aaf944678a63469cbf514015 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 18 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:14.430624 23533 raft_consensus.cc:515] T 3884f864a96d428785895e5463e5728f P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:14.440761 16297 raft_consensus.cc:2393] T 3884f864a96d428785895e5463e5728f P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 16: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
I20260709 13:22:14.441604 23474 consensus_queue.cc:1048] T af57996a8be84d398f0f023e8d08ef77 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:22:14.444156 23533 leader_election.cc:290] T 3884f864a96d428785895e5463e5728f P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 16 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:14.423832 23557 raft_consensus.cc:2749] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:22:14.445308 16291 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48684 (request call id 3252) took 1176 ms. Trace:
I20260709 13:22:14.446518 23399 raft_consensus.cc:493] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:22:14.446051 16291 rpcz_store.cc:276] 0709 13:22:13.268656 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:22:13.967638 (+698982us) service_pool.cc:224] Handling call
0709 13:22:13.967933 (+   295us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:22:13.968247 (+   314us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:22:13.968274 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:22:13.968682 (+   408us) log.cc:844] Serialized 26 byte log entry
0709 13:22:14.170310 (+201628us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:22:14.170369 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:22:14.170438 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:22:14.445056 (+274618us) raft_consensus.cc:1685] finished
0709 13:22:14.445078 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:22:14.445243 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":73590,"raft.run_cpu_time_us":135,"raft.run_wall_time_us":24188,"thread_start_us":82061,"threads_started":1,"wal-append.queue_time_us":366569}
I20260709 13:22:14.446991 23173 raft_consensus.cc:515] T e5ad4eb281564936a59f41e387a47156 P 1ca2725d290e47d68b1a4b609c9816ac [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:14.447379 23399 raft_consensus.cc:515] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:14.448388 16291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fffab26427e24d398e2695aad40ca134" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 21 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:14.449295 16291 raft_consensus.cc:2410] T fffab26427e24d398e2695aad40ca134 P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b for term 21 because replica has last-logged OpId of term: 19 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 11.
I20260709 13:22:14.449808 23399 leader_election.cc:290] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:14.450017 23173 leader_election.cc:290] T e5ad4eb281564936a59f41e387a47156 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 20 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:14.450156 16291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434bff3082d141838068f323bdb0d1ed" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:22:14.445292 23474 consensus_queue.cc:1048] T 18ad7e3af5504571a69d10c00750a01c P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.007s
I20260709 13:22:14.452585 16291 raft_consensus.cc:2368] T 434bff3082d141838068f323bdb0d1ed P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 for earlier term 19. Current term is 20.
I20260709 13:22:14.453692 16291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:22:14.454705 16291 raft_consensus.cc:3060] T ecc1e363230043fe9fef390f0b979f20 P 90f3da57796b488c9087a2f523adf045 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:22:14.455642 16292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3884f864a96d428785895e5463e5728f" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 16 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:22:14.453975 23311 leader_election.cc:290] T 08c74490c40243a88e62d1760b5cd814 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 20 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:14.456867 16292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cc959fb7f704fba9009c0595467d074" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:14.459151 23403 raft_consensus.cc:2804] T 1df93779209748b9860f6d893dd3de53 P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:22:14.459621 23403 raft_consensus.cc:493] T 1df93779209748b9860f6d893dd3de53 P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:14.460039 23403 raft_consensus.cc:3060] T 1df93779209748b9860f6d893dd3de53 P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:22:14.459705 23474 consensus_queue.cc:1048] T b6f07842d3aa47858df5665874c86f3a P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:22:14.417692 23024 raft_consensus.cc:515] T 26354361e9b044cea1c2b86c4882fe92 P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:14.461556 16191 leader_election.cc:304] T 746aabf856ab401c9ef5e859371e7b67 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
I20260709 13:22:14.464310 16192 leader_election.cc:304] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:22:14.464970 16292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02838c86531d4f98bf58d1c22b8baf77" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:14.468561 16144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b7d1ab519044eda2d5e04a07242f25" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:22:14.465770 16292 raft_consensus.cc:2468] T 02838c86531d4f98bf58d1c22b8baf77 P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 20.
I20260709 13:22:14.469509 16144 raft_consensus.cc:2468] T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 19.
I20260709 13:22:14.470050 16292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31969392ccdb43b491911a319e8bc7e1" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:14.413414 23292 raft_consensus.cc:2749] T 66f75124215f4ca6bef831e174a792d8 P cb82cc7bd6db46d980c75237a1402860 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:22:14.470860 16292 raft_consensus.cc:2468] T 31969392ccdb43b491911a319e8bc7e1 P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 17.
W20260709 13:22:14.418740 23077 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.581s	user 0.007s	sys 0.007s
I20260709 13:22:14.493777 16292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ab0ac8347641e6955e9bce7d90c1a2" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:14.500079 22677 raft_consensus.cc:2749] T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:22:14.490561 23478 raft_consensus.cc:2749] T 746aabf856ab401c9ef5e859371e7b67 P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:22:14.504316 16219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a959dbecc2648eb85d1cc550f5a3245" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 21 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:14.506934 16219 raft_consensus.cc:2410] T 5a959dbecc2648eb85d1cc550f5a3245 P 1ca2725d290e47d68b1a4b609c9816ac [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b for term 21 because replica has last-logged OpId of term: 18 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 11.
I20260709 13:22:14.508955 16292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3884f864a96d428785895e5463e5728f" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 15 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:22:14.488327 16191 leader_election.cc:304] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: cb82cc7bd6db46d980c75237a1402860
I20260709 13:22:14.422866 23554 consensus_queue.cc:237] T 66f75124215f4ca6bef831e174a792d8 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.14, Last appended by leader: 14, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:14.513373 16217 raft_consensus.cc:1240] T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 18. Current term is 19. Ops: []
I20260709 13:22:14.514787 16116 leader_election.cc:304] T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860
I20260709 13:22:14.511044 16191 leader_election.cc:304] T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
I20260709 13:22:14.511770 16223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:22:14.519533 16223 raft_consensus.cc:3060] T ecc1e363230043fe9fef390f0b979f20 P 1ca2725d290e47d68b1a4b609c9816ac [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:22:14.520228 16116 leader_election.cc:304] T c0eab25ce4c64e018a041beeda2867db P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:22:14.521356 23433 raft_consensus.cc:2749] T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:22:14.522339 16219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c2095a5f4ad49408dbc7f69c606120d" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:22:14.521468 16146 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e5ad4eb281564936a59f41e387a47156"
dest_uuid: "647a343a542d4a139fe106c19095129b"
mode: GRACEFUL
new_leader_uuid: "90f3da57796b488c9087a2f523adf045"
 from {username='slave'} at 127.0.0.1:40290
I20260709 13:22:14.522974 16292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a9de347c711477e87c582d003f36925" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:14.523599 16146 raft_consensus.cc:606] T e5ad4eb281564936a59f41e387a47156 P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Received request to transfer leadership to TS 90f3da57796b488c9087a2f523adf045
I20260709 13:22:14.523828 16292 raft_consensus.cc:2393] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 24: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
I20260709 13:22:14.524099 16146 raft_consensus.cc:614] T e5ad4eb281564936a59f41e387a47156 P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:22:14.525357 16292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2794af5b99984efebdee564035841cb0" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 19 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:14.542758 16292 raft_consensus.cc:2410] T 2794af5b99984efebdee564035841cb0 P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b for term 19 because replica has last-logged OpId of term: 17 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 14.
I20260709 13:22:14.526968 16116 leader_election.cc:304] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860
I20260709 13:22:14.524462 23311 raft_consensus.cc:2804] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:22:14.492678 23024 leader_election.cc:290] T 26354361e9b044cea1c2b86c4882fe92 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 17 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:14.529733 23373 raft_consensus.cc:2804] T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:22:14.505189 23210 leader_election.cc:290] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 24 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:14.531840 23433 raft_consensus.cc:2749] T c0eab25ce4c64e018a041beeda2867db P 647a343a542d4a139fe106c19095129b [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:22:14.569562 16292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87147731a68400e8cce9b879daa7c5d" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:14.493330 23474 raft_consensus.cc:2804] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Leader election won for term 21
W20260709 13:22:14.523841 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 88590.6us
I20260709 13:22:14.546659 23311 raft_consensus.cc:493] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:14.560132 16790 leader_election.cc:304] T 829b4ad8c5a54eababeea1181b960d4b P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
W20260709 13:22:14.505841 23077 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.675s	user 0.008s	sys 0.007s
I20260709 13:22:14.551098 16116 leader_election.cc:304] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860
I20260709 13:22:14.579715 16190 leader_election.cc:304] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860
I20260709 13:22:14.526684 16191 leader_election.cc:304] T 08c74490c40243a88e62d1760b5cd814 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
I20260709 13:22:14.588431 16190 leader_election.cc:400] T c2f5f89186204242bf377f7c42b2e2da P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 22 pre-election: Vote denied by peer 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389) with higher term. Message: Invalid argument: T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac for earlier term 22. Current term is 23.
I20260709 13:22:14.589656 16190 leader_election.cc:403] T c2f5f89186204242bf377f7c42b2e2da P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 22 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:22:14.573563 16223 raft_consensus.cc:2468] T ecc1e363230043fe9fef390f0b979f20 P 1ca2725d290e47d68b1a4b609c9816ac [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 23.
I20260709 13:22:14.592046 16224 raft_consensus.cc:1275] T c2f5f89186204242bf377f7c42b2e2da P 1ca2725d290e47d68b1a4b609c9816ac [term 22 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 20 index: 12. Preceding OpId from leader: term: 22 index: 13. (index mismatch)
I20260709 13:22:14.561474 23373 raft_consensus.cc:493] T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:22:14.593914 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 47091.2us
I20260709 13:22:14.599006 16190 leader_election.cc:304] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:22:14.570523 23292 raft_consensus.cc:493] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:14.547505 23287 raft_consensus.cc:515] T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:14.571692 23256 raft_consensus.cc:3055] T 84f2da98864b4425bf7c0055951fdd4b P 1ca2725d290e47d68b1a4b609c9816ac [term 16 LEADER]: Stepping down as leader of term 16
W20260709 13:22:14.594400 16789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 358378us
I20260709 13:22:14.601970 16790 leader_election.cc:304] T 434bff3082d141838068f323bdb0d1ed P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:22:14.593063 23311 raft_consensus.cc:3060] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:22:14.608564 16790 leader_election.cc:400] T 3884f864a96d428785895e5463e5728f P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 15 election: Vote denied by peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741) with higher term. Message: Invalid argument: T 3884f864a96d428785895e5463e5728f P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 for earlier term 15. Current term is 16.
I20260709 13:22:14.607863 16116 leader_election.cc:304] T 1df93779209748b9860f6d893dd3de53 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
I20260709 13:22:14.614488 16790 leader_election.cc:403] T 3884f864a96d428785895e5463e5728f P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 15 election: Cancelling election due to peer responding with higher term
I20260709 13:22:14.594942 23433 raft_consensus.cc:2749] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:22:14.612159 16819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26354361e9b044cea1c2b86c4882fe92" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:22:14.592948 23077 raft_consensus.cc:697] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045 [term 23 LEADER]: Becoming Leader. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:22:14.615952 16116 leader_election.cc:304] T ee9ed67006144797b1de6ea8c2013a68 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860
I20260709 13:22:14.611589 16820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434bff3082d141838068f323bdb0d1ed" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:22:14.617364 16825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a9de347c711477e87c582d003f36925" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:22:14.605201 16190 leader_election.cc:304] T 5a96373bb0e14dce8a266b2fc6c58507 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:22:14.653065 16821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2372981006d4b218215c28964dea623" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:14.621748 16817 raft_consensus.cc:1240] T 66f75124215f4ca6bef831e174a792d8 P cb82cc7bd6db46d980c75237a1402860 [term 19 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 18. Current term is 19. Ops: [18.14-18.14]
W20260709 13:22:14.675905 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 149270us
I20260709 13:22:14.678071 16818 raft_consensus.cc:1240] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860 [term 19 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 18. Current term is 19. Ops: []
I20260709 13:22:14.618734 16790 leader_election.cc:304] T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:22:14.600745 23373 raft_consensus.cc:3060] T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:22:14.596257 23474 raft_consensus.cc:697] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 1ca2725d290e47d68b1a4b609c9816ac [term 21 LEADER]: Becoming Leader. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:22:14.658703 16822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3884f864a96d428785895e5463e5728f" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 16 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:22:14.603899 23409 raft_consensus.cc:2749] T 829b4ad8c5a54eababeea1181b960d4b P cb82cc7bd6db46d980c75237a1402860 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:22:14.603734 23372 raft_consensus.cc:3060] T c2f5f89186204242bf377f7c42b2e2da P 1ca2725d290e47d68b1a4b609c9816ac [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:22:14.677807 16819 raft_consensus.cc:3060] T 26354361e9b044cea1c2b86c4882fe92 P cb82cc7bd6db46d980c75237a1402860 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:22:14.707762 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 89123.6us
I20260709 13:22:14.594731 23566 raft_consensus.cc:2749] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:22:14.614775 23563 raft_consensus.cc:2804] T 434bff3082d141838068f323bdb0d1ed P cb82cc7bd6db46d980c75237a1402860 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:22:14.693424 16823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3884f864a96d428785895e5463e5728f" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 16 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:22:14.697393 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02838c86531d4f98bf58d1c22b8baf77" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:22:14.721911 16150 raft_consensus.cc:3060] T 02838c86531d4f98bf58d1c22b8baf77 P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Advancing to term 21
I20260709 13:22:14.697901 16300 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40474 (request call id 1343) took 1231 ms. Trace:
I20260709 13:22:14.585005 23326 raft_consensus.cc:515] T b87147731a68400e8cce9b879daa7c5d P 90f3da57796b488c9087a2f523adf045 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:14.611757 23023 raft_consensus.cc:2749] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
W20260709 13:22:14.689539 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 84416.3us
I20260709 13:22:14.681936 23399 raft_consensus.cc:3060] T 3884f864a96d428785895e5463e5728f P cb82cc7bd6db46d980c75237a1402860 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:22:14.607491 23256 raft_consensus.cc:740] T 84f2da98864b4425bf7c0055951fdd4b P 1ca2725d290e47d68b1a4b609c9816ac [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:22:14.609614 23292 raft_consensus.cc:515] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:14.695689 16818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:22:14.787909 16150 raft_consensus.cc:2468] T 02838c86531d4f98bf58d1c22b8baf77 P 647a343a542d4a139fe106c19095129b [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 21.
W20260709 13:22:14.788233 16265 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.546s	user 0.000s	sys 0.027s
I20260709 13:22:14.618825 23286 consensus_queue.cc:1059] T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.133s
I20260709 13:22:14.614424 23210 consensus_queue.cc:1059] T 57a744f5aaf944678a63469cbf514015 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: INVALID_TERM, Last received: 15.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.134s
I20260709 13:22:14.781009 23077 consensus_queue.cc:237] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 21.12, Last appended by leader: 12, Current term: 23, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:14.675532 23331 raft_consensus.cc:2804] T 1df93779209748b9860f6d893dd3de53 P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:22:14.782598 23326 leader_election.cc:290] T b87147731a68400e8cce9b879daa7c5d P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 21 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:14.782792 16816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87147731a68400e8cce9b879daa7c5d" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:22:14.783074 23287 leader_election.cc:290] T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 20 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:22:14.784603 16292 raft_consensus.cc:2393] T b87147731a68400e8cce9b879daa7c5d P 90f3da57796b488c9087a2f523adf045 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 21: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
I20260709 13:22:14.779850 16819 raft_consensus.cc:2468] T 26354361e9b044cea1c2b86c4882fe92 P cb82cc7bd6db46d980c75237a1402860 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 17.
I20260709 13:22:14.717563 16291 raft_consensus.cc:2468] T ecc1e363230043fe9fef390f0b979f20 P 90f3da57796b488c9087a2f523adf045 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 23.
I20260709 13:22:14.788259 16818 raft_consensus.cc:3060] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860 [term 22 FOLLOWER]: Advancing to term 23
W20260709 13:22:14.789040 16265 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.546s	user 0.000s	sys 0.028s
I20260709 13:22:14.781206 16146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57a744f5aaf944678a63469cbf514015" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:22:14.676278 23565 raft_consensus.cc:2749] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:22:14.692991 22946 raft_consensus.cc:2804] T 08c74490c40243a88e62d1760b5cd814 P 1ca2725d290e47d68b1a4b609c9816ac [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:22:14.730842 16300 rpcz_store.cc:276] 0709 13:22:13.466686 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:22:14.044730 (+578044us) service_pool.cc:224] Handling call
0709 13:22:14.045015 (+   285us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:22:14.045206 (+   191us) raft_consensus.cc:1534] Early marking committed up to index 16
0709 13:22:14.585296 (+540090us) log.cc:844] Serialized 14 byte log entry
0709 13:22:14.696230 (+110934us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:22:14.696289 (+    59us) raft_consensus.cc:1645] Marking committed up to 16
0709 13:22:14.696339 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:22:14.696399 (+    60us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:22:14.697844 (+  1445us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":38346,"prepare.queue_time_us":570379,"prepare.run_wall_time_us":36,"raft.queue_time_us":10209,"raft.run_wall_time_us":11277,"spinlock_wait_cycles":9787136,"thread_start_us":595659,"threads_started":2,"wal-append.queue_time_us":171935}
I20260709 13:22:14.712520 23398 raft_consensus.cc:2749] T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:22:14.710022 23552 raft_consensus.cc:3060] T 5a96373bb0e14dce8a266b2fc6c58507 P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:22:14.805542 16146 raft_consensus.cc:3060] T 57a744f5aaf944678a63469cbf514015 P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:22:14.804872 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 562436us
I20260709 13:22:14.838893 16265 leader_election.cc:304] T 5a96373bb0e14dce8a266b2fc6c58507 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:22:14.839607 16821 raft_consensus.cc:2393] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 24: Already voted for candidate cb82cc7bd6db46d980c75237a1402860 in this term.
W20260709 13:22:14.830279 23570 log.cc:927] Time spent T 18ad7e3af5504571a69d10c00750a01c P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.229s	user 0.001s	sys 0.000s
I20260709 13:22:14.845499 16265 leader_election.cc:304] T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 647a343a542d4a139fe106c19095129b
W20260709 13:22:14.848246 16192 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 136341us
I20260709 13:22:14.713143 23478 consensus_queue.cc:1048] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.122s
I20260709 13:22:14.804226 23453 raft_consensus.cc:2749] T ee9ed67006144797b1de6ea8c2013a68 P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:22:14.832144 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57a744f5aaf944678a63469cbf514015" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:22:14.719192 23558 raft_consensus.cc:515] T 8158bbd17b994d2eb96f560bd0f1b88d P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:22:14.790273 16117 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 124288us
I20260709 13:22:14.834570 16291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7ca61592e848ea9b0fb81a2ce33cdc" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:14.854949 16293 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48684 (request call id 3248) took 1744 ms. Trace:
I20260709 13:22:14.855551 16291 raft_consensus.cc:2468] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 20.
I20260709 13:22:14.851840 23565 consensus_queue.cc:1059] T 84f2da98864b4425bf7c0055951fdd4b P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.007s
I20260709 13:22:14.843149 22677 raft_consensus.cc:493] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:22:14.802949 23331 raft_consensus.cc:493] T 1df93779209748b9860f6d893dd3de53 P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:14.830947 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5ad4eb281564936a59f41e387a47156" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:22:14.860942 16291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8158bbd17b994d2eb96f560bd0f1b88d" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:14.861938 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f5f89186204242bf377f7c42b2e2da" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 23 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:22:14.841224 23538 raft_consensus.cc:3055] T 57a744f5aaf944678a63469cbf514015 P 90f3da57796b488c9087a2f523adf045 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:22:14.858170 22677 raft_consensus.cc:515] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:14.863138 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fffab26427e24d398e2695aad40ca134" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:22:14.883142 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3884f864a96d428785895e5463e5728f" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 16 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:22:14.841001 22946 raft_consensus.cc:697] T 08c74490c40243a88e62d1760b5cd814 P 1ca2725d290e47d68b1a4b609c9816ac [term 20 LEADER]: Becoming Leader. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:22:14.844890 23286 consensus_queue.cc:1243] T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b [LEADER]: Peer 1ca2725d290e47d68b1a4b609c9816ac log is divergent from this leader: its last log entry 17.14 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 13
I20260709 13:22:14.846724 23210 raft_consensus.cc:2749] T 5a96373bb0e14dce8a266b2fc6c58507 P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:22:14.888710 16150 raft_consensus.cc:2393] T 3884f864a96d428785895e5463e5728f P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 16: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
I20260709 13:22:14.891886 16146 raft_consensus.cc:2468] T 57a744f5aaf944678a63469cbf514015 P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 18.
I20260709 13:22:14.892207 16291 raft_consensus.cc:2393] T 8158bbd17b994d2eb96f560bd0f1b88d P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 19: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
I20260709 13:22:14.858898 23287 raft_consensus.cc:2804] T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:22:14.894243 16146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6f07842d3aa47858df5665874c86f3a" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:22:14.865008 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f5f89186204242bf377f7c42b2e2da" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 23 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:22:14.876176 16115 leader_election.cc:304] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:22:14.878023 23538 raft_consensus.cc:740] T 57a744f5aaf944678a63469cbf514015 P 90f3da57796b488c9087a2f523adf045 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:22:14.866354 23474 consensus_queue.cc:237] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:14.903066 23538 consensus_queue.cc:260] T 57a744f5aaf944678a63469cbf514015 P 90f3da57796b488c9087a2f523adf045 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.14, Last appended by leader: 14, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:14.906695 23372 raft_consensus.cc:2749] T c2f5f89186204242bf377f7c42b2e2da P 1ca2725d290e47d68b1a4b609c9816ac [term 23 FOLLOWER]: Leader pre-election lost for term 22. Reason: Vote denied by peer 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389) with higher term. Message: Invalid argument: T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac for earlier term 22. Current term is 23.
I20260709 13:22:14.906945 23538 raft_consensus.cc:3060] T 57a744f5aaf944678a63469cbf514015 P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:22:14.877635 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 90711.6us
I20260709 13:22:14.866500 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0eab25ce4c64e018a041beeda2867db" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:14.912667 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c2095a5f4ad49408dbc7f69c606120d" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:22:14.916724 22677 leader_election.cc:290] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:14.864713 16224 raft_consensus.cc:1240] T c0eab25ce4c64e018a041beeda2867db P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 20. Current term is 21. Ops: []
I20260709 13:22:14.863637 16223 raft_consensus.cc:1240] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 20. Current term is 21. Ops: []
I20260709 13:22:14.920644 16300 raft_consensus.cc:2468] T 7c2095a5f4ad49408dbc7f69c606120d P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 17.
I20260709 13:22:14.923241 16295 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48684 (request call id 3251) took 1654 ms. Trace:
I20260709 13:22:14.923492 16143 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46224 (request call id 1235) took 1615 ms. Trace:
I20260709 13:22:14.892100 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26354361e9b044cea1c2b86c4882fe92" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:22:14.900355 16146 raft_consensus.cc:3060] T b6f07842d3aa47858df5665874c86f3a P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:22:14.901868 23573 log.cc:927] Time spent T 68a6838d82ef4cac83507f906139d90f P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 0.102s	user 0.001s	sys 0.000s
I20260709 13:22:14.877015 23331 raft_consensus.cc:3060] T 1df93779209748b9860f6d893dd3de53 P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:22:14.928788 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87147731a68400e8cce9b879daa7c5d" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
W20260709 13:22:14.922580 16115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 46499.3us
I20260709 13:22:14.930567 16115 leader_election.cc:304] T b6f07842d3aa47858df5665874c86f3a P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac
I20260709 13:22:14.923679 16295 rpcz_store.cc:276] 0709 13:22:13.268373 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:22:13.966340 (+697967us) service_pool.cc:224] Handling call
0709 13:22:13.966611 (+   271us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:22:13.966818 (+   207us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:22:13.966842 (+    24us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:22:14.056322 (+ 89480us) log.cc:844] Serialized 26 byte log entry
0709 13:22:14.424722 (+368400us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:22:14.424759 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:22:14.424833 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:22:14.922991 (+498158us) raft_consensus.cc:1685] finished
0709 13:22:14.923016 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:22:14.923155 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":14592,"thread_start_us":326231,"threads_started":1,"wal-append.queue_time_us":493153}
I20260709 13:22:14.924086 16143 rpcz_store.cc:276] 0709 13:22:13.308428 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:22:13.724008 (+415580us) service_pool.cc:224] Handling call
0709 13:22:13.724305 (+   297us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:22:13.749941 (+ 25636us) raft_consensus.cc:1534] Early marking committed up to index 15
0709 13:22:13.769241 (+ 19300us) log.cc:844] Serialized 14 byte log entry
0709 13:22:14.209406 (+440165us) spinlock_profiling.cc:243] Waited 5.14 ms on lock 0x7b18003df000. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b693ade1 00007f69b69475b1 00007f69ada723f3 00007f69ada747c1 00007f69ada74779 00007f69ada74709 00007f69ada746d1 00007f69ada739cd 000000000038f354 000000000038f199 00007f69ada5583a
0709 13:22:14.211556 (+  2150us) spinlock_profiling.cc:243] Waited 3.19 ms on lock 0x7b18003df000. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b693ade1 00007f69b69475b1 00007f69ada54e07 00007f69b68afab9 00007f69ada7169b 00007f69ada6f93f 00007f69ada71c02 00007f69b06f920b 00007f69b06fed23 00007f69b06ff23a 00007f69b0b69a2a
0709 13:22:14.600633 (+389077us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:22:14.601879 (+  1246us) log.cc:844] Serialized 26 byte log entry
0709 13:22:14.602050 (+   171us) raft_consensus.cc:1645] Marking committed up to 16
0709 13:22:14.606252 (+  4202us) log.cc:844] Serialized 14 byte log entry
0709 13:22:14.611469 (+  5217us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:22:14.611562 (+    93us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:22:14.923231 (+311669us) raft_consensus.cc:1685] finished
0709 13:22:14.923262 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:22:14.923439 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":64205,"prepare.queue_time_us":23468,"prepare.run_cpu_time_us":42,"prepare.run_wall_time_us":81,"raft.queue_time_us":34327,"raft.run_cpu_time_us":425,"raft.run_wall_time_us":71392,"spinlock_wait_cycles":151988096,"thread_start_us":383666,"threads_started":1,"wal-append.queue_time_us":1065860}
I20260709 13:22:14.924273 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66ae176961b74813a58bc15c9c4c6f2a" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 27 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:14.855523 16293 rpcz_store.cc:276] 0709 13:22:13.110990 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:22:13.955451 (+844461us) service_pool.cc:224] Handling call
0709 13:22:13.955683 (+   232us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:22:13.955854 (+   171us) raft_consensus.cc:1534] Early marking committed up to index 14
0709 13:22:14.539959 (+584105us) log.cc:844] Serialized 14 byte log entry
0709 13:22:14.854533 (+314574us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:22:14.854620 (+    87us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:22:14.854677 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:22:14.854758 (+    81us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:22:14.854885 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":612942,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":7813,"raft.queue_time_us":95,"raft.run_cpu_time_us":96,"raft.run_wall_time_us":96,"thread_start_us":801786,"threads_started":2}
I20260709 13:22:14.933421 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18ad7e3af5504571a69d10c00750a01c" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:14.934459 16115 leader_election.cc:304] T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
W20260709 13:22:14.864943 16265 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:22:14.935927 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87ac0d1f7e8a419f9bcbc04aee0e067e" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:14.937525 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3884f864a96d428785895e5463e5728f" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 16 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:14.938017 16223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ab0ac8347641e6955e9bce7d90c1a2" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:22:14.938017 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68a6838d82ef4cac83507f906139d90f" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
W20260709 13:22:14.863619 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 247722us
I20260709 13:22:14.933086 16295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8158bbd17b994d2eb96f560bd0f1b88d" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:22:14.946218 16115 leader_election.cc:304] T 68a6838d82ef4cac83507f906139d90f P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac
I20260709 13:22:14.955932 16146 raft_consensus.cc:2468] T b6f07842d3aa47858df5665874c86f3a P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 20.
I20260709 13:22:14.954945 16115 leader_election.cc:304] T 08c74490c40243a88e62d1760b5cd814 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:22:14.964234 16147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e9b2bf7011f4ae688a79d7bb5a447b4" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:22:14.963836 16295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a959dbecc2648eb85d1cc550f5a3245" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 21 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:14.966002 16223 raft_consensus.cc:2468] T d2ab0ac8347641e6955e9bce7d90c1a2 P 1ca2725d290e47d68b1a4b609c9816ac [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 20.
I20260709 13:22:14.940553 16116 leader_election.cc:304] T 8158bbd17b994d2eb96f560bd0f1b88d P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
I20260709 13:22:14.972097 16143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1df93779209748b9860f6d893dd3de53" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:22:14.972728 16300 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40474 (request call id 1344) took 1481 ms. Trace:
I20260709 13:22:14.900163 23287 raft_consensus.cc:493] T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:22:14.973306 16300 rpcz_store.cc:276] 0709 13:22:13.491128 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:22:14.971416 (+1480288us) service_pool.cc:224] Handling call
0709 13:22:14.972098 (+   682us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:22:14.972269 (+   171us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:22:14.972294 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:22:14.972333 (+    39us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:22:14.972361 (+    28us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:22:14.972474 (+   113us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:22:14.972673 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:22:14.974140 16295 raft_consensus.cc:1240] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045 [term 23 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 22. Current term is 23. Ops: []
I20260709 13:22:14.889314 23322 consensus_queue.cc:1059] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 647a343a542d4a139fe106c19095129b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: INVALID_TERM, Last received: 18.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.047s
I20260709 13:22:14.974961 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434bff3082d141838068f323bdb0d1ed" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:22:14.961073 16115 leader_election.cc:304] T 9e65676b21b748728ad39668cfe722ec P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860
I20260709 13:22:14.976969 16295 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48674 (request call id 3519) took 1507 ms. Trace:
I20260709 13:22:14.978494 16147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d4ab870a05347868956aa8b841b2981" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:22:14.979449 16146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a9de347c711477e87c582d003f36925" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:22:14.985074 16223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1df93779209748b9860f6d893dd3de53" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:14.894837 23086 consensus_queue.cc:1059] T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: INVALID_TERM, Last received: 16.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.075s
I20260709 13:22:15.017468 16150 raft_consensus.cc:3060] T 434bff3082d141838068f323bdb0d1ed P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:22:15.004195 16295 rpcz_store.cc:276] 0709 13:22:13.469802 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:22:14.973681 (+1503879us) service_pool.cc:224] Handling call
0709 13:22:14.973958 (+   277us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:22:14.976651 (+  2693us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:22:14.976897 (+   246us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:22:15.023406 16295 raft_consensus.cc:1240] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [term 24 FOLLOWER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 23. Current term is 24. Ops: []
I20260709 13:22:14.960901 23286 raft_consensus.cc:2749] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:22:15.026181 16295 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40474 (request call id 1346) took 1529 ms. Trace:
I20260709 13:22:15.026643 16295 rpcz_store.cc:276] 0709 13:22:13.497265 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:22:15.022900 (+1525635us) service_pool.cc:224] Handling call
0709 13:22:15.023229 (+   329us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:22:15.025898 (+  2669us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:22:15.026125 (+   227us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:22:15.028918 16295 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40474 (request call id 1348) took 1503 ms. Trace:
I20260709 13:22:15.029824 16295 rpcz_store.cc:276] 0709 13:22:13.526010 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:22:15.027499 (+1501489us) service_pool.cc:224] Handling call
0709 13:22:15.027965 (+   466us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:22:15.028595 (+   630us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:22:15.028621 (+    26us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:22:15.028664 (+    43us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:22:15.028704 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:22:15.028761 (+    57us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:22:15.028867 (+   106us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:22:14.965185 23453 raft_consensus.cc:2749] T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:22:15.040701 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fffab26427e24d398e2695aad40ca134" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:14.980891 23287 raft_consensus.cc:3060] T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:22:14.962285 23585 raft_consensus.cc:2764] T b6f07842d3aa47858df5665874c86f3a P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Leader pre-election decision vote started in defunct term 19: won
W20260709 13:22:15.069242 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 151336us
W20260709 13:22:15.083701 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 120798us
I20260709 13:22:14.972797 23024 consensus_queue.cc:1048] T 5a959dbecc2648eb85d1cc550f5a3245 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.065s
W20260709 13:22:15.091286 16115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 131015us
I20260709 13:22:15.093025 16150 raft_consensus.cc:2468] T 434bff3082d141838068f323bdb0d1ed P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 20.
I20260709 13:22:15.017370 23577 raft_consensus.cc:493] T d16dfad0c3b540e5b4b86492ddf87208 P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:22:15.095010 16142 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40844 (request call id 3035) took 1016 ms. Trace:
I20260709 13:22:15.094926 16265 leader_election.cc:304] T 8cc959fb7f704fba9009c0595467d074 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac
W20260709 13:22:15.176426 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 81574.4us
I20260709 13:22:15.004031 23590 raft_consensus.cc:493] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [term 26 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
W20260709 13:22:15.176810 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 76935.6us
I20260709 13:22:15.178683 16265 leader_election.cc:304] T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:22:15.181449 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d4ab870a05347868956aa8b841b2981" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:15.095525 16142 rpcz_store.cc:276] 0709 13:22:14.078869 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:22:14.137598 (+ 58729us) service_pool.cc:224] Handling call
0709 13:22:14.150547 (+ 12949us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:22:14.151094 (+   547us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:22:14.151135 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:22:14.161366 (+ 10231us) log.cc:844] Serialized 26 byte log entry
0709 13:22:14.560865 (+399499us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:22:14.560923 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:22:14.561016 (+    93us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:22:15.094799 (+533783us) raft_consensus.cc:1685] finished
0709 13:22:15.094818 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:22:15.094949 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1477,"raft.run_cpu_time_us":123,"raft.run_wall_time_us":2783,"thread_start_us":368968,"threads_started":1,"wal-append.queue_time_us":507303}
I20260709 13:22:15.031280 23539 raft_consensus.cc:493] T 8638ee363273426db1393dd81e5c7b72 P cb82cc7bd6db46d980c75237a1402860 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
W20260709 13:22:15.238922 16115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 145633us
I20260709 13:22:15.047156 23437 raft_consensus.cc:2804] T 68a6838d82ef4cac83507f906139d90f P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Leader pre-election won for term 20
W20260709 13:22:15.244621 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 50708.9us
I20260709 13:22:15.041718 23563 raft_consensus.cc:697] T 434bff3082d141838068f323bdb0d1ed P cb82cc7bd6db46d980c75237a1402860 [term 19 LEADER]: Becoming Leader. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:22:15.040510 23372 raft_consensus.cc:493] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:22:15.055101 23565 consensus_queue.cc:1048] T fffab26427e24d398e2695aad40ca134 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.193s
I20260709 13:22:15.245693 16190 leader_election.cc:304] T e4dc6b0541ca431ebc757beac00521f1 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:22:15.262318 16142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2372981006d4b218215c28964dea623" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 25 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:22:15.263402 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68a6838d82ef4cac83507f906139d90f" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:22:15.034962 23542 consensus_queue.cc:1059] T c0eab25ce4c64e018a041beeda2867db P 90f3da57796b488c9087a2f523adf045 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: INVALID_TERM, Last received: 18.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.212s
I20260709 13:22:15.177919 23024 consensus_queue.cc:1243] T 5a959dbecc2648eb85d1cc550f5a3245 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Peer 647a343a542d4a139fe106c19095129b log is divergent from this leader: its last log entry 17.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 13:22:15.177786 23491 raft_consensus.cc:493] T ada5a004ba7c46819051e3ccec71ffdb P cb82cc7bd6db46d980c75237a1402860 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:15.266682 16142 raft_consensus.cc:2410] T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 for term 25 because replica has last-logged OpId of term: 23 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 22 index: 15.
I20260709 13:22:15.249384 23437 raft_consensus.cc:493] T 68a6838d82ef4cac83507f906139d90f P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Starting leader election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:22:15.249112 23594 raft_consensus.cc:2749] T 08c74490c40243a88e62d1760b5cd814 P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:22:15.304692 16115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 64819.4us
I20260709 13:22:15.308481 16224 raft_consensus.cc:1275] T 5a959dbecc2648eb85d1cc550f5a3245 P 1ca2725d290e47d68b1a4b609c9816ac [term 20 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 13:22:15.312320 16115 leader_election.cc:304] T 8cc959fb7f704fba9009c0595467d074 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:22:15.238300 23474 consensus_queue.cc:1048] T b6f07842d3aa47858df5665874c86f3a P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.050s
I20260709 13:22:15.332954 23235 raft_consensus.cc:3060] T ada5a004ba7c46819051e3ccec71ffdb P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:22:15.313329 23602 raft_consensus.cc:2804] T 8cc959fb7f704fba9009c0595467d074 P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:22:15.335637 23602 raft_consensus.cc:493] T 8cc959fb7f704fba9009c0595467d074 P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:22:15.336004 23602 raft_consensus.cc:3060] T 8cc959fb7f704fba9009c0595467d074 P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:22:15.264853 23372 raft_consensus.cc:515] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:15.341579 16142 raft_consensus.cc:1240] T b6f07842d3aa47858df5665874c86f3a P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 19. Current term is 20. Ops: [19.13-19.13]
W20260709 13:22:15.343429 16818 consensus_meta.cc:287] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860: Time spent flushing consensus metadata: real 0.512s	user 0.009s	sys 0.000s
I20260709 13:22:15.344717 16818 raft_consensus.cc:2468] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 23.
I20260709 13:22:15.339731 16790 leader_election.cc:304] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:22:15.350045 16790 leader_election.cc:304] T 57a744f5aaf944678a63469cbf514015 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:22:15.053812 22946 consensus_queue.cc:237] T 08c74490c40243a88e62d1760b5cd814 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.11, Last appended by leader: 11, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:15.291788 23554 raft_consensus.cc:2749] T e4dc6b0541ca431ebc757beac00521f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:22:15.306205 23437 raft_consensus.cc:3060] T 68a6838d82ef4cac83507f906139d90f P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:22:15.372326 16115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 60073.9us
I20260709 13:22:15.313454 23326 raft_consensus.cc:493] T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:15.313511 23292 leader_election.cc:290] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:22:15.319090 23562 raft_consensus.cc:2804] T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:22:15.373654 16224 raft_consensus.cc:1240] T e5ad4eb281564936a59f41e387a47156 P 1ca2725d290e47d68b1a4b609c9816ac [term 20 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 19. Current term is 20. Ops: []
I20260709 13:22:15.293552 23491 raft_consensus.cc:515] T ada5a004ba7c46819051e3ccec71ffdb P cb82cc7bd6db46d980c75237a1402860 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:15.240759 23577 raft_consensus.cc:515] T d16dfad0c3b540e5b4b86492ddf87208 P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:15.407925 16115 leader_election.cc:304] T 02838c86531d4f98bf58d1c22b8baf77 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac
I20260709 13:22:15.247186 23590 raft_consensus.cc:515] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:15.248872 23539 raft_consensus.cc:515] T 8638ee363273426db1393dd81e5c7b72 P cb82cc7bd6db46d980c75237a1402860 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:15.411032 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6f07842d3aa47858df5665874c86f3a" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:22:15.324538 23586 raft_consensus.cc:2804] T 8158bbd17b994d2eb96f560bd0f1b88d P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:22:15.416915 16818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74ca8140f35c4ae29e8a664f6bdfc829" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:15.417728 16818 raft_consensus.cc:2393] T 74ca8140f35c4ae29e8a664f6bdfc829 P cb82cc7bd6db46d980c75237a1402860 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 22: Already voted for candidate cb82cc7bd6db46d980c75237a1402860 in this term.
I20260709 13:22:15.417856 16226 raft_consensus.cc:2468] T c2f5f89186204242bf377f7c42b2e2da P 1ca2725d290e47d68b1a4b609c9816ac [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 23.
W20260709 13:22:15.419700 23589 log.cc:927] Time spent T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 0.136s	user 0.000s	sys 0.000s
I20260709 13:22:15.423843 23562 raft_consensus.cc:697] T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045 [term 20 LEADER]: Becoming Leader. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:22:15.424816 23562 consensus_queue.cc:237] T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:15.366055 23587 raft_consensus.cc:2749] T 9e65676b21b748728ad39668cfe722ec P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:22:15.370751 23398 raft_consensus.cc:2804] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:22:15.427798 23398 raft_consensus.cc:493] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:15.428171 23398 raft_consensus.cc:3060] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:22:15.430418 23491 leader_election.cc:290] T ada5a004ba7c46819051e3ccec71ffdb P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
W20260709 13:22:15.431739 23595 log.cc:927] Time spent T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 0.166s	user 0.000s	sys 0.001s
I20260709 13:22:15.435271 16818 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20"
dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
mode: GRACEFUL
new_leader_uuid: "90f3da57796b488c9087a2f523adf045"
 from {username='slave'} at 127.0.0.1:37818
I20260709 13:22:15.435878 16818 raft_consensus.cc:606] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860 [term 23 FOLLOWER]: Received request to transfer leadership to TS 90f3da57796b488c9087a2f523adf045
I20260709 13:22:15.436635 16818 raft_consensus.cc:614] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860 [term 23 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 13:22:15.406891 23582 log.cc:927] Time spent T 84f2da98864b4425bf7c0055951fdd4b P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 0.478s	user 0.001s	sys 0.000s
I20260709 13:22:15.413820 23542 raft_consensus.cc:3055] T c0eab25ce4c64e018a041beeda2867db P 90f3da57796b488c9087a2f523adf045 [term 20 LEADER]: Stepping down as leader of term 20
W20260709 13:22:15.500880 23331 consensus_meta.cc:287] T 1df93779209748b9860f6d893dd3de53 P 647a343a542d4a139fe106c19095129b: Time spent flushing consensus metadata: real 0.533s	user 0.009s	sys 0.000s
I20260709 13:22:15.501458 23331 raft_consensus.cc:515] T 1df93779209748b9860f6d893dd3de53 P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:15.406759 23326 raft_consensus.cc:515] T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:15.503322 23478 raft_consensus.cc:1064] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac: attempting to promote NON_VOTER cb82cc7bd6db46d980c75237a1402860 to VOTER
I20260709 13:22:15.505179 23326 leader_election.cc:290] T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:15.469892 23542 raft_consensus.cc:740] T c0eab25ce4c64e018a041beeda2867db P 90f3da57796b488c9087a2f523adf045 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:22:15.508199 23542 consensus_queue.cc:260] T c0eab25ce4c64e018a041beeda2867db P 90f3da57796b488c9087a2f523adf045 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.11, Last appended by leader: 11, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:15.509642 23542 raft_consensus.cc:3060] T c0eab25ce4c64e018a041beeda2867db P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:22:15.510465 23539 leader_election.cc:290] T 8638ee363273426db1393dd81e5c7b72 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:15.468483 16821 raft_consensus.cc:3060] T 8638ee363273426db1393dd81e5c7b72 P cb82cc7bd6db46d980c75237a1402860 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:22:15.512312 16143 raft_consensus.cc:2393] T 1df93779209748b9860f6d893dd3de53 P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 19: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
I20260709 13:22:15.385464 23409 raft_consensus.cc:2804] T 57a744f5aaf944678a63469cbf514015 P cb82cc7bd6db46d980c75237a1402860 [term 18 FOLLOWER]: Leader election won for term 18
W20260709 13:22:15.508811 23061 consensus_meta.cc:287] T 7c2095a5f4ad49408dbc7f69c606120d P 1ca2725d290e47d68b1a4b609c9816ac: Time spent flushing consensus metadata: real 1.086s	user 0.011s	sys 0.000s
I20260709 13:22:15.528376 16821 raft_consensus.cc:1275] T 8638ee363273426db1393dd81e5c7b72 P cb82cc7bd6db46d980c75237a1402860 [term 20 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 20 index: 16. (index mismatch)
I20260709 13:22:15.438832 16115 leader_election.cc:304] T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: cb82cc7bd6db46d980c75237a1402860
W20260709 13:22:15.530254 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 351638us
I20260709 13:22:15.625138 16265 leader_election.cc:304] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
I20260709 13:22:15.626602 16265 leader_election.cc:304] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
I20260709 13:22:15.407866 23594 consensus_queue.cc:1059] T 5a96373bb0e14dce8a266b2fc6c58507 P 647a343a542d4a139fe106c19095129b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: INVALID_TERM, Last received: 16.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.107s
I20260709 13:22:15.506546 23602 raft_consensus.cc:515] T 8cc959fb7f704fba9009c0595467d074 P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:22:15.596345 23591 log.cc:927] Time spent T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 0.532s	user 0.001s	sys 0.000s
W20260709 13:22:15.508199 23311 consensus_meta.cc:287] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac: Time spent flushing consensus metadata: real 0.829s	user 0.007s	sys 0.003s
I20260709 13:22:15.661070 16224 raft_consensus.cc:1240] T e4dc6b0541ca431ebc757beac00521f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 20. Current term is 21. Ops: []
I20260709 13:22:15.661830 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74ca8140f35c4ae29e8a664f6bdfc829" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
W20260709 13:22:15.663395 16115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 224631us
I20260709 13:22:15.671221 16115 leader_election.cc:304] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860
I20260709 13:22:15.672328 15046 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 13:22:15.622668 23601 raft_consensus.cc:493] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:15.667392 16265 leader_election.cc:304] T fffab26427e24d398e2695aad40ca134 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:22:15.677630 16265 leader_election.cc:304] T 68a6838d82ef4cac83507f906139d90f P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:22:15.503350 23608 raft_consensus.cc:2749] T 8cc959fb7f704fba9009c0595467d074 P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:22:15.665095 16189 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.747s	user 0.003s	sys 0.007s
W20260709 13:22:15.675560 16191 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.741s	user 0.003s	sys 0.007s
I20260709 13:22:15.617420 23586 raft_consensus.cc:697] T 8158bbd17b994d2eb96f560bd0f1b88d P 647a343a542d4a139fe106c19095129b [term 19 LEADER]: Becoming Leader. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:22:15.692041 16151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8158bbd17b994d2eb96f560bd0f1b88d" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
W20260709 13:22:15.682474 16191 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.748s	user 0.003s	sys 0.007s
W20260709 13:22:15.682180 16189 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.764s	user 0.004s	sys 0.007s
W20260709 13:22:15.682947 16116 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.716s	user 0.005s	sys 0.026s
I20260709 13:22:15.683594 16115 leader_election.cc:304] T 2794af5b99984efebdee564035841cb0 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860
I20260709 13:22:15.684543 23357 raft_consensus.cc:3060] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:22:15.658243 23566 raft_consensus.cc:493] T 8a878999c0024d2abe517484180290fc P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:22:15.410166 23563 consensus_queue.cc:237] T 434bff3082d141838068f323bdb0d1ed P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 18.15, Last appended by leader: 15, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:22:15.672397 23600 log.cc:927] Time spent T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 0.433s	user 0.001s	sys 0.000s
I20260709 13:22:15.666826 23326 raft_consensus.cc:2749] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:22:15.662767 23023 raft_consensus.cc:2764] T 02838c86531d4f98bf58d1c22b8baf77 P 647a343a542d4a139fe106c19095129b [term 21 FOLLOWER]: Leader pre-election decision vote started in defunct term 19: won
W20260709 13:22:15.693991 16116 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.727s	user 0.005s	sys 0.026s
W20260709 13:22:15.729347 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 788795us
I20260709 13:22:15.730438 16116 leader_election.cc:304] T fffab26427e24d398e2695aad40ca134 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:22:15.731743 16116 leader_election.cc:304] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:22:15.696054 16115 leader_election.cc:304] T 7c2095a5f4ad49408dbc7f69c606120d P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:22:15.669193 23077 raft_consensus.cc:2749] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
W20260709 13:22:15.685663 23403 consensus_meta.cc:287] T 1df93779209748b9860f6d893dd3de53 P 1ca2725d290e47d68b1a4b609c9816ac: Time spent flushing consensus metadata: real 1.225s	user 0.013s	sys 0.000s
I20260709 13:22:15.733577 16115 leader_election.cc:304] T 18ad7e3af5504571a69d10c00750a01c P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:22:15.708861 16221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ab0ac8347641e6955e9bce7d90c1a2" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:15.734573 16221 raft_consensus.cc:2368] T d2ab0ac8347641e6955e9bce7d90c1a2 P 1ca2725d290e47d68b1a4b609c9816ac [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 for earlier term 19. Current term is 20.
I20260709 13:22:15.709050 16220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b7d1ab519044eda2d5e04a07242f25" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:15.735325 16220 raft_consensus.cc:2468] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 19.
I20260709 13:22:15.736328 16220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7ca61592e848ea9b0fb81a2ce33cdc" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:15.737133 16292 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48674 (request call id 3533) took 1481 ms. Trace:
W20260709 13:22:15.723847 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 46281.8us
I20260709 13:22:15.737753 16292 rpcz_store.cc:276] 0709 13:22:14.273286 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:22:14.569244 (+295958us) service_pool.cc:224] Handling call
0709 13:22:15.736794 (+1167550us) spinlock_profiling.cc:243] Waited 214 ms on lock 0x7b54001ee954. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:22:15.737027 (+   233us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":492098816}
I20260709 13:22:15.664043 23061 raft_consensus.cc:515] T 7c2095a5f4ad49408dbc7f69c606120d P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:15.673045 23609 raft_consensus.cc:2804] T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:22:15.724123 16142 pending_rounds.cc:85] T 5a959dbecc2648eb85d1cc550f5a3245 P 647a343a542d4a139fe106c19095129b: Aborting all ops after (but not including) 10
I20260709 13:22:15.737308 16220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8638ee363273426db1393dd81e5c7b72" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
W20260709 13:22:15.739473 16297 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48674 (request call id 3507) took 2606 ms (client timeout 2355 ms). Trace:
I20260709 13:22:15.714970 23331 leader_election.cc:290] T 1df93779209748b9860f6d893dd3de53 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 19 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:15.681910 23409 raft_consensus.cc:697] T 57a744f5aaf944678a63469cbf514015 P cb82cc7bd6db46d980c75237a1402860 [term 18 LEADER]: Becoming Leader. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:22:15.740983 16295 raft_consensus.cc:1240] T b87147731a68400e8cce9b879daa7c5d P 90f3da57796b488c9087a2f523adf045 [term 21 FOLLOWER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 20. Current term is 21. Ops: []
I20260709 13:22:15.741999 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66ae176961b74813a58bc15c9c4c6f2a" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 27 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:15.783154 16191 leader_election.cc:304] T 02838c86531d4f98bf58d1c22b8baf77 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
I20260709 13:22:15.744107 16142 pending_rounds.cc:107] T 5a959dbecc2648eb85d1cc550f5a3245 P 647a343a542d4a139fe106c19095129b: Aborting uncommitted NO_OP operation due to leader change: 17.11
I20260709 13:22:15.791141 16142 raft_consensus.cc:2889] T 5a959dbecc2648eb85d1cc550f5a3245 P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:22:15.765350 16220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2372981006d4b218215c28964dea623" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 25 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:15.713879 23551 raft_consensus.cc:493] T ee9ed67006144797b1de6ea8c2013a68 P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
W20260709 13:22:15.769572 23583 log.cc:927] Time spent T c4f65cdf00b748bb963ee6e81ef43b79 P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.325s	user 0.001s	sys 0.000s
I20260709 13:22:15.771283 23603 raft_consensus.cc:3055] T 5a96373bb0e14dce8a266b2fc6c58507 P 647a343a542d4a139fe106c19095129b [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:22:15.771842 23061 leader_election.cc:290] T 7c2095a5f4ad49408dbc7f69c606120d P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 18 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:15.774129 16219 raft_consensus.cc:2368] T 7c2095a5f4ad49408dbc7f69c606120d P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b for earlier term 17. Current term is 18.
I20260709 13:22:15.679569 23611 raft_consensus.cc:493] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:22:15.777671 23326 raft_consensus.cc:493] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:22:15.776130 15046 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:22:15.747900 23403 raft_consensus.cc:515] T 1df93779209748b9860f6d893dd3de53 P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:15.690034 23024 raft_consensus.cc:493] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:22:15.675464 23311 raft_consensus.cc:515] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:22:15.730011 23597 log.cc:927] Time spent T 3a7ca61592e848ea9b0fb81a2ce33cdc P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 0.406s	user 0.000s	sys 0.001s
I20260709 13:22:15.699985 23610 raft_consensus.cc:2749] T fffab26427e24d398e2695aad40ca134 P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:22:15.698287 23322 raft_consensus.cc:2749] T 2794af5b99984efebdee564035841cb0 P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:22:15.724686 23566 raft_consensus.cc:515] T 8a878999c0024d2abe517484180290fc P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:15.729501 23372 leader_election.cc:290] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
W20260709 13:22:15.786017 16263 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:22:15.704891 23474 raft_consensus.cc:493] T 829b4ad8c5a54eababeea1181b960d4b P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
W20260709 13:22:15.832463 23552 consensus_meta.cc:287] T 5a96373bb0e14dce8a266b2fc6c58507 P 1ca2725d290e47d68b1a4b609c9816ac: Time spent flushing consensus metadata: real 0.983s	user 0.008s	sys 0.001s
I20260709 13:22:15.832538 23474 raft_consensus.cc:515] T 829b4ad8c5a54eababeea1181b960d4b P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:15.833405 23577 leader_election.cc:290] T d16dfad0c3b540e5b4b86492ddf87208 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:22:15.834206 23590 leader_election.cc:290] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:15.835289 23602 leader_election.cc:290] T 8cc959fb7f704fba9009c0595467d074 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 19 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
W20260709 13:22:15.836937 23235 consensus_meta.cc:287] T ada5a004ba7c46819051e3ccec71ffdb P 90f3da57796b488c9087a2f523adf045: Time spent flushing consensus metadata: real 0.503s	user 0.011s	sys 0.001s
I20260709 13:22:15.836650 23311 leader_election.cc:290] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 21 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:15.837874 16218 raft_consensus.cc:1240] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 20. Current term is 21. Ops: [20.16-20.16]
W20260709 13:22:15.839192 23373 consensus_meta.cc:287] T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac: Time spent flushing consensus metadata: real 1.130s	user 0.006s	sys 0.005s
I20260709 13:22:15.839632 23373 raft_consensus.cc:515] T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:22:15.744845 16297 rpcz_store.cc:269] 0709 13:22:13.132614 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:22:13.790735 (+658121us) service_pool.cc:224] Handling call
0709 13:22:15.739185 (+1948450us) spinlock_profiling.cc:243] Waited 637 ms on lock 0x7b5400178c54. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:22:15.739378 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1464548096}
I20260709 13:22:15.841045 23559 raft_consensus.cc:493] T ada5a004ba7c46819051e3ccec71ffdb P 90f3da57796b488c9087a2f523adf045 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:15.841593 23559 raft_consensus.cc:515] T ada5a004ba7c46819051e3ccec71ffdb P 90f3da57796b488c9087a2f523adf045 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:15.841780 23373 leader_election.cc:290] T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 22 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:15.842569 16294 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48684 (request call id 3253) took 2573 ms. Trace:
I20260709 13:22:15.843484 16223 raft_consensus.cc:2393] T 1df93779209748b9860f6d893dd3de53 P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 19: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
I20260709 13:22:15.844024 23403 leader_election.cc:290] T 1df93779209748b9860f6d893dd3de53 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 19 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:15.740505 23594 raft_consensus.cc:2804] T 7c2095a5f4ad49408dbc7f69c606120d P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:22:15.741514 23606 raft_consensus.cc:2749] T fffab26427e24d398e2695aad40ca134 P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:22:15.820749 23061 raft_consensus.cc:2804] T 02838c86531d4f98bf58d1c22b8baf77 P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:22:15.775527 16221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada5a004ba7c46819051e3ccec71ffdb" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:15.796900 23331 raft_consensus.cc:493] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:15.778154 23622 raft_consensus.cc:493] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:22:15.819451 16292 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48674 (request call id 3530) took 1565 ms. Trace:
I20260709 13:22:15.815443 23617 raft_consensus.cc:2749] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:22:15.682600 23601 raft_consensus.cc:515] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:15.822451 23024 raft_consensus.cc:515] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:15.824115 23612 consensus_queue.cc:1048] T 5a959dbecc2648eb85d1cc550f5a3245 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.005s
I20260709 13:22:15.827898 23620 raft_consensus.cc:493] T c4f65cdf00b748bb963ee6e81ef43b79 P 90f3da57796b488c9087a2f523adf045 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:22:15.820760 23326 raft_consensus.cc:515] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:15.820382 23586 consensus_queue.cc:237] T 8158bbd17b994d2eb96f560bd0f1b88d P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:15.833097 23552 raft_consensus.cc:2749] T 5a96373bb0e14dce8a266b2fc6c58507 P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:22:15.798045 16220 raft_consensus.cc:2410] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 for term 25 because replica has last-logged OpId of term: 23 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 22 index: 15.
I20260709 13:22:15.748365 23609 raft_consensus.cc:493] T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Starting leader election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:22:15.811610 23077 raft_consensus.cc:493] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:15.876312 23566 leader_election.cc:290] T 8a878999c0024d2abe517484180290fc P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:15.878446 23609 raft_consensus.cc:3060] T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:22:15.879698 23357 raft_consensus.cc:2749] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:22:15.841928 16295 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40474 (request call id 1350) took 2106 ms. Trace:
I20260709 13:22:15.880889 16295 rpcz_store.cc:276] 0709 13:22:13.735522 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:22:15.046544 (+1311022us) service_pool.cc:224] Handling call
0709 13:22:15.736985 (+690441us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:22:15.787191 (+ 50206us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:22:15.841749 (+ 54558us) spinlock_profiling.cc:243] Waited 683 ms on lock 0x7b54001ee950. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5e8ad 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:22:15.841876 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1570746752}
I20260709 13:22:15.882728 16295 raft_consensus.cc:1240] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 18. Current term is 19. Ops: []
I20260709 13:22:15.883502 16295 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48674 (request call id 3542) took 1349 ms. Trace:
I20260709 13:22:15.883848 16295 rpcz_store.cc:276] 0709 13:22:14.534604 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:22:15.882174 (+1347570us) service_pool.cc:224] Handling call
0709 13:22:15.882529 (+   355us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:22:15.883222 (+   693us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:22:15.883444 (+   222us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:22:15.885442 16295 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48674 (request call id 3543) took 1344 ms. Trace:
I20260709 13:22:15.885772 16295 rpcz_store.cc:276] 0709 13:22:14.540642 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:22:15.884593 (+1343951us) service_pool.cc:224] Handling call
0709 13:22:15.884914 (+   321us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:22:15.885103 (+   189us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:22:15.885131 (+    28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:22:15.885175 (+    44us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:22:15.885215 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:22:15.885270 (+    55us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:22:15.885387 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:22:15.886611 23287 consensus_meta.cc:287] T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045: Time spent flushing consensus metadata: real 0.708s	user 0.006s	sys 0.002s
I20260709 13:22:15.887650 16219 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57604 (request call id 3460) took 1373 ms. Trace:
I20260709 13:22:15.803654 23603 raft_consensus.cc:740] T 5a96373bb0e14dce8a266b2fc6c58507 P 647a343a542d4a139fe106c19095129b [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:22:15.888218 23559 leader_election.cc:290] T ada5a004ba7c46819051e3ccec71ffdb P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:15.888064 23398 raft_consensus.cc:515] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:15.889178 23603 consensus_queue.cc:260] T 5a96373bb0e14dce8a266b2fc6c58507 P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:15.890400 23603 raft_consensus.cc:3060] T 5a96373bb0e14dce8a266b2fc6c58507 P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:22:15.890599 23601 leader_election.cc:290] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:15.871326 23061 raft_consensus.cc:697] T 02838c86531d4f98bf58d1c22b8baf77 P 1ca2725d290e47d68b1a4b609c9816ac [term 21 LEADER]: Becoming Leader. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:22:15.892585 23024 leader_election.cc:290] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
W20260709 13:22:15.809795 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 78092.6us
I20260709 13:22:15.892992 23061 consensus_queue.cc:237] T 02838c86531d4f98bf58d1c22b8baf77 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:15.893534 16116 leader_election.cc:304] T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
I20260709 13:22:15.895314 23357 raft_consensus.cc:2804] T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:22:15.862258 23622 raft_consensus.cc:515] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:15.895467 16116 leader_election.cc:304] T 5a959dbecc2648eb85d1cc550f5a3245 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:22:15.895771 23357 raft_consensus.cc:493] T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b [term 23 FOLLOWER]: Starting leader election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:22:15.896232 23357 raft_consensus.cc:3060] T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:22:15.896711 16116 leader_election.cc:304] T ecc1e363230043fe9fef390f0b979f20 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:22:15.897892 23622 leader_election.cc:290] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:15.898479 23622 raft_consensus.cc:493] T d16dfad0c3b540e5b4b86492ddf87208 P 1ca2725d290e47d68b1a4b609c9816ac [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:22:15.817778 23611 raft_consensus.cc:515] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:15.874382 16221 raft_consensus.cc:2468] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 21.
I20260709 13:22:15.899931 23437 raft_consensus.cc:515] T 68a6838d82ef4cac83507f906139d90f P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:15.900957 23611 leader_election.cc:290] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:15.888091 16219 rpcz_store.cc:276] 0709 13:22:14.514160 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:22:14.514530 (+   370us) service_pool.cc:224] Handling call
0709 13:22:15.887437 (+1372907us) spinlock_profiling.cc:243] Waited 1.25 s on lock 0x7b54002c8154. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:22:15.887588 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2877108608}
W20260709 13:22:15.849395 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 110315us
I20260709 13:22:15.842972 16294 rpcz_store.cc:276] 0709 13:22:13.268886 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:22:13.967824 (+698938us) service_pool.cc:224] Handling call
0709 13:22:14.077487 (+109663us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:22:14.077713 (+   226us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:22:14.077736 (+    23us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:22:14.078194 (+   458us) log.cc:844] Serialized 26 byte log entry
0709 13:22:14.419130 (+340936us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:22:14.419191 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:22:14.419262 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:22:15.837472 (+1418210us) spinlock_profiling.cc:243] Waited 1.51 ms on lock 0x7b240024ba38. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69ad8fadb1 00007f69ad8f89c4 00007f69ad8f6acb 00007f69adab1dc9 00007f69adab1a55 00007f69adab0fb0 00007f69b0727488 00007f69b0727be8 00007f69b06ff398 00007f69b06fa76a 00007f69b06f9bb6
0709 13:22:15.842255 (+  4783us) raft_consensus.cc:1685] finished
0709 13:22:15.842275 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:22:15.842506 (+   231us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3468544,"thread_start_us":333778,"threads_started":1,"wal-append.queue_time_us":534563}
I20260709 13:22:15.834900 23474 leader_election.cc:290] T 829b4ad8c5a54eababeea1181b960d4b P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:15.857379 16292 rpcz_store.cc:276] 0709 13:22:14.254337 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:22:15.742905 (+1488568us) service_pool.cc:224] Handling call
0709 13:22:15.743245 (+   340us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:22:15.743472 (+   227us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:22:15.807397 (+ 63925us) log.cc:844] Serialized 14 byte log entry
0709 13:22:15.818984 (+ 11587us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:22:15.819041 (+    57us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:22:15.819096 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:22:15.819250 (+   154us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:22:15.819398 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10120,"raft.queue_time_us":19749,"raft.run_cpu_time_us":79,"raft.run_wall_time_us":78,"thread_start_us":8830,"threads_started":2}
I20260709 13:22:15.786294 23539 raft_consensus.cc:493] T c9944d5e4b8c4109af9955c4f08c9095 P cb82cc7bd6db46d980c75237a1402860 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
W20260709 13:22:15.858681 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 69227.6us
I20260709 13:22:15.821604 23625 raft_consensus.cc:2749] T 68a6838d82ef4cac83507f906139d90f P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:22:15.850894 23594 raft_consensus.cc:697] T 7c2095a5f4ad49408dbc7f69c606120d P 647a343a542d4a139fe106c19095129b [term 17 LEADER]: Becoming Leader. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:22:15.802497 23551 raft_consensus.cc:515] T ee9ed67006144797b1de6ea8c2013a68 P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:15.926060 16220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1df93779209748b9860f6d893dd3de53" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:22:15.873438 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1df93779209748b9860f6d893dd3de53" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:22:15.928907 16293 raft_consensus.cc:3060] T 1df93779209748b9860f6d893dd3de53 P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:22:15.860569 23331 raft_consensus.cc:515] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:15.876102 23077 raft_consensus.cc:515] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:15.841073 16297 raft_consensus.cc:1240] T 26354361e9b044cea1c2b86c4882fe92 P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 16. Current term is 17. Ops: []
I20260709 13:22:15.936828 16148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b7d1ab519044eda2d5e04a07242f25" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:22:15.937167 16297 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48674 (request call id 3541) took 1409 ms. Trace:
I20260709 13:22:15.937666 16148 raft_consensus.cc:3060] T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:22:15.937578 16297 rpcz_store.cc:276] 0709 13:22:14.528190 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:22:15.840674 (+1312484us) service_pool.cc:224] Handling call
0709 13:22:15.840914 (+   240us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:22:15.936906 (+ 95992us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:22:15.937112 (+   206us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:22:15.890673 23398 leader_election.cc:290] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 20 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:22:15.935220 16220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d16dfad0c3b540e5b4b86492ddf87208" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:15.950681 16790 leader_election.cc:400] T d2ab0ac8347641e6955e9bce7d90c1a2 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 19 pre-election: Vote denied by peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943) with higher term. Message: Invalid argument: T d2ab0ac8347641e6955e9bce7d90c1a2 P 1ca2725d290e47d68b1a4b609c9816ac [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 for earlier term 19. Current term is 20.
I20260709 13:22:15.951076 16790 leader_election.cc:403] T d2ab0ac8347641e6955e9bce7d90c1a2 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 19 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:22:15.921296 23606 raft_consensus.cc:2804] T ecc1e363230043fe9fef390f0b979f20 P 647a343a542d4a139fe106c19095129b [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:22:15.875075 23620 raft_consensus.cc:515] T c4f65cdf00b748bb963ee6e81ef43b79 P 90f3da57796b488c9087a2f523adf045 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:15.944777 16219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada5a004ba7c46819051e3ccec71ffdb" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
W20260709 13:22:15.931041 16117 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 85680.6us
I20260709 13:22:15.898947 23622 raft_consensus.cc:515] T d16dfad0c3b540e5b4b86492ddf87208 P 1ca2725d290e47d68b1a4b609c9816ac [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:15.935714 23559 raft_consensus.cc:493] T 8638ee363273426db1393dd81e5c7b72 P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:15.959949 23286 raft_consensus.cc:2749] T 5a959dbecc2648eb85d1cc550f5a3245 P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:22:15.866592 23409 consensus_queue.cc:237] T 57a744f5aaf944678a63469cbf514015 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:15.969797 23551 leader_election.cc:290] T ee9ed67006144797b1de6ea8c2013a68 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:15.971793 23620 leader_election.cc:290] T c4f65cdf00b748bb963ee6e81ef43b79 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:22:15.971973 16219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87ac0d1f7e8a419f9bcbc04aee0e067e" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:15.972776 23331 leader_election.cc:290] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:15.970625 23357 raft_consensus.cc:515] T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:15.974107 23594 consensus_queue.cc:237] T 7c2095a5f4ad49408dbc7f69c606120d P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.8, Last appended by leader: 8, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:22:15.976117 23399 consensus_meta.cc:287] T 3884f864a96d428785895e5463e5728f P cb82cc7bd6db46d980c75237a1402860: Time spent flushing consensus metadata: real 1.176s	user 0.011s	sys 0.001s
I20260709 13:22:16.023447 23399 raft_consensus.cc:2749] T 3884f864a96d428785895e5463e5728f P cb82cc7bd6db46d980c75237a1402860 [term 16 FOLLOWER]: Leader election lost for term 15. Reason: Vote denied by peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741) with higher term. Message: Invalid argument: T 3884f864a96d428785895e5463e5728f P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 for earlier term 15. Current term is 16.
I20260709 13:22:15.940878 23322 consensus_queue.cc:1048] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.039s
I20260709 13:22:15.931703 23539 raft_consensus.cc:515] T c9944d5e4b8c4109af9955c4f08c9095 P cb82cc7bd6db46d980c75237a1402860 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:16.025748 22677 raft_consensus.cc:493] T c0eab25ce4c64e018a041beeda2867db P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:15.977666 16217 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57604 (request call id 3451) took 1709 ms. Trace:
I20260709 13:22:16.026372 22677 raft_consensus.cc:515] T c0eab25ce4c64e018a041beeda2867db P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:16.026677 23539 leader_election.cc:290] T c9944d5e4b8c4109af9955c4f08c9095 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:16.026619 16217 rpcz_store.cc:276] 0709 13:22:14.268581 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:22:14.270632 (+  2051us) service_pool.cc:224] Handling call
0709 13:22:14.297546 (+ 26914us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:22:14.514131 (+216585us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:22:15.977431 (+1463300us) spinlock_profiling.cc:243] Waited 216 ms on lock 0x7b5400396054. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:22:15.977605 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":495748352}
I20260709 13:22:16.028234 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c74490c40243a88e62d1760b5cd814" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:16.028826 22677 leader_election.cc:290] T c0eab25ce4c64e018a041beeda2867db P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:15.887084 23287 raft_consensus.cc:515] T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:22:16.033918 16265 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:22:15.966786 16293 raft_consensus.cc:2468] T 1df93779209748b9860f6d893dd3de53 P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 19.
I20260709 13:22:15.963378 23398 raft_consensus.cc:2749] T d2ab0ac8347641e6955e9bce7d90c1a2 P cb82cc7bd6db46d980c75237a1402860 [term 20 FOLLOWER]: Leader pre-election lost for term 19. Reason: Vote denied by peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943) with higher term. Message: Invalid argument: T d2ab0ac8347641e6955e9bce7d90c1a2 P 1ca2725d290e47d68b1a4b609c9816ac [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 for earlier term 19. Current term is 20.
I20260709 13:22:15.991513 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d16dfad0c3b540e5b4b86492ddf87208" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:22:16.043370 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c74490c40243a88e62d1760b5cd814" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:22:16.044432 16149 raft_consensus.cc:2393] T 08c74490c40243a88e62d1760b5cd814 P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 20: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
I20260709 13:22:15.920205 23553 consensus_queue.cc:1059] T e5ad4eb281564936a59f41e387a47156 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: INVALID_TERM, Last received: 19.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.032s
I20260709 13:22:15.932204 23024 consensus_queue.cc:1059] T e4dc6b0541ca431ebc757beac00521f1 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: INVALID_TERM, Last received: 20.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.049s
I20260709 13:22:16.048533 16299 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48684 (request call id 3229) took 2942 ms. Trace:
I20260709 13:22:16.049209 23553 raft_consensus.cc:3055] T e5ad4eb281564936a59f41e387a47156 P 90f3da57796b488c9087a2f523adf045 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:22:16.049757 23553 raft_consensus.cc:740] T e5ad4eb281564936a59f41e387a47156 P 90f3da57796b488c9087a2f523adf045 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:22:15.989634 23559 raft_consensus.cc:515] T 8638ee363273426db1393dd81e5c7b72 P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:16.044965 16220 raft_consensus.cc:2468] T d16dfad0c3b540e5b4b86492ddf87208 P 1ca2725d290e47d68b1a4b609c9816ac [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 20.
I20260709 13:22:16.046886 23287 leader_election.cc:290] T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 21 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:22:16.001837 16219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4f65cdf00b748bb963ee6e81ef43b79" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:16.053470 22610 raft_consensus.cc:3060] T 8d4ab870a05347868956aa8b841b2981 P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:22:16.002393 23077 leader_election.cc:290] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:22:16.055687 16142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c61a39fd4f84424b129c1a6f9ba404d" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:22:16.056533 16142 raft_consensus.cc:2393] T 9c61a39fd4f84424b129c1a6f9ba404d P 647a343a542d4a139fe106c19095129b [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 21: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
I20260709 13:22:16.049301 23024 raft_consensus.cc:3055] T e4dc6b0541ca431ebc757beac00521f1 P 90f3da57796b488c9087a2f523adf045 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:22:16.057150 23024 raft_consensus.cc:740] T e4dc6b0541ca431ebc757beac00521f1 P 90f3da57796b488c9087a2f523adf045 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:22:16.057461 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4f65cdf00b748bb963ee6e81ef43b79" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:22:16.013367 16223 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57600 (request call id 2161) took 1033 ms. Trace:
I20260709 13:22:15.961831 23474 consensus_queue.cc:1059] T b6f07842d3aa47858df5665874c86f3a P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: INVALID_TERM, Last received: 18.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.085s
I20260709 13:22:16.058015 23024 consensus_queue.cc:260] T e4dc6b0541ca431ebc757beac00521f1 P 90f3da57796b488c9087a2f523adf045 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.11, Last appended by leader: 11, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:16.058081 16223 rpcz_store.cc:276] 0709 13:22:14.980151 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:22:14.984745 (+  4594us) service_pool.cc:224] Handling call
0709 13:22:16.013130 (+1028385us) spinlock_profiling.cc:243] Waited 836 ms on lock 0x7b5400246654. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:22:16.013298 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1923536256}
I20260709 13:22:16.058816 23474 raft_consensus.cc:3055] T b6f07842d3aa47858df5665874c86f3a P 1ca2725d290e47d68b1a4b609c9816ac [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:22:16.059242 23474 raft_consensus.cc:740] T b6f07842d3aa47858df5665874c86f3a P 1ca2725d290e47d68b1a4b609c9816ac [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:22:16.059782 16223 raft_consensus.cc:1240] T 66f75124215f4ca6bef831e174a792d8 P 1ca2725d290e47d68b1a4b609c9816ac [term 19 LEADER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 18. Current term is 19. Ops: [18.14-18.14]
W20260709 13:22:16.016021 23578 log.cc:927] Time spent T 806d7690c2b644959152074bb2cab0a7 P cb82cc7bd6db46d980c75237a1402860: Append to log took a long time: real 0.986s	user 0.005s	sys 0.000s
I20260709 13:22:16.060012 23474 consensus_queue.cc:260] T b6f07842d3aa47858df5665874c86f3a P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.13, Last appended by leader: 13, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:15.984388 23606 raft_consensus.cc:697] T ecc1e363230043fe9fef390f0b979f20 P 647a343a542d4a139fe106c19095129b [term 23 LEADER]: Becoming Leader. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:22:16.019459 23351 raft_consensus.cc:493] T 23f82224f4564fea93572e8206bad1fe P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:22:16.061702 16224 raft_consensus.cc:1275] T d2ab0ac8347641e6955e9bce7d90c1a2 P 1ca2725d290e47d68b1a4b609c9816ac [term 20 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 16 index: 14. Preceding OpId from leader: term: 20 index: 16. (index mismatch)
I20260709 13:22:16.061650 23351 raft_consensus.cc:515] T 23f82224f4564fea93572e8206bad1fe P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:16.029280 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cc959fb7f704fba9009c0595467d074" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:22:15.978480 16222 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57600 (request call id 2159) took 1085 ms. Trace:
I20260709 13:22:16.029765 16824 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49896 (request call id 1617) took 2323 ms. Trace:
I20260709 13:22:16.030592 23326 leader_election.cc:290] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:16.031903 16221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e4bfb9090c8460eb14fa7e0d818e7f1" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:16.032291 23357 leader_election.cc:290] T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 24 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:16.034365 23256 consensus_queue.cc:260] T 84f2da98864b4425bf7c0055951fdd4b P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:22:16.047252 16296 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48674 (request call id 3514) took 2618 ms (client timeout 2091 ms). Trace:
I20260709 13:22:16.054337 16219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b7d1ab519044eda2d5e04a07242f25" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:22:16.012254 23433 raft_consensus.cc:3055] T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:22:16.057648 16142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87ac0d1f7e8a419f9bcbc04aee0e067e" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:22:16.058538 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cc959fb7f704fba9009c0595467d074" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:22:16.017709 23437 leader_election.cc:290] T 68a6838d82ef4cac83507f906139d90f P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 20 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:16.062038 23606 consensus_queue.cc:237] T ecc1e363230043fe9fef390f0b979f20 P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.13, Last appended by leader: 13, Current term: 23, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:16.013876 23491 raft_consensus.cc:493] T 9e65676b21b748728ad39668cfe722ec P cb82cc7bd6db46d980c75237a1402860 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:16.086553 16142 raft_consensus.cc:2468] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 20.
I20260709 13:22:16.042141 23539 raft_consensus.cc:493] T 8c11074579b046d99e585c2818e18a2c P cb82cc7bd6db46d980c75237a1402860 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:22:16.061193 23024 raft_consensus.cc:3060] T e4dc6b0541ca431ebc757beac00521f1 P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:22:15.987998 23612 raft_consensus.cc:493] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:22:16.074859 16265 leader_election.cc:304] T 434bff3082d141838068f323bdb0d1ed P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:22:16.087101 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c61a39fd4f84424b129c1a6f9ba404d" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:22:16.090857 16219 raft_consensus.cc:3060] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:22:16.109959 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02838c86531d4f98bf58d1c22b8baf77" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:22:16.111748 16300 raft_consensus.cc:3060] T 02838c86531d4f98bf58d1c22b8baf77 P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:22:16.085587 16222 rpcz_store.cc:276] 0709 13:22:14.893107 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:22:14.893231 (+   124us) service_pool.cc:224] Handling call
0709 13:22:14.911572 (+ 18341us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:22:14.920727 (+  9155us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:22:14.920774 (+    47us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:22:14.920814 (+    40us) raft_consensus.cc:1645] Marking committed up to 15
0709 13:22:14.920850 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:22:15.978276 (+1057426us) spinlock_profiling.cc:243] Waited 8.95 ms on lock 0x7b5400245c54. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:22:15.978319 (+    43us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:22:15.978426 (+   107us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":20591360}
I20260709 13:22:16.104328 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c61a39fd4f84424b129c1a6f9ba404d" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:16.113852 16293 raft_consensus.cc:2468] T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 21.
I20260709 13:22:16.113130 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7ca61592e848ea9b0fb81a2ce33cdc" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:22:16.115197 16297 raft_consensus.cc:3060] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:22:16.114521 16292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5ad4eb281564936a59f41e387a47156" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:22:16.115857 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c74490c40243a88e62d1760b5cd814" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:22:16.116422 16293 raft_consensus.cc:3060] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:22:16.082581 16148 raft_consensus.cc:2468] T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 20.
I20260709 13:22:16.123533 16293 raft_consensus.cc:2468] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 20.
I20260709 13:22:16.127812 16295 raft_consensus.cc:1275] T 8638ee363273426db1393dd81e5c7b72 P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 20 index: 16. (index mismatch)
I20260709 13:22:16.048925 16299 rpcz_store.cc:276] 0709 13:22:13.105893 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:22:13.953244 (+847351us) service_pool.cc:224] Handling call
0709 13:22:13.953565 (+   321us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:22:14.858824 (+905259us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:22:14.858861 (+    37us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:22:14.858907 (+    46us) raft_consensus.cc:1645] Marking committed up to 16
0709 13:22:14.858947 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:22:16.048282 (+1189335us) spinlock_profiling.cc:243] Waited 905 ms on lock 0x7b54002ce2d4. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:22:16.048340 (+    58us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:22:16.048471 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2081436032}
W20260709 13:22:16.079144 16296 rpcz_store.cc:269] 0709 13:22:13.428646 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:22:13.813548 (+384902us) service_pool.cc:224] Handling call
0709 13:22:16.041930 (+2228382us) spinlock_profiling.cc:243] Waited 1.81 s on lock 0x7b5400399ed4. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:22:16.047135 (+  5205us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4156049152}
I20260709 13:22:16.133352 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a378d7f796f94615b3b347119a9880fd" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:16.000571 23616 raft_consensus.cc:493] T 18ad7e3af5504571a69d10c00750a01c P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:22:16.140815 16148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f82224f4564fea93572e8206bad1fe" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:22:16.133742 16296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c2095a5f4ad49408dbc7f69c606120d" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:22:16.141947 16296 raft_consensus.cc:3060] T 7c2095a5f4ad49408dbc7f69c606120d P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:22:16.134426 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c61a39fd4f84424b129c1a6f9ba404d" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:22:16.053472 23559 leader_election.cc:290] T 8638ee363273426db1393dd81e5c7b72 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:16.087247 16824 rpcz_store.cc:276] 0709 13:22:13.706615 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:22:13.706769 (+   154us) service_pool.cc:224] Handling call
0709 13:22:13.707051 (+   282us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:22:13.826716 (+119665us) raft_consensus.cc:1534] Early marking committed up to index 14
0709 13:22:14.130964 (+304248us) log.cc:844] Serialized 14 byte log entry
0709 13:22:14.594184 (+463220us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:22:14.594240 (+    56us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:22:14.594301 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:22:16.029474 (+1435173us) spinlock_profiling.cc:243] Waited 119 ms on lock 0x7b54007c0e54. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:22:16.029541 (+    67us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:22:16.029694 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":32198,"prepare.queue_time_us":345127,"prepare.run_cpu_time_us":79,"prepare.run_wall_time_us":59846,"raft.queue_time_us":38103,"raft.run_cpu_time_us":127,"raft.run_wall_time_us":127,"spinlock_wait_cycles":274683776,"thread_start_us":731891,"threads_started":2,"wal-append.queue_time_us":470285,"wal-append.run_cpu_time_us":1620,"wal-append.run_wall_time_us":1144365}
I20260709 13:22:16.144250 16265 leader_election.cc:304] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:22:16.153465 16294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74ca8140f35c4ae29e8a664f6bdfc829" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:16.157929 16219 raft_consensus.cc:2468] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 20.
I20260709 13:22:16.158402 16294 raft_consensus.cc:2468] T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 22.
I20260709 13:22:16.159240 16294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0eab25ce4c64e018a041beeda2867db" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:16.144595 16295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1df93779209748b9860f6d893dd3de53" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:22:16.160199 16295 raft_consensus.cc:2393] T 1df93779209748b9860f6d893dd3de53 P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 19: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
I20260709 13:22:16.160955 16295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdbd78acb4cf46298e0de8ba0d130e19" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
W20260709 13:22:16.149924 16191 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:22:16.143461 16299 raft_consensus.cc:3060] T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:22:16.157280 16222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8638ee363273426db1393dd81e5c7b72" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:16.139710 16217 raft_consensus.cc:2368] T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 for earlier term 21. Current term is 22.
I20260709 13:22:16.161257 16297 raft_consensus.cc:2468] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 90f3da57796b488c9087a2f523adf045 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 21.
I20260709 13:22:16.162976 16219 raft_consensus.cc:1275] T 19baa7bc367844c288bca8f81439c413 P 1ca2725d290e47d68b1a4b609c9816ac [term 23 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 23 index: 13. (index mismatch)
I20260709 13:22:16.085978 23404 raft_consensus.cc:3060] T cdbd78acb4cf46298e0de8ba0d130e19 P 90f3da57796b488c9087a2f523adf045 [term 22 FOLLOWER]: Advancing to term 23
W20260709 13:22:16.163681 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 209984us
I20260709 13:22:16.050655 23553 consensus_queue.cc:260] T e5ad4eb281564936a59f41e387a47156 P 90f3da57796b488c9087a2f523adf045 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:16.164975 16190 leader_election.cc:304] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: cb82cc7bd6db46d980c75237a1402860
I20260709 13:22:16.168193 16219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9944d5e4b8c4109af9955c4f08c9095" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:16.086017 23433 raft_consensus.cc:740] T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:22:16.134255 23210 raft_consensus.cc:2804] T 434bff3082d141838068f323bdb0d1ed P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:22:16.088896 23351 leader_election.cc:290] T 23f82224f4564fea93572e8206bad1fe P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:16.142160 23256 raft_consensus.cc:3060] T 84f2da98864b4425bf7c0055951fdd4b P 1ca2725d290e47d68b1a4b609c9816ac [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:22:16.157708 23474 raft_consensus.cc:3060] T b6f07842d3aa47858df5665874c86f3a P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:22:16.179550 23566 raft_consensus.cc:2804] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:22:16.252807 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada5a004ba7c46819051e3ccec71ffdb" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:16.277726 16297 raft_consensus.cc:2468] T ada5a004ba7c46819051e3ccec71ffdb P 90f3da57796b488c9087a2f523adf045 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 21.
I20260709 13:22:16.278647 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e4bfb9090c8460eb14fa7e0d818e7f1" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:16.253693 23061 raft_consensus.cc:493] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:16.254474 23601 raft_consensus.cc:493] T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:16.108656 23491 raft_consensus.cc:515] T 9e65676b21b748728ad39668cfe722ec P cb82cc7bd6db46d980c75237a1402860 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:16.154608 23616 raft_consensus.cc:515] T 18ad7e3af5504571a69d10c00750a01c P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } }
I20260709 13:22:16.166764 23287 raft_consensus.cc:2804] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:22:16.157543 23399 raft_consensus.cc:493] T 66f75124215f4ca6bef831e174a792d8 P cb82cc7bd6db46d980c75237a1402860 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:16.090516 23539 raft_consensus.cc:515] T 8c11074579b046d99e585c2818e18a2c P cb82cc7bd6db46d980c75237a1402860 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:16.093067 23612 raft_consensus.cc:515] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } }
I20260709 13:22:16.280835 16300 raft_consensus.cc:2468] T 02838c86531d4f98bf58d1c22b8baf77 P 90f3da57796b488c9087a2f523adf045 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 21.
I20260709 13:22:16.279825 16297 raft_consensus.cc:2468] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 21.
I20260709 13:22:16.316349 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8638ee363273426db1393dd81e5c7b72" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:16.286197 16296 raft_consensus.cc:2468] T 7c2095a5f4ad49408dbc7f69c606120d P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 18.
I20260709 13:22:16.256474 16266 leader_election.cc:304] T 3884f864a96d428785895e5463e5728f P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
I20260709 13:22:16.188297 23382 consensus_queue.cc:1048] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.101s
I20260709 13:22:16.296872 16299 raft_consensus.cc:2468] T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 22.
I20260709 13:22:16.277351 23566 raft_consensus.cc:493] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Starting leader election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:22:16.316442 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a878999c0024d2abe517484180290fc" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:16.317173 16300 raft_consensus.cc:2393] T 8638ee363273426db1393dd81e5c7b72 P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 in current term 20: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
I20260709 13:22:16.255880 23210 raft_consensus.cc:697] T 434bff3082d141838068f323bdb0d1ed P 90f3da57796b488c9087a2f523adf045 [term 20 LEADER]: Becoming Leader. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:22:16.317553 16296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada5a004ba7c46819051e3ccec71ffdb" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
W20260709 13:22:16.317945 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 140000us
I20260709 13:22:16.321288 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9944d5e4b8c4109af9955c4f08c9095" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
W20260709 13:22:16.321277 16115 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.501s	user 0.006s	sys 0.026s
I20260709 13:22:16.323977 16296 raft_consensus.cc:2468] T ada5a004ba7c46819051e3ccec71ffdb P 90f3da57796b488c9087a2f523adf045 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 21.
W20260709 13:22:16.324396 16115 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.505s	user 0.007s	sys 0.026s
I20260709 13:22:16.325145 16296 raft_consensus.cc:1240] T 68a6838d82ef4cac83507f906139d90f P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 19. Current term is 20. Ops: []
I20260709 13:22:16.326009 16297 raft_consensus.cc:1240] T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045 [term 20 LEADER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 19. Current term is 20. Ops: [19.13-19.13]
I20260709 13:22:16.327260 16297 raft_consensus.cc:1240] T 84f2da98864b4425bf7c0055951fdd4b P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 16. Current term is 17. Ops: [16.10-16.10]
I20260709 13:22:16.322218 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f82224f4564fea93572e8206bad1fe" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
W20260709 13:22:16.323143 16266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 66787.4us
I20260709 13:22:16.328789 16266 leader_election.cc:304] T b87147731a68400e8cce9b879daa7c5d P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
I20260709 13:22:16.329804 16299 raft_consensus.cc:2468] T 23f82224f4564fea93572e8206bad1fe P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 17.
I20260709 13:22:16.313050 23287 raft_consensus.cc:493] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:22:16.331238 16266 leader_election.cc:304] T 26354361e9b044cea1c2b86c4882fe92 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 647a343a542d4a139fe106c19095129b
I20260709 13:22:16.334359 16822 raft_consensus.cc:2468] T 3884f864a96d428785895e5463e5728f P cb82cc7bd6db46d980c75237a1402860 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 16.
I20260709 13:22:16.337218 16190 leader_election.cc:304] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:22:16.321985 23566 raft_consensus.cc:3060] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:22:16.344132 16149 raft_consensus.cc:1240] T 84f2da98864b4425bf7c0055951fdd4b P 647a343a542d4a139fe106c19095129b [term 17 LEADER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 16. Current term is 17. Ops: [16.10-16.10]
I20260709 13:22:16.346302 23622 leader_election.cc:290] T d16dfad0c3b540e5b4b86492ddf87208 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:16.346578 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c2095a5f4ad49408dbc7f69c606120d" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:22:16.328754 23077 raft_consensus.cc:2749] T 3884f864a96d428785895e5463e5728f P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:22:16.336364 23326 raft_consensus.cc:2749] T b87147731a68400e8cce9b879daa7c5d P 90f3da57796b488c9087a2f523adf045 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:22:16.341545 23562 raft_consensus.cc:2804] T 26354361e9b044cea1c2b86c4882fe92 P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:22:16.353540 23562 raft_consensus.cc:697] T 26354361e9b044cea1c2b86c4882fe92 P 90f3da57796b488c9087a2f523adf045 [term 17 LEADER]: Becoming Leader. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:22:16.354473 23562 consensus_queue.cc:237] T 26354361e9b044cea1c2b86c4882fe92 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:16.358258 23612 leader_election.cc:290] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:22:16.344682 23630 raft_consensus.cc:2804] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:22:16.360322 23398 raft_consensus.cc:493] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:22:16.280153 23409 consensus_queue.cc:1059] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: INVALID_TERM, Last received: 18.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.052s
I20260709 13:22:16.361109 23398 raft_consensus.cc:515] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:16.361670 23409 raft_consensus.cc:3055] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:22:16.362262 23409 raft_consensus.cc:740] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:22:16.342134 16190 leader_election.cc:304] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:22:16.348212 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c61a39fd4f84424b129c1a6f9ba404d" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:22:16.290105 23061 raft_consensus.cc:515] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:16.358762 16115 leader_election.cc:304] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
W20260709 13:22:16.359737 23636 log.cc:927] Time spent T 434bff3082d141838068f323bdb0d1ed P cb82cc7bd6db46d980c75237a1402860: Append to log took a long time: real 0.081s	user 0.000s	sys 0.001s
W20260709 13:22:16.340205 23542 consensus_meta.cc:287] T c0eab25ce4c64e018a041beeda2867db P 90f3da57796b488c9087a2f523adf045: Time spent flushing consensus metadata: real 0.830s	user 0.002s	sys 0.013s
I20260709 13:22:16.311010 23601 raft_consensus.cc:515] T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:16.366671 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68a6838d82ef4cac83507f906139d90f" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:22:16.360529 23630 raft_consensus.cc:493] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:16.287338 23433 consensus_queue.cc:260] T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.14, Last appended by leader: 14, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:16.337350 23287 raft_consensus.cc:3060] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:22:16.317564 23399 raft_consensus.cc:515] T 66f75124215f4ca6bef831e174a792d8 P cb82cc7bd6db46d980c75237a1402860 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:16.380277 16299 raft_consensus.cc:2393] T 68a6838d82ef4cac83507f906139d90f P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 20: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
I20260709 13:22:16.393740 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d16dfad0c3b540e5b4b86492ddf87208" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:16.392454 23373 raft_consensus.cc:2804] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:22:16.395702 23373 raft_consensus.cc:493] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Starting leader election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:22:16.396211 23373 raft_consensus.cc:3060] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:22:16.396622 23399 leader_election.cc:290] T 66f75124215f4ca6bef831e174a792d8 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:22:16.397490 23433 raft_consensus.cc:3060] T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:22:16.397233 23210 consensus_queue.cc:237] T 434bff3082d141838068f323bdb0d1ed P 90f3da57796b488c9087a2f523adf045 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 18.15, Last appended by leader: 15, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:16.380895 16790 leader_election.cc:304] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:22:16.341965 23437 raft_consensus.cc:493] T 9c61a39fd4f84424b129c1a6f9ba404d P 647a343a542d4a139fe106c19095129b [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:16.340627 23648 raft_consensus.cc:493] T 74ca8140f35c4ae29e8a664f6bdfc829 P cb82cc7bd6db46d980c75237a1402860 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:16.407545 23648 raft_consensus.cc:515] T 74ca8140f35c4ae29e8a664f6bdfc829 P cb82cc7bd6db46d980c75237a1402860 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:16.409042 23061 leader_election.cc:290] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:16.410372 23648 leader_election.cc:290] T 74ca8140f35c4ae29e8a664f6bdfc829 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:16.412268 23398 leader_election.cc:290] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
W20260709 13:22:16.419704 23609 consensus_meta.cc:287] T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b: Time spent flushing consensus metadata: real 0.541s	user 0.010s	sys 0.001s
I20260709 13:22:16.419535 23437 raft_consensus.cc:515] T 9c61a39fd4f84424b129c1a6f9ba404d P 647a343a542d4a139fe106c19095129b [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:16.420219 23609 raft_consensus.cc:515] T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:16.422075 23437 leader_election.cc:290] T 9c61a39fd4f84424b129c1a6f9ba404d P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:16.422508 23609 leader_election.cc:290] T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 18 election: Requested vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
W20260709 13:22:16.368454 16789 connection.cc:256] RPC call timeout handler was delayed by -1.0621294679999664s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1623s for a response
I20260709 13:22:16.390048 23630 raft_consensus.cc:3060] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:22:16.424099 23539 leader_election.cc:290] T 8c11074579b046d99e585c2818e18a2c P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:16.424249 23557 raft_consensus.cc:493] T cdbd78acb4cf46298e0de8ba0d130e19 P 90f3da57796b488c9087a2f523adf045 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:16.424865 23557 raft_consensus.cc:515] T cdbd78acb4cf46298e0de8ba0d130e19 P 90f3da57796b488c9087a2f523adf045 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:16.382730 23326 raft_consensus.cc:493] T 66ae176961b74813a58bc15c9c4c6f2a P 90f3da57796b488c9087a2f523adf045 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:22:16.413750 23491 leader_election.cc:290] T 9e65676b21b748728ad39668cfe722ec P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:16.363426 23409 consensus_queue.cc:260] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.16, Last appended by leader: 16, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:16.430487 23409 raft_consensus.cc:3060] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:22:16.400813 16790 leader_election.cc:304] T ada5a004ba7c46819051e3ccec71ffdb P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:22:16.431890 16790 leader_election.cc:304] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:22:16.435539 16816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:22:16.436223 16816 raft_consensus.cc:2376] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860 [term 23 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 647a343a542d4a139fe106c19095129b in term 23. Re-sending same reply.
I20260709 13:22:16.416102 23616 leader_election.cc:290] T 18ad7e3af5504571a69d10c00750a01c P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:22:16.436775 16823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2372981006d4b218215c28964dea623" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:22:16.437323 23491 raft_consensus.cc:2804] T ada5a004ba7c46819051e3ccec71ffdb P cb82cc7bd6db46d980c75237a1402860 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:22:16.403607 23399 raft_consensus.cc:2749] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:22:16.437788 23491 raft_consensus.cc:493] T ada5a004ba7c46819051e3ccec71ffdb P cb82cc7bd6db46d980c75237a1402860 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:16.438246 23491 raft_consensus.cc:3060] T ada5a004ba7c46819051e3ccec71ffdb P cb82cc7bd6db46d980c75237a1402860 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:22:16.438304 23399 raft_consensus.cc:2804] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:22:16.427888 23638 raft_consensus.cc:493] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:16.403219 23542 raft_consensus.cc:493] T 2794af5b99984efebdee564035841cb0 P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:16.427798 23326 raft_consensus.cc:515] T 66ae176961b74813a58bc15c9c4c6f2a P 90f3da57796b488c9087a2f523adf045 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:16.439667 16823 raft_consensus.cc:2393] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 24: Already voted for candidate cb82cc7bd6db46d980c75237a1402860 in this term.
I20260709 13:22:16.439828 23542 raft_consensus.cc:515] T 2794af5b99984efebdee564035841cb0 P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:16.440424 16818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3884f864a96d428785895e5463e5728f" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 16 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:22:16.441345 23326 leader_election.cc:290] T 66ae176961b74813a58bc15c9c4c6f2a P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:16.441371 16819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada5a004ba7c46819051e3ccec71ffdb" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:16.441735 16818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8638ee363273426db1393dd81e5c7b72" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:16.441949 23542 leader_election.cc:290] T 2794af5b99984efebdee564035841cb0 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:16.431807 16825 raft_consensus.cc:1240] T 26354361e9b044cea1c2b86c4882fe92 P cb82cc7bd6db46d980c75237a1402860 [term 17 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 16. Current term is 17. Ops: []
I20260709 13:22:16.444720 16825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e4bfb9090c8460eb14fa7e0d818e7f1" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:16.444885 16817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74ca8140f35c4ae29e8a664f6bdfc829" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:16.445533 16817 raft_consensus.cc:2468] T 74ca8140f35c4ae29e8a664f6bdfc829 P cb82cc7bd6db46d980c75237a1402860 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 22.
I20260709 13:22:16.446048 16825 raft_consensus.cc:1275] T 19baa7bc367844c288bca8f81439c413 P cb82cc7bd6db46d980c75237a1402860 [term 23 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 23 index: 13. (index mismatch)
I20260709 13:22:16.442991 16818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f82224f4564fea93572e8206bad1fe" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:16.439558 16824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5d59407eb545b895d8d922e0dd69fd" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 24 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:16.416381 23601 leader_election.cc:290] T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:16.439069 23399 raft_consensus.cc:697] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 [term 20 LEADER]: Becoming Leader. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:22:16.439575 23638 raft_consensus.cc:515] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:16.428966 23539 raft_consensus.cc:493] T 829b4ad8c5a54eababeea1181b960d4b P cb82cc7bd6db46d980c75237a1402860 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
W20260709 13:22:16.416236 23603 consensus_meta.cc:287] T 5a96373bb0e14dce8a266b2fc6c58507 P 647a343a542d4a139fe106c19095129b: Time spent flushing consensus metadata: real 0.525s	user 0.010s	sys 0.003s
I20260709 13:22:16.448500 16789 leader_election.cc:304] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:22:16.449092 16816 raft_consensus.cc:1275] T d2ab0ac8347641e6955e9bce7d90c1a2 P cb82cc7bd6db46d980c75237a1402860 [term 20 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 18 index: 15. Preceding OpId from leader: term: 20 index: 16. (index mismatch)
I20260709 13:22:16.453215 23539 raft_consensus.cc:515] T 829b4ad8c5a54eababeea1181b960d4b P cb82cc7bd6db46d980c75237a1402860 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:16.456918 23539 leader_election.cc:290] T 829b4ad8c5a54eababeea1181b960d4b P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:22:16.458662 23638 leader_election.cc:290] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:16.460106 16789 leader_election.cc:304] T 19baa7bc367844c288bca8f81439c413 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:22:16.461875 16789 leader_election.cc:304] T 2794af5b99984efebdee564035841cb0 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 647a343a542d4a139fe106c19095129b
I20260709 13:22:16.465896 23539 raft_consensus.cc:2804] T 2794af5b99984efebdee564035841cb0 P cb82cc7bd6db46d980c75237a1402860 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:22:16.459554 23398 raft_consensus.cc:2749] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:22:16.463301 23638 raft_consensus.cc:2749] T 19baa7bc367844c288bca8f81439c413 P cb82cc7bd6db46d980c75237a1402860 [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:22:16.469594 23608 raft_consensus.cc:493] T 8d4ab870a05347868956aa8b841b2981 P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:16.467679 23539 raft_consensus.cc:697] T 2794af5b99984efebdee564035841cb0 P cb82cc7bd6db46d980c75237a1402860 [term 18 LEADER]: Becoming Leader. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:22:16.487419 23608 raft_consensus.cc:515] T 8d4ab870a05347868956aa8b841b2981 P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:16.495932 23630 raft_consensus.cc:515] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:16.497802 23630 leader_election.cc:290] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 22 election: Requested vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
W20260709 13:22:16.545403 16117 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:22:16.546489 16117 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:22:16.549348 16149 raft_consensus.cc:3060] T 9c61a39fd4f84424b129c1a6f9ba404d P 647a343a542d4a139fe106c19095129b [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:22:16.550155 16115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 191275us
I20260709 13:22:16.550828 16221 raft_consensus.cc:1275] T b6f07842d3aa47858df5665874c86f3a P 1ca2725d290e47d68b1a4b609c9816ac [term 20 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 20 index: 13. (term mismatch)
I20260709 13:22:16.551403 16221 pending_rounds.cc:85] T b6f07842d3aa47858df5665874c86f3a P 1ca2725d290e47d68b1a4b609c9816ac: Aborting all ops after (but not including) 12
I20260709 13:22:16.551844 16221 pending_rounds.cc:107] T b6f07842d3aa47858df5665874c86f3a P 1ca2725d290e47d68b1a4b609c9816ac: Aborting uncommitted NO_OP operation due to leader change: 19.13
I20260709 13:22:16.552326 16221 raft_consensus.cc:2889] T b6f07842d3aa47858df5665874c86f3a P 1ca2725d290e47d68b1a4b609c9816ac [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:22:16.384351 16190 leader_election.cc:304] T 7c2095a5f4ad49408dbc7f69c606120d P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 647a343a542d4a139fe106c19095129b
I20260709 13:22:16.555130 16115 leader_election.cc:304] T b87147731a68400e8cce9b879daa7c5d P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860
I20260709 13:22:16.560792 16147 raft_consensus.cc:2468] T 8d4ab870a05347868956aa8b841b2981 P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 19.
I20260709 13:22:16.561420 16223 raft_consensus.cc:1275] T 84f2da98864b4425bf7c0055951fdd4b P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 17 index: 10. (term mismatch)
I20260709 13:22:16.561950 16223 pending_rounds.cc:85] T 84f2da98864b4425bf7c0055951fdd4b P 1ca2725d290e47d68b1a4b609c9816ac: Aborting all ops after (but not including) 9
I20260709 13:22:16.562269 16223 pending_rounds.cc:107] T 84f2da98864b4425bf7c0055951fdd4b P 1ca2725d290e47d68b1a4b609c9816ac: Aborting uncommitted NO_OP operation due to leader change: 16.10
I20260709 13:22:16.562564 16223 raft_consensus.cc:2889] T 84f2da98864b4425bf7c0055951fdd4b P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:22:16.562686 16295 raft_consensus.cc:2468] T cdbd78acb4cf46298e0de8ba0d130e19 P 90f3da57796b488c9087a2f523adf045 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 23.
I20260709 13:22:16.564262 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66f75124215f4ca6bef831e174a792d8" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 20 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
W20260709 13:22:16.565124 16266 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:22:16.565352 16299 raft_consensus.cc:1275] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
W20260709 13:22:16.565677 16266 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:22:16.386898 16294 raft_consensus.cc:2468] T c0eab25ce4c64e018a041beeda2867db P 90f3da57796b488c9087a2f523adf045 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 21.
I20260709 13:22:16.569736 16300 raft_consensus.cc:1240] T 5a96373bb0e14dce8a266b2fc6c58507 P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 16. Current term is 17. Ops: []
W20260709 13:22:16.572242 16189 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:22:16.560118 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 175749us
I20260709 13:22:16.574376 16190 leader_election.cc:304] T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:22:16.586370 23024 raft_consensus.cc:493] T 9e65676b21b748728ad39668cfe722ec P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:16.597051 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74ca8140f35c4ae29e8a664f6bdfc829" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 23 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:16.597499 23024 raft_consensus.cc:515] T 9e65676b21b748728ad39668cfe722ec P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:16.492861 23539 consensus_queue.cc:237] T 2794af5b99984efebdee564035841cb0 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.15, Last appended by leader: 15, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:16.475327 23399 consensus_queue.cc:237] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 18.15, Last appended by leader: 15, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:16.603736 23086 raft_consensus.cc:2749] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:22:16.605044 23321 consensus_queue.cc:1059] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:22:16.606594 23322 raft_consensus.cc:3060] T b87147731a68400e8cce9b879daa7c5d P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:22:16.577898 16789 leader_election.cc:304] T 26354361e9b044cea1c2b86c4882fe92 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:22:16.617125 16115 leader_election.cc:304] T 1df93779209748b9860f6d893dd3de53 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:22:16.627413 23437 consensus_queue.cc:1059] T 26354361e9b044cea1c2b86c4882fe92 P 647a343a542d4a139fe106c19095129b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: INVALID_TERM, Last received: 16.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:22:16.629334 23474 raft_consensus.cc:2804] T 7c2095a5f4ad49408dbc7f69c606120d P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:22:16.630216 23474 raft_consensus.cc:697] T 7c2095a5f4ad49408dbc7f69c606120d P 1ca2725d290e47d68b1a4b609c9816ac [term 18 LEADER]: Becoming Leader. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:22:16.631536 23256 raft_consensus.cc:2804] T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:22:16.632299 23256 raft_consensus.cc:697] T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac [term 22 LEADER]: Becoming Leader. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:22:16.632027 23474 consensus_queue.cc:237] T 7c2095a5f4ad49408dbc7f69c606120d P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 16.8, Last appended by leader: 8, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:16.638337 23373 raft_consensus.cc:515] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:16.641981 23601 consensus_queue.cc:1059] T 68a6838d82ef4cac83507f906139d90f P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: INVALID_TERM, Last received: 19.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:22:16.645744 16222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdbd78acb4cf46298e0de8ba0d130e19" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:16.609069 22610 consensus_queue.cc:1059] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 647a343a542d4a139fe106c19095129b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: INVALID_TERM, Last received: 18.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.040s
I20260709 13:22:16.647296 23645 raft_consensus.cc:2749] T 26354361e9b044cea1c2b86c4882fe92 P cb82cc7bd6db46d980c75237a1402860 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:22:16.647872 16789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 70433.7us
I20260709 13:22:16.649288 16789 leader_election.cc:400] T 434bff3082d141838068f323bdb0d1ed P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 19 election: Vote denied by peer 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389) with higher term. Message: Invalid argument: T 434bff3082d141838068f323bdb0d1ed P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 for earlier term 19. Current term is 20.
I20260709 13:22:16.649372 16218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c11074579b046d99e585c2818e18a2c" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:16.652418 16219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "829b4ad8c5a54eababeea1181b960d4b" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:16.647055 16222 raft_consensus.cc:2468] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 22.
I20260709 13:22:16.655359 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746aabf856ab401c9ef5e859371e7b67" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:16.656315 16224 raft_consensus.cc:2468] T 746aabf856ab401c9ef5e859371e7b67 P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 17.
I20260709 13:22:16.659787 23630 consensus_queue.cc:1048] T 08c74490c40243a88e62d1760b5cd814 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
W20260709 13:22:16.660550 16291 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48674 (request call id 3538) took 2157 ms (client timeout 2032 ms). Trace:
I20260709 13:22:16.667891 23321 raft_consensus.cc:3055] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:22:16.668462 23321 raft_consensus.cc:740] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:22:16.669507 16790 leader_election.cc:304] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:22:16.669992 23321 consensus_queue.cc:260] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 22.13, Last appended by leader: 13, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:16.670996 16790 leader_election.cc:304] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:22:16.670928 23645 raft_consensus.cc:2804] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:22:16.678041 23645 raft_consensus.cc:493] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:16.678524 23645 raft_consensus.cc:3060] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:22:16.680888 16817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62fe3346cd3c42c4a55e14af9073bca4" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 27 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:16.685804 16825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e4bfb9090c8460eb14fa7e0d818e7f1" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:16.634028 23256 consensus_queue.cc:237] T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.11, Last appended by leader: 11, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:16.688059 16817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "829b4ad8c5a54eababeea1181b960d4b" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:16.682632 23287 raft_consensus.cc:515] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 90f3da57796b488c9087a2f523adf045 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:16.691670 16817 raft_consensus.cc:2468] T 829b4ad8c5a54eababeea1181b960d4b P cb82cc7bd6db46d980c75237a1402860 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 18.
W20260709 13:22:16.667716 16291 rpcz_store.cc:269] 0709 13:22:14.503488 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:22:14.856420 (+352932us) service_pool.cc:224] Handling call
0709 13:22:16.660018 (+1803598us) spinlock_profiling.cc:243] Waited 30.4 ms on lock 0x7b54000cf154. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:22:16.660323 (+   305us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":69917568}
I20260709 13:22:16.692775 16821 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49906 (request call id 1410) took 1266 ms. Trace:
W20260709 13:22:16.698143 16115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 137866us
I20260709 13:22:16.694861 16821 rpcz_store.cc:276] 0709 13:22:15.426423 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:22:15.427313 (+   890us) service_pool.cc:224] Handling call
0709 13:22:15.427630 (+   317us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:22:15.624275 (+196645us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:22:16.692246 (+1067971us) spinlock_profiling.cc:243] Waited 40.7 ms on lock 0x7b54007706d4. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:22:16.692568 (+   322us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":93641600}
W20260709 13:22:16.703099 16822 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49896 (request call id 1623) took 2050 ms (client timeout 1562 ms). Trace:
I20260709 13:22:16.703879 16142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1df93779209748b9860f6d893dd3de53" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
W20260709 13:22:16.703800 16822 rpcz_store.cc:269] 0709 13:22:14.652868 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:22:14.658239 (+  5371us) service_pool.cc:224] Handling call
0709 13:22:16.700546 (+2042307us) spinlock_profiling.cc:243] Waited 1.32 s on lock 0x7b54006bf654. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:22:16.702817 (+  2271us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3040306560}
I20260709 13:22:16.705991 16816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5718c623ebee4d2bb770f0f9ddb2f164" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 21 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:22:16.705979 16818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada5a004ba7c46819051e3ccec71ffdb" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:16.719991 16296 raft_consensus.cc:1240] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 90f3da57796b488c9087a2f523adf045 [term 21 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 20. Current term is 21. Ops: []
I20260709 13:22:16.724843 16298 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48684 (request call id 3249) took 3613 ms. Trace:
I20260709 13:22:16.724967 23625 raft_consensus.cc:493] T 8c11074579b046d99e585c2818e18a2c P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:22:16.726222 23625 raft_consensus.cc:515] T 8c11074579b046d99e585c2818e18a2c P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:16.727651 23210 raft_consensus.cc:493] T 84f2da98864b4425bf7c0055951fdd4b P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:16.727845 23553 raft_consensus.cc:3060] T e5ad4eb281564936a59f41e387a47156 P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:22:16.728659 23210 raft_consensus.cc:515] T 84f2da98864b4425bf7c0055951fdd4b P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:22:16.732511 16820 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49892 (request call id 1099) took 2121 ms (client timeout 2194 ms). Trace:
I20260709 13:22:16.733107 16817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdbd78acb4cf46298e0de8ba0d130e19" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:16.734659 23373 leader_election.cc:290] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 22 election: Requested vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:16.708258 23437 raft_consensus.cc:2804] T 1df93779209748b9860f6d893dd3de53 P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:22:16.739532 23650 raft_consensus.cc:2804] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:22:16.740332 23650 raft_consensus.cc:493] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:22:16.740954 23650 raft_consensus.cc:3060] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:22:16.739583 16822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee9ed67006144797b1de6ea8c2013a68" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:16.743224 23024 leader_election.cc:290] T 9e65676b21b748728ad39668cfe722ec P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:16.739624 23437 raft_consensus.cc:697] T 1df93779209748b9860f6d893dd3de53 P 647a343a542d4a139fe106c19095129b [term 19 LEADER]: Becoming Leader. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
W20260709 13:22:16.744654 23656 log.cc:927] Time spent T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 0.056s	user 0.000s	sys 0.001s
I20260709 13:22:16.745344 23437 consensus_queue.cc:237] T 1df93779209748b9860f6d893dd3de53 P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:16.725620 16298 rpcz_store.cc:276] 0709 13:22:13.119988 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:22:13.956258 (+836270us) service_pool.cc:224] Handling call
0709 13:22:13.956535 (+   277us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:22:13.956720 (+   185us) raft_consensus.cc:1534] Early marking committed up to index 16
0709 13:22:14.419492 (+462772us) log.cc:844] Serialized 14 byte log entry
0709 13:22:16.682896 (+2263404us) spinlock_profiling.cc:243] Waited 58.1 ms on lock 0x7b1800340cc0. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b693ade1 00007f69b69475b1 00007f69b07074bc 00007f69ada5548b 00007f69b68afab9 00007f69ada7169b 00007f69ada6f93f 00007f69ada71c02 00007f69b06f920b 00007f69b06fed23 00007f69b06ff23a
0709 13:22:16.694389 (+ 11493us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:22:16.701192 (+  6803us) raft_consensus.cc:1645] Marking committed up to 16
0709 13:22:16.721447 (+ 20255us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:22:16.724421 (+  2974us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:22:16.724654 (+   233us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":95394,"prepare.queue_time_us":641460,"prepare.run_cpu_time_us":36,"prepare.run_wall_time_us":39,"raft.queue_time_us":802,"raft.run_cpu_time_us":143,"raft.run_wall_time_us":143,"spinlock_wait_cycles":133597568,"thread_start_us":534859,"threads_started":2,"wal-append.queue_time_us":427137,"wal-append.run_cpu_time_us":1130,"wal-append.run_wall_time_us":1007458}
I20260709 13:22:16.754019 23608 leader_election.cc:290] T 8d4ab870a05347868956aa8b841b2981 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:16.755748 16142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "829b4ad8c5a54eababeea1181b960d4b" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:22:16.757719 16142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "829b4ad8c5a54eababeea1181b960d4b" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:22:16.759673 16142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62fe3346cd3c42c4a55e14af9073bca4" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 27 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:22:16.761073 16142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0eab25ce4c64e018a041beeda2867db" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:22:16.761849 16142 raft_consensus.cc:2468] T c0eab25ce4c64e018a041beeda2867db P 647a343a542d4a139fe106c19095129b [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 21.
W20260709 13:22:16.735445 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 64612.6us
I20260709 13:22:16.738497 16818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a378d7f796f94615b3b347119a9880fd" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
W20260709 13:22:16.736634 16820 rpcz_store.cc:269] 0709 13:22:14.611176 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:22:14.611344 (+   168us) service_pool.cc:224] Handling call
0709 13:22:16.731964 (+2120620us) spinlock_profiling.cc:243] Waited 1.46 s on lock 0x7b54007553d4. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:22:16.732229 (+   265us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3364385024}
W20260709 13:22:16.763093 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 62868.8us
W20260709 13:22:16.764642 16191 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:22:16.779564 16789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 128652us
I20260709 13:22:16.783406 16148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee9ed67006144797b1de6ea8c2013a68" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:22:16.784986 16148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d16dfad0c3b540e5b4b86492ddf87208" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:22:16.786252 16148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e65676b21b748728ad39668cfe722ec" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 21 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
W20260709 13:22:16.762768 23642 log.cc:927] Time spent T 7c2095a5f4ad49408dbc7f69c606120d P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 0.363s	user 0.000s	sys 0.001s
I20260709 13:22:16.782845 16149 raft_consensus.cc:2468] T 9c61a39fd4f84424b129c1a6f9ba404d P 647a343a542d4a139fe106c19095129b [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 22.
I20260709 13:22:16.789431 16148 raft_consensus.cc:2410] T 9e65676b21b748728ad39668cfe722ec P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 for term 21 because replica has last-logged OpId of term: 19 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 14.
I20260709 13:22:16.782866 16789 leader_election.cc:304] T 3a7ca61592e848ea9b0fb81a2ce33cdc P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac
I20260709 13:22:16.792867 16142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a878999c0024d2abe517484180290fc" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:22:16.745525 23605 raft_consensus.cc:3055] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 647a343a542d4a139fe106c19095129b [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:22:16.800005 23210 leader_election.cc:290] T 84f2da98864b4425bf7c0055951fdd4b P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:22:16.801860 23605 raft_consensus.cc:740] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 647a343a542d4a139fe106c19095129b [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:22:16.805645 23491 raft_consensus.cc:515] T ada5a004ba7c46819051e3ccec71ffdb P cb82cc7bd6db46d980c75237a1402860 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:16.806167 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:16.792665 23321 raft_consensus.cc:3060] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:22:16.805687 16294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c61a39fd4f84424b129c1a6f9ba404d" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 22 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:16.808398 16294 raft_consensus.cc:2393] T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 22: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
I20260709 13:22:16.803104 23605 consensus_queue.cc:260] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:16.809485 23645 raft_consensus.cc:515] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:16.811074 16817 raft_consensus.cc:2368] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac for earlier term 23. Current term is 24.
I20260709 13:22:16.812943 16291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31969392ccdb43b491911a319e8bc7e1" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:22:16.813661 16291 raft_consensus.cc:3060] T 31969392ccdb43b491911a319e8bc7e1 P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:22:16.813786 23287 leader_election.cc:290] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 21 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:22:16.814970 16190 leader_election.cc:304] T c0eab25ce4c64e018a041beeda2867db P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:22:16.815526 16819 raft_consensus.cc:2393] T ada5a004ba7c46819051e3ccec71ffdb P cb82cc7bd6db46d980c75237a1402860 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 22: Already voted for candidate cb82cc7bd6db46d980c75237a1402860 in this term.
I20260709 13:22:16.812217 23605 raft_consensus.cc:3060] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:22:16.819643 22677 raft_consensus.cc:2804] T c0eab25ce4c64e018a041beeda2867db P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:22:16.820147 22677 raft_consensus.cc:493] T c0eab25ce4c64e018a041beeda2867db P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:16.820577 22677 raft_consensus.cc:3060] T c0eab25ce4c64e018a041beeda2867db P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:22:16.822563 23650 raft_consensus.cc:515] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:16.825059 23650 leader_election.cc:290] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 18 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:22:16.831591 16820 raft_consensus.cc:1275] T 66f75124215f4ca6bef831e174a792d8 P cb82cc7bd6db46d980c75237a1402860 [term 19 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 13:22:16.841121 16226 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33050 (request call id 1284) took 1129 ms. Trace:
W20260709 13:22:16.844683 16115 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:22:16.847996 23357 raft_consensus.cc:3055] T 26354361e9b044cea1c2b86c4882fe92 P 647a343a542d4a139fe106c19095129b [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:22:16.848503 23357 raft_consensus.cc:740] T 26354361e9b044cea1c2b86c4882fe92 P 647a343a542d4a139fe106c19095129b [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:22:16.849083 16191 leader_election.cc:304] T 1df93779209748b9860f6d893dd3de53 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:22:16.849761 23357 consensus_queue.cc:260] T 26354361e9b044cea1c2b86c4882fe92 P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:16.853379 16818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada5a004ba7c46819051e3ccec71ffdb" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:22:16.841812 16226 rpcz_store.cc:276] 0709 13:22:15.711320 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:22:15.790272 (+ 78952us) service_pool.cc:224] Handling call
0709 13:22:15.790607 (+   335us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:22:15.951998 (+161391us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:22:15.952030 (+    32us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:22:15.952071 (+    41us) raft_consensus.cc:1645] Marking committed up to 15
0709 13:22:15.952100 (+    29us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:22:16.840367 (+888267us) spinlock_profiling.cc:243] Waited 161 ms on lock 0x7b54002b4654. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:22:16.840597 (+   230us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:22:16.840874 (+   277us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":370707840}
I20260709 13:22:16.856086 16822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e4bfb9090c8460eb14fa7e0d818e7f1" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:22:16.859310 16820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2372981006d4b218215c28964dea623" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:16.860213 16142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e65676b21b748728ad39668cfe722ec" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:22:16.860692 23630 raft_consensus.cc:2749] T 1df93779209748b9860f6d893dd3de53 P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:22:16.860567 23373 raft_consensus.cc:493] T 84f2da98864b4425bf7c0055951fdd4b P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:16.861665 23373 raft_consensus.cc:515] T 84f2da98864b4425bf7c0055951fdd4b P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:16.834717 23322 raft_consensus.cc:2749] T b87147731a68400e8cce9b879daa7c5d P 647a343a542d4a139fe106c19095129b [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:22:16.860953 16142 raft_consensus.cc:2468] T 9e65676b21b748728ad39668cfe722ec P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 19.
I20260709 13:22:16.880517 23373 leader_election.cc:290] T 84f2da98864b4425bf7c0055951fdd4b P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:16.886168 16142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87ac0d1f7e8a419f9bcbc04aee0e067e" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:22:16.839390 23566 raft_consensus.cc:515] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:22:16.887741 16790 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:22:16.862841 23478 consensus_queue.cc:237] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.13, Last appended by leader: 12, Current term: 20, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:16.895458 16148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d4ab870a05347868956aa8b841b2981" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:22:16.896896 16223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdbd78acb4cf46298e0de8ba0d130e19" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:22:16.897593 16223 raft_consensus.cc:3060] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [term 22 FOLLOWER]: Advancing to term 24
I20260709 13:22:16.904245 16291 raft_consensus.cc:2468] T 31969392ccdb43b491911a319e8bc7e1 P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 18.
I20260709 13:22:16.905326 23539 raft_consensus.cc:2764] T 3a7ca61592e848ea9b0fb81a2ce33cdc P cb82cc7bd6db46d980c75237a1402860 [term 21 FOLLOWER]: Leader pre-election decision vote started in defunct term 20: won
W20260709 13:22:16.906194 23607 log.cc:927] Time spent T af57996a8be84d398f0f023e8d08ef77 P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 1.231s	user 0.003s	sys 0.001s
I20260709 13:22:16.910079 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d4ab870a05347868956aa8b841b2981" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:16.910981 16293 raft_consensus.cc:2468] T 8d4ab870a05347868956aa8b841b2981 P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 19.
I20260709 13:22:16.911250 23357 raft_consensus.cc:3060] T 26354361e9b044cea1c2b86c4882fe92 P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:22:16.888077 23287 raft_consensus.cc:493] T ecc1e363230043fe9fef390f0b979f20 P 90f3da57796b488c9087a2f523adf045 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:16.914362 16816 raft_consensus.cc:2410] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac for term 21 because replica has last-logged OpId of term: 20 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 15.
I20260709 13:22:16.913810 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746aabf856ab401c9ef5e859371e7b67" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:22:16.919623 16226 raft_consensus.cc:3060] T 746aabf856ab401c9ef5e859371e7b67 P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:22:16.920346 16217 raft_consensus.cc:2410] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 for term 23 because replica has last-logged OpId of term: 21 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 14.
W20260709 13:22:16.925606 16266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 126650us
I20260709 13:22:16.928434 16142 raft_consensus.cc:3060] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:22:16.933302 16789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 150661us
I20260709 13:22:16.931378 16115 leader_election.cc:304] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:22:16.930357 23645 leader_election.cc:290] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 24 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:16.947566 16291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d4ab870a05347868956aa8b841b2981" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
W20260709 13:22:16.951400 16192 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:22:16.954674 16291 raft_consensus.cc:2468] T 8d4ab870a05347868956aa8b841b2981 P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 19.
I20260709 13:22:16.956736 16226 raft_consensus.cc:2468] T 746aabf856ab401c9ef5e859371e7b67 P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 18.
I20260709 13:22:16.947378 23287 raft_consensus.cc:515] T ecc1e363230043fe9fef390f0b979f20 P 90f3da57796b488c9087a2f523adf045 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:16.957523 16148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18ad7e3af5504571a69d10c00750a01c" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:22:16.940886 23491 leader_election.cc:290] T ada5a004ba7c46819051e3ccec71ffdb P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 22 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:16.962118 16789 leader_election.cc:304] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:22:16.967242 23023 raft_consensus.cc:2749] T 18ad7e3af5504571a69d10c00750a01c P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:22:16.967937 16294 raft_consensus.cc:1275] T 02838c86531d4f98bf58d1c22b8baf77 P 90f3da57796b488c9087a2f523adf045 [term 21 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260709 13:22:16.971294 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada5a004ba7c46819051e3ccec71ffdb" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:22:16.972208 16300 raft_consensus.cc:3060] T ada5a004ba7c46819051e3ccec71ffdb P 90f3da57796b488c9087a2f523adf045 [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:22:16.973160 16115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 56342.7us
I20260709 13:22:16.974363 16115 leader_election.cc:304] T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:22:16.975315 23321 raft_consensus.cc:2804] T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:22:16.975334 16145 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40844 (request call id 3046) took 2267 ms. Trace:
I20260709 13:22:16.976445 16115 leader_election.cc:304] T 8d4ab870a05347868956aa8b841b2981 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:22:16.978372 16223 raft_consensus.cc:2468] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 24.
I20260709 13:22:16.981977 22610 raft_consensus.cc:2804] T 8d4ab870a05347868956aa8b841b2981 P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:22:16.982329 22610 raft_consensus.cc:493] T 8d4ab870a05347868956aa8b841b2981 P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:16.982591 23645 raft_consensus.cc:2749] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 13:22:16.975463 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2794af5b99984efebdee564035841cb0" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 19 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:22:16.982681 22610 raft_consensus.cc:3060] T 8d4ab870a05347868956aa8b841b2981 P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:22:16.984417 16265 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:22:16.985677 16142 raft_consensus.cc:2468] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 647a343a542d4a139fe106c19095129b [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 21.
I20260709 13:22:16.988967 16294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e4bfb9090c8460eb14fa7e0d818e7f1" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:22:16.983594 23321 raft_consensus.cc:697] T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b [term 18 LEADER]: Becoming Leader. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
W20260709 13:22:16.990975 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 60935.1us
I20260709 13:22:16.978052 16145 rpcz_store.cc:276] 0709 13:22:14.694602 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:22:14.698966 (+  4364us) service_pool.cc:224] Handling call
0709 13:22:14.699350 (+   384us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:22:14.971543 (+272193us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:22:14.971575 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:22:14.972107 (+   532us) log.cc:844] Serialized 26 byte log entry
0709 13:22:15.264151 (+292044us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:22:15.332526 (+ 68375us) log.cc:844] Serialized 14 byte log entry
0709 13:22:15.372010 (+ 39484us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:22:15.372120 (+   110us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:22:16.032709 (+660589us) spinlock_profiling.cc:243] Waited 68.8 ms on lock 0x7b1800245d40. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b693ade1 00007f69b69475b1 00007f69ada72522 00007f69ada747c1 00007f69ada74779 00007f69ada74709 00007f69ada746d1 00007f69ada739cd 000000000038f354 000000000038f199 00007f69ada5583a
0709 13:22:16.905314 (+872605us) spinlock_profiling.cc:243] Waited 54.9 ms on lock 0x7b4400022b4c. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69ad8fadb1 00007f69ad998c85 00007f69ad9996a2 00007f69b070b1db 00007f69b0706aaf 00007f69b06fa86e 00007f69b06f9bb6 00007f69b06f97d8 00007f69b0703221 00007f69b07031d9 00007f69b0703169
0709 13:22:16.948269 (+ 42955us) raft_consensus.cc:1685] finished
0709 13:22:16.948292 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:22:16.961572 (+ 13280us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":144792,"prepare.queue_time_us":70859,"prepare.run_cpu_time_us":36,"prepare.run_wall_time_us":38,"raft.queue_time_us":241962,"raft.run_cpu_time_us":529,"raft.run_wall_time_us":31485,"spinlock_wait_cycles":284572032,"thread_start_us":361910,"threads_started":2,"wal-append.queue_time_us":470225}
I20260709 13:22:16.993192 16297 raft_consensus.cc:1275] T 8158bbd17b994d2eb96f560bd0f1b88d P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 13:22:16.993997 16145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5718c623ebee4d2bb770f0f9ddb2f164" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 21 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:22:16.996380 23321 consensus_queue.cc:237] T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:16.999017 23557 leader_election.cc:290] T cdbd78acb4cf46298e0de8ba0d130e19 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:16.999698 23287 leader_election.cc:290] T ecc1e363230043fe9fef390f0b979f20 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:17.000012 16142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806d7690c2b644959152074bb2cab0a7" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:22:17.000895 23625 leader_election.cc:290] T 8c11074579b046d99e585c2818e18a2c P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
W20260709 13:22:17.002596 23584 log.cc:927] Time spent T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 2.199s	user 0.004s	sys 0.000s
I20260709 13:22:17.003419 16150 raft_consensus.cc:2468] T 2794af5b99984efebdee564035841cb0 P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 18.
I20260709 13:22:17.004573 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66f75124215f4ca6bef831e174a792d8" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 20 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:22:17.007277 23210 raft_consensus.cc:493] T a378d7f796f94615b3b347119a9880fd P 90f3da57796b488c9087a2f523adf045 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:22:17.007969 23306 raft_consensus.cc:493] T 57a744f5aaf944678a63469cbf514015 P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:17.008390 16265 leader_election.cc:304] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:22:17.008667 23306 raft_consensus.cc:515] T 57a744f5aaf944678a63469cbf514015 P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:17.009593 16297 raft_consensus.cc:1240] T 68a6838d82ef4cac83507f906139d90f P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 19. Current term is 20. Ops: []
I20260709 13:22:17.011108 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f2da98864b4425bf7c0055951fdd4b" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:17.011924 16297 raft_consensus.cc:2468] T 84f2da98864b4425bf7c0055951fdd4b P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 17.
I20260709 13:22:17.008004 23210 raft_consensus.cc:515] T a378d7f796f94615b3b347119a9880fd P 90f3da57796b488c9087a2f523adf045 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:22:17.012784 16789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 64043.4us
I20260709 13:22:16.876802 15046 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:22:17.014209 16151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746aabf856ab401c9ef5e859371e7b67" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:22:17.013487 16040 catalog_manager.cc:5697] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 reported cstate change: term changed from 13 to 17, leader changed from cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) to 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130). New cstate: current_term: 17 leader_uuid: "1ca2725d290e47d68b1a4b609c9816ac" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:22:17.018339 16294 raft_consensus.cc:3060] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:22:16.985778 23437 raft_consensus.cc:2749] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:22:17.015728 23409 consensus_queue.cc:1059] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: INVALID_TERM, Last received: 23.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 13:22:17.016026 16151 raft_consensus.cc:1240] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 647a343a542d4a139fe106c19095129b [term 21 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 20. Current term is 21. Ops: []
I20260709 13:22:17.009902 16265 leader_election.cc:304] T c4f65cdf00b748bb963ee6e81ef43b79 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:22:17.016883 23210 leader_election.cc:290] T a378d7f796f94615b3b347119a9880fd P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:17.008579 16142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746aabf856ab401c9ef5e859371e7b67" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:22:17.027720 16150 raft_consensus.cc:2410] T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 for term 20 because replica has last-logged OpId of term: 18 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 13.
I20260709 13:22:17.027132 16145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f2da98864b4425bf7c0055951fdd4b" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:22:16.981923 23566 leader_election.cc:290] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 22 election: Requested vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:17.036069 23409 raft_consensus.cc:3055] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 13:22:17.036571 23409 raft_consensus.cc:740] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:22:17.037602 23409 consensus_queue.cc:260] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 23.16, Last appended by leader: 16, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:17.032188 23625 raft_consensus.cc:2749] T c4f65cdf00b748bb963ee6e81ef43b79 P 90f3da57796b488c9087a2f523adf045 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:22:17.041807 16192 leader_election.cc:304] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
W20260709 13:22:17.041556 16147 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40830 (request call id 2161) took 2181 ms (client timeout 2692 ms). Trace:
I20260709 13:22:17.047307 16148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2372981006d4b218215c28964dea623" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
W20260709 13:22:17.045958 16147 rpcz_store.cc:269] 0709 13:22:14.860376 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:22:14.978152 (+117776us) service_pool.cc:224] Handling call
0709 13:22:17.039585 (+2061433us) spinlock_profiling.cc:243] Waited 1.56 s on lock 0x7b540041e6d4. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:22:17.041471 (+  1886us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3579103616}
I20260709 13:22:17.049409 16297 raft_consensus.cc:1240] T 8cc959fb7f704fba9009c0595467d074 P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 18. Current term is 19. Ops: []
I20260709 13:22:17.050312 16147 raft_consensus.cc:1275] T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 13:22:17.051617 16789 leader_election.cc:304] T 8638ee363273426db1393dd81e5c7b72 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:22:17.051998 16151 raft_consensus.cc:1240] T 68a6838d82ef4cac83507f906139d90f P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 19. Current term is 20. Ops: []
I20260709 13:22:17.053802 16789 leader_election.cc:304] T c9944d5e4b8c4109af9955c4f08c9095 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
W20260709 13:22:17.055727 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 45957.5us
I20260709 13:22:17.058982 23553 consensus_queue.cc:1048] T d2ab0ac8347641e6955e9bce7d90c1a2 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:22:17.059664 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74ca8140f35c4ae29e8a664f6bdfc829" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:22:17.059811 16191 leader_election.cc:304] T d16dfad0c3b540e5b4b86492ddf87208 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:22:17.060367 16145 raft_consensus.cc:1275] T 08c74490c40243a88e62d1760b5cd814 P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 13:22:17.060698 23539 consensus_queue.cc:1059] T b87147731a68400e8cce9b879daa7c5d P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: INVALID_TERM, Last received: 20.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:22:17.062165 23539 raft_consensus.cc:2749] T 8638ee363273426db1393dd81e5c7b72 P cb82cc7bd6db46d980c75237a1402860 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:22:17.062336 16790 leader_election.cc:304] T 829b4ad8c5a54eababeea1181b960d4b P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:22:17.063350 23650 raft_consensus.cc:3055] T b87147731a68400e8cce9b879daa7c5d P cb82cc7bd6db46d980c75237a1402860 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:22:17.063820 23650 raft_consensus.cc:740] T b87147731a68400e8cce9b879daa7c5d P cb82cc7bd6db46d980c75237a1402860 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:22:17.039065 23409 raft_consensus.cc:3060] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:22:17.063064 23539 raft_consensus.cc:2749] T c9944d5e4b8c4109af9955c4f08c9095 P cb82cc7bd6db46d980c75237a1402860 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:22:17.063149 16145 raft_consensus.cc:1240] T b87147731a68400e8cce9b879daa7c5d P 647a343a542d4a139fe106c19095129b [term 21 FOLLOWER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 20. Current term is 21. Ops: []
I20260709 13:22:17.070537 22610 raft_consensus.cc:515] T 8d4ab870a05347868956aa8b841b2981 P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:17.073201 16148 raft_consensus.cc:2468] T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 24.
W20260709 13:22:17.073832 16192 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 44437.9us
I20260709 13:22:17.074623 16148 raft_consensus.cc:1240] T 434bff3082d141838068f323bdb0d1ed P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 19. Current term is 20. Ops: []
I20260709 13:22:17.074707 16192 leader_election.cc:304] T 829b4ad8c5a54eababeea1181b960d4b P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 647a343a542d4a139fe106c19095129b
I20260709 13:22:17.075836 23614 raft_consensus.cc:2749] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 13:22:17.076787 23286 raft_consensus.cc:493] T 434bff3082d141838068f323bdb0d1ed P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:17.077316 23286 raft_consensus.cc:515] T 434bff3082d141838068f323bdb0d1ed P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:17.077426 23588 consensus_queue.cc:1048] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 13:22:17.078464 16192 leader_election.cc:304] T ee9ed67006144797b1de6ea8c2013a68 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
I20260709 13:22:17.067363 23394 raft_consensus.cc:2749] T 829b4ad8c5a54eababeea1181b960d4b P cb82cc7bd6db46d980c75237a1402860 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:22:17.079411 23566 raft_consensus.cc:2804] T 829b4ad8c5a54eababeea1181b960d4b P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:22:17.070752 23287 raft_consensus.cc:2749] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:22:17.079905 23566 raft_consensus.cc:493] T 829b4ad8c5a54eababeea1181b960d4b P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Starting leader election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:22:17.080240 23556 raft_consensus.cc:2749] T ee9ed67006144797b1de6ea8c2013a68 P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:22:17.080868 23650 consensus_queue.cc:260] T b87147731a68400e8cce9b879daa7c5d P cb82cc7bd6db46d980c75237a1402860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.13, Last appended by leader: 13, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:17.081902 23286 leader_election.cc:290] T 434bff3082d141838068f323bdb0d1ed P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:17.082558 23650 raft_consensus.cc:3060] T b87147731a68400e8cce9b879daa7c5d P cb82cc7bd6db46d980c75237a1402860 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:22:17.083570 16148 raft_consensus.cc:1240] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b [term 24 FOLLOWER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 23. Current term is 24. Ops: []
I20260709 13:22:17.085229 16148 raft_consensus.cc:1275] T 02838c86531d4f98bf58d1c22b8baf77 P 647a343a542d4a139fe106c19095129b [term 21 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260709 13:22:17.085546 16149 raft_consensus.cc:1275] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [term 23 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 21 index: 12. Preceding OpId from leader: term: 23 index: 13. (index mismatch)
I20260709 13:22:17.089465 16299 raft_consensus.cc:1275] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 90f3da57796b488c9087a2f523adf045 [term 21 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 21 index: 16. (index mismatch)
I20260709 13:22:17.089701 16790 leader_election.cc:304] T 66f75124215f4ca6bef831e174a792d8 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:22:17.090713 23491 raft_consensus.cc:2749] T 66f75124215f4ca6bef831e174a792d8 P cb82cc7bd6db46d980c75237a1402860 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:22:17.080418 23566 raft_consensus.cc:3060] T 829b4ad8c5a54eababeea1181b960d4b P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:22:17.091081 23606 consensus_queue.cc:1048] T 8158bbd17b994d2eb96f560bd0f1b88d P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:22:17.099954 22610 leader_election.cc:290] T 8d4ab870a05347868956aa8b841b2981 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 20 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:17.099509 16300 raft_consensus.cc:2468] T ada5a004ba7c46819051e3ccec71ffdb P 90f3da57796b488c9087a2f523adf045 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 22.
I20260709 13:22:17.103470 16147 raft_consensus.cc:1240] T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b [term 18 LEADER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 17. Current term is 18. Ops: []
I20260709 13:22:17.043586 23306 leader_election.cc:290] T 57a744f5aaf944678a63469cbf514015 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:17.108693 16149 raft_consensus.cc:1275] T 26354361e9b044cea1c2b86c4882fe92 P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 13:22:17.112850 16296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d4ab870a05347868956aa8b841b2981" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:22:17.113708 16296 raft_consensus.cc:3060] T 8d4ab870a05347868956aa8b841b2981 P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:22:17.112864 23384 raft_consensus.cc:2749] T d16dfad0c3b540e5b4b86492ddf87208 P 1ca2725d290e47d68b1a4b609c9816ac [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:22:17.116895 16145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f2da98864b4425bf7c0055951fdd4b" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:22:17.118361 16294 raft_consensus.cc:2468] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 22.
W20260709 13:22:17.111358 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 51670.3us
I20260709 13:22:17.120011 16191 leader_election.cc:304] T 8a878999c0024d2abe517484180290fc P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:22:17.125715 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68a6838d82ef4cac83507f906139d90f" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:22:17.125921 23306 consensus_queue.cc:1048] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:22:17.131052 23565 raft_consensus.cc:3055] T 68a6838d82ef4cac83507f906139d90f P 1ca2725d290e47d68b1a4b609c9816ac [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:22:17.131584 23565 raft_consensus.cc:740] T 68a6838d82ef4cac83507f906139d90f P 1ca2725d290e47d68b1a4b609c9816ac [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:22:17.132633 23565 consensus_queue.cc:260] T 68a6838d82ef4cac83507f906139d90f P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:17.133680 23588 raft_consensus.cc:2749] T 8a878999c0024d2abe517484180290fc P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:22:17.134671 23588 consensus_queue.cc:1059] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: INVALID_TERM, Last received: 20.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:22:17.134761 23614 consensus_queue.cc:1048] T 08c74490c40243a88e62d1760b5cd814 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:22:17.136402 23588 raft_consensus.cc:3055] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 1ca2725d290e47d68b1a4b609c9816ac [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:22:17.136875 23588 raft_consensus.cc:740] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 1ca2725d290e47d68b1a4b609c9816ac [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:22:17.138017 23588 consensus_queue.cc:260] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.13, Last appended by leader: 13, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:17.141717 16298 raft_consensus.cc:1275] T 84f2da98864b4425bf7c0055951fdd4b P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 13:22:17.134029 23565 raft_consensus.cc:3060] T 68a6838d82ef4cac83507f906139d90f P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:22:17.134735 16265 leader_election.cc:400] T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 21 election: Vote denied by peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943) with higher term. Message: Invalid argument: T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 for earlier term 21. Current term is 22.
I20260709 13:22:17.144161 16265 leader_election.cc:403] T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 21 election: Cancelling election due to peer responding with higher term
I20260709 13:22:17.144079 22677 raft_consensus.cc:515] T c0eab25ce4c64e018a041beeda2867db P 1ca2725d290e47d68b1a4b609c9816ac [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:17.146121 23625 raft_consensus.cc:2749] T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045 [term 22 FOLLOWER]: Leader election lost for term 21. Reason: Vote denied by peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943) with higher term. Message: Invalid argument: T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 for earlier term 21. Current term is 22.
I20260709 13:22:17.146256 16144 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46224 (request call id 1249) took 2428 ms. Trace:
I20260709 13:22:17.147325 16291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434bff3082d141838068f323bdb0d1ed" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 21 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
W20260709 13:22:17.147866 16143 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40830 (request call id 2159) took 2287 ms (client timeout 1967 ms). Trace:
W20260709 13:22:17.148289 16143 rpcz_store.cc:269] 0709 13:22:14.859817 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:22:14.933017 (+ 73200us) service_pool.cc:224] Handling call
0709 13:22:17.147440 (+2214423us) spinlock_profiling.cc:243] Waited 539 ms on lock 0x7b540011b2d4. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:22:17.147659 (+   219us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1240222976}
I20260709 13:22:17.148365 23210 consensus_queue.cc:1048] T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:22:17.150221 16143 raft_consensus.cc:1275] T b6f07842d3aa47858df5665874c86f3a P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 13:22:17.150836 16265 leader_election.cc:304] T d16dfad0c3b540e5b4b86492ddf87208 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 647a343a542d4a139fe106c19095129b
I20260709 13:22:17.151057 23394 consensus_queue.cc:1059] T 434bff3082d141838068f323bdb0d1ed P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: INVALID_TERM, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 13:22:17.152318 23533 raft_consensus.cc:2804] T d16dfad0c3b540e5b4b86492ddf87208 P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:22:17.152756 23533 raft_consensus.cc:493] T d16dfad0c3b540e5b4b86492ddf87208 P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:22:17.152827 23394 raft_consensus.cc:3055] T 434bff3082d141838068f323bdb0d1ed P cb82cc7bd6db46d980c75237a1402860 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:22:17.153160 23533 raft_consensus.cc:3060] T d16dfad0c3b540e5b4b86492ddf87208 P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:22:17.153316 23394 raft_consensus.cc:740] T 434bff3082d141838068f323bdb0d1ed P cb82cc7bd6db46d980c75237a1402860 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
W20260709 13:22:17.156386 16789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 102759us
I20260709 13:22:17.158344 16789 leader_election.cc:304] T 23f82224f4564fea93572e8206bad1fe P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 647a343a542d4a139fe106c19095129b
I20260709 13:22:17.159525 23648 raft_consensus.cc:2804] T 23f82224f4564fea93572e8206bad1fe P cb82cc7bd6db46d980c75237a1402860 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:22:17.159917 23648 raft_consensus.cc:493] T 23f82224f4564fea93572e8206bad1fe P cb82cc7bd6db46d980c75237a1402860 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:22:17.160311 23648 raft_consensus.cc:3060] T 23f82224f4564fea93572e8206bad1fe P cb82cc7bd6db46d980c75237a1402860 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:22:17.161425 16192 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 83044.9us
I20260709 13:22:17.162374 16192 leader_election.cc:304] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860
I20260709 13:22:17.163362 23384 raft_consensus.cc:2749] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:22:17.164712 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57a744f5aaf944678a63469cbf514015" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:22:17.165647 16150 raft_consensus.cc:2468] T 57a744f5aaf944678a63469cbf514015 P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 18.
I20260709 13:22:17.167009 16146 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40844 (request call id 3048) took 2471 ms. Trace:
I20260709 13:22:17.167471 16146 rpcz_store.cc:276] 0709 13:22:14.700148 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:22:15.020096 (+319948us) service_pool.cc:224] Handling call
0709 13:22:15.020550 (+   454us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:22:16.909589 (+1889039us) spinlock_profiling.cc:243] Waited 1.46 ms on lock 0x7b180004b520. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b693ade1 00007f69b69475b1 00007f69b07074bc 00007f69ada5548b 00007f69b68afab9 00007f69ada7169b 00007f69ada6f93f 00007f69ada6f455 00007f69b0b57060 00007f69b0b57bfb 00007f69b0b6264b
0709 13:22:16.909839 (+   250us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:22:16.909871 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:22:16.910531 (+   660us) log.cc:844] Serialized 26 byte log entry
0709 13:22:16.942276 (+ 31745us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:22:16.942319 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:22:16.956926 (+ 14607us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:22:17.166572 (+209646us) raft_consensus.cc:1685] finished
0709 13:22:17.166590 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:22:17.166826 (+   236us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":44654,"raft.queue_time_us":348715,"raft.run_cpu_time_us":155,"raft.run_wall_time_us":32283,"spinlock_wait_cycles":3358976,"thread_start_us":400847,"threads_started":2,"wal-append.queue_time_us":131507}
I20260709 13:22:17.169060 16192 leader_election.cc:304] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
I20260709 13:22:17.167466 23384 consensus_queue.cc:1048] T 66f75124215f4ca6bef831e174a792d8 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.003s
I20260709 13:22:17.169255 16040 catalog_manager.cc:5697] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 reported cstate change: term changed from 12 to 18, leader changed from 647a343a542d4a139fe106c19095129b (127.14.177.129) to cb82cc7bd6db46d980c75237a1402860 (127.14.177.132). New cstate: current_term: 18 leader_uuid: "cb82cc7bd6db46d980c75237a1402860" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:22:17.190730 16114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:22:17.191308 16143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 24 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:22:17.154691 23394 consensus_queue.cc:260] T 434bff3082d141838068f323bdb0d1ed P cb82cc7bd6db46d980c75237a1402860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.16, Last appended by leader: 16, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:17.193316 23394 raft_consensus.cc:3060] T 434bff3082d141838068f323bdb0d1ed P cb82cc7bd6db46d980c75237a1402860 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:22:17.146664 16144 rpcz_store.cc:276] 0709 13:22:14.718119 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:22:15.337662 (+619543us) service_pool.cc:224] Handling call
0709 13:22:15.337969 (+   307us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:22:15.374934 (+ 36965us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:22:15.374979 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:22:15.375586 (+   607us) log.cc:844] Serialized 26 byte log entry
0709 13:22:17.145256 (+1769670us) spinlock_profiling.cc:243] Waited 8.29 ms on lock 0x7b1800491e00. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b693ade1 00007f69b69475b1 00007f69b07074bc 00007f69ada5548b 00007f69b68afab9 00007f69ada7169b 00007f69ada6f93f 00007f69ada71c02 00007f69b06f920b 00007f69b06fed23 00007f69b06ff0d1
0709 13:22:17.145619 (+   363us) raft_consensus.cc:1645] Marking committed up to 15
0709 13:22:17.145656 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:22:17.145787 (+   131us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:22:17.145834 (+    47us) raft_consensus.cc:1685] finished
0709 13:22:17.145844 (+    10us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:22:17.146087 (+   243us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":36542,"raft.queue_time_us":192109,"raft.run_cpu_time_us":186,"raft.run_wall_time_us":54690,"spinlock_wait_cycles":19056384,"thread_start_us":215476,"threads_started":1,"wal-append.queue_time_us":428478,"wal-append.run_cpu_time_us":1797,"wal-append.run_wall_time_us":1005220}
I20260709 13:22:17.195231 16292 raft_consensus.cc:2468] T e5ad4eb281564936a59f41e387a47156 P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 20.
I20260709 13:22:17.196437 16191 leader_election.cc:304] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: cb82cc7bd6db46d980c75237a1402860
I20260709 13:22:17.196456 23556 consensus_queue.cc:1048] T 66f75124215f4ca6bef831e174a792d8 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:22:17.199407 23373 raft_consensus.cc:2804] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:22:17.199838 23373 raft_consensus.cc:493] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:17.199913 16192 leader_election.cc:400] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 23 pre-election: Vote denied by peer cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923) with higher term. Message: Invalid argument: T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac for earlier term 23. Current term is 24.
I20260709 13:22:17.200225 23373 raft_consensus.cc:3060] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:22:17.200358 16192 leader_election.cc:403] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 23 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:22:17.180819 22677 leader_election.cc:290] T c0eab25ce4c64e018a041beeda2867db P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 22 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:17.201498 22677 consensus_queue.cc:1048] T 02838c86531d4f98bf58d1c22b8baf77 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:22:17.203188 23588 raft_consensus.cc:3060] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 1ca2725d290e47d68b1a4b609c9816ac [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:22:17.204805 16819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74ca8140f35c4ae29e8a664f6bdfc829" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:22:17.205937 16819 raft_consensus.cc:2393] T 74ca8140f35c4ae29e8a664f6bdfc829 P cb82cc7bd6db46d980c75237a1402860 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 22: Already voted for candidate cb82cc7bd6db46d980c75237a1402860 in this term.
I20260709 13:22:17.207321 23409 raft_consensus.cc:493] T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:22:17.207796 16817 raft_consensus.cc:1275] T 3a7ca61592e848ea9b0fb81a2ce33cdc P cb82cc7bd6db46d980c75237a1402860 [term 21 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 21 index: 16. (index mismatch)
I20260709 13:22:17.208009 23409 raft_consensus.cc:515] T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:22:17.185578 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 106231us
I20260709 13:22:17.211341 16790 leader_election.cc:304] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 647a343a542d4a139fe106c19095129b
I20260709 13:22:17.217927 16265 leader_election.cc:304] T 8cc959fb7f704fba9009c0595467d074 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
W20260709 13:22:17.220911 16191 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:22:17.225036 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5d59407eb545b895d8d922e0dd69fd" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 24 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:17.222004 16265 leader_election.cc:304] T 9e65676b21b748728ad39668cfe722ec P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:22:17.238401 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806d7690c2b644959152074bb2cab0a7" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:17.143752 23603 consensus_queue.cc:1048] T 84f2da98864b4425bf7c0055951fdd4b P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.070s
I20260709 13:22:17.242821 16291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0eab25ce4c64e018a041beeda2867db" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:22:17.219660 23539 raft_consensus.cc:2804] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:22:17.200387 23551 raft_consensus.cc:2749] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:22:17.219218 23559 raft_consensus.cc:2749] T 8cc959fb7f704fba9009c0595467d074 P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:22:17.247977 23554 raft_consensus.cc:2749] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [term 24 FOLLOWER]: Leader pre-election lost for term 23. Reason: Vote denied by peer cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923) with higher term. Message: Invalid argument: T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac for earlier term 23. Current term is 24.
I20260709 13:22:17.248065 16291 raft_consensus.cc:3060] T c0eab25ce4c64e018a041beeda2867db P 90f3da57796b488c9087a2f523adf045 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:22:17.252137 16265 leader_election.cc:304] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:22:17.257130 16790 leader_election.cc:304] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:22:17.258950 16220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18ad7e3af5504571a69d10c00750a01c" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:17.241403 23306 raft_consensus.cc:2804] T 9e65676b21b748728ad39668cfe722ec P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:22:17.259682 23648 raft_consensus.cc:515] T 23f82224f4564fea93572e8206bad1fe P cb82cc7bd6db46d980c75237a1402860 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:17.269214 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2372981006d4b218215c28964dea623" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:22:17.216804 23373 raft_consensus.cc:515] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:17.271144 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66ae176961b74813a58bc15c9c4c6f2a" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 27 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:17.272881 23598 raft_consensus.cc:1027] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac [term 20 LEADER]: attempt to promote peer cb82cc7bd6db46d980c75237a1402860: there is already a config change operation in progress. Unable to promote follower until it completes. Doing nothing.
I20260709 13:22:17.273067 16265 leader_election.cc:304] T 2794af5b99984efebdee564035841cb0 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:22:17.274241 23620 raft_consensus.cc:2804] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 90f3da57796b488c9087a2f523adf045 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:22:17.274875 23620 raft_consensus.cc:697] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 90f3da57796b488c9087a2f523adf045 [term 21 LEADER]: Becoming Leader. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:22:17.276093 23620 consensus_queue.cc:237] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 90f3da57796b488c9087a2f523adf045 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.13, Last appended by leader: 13, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:17.256165 16293 raft_consensus.cc:2468] T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 22.
I20260709 13:22:17.283964 16220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c61a39fd4f84424b129c1a6f9ba404d" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 22 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:17.284884 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f2da98864b4425bf7c0055951fdd4b" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:17.286382 16223 raft_consensus.cc:1240] T c2f5f89186204242bf377f7c42b2e2da P 1ca2725d290e47d68b1a4b609c9816ac [term 23 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 22. Current term is 23. Ops: [22.13-22.13]
I20260709 13:22:17.281775 16221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87ac0d1f7e8a419f9bcbc04aee0e067e" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:22:17.289986 16190 leader_election.cc:304] T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 647a343a542d4a139fe106c19095129b
I20260709 13:22:17.291639 16296 raft_consensus.cc:2468] T 8d4ab870a05347868956aa8b841b2981 P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 20.
I20260709 13:22:17.268229 23306 raft_consensus.cc:493] T 9e65676b21b748728ad39668cfe722ec P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:17.285755 16217 raft_consensus.cc:2468] T 84f2da98864b4425bf7c0055951fdd4b P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 17.
I20260709 13:22:17.269438 23605 raft_consensus.cc:493] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:22:17.301102 16219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdbd78acb4cf46298e0de8ba0d130e19" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:17.305552 16218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c11074579b046d99e585c2818e18a2c" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:17.306434 16116 leader_election.cc:400] T 7c2095a5f4ad49408dbc7f69c606120d P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 17 election: Vote denied by peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943) with higher term. Message: Invalid argument: T 7c2095a5f4ad49408dbc7f69c606120d P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b for earlier term 17. Current term is 18.
I20260709 13:22:17.307456 16222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d4ab870a05347868956aa8b841b2981" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
W20260709 13:22:17.311368 16117 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:22:17.314193 16117 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:22:17.311239 23306 raft_consensus.cc:3060] T 9e65676b21b748728ad39668cfe722ec P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:22:17.323558 16226 raft_consensus.cc:2368] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b for earlier term 24. Current term is 25.
I20260709 13:22:17.321695 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d4ab870a05347868956aa8b841b2981" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:17.322254 16219 raft_consensus.cc:2393] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 24: Already voted for candidate cb82cc7bd6db46d980c75237a1402860 in this term.
I20260709 13:22:17.328055 16222 raft_consensus.cc:3060] T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:22:17.331724 16265 leader_election.cc:304] T 8d4ab870a05347868956aa8b841b2981 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac
I20260709 13:22:17.283393 23024 raft_consensus.cc:2804] T 2794af5b99984efebdee564035841cb0 P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:22:17.339671 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 24 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:17.347476 16220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f5f89186204242bf377f7c42b2e2da" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 23 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:22:17.350829 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a378d7f796f94615b3b347119a9880fd" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:17.366945 23685 raft_consensus.cc:2804] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 [term 24 FOLLOWER]: Leader election won for term 24
W20260709 13:22:17.372828 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 115737us
I20260709 13:22:17.335721 23024 raft_consensus.cc:493] T 2794af5b99984efebdee564035841cb0 P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:17.380754 23539 raft_consensus.cc:697] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860 [term 18 LEADER]: Becoming Leader. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:22:17.323470 23533 raft_consensus.cc:515] T d16dfad0c3b540e5b4b86492ddf87208 P 90f3da57796b488c9087a2f523adf045 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:17.379855 16224 raft_consensus.cc:1275] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 13:22:17.345798 23559 raft_consensus.cc:2764] T 8d4ab870a05347868956aa8b841b2981 P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Leader pre-election decision vote started in defunct term 19: won
I20260709 13:22:17.383968 16220 raft_consensus.cc:2376] T c2f5f89186204242bf377f7c42b2e2da P 1ca2725d290e47d68b1a4b609c9816ac [term 23 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 23. Re-sending same reply.
I20260709 13:22:17.389991 16217 raft_consensus.cc:2410] T ecc1e363230043fe9fef390f0b979f20 P 1ca2725d290e47d68b1a4b609c9816ac [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 for term 24 because replica has last-logged OpId of term: 21 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 12.
I20260709 13:22:17.334820 23605 raft_consensus.cc:515] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:17.399814 16291 raft_consensus.cc:2468] T c0eab25ce4c64e018a041beeda2867db P 90f3da57796b488c9087a2f523adf045 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 22.
I20260709 13:22:17.383894 23024 raft_consensus.cc:3060] T 2794af5b99984efebdee564035841cb0 P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:22:17.412432 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 157619us
I20260709 13:22:17.417860 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada5a004ba7c46819051e3ccec71ffdb" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:22:17.408041 23409 leader_election.cc:290] T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:22:17.424299 16224 raft_consensus.cc:2393] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 in current term 22: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
I20260709 13:22:17.425012 23680 consensus_queue.cc:1048] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.039s
I20260709 13:22:17.425323 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5d59407eb545b895d8d922e0dd69fd" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
W20260709 13:22:17.436327 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 115796us
I20260709 13:22:17.444067 23373 leader_election.cc:290] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 25 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:17.445585 23605 leader_election.cc:290] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:17.449465 23559 consensus_queue.cc:1048] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:22:17.454622 16148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f82224f4564fea93572e8206bad1fe" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:22:17.455297 23558 leader_election.cc:290] T 8158bbd17b994d2eb96f560bd0f1b88d P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 19 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:22:17.381912 23539 consensus_queue.cc:237] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:17.464596 23648 leader_election.cc:290] T 23f82224f4564fea93572e8206bad1fe P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 18 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:17.468663 23566 raft_consensus.cc:515] T 829b4ad8c5a54eababeea1181b960d4b P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:17.476701 23533 leader_election.cc:290] T d16dfad0c3b540e5b4b86492ddf87208 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 21 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:22:17.493564 16222 raft_consensus.cc:2468] T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 20.
W20260709 13:22:17.518744 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 228996us
I20260709 13:22:17.496575 23478 raft_consensus.cc:2764] T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac [term 20 FOLLOWER]: Leader pre-election decision vote started in defunct term 19: won
I20260709 13:22:17.543002 16221 raft_consensus.cc:2468] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 21.
I20260709 13:22:17.542378 16265 leader_election.cc:304] T 57a744f5aaf944678a63469cbf514015 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:22:17.554484 16190 leader_election.cc:304] T 84f2da98864b4425bf7c0055951fdd4b P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 647a343a542d4a139fe106c19095129b
W20260709 13:22:17.557216 16115 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:22:17.585850 16147 raft_consensus.cc:1275] T 57a744f5aaf944678a63469cbf514015 P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 13:22:17.599125 16144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1df93779209748b9860f6d893dd3de53" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:22:17.601950 16190 leader_election.cc:304] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: cb82cc7bd6db46d980c75237a1402860
I20260709 13:22:17.601843 23681 consensus_queue.cc:1059] T 8cc959fb7f704fba9009c0595467d074 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: INVALID_TERM, Last received: 18.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:22:17.603039 23588 consensus_queue.cc:1048] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 13:22:17.603436 16145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d16dfad0c3b540e5b4b86492ddf87208" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:22:17.589628 23023 raft_consensus.cc:493] T 08c74490c40243a88e62d1760b5cd814 P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:17.604254 23688 raft_consensus.cc:2804] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:22:17.604985 23681 raft_consensus.cc:3055] T 8cc959fb7f704fba9009c0595467d074 P 1ca2725d290e47d68b1a4b609c9816ac [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:22:17.604857 23023 raft_consensus.cc:515] T 08c74490c40243a88e62d1760b5cd814 P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:17.605127 23682 consensus_queue.cc:1048] T 02838c86531d4f98bf58d1c22b8baf77 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.065s
I20260709 13:22:17.604995 23688 raft_consensus.cc:697] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac [term 22 LEADER]: Becoming Leader. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:22:17.605410 23681 raft_consensus.cc:740] T 8cc959fb7f704fba9009c0595467d074 P 1ca2725d290e47d68b1a4b609c9816ac [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:22:17.607368 23287 raft_consensus.cc:493] T e4dc6b0541ca431ebc757beac00521f1 P 90f3da57796b488c9087a2f523adf045 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:17.607942 23287 raft_consensus.cc:515] T e4dc6b0541ca431ebc757beac00521f1 P 90f3da57796b488c9087a2f523adf045 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:17.608016 23681 consensus_queue.cc:260] T 8cc959fb7f704fba9009c0595467d074 P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:17.609499 23681 raft_consensus.cc:3060] T 8cc959fb7f704fba9009c0595467d074 P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:22:17.610390 23287 leader_election.cc:290] T e4dc6b0541ca431ebc757beac00521f1 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:22:17.606863 23688 consensus_queue.cc:237] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 15, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:17.586269 23306 raft_consensus.cc:515] T 9e65676b21b748728ad39668cfe722ec P 90f3da57796b488c9087a2f523adf045 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:22:17.612601 23673 log.cc:927] Time spent T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 0.339s	user 0.001s	sys 0.000s
I20260709 13:22:17.613371 23306 leader_election.cc:290] T 9e65676b21b748728ad39668cfe722ec P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 21 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:17.616192 23614 raft_consensus.cc:2804] T 84f2da98864b4425bf7c0055951fdd4b P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:22:17.616653 23614 raft_consensus.cc:493] T 84f2da98864b4425bf7c0055951fdd4b P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:17.617012 23614 raft_consensus.cc:3060] T 84f2da98864b4425bf7c0055951fdd4b P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:22:17.553248 23603 raft_consensus.cc:493] T 02838c86531d4f98bf58d1c22b8baf77 P 647a343a542d4a139fe106c19095129b [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:17.623063 23603 raft_consensus.cc:515] T 02838c86531d4f98bf58d1c22b8baf77 P 647a343a542d4a139fe106c19095129b [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:17.623543 16219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d16dfad0c3b540e5b4b86492ddf87208" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:22:17.624446 16219 raft_consensus.cc:3060] T d16dfad0c3b540e5b4b86492ddf87208 P 1ca2725d290e47d68b1a4b609c9816ac [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:22:17.627589 16222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8158bbd17b994d2eb96f560bd0f1b88d" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:22:17.628517 16222 raft_consensus.cc:2393] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 19: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
I20260709 13:22:17.631053 16116 leader_election.cc:304] T 68a6838d82ef4cac83507f906139d90f P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:22:17.631491 22610 consensus_queue.cc:1048] T 84f2da98864b4425bf7c0055951fdd4b P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:22:17.632740 16116 leader_election.cc:304] T 9c61a39fd4f84424b129c1a6f9ba404d P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:22:17.633366 22610 raft_consensus.cc:2749] T 68a6838d82ef4cac83507f906139d90f P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:22:17.634382 22610 raft_consensus.cc:2749] T 9c61a39fd4f84424b129c1a6f9ba404d P 647a343a542d4a139fe106c19095129b [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:22:17.640143 16148 raft_consensus.cc:1275] T 434bff3082d141838068f323bdb0d1ed P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 18 index: 15. Preceding OpId from leader: term: 20 index: 16. (index mismatch)
W20260709 13:22:17.646309 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 50978.6us
W20260709 13:22:17.646320 16266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 52415.1us
I20260709 13:22:17.647670 16142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0eab25ce4c64e018a041beeda2867db" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:22:17.648524 16142 raft_consensus.cc:3060] T c0eab25ce4c64e018a041beeda2867db P 647a343a542d4a139fe106c19095129b [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:22:17.649420 16266 leader_election.cc:304] T 8638ee363273426db1393dd81e5c7b72 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860
I20260709 13:22:17.651328 23625 raft_consensus.cc:493] T 68a6838d82ef4cac83507f906139d90f P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:17.651882 23210 raft_consensus.cc:2749] T 8638ee363273426db1393dd81e5c7b72 P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:22:17.653759 16151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5d59407eb545b895d8d922e0dd69fd" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
W20260709 13:22:17.657796 23675 log.cc:927] Time spent T fffab26427e24d398e2695aad40ca134 P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 0.269s	user 0.001s	sys 0.000s
I20260709 13:22:17.653964 16117 leader_election.cc:304] T 3884f864a96d428785895e5463e5728f P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860
I20260709 13:22:17.675282 16265 leader_election.cc:304] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:22:17.668771 22610 consensus_queue.cc:1048] T 8158bbd17b994d2eb96f560bd0f1b88d P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.008s
I20260709 13:22:17.669008 16116 leader_election.cc:304] T 8d4ab870a05347868956aa8b841b2981 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
I20260709 13:22:17.664860 16266 leader_election.cc:304] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860
W20260709 13:22:17.664659 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 62790.2us
I20260709 13:22:17.539289 23685 raft_consensus.cc:697] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 [term 24 LEADER]: Becoming Leader. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:22:17.677425 16790 leader_election.cc:304] T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:22:17.607025 23023 leader_election.cc:290] T 08c74490c40243a88e62d1760b5cd814 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:17.551515 23558 raft_consensus.cc:2804] T 57a744f5aaf944678a63469cbf514015 P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:22:17.593400 23566 leader_election.cc:290] T 829b4ad8c5a54eababeea1181b960d4b P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 19 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:17.679932 16190 leader_election.cc:304] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: cb82cc7bd6db46d980c75237a1402860
I20260709 13:22:17.690410 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2372981006d4b218215c28964dea623" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:22:17.731545 16150 raft_consensus.cc:3060] T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:22:17.691882 16224 raft_consensus.cc:1240] T 746aabf856ab401c9ef5e859371e7b67 P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 17. Current term is 18. Ops: []
I20260709 13:22:17.661823 23603 leader_election.cc:290] T 02838c86531d4f98bf58d1c22b8baf77 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:17.547646 23533 consensus_queue.cc:1048] T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.137s
W20260709 13:22:17.743038 16036 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 19baa7bc367844c288bca8f81439c413 with cas_config_opid_index 7: ChangeConfig:REMOVE_PEER failed with leader cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica cb82cc7bd6db46d980c75237a1402860 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 23 committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:22:17.739538 22610 raft_consensus.cc:2749] T 3884f864a96d428785895e5463e5728f P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:22:17.746716 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5718c623ebee4d2bb770f0f9ddb2f164" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:17.747602 16224 raft_consensus.cc:2393] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 21: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
I20260709 13:22:17.748489 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02838c86531d4f98bf58d1c22b8baf77" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:17.749524 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c74490c40243a88e62d1760b5cd814" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 21 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:17.750916 16224 raft_consensus.cc:1275] T ecc1e363230043fe9fef390f0b979f20 P 1ca2725d290e47d68b1a4b609c9816ac [term 23 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 21 index: 13. Preceding OpId from leader: term: 23 index: 14. (index mismatch)
I20260709 13:22:17.740165 23306 raft_consensus.cc:2749] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:22:17.751866 23644 log.cc:927] Time spent T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 1.396s	user 0.010s	sys 0.002s
W20260709 13:22:17.752712 16266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 101645us
I20260709 13:22:17.686549 23287 raft_consensus.cc:2749] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:22:17.753685 16150 raft_consensus.cc:2468] T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 25.
W20260709 13:22:17.753815 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 74009.3us
I20260709 13:22:17.754760 23692 raft_consensus.cc:2804] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 22 FOLLOWER]: Leader election won for term 22
W20260709 13:22:17.755307 16791 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:22:17.758158 16223 raft_consensus.cc:1275] T 1df93779209748b9860f6d893dd3de53 P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 13:22:17.682677 23648 consensus_queue.cc:1059] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: INVALID_TERM, Last received: 17.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.078s
I20260709 13:22:17.761000 23023 raft_consensus.cc:493] T 5a96373bb0e14dce8a266b2fc6c58507 P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:17.685211 23605 raft_consensus.cc:2804] T 8d4ab870a05347868956aa8b841b2981 P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:22:17.795903 23605 raft_consensus.cc:697] T 8d4ab870a05347868956aa8b841b2981 P 647a343a542d4a139fe106c19095129b [term 20 LEADER]: Becoming Leader. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:22:17.796950 23605 consensus_queue.cc:237] T 8d4ab870a05347868956aa8b841b2981 P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:17.798178 16295 raft_consensus.cc:1275] T 1df93779209748b9860f6d893dd3de53 P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
W20260709 13:22:17.763101 16292 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48684 (request call id 3261) took 1649 ms (client timeout 1775 ms). Trace:
I20260709 13:22:17.798897 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02838c86531d4f98bf58d1c22b8baf77" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:17.799541 16298 raft_consensus.cc:2468] T 02838c86531d4f98bf58d1c22b8baf77 P 90f3da57796b488c9087a2f523adf045 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 21.
W20260709 13:22:17.799324 16292 rpcz_store.cc:269] 0709 13:22:16.113810 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:22:16.114276 (+   466us) service_pool.cc:224] Handling call
0709 13:22:17.762861 (+1648585us) spinlock_profiling.cc:243] Waited 892 ms on lock 0x7b54001ef354. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:22:17.763032 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2052118272}
I20260709 13:22:17.799811 16818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e65676b21b748728ad39668cfe722ec" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
W20260709 13:22:17.763828 16117 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 109975us
I20260709 13:22:17.800757 16818 raft_consensus.cc:2468] T 9e65676b21b748728ad39668cfe722ec P cb82cc7bd6db46d980c75237a1402860 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 20.
W20260709 13:22:17.764369 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 89233.9us
I20260709 13:22:17.766575 16142 raft_consensus.cc:2468] T c0eab25ce4c64e018a041beeda2867db P 647a343a542d4a139fe106c19095129b [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 22.
I20260709 13:22:17.767409 16148 raft_consensus.cc:3055] T 7c2095a5f4ad49408dbc7f69c606120d P 647a343a542d4a139fe106c19095129b [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:22:17.769984 16819 raft_consensus.cc:1240] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860 [term 18 LEADER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 17. Current term is 18. Ops: []
I20260709 13:22:17.770064 16823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434bff3082d141838068f323bdb0d1ed" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 21 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:17.771544 16817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31969392ccdb43b491911a319e8bc7e1" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:22:17.802215 16148 raft_consensus.cc:740] T 7c2095a5f4ad49408dbc7f69c606120d P 647a343a542d4a139fe106c19095129b [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:22:17.684834 23539 consensus_queue.cc:1048] T 57a744f5aaf944678a63469cbf514015 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 11, Time since last communication: 0.087s
I20260709 13:22:17.687400 23409 raft_consensus.cc:2749] T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:22:17.729204 23558 raft_consensus.cc:493] T 57a744f5aaf944678a63469cbf514015 P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:17.771939 16824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5718c623ebee4d2bb770f0f9ddb2f164" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:17.651939 23625 raft_consensus.cc:515] T 68a6838d82ef4cac83507f906139d90f P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:17.772250 16820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66ae176961b74813a58bc15c9c4c6f2a" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 27 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:17.773041 16822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2794af5b99984efebdee564035841cb0" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 19 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:17.774431 16821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2372981006d4b218215c28964dea623" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:17.783085 16219 raft_consensus.cc:2468] T d16dfad0c3b540e5b4b86492ddf87208 P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 21.
I20260709 13:22:17.784039 16296 raft_consensus.cc:1240] T 7c2095a5f4ad49408dbc7f69c606120d P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 17. Current term is 18. Ops: []
I20260709 13:22:17.813405 16266 leader_election.cc:304] T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac
I20260709 13:22:17.814291 16222 raft_consensus.cc:1240] T d16dfad0c3b540e5b4b86492ddf87208 P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 20. Current term is 21. Ops: []
I20260709 13:22:17.791127 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c74490c40243a88e62d1760b5cd814" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 21 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:17.790743 23023 raft_consensus.cc:515] T 5a96373bb0e14dce8a266b2fc6c58507 P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:17.762790 23024 raft_consensus.cc:515] T 2794af5b99984efebdee564035841cb0 P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:17.802945 16823 raft_consensus.cc:2410] T 434bff3082d141838068f323bdb0d1ed P cb82cc7bd6db46d980c75237a1402860 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b for term 21 because replica has last-logged OpId of term: 19 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 15.
I20260709 13:22:17.805109 16265 leader_election.cc:304] T 18ad7e3af5504571a69d10c00750a01c P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:22:17.779529 23210 raft_consensus.cc:493] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:17.802286 23692 raft_consensus.cc:697] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 22 LEADER]: Becoming Leader. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:22:17.814013 23558 raft_consensus.cc:3060] T 57a744f5aaf944678a63469cbf514015 P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:22:17.828529 16821 raft_consensus.cc:2393] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 24: Already voted for candidate cb82cc7bd6db46d980c75237a1402860 in this term.
I20260709 13:22:17.807070 23650 raft_consensus.cc:493] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:17.829402 16190 leader_election.cc:304] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: cb82cc7bd6db46d980c75237a1402860
I20260709 13:22:17.830488 16817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdbd78acb4cf46298e0de8ba0d130e19" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:17.827716 16117 leader_election.cc:304] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860
I20260709 13:22:17.837637 16293 raft_consensus.cc:1240] T d16dfad0c3b540e5b4b86492ddf87208 P 90f3da57796b488c9087a2f523adf045 [term 21 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 20. Current term is 21. Ops: []
I20260709 13:22:17.841974 16823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2372981006d4b218215c28964dea623" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:22:17.842300 16819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 24 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:17.842674 16823 raft_consensus.cc:3060] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:22:17.818994 22677 raft_consensus.cc:493] T 5a96373bb0e14dce8a266b2fc6c58507 P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:17.765259 23685 consensus_queue.cc:237] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 22.17, Last appended by leader: 17, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:17.825157 23287 raft_consensus.cc:2749] T 18ad7e3af5504571a69d10c00750a01c P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:22:17.826893 23533 raft_consensus.cc:2804] T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:22:17.840595 16148 consensus_queue.cc:260] T 7c2095a5f4ad49408dbc7f69c606120d P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 17.9, Last appended by leader: 9, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:17.852837 16817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c11074579b046d99e585c2818e18a2c" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:17.848942 16265 leader_election.cc:304] T 84f2da98864b4425bf7c0055951fdd4b P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 647a343a542d4a139fe106c19095129b
I20260709 13:22:17.847260 16819 raft_consensus.cc:2468] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 23.
I20260709 13:22:17.850068 22610 raft_consensus.cc:2749] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:22:17.851550 16818 raft_consensus.cc:1275] T 434bff3082d141838068f323bdb0d1ed P cb82cc7bd6db46d980c75237a1402860 [term 20 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 19 index: 16. Preceding OpId from leader: term: 20 index: 16. (term mismatch)
I20260709 13:22:17.850260 16820 raft_consensus.cc:1275] T 806d7690c2b644959152074bb2cab0a7 P cb82cc7bd6db46d980c75237a1402860 [term 20 LEARNER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 20 index: 13. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 13:22:17.855502 16818 pending_rounds.cc:85] T 434bff3082d141838068f323bdb0d1ed P cb82cc7bd6db46d980c75237a1402860: Aborting all ops after (but not including) 15
I20260709 13:22:17.825661 23648 consensus_queue.cc:1243] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Peer 647a343a542d4a139fe106c19095129b log is divergent from this leader: its last log entry 18.15 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 14
I20260709 13:22:17.855733 16818 pending_rounds.cc:107] T 434bff3082d141838068f323bdb0d1ed P cb82cc7bd6db46d980c75237a1402860: Aborting uncommitted NO_OP operation due to leader change: 19.16
I20260709 13:22:17.859371 16818 raft_consensus.cc:2889] T 434bff3082d141838068f323bdb0d1ed P cb82cc7bd6db46d980c75237a1402860 [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:22:17.861296 16293 raft_consensus.cc:1275] T 31969392ccdb43b491911a319e8bc7e1 P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 18 index: 15. (index mismatch)
I20260709 13:22:17.862322 16190 leader_election.cc:304] T c0eab25ce4c64e018a041beeda2867db P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:22:17.866438 16817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a378d7f796f94615b3b347119a9880fd" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:17.872509 23023 leader_election.cc:290] T 5a96373bb0e14dce8a266b2fc6c58507 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:17.873384 16148 raft_consensus.cc:3060] T 7c2095a5f4ad49408dbc7f69c606120d P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:22:17.876058 23614 raft_consensus.cc:515] T 84f2da98864b4425bf7c0055951fdd4b P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:17.876677 16294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdbd78acb4cf46298e0de8ba0d130e19" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:22:17.877139 16820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "829b4ad8c5a54eababeea1181b960d4b" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:22:17.877306 16294 raft_consensus.cc:3060] T cdbd78acb4cf46298e0de8ba0d130e19 P 90f3da57796b488c9087a2f523adf045 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:22:17.877705 16820 raft_consensus.cc:3060] T 829b4ad8c5a54eababeea1181b960d4b P cb82cc7bd6db46d980c75237a1402860 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:22:17.878528 23614 leader_election.cc:290] T 84f2da98864b4425bf7c0055951fdd4b P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 18 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:17.858379 23533 raft_consensus.cc:493] T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:17.879770 23533 raft_consensus.cc:3060] T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:22:17.858785 22677 raft_consensus.cc:515] T 5a96373bb0e14dce8a266b2fc6c58507 P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:17.881057 16266 leader_election.cc:304] T 23f82224f4564fea93572e8206bad1fe P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
I20260709 13:22:17.882309 23598 raft_consensus.cc:2804] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:22:17.884151 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada5a004ba7c46819051e3ccec71ffdb" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:22:17.884977 16300 raft_consensus.cc:2393] T ada5a004ba7c46819051e3ccec71ffdb P 90f3da57796b488c9087a2f523adf045 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 in current term 22: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
I20260709 13:22:17.886540 16115 leader_election.cc:304] T 02838c86531d4f98bf58d1c22b8baf77 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:22:17.886718 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74ca8140f35c4ae29e8a664f6bdfc829" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 23 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:17.886936 23556 consensus_queue.cc:1243] T 66f75124215f4ca6bef831e174a792d8 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Peer 647a343a542d4a139fe106c19095129b log is divergent from this leader: its last log entry 18.14 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 13
I20260709 13:22:17.887595 23551 consensus_queue.cc:1048] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:22:17.888979 22610 consensus_queue.cc:1048] T 1df93779209748b9860f6d893dd3de53 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:22:17.890354 22610 raft_consensus.cc:2804] T 02838c86531d4f98bf58d1c22b8baf77 P 647a343a542d4a139fe106c19095129b [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:22:17.890707 22610 raft_consensus.cc:493] T 02838c86531d4f98bf58d1c22b8baf77 P 647a343a542d4a139fe106c19095129b [term 21 FOLLOWER]: Starting leader election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
W20260709 13:22:17.891739 16117 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 68994.3us
I20260709 13:22:17.892125 16220 raft_consensus.cc:1240] T 84f2da98864b4425bf7c0055951fdd4b P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 17. Current term is 18. Ops: [17.10-17.10]
I20260709 13:22:17.891047 22610 raft_consensus.cc:3060] T 02838c86531d4f98bf58d1c22b8baf77 P 647a343a542d4a139fe106c19095129b [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:22:17.882858 23598 raft_consensus.cc:697] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [term 22 LEADER]: Becoming Leader. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:22:17.901839 16819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57a744f5aaf944678a63469cbf514015" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:17.902182 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c11074579b046d99e585c2818e18a2c" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:17.906041 16823 raft_consensus.cc:2468] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 25.
I20260709 13:22:17.903815 16295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdbd78acb4cf46298e0de8ba0d130e19" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
W20260709 13:22:17.896260 23620 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.509s	user 0.022s	sys 0.059s
I20260709 13:22:17.912899 16296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e65676b21b748728ad39668cfe722ec" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 21 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:17.844794 23650 raft_consensus.cc:515] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:17.917093 16296 raft_consensus.cc:2393] T 9e65676b21b748728ad39668cfe722ec P 90f3da57796b488c9087a2f523adf045 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 in current term 21: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
I20260709 13:22:17.917528 16292 raft_consensus.cc:1240] T 434bff3082d141838068f323bdb0d1ed P 90f3da57796b488c9087a2f523adf045 [term 20 LEADER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 19. Current term is 20. Ops: []
I20260709 13:22:17.914114 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a96373bb0e14dce8a266b2fc6c58507" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:17.919477 16297 raft_consensus.cc:2468] T 5a96373bb0e14dce8a266b2fc6c58507 P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 17.
I20260709 13:22:17.899329 23287 raft_consensus.cc:2749] T 23f82224f4564fea93572e8206bad1fe P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:22:17.854892 23210 raft_consensus.cc:515] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:17.925730 16294 raft_consensus.cc:2468] T cdbd78acb4cf46298e0de8ba0d130e19 P 90f3da57796b488c9087a2f523adf045 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 24.
I20260709 13:22:17.904397 23695 raft_consensus.cc:2804] T c0eab25ce4c64e018a041beeda2867db P 1ca2725d290e47d68b1a4b609c9816ac [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:22:17.920006 23024 leader_election.cc:290] T 2794af5b99984efebdee564035841cb0 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 19 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:17.937077 16298 raft_consensus.cc:1240] T 31969392ccdb43b491911a319e8bc7e1 P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 17. Current term is 18. Ops: []
I20260709 13:22:17.896417 23306 consensus_queue.cc:1048] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 10, Time since last communication: 0.028s
I20260709 13:22:17.939627 16819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434bff3082d141838068f323bdb0d1ed" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:22:17.917068 23614 raft_consensus.cc:493] T ecc1e363230043fe9fef390f0b979f20 P 1ca2725d290e47d68b1a4b609c9816ac [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:17.944096 22677 leader_election.cc:290] T 5a96373bb0e14dce8a266b2fc6c58507 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
W20260709 13:22:17.920125 23620 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.541s	user 0.022s	sys 0.060s
I20260709 13:22:17.924748 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f82224f4564fea93572e8206bad1fe" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
W20260709 13:22:17.951424 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 102553us
I20260709 13:22:17.954941 16148 raft_consensus.cc:1275] T 7c2095a5f4ad49408dbc7f69c606120d P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 17 index: 9. Preceding OpId from leader: term: 18 index: 9. (term mismatch)
I20260709 13:22:17.958866 16265 leader_election.cc:304] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 647a343a542d4a139fe106c19095129b
I20260709 13:22:17.958782 16148 pending_rounds.cc:85] T 7c2095a5f4ad49408dbc7f69c606120d P 647a343a542d4a139fe106c19095129b: Aborting all ops after (but not including) 8
I20260709 13:22:17.967248 16148 pending_rounds.cc:107] T 7c2095a5f4ad49408dbc7f69c606120d P 647a343a542d4a139fe106c19095129b: Aborting uncommitted NO_OP operation due to leader change: 17.9
I20260709 13:22:17.968931 16148 raft_consensus.cc:2889] T 7c2095a5f4ad49408dbc7f69c606120d P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:22:17.971454 16817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e65676b21b748728ad39668cfe722ec" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:22:17.972275 16817 raft_consensus.cc:3060] T 9e65676b21b748728ad39668cfe722ec P cb82cc7bd6db46d980c75237a1402860 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:22:17.972798 16266 leader_election.cc:304] T ada5a004ba7c46819051e3ccec71ffdb P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860
I20260709 13:22:17.899716 23023 consensus_queue.cc:1059] T d16dfad0c3b540e5b4b86492ddf87208 P 647a343a542d4a139fe106c19095129b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: INVALID_TERM, Last received: 20.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.045s
I20260709 13:22:17.971942 16824 raft_consensus.cc:2468] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 21.
I20260709 13:22:17.935578 23024 consensus_queue.cc:1048] T d2ab0ac8347641e6955e9bce7d90c1a2 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.017s
I20260709 13:22:17.988517 16295 raft_consensus.cc:1240] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045 [term 22 FOLLOWER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 21. Current term is 22. Ops: []
I20260709 13:22:17.983733 23695 raft_consensus.cc:697] T c0eab25ce4c64e018a041beeda2867db P 1ca2725d290e47d68b1a4b609c9816ac [term 22 LEADER]: Becoming Leader. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:22:17.979175 23286 consensus_queue.cc:1048] T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.016s
I20260709 13:22:17.996296 23695 consensus_queue.cc:237] T c0eab25ce4c64e018a041beeda2867db P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.11, Last appended by leader: 11, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:18.000708 23023 raft_consensus.cc:493] T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:22:18.001572 23023 raft_consensus.cc:515] T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:17.944590 23614 raft_consensus.cc:515] T ecc1e363230043fe9fef390f0b979f20 P 1ca2725d290e47d68b1a4b609c9816ac [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:18.006012 23614 leader_election.cc:290] T ecc1e363230043fe9fef390f0b979f20 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:18.014249 23023 leader_election.cc:290] T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:18.016803 23708 raft_consensus.cc:3055] T d16dfad0c3b540e5b4b86492ddf87208 P 647a343a542d4a139fe106c19095129b [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:22:18.034142 23708 raft_consensus.cc:740] T d16dfad0c3b540e5b4b86492ddf87208 P 647a343a542d4a139fe106c19095129b [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:22:18.036418 16040 catalog_manager.cc:5697] T 9e65676b21b748728ad39668cfe722ec P cb82cc7bd6db46d980c75237a1402860 reported cstate change: term changed from 15 to 17. New cstate: current_term: 17 leader_uuid: "cb82cc7bd6db46d980c75237a1402860" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:22:18.037730 23708 consensus_queue.cc:260] T d16dfad0c3b540e5b4b86492ddf87208 P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.11, Last appended by leader: 11, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:18.033389 16820 raft_consensus.cc:2468] T 829b4ad8c5a54eababeea1181b960d4b P cb82cc7bd6db46d980c75237a1402860 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 19.
I20260709 13:22:18.039341 23708 raft_consensus.cc:3060] T d16dfad0c3b540e5b4b86492ddf87208 P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:22:18.040184 16816 raft_consensus.cc:1240] T 829b4ad8c5a54eababeea1181b960d4b P cb82cc7bd6db46d980c75237a1402860 [term 19 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 18. Current term is 19. Ops: []
W20260709 13:22:18.021777 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 63015.1us
I20260709 13:22:18.030253 23558 raft_consensus.cc:515] T 57a744f5aaf944678a63469cbf514015 P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:18.041904 23625 leader_election.cc:290] T 68a6838d82ef4cac83507f906139d90f P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:22:18.043920 23650 leader_election.cc:290] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:22:18.043992 23558 leader_election.cc:290] T 57a744f5aaf944678a63469cbf514015 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 19 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:18.026043 23210 leader_election.cc:290] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:18.026715 23710 raft_consensus.cc:2804] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:22:18.027297 23287 raft_consensus.cc:2749] T ada5a004ba7c46819051e3ccec71ffdb P 90f3da57796b488c9087a2f523adf045 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:22:17.879549 23692 consensus_queue.cc:237] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 21.16, Last appended by leader: 16, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:18.028741 23598 consensus_queue.cc:237] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.15, Last appended by leader: 15, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:18.030916 16822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 24 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
W20260709 13:22:18.053012 16191 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:22:18.054530 16265 leader_election.cc:304] T ecc1e363230043fe9fef390f0b979f20 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:22:18.054314 22610 raft_consensus.cc:515] T 02838c86531d4f98bf58d1c22b8baf77 P 647a343a542d4a139fe106c19095129b [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:18.056407 16789 leader_election.cc:304] T ada5a004ba7c46819051e3ccec71ffdb P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:22:18.065081 16822 raft_consensus.cc:2468] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 23.
I20260709 13:22:18.069754 16825 raft_consensus.cc:3055] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:22:18.070159 16825 raft_consensus.cc:740] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:22:18.022780 23533 raft_consensus.cc:515] T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:18.072376 16825 consensus_queue.cc:260] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:18.073561 23533 leader_election.cc:290] T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 23 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:18.074728 16293 raft_consensus.cc:2393] T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 in current term 23: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
I20260709 13:22:18.075847 16825 raft_consensus.cc:3060] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:22:18.076272 22610 leader_election.cc:290] T 02838c86531d4f98bf58d1c22b8baf77 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 22 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:18.086179 16265 leader_election.cc:304] T 8158bbd17b994d2eb96f560bd0f1b88d P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:22:18.071043 16819 raft_consensus.cc:2468] T 434bff3082d141838068f323bdb0d1ed P cb82cc7bd6db46d980c75237a1402860 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 20.
I20260709 13:22:18.088737 16817 raft_consensus.cc:2468] T 9e65676b21b748728ad39668cfe722ec P cb82cc7bd6db46d980c75237a1402860 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 21.
I20260709 13:22:18.038363 16040 catalog_manager.cc:5697] T 3a7ca61592e848ea9b0fb81a2ce33cdc P cb82cc7bd6db46d980c75237a1402860 reported cstate change: term changed from 17 to 19. New cstate: current_term: 19 leader_uuid: "cb82cc7bd6db46d980c75237a1402860" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:22:18.097355 16115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 208758us
I20260709 13:22:18.104492 16294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4dc6b0541ca431ebc757beac00521f1" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:18.105351 16294 raft_consensus.cc:2468] T e4dc6b0541ca431ebc757beac00521f1 P 90f3da57796b488c9087a2f523adf045 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 21.
I20260709 13:22:18.108923 16295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02838c86531d4f98bf58d1c22b8baf77" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:22:18.109697 16295 raft_consensus.cc:3060] T 02838c86531d4f98bf58d1c22b8baf77 P 90f3da57796b488c9087a2f523adf045 [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:22:18.116276 16789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 60006us
I20260709 13:22:18.117152 16789 leader_election.cc:304] T 8c11074579b046d99e585c2818e18a2c P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:22:18.120425 22610 raft_consensus.cc:493] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
W20260709 13:22:18.124195 16042 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:49246 (request call id 333) took 40101 ms (client timeout 15000 ms). Trace:
I20260709 13:22:18.126129 23689 raft_consensus.cc:493] T c2f5f89186204242bf377f7c42b2e2da P 1ca2725d290e47d68b1a4b609c9816ac [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:18.097558 23710 raft_consensus.cc:697] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045 [term 23 LEADER]: Becoming Leader. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:22:18.126909 23689 raft_consensus.cc:515] T c2f5f89186204242bf377f7c42b2e2da P 1ca2725d290e47d68b1a4b609c9816ac [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:18.128129 23710 consensus_queue.cc:237] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.12, Last appended by leader: 12, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:18.128233 16040 catalog_manager.cc:5697] T 806d7690c2b644959152074bb2cab0a7 P cb82cc7bd6db46d980c75237a1402860 reported cstate change: term changed from 15 to 18, leader changed from 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130) to 647a343a542d4a139fe106c19095129b (127.14.177.129). New cstate: current_term: 18 leader_uuid: "647a343a542d4a139fe106c19095129b" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } } }
I20260709 13:22:18.130070 23689 leader_election.cc:290] T c2f5f89186204242bf377f7c42b2e2da P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:18.099079 23287 raft_consensus.cc:2749] T ecc1e363230043fe9fef390f0b979f20 P 90f3da57796b488c9087a2f523adf045 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:22:18.097415 23559 raft_consensus.cc:2804] T 84f2da98864b4425bf7c0055951fdd4b P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:22:18.138511 16294 raft_consensus.cc:1275] T ecc1e363230043fe9fef390f0b979f20 P 90f3da57796b488c9087a2f523adf045 [term 23 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 23 index: 14. (index mismatch)
I20260709 13:22:18.142326 22610 raft_consensus.cc:515] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } }
I20260709 13:22:18.120625 23478 raft_consensus.cc:493] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:22:18.233148 16295 raft_consensus.cc:2468] T 02838c86531d4f98bf58d1c22b8baf77 P 90f3da57796b488c9087a2f523adf045 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 22.
I20260709 13:22:18.141530 23398 raft_consensus.cc:2749] T 8c11074579b046d99e585c2818e18a2c P cb82cc7bd6db46d980c75237a1402860 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:22:18.270061 22610 leader_election.cc:290] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:18.137126 23533 raft_consensus.cc:2749] T 8158bbd17b994d2eb96f560bd0f1b88d P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:22:18.276641 16115 leader_election.cc:304] T 5a96373bb0e14dce8a266b2fc6c58507 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:22:18.280916 16148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "829b4ad8c5a54eababeea1181b960d4b" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
W20260709 13:22:18.268587 16117 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 117529us
W20260709 13:22:18.277331 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 128063us
I20260709 13:22:18.130311 16040 catalog_manager.cc:5697] T 829b4ad8c5a54eababeea1181b960d4b P cb82cc7bd6db46d980c75237a1402860 reported cstate change: term changed from 12 to 16, leader changed from 647a343a542d4a139fe106c19095129b (127.14.177.129) to cb82cc7bd6db46d980c75237a1402860 (127.14.177.132). New cstate: current_term: 16 leader_uuid: "cb82cc7bd6db46d980c75237a1402860" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:22:18.140811 23648 raft_consensus.cc:2749] T ada5a004ba7c46819051e3ccec71ffdb P cb82cc7bd6db46d980c75237a1402860 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:22:18.279426 16825 raft_consensus.cc:1275] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860 [term 18 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 18 index: 15. (index mismatch)
I20260709 13:22:18.289470 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f2da98864b4425bf7c0055951fdd4b" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:22:18.291193 16145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 24 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:22:18.294071 16151 pending_rounds.cc:85] T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b: Aborting all ops after (but not including) 13
I20260709 13:22:18.294554 16151 pending_rounds.cc:107] T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b: Aborting uncommitted NO_OP operation due to leader change: 18.14
I20260709 13:22:18.294903 16151 raft_consensus.cc:2889] T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:22:18.283790 16191 leader_election.cc:304] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
I20260709 13:22:18.302213 16146 raft_consensus.cc:1275] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 20 index: 13. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 13:22:18.306011 16117 leader_election.cc:304] T 434bff3082d141838068f323bdb0d1ed P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860
I20260709 13:22:18.306579 16144 raft_consensus.cc:1275] T 9c61a39fd4f84424b129c1a6f9ba404d P 647a343a542d4a139fe106c19095129b [term 22 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 19 index: 10. Preceding OpId from leader: term: 22 index: 12. (index mismatch)
W20260709 13:22:18.307399 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 221350us
I20260709 13:22:18.309186 16265 leader_election.cc:304] T d16dfad0c3b540e5b4b86492ddf87208 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:22:18.310777 23558 raft_consensus.cc:2804] T d16dfad0c3b540e5b4b86492ddf87208 P 90f3da57796b488c9087a2f523adf045 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:22:18.232445 23478 raft_consensus.cc:515] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:18.311385 23558 raft_consensus.cc:697] T d16dfad0c3b540e5b4b86492ddf87208 P 90f3da57796b488c9087a2f523adf045 [term 21 LEADER]: Becoming Leader. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:22:18.313422 23478 leader_election.cc:290] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:18.312922 16040 catalog_manager.cc:5697] T 74ca8140f35c4ae29e8a664f6bdfc829 P cb82cc7bd6db46d980c75237a1402860 reported cstate change: term changed from 13 to 19, leader changed from 90f3da57796b488c9087a2f523adf045 (127.14.177.131) to cb82cc7bd6db46d980c75237a1402860 (127.14.177.132). New cstate: current_term: 19 leader_uuid: "cb82cc7bd6db46d980c75237a1402860" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:22:18.124774 16042 rpcz_store.cc:269] 0709 13:21:38.000558 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:38.002524 (+  1966us) service_pool.cc:224] Handling call
0709 13:22:18.097422 (+40094898us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:22:04.005152 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:22:04.049999 (+ 44847us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:22:04.050052 (+    53us) write_op.cc:435] Acquired schema lock
0709 13:22:04.050070 (+    18us) tablet.cc:662] Decoding operations
0709 13:22:04.057751 (+  7681us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:22:04.057895 (+   144us) write_op.cc:641] Partition lock acquired for write op
0709 13:22:04.057932 (+    37us) tablet.cc:685] Acquiring locks for 18 operations
0709 13:22:04.059792 (+  1860us) tablet.cc:701] Row locks acquired
0709 13:22:04.059850 (+    58us) write_op.cc:260] PREPARE: finished
0709 13:22:04.060072 (+   222us) write_op.cc:270] Start()
0709 13:22:04.060260 (+   188us) write_op.cc:276] Timestamp: P: 1783603324059995 usec, L: 0
0709 13:22:04.060304 (+    44us) op_driver.cc:348] REPLICATION: starting
0709 13:22:04.308051 (+247747us) log.cc:844] Serialized 7168 byte log entry
0709 13:22:04.699610 (+391559us) spinlock_profiling.cc:243] Waited 14.6 ms on lock 0x7b54003cdf54. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5ae28 00007f69b4595581 00007f69b4594d58 00007f69b45999f1 00007f69b45999a9 00007f69b4599939 00007f69b4599901 00007f69b4598b2d 000000000038f354 000000000038f199
0709 13:22:05.168906 (+469296us) op_driver.cc:464] REPLICATION: finished
0709 13:22:05.669302 (+500396us) write_op.cc:301] APPLY: starting
0709 13:22:05.669424 (+   122us) tablet.cc:1367] starting BulkCheckPresence
0709 13:22:06.128327 (+458903us) tablet.cc:1370] finished BulkCheckPresence
0709 13:22:06.128390 (+    63us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:22:06.857382 (+728992us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:22:06.857845 (+   463us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=36,key_file_lookups=36,delta_file_lookups=0,mrs_lookups=0
0709 13:22:06.857916 (+    71us) write_op.cc:312] APPLY: finished
0709 13:22:06.858536 (+   620us) log.cc:844] Serialized 164 byte log entry
0709 13:22:07.164441 (+305905us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:22:07.266880 (+102439us) write_op.cc:454] Released schema lock
0709 13:22:07.266990 (+   110us) write_op.cc:341] FINISH: Updating metrics
0709 13:22:07.267611 (+   621us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:22:07.267644 (+    33us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":4995611,"reported_tablets":48,"child_traces":[["op",{"apply.queue_time_us":487102,"apply.run_cpu_time_us":107901,"apply.run_wall_time_us":1598262,"cfile_cache_hit":74,"cfile_cache_hit_bytes":79295,"num_ops":18,"prepare.queue_time_us":113874,"prepare.run_cpu_time_us":26371,"prepare.run_wall_time_us":732371,"raft.queue_time_us":266710,"raft.run_cpu_time_us":37106,"raft.run_wall_time_us":404077,"replication_time_us":1081831,"spinlock_wait_cycles":33547520,"thread_start_us":885230,"threads_started":5,"wal-append.queue_time_us":503265,"wal-append.run_cpu_time_us":24306,"wal-append.run_wall_time_us":2961461}]]}
I20260709 13:22:18.317862 23712 raft_consensus.cc:493] T e5ad4eb281564936a59f41e387a47156 P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:22:18.319551 16218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74ca8140f35c4ae29e8a664f6bdfc829" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:22:18.321482 16825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9944d5e4b8c4109af9955c4f08c9095" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:18.312781 23558 consensus_queue.cc:237] T d16dfad0c3b540e5b4b86492ddf87208 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.11, Last appended by leader: 11, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:22:18.322865 16115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 46346.8us
I20260709 13:22:18.328655 16295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806d7690c2b644959152074bb2cab0a7" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:18.321266 16147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a96373bb0e14dce8a266b2fc6c58507" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:22:18.333937 16116 leader_election.cc:304] T 08c74490c40243a88e62d1760b5cd814 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:22:18.336048 16221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68a6838d82ef4cac83507f906139d90f" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:18.318593 16265 leader_election.cc:304] T 1df93779209748b9860f6d893dd3de53 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:22:18.337740 16221 raft_consensus.cc:2468] T 68a6838d82ef4cac83507f906139d90f P 1ca2725d290e47d68b1a4b609c9816ac [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 20.
I20260709 13:22:18.330533 16147 raft_consensus.cc:2468] T 5a96373bb0e14dce8a266b2fc6c58507 P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 17.
W20260709 13:22:18.335744 16789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 50150.8us
I20260709 13:22:18.341369 16825 raft_consensus.cc:2468] T c9944d5e4b8c4109af9955c4f08c9095 P cb82cc7bd6db46d980c75237a1402860 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 22.
I20260709 13:22:18.332042 23689 raft_consensus.cc:2804] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [term 25 FOLLOWER]: Leader election won for term 25
I20260709 13:22:18.345481 23689 raft_consensus.cc:697] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [term 25 LEADER]: Becoming Leader. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:22:18.346064 16294 raft_consensus.cc:1240] T 2794af5b99984efebdee564035841cb0 P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 18. Current term is 19. Ops: []
I20260709 13:22:18.347206 16300 raft_consensus.cc:1240] T 57a744f5aaf944678a63469cbf514015 P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 18. Current term is 19. Ops: []
I20260709 13:22:18.346624 23689 consensus_queue.cc:237] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 23.15, Last appended by leader: 15, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:22:18.347903 23694 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 18 current term: 18 is higher.
I20260709 13:22:18.353072 16295 raft_consensus.cc:2468] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 20.
I20260709 13:22:18.343854 16222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4dc6b0541ca431ebc757beac00521f1" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:18.358268 16192 leader_election.cc:304] T 829b4ad8c5a54eababeea1181b960d4b P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:22:18.318658 23712 raft_consensus.cc:515] T e5ad4eb281564936a59f41e387a47156 P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:18.360306 16825 raft_consensus.cc:1240] T 23f82224f4564fea93572e8206bad1fe P cb82cc7bd6db46d980c75237a1402860 [term 18 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 17. Current term is 18. Ops: []
I20260709 13:22:18.327006 23024 raft_consensus.cc:493] T b87147731a68400e8cce9b879daa7c5d P 90f3da57796b488c9087a2f523adf045 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:18.305091 23551 raft_consensus.cc:493] T e4dc6b0541ca431ebc757beac00521f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:18.286054 23606 consensus_queue.cc:1059] T 746aabf856ab401c9ef5e859371e7b67 P 647a343a542d4a139fe106c19095129b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: INVALID_TERM, Last received: 17.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.058s
I20260709 13:22:18.326746 23625 raft_consensus.cc:493] T 3884f864a96d428785895e5463e5728f P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:22:18.368934 16266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 420115us
I20260709 13:22:18.375505 23681 raft_consensus.cc:2804] T 829b4ad8c5a54eababeea1181b960d4b P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:22:18.376395 22610 raft_consensus.cc:2749] T 434bff3082d141838068f323bdb0d1ed P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:22:18.344086 23688 raft_consensus.cc:493] T 68a6838d82ef4cac83507f906139d90f P 1ca2725d290e47d68b1a4b609c9816ac [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:18.351907 23605 raft_consensus.cc:2804] T 5a96373bb0e14dce8a266b2fc6c58507 P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:22:18.389926 16823 raft_consensus.cc:1275] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860 [term 23 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 23 index: 14. (index mismatch)
I20260709 13:22:18.395334 22610 consensus_queue.cc:1048] T ecc1e363230043fe9fef390f0b979f20 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:22:18.397235 22610 consensus_queue.cc:1048] T 1df93779209748b9860f6d893dd3de53 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:22:18.365280 23023 raft_consensus.cc:493] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:18.418020 16144 raft_consensus.cc:1240] T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b [term 18 LEADER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 17. Current term is 18. Ops: []
I20260709 13:22:18.392339 23681 raft_consensus.cc:697] T 829b4ad8c5a54eababeea1181b960d4b P 1ca2725d290e47d68b1a4b609c9816ac [term 19 LEADER]: Becoming Leader. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:22:18.387256 23694 raft_consensus.cc:493] T 3884f864a96d428785895e5463e5728f P cb82cc7bd6db46d980c75237a1402860 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:18.393388 23357 raft_consensus.cc:493] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:18.428256 16266 leader_election.cc:304] T 66ae176961b74813a58bc15c9c4c6f2a P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860
I20260709 13:22:18.432183 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5718c623ebee4d2bb770f0f9ddb2f164" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:22:18.433131 16149 raft_consensus.cc:2468] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 20.
I20260709 13:22:18.435948 23399 raft_consensus.cc:493] T 26354361e9b044cea1c2b86c4882fe92 P cb82cc7bd6db46d980c75237a1402860 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:18.436710 23399 raft_consensus.cc:515] T 26354361e9b044cea1c2b86c4882fe92 P cb82cc7bd6db46d980c75237a1402860 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:18.421589 23023 raft_consensus.cc:515] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:18.435894 23712 leader_election.cc:290] T e5ad4eb281564936a59f41e387a47156 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:18.443545 23357 raft_consensus.cc:515] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:18.394229 23625 raft_consensus.cc:515] T 3884f864a96d428785895e5463e5728f P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:18.446187 23357 leader_election.cc:290] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:18.440932 23023 leader_election.cc:290] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:18.315080 16040 catalog_manager.cc:5697] T 23f82224f4564fea93572e8206bad1fe P cb82cc7bd6db46d980c75237a1402860 reported cstate change: term changed from 12 to 16. New cstate: current_term: 16 leader_uuid: "cb82cc7bd6db46d980c75237a1402860" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:22:18.383833 23551 raft_consensus.cc:515] T e4dc6b0541ca431ebc757beac00521f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:18.448597 23620 consensus_queue.cc:1048] T 26354361e9b044cea1c2b86c4882fe92 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:22:18.448184 16040 catalog_manager.cc:5697] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 reported cstate change: term changed from 10 to 21. New cstate: current_term: 21 leader_uuid: "cb82cc7bd6db46d980c75237a1402860" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:22:18.450459 16222 raft_consensus.cc:2468] T e4dc6b0541ca431ebc757beac00521f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 21.
I20260709 13:22:18.449857 23551 leader_election.cc:290] T e4dc6b0541ca431ebc757beac00521f1 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:18.418601 23024 raft_consensus.cc:515] T b87147731a68400e8cce9b879daa7c5d P 90f3da57796b488c9087a2f523adf045 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:18.454447 23024 leader_election.cc:290] T b87147731a68400e8cce9b879daa7c5d P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:18.442498 23681 consensus_queue.cc:237] T 829b4ad8c5a54eababeea1181b960d4b P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:18.393916 23688 raft_consensus.cc:515] T 68a6838d82ef4cac83507f906139d90f P 1ca2725d290e47d68b1a4b609c9816ac [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:18.446656 23625 leader_election.cc:290] T 3884f864a96d428785895e5463e5728f P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:18.473862 16142 raft_consensus.cc:1275] T 2794af5b99984efebdee564035841cb0 P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 16 index: 14. Preceding OpId from leader: term: 18 index: 16. (index mismatch)
I20260709 13:22:18.455122 23024 raft_consensus.cc:493] T 7c2095a5f4ad49408dbc7f69c606120d P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:22:18.537562 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 253849us
I20260709 13:22:18.491499 23306 raft_consensus.cc:2749] T 1df93779209748b9860f6d893dd3de53 P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:22:18.542019 16266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 118887us
I20260709 13:22:18.543835 16266 leader_election.cc:304] T cdbd78acb4cf46298e0de8ba0d130e19 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860
W20260709 13:22:18.532697 16192 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 164545us
I20260709 13:22:18.508507 23719 raft_consensus.cc:2749] T 08c74490c40243a88e62d1760b5cd814 P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:22:18.450299 16040 catalog_manager.cc:5697] T ada5a004ba7c46819051e3ccec71ffdb P cb82cc7bd6db46d980c75237a1402860 reported cstate change: term changed from 14 to 19, leader changed from 90f3da57796b488c9087a2f523adf045 (127.14.177.131) to cb82cc7bd6db46d980c75237a1402860 (127.14.177.132). New cstate: current_term: 19 leader_uuid: "cb82cc7bd6db46d980c75237a1402860" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:22:18.546834 16192 leader_election.cc:304] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:22:18.552911 16266 leader_election.cc:304] T 8c11074579b046d99e585c2818e18a2c P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860
I20260709 13:22:18.534008 23606 consensus_queue.cc:1243] T 746aabf856ab401c9ef5e859371e7b67 P 647a343a542d4a139fe106c19095129b [LEADER]: Peer cb82cc7bd6db46d980c75237a1402860 log is divergent from this leader: its last log entry 18.15 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 14
I20260709 13:22:18.555616 16225 raft_consensus.cc:1240] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 22 LEADER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 21. Current term is 22. Ops: []
I20260709 13:22:18.541643 23399 leader_election.cc:290] T 26354361e9b044cea1c2b86c4882fe92 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:18.473461 23720 consensus_queue.cc:1048] T 434bff3082d141838068f323bdb0d1ed P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.114s
W20260709 13:22:18.563306 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 205338us
I20260709 13:22:18.546370 23625 raft_consensus.cc:2749] T 66ae176961b74813a58bc15c9c4c6f2a P 90f3da57796b488c9087a2f523adf045 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 13:22:18.565611 16218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5718c623ebee4d2bb770f0f9ddb2f164" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:18.546151 23210 raft_consensus.cc:493] T fffab26427e24d398e2695aad40ca134 P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:22:18.558894 16266 leader_election.cc:304] T a378d7f796f94615b3b347119a9880fd P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860
I20260709 13:22:18.554167 23551 raft_consensus.cc:2804] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:22:18.574141 16218 raft_consensus.cc:2468] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 21.
W20260709 13:22:18.565728 23722 log.cc:927] Time spent T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860: Append to log took a long time: real 0.083s	user 0.003s	sys 0.000s
I20260709 13:22:18.546546 23692 consensus_queue.cc:1048] T 7c2095a5f4ad49408dbc7f69c606120d P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.024s
I20260709 13:22:18.581158 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4dc6b0541ca431ebc757beac00521f1" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:22:18.587527 16149 raft_consensus.cc:2468] T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 20.
I20260709 13:22:18.582295 16146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e65676b21b748728ad39668cfe722ec" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:22:18.543383 23024 raft_consensus.cc:515] T 7c2095a5f4ad49408dbc7f69c606120d P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:18.563414 23737 raft_consensus.cc:493] T b6f07842d3aa47858df5665874c86f3a P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:18.588460 16146 raft_consensus.cc:3060] T 9e65676b21b748728ad39668cfe722ec P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Advancing to term 21
I20260709 13:22:18.568357 23558 raft_consensus.cc:2749] T cdbd78acb4cf46298e0de8ba0d130e19 P 90f3da57796b488c9087a2f523adf045 [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:22:18.508685 23694 raft_consensus.cc:515] T 3884f864a96d428785895e5463e5728f P cb82cc7bd6db46d980c75237a1402860 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:22:18.596123 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 220397us
I20260709 13:22:18.561805 23603 consensus_queue.cc:1059] T 746aabf856ab401c9ef5e859371e7b67 P 647a343a542d4a139fe106c19095129b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: INVALID_TERM, Last received: 17.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.025s
I20260709 13:22:18.598404 16116 leader_election.cc:400] T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 24 election: Vote denied by peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943) with higher term. Message: Invalid argument: T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b for earlier term 24. Current term is 25.
I20260709 13:22:18.598896 16148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68a6838d82ef4cac83507f906139d90f" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:22:18.599592 16148 raft_consensus.cc:2468] T 68a6838d82ef4cac83507f906139d90f P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 20.
I20260709 13:22:18.601536 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57a744f5aaf944678a63469cbf514015" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:22:18.599112 16116 leader_election.cc:403] T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 24 election: Cancelling election due to peer responding with higher term
I20260709 13:22:18.578816 23732 raft_consensus.cc:493] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:18.581064 23399 raft_consensus.cc:493] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:18.581862 23551 raft_consensus.cc:493] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac [term 22 FOLLOWER]: Starting leader election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
W20260709 13:22:18.675153 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 110010us
W20260709 13:22:18.681030 16266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 122245us
I20260709 13:22:18.684116 16148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2794af5b99984efebdee564035841cb0" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 19 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:22:18.684764 16148 raft_consensus.cc:3060] T 2794af5b99984efebdee564035841cb0 P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:22:18.573261 23210 raft_consensus.cc:515] T fffab26427e24d398e2695aad40ca134 P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:18.586828 23712 consensus_queue.cc:1059] T 829b4ad8c5a54eababeea1181b960d4b P 647a343a542d4a139fe106c19095129b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: INVALID_TERM, Last received: 18.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.096s
I20260709 13:22:18.594586 23023 consensus_queue.cc:1048] T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.092s
I20260709 13:22:18.682317 23558 raft_consensus.cc:2749] T a378d7f796f94615b3b347119a9880fd P 90f3da57796b488c9087a2f523adf045 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:22:18.706307 23606 raft_consensus.cc:2749] T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b [term 25 FOLLOWER]: Leader election lost for term 24. Reason: Vote denied by peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943) with higher term. Message: Invalid argument: T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b for earlier term 24. Current term is 25.
W20260709 13:22:18.706928 23721 log.cc:927] Time spent T c0eab25ce4c64e018a041beeda2867db P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 0.140s	user 0.000s	sys 0.000s
I20260709 13:22:18.706998 23688 leader_election.cc:290] T 68a6838d82ef4cac83507f906139d90f P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:18.706910 23694 leader_election.cc:290] T 3884f864a96d428785895e5463e5728f P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
W20260709 13:22:18.720119 23742 log.cc:927] Time spent T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.196s	user 0.000s	sys 0.005s
I20260709 13:22:18.689428 23551 raft_consensus.cc:3060] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:22:18.688800 23738 raft_consensus.cc:2749] T 8c11074579b046d99e585c2818e18a2c P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:22:18.692004 23732 raft_consensus.cc:515] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:18.715871 16146 raft_consensus.cc:2468] T 9e65676b21b748728ad39668cfe722ec P 647a343a542d4a139fe106c19095129b [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 21.
I20260709 13:22:18.716771 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a9de347c711477e87c582d003f36925" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:22:18.721442 16149 raft_consensus.cc:2468] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 24.
I20260709 13:22:18.722525 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87147731a68400e8cce9b879daa7c5d" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:22:18.722671 16143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3884f864a96d428785895e5463e5728f" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 17 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:22:18.723088 16149 raft_consensus.cc:2468] T b87147731a68400e8cce9b879daa7c5d P 647a343a542d4a139fe106c19095129b [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 21.
I20260709 13:22:18.723338 16143 raft_consensus.cc:2468] T 3884f864a96d428785895e5463e5728f P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 16.
W20260709 13:22:18.732960 16790 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:22:18.733091 23551 raft_consensus.cc:515] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:18.733855 16143 raft_consensus.cc:1275] T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 18 index: 15. Preceding OpId from leader: term: 20 index: 16. (index mismatch)
I20260709 13:22:18.735746 23551 leader_election.cc:290] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 23 election: Requested vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:18.735064 16143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3884f864a96d428785895e5463e5728f" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 17 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:22:18.745988 16143 raft_consensus.cc:2468] T 3884f864a96d428785895e5463e5728f P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 16.
I20260709 13:22:18.747854 16143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26354361e9b044cea1c2b86c4882fe92" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
W20260709 13:22:18.724375 23709 log.cc:927] Time spent T 84f2da98864b4425bf7c0055951fdd4b P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.295s	user 0.000s	sys 0.001s
I20260709 13:22:18.717383 16265 leader_election.cc:304] T 68a6838d82ef4cac83507f906139d90f P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:22:18.735936 23645 consensus_queue.cc:1059] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: INVALID_TERM, Last received: 21.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.008s
I20260709 13:22:18.736917 16790 leader_election.cc:304] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:22:18.748724 16143 raft_consensus.cc:2468] T 26354361e9b044cea1c2b86c4882fe92 P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 17.
W20260709 13:22:18.737092 16266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 51839.1us
I20260709 13:22:18.757970 16266 leader_election.cc:304] T 9e65676b21b748728ad39668cfe722ec P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 
W20260709 13:22:18.745025 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 148159us
I20260709 13:22:18.605003 23737 raft_consensus.cc:515] T b6f07842d3aa47858df5665874c86f3a P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:18.687657 23399 raft_consensus.cc:515] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:18.756482 16148 raft_consensus.cc:2468] T 2794af5b99984efebdee564035841cb0 P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 19.
I20260709 13:22:18.758352 23738 raft_consensus.cc:2804] T 68a6838d82ef4cac83507f906139d90f P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:22:18.757400 16265 leader_election.cc:304] T e4dc6b0541ca431ebc757beac00521f1 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:22:18.759860 16146 raft_consensus.cc:3055] T 746aabf856ab401c9ef5e859371e7b67 P 647a343a542d4a139fe106c19095129b [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:22:18.768093 16146 raft_consensus.cc:740] T 746aabf856ab401c9ef5e859371e7b67 P 647a343a542d4a139fe106c19095129b [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:22:18.775741 16265 leader_election.cc:304] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:22:18.758873 23450 raft_consensus.cc:2804] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:22:18.779318 16265 leader_election.cc:304] T b87147731a68400e8cce9b879daa7c5d P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:22:18.770778 23738 raft_consensus.cc:493] T 68a6838d82ef4cac83507f906139d90f P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:18.784344 23710 raft_consensus.cc:2804] T e4dc6b0541ca431ebc757beac00521f1 P 90f3da57796b488c9087a2f523adf045 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:22:18.811805 23738 raft_consensus.cc:3060] T 68a6838d82ef4cac83507f906139d90f P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:22:18.811879 23710 raft_consensus.cc:493] T e4dc6b0541ca431ebc757beac00521f1 P 90f3da57796b488c9087a2f523adf045 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:18.812448 23710 raft_consensus.cc:3060] T e4dc6b0541ca431ebc757beac00521f1 P 90f3da57796b488c9087a2f523adf045 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:22:18.784675 23720 raft_consensus.cc:2804] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:22:18.815021 23720 raft_consensus.cc:493] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:18.776912 23645 consensus_queue.cc:1243] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Peer 1ca2725d290e47d68b1a4b609c9816ac log is divergent from this leader: its last log entry 22.17 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 16
I20260709 13:22:18.815433 23720 raft_consensus.cc:3060] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:22:18.815749 23645 raft_consensus.cc:493] T d2ab0ac8347641e6955e9bce7d90c1a2 P cb82cc7bd6db46d980c75237a1402860 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:18.816267 23645 raft_consensus.cc:515] T d2ab0ac8347641e6955e9bce7d90c1a2 P cb82cc7bd6db46d980c75237a1402860 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:18.818280 23645 leader_election.cc:290] T d2ab0ac8347641e6955e9bce7d90c1a2 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:18.778132 23745 raft_consensus.cc:2804] T 9e65676b21b748728ad39668cfe722ec P 90f3da57796b488c9087a2f523adf045 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:22:18.789824 16790 leader_election.cc:304] T 3884f864a96d428785895e5463e5728f P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:22:18.819154 16291 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48674 (request call id 3562) took 1042 ms. Trace:
I20260709 13:22:18.819804 16790 leader_election.cc:304] T 26354361e9b044cea1c2b86c4882fe92 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:22:18.819579 16291 rpcz_store.cc:276] 0709 13:22:17.777324 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:22:17.781952 (+  4628us) service_pool.cc:224] Handling call
0709 13:22:17.795530 (+ 13578us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:22:17.796031 (+   501us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:22:17.796068 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:22:17.796703 (+   635us) log.cc:844] Serialized 26 byte log entry
0709 13:22:18.087808 (+291105us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:22:18.087866 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:22:18.088141 (+   275us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:22:18.818908 (+730767us) raft_consensus.cc:1685] finished
0709 13:22:18.818930 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:22:18.819089 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1227,"raft.run_cpu_time_us":91,"raft.run_wall_time_us":92,"thread_start_us":266942,"threads_started":1,"wal-append.queue_time_us":569929}
I20260709 13:22:18.790419 23648 consensus_queue.cc:1048] T 2794af5b99984efebdee564035841cb0 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:22:18.791067 23694 consensus_queue.cc:1048] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 13:22:18.820228 16265 leader_election.cc:304] T 3884f864a96d428785895e5463e5728f P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:22:18.791849 23732 leader_election.cc:290] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:18.792688 23737 leader_election.cc:290] T b6f07842d3aa47858df5665874c86f3a P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:18.782374 23450 raft_consensus.cc:493] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:18.832831 23450 raft_consensus.cc:3060] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:22:18.834214 23713 log.cc:927] Time spent T 8158bbd17b994d2eb96f560bd0f1b88d P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 0.378s	user 0.000s	sys 0.009s
I20260709 13:22:18.794742 23398 raft_consensus.cc:493] T 8638ee363273426db1393dd81e5c7b72 P cb82cc7bd6db46d980c75237a1402860 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:22:18.835441 23398 raft_consensus.cc:515] T 8638ee363273426db1393dd81e5c7b72 P cb82cc7bd6db46d980c75237a1402860 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:22:18.836180 23741 log.cc:927] Time spent T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.130s	user 0.001s	sys 0.000s
I20260709 13:22:18.801450 23559 raft_consensus.cc:493] T 84f2da98864b4425bf7c0055951fdd4b P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:22:18.801836 16117 leader_election.cc:304] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac
I20260709 13:22:18.802217 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26354361e9b044cea1c2b86c4882fe92" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:18.802490 23023 consensus_queue.cc:1059] T 23f82224f4564fea93572e8206bad1fe P 647a343a542d4a139fe106c19095129b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: INVALID_TERM, Last received: 17.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:22:18.803900 16300 raft_consensus.cc:1275] T cdbd78acb4cf46298e0de8ba0d130e19 P 90f3da57796b488c9087a2f523adf045 [term 24 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 22 index: 17. Preceding OpId from leader: term: 24 index: 18. (index mismatch)
I20260709 13:22:18.804668 23638 raft_consensus.cc:493] T b87147731a68400e8cce9b879daa7c5d P cb82cc7bd6db46d980c75237a1402860 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:18.805358 23717 consensus_queue.cc:1059] T 84f2da98864b4425bf7c0055951fdd4b P 647a343a542d4a139fe106c19095129b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:22:18.805146 16146 consensus_queue.cc:260] T 746aabf856ab401c9ef5e859371e7b67 P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:18.802997 16295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3884f864a96d428785895e5463e5728f" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 17 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:18.844636 16295 raft_consensus.cc:2468] T 3884f864a96d428785895e5463e5728f P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 16.
I20260709 13:22:18.807199 23399 leader_election.cc:290] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:18.808236 23024 leader_election.cc:290] T 7c2095a5f4ad49408dbc7f69c606120d P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:22:18.810655 23735 consensus_queue.cc:1048] T ecc1e363230043fe9fef390f0b979f20 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:22:18.757385 16040 catalog_manager.cc:5697] T 3884f864a96d428785895e5463e5728f P cb82cc7bd6db46d980c75237a1402860 reported cstate change: term changed from 8 to 13. New cstate: current_term: 13 leader_uuid: "647a343a542d4a139fe106c19095129b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:22:18.829113 23533 raft_consensus.cc:2804] T b87147731a68400e8cce9b879daa7c5d P 90f3da57796b488c9087a2f523adf045 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:22:18.819274 23745 raft_consensus.cc:697] T 9e65676b21b748728ad39668cfe722ec P 90f3da57796b488c9087a2f523adf045 [term 21 LEADER]: Becoming Leader. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:22:18.831987 16151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a9de347c711477e87c582d003f36925" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:22:18.865283 16151 raft_consensus.cc:2468] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 24.
W20260709 13:22:18.839000 16266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 64952.6us
W20260709 13:22:18.838754 16116 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:22:18.868185 16790 leader_election.cc:304] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:22:18.860059 16221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8638ee363273426db1393dd81e5c7b72" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:18.827967 23645 raft_consensus.cc:2804] T 3884f864a96d428785895e5463e5728f P cb82cc7bd6db46d980c75237a1402860 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:22:18.837704 23398 leader_election.cc:290] T 8638ee363273426db1393dd81e5c7b72 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:18.846448 23559 raft_consensus.cc:3060] T 84f2da98864b4425bf7c0055951fdd4b P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:22:18.869493 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ab0ac8347641e6955e9bce7d90c1a2" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 21 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:18.878779 16223 raft_consensus.cc:1275] T 746aabf856ab401c9ef5e859371e7b67 P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 18 index: 15. (index mismatch)
I20260709 13:22:18.881107 23648 raft_consensus.cc:2804] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:22:18.863935 23399 raft_consensus.cc:2804] T 26354361e9b044cea1c2b86c4882fe92 P cb82cc7bd6db46d980c75237a1402860 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:22:18.857766 23625 raft_consensus.cc:2804] T 3884f864a96d428785895e5463e5728f P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:22:18.887336 16151 raft_consensus.cc:1240] T 2794af5b99984efebdee564035841cb0 P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 18. Current term is 19. Ops: [17.15-18.16]
W20260709 13:22:18.891927 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 45411.8us
I20260709 13:22:18.892975 16265 leader_election.cc:304] T 2794af5b99984efebdee564035841cb0 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:22:18.867954 23533 raft_consensus.cc:493] T b87147731a68400e8cce9b879daa7c5d P 90f3da57796b488c9087a2f523adf045 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:18.864187 23732 raft_consensus.cc:2804] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:22:18.931385 23551 consensus_queue.cc:1048] T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:22:18.932075 16296 raft_consensus.cc:1275] T 8d4ab870a05347868956aa8b841b2981 P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 13:22:18.844183 23620 consensus_queue.cc:1048] T 434bff3082d141838068f323bdb0d1ed P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.036s
I20260709 13:22:18.936993 16294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f5f89186204242bf377f7c42b2e2da" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:18.938474 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a9de347c711477e87c582d003f36925" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:18.940078 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e9b2bf7011f4ae688a79d7bb5a447b4" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:18.940734 16297 raft_consensus.cc:2468] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 19.
I20260709 13:22:18.941653 23710 raft_consensus.cc:515] T e4dc6b0541ca431ebc757beac00521f1 P 90f3da57796b488c9087a2f523adf045 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:18.944566 23720 raft_consensus.cc:515] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:18.897372 23737 raft_consensus.cc:493] T fffab26427e24d398e2695aad40ca134 P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:22:18.903296 16222 raft_consensus.cc:1275] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac [term 20 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 18 index: 15. Preceding OpId from leader: term: 20 index: 16. (index mismatch)
I20260709 13:22:18.945937 23737 raft_consensus.cc:515] T fffab26427e24d398e2695aad40ca134 P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:18.946810 23720 leader_election.cc:290] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 25 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:18.852133 23638 raft_consensus.cc:515] T b87147731a68400e8cce9b879daa7c5d P cb82cc7bd6db46d980c75237a1402860 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:18.973354 23638 leader_election.cc:290] T b87147731a68400e8cce9b879daa7c5d P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:18.919332 23398 consensus_queue.cc:1059] T 2794af5b99984efebdee564035841cb0 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: INVALID_TERM, Last received: 16.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:22:18.974309 23398 raft_consensus.cc:3055] T 2794af5b99984efebdee564035841cb0 P cb82cc7bd6db46d980c75237a1402860 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:22:18.974676 23398 raft_consensus.cc:740] T 2794af5b99984efebdee564035841cb0 P cb82cc7bd6db46d980c75237a1402860 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:22:18.889104 23399 raft_consensus.cc:493] T 26354361e9b044cea1c2b86c4882fe92 P cb82cc7bd6db46d980c75237a1402860 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:18.975500 23399 raft_consensus.cc:3060] T 26354361e9b044cea1c2b86c4882fe92 P cb82cc7bd6db46d980c75237a1402860 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:22:18.977329 16146 raft_consensus.cc:3060] T 746aabf856ab401c9ef5e859371e7b67 P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:22:18.977852 23733 raft_consensus.cc:493] T 5a959dbecc2648eb85d1cc550f5a3245 P 1ca2725d290e47d68b1a4b609c9816ac [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:22:18.881625 23023 raft_consensus.cc:493] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:22:18.979466 23023 raft_consensus.cc:515] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:18.923411 16225 raft_consensus.cc:1240] T 829b4ad8c5a54eababeea1181b960d4b P 1ca2725d290e47d68b1a4b609c9816ac [term 19 LEADER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 18. Current term is 19. Ops: []
I20260709 13:22:18.924350 16221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a96373bb0e14dce8a266b2fc6c58507" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:18.981875 16221 raft_consensus.cc:2468] T 5a96373bb0e14dce8a266b2fc6c58507 P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 17.
I20260709 13:22:18.984081 16151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87147731a68400e8cce9b879daa7c5d" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:22:18.924183 16040 catalog_manager.cc:5697] T 66f75124215f4ca6bef831e174a792d8 P cb82cc7bd6db46d980c75237a1402860 reported cstate change: term changed from 14 to 16. New cstate: current_term: 16 leader_uuid: "cb82cc7bd6db46d980c75237a1402860" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:22:18.875531 23645 raft_consensus.cc:493] T 3884f864a96d428785895e5463e5728f P cb82cc7bd6db46d980c75237a1402860 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:18.895957 23625 raft_consensus.cc:493] T 3884f864a96d428785895e5463e5728f P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:18.925328 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5ad4eb281564936a59f41e387a47156" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:18.984678 16151 raft_consensus.cc:2468] T b87147731a68400e8cce9b879daa7c5d P 647a343a542d4a139fe106c19095129b [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 21.
I20260709 13:22:18.985940 16300 raft_consensus.cc:2393] T e5ad4eb281564936a59f41e387a47156 P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 20: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
I20260709 13:22:18.889163 23648 raft_consensus.cc:493] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:18.926412 23732 raft_consensus.cc:493] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Starting leader election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:22:18.927248 23533 raft_consensus.cc:3060] T b87147731a68400e8cce9b879daa7c5d P 90f3da57796b488c9087a2f523adf045 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:22:18.948216 23737 leader_election.cc:290] T fffab26427e24d398e2695aad40ca134 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:18.949628 23685 raft_consensus.cc:3055] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P cb82cc7bd6db46d980c75237a1402860 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:22:18.949868 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4dc6b0541ca431ebc757beac00521f1" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:18.953284 23694 consensus_queue.cc:1048] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:22:18.953470 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806d7690c2b644959152074bb2cab0a7" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:18.954255 16299 raft_consensus.cc:2393] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 25: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
I20260709 13:22:18.903637 23745 consensus_queue.cc:237] T 9e65676b21b748728ad39668cfe722ec P 90f3da57796b488c9087a2f523adf045 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:18.955145 16222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02838c86531d4f98bf58d1c22b8baf77" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:22:18.955549 16219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5ad4eb281564936a59f41e387a47156" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:18.955857 22677 raft_consensus.cc:493] T 19baa7bc367844c288bca8f81439c413 P 1ca2725d290e47d68b1a4b609c9816ac [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:22:18.956362 23287 raft_consensus.cc:493] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:18.957194 16191 leader_election.cc:304] T 5a96373bb0e14dce8a266b2fc6c58507 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
I20260709 13:22:18.957374 16220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f5f89186204242bf377f7c42b2e2da" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:18.958536 16115 leader_election.cc:304] T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
W20260709 13:22:19.020438 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 63328.9us
W20260709 13:22:19.020761 16115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 62305.3us
I20260709 13:22:19.018854 16820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a9de347c711477e87c582d003f36925" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:19.020452 16220 raft_consensus.cc:2468] T c2f5f89186204242bf377f7c42b2e2da P 1ca2725d290e47d68b1a4b609c9816ac [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 23.
I20260709 13:22:18.959367 23605 raft_consensus.cc:493] T 5a96373bb0e14dce8a266b2fc6c58507 P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:18.959878 23558 raft_consensus.cc:493] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 90f3da57796b488c9087a2f523adf045 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:18.960351 23024 raft_consensus.cc:493] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:22:18.959874 23588 consensus_queue.cc:1048] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:22:18.960979 23306 raft_consensus.cc:2804] T 2794af5b99984efebdee564035841cb0 P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:22:18.962073 16295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6f07842d3aa47858df5665874c86f3a" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:18.963960 16824 raft_consensus.cc:1275] T ada5a004ba7c46819051e3ccec71ffdb P cb82cc7bd6db46d980c75237a1402860 [term 22 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 20 index: 15. Preceding OpId from leader: term: 22 index: 16. (index mismatch)
I20260709 13:22:18.965742 16823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9944d5e4b8c4109af9955c4f08c9095" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:22:19.037684 16823 raft_consensus.cc:3060] T c9944d5e4b8c4109af9955c4f08c9095 P cb82cc7bd6db46d980c75237a1402860 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:22:18.970117 22610 raft_consensus.cc:493] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 647a343a542d4a139fe106c19095129b [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:19.039638 22610 raft_consensus.cc:515] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 647a343a542d4a139fe106c19095129b [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:18.978545 23733 raft_consensus.cc:515] T 5a959dbecc2648eb85d1cc550f5a3245 P 1ca2725d290e47d68b1a4b609c9816ac [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:18.986714 16790 leader_election.cc:304] T b87147731a68400e8cce9b879daa7c5d P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:22:18.991303 16221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6f07842d3aa47858df5665874c86f3a" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:18.969841 23210 leader_election.cc:290] T fffab26427e24d398e2695aad40ca134 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:22:19.005964 16819 raft_consensus.cc:1240] T 829b4ad8c5a54eababeea1181b960d4b P cb82cc7bd6db46d980c75237a1402860 [term 19 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 18. Current term is 19. Ops: []
W20260709 13:22:18.999284 23716 log.cc:927] Time spent T 8d4ab870a05347868956aa8b841b2981 P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 0.580s	user 0.000s	sys 0.002s
W20260709 13:22:18.999876 23749 log.cc:927] Time spent T 829b4ad8c5a54eababeea1181b960d4b P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 0.052s	user 0.000s	sys 0.001s
I20260709 13:22:18.969983 23710 leader_election.cc:290] T e4dc6b0541ca431ebc757beac00521f1 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 22 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:22:18.990293 23645 raft_consensus.cc:3060] T 3884f864a96d428785895e5463e5728f P cb82cc7bd6db46d980c75237a1402860 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:22:19.025996 23737 raft_consensus.cc:493] T af57996a8be84d398f0f023e8d08ef77 P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:22:19.018810 16217 raft_consensus.cc:2393] T e4dc6b0541ca431ebc757beac00521f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 21: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
I20260709 13:22:18.998304 23625 raft_consensus.cc:3060] T 3884f864a96d428785895e5463e5728f P 90f3da57796b488c9087a2f523adf045 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:22:19.024034 16219 raft_consensus.cc:2393] T e5ad4eb281564936a59f41e387a47156 P 1ca2725d290e47d68b1a4b609c9816ac [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 20: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
I20260709 13:22:19.080148 16824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2794af5b99984efebdee564035841cb0" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 19 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:22:19.019516 16821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e9b2bf7011f4ae688a79d7bb5a447b4" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:19.024539 22677 raft_consensus.cc:515] T 19baa7bc367844c288bca8f81439c413 P 1ca2725d290e47d68b1a4b609c9816ac [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:18.999950 23732 raft_consensus.cc:3060] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:22:19.088033 16821 raft_consensus.cc:2468] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 19.
I20260709 13:22:19.031508 16146 raft_consensus.cc:1275] T 746aabf856ab401c9ef5e859371e7b67 P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 18 index: 15. (index mismatch)
I20260709 13:22:19.021828 16115 leader_election.cc:304] T 02838c86531d4f98bf58d1c22b8baf77 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
W20260709 13:22:19.090844 16115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 69088.2us
I20260709 13:22:19.091264 23357 raft_consensus.cc:3055] T 829b4ad8c5a54eababeea1181b960d4b P 647a343a542d4a139fe106c19095129b [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:22:19.091571 16115 leader_election.cc:304] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac
I20260709 13:22:19.091796 23357 raft_consensus.cc:740] T 829b4ad8c5a54eababeea1181b960d4b P 647a343a542d4a139fe106c19095129b [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:22:19.034410 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fffab26427e24d398e2695aad40ca134" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:19.092867 23357 consensus_queue.cc:260] T 829b4ad8c5a54eababeea1181b960d4b P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:19.094110 23357 raft_consensus.cc:3060] T 829b4ad8c5a54eababeea1181b960d4b P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:22:19.036221 23719 consensus_queue.cc:1048] T ecc1e363230043fe9fef390f0b979f20 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.042s
I20260709 13:22:19.009131 23685 raft_consensus.cc:740] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P cb82cc7bd6db46d980c75237a1402860 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:22:19.072361 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ad5dbfa99ee4b8d9e28be92b3f36db5" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:19.105000 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87ac0d1f7e8a419f9bcbc04aee0e067e" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:18.998406 23738 raft_consensus.cc:515] T 68a6838d82ef4cac83507f906139d90f P 90f3da57796b488c9087a2f523adf045 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:19.023983 23398 consensus_queue.cc:260] T 2794af5b99984efebdee564035841cb0 P cb82cc7bd6db46d980c75237a1402860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.16, Last appended by leader: 16, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:19.032018 23566 raft_consensus.cc:2804] T 5a96373bb0e14dce8a266b2fc6c58507 P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:22:19.032315 23606 raft_consensus.cc:2804] T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:22:19.033092 23605 raft_consensus.cc:3060] T 5a96373bb0e14dce8a266b2fc6c58507 P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:22:18.998193 23648 raft_consensus.cc:3060] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:22:19.078321 22610 leader_election.cc:290] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:19.032465 23023 leader_election.cc:290] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:19.104944 16819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57a744f5aaf944678a63469cbf514015" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:22:19.138876 16223 raft_consensus.cc:1275] T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac [term 20 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 13:22:19.073879 23306 raft_consensus.cc:697] T 2794af5b99984efebdee564035841cb0 P 90f3da57796b488c9087a2f523adf045 [term 19 LEADER]: Becoming Leader. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:22:19.033157 23287 raft_consensus.cc:515] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:19.128757 16789 leader_election.cc:304] T 9e65676b21b748728ad39668cfe722ec P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:22:19.038229 23024 raft_consensus.cc:515] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:19.070264 23630 consensus_queue.cc:1048] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.035s
I20260709 13:22:19.072606 23558 raft_consensus.cc:515] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 90f3da57796b488c9087a2f523adf045 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:19.147610 16819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a9de347c711477e87c582d003f36925" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:19.134119 23566 raft_consensus.cc:493] T 5a96373bb0e14dce8a266b2fc6c58507 P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:19.140432 22610 raft_consensus.cc:493] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:22:19.135339 16821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74ca8140f35c4ae29e8a664f6bdfc829" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:22:19.135607 23753 raft_consensus.cc:493] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:19.152148 16821 raft_consensus.cc:3060] T 74ca8140f35c4ae29e8a664f6bdfc829 P cb82cc7bd6db46d980c75237a1402860 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:22:19.136399 23756 raft_consensus.cc:2804] T 02838c86531d4f98bf58d1c22b8baf77 P 647a343a542d4a139fe106c19095129b [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:22:19.151971 22610 raft_consensus.cc:515] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:19.141012 23717 raft_consensus.cc:2804] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:22:19.136724 23606 raft_consensus.cc:493] T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Starting leader election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:22:19.159485 22677 leader_election.cc:290] T 19baa7bc367844c288bca8f81439c413 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:19.148324 23398 raft_consensus.cc:3060] T 2794af5b99984efebdee564035841cb0 P cb82cc7bd6db46d980c75237a1402860 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:22:19.095890 23450 raft_consensus.cc:515] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:19.170364 16819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87147731a68400e8cce9b879daa7c5d" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:19.167922 16116 leader_election.cc:304] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
I20260709 13:22:19.171443 16819 raft_consensus.cc:2468] T b87147731a68400e8cce9b879daa7c5d P cb82cc7bd6db46d980c75237a1402860 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 21.
I20260709 13:22:19.113732 16823 raft_consensus.cc:2468] T c9944d5e4b8c4109af9955c4f08c9095 P cb82cc7bd6db46d980c75237a1402860 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 23.
I20260709 13:22:19.131124 23737 raft_consensus.cc:515] T af57996a8be84d398f0f023e8d08ef77 P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:19.153115 23566 raft_consensus.cc:3060] T 5a96373bb0e14dce8a266b2fc6c58507 P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:22:19.181003 16789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 52362us
I20260709 13:22:19.186103 16192 leader_election.cc:304] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 
W20260709 13:22:19.175154 23708 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 18 current term: 18 is higher.
I20260709 13:22:19.197994 16300 raft_consensus.cc:1275] T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045 [term 22 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 22 index: 12. (index mismatch)
I20260709 13:22:19.198559 16819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3884f864a96d428785895e5463e5728f" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 17 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:19.169834 23606 raft_consensus.cc:3060] T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:22:19.159569 23717 raft_consensus.cc:493] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Starting leader election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:22:19.202986 22610 leader_election.cc:290] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:19.203619 16293 raft_consensus.cc:1275] T 7c2095a5f4ad49408dbc7f69c606120d P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 16 index: 8. Preceding OpId from leader: term: 18 index: 9. (index mismatch)
W20260709 13:22:19.191648 23598 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.749s	user 0.005s	sys 0.016s
W20260709 13:22:19.211392 23598 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.769s	user 0.005s	sys 0.017s
I20260709 13:22:19.214493 16297 raft_consensus.cc:1275] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 20 index: 13. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 13:22:19.194375 23730 raft_consensus.cc:493] T e5ad4eb281564936a59f41e387a47156 P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:19.239624 16291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5ad4eb281564936a59f41e387a47156" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:22:19.243468 16222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:19.201794 23719 raft_consensus.cc:2804] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:22:19.244275 16222 raft_consensus.cc:2468] T 19baa7bc367844c288bca8f81439c413 P 1ca2725d290e47d68b1a4b609c9816ac [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 23.
W20260709 13:22:19.217666 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 231014us
W20260709 13:22:19.247040 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 354151us
I20260709 13:22:19.195129 23756 raft_consensus.cc:697] T 02838c86531d4f98bf58d1c22b8baf77 P 647a343a542d4a139fe106c19095129b [term 22 LEADER]: Becoming Leader. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:22:19.160440 23753 raft_consensus.cc:515] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:19.207389 23717 raft_consensus.cc:3060] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:22:19.261619 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c2095a5f4ad49408dbc7f69c606120d" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 19 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:22:19.258548 16300 raft_consensus.cc:1240] T 02838c86531d4f98bf58d1c22b8baf77 P 90f3da57796b488c9087a2f523adf045 [term 22 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 21. Current term is 22. Ops: [21.12-21.12]
I20260709 13:22:19.259665 23719 raft_consensus.cc:493] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:19.712424 23287 leader_election.cc:290] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
W20260709 13:22:19.424508 16789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 234263us
I20260709 13:22:19.781946 16789 leader_election.cc:304] T 23f82224f4564fea93572e8206bad1fe P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:22:19.717504 16150 raft_consensus.cc:2468] T 7c2095a5f4ad49408dbc7f69c606120d P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 18.
I20260709 13:22:19.248102 23730 raft_consensus.cc:515] T e5ad4eb281564936a59f41e387a47156 P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:19.209735 23023 consensus_queue.cc:1048] T 8d4ab870a05347868956aa8b841b2981 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.275s
W20260709 13:22:19.097306 23735 consensus_queue.cc:1175] T 829b4ad8c5a54eababeea1181b960d4b P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "1ca2725d290e47d68b1a4b609c9816ac" responder_term: 19 status { last_received { term: 19 index: 15 } last_committed_idx: 14 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 18. Current term is 19. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:22:19.357359 23754 raft_consensus.cc:2804] T b87147731a68400e8cce9b879daa7c5d P cb82cc7bd6db46d980c75237a1402860 [term 21 FOLLOWER]: Leader pre-election won for term 22
W20260709 13:22:19.425362 16115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 332460us
W20260709 13:22:19.714119 23645 consensus_meta.cc:287] T 3884f864a96d428785895e5463e5728f P cb82cc7bd6db46d980c75237a1402860: Time spent flushing consensus metadata: real 0.628s	user 0.012s	sys 0.012s
W20260709 13:22:19.720293 16821 consensus_meta.cc:287] T 74ca8140f35c4ae29e8a664f6bdfc829 P cb82cc7bd6db46d980c75237a1402860: Time spent flushing consensus metadata: real 0.563s	user 0.007s	sys 0.004s
I20260709 13:22:19.807021 16294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 24 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:19.807276 16821 raft_consensus.cc:2468] T 74ca8140f35c4ae29e8a664f6bdfc829 P cb82cc7bd6db46d980c75237a1402860 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 23.
I20260709 13:22:19.243861 23306 consensus_queue.cc:237] T 2794af5b99984efebdee564035841cb0 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.15, Last appended by leader: 15, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:19.141683 16040 catalog_manager.cc:5697] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 reported cstate change: term changed from 12 to 24. New cstate: current_term: 24 leader_uuid: "cb82cc7bd6db46d980c75237a1402860" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:22:19.216182 23685 consensus_queue.cc:260] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P cb82cc7bd6db46d980c75237a1402860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 21.16, Last appended by leader: 16, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:19.311244 23559 raft_consensus.cc:515] T 84f2da98864b4425bf7c0055951fdd4b P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:19.716403 16147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fffab26427e24d398e2695aad40ca134" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
W20260709 13:22:19.722831 16825 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.581s	user 0.010s	sys 0.006s
W20260709 13:22:19.778927 16116 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:22:19.825073 16825 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.689s	user 0.011s	sys 0.006s
I20260709 13:22:19.799470 16295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a96373bb0e14dce8a266b2fc6c58507" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:19.827579 16295 raft_consensus.cc:2468] T 5a96373bb0e14dce8a266b2fc6c58507 P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 17.
I20260709 13:22:19.799674 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f2da98864b4425bf7c0055951fdd4b" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:22:19.800895 23733 leader_election.cc:290] T 5a959dbecc2648eb85d1cc550f5a3245 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:19.811796 16294 raft_consensus.cc:2468] T ecc1e363230043fe9fef390f0b979f20 P 90f3da57796b488c9087a2f523adf045 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 23.
I20260709 13:22:19.830294 16821 raft_consensus.cc:1240] T c9944d5e4b8c4109af9955c4f08c9095 P cb82cc7bd6db46d980c75237a1402860 [term 23 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 22. Current term is 23. Ops: []
I20260709 13:22:19.821179 16291 raft_consensus.cc:2376] T e5ad4eb281564936a59f41e387a47156 P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 20. Re-sending same reply.
I20260709 13:22:19.816035 16292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f5f89186204242bf377f7c42b2e2da" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 24 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
W20260709 13:22:19.813372 23758 log.cc:927] Time spent T 66f75124215f4ca6bef831e174a792d8 P cb82cc7bd6db46d980c75237a1402860: Append to log took a long time: real 0.625s	user 0.006s	sys 0.003s
W20260709 13:22:19.824733 16790 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.571s	user 0.015s	sys 0.016s
I20260709 13:22:19.824831 16147 raft_consensus.cc:2468] T fffab26427e24d398e2695aad40ca134 P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 20.
W20260709 13:22:19.786691 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 71686.6us
W20260709 13:22:19.858729 16790 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.605s	user 0.015s	sys 0.017s
I20260709 13:22:19.803045 23754 raft_consensus.cc:493] T b87147731a68400e8cce9b879daa7c5d P cb82cc7bd6db46d980c75237a1402860 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:22:19.822113 23747 log.cc:927] Time spent T d2ab0ac8347641e6955e9bce7d90c1a2 P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.737s	user 0.005s	sys 0.014s
I20260709 13:22:19.866240 16292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9944d5e4b8c4109af9955c4f08c9095" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:19.786862 23719 raft_consensus.cc:3060] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:22:19.866297 16040 catalog_manager.cc:5697] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 reported cstate change: term changed from 17 to 21. New cstate: current_term: 21 leader_uuid: "1ca2725d290e47d68b1a4b609c9816ac" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
W20260709 13:22:19.868832 23760 log.cc:927] Time spent T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 0.071s	user 0.000s	sys 0.000s
I20260709 13:22:19.868969 23287 raft_consensus.cc:493] T 8cc959fb7f704fba9009c0595467d074 P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:19.902029 23287 raft_consensus.cc:515] T 8cc959fb7f704fba9009c0595467d074 P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:19.903369 16144 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40830 (request call id 2198) took 1336 ms. Trace:
I20260709 13:22:19.904217 23559 leader_election.cc:290] T 84f2da98864b4425bf7c0055951fdd4b P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 18 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:22:19.904814 23559 raft_consensus.cc:493] T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:19.905308 23559 raft_consensus.cc:515] T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:19.907094 16295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68a6838d82ef4cac83507f906139d90f" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:19.907882 16295 raft_consensus.cc:2393] T 68a6838d82ef4cac83507f906139d90f P 90f3da57796b488c9087a2f523adf045 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 21: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
W20260709 13:22:19.867405 23606 consensus_meta.cc:287] T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b: Time spent flushing consensus metadata: real 0.628s	user 0.009s	sys 0.003s
I20260709 13:22:19.877456 23754 raft_consensus.cc:3060] T b87147731a68400e8cce9b879daa7c5d P cb82cc7bd6db46d980c75237a1402860 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:22:19.928375 23606 raft_consensus.cc:515] T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:19.930681 23606 leader_election.cc:290] T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 21 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:19.931543 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9944d5e4b8c4109af9955c4f08c9095" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:22:19.931548 23737 leader_election.cc:290] T af57996a8be84d398f0f023e8d08ef77 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:19.932844 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:19.933255 23737 raft_consensus.cc:493] T 8cc959fb7f704fba9009c0595467d074 P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:22:19.933909 23737 raft_consensus.cc:515] T 8cc959fb7f704fba9009c0595467d074 P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:19.934392 16298 raft_consensus.cc:1275] T c0eab25ce4c64e018a041beeda2867db P 90f3da57796b488c9087a2f523adf045 [term 22 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 22 index: 12. (index mismatch)
I20260709 13:22:19.935842 16292 raft_consensus.cc:1275] T ada5a004ba7c46819051e3ccec71ffdb P 90f3da57796b488c9087a2f523adf045 [term 22 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 20 index: 15. Preceding OpId from leader: term: 22 index: 16. (index mismatch)
I20260709 13:22:19.935983 23737 leader_election.cc:290] T 8cc959fb7f704fba9009c0595467d074 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:19.936643 23737 raft_consensus.cc:493] T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:22:19.936829 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4dc6b0541ca431ebc757beac00521f1" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:19.937080 23737 raft_consensus.cc:515] T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:19.937587 16297 raft_consensus.cc:2393] T e4dc6b0541ca431ebc757beac00521f1 P 90f3da57796b488c9087a2f523adf045 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 22: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
I20260709 13:22:19.903759 16144 rpcz_store.cc:276] 0709 13:22:18.566627 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:22:18.566734 (+   107us) service_pool.cc:224] Handling call
0709 13:22:18.583725 (+ 16991us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:22:18.790208 (+206483us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:22:18.790243 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:22:18.790758 (+   515us) log.cc:844] Serialized 26 byte log entry
0709 13:22:18.937021 (+146263us) spinlock_profiling.cc:243] Waited 29.3 ms on lock 0x7b180036c220. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b693ade1 00007f69b69475b1 00007f69ada54e07 00007f69b68afab9 00007f69ada7169b 00007f69ada6f93f 00007f69ada71c02 00007f69b06f920b 00007f69b06fed23 00007f69b06ff0d1 00007f69b0b1aa0f
0709 13:22:19.119063 (+182042us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:22:19.119124 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:22:19.903036 (+783912us) spinlock_profiling.cc:243] Waited 189 ms on lock 0x7b540012fa54. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:22:19.903089 (+    53us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:22:19.903146 (+    57us) raft_consensus.cc:1685] finished
0709 13:22:19.903183 (+    37us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:22:19.903310 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7282,"raft.queue_time_us":8534,"raft.run_cpu_time_us":114,"raft.run_wall_time_us":8585,"spinlock_wait_cycles":501114368,"thread_start_us":181780,"threads_started":1,"wal-append.queue_time_us":333052}
I20260709 13:22:19.835574 23735 raft_consensus.cc:493] T d16dfad0c3b540e5b4b86492ddf87208 P 647a343a542d4a139fe106c19095129b [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:22:19.871033 23398 consensus_meta.cc:287] T 2794af5b99984efebdee564035841cb0 P cb82cc7bd6db46d980c75237a1402860: Time spent flushing consensus metadata: real 0.684s	user 0.004s	sys 0.006s
I20260709 13:22:19.871039 23768 raft_consensus.cc:2749] T 9e65676b21b748728ad39668cfe722ec P cb82cc7bd6db46d980c75237a1402860 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:22:19.890848 23558 leader_election.cc:290] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:19.891389 23774 raft_consensus.cc:493] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [term 26 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
W20260709 13:22:19.893450 23533 consensus_meta.cc:287] T b87147731a68400e8cce9b879daa7c5d P 90f3da57796b488c9087a2f523adf045: Time spent flushing consensus metadata: real 0.883s	user 0.013s	sys 0.001s
I20260709 13:22:19.894230 23780 raft_consensus.cc:493] T 8638ee363273426db1393dd81e5c7b72 P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
W20260709 13:22:19.894743 23746 log.cc:927] Time spent T 57a744f5aaf944678a63469cbf514015 P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 1.129s	user 0.005s	sys 0.000s
I20260709 13:22:19.895601 23762 raft_consensus.cc:493] T 18ad7e3af5504571a69d10c00750a01c P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
W20260709 13:22:19.897192 23648 consensus_meta.cc:287] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860: Time spent flushing consensus metadata: real 0.749s	user 0.008s	sys 0.013s
I20260709 13:22:19.820003 23645 raft_consensus.cc:515] T 3884f864a96d428785895e5463e5728f P cb82cc7bd6db46d980c75237a1402860 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:19.900003 16040 catalog_manager.cc:5697] T ee9ed67006144797b1de6ea8c2013a68 P cb82cc7bd6db46d980c75237a1402860 reported cstate change: term changed from 16 to 19, leader changed from 647a343a542d4a139fe106c19095129b (127.14.177.129) to cb82cc7bd6db46d980c75237a1402860 (127.14.177.132). New cstate: current_term: 19 leader_uuid: "cb82cc7bd6db46d980c75237a1402860" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:22:19.901470 23450 leader_election.cc:290] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 22 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:22:19.938736 16294 raft_consensus.cc:1275] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045 [term 22 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 21 index: 16. Preceding OpId from leader: term: 22 index: 17. (index mismatch)
W20260709 13:22:19.940621 16115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 129297us
W20260709 13:22:19.955366 16192 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 769320us
I20260709 13:22:20.031472 23708 consensus_queue.cc:1048] T 8d4ab870a05347868956aa8b841b2981 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:22:20.032832 23384 raft_consensus.cc:493] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:22:20.033444 23384 raft_consensus.cc:515] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:20.034092 23730 leader_election.cc:290] T e5ad4eb281564936a59f41e387a47156 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:22:20.034698 23730 raft_consensus.cc:493] T c4f65cdf00b748bb963ee6e81ef43b79 P 90f3da57796b488c9087a2f523adf045 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:22:20.035188 23730 raft_consensus.cc:515] T c4f65cdf00b748bb963ee6e81ef43b79 P 90f3da57796b488c9087a2f523adf045 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:20.036309 23588 raft_consensus.cc:493] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:20.037334 16225 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57604 (request call id 3494) took 1083 ms. Trace:
I20260709 13:22:19.885174 23685 raft_consensus.cc:3060] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P cb82cc7bd6db46d980c75237a1402860 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:22:19.967072 16824 raft_consensus.cc:2410] T 2794af5b99984efebdee564035841cb0 P cb82cc7bd6db46d980c75237a1402860 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 for term 19 because replica has last-logged OpId of term: 18 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 15.
W20260709 13:22:19.969673 16789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 187860us
I20260709 13:22:19.956866 23778 raft_consensus.cc:493] T 806d7690c2b644959152074bb2cab0a7 P cb82cc7bd6db46d980c75237a1402860 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
W20260709 13:22:19.982407 23747 log.cc:927] Time spent T d2ab0ac8347641e6955e9bce7d90c1a2 P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.076s	user 0.000s	sys 0.000s
I20260709 13:22:20.047937 16789 leader_election.cc:304] T 74ca8140f35c4ae29e8a664f6bdfc829 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:22:19.966673 23024 leader_election.cc:290] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
W20260709 13:22:19.968186 23605 consensus_meta.cc:287] T 5a96373bb0e14dce8a266b2fc6c58507 P 647a343a542d4a139fe106c19095129b: Time spent flushing consensus metadata: real 0.823s	user 0.013s	sys 0.000s
I20260709 13:22:20.013206 23775 raft_consensus.cc:493] T 66f75124215f4ca6bef831e174a792d8 P cb82cc7bd6db46d980c75237a1402860 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:22:19.995922 23753 leader_election.cc:290] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:20.008617 23774 raft_consensus.cc:515] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:20.015331 23726 raft_consensus.cc:493] T 746aabf856ab401c9ef5e859371e7b67 P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:20.017751 23712 raft_consensus.cc:3055] T 23f82224f4564fea93572e8206bad1fe P 647a343a542d4a139fe106c19095129b [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:22:20.018718 23603 raft_consensus.cc:3055] T 84f2da98864b4425bf7c0055951fdd4b P 647a343a542d4a139fe106c19095129b [term 17 LEADER]: Stepping down as leader of term 17
W20260709 13:22:20.024766 23764 log.cc:927] Time spent T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.221s	user 0.000s	sys 0.000s
I20260709 13:22:20.082247 23778 raft_consensus.cc:515] T 806d7690c2b644959152074bb2cab0a7 P cb82cc7bd6db46d980c75237a1402860 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:22:20.100153 23357 consensus_meta.cc:287] T 829b4ad8c5a54eababeea1181b960d4b P 647a343a542d4a139fe106c19095129b: Time spent flushing consensus metadata: real 1.006s	user 0.013s	sys 0.003s
I20260709 13:22:20.100304 16220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f2da98864b4425bf7c0055951fdd4b" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:22:20.010514 23780 raft_consensus.cc:515] T 8638ee363273426db1393dd81e5c7b72 P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:20.101169 16220 raft_consensus.cc:2393] T 84f2da98864b4425bf7c0055951fdd4b P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 18: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
I20260709 13:22:20.101951 23778 leader_election.cc:290] T 806d7690c2b644959152074bb2cab0a7 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:20.102207 16220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cc959fb7f704fba9009c0595467d074" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:19.904265 23287 leader_election.cc:290] T 8cc959fb7f704fba9009c0595467d074 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:22:19.939762 23756 consensus_queue.cc:237] T 02838c86531d4f98bf58d1c22b8baf77 P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 21.12, Last appended by leader: 12, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:20.088617 23712 raft_consensus.cc:740] T 23f82224f4564fea93572e8206bad1fe P 647a343a542d4a139fe106c19095129b [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:22:20.104825 23712 consensus_queue.cc:260] T 23f82224f4564fea93572e8206bad1fe P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:20.106037 23712 raft_consensus.cc:3060] T 23f82224f4564fea93572e8206bad1fe P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:22:20.043049 23598 raft_consensus.cc:493] T b6f07842d3aa47858df5665874c86f3a P 1ca2725d290e47d68b1a4b609c9816ac [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:22:19.963449 23606 raft_consensus.cc:493] T ee9ed67006144797b1de6ea8c2013a68 P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:22:20.040342 16115 leader_election.cc:304] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:22:19.969481 23556 raft_consensus.cc:2804] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:22:20.112284 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4dc6b0541ca431ebc757beac00521f1" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
W20260709 13:22:20.112635 16789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 64827.1us
I20260709 13:22:20.112905 16226 raft_consensus.cc:3060] T e4dc6b0541ca431ebc757beac00521f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:22:20.013451 23559 leader_election.cc:290] T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:22:20.051411 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fffab26427e24d398e2695aad40ca134" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:20.082949 16218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c2095a5f4ad49408dbc7f69c606120d" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 19 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:20.083774 16819 raft_consensus.cc:2393] T 3884f864a96d428785895e5463e5728f P cb82cc7bd6db46d980c75237a1402860 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 17: Already voted for candidate cb82cc7bd6db46d980c75237a1402860 in this term.
I20260709 13:22:20.083752 16294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a959dbecc2648eb85d1cc550f5a3245" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:19.962236 23735 raft_consensus.cc:515] T d16dfad0c3b540e5b4b86492ddf87208 P 647a343a542d4a139fe106c19095129b [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:20.085235 23769 raft_consensus.cc:2749] T 23f82224f4564fea93572e8206bad1fe P cb82cc7bd6db46d980c75237a1402860 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:22:20.038508 23648 raft_consensus.cc:515] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:20.091926 23726 raft_consensus.cc:515] T 746aabf856ab401c9ef5e859371e7b67 P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:20.094969 23614 raft_consensus.cc:493] T ee9ed67006144797b1de6ea8c2013a68 P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:22:20.095345 23767 consensus_queue.cc:1048] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.086s
I20260709 13:22:20.096298 16816 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49896 (request call id 1644) took 1090 ms. Trace:
I20260709 13:22:20.133766 16218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7ca61592e848ea9b0fb81a2ce33cdc" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 22 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:20.046043 23588 raft_consensus.cc:515] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:20.102869 16220 raft_consensus.cc:2468] T 8cc959fb7f704fba9009c0595467d074 P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 19.
I20260709 13:22:20.121054 23357 raft_consensus.cc:2804] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:22:20.041548 16221 raft_consensus.cc:1275] T c2f5f89186204242bf377f7c42b2e2da P 1ca2725d290e47d68b1a4b609c9816ac [term 23 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 20 index: 12. Preceding OpId from leader: term: 23 index: 13. (index mismatch)
I20260709 13:22:20.106866 23738 leader_election.cc:290] T 68a6838d82ef4cac83507f906139d90f P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 21 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
W20260709 13:22:20.102918 23771 log.cc:927] Time spent T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 0.278s	user 0.001s	sys 0.001s
W20260709 13:22:20.111049 16115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 70814.4us
I20260709 13:22:20.133999 16816 rpcz_store.cc:276] 0709 13:22:19.005642 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:22:19.005749 (+   107us) service_pool.cc:224] Handling call
0709 13:22:19.019614 (+ 13865us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:22:19.207720 (+188106us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:22:19.207785 (+    65us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:22:19.302157 (+ 94372us) log.cc:844] Serialized 52 byte log entry
0709 13:22:19.858757 (+556600us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:22:19.858796 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:22:19.858876 (+    80us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:22:20.096038 (+237162us) raft_consensus.cc:1685] finished
0709 13:22:20.096060 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:22:20.096237 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":130536,"spinlock_wait_cycles":9131136,"thread_start_us":317356,"threads_started":2,"wal-append.queue_time_us":281114}
I20260709 13:22:20.126545 23398 raft_consensus.cc:493] T ada5a004ba7c46819051e3ccec71ffdb P cb82cc7bd6db46d980c75237a1402860 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:20.119199 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ad5dbfa99ee4b8d9e28be92b3f36db5" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:20.006760 23533 raft_consensus.cc:515] T b87147731a68400e8cce9b879daa7c5d P 90f3da57796b488c9087a2f523adf045 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:20.086493 23024 raft_consensus.cc:493] T ada5a004ba7c46819051e3ccec71ffdb P 90f3da57796b488c9087a2f523adf045 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:20.083096 23306 raft_consensus.cc:493] T 8a878999c0024d2abe517484180290fc P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:22:20.092015 23603 raft_consensus.cc:740] T 84f2da98864b4425bf7c0055951fdd4b P 647a343a542d4a139fe106c19095129b [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:22:20.194134 16220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c61a39fd4f84424b129c1a6f9ba404d" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 23 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:20.108282 23753 raft_consensus.cc:2749] T 74ca8140f35c4ae29e8a664f6bdfc829 P cb82cc7bd6db46d980c75237a1402860 [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:22:20.098259 23774 leader_election.cc:290] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
W20260709 13:22:20.120813 23625 consensus_meta.cc:287] T 3884f864a96d428785895e5463e5728f P 90f3da57796b488c9087a2f523adf045: Time spent flushing consensus metadata: real 0.988s	user 0.015s	sys 0.032s
I20260709 13:22:20.102985 23780 leader_election.cc:290] T 8638ee363273426db1393dd81e5c7b72 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
W20260709 13:22:20.123396 23732 consensus_meta.cc:287] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b: Time spent flushing consensus metadata: real 0.982s	user 0.009s	sys 0.039s
I20260709 13:22:20.082835 23605 raft_consensus.cc:515] T 5a96373bb0e14dce8a266b2fc6c58507 P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:22:20.123283 23399 consensus_meta.cc:287] T 26354361e9b044cea1c2b86c4882fe92 P cb82cc7bd6db46d980c75237a1402860: Time spent flushing consensus metadata: real 1.147s	user 0.013s	sys 0.006s
W20260709 13:22:20.108453 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 248514us
W20260709 13:22:20.123363 23566 consensus_meta.cc:287] T 5a96373bb0e14dce8a266b2fc6c58507 P 1ca2725d290e47d68b1a4b609c9816ac: Time spent flushing consensus metadata: real 0.914s	user 0.008s	sys 0.004s
W20260709 13:22:20.127154 23717 consensus_meta.cc:287] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b: Time spent flushing consensus metadata: real 0.732s	user 0.012s	sys 0.000s
I20260709 13:22:20.183979 16820 raft_consensus.cc:2393] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 25: Already voted for candidate cb82cc7bd6db46d980c75237a1402860 in this term.
I20260709 13:22:20.185624 16115 leader_election.cc:304] T e5ad4eb281564936a59f41e387a47156 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:22:20.185969 16221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8638ee363273426db1393dd81e5c7b72" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:20.116685 16266 leader_election.cc:304] T 57a744f5aaf944678a63469cbf514015 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
W20260709 13:22:20.314836 16266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 198215us
I20260709 13:22:20.318536 16266 leader_election.cc:304] T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac
I20260709 13:22:20.098913 23719 raft_consensus.cc:515] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:20.122771 23450 raft_consensus.cc:493] T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:22:20.124195 23556 raft_consensus.cc:697] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac [term 23 LEADER]: Becoming Leader. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:22:20.124284 23023 raft_consensus.cc:493] T 5a959dbecc2648eb85d1cc550f5a3245 P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:22:20.043231 23762 raft_consensus.cc:515] T 18ad7e3af5504571a69d10c00750a01c P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } }
I20260709 13:22:20.203696 16824 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49892 (request call id 1125) took 1191 ms. Trace:
I20260709 13:22:20.211762 16222 raft_consensus.cc:1275] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [term 24 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 22 index: 17. Preceding OpId from leader: term: 24 index: 18. (index mismatch)
I20260709 13:22:20.088284 23775 raft_consensus.cc:515] T 66f75124215f4ca6bef831e174a792d8 P cb82cc7bd6db46d980c75237a1402860 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:20.213712 16151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a9de347c711477e87c582d003f36925" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:22:20.217145 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68a6838d82ef4cac83507f906139d90f" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:22:20.122287 23694 consensus_queue.cc:1059] T 57a744f5aaf944678a63469cbf514015 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.064s
I20260709 13:22:20.220611 16817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:20.182292 23357 raft_consensus.cc:493] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:20.173079 23737 leader_election.cc:290] T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:20.221400 16218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5ad4eb281564936a59f41e387a47156" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:20.118589 23598 raft_consensus.cc:515] T b6f07842d3aa47858df5665874c86f3a P 1ca2725d290e47d68b1a4b609c9816ac [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:20.181416 23730 leader_election.cc:290] T c4f65cdf00b748bb963ee6e81ef43b79 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:22:20.173157 23384 leader_election.cc:290] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:20.212998 23398 raft_consensus.cc:515] T ada5a004ba7c46819051e3ccec71ffdb P cb82cc7bd6db46d980c75237a1402860 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:20.250821 23551 raft_consensus.cc:493] T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:20.252017 23780 raft_consensus.cc:493] T 8d4ab870a05347868956aa8b841b2981 P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:20.259261 23399 raft_consensus.cc:515] T 26354361e9b044cea1c2b86c4882fe92 P cb82cc7bd6db46d980c75237a1402860 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:20.272418 16816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a9de347c711477e87c582d003f36925" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:22:20.225512 23732 raft_consensus.cc:515] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:20.220800 23625 raft_consensus.cc:515] T 3884f864a96d428785895e5463e5728f P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:20.221084 23024 raft_consensus.cc:515] T ada5a004ba7c46819051e3ccec71ffdb P 90f3da57796b488c9087a2f523adf045 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:20.221402 23774 raft_consensus.cc:493] T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:22:20.211726 23776 raft_consensus.cc:493] T af57996a8be84d398f0f023e8d08ef77 P cb82cc7bd6db46d980c75237a1402860 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:22:20.251458 23785 raft_consensus.cc:493] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:20.135761 23606 raft_consensus.cc:515] T ee9ed67006144797b1de6ea8c2013a68 P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:20.403584 23606 leader_election.cc:290] T ee9ed67006144797b1de6ea8c2013a68 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:20.296130 16226 raft_consensus.cc:2468] T e4dc6b0541ca431ebc757beac00521f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 22.
I20260709 13:22:20.302168 16144 raft_consensus.cc:1275] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 647a343a542d4a139fe106c19095129b [term 21 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 20 index: 13. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
I20260709 13:22:20.302591 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4dc6b0541ca431ebc757beac00521f1" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:22:20.404476 23598 leader_election.cc:290] T b6f07842d3aa47858df5665874c86f3a P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:20.404896 16149 raft_consensus.cc:3060] T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:22:20.405344 16144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f2da98864b4425bf7c0055951fdd4b" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:22:20.180572 23614 raft_consensus.cc:515] T ee9ed67006144797b1de6ea8c2013a68 P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:20.407009 16219 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57604 (request call id 3492) took 1453 ms. Trace:
I20260709 13:22:20.407614 23614 leader_election.cc:290] T ee9ed67006144797b1de6ea8c2013a68 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:20.310761 16145 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46224 (request call id 1272) took 1895 ms. Trace:
I20260709 13:22:20.408222 23762 leader_election.cc:290] T 18ad7e3af5504571a69d10c00750a01c P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:22:20.124779 23778 consensus_queue.cc:1048] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.063s
I20260709 13:22:20.409132 23735 leader_election.cc:290] T d16dfad0c3b540e5b4b86492ddf87208 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:20.409294 23775 leader_election.cc:290] T 66f75124215f4ca6bef831e174a792d8 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:22:20.321966 16148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ad5dbfa99ee4b8d9e28be92b3f36db5" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:22:20.380699 23556 consensus_queue.cc:237] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 22.16, Last appended by leader: 16, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:20.410175 23533 leader_election.cc:290] T b87147731a68400e8cce9b879daa7c5d P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 22 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:20.410445 16148 raft_consensus.cc:2468] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 19.
I20260709 13:22:20.410799 23761 raft_consensus.cc:493] T 02838c86531d4f98bf58d1c22b8baf77 P 90f3da57796b488c9087a2f523adf045 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:20.411509 16148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c61a39fd4f84424b129c1a6f9ba404d" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 23 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:22:20.411374 23761 raft_consensus.cc:515] T 02838c86531d4f98bf58d1c22b8baf77 P 90f3da57796b488c9087a2f523adf045 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:20.412114 16148 raft_consensus.cc:2468] T 9c61a39fd4f84424b129c1a6f9ba404d P 647a343a542d4a139fe106c19095129b [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 22.
I20260709 13:22:20.371059 23694 consensus_queue.cc:1243] T 57a744f5aaf944678a63469cbf514015 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Peer 90f3da57796b488c9087a2f523adf045 log is divergent from this leader: its last log entry 17.14 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 13
I20260709 13:22:20.328359 23450 raft_consensus.cc:515] T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:20.412928 16149 raft_consensus.cc:2468] T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 22.
I20260709 13:22:20.413142 16148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cc959fb7f704fba9009c0595467d074" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:22:20.413750 16148 raft_consensus.cc:2468] T 8cc959fb7f704fba9009c0595467d074 P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 18.
I20260709 13:22:20.413703 23761 leader_election.cc:290] T 02838c86531d4f98bf58d1c22b8baf77 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:22:20.413928 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5ad4eb281564936a59f41e387a47156" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:22:20.349552 23794 raft_consensus.cc:3060] T e5ad4eb281564936a59f41e387a47156 P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:22:20.414413 23450 leader_election.cc:290] T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:22:20.414673 16148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4f65cdf00b748bb963ee6e81ef43b79" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
W20260709 13:22:20.323622 16115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 138095us
I20260709 13:22:20.386264 23551 raft_consensus.cc:515] T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:20.415630 16115 leader_election.cc:304] T b6f07842d3aa47858df5665874c86f3a P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:22:20.325083 16147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e9b2bf7011f4ae688a79d7bb5a447b4" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
W20260709 13:22:20.310761 23782 log.cc:927] Time spent T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 0.098s	user 0.001s	sys 0.000s
I20260709 13:22:20.213593 23306 raft_consensus.cc:515] T 8a878999c0024d2abe517484180290fc P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:22:20.335819 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 47811.4us
I20260709 13:22:20.417641 16817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e9b2bf7011f4ae688a79d7bb5a447b4" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:20.418241 16817 raft_consensus.cc:2468] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 19.
I20260709 13:22:20.418828 23306 leader_election.cc:290] T 8a878999c0024d2abe517484180290fc P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:22:20.419030 16817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8638ee363273426db1393dd81e5c7b72" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:20.419356 23306 raft_consensus.cc:493] T a378d7f796f94615b3b347119a9880fd P 90f3da57796b488c9087a2f523adf045 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:22:20.419855 23306 raft_consensus.cc:515] T a378d7f796f94615b3b347119a9880fd P 90f3da57796b488c9087a2f523adf045 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:20.420156 16817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7ca61592e848ea9b0fb81a2ce33cdc" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 22 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:20.337824 16151 raft_consensus.cc:3060] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:22:20.337867 16217 raft_consensus.cc:3060] T 68a6838d82ef4cac83507f906139d90f P 1ca2725d290e47d68b1a4b609c9816ac [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:22:20.421566 16788 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.541s	user 0.018s	sys 0.063s
W20260709 13:22:20.421897 16788 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.672s	user 0.018s	sys 0.063s
I20260709 13:22:20.422255 23790 raft_consensus.cc:493] T 08c74490c40243a88e62d1760b5cd814 P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:22:20.422755 23790 raft_consensus.cc:515] T 08c74490c40243a88e62d1760b5cd814 P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:20.424810 23790 leader_election.cc:290] T 08c74490c40243a88e62d1760b5cd814 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
W20260709 13:22:20.425799 23791 log.cc:927] Time spent T 2794af5b99984efebdee564035841cb0 P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.131s	user 0.000s	sys 0.000s
I20260709 13:22:20.309983 23708 raft_consensus.cc:493] T 434bff3082d141838068f323bdb0d1ed P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:22:20.332369 16824 rpcz_store.cc:276] 0709 13:22:19.012766 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:22:19.069449 (+ 56683us) service_pool.cc:224] Handling call
0709 13:22:20.192577 (+1123128us) spinlock_profiling.cc:243] Waited 882 ms on lock 0x7b54007201d4. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:22:20.203600 (+ 11023us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2028449664}
I20260709 13:22:20.342314 16146 raft_consensus.cc:1240] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b [term 24 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 23. Current term is 24. Ops: []
I20260709 13:22:20.321628 23287 raft_consensus.cc:493] T 8158bbd17b994d2eb96f560bd0f1b88d P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:22:20.328081 23210 raft_consensus.cc:2749] T 57a744f5aaf944678a63469cbf514015 P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:22:20.372112 16218 raft_consensus.cc:2468] T e5ad4eb281564936a59f41e387a47156 P 1ca2725d290e47d68b1a4b609c9816ac [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 20.
W20260709 13:22:20.379792 23757 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.513s	user 0.007s	sys 0.020s
I20260709 13:22:20.342865 23726 leader_election.cc:290] T 746aabf856ab401c9ef5e859371e7b67 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:20.346596 23645 leader_election.cc:290] T 3884f864a96d428785895e5463e5728f P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 17 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:20.350790 23357 raft_consensus.cc:3060] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:22:20.396724 23784 consensus_queue.cc:1059] T c9944d5e4b8c4109af9955c4f08c9095 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: INVALID_TERM, Last received: 22.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 13:22:20.327688 23566 raft_consensus.cc:515] T 5a96373bb0e14dce8a266b2fc6c58507 P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:20.401394 23774 raft_consensus.cc:515] T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:20.417119 23551 leader_election.cc:290] T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:20.427305 16221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5718c623ebee4d2bb770f0f9ddb2f164" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:22:20.615422 16221 raft_consensus.cc:3060] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:22:20.342223 23767 consensus_queue.cc:1048] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.049s
I20260709 13:22:20.432471 16217 raft_consensus.cc:2468] T 68a6838d82ef4cac83507f906139d90f P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 21.
I20260709 13:22:20.622869 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5d59407eb545b895d8d922e0dd69fd" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:20.361578 23603 consensus_queue.cc:260] T 84f2da98864b4425bf7c0055951fdd4b P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:20.246031 16040 catalog_manager.cc:5697] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860 reported cstate change: term changed from 15 to 21. New cstate: current_term: 21 leader_uuid: "cb82cc7bd6db46d980c75237a1402860" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:22:20.517565 16824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af57996a8be84d398f0f023e8d08ef77" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:20.388299 23730 raft_consensus.cc:2804] T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:22:20.339202 23023 raft_consensus.cc:515] T 5a959dbecc2648eb85d1cc550f5a3245 P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:20.517503 16823 raft_consensus.cc:1240] T 26354361e9b044cea1c2b86c4882fe92 P cb82cc7bd6db46d980c75237a1402860 [term 18 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 17. Current term is 18. Ops: []
I20260709 13:22:20.408330 16145 rpcz_store.cc:276] 0709 13:22:18.415387 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:22:18.421045 (+  5658us) service_pool.cc:224] Handling call
0709 13:22:18.434287 (+ 13242us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:22:18.508336 (+ 74049us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 13:22:18.594662 (+ 86326us) log.cc:844] Serialized 14 byte log entry
0709 13:22:18.765549 (+170887us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:22:18.766255 (+   706us) log.cc:844] Serialized 52 byte log entry
0709 13:22:18.766398 (+   143us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:22:18.766613 (+   215us) log.cc:844] Serialized 14 byte log entry
0709 13:22:18.766772 (+   159us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:22:18.766842 (+    70us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:22:18.910222 (+143380us) spinlock_profiling.cc:243] Waited 1.44 ms on lock 0x7b180032e720. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b693ade1 00007f69b69475b1 00007f69ada72522 00007f69ada747c1 00007f69ada74779 00007f69ada74709 00007f69ada746d1 00007f69ada739cd 000000000038f354 000000000038f199 00007f69ada5583a
0709 13:22:19.800335 (+890113us) spinlock_profiling.cc:243] Waited 2.82 ms on lock 0x7b4400022b4c. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69ad8fadb1 00007f69ad998c85 00007f69ad9996a2 00007f69b070b1db 00007f69b0706aaf 00007f69b06fa86e 00007f69b06f9bb6 00007f69b06f97d8 00007f69b0703221 00007f69b07031d9 00007f69b0703169
0709 13:22:20.310512 (+510177us) raft_consensus.cc:1685] finished
0709 13:22:20.310544 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:22:20.310685 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":11407,"prepare.queue_time_us":212382,"prepare.run_cpu_time_us":100,"prepare.run_wall_time_us":76,"raft.queue_time_us":33792,"raft.run_cpu_time_us":347,"raft.run_wall_time_us":7047,"spinlock_wait_cycles":9813120,"thread_start_us":121590,"threads_started":2,"wal-append.queue_time_us":86221}
W20260709 13:22:20.523396 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 92633.3us
W20260709 13:22:20.527508 23688 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.500s	user 0.017s	sys 0.039s
W20260709 13:22:20.543880 16116 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:22:20.526916 23630 raft_consensus.cc:493] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [term 26 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:22:20.555959 23588 leader_election.cc:290] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:20.431435 23785 raft_consensus.cc:515] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:22:20.538075 16117 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 103526us
I20260709 13:22:20.517175 23708 raft_consensus.cc:515] T 434bff3082d141838068f323bdb0d1ed P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:20.431704 23776 raft_consensus.cc:515] T af57996a8be84d398f0f023e8d08ef77 P cb82cc7bd6db46d980c75237a1402860 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:20.579548 23789 raft_consensus.cc:493] T 3a7ca61592e848ea9b0fb81a2ce33cdc P cb82cc7bd6db46d980c75237a1402860 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:22:20.581498 16825 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49892 (request call id 1127) took 1568 ms. Trace:
I20260709 13:22:20.582204 23777 raft_consensus.cc:493] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
W20260709 13:22:20.526233 16192 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 343795us
I20260709 13:22:20.582970 23735 raft_consensus.cc:2749] T b6f07842d3aa47858df5665874c86f3a P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:22:20.323582 23717 raft_consensus.cc:515] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b [term 21 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } }
I20260709 13:22:20.373019 23754 raft_consensus.cc:515] T b87147731a68400e8cce9b879daa7c5d P cb82cc7bd6db46d980c75237a1402860 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:22:20.588533 16115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 172960us
I20260709 13:22:20.608441 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806d7690c2b644959152074bb2cab0a7" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:20.431815 16148 raft_consensus.cc:2468] T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 21.
I20260709 13:22:20.387501 23780 raft_consensus.cc:515] T 8d4ab870a05347868956aa8b841b2981 P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:20.407438 16219 rpcz_store.cc:276] 0709 13:22:18.953727 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:22:19.097510 (+143783us) service_pool.cc:224] Handling call
0709 13:22:19.130704 (+ 33194us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:22:19.904444 (+773740us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:22:19.904477 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:22:19.904985 (+   508us) log.cc:844] Serialized 26 byte log entry
0709 13:22:20.186341 (+281356us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:22:20.186399 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:22:20.186938 (+   539us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:22:20.406818 (+219880us) raft_consensus.cc:1685] finished
0709 13:22:20.406839 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:22:20.406955 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":773348,"raft.queue_time_us":805872,"raft.run_cpu_time_us":78,"raft.run_wall_time_us":78,"thread_start_us":280941,"threads_started":1,"wal-append.queue_time_us":500086}
I20260709 13:22:20.624948 16223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66f75124215f4ca6bef831e174a792d8" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
W20260709 13:22:20.587839 23757 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.772s	user 0.007s	sys 0.020s
I20260709 13:22:20.414438 23761 raft_consensus.cc:493] T c0eab25ce4c64e018a041beeda2867db P 90f3da57796b488c9087a2f523adf045 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:20.574635 23793 raft_consensus.cc:493] T 2794af5b99984efebdee564035841cb0 P cb82cc7bd6db46d980c75237a1402860 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:20.654693 23533 raft_consensus.cc:493] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:22:20.611901 23756 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.506s	user 0.040s	sys 0.036s
W20260709 13:22:20.629922 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 288821us
W20260709 13:22:20.632900 16266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 300403us
I20260709 13:22:20.621645 23384 raft_consensus.cc:493] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:22:20.650631 16291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ab0ac8347641e6955e9bce7d90c1a2" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 21 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:20.652554 16292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e9b2bf7011f4ae688a79d7bb5a447b4" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:20.725327 16292 raft_consensus.cc:2468] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 19.
I20260709 13:22:20.649284 16265 leader_election.cc:304] T 7c2095a5f4ad49408dbc7f69c606120d P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:22:20.664685 16294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8638ee363273426db1393dd81e5c7b72" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:20.658774 16825 rpcz_store.cc:276] 0709 13:22:19.013333 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:22:19.869769 (+856436us) service_pool.cc:224] Handling call
0709 13:22:19.870122 (+   353us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:22:19.937409 (+ 67287us) raft_consensus.cc:1534] Early marking committed up to index 14
0709 13:22:19.937441 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:22:19.937940 (+   499us) log.cc:844] Serialized 26 byte log entry
0709 13:22:20.305192 (+367252us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:22:20.305254 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:22:20.305350 (+    96us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:22:20.581256 (+275906us) raft_consensus.cc:1685] finished
0709 13:22:20.581274 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:22:20.581426 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":66860,"raft.queue_time_us":143770,"raft.run_cpu_time_us":94,"raft.run_wall_time_us":94,"thread_start_us":366854,"threads_started":1,"wal-append.queue_time_us":641391}
I20260709 13:22:20.527207 23287 raft_consensus.cc:515] T 8158bbd17b994d2eb96f560bd0f1b88d P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:20.644460 23737 raft_consensus.cc:493] T 746aabf856ab401c9ef5e859371e7b67 P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:20.697908 16824 raft_consensus.cc:2468] T af57996a8be84d398f0f023e8d08ef77 P cb82cc7bd6db46d980c75237a1402860 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 20.
I20260709 13:22:20.644544 23730 raft_consensus.cc:697] T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045 [term 23 LEADER]: Becoming Leader. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:22:20.705590 16221 raft_consensus.cc:2468] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 22.
I20260709 13:22:20.754144 16221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62fe3346cd3c42c4a55e14af9073bca4" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 27 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
W20260709 13:22:20.555404 16789 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.14.177.131:41389 timed out after 1.540s (SENT) [suppressed 19 similar messages]
I20260709 13:22:20.649559 23306 leader_election.cc:290] T a378d7f796f94615b3b347119a9880fd P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:20.648588 23605 leader_election.cc:290] T 5a96373bb0e14dce8a266b2fc6c58507 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 18 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:20.710130 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a9de347c711477e87c582d003f36925" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:20.713351 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4f65cdf00b748bb963ee6e81ef43b79" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:20.758262 16219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af57996a8be84d398f0f023e8d08ef77" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
W20260709 13:22:20.730904 23756 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.617s	user 0.041s	sys 0.036s
I20260709 13:22:20.711786 23757 raft_consensus.cc:493] T 829b4ad8c5a54eababeea1181b960d4b P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:20.661146 23778 raft_consensus.cc:493] T 434bff3082d141838068f323bdb0d1ed P cb82cc7bd6db46d980c75237a1402860 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:22:20.733696 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4dc6b0541ca431ebc757beac00521f1" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:22:20.741355 16223 raft_consensus.cc:1240] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac [term 23 LEADER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 22. Current term is 23. Ops: []
W20260709 13:22:20.741755 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 92518.5us
I20260709 13:22:20.751775 16291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3884f864a96d428785895e5463e5728f" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 17 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:22:20.709990 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87147731a68400e8cce9b879daa7c5d" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:20.772349 16291 raft_consensus.cc:2393] T 3884f864a96d428785895e5463e5728f P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 in current term 17: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
I20260709 13:22:20.772969 16217 raft_consensus.cc:2368] T e4dc6b0541ca431ebc757beac00521f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b for earlier term 21. Current term is 22.
I20260709 13:22:20.773805 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a878999c0024d2abe517484180290fc" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:20.774353 16217 raft_consensus.cc:2468] T 8a878999c0024d2abe517484180290fc P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 18.
I20260709 13:22:20.775095 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d16dfad0c3b540e5b4b86492ddf87208" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:20.757318 16151 raft_consensus.cc:2468] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 25.
I20260709 13:22:20.760784 16825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee9ed67006144797b1de6ea8c2013a68" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:20.711673 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806d7690c2b644959152074bb2cab0a7" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:20.778038 16298 raft_consensus.cc:2468] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 20.
I20260709 13:22:20.782617 16297 raft_consensus.cc:2393] T b87147731a68400e8cce9b879daa7c5d P 90f3da57796b488c9087a2f523adf045 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 in current term 22: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
I20260709 13:22:20.771507 16265 leader_election.cc:304] T fffab26427e24d398e2695aad40ca134 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:22:20.766273 16221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4f65cdf00b748bb963ee6e81ef43b79" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:20.661140 23630 raft_consensus.cc:515] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:20.790715 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02838c86531d4f98bf58d1c22b8baf77" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 23 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:20.735952 23784 raft_consensus.cc:493] T 23f82224f4564fea93572e8206bad1fe P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
W20260709 13:22:20.760205 16115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 65000.5us
I20260709 13:22:20.752924 23710 raft_consensus.cc:493] T 66ae176961b74813a58bc15c9c4c6f2a P 90f3da57796b488c9087a2f523adf045 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:22:20.802414 16190 leader_election.cc:304] T ecc1e363230043fe9fef390f0b979f20 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 647a343a542d4a139fe106c19095129b
I20260709 13:22:20.762566 23719 leader_election.cc:290] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 24 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:20.729426 23533 raft_consensus.cc:515] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:20.667835 23789 raft_consensus.cc:515] T 3a7ca61592e848ea9b0fb81a2ce33cdc P cb82cc7bd6db46d980c75237a1402860 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:20.787087 23306 raft_consensus.cc:2804] T 7c2095a5f4ad49408dbc7f69c606120d P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Leader pre-election won for term 19
W20260709 13:22:20.836973 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 55310.5us
I20260709 13:22:20.812903 16825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62fe3346cd3c42c4a55e14af9073bca4" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 27 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:20.816323 16223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cc959fb7f704fba9009c0595467d074" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:20.838853 16223 raft_consensus.cc:2468] T 8cc959fb7f704fba9009c0595467d074 P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 19.
I20260709 13:22:20.817189 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a378d7f796f94615b3b347119a9880fd" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:20.694490 23777 raft_consensus.cc:515] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:20.817785 16219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18ad7e3af5504571a69d10c00750a01c" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
W20260709 13:22:20.824172 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 89379.4us
I20260709 13:22:20.792235 23732 leader_election.cc:290] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 21 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:20.844492 16116 leader_election.cc:400] T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 21 election: Vote denied by peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943) with higher term. Message: Invalid argument: T e4dc6b0541ca431ebc757beac00521f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b for earlier term 21. Current term is 22.
I20260709 13:22:20.797686 23398 leader_election.cc:290] T ada5a004ba7c46819051e3ccec71ffdb P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:20.844882 16116 leader_election.cc:403] T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 21 election: Cancelling election due to peer responding with higher term
I20260709 13:22:20.729868 23793 raft_consensus.cc:515] T 2794af5b99984efebdee564035841cb0 P cb82cc7bd6db46d980c75237a1402860 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:20.832538 16190 leader_election.cc:304] T 84f2da98864b4425bf7c0055951fdd4b P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:22:20.759866 23384 raft_consensus.cc:515] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:20.843582 23726 raft_consensus.cc:2804] T ecc1e363230043fe9fef390f0b979f20 P 1ca2725d290e47d68b1a4b609c9816ac [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:22:20.761370 23737 raft_consensus.cc:515] T 746aabf856ab401c9ef5e859371e7b67 P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:20.782493 23556 consensus_queue.cc:1048] T 7c2095a5f4ad49408dbc7f69c606120d P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.045s
I20260709 13:22:20.719395 23761 raft_consensus.cc:515] T c0eab25ce4c64e018a041beeda2867db P 90f3da57796b488c9087a2f523adf045 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:20.790927 23588 consensus_queue.cc:1048] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.049s
I20260709 13:22:20.794576 23688 consensus_queue.cc:1059] T 02838c86531d4f98bf58d1c22b8baf77 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: INVALID_TERM, Last received: 19.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.046s
I20260709 13:22:20.855412 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada5a004ba7c46819051e3ccec71ffdb" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 23 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:20.792531 23778 raft_consensus.cc:515] T 434bff3082d141838068f323bdb0d1ed P cb82cc7bd6db46d980c75237a1402860 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:20.840965 23762 raft_consensus.cc:2804] T fffab26427e24d398e2695aad40ca134 P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:22:20.868701 16825 raft_consensus.cc:1240] T 23f82224f4564fea93572e8206bad1fe P cb82cc7bd6db46d980c75237a1402860 [term 18 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 17. Current term is 18. Ops: []
I20260709 13:22:20.837837 23306 raft_consensus.cc:493] T 7c2095a5f4ad49408dbc7f69c606120d P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:22:20.871418 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 80861.3us
I20260709 13:22:20.787534 23757 raft_consensus.cc:515] T 829b4ad8c5a54eababeea1181b960d4b P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:20.855252 23726 raft_consensus.cc:493] T ecc1e363230043fe9fef390f0b979f20 P 1ca2725d290e47d68b1a4b609c9816ac [term 23 FOLLOWER]: Starting leader election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:22:20.876322 16116 leader_election.cc:304] T 8cc959fb7f704fba9009c0595467d074 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
I20260709 13:22:20.863430 23614 raft_consensus.cc:2749] T 84f2da98864b4425bf7c0055951fdd4b P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:22:20.809782 23784 raft_consensus.cc:515] T 23f82224f4564fea93572e8206bad1fe P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:20.827875 23710 raft_consensus.cc:515] T 66ae176961b74813a58bc15c9c4c6f2a P 90f3da57796b488c9087a2f523adf045 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:20.875758 16265 leader_election.cc:304] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:22:20.871058 23732 raft_consensus.cc:2749] T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b [term 22 FOLLOWER]: Leader election lost for term 21. Reason: Vote denied by peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943) with higher term. Message: Invalid argument: T e4dc6b0541ca431ebc757beac00521f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b for earlier term 21. Current term is 22.
I20260709 13:22:20.870769 23566 leader_election.cc:290] T 5a96373bb0e14dce8a266b2fc6c58507 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 18 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:20.872097 23788 raft_consensus.cc:493] T c0eab25ce4c64e018a041beeda2867db P 647a343a542d4a139fe106c19095129b [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:20.874166 23306 raft_consensus.cc:3060] T 7c2095a5f4ad49408dbc7f69c606120d P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:22:20.892786 23804 log.cc:927] Time spent T 1df93779209748b9860f6d893dd3de53 P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 0.245s	user 0.001s	sys 0.000s
W20260709 13:22:20.893723 23814 log.cc:927] Time spent T 87ac0d1f7e8a419f9bcbc04aee0e067e P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 0.084s	user 0.000s	sys 0.000s
I20260709 13:22:20.877668 16040 catalog_manager.cc:5697] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860 reported cstate change: term changed from 17 to 19. New cstate: current_term: 19 leader_uuid: "cb82cc7bd6db46d980c75237a1402860" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:22:20.894591 23287 leader_election.cc:290] T 8158bbd17b994d2eb96f560bd0f1b88d P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:22:20.883714 23726 raft_consensus.cc:3060] T ecc1e363230043fe9fef390f0b979f20 P 1ca2725d290e47d68b1a4b609c9816ac [term 23 FOLLOWER]: Advancing to term 24
W20260709 13:22:20.895721 23797 log.cc:927] Time spent T 3a7ca61592e848ea9b0fb81a2ce33cdc P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.332s	user 0.001s	sys 0.001s
I20260709 13:22:20.896155 23384 leader_election.cc:290] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:20.896005 23816 consensus_queue.cc:1048] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
W20260709 13:22:20.897765 23805 log.cc:927] Time spent T 9e65676b21b748728ad39668cfe722ec P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.169s	user 0.001s	sys 0.000s
I20260709 13:22:20.898013 23757 leader_election.cc:290] T 829b4ad8c5a54eababeea1181b960d4b P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
W20260709 13:22:20.898442 23813 log.cc:927] Time spent T 19baa7bc367844c288bca8f81439c413 P cb82cc7bd6db46d980c75237a1402860: Append to log took a long time: real 0.150s	user 0.000s	sys 0.000s
I20260709 13:22:20.898644 23776 leader_election.cc:290] T af57996a8be84d398f0f023e8d08ef77 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:22:20.899132 16151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5718c623ebee4d2bb770f0f9ddb2f164" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:22:20.878733 23598 raft_consensus.cc:3055] T 02838c86531d4f98bf58d1c22b8baf77 P 1ca2725d290e47d68b1a4b609c9816ac [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:22:20.876024 23762 raft_consensus.cc:493] T fffab26427e24d398e2695aad40ca134 P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
W20260709 13:22:20.914157 16299 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48674 (request call id 3581) took 1993 ms (client timeout 1947 ms). Trace:
I20260709 13:22:20.914338 16190 leader_election.cc:304] T e5ad4eb281564936a59f41e387a47156 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 647a343a542d4a139fe106c19095129b
I20260709 13:22:20.920024 16291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fffab26427e24d398e2695aad40ca134" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:20.900554 16265 leader_election.cc:304] T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:22:20.924412 16292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87ac0d1f7e8a419f9bcbc04aee0e067e" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
W20260709 13:22:20.899247 23808 log.cc:927] Time spent T 66f75124215f4ca6bef831e174a792d8 P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 0.102s	user 0.000s	sys 0.000s
W20260709 13:22:20.925904 16789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 370522us
I20260709 13:22:20.881208 23774 leader_election.cc:290] T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:20.928576 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ad5dbfa99ee4b8d9e28be92b3f36db5" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:20.891005 23719 raft_consensus.cc:2804] T 8cc959fb7f704fba9009c0595467d074 P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:22:20.890516 23023 leader_election.cc:290] T 5a959dbecc2648eb85d1cc550f5a3245 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:20.920526 23558 raft_consensus.cc:2804] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:22:20.889415 23800 consensus_queue.cc:1048] T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.160s
I20260709 13:22:20.919826 23708 leader_election.cc:290] T 434bff3082d141838068f323bdb0d1ed P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:20.947563 16151 raft_consensus.cc:3060] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:22:20.948330 23754 leader_election.cc:290] T b87147731a68400e8cce9b879daa7c5d P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 22 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:20.915582 23817 raft_consensus.cc:493] T d16dfad0c3b540e5b4b86492ddf87208 P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:20.950832 16293 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48684 (request call id 3307) took 1731 ms. Trace:
I20260709 13:22:20.888051 23688 consensus_queue.cc:1048] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.049s
I20260709 13:22:20.955307 23777 leader_election.cc:290] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
W20260709 13:22:20.958514 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 58038.6us
W20260709 13:22:20.947134 16117 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 100027us
I20260709 13:22:20.929561 23785 leader_election.cc:290] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:22:21.036357 16265 leader_election.cc:304] T e4dc6b0541ca431ebc757beac00521f1 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:22:21.048741 16265 leader_election.cc:304] T 8cc959fb7f704fba9009c0595467d074 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:22:20.933890 23780 leader_election.cc:290] T 8d4ab870a05347868956aa8b841b2981 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
W20260709 13:22:20.958055 16299 rpcz_store.cc:269] 0709 13:22:18.920332 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:22:18.926466 (+  6134us) service_pool.cc:224] Handling call
0709 13:22:20.913922 (+1987456us) spinlock_profiling.cc:243] Waited 14.9 ms on lock 0x7b5400353ed4. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:22:20.914081 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":34352512}
I20260709 13:22:20.970084 23719 raft_consensus.cc:493] T 8cc959fb7f704fba9009c0595467d074 P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Starting leader election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:22:20.872278 23730 consensus_queue.cc:237] T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.15, Last appended by leader: 15, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:22:20.972885 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 81038.3us
I20260709 13:22:20.972995 16291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4dc6b0541ca431ebc757beac00521f1" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:22:20.942966 23717 leader_election.cc:290] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 21 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
W20260709 13:22:20.935564 23810 log.cc:927] Time spent T 26354361e9b044cea1c2b86c4882fe92 P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 0.069s	user 0.000s	sys 0.001s
I20260709 13:22:20.988821 16292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:20.989389 23829 raft_consensus.cc:493] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:22:20.981657 23774 raft_consensus.cc:493] T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:20.989539 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4f65cdf00b748bb963ee6e81ef43b79" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:20.990064 23821 raft_consensus.cc:2804] T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:22:21.076413 16222 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33050 (request call id 1313) took 1077 ms. Trace:
I20260709 13:22:20.991041 23826 raft_consensus.cc:493] T b87147731a68400e8cce9b879daa7c5d P 647a343a542d4a139fe106c19095129b [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:20.991361 23024 leader_election.cc:290] T ada5a004ba7c46819051e3ccec71ffdb P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
W20260709 13:22:20.992019 23823 log.cc:927] Time spent T 746aabf856ab401c9ef5e859371e7b67 P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 0.064s	user 0.000s	sys 0.000s
I20260709 13:22:20.037703 16225 rpcz_store.cc:276] 0709 13:22:18.954282 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:22:19.094030 (+139748us) service_pool.cc:224] Handling call
0709 13:22:19.094436 (+   406us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:22:19.094687 (+   251us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 13:22:19.095012 (+   325us) log.cc:844] Serialized 14 byte log entry
0709 13:22:20.037022 (+942010us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:22:20.037077 (+    55us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:22:20.037119 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:22:20.037177 (+    58us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:22:20.037279 (+   102us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":778686,"prepare.queue_time_us":11379,"prepare.run_cpu_time_us":35,"prepare.run_wall_time_us":37,"raft.queue_time_us":778884,"thread_start_us":162904,"threads_started":1,"wal-append.queue_time_us":921863}
I20260709 13:22:21.077286 23826 raft_consensus.cc:515] T b87147731a68400e8cce9b879daa7c5d P 647a343a542d4a139fe106c19095129b [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:20.992280 23812 consensus_queue.cc:1048] T c0eab25ce4c64e018a041beeda2867db P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.106s
I20260709 13:22:21.078373 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806d7690c2b644959152074bb2cab0a7" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:22:21.079494 23826 leader_election.cc:290] T b87147731a68400e8cce9b879daa7c5d P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:20.992964 23818 consensus_queue.cc:1048] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.067s
I20260709 13:22:21.080040 16115 leader_election.cc:304] T fffab26427e24d398e2695aad40ca134 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:22:20.994226 16151 raft_consensus.cc:2468] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 22.
I20260709 13:22:21.081166 16115 leader_election.cc:304] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:22:21.082393 16115 leader_election.cc:304] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:22:21.082569 23710 leader_election.cc:290] T 66ae176961b74813a58bc15c9c4c6f2a P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:20.976961 23558 raft_consensus.cc:493] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:22:21.083302 23829 raft_consensus.cc:515] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:21.083822 23558 raft_consensus.cc:3060] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:22:21.084441 23786 raft_consensus.cc:2749] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 647a343a542d4a139fe106c19095129b [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:22:21.085171 23786 raft_consensus.cc:2749] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:22:21.085561 23829 leader_election.cc:290] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:21.086494 16115 leader_election.cc:304] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860
I20260709 13:22:20.996739 23638 raft_consensus.cc:3055] T 57a744f5aaf944678a63469cbf514015 P cb82cc7bd6db46d980c75237a1402860 [term 18 LEADER]: Stepping down as leader of term 18
W20260709 13:22:20.997365 23794 consensus_meta.cc:287] T e5ad4eb281564936a59f41e387a47156 P 647a343a542d4a139fe106c19095129b: Time spent flushing consensus metadata: real 0.583s	user 0.004s	sys 0.005s
I20260709 13:22:21.087322 23829 raft_consensus.cc:2749] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:22:21.087591 23794 raft_consensus.cc:2749] T e5ad4eb281564936a59f41e387a47156 P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:22:20.998193 23828 log.cc:927] Time spent T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 0.115s	user 0.000s	sys 0.000s
I20260709 13:22:20.999929 23726 raft_consensus.cc:515] T ecc1e363230043fe9fef390f0b979f20 P 1ca2725d290e47d68b1a4b609c9816ac [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:21.001289 23778 leader_election.cc:290] T 434bff3082d141838068f323bdb0d1ed P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:21.002713 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "829b4ad8c5a54eababeea1181b960d4b" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:21.105159 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a959dbecc2648eb85d1cc550f5a3245" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:21.106360 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66f75124215f4ca6bef831e174a792d8" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:21.003334 23603 raft_consensus.cc:3060] T 84f2da98864b4425bf7c0055951fdd4b P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:22:21.003361 16146 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40830 (request call id 2207) took 1752 ms. Trace:
I20260709 13:22:21.063107 23719 raft_consensus.cc:3060] T 8cc959fb7f704fba9009c0595467d074 P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:22:21.109586 16146 rpcz_store.cc:276] 0709 13:22:19.250883 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:22:19.251024 (+   141us) service_pool.cc:224] Handling call
0709 13:22:19.718237 (+467213us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:22:20.431754 (+713517us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:22:21.003039 (+571285us) spinlock_profiling.cc:243] Waited 624 ms on lock 0x7b540018f954. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:22:21.003285 (+   246us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1434792832}
W20260709 13:22:21.005298 16820 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49896 (request call id 1642) took 2000 ms (client timeout 1947 ms). Trace:
W20260709 13:22:21.110945 16820 rpcz_store.cc:269] 0709 13:22:19.004954 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:22:19.005058 (+   104us) service_pool.cc:224] Handling call
0709 13:22:21.005090 (+2000032us) spinlock_profiling.cc:243] Waited 1.16 s on lock 0x7b5400787154. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:22:21.005221 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2669696640}
I20260709 13:22:21.006592 16822 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49906 (request call id 1439) took 3058 ms. Trace:
I20260709 13:22:21.016525 16220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f5f89186204242bf377f7c42b2e2da" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:22:21.018406 23399 leader_election.cc:290] T 26354361e9b044cea1c2b86c4882fe92 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 18 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:21.020121 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cc959fb7f704fba9009c0595467d074" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:21.022078 23789 leader_election.cc:290] T 3a7ca61592e848ea9b0fb81a2ce33cdc P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:20.963267 23762 raft_consensus.cc:3060] T fffab26427e24d398e2695aad40ca134 P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:22:20.952216 23598 raft_consensus.cc:740] T 02838c86531d4f98bf58d1c22b8baf77 P 1ca2725d290e47d68b1a4b609c9816ac [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:22:21.123813 23598 consensus_queue.cc:260] T 02838c86531d4f98bf58d1c22b8baf77 P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.12, Last appended by leader: 12, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:21.125221 23598 raft_consensus.cc:3060] T 02838c86531d4f98bf58d1c22b8baf77 P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:22:21.128908 23685 consensus_meta.cc:287] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P cb82cc7bd6db46d980c75237a1402860: Time spent flushing consensus metadata: real 1.082s	user 0.012s	sys 0.000s
I20260709 13:22:21.129894 16218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746aabf856ab401c9ef5e859371e7b67" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:21.029426 16293 rpcz_store.cc:276] 0709 13:22:19.219657 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:22:19.219865 (+   208us) service_pool.cc:224] Handling call
0709 13:22:19.220314 (+   449us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:22:19.259302 (+ 38988us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:22:19.259348 (+    46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:22:19.727366 (+468018us) log.cc:844] Serialized 26 byte log entry
0709 13:22:20.188941 (+461575us) raft_consensus.cc:1645] Marking committed up to 15
0709 13:22:20.188998 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:22:20.189100 (+   102us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:22:20.950527 (+761427us) raft_consensus.cc:1685] finished
0709 13:22:20.950556 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:22:20.950728 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":189,"raft.run_cpu_time_us":127,"raft.run_wall_time_us":12739,"thread_start_us":358729,"threads_started":1,"wal-append.queue_time_us":559437}
I20260709 13:22:21.131524 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f5f89186204242bf377f7c42b2e2da" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:22:21.131882 16818 raft_consensus.cc:1275] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P cb82cc7bd6db46d980c75237a1402860 [term 22 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 21 index: 16. Preceding OpId from leader: term: 22 index: 17. (index mismatch)
I20260709 13:22:21.132552 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806d7690c2b644959152074bb2cab0a7" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:22:21.132576 23775 raft_consensus.cc:493] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P cb82cc7bd6db46d980c75237a1402860 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:21.133103 16293 raft_consensus.cc:3060] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:22:21.133460 23775 raft_consensus.cc:515] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P cb82cc7bd6db46d980c75237a1402860 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:22:21.057147 16266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 153736us
I20260709 13:22:21.135466 23780 consensus_queue.cc:1048] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:22:20.977138 23384 raft_consensus.cc:2804] T e5ad4eb281564936a59f41e387a47156 P 1ca2725d290e47d68b1a4b609c9816ac [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:22:21.137989 23384 raft_consensus.cc:697] T e5ad4eb281564936a59f41e387a47156 P 1ca2725d290e47d68b1a4b609c9816ac [term 20 LEADER]: Becoming Leader. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:22:21.138895 23384 consensus_queue.cc:237] T e5ad4eb281564936a59f41e387a47156 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:21.140612 16224 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33050 (request call id 1314) took 1141 ms. Trace:
I20260709 13:22:20.921411 23788 raft_consensus.cc:515] T c0eab25ce4c64e018a041beeda2867db P 647a343a542d4a139fe106c19095129b [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:22:21.036394 23813 log.cc:927] Time spent T 19baa7bc367844c288bca8f81439c413 P cb82cc7bd6db46d980c75237a1402860: Append to log took a long time: real 0.069s	user 0.000s	sys 0.000s
I20260709 13:22:21.140987 16224 rpcz_store.cc:276] 0709 13:22:19.999579 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:22:19.999753 (+   174us) service_pool.cc:224] Handling call
0709 13:22:20.037201 (+ 37448us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:22:20.433360 (+396159us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:22:20.433414 (+    54us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:22:20.433976 (+   562us) log.cc:844] Serialized 26 byte log entry
0709 13:22:20.716824 (+282848us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:22:20.716859 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:22:21.140219 (+423360us) spinlock_profiling.cc:243] Waited 149 ms on lock 0x7b5400139554. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:22:21.140272 (+    53us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:22:21.140324 (+    52us) raft_consensus.cc:1685] finished
0709 13:22:21.140338 (+    14us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:22:21.140555 (+   217us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":58395,"raft.run_cpu_time_us":103,"raft.run_wall_time_us":105,"spinlock_wait_cycles":343299328,"thread_start_us":380530,"threads_started":2,"wal-append.queue_time_us":315530}
I20260709 13:22:21.066361 23827 raft_consensus.cc:493] T 18ad7e3af5504571a69d10c00750a01c P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:22:21.142737 23827 raft_consensus.cc:515] T 18ad7e3af5504571a69d10c00750a01c P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } }
I20260709 13:22:21.143253 23788 leader_election.cc:290] T c0eab25ce4c64e018a041beeda2867db P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:21.144888 23827 leader_election.cc:290] T 18ad7e3af5504571a69d10c00750a01c P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:22:21.145592 16821 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49892 (request call id 1126) took 2132 ms. Trace:
I20260709 13:22:21.145841 16296 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48684 (request call id 3322) took 1238 ms. Trace:
W20260709 13:22:21.146306 23712 consensus_meta.cc:287] T 23f82224f4564fea93572e8206bad1fe P 647a343a542d4a139fe106c19095129b: Time spent flushing consensus metadata: real 1.040s	user 0.012s	sys 0.000s
I20260709 13:22:21.147063 16823 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49892 (request call id 1123) took 2135 ms. Trace:
I20260709 13:22:21.145965 16821 rpcz_store.cc:276] 0709 13:22:19.013078 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:22:19.831276 (+818198us) service_pool.cc:224] Handling call
0709 13:22:19.831645 (+   369us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:22:19.870522 (+ 38877us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:22:20.128317 (+257795us) log.cc:844] Serialized 14 byte log entry
0709 13:22:20.631082 (+502765us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:22:20.639561 (+  8479us) log.cc:844] Serialized 52 byte log entry
0709 13:22:20.646407 (+  6846us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:22:20.646501 (+    94us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:22:20.646680 (+   179us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:22:21.145364 (+498684us) raft_consensus.cc:1685] finished
0709 13:22:21.145383 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:22:21.145536 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":15114,"prepare.queue_time_us":25719,"prepare.run_cpu_time_us":25,"prepare.run_wall_time_us":25,"raft.queue_time_us":146749,"raft.run_cpu_time_us":329,"raft.run_wall_time_us":39770,"spinlock_wait_cycles":12085504,"thread_start_us":671010,"threads_started":2,"wal-append.queue_time_us":482209}
I20260709 13:22:21.069581 16297 raft_consensus.cc:2468] T c4f65cdf00b748bb963ee6e81ef43b79 P 90f3da57796b488c9087a2f523adf045 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 21.
I20260709 13:22:21.147488 16823 rpcz_store.cc:276] 0709 13:22:19.012144 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:22:19.199592 (+187448us) service_pool.cc:224] Handling call
0709 13:22:19.211218 (+ 11626us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:22:20.635908 (+1424690us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:22:21.146837 (+510929us) spinlock_profiling.cc:243] Waited 1.22 s on lock 0x7b5400520dd4. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:22:21.147004 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2808212224}
I20260709 13:22:21.148676 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d4ab870a05347868956aa8b841b2981" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:21.148818 16295 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48684 (request call id 3324) took 1217 ms. Trace:
I20260709 13:22:21.149852 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87ac0d1f7e8a419f9bcbc04aee0e067e" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:21.150954 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a96373bb0e14dce8a266b2fc6c58507" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:22:21.149369 16265 leader_election.cc:304] T e5ad4eb281564936a59f41e387a47156 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:22:21.071847 23831 raft_consensus.cc:2804] T 8cc959fb7f704fba9009c0595467d074 P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Leader pre-election won for term 20
W20260709 13:22:21.151815 23824 log.cc:927] Time spent T 434bff3082d141838068f323bdb0d1ed P cb82cc7bd6db46d980c75237a1402860: Append to log took a long time: real 0.354s	user 0.002s	sys 0.000s
I20260709 13:22:21.152083 23831 raft_consensus.cc:493] T 8cc959fb7f704fba9009c0595467d074 P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:21.071059 16040 catalog_manager.cc:5697] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860 reported cstate change: term changed from 13 to 15. New cstate: current_term: 15 leader_uuid: "1ca2725d290e47d68b1a4b609c9816ac" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:22:21.072376 23784 leader_election.cc:290] T 23f82224f4564fea93572e8206bad1fe P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:21.035359 23286 raft_consensus.cc:493] T 2794af5b99984efebdee564035841cb0 P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:21.153254 23286 raft_consensus.cc:515] T 2794af5b99984efebdee564035841cb0 P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:21.153765 16040 catalog_manager.cc:5697] T 2794af5b99984efebdee564035841cb0 P cb82cc7bd6db46d980c75237a1402860 reported cstate change: term changed from 12 to 15, leader changed from 647a343a542d4a139fe106c19095129b (127.14.177.129) to cb82cc7bd6db46d980c75237a1402860 (127.14.177.132). New cstate: current_term: 15 leader_uuid: "cb82cc7bd6db46d980c75237a1402860" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:22:21.155484 23286 leader_election.cc:290] T 2794af5b99984efebdee564035841cb0 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:21.155543 23559 raft_consensus.cc:2804] T e5ad4eb281564936a59f41e387a47156 P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:22:21.072145 23630 leader_election.cc:290] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:21.072877 23774 raft_consensus.cc:515] T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:22:21.075605 23819 log.cc:927] Time spent T 19baa7bc367844c288bca8f81439c413 P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 0.105s	user 0.001s	sys 0.000s
I20260709 13:22:21.076337 23761 leader_election.cc:290] T c0eab25ce4c64e018a041beeda2867db P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:22:21.076813 23737 leader_election.cc:290] T 746aabf856ab401c9ef5e859371e7b67 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:21.081326 23605 raft_consensus.cc:2749] T fffab26427e24d398e2695aad40ca134 P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:22:20.999279 16223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee9ed67006144797b1de6ea8c2013a68" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:21.088513 16149 raft_consensus.cc:2468] T e5ad4eb281564936a59f41e387a47156 P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 20.
I20260709 13:22:21.079494 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a96373bb0e14dce8a266b2fc6c58507" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:22:21.180723 16225 raft_consensus.cc:2393] T 5a96373bb0e14dce8a266b2fc6c58507 P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 18: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
I20260709 13:22:21.081190 16151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66f75124215f4ca6bef831e174a792d8" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:22:21.076874 16222 rpcz_store.cc:276] 0709 13:22:19.998753 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:22:19.998934 (+   181us) service_pool.cc:224] Handling call
0709 13:22:20.037550 (+ 38616us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:22:20.333295 (+295745us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:22:21.076191 (+742896us) spinlock_profiling.cc:243] Waited 158 ms on lock 0x7b54001423d4. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:22:21.076355 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":363189504}
I20260709 13:22:21.103683 23726 leader_election.cc:290] T ecc1e363230043fe9fef390f0b979f20 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 24 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:21.063534 23287 raft_consensus.cc:2804] T e4dc6b0541ca431ebc757beac00521f1 P 90f3da57796b488c9087a2f523adf045 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:22:21.184715 23774 leader_election.cc:290] T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:21.185209 23287 raft_consensus.cc:697] T e4dc6b0541ca431ebc757beac00521f1 P 90f3da57796b488c9087a2f523adf045 [term 22 LEADER]: Becoming Leader. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:22:21.186188 23287 consensus_queue.cc:237] T e4dc6b0541ca431ebc757beac00521f1 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.11, Last appended by leader: 11, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:21.112100 16220 raft_consensus.cc:3060] T c2f5f89186204242bf377f7c42b2e2da P 1ca2725d290e47d68b1a4b609c9816ac [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:22:21.076860 23821 raft_consensus.cc:493] T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:21.195343 23821 raft_consensus.cc:3060] T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:22:21.196620 16817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746aabf856ab401c9ef5e859371e7b67" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:21.198029 16817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62fe3346cd3c42c4a55e14af9073bca4" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 27 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:21.199635 16823 raft_consensus.cc:1240] T 74ca8140f35c4ae29e8a664f6bdfc829 P cb82cc7bd6db46d980c75237a1402860 [term 23 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 22. Current term is 23. Ops: []
I20260709 13:22:21.200762 16816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdbd78acb4cf46298e0de8ba0d130e19" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 25 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:21.202279 16823 raft_consensus.cc:1275] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 [term 25 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 22 index: 15. Preceding OpId from leader: term: 25 index: 16. (index mismatch)
I20260709 13:22:21.118479 16148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a878999c0024d2abe517484180290fc" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:22:20.961591 23815 raft_consensus.cc:493] T 1df93779209748b9860f6d893dd3de53 P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:22:21.218703 16220 raft_consensus.cc:2468] T c2f5f89186204242bf377f7c42b2e2da P 1ca2725d290e47d68b1a4b609c9816ac [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 24.
I20260709 13:22:21.218919 16142 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40830 (request call id 2211) took 1855 ms. Trace:
I20260709 13:22:21.151577 16297 raft_consensus.cc:3060] T 5a96373bb0e14dce8a266b2fc6c58507 P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:22:21.152509 23831 raft_consensus.cc:3060] T 8cc959fb7f704fba9009c0595467d074 P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:22:21.219411 16142 rpcz_store.cc:276] 0709 13:22:19.363434 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:22:19.715850 (+352416us) service_pool.cc:224] Handling call
0709 13:22:19.723900 (+  8050us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:22:20.044597 (+320697us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:22:20.044628 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:22:20.051047 (+  6419us) log.cc:844] Serialized 26 byte log entry
0709 13:22:20.520327 (+469280us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:22:20.520389 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:22:20.520487 (+    98us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:22:21.218720 (+698233us) raft_consensus.cc:1685] finished
0709 13:22:21.218739 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:22:21.218861 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":55365,"raft.queue_time_us":172539,"raft.run_cpu_time_us":92,"raft.run_wall_time_us":93,"thread_start_us":658900,"threads_started":2,"wal-append.queue_time_us":685482}
I20260709 13:22:21.072783 16292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d16dfad0c3b540e5b4b86492ddf87208" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:21.142858 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5d59407eb545b895d8d922e0dd69fd" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:21.160413 16115 leader_election.cc:304] T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac
I20260709 13:22:21.156041 16221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5718c623ebee4d2bb770f0f9ddb2f164" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:22:21.222800 16221 raft_consensus.cc:2368] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b for earlier term 21. Current term is 22.
I20260709 13:22:21.146353 16296 rpcz_store.cc:276] 0709 13:22:19.907047 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:22:19.908452 (+  1405us) service_pool.cc:224] Handling call
0709 13:22:19.908753 (+   301us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:22:20.360678 (+451925us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:22:20.360728 (+    50us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:22:20.360791 (+    63us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:22:20.360827 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:22:21.145629 (+784802us) spinlock_profiling.cc:243] Waited 452 ms on lock 0x7b540024c554. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:22:21.145673 (+    44us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:22:21.145789 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1038676736}
I20260709 13:22:21.155205 16265 leader_election.cc:304] T 68a6838d82ef4cac83507f906139d90f P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
W20260709 13:22:21.223853 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 69151.9us
I20260709 13:22:21.149224 16295 rpcz_store.cc:276] 0709 13:22:19.931561 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:22:19.935497 (+  3936us) service_pool.cc:224] Handling call
0709 13:22:19.935734 (+   237us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:22:20.103363 (+167629us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:22:20.103397 (+    34us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:22:20.103437 (+    40us) raft_consensus.cc:1645] Marking committed up to 16
0709 13:22:20.103484 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:22:21.148609 (+1045125us) spinlock_profiling.cc:243] Waited 167 ms on lock 0x7b540010ed54. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:22:21.148654 (+    45us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:22:21.148767 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":385054592}
W20260709 13:22:21.150866 23357 consensus_meta.cc:287] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 647a343a542d4a139fe106c19095129b: Time spent flushing consensus metadata: real 0.566s	user 0.007s	sys 0.001s
I20260709 13:22:21.175922 16146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806d7690c2b644959152074bb2cab0a7" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:22:21.227329 16146 raft_consensus.cc:2393] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 in current term 21: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
I20260709 13:22:21.087342 23638 raft_consensus.cc:740] T 57a744f5aaf944678a63469cbf514015 P cb82cc7bd6db46d980c75237a1402860 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:22:21.111842 16822 rpcz_store.cc:276] 0709 13:22:17.947775 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:22:18.095256 (+147481us) service_pool.cc:224] Handling call
0709 13:22:18.095793 (+   537us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:22:18.099636 (+  3843us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:22:18.099680 (+    44us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:22:18.122961 (+ 23281us) log.cc:844] Serialized 52 byte log entry
0709 13:22:18.887139 (+764178us) spinlock_profiling.cc:243] Waited 8.06 ms on lock 0x7b180032b420. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b693ade1 00007f69b69475b1 00007f69ada54e07 00007f69b68afab9 00007f69ada7169b 00007f69ada6f93f 00007f69ada71c02 00007f69b06f920b 00007f69b06fed23 00007f69b06ff0d1 00007f69b0b1aa0f
0709 13:22:19.083116 (+195977us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:22:19.083202 (+    86us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:22:19.083279 (+    77us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:22:21.006135 (+1922856us) spinlock_profiling.cc:243] Waited 4.09 ms on lock 0x7b2c0018a038. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69aee0f291 00007f69aee2a0ca 00007f69b0b584ad 00007f69b0b5ff8e 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2
0709 13:22:21.006252 (+   117us) raft_consensus.cc:1685] finished
0709 13:22:21.006275 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:22:21.006460 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":11662,"raft.queue_time_us":3631,"raft.run_cpu_time_us":180,"raft.run_wall_time_us":18866,"spinlock_wait_cycles":27940864,"thread_start_us":195715,"threads_started":1,"wal-append.queue_time_us":1021218,"wal-append.run_cpu_time_us":13998,"wal-append.run_wall_time_us":1889444}
I20260709 13:22:21.177017 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3884f864a96d428785895e5463e5728f" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 17 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:22:21.229779 16296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a959dbecc2648eb85d1cc550f5a3245" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:21.230104 16150 raft_consensus.cc:3060] T 3884f864a96d428785895e5463e5728f P 647a343a542d4a139fe106c19095129b [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:22:21.230223 16148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02838c86531d4f98bf58d1c22b8baf77" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 23 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:22:21.181202 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada5a004ba7c46819051e3ccec71ffdb" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 23 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:21.181746 16151 raft_consensus.cc:2468] T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 19.
I20260709 13:22:21.096089 23024 raft_consensus.cc:493] T cdbd78acb4cf46298e0de8ba0d130e19 P 90f3da57796b488c9087a2f523adf045 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:21.156188 23559 raft_consensus.cc:493] T e5ad4eb281564936a59f41e387a47156 P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:21.183092 23306 raft_consensus.cc:515] T 7c2095a5f4ad49408dbc7f69c606120d P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:21.228199 22677 consensus_queue.cc:1059] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:22:21.183523 16222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18ad7e3af5504571a69d10c00750a01c" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:21.191818 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2372981006d4b218215c28964dea623" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 26 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:21.100230 23533 leader_election.cc:290] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:21.203080 16816 raft_consensus.cc:1275] T 829b4ad8c5a54eababeea1181b960d4b P cb82cc7bd6db46d980c75237a1402860 [term 19 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 13:22:21.209456 16293 raft_consensus.cc:2468] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 21.
I20260709 13:22:21.205394 16789 leader_election.cc:304] T d2ab0ac8347641e6955e9bce7d90c1a2 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:22:21.199064 16817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee9ed67006144797b1de6ea8c2013a68" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:21.157117 16218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e4bfb9090c8460eb14fa7e0d818e7f1" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:21.047957 23817 raft_consensus.cc:515] T d16dfad0c3b540e5b4b86492ddf87208 P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:22:21.227733 16115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 67454.5us
I20260709 13:22:21.231777 16151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87ac0d1f7e8a419f9bcbc04aee0e067e" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:22:21.239542 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7ca61592e848ea9b0fb81a2ce33cdc" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:21.239673 23638 consensus_queue.cc:260] T 57a744f5aaf944678a63469cbf514015 P cb82cc7bd6db46d980c75237a1402860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.14, Last appended by leader: 14, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:21.241957 23648 leader_election.cc:290] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 25 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:21.243007 16819 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49892 (request call id 1131) took 2153 ms. Trace:
I20260709 13:22:21.243376 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2372981006d4b218215c28964dea623" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 26 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:21.221621 23785 raft_consensus.cc:493] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:22:21.241957 16223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66ae176961b74813a58bc15c9c4c6f2a" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 27 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:21.244227 16142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e9b2bf7011f4ae688a79d7bb5a447b4" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:22:21.244803 23210 raft_consensus.cc:3055] T c9944d5e4b8c4109af9955c4f08c9095 P 90f3da57796b488c9087a2f523adf045 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:22:21.245007 16292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6f07842d3aa47858df5665874c86f3a" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:21.245914 16222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af57996a8be84d398f0f023e8d08ef77" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:21.247433 16148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68a6838d82ef4cac83507f906139d90f" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:22:21.247694 22677 raft_consensus.cc:3055] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:22:21.277456 22677 raft_consensus.cc:740] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:22:21.277509 16148 raft_consensus.cc:2468] T 68a6838d82ef4cac83507f906139d90f P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 20.
W20260709 13:22:21.261788 23839 log.cc:927] Time spent T ecc1e363230043fe9fef390f0b979f20 P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 0.081s	user 0.001s	sys 0.000s
I20260709 13:22:21.249270 16295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a96373bb0e14dce8a266b2fc6c58507" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:22:21.244575 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0eab25ce4c64e018a041beeda2867db" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 23 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:21.251242 23024 raft_consensus.cc:515] T cdbd78acb4cf46298e0de8ba0d130e19 P 90f3da57796b488c9087a2f523adf045 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:21.236157 16822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:22:21.253121 16150 raft_consensus.cc:2468] T 3884f864a96d428785895e5463e5728f P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 17.
I20260709 13:22:21.233978 16265 leader_election.cc:304] T 8a878999c0024d2abe517484180290fc P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
W20260709 13:22:21.287038 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 53123.8us
I20260709 13:22:21.287343 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a9de347c711477e87c582d003f36925" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:22:21.288422 16150 raft_consensus.cc:2393] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 in current term 25: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
I20260709 13:22:21.289439 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:22:21.290256 16150 raft_consensus.cc:2468] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 23.
I20260709 13:22:21.262899 16151 raft_consensus.cc:2468] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 647a343a542d4a139fe106c19095129b [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 21.
I20260709 13:22:21.252992 23793 leader_election.cc:290] T 2794af5b99984efebdee564035841cb0 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:21.294806 16148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f5f89186204242bf377f7c42b2e2da" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 24 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:22:21.295861 16148 raft_consensus.cc:2393] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 24: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
I20260709 13:22:21.297351 16292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8158bbd17b994d2eb96f560bd0f1b88d" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:21.299302 16295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18ad7e3af5504571a69d10c00750a01c" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:21.234714 23357 raft_consensus.cc:515] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:21.271620 16819 rpcz_store.cc:276] 0709 13:22:19.089698 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:22:19.181376 (+ 91678us) service_pool.cc:224] Handling call
0709 13:22:21.242822 (+2061446us) spinlock_profiling.cc:243] Waited 884 ms on lock 0x7b54006bf654. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:22:21.242951 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2033345408}
I20260709 13:22:21.229811 23761 raft_consensus.cc:2804] T 68a6838d82ef4cac83507f906139d90f P 90f3da57796b488c9087a2f523adf045 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:22:21.227072 23778 raft_consensus.cc:493] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:22:21.233525 23774 raft_consensus.cc:2804] T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:22:21.233944 23795 raft_consensus.cc:493] T c9944d5e4b8c4109af9955c4f08c9095 P cb82cc7bd6db46d980c75237a1402860 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:21.248085 16146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5d59407eb545b895d8d922e0dd69fd" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
W20260709 13:22:21.278591 16789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 73287.6us
I20260709 13:22:21.236881 23559 raft_consensus.cc:3060] T e5ad4eb281564936a59f41e387a47156 P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:22:21.282326 16217 raft_consensus.cc:2468] T d16dfad0c3b540e5b4b86492ddf87208 P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 21.
I20260709 13:22:21.278383 22677 consensus_queue.cc:260] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 22.16, Last appended by leader: 16, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:21.313649 16146 raft_consensus.cc:2468] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 23.
I20260709 13:22:21.286041 23645 raft_consensus.cc:2749] T d2ab0ac8347641e6955e9bce7d90c1a2 P cb82cc7bd6db46d980c75237a1402860 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:22:21.314325 22677 raft_consensus.cc:3060] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:22:21.299403 16218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8158bbd17b994d2eb96f560bd0f1b88d" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:21.299326 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d4ab870a05347868956aa8b841b2981" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:21.300982 16790 leader_election.cc:304] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:22:21.229141 23762 raft_consensus.cc:515] T fffab26427e24d398e2695aad40ca134 P 90f3da57796b488c9087a2f523adf045 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:21.313879 16789 leader_election.cc:304] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:22:21.267437 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434bff3082d141838068f323bdb0d1ed" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:21.276328 23694 raft_consensus.cc:493] T 829b4ad8c5a54eababeea1181b960d4b P cb82cc7bd6db46d980c75237a1402860 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:21.254227 23818 consensus_queue.cc:1048] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.036s
I20260709 13:22:21.298074 16292 raft_consensus.cc:2468] T 8158bbd17b994d2eb96f560bd0f1b88d P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 19.
I20260709 13:22:21.304122 16297 raft_consensus.cc:2468] T 5a96373bb0e14dce8a266b2fc6c58507 P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 18.
I20260709 13:22:21.307657 16148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af57996a8be84d398f0f023e8d08ef77" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:22:21.330298 16148 raft_consensus.cc:2468] T af57996a8be84d398f0f023e8d08ef77 P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 20.
I20260709 13:22:21.330343 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66ae176961b74813a58bc15c9c4c6f2a" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 27 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:21.308678 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c74490c40243a88e62d1760b5cd814" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:21.297689 23784 raft_consensus.cc:2804] T 8a878999c0024d2abe517484180290fc P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:22:21.309793 16142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a959dbecc2648eb85d1cc550f5a3245" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:22:21.276327 23210 raft_consensus.cc:740] T c9944d5e4b8c4109af9955c4f08c9095 P 90f3da57796b488c9087a2f523adf045 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:22:21.315522 16218 raft_consensus.cc:2468] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 19.
I20260709 13:22:21.282953 23558 raft_consensus.cc:515] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:21.318531 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87147731a68400e8cce9b879daa7c5d" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:21.406996 16293 raft_consensus.cc:2393] T b87147731a68400e8cce9b879daa7c5d P 90f3da57796b488c9087a2f523adf045 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 22: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
W20260709 13:22:21.402648 16116 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:22:21.335464 16147 raft_consensus.cc:2393] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 20: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
I20260709 13:22:21.320197 16143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2372981006d4b218215c28964dea623" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 26 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:22:21.409658 16143 raft_consensus.cc:2410] T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 for term 26 because replica has last-logged OpId of term: 23 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 22 index: 15.
I20260709 13:22:21.321849 16220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0eab25ce4c64e018a041beeda2867db" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 23 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:21.372080 16817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a378d7f796f94615b3b347119a9880fd" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:21.327430 16819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87147731a68400e8cce9b879daa7c5d" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:22:21.411633 16819 raft_consensus.cc:2393] T b87147731a68400e8cce9b879daa7c5d P cb82cc7bd6db46d980c75237a1402860 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 22: Already voted for candidate cb82cc7bd6db46d980c75237a1402860 in this term.
I20260709 13:22:21.298465 23638 raft_consensus.cc:3060] T 57a744f5aaf944678a63469cbf514015 P cb82cc7bd6db46d980c75237a1402860 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:22:21.332170 23798 raft_consensus.cc:493] T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:22:21.420595 16145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434bff3082d141838068f323bdb0d1ed" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:22:21.421077 23798 raft_consensus.cc:515] T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:21.421778 16145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4dc6b0541ca431ebc757beac00521f1" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:22:21.422501 16145 raft_consensus.cc:2393] T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 22: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
I20260709 13:22:21.418825 23767 raft_consensus.cc:493] T 8c11074579b046d99e585c2818e18a2c P cb82cc7bd6db46d980c75237a1402860 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:22:21.333272 16146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0eab25ce4c64e018a041beeda2867db" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 23 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:22:21.423784 16147 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40830 (request call id 2215) took 1121 ms. Trace:
I20260709 13:22:21.424196 16146 raft_consensus.cc:2468] T c0eab25ce4c64e018a041beeda2867db P 647a343a542d4a139fe106c19095129b [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 22.
I20260709 13:22:21.391921 23761 raft_consensus.cc:697] T 68a6838d82ef4cac83507f906139d90f P 90f3da57796b488c9087a2f523adf045 [term 21 LEADER]: Becoming Leader. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:22:21.425230 16146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f82224f4564fea93572e8206bad1fe" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:22:21.297418 23533 consensus_queue.cc:1059] T c9944d5e4b8c4109af9955c4f08c9095 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: INVALID_TERM, Last received: 22.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.035s
I20260709 13:22:21.424216 16147 rpcz_store.cc:276] 0709 13:22:20.302527 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:22:20.303503 (+   976us) service_pool.cc:224] Handling call
0709 13:22:21.423581 (+1120078us) spinlock_profiling.cc:243] Waited 916 ms on lock 0x7b54000d0554. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:22:21.423725 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2106052864}
I20260709 13:22:21.426509 23533 consensus_queue.cc:1243] T c9944d5e4b8c4109af9955c4f08c9095 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Peer 1ca2725d290e47d68b1a4b609c9816ac log is divergent from this leader: its last log entry 23.17 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 16
I20260709 13:22:21.425704 23761 consensus_queue.cc:237] T 68a6838d82ef4cac83507f906139d90f P 90f3da57796b488c9087a2f523adf045 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.11, Last appended by leader: 11, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:21.398551 16222 raft_consensus.cc:1240] T c2f5f89186204242bf377f7c42b2e2da P 1ca2725d290e47d68b1a4b609c9816ac [term 24 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 23. Current term is 24. Ops: [23.13-23.13]
I20260709 13:22:21.398902 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66ae176961b74813a58bc15c9c4c6f2a" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 27 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:21.376236 16296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87147731a68400e8cce9b879daa7c5d" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:21.310351 16040 catalog_manager.cc:5697] T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860 reported cstate change: term changed from 17 to 20, leader changed from 647a343a542d4a139fe106c19095129b (127.14.177.129) to cb82cc7bd6db46d980c75237a1402860 (127.14.177.132). New cstate: current_term: 20 leader_uuid: "cb82cc7bd6db46d980c75237a1402860" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:22:21.429428 16296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87147731a68400e8cce9b879daa7c5d" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:22:21.432412 16818 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49906 (request call id 1449) took 2469 ms. Trace:
I20260709 13:22:21.332017 23818 consensus_queue.cc:1243] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Peer cb82cc7bd6db46d980c75237a1402860 log is divergent from this leader: its last log entry 22.15 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 14
I20260709 13:22:21.372859 23024 leader_election.cc:290] T cdbd78acb4cf46298e0de8ba0d130e19 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:21.377588 23306 leader_election.cc:290] T 7c2095a5f4ad49408dbc7f69c606120d P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 19 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:22:21.371449 23357 leader_election.cc:290] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 20 election: Requested vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:21.378682 23625 leader_election.cc:290] T 3884f864a96d428785895e5463e5728f P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 17 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:21.316306 23774 raft_consensus.cc:493] T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b [term 21 FOLLOWER]: Starting leader election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:22:21.439481 23774 raft_consensus.cc:3060] T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:22:21.439800 16824 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49896 (request call id 1646) took 1007 ms. Trace:
I20260709 13:22:21.440021 16151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8158bbd17b994d2eb96f560bd0f1b88d" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:22:21.440237 16824 rpcz_store.cc:276] 0709 13:22:20.432260 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:22:20.516636 (+ 84376us) service_pool.cc:224] Handling call
0709 13:22:21.439624 (+922988us) spinlock_profiling.cc:243] Waited 69.1 ms on lock 0x7b54007c01d4. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:22:21.439741 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":158990720}
I20260709 13:22:21.441282 16151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87147731a68400e8cce9b879daa7c5d" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:22:21.441828 16151 raft_consensus.cc:3060] T b87147731a68400e8cce9b879daa7c5d P 647a343a542d4a139fe106c19095129b [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:22:21.323414 23778 raft_consensus.cc:515] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:21.324904 23795 raft_consensus.cc:515] T c9944d5e4b8c4109af9955c4f08c9095 P cb82cc7bd6db46d980c75237a1402860 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:21.400372 23817 leader_election.cc:290] T d16dfad0c3b540e5b4b86492ddf87208 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
W20260709 13:22:21.400993 23820 log.cc:927] Time spent T 434bff3082d141838068f323bdb0d1ed P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.427s	user 0.001s	sys 0.000s
I20260709 13:22:21.401197 16816 raft_consensus.cc:1275] T 2794af5b99984efebdee564035841cb0 P cb82cc7bd6db46d980c75237a1402860 [term 19 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 18 index: 16. Preceding OpId from leader: term: 19 index: 16. (term mismatch)
I20260709 13:22:21.337543 16148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746aabf856ab401c9ef5e859371e7b67" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:22:21.332139 23694 raft_consensus.cc:515] T 829b4ad8c5a54eababeea1181b960d4b P cb82cc7bd6db46d980c75237a1402860 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:21.402637 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2794af5b99984efebdee564035841cb0" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 20 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:22:21.402909 23772 raft_consensus.cc:493] T 9e65676b21b748728ad39668cfe722ec P 647a343a542d4a139fe106c19095129b [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:21.404486 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6f07842d3aa47858df5665874c86f3a" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:22:21.393731 23775 leader_election.cc:290] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:21.335131 23648 consensus_queue.cc:1048] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.085s
W20260709 13:22:21.377666 23809 log.cc:927] Time spent T 02838c86531d4f98bf58d1c22b8baf77 P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 0.526s	user 0.000s	sys 0.002s
I20260709 13:22:21.328029 23793 raft_consensus.cc:2804] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:22:21.328271 23777 raft_consensus.cc:2804] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:22:21.233867 23815 raft_consensus.cc:515] T 1df93779209748b9860f6d893dd3de53 P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:21.290194 23630 consensus_queue.cc:1048] T 829b4ad8c5a54eababeea1181b960d4b P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.041s
W20260709 13:22:21.388202 23835 log.cc:927] Time spent T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860: Append to log took a long time: real 0.302s	user 0.002s	sys 0.000s
I20260709 13:22:21.431754 16297 raft_consensus.cc:2468] T 66ae176961b74813a58bc15c9c4c6f2a P 90f3da57796b488c9087a2f523adf045 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 26.
W20260709 13:22:21.445801 16190 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:22:21.390120 16292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdbd78acb4cf46298e0de8ba0d130e19" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 25 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:21.447435 16147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68a6838d82ef4cac83507f906139d90f" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:22:21.448292 16220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e4bfb9090c8460eb14fa7e0d818e7f1" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:21.506254 16292 raft_consensus.cc:2468] T cdbd78acb4cf46298e0de8ba0d130e19 P 90f3da57796b488c9087a2f523adf045 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 24.
I20260709 13:22:21.506255 16147 raft_consensus.cc:3060] T 68a6838d82ef4cac83507f906139d90f P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:22:21.507148 16220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c2095a5f4ad49408dbc7f69c606120d" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 19 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:22:21.318531 16295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:22:21.452548 16145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18ad7e3af5504571a69d10c00750a01c" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:22:21.517309 16151 raft_consensus.cc:2468] T b87147731a68400e8cce9b879daa7c5d P 647a343a542d4a139fe106c19095129b [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 22.
I20260709 13:22:21.453804 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdbd78acb4cf46298e0de8ba0d130e19" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 25 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:21.518682 16217 raft_consensus.cc:2468] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 24.
I20260709 13:22:21.432426 16296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a9de347c711477e87c582d003f36925" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:22:21.519765 16296 raft_consensus.cc:2393] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 in current term 25: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
I20260709 13:22:21.455868 16224 raft_consensus.cc:1275] T d16dfad0c3b540e5b4b86492ddf87208 P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260709 13:22:21.456159 16821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:21.521245 16821 raft_consensus.cc:2468] T 19baa7bc367844c288bca8f81439c413 P cb82cc7bd6db46d980c75237a1402860 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 23.
I20260709 13:22:21.281266 23785 raft_consensus.cc:515] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:21.457823 16142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87147731a68400e8cce9b879daa7c5d" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:22:21.433017 16818 rpcz_store.cc:276] 0709 13:22:18.962718 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:22:18.968193 (+  5475us) service_pool.cc:224] Handling call
0709 13:22:18.968480 (+   287us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:22:21.132256 (+2163776us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:22:21.432121 (+299865us) spinlock_profiling.cc:243] Waited 2.16 s on lock 0x7b54007c3b54. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:22:21.432344 (+   223us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4972243200}
I20260709 13:22:21.509639 16145 raft_consensus.cc:2468] T 18ad7e3af5504571a69d10c00750a01c P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 17.
I20260709 13:22:21.469259 16144 raft_consensus.cc:2468] T 84f2da98864b4425bf7c0055951fdd4b P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 18.
I20260709 13:22:21.490012 16226 raft_consensus.cc:1275] T 02838c86531d4f98bf58d1c22b8baf77 P 1ca2725d290e47d68b1a4b609c9816ac [term 22 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 21 index: 12. Preceding OpId from leader: term: 22 index: 13. (index mismatch)
I20260709 13:22:21.530264 16145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d4ab870a05347868956aa8b841b2981" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:22:21.427662 16146 raft_consensus.cc:2468] T 23f82224f4564fea93572e8206bad1fe P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 18.
W20260709 13:22:21.369750 23840 log.cc:927] Time spent T ecc1e363230043fe9fef390f0b979f20 P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.093s	user 0.000s	sys 0.000s
I20260709 13:22:21.497354 16149 raft_consensus.cc:2468] T 2794af5b99984efebdee564035841cb0 P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 19.
I20260709 13:22:21.475601 16790 leader_election.cc:304] T 806d7690c2b644959152074bb2cab0a7 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:22:21.496881 16148 raft_consensus.cc:2468] T 746aabf856ab401c9ef5e859371e7b67 P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 18.
I20260709 13:22:21.337908 16266 leader_election.cc:304] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:22:21.489117 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0eab25ce4c64e018a041beeda2867db" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 23 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:21.496842 16150 raft_consensus.cc:2468] T b6f07842d3aa47858df5665874c86f3a P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 20.
I20260709 13:22:21.502972 16816 pending_rounds.cc:85] T 2794af5b99984efebdee564035841cb0 P cb82cc7bd6db46d980c75237a1402860: Aborting all ops after (but not including) 15
I20260709 13:22:21.537855 16142 raft_consensus.cc:2393] T b87147731a68400e8cce9b879daa7c5d P 647a343a542d4a139fe106c19095129b [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 in current term 22: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
I20260709 13:22:21.538272 16816 pending_rounds.cc:107] T 2794af5b99984efebdee564035841cb0 P cb82cc7bd6db46d980c75237a1402860: Aborting uncommitted NO_OP operation due to leader change: 18.16
I20260709 13:22:21.538678 16816 raft_consensus.cc:2889] T 2794af5b99984efebdee564035841cb0 P cb82cc7bd6db46d980c75237a1402860 [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:22:21.537866 16293 raft_consensus.cc:2410] T c0eab25ce4c64e018a041beeda2867db P 90f3da57796b488c9087a2f523adf045 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b for term 23 because replica has last-logged OpId of term: 20 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 10.
I20260709 13:22:21.351024 16117 leader_election.cc:304] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860
W20260709 13:22:21.488636 23843 log.cc:927] Time spent T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 0.179s	user 0.000s	sys 0.001s
I20260709 13:22:21.541867 16149 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46224 (request call id 1288) took 1117 ms. Trace:
I20260709 13:22:21.541673 16218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fffab26427e24d398e2695aad40ca134" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
W20260709 13:22:21.554566 16266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 216754us
I20260709 13:22:21.546124 16143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26354361e9b044cea1c2b86c4882fe92" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
W20260709 13:22:21.558775 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 83315.5us
I20260709 13:22:21.559696 16790 leader_election.cc:304] T 66f75124215f4ca6bef831e174a792d8 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac
I20260709 13:22:21.553442 16149 rpcz_store.cc:276] 0709 13:22:20.424943 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:22:21.541031 (+1116088us) service_pool.cc:224] Handling call
0709 13:22:21.541354 (+   323us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:22:21.541515 (+   161us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:22:21.541547 (+    32us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:22:21.541596 (+    49us) raft_consensus.cc:1645] Marking committed up to 16
0709 13:22:21.541625 (+    29us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:22:21.541679 (+    54us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:22:21.541804 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:22:21.524689 16151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee9ed67006144797b1de6ea8c2013a68" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:22:21.425716 23798 leader_election.cc:290] T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:21.569690 16294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c74490c40243a88e62d1760b5cd814" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:21.570952 16146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3884f864a96d428785895e5463e5728f" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 17 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:22:21.571669 16146 raft_consensus.cc:2393] T 3884f864a96d428785895e5463e5728f P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 17: Already voted for candidate cb82cc7bd6db46d980c75237a1402860 in this term.
W20260709 13:22:21.557302 16117 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 206399us
I20260709 13:22:21.334823 23784 raft_consensus.cc:493] T 8a878999c0024d2abe517484180290fc P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:22:21.423970 23767 raft_consensus.cc:515] T 8c11074579b046d99e585c2818e18a2c P cb82cc7bd6db46d980c75237a1402860 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:21.526019 16265 leader_election.cc:304] T 8158bbd17b994d2eb96f560bd0f1b88d P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:22:21.576083 16790 leader_election.cc:304] T 3884f864a96d428785895e5463e5728f P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860; no voters: 90f3da57796b488c9087a2f523adf045
I20260709 13:22:21.394755 23762 leader_election.cc:290] T fffab26427e24d398e2695aad40ca134 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 21 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:22:21.561188 16151 raft_consensus.cc:2468] T ee9ed67006144797b1de6ea8c2013a68 P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 19.
I20260709 13:22:21.542706 16148 raft_consensus.cc:1275] T d16dfad0c3b540e5b4b86492ddf87208 P 647a343a542d4a139fe106c19095129b [term 21 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260709 13:22:21.310196 23719 raft_consensus.cc:515] T 8cc959fb7f704fba9009c0595467d074 P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:21.584826 16822 raft_consensus.cc:3060] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:22:21.578094 16790 leader_election.cc:304] T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac
I20260709 13:22:21.580176 16295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2794af5b99984efebdee564035841cb0" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 20 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:21.580865 16292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434bff3082d141838068f323bdb0d1ed" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:21.570317 16294 raft_consensus.cc:2468] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 20.
I20260709 13:22:21.503968 23793 raft_consensus.cc:493] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:21.584972 16117 leader_election.cc:304] T af57996a8be84d398f0f023e8d08ef77 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac
I20260709 13:22:21.592047 16790 leader_election.cc:304] T af57996a8be84d398f0f023e8d08ef77 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:22:21.592839 16150 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46224 (request call id 1284) took 1169 ms. Trace:
I20260709 13:22:21.593026 16821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5718c623ebee4d2bb770f0f9ddb2f164" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:22:21.593804 16190 leader_election.cc:304] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:22:21.580425 16148 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40830 (request call id 2224) took 1177 ms. Trace:
W20260709 13:22:21.577227 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 51303.5us
I20260709 13:22:21.380897 23821 raft_consensus.cc:515] T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:21.636576 16291 raft_consensus.cc:1240] T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045 [term 23 LEADER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 22. Current term is 23. Ops: []
W20260709 13:22:21.620421 16191 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:22:21.600646 16823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "829b4ad8c5a54eababeea1181b960d4b" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:21.638134 16148 rpcz_store.cc:276] 0709 13:22:20.402952 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:22:21.542007 (+1139055us) service_pool.cc:224] Handling call
0709 13:22:21.542327 (+   320us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:22:21.580186 (+ 37859us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:22:21.580374 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:22:21.648754 16823 raft_consensus.cc:2468] T 829b4ad8c5a54eababeea1181b960d4b P cb82cc7bd6db46d980c75237a1402860 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 19.
I20260709 13:22:21.639494 22677 raft_consensus.cc:2804] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:22:21.612120 16266 leader_election.cc:304] T 8638ee363273426db1393dd81e5c7b72 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860
I20260709 13:22:21.649835 22677 raft_consensus.cc:493] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Starting leader election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:22:21.650264 22677 raft_consensus.cc:3060] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:22:21.527021 23772 raft_consensus.cc:515] T 9e65676b21b748728ad39668cfe722ec P 647a343a542d4a139fe106c19095129b [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:21.648005 23793 raft_consensus.cc:3060] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:22:21.581399 23775 raft_consensus.cc:3060] T 806d7690c2b644959152074bb2cab0a7 P cb82cc7bd6db46d980c75237a1402860 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:22:21.646021 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada5a004ba7c46819051e3ccec71ffdb" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 23 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:21.692696 16293 raft_consensus.cc:2468] T ada5a004ba7c46819051e3ccec71ffdb P 90f3da57796b488c9087a2f523adf045 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 22.
I20260709 13:22:21.646927 16295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d16dfad0c3b540e5b4b86492ddf87208" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:21.697467 16816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434bff3082d141838068f323bdb0d1ed" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
W20260709 13:22:21.698107 16117 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 113202us
I20260709 13:22:21.581938 23784 raft_consensus.cc:3060] T 8a878999c0024d2abe517484180290fc P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:22:21.603871 16151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d4ab870a05347868956aa8b841b2981" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:22:21.655092 16292 raft_consensus.cc:1240] T 7c2095a5f4ad49408dbc7f69c606120d P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 18. Current term is 19. Ops: [18.9-18.9]
W20260709 13:22:21.655557 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 63627.7us
I20260709 13:22:21.704341 23795 leader_election.cc:290] T c9944d5e4b8c4109af9955c4f08c9095 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:21.656165 16294 raft_consensus.cc:1240] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045 [term 21 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 20. Current term is 21. Ops: [20.14-20.14]
I20260709 13:22:21.704659 16790 leader_election.cc:304] T 2794af5b99984efebdee564035841cb0 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:22:21.705772 16790 leader_election.cc:304] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:22:21.706271 16294 raft_consensus.cc:1240] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 19. Current term is 20. Ops: []
I20260709 13:22:21.706497 16824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a9de347c711477e87c582d003f36925" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:21.542903 23777 raft_consensus.cc:697] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 [term 22 LEADER]: Becoming Leader. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:22:21.707369 23286 raft_consensus.cc:2804] T af57996a8be84d398f0f023e8d08ef77 P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:22:21.707841 23286 raft_consensus.cc:493] T af57996a8be84d398f0f023e8d08ef77 P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Starting leader election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:22:21.657752 16146 raft_consensus.cc:1275] T 2794af5b99984efebdee564035841cb0 P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 16 index: 14. Preceding OpId from leader: term: 19 index: 16. (index mismatch)
I20260709 13:22:21.708249 23286 raft_consensus.cc:3060] T af57996a8be84d398f0f023e8d08ef77 P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:22:21.676738 23306 consensus_queue.cc:1059] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: INVALID_TERM, Last received: 20.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.032s
I20260709 13:22:21.708688 16146 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40830 (request call id 2227) took 1056 ms. Trace:
W20260709 13:22:21.639933 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 46228.8us
I20260709 13:22:21.709362 23306 raft_consensus.cc:3055] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 13:22:21.709973 23306 raft_consensus.cc:740] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:22:21.710623 16190 leader_election.cc:304] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: cb82cc7bd6db46d980c75237a1402860
I20260709 13:22:21.711421 23789 raft_consensus.cc:2749] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:22:21.665660 22677 raft_consensus.cc:515] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:21.712231 23772 leader_election.cc:290] T 9e65676b21b748728ad39668cfe722ec P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:21.712975 16824 raft_consensus.cc:2393] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 25: Already voted for candidate cb82cc7bd6db46d980c75237a1402860 in this term.
I20260709 13:22:21.572384 23730 raft_consensus.cc:2804] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:22:21.568356 23603 raft_consensus.cc:2749] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 13:22:21.637552 16150 rpcz_store.cc:276] 0709 13:22:20.423674 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:22:21.551563 (+1127889us) service_pool.cc:224] Handling call
0709 13:22:21.592140 (+ 40577us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:22:21.592386 (+   246us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:22:21.592430 (+    44us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:22:21.592504 (+    74us) raft_consensus.cc:1645] Marking committed up to 15
0709 13:22:21.592555 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:22:21.592635 (+    80us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:22:21.592782 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:22:21.677719 16148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fffab26427e24d398e2695aad40ca134" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:22:21.729427 16148 raft_consensus.cc:3060] T fffab26427e24d398e2695aad40ca134 P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:22:21.677872 16817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66f75124215f4ca6bef831e174a792d8" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:21.679225 16142 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a9de347c711477e87c582d003f36925" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:22:21.656769 16265 leader_election.cc:304] T c4f65cdf00b748bb963ee6e81ef43b79 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac
I20260709 13:22:21.601112 23776 raft_consensus.cc:2804] T 3884f864a96d428785895e5463e5728f P cb82cc7bd6db46d980c75237a1402860 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:22:21.709142 16146 rpcz_store.cc:276] 0709 13:22:20.652382 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:22:21.583688 (+931306us) service_pool.cc:224] Handling call
0709 13:22:21.657400 (+ 73712us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:22:21.708431 (+ 51031us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:22:21.708634 (+   203us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:22:21.716683 16151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a96373bb0e14dce8a266b2fc6c58507" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:22:21.718133 22677 leader_election.cc:290] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 20 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:21.718525 23730 raft_consensus.cc:493] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:21.719087 23558 leader_election.cc:290] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 20 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:22:21.718554 16824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746aabf856ab401c9ef5e859371e7b67" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:21.722435 23287 raft_consensus.cc:2749] T 8638ee363273426db1393dd81e5c7b72 P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:22:21.723275 16266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 111288us
I20260709 13:22:21.647115 23210 consensus_queue.cc:260] T c9944d5e4b8c4109af9955c4f08c9095 P 90f3da57796b488c9087a2f523adf045 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 22.16, Last appended by leader: 16, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:21.637516 23800 consensus_queue.cc:1048] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.087s
I20260709 13:22:21.691880 23398 raft_consensus.cc:2804] T af57996a8be84d398f0f023e8d08ef77 P cb82cc7bd6db46d980c75237a1402860 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:22:21.682142 23024 raft_consensus.cc:493] T 31969392ccdb43b491911a319e8bc7e1 P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:22:21.601531 23625 raft_consensus.cc:2804] T 8158bbd17b994d2eb96f560bd0f1b88d P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:22:21.646710 23645 raft_consensus.cc:2804] T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:22:21.731530 16142 raft_consensus.cc:2376] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b [term 25 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 25. Re-sending same reply.
I20260709 13:22:21.756968 16295 raft_consensus.cc:1240] T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045 [term 23 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 22. Current term is 23. Ops: [22.12-22.12]
I20260709 13:22:21.734987 16147 raft_consensus.cc:2468] T 68a6838d82ef4cac83507f906139d90f P 647a343a542d4a139fe106c19095129b [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 21.
W20260709 13:22:21.736208 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 79552us
I20260709 13:22:21.739651 16151 raft_consensus.cc:2393] T 5a96373bb0e14dce8a266b2fc6c58507 P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 18: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
W20260709 13:22:21.740926 16790 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:22:21.743084 16266 leader_election.cc:304] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860
I20260709 13:22:21.744306 16218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9944d5e4b8c4109af9955c4f08c9095" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:21.754885 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62fe3346cd3c42c4a55e14af9073bca4" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 27 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:22:21.745604 16115 leader_election.cc:304] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac
I20260709 13:22:21.709388 23450 raft_consensus.cc:2804] T 2794af5b99984efebdee564035841cb0 P cb82cc7bd6db46d980c75237a1402860 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:22:21.756762 16816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66ae176961b74813a58bc15c9c4c6f2a" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 27 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:21.758172 16147 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40830 (request call id 2225) took 1355 ms. Trace:
I20260709 13:22:21.764807 16147 rpcz_store.cc:276] 0709 13:22:20.429361 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:22:21.433283 (+1003922us) service_pool.cc:224] Handling call
0709 13:22:21.758116 (+324833us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:22:21.766872 16148 raft_consensus.cc:2468] T fffab26427e24d398e2695aad40ca134 P 647a343a542d4a139fe106c19095129b [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 21.
I20260709 13:22:21.768074 16115 leader_election.cc:304] T 5a959dbecc2648eb85d1cc550f5a3245 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:22:21.769668 16824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b7d1ab519044eda2d5e04a07242f25" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:21.728556 23798 raft_consensus.cc:493] T 7c2095a5f4ad49408dbc7f69c606120d P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:21.736759 23601 raft_consensus.cc:2804] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:22:21.774355 16265 leader_election.cc:304] T 02838c86531d4f98bf58d1c22b8baf77 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:22:21.756938 23645 raft_consensus.cc:493] T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860 [term 23 FOLLOWER]: Starting leader election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:22:21.816764 23645 raft_consensus.cc:3060] T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:22:21.812566 23603 raft_consensus.cc:2804] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:22:21.775689 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68a6838d82ef4cac83507f906139d90f" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:22:21.819355 23603 raft_consensus.cc:697] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b [term 21 LEADER]: Becoming Leader. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:22:21.820081 16150 raft_consensus.cc:2393] T 68a6838d82ef4cac83507f906139d90f P 647a343a542d4a139fe106c19095129b [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 21: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
I20260709 13:22:21.820550 23603 consensus_queue.cc:237] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.13, Last appended by leader: 13, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } }
I20260709 13:22:21.771714 16817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e9b2bf7011f4ae688a79d7bb5a447b4" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:22:21.823288 23601 raft_consensus.cc:493] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:21.823733 23601 raft_consensus.cc:3060] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:22:21.823822 16151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d16dfad0c3b540e5b4b86492ddf87208" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:22:21.824599 16151 raft_consensus.cc:2468] T d16dfad0c3b540e5b4b86492ddf87208 P 647a343a542d4a139fe106c19095129b [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 21.
I20260709 13:22:21.825380 16146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8158bbd17b994d2eb96f560bd0f1b88d" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:22:21.710877 23306 consensus_queue.cc:260] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 23.13, Last appended by leader: 13, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:21.827273 23306 raft_consensus.cc:3060] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:22:21.827867 23710 raft_consensus.cc:2749] T 02838c86531d4f98bf58d1c22b8baf77 P 90f3da57796b488c9087a2f523adf045 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 13:22:21.829022 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 54787.1us
I20260709 13:22:21.831777 23645 raft_consensus.cc:515] T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:22:21.833640 23558 consensus_queue.cc:1175] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "647a343a542d4a139fe106c19095129b" responder_term: 24 status { last_received { term: 22 index: 13 } last_committed_idx: 11 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 23. Current term is 24. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:22:21.836416 16265 leader_election.cc:304] T c0eab25ce4c64e018a041beeda2867db P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac
I20260709 13:22:21.741807 23730 raft_consensus.cc:3060] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:22:21.840528 23719 leader_election.cc:290] T 8cc959fb7f704fba9009c0595467d074 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 19 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:21.821108 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f5f89186204242bf377f7c42b2e2da" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 24 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:22:21.842866 16150 raft_consensus.cc:2393] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 24: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
W20260709 13:22:21.845436 16266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 102436us
I20260709 13:22:21.784600 16115 leader_election.cc:304] T 5a96373bb0e14dce8a266b2fc6c58507 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac
I20260709 13:22:21.848333 16148 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:22:21.849182 16148 raft_consensus.cc:2468] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 23.
I20260709 13:22:21.852195 16219 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57604 (request call id 3504) took 1118 ms. Trace:
I20260709 13:22:21.850282 16265 leader_election.cc:304] T b87147731a68400e8cce9b879daa7c5d P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:22:21.852782 16219 rpcz_store.cc:276] 0709 13:22:20.733287 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:22:20.851336 (+118049us) service_pool.cc:224] Handling call
0709 13:22:20.857958 (+  6622us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:22:20.867450 (+  9492us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:22:20.867485 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:22:20.870044 (+  2559us) log.cc:844] Serialized 26 byte log entry
0709 13:22:21.024670 (+154626us) spinlock_profiling.cc:243] Waited 2.37 ms on lock 0x7b18003dcc00. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b693ade1 00007f69b69475b1 00007f69ada54e07 00007f69b68afab9 00007f69ada7169b 00007f69ada6f93f 00007f69ada71c02 00007f69b06f920b 00007f69b06fed23 00007f69b06ff0d1 00007f69b0b1aa0f
0709 13:22:21.039663 (+ 14993us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:22:21.039715 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:22:21.039788 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:22:21.842926 (+803138us) raft_consensus.cc:1685] finished
0709 13:22:21.842956 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:22:21.852111 (+  9155us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":12577,"raft.queue_time_us":6088,"raft.run_cpu_time_us":129,"raft.run_wall_time_us":5168,"spinlock_wait_cycles":5444480,"thread_start_us":14785,"threads_started":1,"wal-append.queue_time_us":303285}
I20260709 13:22:21.855604 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4dc6b0541ca431ebc757beac00521f1" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:22:21.751725 23533 raft_consensus.cc:2804] T c4f65cdf00b748bb963ee6e81ef43b79 P 90f3da57796b488c9087a2f523adf045 [term 21 FOLLOWER]: Leader pre-election won for term 22
W20260709 13:22:21.866108 16115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 81604.6us
I20260709 13:22:21.860575 16266 leader_election.cc:304] T a378d7f796f94615b3b347119a9880fd P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860
I20260709 13:22:21.859334 16150 raft_consensus.cc:2393] T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 22: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
I20260709 13:22:21.863641 23772 raft_consensus.cc:493] T 57a744f5aaf944678a63469cbf514015 P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:22:21.867743 16816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f82224f4564fea93572e8206bad1fe" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:21.763999 23450 raft_consensus.cc:493] T 2794af5b99984efebdee564035841cb0 P cb82cc7bd6db46d980c75237a1402860 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:21.754142 23398 raft_consensus.cc:493] T af57996a8be84d398f0f023e8d08ef77 P cb82cc7bd6db46d980c75237a1402860 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:22:21.876927 16816 raft_consensus.cc:2393] T 23f82224f4564fea93572e8206bad1fe P cb82cc7bd6db46d980c75237a1402860 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 18: Already voted for candidate cb82cc7bd6db46d980c75237a1402860 in this term.
I20260709 13:22:21.878283 16147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c2095a5f4ad49408dbc7f69c606120d" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 19 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:22:21.879103 16117 leader_election.cc:304] T 829b4ad8c5a54eababeea1181b960d4b P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac
I20260709 13:22:21.880484 23762 consensus_queue.cc:1059] T 26354361e9b044cea1c2b86c4882fe92 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:22:21.842012 23735 raft_consensus.cc:2749] T 5a959dbecc2648eb85d1cc550f5a3245 P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:22:21.913225 16146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8158bbd17b994d2eb96f560bd0f1b88d" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:22:21.771188 23776 raft_consensus.cc:697] T 3884f864a96d428785895e5463e5728f P cb82cc7bd6db46d980c75237a1402860 [term 17 LEADER]: Becoming Leader. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:22:21.708261 23777 consensus_queue.cc:237] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.16, Last appended by leader: 16, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:21.755338 23831 raft_consensus.cc:515] T 8cc959fb7f704fba9009c0595467d074 P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:21.763689 23024 raft_consensus.cc:515] T 31969392ccdb43b491911a319e8bc7e1 P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:21.865960 23738 raft_consensus.cc:2749] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 90f3da57796b488c9087a2f523adf045 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:22:21.998553 16117 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 119561us
I20260709 13:22:21.913285 16151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:22:21.993985 16145 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40830 (request call id 2226) took 1342 ms. Trace:
I20260709 13:22:22.030251 16819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2372981006d4b218215c28964dea623" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 26 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:22.038141 16819 raft_consensus.cc:2468] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 25.
I20260709 13:22:22.038705 16117 leader_election.cc:304] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860
I20260709 13:22:21.865077 23533 raft_consensus.cc:493] T c4f65cdf00b748bb963ee6e81ef43b79 P 90f3da57796b488c9087a2f523adf045 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:22:22.000488 16146 raft_consensus.cc:1240] T 02838c86531d4f98bf58d1c22b8baf77 P 647a343a542d4a139fe106c19095129b [term 22 LEADER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 21. Current term is 22. Ops: []
I20260709 13:22:22.029031 16816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2794af5b99984efebdee564035841cb0" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 19 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:22:22.031562 16149 raft_consensus.cc:1275] T c0eab25ce4c64e018a041beeda2867db P 647a343a542d4a139fe106c19095129b [term 22 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 22 index: 12. (index mismatch)
I20260709 13:22:21.853490 23798 raft_consensus.cc:515] T 7c2095a5f4ad49408dbc7f69c606120d P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:22:22.020247 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 76701us
W20260709 13:22:21.995857 16190 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:22:22.049636 16819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e4bfb9090c8460eb14fa7e0d818e7f1" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:21.877185 23357 raft_consensus.cc:2804] T 5a96373bb0e14dce8a266b2fc6c58507 P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:22:22.037955 16145 rpcz_store.cc:276] 0709 13:22:20.652068 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:22:21.569143 (+917075us) service_pool.cc:224] Handling call
0709 13:22:21.569438 (+   295us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:22:21.713488 (+144050us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:22:21.713527 (+    39us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:22:21.736282 (+ 22755us) raft_consensus.cc:1645] Marking committed up to 15
0709 13:22:21.736338 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:22:21.993712 (+257374us) spinlock_profiling.cc:243] Waited 113 ms on lock 0x7b54003f0f54. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:22:21.993762 (+    50us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:22:21.993912 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":30427,"raft.queue_time_us":30704,"raft.run_cpu_time_us":138,"raft.run_wall_time_us":14331,"spinlock_wait_cycles":260382592}
W20260709 13:22:21.877329 23823 log.cc:927] Time spent T 746aabf856ab401c9ef5e859371e7b67 P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 0.098s	user 0.000s	sys 0.000s
I20260709 13:22:21.971719 16116 leader_election.cc:304] T ee9ed67006144797b1de6ea8c2013a68 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860
I20260709 13:22:22.040585 16149 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40844 (request call id 3096) took 1240 ms. Trace:
I20260709 13:22:22.055698 16145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31969392ccdb43b491911a319e8bc7e1" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:22:22.055639 16149 rpcz_store.cc:276] 0709 13:22:20.799878 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:22:21.968787 (+1168909us) service_pool.cc:224] Handling call
0709 13:22:22.031146 (+ 62359us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:22:22.040286 (+  9140us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:22:22.040524 (+   238us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:22:22.053000 16790 leader_election.cc:304] T b87147731a68400e8cce9b879daa7c5d P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:22:21.883505 23210 raft_consensus.cc:3060] T c9944d5e4b8c4109af9955c4f08c9095 P 90f3da57796b488c9087a2f523adf045 [term 22 FOLLOWER]: Advancing to term 23
W20260709 13:22:21.986414 23828 log.cc:927] Time spent T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 0.119s	user 0.001s	sys 0.000s
I20260709 13:22:22.024262 23785 leader_election.cc:290] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:21.909782 23685 raft_consensus.cc:493] T 9e65676b21b748728ad39668cfe722ec P cb82cc7bd6db46d980c75237a1402860 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:22.044693 23533 raft_consensus.cc:3060] T c4f65cdf00b748bb963ee6e81ef43b79 P 90f3da57796b488c9087a2f523adf045 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:22:22.050357 16819 raft_consensus.cc:2468] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P cb82cc7bd6db46d980c75237a1402860 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 22.
I20260709 13:22:21.969127 23648 raft_consensus.cc:493] T 74ca8140f35c4ae29e8a664f6bdfc829 P cb82cc7bd6db46d980c75237a1402860 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:22.088879 16147 raft_consensus.cc:3060] T 7c2095a5f4ad49408dbc7f69c606120d P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:22:22.053027 16146 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40844 (request call id 3094) took 1253 ms. Trace:
I20260709 13:22:22.008720 16824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5d59407eb545b895d8d922e0dd69fd" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:22.057778 16298 raft_consensus.cc:2368] T 8cc959fb7f704fba9009c0595467d074 P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b for earlier term 19. Current term is 20.
I20260709 13:22:21.935029 23645 leader_election.cc:290] T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 24 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:22:21.910637 23450 raft_consensus.cc:3060] T 2794af5b99984efebdee564035841cb0 P cb82cc7bd6db46d980c75237a1402860 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:22:22.060356 16223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b7d1ab519044eda2d5e04a07242f25" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:21.969345 23398 raft_consensus.cc:3060] T af57996a8be84d398f0f023e8d08ef77 P cb82cc7bd6db46d980c75237a1402860 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:22:22.066972 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cc959fb7f704fba9009c0595467d074" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:22:22.029680 23762 raft_consensus.cc:2804] T b87147731a68400e8cce9b879daa7c5d P 90f3da57796b488c9087a2f523adf045 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:22:22.026190 23719 raft_consensus.cc:2804] T 829b4ad8c5a54eababeea1181b960d4b P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:22:21.936615 23625 raft_consensus.cc:493] T 8158bbd17b994d2eb96f560bd0f1b88d P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:22:22.022116 23778 leader_election.cc:290] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:22.071085 16218 raft_consensus.cc:1240] T ecc1e363230043fe9fef390f0b979f20 P 1ca2725d290e47d68b1a4b609c9816ac [term 24 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 23. Current term is 24. Ops: []
I20260709 13:22:22.072501 16151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5d59407eb545b895d8d922e0dd69fd" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
I20260709 13:22:22.073711 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:22:22.082597 23815 leader_election.cc:290] T 1df93779209748b9860f6d893dd3de53 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
W20260709 13:22:22.083243 16263 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:22:22.084255 23605 raft_consensus.cc:2749] T ee9ed67006144797b1de6ea8c2013a68 P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:22:22.137107 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26354361e9b044cea1c2b86c4882fe92" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:22:22.085602 23825 raft_consensus.cc:2749] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b [term 25 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:22:22.091790 16146 rpcz_store.cc:276] 0709 13:22:20.799379 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:22:21.914664 (+1115285us) service_pool.cc:224] Handling call
0709 13:22:22.000263 (+ 85599us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:22:22.052766 (+ 52503us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:22:22.052968 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:22:22.097788 16819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87147731a68400e8cce9b879daa7c5d" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:22.083273 23821 leader_election.cc:290] T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 23 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:22:22.099501 16824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada5a004ba7c46819051e3ccec71ffdb" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 23 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:21.999234 23772 raft_consensus.cc:515] T 57a744f5aaf944678a63469cbf514015 P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:22.066404 23795 raft_consensus.cc:2749] T b87147731a68400e8cce9b879daa7c5d P cb82cc7bd6db46d980c75237a1402860 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
W20260709 13:22:22.111212 16820 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.692s	user 0.003s	sys 0.028s
I20260709 13:22:22.125680 16151 raft_consensus.cc:3060] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:22:22.132537 16147 raft_consensus.cc:2468] T 7c2095a5f4ad49408dbc7f69c606120d P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 19.
I20260709 13:22:22.077891 23357 raft_consensus.cc:697] T 5a96373bb0e14dce8a266b2fc6c58507 P 647a343a542d4a139fe106c19095129b [term 18 LEADER]: Becoming Leader. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:22:22.138566 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8638ee363273426db1393dd81e5c7b72" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:22.182036 16297 raft_consensus.cc:2468] T 8638ee363273426db1393dd81e5c7b72 P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 20.
I20260709 13:22:22.125617 23625 raft_consensus.cc:3060] T 8158bbd17b994d2eb96f560bd0f1b88d P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:22:22.116627 23719 raft_consensus.cc:493] T 829b4ad8c5a54eababeea1181b960d4b P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:22.183190 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2794af5b99984efebdee564035841cb0" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 20 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:22.194445 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e4bfb9090c8460eb14fa7e0d818e7f1" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:22.101722 23738 raft_consensus.cc:2749] T a378d7f796f94615b3b347119a9880fd P 90f3da57796b488c9087a2f523adf045 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:22:22.138494 23762 raft_consensus.cc:697] T b87147731a68400e8cce9b879daa7c5d P 90f3da57796b488c9087a2f523adf045 [term 22 LEADER]: Becoming Leader. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
W20260709 13:22:22.176384 16820 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.757s	user 0.003s	sys 0.028s
I20260709 13:22:22.087800 23685 raft_consensus.cc:515] T 9e65676b21b748728ad39668cfe722ec P cb82cc7bd6db46d980c75237a1402860 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:22.189402 16296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7ca61592e848ea9b0fb81a2ce33cdc" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:22.217326 16296 raft_consensus.cc:2468] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 90f3da57796b488c9087a2f523adf045 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 21.
I20260709 13:22:22.222895 16789 leader_election.cc:304] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:22:22.178726 16300 raft_consensus.cc:1240] T 57a744f5aaf944678a63469cbf514015 P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 18. Current term is 19. Ops: [18.14-18.14]
I20260709 13:22:22.102572 23648 raft_consensus.cc:515] T 74ca8140f35c4ae29e8a664f6bdfc829 P cb82cc7bd6db46d980c75237a1402860 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:22.225121 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9944d5e4b8c4109af9955c4f08c9095" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
W20260709 13:22:22.208681 16822 consensus_meta.cc:287] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860: Time spent flushing consensus metadata: real 0.623s	user 0.009s	sys 0.002s
I20260709 13:22:22.226202 16822 raft_consensus.cc:2468] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 24.
I20260709 13:22:22.211894 16226 raft_consensus.cc:2468] T 8cc959fb7f704fba9009c0595467d074 P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 19.
I20260709 13:22:22.228623 16789 leader_election.cc:304] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac
I20260709 13:22:22.231884 16296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31969392ccdb43b491911a319e8bc7e1" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:22.232568 16296 raft_consensus.cc:2468] T 31969392ccdb43b491911a319e8bc7e1 P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 18.
I20260709 13:22:22.233508 16296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c11074579b046d99e585c2818e18a2c" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:22.234596 16296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:22.235486 16824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdbd78acb4cf46298e0de8ba0d130e19" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 25 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:22.240322 16824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3884f864a96d428785895e5463e5728f" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 17 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:22:22.240828 16819 raft_consensus.cc:2393] T b87147731a68400e8cce9b879daa7c5d P cb82cc7bd6db46d980c75237a1402860 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 22: Already voted for candidate cb82cc7bd6db46d980c75237a1402860 in this term.
I20260709 13:22:22.241436 16824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e65676b21b748728ad39668cfe722ec" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 22 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
W20260709 13:22:22.241739 23774 consensus_meta.cc:287] T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b: Time spent flushing consensus metadata: real 0.802s	user 0.007s	sys 0.001s
I20260709 13:22:22.242228 23774 raft_consensus.cc:515] T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:22.202625 23719 raft_consensus.cc:3060] T 829b4ad8c5a54eababeea1181b960d4b P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:22:22.243266 16145 raft_consensus.cc:1240] T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b [term 22 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 21. Current term is 22. Ops: []
I20260709 13:22:22.244454 23772 leader_election.cc:290] T 57a744f5aaf944678a63469cbf514015 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:22.244773 16145 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40844 (request call id 3099) took 1430 ms. Trace:
I20260709 13:22:22.245167 16145 rpcz_store.cc:276] 0709 13:22:20.814433 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:22:22.063268 (+1248835us) service_pool.cc:224] Handling call
0709 13:22:22.063591 (+   323us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:22:22.243784 (+180193us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:22:22.244559 (+   775us) spinlock_profiling.cc:243] Waited 179 ms on lock 0x7b54000d1454. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:22:22.244717 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":412723072}
I20260709 13:22:22.246805 23685 leader_election.cc:290] T 9e65676b21b748728ad39668cfe722ec P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
W20260709 13:22:22.247828 16144 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40830 (request call id 2220) took 1846 ms (client timeout 1518 ms). Trace:
W20260709 13:22:22.248387 16144 rpcz_store.cc:269] 0709 13:22:20.401747 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:22:20.405068 (+  3321us) service_pool.cc:224] Handling call
0709 13:22:22.247605 (+1842537us) spinlock_profiling.cc:243] Waited 976 ms on lock 0x7b54000d1954. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:22:22.247741 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2245684608}
I20260709 13:22:22.176975 23767 leader_election.cc:290] T 8c11074579b046d99e585c2818e18a2c P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:22.253036 23762 consensus_queue.cc:237] T b87147731a68400e8cce9b879daa7c5d P 90f3da57796b488c9087a2f523adf045 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.13, Last appended by leader: 13, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:22.256752 16296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e9b2bf7011f4ae688a79d7bb5a447b4" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:22:22.257835 23694 leader_election.cc:290] T 829b4ad8c5a54eababeea1181b960d4b P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
W20260709 13:22:22.217942 23793 consensus_meta.cc:287] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860: Time spent flushing consensus metadata: real 0.528s	user 0.009s	sys 0.001s
I20260709 13:22:22.171965 23776 consensus_queue.cc:237] T 3884f864a96d428785895e5463e5728f P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 14.14, Last appended by leader: 14, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:22.242986 16819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2794af5b99984efebdee564035841cb0" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 20 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
W20260709 13:22:22.263125 16790 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:22:22.263784 23852 log.cc:927] Time spent T 87ac0d1f7e8a419f9bcbc04aee0e067e P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.071s	user 0.000s	sys 0.000s
I20260709 13:22:22.264731 23357 consensus_queue.cc:237] T 5a96373bb0e14dce8a266b2fc6c58507 P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:22.265460 16147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e65676b21b748728ad39668cfe722ec" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 22 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:22:22.266753 16147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "829b4ad8c5a54eababeea1181b960d4b" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b" is_pre_election: true
I20260709 13:22:22.267547 16150 raft_consensus.cc:1240] T 7c2095a5f4ad49408dbc7f69c606120d P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 18. Current term is 19. Ops: [18.9-18.9]
I20260709 13:22:22.235771 23831 leader_election.cc:290] T 8cc959fb7f704fba9009c0595467d074 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 20 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:22:22.268013 16151 raft_consensus.cc:2468] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 24.
I20260709 13:22:22.267831 16824 raft_consensus.cc:2468] T 9e65676b21b748728ad39668cfe722ec P cb82cc7bd6db46d980c75237a1402860 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 21.
I20260709 13:22:22.274960 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cc959fb7f704fba9009c0595467d074" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:22:22.277396 16144 raft_consensus.cc:1240] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [term 24 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 23. Current term is 24. Ops: []
I20260709 13:22:22.279206 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e65676b21b748728ad39668cfe722ec" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 22 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:22.258730 23793 raft_consensus.cc:515] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:22:22.280067 16789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 51534.8us
I20260709 13:22:22.282217 16789 leader_election.cc:304] T 434bff3082d141838068f323bdb0d1ed P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:22:22.281409 23778 raft_consensus.cc:493] T 23f82224f4564fea93572e8206bad1fe P cb82cc7bd6db46d980c75237a1402860 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:22.270924 23785 raft_consensus.cc:2749] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860 [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 13:22:22.292898 16789 leader_election.cc:304] T ada5a004ba7c46819051e3ccec71ffdb P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac
I20260709 13:22:22.294121 23694 raft_consensus.cc:2749] T 434bff3082d141838068f323bdb0d1ed P cb82cc7bd6db46d980c75237a1402860 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:22:22.297482 16789 leader_election.cc:304] T 26354361e9b044cea1c2b86c4882fe92 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:22:22.298905 23789 raft_consensus.cc:2804] T ada5a004ba7c46819051e3ccec71ffdb P cb82cc7bd6db46d980c75237a1402860 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:22:22.299407 23789 raft_consensus.cc:493] T ada5a004ba7c46819051e3ccec71ffdb P cb82cc7bd6db46d980c75237a1402860 [term 22 FOLLOWER]: Starting leader election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:22:22.299799 23789 raft_consensus.cc:3060] T ada5a004ba7c46819051e3ccec71ffdb P cb82cc7bd6db46d980c75237a1402860 [term 22 FOLLOWER]: Advancing to term 23
W20260709 13:22:22.302567 23849 log.cc:927] Time spent T 66f75124215f4ca6bef831e174a792d8 P cb82cc7bd6db46d980c75237a1402860: Append to log took a long time: real 0.262s	user 0.000s	sys 0.003s
I20260709 13:22:22.298621 23450 raft_consensus.cc:515] T 2794af5b99984efebdee564035841cb0 P cb82cc7bd6db46d980c75237a1402860 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:22.303411 16789 leader_election.cc:304] T 8638ee363273426db1393dd81e5c7b72 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac
I20260709 13:22:22.303815 23645 raft_consensus.cc:2749] T 26354361e9b044cea1c2b86c4882fe92 P cb82cc7bd6db46d980c75237a1402860 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:22:22.304419 16816 raft_consensus.cc:2368] T 2794af5b99984efebdee564035841cb0 P cb82cc7bd6db46d980c75237a1402860 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 for earlier term 19. Current term is 20.
I20260709 13:22:22.305188 23795 raft_consensus.cc:2804] T 8638ee363273426db1393dd81e5c7b72 P cb82cc7bd6db46d980c75237a1402860 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:22:22.305629 23795 raft_consensus.cc:493] T 8638ee363273426db1393dd81e5c7b72 P cb82cc7bd6db46d980c75237a1402860 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:22:22.305828 23450 leader_election.cc:290] T 2794af5b99984efebdee564035841cb0 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 20 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:22.306035 23795 raft_consensus.cc:3060] T 8638ee363273426db1393dd81e5c7b72 P cb82cc7bd6db46d980c75237a1402860 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:22:22.308432 16150 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40844 (request call id 3093) took 1509 ms. Trace:
I20260709 13:22:22.308749 16298 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48674 (request call id 3591) took 1363 ms. Trace:
I20260709 13:22:22.308835 16150 rpcz_store.cc:276] 0709 13:22:20.799105 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:22:21.878308 (+1079203us) service_pool.cc:224] Handling call
0709 13:22:22.267302 (+388994us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:22:22.268372 (+  1070us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:22:22.308228 (+ 39856us) spinlock_profiling.cc:243] Waited 293 ms on lock 0x7b5400072bd0. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5e8ad 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:22:22.308371 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":673921408}
I20260709 13:22:22.309206 16298 rpcz_store.cc:276] 0709 13:22:20.945184 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:22:21.019841 (+ 74657us) service_pool.cc:224] Handling call
0709 13:22:22.308551 (+1288710us) spinlock_profiling.cc:243] Waited 945 ms on lock 0x7b5400178754. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:22:22.308692 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2172881280}
I20260709 13:22:22.310333 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2794af5b99984efebdee564035841cb0" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 20 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "647a343a542d4a139fe106c19095129b"
W20260709 13:22:22.311412 23286 consensus_meta.cc:287] T af57996a8be84d398f0f023e8d08ef77 P 647a343a542d4a139fe106c19095129b: Time spent flushing consensus metadata: real 0.603s	user 0.005s	sys 0.005s
I20260709 13:22:22.311851 23286 raft_consensus.cc:515] T af57996a8be84d398f0f023e8d08ef77 P 647a343a542d4a139fe106c19095129b [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:22.312184 23798 leader_election.cc:290] T 7c2095a5f4ad49408dbc7f69c606120d P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:22.312920 16143 raft_consensus.cc:1240] T af57996a8be84d398f0f023e8d08ef77 P 647a343a542d4a139fe106c19095129b [term 21 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 20. Current term is 21. Ops: []
I20260709 13:22:22.313225 23024 leader_election.cc:290] T 31969392ccdb43b491911a319e8bc7e1 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:22.314218 16143 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40844 (request call id 3095) took 1514 ms. Trace:
I20260709 13:22:22.284216 23685 raft_consensus.cc:2804] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 13:22:22.319514 23685 raft_consensus.cc:493] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [term 26 FOLLOWER]: Starting leader election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:22:22.319748 16819 raft_consensus.cc:1275] T 9e65676b21b748728ad39668cfe722ec P cb82cc7bd6db46d980c75237a1402860 [term 21 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 13:22:22.319938 23685 raft_consensus.cc:3060] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [term 26 FOLLOWER]: Advancing to term 27
I20260709 13:22:22.317539 16824 raft_consensus.cc:1240] T c9944d5e4b8c4109af9955c4f08c9095 P cb82cc7bd6db46d980c75237a1402860 [term 23 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 22. Current term is 23. Ops: []
I20260709 13:22:22.321950 16824 pending_rounds.cc:85] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860: Aborting all ops after (but not including) 14
I20260709 13:22:22.322190 16824 pending_rounds.cc:107] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860: Aborting uncommitted NO_OP operation due to leader change: 22.15
I20260709 13:22:22.322422 16824 raft_consensus.cc:2889] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 [term 25 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:22:22.323963 16144 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40844 (request call id 3104) took 1445 ms. Trace:
I20260709 13:22:22.324353 16144 rpcz_store.cc:276] 0709 13:22:20.878805 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:22:22.249460 (+1370655us) service_pool.cc:224] Handling call
0709 13:22:22.277143 (+ 27683us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:22:22.277883 (+   740us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:22:22.323758 (+ 45875us) spinlock_profiling.cc:243] Waited 27.3 ms on lock 0x7b540012f7d0. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5e8ad 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:22:22.323898 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":62714624}
I20260709 13:22:22.324419 23533 raft_consensus.cc:515] T c4f65cdf00b748bb963ee6e81ef43b79 P 90f3da57796b488c9087a2f523adf045 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:22.325901 23648 leader_election.cc:290] T 74ca8140f35c4ae29e8a664f6bdfc829 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:22.324415 16789 leader_election.cc:304] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:22:22.326900 23533 leader_election.cc:290] T c4f65cdf00b748bb963ee6e81ef43b79 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 22 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:22:22.314728 16143 rpcz_store.cc:276] 0709 13:22:20.799625 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:22:21.918722 (+1119097us) service_pool.cc:224] Handling call
0709 13:22:21.919157 (+   435us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:22:22.313296 (+394139us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:22:22.313953 (+   657us) spinlock_profiling.cc:243] Waited 394 ms on lock 0x7b54000e9a54. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:22:22.314149 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":905112448}
I20260709 13:22:22.310950 16150 raft_consensus.cc:3060] T 2794af5b99984efebdee564035841cb0 P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:22:22.326021 16144 raft_consensus.cc:1240] T fffab26427e24d398e2695aad40ca134 P 647a343a542d4a139fe106c19095129b [term 21 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 20. Current term is 21. Ops: []
I20260709 13:22:22.335289 16144 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40844 (request call id 3107) took 1055 ms. Trace:
I20260709 13:22:22.335465 16817 raft_consensus.cc:2393] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 20: Already voted for candidate cb82cc7bd6db46d980c75237a1402860 in this term.
I20260709 13:22:22.335706 16143 raft_consensus.cc:1275] T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b [term 25 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 23 index: 15. Preceding OpId from leader: term: 25 index: 16. (index mismatch)
I20260709 13:22:22.335561 16789 leader_election.cc:304] T 3a7ca61592e848ea9b0fb81a2ce33cdc P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac
I20260709 13:22:22.335719 16144 rpcz_store.cc:276] 0709 13:22:21.280320 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:22:22.325486 (+1045166us) service_pool.cc:224] Handling call
0709 13:22:22.325827 (+   341us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:22:22.334150 (+  8323us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:22:22.335232 (+  1082us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:22:22.337618 23648 raft_consensus.cc:2749] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P cb82cc7bd6db46d980c75237a1402860 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:22:22.338198 16143 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40844 (request call id 3108) took 1053 ms. Trace:
I20260709 13:22:22.338950 16789 leader_election.cc:304] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 647a343a542d4a139fe106c19095129b
I20260709 13:22:22.338680 16143 rpcz_store.cc:276] 0709 13:22:21.284628 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:22:22.332319 (+1047691us) service_pool.cc:224] Handling call
0709 13:22:22.332704 (+   385us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:22:22.336442 (+  3738us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:22:22.338143 (+  1701us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:22:22.341934 23784 consensus_meta.cc:287] T 8a878999c0024d2abe517484180290fc P 90f3da57796b488c9087a2f523adf045: Time spent flushing consensus metadata: real 0.615s	user 0.005s	sys 0.005s
I20260709 13:22:22.345414 16822 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49906 (request call id 1466) took 1127 ms. Trace:
I20260709 13:22:22.345778 16822 rpcz_store.cc:276] 0709 13:22:21.218065 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:22:21.235862 (+ 17797us) service_pool.cc:224] Handling call
0709 13:22:22.345258 (+1109396us) spinlock_profiling.cc:243] Waited 298 ms on lock 0x7b54007ffdd4. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:22:22.345363 (+   105us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":685489536}
I20260709 13:22:22.348155 16823 raft_consensus.cc:1240] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860 [term 24 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 23. Current term is 24. Ops: []
W20260709 13:22:22.349617 16818 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.639s	user 0.020s	sys 0.023s
I20260709 13:22:22.349583 16789 leader_election.cc:304] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
W20260709 13:22:22.351622 16818 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.642s	user 0.020s	sys 0.023s
I20260709 13:22:22.352429 16789 leader_election.cc:304] T c9944d5e4b8c4109af9955c4f08c9095 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:22:22.350915 16143 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40844 (request call id 3110) took 1042 ms. Trace:
I20260709 13:22:22.359059 16144 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40844 (request call id 3109) took 1058 ms. Trace:
I20260709 13:22:22.359087 16143 rpcz_store.cc:276] 0709 13:22:21.308860 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:22:22.340539 (+1031679us) service_pool.cc:224] Handling call
0709 13:22:22.350447 (+  9908us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:22:22.350618 (+   171us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:22:22.350642 (+    24us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:22:22.350683 (+    41us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:22:22.350714 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:22:22.350766 (+    52us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:22:22.350863 (+    97us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:22:22.359946 16144 rpcz_store.cc:276] 0709 13:22:21.300694 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:22:22.337296 (+1036602us) service_pool.cc:224] Handling call
0709 13:22:22.358220 (+ 20924us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:22:22.358644 (+   424us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:22:22.358689 (+    45us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:22:22.358756 (+    67us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:22:22.358797 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:22:22.358869 (+    72us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:22:22.358998 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:22:22.341181 23559 consensus_meta.cc:287] T e5ad4eb281564936a59f41e387a47156 P 90f3da57796b488c9087a2f523adf045: Time spent flushing consensus metadata: real 1.007s	user 0.006s	sys 0.005s
I20260709 13:22:22.365137 16817 raft_consensus.cc:1275] T c9944d5e4b8c4109af9955c4f08c9095 P cb82cc7bd6db46d980c75237a1402860 [term 23 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 22 index: 16. Preceding OpId from leader: term: 23 index: 17. (index mismatch)
W20260709 13:22:22.378353 16299 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.738s	user 0.013s	sys 0.044s
W20260709 13:22:22.378914 16299 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.738s	user 0.014s	sys 0.044s
I20260709 13:22:22.399797 16150 raft_consensus.cc:2468] T 2794af5b99984efebdee564035841cb0 P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 20.
I20260709 13:22:22.344558 23645 raft_consensus.cc:2804] T 3a7ca61592e848ea9b0fb81a2ce33cdc P cb82cc7bd6db46d980c75237a1402860 [term 21 FOLLOWER]: Leader pre-election won for term 22
W20260709 13:22:22.406649 16789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 54354.9us
I20260709 13:22:22.412870 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2794af5b99984efebdee564035841cb0" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 20 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:22:22.332229 23778 raft_consensus.cc:515] T 23f82224f4564fea93572e8206bad1fe P cb82cc7bd6db46d980c75237a1402860 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:22.424746 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e65676b21b748728ad39668cfe722ec" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 22 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:22.343616 23784 raft_consensus.cc:515] T 8a878999c0024d2abe517484180290fc P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:22.378118 23694 raft_consensus.cc:2749] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860 [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:22:22.404894 23450 raft_consensus.cc:2804] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860 [term 18 FOLLOWER]: Leader pre-election won for term 19
W20260709 13:22:22.427354 16225 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.672s	user 0.022s	sys 0.025s
W20260709 13:22:22.431039 16225 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.686s	user 0.022s	sys 0.025s
W20260709 13:22:22.430297 16291 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.707s	user 0.016s	sys 0.047s
W20260709 13:22:22.435281 16291 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.712s	user 0.016s	sys 0.047s
I20260709 13:22:22.414968 23638 raft_consensus.cc:2749] T c9944d5e4b8c4109af9955c4f08c9095 P cb82cc7bd6db46d980c75237a1402860 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
/apache/dev/git/kudu/src/kudu/master/auto_leader_rebalancer-test.cc:278: Failure
Failed
Bad status: Illegal state: leader_transfer_task size should be 0, but 6
I20260709 13:22:22.378422 23559 raft_consensus.cc:515] T e5ad4eb281564936a59f41e387a47156 P 90f3da57796b488c9087a2f523adf045 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:22.431967 16225 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57604 (request call id 3519) took 1430 ms. Trace:
I20260709 13:22:22.448010 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74ca8140f35c4ae29e8a664f6bdfc829" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 24 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:22.417380 23645 raft_consensus.cc:493] T 3a7ca61592e848ea9b0fb81a2ce33cdc P cb82cc7bd6db46d980c75237a1402860 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:22:22.454104 16294 raft_consensus.cc:1275] T c9944d5e4b8c4109af9955c4f08c9095 P 90f3da57796b488c9087a2f523adf045 [term 23 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 22 index: 16. Preceding OpId from leader: term: 23 index: 17. (index mismatch)
W20260709 13:22:22.452916 23287 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.632s	user 0.028s	sys 0.020s
W20260709 13:22:22.486241 23287 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.666s	user 0.029s	sys 0.020s
/apache/dev/git/kudu/src/kudu/util/test_util.cc:402: Failure
Failed
Timed out waiting for assertion to pass.
I20260709 13:22:22.487993 16790 leader_election.cc:304] T 9e65676b21b748728ad39668cfe722ec P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:22:22.448316 23745 raft_consensus.cc:2804] T c0eab25ce4c64e018a041beeda2867db P 90f3da57796b488c9087a2f523adf045 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:22:22.491817 15046 tablet_server.cc:179] TabletServer@127.14.177.129:0 shutting down...
I20260709 13:22:22.509465 16790 leader_election.cc:304] T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:22:22.483222 23768 raft_consensus.cc:493] T 57a744f5aaf944678a63469cbf514015 P cb82cc7bd6db46d980c75237a1402860 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:22.511260 23638 raft_consensus.cc:2749] T 9e65676b21b748728ad39668cfe722ec P cb82cc7bd6db46d980c75237a1402860 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:22:22.511430 23768 raft_consensus.cc:515] T 57a744f5aaf944678a63469cbf514015 P cb82cc7bd6db46d980c75237a1402860 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:22.512399 23450 raft_consensus.cc:493] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:22:22.512975 23450 raft_consensus.cc:3060] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:22:22.513804 23398 raft_consensus.cc:515] T af57996a8be84d398f0f023e8d08ef77 P cb82cc7bd6db46d980c75237a1402860 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:22.514456 23768 leader_election.cc:290] T 57a744f5aaf944678a63469cbf514015 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:22.449345 23795 raft_consensus.cc:515] T 8638ee363273426db1393dd81e5c7b72 P cb82cc7bd6db46d980c75237a1402860 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:22.515081 16819 raft_consensus.cc:1240] T af57996a8be84d398f0f023e8d08ef77 P cb82cc7bd6db46d980c75237a1402860 [term 21 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 20. Current term is 21. Ops: []
I20260709 13:22:22.515763 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57a744f5aaf944678a63469cbf514015" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:22.516445 16298 raft_consensus.cc:2468] T 57a744f5aaf944678a63469cbf514015 P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 19.
I20260709 13:22:22.517323 16292 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48684 (request call id 3340) took 1378 ms. Trace:
I20260709 13:22:22.517683 16292 rpcz_store.cc:276] 0709 13:22:21.139092 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:22:21.704209 (+565117us) service_pool.cc:224] Handling call
0709 13:22:21.704617 (+   408us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:22:21.705009 (+   392us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:22:21.705043 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:22:21.705532 (+   489us) log.cc:844] Serialized 26 byte log entry
0709 13:22:22.045329 (+339797us) raft_consensus.cc:1645] Marking committed up to 16
0709 13:22:22.045384 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:22:22.045470 (+    86us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:22:22.517117 (+471647us) raft_consensus.cc:1685] finished
0709 13:22:22.517137 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:22:22.517267 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2620,"raft.run_cpu_time_us":92,"raft.run_wall_time_us":92,"thread_start_us":339424,"threads_started":1,"wal-append.queue_time_us":718698}
I20260709 13:22:22.519351 23287 raft_consensus.cc:3055] T 26354361e9b044cea1c2b86c4882fe92 P 90f3da57796b488c9087a2f523adf045 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:22:22.519830 23774 leader_election.cc:290] T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 22 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:22.519930 23287 raft_consensus.cc:740] T 26354361e9b044cea1c2b86c4882fe92 P 90f3da57796b488c9087a2f523adf045 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:22:22.520975 23287 consensus_queue.cc:260] T 26354361e9b044cea1c2b86c4882fe92 P 90f3da57796b488c9087a2f523adf045 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:22.523676 23287 raft_consensus.cc:3060] T 26354361e9b044cea1c2b86c4882fe92 P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:22:22.496790 16220 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57600 (request call id 2242) took 1042 ms. Trace:
I20260709 13:22:22.529770 23778 leader_election.cc:290] T 23f82224f4564fea93572e8206bad1fe P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:22.485741 23645 raft_consensus.cc:3060] T 3a7ca61592e848ea9b0fb81a2ce33cdc P cb82cc7bd6db46d980c75237a1402860 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:22:22.430110 23818 consensus_queue.cc:1048] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.055s
I20260709 13:22:22.529080 16220 rpcz_store.cc:276] 0709 13:22:21.454083 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:22:21.509023 (+ 54940us) service_pool.cc:224] Handling call
0709 13:22:21.568359 (+ 59336us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:22:21.568563 (+   204us) raft_consensus.cc:1534] Early marking committed up to index 14
0709 13:22:22.105070 (+536507us) log.cc:844] Serialized 14 byte log entry
0709 13:22:22.492882 (+387812us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:22:22.492950 (+    68us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:22:22.496519 (+  3569us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:22:22.496604 (+    85us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:22:22.496730 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":618260,"prepare.run_cpu_time_us":56,"prepare.run_wall_time_us":58,"raft.queue_time_us":7008,"thread_start_us":851051,"threads_started":2}
I20260709 13:22:22.535260 23789 raft_consensus.cc:515] T ada5a004ba7c46819051e3ccec71ffdb P cb82cc7bd6db46d980c75237a1402860 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:22.485368 16225 rpcz_store.cc:276] 0709 13:22:21.001274 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:22:21.371683 (+370409us) service_pool.cc:224] Handling call
0709 13:22:21.372447 (+   764us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:22:21.372671 (+   224us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 13:22:21.737295 (+364624us) log.cc:844] Serialized 14 byte log entry
0709 13:22:22.431563 (+694268us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:22:22.431612 (+    49us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:22:22.431661 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:22:22.431721 (+    60us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:22:22.431915 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8775,"prepare.queue_time_us":733187,"prepare.run_cpu_time_us":54,"prepare.run_wall_time_us":10838,"raft.queue_time_us":1787,"raft.run_cpu_time_us":77,"raft.run_wall_time_us":78,"thread_start_us":1050459,"threads_started":2}
I20260709 13:22:22.527396 23630 raft_consensus.cc:493] T 8a878999c0024d2abe517484180290fc P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:22:22.537667 23630 raft_consensus.cc:515] T 8a878999c0024d2abe517484180290fc P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:22.508255 23800 raft_consensus.cc:493] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:22.539465 23800 raft_consensus.cc:515] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:22.539923 16299 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48674 (request call id 3594) took 1509 ms. Trace:
I20260709 13:22:22.541589 23800 leader_election.cc:290] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:22.528311 23784 leader_election.cc:290] T 8a878999c0024d2abe517484180290fc P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 19 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:22:22.542181 23793 leader_election.cc:290] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 20 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:22.543467 23630 leader_election.cc:290] T 8a878999c0024d2abe517484180290fc P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:22.543771 16819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5718c623ebee4d2bb770f0f9ddb2f164" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 23 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:22.545374 16819 raft_consensus.cc:1240] T 8638ee363273426db1393dd81e5c7b72 P cb82cc7bd6db46d980c75237a1402860 [term 21 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 20. Current term is 21. Ops: []
I20260709 13:22:22.546496 16819 raft_consensus.cc:1240] T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860 [term 24 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 23. Current term is 24. Ops: []
W20260709 13:22:22.549768 23730 consensus_meta.cc:287] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 90f3da57796b488c9087a2f523adf045: Time spent flushing consensus metadata: real 0.710s	user 0.009s	sys 0.000s
I20260709 13:22:22.550276 23800 consensus_queue.cc:1059] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: INVALID_TERM, Last received: 20.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 13:22:22.551553 23800 consensus_queue.cc:1059] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: INVALID_TERM, Last received: 23.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.000s
I20260709 13:22:22.552397 23817 raft_consensus.cc:3055] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:22:22.552762 23817 raft_consensus.cc:740] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:22:22.553687 23817 consensus_queue.cc:260] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 20.16, Last appended by leader: 16, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:22.555716 23800 raft_consensus.cc:3055] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac [term 23 LEADER]: Stepping down as leader of term 23
I20260709 13:22:22.556152 23800 raft_consensus.cc:740] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:22:22.554966 23817 raft_consensus.cc:3060] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:22:22.556927 23450 raft_consensus.cc:515] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:22:22.490105 16266 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.604s	user 0.006s	sys 0.026s
W20260709 13:22:22.560647 16266 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.674s	user 0.006s	sys 0.026s
W20260709 13:22:22.561497 16266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 700934us
I20260709 13:22:22.549606 23384 consensus_queue.cc:1059] T af57996a8be84d398f0f023e8d08ef77 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: INVALID_TERM, Last received: 20.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
W20260709 13:22:22.565274 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 55908us
I20260709 13:22:22.566120 16790 leader_election.cc:304] T 2794af5b99984efebdee564035841cb0 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860; no voters: 90f3da57796b488c9087a2f523adf045
I20260709 13:22:22.519412 16292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8638ee363273426db1393dd81e5c7b72" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:22:22.566910 16292 raft_consensus.cc:3060] T 8638ee363273426db1393dd81e5c7b72 P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:22:22.550390 23730 raft_consensus.cc:515] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:22.568931 16789 leader_election.cc:304] T 57a744f5aaf944678a63469cbf514015 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 
W20260709 13:22:22.576406 16117 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 537740us
I20260709 13:22:22.577232 16117 leader_election.cc:304] T 434bff3082d141838068f323bdb0d1ed P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860
W20260709 13:22:22.578109 16265 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.705s	user 0.005s	sys 0.031s
W20260709 13:22:22.578665 16265 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.706s	user 0.005s	sys 0.031s
I20260709 13:22:22.540388 16299 rpcz_store.cc:276] 0709 13:22:21.036175 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:22:21.058570 (+ 22395us) service_pool.cc:224] Handling call
0709 13:22:21.070408 (+ 11838us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:22:21.569365 (+498957us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 13:22:21.640304 (+ 70939us) log.cc:844] Serialized 14 byte log entry
0709 13:22:22.379597 (+739293us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:22:22.379676 (+    79us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:22:22.379741 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:22:22.493927 (+114186us) spinlock_profiling.cc:243] Waited 24.4 ms on lock 0x7b1800238e40. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b693ade1 00007f69b69475b1 00007f69ada402a6 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419
0709 13:22:22.539668 (+ 45741us) spinlock_profiling.cc:243] Waited 499 ms on lock 0x7b540024d954. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:22:22.539718 (+    50us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:22:22.539862 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":70740,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":40,"raft.queue_time_us":143,"raft.run_cpu_time_us":148,"raft.run_wall_time_us":10932,"spinlock_wait_cycles":1203131520,"thread_start_us":738707,"threads_started":1}
W20260709 13:22:22.579447 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 729195us
I20260709 13:22:22.580482 16265 leader_election.cc:304] T 18ad7e3af5504571a69d10c00750a01c P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac
W20260709 13:22:22.583986 16266 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:22:22.591506 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada5a004ba7c46819051e3ccec71ffdb" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:22:22.593684 16117 leader_election.cc:304] T 746aabf856ab401c9ef5e859371e7b67 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860
W20260709 13:22:22.595508 16263 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:22:22.597921 16294 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48684 (request call id 3343) took 1372 ms. Trace:
W20260709 13:22:22.573882 23775 consensus_meta.cc:287] T 806d7690c2b644959152074bb2cab0a7 P cb82cc7bd6db46d980c75237a1402860: Time spent flushing consensus metadata: real 0.859s	user 0.011s	sys 0.000s
I20260709 13:22:22.611629 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e9b2bf7011f4ae688a79d7bb5a447b4" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
W20260709 13:22:22.574141 23601 consensus_meta.cc:287] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac: Time spent flushing consensus metadata: real 0.750s	user 0.009s	sys 0.008s
I20260709 13:22:22.616242 16117 leader_election.cc:304] T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860
I20260709 13:22:22.619570 16265 leader_election.cc:304] T 23f82224f4564fea93572e8206bad1fe P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:22:22.598383 16294 rpcz_store.cc:276] 0709 13:22:21.225464 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:22:21.707114 (+481650us) service_pool.cc:224] Handling call
0709 13:22:21.707421 (+   307us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:22:22.484704 (+777283us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:22:22.564334 (+ 79630us) spinlock_profiling.cc:243] Waited 10.1 ms on lock 0x7b5400072720. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0ad6dc0 00007f69b0b62a02 00007f69b0b5edd6 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2
0709 13:22:22.597586 (+ 33252us) spinlock_profiling.cc:243] Waited 746 ms on lock 0x7b540010f254. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:22:22.597852 (+   266us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1739605248}
I20260709 13:22:22.567377 23789 leader_election.cc:290] T ada5a004ba7c46819051e3ccec71ffdb P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 23 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
W20260709 13:22:22.637959 16266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 50781.8us
I20260709 13:22:22.577787 23638 raft_consensus.cc:2804] T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 13:22:22.517357 23795 leader_election.cc:290] T 8638ee363273426db1393dd81e5c7b72 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 21 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:22.700208 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a878999c0024d2abe517484180290fc" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
W20260709 13:22:22.685947 16117 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 69819.5us
W20260709 13:22:22.683897 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 64450.9us
I20260709 13:22:22.703513 16299 raft_consensus.cc:2393] T 8a878999c0024d2abe517484180290fc P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 19: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
I20260709 13:22:22.703670 16265 leader_election.cc:304] T 8d4ab870a05347868956aa8b841b2981 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:22:22.558704 23800 consensus_queue.cc:260] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 23.17, Last appended by leader: 17, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:22.541465 23719 raft_consensus.cc:515] T 829b4ad8c5a54eababeea1181b960d4b P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:22.685581 23694 raft_consensus.cc:2804] T 57a744f5aaf944678a63469cbf514015 P cb82cc7bd6db46d980c75237a1402860 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:22:22.690282 16296 raft_consensus.cc:2393] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 20: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
I20260709 13:22:22.696774 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f82224f4564fea93572e8206bad1fe" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:22.628340 23450 leader_election.cc:290] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 19 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:22.628336 23784 raft_consensus.cc:2804] T 18ad7e3af5504571a69d10c00750a01c P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:22:22.594070 23745 raft_consensus.cc:493] T c0eab25ce4c64e018a041beeda2867db P 90f3da57796b488c9087a2f523adf045 [term 22 FOLLOWER]: Starting leader election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:22:22.708071 16265 leader_election.cc:304] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: cb82cc7bd6db46d980c75237a1402860
I20260709 13:22:22.681371 23778 raft_consensus.cc:2804] T 2794af5b99984efebdee564035841cb0 P cb82cc7bd6db46d980c75237a1402860 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:22:22.629319 23774 raft_consensus.cc:2749] T 434bff3082d141838068f323bdb0d1ed P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:22:22.702443 16117 leader_election.cc:304] T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:22:22.742898 16117 leader_election.cc:304] T b87147731a68400e8cce9b879daa7c5d P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860
I20260709 13:22:22.743490 16299 raft_consensus.cc:1240] T e5ad4eb281564936a59f41e387a47156 P 90f3da57796b488c9087a2f523adf045 [term 21 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 20. Current term is 21. Ops: []
I20260709 13:22:22.682649 23605 raft_consensus.cc:2749] T 746aabf856ab401c9ef5e859371e7b67 P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:22:22.745863 16117 leader_election.cc:304] T 9e65676b21b748728ad39668cfe722ec P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:22:22.712960 23306 raft_consensus.cc:2749] T 8d4ab870a05347868956aa8b841b2981 P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:22:22.575925 23630 consensus_queue.cc:1059] T 7c2095a5f4ad49408dbc7f69c606120d P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: INVALID_TERM, Last received: 16.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.121s
I20260709 13:22:22.766705 16299 raft_consensus.cc:1240] T fffab26427e24d398e2695aad40ca134 P 90f3da57796b488c9087a2f523adf045 [term 21 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 20. Current term is 21. Ops: []
I20260709 13:22:22.766814 23630 raft_consensus.cc:3055] T 7c2095a5f4ad49408dbc7f69c606120d P 1ca2725d290e47d68b1a4b609c9816ac [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:22:22.767486 23630 raft_consensus.cc:740] T 7c2095a5f4ad49408dbc7f69c606120d P 1ca2725d290e47d68b1a4b609c9816ac [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:22:22.685238 23775 raft_consensus.cc:2749] T 806d7690c2b644959152074bb2cab0a7 P cb82cc7bd6db46d980c75237a1402860 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:22:22.727634 16292 raft_consensus.cc:2468] T 8638ee363273426db1393dd81e5c7b72 P 90f3da57796b488c9087a2f523adf045 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 21.
I20260709 13:22:22.689272 23210 raft_consensus.cc:2804] T 23f82224f4564fea93572e8206bad1fe P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:22:22.691140 23286 leader_election.cc:290] T af57996a8be84d398f0f023e8d08ef77 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 21 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:22.593318 23598 consensus_queue.cc:1059] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: INVALID_TERM, Last received: 20.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.110s
I20260709 13:22:22.741053 16145 raft_consensus.cc:1240] T 829b4ad8c5a54eababeea1181b960d4b P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 19. Current term is 20. Ops: []
I20260709 13:22:22.740760 16265 leader_election.cc:304] T fffab26427e24d398e2695aad40ca134 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:22:22.787089 23605 raft_consensus.cc:2804] T 9e65676b21b748728ad39668cfe722ec P 647a343a542d4a139fe106c19095129b [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:22:22.787593 23605 raft_consensus.cc:493] T 9e65676b21b748728ad39668cfe722ec P 647a343a542d4a139fe106c19095129b [term 21 FOLLOWER]: Starting leader election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
W20260709 13:22:22.787685 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 47041.2us
I20260709 13:22:22.788147 23605 raft_consensus.cc:3060] T 9e65676b21b748728ad39668cfe722ec P 647a343a542d4a139fe106c19095129b [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:22:22.689833 16266 leader_election.cc:304] T 66ae176961b74813a58bc15c9c4c6f2a P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860
W20260709 13:22:22.791806 16266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 102096us
W20260709 13:22:22.792155 23625 consensus_meta.cc:287] T 8158bbd17b994d2eb96f560bd0f1b88d P 90f3da57796b488c9087a2f523adf045: Time spent flushing consensus metadata: real 0.588s	user 0.009s	sys 0.000s
I20260709 13:22:22.792696 23625 raft_consensus.cc:515] T 8158bbd17b994d2eb96f560bd0f1b88d P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:22.793215 16266 leader_election.cc:304] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac
I20260709 13:22:22.794559 16266 leader_election.cc:304] T ada5a004ba7c46819051e3ccec71ffdb P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860
I20260709 13:22:22.713543 23638 raft_consensus.cc:697] T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860 [term 24 LEADER]: Becoming Leader. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:22:22.796125 23638 consensus_queue.cc:237] T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 23.17, Last appended by leader: 17, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:22.689019 23601 raft_consensus.cc:515] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:22.798970 23306 raft_consensus.cc:2804] T fffab26427e24d398e2695aad40ca134 P 90f3da57796b488c9087a2f523adf045 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:22:22.799687 23306 raft_consensus.cc:697] T fffab26427e24d398e2695aad40ca134 P 90f3da57796b488c9087a2f523adf045 [term 21 LEADER]: Becoming Leader. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:22:22.800733 23306 consensus_queue.cc:237] T fffab26427e24d398e2695aad40ca134 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.13, Last appended by leader: 13, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:22.801260 23601 leader_election.cc:290] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 25 election: Requested vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:22.802299 16293 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48684 (request call id 3339) took 1663 ms. Trace:
I20260709 13:22:22.803539 16217 raft_consensus.cc:1240] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [term 25 FOLLOWER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 24. Current term is 25. Ops: [24.18-24.18]
I20260709 13:22:22.803435 23286 consensus_queue.cc:1059] T ecc1e363230043fe9fef390f0b979f20 P 647a343a542d4a139fe106c19095129b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: INVALID_TERM, Last received: 23.14, Next index: 15, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:22:22.804090 16266 leader_election.cc:304] T 3884f864a96d428785895e5463e5728f P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
I20260709 13:22:22.734304 23694 raft_consensus.cc:493] T 57a744f5aaf944678a63469cbf514015 P cb82cc7bd6db46d980c75237a1402860 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:22.802858 16293 rpcz_store.cc:276] 0709 13:22:21.138866 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:22:21.713779 (+574913us) service_pool.cc:224] Handling call
0709 13:22:21.757664 (+ 43885us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:22:21.758058 (+   394us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:22:21.758088 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:22:21.767072 (+  8984us) log.cc:844] Serialized 26 byte log entry
0709 13:22:22.273626 (+506554us) raft_consensus.cc:1645] Marking committed up to 15
0709 13:22:22.273670 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:22:22.273745 (+    75us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:22:22.802015 (+528270us) raft_consensus.cc:1685] finished
0709 13:22:22.802045 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:22:22.802232 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":17862,"raft.queue_time_us":95,"raft.run_cpu_time_us":152,"raft.run_wall_time_us":48367,"thread_start_us":406491,"threads_started":1,"wal-append.queue_time_us":869168}
W20260709 13:22:22.776149 16191 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.859s	user 0.025s	sys 0.020s
W20260709 13:22:22.808950 16191 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.892s	user 0.025s	sys 0.020s
W20260709 13:22:22.809415 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 892508us
I20260709 13:22:22.810638 16191 leader_election.cc:304] T 5a959dbecc2648eb85d1cc550f5a3245 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:22:22.713565 23685 raft_consensus.cc:515] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [term 27 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:22.687928 23815 consensus_queue.cc:1048] T 2794af5b99984efebdee564035841cb0 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 13, Time since last communication: 0.049s
I20260709 13:22:22.815629 16292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31969392ccdb43b491911a319e8bc7e1" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:22:22.723491 23818 consensus_queue.cc:1059] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: INVALID_TERM, Last received: 19.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.055s
I20260709 13:22:22.783792 16817 raft_consensus.cc:1240] T 806d7690c2b644959152074bb2cab0a7 P cb82cc7bd6db46d980c75237a1402860 [term 21 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 20. Current term is 21. Ops: []
I20260709 13:22:22.784734 23719 leader_election.cc:290] T 829b4ad8c5a54eababeea1181b960d4b P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 20 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:22.735482 23784 raft_consensus.cc:493] T 18ad7e3af5504571a69d10c00750a01c P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:22:22.785708 23559 leader_election.cc:290] T e5ad4eb281564936a59f41e387a47156 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 21 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:22:22.742458 23745 raft_consensus.cc:3060] T c0eab25ce4c64e018a041beeda2867db P 90f3da57796b488c9087a2f523adf045 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:22:22.786377 23774 raft_consensus.cc:2749] T b87147731a68400e8cce9b879daa7c5d P 647a343a542d4a139fe106c19095129b [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:22:22.723147 23533 consensus_queue.cc:1048] T d16dfad0c3b540e5b4b86492ddf87208 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.054s
I20260709 13:22:22.778288 16789 leader_election.cc:304] T 8638ee363273426db1393dd81e5c7b72 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 
W20260709 13:22:22.832454 16789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 54254.9us
I20260709 13:22:22.780304 23210 raft_consensus.cc:493] T 23f82224f4564fea93572e8206bad1fe P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:22:22.834779 16823 raft_consensus.cc:1240] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [term 27 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 26. Current term is 27. Ops: []
I20260709 13:22:22.818400 23601 raft_consensus.cc:2749] T 5a959dbecc2648eb85d1cc550f5a3245 P 1ca2725d290e47d68b1a4b609c9816ac [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:22:22.764962 23831 raft_consensus.cc:2804] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [term 25 FOLLOWER]: Leader election won for term 25
W20260709 13:22:22.818881 16264 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.830s	user 0.021s	sys 0.029s
W20260709 13:22:22.853968 16264 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.981s	user 0.021s	sys 0.030s
W20260709 13:22:22.776924 16295 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.895s	user 0.008s	sys 0.020s
W20260709 13:22:22.820116 16115 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.740s	user 0.009s	sys 0.025s
I20260709 13:22:22.770515 23800 raft_consensus.cc:3060] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:22:22.764633 23798 raft_consensus.cc:2804] T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 13:22:22.823628 23719 raft_consensus.cc:3055] T ecc1e363230043fe9fef390f0b979f20 P 647a343a542d4a139fe106c19095129b [term 23 LEADER]: Stepping down as leader of term 23
I20260709 13:22:22.769791 23778 raft_consensus.cc:697] T 2794af5b99984efebdee564035841cb0 P cb82cc7bd6db46d980c75237a1402860 [term 20 LEADER]: Becoming Leader. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:22:22.787817 23598 raft_consensus.cc:3055] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac [term 20 LEADER]: Stepping down as leader of term 20
W20260709 13:22:22.824496 16189 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:22:22.825555 16293 raft_consensus.cc:1240] T 8638ee363273426db1393dd81e5c7b72 P 90f3da57796b488c9087a2f523adf045 [term 21 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 20. Current term is 21. Ops: []
I20260709 13:22:22.768391 23630 consensus_queue.cc:260] T 7c2095a5f4ad49408dbc7f69c606120d P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 18.9, Last appended by leader: 9, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:22.823091 16191 leader_election.cc:304] T 746aabf856ab401c9ef5e859371e7b67 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: cb82cc7bd6db46d980c75237a1402860
W20260709 13:22:22.851593 16116 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.772s	user 0.007s	sys 0.023s
W20260709 13:22:22.936407 16116 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.857s	user 0.008s	sys 0.023s
W20260709 13:22:22.936723 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 113750us
I20260709 13:22:22.937466 16191 leader_election.cc:304] T b6f07842d3aa47858df5665874c86f3a P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 90f3da57796b488c9087a2f523adf045
I20260709 13:22:22.852995 23210 raft_consensus.cc:3060] T 23f82224f4564fea93572e8206bad1fe P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:22:22.854604 23775 raft_consensus.cc:2804] T 8638ee363273426db1393dd81e5c7b72 P cb82cc7bd6db46d980c75237a1402860 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:22:22.940801 16294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8158bbd17b994d2eb96f560bd0f1b88d" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:22:22.944846 16191 leader_election.cc:304] T ee9ed67006144797b1de6ea8c2013a68 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: cb82cc7bd6db46d980c75237a1402860
I20260709 13:22:22.851351 23559 raft_consensus.cc:2749] T 66ae176961b74813a58bc15c9c4c6f2a P 90f3da57796b488c9087a2f523adf045 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
W20260709 13:22:22.872901 16142 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.752s	user 0.015s	sys 0.019s
W20260709 13:22:22.875052 16115 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.795s	user 0.009s	sys 0.029s
W20260709 13:22:22.874898 16295 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.993s	user 0.009s	sys 0.025s
I20260709 13:22:22.805235 23694 raft_consensus.cc:3060] T 57a744f5aaf944678a63469cbf514015 P cb82cc7bd6db46d980c75237a1402860 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:22:22.816973 23625 leader_election.cc:290] T 8158bbd17b994d2eb96f560bd0f1b88d P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 20 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
W20260709 13:22:22.920389 23777 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.582s	user 0.013s	sys 0.007s
W20260709 13:22:22.896337 16149 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.669s	user 0.013s	sys 0.004s
W20260709 13:22:22.923964 16825 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.594s	user 0.006s	sys 0.014s
I20260709 13:22:22.878592 23774 raft_consensus.cc:493] T e5ad4eb281564936a59f41e387a47156 P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:22:22.970492 16149 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.743s	user 0.013s	sys 0.004s
W20260709 13:22:22.970747 16825 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.641s	user 0.007s	sys 0.014s
W20260709 13:22:22.956732 16142 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.836s	user 0.016s	sys 0.020s
I20260709 13:22:22.963287 16292 raft_consensus.cc:1240] T 8a878999c0024d2abe517484180290fc P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 18. Current term is 19. Ops: []
W20260709 13:22:22.964196 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 992517us
W20260709 13:22:22.920753 16300 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.659s	user 0.007s	sys 0.013s
W20260709 13:22:22.920576 23603 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.689s	user 0.007s	sys 0.012s
W20260709 13:22:22.894685 16151 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.610s	user 0.001s	sys 0.024s
W20260709 13:22:22.977905 16151 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.693s	user 0.001s	sys 0.024s
W20260709 13:22:22.936218 23776 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.674s	user 0.003s	sys 0.022s
I20260709 13:22:22.882678 23719 raft_consensus.cc:740] T ecc1e363230043fe9fef390f0b979f20 P 647a343a542d4a139fe106c19095129b [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:22:22.825804 23784 raft_consensus.cc:3060] T 18ad7e3af5504571a69d10c00750a01c P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:22:22.935575 23601 raft_consensus.cc:3055] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 1ca2725d290e47d68b1a4b609c9816ac [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:22:22.933914 23598 raft_consensus.cc:740] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:22:22.987376 16293 raft_consensus.cc:1275] T 02838c86531d4f98bf58d1c22b8baf77 P 90f3da57796b488c9087a2f523adf045 [term 22 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 22 index: 13. (index mismatch)
W20260709 13:22:22.896993 23761 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.713s	user 0.016s	sys 0.007s
I20260709 13:22:22.974704 16296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c2095a5f4ad49408dbc7f69c606120d" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 19 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:22.989254 16296 raft_consensus.cc:2393] T 7c2095a5f4ad49408dbc7f69c606120d P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 19: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
W20260709 13:22:22.976375 16300 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.715s	user 0.008s	sys 0.013s
I20260709 13:22:22.852355 23024 consensus_queue.cc:1048] T 2794af5b99984efebdee564035841cb0 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.107s
W20260709 13:22:22.977665 23603 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.746s	user 0.009s	sys 0.013s
I20260709 13:22:22.967406 16115 leader_election.cc:304] T 08c74490c40243a88e62d1760b5cd814 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:22:22.878440 23798 raft_consensus.cc:493] T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:22.984472 16294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4f65cdf00b748bb963ee6e81ef43b79" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
W20260709 13:22:22.970125 23777 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.656s	user 0.014s	sys 0.007s
I20260709 13:22:22.924413 23398 leader_election.cc:290] T af57996a8be84d398f0f023e8d08ef77 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 21 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:22:22.973299 16116 leader_election.cc:304] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 90f3da57796b488c9087a2f523adf045
W20260709 13:22:22.968032 23357 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.627s	user 0.001s	sys 0.018s
I20260709 13:22:22.983093 16825 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49906 (request call id 1467) took 1733 ms. Trace:
W20260709 13:22:22.981979 23776 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.720s	user 0.003s	sys 0.023s
W20260709 13:22:22.994874 16147 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.659s	user 0.004s	sys 0.020s
W20260709 13:22:23.012835 16147 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.677s	user 0.005s	sys 0.020s
W20260709 13:22:22.983697 23762 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.639s	user 0.009s	sys 0.013s
W20260709 13:22:22.979009 16040 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.740s	user 0.009s	sys 0.017s
I20260709 13:22:22.990491 23864 raft_consensus.cc:2749] T ada5a004ba7c46819051e3ccec71ffdb P 90f3da57796b488c9087a2f523adf045 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 13:22:23.012267 23357 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.674s	user 0.001s	sys 0.018s
W20260709 13:22:23.018548 23859 log.cc:927] Time spent T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860: Append to log took a long time: real 0.082s	user 0.000s	sys 0.000s
W20260709 13:22:23.018657 16040 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.780s	user 0.010s	sys 0.017s
W20260709 13:22:23.018576 23762 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.671s	user 0.010s	sys 0.013s
I20260709 13:22:23.022063 16148 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46224 (request call id 1293) took 2239 ms. Trace:
I20260709 13:22:23.022603 16148 rpcz_store.cc:276] 0709 13:22:20.782340 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:22:21.849939 (+1067599us) service_pool.cc:224] Handling call
0709 13:22:21.850250 (+   311us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:22:21.850692 (+   442us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:22:21.850739 (+    47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:22:21.856808 (+  6069us) log.cc:844] Serialized 26 byte log entry
0709 13:22:22.433727 (+576919us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:22:22.433786 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:22:22.433900 (+   114us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:22:23.021680 (+587780us) raft_consensus.cc:1685] finished
0709 13:22:23.021709 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:22:23.021988 (+   279us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":102,"raft.run_cpu_time_us":186,"raft.run_wall_time_us":18507,"thread_start_us":435648,"threads_started":1,"wal-append.queue_time_us":1114388}
I20260709 13:22:23.035609 23719 consensus_queue.cc:260] T ecc1e363230043fe9fef390f0b979f20 P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 23.14, Last appended by leader: 14, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:23.037742 23719 raft_consensus.cc:3060] T ecc1e363230043fe9fef390f0b979f20 P 647a343a542d4a139fe106c19095129b [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:22:22.987732 23601 raft_consensus.cc:740] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 1ca2725d290e47d68b1a4b609c9816ac [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:22:23.044909 23601 consensus_queue.cc:260] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:22:23.051694 23857 log.cc:927] Time spent T 1df93779209748b9860f6d893dd3de53 P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.130s	user 0.000s	sys 0.001s
I20260709 13:22:23.053057 16291 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48684 (request call id 3338) took 1914 ms. Trace:
I20260709 13:22:22.975059 23866 raft_consensus.cc:2804] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:22:22.990794 23559 raft_consensus.cc:2749] T 3884f864a96d428785895e5463e5728f P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:22:23.067466 16821 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49896 (request call id 1651) took 1479 ms. Trace:
I20260709 13:22:23.067991 16821 rpcz_store.cc:276] 0709 13:22:21.588255 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:22:21.592698 (+  4443us) service_pool.cc:224] Handling call
0709 13:22:23.067253 (+1474555us) spinlock_profiling.cc:243] Waited 1.37 s on lock 0x7b54005c35d4. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:22:23.067399 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3146532736}
I20260709 13:22:22.981598 23730 leader_election.cc:290] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 20 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:23.007893 16823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31969392ccdb43b491911a319e8bc7e1" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:23.008800 16146 raft_consensus.cc:1240] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b [term 21 LEADER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 20. Current term is 21. Ops: [20.14-20.14]
I20260709 13:22:23.009706 16292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57a744f5aaf944678a63469cbf514015" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:23.077849 16292 raft_consensus.cc:2393] T 57a744f5aaf944678a63469cbf514015 P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 19: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
I20260709 13:22:22.977272 23775 raft_consensus.cc:697] T 8638ee363273426db1393dd81e5c7b72 P cb82cc7bd6db46d980c75237a1402860 [term 21 LEADER]: Becoming Leader. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:22:23.079108 23775 consensus_queue.cc:237] T 8638ee363273426db1393dd81e5c7b72 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 20.16, Last appended by leader: 16, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:23.079586 23605 raft_consensus.cc:515] T 9e65676b21b748728ad39668cfe722ec P 647a343a542d4a139fe106c19095129b [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:22:22.991958 23761 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.815s	user 0.016s	sys 0.007s
I20260709 13:22:23.081770 23605 leader_election.cc:290] T 9e65676b21b748728ad39668cfe722ec P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 22 election: Requested vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:22.989069 23754 raft_consensus.cc:2804] T 66f75124215f4ca6bef831e174a792d8 P cb82cc7bd6db46d980c75237a1402860 [term 19 FOLLOWER]: Leader pre-election won for term 20
W20260709 13:22:23.056845 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 83638.8us
I20260709 13:22:23.083618 16116 leader_election.cc:400] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 21 election: Vote denied by peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943) with higher term. Message: Invalid argument: T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b for earlier term 21. Current term is 22.
I20260709 13:22:23.084141 16116 leader_election.cc:403] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 21 election: Cancelling election due to peer responding with higher term
I20260709 13:22:23.053617 16291 rpcz_store.cc:276] 0709 13:22:21.138655 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:22:21.681862 (+543207us) service_pool.cc:224] Handling call
0709 13:22:21.682243 (+   381us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:22:21.682607 (+   364us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:22:21.682641 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:22:21.683151 (+   510us) log.cc:844] Serialized 26 byte log entry
0709 13:22:22.435655 (+752504us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:22:22.435694 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:22:22.435835 (+   141us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:22:23.052845 (+617010us) raft_consensus.cc:1685] finished
0709 13:22:23.052865 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:22:23.052999 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":752,"raft.run_cpu_time_us":181,"raft.run_wall_time_us":33488,"thread_start_us":711975,"threads_started":1,"wal-append.queue_time_us":1212195}
W20260709 13:22:23.084486 23287 consensus_meta.cc:287] T 26354361e9b044cea1c2b86c4882fe92 P 90f3da57796b488c9087a2f523adf045: Time spent flushing consensus metadata: real 0.527s	user 0.009s	sys 0.001s
I20260709 13:22:23.004472 23798 raft_consensus.cc:3060] T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:22:23.053598 23685 leader_election.cc:290] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 27 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:22.980393 23774 raft_consensus.cc:515] T e5ad4eb281564936a59f41e387a47156 P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:22.984539 23630 raft_consensus.cc:3060] T 7c2095a5f4ad49408dbc7f69c606120d P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:22:23.094264 16116 leader_election.cc:304] T 18ad7e3af5504571a69d10c00750a01c P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:22:23.095991 16116 leader_election.cc:304] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860
W20260709 13:22:23.089653 16115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 122323us
I20260709 13:22:23.097512 16115 leader_election.cc:304] T 2794af5b99984efebdee564035841cb0 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860
I20260709 13:22:23.068883 23603 raft_consensus.cc:2804] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b [term 24 FOLLOWER]: Leader election won for term 24
I20260709 13:22:23.067503 23866 raft_consensus.cc:493] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045 [term 22 FOLLOWER]: Starting leader election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
W20260709 13:22:23.117483 16816 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.666s	user 0.009s	sys 0.016s
W20260709 13:22:23.118016 16816 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.666s	user 0.009s	sys 0.017s
I20260709 13:22:23.106032 16115 leader_election.cc:400] T 8cc959fb7f704fba9009c0595467d074 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 19 pre-election: Vote denied by peer 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389) with higher term. Message: Invalid argument: T 8cc959fb7f704fba9009c0595467d074 P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b for earlier term 19. Current term is 20.
I20260709 13:22:23.121533 16291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66ae176961b74813a58bc15c9c4c6f2a" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 27 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
W20260709 13:22:23.148577 16263 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:22:23.098577 23645 consensus_meta.cc:287] T 3a7ca61592e848ea9b0fb81a2ce33cdc P cb82cc7bd6db46d980c75237a1402860: Time spent flushing consensus metadata: real 0.567s	user 0.007s	sys 0.004s
I20260709 13:22:23.119215 23384 raft_consensus.cc:2804] T 746aabf856ab401c9ef5e859371e7b67 P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:22:23.120148 23605 raft_consensus.cc:2749] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b [term 22 FOLLOWER]: Leader election lost for term 21. Reason: Vote denied by peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943) with higher term. Message: Invalid argument: T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b for earlier term 21. Current term is 22.
W20260709 13:22:23.151410 23817 consensus_meta.cc:287] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac: Time spent flushing consensus metadata: real 0.594s	user 0.010s	sys 0.003s
I20260709 13:22:23.118455 23866 raft_consensus.cc:3060] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:22:23.088943 23598 consensus_queue.cc:260] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.14, Last appended by leader: 14, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:23.182713 23754 raft_consensus.cc:493] T 66f75124215f4ca6bef831e174a792d8 P cb82cc7bd6db46d980c75237a1402860 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:22:23.053838 23778 consensus_queue.cc:237] T 2794af5b99984efebdee564035841cb0 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.15, Last appended by leader: 16, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:23.198531 16145 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40844 (request call id 3103) took 2330 ms. Trace:
W20260709 13:22:23.209537 16824 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.795s	user 0.009s	sys 0.025s
W20260709 13:22:23.213852 16824 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.809s	user 0.009s	sys 0.026s
I20260709 13:22:23.213189 16145 rpcz_store.cc:276] 0709 13:22:20.867986 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:22:22.246076 (+1378090us) service_pool.cc:224] Handling call
0709 13:22:22.246368 (+   292us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:22:22.787446 (+541078us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:22:23.187099 (+399653us) spinlock_profiling.cc:243] Waited 494 ms on lock 0x7b540006d6d4. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:22:23.198433 (+ 11334us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1136990976}
W20260709 13:22:23.219779 16266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 415718us
I20260709 13:22:23.180035 23871 raft_consensus.cc:2749] T 18ad7e3af5504571a69d10c00750a01c P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:22:23.221850 16266 leader_election.cc:400] T 2794af5b99984efebdee564035841cb0 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 19 election: Vote denied by peer cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923) with higher term. Message: Invalid argument: T 2794af5b99984efebdee564035841cb0 P cb82cc7bd6db46d980c75237a1402860 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 for earlier term 19. Current term is 20.
I20260709 13:22:23.172425 23384 raft_consensus.cc:493] T 746aabf856ab401c9ef5e859371e7b67 P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Starting leader election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
W20260709 13:22:23.226565 16192 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.516s	user 0.029s	sys 0.021s
I20260709 13:22:23.219012 23754 raft_consensus.cc:3060] T 66f75124215f4ca6bef831e174a792d8 P cb82cc7bd6db46d980c75237a1402860 [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:22:23.228355 16822 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.778s	user 0.013s	sys 0.021s
I20260709 13:22:23.203904 23817 raft_consensus.cc:2804] T b6f07842d3aa47858df5665874c86f3a P 1ca2725d290e47d68b1a4b609c9816ac [term 20 FOLLOWER]: Leader pre-election won for term 21
W20260709 13:22:23.235301 16143 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.783s	user 0.013s	sys 0.023s
W20260709 13:22:23.235679 16143 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.785s	user 0.013s	sys 0.023s
W20260709 13:22:23.237792 16190 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.660s	user 0.018s	sys 0.017s
I20260709 13:22:23.209666 23605 raft_consensus.cc:2749] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 13:22:23.231383 16192 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.521s	user 0.029s	sys 0.021s
W20260709 13:22:23.233903 16822 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.783s	user 0.014s	sys 0.022s
I20260709 13:22:23.232002 23384 raft_consensus.cc:3060] T 746aabf856ab401c9ef5e859371e7b67 P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:22:23.290426 16192 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:22:23.290735 16265 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:22:23.238250 16190 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.661s	user 0.018s	sys 0.017s
W20260709 13:22:23.296001 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 719004us
I20260709 13:22:23.235260 23817 raft_consensus.cc:493] T b6f07842d3aa47858df5665874c86f3a P 1ca2725d290e47d68b1a4b609c9816ac [term 20 FOLLOWER]: Starting leader election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
W20260709 13:22:23.259335 16144 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.835s	user 0.029s	sys 0.024s
I20260709 13:22:23.236282 23598 raft_consensus.cc:3060] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:22:23.292891 16265 leader_election.cc:304] T cdbd78acb4cf46298e0de8ba0d130e19 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: cb82cc7bd6db46d980c75237a1402860
I20260709 13:22:23.231771 23869 raft_consensus.cc:2749] T 2794af5b99984efebdee564035841cb0 P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:22:23.310161 16144 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.886s	user 0.030s	sys 0.024s
I20260709 13:22:23.230580 23745 raft_consensus.cc:515] T c0eab25ce4c64e018a041beeda2867db P 90f3da57796b488c9087a2f523adf045 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:22:23.258867 16790 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.557s	user 0.007s	sys 0.007s
I20260709 13:22:23.197853 23645 raft_consensus.cc:515] T 3a7ca61592e848ea9b0fb81a2ce33cdc P cb82cc7bd6db46d980c75237a1402860 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:23.316110 23817 raft_consensus.cc:3060] T b6f07842d3aa47858df5665874c86f3a P 1ca2725d290e47d68b1a4b609c9816ac [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:22:23.333899 23862 log.cc:927] Time spent T 26354361e9b044cea1c2b86c4882fe92 P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.232s	user 0.000s	sys 0.000s
W20260709 13:22:23.334669 23853 log.cc:927] Time spent T ada5a004ba7c46819051e3ccec71ffdb P cb82cc7bd6db46d980c75237a1402860: Append to log took a long time: real 0.757s	user 0.003s	sys 0.003s
I20260709 13:22:23.336571 16820 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49906 (request call id 1455) took 2146 ms. Trace:
I20260709 13:22:23.336984 16820 rpcz_store.cc:276] 0709 13:22:21.190534 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:22:21.203490 (+ 12956us) service_pool.cc:224] Handling call
0709 13:22:21.406225 (+202735us) spinlock_profiling.cc:243] Waited 17.5 ms on lock 0x7b5800004298. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69ad8fadb1 00007f69ada3ab71 00007f69ada3aa22 00007f69b6953c09 00007f69b5aadee0 00007f69b5b770ab 00007f69b5b77131 00007f69b5b38fc0 00007f69b5b23929 00007f69b5b233d2 00007f69af09abd4
0709 13:22:21.406424 (+   199us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:22:21.406890 (+   466us) raft_consensus.cc:1534] Early marking committed up to index 15
0709 13:22:21.418701 (+ 11811us) log.cc:844] Serialized 14 byte log entry
0709 13:22:22.203990 (+785289us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:22:22.204636 (+   646us) log.cc:844] Serialized 26 byte log entry
0709 13:22:22.204815 (+   179us) raft_consensus.cc:1645] Marking committed up to 15
0709 13:22:22.204864 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:22:22.204971 (+   107us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:22:23.334017 (+1129046us) spinlock_profiling.cc:243] Waited 15.1 ms on lock 0x7b4400022b4c. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69ad8fadb1 00007f69ad998c85 00007f69ad9996a2 00007f69b070b1db 00007f69b0706aaf 00007f69b06fa86e 00007f69b06f9bb6 00007f69b06f97d8 00007f69b0703221 00007f69b07031d9 00007f69b0703169
0709 13:22:23.336287 (+  2270us) raft_consensus.cc:1685] finished
0709 13:22:23.336316 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:22:23.336493 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8392,"prepare.queue_time_us":4801,"prepare.run_cpu_time_us":37,"prepare.run_wall_time_us":38,"raft.queue_time_us":50854,"raft.run_cpu_time_us":284,"raft.run_wall_time_us":84683,"spinlock_wait_cycles":74998784,"thread_start_us":784823,"threads_started":1,"wal-append.queue_time_us":1032062}
W20260709 13:22:23.343916 23885 log.cc:927] Time spent T 5a96373bb0e14dce8a266b2fc6c58507 P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 0.108s	user 0.000s	sys 0.000s
I20260709 13:22:23.007045 16825 rpcz_store.cc:276] 0709 13:22:21.249264 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:22:21.272719 (+ 23455us) service_pool.cc:224] Handling call
0709 13:22:21.284538 (+ 11819us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:22:21.284729 (+   191us) raft_consensus.cc:1534] Early marking committed up to index 15
0709 13:22:21.297302 (+ 12573us) log.cc:844] Serialized 14 byte log entry
0709 13:22:22.301977 (+1004675us) spinlock_profiling.cc:243] Waited 3.63 ms on lock 0x7b4400022b4c. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69ad8fadb1 00007f69ad998c85 00007f69ad9996a2 00007f69b070b1db 00007f69b0706aaf 00007f69b06fa86e 00007f69b06f9bb6 00007f69b06f97d8 00007f69b0703221 00007f69b07031d9 00007f69b0703169
0709 13:22:22.329391 (+ 27414us) spinlock_profiling.cc:243] Waited 1 ms on lock 0x7b18003afcc0. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b693ade1 00007f69b69475b1 00007f69ad9ecc0a 00007f69b712c219 00007f69ada74a40 00007f69ada6f4e4 00007f69ada6f455 00007f69b45573c7 00007f69b0b699a7 00007f69b0b790bf 00007f69b0b79038
0709 13:22:22.978431 (+649040us) log.cc:844] Serialized 14 byte log entry
0709 13:22:22.982454 (+  4023us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:22:22.982523 (+    69us) raft_consensus.cc:1645] Marking committed up to 15
0709 13:22:22.982741 (+   218us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:22:22.982829 (+    88us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:22:22.983004 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":55977,"prepare.queue_time_us":653053,"prepare.run_cpu_time_us":113,"prepare.run_wall_time_us":3927,"raft.queue_time_us":3928,"raft.run_cpu_time_us":255,"raft.run_wall_time_us":12196,"spinlock_wait_cycles":10653440,"thread_start_us":1026954,"threads_started":2,"wal-append.queue_time_us":585729}
I20260709 13:22:23.345667 16825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e9b2bf7011f4ae688a79d7bb5a447b4" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:22:23.346621 16825 raft_consensus.cc:2393] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 20: Already voted for candidate cb82cc7bd6db46d980c75237a1402860 in this term.
W20260709 13:22:23.348551 23873 log.cc:927] Time spent T 87ac0d1f7e8a419f9bcbc04aee0e067e P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 0.341s	user 0.003s	sys 0.000s
W20260709 13:22:23.349191 23878 log.cc:927] Time spent T 31969392ccdb43b491911a319e8bc7e1 P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.123s	user 0.000s	sys 0.000s
W20260709 13:22:23.350694 23861 log.cc:927] Time spent T 1df93779209748b9860f6d893dd3de53 P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 0.373s	user 0.010s	sys 0.001s
I20260709 13:22:23.352658 16819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdbd78acb4cf46298e0de8ba0d130e19" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 25 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:22:23.354653 16819 raft_consensus.cc:1240] T 26354361e9b044cea1c2b86c4882fe92 P cb82cc7bd6db46d980c75237a1402860 [term 18 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 17. Current term is 18. Ops: [17.14-17.14]
I20260709 13:22:23.356240 16824 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49906 (request call id 1471) took 1428 ms. Trace:
I20260709 13:22:23.356750 16824 rpcz_store.cc:276] 0709 13:22:21.927547 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:22:22.321287 (+393740us) service_pool.cc:224] Handling call
0709 13:22:22.321748 (+   461us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:22:22.339342 (+ 17594us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:22:22.339391 (+    49us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:22:22.340090 (+   699us) log.cc:844] Serialized 52 byte log entry
0709 13:22:23.214311 (+874221us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:22:23.214355 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:22:23.214632 (+   277us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:22:23.355973 (+141341us) raft_consensus.cc:1685] finished
0709 13:22:23.356006 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:22:23.356173 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8517,"raft.queue_time_us":11510,"raft.run_cpu_time_us":125,"raft.run_wall_time_us":6198,"thread_start_us":809004,"threads_started":1,"wal-append.queue_time_us":996577}
I20260709 13:22:23.348590 23761 consensus_queue.cc:1048] T d16dfad0c3b540e5b4b86492ddf87208 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.015s
I20260709 13:22:23.364471 16146 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40844 (request call id 3101) took 2517 ms. Trace:
I20260709 13:22:23.364984 16146 rpcz_store.cc:276] 0709 13:22:20.847215 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:22:22.151871 (+1304656us) service_pool.cc:224] Handling call
0709 13:22:22.201686 (+ 49815us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:22:23.077179 (+875493us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:22:23.364072 (+286893us) spinlock_profiling.cc:243] Waited 806 ms on lock 0x7b540011add4. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:22:23.364396 (+   324us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1854921984}
I20260709 13:22:23.366382 16143 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40830 (request call id 2236) took 2057 ms. Trace:
I20260709 13:22:23.367488 23645 leader_election.cc:290] T 3a7ca61592e848ea9b0fb81a2ce33cdc P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 22 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:23.368413 16816 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49906 (request call id 1474) took 1370 ms. Trace:
I20260709 13:22:23.368877 16816 rpcz_store.cc:276] 0709 13:22:21.998153 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:22:22.350322 (+352169us) service_pool.cc:224] Handling call
0709 13:22:22.404672 (+ 54350us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:22:22.417794 (+ 13122us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:22:22.417840 (+    46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:22:22.424105 (+  6265us) log.cc:844] Serialized 26 byte log entry
0709 13:22:23.118500 (+694395us) raft_consensus.cc:1645] Marking committed up to 16
0709 13:22:23.118554 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:22:23.118623 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:22:23.368169 (+249546us) raft_consensus.cc:1685] finished
0709 13:22:23.368188 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:22:23.368351 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":12655,"raft.queue_time_us":12803,"raft.run_cpu_time_us":144,"raft.run_wall_time_us":5585,"thread_start_us":666574,"threads_started":1,"wal-append.queue_time_us":932160}
I20260709 13:22:23.368435 16818 raft_consensus.cc:1240] T 3a7ca61592e848ea9b0fb81a2ce33cdc P cb82cc7bd6db46d980c75237a1402860 [term 22 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 21. Current term is 22. Ops: []
I20260709 13:22:23.366912 16143 rpcz_store.cc:276] 0709 13:22:21.309032 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:22:22.361211 (+1052179us) service_pool.cc:224] Handling call
0709 13:22:22.361559 (+   348us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:22:22.377973 (+ 16414us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:22:22.378017 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:22:22.406080 (+ 28063us) log.cc:844] Serialized 26 byte log entry
0709 13:22:23.236078 (+829998us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:22:23.236126 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:22:23.236205 (+    79us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:22:23.366121 (+129916us) raft_consensus.cc:1685] finished
0709 13:22:23.366145 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:22:23.366316 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1441,"raft.run_cpu_time_us":110,"raft.run_wall_time_us":25653,"thread_start_us":785192,"threads_started":1,"wal-append.queue_time_us":928601}
W20260709 13:22:23.332573 16790 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.631s	user 0.008s	sys 0.008s
I20260709 13:22:23.309916 23730 raft_consensus.cc:493] T ecc1e363230043fe9fef390f0b979f20 P 90f3da57796b488c9087a2f523adf045 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:22:23.388243 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7ca61592e848ea9b0fb81a2ce33cdc" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
W20260709 13:22:23.392412 16264 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.519s	user 0.011s	sys 0.009s
W20260709 13:22:23.393002 16264 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.538s	user 0.011s	sys 0.009s
W20260709 13:22:23.394131 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 449336us
I20260709 13:22:23.395787 16191 leader_election.cc:304] T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:22:23.372233 23559 raft_consensus.cc:493] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:22:23.306249 23533 consensus_queue.cc:1048] T 9e65676b21b748728ad39668cfe722ec P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.077s
W20260709 13:22:23.380137 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 284222us
I20260709 13:22:23.376895 23762 raft_consensus.cc:493] T c9944d5e4b8c4109af9955c4f08c9095 P 90f3da57796b488c9087a2f523adf045 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:22:23.378741 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 682920us
I20260709 13:22:23.381428 23872 raft_consensus.cc:493] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:23.429042 16116 leader_election.cc:304] T c0eab25ce4c64e018a041beeda2867db P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:22:23.434924 16821 raft_consensus.cc:1275] T 74ca8140f35c4ae29e8a664f6bdfc829 P cb82cc7bd6db46d980c75237a1402860 [term 23 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 23 index: 16. (index mismatch)
W20260709 13:22:23.429183 16115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 304492us
I20260709 13:22:23.443423 16115 leader_election.cc:304] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860
I20260709 13:22:23.425148 23898 raft_consensus.cc:493] T 8c11074579b046d99e585c2818e18a2c P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
W20260709 13:22:23.482558 16117 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.721s	user 0.021s	sys 0.054s
W20260709 13:22:23.483299 16117 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.722s	user 0.021s	sys 0.054s
I20260709 13:22:23.430855 23892 raft_consensus.cc:2804] T ee9ed67006144797b1de6ea8c2013a68 P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Leader pre-election won for term 20
W20260709 13:22:23.483834 16117 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 722797us
I20260709 13:22:23.490016 16821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57a744f5aaf944678a63469cbf514015" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:23.435201 23774 leader_election.cc:290] T e5ad4eb281564936a59f41e387a47156 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:23.396379 23730 raft_consensus.cc:515] T ecc1e363230043fe9fef390f0b979f20 P 90f3da57796b488c9087a2f523adf045 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:23.496600 16819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e65676b21b748728ad39668cfe722ec" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 22 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:22:23.500329 16819 raft_consensus.cc:3060] T 9e65676b21b748728ad39668cfe722ec P cb82cc7bd6db46d980c75237a1402860 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:22:23.502382 16265 leader_election.cc:304] T 7c2095a5f4ad49408dbc7f69c606120d P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac
I20260709 13:22:23.429033 23559 raft_consensus.cc:515] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:23.431152 23762 raft_consensus.cc:515] T c9944d5e4b8c4109af9955c4f08c9095 P 90f3da57796b488c9087a2f523adf045 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:23.476528 23894 raft_consensus.cc:493] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:23.511341 16824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af57996a8be84d398f0f023e8d08ef77" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:22:23.508450 16825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "829b4ad8c5a54eababeea1181b960d4b" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:22:23.525924 16819 raft_consensus.cc:2468] T 9e65676b21b748728ad39668cfe722ec P cb82cc7bd6db46d980c75237a1402860 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 22.
I20260709 13:22:23.532068 23745 leader_election.cc:290] T c0eab25ce4c64e018a041beeda2867db P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 23 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
W20260709 13:22:23.541642 23719 consensus_meta.cc:287] T ecc1e363230043fe9fef390f0b979f20 P 647a343a542d4a139fe106c19095129b: Time spent flushing consensus metadata: real 0.503s	user 0.011s	sys 0.001s
W20260709 13:22:23.544013 23210 consensus_meta.cc:287] T 23f82224f4564fea93572e8206bad1fe P 90f3da57796b488c9087a2f523adf045: Time spent flushing consensus metadata: real 0.564s	user 0.008s	sys 0.002s
I20260709 13:22:23.489398 23898 raft_consensus.cc:515] T 8c11074579b046d99e585c2818e18a2c P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:23.544577 23210 raft_consensus.cc:515] T 23f82224f4564fea93572e8206bad1fe P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:23.545799 16298 raft_consensus.cc:2468] T 23f82224f4564fea93572e8206bad1fe P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 18.
I20260709 13:22:23.492569 23892 raft_consensus.cc:493] T ee9ed67006144797b1de6ea8c2013a68 P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Starting leader election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:22:23.546556 23898 leader_election.cc:290] T 8c11074579b046d99e585c2818e18a2c P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:23.432075 23872 raft_consensus.cc:515] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:23.549000 16217 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33050 (request call id 1334) took 1808 ms. Trace:
I20260709 13:22:23.549612 16217 rpcz_store.cc:276] 0709 13:22:21.740435 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:22:21.740581 (+   146us) service_pool.cc:224] Handling call
0709 13:22:21.741006 (+   425us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:22:22.804109 (+1063103us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:22:23.548648 (+744539us) spinlock_profiling.cc:243] Waited 1.06 s on lock 0x7b54001423d4. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:22:23.548915 (+   267us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2442900352}
I20260709 13:22:23.480755 23761 consensus_queue.cc:1048] T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.038s
I20260709 13:22:23.554203 23865 raft_consensus.cc:2804] T 7c2095a5f4ad49408dbc7f69c606120d P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:22:23.554697 23559 leader_election.cc:290] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:22:23.554932 23865 raft_consensus.cc:697] T 7c2095a5f4ad49408dbc7f69c606120d P 90f3da57796b488c9087a2f523adf045 [term 19 LEADER]: Becoming Leader. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:22:23.556041 23865 consensus_queue.cc:237] T 7c2095a5f4ad49408dbc7f69c606120d P 90f3da57796b488c9087a2f523adf045 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.8, Last appended by leader: 8, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:22:23.558750 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 56489.7us
W20260709 13:22:23.564025 23800 consensus_meta.cc:287] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac: Time spent flushing consensus metadata: real 0.683s	user 0.010s	sys 0.000s
W20260709 13:22:23.565026 23888 log.cc:927] Time spent T 829b4ad8c5a54eababeea1181b960d4b P cb82cc7bd6db46d980c75237a1402860: Append to log took a long time: real 0.130s	user 0.000s	sys 0.000s
I20260709 13:22:23.546608 23892 raft_consensus.cc:3060] T ee9ed67006144797b1de6ea8c2013a68 P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:22:23.566418 23901 consensus_queue.cc:1059] T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: INVALID_TERM, Last received: 20.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:22:23.495182 23287 raft_consensus.cc:2804] T cdbd78acb4cf46298e0de8ba0d130e19 P 90f3da57796b488c9087a2f523adf045 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:22:23.567533 16822 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49906 (request call id 1473) took 1570 ms. Trace:
W20260709 13:22:23.490239 23306 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.512s	user 0.003s	sys 0.015s
I20260709 13:22:23.572576 16825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c11074579b046d99e585c2818e18a2c" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
W20260709 13:22:23.544085 23784 consensus_meta.cc:287] T 18ad7e3af5504571a69d10c00750a01c P 90f3da57796b488c9087a2f523adf045: Time spent flushing consensus metadata: real 0.554s	user 0.010s	sys 0.001s
I20260709 13:22:23.572785 16824 raft_consensus.cc:2393] T af57996a8be84d398f0f023e8d08ef77 P cb82cc7bd6db46d980c75237a1402860 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 21: Already voted for candidate cb82cc7bd6db46d980c75237a1402860 in this term.
I20260709 13:22:23.587728 16819 raft_consensus.cc:1275] T b87147731a68400e8cce9b879daa7c5d P cb82cc7bd6db46d980c75237a1402860 [term 22 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 20 index: 13. Preceding OpId from leader: term: 22 index: 14. (index mismatch)
I20260709 13:22:23.581202 16822 rpcz_store.cc:276] 0709 13:22:21.997253 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:22:22.346738 (+349485us) service_pool.cc:224] Handling call
0709 13:22:22.347142 (+   404us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:22:22.404210 (+ 57068us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:22:22.404242 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:22:22.421011 (+ 16769us) log.cc:844] Serialized 26 byte log entry
0709 13:22:23.243506 (+822495us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:22:23.243571 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:22:23.243655 (+    84us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:22:23.567274 (+323619us) raft_consensus.cc:1685] finished
0709 13:22:23.567299 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:22:23.567465 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":15883,"raft.queue_time_us":16070,"raft.run_cpu_time_us":112,"raft.run_wall_time_us":27419,"thread_start_us":792870,"threads_started":1,"wal-append.queue_time_us":938363}
I20260709 13:22:23.592192 16816 raft_consensus.cc:1240] T 9e65676b21b748728ad39668cfe722ec P cb82cc7bd6db46d980c75237a1402860 [term 22 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 21. Current term is 22. Ops: [19.15-21.16]
I20260709 13:22:23.530964 23798 raft_consensus.cc:515] T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b [term 26 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:22:23.597487 23306 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.624s	user 0.004s	sys 0.016s
W20260709 13:22:23.599454 16189 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.505s	user 0.010s	sys 0.028s
W20260709 13:22:23.700009 16189 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.605s	user 0.010s	sys 0.028s
W20260709 13:22:23.700893 16266 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:22:23.702834 16189 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:22:23.703465 16189 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:22:23.703848 16189 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:22:23.704157 16189 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:22:23.651237 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 348898us
I20260709 13:22:23.585175 23894 raft_consensus.cc:515] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:22:23.597891 16790 leader_election.cc:336] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 20 election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:22:23.725344 16192 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 414448us
W20260709 13:22:23.702857 23886 log.cc:927] Time spent T b87147731a68400e8cce9b879daa7c5d P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.271s	user 0.001s	sys 0.001s
I20260709 13:22:23.722540 16790 leader_election.cc:304] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:22:23.609283 23730 leader_election.cc:290] T ecc1e363230043fe9fef390f0b979f20 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
W20260709 13:22:23.775074 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 379530us
I20260709 13:22:23.707017 23872 leader_election.cc:290] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:22:23.783353 16191 leader_election.cc:304] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
I20260709 13:22:23.704268 23603 raft_consensus.cc:697] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b [term 24 LEADER]: Becoming Leader. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
W20260709 13:22:23.776618 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 182650us
I20260709 13:22:23.721486 23598 raft_consensus.cc:2749] T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:22:23.793972 23685 raft_consensus.cc:2749] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:22:23.598351 23784 raft_consensus.cc:515] T 18ad7e3af5504571a69d10c00750a01c P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } }
W20260709 13:22:23.794313 23902 log.cc:927] Time spent T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 0.177s	user 0.000s	sys 0.001s
I20260709 13:22:23.713814 23800 consensus_queue.cc:1059] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: INVALID_TERM, Last received: 26.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.091s
W20260709 13:22:23.796854 16790 leader_election.cc:336] T 57a744f5aaf944678a63469cbf514015 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:22:23.831297 16299 raft_consensus.cc:1240] T 18ad7e3af5504571a69d10c00750a01c P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 17. Current term is 18. Ops: []
W20260709 13:22:23.835793 23906 log.cc:927] Time spent T b6f07842d3aa47858df5665874c86f3a P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
W20260709 13:22:23.844280 16790 leader_election.cc:336] T 23f82224f4564fea93572e8206bad1fe P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:22:23.845412 16790 leader_election.cc:336] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 19 election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:22:23.845849 16790 leader_election.cc:304] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
W20260709 13:22:23.859726 16790 consensus_peers.cc:597] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:22:23.863859 16142 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.510s	user 0.003s	sys 0.032s
I20260709 13:22:23.867801 16817 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49892 (request call id 1152) took 1267 ms. Trace:
W20260709 13:22:23.868573 16790 leader_election.cc:336] T af57996a8be84d398f0f023e8d08ef77 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 21 election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:22:23.871150 16142 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.515s	user 0.004s	sys 0.032s
W20260709 13:22:23.874312 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 169216us
W20260709 13:22:23.876336 16790 consensus_peers.cc:597] T ee9ed67006144797b1de6ea8c2013a68 P cb82cc7bd6db46d980c75237a1402860 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:22:23.867237 23638 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.042s	user 0.039s	sys 0.027s
I20260709 13:22:23.861358 23685 raft_consensus.cc:2749] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:22:23.862247 23210 leader_election.cc:290] T 23f82224f4564fea93572e8206bad1fe P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 18 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:23.794544 23817 raft_consensus.cc:515] T b6f07842d3aa47858df5665874c86f3a P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:23.878610 16820 raft_consensus.cc:1240] T 2794af5b99984efebdee564035841cb0 P cb82cc7bd6db46d980c75237a1402860 [term 20 LEADER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 19. Current term is 20. Ops: [19.16-19.16]
I20260709 13:22:23.875151 23762 leader_election.cc:290] T c9944d5e4b8c4109af9955c4f08c9095 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:23.889379 23784 leader_election.cc:290] T 18ad7e3af5504571a69d10c00750a01c P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 18 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
W20260709 13:22:23.882807 23638 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.059s	user 0.040s	sys 0.027s
I20260709 13:22:23.891525 16295 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40474 (request call id 1386) took 2554 ms. Trace:
I20260709 13:22:23.895062 16295 rpcz_store.cc:276] 0709 13:22:21.337322 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:22:21.776991 (+439669us) service_pool.cc:224] Handling call
0709 13:22:21.844170 (+ 67179us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:22:22.955523 (+1111353us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:22:22.955568 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:22:22.972808 (+ 17240us) log.cc:844] Serialized 26 byte log entry
0709 13:22:23.485204 (+512396us) raft_consensus.cc:1645] Marking committed up to 16
0709 13:22:23.485249 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:22:23.485344 (+    95us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:22:23.891255 (+405911us) raft_consensus.cc:1685] finished
0709 13:22:23.891285 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:22:23.891464 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":37070,"raft.queue_time_us":221122,"raft.run_cpu_time_us":128,"raft.run_wall_time_us":6457,"thread_start_us":1556778,"threads_started":2,"wal-append.queue_time_us":826736}
W20260709 13:22:23.899289 16192 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 152478us
W20260709 13:22:23.881553 16790 consensus_peers.cc:597] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:22:23.900833 16790 consensus_peers.cc:597] T 3884f864a96d428785895e5463e5728f P cb82cc7bd6db46d980c75237a1402860 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:22:23.888020 23694 consensus_meta.cc:287] T 57a744f5aaf944678a63469cbf514015 P cb82cc7bd6db46d980c75237a1402860: Time spent flushing consensus metadata: real 0.911s	user 0.006s	sys 0.006s
W20260709 13:22:23.893646 23630 consensus_meta.cc:287] T 7c2095a5f4ad49408dbc7f69c606120d P 1ca2725d290e47d68b1a4b609c9816ac: Time spent flushing consensus metadata: real 0.792s	user 0.011s	sys 0.003s
W20260709 13:22:23.908319 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 125126us
I20260709 13:22:23.887745 23921 raft_consensus.cc:2749] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 13:22:23.912290 23894 leader_election.cc:290] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:23.916271 23894 consensus_queue.cc:1048] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
W20260709 13:22:23.919375 23754 consensus_meta.cc:287] T 66f75124215f4ca6bef831e174a792d8 P cb82cc7bd6db46d980c75237a1402860: Time spent flushing consensus metadata: real 0.679s	user 0.011s	sys 0.001s
W20260709 13:22:23.919548 23910 log.cc:927] Time spent T 434bff3082d141838068f323bdb0d1ed P cb82cc7bd6db46d980c75237a1402860: Append to log took a long time: real 0.089s	user 0.001s	sys 0.000s
I20260709 13:22:23.919840 23754 raft_consensus.cc:515] T 66f75124215f4ca6bef831e174a792d8 P cb82cc7bd6db46d980c75237a1402860 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:23.921509 16151 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40844 (request call id 3106) took 2655 ms. Trace:
I20260709 13:22:23.871912 23603 consensus_queue.cc:237] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 22.13, Last appended by leader: 13, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:23.922225 16147 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40830 (request call id 2235) took 2821 ms. Trace:
W20260709 13:22:23.925089 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 498308us
I20260709 13:22:23.917721 23894 consensus_queue.cc:1059] T e5ad4eb281564936a59f41e387a47156 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:22:23.926311 16116 leader_election.cc:304] T d16dfad0c3b540e5b4b86492ddf87208 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 90f3da57796b488c9087a2f523adf045
I20260709 13:22:23.927949 23605 raft_consensus.cc:2749] T c0eab25ce4c64e018a041beeda2867db P 647a343a542d4a139fe106c19095129b [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 13:22:23.928830 16115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 485531us
I20260709 13:22:23.922626 16147 rpcz_store.cc:276] 0709 13:22:21.101286 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:22:22.272784 (+1171498us) service_pool.cc:224] Handling call
0709 13:22:22.286338 (+ 13554us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:22:22.286616 (+   278us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 13:22:23.013291 (+726675us) log.cc:844] Serialized 14 byte log entry
0709 13:22:23.921855 (+908564us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:22:23.921924 (+    69us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:22:23.921995 (+    71us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:22:23.922074 (+    79us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:22:23.922175 (+   101us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":523511,"prepare.queue_time_us":805967,"prepare.run_cpu_time_us":47,"prepare.run_wall_time_us":49,"raft.queue_time_us":525124,"raft.run_cpu_time_us":84,"raft.run_wall_time_us":84,"thread_start_us":1062463,"threads_started":2,"wal-append.queue_time_us":688127}
I20260709 13:22:23.927731 16823 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49892 (request call id 1151) took 1328 ms. Trace:
I20260709 13:22:23.929462 23774 raft_consensus.cc:2749] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:22:23.921957 16151 rpcz_store.cc:276] 0709 13:22:21.266365 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:22:22.276181 (+1009816us) service_pool.cc:224] Handling call
0709 13:22:22.276598 (+   417us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:22:22.276908 (+   310us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 13:22:22.978368 (+701460us) log.cc:844] Serialized 14 byte log entry
0709 13:22:23.383299 (+404931us) log.cc:844] Serialized 14 byte log entry
0709 13:22:23.920975 (+537676us) spinlock_profiling.cc:243] Waited 3.7 ms on lock 0x7b18004cd620. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b693ade1 00007f69b69475b1 00007f69ad9ecc0a 00007f69b712c219 00007f69ada74a40 00007f69ada6f4e4 00007f69ada6f455 00007f69b0b57060 00007f69b0b69716 00007f69b0b7a287 00007f69b0b7a208
0709 13:22:23.921121 (+   146us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:22:23.921181 (+    60us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:22:23.921244 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:22:23.921321 (+    77us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:22:23.921448 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":510765,"prepare.queue_time_us":814038,"prepare.run_cpu_time_us":48,"prepare.run_wall_time_us":95,"raft.queue_time_us":549795,"raft.run_cpu_time_us":193,"raft.run_wall_time_us":28008,"spinlock_wait_cycles":19620096,"thread_start_us":1092165,"threads_started":2,"wal-append.queue_time_us":855691}
W20260709 13:22:23.929064 16117 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:22:23.932439 23870 raft_consensus.cc:2804] T 08c74490c40243a88e62d1760b5cd814 P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:22:23.927371 23798 leader_election.cc:290] T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 26 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:23.937353 16144 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46224 (request call id 1300) took 2608 ms. Trace:
I20260709 13:22:23.932003 16823 rpcz_store.cc:276] 0709 13:22:22.599873 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:22:23.081601 (+481728us) service_pool.cc:224] Handling call
0709 13:22:23.081972 (+   371us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:22:23.082199 (+   227us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 13:22:23.400502 (+318303us) log.cc:844] Serialized 14 byte log entry
0709 13:22:23.869698 (+469196us) log.cc:844] Serialized 14 byte log entry
0709 13:22:23.927365 (+ 57667us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:22:23.927426 (+    61us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:22:23.927491 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:22:23.927556 (+    65us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:22:23.927674 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":13350,"prepare.queue_time_us":154124,"prepare.run_cpu_time_us":84,"prepare.run_wall_time_us":84,"raft.queue_time_us":9950,"raft.run_wall_time_us":2021,"spinlock_wait_cycles":24465280,"thread_start_us":801428,"threads_started":3,"wal-append.queue_time_us":502922}
I20260709 13:22:23.937842 16144 rpcz_store.cc:276] 0709 13:22:21.329111 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:22:22.362558 (+1033447us) service_pool.cc:224] Handling call
0709 13:22:22.415806 (+ 53248us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:22:22.416084 (+   278us) raft_consensus.cc:1534] Early marking committed up to index 16
0709 13:22:23.324097 (+908013us) log.cc:844] Serialized 14 byte log entry
0709 13:22:23.936883 (+612786us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:22:23.936967 (+    84us) raft_consensus.cc:1645] Marking committed up to 16
0709 13:22:23.937032 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:22:23.937121 (+    89us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:22:23.937265 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":183389,"prepare.queue_time_us":741438,"prepare.run_cpu_time_us":47,"prepare.run_wall_time_us":48,"thread_start_us":1328095,"threads_started":2,"wal-append.queue_time_us":555054}
W20260709 13:22:23.932804 23866 consensus_meta.cc:287] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045: Time spent flushing consensus metadata: real 0.721s	user 0.012s	sys 0.000s
I20260709 13:22:23.947515 23866 raft_consensus.cc:515] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:23.911315 16191 leader_election.cc:304] T 68a6838d82ef4cac83507f906139d90f P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:22:23.949574 23357 raft_consensus.cc:493] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:22:23.955438 16217 raft_consensus.cc:1275] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [term 23 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 22 index: 16. Preceding OpId from leader: term: 23 index: 16. (term mismatch)
I20260709 13:22:23.955853 16217 pending_rounds.cc:85] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac: Aborting all ops after (but not including) 15
I20260709 13:22:23.955449 16222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cc959fb7f704fba9009c0595467d074" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:22:23.956095 16217 pending_rounds.cc:107] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac: Aborting uncommitted NO_OP operation due to leader change: 22.16
I20260709 13:22:23.956382 16217 raft_consensus.cc:2889] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [term 23 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:22:23.956429 16222 raft_consensus.cc:3060] T 8cc959fb7f704fba9009c0595467d074 P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:22:23.957321 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8158bbd17b994d2eb96f560bd0f1b88d" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:22:23.957806 16217 raft_consensus.cc:2393] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 20: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
I20260709 13:22:23.958434 16221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5ad4eb281564936a59f41e387a47156" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:22:23.908411 23694 raft_consensus.cc:515] T 57a744f5aaf944678a63469cbf514015 P cb82cc7bd6db46d980c75237a1402860 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:23.960583 23357 raft_consensus.cc:515] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:23.962050 23694 leader_election.cc:290] T 57a744f5aaf944678a63469cbf514015 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 20 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:23.951409 23474 raft_consensus.cc:3055] T af57996a8be84d398f0f023e8d08ef77 P 1ca2725d290e47d68b1a4b609c9816ac [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:22:23.962881 23474 raft_consensus.cc:740] T af57996a8be84d398f0f023e8d08ef77 P 1ca2725d290e47d68b1a4b609c9816ac [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
W20260709 13:22:23.953475 16790 consensus_peers.cc:597] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:22:23.963742 23474 consensus_queue.cc:260] T af57996a8be84d398f0f023e8d08ef77 P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.13, Last appended by leader: 13, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:23.964958 23474 raft_consensus.cc:3060] T af57996a8be84d398f0f023e8d08ef77 P 1ca2725d290e47d68b1a4b609c9816ac [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:22:23.965831 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4f65cdf00b748bb963ee6e81ef43b79" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:22:23.966596 16223 raft_consensus.cc:1275] T e4dc6b0541ca431ebc757beac00521f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 22 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 22 index: 12. (index mismatch)
I20260709 13:22:23.967214 16821 raft_consensus.cc:2368] T 57a744f5aaf944678a63469cbf514015 P cb82cc7bd6db46d980c75237a1402860 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b for earlier term 19. Current term is 20.
I20260709 13:22:23.954349 16191 leader_election.cc:304] T 5a96373bb0e14dce8a266b2fc6c58507 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
W20260709 13:22:23.983338 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 57096.5us
I20260709 13:22:23.983511 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ad5dbfa99ee4b8d9e28be92b3f36db5" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:22:23.984771 16219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a878999c0024d2abe517484180290fc" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:22:23.985611 16219 raft_consensus.cc:3060] T 8a878999c0024d2abe517484180290fc P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:22:23.932585 23630 consensus_queue.cc:1059] T fffab26427e24d398e2695aad40ca134 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: INVALID_TERM, Last received: 20.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.027s
I20260709 13:22:23.988698 16291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5ad4eb281564936a59f41e387a47156" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:23.989552 16295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e65676b21b748728ad39668cfe722ec" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 22 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:22:23.990329 16220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1df93779209748b9860f6d893dd3de53" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:23.991629 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c61a39fd4f84424b129c1a6f9ba404d" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 23 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:22:23.996701 16220 raft_consensus.cc:2468] T 1df93779209748b9860f6d893dd3de53 P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 19.
I20260709 13:22:23.999130 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0eab25ce4c64e018a041beeda2867db" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:22:23.949671 23866 leader_election.cc:290] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 23 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:23.988358 23774 consensus_queue.cc:1048] T 02838c86531d4f98bf58d1c22b8baf77 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.014s
I20260709 13:22:24.003082 23913 raft_consensus.cc:493] T b6f07842d3aa47858df5665874c86f3a P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:22:24.003894 23913 raft_consensus.cc:515] T b6f07842d3aa47858df5665874c86f3a P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:24.005420 23817 leader_election.cc:290] T b6f07842d3aa47858df5665874c86f3a P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 21 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:24.009363 16291 raft_consensus.cc:2393] T e5ad4eb281564936a59f41e387a47156 P 90f3da57796b488c9087a2f523adf045 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 21: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
I20260709 13:22:24.011411 23925 raft_consensus.cc:3055] T e5ad4eb281564936a59f41e387a47156 P 1ca2725d290e47d68b1a4b609c9816ac [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:22:24.011876 23925 raft_consensus.cc:740] T e5ad4eb281564936a59f41e387a47156 P 1ca2725d290e47d68b1a4b609c9816ac [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:22:24.012918 23925 consensus_queue.cc:260] T e5ad4eb281564936a59f41e387a47156 P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.13, Last appended by leader: 13, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:24.014266 23925 raft_consensus.cc:3060] T e5ad4eb281564936a59f41e387a47156 P 1ca2725d290e47d68b1a4b609c9816ac [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:22:23.982589 23871 consensus_queue.cc:1059] T 8a878999c0024d2abe517484180290fc P 647a343a542d4a139fe106c19095129b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: INVALID_TERM, Last received: 18.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.033s
I20260709 13:22:24.016203 23357 leader_election.cc:290] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:24.017233 16818 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49906 (request call id 1477) took 1942 ms. Trace:
I20260709 13:22:24.017655 16818 rpcz_store.cc:276] 0709 13:22:22.074463 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:22:22.379939 (+305476us) service_pool.cc:224] Handling call
0709 13:22:22.419220 (+ 39281us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:22:23.375757 (+956537us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:22:24.016950 (+641193us) spinlock_profiling.cc:243] Waited 949 ms on lock 0x7b54006441d4. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:22:24.017164 (+   214us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2182379776}
I20260709 13:22:24.011915 16218 raft_consensus.cc:1275] T 68a6838d82ef4cac83507f906139d90f P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
W20260709 13:22:24.032522 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 48136.4us
I20260709 13:22:24.033558 16116 leader_election.cc:304] T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860
I20260709 13:22:24.035729 16222 raft_consensus.cc:2468] T 8cc959fb7f704fba9009c0595467d074 P 1ca2725d290e47d68b1a4b609c9816ac [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 20.
I20260709 13:22:23.994899 23605 raft_consensus.cc:2804] T d16dfad0c3b540e5b4b86492ddf87208 P 647a343a542d4a139fe106c19095129b [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:22:24.041303 16116 leader_election.cc:304] T 8cc959fb7f704fba9009c0595467d074 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 90f3da57796b488c9087a2f523adf045
I20260709 13:22:24.044492 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c11074579b046d99e585c2818e18a2c" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:24.031684 23932 raft_consensus.cc:2749] T 68a6838d82ef4cac83507f906139d90f P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:22:24.053318 16298 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40474 (request call id 1408) took 1522 ms. Trace:
I20260709 13:22:24.067586 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57a744f5aaf944678a63469cbf514015" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:22:24.068442 16300 raft_consensus.cc:3060] T 57a744f5aaf944678a63469cbf514015 P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:22:24.053743 16298 rpcz_store.cc:276] 0709 13:22:22.530506 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:22:22.683571 (+153065us) service_pool.cc:224] Handling call
0709 13:22:24.053150 (+1369579us) spinlock_profiling.cc:243] Waited 821 ms on lock 0x7b540035fcd4. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:22:24.053261 (+   111us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1887647488}
I20260709 13:22:24.073675 16219 raft_consensus.cc:2468] T 8a878999c0024d2abe517484180290fc P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 19.
I20260709 13:22:24.079761 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c74490c40243a88e62d1760b5cd814" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:24.073108 23897 raft_consensus.cc:493] T ee9ed67006144797b1de6ea8c2013a68 P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:22:24.081799 23897 raft_consensus.cc:515] T ee9ed67006144797b1de6ea8c2013a68 P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:22:24.083956 23930 log.cc:927] Time spent T 00000000000000000000000000000000 P 667a34b27a39474ebefc1af3cf5b4271: Append to log took a long time: real 0.064s	user 0.002s	sys 0.000s
I20260709 13:22:24.084110 23897 leader_election.cc:290] T ee9ed67006144797b1de6ea8c2013a68 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
W20260709 13:22:24.084450 23384 consensus_meta.cc:287] T 746aabf856ab401c9ef5e859371e7b67 P 1ca2725d290e47d68b1a4b609c9816ac: Time spent flushing consensus metadata: real 0.768s	user 0.011s	sys 0.000s
I20260709 13:22:24.084856 23384 raft_consensus.cc:515] T 746aabf856ab401c9ef5e859371e7b67 P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:24.085541 23719 raft_consensus.cc:2804] T 8cc959fb7f704fba9009c0595467d074 P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:22:24.086064 23719 raft_consensus.cc:697] T 8cc959fb7f704fba9009c0595467d074 P 647a343a542d4a139fe106c19095129b [term 19 LEADER]: Becoming Leader. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:22:24.049831 23605 raft_consensus.cc:493] T d16dfad0c3b540e5b4b86492ddf87208 P 647a343a542d4a139fe106c19095129b [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:24.087229 23605 raft_consensus.cc:3060] T d16dfad0c3b540e5b4b86492ddf87208 P 647a343a542d4a139fe106c19095129b [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:22:24.086977 23719 consensus_queue.cc:237] T 8cc959fb7f704fba9009c0595467d074 P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:22:24.051690 23831 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.110s	user 0.043s	sys 0.081s
I20260709 13:22:24.074790 23869 raft_consensus.cc:2749] T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:22:24.065179 23936 raft_consensus.cc:493] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:24.111110 16117 leader_election.cc:304] T 9e65676b21b748728ad39668cfe722ec P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860; no voters: 
W20260709 13:22:24.091008 23831 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.154s	user 0.044s	sys 0.081s
I20260709 13:22:24.051229 23908 consensus_queue.cc:1048] T 02838c86531d4f98bf58d1c22b8baf77 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.044s
I20260709 13:22:24.086994 23384 leader_election.cc:290] T 746aabf856ab401c9ef5e859371e7b67 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 19 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:24.118043 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:24.089998 23685 raft_consensus.cc:493] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:22:24.090940 23865 consensus_queue.cc:1059] T 9e65676b21b748728ad39668cfe722ec P 90f3da57796b488c9087a2f523adf045 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: INVALID_TERM, Last received: 17.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.039s
I20260709 13:22:24.064661 23892 raft_consensus.cc:515] T ee9ed67006144797b1de6ea8c2013a68 P 1ca2725d290e47d68b1a4b609c9816ac [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:24.091435 23929 raft_consensus.cc:493] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P cb82cc7bd6db46d980c75237a1402860 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:22:24.132195 16224 raft_consensus.cc:2393] T ecc1e363230043fe9fef390f0b979f20 P 1ca2725d290e47d68b1a4b609c9816ac [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 24: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
I20260709 13:22:24.098143 23024 consensus_queue.cc:1048] T b87147731a68400e8cce9b879daa7c5d P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.038s
I20260709 13:22:24.128389 23936 raft_consensus.cc:515] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:24.137815 23685 raft_consensus.cc:515] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:24.119216 23831 raft_consensus.cc:697] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [term 25 LEADER]: Becoming Leader. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
W20260709 13:22:24.169173 16189 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:22:24.170925 16115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 122191us
I20260709 13:22:24.174152 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ad5dbfa99ee4b8d9e28be92b3f36db5" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:24.175369 16298 raft_consensus.cc:2393] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 20: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
I20260709 13:22:24.149704 23894 raft_consensus.cc:2749] T 5a96373bb0e14dce8a266b2fc6c58507 P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:22:24.170140 23869 raft_consensus.cc:2804] T 9e65676b21b748728ad39668cfe722ec P 647a343a542d4a139fe106c19095129b [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:22:24.181206 16291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6f07842d3aa47858df5665874c86f3a" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:24.190047 16296 raft_consensus.cc:1240] T 8158bbd17b994d2eb96f560bd0f1b88d P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 19. Current term is 20. Ops: []
W20260709 13:22:24.187539 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 113600us
I20260709 13:22:24.194976 16219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4f65cdf00b748bb963ee6e81ef43b79" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:22:24.197769 16219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5ad4eb281564936a59f41e387a47156" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:24.198855 16219 raft_consensus.cc:2410] T e5ad4eb281564936a59f41e387a47156 P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b for term 21 because replica has last-logged OpId of term: 20 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 12.
I20260709 13:22:24.200718 16219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2372981006d4b218215c28964dea623" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 26 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:22:24.182318 16300 raft_consensus.cc:2468] T 57a744f5aaf944678a63469cbf514015 P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 20.
I20260709 13:22:24.203193 16219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ad5dbfa99ee4b8d9e28be92b3f36db5" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:24.141963 23929 raft_consensus.cc:515] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P cb82cc7bd6db46d980c75237a1402860 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:24.209779 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c2095a5f4ad49408dbc7f69c606120d" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 19 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
W20260709 13:22:24.216310 16189 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:22:24.218869 16224 raft_consensus.cc:2410] T 7c2095a5f4ad49408dbc7f69c606120d P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b for term 19 because replica has last-logged OpId of term: 18 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 8.
I20260709 13:22:24.227213 23943 consensus_queue.cc:1059] T 8158bbd17b994d2eb96f560bd0f1b88d P 647a343a542d4a139fe106c19095129b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: INVALID_TERM, Last received: 19.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:22:24.228729 23943 raft_consensus.cc:493] T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:24.229306 23943 raft_consensus.cc:515] T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:24.233790 16221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee9ed67006144797b1de6ea8c2013a68" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:24.234606 16221 raft_consensus.cc:2393] T ee9ed67006144797b1de6ea8c2013a68 P 1ca2725d290e47d68b1a4b609c9816ac [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 20: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
I20260709 13:22:24.219003 16789 leader_election.cc:304] T 23f82224f4564fea93572e8206bad1fe P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 647a343a542d4a139fe106c19095129b
I20260709 13:22:24.240207 16789 leader_election.cc:304] T 57a744f5aaf944678a63469cbf514015 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:22:24.248816 16218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "829b4ad8c5a54eababeea1181b960d4b" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:22:24.185329 23869 raft_consensus.cc:697] T 9e65676b21b748728ad39668cfe722ec P 647a343a542d4a139fe106c19095129b [term 22 LEADER]: Becoming Leader. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
W20260709 13:22:24.255255 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 301005us
I20260709 13:22:24.259147 16220 raft_consensus.cc:1240] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac [term 20 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 19. Current term is 20. Ops: []
I20260709 13:22:24.259675 16191 leader_election.cc:304] T d16dfad0c3b540e5b4b86492ddf87208 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 90f3da57796b488c9087a2f523adf045
I20260709 13:22:24.261469 16191 leader_election.cc:304] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:22:24.262317 16222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af57996a8be84d398f0f023e8d08ef77" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:22:24.191682 23357 raft_consensus.cc:493] T 5a959dbecc2648eb85d1cc550f5a3245 P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:22:24.231498 23944 raft_consensus.cc:3055] T 8158bbd17b994d2eb96f560bd0f1b88d P 647a343a542d4a139fe106c19095129b [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:22:24.262908 16191 leader_election.cc:304] T c2f5f89186204242bf377f7c42b2e2da P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:22:24.217717 23894 raft_consensus.cc:493] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:24.264338 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6f07842d3aa47858df5665874c86f3a" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:24.276736 16226 raft_consensus.cc:2393] T b6f07842d3aa47858df5665874c86f3a P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 21: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
I20260709 13:22:24.276832 16191 leader_election.cc:304] T 19baa7bc367844c288bca8f81439c413 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860; no voters: 90f3da57796b488c9087a2f523adf045
I20260709 13:22:24.264045 23474 raft_consensus.cc:2749] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
W20260709 13:22:24.289162 16189 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:22:24.268545 23944 raft_consensus.cc:740] T 8158bbd17b994d2eb96f560bd0f1b88d P 647a343a542d4a139fe106c19095129b [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:22:24.291239 16222 raft_consensus.cc:2468] T af57996a8be84d398f0f023e8d08ef77 P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 21.
I20260709 13:22:24.311566 16824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee9ed67006144797b1de6ea8c2013a68" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:24.312198 16822 raft_consensus.cc:1275] T 806d7690c2b644959152074bb2cab0a7 P cb82cc7bd6db46d980c75237a1402860 [term 21 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 20 index: 14. Preceding OpId from leader: term: 21 index: 14. (term mismatch)
I20260709 13:22:24.311915 16823 raft_consensus.cc:1240] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860 [term 19 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 18. Current term is 19. Ops: []
I20260709 13:22:24.271273 23940 raft_consensus.cc:2804] T 23f82224f4564fea93572e8206bad1fe P cb82cc7bd6db46d980c75237a1402860 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:22:24.304381 16818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2372981006d4b218215c28964dea623" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 26 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:22:24.319854 16818 raft_consensus.cc:3060] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:22:24.298087 16297 raft_consensus.cc:1275] T 5a96373bb0e14dce8a266b2fc6c58507 P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 13:22:24.298457 16191 leader_election.cc:304] T e4dc6b0541ca431ebc757beac00521f1 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:22:24.318470 16822 pending_rounds.cc:85] T 806d7690c2b644959152074bb2cab0a7 P cb82cc7bd6db46d980c75237a1402860: Aborting all ops after (but not including) 13
I20260709 13:22:24.285475 23951 raft_consensus.cc:2804] T 57a744f5aaf944678a63469cbf514015 P cb82cc7bd6db46d980c75237a1402860 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:22:24.297729 23938 raft_consensus.cc:2749] T c2f5f89186204242bf377f7c42b2e2da P 1ca2725d290e47d68b1a4b609c9816ac [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 13:22:24.368297 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 69968.2us
I20260709 13:22:24.325941 16822 pending_rounds.cc:107] T 806d7690c2b644959152074bb2cab0a7 P cb82cc7bd6db46d980c75237a1402860: Aborting uncommitted CHANGE_CONFIG_OP operation due to leader change: 20.14
I20260709 13:22:24.372589 16822 raft_consensus.cc:2917] T 806d7690c2b644959152074bb2cab0a7 P cb82cc7bd6db46d980c75237a1402860 [term 21 FOLLOWER]: Aborting config change with OpId 20.14: Aborted: Op aborted by new leader
I20260709 13:22:24.373135 16822 raft_consensus.cc:2889] T 806d7690c2b644959152074bb2cab0a7 P cb82cc7bd6db46d980c75237a1402860 [term 21 LEARNER]: CHANGE_CONFIG_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:22:24.373374 16191 leader_election.cc:304] T ecc1e363230043fe9fef390f0b979f20 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:22:24.269790 23357 raft_consensus.cc:515] T 5a959dbecc2648eb85d1cc550f5a3245 P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:24.285815 23925 raft_consensus.cc:2804] T d16dfad0c3b540e5b4b86492ddf87208 P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:22:24.320506 23921 raft_consensus.cc:2804] T 19baa7bc367844c288bca8f81439c413 P 1ca2725d290e47d68b1a4b609c9816ac [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:22:24.382092 23921 raft_consensus.cc:493] T 19baa7bc367844c288bca8f81439c413 P 1ca2725d290e47d68b1a4b609c9816ac [term 23 FOLLOWER]: Starting leader election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:22:24.320518 23940 raft_consensus.cc:493] T 23f82224f4564fea93572e8206bad1fe P cb82cc7bd6db46d980c75237a1402860 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:24.269795 23831 consensus_queue.cc:237] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 23.16, Last appended by leader: 16, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:24.285537 23894 raft_consensus.cc:515] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:24.382586 23921 raft_consensus.cc:3060] T 19baa7bc367844c288bca8f81439c413 P 1ca2725d290e47d68b1a4b609c9816ac [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:22:24.374294 23943 leader_election.cc:290] T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:24.404268 23286 raft_consensus.cc:493] T 84f2da98864b4425bf7c0055951fdd4b P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:24.404776 23908 raft_consensus.cc:493] T 23f82224f4564fea93572e8206bad1fe P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:24.405300 23908 raft_consensus.cc:515] T 23f82224f4564fea93572e8206bad1fe P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:24.405750 16142 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40844 (request call id 3097) took 3598 ms. Trace:
W20260709 13:22:24.400132 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 412478us
I20260709 13:22:24.380709 23901 raft_consensus.cc:2749] T e4dc6b0541ca431ebc757beac00521f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:22:24.406273 16142 rpcz_store.cc:276] 0709 13:22:20.807616 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:22:22.024278 (+1216662us) service_pool.cc:224] Handling call
0709 13:22:22.047358 (+ 23080us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:22:22.096535 (+ 49177us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:22:22.972672 (+876137us) log.cc:844] Serialized 14 byte log entry
0709 13:22:23.353424 (+380752us) spinlock_profiling.cc:243] Waited 1.2 ms on lock 0x7b18000201a0. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b693ade1 00007f69b69475b1 00007f69ada54e07 00007f69b68afab9 00007f69ada7169b 00007f69ada6f93f 00007f69ada71c02 00007f69b06f920b 00007f69b06fed23 00007f69b06ff23a 00007f69b0b69a2a
0709 13:22:23.872467 (+519043us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:22:23.883060 (+ 10593us) log.cc:844] Serialized 52 byte log entry
0709 13:22:23.883515 (+   455us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:22:24.404984 (+521469us) log.cc:844] Serialized 14 byte log entry
0709 13:22:24.405241 (+   257us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:22:24.405343 (+   102us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:22:24.405491 (+   148us) raft_consensus.cc:1685] finished
0709 13:22:24.405510 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:22:24.405679 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":552785,"prepare.queue_time_us":1400916,"prepare.run_cpu_time_us":101,"prepare.run_wall_time_us":103,"raft.queue_time_us":32326,"raft.run_cpu_time_us":74,"raft.run_wall_time_us":74,"spinlock_wait_cycles":2754816,"thread_start_us":1370620,"threads_started":2,"wal-append.queue_time_us":902989}
I20260709 13:22:24.407745 23778 raft_consensus.cc:493] T b87147731a68400e8cce9b879daa7c5d P cb82cc7bd6db46d980c75237a1402860 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:24.380362 23892 leader_election.cc:290] T ee9ed67006144797b1de6ea8c2013a68 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 20 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:24.411749 16822 raft_consensus.cc:1240] T ada5a004ba7c46819051e3ccec71ffdb P cb82cc7bd6db46d980c75237a1402860 [term 23 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 22. Current term is 23. Ops: []
I20260709 13:22:24.411284 23778 raft_consensus.cc:515] T b87147731a68400e8cce9b879daa7c5d P cb82cc7bd6db46d980c75237a1402860 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:24.413793 16824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746aabf856ab401c9ef5e859371e7b67" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:22:24.414994 23778 leader_election.cc:290] T b87147731a68400e8cce9b879daa7c5d P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:24.416050 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdbd78acb4cf46298e0de8ba0d130e19" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 25 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:22:24.416100 23778 raft_consensus.cc:493] T c9944d5e4b8c4109af9955c4f08c9095 P cb82cc7bd6db46d980c75237a1402860 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:22:24.417531 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6f07842d3aa47858df5665874c86f3a" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:22:24.419368 16296 raft_consensus.cc:1240] T c0eab25ce4c64e018a041beeda2867db P 90f3da57796b488c9087a2f523adf045 [term 23 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 22. Current term is 23. Ops: []
I20260709 13:22:24.420928 16294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74ca8140f35c4ae29e8a664f6bdfc829" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:24.422165 16300 raft_consensus.cc:1240] T c4f65cdf00b748bb963ee6e81ef43b79 P 90f3da57796b488c9087a2f523adf045 [term 22 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 21. Current term is 22. Ops: []
I20260709 13:22:23.872078 16817 rpcz_store.cc:276] 0709 13:22:22.600215 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:22:23.093126 (+492911us) service_pool.cc:224] Handling call
0709 13:22:23.110731 (+ 17605us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:22:23.110960 (+   229us) raft_consensus.cc:1534] Early marking committed up to index 16
0709 13:22:23.461357 (+350397us) log.cc:844] Serialized 14 byte log entry
0709 13:22:23.867263 (+405906us) log.cc:844] Serialized 14 byte log entry
0709 13:22:23.867471 (+   208us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:22:23.867516 (+    45us) raft_consensus.cc:1645] Marking committed up to 16
0709 13:22:23.867564 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:22:23.867630 (+    66us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:22:23.867740 (+   110us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":138921,"prepare.run_cpu_time_us":91,"prepare.run_wall_time_us":93,"raft.queue_time_us":345,"raft.run_cpu_time_us":132,"raft.run_wall_time_us":8668,"spinlock_wait_cycles":14566784,"thread_start_us":699892,"threads_started":2}
I20260709 13:22:24.424811 16819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74ca8140f35c4ae29e8a664f6bdfc829" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:24.440207 16819 raft_consensus.cc:2468] T 74ca8140f35c4ae29e8a664f6bdfc829 P cb82cc7bd6db46d980c75237a1402860 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 23.
I20260709 13:22:24.404819 23286 raft_consensus.cc:515] T 84f2da98864b4425bf7c0055951fdd4b P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:24.428059 23915 raft_consensus.cc:3055] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [term 26 LEADER]: Stepping down as leader of term 26
I20260709 13:22:24.415136 16824 raft_consensus.cc:1240] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [term 27 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 26. Current term is 27. Ops: []
I20260709 13:22:24.416379 23907 raft_consensus.cc:3055] T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:22:24.386543 23940 raft_consensus.cc:3060] T 23f82224f4564fea93572e8206bad1fe P cb82cc7bd6db46d980c75237a1402860 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:22:24.401912 23945 raft_consensus.cc:2749] T ecc1e363230043fe9fef390f0b979f20 P 1ca2725d290e47d68b1a4b609c9816ac [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 13:22:24.447561 16824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee9ed67006144797b1de6ea8c2013a68" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:22:24.447742 16117 leader_election.cc:400] T 57a744f5aaf944678a63469cbf514015 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 19 pre-election: Vote denied by peer cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923) with higher term. Message: Invalid argument: T 57a744f5aaf944678a63469cbf514015 P cb82cc7bd6db46d980c75237a1402860 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b for earlier term 19. Current term is 20.
I20260709 13:22:24.448670 16117 leader_election.cc:403] T 57a744f5aaf944678a63469cbf514015 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 19 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:22:24.378152 23942 consensus_queue.cc:1048] T 5a96373bb0e14dce8a266b2fc6c58507 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.061s
I20260709 13:22:24.472927 16294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87147731a68400e8cce9b879daa7c5d" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 23 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:24.476574 23818 consensus_queue.cc:1059] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: INVALID_TERM, Last received: 21.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.007s
I20260709 13:22:24.483614 23945 consensus_queue.cc:1059] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: INVALID_TERM, Last received: 21.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 13:22:24.484797 23945 raft_consensus.cc:3055] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:22:24.484642 23818 consensus_queue.cc:1059] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: INVALID_TERM, Last received: 22.16, Next index: 17, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 13:22:24.485211 23945 raft_consensus.cc:740] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:22:24.485731 23818 raft_consensus.cc:3055] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 1ca2725d290e47d68b1a4b609c9816ac [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:22:24.486222 23818 raft_consensus.cc:740] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 1ca2725d290e47d68b1a4b609c9816ac [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
W20260709 13:22:24.487254 23950 log.cc:927] Time spent T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 0.104s	user 0.001s	sys 0.000s
I20260709 13:22:24.435720 23871 raft_consensus.cc:493] T 9c61a39fd4f84424b129c1a6f9ba404d P 647a343a542d4a139fe106c19095129b [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
W20260709 13:22:24.445271 23934 log.cc:927] Time spent T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 0.363s	user 0.004s	sys 0.000s
I20260709 13:22:24.435525 23953 raft_consensus.cc:493] T 26354361e9b044cea1c2b86c4882fe92 P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:22:24.480806 16192 leader_election.cc:304] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:22:24.444967 23915 raft_consensus.cc:740] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [term 26 LEADER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:22:24.445658 23907 raft_consensus.cc:740] T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:22:24.449460 23936 leader_election.cc:290] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
W20260709 13:22:24.542688 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 63032.7us
W20260709 13:22:24.547376 16192 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 66635.3us
I20260709 13:22:24.545236 16294 raft_consensus.cc:1240] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045 [term 23 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 22. Current term is 23. Ops: []
I20260709 13:22:24.541549 23959 raft_consensus.cc:3060] T 57a744f5aaf944678a63469cbf514015 P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Advancing to term 20
I20260709 13:22:24.440419 23474 consensus_queue.cc:1048] T c0eab25ce4c64e018a041beeda2867db P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.066s
I20260709 13:22:24.442934 23778 raft_consensus.cc:515] T c9944d5e4b8c4109af9955c4f08c9095 P cb82cc7bd6db46d980c75237a1402860 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:24.565953 16149 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40844 (request call id 3102) took 3701 ms. Trace:
I20260709 13:22:24.490358 23908 leader_election.cc:290] T 23f82224f4564fea93572e8206bad1fe P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:24.387768 23869 consensus_queue.cc:237] T 9e65676b21b748728ad39668cfe722ec P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:22:24.582854 23954 log.cc:927] Time spent T 02838c86531d4f98bf58d1c22b8baf77 P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.112s	user 0.000s	sys 0.000s
I20260709 13:22:24.444866 23943 consensus_queue.cc:1059] T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: INVALID_TERM, Last received: 18.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.100s
I20260709 13:22:24.585734 23943 raft_consensus.cc:493] T 68a6838d82ef4cac83507f906139d90f P 647a343a542d4a139fe106c19095129b [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:24.586232 23943 raft_consensus.cc:515] T 68a6838d82ef4cac83507f906139d90f P 647a343a542d4a139fe106c19095129b [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:24.586563 23870 raft_consensus.cc:493] T 08c74490c40243a88e62d1760b5cd814 P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Starting leader election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:22:24.471233 23774 consensus_queue.cc:1048] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.083s
W20260709 13:22:24.543578 23949 log.cc:927] Time spent T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 0.162s	user 0.000s	sys 0.001s
W20260709 13:22:24.582015 23952 log.cc:927] Time spent T 8d4ab870a05347868956aa8b841b2981 P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.199s	user 0.000s	sys 0.000s
I20260709 13:22:24.495563 23287 raft_consensus.cc:493] T cdbd78acb4cf46298e0de8ba0d130e19 P 90f3da57796b488c9087a2f523adf045 [term 24 FOLLOWER]: Starting leader election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:22:24.591189 23754 leader_election.cc:290] T 66f75124215f4ca6bef831e174a792d8 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 20 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:22:24.573432 16149 rpcz_store.cc:276] 0709 13:22:20.865056 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:22:22.171504 (+1306448us) service_pool.cc:224] Handling call
0709 13:22:22.194345 (+ 22841us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:22:22.194574 (+   229us) raft_consensus.cc:1534] Early marking committed up to index 15
0709 13:22:22.971076 (+776502us) log.cc:844] Serialized 14 byte log entry
0709 13:22:24.402859 (+1431783us) spinlock_profiling.cc:243] Waited 4.06 ms on lock 0x7b180039c160. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b693ade1 00007f69b69475b1 00007f69ad9ecc0a 00007f69b712c219 00007f69ada74a40 00007f69ada6f4e4 00007f69ada6f455 00007f69b45573c7 00007f69b0b699a7 00007f69b0b790bf 00007f69b0b79038
0709 13:22:24.403066 (+   207us) log.cc:844] Serialized 14 byte log entry
0709 13:22:24.565433 (+162367us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:22:24.565516 (+    83us) raft_consensus.cc:1645] Marking committed up to 15
0709 13:22:24.565587 (+    71us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:22:24.565696 (+   109us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:22:24.565871 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":501072,"prepare.queue_time_us":1838197,"prepare.run_cpu_time_us":125,"prepare.run_wall_time_us":2300,"raft.queue_time_us":28377,"raft.run_cpu_time_us":238,"raft.run_wall_time_us":6357,"spinlock_wait_cycles":9328512,"thread_start_us":1309989,"threads_started":3,"wal-append.queue_time_us":634506}
I20260709 13:22:24.588351 23943 leader_election.cc:290] T 68a6838d82ef4cac83507f906139d90f P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:24.592906 16295 raft_consensus.cc:3055] T 2794af5b99984efebdee564035841cb0 P 90f3da57796b488c9087a2f523adf045 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:22:24.438145 23944 consensus_queue.cc:260] T 8158bbd17b994d2eb96f560bd0f1b88d P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:24.637842 16116 leader_election.cc:304] T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:22:24.588531 23915 consensus_queue.cc:260] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 26.16, Last appended by leader: 16, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:24.509970 23941 raft_consensus.cc:493] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac [term 20 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:22:24.543463 23953 raft_consensus.cc:515] T 26354361e9b044cea1c2b86c4882fe92 P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:24.587216 23870 raft_consensus.cc:3060] T 08c74490c40243a88e62d1760b5cd814 P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:22:24.537045 23924 raft_consensus.cc:493] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:22:24.564483 23598 raft_consensus.cc:2804] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:22:24.570457 23772 raft_consensus.cc:2749] T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:22:24.638312 16295 raft_consensus.cc:740] T 2794af5b99984efebdee564035841cb0 P 90f3da57796b488c9087a2f523adf045 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:22:24.542486 23938 raft_consensus.cc:493] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:24.565696 23630 consensus_queue.cc:1059] T fffab26427e24d398e2695aad40ca134 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: INVALID_TERM, Last received: 20.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.056s
I20260709 13:22:24.593703 23929 leader_election.cc:290] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:24.486059 23945 consensus_queue.cc:260] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 21.14, Last appended by leader: 14, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:24.535792 23818 consensus_queue.cc:260] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 21.16, Last appended by leader: 16, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:24.634781 23785 raft_consensus.cc:493] T 434bff3082d141838068f323bdb0d1ed P cb82cc7bd6db46d980c75237a1402860 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:22:24.648763 16116 leader_election.cc:304] T e5ad4eb281564936a59f41e387a47156 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:22:24.590768 23907 consensus_queue.cc:260] T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 22.12, Last appended by leader: 12, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:24.538101 23871 raft_consensus.cc:515] T 9c61a39fd4f84424b129c1a6f9ba404d P 647a343a542d4a139fe106c19095129b [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:24.635865 23287 raft_consensus.cc:3060] T cdbd78acb4cf46298e0de8ba0d130e19 P 90f3da57796b488c9087a2f523adf045 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:22:24.681984 16818 raft_consensus.cc:2410] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b for term 26 because replica has last-logged OpId of term: 25 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 23 index: 15.
I20260709 13:22:24.671974 16295 consensus_queue.cc:260] T 2794af5b99984efebdee564035841cb0 P 90f3da57796b488c9087a2f523adf045 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.16, Last appended by leader: 16, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:24.693192 16295 raft_consensus.cc:3060] T 2794af5b99984efebdee564035841cb0 P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:22:24.653329 23943 raft_consensus.cc:493] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [term 26 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:22:24.604140 23936 raft_consensus.cc:493] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:24.658073 23645 raft_consensus.cc:493] T d2ab0ac8347641e6955e9bce7d90c1a2 P cb82cc7bd6db46d980c75237a1402860 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:22:24.601263 23947 consensus_queue.cc:1048] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.071s
I20260709 13:22:24.689908 16116 leader_election.cc:304] T 7c2095a5f4ad49408dbc7f69c606120d P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:22:24.699673 16825 raft_consensus.cc:1240] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 [term 26 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 25. Current term is 26. Ops: []
I20260709 13:22:24.665453 23598 raft_consensus.cc:493] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:24.639381 23926 consensus_queue.cc:1059] T 829b4ad8c5a54eababeea1181b960d4b P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.050s
I20260709 13:22:24.684790 23944 raft_consensus.cc:3060] T 8158bbd17b994d2eb96f560bd0f1b88d P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:22:24.694818 23942 raft_consensus.cc:3055] T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:22:24.658875 23927 consensus_queue.cc:1059] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: INVALID_TERM, Last received: 26.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.043s
I20260709 13:22:24.711683 23598 raft_consensus.cc:3060] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:22:24.657140 23941 raft_consensus.cc:515] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:22:24.742166 16265 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:22:24.712211 23357 leader_election.cc:290] T 5a959dbecc2648eb85d1cc550f5a3245 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:24.672785 23924 raft_consensus.cc:515] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:24.721199 23907 raft_consensus.cc:3060] T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:22:24.724684 23818 raft_consensus.cc:3060] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:22:24.727125 23942 raft_consensus.cc:740] T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:22:24.713954 23645 raft_consensus.cc:515] T d2ab0ac8347641e6955e9bce7d90c1a2 P cb82cc7bd6db46d980c75237a1402860 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:24.707533 23943 raft_consensus.cc:515] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:24.731884 23945 raft_consensus.cc:3060] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:22:24.684643 23938 raft_consensus.cc:515] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:22:24.767439 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 77605.8us
I20260709 13:22:24.770974 16116 leader_election.cc:304] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:22:24.756608 16295 raft_consensus.cc:1275] T 2794af5b99984efebdee564035841cb0 P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 19 index: 16. Preceding OpId from leader: term: 20 index: 16. (term mismatch)
I20260709 13:22:24.771798 16295 pending_rounds.cc:85] T 2794af5b99984efebdee564035841cb0 P 90f3da57796b488c9087a2f523adf045: Aborting all ops after (but not including) 15
I20260709 13:22:24.772184 16295 pending_rounds.cc:107] T 2794af5b99984efebdee564035841cb0 P 90f3da57796b488c9087a2f523adf045: Aborting uncommitted NO_OP operation due to leader change: 19.16
I20260709 13:22:24.772537 16295 raft_consensus.cc:2889] T 2794af5b99984efebdee564035841cb0 P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:22:24.776531 16116 leader_election.cc:304] T 829b4ad8c5a54eababeea1181b960d4b P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860
W20260709 13:22:24.777566 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 181420us
I20260709 13:22:24.763612 23869 raft_consensus.cc:3060] T e5ad4eb281564936a59f41e387a47156 P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:22:24.754323 23286 leader_election.cc:290] T 84f2da98864b4425bf7c0055951fdd4b P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:24.704147 23785 raft_consensus.cc:515] T 434bff3082d141838068f323bdb0d1ed P cb82cc7bd6db46d980c75237a1402860 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:24.713822 23936 raft_consensus.cc:515] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:24.773398 23774 raft_consensus.cc:2749] T 7c2095a5f4ad49408dbc7f69c606120d P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:22:24.781247 16116 leader_election.cc:304] T ee9ed67006144797b1de6ea8c2013a68 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860
I20260709 13:22:24.820591 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:24.821434 16222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5d59407eb545b895d8d922e0dd69fd" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:22:24.805634 23778 leader_election.cc:290] T c9944d5e4b8c4109af9955c4f08c9095 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:24.824235 16224 raft_consensus.cc:2393] T ecc1e363230043fe9fef390f0b979f20 P 1ca2725d290e47d68b1a4b609c9816ac [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 in current term 24: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
W20260709 13:22:24.805984 23971 log.cc:927] Time spent T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 0.064s	user 0.000s	sys 0.001s
W20260709 13:22:24.806108 23957 log.cc:927] Time spent T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.103s	user 0.001s	sys 0.000s
I20260709 13:22:24.830986 16220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c11074579b046d99e585c2818e18a2c" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:24.832209 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "829b4ad8c5a54eababeea1181b960d4b" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:24.806835 23976 raft_consensus.cc:493] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:24.811689 23975 raft_consensus.cc:3060] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:22:24.811425 23942 consensus_queue.cc:260] T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 18.15, Last appended by leader: 15, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:24.765141 23921 raft_consensus.cc:515] T 19baa7bc367844c288bca8f81439c413 P 1ca2725d290e47d68b1a4b609c9816ac [term 24 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:22:24.812839 23966 log.cc:927] Time spent T 8638ee363273426db1393dd81e5c7b72 P cb82cc7bd6db46d980c75237a1402860: Append to log took a long time: real 0.090s	user 0.000s	sys 0.000s
I20260709 13:22:24.813446 23978 raft_consensus.cc:2749] T 829b4ad8c5a54eababeea1181b960d4b P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:22:24.821851 16217 raft_consensus.cc:1240] T 746aabf856ab401c9ef5e859371e7b67 P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 18. Current term is 19. Ops: []
I20260709 13:22:24.849766 23921 leader_election.cc:290] T 19baa7bc367844c288bca8f81439c413 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 24 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:24.827692 16226 raft_consensus.cc:1240] T ee9ed67006144797b1de6ea8c2013a68 P 1ca2725d290e47d68b1a4b609c9816ac [term 20 FOLLOWER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 19. Current term is 20. Ops: []
I20260709 13:22:24.860354 23645 leader_election.cc:290] T d2ab0ac8347641e6955e9bce7d90c1a2 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:24.861018 23024 consensus_queue.cc:1048] T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 13:22:24.863409 23940 raft_consensus.cc:515] T 23f82224f4564fea93572e8206bad1fe P cb82cc7bd6db46d980c75237a1402860 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:24.865716 23940 leader_election.cc:290] T 23f82224f4564fea93572e8206bad1fe P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 19 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:24.868120 23925 raft_consensus.cc:493] T d16dfad0c3b540e5b4b86492ddf87208 P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Starting leader election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:22:24.868674 23925 raft_consensus.cc:3060] T d16dfad0c3b540e5b4b86492ddf87208 P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:22:24.868758 23941 leader_election.cc:290] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
W20260709 13:22:24.870702 23969 log.cc:927] Time spent T ada5a004ba7c46819051e3ccec71ffdb P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.161s	user 0.001s	sys 0.000s
W20260709 13:22:24.875622 23948 log.cc:927] Time spent T d16dfad0c3b540e5b4b86492ddf87208 P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 0.161s	user 0.000s	sys 0.001s
I20260709 13:22:24.876240 23924 leader_election.cc:290] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:24.830943 23981 raft_consensus.cc:493] T 84f2da98864b4425bf7c0055951fdd4b P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:24.836886 23965 raft_consensus.cc:3060] T ee9ed67006144797b1de6ea8c2013a68 P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:22:24.863947 23605 consensus_meta.cc:287] T d16dfad0c3b540e5b4b86492ddf87208 P 647a343a542d4a139fe106c19095129b: Time spent flushing consensus metadata: real 0.776s	user 0.011s	sys 0.001s
I20260709 13:22:24.905107 16222 raft_consensus.cc:2468] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 24.
W20260709 13:22:24.907079 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 194928us
W20260709 13:22:24.913309 16190 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:22:24.914460 16291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4dc6b0541ca431ebc757beac00521f1" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 23 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:24.916198 16292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:22:24.916842 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8638ee363273426db1393dd81e5c7b72" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:24.917654 16298 raft_consensus.cc:2468] T 8638ee363273426db1393dd81e5c7b72 P 90f3da57796b488c9087a2f523adf045 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 21.
W20260709 13:22:24.865728 23972 log.cc:927] Time spent T c0eab25ce4c64e018a041beeda2867db P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 0.082s	user 0.000s	sys 0.000s
I20260709 13:22:24.920928 16190 leader_election.cc:304] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860
I20260709 13:22:24.935662 16291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f82224f4564fea93572e8206bad1fe" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:24.936583 16222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74ca8140f35c4ae29e8a664f6bdfc829" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 24 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:24.937361 16223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8638ee363273426db1393dd81e5c7b72" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:22:24.943014 16295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a378d7f796f94615b3b347119a9880fd" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:24.880980 23943 leader_election.cc:290] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:24.879125 23953 leader_election.cc:290] T 26354361e9b044cea1c2b86c4882fe92 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:24.960036 23941 consensus_queue.cc:1059] T c0eab25ce4c64e018a041beeda2867db P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: INVALID_TERM, Last received: 22.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:22:24.900104 23976 raft_consensus.cc:515] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:24.859148 23936 leader_election.cc:290] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:24.880656 23785 leader_election.cc:290] T 434bff3082d141838068f323bdb0d1ed P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:24.919772 23605 raft_consensus.cc:515] T d16dfad0c3b540e5b4b86492ddf87208 P 647a343a542d4a139fe106c19095129b [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:24.881238 23981 raft_consensus.cc:515] T 84f2da98864b4425bf7c0055951fdd4b P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:25.143407 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada5a004ba7c46819051e3ccec71ffdb" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
W20260709 13:22:25.112092 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 163372us
W20260709 13:22:25.113060 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 173814us
I20260709 13:22:25.113626 16295 raft_consensus.cc:2468] T a378d7f796f94615b3b347119a9880fd P 90f3da57796b488c9087a2f523adf045 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 21.
I20260709 13:22:24.936537 16291 raft_consensus.cc:2468] T 23f82224f4564fea93572e8206bad1fe P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 18.
I20260709 13:22:24.959725 23964 raft_consensus.cc:2749] T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:22:25.117880 23865 raft_consensus.cc:493] T 26354361e9b044cea1c2b86c4882fe92 P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:25.119704 16292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68a6838d82ef4cac83507f906139d90f" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:24.924957 23774 raft_consensus.cc:493] T ecc1e363230043fe9fef390f0b979f20 P 647a343a542d4a139fe106c19095129b [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:25.123107 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af57996a8be84d398f0f023e8d08ef77" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:22:25.157428 16217 raft_consensus.cc:2393] T af57996a8be84d398f0f023e8d08ef77 P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 in current term 21: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
I20260709 13:22:24.879701 23938 leader_election.cc:290] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:25.150380 16291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26354361e9b044cea1c2b86c4882fe92" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:25.114974 23357 raft_consensus.cc:493] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:25.154086 16190 leader_election.cc:304] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:22:25.157220 16294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a959dbecc2648eb85d1cc550f5a3245" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:25.140590 16224 raft_consensus.cc:1275] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac [term 22 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 18 index: 15. Preceding OpId from leader: term: 20 index: 16. (index mismatch)
W20260709 13:22:25.193733 16263 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:22:25.114741 23924 raft_consensus.cc:2749] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
W20260709 13:22:25.286703 16266 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:22:25.194598 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66ae176961b74813a58bc15c9c4c6f2a" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 27 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
W20260709 13:22:25.179876 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 412880us
I20260709 13:22:25.191483 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f2da98864b4425bf7c0055951fdd4b" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:25.127452 23985 raft_consensus.cc:493] T b87147731a68400e8cce9b879daa7c5d P 647a343a542d4a139fe106c19095129b [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:25.112668 23978 raft_consensus.cc:493] T 746aabf856ab401c9ef5e859371e7b67 P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:22:25.194010 23774 raft_consensus.cc:515] T ecc1e363230043fe9fef390f0b979f20 P 647a343a542d4a139fe106c19095129b [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:25.253278 16292 raft_consensus.cc:1275] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045 [term 24 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 23 index: 13. Preceding OpId from leader: term: 24 index: 14. (index mismatch)
I20260709 13:22:25.147464 23761 raft_consensus.cc:493] T a378d7f796f94615b3b347119a9880fd P 90f3da57796b488c9087a2f523adf045 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:22:25.253888 23601 raft_consensus.cc:3060] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:22:25.249899 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 95895.4us
I20260709 13:22:25.255097 23685 leader_election.cc:290] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:22:25.292135 16293 raft_consensus.cc:2468] T 84f2da98864b4425bf7c0055951fdd4b P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 18.
I20260709 13:22:25.297636 16265 leader_election.cc:304] T 1df93779209748b9860f6d893dd3de53 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:22:25.211421 16223 raft_consensus.cc:2468] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 21.
I20260709 13:22:25.259748 23815 raft_consensus.cc:3055] T 9e65676b21b748728ad39668cfe722ec P 90f3da57796b488c9087a2f523adf045 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:22:25.305773 16190 leader_election.cc:304] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:22:25.148206 23965 raft_consensus.cc:2749] T ee9ed67006144797b1de6ea8c2013a68 P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:22:25.321058 16223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7ca61592e848ea9b0fb81a2ce33cdc" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
W20260709 13:22:25.307109 16192 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.710s	user 0.008s	sys 0.006s
W20260709 13:22:25.322165 16192 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.726s	user 0.009s	sys 0.006s
W20260709 13:22:25.322561 16192 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 774154us
I20260709 13:22:25.143364 23287 raft_consensus.cc:515] T cdbd78acb4cf46298e0de8ba0d130e19 P 90f3da57796b488c9087a2f523adf045 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:22:25.316087 16190 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:22:25.250725 23981 leader_election.cc:290] T 84f2da98864b4425bf7c0055951fdd4b P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:25.190928 23987 raft_consensus.cc:493] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:25.204731 23964 raft_consensus.cc:493] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:22:25.332378 16822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5d59407eb545b895d8d922e0dd69fd" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:25.333456 16818 raft_consensus.cc:1240] T ada5a004ba7c46819051e3ccec71ffdb P cb82cc7bd6db46d980c75237a1402860 [term 23 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 22. Current term is 23. Ops: []
I20260709 13:22:25.305594 23785 raft_consensus.cc:493] T 19baa7bc367844c288bca8f81439c413 P cb82cc7bd6db46d980c75237a1402860 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:22:25.334802 16824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b7d1ab519044eda2d5e04a07242f25" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:25.336899 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f2da98864b4425bf7c0055951fdd4b" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 19 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:25.318461 23938 raft_consensus.cc:2804] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:22:25.334075 16823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:22:25.346383 16293 raft_consensus.cc:2410] T 84f2da98864b4425bf7c0055951fdd4b P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac for term 19 because replica has last-logged OpId of term: 17 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 9.
W20260709 13:22:25.339066 16192 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:22:25.353031 16816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a378d7f796f94615b3b347119a9880fd" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:25.324605 23924 raft_consensus.cc:2804] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:22:25.313804 23815 raft_consensus.cc:740] T 9e65676b21b748728ad39668cfe722ec P 90f3da57796b488c9087a2f523adf045 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:22:25.294759 23357 raft_consensus.cc:515] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:25.356606 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f5f89186204242bf377f7c42b2e2da" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:25.358237 16224 raft_consensus.cc:2468] T c2f5f89186204242bf377f7c42b2e2da P 1ca2725d290e47d68b1a4b609c9816ac [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 24.
I20260709 13:22:25.357251 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5d59407eb545b895d8d922e0dd69fd" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:22:25.356631 16824 raft_consensus.cc:1240] T 66f75124215f4ca6bef831e174a792d8 P cb82cc7bd6db46d980c75237a1402860 [term 20 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 19. Current term is 20. Ops: []
I20260709 13:22:25.161895 23865 raft_consensus.cc:515] T 26354361e9b044cea1c2b86c4882fe92 P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:22:25.343928 23944 consensus_meta.cc:287] T 8158bbd17b994d2eb96f560bd0f1b88d P 647a343a542d4a139fe106c19095129b: Time spent flushing consensus metadata: real 0.622s	user 0.005s	sys 0.006s
I20260709 13:22:25.189877 23907 consensus_queue.cc:1059] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: INVALID_TERM, Last received: 22.17, Next index: 18, Last known committed idx: 16, Time since last communication: 0.166s
I20260709 13:22:25.363135 16818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8638ee363273426db1393dd81e5c7b72" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:25.314972 23936 consensus_queue.cc:1059] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: INVALID_TERM, Last received: 22.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.037s
I20260709 13:22:25.315703 23761 raft_consensus.cc:515] T a378d7f796f94615b3b347119a9880fd P 90f3da57796b488c9087a2f523adf045 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:25.368714 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18ad7e3af5504571a69d10c00750a01c" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:22:25.365954 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9944d5e4b8c4109af9955c4f08c9095" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:25.350221 23871 leader_election.cc:290] T 9c61a39fd4f84424b129c1a6f9ba404d P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:25.391765 16217 raft_consensus.cc:3055] T fffab26427e24d398e2695aad40ca134 P 1ca2725d290e47d68b1a4b609c9816ac [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:22:25.399480 16217 raft_consensus.cc:740] T fffab26427e24d398e2695aad40ca134 P 1ca2725d290e47d68b1a4b609c9816ac [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:22:25.358052 23924 raft_consensus.cc:493] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Starting leader election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:22:25.397198 16291 raft_consensus.cc:2468] T 26354361e9b044cea1c2b86c4882fe92 P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 18.
W20260709 13:22:25.383733 16117 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:22:25.396241 16220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e4bfb9090c8460eb14fa7e0d818e7f1" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:22:25.306649 23985 raft_consensus.cc:515] T b87147731a68400e8cce9b879daa7c5d P 647a343a542d4a139fe106c19095129b [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:25.359326 23921 raft_consensus.cc:493] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:25.400506 16217 consensus_queue.cc:260] T fffab26427e24d398e2695aad40ca134 P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.13, Last appended by leader: 13, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:25.312426 23978 raft_consensus.cc:515] T 746aabf856ab401c9ef5e859371e7b67 P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:22:25.395973 23928 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.259s	user 0.015s	sys 0.018s
W20260709 13:22:25.408007 23975 consensus_meta.cc:287] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b: Time spent flushing consensus metadata: real 0.564s	user 0.010s	sys 0.000s
I20260709 13:22:25.410283 16219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:25.361104 23831 consensus_queue.cc:1048] T e4dc6b0541ca431ebc757beac00521f1 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.042s
W20260709 13:22:25.411692 23983 log.cc:927] Time spent T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860: Append to log took a long time: real 0.117s	user 0.000s	sys 0.001s
W20260709 13:22:25.412379 23979 log.cc:927] Time spent T 9e65676b21b748728ad39668cfe722ec P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 0.173s	user 0.000s	sys 0.001s
I20260709 13:22:25.412249 16218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9944d5e4b8c4109af9955c4f08c9095" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:25.344226 23987 raft_consensus.cc:515] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:22:25.413225 23870 consensus_meta.cc:287] T 08c74490c40243a88e62d1760b5cd814 P 647a343a542d4a139fe106c19095129b: Time spent flushing consensus metadata: real 0.737s	user 0.011s	sys 0.000s
W20260709 13:22:25.414685 23959 consensus_meta.cc:287] T 57a744f5aaf944678a63469cbf514015 P 647a343a542d4a139fe106c19095129b: Time spent flushing consensus metadata: real 0.814s	user 0.010s	sys 0.003s
I20260709 13:22:25.418118 16221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e4bfb9090c8460eb14fa7e0d818e7f1" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:25.419116 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66f75124215f4ca6bef831e174a792d8" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:22:25.404250 23924 raft_consensus.cc:3060] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:22:25.448844 16825 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49906 (request call id 1488) took 1038 ms. Trace:
I20260709 13:22:25.449361 16825 rpcz_store.cc:276] 0709 13:22:24.410595 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:22:24.422640 (+ 12045us) service_pool.cc:224] Handling call
0709 13:22:24.699425 (+276785us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:22:24.703394 (+  3969us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:22:25.448634 (+745240us) spinlock_profiling.cc:243] Waited 276 ms on lock 0x7b54006239d0. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5e8ad 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:22:25.448784 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":635697536}
I20260709 13:22:25.421394 16219 raft_consensus.cc:2468] T ecc1e363230043fe9fef390f0b979f20 P 1ca2725d290e47d68b1a4b609c9816ac [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 24.
W20260709 13:22:25.420171 23928 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.318s	user 0.015s	sys 0.019s
I20260709 13:22:25.386848 23944 raft_consensus.cc:493] T 434bff3082d141838068f323bdb0d1ed P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:22:25.435369 16790 leader_election.cc:304] T 829b4ad8c5a54eababeea1181b960d4b P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:22:25.353533 23785 raft_consensus.cc:515] T 19baa7bc367844c288bca8f81439c413 P cb82cc7bd6db46d980c75237a1402860 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:25.431443 23975 raft_consensus.cc:2749] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:22:25.442137 23865 leader_election.cc:290] T 26354361e9b044cea1c2b86c4882fe92 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:25.443476 16220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ab0ac8347641e6955e9bce7d90c1a2" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
W20260709 13:22:25.454056 23818 consensus_meta.cc:287] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 1ca2725d290e47d68b1a4b609c9816ac: Time spent flushing consensus metadata: real 0.687s	user 0.010s	sys 0.002s
I20260709 13:22:25.457553 16218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada5a004ba7c46819051e3ccec71ffdb" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
W20260709 13:22:25.451692 23869 consensus_meta.cc:287] T e5ad4eb281564936a59f41e387a47156 P 647a343a542d4a139fe106c19095129b: Time spent flushing consensus metadata: real 0.672s	user 0.010s	sys 0.003s
W20260709 13:22:25.449282 23598 consensus_meta.cc:287] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac: Time spent flushing consensus metadata: real 0.695s	user 0.008s	sys 0.012s
I20260709 13:22:25.462441 16823 raft_consensus.cc:3060] T 19baa7bc367844c288bca8f81439c413 P cb82cc7bd6db46d980c75237a1402860 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:22:25.416541 23774 leader_election.cc:290] T ecc1e363230043fe9fef390f0b979f20 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:25.435104 23943 raft_consensus.cc:493] T 18ad7e3af5504571a69d10c00750a01c P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:22:25.419667 23907 consensus_queue.cc:1059] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: INVALID_TERM, Last received: 22.16, Next index: 17, Last known committed idx: 15, Time since last communication: 0.037s
I20260709 13:22:25.465517 23976 leader_election.cc:290] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:25.465862 16218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7ca61592e848ea9b0fb81a2ce33cdc" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:22:25.466981 23978 leader_election.cc:290] T 746aabf856ab401c9ef5e859371e7b67 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:25.467978 16220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2372981006d4b218215c28964dea623" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 26 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:25.468462 23357 leader_election.cc:290] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
W20260709 13:22:25.470000 23925 consensus_meta.cc:287] T d16dfad0c3b540e5b4b86492ddf87208 P 1ca2725d290e47d68b1a4b609c9816ac: Time spent flushing consensus metadata: real 0.601s	user 0.001s	sys 0.009s
I20260709 13:22:25.470448 23785 leader_election.cc:290] T 19baa7bc367844c288bca8f81439c413 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:25.462131 23694 consensus_queue.cc:1059] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: INVALID_TERM, Last received: 18.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.010s
I20260709 13:22:25.474691 23897 raft_consensus.cc:3055] T 8a878999c0024d2abe517484180290fc P 647a343a542d4a139fe106c19095129b [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:22:25.450707 23959 raft_consensus.cc:2749] T 57a744f5aaf944678a63469cbf514015 P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Leader pre-election lost for term 19. Reason: Vote denied by peer cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923) with higher term. Message: Invalid argument: T 57a744f5aaf944678a63469cbf514015 P cb82cc7bd6db46d980c75237a1402860 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b for earlier term 19. Current term is 20.
I20260709 13:22:25.358810 23964 raft_consensus.cc:515] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:25.504776 16820 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49892 (request call id 1156) took 2131 ms. Trace:
I20260709 13:22:25.454380 23993 raft_consensus.cc:2804] T 1df93779209748b9860f6d893dd3de53 P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:22:25.480535 16790 leader_election.cc:304] T af57996a8be84d398f0f023e8d08ef77 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:22:25.549331 16820 rpcz_store.cc:276] 0709 13:22:23.373804 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:22:23.375515 (+  1711us) service_pool.cc:224] Handling call
0709 13:22:23.389309 (+ 13794us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:22:23.886828 (+497519us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:22:25.504489 (+1617661us) spinlock_profiling.cc:243] Waited 489 ms on lock 0x7b54007201d4. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:22:25.504705 (+   216us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1124444416}
I20260709 13:22:25.550140 16220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:25.506616 16218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c11074579b046d99e585c2818e18a2c" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:25.420964 23921 raft_consensus.cc:515] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:25.516920 16223 raft_consensus.cc:2468] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 1ca2725d290e47d68b1a4b609c9816ac [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 22.
I20260709 13:22:25.448248 23870 raft_consensus.cc:515] T 08c74490c40243a88e62d1760b5cd814 P 647a343a542d4a139fe106c19095129b [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:25.464453 23605 leader_election.cc:290] T d16dfad0c3b540e5b4b86492ddf87208 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 22 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
W20260709 13:22:25.523998 16115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 317032us
I20260709 13:22:25.422773 23815 consensus_queue.cc:260] T 9e65676b21b748728ad39668cfe722ec P 90f3da57796b488c9087a2f523adf045 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.16, Last appended by leader: 16, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:22:25.542243 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 244650us
I20260709 13:22:25.460280 23944 raft_consensus.cc:515] T 434bff3082d141838068f323bdb0d1ed P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:25.505658 23869 raft_consensus.cc:2749] T e5ad4eb281564936a59f41e387a47156 P 647a343a542d4a139fe106c19095129b [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:22:25.554442 16115 leader_election.cc:304] T 23f82224f4564fea93572e8206bad1fe P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
W20260709 13:22:25.555555 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 75141.3us
I20260709 13:22:25.521181 23988 raft_consensus.cc:493] T 66ae176961b74813a58bc15c9c4c6f2a P 90f3da57796b488c9087a2f523adf045 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:22:25.556193 16220 raft_consensus.cc:2393] T ecc1e363230043fe9fef390f0b979f20 P 1ca2725d290e47d68b1a4b609c9816ac [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 in current term 24: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
I20260709 13:22:25.519009 23985 leader_election.cc:290] T b87147731a68400e8cce9b879daa7c5d P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:25.559249 16265 leader_election.cc:304] T 8cc959fb7f704fba9009c0595467d074 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:22:25.540426 23981 raft_consensus.cc:493] T e5ad4eb281564936a59f41e387a47156 P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:25.538579 23897 raft_consensus.cc:740] T 8a878999c0024d2abe517484180290fc P 647a343a542d4a139fe106c19095129b [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:22:25.515352 23598 raft_consensus.cc:515] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:25.506060 23943 raft_consensus.cc:515] T 18ad7e3af5504571a69d10c00750a01c P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } }
I20260709 13:22:25.564610 16265 leader_election.cc:304] T 8c11074579b046d99e585c2818e18a2c P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860
I20260709 13:22:25.554215 23993 raft_consensus.cc:493] T 1df93779209748b9860f6d893dd3de53 P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:22:25.527307 23925 raft_consensus.cc:515] T d16dfad0c3b540e5b4b86492ddf87208 P 1ca2725d290e47d68b1a4b609c9816ac [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:25.558840 23694 raft_consensus.cc:2749] T af57996a8be84d398f0f023e8d08ef77 P cb82cc7bd6db46d980c75237a1402860 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:22:25.568327 16265 leader_election.cc:304] T 8a878999c0024d2abe517484180290fc P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:22:25.572736 16265 leader_election.cc:304] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:22:25.564626 23815 raft_consensus.cc:3060] T 9e65676b21b748728ad39668cfe722ec P 90f3da57796b488c9087a2f523adf045 [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:22:25.576936 16192 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 183319us
I20260709 13:22:25.541500 23785 consensus_queue.cc:1059] T ee9ed67006144797b1de6ea8c2013a68 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: INVALID_TERM, Last received: 19.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.023s
I20260709 13:22:25.569969 23959 raft_consensus.cc:493] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 647a343a542d4a139fe106c19095129b [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:22:25.579854 16222 raft_consensus.cc:2393] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 in current term 24: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
I20260709 13:22:25.566049 23865 raft_consensus.cc:2804] T 8cc959fb7f704fba9009c0595467d074 P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:22:25.562582 23831 consensus_queue.cc:1048] T 68a6838d82ef4cac83507f906139d90f P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.014s
I20260709 13:22:25.565398 23605 raft_consensus.cc:2804] T 23f82224f4564fea93572e8206bad1fe P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:22:25.579785 23985 raft_consensus.cc:493] T fffab26427e24d398e2695aad40ca134 P 647a343a542d4a139fe106c19095129b [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:25.582491 16823 raft_consensus.cc:2468] T 19baa7bc367844c288bca8f81439c413 P cb82cc7bd6db46d980c75237a1402860 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 24.
I20260709 13:22:25.569584 23993 raft_consensus.cc:3060] T 1df93779209748b9860f6d893dd3de53 P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:22:25.598337 16292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c61a39fd4f84424b129c1a6f9ba404d" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:25.599429 16292 raft_consensus.cc:2393] T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 23: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
W20260709 13:22:25.599852 15046 thread.cc:527] Waited for 1000ms trying to join with rpc worker (tid 16132)
I20260709 13:22:25.601492 16295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:25.602152 16295 raft_consensus.cc:2468] T ecc1e363230043fe9fef390f0b979f20 P 90f3da57796b488c9087a2f523adf045 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 23.
I20260709 13:22:25.564939 23685 consensus_queue.cc:1048] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 15, Time since last communication: 0.020s
I20260709 13:22:25.606587 16299 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48684 (request call id 3353) took 3020 ms. Trace:
I20260709 13:22:25.607506 16299 rpcz_store.cc:276] 0709 13:22:22.586275 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:22:22.767937 (+181662us) service_pool.cc:224] Handling call
0709 13:22:22.768316 (+   379us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:22:23.832530 (+1064214us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:22:25.606345 (+1773815us) spinlock_profiling.cc:243] Waited 1.06 s on lock 0x7b540024c554. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:22:25.606530 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2443990272}
I20260709 13:22:25.566201 23981 raft_consensus.cc:515] T e5ad4eb281564936a59f41e387a47156 P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:25.609004 16115 leader_election.cc:304] T 84f2da98864b4425bf7c0055951fdd4b P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:22:25.565356 23988 raft_consensus.cc:515] T 66ae176961b74813a58bc15c9c4c6f2a P 90f3da57796b488c9087a2f523adf045 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:22:25.602684 23989 log.cc:927] Time spent T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.197s	user 0.001s	sys 0.000s
I20260709 13:22:25.589108 23996 raft_consensus.cc:2749] T 8c11074579b046d99e585c2818e18a2c P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:22:25.595652 23605 raft_consensus.cc:493] T 23f82224f4564fea93572e8206bad1fe P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:22:25.722172 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 158426us
I20260709 13:22:25.728775 16292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d16dfad0c3b540e5b4b86492ddf87208" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:22:25.740612 16292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87147731a68400e8cce9b879daa7c5d" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 23 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
W20260709 13:22:25.741448 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 155247us
I20260709 13:22:25.588742 23959 raft_consensus.cc:515] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 647a343a542d4a139fe106c19095129b [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:25.698143 23869 raft_consensus.cc:493] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:25.744372 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a9de347c711477e87c582d003f36925" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 26 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:25.697072 23774 raft_consensus.cc:493] T 3884f864a96d428785895e5463e5728f P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:25.721406 23605 raft_consensus.cc:3060] T 23f82224f4564fea93572e8206bad1fe P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:22:25.603698 23985 raft_consensus.cc:515] T fffab26427e24d398e2695aad40ca134 P 647a343a542d4a139fe106c19095129b [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:25.721335 23357 raft_consensus.cc:493] T c0eab25ce4c64e018a041beeda2867db P 647a343a542d4a139fe106c19095129b [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:22:25.733767 23951 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.294s	user 0.003s	sys 0.015s
I20260709 13:22:25.676046 24000 raft_consensus.cc:2804] T 8a878999c0024d2abe517484180290fc P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:22:25.722503 23997 raft_consensus.cc:2804] T 84f2da98864b4425bf7c0055951fdd4b P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Leader pre-election won for term 19
W20260709 13:22:25.766665 16192 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:22:25.833766 16265 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 27 similar messages]
W20260709 13:22:25.832943 23951 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.432s	user 0.003s	sys 0.016s
I20260709 13:22:25.755311 23987 leader_election.cc:290] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:25.676409 23994 consensus_queue.cc:1048] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.168s
I20260709 13:22:25.635871 23897 consensus_queue.cc:260] T 8a878999c0024d2abe517484180290fc P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:25.757164 23992 raft_consensus.cc:493] T 7c2095a5f4ad49408dbc7f69c606120d P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:25.827863 24003 raft_consensus.cc:2804] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:22:25.835860 23997 raft_consensus.cc:493] T 84f2da98864b4425bf7c0055951fdd4b P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:25.878772 23913 leader_election.cc:290] T b6f07842d3aa47858df5665874c86f3a P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:25.881052 24010 raft_consensus.cc:493] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:25.882021 16819 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49906 (request call id 1491) took 1441 ms. Trace:
I20260709 13:22:25.932827 16819 rpcz_store.cc:276] 0709 13:22:24.441032 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:22:24.441405 (+   373us) service_pool.cc:224] Handling call
0709 13:22:24.441709 (+   304us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:22:24.640053 (+198344us) raft_consensus.cc:1534] Early marking committed up to index 17
0709 13:22:24.921311 (+281258us) log.cc:844] Serialized 14 byte log entry
0709 13:22:25.353696 (+432385us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:22:25.353772 (+    76us) raft_consensus.cc:1645] Marking committed up to 17
0709 13:22:25.353841 (+    69us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:22:25.880721 (+526880us) spinlock_profiling.cc:243] Waited 2.07 ms on lock 0x7b180006d360. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b693ade1 00007f69b69475b1 00007f69ada402a6 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419
0709 13:22:25.881763 (+  1042us) spinlock_profiling.cc:243] Waited 172 ms on lock 0x7b54004b8cd4. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:22:25.881807 (+    44us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:22:25.881953 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":133142,"prepare.run_cpu_time_us":66,"prepare.run_wall_time_us":1620,"raft.queue_time_us":628,"raft.run_cpu_time_us":130,"raft.run_wall_time_us":6361,"spinlock_wait_cycles":400061440,"thread_start_us":711618,"threads_started":2,"wal-append.queue_time_us":635565}
I20260709 13:22:25.883021 23598 leader_election.cc:290] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 24 election: Requested vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:25.891688 24004 raft_consensus.cc:493] T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:22:25.894562 23287 leader_election.cc:290] T cdbd78acb4cf46298e0de8ba0d130e19 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 25 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:25.845911 24000 raft_consensus.cc:697] T 8a878999c0024d2abe517484180290fc P 90f3da57796b488c9087a2f523adf045 [term 19 LEADER]: Becoming Leader. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
W20260709 13:22:25.869910 23991 log.cc:927] Time spent T c9944d5e4b8c4109af9955c4f08c9095 P cb82cc7bd6db46d980c75237a1402860: Append to log took a long time: real 0.306s	user 0.000s	sys 0.001s
I20260709 13:22:25.947273 16292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18ad7e3af5504571a69d10c00750a01c" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:25.903683 24003 raft_consensus.cc:493] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:25.870913 23994 consensus_queue.cc:1243] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b [LEADER]: Peer 90f3da57796b488c9087a2f523adf045 log is divergent from this leader: its last log entry 23.13 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 12
I20260709 13:22:25.756982 23869 raft_consensus.cc:515] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:22:25.952633 16190 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:22:25.835788 23357 raft_consensus.cc:515] T c0eab25ce4c64e018a041beeda2867db P 647a343a542d4a139fe106c19095129b [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:25.829075 23774 raft_consensus.cc:515] T 3884f864a96d428785895e5463e5728f P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:25.868903 23865 raft_consensus.cc:697] T 8cc959fb7f704fba9009c0595467d074 P 90f3da57796b488c9087a2f523adf045 [term 20 LEADER]: Becoming Leader. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
W20260709 13:22:25.965121 16263 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:22:25.943722 23931 log.cc:927] Time spent T 7c2095a5f4ad49408dbc7f69c606120d P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 1.857s	user 0.009s	sys 0.009s
I20260709 13:22:25.920430 24009 raft_consensus.cc:3055] T ee9ed67006144797b1de6ea8c2013a68 P cb82cc7bd6db46d980c75237a1402860 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:22:25.970523 16292 raft_consensus.cc:2393] T 18ad7e3af5504571a69d10c00750a01c P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 18: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
I20260709 13:22:25.862051 23951 raft_consensus.cc:697] T 57a744f5aaf944678a63469cbf514015 P cb82cc7bd6db46d980c75237a1402860 [term 20 LEADER]: Becoming Leader. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:22:25.931126 23997 raft_consensus.cc:3060] T 84f2da98864b4425bf7c0055951fdd4b P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:22:25.971011 23287 raft_consensus.cc:493] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 90f3da57796b488c9087a2f523adf045 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:22:25.979095 24003 raft_consensus.cc:3060] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045 [term 24 FOLLOWER]: Advancing to term 25
W20260709 13:22:26.038076 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 283341us
I20260709 13:22:25.990834 23925 leader_election.cc:290] T d16dfad0c3b540e5b4b86492ddf87208 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 22 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
W20260709 13:22:26.044591 16265 leader_election.cc:336] T 84f2da98864b4425bf7c0055951fdd4b P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 18 election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:22:26.047191 23601 consensus_meta.cc:287] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 1ca2725d290e47d68b1a4b609c9816ac: Time spent flushing consensus metadata: real 0.733s	user 0.012s	sys 0.000s
I20260709 13:22:26.047353 24012 raft_consensus.cc:493] T 3884f864a96d428785895e5463e5728f P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
W20260709 13:22:26.049363 24011 log.cc:927] Time spent T 829b4ad8c5a54eababeea1181b960d4b P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
I20260709 13:22:26.067819 24000 consensus_queue.cc:237] T 8a878999c0024d2abe517484180290fc P 90f3da57796b488c9087a2f523adf045 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:26.066545 23997 raft_consensus.cc:515] T 84f2da98864b4425bf7c0055951fdd4b P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:26.069836 23943 leader_election.cc:290] T 18ad7e3af5504571a69d10c00750a01c P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:22:26.109733 16816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9944d5e4b8c4109af9955c4f08c9095" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:25.950942 24010 raft_consensus.cc:515] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:26.058808 23638 raft_consensus.cc:3055] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:22:26.071312 23985 leader_election.cc:290] T fffab26427e24d398e2695aad40ca134 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:25.921293 23992 raft_consensus.cc:515] T 7c2095a5f4ad49408dbc7f69c606120d P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:26.117240 23992 leader_election.cc:290] T 7c2095a5f4ad49408dbc7f69c606120d P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:25.993973 23870 leader_election.cc:290] T 08c74490c40243a88e62d1760b5cd814 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 21 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:26.072366 23981 leader_election.cc:290] T e5ad4eb281564936a59f41e387a47156 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:25.996485 24009 raft_consensus.cc:740] T ee9ed67006144797b1de6ea8c2013a68 P cb82cc7bd6db46d980c75237a1402860 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:22:26.093029 23818 raft_consensus.cc:493] T 68a6838d82ef4cac83507f906139d90f P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:26.099738 23865 consensus_queue.cc:237] T 8cc959fb7f704fba9009c0595467d074 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:26.104055 16292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a9de347c711477e87c582d003f36925" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 26 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:25.964385 24004 raft_consensus.cc:515] T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:26.125448 16292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434bff3082d141838068f323bdb0d1ed" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:26.106590 16825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f82224f4564fea93572e8206bad1fe" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:22:26.107509 16822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e4bfb9090c8460eb14fa7e0d818e7f1" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:22:26.112713 16816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26354361e9b044cea1c2b86c4882fe92" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:26.126714 16825 raft_consensus.cc:2368] T 23f82224f4564fea93572e8206bad1fe P cb82cc7bd6db46d980c75237a1402860 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 for earlier term 18. Current term is 19.
I20260709 13:22:26.126763 16822 raft_consensus.cc:3060] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P cb82cc7bd6db46d980c75237a1402860 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:22:26.113535 16820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:26.129554 16820 raft_consensus.cc:2393] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 24: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
I20260709 13:22:26.114635 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ab0ac8347641e6955e9bce7d90c1a2" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:26.115200 23638 raft_consensus.cc:740] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:22:26.074618 16265 leader_election.cc:304] T 84f2da98864b4425bf7c0055951fdd4b P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:22:26.130925 16820 raft_consensus.cc:1240] T 9e65676b21b748728ad39668cfe722ec P cb82cc7bd6db46d980c75237a1402860 [term 22 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 21. Current term is 22. Ops: [19.15-21.16]
I20260709 13:22:26.131603 23024 raft_consensus.cc:2749] T 84f2da98864b4425bf7c0055951fdd4b P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
W20260709 13:22:26.131722 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 87165.9us
I20260709 13:22:26.132387 16820 raft_consensus.cc:1275] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860 [term 25 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 23 index: 16. Preceding OpId from leader: term: 25 index: 17. (index mismatch)
I20260709 13:22:26.114341 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e4bfb9090c8460eb14fa7e0d818e7f1" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:26.126819 16816 raft_consensus.cc:2468] T 26354361e9b044cea1c2b86c4882fe92 P cb82cc7bd6db46d980c75237a1402860 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 18.
I20260709 13:22:26.134042 16297 raft_consensus.cc:2393] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 in current term 23: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
I20260709 13:22:26.073164 23921 leader_election.cc:290] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:26.135046 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:26.135785 16297 raft_consensus.cc:2468] T ecc1e363230043fe9fef390f0b979f20 P 90f3da57796b488c9087a2f523adf045 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 23.
I20260709 13:22:26.136014 23638 consensus_queue.cc:260] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 18.15, Last appended by leader: 15, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:26.137292 23638 raft_consensus.cc:3060] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:22:26.137718 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9944d5e4b8c4109af9955c4f08c9095" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:26.138439 16297 raft_consensus.cc:2468] T c9944d5e4b8c4109af9955c4f08c9095 P 90f3da57796b488c9087a2f523adf045 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 23.
I20260709 13:22:26.130944 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:26.149734 16225 raft_consensus.cc:2468] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 1ca2725d290e47d68b1a4b609c9816ac [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 20.
I20260709 13:22:26.139365 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f82224f4564fea93572e8206bad1fe" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:22:26.151798 16297 raft_consensus.cc:3060] T 23f82224f4564fea93572e8206bad1fe P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:22:26.154227 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 183574us
I20260709 13:22:26.075409 23287 raft_consensus.cc:515] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 90f3da57796b488c9087a2f523adf045 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:26.121971 23959 leader_election.cc:290] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:26.121198 23988 leader_election.cc:290] T 66ae176961b74813a58bc15c9c4c6f2a P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
W20260709 13:22:26.187903 16790 consensus_peers.cc:597] T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:22:26.192873 16297 raft_consensus.cc:2468] T 23f82224f4564fea93572e8206bad1fe P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 19.
I20260709 13:22:26.154779 23941 raft_consensus.cc:493] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 1ca2725d290e47d68b1a4b609c9816ac [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:26.220142 16822 raft_consensus.cc:2468] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P cb82cc7bd6db46d980c75237a1402860 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 23.
I20260709 13:22:26.153643 23818 raft_consensus.cc:515] T 68a6838d82ef4cac83507f906139d90f P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:26.228287 23818 leader_election.cc:290] T 68a6838d82ef4cac83507f906139d90f P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:26.179710 23951 consensus_queue.cc:237] T 57a744f5aaf944678a63469cbf514015 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.14, Last appended by leader: 14, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:26.225107 23941 raft_consensus.cc:515] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 1ca2725d290e47d68b1a4b609c9816ac [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:26.236832 23915 raft_consensus.cc:3060] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [term 26 FOLLOWER]: Advancing to term 27
I20260709 13:22:26.237867 23774 leader_election.cc:290] T 3884f864a96d428785895e5463e5728f P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
W20260709 13:22:26.240106 16789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 121731us
I20260709 13:22:26.243130 16824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f82224f4564fea93572e8206bad1fe" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:26.243891 16824 raft_consensus.cc:2393] T 23f82224f4564fea93572e8206bad1fe P cb82cc7bd6db46d980c75237a1402860 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 19: Already voted for candidate cb82cc7bd6db46d980c75237a1402860 in this term.
W20260709 13:22:26.243765 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 48087.7us
W20260709 13:22:26.236747 24014 log.cc:927] Time spent T 5a96373bb0e14dce8a266b2fc6c58507 P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.164s	user 0.001s	sys 0.000s
I20260709 13:22:26.105304 24012 raft_consensus.cc:515] T 3884f864a96d428785895e5463e5728f P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:22:26.247758 16117 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 348631us
W20260709 13:22:26.248921 16117 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:22:26.253268 16298 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48674 (request call id 3631) took 1124 ms. Trace:
I20260709 13:22:26.254271 23778 raft_consensus.cc:3055] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 [term 24 LEADER]: Stepping down as leader of term 24
W20260709 13:22:26.249640 16117 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:22:26.257128 16822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26354361e9b044cea1c2b86c4882fe92" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:26.253682 16298 rpcz_store.cc:276] 0709 13:22:25.128490 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:22:25.128630 (+   140us) service_pool.cc:224] Handling call
0709 13:22:25.128945 (+   315us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:22:25.524671 (+395726us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:22:25.524718 (+    47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:22:25.565894 (+ 41176us) log.cc:844] Serialized 26 byte log entry
0709 13:22:25.857277 (+291383us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:22:25.857324 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:22:25.857402 (+    78us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:22:26.253028 (+395626us) raft_consensus.cc:1685] finished
0709 13:22:26.253050 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:22:26.253204 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":112117,"raft.run_cpu_time_us":88,"raft.run_wall_time_us":87,"thread_start_us":634384,"threads_started":2,"wal-append.queue_time_us":404997}
I20260709 13:22:26.258517 16822 raft_consensus.cc:2393] T 26354361e9b044cea1c2b86c4882fe92 P cb82cc7bd6db46d980c75237a1402860 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 18: Already voted for candidate cb82cc7bd6db46d980c75237a1402860 in this term.
I20260709 13:22:26.262244 16816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62fe3346cd3c42c4a55e14af9073bca4" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 27 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:26.283674 16822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a9de347c711477e87c582d003f36925" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 26 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:26.284547 16822 raft_consensus.cc:2468] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 25.
I20260709 13:22:26.222596 23785 consensus_queue.cc:1048] T 2794af5b99984efebdee564035841cb0 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 13, Time since last communication: 0.039s
I20260709 13:22:26.256980 23778 raft_consensus.cc:740] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:22:26.289980 16816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:26.246433 24009 consensus_queue.cc:260] T ee9ed67006144797b1de6ea8c2013a68 P cb82cc7bd6db46d980c75237a1402860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:22:26.293030 23815 consensus_meta.cc:287] T 9e65676b21b748728ad39668cfe722ec P 90f3da57796b488c9087a2f523adf045: Time spent flushing consensus metadata: real 0.710s	user 0.005s	sys 0.007s
I20260709 13:22:26.293299 24009 raft_consensus.cc:3060] T ee9ed67006144797b1de6ea8c2013a68 P cb82cc7bd6db46d980c75237a1402860 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:22:26.290953 16816 raft_consensus.cc:2468] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 24.
I20260709 13:22:26.294428 16296 raft_consensus.cc:1275] T 9e65676b21b748728ad39668cfe722ec P 90f3da57796b488c9087a2f523adf045 [term 22 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 21 index: 16. Preceding OpId from leader: term: 22 index: 16. (term mismatch)
I20260709 13:22:26.294845 16296 pending_rounds.cc:85] T 9e65676b21b748728ad39668cfe722ec P 90f3da57796b488c9087a2f523adf045: Aborting all ops after (but not including) 15
I20260709 13:22:26.295053 16296 pending_rounds.cc:107] T 9e65676b21b748728ad39668cfe722ec P 90f3da57796b488c9087a2f523adf045: Aborting uncommitted NO_OP operation due to leader change: 21.16
I20260709 13:22:26.295289 16296 raft_consensus.cc:2889] T 9e65676b21b748728ad39668cfe722ec P 90f3da57796b488c9087a2f523adf045 [term 22 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:22:26.318614 16822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746aabf856ab401c9ef5e859371e7b67" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:26.322227 16816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5718c623ebee4d2bb770f0f9ddb2f164" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 23 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
W20260709 13:22:26.323127 23993 consensus_meta.cc:287] T 1df93779209748b9860f6d893dd3de53 P 90f3da57796b488c9087a2f523adf045: Time spent flushing consensus metadata: real 0.679s	user 0.005s	sys 0.006s
I20260709 13:22:26.336014 16822 raft_consensus.cc:2468] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 19.
I20260709 13:22:26.334959 23287 leader_election.cc:290] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
W20260709 13:22:26.342913 23605 consensus_meta.cc:287] T 23f82224f4564fea93572e8206bad1fe P 647a343a542d4a139fe106c19095129b: Time spent flushing consensus metadata: real 0.511s	user 0.009s	sys 0.000s
I20260709 13:22:26.343509 16816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5d59407eb545b895d8d922e0dd69fd" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:26.344257 23942 raft_consensus.cc:3060] T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:22:26.349081 16821 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49906 (request call id 1498) took 1012 ms. Trace:
I20260709 13:22:26.349877 23936 raft_consensus.cc:2749] T 829b4ad8c5a54eababeea1181b960d4b P cb82cc7bd6db46d980c75237a1402860 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:22:26.336886 23928 raft_consensus.cc:3055] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:22:26.354450 16816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87147731a68400e8cce9b879daa7c5d" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 23 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:26.352942 16821 rpcz_store.cc:276] 0709 13:22:25.336726 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:22:25.336891 (+   165us) service_pool.cc:224] Handling call
0709 13:22:25.337137 (+   246us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:22:25.337332 (+   195us) raft_consensus.cc:1534] Early marking committed up to index 15
0709 13:22:25.354002 (+ 16670us) log.cc:844] Serialized 14 byte log entry
0709 13:22:26.346931 (+992929us) spinlock_profiling.cc:243] Waited 18.1 ms on lock 0x7b1800176e60. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b693ade1 00007f69b69475b1 00007f69b07074bc 00007f69ada5548b 00007f69b68afab9 00007f69ada7169b 00007f69ada6f93f 00007f69ada6f455 00007f69b0b57060 00007f69b0b69716 00007f69b0b7a287
0709 13:22:26.347056 (+   125us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:22:26.347115 (+    59us) raft_consensus.cc:1645] Marking committed up to 15
0709 13:22:26.348860 (+  1745us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:22:26.348933 (+    73us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:22:26.349030 (+    97us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":30064,"prepare.queue_time_us":19154,"prepare.run_cpu_time_us":48,"prepare.run_wall_time_us":50,"raft.queue_time_us":166429,"spinlock_wait_cycles":116295808,"thread_start_us":617684,"threads_started":2,"wal-append.queue_time_us":646032}
I20260709 13:22:26.357954 16117 leader_election.cc:304] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860; no voters: 90f3da57796b488c9087a2f523adf045
I20260709 13:22:26.359588 16821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3884f864a96d428785895e5463e5728f" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 18 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:26.359934 16117 leader_election.cc:304] T ecc1e363230043fe9fef390f0b979f20 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:22:26.358706 16825 raft_consensus.cc:1275] T 9e65676b21b748728ad39668cfe722ec P cb82cc7bd6db46d980c75237a1402860 [term 22 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 22 index: 16. (index mismatch)
I20260709 13:22:26.339658 23869 leader_election.cc:290] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:26.372114 16823 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49906 (request call id 1495) took 1039 ms. Trace:
W20260709 13:22:26.365621 16790 leader_election.cc:336] T 66f75124215f4ca6bef831e174a792d8 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 20 election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:22:26.366039 24028 log.cc:927] Time spent T 02838c86531d4f98bf58d1c22b8baf77 P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.062s	user 0.000s	sys 0.000s
I20260709 13:22:26.366145 23894 leader_election.cc:290] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:26.375907 23964 leader_election.cc:290] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:26.372536 16823 rpcz_store.cc:276] 0709 13:22:25.332579 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:22:25.333854 (+  1275us) service_pool.cc:224] Handling call
0709 13:22:26.371920 (+1038066us) spinlock_profiling.cc:243] Waited 111 ms on lock 0x7b54006eebd4. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:22:26.372049 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":255232896}
I20260709 13:22:26.366827 16789 leader_election.cc:304] T c9944d5e4b8c4109af9955c4f08c9095 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 
W20260709 13:22:26.369130 16190 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:22:26.365288 23719 raft_consensus.cc:2804] T ecc1e363230043fe9fef390f0b979f20 P 647a343a542d4a139fe106c19095129b [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:22:26.372917 23978 raft_consensus.cc:2804] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 13:22:26.374018 24004 leader_election.cc:290] T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:26.339769 23993 raft_consensus.cc:515] T 1df93779209748b9860f6d893dd3de53 P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:26.387984 16823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e9b2bf7011f4ae688a79d7bb5a447b4" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:26.388706 16823 raft_consensus.cc:2468] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 20.
I20260709 13:22:26.373183 16820 raft_consensus.cc:1240] T 19baa7bc367844c288bca8f81439c413 P cb82cc7bd6db46d980c75237a1402860 [term 24 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 23. Current term is 24. Ops: []
I20260709 13:22:26.358783 23928 raft_consensus.cc:740] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:22:26.386907 23719 raft_consensus.cc:493] T ecc1e363230043fe9fef390f0b979f20 P 647a343a542d4a139fe106c19095129b [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:22:26.387256 16190 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:22:26.392820 16823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2372981006d4b218215c28964dea623" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 26 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:22:26.390777 16117 leader_election.cc:304] T b87147731a68400e8cce9b879daa7c5d P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860
I20260709 13:22:26.355290 23605 raft_consensus.cc:515] T 23f82224f4564fea93572e8206bad1fe P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:26.394097 16823 raft_consensus.cc:2410] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b for term 26 because replica has last-logged OpId of term: 25 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 23 index: 15.
I20260709 13:22:26.339694 23778 consensus_queue.cc:260] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 24.18, Last appended by leader: 18, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:26.382558 23941 leader_election.cc:290] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:26.395937 16117 leader_election.cc:304] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:22:26.394819 24004 raft_consensus.cc:2749] T b87147731a68400e8cce9b879daa7c5d P 647a343a542d4a139fe106c19095129b [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:22:26.398628 23778 raft_consensus.cc:3060] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:22:26.398797 23964 raft_consensus.cc:2804] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:22:26.387566 23978 raft_consensus.cc:493] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:26.399349 23964 raft_consensus.cc:493] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:26.399507 23978 raft_consensus.cc:3060] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:22:26.399765 23964 raft_consensus.cc:3060] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:22:26.401129 23869 consensus_queue.cc:1048] T 9e65676b21b748728ad39668cfe722ec P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.004s
I20260709 13:22:26.403235 23958 raft_consensus.cc:3055] T 829b4ad8c5a54eababeea1181b960d4b P 1ca2725d290e47d68b1a4b609c9816ac [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:22:26.403705 23958 raft_consensus.cc:740] T 829b4ad8c5a54eababeea1181b960d4b P 1ca2725d290e47d68b1a4b609c9816ac [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:22:26.404583 23958 consensus_queue.cc:260] T 829b4ad8c5a54eababeea1181b960d4b P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:22:26.405455 23924 consensus_meta.cc:287] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac: Time spent flushing consensus metadata: real 0.947s	user 0.007s	sys 0.005s
I20260709 13:22:26.405680 23958 raft_consensus.cc:3060] T 829b4ad8c5a54eababeea1181b960d4b P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:22:26.407500 16823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b7d1ab519044eda2d5e04a07242f25" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:26.408851 16117 leader_election.cc:304] T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 26 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860
I20260709 13:22:26.410318 16823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:26.395002 23719 raft_consensus.cc:3060] T ecc1e363230043fe9fef390f0b979f20 P 647a343a542d4a139fe106c19095129b [term 24 FOLLOWER]: Advancing to term 25
W20260709 13:22:26.415143 16266 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:22:26.417826 16266 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:22:26.418228 16266 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:22:26.411938 16823 raft_consensus.cc:2393] T 19baa7bc367844c288bca8f81439c413 P cb82cc7bd6db46d980c75237a1402860 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 24: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
I20260709 13:22:26.405908 23924 raft_consensus.cc:515] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:22:26.418543 16790 leader_election.cc:336] T 57a744f5aaf944678a63469cbf514015 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 20 election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:22:26.394243 23785 raft_consensus.cc:2804] T c9944d5e4b8c4109af9955c4f08c9095 P cb82cc7bd6db46d980c75237a1402860 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:22:26.419634 16226 raft_consensus.cc:2468] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 27.
I20260709 13:22:26.430308 23924 leader_election.cc:290] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 22 election: Requested vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:26.423588 16823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdbd78acb4cf46298e0de8ba0d130e19" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 25 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:22:26.422017 23941 consensus_queue.cc:1059] T 18ad7e3af5504571a69d10c00750a01c P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: INVALID_TERM, Last received: 17.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.014s
W20260709 13:22:26.440639 16790 consensus_peers.cc:597] T 2794af5b99984efebdee564035841cb0 P cb82cc7bd6db46d980c75237a1402860 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:22:26.443482 16821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7ca61592e848ea9b0fb81a2ce33cdc" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:26.444283 16821 raft_consensus.cc:2393] T 3a7ca61592e848ea9b0fb81a2ce33cdc P cb82cc7bd6db46d980c75237a1402860 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 22: Already voted for candidate cb82cc7bd6db46d980c75237a1402860 in this term.
I20260709 13:22:26.445289 16821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66ae176961b74813a58bc15c9c4c6f2a" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 27 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
W20260709 13:22:26.442510 16790 consensus_peers.cc:597] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:22:26.449812 16790 leader_election.cc:336] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:22:26.448922 16821 raft_consensus.cc:2393] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 27: Already voted for candidate cb82cc7bd6db46d980c75237a1402860 in this term.
I20260709 13:22:26.453541 24012 leader_election.cc:290] T 3884f864a96d428785895e5463e5728f P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:26.454834 16790 leader_election.cc:304] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:22:26.456482 23638 raft_consensus.cc:2749] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
W20260709 13:22:26.458069 16790 leader_election.cc:336] T b87147731a68400e8cce9b879daa7c5d P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:22:26.459276 16790 leader_election.cc:304] T b87147731a68400e8cce9b879daa7c5d P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:22:26.462234 23638 raft_consensus.cc:2749] T b87147731a68400e8cce9b879daa7c5d P cb82cc7bd6db46d980c75237a1402860 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 13:22:26.462985 16791 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:22:26.464247 16790 leader_election.cc:336] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:22:26.460592 23944 leader_election.cc:290] T 434bff3082d141838068f323bdb0d1ed P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:26.450109 24003 raft_consensus.cc:515] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:22:26.466166 16790 leader_election.cc:336] T 23f82224f4564fea93572e8206bad1fe P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 19 election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:22:26.421258 23869 raft_consensus.cc:2749] T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b [term 26 FOLLOWER]: Leader election lost for term 26. Reason: could not achieve majority
W20260709 13:22:26.471360 16790 leader_election.cc:336] T 434bff3082d141838068f323bdb0d1ed P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:22:26.396504 23928 consensus_queue.cc:260] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 22.16, Last appended by leader: 16, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:22:26.472343 16790 leader_election.cc:336] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:22:26.479065 16790 leader_election.cc:336] T 19baa7bc367844c288bca8f81439c413 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:22:26.441460 23785 raft_consensus.cc:493] T c9944d5e4b8c4109af9955c4f08c9095 P cb82cc7bd6db46d980c75237a1402860 [term 23 FOLLOWER]: Starting leader election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:22:26.483852 23785 raft_consensus.cc:3060] T c9944d5e4b8c4109af9955c4f08c9095 P cb82cc7bd6db46d980c75237a1402860 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:22:26.493678 23993 leader_election.cc:290] T 1df93779209748b9860f6d893dd3de53 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 20 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:22:26.494691 23928 raft_consensus.cc:3060] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:22:26.498564 23947 raft_consensus.cc:3055] T c0eab25ce4c64e018a041beeda2867db P 1ca2725d290e47d68b1a4b609c9816ac [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:22:26.499003 23947 raft_consensus.cc:740] T c0eab25ce4c64e018a041beeda2867db P 1ca2725d290e47d68b1a4b609c9816ac [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
W20260709 13:22:26.499534 16790 leader_election.cc:336] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:22:26.499895 23947 consensus_queue.cc:260] T c0eab25ce4c64e018a041beeda2867db P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 22.12, Last appended by leader: 12, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:26.501139 23947 raft_consensus.cc:3060] T c0eab25ce4c64e018a041beeda2867db P 1ca2725d290e47d68b1a4b609c9816ac [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:22:26.508656 16222 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33050 (request call id 1344) took 1579 ms. Trace:
I20260709 13:22:26.509254 16222 rpcz_store.cc:276] 0709 13:22:24.928895 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:22:24.936265 (+  7370us) service_pool.cc:224] Handling call
0709 13:22:26.508393 (+1572128us) spinlock_profiling.cc:243] Waited 642 ms on lock 0x7b5400245c54. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:22:26.508546 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1476604160}
I20260709 13:22:26.512022 23980 raft_consensus.cc:493] T 8cc959fb7f704fba9009c0595467d074 P 1ca2725d290e47d68b1a4b609c9816ac [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:26.512604 23980 raft_consensus.cc:515] T 8cc959fb7f704fba9009c0595467d074 P 1ca2725d290e47d68b1a4b609c9816ac [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:26.514562 23800 raft_consensus.cc:493] T 5a959dbecc2648eb85d1cc550f5a3245 P 1ca2725d290e47d68b1a4b609c9816ac [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:22:26.515326 23945 raft_consensus.cc:493] T 5a96373bb0e14dce8a266b2fc6c58507 P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:26.524787 16222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4dc6b0541ca431ebc757beac00521f1" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 23 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:26.525301 23945 raft_consensus.cc:515] T 5a96373bb0e14dce8a266b2fc6c58507 P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:26.525804 16816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f82224f4564fea93572e8206bad1fe" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:22:26.526048 16218 raft_consensus.cc:1240] T 8cc959fb7f704fba9009c0595467d074 P 1ca2725d290e47d68b1a4b609c9816ac [term 20 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 19. Current term is 20. Ops: []
I20260709 13:22:26.526577 16816 raft_consensus.cc:2393] T 23f82224f4564fea93572e8206bad1fe P cb82cc7bd6db46d980c75237a1402860 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 19: Already voted for candidate cb82cc7bd6db46d980c75237a1402860 in this term.
I20260709 13:22:26.527050 16220 raft_consensus.cc:1275] T 5a96373bb0e14dce8a266b2fc6c58507 P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 13:22:26.515126 23800 raft_consensus.cc:515] T 5a959dbecc2648eb85d1cc550f5a3245 P 1ca2725d290e47d68b1a4b609c9816ac [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:26.528367 16218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62fe3346cd3c42c4a55e14af9073bca4" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 27 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:26.528575 16820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434bff3082d141838068f323bdb0d1ed" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:26.529066 16218 raft_consensus.cc:2468] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 26.
I20260709 13:22:26.530107 23945 leader_election.cc:290] T 5a96373bb0e14dce8a266b2fc6c58507 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:26.520067 23605 leader_election.cc:290] T 23f82224f4564fea93572e8206bad1fe P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 19 election: Requested vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:26.521668 16219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f2da98864b4425bf7c0055951fdd4b" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:26.532311 16219 raft_consensus.cc:2468] T 84f2da98864b4425bf7c0055951fdd4b P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 18.
I20260709 13:22:26.515981 16221 raft_consensus.cc:1275] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 20 index: 14. Preceding OpId from leader: term: 21 index: 14. (term mismatch)
I20260709 13:22:26.532822 16221 pending_rounds.cc:85] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac: Aborting all ops after (but not including) 13
I20260709 13:22:26.533043 16221 pending_rounds.cc:107] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac: Aborting uncommitted CHANGE_CONFIG_OP operation due to leader change: 20.14
I20260709 13:22:26.533308 16221 raft_consensus.cc:2917] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Aborting config change with OpId 20.14: Aborted: Op aborted by new leader
I20260709 13:22:26.522555 16223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68a6838d82ef4cac83507f906139d90f" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:26.533933 16221 raft_consensus.cc:2889] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: CHANGE_CONFIG_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:22:26.534287 16223 raft_consensus.cc:2468] T 68a6838d82ef4cac83507f906139d90f P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 21.
I20260709 13:22:26.522640 23938 raft_consensus.cc:493] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Starting leader election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:22:26.514966 23980 leader_election.cc:290] T 8cc959fb7f704fba9009c0595467d074 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:26.535207 16223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d16dfad0c3b540e5b4b86492ddf87208" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:22:26.535410 23938 raft_consensus.cc:3060] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:22:26.521057 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a959dbecc2648eb85d1cc550f5a3245" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:26.536705 16224 raft_consensus.cc:2468] T 5a959dbecc2648eb85d1cc550f5a3245 P 1ca2725d290e47d68b1a4b609c9816ac [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 20.
I20260709 13:22:26.523554 16217 raft_consensus.cc:3060] T fffab26427e24d398e2695aad40ca134 P 1ca2725d290e47d68b1a4b609c9816ac [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:22:26.537467 16816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74ca8140f35c4ae29e8a664f6bdfc829" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 24 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:22:26.537551 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5718c623ebee4d2bb770f0f9ddb2f164" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 23 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:26.538205 16224 raft_consensus.cc:2468] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 22.
I20260709 13:22:26.533231 16219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5d59407eb545b895d8d922e0dd69fd" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:26.536011 16221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:26.539655 16218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c61a39fd4f84424b129c1a6f9ba404d" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:26.540079 16221 raft_consensus.cc:2468] T ecc1e363230043fe9fef390f0b979f20 P 1ca2725d290e47d68b1a4b609c9816ac [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 24.
I20260709 13:22:26.536020 16223 raft_consensus.cc:2393] T d16dfad0c3b540e5b4b86492ddf87208 P 1ca2725d290e47d68b1a4b609c9816ac [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 22: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
I20260709 13:22:26.540414 16218 raft_consensus.cc:2468] T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 23.
I20260709 13:22:26.536046 23997 leader_election.cc:290] T 84f2da98864b4425bf7c0055951fdd4b P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 19 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:26.541551 16222 raft_consensus.cc:2468] T e4dc6b0541ca431ebc757beac00521f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 22.
W20260709 13:22:26.543076 16291 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48674 (request call id 3634) took 1393 ms (client timeout 1652 ms). Trace:
I20260709 13:22:26.543731 16218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c74490c40243a88e62d1760b5cd814" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
W20260709 13:22:26.543543 16291 rpcz_store.cc:269] 0709 13:22:25.149527 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:22:25.150022 (+   495us) service_pool.cc:224] Handling call
0709 13:22:26.542923 (+1392901us) spinlock_profiling.cc:243] Waited 206 ms on lock 0x7b54000cf654. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:22:26.543020 (+    97us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":473665280}
I20260709 13:22:26.544225 16222 raft_consensus.cc:1275] T c2f5f89186204242bf377f7c42b2e2da P 1ca2725d290e47d68b1a4b609c9816ac [term 24 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 20 index: 12. Preceding OpId from leader: term: 24 index: 14. (index mismatch)
W20260709 13:22:26.523792 16225 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57600 (request call id 2245) took 2570 ms (client timeout 1612 ms). Trace:
W20260709 13:22:26.545188 16225 rpcz_store.cc:269] 0709 13:22:23.953571 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:22:23.953755 (+   184us) service_pool.cc:224] Handling call
0709 13:22:26.523591 (+2569836us) spinlock_profiling.cc:243] Waited 2.11 s on lock 0x7b54002b5054. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:22:26.523694 (+   103us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4850642816}
I20260709 13:22:26.549021 23800 leader_election.cc:290] T 5a959dbecc2648eb85d1cc550f5a3245 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:26.539124 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746aabf856ab401c9ef5e859371e7b67" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:26.551872 16224 raft_consensus.cc:2393] T 746aabf856ab401c9ef5e859371e7b67 P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 19: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
I20260709 13:22:26.552628 16823 raft_consensus.cc:2468] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 25.
W20260709 13:22:26.531589 24034 log.cc:927] Time spent T 57a744f5aaf944678a63469cbf514015 P cb82cc7bd6db46d980c75237a1402860: Append to log took a long time: real 0.150s	user 0.001s	sys 0.000s
I20260709 13:22:26.569692 16217 raft_consensus.cc:1275] T fffab26427e24d398e2695aad40ca134 P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 20 index: 13. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
I20260709 13:22:26.570658 16217 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57600 (request call id 2268) took 1180 ms. Trace:
I20260709 13:22:26.571074 16217 rpcz_store.cc:276] 0709 13:22:25.390627 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:22:25.391246 (+   619us) service_pool.cc:224] Handling call
0709 13:22:25.391613 (+   367us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:22:26.515023 (+1123410us) spinlock_profiling.cc:243] Waited 462 ms on lock 0x7b5800887fd0. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0abb0aa 00007f69b0b3d447 00007f69b0b55fe7 00007f69b0b577de 00007f69b0b61b49 00007f69b0b62416 00007f69b0b5ed8f 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4
0709 13:22:26.570297 (+ 55274us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:22:26.570588 (+   291us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1062854912}
I20260709 13:22:26.556758 23719 raft_consensus.cc:515] T ecc1e363230043fe9fef390f0b979f20 P 647a343a542d4a139fe106c19095129b [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:22:26.573446 23384 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 21 current term: 21 is higher.
I20260709 13:22:26.551049 23785 raft_consensus.cc:515] T c9944d5e4b8c4109af9955c4f08c9095 P cb82cc7bd6db46d980c75237a1402860 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:26.595768 23978 raft_consensus.cc:515] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b [term 26 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:22:26.598143 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 186966us
W20260709 13:22:26.600376 15046 thread.cc:527] Waited for 2000ms trying to join with rpc worker (tid 16132)
W20260709 13:22:26.602227 16191 leader_election.cc:336] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:22:26.603510 16191 leader_election.cc:304] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:22:26.576857 23911 raft_consensus.cc:493] T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:26.608472 23800 raft_consensus.cc:2749] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:22:26.611842 23911 raft_consensus.cc:515] T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:26.616127 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18ad7e3af5504571a69d10c00750a01c" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:26.616415 23357 leader_election.cc:290] T c0eab25ce4c64e018a041beeda2867db P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:26.617779 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fffab26427e24d398e2695aad40ca134" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:26.618428 16217 raft_consensus.cc:2468] T fffab26427e24d398e2695aad40ca134 P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 21.
I20260709 13:22:26.617741 23942 consensus_queue.cc:1048] T 9e65676b21b748728ad39668cfe722ec P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.002s
W20260709 13:22:26.621871 23984 log.cc:927] Time spent T 9e4bfb9090c8460eb14fa7e0d818e7f1 P cb82cc7bd6db46d980c75237a1402860: Append to log took a long time: real 1.277s	user 0.003s	sys 0.000s
I20260709 13:22:26.624859 23990 raft_consensus.cc:3055] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:22:26.625357 23990 raft_consensus.cc:740] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:22:26.626247 23990 consensus_queue.cc:260] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 22.17, Last appended by leader: 17, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:26.630352 24010 leader_election.cc:290] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:26.622982 16789 leader_election.cc:304] T 23f82224f4564fea93572e8206bad1fe P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 647a343a542d4a139fe106c19095129b
I20260709 13:22:26.657538 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b7d1ab519044eda2d5e04a07242f25" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:26.655053 16117 leader_election.cc:304] T 746aabf856ab401c9ef5e859371e7b67 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:22:26.658243 16224 raft_consensus.cc:2468] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 20.
W20260709 13:22:26.660797 16790 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:22:26.661849 16225 raft_consensus.cc:1240] T 8a878999c0024d2abe517484180290fc P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 18. Current term is 19. Ops: []
I20260709 13:22:26.663138 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:26.663920 16224 raft_consensus.cc:2393] T 19baa7bc367844c288bca8f81439c413 P 1ca2725d290e47d68b1a4b609c9816ac [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 24: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
I20260709 13:22:26.665561 16116 leader_election.cc:304] T b6f07842d3aa47858df5665874c86f3a P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:22:26.669162 16790 leader_election.cc:304] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 647a343a542d4a139fe106c19095129b
I20260709 13:22:26.671627 16220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f2da98864b4425bf7c0055951fdd4b" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:22:26.672267 16220 raft_consensus.cc:3060] T 84f2da98864b4425bf7c0055951fdd4b P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:22:26.673784 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87ac0d1f7e8a419f9bcbc04aee0e067e" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:26.676433 16116 leader_election.cc:304] T af57996a8be84d398f0f023e8d08ef77 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: cb82cc7bd6db46d980c75237a1402860
I20260709 13:22:26.676555 16226 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33050 (request call id 1349) took 1530 ms. Trace:
I20260709 13:22:26.678560 16226 rpcz_store.cc:276] 0709 13:22:25.146076 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:22:25.146220 (+   144us) service_pool.cc:224] Handling call
0709 13:22:26.676396 (+1530176us) spinlock_profiling.cc:243] Waited 1.12 s on lock 0x7b5400395154. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:22:26.676499 (+   103us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2567281024}
I20260709 13:22:26.680361 16822 raft_consensus.cc:1240] T ada5a004ba7c46819051e3ccec71ffdb P cb82cc7bd6db46d980c75237a1402860 [term 23 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 22. Current term is 23. Ops: []
W20260709 13:22:26.681141 16192 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:22:26.685212 16293 raft_consensus.cc:1240] T 8cc959fb7f704fba9009c0595467d074 P 90f3da57796b488c9087a2f523adf045 [term 20 LEADER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 19. Current term is 20. Ops: []
W20260709 13:22:26.685271 16191 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:22:26.692031 16790 leader_election.cc:304] T 66f75124215f4ca6bef831e174a792d8 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:22:26.693110 16822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806d7690c2b644959152074bb2cab0a7" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:26.694438 16823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a378d7f796f94615b3b347119a9880fd" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:22:26.700910 16224 raft_consensus.cc:1240] T 19baa7bc367844c288bca8f81439c413 P 1ca2725d290e47d68b1a4b609c9816ac [term 24 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 23. Current term is 24. Ops: []
I20260709 13:22:26.704396 16223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdbd78acb4cf46298e0de8ba0d130e19" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 25 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:22:26.705338 16223 raft_consensus.cc:2393] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 25: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
I20260709 13:22:26.705376 16218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66ae176961b74813a58bc15c9c4c6f2a" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 27 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:26.663973 24010 raft_consensus.cc:2804] T 23f82224f4564fea93572e8206bad1fe P cb82cc7bd6db46d980c75237a1402860 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:22:26.663422 23774 raft_consensus.cc:2804] T 746aabf856ab401c9ef5e859371e7b67 P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:22:26.713928 16790 leader_election.cc:304] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:22:26.714596 16116 leader_election.cc:304] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: cb82cc7bd6db46d980c75237a1402860
I20260709 13:22:26.715052 16218 raft_consensus.cc:2393] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 27: Already voted for candidate cb82cc7bd6db46d980c75237a1402860 in this term.
W20260709 13:22:26.715987 16266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 297177us
I20260709 13:22:26.666978 23994 raft_consensus.cc:493] T 8158bbd17b994d2eb96f560bd0f1b88d P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:26.727504 16222 raft_consensus.cc:1240] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [term 24 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 23. Current term is 24. Ops: [23.16-23.16]
I20260709 13:22:26.717649 16266 leader_election.cc:304] T c9944d5e4b8c4109af9955c4f08c9095 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860
I20260709 13:22:26.721844 16790 leader_election.cc:304] T ada5a004ba7c46819051e3ccec71ffdb P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:22:26.683005 23798 raft_consensus.cc:3060] T b6f07842d3aa47858df5665874c86f3a P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:22:26.729970 16220 raft_consensus.cc:2468] T 84f2da98864b4425bf7c0055951fdd4b P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 19.
W20260709 13:22:26.731189 16789 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:22:26.732033 16116 leader_election.cc:304] T 68a6838d82ef4cac83507f906139d90f P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 90f3da57796b488c9087a2f523adf045
I20260709 13:22:26.691037 23990 raft_consensus.cc:3060] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:22:26.729661 23685 raft_consensus.cc:2804] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:22:26.738477 16226 raft_consensus.cc:1240] T b6f07842d3aa47858df5665874c86f3a P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 20. Current term is 21. Ops: []
I20260709 13:22:26.742565 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c74490c40243a88e62d1760b5cd814" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:22:26.747813 16297 raft_consensus.cc:3060] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:22:26.749320 24004 raft_consensus.cc:2804] T 68a6838d82ef4cac83507f906139d90f P 647a343a542d4a139fe106c19095129b [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:22:26.749775 24004 raft_consensus.cc:493] T 68a6838d82ef4cac83507f906139d90f P 647a343a542d4a139fe106c19095129b [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:26.750169 24004 raft_consensus.cc:3060] T 68a6838d82ef4cac83507f906139d90f P 647a343a542d4a139fe106c19095129b [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:22:26.743470 16295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f2da98864b4425bf7c0055951fdd4b" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:22:26.745433 16790 leader_election.cc:304] T 3a7ca61592e848ea9b0fb81a2ce33cdc P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
W20260709 13:22:26.745944 16117 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:22:26.755657 16295 raft_consensus.cc:3060] T 84f2da98864b4425bf7c0055951fdd4b P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:22:26.701949 23997 raft_consensus.cc:2804] T af57996a8be84d398f0f023e8d08ef77 P 647a343a542d4a139fe106c19095129b [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:22:26.761883 16291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87ac0d1f7e8a419f9bcbc04aee0e067e" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:26.763634 23975 consensus_queue.cc:1048] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:22:26.719251 23800 consensus_queue.cc:1059] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: INVALID_TERM, Last received: 25.16, Next index: 17, Last known committed idx: 14, Time since last communication: 0.045s
I20260709 13:22:26.717032 23774 raft_consensus.cc:493] T 746aabf856ab401c9ef5e859371e7b67 P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Starting leader election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:22:26.765561 23774 raft_consensus.cc:3060] T 746aabf856ab401c9ef5e859371e7b67 P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:22:26.767271 16297 raft_consensus.cc:2468] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 21.
I20260709 13:22:26.769189 16295 raft_consensus.cc:2468] T 84f2da98864b4425bf7c0055951fdd4b P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 19.
W20260709 13:22:26.726898 23953 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.313s	user 0.010s	sys 0.031s
W20260709 13:22:26.772893 16265 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:22:26.726442 23915 raft_consensus.cc:493] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:26.780342 23761 leader_election.cc:290] T a378d7f796f94615b3b347119a9880fd P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:26.744305 24010 raft_consensus.cc:697] T 23f82224f4564fea93572e8206bad1fe P cb82cc7bd6db46d980c75237a1402860 [term 19 LEADER]: Becoming Leader. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:22:26.781567 23926 raft_consensus.cc:493] T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:22:26.782133 23474 raft_consensus.cc:493] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:26.782369 24015 raft_consensus.cc:2749] T c9944d5e4b8c4109af9955c4f08c9095 P 90f3da57796b488c9087a2f523adf045 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 13:22:26.832432 24042 log.cc:927] Time spent T 8a878999c0024d2abe517484180290fc P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.052s	user 0.001s	sys 0.000s
I20260709 13:22:26.832552 23474 raft_consensus.cc:515] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:22:26.837630 16190 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:22:26.839389 23924 raft_consensus.cc:3055] T 18ad7e3af5504571a69d10c00750a01c P 1ca2725d290e47d68b1a4b609c9816ac [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:22:26.839790 23911 leader_election.cc:290] T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:26.840062 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f5f89186204242bf377f7c42b2e2da" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:22:26.824818 23999 consensus_queue.cc:1048] T fffab26427e24d398e2695aad40ca134 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.016s
I20260709 13:22:26.841094 23897 raft_consensus.cc:3060] T 8a878999c0024d2abe517484180290fc P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:22:26.842113 23287 consensus_queue.cc:1059] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: INVALID_TERM, Last received: 23.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:22:26.839850 23924 raft_consensus.cc:740] T 18ad7e3af5504571a69d10c00750a01c P 1ca2725d290e47d68b1a4b609c9816ac [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:22:26.843798 23815 consensus_queue.cc:1059] T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: INVALID_TERM, Last received: 21.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 13:22:26.844221 23924 consensus_queue.cc:260] T 18ad7e3af5504571a69d10c00750a01c P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } }
I20260709 13:22:26.844671 24026 consensus_queue.cc:1059] T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: INVALID_TERM, Last received: 20.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:22:26.845382 23924 raft_consensus.cc:3060] T 18ad7e3af5504571a69d10c00750a01c P 1ca2725d290e47d68b1a4b609c9816ac [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:22:26.788065 16296 raft_consensus.cc:1240] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045 [term 25 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 24. Current term is 25. Ops: [22.13-24.14]
I20260709 13:22:26.799227 16222 raft_consensus.cc:1275] T 7c2095a5f4ad49408dbc7f69c606120d P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 18 index: 9. Preceding OpId from leader: term: 19 index: 9. (term mismatch)
I20260709 13:22:26.854126 16222 pending_rounds.cc:85] T 7c2095a5f4ad49408dbc7f69c606120d P 1ca2725d290e47d68b1a4b609c9816ac: Aborting all ops after (but not including) 8
I20260709 13:22:26.805693 16220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7ca61592e848ea9b0fb81a2ce33cdc" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:26.809757 16218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1df93779209748b9860f6d893dd3de53" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:22:26.811225 16295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3884f864a96d428785895e5463e5728f" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 18 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:26.816731 16293 raft_consensus.cc:1240] T 1df93779209748b9860f6d893dd3de53 P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 19. Current term is 20. Ops: []
I20260709 13:22:26.816922 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fffab26427e24d398e2695aad40ca134" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:26.818152 16223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a378d7f796f94615b3b347119a9880fd" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:26.854455 16222 pending_rounds.cc:107] T 7c2095a5f4ad49408dbc7f69c606120d P 1ca2725d290e47d68b1a4b609c9816ac: Aborting uncommitted NO_OP operation due to leader change: 18.9
I20260709 13:22:26.855026 16220 raft_consensus.cc:2393] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 1ca2725d290e47d68b1a4b609c9816ac [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 22: Already voted for candidate cb82cc7bd6db46d980c75237a1402860 in this term.
I20260709 13:22:26.856353 16223 raft_consensus.cc:2393] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 22: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
I20260709 13:22:26.818842 16219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ad5dbfa99ee4b8d9e28be92b3f36db5" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:22:26.856351 16222 raft_consensus.cc:2889] T 7c2095a5f4ad49408dbc7f69c606120d P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:22:26.857573 16219 raft_consensus.cc:2376] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 1ca2725d290e47d68b1a4b609c9816ac [term 20 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 20. Re-sending same reply.
I20260709 13:22:26.740661 23951 raft_consensus.cc:2749] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P cb82cc7bd6db46d980c75237a1402860 [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:22:26.821661 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e9b2bf7011f4ae688a79d7bb5a447b4" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:26.860577 16297 raft_consensus.cc:2468] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 20.
I20260709 13:22:26.824698 16221 raft_consensus.cc:1240] T d16dfad0c3b540e5b4b86492ddf87208 P 1ca2725d290e47d68b1a4b609c9816ac [term 22 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 21. Current term is 22. Ops: []
I20260709 13:22:26.730122 24022 raft_consensus.cc:2749] T 66f75124215f4ca6bef831e174a792d8 P cb82cc7bd6db46d980c75237a1402860 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:22:26.829605 16291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
W20260709 13:22:26.830180 24039 log.cc:927] Time spent T 9e65676b21b748728ad39668cfe722ec P cb82cc7bd6db46d980c75237a1402860: Append to log took a long time: real 0.129s	user 0.000s	sys 0.001s
I20260709 13:22:26.811425 23996 raft_consensus.cc:493] T 2794af5b99984efebdee564035841cb0 P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:26.831599 23926 raft_consensus.cc:515] T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:26.783197 23818 raft_consensus.cc:493] T d2ab0ac8347641e6955e9bce7d90c1a2 P 1ca2725d290e47d68b1a4b609c9816ac [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:22:26.782702 23894 raft_consensus.cc:493] T af57996a8be84d398f0f023e8d08ef77 P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:22:26.781594 16266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 64066.3us
I20260709 13:22:26.854405 23980 raft_consensus.cc:3055] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [term 25 LEADER]: Stepping down as leader of term 25
I20260709 13:22:26.848905 23287 raft_consensus.cc:3055] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 13:22:26.849597 24023 raft_consensus.cc:2749] T 3a7ca61592e848ea9b0fb81a2ce33cdc P cb82cc7bd6db46d980c75237a1402860 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:22:26.849731 23815 raft_consensus.cc:3055] T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 13:22:26.850927 24026 raft_consensus.cc:3055] T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:22:26.746801 23685 raft_consensus.cc:493] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:26.744884 23944 raft_consensus.cc:2804] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [term 26 FOLLOWER]: Leader pre-election won for term 27
W20260709 13:22:26.784225 23953 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.370s	user 0.010s	sys 0.031s
I20260709 13:22:26.780377 24020 raft_consensus.cc:2749] T ada5a004ba7c46819051e3ccec71ffdb P cb82cc7bd6db46d980c75237a1402860 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 13:22:26.716266 23384 consensus_queue.cc:1059] T 66f75124215f4ca6bef831e174a792d8 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: INVALID_TERM, Last received: 19.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.098s
I20260709 13:22:26.786372 24003 leader_election.cc:290] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 25 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:22:26.834973 23474 leader_election.cc:290] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:26.740201 23869 consensus_queue.cc:1059] T 8cc959fb7f704fba9009c0595467d074 P 647a343a542d4a139fe106c19095129b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.112s
I20260709 13:22:26.815555 23941 raft_consensus.cc:493] T 02838c86531d4f98bf58d1c22b8baf77 P 1ca2725d290e47d68b1a4b609c9816ac [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:22:26.734330 23994 raft_consensus.cc:515] T 8158bbd17b994d2eb96f560bd0f1b88d P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:26.827020 23719 leader_election.cc:290] T ecc1e363230043fe9fef390f0b979f20 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 25 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:26.836850 23774 raft_consensus.cc:515] T 746aabf856ab401c9ef5e859371e7b67 P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:26.823676 23915 raft_consensus.cc:515] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:26.889707 23815 raft_consensus.cc:740] T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:22:26.889345 23980 raft_consensus.cc:740] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [term 25 LEADER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:22:26.894443 23685 raft_consensus.cc:3060] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:22:26.891690 24026 raft_consensus.cc:740] T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:22:26.892300 23785 leader_election.cc:290] T c9944d5e4b8c4109af9955c4f08c9095 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 24 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:26.895576 23944 raft_consensus.cc:493] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [term 26 FOLLOWER]: Starting leader election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:22:26.885303 23996 raft_consensus.cc:515] T 2794af5b99984efebdee564035841cb0 P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:26.916097 23944 raft_consensus.cc:3060] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [term 26 FOLLOWER]: Advancing to term 27
W20260709 13:22:26.918583 24040 log.cc:927] Time spent T 8cc959fb7f704fba9009c0595467d074 P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.065s	user 0.000s	sys 0.000s
I20260709 13:22:26.908627 23978 leader_election.cc:290] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 26 election: Requested vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:26.918895 24010 consensus_queue.cc:237] T 23f82224f4564fea93572e8206bad1fe P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:26.938851 24004 raft_consensus.cc:515] T 68a6838d82ef4cac83507f906139d90f P 647a343a542d4a139fe106c19095129b [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:26.890275 23818 raft_consensus.cc:515] T d2ab0ac8347641e6955e9bce7d90c1a2 P 1ca2725d290e47d68b1a4b609c9816ac [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:26.941514 23818 leader_election.cc:290] T d2ab0ac8347641e6955e9bce7d90c1a2 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:26.922178 24026 consensus_queue.cc:260] T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.13, Last appended by leader: 13, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:26.938724 23944 raft_consensus.cc:515] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [term 27 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:26.942816 24026 raft_consensus.cc:3060] T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:22:26.944644 23944 leader_election.cc:290] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 27 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:26.927874 23996 leader_election.cc:290] T 2794af5b99984efebdee564035841cb0 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:26.928639 23994 leader_election.cc:290] T 8158bbd17b994d2eb96f560bd0f1b88d P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:26.905144 23941 raft_consensus.cc:515] T 02838c86531d4f98bf58d1c22b8baf77 P 1ca2725d290e47d68b1a4b609c9816ac [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:26.923365 23980 consensus_queue.cc:260] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 25.16, Last appended by leader: 16, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:26.840744 16217 raft_consensus.cc:3060] T c2f5f89186204242bf377f7c42b2e2da P 1ca2725d290e47d68b1a4b609c9816ac [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:22:26.886607 16266 leader_election.cc:400] T 23f82224f4564fea93572e8206bad1fe P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 18 election: Vote denied by peer cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923) with higher term. Message: Invalid argument: T 23f82224f4564fea93572e8206bad1fe P cb82cc7bd6db46d980c75237a1402860 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 for earlier term 18. Current term is 19.
I20260709 13:22:26.890079 16291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f82224f4564fea93572e8206bad1fe" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:22:26.885334 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434bff3082d141838068f323bdb0d1ed" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:26.887112 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0eab25ce4c64e018a041beeda2867db" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 23 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:26.893833 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26354361e9b044cea1c2b86c4882fe92" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:26.921758 23774 leader_election.cc:290] T 746aabf856ab401c9ef5e859371e7b67 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 19 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:26.890110 23287 raft_consensus.cc:740] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:22:26.926692 23815 consensus_queue.cc:260] T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 23.16, Last appended by leader: 16, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:26.890039 23894 raft_consensus.cc:515] T af57996a8be84d398f0f023e8d08ef77 P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:26.919044 23926 leader_election.cc:290] T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:26.923118 23938 raft_consensus.cc:515] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:27.138211 23815 raft_consensus.cc:3060] T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:22:27.077175 16219 raft_consensus.cc:1240] T 746aabf856ab401c9ef5e859371e7b67 P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 18. Current term is 19. Ops: []
W20260709 13:22:27.065798 16189 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:22:27.181012 16189 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:22:27.155865 23869 raft_consensus.cc:493] T 7c2095a5f4ad49408dbc7f69c606120d P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:27.141462 23287 consensus_queue.cc:260] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 23.13, Last appended by leader: 13, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:22:26.965516 23964 consensus_meta.cc:287] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 647a343a542d4a139fe106c19095129b: Time spent flushing consensus metadata: real 0.565s	user 0.007s	sys 0.003s
I20260709 13:22:27.179287 16219 raft_consensus.cc:1240] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac [term 22 FOLLOWER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 21. Current term is 22. Ops: []
I20260709 13:22:27.117220 16819 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49892 (request call id 1161) took 1011 ms. Trace:
I20260709 13:22:27.182658 23869 raft_consensus.cc:515] T 7c2095a5f4ad49408dbc7f69c606120d P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:27.062403 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:22:27.124281 23605 raft_consensus.cc:3055] T 8cc959fb7f704fba9009c0595467d074 P 647a343a542d4a139fe106c19095129b [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:22:27.207628 23605 raft_consensus.cc:740] T 8cc959fb7f704fba9009c0595467d074 P 647a343a542d4a139fe106c19095129b [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:22:27.207758 16298 raft_consensus.cc:3060] T ecc1e363230043fe9fef390f0b979f20 P 90f3da57796b488c9087a2f523adf045 [term 23 FOLLOWER]: Advancing to term 25
I20260709 13:22:27.209165 23605 consensus_queue.cc:260] T 8cc959fb7f704fba9009c0595467d074 P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.14, Last appended by leader: 14, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:27.190766 23287 raft_consensus.cc:3060] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:22:26.947005 23915 leader_election.cc:290] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:27.191407 16226 raft_consensus.cc:1240] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [term 22 FOLLOWER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 21. Current term is 22. Ops: [21.17-21.17]
I20260709 13:22:27.212141 16299 raft_consensus.cc:2468] T 26354361e9b044cea1c2b86c4882fe92 P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 18.
I20260709 13:22:27.213609 16226 raft_consensus.cc:1240] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [term 25 FOLLOWER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 24. Current term is 25. Ops: [24.18-24.18]
I20260709 13:22:27.213402 16296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31969392ccdb43b491911a319e8bc7e1" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:27.186347 23719 raft_consensus.cc:493] T 2794af5b99984efebdee564035841cb0 P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:27.215320 23988 raft_consensus.cc:493] T ada5a004ba7c46819051e3ccec71ffdb P 90f3da57796b488c9087a2f523adf045 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:22:27.215072 23719 raft_consensus.cc:515] T 2794af5b99984efebdee564035841cb0 P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:27.216024 16266 leader_election.cc:403] T 23f82224f4564fea93572e8206bad1fe P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 18 election: Cancelling election due to peer responding with higher term
I20260709 13:22:27.216101 23988 raft_consensus.cc:515] T ada5a004ba7c46819051e3ccec71ffdb P 90f3da57796b488c9087a2f523adf045 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:27.211145 23605 raft_consensus.cc:3060] T 8cc959fb7f704fba9009c0595467d074 P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:22:27.218200 23719 leader_election.cc:290] T 2794af5b99984efebdee564035841cb0 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:27.163693 23978 raft_consensus.cc:493] T 829b4ad8c5a54eababeea1181b960d4b P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:27.197947 16300 raft_consensus.cc:1240] T 8a878999c0024d2abe517484180290fc P 90f3da57796b488c9087a2f523adf045 [term 19 LEADER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 18. Current term is 19. Ops: []
I20260709 13:22:27.073860 16223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8638ee363273426db1393dd81e5c7b72" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:22:27.203920 16291 raft_consensus.cc:2393] T 23f82224f4564fea93572e8206bad1fe P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 19: Already voted for candidate cb82cc7bd6db46d980c75237a1402860 in this term.
W20260709 13:22:27.035837 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 288502us
W20260709 13:22:26.967947 23958 consensus_meta.cc:287] T 829b4ad8c5a54eababeea1181b960d4b P 1ca2725d290e47d68b1a4b609c9816ac: Time spent flushing consensus metadata: real 0.562s	user 0.013s	sys 0.001s
I20260709 13:22:27.076064 16221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:27.215696 16297 raft_consensus.cc:2393] T c0eab25ce4c64e018a041beeda2867db P 90f3da57796b488c9087a2f523adf045 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 23: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
I20260709 13:22:27.204380 16219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74ca8140f35c4ae29e8a664f6bdfc829" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 24 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:27.194280 16220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66ae176961b74813a58bc15c9c4c6f2a" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 27 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:22:27.063762 16292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a9de347c711477e87c582d003f36925" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 26 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:22:27.221513 24013 raft_consensus.cc:2749] T 23f82224f4564fea93572e8206bad1fe P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Leader election lost for term 18. Reason: Vote denied by peer cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923) with higher term. Message: Invalid argument: T 23f82224f4564fea93572e8206bad1fe P cb82cc7bd6db46d980c75237a1402860 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 for earlier term 18. Current term is 19.
I20260709 13:22:27.248943 16220 raft_consensus.cc:2376] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [term 27 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 27. Re-sending same reply.
W20260709 13:22:27.227478 16264 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:22:27.233366 23894 leader_election.cc:290] T af57996a8be84d398f0f023e8d08ef77 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
W20260709 13:22:27.233904 16190 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:22:27.234552 24062 log.cc:927] Time spent T 9e65676b21b748728ad39668cfe722ec P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.119s	user 0.003s	sys 0.000s
W20260709 13:22:27.234994 24014 log.cc:927] Time spent T 5a96373bb0e14dce8a266b2fc6c58507 P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.080s	user 0.001s	sys 0.000s
I20260709 13:22:26.987537 23997 raft_consensus.cc:697] T af57996a8be84d398f0f023e8d08ef77 P 647a343a542d4a139fe106c19095129b [term 21 LEADER]: Becoming Leader. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:22:27.236475 23768 raft_consensus.cc:493] T af57996a8be84d398f0f023e8d08ef77 P cb82cc7bd6db46d980c75237a1402860 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:22:27.252661 16190 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:22:27.240247 23938 leader_election.cc:290] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 22 election: Requested vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:27.240517 23995 raft_consensus.cc:493] T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:22:27.241441 23964 raft_consensus.cc:515] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 647a343a542d4a139fe106c19095129b [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:27.186866 23994 raft_consensus.cc:493] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:27.242630 23978 raft_consensus.cc:515] T 829b4ad8c5a54eababeea1181b960d4b P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:27.260710 23964 leader_election.cc:290] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 21 election: Requested vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:27.260931 23978 leader_election.cc:290] T 829b4ad8c5a54eababeea1181b960d4b P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:27.227420 23980 raft_consensus.cc:3060] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:22:27.258500 23995 raft_consensus.cc:515] T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:27.262650 23997 consensus_queue.cc:237] T af57996a8be84d398f0f023e8d08ef77 P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.13, Last appended by leader: 13, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:27.265216 23995 leader_election.cc:290] T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
W20260709 13:22:27.218570 16266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 332174us
I20260709 13:22:27.266345 16219 raft_consensus.cc:2393] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 in current term 24: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
W20260709 13:22:27.251197 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 397966us
I20260709 13:22:27.267475 16790 leader_election.cc:304] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:22:27.268141 24026 consensus_queue.cc:1059] T d16dfad0c3b540e5b4b86492ddf87208 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: INVALID_TERM, Last received: 21.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:22:27.269558 16790 leader_election.cc:304] T 8c11074579b046d99e585c2818e18a2c P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:22:27.269877 23768 raft_consensus.cc:515] T af57996a8be84d398f0f023e8d08ef77 P cb82cc7bd6db46d980c75237a1402860 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:27.270601 23785 raft_consensus.cc:2749] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 [term 26 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:22:27.269577 24026 raft_consensus.cc:3055] T d16dfad0c3b540e5b4b86492ddf87208 P 90f3da57796b488c9087a2f523adf045 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:22:27.272233 24026 raft_consensus.cc:740] T d16dfad0c3b540e5b4b86492ddf87208 P 90f3da57796b488c9087a2f523adf045 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:22:27.257670 16223 raft_consensus.cc:2368] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 for earlier term 21. Current term is 22.
I20260709 13:22:27.258509 23994 raft_consensus.cc:515] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:27.249266 24003 consensus_queue.cc:1048] T 7c2095a5f4ad49408dbc7f69c606120d P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.025s
I20260709 13:22:27.275976 23768 leader_election.cc:290] T af57996a8be84d398f0f023e8d08ef77 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:22:27.276039 23994 leader_election.cc:290] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:27.280078 16221 raft_consensus.cc:2393] T 19baa7bc367844c288bca8f81439c413 P 1ca2725d290e47d68b1a4b609c9816ac [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 in current term 24: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
I20260709 13:22:27.280200 24004 leader_election.cc:290] T 68a6838d82ef4cac83507f906139d90f P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 22 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:27.277628 23941 leader_election.cc:290] T 02838c86531d4f98bf58d1c22b8baf77 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:27.282292 23638 raft_consensus.cc:2749] T 8c11074579b046d99e585c2818e18a2c P cb82cc7bd6db46d980c75237a1402860 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:22:27.280264 16266 leader_election.cc:304] T ecc1e363230043fe9fef390f0b979f20 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860
I20260709 13:22:27.299589 16298 raft_consensus.cc:2468] T ecc1e363230043fe9fef390f0b979f20 P 90f3da57796b488c9087a2f523adf045 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 25.
I20260709 13:22:27.304015 16217 raft_consensus.cc:2468] T c2f5f89186204242bf377f7c42b2e2da P 1ca2725d290e47d68b1a4b609c9816ac [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 25.
W20260709 13:22:27.310254 16266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 43655.2us
I20260709 13:22:27.312053 23815 raft_consensus.cc:2749] T ecc1e363230043fe9fef390f0b979f20 P 90f3da57796b488c9087a2f523adf045 [term 25 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:22:27.312973 23945 raft_consensus.cc:3055] T 66f75124215f4ca6bef831e174a792d8 P 1ca2725d290e47d68b1a4b609c9816ac [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:22:27.313436 23945 raft_consensus.cc:740] T 66f75124215f4ca6bef831e174a792d8 P 1ca2725d290e47d68b1a4b609c9816ac [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:22:27.308023 24013 raft_consensus.cc:493] T 02838c86531d4f98bf58d1c22b8baf77 P 90f3da57796b488c9087a2f523adf045 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:22:27.188071 16819 rpcz_store.cc:276] 0709 13:22:26.105296 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:22:26.105428 (+   132us) service_pool.cc:224] Handling call
0709 13:22:26.105720 (+   292us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:22:26.106042 (+   322us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:22:26.106068 (+    26us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:22:26.112695 (+  6627us) log.cc:844] Serialized 52 byte log entry
0709 13:22:26.549225 (+436530us) spinlock_profiling.cc:243] Waited 14.4 ms on lock 0x7b180041b000. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b693ade1 00007f69b69475b1 00007f69b07074bc 00007f69ada5548b 00007f69b68afab9 00007f69ada7169b 00007f69ada6f93f 00007f69ada71c02 00007f69b06f920b 00007f69b06fed23 00007f69b06ff0d1
0709 13:22:26.549384 (+   159us) raft_consensus.cc:1645] Marking committed up to 15
0709 13:22:26.766041 (+216657us) log.cc:844] Serialized 14 byte log entry
0709 13:22:26.903606 (+137565us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:22:26.903679 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:22:26.903722 (+    43us) raft_consensus.cc:1685] finished
0709 13:22:26.907434 (+  3712us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:22:27.116862 (+209428us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":24228,"prepare.queue_time_us":286911,"prepare.run_cpu_time_us":35,"prepare.run_wall_time_us":36,"raft.queue_time_us":102816,"raft.run_cpu_time_us":310,"raft.run_wall_time_us":27118,"spinlock_wait_cycles":33146112,"thread_start_us":360002,"threads_started":3,"wal-append.queue_time_us":143423}
I20260709 13:22:27.319027 24013 raft_consensus.cc:515] T 02838c86531d4f98bf58d1c22b8baf77 P 90f3da57796b488c9087a2f523adf045 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:27.326802 23869 leader_election.cc:290] T 7c2095a5f4ad49408dbc7f69c606120d P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
W20260709 13:22:27.335959 16191 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:22:27.314459 23945 consensus_queue.cc:260] T 66f75124215f4ca6bef831e174a792d8 P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:22:27.345649 16190 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:22:27.349519 16115 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:22:27.344410 16116 leader_election.cc:304] T 5a959dbecc2648eb85d1cc550f5a3245 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 90f3da57796b488c9087a2f523adf045
I20260709 13:22:27.377096 23287 consensus_queue.cc:1048] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 13:22:27.376901 16266 leader_election.cc:304] T 26354361e9b044cea1c2b86c4882fe92 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 
W20260709 13:22:27.399501 16117 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:22:27.378800 23685 raft_consensus.cc:515] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:22:27.405061 16117 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:22:27.408526 16117 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:22:27.406924 23996 raft_consensus.cc:2804] T 26354361e9b044cea1c2b86c4882fe92 P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:22:27.398797 23719 raft_consensus.cc:2804] T 5a959dbecc2648eb85d1cc550f5a3245 P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:22:27.411182 23958 raft_consensus.cc:493] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:27.402603 23945 raft_consensus.cc:3060] T 66f75124215f4ca6bef831e174a792d8 P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:22:27.411583 16298 raft_consensus.cc:1275] T 8638ee363273426db1393dd81e5c7b72 P 90f3da57796b488c9087a2f523adf045 [term 21 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 20 index: 16. Preceding OpId from leader: term: 21 index: 17. (index mismatch)
I20260709 13:22:27.414656 23869 raft_consensus.cc:493] T ee9ed67006144797b1de6ea8c2013a68 P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:27.416399 16295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8158bbd17b994d2eb96f560bd0f1b88d" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:27.412748 23719 raft_consensus.cc:493] T 5a959dbecc2648eb85d1cc550f5a3245 P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Starting leader election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:22:27.411713 23996 raft_consensus.cc:493] T 26354361e9b044cea1c2b86c4882fe92 P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:27.414417 23958 raft_consensus.cc:515] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:27.422966 23719 raft_consensus.cc:3060] T 5a959dbecc2648eb85d1cc550f5a3245 P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:22:27.423507 23996 raft_consensus.cc:3060] T 26354361e9b044cea1c2b86c4882fe92 P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:22:27.430599 16295 raft_consensus.cc:2468] T 8158bbd17b994d2eb96f560bd0f1b88d P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 20.
I20260709 13:22:27.429842 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31969392ccdb43b491911a319e8bc7e1" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:27.433830 23869 raft_consensus.cc:515] T ee9ed67006144797b1de6ea8c2013a68 P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:27.429253 16266 leader_election.cc:304] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac
I20260709 13:22:27.439721 16292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ab0ac8347641e6955e9bce7d90c1a2" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:27.442301 24004 raft_consensus.cc:493] T 8a878999c0024d2abe517484180290fc P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:27.442986 24004 raft_consensus.cc:515] T 8a878999c0024d2abe517484180290fc P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:22:27.444638 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 138479us
I20260709 13:22:27.451907 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434bff3082d141838068f323bdb0d1ed" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:27.452437 16821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ab0ac8347641e6955e9bce7d90c1a2" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:27.454010 16821 raft_consensus.cc:2468] T d2ab0ac8347641e6955e9bce7d90c1a2 P cb82cc7bd6db46d980c75237a1402860 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 20.
W20260709 13:22:27.445994 23798 consensus_meta.cc:287] T b6f07842d3aa47858df5665874c86f3a P 647a343a542d4a139fe106c19095129b: Time spent flushing consensus metadata: real 0.668s	user 0.008s	sys 0.012s
I20260709 13:22:27.458860 23815 raft_consensus.cc:2804] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:22:27.465699 23685 leader_election.cc:290] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 25 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:27.471092 23798 raft_consensus.cc:2749] T b6f07842d3aa47858df5665874c86f3a P 647a343a542d4a139fe106c19095129b [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:22:27.473724 16222 raft_consensus.cc:1275] T 8cc959fb7f704fba9009c0595467d074 P 1ca2725d290e47d68b1a4b609c9816ac [term 20 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 13:22:27.470261 23815 raft_consensus.cc:697] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045 [term 23 LEADER]: Becoming Leader. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:22:27.478544 23815 consensus_queue.cc:237] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 22.17, Last appended by leader: 17, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:27.480576 24004 leader_election.cc:290] T 8a878999c0024d2abe517484180290fc P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:27.482605 23975 raft_consensus.cc:1064] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b: attempting to promote NON_VOTER cb82cc7bd6db46d980c75237a1402860 to VOTER
I20260709 13:22:27.483204 16821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af57996a8be84d398f0f023e8d08ef77" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:27.483852 16821 raft_consensus.cc:2468] T af57996a8be84d398f0f023e8d08ef77 P cb82cc7bd6db46d980c75237a1402860 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 21.
I20260709 13:22:27.485311 16819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8638ee363273426db1393dd81e5c7b72" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:22:27.476456 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:27.486454 16297 raft_consensus.cc:2468] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 24.
I20260709 13:22:27.490367 23869 leader_election.cc:290] T ee9ed67006144797b1de6ea8c2013a68 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:27.490645 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9944d5e4b8c4109af9955c4f08c9095" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:22:27.496984 16824 raft_consensus.cc:1240] T c9944d5e4b8c4109af9955c4f08c9095 P cb82cc7bd6db46d980c75237a1402860 [term 24 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 23. Current term is 24. Ops: []
I20260709 13:22:27.489034 16116 leader_election.cc:304] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: cb82cc7bd6db46d980c75237a1402860
I20260709 13:22:27.498317 23287 raft_consensus.cc:493] T 8c11074579b046d99e585c2818e18a2c P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:22:27.499653 16300 raft_consensus.cc:3060] T c9944d5e4b8c4109af9955c4f08c9095 P 90f3da57796b488c9087a2f523adf045 [term 23 FOLLOWER]: Advancing to term 24
W20260709 13:22:27.503451 16789 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:22:27.506306 23287 raft_consensus.cc:515] T 8c11074579b046d99e585c2818e18a2c P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:22:27.508323 16265 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:22:27.509011 16265 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:22:27.513499 24026 consensus_queue.cc:260] T d16dfad0c3b540e5b4b86492ddf87208 P 90f3da57796b488c9087a2f523adf045 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.12, Last appended by leader: 12, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:27.514385 24013 leader_election.cc:290] T 02838c86531d4f98bf58d1c22b8baf77 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:22:27.524736 23287 leader_election.cc:290] T 8c11074579b046d99e585c2818e18a2c P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:27.527068 16821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a9de347c711477e87c582d003f36925" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 26 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:22:27.534660 16819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31969392ccdb43b491911a319e8bc7e1" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:27.535384 24026 raft_consensus.cc:3060] T d16dfad0c3b540e5b4b86492ddf87208 P 90f3da57796b488c9087a2f523adf045 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:22:27.535529 16819 raft_consensus.cc:2468] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 19.
I20260709 13:22:27.536561 16824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:22:27.537289 16824 raft_consensus.cc:2393] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 25: Already voted for candidate cb82cc7bd6db46d980c75237a1402860 in this term.
I20260709 13:22:27.539247 24013 raft_consensus.cc:493] T 57a744f5aaf944678a63469cbf514015 P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:27.539418 16823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746aabf856ab401c9ef5e859371e7b67" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:22:27.539987 24013 raft_consensus.cc:515] T 57a744f5aaf944678a63469cbf514015 P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:27.542734 24013 leader_election.cc:290] T 57a744f5aaf944678a63469cbf514015 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
W20260709 13:22:27.545378 16192 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:22:27.546180 16192 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:22:27.547024 16265 leader_election.cc:304] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:22:27.547005 16192 leader_election.cc:304] T d2ab0ac8347641e6955e9bce7d90c1a2 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:22:27.552501 24003 raft_consensus.cc:2804] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:22:27.553010 24003 raft_consensus.cc:697] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 90f3da57796b488c9087a2f523adf045 [term 20 LEADER]: Becoming Leader. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:22:27.553347 23975 consensus_queue.cc:237] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 21.14, Last appended by leader: 13, Current term: 21, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:27.554037 24003 consensus_queue.cc:237] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:22:27.564254 23924 consensus_meta.cc:287] T 18ad7e3af5504571a69d10c00750a01c P 1ca2725d290e47d68b1a4b609c9816ac: Time spent flushing consensus metadata: real 0.698s	user 0.015s	sys 0.007s
I20260709 13:22:27.568337 16300 raft_consensus.cc:2468] T c9944d5e4b8c4109af9955c4f08c9095 P 90f3da57796b488c9087a2f523adf045 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 24.
W20260709 13:22:27.570731 16266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 157134us
I20260709 13:22:27.572557 23988 leader_election.cc:290] T ada5a004ba7c46819051e3ccec71ffdb P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:27.572638 23941 raft_consensus.cc:2804] T d2ab0ac8347641e6955e9bce7d90c1a2 P 1ca2725d290e47d68b1a4b609c9816ac [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:22:27.573313 23941 raft_consensus.cc:493] T d2ab0ac8347641e6955e9bce7d90c1a2 P 1ca2725d290e47d68b1a4b609c9816ac [term 20 FOLLOWER]: Starting leader election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:22:27.573712 23941 raft_consensus.cc:3060] T d2ab0ac8347641e6955e9bce7d90c1a2 P 1ca2725d290e47d68b1a4b609c9816ac [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:22:27.574672 16189 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:22:27.578897 16192 leader_election.cc:304] T af57996a8be84d398f0f023e8d08ef77 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:22:27.580017 23924 raft_consensus.cc:2804] T af57996a8be84d398f0f023e8d08ef77 P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:22:27.580403 23924 raft_consensus.cc:493] T af57996a8be84d398f0f023e8d08ef77 P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:27.580739 23924 raft_consensus.cc:3060] T af57996a8be84d398f0f023e8d08ef77 P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:22:27.580657 23938 consensus_queue.cc:1059] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: INVALID_TERM, Last received: 23.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.001s
I20260709 13:22:27.582343 23938 raft_consensus.cc:3055] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac [term 23 LEADER]: Stepping down as leader of term 23
I20260709 13:22:27.582715 23938 raft_consensus.cc:740] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:22:27.583586 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada5a004ba7c46819051e3ccec71ffdb" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:27.584316 16217 raft_consensus.cc:2468] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 23.
I20260709 13:22:27.592224 16821 raft_consensus.cc:3060] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860 [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:22:27.596261 23638 raft_consensus.cc:493] T ee9ed67006144797b1de6ea8c2013a68 P cb82cc7bd6db46d980c75237a1402860 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:27.596503 16819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62fe3346cd3c42c4a55e14af9073bca4" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 27 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:22:27.597096 23638 raft_consensus.cc:515] T ee9ed67006144797b1de6ea8c2013a68 P cb82cc7bd6db46d980c75237a1402860 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:27.599203 16823 raft_consensus.cc:2468] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 19.
W20260709 13:22:27.599429 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 65813.2us
I20260709 13:22:27.594693 23869 raft_consensus.cc:2804] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b [term 22 FOLLOWER]: Leader pre-election won for term 23
W20260709 13:22:27.622416 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 164103us
I20260709 13:22:27.623773 16823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2794af5b99984efebdee564035841cb0" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 21 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
W20260709 13:22:27.626950 16191 leader_election.cc:336] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:22:27.633174 16819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e9b2bf7011f4ae688a79d7bb5a447b4" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:22:27.636832 16191 leader_election.cc:304] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
I20260709 13:22:27.633404 16824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "829b4ad8c5a54eababeea1181b960d4b" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 21 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:27.624697 23869 raft_consensus.cc:493] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:22:27.630605 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 81886.9us
I20260709 13:22:27.644789 16824 raft_consensus.cc:2410] T 829b4ad8c5a54eababeea1181b960d4b P cb82cc7bd6db46d980c75237a1402860 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b for term 21 because replica has last-logged OpId of term: 19 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 14.
I20260709 13:22:27.648578 16819 raft_consensus.cc:3060] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:22:27.644330 23869 raft_consensus.cc:3060] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:22:27.653162 16266 leader_election.cc:304] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860
I20260709 13:22:27.658144 23938 consensus_queue.cc:260] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 23.17, Last appended by leader: 17, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:27.641613 23958 leader_election.cc:290] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
W20260709 13:22:27.671121 24068 log.cc:927] Time spent T af57996a8be84d398f0f023e8d08ef77 P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 0.203s	user 0.000s	sys 0.002s
I20260709 13:22:27.671237 16817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee9ed67006144797b1de6ea8c2013a68" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:27.675011 16817 raft_consensus.cc:2468] T ee9ed67006144797b1de6ea8c2013a68 P cb82cc7bd6db46d980c75237a1402860 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 20.
I20260709 13:22:27.655038 23894 raft_consensus.cc:2749] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:22:27.676064 16117 leader_election.cc:304] T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860; no voters: 
W20260709 13:22:27.680316 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 72690.2us
W20260709 13:22:27.681011 16190 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:22:27.681624 16190 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:22:27.678292 16116 leader_election.cc:304] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860
I20260709 13:22:27.685431 16265 leader_election.cc:304] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
W20260709 13:22:27.681682 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 59126us
I20260709 13:22:27.699734 16822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66f75124215f4ca6bef831e174a792d8" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:27.700690 16822 raft_consensus.cc:2393] T 66f75124215f4ca6bef831e174a792d8 P cb82cc7bd6db46d980c75237a1402860 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 20: Already voted for candidate cb82cc7bd6db46d980c75237a1402860 in this term.
I20260709 13:22:27.707365 16115 leader_election.cc:304] T 08c74490c40243a88e62d1760b5cd814 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:22:27.708714 16115 leader_election.cc:304] T 84f2da98864b4425bf7c0055951fdd4b P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:22:27.710402 16115 leader_election.cc:304] T 3884f864a96d428785895e5463e5728f P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860
I20260709 13:22:27.687845 23287 raft_consensus.cc:3060] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 90f3da57796b488c9087a2f523adf045 [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:22:27.699728 16266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 50054us
I20260709 13:22:27.702905 16117 leader_election.cc:304] T 746aabf856ab401c9ef5e859371e7b67 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:22:27.722932 16819 raft_consensus.cc:2468] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 21.
I20260709 13:22:27.724151 23605 raft_consensus.cc:2804] T 08c74490c40243a88e62d1760b5cd814 P 647a343a542d4a139fe106c19095129b [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:22:27.724803 23605 raft_consensus.cc:697] T 08c74490c40243a88e62d1760b5cd814 P 647a343a542d4a139fe106c19095129b [term 21 LEADER]: Becoming Leader. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:22:27.725903 23605 consensus_queue.cc:237] T 08c74490c40243a88e62d1760b5cd814 P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.12, Last appended by leader: 12, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:27.731741 16266 leader_election.cc:304] T 66ae176961b74813a58bc15c9c4c6f2a P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860
W20260709 13:22:27.740350 16191 leader_election.cc:336] T 8a878999c0024d2abe517484180290fc P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:22:27.741781 16821 raft_consensus.cc:2468] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 26.
W20260709 13:22:27.744055 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 82877.3us
I20260709 13:22:27.727361 23995 raft_consensus.cc:2804] T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:22:27.698734 23996 raft_consensus.cc:515] T 26354361e9b044cea1c2b86c4882fe92 P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:27.747493 16191 leader_election.cc:304] T 8a878999c0024d2abe517484180290fc P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
W20260709 13:22:27.747113 16117 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 44349.7us
I20260709 13:22:27.727377 23938 raft_consensus.cc:3060] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:22:27.753290 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e9b2bf7011f4ae688a79d7bb5a447b4" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
W20260709 13:22:27.753917 16264 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:22:27.754323 16190 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:22:27.754436 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2794af5b99984efebdee564035841cb0" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 21 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:27.760535 16117 leader_election.cc:304] T ee9ed67006144797b1de6ea8c2013a68 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:22:27.741285 23975 raft_consensus.cc:2749] T 3884f864a96d428785895e5463e5728f P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:22:27.743675 23997 raft_consensus.cc:2804] T 84f2da98864b4425bf7c0055951fdd4b P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:22:27.765908 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a878999c0024d2abe517484180290fc" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:27.747814 23995 raft_consensus.cc:493] T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:27.766072 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ad5dbfa99ee4b8d9e28be92b3f36db5" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:27.745177 23638 leader_election.cc:290] T ee9ed67006144797b1de6ea8c2013a68 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:22:27.772891 16292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68a6838d82ef4cac83507f906139d90f" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:22:27.778126 16300 raft_consensus.cc:3060] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:22:27.775599 23798 raft_consensus.cc:2804] T ee9ed67006144797b1de6ea8c2013a68 P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:22:27.766238 16291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c2095a5f4ad49408dbc7f69c606120d" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 20 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:27.783707 16296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:22:27.784497 16296 raft_consensus.cc:2393] T ecc1e363230043fe9fef390f0b979f20 P 90f3da57796b488c9087a2f523adf045 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 in current term 25: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
W20260709 13:22:27.791419 16114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:22:27.793697 16293 raft_consensus.cc:1275] T 57a744f5aaf944678a63469cbf514015 P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 13:22:27.799497 23915 raft_consensus.cc:2749] T 8a878999c0024d2abe517484180290fc P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:22:27.800968 16190 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:22:27.780185 23995 raft_consensus.cc:3060] T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:22:27.802358 23996 leader_election.cc:290] T 26354361e9b044cea1c2b86c4882fe92 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 19 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:27.804100 23865 consensus_queue.cc:1048] T 8cc959fb7f704fba9009c0595467d074 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:22:27.805130 23915 raft_consensus.cc:493] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:27.805876 23915 raft_consensus.cc:515] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } }
I20260709 13:22:27.765206 23994 raft_consensus.cc:2749] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 13:22:27.811476 24077 log.cc:927] Time spent T 9e65676b21b748728ad39668cfe722ec P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 0.102s	user 0.003s	sys 0.000s
W20260709 13:22:27.815618 16266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 83985.8us
I20260709 13:22:27.817335 16266 leader_election.cc:304] T cdbd78acb4cf46298e0de8ba0d130e19 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac
W20260709 13:22:27.818486 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 174402us
I20260709 13:22:27.820307 16116 leader_election.cc:304] T 9c61a39fd4f84424b129c1a6f9ba404d P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 90f3da57796b488c9087a2f523adf045
I20260709 13:22:27.821719 16116 leader_election.cc:304] T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 90f3da57796b488c9087a2f523adf045
W20260709 13:22:27.827323 16789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 75844.1us
I20260709 13:22:27.827433 23915 leader_election.cc:290] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:27.828382 16789 leader_election.cc:304] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:22:27.829282 23785 consensus_queue.cc:1048] T 8638ee363273426db1393dd81e5c7b72 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 13:22:27.830133 16789 leader_election.cc:304] T 434bff3082d141838068f323bdb0d1ed P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:22:27.830981 23785 raft_consensus.cc:2749] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:22:27.831893 23785 raft_consensus.cc:2749] T 434bff3082d141838068f323bdb0d1ed P cb82cc7bd6db46d980c75237a1402860 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:22:27.766602 23815 raft_consensus.cc:3060] T 66ae176961b74813a58bc15c9c4c6f2a P 90f3da57796b488c9087a2f523adf045 [term 26 FOLLOWER]: Advancing to term 27
W20260709 13:22:27.835785 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 120099us
I20260709 13:22:27.836939 16299 raft_consensus.cc:1275] T 23f82224f4564fea93572e8206bad1fe P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
W20260709 13:22:27.838047 16191 leader_election.cc:336] T b6f07842d3aa47858df5665874c86f3a P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 21 election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:22:27.839330 16189 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:22:27.778821 23978 raft_consensus.cc:2804] T 746aabf856ab401c9ef5e859371e7b67 P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:22:27.838774 16191 leader_election.cc:304] T b6f07842d3aa47858df5665874c86f3a P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
W20260709 13:22:27.815680 16115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 105391us
I20260709 13:22:27.765802 23988 raft_consensus.cc:2804] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 13:22:27.860751 23994 raft_consensus.cc:2804] T 9c61a39fd4f84424b129c1a6f9ba404d P 647a343a542d4a139fe106c19095129b [term 22 FOLLOWER]: Leader pre-election won for term 23
W20260709 13:22:27.843060 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 61887.4us
I20260709 13:22:27.885375 23287 raft_consensus.cc:2749] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 90f3da57796b488c9087a2f523adf045 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:22:27.889894 16823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdbd78acb4cf46298e0de8ba0d130e19" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 26 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:27.890997 16823 raft_consensus.cc:2410] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac for term 26 because replica has last-logged OpId of term: 24 index: 18, which is greater than that of the candidate, which has last-logged OpId of term: 22 index: 17.
W20260709 13:22:27.892343 16117 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:22:27.841840 23997 raft_consensus.cc:697] T 84f2da98864b4425bf7c0055951fdd4b P 647a343a542d4a139fe106c19095129b [term 19 LEADER]: Becoming Leader. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
W20260709 13:22:27.895298 24069 log.cc:927] Time spent T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.206s	user 0.001s	sys 0.000s
I20260709 13:22:27.896314 23997 consensus_queue.cc:237] T 84f2da98864b4425bf7c0055951fdd4b P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.10, Last appended by leader: 10, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:27.904217 16048 catalog_manager.cc:5697] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac reported cstate change: term changed from 21 to 24, leader changed from 647a343a542d4a139fe106c19095129b (127.14.177.129) to 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130). New cstate: current_term: 24 leader_uuid: "1ca2725d290e47d68b1a4b609c9816ac" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 13:22:27.906838 16263 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:22:27.906574 16040 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:58400 (request call id 194) took 26340 ms (client timeout 15000 ms). Trace:
I20260709 13:22:27.908080 23996 raft_consensus.cc:2804] T cdbd78acb4cf46298e0de8ba0d130e19 P 90f3da57796b488c9087a2f523adf045 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 13:22:27.909265 16293 raft_consensus.cc:1240] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045 [term 23 LEADER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 22. Current term is 23. Ops: []
I20260709 13:22:27.909416 16296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5ad4eb281564936a59f41e387a47156" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
W20260709 13:22:27.911553 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 73594.6us
I20260709 13:22:27.915818 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74ca8140f35c4ae29e8a664f6bdfc829" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 24 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:22:27.916927 16298 raft_consensus.cc:2410] T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac for term 24 because replica has last-logged OpId of term: 23 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 15.
W20260709 13:22:27.917670 16263 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:22:27.839344 23798 raft_consensus.cc:493] T ee9ed67006144797b1de6ea8c2013a68 P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:27.841616 24004 raft_consensus.cc:2804] T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:22:27.932246 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c2095a5f4ad49408dbc7f69c606120d" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:27.940523 16292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d16dfad0c3b540e5b4b86492ddf87208" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:22:27.906844 16048 catalog_manager.cc:5697] T e5ad4eb281564936a59f41e387a47156 P 1ca2725d290e47d68b1a4b609c9816ac reported cstate change: term changed from 16 to 17, leader changed from 647a343a542d4a139fe106c19095129b (127.14.177.129) to 90f3da57796b488c9087a2f523adf045 (127.14.177.131). New cstate: current_term: 17 leader_uuid: "90f3da57796b488c9087a2f523adf045" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } }
I20260709 13:22:27.941327 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806d7690c2b644959152074bb2cab0a7" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:27.887931 23988 raft_consensus.cc:697] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045 [term 25 LEADER]: Becoming Leader. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
W20260709 13:22:27.925766 16191 leader_election.cc:336] T 746aabf856ab401c9ef5e859371e7b67 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 19 election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:22:27.925410 16296 raft_consensus.cc:2468] T e5ad4eb281564936a59f41e387a47156 P 90f3da57796b488c9087a2f523adf045 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 21.
I20260709 13:22:27.947014 16789 leader_election.cc:304] T c9944d5e4b8c4109af9955c4f08c9095 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:22:27.799026 23719 raft_consensus.cc:515] T 5a959dbecc2648eb85d1cc550f5a3245 P 647a343a542d4a139fe106c19095129b [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:22:27.956564 16192 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:22:27.957348 16192 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:22:27.960402 16299 raft_consensus.cc:1240] T fffab26427e24d398e2695aad40ca134 P 90f3da57796b488c9087a2f523adf045 [term 21 LEADER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 20. Current term is 21. Ops: []
I20260709 13:22:27.887761 23994 raft_consensus.cc:493] T 9c61a39fd4f84424b129c1a6f9ba404d P 647a343a542d4a139fe106c19095129b [term 22 FOLLOWER]: Starting leader election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:22:27.930922 24004 raft_consensus.cc:493] T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:27.956939 16191 leader_election.cc:304] T 746aabf856ab401c9ef5e859371e7b67 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
I20260709 13:22:27.932583 23798 raft_consensus.cc:3060] T ee9ed67006144797b1de6ea8c2013a68 P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:22:27.941787 23996 raft_consensus.cc:697] T cdbd78acb4cf46298e0de8ba0d130e19 P 90f3da57796b488c9087a2f523adf045 [term 25 LEADER]: Becoming Leader. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:22:27.973975 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68a6838d82ef4cac83507f906139d90f" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:27.925797 23915 raft_consensus.cc:2749] T b6f07842d3aa47858df5665874c86f3a P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:22:27.953873 23815 raft_consensus.cc:2749] T 66ae176961b74813a58bc15c9c4c6f2a P 90f3da57796b488c9087a2f523adf045 [term 27 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 13:22:27.918704 23941 raft_consensus.cc:515] T d2ab0ac8347641e6955e9bce7d90c1a2 P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:27.955699 23894 raft_consensus.cc:493] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:22:28.021020 16117 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:22:27.940346 23975 consensus_queue.cc:1059] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.042s
I20260709 13:22:28.034849 16300 raft_consensus.cc:2468] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 90f3da57796b488c9087a2f523adf045 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 21.
I20260709 13:22:27.969537 24004 raft_consensus.cc:3060] T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:22:27.969734 23994 raft_consensus.cc:3060] T 9c61a39fd4f84424b129c1a6f9ba404d P 647a343a542d4a139fe106c19095129b [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:22:27.970343 23685 raft_consensus.cc:2804] T c9944d5e4b8c4109af9955c4f08c9095 P cb82cc7bd6db46d980c75237a1402860 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 13:22:28.051467 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ad5dbfa99ee4b8d9e28be92b3f36db5" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
W20260709 13:22:28.051397 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 278693us
I20260709 13:22:28.043329 23958 raft_consensus.cc:493] T 829b4ad8c5a54eababeea1181b960d4b P 1ca2725d290e47d68b1a4b609c9816ac [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:28.070216 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a378d7f796f94615b3b347119a9880fd" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:22:28.053244 23978 raft_consensus.cc:697] T 746aabf856ab401c9ef5e859371e7b67 P 647a343a542d4a139fe106c19095129b [term 19 LEADER]: Becoming Leader. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:22:28.056344 23980 raft_consensus.cc:493] T c0eab25ce4c64e018a041beeda2867db P 1ca2725d290e47d68b1a4b609c9816ac [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:28.056252 23915 raft_consensus.cc:2749] T 746aabf856ab401c9ef5e859371e7b67 P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:22:28.065809 16048 catalog_manager.cc:5697] T 02838c86531d4f98bf58d1c22b8baf77 P 1ca2725d290e47d68b1a4b609c9816ac reported cstate change: term changed from 13 to 18, leader changed from 90f3da57796b488c9087a2f523adf045 (127.14.177.131) to 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130). New cstate: current_term: 18 leader_uuid: "1ca2725d290e47d68b1a4b609c9816ac" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:28.075929 23980 raft_consensus.cc:515] T c0eab25ce4c64e018a041beeda2867db P 1ca2725d290e47d68b1a4b609c9816ac [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:28.078647 23980 leader_election.cc:290] T c0eab25ce4c64e018a041beeda2867db P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:28.078094 16048 catalog_manager.cc:5697] T 8a878999c0024d2abe517484180290fc P 1ca2725d290e47d68b1a4b609c9816ac reported cstate change: term changed from 14 to 15, leader changed from 647a343a542d4a139fe106c19095129b (127.14.177.129) to 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130). New cstate: current_term: 15 leader_uuid: "1ca2725d290e47d68b1a4b609c9816ac" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:28.079388 23605 consensus_queue.cc:1059] T 1df93779209748b9860f6d893dd3de53 P 647a343a542d4a139fe106c19095129b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: INVALID_TERM, Last received: 19.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:22:28.082311 23978 consensus_queue.cc:237] T 746aabf856ab401c9ef5e859371e7b67 P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.15, Last appended by leader: 15, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:22:28.080360 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 258758us
I20260709 13:22:28.085100 16265 leader_election.cc:304] T ada5a004ba7c46819051e3ccec71ffdb P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:22:28.085794 16116 leader_election.cc:304] T d16dfad0c3b540e5b4b86492ddf87208 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
W20260709 13:22:28.086210 16265 consensus_peers.cc:597] T e4dc6b0541ca431ebc757beac00521f1 P 90f3da57796b488c9087a2f523adf045 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:22:28.086515 24073 raft_consensus.cc:2804] T ada5a004ba7c46819051e3ccec71ffdb P 90f3da57796b488c9087a2f523adf045 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:22:28.086915 16048 catalog_manager.cc:5697] T 68a6838d82ef4cac83507f906139d90f P 1ca2725d290e47d68b1a4b609c9816ac reported cstate change: term changed from 15 to 17. New cstate: current_term: 17 leader_uuid: "647a343a542d4a139fe106c19095129b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } }
I20260709 13:22:28.087016 24073 raft_consensus.cc:493] T ada5a004ba7c46819051e3ccec71ffdb P 90f3da57796b488c9087a2f523adf045 [term 22 FOLLOWER]: Starting leader election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:22:28.088371 24073 raft_consensus.cc:3060] T ada5a004ba7c46819051e3ccec71ffdb P 90f3da57796b488c9087a2f523adf045 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:22:28.088635 16048 catalog_manager.cc:5697] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac reported cstate change: term changed from 17 to 19. New cstate: current_term: 19 leader_uuid: "cb82cc7bd6db46d980c75237a1402860" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
W20260709 13:22:28.092805 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 179123us
W20260709 13:22:28.087013 16265 leader_election.cc:336] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 20 election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:22:28.095840 16191 leader_election.cc:336] T ee9ed67006144797b1de6ea8c2013a68 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 20 election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:22:28.096344 16191 leader_election.cc:304] T ee9ed67006144797b1de6ea8c2013a68 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
I20260709 13:22:28.094244 23996 consensus_queue.cc:237] T cdbd78acb4cf46298e0de8ba0d130e19 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 24.18, Last appended by leader: 18, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:22:28.100900 16189 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:22:28.056730 23975 consensus_queue.cc:1243] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b [LEADER]: Peer 90f3da57796b488c9087a2f523adf045 log is divergent from this leader: its last log entry 23.13 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 12
I20260709 13:22:28.104251 23915 raft_consensus.cc:493] T 18ad7e3af5504571a69d10c00750a01c P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:28.058907 23785 consensus_queue.cc:1048] T 57a744f5aaf944678a63469cbf514015 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.040s
I20260709 13:22:28.109360 16295 raft_consensus.cc:1240] T c9944d5e4b8c4109af9955c4f08c9095 P 90f3da57796b488c9087a2f523adf045 [term 24 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 23. Current term is 24. Ops: []
I20260709 13:22:28.104223 16048 catalog_manager.cc:5697] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac reported cstate change: term changed from 15 to 17. New cstate: current_term: 17 leader_uuid: "cb82cc7bd6db46d980c75237a1402860" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:22:28.059650 23894 raft_consensus.cc:515] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:28.106559 23915 raft_consensus.cc:515] T 18ad7e3af5504571a69d10c00750a01c P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } }
W20260709 13:22:28.155536 16115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 254498us
I20260709 13:22:28.097533 23958 raft_consensus.cc:515] T 829b4ad8c5a54eababeea1181b960d4b P 1ca2725d290e47d68b1a4b609c9816ac [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:22:28.180397 16190 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:22:28.152281 23975 raft_consensus.cc:3055] T 1df93779209748b9860f6d893dd3de53 P 647a343a542d4a139fe106c19095129b [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:22:28.167557 23785 consensus_queue.cc:1243] T 57a744f5aaf944678a63469cbf514015 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Peer 90f3da57796b488c9087a2f523adf045 log is divergent from this leader: its last log entry 17.14 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 13
W20260709 13:22:28.185582 16789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 131208us
I20260709 13:22:28.167757 23941 leader_election.cc:290] T d2ab0ac8347641e6955e9bce7d90c1a2 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 21 election: Requested vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:28.183010 16297 raft_consensus.cc:1240] T 18ad7e3af5504571a69d10c00750a01c P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 17. Current term is 18. Ops: []
I20260709 13:22:28.110739 23980 raft_consensus.cc:2749] T ee9ed67006144797b1de6ea8c2013a68 P 1ca2725d290e47d68b1a4b609c9816ac [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
W20260709 13:22:28.190028 16790 leader_election.cc:336] T 8638ee363273426db1393dd81e5c7b72 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 21 election: RPC error from VoteRequest() call to peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943): Timed out: RequestConsensusVote RPC to 127.14.177.130:35943 timed out after 2.044s (SENT)
W20260709 13:22:28.193719 16789 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:22:28.098414 23988 consensus_queue.cc:237] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 23.13, Last appended by leader: 13, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:28.105357 23869 raft_consensus.cc:515] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:28.182299 23719 leader_election.cc:290] T 5a959dbecc2648eb85d1cc550f5a3245 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 21 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:28.185518 23975 raft_consensus.cc:740] T 1df93779209748b9860f6d893dd3de53 P 647a343a542d4a139fe106c19095129b [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
W20260709 13:22:28.198076 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 87036.2us
W20260709 13:22:28.194597 23945 consensus_meta.cc:287] T 66f75124215f4ca6bef831e174a792d8 P 1ca2725d290e47d68b1a4b609c9816ac: Time spent flushing consensus metadata: real 0.776s	user 0.008s	sys 0.005s
I20260709 13:22:28.195506 16117 leader_election.cc:304] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:22:28.186758 23685 raft_consensus.cc:697] T c9944d5e4b8c4109af9955c4f08c9095 P cb82cc7bd6db46d980c75237a1402860 [term 24 LEADER]: Becoming Leader. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:22:28.207787 24091 raft_consensus.cc:2749] T d16dfad0c3b540e5b4b86492ddf87208 P 647a343a542d4a139fe106c19095129b [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:22:28.209973 23915 leader_election.cc:290] T 18ad7e3af5504571a69d10c00750a01c P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:28.212225 23785 consensus_queue.cc:1048] T 23f82224f4564fea93572e8206bad1fe P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:22:28.213939 23975 consensus_queue.cc:260] T 1df93779209748b9860f6d893dd3de53 P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:28.215962 23941 raft_consensus.cc:493] T e4dc6b0541ca431ebc757beac00521f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:22:28.216866 23995 raft_consensus.cc:515] T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:22:28.217835 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 122045us
I20260709 13:22:28.218442 16816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ab0ac8347641e6955e9bce7d90c1a2" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:22:28.219920 23958 leader_election.cc:290] T 829b4ad8c5a54eababeea1181b960d4b P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:28.239566 16295 raft_consensus.cc:1240] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045 [term 21 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 20. Current term is 21. Ops: []
I20260709 13:22:28.239663 16816 raft_consensus.cc:3060] T d2ab0ac8347641e6955e9bce7d90c1a2 P cb82cc7bd6db46d980c75237a1402860 [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:22:27.914498 16040 rpcz_store.cc:269] 0709 13:22:01.565961 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:22:01.580463 (+ 14502us) service_pool.cc:224] Handling call
0709 13:22:27.906290 (+26325827us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:22:23.368152 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:22:23.368933 (+   781us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:22:23.369005 (+    72us) write_op.cc:435] Acquired schema lock
0709 13:22:23.369021 (+    16us) tablet.cc:662] Decoding operations
0709 13:22:23.533221 (+164200us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:22:23.533337 (+   116us) write_op.cc:641] Partition lock acquired for write op
0709 13:22:23.533370 (+    33us) tablet.cc:685] Acquiring locks for 21 operations
0709 13:22:23.614620 (+ 81250us) tablet.cc:701] Row locks acquired
0709 13:22:23.711418 (+ 96798us) write_op.cc:260] PREPARE: finished
0709 13:22:23.711702 (+   284us) write_op.cc:270] Start()
0709 13:22:23.711908 (+   206us) write_op.cc:276] Timestamp: P: 1783603343711619 usec, L: 0
0709 13:22:23.711956 (+    48us) op_driver.cc:348] REPLICATION: starting
0709 13:22:23.859061 (+147105us) log.cc:844] Serialized 8384 byte log entry
0709 13:22:24.256676 (+397615us) op_driver.cc:464] REPLICATION: finished
0709 13:22:24.410562 (+153886us) spinlock_profiling.cc:243] Waited 16.8 ms on lock 0x7b180010f0c0. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b693ade1 00007f69b69475b1 00007f69b07074bc 00007f69ada5548b 00007f69b68afab9 00007f69ada7169b 00007f69ada6f93f 00007f69ada6f455 00007f69b0adc190 00007f69b0ad5ae7 00007f69b0ad4dd9
0709 13:22:24.811784 (+401222us) write_op.cc:301] APPLY: starting
0709 13:22:24.811905 (+   121us) tablet.cc:1367] starting BulkCheckPresence
0709 13:22:25.963936 (+1152031us) tablet.cc:1370] finished BulkCheckPresence
0709 13:22:25.979308 (+ 15372us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:22:26.699283 (+719975us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:22:26.908124 (+208841us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=42,key_file_lookups=42,delta_file_lookups=0,mrs_lookups=0
0709 13:22:26.914851 (+  6727us) write_op.cc:312] APPLY: finished
0709 13:22:26.921172 (+  6321us) log.cc:844] Serialized 188 byte log entry
0709 13:22:26.983481 (+ 62309us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:22:26.983681 (+   200us) write_op.cc:454] Released schema lock
0709 13:22:26.983765 (+    84us) write_op.cc:341] FINISH: Updating metrics
0709 13:22:26.984238 (+   473us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:22:26.984267 (+    29us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":8569408,"reported_tablets":35,"child_traces":[["op",{"apply.queue_time_us":554752,"apply.run_cpu_time_us":49456,"apply.run_wall_time_us":2172419,"cfile_cache_hit":86,"cfile_cache_hit_bytes":91823,"num_ops":21,"prepare.queue_time_us":1067849,"prepare.run_cpu_time_us":21239,"prepare.run_wall_time_us":707351,"raft.queue_time_us":148189,"raft.run_cpu_time_us":48698,"raft.run_wall_time_us":467192,"replication_time_us":544542,"spinlock_wait_cycles":38692608,"thread_start_us":1519687,"threads_started":5,"wal-append.queue_time_us":216484,"wal-append.run_cpu_time_us":24401,"wal-append.run_wall_time_us":2448992}]]}
I20260709 13:22:28.220656 24073 raft_consensus.cc:515] T ada5a004ba7c46819051e3ccec71ffdb P 90f3da57796b488c9087a2f523adf045 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:28.226600 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8158bbd17b994d2eb96f560bd0f1b88d" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:28.242728 16298 raft_consensus.cc:2468] T 8158bbd17b994d2eb96f560bd0f1b88d P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 20.
I20260709 13:22:28.228711 16296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a959dbecc2648eb85d1cc550f5a3245" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:28.228958 16291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d4ab870a05347868956aa8b841b2981" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:28.240599 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cc959fb7f704fba9009c0595467d074" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:28.186112 23997 consensus_queue.cc:1048] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.040s
I20260709 13:22:28.233995 16117 leader_election.cc:304] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:22:28.256204 24091 raft_consensus.cc:2804] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 647a343a542d4a139fe106c19095129b [term 21 FOLLOWER]: Leader election won for term 21
W20260709 13:22:28.256524 16114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:22:28.224810 23869 leader_election.cc:290] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 23 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:28.242779 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a96373bb0e14dce8a266b2fc6c58507" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:28.267881 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02838c86531d4f98bf58d1c22b8baf77" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:28.268726 16298 raft_consensus.cc:2468] T 02838c86531d4f98bf58d1c22b8baf77 P 90f3da57796b488c9087a2f523adf045 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 22.
I20260709 13:22:28.252935 23975 raft_consensus.cc:3060] T 1df93779209748b9860f6d893dd3de53 P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:22:28.272375 16048 catalog_manager.cc:5697] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac reported cstate change: term changed from 15 to 19. New cstate: current_term: 19 leader_uuid: "1ca2725d290e47d68b1a4b609c9816ac" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:28.227207 24004 raft_consensus.cc:515] T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:22:28.274715 16265 leader_election.cc:336] T 1df93779209748b9860f6d893dd3de53 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:22:28.273965 16296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4f65cdf00b748bb963ee6e81ef43b79" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:28.271152 16295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8638ee363273426db1393dd81e5c7b72" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:22:28.260489 23719 raft_consensus.cc:2804] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b [term 26 FOLLOWER]: Leader election won for term 26
I20260709 13:22:28.276576 16295 raft_consensus.cc:3060] T 8638ee363273426db1393dd81e5c7b72 P 90f3da57796b488c9087a2f523adf045 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:22:28.277171 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdbd78acb4cf46298e0de8ba0d130e19" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 26 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:28.272866 16291 raft_consensus.cc:2468] T 8d4ab870a05347868956aa8b841b2981 P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 20.
I20260709 13:22:28.278496 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5ad4eb281564936a59f41e387a47156" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:28.282945 16298 raft_consensus.cc:2468] T e5ad4eb281564936a59f41e387a47156 P 90f3da57796b488c9087a2f523adf045 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 21.
W20260709 13:22:28.286134 16115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 87716.7us
I20260709 13:22:28.236488 23941 raft_consensus.cc:515] T e4dc6b0541ca431ebc757beac00521f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:22:28.287801 16191 leader_election.cc:336] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:22:28.293955 16191 leader_election.cc:304] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
I20260709 13:22:28.296010 16296 raft_consensus.cc:2468] T c4f65cdf00b748bb963ee6e81ef43b79 P 90f3da57796b488c9087a2f523adf045 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 22.
I20260709 13:22:28.297236 16296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ab0ac8347641e6955e9bce7d90c1a2" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:22:28.298369 23995 leader_election.cc:290] T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 20 election: Requested vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:28.298746 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0eab25ce4c64e018a041beeda2867db" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:28.299523 16293 raft_consensus.cc:2468] T c0eab25ce4c64e018a041beeda2867db P 90f3da57796b488c9087a2f523adf045 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 23.
I20260709 13:22:28.300022 23719 raft_consensus.cc:697] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b [term 26 LEADER]: Becoming Leader. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:22:28.300522 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74ca8140f35c4ae29e8a664f6bdfc829" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 24 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:22:28.301501 16299 raft_consensus.cc:2410] T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac for term 24 because replica has last-logged OpId of term: 23 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 15.
I20260709 13:22:28.301117 23719 consensus_queue.cc:237] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 23.16, Last appended by leader: 16, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:28.302450 23941 leader_election.cc:290] T e4dc6b0541ca431ebc757beac00521f1 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:28.303404 16048 catalog_manager.cc:5697] T 84f2da98864b4425bf7c0055951fdd4b P 1ca2725d290e47d68b1a4b609c9816ac reported cstate change: term changed from 12 to 14, leader changed from 90f3da57796b488c9087a2f523adf045 (127.14.177.131) to 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130). New cstate: current_term: 14 leader_uuid: "1ca2725d290e47d68b1a4b609c9816ac" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } health_report { overall_health: UNKNOWN } } }
W20260709 13:22:28.304853 23924 consensus_meta.cc:287] T af57996a8be84d398f0f023e8d08ef77 P 1ca2725d290e47d68b1a4b609c9816ac: Time spent flushing consensus metadata: real 0.723s	user 0.003s	sys 0.009s
I20260709 13:22:28.305327 23924 raft_consensus.cc:515] T af57996a8be84d398f0f023e8d08ef77 P 1ca2725d290e47d68b1a4b609c9816ac [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:28.305763 23994 raft_consensus.cc:515] T 9c61a39fd4f84424b129c1a6f9ba404d P 647a343a542d4a139fe106c19095129b [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:28.307281 16818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "829b4ad8c5a54eababeea1181b960d4b" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:28.307972 16818 raft_consensus.cc:2468] T 829b4ad8c5a54eababeea1181b960d4b P cb82cc7bd6db46d980c75237a1402860 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 19.
I20260709 13:22:28.308761 16294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ab0ac8347641e6955e9bce7d90c1a2" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:28.309300 16192 leader_election.cc:304] T 829b4ad8c5a54eababeea1181b960d4b P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 
W20260709 13:22:28.287674 16790 consensus_peers.cc:597] T 57a744f5aaf944678a63469cbf514015 P cb82cc7bd6db46d980c75237a1402860 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:22:28.310154 23958 raft_consensus.cc:2804] T 829b4ad8c5a54eababeea1181b960d4b P 1ca2725d290e47d68b1a4b609c9816ac [term 20 FOLLOWER]: Leader pre-election won for term 21
W20260709 13:22:28.310618 16790 leader_election.cc:336] T af57996a8be84d398f0f023e8d08ef77 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:22:28.310734 23958 raft_consensus.cc:493] T 829b4ad8c5a54eababeea1181b960d4b P 1ca2725d290e47d68b1a4b609c9816ac [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:28.292268 16823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e4bfb9090c8460eb14fa7e0d818e7f1" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
W20260709 13:22:28.311390 16790 leader_election.cc:336] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 25 election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:22:28.311497 16823 raft_consensus.cc:2468] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P cb82cc7bd6db46d980c75237a1402860 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 23.
I20260709 13:22:28.311861 16790 leader_election.cc:304] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:22:28.312498 23958 raft_consensus.cc:3060] T 829b4ad8c5a54eababeea1181b960d4b P 1ca2725d290e47d68b1a4b609c9816ac [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:22:28.312615 24004 leader_election.cc:290] T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 23 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:28.313477 24101 raft_consensus.cc:2749] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860 [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 13:22:28.315986 16192 leader_election.cc:304] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 
W20260709 13:22:28.316645 16117 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:22:28.317008 23798 raft_consensus.cc:515] T ee9ed67006144797b1de6ea8c2013a68 P 647a343a542d4a139fe106c19095129b [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:28.317813 16291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806d7690c2b644959152074bb2cab0a7" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
W20260709 13:22:28.318903 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 54026.4us
W20260709 13:22:28.296020 16191 leader_election.cc:336] T 19baa7bc367844c288bca8f81439c413 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 24 election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:22:28.292881 23685 consensus_queue.cc:237] T c9944d5e4b8c4109af9955c4f08c9095 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 23.17, Last appended by leader: 17, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:28.320080 16191 leader_election.cc:304] T 19baa7bc367844c288bca8f81439c413 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:22:28.321327 23924 leader_election.cc:290] T af57996a8be84d398f0f023e8d08ef77 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 22 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
W20260709 13:22:28.321874 16191 leader_election.cc:336] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:22:28.322003 23924 raft_consensus.cc:2749] T 19baa7bc367844c288bca8f81439c413 P 1ca2725d290e47d68b1a4b609c9816ac [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 13:22:28.322346 16191 leader_election.cc:304] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
I20260709 13:22:28.290663 23894 leader_election.cc:290] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
W20260709 13:22:28.322791 16190 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:22:28.310199 24096 raft_consensus.cc:493] T 829b4ad8c5a54eababeea1181b960d4b P cb82cc7bd6db46d980c75237a1402860 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:22:28.323223 23894 raft_consensus.cc:2749] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:22:28.319456 24097 raft_consensus.cc:2804] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:22:28.323498 24096 raft_consensus.cc:515] T 829b4ad8c5a54eababeea1181b960d4b P cb82cc7bd6db46d980c75237a1402860 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:28.324087 24097 raft_consensus.cc:493] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:28.324473 24097 raft_consensus.cc:3060] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 23 FOLLOWER]: Advancing to term 24
W20260709 13:22:28.324641 16190 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:22:28.290390 24073 leader_election.cc:290] T ada5a004ba7c46819051e3ccec71ffdb P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 23 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
W20260709 13:22:28.313064 16790 consensus_peers.cc:597] T 23f82224f4564fea93572e8206bad1fe P cb82cc7bd6db46d980c75237a1402860 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:22:28.326067 24096 leader_election.cc:290] T 829b4ad8c5a54eababeea1181b960d4b P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
W20260709 13:22:28.319772 16265 leader_election.cc:336] T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 23 election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:22:28.328336 16823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af57996a8be84d398f0f023e8d08ef77" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:22:28.326833 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d16dfad0c3b540e5b4b86492ddf87208" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:22:28.305495 16048 catalog_manager.cc:5697] T 5a959dbecc2648eb85d1cc550f5a3245 P 1ca2725d290e47d68b1a4b609c9816ac reported cstate change: term changed from 14 to 15, leader changed from 647a343a542d4a139fe106c19095129b (127.14.177.129) to 90f3da57796b488c9087a2f523adf045 (127.14.177.131). New cstate: current_term: 15 leader_uuid: "90f3da57796b488c9087a2f523adf045" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } }
W20260709 13:22:28.290380 23938 consensus_meta.cc:287] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac: Time spent flushing consensus metadata: real 0.532s	user 0.009s	sys 0.001s
I20260709 13:22:28.303035 23941 raft_consensus.cc:2749] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:22:28.336728 23798 leader_election.cc:290] T ee9ed67006144797b1de6ea8c2013a68 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 21 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
W20260709 13:22:28.340715 16191 leader_election.cc:336] T 84f2da98864b4425bf7c0055951fdd4b P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:22:28.334805 16190 leader_election.cc:304] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860
I20260709 13:22:28.359305 16265 leader_election.cc:304] T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
W20260709 13:22:28.326814 16263 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:22:28.360823 16265 leader_election.cc:336] T 8cc959fb7f704fba9009c0595467d074 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 20 election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:22:28.361063 23941 raft_consensus.cc:2749] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:22:28.362195 16191 leader_election.cc:304] T 84f2da98864b4425bf7c0055951fdd4b P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:22:28.358572 16823 raft_consensus.cc:3060] T af57996a8be84d398f0f023e8d08ef77 P cb82cc7bd6db46d980c75237a1402860 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:22:28.365206 16816 raft_consensus.cc:2468] T d2ab0ac8347641e6955e9bce7d90c1a2 P cb82cc7bd6db46d980c75237a1402860 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 21.
W20260709 13:22:28.348486 16790 leader_election.cc:336] T ee9ed67006144797b1de6ea8c2013a68 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:22:28.362995 24073 raft_consensus.cc:2749] T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 13:22:28.366526 16192 leader_election.cc:304] T d2ab0ac8347641e6955e9bce7d90c1a2 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:22:28.330384 16292 raft_consensus.cc:2468] T d16dfad0c3b540e5b4b86492ddf87208 P 90f3da57796b488c9087a2f523adf045 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 22.
I20260709 13:22:28.328521 24088 raft_consensus.cc:3055] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b [term 24 LEADER]: Stepping down as leader of term 24
I20260709 13:22:28.367551 24088 raft_consensus.cc:740] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:22:28.368647 23915 raft_consensus.cc:2749] T 84f2da98864b4425bf7c0055951fdd4b P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:22:28.368487 24088 consensus_queue.cc:260] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 24.14, Last appended by leader: 14, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:28.369427 23915 raft_consensus.cc:2804] T d2ab0ac8347641e6955e9bce7d90c1a2 P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Leader election won for term 21
W20260709 13:22:28.362481 16265 leader_election.cc:336] T 31969392ccdb43b491911a319e8bc7e1 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:22:28.369748 24088 raft_consensus.cc:3060] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:22:28.370226 23938 raft_consensus.cc:493] T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:28.370409 23894 raft_consensus.cc:493] T fffab26427e24d398e2695aad40ca134 P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:22:28.370832 23938 raft_consensus.cc:515] T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:28.370954 23894 raft_consensus.cc:515] T fffab26427e24d398e2695aad40ca134 P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:28.372808 23638 consensus_queue.cc:1059] T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: INVALID_TERM, Last received: 21.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:22:28.369920 23915 raft_consensus.cc:697] T d2ab0ac8347641e6955e9bce7d90c1a2 P 1ca2725d290e47d68b1a4b609c9816ac [term 21 LEADER]: Becoming Leader. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:22:28.374177 23638 raft_consensus.cc:3055] T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:22:28.374544 23915 consensus_queue.cc:237] T d2ab0ac8347641e6955e9bce7d90c1a2 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 20.16, Last appended by leader: 16, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:28.375633 23785 consensus_queue.cc:1059] T 8638ee363273426db1393dd81e5c7b72 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: INVALID_TERM, Last received: 20.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 13:22:28.376001 16265 leader_election.cc:304] T 31969392ccdb43b491911a319e8bc7e1 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
I20260709 13:22:28.376569 23785 raft_consensus.cc:3055] T 8638ee363273426db1393dd81e5c7b72 P cb82cc7bd6db46d980c75237a1402860 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:22:28.376938 24073 raft_consensus.cc:2749] T 31969392ccdb43b491911a319e8bc7e1 P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:22:28.377002 23785 raft_consensus.cc:740] T 8638ee363273426db1393dd81e5c7b72 P cb82cc7bd6db46d980c75237a1402860 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
W20260709 13:22:28.377458 16265 leader_election.cc:336] T c4f65cdf00b748bb963ee6e81ef43b79 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:22:28.377966 16265 leader_election.cc:304] T c4f65cdf00b748bb963ee6e81ef43b79 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:22:28.374560 23638 raft_consensus.cc:740] T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:22:28.378242 23785 consensus_queue.cc:260] T 8638ee363273426db1393dd81e5c7b72 P cb82cc7bd6db46d980c75237a1402860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 21.17, Last appended by leader: 17, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:28.379487 23638 consensus_queue.cc:260] T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 21.14, Last appended by leader: 14, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:28.335965 23994 leader_election.cc:290] T 9c61a39fd4f84424b129c1a6f9ba404d P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 23 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:28.373103 23938 leader_election.cc:290] T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:28.382465 16823 raft_consensus.cc:2468] T af57996a8be84d398f0f023e8d08ef77 P cb82cc7bd6db46d980c75237a1402860 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 22.
I20260709 13:22:28.413054 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c2095a5f4ad49408dbc7f69c606120d" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:28.413980 16192 leader_election.cc:304] T af57996a8be84d398f0f023e8d08ef77 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 
W20260709 13:22:28.381537 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 295858us
I20260709 13:22:28.380158 23785 raft_consensus.cc:3060] T 8638ee363273426db1393dd81e5c7b72 P cb82cc7bd6db46d980c75237a1402860 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:22:28.383391 24073 raft_consensus.cc:2749] T c4f65cdf00b748bb963ee6e81ef43b79 P 90f3da57796b488c9087a2f523adf045 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:22:28.364450 16190 leader_election.cc:304] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
W20260709 13:22:28.399269 16265 leader_election.cc:336] T 8a878999c0024d2abe517484180290fc P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 19 election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:22:28.402256 16818 raft_consensus.cc:1275] T 806d7690c2b644959152074bb2cab0a7 P cb82cc7bd6db46d980c75237a1402860 [term 21 LEARNER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 21 index: 14. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
I20260709 13:22:28.403918 16292 pending_rounds.cc:85] T 57a744f5aaf944678a63469cbf514015 P 90f3da57796b488c9087a2f523adf045: Aborting all ops after (but not including) 13
I20260709 13:22:28.407351 16819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5718c623ebee4d2bb770f0f9ddb2f164" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 23 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:22:28.408140 16821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31969392ccdb43b491911a319e8bc7e1" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:22:28.438213 16821 raft_consensus.cc:3060] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:22:28.408753 16822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee9ed67006144797b1de6ea8c2013a68" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:22:28.438962 16822 raft_consensus.cc:3060] T ee9ed67006144797b1de6ea8c2013a68 P cb82cc7bd6db46d980c75237a1402860 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:22:28.410180 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806d7690c2b644959152074bb2cab0a7" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:28.410910 16294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18ad7e3af5504571a69d10c00750a01c" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:28.411365 16115 leader_election.cc:304] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860
I20260709 13:22:28.410187 16817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a378d7f796f94615b3b347119a9880fd" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:28.412633 16824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2794af5b99984efebdee564035841cb0" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 21 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:28.415233 16820 raft_consensus.cc:1240] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860 [term 26 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 25. Current term is 26. Ops: [25.17-25.17]
I20260709 13:22:28.422391 24091 raft_consensus.cc:697] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 647a343a542d4a139fe106c19095129b [term 21 LEADER]: Becoming Leader. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:22:28.382650 23894 leader_election.cc:290] T fffab26427e24d398e2695aad40ca134 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:28.450045 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4dc6b0541ca431ebc757beac00521f1" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:28.446861 16823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada5a004ba7c46819051e3ccec71ffdb" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:28.447711 16291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e4bfb9090c8460eb14fa7e0d818e7f1" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:28.435628 16825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c11074579b046d99e585c2818e18a2c" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:28.453299 16816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3884f864a96d428785895e5463e5728f" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 18 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
W20260709 13:22:28.454321 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 90009.7us
I20260709 13:22:28.451932 16294 raft_consensus.cc:2468] T 18ad7e3af5504571a69d10c00750a01c P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 18.
I20260709 13:22:28.437515 16292 pending_rounds.cc:107] T 57a744f5aaf944678a63469cbf514015 P 90f3da57796b488c9087a2f523adf045: Aborting uncommitted NO_OP operation due to leader change: 17.14
I20260709 13:22:28.456764 16292 raft_consensus.cc:2889] T 57a744f5aaf944678a63469cbf514015 P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:22:28.457983 16115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 74887us
I20260709 13:22:28.437248 23938 raft_consensus.cc:2804] T af57996a8be84d398f0f023e8d08ef77 P 1ca2725d290e47d68b1a4b609c9816ac [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:22:28.459993 16819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57a744f5aaf944678a63469cbf514015" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 21 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:28.467608 16824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26354361e9b044cea1c2b86c4882fe92" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:22:28.469365 16295 raft_consensus.cc:2468] T 8638ee363273426db1393dd81e5c7b72 P 90f3da57796b488c9087a2f523adf045 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 22.
I20260709 13:22:28.470096 16298 raft_consensus.cc:1240] T 8638ee363273426db1393dd81e5c7b72 P 90f3da57796b488c9087a2f523adf045 [term 22 FOLLOWER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 21. Current term is 22. Ops: [21.17-21.17]
I20260709 13:22:28.470985 16823 raft_consensus.cc:2393] T ada5a004ba7c46819051e3ccec71ffdb P cb82cc7bd6db46d980c75237a1402860 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 23: Already voted for candidate cb82cc7bd6db46d980c75237a1402860 in this term.
I20260709 13:22:28.475241 16790 leader_election.cc:304] T 74ca8140f35c4ae29e8a664f6bdfc829 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:22:28.478991 16821 raft_consensus.cc:2468] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 20.
W20260709 13:22:28.479698 16117 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:22:28.482487 16824 raft_consensus.cc:3060] T 26354361e9b044cea1c2b86c4882fe92 P cb82cc7bd6db46d980c75237a1402860 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:22:28.473181 23994 raft_consensus.cc:3060] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:22:28.501585 16223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee9ed67006144797b1de6ea8c2013a68" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:28.506781 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ab0ac8347641e6955e9bce7d90c1a2" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:28.508316 16220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
W20260709 13:22:28.510210 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 131029us
I20260709 13:22:28.481230 23941 raft_consensus.cc:493] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 1ca2725d290e47d68b1a4b609c9816ac [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:28.473748 23894 raft_consensus.cc:2804] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:22:28.507614 16221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:22:28.475610 23938 raft_consensus.cc:697] T af57996a8be84d398f0f023e8d08ef77 P 1ca2725d290e47d68b1a4b609c9816ac [term 22 LEADER]: Becoming Leader. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:22:28.514045 16117 leader_election.cc:304] T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:22:28.516649 16220 raft_consensus.cc:2393] T 19baa7bc367844c288bca8f81439c413 P 1ca2725d290e47d68b1a4b609c9816ac [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 in current term 24: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
I20260709 13:22:28.520205 16821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada5a004ba7c46819051e3ccec71ffdb" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:22:28.522830 16219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "829b4ad8c5a54eababeea1181b960d4b" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:28.516129 16223 raft_consensus.cc:2468] T ee9ed67006144797b1de6ea8c2013a68 P 1ca2725d290e47d68b1a4b609c9816ac [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 20.
I20260709 13:22:28.530437 16821 raft_consensus.cc:2393] T ada5a004ba7c46819051e3ccec71ffdb P cb82cc7bd6db46d980c75237a1402860 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 23: Already voted for candidate cb82cc7bd6db46d980c75237a1402860 in this term.
I20260709 13:22:28.523113 16222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9944d5e4b8c4109af9955c4f08c9095" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:22:28.492316 23924 raft_consensus.cc:493] T 1df93779209748b9860f6d893dd3de53 P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
W20260709 13:22:28.534823 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 98497us
I20260709 13:22:28.543149 16221 raft_consensus.cc:3060] T ecc1e363230043fe9fef390f0b979f20 P 1ca2725d290e47d68b1a4b609c9816ac [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:22:28.523554 23894 raft_consensus.cc:493] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:28.525604 23685 raft_consensus.cc:3060] T 74ca8140f35c4ae29e8a664f6bdfc829 P cb82cc7bd6db46d980c75237a1402860 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:22:28.529821 23798 raft_consensus.cc:2804] T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:22:28.533382 23945 raft_consensus.cc:493] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:22:28.572535 16265 leader_election.cc:336] T e5ad4eb281564936a59f41e387a47156 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 21 election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:22:28.537895 23638 raft_consensus.cc:3060] T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:22:28.575675 16265 leader_election.cc:304] T e5ad4eb281564936a59f41e387a47156 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:22:28.576210 16218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af57996a8be84d398f0f023e8d08ef77" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:28.576247 23988 raft_consensus.cc:493] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:22:28.532799 23915 raft_consensus.cc:493] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:28.542832 24073 consensus_queue.cc:1059] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: INVALID_TERM, Last received: 23.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.036s
I20260709 13:22:28.540225 23941 raft_consensus.cc:515] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 1ca2725d290e47d68b1a4b609c9816ac [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:28.578792 23915 raft_consensus.cc:515] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:28.580878 16115 leader_election.cc:304] T 434bff3082d141838068f323bdb0d1ed P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860
W20260709 13:22:28.506072 16036 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet ecc1e363230043fe9fef390f0b979f20 with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER failed with leader cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica cb82cc7bd6db46d980c75237a1402860 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 25 committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
W20260709 13:22:28.579873 16265 leader_election.cc:336] T 8158bbd17b994d2eb96f560bd0f1b88d P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 20 election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:22:28.553303 16822 raft_consensus.cc:2468] T ee9ed67006144797b1de6ea8c2013a68 P cb82cc7bd6db46d980c75237a1402860 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 21.
I20260709 13:22:28.525233 24088 consensus_queue.cc:1048] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.032s
I20260709 13:22:28.579355 23287 raft_consensus.cc:2749] T e5ad4eb281564936a59f41e387a47156 P 90f3da57796b488c9087a2f523adf045 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:22:28.586848 16117 leader_election.cc:304] T ee9ed67006144797b1de6ea8c2013a68 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860; no voters: 
W20260709 13:22:28.587126 16115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 46188.5us
I20260709 13:22:28.587730 23719 raft_consensus.cc:2749] T 434bff3082d141838068f323bdb0d1ed P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:22:28.587832 23997 raft_consensus.cc:2804] T ee9ed67006144797b1de6ea8c2013a68 P 647a343a542d4a139fe106c19095129b [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:22:28.588037 16115 leader_election.cc:304] T 26354361e9b044cea1c2b86c4882fe92 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: cb82cc7bd6db46d980c75237a1402860
I20260709 13:22:28.588310 23915 leader_election.cc:290] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:28.588510 23997 raft_consensus.cc:697] T ee9ed67006144797b1de6ea8c2013a68 P 647a343a542d4a139fe106c19095129b [term 21 LEADER]: Becoming Leader. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:22:28.589533 23997 consensus_queue.cc:237] T ee9ed67006144797b1de6ea8c2013a68 P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:28.591355 23975 consensus_queue.cc:1048] T 5a96373bb0e14dce8a266b2fc6c58507 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:22:28.596077 23988 raft_consensus.cc:515] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } }
I20260709 13:22:28.598582 23988 leader_election.cc:290] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:22:28.547019 24091 consensus_queue.cc:237] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:28.600924 23938 consensus_queue.cc:237] T af57996a8be84d398f0f023e8d08ef77 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.13, Last appended by leader: 13, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:28.582336 23798 raft_consensus.cc:697] T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b [term 20 LEADER]: Becoming Leader. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:22:28.575052 23894 raft_consensus.cc:3060] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:22:28.585129 16036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.ChangeConfig -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 83757.4us
I20260709 13:22:28.609239 16265 leader_election.cc:304] T 8158bbd17b994d2eb96f560bd0f1b88d P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:22:28.548652 23924 raft_consensus.cc:515] T 1df93779209748b9860f6d893dd3de53 P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:22:28.583952 24102 log.cc:927] Time spent T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.054s	user 0.000s	sys 0.001s
I20260709 13:22:28.616369 16222 raft_consensus.cc:2468] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 24.
I20260709 13:22:28.619841 16824 raft_consensus.cc:2468] T 26354361e9b044cea1c2b86c4882fe92 P cb82cc7bd6db46d980c75237a1402860 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 19.
I20260709 13:22:28.548938 23980 consensus_queue.cc:1059] T 08c74490c40243a88e62d1760b5cd814 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: INVALID_TERM, Last received: 20.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.061s
I20260709 13:22:28.630980 23815 raft_consensus.cc:3055] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [term 25 LEADER]: Stepping down as leader of term 25
I20260709 13:22:28.646770 23815 raft_consensus.cc:740] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [term 25 LEADER]: Becoming Follower/Learner. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:22:28.647272 23980 raft_consensus.cc:3055] T 08c74490c40243a88e62d1760b5cd814 P 1ca2725d290e47d68b1a4b609c9816ac [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:22:28.647459 16822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2372981006d4b218215c28964dea623" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 27 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:28.647775 23980 raft_consensus.cc:740] T 08c74490c40243a88e62d1760b5cd814 P 1ca2725d290e47d68b1a4b609c9816ac [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:22:28.648164 16822 raft_consensus.cc:2468] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 26.
I20260709 13:22:28.648710 23980 consensus_queue.cc:260] T 08c74490c40243a88e62d1760b5cd814 P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.12, Last appended by leader: 12, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:28.632942 23719 raft_consensus.cc:2804] T 26354361e9b044cea1c2b86c4882fe92 P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:22:28.655654 23719 raft_consensus.cc:493] T 26354361e9b044cea1c2b86c4882fe92 P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Starting leader election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:22:28.656212 23719 raft_consensus.cc:3060] T 26354361e9b044cea1c2b86c4882fe92 P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:22:28.635638 24108 log.cc:927] Time spent T c9944d5e4b8c4109af9955c4f08c9095 P cb82cc7bd6db46d980c75237a1402860: Append to log took a long time: real 0.051s	user 0.002s	sys 0.000s
I20260709 13:22:28.636534 23941 leader_election.cc:290] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:28.619737 23988 raft_consensus.cc:493] T 9e65676b21b748728ad39668cfe722ec P 90f3da57796b488c9087a2f523adf045 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:22:28.639297 23865 raft_consensus.cc:2749] T 8158bbd17b994d2eb96f560bd0f1b88d P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:22:28.661954 23988 raft_consensus.cc:515] T 9e65676b21b748728ad39668cfe722ec P 90f3da57796b488c9087a2f523adf045 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:28.664888 23988 leader_election.cc:290] T 9e65676b21b748728ad39668cfe722ec P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:28.643637 16116 leader_election.cc:304] T 18ad7e3af5504571a69d10c00750a01c P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:22:28.644726 16823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7ca61592e848ea9b0fb81a2ce33cdc" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:28.648319 23815 consensus_queue.cc:260] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 25.17, Last appended by leader: 17, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:22:28.631533 16115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 43594.9us
W20260709 13:22:28.640687 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 60857.6us
I20260709 13:22:28.622242 16266 leader_election.cc:304] T 26354361e9b044cea1c2b86c4882fe92 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:22:28.653431 16192 leader_election.cc:304] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 
W20260709 13:22:28.620883 24107 log.cc:927] Time spent T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 0.083s	user 0.001s	sys 0.000s
I20260709 13:22:28.677851 16115 leader_election.cc:304] T 23f82224f4564fea93572e8206bad1fe P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860
I20260709 13:22:28.659206 23980 raft_consensus.cc:3060] T 08c74490c40243a88e62d1760b5cd814 P 1ca2725d290e47d68b1a4b609c9816ac [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:22:28.641728 23924 leader_election.cc:290] T 1df93779209748b9860f6d893dd3de53 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:28.666814 23994 raft_consensus.cc:2749] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 13:22:28.652108 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 201001us
I20260709 13:22:28.669529 16221 raft_consensus.cc:2468] T ecc1e363230043fe9fef390f0b979f20 P 1ca2725d290e47d68b1a4b609c9816ac [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 25.
I20260709 13:22:28.577682 23945 raft_consensus.cc:515] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:28.670933 23798 consensus_queue.cc:237] T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 18.15, Last appended by leader: 15, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:22:28.677802 24110 log.cc:927] Time spent T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860: Append to log took a long time: real 0.057s	user 0.000s	sys 0.001s
W20260709 13:22:28.678604 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 57852.2us
I20260709 13:22:28.679189 24088 raft_consensus.cc:3060] T 18ad7e3af5504571a69d10c00750a01c P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:22:28.704288 16823 raft_consensus.cc:2468] T 3a7ca61592e848ea9b0fb81a2ce33cdc P cb82cc7bd6db46d980c75237a1402860 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 22.
I20260709 13:22:28.683616 23941 raft_consensus.cc:2804] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [term 26 FOLLOWER]: Leader pre-election won for term 27
W20260709 13:22:28.679451 16266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 57339us
W20260709 13:22:28.707599 16265 leader_election.cc:336] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 20 election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:22:28.708395 16265 leader_election.cc:304] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
I20260709 13:22:28.681408 23865 raft_consensus.cc:493] T 84f2da98864b4425bf7c0055951fdd4b P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:22:28.716579 16266 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:22:28.713032 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 174150us
I20260709 13:22:28.684293 23988 raft_consensus.cc:2804] T 26354361e9b044cea1c2b86c4882fe92 P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:22:28.733669 23988 raft_consensus.cc:697] T 26354361e9b044cea1c2b86c4882fe92 P 90f3da57796b488c9087a2f523adf045 [term 19 LEADER]: Becoming Leader. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:22:28.734745 23988 consensus_queue.cc:237] T 26354361e9b044cea1c2b86c4882fe92 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:28.713932 16116 leader_election.cc:304] T fffab26427e24d398e2695aad40ca134 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 90f3da57796b488c9087a2f523adf045
I20260709 13:22:28.710273 23941 raft_consensus.cc:493] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:28.737215 23941 raft_consensus.cc:3060] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [term 26 FOLLOWER]: Advancing to term 27
I20260709 13:22:28.738282 16116 leader_election.cc:304] T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: cb82cc7bd6db46d980c75237a1402860
I20260709 13:22:28.715785 24097 raft_consensus.cc:515] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:28.741264 16116 leader_election.cc:304] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:22:28.715777 16192 leader_election.cc:304] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:22:28.741644 24097 leader_election.cc:290] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 24 election: Requested vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:28.742496 24097 raft_consensus.cc:2804] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 1ca2725d290e47d68b1a4b609c9816ac [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:22:28.684530 23975 raft_consensus.cc:2749] T 23f82224f4564fea93572e8206bad1fe P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
W20260709 13:22:28.743058 16191 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:22:28.743724 16191 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:22:28.719388 23685 raft_consensus.cc:2749] T 74ca8140f35c4ae29e8a664f6bdfc829 P cb82cc7bd6db46d980c75237a1402860 [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:22:28.744633 23924 raft_consensus.cc:493] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [term 26 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:22:28.745286 23924 raft_consensus.cc:515] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:28.746028 23815 raft_consensus.cc:3060] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:22:28.719560 23958 raft_consensus.cc:515] T 829b4ad8c5a54eababeea1181b960d4b P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:28.725337 23945 leader_election.cc:290] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:28.748062 16825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e4bfb9090c8460eb14fa7e0d818e7f1" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
W20260709 13:22:28.729285 16117 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:22:28.732795 16819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e65676b21b748728ad39668cfe722ec" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:28.741420 23994 raft_consensus.cc:2804] T fffab26427e24d398e2695aad40ca134 P 647a343a542d4a139fe106c19095129b [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:22:28.723326 24103 raft_consensus.cc:493] T b87147731a68400e8cce9b879daa7c5d P cb82cc7bd6db46d980c75237a1402860 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:22:28.707433 24104 raft_consensus.cc:493] T 8cc959fb7f704fba9009c0595467d074 P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:28.747843 16823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66ae176961b74813a58bc15c9c4c6f2a" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 28 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:28.754576 23994 raft_consensus.cc:493] T fffab26427e24d398e2695aad40ca134 P 647a343a542d4a139fe106c19095129b [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:28.756335 16825 raft_consensus.cc:3060] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P cb82cc7bd6db46d980c75237a1402860 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:22:28.743001 24097 raft_consensus.cc:493] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 1ca2725d290e47d68b1a4b609c9816ac [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:28.763474 16190 leader_election.cc:304] T 02838c86531d4f98bf58d1c22b8baf77 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:22:28.743882 23975 raft_consensus.cc:2804] T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:22:28.716167 23996 raft_consensus.cc:2749] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 90f3da57796b488c9087a2f523adf045 [term 21 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:22:28.771440 16823 raft_consensus.cc:2468] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 27.
I20260709 13:22:28.771637 16790 leader_election.cc:304] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 90f3da57796b488c9087a2f523adf045
I20260709 13:22:28.799568 24120 raft_consensus.cc:2804] T 02838c86531d4f98bf58d1c22b8baf77 P 1ca2725d290e47d68b1a4b609c9816ac [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:22:28.800130 24120 raft_consensus.cc:493] T 02838c86531d4f98bf58d1c22b8baf77 P 1ca2725d290e47d68b1a4b609c9816ac [term 22 FOLLOWER]: Starting leader election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:22:28.800537 24120 raft_consensus.cc:3060] T 02838c86531d4f98bf58d1c22b8baf77 P 1ca2725d290e47d68b1a4b609c9816ac [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:22:28.769665 24097 raft_consensus.cc:3060] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 1ca2725d290e47d68b1a4b609c9816ac [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:22:28.767576 23605 raft_consensus.cc:2749] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 647a343a542d4a139fe106c19095129b [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:22:28.804131 16115 leader_election.cc:304] T 8158bbd17b994d2eb96f560bd0f1b88d P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:22:28.805207 16115 leader_election.cc:304] T 2794af5b99984efebdee564035841cb0 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860
I20260709 13:22:28.806991 24004 raft_consensus.cc:2804] T 8158bbd17b994d2eb96f560bd0f1b88d P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:22:28.807556 24004 raft_consensus.cc:493] T 8158bbd17b994d2eb96f560bd0f1b88d P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:22:28.811375 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 52363.3us
W20260709 13:22:28.812469 16265 consensus_peers.cc:597] T 5a959dbecc2648eb85d1cc550f5a3245 P 90f3da57796b488c9087a2f523adf045 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:22:28.813155 16265 consensus_peers.cc:597] T 434bff3082d141838068f323bdb0d1ed P 90f3da57796b488c9087a2f523adf045 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:22:28.813391 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a959dbecc2648eb85d1cc550f5a3245" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
W20260709 13:22:28.813781 16265 consensus_peers.cc:597] T 68a6838d82ef4cac83507f906139d90f P 90f3da57796b488c9087a2f523adf045 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:22:28.814455 16265 consensus_peers.cc:597] T b87147731a68400e8cce9b879daa7c5d P 90f3da57796b488c9087a2f523adf045 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:22:28.817543 24004 raft_consensus.cc:3060] T 8158bbd17b994d2eb96f560bd0f1b88d P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:22:28.817685 23894 raft_consensus.cc:515] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:28.755885 23865 raft_consensus.cc:515] T 84f2da98864b4425bf7c0055951fdd4b P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:28.820878 23865 leader_election.cc:290] T 84f2da98864b4425bf7c0055951fdd4b P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:22:28.775596 23975 raft_consensus.cc:493] T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Starting leader election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:22:28.792184 16192 leader_election.cc:304] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:22:28.821933 23975 raft_consensus.cc:3060] T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:22:28.791428 16219 raft_consensus.cc:2368] T 829b4ad8c5a54eababeea1181b960d4b P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 for earlier term 20. Current term is 21.
W20260709 13:22:28.815078 16265 leader_election.cc:336] T c0eab25ce4c64e018a041beeda2867db P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 23 election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:22:28.760167 23994 raft_consensus.cc:3060] T fffab26427e24d398e2695aad40ca134 P 647a343a542d4a139fe106c19095129b [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:22:28.818235 23785 raft_consensus.cc:2804] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [term 27 FOLLOWER]: Leader election won for term 27
I20260709 13:22:28.754787 24103 raft_consensus.cc:515] T b87147731a68400e8cce9b879daa7c5d P cb82cc7bd6db46d980c75237a1402860 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:28.824517 16190 leader_election.cc:304] T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:22:28.826197 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c61a39fd4f84424b129c1a6f9ba404d" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
W20260709 13:22:28.829597 16115 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:22:28.807431 24091 raft_consensus.cc:2749] T 2794af5b99984efebdee564035841cb0 P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:22:28.840824 16265 leader_election.cc:304] T c0eab25ce4c64e018a041beeda2867db P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:22:28.849812 16300 raft_consensus.cc:2393] T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 23: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
W20260709 13:22:28.842092 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 100934us
I20260709 13:22:28.851511 16294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4dc6b0541ca431ebc757beac00521f1" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 23 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:22:28.861611 16293 raft_consensus.cc:1275] T 84f2da98864b4425bf7c0055951fdd4b P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 13:22:28.755255 24104 raft_consensus.cc:515] T 8cc959fb7f704fba9009c0595467d074 P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:28.844100 23980 raft_consensus.cc:2804] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [term 27 FOLLOWER]: Leader pre-election won for term 28
W20260709 13:22:28.883598 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 68576.7us
W20260709 13:22:28.884567 16265 leader_election.cc:336] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:22:28.885059 16265 leader_election.cc:304] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:22:28.885947 24085 raft_consensus.cc:2749] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:22:28.886511 16265 leader_election.cc:336] T ecc1e363230043fe9fef390f0b979f20 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:22:28.887277 23865 raft_consensus.cc:2749] T c0eab25ce4c64e018a041beeda2867db P 90f3da57796b488c9087a2f523adf045 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 13:22:28.896602 23958 leader_election.cc:290] T 829b4ad8c5a54eababeea1181b960d4b P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 21 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:28.898061 23958 raft_consensus.cc:2804] T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:22:28.898505 23958 raft_consensus.cc:493] T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac [term 20 FOLLOWER]: Starting leader election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:22:28.898910 23958 raft_consensus.cc:3060] T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:22:28.900336 23980 raft_consensus.cc:493] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [term 27 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:22:28.904259 24119 log.cc:927] Time spent T 00000000000000000000000000000000 P 667a34b27a39474ebefc1af3cf5b4271: Append to log took a long time: real 0.180s	user 0.004s	sys 0.000s
I20260709 13:22:28.860383 23785 raft_consensus.cc:697] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [term 27 LEADER]: Becoming Leader. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
W20260709 13:22:28.949720 16116 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:22:28.947430 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 60955.7us
I20260709 13:22:28.948657 16825 raft_consensus.cc:2468] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P cb82cc7bd6db46d980c75237a1402860 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 24.
I20260709 13:22:28.950747 16817 raft_consensus.cc:2468] T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 22.
I20260709 13:22:28.950170 23798 raft_consensus.cc:493] T e5ad4eb281564936a59f41e387a47156 P 647a343a542d4a139fe106c19095129b [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:28.928278 23980 raft_consensus.cc:3060] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [term 27 FOLLOWER]: Advancing to term 28
I20260709 13:22:28.953192 23894 leader_election.cc:290] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 21 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:28.919780 24004 raft_consensus.cc:515] T 8158bbd17b994d2eb96f560bd0f1b88d P 647a343a542d4a139fe106c19095129b [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:28.890848 23719 raft_consensus.cc:515] T 26354361e9b044cea1c2b86c4882fe92 P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:29.003881 23785 consensus_queue.cc:237] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 26.16, Last appended by leader: 16, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:29.004480 16221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0eab25ce4c64e018a041beeda2867db" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 23 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:29.013012 16221 raft_consensus.cc:2410] T c0eab25ce4c64e018a041beeda2867db P 1ca2725d290e47d68b1a4b609c9816ac [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b for term 23 because replica has last-logged OpId of term: 22 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 10.
W20260709 13:22:28.997735 16265 consensus_peers.cc:597] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 90f3da57796b488c9087a2f523adf045 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:22:28.949963 24091 raft_consensus.cc:493] T 57a744f5aaf944678a63469cbf514015 P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:22:29.015606 16117 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:22:29.018162 16218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:22:28.995654 23924 leader_election.cc:290] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:29.024448 16218 raft_consensus.cc:2393] T ecc1e363230043fe9fef390f0b979f20 P 1ca2725d290e47d68b1a4b609c9816ac [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 25: Already voted for candidate cb82cc7bd6db46d980c75237a1402860 in this term.
I20260709 13:22:29.017989 23638 raft_consensus.cc:493] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:29.026330 24123 raft_consensus.cc:493] T 66f75124215f4ca6bef831e174a792d8 P cb82cc7bd6db46d980c75237a1402860 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:29.026218 23638 raft_consensus.cc:515] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:29.027546 23975 raft_consensus.cc:515] T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:29.028360 23638 leader_election.cc:290] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:29.029224 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ad5dbfa99ee4b8d9e28be92b3f36db5" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:29.029862 16226 raft_consensus.cc:2468] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 1ca2725d290e47d68b1a4b609c9816ac [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 20.
I20260709 13:22:29.031620 16220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8158bbd17b994d2eb96f560bd0f1b88d" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:29.024330 23798 raft_consensus.cc:515] T e5ad4eb281564936a59f41e387a47156 P 647a343a542d4a139fe106c19095129b [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:29.032423 16220 raft_consensus.cc:2393] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 21: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
I20260709 13:22:29.033748 16222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62fe3346cd3c42c4a55e14af9073bca4" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 27 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:22:29.034163 23798 leader_election.cc:290] T e5ad4eb281564936a59f41e387a47156 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:29.034770 16223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c2095a5f4ad49408dbc7f69c606120d" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 20 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:29.026937 24123 raft_consensus.cc:515] T 66f75124215f4ca6bef831e174a792d8 P cb82cc7bd6db46d980c75237a1402860 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:29.036034 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68a6838d82ef4cac83507f906139d90f" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:22:29.036608 16217 raft_consensus.cc:3060] T 68a6838d82ef4cac83507f906139d90f P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:22:29.034297 16222 raft_consensus.cc:3060] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [term 26 FOLLOWER]: Advancing to term 27
I20260709 13:22:29.039693 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "829b4ad8c5a54eababeea1181b960d4b" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 21 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:29.040498 16224 raft_consensus.cc:2393] T 829b4ad8c5a54eababeea1181b960d4b P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 21: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
I20260709 13:22:29.041476 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66f75124215f4ca6bef831e174a792d8" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:29.042064 16225 raft_consensus.cc:2468] T 66f75124215f4ca6bef831e174a792d8 P 1ca2725d290e47d68b1a4b609c9816ac [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 20.
I20260709 13:22:29.038441 23941 raft_consensus.cc:515] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [term 27 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:29.044533 16218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746aabf856ab401c9ef5e859371e7b67" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:22:29.045122 16222 raft_consensus.cc:2468] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 27.
I20260709 13:22:29.045276 16218 raft_consensus.cc:2393] T 746aabf856ab401c9ef5e859371e7b67 P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 19: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
I20260709 13:22:29.050680 16292 raft_consensus.cc:1275] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045 [term 21 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 21 index: 14. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
I20260709 13:22:29.050995 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a878999c0024d2abe517484180290fc" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:29.052567 16218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee9ed67006144797b1de6ea8c2013a68" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:29.053232 16218 raft_consensus.cc:2468] T ee9ed67006144797b1de6ea8c2013a68 P 1ca2725d290e47d68b1a4b609c9816ac [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 20.
W20260709 13:22:29.055207 16265 leader_election.cc:336] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:22:28.994414 24097 raft_consensus.cc:515] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 1ca2725d290e47d68b1a4b609c9816ac [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:29.054853 16225 raft_consensus.cc:2468] T 8a878999c0024d2abe517484180290fc P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 19.
I20260709 13:22:29.058065 16824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "829b4ad8c5a54eababeea1181b960d4b" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:22:29.058660 16824 raft_consensus.cc:3060] T 829b4ad8c5a54eababeea1181b960d4b P cb82cc7bd6db46d980c75237a1402860 [term 19 FOLLOWER]: Advancing to term 21
I20260709 13:22:29.058555 16222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c61a39fd4f84424b129c1a6f9ba404d" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:22:29.000465 23958 raft_consensus.cc:515] T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:22:29.060806 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 311567us
I20260709 13:22:29.062463 16217 raft_consensus.cc:2410] T 68a6838d82ef4cac83507f906139d90f P 1ca2725d290e47d68b1a4b609c9816ac [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b for term 22 because replica has last-logged OpId of term: 21 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 11.
I20260709 13:22:29.064291 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5718c623ebee4d2bb770f0f9ddb2f164" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 23 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:22:29.064903 16225 raft_consensus.cc:3060] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:22:29.066426 16218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4dc6b0541ca431ebc757beac00521f1" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 23 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:22:29.066965 16218 raft_consensus.cc:3060] T e4dc6b0541ca431ebc757beac00521f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:22:29.068727 16222 raft_consensus.cc:2468] T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 23.
I20260709 13:22:29.069341 24129 raft_consensus.cc:493] T c2f5f89186204242bf377f7c42b2e2da P 1ca2725d290e47d68b1a4b609c9816ac [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:29.069864 24129 raft_consensus.cc:515] T c2f5f89186204242bf377f7c42b2e2da P 1ca2725d290e47d68b1a4b609c9816ac [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:29.070497 24004 leader_election.cc:290] T 8158bbd17b994d2eb96f560bd0f1b88d P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 21 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:29.072095 24129 leader_election.cc:290] T c2f5f89186204242bf377f7c42b2e2da P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:29.072253 24104 leader_election.cc:290] T 8cc959fb7f704fba9009c0595467d074 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:29.078027 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee9ed67006144797b1de6ea8c2013a68" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:22:29.079022 16217 raft_consensus.cc:3060] T ee9ed67006144797b1de6ea8c2013a68 P 1ca2725d290e47d68b1a4b609c9816ac [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:22:29.080564 16222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b7d1ab519044eda2d5e04a07242f25" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:22:29.081437 16222 raft_consensus.cc:3060] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:22:29.084619 16220 raft_consensus.cc:1240] T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 20. Current term is 21. Ops: []
I20260709 13:22:29.087236 23958 leader_election.cc:290] T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 21 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
W20260709 13:22:29.087448 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 65558.4us
W20260709 13:22:29.088687 16265 leader_election.cc:336] T 23f82224f4564fea93572e8206bad1fe P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 18 election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:22:29.089597 16265 leader_election.cc:336] T 18ad7e3af5504571a69d10c00750a01c P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 18 election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:22:29.089954 16265 leader_election.cc:304] T 18ad7e3af5504571a69d10c00750a01c P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:22:29.092357 24123 leader_election.cc:290] T 66f75124215f4ca6bef831e174a792d8 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:22:29.093245 16218 raft_consensus.cc:2410] T e4dc6b0541ca431ebc757beac00521f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b for term 23 because replica has last-logged OpId of term: 22 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 11.
I20260709 13:22:29.093602 16790 leader_election.cc:304] T ee9ed67006144797b1de6ea8c2013a68 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 647a343a542d4a139fe106c19095129b
I20260709 13:22:29.094928 16790 leader_election.cc:304] T 19baa7bc367844c288bca8f81439c413 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:22:29.096902 24098 raft_consensus.cc:2764] T ee9ed67006144797b1de6ea8c2013a68 P cb82cc7bd6db46d980c75237a1402860 [term 21 FOLLOWER]: Leader pre-election decision vote started in defunct term 20: won
I20260709 13:22:29.098650 24088 raft_consensus.cc:2749] T 18ad7e3af5504571a69d10c00750a01c P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:22:29.099965 16265 consensus_peers.cc:597] T fffab26427e24d398e2695aad40ca134 P 90f3da57796b488c9087a2f523adf045 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:22:29.096978 24113 raft_consensus.cc:2749] T 19baa7bc367844c288bca8f81439c413 P cb82cc7bd6db46d980c75237a1402860 [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:22:29.087069 24120 raft_consensus.cc:515] T 02838c86531d4f98bf58d1c22b8baf77 P 1ca2725d290e47d68b1a4b609c9816ac [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:29.102547 23988 raft_consensus.cc:2749] T 18ad7e3af5504571a69d10c00750a01c P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:22:29.103466 16219 raft_consensus.cc:1240] T 02838c86531d4f98bf58d1c22b8baf77 P 1ca2725d290e47d68b1a4b609c9816ac [term 23 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 22. Current term is 23. Ops: []
I20260709 13:22:29.104761 16116 leader_election.cc:304] T ecc1e363230043fe9fef390f0b979f20 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860
I20260709 13:22:29.106362 24104 raft_consensus.cc:2749] T ecc1e363230043fe9fef390f0b979f20 P 647a343a542d4a139fe106c19095129b [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 13:22:29.106670 16116 leader_election.cc:304] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 90f3da57796b488c9087a2f523adf045
I20260709 13:22:29.096501 16790 leader_election.cc:304] T d2ab0ac8347641e6955e9bce7d90c1a2 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:22:29.112747 16218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5ad4eb281564936a59f41e387a47156" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:29.113893 16824 raft_consensus.cc:2468] T 829b4ad8c5a54eababeea1181b960d4b P cb82cc7bd6db46d980c75237a1402860 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 21.
I20260709 13:22:29.122283 16217 raft_consensus.cc:2468] T ee9ed67006144797b1de6ea8c2013a68 P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 21.
I20260709 13:22:29.130787 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8158bbd17b994d2eb96f560bd0f1b88d" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:22:29.131994 16217 raft_consensus.cc:2393] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 21: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
I20260709 13:22:29.133831 24120 leader_election.cc:290] T 02838c86531d4f98bf58d1c22b8baf77 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 23 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:29.123971 23958 raft_consensus.cc:493] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:22:29.135839 23958 raft_consensus.cc:515] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:29.136837 24088 raft_consensus.cc:2804] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:22:29.137313 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31969392ccdb43b491911a319e8bc7e1" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:22:29.137979 16298 raft_consensus.cc:3060] T 31969392ccdb43b491911a319e8bc7e1 P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Advancing to term 20
I20260709 13:22:29.138263 23958 leader_election.cc:290] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:29.138674 16818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62fe3346cd3c42c4a55e14af9073bca4" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 27 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:29.134493 23980 raft_consensus.cc:515] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [term 28 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:22:29.140542 24126 log.cc:927] Time spent T 26354361e9b044cea1c2b86c4882fe92 P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.076s	user 0.000s	sys 0.000s
I20260709 13:22:29.141970 24097 leader_election.cc:290] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 23 election: Requested vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
W20260709 13:22:29.142635 24124 log.cc:927] Time spent T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 0.079s	user 0.000s	sys 0.000s
I20260709 13:22:29.137357 24088 raft_consensus.cc:493] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:29.143553 24088 raft_consensus.cc:3060] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:22:29.144723 23719 leader_election.cc:290] T 26354361e9b044cea1c2b86c4882fe92 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 18 election: Requested vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
W20260709 13:22:29.145131 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 48713.3us
I20260709 13:22:29.145824 23994 raft_consensus.cc:515] T fffab26427e24d398e2695aad40ca134 P 647a343a542d4a139fe106c19095129b [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:29.146637 16790 leader_election.cc:304] T af57996a8be84d398f0f023e8d08ef77 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:22:29.150743 16116 leader_election.cc:304] T 7c2095a5f4ad49408dbc7f69c606120d P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:22:29.151626 24123 raft_consensus.cc:2749] T af57996a8be84d398f0f023e8d08ef77 P cb82cc7bd6db46d980c75237a1402860 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:22:29.152181 16116 leader_election.cc:304] T c0eab25ce4c64e018a041beeda2867db P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:22:29.153541 16116 leader_election.cc:304] T 829b4ad8c5a54eababeea1181b960d4b P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860
I20260709 13:22:29.154132 16790 leader_election.cc:400] T 829b4ad8c5a54eababeea1181b960d4b P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 20 pre-election: Vote denied by peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943) with higher term. Message: Invalid argument: T 829b4ad8c5a54eababeea1181b960d4b P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 for earlier term 20. Current term is 21.
I20260709 13:22:29.138017 16816 raft_consensus.cc:1275] T d2ab0ac8347641e6955e9bce7d90c1a2 P cb82cc7bd6db46d980c75237a1402860 [term 21 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 20 index: 16. Preceding OpId from leader: term: 21 index: 17. (index mismatch)
I20260709 13:22:29.154757 24004 raft_consensus.cc:2749] T 7c2095a5f4ad49408dbc7f69c606120d P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:22:29.155565 24004 raft_consensus.cc:3060] T c0eab25ce4c64e018a041beeda2867db P 647a343a542d4a139fe106c19095129b [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:22:29.155855 16790 leader_election.cc:403] T 829b4ad8c5a54eababeea1181b960d4b P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 20 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:22:29.156569 24114 raft_consensus.cc:2749] T d2ab0ac8347641e6955e9bce7d90c1a2 P cb82cc7bd6db46d980c75237a1402860 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:22:29.156827 16223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cc959fb7f704fba9009c0595467d074" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:29.157598 16223 raft_consensus.cc:2468] T 8cc959fb7f704fba9009c0595467d074 P 1ca2725d290e47d68b1a4b609c9816ac [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 20.
I20260709 13:22:29.158546 16223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a959dbecc2648eb85d1cc550f5a3245" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:22:29.148452 23994 leader_election.cc:290] T fffab26427e24d398e2695aad40ca134 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 22 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
W20260709 13:22:29.148653 16263 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:22:29.056914 24091 raft_consensus.cc:515] T 57a744f5aaf944678a63469cbf514015 P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:29.159099 16223 raft_consensus.cc:3060] T 5a959dbecc2648eb85d1cc550f5a3245 P 1ca2725d290e47d68b1a4b609c9816ac [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:22:29.162189 16116 leader_election.cc:304] T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: cb82cc7bd6db46d980c75237a1402860
I20260709 13:22:29.159601 16218 raft_consensus.cc:2410] T e5ad4eb281564936a59f41e387a47156 P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b for term 22 because replica has last-logged OpId of term: 20 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 12.
W20260709 13:22:29.163414 16263 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:22:29.166026 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cc959fb7f704fba9009c0595467d074" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:29.166276 16298 raft_consensus.cc:2468] T 31969392ccdb43b491911a319e8bc7e1 P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 20.
I20260709 13:22:29.167519 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdbd78acb4cf46298e0de8ba0d130e19" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 26 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:29.174216 16222 raft_consensus.cc:2468] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 21.
W20260709 13:22:29.177600 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 77027.6us
I20260709 13:22:29.180462 16300 raft_consensus.cc:1275] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 90f3da57796b488c9087a2f523adf045 [term 21 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
I20260709 13:22:29.182368 16816 raft_consensus.cc:1240] T d2ab0ac8347641e6955e9bce7d90c1a2 P cb82cc7bd6db46d980c75237a1402860 [term 21 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 20. Current term is 21. Ops: []
I20260709 13:22:29.200845 16294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8158bbd17b994d2eb96f560bd0f1b88d" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:22:29.201541 16294 raft_consensus.cc:3060] T 8158bbd17b994d2eb96f560bd0f1b88d P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:22:29.202061 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26354361e9b044cea1c2b86c4882fe92" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:22:29.202215 16295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5ad4eb281564936a59f41e387a47156" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:29.202268 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fffab26427e24d398e2695aad40ca134" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:22:29.202867 16295 raft_consensus.cc:2468] T e5ad4eb281564936a59f41e387a47156 P 90f3da57796b488c9087a2f523adf045 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 21.
I20260709 13:22:29.208331 16299 raft_consensus.cc:1275] T c9944d5e4b8c4109af9955c4f08c9095 P 90f3da57796b488c9087a2f523adf045 [term 24 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 23 index: 17. Preceding OpId from leader: term: 24 index: 18. (index mismatch)
I20260709 13:22:29.210517 16116 leader_election.cc:304] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: cb82cc7bd6db46d980c75237a1402860
W20260709 13:22:29.216794 16265 leader_election.cc:336] T 26354361e9b044cea1c2b86c4882fe92 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:22:29.219195 23605 consensus_queue.cc:1048] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:22:29.219413 24099 consensus_queue.cc:1048] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:22:29.220697 16220 raft_consensus.cc:1275] T 746aabf856ab401c9ef5e859371e7b67 P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 18 index: 15. Preceding OpId from leader: term: 19 index: 16. (index mismatch)
W20260709 13:22:29.222783 16265 consensus_peers.cc:597] T 7c2095a5f4ad49408dbc7f69c606120d P 90f3da57796b488c9087a2f523adf045 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:22:29.225025 16266 leader_election.cc:304] T a378d7f796f94615b3b347119a9880fd P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac
I20260709 13:22:29.226567 23988 raft_consensus.cc:2804] T a378d7f796f94615b3b347119a9880fd P 90f3da57796b488c9087a2f523adf045 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:22:29.227025 23988 raft_consensus.cc:493] T a378d7f796f94615b3b347119a9880fd P 90f3da57796b488c9087a2f523adf045 [term 21 FOLLOWER]: Starting leader election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:22:29.227330 24111 raft_consensus.cc:493] T ada5a004ba7c46819051e3ccec71ffdb P cb82cc7bd6db46d980c75237a1402860 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:29.227684 23988 raft_consensus.cc:3060] T a378d7f796f94615b3b347119a9880fd P 90f3da57796b488c9087a2f523adf045 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:22:29.227989 24111 raft_consensus.cc:515] T ada5a004ba7c46819051e3ccec71ffdb P cb82cc7bd6db46d980c75237a1402860 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:29.230350 24111 leader_election.cc:290] T ada5a004ba7c46819051e3ccec71ffdb P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:29.236995 24091 leader_election.cc:290] T 57a744f5aaf944678a63469cbf514015 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:29.241086 23865 consensus_queue.cc:1059] T d2ab0ac8347641e6955e9bce7d90c1a2 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: INVALID_TERM, Last received: 20.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 13:22:29.242661 24085 raft_consensus.cc:3055] T d2ab0ac8347641e6955e9bce7d90c1a2 P 90f3da57796b488c9087a2f523adf045 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:22:29.243099 24085 raft_consensus.cc:740] T d2ab0ac8347641e6955e9bce7d90c1a2 P 90f3da57796b488c9087a2f523adf045 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:22:29.244134 24085 consensus_queue.cc:260] T d2ab0ac8347641e6955e9bce7d90c1a2 P 90f3da57796b488c9087a2f523adf045 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 20.16, Last appended by leader: 16, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:29.250193 24085 raft_consensus.cc:3060] T d2ab0ac8347641e6955e9bce7d90c1a2 P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:22:29.159379 23997 raft_consensus.cc:3060] T 829b4ad8c5a54eababeea1181b960d4b P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:22:29.255110 16223 raft_consensus.cc:2468] T 5a959dbecc2648eb85d1cc550f5a3245 P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 21.
I20260709 13:22:29.255839 23988 raft_consensus.cc:515] T a378d7f796f94615b3b347119a9880fd P 90f3da57796b488c9087a2f523adf045 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:29.258630 23865 raft_consensus.cc:493] T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:29.197850 24104 raft_consensus.cc:2804] T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:22:29.259565 24104 raft_consensus.cc:493] T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Starting leader election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:22:29.259296 23865 raft_consensus.cc:515] T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:29.260118 24104 raft_consensus.cc:3060] T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:22:29.260860 16817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a378d7f796f94615b3b347119a9880fd" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:22:29.205561 23994 consensus_queue.cc:1048] T 84f2da98864b4425bf7c0055951fdd4b P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.058s
I20260709 13:22:29.264570 23988 leader_election.cc:290] T a378d7f796f94615b3b347119a9880fd P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 22 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:29.265515 16822 raft_consensus.cc:1240] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P cb82cc7bd6db46d980c75237a1402860 [term 24 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 23. Current term is 24. Ops: []
I20260709 13:22:29.267387 23988 consensus_queue.cc:1059] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.000s
I20260709 13:22:29.130278 23941 leader_election.cc:290] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 27 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:29.177318 24123 raft_consensus.cc:2749] T 829b4ad8c5a54eababeea1181b960d4b P cb82cc7bd6db46d980c75237a1402860 [term 21 FOLLOWER]: Leader pre-election lost for term 20. Reason: Vote denied by peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943) with higher term. Message: Invalid argument: T 829b4ad8c5a54eababeea1181b960d4b P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 for earlier term 20. Current term is 21.
I20260709 13:22:29.276473 16225 raft_consensus.cc:2410] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b for term 23 because replica has last-logged OpId of term: 22 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 16.
I20260709 13:22:29.277828 16222 raft_consensus.cc:1240] T c2f5f89186204242bf377f7c42b2e2da P 1ca2725d290e47d68b1a4b609c9816ac [term 25 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 24. Current term is 25. Ops: [22.13-24.14]
W20260709 13:22:29.288851 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 78447.1us
I20260709 13:22:29.287281 16817 raft_consensus.cc:2468] T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 22.
I20260709 13:22:29.289810 16116 leader_election.cc:304] T 8a878999c0024d2abe517484180290fc P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 90f3da57796b488c9087a2f523adf045
I20260709 13:22:29.292083 16266 leader_election.cc:304] T a378d7f796f94615b3b347119a9880fd P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:22:29.292222 16116 leader_election.cc:304] T 68a6838d82ef4cac83507f906139d90f P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:22:29.300541 16116 leader_election.cc:304] T 9c61a39fd4f84424b129c1a6f9ba404d P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 90f3da57796b488c9087a2f523adf045
I20260709 13:22:29.305622 16116 leader_election.cc:304] T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:22:29.298310 23815 raft_consensus.cc:2804] T a378d7f796f94615b3b347119a9880fd P 90f3da57796b488c9087a2f523adf045 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:22:29.311141 24091 raft_consensus.cc:2749] T 68a6838d82ef4cac83507f906139d90f P 647a343a542d4a139fe106c19095129b [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
W20260709 13:22:29.311400 16790 leader_election.cc:336] T 829b4ad8c5a54eababeea1181b960d4b P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:22:29.313422 24111 raft_consensus.cc:493] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:29.314021 24111 raft_consensus.cc:515] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:29.315598 23798 raft_consensus.cc:2804] T 9c61a39fd4f84424b129c1a6f9ba404d P 647a343a542d4a139fe106c19095129b [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:22:29.316298 23798 raft_consensus.cc:697] T 9c61a39fd4f84424b129c1a6f9ba404d P 647a343a542d4a139fe106c19095129b [term 23 LEADER]: Becoming Leader. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
W20260709 13:22:29.317728 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 493279us
I20260709 13:22:29.318548 16190 leader_election.cc:304] T e5ad4eb281564936a59f41e387a47156 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:22:29.317279 23865 leader_election.cc:290] T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:22:29.296428 23719 raft_consensus.cc:2804] T 8a878999c0024d2abe517484180290fc P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:22:29.336263 16190 leader_election.cc:304] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:22:29.317355 23798 consensus_queue.cc:237] T 9c61a39fd4f84424b129c1a6f9ba404d P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 22.12, Last appended by leader: 12, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:29.341214 23941 raft_consensus.cc:2804] T e5ad4eb281564936a59f41e387a47156 P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:22:29.345620 23941 raft_consensus.cc:493] T e5ad4eb281564936a59f41e387a47156 P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:29.346066 23941 raft_consensus.cc:3060] T e5ad4eb281564936a59f41e387a47156 P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:22:29.347101 23975 leader_election.cc:290] T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 21 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:29.348839 16192 leader_election.cc:304] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:22:29.336963 23719 raft_consensus.cc:493] T 8a878999c0024d2abe517484180290fc P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:29.303576 23994 raft_consensus.cc:2804] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [term 27 FOLLOWER]: Leader election won for term 27
I20260709 13:22:29.377961 16816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26354361e9b044cea1c2b86c4882fe92" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:22:29.387730 16816 raft_consensus.cc:2368] T 26354361e9b044cea1c2b86c4882fe92 P cb82cc7bd6db46d980c75237a1402860 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b for earlier term 18. Current term is 19.
W20260709 13:22:29.387877 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 82523.8us
I20260709 13:22:29.389101 16294 raft_consensus.cc:2468] T 8158bbd17b994d2eb96f560bd0f1b88d P 90f3da57796b488c9087a2f523adf045 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 21.
W20260709 13:22:29.395918 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 251703us
I20260709 13:22:29.399380 23605 raft_consensus.cc:2749] T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 13:22:29.400396 23996 raft_consensus.cc:493] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:29.401007 23996 raft_consensus.cc:515] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:29.401899 16818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57a744f5aaf944678a63469cbf514015" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 21 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:29.403520 16819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b7d1ab519044eda2d5e04a07242f25" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:22:29.406483 16817 raft_consensus.cc:1275] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860 [term 26 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 23 index: 16. Preceding OpId from leader: term: 26 index: 17. (index mismatch)
W20260709 13:22:29.409371 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 75578.2us
W20260709 13:22:29.409415 16192 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 60688.1us
I20260709 13:22:29.410866 16190 leader_election.cc:304] T c0eab25ce4c64e018a041beeda2867db P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:22:29.422225 16292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada5a004ba7c46819051e3ccec71ffdb" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:29.422636 16116 leader_election.cc:304] T 8cc959fb7f704fba9009c0595467d074 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 90f3da57796b488c9087a2f523adf045
I20260709 13:22:29.423039 16292 raft_consensus.cc:2468] T ada5a004ba7c46819051e3ccec71ffdb P 90f3da57796b488c9087a2f523adf045 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 23.
I20260709 13:22:29.424419 16192 leader_election.cc:304] T 829b4ad8c5a54eababeea1181b960d4b P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 
W20260709 13:22:29.439543 16191 leader_election.cc:336] T d16dfad0c3b540e5b4b86492ddf87208 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:22:29.441962 16191 leader_election.cc:336] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:22:29.442581 16191 leader_election.cc:304] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
W20260709 13:22:29.444100 16191 leader_election.cc:336] T e5ad4eb281564936a59f41e387a47156 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:22:29.444260 24137 log.cc:927] Time spent T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860: Append to log took a long time: real 0.100s	user 0.002s	sys 0.000s
I20260709 13:22:29.444821 24130 raft_consensus.cc:2804] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 24 FOLLOWER]: Leader election won for term 24
I20260709 13:22:29.446866 24130 raft_consensus.cc:697] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 24 LEADER]: Becoming Leader. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:22:29.447968 24130 consensus_queue.cc:237] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 22.17, Last appended by leader: 17, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:29.452380 24111 leader_election.cc:290] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
W20260709 13:22:29.454596 16191 leader_election.cc:336] T 7c2095a5f4ad49408dbc7f69c606120d P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:22:29.463356 24131 log.cc:927] Time spent T 23f82224f4564fea93572e8206bad1fe P cb82cc7bd6db46d980c75237a1402860: Append to log took a long time: real 0.378s	user 0.001s	sys 0.001s
W20260709 13:22:29.464277 16191 leader_election.cc:336] T 68a6838d82ef4cac83507f906139d90f P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:22:29.388653 23719 raft_consensus.cc:3060] T 8a878999c0024d2abe517484180290fc P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:22:29.467450 23958 raft_consensus.cc:2804] T 829b4ad8c5a54eababeea1181b960d4b P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:22:29.471628 23958 raft_consensus.cc:697] T 829b4ad8c5a54eababeea1181b960d4b P 1ca2725d290e47d68b1a4b609c9816ac [term 21 LEADER]: Becoming Leader. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:22:29.472823 24103 leader_election.cc:290] T b87147731a68400e8cce9b879daa7c5d P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:29.472616 23958 consensus_queue.cc:237] T 829b4ad8c5a54eababeea1181b960d4b P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:29.415593 23915 raft_consensus.cc:2804] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:22:29.475638 23915 raft_consensus.cc:493] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:29.476024 23915 raft_consensus.cc:3060] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:22:29.478382 23997 raft_consensus.cc:2749] T 829b4ad8c5a54eababeea1181b960d4b P 647a343a542d4a139fe106c19095129b [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:22:29.479467 23980 leader_election.cc:290] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 28 election: Requested vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:29.490962 16190 leader_election.cc:304] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860
W20260709 13:22:29.495092 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 72534.4us
I20260709 13:22:29.488648 16191 leader_election.cc:304] T 68a6838d82ef4cac83507f906139d90f P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:22:29.339406 24088 raft_consensus.cc:515] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:29.447191 24127 consensus_queue.cc:1048] T c9944d5e4b8c4109af9955c4f08c9095 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.041s
I20260709 13:22:29.496104 16116 leader_election.cc:304] T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
I20260709 13:22:29.507850 16789 leader_election.cc:304] T ada5a004ba7c46819051e3ccec71ffdb P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:22:29.510520 24125 raft_consensus.cc:2804] T ada5a004ba7c46819051e3ccec71ffdb P cb82cc7bd6db46d980c75237a1402860 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:22:29.511070 24125 raft_consensus.cc:493] T ada5a004ba7c46819051e3ccec71ffdb P cb82cc7bd6db46d980c75237a1402860 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:29.511480 24125 raft_consensus.cc:3060] T ada5a004ba7c46819051e3ccec71ffdb P cb82cc7bd6db46d980c75237a1402860 [term 23 FOLLOWER]: Advancing to term 24
W20260709 13:22:29.524330 16265 leader_election.cc:336] T 3884f864a96d428785895e5463e5728f P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:22:29.524878 16265 leader_election.cc:304] T 3884f864a96d428785895e5463e5728f P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
I20260709 13:22:29.525499 16117 leader_election.cc:400] T 26354361e9b044cea1c2b86c4882fe92 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 18 election: Vote denied by peer cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923) with higher term. Message: Invalid argument: T 26354361e9b044cea1c2b86c4882fe92 P cb82cc7bd6db46d980c75237a1402860 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b for earlier term 18. Current term is 19.
I20260709 13:22:29.526015 16117 leader_election.cc:403] T 26354361e9b044cea1c2b86c4882fe92 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 18 election: Cancelling election due to peer responding with higher term
I20260709 13:22:29.526813 24085 raft_consensus.cc:2749] T 3884f864a96d428785895e5463e5728f P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:22:29.528123 16822 raft_consensus.cc:1275] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860 [term 21 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
W20260709 13:22:29.529680 16265 leader_election.cc:336] T 1df93779209748b9860f6d893dd3de53 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 20 election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:22:29.530368 16265 leader_election.cc:304] T 1df93779209748b9860f6d893dd3de53 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
W20260709 13:22:29.530917 15046 thread.cc:527] Waited for 1000ms trying to join with heartbeat (tid 16176)
I20260709 13:22:29.531657 24085 raft_consensus.cc:2749] T 1df93779209748b9860f6d893dd3de53 P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
W20260709 13:22:29.532560 16265 leader_election.cc:336] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 25 election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:22:29.535212 16265 leader_election.cc:336] T 2794af5b99984efebdee564035841cb0 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:22:29.536024 16265 leader_election.cc:304] T 2794af5b99984efebdee564035841cb0 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
I20260709 13:22:29.536546 16297 raft_consensus.cc:1275] T 66ae176961b74813a58bc15c9c4c6f2a P 90f3da57796b488c9087a2f523adf045 [term 27 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 26 index: 16. Preceding OpId from leader: term: 27 index: 17. (index mismatch)
I20260709 13:22:29.537246 24085 raft_consensus.cc:2749] T 2794af5b99984efebdee564035841cb0 P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:22:29.538393 16265 consensus_peers.cc:597] T 8a878999c0024d2abe517484180290fc P 90f3da57796b488c9087a2f523adf045 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:22:29.543385 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 230668us
I20260709 13:22:29.545099 16291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87147731a68400e8cce9b879daa7c5d" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
W20260709 13:22:29.552244 16265 leader_election.cc:336] T 02838c86531d4f98bf58d1c22b8baf77 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:22:29.538941 24096 consensus_queue.cc:1048] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.015s
I20260709 13:22:29.488165 24097 raft_consensus.cc:2804] T c0eab25ce4c64e018a041beeda2867db P 1ca2725d290e47d68b1a4b609c9816ac [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:22:29.427357 23994 raft_consensus.cc:697] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [term 27 LEADER]: Becoming Leader. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:22:29.505503 24004 raft_consensus.cc:2749] T c0eab25ce4c64e018a041beeda2867db P 647a343a542d4a139fe106c19095129b [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:22:29.507249 24073 raft_consensus.cc:3055] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045 [term 23 LEADER]: Stepping down as leader of term 23
W20260709 13:22:29.561206 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 65226us
W20260709 13:22:29.537017 23938 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.800s	user 0.005s	sys 0.009s
I20260709 13:22:29.506862 24091 raft_consensus.cc:2804] T 8cc959fb7f704fba9009c0595467d074 P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:22:29.562575 16224 raft_consensus.cc:1275] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 21 index: 14. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
W20260709 13:22:29.560451 16192 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 52516.9us
I20260709 13:22:29.599269 16822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7ca61592e848ea9b0fb81a2ce33cdc" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:22:29.600129 16822 raft_consensus.cc:3060] T 3a7ca61592e848ea9b0fb81a2ce33cdc P cb82cc7bd6db46d980c75237a1402860 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:22:29.563952 24129 consensus_queue.cc:1048] T d2ab0ac8347641e6955e9bce7d90c1a2 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 13:22:29.567651 23997 raft_consensus.cc:2804] T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:22:29.600633 23287 raft_consensus.cc:493] T ecc1e363230043fe9fef390f0b979f20 P 90f3da57796b488c9087a2f523adf045 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:22:29.569964 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 105722us
W20260709 13:22:29.601518 16191 leader_election.cc:336] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:22:29.601255 23287 raft_consensus.cc:515] T ecc1e363230043fe9fef390f0b979f20 P 90f3da57796b488c9087a2f523adf045 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:29.601989 16191 leader_election.cc:304] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:22:29.543651 23815 raft_consensus.cc:697] T a378d7f796f94615b3b347119a9880fd P 90f3da57796b488c9087a2f523adf045 [term 22 LEADER]: Becoming Leader. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:22:29.575783 23994 consensus_queue.cc:237] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 26.16, Last appended by leader: 16, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:29.560029 24097 raft_consensus.cc:493] T c0eab25ce4c64e018a041beeda2867db P 1ca2725d290e47d68b1a4b609c9816ac [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:29.581759 23941 raft_consensus.cc:515] T e5ad4eb281564936a59f41e387a47156 P 1ca2725d290e47d68b1a4b609c9816ac [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:29.583266 23915 raft_consensus.cc:515] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:29.585544 24088 leader_election.cc:290] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 21 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:29.578308 24073 raft_consensus.cc:740] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:22:29.549783 23980 raft_consensus.cc:2749] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 13:22:29.563304 24132 raft_consensus.cc:2749] T 68a6838d82ef4cac83507f906139d90f P 1ca2725d290e47d68b1a4b609c9816ac [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:22:29.591517 16036 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet ecc1e363230043fe9fef390f0b979f20 with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER failed with leader cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica cb82cc7bd6db46d980c75237a1402860 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 25 committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:22:29.595268 24091 raft_consensus.cc:493] T 8cc959fb7f704fba9009c0595467d074 P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:29.604061 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02838c86531d4f98bf58d1c22b8baf77" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:29.604563 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c11074579b046d99e585c2818e18a2c" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:29.606050 16816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
W20260709 13:22:29.607887 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 117016us
W20260709 13:22:29.590750 23938 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.854s	user 0.005s	sys 0.009s
W20260709 13:22:29.613013 16115 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:22:29.615296 16819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66ae176961b74813a58bc15c9c4c6f2a" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 28 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:22:29.602936 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57a744f5aaf944678a63469cbf514015" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 21 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:29.624755 16817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2372981006d4b218215c28964dea623" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 27 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:22:29.598232 24120 raft_consensus.cc:2749] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:22:29.706388 16818 raft_consensus.cc:1275] T 26354361e9b044cea1c2b86c4882fe92 P cb82cc7bd6db46d980c75237a1402860 [term 19 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 13:22:29.708392 16218 raft_consensus.cc:1240] T e4dc6b0541ca431ebc757beac00521f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 23 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 22. Current term is 23. Ops: []
I20260709 13:22:29.617004 24097 raft_consensus.cc:3060] T c0eab25ce4c64e018a041beeda2867db P 1ca2725d290e47d68b1a4b609c9816ac [term 23 FOLLOWER]: Advancing to term 24
W20260709 13:22:29.719975 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 118480us
I20260709 13:22:29.712297 16226 raft_consensus.cc:2393] T 02838c86531d4f98bf58d1c22b8baf77 P 1ca2725d290e47d68b1a4b609c9816ac [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 23: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
W20260709 13:22:29.726310 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 130816us
I20260709 13:22:29.730958 16822 raft_consensus.cc:2468] T 3a7ca61592e848ea9b0fb81a2ce33cdc P cb82cc7bd6db46d980c75237a1402860 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 23.
I20260709 13:22:29.718170 16816 raft_consensus.cc:2468] T 19baa7bc367844c288bca8f81439c413 P cb82cc7bd6db46d980c75237a1402860 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 24.
I20260709 13:22:29.607009 23719 raft_consensus.cc:515] T 8a878999c0024d2abe517484180290fc P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:29.738145 23287 leader_election.cc:290] T ecc1e363230043fe9fef390f0b979f20 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
W20260709 13:22:29.724063 16191 leader_election.cc:336] T 8cc959fb7f704fba9009c0595467d074 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:22:29.746093 16191 leader_election.cc:336] T 5a96373bb0e14dce8a266b2fc6c58507 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:22:29.746726 16191 leader_election.cc:304] T 5a96373bb0e14dce8a266b2fc6c58507 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:22:29.743958 16817 raft_consensus.cc:3060] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 [term 26 FOLLOWER]: Advancing to term 27
I20260709 13:22:29.625609 24099 raft_consensus.cc:493] T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:29.626451 24149 raft_consensus.cc:493] T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:29.708194 24091 raft_consensus.cc:3060] T 8cc959fb7f704fba9009c0595467d074 P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:22:29.737325 16115 leader_election.cc:304] T e5ad4eb281564936a59f41e387a47156 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac
I20260709 13:22:29.759192 16221 raft_consensus.cc:1240] T af57996a8be84d398f0f023e8d08ef77 P 1ca2725d290e47d68b1a4b609c9816ac [term 22 LEADER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 21. Current term is 22. Ops: []
W20260709 13:22:29.760031 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 53389.3us
I20260709 13:22:29.761060 16190 leader_election.cc:304] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860
I20260709 13:22:29.753767 16116 leader_election.cc:304] T 5a959dbecc2648eb85d1cc550f5a3245 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 90f3da57796b488c9087a2f523adf045
I20260709 13:22:29.750656 16217 raft_consensus.cc:1240] T 68a6838d82ef4cac83507f906139d90f P 1ca2725d290e47d68b1a4b609c9816ac [term 22 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 21. Current term is 22. Ops: []
I20260709 13:22:29.707683 23958 raft_consensus.cc:2749] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 1ca2725d290e47d68b1a4b609c9816ac [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:22:29.626579 23997 raft_consensus.cc:697] T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b [term 21 LEADER]: Becoming Leader. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:22:29.767695 24140 consensus_queue.cc:1048] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.046s
I20260709 13:22:29.705899 24004 consensus_queue.cc:1059] T 8d4ab870a05347868956aa8b841b2981 P 647a343a542d4a139fe106c19095129b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: INVALID_TERM, Last received: 20.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.126s
I20260709 13:22:29.786962 16190 leader_election.cc:304] T d16dfad0c3b540e5b4b86492ddf87208 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:22:29.788002 16217 raft_consensus.cc:1240] T 5a959dbecc2648eb85d1cc550f5a3245 P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 20. Current term is 21. Ops: []
I20260709 13:22:29.738044 24096 raft_consensus.cc:493] T 8638ee363273426db1393dd81e5c7b72 P cb82cc7bd6db46d980c75237a1402860 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:29.769379 23975 consensus_queue.cc:1048] T 746aabf856ab401c9ef5e859371e7b67 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.039s
I20260709 13:22:29.739020 24088 consensus_queue.cc:1048] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.054s
I20260709 13:22:29.754491 23980 raft_consensus.cc:2749] T 5a96373bb0e14dce8a266b2fc6c58507 P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:22:29.782460 16191 leader_election.cc:336] T 5a959dbecc2648eb85d1cc550f5a3245 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:22:29.824633 16191 leader_election.cc:304] T 5a959dbecc2648eb85d1cc550f5a3245 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:22:29.809505 16190 leader_election.cc:304] T 7c2095a5f4ad49408dbc7f69c606120d P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
W20260709 13:22:29.781337 16115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 44072.6us
I20260709 13:22:29.819882 16221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6f07842d3aa47858df5665874c86f3a" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:29.797422 23815 consensus_queue.cc:237] T a378d7f796f94615b3b347119a9880fd P 90f3da57796b488c9087a2f523adf045 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 21.14, Last appended by leader: 14, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:29.793853 16819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 26 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:29.803647 16817 raft_consensus.cc:2468] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 27.
I20260709 13:22:29.816768 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:29.835692 16225 raft_consensus.cc:2468] T 19baa7bc367844c288bca8f81439c413 P 1ca2725d290e47d68b1a4b609c9816ac [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 24.
I20260709 13:22:29.836225 16819 raft_consensus.cc:2468] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 25.
I20260709 13:22:29.817324 16224 raft_consensus.cc:1240] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 24 LEADER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 23. Current term is 24. Ops: []
I20260709 13:22:29.810070 16217 raft_consensus.cc:1275] T 8a878999c0024d2abe517484180290fc P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 13:22:29.828527 16221 raft_consensus.cc:2468] T b6f07842d3aa47858df5665874c86f3a P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 21.
I20260709 13:22:29.828763 16223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806d7690c2b644959152074bb2cab0a7" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:29.768692 23941 leader_election.cc:290] T e5ad4eb281564936a59f41e387a47156 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 22 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:29.842183 16217 raft_consensus.cc:1275] T c2f5f89186204242bf377f7c42b2e2da P 1ca2725d290e47d68b1a4b609c9816ac [term 25 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 20 index: 12. Preceding OpId from leader: term: 25 index: 14. (index mismatch)
W20260709 13:22:29.843514 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 61101.4us
W20260709 13:22:29.844292 16191 leader_election.cc:336] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:22:29.845486 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a378d7f796f94615b3b347119a9880fd" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:22:29.846505 16225 raft_consensus.cc:2393] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 22: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
I20260709 13:22:29.817129 16222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada5a004ba7c46819051e3ccec71ffdb" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:22:29.793569 24162 raft_consensus.cc:493] T b6f07842d3aa47858df5665874c86f3a P 647a343a542d4a139fe106c19095129b [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:29.838174 16220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f2da98864b4425bf7c0055951fdd4b" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:29.782128 24141 raft_consensus.cc:3060] T 26354361e9b044cea1c2b86c4882fe92 P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:22:29.848129 16220 raft_consensus.cc:2468] T 84f2da98864b4425bf7c0055951fdd4b P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 19.
I20260709 13:22:29.801199 24129 raft_consensus.cc:2749] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:22:29.811090 23996 leader_election.cc:290] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:29.836026 24130 raft_consensus.cc:2749] T 7c2095a5f4ad49408dbc7f69c606120d P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:22:29.871905 24125 raft_consensus.cc:515] T ada5a004ba7c46819051e3ccec71ffdb P cb82cc7bd6db46d980c75237a1402860 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:29.812192 23988 consensus_queue.cc:1048] T 26354361e9b044cea1c2b86c4882fe92 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.044s
I20260709 13:22:29.815332 24096 raft_consensus.cc:515] T 8638ee363273426db1393dd81e5c7b72 P cb82cc7bd6db46d980c75237a1402860 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:29.874331 24125 leader_election.cc:290] T ada5a004ba7c46819051e3ccec71ffdb P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 24 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:29.876238 24096 leader_election.cc:290] T 8638ee363273426db1393dd81e5c7b72 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:29.850481 24113 raft_consensus.cc:493] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:29.877480 24113 raft_consensus.cc:515] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:29.879580 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada5a004ba7c46819051e3ccec71ffdb" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:22:29.880285 16300 raft_consensus.cc:3060] T ada5a004ba7c46819051e3ccec71ffdb P 90f3da57796b488c9087a2f523adf045 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:22:29.882649 16219 raft_consensus.cc:1275] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [term 25 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 22 index: 17. Preceding OpId from leader: term: 25 index: 19. (index mismatch)
I20260709 13:22:29.883600 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8638ee363273426db1393dd81e5c7b72" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 23 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:29.884318 16293 raft_consensus.cc:2468] T 8638ee363273426db1393dd81e5c7b72 P 90f3da57796b488c9087a2f523adf045 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 22.
W20260709 13:22:29.884920 16191 leader_election.cc:336] T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:22:29.866276 23915 leader_election.cc:290] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 23 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:29.867266 24162 raft_consensus.cc:515] T b6f07842d3aa47858df5665874c86f3a P 647a343a542d4a139fe106c19095129b [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:22:29.885995 16191 leader_election.cc:336] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:22:29.887615 24162 leader_election.cc:290] T b6f07842d3aa47858df5665874c86f3a P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:29.887689 16265 leader_election.cc:304] T 8c11074579b046d99e585c2818e18a2c P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860
W20260709 13:22:29.854909 24104 consensus_meta.cc:287] T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b: Time spent flushing consensus metadata: real 0.589s	user 0.003s	sys 0.009s
I20260709 13:22:29.888756 24104 raft_consensus.cc:515] T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:29.889140 24091 raft_consensus.cc:515] T 8cc959fb7f704fba9009c0595467d074 P 647a343a542d4a139fe106c19095129b [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:29.868841 23719 leader_election.cc:290] T 8a878999c0024d2abe517484180290fc P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 20 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:29.851125 23924 raft_consensus.cc:2749] T 5a959dbecc2648eb85d1cc550f5a3245 P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:22:29.790606 24099 raft_consensus.cc:515] T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:29.780475 24073 consensus_queue.cc:260] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 23.18, Last appended by leader: 18, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:29.769578 24149 raft_consensus.cc:515] T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:29.879798 23798 raft_consensus.cc:2804] T 5a959dbecc2648eb85d1cc550f5a3245 P 647a343a542d4a139fe106c19095129b [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:22:29.809044 23958 raft_consensus.cc:2749] T d16dfad0c3b540e5b4b86492ddf87208 P 1ca2725d290e47d68b1a4b609c9816ac [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:22:29.893546 16265 leader_election.cc:304] T 02838c86531d4f98bf58d1c22b8baf77 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:22:29.816620 23975 consensus_queue.cc:1059] T 02838c86531d4f98bf58d1c22b8baf77 P 647a343a542d4a139fe106c19095129b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: INVALID_TERM, Last received: 22.13, Next index: 14, Last known committed idx: 11, Time since last communication: 0.043s
W20260709 13:22:29.916440 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 90449.5us
I20260709 13:22:29.918020 16190 leader_election.cc:304] T 18ad7e3af5504571a69d10c00750a01c P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:22:29.913139 16192 leader_election.cc:304] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 
W20260709 13:22:29.829056 16117 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 201705us
I20260709 13:22:29.910508 16300 raft_consensus.cc:2468] T ada5a004ba7c46819051e3ccec71ffdb P 90f3da57796b488c9087a2f523adf045 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 24.
I20260709 13:22:29.922550 16190 leader_election.cc:304] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: cb82cc7bd6db46d980c75237a1402860
W20260709 13:22:29.928700 16191 leader_election.cc:336] T 02838c86531d4f98bf58d1c22b8baf77 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:22:29.891119 24104 leader_election.cc:290] T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 20 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:29.910171 23996 raft_consensus.cc:2749] T 8c11074579b046d99e585c2818e18a2c P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:22:29.896739 24159 raft_consensus.cc:2804] T e5ad4eb281564936a59f41e387a47156 P 647a343a542d4a139fe106c19095129b [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:22:29.895530 16222 raft_consensus.cc:2468] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 23.
I20260709 13:22:29.935500 23798 raft_consensus.cc:697] T 5a959dbecc2648eb85d1cc550f5a3245 P 647a343a542d4a139fe106c19095129b [term 21 LEADER]: Becoming Leader. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:22:29.939208 16192 leader_election.cc:304] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:22:29.977489 23997 consensus_queue.cc:237] T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 20.16, Last appended by leader: 16, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:29.978252 23798 consensus_queue.cc:237] T 5a959dbecc2648eb85d1cc550f5a3245 P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.12, Last appended by leader: 12, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:29.947798 24149 leader_election.cc:290] T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:29.921914 24146 consensus_queue.cc:1059] T 68a6838d82ef4cac83507f906139d90f P 90f3da57796b488c9087a2f523adf045 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: INVALID_TERM, Last received: 21.12, Next index: 13, Last known committed idx: 10, Time since last communication: 0.032s
I20260709 13:22:29.951511 24163 consensus_queue.cc:1059] T 5a959dbecc2648eb85d1cc550f5a3245 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: INVALID_TERM, Last received: 20.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.047s
I20260709 13:22:29.953056 24097 raft_consensus.cc:515] T c0eab25ce4c64e018a041beeda2867db P 1ca2725d290e47d68b1a4b609c9816ac [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:29.944913 23915 raft_consensus.cc:2804] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 1ca2725d290e47d68b1a4b609c9816ac [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:22:29.911901 24085 raft_consensus.cc:3060] T 02838c86531d4f98bf58d1c22b8baf77 P 90f3da57796b488c9087a2f523adf045 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:22:29.983068 23915 raft_consensus.cc:697] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 1ca2725d290e47d68b1a4b609c9816ac [term 23 LEADER]: Becoming Leader. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:22:29.983342 24130 raft_consensus.cc:2804] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [term 27 FOLLOWER]: Leader election won for term 27
I20260709 13:22:29.983915 24130 raft_consensus.cc:697] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [term 27 LEADER]: Becoming Leader. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:22:29.984287 24097 leader_election.cc:290] T c0eab25ce4c64e018a041beeda2867db P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 24 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:29.984145 23915 consensus_queue.cc:237] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 21.16, Last appended by leader: 16, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:29.984826 24130 consensus_queue.cc:237] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 25.16, Last appended by leader: 16, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:29.981537 16819 raft_consensus.cc:1275] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P cb82cc7bd6db46d980c75237a1402860 [term 24 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 22 index: 17. Preceding OpId from leader: term: 24 index: 18. (index mismatch)
W20260709 13:22:29.958771 16265 leader_election.cc:336] T 57a744f5aaf944678a63469cbf514015 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:22:29.989251 16265 leader_election.cc:304] T 57a744f5aaf944678a63469cbf514015 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
I20260709 13:22:29.981473 23975 raft_consensus.cc:493] T 3884f864a96d428785895e5463e5728f P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:29.954838 24129 raft_consensus.cc:493] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:29.992396 16222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 26 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:30.010723 16222 raft_consensus.cc:2468] T ecc1e363230043fe9fef390f0b979f20 P 1ca2725d290e47d68b1a4b609c9816ac [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 25.
I20260709 13:22:29.932021 23924 raft_consensus.cc:2804] T 18ad7e3af5504571a69d10c00750a01c P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:22:30.013473 16219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74ca8140f35c4ae29e8a664f6bdfc829" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:29.912158 23988 consensus_queue.cc:1059] T e4dc6b0541ca431ebc757beac00521f1 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: INVALID_TERM, Last received: 22.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.076s
I20260709 13:22:30.017421 16222 raft_consensus.cc:1275] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 1ca2725d290e47d68b1a4b609c9816ac [term 20 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 13:22:30.016446 16219 raft_consensus.cc:2468] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 24.
I20260709 13:22:29.938107 23958 raft_consensus.cc:2804] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:22:30.021739 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7ca61592e848ea9b0fb81a2ce33cdc" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:29.988294 24159 raft_consensus.cc:493] T e5ad4eb281564936a59f41e387a47156 P 647a343a542d4a139fe106c19095129b [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:29.981243 24113 leader_election.cc:290] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:30.022241 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cc959fb7f704fba9009c0595467d074" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
W20260709 13:22:30.022275 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 63573.3us
I20260709 13:22:30.023550 16291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fffab26427e24d398e2695aad40ca134" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:30.022850 16293 raft_consensus.cc:2468] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 90f3da57796b488c9087a2f523adf045 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 22.
I20260709 13:22:30.024534 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c61a39fd4f84424b129c1a6f9ba404d" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:30.024698 16296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e4bfb9090c8460eb14fa7e0d818e7f1" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:22:30.025300 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1df93779209748b9860f6d893dd3de53" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
W20260709 13:22:30.025511 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 271811us
I20260709 13:22:30.035169 16299 raft_consensus.cc:2468] T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 23.
I20260709 13:22:30.040361 16295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66ae176961b74813a58bc15c9c4c6f2a" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 28 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:30.041615 16190 leader_election.cc:304] T 8cc959fb7f704fba9009c0595467d074 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:22:30.016100 23996 raft_consensus.cc:3055] T 68a6838d82ef4cac83507f906139d90f P 90f3da57796b488c9087a2f523adf045 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:22:30.043646 16298 raft_consensus.cc:2393] T 1df93779209748b9860f6d893dd3de53 P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 20: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
I20260709 13:22:30.049533 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5ad4eb281564936a59f41e387a47156" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:22:30.050104 16293 raft_consensus.cc:3060] T e5ad4eb281564936a59f41e387a47156 P 90f3da57796b488c9087a2f523adf045 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:22:30.044984 16190 leader_election.cc:304] T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:22:30.051627 16265 leader_election.cc:304] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:22:30.046056 16116 leader_election.cc:304] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860
I20260709 13:22:30.047950 16789 leader_election.cc:304] T 8638ee363273426db1393dd81e5c7b72 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:22:30.019690 23924 raft_consensus.cc:493] T 18ad7e3af5504571a69d10c00750a01c P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:30.039839 24149 raft_consensus.cc:2749] T 57a744f5aaf944678a63469cbf514015 P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:22:30.062863 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d4ab870a05347868956aa8b841b2981" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:22:30.064203 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4f65cdf00b748bb963ee6e81ef43b79" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:22:30.032904 24159 raft_consensus.cc:3060] T e5ad4eb281564936a59f41e387a47156 P 647a343a542d4a139fe106c19095129b [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:22:30.064710 16265 leader_election.cc:304] T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:22:30.071575 16297 raft_consensus.cc:3060] T c4f65cdf00b748bb963ee6e81ef43b79 P 90f3da57796b488c9087a2f523adf045 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:22:30.073957 16819 raft_consensus.cc:1275] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 [term 25 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 24 index: 18. Preceding OpId from leader: term: 25 index: 19. (index mismatch)
I20260709 13:22:30.039405 23958 raft_consensus.cc:697] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [term 22 LEADER]: Becoming Leader. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:22:30.015585 24129 raft_consensus.cc:515] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:30.051438 23938 raft_consensus.cc:2749] T 8cc959fb7f704fba9009c0595467d074 P 1ca2725d290e47d68b1a4b609c9816ac [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:22:30.094758 16223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a878999c0024d2abe517484180290fc" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:22:30.121670 16291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7ca61592e848ea9b0fb81a2ce33cdc" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:22:30.128588 16223 raft_consensus.cc:3060] T 8a878999c0024d2abe517484180290fc P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:22:30.122025 16222 raft_consensus.cc:1275] T ee9ed67006144797b1de6ea8c2013a68 P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 21 index: 16. (index mismatch)
I20260709 13:22:30.130363 16222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cc959fb7f704fba9009c0595467d074" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:22:30.049388 23996 raft_consensus.cc:740] T 68a6838d82ef4cac83507f906139d90f P 90f3da57796b488c9087a2f523adf045 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:22:30.057624 24125 raft_consensus.cc:2804] T 8638ee363273426db1393dd81e5c7b72 P cb82cc7bd6db46d980c75237a1402860 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:22:30.140954 24125 raft_consensus.cc:493] T 8638ee363273426db1393dd81e5c7b72 P cb82cc7bd6db46d980c75237a1402860 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:30.140705 23958 consensus_queue.cc:237] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 20.16, Last appended by leader: 16, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:30.141508 24125 raft_consensus.cc:3060] T 8638ee363273426db1393dd81e5c7b72 P cb82cc7bd6db46d980c75237a1402860 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:22:30.118300 23994 raft_consensus.cc:2749] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 13:22:30.135566 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fffab26427e24d398e2695aad40ca134" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
W20260709 13:22:30.123426 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 58802.7us
I20260709 13:22:30.145174 16222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66f75124215f4ca6bef831e174a792d8" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:22:30.077867 16219 raft_consensus.cc:1275] T 08c74490c40243a88e62d1760b5cd814 P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 20 index: 12. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
I20260709 13:22:30.050325 24146 raft_consensus.cc:3055] T e4dc6b0541ca431ebc757beac00521f1 P 90f3da57796b488c9087a2f523adf045 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:22:30.125516 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6f07842d3aa47858df5665874c86f3a" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:30.149829 16219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2372981006d4b218215c28964dea623" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 27 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:30.150243 16217 raft_consensus.cc:2468] T b6f07842d3aa47858df5665874c86f3a P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 21.
I20260709 13:22:30.116955 16818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74ca8140f35c4ae29e8a664f6bdfc829" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:30.151808 16818 raft_consensus.cc:2468] T 74ca8140f35c4ae29e8a664f6bdfc829 P cb82cc7bd6db46d980c75237a1402860 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 24.
I20260709 13:22:30.031776 23980 consensus_queue.cc:1048] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.044s
I20260709 13:22:30.026031 23975 raft_consensus.cc:515] T 3884f864a96d428785895e5463e5728f P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:30.057106 23924 raft_consensus.cc:3060] T 18ad7e3af5504571a69d10c00750a01c P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:22:30.137156 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ad5dbfa99ee4b8d9e28be92b3f36db5" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:22:30.168434 16225 raft_consensus.cc:3060] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 1ca2725d290e47d68b1a4b609c9816ac [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:22:30.139748 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0eab25ce4c64e018a041beeda2867db" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:22:30.145754 16265 leader_election.cc:304] T 84f2da98864b4425bf7c0055951fdd4b P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:22:30.169567 16299 raft_consensus.cc:3060] T c0eab25ce4c64e018a041beeda2867db P 90f3da57796b488c9087a2f523adf045 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:22:30.128787 16291 raft_consensus.cc:3060] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 90f3da57796b488c9087a2f523adf045 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:22:30.076185 24163 raft_consensus.cc:2804] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:22:30.153014 16824 raft_consensus.cc:1275] T ee9ed67006144797b1de6ea8c2013a68 P cb82cc7bd6db46d980c75237a1402860 [term 21 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 21 index: 16. (index mismatch)
W20260709 13:22:30.162420 16115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 328806us
I20260709 13:22:30.062549 23941 raft_consensus.cc:2804] T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:22:30.166636 16293 raft_consensus.cc:2468] T e5ad4eb281564936a59f41e387a47156 P 90f3da57796b488c9087a2f523adf045 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 22.
I20260709 13:22:30.170792 16265 leader_election.cc:304] T ada5a004ba7c46819051e3ccec71ffdb P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: cb82cc7bd6db46d980c75237a1402860
I20260709 13:22:30.186383 16219 raft_consensus.cc:1240] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac [term 23 FOLLOWER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 22. Current term is 23. Ops: []
I20260709 13:22:30.205657 16824 raft_consensus.cc:1275] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860 [term 20 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 18 index: 15. Preceding OpId from leader: term: 20 index: 16. (index mismatch)
I20260709 13:22:30.202257 16825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66f75124215f4ca6bef831e174a792d8" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:22:30.204962 16223 raft_consensus.cc:2468] T 8a878999c0024d2abe517484180290fc P 1ca2725d290e47d68b1a4b609c9816ac [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 20.
I20260709 13:22:30.207501 16825 raft_consensus.cc:2393] T 66f75124215f4ca6bef831e174a792d8 P cb82cc7bd6db46d980c75237a1402860 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 20: Already voted for candidate cb82cc7bd6db46d980c75237a1402860 in this term.
I20260709 13:22:30.211834 16818 raft_consensus.cc:1275] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860 [term 19 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 18 index: 15. Preceding OpId from leader: term: 19 index: 16. (index mismatch)
W20260709 13:22:30.207937 24170 log.cc:927] Time spent T 8cc959fb7f704fba9009c0595467d074 P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 0.069s	user 0.000s	sys 0.001s
I20260709 13:22:30.215903 23287 raft_consensus.cc:2764] T ada5a004ba7c46819051e3ccec71ffdb P 90f3da57796b488c9087a2f523adf045 [term 24 FOLLOWER]: Leader election decision vote started in defunct term 23: won
I20260709 13:22:30.220907 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66ae176961b74813a58bc15c9c4c6f2a" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 28 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:22:30.134917 23988 raft_consensus.cc:2804] T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:22:30.127830 23997 consensus_queue.cc:1048] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.082s
I20260709 13:22:30.136595 24004 consensus_queue.cc:1059] T af57996a8be84d398f0f023e8d08ef77 P 647a343a542d4a139fe106c19095129b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.073s
I20260709 13:22:30.122874 24166 consensus_queue.cc:1048] T 8a878999c0024d2abe517484180290fc P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.136s
I20260709 13:22:30.209452 23865 raft_consensus.cc:2804] T 84f2da98864b4425bf7c0055951fdd4b P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:22:30.171518 24146 raft_consensus.cc:740] T e4dc6b0541ca431ebc757beac00521f1 P 90f3da57796b488c9087a2f523adf045 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:22:30.205634 24091 leader_election.cc:290] T 8cc959fb7f704fba9009c0595467d074 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 21 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:30.158543 24150 consensus_queue.cc:1048] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.083s
I20260709 13:22:30.225767 23988 raft_consensus.cc:493] T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:30.240751 16226 raft_consensus.cc:1240] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [term 27 FOLLOWER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 26. Current term is 27. Ops: []
I20260709 13:22:30.209615 23941 raft_consensus.cc:493] T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:30.214493 23915 raft_consensus.cc:493] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:30.214728 24163 raft_consensus.cc:493] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:30.252517 16294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87ac0d1f7e8a419f9bcbc04aee0e067e" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:30.255775 16222 raft_consensus.cc:2468] T 66f75124215f4ca6bef831e174a792d8 P 1ca2725d290e47d68b1a4b609c9816ac [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 20.
I20260709 13:22:30.257885 16225 raft_consensus.cc:2468] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 21.
I20260709 13:22:30.258572 16219 raft_consensus.cc:1275] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac [term 24 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 23 index: 17. Preceding OpId from leader: term: 24 index: 18. (index mismatch)
I20260709 13:22:30.206555 16115 leader_election.cc:304] T 8158bbd17b994d2eb96f560bd0f1b88d P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac
I20260709 13:22:30.277714 16219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66f75124215f4ca6bef831e174a792d8" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:30.278457 16219 raft_consensus.cc:2468] T 66f75124215f4ca6bef831e174a792d8 P 1ca2725d290e47d68b1a4b609c9816ac [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 20.
W20260709 13:22:30.260156 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 215238us
I20260709 13:22:30.209342 24149 consensus_queue.cc:1048] T cdbd78acb4cf46298e0de8ba0d130e19 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 17, Time since last communication: 0.033s
I20260709 13:22:30.227941 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02838c86531d4f98bf58d1c22b8baf77" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
W20260709 13:22:30.269868 16117 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 44479.6us
I20260709 13:22:30.243932 23865 raft_consensus.cc:493] T 84f2da98864b4425bf7c0055951fdd4b P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:22:30.282419 16222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada5a004ba7c46819051e3ccec71ffdb" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:30.290299 16225 raft_consensus.cc:1240] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 20. Current term is 21. Ops: []
I20260709 13:22:30.288617 16222 raft_consensus.cc:2468] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 23.
I20260709 13:22:30.292471 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdbd78acb4cf46298e0de8ba0d130e19" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 26 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:30.293133 16226 raft_consensus.cc:2468] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 25.
I20260709 13:22:30.290382 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f5f89186204242bf377f7c42b2e2da" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 26 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:30.270397 23988 raft_consensus.cc:3060] T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045 [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:22:30.294446 16115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 88031.5us
I20260709 13:22:30.276883 23941 raft_consensus.cc:3060] T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:22:30.296398 16294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8158bbd17b994d2eb96f560bd0f1b88d" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:22:30.281447 24163 raft_consensus.cc:3060] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:22:30.270018 23994 raft_consensus.cc:493] T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:30.289417 23865 raft_consensus.cc:3060] T 84f2da98864b4425bf7c0055951fdd4b P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:22:30.302371 16294 raft_consensus.cc:2393] T 8158bbd17b994d2eb96f560bd0f1b88d P 90f3da57796b488c9087a2f523adf045 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 21: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
W20260709 13:22:30.308640 16190 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:22:30.312741 16218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746aabf856ab401c9ef5e859371e7b67" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:30.275112 24088 consensus_queue.cc:1048] T ee9ed67006144797b1de6ea8c2013a68 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.032s
I20260709 13:22:30.283342 24091 consensus_queue.cc:1048] T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.024s
I20260709 13:22:30.304152 16293 raft_consensus.cc:1275] T d2ab0ac8347641e6955e9bce7d90c1a2 P 90f3da57796b488c9087a2f523adf045 [term 21 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 20 index: 16. Preceding OpId from leader: term: 21 index: 17. (index mismatch)
W20260709 13:22:30.294951 16192 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 85623.7us
I20260709 13:22:30.259891 23996 consensus_queue.cc:260] T 68a6838d82ef4cac83507f906139d90f P 90f3da57796b488c9087a2f523adf045 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.12, Last appended by leader: 12, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:30.283754 23915 raft_consensus.cc:515] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:30.349556 16226 raft_consensus.cc:1240] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [term 28 FOLLOWER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 27. Current term is 28. Ops: []
I20260709 13:22:30.362277 16218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada5a004ba7c46819051e3ccec71ffdb" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:22:30.371537 16218 raft_consensus.cc:3060] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac [term 23 FOLLOWER]: Advancing to term 24
W20260709 13:22:30.350937 24167 log.cc:927] Time spent T fffab26427e24d398e2695aad40ca134 P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 0.084s	user 0.000s	sys 0.000s
I20260709 13:22:30.315400 24187 raft_consensus.cc:2804] T 8158bbd17b994d2eb96f560bd0f1b88d P 647a343a542d4a139fe106c19095129b [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:22:30.377524 16219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8638ee363273426db1393dd81e5c7b72" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 23 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:30.316190 23994 raft_consensus.cc:515] T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:30.389693 16825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a378d7f796f94615b3b347119a9880fd" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:30.369920 23996 raft_consensus.cc:3060] T 68a6838d82ef4cac83507f906139d90f P 90f3da57796b488c9087a2f523adf045 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:22:30.375378 24141 raft_consensus.cc:2749] T 26354361e9b044cea1c2b86c4882fe92 P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Leader election lost for term 18. Reason: Vote denied by peer cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923) with higher term. Message: Invalid argument: T 26354361e9b044cea1c2b86c4882fe92 P cb82cc7bd6db46d980c75237a1402860 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b for earlier term 18. Current term is 19.
I20260709 13:22:30.394760 16825 raft_consensus.cc:2468] T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 22.
I20260709 13:22:30.416028 16299 raft_consensus.cc:2468] T c0eab25ce4c64e018a041beeda2867db P 90f3da57796b488c9087a2f523adf045 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 24.
I20260709 13:22:30.383946 24129 leader_election.cc:290] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:30.397071 24004 consensus_queue.cc:1243] T af57996a8be84d398f0f023e8d08ef77 P 647a343a542d4a139fe106c19095129b [LEADER]: Peer 1ca2725d290e47d68b1a4b609c9816ac log is divergent from this leader: its last log entry 22.14 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 13
I20260709 13:22:30.439383 16819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3884f864a96d428785895e5463e5728f" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 18 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:30.313601 24146 consensus_queue.cc:260] T e4dc6b0541ca431ebc757beac00521f1 P 90f3da57796b488c9087a2f523adf045 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 22.12, Last appended by leader: 12, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:30.388756 24187 raft_consensus.cc:697] T 8158bbd17b994d2eb96f560bd0f1b88d P 647a343a542d4a139fe106c19095129b [term 21 LEADER]: Becoming Leader. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:22:30.440238 16192 leader_election.cc:304] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:22:30.363776 23798 consensus_queue.cc:1048] T 746aabf856ab401c9ef5e859371e7b67 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.040s
W20260709 13:22:30.424183 16115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 127688us
I20260709 13:22:30.448348 16115 leader_election.cc:304] T 57a744f5aaf944678a63469cbf514015 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860
I20260709 13:22:30.397045 23975 leader_election.cc:290] T 3884f864a96d428785895e5463e5728f P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:30.443773 16297 raft_consensus.cc:2468] T c4f65cdf00b748bb963ee6e81ef43b79 P 90f3da57796b488c9087a2f523adf045 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 23.
I20260709 13:22:30.451038 16218 raft_consensus.cc:2468] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 24.
W20260709 13:22:30.452853 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 282116us
I20260709 13:22:30.455258 16265 leader_election.cc:304] T ecc1e363230043fe9fef390f0b979f20 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 
W20260709 13:22:30.459508 24172 log.cc:927] Time spent T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 0.209s	user 0.004s	sys 0.004s
W20260709 13:22:30.459863 24174 log.cc:927] Time spent T d2ab0ac8347641e6955e9bce7d90c1a2 P cb82cc7bd6db46d980c75237a1402860: Append to log took a long time: real 0.148s	user 0.000s	sys 0.000s
I20260709 13:22:30.460250 23915 leader_election.cc:290] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:30.460914 24184 raft_consensus.cc:493] T b87147731a68400e8cce9b879daa7c5d P 647a343a542d4a139fe106c19095129b [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:30.464464 16819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5718c623ebee4d2bb770f0f9ddb2f164" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:30.465860 16265 leader_election.cc:304] T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:22:30.468334 16291 raft_consensus.cc:2468] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 90f3da57796b488c9087a2f523adf045 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 23.
W20260709 13:22:30.471033 24178 log.cc:927] Time spent T a378d7f796f94615b3b347119a9880fd P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.187s	user 0.001s	sys 0.000s
I20260709 13:22:30.443859 24004 raft_consensus.cc:493] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:22:30.472815 16265 leader_election.cc:336] T 26354361e9b044cea1c2b86c4882fe92 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 19 election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:22:30.497246 24186 raft_consensus.cc:2804] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac [term 22 FOLLOWER]: Leader pre-election won for term 23
W20260709 13:22:30.503332 24169 log.cc:927] Time spent T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.162s	user 0.001s	sys 0.000s
W20260709 13:22:30.504534 24181 log.cc:927] Time spent T af57996a8be84d398f0f023e8d08ef77 P cb82cc7bd6db46d980c75237a1402860: Append to log took a long time: real 0.156s	user 0.001s	sys 0.000s
I20260709 13:22:30.504630 24104 raft_consensus.cc:3055] T 02838c86531d4f98bf58d1c22b8baf77 P 647a343a542d4a139fe106c19095129b [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:22:30.505373 24187 consensus_queue.cc:237] T 8158bbd17b994d2eb96f560bd0f1b88d P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:22:30.506206 24182 log.cc:927] Time spent T 5a959dbecc2648eb85d1cc550f5a3245 P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 0.074s	user 0.000s	sys 0.000s
W20260709 13:22:30.507412 24183 log.cc:927] Time spent T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 0.178s	user 0.001s	sys 0.000s
I20260709 13:22:30.508599 24176 raft_consensus.cc:3055] T af57996a8be84d398f0f023e8d08ef77 P 647a343a542d4a139fe106c19095129b [term 21 LEADER]: Stepping down as leader of term 21
W20260709 13:22:30.509739 24185 log.cc:927] Time spent T 26354361e9b044cea1c2b86c4882fe92 P cb82cc7bd6db46d980c75237a1402860: Append to log took a long time: real 0.109s	user 0.000s	sys 0.000s
I20260709 13:22:30.510046 23924 raft_consensus.cc:515] T 18ad7e3af5504571a69d10c00750a01c P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } }
W20260709 13:22:30.510591 24177 log.cc:927] Time spent T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 0.197s	user 0.001s	sys 0.000s
W20260709 13:22:30.513690 24085 consensus_meta.cc:287] T 02838c86531d4f98bf58d1c22b8baf77 P 90f3da57796b488c9087a2f523adf045: Time spent flushing consensus metadata: real 0.530s	user 0.006s	sys 0.002s
W20260709 13:22:30.515626 24173 log.cc:927] Time spent T 7c2095a5f4ad49408dbc7f69c606120d P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 0.203s	user 0.000s	sys 0.001s
I20260709 13:22:30.443419 24141 raft_consensus.cc:493] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 647a343a542d4a139fe106c19095129b [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
W20260709 13:22:30.520744 16191 leader_election.cc:336] T af57996a8be84d398f0f023e8d08ef77 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:22:30.566052 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 45355.4us
I20260709 13:22:30.473070 24146 raft_consensus.cc:3060] T e4dc6b0541ca431ebc757beac00521f1 P 90f3da57796b488c9087a2f523adf045 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:22:30.467736 24149 raft_consensus.cc:2804] T ecc1e363230043fe9fef390f0b979f20 P 90f3da57796b488c9087a2f523adf045 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 13:22:30.526599 16819 raft_consensus.cc:1275] T 829b4ad8c5a54eababeea1181b960d4b P cb82cc7bd6db46d980c75237a1402860 [term 21 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 21 index: 16. (index mismatch)
I20260709 13:22:30.474248 24166 raft_consensus.cc:2804] T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:22:30.461540 24184 raft_consensus.cc:515] T b87147731a68400e8cce9b879daa7c5d P 647a343a542d4a139fe106c19095129b [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:22:30.531440 15046 thread.cc:527] Waited for 2000ms trying to join with heartbeat (tid 16176)
W20260709 13:22:30.529762 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 56981.8us
I20260709 13:22:30.472047 24150 consensus_queue.cc:1048] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.057s
I20260709 13:22:30.518965 24091 raft_consensus.cc:2749] T 57a744f5aaf944678a63469cbf514015 P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:22:30.536201 24104 raft_consensus.cc:740] T 02838c86531d4f98bf58d1c22b8baf77 P 647a343a542d4a139fe106c19095129b [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:22:30.538417 23994 leader_election.cc:290] T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:30.523543 23975 raft_consensus.cc:493] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:22:30.547911 24179 log.cc:927] Time spent T 3a7ca61592e848ea9b0fb81a2ce33cdc P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 0.243s	user 0.000s	sys 0.001s
I20260709 13:22:30.547732 24196 consensus_queue.cc:1048] T ee9ed67006144797b1de6ea8c2013a68 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 13:22:30.530225 24186 raft_consensus.cc:493] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:30.534685 24193 raft_consensus.cc:493] T d16dfad0c3b540e5b4b86492ddf87208 P 647a343a542d4a139fe106c19095129b [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:30.554863 16116 leader_election.cc:304] T b6f07842d3aa47858df5665874c86f3a P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
I20260709 13:22:30.493456 24004 raft_consensus.cc:515] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:30.534945 23798 raft_consensus.cc:493] T 1df93779209748b9860f6d893dd3de53 P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:22:30.571046 16191 leader_election.cc:336] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:22:30.616129 16191 leader_election.cc:304] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
W20260709 13:22:30.617532 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 46507.4us
W20260709 13:22:30.618582 16191 leader_election.cc:336] T c0eab25ce4c64e018a041beeda2867db P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:22:30.545977 24176 raft_consensus.cc:740] T af57996a8be84d398f0f023e8d08ef77 P 647a343a542d4a139fe106c19095129b [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:22:30.569870 24149 raft_consensus.cc:493] T ecc1e363230043fe9fef390f0b979f20 P 90f3da57796b488c9087a2f523adf045 [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:30.568794 24085 raft_consensus.cc:2749] T 02838c86531d4f98bf58d1c22b8baf77 P 90f3da57796b488c9087a2f523adf045 [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:22:30.571566 24166 raft_consensus.cc:493] T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:30.632104 24176 consensus_queue.cc:260] T af57996a8be84d398f0f023e8d08ef77 P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 21.14, Last appended by leader: 14, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:30.593892 24141 raft_consensus.cc:515] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 647a343a542d4a139fe106c19095129b [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:30.610141 24184 leader_election.cc:290] T b87147731a68400e8cce9b879daa7c5d P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
W20260709 13:22:30.611694 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 56920us
I20260709 13:22:30.560046 24140 consensus_queue.cc:1048] T 08c74490c40243a88e62d1760b5cd814 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.043s
I20260709 13:22:30.610759 24193 raft_consensus.cc:515] T d16dfad0c3b540e5b4b86492ddf87208 P 647a343a542d4a139fe106c19095129b [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:30.561408 23988 raft_consensus.cc:515] T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:30.621786 23980 raft_consensus.cc:2749] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:22:30.622030 24149 raft_consensus.cc:3060] T ecc1e363230043fe9fef390f0b979f20 P 90f3da57796b488c9087a2f523adf045 [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:22:30.641336 24190 raft_consensus.cc:493] T 434bff3082d141838068f323bdb0d1ed P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:22:30.641945 24190 raft_consensus.cc:515] T 434bff3082d141838068f323bdb0d1ed P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:30.644209 24190 leader_election.cc:290] T 434bff3082d141838068f323bdb0d1ed P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:30.619685 24091 raft_consensus.cc:2804] T b6f07842d3aa47858df5665874c86f3a P 647a343a542d4a139fe106c19095129b [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:22:30.612641 24186 raft_consensus.cc:3060] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:22:30.602671 24120 consensus_queue.cc:1048] T 829b4ad8c5a54eababeea1181b960d4b P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.027s
I20260709 13:22:30.626392 23798 raft_consensus.cc:515] T 1df93779209748b9860f6d893dd3de53 P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:30.661106 23798 leader_election.cc:290] T 1df93779209748b9860f6d893dd3de53 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:30.610201 23975 raft_consensus.cc:515] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:30.661864 24004 leader_election.cc:290] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:30.663492 23975 leader_election.cc:290] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:30.645748 24196 raft_consensus.cc:493] T 23f82224f4564fea93572e8206bad1fe P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:30.665356 24196 raft_consensus.cc:515] T 23f82224f4564fea93572e8206bad1fe P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:30.653662 24163 raft_consensus.cc:515] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:30.634598 24166 raft_consensus.cc:3060] T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:22:30.657101 24193 leader_election.cc:290] T d16dfad0c3b540e5b4b86492ddf87208 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:30.647147 24091 raft_consensus.cc:493] T b6f07842d3aa47858df5665874c86f3a P 647a343a542d4a139fe106c19095129b [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:30.683604 24091 raft_consensus.cc:3060] T b6f07842d3aa47858df5665874c86f3a P 647a343a542d4a139fe106c19095129b [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:22:30.685379 23865 raft_consensus.cc:515] T 84f2da98864b4425bf7c0055951fdd4b P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:30.687729 23865 leader_election.cc:290] T 84f2da98864b4425bf7c0055951fdd4b P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 20 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:22:30.689949 24141 leader_election.cc:290] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:30.690768 16825 raft_consensus.cc:1275] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 [term 27 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 25 index: 16. Preceding OpId from leader: term: 27 index: 17. (index mismatch)
W20260709 13:22:30.624393 16191 leader_election.cc:336] T 18ad7e3af5504571a69d10c00750a01c P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:22:30.646809 24104 consensus_queue.cc:260] T 02838c86531d4f98bf58d1c22b8baf77 P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 22.13, Last appended by leader: 13, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:30.680027 23924 leader_election.cc:290] T 18ad7e3af5504571a69d10c00750a01c P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 19 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
W20260709 13:22:30.682199 24125 consensus_meta.cc:287] T 8638ee363273426db1393dd81e5c7b72 P cb82cc7bd6db46d980c75237a1402860: Time spent flushing consensus metadata: real 0.540s	user 0.007s	sys 0.002s
I20260709 13:22:30.697855 23988 leader_election.cc:290] T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 22 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:22:30.698032 24125 raft_consensus.cc:515] T 8638ee363273426db1393dd81e5c7b72 P cb82cc7bd6db46d980c75237a1402860 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:30.699684 16817 raft_consensus.cc:1240] T 8638ee363273426db1393dd81e5c7b72 P cb82cc7bd6db46d980c75237a1402860 [term 23 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 22. Current term is 23. Ops: []
I20260709 13:22:30.700608 24104 raft_consensus.cc:3060] T 02838c86531d4f98bf58d1c22b8baf77 P 647a343a542d4a139fe106c19095129b [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:22:30.701581 24125 leader_election.cc:290] T 8638ee363273426db1393dd81e5c7b72 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 23 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
W20260709 13:22:30.692641 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 68307.5us
W20260709 13:22:30.709314 16191 leader_election.cc:336] T 829b4ad8c5a54eababeea1181b960d4b P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:22:30.710764 24163 leader_election.cc:290] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 25 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
W20260709 13:22:30.711943 16191 leader_election.cc:336] T e4dc6b0541ca431ebc757beac00521f1 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:22:30.718295 16191 leader_election.cc:304] T e4dc6b0541ca431ebc757beac00521f1 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
W20260709 13:22:30.717365 16266 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:22:30.737344 16191 leader_election.cc:336] T af57996a8be84d398f0f023e8d08ef77 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:22:30.722393 16190 leader_election.cc:304] T e5ad4eb281564936a59f41e387a47156 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:22:30.738224 16291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18ad7e3af5504571a69d10c00750a01c" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:22:30.738531 16190 leader_election.cc:304] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 28 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860
I20260709 13:22:30.738824 16291 raft_consensus.cc:3060] T 18ad7e3af5504571a69d10c00750a01c P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:22:30.659047 24099 leader_election.cc:290] T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:30.745517 16224 raft_consensus.cc:1275] T 84f2da98864b4425bf7c0055951fdd4b P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 13:22:30.747655 16820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2372981006d4b218215c28964dea623" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 27 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:30.748430 16820 raft_consensus.cc:2393] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 27: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
I20260709 13:22:30.749213 16820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5d59407eb545b895d8d922e0dd69fd" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:30.753829 24163 consensus_queue.cc:1048] T cdbd78acb4cf46298e0de8ba0d130e19 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.000s
W20260709 13:22:30.754729 16264 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:22:30.640059 23941 raft_consensus.cc:515] T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:30.728360 24145 raft_consensus.cc:493] T 8d4ab870a05347868956aa8b841b2981 P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
W20260709 13:22:30.729524 24159 consensus_meta.cc:287] T e5ad4eb281564936a59f41e387a47156 P 647a343a542d4a139fe106c19095129b: Time spent flushing consensus metadata: real 0.596s	user 0.011s	sys 0.000s
I20260709 13:22:30.734018 16824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87147731a68400e8cce9b879daa7c5d" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 23 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:30.734776 16822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434bff3082d141838068f323bdb0d1ed" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:30.781103 16190 leader_election.cc:304] T c0eab25ce4c64e018a041beeda2867db P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:22:30.717391 16825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:22:30.737473 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a378d7f796f94615b3b347119a9880fd" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:30.741335 16116 leader_election.cc:304] T fffab26427e24d398e2695aad40ca134 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:22:30.782575 16825 raft_consensus.cc:3060] T 19baa7bc367844c288bca8f81439c413 P cb82cc7bd6db46d980c75237a1402860 [term 24 FOLLOWER]: Advancing to term 25
W20260709 13:22:30.742584 16191 leader_election.cc:336] T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:22:30.784571 16190 leader_election.cc:304] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
W20260709 13:22:30.784890 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 43632.6us
I20260709 13:22:30.785223 16218 raft_consensus.cc:1275] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [term 27 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 26 index: 16. Preceding OpId from leader: term: 27 index: 17. (index mismatch)
I20260709 13:22:30.720448 24091 raft_consensus.cc:515] T b6f07842d3aa47858df5665874c86f3a P 647a343a542d4a139fe106c19095129b [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:30.799360 16222 raft_consensus.cc:1275] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 20 index: 16. Preceding OpId from leader: term: 21 index: 17. (index mismatch)
I20260709 13:22:30.800631 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d16dfad0c3b540e5b4b86492ddf87208" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 23 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:30.801783 16225 raft_consensus.cc:2410] T d16dfad0c3b540e5b4b86492ddf87208 P 1ca2725d290e47d68b1a4b609c9816ac [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b for term 23 because replica has last-logged OpId of term: 21 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 11.
I20260709 13:22:30.759761 16223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4f65cdf00b748bb963ee6e81ef43b79" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:30.785732 16116 leader_election.cc:304] T 8a878999c0024d2abe517484180290fc P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
I20260709 13:22:30.804795 16116 leader_election.cc:304] T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: cb82cc7bd6db46d980c75237a1402860
W20260709 13:22:30.791435 16264 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:22:30.792460 16191 leader_election.cc:336] T fffab26427e24d398e2695aad40ca134 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:22:30.805963 23287 raft_consensus.cc:493] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:22:30.716569 24149 raft_consensus.cc:515] T ecc1e363230043fe9fef390f0b979f20 P 90f3da57796b488c9087a2f523adf045 [term 26 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:30.786149 16117 leader_election.cc:304] T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860
I20260709 13:22:30.784685 16790 leader_election.cc:304] T 66f75124215f4ca6bef831e174a792d8 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:22:30.806455 16191 leader_election.cc:304] T fffab26427e24d398e2695aad40ca134 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:22:30.841547 23287 raft_consensus.cc:515] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:30.872997 16217 raft_consensus.cc:1240] T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac [term 24 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 23. Current term is 24. Ops: []
I20260709 13:22:30.805934 16116 leader_election.cc:304] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
I20260709 13:22:30.797653 16291 raft_consensus.cc:2468] T 18ad7e3af5504571a69d10c00750a01c P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 19.
I20260709 13:22:30.807649 16220 raft_consensus.cc:1240] T 02838c86531d4f98bf58d1c22b8baf77 P 1ca2725d290e47d68b1a4b609c9816ac [term 23 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 22. Current term is 23. Ops: []
I20260709 13:22:30.878728 16220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f5f89186204242bf377f7c42b2e2da" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:30.879690 16220 raft_consensus.cc:2468] T c2f5f89186204242bf377f7c42b2e2da P 1ca2725d290e47d68b1a4b609c9816ac [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 25.
I20260709 13:22:30.880851 23287 leader_election.cc:290] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:30.881055 16220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1df93779209748b9860f6d893dd3de53" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:30.807996 16824 raft_consensus.cc:3055] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 [term 20 LEADER]: Stepping down as leader of term 20
W20260709 13:22:30.806821 16036 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet ecc1e363230043fe9fef390f0b979f20 with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER failed with leader cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica cb82cc7bd6db46d980c75237a1402860 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 25 committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:22:30.882789 16824 raft_consensus.cc:740] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
W20260709 13:22:30.882936 16036 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.ChangeConfig -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 76162.7us
I20260709 13:22:30.883999 16824 consensus_queue.cc:260] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 20.16, Last appended by leader: 16, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:30.885358 16824 raft_consensus.cc:3060] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:22:30.881978 16220 raft_consensus.cc:2468] T 1df93779209748b9860f6d893dd3de53 P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 19.
I20260709 13:22:30.890129 16220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5d59407eb545b895d8d922e0dd69fd" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:30.780020 24120 raft_consensus.cc:2749] T e4dc6b0541ca431ebc757beac00521f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:22:30.807395 23915 raft_consensus.cc:2804] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:22:30.895838 24120 raft_consensus.cc:2749] T fffab26427e24d398e2695aad40ca134 P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:22:30.896032 23915 raft_consensus.cc:697] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac [term 23 LEADER]: Becoming Leader. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:22:30.789124 24180 consensus_queue.cc:1048] T d2ab0ac8347641e6955e9bce7d90c1a2 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.108s
I20260709 13:22:30.897199 23915 consensus_queue.cc:237] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 21.14, Last appended by leader: 14, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:30.789862 24099 raft_consensus.cc:2749] T fffab26427e24d398e2695aad40ca134 P 647a343a542d4a139fe106c19095129b [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:22:30.798422 23980 raft_consensus.cc:2804] T c0eab25ce4c64e018a041beeda2867db P 1ca2725d290e47d68b1a4b609c9816ac [term 24 FOLLOWER]: Leader election won for term 24
I20260709 13:22:30.899322 23980 raft_consensus.cc:697] T c0eab25ce4c64e018a041beeda2867db P 1ca2725d290e47d68b1a4b609c9816ac [term 24 LEADER]: Becoming Leader. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:22:30.757155 24130 raft_consensus.cc:2749] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [term 28 FOLLOWER]: Leader election lost for term 28. Reason: could not achieve majority
I20260709 13:22:30.756939 23924 raft_consensus.cc:2804] T e5ad4eb281564936a59f41e387a47156 P 1ca2725d290e47d68b1a4b609c9816ac [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:22:30.779953 23988 raft_consensus.cc:493] T d2ab0ac8347641e6955e9bce7d90c1a2 P 90f3da57796b488c9087a2f523adf045 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:22:30.913419 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 107548us
I20260709 13:22:30.915390 16218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87ac0d1f7e8a419f9bcbc04aee0e067e" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
W20260709 13:22:30.905983 16117 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 119900us
I20260709 13:22:30.904361 16190 leader_election.cc:304] T 18ad7e3af5504571a69d10c00750a01c P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:22:30.906755 16825 raft_consensus.cc:2468] T 19baa7bc367844c288bca8f81439c413 P cb82cc7bd6db46d980c75237a1402860 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 25.
I20260709 13:22:30.913576 16816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806d7690c2b644959152074bb2cab0a7" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:30.924180 16820 raft_consensus.cc:1275] T 3a7ca61592e848ea9b0fb81a2ce33cdc P cb82cc7bd6db46d980c75237a1402860 [term 23 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 21 index: 16. Preceding OpId from leader: term: 23 index: 17. (index mismatch)
I20260709 13:22:30.780054 24113 consensus_queue.cc:1059] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: INVALID_TERM, Last received: 22.17, Next index: 18, Last known committed idx: 15, Time since last communication: 0.151s
I20260709 13:22:30.872923 24168 raft_consensus.cc:2804] T 8a878999c0024d2abe517484180290fc P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:22:30.873111 24193 raft_consensus.cc:2804] T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:22:30.741813 24166 raft_consensus.cc:515] T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:30.945524 16300 raft_consensus.cc:1275] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 90f3da57796b488c9087a2f523adf045 [term 23 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 21 index: 16. Preceding OpId from leader: term: 23 index: 17. (index mismatch)
I20260709 13:22:30.929075 16824 raft_consensus.cc:1275] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 [term 21 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 20 index: 16. Preceding OpId from leader: term: 21 index: 17. (index mismatch)
I20260709 13:22:30.873338 24096 consensus_queue.cc:1048] T c9944d5e4b8c4109af9955c4f08c9095 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.119s
I20260709 13:22:30.808944 24142 consensus_queue.cc:1059] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: INVALID_TERM, Last received: 26.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.125s
I20260709 13:22:30.924039 23924 raft_consensus.cc:697] T e5ad4eb281564936a59f41e387a47156 P 1ca2725d290e47d68b1a4b609c9816ac [term 22 LEADER]: Becoming Leader. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:22:30.930016 24099 raft_consensus.cc:2804] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:22:30.789584 24159 raft_consensus.cc:515] T e5ad4eb281564936a59f41e387a47156 P 647a343a542d4a139fe106c19095129b [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:30.789696 24145 raft_consensus.cc:515] T 8d4ab870a05347868956aa8b841b2981 P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:22:31.038292 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 63552.1us
I20260709 13:22:30.930562 24104 raft_consensus.cc:3060] T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b [term 26 FOLLOWER]: Advancing to term 27
W20260709 13:22:31.050357 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 266428us
I20260709 13:22:30.933336 23958 raft_consensus.cc:2804] T 18ad7e3af5504571a69d10c00750a01c P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Leader election won for term 19
W20260709 13:22:31.075256 16191 leader_election.cc:336] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:22:31.079144 16191 leader_election.cc:336] T 1df93779209748b9860f6d893dd3de53 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:22:31.079769 16191 leader_election.cc:304] T 1df93779209748b9860f6d893dd3de53 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
W20260709 13:22:31.094362 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 309729us
I20260709 13:22:30.920049 23988 raft_consensus.cc:515] T d2ab0ac8347641e6955e9bce7d90c1a2 P 90f3da57796b488c9087a2f523adf045 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:31.101781 16300 raft_consensus.cc:1275] T 31969392ccdb43b491911a319e8bc7e1 P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 18 index: 15. Preceding OpId from leader: term: 20 index: 16. (index mismatch)
I20260709 13:22:31.102895 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ad5dbfa99ee4b8d9e28be92b3f36db5" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:22:31.104293 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a878999c0024d2abe517484180290fc" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:22:31.106112 23287 raft_consensus.cc:493] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:22:31.077214 24189 log.cc:927] Time spent T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860: Append to log took a long time: real 0.134s	user 0.000s	sys 0.001s
I20260709 13:22:31.106591 23287 raft_consensus.cc:515] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:31.108909 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6f07842d3aa47858df5665874c86f3a" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:31.116289 16299 raft_consensus.cc:2393] T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 22: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
W20260709 13:22:31.111397 16192 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 191439us
I20260709 13:22:31.096174 16790 leader_election.cc:304] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 90f3da57796b488c9087a2f523adf045
I20260709 13:22:31.047107 24168 raft_consensus.cc:697] T 8a878999c0024d2abe517484180290fc P 647a343a542d4a139fe106c19095129b [term 20 LEADER]: Becoming Leader. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:22:31.048372 24125 raft_consensus.cc:2804] T 66f75124215f4ca6bef831e174a792d8 P cb82cc7bd6db46d980c75237a1402860 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:22:31.042484 23994 consensus_queue.cc:1048] T 84f2da98864b4425bf7c0055951fdd4b P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.066s
I20260709 13:22:31.049029 24129 consensus_queue.cc:1048] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.070s
W20260709 13:22:31.108237 24205 log.cc:927] Time spent T d2ab0ac8347641e6955e9bce7d90c1a2 P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 0.438s	user 0.001s	sys 0.000s
I20260709 13:22:31.094085 24120 raft_consensus.cc:3060] T 1df93779209748b9860f6d893dd3de53 P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:22:31.137678 16299 raft_consensus.cc:1275] T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045 [term 23 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 23 index: 13. (index mismatch)
I20260709 13:22:31.109649 23958 raft_consensus.cc:697] T 18ad7e3af5504571a69d10c00750a01c P 1ca2725d290e47d68b1a4b609c9816ac [term 19 LEADER]: Becoming Leader. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:22:31.160094 16300 raft_consensus.cc:1275] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [term 26 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 25 index: 17. Preceding OpId from leader: term: 26 index: 17. (term mismatch)
I20260709 13:22:31.168403 16300 pending_rounds.cc:85] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045: Aborting all ops after (but not including) 16
I20260709 13:22:31.168691 16300 pending_rounds.cc:107] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045: Aborting uncommitted NO_OP operation due to leader change: 25.17
I20260709 13:22:31.168916 16300 raft_consensus.cc:2889] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [term 26 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:22:31.170011 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4f65cdf00b748bb963ee6e81ef43b79" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
W20260709 13:22:31.160848 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 64750us
I20260709 13:22:31.161898 16218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8638ee363273426db1393dd81e5c7b72" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 23 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:22:31.110284 24085 raft_consensus.cc:493] T 1df93779209748b9860f6d893dd3de53 P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:31.101092 24099 raft_consensus.cc:697] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b [term 21 LEADER]: Becoming Leader. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:22:31.165431 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3884f864a96d428785895e5463e5728f" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 18 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:31.166973 16298 raft_consensus.cc:1275] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045 [term 21 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 20 index: 12. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
I20260709 13:22:31.138727 24125 raft_consensus.cc:493] T 66f75124215f4ca6bef831e174a792d8 P cb82cc7bd6db46d980c75237a1402860 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:31.207154 24099 consensus_queue.cc:237] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:31.159054 24130 raft_consensus.cc:493] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:31.162264 24193 raft_consensus.cc:697] T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b [term 20 LEADER]: Becoming Leader. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:22:31.178782 16293 raft_consensus.cc:2393] T c4f65cdf00b748bb963ee6e81ef43b79 P 90f3da57796b488c9087a2f523adf045 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 23: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
I20260709 13:22:31.212890 24125 raft_consensus.cc:3060] T 66f75124215f4ca6bef831e174a792d8 P cb82cc7bd6db46d980c75237a1402860 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:22:31.213358 24085 raft_consensus.cc:515] T 1df93779209748b9860f6d893dd3de53 P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:22:31.211670 16117 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 239982us
I20260709 13:22:31.174034 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cc959fb7f704fba9009c0595467d074" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:22:31.215602 16299 raft_consensus.cc:2410] T 3884f864a96d428785895e5463e5728f P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b for term 18 because replica has last-logged OpId of term: 17 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 14.
I20260709 13:22:31.162537 23815 raft_consensus.cc:493] T c9944d5e4b8c4109af9955c4f08c9095 P 90f3da57796b488c9087a2f523adf045 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
W20260709 13:22:31.221889 16266 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:22:31.223664 16117 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:22:31.232493 16221 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57600 (request call id 2308) took 1407 ms. Trace:
I20260709 13:22:31.222867 23287 leader_election.cc:290] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:31.234444 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87147731a68400e8cce9b879daa7c5d" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 23 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:31.232993 16221 rpcz_store.cc:276] 0709 13:22:29.813988 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:22:29.844889 (+ 30901us) service_pool.cc:224] Handling call
0709 13:22:29.845098 (+   209us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:22:29.878564 (+ 33466us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:22:29.878822 (+   258us) log.cc:844] Serialized 14 byte log entry
0709 13:22:30.155476 (+276654us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:22:30.155538 (+    62us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:22:30.155595 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:22:31.221361 (+1065766us) spinlock_profiling.cc:243] Waited 33.2 ms on lock 0x7b54002c8154. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:22:31.221445 (+    84us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:22:31.221610 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":551,"prepare.run_cpu_time_us":37,"prepare.run_wall_time_us":39,"raft.queue_time_us":93,"raft.run_cpu_time_us":157,"raft.run_wall_time_us":33946,"spinlock_wait_cycles":76458368,"thread_start_us":276301,"threads_started":1,"wal-append.queue_time_us":396746}
I20260709 13:22:31.241053 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c61a39fd4f84424b129c1a6f9ba404d" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:22:31.241955 16293 raft_consensus.cc:3060] T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:22:31.110809 23980 consensus_queue.cc:237] T c0eab25ce4c64e018a041beeda2867db P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 22.12, Last appended by leader: 12, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:31.247006 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434bff3082d141838068f323bdb0d1ed" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
W20260709 13:22:31.256812 16789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 149201us
I20260709 13:22:31.221100 24226 raft_consensus.cc:493] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:22:31.268908 16264 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:22:31.216777 23924 consensus_queue.cc:237] T e5ad4eb281564936a59f41e387a47156 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.13, Last appended by leader: 13, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:31.259544 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d16dfad0c3b540e5b4b86492ddf87208" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 23 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:31.272024 16299 raft_consensus.cc:2410] T d16dfad0c3b540e5b4b86492ddf87208 P 90f3da57796b488c9087a2f523adf045 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b for term 23 because replica has last-logged OpId of term: 21 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 11.
I20260709 13:22:31.229434 24113 raft_consensus.cc:3055] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [term 26 LEADER]: Stepping down as leader of term 26
I20260709 13:22:31.245127 23941 leader_election.cc:290] T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 24 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:31.243660 23815 raft_consensus.cc:515] T c9944d5e4b8c4109af9955c4f08c9095 P 90f3da57796b488c9087a2f523adf045 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:31.233656 24196 leader_election.cc:290] T 23f82224f4564fea93572e8206bad1fe P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
W20260709 13:22:31.264729 24146 consensus_meta.cc:287] T e4dc6b0541ca431ebc757beac00521f1 P 90f3da57796b488c9087a2f523adf045: Time spent flushing consensus metadata: real 0.695s	user 0.013s	sys 0.007s
I20260709 13:22:31.274283 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1df93779209748b9860f6d893dd3de53" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:31.287459 16297 raft_consensus.cc:2468] T 1df93779209748b9860f6d893dd3de53 P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 20.
W20260709 13:22:31.287745 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 116135us
I20260709 13:22:31.231380 24130 raft_consensus.cc:515] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:31.279538 16294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f82224f4564fea93572e8206bad1fe" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:31.279853 16298 raft_consensus.cc:1275] T 8638ee363273426db1393dd81e5c7b72 P 90f3da57796b488c9087a2f523adf045 [term 22 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 20 index: 16. Preceding OpId from leader: term: 22 index: 17. (index mismatch)
I20260709 13:22:31.280556 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87ac0d1f7e8a419f9bcbc04aee0e067e" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
W20260709 13:22:31.288815 16264 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:22:31.289332 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f5f89186204242bf377f7c42b2e2da" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:31.274863 16789 leader_election.cc:304] T ada5a004ba7c46819051e3ccec71ffdb P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:22:31.294104 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8638ee363273426db1393dd81e5c7b72" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 23 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:22:31.294749 16298 raft_consensus.cc:3060] T 8638ee363273426db1393dd81e5c7b72 P 90f3da57796b488c9087a2f523adf045 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:22:31.273478 24113 raft_consensus.cc:740] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [term 26 LEADER]: Becoming Follower/Learner. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:22:31.316064 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e9b2bf7011f4ae688a79d7bb5a447b4" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 22 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:31.292330 24166 leader_election.cc:290] T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 25 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
W20260709 13:22:31.324018 16266 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:22:31.320071 16293 raft_consensus.cc:2468] T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 24.
I20260709 13:22:31.296017 24096 raft_consensus.cc:2804] T ada5a004ba7c46819051e3ccec71ffdb P cb82cc7bd6db46d980c75237a1402860 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 13:22:31.291700 24222 consensus_queue.cc:1059] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: INVALID_TERM, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.048s
W20260709 13:22:31.331717 24221 log.cc:927] Time spent T ee9ed67006144797b1de6ea8c2013a68 P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 0.070s	user 0.001s	sys 0.000s
I20260709 13:22:31.312240 24145 leader_election.cc:290] T 8d4ab870a05347868956aa8b841b2981 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:22:31.323764 24159 leader_election.cc:290] T e5ad4eb281564936a59f41e387a47156 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 22 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:31.354699 16298 raft_consensus.cc:2468] T 8638ee363273426db1393dd81e5c7b72 P 90f3da57796b488c9087a2f523adf045 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 23.
I20260709 13:22:31.266021 24168 consensus_queue.cc:237] T 8a878999c0024d2abe517484180290fc P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:31.336886 23988 leader_election.cc:290] T d2ab0ac8347641e6955e9bce7d90c1a2 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:31.273595 23958 consensus_queue.cc:237] T 18ad7e3af5504571a69d10c00750a01c P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } }
I20260709 13:22:31.347870 24096 raft_consensus.cc:697] T ada5a004ba7c46819051e3ccec71ffdb P cb82cc7bd6db46d980c75237a1402860 [term 24 LEADER]: Becoming Leader. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:22:31.287514 24226 raft_consensus.cc:515] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:31.354198 24222 raft_consensus.cc:3055] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [term 27 LEADER]: Stepping down as leader of term 27
I20260709 13:22:31.349320 24166 raft_consensus.cc:493] T 23f82224f4564fea93572e8206bad1fe P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:22:31.354933 24149 leader_election.cc:290] T ecc1e363230043fe9fef390f0b979f20 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 26 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:31.320201 24193 consensus_queue.cc:237] T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:22:31.395650 24209 log.cc:927] Time spent T 829b4ad8c5a54eababeea1181b960d4b P cb82cc7bd6db46d980c75237a1402860: Append to log took a long time: real 0.213s	user 0.001s	sys 0.000s
W20260709 13:22:31.397755 24218 log.cc:927] Time spent T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 0.084s	user 0.000s	sys 0.001s
I20260709 13:22:31.414311 16823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ab0ac8347641e6955e9bce7d90c1a2" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:31.383289 24085 leader_election.cc:290] T 1df93779209748b9860f6d893dd3de53 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:22:31.381265 24222 raft_consensus.cc:740] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [term 27 LEADER]: Becoming Follower/Learner. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
W20260709 13:22:31.393465 24220 log.cc:927] Time spent T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 0.141s	user 0.001s	sys 0.000s
I20260709 13:22:31.387210 24223 raft_consensus.cc:493] T 18ad7e3af5504571a69d10c00750a01c P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:31.448900 16823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 26 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:22:31.451612 16823 raft_consensus.cc:3060] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860 [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:22:31.394179 24104 raft_consensus.cc:2749] T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b [term 27 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 13:22:31.366170 24216 consensus_queue.cc:1048] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 14, Time since last communication: 0.190s
I20260709 13:22:31.370450 24113 consensus_queue.cc:260] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 26.16, Last appended by leader: 16, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:22:31.455710 16819 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.509s	user 0.002s	sys 0.010s
I20260709 13:22:31.393754 24166 raft_consensus.cc:515] T 23f82224f4564fea93572e8206bad1fe P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:22:31.483430 16822 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.558s	user 0.015s	sys 0.040s
W20260709 13:22:31.484212 16822 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.565s	user 0.015s	sys 0.041s
W20260709 13:22:31.454030 16192 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 252266us
I20260709 13:22:31.489323 16819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74ca8140f35c4ae29e8a664f6bdfc829" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:22:31.490131 16819 raft_consensus.cc:3060] T 74ca8140f35c4ae29e8a664f6bdfc829 P cb82cc7bd6db46d980c75237a1402860 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:22:31.491114 16822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a9de347c711477e87c582d003f36925" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 27 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:31.501485 16823 raft_consensus.cc:2468] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 26.
I20260709 13:22:31.502135 16822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f82224f4564fea93572e8206bad1fe" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:31.495257 23915 consensus_queue.cc:1048] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.012s
I20260709 13:22:31.488607 24130 leader_election.cc:290] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:31.516274 16823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9944d5e4b8c4109af9955c4f08c9095" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:31.515625 24226 leader_election.cc:290] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:31.528523 24130 raft_consensus.cc:493] T 5a96373bb0e14dce8a266b2fc6c58507 P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:22:31.529142 24130 raft_consensus.cc:515] T 5a96373bb0e14dce8a266b2fc6c58507 P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:22:31.530579 24186 consensus_meta.cc:287] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac: Time spent flushing consensus metadata: real 0.880s	user 0.015s	sys 0.000s
I20260709 13:22:31.531771 24130 leader_election.cc:290] T 5a96373bb0e14dce8a266b2fc6c58507 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:31.453174 24223 raft_consensus.cc:515] T 18ad7e3af5504571a69d10c00750a01c P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } }
W20260709 13:22:31.535914 24236 log.cc:927] Time spent T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 0.125s	user 0.001s	sys 0.000s
W20260709 13:22:31.559068 24234 log.cc:927] Time spent T d2ab0ac8347641e6955e9bce7d90c1a2 P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.130s	user 0.001s	sys 0.000s
I20260709 13:22:31.473176 24096 consensus_queue.cc:237] T ada5a004ba7c46819051e3ccec71ffdb P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 22.16, Last appended by leader: 16, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:31.559686 24190 raft_consensus.cc:493] T 68a6838d82ef4cac83507f906139d90f P 647a343a542d4a139fe106c19095129b [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:31.560649 23938 raft_consensus.cc:493] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:31.485117 24222 consensus_queue.cc:260] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 27.17, Last appended by leader: 17, Current term: 27, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:31.573771 24222 raft_consensus.cc:3060] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [term 27 FOLLOWER]: Advancing to term 28
I20260709 13:22:31.531061 24186 raft_consensus.cc:515] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:31.585390 24186 leader_election.cc:290] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 23 election: Requested vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:31.587671 23938 raft_consensus.cc:515] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:31.590349 23938 leader_election.cc:290] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
W20260709 13:22:31.591218 15046 thread.cc:527] Waited for 3000ms trying to join with heartbeat (tid 16176)
I20260709 13:22:31.579684 24190 raft_consensus.cc:515] T 68a6838d82ef4cac83507f906139d90f P 647a343a542d4a139fe106c19095129b [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:22:31.593354 24194 log.cc:927] Time spent T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860: Append to log took a long time: real 1.130s	user 0.005s	sys 0.000s
I20260709 13:22:31.594718 24161 raft_consensus.cc:3055] T 5a959dbecc2648eb85d1cc550f5a3245 P 90f3da57796b488c9087a2f523adf045 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:22:31.595134 24190 leader_election.cc:290] T 68a6838d82ef4cac83507f906139d90f P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:31.595223 24161 raft_consensus.cc:740] T 5a959dbecc2648eb85d1cc550f5a3245 P 90f3da57796b488c9087a2f523adf045 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:22:31.611714 16224 raft_consensus.cc:1240] T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 20. Current term is 21. Ops: []
I20260709 13:22:31.613647 16219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68a6838d82ef4cac83507f906139d90f" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 23 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:31.613587 16223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:31.633152 16223 raft_consensus.cc:2468] T 19baa7bc367844c288bca8f81439c413 P 1ca2725d290e47d68b1a4b609c9816ac [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 24.
I20260709 13:22:31.624379 16821 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49896 (request call id 1721) took 1414 ms. Trace:
W20260709 13:22:31.627800 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 337484us
I20260709 13:22:31.609876 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6f07842d3aa47858df5665874c86f3a" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:22:31.638633 16226 raft_consensus.cc:3060] T b6f07842d3aa47858df5665874c86f3a P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:22:31.638240 16822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:31.639912 16219 raft_consensus.cc:2410] T 68a6838d82ef4cac83507f906139d90f P 1ca2725d290e47d68b1a4b609c9816ac [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b for term 23 because replica has last-logged OpId of term: 21 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 11.
I20260709 13:22:31.635054 16821 rpcz_store.cc:276] 0709 13:22:30.210392 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:22:30.210590 (+   198us) service_pool.cc:224] Handling call
0709 13:22:30.215744 (+  5154us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:22:30.224624 (+  8880us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:22:30.224657 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:22:30.269150 (+ 44493us) log.cc:844] Serialized 26 byte log entry
0709 13:22:30.343257 (+ 74107us) raft_consensus.cc:1645] Marking committed up to 16
0709 13:22:30.343311 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:22:30.343408 (+    97us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:22:31.592744 (+1249336us) spinlock_profiling.cc:243] Waited 29.6 ms on lock 0x7b4400022b4c. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69ad8fadb1 00007f69ad998c85 00007f69ad9996a2 00007f69b070b1db 00007f69b0706aaf 00007f69b06fa86e 00007f69b06f9bb6 00007f69b06f97d8 00007f69b0703221 00007f69b07031d9 00007f69b0703169
0709 13:22:31.623998 (+ 31254us) raft_consensus.cc:1685] finished
0709 13:22:31.624034 (+    36us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:22:31.624293 (+   259us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":10696,"raft.run_cpu_time_us":135,"raft.run_wall_time_us":4152,"spinlock_wait_cycles":68044672,"thread_start_us":41558,"threads_started":1,"wal-append.queue_time_us":99839}
I20260709 13:22:31.610167 16221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5ad4eb281564936a59f41e387a47156" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:22:31.645728 16822 raft_consensus.cc:2393] T 19baa7bc367844c288bca8f81439c413 P cb82cc7bd6db46d980c75237a1402860 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 25: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
I20260709 13:22:31.647042 16822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f82224f4564fea93572e8206bad1fe" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:31.650417 23958 raft_consensus.cc:493] T 746aabf856ab401c9ef5e859371e7b67 P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:22:31.623194 24166 leader_election.cc:290] T 23f82224f4564fea93572e8206bad1fe P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:31.651296 23958 raft_consensus.cc:515] T 746aabf856ab401c9ef5e859371e7b67 P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:31.609875 24223 leader_election.cc:290] T 18ad7e3af5504571a69d10c00750a01c P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:22:31.646732 24125 raft_consensus.cc:515] T 66f75124215f4ca6bef831e174a792d8 P cb82cc7bd6db46d980c75237a1402860 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:31.655305 16116 leader_election.cc:304] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
I20260709 13:22:31.661492 16822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a378d7f796f94615b3b347119a9880fd" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:22:31.662387 16822 raft_consensus.cc:3060] T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860 [term 22 FOLLOWER]: Advancing to term 23
W20260709 13:22:31.662539 16790 leader_election.cc:336] T 66f75124215f4ca6bef831e174a792d8 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:22:31.626153 24162 raft_consensus.cc:3055] T 8d4ab870a05347868956aa8b841b2981 P 647a343a542d4a139fe106c19095129b [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:22:31.615806 24073 raft_consensus.cc:3060] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045 [term 23 FOLLOWER]: Advancing to term 24
W20260709 13:22:31.623927 24195 log.cc:927] Time spent T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 1.181s	user 0.020s	sys 0.004s
I20260709 13:22:31.609064 24192 raft_consensus.cc:493] T ecc1e363230043fe9fef390f0b979f20 P 647a343a542d4a139fe106c19095129b [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:31.606252 16819 raft_consensus.cc:2468] T 74ca8140f35c4ae29e8a664f6bdfc829 P cb82cc7bd6db46d980c75237a1402860 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 25.
I20260709 13:22:31.602396 24120 raft_consensus.cc:2749] T 1df93779209748b9860f6d893dd3de53 P 1ca2725d290e47d68b1a4b609c9816ac [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:22:31.620350 24113 raft_consensus.cc:3060] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [term 26 FOLLOWER]: Advancing to term 27
I20260709 13:22:31.673295 16823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b7d1ab519044eda2d5e04a07242f25" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:31.674260 16823 raft_consensus.cc:2468] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 21.
I20260709 13:22:31.675760 16192 leader_election.cc:304] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:22:31.665640 24162 raft_consensus.cc:740] T 8d4ab870a05347868956aa8b841b2981 P 647a343a542d4a139fe106c19095129b [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:22:31.613915 24226 raft_consensus.cc:493] T 829b4ad8c5a54eababeea1181b960d4b P 647a343a542d4a139fe106c19095129b [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:31.667439 16116 leader_election.cc:304] T 1df93779209748b9860f6d893dd3de53 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
I20260709 13:22:31.702652 16116 leader_election.cc:304] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860
I20260709 13:22:31.613940 24216 raft_consensus.cc:493] T 7c2095a5f4ad49408dbc7f69c606120d P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
W20260709 13:22:31.702217 16790 leader_election.cc:336] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:22:31.598500 24091 leader_election.cc:290] T b6f07842d3aa47858df5665874c86f3a P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 22 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:31.703897 16790 leader_election.cc:304] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:22:31.666743 24227 raft_consensus.cc:2804] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 13:22:31.705900 16822 raft_consensus.cc:2468] T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 23.
I20260709 13:22:31.708328 16192 leader_election.cc:304] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:22:31.617223 24168 consensus_queue.cc:1048] T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.062s
I20260709 13:22:31.617168 24193 consensus_queue.cc:1048] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.079s
W20260709 13:22:31.710438 16790 leader_election.cc:336] T b87147731a68400e8cce9b879daa7c5d P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:22:31.616716 24161 consensus_queue.cc:260] T 5a959dbecc2648eb85d1cc550f5a3245 P 90f3da57796b488c9087a2f523adf045 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.12, Last appended by leader: 12, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:31.715765 16790 leader_election.cc:304] T b87147731a68400e8cce9b879daa7c5d P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:22:31.694494 23938 raft_consensus.cc:2804] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:22:31.734036 16219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18ad7e3af5504571a69d10c00750a01c" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:31.710657 24223 raft_consensus.cc:2804] T 1df93779209748b9860f6d893dd3de53 P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:22:31.709722 24227 raft_consensus.cc:493] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:31.708231 24231 raft_consensus.cc:2749] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:22:31.717432 24228 raft_consensus.cc:2749] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:22:31.711367 23980 raft_consensus.cc:2804] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac [term 23 FOLLOWER]: Leader election won for term 23
W20260709 13:22:31.735056 16790 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 164 similar messages]
W20260709 13:22:31.753034 16264 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:22:31.731562 23938 raft_consensus.cc:493] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:31.738773 24238 raft_consensus.cc:2749] T b87147731a68400e8cce9b879daa7c5d P cb82cc7bd6db46d980c75237a1402860 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:22:31.714730 24192 raft_consensus.cc:515] T ecc1e363230043fe9fef390f0b979f20 P 647a343a542d4a139fe106c19095129b [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:31.744714 24161 raft_consensus.cc:3060] T 5a959dbecc2648eb85d1cc550f5a3245 P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:22:31.747915 24227 raft_consensus.cc:3060] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:22:31.726274 24226 raft_consensus.cc:515] T 829b4ad8c5a54eababeea1181b960d4b P 647a343a542d4a139fe106c19095129b [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:31.791028 24226 leader_election.cc:290] T 829b4ad8c5a54eababeea1181b960d4b P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:31.791807 16819 raft_consensus.cc:1240] T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860 [term 23 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 22. Current term is 23. Ops: []
W20260709 13:22:31.792945 16266 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:22:31.745498 24223 raft_consensus.cc:493] T 1df93779209748b9860f6d893dd3de53 P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:31.711413 24162 consensus_queue.cc:260] T 8d4ab870a05347868956aa8b841b2981 P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.12, Last appended by leader: 12, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:31.777881 23605 raft_consensus.cc:493] T 7c2095a5f4ad49408dbc7f69c606120d P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:31.778702 23938 raft_consensus.cc:3060] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:22:31.708122 24216 raft_consensus.cc:515] T 7c2095a5f4ad49408dbc7f69c606120d P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:31.802335 16219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "829b4ad8c5a54eababeea1181b960d4b" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 22 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:31.783896 23980 raft_consensus.cc:697] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac [term 23 LEADER]: Becoming Leader. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:22:31.800297 24223 raft_consensus.cc:3060] T 1df93779209748b9860f6d893dd3de53 P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:22:31.831621 16264 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:22:31.830982 24215 raft_consensus.cc:2804] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 [term 25 FOLLOWER]: Leader pre-election won for term 26
W20260709 13:22:31.834607 24248 log.cc:927] Time spent T c0eab25ce4c64e018a041beeda2867db P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 0.095s	user 0.000s	sys 0.001s
I20260709 13:22:31.835144 24216 leader_election.cc:290] T 7c2095a5f4ad49408dbc7f69c606120d P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:31.836675 16817 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49906 (request call id 1539) took 1173 ms. Trace:
I20260709 13:22:31.837122 16817 rpcz_store.cc:276] 0709 13:22:30.663561 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:22:30.666383 (+  2822us) service_pool.cc:224] Handling call
0709 13:22:30.670153 (+  3770us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:22:30.700122 (+ 29969us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:22:31.836364 (+1136242us) spinlock_profiling.cc:243] Waited 29.3 ms on lock 0x7b54007706d4. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:22:31.836617 (+   253us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":67416320}
I20260709 13:22:31.838209 23958 leader_election.cc:290] T 746aabf856ab401c9ef5e859371e7b67 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:31.840937 24192 leader_election.cc:290] T ecc1e363230043fe9fef390f0b979f20 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:31.840765 23980 consensus_queue.cc:237] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 21.14, Last appended by leader: 14, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:31.844336 23924 consensus_queue.cc:1059] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 13:22:31.845279 23924 consensus_queue.cc:1243] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Peer cb82cc7bd6db46d980c75237a1402860 log is divergent from this leader: its last log entry 21.17 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 16
I20260709 13:22:31.845798 23924 raft_consensus.cc:3055] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:22:31.846150 23924 raft_consensus.cc:740] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:22:31.846323 24125 leader_election.cc:290] T 66f75124215f4ca6bef831e174a792d8 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 21 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:22:31.802567 24197 consensus_queue.cc:1059] T 8d4ab870a05347868956aa8b841b2981 P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: INVALID_TERM, Last received: 20.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.047s
I20260709 13:22:31.848434 24073 raft_consensus.cc:493] T e4dc6b0541ca431ebc757beac00521f1 P 90f3da57796b488c9087a2f523adf045 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:31.849007 24073 raft_consensus.cc:515] T e4dc6b0541ca431ebc757beac00521f1 P 90f3da57796b488c9087a2f523adf045 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:31.819693 23605 raft_consensus.cc:515] T 7c2095a5f4ad49408dbc7f69c606120d P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:31.853314 23605 leader_election.cc:290] T 7c2095a5f4ad49408dbc7f69c606120d P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:31.854826 24145 raft_consensus.cc:493] T 68a6838d82ef4cac83507f906139d90f P 90f3da57796b488c9087a2f523adf045 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:31.855279 23915 raft_consensus.cc:493] T 8cc959fb7f704fba9009c0595467d074 P 1ca2725d290e47d68b1a4b609c9816ac [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:22:31.855423 24145 raft_consensus.cc:515] T 68a6838d82ef4cac83507f906139d90f P 90f3da57796b488c9087a2f523adf045 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:31.855842 23915 raft_consensus.cc:515] T 8cc959fb7f704fba9009c0595467d074 P 1ca2725d290e47d68b1a4b609c9816ac [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:31.857560 16819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "829b4ad8c5a54eababeea1181b960d4b" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 22 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:31.858294 23915 leader_election.cc:290] T 8cc959fb7f704fba9009c0595467d074 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:31.857659 24145 leader_election.cc:290] T 68a6838d82ef4cac83507f906139d90f P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
W20260709 13:22:31.859124 16264 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:22:31.860998 16226 raft_consensus.cc:2468] T b6f07842d3aa47858df5665874c86f3a P 1ca2725d290e47d68b1a4b609c9816ac [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 22.
W20260709 13:22:31.862195 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 127180us
I20260709 13:22:31.862525 16817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746aabf856ab401c9ef5e859371e7b67" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:31.847091 23924 consensus_queue.cc:260] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 22.17, Last appended by leader: 17, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:31.866703 23924 raft_consensus.cc:3060] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:22:31.858852 23865 raft_consensus.cc:493] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:31.868113 23865 raft_consensus.cc:515] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:31.869628 16296 raft_consensus.cc:2410] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac for term 24 because replica has last-logged OpId of term: 23 index: 18, which is greater than that of the candidate, which has last-logged OpId of term: 22 index: 17.
I20260709 13:22:31.877493 16820 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49892 (request call id 1202) took 1086 ms. Trace:
I20260709 13:22:31.881543 16225 raft_consensus.cc:1240] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac [term 23 FOLLOWER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 22. Current term is 23. Ops: []
I20260709 13:22:31.884390 16222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66f75124215f4ca6bef831e174a792d8" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:22:31.885172 16222 raft_consensus.cc:3060] T 66f75124215f4ca6bef831e174a792d8 P 1ca2725d290e47d68b1a4b609c9816ac [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:22:31.884390 16819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 26 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:31.878218 16820 rpcz_store.cc:276] 0709 13:22:30.790779 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:22:31.034400 (+243621us) service_pool.cc:224] Handling call
0709 13:22:31.034871 (+   471us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:22:31.369784 (+334913us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:22:31.369820 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:22:31.451783 (+ 81963us) log.cc:844] Serialized 26 byte log entry
0709 13:22:31.742957 (+291174us) raft_consensus.cc:1645] Marking committed up to 18
0709 13:22:31.743004 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:22:31.743096 (+    92us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:22:31.877152 (+134056us) raft_consensus.cc:1685] finished
0709 13:22:31.877184 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:22:31.877385 (+   201us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":165492,"raft.run_cpu_time_us":133,"raft.run_wall_time_us":11774,"thread_start_us":559496,"threads_started":2,"wal-append.queue_time_us":324778}
I20260709 13:22:31.886030 16819 raft_consensus.cc:2393] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 26: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
I20260709 13:22:31.864006 24073 leader_election.cc:290] T e4dc6b0541ca431ebc757beac00521f1 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:22:31.907042 16222 raft_consensus.cc:2468] T 66f75124215f4ca6bef831e174a792d8 P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 21.
I20260709 13:22:31.913829 16217 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57604 (request call id 3604) took 1819 ms. Trace:
I20260709 13:22:31.914340 16217 rpcz_store.cc:276] 0709 13:22:30.094261 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:22:30.161153 (+ 66892us) service_pool.cc:224] Handling call
0709 13:22:30.161572 (+   419us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:22:30.873836 (+712264us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:22:31.913558 (+1039722us) spinlock_profiling.cc:243] Waited 711 ms on lock 0x7b54000154d4. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:22:31.913746 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1635575168}
I20260709 13:22:31.927546 24142 raft_consensus.cc:3055] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:22:31.928258 24142 raft_consensus.cc:740] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:22:31.937014 16790 leader_election.cc:304] T 66f75124215f4ca6bef831e174a792d8 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:22:31.939020 16819 raft_consensus.cc:1240] T af57996a8be84d398f0f023e8d08ef77 P cb82cc7bd6db46d980c75237a1402860 [term 22 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 21. Current term is 22. Ops: []
W20260709 13:22:31.943608 16265 consensus_peers.cc:597] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 90f3da57796b488c9087a2f523adf045 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:22:31.944149 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 1.34941e+06us
W20260709 13:22:31.944792 16265 consensus_peers.cc:597] T 8cc959fb7f704fba9009c0595467d074 P 90f3da57796b488c9087a2f523adf045 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:22:31.946892 16265 leader_election.cc:336] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:22:31.947453 16265 leader_election.cc:304] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:22:31.929570 24142 consensus_queue.cc:260] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 22.17, Last appended by leader: 17, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:22:31.950342 16265 leader_election.cc:336] T 9e65676b21b748728ad39668cfe722ec P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:22:31.951627 16265 leader_election.cc:304] T 9e65676b21b748728ad39668cfe722ec P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
W20260709 13:22:31.953342 16265 consensus_peers.cc:597] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:22:31.955322 16265 leader_election.cc:336] T 84f2da98864b4425bf7c0055951fdd4b P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:22:31.934741 23938 raft_consensus.cc:515] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:31.950340 24073 raft_consensus.cc:2749] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:22:31.953670 24222 raft_consensus.cc:2804] T 66f75124215f4ca6bef831e174a792d8 P cb82cc7bd6db46d980c75237a1402860 [term 21 FOLLOWER]: Leader election won for term 21
W20260709 13:22:31.965193 16265 leader_election.cc:336] T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:22:31.954969 24161 raft_consensus.cc:2749] T 9e65676b21b748728ad39668cfe722ec P 90f3da57796b488c9087a2f523adf045 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:22:31.976851 16823 raft_consensus.cc:1275] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [term 27 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 26 index: 16. Preceding OpId from leader: term: 27 index: 17. (index mismatch)
W20260709 13:22:31.978111 16265 consensus_peers.cc:597] T 26354361e9b044cea1c2b86c4882fe92 P 90f3da57796b488c9087a2f523adf045 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:22:31.982796 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a378d7f796f94615b3b347119a9880fd" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:22:31.983016 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a96373bb0e14dce8a266b2fc6c58507" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:31.984211 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cc959fb7f704fba9009c0595467d074" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:31.985081 16190 leader_election.cc:304] T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:22:31.986053 16291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9944d5e4b8c4109af9955c4f08c9095" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:31.986965 16291 raft_consensus.cc:2410] T c9944d5e4b8c4109af9955c4f08c9095 P 90f3da57796b488c9087a2f523adf045 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac for term 25 because replica has last-logged OpId of term: 24 index: 18, which is greater than that of the candidate, which has last-logged OpId of term: 23 index: 17.
I20260709 13:22:31.989032 24091 consensus_queue.cc:1048] T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 13:22:31.989090 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c2095a5f4ad49408dbc7f69c606120d" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 20 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:31.950600 24240 consensus_queue.cc:1059] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: INVALID_TERM, Last received: 22.17, Next index: 18, Last known committed idx: 15, Time since last communication: 0.025s
I20260709 13:22:31.991865 16116 leader_election.cc:304] T b6f07842d3aa47858df5665874c86f3a P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
I20260709 13:22:31.992821 24171 raft_consensus.cc:2804] T b6f07842d3aa47858df5665874c86f3a P 647a343a542d4a139fe106c19095129b [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:22:31.993419 24171 raft_consensus.cc:697] T b6f07842d3aa47858df5665874c86f3a P 647a343a542d4a139fe106c19095129b [term 22 LEADER]: Becoming Leader. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:22:31.995477 24222 raft_consensus.cc:697] T 66f75124215f4ca6bef831e174a792d8 P cb82cc7bd6db46d980c75237a1402860 [term 21 LEADER]: Becoming Leader. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:22:31.996692 24222 consensus_queue.cc:237] T 66f75124215f4ca6bef831e174a792d8 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:31.995362 24196 consensus_queue.cc:1059] T 9c61a39fd4f84424b129c1a6f9ba404d P 647a343a542d4a139fe106c19095129b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:22:32.034660 16221 raft_consensus.cc:1240] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac [term 22 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 21. Current term is 22. Ops: [21.17-21.17]
W20260709 13:22:32.036129 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 51125.9us
I20260709 13:22:32.014477 16266 leader_election.cc:304] T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:22:32.016057 16299 raft_consensus.cc:1275] T e5ad4eb281564936a59f41e387a47156 P 90f3da57796b488c9087a2f523adf045 [term 22 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 22 index: 14. (index mismatch)
W20260709 13:22:32.018926 16036 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet ecc1e363230043fe9fef390f0b979f20 with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER failed with leader cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica cb82cc7bd6db46d980c75237a1402860 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 26 committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
W20260709 13:22:32.039482 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 59020.7us
I20260709 13:22:31.994442 24171 consensus_queue.cc:237] T b6f07842d3aa47858df5665874c86f3a P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.13, Last appended by leader: 13, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:32.038583 24145 raft_consensus.cc:493] T ada5a004ba7c46819051e3ccec71ffdb P 90f3da57796b488c9087a2f523adf045 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:22:32.042052 16117 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 53288.9us
W20260709 13:22:32.053161 16790 leader_election.cc:336] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:22:32.039098 24161 raft_consensus.cc:493] T e5ad4eb281564936a59f41e387a47156 P 90f3da57796b488c9087a2f523adf045 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:32.040539 24217 raft_consensus.cc:493] T 68a6838d82ef4cac83507f906139d90f P 1ca2725d290e47d68b1a4b609c9816ac [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:32.055081 16790 leader_election.cc:304] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:22:32.042887 23915 raft_consensus.cc:2804] T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac [term 24 FOLLOWER]: Leader election won for term 24
I20260709 13:22:32.041724 24166 raft_consensus.cc:2804] T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 13:22:32.063977 16190 leader_election.cc:304] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860
I20260709 13:22:32.065487 16117 leader_election.cc:304] T 829b4ad8c5a54eababeea1181b960d4b P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860
W20260709 13:22:32.067981 16789 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:22:32.046586 24225 raft_consensus.cc:3055] T 9c61a39fd4f84424b129c1a6f9ba404d P 647a343a542d4a139fe106c19095129b [term 23 LEADER]: Stepping down as leader of term 23
W20260709 13:22:32.078488 16790 leader_election.cc:336] T 66f75124215f4ca6bef831e174a792d8 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 21 election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:22:32.055999 24187 raft_consensus.cc:493] T 26354361e9b044cea1c2b86c4882fe92 P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:32.035737 24227 raft_consensus.cc:515] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:32.058604 23798 raft_consensus.cc:493] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:32.092792 23798 raft_consensus.cc:515] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:32.096127 24227 leader_election.cc:290] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 26 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:32.097301 23798 leader_election.cc:290] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:32.048910 24145 raft_consensus.cc:515] T ada5a004ba7c46819051e3ccec71ffdb P 90f3da57796b488c9087a2f523adf045 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:32.109645 24145 leader_election.cc:290] T ada5a004ba7c46819051e3ccec71ffdb P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:32.074436 24225 raft_consensus.cc:740] T 9c61a39fd4f84424b129c1a6f9ba404d P 647a343a542d4a139fe106c19095129b [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:22:32.111246 24225 consensus_queue.cc:260] T 9c61a39fd4f84424b129c1a6f9ba404d P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 23.13, Last appended by leader: 13, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:32.112998 24225 raft_consensus.cc:3060] T 9c61a39fd4f84424b129c1a6f9ba404d P 647a343a542d4a139fe106c19095129b [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:22:32.121799 23938 leader_election.cc:290] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 22 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:32.074184 23605 raft_consensus.cc:2749] T 829b4ad8c5a54eababeea1181b960d4b P 647a343a542d4a139fe106c19095129b [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:22:32.140647 16266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 97855.3us
I20260709 13:22:32.076174 24230 raft_consensus.cc:2749] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:22:32.083025 23958 raft_consensus.cc:493] T ecc1e363230043fe9fef390f0b979f20 P 1ca2725d290e47d68b1a4b609c9816ac [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:32.090849 24120 raft_consensus.cc:2749] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:22:32.060792 24161 raft_consensus.cc:515] T e5ad4eb281564936a59f41e387a47156 P 90f3da57796b488c9087a2f523adf045 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:22:32.145831 24219 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.819s	user 0.024s	sys 0.050s
I20260709 13:22:32.163448 16297 raft_consensus.cc:1275] T 18ad7e3af5504571a69d10c00750a01c P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 13:22:32.157034 16824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5718c623ebee4d2bb770f0f9ddb2f164" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 24 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
W20260709 13:22:32.165162 16114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:22:32.065516 24217 raft_consensus.cc:515] T 68a6838d82ef4cac83507f906139d90f P 1ca2725d290e47d68b1a4b609c9816ac [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:32.166872 16819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada5a004ba7c46819051e3ccec71ffdb" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 25 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:32.173557 24129 consensus_queue.cc:1048] T e5ad4eb281564936a59f41e387a47156 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.001s
W20260709 13:22:32.166338 24219 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.849s	user 0.024s	sys 0.051s
I20260709 13:22:32.181643 16297 raft_consensus.cc:1275] T c4f65cdf00b748bb963ee6e81ef43b79 P 90f3da57796b488c9087a2f523adf045 [term 23 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 21 index: 14. Preceding OpId from leader: term: 23 index: 15. (index mismatch)
W20260709 13:22:32.145550 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 163260us
I20260709 13:22:32.174222 23815 leader_election.cc:290] T c9944d5e4b8c4109af9955c4f08c9095 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:32.166853 24216 raft_consensus.cc:493] T d16dfad0c3b540e5b4b86492ddf87208 P 1ca2725d290e47d68b1a4b609c9816ac [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:32.141494 24187 raft_consensus.cc:515] T 26354361e9b044cea1c2b86c4882fe92 P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:22:32.192219 16265 leader_election.cc:336] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:22:32.156525 24145 consensus_queue.cc:1059] T a378d7f796f94615b3b347119a9880fd P 90f3da57796b488c9087a2f523adf045 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.019s
I20260709 13:22:32.199679 16819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9944d5e4b8c4109af9955c4f08c9095" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 25 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:32.202044 16817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b7d1ab519044eda2d5e04a07242f25" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:22:32.176306 24120 raft_consensus.cc:493] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:32.185431 24166 raft_consensus.cc:697] T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045 [term 25 LEADER]: Becoming Leader. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:22:32.211139 24216 raft_consensus.cc:515] T d16dfad0c3b540e5b4b86492ddf87208 P 1ca2725d290e47d68b1a4b609c9816ac [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:32.213459 24142 raft_consensus.cc:3060] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:22:32.217178 16220 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57604 (request call id 3615) took 1454 ms. Trace:
I20260709 13:22:32.162246 23958 raft_consensus.cc:515] T ecc1e363230043fe9fef390f0b979f20 P 1ca2725d290e47d68b1a4b609c9816ac [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:32.174973 23938 consensus_queue.cc:1048] T 18ad7e3af5504571a69d10c00750a01c P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.027s
I20260709 13:22:32.237375 23815 raft_consensus.cc:3055] T a378d7f796f94615b3b347119a9880fd P 90f3da57796b488c9087a2f523adf045 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:22:32.221122 16220 rpcz_store.cc:276] 0709 13:22:30.762586 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:22:30.891134 (+128548us) service_pool.cc:224] Handling call
0709 13:22:30.891482 (+   348us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:22:31.075222 (+183740us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:22:31.075253 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:22:31.109970 (+ 34717us) log.cc:844] Serialized 26 byte log entry
0709 13:22:31.381786 (+271816us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:22:31.381852 (+    66us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:22:31.381930 (+    78us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:22:32.215511 (+833581us) spinlock_profiling.cc:243] Waited 982 us on lock 0x7b5400421920. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0ad4dbe 00007f69b0adf930 00007f69b0adf8d8 00007f69b0adf858 00007f69b0adf813 00007f69b0adea9c 00007f69b69ca626 00007f69b69c4dd8 00007f69b0b1b7cb 00007f69b0b1e2c0
0709 13:22:32.216822 (+  1311us) raft_consensus.cc:1685] finished
0709 13:22:32.216847 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:22:32.217115 (+   268us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":20407,"raft.queue_time_us":4886,"raft.run_cpu_time_us":216,"raft.run_wall_time_us":29497,"spinlock_wait_cycles":2259328,"thread_start_us":369417,"threads_started":2,"wal-append.queue_time_us":240117}
I20260709 13:22:32.237946 23815 raft_consensus.cc:740] T a378d7f796f94615b3b347119a9880fd P 90f3da57796b488c9087a2f523adf045 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:22:32.201617 24180 consensus_queue.cc:1048] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.037s
I20260709 13:22:32.219126 23915 raft_consensus.cc:697] T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac [term 24 LEADER]: Becoming Leader. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:22:32.220332 24120 raft_consensus.cc:515] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:32.254734 23815 consensus_queue.cc:260] T a378d7f796f94615b3b347119a9880fd P 90f3da57796b488c9087a2f523adf045 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 22.15, Last appended by leader: 15, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:32.269423 23815 raft_consensus.cc:3060] T a378d7f796f94615b3b347119a9880fd P 90f3da57796b488c9087a2f523adf045 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:22:32.270202 24216 leader_election.cc:290] T d16dfad0c3b540e5b4b86492ddf87208 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:32.272977 24223 raft_consensus.cc:515] T 1df93779209748b9860f6d893dd3de53 P 647a343a542d4a139fe106c19095129b [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:32.276777 23958 leader_election.cc:290] T ecc1e363230043fe9fef390f0b979f20 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:32.254226 24129 raft_consensus.cc:493] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac [term 24 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:22:32.278187 24129 raft_consensus.cc:515] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:32.281124 24129 leader_election.cc:290] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:32.281541 16824 raft_consensus.cc:2410] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b for term 24 because replica has last-logged OpId of term: 22 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 16.
W20260709 13:22:32.282822 24266 log.cc:927] Time spent T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 0.083s	user 0.001s	sys 0.000s
I20260709 13:22:32.286803 24120 leader_election.cc:290] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:32.288021 23915 consensus_queue.cc:237] T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 22.12, Last appended by leader: 12, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:32.292703 24187 leader_election.cc:290] T 26354361e9b044cea1c2b86c4882fe92 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:32.295789 24217 leader_election.cc:290] T 68a6838d82ef4cac83507f906139d90f P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:32.253763 24166 consensus_queue.cc:237] T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 23.16, Last appended by leader: 16, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:32.299422 16220 raft_consensus.cc:1275] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac [term 24 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 22 index: 16. Preceding OpId from leader: term: 24 index: 17. (index mismatch)
W20260709 13:22:32.299697 16296 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48684 (request call id 3412) took 2286 ms (client timeout 1527 ms). Trace:
W20260709 13:22:32.300105 16296 rpcz_store.cc:269] 0709 13:22:30.012835 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:22:30.012984 (+   149us) service_pool.cc:224] Handling call
0709 13:22:32.299452 (+2286468us) spinlock_profiling.cc:243] Waited 1.83 s on lock 0x7b540030c0d4. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:22:32.299608 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4216804736}
I20260709 13:22:32.300500 24168 consensus_queue.cc:1059] T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: INVALID_TERM, Last received: 20.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 13:22:32.301489 24168 raft_consensus.cc:3055] T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:22:32.301846 24168 raft_consensus.cc:740] T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:22:32.303763 23865 leader_election.cc:290] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:32.304694 16292 raft_consensus.cc:1275] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045 [term 24 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 23 index: 18. Preceding OpId from leader: term: 24 index: 18. (term mismatch)
I20260709 13:22:32.305069 16292 pending_rounds.cc:85] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045: Aborting all ops after (but not including) 17
I20260709 13:22:32.305300 16292 pending_rounds.cc:107] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045: Aborting uncommitted NO_OP operation due to leader change: 23.18
I20260709 13:22:32.305570 16292 raft_consensus.cc:2889] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045 [term 24 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:22:32.306459 16819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26354361e9b044cea1c2b86c4882fe92" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
W20260709 13:22:32.307679 16788 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:22:32.309641 24215 raft_consensus.cc:493] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 [term 25 FOLLOWER]: Starting leader election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:22:32.310204 24215 raft_consensus.cc:3060] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:22:32.307044 16292 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48684 (request call id 3426) took 1569 ms. Trace:
I20260709 13:22:32.311138 16292 rpcz_store.cc:276] 0709 13:22:30.737430 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:22:30.737606 (+   176us) service_pool.cc:224] Handling call
0709 13:22:32.304333 (+1566727us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:22:32.306025 (+  1692us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:22:32.306833 (+   808us) spinlock_profiling.cc:243] Waited 1.56 s on lock 0x7b540030c0d0. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5e8ad 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:22:32.306963 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3581562752}
I20260709 13:22:32.252450 24230 raft_consensus.cc:493] T 806d7690c2b644959152074bb2cab0a7 P cb82cc7bd6db46d980c75237a1402860 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:22:32.257186 24274 raft_consensus.cc:493] T 5a959dbecc2648eb85d1cc550f5a3245 P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:32.302685 24168 consensus_queue.cc:260] T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 21.17, Last appended by leader: 17, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:32.342868 24142 consensus_queue.cc:1048] T ada5a004ba7c46819051e3ccec71ffdb P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.017s
I20260709 13:22:32.359251 16292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e4bfb9090c8460eb14fa7e0d818e7f1" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:22:32.319674 24278 raft_consensus.cc:493] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 90f3da57796b488c9087a2f523adf045 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:22:32.361792 16819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e4bfb9090c8460eb14fa7e0d818e7f1" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 25 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:32.362600 16825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 26 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:32.363983 16825 raft_consensus.cc:2393] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 26: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
I20260709 13:22:32.367202 16220 raft_consensus.cc:1240] T 68a6838d82ef4cac83507f906139d90f P 1ca2725d290e47d68b1a4b609c9816ac [term 22 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 21. Current term is 22. Ops: []
W20260709 13:22:32.368065 24262 log.cc:927] Time spent T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860: Append to log took a long time: real 0.425s	user 0.002s	sys 0.002s
I20260709 13:22:32.370847 16817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5d59407eb545b895d8d922e0dd69fd" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 25 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:32.370177 16220 raft_consensus.cc:1240] T 8a878999c0024d2abe517484180290fc P 1ca2725d290e47d68b1a4b609c9816ac [term 20 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 19. Current term is 20. Ops: [19.11-19.11]
I20260709 13:22:32.379660 16218 raft_consensus.cc:1240] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 20. Current term is 21. Ops: [20.13-20.13]
I20260709 13:22:32.381528 16218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f2da98864b4425bf7c0055951fdd4b" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:22:32.383352 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6f07842d3aa47858df5665874c86f3a" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:22:32.384238 16226 raft_consensus.cc:2393] T b6f07842d3aa47858df5665874c86f3a P 1ca2725d290e47d68b1a4b609c9816ac [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 22: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
I20260709 13:22:32.389297 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:22:32.389876 16226 raft_consensus.cc:3060] T 19baa7bc367844c288bca8f81439c413 P 1ca2725d290e47d68b1a4b609c9816ac [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:22:32.359789 24217 raft_consensus.cc:493] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:22:32.404019 16817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada5a004ba7c46819051e3ccec71ffdb" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 25 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:32.412082 16292 raft_consensus.cc:2468] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 24.
I20260709 13:22:32.380411 24223 leader_election.cc:290] T 1df93779209748b9860f6d893dd3de53 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 21 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:32.369148 24278 raft_consensus.cc:515] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 90f3da57796b488c9087a2f523adf045 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:32.345247 24230 raft_consensus.cc:515] T 806d7690c2b644959152074bb2cab0a7 P cb82cc7bd6db46d980c75237a1402860 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:22:32.426400 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 178005us
I20260709 13:22:32.402407 24225 raft_consensus.cc:493] T 57a744f5aaf944678a63469cbf514015 P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:32.428073 16292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d16dfad0c3b540e5b4b86492ddf87208" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:32.428450 16294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 26 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:32.428761 16292 raft_consensus.cc:2468] T d16dfad0c3b540e5b4b86492ddf87208 P 90f3da57796b488c9087a2f523adf045 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 22.
I20260709 13:22:32.429497 16294 raft_consensus.cc:2393] T ecc1e363230043fe9fef390f0b979f20 P 90f3da57796b488c9087a2f523adf045 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 26: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
I20260709 13:22:32.430416 16226 raft_consensus.cc:2468] T 19baa7bc367844c288bca8f81439c413 P 1ca2725d290e47d68b1a4b609c9816ac [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 25.
I20260709 13:22:32.433008 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada5a004ba7c46819051e3ccec71ffdb" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 25 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:32.433672 16297 raft_consensus.cc:2468] T ada5a004ba7c46819051e3ccec71ffdb P 90f3da57796b488c9087a2f523adf045 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 24.
I20260709 13:22:32.406921 23719 raft_consensus.cc:493] T 02838c86531d4f98bf58d1c22b8baf77 P 647a343a542d4a139fe106c19095129b [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:32.436885 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68a6838d82ef4cac83507f906139d90f" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
W20260709 13:22:32.437249 16117 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:22:32.437678 16298 raft_consensus.cc:2468] T 68a6838d82ef4cac83507f906139d90f P 90f3da57796b488c9087a2f523adf045 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 22.
I20260709 13:22:32.356967 24274 raft_consensus.cc:515] T 5a959dbecc2648eb85d1cc550f5a3245 P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:32.440718 16190 leader_election.cc:304] T d16dfad0c3b540e5b4b86492ddf87208 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:22:32.443938 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806d7690c2b644959152074bb2cab0a7" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:32.444962 16190 leader_election.cc:304] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:22:32.452909 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 26 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:32.465274 16225 raft_consensus.cc:2468] T ecc1e363230043fe9fef390f0b979f20 P 1ca2725d290e47d68b1a4b609c9816ac [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 25.
I20260709 13:22:32.466523 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1df93779209748b9860f6d893dd3de53" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:22:32.467247 16225 raft_consensus.cc:3060] T 1df93779209748b9860f6d893dd3de53 P 1ca2725d290e47d68b1a4b609c9816ac [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:22:32.470089 16264 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:22:32.470633 16264 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:22:32.453635 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c2095a5f4ad49408dbc7f69c606120d" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 20 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:32.445235 24120 raft_consensus.cc:2804] T d16dfad0c3b540e5b4b86492ddf87208 P 1ca2725d290e47d68b1a4b609c9816ac [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:22:32.471850 16224 raft_consensus.cc:2410] T 7c2095a5f4ad49408dbc7f69c606120d P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b for term 20 because replica has last-logged OpId of term: 19 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 8.
I20260709 13:22:32.472007 24120 raft_consensus.cc:493] T d16dfad0c3b540e5b4b86492ddf87208 P 1ca2725d290e47d68b1a4b609c9816ac [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:32.472671 24120 raft_consensus.cc:3060] T d16dfad0c3b540e5b4b86492ddf87208 P 1ca2725d290e47d68b1a4b609c9816ac [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:22:32.473071 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f5f89186204242bf377f7c42b2e2da" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:22:32.474246 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5718c623ebee4d2bb770f0f9ddb2f164" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 24 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:32.475199 16224 raft_consensus.cc:2410] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b for term 24 because replica has last-logged OpId of term: 22 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 16.
I20260709 13:22:32.440662 23719 raft_consensus.cc:515] T 02838c86531d4f98bf58d1c22b8baf77 P 647a343a542d4a139fe106c19095129b [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:32.476435 16224 raft_consensus.cc:1275] T 8a878999c0024d2abe517484180290fc P 1ca2725d290e47d68b1a4b609c9816ac [term 20 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 13:22:32.477231 23719 leader_election.cc:290] T 02838c86531d4f98bf58d1c22b8baf77 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:32.415511 24217 raft_consensus.cc:515] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:32.445616 24273 consensus_queue.cc:1048] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 16, Time since last communication: 0.037s
I20260709 13:22:32.479394 16226 raft_consensus.cc:2468] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 21.
I20260709 13:22:32.480206 24162 raft_consensus.cc:3060] T 8d4ab870a05347868956aa8b841b2981 P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:22:32.471112 16264 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:22:32.486467 16225 raft_consensus.cc:2468] T 1df93779209748b9860f6d893dd3de53 P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 21.
I20260709 13:22:32.436847 24225 raft_consensus.cc:515] T 57a744f5aaf944678a63469cbf514015 P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:22:32.512892 16114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:22:32.514267 24291 log.cc:927] Time spent T c2f5f89186204242bf377f7c42b2e2da P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 0.059s	user 0.001s	sys 0.001s
I20260709 13:22:32.523689 16190 leader_election.cc:304] T 68a6838d82ef4cac83507f906139d90f P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:22:32.526278 16789 leader_election.cc:304] T 8638ee363273426db1393dd81e5c7b72 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac
I20260709 13:22:32.537546 16223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4dc6b0541ca431ebc757beac00521f1" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:32.538362 16223 raft_consensus.cc:2468] T e4dc6b0541ca431ebc757beac00521f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 23.
I20260709 13:22:32.540139 16220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada5a004ba7c46819051e3ccec71ffdb" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 25 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:32.546778 16192 leader_election.cc:304] T ecc1e363230043fe9fef390f0b979f20 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860
I20260709 13:22:32.551335 16116 leader_election.cc:304] T 1df93779209748b9860f6d893dd3de53 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
I20260709 13:22:32.546435 24113 raft_consensus.cc:2804] T 8638ee363273426db1393dd81e5c7b72 P cb82cc7bd6db46d980c75237a1402860 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:22:32.590710 16294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806d7690c2b644959152074bb2cab0a7" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:32.591506 16294 raft_consensus.cc:2468] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 21.
W20260709 13:22:32.592453 16191 leader_election.cc:336] T e5ad4eb281564936a59f41e387a47156 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Timed out: RequestConsensusVote RPC to 127.14.177.129:36741 timed out after 1.509s (SENT)
W20260709 13:22:32.594103 15046 thread.cc:527] Waited for 4000ms trying to join with heartbeat (tid 16176)
I20260709 13:22:32.588171 16822 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49906 (request call id 1549) took 1060 ms. Trace:
W20260709 13:22:32.595549 16114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:22:32.603986 16789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 71825.8us
I20260709 13:22:32.595729 16822 rpcz_store.cc:276] 0709 13:22:31.528054 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:22:31.715299 (+187245us) service_pool.cc:224] Handling call
0709 13:22:31.715627 (+   328us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:22:31.720155 (+  4528us) raft_consensus.cc:1534] Early marking committed up to index 16
0709 13:22:31.741939 (+ 21784us) log.cc:844] Serialized 14 byte log entry
0709 13:22:31.909520 (+167581us) log.cc:844] Serialized 14 byte log entry
0709 13:22:31.910518 (+   998us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:22:31.953000 (+ 42482us) log.cc:844] Serialized 26 byte log entry
0709 13:22:31.953215 (+   215us) raft_consensus.cc:1645] Marking committed up to 16
0709 13:22:31.953263 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:22:31.953331 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:22:32.341408 (+388077us) spinlock_profiling.cc:243] Waited 1.16 ms on lock 0x7b4400022b4c. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69ad8fadb1 00007f69ad998c85 00007f69ad9996a2 00007f69b070b1db 00007f69b0706aaf 00007f69b06fa86e 00007f69b06f9bb6 00007f69b06f97d8 00007f69b0703221 00007f69b07031d9 00007f69b0703169
0709 13:22:32.587835 (+246427us) raft_consensus.cc:1685] finished
0709 13:22:32.587872 (+    37us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:22:32.588090 (+   218us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":453,"prepare.queue_time_us":22535,"prepare.run_cpu_time_us":118,"prepare.run_wall_time_us":2261,"raft.queue_time_us":2150,"raft.run_cpu_time_us":253,"raft.run_wall_time_us":14209,"spinlock_wait_cycles":18081280,"thread_start_us":115613,"threads_started":1,"wal-append.queue_time_us":162099}
I20260709 13:22:32.558627 16293 raft_consensus.cc:1275] T c0eab25ce4c64e018a041beeda2867db P 90f3da57796b488c9087a2f523adf045 [term 24 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 22 index: 12. Preceding OpId from leader: term: 24 index: 13. (index mismatch)
I20260709 13:22:32.559846 24216 raft_consensus.cc:3060] T ecc1e363230043fe9fef390f0b979f20 P 1ca2725d290e47d68b1a4b609c9816ac [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:22:32.581003 24281 raft_consensus.cc:2804] T 68a6838d82ef4cac83507f906139d90f P 1ca2725d290e47d68b1a4b609c9816ac [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:22:32.623486 24281 raft_consensus.cc:493] T 68a6838d82ef4cac83507f906139d90f P 1ca2725d290e47d68b1a4b609c9816ac [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:22:32.623919 24287 log.cc:927] Time spent T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.107s	user 0.001s	sys 0.000s
W20260709 13:22:32.625861 24285 log.cc:927] Time spent T b6f07842d3aa47858df5665874c86f3a P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 0.099s	user 0.000s	sys 0.000s
I20260709 13:22:32.626009 24274 leader_election.cc:290] T 5a959dbecc2648eb85d1cc550f5a3245 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:32.558748 24223 raft_consensus.cc:2804] T 1df93779209748b9860f6d893dd3de53 P 647a343a542d4a139fe106c19095129b [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:22:32.579809 23958 raft_consensus.cc:493] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:32.581568 23975 raft_consensus.cc:493] T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:32.564198 24219 raft_consensus.cc:493] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:22:32.629829 16191 leader_election.cc:336] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 23 election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Timed out: RequestConsensusVote RPC to 127.14.177.129:36741 timed out after 1.539s (SENT)
W20260709 13:22:32.702044 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 72237us
W20260709 13:22:32.703143 16191 leader_election.cc:336] T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 21 election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Timed out: RequestConsensusVote RPC to 127.14.177.129:36741 timed out after 1.574s (SENT)
I20260709 13:22:32.703804 16191 leader_election.cc:304] T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:22:32.633322 16221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ab0ac8347641e6955e9bce7d90c1a2" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:32.558455 24088 consensus_queue.cc:1048] T 8a878999c0024d2abe517484180290fc P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.044s
I20260709 13:22:32.597520 24289 raft_consensus.cc:493] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:32.723603 16221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d4ab870a05347868956aa8b841b2981" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:32.612933 24113 raft_consensus.cc:697] T 8638ee363273426db1393dd81e5c7b72 P cb82cc7bd6db46d980c75237a1402860 [term 23 LEADER]: Becoming Leader. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
W20260709 13:22:32.732874 16788 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:22:32.728384 16221 raft_consensus.cc:2393] T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 21: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
I20260709 13:22:32.710507 23865 raft_consensus.cc:493] T c0eab25ce4c64e018a041beeda2867db P 90f3da57796b488c9087a2f523adf045 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:32.635543 24223 raft_consensus.cc:697] T 1df93779209748b9860f6d893dd3de53 P 647a343a542d4a139fe106c19095129b [term 21 LEADER]: Becoming Leader. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
W20260709 13:22:32.735896 16789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 126244us
I20260709 13:22:32.623998 24281 raft_consensus.cc:3060] T 68a6838d82ef4cac83507f906139d90f P 1ca2725d290e47d68b1a4b609c9816ac [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:22:32.719763 24275 raft_consensus.cc:493] T 8158bbd17b994d2eb96f560bd0f1b88d P 90f3da57796b488c9087a2f523adf045 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:32.720316 24163 raft_consensus.cc:493] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:32.720086 24166 raft_consensus.cc:493] T 31969392ccdb43b491911a319e8bc7e1 P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:32.742138 24278 leader_election.cc:290] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:32.708961 23958 raft_consensus.cc:515] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:32.735535 24217 leader_election.cc:290] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:32.716358 24219 raft_consensus.cc:515] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:32.716893 23975 raft_consensus.cc:515] T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:32.775637 16221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 26 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:22:32.717108 24274 consensus_queue.cc:1048] T c0eab25ce4c64e018a041beeda2867db P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.033s
I20260709 13:22:32.746146 23815 consensus_queue.cc:1059] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: INVALID_TERM, Last received: 19.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.034s
I20260709 13:22:32.783026 24166 raft_consensus.cc:515] T 31969392ccdb43b491911a319e8bc7e1 P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:32.796953 24113 consensus_queue.cc:237] T 8638ee363273426db1393dd81e5c7b72 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 21.17, Last appended by leader: 17, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:32.734997 24271 consensus_queue.cc:1059] T 8a878999c0024d2abe517484180290fc P 90f3da57796b488c9087a2f523adf045 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: INVALID_TERM, Last received: 18.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.046s
I20260709 13:22:32.741255 23865 raft_consensus.cc:515] T c0eab25ce4c64e018a041beeda2867db P 90f3da57796b488c9087a2f523adf045 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:22:32.772347 16116 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:22:32.803903 23958 leader_election.cc:290] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:32.804185 24215 raft_consensus.cc:515] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 [term 26 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:32.769446 24120 raft_consensus.cc:515] T d16dfad0c3b540e5b4b86492ddf87208 P 1ca2725d290e47d68b1a4b609c9816ac [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:22:32.807566 24290 log.cc:927] Time spent T 08c74490c40243a88e62d1760b5cd814 P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 0.259s	user 0.002s	sys 0.000s
I20260709 13:22:32.801582 24163 raft_consensus.cc:515] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:32.814695 23975 leader_election.cc:290] T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:32.781215 24275 raft_consensus.cc:515] T 8158bbd17b994d2eb96f560bd0f1b88d P 90f3da57796b488c9087a2f523adf045 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:32.736511 24289 raft_consensus.cc:515] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:22:32.823675 24282 log.cc:927] Time spent T 66f75124215f4ca6bef831e174a792d8 P cb82cc7bd6db46d980c75237a1402860: Append to log took a long time: real 0.284s	user 0.001s	sys 0.000s
I20260709 13:22:32.824441 24223 consensus_queue.cc:237] T 1df93779209748b9860f6d893dd3de53 P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:32.825706 24225 leader_election.cc:290] T 57a744f5aaf944678a63469cbf514015 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
W20260709 13:22:32.851442 16192 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 240107us
W20260709 13:22:32.920094 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 189893us
I20260709 13:22:33.000599 16265 leader_election.cc:304] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:22:32.970662 24284 raft_consensus.cc:3055] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 90f3da57796b488c9087a2f523adf045 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:22:32.800992 24217 raft_consensus.cc:2749] T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
W20260709 13:22:32.943611 16114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:22:32.972090 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 205259us
I20260709 13:22:32.996419 16818 raft_consensus.cc:1240] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 [term 26 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 25. Current term is 26. Ops: []
W20260709 13:22:32.911048 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 300698us
I20260709 13:22:33.008121 16825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66ae176961b74813a58bc15c9c4c6f2a" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 29 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:32.969136 24193 raft_consensus.cc:493] T 2794af5b99984efebdee564035841cb0 P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:33.009078 16825 raft_consensus.cc:2410] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac for term 29 because replica has last-logged OpId of term: 27 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 26 index: 16.
I20260709 13:22:32.977502 23815 raft_consensus.cc:493] T 57a744f5aaf944678a63469cbf514015 P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:22:33.004062 16822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806d7690c2b644959152074bb2cab0a7" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:33.010936 16822 raft_consensus.cc:2468] T 806d7690c2b644959152074bb2cab0a7 P cb82cc7bd6db46d980c75237a1402860 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 21.
W20260709 13:22:33.009567 16191 leader_election.cc:336] T 829b4ad8c5a54eababeea1181b960d4b P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 21 election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Timed out: RequestConsensusVote RPC to 127.14.177.129:36741 timed out after 1.685s (SENT)
I20260709 13:22:32.922235 24278 raft_consensus.cc:493] T 3884f864a96d428785895e5463e5728f P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:22:32.993505 24230 leader_election.cc:290] T 806d7690c2b644959152074bb2cab0a7 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:33.039861 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74ca8140f35c4ae29e8a664f6bdfc829" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:22:33.040649 16225 raft_consensus.cc:3060] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:22:33.004899 24284 raft_consensus.cc:740] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 90f3da57796b488c9087a2f523adf045 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
W20260709 13:22:33.088805 16264 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:22:33.076239 24295 log.cc:927] Time spent T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 0.441s	user 0.001s	sys 0.000s
I20260709 13:22:33.076707 24230 raft_consensus.cc:493] T ee9ed67006144797b1de6ea8c2013a68 P cb82cc7bd6db46d980c75237a1402860 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:22:33.015616 24193 raft_consensus.cc:515] T 2794af5b99984efebdee564035841cb0 P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:33.018060 23815 raft_consensus.cc:515] T 57a744f5aaf944678a63469cbf514015 P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:33.020066 24278 raft_consensus.cc:515] T 3884f864a96d428785895e5463e5728f P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:22:33.148146 16263 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:22:33.106114 24300 raft_consensus.cc:493] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:33.100307 24142 raft_consensus.cc:493] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
W20260709 13:22:33.156064 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 155581us
W20260709 13:22:33.158974 16114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:22:33.166438 16292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a959dbecc2648eb85d1cc550f5a3245" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:33.167094 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66ae176961b74813a58bc15c9c4c6f2a" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 29 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:33.167436 16292 raft_consensus.cc:2468] T 5a959dbecc2648eb85d1cc550f5a3245 P 90f3da57796b488c9087a2f523adf045 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 21.
I20260709 13:22:33.167955 16293 raft_consensus.cc:2410] T 66ae176961b74813a58bc15c9c4c6f2a P 90f3da57796b488c9087a2f523adf045 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac for term 29 because replica has last-logged OpId of term: 27 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 26 index: 16.
I20260709 13:22:33.168187 16823 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49896 (request call id 1740) took 1717 ms. Trace:
W20260709 13:22:33.138327 24292 log.cc:927] Time spent T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 0.621s	user 0.000s	sys 0.001s
I20260709 13:22:33.168545 16823 rpcz_store.cc:276] 0709 13:22:31.450812 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:22:31.676503 (+225691us) service_pool.cc:224] Handling call
0709 13:22:31.677286 (+   783us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:22:31.981503 (+304217us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:22:33.167960 (+1186457us) spinlock_profiling.cc:243] Waited 299 ms on lock 0x7b54006423d4. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:22:33.168133 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":688048000}
I20260709 13:22:33.183780 16816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e9b2bf7011f4ae688a79d7bb5a447b4" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:33.184610 16816 raft_consensus.cc:2468] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 21.
I20260709 13:22:33.147478 24284 consensus_queue.cc:260] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 90f3da57796b488c9087a2f523adf045 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.13, Last appended by leader: 13, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:33.187865 16225 raft_consensus.cc:2468] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 25.
W20260709 13:22:33.197934 16191 leader_election.cc:336] T c0eab25ce4c64e018a041beeda2867db P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 24 election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Timed out: RequestConsensusVote RPC to 127.14.177.129:36741 timed out after 1.882s (SENT)
W20260709 13:22:33.199103 16191 leader_election.cc:336] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Timed out: RequestConsensusVote RPC to 127.14.177.129:36741 timed out after 1.891s (SENT)
I20260709 13:22:33.199606 16191 leader_election.cc:304] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
I20260709 13:22:33.162850 24240 raft_consensus.cc:493] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:33.125547 24230 raft_consensus.cc:515] T ee9ed67006144797b1de6ea8c2013a68 P cb82cc7bd6db46d980c75237a1402860 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:33.185883 23605 raft_consensus.cc:493] T fffab26427e24d398e2695aad40ca134 P 647a343a542d4a139fe106c19095129b [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:33.189898 24222 raft_consensus.cc:493] T 74ca8140f35c4ae29e8a664f6bdfc829 P cb82cc7bd6db46d980c75237a1402860 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:33.188508 24284 raft_consensus.cc:3060] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:22:33.170395 24142 raft_consensus.cc:515] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:33.170643 24300 raft_consensus.cc:515] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:22:33.238698 16191 leader_election.cc:336] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 27 election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Timed out: RequestConsensusVote RPC to 127.14.177.129:36741 timed out after 1.955s (SENT)
I20260709 13:22:33.212001 24166 leader_election.cc:290] T 31969392ccdb43b491911a319e8bc7e1 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:33.214895 23865 leader_election.cc:290] T c0eab25ce4c64e018a041beeda2867db P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:22:33.222724 24215 leader_election.cc:290] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 26 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
W20260709 13:22:33.243629 24305 log.cc:927] Time spent T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.119s	user 0.001s	sys 0.001s
I20260709 13:22:33.245404 24321 raft_consensus.cc:2804] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 13:22:33.245870 24120 leader_election.cc:290] T d16dfad0c3b540e5b4b86492ddf87208 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 23 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:33.234911 24219 leader_election.cc:290] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:33.209192 24240 raft_consensus.cc:515] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:33.234617 24163 leader_election.cc:290] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:22:33.220000 23605 raft_consensus.cc:515] T fffab26427e24d398e2695aad40ca134 P 647a343a542d4a139fe106c19095129b [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:33.221139 24222 raft_consensus.cc:515] T 74ca8140f35c4ae29e8a664f6bdfc829 P cb82cc7bd6db46d980c75237a1402860 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:22:33.270601 16266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 96203.3us
I20260709 13:22:33.243934 24289 leader_election.cc:290] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:33.273540 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806d7690c2b644959152074bb2cab0a7" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:33.274300 16298 raft_consensus.cc:2468] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 21.
I20260709 13:22:33.276597 16294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdbd78acb4cf46298e0de8ba0d130e19" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 26 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
W20260709 13:22:33.285864 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 47268.5us
W20260709 13:22:33.286716 16191 leader_election.cc:336] T 02838c86531d4f98bf58d1c22b8baf77 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 23 election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Timed out: RequestConsensusVote RPC to 127.14.177.129:36741 timed out after 1.964s (SENT)
I20260709 13:22:33.287238 16191 leader_election.cc:304] T 02838c86531d4f98bf58d1c22b8baf77 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:22:33.289994 16816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31969392ccdb43b491911a319e8bc7e1" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 21 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:33.291250 16816 raft_consensus.cc:2410] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 for term 21 because replica has last-logged OpId of term: 20 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 15.
I20260709 13:22:33.293749 16825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3884f864a96d428785895e5463e5728f" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:33.298403 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9944d5e4b8c4109af9955c4f08c9095" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 25 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
W20260709 13:22:33.300522 16191 leader_election.cc:336] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Timed out: RequestConsensusVote RPC to 127.14.177.129:36741 timed out after 2.200s (SENT)
I20260709 13:22:33.300973 16191 leader_election.cc:304] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:22:33.301280 16190 leader_election.cc:304] T 5a959dbecc2648eb85d1cc550f5a3245 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:22:33.305787 16823 raft_consensus.cc:1275] T 74ca8140f35c4ae29e8a664f6bdfc829 P cb82cc7bd6db46d980c75237a1402860 [term 25 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 23 index: 16. Preceding OpId from leader: term: 25 index: 17. (index mismatch)
I20260709 13:22:33.306087 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68a6838d82ef4cac83507f906139d90f" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:33.270823 24278 leader_election.cc:290] T 3884f864a96d428785895e5463e5728f P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:33.136924 24184 raft_consensus.cc:2955] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b [term 21 LEADER]: Committing config change with OpId 21.15: config changed from index 2 to 15, cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) changed from NON_VOTER to VOTER. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:22:33.305190 23938 raft_consensus.cc:2749] T 02838c86531d4f98bf58d1c22b8baf77 P 1ca2725d290e47d68b1a4b609c9816ac [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 13:22:33.303207 24313 raft_consensus.cc:493] T c4f65cdf00b748bb963ee6e81ef43b79 P 90f3da57796b488c9087a2f523adf045 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:33.305327 24193 leader_election.cc:290] T 2794af5b99984efebdee564035841cb0 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:33.313977 24321 raft_consensus.cc:697] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045 [term 25 LEADER]: Becoming Leader. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:22:33.313215 24289 raft_consensus.cc:2749] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:22:33.338472 16266 leader_election.cc:304] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:22:33.341563 16292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ad5dbfa99ee4b8d9e28be92b3f36db5" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
W20260709 13:22:33.342042 16189 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.566s	user 0.047s	sys 0.056s
W20260709 13:22:33.342406 16189 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.567s	user 0.047s	sys 0.056s
I20260709 13:22:33.343475 24320 raft_consensus.cc:2804] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 90f3da57796b488c9087a2f523adf045 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:22:33.344065 24320 raft_consensus.cc:493] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 90f3da57796b488c9087a2f523adf045 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:22:33.344544 24320 raft_consensus.cc:3060] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 90f3da57796b488c9087a2f523adf045 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:22:33.314530 24303 raft_consensus.cc:2804] T 5a959dbecc2648eb85d1cc550f5a3245 P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Leader pre-election won for term 22
W20260709 13:22:33.349750 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 196580us
I20260709 13:22:33.355576 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d16dfad0c3b540e5b4b86492ddf87208" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:22:33.356940 16817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57a744f5aaf944678a63469cbf514015" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:33.356374 16300 raft_consensus.cc:3060] T d16dfad0c3b540e5b4b86492ddf87208 P 90f3da57796b488c9087a2f523adf045 [term 22 FOLLOWER]: Advancing to term 23
W20260709 13:22:33.351619 16790 consensus_peers.cc:597] T 3884f864a96d428785895e5463e5728f P cb82cc7bd6db46d980c75237a1402860 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:22:33.377538 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74ca8140f35c4ae29e8a664f6bdfc829" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 25 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:33.394892 16300 raft_consensus.cc:2468] T d16dfad0c3b540e5b4b86492ddf87208 P 90f3da57796b488c9087a2f523adf045 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 23.
I20260709 13:22:33.397550 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18ad7e3af5504571a69d10c00750a01c" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:33.397593 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 26 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:33.399389 16300 raft_consensus.cc:2393] T ecc1e363230043fe9fef390f0b979f20 P 90f3da57796b488c9087a2f523adf045 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 26: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
I20260709 13:22:33.400704 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6f07842d3aa47858df5665874c86f3a" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:22:33.394987 16297 raft_consensus.cc:1240] T 84f2da98864b4425bf7c0055951fdd4b P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 19. Current term is 20. Ops: []
I20260709 13:22:33.349247 24215 raft_consensus.cc:493] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:33.405355 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f5f89186204242bf377f7c42b2e2da" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:22:33.402283 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5ad4eb281564936a59f41e387a47156" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:22:33.406378 16295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:33.352875 24303 raft_consensus.cc:493] T 5a959dbecc2648eb85d1cc550f5a3245 P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:33.407838 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c2095a5f4ad49408dbc7f69c606120d" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 20 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:33.410349 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68a6838d82ef4cac83507f906139d90f" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 23 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:33.411314 16299 raft_consensus.cc:2410] T 68a6838d82ef4cac83507f906139d90f P 90f3da57796b488c9087a2f523adf045 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b for term 23 because replica has last-logged OpId of term: 21 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 11.
W20260709 13:22:33.411945 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 255186us
I20260709 13:22:33.376935 23815 leader_election.cc:290] T 57a744f5aaf944678a63469cbf514015 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:33.413344 16265 leader_election.cc:304] T e4dc6b0541ca431ebc757beac00521f1 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:22:33.414188 16300 raft_consensus.cc:2393] T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 22: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
W20260709 13:22:33.414573 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 72527.7us
I20260709 13:22:33.417629 16265 leader_election.cc:304] T ada5a004ba7c46819051e3ccec71ffdb P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860
I20260709 13:22:33.418268 16190 leader_election.cc:304] T d16dfad0c3b540e5b4b86492ddf87208 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:22:33.417487 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26354361e9b044cea1c2b86c4882fe92" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:33.411959 16298 raft_consensus.cc:2955] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045 [term 21 FOLLOWER]: Committing config change with OpId 21.15: config changed from index 2 to 15, cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) changed from NON_VOTER to VOTER. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:22:33.424448 16295 raft_consensus.cc:1275] T 5a959dbecc2648eb85d1cc550f5a3245 P 90f3da57796b488c9087a2f523adf045 [term 21 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 20 index: 12. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
I20260709 13:22:33.324450 24313 raft_consensus.cc:515] T c4f65cdf00b748bb963ee6e81ef43b79 P 90f3da57796b488c9087a2f523adf045 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:33.410024 24303 raft_consensus.cc:3060] T 5a959dbecc2648eb85d1cc550f5a3245 P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:22:33.444904 16825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74ca8140f35c4ae29e8a664f6bdfc829" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 25 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:33.414269 24307 raft_consensus.cc:493] T 8c11074579b046d99e585c2818e18a2c P cb82cc7bd6db46d980c75237a1402860 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:22:33.451886 16265 leader_election.cc:304] T d2ab0ac8347641e6955e9bce7d90c1a2 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860
I20260709 13:22:33.413954 24215 raft_consensus.cc:515] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:33.450423 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57a744f5aaf944678a63469cbf514015" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 21 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:33.461364 16300 raft_consensus.cc:2410] T 57a744f5aaf944678a63469cbf514015 P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b for term 21 because replica has last-logged OpId of term: 20 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 14.
I20260709 13:22:33.457810 16295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1df93779209748b9860f6d893dd3de53" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:22:33.462632 16295 raft_consensus.cc:3060] T 1df93779209748b9860f6d893dd3de53 P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:22:33.466109 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4dc6b0541ca431ebc757beac00521f1" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 24 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
W20260709 13:22:33.466828 24162 consensus_meta.cc:287] T 8d4ab870a05347868956aa8b841b2981 P 647a343a542d4a139fe106c19095129b: Time spent flushing consensus metadata: real 0.986s	user 0.011s	sys 0.002s
I20260709 13:22:33.471536 24330 raft_consensus.cc:493] T 8a878999c0024d2abe517484180290fc P 1ca2725d290e47d68b1a4b609c9816ac [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:33.472411 24330 raft_consensus.cc:515] T 8a878999c0024d2abe517484180290fc P 1ca2725d290e47d68b1a4b609c9816ac [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:33.480620 16265 leader_election.cc:304] T c9944d5e4b8c4109af9955c4f08c9095 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860
I20260709 13:22:33.480813 16295 raft_consensus.cc:2468] T 1df93779209748b9860f6d893dd3de53 P 90f3da57796b488c9087a2f523adf045 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 21.
I20260709 13:22:33.421753 24230 leader_election.cc:290] T ee9ed67006144797b1de6ea8c2013a68 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:22:33.446197 23815 raft_consensus.cc:2749] T ada5a004ba7c46819051e3ccec71ffdb P 90f3da57796b488c9087a2f523adf045 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:22:33.491533 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02838c86531d4f98bf58d1c22b8baf77" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:33.504271 16299 raft_consensus.cc:2468] T 02838c86531d4f98bf58d1c22b8baf77 P 90f3da57796b488c9087a2f523adf045 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 23.
W20260709 13:22:33.454257 16115 connection.cc:256] RPC call timeout handler was delayed by -1.3917868490000274s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15960000000000002s for a response
W20260709 13:22:33.505291 16115 connection.cc:256] RPC call timeout handler was delayed by -1.2553113930000563s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15990000000000001s for a response
W20260709 13:22:33.505695 16115 connection.cc:256] RPC call timeout handler was delayed by -1.2422146400001566s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18810000000000002s for a response
W20260709 13:22:33.506028 16115 connection.cc:256] RPC call timeout handler was delayed by -1.1819146400000591s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1948s for a response
W20260709 13:22:33.506346 16115 connection.cc:256] RPC call timeout handler was delayed by -1.0982146400001511s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2041s for a response
W20260709 13:22:33.506644 16115 connection.cc:256] RPC call timeout handler was delayed by -1.052473824000117s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18530000000000002s for a response
W20260709 13:22:33.506942 16115 connection.cc:256] RPC call timeout handler was delayed by -1.0379146400000536s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21080000000000002s for a response
I20260709 13:22:33.440142 24326 raft_consensus.cc:2804] T e4dc6b0541ca431ebc757beac00521f1 P 90f3da57796b488c9087a2f523adf045 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:22:33.509222 23865 raft_consensus.cc:2749] T c9944d5e4b8c4109af9955c4f08c9095 P 90f3da57796b488c9087a2f523adf045 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:22:33.515945 24162 consensus_queue.cc:1048] T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.003s
I20260709 13:22:33.477272 24223 raft_consensus.cc:493] T 8d4ab870a05347868956aa8b841b2981 P 647a343a542d4a139fe106c19095129b [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:33.461915 24300 leader_election.cc:290] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:33.477453 24240 leader_election.cc:290] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:22:33.511286 24327 raft_consensus.cc:2804] T d16dfad0c3b540e5b4b86492ddf87208 P 1ca2725d290e47d68b1a4b609c9816ac [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:22:33.503599 24163 raft_consensus.cc:2749] T d2ab0ac8347641e6955e9bce7d90c1a2 P 90f3da57796b488c9087a2f523adf045 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:22:33.512075 24326 raft_consensus.cc:493] T e4dc6b0541ca431ebc757beac00521f1 P 90f3da57796b488c9087a2f523adf045 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:33.532522 16299 raft_consensus.cc:1275] T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045 [term 22 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 20 index: 13. Preceding OpId from leader: term: 22 index: 14. (index mismatch)
W20260709 13:22:33.532612 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 114416us
I20260709 13:22:33.477998 24289 raft_consensus.cc:493] T fffab26427e24d398e2695aad40ca134 P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:22:33.541594 16825 raft_consensus.cc:1240] T 66f75124215f4ca6bef831e174a792d8 P cb82cc7bd6db46d980c75237a1402860 [term 21 LEADER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 20. Current term is 21. Ops: []
I20260709 13:22:33.545521 16822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57a744f5aaf944678a63469cbf514015" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 21 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:33.545631 16824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2794af5b99984efebdee564035841cb0" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 21 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:33.525962 23605 leader_election.cc:290] T fffab26427e24d398e2695aad40ca134 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:33.509518 24330 leader_election.cc:290] T 8a878999c0024d2abe517484180290fc P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:33.426453 24321 consensus_queue.cc:237] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 23.13, Last appended by leader: 13, Current term: 25, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:33.553062 24240 raft_consensus.cc:493] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:22:33.557188 24240 raft_consensus.cc:515] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:33.559373 24161 leader_election.cc:290] T e5ad4eb281564936a59f41e387a47156 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:22:33.559373 24240 leader_election.cc:290] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:22:33.564635 24142 leader_election.cc:290] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:33.525390 24227 consensus_queue.cc:1048] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.026s
I20260709 13:22:33.569370 24215 leader_election.cc:290] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:22:33.462739 24307 raft_consensus.cc:515] T 8c11074579b046d99e585c2818e18a2c P cb82cc7bd6db46d980c75237a1402860 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:22:33.572028 16191 leader_election.cc:336] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 21 election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Timed out: RequestConsensusVote RPC to 127.14.177.129:36741 timed out after 2.297s (SENT)
I20260709 13:22:33.572685 16191 leader_election.cc:304] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:22:33.530157 24223 raft_consensus.cc:515] T 8d4ab870a05347868956aa8b841b2981 P 647a343a542d4a139fe106c19095129b [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:33.576226 24223 leader_election.cc:290] T 8d4ab870a05347868956aa8b841b2981 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:33.542253 24326 raft_consensus.cc:3060] T e4dc6b0541ca431ebc757beac00521f1 P 90f3da57796b488c9087a2f523adf045 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:22:33.585433 16295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a878999c0024d2abe517484180290fc" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:33.587333 24168 raft_consensus.cc:3060] T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:22:33.589891 24307 leader_election.cc:290] T 8c11074579b046d99e585c2818e18a2c P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:33.590004 16818 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49892 (request call id 1216) took 1236 ms. Trace:
I20260709 13:22:33.591450 24313 leader_election.cc:290] T c4f65cdf00b748bb963ee6e81ef43b79 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
W20260709 13:22:33.591727 16116 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:22:33.594653 15046 thread.cc:527] Waited for 5000ms trying to join with heartbeat (tid 16176)
I20260709 13:22:33.600499 16818 rpcz_store.cc:276] 0709 13:22:32.353634 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:22:32.353791 (+   157us) service_pool.cc:224] Handling call
0709 13:22:32.363537 (+  9746us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:22:33.010475 (+646938us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:22:33.589778 (+579303us) spinlock_profiling.cc:243] Waited 633 ms on lock 0x7b54004b5d54. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:22:33.589946 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1455037184}
I20260709 13:22:33.556492 24327 raft_consensus.cc:697] T d16dfad0c3b540e5b4b86492ddf87208 P 1ca2725d290e47d68b1a4b609c9816ac [term 23 LEADER]: Becoming Leader. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
W20260709 13:22:33.582243 24281 consensus_meta.cc:287] T 68a6838d82ef4cac83507f906139d90f P 1ca2725d290e47d68b1a4b609c9816ac: Time spent flushing consensus metadata: real 0.814s	user 0.008s	sys 0.004s
I20260709 13:22:33.612478 23994 raft_consensus.cc:493] T 9c61a39fd4f84424b129c1a6f9ba404d P 647a343a542d4a139fe106c19095129b [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:33.592577 24317 raft_consensus.cc:2749] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:22:33.633033 16192 leader_election.cc:304] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:22:33.637144 23994 raft_consensus.cc:515] T 9c61a39fd4f84424b129c1a6f9ba404d P 647a343a542d4a139fe106c19095129b [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:33.646214 16295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66ae176961b74813a58bc15c9c4c6f2a" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 29 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:33.639871 16294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2794af5b99984efebdee564035841cb0" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 21 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:33.647632 23994 leader_election.cc:290] T 9c61a39fd4f84424b129c1a6f9ba404d P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:33.643107 24222 leader_election.cc:290] T 74ca8140f35c4ae29e8a664f6bdfc829 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:33.583647 24301 raft_consensus.cc:493] T 08c74490c40243a88e62d1760b5cd814 P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:22:33.646950 16295 raft_consensus.cc:2468] T 66ae176961b74813a58bc15c9c4c6f2a P 90f3da57796b488c9087a2f523adf045 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 27.
I20260709 13:22:33.650929 16192 leader_election.cc:304] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860
I20260709 13:22:33.655543 24334 raft_consensus.cc:2749] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 13:22:33.655894 16192 leader_election.cc:304] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860
W20260709 13:22:33.652439 16114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:22:33.657119 24331 raft_consensus.cc:2749] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [term 25 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:22:33.549714 24289 raft_consensus.cc:515] T fffab26427e24d398e2695aad40ca134 P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:22:33.658250 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 137737us
I20260709 13:22:33.659642 24289 leader_election.cc:290] T fffab26427e24d398e2695aad40ca134 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
W20260709 13:22:33.660440 16115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 143419us
I20260709 13:22:33.660879 16218 raft_consensus.cc:1240] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac [term 23 LEADER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 22. Current term is 23. Ops: []
I20260709 13:22:33.661580 16115 leader_election.cc:304] T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:22:33.661937 24289 raft_consensus.cc:493] T 66f75124215f4ca6bef831e174a792d8 P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:33.663339 16115 leader_election.cc:304] T 8cc959fb7f704fba9009c0595467d074 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:22:33.664435 24162 raft_consensus.cc:3060] T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b [term 22 FOLLOWER]: Advancing to term 23
W20260709 13:22:33.665422 16265 leader_election.cc:336] T ecc1e363230043fe9fef390f0b979f20 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:22:33.665781 16191 leader_election.cc:336] T c2f5f89186204242bf377f7c42b2e2da P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:22:33.662874 23994 consensus_queue.cc:1048] T 08c74490c40243a88e62d1760b5cd814 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:22:33.666688 24341 raft_consensus.cc:2749] T 8cc959fb7f704fba9009c0595467d074 P 647a343a542d4a139fe106c19095129b [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:22:33.668574 16191 leader_election.cc:304] T c2f5f89186204242bf377f7c42b2e2da P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:22:33.663975 24289 raft_consensus.cc:515] T 66f75124215f4ca6bef831e174a792d8 P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:33.670416 23980 raft_consensus.cc:2749] T c2f5f89186204242bf377f7c42b2e2da P 1ca2725d290e47d68b1a4b609c9816ac [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:22:33.672309 24289 leader_election.cc:290] T 66f75124215f4ca6bef831e174a792d8 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
W20260709 13:22:33.673085 16191 consensus_peers.cc:597] T af57996a8be84d398f0f023e8d08ef77 P 1ca2725d290e47d68b1a4b609c9816ac -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:22:33.673463 16819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66f75124215f4ca6bef831e174a792d8" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 22 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:33.677575 16115 leader_election.cc:304] T 3884f864a96d428785895e5463e5728f P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860
I20260709 13:22:33.678740 23994 raft_consensus.cc:2749] T 3884f864a96d428785895e5463e5728f P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:22:33.681361 16115 leader_election.cc:304] T b87147731a68400e8cce9b879daa7c5d P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860
I20260709 13:22:33.686878 16115 leader_election.cc:304] T 434bff3082d141838068f323bdb0d1ed P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860
W20260709 13:22:33.689023 16264 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:22:33.686780 23994 raft_consensus.cc:2749] T b87147731a68400e8cce9b879daa7c5d P 647a343a542d4a139fe106c19095129b [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:22:33.691980 16217 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33050 (request call id 1402) took 1249 ms. Trace:
I20260709 13:22:33.692732 16217 rpcz_store.cc:276] 0709 13:22:32.442146 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:22:32.442290 (+   144us) service_pool.cc:224] Handling call
0709 13:22:32.442615 (+   325us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:22:32.442939 (+   324us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:22:32.442968 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:22:33.577875 (+1134907us) spinlock_profiling.cc:243] Waited 1.8 ms on lock 0x7b1800431d40. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b693ade1 00007f69b69475b1 00007f69ada402a6 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69af458a11 00007f69af468372 00007f69b0b9aa01 00007f69b0b5f43a
0709 13:22:33.578335 (+   460us) log.cc:844] Serialized 26 byte log entry
0709 13:22:33.690408 (+112073us) raft_consensus.cc:1645] Marking committed up to 15
0709 13:22:33.690456 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:22:33.690609 (+   153us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:22:33.691472 (+   863us) raft_consensus.cc:1685] finished
0709 13:22:33.691484 (+    12us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:22:33.691665 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":490,"raft.run_cpu_time_us":127,"raft.run_wall_time_us":9824,"spinlock_wait_cycles":5009024,"thread_start_us":111662,"threads_started":1,"wal-append.queue_time_us":106586}
I20260709 13:22:33.679836 24162 raft_consensus.cc:2749] T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:22:33.698123 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02838c86531d4f98bf58d1c22b8baf77" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:33.698942 16217 raft_consensus.cc:2468] T 02838c86531d4f98bf58d1c22b8baf77 P 1ca2725d290e47d68b1a4b609c9816ac [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 23.
I20260709 13:22:33.699889 16217 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57604 (request call id 3642) took 1162 ms. Trace:
I20260709 13:22:33.701222 16217 rpcz_store.cc:276] 0709 13:22:32.537127 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:22:33.697876 (+1160749us) service_pool.cc:224] Handling call
0709 13:22:33.699838 (+  1962us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:22:33.690876 23924 raft_consensus.cc:2804] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:22:33.699832 24313 raft_consensus.cc:493] T 5a96373bb0e14dce8a266b2fc6c58507 P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:22:33.695969 16115 leader_election.cc:304] T d16dfad0c3b540e5b4b86492ddf87208 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
W20260709 13:22:33.704993 16191 leader_election.cc:336] T 18ad7e3af5504571a69d10c00750a01c P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 19 election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:22:33.716068 16116 leader_election.cc:304] T 02838c86531d4f98bf58d1c22b8baf77 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
I20260709 13:22:33.691972 24227 raft_consensus.cc:2749] T 434bff3082d141838068f323bdb0d1ed P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:22:33.707645 24222 raft_consensus.cc:493] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:33.708119 24320 raft_consensus.cc:515] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 90f3da57796b488c9087a2f523adf045 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:33.633817 24281 raft_consensus.cc:515] T 68a6838d82ef4cac83507f906139d90f P 1ca2725d290e47d68b1a4b609c9816ac [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:33.684036 24301 raft_consensus.cc:515] T 08c74490c40243a88e62d1760b5cd814 P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:33.720876 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1df93779209748b9860f6d893dd3de53" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:33.726329 16291 raft_consensus.cc:1240] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 90f3da57796b488c9087a2f523adf045 [term 22 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 21. Current term is 22. Ops: []
I20260709 13:22:33.728407 16225 raft_consensus.cc:2393] T 68a6838d82ef4cac83507f906139d90f P 1ca2725d290e47d68b1a4b609c9816ac [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 23: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
I20260709 13:22:33.748178 16217 raft_consensus.cc:2393] T 1df93779209748b9860f6d893dd3de53 P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 21: Already voted for candidate 647a343a542d4a139fe106c19095129b in this term.
W20260709 13:22:33.740772 16115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 44899.5us
I20260709 13:22:33.751564 16115 leader_election.cc:304] T 23f82224f4564fea93572e8206bad1fe P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860
I20260709 13:22:33.756219 16115 leader_election.cc:304] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
W20260709 13:22:33.723637 24288 log.cc:927] Time spent T e5ad4eb281564936a59f41e387a47156 P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
W20260709 13:22:33.728755 24325 log.cc:927] Time spent T 1df93779209748b9860f6d893dd3de53 P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 0.113s	user 0.000s	sys 0.000s
I20260709 13:22:33.758419 24162 raft_consensus.cc:2749] T 23f82224f4564fea93572e8206bad1fe P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:22:33.764196 16217 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57600 (request call id 2332) took 1244 ms. Trace:
I20260709 13:22:33.792212 16217 rpcz_store.cc:276] 0709 13:22:32.519452 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:22:33.720594 (+1201142us) service_pool.cc:224] Handling call
0709 13:22:33.764135 (+ 43541us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:22:33.793236 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e4bfb9090c8460eb14fa7e0d818e7f1" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 25 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:33.724360 24276 raft_consensus.cc:2804] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:22:33.793875 24276 raft_consensus.cc:493] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Starting leader election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:22:33.794359 24276 raft_consensus.cc:3060] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:22:33.794440 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806d7690c2b644959152074bb2cab0a7" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
W20260709 13:22:33.788724 16191 leader_election.cc:336] T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 24 election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:22:33.750880 24284 consensus_meta.cc:287] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 90f3da57796b488c9087a2f523adf045: Time spent flushing consensus metadata: real 0.515s	user 0.001s	sys 0.010s
I20260709 13:22:33.797703 16292 raft_consensus.cc:2410] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 90f3da57796b488c9087a2f523adf045 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac for term 22 because replica has last-logged OpId of term: 20 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 12.
I20260709 13:22:33.724138 24341 raft_consensus.cc:2804] T 02838c86531d4f98bf58d1c22b8baf77 P 647a343a542d4a139fe106c19095129b [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:22:33.799449 24341 raft_consensus.cc:493] T 02838c86531d4f98bf58d1c22b8baf77 P 647a343a542d4a139fe106c19095129b [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:33.800099 24341 raft_consensus.cc:3060] T 02838c86531d4f98bf58d1c22b8baf77 P 647a343a542d4a139fe106c19095129b [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:22:33.806504 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d4ab870a05347868956aa8b841b2981" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:33.807322 16217 raft_consensus.cc:2468] T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 21.
I20260709 13:22:33.728080 24281 leader_election.cc:290] T 68a6838d82ef4cac83507f906139d90f P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 23 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:33.725411 24334 raft_consensus.cc:493] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:33.822032 24334 raft_consensus.cc:515] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:33.826840 24142 raft_consensus.cc:493] T 19baa7bc367844c288bca8f81439c413 P cb82cc7bd6db46d980c75237a1402860 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:33.827502 24142 raft_consensus.cc:515] T 19baa7bc367844c288bca8f81439c413 P cb82cc7bd6db46d980c75237a1402860 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:33.740379 23994 raft_consensus.cc:2749] T d16dfad0c3b540e5b4b86492ddf87208 P 647a343a542d4a139fe106c19095129b [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:22:33.824456 24334 leader_election.cc:290] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
W20260709 13:22:33.833249 24216 consensus_meta.cc:287] T ecc1e363230043fe9fef390f0b979f20 P 1ca2725d290e47d68b1a4b609c9816ac: Time spent flushing consensus metadata: real 1.197s	user 0.009s	sys 0.005s
I20260709 13:22:33.835285 24216 raft_consensus.cc:2749] T ecc1e363230043fe9fef390f0b979f20 P 1ca2725d290e47d68b1a4b609c9816ac [term 26 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:22:33.787346 24227 raft_consensus.cc:2749] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 647a343a542d4a139fe106c19095129b [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:22:33.788345 24338 log.cc:927] Time spent T e5ad4eb281564936a59f41e387a47156 P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 0.086s	user 0.000s	sys 0.000s
I20260709 13:22:33.817445 24215 raft_consensus.cc:493] T 9e65676b21b748728ad39668cfe722ec P cb82cc7bd6db46d980c75237a1402860 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:22:33.724128 24327 consensus_queue.cc:237] T d16dfad0c3b540e5b4b86492ddf87208 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 21.12, Last appended by leader: 12, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:33.827876 24176 raft_consensus.cc:3060] T af57996a8be84d398f0f023e8d08ef77 P 647a343a542d4a139fe106c19095129b [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:22:33.830883 24142 leader_election.cc:290] T 19baa7bc367844c288bca8f81439c413 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:33.727715 24320 leader_election.cc:290] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 22 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:33.721968 24313 raft_consensus.cc:515] T 5a96373bb0e14dce8a266b2fc6c58507 P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:33.819257 16116 leader_election.cc:304] T 8d4ab870a05347868956aa8b841b2981 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
W20260709 13:22:33.872265 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 53127.4us
I20260709 13:22:33.807626 16265 leader_election.cc:304] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860
I20260709 13:22:33.837250 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31969392ccdb43b491911a319e8bc7e1" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:33.873708 16299 raft_consensus.cc:2468] T 31969392ccdb43b491911a319e8bc7e1 P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 20.
I20260709 13:22:33.762024 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74ca8140f35c4ae29e8a664f6bdfc829" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 26 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
W20260709 13:22:33.843945 16191 leader_election.cc:336] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:22:33.854219 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5ad4eb281564936a59f41e387a47156" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 23 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:33.858940 16819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8638ee363273426db1393dd81e5c7b72" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
W20260709 13:22:33.877110 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 69559.5us
I20260709 13:22:33.875466 16191 leader_election.cc:304] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
I20260709 13:22:33.843350 24190 raft_consensus.cc:493] T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:33.748994 24222 raft_consensus.cc:515] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:33.878767 16221 raft_consensus.cc:2468] T ecc1e363230043fe9fef390f0b979f20 P 1ca2725d290e47d68b1a4b609c9816ac [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 26.
W20260709 13:22:33.883245 16191 consensus_peers.cc:597] T 829b4ad8c5a54eababeea1181b960d4b P 1ca2725d290e47d68b1a4b609c9816ac -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:22:33.884711 16191 consensus_peers.cc:597] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:22:33.885249 16191 leader_election.cc:336] T 5a96373bb0e14dce8a266b2fc6c58507 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:22:33.887413 16294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c11074579b046d99e585c2818e18a2c" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:33.888693 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fffab26427e24d398e2695aad40ca134" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 23 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:33.885643 16191 leader_election.cc:304] T 5a96373bb0e14dce8a266b2fc6c58507 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:22:33.890818 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c74490c40243a88e62d1760b5cd814" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:33.892024 16217 raft_consensus.cc:2410] T 08c74490c40243a88e62d1760b5cd814 P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 for term 22 because replica has last-logged OpId of term: 21 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 12.
I20260709 13:22:33.864225 24096 raft_consensus.cc:493] T d2ab0ac8347641e6955e9bce7d90c1a2 P cb82cc7bd6db46d980c75237a1402860 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:22:33.880187 24320 raft_consensus.cc:2749] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:22:33.808509 24326 raft_consensus.cc:515] T e4dc6b0541ca431ebc757beac00521f1 P 90f3da57796b488c9087a2f523adf045 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:33.875774 24180 raft_consensus.cc:493] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac [term 24 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
W20260709 13:22:33.895444 16191 leader_election.cc:336] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:22:33.857570 24215 raft_consensus.cc:515] T 9e65676b21b748728ad39668cfe722ec P cb82cc7bd6db46d980c75237a1402860 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:33.882488 24227 raft_consensus.cc:2804] T 8d4ab870a05347868956aa8b841b2981 P 647a343a542d4a139fe106c19095129b [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:22:33.858891 24341 raft_consensus.cc:515] T 02838c86531d4f98bf58d1c22b8baf77 P 647a343a542d4a139fe106c19095129b [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:22:33.905268 16191 leader_election.cc:336] T 746aabf856ab401c9ef5e859371e7b67 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:22:33.905449 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d4ab870a05347868956aa8b841b2981" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:33.914484 16191 leader_election.cc:304] T 746aabf856ab401c9ef5e859371e7b67 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
W20260709 13:22:33.916303 16191 leader_election.cc:336] T 7c2095a5f4ad49408dbc7f69c606120d P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:22:33.916415 16300 raft_consensus.cc:2393] T e4dc6b0541ca431ebc757beac00521f1 P 90f3da57796b488c9087a2f523adf045 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 24: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
I20260709 13:22:33.916796 16191 leader_election.cc:304] T 7c2095a5f4ad49408dbc7f69c606120d P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:22:33.898466 23938 raft_consensus.cc:2749] T 5a96373bb0e14dce8a266b2fc6c58507 P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:22:33.886417 24281 raft_consensus.cc:2749] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:22:33.907042 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c61a39fd4f84424b129c1a6f9ba404d" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:33.901897 24217 raft_consensus.cc:2749] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [term 27 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
W20260709 13:22:33.919713 16191 leader_election.cc:336] T 8cc959fb7f704fba9009c0595467d074 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:22:33.921108 16191 leader_election.cc:304] T 8cc959fb7f704fba9009c0595467d074 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:22:33.888149 24190 raft_consensus.cc:515] T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:33.928670 24341 leader_election.cc:290] T 02838c86531d4f98bf58d1c22b8baf77 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 24 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:33.929157 24222 leader_election.cc:290] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:33.932181 24326 leader_election.cc:290] T e4dc6b0541ca431ebc757beac00521f1 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 24 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:22:33.932518 24303 raft_consensus.cc:515] T 5a959dbecc2648eb85d1cc550f5a3245 P 1ca2725d290e47d68b1a4b609c9816ac [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:33.909315 24227 raft_consensus.cc:493] T 8d4ab870a05347868956aa8b841b2981 P 647a343a542d4a139fe106c19095129b [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:33.901230 24096 raft_consensus.cc:515] T d2ab0ac8347641e6955e9bce7d90c1a2 P cb82cc7bd6db46d980c75237a1402860 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:33.946427 24227 raft_consensus.cc:3060] T 8d4ab870a05347868956aa8b841b2981 P 647a343a542d4a139fe106c19095129b [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:22:33.922272 24334 raft_consensus.cc:2749] T 7c2095a5f4ad49408dbc7f69c606120d P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:22:33.919931 24216 raft_consensus.cc:2749] T 746aabf856ab401c9ef5e859371e7b67 P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:22:33.958050 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c61a39fd4f84424b129c1a6f9ba404d" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:33.959566 24301 leader_election.cc:290] T 08c74490c40243a88e62d1760b5cd814 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:33.910318 24180 raft_consensus.cc:515] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:33.968917 24190 leader_election.cc:290] T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:33.970707 24215 leader_election.cc:290] T 9e65676b21b748728ad39668cfe722ec P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:33.976168 24274 raft_consensus.cc:2749] T 8cc959fb7f704fba9009c0595467d074 P 1ca2725d290e47d68b1a4b609c9816ac [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:22:33.977792 16299 raft_consensus.cc:2468] T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 24.
W20260709 13:22:33.974855 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 55180.5us
W20260709 13:22:34.048273 16191 consensus_peers.cc:597] T c0eab25ce4c64e018a041beeda2867db P 1ca2725d290e47d68b1a4b609c9816ac -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:22:34.048959 16191 consensus_peers.cc:597] T e5ad4eb281564936a59f41e387a47156 P 1ca2725d290e47d68b1a4b609c9816ac -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:22:33.968930 24303 leader_election.cc:290] T 5a959dbecc2648eb85d1cc550f5a3245 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 22 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:33.984193 24307 raft_consensus.cc:493] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:22:34.056303 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e65676b21b748728ad39668cfe722ec" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:33.977777 24281 raft_consensus.cc:493] T ee9ed67006144797b1de6ea8c2013a68 P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
W20260709 13:22:34.061039 16191 consensus_peers.cc:597] T 18ad7e3af5504571a69d10c00750a01c P 1ca2725d290e47d68b1a4b609c9816ac -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:22:34.067351 24308 log.cc:927] Time spent T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 0.893s	user 0.009s	sys 0.006s
I20260709 13:22:34.069926 16291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 27 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:34.070680 16291 raft_consensus.cc:2468] T ecc1e363230043fe9fef390f0b979f20 P 90f3da57796b488c9087a2f523adf045 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 26.
I20260709 13:22:34.071660 16291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 26 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:34.077641 16218 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57600 (request call id 2322) took 1674 ms. Trace:
I20260709 13:22:34.078091 16218 rpcz_store.cc:276] 0709 13:22:32.403482 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:22:32.403811 (+   329us) service_pool.cc:224] Handling call
0709 13:22:32.404094 (+   283us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:22:33.661293 (+1257199us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:22:34.077382 (+416089us) spinlock_profiling.cc:243] Waited 1.26 s on lock 0x7b5400138b54. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:22:34.077579 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2889939840}
I20260709 13:22:34.079341 16218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4dc6b0541ca431ebc757beac00521f1" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:22:34.079897 16218 raft_consensus.cc:3060] T e4dc6b0541ca431ebc757beac00521f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:22:34.084189 16220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0eab25ce4c64e018a041beeda2867db" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 25 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:34.085474 16220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02838c86531d4f98bf58d1c22b8baf77" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:22:34.085989 16220 raft_consensus.cc:3060] T 02838c86531d4f98bf58d1c22b8baf77 P 1ca2725d290e47d68b1a4b609c9816ac [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:22:34.090467 16218 raft_consensus.cc:2468] T e4dc6b0541ca431ebc757beac00521f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 24.
I20260709 13:22:34.092543 16222 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57604 (request call id 3628) took 1646 ms. Trace:
I20260709 13:22:34.094671 16265 leader_election.cc:304] T e4dc6b0541ca431ebc757beac00521f1 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
W20260709 13:22:34.097822 16191 leader_election.cc:336] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:22:34.099264 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02838c86531d4f98bf58d1c22b8baf77" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:22:34.099869 16299 raft_consensus.cc:3060] T 02838c86531d4f98bf58d1c22b8baf77 P 90f3da57796b488c9087a2f523adf045 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:22:34.093022 16222 rpcz_store.cc:276] 0709 13:22:32.446121 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:22:32.446248 (+   127us) service_pool.cc:224] Handling call
0709 13:22:32.457415 (+ 11167us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:22:32.457802 (+   387us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:22:32.457844 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:22:33.831866 (+1374022us) spinlock_profiling.cc:243] Waited 3.93 ms on lock 0x7b18004b0e20. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b693ade1 00007f69b69475b1 00007f69ada402a6 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69af458a11 00007f69af468372 00007f69b0b9aa01 00007f69b0b5f43a
0709 13:22:33.832897 (+  1031us) spinlock_profiling.cc:243] Waited 1.94 ms on lock 0x7b2c00115ac8. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69af458a11 00007f69af468372 00007f69b0b9aa01 00007f69b0b5f43a 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2
0709 13:22:33.833416 (+   519us) log.cc:844] Serialized 26 byte log entry
0709 13:22:33.899393 (+ 65977us) raft_consensus.cc:1645] Marking committed up to 16
0709 13:22:33.899446 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:22:33.899565 (+   119us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:22:34.091099 (+191534us) spinlock_profiling.cc:243] Waited 1.63 ms on lock 0x7b4400022b4c. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69ad8fadb1 00007f69ad998c85 00007f69ad9996a2 00007f69b070b1db 00007f69b0706aaf 00007f69b06fa86e 00007f69b06f9bb6 00007f69b06f97d8 00007f69b0703221 00007f69b07031d9 00007f69b0703169
0709 13:22:34.092238 (+  1139us) raft_consensus.cc:1685] finished
0709 13:22:34.092271 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:22:34.092474 (+   203us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":17284,"raft.queue_time_us":93,"raft.run_cpu_time_us":108,"raft.run_wall_time_us":16851,"spinlock_wait_cycles":17238912,"thread_start_us":65566,"threads_started":1,"wal-append.queue_time_us":124048}
I20260709 13:22:34.098294 16191 leader_election.cc:304] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
W20260709 13:22:34.105290 16191 leader_election.cc:336] T d16dfad0c3b540e5b4b86492ddf87208 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:22:34.092615 24350 log.cc:927] Time spent T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.116s	user 0.000s	sys 0.001s
I20260709 13:22:34.108575 16222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4f65cdf00b748bb963ee6e81ef43b79" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 24 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
W20260709 13:22:34.090544 24347 log.cc:927] Time spent T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860: Append to log took a long time: real 0.138s	user 0.000s	sys 0.001s
I20260709 13:22:34.119035 16218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b7d1ab519044eda2d5e04a07242f25" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 23 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:34.105415 24326 raft_consensus.cc:2804] T e4dc6b0541ca431ebc757beac00521f1 P 90f3da57796b488c9087a2f523adf045 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 13:22:34.106431 24327 raft_consensus.cc:2749] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:22:34.104259 24096 leader_election.cc:290] T d2ab0ac8347641e6955e9bce7d90c1a2 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:34.062098 24281 raft_consensus.cc:515] T ee9ed67006144797b1de6ea8c2013a68 P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:34.060140 24307 raft_consensus.cc:515] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:34.136564 16218 raft_consensus.cc:2468] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 22.
W20260709 13:22:34.121109 16191 leader_election.cc:336] T ecc1e363230043fe9fef390f0b979f20 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:22:34.140587 16191 leader_election.cc:336] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:22:34.141008 16191 leader_election.cc:304] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
W20260709 13:22:34.142046 16191 leader_election.cc:336] T 68a6838d82ef4cac83507f906139d90f P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:22:34.143034 16191 consensus_peers.cc:597] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:22:34.143558 16191 leader_election.cc:336] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:22:34.144819 16191 leader_election.cc:336] T 5a959dbecc2648eb85d1cc550f5a3245 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:22:34.145009 16299 raft_consensus.cc:2468] T 02838c86531d4f98bf58d1c22b8baf77 P 90f3da57796b488c9087a2f523adf045 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 24.
W20260709 13:22:34.145699 16191 consensus_peers.cc:597] T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:22:34.146488 16191 leader_election.cc:336] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:22:34.146526 16299 raft_consensus.cc:3055] T 8a878999c0024d2abe517484180290fc P 90f3da57796b488c9087a2f523adf045 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:22:34.147037 16299 raft_consensus.cc:740] T 8a878999c0024d2abe517484180290fc P 90f3da57796b488c9087a2f523adf045 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:22:34.130764 24231 raft_consensus.cc:493] T af57996a8be84d398f0f023e8d08ef77 P 647a343a542d4a139fe106c19095129b [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:34.104949 24276 raft_consensus.cc:515] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac [term 22 FOLLOWER]: Starting leader election with config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:34.150300 16298 raft_consensus.cc:1275] T 1df93779209748b9860f6d893dd3de53 P 90f3da57796b488c9087a2f523adf045 [term 21 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
W20260709 13:22:34.148921 16191 leader_election.cc:336] T d16dfad0c3b540e5b4b86492ddf87208 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 23 election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:22:34.135568 24326 raft_consensus.cc:697] T e4dc6b0541ca431ebc757beac00521f1 P 90f3da57796b488c9087a2f523adf045 [term 24 LEADER]: Becoming Leader. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
W20260709 13:22:34.153975 16191 leader_election.cc:336] T 8a878999c0024d2abe517484180290fc P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:22:34.158811 16298 raft_consensus.cc:1275] T ada5a004ba7c46819051e3ccec71ffdb P 90f3da57796b488c9087a2f523adf045 [term 24 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 22 index: 16. Preceding OpId from leader: term: 24 index: 17. (index mismatch)
W20260709 13:22:34.160759 16114 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:22:34.170689 24281 leader_election.cc:290] T ee9ed67006144797b1de6ea8c2013a68 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:34.154569 16191 leader_election.cc:304] T 8a878999c0024d2abe517484180290fc P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:22:34.148037 24327 raft_consensus.cc:2749] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:22:34.154894 16299 consensus_queue.cc:260] T 8a878999c0024d2abe517484180290fc P 90f3da57796b488c9087a2f523adf045 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.11, Last appended by leader: 11, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:22:34.180871 16191 leader_election.cc:336] T fffab26427e24d398e2695aad40ca134 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:22:34.180430 16299 raft_consensus.cc:3060] T 8a878999c0024d2abe517484180290fc P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:22:34.181607 16191 leader_election.cc:336] T 66f75124215f4ca6bef831e174a792d8 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:22:34.182109 16191 leader_election.cc:304] T 66f75124215f4ca6bef831e174a792d8 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
W20260709 13:22:34.184500 16191 leader_election.cc:336] T 68a6838d82ef4cac83507f906139d90f P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 23 election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:22:34.158106 24231 raft_consensus.cc:515] T af57996a8be84d398f0f023e8d08ef77 P 647a343a542d4a139fe106c19095129b [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:22:34.189388 16191 leader_election.cc:336] T 08c74490c40243a88e62d1760b5cd814 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:22:34.181553 24281 raft_consensus.cc:2749] T 8a878999c0024d2abe517484180290fc P 1ca2725d290e47d68b1a4b609c9816ac [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:22:34.193884 16191 leader_election.cc:336] T 5a959dbecc2648eb85d1cc550f5a3245 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:22:34.185773 24327 raft_consensus.cc:2749] T 66f75124215f4ca6bef831e174a792d8 P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:22:34.197757 16299 raft_consensus.cc:1275] T 8a878999c0024d2abe517484180290fc P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 20 index: 11. (term mismatch)
I20260709 13:22:34.198215 16299 pending_rounds.cc:85] T 8a878999c0024d2abe517484180290fc P 90f3da57796b488c9087a2f523adf045: Aborting all ops after (but not including) 10
I20260709 13:22:34.198505 16299 pending_rounds.cc:107] T 8a878999c0024d2abe517484180290fc P 90f3da57796b488c9087a2f523adf045: Aborting uncommitted NO_OP operation due to leader change: 19.11
I20260709 13:22:34.198807 16299 raft_consensus.cc:2889] T 8a878999c0024d2abe517484180290fc P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:22:34.203469 16115 leader_election.cc:304] T ecc1e363230043fe9fef390f0b979f20 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860
I20260709 13:22:34.204486 24184 raft_consensus.cc:3060] T ecc1e363230043fe9fef390f0b979f20 P 647a343a542d4a139fe106c19095129b [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:22:34.207376 16219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4dc6b0541ca431ebc757beac00521f1" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 24 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:34.208189 16219 raft_consensus.cc:2393] T e4dc6b0541ca431ebc757beac00521f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 24: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
I20260709 13:22:34.209023 16219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fffab26427e24d398e2695aad40ca134" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 23 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
W20260709 13:22:34.210774 16225 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57600 (request call id 2333) took 1691 ms (client timeout 1941 ms). Trace:
W20260709 13:22:34.211279 16225 rpcz_store.cc:269] 0709 13:22:32.519708 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:22:33.305759 (+786051us) service_pool.cc:224] Handling call
0709 13:22:34.210589 (+904830us) spinlock_profiling.cc:243] Waited 421 ms on lock 0x7b5400297cd4. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:22:34.210718 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":968905088}
I20260709 13:22:34.212119 16115 leader_election.cc:304] T 18ad7e3af5504571a69d10c00750a01c P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:22:34.212500 16225 raft_consensus.cc:1240] T 5a959dbecc2648eb85d1cc550f5a3245 P 1ca2725d290e47d68b1a4b609c9816ac [term 22 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 20. Current term is 22. Ops: []
I20260709 13:22:34.213081 24223 raft_consensus.cc:2749] T 18ad7e3af5504571a69d10c00750a01c P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:22:34.213838 16115 leader_election.cc:304] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 26 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:22:34.213917 24223 consensus_queue.cc:1059] T 84f2da98864b4425bf7c0055951fdd4b P 647a343a542d4a139fe106c19095129b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: INVALID_TERM, Last received: 19.11, Next index: 12, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:22:34.214987 16115 leader_election.cc:304] T e5ad4eb281564936a59f41e387a47156 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:22:34.215308 23994 raft_consensus.cc:3055] T 84f2da98864b4425bf7c0055951fdd4b P 647a343a542d4a139fe106c19095129b [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:22:34.215720 23994 raft_consensus.cc:740] T 84f2da98864b4425bf7c0055951fdd4b P 647a343a542d4a139fe106c19095129b [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:22:34.216748 24184 raft_consensus.cc:2749] T ecc1e363230043fe9fef390f0b979f20 P 647a343a542d4a139fe106c19095129b [term 26 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:22:34.216554 23994 consensus_queue.cc:260] T 84f2da98864b4425bf7c0055951fdd4b P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:34.218937 23994 raft_consensus.cc:3060] T 84f2da98864b4425bf7c0055951fdd4b P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:22:34.219791 23924 raft_consensus.cc:493] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac [term 24 FOLLOWER]: Starting leader election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:22:34.220333 23924 raft_consensus.cc:3060] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:22:34.213145 16225 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57600 (request call id 2325) took 1755 ms. Trace:
I20260709 13:22:34.217329 24199 raft_consensus.cc:2749] T e5ad4eb281564936a59f41e387a47156 P 647a343a542d4a139fe106c19095129b [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:22:34.215044 24223 raft_consensus.cc:2749] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b [term 26 FOLLOWER]: Leader election lost for term 26. Reason: could not achieve majority
I20260709 13:22:34.223619 16297 raft_consensus.cc:1275] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 90f3da57796b488c9087a2f523adf045 [term 21 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 20 index: 13. Preceding OpId from leader: term: 21 index: 13. (term mismatch)
I20260709 13:22:34.224050 16297 pending_rounds.cc:85] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 90f3da57796b488c9087a2f523adf045: Aborting all ops after (but not including) 12
I20260709 13:22:34.223788 16225 rpcz_store.cc:276] 0709 13:22:32.458171 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:22:34.212074 (+1753903us) service_pool.cc:224] Handling call
0709 13:22:34.212349 (+   275us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:22:34.212886 (+   537us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:22:34.213096 (+   210us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:22:34.224438 16297 pending_rounds.cc:107] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 90f3da57796b488c9087a2f523adf045: Aborting uncommitted NO_OP operation due to leader change: 20.13
I20260709 13:22:34.224707 16297 raft_consensus.cc:2889] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 90f3da57796b488c9087a2f523adf045 [term 21 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:22:34.191118 24283 raft_consensus.cc:493] T 84f2da98864b4425bf7c0055951fdd4b P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:22:34.230602 16115 leader_election.cc:304] T 7c2095a5f4ad49408dbc7f69c606120d P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:22:34.231094 24283 raft_consensus.cc:515] T 84f2da98864b4425bf7c0055951fdd4b P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:34.231899 16115 leader_election.cc:304] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860
I20260709 13:22:34.232693 24242 raft_consensus.cc:2749] T 7c2095a5f4ad49408dbc7f69c606120d P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:22:34.233208 24283 leader_election.cc:290] T 84f2da98864b4425bf7c0055951fdd4b P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:34.233421 24242 raft_consensus.cc:3060] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:22:34.234831 16115 leader_election.cc:304] T 68a6838d82ef4cac83507f906139d90f P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
W20260709 13:22:34.237378 16263 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:22:34.189533 24120 raft_consensus.cc:493] T b6f07842d3aa47858df5665874c86f3a P 1ca2725d290e47d68b1a4b609c9816ac [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:34.191627 24200 raft_consensus.cc:493] T c0eab25ce4c64e018a041beeda2867db P 647a343a542d4a139fe106c19095129b [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:34.238935 16219 raft_consensus.cc:2410] T fffab26427e24d398e2695aad40ca134 P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b for term 23 because replica has last-logged OpId of term: 21 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 13.
I20260709 13:22:34.241214 16117 leader_election.cc:304] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860
I20260709 13:22:34.241652 24307 leader_election.cc:290] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:22:34.242637 16219 raft_consensus.cc:1240] T 5a959dbecc2648eb85d1cc550f5a3245 P 1ca2725d290e47d68b1a4b609c9816ac [term 22 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 21. Current term is 22. Ops: []
I20260709 13:22:34.243222 16219 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57604 (request call id 3639) took 1760 ms. Trace:
I20260709 13:22:34.243637 16219 rpcz_store.cc:276] 0709 13:22:32.483074 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:22:34.242259 (+1759185us) service_pool.cc:224] Handling call
0709 13:22:34.242513 (+   254us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:22:34.242990 (+   477us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:22:34.243125 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:22:34.248118 24242 raft_consensus.cc:2749] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [term 25 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:22:34.247887 24227 raft_consensus.cc:515] T 8d4ab870a05347868956aa8b841b2981 P 647a343a542d4a139fe106c19095129b [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:34.250447 16225 raft_consensus.cc:1240] T 66f75124215f4ca6bef831e174a792d8 P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 20. Current term is 21. Ops: []
I20260709 13:22:34.250823 24242 raft_consensus.cc:2749] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:22:34.249929 24120 raft_consensus.cc:515] T b6f07842d3aa47858df5665874c86f3a P 1ca2725d290e47d68b1a4b609c9816ac [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:34.195329 24326 consensus_queue.cc:237] T e4dc6b0541ca431ebc757beac00521f1 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 22.12, Last appended by leader: 12, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:34.253721 24276 leader_election.cc:290] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 22 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:34.250165 24200 raft_consensus.cc:515] T c0eab25ce4c64e018a041beeda2867db P 647a343a542d4a139fe106c19095129b [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:34.260648 16220 raft_consensus.cc:2468] T 02838c86531d4f98bf58d1c22b8baf77 P 1ca2725d290e47d68b1a4b609c9816ac [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 24.
I20260709 13:22:34.261729 16217 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57600 (request call id 2326) took 1748 ms. Trace:
I20260709 13:22:34.262290 16217 rpcz_store.cc:276] 0709 13:22:32.513561 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:22:34.260633 (+1747072us) service_pool.cc:224] Handling call
0709 13:22:34.261066 (+   433us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:22:34.261306 (+   240us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:22:34.261345 (+    39us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:22:34.261409 (+    64us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:22:34.261463 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:22:34.261524 (+    61us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:22:34.261657 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:22:34.267918 16223 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57604 (request call id 3643) took 1495 ms. Trace:
I20260709 13:22:34.268488 16115 leader_election.cc:304] T 26354361e9b044cea1c2b86c4882fe92 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860
I20260709 13:22:34.268450 16223 rpcz_store.cc:276] 0709 13:22:32.772497 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:22:34.266900 (+1494403us) service_pool.cc:224] Handling call
0709 13:22:34.267251 (+   351us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:22:34.267484 (+   233us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:22:34.267524 (+    40us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:22:34.267587 (+    63us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:22:34.267633 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:22:34.267710 (+    77us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:22:34.267848 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:22:34.271828 16223 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33050 (request call id 1405) took 1128 ms. Trace:
I20260709 13:22:34.271884 16115 leader_election.cc:304] T 57a744f5aaf944678a63469cbf514015 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860
I20260709 13:22:34.272536 16223 rpcz_store.cc:276] 0709 13:22:33.143403 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:22:34.270455 (+1127052us) service_pool.cc:224] Handling call
0709 13:22:34.270876 (+   421us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:22:34.271410 (+   534us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:22:34.271460 (+    50us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:22:34.271520 (+    60us) raft_consensus.cc:1645] Marking committed up to 18
0709 13:22:34.271568 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:22:34.271644 (+    76us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:22:34.271760 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:22:34.253762 24223 raft_consensus.cc:2749] T 68a6838d82ef4cac83507f906139d90f P 647a343a542d4a139fe106c19095129b [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:22:34.276973 24190 consensus_queue.cc:1048] T 5a959dbecc2648eb85d1cc550f5a3245 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:22:34.253741 24120 leader_election.cc:290] T b6f07842d3aa47858df5665874c86f3a P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:34.283878 16219 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57604 (request call id 3640) took 1797 ms. Trace:
I20260709 13:22:34.278208 24199 raft_consensus.cc:2749] T 26354361e9b044cea1c2b86c4882fe92 P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:22:34.290084 16115 leader_election.cc:304] T 2794af5b99984efebdee564035841cb0 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860
W20260709 13:22:34.287257 24314 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 20 current term: 20 is higher.
I20260709 13:22:34.269520 23994 consensus_queue.cc:1048] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.022s
W20260709 13:22:34.291785 16265 leader_election.cc:336] T 84f2da98864b4425bf7c0055951fdd4b P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 20 election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:22:34.297554 16115 leader_election.cc:304] T 9c61a39fd4f84424b129c1a6f9ba404d P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac
I20260709 13:22:34.281731 24223 consensus_queue.cc:1059] T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.011s
I20260709 13:22:34.285696 24190 raft_consensus.cc:2749] T 57a744f5aaf944678a63469cbf514015 P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:22:34.295583 16265 leader_election.cc:304] T 84f2da98864b4425bf7c0055951fdd4b P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
W20260709 13:22:34.308036 16265 leader_election.cc:336] T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:22:34.308600 16265 leader_election.cc:304] T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:22:34.309877 16291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ab0ac8347641e6955e9bce7d90c1a2" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:34.310010 16825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b7d1ab519044eda2d5e04a07242f25" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 23 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:34.311100 16825 raft_consensus.cc:2468] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 21.
I20260709 13:22:34.312180 16789 leader_election.cc:304] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:22:34.313359 24096 raft_consensus.cc:2804] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 13:22:34.313741 24096 raft_consensus.cc:493] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [term 28 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:34.307969 16220 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57604 (request call id 3641) took 1774 ms. Trace:
I20260709 13:22:34.315574 16115 leader_election.cc:304] T 02838c86531d4f98bf58d1c22b8baf77 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:22:34.309062 16816 raft_consensus.cc:2955] T 806d7690c2b644959152074bb2cab0a7 P cb82cc7bd6db46d980c75237a1402860 [term 21 FOLLOWER]: Committing config change with OpId 21.15: config changed from index 2 to 15, cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) changed from NON_VOTER to VOTER. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:22:34.315052 16220 rpcz_store.cc:276] 0709 13:22:32.533958 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:22:34.261350 (+1727392us) service_pool.cc:224] Handling call
0709 13:22:34.261579 (+   229us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:22:34.261715 (+   136us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 13:22:34.275473 (+ 13758us) log.cc:844] Serialized 14 byte log entry
0709 13:22:34.307657 (+ 32184us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:22:34.307707 (+    50us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:22:34.307748 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:22:34.307807 (+    59us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:22:34.307912 (+   105us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":13581,"prepare.run_cpu_time_us":52,"prepare.run_wall_time_us":1066,"raft.queue_time_us":3525,"thread_start_us":31778,"threads_started":1}
I20260709 13:22:34.303825 24199 raft_consensus.cc:2749] T 2794af5b99984efebdee564035841cb0 P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:22:34.314354 16789 leader_election.cc:304] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:22:34.304944 24223 consensus_queue.cc:1243] T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b [LEADER]: Peer cb82cc7bd6db46d980c75237a1402860 log is divergent from this leader: its last log entry 21.16 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 15
I20260709 13:22:34.293670 24184 consensus_queue.cc:1048] T b6f07842d3aa47858df5665874c86f3a P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.024s
I20260709 13:22:34.307588 24314 raft_consensus.cc:2749] T 84f2da98864b4425bf7c0055951fdd4b P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:22:34.307570 24200 leader_election.cc:290] T c0eab25ce4c64e018a041beeda2867db P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:34.311275 24242 raft_consensus.cc:2804] T 9c61a39fd4f84424b129c1a6f9ba404d P 647a343a542d4a139fe106c19095129b [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:22:34.314083 24096 raft_consensus.cc:3060] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [term 28 FOLLOWER]: Advancing to term 29
I20260709 13:22:34.329880 16220 raft_consensus.cc:1275] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
I20260709 13:22:34.331203 24142 consensus_queue.cc:1048] T ada5a004ba7c46819051e3ccec71ffdb P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
W20260709 13:22:34.333743 16036 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet ecc1e363230043fe9fef390f0b979f20 with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER failed with leader cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica cb82cc7bd6db46d980c75237a1402860 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 26 committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:22:34.340524 24180 leader_election.cc:290] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:34.319993 24341 raft_consensus.cc:2804] T 02838c86531d4f98bf58d1c22b8baf77 P 647a343a542d4a139fe106c19095129b [term 24 FOLLOWER]: Leader election won for term 24
I20260709 13:22:34.332466 24215 raft_consensus.cc:2804] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:22:34.343201 24215 raft_consensus.cc:493] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:22:34.343572 24215 raft_consensus.cc:3060] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:22:34.356895 16225 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57604 (request call id 3637) took 1898 ms. Trace:
I20260709 13:22:34.357496 16225 rpcz_store.cc:276] 0709 13:22:32.458839 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:22:34.224764 (+1765925us) service_pool.cc:224] Handling call
0709 13:22:34.225045 (+   281us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:22:34.250867 (+ 25822us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:22:34.356527 (+105660us) spinlock_profiling.cc:243] Waited 25.2 ms on lock 0x7b54001428d4. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:22:34.356811 (+   284us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":57998592}
I20260709 13:22:34.359982 24227 leader_election.cc:290] T 8d4ab870a05347868956aa8b841b2981 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 22 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:34.335035 24242 raft_consensus.cc:493] T 9c61a39fd4f84424b129c1a6f9ba404d P 647a343a542d4a139fe106c19095129b [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:34.361517 24242 raft_consensus.cc:3060] T 9c61a39fd4f84424b129c1a6f9ba404d P 647a343a542d4a139fe106c19095129b [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:22:34.368171 16218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b7d1ab519044eda2d5e04a07242f25" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:34.368964 16218 raft_consensus.cc:2393] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 in current term 22: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
I20260709 13:22:34.369814 16218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62fe3346cd3c42c4a55e14af9073bca4" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 28 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:34.287215 16219 rpcz_store.cc:276] 0709 13:22:32.486444 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:22:34.244708 (+1758264us) service_pool.cc:224] Handling call
0709 13:22:34.245049 (+   341us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:22:34.245289 (+   240us) raft_consensus.cc:1534] Early marking committed up to index 16
0709 13:22:34.245867 (+   578us) log.cc:844] Serialized 14 byte log entry
0709 13:22:34.281286 (+ 35419us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:22:34.281348 (+    62us) raft_consensus.cc:1645] Marking committed up to 16
0709 13:22:34.283666 (+  2318us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:22:34.283728 (+    62us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:22:34.283830 (+   102us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":177,"prepare.run_cpu_time_us":32,"prepare.run_wall_time_us":33,"raft.queue_time_us":821,"raft.run_cpu_time_us":116,"raft.run_wall_time_us":2975,"thread_start_us":34985,"threads_started":1}
I20260709 13:22:34.373224 16219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74ca8140f35c4ae29e8a664f6bdfc829" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 26 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:34.373839 16219 raft_consensus.cc:2468] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 25.
I20260709 13:22:34.374347 16217 raft_consensus.cc:1275] T 66f75124215f4ca6bef831e174a792d8 P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 21 index: 16. (index mismatch)
I20260709 13:22:34.317951 24298 raft_consensus.cc:2749] T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:22:34.333962 16117 leader_election.cc:304] T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:22:34.378423 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdbd78acb4cf46298e0de8ba0d130e19" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 26 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
W20260709 13:22:34.373351 16221 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57600 (request call id 2330) took 1854 ms (client timeout 1769 ms). Trace:
W20260709 13:22:34.379403 16221 rpcz_store.cc:269] 0709 13:22:32.518922 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:22:32.746888 (+227966us) service_pool.cc:224] Handling call
0709 13:22:34.373162 (+1626274us) spinlock_profiling.cc:243] Waited 1.04 s on lock 0x7b54002b4b54. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:22:34.373264 (+   102us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2386403072}
I20260709 13:22:34.380822 16221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee9ed67006144797b1de6ea8c2013a68" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:34.386065 24341 raft_consensus.cc:697] T 02838c86531d4f98bf58d1c22b8baf77 P 647a343a542d4a139fe106c19095129b [term 24 LEADER]: Becoming Leader. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:22:34.386282 16219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5718c623ebee4d2bb770f0f9ddb2f164" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:34.386906 16219 raft_consensus.cc:2468] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 23.
I20260709 13:22:34.387113 24341 consensus_queue.cc:237] T 02838c86531d4f98bf58d1c22b8baf77 P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 22.13, Last appended by leader: 13, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:34.387838 16219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ab0ac8347641e6955e9bce7d90c1a2" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:34.388705 16221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66ae176961b74813a58bc15c9c4c6f2a" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 29 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:34.388909 16219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c11074579b046d99e585c2818e18a2c" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:34.389557 16221 raft_consensus.cc:2468] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 28.
I20260709 13:22:34.327301 24223 consensus_queue.cc:1048] T 1df93779209748b9860f6d893dd3de53 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.063s
I20260709 13:22:34.390389 24275 leader_election.cc:290] T 8158bbd17b994d2eb96f560bd0f1b88d P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:22:34.390923 24313 leader_election.cc:290] T 5a96373bb0e14dce8a266b2fc6c58507 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:22:34.318173 24190 consensus_queue.cc:1059] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 647a343a542d4a139fe106c19095129b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: INVALID_TERM, Last received: 21.14, Next index: 15, Last known committed idx: 13, Time since last communication: 0.017s
W20260709 13:22:34.391927 16226 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57600 (request call id 2324) took 1950 ms (client timeout 1881 ms). Trace:
I20260709 13:22:34.392169 16224 raft_consensus.cc:1240] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac [term 22 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 21. Current term is 22. Ops: []
I20260709 13:22:34.394229 16224 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57604 (request call id 3633) took 1940 ms. Trace:
I20260709 13:22:34.398615 24231 leader_election.cc:290] T af57996a8be84d398f0f023e8d08ef77 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:34.403854 16219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a96373bb0e14dce8a266b2fc6c58507" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:34.404497 23719 raft_consensus.cc:3055] T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:22:34.404906 23719 raft_consensus.cc:740] T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:22:34.405129 16219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8158bbd17b994d2eb96f560bd0f1b88d" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 22 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:34.405795 16219 raft_consensus.cc:2468] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 21.
W20260709 13:22:34.408386 16117 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 74491.4us
I20260709 13:22:34.411382 24227 raft_consensus.cc:2804] T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:22:34.411826 24227 raft_consensus.cc:493] T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:34.412159 24227 raft_consensus.cc:3060] T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:22:34.392256 24190 raft_consensus.cc:3055] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 647a343a542d4a139fe106c19095129b [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:22:34.412613 23865 consensus_queue.cc:1059] T cdbd78acb4cf46298e0de8ba0d130e19 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: INVALID_TERM, Last received: 25.19, Next index: 20, Last known committed idx: 18, Time since last communication: 0.001s
I20260709 13:22:34.413797 23865 consensus_queue.cc:1048] T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 13:22:34.415375 24304 raft_consensus.cc:3055] T cdbd78acb4cf46298e0de8ba0d130e19 P 90f3da57796b488c9087a2f523adf045 [term 25 LEADER]: Stepping down as leader of term 25
I20260709 13:22:34.415895 24304 raft_consensus.cc:740] T cdbd78acb4cf46298e0de8ba0d130e19 P 90f3da57796b488c9087a2f523adf045 [term 25 LEADER]: Becoming Follower/Learner. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:22:34.417079 24304 consensus_queue.cc:260] T cdbd78acb4cf46298e0de8ba0d130e19 P 90f3da57796b488c9087a2f523adf045 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 25.19, Last appended by leader: 19, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:34.419255 16219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 26 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:34.422133 16190 leader_election.cc:304] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:22:34.424093 24304 raft_consensus.cc:3060] T cdbd78acb4cf46298e0de8ba0d130e19 P 90f3da57796b488c9087a2f523adf045 [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:22:34.426508 16817 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49906 (request call id 1570) took 1011 ms. Trace:
W20260709 13:22:34.392331 16226 rpcz_store.cc:269] 0709 13:22:32.442003 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:22:32.443711 (+  1708us) service_pool.cc:224] Handling call
0709 13:22:34.391764 (+1948053us) spinlock_profiling.cc:243] Waited 34.7 ms on lock 0x7b54000159d4. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:22:34.391868 (+   104us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":79704832}
I20260709 13:22:34.428195 16217 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33050 (request call id 1404) took 1285 ms. Trace:
I20260709 13:22:34.429032 16824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e9b2bf7011f4ae688a79d7bb5a447b4" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:22:34.437932 16817 rpcz_store.cc:276] 0709 13:22:33.415090 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:22:33.415498 (+   408us) service_pool.cc:224] Handling call
0709 13:22:33.415774 (+   276us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:22:33.574476 (+158702us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:22:33.574539 (+    63us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:22:33.574599 (+    60us) raft_consensus.cc:1645] Marking committed up to 16
0709 13:22:33.574646 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:22:34.426283 (+851637us) spinlock_profiling.cc:243] Waited 158 ms on lock 0x7b540051b154. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:22:34.426327 (+    44us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:22:34.426453 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":364359424}
I20260709 13:22:34.439116 16217 rpcz_store.cc:276] 0709 13:22:33.143086 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:22:34.264027 (+1120941us) service_pool.cc:224] Handling call
0709 13:22:34.374062 (+110035us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:22:34.374739 (+   677us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:22:34.427939 (+ 53200us) spinlock_profiling.cc:243] Waited 110 ms on lock 0x7b54001428d0. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5e8ad 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:22:34.428119 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":252348288}
I20260709 13:22:34.413358 24190 raft_consensus.cc:740] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 647a343a542d4a139fe106c19095129b [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:22:34.434343 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 26 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:22:34.447073 16225 raft_consensus.cc:2376] T ecc1e363230043fe9fef390f0b979f20 P 1ca2725d290e47d68b1a4b609c9816ac [term 26 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 26. Re-sending same reply.
I20260709 13:22:34.435530 16291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fffab26427e24d398e2695aad40ca134" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:34.436908 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68a6838d82ef4cac83507f906139d90f" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:22:34.448489 16297 raft_consensus.cc:3060] T 68a6838d82ef4cac83507f906139d90f P 90f3da57796b488c9087a2f523adf045 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:22:34.438745 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806d7690c2b644959152074bb2cab0a7" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:34.449925 16226 raft_consensus.cc:2393] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 22: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
I20260709 13:22:34.394723 16224 rpcz_store.cc:276] 0709 13:22:32.453824 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:22:32.477236 (+ 23412us) service_pool.cc:224] Handling call
0709 13:22:34.391928 (+1914692us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:22:34.393014 (+  1086us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:22:34.393977 (+   963us) spinlock_profiling.cc:243] Waited 1.91 s on lock 0x7b54000159d0. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5e8ad 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:22:34.394159 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4402767616}
I20260709 13:22:34.430949 16219 raft_consensus.cc:2468] T 19baa7bc367844c288bca8f81439c413 P 1ca2725d290e47d68b1a4b609c9816ac [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 25.
I20260709 13:22:34.442427 16292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8638ee363273426db1393dd81e5c7b72" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:34.451833 16292 raft_consensus.cc:2468] T 8638ee363273426db1393dd81e5c7b72 P 90f3da57796b488c9087a2f523adf045 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 23.
I20260709 13:22:34.452368 16224 raft_consensus.cc:1275] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
I20260709 13:22:34.445832 16825 raft_consensus.cc:1275] T 19baa7bc367844c288bca8f81439c413 P cb82cc7bd6db46d980c75237a1402860 [term 25 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 23 index: 13. Preceding OpId from leader: term: 25 index: 14. (index mismatch)
I20260709 13:22:34.390694 16221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 27 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:34.453876 16221 raft_consensus.cc:2468] T ecc1e363230043fe9fef390f0b979f20 P 1ca2725d290e47d68b1a4b609c9816ac [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 26.
I20260709 13:22:34.455549 16292 raft_consensus.cc:1275] T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045 [term 24 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 24 index: 13. (index mismatch)
I20260709 13:22:34.456712 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c74490c40243a88e62d1760b5cd814" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:34.457091 16292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a959dbecc2648eb85d1cc550f5a3245" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:22:34.457756 16292 raft_consensus.cc:3060] T 5a959dbecc2648eb85d1cc550f5a3245 P 90f3da57796b488c9087a2f523adf045 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:22:34.459962 24190 consensus_queue.cc:260] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 21.14, Last appended by leader: 14, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:34.461313 24190 raft_consensus.cc:3060] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 647a343a542d4a139fe106c19095129b [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:22:34.461490 16221 raft_consensus.cc:1275] T 1df93779209748b9860f6d893dd3de53 P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
I20260709 13:22:34.463447 24180 raft_consensus.cc:2749] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:22:34.465621 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746aabf856ab401c9ef5e859371e7b67" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:34.473526 16224 raft_consensus.cc:1275] T b6f07842d3aa47858df5665874c86f3a P 1ca2725d290e47d68b1a4b609c9816ac [term 22 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 20 index: 13. Preceding OpId from leader: term: 22 index: 14. (index mismatch)
I20260709 13:22:34.474886 16818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af57996a8be84d398f0f023e8d08ef77" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
W20260709 13:22:34.479460 16115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 159019us
I20260709 13:22:34.483052 16221 raft_consensus.cc:1275] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [term 23 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 22 index: 17. Preceding OpId from leader: term: 21 index: 17. (term mismatch)
I20260709 13:22:34.483984 16221 pending_rounds.cc:85] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac: Aborting all ops after (but not including) 16
I20260709 13:22:34.483987 16294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9944d5e4b8c4109af9955c4f08c9095" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 25 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:34.484635 16221 pending_rounds.cc:107] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac: Aborting uncommitted NO_OP operation due to leader change: 22.17
I20260709 13:22:34.486106 16217 raft_consensus.cc:1275] T 19baa7bc367844c288bca8f81439c413 P 1ca2725d290e47d68b1a4b609c9816ac [term 25 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 23 index: 13. Preceding OpId from leader: term: 25 index: 14. (index mismatch)
I20260709 13:22:34.486681 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f2da98864b4425bf7c0055951fdd4b" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:34.487290 16221 raft_consensus.cc:2889] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [term 23 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:22:34.487723 16293 raft_consensus.cc:2393] T 84f2da98864b4425bf7c0055951fdd4b P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 20: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
I20260709 13:22:34.491888 16297 raft_consensus.cc:2468] T 68a6838d82ef4cac83507f906139d90f P 90f3da57796b488c9087a2f523adf045 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 23.
I20260709 13:22:34.493662 16224 raft_consensus.cc:1275] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [term 25 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 21 index: 15. Preceding OpId from leader: term: 25 index: 17. (index mismatch)
I20260709 13:22:34.495379 16292 raft_consensus.cc:2468] T 5a959dbecc2648eb85d1cc550f5a3245 P 90f3da57796b488c9087a2f523adf045 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 22.
I20260709 13:22:34.500029 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806d7690c2b644959152074bb2cab0a7" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:22:34.502488 16293 raft_consensus.cc:1240] T 5a959dbecc2648eb85d1cc550f5a3245 P 90f3da57796b488c9087a2f523adf045 [term 22 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 21. Current term is 22. Ops: [21.13-21.13]
I20260709 13:22:34.511029 16297 raft_consensus.cc:1275] T 8158bbd17b994d2eb96f560bd0f1b88d P 90f3da57796b488c9087a2f523adf045 [term 21 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
I20260709 13:22:34.513191 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0eab25ce4c64e018a041beeda2867db" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 24 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:34.513533 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6f07842d3aa47858df5665874c86f3a" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 23 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:34.514153 16293 raft_consensus.cc:2468] T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 22.
I20260709 13:22:34.514161 16294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d4ab870a05347868956aa8b841b2981" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:22:34.521081 16294 pending_rounds.cc:85] T a378d7f796f94615b3b347119a9880fd P 90f3da57796b488c9087a2f523adf045: Aborting all ops after (but not including) 14
I20260709 13:22:34.521517 16294 pending_rounds.cc:107] T a378d7f796f94615b3b347119a9880fd P 90f3da57796b488c9087a2f523adf045: Aborting uncommitted NO_OP operation due to leader change: 22.15
I20260709 13:22:34.521816 16294 raft_consensus.cc:2889] T a378d7f796f94615b3b347119a9880fd P 90f3da57796b488c9087a2f523adf045 [term 23 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:22:34.528052 16190 leader_election.cc:304] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: cb82cc7bd6db46d980c75237a1402860
W20260709 13:22:34.529584 16191 leader_election.cc:336] T ee9ed67006144797b1de6ea8c2013a68 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:22:34.536605 16190 leader_election.cc:304] T 08c74490c40243a88e62d1760b5cd814 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:22:34.538206 16190 leader_election.cc:304] T fffab26427e24d398e2695aad40ca134 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:22:34.541905 24120 raft_consensus.cc:2749] T 08c74490c40243a88e62d1760b5cd814 P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:22:34.542874 24120 raft_consensus.cc:2749] T fffab26427e24d398e2695aad40ca134 P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:22:34.547590 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 63517.4us
I20260709 13:22:34.547901 16218 raft_consensus.cc:2410] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 for term 28 because replica has last-logged OpId of term: 27 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 26 index: 16.
I20260709 13:22:34.511053 24231 consensus_queue.cc:1048] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.030s
I20260709 13:22:34.549614 16190 leader_election.cc:304] T 68a6838d82ef4cac83507f906139d90f P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 647a343a542d4a139fe106c19095129b
W20260709 13:22:34.550057 16790 consensus_peers.cc:597] T 66f75124215f4ca6bef831e174a792d8 P cb82cc7bd6db46d980c75237a1402860 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:22:34.496877 24141 consensus_queue.cc:1048] T 8a878999c0024d2abe517484180290fc P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.030s
W20260709 13:22:34.553949 16790 leader_election.cc:336] T 806d7690c2b644959152074bb2cab0a7 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:22:34.553061 23958 consensus_queue.cc:1048] T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.006s
I20260709 13:22:34.532240 24180 raft_consensus.cc:2804] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:22:34.559053 24180 raft_consensus.cc:493] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:34.559587 24180 raft_consensus.cc:3060] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:22:34.559353 24321 consensus_queue.cc:1048] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:22:34.561492 23994 consensus_queue.cc:1059] T 5a959dbecc2648eb85d1cc550f5a3245 P 647a343a542d4a139fe106c19095129b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: INVALID_TERM, Last received: 20.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:22:34.561715 24362 consensus_queue.cc:1048] T 8158bbd17b994d2eb96f560bd0f1b88d P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:22:34.562460 23994 raft_consensus.cc:3055] T 5a959dbecc2648eb85d1cc550f5a3245 P 647a343a542d4a139fe106c19095129b [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:22:34.562844 23994 raft_consensus.cc:740] T 5a959dbecc2648eb85d1cc550f5a3245 P 647a343a542d4a139fe106c19095129b [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:22:34.564172 16226 raft_consensus.cc:1240] T 8a878999c0024d2abe517484180290fc P 1ca2725d290e47d68b1a4b609c9816ac [term 20 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 19. Current term is 20. Ops: [19.11-19.11]
I20260709 13:22:34.563804 23994 consensus_queue.cc:260] T 5a959dbecc2648eb85d1cc550f5a3245 P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.13, Last appended by leader: 13, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:34.565306 23994 raft_consensus.cc:3060] T 5a959dbecc2648eb85d1cc550f5a3245 P 647a343a542d4a139fe106c19095129b [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:22:34.523334 23719 consensus_queue.cc:260] T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.16, Last appended by leader: 16, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:34.566237 24242 raft_consensus.cc:515] T 9c61a39fd4f84424b129c1a6f9ba404d P 647a343a542d4a139fe106c19095129b [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:22:34.573267 16115 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:22:34.578150 16263 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:22:34.516973 23924 raft_consensus.cc:515] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:34.591905 16190 leader_election.cc:304] T 5a959dbecc2648eb85d1cc550f5a3245 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 647a343a542d4a139fe106c19095129b
W20260709 13:22:34.603361 16264 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:22:34.604386 15046 thread.cc:527] Waited for 6000ms trying to join with heartbeat (tid 16176)
I20260709 13:22:34.612090 16790 leader_election.cc:304] T 806d7690c2b644959152074bb2cab0a7 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
W20260709 13:22:34.621631 16263 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:22:34.622982 16300 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48674 (request call id 3734) took 1161 ms. Trace:
W20260709 13:22:34.628255 16116 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:22:34.628650 16191 leader_election.cc:336] T 84f2da98864b4425bf7c0055951fdd4b P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:22:34.623561 16300 rpcz_store.cc:276] 0709 13:22:33.453079 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:22:33.465704 (+ 12625us) service_pool.cc:224] Handling call
0709 13:22:34.614235 (+1148531us) spinlock_profiling.cc:243] Waited 449 ms on lock 0x7b540030bbd4. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:22:34.614407 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1032840448}
I20260709 13:22:34.631134 16790 leader_election.cc:304] T 74ca8140f35c4ae29e8a664f6bdfc829 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 90f3da57796b488c9087a2f523adf045
I20260709 13:22:34.607921 24219 raft_consensus.cc:2804] T 68a6838d82ef4cac83507f906139d90f P 1ca2725d290e47d68b1a4b609c9816ac [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:22:34.617878 23719 raft_consensus.cc:3060] T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:22:34.624544 24125 raft_consensus.cc:2749] T 806d7690c2b644959152074bb2cab0a7 P cb82cc7bd6db46d980c75237a1402860 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:22:34.651481 16191 leader_election.cc:336] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:22:34.644709 16116 leader_election.cc:304] T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
W20260709 13:22:34.655086 16117 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:22:34.662837 16116 leader_election.cc:304] T fffab26427e24d398e2695aad40ca134 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
W20260709 13:22:34.661528 16191 leader_election.cc:336] T b6f07842d3aa47858df5665874c86f3a P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:22:34.634756 24215 raft_consensus.cc:515] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:34.682020 24307 raft_consensus.cc:2804] T 74ca8140f35c4ae29e8a664f6bdfc829 P cb82cc7bd6db46d980c75237a1402860 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 13:22:34.683233 24215 leader_election.cc:290] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 21 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:34.683408 24307 raft_consensus.cc:493] T 74ca8140f35c4ae29e8a664f6bdfc829 P cb82cc7bd6db46d980c75237a1402860 [term 25 FOLLOWER]: Starting leader election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:22:34.684105 24307 raft_consensus.cc:3060] T 74ca8140f35c4ae29e8a664f6bdfc829 P cb82cc7bd6db46d980c75237a1402860 [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:22:34.696524 16824 raft_consensus.cc:1240] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860 [term 21 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 20. Current term is 21. Ops: []
W20260709 13:22:34.706770 24372 log.cc:927] Time spent T 8638ee363273426db1393dd81e5c7b72 P cb82cc7bd6db46d980c75237a1402860: Append to log took a long time: real 0.077s	user 0.001s	sys 0.000s
I20260709 13:22:34.706594 16296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31969392ccdb43b491911a319e8bc7e1" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:22:34.708293 24088 raft_consensus.cc:493] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:34.708926 24088 raft_consensus.cc:515] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:34.710739 24227 raft_consensus.cc:515] T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:34.665729 24184 raft_consensus.cc:2749] T fffab26427e24d398e2695aad40ca134 P 647a343a542d4a139fe106c19095129b [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:22:34.715785 16825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b7d1ab519044eda2d5e04a07242f25" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 23 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:22:34.716602 16825 raft_consensus.cc:3060] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 [term 21 FOLLOWER]: Advancing to term 23
I20260709 13:22:34.666041 24362 raft_consensus.cc:3060] T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:22:34.732740 16825 raft_consensus.cc:2468] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 23.
I20260709 13:22:34.735841 24227 leader_election.cc:290] T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 23 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:34.736467 16117 leader_election.cc:304] T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:22:34.743820 24088 leader_election.cc:290] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
W20260709 13:22:34.738385 24369 log.cc:927] Time spent T c2f5f89186204242bf377f7c42b2e2da P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 0.166s	user 0.000s	sys 0.001s
W20260709 13:22:34.747655 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 116606us
I20260709 13:22:34.750581 16825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5d59407eb545b895d8d922e0dd69fd" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:34.749938 16790 leader_election.cc:304] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:22:34.754798 16790 leader_election.cc:304] T d2ab0ac8347641e6955e9bce7d90c1a2 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:22:34.756356 24341 raft_consensus.cc:2804] T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:22:34.757570 24381 raft_consensus.cc:2804] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:22:34.762106 24381 raft_consensus.cc:493] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:34.762440 24381 raft_consensus.cc:3060] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:22:34.768834 16790 leader_election.cc:304] T 8c11074579b046d99e585c2818e18a2c P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:22:34.775184 16218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68a6838d82ef4cac83507f906139d90f" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
W20260709 13:22:34.775949 16265 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:22:34.777041 24341 raft_consensus.cc:697] T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b [term 23 LEADER]: Becoming Leader. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:22:34.778330 16219 raft_consensus.cc:1275] T e4dc6b0541ca431ebc757beac00521f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 24 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 22 index: 12. Preceding OpId from leader: term: 24 index: 13. (index mismatch)
W20260709 13:22:34.785981 16265 leader_election.cc:336] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 25 election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:22:34.788821 24368 log.cc:927] Time spent T c4f65cdf00b748bb963ee6e81ef43b79 P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.063s	user 0.000s	sys 0.000s
I20260709 13:22:34.789834 24215 raft_consensus.cc:2749] T 8c11074579b046d99e585c2818e18a2c P cb82cc7bd6db46d980c75237a1402860 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:22:34.789829 16790 leader_election.cc:304] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 26 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:22:34.790635 24242 leader_election.cc:290] T 9c61a39fd4f84424b129c1a6f9ba404d P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 25 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:34.791766 24379 raft_consensus.cc:2749] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 [term 26 FOLLOWER]: Leader election lost for term 26. Reason: could not achieve majority
I20260709 13:22:34.792495 24379 consensus_queue.cc:1048] T 66f75124215f4ca6bef831e174a792d8 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 13:22:34.793620 16790 leader_election.cc:304] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:22:34.795084 24113 raft_consensus.cc:2804] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 13:22:34.795490 24113 raft_consensus.cc:493] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860 [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:34.795826 24113 raft_consensus.cc:3060] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860 [term 26 FOLLOWER]: Advancing to term 27
I20260709 13:22:34.797799 16824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee9ed67006144797b1de6ea8c2013a68" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
W20260709 13:22:34.803916 16265 leader_election.cc:336] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:22:34.804394 16265 leader_election.cc:304] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
I20260709 13:22:34.806914 16825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806d7690c2b644959152074bb2cab0a7" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:22:34.809209 16222 raft_consensus.cc:1240] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac [term 25 FOLLOWER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 24. Current term is 25. Ops: []
I20260709 13:22:34.811332 24271 raft_consensus.cc:2749] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:22:34.814675 16818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9944d5e4b8c4109af9955c4f08c9095" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 25 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
W20260709 13:22:34.814445 16265 leader_election.cc:336] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:22:34.836513 16265 leader_election.cc:304] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
I20260709 13:22:34.814462 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c61a39fd4f84424b129c1a6f9ba404d" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:22:34.778036 24341 consensus_queue.cc:237] T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 21.17, Last appended by leader: 17, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:34.767148 24362 raft_consensus.cc:2749] T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 13:22:34.838546 16265 leader_election.cc:336] T ecc1e363230043fe9fef390f0b979f20 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 26 election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:22:34.838861 16299 raft_consensus.cc:3060] T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:22:34.839231 24271 raft_consensus.cc:2749] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
W20260709 13:22:34.846268 16265 leader_election.cc:336] T 8d4ab870a05347868956aa8b841b2981 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:22:34.852656 16265 leader_election.cc:304] T 8d4ab870a05347868956aa8b841b2981 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
W20260709 13:22:34.856289 16265 leader_election.cc:336] T 1df93779209748b9860f6d893dd3de53 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:22:34.856730 16265 leader_election.cc:304] T 1df93779209748b9860f6d893dd3de53 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
W20260709 13:22:34.857723 16265 leader_election.cc:336] T 23f82224f4564fea93572e8206bad1fe P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:22:34.858088 16265 leader_election.cc:304] T 23f82224f4564fea93572e8206bad1fe P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
I20260709 13:22:34.815666 23924 leader_election.cc:290] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 25 election: Requested vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:34.801843 24378 raft_consensus.cc:2749] T d2ab0ac8347641e6955e9bce7d90c1a2 P cb82cc7bd6db46d980c75237a1402860 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:22:34.873267 24222 consensus_queue.cc:1059] T ada5a004ba7c46819051e3ccec71ffdb P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: INVALID_TERM, Last received: 24.17, Next index: 18, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 13:22:34.860000 24307 raft_consensus.cc:515] T 74ca8140f35c4ae29e8a664f6bdfc829 P cb82cc7bd6db46d980c75237a1402860 [term 26 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:22:34.876623 24377 log.cc:927] Time spent T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.108s	user 0.001s	sys 0.000s
I20260709 13:22:34.877554 24307 leader_election.cc:290] T 74ca8140f35c4ae29e8a664f6bdfc829 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 26 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:34.855955 24096 raft_consensus.cc:515] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [term 29 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:22:34.859048 16265 leader_election.cc:336] T e4dc6b0541ca431ebc757beac00521f1 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:22:34.880641 16265 leader_election.cc:336] T 68a6838d82ef4cac83507f906139d90f P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:22:34.869460 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 199619us
I20260709 13:22:34.881415 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74ca8140f35c4ae29e8a664f6bdfc829" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:22:34.872387 16299 raft_consensus.cc:2468] T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 25.
I20260709 13:22:34.861058 23865 raft_consensus.cc:3060] T 8d4ab870a05347868956aa8b841b2981 P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:22:34.786398 24180 raft_consensus.cc:515] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:34.815486 24125 consensus_queue.cc:1048] T 8638ee363273426db1393dd81e5c7b72 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 16, Time since last communication: 0.046s
I20260709 13:22:34.861565 24275 raft_consensus.cc:2749] T 23f82224f4564fea93572e8206bad1fe P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:22:34.889858 24223 consensus_queue.cc:1048] T 8158bbd17b994d2eb96f560bd0f1b88d P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:22:34.892341 24378 raft_consensus.cc:3055] T ada5a004ba7c46819051e3ccec71ffdb P cb82cc7bd6db46d980c75237a1402860 [term 24 LEADER]: Stepping down as leader of term 24
I20260709 13:22:34.892624 24180 leader_election.cc:290] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 24 election: Requested vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:34.893991 24176 consensus_queue.cc:1059] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true }, Status: INVALID_TERM, Last received: 21.15, Next index: 16, Last known committed idx: 14, Time since last communication: 0.000s
W20260709 13:22:34.897521 16265 leader_election.cc:336] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:22:34.899315 16265 leader_election.cc:336] T 31969392ccdb43b491911a319e8bc7e1 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:22:34.898850 16115 leader_election.cc:304] T 9c61a39fd4f84424b129c1a6f9ba404d P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:22:34.859867 24231 consensus_queue.cc:1059] T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: INVALID_TERM, Last received: 20.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.027s
I20260709 13:22:34.899740 16265 leader_election.cc:304] T 31969392ccdb43b491911a319e8bc7e1 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
I20260709 13:22:34.912811 16818 raft_consensus.cc:1240] T 74ca8140f35c4ae29e8a664f6bdfc829 P cb82cc7bd6db46d980c75237a1402860 [term 26 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 25. Current term is 26. Ops: []
W20260709 13:22:34.913991 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 76863.8us
W20260709 13:22:34.912825 16265 leader_election.cc:336] T c0eab25ce4c64e018a041beeda2867db P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:22:34.915062 16790 leader_election.cc:336] T ee9ed67006144797b1de6ea8c2013a68 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:22:34.914997 16265 leader_election.cc:304] T c0eab25ce4c64e018a041beeda2867db P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:22:34.915591 16790 leader_election.cc:304] T ee9ed67006144797b1de6ea8c2013a68 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
W20260709 13:22:34.916559 16265 leader_election.cc:336] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:22:34.916762 16790 leader_election.cc:336] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:22:34.892781 24378 raft_consensus.cc:740] T ada5a004ba7c46819051e3ccec71ffdb P cb82cc7bd6db46d980c75237a1402860 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:22:34.917053 16265 leader_election.cc:304] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
W20260709 13:22:34.917796 16790 leader_election.cc:336] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:22:34.920804 16790 leader_election.cc:304] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
W20260709 13:22:34.923964 16265 leader_election.cc:336] T 3884f864a96d428785895e5463e5728f P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:22:34.924928 16265 leader_election.cc:304] T 3884f864a96d428785895e5463e5728f P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
I20260709 13:22:34.900300 23994 consensus_queue.cc:1048] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.028s
I20260709 13:22:34.911698 24298 raft_consensus.cc:2749] T 1df93779209748b9860f6d893dd3de53 P 90f3da57796b488c9087a2f523adf045 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:22:34.917029 24321 raft_consensus.cc:2749] T 31969392ccdb43b491911a319e8bc7e1 P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:22:34.914224 24275 raft_consensus.cc:493] T 9e65676b21b748728ad39668cfe722ec P 90f3da57796b488c9087a2f523adf045 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:22:34.919672 24313 raft_consensus.cc:2749] T c0eab25ce4c64e018a041beeda2867db P 90f3da57796b488c9087a2f523adf045 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:22:34.933853 24176 raft_consensus.cc:2804] T 9c61a39fd4f84424b129c1a6f9ba404d P 647a343a542d4a139fe106c19095129b [term 25 FOLLOWER]: Leader election won for term 25
I20260709 13:22:34.929332 24314 raft_consensus.cc:2749] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:22:34.928815 24200 consensus_queue.cc:1048] T 1df93779209748b9860f6d893dd3de53 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.037s
I20260709 13:22:34.966399 24096 leader_election.cc:290] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 29 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:34.968098 24383 raft_consensus.cc:3055] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:22:34.940465 24390 raft_consensus.cc:2749] T ee9ed67006144797b1de6ea8c2013a68 P cb82cc7bd6db46d980c75237a1402860 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:22:34.929019 24190 consensus_queue.cc:1048] T b6f07842d3aa47858df5665874c86f3a P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.043s
I20260709 13:22:34.972361 24275 raft_consensus.cc:515] T 9e65676b21b748728ad39668cfe722ec P 90f3da57796b488c9087a2f523adf045 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:34.973594 24321 raft_consensus.cc:2749] T 3884f864a96d428785895e5463e5728f P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:22:34.942113 24385 log.cc:927] Time spent T ada5a004ba7c46819051e3ccec71ffdb P cb82cc7bd6db46d980c75237a1402860: Append to log took a long time: real 0.058s	user 0.000s	sys 0.001s
I20260709 13:22:34.986088 24307 raft_consensus.cc:2749] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 13:22:34.978072 16294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66ae176961b74813a58bc15c9c4c6f2a" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 29 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:22:34.943754 24381 raft_consensus.cc:515] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:34.939432 24125 raft_consensus.cc:493] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:22:34.994761 24383 raft_consensus.cc:740] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:22:35.012503 24125 raft_consensus.cc:515] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:35.013072 24383 consensus_queue.cc:260] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.15, Last appended by leader: 15, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:35.014624 24125 leader_election.cc:290] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
W20260709 13:22:34.990147 16263 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:22:34.990760 16264 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:22:35.004405 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 412613us
W20260709 13:22:35.015791 16263 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:22:35.016145 16190 leader_election.cc:304] T 84f2da98864b4425bf7c0055951fdd4b P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:22:35.009485 24219 raft_consensus.cc:697] T 68a6838d82ef4cac83507f906139d90f P 1ca2725d290e47d68b1a4b609c9816ac [term 23 LEADER]: Becoming Leader. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:22:35.018139 24219 consensus_queue.cc:237] T 68a6838d82ef4cac83507f906139d90f P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 21.12, Last appended by leader: 12, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:35.018616 16192 leader_election.cc:304] T ee9ed67006144797b1de6ea8c2013a68 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
I20260709 13:22:34.998636 24113 raft_consensus.cc:515] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860 [term 27 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:22:34.975205 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 51297.1us
W20260709 13:22:35.022007 16265 leader_election.cc:336] T 57a744f5aaf944678a63469cbf514015 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:22:35.022451 16265 leader_election.cc:304] T 57a744f5aaf944678a63469cbf514015 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
I20260709 13:22:35.019661 16295 raft_consensus.cc:1275] T 02838c86531d4f98bf58d1c22b8baf77 P 90f3da57796b488c9087a2f523adf045 [term 24 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 22 index: 13. Preceding OpId from leader: term: 24 index: 14. (index mismatch)
W20260709 13:22:35.023702 16265 leader_election.cc:336] T e5ad4eb281564936a59f41e387a47156 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:22:35.024289 16265 leader_election.cc:304] T e5ad4eb281564936a59f41e387a47156 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
W20260709 13:22:35.005568 16266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 75429.8us
I20260709 13:22:35.010231 16294 raft_consensus.cc:3060] T 66ae176961b74813a58bc15c9c4c6f2a P 90f3da57796b488c9087a2f523adf045 [term 27 FOLLOWER]: Advancing to term 29
I20260709 13:22:34.985448 16225 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57600 (request call id 2345) took 1255 ms. Trace:
I20260709 13:22:35.019872 16192 leader_election.cc:304] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
I20260709 13:22:35.014803 24275 leader_election.cc:290] T 9e65676b21b748728ad39668cfe722ec P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:35.021196 24180 raft_consensus.cc:2749] T ee9ed67006144797b1de6ea8c2013a68 P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:22:34.943809 24304 consensus_queue.cc:1059] T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: INVALID_TERM, Last received: 25.17, Next index: 18, Last known committed idx: 15, Time since last communication: 0.057s
W20260709 13:22:35.030012 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 112260us
I20260709 13:22:35.027134 16225 rpcz_store.cc:276] 0709 13:22:33.730133 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:22:34.448361 (+718228us) service_pool.cc:224] Handling call
0709 13:22:34.448700 (+   339us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:22:34.449041 (+   341us) raft_consensus.cc:1534] Early marking committed up to index 14
0709 13:22:34.478500 (+ 29459us) log.cc:844] Serialized 14 byte log entry
0709 13:22:34.557125 (+ 78625us) log.cc:844] Serialized 14 byte log entry
0709 13:22:34.984886 (+427761us) spinlock_profiling.cc:243] Waited 2.06 ms on lock 0x7b18004122a0. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b693ade1 00007f69b69475b1 00007f69ad9ecc0a 00007f69b712c219 00007f69ada74a40 00007f69ada6f4e4 00007f69ada6f455 00007f69b0b57060 00007f69b0b69716 00007f69b0b7a287 00007f69b0b7a208
0709 13:22:34.985027 (+   141us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:22:34.985093 (+    66us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:22:34.985156 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:22:34.985245 (+    89us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:22:34.985387 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":471834,"prepare.queue_time_us":30819,"prepare.run_cpu_time_us":88,"prepare.run_wall_time_us":1797,"raft.queue_time_us":475732,"raft.run_cpu_time_us":300,"raft.run_wall_time_us":9430,"spinlock_wait_cycles":4726528,"thread_start_us":48029,"threads_started":1,"wal-append.queue_time_us":93322}
W20260709 13:22:35.050264 16790 leader_election.cc:336] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:22:35.050803 16265 leader_election.cc:336] T c4f65cdf00b748bb963ee6e81ef43b79 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:22:35.050895 16790 leader_election.cc:304] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
W20260709 13:22:35.051661 16265 leader_election.cc:336] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:22:35.052156 16265 leader_election.cc:304] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
I20260709 13:22:35.054140 24280 raft_consensus.cc:2749] T e5ad4eb281564936a59f41e387a47156 P 90f3da57796b488c9087a2f523adf045 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:22:35.055123 24313 raft_consensus.cc:2749] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 90f3da57796b488c9087a2f523adf045 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:22:35.007579 24120 raft_consensus.cc:2804] T 5a959dbecc2648eb85d1cc550f5a3245 P 1ca2725d290e47d68b1a4b609c9816ac [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:22:35.048720 16818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e65676b21b748728ad39668cfe722ec" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:35.020655 23865 raft_consensus.cc:2749] T 8d4ab870a05347868956aa8b841b2981 P 90f3da57796b488c9087a2f523adf045 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:22:35.066427 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 50380.7us
I20260709 13:22:35.067605 16190 leader_election.cc:304] T b6f07842d3aa47858df5665874c86f3a P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 647a343a542d4a139fe106c19095129b
W20260709 13:22:35.059926 16265 leader_election.cc:336] T e4dc6b0541ca431ebc757beac00521f1 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 24 election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:22:35.014827 24383 raft_consensus.cc:3060] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:22:34.994313 24378 consensus_queue.cc:260] T ada5a004ba7c46819051e3ccec71ffdb P cb82cc7bd6db46d980c75237a1402860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 24.17, Last appended by leader: 17, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:35.048028 24321 raft_consensus.cc:2749] T 57a744f5aaf944678a63469cbf514015 P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:22:35.064772 24366 log.cc:927] Time spent T ada5a004ba7c46819051e3ccec71ffdb P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
I20260709 13:22:35.075476 16265 leader_election.cc:304] T c4f65cdf00b748bb963ee6e81ef43b79 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
W20260709 13:22:35.072399 16790 leader_election.cc:336] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:22:35.056757 24176 raft_consensus.cc:697] T 9c61a39fd4f84424b129c1a6f9ba404d P 647a343a542d4a139fe106c19095129b [term 25 LEADER]: Becoming Leader. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:22:35.081176 16294 raft_consensus.cc:2468] T 66ae176961b74813a58bc15c9c4c6f2a P 90f3da57796b488c9087a2f523adf045 [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 29.
I20260709 13:22:35.067183 24304 raft_consensus.cc:493] T 18ad7e3af5504571a69d10c00750a01c P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
W20260709 13:22:35.081122 16192 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 61347.7us
I20260709 13:22:35.086205 24217 raft_consensus.cc:3060] T 84f2da98864b4425bf7c0055951fdd4b P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:22:35.114413 16790 leader_election.cc:336] T 19baa7bc367844c288bca8f81439c413 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:22:35.124400 16192 leader_election.cc:304] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860
I20260709 13:22:35.124194 16790 leader_election.cc:304] T 19baa7bc367844c288bca8f81439c413 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
W20260709 13:22:35.127496 16790 leader_election.cc:336] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:22:35.128849 16790 leader_election.cc:336] T 9e65676b21b748728ad39668cfe722ec P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:22:35.129252 16790 leader_election.cc:304] T 9e65676b21b748728ad39668cfe722ec P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:22:35.063323 24307 raft_consensus.cc:493] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
W20260709 13:22:35.148293 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 72895.9us
I20260709 13:22:35.080538 24222 raft_consensus.cc:2749] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 [term 23 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:22:35.186734 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 57918.1us
W20260709 13:22:35.181258 16789 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:22:35.187829 16790 leader_election.cc:336] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:22:35.182498 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 96305.7us
I20260709 13:22:35.177214 16040 catalog_manager.cc:5697] T 434bff3082d141838068f323bdb0d1ed P cb82cc7bd6db46d980c75237a1402860 reported cstate change: term changed from 15 to 20. New cstate: current_term: 20 leader_uuid: "90f3da57796b488c9087a2f523adf045" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:22:35.188476 16790 leader_election.cc:304] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:22:35.181252 16265 leader_election.cc:304] T 8158bbd17b994d2eb96f560bd0f1b88d P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:22:35.189812 24113 leader_election.cc:290] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 27 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:35.111413 24180 raft_consensus.cc:2749] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
W20260709 13:22:35.221740 16790 consensus_peers.cc:597] T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:22:35.235754 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 54573.9us
I20260709 13:22:35.236594 16265 leader_election.cc:304] T ecc1e363230043fe9fef390f0b979f20 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 647a343a542d4a139fe106c19095129b
I20260709 13:22:35.240528 16291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 27 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:22:35.241226 16291 raft_consensus.cc:3060] T ecc1e363230043fe9fef390f0b979f20 P 90f3da57796b488c9087a2f523adf045 [term 26 FOLLOWER]: Advancing to term 27
I20260709 13:22:35.241606 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74ca8140f35c4ae29e8a664f6bdfc829" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:22:35.242471 16225 raft_consensus.cc:3060] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:22:35.243752 16219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66ae176961b74813a58bc15c9c4c6f2a" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 29 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:22:35.244585 16219 raft_consensus.cc:3060] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [term 28 FOLLOWER]: Advancing to term 29
I20260709 13:22:35.209345 24389 raft_consensus.cc:2749] T 9e65676b21b748728ad39668cfe722ec P cb82cc7bd6db46d980c75237a1402860 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:22:35.168844 24275 raft_consensus.cc:2749] T c4f65cdf00b748bb963ee6e81ef43b79 P 90f3da57796b488c9087a2f523adf045 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 13:22:35.248740 16790 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:22:35.231341 16789 leader_election.cc:304] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 29 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:22:35.220705 24390 raft_consensus.cc:2749] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:22:35.193527 24398 raft_consensus.cc:2749] T 19baa7bc367844c288bca8f81439c413 P cb82cc7bd6db46d980c75237a1402860 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:22:35.252820 16220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 27 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:22:35.253545 16223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2372981006d4b218215c28964dea623" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 28 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:35.262274 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af57996a8be84d398f0f023e8d08ef77" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
W20260709 13:22:35.257390 16192 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 133153us
I20260709 13:22:35.268760 16220 raft_consensus.cc:3060] T ecc1e363230043fe9fef390f0b979f20 P 1ca2725d290e47d68b1a4b609c9816ac [term 26 FOLLOWER]: Advancing to term 27
I20260709 13:22:35.272070 16818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada5a004ba7c46819051e3ccec71ffdb" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:22:35.272656 24413 raft_consensus.cc:2804] T 8158bbd17b994d2eb96f560bd0f1b88d P 90f3da57796b488c9087a2f523adf045 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:22:35.273111 24413 raft_consensus.cc:493] T 8158bbd17b994d2eb96f560bd0f1b88d P 90f3da57796b488c9087a2f523adf045 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:22:35.273634 24413 raft_consensus.cc:3060] T 8158bbd17b994d2eb96f560bd0f1b88d P 90f3da57796b488c9087a2f523adf045 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:22:35.274372 16816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8638ee363273426db1393dd81e5c7b72" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 24 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:22:35.256409 24399 consensus_queue.cc:1048] T 02838c86531d4f98bf58d1c22b8baf77 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.026s
I20260709 13:22:35.258903 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d4ab870a05347868956aa8b841b2981" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:22:35.276719 16224 raft_consensus.cc:3060] T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:22:35.267835 24176 consensus_queue.cc:237] T 9c61a39fd4f84424b129c1a6f9ba404d P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 23.13, Last appended by leader: 13, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:35.267967 16222 raft_consensus.cc:1240] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [term 24 FOLLOWER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 23. Current term is 24. Ops: [21.17-23.18]
I20260709 13:22:35.293576 16220 raft_consensus.cc:2468] T ecc1e363230043fe9fef390f0b979f20 P 1ca2725d290e47d68b1a4b609c9816ac [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 27.
I20260709 13:22:35.183449 24307 raft_consensus.cc:515] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:35.332922 24307 leader_election.cc:290] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:35.248152 24273 raft_consensus.cc:2749] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 13:22:35.121565 24125 consensus_queue.cc:1175] T ada5a004ba7c46819051e3ccec71ffdb P cb82cc7bd6db46d980c75237a1402860 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "90f3da57796b488c9087a2f523adf045" responder_term: 24 status { last_received { term: 24 index: 17 } last_committed_idx: 17 last_received_current_leader { term: 24 index: 17 } }
I20260709 13:22:35.241778 24219 raft_consensus.cc:2804] T b6f07842d3aa47858df5665874c86f3a P 1ca2725d290e47d68b1a4b609c9816ac [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:22:35.341610 24219 raft_consensus.cc:493] T b6f07842d3aa47858df5665874c86f3a P 1ca2725d290e47d68b1a4b609c9816ac [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:35.342036 24219 raft_consensus.cc:3060] T b6f07842d3aa47858df5665874c86f3a P 1ca2725d290e47d68b1a4b609c9816ac [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:22:35.171082 24304 raft_consensus.cc:515] T 18ad7e3af5504571a69d10c00750a01c P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } }
I20260709 13:22:35.339418 24217 raft_consensus.cc:2749] T 84f2da98864b4425bf7c0055951fdd4b P 1ca2725d290e47d68b1a4b609c9816ac [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:22:35.348101 24381 leader_election.cc:290] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 24 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:22:35.348754 24400 consensus_queue.cc:1048] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.028s
I20260709 13:22:35.296664 24389 raft_consensus.cc:2804] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [term 29 FOLLOWER]: Leader election won for term 29
I20260709 13:22:35.311503 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c61a39fd4f84424b129c1a6f9ba404d" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
W20260709 13:22:35.350358 24404 log.cc:927] Time spent T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.111s	user 0.001s	sys 0.000s
W20260709 13:22:35.311657 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 122903us
I20260709 13:22:35.312188 16221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0eab25ce4c64e018a041beeda2867db" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 24 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:35.353953 16221 raft_consensus.cc:1275] T 02838c86531d4f98bf58d1c22b8baf77 P 1ca2725d290e47d68b1a4b609c9816ac [term 24 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 22 index: 13. Preceding OpId from leader: term: 24 index: 14. (index mismatch)
I20260709 13:22:35.313452 16223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b7d1ab519044eda2d5e04a07242f25" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 23 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:22:35.355440 16223 raft_consensus.cc:3060] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac [term 22 FOLLOWER]: Advancing to term 23
W20260709 13:22:35.296891 24394 log.cc:927] Time spent T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 0.074s	user 0.000s	sys 0.000s
I20260709 13:22:35.317906 16224 raft_consensus.cc:2468] T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 22.
I20260709 13:22:35.320333 16291 raft_consensus.cc:2468] T ecc1e363230043fe9fef390f0b979f20 P 90f3da57796b488c9087a2f523adf045 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 27.
I20260709 13:22:35.320412 16225 raft_consensus.cc:2468] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 26.
I20260709 13:22:35.329324 16219 raft_consensus.cc:2468] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 29.
I20260709 13:22:35.374370 16291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada5a004ba7c46819051e3ccec71ffdb" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:22:35.322691 16220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5d59407eb545b895d8d922e0dd69fd" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:35.366658 16223 raft_consensus.cc:2468] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 23.
W20260709 13:22:35.391050 16115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 184684us
I20260709 13:22:35.394541 16291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8638ee363273426db1393dd81e5c7b72" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 24 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:22:35.394951 16296 raft_consensus.cc:1240] T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045 [term 25 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 24. Current term is 25. Ops: [22.12-24.13]
I20260709 13:22:35.395253 16291 raft_consensus.cc:3060] T 8638ee363273426db1393dd81e5c7b72 P 90f3da57796b488c9087a2f523adf045 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:22:35.402697 24389 raft_consensus.cc:697] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [term 29 LEADER]: Becoming Leader. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:22:35.403806 24389 consensus_queue.cc:237] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 27.17, Last appended by leader: 17, Current term: 29, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:35.413913 24378 raft_consensus.cc:3060] T ada5a004ba7c46819051e3ccec71ffdb P cb82cc7bd6db46d980c75237a1402860 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:22:35.348799 24304 leader_election.cc:290] T 18ad7e3af5504571a69d10c00750a01c P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:22:35.413347 16116 leader_election.cc:304] T af57996a8be84d398f0f023e8d08ef77 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860
W20260709 13:22:35.420209 16263 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:22:35.385147 23865 raft_consensus.cc:2764] T ecc1e363230043fe9fef390f0b979f20 P 90f3da57796b488c9087a2f523adf045 [term 27 FOLLOWER]: Leader election decision vote started in defunct term 26: won
I20260709 13:22:35.385174 24402 raft_consensus.cc:493] T d2ab0ac8347641e6955e9bce7d90c1a2 P 90f3da57796b488c9087a2f523adf045 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
W20260709 13:22:35.456944 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 43671.4us
I20260709 13:22:35.458727 16116 leader_election.cc:304] T c0eab25ce4c64e018a041beeda2867db P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:22:35.385068 24120 raft_consensus.cc:697] T 5a959dbecc2648eb85d1cc550f5a3245 P 1ca2725d290e47d68b1a4b609c9816ac [term 22 LEADER]: Becoming Leader. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:22:35.475838 16116 leader_election.cc:304] T 8d4ab870a05347868956aa8b841b2981 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 90f3da57796b488c9087a2f523adf045
W20260709 13:22:35.476946 16789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 245770us
I20260709 13:22:35.479987 16299 raft_consensus.cc:1275] T d16dfad0c3b540e5b4b86492ddf87208 P 90f3da57796b488c9087a2f523adf045 [term 23 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 21 index: 12. Preceding OpId from leader: term: 23 index: 13. (index mismatch)
I20260709 13:22:35.469600 16291 raft_consensus.cc:2468] T 8638ee363273426db1393dd81e5c7b72 P 90f3da57796b488c9087a2f523adf045 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 24.
I20260709 13:22:35.484536 24383 raft_consensus.cc:2749] T af57996a8be84d398f0f023e8d08ef77 P 647a343a542d4a139fe106c19095129b [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:22:35.506031 16190 leader_election.cc:304] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: cb82cc7bd6db46d980c75237a1402860
I20260709 13:22:35.507954 16116 leader_election.cc:304] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860
I20260709 13:22:35.466831 24402 raft_consensus.cc:515] T d2ab0ac8347641e6955e9bce7d90c1a2 P 90f3da57796b488c9087a2f523adf045 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:35.494200 24417 raft_consensus.cc:2749] T c0eab25ce4c64e018a041beeda2867db P 647a343a542d4a139fe106c19095129b [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:22:35.514462 24199 raft_consensus.cc:2749] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 13:22:35.516542 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 261072us
I20260709 13:22:35.502821 24396 consensus_queue.cc:1048] T 02838c86531d4f98bf58d1c22b8baf77 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 11, Time since last communication: 0.011s
I20260709 13:22:35.528832 24383 raft_consensus.cc:493] T 8cc959fb7f704fba9009c0595467d074 P 647a343a542d4a139fe106c19095129b [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:35.529373 24383 raft_consensus.cc:515] T 8cc959fb7f704fba9009c0595467d074 P 647a343a542d4a139fe106c19095129b [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:35.505422 24219 raft_consensus.cc:515] T b6f07842d3aa47858df5665874c86f3a P 1ca2725d290e47d68b1a4b609c9816ac [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:22:35.531769 16265 leader_election.cc:336] T 5a96373bb0e14dce8a266b2fc6c58507 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:22:35.532166 16819 raft_consensus.cc:1240] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860 [term 21 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 20. Current term is 21. Ops: []
I20260709 13:22:35.532269 16265 leader_election.cc:304] T 5a96373bb0e14dce8a266b2fc6c58507 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:22:35.533339 24298 consensus_queue.cc:1048] T e4dc6b0541ca431ebc757beac00521f1 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:22:35.535544 24298 raft_consensus.cc:493] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 90f3da57796b488c9087a2f523adf045 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:22:35.536232 24298 raft_consensus.cc:515] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 90f3da57796b488c9087a2f523adf045 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:22:35.540925 24410 log.cc:927] Time spent T 8a878999c0024d2abe517484180290fc P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 0.074s	user 0.000s	sys 0.000s
I20260709 13:22:35.541546 24304 raft_consensus.cc:2749] T 5a96373bb0e14dce8a266b2fc6c58507 P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:22:35.542815 24402 leader_election.cc:290] T d2ab0ac8347641e6955e9bce7d90c1a2 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:35.533366 24219 leader_election.cc:290] T b6f07842d3aa47858df5665874c86f3a P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 23 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:35.511449 24415 raft_consensus.cc:2804] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [term 24 FOLLOWER]: Leader election won for term 24
I20260709 13:22:35.523350 24401 consensus_queue.cc:1048] T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 15, Time since last communication: 0.000s
W20260709 13:22:35.556456 16192 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 211099us
I20260709 13:22:35.508533 24231 raft_consensus.cc:2804] T 8d4ab870a05347868956aa8b841b2981 P 647a343a542d4a139fe106c19095129b [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:22:35.539076 24298 leader_election.cc:290] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:35.563630 16294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6f07842d3aa47858df5665874c86f3a" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 23 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:22:35.509645 24276 consensus_queue.cc:1048] T d16dfad0c3b540e5b4b86492ddf87208 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.043s
I20260709 13:22:35.508656 24411 consensus_queue.cc:1059] T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: INVALID_TERM, Last received: 20.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.042s
I20260709 13:22:35.567869 24389 consensus_queue.cc:1059] T 8638ee363273426db1393dd81e5c7b72 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: INVALID_TERM, Last received: 20.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.002s
I20260709 13:22:35.570465 24389 raft_consensus.cc:3055] T 8638ee363273426db1393dd81e5c7b72 P cb82cc7bd6db46d980c75237a1402860 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 13:22:35.570801 24389 raft_consensus.cc:740] T 8638ee363273426db1393dd81e5c7b72 P cb82cc7bd6db46d980c75237a1402860 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:22:35.561267 24415 raft_consensus.cc:697] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [term 24 LEADER]: Becoming Leader. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
W20260709 13:22:35.571394 24424 log.cc:927] Time spent T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 0.054s	user 0.001s	sys 0.000s
I20260709 13:22:35.571724 24389 consensus_queue.cc:260] T 8638ee363273426db1393dd81e5c7b72 P cb82cc7bd6db46d980c75237a1402860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 23.18, Last appended by leader: 18, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:22:35.582260 16790 leader_election.cc:336] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 21 election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:22:35.585268 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 53531.5us
W20260709 13:22:35.586079 16265 leader_election.cc:336] T 8158bbd17b994d2eb96f560bd0f1b88d P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:22:35.582827 16790 leader_election.cc:304] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:22:35.568464 24231 raft_consensus.cc:697] T 8d4ab870a05347868956aa8b841b2981 P 647a343a542d4a139fe106c19095129b [term 22 LEADER]: Becoming Leader. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
W20260709 13:22:35.588085 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 99156.6us
I20260709 13:22:35.575901 24383 leader_election.cc:290] T 8cc959fb7f704fba9009c0595467d074 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
W20260709 13:22:35.605026 15046 thread.cc:527] Waited for 7000ms trying to join with heartbeat (tid 16176)
I20260709 13:22:35.550114 24120 consensus_queue.cc:237] T 5a959dbecc2648eb85d1cc550f5a3245 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.12, Last appended by leader: 12, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:35.589149 24389 raft_consensus.cc:3060] T 8638ee363273426db1393dd81e5c7b72 P cb82cc7bd6db46d980c75237a1402860 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:22:35.600095 24416 raft_consensus.cc:493] T 84f2da98864b4425bf7c0055951fdd4b P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:35.585835 24413 raft_consensus.cc:515] T 8158bbd17b994d2eb96f560bd0f1b88d P 90f3da57796b488c9087a2f523adf045 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:35.635571 24341 raft_consensus.cc:493] T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:35.647789 24199 raft_consensus.cc:493] T 829b4ad8c5a54eababeea1181b960d4b P 647a343a542d4a139fe106c19095129b [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:35.665809 16218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18ad7e3af5504571a69d10c00750a01c" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:35.642762 24231 consensus_queue.cc:237] T 8d4ab870a05347868956aa8b841b2981 P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.12, Last appended by leader: 12, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:35.572091 24415 consensus_queue.cc:237] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 20.16, Last appended by leader: 17, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:35.562690 24384 consensus_queue.cc:1059] T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: INVALID_TERM, Last received: 20.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.111s
I20260709 13:22:35.663278 24199 raft_consensus.cc:515] T 829b4ad8c5a54eababeea1181b960d4b P 647a343a542d4a139fe106c19095129b [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:22:35.690007 24423 log.cc:927] Time spent T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 0.068s	user 0.000s	sys 0.000s
I20260709 13:22:35.692303 24199 leader_election.cc:290] T 829b4ad8c5a54eababeea1181b960d4b P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:35.640031 24416 raft_consensus.cc:515] T 84f2da98864b4425bf7c0055951fdd4b P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:35.697216 24416 leader_election.cc:290] T 84f2da98864b4425bf7c0055951fdd4b P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
W20260709 13:22:35.699893 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 112807us
W20260709 13:22:35.697057 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 106858us
W20260709 13:22:35.705513 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 123268us
I20260709 13:22:35.709518 16823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7ca61592e848ea9b0fb81a2ce33cdc" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:35.655990 24341 raft_consensus.cc:515] T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:22:35.742635 16788 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:22:35.783356 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 48535.4us
W20260709 13:22:35.700675 16265 consensus_peers.cc:597] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:22:35.806125 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 105503us
W20260709 13:22:35.786839 16115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 53465.9us
I20260709 13:22:35.728523 24378 raft_consensus.cc:493] T 3a7ca61592e848ea9b0fb81a2ce33cdc P cb82cc7bd6db46d980c75237a1402860 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:22:35.789291 16295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cc959fb7f704fba9009c0595467d074" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 22 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:35.790000 16296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f2da98864b4425bf7c0055951fdd4b" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:35.790263 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e9b2bf7011f4ae688a79d7bb5a447b4" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:35.791785 16818 raft_consensus.cc:2468] T ada5a004ba7c46819051e3ccec71ffdb P cb82cc7bd6db46d980c75237a1402860 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 25.
I20260709 13:22:35.798106 16823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ab0ac8347641e6955e9bce7d90c1a2" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:35.798307 16223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5718c623ebee4d2bb770f0f9ddb2f164" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
W20260709 13:22:35.810156 16790 consensus_peers.cc:597] T 2794af5b99984efebdee564035841cb0 P cb82cc7bd6db46d980c75237a1402860 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:22:35.811595 16217 raft_consensus.cc:1240] T b6f07842d3aa47858df5665874c86f3a P 1ca2725d290e47d68b1a4b609c9816ac [term 23 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 22. Current term is 23. Ops: [22.14-22.14]
I20260709 13:22:35.815032 16218 raft_consensus.cc:1275] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac [term 23 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 20 index: 16. Preceding OpId from leader: term: 21 index: 17. (index mismatch)
I20260709 13:22:35.831205 16223 raft_consensus.cc:3060] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:22:35.830734 16300 raft_consensus.cc:2393] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 90f3da57796b488c9087a2f523adf045 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 in current term 22: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
I20260709 13:22:35.833016 16790 leader_election.cc:304] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 
W20260709 13:22:35.834707 24425 log.cc:927] Time spent T 66f75124215f4ca6bef831e174a792d8 P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 0.220s	user 0.001s	sys 0.000s
I20260709 13:22:35.840096 24216 raft_consensus.cc:493] T 746aabf856ab401c9ef5e859371e7b67 P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:22:35.840803 24216 raft_consensus.cc:515] T 746aabf856ab401c9ef5e859371e7b67 P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:35.841961 24420 raft_consensus.cc:2804] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860 [term 27 FOLLOWER]: Leader election won for term 27
I20260709 13:22:35.842743 24420 raft_consensus.cc:697] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860 [term 27 LEADER]: Becoming Leader. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:22:35.844003 24413 leader_election.cc:290] T 8158bbd17b994d2eb96f560bd0f1b88d P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 22 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:22:35.846114 24129 raft_consensus.cc:493] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:22:35.846725 24129 raft_consensus.cc:515] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:35.818421 23938 raft_consensus.cc:493] T 5a96373bb0e14dce8a266b2fc6c58507 P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:22:35.829691 24409 raft_consensus.cc:493] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:22:35.848876 24409 raft_consensus.cc:515] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:35.850366 16299 raft_consensus.cc:1240] T 8638ee363273426db1393dd81e5c7b72 P 90f3da57796b488c9087a2f523adf045 [term 24 FOLLOWER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 23. Current term is 24. Ops: []
I20260709 13:22:35.850955 16296 raft_consensus.cc:2468] T 84f2da98864b4425bf7c0055951fdd4b P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 20.
I20260709 13:22:35.854133 16300 raft_consensus.cc:1275] T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045 [term 25 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 25 index: 14. (index mismatch)
I20260709 13:22:35.853986 16790 leader_election.cc:304] T 74ca8140f35c4ae29e8a664f6bdfc829 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 90f3da57796b488c9087a2f523adf045
W20260709 13:22:35.856323 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 48291.2us
I20260709 13:22:35.858958 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f2da98864b4425bf7c0055951fdd4b" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:35.848100 23938 raft_consensus.cc:515] T 5a96373bb0e14dce8a266b2fc6c58507 P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:35.787732 24222 raft_consensus.cc:2749] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:22:35.863943 16220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cc959fb7f704fba9009c0595467d074" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 22 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:35.868103 24222 raft_consensus.cc:2804] T 74ca8140f35c4ae29e8a664f6bdfc829 P cb82cc7bd6db46d980c75237a1402860 [term 26 FOLLOWER]: Leader election won for term 26
I20260709 13:22:35.868683 24222 raft_consensus.cc:697] T 74ca8140f35c4ae29e8a664f6bdfc829 P cb82cc7bd6db46d980c75237a1402860 [term 26 LEADER]: Becoming Leader. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:22:35.869247 24362 consensus_queue.cc:1048] T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 13:22:35.869565 24222 consensus_queue.cc:237] T 74ca8140f35c4ae29e8a664f6bdfc829 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 25.17, Last appended by leader: 17, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:35.877350 16265 leader_election.cc:304] T 68a6838d82ef4cac83507f906139d90f P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
W20260709 13:22:35.884953 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 50714us
I20260709 13:22:35.887353 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdbd78acb4cf46298e0de8ba0d130e19" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 26 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:35.888504 16298 raft_consensus.cc:1275] T 68a6838d82ef4cac83507f906139d90f P 90f3da57796b488c9087a2f523adf045 [term 23 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 21 index: 12. Preceding OpId from leader: term: 23 index: 13. (index mismatch)
I20260709 13:22:36.000550 16297 raft_consensus.cc:2468] T cdbd78acb4cf46298e0de8ba0d130e19 P 90f3da57796b488c9087a2f523adf045 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 26.
I20260709 13:22:35.889581 16823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "829b4ad8c5a54eababeea1181b960d4b" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 22 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:35.890769 16292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a96373bb0e14dce8a266b2fc6c58507" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:35.796864 24242 raft_consensus.cc:3055] T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:22:35.849092 24216 leader_election.cc:290] T 746aabf856ab401c9ef5e859371e7b67 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
W20260709 13:22:35.885247 16265 consensus_peers.cc:597] T e4dc6b0541ca431ebc757beac00521f1 P 90f3da57796b488c9087a2f523adf045 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:22:35.895774 16222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "829b4ad8c5a54eababeea1181b960d4b" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 22 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:35.925221 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7ca61592e848ea9b0fb81a2ce33cdc" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
W20260709 13:22:35.949280 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 58676.8us
I20260709 13:22:35.952056 16816 raft_consensus.cc:1275] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 [term 23 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 21 index: 17. Preceding OpId from leader: term: 23 index: 18. (index mismatch)
I20260709 13:22:35.890111 24362 raft_consensus.cc:493] T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:35.891902 24334 raft_consensus.cc:493] T 7c2095a5f4ad49408dbc7f69c606120d P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:22:35.892068 24412 raft_consensus.cc:493] T 8cc959fb7f704fba9009c0595467d074 P 1ca2725d290e47d68b1a4b609c9816ac [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:22:35.869606 24223 consensus_queue.cc:1059] T b6f07842d3aa47858df5665874c86f3a P 647a343a542d4a139fe106c19095129b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: INVALID_TERM, Last received: 20.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.080s
I20260709 13:22:35.969877 16221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ab0ac8347641e6955e9bce7d90c1a2" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:35.882793 24378 raft_consensus.cc:515] T 3a7ca61592e848ea9b0fb81a2ce33cdc P cb82cc7bd6db46d980c75237a1402860 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:35.978332 16115 leader_election.cc:304] T 8cc959fb7f704fba9009c0595467d074 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:22:35.992637 16299 raft_consensus.cc:1275] T 66ae176961b74813a58bc15c9c4c6f2a P 90f3da57796b488c9087a2f523adf045 [term 29 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 27 index: 17. Preceding OpId from leader: term: 29 index: 18. (index mismatch)
I20260709 13:22:35.844012 24420 consensus_queue.cc:237] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 23.14, Last appended by leader: 14, Current term: 27, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:22:36.003794 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 118586us
I20260709 13:22:36.003340 24242 raft_consensus.cc:740] T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:22:36.016018 24227 raft_consensus.cc:2749] T 8cc959fb7f704fba9009c0595467d074 P 647a343a542d4a139fe106c19095129b [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:22:36.023757 24242 consensus_queue.cc:260] T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 20.16, Last appended by leader: 16, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:36.025297 24242 raft_consensus.cc:3060] T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:22:36.003561 24413 raft_consensus.cc:2749] T 68a6838d82ef4cac83507f906139d90f P 90f3da57796b488c9087a2f523adf045 [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:22:36.016150 24141 raft_consensus.cc:493] T 5a959dbecc2648eb85d1cc550f5a3245 P 647a343a542d4a139fe106c19095129b [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:36.027990 24141 raft_consensus.cc:515] T 5a959dbecc2648eb85d1cc550f5a3245 P 647a343a542d4a139fe106c19095129b [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:36.008900 24412 raft_consensus.cc:515] T 8cc959fb7f704fba9009c0595467d074 P 1ca2725d290e47d68b1a4b609c9816ac [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:36.030117 24141 leader_election.cc:290] T 5a959dbecc2648eb85d1cc550f5a3245 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:36.030606 24412 leader_election.cc:290] T 8cc959fb7f704fba9009c0595467d074 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:36.008692 24334 raft_consensus.cc:515] T 7c2095a5f4ad49408dbc7f69c606120d P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:22:36.036398 16191 consensus_peers.cc:597] T d16dfad0c3b540e5b4b86492ddf87208 P 1ca2725d290e47d68b1a4b609c9816ac -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:22:36.037792 16191 leader_election.cc:336] T b6f07842d3aa47858df5665874c86f3a P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 23 election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:22:36.038211 16191 leader_election.cc:304] T b6f07842d3aa47858df5665874c86f3a P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
W20260709 13:22:36.009184 16265 leader_election.cc:336] T 9e65676b21b748728ad39668cfe722ec P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:22:36.013530 16297 raft_consensus.cc:1240] T 8158bbd17b994d2eb96f560bd0f1b88d P 90f3da57796b488c9087a2f523adf045 [term 22 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 21. Current term is 22. Ops: []
I20260709 13:22:36.040438 16223 raft_consensus.cc:2468] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 24.
I20260709 13:22:36.014564 16116 leader_election.cc:304] T 829b4ad8c5a54eababeea1181b960d4b P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860
I20260709 13:22:36.018618 23719 consensus_queue.cc:1048] T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 13:22:36.019218 16190 leader_election.cc:304] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:22:36.040304 16265 leader_election.cc:304] T 9e65676b21b748728ad39668cfe722ec P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
I20260709 13:22:36.047905 16223 raft_consensus.cc:1240] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [term 26 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 25. Current term is 26. Ops: [23.16-25.17]
I20260709 13:22:36.022248 24411 consensus_queue.cc:1048] T 68a6838d82ef4cac83507f906139d90f P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:22:36.022518 16221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2372981006d4b218215c28964dea623" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 28 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:36.050024 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cc959fb7f704fba9009c0595467d074" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:36.053180 24216 raft_consensus.cc:2804] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [term 25 FOLLOWER]: Leader pre-election won for term 26
W20260709 13:22:36.053578 16265 leader_election.cc:336] T 18ad7e3af5504571a69d10c00750a01c P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:22:36.054062 16265 leader_election.cc:304] T 18ad7e3af5504571a69d10c00750a01c P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:22:36.053622 24216 raft_consensus.cc:493] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [term 25 FOLLOWER]: Starting leader election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:22:36.054756 24216 raft_consensus.cc:3060] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:22:36.050483 24427 raft_consensus.cc:2749] T b6f07842d3aa47858df5665874c86f3a P 1ca2725d290e47d68b1a4b609c9816ac [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 13:22:36.054260 24280 raft_consensus.cc:2749] T 9e65676b21b748728ad39668cfe722ec P 90f3da57796b488c9087a2f523adf045 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:22:36.059185 24341 leader_election.cc:290] T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:36.059894 24409 leader_election.cc:290] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:36.062772 16222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8158bbd17b994d2eb96f560bd0f1b88d" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:22:36.066295 16265 leader_election.cc:304] T d2ab0ac8347641e6955e9bce7d90c1a2 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860
I20260709 13:22:36.067973 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87ac0d1f7e8a419f9bcbc04aee0e067e" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:36.069250 16823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2372981006d4b218215c28964dea623" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 28 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
W20260709 13:22:36.070801 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 56367.1us
I20260709 13:22:36.078485 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a959dbecc2648eb85d1cc550f5a3245" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:36.079481 16299 raft_consensus.cc:2468] T 5a959dbecc2648eb85d1cc550f5a3245 P 90f3da57796b488c9087a2f523adf045 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 22.
I20260709 13:22:36.088390 16117 leader_election.cc:304] T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860
I20260709 13:22:36.007799 24362 raft_consensus.cc:515] T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:36.074905 24196 raft_consensus.cc:2749] T 829b4ad8c5a54eababeea1181b960d4b P 647a343a542d4a139fe106c19095129b [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:22:36.081943 24271 raft_consensus.cc:2749] T 18ad7e3af5504571a69d10c00750a01c P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:22:36.109160 16265 leader_election.cc:336] T 8158bbd17b994d2eb96f560bd0f1b88d P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:22:36.106810 24223 raft_consensus.cc:493] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:36.110258 24223 raft_consensus.cc:515] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:36.116042 16294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f5f89186204242bf377f7c42b2e2da" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 27 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:36.108300 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c2095a5f4ad49408dbc7f69c606120d" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 20 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:36.119031 24223 leader_election.cc:290] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:36.085552 24176 raft_consensus.cc:493] T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:36.135967 16222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a959dbecc2648eb85d1cc550f5a3245" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
W20260709 13:22:36.150790 24442 log.cc:927] Time spent T 1df93779209748b9860f6d893dd3de53 P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 0.095s	user 0.000s	sys 0.000s
I20260709 13:22:36.147418 24176 raft_consensus.cc:515] T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:36.156839 23938 leader_election.cc:290] T 5a96373bb0e14dce8a266b2fc6c58507 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:36.157856 24176 leader_election.cc:290] T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:36.161479 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7ca61592e848ea9b0fb81a2ce33cdc" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:36.162487 16265 leader_election.cc:304] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860
W20260709 13:22:36.162958 24431 log.cc:927] Time spent T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860: Append to log took a long time: real 0.574s	user 0.012s	sys 0.005s
I20260709 13:22:36.155377 24378 leader_election.cc:290] T 3a7ca61592e848ea9b0fb81a2ce33cdc P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:36.164464 16265 leader_election.cc:304] T 8158bbd17b994d2eb96f560bd0f1b88d P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:22:36.096494 24280 raft_consensus.cc:2749] T d2ab0ac8347641e6955e9bce7d90c1a2 P 90f3da57796b488c9087a2f523adf045 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:22:36.172003 16192 leader_election.cc:304] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 90f3da57796b488c9087a2f523adf045
I20260709 13:22:36.174598 16115 leader_election.cc:304] T 84f2da98864b4425bf7c0055951fdd4b P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac
I20260709 13:22:36.175777 16294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4f65cdf00b748bb963ee6e81ef43b79" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 24 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:36.176783 16823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746aabf856ab401c9ef5e859371e7b67" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:36.144074 24196 raft_consensus.cc:2749] T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 13:22:36.126076 24334 leader_election.cc:290] T 7c2095a5f4ad49408dbc7f69c606120d P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
W20260709 13:22:36.172982 24439 log.cc:927] Time spent T d16dfad0c3b540e5b4b86492ddf87208 P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.073s	user 0.000s	sys 0.000s
I20260709 13:22:36.185250 24418 consensus_queue.cc:1059] T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: INVALID_TERM, Last received: 21.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.014s
I20260709 13:22:36.175381 24407 raft_consensus.cc:493] T d16dfad0c3b540e5b4b86492ddf87208 P 647a343a542d4a139fe106c19095129b [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:36.178154 24415 raft_consensus.cc:2804] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac [term 25 FOLLOWER]: Leader election won for term 25
I20260709 13:22:36.204780 24415 raft_consensus.cc:697] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac [term 25 LEADER]: Becoming Leader. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:22:36.205849 24415 consensus_queue.cc:237] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 24.17, Last appended by leader: 17, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:36.186996 24271 raft_consensus.cc:2749] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 90f3da57796b488c9087a2f523adf045 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:22:36.208389 23994 consensus_queue.cc:1059] T 8158bbd17b994d2eb96f560bd0f1b88d P 647a343a542d4a139fe106c19095129b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: INVALID_TERM, Last received: 21.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:22:36.209605 24362 leader_election.cc:290] T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:36.183331 24224 raft_consensus.cc:2804] T 84f2da98864b4425bf7c0055951fdd4b P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:22:36.185408 24400 raft_consensus.cc:2749] T 8158bbd17b994d2eb96f560bd0f1b88d P 90f3da57796b488c9087a2f523adf045 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:22:36.179953 24389 consensus_queue.cc:1048] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.055s
I20260709 13:22:36.202979 24407 raft_consensus.cc:515] T d16dfad0c3b540e5b4b86492ddf87208 P 647a343a542d4a139fe106c19095129b [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:36.279572 24407 leader_election.cc:290] T d16dfad0c3b540e5b4b86492ddf87208 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:36.279961 24216 raft_consensus.cc:515] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [term 26 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:36.282342 24216 leader_election.cc:290] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 26 election: Requested vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:36.238632 24224 raft_consensus.cc:493] T 84f2da98864b4425bf7c0055951fdd4b P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:36.196154 24176 consensus_queue.cc:1048] T 9c61a39fd4f84424b129c1a6f9ba404d P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 11, Time since last communication: 0.078s
I20260709 13:22:36.290122 16817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdbd78acb4cf46298e0de8ba0d130e19" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 26 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:22:36.291456 16817 raft_consensus.cc:2393] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 26: Already voted for candidate cb82cc7bd6db46d980c75237a1402860 in this term.
I20260709 13:22:36.283093 24224 raft_consensus.cc:3060] T 84f2da98864b4425bf7c0055951fdd4b P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:22:36.295993 24176 raft_consensus.cc:493] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:36.297148 24176 raft_consensus.cc:515] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:36.306958 16823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66f75124215f4ca6bef831e174a792d8" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:36.325940 16819 raft_consensus.cc:1240] T 74ca8140f35c4ae29e8a664f6bdfc829 P cb82cc7bd6db46d980c75237a1402860 [term 26 LEADER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 25. Current term is 26. Ops: []
I20260709 13:22:36.303515 24313 raft_consensus.cc:493] T ada5a004ba7c46819051e3ccec71ffdb P 90f3da57796b488c9087a2f523adf045 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
W20260709 13:22:36.337286 16115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 148395us
I20260709 13:22:36.339206 16115 leader_election.cc:304] T 5a959dbecc2648eb85d1cc550f5a3245 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:22:36.341734 24184 raft_consensus.cc:2804] T 5a959dbecc2648eb85d1cc550f5a3245 P 647a343a542d4a139fe106c19095129b [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:22:36.342253 24184 raft_consensus.cc:493] T 5a959dbecc2648eb85d1cc550f5a3245 P 647a343a542d4a139fe106c19095129b [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:36.342742 24184 raft_consensus.cc:3060] T 5a959dbecc2648eb85d1cc550f5a3245 P 647a343a542d4a139fe106c19095129b [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:22:36.317895 24416 raft_consensus.cc:493] T 434bff3082d141838068f323bdb0d1ed P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:22:36.363919 24129 leader_election.cc:290] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:36.344045 24190 raft_consensus.cc:493] T 23f82224f4564fea93572e8206bad1fe P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:36.343851 24227 raft_consensus.cc:493] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 647a343a542d4a139fe106c19095129b [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:36.393326 16220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4f65cdf00b748bb963ee6e81ef43b79" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 24 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:36.396999 16225 raft_consensus.cc:1275] T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac [term 22 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 20 index: 12. Preceding OpId from leader: term: 22 index: 13. (index mismatch)
I20260709 13:22:36.399678 16825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdbd78acb4cf46298e0de8ba0d130e19" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 26 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:36.397956 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f5f89186204242bf377f7c42b2e2da" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 27 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:36.398108 16219 raft_consensus.cc:1240] T b6f07842d3aa47858df5665874c86f3a P 1ca2725d290e47d68b1a4b609c9816ac [term 23 FOLLOWER]: Rejecting Update request from peer 647a343a542d4a139fe106c19095129b for earlier term 22. Current term is 23. Ops: [22.14-22.14]
I20260709 13:22:36.403862 16825 raft_consensus.cc:2393] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 26: Already voted for candidate cb82cc7bd6db46d980c75237a1402860 in this term.
I20260709 13:22:36.400537 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d16dfad0c3b540e5b4b86492ddf87208" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 23 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:36.368351 24190 raft_consensus.cc:515] T 23f82224f4564fea93572e8206bad1fe P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:36.443029 16222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66f75124215f4ca6bef831e174a792d8" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:36.398422 24217 raft_consensus.cc:493] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac [term 24 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:22:36.374776 24416 raft_consensus.cc:515] T 434bff3082d141838068f323bdb0d1ed P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:36.444260 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d16dfad0c3b540e5b4b86492ddf87208" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 23 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:36.356648 24313 raft_consensus.cc:515] T ada5a004ba7c46819051e3ccec71ffdb P 90f3da57796b488c9087a2f523adf045 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:36.393577 24227 raft_consensus.cc:515] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 647a343a542d4a139fe106c19095129b [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:36.459519 16817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806d7690c2b644959152074bb2cab0a7" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:36.477241 16817 raft_consensus.cc:1275] T 8638ee363273426db1393dd81e5c7b72 P cb82cc7bd6db46d980c75237a1402860 [term 24 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 23 index: 18. Preceding OpId from leader: term: 24 index: 17. (term mismatch)
I20260709 13:22:36.477712 16817 pending_rounds.cc:85] T 8638ee363273426db1393dd81e5c7b72 P cb82cc7bd6db46d980c75237a1402860: Aborting all ops after (but not including) 16
I20260709 13:22:36.477975 16817 pending_rounds.cc:107] T 8638ee363273426db1393dd81e5c7b72 P cb82cc7bd6db46d980c75237a1402860: Aborting uncommitted NO_OP operation due to leader change: 21.17
I20260709 13:22:36.478225 16817 raft_consensus.cc:2889] T 8638ee363273426db1393dd81e5c7b72 P cb82cc7bd6db46d980c75237a1402860 [term 24 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:22:36.478557 16817 pending_rounds.cc:107] T 8638ee363273426db1393dd81e5c7b72 P cb82cc7bd6db46d980c75237a1402860: Aborting uncommitted NO_OP operation due to leader change: 23.18
I20260709 13:22:36.478871 16817 raft_consensus.cc:2889] T 8638ee363273426db1393dd81e5c7b72 P cb82cc7bd6db46d980c75237a1402860 [term 24 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:22:36.484341 24190 leader_election.cc:290] T 23f82224f4564fea93572e8206bad1fe P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:36.448945 24217 raft_consensus.cc:515] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:36.485442 16825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f82224f4564fea93572e8206bad1fe" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
W20260709 13:22:36.485641 16191 consensus_peers.cc:597] T 68a6838d82ef4cac83507f906139d90f P 1ca2725d290e47d68b1a4b609c9816ac -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:22:36.487850 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 445186us
I20260709 13:22:36.491281 24217 leader_election.cc:290] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:36.491133 23924 consensus_queue.cc:1048] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 13:22:36.459112 24176 leader_election.cc:290] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:36.496479 16825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5d59407eb545b895d8d922e0dd69fd" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 25 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:36.499368 24461 raft_consensus.cc:493] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:22:36.499994 24461 raft_consensus.cc:515] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:36.514979 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f82224f4564fea93572e8206bad1fe" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:36.516012 16291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806d7690c2b644959152074bb2cab0a7" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:36.516660 16291 raft_consensus.cc:2468] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 21.
W20260709 13:22:36.525188 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 46664.1us
I20260709 13:22:36.449024 24224 raft_consensus.cc:515] T 84f2da98864b4425bf7c0055951fdd4b P 647a343a542d4a139fe106c19095129b [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:22:36.494859 24454 log.cc:927] Time spent T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860: Append to log took a long time: real 0.107s	user 0.001s	sys 0.000s
I20260709 13:22:36.578498 24397 raft_consensus.cc:3055] T b6f07842d3aa47858df5665874c86f3a P 647a343a542d4a139fe106c19095129b [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:22:36.579023 24397 raft_consensus.cc:740] T b6f07842d3aa47858df5665874c86f3a P 647a343a542d4a139fe106c19095129b [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:22:36.579869 24397 consensus_queue.cc:260] T b6f07842d3aa47858df5665874c86f3a P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 22.14, Last appended by leader: 14, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:36.581110 24397 raft_consensus.cc:3060] T b6f07842d3aa47858df5665874c86f3a P 647a343a542d4a139fe106c19095129b [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:22:36.582456 24200 raft_consensus.cc:3055] T 8158bbd17b994d2eb96f560bd0f1b88d P 647a343a542d4a139fe106c19095129b [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:22:36.585304 24453 raft_consensus.cc:493] T 806d7690c2b644959152074bb2cab0a7 P cb82cc7bd6db46d980c75237a1402860 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:22:36.586179 24453 raft_consensus.cc:515] T 806d7690c2b644959152074bb2cab0a7 P cb82cc7bd6db46d980c75237a1402860 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:36.593174 16297 raft_consensus.cc:1275] T ecc1e363230043fe9fef390f0b979f20 P 90f3da57796b488c9087a2f523adf045 [term 27 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 23 index: 14. Preceding OpId from leader: term: 27 index: 15. (index mismatch)
I20260709 13:22:36.583041 24200 raft_consensus.cc:740] T 8158bbd17b994d2eb96f560bd0f1b88d P 647a343a542d4a139fe106c19095129b [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:22:36.595077 23719 raft_consensus.cc:493] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 647a343a542d4a139fe106c19095129b [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:22:36.595882 23719 raft_consensus.cc:515] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 647a343a542d4a139fe106c19095129b [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:36.596149 24200 consensus_queue.cc:260] T 8158bbd17b994d2eb96f560bd0f1b88d P 647a343a542d4a139fe106c19095129b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 21.14, Last appended by leader: 14, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:36.597509 24200 raft_consensus.cc:3060] T 8158bbd17b994d2eb96f560bd0f1b88d P 647a343a542d4a139fe106c19095129b [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:22:36.585168 16221 raft_consensus.cc:3055] T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac [term 24 LEADER]: Stepping down as leader of term 24
I20260709 13:22:36.603463 16221 raft_consensus.cc:740] T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
W20260709 13:22:36.605722 15046 thread.cc:527] Waited for 8000ms trying to join with heartbeat (tid 16176)
I20260709 13:22:36.604808 16221 consensus_queue.cc:260] T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 24.13, Last appended by leader: 13, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:36.606504 16221 raft_consensus.cc:3060] T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac [term 24 FOLLOWER]: Advancing to term 25
W20260709 13:22:36.651687 16790 consensus_peers.cc:597] T 57a744f5aaf944678a63469cbf514015 P cb82cc7bd6db46d980c75237a1402860 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:22:36.652374 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 430977us
W20260709 13:22:36.653199 16790 leader_election.cc:336] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:22:36.653806 16790 leader_election.cc:304] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:22:36.684212 16226 raft_consensus.cc:1240] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [term 26 FOLLOWER]: Rejecting Update request from peer 90f3da57796b488c9087a2f523adf045 for earlier term 25. Current term is 26. Ops: [23.16-25.17]
W20260709 13:22:36.699555 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 46391.4us
W20260709 13:22:36.700704 16790 leader_election.cc:336] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 27 election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:22:36.702081 16790 leader_election.cc:336] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 24 election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:22:36.688468 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 190782us
W20260709 13:22:36.703459 16790 leader_election.cc:336] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:22:36.550868 24224 leader_election.cc:290] T 84f2da98864b4425bf7c0055951fdd4b P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 21 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:36.739367 16221 raft_consensus.cc:1275] T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac [term 25 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 24 index: 13. Preceding OpId from leader: term: 25 index: 14. (index mismatch)
I20260709 13:22:36.738078 16116 leader_election.cc:304] T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
W20260709 13:22:36.747956 16189 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:22:36.736084 16790 leader_election.cc:304] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:22:36.685822 24222 raft_consensus.cc:2749] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
W20260709 13:22:36.753887 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 50465.5us
W20260709 13:22:36.757121 16790 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 143 similar messages]
W20260709 13:22:36.738152 24430 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.677s	user 0.007s	sys 0.007s
I20260709 13:22:36.755385 16116 leader_election.cc:304] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
W20260709 13:22:36.768024 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 279522us
W20260709 13:22:36.754398 23958 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 25 current term: 25 is higher.
I20260709 13:22:36.750042 24129 raft_consensus.cc:493] T 66f75124215f4ca6bef831e174a792d8 P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:22:36.752205 24223 raft_consensus.cc:2749] T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:22:36.759204 24420 raft_consensus.cc:3060] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:22:36.740777 24398 consensus_queue.cc:1048] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 12, Time since last communication: 0.031s
I20260709 13:22:36.793766 16221 raft_consensus.cc:1275] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [term 29 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 26 index: 16. Preceding OpId from leader: term: 29 index: 18. (index mismatch)
I20260709 13:22:36.796475 16218 raft_consensus.cc:1275] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [term 26 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 21 index: 15. Preceding OpId from leader: term: 26 index: 18. (index mismatch)
I20260709 13:22:36.786572 24129 raft_consensus.cc:515] T 66f75124215f4ca6bef831e174a792d8 P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:36.792236 23719 leader_election.cc:290] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:36.790621 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7ca61592e848ea9b0fb81a2ce33cdc" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:36.777731 24088 raft_consensus.cc:2749] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 13:22:36.758785 24176 consensus_queue.cc:1048] T 8d4ab870a05347868956aa8b841b2981 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.029s
W20260709 13:22:36.774518 24430 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.714s	user 0.008s	sys 0.008s
I20260709 13:22:36.780153 16116 leader_election.cc:304] T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860
I20260709 13:22:36.832412 16790 leader_election.cc:304] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 647a343a542d4a139fe106c19095129b
I20260709 13:22:36.798228 16218 raft_consensus.cc:1275] T ecc1e363230043fe9fef390f0b979f20 P 1ca2725d290e47d68b1a4b609c9816ac [term 27 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 23 index: 14. Preceding OpId from leader: term: 27 index: 15. (index mismatch)
I20260709 13:22:36.868186 16040 catalog_manager.cc:5697] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 reported cstate change: term changed from 17 to 20, leader changed from 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130) to cb82cc7bd6db46d980c75237a1402860 (127.14.177.132). New cstate: current_term: 20 leader_uuid: "cb82cc7bd6db46d980c75237a1402860" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:22:36.871712 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 91634.4us
I20260709 13:22:36.838359 24227 leader_election.cc:290] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:36.880776 16218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806d7690c2b644959152074bb2cab0a7" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:36.881518 16218 raft_consensus.cc:2468] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 22.
I20260709 13:22:36.882819 16219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87ac0d1f7e8a419f9bcbc04aee0e067e" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:36.905182 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f2da98864b4425bf7c0055951fdd4b" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:22:36.905922 16224 raft_consensus.cc:3060] T 84f2da98864b4425bf7c0055951fdd4b P 1ca2725d290e47d68b1a4b609c9816ac [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:22:36.905776 16790 leader_election.cc:304] T 3a7ca61592e848ea9b0fb81a2ce33cdc P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:22:36.918476 16823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e9b2bf7011f4ae688a79d7bb5a447b4" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:36.864039 24444 raft_consensus.cc:493] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:22:36.913450 24088 raft_consensus.cc:493] T ecc1e363230043fe9fef390f0b979f20 P 647a343a542d4a139fe106c19095129b [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:36.878849 24463 raft_consensus.cc:2749] T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:22:36.903510 24125 raft_consensus.cc:2804] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 13:22:36.924237 24227 raft_consensus.cc:493] T 68a6838d82ef4cac83507f906139d90f P 647a343a542d4a139fe106c19095129b [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:22:36.950572 24184 consensus_meta.cc:287] T 5a959dbecc2648eb85d1cc550f5a3245 P 647a343a542d4a139fe106c19095129b: Time spent flushing consensus metadata: real 0.607s	user 0.012s	sys 0.005s
I20260709 13:22:37.037362 16224 raft_consensus.cc:2468] T 84f2da98864b4425bf7c0055951fdd4b P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 21.
I20260709 13:22:36.903800 24399 consensus_queue.cc:1048] T 9c61a39fd4f84424b129c1a6f9ba404d P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 11, Time since last communication: 0.099s
I20260709 13:22:36.932633 24224 raft_consensus.cc:493] T 18ad7e3af5504571a69d10c00750a01c P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:36.992182 24384 raft_consensus.cc:493] T b87147731a68400e8cce9b879daa7c5d P 647a343a542d4a139fe106c19095129b [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:22:37.053572 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 147901us
I20260709 13:22:37.061570 16040 catalog_manager.cc:5697] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 reported cstate change: term changed from 18 to 22. New cstate: current_term: 22 leader_uuid: "cb82cc7bd6db46d980c75237a1402860" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:22:37.049119 24280 raft_consensus.cc:493] T a378d7f796f94615b3b347119a9880fd P 90f3da57796b488c9087a2f523adf045 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:22:36.993300 24304 raft_consensus.cc:493] T 31969392ccdb43b491911a319e8bc7e1 P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:22:36.931427 24398 consensus_queue.cc:1048] T 74ca8140f35c4ae29e8a664f6bdfc829 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 15, Time since last communication: 0.120s
I20260709 13:22:37.031911 24125 raft_consensus.cc:697] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 [term 24 LEADER]: Becoming Leader. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:22:36.950188 24444 raft_consensus.cc:515] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:36.920569 24378 consensus_queue.cc:1048] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 16, Time since last communication: 0.113s
W20260709 13:22:37.075098 24475 log.cc:927] Time spent T 66ae176961b74813a58bc15c9c4c6f2a P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.210s	user 0.000s	sys 0.000s
I20260709 13:22:37.071935 24399 consensus_queue.cc:1243] T 9c61a39fd4f84424b129c1a6f9ba404d P 647a343a542d4a139fe106c19095129b [LEADER]: Peer 1ca2725d290e47d68b1a4b609c9816ac log is divergent from this leader: its last log entry 24.13 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 11
I20260709 13:22:37.000617 24088 raft_consensus.cc:515] T ecc1e363230043fe9fef390f0b979f20 P 647a343a542d4a139fe106c19095129b [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:22:37.099999 16789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 179439us
I20260709 13:22:37.081725 24313 leader_election.cc:290] T ada5a004ba7c46819051e3ccec71ffdb P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:37.072618 24184 raft_consensus.cc:515] T 5a959dbecc2648eb85d1cc550f5a3245 P 647a343a542d4a139fe106c19095129b [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:37.073109 24227 raft_consensus.cc:515] T 68a6838d82ef4cac83507f906139d90f P 647a343a542d4a139fe106c19095129b [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:37.107547 24459 raft_consensus.cc:493] T 7c2095a5f4ad49408dbc7f69c606120d P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:37.079985 24224 raft_consensus.cc:515] T 18ad7e3af5504571a69d10c00750a01c P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } }
I20260709 13:22:37.083367 24384 raft_consensus.cc:515] T b87147731a68400e8cce9b879daa7c5d P 647a343a542d4a139fe106c19095129b [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:37.087065 24304 raft_consensus.cc:515] T 31969392ccdb43b491911a319e8bc7e1 P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:37.085832 24280 raft_consensus.cc:515] T a378d7f796f94615b3b347119a9880fd P 90f3da57796b488c9087a2f523adf045 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:22:37.123610 24474 log.cc:927] Time spent T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.250s	user 0.001s	sys 0.001s
I20260709 13:22:37.150593 16115 leader_election.cc:304] T d16dfad0c3b540e5b4b86492ddf87208 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:22:37.121507 24453 leader_election.cc:290] T 806d7690c2b644959152074bb2cab0a7 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:37.154407 16115 leader_election.cc:304] T 23f82224f4564fea93572e8206bad1fe P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860
I20260709 13:22:37.150817 24125 consensus_queue.cc:237] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 22.17, Last appended by leader: 17, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:37.155390 24463 raft_consensus.cc:2749] T d16dfad0c3b540e5b4b86492ddf87208 P 647a343a542d4a139fe106c19095129b [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:22:37.156180 24463 raft_consensus.cc:2749] T 23f82224f4564fea93572e8206bad1fe P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:22:37.156529 24184 leader_election.cc:290] T 5a959dbecc2648eb85d1cc550f5a3245 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 23 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:37.142779 24485 raft_consensus.cc:2749] T 3a7ca61592e848ea9b0fb81a2ce33cdc P cb82cc7bd6db46d980c75237a1402860 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 13:22:37.159971 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 434881us
I20260709 13:22:37.160349 24384 leader_election.cc:290] T b87147731a68400e8cce9b879daa7c5d P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:37.155694 16115 leader_election.cc:304] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: cb82cc7bd6db46d980c75237a1402860
I20260709 13:22:37.165180 23719 raft_consensus.cc:2804] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:22:37.165619 23719 raft_consensus.cc:493] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:37.165974 23719 raft_consensus.cc:3060] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:22:37.167584 24321 raft_consensus.cc:3055] T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045 [term 25 LEADER]: Stepping down as leader of term 25
I20260709 13:22:37.168066 24321 raft_consensus.cc:740] T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045 [term 25 LEADER]: Becoming Follower/Learner. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:22:37.169334 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434bff3082d141838068f323bdb0d1ed" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:37.169010 24321 consensus_queue.cc:260] T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 25.17, Last appended by leader: 17, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:37.170281 24321 raft_consensus.cc:3060] T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045 [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:22:37.170571 16295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87ac0d1f7e8a419f9bcbc04aee0e067e" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:37.171444 16297 raft_consensus.cc:3060] T 8d4ab870a05347868956aa8b841b2981 P 90f3da57796b488c9087a2f523adf045 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:22:37.175599 16291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e9b2bf7011f4ae688a79d7bb5a447b4" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:37.176184 24444 leader_election.cc:290] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:37.176338 16291 raft_consensus.cc:2468] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 90f3da57796b488c9087a2f523adf045 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 22.
I20260709 13:22:37.177206 24398 consensus_queue.cc:1048] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:22:37.178776 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a959dbecc2648eb85d1cc550f5a3245" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:22:37.179489 16298 raft_consensus.cc:3060] T 5a959dbecc2648eb85d1cc550f5a3245 P 90f3da57796b488c9087a2f523adf045 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:22:37.180430 24088 leader_election.cc:290] T ecc1e363230043fe9fef390f0b979f20 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:37.187742 24378 raft_consensus.cc:493] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:37.188316 24378 raft_consensus.cc:515] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:37.190655 24378 leader_election.cc:290] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:37.195775 24463 raft_consensus.cc:493] T fffab26427e24d398e2695aad40ca134 P 647a343a542d4a139fe106c19095129b [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:37.196444 24463 raft_consensus.cc:515] T fffab26427e24d398e2695aad40ca134 P 647a343a542d4a139fe106c19095129b [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:37.199121 24461 leader_election.cc:290] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:37.198734 24463 leader_election.cc:290] T fffab26427e24d398e2695aad40ca134 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:37.199723 16294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f2da98864b4425bf7c0055951fdd4b" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:22:37.200557 16294 raft_consensus.cc:3060] T 84f2da98864b4425bf7c0055951fdd4b P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:22:37.204288 24224 leader_election.cc:290] T 18ad7e3af5504571a69d10c00750a01c P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:22:37.208415 24129 leader_election.cc:290] T 66f75124215f4ca6bef831e174a792d8 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:37.210237 16295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdbd78acb4cf46298e0de8ba0d130e19" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 26 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:22:37.210346 24486 raft_consensus.cc:493] T 829b4ad8c5a54eababeea1181b960d4b P cb82cc7bd6db46d980c75237a1402860 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:22:37.211236 24486 raft_consensus.cc:515] T 829b4ad8c5a54eababeea1181b960d4b P cb82cc7bd6db46d980c75237a1402860 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:37.213945 24486 leader_election.cc:290] T 829b4ad8c5a54eababeea1181b960d4b P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:22:37.188591 16819 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49892 (request call id 1246) took 1247 ms. Trace:
I20260709 13:22:37.214679 16819 rpcz_store.cc:276] 0709 13:22:35.940766 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:22:35.942167 (+  1401us) service_pool.cc:224] Handling call
0709 13:22:35.942454 (+   287us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:22:36.327410 (+384956us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:22:37.188330 (+860920us) spinlock_profiling.cc:243] Waited 382 ms on lock 0x7b54005a0d54. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b5fe01 00007f69b0b5e83e 00007f69b5b23419 00007f69af09abd4 00007f69af09ab62 00007f69af09aa91 00007f69af09aa0c 00007f69af099cc2 00007f69aee9a48c 00007f69aee998c6
0709 13:22:37.188525 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":877922048}
I20260709 13:22:37.218508 16291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87147731a68400e8cce9b879daa7c5d" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 23 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
W20260709 13:22:37.214504 24397 consensus_meta.cc:287] T b6f07842d3aa47858df5665874c86f3a P 647a343a542d4a139fe106c19095129b: Time spent flushing consensus metadata: real 0.633s	user 0.012s	sys 0.001s
I20260709 13:22:37.230732 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 27 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:37.233515 16291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fffab26427e24d398e2695aad40ca134" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 23 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:37.234287 16115 leader_election.cc:304] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:22:37.231343 16295 raft_consensus.cc:2468] T cdbd78acb4cf46298e0de8ba0d130e19 P 90f3da57796b488c9087a2f523adf045 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 26.
I20260709 13:22:37.236127 16295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18ad7e3af5504571a69d10c00750a01c" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:37.240828 16294 raft_consensus.cc:2468] T 84f2da98864b4425bf7c0055951fdd4b P 90f3da57796b488c9087a2f523adf045 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 21.
I20260709 13:22:37.238162 16297 raft_consensus.cc:1275] T 8d4ab870a05347868956aa8b841b2981 P 90f3da57796b488c9087a2f523adf045 [term 22 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 20 index: 12. Preceding OpId from leader: term: 22 index: 13. (index mismatch)
I20260709 13:22:37.200989 24416 leader_election.cc:290] T 434bff3082d141838068f323bdb0d1ed P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:37.201088 23719 raft_consensus.cc:515] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b [term 23 FOLLOWER]: Starting leader election with config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:37.236487 16298 raft_consensus.cc:2468] T 5a959dbecc2648eb85d1cc550f5a3245 P 90f3da57796b488c9087a2f523adf045 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 23.
I20260709 13:22:37.222712 24420 raft_consensus.cc:2749] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:22:37.122213 24459 raft_consensus.cc:515] T 7c2095a5f4ad49408dbc7f69c606120d P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:37.271826 16040 catalog_manager.cc:5697] T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860 reported cstate change: term changed from 11 to 21. New cstate: current_term: 21 leader_uuid: "cb82cc7bd6db46d980c75237a1402860" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:22:37.274982 16115 leader_election.cc:304] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:22:37.293097 24231 raft_consensus.cc:2749] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 647a343a542d4a139fe106c19095129b [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:22:37.297309 24304 leader_election.cc:290] T 31969392ccdb43b491911a319e8bc7e1 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:37.298136 24227 leader_election.cc:290] T 68a6838d82ef4cac83507f906139d90f P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:37.299480 16299 raft_consensus.cc:1217] T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045 [term 25 FOLLOWER]: Deduplicated request from leader. Original: 20.11->[22.12-24.13]   Dedup: 22.12->[24.13-24.13]
I20260709 13:22:37.310649 16299 raft_consensus.cc:1240] T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045 [term 25 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 24. Current term is 25. Ops: [22.12-24.13]
W20260709 13:22:37.265815 16191 leader_election.cc:336] T 746aabf856ab401c9ef5e859371e7b67 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:22:37.313040 16191 leader_election.cc:304] T 746aabf856ab401c9ef5e859371e7b67 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
I20260709 13:22:37.305042 16040 catalog_manager.cc:5697] T 8c11074579b046d99e585c2818e18a2c P cb82cc7bd6db46d980c75237a1402860 reported cstate change: term changed from 15 to 20, leader changed from cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) to 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130). New cstate: current_term: 20 leader_uuid: "1ca2725d290e47d68b1a4b609c9816ac" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:22:37.314855 16040 catalog_manager.cc:5697] T 8638ee363273426db1393dd81e5c7b72 P cb82cc7bd6db46d980c75237a1402860 reported cstate change: term changed from 17 to 21. New cstate: current_term: 21 leader_uuid: "cb82cc7bd6db46d980c75237a1402860" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:22:37.317657 16040 catalog_manager.cc:5697] T 19baa7bc367844c288bca8f81439c413 P cb82cc7bd6db46d980c75237a1402860 reported cstate change: term changed from 17 to 23, leader changed from cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) to 90f3da57796b488c9087a2f523adf045 (127.14.177.131). New cstate: current_term: 23 leader_uuid: "90f3da57796b488c9087a2f523adf045" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:22:37.326036 16219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a959dbecc2648eb85d1cc550f5a3245" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:22:37.320510 16040 catalog_manager.cc:5697] T 829b4ad8c5a54eababeea1181b960d4b P cb82cc7bd6db46d980c75237a1402860 reported cstate change: term changed from 16 to 19, leader changed from cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) to 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130). New cstate: current_term: 19 leader_uuid: "1ca2725d290e47d68b1a4b609c9816ac" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:22:37.309826 16823 raft_consensus.cc:2468] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 22.
I20260709 13:22:37.311475 16115 leader_election.cc:304] T 84f2da98864b4425bf7c0055951fdd4b P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:22:37.352052 24227 raft_consensus.cc:2804] T 84f2da98864b4425bf7c0055951fdd4b P 647a343a542d4a139fe106c19095129b [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:22:37.352382 16115 leader_election.cc:304] T 5a959dbecc2648eb85d1cc550f5a3245 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:22:37.352583 24199 consensus_queue.cc:1048] T 8d4ab870a05347868956aa8b841b2981 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:22:37.352634 24227 raft_consensus.cc:697] T 84f2da98864b4425bf7c0055951fdd4b P 647a343a542d4a139fe106c19095129b [term 21 LEADER]: Becoming Leader. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:22:37.353940 24227 consensus_queue.cc:237] T 84f2da98864b4425bf7c0055951fdd4b P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:37.354708 24492 raft_consensus.cc:493] T 26354361e9b044cea1c2b86c4882fe92 P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:37.333047 16219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 27 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:37.355948 16295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806d7690c2b644959152074bb2cab0a7" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
W20260709 13:22:37.334403 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 68632.7us
I20260709 13:22:37.342620 16817 raft_consensus.cc:1275] T ada5a004ba7c46819051e3ccec71ffdb P cb82cc7bd6db46d980c75237a1402860 [term 25 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 24 index: 17. Preceding OpId from leader: term: 25 index: 18. (index mismatch)
I20260709 13:22:37.356670 16295 raft_consensus.cc:3060] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045 [term 21 FOLLOWER]: Advancing to term 23
I20260709 13:22:37.340766 16299 raft_consensus.cc:1240] T 5a959dbecc2648eb85d1cc550f5a3245 P 90f3da57796b488c9087a2f523adf045 [term 23 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 22. Current term is 23. Ops: []
I20260709 13:22:37.355645 16294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68a6838d82ef4cac83507f906139d90f" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 23 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:37.346256 16222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fffab26427e24d398e2695aad40ca134" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 23 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:37.361459 23719 leader_election.cc:290] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 23 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:37.355288 24492 raft_consensus.cc:515] T 26354361e9b044cea1c2b86c4882fe92 P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:37.363772 24280 leader_election.cc:290] T a378d7f796f94615b3b347119a9880fd P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:37.365541 16219 raft_consensus.cc:2393] T ecc1e363230043fe9fef390f0b979f20 P 1ca2725d290e47d68b1a4b609c9816ac [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 27: Already voted for candidate cb82cc7bd6db46d980c75237a1402860 in this term.
W20260709 13:22:37.371708 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 293523us
I20260709 13:22:37.377079 16222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18ad7e3af5504571a69d10c00750a01c" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:37.381896 16295 raft_consensus.cc:2468] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 23.
W20260709 13:22:37.383038 16191 leader_election.cc:336] T 5a96373bb0e14dce8a266b2fc6c58507 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:22:37.309695 24418 raft_consensus.cc:493] T c4f65cdf00b748bb963ee6e81ef43b79 P 90f3da57796b488c9087a2f523adf045 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:22:37.332435 24383 raft_consensus.cc:2804] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 647a343a542d4a139fe106c19095129b [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:22:37.387039 16222 raft_consensus.cc:1217] T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac [term 25 FOLLOWER]: Deduplicated request from leader. Original: 20.11->[22.12-25.14]   Dedup: 22.12->[23.13-25.14]
I20260709 13:22:37.330821 24120 raft_consensus.cc:493] T ee9ed67006144797b1de6ea8c2013a68 P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:22:37.338434 24397 raft_consensus.cc:493] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:37.344293 24456 raft_consensus.cc:2749] T 746aabf856ab401c9ef5e859371e7b67 P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:22:37.389158 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdbd78acb4cf46298e0de8ba0d130e19" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 27 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:37.394850 16298 raft_consensus.cc:2468] T cdbd78acb4cf46298e0de8ba0d130e19 P 90f3da57796b488c9087a2f523adf045 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 26.
I20260709 13:22:37.396575 16298 raft_consensus.cc:1275] T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045 [term 26 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 25 index: 17. Preceding OpId from leader: term: 26 index: 18. (index mismatch)
I20260709 13:22:37.389408 16219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68a6838d82ef4cac83507f906139d90f" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 23 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:37.392468 16222 pending_rounds.cc:85] T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac: Aborting all ops after (but not including) 12
I20260709 13:22:37.393227 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26354361e9b044cea1c2b86c4882fe92" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:37.353955 24176 raft_consensus.cc:2804] T 5a959dbecc2648eb85d1cc550f5a3245 P 647a343a542d4a139fe106c19095129b [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:22:37.393816 16816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62fe3346cd3c42c4a55e14af9073bca4" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 28 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:37.402573 16817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada5a004ba7c46819051e3ccec71ffdb" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:37.403012 16298 raft_consensus.cc:1275] T 8638ee363273426db1393dd81e5c7b72 P 90f3da57796b488c9087a2f523adf045 [term 24 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 20 index: 16. Preceding OpId from leader: term: 24 index: 17. (index mismatch)
I20260709 13:22:37.403576 16817 raft_consensus.cc:2393] T ada5a004ba7c46819051e3ccec71ffdb P cb82cc7bd6db46d980c75237a1402860 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 25: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
I20260709 13:22:37.400276 16222 pending_rounds.cc:107] T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac: Aborting uncommitted NO_OP operation due to leader change: 24.13
I20260709 13:22:37.406297 16222 raft_consensus.cc:2889] T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac [term 25 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:22:37.413050 16298 raft_consensus.cc:3060] T ada5a004ba7c46819051e3ccec71ffdb P 90f3da57796b488c9087a2f523adf045 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:22:37.416694 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806d7690c2b644959152074bb2cab0a7" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:37.418001 16819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806d7690c2b644959152074bb2cab0a7" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:37.418983 16819 raft_consensus.cc:2468] T 806d7690c2b644959152074bb2cab0a7 P cb82cc7bd6db46d980c75237a1402860 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 21.
I20260709 13:22:37.423666 16300 raft_consensus.cc:2368] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 for earlier term 22. Current term is 23.
W20260709 13:22:37.401051 24200 consensus_meta.cc:287] T 8158bbd17b994d2eb96f560bd0f1b88d P 647a343a542d4a139fe106c19095129b: Time spent flushing consensus metadata: real 0.803s	user 0.017s	sys 0.001s
I20260709 13:22:37.384286 24276 raft_consensus.cc:493] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:22:37.387887 24492 leader_election.cc:290] T 26354361e9b044cea1c2b86c4882fe92 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:37.393879 24383 raft_consensus.cc:493] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 647a343a542d4a139fe106c19095129b [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:37.424558 24176 raft_consensus.cc:697] T 5a959dbecc2648eb85d1cc550f5a3245 P 647a343a542d4a139fe106c19095129b [term 23 LEADER]: Becoming Leader. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:22:37.409356 24120 raft_consensus.cc:515] T ee9ed67006144797b1de6ea8c2013a68 P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:37.409507 24397 raft_consensus.cc:515] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:37.441135 24383 raft_consensus.cc:3060] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 647a343a542d4a139fe106c19095129b [term 22 FOLLOWER]: Advancing to term 23
W20260709 13:22:37.442138 24362 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.946s	user 0.011s	sys 0.025s
I20260709 13:22:37.402190 24418 raft_consensus.cc:515] T c4f65cdf00b748bb963ee6e81ef43b79 P 90f3da57796b488c9087a2f523adf045 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:37.460364 24459 leader_election.cc:290] T 7c2095a5f4ad49408dbc7f69c606120d P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:37.461010 24397 leader_election.cc:290] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:37.465894 24418 leader_election.cc:290] T c4f65cdf00b748bb963ee6e81ef43b79 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:22:37.498279 16040 catalog_manager.cc:5697] T d2ab0ac8347641e6955e9bce7d90c1a2 P cb82cc7bd6db46d980c75237a1402860 reported cstate change: term changed from 14 to 20, leader changed from cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) to 90f3da57796b488c9087a2f523adf045 (127.14.177.131). New cstate: current_term: 20 leader_uuid: "90f3da57796b488c9087a2f523adf045" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
W20260709 13:22:37.501659 16048 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:49250 (request call id 324) took 40659 ms (client timeout 15000 ms). Trace:
W20260709 13:22:37.502561 16048 rpcz_store.cc:269] 0709 13:21:56.842284 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:56.909366 (+ 67082us) service_pool.cc:224] Handling call
0709 13:22:37.501191 (+40591825us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:22:28.601496 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:22:28.602216 (+   720us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:22:28.602267 (+    51us) write_op.cc:435] Acquired schema lock
0709 13:22:28.602285 (+    18us) tablet.cc:662] Decoding operations
0709 13:22:28.603410 (+  1125us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:22:28.603516 (+   106us) write_op.cc:641] Partition lock acquired for write op
0709 13:22:28.603545 (+    29us) tablet.cc:685] Acquiring locks for 10 operations
0709 13:22:28.604422 (+   877us) tablet.cc:701] Row locks acquired
0709 13:22:28.604472 (+    50us) write_op.cc:260] PREPARE: finished
0709 13:22:28.604636 (+   164us) write_op.cc:270] Start()
0709 13:22:28.604783 (+   147us) write_op.cc:276] Timestamp: P: 1783603348604575 usec, L: 0
0709 13:22:28.604832 (+    49us) op_driver.cc:348] REPLICATION: starting
0709 13:22:28.605478 (+   646us) log.cc:844] Serialized 4049 byte log entry
0709 13:22:28.902325 (+296847us) spinlock_profiling.cc:243] Waited 24.9 ms on lock 0x7b4400022b4c. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69ad8fadb1 00007f69ad998c85 00007f69ad9996a2 00007f69b070b1db 00007f69b0706aaf 00007f69b06fa86e 00007f69b06f9bb6 00007f69b06f97d8 00007f69b0703221 00007f69b07031d9 00007f69b0703169
0709 13:22:29.307463 (+405138us) op_driver.cc:464] REPLICATION: finished
0709 13:22:29.768424 (+460961us) write_op.cc:301] APPLY: starting
0709 13:22:29.768581 (+   157us) tablet.cc:1367] starting BulkCheckPresence
0709 13:22:29.956994 (+188413us) tablet.cc:1370] finished BulkCheckPresence
0709 13:22:29.957038 (+    44us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:22:30.515994 (+558956us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:22:30.516425 (+   431us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=20,key_file_lookups=20,delta_file_lookups=0,mrs_lookups=0
0709 13:22:30.516513 (+    88us) write_op.cc:312] APPLY: finished
0709 13:22:30.516989 (+   476us) log.cc:844] Serialized 97 byte log entry
0709 13:22:30.885746 (+368757us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:22:30.885885 (+   139us) write_op.cc:454] Released schema lock
0709 13:22:30.885958 (+    73us) write_op.cc:341] FINISH: Updating metrics
0709 13:22:30.886414 (+   456us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:22:30.886448 (+    34us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":1424470,"reported_tablets":48,"child_traces":[["op",{"apply.queue_time_us":400192,"apply.run_cpu_time_us":57998,"apply.run_wall_time_us":1139731,"cfile_cache_hit":42,"cfile_cache_hit_bytes":45887,"num_ops":10,"prepare.queue_time_us":165840,"prepare.run_cpu_time_us":6964,"prepare.run_wall_time_us":115811,"raft.queue_time_us":258092,"raft.run_cpu_time_us":47818,"raft.run_wall_time_us":299525,"replication_time_us":702452,"spinlock_wait_cycles":57169920,"thread_start_us":919282,"threads_started":5,"wal-append.queue_time_us":455896,"wal-append.run_cpu_time_us":21355,"wal-append.run_wall_time_us":2490975}]]}
I20260709 13:22:37.504174 16040 catalog_manager.cc:5697] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 reported cstate change: term changed from 21 to 24. New cstate: current_term: 24 leader_uuid: "cb82cc7bd6db46d980c75237a1402860" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:22:37.520352 16040 catalog_manager.cc:5697] T 3884f864a96d428785895e5463e5728f P cb82cc7bd6db46d980c75237a1402860 reported cstate change: term changed from 13 to 17, leader changed from 647a343a542d4a139fe106c19095129b (127.14.177.129) to cb82cc7bd6db46d980c75237a1402860 (127.14.177.132). New cstate: current_term: 17 leader_uuid: "cb82cc7bd6db46d980c75237a1402860" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:22:37.523990 16040 catalog_manager.cc:5697] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 reported cstate change: term changed from 21 to 25. New cstate: current_term: 25 leader_uuid: "1ca2725d290e47d68b1a4b609c9816ac" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
W20260709 13:22:37.539781 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 201004us
I20260709 13:22:37.542361 16040 catalog_manager.cc:5697] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860 reported cstate change: term changed from 19 to 23, leader changed from cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) to 647a343a542d4a139fe106c19095129b (127.14.177.129). New cstate: current_term: 23 leader_uuid: "647a343a542d4a139fe106c19095129b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:22:37.545070 16040 catalog_manager.cc:5697] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860 reported cstate change: term changed from 15 to 18, leader changed from 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130) to cb82cc7bd6db46d980c75237a1402860 (127.14.177.132). New cstate: current_term: 18 leader_uuid: "cb82cc7bd6db46d980c75237a1402860" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:22:37.547909 16116 leader_election.cc:304] T fffab26427e24d398e2695aad40ca134 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:22:37.548233 16040 catalog_manager.cc:5697] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 reported cstate change: term changed from 24 to 26, leader changed from 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130) to cb82cc7bd6db46d980c75237a1402860 (127.14.177.132). New cstate: current_term: 26 leader_uuid: "cb82cc7bd6db46d980c75237a1402860" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:22:37.550199 16116 leader_election.cc:304] T 18ad7e3af5504571a69d10c00750a01c P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:22:37.551926 16040 catalog_manager.cc:5697] T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860 reported cstate change: term changed from 20 to 23, leader changed from cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) to 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130). New cstate: current_term: 23 leader_uuid: "1ca2725d290e47d68b1a4b609c9816ac" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:22:37.561602 16040 catalog_manager.cc:5697] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860 reported cstate change: term changed from 12 to 18. New cstate: current_term: 18 leader_uuid: "647a343a542d4a139fe106c19095129b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
W20260709 13:22:37.390901 16189 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:22:37.565153 16190 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:22:37.390710 16191 leader_election.cc:304] T 5a96373bb0e14dce8a266b2fc6c58507 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
W20260709 13:22:37.569774 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 186656us
W20260709 13:22:37.576481 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 414496us
W20260709 13:22:37.578266 16263 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:22:37.595052 23958 raft_consensus.cc:2749] T 5a96373bb0e14dce8a266b2fc6c58507 P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:22:37.595975 16817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 27 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:37.598502 16825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434bff3082d141838068f323bdb0d1ed" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:37.596441 24120 leader_election.cc:290] T ee9ed67006144797b1de6ea8c2013a68 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:37.602272 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c2095a5f4ad49408dbc7f69c606120d" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 20 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:37.606621 24125 raft_consensus.cc:493] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:22:37.607407 24125 raft_consensus.cc:515] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:22:37.611456 15046 thread.cc:527] Waited for 9000ms trying to join with heartbeat (tid 16176)
I20260709 13:22:37.594779 23719 raft_consensus.cc:493] T af57996a8be84d398f0f023e8d08ef77 P 647a343a542d4a139fe106c19095129b [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:37.612710 24141 raft_consensus.cc:2749] T fffab26427e24d398e2695aad40ca134 P 647a343a542d4a139fe106c19095129b [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:22:37.612531 23719 raft_consensus.cc:515] T af57996a8be84d398f0f023e8d08ef77 P 647a343a542d4a139fe106c19095129b [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:37.615898 16816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806d7690c2b644959152074bb2cab0a7" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:22:37.616832 24141 raft_consensus.cc:2749] T 18ad7e3af5504571a69d10c00750a01c P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:22:37.618232 16824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26354361e9b044cea1c2b86c4882fe92" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:37.620745 24176 consensus_queue.cc:237] T 5a959dbecc2648eb85d1cc550f5a3245 P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.13, Last appended by leader: 13, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:37.623872 16190 leader_election.cc:304] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: cb82cc7bd6db46d980c75237a1402860
W20260709 13:22:37.626289 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 51112.8us
I20260709 13:22:37.617000 16816 raft_consensus.cc:3060] T 806d7690c2b644959152074bb2cab0a7 P cb82cc7bd6db46d980c75237a1402860 [term 21 FOLLOWER]: Advancing to term 23
I20260709 13:22:37.631585 23958 raft_consensus.cc:2804] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [term 26 FOLLOWER]: Leader election won for term 26
I20260709 13:22:37.441848 24276 raft_consensus.cc:515] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:37.658668 24276 leader_election.cc:290] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:37.659217 23958 raft_consensus.cc:697] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [term 26 LEADER]: Becoming Leader. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:22:37.660234 16819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66f75124215f4ca6bef831e174a792d8" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
W20260709 13:22:37.661731 16192 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 70491.9us
W20260709 13:22:37.451237 24362 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.958s	user 0.012s	sys 0.025s
I20260709 13:22:37.661919 23958 consensus_queue.cc:237] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 25.19, Last appended by leader: 19, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:37.663264 24427 consensus_queue.cc:1048] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.001s
I20260709 13:22:37.664753 16223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806d7690c2b644959152074bb2cab0a7" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:37.665966 16223 raft_consensus.cc:2393] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 in current term 22: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
I20260709 13:22:37.666281 24271 raft_consensus.cc:493] T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:22:37.667199 24271 raft_consensus.cc:515] T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:37.667945 24304 raft_consensus.cc:493] T 57a744f5aaf944678a63469cbf514015 P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:22:37.690816 16818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87147731a68400e8cce9b879daa7c5d" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 23 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:37.705976 24420 raft_consensus.cc:493] T ee9ed67006144797b1de6ea8c2013a68 P cb82cc7bd6db46d980c75237a1402860 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:22:37.709686 24125 leader_election.cc:290] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:22:37.713085 24304 raft_consensus.cc:515] T 57a744f5aaf944678a63469cbf514015 P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:37.715766 16818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31969392ccdb43b491911a319e8bc7e1" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:37.727187 16218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdbd78acb4cf46298e0de8ba0d130e19" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 27 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:37.734529 24488 raft_consensus.cc:493] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:22:37.760430 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 210348us
I20260709 13:22:37.762878 16116 leader_election.cc:304] T ecc1e363230043fe9fef390f0b979f20 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:22:37.717995 24484 raft_consensus.cc:493] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:37.771189 16824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a378d7f796f94615b3b347119a9880fd" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
W20260709 13:22:37.773459 16192 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 52709.4us
I20260709 13:22:37.780726 16789 leader_election.cc:304] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:22:37.782685 24495 raft_consensus.cc:2804] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 13:22:37.783252 24495 raft_consensus.cc:493] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:37.783753 24495 raft_consensus.cc:3060] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 [term 26 FOLLOWER]: Advancing to term 27
I20260709 13:22:37.783421 24490 consensus_queue.cc:1048] T 74ca8140f35c4ae29e8a664f6bdfc829 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.001s
I20260709 13:22:37.785225 24456 consensus_queue.cc:1059] T 5a959dbecc2648eb85d1cc550f5a3245 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:22:37.791518 24304 leader_election.cc:290] T 57a744f5aaf944678a63469cbf514015 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:37.792807 24488 raft_consensus.cc:515] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:37.795406 24484 raft_consensus.cc:515] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:37.798367 24484 leader_election.cc:290] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:37.799072 16818 raft_consensus.cc:2393] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 21: Already voted for candidate cb82cc7bd6db46d980c75237a1402860 in this term.
W20260709 13:22:37.774801 16191 leader_election.cc:336] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:22:37.806213 16191 leader_election.cc:304] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:22:37.808332 24456 raft_consensus.cc:3055] T 5a959dbecc2648eb85d1cc550f5a3245 P 1ca2725d290e47d68b1a4b609c9816ac [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:22:37.808882 24456 raft_consensus.cc:740] T 5a959dbecc2648eb85d1cc550f5a3245 P 1ca2725d290e47d68b1a4b609c9816ac [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:22:37.810073 24456 consensus_queue.cc:260] T 5a959dbecc2648eb85d1cc550f5a3245 P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 22.13, Last appended by leader: 13, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:37.812424 24276 consensus_queue.cc:1048] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.002s
I20260709 13:22:37.815526 24459 raft_consensus.cc:493] T 8158bbd17b994d2eb96f560bd0f1b88d P 647a343a542d4a139fe106c19095129b [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:37.816380 24459 raft_consensus.cc:515] T 8158bbd17b994d2eb96f560bd0f1b88d P 647a343a542d4a139fe106c19095129b [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:22:37.818787 16189 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:22:37.819587 16115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 77967us
I20260709 13:22:37.820045 23958 raft_consensus.cc:493] T e4dc6b0541ca431ebc757beac00521f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:22:37.820996 23958 raft_consensus.cc:515] T e4dc6b0541ca431ebc757beac00521f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:37.827665 24463 raft_consensus.cc:493] T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:37.828557 24463 raft_consensus.cc:515] T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:37.830792 24125 raft_consensus.cc:493] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
W20260709 13:22:37.831146 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 68533.3us
I20260709 13:22:37.833331 16116 leader_election.cc:304] T 68a6838d82ef4cac83507f906139d90f P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
W20260709 13:22:37.831350 16790 consensus_peers.cc:597] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:22:37.835846 16790 consensus_peers.cc:597] T 23f82224f4564fea93572e8206bad1fe P cb82cc7bd6db46d980c75237a1402860 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:22:37.836059 24200 raft_consensus.cc:2749] T 68a6838d82ef4cac83507f906139d90f P 647a343a542d4a139fe106c19095129b [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:22:37.837754 24455 raft_consensus.cc:2749] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:22:37.836622 16823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee9ed67006144797b1de6ea8c2013a68" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:37.775861 24420 raft_consensus.cc:515] T ee9ed67006144797b1de6ea8c2013a68 P cb82cc7bd6db46d980c75237a1402860 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:37.831455 24141 raft_consensus.cc:3060] T ecc1e363230043fe9fef390f0b979f20 P 647a343a542d4a139fe106c19095129b [term 26 FOLLOWER]: Advancing to term 27
I20260709 13:22:37.841008 24420 leader_election.cc:290] T ee9ed67006144797b1de6ea8c2013a68 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
W20260709 13:22:37.836462 16790 leader_election.cc:336] T 806d7690c2b644959152074bb2cab0a7 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:22:37.842285 16218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "829b4ad8c5a54eababeea1181b960d4b" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:37.848554 16789 leader_election.cc:400] T 806d7690c2b644959152074bb2cab0a7 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 22 pre-election: Vote denied by peer 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389) with higher term. Message: Invalid argument: T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 for earlier term 22. Current term is 23.
I20260709 13:22:37.848974 16789 leader_election.cc:403] T 806d7690c2b644959152074bb2cab0a7 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 22 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:22:37.861027 16294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8158bbd17b994d2eb96f560bd0f1b88d" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:37.862212 16294 raft_consensus.cc:2393] T 8158bbd17b994d2eb96f560bd0f1b88d P 90f3da57796b488c9087a2f523adf045 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 22: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
I20260709 13:22:37.867336 23719 leader_election.cc:290] T af57996a8be84d398f0f023e8d08ef77 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:37.870111 16298 raft_consensus.cc:1275] T ada5a004ba7c46819051e3ccec71ffdb P 90f3da57796b488c9087a2f523adf045 [term 25 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 24 index: 17. Preceding OpId from leader: term: 25 index: 18. (index mismatch)
I20260709 13:22:37.872260 16221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada5a004ba7c46819051e3ccec71ffdb" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:37.882652 24471 consensus_queue.cc:1048] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.001s
I20260709 13:22:37.894053 24456 raft_consensus.cc:3060] T 5a959dbecc2648eb85d1cc550f5a3245 P 1ca2725d290e47d68b1a4b609c9816ac [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:22:37.911384 24271 leader_election.cc:290] T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:22:37.911715 16297 raft_consensus.cc:1275] T 84f2da98864b4425bf7c0055951fdd4b P 90f3da57796b488c9087a2f523adf045 [term 21 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260709 13:22:37.912073 24495 raft_consensus.cc:515] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 [term 27 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:37.913879 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 26 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:37.915939 24463 leader_election.cc:290] T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:37.916824 23958 leader_election.cc:290] T e4dc6b0541ca431ebc757beac00521f1 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:37.831571 24125 raft_consensus.cc:515] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:37.910912 24313 raft_consensus.cc:493] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:22:37.920001 24125 leader_election.cc:290] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:22:37.919742 24313 raft_consensus.cc:515] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:37.922971 24459 leader_election.cc:290] T 8158bbd17b994d2eb96f560bd0f1b88d P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:37.925264 24488 leader_election.cc:290] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:37.925938 16218 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806d7690c2b644959152074bb2cab0a7" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:22:37.926631 16218 raft_consensus.cc:3060] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:22:37.923035 24313 leader_election.cc:290] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:37.930246 16823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5d59407eb545b895d8d922e0dd69fd" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
W20260709 13:22:37.934154 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 159504us
I20260709 13:22:37.941505 16824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57a744f5aaf944678a63469cbf514015" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:37.946265 16816 raft_consensus.cc:2468] T 806d7690c2b644959152074bb2cab0a7 P cb82cc7bd6db46d980c75237a1402860 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 23.
I20260709 13:22:37.948935 16825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a9de347c711477e87c582d003f36925" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 27 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
W20260709 13:22:37.925292 16790 leader_election.cc:336] T 829b4ad8c5a54eababeea1181b960d4b P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:22:37.955066 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31969392ccdb43b491911a319e8bc7e1" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:37.955972 16298 raft_consensus.cc:2468] T 31969392ccdb43b491911a319e8bc7e1 P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 20.
I20260709 13:22:37.956458 16822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af57996a8be84d398f0f023e8d08ef77" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:37.957895 24493 raft_consensus.cc:2749] T 806d7690c2b644959152074bb2cab0a7 P cb82cc7bd6db46d980c75237a1402860 [term 23 FOLLOWER]: Leader pre-election lost for term 22. Reason: Vote denied by peer 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389) with higher term. Message: Invalid argument: T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 for earlier term 22. Current term is 23.
I20260709 13:22:37.956530 24463 raft_consensus.cc:493] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:37.959904 24463 raft_consensus.cc:515] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:37.960708 16222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c2095a5f4ad49408dbc7f69c606120d" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 20 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:37.956601 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806d7690c2b644959152074bb2cab0a7" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
W20260709 13:22:37.962693 16191 leader_election.cc:336] T 8cc959fb7f704fba9009c0595467d074 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:22:37.962911 24463 leader_election.cc:290] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:37.967643 24227 raft_consensus.cc:493] T e5ad4eb281564936a59f41e387a47156 P 647a343a542d4a139fe106c19095129b [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:37.967789 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a378d7f796f94615b3b347119a9880fd" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:37.968240 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5d59407eb545b895d8d922e0dd69fd" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:37.971800 16220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af57996a8be84d398f0f023e8d08ef77" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:37.983991 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 26 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:37.984813 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5718c623ebee4d2bb770f0f9ddb2f164" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 24 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:37.985668 16226 raft_consensus.cc:2393] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 24: Already voted for candidate cb82cc7bd6db46d980c75237a1402860 in this term.
I20260709 13:22:37.974100 24495 leader_election.cc:290] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 27 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
W20260709 13:22:37.965013 16117 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 46256.9us
I20260709 13:22:37.980119 16116 leader_election.cc:304] T 7c2095a5f4ad49408dbc7f69c606120d P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:22:37.983033 24227 raft_consensus.cc:515] T e5ad4eb281564936a59f41e387a47156 P 647a343a542d4a139fe106c19095129b [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:37.968392 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6f07842d3aa47858df5665874c86f3a" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:37.988775 16224 raft_consensus.cc:2393] T b6f07842d3aa47858df5665874c86f3a P 1ca2725d290e47d68b1a4b609c9816ac [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 23: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
I20260709 13:22:37.989331 24488 raft_consensus.cc:2749] T 7c2095a5f4ad49408dbc7f69c606120d P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:22:37.963275 16191 leader_election.cc:304] T 8cc959fb7f704fba9009c0595467d074 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:22:37.972716 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8158bbd17b994d2eb96f560bd0f1b88d" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:37.995070 16225 raft_consensus.cc:2468] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 21.
I20260709 13:22:37.996960 16117 leader_election.cc:304] T 434bff3082d141838068f323bdb0d1ed P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860
W20260709 13:22:37.997710 16266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 63106.2us
I20260709 13:22:37.999481 24199 raft_consensus.cc:2749] T 434bff3082d141838068f323bdb0d1ed P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:22:38.000484 24383 consensus_meta.cc:287] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 647a343a542d4a139fe106c19095129b: Time spent flushing consensus metadata: real 0.546s	user 0.010s	sys 0.008s
I20260709 13:22:38.000970 23719 raft_consensus.cc:493] T b6f07842d3aa47858df5665874c86f3a P 647a343a542d4a139fe106c19095129b [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:37.998747 24227 leader_election.cc:290] T e5ad4eb281564936a59f41e387a47156 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:38.001717 23719 raft_consensus.cc:515] T b6f07842d3aa47858df5665874c86f3a P 647a343a542d4a139fe106c19095129b [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:38.001070 24383 raft_consensus.cc:515] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 647a343a542d4a139fe106c19095129b [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:38.004189 23719 leader_election.cc:290] T b6f07842d3aa47858df5665874c86f3a P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:38.004940 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8158bbd17b994d2eb96f560bd0f1b88d" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:37.975831 16265 leader_election.cc:304] T ada5a004ba7c46819051e3ccec71ffdb P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860
I20260709 13:22:38.009768 16299 raft_consensus.cc:2468] T 8158bbd17b994d2eb96f560bd0f1b88d P 90f3da57796b488c9087a2f523adf045 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 22.
I20260709 13:22:38.011008 16294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31969392ccdb43b491911a319e8bc7e1" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:37.975140 16222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4f65cdf00b748bb963ee6e81ef43b79" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:38.011281 24314 raft_consensus.cc:2749] T ada5a004ba7c46819051e3ccec71ffdb P 90f3da57796b488c9087a2f523adf045 [term 25 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:22:38.012485 16294 raft_consensus.cc:2468] T 31969392ccdb43b491911a319e8bc7e1 P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 20.
I20260709 13:22:38.015551 16116 leader_election.cc:304] T 8158bbd17b994d2eb96f560bd0f1b88d P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
I20260709 13:22:38.016079 16222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5ad4eb281564936a59f41e387a47156" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 23 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
W20260709 13:22:38.017006 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 82207.6us
I20260709 13:22:38.017257 23719 raft_consensus.cc:2804] T 8158bbd17b994d2eb96f560bd0f1b88d P 647a343a542d4a139fe106c19095129b [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:22:38.017671 23719 raft_consensus.cc:493] T 8158bbd17b994d2eb96f560bd0f1b88d P 647a343a542d4a139fe106c19095129b [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:38.018021 23719 raft_consensus.cc:3060] T 8158bbd17b994d2eb96f560bd0f1b88d P 647a343a542d4a139fe106c19095129b [term 22 FOLLOWER]: Advancing to term 23
W20260709 13:22:38.018410 16790 leader_election.cc:336] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:22:38.014281 24141 raft_consensus.cc:2749] T ecc1e363230043fe9fef390f0b979f20 P 647a343a542d4a139fe106c19095129b [term 27 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 13:22:38.023989 16222 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6f07842d3aa47858df5665874c86f3a" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:38.024717 16222 raft_consensus.cc:2468] T b6f07842d3aa47858df5665874c86f3a P 1ca2725d290e47d68b1a4b609c9816ac [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 23.
I20260709 13:22:38.030331 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4dc6b0541ca431ebc757beac00521f1" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
W20260709 13:22:38.032718 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 70150.1us
W20260709 13:22:38.033674 16191 leader_election.cc:336] T 7c2095a5f4ad49408dbc7f69c606120d P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:22:38.034222 16191 leader_election.cc:304] T 7c2095a5f4ad49408dbc7f69c606120d P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:22:38.034364 16115 leader_election.cc:304] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:22:38.035252 24459 raft_consensus.cc:2749] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:22:38.035712 24496 raft_consensus.cc:2749] T 8cc959fb7f704fba9009c0595467d074 P 1ca2725d290e47d68b1a4b609c9816ac [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:22:38.035769 24415 raft_consensus.cc:2749] T 7c2095a5f4ad49408dbc7f69c606120d P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:22:38.034638 24141 consensus_queue.cc:1048] T 84f2da98864b4425bf7c0055951fdd4b P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 9, Time since last communication: 0.003s
I20260709 13:22:38.039624 16295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5ad4eb281564936a59f41e387a47156" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 23 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:38.039841 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6f07842d3aa47858df5665874c86f3a" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:38.040747 16298 raft_consensus.cc:2468] T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 22.
I20260709 13:22:38.041179 16789 leader_election.cc:304] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 
W20260709 13:22:38.041513 16191 leader_election.cc:336] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:22:38.042064 24490 raft_consensus.cc:2804] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:22:38.042531 24490 raft_consensus.cc:493] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:38.043028 24490 raft_consensus.cc:3060] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860 [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:22:38.047228 16117 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 48389us
W20260709 13:22:38.048596 16192 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 66242us
I20260709 13:22:38.048900 16117 leader_election.cc:304] T 26354361e9b044cea1c2b86c4882fe92 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860
I20260709 13:22:38.050207 24488 raft_consensus.cc:2749] T 26354361e9b044cea1c2b86c4882fe92 P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:22:38.005263 24383 leader_election.cc:290] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 23 election: Requested vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:38.053473 16116 leader_election.cc:304] T b6f07842d3aa47858df5665874c86f3a P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
I20260709 13:22:38.042034 16191 leader_election.cc:304] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
I20260709 13:22:38.081230 16115 leader_election.cc:304] T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:22:38.081550 16218 raft_consensus.cc:2468] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 23.
I20260709 13:22:38.082187 23719 raft_consensus.cc:515] T 8158bbd17b994d2eb96f560bd0f1b88d P 647a343a542d4a139fe106c19095129b [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:38.084300 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdbd78acb4cf46298e0de8ba0d130e19" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 27 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:22:38.083793 16115 leader_election.cc:304] T e5ad4eb281564936a59f41e387a47156 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:22:38.050323 16117 leader_election.cc:304] T b87147731a68400e8cce9b879daa7c5d P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860
I20260709 13:22:38.088229 16265 leader_election.cc:304] T a378d7f796f94615b3b347119a9880fd P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860
I20260709 13:22:38.100675 24488 raft_consensus.cc:2749] T e5ad4eb281564936a59f41e387a47156 P 647a343a542d4a139fe106c19095129b [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:22:38.104230 23719 leader_election.cc:290] T 8158bbd17b994d2eb96f560bd0f1b88d P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 23 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
W20260709 13:22:38.105998 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 64577us
W20260709 13:22:38.107375 16191 leader_election.cc:336] T 66f75124215f4ca6bef831e174a792d8 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:22:38.107784 16191 leader_election.cc:304] T 66f75124215f4ca6bef831e174a792d8 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
W20260709 13:22:38.109198 16191 leader_election.cc:336] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:22:38.109630 16191 leader_election.cc:304] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
I20260709 13:22:38.110591 16298 raft_consensus.cc:3060] T cdbd78acb4cf46298e0de8ba0d130e19 P 90f3da57796b488c9087a2f523adf045 [term 26 FOLLOWER]: Advancing to term 27
I20260709 13:22:38.116149 16292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e9b2bf7011f4ae688a79d7bb5a447b4" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
W20260709 13:22:38.128099 16790 leader_election.cc:336] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:22:38.124991 24271 raft_consensus.cc:2749] T a378d7f796f94615b3b347119a9880fd P 90f3da57796b488c9087a2f523adf045 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:22:38.115500 24120 raft_consensus.cc:2749] T 66f75124215f4ca6bef831e174a792d8 P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:22:38.133481 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8158bbd17b994d2eb96f560bd0f1b88d" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:22:38.134104 16224 raft_consensus.cc:3060] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Advancing to term 23
I20260709 13:22:38.094453 24459 raft_consensus.cc:2804] T b6f07842d3aa47858df5665874c86f3a P 647a343a542d4a139fe106c19095129b [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:22:38.096312 24141 raft_consensus.cc:2804] T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:22:38.111086 24409 raft_consensus.cc:2749] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 13:22:38.147266 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 86687.8us
I20260709 13:22:38.148238 24492 raft_consensus.cc:2749] T b87147731a68400e8cce9b879daa7c5d P 647a343a542d4a139fe106c19095129b [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:22:38.149544 16292 raft_consensus.cc:3060] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 90f3da57796b488c9087a2f523adf045 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:22:38.118352 24496 raft_consensus.cc:2749] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
W20260709 13:22:38.150732 16790 leader_election.cc:336] T ee9ed67006144797b1de6ea8c2013a68 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:22:38.114576 24190 raft_consensus.cc:493] T 57a744f5aaf944678a63469cbf514015 P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:38.141134 24459 raft_consensus.cc:493] T b6f07842d3aa47858df5665874c86f3a P 647a343a542d4a139fe106c19095129b [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:38.146349 24141 raft_consensus.cc:493] T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:22:38.171914 16790 leader_election.cc:336] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:22:38.163766 24459 raft_consensus.cc:3060] T b6f07842d3aa47858df5665874c86f3a P 647a343a542d4a139fe106c19095129b [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:22:38.198151 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8158bbd17b994d2eb96f560bd0f1b88d" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:22:38.199092 16293 raft_consensus.cc:3060] T 8158bbd17b994d2eb96f560bd0f1b88d P 90f3da57796b488c9087a2f523adf045 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:22:38.205565 16116 leader_election.cc:304] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:22:38.207450 24488 raft_consensus.cc:2804] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:22:38.167567 24141 raft_consensus.cc:3060] T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:22:38.164599 24492 raft_consensus.cc:493] T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:38.213550 24488 raft_consensus.cc:697] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b [term 23 LEADER]: Becoming Leader. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:22:38.223135 16220 raft_consensus.cc:1275] T 5a959dbecc2648eb85d1cc550f5a3245 P 1ca2725d290e47d68b1a4b609c9816ac [term 23 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 22 index: 13. Preceding OpId from leader: term: 21 index: 13. (term mismatch)
I20260709 13:22:38.223578 16220 pending_rounds.cc:85] T 5a959dbecc2648eb85d1cc550f5a3245 P 1ca2725d290e47d68b1a4b609c9816ac: Aborting all ops after (but not including) 12
I20260709 13:22:38.223814 16220 pending_rounds.cc:107] T 5a959dbecc2648eb85d1cc550f5a3245 P 1ca2725d290e47d68b1a4b609c9816ac: Aborting uncommitted NO_OP operation due to leader change: 22.13
I20260709 13:22:38.224093 16220 raft_consensus.cc:2889] T 5a959dbecc2648eb85d1cc550f5a3245 P 1ca2725d290e47d68b1a4b609c9816ac [term 23 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:22:38.224203 24492 raft_consensus.cc:515] T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:38.226806 24492 leader_election.cc:290] T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:38.222640 24488 consensus_queue.cc:237] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.15, Last appended by leader: 15, Current term: 23, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:22:38.208726 16790 consensus_peers.cc:597] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:22:38.243239 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4dc6b0541ca431ebc757beac00521f1" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 25 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
W20260709 13:22:38.263921 16117 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 213655us
I20260709 13:22:38.237995 24120 raft_consensus.cc:493] T c2f5f89186204242bf377f7c42b2e2da P 1ca2725d290e47d68b1a4b609c9816ac [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:22:38.268486 16117 leader_election.cc:304] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860
I20260709 13:22:38.280499 16822 raft_consensus.cc:1240] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 [term 27 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 26. Current term is 27. Ops: []
I20260709 13:22:38.283949 16292 raft_consensus.cc:2468] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 90f3da57796b488c9087a2f523adf045 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 23.
I20260709 13:22:38.218600 24190 raft_consensus.cc:515] T 57a744f5aaf944678a63469cbf514015 P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:38.295358 24281 consensus_queue.cc:1059] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 20, Last known committed idx: 18, Time since last communication: 0.011s
I20260709 13:22:38.296401 24281 raft_consensus.cc:3055] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [term 26 LEADER]: Stepping down as leader of term 26
I20260709 13:22:38.296804 24281 raft_consensus.cc:740] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [term 26 LEADER]: Becoming Follower/Learner. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:22:38.301841 24190 leader_election.cc:290] T 57a744f5aaf944678a63469cbf514015 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:38.302563 16296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4dc6b0541ca431ebc757beac00521f1" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 25 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:38.303117 16298 raft_consensus.cc:2468] T cdbd78acb4cf46298e0de8ba0d130e19 P 90f3da57796b488c9087a2f523adf045 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 27.
I20260709 13:22:38.306203 16296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57a744f5aaf944678a63469cbf514015" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 21 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:38.323513 16224 raft_consensus.cc:2468] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 23.
I20260709 13:22:38.273821 24120 raft_consensus.cc:515] T c2f5f89186204242bf377f7c42b2e2da P 1ca2725d290e47d68b1a4b609c9816ac [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:38.303988 24491 consensus_queue.cc:1048] T 5a959dbecc2648eb85d1cc550f5a3245 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.101s
I20260709 13:22:38.346012 16293 raft_consensus.cc:2468] T 8158bbd17b994d2eb96f560bd0f1b88d P 90f3da57796b488c9087a2f523adf045 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 23.
I20260709 13:22:38.345621 16789 leader_election.cc:304] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:22:38.354943 24420 raft_consensus.cc:2804] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 [term 27 FOLLOWER]: Leader election won for term 27
I20260709 13:22:38.355657 24420 raft_consensus.cc:697] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 [term 27 LEADER]: Becoming Leader. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:22:38.356887 24420 consensus_queue.cc:237] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 25.19, Last appended by leader: 19, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:38.402645 16115 leader_election.cc:304] T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
W20260709 13:22:38.406308 16117 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 137976us
I20260709 13:22:38.407042 16117 leader_election.cc:304] T af57996a8be84d398f0f023e8d08ef77 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860
I20260709 13:22:38.355000 24281 consensus_queue.cc:260] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 26.20, Last appended by leader: 20, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:38.403353 24227 raft_consensus.cc:2749] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:22:38.420498 24490 raft_consensus.cc:515] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:22:38.438179 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 307787us
I20260709 13:22:38.440279 16115 leader_election.cc:304] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:22:38.456176 16295 raft_consensus.cc:1240] T cdbd78acb4cf46298e0de8ba0d130e19 P 90f3da57796b488c9087a2f523adf045 [term 27 FOLLOWER]: Rejecting Update request from peer 1ca2725d290e47d68b1a4b609c9816ac for earlier term 26. Current term is 27. Ops: []
I20260709 13:22:38.447018 24199 raft_consensus.cc:2749] T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:22:38.437297 24281 raft_consensus.cc:3060] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [term 26 FOLLOWER]: Advancing to term 27
I20260709 13:22:38.469964 16116 leader_election.cc:304] T 8158bbd17b994d2eb96f560bd0f1b88d P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b; no voters: 
I20260709 13:22:38.479267 16220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b7d1ab519044eda2d5e04a07242f25" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 24 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:38.480623 24129 raft_consensus.cc:493] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:38.481499 24129 raft_consensus.cc:515] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:38.448537 24190 raft_consensus.cc:2749] T af57996a8be84d398f0f023e8d08ef77 P 647a343a542d4a139fe106c19095129b [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:22:38.520135 16219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee9ed67006144797b1de6ea8c2013a68" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
W20260709 13:22:38.527935 16115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 87734.2us
I20260709 13:22:38.531785 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62fe3346cd3c42c4a55e14af9073bca4" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 28 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:38.537891 24362 raft_consensus.cc:2804] T 8158bbd17b994d2eb96f560bd0f1b88d P 647a343a542d4a139fe106c19095129b [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:22:38.538796 24362 raft_consensus.cc:697] T 8158bbd17b994d2eb96f560bd0f1b88d P 647a343a542d4a139fe106c19095129b [term 23 LEADER]: Becoming Leader. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:22:38.540182 24362 consensus_queue.cc:237] T 8158bbd17b994d2eb96f560bd0f1b88d P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 21.14, Last appended by leader: 14, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:22:38.548154 24521 log.cc:927] Time spent T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860: Append to log took a long time: real 0.180s	user 0.000s	sys 0.001s
I20260709 13:22:38.560753 16221 raft_consensus.cc:1275] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac [term 24 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 22 index: 17. Preceding OpId from leader: term: 24 index: 18. (index mismatch)
W20260709 13:22:38.573081 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 52530.2us
W20260709 13:22:38.578056 24527 log.cc:927] Time spent T ada5a004ba7c46819051e3ccec71ffdb P cb82cc7bd6db46d980c75237a1402860: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
W20260709 13:22:38.595849 24524 log.cc:927] Time spent T 8d4ab870a05347868956aa8b841b2981 P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 0.195s	user 0.000s	sys 0.001s
I20260709 13:22:38.596773 24120 leader_election.cc:290] T c2f5f89186204242bf377f7c42b2e2da P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:38.597980 16292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f5f89186204242bf377f7c42b2e2da" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:38.599625 24129 leader_election.cc:290] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:38.559271 24190 raft_consensus.cc:2804] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 647a343a542d4a139fe106c19095129b [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:22:38.583693 16225 raft_consensus.cc:1275] T 84f2da98864b4425bf7c0055951fdd4b P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
W20260709 13:22:38.611948 15046 thread.cc:527] Waited for 10000ms trying to join with heartbeat (tid 16176)
I20260709 13:22:38.612260 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdbd78acb4cf46298e0de8ba0d130e19" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 27 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:22:38.618238 24458 raft_consensus.cc:493] T 8cc959fb7f704fba9009c0595467d074 P 647a343a542d4a139fe106c19095129b [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:38.618903 24458 raft_consensus.cc:515] T 8cc959fb7f704fba9009c0595467d074 P 647a343a542d4a139fe106c19095129b [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:38.621451 24458 leader_election.cc:290] T 8cc959fb7f704fba9009c0595467d074 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:38.622632 16296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cc959fb7f704fba9009c0595467d074" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 22 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:38.627699 16295 raft_consensus.cc:1275] T 5a959dbecc2648eb85d1cc550f5a3245 P 90f3da57796b488c9087a2f523adf045 [term 23 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 20 index: 12. Preceding OpId from leader: term: 23 index: 14. (index mismatch)
I20260709 13:22:38.609725 24459 raft_consensus.cc:515] T b6f07842d3aa47858df5665874c86f3a P 647a343a542d4a139fe106c19095129b [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:38.633455 24459 leader_election.cc:290] T b6f07842d3aa47858df5665874c86f3a P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 24 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:38.628015 24141 raft_consensus.cc:515] T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:38.638723 16295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6f07842d3aa47858df5665874c86f3a" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:22:38.639606 16295 raft_consensus.cc:3060] T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045 [term 22 FOLLOWER]: Advancing to term 24
W20260709 13:22:38.641108 24518 log.cc:927] Time spent T 19baa7bc367844c288bca8f81439c413 P cb82cc7bd6db46d980c75237a1402860: Append to log took a long time: real 0.319s	user 0.002s	sys 0.001s
W20260709 13:22:38.643270 16291 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.560s	user 0.071s	sys 0.130s
W20260709 13:22:38.643718 16291 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.681s	user 0.071s	sys 0.130s
I20260709 13:22:38.658440 16790 leader_election.cc:304] T 829b4ad8c5a54eababeea1181b960d4b P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:22:38.659595 24498 raft_consensus.cc:2749] T 829b4ad8c5a54eababeea1181b960d4b P cb82cc7bd6db46d980c75237a1402860 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:22:38.662427 16219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cc959fb7f704fba9009c0595467d074" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 22 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:38.666126 16295 raft_consensus.cc:2468] T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 24.
I20260709 13:22:38.670480 16790 leader_election.cc:304] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:22:38.671514 16220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6f07842d3aa47858df5665874c86f3a" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:22:38.672451 16220 raft_consensus.cc:3060] T b6f07842d3aa47858df5665874c86f3a P 1ca2725d290e47d68b1a4b609c9816ac [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:22:38.679029 16790 leader_election.cc:304] T ee9ed67006144797b1de6ea8c2013a68 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:22:38.691975 24490 leader_election.cc:290] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 22 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:38.692955 24490 consensus_queue.cc:1048] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 13:22:38.694497 24484 raft_consensus.cc:2749] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:22:38.696811 24141 leader_election.cc:290] T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 22 election: Requested vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:38.699565 24493 raft_consensus.cc:2749] T ee9ed67006144797b1de6ea8c2013a68 P cb82cc7bd6db46d980c75237a1402860 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:22:38.661144 24190 raft_consensus.cc:697] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 647a343a542d4a139fe106c19095129b [term 23 LEADER]: Becoming Leader. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
W20260709 13:22:38.763602 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 79916.1us
I20260709 13:22:38.747637 16819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5718c623ebee4d2bb770f0f9ddb2f164" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 25 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:38.765705 16790 leader_election.cc:304] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
W20260709 13:22:38.773257 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 99497.5us
I20260709 13:22:38.768977 24125 raft_consensus.cc:2749] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 13:22:38.795794 16226 raft_consensus.cc:2410] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [term 27 FOLLOWER]: Leader election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 for term 27 because replica has last-logged OpId of term: 26 index: 20, which is greater than that of the candidate, which has last-logged OpId of term: 25 index: 19.
W20260709 13:22:38.789543 16115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 115884us
I20260709 13:22:38.782574 24362 consensus_queue.cc:1048] T 84f2da98864b4425bf7c0055951fdd4b P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.041s
I20260709 13:22:38.821832 24199 raft_consensus.cc:493] T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:38.829787 16115 leader_election.cc:304] T 8cc959fb7f704fba9009c0595467d074 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:22:38.789784 24484 raft_consensus.cc:493] T 9e65676b21b748728ad39668cfe722ec P cb82cc7bd6db46d980c75237a1402860 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:22:38.851516 24199 raft_consensus.cc:515] T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:38.854202 24199 leader_election.cc:290] T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:38.861033 24541 raft_consensus.cc:2749] T 8cc959fb7f704fba9009c0595467d074 P 647a343a542d4a139fe106c19095129b [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:22:38.850232 16291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31969392ccdb43b491911a319e8bc7e1" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:22:38.870693 16291 raft_consensus.cc:3060] T 31969392ccdb43b491911a319e8bc7e1 P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Advancing to term 22
I20260709 13:22:38.879559 16115 leader_election.cc:304] T b6f07842d3aa47858df5665874c86f3a P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:22:38.893721 24199 consensus_queue.cc:1048] T 5a959dbecc2648eb85d1cc550f5a3245 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:22:38.896173 24199 raft_consensus.cc:2804] T b6f07842d3aa47858df5665874c86f3a P 647a343a542d4a139fe106c19095129b [term 24 FOLLOWER]: Leader election won for term 24
I20260709 13:22:38.896788 24199 raft_consensus.cc:697] T b6f07842d3aa47858df5665874c86f3a P 647a343a542d4a139fe106c19095129b [term 24 LEADER]: Becoming Leader. State: Replica: 647a343a542d4a139fe106c19095129b, State: Running, Role: LEADER
I20260709 13:22:38.897946 24199 consensus_queue.cc:237] T b6f07842d3aa47858df5665874c86f3a P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 22.14, Last appended by leader: 14, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:22:38.917327 24535 log.cc:927] Time spent T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860: Append to log took a long time: real 0.070s	user 0.000s	sys 0.000s
W20260709 13:22:38.919196 24537 log.cc:927] Time spent T 8158bbd17b994d2eb96f560bd0f1b88d P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 0.094s	user 0.000s	sys 0.001s
I20260709 13:22:38.872586 23958 raft_consensus.cc:493] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:22:38.942005 16292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31969392ccdb43b491911a319e8bc7e1" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
W20260709 13:22:38.944512 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 99975.2us
W20260709 13:22:38.950421 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 103726us
I20260709 13:22:38.873668 24484 raft_consensus.cc:515] T 9e65676b21b748728ad39668cfe722ec P cb82cc7bd6db46d980c75237a1402860 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:38.971181 16824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 26 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:38.977584 16818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31969392ccdb43b491911a319e8bc7e1" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:38.979570 16818 raft_consensus.cc:2393] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 22: Already voted for candidate cb82cc7bd6db46d980c75237a1402860 in this term.
I20260709 13:22:38.934897 24190 consensus_queue.cc:237] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 647a343a542d4a139fe106c19095129b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 21.14, Last appended by leader: 14, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:38.993803 16817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5718c623ebee4d2bb770f0f9ddb2f164" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 24 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
W20260709 13:22:38.997359 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 230035us
I20260709 13:22:38.962503 23958 raft_consensus.cc:515] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:39.011124 16217 raft_consensus.cc:1275] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [term 27 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 26 index: 20. Preceding OpId from leader: term: 27 index: 20. (term mismatch)
I20260709 13:22:39.011613 16217 pending_rounds.cc:85] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac: Aborting all ops after (but not including) 19
I20260709 13:22:39.011914 16217 pending_rounds.cc:107] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac: Aborting uncommitted NO_OP operation due to leader change: 26.20
I20260709 13:22:39.012194 16217 raft_consensus.cc:2889] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [term 27 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:22:39.018409 16817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e9b2bf7011f4ae688a79d7bb5a447b4" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:22:39.025297 23958 leader_election.cc:290] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:39.028384 16819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b7d1ab519044eda2d5e04a07242f25" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 24 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:39.029026 16220 raft_consensus.cc:2468] T b6f07842d3aa47858df5665874c86f3a P 1ca2725d290e47d68b1a4b609c9816ac [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 24.
I20260709 13:22:39.031872 24484 leader_election.cc:290] T 9e65676b21b748728ad39668cfe722ec P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
W20260709 13:22:39.033480 16790 leader_election.cc:336] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:22:39.021225 16115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 54510.4us
I20260709 13:22:39.042531 16816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31969392ccdb43b491911a319e8bc7e1" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:22:39.043502 16816 raft_consensus.cc:2393] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b in current term 22: Already voted for candidate cb82cc7bd6db46d980c75237a1402860 in this term.
I20260709 13:22:39.045065 16824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57a744f5aaf944678a63469cbf514015" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 21 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:39.048017 16817 raft_consensus.cc:3060] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:22:39.069059 16819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66f75124215f4ca6bef831e174a792d8" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:39.086314 16817 raft_consensus.cc:2468] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 647a343a542d4a139fe106c19095129b in term 23.
I20260709 13:22:39.061622 16291 raft_consensus.cc:2468] T 31969392ccdb43b491911a319e8bc7e1 P 90f3da57796b488c9087a2f523adf045 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 22.
I20260709 13:22:39.077620 16219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66f75124215f4ca6bef831e174a792d8" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:39.071118 16822 consensus_queue.cc:237] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 27.15, Last appended by leader: 14, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:39.064363 24176 raft_consensus.cc:493] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 647a343a542d4a139fe106c19095129b [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
W20260709 13:22:39.131529 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 58283us
I20260709 13:22:39.133512 16117 leader_election.cc:304] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860
I20260709 13:22:39.122295 24484 raft_consensus.cc:493] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:22:39.143702 23719 raft_consensus.cc:3060] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:22:39.150666 16291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e65676b21b748728ad39668cfe722ec" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:39.160015 16297 raft_consensus.cc:1275] T cdbd78acb4cf46298e0de8ba0d130e19 P 90f3da57796b488c9087a2f523adf045 [term 27 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 25 index: 19. Preceding OpId from leader: term: 27 index: 20. (index mismatch)
I20260709 13:22:39.175529 16117 leader_election.cc:304] T 57a744f5aaf944678a63469cbf514015 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860
W20260709 13:22:39.238090 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 105358us
I20260709 13:22:39.149838 24484 raft_consensus.cc:515] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:39.168363 24176 raft_consensus.cc:515] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 647a343a542d4a139fe106c19095129b [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:22:39.250913 24556 log.cc:927] Time spent T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 0.073s	user 0.000s	sys 0.000s
W20260709 13:22:39.251657 24550 log.cc:927] Time spent T 74ca8140f35c4ae29e8a664f6bdfc829 P cb82cc7bd6db46d980c75237a1402860: Append to log took a long time: real 0.130s	user 0.000s	sys 0.001s
I20260709 13:22:39.241189 24200 raft_consensus.cc:2749] T 57a744f5aaf944678a63469cbf514015 P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:22:39.262953 23719 raft_consensus.cc:2749] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:22:39.264292 24484 leader_election.cc:290] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:22:39.309496 16226 raft_consensus.cc:1275] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac [term 23 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 21 index: 14. Preceding OpId from leader: term: 23 index: 15. (index mismatch)
I20260709 13:22:39.330184 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87ac0d1f7e8a419f9bcbc04aee0e067e" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:39.335496 23719 consensus_queue.cc:1048] T 8158bbd17b994d2eb96f560bd0f1b88d P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:22:39.453987 24227 raft_consensus.cc:493] T 3884f864a96d428785895e5463e5728f P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:22:39.482897 16117 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 281682us
I20260709 13:22:39.484758 16117 leader_election.cc:304] T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860
I20260709 13:22:39.494177 24227 raft_consensus.cc:515] T 3884f864a96d428785895e5463e5728f P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:39.504463 24568 raft_consensus.cc:2749] T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:22:39.508246 16824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3884f864a96d428785895e5463e5728f" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 18 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:39.500926 24227 leader_election.cc:290] T 3884f864a96d428785895e5463e5728f P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
W20260709 13:22:39.512180 16265 leader_election.cc:336] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:22:39.512782 16265 leader_election.cc:304] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:22:39.513834 16818 raft_consensus.cc:1275] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860 [term 23 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 21 index: 14. Preceding OpId from leader: term: 23 index: 15. (index mismatch)
I20260709 13:22:39.516774 24413 raft_consensus.cc:2749] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045 [term 23 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:22:39.530387 16292 raft_consensus.cc:1275] T 8158bbd17b994d2eb96f560bd0f1b88d P 90f3da57796b488c9087a2f523adf045 [term 23 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 21 index: 14. Preceding OpId from leader: term: 23 index: 15. (index mismatch)
I20260709 13:22:39.532881 16297 raft_consensus.cc:1275] T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045 [term 24 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 22 index: 14. Preceding OpId from leader: term: 24 index: 15. (index mismatch)
I20260709 13:22:39.533838 24490 consensus_queue.cc:1048] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.001s
I20260709 13:22:39.535796 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3884f864a96d428785895e5463e5728f" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 18 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
W20260709 13:22:39.538463 16791 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:22:39.562949 24459 consensus_queue.cc:1048] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:22:39.564376 24575 raft_consensus.cc:493] T 746aabf856ab401c9ef5e859371e7b67 P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:22:39.565369 24575 raft_consensus.cc:515] T 746aabf856ab401c9ef5e859371e7b67 P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:39.581328 24575 leader_election.cc:290] T 746aabf856ab401c9ef5e859371e7b67 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:39.602880 24176 leader_election.cc:290] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
W20260709 13:22:39.602645 16790 leader_election.cc:336] T 9e65676b21b748728ad39668cfe722ec P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:22:39.605309 16790 leader_election.cc:336] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:22:39.612427 15046 thread.cc:527] Waited for 11000ms trying to join with heartbeat (tid 16176)
I20260709 13:22:39.609717 16789 leader_election.cc:304] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 647a343a542d4a139fe106c19095129b
W20260709 13:22:39.628435 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 49896.3us
I20260709 13:22:39.632715 24490 raft_consensus.cc:2804] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:22:39.633467 24490 raft_consensus.cc:697] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860 [term 22 LEADER]: Becoming Leader. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:22:39.635201 16789 leader_election.cc:304] T 9e65676b21b748728ad39668cfe722ec P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:22:39.663836 16817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746aabf856ab401c9ef5e859371e7b67" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
W20260709 13:22:39.666147 16115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 60833.6us
I20260709 13:22:39.676116 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87ac0d1f7e8a419f9bcbc04aee0e067e" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:39.678005 24420 raft_consensus.cc:493] T 806d7690c2b644959152074bb2cab0a7 P cb82cc7bd6db46d980c75237a1402860 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:39.685571 16115 leader_election.cc:304] T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860
I20260709 13:22:39.636003 24490 consensus_queue.cc:237] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 20.16, Last appended by leader: 16, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:39.688184 24541 raft_consensus.cc:2749] T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
W20260709 13:22:39.688241 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 176082us
W20260709 13:22:39.690979 24585 log.cc:927] Time spent T 5a959dbecc2648eb85d1cc550f5a3245 P 647a343a542d4a139fe106c19095129b: Append to log took a long time: real 0.063s	user 0.000s	sys 0.000s
I20260709 13:22:39.691596 24495 raft_consensus.cc:2749] T 9e65676b21b748728ad39668cfe722ec P cb82cc7bd6db46d980c75237a1402860 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 13:22:39.692098 16117 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 55363.1us
I20260709 13:22:39.678696 24420 raft_consensus.cc:515] T 806d7690c2b644959152074bb2cab0a7 P cb82cc7bd6db46d980c75237a1402860 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:22:39.699409 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 62798.2us
I20260709 13:22:39.700440 24420 leader_election.cc:290] T 806d7690c2b644959152074bb2cab0a7 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:39.706805 24271 raft_consensus.cc:493] T ada5a004ba7c46819051e3ccec71ffdb P 90f3da57796b488c9087a2f523adf045 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
W20260709 13:22:39.707285 16265 consensus_peers.cc:597] T fffab26427e24d398e2695aad40ca134 P 90f3da57796b488c9087a2f523adf045 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:22:39.707505 24271 raft_consensus.cc:515] T ada5a004ba7c46819051e3ccec71ffdb P 90f3da57796b488c9087a2f523adf045 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:39.710532 24271 leader_election.cc:290] T ada5a004ba7c46819051e3ccec71ffdb P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:39.719417 24541 consensus_queue.cc:1048] T 8158bbd17b994d2eb96f560bd0f1b88d P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:22:39.720834 24492 consensus_queue.cc:1048] T b6f07842d3aa47858df5665874c86f3a P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:22:39.725472 16115 leader_election.cc:304] T 3884f864a96d428785895e5463e5728f P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860
I20260709 13:22:39.727098 24492 raft_consensus.cc:2749] T 3884f864a96d428785895e5463e5728f P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:22:39.727319 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 90722.1us
I20260709 13:22:39.730578 16115 leader_election.cc:304] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:22:39.733651 16225 raft_consensus.cc:1275] T b6f07842d3aa47858df5665874c86f3a P 1ca2725d290e47d68b1a4b609c9816ac [term 24 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 20 index: 13. Preceding OpId from leader: term: 24 index: 15. (index mismatch)
I20260709 13:22:39.733741 24420 consensus_queue.cc:1048] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.001s
W20260709 13:22:39.734331 16265 leader_election.cc:336] T 31969392ccdb43b491911a319e8bc7e1 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:22:39.748140 16265 leader_election.cc:304] T 31969392ccdb43b491911a319e8bc7e1 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
W20260709 13:22:39.733592 16789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 96474us
I20260709 13:22:39.741453 16224 raft_consensus.cc:1275] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac [term 23 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 21 index: 15. Preceding OpId from leader: term: 23 index: 16. (index mismatch)
W20260709 13:22:39.745460 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 204749us
W20260709 13:22:39.754365 16191 leader_election.cc:336] T ee9ed67006144797b1de6ea8c2013a68 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:22:39.755095 16191 leader_election.cc:304] T ee9ed67006144797b1de6ea8c2013a68 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
I20260709 13:22:39.761981 16822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada5a004ba7c46819051e3ccec71ffdb" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 26 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:39.768803 16295 raft_consensus.cc:1275] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045 [term 23 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 21 index: 15. Preceding OpId from leader: term: 23 index: 16. (index mismatch)
W20260709 13:22:39.769860 16790 consensus_peers.cc:597] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:22:39.770366 16191 leader_election.cc:336] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:22:39.771066 16191 leader_election.cc:304] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
W20260709 13:22:39.773803 16191 leader_election.cc:336] T e4dc6b0541ca431ebc757beac00521f1 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:22:39.774284 16191 leader_election.cc:304] T e4dc6b0541ca431ebc757beac00521f1 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:22:39.765430 24200 raft_consensus.cc:493] T c0eab25ce4c64e018a041beeda2867db P 647a343a542d4a139fe106c19095129b [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:39.748129 24495 raft_consensus.cc:493] T 19baa7bc367844c288bca8f81439c413 P cb82cc7bd6db46d980c75237a1402860 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:22:39.757166 24176 raft_consensus.cc:2749] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 647a343a542d4a139fe106c19095129b [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:22:39.785012 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 50706.7us
I20260709 13:22:39.765884 24563 raft_consensus.cc:2749] T 31969392ccdb43b491911a319e8bc7e1 P 90f3da57796b488c9087a2f523adf045 [term 22 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:22:39.789696 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada5a004ba7c46819051e3ccec71ffdb" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 26 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
W20260709 13:22:39.786357 16191 leader_election.cc:336] T c2f5f89186204242bf377f7c42b2e2da P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:22:39.794602 16817 raft_consensus.cc:1275] T 806d7690c2b644959152074bb2cab0a7 P cb82cc7bd6db46d980c75237a1402860 [term 23 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 21 index: 15. Preceding OpId from leader: term: 23 index: 16. (index mismatch)
I20260709 13:22:39.772735 24120 raft_consensus.cc:2749] T ee9ed67006144797b1de6ea8c2013a68 P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:22:39.794791 16292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806d7690c2b644959152074bb2cab0a7" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 24 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:39.802771 16292 raft_consensus.cc:2468] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 23.
I20260709 13:22:39.784554 24276 raft_consensus.cc:2749] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac [term 23 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:22:39.780193 24200 raft_consensus.cc:515] T c0eab25ce4c64e018a041beeda2867db P 647a343a542d4a139fe106c19095129b [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:39.793769 24415 raft_consensus.cc:2749] T e4dc6b0541ca431ebc757beac00521f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:22:39.801995 16191 leader_election.cc:304] T c2f5f89186204242bf377f7c42b2e2da P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:22:39.831281 16292 raft_consensus.cc:1275] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 90f3da57796b488c9087a2f523adf045 [term 23 FOLLOWER]: Refusing update from remote peer 647a343a542d4a139fe106c19095129b: Log matching property violated. Preceding OpId in replica: term: 21 index: 14. Preceding OpId from leader: term: 23 index: 15. (index mismatch)
W20260709 13:22:39.833416 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 47085.4us
I20260709 13:22:39.842572 24129 raft_consensus.cc:2749] T c2f5f89186204242bf377f7c42b2e2da P 1ca2725d290e47d68b1a4b609c9816ac [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:22:39.846575 24397 consensus_queue.cc:1048] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:22:39.849027 16265 leader_election.cc:304] T ada5a004ba7c46819051e3ccec71ffdb P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860
W20260709 13:22:39.849102 24583 log.cc:927] Time spent T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860: Append to log took a long time: real 0.093s	user 0.000s	sys 0.000s
I20260709 13:22:39.855840 24459 consensus_queue.cc:1048] T b6f07842d3aa47858df5665874c86f3a P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:22:39.858707 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0eab25ce4c64e018a041beeda2867db" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 24 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:39.795799 24495 raft_consensus.cc:515] T 19baa7bc367844c288bca8f81439c413 P cb82cc7bd6db46d980c75237a1402860 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:22:39.863932 16790 leader_election.cc:336] T 806d7690c2b644959152074bb2cab0a7 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:22:39.799623 24190 consensus_queue.cc:1048] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.080s
I20260709 13:22:39.887850 24574 consensus_queue.cc:1048] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.001s
W20260709 13:22:39.898702 16191 leader_election.cc:336] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:22:39.899308 16191 leader_election.cc:304] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
I20260709 13:22:39.912412 24129 raft_consensus.cc:2749] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 13:22:39.912925 24591 log.cc:927] Time spent T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.063s	user 0.001s	sys 0.000s
I20260709 13:22:39.906134 24276 raft_consensus.cc:493] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:39.915763 24276 raft_consensus.cc:515] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:39.918279 24276 leader_election.cc:290] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:39.921152 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdbd78acb4cf46298e0de8ba0d130e19" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 28 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:39.921152 16824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdbd78acb4cf46298e0de8ba0d130e19" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 28 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:39.924993 16192 leader_election.cc:304] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860
I20260709 13:22:39.926124 24120 raft_consensus.cc:2749] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 13:22:39.932356 16225 raft_consensus.cc:2955] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac [term 23 FOLLOWER]: Committing config change with OpId 21.15: config changed from index 2 to 15, cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) changed from NON_VOTER to VOTER. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
W20260709 13:22:39.934559 16790 leader_election.cc:336] T 19baa7bc367844c288bca8f81439c413 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:22:39.939203 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0eab25ce4c64e018a041beeda2867db" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 24 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:39.943589 16221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746aabf856ab401c9ef5e859371e7b67" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:39.947830 16115 leader_election.cc:304] T c0eab25ce4c64e018a041beeda2867db P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:22:39.940488 24200 leader_election.cc:290] T c0eab25ce4c64e018a041beeda2867db P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:39.944334 24413 raft_consensus.cc:2749] T ada5a004ba7c46819051e3ccec71ffdb P 90f3da57796b488c9087a2f523adf045 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:22:39.954726 24190 raft_consensus.cc:2749] T c0eab25ce4c64e018a041beeda2867db P 647a343a542d4a139fe106c19095129b [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 13:22:39.961207 16265 consensus_peers.cc:597] T 434bff3082d141838068f323bdb0d1ed P 90f3da57796b488c9087a2f523adf045 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:22:39.962066 16265 leader_election.cc:336] T c4f65cdf00b748bb963ee6e81ef43b79 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:22:39.970191 16265 leader_election.cc:304] T c4f65cdf00b748bb963ee6e81ef43b79 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:22:39.997885 24563 raft_consensus.cc:2749] T c4f65cdf00b748bb963ee6e81ef43b79 P 90f3da57796b488c9087a2f523adf045 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:22:39.999091 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806d7690c2b644959152074bb2cab0a7" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 24 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:40.027650 16226 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:40.020852 24397 raft_consensus.cc:493] T 23f82224f4564fea93572e8206bad1fe P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:22:40.031966 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 69930.5us
I20260709 13:22:40.043473 24397 raft_consensus.cc:515] T 23f82224f4564fea93572e8206bad1fe P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:22:40.146136 16791 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:22:40.148686 16265 consensus_peers.cc:597] T 7c2095a5f4ad49408dbc7f69c606120d P 90f3da57796b488c9087a2f523adf045 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:22:40.020854 16790 consensus_peers.cc:597] T 3884f864a96d428785895e5463e5728f P cb82cc7bd6db46d980c75237a1402860 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
W20260709 13:22:40.171638 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 151570us
W20260709 13:22:40.171854 16265 consensus_peers.cc:597] T b87147731a68400e8cce9b879daa7c5d P 90f3da57796b488c9087a2f523adf045 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:22:40.175608 16790 consensus_peers.cc:597] T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:22:40.185949 16291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
W20260709 13:22:40.188413 16265 leader_election.cc:336] T 57a744f5aaf944678a63469cbf514015 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:22:40.189826 16265 leader_election.cc:304] T 57a744f5aaf944678a63469cbf514015 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
I20260709 13:22:40.198051 16790 leader_election.cc:304] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
W20260709 13:22:40.201061 16265 leader_election.cc:336] T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:22:40.203560 16265 leader_election.cc:304] T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:22:40.204289 16789 leader_election.cc:304] T 19baa7bc367844c288bca8f81439c413 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
W20260709 13:22:40.204804 16265 leader_election.cc:336] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:22:40.205327 16265 leader_election.cc:304] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
I20260709 13:22:40.209911 24541 raft_consensus.cc:493] T d16dfad0c3b540e5b4b86492ddf87208 P 647a343a542d4a139fe106c19095129b [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:40.210536 24541 raft_consensus.cc:515] T d16dfad0c3b540e5b4b86492ddf87208 P 647a343a542d4a139fe106c19095129b [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:40.212937 24541 leader_election.cc:290] T d16dfad0c3b540e5b4b86492ddf87208 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:40.216917 24397 leader_election.cc:290] T 23f82224f4564fea93572e8206bad1fe P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:40.207054 24314 raft_consensus.cc:2749] T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045 [term 24 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:22:40.201215 24271 raft_consensus.cc:2749] T 57a744f5aaf944678a63469cbf514015 P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:22:40.230569 24490 raft_consensus.cc:2749] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:22:40.208101 24563 raft_consensus.cc:2749] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 13:22:40.246519 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d16dfad0c3b540e5b4b86492ddf87208" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 23 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:40.246663 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f82224f4564fea93572e8206bad1fe" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:40.214905 24541 consensus_queue.cc:1048] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.026s
I20260709 13:22:40.231614 24463 raft_consensus.cc:493] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:40.241130 24544 raft_consensus.cc:2749] T 19baa7bc367844c288bca8f81439c413 P cb82cc7bd6db46d980c75237a1402860 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
W20260709 13:22:40.280640 16191 leader_election.cc:336] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:22:40.299508 16191 leader_election.cc:304] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
W20260709 13:22:40.280926 24595 log.cc:927] Time spent T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.134s	user 0.002s	sys 0.000s
W20260709 13:22:40.286214 24592 log.cc:927] Time spent T ada5a004ba7c46819051e3ccec71ffdb P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.123s	user 0.000s	sys 0.001s
I20260709 13:22:40.326889 16220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d16dfad0c3b540e5b4b86492ddf87208" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 23 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:40.331308 24463 raft_consensus.cc:515] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:22:40.339463 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 141507us
I20260709 13:22:40.328714 24496 raft_consensus.cc:2749] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:22:40.364003 16790 leader_election.cc:304] T 806d7690c2b644959152074bb2cab0a7 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
W20260709 13:22:40.374323 16790 consensus_peers.cc:597] T 66f75124215f4ca6bef831e174a792d8 P cb82cc7bd6db46d980c75237a1402860 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:22:40.372223 24493 raft_consensus.cc:2749] T 806d7690c2b644959152074bb2cab0a7 P cb82cc7bd6db46d980c75237a1402860 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:22:40.376464 24463 leader_election.cc:290] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
W20260709 13:22:40.405520 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 86360.2us
I20260709 13:22:40.396785 24544 raft_consensus.cc:493] T ada5a004ba7c46819051e3ccec71ffdb P cb82cc7bd6db46d980c75237a1402860 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:22:40.426476 24495 leader_election.cc:290] T 19baa7bc367844c288bca8f81439c413 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:40.404788 24543 raft_consensus.cc:493] T 829b4ad8c5a54eababeea1181b960d4b P cb82cc7bd6db46d980c75237a1402860 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
W20260709 13:22:40.434253 24599 log.cc:927] Time spent T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860: Append to log took a long time: real 0.286s	user 0.002s	sys 0.000s
I20260709 13:22:40.432760 24543 raft_consensus.cc:515] T 829b4ad8c5a54eababeea1181b960d4b P cb82cc7bd6db46d980c75237a1402860 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:40.430757 24544 raft_consensus.cc:515] T ada5a004ba7c46819051e3ccec71ffdb P cb82cc7bd6db46d980c75237a1402860 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:40.442684 24544 leader_election.cc:290] T ada5a004ba7c46819051e3ccec71ffdb P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:40.453619 24580 raft_consensus.cc:493] T ecc1e363230043fe9fef390f0b979f20 P 90f3da57796b488c9087a2f523adf045 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:22:40.456985 24580 raft_consensus.cc:515] T ecc1e363230043fe9fef390f0b979f20 P 90f3da57796b488c9087a2f523adf045 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:40.453709 16221 raft_consensus.cc:1275] T ecc1e363230043fe9fef390f0b979f20 P 1ca2725d290e47d68b1a4b609c9816ac [term 27 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 27 index: 15. Preceding OpId from leader: term: 27 index: 16. (index mismatch)
I20260709 13:22:40.463305 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "829b4ad8c5a54eababeea1181b960d4b" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:40.464602 16219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada5a004ba7c46819051e3ccec71ffdb" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 26 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:40.455528 24543 leader_election.cc:290] T 829b4ad8c5a54eababeea1181b960d4b P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:22:40.469444 24580 leader_election.cc:290] T ecc1e363230043fe9fef390f0b979f20 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:40.501017 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 28 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
W20260709 13:22:40.502404 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 95059.6us
W20260709 13:22:40.510502 16789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 306339us
W20260709 13:22:40.512706 16790 consensus_peers.cc:597] T 2794af5b99984efebdee564035841cb0 P cb82cc7bd6db46d980c75237a1402860 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
W20260709 13:22:40.520150 16790 consensus_peers.cc:597] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:22:40.521147 16115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 218695us
I20260709 13:22:40.525576 16299 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f5f89186204242bf377f7c42b2e2da" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 27 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:40.526041 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f5f89186204242bf377f7c42b2e2da" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 27 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
W20260709 13:22:40.538183 16117 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 354520us
I20260709 13:22:40.540666 16297 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada5a004ba7c46819051e3ccec71ffdb" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 26 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:40.543145 16293 raft_consensus.cc:1275] T 31969392ccdb43b491911a319e8bc7e1 P 90f3da57796b488c9087a2f523adf045 [term 22 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 20 index: 16. Preceding OpId from leader: term: 22 index: 17. (index mismatch)
I20260709 13:22:40.559701 16115 leader_election.cc:304] T d16dfad0c3b540e5b4b86492ddf87208 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
W20260709 13:22:40.563220 16265 consensus_peers.cc:597] T 26354361e9b044cea1c2b86c4882fe92 P 90f3da57796b488c9087a2f523adf045 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:22:40.570613 16818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 28 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:40.583559 16266 leader_election.cc:304] T ecc1e363230043fe9fef390f0b979f20 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860
I20260709 13:22:40.585085 24314 raft_consensus.cc:2749] T ecc1e363230043fe9fef390f0b979f20 P 90f3da57796b488c9087a2f523adf045 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 13:22:40.558118 24541 raft_consensus.cc:493] T 26354361e9b044cea1c2b86c4882fe92 P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:40.590672 24541 raft_consensus.cc:515] T 26354361e9b044cea1c2b86c4882fe92 P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:40.592928 24541 leader_election.cc:290] T 26354361e9b044cea1c2b86c4882fe92 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:40.578902 24463 raft_consensus.cc:2749] T d16dfad0c3b540e5b4b86492ddf87208 P 647a343a542d4a139fe106c19095129b [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 13:22:40.595834 16790 consensus_peers.cc:597] T 23f82224f4564fea93572e8206bad1fe P cb82cc7bd6db46d980c75237a1402860 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
W20260709 13:22:40.596359 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 75569.7us
I20260709 13:22:40.596980 16817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f82224f4564fea93572e8206bad1fe" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
W20260709 13:22:40.612919 15046 thread.cc:527] Waited for 12000ms trying to join with heartbeat (tid 16176)
I20260709 13:22:40.614874 24543 consensus_queue.cc:1048] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
W20260709 13:22:40.596980 16790 consensus_peers.cc:597] T 57a744f5aaf944678a63469cbf514015 P cb82cc7bd6db46d980c75237a1402860 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:22:40.624064 24125 consensus_queue.cc:1048] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 13:22:40.627300 16789 leader_election.cc:304] T ada5a004ba7c46819051e3ccec71ffdb P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:22:40.633854 16117 leader_election.cc:304] T 23f82224f4564fea93572e8206bad1fe P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860
W20260709 13:22:40.638727 16040 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:58400 (request call id 195) took 14316 ms (client timeout 15000 ms). Trace:
I20260709 13:22:40.645777 16822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26354361e9b044cea1c2b86c4882fe92" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
W20260709 13:22:40.647704 16790 leader_election.cc:336] T 829b4ad8c5a54eababeea1181b960d4b P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:22:40.648800 16790 leader_election.cc:304] T 829b4ad8c5a54eababeea1181b960d4b P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:22:40.652761 24543 raft_consensus.cc:2749] T 829b4ad8c5a54eababeea1181b960d4b P cb82cc7bd6db46d980c75237a1402860 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:22:40.649369 16115 leader_election.cc:304] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:22:40.636678 24493 raft_consensus.cc:2749] T ada5a004ba7c46819051e3ccec71ffdb P cb82cc7bd6db46d980c75237a1402860 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
W20260709 13:22:40.643983 16040 rpcz_store.cc:269] 0709 13:22:26.322389 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:22:28.256097 (+1933708us) service_pool.cc:224] Handling call
0709 13:22:40.638632 (+12382535us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:22:38.414555 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:22:38.427107 (+ 12552us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:22:38.434243 (+  7136us) write_op.cc:435] Acquired schema lock
0709 13:22:38.437222 (+  2979us) tablet.cc:662] Decoding operations
0709 13:22:38.653489 (+216267us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:22:38.653585 (+    96us) write_op.cc:641] Partition lock acquired for write op
0709 13:22:38.653614 (+    29us) tablet.cc:685] Acquiring locks for 17 operations
0709 13:22:38.655105 (+  1491us) tablet.cc:701] Row locks acquired
0709 13:22:38.655152 (+    47us) write_op.cc:260] PREPARE: finished
0709 13:22:38.655362 (+   210us) write_op.cc:270] Start()
0709 13:22:38.655517 (+   155us) write_op.cc:276] Timestamp: P: 1783603358655296 usec, L: 0
0709 13:22:38.655557 (+    40us) op_driver.cc:348] REPLICATION: starting
0709 13:22:38.656384 (+   827us) log.cc:844] Serialized 6844 byte log entry
0709 13:22:39.026488 (+370104us) op_driver.cc:464] REPLICATION: finished
0709 13:22:39.227476 (+200988us) write_op.cc:301] APPLY: starting
0709 13:22:39.238796 (+ 11320us) tablet.cc:1367] starting BulkCheckPresence
0709 13:22:39.583301 (+344505us) tablet.cc:1370] finished BulkCheckPresence
0709 13:22:39.583434 (+   133us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:22:39.810370 (+226936us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:22:39.810783 (+   413us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=34,key_file_lookups=34,delta_file_lookups=0,mrs_lookups=0
0709 13:22:39.810850 (+    67us) write_op.cc:312] APPLY: finished
0709 13:22:39.900000 (+ 89150us) log.cc:844] Serialized 156 byte log entry
0709 13:22:39.900913 (+   913us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:22:39.901091 (+   178us) write_op.cc:454] Released schema lock
0709 13:22:39.901173 (+    82us) write_op.cc:341] FINISH: Updating metrics
0709 13:22:39.912395 (+ 11222us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:22:39.912430 (+    35us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":2868447,"reported_tablets":35,"child_traces":[["op",{"apply.queue_time_us":125664,"apply.run_cpu_time_us":30710,"apply.run_wall_time_us":731685,"cfile_cache_hit":70,"cfile_cache_hit_bytes":75119,"num_ops":17,"prepare.queue_time_us":504073,"prepare.run_cpu_time_us":17174,"prepare.run_wall_time_us":349642,"raft.queue_time_us":246307,"raft.run_cpu_time_us":9311,"raft.run_wall_time_us":111988,"replication_time_us":363877,"thread_start_us":764360,"threads_started":4,"wal-append.queue_time_us":94437}]]}
W20260709 13:22:40.662962 16265 consensus_peers.cc:597] T 8cc959fb7f704fba9009c0595467d074 P 90f3da57796b488c9087a2f523adf045 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:22:40.658133 24603 raft_consensus.cc:2749] T 23f82224f4564fea93572e8206bad1fe P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:22:40.686546 16294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26354361e9b044cea1c2b86c4882fe92" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:40.666204 24141 raft_consensus.cc:2749] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
W20260709 13:22:40.670017 16191 leader_election.cc:336] T 746aabf856ab401c9ef5e859371e7b67 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:22:40.687618 16191 leader_election.cc:304] T 746aabf856ab401c9ef5e859371e7b67 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
I20260709 13:22:40.688854 24120 raft_consensus.cc:2749] T 746aabf856ab401c9ef5e859371e7b67 P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:22:40.697000 16265 leader_election.cc:336] T ecc1e363230043fe9fef390f0b979f20 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:22:40.715508 16265 consensus_peers.cc:597] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:22:40.674222 16041 catalog_manager.cc:5697] T fffab26427e24d398e2695aad40ca134 P 90f3da57796b488c9087a2f523adf045 reported cstate change: term changed from 9 to 15, leader changed from 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130) to 90f3da57796b488c9087a2f523adf045 (127.14.177.131). New cstate: current_term: 15 leader_uuid: "90f3da57796b488c9087a2f523adf045" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } health_report { overall_health: HEALTHY } } }
I20260709 13:22:40.736555 16115 leader_election.cc:304] T 26354361e9b044cea1c2b86c4882fe92 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860
W20260709 13:22:40.742553 16265 consensus_peers.cc:597] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 90f3da57796b488c9087a2f523adf045 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:22:40.746448 24493 raft_consensus.cc:2955] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860 [term 27 LEADER]: Committing config change with OpId 27.16: config changed from index 4 to 16, VOTER 90f3da57796b488c9087a2f523adf045 (127.14.177.131) evicted. New config: { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:22:40.756315 16041 catalog_manager.cc:5697] T e4dc6b0541ca431ebc757beac00521f1 P 90f3da57796b488c9087a2f523adf045 reported cstate change: term changed from 12 to 14, leader changed from 647a343a542d4a139fe106c19095129b (127.14.177.129) to 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130). New cstate: current_term: 14 leader_uuid: "1ca2725d290e47d68b1a4b609c9816ac" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } }
I20260709 13:22:40.762403 16224 raft_consensus.cc:2955] T ecc1e363230043fe9fef390f0b979f20 P 1ca2725d290e47d68b1a4b609c9816ac [term 27 FOLLOWER]: Committing config change with OpId 27.16: config changed from index 4 to 16, VOTER 90f3da57796b488c9087a2f523adf045 (127.14.177.131) evicted. New config: { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:22:40.772843 24541 raft_consensus.cc:2749] T 26354361e9b044cea1c2b86c4882fe92 P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:22:40.782881 16115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 46437.7us
I20260709 13:22:40.824517 24493 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.ChangeConfig from 127.0.0.1:60962 (request call id 21) took 1759 ms. Trace:
I20260709 13:22:40.828584 24493 rpcz_store.cc:276] 0709 13:22:39.064792 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:22:39.069101 (+  4309us) service_pool.cc:224] Handling call
0709 13:22:39.150103 (+ 81002us) log.cc:844] Serialized 508 byte log entry
0709 13:22:39.623539 (+473436us) spinlock_profiling.cc:243] Waited 2.68 ms on lock 0x7b540056f020. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0ad8142 00007f69b0ab8eaa 00007f69b0abfd53 00007f69b0abfcd9 00007f69b0abfc69 00007f69b0abfc31 00007f69b0abedfd 000000000038f354 000000000038f199 00007f69ada72475
0709 13:22:40.824418 (+1200879us) inbound_call.cc:177] Queueing success response
Metrics: {"dns_us":2656,"mutex_wait_us":2473,"raft.queue_time_us":5039,"raft.run_cpu_time_us":2917,"raft.run_wall_time_us":303282,"spinlock_wait_cycles":6167296,"thread_start_us":130337,"threads_started":1,"wal-append.queue_time_us":300003,"wal-append.run_cpu_time_us":3672,"wal-append.run_wall_time_us":1102716}
W20260709 13:22:40.848052 16191 consensus_peers.cc:597] T 829b4ad8c5a54eababeea1181b960d4b P 1ca2725d290e47d68b1a4b609c9816ac -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:22:40.977808 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 109282us
I20260709 13:22:40.991773 16036 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet ecc1e363230043fe9fef390f0b979f20 with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER succeeded (attempt 10)
W20260709 13:22:41.008095 16115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 47769.8us
I20260709 13:22:41.012501 24574 raft_consensus.cc:493] T af57996a8be84d398f0f023e8d08ef77 P 647a343a542d4a139fe106c19095129b [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:41.013139 24574 raft_consensus.cc:515] T af57996a8be84d398f0f023e8d08ef77 P 647a343a542d4a139fe106c19095129b [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:41.016927 24574 leader_election.cc:290] T af57996a8be84d398f0f023e8d08ef77 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:41.018257 24574 raft_consensus.cc:493] T fffab26427e24d398e2695aad40ca134 P 647a343a542d4a139fe106c19095129b [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:41.018962 24574 raft_consensus.cc:515] T fffab26427e24d398e2695aad40ca134 P 647a343a542d4a139fe106c19095129b [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:41.021522 16823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af57996a8be84d398f0f023e8d08ef77" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
W20260709 13:22:41.031406 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 68010.9us
W20260709 13:22:41.035863 16191 consensus_peers.cc:597] T c0eab25ce4c64e018a041beeda2867db P 1ca2725d290e47d68b1a4b609c9816ac -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:22:41.036710 16191 consensus_peers.cc:597] T e5ad4eb281564936a59f41e387a47156 P 1ca2725d290e47d68b1a4b609c9816ac -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:22:41.037424 16191 consensus_peers.cc:597] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:22:41.043339 16221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fffab26427e24d398e2695aad40ca134" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 23 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:41.043798 16295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fffab26427e24d398e2695aad40ca134" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 23 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:41.053131 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af57996a8be84d398f0f023e8d08ef77" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
W20260709 13:22:41.063547 16191 consensus_peers.cc:597] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:22:41.046970 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 53608.7us
I20260709 13:22:41.064857 16115 leader_election.cc:304] T fffab26427e24d398e2695aad40ca134 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
W20260709 13:22:41.019011 16041 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:49252 (request call id 291) took 61818 ms (client timeout 15000 ms). Trace:
I20260709 13:22:41.021636 24574 leader_election.cc:290] T fffab26427e24d398e2695aad40ca134 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:41.066686 24141 raft_consensus.cc:2749] T fffab26427e24d398e2695aad40ca134 P 647a343a542d4a139fe106c19095129b [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:22:41.088690 16116 leader_election.cc:304] T af57996a8be84d398f0f023e8d08ef77 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860
W20260709 13:22:41.091259 16191 consensus_peers.cc:597] T af57996a8be84d398f0f023e8d08ef77 P 1ca2725d290e47d68b1a4b609c9816ac -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:22:41.096345 24574 raft_consensus.cc:2749] T af57996a8be84d398f0f023e8d08ef77 P 647a343a542d4a139fe106c19095129b [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 13:22:41.077842 16041 rpcz_store.cc:269] 0709 13:21:39.200517 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:39.231693 (+ 31176us) service_pool.cc:224] Handling call
0709 13:22:41.018906 (+61787213us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:22:40.803834 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:22:40.804432 (+   598us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:22:40.804471 (+    39us) write_op.cc:435] Acquired schema lock
0709 13:22:40.804490 (+    19us) tablet.cc:662] Decoding operations
0709 13:22:40.804906 (+   416us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:22:40.804990 (+    84us) write_op.cc:641] Partition lock acquired for write op
0709 13:22:40.805021 (+    31us) tablet.cc:685] Acquiring locks for 2 operations
0709 13:22:40.805391 (+   370us) tablet.cc:701] Row locks acquired
0709 13:22:40.805414 (+    23us) write_op.cc:260] PREPARE: finished
0709 13:22:40.805567 (+   153us) write_op.cc:270] Start()
0709 13:22:40.805697 (+   130us) write_op.cc:276] Timestamp: P: 1783603360805521 usec, L: 0
0709 13:22:40.805723 (+    26us) op_driver.cc:348] REPLICATION: starting
0709 13:22:40.806429 (+   706us) log.cc:844] Serialized 935 byte log entry
0709 13:22:40.828628 (+ 22199us) op_driver.cc:464] REPLICATION: finished
0709 13:22:40.829836 (+  1208us) write_op.cc:301] APPLY: starting
0709 13:22:40.829923 (+    87us) tablet.cc:1367] starting BulkCheckPresence
0709 13:22:40.834263 (+  4340us) tablet.cc:1370] finished BulkCheckPresence
0709 13:22:40.834393 (+   130us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:22:40.835852 (+  1459us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:22:40.836072 (+   220us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=4,key_file_lookups=4,delta_file_lookups=0,mrs_lookups=0
0709 13:22:40.836124 (+    52us) write_op.cc:312] APPLY: finished
0709 13:22:40.836366 (+   242us) log.cc:844] Serialized 33 byte log entry
0709 13:22:40.837265 (+   899us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:22:40.837370 (+   105us) write_op.cc:454] Released schema lock
0709 13:22:40.837413 (+    43us) write_op.cc:341] FINISH: Updating metrics
0709 13:22:40.851242 (+ 13829us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:22:40.851282 (+    40us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":410374,"reported_tablets":49,"child_traces":[["op",{"apply.queue_time_us":960,"apply.run_cpu_time_us":4977,"apply.run_wall_time_us":21313,"cfile_cache_hit":10,"cfile_cache_hit_bytes":12479,"num_ops":2,"prepare.queue_time_us":21835,"prepare.run_cpu_time_us":2944,"prepare.run_wall_time_us":2945,"replication_time_us":22737,"thread_start_us":23237,"threads_started":2}]]}
I20260709 13:22:41.141213 24610 raft_consensus.cc:493] T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:41.141897 24610 raft_consensus.cc:515] T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:41.148170 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4f65cdf00b748bb963ee6e81ef43b79" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 24 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:41.159250 24610 leader_election.cc:290] T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:41.160580 16042 catalog_manager.cc:5697] T 68a6838d82ef4cac83507f906139d90f P 90f3da57796b488c9087a2f523adf045 reported cstate change: term changed from 17 to 21, leader changed from 647a343a542d4a139fe106c19095129b (127.14.177.129) to 90f3da57796b488c9087a2f523adf045 (127.14.177.131). New cstate: current_term: 21 leader_uuid: "90f3da57796b488c9087a2f523adf045" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } health_report { overall_health: HEALTHY } } }
I20260709 13:22:41.166293 16295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4f65cdf00b748bb963ee6e81ef43b79" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 24 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
W20260709 13:22:41.172493 16790 consensus_peers.cc:597] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:22:41.176716 16115 leader_election.cc:304] T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:22:41.177686 24459 raft_consensus.cc:2749] T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:22:41.162664 16042 catalog_manager.cc:5697] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 90f3da57796b488c9087a2f523adf045 reported cstate change: term changed from 16 to 21, leader changed from 647a343a542d4a139fe106c19095129b (127.14.177.129) to 90f3da57796b488c9087a2f523adf045 (127.14.177.131). New cstate: current_term: 21 leader_uuid: "90f3da57796b488c9087a2f523adf045" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } health_report { overall_health: HEALTHY } } }
I20260709 13:22:41.199033 16042 catalog_manager.cc:5697] T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045 reported cstate change: term changed from 19 to 23, leader changed from cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) to 90f3da57796b488c9087a2f523adf045 (127.14.177.131). New cstate: current_term: 23 leader_uuid: "90f3da57796b488c9087a2f523adf045" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:41.206516 16042 catalog_manager.cc:5697] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045 reported cstate change: term changed from 18 to 21. New cstate: current_term: 21 leader_uuid: "647a343a542d4a139fe106c19095129b" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } } }
I20260709 13:22:41.209343 24459 raft_consensus.cc:493] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:41.209925 24459 raft_consensus.cc:515] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:41.208494 16042 catalog_manager.cc:5697] T 5a959dbecc2648eb85d1cc550f5a3245 P 90f3da57796b488c9087a2f523adf045 reported cstate change: term changed from 15 to 20. New cstate: current_term: 20 leader_uuid: "90f3da57796b488c9087a2f523adf045" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } health_report { overall_health: HEALTHY } } }
I20260709 13:22:41.213711 24459 leader_election.cc:290] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:41.217595 16823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5718c623ebee4d2bb770f0f9ddb2f164" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 25 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:41.230249 16221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5718c623ebee4d2bb770f0f9ddb2f164" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 25 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:41.236920 16116 leader_election.cc:304] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860
I20260709 13:22:41.239738 24610 raft_consensus.cc:2749] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:22:41.247660 16042 catalog_manager.cc:5697] T 1df93779209748b9860f6d893dd3de53 P 90f3da57796b488c9087a2f523adf045 reported cstate change: term changed from 6 to 19. New cstate: current_term: 19 leader_uuid: "647a343a542d4a139fe106c19095129b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } }
I20260709 13:22:41.255467 24459 raft_consensus.cc:493] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:41.258445 24459 raft_consensus.cc:515] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:41.266112 24459 leader_election.cc:290] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:41.266670 16221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 26 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:41.283228 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 26 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:41.288265 16115 leader_election.cc:304] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:22:41.289786 24610 raft_consensus.cc:2749] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:22:41.290949 16823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 26 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
W20260709 13:22:41.294585 16191 consensus_peers.cc:597] T 18ad7e3af5504571a69d10c00750a01c P 1ca2725d290e47d68b1a4b609c9816ac -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:22:41.295177 16042 catalog_manager.cc:5697] T 8d4ab870a05347868956aa8b841b2981 P 90f3da57796b488c9087a2f523adf045 reported cstate change: term changed from 14 to 20. New cstate: current_term: 20 leader_uuid: "647a343a542d4a139fe106c19095129b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } }
I20260709 13:22:41.297245 16042 catalog_manager.cc:5697] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 90f3da57796b488c9087a2f523adf045 reported cstate change: term changed from 19 to 21, leader changed from cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) to 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130). New cstate: current_term: 21 leader_uuid: "1ca2725d290e47d68b1a4b609c9816ac" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:22:41.298862 16042 catalog_manager.cc:5697] T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045 reported cstate change: term changed from 15 to 20, leader changed from 647a343a542d4a139fe106c19095129b (127.14.177.129) to 90f3da57796b488c9087a2f523adf045 (127.14.177.131). New cstate: current_term: 20 leader_uuid: "90f3da57796b488c9087a2f523adf045" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } health_report { overall_health: HEALTHY } } }
I20260709 13:22:41.304343 16042 catalog_manager.cc:5697] T 66ae176961b74813a58bc15c9c4c6f2a P 90f3da57796b488c9087a2f523adf045 reported cstate change: term changed from 24 to 26, leader changed from cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) to 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130). New cstate: current_term: 26 leader_uuid: "1ca2725d290e47d68b1a4b609c9816ac" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:22:41.320156 16042 catalog_manager.cc:5697] T 9e65676b21b748728ad39668cfe722ec P 90f3da57796b488c9087a2f523adf045 reported cstate change: term changed from 17 to 21, leader changed from cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) to 90f3da57796b488c9087a2f523adf045 (127.14.177.131). New cstate: current_term: 21 leader_uuid: "90f3da57796b488c9087a2f523adf045" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:41.322358 16042 catalog_manager.cc:5697] T 23f82224f4564fea93572e8206bad1fe P 90f3da57796b488c9087a2f523adf045 reported cstate change: term changed from 16 to 17, leader changed from cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) to 647a343a542d4a139fe106c19095129b (127.14.177.129). New cstate: current_term: 17 leader_uuid: "647a343a542d4a139fe106c19095129b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:22:41.324312 16042 catalog_manager.cc:5697] T 26354361e9b044cea1c2b86c4882fe92 P 90f3da57796b488c9087a2f523adf045 reported cstate change: term changed from 13 to 17, leader changed from cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) to 90f3da57796b488c9087a2f523adf045 (127.14.177.131). New cstate: current_term: 17 leader_uuid: "90f3da57796b488c9087a2f523adf045" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:41.333869 16042 catalog_manager.cc:5697] T d16dfad0c3b540e5b4b86492ddf87208 P 90f3da57796b488c9087a2f523adf045 reported cstate change: term changed from 15 to 21, leader changed from 647a343a542d4a139fe106c19095129b (127.14.177.129) to 90f3da57796b488c9087a2f523adf045 (127.14.177.131). New cstate: current_term: 21 leader_uuid: "90f3da57796b488c9087a2f523adf045" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } health_report { overall_health: HEALTHY } } }
I20260709 13:22:41.345297 16042 catalog_manager.cc:5697] T 84f2da98864b4425bf7c0055951fdd4b P 90f3da57796b488c9087a2f523adf045 reported cstate change: term changed from 14 to 17, leader changed from 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130) to 647a343a542d4a139fe106c19095129b (127.14.177.129). New cstate: current_term: 17 leader_uuid: "647a343a542d4a139fe106c19095129b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } }
I20260709 13:22:41.350093 24610 raft_consensus.cc:493] T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:41.371397 24610 raft_consensus.cc:515] T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:41.373994 24610 leader_election.cc:290] T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:41.376106 16042 catalog_manager.cc:5697] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045 reported cstate change: term changed from 14 to 20, leader changed from 647a343a542d4a139fe106c19095129b (127.14.177.129) to 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130). New cstate: current_term: 20 leader_uuid: "1ca2725d290e47d68b1a4b609c9816ac" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } }
I20260709 13:22:41.379669 16823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31969392ccdb43b491911a319e8bc7e1" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 23 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:41.386149 16295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31969392ccdb43b491911a319e8bc7e1" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 23 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:41.379060 16042 catalog_manager.cc:5697] T e4dc6b0541ca431ebc757beac00521f1 P 90f3da57796b488c9087a2f523adf045 reported cstate change: term changed from 14 to 22, leader changed from 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130) to 90f3da57796b488c9087a2f523adf045 (127.14.177.131). New cstate: current_term: 22 leader_uuid: "90f3da57796b488c9087a2f523adf045" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } health_report { overall_health: HEALTHY } } }
I20260709 13:22:41.387969 16115 leader_election.cc:304] T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860
I20260709 13:22:41.389355 24610 raft_consensus.cc:2749] T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:22:41.390491 16042 catalog_manager.cc:5697] T b87147731a68400e8cce9b879daa7c5d P 90f3da57796b488c9087a2f523adf045 reported cstate change: term changed from 16 to 22, leader changed from cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) to 90f3da57796b488c9087a2f523adf045 (127.14.177.131). New cstate: current_term: 22 leader_uuid: "90f3da57796b488c9087a2f523adf045" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:41.396103 16042 catalog_manager.cc:5697] T 18ad7e3af5504571a69d10c00750a01c P 90f3da57796b488c9087a2f523adf045 reported cstate change: term changed from 13 to 17, leader changed from 647a343a542d4a139fe106c19095129b (127.14.177.129) to 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130). New cstate: current_term: 17 leader_uuid: "1ca2725d290e47d68b1a4b609c9816ac" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } }
W20260709 13:22:41.397967 16265 consensus_peers.cc:597] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:22:41.418587 16042 catalog_manager.cc:5697] T 2794af5b99984efebdee564035841cb0 P 90f3da57796b488c9087a2f523adf045 reported cstate change: term changed from 15 to 19, leader changed from cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) to 90f3da57796b488c9087a2f523adf045 (127.14.177.131). New cstate: current_term: 19 leader_uuid: "90f3da57796b488c9087a2f523adf045" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 13:22:41.613418 15046 thread.cc:527] Waited for 13000ms trying to join with heartbeat (tid 16176)
W20260709 13:22:41.658739 16265 consensus_peers.cc:597] T e4dc6b0541ca431ebc757beac00521f1 P 90f3da57796b488c9087a2f523adf045 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:22:41.700747 16117 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 72923.6us
W20260709 13:22:41.744277 16191 consensus_peers.cc:597] T 68a6838d82ef4cac83507f906139d90f P 1ca2725d290e47d68b1a4b609c9816ac -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:22:41.750567 16191 consensus_peers.cc:597] T d16dfad0c3b540e5b4b86492ddf87208 P 1ca2725d290e47d68b1a4b609c9816ac -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:22:41.788704 16265 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 175 similar messages]
I20260709 13:22:42.051390 24606 raft_consensus.cc:493] T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:42.052053 24606 raft_consensus.cc:515] T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:42.055923 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66f75124215f4ca6bef831e174a792d8" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:42.069260 16044 catalog_manager.cc:5697] T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac reported cstate change: term changed from 13 to 15. New cstate: current_term: 15 leader_uuid: "1ca2725d290e47d68b1a4b609c9816ac" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:42.071609 24606 leader_election.cc:290] T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:42.072645 16818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66f75124215f4ca6bef831e174a792d8" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:42.076685 16117 leader_election.cc:304] T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860
I20260709 13:22:42.079144 24606 raft_consensus.cc:2749] T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:22:42.091877 16042 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:49252 (request call id 293) took 17311 ms (client timeout 15000 ms). Trace:
W20260709 13:22:42.095311 16042 rpcz_store.cc:269] 0709 13:22:24.805487 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:22:24.805942 (+   455us) service_pool.cc:224] Handling call
0709 13:22:42.091743 (+17285801us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:22:41.825813 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:22:41.826528 (+   715us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:22:41.826576 (+    48us) write_op.cc:435] Acquired schema lock
0709 13:22:41.826603 (+    27us) tablet.cc:662] Decoding operations
0709 13:22:41.842788 (+ 16185us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:22:41.842873 (+    85us) write_op.cc:641] Partition lock acquired for write op
0709 13:22:41.842899 (+    26us) tablet.cc:685] Acquiring locks for 20 operations
0709 13:22:41.857915 (+ 15016us) tablet.cc:701] Row locks acquired
0709 13:22:41.857933 (+    18us) write_op.cc:260] PREPARE: finished
0709 13:22:41.858097 (+   164us) write_op.cc:270] Start()
0709 13:22:41.858236 (+   139us) write_op.cc:276] Timestamp: P: 1783603361858043 usec, L: 0
0709 13:22:41.858266 (+    30us) op_driver.cc:348] REPLICATION: starting
0709 13:22:41.859009 (+   743us) log.cc:844] Serialized 7985 byte log entry
0709 13:22:41.895534 (+ 36525us) spinlock_profiling.cc:243] Waited 5.25 ms on lock 0x7b180043af80. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b693ade1 00007f69b69475b1 00007f69ada723f3 00007f69ada747c1 00007f69ada74779 00007f69ada74709 00007f69ada746d1 00007f69ada739cd 000000000038f354 000000000038f199 00007f69ada5583a
0709 13:22:41.904134 (+  8600us) op_driver.cc:464] REPLICATION: finished
0709 13:22:41.905492 (+  1358us) write_op.cc:301] APPLY: starting
0709 13:22:41.905576 (+    84us) tablet.cc:1367] starting BulkCheckPresence
0709 13:22:41.921150 (+ 15574us) tablet.cc:1370] finished BulkCheckPresence
0709 13:22:41.921184 (+    34us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:22:41.953838 (+ 32654us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:22:41.954143 (+   305us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=40,key_file_lookups=40,delta_file_lookups=0,mrs_lookups=0
0709 13:22:41.954202 (+    59us) write_op.cc:312] APPLY: finished
0709 13:22:41.954653 (+   451us) log.cc:844] Serialized 180 byte log entry
0709 13:22:41.959349 (+  4696us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:22:41.959537 (+   188us) write_op.cc:454] Released schema lock
0709 13:22:41.959606 (+    69us) write_op.cc:341] FINISH: Updating metrics
0709 13:22:41.960115 (+   509us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:22:41.960145 (+    30us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":6,"reported_tablets":49,"child_traces":[["op",{"apply.queue_time_us":1130,"apply.run_cpu_time_us":25748,"apply.run_wall_time_us":54577,"cfile_cache_hit":82,"cfile_cache_hit_bytes":87647,"num_ops":20,"prepare.queue_time_us":375784,"prepare.run_cpu_time_us":14955,"prepare.run_wall_time_us":62416,"raft.queue_time_us":5969,"raft.run_cpu_time_us":905,"raft.run_wall_time_us":5472,"replication_time_us":45704,"spinlock_wait_cycles":12079360,"thread_start_us":418724,"threads_started":4,"wal-append.queue_time_us":28794}]]}
I20260709 13:22:42.155879 24459 raft_consensus.cc:493] T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:42.156448 24459 raft_consensus.cc:515] T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:42.160228 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4dc6b0541ca431ebc757beac00521f1" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 25 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:42.162094 24459 leader_election.cc:290] T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:42.164197 16295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4dc6b0541ca431ebc757beac00521f1" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 25 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:42.167587 24459 raft_consensus.cc:493] T e5ad4eb281564936a59f41e387a47156 P 647a343a542d4a139fe106c19095129b [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:42.169736 16115 leader_election.cc:304] T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:22:42.169325 24459 raft_consensus.cc:515] T e5ad4eb281564936a59f41e387a47156 P 647a343a542d4a139fe106c19095129b [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:42.171327 24541 raft_consensus.cc:2749] T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:22:42.172259 24459 leader_election.cc:290] T e5ad4eb281564936a59f41e387a47156 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:42.176299 24271 raft_consensus.cc:493] T ecc1e363230043fe9fef390f0b979f20 P 90f3da57796b488c9087a2f523adf045 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:22:42.176916 24271 raft_consensus.cc:515] T ecc1e363230043fe9fef390f0b979f20 P 90f3da57796b488c9087a2f523adf045 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:42.180481 16295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5ad4eb281564936a59f41e387a47156" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 23 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:42.184247 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5ad4eb281564936a59f41e387a47156" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 23 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:42.186342 16116 leader_election.cc:304] T e5ad4eb281564936a59f41e387a47156 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:22:42.187124 24606 raft_consensus.cc:2749] T e5ad4eb281564936a59f41e387a47156 P 647a343a542d4a139fe106c19095129b [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 13:22:42.188921 16265 leader_election.cc:336] T ecc1e363230043fe9fef390f0b979f20 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:22:42.190173 24271 leader_election.cc:290] T ecc1e363230043fe9fef390f0b979f20 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:42.190855 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 28 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:42.206933 16265 leader_election.cc:304] T ecc1e363230043fe9fef390f0b979f20 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:22:42.210597 24271 raft_consensus.cc:2749] T ecc1e363230043fe9fef390f0b979f20 P 90f3da57796b488c9087a2f523adf045 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
W20260709 13:22:42.221143 16265 consensus_peers.cc:597] T b87147731a68400e8cce9b879daa7c5d P 90f3da57796b488c9087a2f523adf045 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:22:42.213434 16043 catalog_manager.cc:5697] T 8158bbd17b994d2eb96f560bd0f1b88d P 647a343a542d4a139fe106c19095129b reported cstate change: term changed from 14 to 17, leader changed from 647a343a542d4a139fe106c19095129b (127.14.177.129) to 90f3da57796b488c9087a2f523adf045 (127.14.177.131). New cstate: current_term: 17 leader_uuid: "90f3da57796b488c9087a2f523adf045" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } }
I20260709 13:22:42.238132 16043 catalog_manager.cc:5697] T 8a878999c0024d2abe517484180290fc P 647a343a542d4a139fe106c19095129b reported cstate change: term changed from 15 to 16, leader changed from 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130) to 647a343a542d4a139fe106c19095129b (127.14.177.129). New cstate: current_term: 16 leader_uuid: "647a343a542d4a139fe106c19095129b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } health_report { overall_health: HEALTHY } } }
I20260709 13:22:42.244868 16043 catalog_manager.cc:5697] T 02838c86531d4f98bf58d1c22b8baf77 P 647a343a542d4a139fe106c19095129b reported cstate change: term changed from 18 to 19, leader changed from 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130) to 90f3da57796b488c9087a2f523adf045 (127.14.177.131). New cstate: current_term: 19 leader_uuid: "90f3da57796b488c9087a2f523adf045" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } }
I20260709 13:22:42.248783 16043 catalog_manager.cc:5697] T c0eab25ce4c64e018a041beeda2867db P 647a343a542d4a139fe106c19095129b reported cstate change: term changed from 16 to 18. New cstate: current_term: 18 leader_uuid: "647a343a542d4a139fe106c19095129b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } health_report { overall_health: HEALTHY } } }
I20260709 13:22:42.256498 16818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 28 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:42.281672 24459 raft_consensus.cc:493] T 18ad7e3af5504571a69d10c00750a01c P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:42.282303 24459 raft_consensus.cc:515] T 18ad7e3af5504571a69d10c00750a01c P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } }
I20260709 13:22:42.286489 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18ad7e3af5504571a69d10c00750a01c" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:42.286819 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18ad7e3af5504571a69d10c00750a01c" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:42.287442 24459 leader_election.cc:290] T 18ad7e3af5504571a69d10c00750a01c P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
W20260709 13:22:42.288364 16044 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:49250 (request call id 323) took 75047 ms (client timeout 15000 ms). Trace:
I20260709 13:22:42.289155 16115 leader_election.cc:304] T 18ad7e3af5504571a69d10c00750a01c P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:22:42.290135 24459 raft_consensus.cc:2749] T 18ad7e3af5504571a69d10c00750a01c P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:22:42.288769 16044 rpcz_store.cc:269] 0709 13:21:27.240849 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:27.241152 (+   303us) service_pool.cc:224] Handling call
0709 13:21:27.917069 (+675917us) spinlock_profiling.cc:243] Waited 75 ms on lock 0x7b5400002490. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69ad8fadb1 00007f69ada3ab71 00007f69ada3b0b2 00007f69b69b4a89 00007f69b694cda6 00007f69b691e088 00007f69b6a1c37e 00007f69b4edd2a4 00007f69b4edd232 00007f69b4edd161 00007f69b4edd0dc
0709 13:22:42.288293 (+74371224us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:22:42.100073 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:22:42.100645 (+   572us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:22:42.100680 (+    35us) write_op.cc:435] Acquired schema lock
0709 13:22:42.100693 (+    13us) tablet.cc:662] Decoding operations
0709 13:22:42.101046 (+   353us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:22:42.101103 (+    57us) write_op.cc:641] Partition lock acquired for write op
0709 13:22:42.101131 (+    28us) tablet.cc:685] Acquiring locks for 1 operations
0709 13:22:42.101359 (+   228us) tablet.cc:701] Row locks acquired
0709 13:22:42.101373 (+    14us) write_op.cc:260] PREPARE: finished
0709 13:22:42.101538 (+   165us) write_op.cc:270] Start()
0709 13:22:42.101654 (+   116us) write_op.cc:276] Timestamp: P: 1783603362101487 usec, L: 0
0709 13:22:42.101684 (+    30us) op_driver.cc:348] REPLICATION: starting
0709 13:22:42.102338 (+   654us) log.cc:844] Serialized 544 byte log entry
0709 13:22:42.113059 (+ 10721us) op_driver.cc:464] REPLICATION: finished
0709 13:22:42.113573 (+   514us) write_op.cc:301] APPLY: starting
0709 13:22:42.113652 (+    79us) tablet.cc:1367] starting BulkCheckPresence
0709 13:22:42.114521 (+   869us) tablet.cc:1370] finished BulkCheckPresence
0709 13:22:42.114609 (+    88us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:22:42.115294 (+   685us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:22:42.115442 (+   148us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=2,key_file_lookups=2,delta_file_lookups=0,mrs_lookups=0
0709 13:22:42.115487 (+    45us) write_op.cc:312] APPLY: finished
0709 13:22:42.115687 (+   200us) log.cc:844] Serialized 25 byte log entry
0709 13:22:42.115850 (+   163us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:22:42.115930 (+    80us) write_op.cc:454] Released schema lock
0709 13:22:42.115966 (+    36us) write_op.cc:341] FINISH: Updating metrics
0709 13:22:42.116343 (+   377us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:22:42.116369 (+    26us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":1022487,"reported_tablets":48,"spinlock_wait_cycles":172530944,"child_traces":[["op",{"apply.queue_time_us":272,"apply.run_cpu_time_us":2708,"apply.run_wall_time_us":2708,"cfile_cache_hit":4,"cfile_cache_hit_bytes":4176,"num_ops":1,"prepare.queue_time_us":1170,"prepare.run_cpu_time_us":2603,"prepare.run_wall_time_us":2603,"replication_time_us":11226}]]}
I20260709 13:22:42.286947 16043 catalog_manager.cc:5697] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b reported cstate change: term changed from 14 to 18. New cstate: current_term: 18 leader_uuid: "647a343a542d4a139fe106c19095129b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:42.301292 16043 catalog_manager.cc:5697] T 9c61a39fd4f84424b129c1a6f9ba404d P 647a343a542d4a139fe106c19095129b reported cstate change: term changed from 15 to 19, leader changed from 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130) to 647a343a542d4a139fe106c19095129b (127.14.177.129). New cstate: current_term: 19 leader_uuid: "647a343a542d4a139fe106c19095129b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:42.308466 16043 catalog_manager.cc:5697] T 8cc959fb7f704fba9009c0595467d074 P 647a343a542d4a139fe106c19095129b reported cstate change: term changed from 14 to 16, leader changed from 90f3da57796b488c9087a2f523adf045 (127.14.177.131) to 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130). New cstate: current_term: 16 leader_uuid: "1ca2725d290e47d68b1a4b609c9816ac" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } }
I20260709 13:22:42.313115 16043 catalog_manager.cc:5697] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 647a343a542d4a139fe106c19095129b reported cstate change: term changed from 13 to 18, leader changed from cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) to 647a343a542d4a139fe106c19095129b (127.14.177.129). New cstate: current_term: 18 leader_uuid: "647a343a542d4a139fe106c19095129b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:42.323889 16043 catalog_manager.cc:5697] T 5a96373bb0e14dce8a266b2fc6c58507 P 647a343a542d4a139fe106c19095129b reported cstate change: term changed from 12 to 14. New cstate: current_term: 14 leader_uuid: "647a343a542d4a139fe106c19095129b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } health_report { overall_health: HEALTHY } } }
W20260709 13:22:42.454905 16043 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:49246 (request call id 334) took 35320 ms (client timeout 15000 ms). Trace:
W20260709 13:22:42.455437 16790 consensus_peers.cc:597] T 3884f864a96d428785895e5463e5728f P cb82cc7bd6db46d980c75237a1402860 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
I20260709 13:22:42.471518 16041 catalog_manager.cc:5697] T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045 reported cstate change: term changed from 19 to 25. New cstate: current_term: 25 leader_uuid: "647a343a542d4a139fe106c19095129b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } }
I20260709 13:22:42.478273 16041 catalog_manager.cc:5697] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 90f3da57796b488c9087a2f523adf045 reported cstate change: term changed from 18 to 21. New cstate: current_term: 21 leader_uuid: "647a343a542d4a139fe106c19095129b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } }
W20260709 13:22:42.475347 16043 rpcz_store.cc:269] 0709 13:22:07.134547 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:22:07.136519 (+  1972us) service_pool.cc:224] Handling call
0709 13:22:07.643221 (+506702us) spinlock_profiling.cc:243] Waited 86.1 ms on lock 0x7b5400002490. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69ad8fadb1 00007f69ada3ab71 00007f69ada3b0b2 00007f69b69b4a89 00007f69b694cda6 00007f69b691e088 00007f69b6a1c37e 00007f69b4edd2a4 00007f69b4edd232 00007f69b4edd161 00007f69b4edd0dc
0709 13:22:42.454810 (+34811589us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:22:42.375592 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:22:42.376136 (+   544us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:22:42.376171 (+    35us) write_op.cc:435] Acquired schema lock
0709 13:22:42.376193 (+    22us) tablet.cc:662] Decoding operations
0709 13:22:42.377135 (+   942us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:22:42.377192 (+    57us) write_op.cc:641] Partition lock acquired for write op
0709 13:22:42.377232 (+    40us) tablet.cc:685] Acquiring locks for 9 operations
0709 13:22:42.378016 (+   784us) tablet.cc:701] Row locks acquired
0709 13:22:42.378036 (+    20us) write_op.cc:260] PREPARE: finished
0709 13:22:42.378186 (+   150us) write_op.cc:270] Start()
0709 13:22:42.378324 (+   138us) write_op.cc:276] Timestamp: P: 1783603362378133 usec, L: 0
0709 13:22:42.378349 (+    25us) op_driver.cc:348] REPLICATION: starting
0709 13:22:42.379030 (+   681us) log.cc:844] Serialized 3668 byte log entry
0709 13:22:42.381936 (+  2906us) op_driver.cc:464] REPLICATION: finished
0709 13:22:42.382411 (+   475us) write_op.cc:301] APPLY: starting
0709 13:22:42.382494 (+    83us) tablet.cc:1367] starting BulkCheckPresence
0709 13:22:42.388305 (+  5811us) tablet.cc:1370] finished BulkCheckPresence
0709 13:22:42.388408 (+   103us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:22:42.397108 (+  8700us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:22:42.397346 (+   238us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=18,key_file_lookups=18,delta_file_lookups=0,mrs_lookups=0
0709 13:22:42.397402 (+    56us) write_op.cc:312] APPLY: finished
0709 13:22:42.397739 (+   337us) log.cc:844] Serialized 89 byte log entry
0709 13:22:42.399744 (+  2005us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:22:42.399891 (+   147us) write_op.cc:454] Released schema lock
0709 13:22:42.399941 (+    50us) write_op.cc:341] FINISH: Updating metrics
0709 13:22:42.418012 (+ 18071us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:22:42.418046 (+    34us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":219993,"reported_tablets":48,"spinlock_wait_cycles":197967104,"child_traces":[["op",{"apply.queue_time_us":278,"apply.run_cpu_time_us":12988,"apply.run_wall_time_us":35543,"cfile_cache_hit":36,"cfile_cache_hit_bytes":37584,"num_ops":9,"prepare.queue_time_us":175,"prepare.run_cpu_time_us":3770,"prepare.run_wall_time_us":6000,"replication_time_us":3447}]]}
I20260709 13:22:42.480815 16041 catalog_manager.cc:5697] T e5ad4eb281564936a59f41e387a47156 P 90f3da57796b488c9087a2f523adf045 reported cstate change: term changed from 17 to 22, leader changed from 90f3da57796b488c9087a2f523adf045 (127.14.177.131) to 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130). New cstate: current_term: 22 leader_uuid: "1ca2725d290e47d68b1a4b609c9816ac" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } }
I20260709 13:22:42.488837 16041 catalog_manager.cc:5697] T 8158bbd17b994d2eb96f560bd0f1b88d P 90f3da57796b488c9087a2f523adf045 reported cstate change: term changed from 17 to 23, leader changed from 90f3da57796b488c9087a2f523adf045 (127.14.177.131) to 647a343a542d4a139fe106c19095129b (127.14.177.129). New cstate: current_term: 23 leader_uuid: "647a343a542d4a139fe106c19095129b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } }
I20260709 13:22:42.490788 16041 catalog_manager.cc:5697] T 57a744f5aaf944678a63469cbf514015 P 90f3da57796b488c9087a2f523adf045 reported cstate change: term changed from 12 to 20, leader changed from 647a343a542d4a139fe106c19095129b (127.14.177.129) to cb82cc7bd6db46d980c75237a1402860 (127.14.177.132). New cstate: current_term: 20 leader_uuid: "cb82cc7bd6db46d980c75237a1402860" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
W20260709 13:22:42.496235 16790 consensus_peers.cc:597] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:22:42.493947 16041 catalog_manager.cc:5697] T ecc1e363230043fe9fef390f0b979f20 P 90f3da57796b488c9087a2f523adf045 reported cstate change: term changed from 23 to 27, leader changed from 647a343a542d4a139fe106c19095129b (127.14.177.129) to cb82cc7bd6db46d980c75237a1402860 (127.14.177.132). New cstate: current_term: 27 leader_uuid: "cb82cc7bd6db46d980c75237a1402860" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:22:42.501519 16041 catalog_manager.cc:5697] T 68a6838d82ef4cac83507f906139d90f P 90f3da57796b488c9087a2f523adf045 reported cstate change: term changed from 21 to 23, leader changed from 90f3da57796b488c9087a2f523adf045 (127.14.177.131) to 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130). New cstate: current_term: 23 leader_uuid: "1ca2725d290e47d68b1a4b609c9816ac" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } }
I20260709 13:22:42.508195 16041 catalog_manager.cc:5697] T c9944d5e4b8c4109af9955c4f08c9095 P 90f3da57796b488c9087a2f523adf045 reported cstate change: term changed from 19 to 24. New cstate: current_term: 24 leader_uuid: "cb82cc7bd6db46d980c75237a1402860" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:22:42.509939 16041 catalog_manager.cc:5697] T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045 reported cstate change: term changed from 23 to 26, leader changed from 90f3da57796b488c9087a2f523adf045 (127.14.177.131) to cb82cc7bd6db46d980c75237a1402860 (127.14.177.132). New cstate: current_term: 26 leader_uuid: "cb82cc7bd6db46d980c75237a1402860" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:22:42.517503 16041 catalog_manager.cc:5697] T a378d7f796f94615b3b347119a9880fd P 90f3da57796b488c9087a2f523adf045 reported cstate change: term changed from 21 to 23, leader changed from cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) to 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130). New cstate: current_term: 23 leader_uuid: "1ca2725d290e47d68b1a4b609c9816ac" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:22:42.521443 16041 catalog_manager.cc:5697] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045 reported cstate change: config changed from index 2 to 15, term changed from 21 to 23, cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) changed from NON_VOTER to VOTER. New cstate: current_term: 23 leader_uuid: "647a343a542d4a139fe106c19095129b" committed_config { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:22:42.525472 16041 catalog_manager.cc:5697] T 8cc959fb7f704fba9009c0595467d074 P 90f3da57796b488c9087a2f523adf045 reported cstate change: term changed from 16 to 20, leader changed from 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130) to 90f3da57796b488c9087a2f523adf045 (127.14.177.131). New cstate: current_term: 20 leader_uuid: "90f3da57796b488c9087a2f523adf045" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } health_report { overall_health: HEALTHY } } }
I20260709 13:22:42.526700 24541 raft_consensus.cc:493] T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:42.529461 24541 raft_consensus.cc:515] T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:42.527843 16041 catalog_manager.cc:5697] T 5a959dbecc2648eb85d1cc550f5a3245 P 90f3da57796b488c9087a2f523adf045 reported cstate change: term changed from 20 to 23, leader changed from 90f3da57796b488c9087a2f523adf045 (127.14.177.131) to 647a343a542d4a139fe106c19095129b (127.14.177.129). New cstate: current_term: 23 leader_uuid: "647a343a542d4a139fe106c19095129b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } }
I20260709 13:22:42.532142 24541 leader_election.cc:290] T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:42.531147 16041 catalog_manager.cc:5697] T cdbd78acb4cf46298e0de8ba0d130e19 P 90f3da57796b488c9087a2f523adf045 reported cstate change: term changed from 24 to 27. New cstate: current_term: 27 leader_uuid: "cb82cc7bd6db46d980c75237a1402860" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:22:42.533602 16823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2372981006d4b218215c28964dea623" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 28 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:42.536402 16041 catalog_manager.cc:5697] T 1df93779209748b9860f6d893dd3de53 P 90f3da57796b488c9087a2f523adf045 reported cstate change: term changed from 19 to 21. New cstate: current_term: 21 leader_uuid: "647a343a542d4a139fe106c19095129b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } }
I20260709 13:22:42.544386 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2372981006d4b218215c28964dea623" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 28 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:42.544910 16041 catalog_manager.cc:5697] T 8d4ab870a05347868956aa8b841b2981 P 90f3da57796b488c9087a2f523adf045 reported cstate change: term changed from 20 to 22. New cstate: current_term: 22 leader_uuid: "647a343a542d4a139fe106c19095129b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } }
I20260709 13:22:42.546516 16041 catalog_manager.cc:5697] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045 reported cstate change: term changed from 23 to 25. New cstate: current_term: 25 leader_uuid: "90f3da57796b488c9087a2f523adf045" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:22:42.548499 16041 catalog_manager.cc:5697] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 90f3da57796b488c9087a2f523adf045 reported cstate change: term changed from 21 to 23. New cstate: current_term: 23 leader_uuid: "1ca2725d290e47d68b1a4b609c9816ac" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:22:42.549321 16116 leader_election.cc:304] T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860
I20260709 13:22:42.550150 24459 raft_consensus.cc:2749] T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 13:22:42.550748 16041 catalog_manager.cc:5697] T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045 reported cstate change: term changed from 20 to 24, leader changed from 90f3da57796b488c9087a2f523adf045 (127.14.177.131) to 647a343a542d4a139fe106c19095129b (127.14.177.129). New cstate: current_term: 24 leader_uuid: "647a343a542d4a139fe106c19095129b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } }
I20260709 13:22:42.554288 16041 catalog_manager.cc:5697] T 66ae176961b74813a58bc15c9c4c6f2a P 90f3da57796b488c9087a2f523adf045 reported cstate change: term changed from 26 to 29, leader changed from 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130) to cb82cc7bd6db46d980c75237a1402860 (127.14.177.132). New cstate: current_term: 29 leader_uuid: "cb82cc7bd6db46d980c75237a1402860" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:22:42.560273 16041 catalog_manager.cc:5697] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045 reported cstate change: term changed from 17 to 25, leader changed from 647a343a542d4a139fe106c19095129b (127.14.177.129) to 90f3da57796b488c9087a2f523adf045 (127.14.177.131). New cstate: current_term: 25 leader_uuid: "90f3da57796b488c9087a2f523adf045" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } health_report { overall_health: HEALTHY } } }
I20260709 13:22:42.562008 16041 catalog_manager.cc:5697] T 9e65676b21b748728ad39668cfe722ec P 90f3da57796b488c9087a2f523adf045 reported cstate change: term changed from 21 to 22, leader changed from 90f3da57796b488c9087a2f523adf045 (127.14.177.131) to 647a343a542d4a139fe106c19095129b (127.14.177.129). New cstate: current_term: 22 leader_uuid: "647a343a542d4a139fe106c19095129b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
W20260709 13:22:42.572750 16265 consensus_peers.cc:597] T fffab26427e24d398e2695aad40ca134 P 90f3da57796b488c9087a2f523adf045 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:22:42.576023 16041 catalog_manager.cc:5697] T 7c2095a5f4ad49408dbc7f69c606120d P 90f3da57796b488c9087a2f523adf045 reported cstate change: term changed from 9 to 19, leader changed from 647a343a542d4a139fe106c19095129b (127.14.177.129) to 90f3da57796b488c9087a2f523adf045 (127.14.177.131). New cstate: current_term: 19 leader_uuid: "90f3da57796b488c9087a2f523adf045" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } health_report { overall_health: HEALTHY } } }
I20260709 13:22:42.577804 16041 catalog_manager.cc:5697] T 23f82224f4564fea93572e8206bad1fe P 90f3da57796b488c9087a2f523adf045 reported cstate change: term changed from 17 to 19, leader changed from 647a343a542d4a139fe106c19095129b (127.14.177.129) to cb82cc7bd6db46d980c75237a1402860 (127.14.177.132). New cstate: current_term: 19 leader_uuid: "cb82cc7bd6db46d980c75237a1402860" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:22:42.579244 16041 catalog_manager.cc:5697] T fffab26427e24d398e2695aad40ca134 P 90f3da57796b488c9087a2f523adf045 reported cstate change: term changed from 15 to 21. New cstate: current_term: 21 leader_uuid: "90f3da57796b488c9087a2f523adf045" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } health_report { overall_health: HEALTHY } } }
I20260709 13:22:42.580610 16041 catalog_manager.cc:5697] T 26354361e9b044cea1c2b86c4882fe92 P 90f3da57796b488c9087a2f523adf045 reported cstate change: term changed from 17 to 19. New cstate: current_term: 19 leader_uuid: "90f3da57796b488c9087a2f523adf045" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:22:42.582162 16041 catalog_manager.cc:5697] T 8a878999c0024d2abe517484180290fc P 90f3da57796b488c9087a2f523adf045 reported cstate change: term changed from 16 to 20. New cstate: current_term: 20 leader_uuid: "647a343a542d4a139fe106c19095129b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } }
I20260709 13:22:42.583405 16041 catalog_manager.cc:5697] T ada5a004ba7c46819051e3ccec71ffdb P 90f3da57796b488c9087a2f523adf045 reported cstate change: term changed from 19 to 25, leader changed from cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) to 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130). New cstate: current_term: 25 leader_uuid: "1ca2725d290e47d68b1a4b609c9816ac" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
W20260709 13:22:42.585474 16790 consensus_peers.cc:597] T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
I20260709 13:22:42.587599 16041 catalog_manager.cc:5697] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 reported cstate change: term changed from 21 to 26, leader changed from cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) to 647a343a542d4a139fe106c19095129b (127.14.177.129). New cstate: current_term: 26 leader_uuid: "647a343a542d4a139fe106c19095129b" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:22:42.589229 16041 catalog_manager.cc:5697] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 90f3da57796b488c9087a2f523adf045 reported cstate change: term changed from 18 to 23. New cstate: current_term: 23 leader_uuid: "647a343a542d4a139fe106c19095129b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:22:42.599502 16041 catalog_manager.cc:5697] T 5a96373bb0e14dce8a266b2fc6c58507 P 90f3da57796b488c9087a2f523adf045 reported cstate change: term changed from 14 to 18. New cstate: current_term: 18 leader_uuid: "647a343a542d4a139fe106c19095129b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } }
I20260709 13:22:42.601073 16041 catalog_manager.cc:5697] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045 reported cstate change: term changed from 15 to 24. New cstate: current_term: 24 leader_uuid: "1ca2725d290e47d68b1a4b609c9816ac" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:22:42.604192 16041 catalog_manager.cc:5697] T c4f65cdf00b748bb963ee6e81ef43b79 P 90f3da57796b488c9087a2f523adf045 reported cstate change: term changed from 19 to 23. New cstate: current_term: 23 leader_uuid: "1ca2725d290e47d68b1a4b609c9816ac" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } }
I20260709 13:22:42.607651 24606 raft_consensus.cc:493] T 8cc959fb7f704fba9009c0595467d074 P 647a343a542d4a139fe106c19095129b [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:42.605588 16041 catalog_manager.cc:5697] T d16dfad0c3b540e5b4b86492ddf87208 P 90f3da57796b488c9087a2f523adf045 reported cstate change: term changed from 21 to 23, leader changed from 90f3da57796b488c9087a2f523adf045 (127.14.177.131) to 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130). New cstate: current_term: 23 leader_uuid: "1ca2725d290e47d68b1a4b609c9816ac" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } }
I20260709 13:22:42.608180 24606 raft_consensus.cc:515] T 8cc959fb7f704fba9009c0595467d074 P 647a343a542d4a139fe106c19095129b [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:42.609166 16041 catalog_manager.cc:5697] T 84f2da98864b4425bf7c0055951fdd4b P 90f3da57796b488c9087a2f523adf045 reported cstate change: term changed from 17 to 21. New cstate: current_term: 21 leader_uuid: "647a343a542d4a139fe106c19095129b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } }
I20260709 13:22:42.610262 24606 leader_election.cc:290] T 8cc959fb7f704fba9009c0595467d074 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:42.611687 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cc959fb7f704fba9009c0595467d074" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 22 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:42.613091 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cc959fb7f704fba9009c0595467d074" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 22 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
W20260709 13:22:42.613929 15046 thread.cc:527] Waited for 14000ms trying to join with heartbeat (tid 16176)
W20260709 13:22:42.621104 16790 consensus_peers.cc:597] T 2794af5b99984efebdee564035841cb0 P cb82cc7bd6db46d980c75237a1402860 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
I20260709 13:22:42.620656 16041 catalog_manager.cc:5697] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045 reported cstate change: term changed from 20 to 21, leader changed from 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130) to 647a343a542d4a139fe106c19095129b (127.14.177.129). New cstate: current_term: 21 leader_uuid: "647a343a542d4a139fe106c19095129b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } }
I20260709 13:22:42.622495 16115 leader_election.cc:304] T 8cc959fb7f704fba9009c0595467d074 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:22:42.622165 16041 catalog_manager.cc:5697] T 8638ee363273426db1393dd81e5c7b72 P 90f3da57796b488c9087a2f523adf045 reported cstate change: term changed from 21 to 24, leader changed from cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) to 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130). New cstate: current_term: 24 leader_uuid: "1ca2725d290e47d68b1a4b609c9816ac" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:22:42.623687 16041 catalog_manager.cc:5697] T 02838c86531d4f98bf58d1c22b8baf77 P 90f3da57796b488c9087a2f523adf045 reported cstate change: term changed from 19 to 24, leader changed from 90f3da57796b488c9087a2f523adf045 (127.14.177.131) to 647a343a542d4a139fe106c19095129b (127.14.177.129). New cstate: current_term: 24 leader_uuid: "647a343a542d4a139fe106c19095129b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } }
I20260709 13:22:42.624217 24606 raft_consensus.cc:2749] T 8cc959fb7f704fba9009c0595467d074 P 647a343a542d4a139fe106c19095129b [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:22:42.625739 16041 catalog_manager.cc:5697] T e4dc6b0541ca431ebc757beac00521f1 P 90f3da57796b488c9087a2f523adf045 reported cstate change: term changed from 22 to 24. New cstate: current_term: 24 leader_uuid: "90f3da57796b488c9087a2f523adf045" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } health_report { overall_health: HEALTHY } } }
I20260709 13:22:42.627151 16041 catalog_manager.cc:5697] T d2ab0ac8347641e6955e9bce7d90c1a2 P 90f3da57796b488c9087a2f523adf045 reported cstate change: term changed from 20 to 21, leader changed from 90f3da57796b488c9087a2f523adf045 (127.14.177.131) to 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130). New cstate: current_term: 21 leader_uuid: "1ca2725d290e47d68b1a4b609c9816ac" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:22:42.628835 16041 catalog_manager.cc:5697] T 18ad7e3af5504571a69d10c00750a01c P 90f3da57796b488c9087a2f523adf045 reported cstate change: term changed from 17 to 19. New cstate: current_term: 19 leader_uuid: "1ca2725d290e47d68b1a4b609c9816ac" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } }
I20260709 13:22:42.630029 16041 catalog_manager.cc:5697] T 2794af5b99984efebdee564035841cb0 P 90f3da57796b488c9087a2f523adf045 reported cstate change: term changed from 19 to 20, leader changed from 90f3da57796b488c9087a2f523adf045 (127.14.177.131) to cb82cc7bd6db46d980c75237a1402860 (127.14.177.132). New cstate: current_term: 20 leader_uuid: "cb82cc7bd6db46d980c75237a1402860" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:22:42.636181 16041 catalog_manager.cc:5697] T c0eab25ce4c64e018a041beeda2867db P 90f3da57796b488c9087a2f523adf045 reported cstate change: term changed from 18 to 24, leader changed from 647a343a542d4a139fe106c19095129b (127.14.177.129) to 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130). New cstate: current_term: 24 leader_uuid: "1ca2725d290e47d68b1a4b609c9816ac" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } }
W20260709 13:22:42.639820 16265 consensus_peers.cc:597] T 434bff3082d141838068f323bdb0d1ed P 90f3da57796b488c9087a2f523adf045 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
W20260709 13:22:42.684755 16790 consensus_peers.cc:597] T 66f75124215f4ca6bef831e174a792d8 P cb82cc7bd6db46d980c75237a1402860 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
W20260709 13:22:42.723479 16265 consensus_peers.cc:597] T 7c2095a5f4ad49408dbc7f69c606120d P 90f3da57796b488c9087a2f523adf045 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
W20260709 13:22:42.744990 16265 consensus_peers.cc:597] T 26354361e9b044cea1c2b86c4882fe92 P 90f3da57796b488c9087a2f523adf045 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
W20260709 13:22:42.795524 16191 consensus_peers.cc:597] T 829b4ad8c5a54eababeea1181b960d4b P 1ca2725d290e47d68b1a4b609c9816ac -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:22:42.805627 24541 raft_consensus.cc:493] T 23f82224f4564fea93572e8206bad1fe P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:42.806231 24541 raft_consensus.cc:515] T 23f82224f4564fea93572e8206bad1fe P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:22:42.808470 16790 consensus_peers.cc:597] T 23f82224f4564fea93572e8206bad1fe P cb82cc7bd6db46d980c75237a1402860 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
I20260709 13:22:42.810190 24541 leader_election.cc:290] T 23f82224f4564fea93572e8206bad1fe P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:42.811327 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f82224f4564fea93572e8206bad1fe" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:42.811681 16823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23f82224f4564fea93572e8206bad1fe" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:42.816104 16115 leader_election.cc:304] T 23f82224f4564fea93572e8206bad1fe P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860
I20260709 13:22:42.817425 24541 raft_consensus.cc:2749] T 23f82224f4564fea93572e8206bad1fe P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:22:42.869208 16790 consensus_peers.cc:597] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:22:42.914142 24606 raft_consensus.cc:493] T 434bff3082d141838068f323bdb0d1ed P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:22:42.914917 24606 raft_consensus.cc:515] T 434bff3082d141838068f323bdb0d1ed P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:42.918557 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434bff3082d141838068f323bdb0d1ed" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:42.921361 24606 leader_election.cc:290] T 434bff3082d141838068f323bdb0d1ed P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:42.922559 16823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434bff3082d141838068f323bdb0d1ed" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:42.924299 16117 leader_election.cc:304] T 434bff3082d141838068f323bdb0d1ed P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860
I20260709 13:22:42.925612 24606 raft_consensus.cc:2749] T 434bff3082d141838068f323bdb0d1ed P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:22:42.942922 16042 catalog_manager.cc:5697] T af57996a8be84d398f0f023e8d08ef77 P cb82cc7bd6db46d980c75237a1402860 reported cstate change: term changed from 18 to 22. New cstate: current_term: 22 leader_uuid: "1ca2725d290e47d68b1a4b609c9816ac" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:22:42.944694 16042 catalog_manager.cc:5697] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 reported cstate change: term changed from 20 to 23, leader changed from cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) to 647a343a542d4a139fe106c19095129b (127.14.177.129). New cstate: current_term: 23 leader_uuid: "647a343a542d4a139fe106c19095129b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:22:42.946113 16042 catalog_manager.cc:5697] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 reported cstate change: term changed from 22 to 24. New cstate: current_term: 24 leader_uuid: "cb82cc7bd6db46d980c75237a1402860" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:22:42.951049 16042 catalog_manager.cc:5697] T 829b4ad8c5a54eababeea1181b960d4b P cb82cc7bd6db46d980c75237a1402860 reported cstate change: term changed from 19 to 21. New cstate: current_term: 21 leader_uuid: "1ca2725d290e47d68b1a4b609c9816ac" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:22:42.954016 16042 catalog_manager.cc:5697] T 66f75124215f4ca6bef831e174a792d8 P cb82cc7bd6db46d980c75237a1402860 reported cstate change: term changed from 16 to 21. New cstate: current_term: 21 leader_uuid: "cb82cc7bd6db46d980c75237a1402860" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:22:42.985152 16042 catalog_manager.cc:5697] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 reported cstate change: term changed from 25 to 27. New cstate: current_term: 27 leader_uuid: "1ca2725d290e47d68b1a4b609c9816ac" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:22:42.987008 16042 catalog_manager.cc:5697] T ee9ed67006144797b1de6ea8c2013a68 P cb82cc7bd6db46d980c75237a1402860 reported cstate change: term changed from 19 to 21, leader changed from cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) to 647a343a542d4a139fe106c19095129b (127.14.177.129). New cstate: current_term: 21 leader_uuid: "647a343a542d4a139fe106c19095129b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:22:42.989382 16042 catalog_manager.cc:5697] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860 reported cstate change: term changed from 18 to 19, leader changed from cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) to 647a343a542d4a139fe106c19095129b (127.14.177.129). New cstate: current_term: 19 leader_uuid: "647a343a542d4a139fe106c19095129b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:22:43.008311 16041 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:49252 (request call id 294) took 2402 ms. Trace:
I20260709 13:22:43.008781 16041 rpcz_store.cc:276] 0709 13:22:40.605636 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:22:41.152291 (+546655us) service_pool.cc:224] Handling call
0709 13:22:43.008204 (+1855913us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:22:42.677587 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:22:42.680533 (+  2946us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:22:42.680572 (+    39us) write_op.cc:435] Acquired schema lock
0709 13:22:42.680586 (+    14us) tablet.cc:662] Decoding operations
0709 13:22:42.683375 (+  2789us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:22:42.683434 (+    59us) write_op.cc:641] Partition lock acquired for write op
0709 13:22:42.683465 (+    31us) tablet.cc:685] Acquiring locks for 43 operations
0709 13:22:42.686506 (+  3041us) tablet.cc:701] Row locks acquired
0709 13:22:42.686525 (+    19us) write_op.cc:260] PREPARE: finished
0709 13:22:42.686658 (+   133us) write_op.cc:270] Start()
0709 13:22:42.686775 (+   117us) write_op.cc:276] Timestamp: P: 1783603362686613 usec, L: 0
0709 13:22:42.686810 (+    35us) op_driver.cc:348] REPLICATION: starting
0709 13:22:42.687584 (+   774us) log.cc:844] Serialized 17055 byte log entry
0709 13:22:42.695135 (+  7551us) op_driver.cc:464] REPLICATION: finished
0709 13:22:42.695653 (+   518us) write_op.cc:301] APPLY: starting
0709 13:22:42.695734 (+    81us) tablet.cc:1367] starting BulkCheckPresence
0709 13:22:42.782282 (+ 86548us) tablet.cc:1370] finished BulkCheckPresence
0709 13:22:42.782397 (+   115us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:22:42.879407 (+ 97010us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:22:42.891485 (+ 12078us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=86,key_file_lookups=86,delta_file_lookups=0,mrs_lookups=0
0709 13:22:42.891532 (+    47us) write_op.cc:312] APPLY: finished
0709 13:22:42.892169 (+   637us) log.cc:844] Serialized 364 byte log entry
0709 13:22:42.892996 (+   827us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:22:42.893302 (+   306us) write_op.cc:454] Released schema lock
0709 13:22:42.893377 (+    75us) write_op.cc:341] FINISH: Updating metrics
0709 13:22:42.893754 (+   377us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:22:42.893779 (+    25us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":21741,"reported_tablets":49,"child_traces":[["op",{"apply.queue_time_us":293,"apply.run_cpu_time_us":48722,"apply.run_wall_time_us":198045,"cfile_cache_hit":172,"cfile_cache_hit_bytes":179568,"num_ops":43,"prepare.queue_time_us":106,"prepare.run_cpu_time_us":7857,"prepare.run_wall_time_us":67973,"replication_time_us":8197}]]}
I20260709 13:22:43.027021 16042 catalog_manager.cc:5697] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 reported cstate change: term changed from 26 to 27, leader changed from cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) to 647a343a542d4a139fe106c19095129b (127.14.177.129). New cstate: current_term: 27 leader_uuid: "647a343a542d4a139fe106c19095129b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:22:43.030222 24541 raft_consensus.cc:493] T ecc1e363230043fe9fef390f0b979f20 P 647a343a542d4a139fe106c19095129b [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:43.030866 24541 raft_consensus.cc:515] T ecc1e363230043fe9fef390f0b979f20 P 647a343a542d4a139fe106c19095129b [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:43.033870 24541 leader_election.cc:290] T ecc1e363230043fe9fef390f0b979f20 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:43.036801 16295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 28 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:43.037772 16295 raft_consensus.cc:2410] T ecc1e363230043fe9fef390f0b979f20 P 90f3da57796b488c9087a2f523adf045 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 647a343a542d4a139fe106c19095129b for term 28 because replica has last-logged OpId of term: 27 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 23 index: 14.
I20260709 13:22:43.037598 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 28 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:43.040480 16115 leader_election.cc:304] T ecc1e363230043fe9fef390f0b979f20 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:22:43.040174 16042 catalog_manager.cc:5697] T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860 reported cstate change: term changed from 23 to 24, leader changed from 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130) to cb82cc7bd6db46d980c75237a1402860 (127.14.177.132). New cstate: current_term: 24 leader_uuid: "cb82cc7bd6db46d980c75237a1402860" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:22:43.043064 24459 raft_consensus.cc:2749] T ecc1e363230043fe9fef390f0b979f20 P 647a343a542d4a139fe106c19095129b [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 13:22:43.044754 16818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 28 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:43.045138 16042 catalog_manager.cc:5697] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860 reported cstate change: term changed from 18 to 22, leader changed from 647a343a542d4a139fe106c19095129b (127.14.177.129) to cb82cc7bd6db46d980c75237a1402860 (127.14.177.132). New cstate: current_term: 22 leader_uuid: "cb82cc7bd6db46d980c75237a1402860" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:22:43.162272 16042 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:58400 (request call id 196) took 2087 ms. Trace:
W20260709 13:22:43.169001 16790 consensus_peers.cc:597] T 57a744f5aaf944678a63469cbf514015 P cb82cc7bd6db46d980c75237a1402860 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
I20260709 13:22:43.162828 16042 rpcz_store.cc:276] 0709 13:22:41.074424 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:22:42.116805 (+1042381us) service_pool.cc:224] Handling call
0709 13:22:43.162170 (+1045365us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:22:43.086083 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:22:43.086614 (+   531us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:22:43.086645 (+    31us) write_op.cc:435] Acquired schema lock
0709 13:22:43.086663 (+    18us) tablet.cc:662] Decoding operations
0709 13:22:43.087684 (+  1021us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:22:43.087740 (+    56us) write_op.cc:641] Partition lock acquired for write op
0709 13:22:43.087768 (+    28us) tablet.cc:685] Acquiring locks for 11 operations
0709 13:22:43.088727 (+   959us) tablet.cc:701] Row locks acquired
0709 13:22:43.088745 (+    18us) write_op.cc:260] PREPARE: finished
0709 13:22:43.088891 (+   146us) write_op.cc:270] Start()
0709 13:22:43.089009 (+   118us) write_op.cc:276] Timestamp: P: 1783603363088846 usec, L: 0
0709 13:22:43.089031 (+    22us) op_driver.cc:348] REPLICATION: starting
0709 13:22:43.089633 (+   602us) log.cc:844] Serialized 4400 byte log entry
0709 13:22:43.093216 (+  3583us) op_driver.cc:464] REPLICATION: finished
0709 13:22:43.093679 (+   463us) write_op.cc:301] APPLY: starting
0709 13:22:43.093756 (+    77us) tablet.cc:1367] starting BulkCheckPresence
0709 13:22:43.101391 (+  7635us) tablet.cc:1370] finished BulkCheckPresence
0709 13:22:43.101498 (+   107us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:22:43.107914 (+  6416us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:22:43.108122 (+   208us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=22,key_file_lookups=22,delta_file_lookups=0,mrs_lookups=0
0709 13:22:43.108165 (+    43us) write_op.cc:312] APPLY: finished
0709 13:22:43.108461 (+   296us) log.cc:844] Serialized 105 byte log entry
0709 13:22:43.109226 (+   765us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:22:43.109353 (+   127us) write_op.cc:454] Released schema lock
0709 13:22:43.109405 (+    52us) write_op.cc:341] FINISH: Updating metrics
0709 13:22:43.143414 (+ 34009us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:22:43.143445 (+    31us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":76873,"reported_tablets":34,"spinlock_wait_cycles":17792,"child_traces":[["op",{"apply.queue_time_us":255,"apply.run_cpu_time_us":13434,"apply.run_wall_time_us":49691,"cfile_cache_hit":44,"cfile_cache_hit_bytes":45936,"num_ops":11,"prepare.queue_time_us":98,"prepare.run_cpu_time_us":3853,"prepare.run_wall_time_us":6002,"replication_time_us":4070}]]}
W20260709 13:22:43.179630 16116 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 100720us
W20260709 13:22:43.227873 16265 consensus_peers.cc:597] T 8cc959fb7f704fba9009c0595467d074 P 90f3da57796b488c9087a2f523adf045 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
W20260709 13:22:43.228286 16115 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 43526.4us
W20260709 13:22:43.228600 16265 consensus_peers.cc:597] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 90f3da57796b488c9087a2f523adf045 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
W20260709 13:22:43.229393 16265 consensus_peers.cc:597] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:22:43.233840 24630 raft_consensus.cc:493] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:43.234661 24630 raft_consensus.cc:515] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:43.239600 24630 leader_election.cc:290] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:43.246958 16818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5d59407eb545b895d8d922e0dd69fd" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
W20260709 13:22:43.263955 16191 consensus_peers.cc:597] T e5ad4eb281564936a59f41e387a47156 P 1ca2725d290e47d68b1a4b609c9816ac -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:22:43.272611 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5d59407eb545b895d8d922e0dd69fd" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:43.276964 16116 leader_election.cc:304] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860
W20260709 13:22:43.278441 16191 consensus_peers.cc:597] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:22:43.281520 24632 raft_consensus.cc:2749] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:22:43.302286 24630 raft_consensus.cc:493] T 57a744f5aaf944678a63469cbf514015 P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:43.302867 24630 raft_consensus.cc:515] T 57a744f5aaf944678a63469cbf514015 P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:43.306006 24630 leader_election.cc:290] T 57a744f5aaf944678a63469cbf514015 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:43.307472 16295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57a744f5aaf944678a63469cbf514015" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 21 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:43.310834 16818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57a744f5aaf944678a63469cbf514015" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 21 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:43.314239 16042 catalog_manager.cc:5697] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860 reported cstate change: config changed from index 4 to 16, VOTER 90f3da57796b488c9087a2f523adf045 (127.14.177.131) evicted. New cstate: current_term: 27 leader_uuid: "cb82cc7bd6db46d980c75237a1402860" committed_config { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:22:43.315263 16117 leader_election.cc:304] T 57a744f5aaf944678a63469cbf514015 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860
W20260709 13:22:43.315795 16046 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:49246 (request call id 332) took 88840 ms (client timeout 15000 ms). Trace:
I20260709 13:22:43.316823 24632 raft_consensus.cc:2749] T 57a744f5aaf944678a63469cbf514015 P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:22:43.316342 16046 rpcz_store.cc:269] 0709 13:21:14.511405 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:14.511849 (+   444us) service_pool.cc:224] Handling call
0709 13:22:43.297318 (+88785469us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7,"reported_tablets":48,"spinlock_wait_cycles":30080}
W20260709 13:22:43.379125 16047 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:49252 (request call id 292) took 45039 ms (client timeout 15000 ms). Trace:
W20260709 13:22:43.385766 16047 rpcz_store.cc:269] 0709 13:21:58.339893 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:58.340170 (+   277us) service_pool.cc:224] Handling call
0709 13:22:43.378930 (+45038760us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":40591,"reported_tablets":49}
W20260709 13:22:43.391590 16191 consensus_peers.cc:597] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
W20260709 13:22:43.405359 16191 consensus_peers.cc:597] T af57996a8be84d398f0f023e8d08ef77 P 1ca2725d290e47d68b1a4b609c9816ac -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:22:43.423484 16286 tablet_service.cc:1558] Processing DeleteTablet for tablet ecc1e363230043fe9fef390f0b979f20 with delete_type TABLET_DATA_TOMBSTONED (TS 90f3da57796b488c9087a2f523adf045 not found in new config with opid_index 16) from {username='slave'} at 127.0.0.1:55454
I20260709 13:22:43.432268 24630 raft_consensus.cc:493] T 829b4ad8c5a54eababeea1181b960d4b P 647a343a542d4a139fe106c19095129b [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:43.432926 24630 raft_consensus.cc:515] T 829b4ad8c5a54eababeea1181b960d4b P 647a343a542d4a139fe106c19095129b [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:43.436863 16818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "829b4ad8c5a54eababeea1181b960d4b" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 22 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:43.443269 24630 leader_election.cc:290] T 829b4ad8c5a54eababeea1181b960d4b P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
W20260709 13:22:43.448246 16049 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:49252 (request call id 289) took 110208 ms (client timeout 15000 ms). Trace:
W20260709 13:22:43.448756 16049 rpcz_store.cc:269] 0709 13:20:53.240093 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:53.248634 (+  8541us) service_pool.cc:224] Handling call
0709 13:20:53.360406 (+111772us) spinlock_profiling.cc:243] Waited 1.26 ms on lock 0x7b54003cdf54. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac9090 00007f69b0b67f3a 00007f69b691e1cc 00007f69b6a1c37e 00007f69b4edd2a4 00007f69b4edd232 00007f69b4edd161 00007f69b4edd0dc 00007f69b4edc392 00007f69aee9a48c 00007f69aee998c6
0709 13:22:43.448021 (+110087615us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":275071,"reported_tablets":49,"spinlock_wait_cycles":2902272}
I20260709 13:22:43.463353 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "829b4ad8c5a54eababeea1181b960d4b" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 22 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:43.473840 16116 leader_election.cc:304] T 829b4ad8c5a54eababeea1181b960d4b P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860
I20260709 13:22:43.475277 24632 raft_consensus.cc:2749] T 829b4ad8c5a54eababeea1181b960d4b P 647a343a542d4a139fe106c19095129b [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:22:43.492383 24635 tablet_replica.cc:333] T ecc1e363230043fe9fef390f0b979f20 P 90f3da57796b488c9087a2f523adf045: stopping tablet replica
I20260709 13:22:43.493835 24635 raft_consensus.cc:2243] T ecc1e363230043fe9fef390f0b979f20 P 90f3da57796b488c9087a2f523adf045 [term 27 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:43.494984 24635 pending_rounds.cc:70] T ecc1e363230043fe9fef390f0b979f20 P 90f3da57796b488c9087a2f523adf045: Trying to abort 3 pending ops.
I20260709 13:22:43.495395 24635 pending_rounds.cc:77] T ecc1e363230043fe9fef390f0b979f20 P 90f3da57796b488c9087a2f523adf045: Aborting op as it isn't in flight: id { term: 27 index: 15 } timestamp: 7305639346261733376 op_type: NO_OP noop_request { }
I20260709 13:22:43.496198 24635 raft_consensus.cc:2889] T ecc1e363230043fe9fef390f0b979f20 P 90f3da57796b488c9087a2f523adf045 [term 27 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:22:43.496644 24635 pending_rounds.cc:77] T ecc1e363230043fe9fef390f0b979f20 P 90f3da57796b488c9087a2f523adf045: Aborting op as it isn't in flight: id { term: 23 index: 14 } timestamp: 7305639267699224576 op_type: NO_OP noop_request { }
I20260709 13:22:43.497151 24635 raft_consensus.cc:2889] T ecc1e363230043fe9fef390f0b979f20 P 90f3da57796b488c9087a2f523adf045 [term 27 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:22:43.497555 24635 pending_rounds.cc:77] T ecc1e363230043fe9fef390f0b979f20 P 90f3da57796b488c9087a2f523adf045: Aborting op as it isn't in flight: id { term: 21 index: 13 } timestamp: 7305639218047377408 op_type: NO_OP noop_request { }
I20260709 13:22:43.497959 24635 raft_consensus.cc:2889] T ecc1e363230043fe9fef390f0b979f20 P 90f3da57796b488c9087a2f523adf045 [term 27 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:22:43.498656 24635 raft_consensus.cc:2272] T ecc1e363230043fe9fef390f0b979f20 P 90f3da57796b488c9087a2f523adf045 [term 27 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:43.502490 24635 ts_tablet_manager.cc:1916] T ecc1e363230043fe9fef390f0b979f20 P 90f3da57796b488c9087a2f523adf045: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:22:43.534760 24635 ts_tablet_manager.cc:1929] T ecc1e363230043fe9fef390f0b979f20 P 90f3da57796b488c9087a2f523adf045: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 27.15
W20260709 13:22:43.534678 16048 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:49250 (request call id 325) took 16518 ms (client timeout 15000 ms). Trace:
W20260709 13:22:43.539203 16191 consensus_peers.cc:597] T c0eab25ce4c64e018a041beeda2867db P 1ca2725d290e47d68b1a4b609c9816ac -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:22:43.537500 24635 log.cc:1199] T ecc1e363230043fe9fef390f0b979f20 P 90f3da57796b488c9087a2f523adf045: Deleting WAL directory at /tmp/dist-test-taskdH_Ma7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783603093801613-15046-0/minicluster-data/ts-2-root/wals/ecc1e363230043fe9fef390f0b979f20
W20260709 13:22:43.539443 16048 rpcz_store.cc:269] 0709 13:22:27.016985 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:22:37.511859 (+10494874us) service_pool.cc:224] Handling call
0709 13:22:43.528062 (+6016203us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10548,"reported_tablets":48}
I20260709 13:22:43.544335 16034 catalog_manager.cc:5028] TS 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389): tablet ecc1e363230043fe9fef390f0b979f20 (table test-workload [id=ed3d15d8c9ad46bebc4fe3a402d2e54f]) successfully deleted
W20260709 13:22:43.568809 16192 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 43921.3us
I20260709 13:22:43.599987 16040 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:49246 (request call id 335) took 7761 ms. Trace:
I20260709 13:22:43.615314 16040 rpcz_store.cc:276] 0709 13:22:35.838586 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:22:40.670803 (+4832217us) service_pool.cc:224] Handling call
0709 13:22:43.599896 (+2929093us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4077,"reported_tablets":48,"spinlock_wait_cycles":17280}
I20260709 13:22:43.620008 24630 raft_consensus.cc:493] T 2794af5b99984efebdee564035841cb0 P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:43.623822 24630 raft_consensus.cc:515] T 2794af5b99984efebdee564035841cb0 P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:43.626056 24630 leader_election.cc:290] T 2794af5b99984efebdee564035841cb0 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:43.627822 15046 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:22:43.628973 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2794af5b99984efebdee564035841cb0" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 21 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:43.630335 16823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2794af5b99984efebdee564035841cb0" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 21 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:43.631021 15046 tablet_replica.cc:333] T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b: stopping tablet replica
I20260709 13:22:43.632769 15046 raft_consensus.cc:2243] T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:43.633507 15046 pending_rounds.cc:70] T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b: Trying to abort 1 pending ops.
I20260709 13:22:43.633771 15046 pending_rounds.cc:77] T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b: Aborting op as it isn't in flight: id { term: 20 index: 16 } timestamp: 7305639327321866240 op_type: NO_OP noop_request { }
I20260709 13:22:43.634161 15046 raft_consensus.cc:2889] T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b [term 21 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:22:43.634539 15046 raft_consensus.cc:2272] T 66f75124215f4ca6bef831e174a792d8 P 647a343a542d4a139fe106c19095129b [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:43.637950 15046 tablet_replica.cc:333] T 57a744f5aaf944678a63469cbf514015 P 647a343a542d4a139fe106c19095129b: stopping tablet replica
I20260709 13:22:43.640817 15046 raft_consensus.cc:2243] T 57a744f5aaf944678a63469cbf514015 P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:43.641546 15046 pending_rounds.cc:70] T 57a744f5aaf944678a63469cbf514015 P 647a343a542d4a139fe106c19095129b: Trying to abort 1 pending ops.
I20260709 13:22:43.641904 15046 pending_rounds.cc:77] T 57a744f5aaf944678a63469cbf514015 P 647a343a542d4a139fe106c19095129b: Aborting op as it isn't in flight: id { term: 18 index: 14 } timestamp: 7305639264324698112 op_type: NO_OP noop_request { }
I20260709 13:22:43.642338 15046 raft_consensus.cc:2889] T 57a744f5aaf944678a63469cbf514015 P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:22:43.642767 15046 raft_consensus.cc:2272] T 57a744f5aaf944678a63469cbf514015 P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:43.646137 16115 leader_election.cc:304] T 2794af5b99984efebdee564035841cb0 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860
I20260709 13:22:43.649077 24541 raft_consensus.cc:2749] T 2794af5b99984efebdee564035841cb0 P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:22:43.651285 16191 consensus_peers.cc:597] T 18ad7e3af5504571a69d10c00750a01c P 1ca2725d290e47d68b1a4b609c9816ac -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:22:43.653780 15046 tablet_replica.cc:333] T e5ad4eb281564936a59f41e387a47156 P 647a343a542d4a139fe106c19095129b: stopping tablet replica
W20260709 13:22:43.654839 16045 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:49246 (request call id 331) took 114755 ms (client timeout 15000 ms). Trace:
W20260709 13:22:43.663023 16045 rpcz_store.cc:269] 0709 13:20:48.899696 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:48.902473 (+  2777us) service_pool.cc:224] Handling call
0709 13:20:49.117446 (+214973us) spinlock_profiling.cc:243] Waited 20.3 ms on lock 0x7b54003cdf54. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac9090 00007f69b0b67f3a 00007f69b691e1cc 00007f69b6a1c37e 00007f69b4edd2a4 00007f69b4edd232 00007f69b4edd161 00007f69b4edd0dc 00007f69b4edc392 00007f69aee9a48c 00007f69aee998c6
0709 13:22:43.654586 (+114537140us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":11487,"reported_tablets":48,"spinlock_wait_cycles":46759680}
I20260709 13:22:43.655655 15046 raft_consensus.cc:2243] T e5ad4eb281564936a59f41e387a47156 P 647a343a542d4a139fe106c19095129b [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:43.665813 15046 raft_consensus.cc:2272] T e5ad4eb281564936a59f41e387a47156 P 647a343a542d4a139fe106c19095129b [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:43.669235 15046 tablet_replica.cc:333] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b: stopping tablet replica
I20260709 13:22:43.670676 15046 raft_consensus.cc:2243] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b [term 21 LEADER]: Raft consensus shutting down.
W20260709 13:22:43.675475 16790 consensus_peers.cc:597] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:22:43.678146 15046 raft_consensus.cc:2272] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 647a343a542d4a139fe106c19095129b [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:43.687994 15046 tablet_replica.cc:333] T 8d4ab870a05347868956aa8b841b2981 P 647a343a542d4a139fe106c19095129b: stopping tablet replica
I20260709 13:22:43.688723 15046 raft_consensus.cc:2243] T 8d4ab870a05347868956aa8b841b2981 P 647a343a542d4a139fe106c19095129b [term 22 LEADER]: Raft consensus shutting down.
I20260709 13:22:43.689620 15046 raft_consensus.cc:2272] T 8d4ab870a05347868956aa8b841b2981 P 647a343a542d4a139fe106c19095129b [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:43.699298 15046 tablet_replica.cc:333] T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b: stopping tablet replica
I20260709 13:22:43.702976 16044 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:49250 (request call id 326) took 1705 ms. Trace:
I20260709 13:22:43.702481 15046 raft_consensus.cc:2243] T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:43.703754 16044 rpcz_store.cc:276] 0709 13:22:41.997895 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:22:42.295585 (+297690us) service_pool.cc:224] Handling call
0709 13:22:43.702886 (+1407301us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8490,"reported_tablets":48,"spinlock_wait_cycles":9088}
I20260709 13:22:43.704520 15046 raft_consensus.cc:2272] T c4f65cdf00b748bb963ee6e81ef43b79 P 647a343a542d4a139fe106c19095129b [term 23 FOLLOWER]: Raft consensus is shut down!
W20260709 13:22:43.706368 16035 catalog_manager.cc:4755] TS 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): ChangeConfig:REMOVE_PEER RPC failed for tablet 806d7690c2b644959152074bb2cab0a7: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:22:43.708873 15046 tablet_replica.cc:333] T 08c74490c40243a88e62d1760b5cd814 P 647a343a542d4a139fe106c19095129b: stopping tablet replica
I20260709 13:22:43.710162 15046 raft_consensus.cc:2243] T 08c74490c40243a88e62d1760b5cd814 P 647a343a542d4a139fe106c19095129b [term 21 LEADER]: Raft consensus shutting down.
I20260709 13:22:43.711406 15046 raft_consensus.cc:2272] T 08c74490c40243a88e62d1760b5cd814 P 647a343a542d4a139fe106c19095129b [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:43.715335 15046 tablet_replica.cc:333] T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b: stopping tablet replica
I20260709 13:22:43.723414 24541 raft_consensus.cc:493] T fffab26427e24d398e2695aad40ca134 P 647a343a542d4a139fe106c19095129b [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:43.728106 24541 raft_consensus.cc:515] T fffab26427e24d398e2695aad40ca134 P 647a343a542d4a139fe106c19095129b [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:43.728562 15046 raft_consensus.cc:2243] T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b [term 27 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:43.729489 15046 pending_rounds.cc:70] T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b: Trying to abort 1 pending ops.
I20260709 13:22:43.729779 15046 pending_rounds.cc:77] T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b: Aborting op as it isn't in flight: id { term: 23 index: 15 } timestamp: 7305639233095344128 op_type: NO_OP noop_request { }
I20260709 13:22:43.730270 15046 raft_consensus.cc:2889] T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b [term 27 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:22:43.730664 15046 raft_consensus.cc:2272] T d2372981006d4b218215c28964dea623 P 647a343a542d4a139fe106c19095129b [term 27 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:43.732713 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fffab26427e24d398e2695aad40ca134" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 23 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:43.734706 15046 tablet_replica.cc:333] T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b: stopping tablet replica
I20260709 13:22:43.736579 15046 raft_consensus.cc:2243] T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:43.741544 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fffab26427e24d398e2695aad40ca134" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 23 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:43.749908 15046 raft_consensus.cc:2272] T 31969392ccdb43b491911a319e8bc7e1 P 647a343a542d4a139fe106c19095129b [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:43.754524 16115 leader_election.cc:304] T fffab26427e24d398e2695aad40ca134 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:22:43.755546 24630 raft_consensus.cc:2749] T fffab26427e24d398e2695aad40ca134 P 647a343a542d4a139fe106c19095129b [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:22:43.730567 24541 leader_election.cc:290] T fffab26427e24d398e2695aad40ca134 P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:43.774770 15046 tablet_replica.cc:333] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b: stopping tablet replica
I20260709 13:22:43.776047 15046 raft_consensus.cc:2243] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:43.776775 15046 raft_consensus.cc:2272] T 1d5d59407eb545b895d8d922e0dd69fd P 647a343a542d4a139fe106c19095129b [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:43.789016 15046 tablet_replica.cc:333] T 8158bbd17b994d2eb96f560bd0f1b88d P 647a343a542d4a139fe106c19095129b: stopping tablet replica
I20260709 13:22:43.790217 15046 raft_consensus.cc:2243] T 8158bbd17b994d2eb96f560bd0f1b88d P 647a343a542d4a139fe106c19095129b [term 23 LEADER]: Raft consensus shutting down.
I20260709 13:22:43.796042 15046 raft_consensus.cc:2272] T 8158bbd17b994d2eb96f560bd0f1b88d P 647a343a542d4a139fe106c19095129b [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:43.799814 15046 tablet_replica.cc:333] T 1df93779209748b9860f6d893dd3de53 P 647a343a542d4a139fe106c19095129b: stopping tablet replica
I20260709 13:22:43.800911 15046 raft_consensus.cc:2243] T 1df93779209748b9860f6d893dd3de53 P 647a343a542d4a139fe106c19095129b [term 21 LEADER]: Raft consensus shutting down.
I20260709 13:22:43.805297 15046 raft_consensus.cc:2272] T 1df93779209748b9860f6d893dd3de53 P 647a343a542d4a139fe106c19095129b [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:43.814270 15046 tablet_replica.cc:333] T 2794af5b99984efebdee564035841cb0 P 647a343a542d4a139fe106c19095129b: stopping tablet replica
I20260709 13:22:43.820598 15046 raft_consensus.cc:2243] T 2794af5b99984efebdee564035841cb0 P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:43.821354 15046 raft_consensus.cc:2272] T 2794af5b99984efebdee564035841cb0 P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:43.824782 15046 tablet_replica.cc:333] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b: stopping tablet replica
I20260709 13:22:43.836826 15046 raft_consensus.cc:2243] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [term 25 FOLLOWER]: Raft consensus shutting down.
W20260709 13:22:43.832224 16265 consensus_peers.cc:597] T e4dc6b0541ca431ebc757beac00521f1 P 90f3da57796b488c9087a2f523adf045 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:22:43.843269 15046 raft_consensus.cc:2272] T 19baa7bc367844c288bca8f81439c413 P 647a343a542d4a139fe106c19095129b [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:43.847229 15046 tablet_replica.cc:333] T 746aabf856ab401c9ef5e859371e7b67 P 647a343a542d4a139fe106c19095129b: stopping tablet replica
I20260709 13:22:43.848786 15046 raft_consensus.cc:2243] T 746aabf856ab401c9ef5e859371e7b67 P 647a343a542d4a139fe106c19095129b [term 19 LEADER]: Raft consensus shutting down.
I20260709 13:22:43.850190 15046 raft_consensus.cc:2272] T 746aabf856ab401c9ef5e859371e7b67 P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:43.864867 15046 tablet_replica.cc:333] T ee9ed67006144797b1de6ea8c2013a68 P 647a343a542d4a139fe106c19095129b: stopping tablet replica
I20260709 13:22:43.866111 15046 raft_consensus.cc:2243] T ee9ed67006144797b1de6ea8c2013a68 P 647a343a542d4a139fe106c19095129b [term 21 LEADER]: Raft consensus shutting down.
I20260709 13:22:43.869052 15046 raft_consensus.cc:2272] T ee9ed67006144797b1de6ea8c2013a68 P 647a343a542d4a139fe106c19095129b [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:43.873633 15046 tablet_replica.cc:333] T b87147731a68400e8cce9b879daa7c5d P 647a343a542d4a139fe106c19095129b: stopping tablet replica
W20260709 13:22:43.886190 16265 consensus_peers.cc:597] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:22:43.894560 15046 raft_consensus.cc:2243] T b87147731a68400e8cce9b879daa7c5d P 647a343a542d4a139fe106c19095129b [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:43.895536 15046 raft_consensus.cc:2272] T b87147731a68400e8cce9b879daa7c5d P 647a343a542d4a139fe106c19095129b [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:43.902781 24541 raft_consensus.cc:493] T 7c2095a5f4ad49408dbc7f69c606120d P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:43.905812 24541 raft_consensus.cc:515] T 7c2095a5f4ad49408dbc7f69c606120d P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:43.909444 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c2095a5f4ad49408dbc7f69c606120d" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 20 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:43.910851 15046 tablet_replica.cc:333] T af57996a8be84d398f0f023e8d08ef77 P 647a343a542d4a139fe106c19095129b: stopping tablet replica
I20260709 13:22:43.914049 24541 leader_election.cc:290] T 7c2095a5f4ad49408dbc7f69c606120d P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:43.914202 16295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c2095a5f4ad49408dbc7f69c606120d" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 20 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:43.918365 15046 raft_consensus.cc:2243] T af57996a8be84d398f0f023e8d08ef77 P 647a343a542d4a139fe106c19095129b [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:43.919312 16115 leader_election.cc:304] T 7c2095a5f4ad49408dbc7f69c606120d P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:22:43.921299 15046 pending_rounds.cc:70] T af57996a8be84d398f0f023e8d08ef77 P 647a343a542d4a139fe106c19095129b: Trying to abort 1 pending ops.
I20260709 13:22:43.921614 15046 pending_rounds.cc:77] T af57996a8be84d398f0f023e8d08ef77 P 647a343a542d4a139fe106c19095129b: Aborting op as it isn't in flight: id { term: 21 index: 14 } timestamp: 7305639310544654336 op_type: NO_OP noop_request { }
I20260709 13:22:43.921725 24541 raft_consensus.cc:2749] T 7c2095a5f4ad49408dbc7f69c606120d P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:22:43.922086 15046 raft_consensus.cc:2889] T af57996a8be84d398f0f023e8d08ef77 P 647a343a542d4a139fe106c19095129b [term 22 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:22:43.932938 15046 raft_consensus.cc:2272] T af57996a8be84d398f0f023e8d08ef77 P 647a343a542d4a139fe106c19095129b [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:43.938191 15046 tablet_replica.cc:333] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b: stopping tablet replica
I20260709 13:22:43.943536 15046 raft_consensus.cc:2243] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [term 27 LEADER]: Raft consensus shutting down.
I20260709 13:22:43.944756 15046 raft_consensus.cc:2272] T 62fe3346cd3c42c4a55e14af9073bca4 P 647a343a542d4a139fe106c19095129b [term 27 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:43.947266 24541 raft_consensus.cc:493] T 68a6838d82ef4cac83507f906139d90f P 647a343a542d4a139fe106c19095129b [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:43.960209 24541 raft_consensus.cc:515] T 68a6838d82ef4cac83507f906139d90f P 647a343a542d4a139fe106c19095129b [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:43.962564 15046 tablet_replica.cc:333] T 68a6838d82ef4cac83507f906139d90f P 647a343a542d4a139fe106c19095129b: stopping tablet replica
I20260709 13:22:43.963687 15046 raft_consensus.cc:2243] T 68a6838d82ef4cac83507f906139d90f P 647a343a542d4a139fe106c19095129b [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:43.963898 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68a6838d82ef4cac83507f906139d90f" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 23 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:43.968869 16295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68a6838d82ef4cac83507f906139d90f" candidate_uuid: "647a343a542d4a139fe106c19095129b" candidate_term: 23 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:43.971385 15046 pending_rounds.cc:70] T 68a6838d82ef4cac83507f906139d90f P 647a343a542d4a139fe106c19095129b: Trying to abort 1 pending ops.
I20260709 13:22:43.971786 15046 pending_rounds.cc:77] T 68a6838d82ef4cac83507f906139d90f P 647a343a542d4a139fe106c19095129b: Aborting op as it isn't in flight: id { term: 19 index: 11 } timestamp: 7305639240884482048 op_type: NO_OP noop_request { }
I20260709 13:22:43.972177 15046 raft_consensus.cc:2889] T 68a6838d82ef4cac83507f906139d90f P 647a343a542d4a139fe106c19095129b [term 22 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:22:43.972610 15046 raft_consensus.cc:2272] T 68a6838d82ef4cac83507f906139d90f P 647a343a542d4a139fe106c19095129b [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:43.984144 16115 leader_election.cc:304] T 68a6838d82ef4cac83507f906139d90f P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 647a343a542d4a139fe106c19095129b; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
W20260709 13:22:44.000276 16115 raft_consensus.cc:2686] T 68a6838d82ef4cac83507f906139d90f P 647a343a542d4a139fe106c19095129b: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260709 13:22:44.018431 24541 leader_election.cc:290] T 68a6838d82ef4cac83507f906139d90f P 647a343a542d4a139fe106c19095129b [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:44.022375 15046 tablet_replica.cc:333] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 647a343a542d4a139fe106c19095129b: stopping tablet replica
I20260709 13:22:44.028232 15046 raft_consensus.cc:2243] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 647a343a542d4a139fe106c19095129b [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:44.031594 15046 pending_rounds.cc:70] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 647a343a542d4a139fe106c19095129b: Trying to abort 1 pending ops.
I20260709 13:22:44.031965 15046 pending_rounds.cc:77] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 647a343a542d4a139fe106c19095129b: Aborting op as it isn't in flight: id { term: 21 index: 14 } timestamp: 7305639269867565056 op_type: NO_OP noop_request { }
I20260709 13:22:44.032416 15046 raft_consensus.cc:2889] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 647a343a542d4a139fe106c19095129b [term 21 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:22:44.034833 15046 raft_consensus.cc:2272] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 647a343a542d4a139fe106c19095129b [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:44.037964 15046 tablet_replica.cc:333] T ecc1e363230043fe9fef390f0b979f20 P 647a343a542d4a139fe106c19095129b: stopping tablet replica
I20260709 13:22:44.100720 15046 raft_consensus.cc:2243] T ecc1e363230043fe9fef390f0b979f20 P 647a343a542d4a139fe106c19095129b [term 27 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:44.107424 15046 raft_consensus.cc:2272] T ecc1e363230043fe9fef390f0b979f20 P 647a343a542d4a139fe106c19095129b [term 27 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:44.116565 15046 tablet_replica.cc:333] T c0eab25ce4c64e018a041beeda2867db P 647a343a542d4a139fe106c19095129b: stopping tablet replica
I20260709 13:22:44.128677 15046 raft_consensus.cc:2243] T c0eab25ce4c64e018a041beeda2867db P 647a343a542d4a139fe106c19095129b [term 23 FOLLOWER]: Raft consensus shutting down.
W20260709 13:22:44.129613 16191 consensus_peers.cc:597] T d16dfad0c3b540e5b4b86492ddf87208 P 1ca2725d290e47d68b1a4b609c9816ac -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:22:44.172254 15046 raft_consensus.cc:2272] T c0eab25ce4c64e018a041beeda2867db P 647a343a542d4a139fe106c19095129b [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:44.178807 15046 tablet_replica.cc:333] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b: stopping tablet replica
I20260709 13:22:44.181723 15046 raft_consensus.cc:2243] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b [term 23 LEADER]: Raft consensus shutting down.
I20260709 13:22:44.183247 15046 raft_consensus.cc:2272] T 806d7690c2b644959152074bb2cab0a7 P 647a343a542d4a139fe106c19095129b [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:44.188527 15046 tablet_replica.cc:333] T 829b4ad8c5a54eababeea1181b960d4b P 647a343a542d4a139fe106c19095129b: stopping tablet replica
I20260709 13:22:44.189553 15046 raft_consensus.cc:2243] T 829b4ad8c5a54eababeea1181b960d4b P 647a343a542d4a139fe106c19095129b [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:44.190218 15046 raft_consensus.cc:2272] T 829b4ad8c5a54eababeea1181b960d4b P 647a343a542d4a139fe106c19095129b [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:44.222440 15046 tablet_replica.cc:333] T 84f2da98864b4425bf7c0055951fdd4b P 647a343a542d4a139fe106c19095129b: stopping tablet replica
I20260709 13:22:44.223626 15046 raft_consensus.cc:2243] T 84f2da98864b4425bf7c0055951fdd4b P 647a343a542d4a139fe106c19095129b [term 21 LEADER]: Raft consensus shutting down.
I20260709 13:22:44.225214 15046 raft_consensus.cc:2272] T 84f2da98864b4425bf7c0055951fdd4b P 647a343a542d4a139fe106c19095129b [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:44.239377 15046 tablet_replica.cc:333] T d16dfad0c3b540e5b4b86492ddf87208 P 647a343a542d4a139fe106c19095129b: stopping tablet replica
I20260709 13:22:44.240628 15046 raft_consensus.cc:2243] T d16dfad0c3b540e5b4b86492ddf87208 P 647a343a542d4a139fe106c19095129b [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:44.246300 15046 raft_consensus.cc:2272] T d16dfad0c3b540e5b4b86492ddf87208 P 647a343a542d4a139fe106c19095129b [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:44.253895 15046 tablet_replica.cc:333] T 02838c86531d4f98bf58d1c22b8baf77 P 647a343a542d4a139fe106c19095129b: stopping tablet replica
I20260709 13:22:44.255101 15046 raft_consensus.cc:2243] T 02838c86531d4f98bf58d1c22b8baf77 P 647a343a542d4a139fe106c19095129b [term 24 LEADER]: Raft consensus shutting down.
I20260709 13:22:44.256632 15046 raft_consensus.cc:2272] T 02838c86531d4f98bf58d1c22b8baf77 P 647a343a542d4a139fe106c19095129b [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:44.273315 15046 tablet_replica.cc:333] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 647a343a542d4a139fe106c19095129b: stopping tablet replica
I20260709 13:22:44.274566 15046 raft_consensus.cc:2243] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 647a343a542d4a139fe106c19095129b [term 23 LEADER]: Raft consensus shutting down.
I20260709 13:22:44.275882 15046 raft_consensus.cc:2272] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 647a343a542d4a139fe106c19095129b [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:44.279438 15046 tablet_replica.cc:333] T 7c2095a5f4ad49408dbc7f69c606120d P 647a343a542d4a139fe106c19095129b: stopping tablet replica
I20260709 13:22:44.280597 15046 raft_consensus.cc:2243] T 7c2095a5f4ad49408dbc7f69c606120d P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:44.286140 15046 raft_consensus.cc:2272] T 7c2095a5f4ad49408dbc7f69c606120d P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:44.290174 15046 tablet_replica.cc:333] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b: stopping tablet replica
I20260709 13:22:44.298858 15046 raft_consensus.cc:2243] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b [term 26 LEADER]: Raft consensus shutting down.
I20260709 13:22:44.300357 15046 raft_consensus.cc:2272] T 0a9de347c711477e87c582d003f36925 P 647a343a542d4a139fe106c19095129b [term 26 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:44.305902 15046 tablet_replica.cc:333] T b6f07842d3aa47858df5665874c86f3a P 647a343a542d4a139fe106c19095129b: stopping tablet replica
I20260709 13:22:44.306995 15046 raft_consensus.cc:2243] T b6f07842d3aa47858df5665874c86f3a P 647a343a542d4a139fe106c19095129b [term 24 LEADER]: Raft consensus shutting down.
I20260709 13:22:44.308333 15046 raft_consensus.cc:2272] T b6f07842d3aa47858df5665874c86f3a P 647a343a542d4a139fe106c19095129b [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:44.311570 15046 tablet_replica.cc:333] T 23f82224f4564fea93572e8206bad1fe P 647a343a542d4a139fe106c19095129b: stopping tablet replica
I20260709 13:22:44.312662 15046 raft_consensus.cc:2243] T 23f82224f4564fea93572e8206bad1fe P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:44.313342 15046 raft_consensus.cc:2272] T 23f82224f4564fea93572e8206bad1fe P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:44.322300 15046 tablet_replica.cc:333] T 26354361e9b044cea1c2b86c4882fe92 P 647a343a542d4a139fe106c19095129b: stopping tablet replica
I20260709 13:22:44.323438 15046 raft_consensus.cc:2243] T 26354361e9b044cea1c2b86c4882fe92 P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:44.324164 15046 pending_rounds.cc:70] T 26354361e9b044cea1c2b86c4882fe92 P 647a343a542d4a139fe106c19095129b: Trying to abort 1 pending ops.
I20260709 13:22:44.324738 15046 pending_rounds.cc:77] T 26354361e9b044cea1c2b86c4882fe92 P 647a343a542d4a139fe106c19095129b: Aborting op as it isn't in flight: id { term: 17 index: 14 } timestamp: 7305639265717792768 op_type: NO_OP noop_request { }
I20260709 13:22:44.325220 15046 raft_consensus.cc:2889] T 26354361e9b044cea1c2b86c4882fe92 P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:22:44.325663 15046 raft_consensus.cc:2272] T 26354361e9b044cea1c2b86c4882fe92 P 647a343a542d4a139fe106c19095129b [term 19 FOLLOWER]: Raft consensus is shut down!
W20260709 13:22:44.338935 16265 consensus_peers.cc:597] T b87147731a68400e8cce9b879daa7c5d P 90f3da57796b488c9087a2f523adf045 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
I20260709 13:22:44.348100 15046 tablet_replica.cc:333] T 3884f864a96d428785895e5463e5728f P 647a343a542d4a139fe106c19095129b: stopping tablet replica
I20260709 13:22:44.349355 15046 raft_consensus.cc:2243] T 3884f864a96d428785895e5463e5728f P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:44.350091 15046 pending_rounds.cc:70] T 3884f864a96d428785895e5463e5728f P 647a343a542d4a139fe106c19095129b: Trying to abort 1 pending ops.
I20260709 13:22:44.350353 15046 pending_rounds.cc:77] T 3884f864a96d428785895e5463e5728f P 647a343a542d4a139fe106c19095129b: Aborting op as it isn't in flight: id { term: 14 index: 14 } timestamp: 7305639204671614976 op_type: NO_OP noop_request { }
I20260709 13:22:44.350725 15046 raft_consensus.cc:2889] T 3884f864a96d428785895e5463e5728f P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:22:44.351104 15046 raft_consensus.cc:2272] T 3884f864a96d428785895e5463e5728f P 647a343a542d4a139fe106c19095129b [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:44.354219 15046 tablet_replica.cc:333] T fffab26427e24d398e2695aad40ca134 P 647a343a542d4a139fe106c19095129b: stopping tablet replica
I20260709 13:22:44.355137 15046 raft_consensus.cc:2243] T fffab26427e24d398e2695aad40ca134 P 647a343a542d4a139fe106c19095129b [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:44.355880 15046 raft_consensus.cc:2272] T fffab26427e24d398e2695aad40ca134 P 647a343a542d4a139fe106c19095129b [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:44.358695 15046 tablet_replica.cc:333] T 9c61a39fd4f84424b129c1a6f9ba404d P 647a343a542d4a139fe106c19095129b: stopping tablet replica
I20260709 13:22:44.359575 15046 raft_consensus.cc:2243] T 9c61a39fd4f84424b129c1a6f9ba404d P 647a343a542d4a139fe106c19095129b [term 25 LEADER]: Raft consensus shutting down.
I20260709 13:22:44.360770 15046 raft_consensus.cc:2272] T 9c61a39fd4f84424b129c1a6f9ba404d P 647a343a542d4a139fe106c19095129b [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:44.363667 15046 tablet_replica.cc:333] T 5a96373bb0e14dce8a266b2fc6c58507 P 647a343a542d4a139fe106c19095129b: stopping tablet replica
I20260709 13:22:44.364547 15046 raft_consensus.cc:2243] T 5a96373bb0e14dce8a266b2fc6c58507 P 647a343a542d4a139fe106c19095129b [term 18 LEADER]: Raft consensus shutting down.
I20260709 13:22:44.365828 15046 raft_consensus.cc:2272] T 5a96373bb0e14dce8a266b2fc6c58507 P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:44.369199 15046 tablet_replica.cc:333] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b: stopping tablet replica
I20260709 13:22:44.370194 15046 raft_consensus.cc:2243] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b [term 26 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:44.370847 15046 pending_rounds.cc:70] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b: Trying to abort 3 pending ops.
I20260709 13:22:44.371111 15046 pending_rounds.cc:77] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b: Aborting op as it isn't in flight: id { term: 24 index: 14 } timestamp: 7305639296711651328 op_type: NO_OP noop_request { }
I20260709 13:22:44.371601 15046 raft_consensus.cc:2889] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b [term 26 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:22:44.371932 15046 pending_rounds.cc:77] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b: Aborting op as it isn't in flight: id { term: 22 index: 13 } timestamp: 7305639238901018624 op_type: NO_OP noop_request { }
I20260709 13:22:44.372303 15046 raft_consensus.cc:2889] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b [term 26 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:22:44.372623 15046 pending_rounds.cc:77] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b: Aborting op as it isn't in flight: id { term: 20 index: 12 } timestamp: 7305639197220360192 op_type: NO_OP noop_request { }
I20260709 13:22:44.372996 15046 raft_consensus.cc:2889] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b [term 26 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:22:44.373350 15046 raft_consensus.cc:2272] T c2f5f89186204242bf377f7c42b2e2da P 647a343a542d4a139fe106c19095129b [term 26 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:44.376618 15046 tablet_replica.cc:333] T 5a959dbecc2648eb85d1cc550f5a3245 P 647a343a542d4a139fe106c19095129b: stopping tablet replica
I20260709 13:22:44.377385 15046 raft_consensus.cc:2243] T 5a959dbecc2648eb85d1cc550f5a3245 P 647a343a542d4a139fe106c19095129b [term 23 LEADER]: Raft consensus shutting down.
I20260709 13:22:44.378549 15046 raft_consensus.cc:2272] T 5a959dbecc2648eb85d1cc550f5a3245 P 647a343a542d4a139fe106c19095129b [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:44.381376 15046 tablet_replica.cc:333] T 8cc959fb7f704fba9009c0595467d074 P 647a343a542d4a139fe106c19095129b: stopping tablet replica
I20260709 13:22:44.382252 15046 raft_consensus.cc:2243] T 8cc959fb7f704fba9009c0595467d074 P 647a343a542d4a139fe106c19095129b [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:44.382875 15046 pending_rounds.cc:70] T 8cc959fb7f704fba9009c0595467d074 P 647a343a542d4a139fe106c19095129b: Trying to abort 1 pending ops.
I20260709 13:22:44.383112 15046 pending_rounds.cc:77] T 8cc959fb7f704fba9009c0595467d074 P 647a343a542d4a139fe106c19095129b: Aborting op as it isn't in flight: id { term: 19 index: 14 } timestamp: 7305639297760362496 op_type: NO_OP noop_request { }
I20260709 13:22:44.383630 15046 raft_consensus.cc:2889] T 8cc959fb7f704fba9009c0595467d074 P 647a343a542d4a139fe106c19095129b [term 21 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:22:44.383970 15046 raft_consensus.cc:2272] T 8cc959fb7f704fba9009c0595467d074 P 647a343a542d4a139fe106c19095129b [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:44.401041 15046 tablet_replica.cc:333] T 8a878999c0024d2abe517484180290fc P 647a343a542d4a139fe106c19095129b: stopping tablet replica
I20260709 13:22:44.401887 15046 raft_consensus.cc:2243] T 8a878999c0024d2abe517484180290fc P 647a343a542d4a139fe106c19095129b [term 20 LEADER]: Raft consensus shutting down.
I20260709 13:22:44.403077 15046 raft_consensus.cc:2272] T 8a878999c0024d2abe517484180290fc P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:44.405954 15046 tablet_replica.cc:333] T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b: stopping tablet replica
I20260709 13:22:44.406709 15046 raft_consensus.cc:2243] T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b [term 23 LEADER]: Raft consensus shutting down.
I20260709 13:22:44.416082 15046 raft_consensus.cc:2272] T 08b7d1ab519044eda2d5e04a07242f25 P 647a343a542d4a139fe106c19095129b [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:44.428536 15046 tablet_replica.cc:333] T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b: stopping tablet replica
I20260709 13:22:44.429370 15046 raft_consensus.cc:2243] T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:44.430069 15046 raft_consensus.cc:2272] T e4dc6b0541ca431ebc757beac00521f1 P 647a343a542d4a139fe106c19095129b [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:44.432987 15046 tablet_replica.cc:333] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b: stopping tablet replica
I20260709 13:22:44.433758 15046 raft_consensus.cc:2243] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:44.434286 15046 pending_rounds.cc:70] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b: Trying to abort 1 pending ops.
I20260709 13:22:44.434579 15046 pending_rounds.cc:77] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b: Aborting op as it isn't in flight: id { term: 20 index: 16 } timestamp: 7305639249671806976 op_type: NO_OP noop_request { }
I20260709 13:22:44.434969 15046 raft_consensus.cc:2889] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b [term 24 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:22:44.449604 15046 raft_consensus.cc:2272] T 5718c623ebee4d2bb770f0f9ddb2f164 P 647a343a542d4a139fe106c19095129b [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:44.453042 15046 tablet_replica.cc:333] T 434bff3082d141838068f323bdb0d1ed P 647a343a542d4a139fe106c19095129b: stopping tablet replica
I20260709 13:22:44.453985 15046 raft_consensus.cc:2243] T 434bff3082d141838068f323bdb0d1ed P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:44.454711 15046 raft_consensus.cc:2272] T 434bff3082d141838068f323bdb0d1ed P 647a343a542d4a139fe106c19095129b [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:44.474143 15046 tablet_replica.cc:333] T 9e65676b21b748728ad39668cfe722ec P 647a343a542d4a139fe106c19095129b: stopping tablet replica
I20260709 13:22:44.474998 15046 raft_consensus.cc:2243] T 9e65676b21b748728ad39668cfe722ec P 647a343a542d4a139fe106c19095129b [term 22 LEADER]: Raft consensus shutting down.
I20260709 13:22:44.476274 15046 raft_consensus.cc:2272] T 9e65676b21b748728ad39668cfe722ec P 647a343a542d4a139fe106c19095129b [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:44.501467 15046 tablet_replica.cc:333] T 18ad7e3af5504571a69d10c00750a01c P 647a343a542d4a139fe106c19095129b: stopping tablet replica
I20260709 13:22:44.502332 15046 raft_consensus.cc:2243] T 18ad7e3af5504571a69d10c00750a01c P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:44.503084 15046 raft_consensus.cc:2272] T 18ad7e3af5504571a69d10c00750a01c P 647a343a542d4a139fe106c19095129b [term 18 FOLLOWER]: Raft consensus is shut down!
W20260709 13:22:44.551683 16191 consensus_peers.cc:597] T 68a6838d82ef4cac83507f906139d90f P 1ca2725d290e47d68b1a4b609c9816ac -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
W20260709 13:22:44.714087 16097 auto_rebalancer.cc:629] Failed to schedule move for tablet 8d4ab870a05347868956aa8b841b2981: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:22:44.776803 24578 raft_consensus.cc:493] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:22:44.777560 24578 raft_consensus.cc:515] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:44.781360 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ad5dbfa99ee4b8d9e28be92b3f36db5" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:44.783246 24578 leader_election.cc:290] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
W20260709 13:22:44.783597 16191 leader_election.cc:336] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:22:44.784082 16191 leader_election.cc:304] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:22:44.784741 16295 consensus_queue.cc:237] T 7c2095a5f4ad49408dbc7f69c606120d P 90f3da57796b488c9087a2f523adf045 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 19.9, Last appended by leader: 8, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } }
I20260709 13:22:44.799526 24578 raft_consensus.cc:2749] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:22:44.830327 24640 raft_consensus.cc:493] T 8d4ab870a05347868956aa8b841b2981 P 90f3da57796b488c9087a2f523adf045 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:22:44.831010 24640 raft_consensus.cc:515] T 8d4ab870a05347868956aa8b841b2981 P 90f3da57796b488c9087a2f523adf045 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:44.835374 24640 leader_election.cc:290] T 8d4ab870a05347868956aa8b841b2981 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:22:44.840757 24276 raft_consensus.cc:493] T 08c74490c40243a88e62d1760b5cd814 P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:22:44.841061 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d4ab870a05347868956aa8b841b2981" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:44.841428 24276 raft_consensus.cc:515] T 08c74490c40243a88e62d1760b5cd814 P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:22:44.848786 16266 consensus_peers.cc:597] T 7c2095a5f4ad49408dbc7f69c606120d P 90f3da57796b488c9087a2f523adf045 -> Peer cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923): Couldn't send request to peer cb82cc7bd6db46d980c75237a1402860. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 7c2095a5f4ad49408dbc7f69c606120d. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:22:44.851677 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c74490c40243a88e62d1760b5cd814" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:44.853471 24276 leader_election.cc:290] T 08c74490c40243a88e62d1760b5cd814 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
W20260709 13:22:44.853909 16191 leader_election.cc:336] T 08c74490c40243a88e62d1760b5cd814 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:22:44.854434 16191 leader_election.cc:304] T 08c74490c40243a88e62d1760b5cd814 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:22:44.855734 24276 raft_consensus.cc:2749] T 08c74490c40243a88e62d1760b5cd814 P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:22:44.858991 16265 leader_election.cc:336] T 8d4ab870a05347868956aa8b841b2981 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:22:44.860383 16265 leader_election.cc:304] T 8d4ab870a05347868956aa8b841b2981 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:22:44.863523 24640 raft_consensus.cc:2749] T 8d4ab870a05347868956aa8b841b2981 P 90f3da57796b488c9087a2f523adf045 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:22:44.868914 16224 raft_consensus.cc:1275] T 7c2095a5f4ad49408dbc7f69c606120d P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 19 index: 9. Preceding OpId from leader: term: 19 index: 10. (index mismatch)
I20260709 13:22:44.871551 24641 consensus_queue.cc:1048] T 7c2095a5f4ad49408dbc7f69c606120d P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
W20260709 13:22:44.875052 16265 consensus_peers.cc:597] T 7c2095a5f4ad49408dbc7f69c606120d P 90f3da57796b488c9087a2f523adf045 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:22:44.887308 24641 raft_consensus.cc:493] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 90f3da57796b488c9087a2f523adf045 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:22:44.888059 24641 raft_consensus.cc:515] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 90f3da57796b488c9087a2f523adf045 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:44.897387 24641 leader_election.cc:290] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:22:44.902024 24578 raft_consensus.cc:493] T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:22:44.902709 24578 raft_consensus.cc:515] T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:44.906785 24578 leader_election.cc:290] T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:44.906986 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d4ab870a05347868956aa8b841b2981" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:44.907773 16293 raft_consensus.cc:2468] T 8d4ab870a05347868956aa8b841b2981 P 90f3da57796b488c9087a2f523adf045 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 22.
I20260709 13:22:44.909145 16190 leader_election.cc:304] T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:22:44.909961 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ad5dbfa99ee4b8d9e28be92b3f36db5" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:44.910135 24578 raft_consensus.cc:2804] T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:22:44.910683 16217 raft_consensus.cc:2468] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 21.
I20260709 13:22:44.910951 24578 raft_consensus.cc:493] T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac [term 22 FOLLOWER]: Starting leader election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:22:44.911458 24578 raft_consensus.cc:3060] T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:22:44.912348 16265 leader_election.cc:304] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:22:44.913337 24641 raft_consensus.cc:2804] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 90f3da57796b488c9087a2f523adf045 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:22:44.913794 24641 raft_consensus.cc:493] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 90f3da57796b488c9087a2f523adf045 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:22:44.914224 24641 raft_consensus.cc:3060] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 90f3da57796b488c9087a2f523adf045 [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:22:44.921130 16035 catalog_manager.cc:4755] TS 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): ChangeConfig:REMOVE_PEER RPC failed for tablet 806d7690c2b644959152074bb2cab0a7: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111)
W20260709 13:22:44.922552 16265 leader_election.cc:336] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111)
W20260709 13:22:44.932600 16191 leader_election.cc:336] T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111)
I20260709 13:22:44.937080 24578 raft_consensus.cc:515] T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:44.945147 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d4ab870a05347868956aa8b841b2981" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:22:44.945899 16293 raft_consensus.cc:3060] T 8d4ab870a05347868956aa8b841b2981 P 90f3da57796b488c9087a2f523adf045 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:22:44.943765 24578 leader_election.cc:290] T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 23 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:44.948146 24628 raft_consensus.cc:2955] T 7c2095a5f4ad49408dbc7f69c606120d P 90f3da57796b488c9087a2f523adf045 [term 19 LEADER]: Committing config change with OpId 19.10: config changed from index -1 to 10, NON_VOTER cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) added. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } } }
I20260709 13:22:44.957299 24641 raft_consensus.cc:515] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 90f3da57796b488c9087a2f523adf045 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:44.959213 16293 raft_consensus.cc:2468] T 8d4ab870a05347868956aa8b841b2981 P 90f3da57796b488c9087a2f523adf045 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 23.
I20260709 13:22:44.960865 16190 leader_election.cc:304] T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:22:44.964287 24578 raft_consensus.cc:2804] T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac [term 23 FOLLOWER]: Leader election won for term 23
W20260709 13:22:44.967360 16191 leader_election.cc:336] T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 23 election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111)
W20260709 13:22:44.970774 16790 consensus_peers.cc:597] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:22:44.964301 24641 leader_election.cc:290] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 22 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:22:44.990084 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ad5dbfa99ee4b8d9e28be92b3f36db5" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:22:44.990674 16224 raft_consensus.cc:3060] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:22:45.017831 24578 raft_consensus.cc:697] T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac [term 23 LEADER]: Becoming Leader. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:22:45.021495 24578 consensus_queue.cc:237] T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 22.13, Last appended by leader: 13, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:45.034428 16224 raft_consensus.cc:2468] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 1ca2725d290e47d68b1a4b609c9816ac [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 22.
I20260709 13:22:45.040330 16265 leader_election.cc:304] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:22:45.016549 24653 raft_consensus.cc:493] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
W20260709 13:22:45.048714 16097 auto_rebalancer.cc:629] Failed to schedule move for tablet 8158bbd17b994d2eb96f560bd0f1b88d: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111)
I20260709 13:22:45.051075 16224 raft_consensus.cc:2955] T 7c2095a5f4ad49408dbc7f69c606120d P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Committing config change with OpId 19.10: config changed from index -1 to 10, NON_VOTER cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) added. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } } }
I20260709 13:22:45.066387 24656 raft_consensus.cc:493] T 1df93779209748b9860f6d893dd3de53 P 90f3da57796b488c9087a2f523adf045 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:22:45.077124 24656 raft_consensus.cc:515] T 1df93779209748b9860f6d893dd3de53 P 90f3da57796b488c9087a2f523adf045 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:22:45.081475 16790 consensus_peers.cc:597] T 3884f864a96d428785895e5463e5728f P cb82cc7bd6db46d980c75237a1402860 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 13:22:45.083861 24658 raft_consensus.cc:2804] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 90f3da57796b488c9087a2f523adf045 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:22:45.046514 24653 raft_consensus.cc:515] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:45.087302 24639 raft_consensus.cc:493] T ee9ed67006144797b1de6ea8c2013a68 P cb82cc7bd6db46d980c75237a1402860 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:22:45.088014 24639 raft_consensus.cc:515] T ee9ed67006144797b1de6ea8c2013a68 P cb82cc7bd6db46d980c75237a1402860 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:45.089515 24653 leader_election.cc:290] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:22:45.090792 24639 leader_election.cc:290] T ee9ed67006144797b1de6ea8c2013a68 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:22:45.093825 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee9ed67006144797b1de6ea8c2013a68" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
W20260709 13:22:45.095893 16265 leader_election.cc:336] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111)
I20260709 13:22:45.099807 24656 leader_election.cc:290] T 1df93779209748b9860f6d893dd3de53 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
W20260709 13:22:45.103631 16265 consensus_peers.cc:597] T fffab26427e24d398e2695aad40ca134 P 90f3da57796b488c9087a2f523adf045 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:22:45.104094 16265 leader_election.cc:336] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111)
W20260709 13:22:45.106644 16265 leader_election.cc:336] T 1df93779209748b9860f6d893dd3de53 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111)
I20260709 13:22:45.111428 24658 raft_consensus.cc:697] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 90f3da57796b488c9087a2f523adf045 [term 22 LEADER]: Becoming Leader. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:22:45.112095 24578 raft_consensus.cc:493] T 1df93779209748b9860f6d893dd3de53 P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:22:45.112627 24658 consensus_queue.cc:237] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 21.13, Last appended by leader: 13, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:45.116762 24654 raft_consensus.cc:493] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:22:45.117405 24654 raft_consensus.cc:515] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:22:45.126278 16790 leader_election.cc:336] T ee9ed67006144797b1de6ea8c2013a68 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111)
I20260709 13:22:45.127395 16790 leader_election.cc:304] T ee9ed67006144797b1de6ea8c2013a68 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
W20260709 13:22:45.132342 16789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 65941.1us
I20260709 13:22:45.125629 24660 raft_consensus.cc:493] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:22:45.133409 24663 raft_consensus.cc:493] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:22:45.133534 24660 raft_consensus.cc:515] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:45.134137 24663 raft_consensus.cc:515] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:45.136144 24660 leader_election.cc:290] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:45.136433 24663 leader_election.cc:290] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:22:45.122272 24578 raft_consensus.cc:515] T 1df93779209748b9860f6d893dd3de53 P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:45.136865 16044 catalog_manager.cc:5697] T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac reported cstate change: term changed from 22 to 23, leader changed from 647a343a542d4a139fe106c19095129b (127.14.177.129) to 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130). New cstate: current_term: 23 leader_uuid: "1ca2725d290e47d68b1a4b609c9816ac" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:45.139067 24578 leader_election.cc:290] T 1df93779209748b9860f6d893dd3de53 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:45.139807 24654 leader_election.cc:290] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
W20260709 13:22:45.147099 16191 leader_election.cc:336] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111)
W20260709 13:22:45.147926 16265 consensus_peers.cc:597] T 434bff3082d141838068f323bdb0d1ed P 90f3da57796b488c9087a2f523adf045 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:22:45.148005 16191 leader_election.cc:336] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111)
W20260709 13:22:45.148823 16191 leader_election.cc:336] T 1df93779209748b9860f6d893dd3de53 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111)
I20260709 13:22:45.154023 24656 raft_consensus.cc:493] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:22:45.154834 24656 raft_consensus.cc:515] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:45.158376 24656 leader_election.cc:290] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:45.172012 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746aabf856ab401c9ef5e859371e7b67" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:45.174094 16818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806d7690c2b644959152074bb2cab0a7" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
W20260709 13:22:45.182862 16265 leader_election.cc:336] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111)
W20260709 13:22:45.187331 16790 consensus_peers.cc:597] T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 13:22:45.189014 16266 leader_election.cc:304] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
I20260709 13:22:45.192888 24659 raft_consensus.cc:2749] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 13:22:45.190459 16790 consensus_peers.cc:597] T 2794af5b99984efebdee564035841cb0 P cb82cc7bd6db46d980c75237a1402860 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:22:45.204970 16790 leader_election.cc:336] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111)
I20260709 13:22:45.205708 16790 leader_election.cc:304] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
W20260709 13:22:45.207588 16790 consensus_peers.cc:597] T 66f75124215f4ca6bef831e174a792d8 P cb82cc7bd6db46d980c75237a1402860 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:22:45.214116 24663 raft_consensus.cc:2749] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:22:45.164837 24639 raft_consensus.cc:2749] T ee9ed67006144797b1de6ea8c2013a68 P cb82cc7bd6db46d980c75237a1402860 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:22:45.238942 24656 raft_consensus.cc:493] T 84f2da98864b4425bf7c0055951fdd4b P 90f3da57796b488c9087a2f523adf045 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:22:45.239632 24656 raft_consensus.cc:515] T 84f2da98864b4425bf7c0055951fdd4b P 90f3da57796b488c9087a2f523adf045 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:45.241918 24656 leader_election.cc:290] T 84f2da98864b4425bf7c0055951fdd4b P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:22:45.228385 24659 raft_consensus.cc:493] T 8158bbd17b994d2eb96f560bd0f1b88d P 90f3da57796b488c9087a2f523adf045 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:22:45.242877 24659 raft_consensus.cc:515] T 8158bbd17b994d2eb96f560bd0f1b88d P 90f3da57796b488c9087a2f523adf045 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:45.245445 24659 leader_election.cc:290] T 8158bbd17b994d2eb96f560bd0f1b88d P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:22:45.258335 16822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62fe3346cd3c42c4a55e14af9073bca4" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 28 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
W20260709 13:22:45.263631 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 138732us
I20260709 13:22:45.264276 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c74490c40243a88e62d1760b5cd814" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:45.265030 16217 raft_consensus.cc:2468] T 08c74490c40243a88e62d1760b5cd814 P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 21.
I20260709 13:22:45.265058 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1df93779209748b9860f6d893dd3de53" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:45.265796 16224 raft_consensus.cc:2468] T 1df93779209748b9860f6d893dd3de53 P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 21.
I20260709 13:22:45.222210 24664 raft_consensus.cc:493] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
W20260709 13:22:45.271297 16265 leader_election.cc:336] T 84f2da98864b4425bf7c0055951fdd4b P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111)
I20260709 13:22:45.272472 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1df93779209748b9860f6d893dd3de53" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:45.273243 16293 raft_consensus.cc:2468] T 1df93779209748b9860f6d893dd3de53 P 90f3da57796b488c9087a2f523adf045 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 21.
I20260709 13:22:45.273591 16295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8158bbd17b994d2eb96f560bd0f1b88d" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:45.275068 16295 raft_consensus.cc:2468] T 8158bbd17b994d2eb96f560bd0f1b88d P 90f3da57796b488c9087a2f523adf045 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 23.
I20260709 13:22:45.275444 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806d7690c2b644959152074bb2cab0a7" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
W20260709 13:22:45.272048 16265 leader_election.cc:336] T 8158bbd17b994d2eb96f560bd0f1b88d P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111)
I20260709 13:22:45.275144 24664 raft_consensus.cc:515] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:45.284585 16822 raft_consensus.cc:2468] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 27.
I20260709 13:22:45.285786 24664 leader_election.cc:290] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:22:45.291594 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62fe3346cd3c42c4a55e14af9073bca4" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 28 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:45.292445 16217 raft_consensus.cc:2468] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 27.
I20260709 13:22:45.294549 16220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f2da98864b4425bf7c0055951fdd4b" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:45.294735 16219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8158bbd17b994d2eb96f560bd0f1b88d" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:45.279126 16190 leader_election.cc:304] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 647a343a542d4a139fe106c19095129b
I20260709 13:22:45.295374 16219 raft_consensus.cc:2468] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 23.
I20260709 13:22:45.295768 24654 raft_consensus.cc:493] T 746aabf856ab401c9ef5e859371e7b67 P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:22:45.296208 16190 leader_election.cc:304] T 1df93779209748b9860f6d893dd3de53 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 647a343a542d4a139fe106c19095129b
I20260709 13:22:45.296296 24654 raft_consensus.cc:515] T 746aabf856ab401c9ef5e859371e7b67 P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:45.297183 24578 raft_consensus.cc:2804] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:22:45.297631 24578 raft_consensus.cc:493] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac [term 23 FOLLOWER]: Starting leader election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:22:45.298084 24578 raft_consensus.cc:3060] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:22:45.299077 24654 leader_election.cc:290] T 746aabf856ab401c9ef5e859371e7b67 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
W20260709 13:22:45.310318 16191 leader_election.cc:336] T 746aabf856ab401c9ef5e859371e7b67 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111)
I20260709 13:22:45.311777 16224 consensus_queue.cc:237] T 18ad7e3af5504571a69d10c00750a01c P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 13, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } }
I20260709 13:22:45.297636 24667 raft_consensus.cc:2804] T 1df93779209748b9860f6d893dd3de53 P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:22:45.315352 24667 raft_consensus.cc:493] T 1df93779209748b9860f6d893dd3de53 P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Starting leader election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:22:45.315843 24667 raft_consensus.cc:3060] T 1df93779209748b9860f6d893dd3de53 P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:22:45.316044 16790 leader_election.cc:304] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 
W20260709 13:22:45.317229 16790 leader_election.cc:336] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111)
I20260709 13:22:45.319414 24664 raft_consensus.cc:2804] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 13:22:45.323046 24578 raft_consensus.cc:515] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:45.300447 16192 leader_election.cc:304] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 647a343a542d4a139fe106c19095129b
I20260709 13:22:45.339401 24654 raft_consensus.cc:2804] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 13:22:45.339913 24654 raft_consensus.cc:493] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [term 27 FOLLOWER]: Starting leader election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:22:45.340375 24654 raft_consensus.cc:3060] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [term 27 FOLLOWER]: Advancing to term 28
I20260709 13:22:45.341267 24578 leader_election.cc:290] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 24 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:45.343477 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8158bbd17b994d2eb96f560bd0f1b88d" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:22:45.344175 16293 raft_consensus.cc:3060] T 8158bbd17b994d2eb96f560bd0f1b88d P 90f3da57796b488c9087a2f523adf045 [term 23 FOLLOWER]: Advancing to term 24
W20260709 13:22:45.346123 16191 leader_election.cc:336] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 24 election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111)
I20260709 13:22:45.347085 16818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746aabf856ab401c9ef5e859371e7b67" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:45.347754 16818 raft_consensus.cc:2468] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 19.
I20260709 13:22:45.320007 24664 raft_consensus.cc:493] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [term 27 FOLLOWER]: Starting leader election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:22:45.321650 16265 leader_election.cc:304] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 647a343a542d4a139fe106c19095129b
I20260709 13:22:45.351433 24664 raft_consensus.cc:3060] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [term 27 FOLLOWER]: Advancing to term 28
I20260709 13:22:45.352221 16265 leader_election.cc:304] T 1df93779209748b9860f6d893dd3de53 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 647a343a542d4a139fe106c19095129b
I20260709 13:22:45.353166 24669 raft_consensus.cc:2804] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:22:45.353605 24669 raft_consensus.cc:493] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:22:45.354008 24669 raft_consensus.cc:3060] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:22:45.357483 24654 raft_consensus.cc:515] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [term 28 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:45.357867 16192 leader_election.cc:304] T 746aabf856ab401c9ef5e859371e7b67 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 647a343a542d4a139fe106c19095129b
I20260709 13:22:45.359936 24276 raft_consensus.cc:493] T ee9ed67006144797b1de6ea8c2013a68 P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:22:45.362515 24276 raft_consensus.cc:515] T ee9ed67006144797b1de6ea8c2013a68 P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:45.364820 24276 leader_election.cc:290] T ee9ed67006144797b1de6ea8c2013a68 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:45.365378 24276 raft_consensus.cc:2804] T 746aabf856ab401c9ef5e859371e7b67 P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:22:45.365721 24276 raft_consensus.cc:493] T 746aabf856ab401c9ef5e859371e7b67 P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Starting leader election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:22:45.366065 24276 raft_consensus.cc:3060] T 746aabf856ab401c9ef5e859371e7b67 P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:22:45.361553 24659 raft_consensus.cc:2804] T 1df93779209748b9860f6d893dd3de53 P 90f3da57796b488c9087a2f523adf045 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:22:45.368000 24659 raft_consensus.cc:493] T 1df93779209748b9860f6d893dd3de53 P 90f3da57796b488c9087a2f523adf045 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:22:45.368494 24659 raft_consensus.cc:3060] T 1df93779209748b9860f6d893dd3de53 P 90f3da57796b488c9087a2f523adf045 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:22:45.370311 24667 raft_consensus.cc:515] T 1df93779209748b9860f6d893dd3de53 P 1ca2725d290e47d68b1a4b609c9816ac [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:45.393216 24667 leader_election.cc:290] T 1df93779209748b9860f6d893dd3de53 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 22 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:45.371516 24669 raft_consensus.cc:515] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:22:45.376477 16790 consensus_peers.cc:597] T 23f82224f4564fea93572e8206bad1fe P cb82cc7bd6db46d980c75237a1402860 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 13:22:45.395572 24669 leader_election.cc:290] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 22 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:22:45.376685 16818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee9ed67006144797b1de6ea8c2013a68" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:45.401594 16818 raft_consensus.cc:2468] T ee9ed67006144797b1de6ea8c2013a68 P cb82cc7bd6db46d980c75237a1402860 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 21.
I20260709 13:22:45.403280 24664 raft_consensus.cc:515] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [term 28 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:45.403867 24654 leader_election.cc:290] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 28 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:45.403416 16045 catalog_manager.cc:5697] T 7c2095a5f4ad49408dbc7f69c606120d P 90f3da57796b488c9087a2f523adf045 reported cstate change: config changed from index -1 to 10, NON_VOTER cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) added. New cstate: current_term: 19 leader_uuid: "90f3da57796b488c9087a2f523adf045" committed_config { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:45.391038 24276 raft_consensus.cc:515] T 746aabf856ab401c9ef5e859371e7b67 P 1ca2725d290e47d68b1a4b609c9816ac [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:45.405550 24664 leader_election.cc:290] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 28 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:22:45.406450 24276 leader_election.cc:290] T 746aabf856ab401c9ef5e859371e7b67 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 20 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:45.405902 16045 catalog_manager.cc:5697] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 90f3da57796b488c9087a2f523adf045 reported cstate change: term changed from 21 to 22, leader changed from 647a343a542d4a139fe106c19095129b (127.14.177.129) to 90f3da57796b488c9087a2f523adf045 (127.14.177.131). New cstate: current_term: 22 leader_uuid: "90f3da57796b488c9087a2f523adf045" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } health_report { overall_health: HEALTHY } } }
I20260709 13:22:45.410912 16295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1df93779209748b9860f6d893dd3de53" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:22:45.413417 24666 raft_consensus.cc:493] T 8a878999c0024d2abe517484180290fc P 1ca2725d290e47d68b1a4b609c9816ac [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
W20260709 13:22:45.417156 16790 leader_election.cc:336] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 28 election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111)
W20260709 13:22:45.377574 16191 leader_election.cc:336] T ee9ed67006144797b1de6ea8c2013a68 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111)
I20260709 13:22:45.382064 16265 leader_election.cc:304] T 84f2da98864b4425bf7c0055951fdd4b P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:22:45.385955 16293 raft_consensus.cc:2468] T 8158bbd17b994d2eb96f560bd0f1b88d P 90f3da57796b488c9087a2f523adf045 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 24.
I20260709 13:22:45.386047 24659 raft_consensus.cc:515] T 1df93779209748b9860f6d893dd3de53 P 90f3da57796b488c9087a2f523adf045 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:45.390913 16300 raft_consensus.cc:1275] T 18ad7e3af5504571a69d10c00750a01c P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
W20260709 13:22:45.429096 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 47128.9us
I20260709 13:22:45.429925 16295 raft_consensus.cc:2393] T 1df93779209748b9860f6d893dd3de53 P 90f3da57796b488c9087a2f523adf045 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 22: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
I20260709 13:22:45.430313 24628 raft_consensus.cc:2749] T 84f2da98864b4425bf7c0055951fdd4b P 90f3da57796b488c9087a2f523adf045 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:22:45.426463 16818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62fe3346cd3c42c4a55e14af9073bca4" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 28 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:22:45.433382 16818 raft_consensus.cc:2393] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [term 28 FOLLOWER]: Leader election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 28: Already voted for candidate cb82cc7bd6db46d980c75237a1402860 in this term.
I20260709 13:22:45.414131 24666 raft_consensus.cc:515] T 8a878999c0024d2abe517484180290fc P 1ca2725d290e47d68b1a4b609c9816ac [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:45.435959 24666 leader_election.cc:290] T 8a878999c0024d2abe517484180290fc P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:45.427596 16190 leader_election.cc:304] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 647a343a542d4a139fe106c19095129b
I20260709 13:22:45.438472 16295 raft_consensus.cc:1275] T 8d4ab870a05347868956aa8b841b2981 P 90f3da57796b488c9087a2f523adf045 [term 23 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 22 index: 13. Preceding OpId from leader: term: 23 index: 14. (index mismatch)
W20260709 13:22:45.426712 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 49178.3us
I20260709 13:22:45.429812 16265 leader_election.cc:304] T 8158bbd17b994d2eb96f560bd0f1b88d P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 647a343a542d4a139fe106c19095129b
I20260709 13:22:45.440191 24669 raft_consensus.cc:2764] T 8158bbd17b994d2eb96f560bd0f1b88d P 90f3da57796b488c9087a2f523adf045 [term 24 FOLLOWER]: Leader pre-election decision vote started in defunct term 23: won
I20260709 13:22:45.430439 24659 leader_election.cc:290] T 1df93779209748b9860f6d893dd3de53 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 22 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:22:45.426297 16823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746aabf856ab401c9ef5e859371e7b67" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:22:45.446597 16823 raft_consensus.cc:3060] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860 [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:22:45.446935 16265 consensus_peers.cc:597] T 26354361e9b044cea1c2b86c4882fe92 P 90f3da57796b488c9087a2f523adf045 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:22:45.447491 16265 consensus_peers.cc:597] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:22:45.447907 16265 leader_election.cc:336] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111)
W20260709 13:22:45.448333 16265 leader_election.cc:336] T 1df93779209748b9860f6d893dd3de53 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111)
I20260709 13:22:45.448370 16295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a878999c0024d2abe517484180290fc" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:45.453929 16823 raft_consensus.cc:2468] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 20.
I20260709 13:22:45.457938 24276 raft_consensus.cc:2804] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac [term 24 FOLLOWER]: Leader election won for term 24
I20260709 13:22:45.458756 24276 raft_consensus.cc:697] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac [term 24 LEADER]: Becoming Leader. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:22:45.458876 24667 consensus_queue.cc:1048] T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:22:45.458765 24654 consensus_queue.cc:1048] T 18ad7e3af5504571a69d10c00750a01c P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.002s
I20260709 13:22:45.459995 24276 consensus_queue.cc:237] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 23.15, Last appended by leader: 15, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:45.427954 16192 leader_election.cc:304] T ee9ed67006144797b1de6ea8c2013a68 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 647a343a542d4a139fe106c19095129b
I20260709 13:22:45.467821 24578 raft_consensus.cc:493] T 84f2da98864b4425bf7c0055951fdd4b P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:22:45.468444 24578 raft_consensus.cc:515] T 84f2da98864b4425bf7c0055951fdd4b P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:45.470443 24578 leader_election.cc:290] T 84f2da98864b4425bf7c0055951fdd4b P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:45.473031 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62fe3346cd3c42c4a55e14af9073bca4" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 28 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:22:45.473851 16217 raft_consensus.cc:2393] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [term 28 FOLLOWER]: Leader election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 in current term 28: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
W20260709 13:22:45.474449 16192 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 47038.4us
W20260709 13:22:45.476430 16192 consensus_peers.cc:597] T 18ad7e3af5504571a69d10c00750a01c P 1ca2725d290e47d68b1a4b609c9816ac -> Peer cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923): Couldn't send request to peer cb82cc7bd6db46d980c75237a1402860. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 18ad7e3af5504571a69d10c00750a01c. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:22:45.476614 16221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1df93779209748b9860f6d893dd3de53" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:22:45.477449 16221 raft_consensus.cc:2393] T 1df93779209748b9860f6d893dd3de53 P 1ca2725d290e47d68b1a4b609c9816ac [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 22: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
I20260709 13:22:45.477344 16192 leader_election.cc:304] T 746aabf856ab401c9ef5e859371e7b67 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:22:45.477973 24578 raft_consensus.cc:2804] T ee9ed67006144797b1de6ea8c2013a68 P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:22:45.478257 24657 raft_consensus.cc:493] T 02838c86531d4f98bf58d1c22b8baf77 P 1ca2725d290e47d68b1a4b609c9816ac [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:22:45.478433 24578 raft_consensus.cc:493] T ee9ed67006144797b1de6ea8c2013a68 P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Starting leader election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:22:45.478696 24666 raft_consensus.cc:2804] T 746aabf856ab401c9ef5e859371e7b67 P 1ca2725d290e47d68b1a4b609c9816ac [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:22:45.479027 24578 raft_consensus.cc:3060] T ee9ed67006144797b1de6ea8c2013a68 P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:22:45.478890 24657 raft_consensus.cc:515] T 02838c86531d4f98bf58d1c22b8baf77 P 1ca2725d290e47d68b1a4b609c9816ac [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:45.479514 24666 raft_consensus.cc:697] T 746aabf856ab401c9ef5e859371e7b67 P 1ca2725d290e47d68b1a4b609c9816ac [term 20 LEADER]: Becoming Leader. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:22:45.480017 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c74490c40243a88e62d1760b5cd814" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:22:45.480592 16217 raft_consensus.cc:3060] T 08c74490c40243a88e62d1760b5cd814 P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:22:45.481199 24657 leader_election.cc:290] T 02838c86531d4f98bf58d1c22b8baf77 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:45.481803 24664 raft_consensus.cc:493] T 806d7690c2b644959152074bb2cab0a7 P cb82cc7bd6db46d980c75237a1402860 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:22:45.479461 16265 leader_election.cc:304] T 1df93779209748b9860f6d893dd3de53 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:22:45.482465 24664 raft_consensus.cc:515] T 806d7690c2b644959152074bb2cab0a7 P cb82cc7bd6db46d980c75237a1402860 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:45.482955 24659 raft_consensus.cc:2749] T 1df93779209748b9860f6d893dd3de53 P 90f3da57796b488c9087a2f523adf045 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:22:45.486819 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806d7690c2b644959152074bb2cab0a7" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:45.487520 16300 raft_consensus.cc:2468] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 23.
I20260709 13:22:45.489081 24664 leader_election.cc:290] T 806d7690c2b644959152074bb2cab0a7 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:45.494479 16217 raft_consensus.cc:2468] T 08c74490c40243a88e62d1760b5cd814 P 1ca2725d290e47d68b1a4b609c9816ac [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 22.
I20260709 13:22:45.497921 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f2da98864b4425bf7c0055951fdd4b" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:45.498750 16300 raft_consensus.cc:2468] T 84f2da98864b4425bf7c0055951fdd4b P 90f3da57796b488c9087a2f523adf045 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 21.
I20260709 13:22:45.499850 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02838c86531d4f98bf58d1c22b8baf77" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
W20260709 13:22:45.500427 16790 consensus_peers.cc:597] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:22:45.502049 16190 leader_election.cc:304] T 84f2da98864b4425bf7c0055951fdd4b P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:22:45.481710 24666 consensus_queue.cc:237] T 746aabf856ab401c9ef5e859371e7b67 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 19.16, Last appended by leader: 16, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:45.504496 24672 ts_tablet_manager.cc:933] T 7c2095a5f4ad49408dbc7f69c606120d P cb82cc7bd6db46d980c75237a1402860: Initiating tablet copy from peer 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:45.511905 24657 raft_consensus.cc:2804] T 84f2da98864b4425bf7c0055951fdd4b P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:22:45.512406 24657 raft_consensus.cc:493] T 84f2da98864b4425bf7c0055951fdd4b P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Starting leader election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:22:45.512804 24657 raft_consensus.cc:3060] T 84f2da98864b4425bf7c0055951fdd4b P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:22:45.521548 24655 raft_consensus.cc:2955] T 18ad7e3af5504571a69d10c00750a01c P 1ca2725d290e47d68b1a4b609c9816ac [term 19 LEADER]: Committing config change with OpId 19.15: config changed from index -1 to 15, NON_VOTER cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) added. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } } }
I20260709 13:22:45.486999 24578 raft_consensus.cc:515] T ee9ed67006144797b1de6ea8c2013a68 P 1ca2725d290e47d68b1a4b609c9816ac [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:45.505810 24659 raft_consensus.cc:493] T 9e65676b21b748728ad39668cfe722ec P 90f3da57796b488c9087a2f523adf045 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:22:45.508255 24628 raft_consensus.cc:493] T 5a959dbecc2648eb85d1cc550f5a3245 P 90f3da57796b488c9087a2f523adf045 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:22:45.527611 24660 raft_consensus.cc:493] T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:22:45.527968 24628 raft_consensus.cc:515] T 5a959dbecc2648eb85d1cc550f5a3245 P 90f3da57796b488c9087a2f523adf045 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:45.528158 24660 raft_consensus.cc:515] T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:45.530076 24660 leader_election.cc:290] T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:45.527585 24659 raft_consensus.cc:515] T 9e65676b21b748728ad39668cfe722ec P 90f3da57796b488c9087a2f523adf045 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:45.532997 24659 leader_election.cc:290] T 9e65676b21b748728ad39668cfe722ec P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:45.530100 24628 leader_election.cc:290] T 5a959dbecc2648eb85d1cc550f5a3245 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:22:45.534070 16822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e65676b21b748728ad39668cfe722ec" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
W20260709 13:22:45.535213 16265 leader_election.cc:336] T 5a959dbecc2648eb85d1cc550f5a3245 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111)
I20260709 13:22:45.536535 24578 leader_election.cc:290] T ee9ed67006144797b1de6ea8c2013a68 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 22 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:45.536784 16295 raft_consensus.cc:2955] T 18ad7e3af5504571a69d10c00750a01c P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Committing config change with OpId 19.15: config changed from index -1 to 15, NON_VOTER cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) added. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } } }
I20260709 13:22:45.538699 16291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c61a39fd4f84424b129c1a6f9ba404d" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
W20260709 13:22:45.541572 16265 leader_election.cc:336] T 9e65676b21b748728ad39668cfe722ec P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111)
I20260709 13:22:45.542240 16265 leader_election.cc:304] T 9e65676b21b748728ad39668cfe722ec P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
I20260709 13:22:45.543236 24628 raft_consensus.cc:2749] T 9e65676b21b748728ad39668cfe722ec P 90f3da57796b488c9087a2f523adf045 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:22:45.547144 24664 raft_consensus.cc:493] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:22:45.547719 24664 raft_consensus.cc:515] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:45.550558 24664 leader_election.cc:290] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:45.550853 16291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a9de347c711477e87c582d003f36925" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 27 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:45.553423 16823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee9ed67006144797b1de6ea8c2013a68" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:22:45.553957 16217 raft_consensus.cc:1275] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 1ca2725d290e47d68b1a4b609c9816ac [term 22 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 21 index: 13. Preceding OpId from leader: term: 22 index: 14. (index mismatch)
I20260709 13:22:45.555289 16221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a959dbecc2648eb85d1cc550f5a3245" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:45.556533 16265 leader_election.cc:304] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 647a343a542d4a139fe106c19095129b
I20260709 13:22:45.557278 24628 raft_consensus.cc:2804] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:22:45.557971 24628 raft_consensus.cc:697] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045 [term 22 LEADER]: Becoming Leader. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:22:45.558848 24628 consensus_queue.cc:237] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 21.13, Last appended by leader: 13, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:45.560045 24664 raft_consensus.cc:493] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:22:45.560437 24669 raft_consensus.cc:493] T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:22:45.560672 24664 raft_consensus.cc:515] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:45.561003 24669 raft_consensus.cc:515] T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:45.562984 24664 leader_election.cc:290] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:22:45.563374 24669 leader_election.cc:290] T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:22:45.554013 16823 raft_consensus.cc:3060] T ee9ed67006144797b1de6ea8c2013a68 P cb82cc7bd6db46d980c75237a1402860 [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:22:45.564869 16790 leader_election.cc:336] T 806d7690c2b644959152074bb2cab0a7 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111)
I20260709 13:22:45.571408 24669 raft_consensus.cc:493] T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
W20260709 13:22:45.572135 16790 consensus_peers.cc:597] T 57a744f5aaf944678a63469cbf514015 P cb82cc7bd6db46d980c75237a1402860 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 13:22:45.572235 24669 raft_consensus.cc:515] T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:45.574717 24669 leader_election.cc:290] T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
W20260709 13:22:45.576983 16191 consensus_peers.cc:597] T 18ad7e3af5504571a69d10c00750a01c P 1ca2725d290e47d68b1a4b609c9816ac -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:22:45.577521 16191 leader_election.cc:336] T 1df93779209748b9860f6d893dd3de53 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111)
I20260709 13:22:45.577965 16191 leader_election.cc:304] T 1df93779209748b9860f6d893dd3de53 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:22:45.526500 24657 raft_consensus.cc:515] T 84f2da98864b4425bf7c0055951fdd4b P 1ca2725d290e47d68b1a4b609c9816ac [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:45.579442 16823 raft_consensus.cc:2468] T ee9ed67006144797b1de6ea8c2013a68 P cb82cc7bd6db46d980c75237a1402860 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 22.
W20260709 13:22:45.580528 16191 leader_election.cc:336] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 28 election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111)
I20260709 13:22:45.582803 16191 leader_election.cc:304] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 28 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
I20260709 13:22:45.583781 24578 raft_consensus.cc:2749] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [term 28 FOLLOWER]: Leader election lost for term 28. Reason: could not achieve majority
W20260709 13:22:45.584028 16191 leader_election.cc:336] T 746aabf856ab401c9ef5e859371e7b67 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 20 election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111)
W20260709 13:22:45.585100 16191 leader_election.cc:336] T 8a878999c0024d2abe517484180290fc P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111)
I20260709 13:22:45.585590 16191 leader_election.cc:304] T 8a878999c0024d2abe517484180290fc P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:22:45.592697 16221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c61a39fd4f84424b129c1a6f9ba404d" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:45.593852 16221 raft_consensus.cc:2468] T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 25.
I20260709 13:22:45.579703 24655 raft_consensus.cc:2749] T 1df93779209748b9860f6d893dd3de53 P 1ca2725d290e47d68b1a4b609c9816ac [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:22:45.603471 16790 leader_election.cc:304] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 28 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:22:45.606389 24578 raft_consensus.cc:493] T 5a959dbecc2648eb85d1cc550f5a3245 P 1ca2725d290e47d68b1a4b609c9816ac [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:22:45.611841 24578 raft_consensus.cc:515] T 5a959dbecc2648eb85d1cc550f5a3245 P 1ca2725d290e47d68b1a4b609c9816ac [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:45.612677 16265 leader_election.cc:304] T 5a959dbecc2648eb85d1cc550f5a3245 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:22:45.592855 16192 leader_election.cc:304] T ee9ed67006144797b1de6ea8c2013a68 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:22:45.633471 24578 leader_election.cc:290] T 5a959dbecc2648eb85d1cc550f5a3245 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:45.619719 16221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806d7690c2b644959152074bb2cab0a7" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:45.635143 16291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a959dbecc2648eb85d1cc550f5a3245" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:45.635983 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b7d1ab519044eda2d5e04a07242f25" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 24 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:45.634143 24578 raft_consensus.cc:493] T b6f07842d3aa47858df5665874c86f3a P 1ca2725d290e47d68b1a4b609c9816ac [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:22:45.637524 24578 raft_consensus.cc:515] T b6f07842d3aa47858df5665874c86f3a P 1ca2725d290e47d68b1a4b609c9816ac [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:45.635844 16291 raft_consensus.cc:2468] T 5a959dbecc2648eb85d1cc550f5a3245 P 90f3da57796b488c9087a2f523adf045 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 23.
I20260709 13:22:45.616716 24276 raft_consensus.cc:493] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
W20260709 13:22:45.642704 16192 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 49948.8us
I20260709 13:22:45.613291 24660 raft_consensus.cc:2749] T 8a878999c0024d2abe517484180290fc P 1ca2725d290e47d68b1a4b609c9816ac [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:22:45.648447 16790 leader_election.cc:336] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111)
I20260709 13:22:45.640162 16190 leader_election.cc:304] T 5a959dbecc2648eb85d1cc550f5a3245 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:22:45.650847 24578 leader_election.cc:290] T b6f07842d3aa47858df5665874c86f3a P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:45.655532 16790 leader_election.cc:304] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:22:45.611230 24669 raft_consensus.cc:493] T 02838c86531d4f98bf58d1c22b8baf77 P 90f3da57796b488c9087a2f523adf045 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:22:45.641873 24276 raft_consensus.cc:515] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:45.613612 24659 raft_consensus.cc:493] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 90f3da57796b488c9087a2f523adf045 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:22:45.655447 24667 raft_consensus.cc:2804] T ee9ed67006144797b1de6ea8c2013a68 P 1ca2725d290e47d68b1a4b609c9816ac [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:22:45.637622 24639 raft_consensus.cc:2749] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [term 28 FOLLOWER]: Leader election lost for term 28. Reason: could not achieve majority
I20260709 13:22:45.669040 16221 raft_consensus.cc:2468] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 23.
I20260709 13:22:45.614995 24641 consensus_queue.cc:1048] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.048s
I20260709 13:22:45.675256 16790 leader_election.cc:304] T 806d7690c2b644959152074bb2cab0a7 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 647a343a542d4a139fe106c19095129b
I20260709 13:22:45.672158 24659 raft_consensus.cc:515] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 90f3da57796b488c9087a2f523adf045 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:45.667701 24627 raft_consensus.cc:2749] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 13:22:45.663614 24656 raft_consensus.cc:2749] T 5a959dbecc2648eb85d1cc550f5a3245 P 90f3da57796b488c9087a2f523adf045 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 13:22:45.696447 16192 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 46788.1us
I20260709 13:22:45.711359 16221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6f07842d3aa47858df5665874c86f3a" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:45.712075 16221 raft_consensus.cc:2468] T b6f07842d3aa47858df5665874c86f3a P 1ca2725d290e47d68b1a4b609c9816ac [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 24.
I20260709 13:22:45.681505 24660 raft_consensus.cc:2804] T 5a959dbecc2648eb85d1cc550f5a3245 P 1ca2725d290e47d68b1a4b609c9816ac [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:22:45.716388 24672 tablet_copy_client.cc:323] T 7c2095a5f4ad49408dbc7f69c606120d P cb82cc7bd6db46d980c75237a1402860: tablet copy: Beginning tablet copy session from remote peer at address 127.14.177.131:41389
I20260709 13:22:45.720003 16818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806d7690c2b644959152074bb2cab0a7" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:45.714588 24276 leader_election.cc:290] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:45.688680 24667 raft_consensus.cc:697] T ee9ed67006144797b1de6ea8c2013a68 P 1ca2725d290e47d68b1a4b609c9816ac [term 22 LEADER]: Becoming Leader. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:22:45.698166 24663 raft_consensus.cc:2804] T 806d7690c2b644959152074bb2cab0a7 P cb82cc7bd6db46d980c75237a1402860 [term 23 FOLLOWER]: Leader pre-election won for term 24
W20260709 13:22:45.724516 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 112099us
I20260709 13:22:45.720283 24660 raft_consensus.cc:493] T 5a959dbecc2648eb85d1cc550f5a3245 P 1ca2725d290e47d68b1a4b609c9816ac [term 23 FOLLOWER]: Starting leader election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:22:45.670306 24669 raft_consensus.cc:515] T 02838c86531d4f98bf58d1c22b8baf77 P 90f3da57796b488c9087a2f523adf045 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:45.718811 24659 leader_election.cc:290] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
W20260709 13:22:45.745150 16191 leader_election.cc:336] T 84f2da98864b4425bf7c0055951fdd4b P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111)
I20260709 13:22:45.719056 24628 raft_consensus.cc:493] T 5a96373bb0e14dce8a266b2fc6c58507 P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:22:45.753064 16823 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e9b2bf7011f4ae688a79d7bb5a447b4" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:45.735971 24663 raft_consensus.cc:493] T 806d7690c2b644959152074bb2cab0a7 P cb82cc7bd6db46d980c75237a1402860 [term 23 FOLLOWER]: Starting leader election (detected failure of leader 647a343a542d4a139fe106c19095129b)
W20260709 13:22:45.747838 16191 leader_election.cc:336] T 02838c86531d4f98bf58d1c22b8baf77 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111)
W20260709 13:22:45.745260 16265 leader_election.cc:336] T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111)
I20260709 13:22:45.759135 16191 leader_election.cc:304] T 02838c86531d4f98bf58d1c22b8baf77 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:22:45.744860 24660 raft_consensus.cc:3060] T 5a959dbecc2648eb85d1cc550f5a3245 P 1ca2725d290e47d68b1a4b609c9816ac [term 23 FOLLOWER]: Advancing to term 24
W20260709 13:22:45.761457 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 121381us
W20260709 13:22:45.763876 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 88733us
I20260709 13:22:45.764536 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6f07842d3aa47858df5665874c86f3a" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:45.765172 16293 raft_consensus.cc:2468] T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 24.
I20260709 13:22:45.767629 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806d7690c2b644959152074bb2cab0a7" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:45.768213 16293 raft_consensus.cc:2468] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 23.
I20260709 13:22:45.770277 16291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f2da98864b4425bf7c0055951fdd4b" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:22:45.770918 16291 raft_consensus.cc:3060] T 84f2da98864b4425bf7c0055951fdd4b P 90f3da57796b488c9087a2f523adf045 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:22:45.760993 24663 raft_consensus.cc:3060] T 806d7690c2b644959152074bb2cab0a7 P cb82cc7bd6db46d980c75237a1402860 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:22:45.773010 16190 leader_election.cc:304] T b6f07842d3aa47858df5665874c86f3a P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:22:45.773962 24276 raft_consensus.cc:2804] T b6f07842d3aa47858df5665874c86f3a P 1ca2725d290e47d68b1a4b609c9816ac [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:22:45.774338 24276 raft_consensus.cc:493] T b6f07842d3aa47858df5665874c86f3a P 1ca2725d290e47d68b1a4b609c9816ac [term 24 FOLLOWER]: Starting leader election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:22:45.774681 24276 raft_consensus.cc:3060] T b6f07842d3aa47858df5665874c86f3a P 1ca2725d290e47d68b1a4b609c9816ac [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:22:45.775957 16310 tablet_copy_service.cc:140] P 90f3da57796b488c9087a2f523adf045: Received BeginTabletCopySession request for tablet 7c2095a5f4ad49408dbc7f69c606120d from peer cb82cc7bd6db46d980c75237a1402860 ({username='slave'} at 127.0.0.1:40474)
I20260709 13:22:45.776590 16310 tablet_copy_service.cc:161] P 90f3da57796b488c9087a2f523adf045: Beginning new tablet copy session on tablet 7c2095a5f4ad49408dbc7f69c606120d from peer cb82cc7bd6db46d980c75237a1402860 at {username='slave'} at 127.0.0.1:40474: session id = cb82cc7bd6db46d980c75237a1402860-7c2095a5f4ad49408dbc7f69c606120d
I20260709 13:22:45.767199 24654 raft_consensus.cc:2749] T 02838c86531d4f98bf58d1c22b8baf77 P 1ca2725d290e47d68b1a4b609c9816ac [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:22:45.758419 24628 raft_consensus.cc:515] T 5a96373bb0e14dce8a266b2fc6c58507 P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:45.783246 16045 catalog_manager.cc:5697] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac reported cstate change: term changed from 23 to 24, leader changed from 647a343a542d4a139fe106c19095129b (127.14.177.129) to 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130). New cstate: current_term: 24 leader_uuid: "1ca2725d290e47d68b1a4b609c9816ac" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:45.785836 24628 leader_election.cc:290] T 5a96373bb0e14dce8a266b2fc6c58507 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:22:45.785286 16045 catalog_manager.cc:5697] T 746aabf856ab401c9ef5e859371e7b67 P 1ca2725d290e47d68b1a4b609c9816ac reported cstate change: term changed from 19 to 20, leader changed from 647a343a542d4a139fe106c19095129b (127.14.177.129) to 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130). New cstate: current_term: 20 leader_uuid: "1ca2725d290e47d68b1a4b609c9816ac" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:45.762387 24657 leader_election.cc:290] T 84f2da98864b4425bf7c0055951fdd4b P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 22 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:45.787544 16045 catalog_manager.cc:5697] T 18ad7e3af5504571a69d10c00750a01c P 1ca2725d290e47d68b1a4b609c9816ac reported cstate change: config changed from index -1 to 15, NON_VOTER cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) added. New cstate: current_term: 19 leader_uuid: "1ca2725d290e47d68b1a4b609c9816ac" committed_config { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:45.790247 16291 raft_consensus.cc:2468] T 84f2da98864b4425bf7c0055951fdd4b P 90f3da57796b488c9087a2f523adf045 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 22.
I20260709 13:22:45.794523 24688 raft_consensus.cc:493] T 8a878999c0024d2abe517484180290fc P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:22:45.795125 24688 raft_consensus.cc:515] T 8a878999c0024d2abe517484180290fc P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:45.797319 24688 leader_election.cc:290] T 8a878999c0024d2abe517484180290fc P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
W20260709 13:22:45.813637 16191 leader_election.cc:336] T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111)
I20260709 13:22:45.814728 16191 leader_election.cc:304] T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
W20260709 13:22:45.816320 16191 leader_election.cc:336] T ee9ed67006144797b1de6ea8c2013a68 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111)
I20260709 13:22:45.771124 24687 raft_consensus.cc:493] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:22:45.828428 24654 raft_consensus.cc:493] T 5a96373bb0e14dce8a266b2fc6c58507 P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:22:45.809870 16310 tablet_copy_source_session.cc:215] T 7c2095a5f4ad49408dbc7f69c606120d P 90f3da57796b488c9087a2f523adf045: Tablet Copy: opened 0 blocks and 1 log segments
W20260709 13:22:45.828370 16191 consensus_peers.cc:597] T 829b4ad8c5a54eababeea1181b960d4b P 1ca2725d290e47d68b1a4b609c9816ac -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:22:45.845916 16191 consensus_peers.cc:597] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:22:45.831282 24276 raft_consensus.cc:515] T b6f07842d3aa47858df5665874c86f3a P 1ca2725d290e47d68b1a4b609c9816ac [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:22:45.846411 16191 leader_election.cc:336] T 5a959dbecc2648eb85d1cc550f5a3245 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111)
W20260709 13:22:45.848492 16191 consensus_peers.cc:597] T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:22:45.844944 24654 raft_consensus.cc:515] T 5a96373bb0e14dce8a266b2fc6c58507 P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:45.790931 24685 raft_consensus.cc:493] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
W20260709 13:22:45.849125 16191 leader_election.cc:336] T b6f07842d3aa47858df5665874c86f3a P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111)
W20260709 13:22:45.852900 16191 leader_election.cc:336] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111)
W20260709 13:22:45.853613 16191 leader_election.cc:336] T 84f2da98864b4425bf7c0055951fdd4b P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111)
W20260709 13:22:45.854269 16191 consensus_peers.cc:597] T e5ad4eb281564936a59f41e387a47156 P 1ca2725d290e47d68b1a4b609c9816ac -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:22:45.812599 24627 raft_consensus.cc:493] T 9e65676b21b748728ad39668cfe722ec P cb82cc7bd6db46d980c75237a1402860 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:22:45.840934 24657 raft_consensus.cc:2749] T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:22:45.846432 24698 raft_consensus.cc:493] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:22:45.859831 24627 raft_consensus.cc:515] T 9e65676b21b748728ad39668cfe722ec P cb82cc7bd6db46d980c75237a1402860 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:45.771622 24667 consensus_queue.cc:237] T ee9ed67006144797b1de6ea8c2013a68 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 21.16, Last appended by leader: 16, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:45.862892 24627 leader_election.cc:290] T 9e65676b21b748728ad39668cfe722ec P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:45.866657 24698 raft_consensus.cc:515] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:22:45.867689 24684 log.cc:927] Time spent T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045: Append to log took a long time: real 0.131s	user 0.001s	sys 0.000s
I20260709 13:22:45.871601 24669 leader_election.cc:290] T 02838c86531d4f98bf58d1c22b8baf77 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:22:45.874871 24698 leader_election.cc:290] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:45.868271 24685 raft_consensus.cc:515] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:45.877658 24685 leader_election.cc:290] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:45.879590 16265 leader_election.cc:304] T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 647a343a542d4a139fe106c19095129b
I20260709 13:22:45.880517 24669 raft_consensus.cc:2804] T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 13:22:45.880941 24669 raft_consensus.cc:493] T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045 [term 25 FOLLOWER]: Starting leader election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:22:45.881316 24669 raft_consensus.cc:3060] T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045 [term 25 FOLLOWER]: Advancing to term 26
W20260709 13:22:45.881965 15046 thread.cc:527] Waited for 1000ms trying to join with diag-logger (tid 16172)
I20260709 13:22:45.845152 24687 raft_consensus.cc:515] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:45.885316 24687 leader_election.cc:290] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:45.889734 16265 leader_election.cc:304] T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:22:45.890625 24691 raft_consensus.cc:2804] T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:22:45.891040 24691 raft_consensus.cc:493] T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045 [term 24 FOLLOWER]: Starting leader election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:22:45.891969 24691 raft_consensus.cc:3060] T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:22:45.892411 24276 leader_election.cc:290] T b6f07842d3aa47858df5665874c86f3a P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 25 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:45.901818 24691 raft_consensus.cc:515] T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:45.908139 24654 leader_election.cc:290] T 5a96373bb0e14dce8a266b2fc6c58507 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:45.910194 16044 catalog_manager.cc:5697] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045 reported cstate change: term changed from 21 to 22, leader changed from 647a343a542d4a139fe106c19095129b (127.14.177.129) to 90f3da57796b488c9087a2f523adf045 (127.14.177.131). New cstate: current_term: 22 leader_uuid: "90f3da57796b488c9087a2f523adf045" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } health_report { overall_health: HEALTHY } } }
I20260709 13:22:45.931704 24691 leader_election.cc:290] T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 25 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
W20260709 13:22:45.942826 16265 leader_election.cc:336] T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111)
W20260709 13:22:45.944577 16265 leader_election.cc:336] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111)
I20260709 13:22:45.944953 24669 raft_consensus.cc:515] T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045 [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:45.945878 24660 raft_consensus.cc:515] T 5a959dbecc2648eb85d1cc550f5a3245 P 1ca2725d290e47d68b1a4b609c9816ac [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:45.945012 16265 leader_election.cc:304] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
I20260709 13:22:45.949553 24691 raft_consensus.cc:2749] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 90f3da57796b488c9087a2f523adf045 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:22:45.951963 24669 leader_election.cc:290] T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 26 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
W20260709 13:22:45.954186 16191 leader_election.cc:336] T 5a96373bb0e14dce8a266b2fc6c58507 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111)
W20260709 13:22:45.954891 16191 consensus_peers.cc:597] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:22:45.978212 16191 leader_election.cc:336] T b6f07842d3aa47858df5665874c86f3a P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 25 election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111)
I20260709 13:22:45.982142 16219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a878999c0024d2abe517484180290fc" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
W20260709 13:22:45.956837 16265 consensus_peers.cc:597] T 8cc959fb7f704fba9009c0595467d074 P 90f3da57796b488c9087a2f523adf045 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:22:45.965059 24672 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7c2095a5f4ad49408dbc7f69c606120d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:45.984874 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6f07842d3aa47858df5665874c86f3a" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:22:45.985349 16223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c61a39fd4f84424b129c1a6f9ba404d" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:22:45.985736 16225 raft_consensus.cc:2393] T b6f07842d3aa47858df5665874c86f3a P 1ca2725d290e47d68b1a4b609c9816ac [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 90f3da57796b488c9087a2f523adf045 in current term 25: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
I20260709 13:22:45.986019 16223 raft_consensus.cc:3060] T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:22:45.987783 16819 raft_consensus.cc:1275] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860 [term 20 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 19 index: 16. Preceding OpId from leader: term: 20 index: 17. (index mismatch)
I20260709 13:22:45.984925 16220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02838c86531d4f98bf58d1c22b8baf77" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:45.967065 16291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e65676b21b748728ad39668cfe722ec" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:45.989362 16220 raft_consensus.cc:2468] T 02838c86531d4f98bf58d1c22b8baf77 P 1ca2725d290e47d68b1a4b609c9816ac [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 24.
I20260709 13:22:45.989745 16291 raft_consensus.cc:2468] T 9e65676b21b748728ad39668cfe722ec P 90f3da57796b488c9087a2f523adf045 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 22.
I20260709 13:22:45.968634 16817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a9de347c711477e87c582d003f36925" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 27 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:45.970551 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e9b2bf7011f4ae688a79d7bb5a447b4" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:45.990832 16817 raft_consensus.cc:2468] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 26.
I20260709 13:22:45.990931 16300 raft_consensus.cc:2468] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 90f3da57796b488c9087a2f523adf045 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 23.
I20260709 13:22:45.982905 16219 raft_consensus.cc:2468] T 8a878999c0024d2abe517484180290fc P 1ca2725d290e47d68b1a4b609c9816ac [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 20.
I20260709 13:22:45.973527 16821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b7d1ab519044eda2d5e04a07242f25" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 24 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:45.974705 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a96373bb0e14dce8a266b2fc6c58507" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
W20260709 13:22:45.992940 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 152390us
I20260709 13:22:45.996747 24705 ts_tablet_manager.cc:933] T 18ad7e3af5504571a69d10c00750a01c P cb82cc7bd6db46d980c75237a1402860: Initiating tablet copy from peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:22:45.993288 16224 raft_consensus.cc:2468] T 5a96373bb0e14dce8a266b2fc6c58507 P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 18.
I20260709 13:22:45.993196 16821 raft_consensus.cc:2468] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 23.
I20260709 13:22:45.993835 16266 leader_election.cc:304] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:22:46.005371 24669 raft_consensus.cc:2804] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 13:22:46.003829 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a96373bb0e14dce8a266b2fc6c58507" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:22:46.007822 24669 raft_consensus.cc:493] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [term 26 FOLLOWER]: Starting leader election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:22:46.008010 16298 raft_consensus.cc:2468] T 5a96373bb0e14dce8a266b2fc6c58507 P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 18.
I20260709 13:22:46.008260 24669 raft_consensus.cc:3060] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [term 26 FOLLOWER]: Advancing to term 27
I20260709 13:22:46.008579 16223 raft_consensus.cc:2468] T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 26.
W20260709 13:22:46.003206 16790 leader_election.cc:336] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111)
I20260709 13:22:46.009366 16790 leader_election.cc:304] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:22:46.011386 24701 raft_consensus.cc:2749] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:22:46.006388 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6f07842d3aa47858df5665874c86f3a" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
W20260709 13:22:46.018446 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 61642.3us
I20260709 13:22:46.019304 16293 raft_consensus.cc:2393] T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 25: Already voted for candidate 90f3da57796b488c9087a2f523adf045 in this term.
I20260709 13:22:46.002550 24660 leader_election.cc:290] T 5a959dbecc2648eb85d1cc550f5a3245 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 24 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
W20260709 13:22:45.978943 16191 leader_election.cc:336] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111)
I20260709 13:22:46.006372 16789 leader_election.cc:304] T 9e65676b21b748728ad39668cfe722ec P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:22:46.022921 24701 raft_consensus.cc:2804] T 9e65676b21b748728ad39668cfe722ec P cb82cc7bd6db46d980c75237a1402860 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:22:46.023329 24701 raft_consensus.cc:493] T 9e65676b21b748728ad39668cfe722ec P cb82cc7bd6db46d980c75237a1402860 [term 22 FOLLOWER]: Starting leader election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:22:46.023708 24701 raft_consensus.cc:3060] T 9e65676b21b748728ad39668cfe722ec P cb82cc7bd6db46d980c75237a1402860 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:22:46.034094 24701 raft_consensus.cc:515] T 9e65676b21b748728ad39668cfe722ec P cb82cc7bd6db46d980c75237a1402860 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:46.035543 16789 leader_election.cc:304] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:22:46.036448 24697 raft_consensus.cc:2804] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:22:46.036945 24697 raft_consensus.cc:493] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860 [term 23 FOLLOWER]: Starting leader election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:22:46.037613 24697 raft_consensus.cc:3060] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:22:46.038592 24276 consensus_queue.cc:1048] T 746aabf856ab401c9ef5e859371e7b67 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 13:22:46.040488 16192 leader_election.cc:304] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 647a343a542d4a139fe106c19095129b
I20260709 13:22:45.933171 24663 raft_consensus.cc:515] T 806d7690c2b644959152074bb2cab0a7 P cb82cc7bd6db46d980c75237a1402860 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:22:46.036856 16035 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.ChangeConfig -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 70261us
I20260709 13:22:46.041630 16818 raft_consensus.cc:2393] T 806d7690c2b644959152074bb2cab0a7 P cb82cc7bd6db46d980c75237a1402860 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 24: Already voted for candidate cb82cc7bd6db46d980c75237a1402860 in this term.
I20260709 13:22:46.041555 24276 raft_consensus.cc:2804] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:22:46.042660 24276 raft_consensus.cc:493] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac [term 23 FOLLOWER]: Starting leader election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:22:46.042995 24276 raft_consensus.cc:3060] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:22:46.046185 24663 leader_election.cc:290] T 806d7690c2b644959152074bb2cab0a7 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 24 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:46.018133 24669 raft_consensus.cc:515] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [term 27 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:46.048198 16044 catalog_manager.cc:5697] T ee9ed67006144797b1de6ea8c2013a68 P 1ca2725d290e47d68b1a4b609c9816ac reported cstate change: term changed from 21 to 22, leader changed from 647a343a542d4a139fe106c19095129b (127.14.177.129) to 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130). New cstate: current_term: 22 leader_uuid: "1ca2725d290e47d68b1a4b609c9816ac" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:46.050359 24669 leader_election.cc:290] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 27 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:46.048238 16190 leader_election.cc:304] T 84f2da98864b4425bf7c0055951fdd4b P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 647a343a542d4a139fe106c19095129b
I20260709 13:22:46.051685 24578 raft_consensus.cc:2804] T 84f2da98864b4425bf7c0055951fdd4b P 1ca2725d290e47d68b1a4b609c9816ac [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:22:46.052129 24578 raft_consensus.cc:697] T 84f2da98864b4425bf7c0055951fdd4b P 1ca2725d290e47d68b1a4b609c9816ac [term 22 LEADER]: Becoming Leader. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:22:46.052924 24578 consensus_queue.cc:237] T 84f2da98864b4425bf7c0055951fdd4b P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.12, Last appended by leader: 12, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:22:46.053421 16790 leader_election.cc:336] T 9e65676b21b748728ad39668cfe722ec P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111)
W20260709 13:22:46.063436 16035 catalog_manager.cc:4755] TS 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): ChangeConfig:REMOVE_PEER RPC failed for tablet 806d7690c2b644959152074bb2cab0a7: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111)
I20260709 13:22:46.074051 24697 raft_consensus.cc:515] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:46.074483 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806d7690c2b644959152074bb2cab0a7" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:22:46.094132 16293 raft_consensus.cc:3060] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045 [term 23 FOLLOWER]: Advancing to term 24
W20260709 13:22:46.020596 16265 consensus_peers.cc:597] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 90f3da57796b488c9087a2f523adf045 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:22:46.099459 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 80332.3us
W20260709 13:22:46.100062 16265 leader_election.cc:336] T 5a96373bb0e14dce8a266b2fc6c58507 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111)
W20260709 13:22:46.100546 16265 leader_election.cc:336] T 8a878999c0024d2abe517484180290fc P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111)
W20260709 13:22:46.074873 16790 leader_election.cc:336] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111)
I20260709 13:22:46.091017 16821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a9de347c711477e87c582d003f36925" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 27 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
W20260709 13:22:46.101081 16265 leader_election.cc:336] T 02838c86531d4f98bf58d1c22b8baf77 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111)
I20260709 13:22:46.101207 16821 raft_consensus.cc:3060] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860 [term 26 FOLLOWER]: Advancing to term 27
W20260709 13:22:46.101614 16265 consensus_peers.cc:597] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 90f3da57796b488c9087a2f523adf045 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:22:46.102028 16265 leader_election.cc:336] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111)
W20260709 13:22:46.107340 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 50936.6us
I20260709 13:22:46.108071 16190 leader_election.cc:304] T 5a96373bb0e14dce8a266b2fc6c58507 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 647a343a542d4a139fe106c19095129b
I20260709 13:22:46.108196 24697 leader_election.cc:290] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 24 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:46.109215 16190 leader_election.cc:304] T b6f07842d3aa47858df5665874c86f3a P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
W20260709 13:22:46.093379 16191 consensus_peers.cc:597] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:22:46.110440 16191 consensus_peers.cc:597] T c0eab25ce4c64e018a041beeda2867db P 1ca2725d290e47d68b1a4b609c9816ac -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:22:46.110755 24660 raft_consensus.cc:2804] T 5a96373bb0e14dce8a266b2fc6c58507 P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:22:46.111340 24687 raft_consensus.cc:2749] T b6f07842d3aa47858df5665874c86f3a P 1ca2725d290e47d68b1a4b609c9816ac [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 13:22:46.112351 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a959dbecc2648eb85d1cc550f5a3245" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:22:46.112937 16298 raft_consensus.cc:3060] T 5a959dbecc2648eb85d1cc550f5a3245 P 90f3da57796b488c9087a2f523adf045 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:22:46.113387 16192 leader_election.cc:304] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
I20260709 13:22:46.114108 24578 raft_consensus.cc:3060] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:22:46.115974 16295 raft_consensus.cc:1275] T 8158bbd17b994d2eb96f560bd0f1b88d P 90f3da57796b488c9087a2f523adf045 [term 24 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 23 index: 15. Preceding OpId from leader: term: 24 index: 16. (index mismatch)
I20260709 13:22:46.116428 24672 tablet_copy_client.cc:806] T 7c2095a5f4ad49408dbc7f69c606120d P cb82cc7bd6db46d980c75237a1402860: tablet copy: Starting download of 0 data blocks...
I20260709 13:22:46.117012 24672 tablet_copy_client.cc:670] T 7c2095a5f4ad49408dbc7f69c606120d P cb82cc7bd6db46d980c75237a1402860: tablet copy: Starting download of 1 WAL segments...
I20260709 13:22:46.119470 16295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e9b2bf7011f4ae688a79d7bb5a447b4" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:22:46.070859 24276 raft_consensus.cc:515] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:46.119612 16821 raft_consensus.cc:2468] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 27.
I20260709 13:22:46.120106 16295 raft_consensus.cc:3060] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 90f3da57796b488c9087a2f523adf045 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:22:46.117704 24660 raft_consensus.cc:493] T 5a96373bb0e14dce8a266b2fc6c58507 P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Starting leader election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:22:46.124539 24660 raft_consensus.cc:3060] T 5a96373bb0e14dce8a266b2fc6c58507 P 1ca2725d290e47d68b1a4b609c9816ac [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:22:46.126626 24578 raft_consensus.cc:2749] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:22:46.129084 24690 consensus_queue.cc:1048] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 13:22:46.137434 16298 raft_consensus.cc:2468] T 5a959dbecc2648eb85d1cc550f5a3245 P 90f3da57796b488c9087a2f523adf045 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 24.
I20260709 13:22:46.138548 24705 tablet_copy_client.cc:323] T 18ad7e3af5504571a69d10c00750a01c P cb82cc7bd6db46d980c75237a1402860: tablet copy: Beginning tablet copy session from remote peer at address 127.14.177.130:35943
I20260709 13:22:46.119612 16293 raft_consensus.cc:2468] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 24.
I20260709 13:22:46.143786 16190 leader_election.cc:304] T 5a959dbecc2648eb85d1cc550f5a3245 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:22:46.147433 24699 raft_consensus.cc:2804] T 5a959dbecc2648eb85d1cc550f5a3245 P 1ca2725d290e47d68b1a4b609c9816ac [term 24 FOLLOWER]: Leader election won for term 24
I20260709 13:22:46.148289 24276 leader_election.cc:290] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 24 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:46.148710 16295 raft_consensus.cc:2468] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 90f3da57796b488c9087a2f523adf045 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 24.
W20260709 13:22:46.151218 16191 leader_election.cc:336] T 5a959dbecc2648eb85d1cc550f5a3245 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 24 election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111)
I20260709 13:22:46.147972 24699 raft_consensus.cc:697] T 5a959dbecc2648eb85d1cc550f5a3245 P 1ca2725d290e47d68b1a4b609c9816ac [term 24 LEADER]: Becoming Leader. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:22:46.157374 24701 leader_election.cc:290] T 9e65676b21b748728ad39668cfe722ec P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 23 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:46.157290 24699 consensus_queue.cc:237] T 5a959dbecc2648eb85d1cc550f5a3245 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 23.14, Last appended by leader: 14, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:46.154409 16266 leader_election.cc:304] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 
W20260709 13:22:46.163767 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 45173.6us
I20260709 13:22:46.164198 16821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b7d1ab519044eda2d5e04a07242f25" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 24 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:22:46.165132 24660 raft_consensus.cc:515] T 5a96373bb0e14dce8a266b2fc6c58507 P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:46.166728 16224 raft_consensus.cc:1275] T 08c74490c40243a88e62d1760b5cd814 P 1ca2725d290e47d68b1a4b609c9816ac [term 22 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 21 index: 13. Preceding OpId from leader: term: 22 index: 14. (index mismatch)
I20260709 13:22:46.156066 16789 leader_election.cc:304] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:22:46.172310 16821 raft_consensus.cc:3060] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:22:46.178515 24691 raft_consensus.cc:2804] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [term 27 FOLLOWER]: Leader election won for term 27
I20260709 13:22:46.179092 24691 raft_consensus.cc:697] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [term 27 LEADER]: Becoming Leader. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:22:46.179729 24701 raft_consensus.cc:2804] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 13:22:46.179963 24691 consensus_queue.cc:237] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 26.17, Last appended by leader: 17, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:46.180477 24660 leader_election.cc:290] T 5a96373bb0e14dce8a266b2fc6c58507 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 19 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:46.180539 24701 raft_consensus.cc:697] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860 [term 24 LEADER]: Becoming Leader. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:22:46.181731 24701 consensus_queue.cc:237] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 23.15, Last appended by leader: 15, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:46.184149 16298 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a96373bb0e14dce8a266b2fc6c58507" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:22:46.184865 16298 raft_consensus.cc:3060] T 5a96373bb0e14dce8a266b2fc6c58507 P 90f3da57796b488c9087a2f523adf045 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:22:46.184985 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806d7690c2b644959152074bb2cab0a7" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:22:46.186141 16295 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e65676b21b748728ad39668cfe722ec" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:22:46.186697 16295 raft_consensus.cc:3060] T 9e65676b21b748728ad39668cfe722ec P 90f3da57796b488c9087a2f523adf045 [term 22 FOLLOWER]: Advancing to term 23
W20260709 13:22:46.189244 16191 consensus_peers.cc:597] T 746aabf856ab401c9ef5e859371e7b67 P 1ca2725d290e47d68b1a4b609c9816ac -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:22:46.190759 16821 raft_consensus.cc:2468] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 24.
I20260709 13:22:46.192286 16192 leader_election.cc:304] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:22:46.197340 24578 raft_consensus.cc:2804] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac [term 24 FOLLOWER]: Leader election won for term 24
I20260709 13:22:46.197991 24578 raft_consensus.cc:697] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac [term 24 LEADER]: Becoming Leader. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:22:46.207369 24578 consensus_queue.cc:237] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 23.18, Last appended by leader: 18, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:22:46.216723 16790 leader_election.cc:336] T 806d7690c2b644959152074bb2cab0a7 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 24 election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111)
I20260709 13:22:46.214011 16224 raft_consensus.cc:2468] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 24.
I20260709 13:22:46.217677 16298 raft_consensus.cc:2468] T 5a96373bb0e14dce8a266b2fc6c58507 P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 19.
I20260709 13:22:46.206282 16295 raft_consensus.cc:2468] T 9e65676b21b748728ad39668cfe722ec P 90f3da57796b488c9087a2f523adf045 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 23.
I20260709 13:22:46.206562 24672 tablet_copy_client.cc:538] T 7c2095a5f4ad49408dbc7f69c606120d P cb82cc7bd6db46d980c75237a1402860: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:22:46.230558 16190 leader_election.cc:304] T 5a96373bb0e14dce8a266b2fc6c58507 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:22:46.232489 16790 leader_election.cc:304] T 806d7690c2b644959152074bb2cab0a7 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 647a343a542d4a139fe106c19095129b
I20260709 13:22:46.234341 16789 leader_election.cc:304] T 9e65676b21b748728ad39668cfe722ec P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:22:46.240607 16265 leader_election.cc:304] T 02838c86531d4f98bf58d1c22b8baf77 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 647a343a542d4a139fe106c19095129b
I20260709 13:22:46.246279 24689 raft_consensus.cc:2804] T 9e65676b21b748728ad39668cfe722ec P cb82cc7bd6db46d980c75237a1402860 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:22:46.247078 16236 tablet_copy_service.cc:140] P 1ca2725d290e47d68b1a4b609c9816ac: Received BeginTabletCopySession request for tablet 18ad7e3af5504571a69d10c00750a01c from peer cb82cc7bd6db46d980c75237a1402860 ({username='slave'} at 127.0.0.1:33050)
I20260709 13:22:46.247766 16236 tablet_copy_service.cc:161] P 1ca2725d290e47d68b1a4b609c9816ac: Beginning new tablet copy session on tablet 18ad7e3af5504571a69d10c00750a01c from peer cb82cc7bd6db46d980c75237a1402860 at {username='slave'} at 127.0.0.1:33050: session id = cb82cc7bd6db46d980c75237a1402860-18ad7e3af5504571a69d10c00750a01c
I20260709 13:22:46.241779 24699 raft_consensus.cc:2804] T 5a96373bb0e14dce8a266b2fc6c58507 P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:22:46.251761 24699 raft_consensus.cc:697] T 5a96373bb0e14dce8a266b2fc6c58507 P 1ca2725d290e47d68b1a4b609c9816ac [term 19 LEADER]: Becoming Leader. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:22:46.252274 24672 tablet_bootstrap.cc:492] T 7c2095a5f4ad49408dbc7f69c606120d P cb82cc7bd6db46d980c75237a1402860: Bootstrap starting.
I20260709 13:22:46.252833 24699 consensus_queue.cc:237] T 5a96373bb0e14dce8a266b2fc6c58507 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:22:46.253522 16191 leader_election.cc:336] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 24 election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111)
W20260709 13:22:46.254318 16191 leader_election.cc:336] T 5a96373bb0e14dce8a266b2fc6c58507 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 19 election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111)
I20260709 13:22:46.258896 16265 leader_election.cc:304] T 8a878999c0024d2abe517484180290fc P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 647a343a542d4a139fe106c19095129b
W20260709 13:22:46.265010 16790 leader_election.cc:336] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 24 election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111)
W20260709 13:22:46.266549 16790 leader_election.cc:336] T 9e65676b21b748728ad39668cfe722ec P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 23 election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111)
I20260709 13:22:46.271531 16265 leader_election.cc:304] T 5a96373bb0e14dce8a266b2fc6c58507 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 647a343a542d4a139fe106c19095129b
I20260709 13:22:46.267297 24689 raft_consensus.cc:697] T 9e65676b21b748728ad39668cfe722ec P cb82cc7bd6db46d980c75237a1402860 [term 23 LEADER]: Becoming Leader. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:22:46.273283 24641 raft_consensus.cc:2804] T 8a878999c0024d2abe517484180290fc P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:22:46.273746 24641 raft_consensus.cc:493] T 8a878999c0024d2abe517484180290fc P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:22:46.274154 24641 raft_consensus.cc:3060] T 8a878999c0024d2abe517484180290fc P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:22:46.273844 24689 consensus_queue.cc:237] T 9e65676b21b748728ad39668cfe722ec P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 22.16, Last appended by leader: 16, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:46.282716 16265 leader_election.cc:304] T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:22:46.283378 24691 raft_consensus.cc:2764] T 5a96373bb0e14dce8a266b2fc6c58507 P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Leader pre-election decision vote started in defunct term 18: won
I20260709 13:22:46.241281 24664 raft_consensus.cc:2804] T 806d7690c2b644959152074bb2cab0a7 P cb82cc7bd6db46d980c75237a1402860 [term 24 FOLLOWER]: Leader election won for term 24
W20260709 13:22:46.292562 16265 leader_election.cc:336] T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 25 election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111)
I20260709 13:22:46.293154 16265 leader_election.cc:304] T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
W20260709 13:22:46.296605 16265 leader_election.cc:336] T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 26 election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111)
I20260709 13:22:46.294682 24659 raft_consensus.cc:2804] T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045 [term 26 FOLLOWER]: Leader election won for term 26
I20260709 13:22:46.298877 24659 raft_consensus.cc:697] T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045 [term 26 LEADER]: Becoming Leader. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:22:46.299886 24659 consensus_queue.cc:237] T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 25.14, Last appended by leader: 14, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:46.267590 24658 raft_consensus.cc:2804] T 02838c86531d4f98bf58d1c22b8baf77 P 90f3da57796b488c9087a2f523adf045 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:22:46.296140 24685 consensus_queue.cc:1048] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.024s
I20260709 13:22:46.319734 24641 raft_consensus.cc:515] T 8a878999c0024d2abe517484180290fc P 90f3da57796b488c9087a2f523adf045 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:46.303874 24688 raft_consensus.cc:2749] T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045 [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 13:22:46.322183 24641 leader_election.cc:290] T 8a878999c0024d2abe517484180290fc P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 21 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:22:46.337695 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a878999c0024d2abe517484180290fc" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:22:46.338320 16224 raft_consensus.cc:3060] T 8a878999c0024d2abe517484180290fc P 1ca2725d290e47d68b1a4b609c9816ac [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:22:46.346489 16224 raft_consensus.cc:2468] T 8a878999c0024d2abe517484180290fc P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 21.
I20260709 13:22:46.311434 24664 raft_consensus.cc:697] T 806d7690c2b644959152074bb2cab0a7 P cb82cc7bd6db46d980c75237a1402860 [term 24 LEADER]: Becoming Leader. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:22:46.311205 24658 raft_consensus.cc:493] T 02838c86531d4f98bf58d1c22b8baf77 P 90f3da57796b488c9087a2f523adf045 [term 24 FOLLOWER]: Starting leader election (detected failure of leader 647a343a542d4a139fe106c19095129b)
I20260709 13:22:46.349258 16265 leader_election.cc:304] T 8a878999c0024d2abe517484180290fc P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:22:46.351264 24658 raft_consensus.cc:3060] T 02838c86531d4f98bf58d1c22b8baf77 P 90f3da57796b488c9087a2f523adf045 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:22:46.356485 24686 raft_consensus.cc:2804] T 8a878999c0024d2abe517484180290fc P 90f3da57796b488c9087a2f523adf045 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:22:46.357123 24686 raft_consensus.cc:697] T 8a878999c0024d2abe517484180290fc P 90f3da57796b488c9087a2f523adf045 [term 21 LEADER]: Becoming Leader. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:22:46.358054 24686 consensus_queue.cc:237] T 8a878999c0024d2abe517484180290fc P 90f3da57796b488c9087a2f523adf045 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.11, Last appended by leader: 11, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:22:46.373354 16265 consensus_peers.cc:597] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:22:46.373761 16236 tablet_copy_source_session.cc:215] T 18ad7e3af5504571a69d10c00750a01c P 1ca2725d290e47d68b1a4b609c9816ac: Tablet Copy: opened 0 blocks and 1 log segments
W20260709 13:22:46.373862 16265 leader_election.cc:336] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 27 election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111)
W20260709 13:22:46.379441 16790 consensus_peers.cc:597] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:22:46.379799 16044 catalog_manager.cc:5697] T 84f2da98864b4425bf7c0055951fdd4b P 1ca2725d290e47d68b1a4b609c9816ac reported cstate change: term changed from 21 to 22, leader changed from 647a343a542d4a139fe106c19095129b (127.14.177.129) to 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130). New cstate: current_term: 22 leader_uuid: "1ca2725d290e47d68b1a4b609c9816ac" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:46.386181 16044 catalog_manager.cc:5697] T 5a959dbecc2648eb85d1cc550f5a3245 P 1ca2725d290e47d68b1a4b609c9816ac reported cstate change: term changed from 23 to 24, leader changed from 647a343a542d4a139fe106c19095129b (127.14.177.129) to 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130). New cstate: current_term: 24 leader_uuid: "1ca2725d290e47d68b1a4b609c9816ac" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:46.374302 24664 consensus_queue.cc:237] T 806d7690c2b644959152074bb2cab0a7 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 23.16, Last appended by leader: 16, Current term: 24, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:46.431787 24705 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 18ad7e3af5504571a69d10c00750a01c. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:22:46.453138 16265 leader_election.cc:336] T 8a878999c0024d2abe517484180290fc P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 21 election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111)
I20260709 13:22:46.458457 16821 raft_consensus.cc:1275] T ee9ed67006144797b1de6ea8c2013a68 P cb82cc7bd6db46d980c75237a1402860 [term 22 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 21 index: 16. Preceding OpId from leader: term: 22 index: 17. (index mismatch)
I20260709 13:22:46.460526 24276 consensus_queue.cc:1048] T ee9ed67006144797b1de6ea8c2013a68 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
W20260709 13:22:46.476084 16265 consensus_peers.cc:597] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:22:46.476653 16265 consensus_peers.cc:597] T e4dc6b0541ca431ebc757beac00521f1 P 90f3da57796b488c9087a2f523adf045 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:22:46.482832 24658 raft_consensus.cc:515] T 02838c86531d4f98bf58d1c22b8baf77 P 90f3da57796b488c9087a2f523adf045 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:46.514299 16045 catalog_manager.cc:5697] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860 reported cstate change: term changed from 23 to 24, leader changed from 647a343a542d4a139fe106c19095129b (127.14.177.129) to cb82cc7bd6db46d980c75237a1402860 (127.14.177.132). New cstate: current_term: 24 leader_uuid: "cb82cc7bd6db46d980c75237a1402860" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:22:46.515587 16040 catalog_manager.cc:5697] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 reported cstate change: term changed from 26 to 27, leader changed from 647a343a542d4a139fe106c19095129b (127.14.177.129) to 90f3da57796b488c9087a2f523adf045 (127.14.177.131). New cstate: current_term: 27 leader_uuid: "90f3da57796b488c9087a2f523adf045" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 13:22:46.516538 16191 consensus_peers.cc:597] T ee9ed67006144797b1de6ea8c2013a68 P 1ca2725d290e47d68b1a4b609c9816ac -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:22:46.517107 16191 consensus_peers.cc:597] T af57996a8be84d398f0f023e8d08ef77 P 1ca2725d290e47d68b1a4b609c9816ac -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:22:46.516324 16045 catalog_manager.cc:5697] T 9e65676b21b748728ad39668cfe722ec P cb82cc7bd6db46d980c75237a1402860 reported cstate change: term changed from 22 to 23, leader changed from 647a343a542d4a139fe106c19095129b (127.14.177.129) to cb82cc7bd6db46d980c75237a1402860 (127.14.177.132). New cstate: current_term: 23 leader_uuid: "cb82cc7bd6db46d980c75237a1402860" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:22:46.517465 16040 catalog_manager.cc:5697] T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045 reported cstate change: term changed from 25 to 26, leader changed from 647a343a542d4a139fe106c19095129b (127.14.177.129) to 90f3da57796b488c9087a2f523adf045 (127.14.177.131). New cstate: current_term: 26 leader_uuid: "90f3da57796b488c9087a2f523adf045" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } health_report { overall_health: HEALTHY } } }
I20260709 13:22:46.513809 24658 leader_election.cc:290] T 02838c86531d4f98bf58d1c22b8baf77 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 25 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
W20260709 13:22:46.518500 16191 consensus_peers.cc:597] T 84f2da98864b4425bf7c0055951fdd4b P 1ca2725d290e47d68b1a4b609c9816ac -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:22:46.529207 16295 raft_consensus.cc:1275] T 84f2da98864b4425bf7c0055951fdd4b P 90f3da57796b488c9087a2f523adf045 [term 22 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 21 index: 12. Preceding OpId from leader: term: 22 index: 13. (index mismatch)
I20260709 13:22:46.531358 24667 consensus_queue.cc:1048] T 84f2da98864b4425bf7c0055951fdd4b P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:22:46.556355 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02838c86531d4f98bf58d1c22b8baf77" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:22:46.557652 16224 raft_consensus.cc:3060] T 02838c86531d4f98bf58d1c22b8baf77 P 1ca2725d290e47d68b1a4b609c9816ac [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:22:46.570114 24705 tablet_copy_client.cc:806] T 18ad7e3af5504571a69d10c00750a01c P cb82cc7bd6db46d980c75237a1402860: tablet copy: Starting download of 0 data blocks...
I20260709 13:22:46.570922 24705 tablet_copy_client.cc:670] T 18ad7e3af5504571a69d10c00750a01c P cb82cc7bd6db46d980c75237a1402860: tablet copy: Starting download of 1 WAL segments...
I20260709 13:22:46.579082 24672 tablet_bootstrap.cc:492] T 7c2095a5f4ad49408dbc7f69c606120d P cb82cc7bd6db46d980c75237a1402860: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=4 applied=10 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:22:46.594750 24672 tablet_bootstrap.cc:492] T 7c2095a5f4ad49408dbc7f69c606120d P cb82cc7bd6db46d980c75237a1402860: Bootstrap complete.
I20260709 13:22:46.595628 24672 ts_tablet_manager.cc:1403] T 7c2095a5f4ad49408dbc7f69c606120d P cb82cc7bd6db46d980c75237a1402860: Time spent bootstrapping tablet: real 0.344s	user 0.052s	sys 0.039s
I20260709 13:22:46.595424 16045 catalog_manager.cc:5697] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac reported cstate change: term changed from 23 to 24, leader changed from 647a343a542d4a139fe106c19095129b (127.14.177.129) to 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130). New cstate: current_term: 24 leader_uuid: "1ca2725d290e47d68b1a4b609c9816ac" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:46.602039 16045 catalog_manager.cc:5697] T 5a96373bb0e14dce8a266b2fc6c58507 P 1ca2725d290e47d68b1a4b609c9816ac reported cstate change: term changed from 18 to 19, leader changed from 647a343a542d4a139fe106c19095129b (127.14.177.129) to 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130). New cstate: current_term: 19 leader_uuid: "1ca2725d290e47d68b1a4b609c9816ac" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:46.612599 24705 tablet_copy_client.cc:538] T 18ad7e3af5504571a69d10c00750a01c P cb82cc7bd6db46d980c75237a1402860: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:22:46.617290 24672 raft_consensus.cc:359] T 7c2095a5f4ad49408dbc7f69c606120d P cb82cc7bd6db46d980c75237a1402860 [term 19 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } }
I20260709 13:22:46.618270 24672 raft_consensus.cc:740] T 7c2095a5f4ad49408dbc7f69c606120d P cb82cc7bd6db46d980c75237a1402860 [term 19 LEARNER]: Becoming Follower/Learner. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Initialized, Role: LEARNER
I20260709 13:22:46.618985 24672 consensus_queue.cc:260] T 7c2095a5f4ad49408dbc7f69c606120d P cb82cc7bd6db46d980c75237a1402860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 10, Last appended: 19.10, Last appended by leader: 10, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } }
I20260709 13:22:46.622761 16224 raft_consensus.cc:2468] T 02838c86531d4f98bf58d1c22b8baf77 P 1ca2725d290e47d68b1a4b609c9816ac [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 25.
I20260709 13:22:46.614372 16040 catalog_manager.cc:5697] T 806d7690c2b644959152074bb2cab0a7 P cb82cc7bd6db46d980c75237a1402860 reported cstate change: term changed from 23 to 24, leader changed from 647a343a542d4a139fe106c19095129b (127.14.177.129) to cb82cc7bd6db46d980c75237a1402860 (127.14.177.132). New cstate: current_term: 24 leader_uuid: "cb82cc7bd6db46d980c75237a1402860" committed_config { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:22:46.646654 16821 raft_consensus.cc:1275] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 [term 24 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 23 index: 18. Preceding OpId from leader: term: 24 index: 19. (index mismatch)
W20260709 13:22:46.633208 16265 leader_election.cc:336] T 02838c86531d4f98bf58d1c22b8baf77 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 25 election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111)
I20260709 13:22:46.633870 16291 raft_consensus.cc:1275] T 5a959dbecc2648eb85d1cc550f5a3245 P 90f3da57796b488c9087a2f523adf045 [term 24 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 23 index: 14. Preceding OpId from leader: term: 24 index: 15. (index mismatch)
I20260709 13:22:46.652351 24276 consensus_queue.cc:1048] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.000s
I20260709 13:22:46.658012 24276 consensus_queue.cc:1048] T 5a959dbecc2648eb85d1cc550f5a3245 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:22:46.646746 24705 tablet_bootstrap.cc:492] T 18ad7e3af5504571a69d10c00750a01c P cb82cc7bd6db46d980c75237a1402860: Bootstrap starting.
I20260709 13:22:46.638489 24672 ts_tablet_manager.cc:1434] T 7c2095a5f4ad49408dbc7f69c606120d P cb82cc7bd6db46d980c75237a1402860: Time spent starting tablet: real 0.042s	user 0.005s	sys 0.002s
I20260709 13:22:46.676158 16310 tablet_copy_service.cc:342] P 90f3da57796b488c9087a2f523adf045: Request end of tablet copy session cb82cc7bd6db46d980c75237a1402860-7c2095a5f4ad49408dbc7f69c606120d received from {username='slave'} at 127.0.0.1:40474
I20260709 13:22:46.676736 16310 tablet_copy_service.cc:434] P 90f3da57796b488c9087a2f523adf045: ending tablet copy session cb82cc7bd6db46d980c75237a1402860-7c2095a5f4ad49408dbc7f69c606120d on tablet 7c2095a5f4ad49408dbc7f69c606120d with peer cb82cc7bd6db46d980c75237a1402860
I20260709 13:22:46.677095 16265 leader_election.cc:304] T 02838c86531d4f98bf58d1c22b8baf77 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 647a343a542d4a139fe106c19095129b
I20260709 13:22:46.679350 24658 raft_consensus.cc:2804] T 02838c86531d4f98bf58d1c22b8baf77 P 90f3da57796b488c9087a2f523adf045 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 13:22:46.679903 24658 raft_consensus.cc:697] T 02838c86531d4f98bf58d1c22b8baf77 P 90f3da57796b488c9087a2f523adf045 [term 25 LEADER]: Becoming Leader. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:22:46.681615 24658 consensus_queue.cc:237] T 02838c86531d4f98bf58d1c22b8baf77 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 24.14, Last appended by leader: 14, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:22:46.695583 16191 consensus_peers.cc:597] T 5a959dbecc2648eb85d1cc550f5a3245 P 1ca2725d290e47d68b1a4b609c9816ac -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:22:46.692399 16045 catalog_manager.cc:5697] T 8a878999c0024d2abe517484180290fc P 90f3da57796b488c9087a2f523adf045 reported cstate change: term changed from 20 to 21, leader changed from 647a343a542d4a139fe106c19095129b (127.14.177.129) to 90f3da57796b488c9087a2f523adf045 (127.14.177.131). New cstate: current_term: 21 leader_uuid: "90f3da57796b488c9087a2f523adf045" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } health_report { overall_health: HEALTHY } } }
I20260709 13:22:46.712199 16223 raft_consensus.cc:1275] T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac [term 26 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 25 index: 14. Preceding OpId from leader: term: 26 index: 15. (index mismatch)
I20260709 13:22:46.721863 24695 consensus_queue.cc:1048] T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:22:46.755679 16817 raft_consensus.cc:1275] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860 [term 27 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 26 index: 17. Preceding OpId from leader: term: 27 index: 18. (index mismatch)
I20260709 13:22:46.760143 24685 consensus_queue.cc:1048] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.002s
W20260709 13:22:46.766808 16191 consensus_peers.cc:597] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:22:46.773296 16045 catalog_manager.cc:5697] T 02838c86531d4f98bf58d1c22b8baf77 P 90f3da57796b488c9087a2f523adf045 reported cstate change: term changed from 24 to 25, leader changed from 647a343a542d4a139fe106c19095129b (127.14.177.129) to 90f3da57796b488c9087a2f523adf045 (127.14.177.131). New cstate: current_term: 25 leader_uuid: "90f3da57796b488c9087a2f523adf045" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } health_report { overall_health: HEALTHY } } }
W20260709 13:22:46.776202 16265 consensus_peers.cc:597] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:22:46.778889 16790 consensus_peers.cc:597] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:22:46.779224 16265 consensus_peers.cc:597] T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:22:46.823523 16298 raft_consensus.cc:1275] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 90f3da57796b488c9087a2f523adf045 [term 24 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 23 index: 15. Preceding OpId from leader: term: 24 index: 16. (index mismatch)
I20260709 13:22:46.831385 24664 consensus_queue.cc:1048] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 13:22:46.831821 16224 raft_consensus.cc:1275] T 8a878999c0024d2abe517484180290fc P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
W20260709 13:22:46.839651 16265 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111) [suppressed 371 similar messages]
I20260709 13:22:46.851312 16291 raft_consensus.cc:1275] T 5a96373bb0e14dce8a266b2fc6c58507 P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 13:22:46.855777 16295 raft_consensus.cc:1275] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045 [term 24 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 23 index: 16. Preceding OpId from leader: term: 24 index: 17. (index mismatch)
I20260709 13:22:46.857550 24627 consensus_queue.cc:1048] T 806d7690c2b644959152074bb2cab0a7 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 13:22:46.871546 24658 consensus_queue.cc:1048] T 8a878999c0024d2abe517484180290fc P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
W20260709 13:22:46.875241 16191 consensus_peers.cc:597] T 5a96373bb0e14dce8a266b2fc6c58507 P 1ca2725d290e47d68b1a4b609c9816ac -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:22:46.879340 24690 consensus_queue.cc:1048] T 5a96373bb0e14dce8a266b2fc6c58507 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
W20260709 13:22:46.882486 15046 thread.cc:527] Waited for 2000ms trying to join with diag-logger (tid 16172)
I20260709 13:22:46.884639 16293 raft_consensus.cc:1275] T 9e65676b21b748728ad39668cfe722ec P 90f3da57796b488c9087a2f523adf045 [term 23 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 22 index: 16. Preceding OpId from leader: term: 23 index: 17. (index mismatch)
I20260709 13:22:46.894676 16817 raft_consensus.cc:1217] T 7c2095a5f4ad49408dbc7f69c606120d P cb82cc7bd6db46d980c75237a1402860 [term 19 LEARNER]: Deduplicated request from leader. Original: 19.9->[19.10-19.10]   Dedup: 19.10->[]
I20260709 13:22:46.894637 24664 consensus_queue.cc:1048] T 9e65676b21b748728ad39668cfe722ec P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
W20260709 13:22:46.915639 16191 consensus_peers.cc:597] T d16dfad0c3b540e5b4b86492ddf87208 P 1ca2725d290e47d68b1a4b609c9816ac -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:22:47.002039 16790 consensus_peers.cc:597] T 9e65676b21b748728ad39668cfe722ec P cb82cc7bd6db46d980c75237a1402860 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:22:47.008654 16224 raft_consensus.cc:1275] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac [term 24 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 23 index: 16. Preceding OpId from leader: term: 24 index: 17. (index mismatch)
W20260709 13:22:47.006577 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 50500us
I20260709 13:22:47.018254 24627 consensus_queue.cc:1048] T 806d7690c2b644959152074bb2cab0a7 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.003s
I20260709 13:22:47.059199 24686 raft_consensus.cc:493] T 1df93779209748b9860f6d893dd3de53 P 90f3da57796b488c9087a2f523adf045 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:22:47.111687 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 97021.4us
I20260709 13:22:47.116276 24686 raft_consensus.cc:515] T 1df93779209748b9860f6d893dd3de53 P 90f3da57796b488c9087a2f523adf045 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:47.215072 24686 leader_election.cc:290] T 1df93779209748b9860f6d893dd3de53 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
W20260709 13:22:47.229610 16790 consensus_peers.cc:597] T 806d7690c2b644959152074bb2cab0a7 P cb82cc7bd6db46d980c75237a1402860 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:22:47.265710 24705 tablet_bootstrap.cc:492] T 18ad7e3af5504571a69d10c00750a01c P cb82cc7bd6db46d980c75237a1402860: Bootstrap replayed 1/1 log segments. Stats: ops{read=15 overwritten=0 applied=15 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:22:47.267328 24705 tablet_bootstrap.cc:492] T 18ad7e3af5504571a69d10c00750a01c P cb82cc7bd6db46d980c75237a1402860: Bootstrap complete.
I20260709 13:22:47.271301 24705 ts_tablet_manager.cc:1403] T 18ad7e3af5504571a69d10c00750a01c P cb82cc7bd6db46d980c75237a1402860: Time spent bootstrapping tablet: real 0.625s	user 0.048s	sys 0.013s
I20260709 13:22:47.276186 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1df93779209748b9860f6d893dd3de53" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
W20260709 13:22:47.278523 16265 consensus_peers.cc:597] T 8a878999c0024d2abe517484180290fc P 90f3da57796b488c9087a2f523adf045 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:22:47.280083 16224 raft_consensus.cc:2468] T 1df93779209748b9860f6d893dd3de53 P 1ca2725d290e47d68b1a4b609c9816ac [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 22.
W20260709 13:22:47.280211 16265 consensus_peers.cc:597] T 02838c86531d4f98bf58d1c22b8baf77 P 90f3da57796b488c9087a2f523adf045 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:22:47.280889 16265 consensus_peers.cc:597] T b87147731a68400e8cce9b879daa7c5d P 90f3da57796b488c9087a2f523adf045 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:22:47.284464 16191 consensus_peers.cc:597] T 68a6838d82ef4cac83507f906139d90f P 1ca2725d290e47d68b1a4b609c9816ac -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:22:47.295454 24705 raft_consensus.cc:359] T 18ad7e3af5504571a69d10c00750a01c P cb82cc7bd6db46d980c75237a1402860 [term 19 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } }
I20260709 13:22:47.303938 16265 leader_election.cc:304] T 1df93779209748b9860f6d893dd3de53 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 
I20260709 13:22:47.304818 24686 raft_consensus.cc:2804] T 1df93779209748b9860f6d893dd3de53 P 90f3da57796b488c9087a2f523adf045 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:22:47.305243 24686 raft_consensus.cc:493] T 1df93779209748b9860f6d893dd3de53 P 90f3da57796b488c9087a2f523adf045 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:47.305596 24686 raft_consensus.cc:3060] T 1df93779209748b9860f6d893dd3de53 P 90f3da57796b488c9087a2f523adf045 [term 22 FOLLOWER]: Advancing to term 23
W20260709 13:22:47.310482 16265 leader_election.cc:336] T 1df93779209748b9860f6d893dd3de53 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111)
I20260709 13:22:47.296589 24705 raft_consensus.cc:740] T 18ad7e3af5504571a69d10c00750a01c P cb82cc7bd6db46d980c75237a1402860 [term 19 LEARNER]: Becoming Follower/Learner. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Initialized, Role: LEARNER
I20260709 13:22:47.322041 24705 consensus_queue.cc:260] T 18ad7e3af5504571a69d10c00750a01c P cb82cc7bd6db46d980c75237a1402860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } }
I20260709 13:22:47.335451 24705 ts_tablet_manager.cc:1434] T 18ad7e3af5504571a69d10c00750a01c P cb82cc7bd6db46d980c75237a1402860: Time spent starting tablet: real 0.056s	user 0.006s	sys 0.002s
I20260709 13:22:47.345649 24686 raft_consensus.cc:515] T 1df93779209748b9860f6d893dd3de53 P 90f3da57796b488c9087a2f523adf045 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:47.348225 24686 leader_election.cc:290] T 1df93779209748b9860f6d893dd3de53 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 23 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:22:47.364319 16223 raft_consensus.cc:1275] T 02838c86531d4f98bf58d1c22b8baf77 P 1ca2725d290e47d68b1a4b609c9816ac [term 25 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 24 index: 14. Preceding OpId from leader: term: 25 index: 15. (index mismatch)
W20260709 13:22:47.387737 16265 leader_election.cc:336] T 1df93779209748b9860f6d893dd3de53 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 23 election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111)
I20260709 13:22:47.389372 16223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1df93779209748b9860f6d893dd3de53" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:22:47.394003 16223 raft_consensus.cc:3060] T 1df93779209748b9860f6d893dd3de53 P 1ca2725d290e47d68b1a4b609c9816ac [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:22:47.397747 16236 tablet_copy_service.cc:342] P 1ca2725d290e47d68b1a4b609c9816ac: Request end of tablet copy session cb82cc7bd6db46d980c75237a1402860-18ad7e3af5504571a69d10c00750a01c received from {username='slave'} at 127.0.0.1:33050
I20260709 13:22:47.398312 16236 tablet_copy_service.cc:434] P 1ca2725d290e47d68b1a4b609c9816ac: ending tablet copy session cb82cc7bd6db46d980c75237a1402860-18ad7e3af5504571a69d10c00750a01c on tablet 18ad7e3af5504571a69d10c00750a01c with peer cb82cc7bd6db46d980c75237a1402860
I20260709 13:22:47.413341 16817 raft_consensus.cc:1217] T 18ad7e3af5504571a69d10c00750a01c P cb82cc7bd6db46d980c75237a1402860 [term 19 LEARNER]: Deduplicated request from leader. Original: 19.14->[19.15-19.15]   Dedup: 19.15->[]
I20260709 13:22:47.438890 16223 raft_consensus.cc:2468] T 1df93779209748b9860f6d893dd3de53 P 1ca2725d290e47d68b1a4b609c9816ac [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 23.
I20260709 13:22:47.443029 24686 consensus_queue.cc:1048] T 02838c86531d4f98bf58d1c22b8baf77 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
W20260709 13:22:47.451120 16790 consensus_peers.cc:597] T 3884f864a96d428785895e5463e5728f P cb82cc7bd6db46d980c75237a1402860 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 13:22:47.466771 16265 leader_election.cc:304] T 1df93779209748b9860f6d893dd3de53 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 647a343a542d4a139fe106c19095129b
I20260709 13:22:47.467620 24686 raft_consensus.cc:2804] T 1df93779209748b9860f6d893dd3de53 P 90f3da57796b488c9087a2f523adf045 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:22:47.468472 24686 raft_consensus.cc:697] T 1df93779209748b9860f6d893dd3de53 P 90f3da57796b488c9087a2f523adf045 [term 23 LEADER]: Becoming Leader. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:22:47.469447 24686 consensus_queue.cc:237] T 1df93779209748b9860f6d893dd3de53 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 21.13, Last appended by leader: 13, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:47.486487 24745 raft_consensus.cc:1064] T 7c2095a5f4ad49408dbc7f69c606120d P 90f3da57796b488c9087a2f523adf045: attempting to promote NON_VOTER cb82cc7bd6db46d980c75237a1402860 to VOTER
I20260709 13:22:47.491881 24745 consensus_queue.cc:237] T 7c2095a5f4ad49408dbc7f69c606120d P 90f3da57796b488c9087a2f523adf045 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.10, Last appended by leader: 8, Current term: 19, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:22:47.513002 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 46323.3us
W20260709 13:22:47.593516 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 53466.6us
I20260709 13:22:47.610662 16817 raft_consensus.cc:1275] T 7c2095a5f4ad49408dbc7f69c606120d P cb82cc7bd6db46d980c75237a1402860 [term 19 LEARNER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 19 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 13:22:47.612478 24695 consensus_queue.cc:1048] T 7c2095a5f4ad49408dbc7f69c606120d P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
W20260709 13:22:47.632588 16790 consensus_peers.cc:597] T 2794af5b99984efebdee564035841cb0 P cb82cc7bd6db46d980c75237a1402860 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 13:22:47.647197 16045 catalog_manager.cc:5697] T 1df93779209748b9860f6d893dd3de53 P 90f3da57796b488c9087a2f523adf045 reported cstate change: term changed from 21 to 23, leader changed from 647a343a542d4a139fe106c19095129b (127.14.177.129) to 90f3da57796b488c9087a2f523adf045 (127.14.177.131). New cstate: current_term: 23 leader_uuid: "90f3da57796b488c9087a2f523adf045" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } health_report { overall_health: HEALTHY } } }
I20260709 13:22:47.662312 24736 raft_consensus.cc:493] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:47.662943 24736 raft_consensus.cc:515] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:47.665185 24736 leader_election.cc:290] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:22:47.670485 16224 raft_consensus.cc:1275] T 7c2095a5f4ad49408dbc7f69c606120d P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 19 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 13:22:47.674744 24740 raft_consensus.cc:493] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:47.675374 24740 raft_consensus.cc:515] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:47.677690 24740 leader_election.cc:290] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:47.684844 16817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62fe3346cd3c42c4a55e14af9073bca4" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 29 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:47.685168 24695 consensus_queue.cc:1048] T 7c2095a5f4ad49408dbc7f69c606120d P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:22:47.685899 16817 raft_consensus.cc:2468] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 28.
W20260709 13:22:47.689019 16265 consensus_peers.cc:597] T 434bff3082d141838068f323bdb0d1ed P 90f3da57796b488c9087a2f523adf045 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:22:47.691263 16265 consensus_peers.cc:597] T 7c2095a5f4ad49408dbc7f69c606120d P 90f3da57796b488c9087a2f523adf045 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:22:47.695982 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62fe3346cd3c42c4a55e14af9073bca4" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 29 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:47.696878 16224 raft_consensus.cc:2468] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 28.
I20260709 13:22:47.706624 16790 leader_election.cc:304] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:22:47.709844 16192 leader_election.cc:304] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:22:47.726447 24687 raft_consensus.cc:2804] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 13:22:47.726979 24687 raft_consensus.cc:493] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [term 28 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:47.728179 24687 raft_consensus.cc:3060] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [term 28 FOLLOWER]: Advancing to term 29
W20260709 13:22:47.728245 16265 consensus_peers.cc:597] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 13:22:47.718199 24664 raft_consensus.cc:2804] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 13:22:47.735744 24664 raft_consensus.cc:493] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [term 28 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:47.736469 24664 raft_consensus.cc:3060] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [term 28 FOLLOWER]: Advancing to term 29
W20260709 13:22:47.753262 16790 leader_election.cc:336] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111)
W20260709 13:22:47.789335 16192 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 79970.4us
W20260709 13:22:47.818349 16790 consensus_peers.cc:597] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:22:47.832674 24664 raft_consensus.cc:515] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [term 29 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:47.834324 24687 raft_consensus.cc:515] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [term 29 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:22:47.871129 16191 leader_election.cc:336] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111)
W20260709 13:22:47.883252 15046 thread.cc:527] Waited for 3000ms trying to join with diag-logger (tid 16172)
I20260709 13:22:47.959046 24745 raft_consensus.cc:493] T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:47.962373 24745 raft_consensus.cc:515] T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:22:47.966348 24745 leader_election.cc:290] T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:22:47.954646 24687 leader_election.cc:290] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 29 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:47.979538 16821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62fe3346cd3c42c4a55e14af9073bca4" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 29 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:22:47.981037 16821 raft_consensus.cc:2393] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [term 29 FOLLOWER]: Leader election vote request: Denying vote to candidate 1ca2725d290e47d68b1a4b609c9816ac in current term 29: Already voted for candidate cb82cc7bd6db46d980c75237a1402860 in this term.
I20260709 13:22:47.989902 24664 leader_election.cc:290] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 29 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
W20260709 13:22:47.997308 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 92101.5us
W20260709 13:22:48.005990 16265 consensus_peers.cc:597] T fffab26427e24d398e2695aad40ca134 P 90f3da57796b488c9087a2f523adf045 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:22:48.054430 16191 leader_election.cc:336] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 29 election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111)
I20260709 13:22:48.057538 16191 leader_election.cc:304] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 29 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
I20260709 13:22:48.058964 16223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6f07842d3aa47858df5665874c86f3a" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 26 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:22:48.068955 16220 raft_consensus.cc:2955] T 7c2095a5f4ad49408dbc7f69c606120d P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Committing config change with OpId 19.11: config changed from index 10 to 11, cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:22:48.081138 24658 raft_consensus.cc:2955] T 7c2095a5f4ad49408dbc7f69c606120d P 90f3da57796b488c9087a2f523adf045 [term 19 LEADER]: Committing config change with OpId 19.11: config changed from index 10 to 11, cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:22:48.085489 24667 raft_consensus.cc:2749] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [term 29 FOLLOWER]: Leader election lost for term 29. Reason: could not achieve majority
I20260709 13:22:48.095932 16223 raft_consensus.cc:2468] T b6f07842d3aa47858df5665874c86f3a P 1ca2725d290e47d68b1a4b609c9816ac [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 25.
I20260709 13:22:48.105746 24763 raft_consensus.cc:1064] T 18ad7e3af5504571a69d10c00750a01c P 1ca2725d290e47d68b1a4b609c9816ac: attempting to promote NON_VOTER cb82cc7bd6db46d980c75237a1402860 to VOTER
I20260709 13:22:48.110872 24763 consensus_queue.cc:237] T 18ad7e3af5504571a69d10c00750a01c P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 13, Current term: 19, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:22:48.140769 16790 consensus_peers.cc:597] T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:22:48.145457 16265 consensus_peers.cc:597] T 26354361e9b044cea1c2b86c4882fe92 P 90f3da57796b488c9087a2f523adf045 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:22:48.148682 16265 leader_election.cc:336] T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111)
W20260709 13:22:48.149303 16265 consensus_peers.cc:597] T 1df93779209748b9860f6d893dd3de53 P 90f3da57796b488c9087a2f523adf045 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:22:48.154068 16265 leader_election.cc:304] T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 647a343a542d4a139fe106c19095129b
I20260709 13:22:48.155073 24745 raft_consensus.cc:2804] T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 13:22:48.159195 24745 raft_consensus.cc:493] T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045 [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:48.173110 16219 raft_consensus.cc:1275] T 1df93779209748b9860f6d893dd3de53 P 1ca2725d290e47d68b1a4b609c9816ac [term 23 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 21 index: 13. Preceding OpId from leader: term: 23 index: 14. (index mismatch)
I20260709 13:22:48.171590 24745 raft_consensus.cc:3060] T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045 [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:22:48.180806 16821 consensus_queue.cc:237] T 806d7690c2b644959152074bb2cab0a7 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 24.17, Last appended by leader: 16, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:48.175076 16817 raft_consensus.cc:2955] T 7c2095a5f4ad49408dbc7f69c606120d P cb82cc7bd6db46d980c75237a1402860 [term 19 FOLLOWER]: Committing config change with OpId 19.11: config changed from index 10 to 11, cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:22:48.202775 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62fe3346cd3c42c4a55e14af9073bca4" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 29 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:22:48.209924 16224 raft_consensus.cc:2393] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [term 29 FOLLOWER]: Leader election vote request: Denying vote to candidate cb82cc7bd6db46d980c75237a1402860 in current term 29: Already voted for candidate 1ca2725d290e47d68b1a4b609c9816ac in this term.
I20260709 13:22:48.208621 24745 raft_consensus.cc:515] T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045 [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:22:48.221185 16191 consensus_peers.cc:597] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 13:22:48.221928 24745 leader_election.cc:290] T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 26 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:22:48.226078 16299 raft_consensus.cc:1275] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045 [term 24 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 24 index: 17. Preceding OpId from leader: term: 24 index: 18. (index mismatch)
I20260709 13:22:48.228982 24769 consensus_queue.cc:1048] T 806d7690c2b644959152074bb2cab0a7 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.001s
W20260709 13:22:48.249558 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 88798.7us
I20260709 13:22:48.248549 24769 raft_consensus.cc:2955] T 806d7690c2b644959152074bb2cab0a7 P cb82cc7bd6db46d980c75237a1402860 [term 24 LEADER]: Committing config change with OpId 24.18: config changed from index 15 to 18, VOTER 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130) evicted. New config: { opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:22:48.254662 16295 raft_consensus.cc:2955] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045 [term 24 FOLLOWER]: Committing config change with OpId 24.18: config changed from index 15 to 18, VOTER 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130) evicted. New config: { opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
W20260709 13:22:48.263808 16790 consensus_peers.cc:597] T 23f82224f4564fea93572e8206bad1fe P cb82cc7bd6db46d980c75237a1402860 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:22:48.264401 16790 leader_election.cc:336] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 29 election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111)
I20260709 13:22:48.266819 16044 catalog_manager.cc:5697] T 7c2095a5f4ad49408dbc7f69c606120d P 90f3da57796b488c9087a2f523adf045 reported cstate change: config changed from index 10 to 11, cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) changed from NON_VOTER to VOTER. New cstate: current_term: 19 leader_uuid: "90f3da57796b488c9087a2f523adf045" committed_config { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:22:48.279731 16790 leader_election.cc:304] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 29 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:22:48.298269 24658 consensus_queue.cc:1048] T 1df93779209748b9860f6d893dd3de53 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
W20260709 13:22:48.304996 16265 consensus_peers.cc:597] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 90f3da57796b488c9087a2f523adf045 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 13:22:48.309829 16817 raft_consensus.cc:1275] T 18ad7e3af5504571a69d10c00750a01c P cb82cc7bd6db46d980c75237a1402860 [term 19 LEARNER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 19 index: 16. (index mismatch)
I20260709 13:22:48.311069 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6f07842d3aa47858df5665874c86f3a" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 26 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:22:48.311784 16224 raft_consensus.cc:3060] T b6f07842d3aa47858df5665874c86f3a P 1ca2725d290e47d68b1a4b609c9816ac [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:22:48.316385 16036 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 806d7690c2b644959152074bb2cab0a7 with cas_config_opid_index 15: ChangeConfig:REMOVE_PEER succeeded (attempt 9)
W20260709 13:22:48.318113 16191 consensus_peers.cc:597] T 18ad7e3af5504571a69d10c00750a01c P 1ca2725d290e47d68b1a4b609c9816ac -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:22:48.311105 24737 raft_consensus.cc:2749] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [term 29 FOLLOWER]: Leader election lost for term 29. Reason: could not achieve majority
I20260709 13:22:48.325285 24667 consensus_queue.cc:1048] T 18ad7e3af5504571a69d10c00750a01c P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
W20260709 13:22:48.332476 16265 leader_election.cc:336] T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 26 election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111)
W20260709 13:22:48.326758 16790 consensus_peers.cc:597] T 66f75124215f4ca6bef831e174a792d8 P cb82cc7bd6db46d980c75237a1402860 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 13:22:48.342723 16224 raft_consensus.cc:2468] T b6f07842d3aa47858df5665874c86f3a P 1ca2725d290e47d68b1a4b609c9816ac [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 26.
I20260709 13:22:48.344501 16299 raft_consensus.cc:1275] T 18ad7e3af5504571a69d10c00750a01c P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 19 index: 16. (index mismatch)
W20260709 13:22:48.335489 16790 consensus_peers.cc:597] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:22:48.357161 16191 consensus_peers.cc:597] T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:22:48.359838 16790 consensus_peers.cc:597] T 806d7690c2b644959152074bb2cab0a7 P cb82cc7bd6db46d980c75237a1402860 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:22:48.363829 16191 consensus_peers.cc:597] T 829b4ad8c5a54eababeea1181b960d4b P 1ca2725d290e47d68b1a4b609c9816ac -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 13:22:48.365612 24667 consensus_queue.cc:1048] T 18ad7e3af5504571a69d10c00750a01c P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 13:22:48.370381 16265 leader_election.cc:304] T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: 647a343a542d4a139fe106c19095129b
I20260709 13:22:48.374137 24745 raft_consensus.cc:2804] T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045 [term 26 FOLLOWER]: Leader election won for term 26
I20260709 13:22:48.380553 24745 raft_consensus.cc:697] T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045 [term 26 LEADER]: Becoming Leader. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:22:48.381718 24745 consensus_queue.cc:237] T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 24.15, Last appended by leader: 15, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:22:48.383522 16191 consensus_peers.cc:597] T c0eab25ce4c64e018a041beeda2867db P 1ca2725d290e47d68b1a4b609c9816ac -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:22:48.406562 16790 consensus_peers.cc:597] T 57a744f5aaf944678a63469cbf514015 P cb82cc7bd6db46d980c75237a1402860 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 13:22:48.425348 24667 raft_consensus.cc:2955] T 18ad7e3af5504571a69d10c00750a01c P 1ca2725d290e47d68b1a4b609c9816ac [term 19 LEADER]: Committing config change with OpId 19.16: config changed from index 15 to 16, cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) changed from NON_VOTER to VOTER. New config: { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:22:48.429821 16299 raft_consensus.cc:2955] T 18ad7e3af5504571a69d10c00750a01c P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Committing config change with OpId 19.16: config changed from index 15 to 16, cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) changed from NON_VOTER to VOTER. New config: { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
W20260709 13:22:48.436440 16265 consensus_peers.cc:597] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 90f3da57796b488c9087a2f523adf045 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:22:48.462430 16191 consensus_peers.cc:597] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 13:22:48.493201 16044 catalog_manager.cc:5697] T 806d7690c2b644959152074bb2cab0a7 P cb82cc7bd6db46d980c75237a1402860 reported cstate change: config changed from index 15 to 18, VOTER 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130) evicted. New cstate: current_term: 24 leader_uuid: "cb82cc7bd6db46d980c75237a1402860" committed_config { opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:22:48.516441 16299 consensus_queue.cc:237] T 7c2095a5f4ad49408dbc7f69c606120d P 90f3da57796b488c9087a2f523adf045 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 8, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:48.515815 16040 catalog_manager.cc:5697] T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045 reported cstate change: term changed from 24 to 26, leader changed from 647a343a542d4a139fe106c19095129b (127.14.177.129) to 90f3da57796b488c9087a2f523adf045 (127.14.177.131). New cstate: current_term: 26 leader_uuid: "90f3da57796b488c9087a2f523adf045" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } health_report { overall_health: HEALTHY } } }
I20260709 13:22:48.536056 16040 catalog_manager.cc:5697] T 18ad7e3af5504571a69d10c00750a01c P 90f3da57796b488c9087a2f523adf045 reported cstate change: config changed from index 15 to 16, cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) changed from NON_VOTER to VOTER. New cstate: current_term: 19 leader_uuid: "1ca2725d290e47d68b1a4b609c9816ac" committed_config { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:22:48.530913 16821 raft_consensus.cc:2955] T 18ad7e3af5504571a69d10c00750a01c P cb82cc7bd6db46d980c75237a1402860 [term 19 FOLLOWER]: Committing config change with OpId 19.16: config changed from index 15 to 16, cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) changed from NON_VOTER to VOTER. New config: { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
W20260709 13:22:48.534397 16191 consensus_peers.cc:597] T e5ad4eb281564936a59f41e387a47156 P 1ca2725d290e47d68b1a4b609c9816ac -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 13:22:48.554294 16817 raft_consensus.cc:1275] T 7c2095a5f4ad49408dbc7f69c606120d P cb82cc7bd6db46d980c75237a1402860 [term 19 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 13:22:48.571125 16209 tablet_service.cc:1558] Processing DeleteTablet for tablet 806d7690c2b644959152074bb2cab0a7 with delete_type TABLET_DATA_TOMBSTONED (TS 1ca2725d290e47d68b1a4b609c9816ac not found in new config with opid_index 18) from {username='slave'} at 127.0.0.1:39558
I20260709 13:22:48.574498 24762 consensus_queue.cc:1048] T 7c2095a5f4ad49408dbc7f69c606120d P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:22:48.581058 16220 raft_consensus.cc:1275] T 7c2095a5f4ad49408dbc7f69c606120d P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 13:22:48.585745 24766 consensus_queue.cc:1048] T 7c2095a5f4ad49408dbc7f69c606120d P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
W20260709 13:22:48.596998 16265 consensus_peers.cc:597] T 8cc959fb7f704fba9009c0595467d074 P 90f3da57796b488c9087a2f523adf045 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 13:22:48.602281 24762 raft_consensus.cc:2955] T 7c2095a5f4ad49408dbc7f69c606120d P 90f3da57796b488c9087a2f523adf045 [term 19 LEADER]: Committing config change with OpId 19.12: config changed from index 11 to 12, VOTER 647a343a542d4a139fe106c19095129b (127.14.177.129) evicted. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:22:48.607882 16220 raft_consensus.cc:2955] T 7c2095a5f4ad49408dbc7f69c606120d P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Committing config change with OpId 19.12: config changed from index 11 to 12, VOTER 647a343a542d4a139fe106c19095129b (127.14.177.129) evicted. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:22:48.617759 16822 raft_consensus.cc:2955] T 7c2095a5f4ad49408dbc7f69c606120d P cb82cc7bd6db46d980c75237a1402860 [term 19 FOLLOWER]: Committing config change with OpId 19.12: config changed from index 11 to 12, VOTER 647a343a542d4a139fe106c19095129b (127.14.177.129) evicted. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
W20260709 13:22:48.640053 16191 consensus_peers.cc:597] T 746aabf856ab401c9ef5e859371e7b67 P 1ca2725d290e47d68b1a4b609c9816ac -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:22:48.666115 16040 catalog_manager.cc:5697] T 7c2095a5f4ad49408dbc7f69c606120d P cb82cc7bd6db46d980c75237a1402860 reported cstate change: config changed from index 11 to 12, VOTER 647a343a542d4a139fe106c19095129b (127.14.177.129) evicted. New cstate: current_term: 19 leader_uuid: "90f3da57796b488c9087a2f523adf045" committed_config { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:22:48.691951 16220 consensus_queue.cc:237] T 18ad7e3af5504571a69d10c00750a01c P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 19.16, Last appended by leader: 13, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:48.708144 16821 raft_consensus.cc:1275] T 18ad7e3af5504571a69d10c00750a01c P cb82cc7bd6db46d980c75237a1402860 [term 19 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 19 index: 16. Preceding OpId from leader: term: 19 index: 17. (index mismatch)
I20260709 13:22:48.710886 24667 consensus_queue.cc:1048] T 18ad7e3af5504571a69d10c00750a01c P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
W20260709 13:22:48.711814 16265 consensus_peers.cc:597] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:22:48.731500 24782 tablet_replica.cc:333] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac: stopping tablet replica
I20260709 13:22:48.732939 24782 raft_consensus.cc:2243] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:48.733947 24782 raft_consensus.cc:2272] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:48.749233 24764 raft_consensus.cc:2955] T 18ad7e3af5504571a69d10c00750a01c P 1ca2725d290e47d68b1a4b609c9816ac [term 19 LEADER]: Committing config change with OpId 19.17: config changed from index 16 to 17, VOTER 647a343a542d4a139fe106c19095129b (127.14.177.129) evicted. New config: { opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:22:48.758154 16295 raft_consensus.cc:1275] T 18ad7e3af5504571a69d10c00750a01c P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 19 index: 16. Preceding OpId from leader: term: 19 index: 17. (index mismatch)
I20260709 13:22:48.764741 16821 raft_consensus.cc:2955] T 18ad7e3af5504571a69d10c00750a01c P cb82cc7bd6db46d980c75237a1402860 [term 19 FOLLOWER]: Committing config change with OpId 19.17: config changed from index 16 to 17, VOTER 647a343a542d4a139fe106c19095129b (127.14.177.129) evicted. New config: { opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
W20260709 13:22:48.778822 16035 catalog_manager.cc:4755] TS 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet 7c2095a5f4ad49408dbc7f69c606120d: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111)
I20260709 13:22:48.782737 16034 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 7c2095a5f4ad49408dbc7f69c606120d with cas_config_opid_index 11: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:22:48.783434 24740 consensus_queue.cc:1048] T 18ad7e3af5504571a69d10c00750a01c P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 13:22:48.786756 24782 ts_tablet_manager.cc:1916] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:22:48.788188 16293 raft_consensus.cc:2955] T 18ad7e3af5504571a69d10c00750a01c P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Committing config change with OpId 19.17: config changed from index 16 to 17, VOTER 647a343a542d4a139fe106c19095129b (127.14.177.129) evicted. New config: { opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:22:48.815449 16035 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 18ad7e3af5504571a69d10c00750a01c with cas_config_opid_index 16: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:22:48.810770 16040 catalog_manager.cc:5697] T 18ad7e3af5504571a69d10c00750a01c P cb82cc7bd6db46d980c75237a1402860 reported cstate change: config changed from index 16 to 17, VOTER 647a343a542d4a139fe106c19095129b (127.14.177.129) evicted. New cstate: current_term: 19 leader_uuid: "1ca2725d290e47d68b1a4b609c9816ac" committed_config { opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
W20260709 13:22:48.884033 15046 thread.cc:527] Waited for 4000ms trying to join with diag-logger (tid 16172)
I20260709 13:22:48.899586 16219 raft_consensus.cc:1275] T b6f07842d3aa47858df5665874c86f3a P 1ca2725d290e47d68b1a4b609c9816ac [term 26 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 24 index: 15. Preceding OpId from leader: term: 26 index: 16. (index mismatch)
I20260709 13:22:48.908581 24762 consensus_queue.cc:1048] T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 13:22:48.942749 24782 ts_tablet_manager.cc:1929] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 24.17
I20260709 13:22:48.943459 24782 log.cc:1199] T 806d7690c2b644959152074bb2cab0a7 P 1ca2725d290e47d68b1a4b609c9816ac: Deleting WAL directory at /tmp/dist-test-taskdH_Ma7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783603093801613-15046-0/minicluster-data/ts-1-root/wals/806d7690c2b644959152074bb2cab0a7
I20260709 13:22:48.956151 16035 catalog_manager.cc:5028] TS 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943): tablet 806d7690c2b644959152074bb2cab0a7 (table test-workload [id=ed3d15d8c9ad46bebc4fe3a402d2e54f]) successfully deleted
W20260709 13:22:48.964037 16265 consensus_peers.cc:597] T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:22:49.008831 16191 consensus_peers.cc:597] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:22:49.043551 16191 consensus_peers.cc:597] T 5a959dbecc2648eb85d1cc550f5a3245 P 1ca2725d290e47d68b1a4b609c9816ac -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:22:49.091981 16790 consensus_peers.cc:597] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:22:49.093294 16265 consensus_peers.cc:597] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:22:49.162132 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 43596.8us
W20260709 13:22:49.186691 16191 consensus_peers.cc:597] T af57996a8be84d398f0f023e8d08ef77 P 1ca2725d290e47d68b1a4b609c9816ac -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:22:49.193818 16191 consensus_peers.cc:597] T 84f2da98864b4425bf7c0055951fdd4b P 1ca2725d290e47d68b1a4b609c9816ac -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:22:49.190290 16265 consensus_peers.cc:597] T e4dc6b0541ca431ebc757beac00521f1 P 90f3da57796b488c9087a2f523adf045 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 13:22:49.205349 16038 maintenance_manager.cc:419] P 667a34b27a39474ebefc1af3cf5b4271: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=0.033336
W20260709 13:22:49.255385 16790 consensus_peers.cc:597] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:22:49.268824 16191 consensus_peers.cc:597] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:22:49.286549 16191 consensus_peers.cc:597] T ee9ed67006144797b1de6ea8c2013a68 P 1ca2725d290e47d68b1a4b609c9816ac -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:22:49.320638 16191 consensus_peers.cc:597] T 5a96373bb0e14dce8a266b2fc6c58507 P 1ca2725d290e47d68b1a4b609c9816ac -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:22:49.322232 16265 consensus_peers.cc:597] T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:22:49.326782 16037 maintenance_manager.cc:643] P 667a34b27a39474ebefc1af3cf5b4271: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.109s	user 0.041s	sys 0.003s Metrics: {"bytes_written":7797,"delete_count":0,"lbm_write_time_us":1190,"lbm_writes_lt_1ms":7,"reinsert_count":0,"update_count":312}
I20260709 13:22:49.329212 16038 maintenance_manager.cc:419] P 667a34b27a39474ebefc1af3cf5b4271: Scheduling MajorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.332438
W20260709 13:22:49.511705 16265 consensus_peers.cc:597] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:22:49.526672 16790 consensus_peers.cc:597] T 9e65676b21b748728ad39668cfe722ec P cb82cc7bd6db46d980c75237a1402860 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:22:49.534425 16191 consensus_peers.cc:597] T d16dfad0c3b540e5b4b86492ddf87208 P 1ca2725d290e47d68b1a4b609c9816ac -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 13:22:49.627712 16037 maintenance_manager.cc:643] P 667a34b27a39474ebefc1af3cf5b4271: MajorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.298s	user 0.103s	sys 0.012s Metrics: {"cfile_cache_miss":8,"cfile_cache_miss_bytes":28693,"cfile_init":4,"delete_count":0,"delta_blocks_compacted":1,"delta_iterators_relevant":1,"dirs.queue_time_us":13051,"dirs.run_cpu_time_us":293,"dirs.run_wall_time_us":294,"lbm_read_time_us":3713,"lbm_reads_1-10_ms":1,"lbm_reads_lt_1ms":23,"lbm_write_time_us":1781,"lbm_writes_lt_1ms":12,"peak_mem_usage":313428,"reinsert_count":0,"thread_start_us":12492,"threads_started":1,"update_count":312}
W20260709 13:22:49.700513 16191 consensus_peers.cc:597] T 68a6838d82ef4cac83507f906139d90f P 1ca2725d290e47d68b1a4b609c9816ac -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:22:49.705583 16265 consensus_peers.cc:597] T b87147731a68400e8cce9b879daa7c5d P 90f3da57796b488c9087a2f523adf045 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:22:49.714975 16265 consensus_peers.cc:597] T 8a878999c0024d2abe517484180290fc P 90f3da57796b488c9087a2f523adf045 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:22:49.807996 16265 consensus_peers.cc:597] T 02838c86531d4f98bf58d1c22b8baf77 P 90f3da57796b488c9087a2f523adf045 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:22:49.884589 15046 thread.cc:527] Waited for 5000ms trying to join with diag-logger (tid 16172)
W20260709 13:22:49.980718 16035 catalog_manager.cc:4755] TS 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet 7c2095a5f4ad49408dbc7f69c606120d: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111)
W20260709 13:22:50.049186 16265 consensus_peers.cc:597] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:22:50.071318 16790 consensus_peers.cc:597] T 2794af5b99984efebdee564035841cb0 P cb82cc7bd6db46d980c75237a1402860 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
I20260709 13:22:50.079597 16219 consensus_queue.cc:237] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 23.15, Last appended by leader: 14, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } }
W20260709 13:22:50.127179 16790 consensus_peers.cc:597] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 13:22:50.146021 16293 raft_consensus.cc:1275] T c4f65cdf00b748bb963ee6e81ef43b79 P 90f3da57796b488c9087a2f523adf045 [term 23 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 23 index: 15. Preceding OpId from leader: term: 23 index: 16. (index mismatch)
W20260709 13:22:50.146773 16192 consensus_peers.cc:597] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac -> Peer cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923): Couldn't send request to peer cb82cc7bd6db46d980c75237a1402860. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: c4f65cdf00b748bb963ee6e81ef43b79. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:22:50.148109 24763 consensus_queue.cc:1048] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
W20260709 13:22:50.152284 16265 consensus_peers.cc:597] T 434bff3082d141838068f323bdb0d1ed P 90f3da57796b488c9087a2f523adf045 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:22:50.171072 16191 consensus_peers.cc:597] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:22:50.212484 24666 raft_consensus.cc:2955] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac [term 23 LEADER]: Committing config change with OpId 23.16: config changed from index -1 to 16, NON_VOTER cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) added. New config: { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } } }
I20260709 13:22:50.215888 16293 raft_consensus.cc:2955] T c4f65cdf00b748bb963ee6e81ef43b79 P 90f3da57796b488c9087a2f523adf045 [term 23 FOLLOWER]: Committing config change with OpId 23.16: config changed from index -1 to 16, NON_VOTER cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) added. New config: { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } } }
I20260709 13:22:50.284189 16219 consensus_queue.cc:237] T 5a959dbecc2648eb85d1cc550f5a3245 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 24.15, Last appended by leader: 14, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } }
I20260709 13:22:50.285432 16045 catalog_manager.cc:5697] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac reported cstate change: config changed from index -1 to 16, NON_VOTER cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) added. New cstate: current_term: 23 leader_uuid: "1ca2725d290e47d68b1a4b609c9816ac" committed_config { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:50.317185 16293 raft_consensus.cc:1275] T 5a959dbecc2648eb85d1cc550f5a3245 P 90f3da57796b488c9087a2f523adf045 [term 24 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 24 index: 15. Preceding OpId from leader: term: 24 index: 16. (index mismatch)
I20260709 13:22:50.320623 24763 consensus_queue.cc:1048] T 5a959dbecc2648eb85d1cc550f5a3245 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
W20260709 13:22:50.326973 16191 consensus_peers.cc:597] T 5a959dbecc2648eb85d1cc550f5a3245 P 1ca2725d290e47d68b1a4b609c9816ac -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:22:50.341183 24667 raft_consensus.cc:2955] T 5a959dbecc2648eb85d1cc550f5a3245 P 1ca2725d290e47d68b1a4b609c9816ac [term 24 LEADER]: Committing config change with OpId 24.16: config changed from index -1 to 16, NON_VOTER cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) added. New config: { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } } }
I20260709 13:22:50.351851 16300 raft_consensus.cc:2955] T 5a959dbecc2648eb85d1cc550f5a3245 P 90f3da57796b488c9087a2f523adf045 [term 24 FOLLOWER]: Committing config change with OpId 24.16: config changed from index -1 to 16, NON_VOTER cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) added. New config: { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } } }
I20260709 13:22:50.365557 16293 consensus_queue.cc:237] T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 26.16, Last appended by leader: 15, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } }
I20260709 13:22:50.401631 16040 catalog_manager.cc:5697] T 5a959dbecc2648eb85d1cc550f5a3245 P 90f3da57796b488c9087a2f523adf045 reported cstate change: config changed from index -1 to 16, NON_VOTER cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) added. New cstate: current_term: 24 leader_uuid: "1ca2725d290e47d68b1a4b609c9816ac" committed_config { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } } }
W20260709 13:22:50.435650 16790 consensus_peers.cc:597] T 3884f864a96d428785895e5463e5728f P cb82cc7bd6db46d980c75237a1402860 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:22:50.470008 16192 consensus_peers.cc:597] T 5a959dbecc2648eb85d1cc550f5a3245 P 1ca2725d290e47d68b1a4b609c9816ac -> Peer cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923): Couldn't send request to peer cb82cc7bd6db46d980c75237a1402860. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 5a959dbecc2648eb85d1cc550f5a3245. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:22:50.477244 16266 consensus_peers.cc:597] T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045 -> Peer cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923): Couldn't send request to peer cb82cc7bd6db46d980c75237a1402860. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b6f07842d3aa47858df5665874c86f3a. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:22:50.480379 16790 consensus_peers.cc:597] T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
I20260709 13:22:50.496050 16219 raft_consensus.cc:1275] T b6f07842d3aa47858df5665874c86f3a P 1ca2725d290e47d68b1a4b609c9816ac [term 26 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 26 index: 16. Preceding OpId from leader: term: 26 index: 17. (index mismatch)
W20260709 13:22:50.500155 16790 consensus_peers.cc:597] T 23f82224f4564fea93572e8206bad1fe P cb82cc7bd6db46d980c75237a1402860 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:22:50.502982 16191 consensus_peers.cc:597] T 829b4ad8c5a54eababeea1181b960d4b P 1ca2725d290e47d68b1a4b609c9816ac -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 13:22:50.512907 24788 consensus_queue.cc:1048] T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.002s
W20260709 13:22:50.523047 16265 consensus_peers.cc:597] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 90f3da57796b488c9087a2f523adf045 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:22:50.529461 16265 consensus_peers.cc:597] T 26354361e9b044cea1c2b86c4882fe92 P 90f3da57796b488c9087a2f523adf045 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:22:50.536991 16265 consensus_peers.cc:597] T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:22:50.542915 16265 consensus_peers.cc:597] T 1df93779209748b9860f6d893dd3de53 P 90f3da57796b488c9087a2f523adf045 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:22:50.547789 24767 raft_consensus.cc:2955] T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045 [term 26 LEADER]: Committing config change with OpId 26.17: config changed from index -1 to 17, NON_VOTER cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) added. New config: { opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } } }
I20260709 13:22:50.572923 16220 raft_consensus.cc:2955] T b6f07842d3aa47858df5665874c86f3a P 1ca2725d290e47d68b1a4b609c9816ac [term 26 FOLLOWER]: Committing config change with OpId 26.17: config changed from index -1 to 17, NON_VOTER cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) added. New config: { opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } } }
I20260709 13:22:50.570849 16219 consensus_queue.cc:237] T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 23.14, Last appended by leader: 13, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } }
I20260709 13:22:50.597615 16040 catalog_manager.cc:5697] T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045 reported cstate change: config changed from index -1 to 17, NON_VOTER cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) added. New cstate: current_term: 26 leader_uuid: "90f3da57796b488c9087a2f523adf045" committed_config { opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
W20260709 13:22:50.613242 16192 consensus_peers.cc:597] T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac -> Peer cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923): Couldn't send request to peer cb82cc7bd6db46d980c75237a1402860. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 8d4ab870a05347868956aa8b841b2981. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:22:50.618834 16790 consensus_peers.cc:597] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 13:22:50.619207 16293 raft_consensus.cc:1275] T 8d4ab870a05347868956aa8b841b2981 P 90f3da57796b488c9087a2f523adf045 [term 23 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 23 index: 14. Preceding OpId from leader: term: 23 index: 15. (index mismatch)
I20260709 13:22:50.624397 24763 consensus_queue.cc:1048] T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
W20260709 13:22:50.625293 16191 consensus_peers.cc:597] T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:22:50.625988 16790 consensus_peers.cc:597] T 66f75124215f4ca6bef831e174a792d8 P cb82cc7bd6db46d980c75237a1402860 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:22:50.685732 16265 consensus_peers.cc:597] T fffab26427e24d398e2695aad40ca134 P 90f3da57796b488c9087a2f523adf045 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:22:50.695604 16790 consensus_peers.cc:597] T 806d7690c2b644959152074bb2cab0a7 P cb82cc7bd6db46d980c75237a1402860 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:22:50.656253 24803 log.cc:927] Time spent T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac: Append to log took a long time: real 0.050s	user 0.001s	sys 0.000s
I20260709 13:22:50.705652 24798 raft_consensus.cc:2955] T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac [term 23 LEADER]: Committing config change with OpId 23.15: config changed from index -1 to 15, NON_VOTER cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) added. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } } }
I20260709 13:22:50.709312 16295 raft_consensus.cc:2955] T 8d4ab870a05347868956aa8b841b2981 P 90f3da57796b488c9087a2f523adf045 [term 23 FOLLOWER]: Committing config change with OpId 23.15: config changed from index -1 to 15, NON_VOTER cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) added. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } } }
I20260709 13:22:50.727156 16040 catalog_manager.cc:5697] T 8d4ab870a05347868956aa8b841b2981 P 90f3da57796b488c9087a2f523adf045 reported cstate change: config changed from index -1 to 15, NON_VOTER cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) added. New cstate: current_term: 23 leader_uuid: "1ca2725d290e47d68b1a4b609c9816ac" committed_config { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } } }
I20260709 13:22:50.743224 24805 ts_tablet_manager.cc:933] T c4f65cdf00b748bb963ee6e81ef43b79 P cb82cc7bd6db46d980c75237a1402860: Initiating tablet copy from peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:22:50.770761 24805 tablet_copy_client.cc:323] T c4f65cdf00b748bb963ee6e81ef43b79 P cb82cc7bd6db46d980c75237a1402860: tablet copy: Beginning tablet copy session from remote peer at address 127.14.177.130:35943
I20260709 13:22:50.773315 16236 tablet_copy_service.cc:140] P 1ca2725d290e47d68b1a4b609c9816ac: Received BeginTabletCopySession request for tablet c4f65cdf00b748bb963ee6e81ef43b79 from peer cb82cc7bd6db46d980c75237a1402860 ({username='slave'} at 127.0.0.1:33050)
I20260709 13:22:50.773970 16236 tablet_copy_service.cc:161] P 1ca2725d290e47d68b1a4b609c9816ac: Beginning new tablet copy session on tablet c4f65cdf00b748bb963ee6e81ef43b79 from peer cb82cc7bd6db46d980c75237a1402860 at {username='slave'} at 127.0.0.1:33050: session id = cb82cc7bd6db46d980c75237a1402860-c4f65cdf00b748bb963ee6e81ef43b79
I20260709 13:22:50.811939 16236 tablet_copy_source_session.cc:215] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:22:50.877619 24805 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c4f65cdf00b748bb963ee6e81ef43b79. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:22:50.885066 15046 thread.cc:527] Waited for 6000ms trying to join with diag-logger (tid 16172)
I20260709 13:22:50.945469 24807 ts_tablet_manager.cc:933] T 5a959dbecc2648eb85d1cc550f5a3245 P cb82cc7bd6db46d980c75237a1402860: Initiating tablet copy from peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:22:50.960338 24807 tablet_copy_client.cc:323] T 5a959dbecc2648eb85d1cc550f5a3245 P cb82cc7bd6db46d980c75237a1402860: tablet copy: Beginning tablet copy session from remote peer at address 127.14.177.130:35943
I20260709 13:22:50.963142 16236 tablet_copy_service.cc:140] P 1ca2725d290e47d68b1a4b609c9816ac: Received BeginTabletCopySession request for tablet 5a959dbecc2648eb85d1cc550f5a3245 from peer cb82cc7bd6db46d980c75237a1402860 ({username='slave'} at 127.0.0.1:33050)
I20260709 13:22:50.964733 16236 tablet_copy_service.cc:161] P 1ca2725d290e47d68b1a4b609c9816ac: Beginning new tablet copy session on tablet 5a959dbecc2648eb85d1cc550f5a3245 from peer cb82cc7bd6db46d980c75237a1402860 at {username='slave'} at 127.0.0.1:33050: session id = cb82cc7bd6db46d980c75237a1402860-5a959dbecc2648eb85d1cc550f5a3245
W20260709 13:22:50.989713 16265 consensus_peers.cc:597] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:22:51.022784 16236 tablet_copy_source_session.cc:215] T 5a959dbecc2648eb85d1cc550f5a3245 P 1ca2725d290e47d68b1a4b609c9816ac: Tablet Copy: opened 0 blocks and 1 log segments
W20260709 13:22:51.040269 16191 consensus_peers.cc:597] T c0eab25ce4c64e018a041beeda2867db P 1ca2725d290e47d68b1a4b609c9816ac -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 13:22:51.043323 24809 ts_tablet_manager.cc:933] T 8d4ab870a05347868956aa8b841b2981 P cb82cc7bd6db46d980c75237a1402860: Initiating tablet copy from peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
W20260709 13:22:51.058892 16191 consensus_peers.cc:597] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 13:22:51.081909 24807 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5a959dbecc2648eb85d1cc550f5a3245. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:51.103052 24809 tablet_copy_client.cc:323] T 8d4ab870a05347868956aa8b841b2981 P cb82cc7bd6db46d980c75237a1402860: tablet copy: Beginning tablet copy session from remote peer at address 127.14.177.130:35943
I20260709 13:22:51.108019 16236 tablet_copy_service.cc:140] P 1ca2725d290e47d68b1a4b609c9816ac: Received BeginTabletCopySession request for tablet 8d4ab870a05347868956aa8b841b2981 from peer cb82cc7bd6db46d980c75237a1402860 ({username='slave'} at 127.0.0.1:33050)
I20260709 13:22:51.108752 16236 tablet_copy_service.cc:161] P 1ca2725d290e47d68b1a4b609c9816ac: Beginning new tablet copy session on tablet 8d4ab870a05347868956aa8b841b2981 from peer cb82cc7bd6db46d980c75237a1402860 at {username='slave'} at 127.0.0.1:33050: session id = cb82cc7bd6db46d980c75237a1402860-8d4ab870a05347868956aa8b841b2981
W20260709 13:22:51.097776 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 48871.9us
W20260709 13:22:51.124353 16790 consensus_peers.cc:597] T 57a744f5aaf944678a63469cbf514015 P cb82cc7bd6db46d980c75237a1402860 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:22:51.124938 16265 consensus_peers.cc:597] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 90f3da57796b488c9087a2f523adf045 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:22:51.137668 24805 tablet_copy_client.cc:806] T c4f65cdf00b748bb963ee6e81ef43b79 P cb82cc7bd6db46d980c75237a1402860: tablet copy: Starting download of 0 data blocks...
I20260709 13:22:51.138396 24805 tablet_copy_client.cc:670] T c4f65cdf00b748bb963ee6e81ef43b79 P cb82cc7bd6db46d980c75237a1402860: tablet copy: Starting download of 1 WAL segments...
I20260709 13:22:51.148278 16236 tablet_copy_source_session.cc:215] T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac: Tablet Copy: opened 0 blocks and 1 log segments
W20260709 13:22:51.181878 16191 consensus_peers.cc:597] T 746aabf856ab401c9ef5e859371e7b67 P 1ca2725d290e47d68b1a4b609c9816ac -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:22:51.183151 16035 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 61893.5us
I20260709 13:22:51.187489 24814 ts_tablet_manager.cc:933] T b6f07842d3aa47858df5665874c86f3a P cb82cc7bd6db46d980c75237a1402860: Initiating tablet copy from peer 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:22:51.188176 24805 tablet_copy_client.cc:538] T c4f65cdf00b748bb963ee6e81ef43b79 P cb82cc7bd6db46d980c75237a1402860: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:22:51.196766 24809 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8d4ab870a05347868956aa8b841b2981. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:51.227463 24807 tablet_copy_client.cc:806] T 5a959dbecc2648eb85d1cc550f5a3245 P cb82cc7bd6db46d980c75237a1402860: tablet copy: Starting download of 0 data blocks...
I20260709 13:22:51.228216 24807 tablet_copy_client.cc:670] T 5a959dbecc2648eb85d1cc550f5a3245 P cb82cc7bd6db46d980c75237a1402860: tablet copy: Starting download of 1 WAL segments...
I20260709 13:22:51.235294 24807 tablet_copy_client.cc:538] T 5a959dbecc2648eb85d1cc550f5a3245 P cb82cc7bd6db46d980c75237a1402860: tablet copy: Tablet Copy complete. Replacing tablet superblock.
W20260709 13:22:51.231532 16035 catalog_manager.cc:4755] TS 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet 7c2095a5f4ad49408dbc7f69c606120d: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111)
I20260709 13:22:51.237547 24805 tablet_bootstrap.cc:492] T c4f65cdf00b748bb963ee6e81ef43b79 P cb82cc7bd6db46d980c75237a1402860: Bootstrap starting.
I20260709 13:22:51.254973 24798 raft_consensus.cc:493] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:51.255633 24798 raft_consensus.cc:515] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:51.258108 24798 leader_election.cc:290] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:51.258602 24814 tablet_copy_client.cc:323] T b6f07842d3aa47858df5665874c86f3a P cb82cc7bd6db46d980c75237a1402860: tablet copy: Beginning tablet copy session from remote peer at address 127.14.177.131:41389
I20260709 13:22:51.259950 16817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62fe3346cd3c42c4a55e14af9073bca4" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 30 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:22:51.260730 16817 raft_consensus.cc:2468] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 29.
I20260709 13:22:51.267568 16192 leader_election.cc:304] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:22:51.268610 24798 raft_consensus.cc:2804] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [term 29 FOLLOWER]: Leader pre-election won for term 30
I20260709 13:22:51.269069 24798 raft_consensus.cc:493] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [term 29 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:51.269472 24798 raft_consensus.cc:3060] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [term 29 FOLLOWER]: Advancing to term 30
I20260709 13:22:51.277348 16310 tablet_copy_service.cc:140] P 90f3da57796b488c9087a2f523adf045: Received BeginTabletCopySession request for tablet b6f07842d3aa47858df5665874c86f3a from peer cb82cc7bd6db46d980c75237a1402860 ({username='slave'} at 127.0.0.1:40474)
I20260709 13:22:51.278055 16310 tablet_copy_service.cc:161] P 90f3da57796b488c9087a2f523adf045: Beginning new tablet copy session on tablet b6f07842d3aa47858df5665874c86f3a from peer cb82cc7bd6db46d980c75237a1402860 at {username='slave'} at 127.0.0.1:40474: session id = cb82cc7bd6db46d980c75237a1402860-b6f07842d3aa47858df5665874c86f3a
W20260709 13:22:51.297119 16790 consensus_peers.cc:597] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:22:51.322682 16191 leader_election.cc:336] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 30 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111)
I20260709 13:22:51.339274 24809 tablet_copy_client.cc:806] T 8d4ab870a05347868956aa8b841b2981 P cb82cc7bd6db46d980c75237a1402860: tablet copy: Starting download of 0 data blocks...
I20260709 13:22:51.340029 24809 tablet_copy_client.cc:670] T 8d4ab870a05347868956aa8b841b2981 P cb82cc7bd6db46d980c75237a1402860: tablet copy: Starting download of 1 WAL segments...
I20260709 13:22:51.334404 16310 tablet_copy_source_session.cc:215] T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:22:51.327814 24807 tablet_bootstrap.cc:492] T 5a959dbecc2648eb85d1cc550f5a3245 P cb82cc7bd6db46d980c75237a1402860: Bootstrap starting.
W20260709 13:22:51.269091 16265 consensus_peers.cc:597] T 8cc959fb7f704fba9009c0595467d074 P 90f3da57796b488c9087a2f523adf045 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:22:51.349267 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 80227.2us
W20260709 13:22:51.362289 16191 consensus_peers.cc:597] T e5ad4eb281564936a59f41e387a47156 P 1ca2725d290e47d68b1a4b609c9816ac -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 13:22:51.365366 24814 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b6f07842d3aa47858df5665874c86f3a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:51.343010 24798 raft_consensus.cc:515] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [term 30 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:51.384271 24809 tablet_copy_client.cc:538] T 8d4ab870a05347868956aa8b841b2981 P cb82cc7bd6db46d980c75237a1402860: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:22:51.401536 24798 leader_election.cc:290] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 30 election: Requested vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:22:51.403055 16817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62fe3346cd3c42c4a55e14af9073bca4" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 30 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:22:51.412250 16817 raft_consensus.cc:3060] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [term 29 FOLLOWER]: Advancing to term 30
I20260709 13:22:51.420372 16817 raft_consensus.cc:2468] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [term 30 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ca2725d290e47d68b1a4b609c9816ac in term 30.
I20260709 13:22:51.429804 16192 leader_election.cc:304] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 30 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:22:51.432395 24798 raft_consensus.cc:2804] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [term 30 FOLLOWER]: Leader election won for term 30
I20260709 13:22:51.435307 24809 tablet_bootstrap.cc:492] T 8d4ab870a05347868956aa8b841b2981 P cb82cc7bd6db46d980c75237a1402860: Bootstrap starting.
W20260709 13:22:51.435395 16191 consensus_peers.cc:597] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:22:51.436231 16191 leader_election.cc:336] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 30 election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111)
I20260709 13:22:51.435396 24798 raft_consensus.cc:697] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [term 30 LEADER]: Becoming Leader. State: Replica: 1ca2725d290e47d68b1a4b609c9816ac, State: Running, Role: LEADER
I20260709 13:22:51.441431 24798 consensus_queue.cc:237] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 27.17, Last appended by leader: 17, Current term: 30, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:51.460542 24814 tablet_copy_client.cc:806] T b6f07842d3aa47858df5665874c86f3a P cb82cc7bd6db46d980c75237a1402860: tablet copy: Starting download of 0 data blocks...
I20260709 13:22:51.461244 24814 tablet_copy_client.cc:670] T b6f07842d3aa47858df5665874c86f3a P cb82cc7bd6db46d980c75237a1402860: tablet copy: Starting download of 1 WAL segments...
W20260709 13:22:51.491205 16191 consensus_peers.cc:597] T ee9ed67006144797b1de6ea8c2013a68 P 1ca2725d290e47d68b1a4b609c9816ac -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:22:51.493108 24814 tablet_copy_client.cc:538] T b6f07842d3aa47858df5665874c86f3a P cb82cc7bd6db46d980c75237a1402860: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:22:51.604584 24814 tablet_bootstrap.cc:492] T b6f07842d3aa47858df5665874c86f3a P cb82cc7bd6db46d980c75237a1402860: Bootstrap starting.
I20260709 13:22:51.624665 16040 catalog_manager.cc:5697] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac reported cstate change: term changed from 27 to 30, leader changed from 647a343a542d4a139fe106c19095129b (127.14.177.129) to 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130). New cstate: current_term: 30 leader_uuid: "1ca2725d290e47d68b1a4b609c9816ac" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 13:22:51.640854 16265 consensus_peers.cc:597] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:22:51.647289 16191 consensus_peers.cc:597] T af57996a8be84d398f0f023e8d08ef77 P 1ca2725d290e47d68b1a4b609c9816ac -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:22:51.652801 16191 consensus_peers.cc:597] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:22:51.655642 24807 tablet_bootstrap.cc:492] T 5a959dbecc2648eb85d1cc550f5a3245 P cb82cc7bd6db46d980c75237a1402860: Bootstrap replayed 1/1 log segments. Stats: ops{read=17 overwritten=1 applied=16 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:22:51.656816 24807 tablet_bootstrap.cc:492] T 5a959dbecc2648eb85d1cc550f5a3245 P cb82cc7bd6db46d980c75237a1402860: Bootstrap complete.
I20260709 13:22:51.658078 24807 ts_tablet_manager.cc:1403] T 5a959dbecc2648eb85d1cc550f5a3245 P cb82cc7bd6db46d980c75237a1402860: Time spent bootstrapping tablet: real 0.331s	user 0.044s	sys 0.017s
I20260709 13:22:51.661664 24807 raft_consensus.cc:359] T 5a959dbecc2648eb85d1cc550f5a3245 P cb82cc7bd6db46d980c75237a1402860 [term 24 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } }
I20260709 13:22:51.662601 24807 raft_consensus.cc:740] T 5a959dbecc2648eb85d1cc550f5a3245 P cb82cc7bd6db46d980c75237a1402860 [term 24 LEARNER]: Becoming Follower/Learner. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Initialized, Role: LEARNER
I20260709 13:22:51.663379 24807 consensus_queue.cc:260] T 5a959dbecc2648eb85d1cc550f5a3245 P cb82cc7bd6db46d980c75237a1402860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 16, Last appended: 24.16, Last appended by leader: 16, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } }
I20260709 13:22:51.666182 24807 ts_tablet_manager.cc:1434] T 5a959dbecc2648eb85d1cc550f5a3245 P cb82cc7bd6db46d980c75237a1402860: Time spent starting tablet: real 0.008s	user 0.002s	sys 0.003s
I20260709 13:22:51.674924 16236 tablet_copy_service.cc:342] P 1ca2725d290e47d68b1a4b609c9816ac: Request end of tablet copy session cb82cc7bd6db46d980c75237a1402860-5a959dbecc2648eb85d1cc550f5a3245 received from {username='slave'} at 127.0.0.1:33050
I20260709 13:22:51.675499 16236 tablet_copy_service.cc:434] P 1ca2725d290e47d68b1a4b609c9816ac: ending tablet copy session cb82cc7bd6db46d980c75237a1402860-5a959dbecc2648eb85d1cc550f5a3245 on tablet 5a959dbecc2648eb85d1cc550f5a3245 with peer cb82cc7bd6db46d980c75237a1402860
W20260709 13:22:51.689141 16191 consensus_peers.cc:597] T 84f2da98864b4425bf7c0055951fdd4b P 1ca2725d290e47d68b1a4b609c9816ac -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:22:51.753291 24805 tablet_bootstrap.cc:492] T c4f65cdf00b748bb963ee6e81ef43b79 P cb82cc7bd6db46d980c75237a1402860: Bootstrap replayed 1/1 log segments. Stats: ops{read=18 overwritten=2 applied=16 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:22:51.754396 24805 tablet_bootstrap.cc:492] T c4f65cdf00b748bb963ee6e81ef43b79 P cb82cc7bd6db46d980c75237a1402860: Bootstrap complete.
I20260709 13:22:51.770859 24805 ts_tablet_manager.cc:1403] T c4f65cdf00b748bb963ee6e81ef43b79 P cb82cc7bd6db46d980c75237a1402860: Time spent bootstrapping tablet: real 0.548s	user 0.054s	sys 0.015s
I20260709 13:22:51.780208 24809 tablet_bootstrap.cc:492] T 8d4ab870a05347868956aa8b841b2981 P cb82cc7bd6db46d980c75237a1402860: Bootstrap replayed 1/1 log segments. Stats: ops{read=17 overwritten=2 applied=15 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:22:51.784649 24809 tablet_bootstrap.cc:492] T 8d4ab870a05347868956aa8b841b2981 P cb82cc7bd6db46d980c75237a1402860: Bootstrap complete.
I20260709 13:22:51.785418 24809 ts_tablet_manager.cc:1403] T 8d4ab870a05347868956aa8b841b2981 P cb82cc7bd6db46d980c75237a1402860: Time spent bootstrapping tablet: real 0.373s	user 0.051s	sys 0.013s
I20260709 13:22:51.776707 24805 raft_consensus.cc:359] T c4f65cdf00b748bb963ee6e81ef43b79 P cb82cc7bd6db46d980c75237a1402860 [term 23 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } }
I20260709 13:22:51.788533 24805 raft_consensus.cc:740] T c4f65cdf00b748bb963ee6e81ef43b79 P cb82cc7bd6db46d980c75237a1402860 [term 23 LEARNER]: Becoming Follower/Learner. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Initialized, Role: LEARNER
I20260709 13:22:51.793484 24805 consensus_queue.cc:260] T c4f65cdf00b748bb963ee6e81ef43b79 P cb82cc7bd6db46d980c75237a1402860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 16, Last appended: 23.16, Last appended by leader: 16, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } }
W20260709 13:22:51.824954 16191 consensus_peers.cc:597] T 5a96373bb0e14dce8a266b2fc6c58507 P 1ca2725d290e47d68b1a4b609c9816ac -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:22:51.850898 16191 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111) [suppressed 470 similar messages]
W20260709 13:22:51.885973 15046 thread.cc:527] Waited for 7000ms trying to join with diag-logger (tid 16172)
I20260709 13:22:51.880699 24809 raft_consensus.cc:359] T 8d4ab870a05347868956aa8b841b2981 P cb82cc7bd6db46d980c75237a1402860 [term 23 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } }
I20260709 13:22:51.896401 24809 raft_consensus.cc:740] T 8d4ab870a05347868956aa8b841b2981 P cb82cc7bd6db46d980c75237a1402860 [term 23 LEARNER]: Becoming Follower/Learner. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Initialized, Role: LEARNER
I20260709 13:22:51.917768 24805 ts_tablet_manager.cc:1434] T c4f65cdf00b748bb963ee6e81ef43b79 P cb82cc7bd6db46d980c75237a1402860: Time spent starting tablet: real 0.146s	user 0.007s	sys 0.000s
W20260709 13:22:51.919134 16790 consensus_peers.cc:597] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:22:51.928902 16236 tablet_copy_service.cc:342] P 1ca2725d290e47d68b1a4b609c9816ac: Request end of tablet copy session cb82cc7bd6db46d980c75237a1402860-c4f65cdf00b748bb963ee6e81ef43b79 received from {username='slave'} at 127.0.0.1:33050
I20260709 13:22:51.929535 16236 tablet_copy_service.cc:434] P 1ca2725d290e47d68b1a4b609c9816ac: ending tablet copy session cb82cc7bd6db46d980c75237a1402860-c4f65cdf00b748bb963ee6e81ef43b79 on tablet c4f65cdf00b748bb963ee6e81ef43b79 with peer cb82cc7bd6db46d980c75237a1402860
I20260709 13:22:51.941905 16817 raft_consensus.cc:1217] T c4f65cdf00b748bb963ee6e81ef43b79 P cb82cc7bd6db46d980c75237a1402860 [term 23 LEARNER]: Deduplicated request from leader. Original: 23.15->[23.16-23.16]   Dedup: 23.16->[]
W20260709 13:22:51.953073 16265 consensus_peers.cc:597] T e4dc6b0541ca431ebc757beac00521f1 P 90f3da57796b488c9087a2f523adf045 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:22:51.953627 16265 consensus_peers.cc:597] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:22:51.954035 16265 consensus_peers.cc:597] T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:22:51.897399 24809 consensus_queue.cc:260] T 8d4ab870a05347868956aa8b841b2981 P cb82cc7bd6db46d980c75237a1402860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 15, Last appended: 23.15, Last appended by leader: 15, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } }
I20260709 13:22:51.966166 24809 ts_tablet_manager.cc:1434] T 8d4ab870a05347868956aa8b841b2981 P cb82cc7bd6db46d980c75237a1402860: Time spent starting tablet: real 0.180s	user 0.011s	sys 0.000s
I20260709 13:22:51.968338 16236 tablet_copy_service.cc:342] P 1ca2725d290e47d68b1a4b609c9816ac: Request end of tablet copy session cb82cc7bd6db46d980c75237a1402860-8d4ab870a05347868956aa8b841b2981 received from {username='slave'} at 127.0.0.1:33050
I20260709 13:22:51.968921 16236 tablet_copy_service.cc:434] P 1ca2725d290e47d68b1a4b609c9816ac: ending tablet copy session cb82cc7bd6db46d980c75237a1402860-8d4ab870a05347868956aa8b841b2981 on tablet 8d4ab870a05347868956aa8b841b2981 with peer cb82cc7bd6db46d980c75237a1402860
I20260709 13:22:51.974885 16822 raft_consensus.cc:1275] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [term 30 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 27 index: 17. Preceding OpId from leader: term: 30 index: 18. (index mismatch)
I20260709 13:22:51.981343 24829 consensus_queue.cc:1048] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.001s
I20260709 13:22:51.987403 16817 raft_consensus.cc:1217] T 5a959dbecc2648eb85d1cc550f5a3245 P cb82cc7bd6db46d980c75237a1402860 [term 24 LEARNER]: Deduplicated request from leader. Original: 24.15->[24.16-24.16]   Dedup: 24.16->[]
W20260709 13:22:52.006271 16265 consensus_peers.cc:597] T 8a878999c0024d2abe517484180290fc P 90f3da57796b488c9087a2f523adf045 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:22:52.023603 16790 consensus_peers.cc:597] T 9e65676b21b748728ad39668cfe722ec P cb82cc7bd6db46d980c75237a1402860 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:22:52.111584 16191 consensus_peers.cc:597] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:22:52.151333 16191 consensus_peers.cc:597] T d16dfad0c3b540e5b4b86492ddf87208 P 1ca2725d290e47d68b1a4b609c9816ac -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 13:22:52.170130 16822 raft_consensus.cc:1217] T 8d4ab870a05347868956aa8b841b2981 P cb82cc7bd6db46d980c75237a1402860 [term 23 LEARNER]: Deduplicated request from leader. Original: 23.14->[23.15-23.15]   Dedup: 23.15->[]
I20260709 13:22:52.168555 24814 tablet_bootstrap.cc:492] T b6f07842d3aa47858df5665874c86f3a P cb82cc7bd6db46d980c75237a1402860: Bootstrap replayed 1/1 log segments. Stats: ops{read=17 overwritten=0 applied=17 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:22:52.180266 24814 tablet_bootstrap.cc:492] T b6f07842d3aa47858df5665874c86f3a P cb82cc7bd6db46d980c75237a1402860: Bootstrap complete.
I20260709 13:22:52.189522 24814 ts_tablet_manager.cc:1403] T b6f07842d3aa47858df5665874c86f3a P cb82cc7bd6db46d980c75237a1402860: Time spent bootstrapping tablet: real 0.585s	user 0.063s	sys 0.016s
I20260709 13:22:52.195851 24814 raft_consensus.cc:359] T b6f07842d3aa47858df5665874c86f3a P cb82cc7bd6db46d980c75237a1402860 [term 26 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } }
I20260709 13:22:52.201620 24814 raft_consensus.cc:740] T b6f07842d3aa47858df5665874c86f3a P cb82cc7bd6db46d980c75237a1402860 [term 26 LEARNER]: Becoming Follower/Learner. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Initialized, Role: LEARNER
I20260709 13:22:52.203708 24814 consensus_queue.cc:260] T b6f07842d3aa47858df5665874c86f3a P cb82cc7bd6db46d980c75237a1402860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 17, Last appended: 26.17, Last appended by leader: 17, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: NON_VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: true } }
I20260709 13:22:52.213090 24814 ts_tablet_manager.cc:1434] T b6f07842d3aa47858df5665874c86f3a P cb82cc7bd6db46d980c75237a1402860: Time spent starting tablet: real 0.020s	user 0.002s	sys 0.004s
W20260709 13:22:52.216704 16265 consensus_peers.cc:597] T 02838c86531d4f98bf58d1c22b8baf77 P 90f3da57796b488c9087a2f523adf045 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:22:52.218837 16310 tablet_copy_service.cc:342] P 90f3da57796b488c9087a2f523adf045: Request end of tablet copy session cb82cc7bd6db46d980c75237a1402860-b6f07842d3aa47858df5665874c86f3a received from {username='slave'} at 127.0.0.1:40474
I20260709 13:22:52.219431 16310 tablet_copy_service.cc:434] P 90f3da57796b488c9087a2f523adf045: ending tablet copy session cb82cc7bd6db46d980c75237a1402860-b6f07842d3aa47858df5665874c86f3a on tablet b6f07842d3aa47858df5665874c86f3a with peer cb82cc7bd6db46d980c75237a1402860
I20260709 13:22:52.233292 24835 raft_consensus.cc:1064] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac: attempting to promote NON_VOTER cb82cc7bd6db46d980c75237a1402860 to VOTER
I20260709 13:22:52.239626 24835 consensus_queue.cc:237] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 23.16, Last appended by leader: 14, Current term: 23, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:22:52.246217 16265 consensus_peers.cc:597] T b87147731a68400e8cce9b879daa7c5d P 90f3da57796b488c9087a2f523adf045 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:22:52.277769 16265 consensus_peers.cc:597] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:22:52.285707 16191 consensus_peers.cc:597] T 68a6838d82ef4cac83507f906139d90f P 1ca2725d290e47d68b1a4b609c9816ac -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 13:22:52.306504 16817 raft_consensus.cc:1275] T c4f65cdf00b748bb963ee6e81ef43b79 P cb82cc7bd6db46d980c75237a1402860 [term 23 LEARNER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 23 index: 16. Preceding OpId from leader: term: 23 index: 17. (index mismatch)
I20260709 13:22:52.307310 16295 raft_consensus.cc:1275] T c4f65cdf00b748bb963ee6e81ef43b79 P 90f3da57796b488c9087a2f523adf045 [term 23 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 23 index: 16. Preceding OpId from leader: term: 23 index: 17. (index mismatch)
I20260709 13:22:52.310276 24829 consensus_queue.cc:1048] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 13:22:52.316099 24792 consensus_queue.cc:1048] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
W20260709 13:22:52.338855 16191 consensus_peers.cc:597] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:22:52.363231 24835 raft_consensus.cc:2955] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac [term 23 LEADER]: Committing config change with OpId 23.17: config changed from index 16 to 17, cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) changed from NON_VOTER to VOTER. New config: { opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:22:52.365695 16822 raft_consensus.cc:2955] T c4f65cdf00b748bb963ee6e81ef43b79 P cb82cc7bd6db46d980c75237a1402860 [term 23 FOLLOWER]: Committing config change with OpId 23.17: config changed from index 16 to 17, cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) changed from NON_VOTER to VOTER. New config: { opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:22:52.384821 16293 raft_consensus.cc:2955] T c4f65cdf00b748bb963ee6e81ef43b79 P 90f3da57796b488c9087a2f523adf045 [term 23 FOLLOWER]: Committing config change with OpId 23.17: config changed from index 16 to 17, cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) changed from NON_VOTER to VOTER. New config: { opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:22:52.394645 16045 catalog_manager.cc:5697] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac reported cstate change: config changed from index 16 to 17, cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) changed from NON_VOTER to VOTER. New cstate: current_term: 23 leader_uuid: "1ca2725d290e47d68b1a4b609c9816ac" committed_config { opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:22:52.694139 16822 raft_consensus.cc:1217] T b6f07842d3aa47858df5665874c86f3a P cb82cc7bd6db46d980c75237a1402860 [term 26 LEARNER]: Deduplicated request from leader. Original: 26.16->[26.17-26.17]   Dedup: 26.17->[]
W20260709 13:22:52.748100 16790 consensus_peers.cc:597] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:22:52.757555 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 68755.8us
W20260709 13:22:52.760978 16790 consensus_peers.cc:597] T 2794af5b99984efebdee564035841cb0 P cb82cc7bd6db46d980c75237a1402860 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:22:52.758383 16265 consensus_peers.cc:597] T 434bff3082d141838068f323bdb0d1ed P 90f3da57796b488c9087a2f523adf045 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
I20260709 13:22:52.793185 24854 raft_consensus.cc:1064] T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac: attempting to promote NON_VOTER cb82cc7bd6db46d980c75237a1402860 to VOTER
I20260709 13:22:52.797363 24854 consensus_queue.cc:237] T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 23.15, Last appended by leader: 13, Current term: 23, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:22:52.802539 16266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 105940us
I20260709 13:22:52.821854 24855 raft_consensus.cc:1064] T 5a959dbecc2648eb85d1cc550f5a3245 P 1ca2725d290e47d68b1a4b609c9816ac: attempting to promote NON_VOTER cb82cc7bd6db46d980c75237a1402860 to VOTER
I20260709 13:22:52.825275 24855 consensus_queue.cc:237] T 5a959dbecc2648eb85d1cc550f5a3245 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 24.16, Last appended by leader: 14, Current term: 24, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:22:52.867491 16191 consensus_peers.cc:597] T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:22:52.878959 16298 raft_consensus.cc:1275] T 8d4ab870a05347868956aa8b841b2981 P 90f3da57796b488c9087a2f523adf045 [term 23 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 23 index: 15. Preceding OpId from leader: term: 23 index: 16. (index mismatch)
W20260709 13:22:52.886430 15046 thread.cc:527] Waited for 8000ms trying to join with diag-logger (tid 16172)
W20260709 13:22:52.921023 16191 consensus_peers.cc:597] T 5a959dbecc2648eb85d1cc550f5a3245 P 1ca2725d290e47d68b1a4b609c9816ac -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:22:52.923792 16790 consensus_peers.cc:597] T 3884f864a96d428785895e5463e5728f P cb82cc7bd6db46d980c75237a1402860 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
I20260709 13:22:52.940977 16817 raft_consensus.cc:1275] T 8d4ab870a05347868956aa8b841b2981 P cb82cc7bd6db46d980c75237a1402860 [term 23 LEARNER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 23 index: 15. Preceding OpId from leader: term: 23 index: 16. (index mismatch)
I20260709 13:22:52.949277 16298 raft_consensus.cc:1275] T 5a959dbecc2648eb85d1cc550f5a3245 P 90f3da57796b488c9087a2f523adf045 [term 24 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 24 index: 16. Preceding OpId from leader: term: 24 index: 17. (index mismatch)
I20260709 13:22:52.950980 24855 consensus_queue.cc:1048] T 5a959dbecc2648eb85d1cc550f5a3245 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 13:22:52.951258 24854 consensus_queue.cc:1048] T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 13:22:52.964434 16822 raft_consensus.cc:1275] T 5a959dbecc2648eb85d1cc550f5a3245 P cb82cc7bd6db46d980c75237a1402860 [term 24 LEARNER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 24 index: 16. Preceding OpId from leader: term: 24 index: 17. (index mismatch)
I20260709 13:22:52.980623 24855 consensus_queue.cc:1048] T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 13:22:52.984129 24829 consensus_queue.cc:1048] T 5a959dbecc2648eb85d1cc550f5a3245 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
W20260709 13:22:53.023654 16265 consensus_peers.cc:597] T fffab26427e24d398e2695aad40ca134 P 90f3da57796b488c9087a2f523adf045 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:22:53.040681 16265 consensus_peers.cc:597] T 1df93779209748b9860f6d893dd3de53 P 90f3da57796b488c9087a2f523adf045 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:22:53.055452 24829 raft_consensus.cc:2955] T 5a959dbecc2648eb85d1cc550f5a3245 P 1ca2725d290e47d68b1a4b609c9816ac [term 24 LEADER]: Committing config change with OpId 24.17: config changed from index 16 to 17, cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) changed from NON_VOTER to VOTER. New config: { opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
W20260709 13:22:53.068236 16790 consensus_peers.cc:597] T 23f82224f4564fea93572e8206bad1fe P cb82cc7bd6db46d980c75237a1402860 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:22:53.070668 16265 consensus_peers.cc:597] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 90f3da57796b488c9087a2f523adf045 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
I20260709 13:22:53.077761 16821 raft_consensus.cc:2955] T 5a959dbecc2648eb85d1cc550f5a3245 P cb82cc7bd6db46d980c75237a1402860 [term 24 FOLLOWER]: Committing config change with OpId 24.17: config changed from index 16 to 17, cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) changed from NON_VOTER to VOTER. New config: { opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
W20260709 13:22:53.100725 16265 consensus_peers.cc:597] T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:22:53.103466 24855 raft_consensus.cc:2955] T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac [term 23 LEADER]: Committing config change with OpId 23.16: config changed from index 15 to 16, cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) changed from NON_VOTER to VOTER. New config: { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:22:53.105113 24830 raft_consensus.cc:1064] T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045: attempting to promote NON_VOTER cb82cc7bd6db46d980c75237a1402860 to VOTER
I20260709 13:22:53.120241 24830 consensus_queue.cc:237] T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 26.17, Last appended by leader: 15, Current term: 26, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:53.114701 16298 raft_consensus.cc:2955] T 5a959dbecc2648eb85d1cc550f5a3245 P 90f3da57796b488c9087a2f523adf045 [term 24 FOLLOWER]: Committing config change with OpId 24.17: config changed from index 16 to 17, cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) changed from NON_VOTER to VOTER. New config: { opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
W20260709 13:22:53.129480 16790 consensus_peers.cc:597] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 13:22:53.138623 16295 raft_consensus.cc:2955] T 8d4ab870a05347868956aa8b841b2981 P 90f3da57796b488c9087a2f523adf045 [term 23 FOLLOWER]: Committing config change with OpId 23.16: config changed from index 15 to 16, cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) changed from NON_VOTER to VOTER. New config: { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
W20260709 13:22:53.149968 16790 consensus_peers.cc:597] T 66f75124215f4ca6bef831e174a792d8 P cb82cc7bd6db46d980c75237a1402860 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
I20260709 13:22:53.155449 16824 raft_consensus.cc:2955] T 8d4ab870a05347868956aa8b841b2981 P cb82cc7bd6db46d980c75237a1402860 [term 23 FOLLOWER]: Committing config change with OpId 23.16: config changed from index 15 to 16, cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) changed from NON_VOTER to VOTER. New config: { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:22:53.162653 16044 catalog_manager.cc:5697] T 5a959dbecc2648eb85d1cc550f5a3245 P 1ca2725d290e47d68b1a4b609c9816ac reported cstate change: config changed from index 16 to 17, cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) changed from NON_VOTER to VOTER. New cstate: current_term: 24 leader_uuid: "1ca2725d290e47d68b1a4b609c9816ac" committed_config { opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:22:53.205583 16191 consensus_peers.cc:597] T 829b4ad8c5a54eababeea1181b960d4b P 1ca2725d290e47d68b1a4b609c9816ac -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:22:53.236723 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 44080us
W20260709 13:22:53.250978 16790 consensus_peers.cc:597] T 806d7690c2b644959152074bb2cab0a7 P cb82cc7bd6db46d980c75237a1402860 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:22:53.256733 16040 catalog_manager.cc:5697] T 8d4ab870a05347868956aa8b841b2981 P 90f3da57796b488c9087a2f523adf045 reported cstate change: config changed from index 15 to 16, cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) changed from NON_VOTER to VOTER. New cstate: current_term: 23 leader_uuid: "1ca2725d290e47d68b1a4b609c9816ac" committed_config { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
W20260709 13:22:53.263345 16790 consensus_peers.cc:597] T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
I20260709 13:22:53.287114 16821 raft_consensus.cc:1275] T b6f07842d3aa47858df5665874c86f3a P cb82cc7bd6db46d980c75237a1402860 [term 26 LEARNER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 26 index: 17. Preceding OpId from leader: term: 26 index: 18. (index mismatch)
I20260709 13:22:53.299063 24857 consensus_queue.cc:1048] T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.001s
I20260709 13:22:53.330895 16223 raft_consensus.cc:1275] T b6f07842d3aa47858df5665874c86f3a P 1ca2725d290e47d68b1a4b609c9816ac [term 26 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 26 index: 17. Preceding OpId from leader: term: 26 index: 18. (index mismatch)
I20260709 13:22:53.339525 24830 consensus_queue.cc:1048] T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.000s
W20260709 13:22:53.350636 16265 consensus_peers.cc:597] T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:22:53.388461 16265 consensus_peers.cc:597] T 26354361e9b044cea1c2b86c4882fe92 P 90f3da57796b488c9087a2f523adf045 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
I20260709 13:22:53.426258 24850 raft_consensus.cc:2955] T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045 [term 26 LEADER]: Committing config change with OpId 26.18: config changed from index 17 to 18, cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) changed from NON_VOTER to VOTER. New config: { opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
W20260709 13:22:53.433270 16035 catalog_manager.cc:4755] TS 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet 18ad7e3af5504571a69d10c00750a01c: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111)
I20260709 13:22:53.434378 16223 consensus_queue.cc:237] T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 23.16, Last appended by leader: 13, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:22:53.442512 16219 raft_consensus.cc:2955] T b6f07842d3aa47858df5665874c86f3a P 1ca2725d290e47d68b1a4b609c9816ac [term 26 FOLLOWER]: Committing config change with OpId 26.18: config changed from index 17 to 18, cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) changed from NON_VOTER to VOTER. New config: { opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:22:53.449419 16821 raft_consensus.cc:1275] T 8d4ab870a05347868956aa8b841b2981 P cb82cc7bd6db46d980c75237a1402860 [term 23 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 23 index: 16. Preceding OpId from leader: term: 23 index: 17. (index mismatch)
I20260709 13:22:53.449419 16298 raft_consensus.cc:1275] T 8d4ab870a05347868956aa8b841b2981 P 90f3da57796b488c9087a2f523adf045 [term 23 FOLLOWER]: Refusing update from remote peer 1ca2725d290e47d68b1a4b609c9816ac: Log matching property violated. Preceding OpId in replica: term: 23 index: 16. Preceding OpId from leader: term: 23 index: 17. (index mismatch)
I20260709 13:22:53.452847 24854 consensus_queue.cc:1048] T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 13:22:53.453640 24667 consensus_queue.cc:1048] T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 13:22:53.463517 24851 raft_consensus.cc:2955] T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac [term 23 LEADER]: Committing config change with OpId 23.17: config changed from index 16 to 17, VOTER 647a343a542d4a139fe106c19095129b (127.14.177.129) evicted. New config: { opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:22:53.465227 16821 raft_consensus.cc:2955] T 8d4ab870a05347868956aa8b841b2981 P cb82cc7bd6db46d980c75237a1402860 [term 23 FOLLOWER]: Committing config change with OpId 23.17: config changed from index 16 to 17, VOTER 647a343a542d4a139fe106c19095129b (127.14.177.129) evicted. New config: { opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:22:53.471396 16824 raft_consensus.cc:2955] T b6f07842d3aa47858df5665874c86f3a P cb82cc7bd6db46d980c75237a1402860 [term 26 FOLLOWER]: Committing config change with OpId 26.18: config changed from index 17 to 18, cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) changed from NON_VOTER to VOTER. New config: { opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:22:53.473380 16295 raft_consensus.cc:2955] T 8d4ab870a05347868956aa8b841b2981 P 90f3da57796b488c9087a2f523adf045 [term 23 FOLLOWER]: Committing config change with OpId 23.17: config changed from index 16 to 17, VOTER 647a343a542d4a139fe106c19095129b (127.14.177.129) evicted. New config: { opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:22:53.489001 16045 catalog_manager.cc:5697] T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045 reported cstate change: config changed from index 17 to 18, cb82cc7bd6db46d980c75237a1402860 (127.14.177.132) changed from NON_VOTER to VOTER. New cstate: current_term: 26 leader_uuid: "90f3da57796b488c9087a2f523adf045" committed_config { opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:22:53.541251 16035 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 8d4ab870a05347868956aa8b841b2981 with cas_config_opid_index 16: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:22:53.558988 16044 catalog_manager.cc:5697] T 8d4ab870a05347868956aa8b841b2981 P cb82cc7bd6db46d980c75237a1402860 reported cstate change: config changed from index 16 to 17, VOTER 647a343a542d4a139fe106c19095129b (127.14.177.129) evicted. New cstate: current_term: 23 leader_uuid: "1ca2725d290e47d68b1a4b609c9816ac" committed_config { opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
W20260709 13:22:53.623035 16191 consensus_peers.cc:597] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:22:53.655629 16265 consensus_peers.cc:597] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:22:53.715989 16191 consensus_peers.cc:597] T c0eab25ce4c64e018a041beeda2867db P 1ca2725d290e47d68b1a4b609c9816ac -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:22:53.741623 16790 consensus_peers.cc:597] T 57a744f5aaf944678a63469cbf514015 P cb82cc7bd6db46d980c75237a1402860 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:22:53.886962 15046 thread.cc:527] Waited for 9000ms trying to join with diag-logger (tid 16172)
W20260709 13:22:53.909586 16191 consensus_peers.cc:597] T ee9ed67006144797b1de6ea8c2013a68 P 1ca2725d290e47d68b1a4b609c9816ac -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:22:53.910604 16265 consensus_peers.cc:597] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 90f3da57796b488c9087a2f523adf045 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:22:53.933079 16191 consensus_peers.cc:597] T 746aabf856ab401c9ef5e859371e7b67 P 1ca2725d290e47d68b1a4b609c9816ac -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:22:53.947870 16790 consensus_peers.cc:597] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:22:53.994655 16265 consensus_peers.cc:597] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:22:54.054014 16191 consensus_peers.cc:597] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:22:54.064622 16265 consensus_peers.cc:597] T 8cc959fb7f704fba9009c0595467d074 P 90f3da57796b488c9087a2f523adf045 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:22:54.114986 16191 consensus_peers.cc:597] T e5ad4eb281564936a59f41e387a47156 P 1ca2725d290e47d68b1a4b609c9816ac -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:22:54.227216 16191 consensus_peers.cc:597] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:22:54.231195 16265 consensus_peers.cc:597] T e4dc6b0541ca431ebc757beac00521f1 P 90f3da57796b488c9087a2f523adf045 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:22:54.290807 16790 consensus_peers.cc:597] T 9e65676b21b748728ad39668cfe722ec P cb82cc7bd6db46d980c75237a1402860 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:22:54.373399 16265 consensus_peers.cc:597] T 8a878999c0024d2abe517484180290fc P 90f3da57796b488c9087a2f523adf045 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:22:54.422549 16191 consensus_peers.cc:597] T af57996a8be84d398f0f023e8d08ef77 P 1ca2725d290e47d68b1a4b609c9816ac -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:22:54.508348 16790 consensus_peers.cc:597] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:22:54.546661 16191 consensus_peers.cc:597] T 84f2da98864b4425bf7c0055951fdd4b P 1ca2725d290e47d68b1a4b609c9816ac -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:22:54.566957 16191 consensus_peers.cc:597] T 5a96373bb0e14dce8a266b2fc6c58507 P 1ca2725d290e47d68b1a4b609c9816ac -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:22:54.646943 16265 consensus_peers.cc:597] T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:22:54.665445 16191 consensus_peers.cc:597] T 68a6838d82ef4cac83507f906139d90f P 1ca2725d290e47d68b1a4b609c9816ac -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:22:54.796588 16191 consensus_peers.cc:597] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:22:54.827615 16265 consensus_peers.cc:597] T b87147731a68400e8cce9b879daa7c5d P 90f3da57796b488c9087a2f523adf045 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:22:54.887475 15046 thread.cc:527] Waited for 10000ms trying to join with diag-logger (tid 16172)
W20260709 13:22:54.889739 16265 consensus_peers.cc:597] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:22:54.896307 16035 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 45285.5us
W20260709 13:22:54.913247 16035 catalog_manager.cc:4755] TS 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet 8d4ab870a05347868956aa8b841b2981: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111)
W20260709 13:22:54.940567 16191 consensus_peers.cc:597] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:22:54.951237 16790 consensus_peers.cc:597] T 2794af5b99984efebdee564035841cb0 P cb82cc7bd6db46d980c75237a1402860 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:22:54.982602 16191 consensus_peers.cc:597] T d16dfad0c3b540e5b4b86492ddf87208 P 1ca2725d290e47d68b1a4b609c9816ac -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:22:54.987911 16265 consensus_peers.cc:597] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:22:54.995117 16265 consensus_peers.cc:597] T 434bff3082d141838068f323bdb0d1ed P 90f3da57796b488c9087a2f523adf045 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:22:55.064019 16790 consensus_peers.cc:597] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:22:55.094203 16265 consensus_peers.cc:597] T 02838c86531d4f98bf58d1c22b8baf77 P 90f3da57796b488c9087a2f523adf045 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:22:55.353649 16191 consensus_peers.cc:597] T 5a959dbecc2648eb85d1cc550f5a3245 P 1ca2725d290e47d68b1a4b609c9816ac -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:22:55.405277 16790 consensus_peers.cc:597] T 3884f864a96d428785895e5463e5728f P cb82cc7bd6db46d980c75237a1402860 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:22:55.405879 16790 consensus_peers.cc:597] T 23f82224f4564fea93572e8206bad1fe P cb82cc7bd6db46d980c75237a1402860 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:22:55.428706 16265 consensus_peers.cc:597] T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:22:55.555104 16265 consensus_peers.cc:597] T 1df93779209748b9860f6d893dd3de53 P 90f3da57796b488c9087a2f523adf045 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:22:55.573558 16265 consensus_peers.cc:597] T fffab26427e24d398e2695aad40ca134 P 90f3da57796b488c9087a2f523adf045 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:22:55.590967 16790 consensus_peers.cc:597] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:22:55.625815 16790 consensus_peers.cc:597] T 66f75124215f4ca6bef831e174a792d8 P cb82cc7bd6db46d980c75237a1402860 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:22:55.723809 16790 consensus_peers.cc:597] T 806d7690c2b644959152074bb2cab0a7 P cb82cc7bd6db46d980c75237a1402860 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:22:55.730374 16191 consensus_peers.cc:597] T 829b4ad8c5a54eababeea1181b960d4b P 1ca2725d290e47d68b1a4b609c9816ac -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:22:55.763583 16790 consensus_peers.cc:597] T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:22:55.764753 16266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 56771.6us
W20260709 13:22:55.768709 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 50312.2us
W20260709 13:22:55.809917 16265 consensus_peers.cc:597] T 26354361e9b044cea1c2b86c4882fe92 P 90f3da57796b488c9087a2f523adf045 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:22:55.810555 16265 consensus_peers.cc:597] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 90f3da57796b488c9087a2f523adf045 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:22:55.886471 16265 consensus_peers.cc:597] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:22:55.887975 15046 thread.cc:527] Waited for 11000ms trying to join with diag-logger (tid 16172)
W20260709 13:22:56.055142 16035 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 76854.1us
W20260709 13:22:56.067233 16035 catalog_manager.cc:4755] TS 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet 8d4ab870a05347868956aa8b841b2981: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111)
W20260709 13:22:56.163544 16191 consensus_peers.cc:597] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:22:56.208431 16265 consensus_peers.cc:597] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:22:56.283511 16790 consensus_peers.cc:597] T 57a744f5aaf944678a63469cbf514015 P cb82cc7bd6db46d980c75237a1402860 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:22:56.331295 16191 consensus_peers.cc:597] T 746aabf856ab401c9ef5e859371e7b67 P 1ca2725d290e47d68b1a4b609c9816ac -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:22:56.373358 16191 consensus_peers.cc:597] T c0eab25ce4c64e018a041beeda2867db P 1ca2725d290e47d68b1a4b609c9816ac -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:22:56.448875 16790 consensus_peers.cc:597] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:22:56.455341 16265 consensus_peers.cc:597] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 90f3da57796b488c9087a2f523adf045 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:22:56.472388 16191 consensus_peers.cc:597] T e5ad4eb281564936a59f41e387a47156 P 1ca2725d290e47d68b1a4b609c9816ac -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:22:56.499977 16191 consensus_peers.cc:597] T ee9ed67006144797b1de6ea8c2013a68 P 1ca2725d290e47d68b1a4b609c9816ac -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:22:56.607875 16265 consensus_peers.cc:597] T 8cc959fb7f704fba9009c0595467d074 P 90f3da57796b488c9087a2f523adf045 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:22:56.632923 16191 consensus_peers.cc:597] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:22:56.715379 16265 consensus_peers.cc:597] T e4dc6b0541ca431ebc757beac00521f1 P 90f3da57796b488c9087a2f523adf045 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:22:56.757393 16790 consensus_peers.cc:597] T 9e65676b21b748728ad39668cfe722ec P cb82cc7bd6db46d980c75237a1402860 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:22:56.783550 16191 consensus_peers.cc:597] T 5a96373bb0e14dce8a266b2fc6c58507 P 1ca2725d290e47d68b1a4b609c9816ac -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:22:56.864152 16265 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111) [suppressed 455 similar messages]
W20260709 13:22:56.888470 15046 thread.cc:527] Waited for 12000ms trying to join with diag-logger (tid 16172)
W20260709 13:22:56.905803 16191 consensus_peers.cc:597] T 84f2da98864b4425bf7c0055951fdd4b P 1ca2725d290e47d68b1a4b609c9816ac -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:22:56.937754 16265 consensus_peers.cc:597] T 8a878999c0024d2abe517484180290fc P 90f3da57796b488c9087a2f523adf045 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:22:56.957490 16191 consensus_peers.cc:597] T af57996a8be84d398f0f023e8d08ef77 P 1ca2725d290e47d68b1a4b609c9816ac -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:22:56.973932 16790 consensus_peers.cc:597] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:22:56.987485 16191 consensus_peers.cc:597] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:22:57.200654 16265 consensus_peers.cc:597] T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:22:57.202397 16191 consensus_peers.cc:597] T 68a6838d82ef4cac83507f906139d90f P 1ca2725d290e47d68b1a4b609c9816ac -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:22:57.248119 16191 consensus_peers.cc:597] T d16dfad0c3b540e5b4b86492ddf87208 P 1ca2725d290e47d68b1a4b609c9816ac -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:22:57.305670 16265 consensus_peers.cc:597] T b87147731a68400e8cce9b879daa7c5d P 90f3da57796b488c9087a2f523adf045 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:22:57.322407 16265 consensus_peers.cc:597] T 02838c86531d4f98bf58d1c22b8baf77 P 90f3da57796b488c9087a2f523adf045 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:22:57.432201 16265 consensus_peers.cc:597] T 434bff3082d141838068f323bdb0d1ed P 90f3da57796b488c9087a2f523adf045 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:22:57.434403 16790 consensus_peers.cc:597] T 2794af5b99984efebdee564035841cb0 P cb82cc7bd6db46d980c75237a1402860 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:22:57.452231 16191 consensus_peers.cc:597] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:22:57.452148 16265 consensus_peers.cc:597] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:22:57.607826 16035 catalog_manager.cc:4755] TS 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet 18ad7e3af5504571a69d10c00750a01c: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111)
W20260709 13:22:57.712816 16191 consensus_peers.cc:597] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:22:57.831568 16790 consensus_peers.cc:597] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:22:57.841653 16265 consensus_peers.cc:597] T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:22:57.888989 15046 thread.cc:527] Waited for 13000ms trying to join with diag-logger (tid 16172)
W20260709 13:22:57.914343 16265 consensus_peers.cc:597] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:22:57.938886 16789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 186860us
W20260709 13:22:58.005522 16265 consensus_peers.cc:597] T fffab26427e24d398e2695aad40ca134 P 90f3da57796b488c9087a2f523adf045 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:22:58.008867 16192 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 100866us
W20260709 13:22:58.017577 16789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 69496.3us
W20260709 13:22:58.019377 16790 consensus_peers.cc:597] T 66f75124215f4ca6bef831e174a792d8 P cb82cc7bd6db46d980c75237a1402860 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:22:58.022488 16790 consensus_peers.cc:597] T 23f82224f4564fea93572e8206bad1fe P cb82cc7bd6db46d980c75237a1402860 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:22:58.054376 16790 consensus_peers.cc:597] T 3884f864a96d428785895e5463e5728f P cb82cc7bd6db46d980c75237a1402860 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:22:58.058554 16265 consensus_peers.cc:597] T 1df93779209748b9860f6d893dd3de53 P 90f3da57796b488c9087a2f523adf045 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:22:58.121834 16192 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 92615.5us
W20260709 13:22:58.165320 16265 consensus_peers.cc:597] T 26354361e9b044cea1c2b86c4882fe92 P 90f3da57796b488c9087a2f523adf045 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:22:58.186467 16790 consensus_peers.cc:597] T 806d7690c2b644959152074bb2cab0a7 P cb82cc7bd6db46d980c75237a1402860 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:22:58.197831 16790 consensus_peers.cc:597] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:22:58.327045 16265 consensus_peers.cc:597] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 90f3da57796b488c9087a2f523adf045 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:22:58.340389 16191 consensus_peers.cc:597] T 5a959dbecc2648eb85d1cc550f5a3245 P 1ca2725d290e47d68b1a4b609c9816ac -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:22:58.370121 16265 consensus_peers.cc:597] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:22:58.435770 16790 consensus_peers.cc:597] T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:22:58.482149 16265 consensus_peers.cc:597] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:22:58.525547 16191 consensus_peers.cc:597] T 829b4ad8c5a54eababeea1181b960d4b P 1ca2725d290e47d68b1a4b609c9816ac -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:22:58.620096 16191 consensus_peers.cc:597] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:22:58.638984 16191 consensus_peers.cc:597] T 746aabf856ab401c9ef5e859371e7b67 P 1ca2725d290e47d68b1a4b609c9816ac -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:22:58.696905 16790 consensus_peers.cc:597] T 57a744f5aaf944678a63469cbf514015 P cb82cc7bd6db46d980c75237a1402860 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:22:58.827066 16265 consensus_peers.cc:597] T 8cc959fb7f704fba9009c0595467d074 P 90f3da57796b488c9087a2f523adf045 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:22:58.840518 16191 consensus_peers.cc:597] T c0eab25ce4c64e018a041beeda2867db P 1ca2725d290e47d68b1a4b609c9816ac -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:22:58.889747 15046 thread.cc:527] Waited for 14000ms trying to join with diag-logger (tid 16172)
W20260709 13:22:58.977469 16790 consensus_peers.cc:597] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:22:58.988703 16790 consensus_peers.cc:597] T 9e65676b21b748728ad39668cfe722ec P cb82cc7bd6db46d980c75237a1402860 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:22:58.991235 16265 consensus_peers.cc:597] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 90f3da57796b488c9087a2f523adf045 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:22:58.996169 16191 consensus_peers.cc:597] T e5ad4eb281564936a59f41e387a47156 P 1ca2725d290e47d68b1a4b609c9816ac -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:22:59.141233 16191 consensus_peers.cc:597] T ee9ed67006144797b1de6ea8c2013a68 P 1ca2725d290e47d68b1a4b609c9816ac -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:22:59.215008 16265 consensus_peers.cc:597] T e4dc6b0541ca431ebc757beac00521f1 P 90f3da57796b488c9087a2f523adf045 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:22:59.317552 16265 consensus_peers.cc:597] T 8a878999c0024d2abe517484180290fc P 90f3da57796b488c9087a2f523adf045 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:22:59.413312 16191 consensus_peers.cc:597] T 5a96373bb0e14dce8a266b2fc6c58507 P 1ca2725d290e47d68b1a4b609c9816ac -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:22:59.423079 16191 consensus_peers.cc:597] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:22:59.537396 16191 consensus_peers.cc:597] T af57996a8be84d398f0f023e8d08ef77 P 1ca2725d290e47d68b1a4b609c9816ac -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:22:59.538136 16191 consensus_peers.cc:597] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:22:59.564672 16265 consensus_peers.cc:597] T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:22:59.597215 16265 consensus_peers.cc:597] T 02838c86531d4f98bf58d1c22b8baf77 P 90f3da57796b488c9087a2f523adf045 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:22:59.716629 16191 consensus_peers.cc:597] T 84f2da98864b4425bf7c0055951fdd4b P 1ca2725d290e47d68b1a4b609c9816ac -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:22:59.718752 16790 consensus_peers.cc:597] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:22:59.793262 16265 consensus_peers.cc:597] T 434bff3082d141838068f323bdb0d1ed P 90f3da57796b488c9087a2f523adf045 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:22:59.905665 15046 thread.cc:527] Waited for 15000ms trying to join with diag-logger (tid 16172)
W20260709 13:22:59.909168 16265 consensus_peers.cc:597] T b87147731a68400e8cce9b879daa7c5d P 90f3da57796b488c9087a2f523adf045 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:22:59.959192 16790 consensus_peers.cc:597] T 2794af5b99984efebdee564035841cb0 P cb82cc7bd6db46d980c75237a1402860 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:23:00.086277 16191 consensus_peers.cc:597] T 68a6838d82ef4cac83507f906139d90f P 1ca2725d290e47d68b1a4b609c9816ac -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:23:00.126348 16265 consensus_peers.cc:597] T fffab26427e24d398e2695aad40ca134 P 90f3da57796b488c9087a2f523adf045 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:23:00.154145 16265 consensus_peers.cc:597] T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:23:00.378132 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 224035us
W20260709 13:23:00.562846 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 410045us
W20260709 13:23:00.659852 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 77818.3us
W20260709 13:23:00.776016 16790 consensus_peers.cc:597] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:23:00.777535 16790 consensus_peers.cc:597] T 806d7690c2b644959152074bb2cab0a7 P cb82cc7bd6db46d980c75237a1402860 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:23:00.778815 16790 consensus_peers.cc:597] T 23f82224f4564fea93572e8206bad1fe P cb82cc7bd6db46d980c75237a1402860 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:23:00.781456 16790 consensus_peers.cc:597] T 66f75124215f4ca6bef831e174a792d8 P cb82cc7bd6db46d980c75237a1402860 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:23:00.782413 16790 consensus_peers.cc:597] T 3884f864a96d428785895e5463e5728f P cb82cc7bd6db46d980c75237a1402860 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:23:00.783977 16266 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 51627us
W20260709 13:23:00.822055 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 55162.9us
W20260709 13:23:00.851154 16192 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 49599.6us
W20260709 13:23:00.871846 16191 consensus_peers.cc:597] T d16dfad0c3b540e5b4b86492ddf87208 P 1ca2725d290e47d68b1a4b609c9816ac -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:23:00.872967 16191 consensus_peers.cc:597] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:23:00.875326 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 152131us
W20260709 13:23:00.901016 16790 consensus_peers.cc:597] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:23:00.906221 15046 thread.cc:527] Waited for 16000ms trying to join with diag-logger (tid 16172)
W20260709 13:23:00.954131 16265 consensus_peers.cc:597] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:23:00.956141 16265 consensus_peers.cc:597] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:23:00.962739 16265 consensus_peers.cc:597] T 1df93779209748b9860f6d893dd3de53 P 90f3da57796b488c9087a2f523adf045 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:23:00.965344 16191 consensus_peers.cc:597] T 829b4ad8c5a54eababeea1181b960d4b P 1ca2725d290e47d68b1a4b609c9816ac -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:23:01.016165 16265 consensus_peers.cc:597] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:23:01.124567 16790 consensus_peers.cc:597] T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:23:01.172240 16080 debug-util.cc:398] Leaking SignalData structure 0x7b0800091c40 after lost signal to thread 15049
W20260709 13:23:01.173558 16080 debug-util.cc:398] Leaking SignalData structure 0x7b080036dd40 after lost signal to thread 16083
W20260709 13:23:01.175064 16080 debug-util.cc:398] Leaking SignalData structure 0x7b080037d7a0 after lost signal to thread 16250
W20260709 13:23:01.176558 16080 debug-util.cc:398] Leaking SignalData structure 0x7b080029b4c0 after lost signal to thread 16324
W20260709 13:23:01.178068 16080 debug-util.cc:398] Leaking SignalData structure 0x7b08002f1e20 after lost signal to thread 16849
W20260709 13:23:01.192003 16191 consensus_peers.cc:597] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:23:01.241389 16265 consensus_peers.cc:597] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:23:01.242753 16265 consensus_peers.cc:597] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 90f3da57796b488c9087a2f523adf045 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:23:01.266120 16191 consensus_peers.cc:597] T 5a959dbecc2648eb85d1cc550f5a3245 P 1ca2725d290e47d68b1a4b609c9816ac -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:23:01.397260 16265 consensus_peers.cc:597] T e4dc6b0541ca431ebc757beac00521f1 P 90f3da57796b488c9087a2f523adf045 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:23:01.417898 16265 consensus_peers.cc:597] T 26354361e9b044cea1c2b86c4882fe92 P 90f3da57796b488c9087a2f523adf045 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:23:01.456954 16191 consensus_peers.cc:597] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:23:01.561260 16790 consensus_peers.cc:597] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:23:01.672703 16191 consensus_peers.cc:597] T e5ad4eb281564936a59f41e387a47156 P 1ca2725d290e47d68b1a4b609c9816ac -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:23:01.766275 16265 consensus_peers.cc:597] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 90f3da57796b488c9087a2f523adf045 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:23:01.779853 16790 consensus_peers.cc:597] T 57a744f5aaf944678a63469cbf514015 P cb82cc7bd6db46d980c75237a1402860 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:23:01.782578 16790 consensus_peers.cc:597] T 9e65676b21b748728ad39668cfe722ec P cb82cc7bd6db46d980c75237a1402860 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:23:01.840965 16191 consensus_peers.cc:597] T c0eab25ce4c64e018a041beeda2867db P 1ca2725d290e47d68b1a4b609c9816ac -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:23:01.869201 16790 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111) [suppressed 425 similar messages]
W20260709 13:23:01.906738 15046 thread.cc:527] Waited for 17000ms trying to join with diag-logger (tid 16172)
W20260709 13:23:01.956463 16191 consensus_peers.cc:597] T ee9ed67006144797b1de6ea8c2013a68 P 1ca2725d290e47d68b1a4b609c9816ac -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:23:01.989037 16265 consensus_peers.cc:597] T 8cc959fb7f704fba9009c0595467d074 P 90f3da57796b488c9087a2f523adf045 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:23:02.016450 16191 consensus_peers.cc:597] T 746aabf856ab401c9ef5e859371e7b67 P 1ca2725d290e47d68b1a4b609c9816ac -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:23:02.134806 16191 consensus_peers.cc:597] T 5a96373bb0e14dce8a266b2fc6c58507 P 1ca2725d290e47d68b1a4b609c9816ac -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:23:02.138545 16191 consensus_peers.cc:597] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:23:02.311873 16191 consensus_peers.cc:597] T af57996a8be84d398f0f023e8d08ef77 P 1ca2725d290e47d68b1a4b609c9816ac -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:23:02.389405 16035 catalog_manager.cc:4755] TS 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet 8d4ab870a05347868956aa8b841b2981: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111)
W20260709 13:23:02.465708 16265 consensus_peers.cc:597] T fffab26427e24d398e2695aad40ca134 P 90f3da57796b488c9087a2f523adf045 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:23:02.478014 16265 consensus_peers.cc:597] T 8a878999c0024d2abe517484180290fc P 90f3da57796b488c9087a2f523adf045 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:23:02.505120 16265 consensus_peers.cc:597] T 02838c86531d4f98bf58d1c22b8baf77 P 90f3da57796b488c9087a2f523adf045 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:23:02.551229 16191 consensus_peers.cc:597] T 84f2da98864b4425bf7c0055951fdd4b P 1ca2725d290e47d68b1a4b609c9816ac -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:23:02.562091 16265 consensus_peers.cc:597] T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:23:02.622521 16790 consensus_peers.cc:597] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:23:02.721673 16191 consensus_peers.cc:597] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:23:02.800575 16265 consensus_peers.cc:597] T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:23:02.828207 16789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 72056.9us
W20260709 13:23:02.849995 16191 consensus_peers.cc:597] T 68a6838d82ef4cac83507f906139d90f P 1ca2725d290e47d68b1a4b609c9816ac -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:23:02.851751 16790 consensus_peers.cc:597] T 2794af5b99984efebdee564035841cb0 P cb82cc7bd6db46d980c75237a1402860 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:23:02.908103 15046 thread.cc:527] Waited for 18000ms trying to join with diag-logger (tid 16172)
W20260709 13:23:02.925539 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.130:35943, user_credentials={real_user=slave}} blocked reactor thread for 69384.3us
W20260709 13:23:02.966645 16790 consensus_peers.cc:597] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:23:02.968475 16265 consensus_peers.cc:597] T b87147731a68400e8cce9b879daa7c5d P 90f3da57796b488c9087a2f523adf045 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:23:03.096655 16191 consensus_peers.cc:597] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:23:03.111375 16790 consensus_peers.cc:597] T 23f82224f4564fea93572e8206bad1fe P cb82cc7bd6db46d980c75237a1402860 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:23:03.120906 16265 consensus_peers.cc:597] T 434bff3082d141838068f323bdb0d1ed P 90f3da57796b488c9087a2f523adf045 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:23:03.127930 16265 consensus_peers.cc:597] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:23:03.161305 16265 consensus_peers.cc:597] T 1df93779209748b9860f6d893dd3de53 P 90f3da57796b488c9087a2f523adf045 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:23:03.176601 16790 consensus_peers.cc:597] T 806d7690c2b644959152074bb2cab0a7 P cb82cc7bd6db46d980c75237a1402860 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:23:03.336285 16790 consensus_peers.cc:597] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:23:03.436228 16265 consensus_peers.cc:597] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 90f3da57796b488c9087a2f523adf045 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:23:03.438107 16265 consensus_peers.cc:597] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:23:03.443687 16790 consensus_peers.cc:597] T 3884f864a96d428785895e5463e5728f P cb82cc7bd6db46d980c75237a1402860 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:23:03.459367 16790 consensus_peers.cc:597] T 66f75124215f4ca6bef831e174a792d8 P cb82cc7bd6db46d980c75237a1402860 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:23:03.473438 16265 consensus_peers.cc:597] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:23:03.477124 16191 consensus_peers.cc:597] T 829b4ad8c5a54eababeea1181b960d4b P 1ca2725d290e47d68b1a4b609c9816ac -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:23:03.484900 16790 consensus_peers.cc:597] T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:23:03.531894 16191 consensus_peers.cc:597] T d16dfad0c3b540e5b4b86492ddf87208 P 1ca2725d290e47d68b1a4b609c9816ac -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:23:03.536603 16191 consensus_peers.cc:597] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:23:03.566798 16265 consensus_peers.cc:597] T e4dc6b0541ca431ebc757beac00521f1 P 90f3da57796b488c9087a2f523adf045 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:23:03.765355 16321 debug-util.cc:398] Leaking SignalData structure 0x7b080023f800 after lost signal to thread 15049
W20260709 13:23:03.766657 16321 debug-util.cc:398] Leaking SignalData structure 0x7b0800159540 after lost signal to thread 16083
W20260709 13:23:03.768185 16321 debug-util.cc:398] Leaking SignalData structure 0x7b08002baf80 after lost signal to thread 16250
W20260709 13:23:03.769469 16321 debug-util.cc:398] Leaking SignalData structure 0x7b08003b8c20 after lost signal to thread 16324
W20260709 13:23:03.771504 16321 debug-util.cc:398] Leaking SignalData structure 0x7b0800222e60 after lost signal to thread 16849
W20260709 13:23:03.908608 15046 thread.cc:527] Waited for 19000ms trying to join with diag-logger (tid 16172)
W20260709 13:23:03.926003 16191 consensus_peers.cc:597] T 5a959dbecc2648eb85d1cc550f5a3245 P 1ca2725d290e47d68b1a4b609c9816ac -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:23:04.022851 16191 consensus_peers.cc:597] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:23:04.041663 16265 consensus_peers.cc:597] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:23:04.077850 16265 consensus_peers.cc:597] T 26354361e9b044cea1c2b86c4882fe92 P 90f3da57796b488c9087a2f523adf045 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:23:04.143862 16790 consensus_peers.cc:597] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:23:04.168562 16191 consensus_peers.cc:597] T e5ad4eb281564936a59f41e387a47156 P 1ca2725d290e47d68b1a4b609c9816ac -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:23:04.264607 16265 consensus_peers.cc:597] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 90f3da57796b488c9087a2f523adf045 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:23:04.279049 16790 consensus_peers.cc:597] T 57a744f5aaf944678a63469cbf514015 P cb82cc7bd6db46d980c75237a1402860 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:23:04.291913 16191 consensus_peers.cc:597] T c0eab25ce4c64e018a041beeda2867db P 1ca2725d290e47d68b1a4b609c9816ac -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:23:04.519582 16191 consensus_peers.cc:597] T ee9ed67006144797b1de6ea8c2013a68 P 1ca2725d290e47d68b1a4b609c9816ac -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:23:04.617632 16191 consensus_peers.cc:597] T 746aabf856ab401c9ef5e859371e7b67 P 1ca2725d290e47d68b1a4b609c9816ac -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:23:04.727985 16790 consensus_peers.cc:597] T 9e65676b21b748728ad39668cfe722ec P cb82cc7bd6db46d980c75237a1402860 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:23:04.744053 16789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 94748.2us
W20260709 13:23:04.777376 16191 consensus_peers.cc:597] T 5a96373bb0e14dce8a266b2fc6c58507 P 1ca2725d290e47d68b1a4b609c9816ac -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:23:04.818583 16191 consensus_peers.cc:597] T 84f2da98864b4425bf7c0055951fdd4b P 1ca2725d290e47d68b1a4b609c9816ac -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:23:04.831367 16191 consensus_peers.cc:597] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:23:04.832695 16265 consensus_peers.cc:597] T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:23:04.889319 16265 consensus_peers.cc:597] T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:23:04.909178 15046 thread.cc:527] Waited for 20000ms trying to join with diag-logger (tid 16172)
W20260709 13:23:04.913502 16790 consensus_peers.cc:597] T 2794af5b99984efebdee564035841cb0 P cb82cc7bd6db46d980c75237a1402860 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 13:23:04.943387 16790 consensus_peers.cc:597] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:23:04.958076 16265 consensus_peers.cc:597] T 02838c86531d4f98bf58d1c22b8baf77 P 90f3da57796b488c9087a2f523adf045 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:23:04.983801 16265 consensus_peers.cc:597] T 8cc959fb7f704fba9009c0595467d074 P 90f3da57796b488c9087a2f523adf045 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:23:05.021924 16265 consensus_peers.cc:597] T 8a878999c0024d2abe517484180290fc P 90f3da57796b488c9087a2f523adf045 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:23:05.033465 16191 consensus_peers.cc:597] T af57996a8be84d398f0f023e8d08ef77 P 1ca2725d290e47d68b1a4b609c9816ac -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:23:05.084913 16265 consensus_peers.cc:597] T fffab26427e24d398e2695aad40ca134 P 90f3da57796b488c9087a2f523adf045 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:23:05.105343 16191 consensus_peers.cc:597] T 68a6838d82ef4cac83507f906139d90f P 1ca2725d290e47d68b1a4b609c9816ac -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:23:05.151026 16191 consensus_peers.cc:597] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:23:05.160578 16790 consensus_peers.cc:597] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:23:05.414081 16265 consensus_peers.cc:597] T b87147731a68400e8cce9b879daa7c5d P 90f3da57796b488c9087a2f523adf045 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:23:05.445912 16265 consensus_peers.cc:597] T 434bff3082d141838068f323bdb0d1ed P 90f3da57796b488c9087a2f523adf045 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:23:05.452356 16790 consensus_peers.cc:597] T 23f82224f4564fea93572e8206bad1fe P cb82cc7bd6db46d980c75237a1402860 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 13:23:05.465842 16191 consensus_peers.cc:597] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:23:05.492828 16265 consensus_peers.cc:597] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:23:05.533520 16265 consensus_peers.cc:597] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 90f3da57796b488c9087a2f523adf045 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:23:05.591207 16790 consensus_peers.cc:597] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:23:05.594081 16265 consensus_peers.cc:597] T 1df93779209748b9860f6d893dd3de53 P 90f3da57796b488c9087a2f523adf045 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:23:05.662529 16265 consensus_peers.cc:597] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:23:05.800046 16790 consensus_peers.cc:597] T 806d7690c2b644959152074bb2cab0a7 P cb82cc7bd6db46d980c75237a1402860 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:23:05.909663 15046 thread.cc:527] Waited for 21000ms trying to join with diag-logger (tid 16172)
W20260709 13:23:05.961061 16790 consensus_peers.cc:597] T 66f75124215f4ca6bef831e174a792d8 P cb82cc7bd6db46d980c75237a1402860 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:23:05.976861 16790 consensus_peers.cc:597] T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 13:23:06.007779 16035 catalog_manager.cc:4755] TS 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet 18ad7e3af5504571a69d10c00750a01c: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111)
W20260709 13:23:06.019642 16191 consensus_peers.cc:597] T d16dfad0c3b540e5b4b86492ddf87208 P 1ca2725d290e47d68b1a4b609c9816ac -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:23:06.029393 16265 consensus_peers.cc:597] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:23:06.151332 16790 consensus_peers.cc:597] T 3884f864a96d428785895e5463e5728f P cb82cc7bd6db46d980c75237a1402860 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 13:23:06.201784 16265 consensus_peers.cc:597] T e4dc6b0541ca431ebc757beac00521f1 P 90f3da57796b488c9087a2f523adf045 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:23:06.354162 16191 consensus_peers.cc:597] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:23:06.377378 16191 consensus_peers.cc:597] T 829b4ad8c5a54eababeea1181b960d4b P 1ca2725d290e47d68b1a4b609c9816ac -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:23:06.397472 16790 consensus_peers.cc:597] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:23:06.440104 16191 consensus_peers.cc:597] T e5ad4eb281564936a59f41e387a47156 P 1ca2725d290e47d68b1a4b609c9816ac -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:23:06.451785 16265 consensus_peers.cc:597] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:23:06.707960 16191 consensus_peers.cc:597] T c0eab25ce4c64e018a041beeda2867db P 1ca2725d290e47d68b1a4b609c9816ac -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:23:06.731498 16191 consensus_peers.cc:597] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:23:06.785429 16265 consensus_peers.cc:597] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 90f3da57796b488c9087a2f523adf045 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:23:06.786203 16191 consensus_peers.cc:597] T 5a959dbecc2648eb85d1cc550f5a3245 P 1ca2725d290e47d68b1a4b609c9816ac -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:23:06.848855 16790 consensus_peers.cc:597] T 57a744f5aaf944678a63469cbf514015 P cb82cc7bd6db46d980c75237a1402860 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 13:23:06.850436 16265 consensus_peers.cc:597] T 26354361e9b044cea1c2b86c4882fe92 P 90f3da57796b488c9087a2f523adf045 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:23:06.911279 15046 thread.cc:527] Waited for 22000ms trying to join with diag-logger (tid 16172)
W20260709 13:23:07.911897 15046 thread.cc:527] Waited for 23000ms trying to join with diag-logger (tid 16172)
I20260709 13:23:07.978116 24973 raft_consensus.cc:493] T 2794af5b99984efebdee564035841cb0 P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:23:07.979369 24973 raft_consensus.cc:515] T 2794af5b99984efebdee564035841cb0 P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:23:07.994649 24968 raft_consensus.cc:493] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [term 29 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:23:08.029137 24968 raft_consensus.cc:515] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:23:07.993709 24978 raft_consensus.cc:493] T 8c11074579b046d99e585c2818e18a2c P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:23:08.033088 24978 raft_consensus.cc:515] T 8c11074579b046d99e585c2818e18a2c P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:23:07.981441 16265 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111) [suppressed 451 similar messages]
I20260709 13:23:08.036091 24978 leader_election.cc:290] T 8c11074579b046d99e585c2818e18a2c P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:23:08.038095 24978 raft_consensus.cc:493] T 9e65676b21b748728ad39668cfe722ec P 90f3da57796b488c9087a2f523adf045 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:23:07.997402 24973 leader_election.cc:290] T 2794af5b99984efebdee564035841cb0 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
W20260709 13:23:07.999436 16790 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.931s	user 0.007s	sys 0.010s
I20260709 13:23:08.001984 24976 raft_consensus.cc:493] T a378d7f796f94615b3b347119a9880fd P 90f3da57796b488c9087a2f523adf045 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:23:08.039574 24976 raft_consensus.cc:515] T a378d7f796f94615b3b347119a9880fd P 90f3da57796b488c9087a2f523adf045 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:23:08.040659 16790 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.972s	user 0.008s	sys 0.010s
I20260709 13:23:08.042073 24976 leader_election.cc:290] T a378d7f796f94615b3b347119a9880fd P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:23:08.043773 24976 raft_consensus.cc:493] T 8d4ab870a05347868956aa8b841b2981 P 90f3da57796b488c9087a2f523adf045 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:23:08.030263 24971 raft_consensus.cc:493] T 3884f864a96d428785895e5463e5728f P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:23:07.987545 24967 raft_consensus.cc:493] T c0eab25ce4c64e018a041beeda2867db P 90f3da57796b488c9087a2f523adf045 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:23:08.047557 24971 raft_consensus.cc:515] T 3884f864a96d428785895e5463e5728f P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:23:08.048059 24967 raft_consensus.cc:515] T c0eab25ce4c64e018a041beeda2867db P 90f3da57796b488c9087a2f523adf045 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:23:08.050474 24971 leader_election.cc:290] T 3884f864a96d428785895e5463e5728f P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:23:08.038674 24978 raft_consensus.cc:515] T 9e65676b21b748728ad39668cfe722ec P 90f3da57796b488c9087a2f523adf045 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:23:08.029587 24975 raft_consensus.cc:493] T 8158bbd17b994d2eb96f560bd0f1b88d P 90f3da57796b488c9087a2f523adf045 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:23:08.051081 24967 leader_election.cc:290] T c0eab25ce4c64e018a041beeda2867db P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:23:08.051544 24975 raft_consensus.cc:515] T 8158bbd17b994d2eb96f560bd0f1b88d P 90f3da57796b488c9087a2f523adf045 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:23:08.053748 24978 leader_election.cc:290] T 9e65676b21b748728ad39668cfe722ec P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:23:08.054560 24976 raft_consensus.cc:515] T 8d4ab870a05347868956aa8b841b2981 P 90f3da57796b488c9087a2f523adf045 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:23:08.045821 24968 leader_election.cc:290] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:23:08.056845 24970 raft_consensus.cc:493] T 66f75124215f4ca6bef831e174a792d8 P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:23:08.076606 24970 raft_consensus.cc:515] T 66f75124215f4ca6bef831e174a792d8 P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:23:08.081180 24970 leader_election.cc:290] T 66f75124215f4ca6bef831e174a792d8 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:23:08.055821 24975 leader_election.cc:290] T 8158bbd17b994d2eb96f560bd0f1b88d P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:23:08.087944 24975 raft_consensus.cc:493] T ada5a004ba7c46819051e3ccec71ffdb P 90f3da57796b488c9087a2f523adf045 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:23:08.088670 24975 raft_consensus.cc:515] T ada5a004ba7c46819051e3ccec71ffdb P 90f3da57796b488c9087a2f523adf045 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:23:08.104919 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66ae176961b74813a58bc15c9c4c6f2a" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 30 candidate_status { last_received { term: 29 index: 18 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
W20260709 13:23:08.110199 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 44966.3us
I20260709 13:23:08.118842 16820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2794af5b99984efebdee564035841cb0" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:23:08.119295 16819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c11074579b046d99e585c2818e18a2c" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:23:08.120374 24995 raft_consensus.cc:493] T 5a959dbecc2648eb85d1cc550f5a3245 P cb82cc7bd6db46d980c75237a1402860 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:23:08.121155 24995 raft_consensus.cc:515] T 5a959dbecc2648eb85d1cc550f5a3245 P cb82cc7bd6db46d980c75237a1402860 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:23:08.121682 16821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a378d7f796f94615b3b347119a9880fd" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
W20260709 13:23:08.128925 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 147519us
I20260709 13:23:08.130867 16820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3884f864a96d428785895e5463e5728f" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:23:08.140281 16821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e65676b21b748728ad39668cfe722ec" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:23:08.144855 24994 raft_consensus.cc:493] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [term 26 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:23:08.145054 24970 raft_consensus.cc:493] T 08c74490c40243a88e62d1760b5cd814 P 1ca2725d290e47d68b1a4b609c9816ac [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:23:08.145733 24970 raft_consensus.cc:515] T 08c74490c40243a88e62d1760b5cd814 P 1ca2725d290e47d68b1a4b609c9816ac [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:23:08.145655 24994 raft_consensus.cc:515] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:23:08.148129 24970 leader_election.cc:290] T 08c74490c40243a88e62d1760b5cd814 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:23:08.148985 24975 leader_election.cc:290] T ada5a004ba7c46819051e3ccec71ffdb P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:23:08.157508 24989 raft_consensus.cc:493] T af57996a8be84d398f0f023e8d08ef77 P cb82cc7bd6db46d980c75237a1402860 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:23:08.135056 24976 leader_election.cc:290] T 8d4ab870a05347868956aa8b841b2981 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:23:08.168939 24985 raft_consensus.cc:493] T 7c2095a5f4ad49408dbc7f69c606120d P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:23:08.169824 24992 raft_consensus.cc:493] T e4dc6b0541ca431ebc757beac00521f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:23:08.169549 24985 raft_consensus.cc:515] T 7c2095a5f4ad49408dbc7f69c606120d P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:23:08.170392 24992 raft_consensus.cc:515] T e4dc6b0541ca431ebc757beac00521f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:23:08.167090 24976 raft_consensus.cc:493] T 84f2da98864b4425bf7c0055951fdd4b P 90f3da57796b488c9087a2f523adf045 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:23:08.171653 24976 raft_consensus.cc:515] T 84f2da98864b4425bf7c0055951fdd4b P 90f3da57796b488c9087a2f523adf045 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:23:08.171977 24985 leader_election.cc:290] T 7c2095a5f4ad49408dbc7f69c606120d P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:23:08.173658 24976 leader_election.cc:290] T 84f2da98864b4425bf7c0055951fdd4b P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:23:08.166081 24989 raft_consensus.cc:515] T af57996a8be84d398f0f023e8d08ef77 P cb82cc7bd6db46d980c75237a1402860 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:23:08.176211 24986 raft_consensus.cc:493] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac [term 24 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:23:08.176893 24986 raft_consensus.cc:515] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:23:08.189440 24976 raft_consensus.cc:493] T cdbd78acb4cf46298e0de8ba0d130e19 P 90f3da57796b488c9087a2f523adf045 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:23:08.190402 24976 raft_consensus.cc:515] T cdbd78acb4cf46298e0de8ba0d130e19 P 90f3da57796b488c9087a2f523adf045 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:23:08.191820 24996 raft_consensus.cc:493] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [term 30 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:23:08.192572 24996 raft_consensus.cc:515] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:23:08.193781 24976 leader_election.cc:290] T cdbd78acb4cf46298e0de8ba0d130e19 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:23:08.195449 24996 leader_election.cc:290] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:23:08.196838 24972 raft_consensus.cc:493] T c2f5f89186204242bf377f7c42b2e2da P 1ca2725d290e47d68b1a4b609c9816ac [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:23:08.178463 24995 leader_election.cc:290] T 5a959dbecc2648eb85d1cc550f5a3245 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:23:08.204134 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74ca8140f35c4ae29e8a664f6bdfc829" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 27 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:23:08.208666 24981 raft_consensus.cc:493] T 02838c86531d4f98bf58d1c22b8baf77 P 1ca2725d290e47d68b1a4b609c9816ac [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:23:08.208930 24986 leader_election.cc:290] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:23:08.209276 24981 raft_consensus.cc:515] T 02838c86531d4f98bf58d1c22b8baf77 P 1ca2725d290e47d68b1a4b609c9816ac [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:23:08.209585 16292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c74490c40243a88e62d1760b5cd814" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:23:08.197398 24972 raft_consensus.cc:515] T c2f5f89186204242bf377f7c42b2e2da P 1ca2725d290e47d68b1a4b609c9816ac [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:23:08.213615 24972 leader_election.cc:290] T c2f5f89186204242bf377f7c42b2e2da P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:23:08.214831 24968 raft_consensus.cc:493] T fffab26427e24d398e2695aad40ca134 P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:23:08.209709 24993 raft_consensus.cc:493] T 19baa7bc367844c288bca8f81439c413 P 1ca2725d290e47d68b1a4b609c9816ac [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:23:08.215559 24968 raft_consensus.cc:515] T fffab26427e24d398e2695aad40ca134 P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:23:08.216034 24993 raft_consensus.cc:515] T 19baa7bc367844c288bca8f81439c413 P 1ca2725d290e47d68b1a4b609c9816ac [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } attrs { replace: true } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:23:08.218222 24968 leader_election.cc:290] T fffab26427e24d398e2695aad40ca134 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:23:08.223491 16292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c2095a5f4ad49408dbc7f69c606120d" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:23:08.180552 24994 leader_election.cc:290] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:23:08.239814 16821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d4ab870a05347868956aa8b841b2981" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:23:08.238456 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9944d5e4b8c4109af9955c4f08c9095" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 25 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:23:08.244220 16292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f5f89186204242bf377f7c42b2e2da" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:23:08.247786 16821 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada5a004ba7c46819051e3ccec71ffdb" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 26 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:23:08.252976 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fffab26427e24d398e2695aad40ca134" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:23:08.254822 16292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:23:08.261842 24992 leader_election.cc:290] T e4dc6b0541ca431ebc757beac00521f1 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:23:08.262292 24979 raft_consensus.cc:493] T b6f07842d3aa47858df5665874c86f3a P 1ca2725d290e47d68b1a4b609c9816ac [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:23:08.262599 24992 raft_consensus.cc:493] T 8cc959fb7f704fba9009c0595467d074 P 1ca2725d290e47d68b1a4b609c9816ac [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:23:08.262931 24979 raft_consensus.cc:515] T b6f07842d3aa47858df5665874c86f3a P 1ca2725d290e47d68b1a4b609c9816ac [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:23:08.263257 24992 raft_consensus.cc:515] T 8cc959fb7f704fba9009c0595467d074 P 1ca2725d290e47d68b1a4b609c9816ac [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:23:08.263916 24985 raft_consensus.cc:493] T 1df93779209748b9860f6d893dd3de53 P 1ca2725d290e47d68b1a4b609c9816ac [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:23:08.264477 24985 raft_consensus.cc:515] T 1df93779209748b9860f6d893dd3de53 P 1ca2725d290e47d68b1a4b609c9816ac [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:23:08.265643 24992 leader_election.cc:290] T 8cc959fb7f704fba9009c0595467d074 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:23:08.265973 24979 leader_election.cc:290] T b6f07842d3aa47858df5665874c86f3a P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:23:08.266486 24985 leader_election.cc:290] T 1df93779209748b9860f6d893dd3de53 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:23:08.250078 24993 leader_election.cc:290] T 19baa7bc367844c288bca8f81439c413 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:23:08.275527 25000 raft_consensus.cc:493] T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:23:08.275815 24981 leader_election.cc:290] T 02838c86531d4f98bf58d1c22b8baf77 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:23:08.276315 25000 raft_consensus.cc:515] T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:23:08.277396 24989 leader_election.cc:290] T af57996a8be84d398f0f023e8d08ef77 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:23:08.277056 16292 consensus_queue.cc:237] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 25.14, Last appended by leader: 13, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:23:08.278944 25000 leader_election.cc:290] T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:23:08.270888 24998 raft_consensus.cc:493] T 31969392ccdb43b491911a319e8bc7e1 P 90f3da57796b488c9087a2f523adf045 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:23:08.285243 16817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66ae176961b74813a58bc15c9c4c6f2a" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 30 candidate_status { last_received { term: 29 index: 18 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:23:08.286017 24998 raft_consensus.cc:515] T 31969392ccdb43b491911a319e8bc7e1 P 90f3da57796b488c9087a2f523adf045 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:23:08.287351 24976 raft_consensus.cc:493] T e5ad4eb281564936a59f41e387a47156 P 90f3da57796b488c9087a2f523adf045 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:23:08.288242 24976 raft_consensus.cc:515] T e5ad4eb281564936a59f41e387a47156 P 90f3da57796b488c9087a2f523adf045 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:23:08.281636 24968 raft_consensus.cc:493] T 8a878999c0024d2abe517484180290fc P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:23:08.297680 24987 raft_consensus.cc:493] T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:23:08.298408 24987 raft_consensus.cc:515] T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:23:08.301005 24987 leader_election.cc:290] T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:23:08.301206 25002 raft_consensus.cc:493] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:23:08.301793 16824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66f75124215f4ca6bef831e174a792d8" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:23:08.301862 25002 raft_consensus.cc:515] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:23:08.302804 24976 leader_election.cc:290] T e5ad4eb281564936a59f41e387a47156 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:23:08.304533 25002 leader_election.cc:290] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:23:08.305680 24976 raft_consensus.cc:493] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:23:08.306340 24976 raft_consensus.cc:515] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:23:08.308969 24976 leader_election.cc:290] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:23:08.304879 24998 leader_election.cc:290] T 31969392ccdb43b491911a319e8bc7e1 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:23:08.312239 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a959dbecc2648eb85d1cc550f5a3245" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:23:08.312429 24998 raft_consensus.cc:493] T 57a744f5aaf944678a63469cbf514015 P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:23:08.313122 24998 raft_consensus.cc:515] T 57a744f5aaf944678a63469cbf514015 P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:23:08.295910 24968 raft_consensus.cc:515] T 8a878999c0024d2abe517484180290fc P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:23:08.317404 24996 raft_consensus.cc:493] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:23:08.317865 25002 raft_consensus.cc:493] T 8d4ab870a05347868956aa8b841b2981 P cb82cc7bd6db46d980c75237a1402860 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:23:08.318059 24984 raft_consensus.cc:493] T ee9ed67006144797b1de6ea8c2013a68 P cb82cc7bd6db46d980c75237a1402860 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:23:08.318607 24998 leader_election.cc:290] T 57a744f5aaf944678a63469cbf514015 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:23:08.318544 16824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdbd78acb4cf46298e0de8ba0d130e19" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 28 candidate_status { last_received { term: 27 index: 20 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:23:08.318712 16825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74ca8140f35c4ae29e8a664f6bdfc829" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 27 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:23:08.320724 16816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31969392ccdb43b491911a319e8bc7e1" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:23:08.321024 16822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "806d7690c2b644959152074bb2cab0a7" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 25 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:23:08.321736 16819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57a744f5aaf944678a63469cbf514015" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:23:08.322122 24968 leader_election.cc:290] T 8a878999c0024d2abe517484180290fc P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:23:08.332592 25001 raft_consensus.cc:493] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:23:08.334684 25001 raft_consensus.cc:515] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:23:08.322213 24996 raft_consensus.cc:515] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:23:08.337352 25001 leader_election.cc:290] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
W20260709 13:23:08.337746 16790 leader_election.cc:336] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 31 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111)
I20260709 13:23:08.338366 24996 leader_election.cc:290] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
W20260709 13:23:08.338470 16790 leader_election.cc:336] T af57996a8be84d398f0f023e8d08ef77 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111)
W20260709 13:23:08.382676 16190 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 124365us
W20260709 13:23:08.416508 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 77937.7us
I20260709 13:23:08.394544 24979 raft_consensus.cc:493] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 1ca2725d290e47d68b1a4b609c9816ac [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:23:08.417724 24979 raft_consensus.cc:515] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 1ca2725d290e47d68b1a4b609c9816ac [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:23:08.421900 24979 leader_election.cc:290] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:23:08.327339 24995 raft_consensus.cc:493] T 8c11074579b046d99e585c2818e18a2c P cb82cc7bd6db46d980c75237a1402860 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:23:08.423223 24995 raft_consensus.cc:515] T 8c11074579b046d99e585c2818e18a2c P cb82cc7bd6db46d980c75237a1402860 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:23:08.426617 24995 leader_election.cc:290] T 8c11074579b046d99e585c2818e18a2c P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:23:08.428398 16824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9944d5e4b8c4109af9955c4f08c9095" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 25 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:23:08.400038 24974 raft_consensus.cc:493] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:23:08.429744 24974 raft_consensus.cc:515] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:23:08.431042 16817 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6f07842d3aa47858df5665874c86f3a" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 27 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:23:08.432533 16818 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19baa7bc367844c288bca8f81439c413" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:23:08.432828 24974 leader_election.cc:290] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:23:08.323197 24984 raft_consensus.cc:515] T ee9ed67006144797b1de6ea8c2013a68 P cb82cc7bd6db46d980c75237a1402860 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:23:08.436002 24984 leader_election.cc:290] T ee9ed67006144797b1de6ea8c2013a68 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:23:08.324481 25002 raft_consensus.cc:515] T 8d4ab870a05347868956aa8b841b2981 P cb82cc7bd6db46d980c75237a1402860 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:23:08.403380 16816 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c2095a5f4ad49408dbc7f69c606120d" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:23:08.407648 16822 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74ca8140f35c4ae29e8a664f6bdfc829" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 27 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:23:08.415489 16825 raft_consensus.cc:1275] T 19baa7bc367844c288bca8f81439c413 P cb82cc7bd6db46d980c75237a1402860 [term 25 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 25 index: 14. Preceding OpId from leader: term: 25 index: 15. (index mismatch)
I20260709 13:23:08.440274 25002 leader_election.cc:290] T 8d4ab870a05347868956aa8b841b2981 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
W20260709 13:23:08.390173 16265 leader_election.cc:336] T 2794af5b99984efebdee564035841cb0 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111)
I20260709 13:23:08.449483 16292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d4ab870a05347868956aa8b841b2981" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:23:08.449138 24998 consensus_queue.cc:1048] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:23:08.450734 16219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c11074579b046d99e585c2818e18a2c" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:23:08.451741 16296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4dc6b0541ca431ebc757beac00521f1" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:23:08.449393 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a378d7f796f94615b3b347119a9880fd" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
W20260709 13:23:08.453431 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 74838.2us
I20260709 13:23:08.453807 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c11074579b046d99e585c2818e18a2c" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:23:08.457062 16296 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cc959fb7f704fba9009c0595467d074" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:23:08.458287 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02838c86531d4f98bf58d1c22b8baf77" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 26 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:23:08.458833 16292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1df93779209748b9860f6d893dd3de53" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:23:08.461699 24994 raft_consensus.cc:493] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac [term 24 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:23:08.462575 24994 raft_consensus.cc:515] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:23:08.463080 24982 raft_consensus.cc:493] T ecc1e363230043fe9fef390f0b979f20 P 1ca2725d290e47d68b1a4b609c9816ac [term 27 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:23:08.464105 24982 raft_consensus.cc:515] T ecc1e363230043fe9fef390f0b979f20 P 1ca2725d290e47d68b1a4b609c9816ac [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:23:08.466022 24994 leader_election.cc:290] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:23:08.467017 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6f07842d3aa47858df5665874c86f3a" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 27 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:23:08.467564 24982 leader_election.cc:290] T ecc1e363230043fe9fef390f0b979f20 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:23:08.469086 16294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c61a39fd4f84424b129c1a6f9ba404d" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 27 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:23:08.471144 16294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a878999c0024d2abe517484180290fc" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:23:08.475425 16294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ad5dbfa99ee4b8d9e28be92b3f36db5" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:23:08.475425 16293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87ac0d1f7e8a419f9bcbc04aee0e067e" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:23:08.474004 16265 leader_election.cc:304] T 2794af5b99984efebdee564035841cb0 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
I20260709 13:23:08.487721 24998 raft_consensus.cc:2749] T 2794af5b99984efebdee564035841cb0 P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:23:08.489418 16223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0eab25ce4c64e018a041beeda2867db" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
W20260709 13:23:08.482007 16790 leader_election.cc:336] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111)
W20260709 13:23:08.493813 16790 consensus_peers.cc:597] T 9e65676b21b748728ad39668cfe722ec P cb82cc7bd6db46d980c75237a1402860 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
I20260709 13:23:08.487982 24994 raft_consensus.cc:493] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac [term 24 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:23:08.495916 24994 raft_consensus.cc:515] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:23:08.498533 24994 leader_election.cc:290] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
W20260709 13:23:08.501554 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 111726us
I20260709 13:23:08.504293 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8158bbd17b994d2eb96f560bd0f1b88d" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:23:08.508369 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d4ab870a05347868956aa8b841b2981" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:23:08.509764 16219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdbd78acb4cf46298e0de8ba0d130e19" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 28 candidate_status { last_received { term: 27 index: 20 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:23:08.510243 16220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f2da98864b4425bf7c0055951fdd4b" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:23:08.511759 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5ad4eb281564936a59f41e387a47156" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:23:08.514195 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a959dbecc2648eb85d1cc550f5a3245" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:23:08.508400 16221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada5a004ba7c46819051e3ccec71ffdb" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 26 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:23:08.517272 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74ca8140f35c4ae29e8a664f6bdfc829" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 27 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:23:08.517987 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af57996a8be84d398f0f023e8d08ef77" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:23:08.518879 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62fe3346cd3c42c4a55e14af9073bca4" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 31 candidate_status { last_received { term: 30 index: 18 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:23:08.520972 16820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d5d59407eb545b895d8d922e0dd69fd" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 25 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:23:08.521204 16824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecc1e363230043fe9fef390f0b979f20" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 28 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:23:08.521507 16825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5718c623ebee4d2bb770f0f9ddb2f164" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 25 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:23:08.522630 16224 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08b7d1ab519044eda2d5e04a07242f25" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 25 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:23:08.525590 16217 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2372981006d4b218215c28964dea623" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 28 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:23:08.527199 24994 raft_consensus.cc:493] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [term 27 FOLLOWER]: Starting pre-election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:23:08.526932 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746aabf856ab401c9ef5e859371e7b67" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 21 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:23:08.526278 16221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c11074579b046d99e585c2818e18a2c" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
W20260709 13:23:08.535511 16265 consensus_peers.cc:597] T 8a878999c0024d2abe517484180290fc P 90f3da57796b488c9087a2f523adf045 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:23:08.536319 16265 leader_election.cc:336] T 3884f864a96d428785895e5463e5728f P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111)
I20260709 13:23:08.527981 24994 raft_consensus.cc:515] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:23:08.536815 16265 leader_election.cc:304] T 3884f864a96d428785895e5463e5728f P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
I20260709 13:23:08.537375 16219 raft_consensus.cc:2468] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 27.
I20260709 13:23:08.537704 24998 raft_consensus.cc:2749] T 3884f864a96d428785895e5463e5728f P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:23:08.540827 16265 leader_election.cc:336] T c0eab25ce4c64e018a041beeda2867db P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111)
W20260709 13:23:08.541759 16265 leader_election.cc:336] T 9e65676b21b748728ad39668cfe722ec P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111)
W20260709 13:23:08.541785 16264 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:23:08.542323 16265 leader_election.cc:304] T 9e65676b21b748728ad39668cfe722ec P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
W20260709 13:23:08.546423 16191 leader_election.cc:336] T 66f75124215f4ca6bef831e174a792d8 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111)
W20260709 13:23:08.547509 16191 leader_election.cc:336] T 08c74490c40243a88e62d1760b5cd814 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111)
I20260709 13:23:08.548026 16191 leader_election.cc:304] T 08c74490c40243a88e62d1760b5cd814 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:23:08.549033 24979 raft_consensus.cc:2749] T 08c74490c40243a88e62d1760b5cd814 P 1ca2725d290e47d68b1a4b609c9816ac [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 13:23:08.549949 16790 consensus_peers.cc:597] T 2794af5b99984efebdee564035841cb0 P cb82cc7bd6db46d980c75237a1402860 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 66: this message will repeat every 5th retry.
W20260709 13:23:08.555593 16790 leader_election.cc:336] T 5a959dbecc2648eb85d1cc550f5a3245 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111)
I20260709 13:23:08.556209 16790 leader_election.cc:304] T 5a959dbecc2648eb85d1cc550f5a3245 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
W20260709 13:23:08.551154 16191 leader_election.cc:336] T c2f5f89186204242bf377f7c42b2e2da P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111)
I20260709 13:23:08.551524 24994 leader_election.cc:290] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:23:08.557289 16191 leader_election.cc:304] T c2f5f89186204242bf377f7c42b2e2da P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:23:08.558209 24994 raft_consensus.cc:2749] T c2f5f89186204242bf377f7c42b2e2da P 1ca2725d290e47d68b1a4b609c9816ac [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
W20260709 13:23:08.558830 16191 leader_election.cc:336] T fffab26427e24d398e2695aad40ca134 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111)
I20260709 13:23:08.559456 16191 leader_election.cc:304] T fffab26427e24d398e2695aad40ca134 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:23:08.560268 24994 raft_consensus.cc:2749] T fffab26427e24d398e2695aad40ca134 P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:23:08.560827 16191 leader_election.cc:336] T 19baa7bc367844c288bca8f81439c413 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111)
I20260709 13:23:08.561313 16191 leader_election.cc:304] T 19baa7bc367844c288bca8f81439c413 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
W20260709 13:23:08.562108 16191 leader_election.cc:336] T 02838c86531d4f98bf58d1c22b8baf77 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111)
I20260709 13:23:08.563462 25002 raft_consensus.cc:2749] T 5a959dbecc2648eb85d1cc550f5a3245 P cb82cc7bd6db46d980c75237a1402860 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:23:08.562127 24994 raft_consensus.cc:2749] T 19baa7bc367844c288bca8f81439c413 P 1ca2725d290e47d68b1a4b609c9816ac [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
W20260709 13:23:08.570200 16790 leader_election.cc:336] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111)
I20260709 13:23:08.562711 16191 leader_election.cc:304] T 02838c86531d4f98bf58d1c22b8baf77 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
W20260709 13:23:08.570959 16790 leader_election.cc:336] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111)
W20260709 13:23:08.580775 16191 leader_election.cc:336] T e4dc6b0541ca431ebc757beac00521f1 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111)
I20260709 13:23:08.586715 16191 leader_election.cc:304] T e4dc6b0541ca431ebc757beac00521f1 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
W20260709 13:23:08.582495 16192 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.132:38923, user_credentials={real_user=slave}} blocked reactor thread for 67231.8us
W20260709 13:23:08.588532 16191 leader_election.cc:336] T b6f07842d3aa47858df5665874c86f3a P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111)
I20260709 13:23:08.588788 16192 leader_election.cc:304] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860
I20260709 13:23:08.589907 24994 raft_consensus.cc:2749] T e4dc6b0541ca431ebc757beac00521f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:23:08.590806 24994 raft_consensus.cc:2749] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 13:23:08.590389 24977 raft_consensus.cc:2955] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045 [term 25 LEADER]: Committing config change with OpId 25.15: config changed from index 7 to 15, VOTER 647a343a542d4a139fe106c19095129b (127.14.177.129) evicted. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:23:08.583276 24998 raft_consensus.cc:2749] T 9e65676b21b748728ad39668cfe722ec P 90f3da57796b488c9087a2f523adf045 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:23:08.594101 16294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdbd78acb4cf46298e0de8ba0d130e19" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 28 candidate_status { last_received { term: 27 index: 20 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:23:08.597502 16819 raft_consensus.cc:2955] T 19baa7bc367844c288bca8f81439c413 P cb82cc7bd6db46d980c75237a1402860 [term 25 FOLLOWER]: Committing config change with OpId 25.15: config changed from index 7 to 15, VOTER 647a343a542d4a139fe106c19095129b (127.14.177.129) evicted. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:23:08.589233 16191 leader_election.cc:304] T b6f07842d3aa47858df5665874c86f3a P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
W20260709 13:23:08.584331 16265 leader_election.cc:336] T 8158bbd17b994d2eb96f560bd0f1b88d P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111)
I20260709 13:23:08.586105 16790 leader_election.cc:304] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:23:08.581089 24974 raft_consensus.cc:2749] T 02838c86531d4f98bf58d1c22b8baf77 P 1ca2725d290e47d68b1a4b609c9816ac [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:23:08.609885 25002 raft_consensus.cc:2749] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
I20260709 13:23:08.609522 16790 leader_election.cc:304] T af57996a8be84d398f0f023e8d08ef77 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:23:08.612865 24995 raft_consensus.cc:2749] T af57996a8be84d398f0f023e8d08ef77 P cb82cc7bd6db46d980c75237a1402860 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:23:08.612936 16790 leader_election.cc:304] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:23:08.614176 24995 raft_consensus.cc:2749] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:23:08.615732 16790 leader_election.cc:304] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:23:08.616443 24995 raft_consensus.cc:2749] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 13:23:08.621037 16790 leader_election.cc:304] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:23:08.621826 24995 raft_consensus.cc:2749] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:23:08.623276 16790 leader_election.cc:304] T 8c11074579b046d99e585c2818e18a2c P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:23:08.623546 24974 raft_consensus.cc:2749] T b6f07842d3aa47858df5665874c86f3a P 1ca2725d290e47d68b1a4b609c9816ac [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 13:23:08.624038 24995 raft_consensus.cc:2749] T 8c11074579b046d99e585c2818e18a2c P cb82cc7bd6db46d980c75237a1402860 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:23:08.624455 16191 leader_election.cc:336] T 8cc959fb7f704fba9009c0595467d074 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111)
I20260709 13:23:08.625010 16191 leader_election.cc:304] T 8cc959fb7f704fba9009c0595467d074 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:23:08.625936 24974 raft_consensus.cc:2749] T 8cc959fb7f704fba9009c0595467d074 P 1ca2725d290e47d68b1a4b609c9816ac [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:23:08.626228 16191 leader_election.cc:336] T 1df93779209748b9860f6d893dd3de53 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111)
I20260709 13:23:08.626730 16191 leader_election.cc:304] T 1df93779209748b9860f6d893dd3de53 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
W20260709 13:23:08.636691 16790 leader_election.cc:336] T ee9ed67006144797b1de6ea8c2013a68 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111)
W20260709 13:23:08.654665 16790 consensus_peers.cc:597] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
I20260709 13:23:08.659150 16044 catalog_manager.cc:5697] T 19baa7bc367844c288bca8f81439c413 P cb82cc7bd6db46d980c75237a1402860 reported cstate change: config changed from index 7 to 15, VOTER 647a343a542d4a139fe106c19095129b (127.14.177.129) evicted. New cstate: current_term: 25 leader_uuid: "90f3da57796b488c9087a2f523adf045" committed_config { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
I20260709 13:23:08.663949 25007 raft_consensus.cc:2749] T 1df93779209748b9860f6d893dd3de53 P 1ca2725d290e47d68b1a4b609c9816ac [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:23:08.670593 16192 leader_election.cc:304] T 66f75124215f4ca6bef831e174a792d8 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
I20260709 13:23:08.672900 24979 raft_consensus.cc:2749] T 66f75124215f4ca6bef831e174a792d8 P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:23:08.675602 16192 leader_election.cc:304] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860
I20260709 13:23:08.676595 24982 raft_consensus.cc:2749] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:23:08.682246 16034 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 19baa7bc367844c288bca8f81439c413 with cas_config_opid_index 7: ChangeConfig:REMOVE_PEER succeeded (attempt 13)
W20260709 13:23:08.685436 16265 leader_election.cc:336] T 84f2da98864b4425bf7c0055951fdd4b P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111)
W20260709 13:23:08.666050 16191 leader_election.cc:336] T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111)
I20260709 13:23:08.692231 16191 leader_election.cc:304] T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
W20260709 13:23:08.693399 16265 leader_election.cc:336] T e5ad4eb281564936a59f41e387a47156 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111)
I20260709 13:23:08.699371 24982 raft_consensus.cc:2749] T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 13:23:08.702283 16192 leader_election.cc:304] T 7c2095a5f4ad49408dbc7f69c606120d P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860
I20260709 13:23:08.703110 24982 raft_consensus.cc:2749] T 7c2095a5f4ad49408dbc7f69c606120d P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:23:08.699025 16265 leader_election.cc:336] T 31969392ccdb43b491911a319e8bc7e1 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111)
I20260709 13:23:08.704670 16192 leader_election.cc:304] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860
I20260709 13:23:08.704928 16265 leader_election.cc:304] T 31969392ccdb43b491911a319e8bc7e1 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
I20260709 13:23:08.705637 24982 raft_consensus.cc:2749] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 13:23:08.706507 24977 raft_consensus.cc:2749] T 31969392ccdb43b491911a319e8bc7e1 P 90f3da57796b488c9087a2f523adf045 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 13:23:08.710211 16265 leader_election.cc:336] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111)
I20260709 13:23:08.710883 16265 leader_election.cc:304] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
I20260709 13:23:08.715720 16819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdbd78acb4cf46298e0de8ba0d130e19" candidate_uuid: "1ca2725d290e47d68b1a4b609c9816ac" candidate_term: 28 candidate_status { last_received { term: 27 index: 20 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:23:08.717044 24977 raft_consensus.cc:2749] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 13:23:08.714042 16790 consensus_peers.cc:597] T 23f82224f4564fea93572e8206bad1fe P cb82cc7bd6db46d980c75237a1402860 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 66: this message will repeat every 5th retry.
W20260709 13:23:08.725411 16265 consensus_peers.cc:597] T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
I20260709 13:23:08.749307 16223 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee9ed67006144797b1de6ea8c2013a68" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
W20260709 13:23:08.750562 16191 leader_election.cc:336] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111)
I20260709 13:23:08.750685 16219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d4ab870a05347868956aa8b841b2981" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:23:08.768625 16223 raft_consensus.cc:1275] T 19baa7bc367844c288bca8f81439c413 P 1ca2725d290e47d68b1a4b609c9816ac [term 25 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 25 index: 14. Preceding OpId from leader: term: 25 index: 15. (index mismatch)
I20260709 13:23:08.771910 16265 leader_election.cc:304] T 8c11074579b046d99e585c2818e18a2c P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860
I20260709 13:23:08.772796 24977 raft_consensus.cc:2749] T 8c11074579b046d99e585c2818e18a2c P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:23:08.774524 16265 leader_election.cc:304] T a378d7f796f94615b3b347119a9880fd P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860
I20260709 13:23:08.751061 16191 leader_election.cc:304] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
I20260709 13:23:08.775840 24994 raft_consensus.cc:2749] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 13:23:08.776947 16191 leader_election.cc:336] T 8a878999c0024d2abe517484180290fc P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111)
I20260709 13:23:08.777379 16191 leader_election.cc:304] T 8a878999c0024d2abe517484180290fc P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:23:08.778097 24994 raft_consensus.cc:2749] T 8a878999c0024d2abe517484180290fc P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:23:08.779227 16191 leader_election.cc:336] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111)
I20260709 13:23:08.779640 16191 leader_election.cc:304] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:23:08.780303 24994 raft_consensus.cc:2749] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 1ca2725d290e47d68b1a4b609c9816ac [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:23:08.781296 16192 leader_election.cc:304] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860
I20260709 13:23:08.782225 24994 raft_consensus.cc:2749] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 13:23:08.782946 24977 raft_consensus.cc:2749] T a378d7f796f94615b3b347119a9880fd P 90f3da57796b488c9087a2f523adf045 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 13:23:08.781322 16191 leader_election.cc:336] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111)
I20260709 13:23:08.785132 16191 leader_election.cc:304] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:23:08.788183 16265 leader_election.cc:304] T c0eab25ce4c64e018a041beeda2867db P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:23:08.792923 24994 raft_consensus.cc:2749] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:23:08.794173 16191 leader_election.cc:336] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111)
I20260709 13:23:08.794636 16191 leader_election.cc:304] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
I20260709 13:23:08.795364 24977 raft_consensus.cc:2749] T c0eab25ce4c64e018a041beeda2867db P 90f3da57796b488c9087a2f523adf045 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:23:08.797056 16265 leader_election.cc:304] T 8158bbd17b994d2eb96f560bd0f1b88d P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:23:08.798247 25007 raft_consensus.cc:2749] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:23:08.798849 24977 raft_consensus.cc:2749] T 8158bbd17b994d2eb96f560bd0f1b88d P 90f3da57796b488c9087a2f523adf045 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 13:23:08.798367 16191 leader_election.cc:336] T ecc1e363230043fe9fef390f0b979f20 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111)
I20260709 13:23:08.799854 16191 leader_election.cc:304] T ecc1e363230043fe9fef390f0b979f20 P 1ca2725d290e47d68b1a4b609c9816ac [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
I20260709 13:23:08.800900 24994 raft_consensus.cc:2749] T ecc1e363230043fe9fef390f0b979f20 P 1ca2725d290e47d68b1a4b609c9816ac [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
W20260709 13:23:08.802098 16191 consensus_peers.cc:597] T 746aabf856ab401c9ef5e859371e7b67 P 1ca2725d290e47d68b1a4b609c9816ac -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:23:08.802603 16191 consensus_peers.cc:597] T ee9ed67006144797b1de6ea8c2013a68 P 1ca2725d290e47d68b1a4b609c9816ac -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:23:08.803077 16191 consensus_peers.cc:597] T 5a96373bb0e14dce8a266b2fc6c58507 P 1ca2725d290e47d68b1a4b609c9816ac -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:23:08.811936 16790 consensus_peers.cc:597] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:23:08.826093 16035 catalog_manager.cc:4755] TS 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet 19baa7bc367844c288bca8f81439c413: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111)
I20260709 13:23:08.826527 16265 leader_election.cc:304] T 8d4ab870a05347868956aa8b841b2981 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860
I20260709 13:23:08.834847 16790 leader_election.cc:304] T ee9ed67006144797b1de6ea8c2013a68 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:23:08.835242 24977 raft_consensus.cc:2749] T 8d4ab870a05347868956aa8b841b2981 P 90f3da57796b488c9087a2f523adf045 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:23:08.835732 24995 raft_consensus.cc:2749] T ee9ed67006144797b1de6ea8c2013a68 P cb82cc7bd6db46d980c75237a1402860 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:23:08.837196 16265 leader_election.cc:304] T 84f2da98864b4425bf7c0055951fdd4b P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:23:08.838547 16790 leader_election.cc:304] T 8d4ab870a05347868956aa8b841b2981 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:23:08.839046 24977 raft_consensus.cc:2749] T 84f2da98864b4425bf7c0055951fdd4b P 90f3da57796b488c9087a2f523adf045 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:23:08.841135 16265 leader_election.cc:304] T e5ad4eb281564936a59f41e387a47156 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:23:08.841043 24989 raft_consensus.cc:2749] T 8d4ab870a05347868956aa8b841b2981 P cb82cc7bd6db46d980c75237a1402860 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:23:08.843464 24977 raft_consensus.cc:2749] T e5ad4eb281564936a59f41e387a47156 P 90f3da57796b488c9087a2f523adf045 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:23:08.844208 16265 leader_election.cc:304] T ada5a004ba7c46819051e3ccec71ffdb P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860
I20260709 13:23:08.844996 24977 raft_consensus.cc:2749] T ada5a004ba7c46819051e3ccec71ffdb P 90f3da57796b488c9087a2f523adf045 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:23:08.845914 16265 leader_election.cc:304] T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860
I20260709 13:23:08.847347 24977 raft_consensus.cc:2749] T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 13:23:08.848058 16265 leader_election.cc:304] T cdbd78acb4cf46298e0de8ba0d130e19 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045; no voters: cb82cc7bd6db46d980c75237a1402860
I20260709 13:23:08.848905 24977 raft_consensus.cc:2804] T cdbd78acb4cf46298e0de8ba0d130e19 P 90f3da57796b488c9087a2f523adf045 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 13:23:08.849355 24977 raft_consensus.cc:493] T cdbd78acb4cf46298e0de8ba0d130e19 P 90f3da57796b488c9087a2f523adf045 [term 27 FOLLOWER]: Starting leader election (detected failure of leader cb82cc7bd6db46d980c75237a1402860)
I20260709 13:23:08.849718 24977 raft_consensus.cc:3060] T cdbd78acb4cf46298e0de8ba0d130e19 P 90f3da57796b488c9087a2f523adf045 [term 27 FOLLOWER]: Advancing to term 28
W20260709 13:23:08.852387 16265 leader_election.cc:336] T 57a744f5aaf944678a63469cbf514015 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111)
I20260709 13:23:08.852382 24987 consensus_queue.cc:1048] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:23:08.853122 16265 leader_election.cc:304] T 57a744f5aaf944678a63469cbf514015 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 647a343a542d4a139fe106c19095129b, cb82cc7bd6db46d980c75237a1402860
I20260709 13:23:08.853909 24998 raft_consensus.cc:2749] T 57a744f5aaf944678a63469cbf514015 P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:23:08.862701 24977 raft_consensus.cc:515] T cdbd78acb4cf46298e0de8ba0d130e19 P 90f3da57796b488c9087a2f523adf045 [term 28 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:23:08.864863 24977 leader_election.cc:290] T cdbd78acb4cf46298e0de8ba0d130e19 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 28 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:23:08.865479 16219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdbd78acb4cf46298e0de8ba0d130e19" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 28 candidate_status { last_received { term: 27 index: 20 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:23:08.866446 16819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdbd78acb4cf46298e0de8ba0d130e19" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 28 candidate_status { last_received { term: 27 index: 20 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:23:08.876847 16292 raft_consensus.cc:1240] T cdbd78acb4cf46298e0de8ba0d130e19 P 90f3da57796b488c9087a2f523adf045 [term 28 FOLLOWER]: Rejecting Update request from peer cb82cc7bd6db46d980c75237a1402860 for earlier term 27. Current term is 28. Ops: []
I20260709 13:23:08.879269 16265 leader_election.cc:304] T cdbd78acb4cf46298e0de8ba0d130e19 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 28 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860
I20260709 13:23:08.881515 24977 raft_consensus.cc:2749] T cdbd78acb4cf46298e0de8ba0d130e19 P 90f3da57796b488c9087a2f523adf045 [term 28 FOLLOWER]: Leader election lost for term 28. Reason: could not achieve majority
I20260709 13:23:08.881470 24984 consensus_queue.cc:1059] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: INVALID_TERM, Last received: 27.20, Next index: 21, Last known committed idx: 20, Time since last communication: 0.001s
I20260709 13:23:08.882974 24984 raft_consensus.cc:3055] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 [term 27 LEADER]: Stepping down as leader of term 27
I20260709 13:23:08.883383 24984 raft_consensus.cc:740] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 [term 27 LEADER]: Becoming Follower/Learner. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:23:08.884636 24984 consensus_queue.cc:260] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 27.20, Last appended by leader: 20, Current term: 27, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:23:08.888970 24984 raft_consensus.cc:3060] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 [term 27 FOLLOWER]: Advancing to term 28
W20260709 13:23:08.912606 15046 thread.cc:527] Waited for 24000ms trying to join with diag-logger (tid 16172)
I20260709 13:23:08.906265 16223 raft_consensus.cc:2955] T 19baa7bc367844c288bca8f81439c413 P 1ca2725d290e47d68b1a4b609c9816ac [term 25 FOLLOWER]: Committing config change with OpId 25.15: config changed from index 7 to 15, VOTER 647a343a542d4a139fe106c19095129b (127.14.177.129) evicted. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } } }
W20260709 13:23:08.994462 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 82382us
W20260709 13:23:09.007669 16265 consensus_peers.cc:597] T fffab26427e24d398e2695aad40ca134 P 90f3da57796b488c9087a2f523adf045 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 13:23:09.028980 16846 debug-util.cc:398] Leaking SignalData structure 0x7b0800204780 after lost signal to thread 15049
W20260709 13:23:09.030627 16846 debug-util.cc:398] Leaking SignalData structure 0x7b08002d7900 after lost signal to thread 16083
W20260709 13:23:09.032670 16846 debug-util.cc:398] Leaking SignalData structure 0x7b08002be7e0 after lost signal to thread 16250
W20260709 13:23:09.034235 16846 debug-util.cc:398] Leaking SignalData structure 0x7b0800386a40 after lost signal to thread 16324
W20260709 13:23:09.036125 16846 debug-util.cc:398] Leaking SignalData structure 0x7b08003ed920 after lost signal to thread 16849
W20260709 13:23:09.040808 16265 consensus_peers.cc:597] T 434bff3082d141838068f323bdb0d1ed P 90f3da57796b488c9087a2f523adf045 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 13:23:09.053390 16265 consensus_peers.cc:597] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 13:23:09.079654 16265 consensus_peers.cc:597] T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:23:09.121507 16265 consensus_peers.cc:597] T 02838c86531d4f98bf58d1c22b8baf77 P 90f3da57796b488c9087a2f523adf045 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:23:09.123500 16191 consensus_peers.cc:597] T 68a6838d82ef4cac83507f906139d90f P 1ca2725d290e47d68b1a4b609c9816ac -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:23:09.172976 16790 consensus_peers.cc:597] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:23:09.210525 16265 consensus_peers.cc:597] T 8cc959fb7f704fba9009c0595467d074 P 90f3da57796b488c9087a2f523adf045 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:23:09.215200 16191 consensus_peers.cc:597] T af57996a8be84d398f0f023e8d08ef77 P 1ca2725d290e47d68b1a4b609c9816ac -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:23:09.233227 16265 consensus_peers.cc:597] T b87147731a68400e8cce9b879daa7c5d P 90f3da57796b488c9087a2f523adf045 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 13:23:09.234031 16191 consensus_peers.cc:597] T 84f2da98864b4425bf7c0055951fdd4b P 1ca2725d290e47d68b1a4b609c9816ac -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:23:09.267025 16790 consensus_peers.cc:597] T 806d7690c2b644959152074bb2cab0a7 P cb82cc7bd6db46d980c75237a1402860 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:23:09.319054 16265 consensus_peers.cc:597] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:23:09.330222 16191 consensus_peers.cc:597] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:23:09.373489 16191 consensus_peers.cc:597] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:23:09.374002 16191 consensus_peers.cc:597] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:23:09.434621 16265 consensus_peers.cc:597] T 1df93779209748b9860f6d893dd3de53 P 90f3da57796b488c9087a2f523adf045 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:23:09.521526 16790 consensus_peers.cc:597] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 13:23:09.543601 16265 consensus_peers.cc:597] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:23:09.566227 16265 consensus_peers.cc:597] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 90f3da57796b488c9087a2f523adf045 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 13:23:09.653128 16790 consensus_peers.cc:597] T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 66: this message will repeat every 5th retry.
W20260709 13:23:09.733639 16790 consensus_peers.cc:597] T 66f75124215f4ca6bef831e174a792d8 P cb82cc7bd6db46d980c75237a1402860 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 13:23:09.818804 16191 consensus_peers.cc:597] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:23:09.969172 16790 consensus_peers.cc:597] T 3884f864a96d428785895e5463e5728f P cb82cc7bd6db46d980c75237a1402860 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 66: this message will repeat every 5th retry.
W20260709 13:23:10.003151 15046 thread.cc:527] Waited for 25000ms trying to join with diag-logger (tid 16172)
I20260709 13:23:10.016320 24984 raft_consensus.cc:493] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:23:10.017202 24984 raft_consensus.cc:515] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:23:10.052958 16265 consensus_peers.cc:597] T e4dc6b0541ca431ebc757beac00521f1 P 90f3da57796b488c9087a2f523adf045 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
I20260709 13:23:10.059085 16294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdbd78acb4cf46298e0de8ba0d130e19" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 29 candidate_status { last_received { term: 27 index: 20 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:23:10.060537 16294 raft_consensus.cc:2468] T cdbd78acb4cf46298e0de8ba0d130e19 P 90f3da57796b488c9087a2f523adf045 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 28.
I20260709 13:23:10.066754 16789 leader_election.cc:304] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:23:10.067919 24989 raft_consensus.cc:2804] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 13:23:10.068419 24989 raft_consensus.cc:493] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 [term 28 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:23:10.068810 24989 raft_consensus.cc:3060] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 [term 28 FOLLOWER]: Advancing to term 29
I20260709 13:23:10.173537 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdbd78acb4cf46298e0de8ba0d130e19" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 29 candidate_status { last_received { term: 27 index: 20 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac" is_pre_election: true
I20260709 13:23:10.173877 24984 leader_election.cc:290] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:23:10.174324 16225 raft_consensus.cc:2468] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 27.
W20260709 13:23:10.216217 16265 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 127231us
I20260709 13:23:10.238888 24989 raft_consensus.cc:515] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 [term 29 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:23:10.241089 16191 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 67385.9us
W20260709 13:23:10.269800 16790 consensus_peers.cc:597] T 57a744f5aaf944678a63469cbf514015 P cb82cc7bd6db46d980c75237a1402860 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 66: this message will repeat every 5th retry.
W20260709 13:23:10.361217 16790 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 91469us
W20260709 13:23:10.393278 16035 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 272175us
I20260709 13:23:10.405697 24989 leader_election.cc:290] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 29 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
W20260709 13:23:10.406256 16265 consensus_peers.cc:597] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 90f3da57796b488c9087a2f523adf045 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:23:10.407434 16265 consensus_peers.cc:597] T 26354361e9b044cea1c2b86c4882fe92 P 90f3da57796b488c9087a2f523adf045 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 13:23:10.414160 16035 catalog_manager.cc:4755] TS 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet 19baa7bc367844c288bca8f81439c413: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111)
I20260709 13:23:10.420866 16292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdbd78acb4cf46298e0de8ba0d130e19" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 29 candidate_status { last_received { term: 27 index: 20 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:23:10.421537 16292 raft_consensus.cc:3060] T cdbd78acb4cf46298e0de8ba0d130e19 P 90f3da57796b488c9087a2f523adf045 [term 28 FOLLOWER]: Advancing to term 29
I20260709 13:23:10.454082 16292 raft_consensus.cc:2468] T cdbd78acb4cf46298e0de8ba0d130e19 P 90f3da57796b488c9087a2f523adf045 [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 29.
I20260709 13:23:10.470825 16789 leader_election.cc:304] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 29 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:23:10.475332 24989 raft_consensus.cc:2804] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 [term 29 FOLLOWER]: Leader election won for term 29
I20260709 13:23:10.478065 24989 raft_consensus.cc:697] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 [term 29 LEADER]: Becoming Leader. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:23:10.479302 24989 consensus_queue.cc:237] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 27.20, Last appended by leader: 20, Current term: 29, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:23:10.487385 16225 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdbd78acb4cf46298e0de8ba0d130e19" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 29 candidate_status { last_received { term: 27 index: 20 } } ignore_live_leader: false dest_uuid: "1ca2725d290e47d68b1a4b609c9816ac"
I20260709 13:23:10.488101 16225 raft_consensus.cc:3060] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [term 27 FOLLOWER]: Advancing to term 29
I20260709 13:23:10.497740 16225 raft_consensus.cc:2468] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 29.
W20260709 13:23:10.499712 16191 consensus_peers.cc:597] T 829b4ad8c5a54eababeea1181b960d4b P 1ca2725d290e47d68b1a4b609c9816ac -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 13:23:10.499858 16265 consensus_peers.cc:597] T 8a878999c0024d2abe517484180290fc P 90f3da57796b488c9087a2f523adf045 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
I20260709 13:23:10.520495 16044 catalog_manager.cc:5697] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 reported cstate change: term changed from 27 to 29. New cstate: current_term: 29 leader_uuid: "cb82cc7bd6db46d980c75237a1402860" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:23:10.526571 16191 consensus_peers.cc:597] T 5a959dbecc2648eb85d1cc550f5a3245 P 1ca2725d290e47d68b1a4b609c9816ac -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:23:10.527282 16191 consensus_peers.cc:597] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 13:23:10.527771 16191 consensus_peers.cc:597] T d16dfad0c3b540e5b4b86492ddf87208 P 1ca2725d290e47d68b1a4b609c9816ac -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:23:10.529845 16191 consensus_peers.cc:597] T c0eab25ce4c64e018a041beeda2867db P 1ca2725d290e47d68b1a4b609c9816ac -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
I20260709 13:23:10.868319 16292 raft_consensus.cc:1275] T cdbd78acb4cf46298e0de8ba0d130e19 P 90f3da57796b488c9087a2f523adf045 [term 29 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 27 index: 20. Preceding OpId from leader: term: 29 index: 21. (index mismatch)
I20260709 13:23:10.870841 24989 consensus_queue.cc:1048] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 20, Time since last communication: 0.000s
W20260709 13:23:10.920394 16790 consensus_peers.cc:597] T 9e65676b21b748728ad39668cfe722ec P cb82cc7bd6db46d980c75237a1402860 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
I20260709 13:23:10.949146 16221 raft_consensus.cc:1275] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [term 29 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 27 index: 20. Preceding OpId from leader: term: 29 index: 21. (index mismatch)
W20260709 13:23:10.953042 16790 consensus_peers.cc:597] T 2794af5b99984efebdee564035841cb0 P cb82cc7bd6db46d980c75237a1402860 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 71: this message will repeat every 5th retry.
I20260709 13:23:10.976284 25024 consensus_queue.cc:1048] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 20, Time since last communication: 0.001s
W20260709 13:23:11.003728 16191 consensus_peers.cc:597] T e5ad4eb281564936a59f41e387a47156 P 1ca2725d290e47d68b1a4b609c9816ac -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 13:23:11.035228 16191 consensus_peers.cc:597] T 746aabf856ab401c9ef5e859371e7b67 P 1ca2725d290e47d68b1a4b609c9816ac -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:23:11.065135 16790 consensus_peers.cc:597] T 23f82224f4564fea93572e8206bad1fe P cb82cc7bd6db46d980c75237a1402860 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 71: this message will repeat every 5th retry.
W20260709 13:23:11.165690 16191 consensus_peers.cc:597] T 5a96373bb0e14dce8a266b2fc6c58507 P 1ca2725d290e47d68b1a4b609c9816ac -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:23:11.228056 16191 consensus_peers.cc:597] T ee9ed67006144797b1de6ea8c2013a68 P 1ca2725d290e47d68b1a4b609c9816ac -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:23:11.239619 16265 consensus_peers.cc:597] T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:23:11.306946 16790 consensus_peers.cc:597] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:23:11.325148 16790 consensus_peers.cc:597] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 13:23:11.543298 16265 consensus_peers.cc:597] T 02838c86531d4f98bf58d1c22b8baf77 P 90f3da57796b488c9087a2f523adf045 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:23:11.548554 16265 consensus_peers.cc:597] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 66: this message will repeat every 5th retry.
W20260709 13:23:11.555884 16035 catalog_manager.cc:4755] TS 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet 19baa7bc367844c288bca8f81439c413: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111)
W20260709 13:23:11.559043 16265 consensus_peers.cc:597] T fffab26427e24d398e2695aad40ca134 P 90f3da57796b488c9087a2f523adf045 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 66: this message will repeat every 5th retry.
W20260709 13:23:11.606364 16265 consensus_peers.cc:597] T 434bff3082d141838068f323bdb0d1ed P 90f3da57796b488c9087a2f523adf045 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 66: this message will repeat every 5th retry.
I20260709 13:23:11.612847 15046 tablet_server.cc:196] TabletServer@127.14.177.129:0 shutdown complete.
W20260709 13:23:11.699259 16191 consensus_peers.cc:597] T af57996a8be84d398f0f023e8d08ef77 P 1ca2725d290e47d68b1a4b609c9816ac -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 13:23:11.734871 16265 consensus_peers.cc:597] T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:23:11.755522 16265 consensus_peers.cc:597] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:23:11.784343 16790 consensus_peers.cc:597] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 13:23:11.814546 16790 consensus_peers.cc:597] T 806d7690c2b644959152074bb2cab0a7 P cb82cc7bd6db46d980c75237a1402860 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:23:11.816946 16191 consensus_peers.cc:597] T 68a6838d82ef4cac83507f906139d90f P 1ca2725d290e47d68b1a4b609c9816ac -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 13:23:11.847126 16265 consensus_peers.cc:597] T 8cc959fb7f704fba9009c0595467d074 P 90f3da57796b488c9087a2f523adf045 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
I20260709 13:23:11.850579 15046 tablet_server.cc:179] TabletServer@127.14.177.130:0 shutting down...
W20260709 13:23:11.852469 16191 consensus_peers.cc:597] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:23:11.859658 16790 consensus_peers.cc:597] T 66f75124215f4ca6bef831e174a792d8 P cb82cc7bd6db46d980c75237a1402860 -> Peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943): Couldn't send request to peer 1ca2725d290e47d68b1a4b609c9816ac. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:23:11.883988 16265 consensus_peers.cc:597] T 7c2095a5f4ad49408dbc7f69c606120d P 90f3da57796b488c9087a2f523adf045 -> Peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943): Couldn't send request to peer 1ca2725d290e47d68b1a4b609c9816ac. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:23:11.884703 16265 consensus_peers.cc:597] T 8cc959fb7f704fba9009c0595467d074 P 90f3da57796b488c9087a2f523adf045 -> Peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943): Couldn't send request to peer 1ca2725d290e47d68b1a4b609c9816ac. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:23:11.902089 16265 consensus_peers.cc:597] T 1df93779209748b9860f6d893dd3de53 P 90f3da57796b488c9087a2f523adf045 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:23:11.913811 16790 consensus_peers.cc:597] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 66: this message will repeat every 5th retry.
W20260709 13:23:11.919996 16265 consensus_peers.cc:597] T b87147731a68400e8cce9b879daa7c5d P 90f3da57796b488c9087a2f523adf045 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 66: this message will repeat every 5th retry.
W20260709 13:23:11.925995 16265 consensus_peers.cc:597] T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045 -> Peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943): Couldn't send request to peer 1ca2725d290e47d68b1a4b609c9816ac. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:23:11.938153 15046 ts_tablet_manager.cc:1507] Shutting down tablet manager...
W20260709 13:23:11.947238 16191 consensus_peers.cc:597] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:23:11.951488 16790 consensus_peers.cc:597] T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860 -> Peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943): Couldn't send request to peer 1ca2725d290e47d68b1a4b609c9816ac. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:23:11.954088 15046 tablet_replica.cc:333] T 66f75124215f4ca6bef831e174a792d8 P 1ca2725d290e47d68b1a4b609c9816ac: stopping tablet replica
I20260709 13:23:11.955317 15046 raft_consensus.cc:2243] T 66f75124215f4ca6bef831e174a792d8 P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Raft consensus shutting down.
W20260709 13:23:11.956961 16265 consensus_peers.cc:597] T fffab26427e24d398e2695aad40ca134 P 90f3da57796b488c9087a2f523adf045 -> Peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943): Couldn't send request to peer 1ca2725d290e47d68b1a4b609c9816ac. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:23:11.957754 15046 raft_consensus.cc:2272] T 66f75124215f4ca6bef831e174a792d8 P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:11.960911 15046 tablet_replica.cc:333] T e5ad4eb281564936a59f41e387a47156 P 1ca2725d290e47d68b1a4b609c9816ac: stopping tablet replica
I20260709 13:23:11.961838 15046 raft_consensus.cc:2243] T e5ad4eb281564936a59f41e387a47156 P 1ca2725d290e47d68b1a4b609c9816ac [term 22 LEADER]: Raft consensus shutting down.
I20260709 13:23:11.963737 15046 raft_consensus.cc:2272] T e5ad4eb281564936a59f41e387a47156 P 1ca2725d290e47d68b1a4b609c9816ac [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:11.966774 15046 tablet_replica.cc:333] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 1ca2725d290e47d68b1a4b609c9816ac: stopping tablet replica
I20260709 13:23:11.967947 15046 raft_consensus.cc:2243] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 1ca2725d290e47d68b1a4b609c9816ac [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:23:11.968933 15046 raft_consensus.cc:2272] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 1ca2725d290e47d68b1a4b609c9816ac [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:11.973910 15046 tablet_replica.cc:333] T e4dc6b0541ca431ebc757beac00521f1 P 1ca2725d290e47d68b1a4b609c9816ac: stopping tablet replica
I20260709 13:23:11.974812 15046 raft_consensus.cc:2243] T e4dc6b0541ca431ebc757beac00521f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 13:23:11.975769 15046 raft_consensus.cc:2272] T e4dc6b0541ca431ebc757beac00521f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:11.983337 15046 tablet_replica.cc:333] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac: stopping tablet replica
I20260709 13:23:11.984257 15046 raft_consensus.cc:2243] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 13:23:11.985522 15046 raft_consensus.cc:2272] T c9944d5e4b8c4109af9955c4f08c9095 P 1ca2725d290e47d68b1a4b609c9816ac [term 24 FOLLOWER]: Raft consensus is shut down!
W20260709 13:23:11.985867 16191 consensus_peers.cc:597] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
I20260709 13:23:11.988564 15046 tablet_replica.cc:333] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac: stopping tablet replica
I20260709 13:23:11.989521 15046 raft_consensus.cc:2243] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [term 29 FOLLOWER]: Raft consensus shutting down.
I20260709 13:23:11.990095 15046 raft_consensus.cc:2272] T cdbd78acb4cf46298e0de8ba0d130e19 P 1ca2725d290e47d68b1a4b609c9816ac [term 29 FOLLOWER]: Raft consensus is shut down!
W20260709 13:23:11.991215 16265 consensus_peers.cc:597] T e4dc6b0541ca431ebc757beac00521f1 P 90f3da57796b488c9087a2f523adf045 -> Peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943): Couldn't send request to peer 1ca2725d290e47d68b1a4b609c9816ac. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:23:11.994422 15046 tablet_replica.cc:333] T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac: stopping tablet replica
I20260709 13:23:11.995400 15046 raft_consensus.cc:2243] T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac [term 23 LEADER]: Raft consensus shutting down.
I20260709 13:23:11.996546 15046 raft_consensus.cc:2272] T 8d4ab870a05347868956aa8b841b2981 P 1ca2725d290e47d68b1a4b609c9816ac [term 23 FOLLOWER]: Raft consensus is shut down!
W20260709 13:23:11.997284 16265 consensus_peers.cc:597] T 8a878999c0024d2abe517484180290fc P 90f3da57796b488c9087a2f523adf045 -> Peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943): Couldn't send request to peer 1ca2725d290e47d68b1a4b609c9816ac. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:23:12.000963 15046 tablet_replica.cc:333] T 1df93779209748b9860f6d893dd3de53 P 1ca2725d290e47d68b1a4b609c9816ac: stopping tablet replica
I20260709 13:23:12.001884 15046 raft_consensus.cc:2243] T 1df93779209748b9860f6d893dd3de53 P 1ca2725d290e47d68b1a4b609c9816ac [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 13:23:12.002632 15046 raft_consensus.cc:2272] T 1df93779209748b9860f6d893dd3de53 P 1ca2725d290e47d68b1a4b609c9816ac [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:12.005844 15046 tablet_replica.cc:333] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac: stopping tablet replica
W20260709 13:23:12.006261 16790 consensus_peers.cc:597] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 -> Peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943): Couldn't send request to peer 1ca2725d290e47d68b1a4b609c9816ac. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:23:12.006647 15046 raft_consensus.cc:2243] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [term 27 LEADER]: Raft consensus shutting down.
I20260709 13:23:12.009634 15046 raft_consensus.cc:2272] T d2372981006d4b218215c28964dea623 P 1ca2725d290e47d68b1a4b609c9816ac [term 27 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:12.012717 15046 tablet_replica.cc:333] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac: stopping tablet replica
I20260709 13:23:12.013757 15046 raft_consensus.cc:2243] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [term 24 LEADER]: Raft consensus shutting down.
I20260709 13:23:12.015291 15046 raft_consensus.cc:2272] T 8638ee363273426db1393dd81e5c7b72 P 1ca2725d290e47d68b1a4b609c9816ac [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:12.018921 15046 tablet_replica.cc:333] T 08c74490c40243a88e62d1760b5cd814 P 1ca2725d290e47d68b1a4b609c9816ac: stopping tablet replica
I20260709 13:23:12.020308 15046 raft_consensus.cc:2243] T 08c74490c40243a88e62d1760b5cd814 P 1ca2725d290e47d68b1a4b609c9816ac [term 22 FOLLOWER]: Raft consensus shutting down.
W20260709 13:23:12.021072 16790 consensus_peers.cc:597] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860 -> Peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943): Couldn't send request to peer 1ca2725d290e47d68b1a4b609c9816ac. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:23:12.028534 15046 raft_consensus.cc:2272] T 08c74490c40243a88e62d1760b5cd814 P 1ca2725d290e47d68b1a4b609c9816ac [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:12.031777 15046 tablet_replica.cc:333] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac: stopping tablet replica
W20260709 13:23:12.031845 16265 consensus_peers.cc:597] T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045 -> Peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943): Couldn't send request to peer 1ca2725d290e47d68b1a4b609c9816ac. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:23:12.034552 15046 raft_consensus.cc:2243] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac [term 23 LEADER]: Raft consensus shutting down.
I20260709 13:23:12.036409 15046 raft_consensus.cc:2272] T c4f65cdf00b748bb963ee6e81ef43b79 P 1ca2725d290e47d68b1a4b609c9816ac [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:12.039397 15046 tablet_replica.cc:333] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac: stopping tablet replica
I20260709 13:23:12.040907 15046 raft_consensus.cc:2243] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 13:23:12.041563 15046 raft_consensus.cc:2272] T 1d5d59407eb545b895d8d922e0dd69fd P 1ca2725d290e47d68b1a4b609c9816ac [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:12.044538 15046 tablet_replica.cc:333] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac: stopping tablet replica
I20260709 13:23:12.045452 15046 raft_consensus.cc:2243] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac [term 24 LEADER]: Raft consensus shutting down.
I20260709 13:23:12.046736 15046 raft_consensus.cc:2272] T 8158bbd17b994d2eb96f560bd0f1b88d P 1ca2725d290e47d68b1a4b609c9816ac [term 24 FOLLOWER]: Raft consensus is shut down!
W20260709 13:23:12.046634 16790 consensus_peers.cc:597] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 -> Peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943): Couldn't send request to peer 1ca2725d290e47d68b1a4b609c9816ac. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:23:12.050200 15046 tablet_replica.cc:333] T ee9ed67006144797b1de6ea8c2013a68 P 1ca2725d290e47d68b1a4b609c9816ac: stopping tablet replica
I20260709 13:23:12.051215 15046 raft_consensus.cc:2243] T ee9ed67006144797b1de6ea8c2013a68 P 1ca2725d290e47d68b1a4b609c9816ac [term 22 LEADER]: Raft consensus shutting down.
I20260709 13:23:12.052558 15046 raft_consensus.cc:2272] T ee9ed67006144797b1de6ea8c2013a68 P 1ca2725d290e47d68b1a4b609c9816ac [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:12.057009 15046 tablet_replica.cc:333] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac: stopping tablet replica
I20260709 13:23:12.058660 15046 raft_consensus.cc:2243] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [term 30 LEADER]: Raft consensus shutting down.
I20260709 13:23:12.060302 15046 raft_consensus.cc:2272] T 62fe3346cd3c42c4a55e14af9073bca4 P 1ca2725d290e47d68b1a4b609c9816ac [term 30 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:12.064404 15046 tablet_replica.cc:333] T 19baa7bc367844c288bca8f81439c413 P 1ca2725d290e47d68b1a4b609c9816ac: stopping tablet replica
I20260709 13:23:12.065310 15046 raft_consensus.cc:2243] T 19baa7bc367844c288bca8f81439c413 P 1ca2725d290e47d68b1a4b609c9816ac [term 25 FOLLOWER]: Raft consensus shutting down.
I20260709 13:23:12.066856 15046 raft_consensus.cc:2272] T 19baa7bc367844c288bca8f81439c413 P 1ca2725d290e47d68b1a4b609c9816ac [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:12.069653 15046 tablet_replica.cc:333] T 746aabf856ab401c9ef5e859371e7b67 P 1ca2725d290e47d68b1a4b609c9816ac: stopping tablet replica
I20260709 13:23:12.070591 15046 raft_consensus.cc:2243] T 746aabf856ab401c9ef5e859371e7b67 P 1ca2725d290e47d68b1a4b609c9816ac [term 20 LEADER]: Raft consensus shutting down.
I20260709 13:23:12.071833 15046 raft_consensus.cc:2272] T 746aabf856ab401c9ef5e859371e7b67 P 1ca2725d290e47d68b1a4b609c9816ac [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:12.074564 15046 tablet_replica.cc:333] T 68a6838d82ef4cac83507f906139d90f P 1ca2725d290e47d68b1a4b609c9816ac: stopping tablet replica
I20260709 13:23:12.075371 15046 raft_consensus.cc:2243] T 68a6838d82ef4cac83507f906139d90f P 1ca2725d290e47d68b1a4b609c9816ac [term 23 LEADER]: Raft consensus shutting down.
I20260709 13:23:12.076514 15046 raft_consensus.cc:2272] T 68a6838d82ef4cac83507f906139d90f P 1ca2725d290e47d68b1a4b609c9816ac [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:12.080055 15046 tablet_replica.cc:333] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 1ca2725d290e47d68b1a4b609c9816ac: stopping tablet replica
I20260709 13:23:12.081817 15046 raft_consensus.cc:2243] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:23:12.082420 15046 raft_consensus.cc:2272] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:12.086146 15046 tablet_replica.cc:333] T ecc1e363230043fe9fef390f0b979f20 P 1ca2725d290e47d68b1a4b609c9816ac: stopping tablet replica
I20260709 13:23:12.086885 15046 raft_consensus.cc:2243] T ecc1e363230043fe9fef390f0b979f20 P 1ca2725d290e47d68b1a4b609c9816ac [term 27 FOLLOWER]: Raft consensus shutting down.
I20260709 13:23:12.088080 15046 raft_consensus.cc:2272] T ecc1e363230043fe9fef390f0b979f20 P 1ca2725d290e47d68b1a4b609c9816ac [term 27 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:12.090826 15046 tablet_replica.cc:333] T c0eab25ce4c64e018a041beeda2867db P 1ca2725d290e47d68b1a4b609c9816ac: stopping tablet replica
I20260709 13:23:12.091610 15046 raft_consensus.cc:2243] T c0eab25ce4c64e018a041beeda2867db P 1ca2725d290e47d68b1a4b609c9816ac [term 24 LEADER]: Raft consensus shutting down.
I20260709 13:23:12.092828 15046 raft_consensus.cc:2272] T c0eab25ce4c64e018a041beeda2867db P 1ca2725d290e47d68b1a4b609c9816ac [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:12.096769 15046 tablet_replica.cc:333] T 829b4ad8c5a54eababeea1181b960d4b P 1ca2725d290e47d68b1a4b609c9816ac: stopping tablet replica
I20260709 13:23:12.097534 15046 raft_consensus.cc:2243] T 829b4ad8c5a54eababeea1181b960d4b P 1ca2725d290e47d68b1a4b609c9816ac [term 21 LEADER]: Raft consensus shutting down.
W20260709 13:23:12.098613 16265 consensus_peers.cc:597] T 02838c86531d4f98bf58d1c22b8baf77 P 90f3da57796b488c9087a2f523adf045 -> Peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943): Couldn't send request to peer 1ca2725d290e47d68b1a4b609c9816ac. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:23:12.098768 15046 raft_consensus.cc:2272] T 829b4ad8c5a54eababeea1181b960d4b P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:12.106690 15046 tablet_replica.cc:333] T af57996a8be84d398f0f023e8d08ef77 P 1ca2725d290e47d68b1a4b609c9816ac: stopping tablet replica
I20260709 13:23:12.110560 15046 raft_consensus.cc:2243] T af57996a8be84d398f0f023e8d08ef77 P 1ca2725d290e47d68b1a4b609c9816ac [term 22 LEADER]: Raft consensus shutting down.
I20260709 13:23:12.111726 15046 raft_consensus.cc:2272] T af57996a8be84d398f0f023e8d08ef77 P 1ca2725d290e47d68b1a4b609c9816ac [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:12.114573 15046 tablet_replica.cc:333] T 84f2da98864b4425bf7c0055951fdd4b P 1ca2725d290e47d68b1a4b609c9816ac: stopping tablet replica
I20260709 13:23:12.115350 15046 raft_consensus.cc:2243] T 84f2da98864b4425bf7c0055951fdd4b P 1ca2725d290e47d68b1a4b609c9816ac [term 22 LEADER]: Raft consensus shutting down.
I20260709 13:23:12.116518 15046 raft_consensus.cc:2272] T 84f2da98864b4425bf7c0055951fdd4b P 1ca2725d290e47d68b1a4b609c9816ac [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:12.119304 15046 tablet_replica.cc:333] T d16dfad0c3b540e5b4b86492ddf87208 P 1ca2725d290e47d68b1a4b609c9816ac: stopping tablet replica
I20260709 13:23:12.120034 15046 raft_consensus.cc:2243] T d16dfad0c3b540e5b4b86492ddf87208 P 1ca2725d290e47d68b1a4b609c9816ac [term 23 LEADER]: Raft consensus shutting down.
I20260709 13:23:12.122290 15046 raft_consensus.cc:2272] T d16dfad0c3b540e5b4b86492ddf87208 P 1ca2725d290e47d68b1a4b609c9816ac [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:12.125942 15046 tablet_replica.cc:333] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac: stopping tablet replica
I20260709 13:23:12.126780 15046 raft_consensus.cc:2243] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac [term 23 LEADER]: Raft consensus shutting down.
I20260709 13:23:12.127837 15046 raft_consensus.cc:2272] T a378d7f796f94615b3b347119a9880fd P 1ca2725d290e47d68b1a4b609c9816ac [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:12.130640 15046 tablet_replica.cc:333] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac: stopping tablet replica
I20260709 13:23:12.131453 15046 raft_consensus.cc:2243] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [term 26 FOLLOWER]: Raft consensus shutting down.
I20260709 13:23:12.132344 15046 raft_consensus.cc:2272] T 74ca8140f35c4ae29e8a664f6bdfc829 P 1ca2725d290e47d68b1a4b609c9816ac [term 26 FOLLOWER]: Raft consensus is shut down!
W20260709 13:23:12.133462 16265 consensus_peers.cc:597] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045 -> Peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943): Couldn't send request to peer 1ca2725d290e47d68b1a4b609c9816ac. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:23:12.135133 15046 tablet_replica.cc:333] T d2ab0ac8347641e6955e9bce7d90c1a2 P 1ca2725d290e47d68b1a4b609c9816ac: stopping tablet replica
I20260709 13:23:12.140517 15046 raft_consensus.cc:2243] T d2ab0ac8347641e6955e9bce7d90c1a2 P 1ca2725d290e47d68b1a4b609c9816ac [term 21 LEADER]: Raft consensus shutting down.
I20260709 13:23:12.141693 15046 raft_consensus.cc:2272] T d2ab0ac8347641e6955e9bce7d90c1a2 P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Raft consensus is shut down!
W20260709 13:23:12.139295 16790 consensus_peers.cc:597] T 74ca8140f35c4ae29e8a664f6bdfc829 P cb82cc7bd6db46d980c75237a1402860 -> Peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943): Couldn't send request to peer 1ca2725d290e47d68b1a4b609c9816ac. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:23:12.144781 15046 tablet_replica.cc:333] T 8cc959fb7f704fba9009c0595467d074 P 1ca2725d290e47d68b1a4b609c9816ac: stopping tablet replica
I20260709 13:23:12.145671 15046 raft_consensus.cc:2243] T 8cc959fb7f704fba9009c0595467d074 P 1ca2725d290e47d68b1a4b609c9816ac [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:23:12.146297 15046 raft_consensus.cc:2272] T 8cc959fb7f704fba9009c0595467d074 P 1ca2725d290e47d68b1a4b609c9816ac [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:12.149544 15046 tablet_replica.cc:333] T 5a959dbecc2648eb85d1cc550f5a3245 P 1ca2725d290e47d68b1a4b609c9816ac: stopping tablet replica
I20260709 13:23:12.150357 15046 raft_consensus.cc:2243] T 5a959dbecc2648eb85d1cc550f5a3245 P 1ca2725d290e47d68b1a4b609c9816ac [term 24 LEADER]: Raft consensus shutting down.
I20260709 13:23:12.152235 15046 raft_consensus.cc:2272] T 5a959dbecc2648eb85d1cc550f5a3245 P 1ca2725d290e47d68b1a4b609c9816ac [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:12.155026 15046 tablet_replica.cc:333] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 1ca2725d290e47d68b1a4b609c9816ac: stopping tablet replica
I20260709 13:23:12.155920 15046 raft_consensus.cc:2243] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 1ca2725d290e47d68b1a4b609c9816ac [term 23 LEADER]: Raft consensus shutting down.
I20260709 13:23:12.157181 15046 raft_consensus.cc:2272] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 1ca2725d290e47d68b1a4b609c9816ac [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:12.160290 15046 tablet_replica.cc:333] T 02838c86531d4f98bf58d1c22b8baf77 P 1ca2725d290e47d68b1a4b609c9816ac: stopping tablet replica
I20260709 13:23:12.161120 15046 raft_consensus.cc:2243] T 02838c86531d4f98bf58d1c22b8baf77 P 1ca2725d290e47d68b1a4b609c9816ac [term 25 FOLLOWER]: Raft consensus shutting down.
I20260709 13:23:12.162036 15046 raft_consensus.cc:2272] T 02838c86531d4f98bf58d1c22b8baf77 P 1ca2725d290e47d68b1a4b609c9816ac [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:12.164990 15046 tablet_replica.cc:333] T 7c2095a5f4ad49408dbc7f69c606120d P 1ca2725d290e47d68b1a4b609c9816ac: stopping tablet replica
I20260709 13:23:12.165737 15046 raft_consensus.cc:2243] T 7c2095a5f4ad49408dbc7f69c606120d P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:23:12.167049 15046 raft_consensus.cc:2272] T 7c2095a5f4ad49408dbc7f69c606120d P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:12.170025 15046 tablet_replica.cc:333] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac: stopping tablet replica
I20260709 13:23:12.170778 15046 raft_consensus.cc:2243] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [term 29 FOLLOWER]: Raft consensus shutting down.
I20260709 13:23:12.171466 15046 raft_consensus.cc:2272] T 66ae176961b74813a58bc15c9c4c6f2a P 1ca2725d290e47d68b1a4b609c9816ac [term 29 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:12.175014 15046 tablet_replica.cc:333] T b6f07842d3aa47858df5665874c86f3a P 1ca2725d290e47d68b1a4b609c9816ac: stopping tablet replica
W20260709 13:23:12.175423 16790 consensus_peers.cc:597] T c9944d5e4b8c4109af9955c4f08c9095 P cb82cc7bd6db46d980c75237a1402860 -> Peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943): Couldn't send request to peer 1ca2725d290e47d68b1a4b609c9816ac. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:23:12.175843 15046 raft_consensus.cc:2243] T b6f07842d3aa47858df5665874c86f3a P 1ca2725d290e47d68b1a4b609c9816ac [term 26 FOLLOWER]: Raft consensus shutting down.
I20260709 13:23:12.176869 15046 raft_consensus.cc:2272] T b6f07842d3aa47858df5665874c86f3a P 1ca2725d290e47d68b1a4b609c9816ac [term 26 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:12.180267 15046 tablet_replica.cc:333] T fffab26427e24d398e2695aad40ca134 P 1ca2725d290e47d68b1a4b609c9816ac: stopping tablet replica
I20260709 13:23:12.181139 15046 raft_consensus.cc:2243] T fffab26427e24d398e2695aad40ca134 P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:23:12.181852 15046 raft_consensus.cc:2272] T fffab26427e24d398e2695aad40ca134 P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:12.184952 15046 tablet_replica.cc:333] T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac: stopping tablet replica
I20260709 13:23:12.187002 15046 raft_consensus.cc:2243] T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac [term 26 FOLLOWER]: Raft consensus shutting down.
I20260709 13:23:12.187922 15046 raft_consensus.cc:2272] T 9c61a39fd4f84424b129c1a6f9ba404d P 1ca2725d290e47d68b1a4b609c9816ac [term 26 FOLLOWER]: Raft consensus is shut down!
W20260709 13:23:12.189291 16265 consensus_peers.cc:597] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 90f3da57796b488c9087a2f523adf045 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 66: this message will repeat every 5th retry.
I20260709 13:23:12.192271 15046 tablet_replica.cc:333] T 8c11074579b046d99e585c2818e18a2c P 1ca2725d290e47d68b1a4b609c9816ac: stopping tablet replica
I20260709 13:23:12.193243 15046 raft_consensus.cc:2243] T 8c11074579b046d99e585c2818e18a2c P 1ca2725d290e47d68b1a4b609c9816ac [term 20 LEADER]: Raft consensus shutting down.
I20260709 13:23:12.194530 15046 raft_consensus.cc:2272] T 8c11074579b046d99e585c2818e18a2c P 1ca2725d290e47d68b1a4b609c9816ac [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:12.197973 15046 tablet_replica.cc:333] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac: stopping tablet replica
I20260709 13:23:12.198792 15046 raft_consensus.cc:2243] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac [term 24 LEADER]: Raft consensus shutting down.
I20260709 13:23:12.200048 15046 raft_consensus.cc:2272] T 08b7d1ab519044eda2d5e04a07242f25 P 1ca2725d290e47d68b1a4b609c9816ac [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:12.202844 15046 tablet_replica.cc:333] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac: stopping tablet replica
I20260709 13:23:12.203675 15046 raft_consensus.cc:2243] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 13:23:12.204780 15046 raft_consensus.cc:2272] T 5718c623ebee4d2bb770f0f9ddb2f164 P 1ca2725d290e47d68b1a4b609c9816ac [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:12.207736 15046 tablet_replica.cc:333] T 18ad7e3af5504571a69d10c00750a01c P 1ca2725d290e47d68b1a4b609c9816ac: stopping tablet replica
I20260709 13:23:12.208508 15046 raft_consensus.cc:2243] T 18ad7e3af5504571a69d10c00750a01c P 1ca2725d290e47d68b1a4b609c9816ac [term 19 LEADER]: Raft consensus shutting down.
I20260709 13:23:12.209779 15046 raft_consensus.cc:2272] T 18ad7e3af5504571a69d10c00750a01c P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:12.212904 15046 tablet_replica.cc:333] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac: stopping tablet replica
I20260709 13:23:12.213696 15046 raft_consensus.cc:2243] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac [term 25 LEADER]: Raft consensus shutting down.
I20260709 13:23:12.214839 15046 raft_consensus.cc:2272] T ada5a004ba7c46819051e3ccec71ffdb P 1ca2725d290e47d68b1a4b609c9816ac [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:12.217639 15046 tablet_replica.cc:333] T 8a878999c0024d2abe517484180290fc P 1ca2725d290e47d68b1a4b609c9816ac: stopping tablet replica
I20260709 13:23:12.218393 15046 raft_consensus.cc:2243] T 8a878999c0024d2abe517484180290fc P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:23:12.219187 15046 raft_consensus.cc:2272] T 8a878999c0024d2abe517484180290fc P 1ca2725d290e47d68b1a4b609c9816ac [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:12.221946 15046 tablet_replica.cc:333] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 1ca2725d290e47d68b1a4b609c9816ac: stopping tablet replica
I20260709 13:23:12.222589 15046 raft_consensus.cc:2243] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 24 LEADER]: Raft consensus shutting down.
I20260709 13:23:12.223693 15046 raft_consensus.cc:2272] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 1ca2725d290e47d68b1a4b609c9816ac [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:12.226476 15046 tablet_replica.cc:333] T 5a96373bb0e14dce8a266b2fc6c58507 P 1ca2725d290e47d68b1a4b609c9816ac: stopping tablet replica
I20260709 13:23:12.227216 15046 raft_consensus.cc:2243] T 5a96373bb0e14dce8a266b2fc6c58507 P 1ca2725d290e47d68b1a4b609c9816ac [term 19 LEADER]: Raft consensus shutting down.
I20260709 13:23:12.228447 15046 raft_consensus.cc:2272] T 5a96373bb0e14dce8a266b2fc6c58507 P 1ca2725d290e47d68b1a4b609c9816ac [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:12.231206 15046 tablet_replica.cc:333] T c2f5f89186204242bf377f7c42b2e2da P 1ca2725d290e47d68b1a4b609c9816ac: stopping tablet replica
I20260709 13:23:12.231829 15046 raft_consensus.cc:2243] T c2f5f89186204242bf377f7c42b2e2da P 1ca2725d290e47d68b1a4b609c9816ac [term 25 FOLLOWER]: Raft consensus shutting down.
I20260709 13:23:12.232668 15046 raft_consensus.cc:2272] T c2f5f89186204242bf377f7c42b2e2da P 1ca2725d290e47d68b1a4b609c9816ac [term 25 FOLLOWER]: Raft consensus is shut down!
W20260709 13:23:12.282764 16265 consensus_peers.cc:597] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:23:12.309034 16265 consensus_peers.cc:597] T 1df93779209748b9860f6d893dd3de53 P 90f3da57796b488c9087a2f523adf045 -> Peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943): Couldn't send request to peer 1ca2725d290e47d68b1a4b609c9816ac. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:23:12.331571 16265 consensus_peers.cc:597] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045 -> Peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943): Couldn't send request to peer 1ca2725d290e47d68b1a4b609c9816ac. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:23:12.362496 16265 connection.cc:769] client connection to 127.14.177.130:35943 send error: Network error: sendmsg error: Broken pipe (error 32)
W20260709 13:23:12.373440 16265 consensus_peers.cc:597] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 90f3da57796b488c9087a2f523adf045 -> Peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943): Couldn't send request to peer 1ca2725d290e47d68b1a4b609c9816ac. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.130:35943: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:23:12.374264 16265 consensus_peers.cc:597] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045 -> Peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943): Couldn't send request to peer 1ca2725d290e47d68b1a4b609c9816ac. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.130:35943: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:23:12.398733 16790 consensus_peers.cc:597] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 -> Peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943): Couldn't send request to peer 1ca2725d290e47d68b1a4b609c9816ac. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.130:35943: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:23:12.407482 16790 consensus_peers.cc:597] T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 71: this message will repeat every 5th retry.
W20260709 13:23:12.416704 16265 consensus_peers.cc:597] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 90f3da57796b488c9087a2f523adf045 -> Peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943): Couldn't send request to peer 1ca2725d290e47d68b1a4b609c9816ac. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.130:35943: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:23:12.417742 16790 consensus_peers.cc:597] T 3884f864a96d428785895e5463e5728f P cb82cc7bd6db46d980c75237a1402860 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 71: this message will repeat every 5th retry.
W20260709 13:23:12.545490 16790 consensus_peers.cc:597] T 57a744f5aaf944678a63469cbf514015 P cb82cc7bd6db46d980c75237a1402860 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 71: this message will repeat every 5th retry.
W20260709 13:23:12.582252 16265 consensus_peers.cc:597] T e4dc6b0541ca431ebc757beac00521f1 P 90f3da57796b488c9087a2f523adf045 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 66: this message will repeat every 5th retry.
W20260709 13:23:12.607561 16790 consensus_peers.cc:597] T 66f75124215f4ca6bef831e174a792d8 P cb82cc7bd6db46d980c75237a1402860 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 66: this message will repeat every 5th retry.
W20260709 13:23:12.845429 16265 consensus_peers.cc:597] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 90f3da57796b488c9087a2f523adf045 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:23:12.845989 16265 consensus_peers.cc:597] T 26354361e9b044cea1c2b86c4882fe92 P 90f3da57796b488c9087a2f523adf045 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 66: this message will repeat every 5th retry.
I20260709 13:23:12.962169 25024 raft_consensus.cc:493] T 8d4ab870a05347868956aa8b841b2981 P cb82cc7bd6db46d980c75237a1402860 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:23:12.962966 25024 raft_consensus.cc:515] T 8d4ab870a05347868956aa8b841b2981 P cb82cc7bd6db46d980c75237a1402860 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:23:12.966178 25024 leader_election.cc:290] T 8d4ab870a05347868956aa8b841b2981 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:23:12.966733 16292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d4ab870a05347868956aa8b841b2981" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
W20260709 13:23:12.973119 16790 leader_election.cc:336] T 8d4ab870a05347868956aa8b841b2981 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943): Network error: Client connection negotiation failed: client connection to 127.14.177.130:35943: connect: Connection refused (error 111)
I20260709 13:23:12.973620 16790 leader_election.cc:304] T 8d4ab870a05347868956aa8b841b2981 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:23:12.974433 25024 raft_consensus.cc:2749] T 8d4ab870a05347868956aa8b841b2981 P cb82cc7bd6db46d980c75237a1402860 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:23:12.982030 24975 raft_consensus.cc:493] T 8638ee363273426db1393dd81e5c7b72 P 90f3da57796b488c9087a2f523adf045 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:23:12.982693 24975 raft_consensus.cc:515] T 8638ee363273426db1393dd81e5c7b72 P 90f3da57796b488c9087a2f523adf045 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:23:12.985204 24975 leader_election.cc:290] T 8638ee363273426db1393dd81e5c7b72 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:23:12.987071 16819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8638ee363273426db1393dd81e5c7b72" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
W20260709 13:23:12.990784 16265 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.14.177.130:35943: connect: Connection refused (error 111) [suppressed 485 similar messages]
W20260709 13:23:12.995005 16265 leader_election.cc:336] T 8638ee363273426db1393dd81e5c7b72 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943): Network error: Client connection negotiation failed: client connection to 127.14.177.130:35943: connect: Connection refused (error 111)
I20260709 13:23:12.995741 16265 leader_election.cc:304] T 8638ee363273426db1393dd81e5c7b72 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860
I20260709 13:23:12.997193 24975 raft_consensus.cc:2749] T 8638ee363273426db1393dd81e5c7b72 P 90f3da57796b488c9087a2f523adf045 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:23:13.011451 24975 raft_consensus.cc:493] T d16dfad0c3b540e5b4b86492ddf87208 P 90f3da57796b488c9087a2f523adf045 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:23:13.012122 24975 raft_consensus.cc:515] T d16dfad0c3b540e5b4b86492ddf87208 P 90f3da57796b488c9087a2f523adf045 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:23:13.014793 24975 leader_election.cc:290] T d16dfad0c3b540e5b4b86492ddf87208 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
W20260709 13:23:13.019642 16265 leader_election.cc:336] T d16dfad0c3b540e5b4b86492ddf87208 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943): Network error: Client connection negotiation failed: client connection to 127.14.177.130:35943: connect: Connection refused (error 111)
W20260709 13:23:13.023542 16265 leader_election.cc:336] T d16dfad0c3b540e5b4b86492ddf87208 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111)
I20260709 13:23:13.024039 16265 leader_election.cc:304] T d16dfad0c3b540e5b4b86492ddf87208 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:23:13.024881 24975 raft_consensus.cc:2749] T d16dfad0c3b540e5b4b86492ddf87208 P 90f3da57796b488c9087a2f523adf045 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:23:13.071471 25038 raft_consensus.cc:493] T 8d4ab870a05347868956aa8b841b2981 P 90f3da57796b488c9087a2f523adf045 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:23:13.072414 25038 raft_consensus.cc:515] T 8d4ab870a05347868956aa8b841b2981 P 90f3da57796b488c9087a2f523adf045 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:23:13.078634 25038 leader_election.cc:290] T 8d4ab870a05347868956aa8b841b2981 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:23:13.080358 16819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d4ab870a05347868956aa8b841b2981" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:23:13.081168 16819 raft_consensus.cc:2468] T 8d4ab870a05347868956aa8b841b2981 P cb82cc7bd6db46d980c75237a1402860 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 23.
I20260709 13:23:13.082623 16266 leader_election.cc:304] T 8d4ab870a05347868956aa8b841b2981 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:23:13.083736 24975 raft_consensus.cc:2804] T 8d4ab870a05347868956aa8b841b2981 P 90f3da57796b488c9087a2f523adf045 [term 23 FOLLOWER]: Leader pre-election won for term 24
W20260709 13:23:13.086320 16265 consensus_peers.cc:597] T 8a878999c0024d2abe517484180290fc P 90f3da57796b488c9087a2f523adf045 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
I20260709 13:23:13.085757 24975 raft_consensus.cc:493] T 8d4ab870a05347868956aa8b841b2981 P 90f3da57796b488c9087a2f523adf045 [term 23 FOLLOWER]: Starting leader election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:23:13.087814 24975 raft_consensus.cc:3060] T 8d4ab870a05347868956aa8b841b2981 P 90f3da57796b488c9087a2f523adf045 [term 23 FOLLOWER]: Advancing to term 24
W20260709 13:23:13.100493 16265 leader_election.cc:336] T 8d4ab870a05347868956aa8b841b2981 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943): Network error: Client connection negotiation failed: client connection to 127.14.177.130:35943: connect: Connection refused (error 111)
I20260709 13:23:13.105176 24975 raft_consensus.cc:515] T 8d4ab870a05347868956aa8b841b2981 P 90f3da57796b488c9087a2f523adf045 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:23:13.108047 24975 leader_election.cc:290] T 8d4ab870a05347868956aa8b841b2981 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 24 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:23:13.109393 16819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d4ab870a05347868956aa8b841b2981" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:23:13.110106 16819 raft_consensus.cc:3060] T 8d4ab870a05347868956aa8b841b2981 P cb82cc7bd6db46d980c75237a1402860 [term 23 FOLLOWER]: Advancing to term 24
W20260709 13:23:13.118067 16265 leader_election.cc:336] T 8d4ab870a05347868956aa8b841b2981 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 24 election: RPC error from VoteRequest() call to peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943): Network error: Client connection negotiation failed: client connection to 127.14.177.130:35943: connect: Connection refused (error 111)
I20260709 13:23:13.123579 16819 raft_consensus.cc:2468] T 8d4ab870a05347868956aa8b841b2981 P cb82cc7bd6db46d980c75237a1402860 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 24.
I20260709 13:23:13.125293 16266 leader_election.cc:304] T 8d4ab870a05347868956aa8b841b2981 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac
I20260709 13:23:13.127103 25038 raft_consensus.cc:2804] T 8d4ab870a05347868956aa8b841b2981 P 90f3da57796b488c9087a2f523adf045 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 13:23:13.127708 25038 raft_consensus.cc:697] T 8d4ab870a05347868956aa8b841b2981 P 90f3da57796b488c9087a2f523adf045 [term 24 LEADER]: Becoming Leader. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:23:13.128813 25038 consensus_queue.cc:237] T 8d4ab870a05347868956aa8b841b2981 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 23.17, Last appended by leader: 17, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:23:13.131912 25051 raft_consensus.cc:493] T a378d7f796f94615b3b347119a9880fd P 90f3da57796b488c9087a2f523adf045 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:23:13.132583 25051 raft_consensus.cc:515] T a378d7f796f94615b3b347119a9880fd P 90f3da57796b488c9087a2f523adf045 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:23:13.135233 25051 leader_election.cc:290] T a378d7f796f94615b3b347119a9880fd P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:23:13.136637 16819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a378d7f796f94615b3b347119a9880fd" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
W20260709 13:23:13.146822 16265 leader_election.cc:336] T a378d7f796f94615b3b347119a9880fd P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943): Network error: Client connection negotiation failed: client connection to 127.14.177.130:35943: connect: Connection refused (error 111)
I20260709 13:23:13.147349 16265 leader_election.cc:304] T a378d7f796f94615b3b347119a9880fd P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860
I20260709 13:23:13.148090 24975 raft_consensus.cc:2749] T a378d7f796f94615b3b347119a9880fd P 90f3da57796b488c9087a2f523adf045 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:23:13.165333 25024 raft_consensus.cc:493] T 5a959dbecc2648eb85d1cc550f5a3245 P cb82cc7bd6db46d980c75237a1402860 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:23:13.166013 25024 raft_consensus.cc:515] T 5a959dbecc2648eb85d1cc550f5a3245 P cb82cc7bd6db46d980c75237a1402860 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:23:13.164357 16044 catalog_manager.cc:5697] T 8d4ab870a05347868956aa8b841b2981 P 90f3da57796b488c9087a2f523adf045 reported cstate change: term changed from 23 to 24, leader changed from 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130) to 90f3da57796b488c9087a2f523adf045 (127.14.177.131). New cstate: current_term: 24 leader_uuid: "90f3da57796b488c9087a2f523adf045" committed_config { opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:23:13.171258 25038 raft_consensus.cc:493] T 8158bbd17b994d2eb96f560bd0f1b88d P 90f3da57796b488c9087a2f523adf045 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:23:13.171852 25038 raft_consensus.cc:515] T 8158bbd17b994d2eb96f560bd0f1b88d P 90f3da57796b488c9087a2f523adf045 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:23:13.174139 25038 leader_election.cc:290] T 8158bbd17b994d2eb96f560bd0f1b88d P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:23:13.174960 16292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a959dbecc2648eb85d1cc550f5a3245" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:23:13.178308 25024 leader_election.cc:290] T 5a959dbecc2648eb85d1cc550f5a3245 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
W20260709 13:23:13.182329 16790 leader_election.cc:336] T 5a959dbecc2648eb85d1cc550f5a3245 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111)
I20260709 13:23:13.182811 16790 leader_election.cc:304] T 5a959dbecc2648eb85d1cc550f5a3245 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:23:13.185590 25024 raft_consensus.cc:493] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [term 30 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:23:13.186224 25024 raft_consensus.cc:515] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:23:13.187731 25050 raft_consensus.cc:2749] T 5a959dbecc2648eb85d1cc550f5a3245 P cb82cc7bd6db46d980c75237a1402860 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 13:23:13.188295 16790 leader_election.cc:336] T 5a959dbecc2648eb85d1cc550f5a3245 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943): Network error: Client connection negotiation failed: client connection to 127.14.177.130:35943: connect: Connection refused (error 111)
I20260709 13:23:13.193593 25024 leader_election.cc:290] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:23:13.201507 25024 raft_consensus.cc:493] T c4f65cdf00b748bb963ee6e81ef43b79 P cb82cc7bd6db46d980c75237a1402860 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:23:13.202203 25024 raft_consensus.cc:515] T c4f65cdf00b748bb963ee6e81ef43b79 P cb82cc7bd6db46d980c75237a1402860 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:23:13.205391 25024 leader_election.cc:290] T c4f65cdf00b748bb963ee6e81ef43b79 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
W20260709 13:23:13.207880 16790 leader_election.cc:336] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 31 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111)
I20260709 13:23:13.207759 16292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4f65cdf00b748bb963ee6e81ef43b79" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
W20260709 13:23:13.209273 16790 leader_election.cc:336] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 31 pre-election: RPC error from VoteRequest() call to peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943): Network error: Client connection negotiation failed: client connection to 127.14.177.130:35943: connect: Connection refused (error 111)
I20260709 13:23:13.209770 16790 leader_election.cc:304] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
W20260709 13:23:13.211427 16265 leader_election.cc:336] T 8158bbd17b994d2eb96f560bd0f1b88d P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111)
I20260709 13:23:13.211352 25024 raft_consensus.cc:2749] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
W20260709 13:23:13.214800 16790 leader_election.cc:336] T c4f65cdf00b748bb963ee6e81ef43b79 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111)
I20260709 13:23:13.215384 16790 leader_election.cc:304] T c4f65cdf00b748bb963ee6e81ef43b79 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:23:13.216156 25024 raft_consensus.cc:2749] T c4f65cdf00b748bb963ee6e81ef43b79 P cb82cc7bd6db46d980c75237a1402860 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 13:23:13.217175 16790 leader_election.cc:336] T c4f65cdf00b748bb963ee6e81ef43b79 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943): Network error: Client connection negotiation failed: client connection to 127.14.177.130:35943: connect: Connection refused (error 111)
W20260709 13:23:13.212860 16265 leader_election.cc:336] T 8158bbd17b994d2eb96f560bd0f1b88d P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943): Network error: Client connection negotiation failed: client connection to 127.14.177.130:35943: connect: Connection refused (error 111)
I20260709 13:23:13.218957 16265 leader_election.cc:304] T 8158bbd17b994d2eb96f560bd0f1b88d P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:23:13.219789 25038 raft_consensus.cc:2749] T 8158bbd17b994d2eb96f560bd0f1b88d P 90f3da57796b488c9087a2f523adf045 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:23:13.222029 25038 raft_consensus.cc:493] T c0eab25ce4c64e018a041beeda2867db P 90f3da57796b488c9087a2f523adf045 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:23:13.222611 25038 raft_consensus.cc:515] T c0eab25ce4c64e018a041beeda2867db P 90f3da57796b488c9087a2f523adf045 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:23:13.225209 25038 leader_election.cc:290] T c0eab25ce4c64e018a041beeda2867db P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
W20260709 13:23:13.240253 16265 leader_election.cc:336] T c0eab25ce4c64e018a041beeda2867db P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111)
W20260709 13:23:13.241101 16265 leader_election.cc:336] T c0eab25ce4c64e018a041beeda2867db P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943): Network error: Client connection negotiation failed: client connection to 127.14.177.130:35943: connect: Connection refused (error 111)
I20260709 13:23:13.241629 16265 leader_election.cc:304] T c0eab25ce4c64e018a041beeda2867db P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:23:13.242384 25038 raft_consensus.cc:2749] T c0eab25ce4c64e018a041beeda2867db P 90f3da57796b488c9087a2f523adf045 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:23:13.259467 25054 raft_consensus.cc:493] T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:23:13.260082 25054 raft_consensus.cc:515] T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:23:13.263875 16292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a378d7f796f94615b3b347119a9880fd" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:23:13.264570 16292 raft_consensus.cc:2468] T a378d7f796f94615b3b347119a9880fd P 90f3da57796b488c9087a2f523adf045 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 23.
I20260709 13:23:13.265833 16789 leader_election.cc:304] T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:23:13.266728 25024 raft_consensus.cc:2804] T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:23:13.267143 25024 raft_consensus.cc:493] T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860 [term 23 FOLLOWER]: Starting leader election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:23:13.267578 25024 raft_consensus.cc:3060] T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860 [term 23 FOLLOWER]: Advancing to term 24
W20260709 13:23:13.268437 16790 leader_election.cc:336] T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943): Network error: Client connection negotiation failed: client connection to 127.14.177.130:35943: connect: Connection refused (error 111)
I20260709 13:23:13.263906 25038 raft_consensus.cc:493] T 68a6838d82ef4cac83507f906139d90f P 90f3da57796b488c9087a2f523adf045 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:23:13.275391 25038 raft_consensus.cc:515] T 68a6838d82ef4cac83507f906139d90f P 90f3da57796b488c9087a2f523adf045 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:23:13.276257 25024 raft_consensus.cc:515] T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:23:13.278810 25054 leader_election.cc:290] T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:23:13.281288 16292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a378d7f796f94615b3b347119a9880fd" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:23:13.279536 25038 leader_election.cc:290] T 68a6838d82ef4cac83507f906139d90f P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:23:13.282752 16292 raft_consensus.cc:3060] T a378d7f796f94615b3b347119a9880fd P 90f3da57796b488c9087a2f523adf045 [term 23 FOLLOWER]: Advancing to term 24
W20260709 13:23:13.287093 16265 leader_election.cc:336] T 68a6838d82ef4cac83507f906139d90f P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111)
I20260709 13:23:13.291944 25054 raft_consensus.cc:493] T 8638ee363273426db1393dd81e5c7b72 P cb82cc7bd6db46d980c75237a1402860 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:23:13.292677 25054 raft_consensus.cc:515] T 8638ee363273426db1393dd81e5c7b72 P cb82cc7bd6db46d980c75237a1402860 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:23:13.295336 25054 leader_election.cc:290] T 8638ee363273426db1393dd81e5c7b72 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:23:13.298457 16294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8638ee363273426db1393dd81e5c7b72" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:23:13.299288 16294 raft_consensus.cc:2468] T 8638ee363273426db1393dd81e5c7b72 P 90f3da57796b488c9087a2f523adf045 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 24.
I20260709 13:23:13.300637 16789 leader_election.cc:304] T 8638ee363273426db1393dd81e5c7b72 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:23:13.303047 16292 raft_consensus.cc:2468] T a378d7f796f94615b3b347119a9880fd P 90f3da57796b488c9087a2f523adf045 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 24.
I20260709 13:23:13.283121 25024 leader_election.cc:290] T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 24 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:23:13.303323 25054 raft_consensus.cc:2804] T 8638ee363273426db1393dd81e5c7b72 P cb82cc7bd6db46d980c75237a1402860 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:23:13.307054 25054 raft_consensus.cc:493] T 8638ee363273426db1393dd81e5c7b72 P cb82cc7bd6db46d980c75237a1402860 [term 24 FOLLOWER]: Starting leader election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:23:13.307703 25054 raft_consensus.cc:3060] T 8638ee363273426db1393dd81e5c7b72 P cb82cc7bd6db46d980c75237a1402860 [term 24 FOLLOWER]: Advancing to term 25
W20260709 13:23:13.304814 16790 leader_election.cc:336] T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 24 election: RPC error from VoteRequest() call to peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943): Network error: Client connection negotiation failed: client connection to 127.14.177.130:35943: connect: Connection refused (error 111)
I20260709 13:23:13.308708 16789 leader_election.cc:304] T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac
I20260709 13:23:13.309636 25024 raft_consensus.cc:2804] T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 13:23:13.310225 25024 raft_consensus.cc:697] T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860 [term 24 LEADER]: Becoming Leader. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
W20260709 13:23:13.303112 16265 leader_election.cc:336] T 68a6838d82ef4cac83507f906139d90f P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943): Network error: Client connection negotiation failed: client connection to 127.14.177.130:35943: connect: Connection refused (error 111)
I20260709 13:23:13.311901 16265 leader_election.cc:304] T 68a6838d82ef4cac83507f906139d90f P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:23:13.312779 24975 raft_consensus.cc:2749] T 68a6838d82ef4cac83507f906139d90f P 90f3da57796b488c9087a2f523adf045 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:23:13.311229 25024 consensus_queue.cc:237] T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 23.15, Last appended by leader: 15, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:23:13.320856 24975 raft_consensus.cc:493] T e5ad4eb281564936a59f41e387a47156 P 90f3da57796b488c9087a2f523adf045 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:23:13.321419 24975 raft_consensus.cc:515] T e5ad4eb281564936a59f41e387a47156 P 90f3da57796b488c9087a2f523adf045 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:23:13.324596 24975 leader_election.cc:290] T e5ad4eb281564936a59f41e387a47156 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
W20260709 13:23:13.325191 16790 leader_election.cc:336] T 8638ee363273426db1393dd81e5c7b72 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943): Network error: Client connection negotiation failed: client connection to 127.14.177.130:35943: connect: Connection refused (error 111)
W20260709 13:23:13.327023 16790 consensus_peers.cc:597] T 23f82224f4564fea93572e8206bad1fe P cb82cc7bd6db46d980c75237a1402860 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 76: this message will repeat every 5th retry.
I20260709 13:23:13.333642 24975 raft_consensus.cc:493] T c4f65cdf00b748bb963ee6e81ef43b79 P 90f3da57796b488c9087a2f523adf045 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:23:13.334267 24975 raft_consensus.cc:515] T c4f65cdf00b748bb963ee6e81ef43b79 P 90f3da57796b488c9087a2f523adf045 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:23:13.337304 24975 leader_election.cc:290] T c4f65cdf00b748bb963ee6e81ef43b79 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:23:13.339944 16819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4f65cdf00b748bb963ee6e81ef43b79" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:23:13.340652 16819 raft_consensus.cc:2468] T c4f65cdf00b748bb963ee6e81ef43b79 P cb82cc7bd6db46d980c75237a1402860 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 23.
I20260709 13:23:13.347242 25054 raft_consensus.cc:515] T 8638ee363273426db1393dd81e5c7b72 P cb82cc7bd6db46d980c75237a1402860 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:23:13.356112 16265 leader_election.cc:336] T e5ad4eb281564936a59f41e387a47156 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111)
W20260709 13:23:13.364530 16265 leader_election.cc:336] T e5ad4eb281564936a59f41e387a47156 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943): Network error: Client connection negotiation failed: client connection to 127.14.177.130:35943: connect: Connection refused (error 111)
I20260709 13:23:13.365062 16265 leader_election.cc:304] T e5ad4eb281564936a59f41e387a47156 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:23:13.366084 24975 raft_consensus.cc:2749] T e5ad4eb281564936a59f41e387a47156 P 90f3da57796b488c9087a2f523adf045 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 13:23:13.366883 15046 thread.cc:527] Waited for 1000ms trying to join with diag-logger (tid 16247)
I20260709 13:23:13.368649 16292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8638ee363273426db1393dd81e5c7b72" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:23:13.368065 16044 catalog_manager.cc:5697] T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860 reported cstate change: term changed from 23 to 24, leader changed from 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130) to cb82cc7bd6db46d980c75237a1402860 (127.14.177.132). New cstate: current_term: 24 leader_uuid: "cb82cc7bd6db46d980c75237a1402860" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:23:13.369333 16292 raft_consensus.cc:3060] T 8638ee363273426db1393dd81e5c7b72 P 90f3da57796b488c9087a2f523adf045 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:23:13.374433 25054 leader_election.cc:290] T 8638ee363273426db1393dd81e5c7b72 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 25 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
W20260709 13:23:13.375509 16265 leader_election.cc:336] T c4f65cdf00b748bb963ee6e81ef43b79 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111)
W20260709 13:23:13.377758 16265 leader_election.cc:336] T c4f65cdf00b748bb963ee6e81ef43b79 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943): Network error: Client connection negotiation failed: client connection to 127.14.177.130:35943: connect: Connection refused (error 111)
I20260709 13:23:13.378252 16265 leader_election.cc:304] T c4f65cdf00b748bb963ee6e81ef43b79 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:23:13.382710 25038 raft_consensus.cc:2749] T c4f65cdf00b748bb963ee6e81ef43b79 P 90f3da57796b488c9087a2f523adf045 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 13:23:13.384982 16790 leader_election.cc:336] T 8638ee363273426db1393dd81e5c7b72 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 25 election: RPC error from VoteRequest() call to peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943): Network error: Client connection negotiation failed: client connection to 127.14.177.130:35943: connect: Connection refused (error 111)
I20260709 13:23:13.388414 24975 raft_consensus.cc:493] T 5a959dbecc2648eb85d1cc550f5a3245 P 90f3da57796b488c9087a2f523adf045 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:23:13.389143 24975 raft_consensus.cc:515] T 5a959dbecc2648eb85d1cc550f5a3245 P 90f3da57796b488c9087a2f523adf045 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:23:13.393888 16819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a959dbecc2648eb85d1cc550f5a3245" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:23:13.394591 16819 raft_consensus.cc:2468] T 5a959dbecc2648eb85d1cc550f5a3245 P cb82cc7bd6db46d980c75237a1402860 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 24.
I20260709 13:23:13.399627 24975 leader_election.cc:290] T 5a959dbecc2648eb85d1cc550f5a3245 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
W20260709 13:23:13.401750 16265 leader_election.cc:336] T 5a959dbecc2648eb85d1cc550f5a3245 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943): Network error: Client connection negotiation failed: client connection to 127.14.177.130:35943: connect: Connection refused (error 111)
W20260709 13:23:13.403273 16265 leader_election.cc:336] T 5a959dbecc2648eb85d1cc550f5a3245 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111)
I20260709 13:23:13.403728 16265 leader_election.cc:304] T 5a959dbecc2648eb85d1cc550f5a3245 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:23:13.405891 24975 raft_consensus.cc:2749] T 5a959dbecc2648eb85d1cc550f5a3245 P 90f3da57796b488c9087a2f523adf045 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:23:13.406790 16292 raft_consensus.cc:2468] T 8638ee363273426db1393dd81e5c7b72 P 90f3da57796b488c9087a2f523adf045 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 25.
I20260709 13:23:13.407464 25054 raft_consensus.cc:493] T d2ab0ac8347641e6955e9bce7d90c1a2 P cb82cc7bd6db46d980c75237a1402860 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:23:13.408186 16789 leader_election.cc:304] T 8638ee363273426db1393dd81e5c7b72 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac
I20260709 13:23:13.408128 25054 raft_consensus.cc:515] T d2ab0ac8347641e6955e9bce7d90c1a2 P cb82cc7bd6db46d980c75237a1402860 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:23:13.409646 25047 raft_consensus.cc:2804] T 8638ee363273426db1393dd81e5c7b72 P cb82cc7bd6db46d980c75237a1402860 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 13:23:13.410247 25047 raft_consensus.cc:697] T 8638ee363273426db1393dd81e5c7b72 P cb82cc7bd6db46d980c75237a1402860 [term 25 LEADER]: Becoming Leader. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:23:13.411260 25024 raft_consensus.cc:493] T 8c11074579b046d99e585c2818e18a2c P cb82cc7bd6db46d980c75237a1402860 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:23:13.411226 25047 consensus_queue.cc:237] T 8638ee363273426db1393dd81e5c7b72 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 24.17, Last appended by leader: 17, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:23:13.411916 25024 raft_consensus.cc:515] T 8c11074579b046d99e585c2818e18a2c P cb82cc7bd6db46d980c75237a1402860 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:23:13.413486 25054 leader_election.cc:290] T d2ab0ac8347641e6955e9bce7d90c1a2 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:23:13.414863 25024 leader_election.cc:290] T 8c11074579b046d99e585c2818e18a2c P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
W20260709 13:23:13.416406 16790 leader_election.cc:336] T d2ab0ac8347641e6955e9bce7d90c1a2 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943): Network error: Client connection negotiation failed: client connection to 127.14.177.130:35943: connect: Connection refused (error 111)
I20260709 13:23:13.416527 16300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c11074579b046d99e585c2818e18a2c" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:23:13.416838 16294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ab0ac8347641e6955e9bce7d90c1a2" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:23:13.418670 25050 raft_consensus.cc:493] T 3a7ca61592e848ea9b0fb81a2ce33cdc P cb82cc7bd6db46d980c75237a1402860 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:23:13.419337 25050 raft_consensus.cc:515] T 3a7ca61592e848ea9b0fb81a2ce33cdc P cb82cc7bd6db46d980c75237a1402860 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:23:13.421819 25050 leader_election.cc:290] T 3a7ca61592e848ea9b0fb81a2ce33cdc P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:23:13.423800 16789 leader_election.cc:304] T d2ab0ac8347641e6955e9bce7d90c1a2 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
W20260709 13:23:13.425796 16790 leader_election.cc:336] T 8c11074579b046d99e585c2818e18a2c P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943): Network error: Client connection negotiation failed: client connection to 127.14.177.130:35943: connect: Connection refused (error 111)
I20260709 13:23:13.425997 25024 raft_consensus.cc:2749] T d2ab0ac8347641e6955e9bce7d90c1a2 P cb82cc7bd6db46d980c75237a1402860 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:23:13.426369 16790 leader_election.cc:304] T 8c11074579b046d99e585c2818e18a2c P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:23:13.427008 25050 raft_consensus.cc:493] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:23:13.425820 16294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7ca61592e848ea9b0fb81a2ce33cdc" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:23:13.426819 25038 raft_consensus.cc:493] T 84f2da98864b4425bf7c0055951fdd4b P 90f3da57796b488c9087a2f523adf045 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:23:13.439503 25038 raft_consensus.cc:515] T 84f2da98864b4425bf7c0055951fdd4b P 90f3da57796b488c9087a2f523adf045 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
W20260709 13:23:13.440037 16790 leader_election.cc:336] T 3a7ca61592e848ea9b0fb81a2ce33cdc P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943): Network error: Client connection negotiation failed: client connection to 127.14.177.130:35943: connect: Connection refused (error 111)
I20260709 13:23:13.441982 25038 leader_election.cc:290] T 84f2da98864b4425bf7c0055951fdd4b P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:23:13.437480 25024 raft_consensus.cc:2749] T 8c11074579b046d99e585c2818e18a2c P cb82cc7bd6db46d980c75237a1402860 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:23:13.445990 16789 leader_election.cc:304] T 3a7ca61592e848ea9b0fb81a2ce33cdc P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:23:13.448640 25059 raft_consensus.cc:2749] T 3a7ca61592e848ea9b0fb81a2ce33cdc P cb82cc7bd6db46d980c75237a1402860 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 13:23:13.450515 16265 leader_election.cc:336] T 84f2da98864b4425bf7c0055951fdd4b P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111)
I20260709 13:23:13.438469 25050 raft_consensus.cc:515] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:23:13.455425 16265 leader_election.cc:336] T 84f2da98864b4425bf7c0055951fdd4b P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943): Network error: Client connection negotiation failed: client connection to 127.14.177.130:35943: connect: Connection refused (error 111)
I20260709 13:23:13.456291 16265 leader_election.cc:304] T 84f2da98864b4425bf7c0055951fdd4b P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:23:13.457942 25038 raft_consensus.cc:2749] T 84f2da98864b4425bf7c0055951fdd4b P 90f3da57796b488c9087a2f523adf045 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:23:13.458513 25024 raft_consensus.cc:493] T 829b4ad8c5a54eababeea1181b960d4b P cb82cc7bd6db46d980c75237a1402860 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:23:13.461112 25050 leader_election.cc:290] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:23:13.459036 25024 raft_consensus.cc:515] T 829b4ad8c5a54eababeea1181b960d4b P cb82cc7bd6db46d980c75237a1402860 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:23:13.468189 16790 leader_election.cc:336] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943): Network error: Client connection negotiation failed: client connection to 127.14.177.130:35943: connect: Connection refused (error 111)
W20260709 13:23:13.468910 16790 leader_election.cc:336] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111)
I20260709 13:23:13.469332 16790 leader_election.cc:304] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:23:13.469396 25024 leader_election.cc:290] T 829b4ad8c5a54eababeea1181b960d4b P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:23:13.470141 25024 raft_consensus.cc:2749] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 13:23:13.475024 25051 raft_consensus.cc:493] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 90f3da57796b488c9087a2f523adf045 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:23:13.475111 25059 raft_consensus.cc:493] T 18ad7e3af5504571a69d10c00750a01c P cb82cc7bd6db46d980c75237a1402860 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:23:13.475916 25051 raft_consensus.cc:515] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 90f3da57796b488c9087a2f523adf045 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:23:13.476128 25059 raft_consensus.cc:515] T 18ad7e3af5504571a69d10c00750a01c P cb82cc7bd6db46d980c75237a1402860 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:23:13.479101 25051 leader_election.cc:290] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:23:13.479333 16824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7ca61592e848ea9b0fb81a2ce33cdc" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:23:13.480073 16824 raft_consensus.cc:2468] T 3a7ca61592e848ea9b0fb81a2ce33cdc P cb82cc7bd6db46d980c75237a1402860 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 23.
I20260709 13:23:13.481030 25024 raft_consensus.cc:493] T af57996a8be84d398f0f023e8d08ef77 P cb82cc7bd6db46d980c75237a1402860 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:23:13.481492 16266 leader_election.cc:304] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:23:13.482897 25051 raft_consensus.cc:2804] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 90f3da57796b488c9087a2f523adf045 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:23:13.483386 25051 raft_consensus.cc:493] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 90f3da57796b488c9087a2f523adf045 [term 23 FOLLOWER]: Starting leader election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:23:13.484164 25051 raft_consensus.cc:3060] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 90f3da57796b488c9087a2f523adf045 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:23:13.487854 25050 raft_consensus.cc:493] T ee9ed67006144797b1de6ea8c2013a68 P cb82cc7bd6db46d980c75237a1402860 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:23:13.489538 25050 raft_consensus.cc:515] T ee9ed67006144797b1de6ea8c2013a68 P cb82cc7bd6db46d980c75237a1402860 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:23:13.486377 25024 raft_consensus.cc:515] T af57996a8be84d398f0f023e8d08ef77 P cb82cc7bd6db46d980c75237a1402860 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:23:13.491314 16265 leader_election.cc:336] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943): Network error: Client connection negotiation failed: client connection to 127.14.177.130:35943: connect: Connection refused (error 111)
I20260709 13:23:13.493633 16292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18ad7e3af5504571a69d10c00750a01c" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 20 candidate_status { last_received { term: 19 index: 17 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:23:13.493605 25059 leader_election.cc:290] T 18ad7e3af5504571a69d10c00750a01c P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:23:13.500901 25051 raft_consensus.cc:515] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 90f3da57796b488c9087a2f523adf045 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:23:13.502923 25024 leader_election.cc:290] T af57996a8be84d398f0f023e8d08ef77 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:23:13.503307 25051 leader_election.cc:290] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 24 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:23:13.500927 25050 leader_election.cc:290] T ee9ed67006144797b1de6ea8c2013a68 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:23:13.504411 16824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a7ca61592e848ea9b0fb81a2ce33cdc" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:23:13.507812 16824 raft_consensus.cc:3060] T 3a7ca61592e848ea9b0fb81a2ce33cdc P cb82cc7bd6db46d980c75237a1402860 [term 23 FOLLOWER]: Advancing to term 24
W20260709 13:23:13.504411 16790 leader_election.cc:336] T 829b4ad8c5a54eababeea1181b960d4b P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111)
I20260709 13:23:13.512662 25038 raft_consensus.cc:493] T d2ab0ac8347641e6955e9bce7d90c1a2 P 90f3da57796b488c9087a2f523adf045 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:23:13.513019 24975 raft_consensus.cc:493] T ada5a004ba7c46819051e3ccec71ffdb P 90f3da57796b488c9087a2f523adf045 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:23:13.513317 25038 raft_consensus.cc:515] T d2ab0ac8347641e6955e9bce7d90c1a2 P 90f3da57796b488c9087a2f523adf045 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:23:13.513625 24975 raft_consensus.cc:515] T ada5a004ba7c46819051e3ccec71ffdb P 90f3da57796b488c9087a2f523adf045 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:23:13.514719 16044 catalog_manager.cc:5697] T 8638ee363273426db1393dd81e5c7b72 P cb82cc7bd6db46d980c75237a1402860 reported cstate change: term changed from 24 to 25, leader changed from 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130) to cb82cc7bd6db46d980c75237a1402860 (127.14.177.132). New cstate: current_term: 25 leader_uuid: "cb82cc7bd6db46d980c75237a1402860" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:23:13.515808 24975 leader_election.cc:290] T ada5a004ba7c46819051e3ccec71ffdb P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:23:13.516964 16819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ab0ac8347641e6955e9bce7d90c1a2" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:23:13.517282 16825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada5a004ba7c46819051e3ccec71ffdb" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 26 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:23:13.517570 16819 raft_consensus.cc:2468] T d2ab0ac8347641e6955e9bce7d90c1a2 P cb82cc7bd6db46d980c75237a1402860 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 21.
I20260709 13:23:13.519074 16266 leader_election.cc:304] T d2ab0ac8347641e6955e9bce7d90c1a2 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 
W20260709 13:23:13.519850 16790 leader_election.cc:336] T 829b4ad8c5a54eababeea1181b960d4b P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943): Network error: Client connection negotiation failed: client connection to 127.14.177.130:35943: connect: Connection refused (error 111)
I20260709 13:23:13.520210 25038 leader_election.cc:290] T d2ab0ac8347641e6955e9bce7d90c1a2 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:23:13.520393 16790 leader_election.cc:304] T 829b4ad8c5a54eababeea1181b960d4b P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
W20260709 13:23:13.513391 16265 leader_election.cc:336] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 24 election: RPC error from VoteRequest() call to peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943): Network error: Client connection negotiation failed: client connection to 127.14.177.130:35943: connect: Connection refused (error 111)
I20260709 13:23:13.519927 24975 raft_consensus.cc:2804] T d2ab0ac8347641e6955e9bce7d90c1a2 P 90f3da57796b488c9087a2f523adf045 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:23:13.529055 24975 raft_consensus.cc:493] T d2ab0ac8347641e6955e9bce7d90c1a2 P 90f3da57796b488c9087a2f523adf045 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:23:13.529564 24975 raft_consensus.cc:3060] T d2ab0ac8347641e6955e9bce7d90c1a2 P 90f3da57796b488c9087a2f523adf045 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:23:13.529995 25060 raft_consensus.cc:493] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:23:13.530575 25060 raft_consensus.cc:515] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:23:13.532804 25060 leader_election.cc:290] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:23:13.528081 25050 raft_consensus.cc:2749] T 829b4ad8c5a54eababeea1181b960d4b P cb82cc7bd6db46d980c75237a1402860 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:23:13.538664 16265 leader_election.cc:336] T d2ab0ac8347641e6955e9bce7d90c1a2 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943): Network error: Client connection negotiation failed: client connection to 127.14.177.130:35943: connect: Connection refused (error 111)
W20260709 13:23:13.539570 16265 leader_election.cc:336] T ada5a004ba7c46819051e3ccec71ffdb P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943): Network error: Client connection negotiation failed: client connection to 127.14.177.130:35943: connect: Connection refused (error 111)
I20260709 13:23:13.539992 16265 leader_election.cc:304] T ada5a004ba7c46819051e3ccec71ffdb P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, cb82cc7bd6db46d980c75237a1402860
I20260709 13:23:13.540730 25062 raft_consensus.cc:2749] T ada5a004ba7c46819051e3ccec71ffdb P 90f3da57796b488c9087a2f523adf045 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:23:13.541126 16824 raft_consensus.cc:2468] T 3a7ca61592e848ea9b0fb81a2ce33cdc P cb82cc7bd6db46d980c75237a1402860 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 24.
I20260709 13:23:13.542750 16266 leader_election.cc:304] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac
I20260709 13:23:13.543599 25062 raft_consensus.cc:2804] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 90f3da57796b488c9087a2f523adf045 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 13:23:13.544486 25062 raft_consensus.cc:697] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 90f3da57796b488c9087a2f523adf045 [term 24 LEADER]: Becoming Leader. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:23:13.545462 25062 consensus_queue.cc:237] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 90f3da57796b488c9087a2f523adf045 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 23.17, Last appended by leader: 17, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:23:13.546307 24975 raft_consensus.cc:515] T d2ab0ac8347641e6955e9bce7d90c1a2 P 90f3da57796b488c9087a2f523adf045 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:23:13.545835 16790 leader_election.cc:336] T ee9ed67006144797b1de6ea8c2013a68 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111)
W20260709 13:23:13.547585 16790 leader_election.cc:336] T af57996a8be84d398f0f023e8d08ef77 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111)
W20260709 13:23:13.549072 16790 leader_election.cc:336] T 18ad7e3af5504571a69d10c00750a01c P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943): Network error: Client connection negotiation failed: client connection to 127.14.177.130:35943: connect: Connection refused (error 111)
I20260709 13:23:13.549506 16819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2ab0ac8347641e6955e9bce7d90c1a2" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:23:13.550372 16819 raft_consensus.cc:3060] T d2ab0ac8347641e6955e9bce7d90c1a2 P cb82cc7bd6db46d980c75237a1402860 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:23:13.549504 16790 leader_election.cc:304] T 18ad7e3af5504571a69d10c00750a01c P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
W20260709 13:23:13.558297 16265 leader_election.cc:336] T d2ab0ac8347641e6955e9bce7d90c1a2 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943): Network error: Client connection negotiation failed: client connection to 127.14.177.130:35943: connect: Connection refused (error 111)
I20260709 13:23:13.560596 24975 leader_election.cc:290] T d2ab0ac8347641e6955e9bce7d90c1a2 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 22 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
W20260709 13:23:13.561430 16790 leader_election.cc:336] T ee9ed67006144797b1de6ea8c2013a68 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943): Network error: Client connection negotiation failed: client connection to 127.14.177.130:35943: connect: Connection refused (error 111)
I20260709 13:23:13.561905 16790 leader_election.cc:304] T ee9ed67006144797b1de6ea8c2013a68 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:23:13.560453 25024 raft_consensus.cc:493] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:23:13.561175 25050 raft_consensus.cc:2749] T 18ad7e3af5504571a69d10c00750a01c P cb82cc7bd6db46d980c75237a1402860 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:23:13.563395 25024 raft_consensus.cc:515] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:23:13.564469 16819 raft_consensus.cc:2468] T d2ab0ac8347641e6955e9bce7d90c1a2 P cb82cc7bd6db46d980c75237a1402860 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 22.
I20260709 13:23:13.565037 24975 raft_consensus.cc:493] T 18ad7e3af5504571a69d10c00750a01c P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
W20260709 13:23:13.565577 16790 leader_election.cc:336] T af57996a8be84d398f0f023e8d08ef77 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943): Network error: Client connection negotiation failed: client connection to 127.14.177.130:35943: connect: Connection refused (error 111)
I20260709 13:23:13.569475 16790 leader_election.cc:304] T af57996a8be84d398f0f023e8d08ef77 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:23:13.571290 25060 raft_consensus.cc:2749] T af57996a8be84d398f0f023e8d08ef77 P cb82cc7bd6db46d980c75237a1402860 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:23:13.575829 16044 catalog_manager.cc:5697] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 90f3da57796b488c9087a2f523adf045 reported cstate change: term changed from 23 to 24, leader changed from 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130) to 90f3da57796b488c9087a2f523adf045 (127.14.177.131). New cstate: current_term: 24 leader_uuid: "90f3da57796b488c9087a2f523adf045" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:23:13.566097 25059 raft_consensus.cc:2749] T ee9ed67006144797b1de6ea8c2013a68 P cb82cc7bd6db46d980c75237a1402860 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 13:23:13.579574 16790 consensus_peers.cc:597] T 9e65676b21b748728ad39668cfe722ec P cb82cc7bd6db46d980c75237a1402860 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:23:13.580113 16790 consensus_peers.cc:597] T 2794af5b99984efebdee564035841cb0 P cb82cc7bd6db46d980c75237a1402860 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 76: this message will repeat every 5th retry.
W20260709 13:23:13.580500 16790 leader_election.cc:336] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111)
W20260709 13:23:13.581116 16790 leader_election.cc:336] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943): Network error: Client connection negotiation failed: client connection to 127.14.177.130:35943: connect: Connection refused (error 111)
I20260709 13:23:13.581512 16790 leader_election.cc:304] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:23:13.565743 16266 leader_election.cc:304] T d2ab0ac8347641e6955e9bce7d90c1a2 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac
I20260709 13:23:13.582271 25060 raft_consensus.cc:2749] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:23:13.582594 25062 raft_consensus.cc:2804] T d2ab0ac8347641e6955e9bce7d90c1a2 P 90f3da57796b488c9087a2f523adf045 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:23:13.583209 25062 raft_consensus.cc:697] T d2ab0ac8347641e6955e9bce7d90c1a2 P 90f3da57796b488c9087a2f523adf045 [term 22 LEADER]: Becoming Leader. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:23:13.584462 25051 raft_consensus.cc:493] T 8c11074579b046d99e585c2818e18a2c P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:23:13.584157 25062 consensus_queue.cc:237] T d2ab0ac8347641e6955e9bce7d90c1a2 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 21.17, Last appended by leader: 17, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:23:13.585035 25051 raft_consensus.cc:515] T 8c11074579b046d99e585c2818e18a2c P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:23:13.589712 25051 leader_election.cc:290] T 8c11074579b046d99e585c2818e18a2c P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
W20260709 13:23:13.592021 16265 leader_election.cc:336] T 8c11074579b046d99e585c2818e18a2c P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943): Network error: Client connection negotiation failed: client connection to 127.14.177.130:35943: connect: Connection refused (error 111)
I20260709 13:23:13.594282 16819 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c11074579b046d99e585c2818e18a2c" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:23:13.594936 16819 raft_consensus.cc:2468] T 8c11074579b046d99e585c2818e18a2c P cb82cc7bd6db46d980c75237a1402860 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 20.
I20260709 13:23:13.596700 16266 leader_election.cc:304] T 8c11074579b046d99e585c2818e18a2c P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac
I20260709 13:23:13.597599 25038 raft_consensus.cc:2804] T 8c11074579b046d99e585c2818e18a2c P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:23:13.597992 25038 raft_consensus.cc:493] T 8c11074579b046d99e585c2818e18a2c P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:23:13.598337 25038 raft_consensus.cc:3060] T 8c11074579b046d99e585c2818e18a2c P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:23:13.609436 16819 raft_consensus.cc:1275] T 8d4ab870a05347868956aa8b841b2981 P cb82cc7bd6db46d980c75237a1402860 [term 24 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 23 index: 17. Preceding OpId from leader: term: 24 index: 18. (index mismatch)
I20260709 13:23:13.568821 24975 raft_consensus.cc:515] T 18ad7e3af5504571a69d10c00750a01c P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:23:13.617761 25038 raft_consensus.cc:515] T 8c11074579b046d99e585c2818e18a2c P 90f3da57796b488c9087a2f523adf045 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:23:13.619484 24975 leader_election.cc:290] T 18ad7e3af5504571a69d10c00750a01c P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:23:13.620146 25038 leader_election.cc:290] T 8c11074579b046d99e585c2818e18a2c P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 21 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:23:13.620676 25062 consensus_queue.cc:1048] T 8d4ab870a05347868956aa8b841b2981 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.000s
I20260709 13:23:13.626679 16044 catalog_manager.cc:5697] T d2ab0ac8347641e6955e9bce7d90c1a2 P 90f3da57796b488c9087a2f523adf045 reported cstate change: term changed from 21 to 22, leader changed from 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130) to 90f3da57796b488c9087a2f523adf045 (127.14.177.131). New cstate: current_term: 22 leader_uuid: "90f3da57796b488c9087a2f523adf045" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 13:23:13.629681 16265 leader_election.cc:336] T 18ad7e3af5504571a69d10c00750a01c P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943): Network error: Client connection negotiation failed: client connection to 127.14.177.130:35943: connect: Connection refused (error 111)
I20260709 13:23:13.629916 25024 leader_election.cc:290] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
W20260709 13:23:13.630256 16265 leader_election.cc:336] T 8c11074579b046d99e585c2818e18a2c P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 21 election: RPC error from VoteRequest() call to peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943): Network error: Client connection negotiation failed: client connection to 127.14.177.130:35943: connect: Connection refused (error 111)
W20260709 13:23:13.631057 16790 consensus_peers.cc:597] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 66: this message will repeat every 5th retry.
I20260709 13:23:13.638192 16824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18ad7e3af5504571a69d10c00750a01c" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 20 candidate_status { last_received { term: 19 index: 17 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:23:13.638939 16824 raft_consensus.cc:2468] T 18ad7e3af5504571a69d10c00750a01c P cb82cc7bd6db46d980c75237a1402860 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 19.
I20260709 13:23:13.639889 16825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c11074579b046d99e585c2818e18a2c" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:23:13.640504 16825 raft_consensus.cc:3060] T 8c11074579b046d99e585c2818e18a2c P cb82cc7bd6db46d980c75237a1402860 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:23:13.645053 16266 leader_election.cc:304] T 18ad7e3af5504571a69d10c00750a01c P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac
I20260709 13:23:13.646783 25062 raft_consensus.cc:2804] T 18ad7e3af5504571a69d10c00750a01c P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:23:13.647202 25062 raft_consensus.cc:493] T 18ad7e3af5504571a69d10c00750a01c P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:23:13.648065 25024 raft_consensus.cc:493] T ada5a004ba7c46819051e3ccec71ffdb P cb82cc7bd6db46d980c75237a1402860 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:23:13.648662 25024 raft_consensus.cc:515] T ada5a004ba7c46819051e3ccec71ffdb P cb82cc7bd6db46d980c75237a1402860 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:23:13.650873 25024 leader_election.cc:290] T ada5a004ba7c46819051e3ccec71ffdb P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:23:13.652817 16292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada5a004ba7c46819051e3ccec71ffdb" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 26 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:23:13.653499 16292 raft_consensus.cc:2468] T ada5a004ba7c46819051e3ccec71ffdb P 90f3da57796b488c9087a2f523adf045 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 25.
I20260709 13:23:13.654665 25024 raft_consensus.cc:493] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P cb82cc7bd6db46d980c75237a1402860 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:23:13.655310 25024 raft_consensus.cc:515] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P cb82cc7bd6db46d980c75237a1402860 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:23:13.657311 16825 raft_consensus.cc:2468] T 8c11074579b046d99e585c2818e18a2c P cb82cc7bd6db46d980c75237a1402860 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 21.
I20260709 13:23:13.659106 16789 leader_election.cc:304] T ada5a004ba7c46819051e3ccec71ffdb P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:23:13.660071 25050 raft_consensus.cc:2804] T ada5a004ba7c46819051e3ccec71ffdb P cb82cc7bd6db46d980c75237a1402860 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 13:23:13.660573 25050 raft_consensus.cc:493] T ada5a004ba7c46819051e3ccec71ffdb P cb82cc7bd6db46d980c75237a1402860 [term 25 FOLLOWER]: Starting leader election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:23:13.660948 25050 raft_consensus.cc:3060] T ada5a004ba7c46819051e3ccec71ffdb P cb82cc7bd6db46d980c75237a1402860 [term 25 FOLLOWER]: Advancing to term 26
W20260709 13:23:13.662354 16790 leader_election.cc:336] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111)
I20260709 13:23:13.667970 16292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e4bfb9090c8460eb14fa7e0d818e7f1" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 25 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045" is_pre_election: true
I20260709 13:23:13.670819 16266 leader_election.cc:304] T 8c11074579b046d99e585c2818e18a2c P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac
I20260709 13:23:13.672360 25038 raft_consensus.cc:2804] T 8c11074579b046d99e585c2818e18a2c P 90f3da57796b488c9087a2f523adf045 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:23:13.673153 25038 raft_consensus.cc:697] T 8c11074579b046d99e585c2818e18a2c P 90f3da57796b488c9087a2f523adf045 [term 21 LEADER]: Becoming Leader. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:23:13.658217 25024 leader_election.cc:290] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:23:13.674306 25038 consensus_queue.cc:237] T 8c11074579b046d99e585c2818e18a2c P 90f3da57796b488c9087a2f523adf045 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 20.16, Last appended by leader: 16, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:23:13.677188 16265 consensus_peers.cc:597] T 8d4ab870a05347868956aa8b841b2981 P 90f3da57796b488c9087a2f523adf045 -> Peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943): Couldn't send request to peer 1ca2725d290e47d68b1a4b609c9816ac. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.130:35943: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:23:13.647593 25062 raft_consensus.cc:3060] T 18ad7e3af5504571a69d10c00750a01c P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:23:13.689851 25050 raft_consensus.cc:515] T ada5a004ba7c46819051e3ccec71ffdb P cb82cc7bd6db46d980c75237a1402860 [term 26 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
W20260709 13:23:13.702665 16035 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.14.177.129:36741, user_credentials={real_user=slave}} blocked reactor thread for 61440.3us
W20260709 13:23:13.703568 16790 leader_election.cc:336] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943): Network error: Client connection negotiation failed: client connection to 127.14.177.130:35943: connect: Connection refused (error 111)
I20260709 13:23:13.706542 16790 leader_election.cc:304] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
W20260709 13:23:13.705313 16035 catalog_manager.cc:4755] TS 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet 19baa7bc367844c288bca8f81439c413: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111)
I20260709 13:23:13.706082 25050 leader_election.cc:290] T ada5a004ba7c46819051e3ccec71ffdb P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 26 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:23:13.704469 25062 raft_consensus.cc:515] T 18ad7e3af5504571a69d10c00750a01c P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:23:13.708029 25024 raft_consensus.cc:2749] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:23:13.708215 16292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada5a004ba7c46819051e3ccec71ffdb" candidate_uuid: "cb82cc7bd6db46d980c75237a1402860" candidate_term: 26 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "90f3da57796b488c9087a2f523adf045"
I20260709 13:23:13.708992 16292 raft_consensus.cc:3060] T ada5a004ba7c46819051e3ccec71ffdb P 90f3da57796b488c9087a2f523adf045 [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:23:13.710705 25062 leader_election.cc:290] T 18ad7e3af5504571a69d10c00750a01c P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 20 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
W20260709 13:23:13.715837 16265 leader_election.cc:336] T 18ad7e3af5504571a69d10c00750a01c P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 20 election: RPC error from VoteRequest() call to peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943): Network error: Client connection negotiation failed: client connection to 127.14.177.130:35943: connect: Connection refused (error 111)
I20260709 13:23:13.722091 16820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18ad7e3af5504571a69d10c00750a01c" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 20 candidate_status { last_received { term: 19 index: 17 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:23:13.722849 16820 raft_consensus.cc:3060] T 18ad7e3af5504571a69d10c00750a01c P cb82cc7bd6db46d980c75237a1402860 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:23:13.723372 25038 raft_consensus.cc:493] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:23:13.723938 25038 raft_consensus.cc:515] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:23:13.724462 16292 raft_consensus.cc:2468] T ada5a004ba7c46819051e3ccec71ffdb P 90f3da57796b488c9087a2f523adf045 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb82cc7bd6db46d980c75237a1402860 in term 26.
I20260709 13:23:13.725764 16789 leader_election.cc:304] T ada5a004ba7c46819051e3ccec71ffdb P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:23:13.726325 16044 catalog_manager.cc:5697] T 8c11074579b046d99e585c2818e18a2c P 90f3da57796b488c9087a2f523adf045 reported cstate change: term changed from 20 to 21, leader changed from 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130) to 90f3da57796b488c9087a2f523adf045 (127.14.177.131). New cstate: current_term: 21 leader_uuid: "90f3da57796b488c9087a2f523adf045" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:23:13.728425 25024 raft_consensus.cc:2804] T ada5a004ba7c46819051e3ccec71ffdb P cb82cc7bd6db46d980c75237a1402860 [term 26 FOLLOWER]: Leader election won for term 26
I20260709 13:23:13.729914 25051 raft_consensus.cc:493] T 5a96373bb0e14dce8a266b2fc6c58507 P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:23:13.730495 25024 raft_consensus.cc:697] T ada5a004ba7c46819051e3ccec71ffdb P cb82cc7bd6db46d980c75237a1402860 [term 26 LEADER]: Becoming Leader. State: Replica: cb82cc7bd6db46d980c75237a1402860, State: Running, Role: LEADER
I20260709 13:23:13.730515 25051 raft_consensus.cc:515] T 5a96373bb0e14dce8a266b2fc6c58507 P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:23:13.731494 25024 consensus_queue.cc:237] T ada5a004ba7c46819051e3ccec71ffdb P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 25.18, Last appended by leader: 18, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:23:13.726079 25038 leader_election.cc:290] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:23:13.733299 25051 leader_election.cc:290] T 5a96373bb0e14dce8a266b2fc6c58507 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:23:13.737673 16825 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e4bfb9090c8460eb14fa7e0d818e7f1" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 25 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:23:13.738399 16825 raft_consensus.cc:2468] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P cb82cc7bd6db46d980c75237a1402860 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 24.
I20260709 13:23:13.740056 16266 leader_election.cc:304] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 
I20260709 13:23:13.740937 25038 raft_consensus.cc:2804] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:23:13.741328 25038 raft_consensus.cc:493] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045 [term 24 FOLLOWER]: Starting leader election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:23:13.741688 25038 raft_consensus.cc:3060] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045 [term 24 FOLLOWER]: Advancing to term 25
W20260709 13:23:13.743486 16265 leader_election.cc:336] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943): Network error: Client connection negotiation failed: client connection to 127.14.177.130:35943: connect: Connection refused (error 111)
W20260709 13:23:13.738886 16790 leader_election.cc:336] T ada5a004ba7c46819051e3ccec71ffdb P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943): Network error: Client connection negotiation failed: client connection to 127.14.177.130:35943: connect: Connection refused (error 111)
W20260709 13:23:13.754694 16790 leader_election.cc:336] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943): Network error: Client connection negotiation failed: client connection to 127.14.177.130:35943: connect: Connection refused (error 111)
I20260709 13:23:13.755234 16790 leader_election.cc:304] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
W20260709 13:23:13.764137 16265 leader_election.cc:336] T 5a96373bb0e14dce8a266b2fc6c58507 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943): Network error: Client connection negotiation failed: client connection to 127.14.177.130:35943: connect: Connection refused (error 111)
W20260709 13:23:13.765364 16265 leader_election.cc:336] T 5a96373bb0e14dce8a266b2fc6c58507 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111)
I20260709 13:23:13.765836 16265 leader_election.cc:304] T 5a96373bb0e14dce8a266b2fc6c58507 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:23:13.759878 25038 raft_consensus.cc:515] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:23:13.764408 16820 raft_consensus.cc:2468] T 18ad7e3af5504571a69d10c00750a01c P cb82cc7bd6db46d980c75237a1402860 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 20.
I20260709 13:23:13.769080 25062 raft_consensus.cc:2749] T 5a96373bb0e14dce8a266b2fc6c58507 P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:23:13.763443 25050 raft_consensus.cc:2749] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P cb82cc7bd6db46d980c75237a1402860 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:23:13.770625 25038 leader_election.cc:290] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 25 election: Requested vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
W20260709 13:23:13.771739 16790 leader_election.cc:336] T ada5a004ba7c46819051e3ccec71ffdb P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 26 election: RPC error from VoteRequest() call to peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943): Network error: Client connection negotiation failed: client connection to 127.14.177.130:35943: connect: Connection refused (error 111)
I20260709 13:23:13.773669 16266 leader_election.cc:304] T 18ad7e3af5504571a69d10c00750a01c P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac
I20260709 13:23:13.775417 25062 raft_consensus.cc:2804] T 18ad7e3af5504571a69d10c00750a01c P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:23:13.775933 25062 raft_consensus.cc:697] T 18ad7e3af5504571a69d10c00750a01c P 90f3da57796b488c9087a2f523adf045 [term 20 LEADER]: Becoming Leader. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:23:13.776865 25062 consensus_queue.cc:237] T 18ad7e3af5504571a69d10c00750a01c P 90f3da57796b488c9087a2f523adf045 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 19.17, Last appended by leader: 17, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:23:13.779346 16824 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e4bfb9090c8460eb14fa7e0d818e7f1" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 25 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860"
I20260709 13:23:13.780149 16824 raft_consensus.cc:3060] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P cb82cc7bd6db46d980c75237a1402860 [term 24 FOLLOWER]: Advancing to term 25
W20260709 13:23:13.783576 16790 consensus_peers.cc:597] T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860 -> Peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943): Couldn't send request to peer 1ca2725d290e47d68b1a4b609c9816ac. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.130:35943: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:23:13.792109 16040 catalog_manager.cc:5697] T ada5a004ba7c46819051e3ccec71ffdb P cb82cc7bd6db46d980c75237a1402860 reported cstate change: term changed from 25 to 26, leader changed from 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130) to cb82cc7bd6db46d980c75237a1402860 (127.14.177.132). New cstate: current_term: 26 leader_uuid: "cb82cc7bd6db46d980c75237a1402860" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:23:13.797250 16265 leader_election.cc:336] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 25 election: RPC error from VoteRequest() call to peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943): Network error: Client connection negotiation failed: client connection to 127.14.177.130:35943: connect: Connection refused (error 111)
I20260709 13:23:13.799399 16824 raft_consensus.cc:2468] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P cb82cc7bd6db46d980c75237a1402860 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 25.
I20260709 13:23:13.800704 16266 leader_election.cc:304] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac
I20260709 13:23:13.802513 25062 raft_consensus.cc:2804] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 13:23:13.805258 25062 raft_consensus.cc:697] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045 [term 25 LEADER]: Becoming Leader. State: Replica: 90f3da57796b488c9087a2f523adf045, State: Running, Role: LEADER
I20260709 13:23:13.806334 25062 consensus_queue.cc:237] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 24.18, Last appended by leader: 18, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:23:13.814759 16040 catalog_manager.cc:5697] T 18ad7e3af5504571a69d10c00750a01c P 90f3da57796b488c9087a2f523adf045 reported cstate change: term changed from 19 to 20, leader changed from 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130) to 90f3da57796b488c9087a2f523adf045 (127.14.177.131). New cstate: current_term: 20 leader_uuid: "90f3da57796b488c9087a2f523adf045" committed_config { opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 13:23:13.835640 16265 consensus_peers.cc:597] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 71: this message will repeat every 5th retry.
W20260709 13:23:13.845141 16265 consensus_peers.cc:597] T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:23:13.853609 16790 consensus_peers.cc:597] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
I20260709 13:23:13.870993 16040 catalog_manager.cc:5697] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045 reported cstate change: term changed from 24 to 25, leader changed from 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130) to 90f3da57796b488c9087a2f523adf045 (127.14.177.131). New cstate: current_term: 25 leader_uuid: "90f3da57796b488c9087a2f523adf045" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 13:23:13.912756 16265 consensus_peers.cc:597] T 02838c86531d4f98bf58d1c22b8baf77 P 90f3da57796b488c9087a2f523adf045 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
I20260709 13:23:13.918889 16292 raft_consensus.cc:1275] T a378d7f796f94615b3b347119a9880fd P 90f3da57796b488c9087a2f523adf045 [term 24 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 23 index: 15. Preceding OpId from leader: term: 24 index: 16. (index mismatch)
I20260709 13:23:13.920694 25024 consensus_queue.cc:1048] T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 13:23:13.946434 16292 raft_consensus.cc:1275] T 8638ee363273426db1393dd81e5c7b72 P 90f3da57796b488c9087a2f523adf045 [term 25 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 24 index: 17. Preceding OpId from leader: term: 25 index: 18. (index mismatch)
I20260709 13:23:13.948463 25024 consensus_queue.cc:1048] T 8638ee363273426db1393dd81e5c7b72 P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.000s
W20260709 13:23:13.960878 16790 consensus_peers.cc:597] T 8638ee363273426db1393dd81e5c7b72 P cb82cc7bd6db46d980c75237a1402860 -> Peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943): Couldn't send request to peer 1ca2725d290e47d68b1a4b609c9816ac. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.130:35943: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:23:13.993728 16265 consensus_peers.cc:597] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 90f3da57796b488c9087a2f523adf045 -> Peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943): Couldn't send request to peer 1ca2725d290e47d68b1a4b609c9816ac. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.130:35943: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:23:13.994964 16824 raft_consensus.cc:1275] T 3a7ca61592e848ea9b0fb81a2ce33cdc P cb82cc7bd6db46d980c75237a1402860 [term 24 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 23 index: 17. Preceding OpId from leader: term: 24 index: 18. (index mismatch)
I20260709 13:23:14.000658 25038 consensus_queue.cc:1048] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.001s
W20260709 13:23:14.062759 16790 consensus_peers.cc:597] T 66f75124215f4ca6bef831e174a792d8 P cb82cc7bd6db46d980c75237a1402860 -> Peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943): Couldn't send request to peer 1ca2725d290e47d68b1a4b609c9816ac. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.130:35943: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:23:14.078775 16265 consensus_peers.cc:597] T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:23:14.134713 16265 consensus_peers.cc:597] T 434bff3082d141838068f323bdb0d1ed P 90f3da57796b488c9087a2f523adf045 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 71: this message will repeat every 5th retry.
I20260709 13:23:14.136734 16820 raft_consensus.cc:1275] T d2ab0ac8347641e6955e9bce7d90c1a2 P cb82cc7bd6db46d980c75237a1402860 [term 22 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 21 index: 17. Preceding OpId from leader: term: 22 index: 18. (index mismatch)
I20260709 13:23:14.138705 25051 consensus_queue.cc:1048] T d2ab0ac8347641e6955e9bce7d90c1a2 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.000s
W20260709 13:23:14.148597 16265 consensus_peers.cc:597] T 8c11074579b046d99e585c2818e18a2c P 90f3da57796b488c9087a2f523adf045 -> Peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943): Couldn't send request to peer 1ca2725d290e47d68b1a4b609c9816ac. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.130:35943: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:23:14.165628 16265 consensus_peers.cc:597] T d2ab0ac8347641e6955e9bce7d90c1a2 P 90f3da57796b488c9087a2f523adf045 -> Peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943): Couldn't send request to peer 1ca2725d290e47d68b1a4b609c9816ac. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.130:35943: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:23:14.172211 16790 consensus_peers.cc:597] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 71: this message will repeat every 5th retry.
W20260709 13:23:14.193676 16265 consensus_peers.cc:597] T fffab26427e24d398e2695aad40ca134 P 90f3da57796b488c9087a2f523adf045 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 71: this message will repeat every 5th retry.
W20260709 13:23:14.198055 16265 consensus_peers.cc:597] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045 -> Peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943): Couldn't send request to peer 1ca2725d290e47d68b1a4b609c9816ac. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.130:35943: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:23:14.201880 16265 consensus_peers.cc:597] T 18ad7e3af5504571a69d10c00750a01c P 90f3da57796b488c9087a2f523adf045 -> Peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943): Couldn't send request to peer 1ca2725d290e47d68b1a4b609c9816ac. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.130:35943: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:23:14.203718 16790 consensus_peers.cc:597] T ada5a004ba7c46819051e3ccec71ffdb P cb82cc7bd6db46d980c75237a1402860 -> Peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943): Couldn't send request to peer 1ca2725d290e47d68b1a4b609c9816ac. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.130:35943: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:23:14.244905 16820 raft_consensus.cc:1275] T 8c11074579b046d99e585c2818e18a2c P cb82cc7bd6db46d980c75237a1402860 [term 21 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 20 index: 16. Preceding OpId from leader: term: 21 index: 17. (index mismatch)
I20260709 13:23:14.246393 25062 consensus_queue.cc:1048] T 8c11074579b046d99e585c2818e18a2c P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 13:23:14.289180 16292 raft_consensus.cc:1275] T ada5a004ba7c46819051e3ccec71ffdb P 90f3da57796b488c9087a2f523adf045 [term 26 FOLLOWER]: Refusing update from remote peer cb82cc7bd6db46d980c75237a1402860: Log matching property violated. Preceding OpId in replica: term: 25 index: 18. Preceding OpId from leader: term: 26 index: 19. (index mismatch)
I20260709 13:23:14.291997 25059 consensus_queue.cc:1048] T ada5a004ba7c46819051e3ccec71ffdb P cb82cc7bd6db46d980c75237a1402860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.000s
I20260709 13:23:14.315049 16824 raft_consensus.cc:1275] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P cb82cc7bd6db46d980c75237a1402860 [term 25 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 24 index: 18. Preceding OpId from leader: term: 25 index: 19. (index mismatch)
I20260709 13:23:14.317546 25074 consensus_queue.cc:1048] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.001s
W20260709 13:23:14.320652 16790 consensus_peers.cc:597] T 806d7690c2b644959152074bb2cab0a7 P cb82cc7bd6db46d980c75237a1402860 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
I20260709 13:23:14.326305 16824 raft_consensus.cc:1275] T 18ad7e3af5504571a69d10c00750a01c P cb82cc7bd6db46d980c75237a1402860 [term 20 FOLLOWER]: Refusing update from remote peer 90f3da57796b488c9087a2f523adf045: Log matching property violated. Preceding OpId in replica: term: 19 index: 17. Preceding OpId from leader: term: 20 index: 18. (index mismatch)
W20260709 13:23:14.339608 16265 consensus_peers.cc:597] T 1df93779209748b9860f6d893dd3de53 P 90f3da57796b488c9087a2f523adf045 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
I20260709 13:23:14.343822 25062 consensus_queue.cc:1048] T 18ad7e3af5504571a69d10c00750a01c P 90f3da57796b488c9087a2f523adf045 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.003s
W20260709 13:23:14.367379 15046 thread.cc:527] Waited for 2000ms trying to join with diag-logger (tid 16247)
W20260709 13:23:14.372738 16265 consensus_peers.cc:597] T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045 -> Peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943): Couldn't send request to peer 1ca2725d290e47d68b1a4b609c9816ac. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.130:35943: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:23:14.389416 16790 consensus_peers.cc:597] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 66: this message will repeat every 5th retry.
W20260709 13:23:14.424826 16790 consensus_peers.cc:597] T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860 -> Peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943): Couldn't send request to peer 1ca2725d290e47d68b1a4b609c9816ac. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.130:35943: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:23:14.429288 16265 consensus_peers.cc:597] T b87147731a68400e8cce9b879daa7c5d P 90f3da57796b488c9087a2f523adf045 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 71: this message will repeat every 5th retry.
W20260709 13:23:14.431396 16265 consensus_peers.cc:597] T 8a878999c0024d2abe517484180290fc P 90f3da57796b488c9087a2f523adf045 -> Peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943): Couldn't send request to peer 1ca2725d290e47d68b1a4b609c9816ac. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.130:35943: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:23:14.435984 16265 consensus_peers.cc:597] T fffab26427e24d398e2695aad40ca134 P 90f3da57796b488c9087a2f523adf045 -> Peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943): Couldn't send request to peer 1ca2725d290e47d68b1a4b609c9816ac. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.130:35943: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:23:14.436599 16265 consensus_peers.cc:597] T 7c2095a5f4ad49408dbc7f69c606120d P 90f3da57796b488c9087a2f523adf045 -> Peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943): Couldn't send request to peer 1ca2725d290e47d68b1a4b609c9816ac. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.130:35943: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:23:14.499264 16790 consensus_peers.cc:597] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 -> Peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943): Couldn't send request to peer 1ca2725d290e47d68b1a4b609c9816ac. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.130:35943: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:23:14.499567 16265 consensus_peers.cc:597] T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045 -> Peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943): Couldn't send request to peer 1ca2725d290e47d68b1a4b609c9816ac. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.130:35943: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:23:14.545542 16790 consensus_peers.cc:597] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 -> Peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943): Couldn't send request to peer 1ca2725d290e47d68b1a4b609c9816ac. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.130:35943: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:23:14.546003 16265 consensus_peers.cc:597] T 8cc959fb7f704fba9009c0595467d074 P 90f3da57796b488c9087a2f523adf045 -> Peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943): Couldn't send request to peer 1ca2725d290e47d68b1a4b609c9816ac. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.130:35943: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:23:14.546237 16790 consensus_peers.cc:597] T 74ca8140f35c4ae29e8a664f6bdfc829 P cb82cc7bd6db46d980c75237a1402860 -> Peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943): Couldn't send request to peer 1ca2725d290e47d68b1a4b609c9816ac. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.130:35943: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:23:14.549731 16265 consensus_peers.cc:597] T 8cc959fb7f704fba9009c0595467d074 P 90f3da57796b488c9087a2f523adf045 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 66: this message will repeat every 5th retry.
W20260709 13:23:14.570303 16265 consensus_peers.cc:597] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045 -> Peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943): Couldn't send request to peer 1ca2725d290e47d68b1a4b609c9816ac. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.130:35943: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:23:14.668268 16790 consensus_peers.cc:597] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860 -> Peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943): Couldn't send request to peer 1ca2725d290e47d68b1a4b609c9816ac. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.130:35943: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:23:14.668427 16265 consensus_peers.cc:597] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:23:14.669981 16265 consensus_peers.cc:597] T 02838c86531d4f98bf58d1c22b8baf77 P 90f3da57796b488c9087a2f523adf045 -> Peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943): Couldn't send request to peer 1ca2725d290e47d68b1a4b609c9816ac. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.130:35943: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:23:14.670394 16265 consensus_peers.cc:597] T 1df93779209748b9860f6d893dd3de53 P 90f3da57796b488c9087a2f523adf045 -> Peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943): Couldn't send request to peer 1ca2725d290e47d68b1a4b609c9816ac. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.130:35943: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:23:14.674297 16265 consensus_peers.cc:597] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 90f3da57796b488c9087a2f523adf045 -> Peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943): Couldn't send request to peer 1ca2725d290e47d68b1a4b609c9816ac. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.130:35943: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:23:14.687410 16265 consensus_peers.cc:597] T e4dc6b0541ca431ebc757beac00521f1 P 90f3da57796b488c9087a2f523adf045 -> Peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943): Couldn't send request to peer 1ca2725d290e47d68b1a4b609c9816ac. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.130:35943: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:23:14.702253 16790 consensus_peers.cc:597] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 -> Peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943): Couldn't send request to peer 1ca2725d290e47d68b1a4b609c9816ac. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.130:35943: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:23:14.743503 25024 raft_consensus.cc:493] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [term 30 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:23:14.744011 25024 raft_consensus.cc:515] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:23:14.745748 25024 leader_election.cc:290] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
W20260709 13:23:14.752808 16790 leader_election.cc:336] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 31 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111)
W20260709 13:23:14.753379 16790 leader_election.cc:336] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 31 pre-election: RPC error from VoteRequest() call to peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943): Network error: Client connection negotiation failed: client connection to 127.14.177.130:35943: connect: Connection refused (error 111)
I20260709 13:23:14.753682 16790 leader_election.cc:304] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:23:14.754251 25024 raft_consensus.cc:2749] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
W20260709 13:23:14.821792 16790 consensus_peers.cc:597] T c9944d5e4b8c4109af9955c4f08c9095 P cb82cc7bd6db46d980c75237a1402860 -> Peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943): Couldn't send request to peer 1ca2725d290e47d68b1a4b609c9816ac. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.130:35943: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:23:14.825258 25051 raft_consensus.cc:493] T c0eab25ce4c64e018a041beeda2867db P 90f3da57796b488c9087a2f523adf045 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:23:14.825801 25051 raft_consensus.cc:515] T c0eab25ce4c64e018a041beeda2867db P 90f3da57796b488c9087a2f523adf045 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:23:14.831243 25051 leader_election.cc:290] T c0eab25ce4c64e018a041beeda2867db P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
W20260709 13:23:14.824052 16790 consensus_peers.cc:597] T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 76: this message will repeat every 5th retry.
W20260709 13:23:14.832242 16790 consensus_peers.cc:597] T 3884f864a96d428785895e5463e5728f P cb82cc7bd6db46d980c75237a1402860 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 76: this message will repeat every 5th retry.
W20260709 13:23:14.835750 16265 consensus_peers.cc:597] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:23:14.838023 16265 leader_election.cc:336] T c0eab25ce4c64e018a041beeda2867db P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943): Network error: Client connection negotiation failed: client connection to 127.14.177.130:35943: connect: Connection refused (error 111)
W20260709 13:23:14.839025 16265 leader_election.cc:336] T c0eab25ce4c64e018a041beeda2867db P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111)
I20260709 13:23:14.839486 16265 leader_election.cc:304] T c0eab25ce4c64e018a041beeda2867db P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:23:14.840086 25051 raft_consensus.cc:2749] T c0eab25ce4c64e018a041beeda2867db P 90f3da57796b488c9087a2f523adf045 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 13:23:14.862356 16265 consensus_peers.cc:597] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 90f3da57796b488c9087a2f523adf045 -> Peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943): Couldn't send request to peer 1ca2725d290e47d68b1a4b609c9816ac. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.130:35943: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:23:14.873935 25051 raft_consensus.cc:493] T d16dfad0c3b540e5b4b86492ddf87208 P 90f3da57796b488c9087a2f523adf045 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:23:14.874456 25051 raft_consensus.cc:515] T d16dfad0c3b540e5b4b86492ddf87208 P 90f3da57796b488c9087a2f523adf045 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:23:14.876499 25051 leader_election.cc:290] T d16dfad0c3b540e5b4b86492ddf87208 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
W20260709 13:23:14.881621 16265 leader_election.cc:336] T d16dfad0c3b540e5b4b86492ddf87208 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111)
W20260709 13:23:14.883682 16265 leader_election.cc:336] T d16dfad0c3b540e5b4b86492ddf87208 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943): Network error: Client connection negotiation failed: client connection to 127.14.177.130:35943: connect: Connection refused (error 111)
I20260709 13:23:14.884093 16265 leader_election.cc:304] T d16dfad0c3b540e5b4b86492ddf87208 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:23:14.884864 25051 raft_consensus.cc:2749] T d16dfad0c3b540e5b4b86492ddf87208 P 90f3da57796b488c9087a2f523adf045 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 13:23:14.897903 16265 consensus_peers.cc:597] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 90f3da57796b488c9087a2f523adf045 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 71: this message will repeat every 5th retry.
W20260709 13:23:14.923566 16790 consensus_peers.cc:597] T 57a744f5aaf944678a63469cbf514015 P cb82cc7bd6db46d980c75237a1402860 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 76: this message will repeat every 5th retry.
I20260709 13:23:14.956534 25051 raft_consensus.cc:493] T 68a6838d82ef4cac83507f906139d90f P 90f3da57796b488c9087a2f523adf045 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:23:14.957180 25051 raft_consensus.cc:515] T 68a6838d82ef4cac83507f906139d90f P 90f3da57796b488c9087a2f523adf045 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:23:14.960328 25051 leader_election.cc:290] T 68a6838d82ef4cac83507f906139d90f P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
W20260709 13:23:14.965703 16265 leader_election.cc:336] T 68a6838d82ef4cac83507f906139d90f P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943): Network error: Client connection negotiation failed: client connection to 127.14.177.130:35943: connect: Connection refused (error 111)
W20260709 13:23:14.966456 16265 leader_election.cc:336] T 68a6838d82ef4cac83507f906139d90f P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111)
I20260709 13:23:14.966885 16265 leader_election.cc:304] T 68a6838d82ef4cac83507f906139d90f P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:23:14.967684 25051 raft_consensus.cc:2749] T 68a6838d82ef4cac83507f906139d90f P 90f3da57796b488c9087a2f523adf045 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:23:14.968524 25038 raft_consensus.cc:493] T 84f2da98864b4425bf7c0055951fdd4b P 90f3da57796b488c9087a2f523adf045 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:23:14.969056 25038 raft_consensus.cc:515] T 84f2da98864b4425bf7c0055951fdd4b P 90f3da57796b488c9087a2f523adf045 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:23:14.971057 25038 leader_election.cc:290] T 84f2da98864b4425bf7c0055951fdd4b P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
W20260709 13:23:14.981473 16265 consensus_peers.cc:597] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045 -> Peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943): Couldn't send request to peer 1ca2725d290e47d68b1a4b609c9816ac. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.130:35943: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:23:14.982051 16265 leader_election.cc:336] T 84f2da98864b4425bf7c0055951fdd4b P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943): Network error: Client connection negotiation failed: client connection to 127.14.177.130:35943: connect: Connection refused (error 111)
W20260709 13:23:14.982728 16265 leader_election.cc:336] T 84f2da98864b4425bf7c0055951fdd4b P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111)
I20260709 13:23:14.983157 16265 leader_election.cc:304] T 84f2da98864b4425bf7c0055951fdd4b P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:23:14.983915 25038 raft_consensus.cc:2749] T 84f2da98864b4425bf7c0055951fdd4b P 90f3da57796b488c9087a2f523adf045 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 13:23:15.004576 16265 consensus_peers.cc:597] T e4dc6b0541ca431ebc757beac00521f1 P 90f3da57796b488c9087a2f523adf045 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 71: this message will repeat every 5th retry.
W20260709 13:23:15.054821 16790 consensus_peers.cc:597] T 66f75124215f4ca6bef831e174a792d8 P cb82cc7bd6db46d980c75237a1402860 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 71: this message will repeat every 5th retry.
I20260709 13:23:15.131556 25038 raft_consensus.cc:493] T 8158bbd17b994d2eb96f560bd0f1b88d P 90f3da57796b488c9087a2f523adf045 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:23:15.132180 25038 raft_consensus.cc:515] T 8158bbd17b994d2eb96f560bd0f1b88d P 90f3da57796b488c9087a2f523adf045 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:23:15.134184 25038 leader_election.cc:290] T 8158bbd17b994d2eb96f560bd0f1b88d P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
W20260709 13:23:15.140942 16265 leader_election.cc:336] T 8158bbd17b994d2eb96f560bd0f1b88d P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943): Network error: Client connection negotiation failed: client connection to 127.14.177.130:35943: connect: Connection refused (error 111)
W20260709 13:23:15.143281 16265 leader_election.cc:336] T 8158bbd17b994d2eb96f560bd0f1b88d P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111)
I20260709 13:23:15.143821 16265 leader_election.cc:304] T 8158bbd17b994d2eb96f560bd0f1b88d P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:23:15.144734 25038 raft_consensus.cc:2749] T 8158bbd17b994d2eb96f560bd0f1b88d P 90f3da57796b488c9087a2f523adf045 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 13:23:15.183600 16265 consensus_peers.cc:597] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 90f3da57796b488c9087a2f523adf045 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:23:15.196003 16265 consensus_peers.cc:597] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045 -> Peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943): Couldn't send request to peer 1ca2725d290e47d68b1a4b609c9816ac. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.130:35943: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:23:15.202553 25024 raft_consensus.cc:493] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:23:15.203056 25024 raft_consensus.cc:515] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:23:15.204851 25024 leader_election.cc:290] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
W20260709 13:23:15.210088 16790 leader_election.cc:336] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111)
W20260709 13:23:15.212002 16790 leader_election.cc:336] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943): Network error: Client connection negotiation failed: client connection to 127.14.177.130:35943: connect: Connection refused (error 111)
I20260709 13:23:15.212591 16790 leader_election.cc:304] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:23:15.213430 25024 raft_consensus.cc:2749] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 13:23:15.236550 16265 consensus_peers.cc:597] T 26354361e9b044cea1c2b86c4882fe92 P 90f3da57796b488c9087a2f523adf045 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 71: this message will repeat every 5th retry.
I20260709 13:23:15.252005 25038 raft_consensus.cc:493] T e5ad4eb281564936a59f41e387a47156 P 90f3da57796b488c9087a2f523adf045 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:23:15.252632 25038 raft_consensus.cc:515] T e5ad4eb281564936a59f41e387a47156 P 90f3da57796b488c9087a2f523adf045 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:23:15.254724 25038 leader_election.cc:290] T e5ad4eb281564936a59f41e387a47156 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
W20260709 13:23:15.260185 16265 leader_election.cc:336] T e5ad4eb281564936a59f41e387a47156 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111)
I20260709 13:23:15.315828 25038 raft_consensus.cc:493] T 5a959dbecc2648eb85d1cc550f5a3245 P 90f3da57796b488c9087a2f523adf045 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:23:15.316382 25038 raft_consensus.cc:515] T 5a959dbecc2648eb85d1cc550f5a3245 P 90f3da57796b488c9087a2f523adf045 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:23:15.318977 25038 leader_election.cc:290] T 5a959dbecc2648eb85d1cc550f5a3245 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:23:15.322124 16820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a959dbecc2648eb85d1cc550f5a3245" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:23:15.322863 16820 raft_consensus.cc:2468] T 5a959dbecc2648eb85d1cc550f5a3245 P cb82cc7bd6db46d980c75237a1402860 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 24.
W20260709 13:23:15.324054 16265 leader_election.cc:336] T 5a959dbecc2648eb85d1cc550f5a3245 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111)
W20260709 13:23:15.367983 15046 thread.cc:527] Waited for 3000ms trying to join with diag-logger (tid 16247)
I20260709 13:23:15.381635 25024 raft_consensus.cc:493] T ee9ed67006144797b1de6ea8c2013a68 P cb82cc7bd6db46d980c75237a1402860 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:23:15.382292 25024 raft_consensus.cc:515] T ee9ed67006144797b1de6ea8c2013a68 P cb82cc7bd6db46d980c75237a1402860 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:23:15.384569 25024 leader_election.cc:290] T ee9ed67006144797b1de6ea8c2013a68 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
W20260709 13:23:15.390599 16790 leader_election.cc:336] T ee9ed67006144797b1de6ea8c2013a68 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111)
W20260709 13:23:15.391471 16790 leader_election.cc:336] T ee9ed67006144797b1de6ea8c2013a68 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943): Network error: Client connection negotiation failed: client connection to 127.14.177.130:35943: connect: Connection refused (error 111)
I20260709 13:23:15.391899 16790 leader_election.cc:304] T ee9ed67006144797b1de6ea8c2013a68 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:23:15.392602 25024 raft_consensus.cc:2749] T ee9ed67006144797b1de6ea8c2013a68 P cb82cc7bd6db46d980c75237a1402860 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:23:15.457417 25024 raft_consensus.cc:493] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:23:15.457964 25024 raft_consensus.cc:515] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:23:15.460173 25024 leader_election.cc:290] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
W20260709 13:23:15.474169 16790 leader_election.cc:336] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943): Network error: Client connection negotiation failed: client connection to 127.14.177.130:35943: connect: Connection refused (error 111)
W20260709 13:23:15.475416 16790 leader_election.cc:336] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111)
I20260709 13:23:15.476052 16790 leader_election.cc:304] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:23:15.476836 25024 raft_consensus.cc:2749] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 13:23:15.491915 25059 raft_consensus.cc:493] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:23:15.492544 25059 raft_consensus.cc:515] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:23:15.494987 25059 leader_election.cc:290] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
W20260709 13:23:15.500188 16790 leader_election.cc:336] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111)
W20260709 13:23:15.504544 16790 leader_election.cc:336] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943): Network error: Client connection negotiation failed: client connection to 127.14.177.130:35943: connect: Connection refused (error 111)
I20260709 13:23:15.505064 16790 leader_election.cc:304] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:23:15.505968 25059 raft_consensus.cc:2749] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:23:15.532002 24975 raft_consensus.cc:493] T c4f65cdf00b748bb963ee6e81ef43b79 P 90f3da57796b488c9087a2f523adf045 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:23:15.532594 24975 raft_consensus.cc:515] T c4f65cdf00b748bb963ee6e81ef43b79 P 90f3da57796b488c9087a2f523adf045 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:23:15.536350 16820 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4f65cdf00b748bb963ee6e81ef43b79" candidate_uuid: "90f3da57796b488c9087a2f523adf045" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "cb82cc7bd6db46d980c75237a1402860" is_pre_election: true
I20260709 13:23:15.536500 24975 leader_election.cc:290] T c4f65cdf00b748bb963ee6e81ef43b79 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), cb82cc7bd6db46d980c75237a1402860 (127.14.177.132:38923)
I20260709 13:23:15.537053 16820 raft_consensus.cc:2468] T c4f65cdf00b748bb963ee6e81ef43b79 P cb82cc7bd6db46d980c75237a1402860 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90f3da57796b488c9087a2f523adf045 in term 23.
W20260709 13:23:15.540481 16265 leader_election.cc:336] T c4f65cdf00b748bb963ee6e81ef43b79 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111)
W20260709 13:23:15.611967 16265 consensus_peers.cc:597] T 8a878999c0024d2abe517484180290fc P 90f3da57796b488c9087a2f523adf045 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
I20260709 13:23:15.623685 25024 raft_consensus.cc:493] T 829b4ad8c5a54eababeea1181b960d4b P cb82cc7bd6db46d980c75237a1402860 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:23:15.624293 25024 raft_consensus.cc:515] T 829b4ad8c5a54eababeea1181b960d4b P cb82cc7bd6db46d980c75237a1402860 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:23:15.626430 25024 leader_election.cc:290] T 829b4ad8c5a54eababeea1181b960d4b P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
W20260709 13:23:15.632208 16790 leader_election.cc:336] T 829b4ad8c5a54eababeea1181b960d4b P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111)
W20260709 13:23:15.632902 16790 leader_election.cc:336] T 829b4ad8c5a54eababeea1181b960d4b P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943): Network error: Client connection negotiation failed: client connection to 127.14.177.130:35943: connect: Connection refused (error 111)
I20260709 13:23:15.633301 16790 leader_election.cc:304] T 829b4ad8c5a54eababeea1181b960d4b P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:23:15.633996 25024 raft_consensus.cc:2749] T 829b4ad8c5a54eababeea1181b960d4b P cb82cc7bd6db46d980c75237a1402860 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:23:15.786119 25024 raft_consensus.cc:493] T af57996a8be84d398f0f023e8d08ef77 P cb82cc7bd6db46d980c75237a1402860 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:23:15.786738 25024 raft_consensus.cc:515] T af57996a8be84d398f0f023e8d08ef77 P cb82cc7bd6db46d980c75237a1402860 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:23:15.788766 25024 leader_election.cc:290] T af57996a8be84d398f0f023e8d08ef77 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
W20260709 13:23:15.794564 16790 leader_election.cc:336] T af57996a8be84d398f0f023e8d08ef77 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111)
W20260709 13:23:15.795264 16790 leader_election.cc:336] T af57996a8be84d398f0f023e8d08ef77 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943): Network error: Client connection negotiation failed: client connection to 127.14.177.130:35943: connect: Connection refused (error 111)
I20260709 13:23:15.795656 16790 leader_election.cc:304] T af57996a8be84d398f0f023e8d08ef77 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:23:15.796348 25024 raft_consensus.cc:2749] T af57996a8be84d398f0f023e8d08ef77 P cb82cc7bd6db46d980c75237a1402860 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 13:23:15.885816 16790 consensus_peers.cc:597] T 2794af5b99984efebdee564035841cb0 P cb82cc7bd6db46d980c75237a1402860 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 81: this message will repeat every 5th retry.
I20260709 13:23:15.900861 25051 raft_consensus.cc:493] T 5a96373bb0e14dce8a266b2fc6c58507 P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:23:15.901467 25051 raft_consensus.cc:515] T 5a96373bb0e14dce8a266b2fc6c58507 P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } }
I20260709 13:23:15.903753 25051 leader_election.cc:290] T 5a96373bb0e14dce8a266b2fc6c58507 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
W20260709 13:23:15.908461 16265 leader_election.cc:336] T 5a96373bb0e14dce8a266b2fc6c58507 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111)
W20260709 13:23:15.912567 16790 consensus_peers.cc:597] T 23f82224f4564fea93572e8206bad1fe P cb82cc7bd6db46d980c75237a1402860 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 81: this message will repeat every 5th retry.
W20260709 13:23:15.965459 16790 consensus_peers.cc:597] T 9e65676b21b748728ad39668cfe722ec P cb82cc7bd6db46d980c75237a1402860 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
I20260709 13:23:16.028445 15046 tablet_server.cc:196] TabletServer@127.14.177.130:0 shutdown complete.
W20260709 13:23:16.146518 16790 consensus_peers.cc:597] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 71: this message will repeat every 5th retry.
I20260709 13:23:16.173316 15046 tablet_server.cc:179] TabletServer@127.14.177.131:0 shutting down...
I20260709 13:23:16.233517 15046 ts_tablet_manager.cc:1507] Shutting down tablet manager...
W20260709 13:23:16.236363 16789 consensus_peers.cc:597] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 -> Peer 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389): Couldn't send request to peer 90f3da57796b488c9087a2f523adf045. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:23:16.237390 15046 tablet_replica.cc:333] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 90f3da57796b488c9087a2f523adf045: stopping tablet replica
I20260709 13:23:16.238606 15046 raft_consensus.cc:2243] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 90f3da57796b488c9087a2f523adf045 [term 22 LEADER]: Raft consensus shutting down.
W20260709 13:23:16.239403 16789 consensus_peers.cc:597] T ada5a004ba7c46819051e3ccec71ffdb P cb82cc7bd6db46d980c75237a1402860 -> Peer 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389): Couldn't send request to peer 90f3da57796b488c9087a2f523adf045. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:23:16.240027 15046 raft_consensus.cc:2272] T 9ad5dbfa99ee4b8d9e28be92b3f36db5 P 90f3da57796b488c9087a2f523adf045 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:16.243263 15046 tablet_replica.cc:333] T cdbd78acb4cf46298e0de8ba0d130e19 P 90f3da57796b488c9087a2f523adf045: stopping tablet replica
I20260709 13:23:16.244254 15046 raft_consensus.cc:2243] T cdbd78acb4cf46298e0de8ba0d130e19 P 90f3da57796b488c9087a2f523adf045 [term 29 FOLLOWER]: Raft consensus shutting down.
I20260709 13:23:16.244956 15046 raft_consensus.cc:2272] T cdbd78acb4cf46298e0de8ba0d130e19 P 90f3da57796b488c9087a2f523adf045 [term 29 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:16.247921 15046 tablet_replica.cc:333] T c9944d5e4b8c4109af9955c4f08c9095 P 90f3da57796b488c9087a2f523adf045: stopping tablet replica
I20260709 13:23:16.248965 15046 raft_consensus.cc:2243] T c9944d5e4b8c4109af9955c4f08c9095 P 90f3da57796b488c9087a2f523adf045 [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 13:23:16.249625 15046 raft_consensus.cc:2272] T c9944d5e4b8c4109af9955c4f08c9095 P 90f3da57796b488c9087a2f523adf045 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:16.252694 15046 tablet_replica.cc:333] T 8d4ab870a05347868956aa8b841b2981 P 90f3da57796b488c9087a2f523adf045: stopping tablet replica
I20260709 13:23:16.253669 15046 raft_consensus.cc:2243] T 8d4ab870a05347868956aa8b841b2981 P 90f3da57796b488c9087a2f523adf045 [term 24 LEADER]: Raft consensus shutting down.
I20260709 13:23:16.254976 15046 raft_consensus.cc:2272] T 8d4ab870a05347868956aa8b841b2981 P 90f3da57796b488c9087a2f523adf045 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:16.261736 15046 tablet_replica.cc:333] T 57a744f5aaf944678a63469cbf514015 P 90f3da57796b488c9087a2f523adf045: stopping tablet replica
W20260709 13:23:16.256235 16789 consensus_peers.cc:597] T 23f82224f4564fea93572e8206bad1fe P cb82cc7bd6db46d980c75237a1402860 -> Peer 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389): Couldn't send request to peer 90f3da57796b488c9087a2f523adf045. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:23:16.263139 15046 raft_consensus.cc:2243] T 57a744f5aaf944678a63469cbf514015 P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:23:16.264204 15046 raft_consensus.cc:2272] T 57a744f5aaf944678a63469cbf514015 P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:16.266183 15046 tablet_replica.cc:333] T e5ad4eb281564936a59f41e387a47156 P 90f3da57796b488c9087a2f523adf045: stopping tablet replica
I20260709 13:23:16.266863 15046 raft_consensus.cc:2243] T e5ad4eb281564936a59f41e387a47156 P 90f3da57796b488c9087a2f523adf045 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:23:16.267450 15046 raft_consensus.cc:2272] T e5ad4eb281564936a59f41e387a47156 P 90f3da57796b488c9087a2f523adf045 [term 22 FOLLOWER]: Raft consensus is shut down!
W20260709 13:23:16.267705 16265 leader_election.cc:336] T e5ad4eb281564936a59f41e387a47156 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943): Network error: Client connection negotiation failed: client connection to 127.14.177.130:35943: connect: Connection refused (error 111)
I20260709 13:23:16.268283 16265 leader_election.cc:304] T e5ad4eb281564936a59f41e387a47156 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
W20260709 13:23:16.268978 16265 raft_consensus.cc:2686] T e5ad4eb281564936a59f41e387a47156 P 90f3da57796b488c9087a2f523adf045: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260709 13:23:16.269754 15046 tablet_replica.cc:333] T 1df93779209748b9860f6d893dd3de53 P 90f3da57796b488c9087a2f523adf045: stopping tablet replica
W20260709 13:23:16.269742 16265 leader_election.cc:336] T 5a959dbecc2648eb85d1cc550f5a3245 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943): Network error: Client connection negotiation failed: client connection to 127.14.177.130:35943: connect: Connection refused (error 111)
I20260709 13:23:16.270391 16265 leader_election.cc:304] T 5a959dbecc2648eb85d1cc550f5a3245 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:23:16.271261 25051 raft_consensus.cc:2749] T 5a959dbecc2648eb85d1cc550f5a3245 P 90f3da57796b488c9087a2f523adf045 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:23:16.270964 15046 raft_consensus.cc:2243] T 1df93779209748b9860f6d893dd3de53 P 90f3da57796b488c9087a2f523adf045 [term 23 LEADER]: Raft consensus shutting down.
W20260709 13:23:16.272737 16265 leader_election.cc:336] T c4f65cdf00b748bb963ee6e81ef43b79 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943): Network error: Client connection negotiation failed: client connection to 127.14.177.130:35943: connect: Connection refused (error 111)
I20260709 13:23:16.272891 15046 raft_consensus.cc:2272] T 1df93779209748b9860f6d893dd3de53 P 90f3da57796b488c9087a2f523adf045 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:16.273322 16265 leader_election.cc:304] T c4f65cdf00b748bb963ee6e81ef43b79 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045, cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:23:16.274242 25051 raft_consensus.cc:2749] T c4f65cdf00b748bb963ee6e81ef43b79 P 90f3da57796b488c9087a2f523adf045 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 13:23:16.275753 16265 leader_election.cc:336] T 5a96373bb0e14dce8a266b2fc6c58507 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943): Network error: Client connection negotiation failed: client connection to 127.14.177.130:35943: connect: Connection refused (error 111)
I20260709 13:23:16.276360 16265 leader_election.cc:304] T 5a96373bb0e14dce8a266b2fc6c58507 P 90f3da57796b488c9087a2f523adf045 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90f3da57796b488c9087a2f523adf045; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:23:16.276775 15046 tablet_replica.cc:333] T 8158bbd17b994d2eb96f560bd0f1b88d P 90f3da57796b488c9087a2f523adf045: stopping tablet replica
I20260709 13:23:16.277148 25051 raft_consensus.cc:2749] T 5a96373bb0e14dce8a266b2fc6c58507 P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:23:16.277849 15046 raft_consensus.cc:2243] T 8158bbd17b994d2eb96f560bd0f1b88d P 90f3da57796b488c9087a2f523adf045 [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 13:23:16.278625 15046 raft_consensus.cc:2272] T 8158bbd17b994d2eb96f560bd0f1b88d P 90f3da57796b488c9087a2f523adf045 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:16.281363 15046 tablet_replica.cc:333] T 31969392ccdb43b491911a319e8bc7e1 P 90f3da57796b488c9087a2f523adf045: stopping tablet replica
I20260709 13:23:16.282171 15046 raft_consensus.cc:2243] T 31969392ccdb43b491911a319e8bc7e1 P 90f3da57796b488c9087a2f523adf045 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:23:16.282857 15046 raft_consensus.cc:2272] T 31969392ccdb43b491911a319e8bc7e1 P 90f3da57796b488c9087a2f523adf045 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:16.284953 15046 tablet_replica.cc:333] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045: stopping tablet replica
I20260709 13:23:16.285534 15046 raft_consensus.cc:2243] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045 [term 25 LEADER]: Raft consensus shutting down.
I20260709 13:23:16.286530 15046 raft_consensus.cc:2272] T 19baa7bc367844c288bca8f81439c413 P 90f3da57796b488c9087a2f523adf045 [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:16.288702 15046 tablet_replica.cc:333] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045: stopping tablet replica
I20260709 13:23:16.289378 15046 raft_consensus.cc:2243] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045 [term 22 LEADER]: Raft consensus shutting down.
I20260709 13:23:16.290472 15046 raft_consensus.cc:2272] T 08c74490c40243a88e62d1760b5cd814 P 90f3da57796b488c9087a2f523adf045 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:16.293485 15046 tablet_replica.cc:333] T 8638ee363273426db1393dd81e5c7b72 P 90f3da57796b488c9087a2f523adf045: stopping tablet replica
I20260709 13:23:16.294116 15046 raft_consensus.cc:2243] T 8638ee363273426db1393dd81e5c7b72 P 90f3da57796b488c9087a2f523adf045 [term 25 FOLLOWER]: Raft consensus shutting down.
I20260709 13:23:16.294643 15046 raft_consensus.cc:2272] T 8638ee363273426db1393dd81e5c7b72 P 90f3da57796b488c9087a2f523adf045 [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:16.297286 15046 tablet_replica.cc:333] T c4f65cdf00b748bb963ee6e81ef43b79 P 90f3da57796b488c9087a2f523adf045: stopping tablet replica
I20260709 13:23:16.298126 15046 raft_consensus.cc:2243] T c4f65cdf00b748bb963ee6e81ef43b79 P 90f3da57796b488c9087a2f523adf045 [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 13:23:16.299324 15046 raft_consensus.cc:2272] T c4f65cdf00b748bb963ee6e81ef43b79 P 90f3da57796b488c9087a2f523adf045 [term 23 FOLLOWER]: Raft consensus is shut down!
W20260709 13:23:16.301729 16789 consensus_peers.cc:597] T 74ca8140f35c4ae29e8a664f6bdfc829 P cb82cc7bd6db46d980c75237a1402860 -> Peer 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389): Couldn't send request to peer 90f3da57796b488c9087a2f523adf045. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:23:16.302340 15046 tablet_replica.cc:333] T 2794af5b99984efebdee564035841cb0 P 90f3da57796b488c9087a2f523adf045: stopping tablet replica
W20260709 13:23:16.302836 16789 consensus_peers.cc:597] T 3884f864a96d428785895e5463e5728f P cb82cc7bd6db46d980c75237a1402860 -> Peer 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389): Couldn't send request to peer 90f3da57796b488c9087a2f523adf045. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:23:16.303366 15046 raft_consensus.cc:2243] T 2794af5b99984efebdee564035841cb0 P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:23:16.304029 15046 raft_consensus.cc:2272] T 2794af5b99984efebdee564035841cb0 P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:16.306954 15046 tablet_replica.cc:333] T b87147731a68400e8cce9b879daa7c5d P 90f3da57796b488c9087a2f523adf045: stopping tablet replica
I20260709 13:23:16.307857 15046 raft_consensus.cc:2243] T b87147731a68400e8cce9b879daa7c5d P 90f3da57796b488c9087a2f523adf045 [term 22 LEADER]: Raft consensus shutting down.
I20260709 13:23:16.309001 15046 raft_consensus.cc:2272] T b87147731a68400e8cce9b879daa7c5d P 90f3da57796b488c9087a2f523adf045 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:16.311292 15046 tablet_replica.cc:333] T 84f2da98864b4425bf7c0055951fdd4b P 90f3da57796b488c9087a2f523adf045: stopping tablet replica
I20260709 13:23:16.311880 15046 raft_consensus.cc:2243] T 84f2da98864b4425bf7c0055951fdd4b P 90f3da57796b488c9087a2f523adf045 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:23:16.312320 15046 raft_consensus.cc:2272] T 84f2da98864b4425bf7c0055951fdd4b P 90f3da57796b488c9087a2f523adf045 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:16.314291 15046 tablet_replica.cc:333] T d16dfad0c3b540e5b4b86492ddf87208 P 90f3da57796b488c9087a2f523adf045: stopping tablet replica
I20260709 13:23:16.314883 15046 raft_consensus.cc:2243] T d16dfad0c3b540e5b4b86492ddf87208 P 90f3da57796b488c9087a2f523adf045 [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 13:23:16.315579 15046 raft_consensus.cc:2272] T d16dfad0c3b540e5b4b86492ddf87208 P 90f3da57796b488c9087a2f523adf045 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:16.318267 15046 tablet_replica.cc:333] T 68a6838d82ef4cac83507f906139d90f P 90f3da57796b488c9087a2f523adf045: stopping tablet replica
I20260709 13:23:16.319097 15046 raft_consensus.cc:2243] T 68a6838d82ef4cac83507f906139d90f P 90f3da57796b488c9087a2f523adf045 [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 13:23:16.319759 15046 raft_consensus.cc:2272] T 68a6838d82ef4cac83507f906139d90f P 90f3da57796b488c9087a2f523adf045 [term 23 FOLLOWER]: Raft consensus is shut down!
W20260709 13:23:16.320523 16790 consensus_peers.cc:597] T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860 -> Peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943): Couldn't send request to peer 1ca2725d290e47d68b1a4b609c9816ac. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.130:35943: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:23:16.322470 15046 tablet_replica.cc:333] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 90f3da57796b488c9087a2f523adf045: stopping tablet replica
I20260709 13:23:16.323345 15046 raft_consensus.cc:2243] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 90f3da57796b488c9087a2f523adf045 [term 21 LEADER]: Raft consensus shutting down.
I20260709 13:23:16.324378 15046 raft_consensus.cc:2272] T 87ac0d1f7e8a419f9bcbc04aee0e067e P 90f3da57796b488c9087a2f523adf045 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:16.326836 15046 tablet_replica.cc:333] T c0eab25ce4c64e018a041beeda2867db P 90f3da57796b488c9087a2f523adf045: stopping tablet replica
I20260709 13:23:16.327471 15046 raft_consensus.cc:2243] T c0eab25ce4c64e018a041beeda2867db P 90f3da57796b488c9087a2f523adf045 [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 13:23:16.327999 15046 raft_consensus.cc:2272] T c0eab25ce4c64e018a041beeda2867db P 90f3da57796b488c9087a2f523adf045 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:16.330070 15046 tablet_replica.cc:333] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045: stopping tablet replica
I20260709 13:23:16.330754 15046 raft_consensus.cc:2243] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045 [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 13:23:16.331421 15046 raft_consensus.cc:2272] T 806d7690c2b644959152074bb2cab0a7 P 90f3da57796b488c9087a2f523adf045 [term 24 FOLLOWER]: Raft consensus is shut down!
W20260709 13:23:16.333474 16789 consensus_peers.cc:597] T 57a744f5aaf944678a63469cbf514015 P cb82cc7bd6db46d980c75237a1402860 -> Peer 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389): Couldn't send request to peer 90f3da57796b488c9087a2f523adf045. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:23:16.334045 15046 tablet_replica.cc:333] T 8cc959fb7f704fba9009c0595467d074 P 90f3da57796b488c9087a2f523adf045: stopping tablet replica
I20260709 13:23:16.334708 15046 raft_consensus.cc:2243] T 8cc959fb7f704fba9009c0595467d074 P 90f3da57796b488c9087a2f523adf045 [term 20 LEADER]: Raft consensus shutting down.
I20260709 13:23:16.335685 15046 raft_consensus.cc:2272] T 8cc959fb7f704fba9009c0595467d074 P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:16.338245 15046 tablet_replica.cc:333] T 5a959dbecc2648eb85d1cc550f5a3245 P 90f3da57796b488c9087a2f523adf045: stopping tablet replica
I20260709 13:23:16.339046 15046 raft_consensus.cc:2243] T 5a959dbecc2648eb85d1cc550f5a3245 P 90f3da57796b488c9087a2f523adf045 [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 13:23:16.339869 15046 raft_consensus.cc:2272] T 5a959dbecc2648eb85d1cc550f5a3245 P 90f3da57796b488c9087a2f523adf045 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:16.342051 15046 tablet_replica.cc:333] T a378d7f796f94615b3b347119a9880fd P 90f3da57796b488c9087a2f523adf045: stopping tablet replica
I20260709 13:23:16.342634 15046 raft_consensus.cc:2243] T a378d7f796f94615b3b347119a9880fd P 90f3da57796b488c9087a2f523adf045 [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 13:23:16.343276 15046 raft_consensus.cc:2272] T a378d7f796f94615b3b347119a9880fd P 90f3da57796b488c9087a2f523adf045 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:16.345731 15046 tablet_replica.cc:333] T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045: stopping tablet replica
I20260709 13:23:16.346557 15046 raft_consensus.cc:2243] T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045 [term 26 FOLLOWER]: Raft consensus shutting down.
I20260709 13:23:16.347419 15046 raft_consensus.cc:2272] T 74ca8140f35c4ae29e8a664f6bdfc829 P 90f3da57796b488c9087a2f523adf045 [term 26 FOLLOWER]: Raft consensus is shut down!
W20260709 13:23:16.348111 16265 consensus_peers.cc:597] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 76: this message will repeat every 5th retry.
I20260709 13:23:16.349756 15046 tablet_replica.cc:333] T d2ab0ac8347641e6955e9bce7d90c1a2 P 90f3da57796b488c9087a2f523adf045: stopping tablet replica
I20260709 13:23:16.350385 15046 raft_consensus.cc:2243] T d2ab0ac8347641e6955e9bce7d90c1a2 P 90f3da57796b488c9087a2f523adf045 [term 22 LEADER]: Raft consensus shutting down.
I20260709 13:23:16.351330 15046 raft_consensus.cc:2272] T d2ab0ac8347641e6955e9bce7d90c1a2 P 90f3da57796b488c9087a2f523adf045 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:16.353232 15046 tablet_replica.cc:333] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 90f3da57796b488c9087a2f523adf045: stopping tablet replica
I20260709 13:23:16.353793 15046 raft_consensus.cc:2243] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 90f3da57796b488c9087a2f523adf045 [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 13:23:16.354346 15046 raft_consensus.cc:2272] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P 90f3da57796b488c9087a2f523adf045 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:16.356426 15046 tablet_replica.cc:333] T 02838c86531d4f98bf58d1c22b8baf77 P 90f3da57796b488c9087a2f523adf045: stopping tablet replica
I20260709 13:23:16.356978 15046 raft_consensus.cc:2243] T 02838c86531d4f98bf58d1c22b8baf77 P 90f3da57796b488c9087a2f523adf045 [term 25 LEADER]: Raft consensus shutting down.
I20260709 13:23:16.357828 15046 raft_consensus.cc:2272] T 02838c86531d4f98bf58d1c22b8baf77 P 90f3da57796b488c9087a2f523adf045 [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:16.359798 15046 tablet_replica.cc:333] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 90f3da57796b488c9087a2f523adf045: stopping tablet replica
I20260709 13:23:16.360342 15046 raft_consensus.cc:2243] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 90f3da57796b488c9087a2f523adf045 [term 24 LEADER]: Raft consensus shutting down.
I20260709 13:23:16.361346 15046 raft_consensus.cc:2272] T 3a7ca61592e848ea9b0fb81a2ce33cdc P 90f3da57796b488c9087a2f523adf045 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:16.363361 15046 tablet_replica.cc:333] T 8c11074579b046d99e585c2818e18a2c P 90f3da57796b488c9087a2f523adf045: stopping tablet replica
I20260709 13:23:16.363903 15046 raft_consensus.cc:2243] T 8c11074579b046d99e585c2818e18a2c P 90f3da57796b488c9087a2f523adf045 [term 21 LEADER]: Raft consensus shutting down.
I20260709 13:23:16.364708 15046 raft_consensus.cc:2272] T 8c11074579b046d99e585c2818e18a2c P 90f3da57796b488c9087a2f523adf045 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:16.366566 15046 tablet_replica.cc:333] T e4dc6b0541ca431ebc757beac00521f1 P 90f3da57796b488c9087a2f523adf045: stopping tablet replica
I20260709 13:23:16.367144 15046 raft_consensus.cc:2243] T e4dc6b0541ca431ebc757beac00521f1 P 90f3da57796b488c9087a2f523adf045 [term 24 LEADER]: Raft consensus shutting down.
I20260709 13:23:16.368278 15046 raft_consensus.cc:2272] T e4dc6b0541ca431ebc757beac00521f1 P 90f3da57796b488c9087a2f523adf045 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:16.370419 15046 tablet_replica.cc:333] T 434bff3082d141838068f323bdb0d1ed P 90f3da57796b488c9087a2f523adf045: stopping tablet replica
I20260709 13:23:16.371008 15046 raft_consensus.cc:2243] T 434bff3082d141838068f323bdb0d1ed P 90f3da57796b488c9087a2f523adf045 [term 20 LEADER]: Raft consensus shutting down.
I20260709 13:23:16.372045 15046 raft_consensus.cc:2272] T 434bff3082d141838068f323bdb0d1ed P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:16.374603 15046 tablet_replica.cc:333] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045: stopping tablet replica
I20260709 13:23:16.375494 15046 raft_consensus.cc:2243] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045 [term 25 LEADER]: Raft consensus shutting down.
I20260709 13:23:16.376633 15046 raft_consensus.cc:2272] T c2f5f89186204242bf377f7c42b2e2da P 90f3da57796b488c9087a2f523adf045 [term 25 FOLLOWER]: Raft consensus is shut down!
W20260709 13:23:16.377147 16789 consensus_peers.cc:597] T c9944d5e4b8c4109af9955c4f08c9095 P cb82cc7bd6db46d980c75237a1402860 -> Peer 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389): Couldn't send request to peer 90f3da57796b488c9087a2f523adf045. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:23:16.378961 15046 tablet_replica.cc:333] T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045: stopping tablet replica
I20260709 13:23:16.379768 15046 raft_consensus.cc:2243] T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045 [term 26 LEADER]: Raft consensus shutting down.
I20260709 13:23:16.381335 15046 raft_consensus.cc:2272] T b6f07842d3aa47858df5665874c86f3a P 90f3da57796b488c9087a2f523adf045 [term 26 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:16.384049 15046 tablet_replica.cc:333] T 66ae176961b74813a58bc15c9c4c6f2a P 90f3da57796b488c9087a2f523adf045: stopping tablet replica
I20260709 13:23:16.384828 15046 raft_consensus.cc:2243] T 66ae176961b74813a58bc15c9c4c6f2a P 90f3da57796b488c9087a2f523adf045 [term 29 FOLLOWER]: Raft consensus shutting down.
I20260709 13:23:16.385317 15046 raft_consensus.cc:2272] T 66ae176961b74813a58bc15c9c4c6f2a P 90f3da57796b488c9087a2f523adf045 [term 29 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:16.387724 15046 tablet_replica.cc:333] T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045: stopping tablet replica
I20260709 13:23:16.388322 15046 raft_consensus.cc:2243] T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045 [term 26 LEADER]: Raft consensus shutting down.
I20260709 13:23:16.389256 15046 raft_consensus.cc:2272] T 9c61a39fd4f84424b129c1a6f9ba404d P 90f3da57796b488c9087a2f523adf045 [term 26 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:16.391566 15046 tablet_replica.cc:333] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045: stopping tablet replica
I20260709 13:23:16.392120 15046 raft_consensus.cc:2243] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [term 27 LEADER]: Raft consensus shutting down.
I20260709 13:23:16.393090 15046 raft_consensus.cc:2272] T 0a9de347c711477e87c582d003f36925 P 90f3da57796b488c9087a2f523adf045 [term 27 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:16.396087 15046 tablet_replica.cc:333] T 23f82224f4564fea93572e8206bad1fe P 90f3da57796b488c9087a2f523adf045: stopping tablet replica
I20260709 13:23:16.396776 15046 raft_consensus.cc:2243] T 23f82224f4564fea93572e8206bad1fe P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:23:16.397399 15046 raft_consensus.cc:2272] T 23f82224f4564fea93572e8206bad1fe P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:16.400243 15046 tablet_replica.cc:333] T 26354361e9b044cea1c2b86c4882fe92 P 90f3da57796b488c9087a2f523adf045: stopping tablet replica
W20260709 13:23:16.400715 16789 consensus_peers.cc:597] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 -> Peer 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389): Couldn't send request to peer 90f3da57796b488c9087a2f523adf045. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:23:16.401074 15046 raft_consensus.cc:2243] T 26354361e9b044cea1c2b86c4882fe92 P 90f3da57796b488c9087a2f523adf045 [term 19 LEADER]: Raft consensus shutting down.
I20260709 13:23:16.402319 15046 raft_consensus.cc:2272] T 26354361e9b044cea1c2b86c4882fe92 P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:16.404457 15046 tablet_replica.cc:333] T 3884f864a96d428785895e5463e5728f P 90f3da57796b488c9087a2f523adf045: stopping tablet replica
I20260709 13:23:16.405051 15046 raft_consensus.cc:2243] T 3884f864a96d428785895e5463e5728f P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 13:23:16.405496 15046 raft_consensus.cc:2272] T 3884f864a96d428785895e5463e5728f P 90f3da57796b488c9087a2f523adf045 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:16.407814 15046 tablet_replica.cc:333] T fffab26427e24d398e2695aad40ca134 P 90f3da57796b488c9087a2f523adf045: stopping tablet replica
I20260709 13:23:16.408372 15046 raft_consensus.cc:2243] T fffab26427e24d398e2695aad40ca134 P 90f3da57796b488c9087a2f523adf045 [term 21 LEADER]: Raft consensus shutting down.
I20260709 13:23:16.409241 15046 raft_consensus.cc:2272] T fffab26427e24d398e2695aad40ca134 P 90f3da57796b488c9087a2f523adf045 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:16.411580 15046 tablet_replica.cc:333] T ada5a004ba7c46819051e3ccec71ffdb P 90f3da57796b488c9087a2f523adf045: stopping tablet replica
I20260709 13:23:16.412153 15046 raft_consensus.cc:2243] T ada5a004ba7c46819051e3ccec71ffdb P 90f3da57796b488c9087a2f523adf045 [term 26 FOLLOWER]: Raft consensus shutting down.
I20260709 13:23:16.412902 15046 raft_consensus.cc:2272] T ada5a004ba7c46819051e3ccec71ffdb P 90f3da57796b488c9087a2f523adf045 [term 26 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:16.415074 15046 tablet_replica.cc:333] T 8a878999c0024d2abe517484180290fc P 90f3da57796b488c9087a2f523adf045: stopping tablet replica
I20260709 13:23:16.415838 15046 raft_consensus.cc:2243] T 8a878999c0024d2abe517484180290fc P 90f3da57796b488c9087a2f523adf045 [term 21 LEADER]: Raft consensus shutting down.
I20260709 13:23:16.417073 15046 raft_consensus.cc:2272] T 8a878999c0024d2abe517484180290fc P 90f3da57796b488c9087a2f523adf045 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:16.420028 15046 tablet_replica.cc:333] T 7c2095a5f4ad49408dbc7f69c606120d P 90f3da57796b488c9087a2f523adf045: stopping tablet replica
I20260709 13:23:16.420768 15046 raft_consensus.cc:2243] T 7c2095a5f4ad49408dbc7f69c606120d P 90f3da57796b488c9087a2f523adf045 [term 19 LEADER]: Raft consensus shutting down.
I20260709 13:23:16.421876 15046 raft_consensus.cc:2272] T 7c2095a5f4ad49408dbc7f69c606120d P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:16.424605 15046 tablet_replica.cc:333] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045: stopping tablet replica
W20260709 13:23:16.425273 16789 consensus_peers.cc:597] T 8638ee363273426db1393dd81e5c7b72 P cb82cc7bd6db46d980c75237a1402860 -> Peer 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389): Couldn't send request to peer 90f3da57796b488c9087a2f523adf045. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:23:16.425319 15046 raft_consensus.cc:2243] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045 [term 25 LEADER]: Raft consensus shutting down.
I20260709 13:23:16.429078 15046 raft_consensus.cc:2272] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P 90f3da57796b488c9087a2f523adf045 [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:16.432063 15046 tablet_replica.cc:333] T 5a96373bb0e14dce8a266b2fc6c58507 P 90f3da57796b488c9087a2f523adf045: stopping tablet replica
I20260709 13:23:16.433025 15046 raft_consensus.cc:2243] T 5a96373bb0e14dce8a266b2fc6c58507 P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:23:16.433732 15046 raft_consensus.cc:2272] T 5a96373bb0e14dce8a266b2fc6c58507 P 90f3da57796b488c9087a2f523adf045 [term 19 FOLLOWER]: Raft consensus is shut down!
W20260709 13:23:16.434662 16789 consensus_peers.cc:597] T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860 -> Peer 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389): Couldn't send request to peer 90f3da57796b488c9087a2f523adf045. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:23:16.436645 15046 tablet_replica.cc:333] T 9e65676b21b748728ad39668cfe722ec P 90f3da57796b488c9087a2f523adf045: stopping tablet replica
I20260709 13:23:16.437340 15046 raft_consensus.cc:2243] T 9e65676b21b748728ad39668cfe722ec P 90f3da57796b488c9087a2f523adf045 [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 13:23:16.438033 15046 raft_consensus.cc:2272] T 9e65676b21b748728ad39668cfe722ec P 90f3da57796b488c9087a2f523adf045 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:17.078451 15046 tablet_replica.cc:333] T 18ad7e3af5504571a69d10c00750a01c P 90f3da57796b488c9087a2f523adf045: stopping tablet replica
I20260709 13:23:17.079437 15046 raft_consensus.cc:2243] T 18ad7e3af5504571a69d10c00750a01c P 90f3da57796b488c9087a2f523adf045 [term 20 LEADER]: Raft consensus shutting down.
I20260709 13:23:17.087663 15046 raft_consensus.cc:2272] T 18ad7e3af5504571a69d10c00750a01c P 90f3da57796b488c9087a2f523adf045 [term 20 FOLLOWER]: Raft consensus is shut down!
W20260709 13:23:17.157172 16789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 59206.6us
I20260709 13:23:17.290649 25085 raft_consensus.cc:493] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:23:17.291242 25085 raft_consensus.cc:515] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:23:17.293408 25085 leader_election.cc:290] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
I20260709 13:23:17.293944 25085 raft_consensus.cc:493] T 5a959dbecc2648eb85d1cc550f5a3245 P cb82cc7bd6db46d980c75237a1402860 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:23:17.294507 25085 raft_consensus.cc:515] T 5a959dbecc2648eb85d1cc550f5a3245 P cb82cc7bd6db46d980c75237a1402860 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:23:17.297509 25085 leader_election.cc:290] T 5a959dbecc2648eb85d1cc550f5a3245 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
W20260709 13:23:17.182065 16789 consensus_peers.cc:597] T 9e65676b21b748728ad39668cfe722ec P cb82cc7bd6db46d980c75237a1402860 -> Peer 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389): Couldn't send request to peer 90f3da57796b488c9087a2f523adf045. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:23:17.300369 16789 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41389, user_credentials={real_user=slave}} blocked reactor thread for 118351us
W20260709 13:23:17.301049 16789 consensus_peers.cc:597] T 806d7690c2b644959152074bb2cab0a7 P cb82cc7bd6db46d980c75237a1402860 -> Peer 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389): Couldn't send request to peer 90f3da57796b488c9087a2f523adf045. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:23:17.301558 16789 consensus_peers.cc:597] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860 -> Peer 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389): Couldn't send request to peer 90f3da57796b488c9087a2f523adf045. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:23:17.302690 16789 consensus_peers.cc:597] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860 -> Peer 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389): Couldn't send request to peer 90f3da57796b488c9087a2f523adf045. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:23:17.303634 16789 consensus_peers.cc:597] T 2794af5b99984efebdee564035841cb0 P cb82cc7bd6db46d980c75237a1402860 -> Peer 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389): Couldn't send request to peer 90f3da57796b488c9087a2f523adf045. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:23:17.320701 16790 consensus_peers.cc:597] T ada5a004ba7c46819051e3ccec71ffdb P cb82cc7bd6db46d980c75237a1402860 -> Peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943): Couldn't send request to peer 1ca2725d290e47d68b1a4b609c9816ac. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.130:35943: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:23:17.321452 16790 consensus_peers.cc:597] T 66f75124215f4ca6bef831e174a792d8 P cb82cc7bd6db46d980c75237a1402860 -> Peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943): Couldn't send request to peer 1ca2725d290e47d68b1a4b609c9816ac. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.130:35943: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:23:17.321835 16790 consensus_peers.cc:597] T 8638ee363273426db1393dd81e5c7b72 P cb82cc7bd6db46d980c75237a1402860 -> Peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943): Couldn't send request to peer 1ca2725d290e47d68b1a4b609c9816ac. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.130:35943: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:23:17.322376 16790 consensus_peers.cc:597] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 -> Peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943): Couldn't send request to peer 1ca2725d290e47d68b1a4b609c9816ac. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.130:35943: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:23:17.323468 16790 consensus_peers.cc:597] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 71: this message will repeat every 5th retry.
W20260709 13:23:17.323884 16790 consensus_peers.cc:597] T 806d7690c2b644959152074bb2cab0a7 P cb82cc7bd6db46d980c75237a1402860 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:23:17.324223 16790 consensus_peers.cc:597] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 76: this message will repeat every 5th retry.
W20260709 13:23:17.324587 16790 consensus_peers.cc:597] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:23:17.331956 16263 connection.cc:466] server connection from 127.0.0.1:40474 torn down before Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40474 (request call id 2442) could send its response
W20260709 13:23:17.332989 16263 connection.cc:466] server connection from 127.0.0.1:40474 torn down before Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40474 (request call id 2443) could send its response
W20260709 13:23:17.333786 16263 connection.cc:466] server connection from 127.0.0.1:40474 torn down before Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40474 (request call id 2444) could send its response
W20260709 13:23:17.334858 16790 leader_election.cc:336] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943): Network error: Client connection negotiation failed: client connection to 127.14.177.130:35943: connect: Connection refused (error 111)
W20260709 13:23:17.335520 16790 leader_election.cc:336] T 5a959dbecc2648eb85d1cc550f5a3245 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943): Network error: Client connection negotiation failed: client connection to 127.14.177.130:35943: connect: Connection refused (error 111)
W20260709 13:23:17.336156 16790 leader_election.cc:336] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111)
I20260709 13:23:17.336545 16790 leader_election.cc:304] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:23:17.337404 25087 raft_consensus.cc:2749] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:23:17.337883 16790 leader_election.cc:336] T 5a959dbecc2648eb85d1cc550f5a3245 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111)
I20260709 13:23:17.338366 16790 leader_election.cc:304] T 5a959dbecc2648eb85d1cc550f5a3245 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
W20260709 13:23:17.340430 16789 leader_election.cc:336] T 5a959dbecc2648eb85d1cc550f5a3245 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389): Network error: Client connection negotiation failed: client connection to 127.14.177.131:41389: connect: Connection refused (error 111)
I20260709 13:23:17.340559 25087 raft_consensus.cc:2749] T 5a959dbecc2648eb85d1cc550f5a3245 P cb82cc7bd6db46d980c75237a1402860 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:23:17.352744 25087 raft_consensus.cc:493] T 26354361e9b044cea1c2b86c4882fe92 P cb82cc7bd6db46d980c75237a1402860 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:23:17.353381 25087 raft_consensus.cc:515] T 26354361e9b044cea1c2b86c4882fe92 P cb82cc7bd6db46d980c75237a1402860 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:23:17.356046 25087 leader_election.cc:290] T 26354361e9b044cea1c2b86c4882fe92 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
W20260709 13:23:17.362224 16789 leader_election.cc:336] T 26354361e9b044cea1c2b86c4882fe92 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389): Network error: Client connection negotiation failed: client connection to 127.14.177.131:41389: connect: Connection refused (error 111)
W20260709 13:23:17.364964 16790 leader_election.cc:336] T 26354361e9b044cea1c2b86c4882fe92 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111)
I20260709 13:23:17.365412 16790 leader_election.cc:304] T 26354361e9b044cea1c2b86c4882fe92 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:23:17.366211 25087 raft_consensus.cc:2749] T 26354361e9b044cea1c2b86c4882fe92 P cb82cc7bd6db46d980c75237a1402860 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:23:17.425587 16790 consensus_peers.cc:597] T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860 -> Peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943): Couldn't send request to peer 1ca2725d290e47d68b1a4b609c9816ac. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.130:35943: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:23:17.494971 16790 consensus_peers.cc:597] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860 -> Peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943): Couldn't send request to peer 1ca2725d290e47d68b1a4b609c9816ac. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.130:35943: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:23:17.511955 25084 raft_consensus.cc:493] T 829b4ad8c5a54eababeea1181b960d4b P cb82cc7bd6db46d980c75237a1402860 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:23:17.512598 25084 raft_consensus.cc:515] T 829b4ad8c5a54eababeea1181b960d4b P cb82cc7bd6db46d980c75237a1402860 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:23:17.514835 25084 leader_election.cc:290] T 829b4ad8c5a54eababeea1181b960d4b P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
W20260709 13:23:17.524276 16790 leader_election.cc:336] T 829b4ad8c5a54eababeea1181b960d4b P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111)
W20260709 13:23:17.525177 16790 leader_election.cc:336] T 829b4ad8c5a54eababeea1181b960d4b P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943): Network error: Client connection negotiation failed: client connection to 127.14.177.130:35943: connect: Connection refused (error 111)
I20260709 13:23:17.525563 16790 leader_election.cc:304] T 829b4ad8c5a54eababeea1181b960d4b P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:23:17.526284 25084 raft_consensus.cc:2749] T 829b4ad8c5a54eababeea1181b960d4b P cb82cc7bd6db46d980c75237a1402860 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:23:17.533517 16790 consensus_peers.cc:597] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 -> Peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943): Couldn't send request to peer 1ca2725d290e47d68b1a4b609c9816ac. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.130:35943: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:23:17.541354 25084 raft_consensus.cc:493] T 3a7ca61592e848ea9b0fb81a2ce33cdc P cb82cc7bd6db46d980c75237a1402860 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:23:17.543201 25084 raft_consensus.cc:515] T 3a7ca61592e848ea9b0fb81a2ce33cdc P cb82cc7bd6db46d980c75237a1402860 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:23:17.550225 25084 leader_election.cc:290] T 3a7ca61592e848ea9b0fb81a2ce33cdc P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:23:17.551580 25084 raft_consensus.cc:493] T 18ad7e3af5504571a69d10c00750a01c P cb82cc7bd6db46d980c75237a1402860 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:23:17.552294 25084 raft_consensus.cc:515] T 18ad7e3af5504571a69d10c00750a01c P cb82cc7bd6db46d980c75237a1402860 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:23:17.554719 25084 leader_election.cc:290] T 18ad7e3af5504571a69d10c00750a01c P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
W20260709 13:23:17.564815 16789 leader_election.cc:336] T 3a7ca61592e848ea9b0fb81a2ce33cdc P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389): Network error: Client connection negotiation failed: client connection to 127.14.177.131:41389: connect: Connection refused (error 111)
W20260709 13:23:17.566609 16789 leader_election.cc:336] T 18ad7e3af5504571a69d10c00750a01c P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389): Network error: Client connection negotiation failed: client connection to 127.14.177.131:41389: connect: Connection refused (error 111)
W20260709 13:23:17.574987 16790 consensus_peers.cc:597] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 -> Peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943): Couldn't send request to peer 1ca2725d290e47d68b1a4b609c9816ac. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.130:35943: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:23:17.575469 16790 leader_election.cc:336] T 3a7ca61592e848ea9b0fb81a2ce33cdc P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943): Network error: Client connection negotiation failed: client connection to 127.14.177.130:35943: connect: Connection refused (error 111)
I20260709 13:23:17.575811 16790 leader_election.cc:304] T 3a7ca61592e848ea9b0fb81a2ce33cdc P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:23:17.576480 25084 raft_consensus.cc:2749] T 3a7ca61592e848ea9b0fb81a2ce33cdc P cb82cc7bd6db46d980c75237a1402860 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 13:23:17.576629 16790 leader_election.cc:336] T 18ad7e3af5504571a69d10c00750a01c P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943): Network error: Client connection negotiation failed: client connection to 127.14.177.130:35943: connect: Connection refused (error 111)
I20260709 13:23:17.577134 16790 leader_election.cc:304] T 18ad7e3af5504571a69d10c00750a01c P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:23:17.577943 25084 raft_consensus.cc:2749] T 18ad7e3af5504571a69d10c00750a01c P cb82cc7bd6db46d980c75237a1402860 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:23:17.578137 16790 consensus_peers.cc:597] T 74ca8140f35c4ae29e8a664f6bdfc829 P cb82cc7bd6db46d980c75237a1402860 -> Peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943): Couldn't send request to peer 1ca2725d290e47d68b1a4b609c9816ac. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.130:35943: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:23:17.583930 16790 consensus_peers.cc:597] T c9944d5e4b8c4109af9955c4f08c9095 P cb82cc7bd6db46d980c75237a1402860 -> Peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943): Couldn't send request to peer 1ca2725d290e47d68b1a4b609c9816ac. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.130:35943: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:23:17.585047 25084 raft_consensus.cc:493] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P cb82cc7bd6db46d980c75237a1402860 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:23:17.585594 25084 raft_consensus.cc:515] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P cb82cc7bd6db46d980c75237a1402860 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:23:17.587826 25084 leader_election.cc:290] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
W20260709 13:23:17.592901 16789 leader_election.cc:336] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389): Network error: Client connection negotiation failed: client connection to 127.14.177.131:41389: connect: Connection refused (error 111)
W20260709 13:23:17.601112 16790 leader_election.cc:336] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943): Network error: Client connection negotiation failed: client connection to 127.14.177.130:35943: connect: Connection refused (error 111)
I20260709 13:23:17.601708 16790 leader_election.cc:304] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:23:17.602461 25084 raft_consensus.cc:2749] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P cb82cc7bd6db46d980c75237a1402860 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:23:17.604741 25092 raft_consensus.cc:493] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [term 30 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:23:17.605245 25092 raft_consensus.cc:515] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:23:17.607259 25092 leader_election.cc:290] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
W20260709 13:23:17.617642 16790 leader_election.cc:336] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 31 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111)
I20260709 13:23:17.618561 25092 raft_consensus.cc:493] T d2ab0ac8347641e6955e9bce7d90c1a2 P cb82cc7bd6db46d980c75237a1402860 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:23:17.619151 25092 raft_consensus.cc:515] T d2ab0ac8347641e6955e9bce7d90c1a2 P cb82cc7bd6db46d980c75237a1402860 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:23:17.621592 25092 leader_election.cc:290] T d2ab0ac8347641e6955e9bce7d90c1a2 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
W20260709 13:23:17.622182 16790 leader_election.cc:336] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 31 pre-election: RPC error from VoteRequest() call to peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943): Network error: Client connection negotiation failed: client connection to 127.14.177.130:35943: connect: Connection refused (error 111)
I20260709 13:23:17.622695 16790 leader_election.cc:304] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:23:17.623442 25092 raft_consensus.cc:2749] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
W20260709 13:23:17.627115 16789 leader_election.cc:336] T d2ab0ac8347641e6955e9bce7d90c1a2 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389): Network error: Client connection negotiation failed: client connection to 127.14.177.131:41389: connect: Connection refused (error 111)
I20260709 13:23:17.629381 25092 raft_consensus.cc:493] T 8c11074579b046d99e585c2818e18a2c P cb82cc7bd6db46d980c75237a1402860 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:23:17.629946 25092 raft_consensus.cc:515] T 8c11074579b046d99e585c2818e18a2c P cb82cc7bd6db46d980c75237a1402860 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:23:17.631314 25084 raft_consensus.cc:493] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:23:17.632017 25092 leader_election.cc:290] T 8c11074579b046d99e585c2818e18a2c P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:23:17.631924 25084 raft_consensus.cc:515] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:23:17.636382 25084 leader_election.cc:290] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
W20260709 13:23:17.636543 16790 leader_election.cc:336] T d2ab0ac8347641e6955e9bce7d90c1a2 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943): Network error: Client connection negotiation failed: client connection to 127.14.177.130:35943: connect: Connection refused (error 111)
I20260709 13:23:17.637964 16790 leader_election.cc:304] T d2ab0ac8347641e6955e9bce7d90c1a2 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
W20260709 13:23:17.639426 16789 leader_election.cc:336] T 8c11074579b046d99e585c2818e18a2c P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389): Network error: Client connection negotiation failed: client connection to 127.14.177.131:41389: connect: Connection refused (error 111)
I20260709 13:23:17.640304 25084 raft_consensus.cc:2749] T d2ab0ac8347641e6955e9bce7d90c1a2 P cb82cc7bd6db46d980c75237a1402860 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 13:23:17.642772 16790 leader_election.cc:336] T 8c11074579b046d99e585c2818e18a2c P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943): Network error: Client connection negotiation failed: client connection to 127.14.177.130:35943: connect: Connection refused (error 111)
I20260709 13:23:17.643206 16790 leader_election.cc:304] T 8c11074579b046d99e585c2818e18a2c P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:23:17.643875 25084 raft_consensus.cc:2749] T 8c11074579b046d99e585c2818e18a2c P cb82cc7bd6db46d980c75237a1402860 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:23:17.644899 16789 leader_election.cc:336] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389): Network error: Client connection negotiation failed: client connection to 127.14.177.131:41389: connect: Connection refused (error 111)
I20260709 13:23:17.646034 25084 raft_consensus.cc:493] T 8d4ab870a05347868956aa8b841b2981 P cb82cc7bd6db46d980c75237a1402860 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
W20260709 13:23:17.647305 16790 leader_election.cc:336] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111)
I20260709 13:23:17.646692 25084 raft_consensus.cc:515] T 8d4ab870a05347868956aa8b841b2981 P cb82cc7bd6db46d980c75237a1402860 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:23:17.647807 16790 leader_election.cc:304] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:23:17.648612 25092 raft_consensus.cc:2749] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 13:23:17.650321 25084 leader_election.cc:290] T 8d4ab870a05347868956aa8b841b2981 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:23:17.651330 25084 raft_consensus.cc:493] T 7c2095a5f4ad49408dbc7f69c606120d P cb82cc7bd6db46d980c75237a1402860 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:23:17.651855 25084 raft_consensus.cc:515] T 7c2095a5f4ad49408dbc7f69c606120d P cb82cc7bd6db46d980c75237a1402860 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:23:17.655746 25084 leader_election.cc:290] T 7c2095a5f4ad49408dbc7f69c606120d P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
W20260709 13:23:17.659312 16789 leader_election.cc:336] T 8d4ab870a05347868956aa8b841b2981 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389): Network error: Client connection negotiation failed: client connection to 127.14.177.131:41389: connect: Connection refused (error 111)
W20260709 13:23:17.661198 16790 leader_election.cc:336] T 8d4ab870a05347868956aa8b841b2981 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943): Network error: Client connection negotiation failed: client connection to 127.14.177.130:35943: connect: Connection refused (error 111)
I20260709 13:23:17.662498 16790 leader_election.cc:304] T 8d4ab870a05347868956aa8b841b2981 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:23:17.663410 25084 raft_consensus.cc:2749] T 8d4ab870a05347868956aa8b841b2981 P cb82cc7bd6db46d980c75237a1402860 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 13:23:17.666409 16789 leader_election.cc:336] T 7c2095a5f4ad49408dbc7f69c606120d P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389): Network error: Client connection negotiation failed: client connection to 127.14.177.131:41389: connect: Connection refused (error 111)
W20260709 13:23:17.669262 16790 leader_election.cc:336] T 7c2095a5f4ad49408dbc7f69c606120d P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943): Network error: Client connection negotiation failed: client connection to 127.14.177.130:35943: connect: Connection refused (error 111)
I20260709 13:23:17.669787 16790 leader_election.cc:304] T 7c2095a5f4ad49408dbc7f69c606120d P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:23:17.670492 25084 raft_consensus.cc:2749] T 7c2095a5f4ad49408dbc7f69c606120d P cb82cc7bd6db46d980c75237a1402860 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:23:17.670903 16790 consensus_peers.cc:597] T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 81: this message will repeat every 5th retry.
I20260709 13:23:17.697959 25084 raft_consensus.cc:493] T 19baa7bc367844c288bca8f81439c413 P cb82cc7bd6db46d980c75237a1402860 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:23:17.698582 25084 raft_consensus.cc:515] T 19baa7bc367844c288bca8f81439c413 P cb82cc7bd6db46d980c75237a1402860 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:23:17.704530 25084 leader_election.cc:290] T 19baa7bc367844c288bca8f81439c413 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
W20260709 13:23:17.709563 16790 leader_election.cc:336] T 19baa7bc367844c288bca8f81439c413 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943): Network error: Client connection negotiation failed: client connection to 127.14.177.130:35943: connect: Connection refused (error 111)
W20260709 13:23:17.712229 16790 consensus_peers.cc:597] T 3884f864a96d428785895e5463e5728f P cb82cc7bd6db46d980c75237a1402860 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 81: this message will repeat every 5th retry.
W20260709 13:23:17.712831 16789 leader_election.cc:336] T 19baa7bc367844c288bca8f81439c413 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389): Network error: Client connection negotiation failed: client connection to 127.14.177.131:41389: connect: Connection refused (error 111)
I20260709 13:23:17.713337 16789 leader_election.cc:304] T 19baa7bc367844c288bca8f81439c413 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 90f3da57796b488c9087a2f523adf045
I20260709 13:23:17.714071 25084 raft_consensus.cc:2749] T 19baa7bc367844c288bca8f81439c413 P cb82cc7bd6db46d980c75237a1402860 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:23:17.733877 25084 raft_consensus.cc:493] T b87147731a68400e8cce9b879daa7c5d P cb82cc7bd6db46d980c75237a1402860 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:23:17.734565 25084 raft_consensus.cc:515] T b87147731a68400e8cce9b879daa7c5d P cb82cc7bd6db46d980c75237a1402860 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:23:17.736707 25092 raft_consensus.cc:493] T c4f65cdf00b748bb963ee6e81ef43b79 P cb82cc7bd6db46d980c75237a1402860 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:23:17.737385 25092 raft_consensus.cc:515] T c4f65cdf00b748bb963ee6e81ef43b79 P cb82cc7bd6db46d980c75237a1402860 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } attrs { replace: true } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:23:17.739861 25092 leader_election.cc:290] T c4f65cdf00b748bb963ee6e81ef43b79 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
I20260709 13:23:17.736800 25084 leader_election.cc:290] T b87147731a68400e8cce9b879daa7c5d P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
W20260709 13:23:17.742988 16789 leader_election.cc:336] T b87147731a68400e8cce9b879daa7c5d P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389): Network error: Client connection negotiation failed: client connection to 127.14.177.131:41389: connect: Connection refused (error 111)
W20260709 13:23:17.747619 16789 leader_election.cc:336] T c4f65cdf00b748bb963ee6e81ef43b79 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389): Network error: Client connection negotiation failed: client connection to 127.14.177.131:41389: connect: Connection refused (error 111)
W20260709 13:23:17.748620 16790 leader_election.cc:336] T b87147731a68400e8cce9b879daa7c5d P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111)
I20260709 13:23:17.749172 16790 leader_election.cc:304] T b87147731a68400e8cce9b879daa7c5d P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:23:17.749927 25084 raft_consensus.cc:2749] T b87147731a68400e8cce9b879daa7c5d P cb82cc7bd6db46d980c75237a1402860 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 13:23:17.750087 16790 leader_election.cc:336] T c4f65cdf00b748bb963ee6e81ef43b79 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111)
I20260709 13:23:17.750571 16790 leader_election.cc:304] T c4f65cdf00b748bb963ee6e81ef43b79 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:23:17.751263 25084 raft_consensus.cc:2749] T c4f65cdf00b748bb963ee6e81ef43b79 P cb82cc7bd6db46d980c75237a1402860 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 13:23:17.751444 16790 leader_election.cc:336] T c4f65cdf00b748bb963ee6e81ef43b79 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943): Network error: Client connection negotiation failed: client connection to 127.14.177.130:35943: connect: Connection refused (error 111)
I20260709 13:23:17.752735 25084 raft_consensus.cc:493] T 434bff3082d141838068f323bdb0d1ed P cb82cc7bd6db46d980c75237a1402860 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:23:17.753211 25084 raft_consensus.cc:515] T 434bff3082d141838068f323bdb0d1ed P cb82cc7bd6db46d980c75237a1402860 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:23:17.755319 25084 leader_election.cc:290] T 434bff3082d141838068f323bdb0d1ed P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
W20260709 13:23:17.759994 16790 leader_election.cc:336] T 434bff3082d141838068f323bdb0d1ed P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111)
W20260709 13:23:17.760511 16789 leader_election.cc:336] T 434bff3082d141838068f323bdb0d1ed P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389): Network error: Client connection negotiation failed: client connection to 127.14.177.131:41389: connect: Connection refused (error 111)
I20260709 13:23:17.760943 16789 leader_election.cc:304] T 434bff3082d141838068f323bdb0d1ed P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:23:17.761646 25084 raft_consensus.cc:2749] T 434bff3082d141838068f323bdb0d1ed P cb82cc7bd6db46d980c75237a1402860 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:23:17.793185 25084 raft_consensus.cc:493] T b6f07842d3aa47858df5665874c86f3a P cb82cc7bd6db46d980c75237a1402860 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 90f3da57796b488c9087a2f523adf045)
I20260709 13:23:17.793772 25084 raft_consensus.cc:515] T b6f07842d3aa47858df5665874c86f3a P cb82cc7bd6db46d980c75237a1402860 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } attrs { replace: true } } peers { permanent_uuid: "90f3da57796b488c9087a2f523adf045" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41389 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:23:17.796793 25084 leader_election.cc:290] T b6f07842d3aa47858df5665874c86f3a P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943), 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389)
W20260709 13:23:17.802098 16789 leader_election.cc:336] T b6f07842d3aa47858df5665874c86f3a P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer 90f3da57796b488c9087a2f523adf045 (127.14.177.131:41389): Network error: Client connection negotiation failed: client connection to 127.14.177.131:41389: connect: Connection refused (error 111)
W20260709 13:23:17.803717 16790 leader_election.cc:336] T b6f07842d3aa47858df5665874c86f3a P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111)
I20260709 13:23:17.804188 16790 leader_election.cc:304] T b6f07842d3aa47858df5665874c86f3a P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 647a343a542d4a139fe106c19095129b, 90f3da57796b488c9087a2f523adf045
I20260709 13:23:17.804948 25084 raft_consensus.cc:2749] T b6f07842d3aa47858df5665874c86f3a P cb82cc7bd6db46d980c75237a1402860 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
W20260709 13:23:17.805078 16790 leader_election.cc:336] T b6f07842d3aa47858df5665874c86f3a P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943): Network error: Client connection negotiation failed: client connection to 127.14.177.130:35943: connect: Connection refused (error 111)
I20260709 13:23:17.812526 25084 raft_consensus.cc:493] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:23:17.813095 25084 raft_consensus.cc:515] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:23:17.815341 25084 leader_election.cc:290] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
W20260709 13:23:17.821183 16790 leader_election.cc:336] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111)
W20260709 13:23:17.822230 16790 leader_election.cc:336] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943): Network error: Client connection negotiation failed: client connection to 127.14.177.130:35943: connect: Connection refused (error 111)
I20260709 13:23:17.822674 16790 leader_election.cc:304] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:23:17.823473 25084 raft_consensus.cc:2749] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 13:23:17.843870 16035 catalog_manager.cc:4755] TS 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet 19baa7bc367844c288bca8f81439c413: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111)
W20260709 13:23:17.970657 16790 consensus_peers.cc:597] T 57a744f5aaf944678a63469cbf514015 P cb82cc7bd6db46d980c75237a1402860 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 81: this message will repeat every 5th retry.
W20260709 13:23:17.993008 16790 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111) [suppressed 594 similar messages]
W20260709 13:23:18.028179 16790 consensus_peers.cc:597] T 66f75124215f4ca6bef831e174a792d8 P cb82cc7bd6db46d980c75237a1402860 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 76: this message will repeat every 5th retry.
W20260709 13:23:18.049607 16846 debug-util.cc:398] Leaking SignalData structure 0x7b08003b6500 after lost signal to thread 15049
W20260709 13:23:18.050889 16846 debug-util.cc:398] Leaking SignalData structure 0x7b0800383ae0 after lost signal to thread 16083
W20260709 13:23:18.051571 16846 debug-util.cc:398] Leaking SignalData structure 0x7b08003623c0 after lost signal to thread 16324
W20260709 13:23:18.052884 16846 debug-util.cc:398] Leaking SignalData structure 0x7b08003ecc40 after lost signal to thread 16849
I20260709 13:23:18.169533 15046 tablet_server.cc:196] TabletServer@127.14.177.131:0 shutdown complete.
W20260709 13:23:18.184427 16080 debug-util.cc:398] Leaking SignalData structure 0x7b0800408760 after lost signal to thread 15049
W20260709 13:23:18.185559 16080 debug-util.cc:398] Leaking SignalData structure 0x7b08002fb6a0 after lost signal to thread 16083
W20260709 13:23:18.186175 16080 debug-util.cc:398] Leaking SignalData structure 0x7b08002c1240 after lost signal to thread 16324
W20260709 13:23:18.187498 16080 debug-util.cc:398] Leaking SignalData structure 0x7b0800285fe0 after lost signal to thread 16849
I20260709 13:23:18.274335 25092 raft_consensus.cc:493] T ee9ed67006144797b1de6ea8c2013a68 P cb82cc7bd6db46d980c75237a1402860 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 1ca2725d290e47d68b1a4b609c9816ac)
I20260709 13:23:18.275034 25092 raft_consensus.cc:515] T ee9ed67006144797b1de6ea8c2013a68 P cb82cc7bd6db46d980c75237a1402860 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "647a343a542d4a139fe106c19095129b" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36741 } } peers { permanent_uuid: "1ca2725d290e47d68b1a4b609c9816ac" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35943 } } peers { permanent_uuid: "cb82cc7bd6db46d980c75237a1402860" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 38923 } attrs { promote: false } }
I20260709 13:23:18.283308 15046 tablet_server.cc:179] TabletServer@127.14.177.132:0 shutting down...
I20260709 13:23:18.278980 25092 leader_election.cc:290] T ee9ed67006144797b1de6ea8c2013a68 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 647a343a542d4a139fe106c19095129b (127.14.177.129:36741), 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943)
W20260709 13:23:18.291154 16790 leader_election.cc:336] T ee9ed67006144797b1de6ea8c2013a68 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111)
W20260709 13:23:18.295110 16790 leader_election.cc:336] T ee9ed67006144797b1de6ea8c2013a68 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 1ca2725d290e47d68b1a4b609c9816ac (127.14.177.130:35943): Network error: Client connection negotiation failed: client connection to 127.14.177.130:35943: connect: Connection refused (error 111)
I20260709 13:23:18.295940 16790 leader_election.cc:304] T ee9ed67006144797b1de6ea8c2013a68 P cb82cc7bd6db46d980c75237a1402860 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb82cc7bd6db46d980c75237a1402860; no voters: 1ca2725d290e47d68b1a4b609c9816ac, 647a343a542d4a139fe106c19095129b
I20260709 13:23:18.296828 25092 raft_consensus.cc:2749] T ee9ed67006144797b1de6ea8c2013a68 P cb82cc7bd6db46d980c75237a1402860 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:23:18.367996 15046 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:23:18.369073 15046 tablet_replica.cc:333] T 5a959dbecc2648eb85d1cc550f5a3245 P cb82cc7bd6db46d980c75237a1402860: stopping tablet replica
I20260709 13:23:18.370267 15046 raft_consensus.cc:2243] T 5a959dbecc2648eb85d1cc550f5a3245 P cb82cc7bd6db46d980c75237a1402860 [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 13:23:18.371156 15046 raft_consensus.cc:2272] T 5a959dbecc2648eb85d1cc550f5a3245 P cb82cc7bd6db46d980c75237a1402860 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:18.374550 15046 tablet_replica.cc:333] T 18ad7e3af5504571a69d10c00750a01c P cb82cc7bd6db46d980c75237a1402860: stopping tablet replica
I20260709 13:23:18.375547 15046 raft_consensus.cc:2243] T 18ad7e3af5504571a69d10c00750a01c P cb82cc7bd6db46d980c75237a1402860 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:23:18.387414 15046 raft_consensus.cc:2272] T 18ad7e3af5504571a69d10c00750a01c P cb82cc7bd6db46d980c75237a1402860 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:18.390010 15046 tablet_replica.cc:333] T 7c2095a5f4ad49408dbc7f69c606120d P cb82cc7bd6db46d980c75237a1402860: stopping tablet replica
I20260709 13:23:18.390710 15046 raft_consensus.cc:2243] T 7c2095a5f4ad49408dbc7f69c606120d P cb82cc7bd6db46d980c75237a1402860 [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:23:18.391378 15046 raft_consensus.cc:2272] T 7c2095a5f4ad49408dbc7f69c606120d P cb82cc7bd6db46d980c75237a1402860 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:18.393710 15046 tablet_replica.cc:333] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860: stopping tablet replica
I20260709 13:23:18.394392 15046 raft_consensus.cc:2243] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860 [term 27 LEADER]: Raft consensus shutting down.
I20260709 13:23:18.395489 15046 raft_consensus.cc:2272] T ecc1e363230043fe9fef390f0b979f20 P cb82cc7bd6db46d980c75237a1402860 [term 27 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:18.398718 15046 tablet_replica.cc:333] T b6f07842d3aa47858df5665874c86f3a P cb82cc7bd6db46d980c75237a1402860: stopping tablet replica
I20260709 13:23:18.399725 15046 raft_consensus.cc:2243] T b6f07842d3aa47858df5665874c86f3a P cb82cc7bd6db46d980c75237a1402860 [term 26 FOLLOWER]: Raft consensus shutting down.
I20260709 13:23:18.400418 15046 raft_consensus.cc:2272] T b6f07842d3aa47858df5665874c86f3a P cb82cc7bd6db46d980c75237a1402860 [term 26 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:18.403640 15046 tablet_replica.cc:333] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860: stopping tablet replica
I20260709 13:23:18.404628 15046 raft_consensus.cc:2243] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860 [term 27 FOLLOWER]: Raft consensus shutting down.
I20260709 13:23:18.405296 15046 raft_consensus.cc:2272] T 0a9de347c711477e87c582d003f36925 P cb82cc7bd6db46d980c75237a1402860 [term 27 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:18.408402 15046 tablet_replica.cc:333] T 8c11074579b046d99e585c2818e18a2c P cb82cc7bd6db46d980c75237a1402860: stopping tablet replica
I20260709 13:23:18.409322 15046 raft_consensus.cc:2243] T 8c11074579b046d99e585c2818e18a2c P cb82cc7bd6db46d980c75237a1402860 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:23:18.410068 15046 raft_consensus.cc:2272] T 8c11074579b046d99e585c2818e18a2c P cb82cc7bd6db46d980c75237a1402860 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:18.413174 15046 tablet_replica.cc:333] T 806d7690c2b644959152074bb2cab0a7 P cb82cc7bd6db46d980c75237a1402860: stopping tablet replica
I20260709 13:23:18.414039 15046 raft_consensus.cc:2243] T 806d7690c2b644959152074bb2cab0a7 P cb82cc7bd6db46d980c75237a1402860 [term 24 LEADER]: Raft consensus shutting down.
I20260709 13:23:18.415825 15046 raft_consensus.cc:2272] T 806d7690c2b644959152074bb2cab0a7 P cb82cc7bd6db46d980c75237a1402860 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:18.419551 15046 tablet_replica.cc:333] T 829b4ad8c5a54eababeea1181b960d4b P cb82cc7bd6db46d980c75237a1402860: stopping tablet replica
I20260709 13:23:18.420665 15046 raft_consensus.cc:2243] T 829b4ad8c5a54eababeea1181b960d4b P cb82cc7bd6db46d980c75237a1402860 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:23:18.421497 15046 raft_consensus.cc:2272] T 829b4ad8c5a54eababeea1181b960d4b P cb82cc7bd6db46d980c75237a1402860 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:18.425058 15046 tablet_replica.cc:333] T c9944d5e4b8c4109af9955c4f08c9095 P cb82cc7bd6db46d980c75237a1402860: stopping tablet replica
I20260709 13:23:18.426178 15046 raft_consensus.cc:2243] T c9944d5e4b8c4109af9955c4f08c9095 P cb82cc7bd6db46d980c75237a1402860 [term 24 LEADER]: Raft consensus shutting down.
I20260709 13:23:18.427546 15046 raft_consensus.cc:2272] T c9944d5e4b8c4109af9955c4f08c9095 P cb82cc7bd6db46d980c75237a1402860 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:18.431003 15046 tablet_replica.cc:333] T af57996a8be84d398f0f023e8d08ef77 P cb82cc7bd6db46d980c75237a1402860: stopping tablet replica
I20260709 13:23:18.432075 15046 raft_consensus.cc:2243] T af57996a8be84d398f0f023e8d08ef77 P cb82cc7bd6db46d980c75237a1402860 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:23:18.432924 15046 raft_consensus.cc:2272] T af57996a8be84d398f0f023e8d08ef77 P cb82cc7bd6db46d980c75237a1402860 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:18.436357 15046 tablet_replica.cc:333] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P cb82cc7bd6db46d980c75237a1402860: stopping tablet replica
I20260709 13:23:18.437368 15046 raft_consensus.cc:2243] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P cb82cc7bd6db46d980c75237a1402860 [term 25 FOLLOWER]: Raft consensus shutting down.
I20260709 13:23:18.438086 15046 raft_consensus.cc:2272] T 9e4bfb9090c8460eb14fa7e0d818e7f1 P cb82cc7bd6db46d980c75237a1402860 [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:18.441332 15046 tablet_replica.cc:333] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860: stopping tablet replica
I20260709 13:23:18.442473 15046 raft_consensus.cc:2243] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860 [term 24 LEADER]: Raft consensus shutting down.
I20260709 13:23:18.443962 15046 raft_consensus.cc:2272] T 4e9b2bf7011f4ae688a79d7bb5a447b4 P cb82cc7bd6db46d980c75237a1402860 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:18.447528 15046 tablet_replica.cc:333] T 19baa7bc367844c288bca8f81439c413 P cb82cc7bd6db46d980c75237a1402860: stopping tablet replica
I20260709 13:23:18.448652 15046 raft_consensus.cc:2243] T 19baa7bc367844c288bca8f81439c413 P cb82cc7bd6db46d980c75237a1402860 [term 25 FOLLOWER]: Raft consensus shutting down.
I20260709 13:23:18.449577 15046 raft_consensus.cc:2272] T 19baa7bc367844c288bca8f81439c413 P cb82cc7bd6db46d980c75237a1402860 [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:18.453022 15046 tablet_replica.cc:333] T 3a7ca61592e848ea9b0fb81a2ce33cdc P cb82cc7bd6db46d980c75237a1402860: stopping tablet replica
I20260709 13:23:18.454052 15046 raft_consensus.cc:2243] T 3a7ca61592e848ea9b0fb81a2ce33cdc P cb82cc7bd6db46d980c75237a1402860 [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 13:23:18.454795 15046 raft_consensus.cc:2272] T 3a7ca61592e848ea9b0fb81a2ce33cdc P cb82cc7bd6db46d980c75237a1402860 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:18.457937 15046 tablet_replica.cc:333] T c4f65cdf00b748bb963ee6e81ef43b79 P cb82cc7bd6db46d980c75237a1402860: stopping tablet replica
I20260709 13:23:18.459010 15046 raft_consensus.cc:2243] T c4f65cdf00b748bb963ee6e81ef43b79 P cb82cc7bd6db46d980c75237a1402860 [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 13:23:18.459870 15046 raft_consensus.cc:2272] T c4f65cdf00b748bb963ee6e81ef43b79 P cb82cc7bd6db46d980c75237a1402860 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:18.463523 15046 tablet_replica.cc:333] T 8638ee363273426db1393dd81e5c7b72 P cb82cc7bd6db46d980c75237a1402860: stopping tablet replica
I20260709 13:23:18.464587 15046 raft_consensus.cc:2243] T 8638ee363273426db1393dd81e5c7b72 P cb82cc7bd6db46d980c75237a1402860 [term 25 LEADER]: Raft consensus shutting down.
I20260709 13:23:18.466116 15046 raft_consensus.cc:2272] T 8638ee363273426db1393dd81e5c7b72 P cb82cc7bd6db46d980c75237a1402860 [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:18.469482 15046 tablet_replica.cc:333] T 66f75124215f4ca6bef831e174a792d8 P cb82cc7bd6db46d980c75237a1402860: stopping tablet replica
I20260709 13:23:18.470526 15046 raft_consensus.cc:2243] T 66f75124215f4ca6bef831e174a792d8 P cb82cc7bd6db46d980c75237a1402860 [term 21 LEADER]: Raft consensus shutting down.
I20260709 13:23:18.471879 15046 raft_consensus.cc:2272] T 66f75124215f4ca6bef831e174a792d8 P cb82cc7bd6db46d980c75237a1402860 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:18.475059 15046 tablet_replica.cc:333] T 3884f864a96d428785895e5463e5728f P cb82cc7bd6db46d980c75237a1402860: stopping tablet replica
I20260709 13:23:18.475976 15046 raft_consensus.cc:2243] T 3884f864a96d428785895e5463e5728f P cb82cc7bd6db46d980c75237a1402860 [term 17 LEADER]: Raft consensus shutting down.
I20260709 13:23:18.477417 15046 raft_consensus.cc:2272] T 3884f864a96d428785895e5463e5728f P cb82cc7bd6db46d980c75237a1402860 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:18.480741 15046 tablet_replica.cc:333] T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860: stopping tablet replica
I20260709 13:23:18.481730 15046 raft_consensus.cc:2243] T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860 [term 24 LEADER]: Raft consensus shutting down.
I20260709 13:23:18.483104 15046 raft_consensus.cc:2272] T 1d5d59407eb545b895d8d922e0dd69fd P cb82cc7bd6db46d980c75237a1402860 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:18.486419 15046 tablet_replica.cc:333] T 57a744f5aaf944678a63469cbf514015 P cb82cc7bd6db46d980c75237a1402860: stopping tablet replica
I20260709 13:23:18.487591 15046 raft_consensus.cc:2243] T 57a744f5aaf944678a63469cbf514015 P cb82cc7bd6db46d980c75237a1402860 [term 20 LEADER]: Raft consensus shutting down.
I20260709 13:23:18.489154 15046 raft_consensus.cc:2272] T 57a744f5aaf944678a63469cbf514015 P cb82cc7bd6db46d980c75237a1402860 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:18.493009 15046 tablet_replica.cc:333] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860: stopping tablet replica
I20260709 13:23:18.494074 15046 raft_consensus.cc:2243] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [term 30 FOLLOWER]: Raft consensus shutting down.
I20260709 13:23:18.494910 15046 raft_consensus.cc:2272] T 62fe3346cd3c42c4a55e14af9073bca4 P cb82cc7bd6db46d980c75237a1402860 [term 30 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:18.504812 15046 tablet_replica.cc:333] T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860: stopping tablet replica
I20260709 13:23:18.505978 15046 raft_consensus.cc:2243] T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860 [term 24 LEADER]: Raft consensus shutting down.
I20260709 13:23:18.507504 15046 raft_consensus.cc:2272] T a378d7f796f94615b3b347119a9880fd P cb82cc7bd6db46d980c75237a1402860 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:18.511057 15046 tablet_replica.cc:333] T ada5a004ba7c46819051e3ccec71ffdb P cb82cc7bd6db46d980c75237a1402860: stopping tablet replica
I20260709 13:23:18.557009 15046 raft_consensus.cc:2243] T ada5a004ba7c46819051e3ccec71ffdb P cb82cc7bd6db46d980c75237a1402860 [term 26 LEADER]: Raft consensus shutting down.
I20260709 13:23:18.563805 15046 raft_consensus.cc:2272] T ada5a004ba7c46819051e3ccec71ffdb P cb82cc7bd6db46d980c75237a1402860 [term 26 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:18.581005 15046 tablet_replica.cc:333] T 9e65676b21b748728ad39668cfe722ec P cb82cc7bd6db46d980c75237a1402860: stopping tablet replica
I20260709 13:23:18.582696 15046 raft_consensus.cc:2243] T 9e65676b21b748728ad39668cfe722ec P cb82cc7bd6db46d980c75237a1402860 [term 23 LEADER]: Raft consensus shutting down.
I20260709 13:23:18.585312 15046 raft_consensus.cc:2272] T 9e65676b21b748728ad39668cfe722ec P cb82cc7bd6db46d980c75237a1402860 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:18.589982 15046 tablet_replica.cc:333] T ee9ed67006144797b1de6ea8c2013a68 P cb82cc7bd6db46d980c75237a1402860: stopping tablet replica
I20260709 13:23:18.592219 15046 raft_consensus.cc:2243] T ee9ed67006144797b1de6ea8c2013a68 P cb82cc7bd6db46d980c75237a1402860 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:23:18.593444 15046 raft_consensus.cc:2272] T ee9ed67006144797b1de6ea8c2013a68 P cb82cc7bd6db46d980c75237a1402860 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:18.597661 15046 tablet_replica.cc:333] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860: stopping tablet replica
I20260709 13:23:18.599345 15046 raft_consensus.cc:2243] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860 [term 22 LEADER]: Raft consensus shutting down.
I20260709 13:23:18.602602 15046 raft_consensus.cc:2272] T 31969392ccdb43b491911a319e8bc7e1 P cb82cc7bd6db46d980c75237a1402860 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:18.607064 15046 tablet_replica.cc:333] T 26354361e9b044cea1c2b86c4882fe92 P cb82cc7bd6db46d980c75237a1402860: stopping tablet replica
I20260709 13:23:18.608215 15046 raft_consensus.cc:2243] T 26354361e9b044cea1c2b86c4882fe92 P cb82cc7bd6db46d980c75237a1402860 [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:23:18.609450 15046 raft_consensus.cc:2272] T 26354361e9b044cea1c2b86c4882fe92 P cb82cc7bd6db46d980c75237a1402860 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:18.613212 15046 tablet_replica.cc:333] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860: stopping tablet replica
I20260709 13:23:18.614616 15046 raft_consensus.cc:2243] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:23:18.615839 15046 raft_consensus.cc:2272] T 746aabf856ab401c9ef5e859371e7b67 P cb82cc7bd6db46d980c75237a1402860 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:18.621891 15046 tablet_replica.cc:333] T 434bff3082d141838068f323bdb0d1ed P cb82cc7bd6db46d980c75237a1402860: stopping tablet replica
I20260709 13:23:18.622785 15046 raft_consensus.cc:2243] T 434bff3082d141838068f323bdb0d1ed P cb82cc7bd6db46d980c75237a1402860 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:23:18.624236 15046 raft_consensus.cc:2272] T 434bff3082d141838068f323bdb0d1ed P cb82cc7bd6db46d980c75237a1402860 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:18.628738 15046 tablet_replica.cc:333] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860: stopping tablet replica
I20260709 13:23:18.629633 15046 raft_consensus.cc:2243] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 [term 24 LEADER]: Raft consensus shutting down.
I20260709 13:23:18.632187 15046 raft_consensus.cc:2272] T 5718c623ebee4d2bb770f0f9ddb2f164 P cb82cc7bd6db46d980c75237a1402860 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:18.636749 15046 tablet_replica.cc:333] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860: stopping tablet replica
I20260709 13:23:18.637809 15046 raft_consensus.cc:2243] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 13:23:18.639137 15046 raft_consensus.cc:2272] T 08b7d1ab519044eda2d5e04a07242f25 P cb82cc7bd6db46d980c75237a1402860 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:18.643430 15046 tablet_replica.cc:333] T 2794af5b99984efebdee564035841cb0 P cb82cc7bd6db46d980c75237a1402860: stopping tablet replica
I20260709 13:23:18.644383 15046 raft_consensus.cc:2243] T 2794af5b99984efebdee564035841cb0 P cb82cc7bd6db46d980c75237a1402860 [term 20 LEADER]: Raft consensus shutting down.
W20260709 13:23:18.645758 16790 consensus_peers.cc:597] T 23f82224f4564fea93572e8206bad1fe P cb82cc7bd6db46d980c75237a1402860 -> Peer 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): Couldn't send request to peer 647a343a542d4a139fe106c19095129b. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111). This is attempt 86: this message will repeat every 5th retry.
I20260709 13:23:18.647572 15046 raft_consensus.cc:2272] T 2794af5b99984efebdee564035841cb0 P cb82cc7bd6db46d980c75237a1402860 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:18.651722 15046 tablet_replica.cc:333] T 8d4ab870a05347868956aa8b841b2981 P cb82cc7bd6db46d980c75237a1402860: stopping tablet replica
I20260709 13:23:18.652899 15046 raft_consensus.cc:2243] T 8d4ab870a05347868956aa8b841b2981 P cb82cc7bd6db46d980c75237a1402860 [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 13:23:18.655476 15046 raft_consensus.cc:2272] T 8d4ab870a05347868956aa8b841b2981 P cb82cc7bd6db46d980c75237a1402860 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:18.660696 15046 tablet_replica.cc:333] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860: stopping tablet replica
I20260709 13:23:18.661755 15046 raft_consensus.cc:2243] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 [term 29 LEADER]: Raft consensus shutting down.
I20260709 13:23:18.663797 15046 raft_consensus.cc:2272] T cdbd78acb4cf46298e0de8ba0d130e19 P cb82cc7bd6db46d980c75237a1402860 [term 29 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:18.667981 15046 tablet_replica.cc:333] T 23f82224f4564fea93572e8206bad1fe P cb82cc7bd6db46d980c75237a1402860: stopping tablet replica
I20260709 13:23:18.668955 15046 raft_consensus.cc:2243] T 23f82224f4564fea93572e8206bad1fe P cb82cc7bd6db46d980c75237a1402860 [term 19 LEADER]: Raft consensus shutting down.
I20260709 13:23:18.671069 15046 raft_consensus.cc:2272] T 23f82224f4564fea93572e8206bad1fe P cb82cc7bd6db46d980c75237a1402860 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:18.675559 15046 tablet_replica.cc:333] T d2ab0ac8347641e6955e9bce7d90c1a2 P cb82cc7bd6db46d980c75237a1402860: stopping tablet replica
I20260709 13:23:18.676509 15046 raft_consensus.cc:2243] T d2ab0ac8347641e6955e9bce7d90c1a2 P cb82cc7bd6db46d980c75237a1402860 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:23:18.677846 15046 raft_consensus.cc:2272] T d2ab0ac8347641e6955e9bce7d90c1a2 P cb82cc7bd6db46d980c75237a1402860 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:18.681155 15046 tablet_replica.cc:333] T 74ca8140f35c4ae29e8a664f6bdfc829 P cb82cc7bd6db46d980c75237a1402860: stopping tablet replica
I20260709 13:23:18.682024 15046 raft_consensus.cc:2243] T 74ca8140f35c4ae29e8a664f6bdfc829 P cb82cc7bd6db46d980c75237a1402860 [term 26 LEADER]: Raft consensus shutting down.
I20260709 13:23:18.684185 15046 raft_consensus.cc:2272] T 74ca8140f35c4ae29e8a664f6bdfc829 P cb82cc7bd6db46d980c75237a1402860 [term 26 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:18.687652 15046 tablet_replica.cc:333] T b87147731a68400e8cce9b879daa7c5d P cb82cc7bd6db46d980c75237a1402860: stopping tablet replica
I20260709 13:23:18.688468 15046 raft_consensus.cc:2243] T b87147731a68400e8cce9b879daa7c5d P cb82cc7bd6db46d980c75237a1402860 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:23:18.689447 15046 raft_consensus.cc:2272] T b87147731a68400e8cce9b879daa7c5d P cb82cc7bd6db46d980c75237a1402860 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:18.692551 15046 tablet_replica.cc:333] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860: stopping tablet replica
I20260709 13:23:18.693364 15046 raft_consensus.cc:2243] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [term 29 LEADER]: Raft consensus shutting down.
I20260709 13:23:18.695242 15046 raft_consensus.cc:2272] T 66ae176961b74813a58bc15c9c4c6f2a P cb82cc7bd6db46d980c75237a1402860 [term 29 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:18.699319 15046 tablet_replica.cc:333] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860: stopping tablet replica
I20260709 13:23:18.700151 15046 raft_consensus.cc:2243] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 [term 27 FOLLOWER]: Raft consensus shutting down.
I20260709 13:23:18.701889 15046 raft_consensus.cc:2272] T d2372981006d4b218215c28964dea623 P cb82cc7bd6db46d980c75237a1402860 [term 27 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:19.707769 15046 tablet_server.cc:196] TabletServer@127.14.177.132:0 shutdown complete.
I20260709 13:23:19.817195 15046 master.cc:562] Master@127.14.177.190:44079 shutting down...
W20260709 13:23:19.878360 16097 auto_rebalancer.cc:873] Removing replace marker failed: Catalog manager is not initialized. State: Closing
W20260709 13:23:19.878813 16097 auto_rebalancer.cc:901] Could not move replica: Service unavailable: Catalog manager is not initialized. State: Closing
W20260709 13:23:19.879042 16097 auto_rebalancer.cc:330] scheduled replica move failed to complete: Service unavailable: Catalog manager is not initialized. State: Closing
W20260709 13:23:22.421921 16035 catalog_manager.cc:4755] TS 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet 18ad7e3af5504571a69d10c00750a01c: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111)
W20260709 13:23:26.089519 16035 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111) [suppressed 49 similar messages]
W20260709 13:23:26.093366 16035 catalog_manager.cc:4755] TS 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet 19baa7bc367844c288bca8f81439c413: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111)
W20260709 13:23:27.094089 16035 catalog_manager.cc:4755] TS 647a343a542d4a139fe106c19095129b (127.14.177.129:36741): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet 8d4ab870a05347868956aa8b841b2981: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36741: connect: Connection refused (error 111)
I20260709 13:23:28.611429 15046 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 667a34b27a39474ebefc1af3cf5b4271 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:23:28.612583 15046 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 667a34b27a39474ebefc1af3cf5b4271 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:28.612987 15046 tablet_replica.cc:333] T 00000000000000000000000000000000 P 667a34b27a39474ebefc1af3cf5b4271: stopping tablet replica
I20260709 13:23:28.778277 15046 master.cc:584] Master@127.14.177.190:44079 shutdown complete.
I20260709 13:23:28.832963 15046 test_util.cc:182] -----------------------------------------------
I20260709 13:23:28.833196 15046 test_util.cc:183] Had failures, leaving test files at /tmp/dist-test-taskdH_Ma7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783603093801613-15046-0
[  FAILED  ] LeaderRebalancerTest.AddTserver (291828 ms)
[ RUN      ] LeaderRebalancerTest.RestartTserver
I20260709 13:23:28.836953 15046 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.14.177.190:45841
I20260709 13:23:28.838176 15046 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:23:28.844812 25109 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:23:28.845351 25110 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:23:28.845531 25112 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:23:28.846570 15046 server_base.cc:1061] running on GCE node
I20260709 13:23:28.849891 15046 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:23:28.850121 15046 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:23:28.850334 15046 hybrid_clock.cc:648] HybridClock initialized: now 1783603408850299 us; error 0 us; skew 500 ppm
I20260709 13:23:28.850869 15046 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:23:28.853605 15046 webserver.cc:533] Webserver started at http://127.14.177.190:34797/ using document root <none> and password file <none>
I20260709 13:23:28.854185 15046 fs_manager.cc:362] Metadata directory not provided
I20260709 13:23:28.854413 15046 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:23:28.854700 15046 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:23:28.856251 15046 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskdH_Ma7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783603093801613-15046-0/minicluster-data/master-0-root/instance:
uuid: "4b6484c4860b4e75a6322b756be0a819"
format_stamp: "Formatted at 2026-07-09 13:23:28 on dist-test-slave-v5z2"
I20260709 13:23:28.860945 15046 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.005s	sys 0.000s
I20260709 13:23:28.864398 25117 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:23:28.865181 15046 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.001s
I20260709 13:23:28.865607 15046 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskdH_Ma7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783603093801613-15046-0/minicluster-data/master-0-root
uuid: "4b6484c4860b4e75a6322b756be0a819"
format_stamp: "Formatted at 2026-07-09 13:23:28 on dist-test-slave-v5z2"
I20260709 13:23:28.865872 15046 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdH_Ma7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783603093801613-15046-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskdH_Ma7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783603093801613-15046-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskdH_Ma7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783603093801613-15046-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:23:28.893705 15046 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:23:28.895047 15046 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:23:28.938149 15046 rpc_server.cc:307] RPC server started. Bound to: 127.14.177.190:45841
I20260709 13:23:28.938277 25168 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.14.177.190:45841 every 8 connection(s)
I20260709 13:23:28.942814 25169 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:23:28.956512 25169 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4b6484c4860b4e75a6322b756be0a819: Bootstrap starting.
I20260709 13:23:28.962144 25169 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4b6484c4860b4e75a6322b756be0a819: Neither blocks nor log segments found. Creating new log.
I20260709 13:23:28.967124 25169 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4b6484c4860b4e75a6322b756be0a819: No bootstrap required, opened a new log
I20260709 13:23:28.969640 25169 raft_consensus.cc:359] T 00000000000000000000000000000000 P 4b6484c4860b4e75a6322b756be0a819 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b6484c4860b4e75a6322b756be0a819" member_type: VOTER }
I20260709 13:23:28.970158 25169 raft_consensus.cc:385] T 00000000000000000000000000000000 P 4b6484c4860b4e75a6322b756be0a819 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:23:28.970391 25169 raft_consensus.cc:740] T 00000000000000000000000000000000 P 4b6484c4860b4e75a6322b756be0a819 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b6484c4860b4e75a6322b756be0a819, State: Initialized, Role: FOLLOWER
I20260709 13:23:28.971001 25169 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4b6484c4860b4e75a6322b756be0a819 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b6484c4860b4e75a6322b756be0a819" member_type: VOTER }
I20260709 13:23:28.971652 25169 raft_consensus.cc:399] T 00000000000000000000000000000000 P 4b6484c4860b4e75a6322b756be0a819 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 13:23:28.971901 25169 raft_consensus.cc:493] T 00000000000000000000000000000000 P 4b6484c4860b4e75a6322b756be0a819 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 13:23:28.972182 25169 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 4b6484c4860b4e75a6322b756be0a819 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:23:28.977267 25169 raft_consensus.cc:515] T 00000000000000000000000000000000 P 4b6484c4860b4e75a6322b756be0a819 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b6484c4860b4e75a6322b756be0a819" member_type: VOTER }
I20260709 13:23:28.977877 25169 leader_election.cc:304] T 00000000000000000000000000000000 P 4b6484c4860b4e75a6322b756be0a819 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4b6484c4860b4e75a6322b756be0a819; no voters: 
I20260709 13:23:28.979215 25169 leader_election.cc:290] T 00000000000000000000000000000000 P 4b6484c4860b4e75a6322b756be0a819 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 13:23:28.979592 25172 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 4b6484c4860b4e75a6322b756be0a819 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:23:28.981086 25172 raft_consensus.cc:697] T 00000000000000000000000000000000 P 4b6484c4860b4e75a6322b756be0a819 [term 1 LEADER]: Becoming Leader. State: Replica: 4b6484c4860b4e75a6322b756be0a819, State: Running, Role: LEADER
I20260709 13:23:28.981884 25172 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4b6484c4860b4e75a6322b756be0a819 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b6484c4860b4e75a6322b756be0a819" member_type: VOTER }
I20260709 13:23:28.982332 25169 sys_catalog.cc:565] T 00000000000000000000000000000000 P 4b6484c4860b4e75a6322b756be0a819 [sys.catalog]: configured and running, proceeding with master startup.
I20260709 13:23:28.989584 25173 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4b6484c4860b4e75a6322b756be0a819 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4b6484c4860b4e75a6322b756be0a819" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b6484c4860b4e75a6322b756be0a819" member_type: VOTER } }
I20260709 13:23:28.990633 25174 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4b6484c4860b4e75a6322b756be0a819 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4b6484c4860b4e75a6322b756be0a819. Latest consensus state: current_term: 1 leader_uuid: "4b6484c4860b4e75a6322b756be0a819" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b6484c4860b4e75a6322b756be0a819" member_type: VOTER } }
I20260709 13:23:28.990644 25173 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4b6484c4860b4e75a6322b756be0a819 [sys.catalog]: This master's current role is: LEADER
I20260709 13:23:28.991331 25174 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4b6484c4860b4e75a6322b756be0a819 [sys.catalog]: This master's current role is: LEADER
I20260709 13:23:28.993155 25181 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 13:23:28.998261 25181 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 13:23:29.000655 15046 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 13:23:29.009536 25181 catalog_manager.cc:1383] Generated new cluster ID: 14d1813b35104b228368c3cbb347dd1f
I20260709 13:23:29.009876 25181 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 13:23:29.039606 25181 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 13:23:29.040985 25181 catalog_manager.cc:1540] Loading token signing keys...
I20260709 13:23:29.053020 25181 catalog_manager.cc:6070] T 00000000000000000000000000000000 P 4b6484c4860b4e75a6322b756be0a819: Generated new TSK 0
I20260709 13:23:29.053822 25181 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 13:23:29.067893 15046 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:23:29.075054 25190 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:23:29.076045 25191 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:23:29.077701 25193 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:23:29.078399 15046 server_base.cc:1061] running on GCE node
I20260709 13:23:29.081013 15046 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:23:29.081228 15046 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:23:29.081442 15046 hybrid_clock.cc:648] HybridClock initialized: now 1783603409081404 us; error 0 us; skew 500 ppm
I20260709 13:23:29.082047 15046 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:23:29.084751 15046 webserver.cc:533] Webserver started at http://127.14.177.129:35817/ using document root <none> and password file <none>
I20260709 13:23:29.085244 15046 fs_manager.cc:362] Metadata directory not provided
I20260709 13:23:29.085443 15046 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:23:29.085727 15046 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:23:29.086963 15046 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskdH_Ma7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783603093801613-15046-0/minicluster-data/ts-0-root/instance:
uuid: "43999d813e564dc688fad116090c95ca"
format_stamp: "Formatted at 2026-07-09 13:23:29 on dist-test-slave-v5z2"
I20260709 13:23:29.091892 15046 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.004s	sys 0.000s
I20260709 13:23:29.095762 25198 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:23:29.096616 15046 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.002s	sys 0.002s
I20260709 13:23:29.097002 15046 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskdH_Ma7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783603093801613-15046-0/minicluster-data/ts-0-root
uuid: "43999d813e564dc688fad116090c95ca"
format_stamp: "Formatted at 2026-07-09 13:23:29 on dist-test-slave-v5z2"
I20260709 13:23:29.097316 15046 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdH_Ma7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783603093801613-15046-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskdH_Ma7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783603093801613-15046-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskdH_Ma7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783603093801613-15046-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:23:29.133555 15046 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:23:29.134869 15046 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:23:29.136683 15046 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:23:29.139266 15046 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:23:29.139523 15046 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:23:29.139781 15046 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:23:29.140048 15046 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:23:29.185971 15046 rpc_server.cc:307] RPC server started. Bound to: 127.14.177.129:38155
I20260709 13:23:29.186096 25260 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.14.177.129:38155 every 8 connection(s)
I20260709 13:23:29.191469 15046 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:23:29.199520 25265 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:23:29.201617 25266 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:23:29.205327 25268 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:23:29.205981 25261 heartbeater.cc:344] Connected to a master server at 127.14.177.190:45841
I20260709 13:23:29.206467 25261 heartbeater.cc:461] Registering TS with master...
I20260709 13:23:29.207113 15046 server_base.cc:1061] running on GCE node
I20260709 13:23:29.207360 25261 heartbeater.cc:507] Master 127.14.177.190:45841 requested a full tablet report, sending...
I20260709 13:23:29.210042 25134 ts_manager.cc:194] Registered new tserver with Master: 43999d813e564dc688fad116090c95ca (127.14.177.129:38155)
I20260709 13:23:29.210343 15046 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:23:29.210662 15046 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:23:29.210845 15046 hybrid_clock.cc:648] HybridClock initialized: now 1783603409210812 us; error 0 us; skew 500 ppm
I20260709 13:23:29.211535 15046 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:23:29.212388 25134 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:35118
I20260709 13:23:29.214711 15046 webserver.cc:533] Webserver started at http://127.14.177.130:32939/ using document root <none> and password file <none>
I20260709 13:23:29.215396 15046 fs_manager.cc:362] Metadata directory not provided
I20260709 13:23:29.215636 15046 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:23:29.215939 15046 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:23:29.217267 15046 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskdH_Ma7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783603093801613-15046-0/minicluster-data/ts-1-root/instance:
uuid: "233e7d29aac04505bf4e65f0a39a0c4c"
format_stamp: "Formatted at 2026-07-09 13:23:29 on dist-test-slave-v5z2"
I20260709 13:23:29.222170 15046 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.006s	sys 0.000s
I20260709 13:23:29.226135 25273 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:23:29.226990 15046 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.000s	sys 0.002s
I20260709 13:23:29.227586 15046 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskdH_Ma7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783603093801613-15046-0/minicluster-data/ts-1-root
uuid: "233e7d29aac04505bf4e65f0a39a0c4c"
format_stamp: "Formatted at 2026-07-09 13:23:29 on dist-test-slave-v5z2"
I20260709 13:23:29.227944 15046 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdH_Ma7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783603093801613-15046-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskdH_Ma7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783603093801613-15046-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskdH_Ma7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783603093801613-15046-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:23:29.247720 15046 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:23:29.249047 15046 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:23:29.250641 15046 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:23:29.253124 15046 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:23:29.253399 15046 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:23:29.253669 15046 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:23:29.253847 15046 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:23:29.299649 15046 rpc_server.cc:307] RPC server started. Bound to: 127.14.177.130:42957
I20260709 13:23:29.299764 25335 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.14.177.130:42957 every 8 connection(s)
I20260709 13:23:29.304796 15046 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:23:29.313045 25339 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:23:29.314704 25340 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:23:29.317131 15046 server_base.cc:1061] running on GCE node
I20260709 13:23:29.318389 25336 heartbeater.cc:344] Connected to a master server at 127.14.177.190:45841
I20260709 13:23:29.318980 25336 heartbeater.cc:461] Registering TS with master...
W20260709 13:23:29.319324 25342 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:23:29.320041 25336 heartbeater.cc:507] Master 127.14.177.190:45841 requested a full tablet report, sending...
I20260709 13:23:29.320454 15046 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:23:29.320731 15046 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:23:29.320986 15046 hybrid_clock.cc:648] HybridClock initialized: now 1783603409320966 us; error 0 us; skew 500 ppm
I20260709 13:23:29.321841 15046 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:23:29.322468 25134 ts_manager.cc:194] Registered new tserver with Master: 233e7d29aac04505bf4e65f0a39a0c4c (127.14.177.130:42957)
I20260709 13:23:29.324158 25134 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:35126
I20260709 13:23:29.324545 15046 webserver.cc:533] Webserver started at http://127.14.177.131:46107/ using document root <none> and password file <none>
I20260709 13:23:29.325189 15046 fs_manager.cc:362] Metadata directory not provided
I20260709 13:23:29.325461 15046 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:23:29.325758 15046 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:23:29.327013 15046 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskdH_Ma7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783603093801613-15046-0/minicluster-data/ts-2-root/instance:
uuid: "3e07d64466934da4bb3c02262b1c2e38"
format_stamp: "Formatted at 2026-07-09 13:23:29 on dist-test-slave-v5z2"
I20260709 13:23:29.331851 15046 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.004s	sys 0.000s
I20260709 13:23:29.335263 25347 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:23:29.336247 15046 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.001s
I20260709 13:23:29.336511 15046 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskdH_Ma7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783603093801613-15046-0/minicluster-data/ts-2-root
uuid: "3e07d64466934da4bb3c02262b1c2e38"
format_stamp: "Formatted at 2026-07-09 13:23:29 on dist-test-slave-v5z2"
I20260709 13:23:29.336753 15046 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdH_Ma7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783603093801613-15046-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskdH_Ma7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783603093801613-15046-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskdH_Ma7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783603093801613-15046-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:23:29.354252 15046 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:23:29.355891 15046 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:23:29.357676 15046 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:23:29.360210 15046 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:23:29.360476 15046 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:23:29.360770 15046 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:23:29.360968 15046 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:23:29.416497 15046 rpc_server.cc:307] RPC server started. Bound to: 127.14.177.131:41539
I20260709 13:23:29.416558 25409 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.14.177.131:41539 every 8 connection(s)
I20260709 13:23:29.422307 15046 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:23:29.431265 25413 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:23:29.433511 25414 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:23:29.434800 15046 server_base.cc:1061] running on GCE node
W20260709 13:23:29.435220 25416 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:23:29.436425 15046 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:23:29.436721 15046 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:23:29.436980 15046 hybrid_clock.cc:648] HybridClock initialized: now 1783603409436953 us; error 0 us; skew 500 ppm
I20260709 13:23:29.437683 15046 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:23:29.440184 25410 heartbeater.cc:344] Connected to a master server at 127.14.177.190:45841
I20260709 13:23:29.440582 25410 heartbeater.cc:461] Registering TS with master...
I20260709 13:23:29.441170 15046 webserver.cc:533] Webserver started at http://127.14.177.132:36441/ using document root <none> and password file <none>
I20260709 13:23:29.441572 25410 heartbeater.cc:507] Master 127.14.177.190:45841 requested a full tablet report, sending...
I20260709 13:23:29.441758 15046 fs_manager.cc:362] Metadata directory not provided
I20260709 13:23:29.442085 15046 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:23:29.442371 15046 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:23:29.443884 15046 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskdH_Ma7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783603093801613-15046-0/minicluster-data/ts-3-root/instance:
uuid: "9868dce521294d0b816bba7648f4ae5a"
format_stamp: "Formatted at 2026-07-09 13:23:29 on dist-test-slave-v5z2"
I20260709 13:23:29.443946 25134 ts_manager.cc:194] Registered new tserver with Master: 3e07d64466934da4bb3c02262b1c2e38 (127.14.177.131:41539)
I20260709 13:23:29.445947 25134 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:35142
I20260709 13:23:29.449506 15046 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.005s	sys 0.000s
I20260709 13:23:29.453238 25421 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:23:29.454054 15046 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.000s	sys 0.002s
I20260709 13:23:29.454421 15046 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskdH_Ma7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783603093801613-15046-0/minicluster-data/ts-3-root
uuid: "9868dce521294d0b816bba7648f4ae5a"
format_stamp: "Formatted at 2026-07-09 13:23:29 on dist-test-slave-v5z2"
I20260709 13:23:29.454738 15046 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdH_Ma7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783603093801613-15046-0/minicluster-data/ts-3-root
metadata directory: /tmp/dist-test-taskdH_Ma7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783603093801613-15046-0/minicluster-data/ts-3-root
1 data directories: /tmp/dist-test-taskdH_Ma7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783603093801613-15046-0/minicluster-data/ts-3-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:23:29.482357 15046 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:23:29.483719 15046 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:23:29.485330 15046 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:23:29.487670 15046 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:23:29.487919 15046 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:23:29.488165 15046 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:23:29.488349 15046 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:23:29.532183 15046 rpc_server.cc:307] RPC server started. Bound to: 127.14.177.132:33683
I20260709 13:23:29.532312 25483 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.14.177.132:33683 every 8 connection(s)
I20260709 13:23:29.548017 25484 heartbeater.cc:344] Connected to a master server at 127.14.177.190:45841
I20260709 13:23:29.548493 25484 heartbeater.cc:461] Registering TS with master...
I20260709 13:23:29.549297 25484 heartbeater.cc:507] Master 127.14.177.190:45841 requested a full tablet report, sending...
I20260709 13:23:29.551401 25134 ts_manager.cc:194] Registered new tserver with Master: 9868dce521294d0b816bba7648f4ae5a (127.14.177.132:33683)
I20260709 13:23:29.552114 15046 internal_mini_cluster.cc:371] 4 TS(s) registered with all masters after 0.016534261s
I20260709 13:23:29.553332 25134 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:35154
I20260709 13:23:29.580868 25134 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35158:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\313c+\002\004\001\000\226\307V\004\004\001\000a+\202\006\004\001\000,\217\255\010\004\001\000\367\362\330\n\004\001\000\302V\004\r\004\001\000\215\272/\017\004\001\000X\036[\021\004\001\000#\202\206\023\004\001\000\356\345\261\025\004\001\000\271I\335\027\004\001\000\204\255\010\032\004\001\000O\0214\034\004\001\000\032u_\036\004\001\000\345\330\212 \004\001\000\260<\266\"\004\001\000{\240\341$\004\001\000F\004\r\'\004\001\000\021h8)\004\001\000\334\313c+\004\001\000\247/\217-\004\001\000r\223\272/\004\001\000=\367\3451\004\001\000\010[\0214\004\001\000\323\276<6\004\001\000\236\"h8\004\001\000i\206\223:\004\001\0004\352\276<\004\001\000\377M\352>\004\001\000\312\261\025A\004\001\000\225\025AC\004\001\000`ylE\004\001\000+\335\227G\004\001\000\366@\303I\004\001\000\301\244\356K\004\001\000\214\010\032N\004\001\000WlEP\004\001\000\"\320pR\004\001\000\3553\234T\004\001\000\270\227\307V\004\001\000\203\373\362X\004\001\000N_\036[\004\001\000\031\303I]\004\001\000\344&u_\004\001\000\257\212\240a\004\001\000z\356\313c\004\001\000ER\367e\004\001\000\020\266\"h\004\001\000\333\031Nj\004\001\000\246}yl\004\001\000q\341\244n\004\001\000<E\320p\004\001\000\007\251\373r\004\001\000\322\014\'u\004\001\000\235pRw\004\001\000h\324}y\004\001\00038\251{\004\001\000\376\233\324}""\004\001\000\313c+\002\004\001\000\226\307V\004\004\001\000a+\202\006\004\001\000,\217\255\010\004\001\000\367\362\330\n\004\001\000\302V\004\r\004\001\000\215\272/\017\004\001\000X\036[\021\004\001\000#\202\206\023\004\001\000\356\345\261\025\004\001\000\271I\335\027\004\001\000\204\255\010\032\004\001\000O\0214\034\004\001\000\032u_\036\004\001\000\345\330\212 \004\001\000\260<\266\"\004\001\000{\240\341$\004\001\000F\004\r\'\004\001\000\021h8)\004\001\000\334\313c+\004\001\000\247/\217-\004\001\000r\223\272/\004\001\000=\367\3451\004\001\000\010[\0214\004\001\000\323\276<6\004\001\000\236\"h8\004\001\000i\206\223:\004\001\0004\352\276<\004\001\000\377M\352>\004\001\000\312\261\025A\004\001\000\225\025AC\004\001\000`ylE\004\001\000+\335\227G\004\001\000\366@\303I\004\001\000\301\244\356K\004\001\000\214\010\032N\004\001\000WlEP\004\001\000\"\320pR\004\001\000\3553\234T\004\001\000\270\227\307V\004\001\000\203\373\362X\004\001\000N_\036[\004\001\000\031\303I]\004\001\000\344&u_\004\001\000\257\212\240a\004\001\000z\356\313c\004\001\000ER\367e\004\001\000\020\266\"h\004\001\000\333\031Nj\004\001\000\246}yl\004\001\000q\341\244n\004\001\000<E\320p\004\001\000\007\251\373r\004\001\000\322\014\'u\004\001\000\235pRw\004\001\000h\324}y\004\001\00038\251{\004\001\000\376\233\324}"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
I20260709 13:23:29.848800 25301 tablet_service.cc:1511] Processing CreateTablet for tablet b2f14aed05e24a98ad83535029b795dc (DEFAULT_TABLE table=test-workload [id=2f3f768873514f53bb58e92ba96f4813]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 13:23:29.850523 25301 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b2f14aed05e24a98ad83535029b795dc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:23:29.855118 25300 tablet_service.cc:1511] Processing CreateTablet for tablet 579da2d6aabb4ae9a2b1802e1358068b (DEFAULT_TABLE table=test-workload [id=2f3f768873514f53bb58e92ba96f4813]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 13:23:29.857220 25300 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 579da2d6aabb4ae9a2b1802e1358068b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:23:29.871752 25299 tablet_service.cc:1511] Processing CreateTablet for tablet 467a6b42ca6548269c12dddde696f7af (DEFAULT_TABLE table=test-workload [id=2f3f768873514f53bb58e92ba96f4813]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 13:23:29.872574 25374 tablet_service.cc:1511] Processing CreateTablet for tablet b7a891bfc89e4e259f53da6cb16f4989 (DEFAULT_TABLE table=test-workload [id=2f3f768873514f53bb58e92ba96f4813]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 13:23:29.873485 25299 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 467a6b42ca6548269c12dddde696f7af. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:23:29.874140 25374 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b7a891bfc89e4e259f53da6cb16f4989. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:23:29.877903 25225 tablet_service.cc:1511] Processing CreateTablet for tablet 579da2d6aabb4ae9a2b1802e1358068b (DEFAULT_TABLE table=test-workload [id=2f3f768873514f53bb58e92ba96f4813]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 13:23:29.879096 25223 tablet_service.cc:1511] Processing CreateTablet for tablet 28c0b6fc9bd64d60828d705c6a9ab1e4 (DEFAULT_TABLE table=test-workload [id=2f3f768873514f53bb58e92ba96f4813]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 13:23:29.880630 25298 tablet_service.cc:1511] Processing CreateTablet for tablet 28c0b6fc9bd64d60828d705c6a9ab1e4 (DEFAULT_TABLE table=test-workload [id=2f3f768873514f53bb58e92ba96f4813]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 13:23:29.880767 25373 tablet_service.cc:1511] Processing CreateTablet for tablet 467a6b42ca6548269c12dddde696f7af (DEFAULT_TABLE table=test-workload [id=2f3f768873514f53bb58e92ba96f4813]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 13:23:29.881374 25223 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 28c0b6fc9bd64d60828d705c6a9ab1e4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:23:29.882212 25298 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 28c0b6fc9bd64d60828d705c6a9ab1e4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:23:29.882382 25373 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 467a6b42ca6548269c12dddde696f7af. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:23:29.876839 25375 tablet_service.cc:1511] Processing CreateTablet for tablet b2f14aed05e24a98ad83535029b795dc (DEFAULT_TABLE table=test-workload [id=2f3f768873514f53bb58e92ba96f4813]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 13:23:29.883843 25224 tablet_service.cc:1511] Processing CreateTablet for tablet b7a891bfc89e4e259f53da6cb16f4989 (DEFAULT_TABLE table=test-workload [id=2f3f768873514f53bb58e92ba96f4813]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 13:23:29.884822 25375 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b2f14aed05e24a98ad83535029b795dc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:23:29.885458 25224 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b7a891bfc89e4e259f53da6cb16f4989. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:23:29.887319 25372 tablet_service.cc:1511] Processing CreateTablet for tablet 28c0b6fc9bd64d60828d705c6a9ab1e4 (DEFAULT_TABLE table=test-workload [id=2f3f768873514f53bb58e92ba96f4813]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 13:23:29.888960 25372 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 28c0b6fc9bd64d60828d705c6a9ab1e4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:23:29.892669 25371 tablet_service.cc:1511] Processing CreateTablet for tablet d0a0e4d32ce943fbb773bb840863647b (DEFAULT_TABLE table=test-workload [id=2f3f768873514f53bb58e92ba96f4813]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 13:23:29.876695 25226 tablet_service.cc:1511] Processing CreateTablet for tablet b2f14aed05e24a98ad83535029b795dc (DEFAULT_TABLE table=test-workload [id=2f3f768873514f53bb58e92ba96f4813]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 13:23:29.894423 25371 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d0a0e4d32ce943fbb773bb840863647b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:23:29.894002 25297 tablet_service.cc:1511] Processing CreateTablet for tablet 441d780c3945463f92f31170f31e37a8 (DEFAULT_TABLE table=test-workload [id=2f3f768873514f53bb58e92ba96f4813]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 13:23:29.894721 25226 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b2f14aed05e24a98ad83535029b795dc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:23:29.895658 25297 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 441d780c3945463f92f31170f31e37a8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:23:29.900715 25296 tablet_service.cc:1511] Processing CreateTablet for tablet 3fe5a944a342411ea923eb39071ac770 (DEFAULT_TABLE table=test-workload [id=2f3f768873514f53bb58e92ba96f4813]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 13:23:29.901832 25295 tablet_service.cc:1511] Processing CreateTablet for tablet b7ff8dde9f5d4c49b9203d902f6b94c4 (DEFAULT_TABLE table=test-workload [id=2f3f768873514f53bb58e92ba96f4813]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 13:23:29.902452 25296 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3fe5a944a342411ea923eb39071ac770. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:23:29.905146 25225 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 579da2d6aabb4ae9a2b1802e1358068b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:23:29.904788 25370 tablet_service.cc:1511] Processing CreateTablet for tablet 3fe5a944a342411ea923eb39071ac770 (DEFAULT_TABLE table=test-workload [id=2f3f768873514f53bb58e92ba96f4813]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 13:23:29.906397 25370 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3fe5a944a342411ea923eb39071ac770. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:23:29.907661 25295 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b7ff8dde9f5d4c49b9203d902f6b94c4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:23:29.911036 25369 tablet_service.cc:1511] Processing CreateTablet for tablet b7ff8dde9f5d4c49b9203d902f6b94c4 (DEFAULT_TABLE table=test-workload [id=2f3f768873514f53bb58e92ba96f4813]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 13:23:29.912755 25369 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b7ff8dde9f5d4c49b9203d902f6b94c4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:23:29.913210 25294 tablet_service.cc:1511] Processing CreateTablet for tablet e9e342ecda4b46619ce12452b2cf7154 (DEFAULT_TABLE table=test-workload [id=2f3f768873514f53bb58e92ba96f4813]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 13:23:29.914996 25294 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e9e342ecda4b46619ce12452b2cf7154. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:23:29.917109 25368 tablet_service.cc:1511] Processing CreateTablet for tablet 0e71b6f5c69b4b35a8ac51c22c7d5f0d (DEFAULT_TABLE table=test-workload [id=2f3f768873514f53bb58e92ba96f4813]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 13:23:29.918733 25368 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0e71b6f5c69b4b35a8ac51c22c7d5f0d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:23:29.921319 25293 tablet_service.cc:1511] Processing CreateTablet for tablet a820954e07a94582ac0613342fcffde5 (DEFAULT_TABLE table=test-workload [id=2f3f768873514f53bb58e92ba96f4813]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 13:23:29.925417 25292 tablet_service.cc:1511] Processing CreateTablet for tablet 0d29b499fa4c41f1b974782cd0c9c1c2 (DEFAULT_TABLE table=test-workload [id=2f3f768873514f53bb58e92ba96f4813]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 13:23:29.923060 25293 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a820954e07a94582ac0613342fcffde5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:23:29.928653 25366 tablet_service.cc:1511] Processing CreateTablet for tablet 0d29b499fa4c41f1b974782cd0c9c1c2 (DEFAULT_TABLE table=test-workload [id=2f3f768873514f53bb58e92ba96f4813]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 13:23:29.930347 25366 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0d29b499fa4c41f1b974782cd0c9c1c2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:23:29.930743 25292 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0d29b499fa4c41f1b974782cd0c9c1c2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:23:29.926939 25367 tablet_service.cc:1511] Processing CreateTablet for tablet a820954e07a94582ac0613342fcffde5 (DEFAULT_TABLE table=test-workload [id=2f3f768873514f53bb58e92ba96f4813]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 13:23:29.933790 25367 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a820954e07a94582ac0613342fcffde5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:23:29.977788 25221 tablet_service.cc:1511] Processing CreateTablet for tablet d0a0e4d32ce943fbb773bb840863647b (DEFAULT_TABLE table=test-workload [id=2f3f768873514f53bb58e92ba96f4813]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 13:23:29.979238 25219 tablet_service.cc:1511] Processing CreateTablet for tablet e9e342ecda4b46619ce12452b2cf7154 (DEFAULT_TABLE table=test-workload [id=2f3f768873514f53bb58e92ba96f4813]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 13:23:29.980458 25220 tablet_service.cc:1511] Processing CreateTablet for tablet b7ff8dde9f5d4c49b9203d902f6b94c4 (DEFAULT_TABLE table=test-workload [id=2f3f768873514f53bb58e92ba96f4813]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 13:23:29.982225 25221 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d0a0e4d32ce943fbb773bb840863647b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:23:29.984511 25222 tablet_service.cc:1511] Processing CreateTablet for tablet 441d780c3945463f92f31170f31e37a8 (DEFAULT_TABLE table=test-workload [id=2f3f768873514f53bb58e92ba96f4813]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 13:23:29.981913 25218 tablet_service.cc:1511] Processing CreateTablet for tablet 0e71b6f5c69b4b35a8ac51c22c7d5f0d (DEFAULT_TABLE table=test-workload [id=2f3f768873514f53bb58e92ba96f4813]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 13:23:29.985882 25220 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b7ff8dde9f5d4c49b9203d902f6b94c4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:23:29.989149 25219 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e9e342ecda4b46619ce12452b2cf7154. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:23:29.991557 25222 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 441d780c3945463f92f31170f31e37a8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:23:29.987934 25217 tablet_service.cc:1511] Processing CreateTablet for tablet 0d29b499fa4c41f1b974782cd0c9c1c2 (DEFAULT_TABLE table=test-workload [id=2f3f768873514f53bb58e92ba96f4813]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 13:23:29.992638 25217 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0d29b499fa4c41f1b974782cd0c9c1c2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:23:29.995523 25218 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0e71b6f5c69b4b35a8ac51c22c7d5f0d. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:23:29.996031 25182 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:23:30.030067 25449 tablet_service.cc:1511] Processing CreateTablet for tablet 579da2d6aabb4ae9a2b1802e1358068b (DEFAULT_TABLE table=test-workload [id=2f3f768873514f53bb58e92ba96f4813]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 13:23:30.032582 25448 tablet_service.cc:1511] Processing CreateTablet for tablet b7a891bfc89e4e259f53da6cb16f4989 (DEFAULT_TABLE table=test-workload [id=2f3f768873514f53bb58e92ba96f4813]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 13:23:30.034933 25447 tablet_service.cc:1511] Processing CreateTablet for tablet 467a6b42ca6548269c12dddde696f7af (DEFAULT_TABLE table=test-workload [id=2f3f768873514f53bb58e92ba96f4813]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 13:23:30.036206 25298 tablet_service.cc:1511] Processing CreateTablet for tablet e15820fa8a7d4e1fb6ad23a64ca22f90 (DEFAULT_TABLE table=test-workload [id=2f3f768873514f53bb58e92ba96f4813]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 13:23:30.037211 25446 tablet_service.cc:1511] Processing CreateTablet for tablet 441d780c3945463f92f31170f31e37a8 (DEFAULT_TABLE table=test-workload [id=2f3f768873514f53bb58e92ba96f4813]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 13:23:30.037845 25298 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e15820fa8a7d4e1fb6ad23a64ca22f90. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:23:30.039491 25445 tablet_service.cc:1511] Processing CreateTablet for tablet d0a0e4d32ce943fbb773bb840863647b (DEFAULT_TABLE table=test-workload [id=2f3f768873514f53bb58e92ba96f4813]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 13:23:30.018939 25374 tablet_service.cc:1511] Processing CreateTablet for tablet 1f15ec62698c4a8cbd84f2b3d516d2ea (DEFAULT_TABLE table=test-workload [id=2f3f768873514f53bb58e92ba96f4813]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 13:23:30.041548 25374 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1f15ec62698c4a8cbd84f2b3d516d2ea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:23:30.041711 25444 tablet_service.cc:1511] Processing CreateTablet for tablet 3fe5a944a342411ea923eb39071ac770 (DEFAULT_TABLE table=test-workload [id=2f3f768873514f53bb58e92ba96f4813]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 13:23:30.044232 25443 tablet_service.cc:1511] Processing CreateTablet for tablet e9e342ecda4b46619ce12452b2cf7154 (DEFAULT_TABLE table=test-workload [id=2f3f768873514f53bb58e92ba96f4813]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 13:23:30.048872 25442 tablet_service.cc:1511] Processing CreateTablet for tablet 0e71b6f5c69b4b35a8ac51c22c7d5f0d (DEFAULT_TABLE table=test-workload [id=2f3f768873514f53bb58e92ba96f4813]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 13:23:30.050673 25441 tablet_service.cc:1511] Processing CreateTablet for tablet a820954e07a94582ac0613342fcffde5 (DEFAULT_TABLE table=test-workload [id=2f3f768873514f53bb58e92ba96f4813]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 13:23:30.055303 25449 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 579da2d6aabb4ae9a2b1802e1358068b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:23:30.032855 25301 tablet_service.cc:1511] Processing CreateTablet for tablet ba8a000f5abe4158965449995af84b18 (DEFAULT_TABLE table=test-workload [id=2f3f768873514f53bb58e92ba96f4813]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 13:23:30.068394 25301 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ba8a000f5abe4158965449995af84b18. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:23:30.068827 25294 tablet_service.cc:1511] Processing CreateTablet for tablet d053b9ae38de4173a0aded4da494bd98 (DEFAULT_TABLE table=test-workload [id=2f3f768873514f53bb58e92ba96f4813]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 13:23:30.070068 25445 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d0a0e4d32ce943fbb773bb840863647b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:23:30.070273 25294 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d053b9ae38de4173a0aded4da494bd98. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:23:30.084173 25295 tablet_service.cc:1511] Processing CreateTablet for tablet 3a3317f4c3514df1b2bb8e44a86be92f (DEFAULT_TABLE table=test-workload [id=2f3f768873514f53bb58e92ba96f4813]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 13:23:30.086722 25372 tablet_service.cc:1511] Processing CreateTablet for tablet ba8a000f5abe4158965449995af84b18 (DEFAULT_TABLE table=test-workload [id=2f3f768873514f53bb58e92ba96f4813]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 13:23:30.088313 25372 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ba8a000f5abe4158965449995af84b18. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:23:30.088639 25440 tablet_service.cc:1511] Processing CreateTablet for tablet 3a3317f4c3514df1b2bb8e44a86be92f (DEFAULT_TABLE table=test-workload [id=2f3f768873514f53bb58e92ba96f4813]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 13:23:30.100474 25503 tablet_bootstrap.cc:492] T 579da2d6aabb4ae9a2b1802e1358068b P 233e7d29aac04505bf4e65f0a39a0c4c: Bootstrap starting.
I20260709 13:23:30.105356 25300 tablet_service.cc:1511] Processing CreateTablet for tablet 5c4ab1978a47481c8576d466bacf49d2 (DEFAULT_TABLE table=test-workload [id=2f3f768873514f53bb58e92ba96f4813]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 13:23:30.110498 25221 tablet_service.cc:1511] Processing CreateTablet for tablet 3a3317f4c3514df1b2bb8e44a86be92f (DEFAULT_TABLE table=test-workload [id=2f3f768873514f53bb58e92ba96f4813]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 13:23:30.112149 25221 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3a3317f4c3514df1b2bb8e44a86be92f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:23:30.115204 25447 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 467a6b42ca6548269c12dddde696f7af. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:23:30.115996 25444 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3fe5a944a342411ea923eb39071ac770. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:23:30.116851 25295 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3a3317f4c3514df1b2bb8e44a86be92f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:23:30.117136 25442 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0e71b6f5c69b4b35a8ac51c22c7d5f0d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:23:30.117023 25222 tablet_service.cc:1511] Processing CreateTablet for tablet e15820fa8a7d4e1fb6ad23a64ca22f90 (DEFAULT_TABLE table=test-workload [id=2f3f768873514f53bb58e92ba96f4813]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 13:23:30.118367 25443 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e9e342ecda4b46619ce12452b2cf7154. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:23:30.118583 25222 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e15820fa8a7d4e1fb6ad23a64ca22f90. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:23:30.119488 25446 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 441d780c3945463f92f31170f31e37a8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:23:30.120675 25448 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b7a891bfc89e4e259f53da6cb16f4989. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:23:30.121352 25441 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a820954e07a94582ac0613342fcffde5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:23:30.122730 25440 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3a3317f4c3514df1b2bb8e44a86be92f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:23:30.122943 25224 tablet_service.cc:1511] Processing CreateTablet for tablet d053b9ae38de4173a0aded4da494bd98 (DEFAULT_TABLE table=test-workload [id=2f3f768873514f53bb58e92ba96f4813]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 13:23:30.132392 25224 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d053b9ae38de4173a0aded4da494bd98. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:23:30.123204 25217 tablet_service.cc:1511] Processing CreateTablet for tablet 1f15ec62698c4a8cbd84f2b3d516d2ea (DEFAULT_TABLE table=test-workload [id=2f3f768873514f53bb58e92ba96f4813]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 13:23:30.140980 25367 tablet_service.cc:1511] Processing CreateTablet for tablet e15820fa8a7d4e1fb6ad23a64ca22f90 (DEFAULT_TABLE table=test-workload [id=2f3f768873514f53bb58e92ba96f4813]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 13:23:30.142753 25217 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1f15ec62698c4a8cbd84f2b3d516d2ea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:23:30.144032 25300 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5c4ab1978a47481c8576d466bacf49d2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:23:30.132942 25505 tablet_bootstrap.cc:492] T b7a891bfc89e4e259f53da6cb16f4989 P 43999d813e564dc688fad116090c95ca: Bootstrap starting.
I20260709 13:23:30.136624 25504 tablet_bootstrap.cc:492] T 28c0b6fc9bd64d60828d705c6a9ab1e4 P 3e07d64466934da4bb3c02262b1c2e38: Bootstrap starting.
I20260709 13:23:30.137681 25371 tablet_service.cc:1511] Processing CreateTablet for tablet 280d65245f4f4358b54bef93e18aca92 (DEFAULT_TABLE table=test-workload [id=2f3f768873514f53bb58e92ba96f4813]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 13:23:30.175304 25371 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 280d65245f4f4358b54bef93e18aca92. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:23:30.179057 25367 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e15820fa8a7d4e1fb6ad23a64ca22f90. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:23:30.209728 25120 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.CreateTablet -> {remote=127.14.177.129:38155, user_credentials={real_user=slave}} blocked reactor thread for 92450us
I20260709 13:23:30.215776 25261 heartbeater.cc:499] Master 127.14.177.190:45841 was elected leader, sending a full tablet report...
I20260709 13:23:30.227588 25368 tablet_service.cc:1511] Processing CreateTablet for tablet 5c4ab1978a47481c8576d466bacf49d2 (DEFAULT_TABLE table=test-workload [id=2f3f768873514f53bb58e92ba96f4813]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 13:23:30.229265 25368 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5c4ab1978a47481c8576d466bacf49d2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:23:30.241815 25505 tablet_bootstrap.cc:654] T b7a891bfc89e4e259f53da6cb16f4989 P 43999d813e564dc688fad116090c95ca: Neither blocks nor log segments found. Creating new log.
I20260709 13:23:30.218735 25292 tablet_service.cc:1511] Processing CreateTablet for tablet beacc815ef954fb692e90544bfa93f05 (DEFAULT_TABLE table=test-workload [id=2f3f768873514f53bb58e92ba96f4813]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 13:23:30.255726 25292 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet beacc815ef954fb692e90544bfa93f05. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:23:30.261436 25503 tablet_bootstrap.cc:654] T 579da2d6aabb4ae9a2b1802e1358068b P 233e7d29aac04505bf4e65f0a39a0c4c: Neither blocks nor log segments found. Creating new log.
W20260709 13:23:30.271663 25118 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.CreateTablet -> {remote=127.14.177.131:41539, user_credentials={real_user=slave}} blocked reactor thread for 134443us
I20260709 13:23:30.284310 25504 tablet_bootstrap.cc:654] T 28c0b6fc9bd64d60828d705c6a9ab1e4 P 3e07d64466934da4bb3c02262b1c2e38: Neither blocks nor log segments found. Creating new log.
I20260709 13:23:30.300503 25225 tablet_service.cc:1511] Processing CreateTablet for tablet beacc815ef954fb692e90544bfa93f05 (DEFAULT_TABLE table=test-workload [id=2f3f768873514f53bb58e92ba96f4813]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 13:23:30.301575 25218 tablet_service.cc:1511] Processing CreateTablet for tablet 280d65245f4f4358b54bef93e18aca92 (DEFAULT_TABLE table=test-workload [id=2f3f768873514f53bb58e92ba96f4813]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 13:23:30.340137 25336 heartbeater.cc:499] Master 127.14.177.190:45841 was elected leader, sending a full tablet report...
I20260709 13:23:30.339947 25220 tablet_service.cc:1511] Processing CreateTablet for tablet 9e5d70664720424d951f269d9d7f4346 (DEFAULT_TABLE table=test-workload [id=2f3f768873514f53bb58e92ba96f4813]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 13:23:30.311476 25298 tablet_service.cc:1511] Processing CreateTablet for tablet 9e5d70664720424d951f269d9d7f4346 (DEFAULT_TABLE table=test-workload [id=2f3f768873514f53bb58e92ba96f4813]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 13:23:30.342447 25298 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9e5d70664720424d951f269d9d7f4346. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:23:30.365461 25218 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 280d65245f4f4358b54bef93e18aca92. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:23:30.370754 25225 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet beacc815ef954fb692e90544bfa93f05. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:23:30.388330 25220 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9e5d70664720424d951f269d9d7f4346. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:23:30.449306 25410 heartbeater.cc:499] Master 127.14.177.190:45841 was elected leader, sending a full tablet report...
I20260709 13:23:30.457248 25505 tablet_bootstrap.cc:492] T b7a891bfc89e4e259f53da6cb16f4989 P 43999d813e564dc688fad116090c95ca: No bootstrap required, opened a new log
I20260709 13:23:30.457888 25505 ts_tablet_manager.cc:1403] T b7a891bfc89e4e259f53da6cb16f4989 P 43999d813e564dc688fad116090c95ca: Time spent bootstrapping tablet: real 0.325s	user 0.007s	sys 0.051s
W20260709 13:23:30.461608 25120 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.CreateTablet -> {remote=127.14.177.129:38155, user_credentials={real_user=slave}} blocked reactor thread for 149566us
I20260709 13:23:30.469322 25366 tablet_service.cc:1511] Processing CreateTablet for tablet beacc815ef954fb692e90544bfa93f05 (DEFAULT_TABLE table=test-workload [id=2f3f768873514f53bb58e92ba96f4813]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 13:23:30.470767 25366 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet beacc815ef954fb692e90544bfa93f05. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:23:30.472977 25445 tablet_service.cc:1511] Processing CreateTablet for tablet 1f15ec62698c4a8cbd84f2b3d516d2ea (DEFAULT_TABLE table=test-workload [id=2f3f768873514f53bb58e92ba96f4813]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
W20260709 13:23:30.475553 25118 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.CreateTablet -> {remote=127.14.177.131:41539, user_credentials={real_user=slave}} blocked reactor thread for 86815.1us
I20260709 13:23:30.479302 25505 raft_consensus.cc:359] T b7a891bfc89e4e259f53da6cb16f4989 P 43999d813e564dc688fad116090c95ca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } }
I20260709 13:23:30.480271 25505 raft_consensus.cc:385] T b7a891bfc89e4e259f53da6cb16f4989 P 43999d813e564dc688fad116090c95ca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:23:30.480615 25505 raft_consensus.cc:740] T b7a891bfc89e4e259f53da6cb16f4989 P 43999d813e564dc688fad116090c95ca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43999d813e564dc688fad116090c95ca, State: Initialized, Role: FOLLOWER
I20260709 13:23:30.484614 25444 tablet_service.cc:1511] Processing CreateTablet for tablet d053b9ae38de4173a0aded4da494bd98 (DEFAULT_TABLE table=test-workload [id=2f3f768873514f53bb58e92ba96f4813]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 13:23:30.487545 25505 consensus_queue.cc:260] T b7a891bfc89e4e259f53da6cb16f4989 P 43999d813e564dc688fad116090c95ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } }
I20260709 13:23:30.488626 25508 tablet_bootstrap.cc:492] T d0a0e4d32ce943fbb773bb840863647b P 9868dce521294d0b816bba7648f4ae5a: Bootstrap starting.
I20260709 13:23:30.489480 25448 tablet_service.cc:1511] Processing CreateTablet for tablet 280d65245f4f4358b54bef93e18aca92 (DEFAULT_TABLE table=test-workload [id=2f3f768873514f53bb58e92ba96f4813]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 13:23:30.483013 25446 tablet_service.cc:1511] Processing CreateTablet for tablet ba8a000f5abe4158965449995af84b18 (DEFAULT_TABLE table=test-workload [id=2f3f768873514f53bb58e92ba96f4813]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 13:23:30.475544 25445 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1f15ec62698c4a8cbd84f2b3d516d2ea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:23:30.509379 25503 tablet_bootstrap.cc:492] T 579da2d6aabb4ae9a2b1802e1358068b P 233e7d29aac04505bf4e65f0a39a0c4c: No bootstrap required, opened a new log
I20260709 13:23:30.510407 25503 ts_tablet_manager.cc:1403] T 579da2d6aabb4ae9a2b1802e1358068b P 233e7d29aac04505bf4e65f0a39a0c4c: Time spent bootstrapping tablet: real 0.414s	user 0.018s	sys 0.010s
I20260709 13:23:30.560599 25484 heartbeater.cc:499] Master 127.14.177.190:45841 was elected leader, sending a full tablet report...
I20260709 13:23:30.586429 25222 tablet_service.cc:1511] Processing CreateTablet for tablet 9ffa42c3b2ff48bc87310e9ab5d218ce (DEFAULT_TABLE table=test-workload [id=2f3f768873514f53bb58e92ba96f4813]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 13:23:30.587944 25222 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9ffa42c3b2ff48bc87310e9ab5d218ce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:23:30.521647 25504 tablet_bootstrap.cc:492] T 28c0b6fc9bd64d60828d705c6a9ab1e4 P 3e07d64466934da4bb3c02262b1c2e38: No bootstrap required, opened a new log
I20260709 13:23:30.514434 25446 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ba8a000f5abe4158965449995af84b18. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:23:30.601068 25444 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d053b9ae38de4173a0aded4da494bd98. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:23:30.602972 25293 tablet_service.cc:1511] Processing CreateTablet for tablet aad1f7937f854cc1a726223269da5d5a (DEFAULT_TABLE table=test-workload [id=2f3f768873514f53bb58e92ba96f4813]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 13:23:30.607414 25503 raft_consensus.cc:359] T 579da2d6aabb4ae9a2b1802e1358068b P 233e7d29aac04505bf4e65f0a39a0c4c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } }
I20260709 13:23:30.608418 25503 raft_consensus.cc:385] T 579da2d6aabb4ae9a2b1802e1358068b P 233e7d29aac04505bf4e65f0a39a0c4c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:23:30.610702 25503 raft_consensus.cc:740] T 579da2d6aabb4ae9a2b1802e1358068b P 233e7d29aac04505bf4e65f0a39a0c4c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 233e7d29aac04505bf4e65f0a39a0c4c, State: Initialized, Role: FOLLOWER
I20260709 13:23:30.599061 25504 ts_tablet_manager.cc:1403] T 28c0b6fc9bd64d60828d705c6a9ab1e4 P 3e07d64466934da4bb3c02262b1c2e38: Time spent bootstrapping tablet: real 0.463s	user 0.021s	sys 0.032s
I20260709 13:23:30.523320 25371 tablet_service.cc:1511] Processing CreateTablet for tablet 9ffa42c3b2ff48bc87310e9ab5d218ce (DEFAULT_TABLE table=test-workload [id=2f3f768873514f53bb58e92ba96f4813]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 13:23:30.619403 25371 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9ffa42c3b2ff48bc87310e9ab5d218ce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:23:30.609987 25440 tablet_service.cc:1511] Processing CreateTablet for tablet 9e5d70664720424d951f269d9d7f4346 (DEFAULT_TABLE table=test-workload [id=2f3f768873514f53bb58e92ba96f4813]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 13:23:30.586912 25443 tablet_service.cc:1511] Processing CreateTablet for tablet 5c4ab1978a47481c8576d466bacf49d2 (DEFAULT_TABLE table=test-workload [id=2f3f768873514f53bb58e92ba96f4813]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 13:23:30.624450 25510 raft_consensus.cc:493] T b7a891bfc89e4e259f53da6cb16f4989 P 43999d813e564dc688fad116090c95ca [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:23:30.609146 25448 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 280d65245f4f4358b54bef93e18aca92. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:23:30.625062 25510 raft_consensus.cc:515] T b7a891bfc89e4e259f53da6cb16f4989 P 43999d813e564dc688fad116090c95ca [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } }
I20260709 13:23:30.576198 25505 ts_tablet_manager.cc:1434] T b7a891bfc89e4e259f53da6cb16f4989 P 43999d813e564dc688fad116090c95ca: Time spent starting tablet: real 0.118s	user 0.023s	sys 0.038s
I20260709 13:23:30.628239 25505 tablet_bootstrap.cc:492] T d0a0e4d32ce943fbb773bb840863647b P 43999d813e564dc688fad116090c95ca: Bootstrap starting.
I20260709 13:23:30.635396 25443 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5c4ab1978a47481c8576d466bacf49d2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:23:30.614253 25503 consensus_queue.cc:260] T 579da2d6aabb4ae9a2b1802e1358068b P 233e7d29aac04505bf4e65f0a39a0c4c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } }
I20260709 13:23:30.642135 25440 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9e5d70664720424d951f269d9d7f4346. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:23:30.653600 25293 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet aad1f7937f854cc1a726223269da5d5a. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:23:30.677158 25118 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.CreateTablet -> {remote=127.14.177.132:33683, user_credentials={real_user=slave}} blocked reactor thread for 67877.1us
I20260709 13:23:30.684633 25503 ts_tablet_manager.cc:1434] T 579da2d6aabb4ae9a2b1802e1358068b P 233e7d29aac04505bf4e65f0a39a0c4c: Time spent starting tablet: real 0.171s	user 0.004s	sys 0.015s
I20260709 13:23:30.686229 25503 tablet_bootstrap.cc:492] T b7ff8dde9f5d4c49b9203d902f6b94c4 P 233e7d29aac04505bf4e65f0a39a0c4c: Bootstrap starting.
I20260709 13:23:30.690161 25510 leader_election.cc:290] T b7a891bfc89e4e259f53da6cb16f4989 P 43999d813e564dc688fad116090c95ca [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3e07d64466934da4bb3c02262b1c2e38 (127.14.177.131:41539), 9868dce521294d0b816bba7648f4ae5a (127.14.177.132:33683)
W20260709 13:23:30.692397 25262 tablet.cc:2406] T b7a891bfc89e4e259f53da6cb16f4989 P 43999d813e564dc688fad116090c95ca: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260709 13:23:30.697572 25367 tablet_service.cc:1511] Processing CreateTablet for tablet aad1f7937f854cc1a726223269da5d5a (DEFAULT_TABLE table=test-workload [id=2f3f768873514f53bb58e92ba96f4813]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 13:23:30.720412 25367 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet aad1f7937f854cc1a726223269da5d5a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:23:30.704314 25504 raft_consensus.cc:359] T 28c0b6fc9bd64d60828d705c6a9ab1e4 P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } }
I20260709 13:23:30.708319 25442 tablet_service.cc:1511] Processing CreateTablet for tablet aad1f7937f854cc1a726223269da5d5a (DEFAULT_TABLE table=test-workload [id=2f3f768873514f53bb58e92ba96f4813]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 13:23:30.709753 25441 tablet_service.cc:1511] Processing CreateTablet for tablet 9ffa42c3b2ff48bc87310e9ab5d218ce (DEFAULT_TABLE table=test-workload [id=2f3f768873514f53bb58e92ba96f4813]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 13:23:30.717907 25505 tablet_bootstrap.cc:654] T d0a0e4d32ce943fbb773bb840863647b P 43999d813e564dc688fad116090c95ca: Neither blocks nor log segments found. Creating new log.
I20260709 13:23:30.748672 25504 raft_consensus.cc:385] T 28c0b6fc9bd64d60828d705c6a9ab1e4 P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:23:30.728605 25220 tablet_service.cc:1511] Processing CreateTablet for tablet ce98a3f3b3eb4c63bcaa355ec7cd64a4 (DEFAULT_TABLE table=test-workload [id=2f3f768873514f53bb58e92ba96f4813]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 13:23:30.749140 25504 raft_consensus.cc:740] T 28c0b6fc9bd64d60828d705c6a9ab1e4 P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e07d64466934da4bb3c02262b1c2e38, State: Initialized, Role: FOLLOWER
I20260709 13:23:30.750077 25220 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ce98a3f3b3eb4c63bcaa355ec7cd64a4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:23:30.746526 25508 tablet_bootstrap.cc:654] T d0a0e4d32ce943fbb773bb840863647b P 9868dce521294d0b816bba7648f4ae5a: Neither blocks nor log segments found. Creating new log.
I20260709 13:23:30.750168 25504 consensus_queue.cc:260] T 28c0b6fc9bd64d60828d705c6a9ab1e4 P 3e07d64466934da4bb3c02262b1c2e38 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } }
I20260709 13:23:30.731042 25372 tablet_service.cc:1511] Processing CreateTablet for tablet ce98a3f3b3eb4c63bcaa355ec7cd64a4 (DEFAULT_TABLE table=test-workload [id=2f3f768873514f53bb58e92ba96f4813]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 13:23:30.752873 25372 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ce98a3f3b3eb4c63bcaa355ec7cd64a4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:23:30.754920 25442 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet aad1f7937f854cc1a726223269da5d5a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:23:30.756448 25301 tablet_service.cc:1511] Processing CreateTablet for tablet 51a11db644124b63bdf5f457c967473d (DEFAULT_TABLE table=test-workload [id=2f3f768873514f53bb58e92ba96f4813]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 13:23:30.761839 25449 tablet_service.cc:1511] Processing CreateTablet for tablet 51a11db644124b63bdf5f457c967473d (DEFAULT_TABLE table=test-workload [id=2f3f768873514f53bb58e92ba96f4813]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 13:23:30.763729 25449 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 51a11db644124b63bdf5f457c967473d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:23:30.768270 25441 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9ffa42c3b2ff48bc87310e9ab5d218ce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:23:30.763096 25503 tablet_bootstrap.cc:654] T b7ff8dde9f5d4c49b9203d902f6b94c4 P 233e7d29aac04505bf4e65f0a39a0c4c: Neither blocks nor log segments found. Creating new log.
I20260709 13:23:30.729667 25298 tablet_service.cc:1511] Processing CreateTablet for tablet ce98a3f3b3eb4c63bcaa355ec7cd64a4 (DEFAULT_TABLE table=test-workload [id=2f3f768873514f53bb58e92ba96f4813]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 13:23:30.777597 25301 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 51a11db644124b63bdf5f457c967473d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:23:30.776177 25225 tablet_service.cc:1511] Processing CreateTablet for tablet 51a11db644124b63bdf5f457c967473d (DEFAULT_TABLE table=test-workload [id=2f3f768873514f53bb58e92ba96f4813]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 13:23:30.780958 25225 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 51a11db644124b63bdf5f457c967473d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:23:30.782253 25298 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ce98a3f3b3eb4c63bcaa355ec7cd64a4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:23:30.794422 25370 tablet_service.cc:1511] Processing CreateTablet for tablet c902bbc7ec0848ba82a04870eab1e7f5 (DEFAULT_TABLE table=test-workload [id=2f3f768873514f53bb58e92ba96f4813]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 13:23:30.796746 25385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7a891bfc89e4e259f53da6cb16f4989" candidate_uuid: "43999d813e564dc688fad116090c95ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e07d64466934da4bb3c02262b1c2e38" is_pre_election: true
W20260709 13:23:30.799993 25199 leader_election.cc:343] T b7a891bfc89e4e259f53da6cb16f4989 P 43999d813e564dc688fad116090c95ca [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3e07d64466934da4bb3c02262b1c2e38 (127.14.177.131:41539): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:23:30.800515 25370 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c902bbc7ec0848ba82a04870eab1e7f5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:23:30.833199 25504 ts_tablet_manager.cc:1434] T 28c0b6fc9bd64d60828d705c6a9ab1e4 P 3e07d64466934da4bb3c02262b1c2e38: Time spent starting tablet: real 0.218s	user 0.014s	sys 0.014s
I20260709 13:23:30.833988 25222 tablet_service.cc:1511] Processing CreateTablet for tablet c902bbc7ec0848ba82a04870eab1e7f5 (DEFAULT_TABLE table=test-workload [id=2f3f768873514f53bb58e92ba96f4813]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 13:23:30.840698 25222 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c902bbc7ec0848ba82a04870eab1e7f5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:23:30.850140 25448 tablet_service.cc:1511] Processing CreateTablet for tablet c902bbc7ec0848ba82a04870eab1e7f5 (DEFAULT_TABLE table=test-workload [id=2f3f768873514f53bb58e92ba96f4813]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 13:23:30.851800 25504 tablet_bootstrap.cc:492] T b7a891bfc89e4e259f53da6cb16f4989 P 3e07d64466934da4bb3c02262b1c2e38: Bootstrap starting.
I20260709 13:23:30.875671 25448 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c902bbc7ec0848ba82a04870eab1e7f5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:23:30.885212 25367 tablet_service.cc:1511] Processing CreateTablet for tablet ad32097cd4dc48b9a69595f1651abd07 (DEFAULT_TABLE table=test-workload [id=2f3f768873514f53bb58e92ba96f4813]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 13:23:30.887501 25367 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ad32097cd4dc48b9a69595f1651abd07. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:23:30.904877 25459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7a891bfc89e4e259f53da6cb16f4989" candidate_uuid: "43999d813e564dc688fad116090c95ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9868dce521294d0b816bba7648f4ae5a" is_pre_election: true
W20260709 13:23:30.907123 25199 leader_election.cc:343] T b7a891bfc89e4e259f53da6cb16f4989 P 43999d813e564dc688fad116090c95ca [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9868dce521294d0b816bba7648f4ae5a (127.14.177.132:33683): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:23:30.907617 25199 leader_election.cc:304] T b7a891bfc89e4e259f53da6cb16f4989 P 43999d813e564dc688fad116090c95ca [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 43999d813e564dc688fad116090c95ca; no voters: 3e07d64466934da4bb3c02262b1c2e38, 9868dce521294d0b816bba7648f4ae5a
I20260709 13:23:30.907562 25218 tablet_service.cc:1511] Processing CreateTablet for tablet 0cb89d70b4ce4aa8992fb68480bec8a2 (DEFAULT_TABLE table=test-workload [id=2f3f768873514f53bb58e92ba96f4813]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 13:23:30.909046 25510 raft_consensus.cc:2749] T b7a891bfc89e4e259f53da6cb16f4989 P 43999d813e564dc688fad116090c95ca [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:23:30.909122 25218 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0cb89d70b4ce4aa8992fb68480bec8a2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:23:30.912420 25294 tablet_service.cc:1511] Processing CreateTablet for tablet 0cb89d70b4ce4aa8992fb68480bec8a2 (DEFAULT_TABLE table=test-workload [id=2f3f768873514f53bb58e92ba96f4813]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 13:23:30.913877 25294 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0cb89d70b4ce4aa8992fb68480bec8a2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:23:30.915635 25512 raft_consensus.cc:493] T 579da2d6aabb4ae9a2b1802e1358068b P 233e7d29aac04505bf4e65f0a39a0c4c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:23:30.916209 25512 raft_consensus.cc:515] T 579da2d6aabb4ae9a2b1802e1358068b P 233e7d29aac04505bf4e65f0a39a0c4c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } }
I20260709 13:23:30.909119 25443 tablet_service.cc:1511] Processing CreateTablet for tablet ad32097cd4dc48b9a69595f1651abd07 (DEFAULT_TABLE table=test-workload [id=2f3f768873514f53bb58e92ba96f4813]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 13:23:30.918865 25443 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ad32097cd4dc48b9a69595f1651abd07. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:23:30.919919 25505 tablet_bootstrap.cc:492] T d0a0e4d32ce943fbb773bb840863647b P 43999d813e564dc688fad116090c95ca: No bootstrap required, opened a new log
I20260709 13:23:30.920608 25505 ts_tablet_manager.cc:1403] T d0a0e4d32ce943fbb773bb840863647b P 43999d813e564dc688fad116090c95ca: Time spent bootstrapping tablet: real 0.293s	user 0.010s	sys 0.021s
I20260709 13:23:30.922137 25508 tablet_bootstrap.cc:492] T d0a0e4d32ce943fbb773bb840863647b P 9868dce521294d0b816bba7648f4ae5a: No bootstrap required, opened a new log
I20260709 13:23:30.929689 25508 ts_tablet_manager.cc:1403] T d0a0e4d32ce943fbb773bb840863647b P 9868dce521294d0b816bba7648f4ae5a: Time spent bootstrapping tablet: real 0.468s	user 0.041s	sys 0.076s
I20260709 13:23:30.927516 25512 leader_election.cc:290] T 579da2d6aabb4ae9a2b1802e1358068b P 233e7d29aac04505bf4e65f0a39a0c4c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9868dce521294d0b816bba7648f4ae5a (127.14.177.132:33683), 43999d813e564dc688fad116090c95ca (127.14.177.129:38155)
I20260709 13:23:30.934181 25368 tablet_service.cc:1511] Processing CreateTablet for tablet 0cb89d70b4ce4aa8992fb68480bec8a2 (DEFAULT_TABLE table=test-workload [id=2f3f768873514f53bb58e92ba96f4813]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 13:23:30.944161 25446 tablet_service.cc:1511] Processing CreateTablet for tablet 0c4f6b7401db47d59a4f34ed0e8b5350 (DEFAULT_TABLE table=test-workload [id=2f3f768873514f53bb58e92ba96f4813]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 13:23:30.945521 25446 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0c4f6b7401db47d59a4f34ed0e8b5350. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:23:30.947893 25503 tablet_bootstrap.cc:492] T b7ff8dde9f5d4c49b9203d902f6b94c4 P 233e7d29aac04505bf4e65f0a39a0c4c: No bootstrap required, opened a new log
I20260709 13:23:30.948467 25503 ts_tablet_manager.cc:1403] T b7ff8dde9f5d4c49b9203d902f6b94c4 P 233e7d29aac04505bf4e65f0a39a0c4c: Time spent bootstrapping tablet: real 0.263s	user 0.047s	sys 0.021s
I20260709 13:23:30.950078 25508 raft_consensus.cc:359] T d0a0e4d32ce943fbb773bb840863647b P 9868dce521294d0b816bba7648f4ae5a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } }
I20260709 13:23:30.950897 25508 raft_consensus.cc:385] T d0a0e4d32ce943fbb773bb840863647b P 9868dce521294d0b816bba7648f4ae5a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:23:30.951239 25508 raft_consensus.cc:740] T d0a0e4d32ce943fbb773bb840863647b P 9868dce521294d0b816bba7648f4ae5a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9868dce521294d0b816bba7648f4ae5a, State: Initialized, Role: FOLLOWER
I20260709 13:23:30.952082 25508 consensus_queue.cc:260] T d0a0e4d32ce943fbb773bb840863647b P 9868dce521294d0b816bba7648f4ae5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } }
I20260709 13:23:30.953423 25504 tablet_bootstrap.cc:654] T b7a891bfc89e4e259f53da6cb16f4989 P 3e07d64466934da4bb3c02262b1c2e38: Neither blocks nor log segments found. Creating new log.
I20260709 13:23:30.953796 25503 raft_consensus.cc:359] T b7ff8dde9f5d4c49b9203d902f6b94c4 P 233e7d29aac04505bf4e65f0a39a0c4c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } }
I20260709 13:23:30.959867 25503 raft_consensus.cc:385] T b7ff8dde9f5d4c49b9203d902f6b94c4 P 233e7d29aac04505bf4e65f0a39a0c4c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:23:30.967955 25368 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0cb89d70b4ce4aa8992fb68480bec8a2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:23:30.965711 25503 raft_consensus.cc:740] T b7ff8dde9f5d4c49b9203d902f6b94c4 P 233e7d29aac04505bf4e65f0a39a0c4c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 233e7d29aac04505bf4e65f0a39a0c4c, State: Initialized, Role: FOLLOWER
I20260709 13:23:30.967213 25293 tablet_service.cc:1511] Processing CreateTablet for tablet ad32097cd4dc48b9a69595f1651abd07 (DEFAULT_TABLE table=test-workload [id=2f3f768873514f53bb58e92ba96f4813]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 13:23:30.973001 25505 raft_consensus.cc:359] T d0a0e4d32ce943fbb773bb840863647b P 43999d813e564dc688fad116090c95ca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } }
I20260709 13:23:30.980590 25505 raft_consensus.cc:385] T d0a0e4d32ce943fbb773bb840863647b P 43999d813e564dc688fad116090c95ca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:23:30.981630 25505 raft_consensus.cc:740] T d0a0e4d32ce943fbb773bb840863647b P 43999d813e564dc688fad116090c95ca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43999d813e564dc688fad116090c95ca, State: Initialized, Role: FOLLOWER
I20260709 13:23:30.973999 25225 tablet_service.cc:1511] Processing CreateTablet for tablet 0c4f6b7401db47d59a4f34ed0e8b5350 (DEFAULT_TABLE table=test-workload [id=2f3f768873514f53bb58e92ba96f4813]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 13:23:30.984162 25503 consensus_queue.cc:260] T b7ff8dde9f5d4c49b9203d902f6b94c4 P 233e7d29aac04505bf4e65f0a39a0c4c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } }
I20260709 13:23:30.985685 25301 tablet_service.cc:1511] Processing CreateTablet for tablet 0c4f6b7401db47d59a4f34ed0e8b5350 (DEFAULT_TABLE table=test-workload [id=2f3f768873514f53bb58e92ba96f4813]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 13:23:30.986526 25503 ts_tablet_manager.cc:1434] T b7ff8dde9f5d4c49b9203d902f6b94c4 P 233e7d29aac04505bf4e65f0a39a0c4c: Time spent starting tablet: real 0.038s	user 0.005s	sys 0.003s
I20260709 13:23:30.988794 25225 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0c4f6b7401db47d59a4f34ed0e8b5350. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:23:30.980777 25293 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ad32097cd4dc48b9a69595f1651abd07. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:23:30.992079 25503 tablet_bootstrap.cc:492] T b2f14aed05e24a98ad83535029b795dc P 233e7d29aac04505bf4e65f0a39a0c4c: Bootstrap starting.
I20260709 13:23:30.997440 25508 ts_tablet_manager.cc:1434] T d0a0e4d32ce943fbb773bb840863647b P 9868dce521294d0b816bba7648f4ae5a: Time spent starting tablet: real 0.060s	user 0.007s	sys 0.002s
W20260709 13:23:31.000171 25182 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:23:31.002671 25508 tablet_bootstrap.cc:492] T 3fe5a944a342411ea923eb39071ac770 P 9868dce521294d0b816bba7648f4ae5a: Bootstrap starting.
I20260709 13:23:31.003703 25505 consensus_queue.cc:260] T d0a0e4d32ce943fbb773bb840863647b P 43999d813e564dc688fad116090c95ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } }
I20260709 13:23:31.006371 25505 ts_tablet_manager.cc:1434] T d0a0e4d32ce943fbb773bb840863647b P 43999d813e564dc688fad116090c95ca: Time spent starting tablet: real 0.085s	user 0.001s	sys 0.008s
I20260709 13:23:31.010200 25301 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0c4f6b7401db47d59a4f34ed0e8b5350. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:23:31.010392 25442 tablet_service.cc:1511] Processing CreateTablet for tablet 1550ed2d2a764c08adffae679599ba41 (DEFAULT_TABLE table=test-workload [id=2f3f768873514f53bb58e92ba96f4813]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 13:23:31.011838 25442 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1550ed2d2a764c08adffae679599ba41. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:23:31.017618 25459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "579da2d6aabb4ae9a2b1802e1358068b" candidate_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9868dce521294d0b816bba7648f4ae5a" is_pre_election: true
I20260709 13:23:31.020237 25367 tablet_service.cc:1511] Processing CreateTablet for tablet 1550ed2d2a764c08adffae679599ba41 (DEFAULT_TABLE table=test-workload [id=2f3f768873514f53bb58e92ba96f4813]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 13:23:31.021600 25367 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1550ed2d2a764c08adffae679599ba41. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:23:31.021427 25220 tablet_service.cc:1511] Processing CreateTablet for tablet 1550ed2d2a764c08adffae679599ba41 (DEFAULT_TABLE table=test-workload [id=2f3f768873514f53bb58e92ba96f4813]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 13:23:31.022735 25220 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1550ed2d2a764c08adffae679599ba41. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:23:31.025038 25274 leader_election.cc:343] T 579da2d6aabb4ae9a2b1802e1358068b P 233e7d29aac04505bf4e65f0a39a0c4c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9868dce521294d0b816bba7648f4ae5a (127.14.177.132:33683): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:23:31.032745 25371 tablet_service.cc:1511] Processing CreateTablet for tablet 89a26dd3ac414f6081b98fa32c2f346f (DEFAULT_TABLE table=test-workload [id=2f3f768873514f53bb58e92ba96f4813]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 13:23:31.034102 25371 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 89a26dd3ac414f6081b98fa32c2f346f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:23:31.019022 25505 tablet_bootstrap.cc:492] T 0d29b499fa4c41f1b974782cd0c9c1c2 P 43999d813e564dc688fad116090c95ca: Bootstrap starting.
I20260709 13:23:31.045477 25508 tablet_bootstrap.cc:654] T 3fe5a944a342411ea923eb39071ac770 P 9868dce521294d0b816bba7648f4ae5a: Neither blocks nor log segments found. Creating new log.
I20260709 13:23:31.056111 25298 tablet_service.cc:1511] Processing CreateTablet for tablet 13bb3a23315a41c7a40e10b954b15c8e (DEFAULT_TABLE table=test-workload [id=2f3f768873514f53bb58e92ba96f4813]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 13:23:31.057507 25298 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 13bb3a23315a41c7a40e10b954b15c8e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:23:31.057871 25504 tablet_bootstrap.cc:492] T b7a891bfc89e4e259f53da6cb16f4989 P 3e07d64466934da4bb3c02262b1c2e38: No bootstrap required, opened a new log
I20260709 13:23:31.058420 25504 ts_tablet_manager.cc:1403] T b7a891bfc89e4e259f53da6cb16f4989 P 3e07d64466934da4bb3c02262b1c2e38: Time spent bootstrapping tablet: real 0.207s	user 0.018s	sys 0.013s
I20260709 13:23:31.061569 25504 raft_consensus.cc:359] T b7a891bfc89e4e259f53da6cb16f4989 P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } }
I20260709 13:23:31.062330 25504 raft_consensus.cc:385] T b7a891bfc89e4e259f53da6cb16f4989 P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:23:31.062640 25504 raft_consensus.cc:740] T b7a891bfc89e4e259f53da6cb16f4989 P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e07d64466934da4bb3c02262b1c2e38, State: Initialized, Role: FOLLOWER
I20260709 13:23:31.066314 25370 tablet_service.cc:1511] Processing CreateTablet for tablet 13bb3a23315a41c7a40e10b954b15c8e (DEFAULT_TABLE table=test-workload [id=2f3f768873514f53bb58e92ba96f4813]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 13:23:31.067791 25370 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 13bb3a23315a41c7a40e10b954b15c8e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:23:31.045934 25294 tablet_service.cc:1511] Processing CreateTablet for tablet 89a26dd3ac414f6081b98fa32c2f346f (DEFAULT_TABLE table=test-workload [id=2f3f768873514f53bb58e92ba96f4813]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 13:23:31.049569 25505 tablet_bootstrap.cc:654] T 0d29b499fa4c41f1b974782cd0c9c1c2 P 43999d813e564dc688fad116090c95ca: Neither blocks nor log segments found. Creating new log.
I20260709 13:23:31.049728 25518 raft_consensus.cc:493] T 28c0b6fc9bd64d60828d705c6a9ab1e4 P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:23:31.051044 25446 tablet_service.cc:1511] Processing CreateTablet for tablet 89a26dd3ac414f6081b98fa32c2f346f (DEFAULT_TABLE table=test-workload [id=2f3f768873514f53bb58e92ba96f4813]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 13:23:31.055388 25503 tablet_bootstrap.cc:654] T b2f14aed05e24a98ad83535029b795dc P 233e7d29aac04505bf4e65f0a39a0c4c: Neither blocks nor log segments found. Creating new log.
I20260709 13:23:31.074898 25518 raft_consensus.cc:515] T 28c0b6fc9bd64d60828d705c6a9ab1e4 P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } }
I20260709 13:23:31.083812 25518 leader_election.cc:290] T 28c0b6fc9bd64d60828d705c6a9ab1e4 P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 43999d813e564dc688fad116090c95ca (127.14.177.129:38155), 233e7d29aac04505bf4e65f0a39a0c4c (127.14.177.130:42957)
I20260709 13:23:31.096915 25236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "579da2d6aabb4ae9a2b1802e1358068b" candidate_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43999d813e564dc688fad116090c95ca" is_pre_election: true
I20260709 13:23:31.100009 25225 tablet_service.cc:1511] Processing CreateTablet for tablet a5df750b84e0487fa530161079deb6c2 (DEFAULT_TABLE table=test-workload [id=2f3f768873514f53bb58e92ba96f4813]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 13:23:31.101255 25294 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 89a26dd3ac414f6081b98fa32c2f346f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:23:31.101958 25225 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a5df750b84e0487fa530161079deb6c2. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:23:31.104230 25276 leader_election.cc:343] T 579da2d6aabb4ae9a2b1802e1358068b P 233e7d29aac04505bf4e65f0a39a0c4c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 43999d813e564dc688fad116090c95ca (127.14.177.129:38155): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:23:31.105057 25276 leader_election.cc:304] T 579da2d6aabb4ae9a2b1802e1358068b P 233e7d29aac04505bf4e65f0a39a0c4c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 233e7d29aac04505bf4e65f0a39a0c4c; no voters: 43999d813e564dc688fad116090c95ca, 9868dce521294d0b816bba7648f4ae5a
I20260709 13:23:31.106143 25512 raft_consensus.cc:2749] T 579da2d6aabb4ae9a2b1802e1358068b P 233e7d29aac04505bf4e65f0a39a0c4c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:23:31.090035 25504 consensus_queue.cc:260] T b7a891bfc89e4e259f53da6cb16f4989 P 3e07d64466934da4bb3c02262b1c2e38 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } }
I20260709 13:23:31.108098 25300 tablet_service.cc:1511] Processing CreateTablet for tablet a5df750b84e0487fa530161079deb6c2 (DEFAULT_TABLE table=test-workload [id=2f3f768873514f53bb58e92ba96f4813]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 13:23:31.109557 25300 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a5df750b84e0487fa530161079deb6c2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:23:31.104583 25446 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 89a26dd3ac414f6081b98fa32c2f346f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:23:31.117378 25504 ts_tablet_manager.cc:1434] T b7a891bfc89e4e259f53da6cb16f4989 P 3e07d64466934da4bb3c02262b1c2e38: Time spent starting tablet: real 0.059s	user 0.000s	sys 0.008s
I20260709 13:23:31.104223 25222 tablet_service.cc:1511] Processing CreateTablet for tablet 13bb3a23315a41c7a40e10b954b15c8e (DEFAULT_TABLE table=test-workload [id=2f3f768873514f53bb58e92ba96f4813]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 13:23:31.119789 25367 tablet_service.cc:1511] Processing CreateTablet for tablet f1ac2f4d2fe2424bb470df42199219e2 (DEFAULT_TABLE table=test-workload [id=2f3f768873514f53bb58e92ba96f4813]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260709 13:23:31.121124 25367 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f1ac2f4d2fe2424bb470df42199219e2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:23:31.122324 25505 tablet_bootstrap.cc:492] T 0d29b499fa4c41f1b974782cd0c9c1c2 P 43999d813e564dc688fad116090c95ca: No bootstrap required, opened a new log
I20260709 13:23:31.122982 25505 ts_tablet_manager.cc:1403] T 0d29b499fa4c41f1b974782cd0c9c1c2 P 43999d813e564dc688fad116090c95ca: Time spent bootstrapping tablet: real 0.116s	user 0.010s	sys 0.007s
I20260709 13:23:31.100270 25442 tablet_service.cc:1511] Processing CreateTablet for tablet a5df750b84e0487fa530161079deb6c2 (DEFAULT_TABLE table=test-workload [id=2f3f768873514f53bb58e92ba96f4813]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 13:23:31.121125 25222 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 13bb3a23315a41c7a40e10b954b15c8e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:23:31.142226 25503 tablet_bootstrap.cc:492] T b2f14aed05e24a98ad83535029b795dc P 233e7d29aac04505bf4e65f0a39a0c4c: No bootstrap required, opened a new log
I20260709 13:23:31.142839 25503 ts_tablet_manager.cc:1403] T b2f14aed05e24a98ad83535029b795dc P 233e7d29aac04505bf4e65f0a39a0c4c: Time spent bootstrapping tablet: real 0.151s	user 0.007s	sys 0.010s
I20260709 13:23:31.152778 25448 tablet_service.cc:1511] Processing CreateTablet for tablet f1ac2f4d2fe2424bb470df42199219e2 (DEFAULT_TABLE table=test-workload [id=2f3f768873514f53bb58e92ba96f4813]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260709 13:23:31.156502 25503 raft_consensus.cc:359] T b2f14aed05e24a98ad83535029b795dc P 233e7d29aac04505bf4e65f0a39a0c4c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } }
I20260709 13:23:31.157441 25503 raft_consensus.cc:385] T b2f14aed05e24a98ad83535029b795dc P 233e7d29aac04505bf4e65f0a39a0c4c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:23:31.157755 25503 raft_consensus.cc:740] T b2f14aed05e24a98ad83535029b795dc P 233e7d29aac04505bf4e65f0a39a0c4c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 233e7d29aac04505bf4e65f0a39a0c4c, State: Initialized, Role: FOLLOWER
I20260709 13:23:31.149678 25504 tablet_bootstrap.cc:492] T d0a0e4d32ce943fbb773bb840863647b P 3e07d64466934da4bb3c02262b1c2e38: Bootstrap starting.
I20260709 13:23:31.158722 25503 consensus_queue.cc:260] T b2f14aed05e24a98ad83535029b795dc P 233e7d29aac04505bf4e65f0a39a0c4c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } }
I20260709 13:23:31.170183 25505 raft_consensus.cc:359] T 0d29b499fa4c41f1b974782cd0c9c1c2 P 43999d813e564dc688fad116090c95ca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } }
I20260709 13:23:31.171047 25505 raft_consensus.cc:385] T 0d29b499fa4c41f1b974782cd0c9c1c2 P 43999d813e564dc688fad116090c95ca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:23:31.171381 25505 raft_consensus.cc:740] T 0d29b499fa4c41f1b974782cd0c9c1c2 P 43999d813e564dc688fad116090c95ca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43999d813e564dc688fad116090c95ca, State: Initialized, Role: FOLLOWER
I20260709 13:23:31.172181 25505 consensus_queue.cc:260] T 0d29b499fa4c41f1b974782cd0c9c1c2 P 43999d813e564dc688fad116090c95ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } }
I20260709 13:23:31.174463 25505 ts_tablet_manager.cc:1434] T 0d29b499fa4c41f1b974782cd0c9c1c2 P 43999d813e564dc688fad116090c95ca: Time spent starting tablet: real 0.051s	user 0.005s	sys 0.000s
I20260709 13:23:31.175024 25442 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a5df750b84e0487fa530161079deb6c2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:23:31.178265 25505 tablet_bootstrap.cc:492] T 441d780c3945463f92f31170f31e37a8 P 43999d813e564dc688fad116090c95ca: Bootstrap starting.
I20260709 13:23:31.188114 25504 tablet_bootstrap.cc:654] T d0a0e4d32ce943fbb773bb840863647b P 3e07d64466934da4bb3c02262b1c2e38: Neither blocks nor log segments found. Creating new log.
I20260709 13:23:31.190467 25220 tablet_service.cc:1511] Processing CreateTablet for tablet f1ac2f4d2fe2424bb470df42199219e2 (DEFAULT_TABLE table=test-workload [id=2f3f768873514f53bb58e92ba96f4813]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260709 13:23:31.191905 25220 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f1ac2f4d2fe2424bb470df42199219e2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:23:31.194293 25505 tablet_bootstrap.cc:654] T 441d780c3945463f92f31170f31e37a8 P 43999d813e564dc688fad116090c95ca: Neither blocks nor log segments found. Creating new log.
I20260709 13:23:31.181792 25508 tablet_bootstrap.cc:492] T 3fe5a944a342411ea923eb39071ac770 P 9868dce521294d0b816bba7648f4ae5a: No bootstrap required, opened a new log
I20260709 13:23:31.195725 25508 ts_tablet_manager.cc:1403] T 3fe5a944a342411ea923eb39071ac770 P 9868dce521294d0b816bba7648f4ae5a: Time spent bootstrapping tablet: real 0.193s	user 0.025s	sys 0.033s
I20260709 13:23:31.196249 25368 tablet_service.cc:1511] Processing CreateTablet for tablet bbab48ba177049dcba6de106fd889cfb (DEFAULT_TABLE table=test-workload [id=2f3f768873514f53bb58e92ba96f4813]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 13:23:31.197671 25368 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bbab48ba177049dcba6de106fd889cfb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:23:31.176110 25503 ts_tablet_manager.cc:1434] T b2f14aed05e24a98ad83535029b795dc P 233e7d29aac04505bf4e65f0a39a0c4c: Time spent starting tablet: real 0.033s	user 0.007s	sys 0.000s
I20260709 13:23:31.187048 25512 raft_consensus.cc:493] T b2f14aed05e24a98ad83535029b795dc P 233e7d29aac04505bf4e65f0a39a0c4c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:23:31.201128 25508 raft_consensus.cc:359] T 3fe5a944a342411ea923eb39071ac770 P 9868dce521294d0b816bba7648f4ae5a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } }
I20260709 13:23:31.201347 25512 raft_consensus.cc:515] T b2f14aed05e24a98ad83535029b795dc P 233e7d29aac04505bf4e65f0a39a0c4c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } }
I20260709 13:23:31.201967 25508 raft_consensus.cc:385] T 3fe5a944a342411ea923eb39071ac770 P 9868dce521294d0b816bba7648f4ae5a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:23:31.202344 25508 raft_consensus.cc:740] T 3fe5a944a342411ea923eb39071ac770 P 9868dce521294d0b816bba7648f4ae5a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9868dce521294d0b816bba7648f4ae5a, State: Initialized, Role: FOLLOWER
I20260709 13:23:31.203282 25508 consensus_queue.cc:260] T 3fe5a944a342411ea923eb39071ac770 P 9868dce521294d0b816bba7648f4ae5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } }
I20260709 13:23:31.203855 25512 leader_election.cc:290] T b2f14aed05e24a98ad83535029b795dc P 233e7d29aac04505bf4e65f0a39a0c4c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3e07d64466934da4bb3c02262b1c2e38 (127.14.177.131:41539), 43999d813e564dc688fad116090c95ca (127.14.177.129:38155)
I20260709 13:23:31.204567 25448 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f1ac2f4d2fe2424bb470df42199219e2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:23:31.205643 25508 ts_tablet_manager.cc:1434] T 3fe5a944a342411ea923eb39071ac770 P 9868dce521294d0b816bba7648f4ae5a: Time spent starting tablet: real 0.010s	user 0.005s	sys 0.001s
I20260709 13:23:31.206768 25508 tablet_bootstrap.cc:492] T 441d780c3945463f92f31170f31e37a8 P 9868dce521294d0b816bba7648f4ae5a: Bootstrap starting.
I20260709 13:23:31.210193 25510 raft_consensus.cc:493] T d0a0e4d32ce943fbb773bb840863647b P 43999d813e564dc688fad116090c95ca [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:23:31.210749 25510 raft_consensus.cc:515] T d0a0e4d32ce943fbb773bb840863647b P 43999d813e564dc688fad116090c95ca [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } }
I20260709 13:23:31.212585 25236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2f14aed05e24a98ad83535029b795dc" candidate_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43999d813e564dc688fad116090c95ca" is_pre_election: true
I20260709 13:23:31.212915 25510 leader_election.cc:290] T d0a0e4d32ce943fbb773bb840863647b P 43999d813e564dc688fad116090c95ca [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3e07d64466934da4bb3c02262b1c2e38 (127.14.177.131:41539), 9868dce521294d0b816bba7648f4ae5a (127.14.177.132:33683)
I20260709 13:23:31.195181 25300 tablet_service.cc:1511] Processing CreateTablet for tablet bbab48ba177049dcba6de106fd889cfb (DEFAULT_TABLE table=test-workload [id=2f3f768873514f53bb58e92ba96f4813]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 13:23:31.214664 25311 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28c0b6fc9bd64d60828d705c6a9ab1e4" candidate_uuid: "3e07d64466934da4bb3c02262b1c2e38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" is_pre_election: true
I20260709 13:23:31.214954 25300 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bbab48ba177049dcba6de106fd889cfb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:23:31.217348 25459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0a0e4d32ce943fbb773bb840863647b" candidate_uuid: "43999d813e564dc688fad116090c95ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9868dce521294d0b816bba7648f4ae5a" is_pre_election: true
I20260709 13:23:31.218055 25459 raft_consensus.cc:2468] T d0a0e4d32ce943fbb773bb840863647b P 9868dce521294d0b816bba7648f4ae5a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 43999d813e564dc688fad116090c95ca in term 0.
I20260709 13:23:31.219604 25236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28c0b6fc9bd64d60828d705c6a9ab1e4" candidate_uuid: "3e07d64466934da4bb3c02262b1c2e38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43999d813e564dc688fad116090c95ca" is_pre_election: true
W20260709 13:23:31.220268 25351 leader_election.cc:343] T 28c0b6fc9bd64d60828d705c6a9ab1e4 P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 233e7d29aac04505bf4e65f0a39a0c4c (127.14.177.130:42957): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:23:31.220899 25276 leader_election.cc:343] T b2f14aed05e24a98ad83535029b795dc P 233e7d29aac04505bf4e65f0a39a0c4c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 43999d813e564dc688fad116090c95ca (127.14.177.129:38155): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:23:31.221885 25350 leader_election.cc:343] T 28c0b6fc9bd64d60828d705c6a9ab1e4 P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 43999d813e564dc688fad116090c95ca (127.14.177.129:38155): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:23:31.221910 25199 leader_election.cc:304] T d0a0e4d32ce943fbb773bb840863647b P 43999d813e564dc688fad116090c95ca [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 43999d813e564dc688fad116090c95ca, 9868dce521294d0b816bba7648f4ae5a; no voters: 
I20260709 13:23:31.221235 25385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0a0e4d32ce943fbb773bb840863647b" candidate_uuid: "43999d813e564dc688fad116090c95ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e07d64466934da4bb3c02262b1c2e38" is_pre_election: true
I20260709 13:23:31.222600 25350 leader_election.cc:304] T 28c0b6fc9bd64d60828d705c6a9ab1e4 P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3e07d64466934da4bb3c02262b1c2e38; no voters: 233e7d29aac04505bf4e65f0a39a0c4c, 43999d813e564dc688fad116090c95ca
I20260709 13:23:31.226140 25518 raft_consensus.cc:2749] T 28c0b6fc9bd64d60828d705c6a9ab1e4 P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:23:31.227087 25199 leader_election.cc:343] T d0a0e4d32ce943fbb773bb840863647b P 43999d813e564dc688fad116090c95ca [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3e07d64466934da4bb3c02262b1c2e38 (127.14.177.131:41539): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:23:31.225270 25512 raft_consensus.cc:493] T b7ff8dde9f5d4c49b9203d902f6b94c4 P 233e7d29aac04505bf4e65f0a39a0c4c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:23:31.228612 25510 raft_consensus.cc:2804] T d0a0e4d32ce943fbb773bb840863647b P 43999d813e564dc688fad116090c95ca [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:23:31.229081 25510 raft_consensus.cc:493] T d0a0e4d32ce943fbb773bb840863647b P 43999d813e564dc688fad116090c95ca [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:23:31.229548 25510 raft_consensus.cc:3060] T d0a0e4d32ce943fbb773bb840863647b P 43999d813e564dc688fad116090c95ca [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:23:31.228932 25512 raft_consensus.cc:515] T b7ff8dde9f5d4c49b9203d902f6b94c4 P 233e7d29aac04505bf4e65f0a39a0c4c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } }
I20260709 13:23:31.231527 25503 tablet_bootstrap.cc:492] T 28c0b6fc9bd64d60828d705c6a9ab1e4 P 233e7d29aac04505bf4e65f0a39a0c4c: Bootstrap starting.
I20260709 13:23:31.233707 25512 leader_election.cc:290] T b7ff8dde9f5d4c49b9203d902f6b94c4 P 233e7d29aac04505bf4e65f0a39a0c4c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 43999d813e564dc688fad116090c95ca (127.14.177.129:38155), 3e07d64466934da4bb3c02262b1c2e38 (127.14.177.131:41539)
I20260709 13:23:31.234556 25236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7ff8dde9f5d4c49b9203d902f6b94c4" candidate_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43999d813e564dc688fad116090c95ca" is_pre_election: true
I20260709 13:23:31.237977 25384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7ff8dde9f5d4c49b9203d902f6b94c4" candidate_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e07d64466934da4bb3c02262b1c2e38" is_pre_election: true
W20260709 13:23:31.239391 25276 leader_election.cc:343] T b7ff8dde9f5d4c49b9203d902f6b94c4 P 233e7d29aac04505bf4e65f0a39a0c4c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 43999d813e564dc688fad116090c95ca (127.14.177.129:38155): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:23:31.244237 25274 leader_election.cc:343] T b7ff8dde9f5d4c49b9203d902f6b94c4 P 233e7d29aac04505bf4e65f0a39a0c4c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3e07d64466934da4bb3c02262b1c2e38 (127.14.177.131:41539): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:23:31.244910 25274 leader_election.cc:304] T b7ff8dde9f5d4c49b9203d902f6b94c4 P 233e7d29aac04505bf4e65f0a39a0c4c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 233e7d29aac04505bf4e65f0a39a0c4c; no voters: 3e07d64466934da4bb3c02262b1c2e38, 43999d813e564dc688fad116090c95ca
I20260709 13:23:31.246074 25512 raft_consensus.cc:2749] T b7ff8dde9f5d4c49b9203d902f6b94c4 P 233e7d29aac04505bf4e65f0a39a0c4c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:23:31.246798 25385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2f14aed05e24a98ad83535029b795dc" candidate_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e07d64466934da4bb3c02262b1c2e38" is_pre_election: true
W20260709 13:23:31.249490 25274 leader_election.cc:343] T b2f14aed05e24a98ad83535029b795dc P 233e7d29aac04505bf4e65f0a39a0c4c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3e07d64466934da4bb3c02262b1c2e38 (127.14.177.131:41539): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:23:31.250137 25274 leader_election.cc:304] T b2f14aed05e24a98ad83535029b795dc P 233e7d29aac04505bf4e65f0a39a0c4c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 233e7d29aac04505bf4e65f0a39a0c4c; no voters: 3e07d64466934da4bb3c02262b1c2e38, 43999d813e564dc688fad116090c95ca
I20260709 13:23:31.252146 25512 raft_consensus.cc:2749] T b2f14aed05e24a98ad83535029b795dc P 233e7d29aac04505bf4e65f0a39a0c4c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:23:31.220263 25446 tablet_service.cc:1511] Processing CreateTablet for tablet bbab48ba177049dcba6de106fd889cfb (DEFAULT_TABLE table=test-workload [id=2f3f768873514f53bb58e92ba96f4813]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 13:23:31.267244 25446 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bbab48ba177049dcba6de106fd889cfb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:23:31.272028 25225 tablet_service.cc:1511] Processing CreateTablet for tablet 79d4a12ef58140568c60a24ed4c12f83 (DEFAULT_TABLE table=test-workload [id=2f3f768873514f53bb58e92ba96f4813]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260709 13:23:31.273530 25505 tablet_bootstrap.cc:492] T 441d780c3945463f92f31170f31e37a8 P 43999d813e564dc688fad116090c95ca: No bootstrap required, opened a new log
I20260709 13:23:31.274290 25505 ts_tablet_manager.cc:1403] T 441d780c3945463f92f31170f31e37a8 P 43999d813e564dc688fad116090c95ca: Time spent bootstrapping tablet: real 0.096s	user 0.004s	sys 0.021s
I20260709 13:23:31.277536 25503 tablet_bootstrap.cc:654] T 28c0b6fc9bd64d60828d705c6a9ab1e4 P 233e7d29aac04505bf4e65f0a39a0c4c: Neither blocks nor log segments found. Creating new log.
I20260709 13:23:31.279881 25370 tablet_service.cc:1511] Processing CreateTablet for tablet 79d4a12ef58140568c60a24ed4c12f83 (DEFAULT_TABLE table=test-workload [id=2f3f768873514f53bb58e92ba96f4813]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260709 13:23:31.277644 25505 raft_consensus.cc:359] T 441d780c3945463f92f31170f31e37a8 P 43999d813e564dc688fad116090c95ca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } }
I20260709 13:23:31.277216 25298 tablet_service.cc:1511] Processing CreateTablet for tablet 79d4a12ef58140568c60a24ed4c12f83 (DEFAULT_TABLE table=test-workload [id=2f3f768873514f53bb58e92ba96f4813]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260709 13:23:31.273507 25225 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 79d4a12ef58140568c60a24ed4c12f83. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:23:31.282048 25508 tablet_bootstrap.cc:654] T 441d780c3945463f92f31170f31e37a8 P 9868dce521294d0b816bba7648f4ae5a: Neither blocks nor log segments found. Creating new log.
I20260709 13:23:31.282653 25298 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 79d4a12ef58140568c60a24ed4c12f83. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:23:31.281368 25505 raft_consensus.cc:385] T 441d780c3945463f92f31170f31e37a8 P 43999d813e564dc688fad116090c95ca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:23:31.283886 25505 raft_consensus.cc:740] T 441d780c3945463f92f31170f31e37a8 P 43999d813e564dc688fad116090c95ca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43999d813e564dc688fad116090c95ca, State: Initialized, Role: FOLLOWER
I20260709 13:23:31.284228 25293 tablet_service.cc:1511] Processing CreateTablet for tablet 6242888a4d4146dcacd87d0fd9a832f4 (DEFAULT_TABLE table=test-workload [id=2f3f768873514f53bb58e92ba96f4813]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 13:23:31.284857 25505 consensus_queue.cc:260] T 441d780c3945463f92f31170f31e37a8 P 43999d813e564dc688fad116090c95ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } }
I20260709 13:23:31.285571 25293 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6242888a4d4146dcacd87d0fd9a832f4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:23:31.281316 25370 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 79d4a12ef58140568c60a24ed4c12f83. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:23:31.288245 25441 tablet_service.cc:1511] Processing CreateTablet for tablet 6242888a4d4146dcacd87d0fd9a832f4 (DEFAULT_TABLE table=test-workload [id=2f3f768873514f53bb58e92ba96f4813]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 13:23:31.289593 25441 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6242888a4d4146dcacd87d0fd9a832f4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:23:31.261922 25510 raft_consensus.cc:515] T d0a0e4d32ce943fbb773bb840863647b P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } }
I20260709 13:23:31.301554 25385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0a0e4d32ce943fbb773bb840863647b" candidate_uuid: "43999d813e564dc688fad116090c95ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e07d64466934da4bb3c02262b1c2e38"
W20260709 13:23:31.303118 25199 leader_election.cc:343] T d0a0e4d32ce943fbb773bb840863647b P 43999d813e564dc688fad116090c95ca [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3e07d64466934da4bb3c02262b1c2e38 (127.14.177.131:41539): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:23:31.306483 25504 tablet_bootstrap.cc:492] T d0a0e4d32ce943fbb773bb840863647b P 3e07d64466934da4bb3c02262b1c2e38: No bootstrap required, opened a new log
I20260709 13:23:31.308249 25504 ts_tablet_manager.cc:1403] T d0a0e4d32ce943fbb773bb840863647b P 3e07d64466934da4bb3c02262b1c2e38: Time spent bootstrapping tablet: real 0.171s	user 0.021s	sys 0.027s
I20260709 13:23:31.311429 25533 raft_consensus.cc:493] T 0d29b499fa4c41f1b974782cd0c9c1c2 P 43999d813e564dc688fad116090c95ca [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:23:31.319164 25222 tablet_service.cc:1511] Processing CreateTablet for tablet 6242888a4d4146dcacd87d0fd9a832f4 (DEFAULT_TABLE table=test-workload [id=2f3f768873514f53bb58e92ba96f4813]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 13:23:31.319342 25533 raft_consensus.cc:515] T 0d29b499fa4c41f1b974782cd0c9c1c2 P 43999d813e564dc688fad116090c95ca [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } }
I20260709 13:23:31.321934 25533 leader_election.cc:290] T 0d29b499fa4c41f1b974782cd0c9c1c2 P 43999d813e564dc688fad116090c95ca [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 233e7d29aac04505bf4e65f0a39a0c4c (127.14.177.130:42957), 3e07d64466934da4bb3c02262b1c2e38 (127.14.177.131:41539)
I20260709 13:23:31.323530 25385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d29b499fa4c41f1b974782cd0c9c1c2" candidate_uuid: "43999d813e564dc688fad116090c95ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e07d64466934da4bb3c02262b1c2e38" is_pre_election: true
I20260709 13:23:31.325062 25371 tablet_service.cc:1511] Processing CreateTablet for tablet 91ac744179de48e5bb924b6dfd9f9c9b (DEFAULT_TABLE table=test-workload [id=2f3f768873514f53bb58e92ba96f4813]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260709 13:23:31.326625 25371 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 91ac744179de48e5bb924b6dfd9f9c9b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:23:31.326256 25504 raft_consensus.cc:359] T d0a0e4d32ce943fbb773bb840863647b P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } }
I20260709 13:23:31.327059 25504 raft_consensus.cc:385] T d0a0e4d32ce943fbb773bb840863647b P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260709 13:23:31.325207 25199 leader_election.cc:343] T 0d29b499fa4c41f1b974782cd0c9c1c2 P 43999d813e564dc688fad116090c95ca [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3e07d64466934da4bb3c02262b1c2e38 (127.14.177.131:41539): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:23:31.327436 25504 raft_consensus.cc:740] T d0a0e4d32ce943fbb773bb840863647b P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e07d64466934da4bb3c02262b1c2e38, State: Initialized, Role: FOLLOWER
I20260709 13:23:31.334038 25518 raft_consensus.cc:493] T b7a891bfc89e4e259f53da6cb16f4989 P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:23:31.334681 25518 raft_consensus.cc:515] T b7a891bfc89e4e259f53da6cb16f4989 P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } }
I20260709 13:23:31.334856 25504 consensus_queue.cc:260] T d0a0e4d32ce943fbb773bb840863647b P 3e07d64466934da4bb3c02262b1c2e38 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } }
I20260709 13:23:31.338457 25443 tablet_service.cc:1511] Processing CreateTablet for tablet 91ac744179de48e5bb924b6dfd9f9c9b (DEFAULT_TABLE table=test-workload [id=2f3f768873514f53bb58e92ba96f4813]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260709 13:23:31.343428 25236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7a891bfc89e4e259f53da6cb16f4989" candidate_uuid: "3e07d64466934da4bb3c02262b1c2e38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43999d813e564dc688fad116090c95ca" is_pre_election: true
I20260709 13:23:31.343999 25443 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 91ac744179de48e5bb924b6dfd9f9c9b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:23:31.344185 25236 raft_consensus.cc:2468] T b7a891bfc89e4e259f53da6cb16f4989 P 43999d813e564dc688fad116090c95ca [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3e07d64466934da4bb3c02262b1c2e38 in term 0.
I20260709 13:23:31.345654 25350 leader_election.cc:304] T b7a891bfc89e4e259f53da6cb16f4989 P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e07d64466934da4bb3c02262b1c2e38, 43999d813e564dc688fad116090c95ca; no voters: 
I20260709 13:23:31.337349 25518 leader_election.cc:290] T b7a891bfc89e4e259f53da6cb16f4989 P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9868dce521294d0b816bba7648f4ae5a (127.14.177.132:33683), 43999d813e564dc688fad116090c95ca (127.14.177.129:38155)
I20260709 13:23:31.334277 25222 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6242888a4d4146dcacd87d0fd9a832f4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:23:31.354748 25459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0a0e4d32ce943fbb773bb840863647b" candidate_uuid: "43999d813e564dc688fad116090c95ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9868dce521294d0b816bba7648f4ae5a"
I20260709 13:23:31.355461 25298 tablet_service.cc:1511] Processing CreateTablet for tablet 434a18a5109e4ea78349ded3968b2709 (DEFAULT_TABLE table=test-workload [id=2f3f768873514f53bb58e92ba96f4813]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260709 13:23:31.355501 25459 raft_consensus.cc:3060] T d0a0e4d32ce943fbb773bb840863647b P 9868dce521294d0b816bba7648f4ae5a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:23:31.355576 25311 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d29b499fa4c41f1b974782cd0c9c1c2" candidate_uuid: "43999d813e564dc688fad116090c95ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" is_pre_election: true
I20260709 13:23:31.351269 25518 raft_consensus.cc:2804] T b7a891bfc89e4e259f53da6cb16f4989 P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:23:31.364655 25518 raft_consensus.cc:493] T b7a891bfc89e4e259f53da6cb16f4989 P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:23:31.365212 25518 raft_consensus.cc:3060] T b7a891bfc89e4e259f53da6cb16f4989 P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:23:31.355625 25220 tablet_service.cc:1511] Processing CreateTablet for tablet 91ac744179de48e5bb924b6dfd9f9c9b (DEFAULT_TABLE table=test-workload [id=2f3f768873514f53bb58e92ba96f4813]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260709 13:23:31.375747 25505 ts_tablet_manager.cc:1434] T 441d780c3945463f92f31170f31e37a8 P 43999d813e564dc688fad116090c95ca: Time spent starting tablet: real 0.101s	user 0.008s	sys 0.013s
I20260709 13:23:31.357237 25510 leader_election.cc:290] T d0a0e4d32ce943fbb773bb840863647b P 43999d813e564dc688fad116090c95ca [CANDIDATE]: Term 1 election: Requested vote from peers 3e07d64466934da4bb3c02262b1c2e38 (127.14.177.131:41539), 9868dce521294d0b816bba7648f4ae5a (127.14.177.132:33683)
I20260709 13:23:31.376925 25458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7a891bfc89e4e259f53da6cb16f4989" candidate_uuid: "3e07d64466934da4bb3c02262b1c2e38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9868dce521294d0b816bba7648f4ae5a" is_pre_election: true
I20260709 13:23:31.378031 25220 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 91ac744179de48e5bb924b6dfd9f9c9b. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:23:31.378867 25348 leader_election.cc:343] T b7a891bfc89e4e259f53da6cb16f4989 P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9868dce521294d0b816bba7648f4ae5a (127.14.177.132:33683): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:23:31.379338 25508 tablet_bootstrap.cc:492] T 441d780c3945463f92f31170f31e37a8 P 9868dce521294d0b816bba7648f4ae5a: No bootstrap required, opened a new log
I20260709 13:23:31.379889 25508 ts_tablet_manager.cc:1403] T 441d780c3945463f92f31170f31e37a8 P 9868dce521294d0b816bba7648f4ae5a: Time spent bootstrapping tablet: real 0.173s	user 0.017s	sys 0.002s
I20260709 13:23:31.375811 25504 ts_tablet_manager.cc:1434] T d0a0e4d32ce943fbb773bb840863647b P 3e07d64466934da4bb3c02262b1c2e38: Time spent starting tablet: real 0.066s	user 0.008s	sys 0.000s
I20260709 13:23:31.357314 25298 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 434a18a5109e4ea78349ded3968b2709. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:23:31.363404 25202 leader_election.cc:343] T 0d29b499fa4c41f1b974782cd0c9c1c2 P 43999d813e564dc688fad116090c95ca [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 233e7d29aac04505bf4e65f0a39a0c4c (127.14.177.130:42957): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:23:31.382982 25508 raft_consensus.cc:359] T 441d780c3945463f92f31170f31e37a8 P 9868dce521294d0b816bba7648f4ae5a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } }
I20260709 13:23:31.383476 25202 leader_election.cc:304] T 0d29b499fa4c41f1b974782cd0c9c1c2 P 43999d813e564dc688fad116090c95ca [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 43999d813e564dc688fad116090c95ca; no voters: 233e7d29aac04505bf4e65f0a39a0c4c, 3e07d64466934da4bb3c02262b1c2e38
I20260709 13:23:31.383819 25508 raft_consensus.cc:385] T 441d780c3945463f92f31170f31e37a8 P 9868dce521294d0b816bba7648f4ae5a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:23:31.384230 25508 raft_consensus.cc:740] T 441d780c3945463f92f31170f31e37a8 P 9868dce521294d0b816bba7648f4ae5a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9868dce521294d0b816bba7648f4ae5a, State: Initialized, Role: FOLLOWER
I20260709 13:23:31.384716 25510 raft_consensus.cc:2749] T 0d29b499fa4c41f1b974782cd0c9c1c2 P 43999d813e564dc688fad116090c95ca [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:23:31.388429 25505 tablet_bootstrap.cc:492] T 0e71b6f5c69b4b35a8ac51c22c7d5f0d P 43999d813e564dc688fad116090c95ca: Bootstrap starting.
I20260709 13:23:31.400905 25368 tablet_service.cc:1511] Processing CreateTablet for tablet 434a18a5109e4ea78349ded3968b2709 (DEFAULT_TABLE table=test-workload [id=2f3f768873514f53bb58e92ba96f4813]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260709 13:23:31.401840 25503 tablet_bootstrap.cc:492] T 28c0b6fc9bd64d60828d705c6a9ab1e4 P 233e7d29aac04505bf4e65f0a39a0c4c: No bootstrap required, opened a new log
I20260709 13:23:31.402383 25503 ts_tablet_manager.cc:1403] T 28c0b6fc9bd64d60828d705c6a9ab1e4 P 233e7d29aac04505bf4e65f0a39a0c4c: Time spent bootstrapping tablet: real 0.171s	user 0.024s	sys 0.004s
I20260709 13:23:31.402434 25368 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 434a18a5109e4ea78349ded3968b2709. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:23:31.405546 25504 tablet_bootstrap.cc:492] T a820954e07a94582ac0613342fcffde5 P 3e07d64466934da4bb3c02262b1c2e38: Bootstrap starting.
I20260709 13:23:31.405684 25459 raft_consensus.cc:2468] T d0a0e4d32ce943fbb773bb840863647b P 9868dce521294d0b816bba7648f4ae5a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 43999d813e564dc688fad116090c95ca in term 1.
I20260709 13:23:31.420359 25199 leader_election.cc:304] T d0a0e4d32ce943fbb773bb840863647b P 43999d813e564dc688fad116090c95ca [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 43999d813e564dc688fad116090c95ca, 9868dce521294d0b816bba7648f4ae5a; no voters: 3e07d64466934da4bb3c02262b1c2e38
I20260709 13:23:31.421862 25510 raft_consensus.cc:2804] T d0a0e4d32ce943fbb773bb840863647b P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:23:31.422341 25510 raft_consensus.cc:697] T d0a0e4d32ce943fbb773bb840863647b P 43999d813e564dc688fad116090c95ca [term 1 LEADER]: Becoming Leader. State: Replica: 43999d813e564dc688fad116090c95ca, State: Running, Role: LEADER
I20260709 13:23:31.408639 25448 tablet_service.cc:1511] Processing CreateTablet for tablet 434a18a5109e4ea78349ded3968b2709 (DEFAULT_TABLE table=test-workload [id=2f3f768873514f53bb58e92ba96f4813]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260709 13:23:31.398834 25508 consensus_queue.cc:260] T 441d780c3945463f92f31170f31e37a8 P 9868dce521294d0b816bba7648f4ae5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } }
I20260709 13:23:31.424263 25524 raft_consensus.cc:493] T 3fe5a944a342411ea923eb39071ac770 P 9868dce521294d0b816bba7648f4ae5a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:23:31.407310 25503 raft_consensus.cc:359] T 28c0b6fc9bd64d60828d705c6a9ab1e4 P 233e7d29aac04505bf4e65f0a39a0c4c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } }
I20260709 13:23:31.425381 25503 raft_consensus.cc:385] T 28c0b6fc9bd64d60828d705c6a9ab1e4 P 233e7d29aac04505bf4e65f0a39a0c4c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:23:31.425199 25524 raft_consensus.cc:515] T 3fe5a944a342411ea923eb39071ac770 P 9868dce521294d0b816bba7648f4ae5a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } }
I20260709 13:23:31.425797 25503 raft_consensus.cc:740] T 28c0b6fc9bd64d60828d705c6a9ab1e4 P 233e7d29aac04505bf4e65f0a39a0c4c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 233e7d29aac04505bf4e65f0a39a0c4c, State: Initialized, Role: FOLLOWER
I20260709 13:23:31.426797 25503 consensus_queue.cc:260] T 28c0b6fc9bd64d60828d705c6a9ab1e4 P 233e7d29aac04505bf4e65f0a39a0c4c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } }
I20260709 13:23:31.427448 25504 tablet_bootstrap.cc:654] T a820954e07a94582ac0613342fcffde5 P 3e07d64466934da4bb3c02262b1c2e38: Neither blocks nor log segments found. Creating new log.
I20260709 13:23:31.427877 25510 consensus_queue.cc:237] T d0a0e4d32ce943fbb773bb840863647b P 43999d813e564dc688fad116090c95ca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } }
I20260709 13:23:31.429441 25503 ts_tablet_manager.cc:1434] T 28c0b6fc9bd64d60828d705c6a9ab1e4 P 233e7d29aac04505bf4e65f0a39a0c4c: Time spent starting tablet: real 0.027s	user 0.009s	sys 0.000s
I20260709 13:23:31.433001 25225 tablet_service.cc:1511] Processing CreateTablet for tablet 87085ac2575e4b6a86dd1fc21e6f718b (DEFAULT_TABLE table=test-workload [id=2f3f768873514f53bb58e92ba96f4813]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260709 13:23:31.436816 25225 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 87085ac2575e4b6a86dd1fc21e6f718b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:23:31.424293 25448 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 434a18a5109e4ea78349ded3968b2709. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:23:31.441993 25503 tablet_bootstrap.cc:492] T e9e342ecda4b46619ce12452b2cf7154 P 233e7d29aac04505bf4e65f0a39a0c4c: Bootstrap starting.
W20260709 13:23:31.409090 25350 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:38155, user_credentials={real_user=slave}} blocked reactor thread for 63534.8us
I20260709 13:23:31.449222 25293 tablet_service.cc:1511] Processing CreateTablet for tablet 87085ac2575e4b6a86dd1fc21e6f718b (DEFAULT_TABLE table=test-workload [id=2f3f768873514f53bb58e92ba96f4813]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260709 13:23:31.452581 25518 raft_consensus.cc:515] T b7a891bfc89e4e259f53da6cb16f4989 P 3e07d64466934da4bb3c02262b1c2e38 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } }
I20260709 13:23:31.461504 25505 tablet_bootstrap.cc:654] T 0e71b6f5c69b4b35a8ac51c22c7d5f0d P 43999d813e564dc688fad116090c95ca: Neither blocks nor log segments found. Creating new log.
I20260709 13:23:31.460577 25293 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 87085ac2575e4b6a86dd1fc21e6f718b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:23:31.463425 25459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7a891bfc89e4e259f53da6cb16f4989" candidate_uuid: "3e07d64466934da4bb3c02262b1c2e38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9868dce521294d0b816bba7648f4ae5a"
W20260709 13:23:31.464962 25348 leader_election.cc:343] T b7a891bfc89e4e259f53da6cb16f4989 P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 9868dce521294d0b816bba7648f4ae5a (127.14.177.132:33683): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:23:31.439656 25524 leader_election.cc:290] T 3fe5a944a342411ea923eb39071ac770 P 9868dce521294d0b816bba7648f4ae5a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 233e7d29aac04505bf4e65f0a39a0c4c (127.14.177.130:42957), 3e07d64466934da4bb3c02262b1c2e38 (127.14.177.131:41539)
I20260709 13:23:31.470616 25300 tablet_service.cc:1511] Processing CreateTablet for tablet ce9c60e3922c4219b6e4453fe0259b6a (DEFAULT_TABLE table=test-workload [id=2f3f768873514f53bb58e92ba96f4813]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260709 13:23:31.475828 25518 leader_election.cc:290] T b7a891bfc89e4e259f53da6cb16f4989 P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 1 election: Requested vote from peers 9868dce521294d0b816bba7648f4ae5a (127.14.177.132:33683), 43999d813e564dc688fad116090c95ca (127.14.177.129:38155)
I20260709 13:23:31.476241 25300 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ce9c60e3922c4219b6e4453fe0259b6a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:23:31.471294 25220 tablet_service.cc:1511] Processing CreateTablet for tablet ce9c60e3922c4219b6e4453fe0259b6a (DEFAULT_TABLE table=test-workload [id=2f3f768873514f53bb58e92ba96f4813]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260709 13:23:31.480001 25220 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ce9c60e3922c4219b6e4453fe0259b6a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:23:31.485128 25370 tablet_service.cc:1511] Processing CreateTablet for tablet 87085ac2575e4b6a86dd1fc21e6f718b (DEFAULT_TABLE table=test-workload [id=2f3f768873514f53bb58e92ba96f4813]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260709 13:23:31.486583 25370 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 87085ac2575e4b6a86dd1fc21e6f718b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:23:31.488680 25446 tablet_service.cc:1511] Processing CreateTablet for tablet ce9c60e3922c4219b6e4453fe0259b6a (DEFAULT_TABLE table=test-workload [id=2f3f768873514f53bb58e92ba96f4813]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260709 13:23:31.490031 25446 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ce9c60e3922c4219b6e4453fe0259b6a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:23:31.492653 25236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7a891bfc89e4e259f53da6cb16f4989" candidate_uuid: "3e07d64466934da4bb3c02262b1c2e38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43999d813e564dc688fad116090c95ca"
I20260709 13:23:31.494087 25236 raft_consensus.cc:3060] T b7a891bfc89e4e259f53da6cb16f4989 P 43999d813e564dc688fad116090c95ca [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:23:31.502591 25236 raft_consensus.cc:2468] T b7a891bfc89e4e259f53da6cb16f4989 P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3e07d64466934da4bb3c02262b1c2e38 in term 1.
I20260709 13:23:31.508169 25367 tablet_service.cc:1511] Processing CreateTablet for tablet 0ed5edbe32044d1c83b0590dbb899801 (DEFAULT_TABLE table=test-workload [id=2f3f768873514f53bb58e92ba96f4813]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 13:23:31.508847 25385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe5a944a342411ea923eb39071ac770" candidate_uuid: "9868dce521294d0b816bba7648f4ae5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e07d64466934da4bb3c02262b1c2e38" is_pre_election: true
I20260709 13:23:31.509519 25367 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0ed5edbe32044d1c83b0590dbb899801. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:23:31.513545 25504 tablet_bootstrap.cc:492] T a820954e07a94582ac0613342fcffde5 P 3e07d64466934da4bb3c02262b1c2e38: No bootstrap required, opened a new log
I20260709 13:23:31.514139 25504 ts_tablet_manager.cc:1403] T a820954e07a94582ac0613342fcffde5 P 3e07d64466934da4bb3c02262b1c2e38: Time spent bootstrapping tablet: real 0.129s	user 0.021s	sys 0.005s
I20260709 13:23:31.518162 25504 raft_consensus.cc:359] T a820954e07a94582ac0613342fcffde5 P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } }
I20260709 13:23:31.518935 25504 raft_consensus.cc:385] T a820954e07a94582ac0613342fcffde5 P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:23:31.519287 25504 raft_consensus.cc:740] T a820954e07a94582ac0613342fcffde5 P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e07d64466934da4bb3c02262b1c2e38, State: Initialized, Role: FOLLOWER
I20260709 13:23:31.520061 25504 consensus_queue.cc:260] T a820954e07a94582ac0613342fcffde5 P 3e07d64466934da4bb3c02262b1c2e38 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } }
I20260709 13:23:31.522461 25504 ts_tablet_manager.cc:1434] T a820954e07a94582ac0613342fcffde5 P 3e07d64466934da4bb3c02262b1c2e38: Time spent starting tablet: real 0.008s	user 0.002s	sys 0.005s
W20260709 13:23:31.510272 25422 leader_election.cc:343] T 3fe5a944a342411ea923eb39071ac770 P 9868dce521294d0b816bba7648f4ae5a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3e07d64466934da4bb3c02262b1c2e38 (127.14.177.131:41539): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:23:31.533360 25503 tablet_bootstrap.cc:654] T e9e342ecda4b46619ce12452b2cf7154 P 233e7d29aac04505bf4e65f0a39a0c4c: Neither blocks nor log segments found. Creating new log.
I20260709 13:23:31.537187 25508 ts_tablet_manager.cc:1434] T 441d780c3945463f92f31170f31e37a8 P 9868dce521294d0b816bba7648f4ae5a: Time spent starting tablet: real 0.157s	user 0.035s	sys 0.004s
I20260709 13:23:31.537578 25350 leader_election.cc:304] T b7a891bfc89e4e259f53da6cb16f4989 P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3e07d64466934da4bb3c02262b1c2e38, 43999d813e564dc688fad116090c95ca; no voters: 9868dce521294d0b816bba7648f4ae5a
I20260709 13:23:31.539832 25508 tablet_bootstrap.cc:492] T b7a891bfc89e4e259f53da6cb16f4989 P 9868dce521294d0b816bba7648f4ae5a: Bootstrap starting.
I20260709 13:23:31.538926 25518 raft_consensus.cc:2804] T b7a891bfc89e4e259f53da6cb16f4989 P 3e07d64466934da4bb3c02262b1c2e38 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:23:31.541541 25518 raft_consensus.cc:697] T b7a891bfc89e4e259f53da6cb16f4989 P 3e07d64466934da4bb3c02262b1c2e38 [term 1 LEADER]: Becoming Leader. State: Replica: 3e07d64466934da4bb3c02262b1c2e38, State: Running, Role: LEADER
I20260709 13:23:31.542372 25518 consensus_queue.cc:237] T b7a891bfc89e4e259f53da6cb16f4989 P 3e07d64466934da4bb3c02262b1c2e38 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } }
I20260709 13:23:31.545706 25512 raft_consensus.cc:493] T 28c0b6fc9bd64d60828d705c6a9ab1e4 P 233e7d29aac04505bf4e65f0a39a0c4c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:23:31.546195 25512 raft_consensus.cc:515] T 28c0b6fc9bd64d60828d705c6a9ab1e4 P 233e7d29aac04505bf4e65f0a39a0c4c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } }
I20260709 13:23:31.546887 25446 tablet_service.cc:1511] Processing CreateTablet for tablet 0ed5edbe32044d1c83b0590dbb899801 (DEFAULT_TABLE table=test-workload [id=2f3f768873514f53bb58e92ba96f4813]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 13:23:31.548264 25446 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0ed5edbe32044d1c83b0590dbb899801. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:23:31.547366 25131 catalog_manager.cc:5697] T d0a0e4d32ce943fbb773bb840863647b P 43999d813e564dc688fad116090c95ca reported cstate change: term changed from 0 to 1, leader changed from <none> to 43999d813e564dc688fad116090c95ca (127.14.177.129). New cstate: current_term: 1 leader_uuid: "43999d813e564dc688fad116090c95ca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } health_report { overall_health: HEALTHY } } }
I20260709 13:23:31.556974 25512 leader_election.cc:290] T 28c0b6fc9bd64d60828d705c6a9ab1e4 P 233e7d29aac04505bf4e65f0a39a0c4c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 43999d813e564dc688fad116090c95ca (127.14.177.129:38155), 3e07d64466934da4bb3c02262b1c2e38 (127.14.177.131:41539)
I20260709 13:23:31.558640 25385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28c0b6fc9bd64d60828d705c6a9ab1e4" candidate_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e07d64466934da4bb3c02262b1c2e38" is_pre_election: true
I20260709 13:23:31.557642 25236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28c0b6fc9bd64d60828d705c6a9ab1e4" candidate_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43999d813e564dc688fad116090c95ca" is_pre_election: true
I20260709 13:23:31.558902 25293 tablet_service.cc:1511] Processing CreateTablet for tablet 9ab90e3e08fb4528b2a47d995e9e26d3 (DEFAULT_TABLE table=test-workload [id=2f3f768873514f53bb58e92ba96f4813]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 13:23:31.559391 25385 raft_consensus.cc:2468] T 28c0b6fc9bd64d60828d705c6a9ab1e4 P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 233e7d29aac04505bf4e65f0a39a0c4c in term 0.
I20260709 13:23:31.560261 25293 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9ab90e3e08fb4528b2a47d995e9e26d3. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:23:31.560354 25276 leader_election.cc:343] T 28c0b6fc9bd64d60828d705c6a9ab1e4 P 233e7d29aac04505bf4e65f0a39a0c4c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 43999d813e564dc688fad116090c95ca (127.14.177.129:38155): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:23:31.560997 25274 leader_election.cc:304] T 28c0b6fc9bd64d60828d705c6a9ab1e4 P 233e7d29aac04505bf4e65f0a39a0c4c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 233e7d29aac04505bf4e65f0a39a0c4c, 3e07d64466934da4bb3c02262b1c2e38; no voters: 43999d813e564dc688fad116090c95ca
I20260709 13:23:31.565286 25538 raft_consensus.cc:493] T d0a0e4d32ce943fbb773bb840863647b P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:23:31.565768 25538 raft_consensus.cc:515] T d0a0e4d32ce943fbb773bb840863647b P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } }
I20260709 13:23:31.567930 25538 leader_election.cc:290] T d0a0e4d32ce943fbb773bb840863647b P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9868dce521294d0b816bba7648f4ae5a (127.14.177.132:33683), 43999d813e564dc688fad116090c95ca (127.14.177.129:38155)
I20260709 13:23:31.568351 25311 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe5a944a342411ea923eb39071ac770" candidate_uuid: "9868dce521294d0b816bba7648f4ae5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" is_pre_election: true
I20260709 13:23:31.569445 25459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0a0e4d32ce943fbb773bb840863647b" candidate_uuid: "3e07d64466934da4bb3c02262b1c2e38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9868dce521294d0b816bba7648f4ae5a" is_pre_election: true
I20260709 13:23:31.570207 25459 raft_consensus.cc:2393] T d0a0e4d32ce943fbb773bb840863647b P 9868dce521294d0b816bba7648f4ae5a [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3e07d64466934da4bb3c02262b1c2e38 in current term 1: Already voted for candidate 43999d813e564dc688fad116090c95ca in this term.
I20260709 13:23:31.572626 25236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0a0e4d32ce943fbb773bb840863647b" candidate_uuid: "3e07d64466934da4bb3c02262b1c2e38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43999d813e564dc688fad116090c95ca" is_pre_election: true
I20260709 13:23:31.561935 25225 tablet_service.cc:1511] Processing CreateTablet for tablet 0ed5edbe32044d1c83b0590dbb899801 (DEFAULT_TABLE table=test-workload [id=2f3f768873514f53bb58e92ba96f4813]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 13:23:31.574213 25350 leader_election.cc:304] T d0a0e4d32ce943fbb773bb840863647b P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3e07d64466934da4bb3c02262b1c2e38; no voters: 43999d813e564dc688fad116090c95ca, 9868dce521294d0b816bba7648f4ae5a
I20260709 13:23:31.574558 25225 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0ed5edbe32044d1c83b0590dbb899801. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:23:31.564035 25512 raft_consensus.cc:2804] T 28c0b6fc9bd64d60828d705c6a9ab1e4 P 233e7d29aac04505bf4e65f0a39a0c4c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:23:31.581189 25512 raft_consensus.cc:493] T 28c0b6fc9bd64d60828d705c6a9ab1e4 P 233e7d29aac04505bf4e65f0a39a0c4c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:23:31.575371 25538 raft_consensus.cc:3060] T d0a0e4d32ce943fbb773bb840863647b P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:23:31.581702 25425 leader_election.cc:343] T 3fe5a944a342411ea923eb39071ac770 P 9868dce521294d0b816bba7648f4ae5a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 233e7d29aac04505bf4e65f0a39a0c4c (127.14.177.130:42957): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:23:31.582285 25425 leader_election.cc:304] T 3fe5a944a342411ea923eb39071ac770 P 9868dce521294d0b816bba7648f4ae5a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9868dce521294d0b816bba7648f4ae5a; no voters: 233e7d29aac04505bf4e65f0a39a0c4c, 3e07d64466934da4bb3c02262b1c2e38
I20260709 13:23:31.583724 25539 raft_consensus.cc:2749] T 3fe5a944a342411ea923eb39071ac770 P 9868dce521294d0b816bba7648f4ae5a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:23:31.581703 25512 raft_consensus.cc:3060] T 28c0b6fc9bd64d60828d705c6a9ab1e4 P 233e7d29aac04505bf4e65f0a39a0c4c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:23:31.580147 25443 tablet_service.cc:1511] Processing CreateTablet for tablet 9ab90e3e08fb4528b2a47d995e9e26d3 (DEFAULT_TABLE table=test-workload [id=2f3f768873514f53bb58e92ba96f4813]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 13:23:31.587392 25222 tablet_service.cc:1511] Processing CreateTablet for tablet 434a54fcee7d4aeeab42f739855c0e8a (DEFAULT_TABLE table=test-workload [id=2f3f768873514f53bb58e92ba96f4813]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 13:23:31.588008 25443 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9ab90e3e08fb4528b2a47d995e9e26d3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:23:31.588781 25222 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 434a54fcee7d4aeeab42f739855c0e8a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:23:31.589740 25298 tablet_service.cc:1511] Processing CreateTablet for tablet 434a54fcee7d4aeeab42f739855c0e8a (DEFAULT_TABLE table=test-workload [id=2f3f768873514f53bb58e92ba96f4813]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 13:23:31.591005 25298 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 434a54fcee7d4aeeab42f739855c0e8a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:23:31.592325 25510 raft_consensus.cc:493] T 441d780c3945463f92f31170f31e37a8 P 43999d813e564dc688fad116090c95ca [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:23:31.592828 25510 raft_consensus.cc:515] T 441d780c3945463f92f31170f31e37a8 P 43999d813e564dc688fad116090c95ca [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } }
I20260709 13:23:31.562983 25504 tablet_bootstrap.cc:492] T 0e71b6f5c69b4b35a8ac51c22c7d5f0d P 3e07d64466934da4bb3c02262b1c2e38: Bootstrap starting.
I20260709 13:23:31.594985 25510 leader_election.cc:290] T 441d780c3945463f92f31170f31e37a8 P 43999d813e564dc688fad116090c95ca [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9868dce521294d0b816bba7648f4ae5a (127.14.177.132:33683), 233e7d29aac04505bf4e65f0a39a0c4c (127.14.177.130:42957)
I20260709 13:23:31.598045 25459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "441d780c3945463f92f31170f31e37a8" candidate_uuid: "43999d813e564dc688fad116090c95ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9868dce521294d0b816bba7648f4ae5a" is_pre_election: true
I20260709 13:23:31.598971 25459 raft_consensus.cc:2468] T 441d780c3945463f92f31170f31e37a8 P 9868dce521294d0b816bba7648f4ae5a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 43999d813e564dc688fad116090c95ca in term 0.
I20260709 13:23:31.598153 25311 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "441d780c3945463f92f31170f31e37a8" candidate_uuid: "43999d813e564dc688fad116090c95ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" is_pre_election: true
W20260709 13:23:31.601243 25202 leader_election.cc:343] T 441d780c3945463f92f31170f31e37a8 P 43999d813e564dc688fad116090c95ca [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 233e7d29aac04505bf4e65f0a39a0c4c (127.14.177.130:42957): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:23:31.602005 25512 raft_consensus.cc:515] T 28c0b6fc9bd64d60828d705c6a9ab1e4 P 233e7d29aac04505bf4e65f0a39a0c4c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } }
I20260709 13:23:31.602756 25199 leader_election.cc:304] T 441d780c3945463f92f31170f31e37a8 P 43999d813e564dc688fad116090c95ca [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 43999d813e564dc688fad116090c95ca, 9868dce521294d0b816bba7648f4ae5a; no voters: 233e7d29aac04505bf4e65f0a39a0c4c
I20260709 13:23:31.604174 25512 leader_election.cc:290] T 28c0b6fc9bd64d60828d705c6a9ab1e4 P 233e7d29aac04505bf4e65f0a39a0c4c [CANDIDATE]: Term 1 election: Requested vote from peers 43999d813e564dc688fad116090c95ca (127.14.177.129:38155), 3e07d64466934da4bb3c02262b1c2e38 (127.14.177.131:41539)
I20260709 13:23:31.604389 25510 raft_consensus.cc:2804] T 441d780c3945463f92f31170f31e37a8 P 43999d813e564dc688fad116090c95ca [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:23:31.605708 25510 raft_consensus.cc:493] T 441d780c3945463f92f31170f31e37a8 P 43999d813e564dc688fad116090c95ca [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:23:31.605743 25236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28c0b6fc9bd64d60828d705c6a9ab1e4" candidate_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43999d813e564dc688fad116090c95ca"
I20260709 13:23:31.606199 25510 raft_consensus.cc:3060] T 441d780c3945463f92f31170f31e37a8 P 43999d813e564dc688fad116090c95ca [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:23:31.606207 25385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28c0b6fc9bd64d60828d705c6a9ab1e4" candidate_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e07d64466934da4bb3c02262b1c2e38"
I20260709 13:23:31.606989 25385 raft_consensus.cc:3060] T 28c0b6fc9bd64d60828d705c6a9ab1e4 P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:23:31.607440 25276 leader_election.cc:343] T 28c0b6fc9bd64d60828d705c6a9ab1e4 P 233e7d29aac04505bf4e65f0a39a0c4c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 43999d813e564dc688fad116090c95ca (127.14.177.129:38155): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:23:31.607913 25508 tablet_bootstrap.cc:654] T b7a891bfc89e4e259f53da6cb16f4989 P 9868dce521294d0b816bba7648f4ae5a: Neither blocks nor log segments found. Creating new log.
I20260709 13:23:31.609278 25503 tablet_bootstrap.cc:492] T e9e342ecda4b46619ce12452b2cf7154 P 233e7d29aac04505bf4e65f0a39a0c4c: No bootstrap required, opened a new log
I20260709 13:23:31.611989 25368 tablet_service.cc:1511] Processing CreateTablet for tablet 9ab90e3e08fb4528b2a47d995e9e26d3 (DEFAULT_TABLE table=test-workload [id=2f3f768873514f53bb58e92ba96f4813]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 13:23:31.613318 25510 raft_consensus.cc:515] T 441d780c3945463f92f31170f31e37a8 P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } }
I20260709 13:23:31.615937 25385 raft_consensus.cc:2468] T 28c0b6fc9bd64d60828d705c6a9ab1e4 P 3e07d64466934da4bb3c02262b1c2e38 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 233e7d29aac04505bf4e65f0a39a0c4c in term 1.
I20260709 13:23:31.618710 25368 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9ab90e3e08fb4528b2a47d995e9e26d3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:23:31.620524 25505 tablet_bootstrap.cc:492] T 0e71b6f5c69b4b35a8ac51c22c7d5f0d P 43999d813e564dc688fad116090c95ca: No bootstrap required, opened a new log
I20260709 13:23:31.620903 25503 ts_tablet_manager.cc:1403] T e9e342ecda4b46619ce12452b2cf7154 P 233e7d29aac04505bf4e65f0a39a0c4c: Time spent bootstrapping tablet: real 0.179s	user 0.019s	sys 0.003s
I20260709 13:23:31.629878 25459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "441d780c3945463f92f31170f31e37a8" candidate_uuid: "43999d813e564dc688fad116090c95ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9868dce521294d0b816bba7648f4ae5a"
I20260709 13:23:31.627585 25371 tablet_service.cc:1511] Processing CreateTablet for tablet 434a54fcee7d4aeeab42f739855c0e8a (DEFAULT_TABLE table=test-workload [id=2f3f768873514f53bb58e92ba96f4813]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 13:23:31.631747 25459 raft_consensus.cc:3060] T 441d780c3945463f92f31170f31e37a8 P 9868dce521294d0b816bba7648f4ae5a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:23:31.628520 25505 ts_tablet_manager.cc:1403] T 0e71b6f5c69b4b35a8ac51c22c7d5f0d P 43999d813e564dc688fad116090c95ca: Time spent bootstrapping tablet: real 0.250s	user 0.020s	sys 0.007s
I20260709 13:23:31.635667 25371 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 434a54fcee7d4aeeab42f739855c0e8a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:23:31.621028 25538 raft_consensus.cc:2749] T d0a0e4d32ce943fbb773bb840863647b P 3e07d64466934da4bb3c02262b1c2e38 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:23:31.637740 25518 raft_consensus.cc:493] T a820954e07a94582ac0613342fcffde5 P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:23:31.638371 25518 raft_consensus.cc:515] T a820954e07a94582ac0613342fcffde5 P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } }
I20260709 13:23:31.638756 25274 leader_election.cc:304] T 28c0b6fc9bd64d60828d705c6a9ab1e4 P 233e7d29aac04505bf4e65f0a39a0c4c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 233e7d29aac04505bf4e65f0a39a0c4c, 3e07d64466934da4bb3c02262b1c2e38; no voters: 43999d813e564dc688fad116090c95ca
I20260709 13:23:31.639012 25311 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "441d780c3945463f92f31170f31e37a8" candidate_uuid: "43999d813e564dc688fad116090c95ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "233e7d29aac04505bf4e65f0a39a0c4c"
I20260709 13:23:31.641714 25518 leader_election.cc:290] T a820954e07a94582ac0613342fcffde5 P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 233e7d29aac04505bf4e65f0a39a0c4c (127.14.177.130:42957), 9868dce521294d0b816bba7648f4ae5a (127.14.177.132:33683)
I20260709 13:23:31.641961 25458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a820954e07a94582ac0613342fcffde5" candidate_uuid: "3e07d64466934da4bb3c02262b1c2e38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9868dce521294d0b816bba7648f4ae5a" is_pre_election: true
W20260709 13:23:31.642282 25202 leader_election.cc:343] T 441d780c3945463f92f31170f31e37a8 P 43999d813e564dc688fad116090c95ca [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 233e7d29aac04505bf4e65f0a39a0c4c (127.14.177.130:42957): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:23:31.641984 25311 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a820954e07a94582ac0613342fcffde5" candidate_uuid: "3e07d64466934da4bb3c02262b1c2e38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" is_pre_election: true
W20260709 13:23:31.643464 25348 leader_election.cc:343] T a820954e07a94582ac0613342fcffde5 P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9868dce521294d0b816bba7648f4ae5a (127.14.177.132:33683): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:23:31.644510 25351 leader_election.cc:343] T a820954e07a94582ac0613342fcffde5 P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 233e7d29aac04505bf4e65f0a39a0c4c (127.14.177.130:42957): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:23:31.644941 25351 leader_election.cc:304] T a820954e07a94582ac0613342fcffde5 P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3e07d64466934da4bb3c02262b1c2e38; no voters: 233e7d29aac04505bf4e65f0a39a0c4c, 9868dce521294d0b816bba7648f4ae5a
I20260709 13:23:31.645360 25512 raft_consensus.cc:2804] T 28c0b6fc9bd64d60828d705c6a9ab1e4 P 233e7d29aac04505bf4e65f0a39a0c4c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:23:31.645069 25505 raft_consensus.cc:359] T 0e71b6f5c69b4b35a8ac51c22c7d5f0d P 43999d813e564dc688fad116090c95ca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } }
I20260709 13:23:31.647919 25367 tablet_service.cc:1511] Processing CreateTablet for tablet 4d837697cd0f402cacf9db89a5d92652 (DEFAULT_TABLE table=test-workload [id=2f3f768873514f53bb58e92ba96f4813]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 13:23:31.645905 25505 raft_consensus.cc:385] T 0e71b6f5c69b4b35a8ac51c22c7d5f0d P 43999d813e564dc688fad116090c95ca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:23:31.649238 25367 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4d837697cd0f402cacf9db89a5d92652. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:23:31.649592 25518 raft_consensus.cc:2749] T a820954e07a94582ac0613342fcffde5 P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:23:31.642969 25293 tablet_service.cc:1511] Processing CreateTablet for tablet b85a238bbc3b4621881603a534c7bd9b (DEFAULT_TABLE table=test-workload [id=2f3f768873514f53bb58e92ba96f4813]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260709 13:23:31.652438 25293 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b85a238bbc3b4621881603a534c7bd9b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:23:31.652927 25503 raft_consensus.cc:359] T e9e342ecda4b46619ce12452b2cf7154 P 233e7d29aac04505bf4e65f0a39a0c4c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } }
I20260709 13:23:31.626327 25133 catalog_manager.cc:5697] T b7a891bfc89e4e259f53da6cb16f4989 P 3e07d64466934da4bb3c02262b1c2e38 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3e07d64466934da4bb3c02262b1c2e38 (127.14.177.131). New cstate: current_term: 1 leader_uuid: "3e07d64466934da4bb3c02262b1c2e38" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } health_report { overall_health: UNKNOWN } } }
I20260709 13:23:31.656245 25225 tablet_service.cc:1511] Processing CreateTablet for tablet b85a238bbc3b4621881603a534c7bd9b (DEFAULT_TABLE table=test-workload [id=2f3f768873514f53bb58e92ba96f4813]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260709 13:23:31.660756 25225 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b85a238bbc3b4621881603a534c7bd9b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:23:31.637816 25510 leader_election.cc:290] T 441d780c3945463f92f31170f31e37a8 P 43999d813e564dc688fad116090c95ca [CANDIDATE]: Term 1 election: Requested vote from peers 9868dce521294d0b816bba7648f4ae5a (127.14.177.132:33683), 233e7d29aac04505bf4e65f0a39a0c4c (127.14.177.130:42957)
I20260709 13:23:31.655380 25503 raft_consensus.cc:385] T e9e342ecda4b46619ce12452b2cf7154 P 233e7d29aac04505bf4e65f0a39a0c4c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:23:31.655565 25505 raft_consensus.cc:740] T 0e71b6f5c69b4b35a8ac51c22c7d5f0d P 43999d813e564dc688fad116090c95ca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43999d813e564dc688fad116090c95ca, State: Initialized, Role: FOLLOWER
I20260709 13:23:31.651150 25459 raft_consensus.cc:2468] T 441d780c3945463f92f31170f31e37a8 P 9868dce521294d0b816bba7648f4ae5a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 43999d813e564dc688fad116090c95ca in term 1.
I20260709 13:23:31.674957 25441 tablet_service.cc:1511] Processing CreateTablet for tablet 4d837697cd0f402cacf9db89a5d92652 (DEFAULT_TABLE table=test-workload [id=2f3f768873514f53bb58e92ba96f4813]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 13:23:31.653461 25448 tablet_service.cc:1511] Processing CreateTablet for tablet b85a238bbc3b4621881603a534c7bd9b (DEFAULT_TABLE table=test-workload [id=2f3f768873514f53bb58e92ba96f4813]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260709 13:23:31.685508 25199 leader_election.cc:304] T 441d780c3945463f92f31170f31e37a8 P 43999d813e564dc688fad116090c95ca [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 43999d813e564dc688fad116090c95ca, 9868dce521294d0b816bba7648f4ae5a; no voters: 233e7d29aac04505bf4e65f0a39a0c4c
I20260709 13:23:31.686712 25510 raft_consensus.cc:2804] T 441d780c3945463f92f31170f31e37a8 P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:23:31.687126 25441 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4d837697cd0f402cacf9db89a5d92652. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:23:31.687513 25510 raft_consensus.cc:697] T 441d780c3945463f92f31170f31e37a8 P 43999d813e564dc688fad116090c95ca [term 1 LEADER]: Becoming Leader. State: Replica: 43999d813e564dc688fad116090c95ca, State: Running, Role: LEADER
I20260709 13:23:31.690363 25510 consensus_queue.cc:237] T 441d780c3945463f92f31170f31e37a8 P 43999d813e564dc688fad116090c95ca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } }
I20260709 13:23:31.687705 25505 consensus_queue.cc:260] T 0e71b6f5c69b4b35a8ac51c22c7d5f0d P 43999d813e564dc688fad116090c95ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } }
I20260709 13:23:31.694957 25505 ts_tablet_manager.cc:1434] T 0e71b6f5c69b4b35a8ac51c22c7d5f0d P 43999d813e564dc688fad116090c95ca: Time spent starting tablet: real 0.062s	user 0.006s	sys 0.001s
I20260709 13:23:31.695698 25508 tablet_bootstrap.cc:492] T b7a891bfc89e4e259f53da6cb16f4989 P 9868dce521294d0b816bba7648f4ae5a: No bootstrap required, opened a new log
I20260709 13:23:31.696225 25505 tablet_bootstrap.cc:492] T 579da2d6aabb4ae9a2b1802e1358068b P 43999d813e564dc688fad116090c95ca: Bootstrap starting.
I20260709 13:23:31.696504 25508 ts_tablet_manager.cc:1403] T b7a891bfc89e4e259f53da6cb16f4989 P 9868dce521294d0b816bba7648f4ae5a: Time spent bootstrapping tablet: real 0.157s	user 0.015s	sys 0.001s
I20260709 13:23:31.689056 25220 tablet_service.cc:1511] Processing CreateTablet for tablet 4d837697cd0f402cacf9db89a5d92652 (DEFAULT_TABLE table=test-workload [id=2f3f768873514f53bb58e92ba96f4813]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 13:23:31.700313 25220 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4d837697cd0f402cacf9db89a5d92652. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:23:31.705519 25508 raft_consensus.cc:359] T b7a891bfc89e4e259f53da6cb16f4989 P 9868dce521294d0b816bba7648f4ae5a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } }
I20260709 13:23:31.706395 25508 raft_consensus.cc:385] T b7a891bfc89e4e259f53da6cb16f4989 P 9868dce521294d0b816bba7648f4ae5a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:23:31.706699 25508 raft_consensus.cc:740] T b7a891bfc89e4e259f53da6cb16f4989 P 9868dce521294d0b816bba7648f4ae5a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9868dce521294d0b816bba7648f4ae5a, State: Initialized, Role: FOLLOWER
I20260709 13:23:31.707775 25448 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b85a238bbc3b4621881603a534c7bd9b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:23:31.678946 25503 raft_consensus.cc:740] T e9e342ecda4b46619ce12452b2cf7154 P 233e7d29aac04505bf4e65f0a39a0c4c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 233e7d29aac04505bf4e65f0a39a0c4c, State: Initialized, Role: FOLLOWER
I20260709 13:23:31.720680 25298 tablet_service.cc:1511] Processing CreateTablet for tablet cfad575709c943cba40ccab278350770 (DEFAULT_TABLE table=test-workload [id=2f3f768873514f53bb58e92ba96f4813]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260709 13:23:31.721581 25298 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cfad575709c943cba40ccab278350770. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:23:31.720062 25512 raft_consensus.cc:697] T 28c0b6fc9bd64d60828d705c6a9ab1e4 P 233e7d29aac04505bf4e65f0a39a0c4c [term 1 LEADER]: Becoming Leader. State: Replica: 233e7d29aac04505bf4e65f0a39a0c4c, State: Running, Role: LEADER
I20260709 13:23:31.722826 25508 consensus_queue.cc:260] T b7a891bfc89e4e259f53da6cb16f4989 P 9868dce521294d0b816bba7648f4ae5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } }
I20260709 13:23:31.729106 25367 tablet_service.cc:1511] Processing CreateTablet for tablet cfad575709c943cba40ccab278350770 (DEFAULT_TABLE table=test-workload [id=2f3f768873514f53bb58e92ba96f4813]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260709 13:23:31.731474 25367 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cfad575709c943cba40ccab278350770. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:23:31.732532 25443 tablet_service.cc:1511] Processing CreateTablet for tablet cfad575709c943cba40ccab278350770 (DEFAULT_TABLE table=test-workload [id=2f3f768873514f53bb58e92ba96f4813]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260709 13:23:31.740435 25443 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cfad575709c943cba40ccab278350770. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:23:31.744932 25508 ts_tablet_manager.cc:1434] T b7a891bfc89e4e259f53da6cb16f4989 P 9868dce521294d0b816bba7648f4ae5a: Time spent starting tablet: real 0.048s	user 0.007s	sys 0.000s
I20260709 13:23:31.746049 25508 tablet_bootstrap.cc:492] T e9e342ecda4b46619ce12452b2cf7154 P 9868dce521294d0b816bba7648f4ae5a: Bootstrap starting.
I20260709 13:23:31.720041 25503 consensus_queue.cc:260] T e9e342ecda4b46619ce12452b2cf7154 P 233e7d29aac04505bf4e65f0a39a0c4c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } }
I20260709 13:23:31.748023 25225 tablet_service.cc:1511] Processing CreateTablet for tablet b155b866f10547fd8239579a81ecf5f9 (DEFAULT_TABLE table=test-workload [id=2f3f768873514f53bb58e92ba96f4813]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 13:23:31.748991 25503 ts_tablet_manager.cc:1434] T e9e342ecda4b46619ce12452b2cf7154 P 233e7d29aac04505bf4e65f0a39a0c4c: Time spent starting tablet: real 0.120s	user 0.007s	sys 0.001s
I20260709 13:23:31.749241 25225 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b155b866f10547fd8239579a81ecf5f9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:23:31.750025 25503 tablet_bootstrap.cc:492] T 0d29b499fa4c41f1b974782cd0c9c1c2 P 233e7d29aac04505bf4e65f0a39a0c4c: Bootstrap starting.
I20260709 13:23:31.750715 25222 tablet_service.cc:1511] Processing CreateTablet for tablet 8ef061028f38450e83324ac78cdaa14c (DEFAULT_TABLE table=test-workload [id=2f3f768873514f53bb58e92ba96f4813]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260709 13:23:31.752007 25222 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8ef061028f38450e83324ac78cdaa14c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:23:31.752683 25294 tablet_service.cc:1511] Processing CreateTablet for tablet 8ef061028f38450e83324ac78cdaa14c (DEFAULT_TABLE table=test-workload [id=2f3f768873514f53bb58e92ba96f4813]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260709 13:23:31.754307 25294 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8ef061028f38450e83324ac78cdaa14c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:23:31.753708 25512 consensus_queue.cc:237] T 28c0b6fc9bd64d60828d705c6a9ab1e4 P 233e7d29aac04505bf4e65f0a39a0c4c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } }
I20260709 13:23:31.747000 25300 tablet_service.cc:1511] Processing CreateTablet for tablet b155b866f10547fd8239579a81ecf5f9 (DEFAULT_TABLE table=test-workload [id=2f3f768873514f53bb58e92ba96f4813]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 13:23:31.763126 25300 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b155b866f10547fd8239579a81ecf5f9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:23:31.786341 25551 raft_consensus.cc:493] T e9e342ecda4b46619ce12452b2cf7154 P 233e7d29aac04505bf4e65f0a39a0c4c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:23:31.787050 25551 raft_consensus.cc:515] T e9e342ecda4b46619ce12452b2cf7154 P 233e7d29aac04505bf4e65f0a39a0c4c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } }
I20260709 13:23:31.800676 25370 tablet_service.cc:1511] Processing CreateTablet for tablet b155b866f10547fd8239579a81ecf5f9 (DEFAULT_TABLE table=test-workload [id=2f3f768873514f53bb58e92ba96f4813]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 13:23:31.802621 25370 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b155b866f10547fd8239579a81ecf5f9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:23:31.816828 25505 tablet_bootstrap.cc:654] T 579da2d6aabb4ae9a2b1802e1358068b P 43999d813e564dc688fad116090c95ca: Neither blocks nor log segments found. Creating new log.
I20260709 13:23:31.822378 25504 tablet_bootstrap.cc:654] T 0e71b6f5c69b4b35a8ac51c22c7d5f0d P 3e07d64466934da4bb3c02262b1c2e38: Neither blocks nor log segments found. Creating new log.
I20260709 13:23:31.826540 25459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9e342ecda4b46619ce12452b2cf7154" candidate_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9868dce521294d0b816bba7648f4ae5a" is_pre_election: true
W20260709 13:23:31.827972 25274 leader_election.cc:343] T e9e342ecda4b46619ce12452b2cf7154 P 233e7d29aac04505bf4e65f0a39a0c4c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9868dce521294d0b816bba7648f4ae5a (127.14.177.132:33683): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:23:31.831782 25551 leader_election.cc:290] T e9e342ecda4b46619ce12452b2cf7154 P 233e7d29aac04505bf4e65f0a39a0c4c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9868dce521294d0b816bba7648f4ae5a (127.14.177.132:33683), 43999d813e564dc688fad116090c95ca (127.14.177.129:38155)
I20260709 13:23:31.837198 25236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9e342ecda4b46619ce12452b2cf7154" candidate_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43999d813e564dc688fad116090c95ca" is_pre_election: true
I20260709 13:23:31.837502 25446 tablet_service.cc:1511] Processing CreateTablet for tablet 8ef061028f38450e83324ac78cdaa14c (DEFAULT_TABLE table=test-workload [id=2f3f768873514f53bb58e92ba96f4813]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
W20260709 13:23:31.839509 25276 leader_election.cc:343] T e9e342ecda4b46619ce12452b2cf7154 P 233e7d29aac04505bf4e65f0a39a0c4c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 43999d813e564dc688fad116090c95ca (127.14.177.129:38155): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:23:31.840152 25276 leader_election.cc:304] T e9e342ecda4b46619ce12452b2cf7154 P 233e7d29aac04505bf4e65f0a39a0c4c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 233e7d29aac04505bf4e65f0a39a0c4c; no voters: 43999d813e564dc688fad116090c95ca, 9868dce521294d0b816bba7648f4ae5a
I20260709 13:23:31.841629 25551 raft_consensus.cc:2749] T e9e342ecda4b46619ce12452b2cf7154 P 233e7d29aac04505bf4e65f0a39a0c4c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:23:31.868207 25442 tablet_service.cc:1511] Processing CreateTablet for tablet b9d550d881b04735bbf89751952c09ec (DEFAULT_TABLE table=test-workload [id=2f3f768873514f53bb58e92ba96f4813]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260709 13:23:31.868363 25367 tablet_service.cc:1511] Processing CreateTablet for tablet b9d550d881b04735bbf89751952c09ec (DEFAULT_TABLE table=test-workload [id=2f3f768873514f53bb58e92ba96f4813]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260709 13:23:31.869452 25446 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8ef061028f38450e83324ac78cdaa14c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:23:31.869897 25367 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b9d550d881b04735bbf89751952c09ec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:23:31.876459 25222 tablet_service.cc:1511] Processing CreateTablet for tablet b9d550d881b04735bbf89751952c09ec (DEFAULT_TABLE table=test-workload [id=2f3f768873514f53bb58e92ba96f4813]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260709 13:23:31.878062 25222 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b9d550d881b04735bbf89751952c09ec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:23:31.905687 25442 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b9d550d881b04735bbf89751952c09ec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:23:31.915814 25293 tablet_service.cc:1511] Processing CreateTablet for tablet e311eb90bad44c80a131b72edcd22b06 (DEFAULT_TABLE table=test-workload [id=2f3f768873514f53bb58e92ba96f4813]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 13:23:31.917193 25293 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e311eb90bad44c80a131b72edcd22b06. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:23:31.921449 25508 tablet_bootstrap.cc:654] T e9e342ecda4b46619ce12452b2cf7154 P 9868dce521294d0b816bba7648f4ae5a: Neither blocks nor log segments found. Creating new log.
I20260709 13:23:31.921912 25505 tablet_bootstrap.cc:492] T 579da2d6aabb4ae9a2b1802e1358068b P 43999d813e564dc688fad116090c95ca: No bootstrap required, opened a new log
I20260709 13:23:31.925665 25503 tablet_bootstrap.cc:654] T 0d29b499fa4c41f1b974782cd0c9c1c2 P 233e7d29aac04505bf4e65f0a39a0c4c: Neither blocks nor log segments found. Creating new log.
I20260709 13:23:31.922689 25505 ts_tablet_manager.cc:1403] T 579da2d6aabb4ae9a2b1802e1358068b P 43999d813e564dc688fad116090c95ca: Time spent bootstrapping tablet: real 0.227s	user 0.004s	sys 0.032s
I20260709 13:23:31.937621 25134 catalog_manager.cc:5697] T 441d780c3945463f92f31170f31e37a8 P 43999d813e564dc688fad116090c95ca reported cstate change: term changed from 0 to 1, leader changed from <none> to 43999d813e564dc688fad116090c95ca (127.14.177.129). New cstate: current_term: 1 leader_uuid: "43999d813e564dc688fad116090c95ca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } health_report { overall_health: UNKNOWN } } }
I20260709 13:23:31.940995 25368 tablet_service.cc:1511] Processing CreateTablet for tablet e311eb90bad44c80a131b72edcd22b06 (DEFAULT_TABLE table=test-workload [id=2f3f768873514f53bb58e92ba96f4813]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 13:23:31.941054 25441 tablet_service.cc:1511] Processing CreateTablet for tablet e311eb90bad44c80a131b72edcd22b06 (DEFAULT_TABLE table=test-workload [id=2f3f768873514f53bb58e92ba96f4813]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 13:23:31.942564 25368 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e311eb90bad44c80a131b72edcd22b06. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:23:31.942564 25441 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e311eb90bad44c80a131b72edcd22b06. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:23:31.927385 25510 consensus_queue.cc:1048] T d0a0e4d32ce943fbb773bb840863647b P 43999d813e564dc688fad116090c95ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.016s
I20260709 13:23:31.928747 25535 raft_consensus.cc:493] T 0e71b6f5c69b4b35a8ac51c22c7d5f0d P 43999d813e564dc688fad116090c95ca [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:23:31.943847 25535 raft_consensus.cc:515] T 0e71b6f5c69b4b35a8ac51c22c7d5f0d P 43999d813e564dc688fad116090c95ca [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } }
I20260709 13:23:31.945894 25535 leader_election.cc:290] T 0e71b6f5c69b4b35a8ac51c22c7d5f0d P 43999d813e564dc688fad116090c95ca [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3e07d64466934da4bb3c02262b1c2e38 (127.14.177.131:41539), 9868dce521294d0b816bba7648f4ae5a (127.14.177.132:33683)
I20260709 13:23:31.949357 25504 tablet_bootstrap.cc:492] T 0e71b6f5c69b4b35a8ac51c22c7d5f0d P 3e07d64466934da4bb3c02262b1c2e38: No bootstrap required, opened a new log
I20260709 13:23:31.949913 25504 ts_tablet_manager.cc:1403] T 0e71b6f5c69b4b35a8ac51c22c7d5f0d P 3e07d64466934da4bb3c02262b1c2e38: Time spent bootstrapping tablet: real 0.406s	user 0.024s	sys 0.041s
I20260709 13:23:31.953171 25504 raft_consensus.cc:359] T 0e71b6f5c69b4b35a8ac51c22c7d5f0d P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } }
I20260709 13:23:31.953939 25504 raft_consensus.cc:385] T 0e71b6f5c69b4b35a8ac51c22c7d5f0d P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:23:31.954226 25504 raft_consensus.cc:740] T 0e71b6f5c69b4b35a8ac51c22c7d5f0d P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e07d64466934da4bb3c02262b1c2e38, State: Initialized, Role: FOLLOWER
I20260709 13:23:31.957257 25504 consensus_queue.cc:260] T 0e71b6f5c69b4b35a8ac51c22c7d5f0d P 3e07d64466934da4bb3c02262b1c2e38 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } }
I20260709 13:23:31.959524 25504 ts_tablet_manager.cc:1434] T 0e71b6f5c69b4b35a8ac51c22c7d5f0d P 3e07d64466934da4bb3c02262b1c2e38: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.002s
I20260709 13:23:31.960582 25504 tablet_bootstrap.cc:492] T 467a6b42ca6548269c12dddde696f7af P 3e07d64466934da4bb3c02262b1c2e38: Bootstrap starting.
I20260709 13:23:31.968211 25504 tablet_bootstrap.cc:654] T 467a6b42ca6548269c12dddde696f7af P 3e07d64466934da4bb3c02262b1c2e38: Neither blocks nor log segments found. Creating new log.
I20260709 13:23:31.969559 25300 tablet_service.cc:1511] Processing CreateTablet for tablet 72b56266e8a54776b9113c7a786e1952 (DEFAULT_TABLE table=test-workload [id=2f3f768873514f53bb58e92ba96f4813]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 13:23:31.970902 25300 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 72b56266e8a54776b9113c7a786e1952. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:23:31.970615 25367 tablet_service.cc:1511] Processing CreateTablet for tablet 72b56266e8a54776b9113c7a786e1952 (DEFAULT_TABLE table=test-workload [id=2f3f768873514f53bb58e92ba96f4813]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 13:23:31.972141 25367 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 72b56266e8a54776b9113c7a786e1952. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:23:31.978320 25222 tablet_service.cc:1511] Processing CreateTablet for tablet 72b56266e8a54776b9113c7a786e1952 (DEFAULT_TABLE table=test-workload [id=2f3f768873514f53bb58e92ba96f4813]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 13:23:31.979853 25222 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 72b56266e8a54776b9113c7a786e1952. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:23:31.988616 25459 raft_consensus.cc:3060] T b7a891bfc89e4e259f53da6cb16f4989 P 9868dce521294d0b816bba7648f4ae5a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:23:31.993284 25505 raft_consensus.cc:359] T 579da2d6aabb4ae9a2b1802e1358068b P 43999d813e564dc688fad116090c95ca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } }
I20260709 13:23:31.997507 25538 consensus_queue.cc:1048] T b7a891bfc89e4e259f53da6cb16f4989 P 3e07d64466934da4bb3c02262b1c2e38 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:23:32.002653 25385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e71b6f5c69b4b35a8ac51c22c7d5f0d" candidate_uuid: "43999d813e564dc688fad116090c95ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e07d64466934da4bb3c02262b1c2e38" is_pre_election: true
I20260709 13:23:32.002758 25225 tablet_service.cc:1511] Processing CreateTablet for tablet fa0da2c845dc4059968332f4db2e0aa6 (DEFAULT_TABLE table=test-workload [id=2f3f768873514f53bb58e92ba96f4813]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260709 13:23:32.003487 25385 raft_consensus.cc:2468] T 0e71b6f5c69b4b35a8ac51c22c7d5f0d P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 43999d813e564dc688fad116090c95ca in term 0.
I20260709 13:23:32.004139 25225 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fa0da2c845dc4059968332f4db2e0aa6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:23:32.003465 25443 tablet_service.cc:1511] Processing CreateTablet for tablet fa0da2c845dc4059968332f4db2e0aa6 (DEFAULT_TABLE table=test-workload [id=2f3f768873514f53bb58e92ba96f4813]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260709 13:23:32.004981 25199 leader_election.cc:304] T 0e71b6f5c69b4b35a8ac51c22c7d5f0d P 43999d813e564dc688fad116090c95ca [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e07d64466934da4bb3c02262b1c2e38, 43999d813e564dc688fad116090c95ca; no voters: 
W20260709 13:23:32.005508 25182 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:23:32.005838 25535 consensus_queue.cc:1048] T d0a0e4d32ce943fbb773bb840863647b P 43999d813e564dc688fad116090c95ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:23:32.006722 25456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e71b6f5c69b4b35a8ac51c22c7d5f0d" candidate_uuid: "43999d813e564dc688fad116090c95ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9868dce521294d0b816bba7648f4ae5a" is_pre_election: true
I20260709 13:23:31.986104 25133 catalog_manager.cc:5697] T 28c0b6fc9bd64d60828d705c6a9ab1e4 P 233e7d29aac04505bf4e65f0a39a0c4c reported cstate change: term changed from 0 to 1, leader changed from <none> to 233e7d29aac04505bf4e65f0a39a0c4c (127.14.177.130). New cstate: current_term: 1 leader_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } health_report { overall_health: UNKNOWN } } }
I20260709 13:23:32.007429 25535 raft_consensus.cc:2804] T 0e71b6f5c69b4b35a8ac51c22c7d5f0d P 43999d813e564dc688fad116090c95ca [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:23:32.008462 25535 raft_consensus.cc:493] T 0e71b6f5c69b4b35a8ac51c22c7d5f0d P 43999d813e564dc688fad116090c95ca [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:23:32.008893 25535 raft_consensus.cc:3060] T 0e71b6f5c69b4b35a8ac51c22c7d5f0d P 43999d813e564dc688fad116090c95ca [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:23:32.005899 25443 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fa0da2c845dc4059968332f4db2e0aa6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:23:31.994130 25505 raft_consensus.cc:385] T 579da2d6aabb4ae9a2b1802e1358068b P 43999d813e564dc688fad116090c95ca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:23:32.019651 25505 raft_consensus.cc:740] T 579da2d6aabb4ae9a2b1802e1358068b P 43999d813e564dc688fad116090c95ca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43999d813e564dc688fad116090c95ca, State: Initialized, Role: FOLLOWER
I20260709 13:23:32.044394 25505 consensus_queue.cc:260] T 579da2d6aabb4ae9a2b1802e1358068b P 43999d813e564dc688fad116090c95ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } }
W20260709 13:23:32.020221 25199 leader_election.cc:343] T 0e71b6f5c69b4b35a8ac51c22c7d5f0d P 43999d813e564dc688fad116090c95ca [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9868dce521294d0b816bba7648f4ae5a (127.14.177.132:33683): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:23:32.063978 25505 ts_tablet_manager.cc:1434] T 579da2d6aabb4ae9a2b1802e1358068b P 43999d813e564dc688fad116090c95ca: Time spent starting tablet: real 0.128s	user 0.004s	sys 0.005s
I20260709 13:23:32.066627 25293 tablet_service.cc:1511] Processing CreateTablet for tablet fa0da2c845dc4059968332f4db2e0aa6 (DEFAULT_TABLE table=test-workload [id=2f3f768873514f53bb58e92ba96f4813]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260709 13:23:32.067951 25293 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fa0da2c845dc4059968332f4db2e0aa6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:23:32.073040 25370 tablet_service.cc:1511] Processing CreateTablet for tablet 5c7e7acbb09f4d49828fa652ca6abaac (DEFAULT_TABLE table=test-workload [id=2f3f768873514f53bb58e92ba96f4813]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260709 13:23:32.082620 25225 tablet_service.cc:1511] Processing CreateTablet for tablet 5c7e7acbb09f4d49828fa652ca6abaac (DEFAULT_TABLE table=test-workload [id=2f3f768873514f53bb58e92ba96f4813]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260709 13:23:32.085371 25294 tablet_service.cc:1511] Processing CreateTablet for tablet 55e8ed3aee404ce698d0626a87bd03aa (DEFAULT_TABLE table=test-workload [id=2f3f768873514f53bb58e92ba96f4813]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260709 13:23:32.086732 25294 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 55e8ed3aee404ce698d0626a87bd03aa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:23:32.086790 25370 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5c7e7acbb09f4d49828fa652ca6abaac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:23:32.088690 25448 tablet_service.cc:1511] Processing CreateTablet for tablet 5c7e7acbb09f4d49828fa652ca6abaac (DEFAULT_TABLE table=test-workload [id=2f3f768873514f53bb58e92ba96f4813]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260709 13:23:32.089252 25508 tablet_bootstrap.cc:492] T e9e342ecda4b46619ce12452b2cf7154 P 9868dce521294d0b816bba7648f4ae5a: No bootstrap required, opened a new log
I20260709 13:23:32.089886 25508 ts_tablet_manager.cc:1403] T e9e342ecda4b46619ce12452b2cf7154 P 9868dce521294d0b816bba7648f4ae5a: Time spent bootstrapping tablet: real 0.344s	user 0.017s	sys 0.022s
I20260709 13:23:32.090071 25448 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5c7e7acbb09f4d49828fa652ca6abaac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:23:32.092782 25505 tablet_bootstrap.cc:492] T d053b9ae38de4173a0aded4da494bd98 P 43999d813e564dc688fad116090c95ca: Bootstrap starting.
I20260709 13:23:32.093360 25225 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5c7e7acbb09f4d49828fa652ca6abaac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:23:32.079546 25535 raft_consensus.cc:515] T 0e71b6f5c69b4b35a8ac51c22c7d5f0d P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } }
I20260709 13:23:32.094097 25504 tablet_bootstrap.cc:492] T 467a6b42ca6548269c12dddde696f7af P 3e07d64466934da4bb3c02262b1c2e38: No bootstrap required, opened a new log
I20260709 13:23:32.094705 25504 ts_tablet_manager.cc:1403] T 467a6b42ca6548269c12dddde696f7af P 3e07d64466934da4bb3c02262b1c2e38: Time spent bootstrapping tablet: real 0.134s	user 0.011s	sys 0.018s
I20260709 13:23:32.098212 25504 raft_consensus.cc:359] T 467a6b42ca6548269c12dddde696f7af P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } }
I20260709 13:23:32.098631 25384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e71b6f5c69b4b35a8ac51c22c7d5f0d" candidate_uuid: "43999d813e564dc688fad116090c95ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e07d64466934da4bb3c02262b1c2e38"
I20260709 13:23:32.099308 25384 raft_consensus.cc:3060] T 0e71b6f5c69b4b35a8ac51c22c7d5f0d P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:23:32.098978 25504 raft_consensus.cc:385] T 467a6b42ca6548269c12dddde696f7af P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:23:32.103551 25371 tablet_service.cc:1511] Processing CreateTablet for tablet 55e8ed3aee404ce698d0626a87bd03aa (DEFAULT_TABLE table=test-workload [id=2f3f768873514f53bb58e92ba96f4813]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260709 13:23:32.110913 25459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e71b6f5c69b4b35a8ac51c22c7d5f0d" candidate_uuid: "43999d813e564dc688fad116090c95ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9868dce521294d0b816bba7648f4ae5a"
I20260709 13:23:32.111769 25371 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 55e8ed3aee404ce698d0626a87bd03aa. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:23:32.112515 25199 leader_election.cc:343] T 0e71b6f5c69b4b35a8ac51c22c7d5f0d P 43999d813e564dc688fad116090c95ca [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 9868dce521294d0b816bba7648f4ae5a (127.14.177.132:33683): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:23:32.112766 25503 tablet_bootstrap.cc:492] T 0d29b499fa4c41f1b974782cd0c9c1c2 P 233e7d29aac04505bf4e65f0a39a0c4c: No bootstrap required, opened a new log
I20260709 13:23:32.113304 25503 ts_tablet_manager.cc:1403] T 0d29b499fa4c41f1b974782cd0c9c1c2 P 233e7d29aac04505bf4e65f0a39a0c4c: Time spent bootstrapping tablet: real 0.364s	user 0.019s	sys 0.007s
I20260709 13:23:32.113627 25535 leader_election.cc:290] T 0e71b6f5c69b4b35a8ac51c22c7d5f0d P 43999d813e564dc688fad116090c95ca [CANDIDATE]: Term 1 election: Requested vote from peers 3e07d64466934da4bb3c02262b1c2e38 (127.14.177.131:41539), 9868dce521294d0b816bba7648f4ae5a (127.14.177.132:33683)
I20260709 13:23:32.110396 25446 tablet_service.cc:1511] Processing CreateTablet for tablet 55e8ed3aee404ce698d0626a87bd03aa (DEFAULT_TABLE table=test-workload [id=2f3f768873514f53bb58e92ba96f4813]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260709 13:23:32.117771 25446 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 55e8ed3aee404ce698d0626a87bd03aa. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:23:32.118683 25202 consensus_peers.cc:597] T 441d780c3945463f92f31170f31e37a8 P 43999d813e564dc688fad116090c95ca -> Peer 233e7d29aac04505bf4e65f0a39a0c4c (127.14.177.130:42957): Couldn't send request to peer 233e7d29aac04505bf4e65f0a39a0c4c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:23:32.120800 25503 raft_consensus.cc:359] T 0d29b499fa4c41f1b974782cd0c9c1c2 P 233e7d29aac04505bf4e65f0a39a0c4c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } }
I20260709 13:23:32.121613 25503 raft_consensus.cc:385] T 0d29b499fa4c41f1b974782cd0c9c1c2 P 233e7d29aac04505bf4e65f0a39a0c4c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:23:32.116034 25222 tablet_service.cc:1511] Processing CreateTablet for tablet 03e9bfbd2f774476a4a377367796a2e8 (DEFAULT_TABLE table=test-workload [id=2f3f768873514f53bb58e92ba96f4813]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260709 13:23:32.121901 25503 raft_consensus.cc:740] T 0d29b499fa4c41f1b974782cd0c9c1c2 P 233e7d29aac04505bf4e65f0a39a0c4c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 233e7d29aac04505bf4e65f0a39a0c4c, State: Initialized, Role: FOLLOWER
I20260709 13:23:32.122571 25222 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 03e9bfbd2f774476a4a377367796a2e8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:23:32.125130 25504 raft_consensus.cc:740] T 467a6b42ca6548269c12dddde696f7af P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e07d64466934da4bb3c02262b1c2e38, State: Initialized, Role: FOLLOWER
I20260709 13:23:32.125653 25505 tablet_bootstrap.cc:654] T d053b9ae38de4173a0aded4da494bd98 P 43999d813e564dc688fad116090c95ca: Neither blocks nor log segments found. Creating new log.
I20260709 13:23:32.126008 25504 consensus_queue.cc:260] T 467a6b42ca6548269c12dddde696f7af P 3e07d64466934da4bb3c02262b1c2e38 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } }
I20260709 13:23:32.128901 25293 tablet_service.cc:1511] Processing CreateTablet for tablet 03e9bfbd2f774476a4a377367796a2e8 (DEFAULT_TABLE table=test-workload [id=2f3f768873514f53bb58e92ba96f4813]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260709 13:23:32.132169 25293 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 03e9bfbd2f774476a4a377367796a2e8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:23:32.139549 25384 raft_consensus.cc:2468] T 0e71b6f5c69b4b35a8ac51c22c7d5f0d P 3e07d64466934da4bb3c02262b1c2e38 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 43999d813e564dc688fad116090c95ca in term 1.
I20260709 13:23:32.141647 25518 raft_consensus.cc:493] T 467a6b42ca6548269c12dddde696f7af P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:23:32.147583 25518 raft_consensus.cc:515] T 467a6b42ca6548269c12dddde696f7af P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } }
I20260709 13:23:32.150146 25518 leader_election.cc:290] T 467a6b42ca6548269c12dddde696f7af P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 233e7d29aac04505bf4e65f0a39a0c4c (127.14.177.130:42957), 9868dce521294d0b816bba7648f4ae5a (127.14.177.132:33683)
I20260709 13:23:32.152177 25311 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "467a6b42ca6548269c12dddde696f7af" candidate_uuid: "3e07d64466934da4bb3c02262b1c2e38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" is_pre_election: true
W20260709 13:23:32.153712 25351 leader_election.cc:343] T 467a6b42ca6548269c12dddde696f7af P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 233e7d29aac04505bf4e65f0a39a0c4c (127.14.177.130:42957): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:23:32.092970 25508 raft_consensus.cc:359] T e9e342ecda4b46619ce12452b2cf7154 P 9868dce521294d0b816bba7648f4ae5a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } }
I20260709 13:23:32.153934 25442 tablet_service.cc:1511] Processing CreateTablet for tablet 11949f56648c47e0a43c3140af99a304 (DEFAULT_TABLE table=test-workload [id=2f3f768873514f53bb58e92ba96f4813]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260709 13:23:32.154495 25508 raft_consensus.cc:385] T e9e342ecda4b46619ce12452b2cf7154 P 9868dce521294d0b816bba7648f4ae5a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:23:32.154843 25508 raft_consensus.cc:740] T e9e342ecda4b46619ce12452b2cf7154 P 9868dce521294d0b816bba7648f4ae5a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9868dce521294d0b816bba7648f4ae5a, State: Initialized, Role: FOLLOWER
I20260709 13:23:32.155306 25442 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 11949f56648c47e0a43c3140af99a304. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:23:32.156736 25367 tablet_service.cc:1511] Processing CreateTablet for tablet 03e9bfbd2f774476a4a377367796a2e8 (DEFAULT_TABLE table=test-workload [id=2f3f768873514f53bb58e92ba96f4813]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260709 13:23:32.157761 25367 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 03e9bfbd2f774476a4a377367796a2e8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:23:32.122661 25503 consensus_queue.cc:260] T 0d29b499fa4c41f1b974782cd0c9c1c2 P 233e7d29aac04505bf4e65f0a39a0c4c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } }
I20260709 13:23:32.160660 25503 ts_tablet_manager.cc:1434] T 0d29b499fa4c41f1b974782cd0c9c1c2 P 233e7d29aac04505bf4e65f0a39a0c4c: Time spent starting tablet: real 0.047s	user 0.004s	sys 0.004s
I20260709 13:23:32.162842 25220 tablet_service.cc:1511] Processing CreateTablet for tablet 11949f56648c47e0a43c3140af99a304 (DEFAULT_TABLE table=test-workload [id=2f3f768873514f53bb58e92ba96f4813]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260709 13:23:32.164399 25220 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 11949f56648c47e0a43c3140af99a304. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:23:32.155817 25508 consensus_queue.cc:260] T e9e342ecda4b46619ce12452b2cf7154 P 9868dce521294d0b816bba7648f4ae5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } }
I20260709 13:23:32.161049 25504 ts_tablet_manager.cc:1434] T 467a6b42ca6548269c12dddde696f7af P 3e07d64466934da4bb3c02262b1c2e38: Time spent starting tablet: real 0.066s	user 0.007s	sys 0.000s
I20260709 13:23:32.189302 25504 tablet_bootstrap.cc:492] T 0d29b499fa4c41f1b974782cd0c9c1c2 P 3e07d64466934da4bb3c02262b1c2e38: Bootstrap starting.
I20260709 13:23:32.191061 25368 tablet_service.cc:1511] Processing CreateTablet for tablet 821a5866429f4173820fb64336abee02 (DEFAULT_TABLE table=test-workload [id=2f3f768873514f53bb58e92ba96f4813]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 13:23:32.192472 25368 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 821a5866429f4173820fb64336abee02. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:23:32.192162 25300 tablet_service.cc:1511] Processing CreateTablet for tablet 11949f56648c47e0a43c3140af99a304 (DEFAULT_TABLE table=test-workload [id=2f3f768873514f53bb58e92ba96f4813]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260709 13:23:32.193427 25300 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 11949f56648c47e0a43c3140af99a304. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:23:32.195688 25503 tablet_bootstrap.cc:492] T 441d780c3945463f92f31170f31e37a8 P 233e7d29aac04505bf4e65f0a39a0c4c: Bootstrap starting.
I20260709 13:23:32.201052 25199 leader_election.cc:304] T 0e71b6f5c69b4b35a8ac51c22c7d5f0d P 43999d813e564dc688fad116090c95ca [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3e07d64466934da4bb3c02262b1c2e38, 43999d813e564dc688fad116090c95ca; no voters: 9868dce521294d0b816bba7648f4ae5a
W20260709 13:23:32.202114 25276 consensus_peers.cc:597] T 28c0b6fc9bd64d60828d705c6a9ab1e4 P 233e7d29aac04505bf4e65f0a39a0c4c -> Peer 43999d813e564dc688fad116090c95ca (127.14.177.129:38155): Couldn't send request to peer 43999d813e564dc688fad116090c95ca. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:23:32.202296 25510 raft_consensus.cc:2804] T 0e71b6f5c69b4b35a8ac51c22c7d5f0d P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:23:32.202811 25510 raft_consensus.cc:697] T 0e71b6f5c69b4b35a8ac51c22c7d5f0d P 43999d813e564dc688fad116090c95ca [term 1 LEADER]: Becoming Leader. State: Replica: 43999d813e564dc688fad116090c95ca, State: Running, Role: LEADER
I20260709 13:23:32.211120 25225 tablet_service.cc:1511] Processing CreateTablet for tablet 821a5866429f4173820fb64336abee02 (DEFAULT_TABLE table=test-workload [id=2f3f768873514f53bb58e92ba96f4813]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 13:23:32.212687 25225 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 821a5866429f4173820fb64336abee02. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:23:32.210737 25348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.132:33683, user_credentials={real_user=slave}} blocked reactor thread for 74699.3us
I20260709 13:23:32.220903 25458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "467a6b42ca6548269c12dddde696f7af" candidate_uuid: "3e07d64466934da4bb3c02262b1c2e38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9868dce521294d0b816bba7648f4ae5a" is_pre_election: true
W20260709 13:23:32.223747 25348 leader_election.cc:343] T 467a6b42ca6548269c12dddde696f7af P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9868dce521294d0b816bba7648f4ae5a (127.14.177.132:33683): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:23:32.224233 25348 leader_election.cc:304] T 467a6b42ca6548269c12dddde696f7af P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3e07d64466934da4bb3c02262b1c2e38; no voters: 233e7d29aac04505bf4e65f0a39a0c4c, 9868dce521294d0b816bba7648f4ae5a
I20260709 13:23:32.232621 25508 ts_tablet_manager.cc:1434] T e9e342ecda4b46619ce12452b2cf7154 P 9868dce521294d0b816bba7648f4ae5a: Time spent starting tablet: real 0.142s	user 0.006s	sys 0.002s
I20260709 13:23:32.233780 25508 tablet_bootstrap.cc:492] T 3a3317f4c3514df1b2bb8e44a86be92f P 9868dce521294d0b816bba7648f4ae5a: Bootstrap starting.
I20260709 13:23:32.203696 25510 consensus_queue.cc:237] T 0e71b6f5c69b4b35a8ac51c22c7d5f0d P 43999d813e564dc688fad116090c95ca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } }
I20260709 13:23:32.240463 25535 consensus_queue.cc:1048] T 441d780c3945463f92f31170f31e37a8 P 43999d813e564dc688fad116090c95ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:23:32.241693 25448 tablet_service.cc:1511] Processing CreateTablet for tablet 821a5866429f4173820fb64336abee02 (DEFAULT_TABLE table=test-workload [id=2f3f768873514f53bb58e92ba96f4813]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 13:23:32.242564 25503 tablet_bootstrap.cc:654] T 441d780c3945463f92f31170f31e37a8 P 233e7d29aac04505bf4e65f0a39a0c4c: Neither blocks nor log segments found. Creating new log.
I20260709 13:23:32.243065 25448 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 821a5866429f4173820fb64336abee02. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:23:32.250846 25568 raft_consensus.cc:2749] T 467a6b42ca6548269c12dddde696f7af P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:23:32.273082 25568 consensus_queue.cc:1048] T b7a891bfc89e4e259f53da6cb16f4989 P 3e07d64466934da4bb3c02262b1c2e38 [LEADER]: Connected to new peer: Peer: permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 13:23:32.283959 25348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:33683, user_credentials={real_user=slave}} blocked reactor thread for 60249.2us
I20260709 13:23:32.284768 25134 catalog_manager.cc:5697] T 0e71b6f5c69b4b35a8ac51c22c7d5f0d P 43999d813e564dc688fad116090c95ca reported cstate change: term changed from 0 to 1, leader changed from <none> to 43999d813e564dc688fad116090c95ca (127.14.177.129). New cstate: current_term: 1 leader_uuid: "43999d813e564dc688fad116090c95ca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } health_report { overall_health: HEALTHY } } }
I20260709 13:23:32.289953 25508 tablet_bootstrap.cc:654] T 3a3317f4c3514df1b2bb8e44a86be92f P 9868dce521294d0b816bba7648f4ae5a: Neither blocks nor log segments found. Creating new log.
I20260709 13:23:32.308812 25504 tablet_bootstrap.cc:654] T 0d29b499fa4c41f1b974782cd0c9c1c2 P 3e07d64466934da4bb3c02262b1c2e38: Neither blocks nor log segments found. Creating new log.
I20260709 13:23:32.312608 25505 tablet_bootstrap.cc:492] T d053b9ae38de4173a0aded4da494bd98 P 43999d813e564dc688fad116090c95ca: No bootstrap required, opened a new log
I20260709 13:23:32.313488 25505 ts_tablet_manager.cc:1403] T d053b9ae38de4173a0aded4da494bd98 P 43999d813e564dc688fad116090c95ca: Time spent bootstrapping tablet: real 0.221s	user 0.020s	sys 0.010s
I20260709 13:23:32.320258 25505 raft_consensus.cc:359] T d053b9ae38de4173a0aded4da494bd98 P 43999d813e564dc688fad116090c95ca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } }
I20260709 13:23:32.320993 25503 tablet_bootstrap.cc:492] T 441d780c3945463f92f31170f31e37a8 P 233e7d29aac04505bf4e65f0a39a0c4c: No bootstrap required, opened a new log
I20260709 13:23:32.321017 25505 raft_consensus.cc:385] T d053b9ae38de4173a0aded4da494bd98 P 43999d813e564dc688fad116090c95ca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:23:32.322371 25505 raft_consensus.cc:740] T d053b9ae38de4173a0aded4da494bd98 P 43999d813e564dc688fad116090c95ca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43999d813e564dc688fad116090c95ca, State: Initialized, Role: FOLLOWER
I20260709 13:23:32.323381 25505 consensus_queue.cc:260] T d053b9ae38de4173a0aded4da494bd98 P 43999d813e564dc688fad116090c95ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } }
I20260709 13:23:32.322189 25503 ts_tablet_manager.cc:1403] T 441d780c3945463f92f31170f31e37a8 P 233e7d29aac04505bf4e65f0a39a0c4c: Time spent bootstrapping tablet: real 0.127s	user 0.009s	sys 0.010s
I20260709 13:23:32.339977 25504 tablet_bootstrap.cc:492] T 0d29b499fa4c41f1b974782cd0c9c1c2 P 3e07d64466934da4bb3c02262b1c2e38: No bootstrap required, opened a new log
I20260709 13:23:32.340576 25504 ts_tablet_manager.cc:1403] T 0d29b499fa4c41f1b974782cd0c9c1c2 P 3e07d64466934da4bb3c02262b1c2e38: Time spent bootstrapping tablet: real 0.151s	user 0.010s	sys 0.005s
I20260709 13:23:32.344333 25504 raft_consensus.cc:359] T 0d29b499fa4c41f1b974782cd0c9c1c2 P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } }
I20260709 13:23:32.345283 25504 raft_consensus.cc:385] T 0d29b499fa4c41f1b974782cd0c9c1c2 P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:23:32.345594 25504 raft_consensus.cc:740] T 0d29b499fa4c41f1b974782cd0c9c1c2 P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e07d64466934da4bb3c02262b1c2e38, State: Initialized, Role: FOLLOWER
I20260709 13:23:32.346364 25504 consensus_queue.cc:260] T 0d29b499fa4c41f1b974782cd0c9c1c2 P 3e07d64466934da4bb3c02262b1c2e38 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } }
I20260709 13:23:32.348780 25504 ts_tablet_manager.cc:1434] T 0d29b499fa4c41f1b974782cd0c9c1c2 P 3e07d64466934da4bb3c02262b1c2e38: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 13:23:32.348809 25505 ts_tablet_manager.cc:1434] T d053b9ae38de4173a0aded4da494bd98 P 43999d813e564dc688fad116090c95ca: Time spent starting tablet: real 0.035s	user 0.006s	sys 0.002s
I20260709 13:23:32.350227 25505 tablet_bootstrap.cc:492] T b7ff8dde9f5d4c49b9203d902f6b94c4 P 43999d813e564dc688fad116090c95ca: Bootstrap starting.
I20260709 13:23:32.350237 25504 tablet_bootstrap.cc:492] T 1f15ec62698c4a8cbd84f2b3d516d2ea P 3e07d64466934da4bb3c02262b1c2e38: Bootstrap starting.
I20260709 13:23:32.359202 25504 tablet_bootstrap.cc:654] T 1f15ec62698c4a8cbd84f2b3d516d2ea P 3e07d64466934da4bb3c02262b1c2e38: Neither blocks nor log segments found. Creating new log.
I20260709 13:23:32.360836 25508 tablet_bootstrap.cc:492] T 3a3317f4c3514df1b2bb8e44a86be92f P 9868dce521294d0b816bba7648f4ae5a: No bootstrap required, opened a new log
I20260709 13:23:32.361542 25508 ts_tablet_manager.cc:1403] T 3a3317f4c3514df1b2bb8e44a86be92f P 9868dce521294d0b816bba7648f4ae5a: Time spent bootstrapping tablet: real 0.128s	user 0.014s	sys 0.010s
I20260709 13:23:32.365705 25508 raft_consensus.cc:359] T 3a3317f4c3514df1b2bb8e44a86be92f P 9868dce521294d0b816bba7648f4ae5a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } }
I20260709 13:23:32.366271 25504 tablet_bootstrap.cc:492] T 1f15ec62698c4a8cbd84f2b3d516d2ea P 3e07d64466934da4bb3c02262b1c2e38: No bootstrap required, opened a new log
I20260709 13:23:32.366590 25508 raft_consensus.cc:385] T 3a3317f4c3514df1b2bb8e44a86be92f P 9868dce521294d0b816bba7648f4ae5a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:23:32.366859 25504 ts_tablet_manager.cc:1403] T 1f15ec62698c4a8cbd84f2b3d516d2ea P 3e07d64466934da4bb3c02262b1c2e38: Time spent bootstrapping tablet: real 0.017s	user 0.012s	sys 0.004s
I20260709 13:23:32.367487 25503 raft_consensus.cc:359] T 441d780c3945463f92f31170f31e37a8 P 233e7d29aac04505bf4e65f0a39a0c4c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } }
I20260709 13:23:32.368392 25503 raft_consensus.cc:385] T 441d780c3945463f92f31170f31e37a8 P 233e7d29aac04505bf4e65f0a39a0c4c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:23:32.368724 25503 raft_consensus.cc:740] T 441d780c3945463f92f31170f31e37a8 P 233e7d29aac04505bf4e65f0a39a0c4c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 233e7d29aac04505bf4e65f0a39a0c4c, State: Initialized, Role: FOLLOWER
I20260709 13:23:32.369606 25503 consensus_queue.cc:260] T 441d780c3945463f92f31170f31e37a8 P 233e7d29aac04505bf4e65f0a39a0c4c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } }
I20260709 13:23:32.370370 25504 raft_consensus.cc:359] T 1f15ec62698c4a8cbd84f2b3d516d2ea P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } }
I20260709 13:23:32.371114 25504 raft_consensus.cc:385] T 1f15ec62698c4a8cbd84f2b3d516d2ea P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:23:32.371431 25504 raft_consensus.cc:740] T 1f15ec62698c4a8cbd84f2b3d516d2ea P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e07d64466934da4bb3c02262b1c2e38, State: Initialized, Role: FOLLOWER
I20260709 13:23:32.372220 25503 ts_tablet_manager.cc:1434] T 441d780c3945463f92f31170f31e37a8 P 233e7d29aac04505bf4e65f0a39a0c4c: Time spent starting tablet: real 0.041s	user 0.006s	sys 0.002s
I20260709 13:23:32.366994 25508 raft_consensus.cc:740] T 3a3317f4c3514df1b2bb8e44a86be92f P 9868dce521294d0b816bba7648f4ae5a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9868dce521294d0b816bba7648f4ae5a, State: Initialized, Role: FOLLOWER
I20260709 13:23:32.372334 25504 consensus_queue.cc:260] T 1f15ec62698c4a8cbd84f2b3d516d2ea P 3e07d64466934da4bb3c02262b1c2e38 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } }
I20260709 13:23:32.373541 25508 consensus_queue.cc:260] T 3a3317f4c3514df1b2bb8e44a86be92f P 9868dce521294d0b816bba7648f4ae5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } }
I20260709 13:23:32.374791 25504 ts_tablet_manager.cc:1434] T 1f15ec62698c4a8cbd84f2b3d516d2ea P 3e07d64466934da4bb3c02262b1c2e38: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 13:23:32.375550 25505 tablet_bootstrap.cc:654] T b7ff8dde9f5d4c49b9203d902f6b94c4 P 43999d813e564dc688fad116090c95ca: Neither blocks nor log segments found. Creating new log.
I20260709 13:23:32.375898 25504 tablet_bootstrap.cc:492] T b2f14aed05e24a98ad83535029b795dc P 3e07d64466934da4bb3c02262b1c2e38: Bootstrap starting.
I20260709 13:23:32.379338 25503 tablet_bootstrap.cc:492] T 3fe5a944a342411ea923eb39071ac770 P 233e7d29aac04505bf4e65f0a39a0c4c: Bootstrap starting.
I20260709 13:23:32.382609 25504 tablet_bootstrap.cc:654] T b2f14aed05e24a98ad83535029b795dc P 3e07d64466934da4bb3c02262b1c2e38: Neither blocks nor log segments found. Creating new log.
I20260709 13:23:32.389750 25505 tablet_bootstrap.cc:492] T b7ff8dde9f5d4c49b9203d902f6b94c4 P 43999d813e564dc688fad116090c95ca: No bootstrap required, opened a new log
I20260709 13:23:32.390302 25505 ts_tablet_manager.cc:1403] T b7ff8dde9f5d4c49b9203d902f6b94c4 P 43999d813e564dc688fad116090c95ca: Time spent bootstrapping tablet: real 0.040s	user 0.010s	sys 0.003s
I20260709 13:23:32.393464 25505 raft_consensus.cc:359] T b7ff8dde9f5d4c49b9203d902f6b94c4 P 43999d813e564dc688fad116090c95ca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } }
I20260709 13:23:32.394232 25505 raft_consensus.cc:385] T b7ff8dde9f5d4c49b9203d902f6b94c4 P 43999d813e564dc688fad116090c95ca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:23:32.394568 25505 raft_consensus.cc:740] T b7ff8dde9f5d4c49b9203d902f6b94c4 P 43999d813e564dc688fad116090c95ca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43999d813e564dc688fad116090c95ca, State: Initialized, Role: FOLLOWER
I20260709 13:23:32.395102 25504 tablet_bootstrap.cc:492] T b2f14aed05e24a98ad83535029b795dc P 3e07d64466934da4bb3c02262b1c2e38: No bootstrap required, opened a new log
I20260709 13:23:32.396056 25504 ts_tablet_manager.cc:1403] T b2f14aed05e24a98ad83535029b795dc P 3e07d64466934da4bb3c02262b1c2e38: Time spent bootstrapping tablet: real 0.020s	user 0.015s	sys 0.004s
I20260709 13:23:32.398628 25508 ts_tablet_manager.cc:1434] T 3a3317f4c3514df1b2bb8e44a86be92f P 9868dce521294d0b816bba7648f4ae5a: Time spent starting tablet: real 0.037s	user 0.010s	sys 0.000s
I20260709 13:23:32.399530 25504 raft_consensus.cc:359] T b2f14aed05e24a98ad83535029b795dc P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } }
I20260709 13:23:32.400383 25504 raft_consensus.cc:385] T b2f14aed05e24a98ad83535029b795dc P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:23:32.400704 25504 raft_consensus.cc:740] T b2f14aed05e24a98ad83535029b795dc P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e07d64466934da4bb3c02262b1c2e38, State: Initialized, Role: FOLLOWER
I20260709 13:23:32.401484 25504 consensus_queue.cc:260] T b2f14aed05e24a98ad83535029b795dc P 3e07d64466934da4bb3c02262b1c2e38 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } }
I20260709 13:23:32.402359 25503 tablet_bootstrap.cc:654] T 3fe5a944a342411ea923eb39071ac770 P 233e7d29aac04505bf4e65f0a39a0c4c: Neither blocks nor log segments found. Creating new log.
I20260709 13:23:32.403434 25508 tablet_bootstrap.cc:492] T a820954e07a94582ac0613342fcffde5 P 9868dce521294d0b816bba7648f4ae5a: Bootstrap starting.
I20260709 13:23:32.403885 25504 ts_tablet_manager.cc:1434] T b2f14aed05e24a98ad83535029b795dc P 3e07d64466934da4bb3c02262b1c2e38: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 13:23:32.405269 25504 tablet_bootstrap.cc:492] T 280d65245f4f4358b54bef93e18aca92 P 3e07d64466934da4bb3c02262b1c2e38: Bootstrap starting.
I20260709 13:23:32.395504 25505 consensus_queue.cc:260] T b7ff8dde9f5d4c49b9203d902f6b94c4 P 43999d813e564dc688fad116090c95ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } }
I20260709 13:23:32.412307 25504 tablet_bootstrap.cc:654] T 280d65245f4f4358b54bef93e18aca92 P 3e07d64466934da4bb3c02262b1c2e38: Neither blocks nor log segments found. Creating new log.
I20260709 13:23:32.417487 25505 ts_tablet_manager.cc:1434] T b7ff8dde9f5d4c49b9203d902f6b94c4 P 43999d813e564dc688fad116090c95ca: Time spent starting tablet: real 0.027s	user 0.009s	sys 0.001s
I20260709 13:23:32.419185 25505 tablet_bootstrap.cc:492] T 1f15ec62698c4a8cbd84f2b3d516d2ea P 43999d813e564dc688fad116090c95ca: Bootstrap starting.
I20260709 13:23:32.419391 25503 tablet_bootstrap.cc:492] T 3fe5a944a342411ea923eb39071ac770 P 233e7d29aac04505bf4e65f0a39a0c4c: No bootstrap required, opened a new log
I20260709 13:23:32.420348 25512 raft_consensus.cc:493] T 0d29b499fa4c41f1b974782cd0c9c1c2 P 233e7d29aac04505bf4e65f0a39a0c4c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:23:32.420898 25512 raft_consensus.cc:515] T 0d29b499fa4c41f1b974782cd0c9c1c2 P 233e7d29aac04505bf4e65f0a39a0c4c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } }
I20260709 13:23:32.423266 25512 leader_election.cc:290] T 0d29b499fa4c41f1b974782cd0c9c1c2 P 233e7d29aac04505bf4e65f0a39a0c4c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 43999d813e564dc688fad116090c95ca (127.14.177.129:38155), 3e07d64466934da4bb3c02262b1c2e38 (127.14.177.131:41539)
I20260709 13:23:32.425632 25236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d29b499fa4c41f1b974782cd0c9c1c2" candidate_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43999d813e564dc688fad116090c95ca" is_pre_election: true
I20260709 13:23:32.426458 25236 raft_consensus.cc:2468] T 0d29b499fa4c41f1b974782cd0c9c1c2 P 43999d813e564dc688fad116090c95ca [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 233e7d29aac04505bf4e65f0a39a0c4c in term 0.
I20260709 13:23:32.427934 25276 leader_election.cc:304] T 0d29b499fa4c41f1b974782cd0c9c1c2 P 233e7d29aac04505bf4e65f0a39a0c4c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 233e7d29aac04505bf4e65f0a39a0c4c, 43999d813e564dc688fad116090c95ca; no voters: 
I20260709 13:23:32.429647 25385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d29b499fa4c41f1b974782cd0c9c1c2" candidate_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e07d64466934da4bb3c02262b1c2e38" is_pre_election: true
I20260709 13:23:32.430430 25385 raft_consensus.cc:2468] T 0d29b499fa4c41f1b974782cd0c9c1c2 P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 233e7d29aac04505bf4e65f0a39a0c4c in term 0.
I20260709 13:23:32.431720 25508 tablet_bootstrap.cc:654] T a820954e07a94582ac0613342fcffde5 P 9868dce521294d0b816bba7648f4ae5a: Neither blocks nor log segments found. Creating new log.
I20260709 13:23:32.432534 25512 raft_consensus.cc:2804] T 0d29b499fa4c41f1b974782cd0c9c1c2 P 233e7d29aac04505bf4e65f0a39a0c4c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:23:32.433035 25512 raft_consensus.cc:493] T 0d29b499fa4c41f1b974782cd0c9c1c2 P 233e7d29aac04505bf4e65f0a39a0c4c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:23:32.433431 25512 raft_consensus.cc:3060] T 0d29b499fa4c41f1b974782cd0c9c1c2 P 233e7d29aac04505bf4e65f0a39a0c4c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:23:32.436420 25505 tablet_bootstrap.cc:654] T 1f15ec62698c4a8cbd84f2b3d516d2ea P 43999d813e564dc688fad116090c95ca: Neither blocks nor log segments found. Creating new log.
I20260709 13:23:32.439248 25503 ts_tablet_manager.cc:1403] T 3fe5a944a342411ea923eb39071ac770 P 233e7d29aac04505bf4e65f0a39a0c4c: Time spent bootstrapping tablet: real 0.060s	user 0.007s	sys 0.013s
I20260709 13:23:32.439425 25504 tablet_bootstrap.cc:492] T 280d65245f4f4358b54bef93e18aca92 P 3e07d64466934da4bb3c02262b1c2e38: No bootstrap required, opened a new log
I20260709 13:23:32.440095 25504 ts_tablet_manager.cc:1403] T 280d65245f4f4358b54bef93e18aca92 P 3e07d64466934da4bb3c02262b1c2e38: Time spent bootstrapping tablet: real 0.035s	user 0.021s	sys 0.012s
I20260709 13:23:32.439946 25563 raft_consensus.cc:493] T e9e342ecda4b46619ce12452b2cf7154 P 9868dce521294d0b816bba7648f4ae5a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:23:32.440665 25563 raft_consensus.cc:515] T e9e342ecda4b46619ce12452b2cf7154 P 9868dce521294d0b816bba7648f4ae5a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } }
I20260709 13:23:32.442062 25503 raft_consensus.cc:359] T 3fe5a944a342411ea923eb39071ac770 P 233e7d29aac04505bf4e65f0a39a0c4c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } }
I20260709 13:23:32.442692 25503 raft_consensus.cc:385] T 3fe5a944a342411ea923eb39071ac770 P 233e7d29aac04505bf4e65f0a39a0c4c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:23:32.442956 25503 raft_consensus.cc:740] T 3fe5a944a342411ea923eb39071ac770 P 233e7d29aac04505bf4e65f0a39a0c4c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 233e7d29aac04505bf4e65f0a39a0c4c, State: Initialized, Role: FOLLOWER
I20260709 13:23:32.443547 25504 raft_consensus.cc:359] T 280d65245f4f4358b54bef93e18aca92 P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } }
I20260709 13:23:32.444211 25563 leader_election.cc:290] T e9e342ecda4b46619ce12452b2cf7154 P 9868dce521294d0b816bba7648f4ae5a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 43999d813e564dc688fad116090c95ca (127.14.177.129:38155), 233e7d29aac04505bf4e65f0a39a0c4c (127.14.177.130:42957)
I20260709 13:23:32.444350 25504 raft_consensus.cc:385] T 280d65245f4f4358b54bef93e18aca92 P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:23:32.444780 25504 raft_consensus.cc:740] T 280d65245f4f4358b54bef93e18aca92 P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e07d64466934da4bb3c02262b1c2e38, State: Initialized, Role: FOLLOWER
I20260709 13:23:32.445006 25311 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9e342ecda4b46619ce12452b2cf7154" candidate_uuid: "9868dce521294d0b816bba7648f4ae5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" is_pre_election: true
I20260709 13:23:32.446015 25311 raft_consensus.cc:2468] T e9e342ecda4b46619ce12452b2cf7154 P 233e7d29aac04505bf4e65f0a39a0c4c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9868dce521294d0b816bba7648f4ae5a in term 0.
I20260709 13:23:32.445860 25504 consensus_queue.cc:260] T 280d65245f4f4358b54bef93e18aca92 P 3e07d64466934da4bb3c02262b1c2e38 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } }
I20260709 13:23:32.447399 25508 tablet_bootstrap.cc:492] T a820954e07a94582ac0613342fcffde5 P 9868dce521294d0b816bba7648f4ae5a: No bootstrap required, opened a new log
I20260709 13:23:32.448417 25508 ts_tablet_manager.cc:1403] T a820954e07a94582ac0613342fcffde5 P 9868dce521294d0b816bba7648f4ae5a: Time spent bootstrapping tablet: real 0.045s	user 0.017s	sys 0.000s
I20260709 13:23:32.449313 25425 leader_election.cc:304] T e9e342ecda4b46619ce12452b2cf7154 P 9868dce521294d0b816bba7648f4ae5a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 233e7d29aac04505bf4e65f0a39a0c4c, 9868dce521294d0b816bba7648f4ae5a; no voters: 
I20260709 13:23:32.450635 25563 raft_consensus.cc:2804] T e9e342ecda4b46619ce12452b2cf7154 P 9868dce521294d0b816bba7648f4ae5a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:23:32.451141 25563 raft_consensus.cc:493] T e9e342ecda4b46619ce12452b2cf7154 P 9868dce521294d0b816bba7648f4ae5a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:23:32.451596 25563 raft_consensus.cc:3060] T e9e342ecda4b46619ce12452b2cf7154 P 9868dce521294d0b816bba7648f4ae5a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:23:32.452055 25508 raft_consensus.cc:359] T a820954e07a94582ac0613342fcffde5 P 9868dce521294d0b816bba7648f4ae5a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } }
I20260709 13:23:32.452919 25508 raft_consensus.cc:385] T a820954e07a94582ac0613342fcffde5 P 9868dce521294d0b816bba7648f4ae5a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:23:32.453027 25504 ts_tablet_manager.cc:1434] T 280d65245f4f4358b54bef93e18aca92 P 3e07d64466934da4bb3c02262b1c2e38: Time spent starting tablet: real 0.013s	user 0.007s	sys 0.001s
I20260709 13:23:32.453347 25508 raft_consensus.cc:740] T a820954e07a94582ac0613342fcffde5 P 9868dce521294d0b816bba7648f4ae5a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9868dce521294d0b816bba7648f4ae5a, State: Initialized, Role: FOLLOWER
I20260709 13:23:32.454212 25504 tablet_bootstrap.cc:492] T b7ff8dde9f5d4c49b9203d902f6b94c4 P 3e07d64466934da4bb3c02262b1c2e38: Bootstrap starting.
I20260709 13:23:32.456665 25505 tablet_bootstrap.cc:492] T 1f15ec62698c4a8cbd84f2b3d516d2ea P 43999d813e564dc688fad116090c95ca: No bootstrap required, opened a new log
I20260709 13:23:32.457284 25505 ts_tablet_manager.cc:1403] T 1f15ec62698c4a8cbd84f2b3d516d2ea P 43999d813e564dc688fad116090c95ca: Time spent bootstrapping tablet: real 0.038s	user 0.014s	sys 0.000s
I20260709 13:23:32.456790 25508 consensus_queue.cc:260] T a820954e07a94582ac0613342fcffde5 P 9868dce521294d0b816bba7648f4ae5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } }
I20260709 13:23:32.460515 25505 raft_consensus.cc:359] T 1f15ec62698c4a8cbd84f2b3d516d2ea P 43999d813e564dc688fad116090c95ca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } }
I20260709 13:23:32.461284 25505 raft_consensus.cc:385] T 1f15ec62698c4a8cbd84f2b3d516d2ea P 43999d813e564dc688fad116090c95ca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:23:32.461601 25505 raft_consensus.cc:740] T 1f15ec62698c4a8cbd84f2b3d516d2ea P 43999d813e564dc688fad116090c95ca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43999d813e564dc688fad116090c95ca, State: Initialized, Role: FOLLOWER
I20260709 13:23:32.462347 25505 consensus_queue.cc:260] T 1f15ec62698c4a8cbd84f2b3d516d2ea P 43999d813e564dc688fad116090c95ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } }
I20260709 13:23:32.463487 25504 tablet_bootstrap.cc:654] T b7ff8dde9f5d4c49b9203d902f6b94c4 P 3e07d64466934da4bb3c02262b1c2e38: Neither blocks nor log segments found. Creating new log.
I20260709 13:23:32.466166 25508 ts_tablet_manager.cc:1434] T a820954e07a94582ac0613342fcffde5 P 9868dce521294d0b816bba7648f4ae5a: Time spent starting tablet: real 0.017s	user 0.007s	sys 0.000s
I20260709 13:23:32.467460 25508 tablet_bootstrap.cc:492] T 467a6b42ca6548269c12dddde696f7af P 9868dce521294d0b816bba7648f4ae5a: Bootstrap starting.
I20260709 13:23:32.473785 25505 ts_tablet_manager.cc:1434] T 1f15ec62698c4a8cbd84f2b3d516d2ea P 43999d813e564dc688fad116090c95ca: Time spent starting tablet: real 0.016s	user 0.003s	sys 0.005s
I20260709 13:23:32.475071 25508 tablet_bootstrap.cc:654] T 467a6b42ca6548269c12dddde696f7af P 9868dce521294d0b816bba7648f4ae5a: Neither blocks nor log segments found. Creating new log.
I20260709 13:23:32.478242 25505 tablet_bootstrap.cc:492] T 3a3317f4c3514df1b2bb8e44a86be92f P 43999d813e564dc688fad116090c95ca: Bootstrap starting.
I20260709 13:23:32.482023 25503 consensus_queue.cc:260] T 3fe5a944a342411ea923eb39071ac770 P 233e7d29aac04505bf4e65f0a39a0c4c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } }
I20260709 13:23:32.491921 25512 raft_consensus.cc:515] T 0d29b499fa4c41f1b974782cd0c9c1c2 P 233e7d29aac04505bf4e65f0a39a0c4c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } }
I20260709 13:23:32.499521 25385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d29b499fa4c41f1b974782cd0c9c1c2" candidate_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e07d64466934da4bb3c02262b1c2e38"
I20260709 13:23:32.500313 25573 raft_consensus.cc:493] T 1f15ec62698c4a8cbd84f2b3d516d2ea P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:23:32.500562 25385 raft_consensus.cc:3060] T 0d29b499fa4c41f1b974782cd0c9c1c2 P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:23:32.501232 25573 raft_consensus.cc:515] T 1f15ec62698c4a8cbd84f2b3d516d2ea P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } }
I20260709 13:23:32.505695 25573 leader_election.cc:290] T 1f15ec62698c4a8cbd84f2b3d516d2ea P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9868dce521294d0b816bba7648f4ae5a (127.14.177.132:33683), 43999d813e564dc688fad116090c95ca (127.14.177.129:38155)
I20260709 13:23:32.505995 25458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f15ec62698c4a8cbd84f2b3d516d2ea" candidate_uuid: "3e07d64466934da4bb3c02262b1c2e38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9868dce521294d0b816bba7648f4ae5a" is_pre_election: true
I20260709 13:23:32.508486 25559 raft_consensus.cc:493] T 579da2d6aabb4ae9a2b1802e1358068b P 43999d813e564dc688fad116090c95ca [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:23:32.508335 25236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f15ec62698c4a8cbd84f2b3d516d2ea" candidate_uuid: "3e07d64466934da4bb3c02262b1c2e38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43999d813e564dc688fad116090c95ca" is_pre_election: true
I20260709 13:23:32.509272 25559 raft_consensus.cc:515] T 579da2d6aabb4ae9a2b1802e1358068b P 43999d813e564dc688fad116090c95ca [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } }
I20260709 13:23:32.509549 25236 raft_consensus.cc:2468] T 1f15ec62698c4a8cbd84f2b3d516d2ea P 43999d813e564dc688fad116090c95ca [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3e07d64466934da4bb3c02262b1c2e38 in term 0.
I20260709 13:23:32.511103 25505 tablet_bootstrap.cc:654] T 3a3317f4c3514df1b2bb8e44a86be92f P 43999d813e564dc688fad116090c95ca: Neither blocks nor log segments found. Creating new log.
I20260709 13:23:32.515240 25236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d29b499fa4c41f1b974782cd0c9c1c2" candidate_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43999d813e564dc688fad116090c95ca"
I20260709 13:23:32.495528 25512 leader_election.cc:290] T 0d29b499fa4c41f1b974782cd0c9c1c2 P 233e7d29aac04505bf4e65f0a39a0c4c [CANDIDATE]: Term 1 election: Requested vote from peers 43999d813e564dc688fad116090c95ca (127.14.177.129:38155), 3e07d64466934da4bb3c02262b1c2e38 (127.14.177.131:41539)
I20260709 13:23:32.515374 25588 raft_consensus.cc:493] T 3fe5a944a342411ea923eb39071ac770 P 233e7d29aac04505bf4e65f0a39a0c4c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:23:32.508735 25563 raft_consensus.cc:515] T e9e342ecda4b46619ce12452b2cf7154 P 9868dce521294d0b816bba7648f4ae5a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } }
I20260709 13:23:32.520166 25236 raft_consensus.cc:3060] T 0d29b499fa4c41f1b974782cd0c9c1c2 P 43999d813e564dc688fad116090c95ca [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:23:32.550683 25503 ts_tablet_manager.cc:1434] T 3fe5a944a342411ea923eb39071ac770 P 233e7d29aac04505bf4e65f0a39a0c4c: Time spent starting tablet: real 0.111s	user 0.009s	sys 0.008s
I20260709 13:23:32.527055 25559 leader_election.cc:290] T 579da2d6aabb4ae9a2b1802e1358068b P 43999d813e564dc688fad116090c95ca [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9868dce521294d0b816bba7648f4ae5a (127.14.177.132:33683), 233e7d29aac04505bf4e65f0a39a0c4c (127.14.177.130:42957)
I20260709 13:23:32.520907 25350 leader_election.cc:304] T 1f15ec62698c4a8cbd84f2b3d516d2ea P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e07d64466934da4bb3c02262b1c2e38, 43999d813e564dc688fad116090c95ca; no voters: 
W20260709 13:23:32.553413 25348 leader_election.cc:343] T 1f15ec62698c4a8cbd84f2b3d516d2ea P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9868dce521294d0b816bba7648f4ae5a (127.14.177.132:33683): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:23:32.553567 25503 tablet_bootstrap.cc:492] T 467a6b42ca6548269c12dddde696f7af P 233e7d29aac04505bf4e65f0a39a0c4c: Bootstrap starting.
I20260709 13:23:32.528678 25311 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "579da2d6aabb4ae9a2b1802e1358068b" candidate_uuid: "43999d813e564dc688fad116090c95ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" is_pre_election: true
I20260709 13:23:32.555917 25311 raft_consensus.cc:2468] T 579da2d6aabb4ae9a2b1802e1358068b P 233e7d29aac04505bf4e65f0a39a0c4c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 43999d813e564dc688fad116090c95ca in term 0.
I20260709 13:23:32.556242 25539 raft_consensus.cc:493] T 3a3317f4c3514df1b2bb8e44a86be92f P 9868dce521294d0b816bba7648f4ae5a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:23:32.556854 25563 leader_election.cc:290] T e9e342ecda4b46619ce12452b2cf7154 P 9868dce521294d0b816bba7648f4ae5a [CANDIDATE]: Term 1 election: Requested vote from peers 43999d813e564dc688fad116090c95ca (127.14.177.129:38155), 233e7d29aac04505bf4e65f0a39a0c4c (127.14.177.130:42957)
I20260709 13:23:32.556746 25539 raft_consensus.cc:515] T 3a3317f4c3514df1b2bb8e44a86be92f P 9868dce521294d0b816bba7648f4ae5a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } }
I20260709 13:23:32.557986 25310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9e342ecda4b46619ce12452b2cf7154" candidate_uuid: "9868dce521294d0b816bba7648f4ae5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "233e7d29aac04505bf4e65f0a39a0c4c"
I20260709 13:23:32.543048 25385 raft_consensus.cc:2468] T 0d29b499fa4c41f1b974782cd0c9c1c2 P 3e07d64466934da4bb3c02262b1c2e38 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 233e7d29aac04505bf4e65f0a39a0c4c in term 1.
I20260709 13:23:32.558831 25310 raft_consensus.cc:3060] T e9e342ecda4b46619ce12452b2cf7154 P 233e7d29aac04505bf4e65f0a39a0c4c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:23:32.560191 25539 leader_election.cc:290] T 3a3317f4c3514df1b2bb8e44a86be92f P 9868dce521294d0b816bba7648f4ae5a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 43999d813e564dc688fad116090c95ca (127.14.177.129:38155), 233e7d29aac04505bf4e65f0a39a0c4c (127.14.177.130:42957)
I20260709 13:23:32.536561 25458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "579da2d6aabb4ae9a2b1802e1358068b" candidate_uuid: "43999d813e564dc688fad116090c95ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9868dce521294d0b816bba7648f4ae5a" is_pre_election: true
W20260709 13:23:32.563074 25199 leader_election.cc:343] T 579da2d6aabb4ae9a2b1802e1358068b P 43999d813e564dc688fad116090c95ca [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9868dce521294d0b816bba7648f4ae5a (127.14.177.132:33683): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:23:32.567615 25573 raft_consensus.cc:2804] T 1f15ec62698c4a8cbd84f2b3d516d2ea P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:23:32.568094 25573 raft_consensus.cc:493] T 1f15ec62698c4a8cbd84f2b3d516d2ea P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:23:32.568179 25309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a3317f4c3514df1b2bb8e44a86be92f" candidate_uuid: "9868dce521294d0b816bba7648f4ae5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" is_pre_election: true
I20260709 13:23:32.568507 25573 raft_consensus.cc:3060] T 1f15ec62698c4a8cbd84f2b3d516d2ea P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:23:32.522583 25588 raft_consensus.cc:515] T 3fe5a944a342411ea923eb39071ac770 P 233e7d29aac04505bf4e65f0a39a0c4c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } }
W20260709 13:23:32.571578 25425 leader_election.cc:343] T 3a3317f4c3514df1b2bb8e44a86be92f P 9868dce521294d0b816bba7648f4ae5a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 233e7d29aac04505bf4e65f0a39a0c4c (127.14.177.130:42957): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:23:32.534647 25504 tablet_bootstrap.cc:492] T b7ff8dde9f5d4c49b9203d902f6b94c4 P 3e07d64466934da4bb3c02262b1c2e38: No bootstrap required, opened a new log
I20260709 13:23:32.573441 25310 raft_consensus.cc:2468] T e9e342ecda4b46619ce12452b2cf7154 P 233e7d29aac04505bf4e65f0a39a0c4c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9868dce521294d0b816bba7648f4ae5a in term 1.
I20260709 13:23:32.572626 25504 ts_tablet_manager.cc:1403] T b7ff8dde9f5d4c49b9203d902f6b94c4 P 3e07d64466934da4bb3c02262b1c2e38: Time spent bootstrapping tablet: real 0.119s	user 0.017s	sys 0.017s
I20260709 13:23:32.574604 25425 leader_election.cc:304] T e9e342ecda4b46619ce12452b2cf7154 P 9868dce521294d0b816bba7648f4ae5a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 233e7d29aac04505bf4e65f0a39a0c4c, 9868dce521294d0b816bba7648f4ae5a; no voters: 
I20260709 13:23:32.575616 25539 raft_consensus.cc:2804] T e9e342ecda4b46619ce12452b2cf7154 P 9868dce521294d0b816bba7648f4ae5a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:23:32.576097 25539 raft_consensus.cc:697] T e9e342ecda4b46619ce12452b2cf7154 P 9868dce521294d0b816bba7648f4ae5a [term 1 LEADER]: Becoming Leader. State: Replica: 9868dce521294d0b816bba7648f4ae5a, State: Running, Role: LEADER
I20260709 13:23:32.576891 25539 consensus_queue.cc:237] T e9e342ecda4b46619ce12452b2cf7154 P 9868dce521294d0b816bba7648f4ae5a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } }
I20260709 13:23:32.577725 25504 raft_consensus.cc:359] T b7ff8dde9f5d4c49b9203d902f6b94c4 P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } }
I20260709 13:23:32.580174 25202 leader_election.cc:304] T 579da2d6aabb4ae9a2b1802e1358068b P 43999d813e564dc688fad116090c95ca [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 233e7d29aac04505bf4e65f0a39a0c4c, 43999d813e564dc688fad116090c95ca; no voters: 9868dce521294d0b816bba7648f4ae5a
I20260709 13:23:32.581070 25510 raft_consensus.cc:493] T d053b9ae38de4173a0aded4da494bd98 P 43999d813e564dc688fad116090c95ca [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:23:32.581632 25559 raft_consensus.cc:2804] T 579da2d6aabb4ae9a2b1802e1358068b P 43999d813e564dc688fad116090c95ca [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:23:32.581560 25510 raft_consensus.cc:515] T d053b9ae38de4173a0aded4da494bd98 P 43999d813e564dc688fad116090c95ca [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } }
I20260709 13:23:32.582049 25559 raft_consensus.cc:493] T 579da2d6aabb4ae9a2b1802e1358068b P 43999d813e564dc688fad116090c95ca [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:23:32.582448 25559 raft_consensus.cc:3060] T 579da2d6aabb4ae9a2b1802e1358068b P 43999d813e564dc688fad116090c95ca [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:23:32.583871 25510 leader_election.cc:290] T d053b9ae38de4173a0aded4da494bd98 P 43999d813e564dc688fad116090c95ca [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9868dce521294d0b816bba7648f4ae5a (127.14.177.132:33683), 233e7d29aac04505bf4e65f0a39a0c4c (127.14.177.130:42957)
I20260709 13:23:32.586146 25510 raft_consensus.cc:493] T b7ff8dde9f5d4c49b9203d902f6b94c4 P 43999d813e564dc688fad116090c95ca [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:23:32.586624 25510 raft_consensus.cc:515] T b7ff8dde9f5d4c49b9203d902f6b94c4 P 43999d813e564dc688fad116090c95ca [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } }
I20260709 13:23:32.587203 25457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d053b9ae38de4173a0aded4da494bd98" candidate_uuid: "43999d813e564dc688fad116090c95ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9868dce521294d0b816bba7648f4ae5a" is_pre_election: true
I20260709 13:23:32.588840 25510 leader_election.cc:290] T b7ff8dde9f5d4c49b9203d902f6b94c4 P 43999d813e564dc688fad116090c95ca [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 233e7d29aac04505bf4e65f0a39a0c4c (127.14.177.130:42957), 3e07d64466934da4bb3c02262b1c2e38 (127.14.177.131:41539)
W20260709 13:23:32.590023 25199 leader_election.cc:343] T d053b9ae38de4173a0aded4da494bd98 P 43999d813e564dc688fad116090c95ca [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9868dce521294d0b816bba7648f4ae5a (127.14.177.132:33683): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:23:32.589567 25588 leader_election.cc:290] T 3fe5a944a342411ea923eb39071ac770 P 233e7d29aac04505bf4e65f0a39a0c4c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3e07d64466934da4bb3c02262b1c2e38 (127.14.177.131:41539), 9868dce521294d0b816bba7648f4ae5a (127.14.177.132:33683)
I20260709 13:23:32.590510 25385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7ff8dde9f5d4c49b9203d902f6b94c4" candidate_uuid: "43999d813e564dc688fad116090c95ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e07d64466934da4bb3c02262b1c2e38" is_pre_election: true
I20260709 13:23:32.595520 25458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe5a944a342411ea923eb39071ac770" candidate_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9868dce521294d0b816bba7648f4ae5a" is_pre_election: true
I20260709 13:23:32.596236 25458 raft_consensus.cc:2468] T 3fe5a944a342411ea923eb39071ac770 P 9868dce521294d0b816bba7648f4ae5a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 233e7d29aac04505bf4e65f0a39a0c4c in term 0.
I20260709 13:23:32.597477 25274 leader_election.cc:304] T 3fe5a944a342411ea923eb39071ac770 P 233e7d29aac04505bf4e65f0a39a0c4c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 233e7d29aac04505bf4e65f0a39a0c4c, 9868dce521294d0b816bba7648f4ae5a; no voters: 
W20260709 13:23:32.597791 25350 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:38155, user_credentials={real_user=slave}} blocked reactor thread for 81636.4us
I20260709 13:23:32.598747 25274 leader_election.cc:304] T 0d29b499fa4c41f1b974782cd0c9c1c2 P 233e7d29aac04505bf4e65f0a39a0c4c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 233e7d29aac04505bf4e65f0a39a0c4c, 3e07d64466934da4bb3c02262b1c2e38; no voters: 
I20260709 13:23:32.598390 25588 raft_consensus.cc:2804] T 3fe5a944a342411ea923eb39071ac770 P 233e7d29aac04505bf4e65f0a39a0c4c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:23:32.599844 25590 raft_consensus.cc:2804] T 0d29b499fa4c41f1b974782cd0c9c1c2 P 233e7d29aac04505bf4e65f0a39a0c4c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:23:32.600071 25588 raft_consensus.cc:493] T 3fe5a944a342411ea923eb39071ac770 P 233e7d29aac04505bf4e65f0a39a0c4c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:23:32.600543 25590 raft_consensus.cc:697] T 0d29b499fa4c41f1b974782cd0c9c1c2 P 233e7d29aac04505bf4e65f0a39a0c4c [term 1 LEADER]: Becoming Leader. State: Replica: 233e7d29aac04505bf4e65f0a39a0c4c, State: Running, Role: LEADER
I20260709 13:23:32.600854 25573 raft_consensus.cc:515] T 1f15ec62698c4a8cbd84f2b3d516d2ea P 3e07d64466934da4bb3c02262b1c2e38 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } }
I20260709 13:23:32.601742 25503 tablet_bootstrap.cc:654] T 467a6b42ca6548269c12dddde696f7af P 233e7d29aac04505bf4e65f0a39a0c4c: Neither blocks nor log segments found. Creating new log.
I20260709 13:23:32.601651 25590 consensus_queue.cc:237] T 0d29b499fa4c41f1b974782cd0c9c1c2 P 233e7d29aac04505bf4e65f0a39a0c4c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } }
I20260709 13:23:32.578439 25504 raft_consensus.cc:385] T b7ff8dde9f5d4c49b9203d902f6b94c4 P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:23:32.604142 25504 raft_consensus.cc:740] T b7ff8dde9f5d4c49b9203d902f6b94c4 P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e07d64466934da4bb3c02262b1c2e38, State: Initialized, Role: FOLLOWER
I20260709 13:23:32.604861 25504 consensus_queue.cc:260] T b7ff8dde9f5d4c49b9203d902f6b94c4 P 3e07d64466934da4bb3c02262b1c2e38 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } }
I20260709 13:23:32.605723 25385 raft_consensus.cc:2468] T b7ff8dde9f5d4c49b9203d902f6b94c4 P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 43999d813e564dc688fad116090c95ca in term 0.
I20260709 13:23:32.609287 25458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f15ec62698c4a8cbd84f2b3d516d2ea" candidate_uuid: "3e07d64466934da4bb3c02262b1c2e38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9868dce521294d0b816bba7648f4ae5a"
I20260709 13:23:32.609378 25199 leader_election.cc:304] T b7ff8dde9f5d4c49b9203d902f6b94c4 P 43999d813e564dc688fad116090c95ca [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e07d64466934da4bb3c02262b1c2e38, 43999d813e564dc688fad116090c95ca; no voters: 
I20260709 13:23:32.610466 25510 raft_consensus.cc:2804] T b7ff8dde9f5d4c49b9203d902f6b94c4 P 43999d813e564dc688fad116090c95ca [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:23:32.610080 25134 catalog_manager.cc:5697] T e9e342ecda4b46619ce12452b2cf7154 P 9868dce521294d0b816bba7648f4ae5a reported cstate change: term changed from 0 to 1, leader changed from <none> to 9868dce521294d0b816bba7648f4ae5a (127.14.177.132). New cstate: current_term: 1 leader_uuid: "9868dce521294d0b816bba7648f4ae5a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } health_report { overall_health: UNKNOWN } } }
W20260709 13:23:32.610780 25348 leader_election.cc:343] T 1f15ec62698c4a8cbd84f2b3d516d2ea P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 9868dce521294d0b816bba7648f4ae5a (127.14.177.132:33683): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:23:32.610931 25510 raft_consensus.cc:493] T b7ff8dde9f5d4c49b9203d902f6b94c4 P 43999d813e564dc688fad116090c95ca [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:23:32.612233 25510 raft_consensus.cc:3060] T b7ff8dde9f5d4c49b9203d902f6b94c4 P 43999d813e564dc688fad116090c95ca [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:23:32.615315 25235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9e342ecda4b46619ce12452b2cf7154" candidate_uuid: "9868dce521294d0b816bba7648f4ae5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43999d813e564dc688fad116090c95ca" is_pre_election: true
I20260709 13:23:32.616827 25504 ts_tablet_manager.cc:1434] T b7ff8dde9f5d4c49b9203d902f6b94c4 P 3e07d64466934da4bb3c02262b1c2e38: Time spent starting tablet: real 0.042s	user 0.005s	sys 0.001s
I20260709 13:23:32.617300 25235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9e342ecda4b46619ce12452b2cf7154" candidate_uuid: "9868dce521294d0b816bba7648f4ae5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43999d813e564dc688fad116090c95ca"
I20260709 13:23:32.617831 25504 tablet_bootstrap.cc:492] T 3fe5a944a342411ea923eb39071ac770 P 3e07d64466934da4bb3c02262b1c2e38: Bootstrap starting.
I20260709 13:23:32.619120 25235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a3317f4c3514df1b2bb8e44a86be92f" candidate_uuid: "9868dce521294d0b816bba7648f4ae5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43999d813e564dc688fad116090c95ca" is_pre_election: true
I20260709 13:23:32.600544 25588 raft_consensus.cc:3060] T 3fe5a944a342411ea923eb39071ac770 P 233e7d29aac04505bf4e65f0a39a0c4c [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:23:32.621081 25424 leader_election.cc:343] T e9e342ecda4b46619ce12452b2cf7154 P 9868dce521294d0b816bba7648f4ae5a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 43999d813e564dc688fad116090c95ca (127.14.177.129:38155): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:23:32.620868 25236 raft_consensus.cc:2468] T 0d29b499fa4c41f1b974782cd0c9c1c2 P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 233e7d29aac04505bf4e65f0a39a0c4c in term 1.
I20260709 13:23:32.620661 25573 leader_election.cc:290] T 1f15ec62698c4a8cbd84f2b3d516d2ea P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 1 election: Requested vote from peers 9868dce521294d0b816bba7648f4ae5a (127.14.177.132:33683), 43999d813e564dc688fad116090c95ca (127.14.177.129:38155)
W20260709 13:23:32.623250 25424 leader_election.cc:343] T e9e342ecda4b46619ce12452b2cf7154 P 9868dce521294d0b816bba7648f4ae5a [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 43999d813e564dc688fad116090c95ca (127.14.177.129:38155): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:23:32.626236 25424 leader_election.cc:343] T 3a3317f4c3514df1b2bb8e44a86be92f P 9868dce521294d0b816bba7648f4ae5a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 43999d813e564dc688fad116090c95ca (127.14.177.129:38155): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:23:32.626828 25424 leader_election.cc:304] T 3a3317f4c3514df1b2bb8e44a86be92f P 9868dce521294d0b816bba7648f4ae5a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9868dce521294d0b816bba7648f4ae5a; no voters: 233e7d29aac04505bf4e65f0a39a0c4c, 43999d813e564dc688fad116090c95ca
I20260709 13:23:32.629787 25311 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7ff8dde9f5d4c49b9203d902f6b94c4" candidate_uuid: "43999d813e564dc688fad116090c95ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" is_pre_election: true
I20260709 13:23:32.628695 25539 raft_consensus.cc:2749] T 3a3317f4c3514df1b2bb8e44a86be92f P 9868dce521294d0b816bba7648f4ae5a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:23:32.634814 25311 raft_consensus.cc:2468] T b7ff8dde9f5d4c49b9203d902f6b94c4 P 233e7d29aac04505bf4e65f0a39a0c4c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 43999d813e564dc688fad116090c95ca in term 0.
I20260709 13:23:32.637933 25504 tablet_bootstrap.cc:654] T 3fe5a944a342411ea923eb39071ac770 P 3e07d64466934da4bb3c02262b1c2e38: Neither blocks nor log segments found. Creating new log.
I20260709 13:23:32.640635 25590 consensus_queue.cc:1048] T 28c0b6fc9bd64d60828d705c6a9ab1e4 P 233e7d29aac04505bf4e65f0a39a0c4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:23:32.640962 25385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe5a944a342411ea923eb39071ac770" candidate_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e07d64466934da4bb3c02262b1c2e38" is_pre_election: true
I20260709 13:23:32.641244 25133 catalog_manager.cc:5697] T 0d29b499fa4c41f1b974782cd0c9c1c2 P 233e7d29aac04505bf4e65f0a39a0c4c reported cstate change: term changed from 0 to 1, leader changed from <none> to 233e7d29aac04505bf4e65f0a39a0c4c (127.14.177.130). New cstate: current_term: 1 leader_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } health_report { overall_health: UNKNOWN } } }
I20260709 13:23:32.634897 25310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d053b9ae38de4173a0aded4da494bd98" candidate_uuid: "43999d813e564dc688fad116090c95ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" is_pre_election: true
W20260709 13:23:32.643518 25274 leader_election.cc:343] T 3fe5a944a342411ea923eb39071ac770 P 233e7d29aac04505bf4e65f0a39a0c4c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3e07d64466934da4bb3c02262b1c2e38 (127.14.177.131:41539): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:23:32.644109 25202 leader_election.cc:343] T d053b9ae38de4173a0aded4da494bd98 P 43999d813e564dc688fad116090c95ca [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 233e7d29aac04505bf4e65f0a39a0c4c (127.14.177.130:42957): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:23:32.644796 25202 leader_election.cc:304] T d053b9ae38de4173a0aded4da494bd98 P 43999d813e564dc688fad116090c95ca [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 43999d813e564dc688fad116090c95ca; no voters: 233e7d29aac04505bf4e65f0a39a0c4c, 9868dce521294d0b816bba7648f4ae5a
I20260709 13:23:32.648556 25535 raft_consensus.cc:2749] T d053b9ae38de4173a0aded4da494bd98 P 43999d813e564dc688fad116090c95ca [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:23:32.631103 25510 raft_consensus.cc:515] T b7ff8dde9f5d4c49b9203d902f6b94c4 P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } }
I20260709 13:23:32.653268 25508 tablet_bootstrap.cc:492] T 467a6b42ca6548269c12dddde696f7af P 9868dce521294d0b816bba7648f4ae5a: No bootstrap required, opened a new log
I20260709 13:23:32.653983 25508 ts_tablet_manager.cc:1403] T 467a6b42ca6548269c12dddde696f7af P 9868dce521294d0b816bba7648f4ae5a: Time spent bootstrapping tablet: real 0.187s	user 0.014s	sys 0.009s
I20260709 13:23:32.662135 25310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7ff8dde9f5d4c49b9203d902f6b94c4" candidate_uuid: "43999d813e564dc688fad116090c95ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "233e7d29aac04505bf4e65f0a39a0c4c"
I20260709 13:23:32.662002 25508 raft_consensus.cc:359] T 467a6b42ca6548269c12dddde696f7af P 9868dce521294d0b816bba7648f4ae5a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } }
I20260709 13:23:32.662864 25310 raft_consensus.cc:3060] T b7ff8dde9f5d4c49b9203d902f6b94c4 P 233e7d29aac04505bf4e65f0a39a0c4c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:23:32.666235 25236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f15ec62698c4a8cbd84f2b3d516d2ea" candidate_uuid: "3e07d64466934da4bb3c02262b1c2e38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43999d813e564dc688fad116090c95ca"
I20260709 13:23:32.667088 25236 raft_consensus.cc:3060] T 1f15ec62698c4a8cbd84f2b3d516d2ea P 43999d813e564dc688fad116090c95ca [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:23:32.672362 25510 leader_election.cc:290] T b7ff8dde9f5d4c49b9203d902f6b94c4 P 43999d813e564dc688fad116090c95ca [CANDIDATE]: Term 1 election: Requested vote from peers 233e7d29aac04505bf4e65f0a39a0c4c (127.14.177.130:42957), 3e07d64466934da4bb3c02262b1c2e38 (127.14.177.131:41539)
I20260709 13:23:32.672838 25310 raft_consensus.cc:2468] T b7ff8dde9f5d4c49b9203d902f6b94c4 P 233e7d29aac04505bf4e65f0a39a0c4c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 43999d813e564dc688fad116090c95ca in term 1.
I20260709 13:23:32.673552 25384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7ff8dde9f5d4c49b9203d902f6b94c4" candidate_uuid: "43999d813e564dc688fad116090c95ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e07d64466934da4bb3c02262b1c2e38"
I20260709 13:23:32.674189 25384 raft_consensus.cc:3060] T b7ff8dde9f5d4c49b9203d902f6b94c4 P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:23:32.674235 25202 leader_election.cc:304] T b7ff8dde9f5d4c49b9203d902f6b94c4 P 43999d813e564dc688fad116090c95ca [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 233e7d29aac04505bf4e65f0a39a0c4c, 43999d813e564dc688fad116090c95ca; no voters: 
I20260709 13:23:32.675328 25510 raft_consensus.cc:2804] T b7ff8dde9f5d4c49b9203d902f6b94c4 P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:23:32.675832 25510 raft_consensus.cc:697] T b7ff8dde9f5d4c49b9203d902f6b94c4 P 43999d813e564dc688fad116090c95ca [term 1 LEADER]: Becoming Leader. State: Replica: 43999d813e564dc688fad116090c95ca, State: Running, Role: LEADER
I20260709 13:23:32.675854 25505 tablet_bootstrap.cc:492] T 3a3317f4c3514df1b2bb8e44a86be92f P 43999d813e564dc688fad116090c95ca: No bootstrap required, opened a new log
I20260709 13:23:32.676869 25505 ts_tablet_manager.cc:1403] T 3a3317f4c3514df1b2bb8e44a86be92f P 43999d813e564dc688fad116090c95ca: Time spent bootstrapping tablet: real 0.199s	user 0.035s	sys 0.016s
I20260709 13:23:32.676939 25510 consensus_queue.cc:237] T b7ff8dde9f5d4c49b9203d902f6b94c4 P 43999d813e564dc688fad116090c95ca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } }
I20260709 13:23:32.662998 25508 raft_consensus.cc:385] T 467a6b42ca6548269c12dddde696f7af P 9868dce521294d0b816bba7648f4ae5a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:23:32.682310 25508 raft_consensus.cc:740] T 467a6b42ca6548269c12dddde696f7af P 9868dce521294d0b816bba7648f4ae5a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9868dce521294d0b816bba7648f4ae5a, State: Initialized, Role: FOLLOWER
I20260709 13:23:32.683359 25504 tablet_bootstrap.cc:492] T 3fe5a944a342411ea923eb39071ac770 P 3e07d64466934da4bb3c02262b1c2e38: No bootstrap required, opened a new log
I20260709 13:23:32.683871 25504 ts_tablet_manager.cc:1403] T 3fe5a944a342411ea923eb39071ac770 P 3e07d64466934da4bb3c02262b1c2e38: Time spent bootstrapping tablet: real 0.066s	user 0.019s	sys 0.031s
I20260709 13:23:32.662887 25588 raft_consensus.cc:515] T 3fe5a944a342411ea923eb39071ac770 P 233e7d29aac04505bf4e65f0a39a0c4c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } }
I20260709 13:23:32.686693 25504 raft_consensus.cc:359] T 3fe5a944a342411ea923eb39071ac770 P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } }
I20260709 13:23:32.687428 25504 raft_consensus.cc:385] T 3fe5a944a342411ea923eb39071ac770 P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:23:32.627125 25559 raft_consensus.cc:515] T 579da2d6aabb4ae9a2b1802e1358068b P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } }
I20260709 13:23:32.682983 25508 consensus_queue.cc:260] T 467a6b42ca6548269c12dddde696f7af P 9868dce521294d0b816bba7648f4ae5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } }
I20260709 13:23:32.687719 25504 raft_consensus.cc:740] T 3fe5a944a342411ea923eb39071ac770 P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e07d64466934da4bb3c02262b1c2e38, State: Initialized, Role: FOLLOWER
I20260709 13:23:32.688813 25504 consensus_queue.cc:260] T 3fe5a944a342411ea923eb39071ac770 P 3e07d64466934da4bb3c02262b1c2e38 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } }
I20260709 13:23:32.689841 25505 raft_consensus.cc:359] T 3a3317f4c3514df1b2bb8e44a86be92f P 43999d813e564dc688fad116090c95ca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } }
I20260709 13:23:32.690536 25505 raft_consensus.cc:385] T 3a3317f4c3514df1b2bb8e44a86be92f P 43999d813e564dc688fad116090c95ca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:23:32.690801 25505 raft_consensus.cc:740] T 3a3317f4c3514df1b2bb8e44a86be92f P 43999d813e564dc688fad116090c95ca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43999d813e564dc688fad116090c95ca, State: Initialized, Role: FOLLOWER
I20260709 13:23:32.690979 25504 ts_tablet_manager.cc:1434] T 3fe5a944a342411ea923eb39071ac770 P 3e07d64466934da4bb3c02262b1c2e38: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.000s
I20260709 13:23:32.691525 25505 consensus_queue.cc:260] T 3a3317f4c3514df1b2bb8e44a86be92f P 43999d813e564dc688fad116090c95ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } }
I20260709 13:23:32.691960 25504 tablet_bootstrap.cc:492] T ba8a000f5abe4158965449995af84b18 P 3e07d64466934da4bb3c02262b1c2e38: Bootstrap starting.
I20260709 13:23:32.692274 25384 raft_consensus.cc:2468] T b7ff8dde9f5d4c49b9203d902f6b94c4 P 3e07d64466934da4bb3c02262b1c2e38 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 43999d813e564dc688fad116090c95ca in term 1.
I20260709 13:23:32.693611 25505 ts_tablet_manager.cc:1434] T 3a3317f4c3514df1b2bb8e44a86be92f P 43999d813e564dc688fad116090c95ca: Time spent starting tablet: real 0.016s	user 0.006s	sys 0.000s
I20260709 13:23:32.695042 25505 tablet_bootstrap.cc:492] T 28c0b6fc9bd64d60828d705c6a9ab1e4 P 43999d813e564dc688fad116090c95ca: Bootstrap starting.
I20260709 13:23:32.698484 25504 tablet_bootstrap.cc:654] T ba8a000f5abe4158965449995af84b18 P 3e07d64466934da4bb3c02262b1c2e38: Neither blocks nor log segments found. Creating new log.
I20260709 13:23:32.699434 25559 leader_election.cc:290] T 579da2d6aabb4ae9a2b1802e1358068b P 43999d813e564dc688fad116090c95ca [CANDIDATE]: Term 1 election: Requested vote from peers 9868dce521294d0b816bba7648f4ae5a (127.14.177.132:33683), 233e7d29aac04505bf4e65f0a39a0c4c (127.14.177.130:42957)
I20260709 13:23:32.700026 25458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "579da2d6aabb4ae9a2b1802e1358068b" candidate_uuid: "43999d813e564dc688fad116090c95ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9868dce521294d0b816bba7648f4ae5a"
I20260709 13:23:32.700409 25310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "579da2d6aabb4ae9a2b1802e1358068b" candidate_uuid: "43999d813e564dc688fad116090c95ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "233e7d29aac04505bf4e65f0a39a0c4c"
I20260709 13:23:32.701092 25310 raft_consensus.cc:3060] T 579da2d6aabb4ae9a2b1802e1358068b P 233e7d29aac04505bf4e65f0a39a0c4c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:23:32.703763 25508 ts_tablet_manager.cc:1434] T 467a6b42ca6548269c12dddde696f7af P 9868dce521294d0b816bba7648f4ae5a: Time spent starting tablet: real 0.049s	user 0.007s	sys 0.001s
I20260709 13:23:32.704914 25588 leader_election.cc:290] T 3fe5a944a342411ea923eb39071ac770 P 233e7d29aac04505bf4e65f0a39a0c4c [CANDIDATE]: Term 1 election: Requested vote from peers 3e07d64466934da4bb3c02262b1c2e38 (127.14.177.131:41539), 9868dce521294d0b816bba7648f4ae5a (127.14.177.132:33683)
I20260709 13:23:32.705744 25505 tablet_bootstrap.cc:654] T 28c0b6fc9bd64d60828d705c6a9ab1e4 P 43999d813e564dc688fad116090c95ca: Neither blocks nor log segments found. Creating new log.
W20260709 13:23:32.707899 25199 leader_election.cc:343] T 579da2d6aabb4ae9a2b1802e1358068b P 43999d813e564dc688fad116090c95ca [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 9868dce521294d0b816bba7648f4ae5a (127.14.177.132:33683): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:23:32.709378 25236 raft_consensus.cc:2468] T 1f15ec62698c4a8cbd84f2b3d516d2ea P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3e07d64466934da4bb3c02262b1c2e38 in term 1.
I20260709 13:23:32.711092 25350 leader_election.cc:304] T 1f15ec62698c4a8cbd84f2b3d516d2ea P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3e07d64466934da4bb3c02262b1c2e38, 43999d813e564dc688fad116090c95ca; no voters: 9868dce521294d0b816bba7648f4ae5a
I20260709 13:23:32.711709 25310 raft_consensus.cc:2468] T 579da2d6aabb4ae9a2b1802e1358068b P 233e7d29aac04505bf4e65f0a39a0c4c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 43999d813e564dc688fad116090c95ca in term 1.
I20260709 13:23:32.712744 25573 raft_consensus.cc:2804] T 1f15ec62698c4a8cbd84f2b3d516d2ea P 3e07d64466934da4bb3c02262b1c2e38 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:23:32.712891 25202 leader_election.cc:304] T 579da2d6aabb4ae9a2b1802e1358068b P 43999d813e564dc688fad116090c95ca [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 233e7d29aac04505bf4e65f0a39a0c4c, 43999d813e564dc688fad116090c95ca; no voters: 9868dce521294d0b816bba7648f4ae5a
I20260709 13:23:32.713316 25573 raft_consensus.cc:697] T 1f15ec62698c4a8cbd84f2b3d516d2ea P 3e07d64466934da4bb3c02262b1c2e38 [term 1 LEADER]: Becoming Leader. State: Replica: 3e07d64466934da4bb3c02262b1c2e38, State: Running, Role: LEADER
I20260709 13:23:32.714251 25573 consensus_queue.cc:237] T 1f15ec62698c4a8cbd84f2b3d516d2ea P 3e07d64466934da4bb3c02262b1c2e38 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } }
I20260709 13:23:32.716625 25559 raft_consensus.cc:2804] T 579da2d6aabb4ae9a2b1802e1358068b P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:23:32.716915 25384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe5a944a342411ea923eb39071ac770" candidate_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e07d64466934da4bb3c02262b1c2e38"
I20260709 13:23:32.718712 25559 raft_consensus.cc:697] T 579da2d6aabb4ae9a2b1802e1358068b P 43999d813e564dc688fad116090c95ca [term 1 LEADER]: Becoming Leader. State: Replica: 43999d813e564dc688fad116090c95ca, State: Running, Role: LEADER
I20260709 13:23:32.719002 25384 raft_consensus.cc:3060] T 3fe5a944a342411ea923eb39071ac770 P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:23:32.720244 25559 consensus_queue.cc:237] T 579da2d6aabb4ae9a2b1802e1358068b P 43999d813e564dc688fad116090c95ca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } }
I20260709 13:23:32.721674 25503 tablet_bootstrap.cc:492] T 467a6b42ca6548269c12dddde696f7af P 233e7d29aac04505bf4e65f0a39a0c4c: No bootstrap required, opened a new log
I20260709 13:23:32.722227 25503 ts_tablet_manager.cc:1403] T 467a6b42ca6548269c12dddde696f7af P 233e7d29aac04505bf4e65f0a39a0c4c: Time spent bootstrapping tablet: real 0.169s	user 0.014s	sys 0.025s
I20260709 13:23:32.725103 25503 raft_consensus.cc:359] T 467a6b42ca6548269c12dddde696f7af P 233e7d29aac04505bf4e65f0a39a0c4c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } }
I20260709 13:23:32.725843 25503 raft_consensus.cc:385] T 467a6b42ca6548269c12dddde696f7af P 233e7d29aac04505bf4e65f0a39a0c4c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:23:32.726135 25503 raft_consensus.cc:740] T 467a6b42ca6548269c12dddde696f7af P 233e7d29aac04505bf4e65f0a39a0c4c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 233e7d29aac04505bf4e65f0a39a0c4c, State: Initialized, Role: FOLLOWER
I20260709 13:23:32.734366 25310 raft_consensus.cc:3060] T 441d780c3945463f92f31170f31e37a8 P 233e7d29aac04505bf4e65f0a39a0c4c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:23:32.735095 25503 consensus_queue.cc:260] T 467a6b42ca6548269c12dddde696f7af P 233e7d29aac04505bf4e65f0a39a0c4c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } }
I20260709 13:23:32.737960 25458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe5a944a342411ea923eb39071ac770" candidate_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9868dce521294d0b816bba7648f4ae5a"
I20260709 13:23:32.739570 25508 tablet_bootstrap.cc:492] T 0e71b6f5c69b4b35a8ac51c22c7d5f0d P 9868dce521294d0b816bba7648f4ae5a: Bootstrap starting.
I20260709 13:23:32.739817 25503 ts_tablet_manager.cc:1434] T 467a6b42ca6548269c12dddde696f7af P 233e7d29aac04505bf4e65f0a39a0c4c: Time spent starting tablet: real 0.017s	user 0.009s	sys 0.000s
I20260709 13:23:32.740818 25503 tablet_bootstrap.cc:492] T e15820fa8a7d4e1fb6ad23a64ca22f90 P 233e7d29aac04505bf4e65f0a39a0c4c: Bootstrap starting.
I20260709 13:23:32.739334 25458 raft_consensus.cc:3060] T 3fe5a944a342411ea923eb39071ac770 P 9868dce521294d0b816bba7648f4ae5a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:23:32.755981 25508 tablet_bootstrap.cc:654] T 0e71b6f5c69b4b35a8ac51c22c7d5f0d P 9868dce521294d0b816bba7648f4ae5a: Neither blocks nor log segments found. Creating new log.
I20260709 13:23:32.780597 25504 tablet_bootstrap.cc:492] T ba8a000f5abe4158965449995af84b18 P 3e07d64466934da4bb3c02262b1c2e38: No bootstrap required, opened a new log
I20260709 13:23:32.781136 25504 ts_tablet_manager.cc:1403] T ba8a000f5abe4158965449995af84b18 P 3e07d64466934da4bb3c02262b1c2e38: Time spent bootstrapping tablet: real 0.089s	user 0.010s	sys 0.019s
I20260709 13:23:32.758005 25384 raft_consensus.cc:2468] T 3fe5a944a342411ea923eb39071ac770 P 3e07d64466934da4bb3c02262b1c2e38 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 233e7d29aac04505bf4e65f0a39a0c4c in term 1.
I20260709 13:23:32.788570 25133 catalog_manager.cc:5697] T b7ff8dde9f5d4c49b9203d902f6b94c4 P 43999d813e564dc688fad116090c95ca reported cstate change: term changed from 0 to 1, leader changed from <none> to 43999d813e564dc688fad116090c95ca (127.14.177.129). New cstate: current_term: 1 leader_uuid: "43999d813e564dc688fad116090c95ca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } health_report { overall_health: UNKNOWN } } }
I20260709 13:23:32.791512 25274 leader_election.cc:304] T 3fe5a944a342411ea923eb39071ac770 P 233e7d29aac04505bf4e65f0a39a0c4c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 233e7d29aac04505bf4e65f0a39a0c4c, 3e07d64466934da4bb3c02262b1c2e38; no voters: 
I20260709 13:23:32.792342 25590 raft_consensus.cc:2804] T 3fe5a944a342411ea923eb39071ac770 P 233e7d29aac04505bf4e65f0a39a0c4c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:23:32.792938 25590 raft_consensus.cc:697] T 3fe5a944a342411ea923eb39071ac770 P 233e7d29aac04505bf4e65f0a39a0c4c [term 1 LEADER]: Becoming Leader. State: Replica: 233e7d29aac04505bf4e65f0a39a0c4c, State: Running, Role: LEADER
I20260709 13:23:32.793759 25590 consensus_queue.cc:237] T 3fe5a944a342411ea923eb39071ac770 P 233e7d29aac04505bf4e65f0a39a0c4c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } }
W20260709 13:23:32.799006 25604 log.cc:927] Time spent T 579da2d6aabb4ae9a2b1802e1358068b P 43999d813e564dc688fad116090c95ca: Append to log took a long time: real 0.061s	user 0.001s	sys 0.000s
I20260709 13:23:32.805864 25503 tablet_bootstrap.cc:654] T e15820fa8a7d4e1fb6ad23a64ca22f90 P 233e7d29aac04505bf4e65f0a39a0c4c: Neither blocks nor log segments found. Creating new log.
I20260709 13:23:32.809171 25458 raft_consensus.cc:2468] T 3fe5a944a342411ea923eb39071ac770 P 9868dce521294d0b816bba7648f4ae5a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 233e7d29aac04505bf4e65f0a39a0c4c in term 1.
I20260709 13:23:32.817796 25504 raft_consensus.cc:359] T ba8a000f5abe4158965449995af84b18 P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } }
I20260709 13:23:32.818833 25504 raft_consensus.cc:385] T ba8a000f5abe4158965449995af84b18 P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260709 13:23:32.821628 25199 consensus_peers.cc:597] T 0e71b6f5c69b4b35a8ac51c22c7d5f0d P 43999d813e564dc688fad116090c95ca -> Peer 9868dce521294d0b816bba7648f4ae5a (127.14.177.132:33683): Couldn't send request to peer 9868dce521294d0b816bba7648f4ae5a. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:23:32.832042 25503 tablet_bootstrap.cc:492] T e15820fa8a7d4e1fb6ad23a64ca22f90 P 233e7d29aac04505bf4e65f0a39a0c4c: No bootstrap required, opened a new log
I20260709 13:23:32.827729 25131 catalog_manager.cc:5697] T 579da2d6aabb4ae9a2b1802e1358068b P 43999d813e564dc688fad116090c95ca reported cstate change: term changed from 0 to 1, leader changed from <none> to 43999d813e564dc688fad116090c95ca (127.14.177.129). New cstate: current_term: 1 leader_uuid: "43999d813e564dc688fad116090c95ca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } health_report { overall_health: UNKNOWN } } }
I20260709 13:23:32.832639 25503 ts_tablet_manager.cc:1403] T e15820fa8a7d4e1fb6ad23a64ca22f90 P 233e7d29aac04505bf4e65f0a39a0c4c: Time spent bootstrapping tablet: real 0.092s	user 0.012s	sys 0.020s
I20260709 13:23:32.830837 25508 tablet_bootstrap.cc:492] T 0e71b6f5c69b4b35a8ac51c22c7d5f0d P 9868dce521294d0b816bba7648f4ae5a: No bootstrap required, opened a new log
I20260709 13:23:32.832978 25602 consensus_queue.cc:1048] T 0e71b6f5c69b4b35a8ac51c22c7d5f0d P 43999d813e564dc688fad116090c95ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:23:32.833506 25508 ts_tablet_manager.cc:1403] T 0e71b6f5c69b4b35a8ac51c22c7d5f0d P 9868dce521294d0b816bba7648f4ae5a: Time spent bootstrapping tablet: real 0.096s	user 0.013s	sys 0.000s
I20260709 13:23:32.835558 25503 raft_consensus.cc:359] T e15820fa8a7d4e1fb6ad23a64ca22f90 P 233e7d29aac04505bf4e65f0a39a0c4c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } }
I20260709 13:23:32.836297 25503 raft_consensus.cc:385] T e15820fa8a7d4e1fb6ad23a64ca22f90 P 233e7d29aac04505bf4e65f0a39a0c4c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:23:32.836649 25503 raft_consensus.cc:740] T e15820fa8a7d4e1fb6ad23a64ca22f90 P 233e7d29aac04505bf4e65f0a39a0c4c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 233e7d29aac04505bf4e65f0a39a0c4c, State: Initialized, Role: FOLLOWER
I20260709 13:23:32.836596 25508 raft_consensus.cc:359] T 0e71b6f5c69b4b35a8ac51c22c7d5f0d P 9868dce521294d0b816bba7648f4ae5a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } }
I20260709 13:23:32.837764 25508 raft_consensus.cc:385] T 0e71b6f5c69b4b35a8ac51c22c7d5f0d P 9868dce521294d0b816bba7648f4ae5a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:23:32.819156 25504 raft_consensus.cc:740] T ba8a000f5abe4158965449995af84b18 P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e07d64466934da4bb3c02262b1c2e38, State: Initialized, Role: FOLLOWER
I20260709 13:23:32.825178 25134 catalog_manager.cc:5697] T 1f15ec62698c4a8cbd84f2b3d516d2ea P 3e07d64466934da4bb3c02262b1c2e38 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3e07d64466934da4bb3c02262b1c2e38 (127.14.177.131). New cstate: current_term: 1 leader_uuid: "3e07d64466934da4bb3c02262b1c2e38" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } health_report { overall_health: UNKNOWN } } }
I20260709 13:23:32.838166 25508 raft_consensus.cc:740] T 0e71b6f5c69b4b35a8ac51c22c7d5f0d P 9868dce521294d0b816bba7648f4ae5a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9868dce521294d0b816bba7648f4ae5a, State: Initialized, Role: FOLLOWER
I20260709 13:23:32.839352 25504 consensus_queue.cc:260] T ba8a000f5abe4158965449995af84b18 P 3e07d64466934da4bb3c02262b1c2e38 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } }
I20260709 13:23:32.840533 25508 consensus_queue.cc:260] T 0e71b6f5c69b4b35a8ac51c22c7d5f0d P 9868dce521294d0b816bba7648f4ae5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } }
I20260709 13:23:32.837779 25503 consensus_queue.cc:260] T e15820fa8a7d4e1fb6ad23a64ca22f90 P 233e7d29aac04505bf4e65f0a39a0c4c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } }
I20260709 13:23:32.843184 25503 ts_tablet_manager.cc:1434] T e15820fa8a7d4e1fb6ad23a64ca22f90 P 233e7d29aac04505bf4e65f0a39a0c4c: Time spent starting tablet: real 0.010s	user 0.006s	sys 0.000s
I20260709 13:23:32.845683 25503 tablet_bootstrap.cc:492] T a820954e07a94582ac0613342fcffde5 P 233e7d29aac04505bf4e65f0a39a0c4c: Bootstrap starting.
I20260709 13:23:32.844647 25504 ts_tablet_manager.cc:1434] T ba8a000f5abe4158965449995af84b18 P 3e07d64466934da4bb3c02262b1c2e38: Time spent starting tablet: real 0.063s	user 0.008s	sys 0.001s
I20260709 13:23:32.848368 25504 tablet_bootstrap.cc:492] T e15820fa8a7d4e1fb6ad23a64ca22f90 P 3e07d64466934da4bb3c02262b1c2e38: Bootstrap starting.
I20260709 13:23:32.827890 25505 tablet_bootstrap.cc:492] T 28c0b6fc9bd64d60828d705c6a9ab1e4 P 43999d813e564dc688fad116090c95ca: No bootstrap required, opened a new log
I20260709 13:23:32.848979 25505 ts_tablet_manager.cc:1403] T 28c0b6fc9bd64d60828d705c6a9ab1e4 P 43999d813e564dc688fad116090c95ca: Time spent bootstrapping tablet: real 0.154s	user 0.009s	sys 0.007s
I20260709 13:23:32.851800 25505 raft_consensus.cc:359] T 28c0b6fc9bd64d60828d705c6a9ab1e4 P 43999d813e564dc688fad116090c95ca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } }
I20260709 13:23:32.852358 25508 ts_tablet_manager.cc:1434] T 0e71b6f5c69b4b35a8ac51c22c7d5f0d P 9868dce521294d0b816bba7648f4ae5a: Time spent starting tablet: real 0.019s	user 0.008s	sys 0.000s
I20260709 13:23:32.855407 25505 raft_consensus.cc:385] T 28c0b6fc9bd64d60828d705c6a9ab1e4 P 43999d813e564dc688fad116090c95ca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:23:32.852762 25133 catalog_manager.cc:5697] T 3fe5a944a342411ea923eb39071ac770 P 233e7d29aac04505bf4e65f0a39a0c4c reported cstate change: term changed from 0 to 1, leader changed from <none> to 233e7d29aac04505bf4e65f0a39a0c4c (127.14.177.130). New cstate: current_term: 1 leader_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } health_report { overall_health: UNKNOWN } } }
I20260709 13:23:32.855808 25505 raft_consensus.cc:740] T 28c0b6fc9bd64d60828d705c6a9ab1e4 P 43999d813e564dc688fad116090c95ca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43999d813e564dc688fad116090c95ca, State: Initialized, Role: FOLLOWER
I20260709 13:23:32.858911 25505 consensus_queue.cc:260] T 28c0b6fc9bd64d60828d705c6a9ab1e4 P 43999d813e564dc688fad116090c95ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } }
I20260709 13:23:32.863198 25505 ts_tablet_manager.cc:1434] T 28c0b6fc9bd64d60828d705c6a9ab1e4 P 43999d813e564dc688fad116090c95ca: Time spent starting tablet: real 0.014s	user 0.005s	sys 0.000s
I20260709 13:23:32.864254 25505 tablet_bootstrap.cc:492] T e15820fa8a7d4e1fb6ad23a64ca22f90 P 43999d813e564dc688fad116090c95ca: Bootstrap starting.
I20260709 13:23:32.857928 25508 tablet_bootstrap.cc:492] T 579da2d6aabb4ae9a2b1802e1358068b P 9868dce521294d0b816bba7648f4ae5a: Bootstrap starting.
I20260709 13:23:32.869666 25539 raft_consensus.cc:493] T 0e71b6f5c69b4b35a8ac51c22c7d5f0d P 9868dce521294d0b816bba7648f4ae5a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:23:32.864549 25503 tablet_bootstrap.cc:654] T a820954e07a94582ac0613342fcffde5 P 233e7d29aac04505bf4e65f0a39a0c4c: Neither blocks nor log segments found. Creating new log.
I20260709 13:23:32.858080 25504 tablet_bootstrap.cc:654] T e15820fa8a7d4e1fb6ad23a64ca22f90 P 3e07d64466934da4bb3c02262b1c2e38: Neither blocks nor log segments found. Creating new log.
I20260709 13:23:32.870217 25539 raft_consensus.cc:515] T 0e71b6f5c69b4b35a8ac51c22c7d5f0d P 9868dce521294d0b816bba7648f4ae5a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } }
I20260709 13:23:32.872567 25539 leader_election.cc:290] T 0e71b6f5c69b4b35a8ac51c22c7d5f0d P 9868dce521294d0b816bba7648f4ae5a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3e07d64466934da4bb3c02262b1c2e38 (127.14.177.131:41539), 43999d813e564dc688fad116090c95ca (127.14.177.129:38155)
I20260709 13:23:32.874204 25385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e71b6f5c69b4b35a8ac51c22c7d5f0d" candidate_uuid: "9868dce521294d0b816bba7648f4ae5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e07d64466934da4bb3c02262b1c2e38" is_pre_election: true
I20260709 13:23:32.874408 25236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e71b6f5c69b4b35a8ac51c22c7d5f0d" candidate_uuid: "9868dce521294d0b816bba7648f4ae5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43999d813e564dc688fad116090c95ca" is_pre_election: true
I20260709 13:23:32.876276 25422 leader_election.cc:304] T 0e71b6f5c69b4b35a8ac51c22c7d5f0d P 9868dce521294d0b816bba7648f4ae5a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9868dce521294d0b816bba7648f4ae5a; no voters: 3e07d64466934da4bb3c02262b1c2e38, 43999d813e564dc688fad116090c95ca
I20260709 13:23:32.877516 25539 raft_consensus.cc:2749] T 0e71b6f5c69b4b35a8ac51c22c7d5f0d P 9868dce521294d0b816bba7648f4ae5a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:23:32.878547 25590 raft_consensus.cc:493] T e15820fa8a7d4e1fb6ad23a64ca22f90 P 233e7d29aac04505bf4e65f0a39a0c4c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:23:32.879034 25590 raft_consensus.cc:515] T e15820fa8a7d4e1fb6ad23a64ca22f90 P 233e7d29aac04505bf4e65f0a39a0c4c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } }
I20260709 13:23:32.879773 25508 tablet_bootstrap.cc:654] T 579da2d6aabb4ae9a2b1802e1358068b P 9868dce521294d0b816bba7648f4ae5a: Neither blocks nor log segments found. Creating new log.
I20260709 13:23:32.882064 25590 leader_election.cc:290] T e15820fa8a7d4e1fb6ad23a64ca22f90 P 233e7d29aac04505bf4e65f0a39a0c4c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 43999d813e564dc688fad116090c95ca (127.14.177.129:38155), 3e07d64466934da4bb3c02262b1c2e38 (127.14.177.131:41539)
I20260709 13:23:32.884073 25385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15820fa8a7d4e1fb6ad23a64ca22f90" candidate_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e07d64466934da4bb3c02262b1c2e38" is_pre_election: true
I20260709 13:23:32.886133 25236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15820fa8a7d4e1fb6ad23a64ca22f90" candidate_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43999d813e564dc688fad116090c95ca" is_pre_election: true
W20260709 13:23:32.888258 25274 leader_election.cc:343] T e15820fa8a7d4e1fb6ad23a64ca22f90 P 233e7d29aac04505bf4e65f0a39a0c4c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3e07d64466934da4bb3c02262b1c2e38 (127.14.177.131:41539): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:23:32.889138 25276 leader_election.cc:343] T e15820fa8a7d4e1fb6ad23a64ca22f90 P 233e7d29aac04505bf4e65f0a39a0c4c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 43999d813e564dc688fad116090c95ca (127.14.177.129:38155): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:23:32.889619 25276 leader_election.cc:304] T e15820fa8a7d4e1fb6ad23a64ca22f90 P 233e7d29aac04505bf4e65f0a39a0c4c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 233e7d29aac04505bf4e65f0a39a0c4c; no voters: 3e07d64466934da4bb3c02262b1c2e38, 43999d813e564dc688fad116090c95ca
I20260709 13:23:32.890424 25590 raft_consensus.cc:2749] T e15820fa8a7d4e1fb6ad23a64ca22f90 P 233e7d29aac04505bf4e65f0a39a0c4c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:23:32.891729 25539 raft_consensus.cc:493] T a820954e07a94582ac0613342fcffde5 P 9868dce521294d0b816bba7648f4ae5a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:23:32.892242 25505 tablet_bootstrap.cc:654] T e15820fa8a7d4e1fb6ad23a64ca22f90 P 43999d813e564dc688fad116090c95ca: Neither blocks nor log segments found. Creating new log.
I20260709 13:23:32.892243 25539 raft_consensus.cc:515] T a820954e07a94582ac0613342fcffde5 P 9868dce521294d0b816bba7648f4ae5a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } }
I20260709 13:23:32.896263 25568 raft_consensus.cc:493] T b2f14aed05e24a98ad83535029b795dc P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:23:32.896453 25539 leader_election.cc:290] T a820954e07a94582ac0613342fcffde5 P 9868dce521294d0b816bba7648f4ae5a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 233e7d29aac04505bf4e65f0a39a0c4c (127.14.177.130:42957), 3e07d64466934da4bb3c02262b1c2e38 (127.14.177.131:41539)
I20260709 13:23:32.896831 25568 raft_consensus.cc:515] T b2f14aed05e24a98ad83535029b795dc P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } }
I20260709 13:23:32.897688 25310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a820954e07a94582ac0613342fcffde5" candidate_uuid: "9868dce521294d0b816bba7648f4ae5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" is_pre_election: true
I20260709 13:23:32.898092 25503 tablet_bootstrap.cc:492] T a820954e07a94582ac0613342fcffde5 P 233e7d29aac04505bf4e65f0a39a0c4c: No bootstrap required, opened a new log
I20260709 13:23:32.898622 25503 ts_tablet_manager.cc:1403] T a820954e07a94582ac0613342fcffde5 P 233e7d29aac04505bf4e65f0a39a0c4c: Time spent bootstrapping tablet: real 0.053s	user 0.010s	sys 0.001s
W20260709 13:23:32.899024 25425 leader_election.cc:343] T a820954e07a94582ac0613342fcffde5 P 9868dce521294d0b816bba7648f4ae5a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 233e7d29aac04505bf4e65f0a39a0c4c (127.14.177.130:42957): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:23:32.900585 25310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2f14aed05e24a98ad83535029b795dc" candidate_uuid: "3e07d64466934da4bb3c02262b1c2e38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" is_pre_election: true
I20260709 13:23:32.901319 25310 raft_consensus.cc:2468] T b2f14aed05e24a98ad83535029b795dc P 233e7d29aac04505bf4e65f0a39a0c4c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3e07d64466934da4bb3c02262b1c2e38 in term 0.
I20260709 13:23:32.901435 25503 raft_consensus.cc:359] T a820954e07a94582ac0613342fcffde5 P 233e7d29aac04505bf4e65f0a39a0c4c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } }
I20260709 13:23:32.902345 25503 raft_consensus.cc:385] T a820954e07a94582ac0613342fcffde5 P 233e7d29aac04505bf4e65f0a39a0c4c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:23:32.902654 25503 raft_consensus.cc:740] T a820954e07a94582ac0613342fcffde5 P 233e7d29aac04505bf4e65f0a39a0c4c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 233e7d29aac04505bf4e65f0a39a0c4c, State: Initialized, Role: FOLLOWER
I20260709 13:23:32.902673 25351 leader_election.cc:304] T b2f14aed05e24a98ad83535029b795dc P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 233e7d29aac04505bf4e65f0a39a0c4c, 3e07d64466934da4bb3c02262b1c2e38; no voters: 
I20260709 13:23:32.903553 25236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2f14aed05e24a98ad83535029b795dc" candidate_uuid: "3e07d64466934da4bb3c02262b1c2e38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43999d813e564dc688fad116090c95ca" is_pre_election: true
I20260709 13:23:32.903690 25573 raft_consensus.cc:2804] T b2f14aed05e24a98ad83535029b795dc P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:23:32.904304 25573 raft_consensus.cc:493] T b2f14aed05e24a98ad83535029b795dc P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:23:32.904726 25573 raft_consensus.cc:3060] T b2f14aed05e24a98ad83535029b795dc P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:23:32.905390 25350 leader_election.cc:343] T b2f14aed05e24a98ad83535029b795dc P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 43999d813e564dc688fad116090c95ca (127.14.177.129:38155): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:23:32.898897 25568 leader_election.cc:290] T b2f14aed05e24a98ad83535029b795dc P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 233e7d29aac04505bf4e65f0a39a0c4c (127.14.177.130:42957), 43999d813e564dc688fad116090c95ca (127.14.177.129:38155)
I20260709 13:23:32.906705 25508 tablet_bootstrap.cc:492] T 579da2d6aabb4ae9a2b1802e1358068b P 9868dce521294d0b816bba7648f4ae5a: No bootstrap required, opened a new log
I20260709 13:23:32.907222 25508 ts_tablet_manager.cc:1403] T 579da2d6aabb4ae9a2b1802e1358068b P 9868dce521294d0b816bba7648f4ae5a: Time spent bootstrapping tablet: real 0.050s	user 0.013s	sys 0.000s
I20260709 13:23:32.907968 25504 tablet_bootstrap.cc:492] T e15820fa8a7d4e1fb6ad23a64ca22f90 P 3e07d64466934da4bb3c02262b1c2e38: No bootstrap required, opened a new log
I20260709 13:23:32.908442 25504 ts_tablet_manager.cc:1403] T e15820fa8a7d4e1fb6ad23a64ca22f90 P 3e07d64466934da4bb3c02262b1c2e38: Time spent bootstrapping tablet: real 0.060s	user 0.010s	sys 0.001s
I20260709 13:23:32.910248 25508 raft_consensus.cc:359] T 579da2d6aabb4ae9a2b1802e1358068b P 9868dce521294d0b816bba7648f4ae5a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } }
I20260709 13:23:32.901232 25385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a820954e07a94582ac0613342fcffde5" candidate_uuid: "9868dce521294d0b816bba7648f4ae5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e07d64466934da4bb3c02262b1c2e38" is_pre_election: true
I20260709 13:23:32.911540 25385 raft_consensus.cc:2468] T a820954e07a94582ac0613342fcffde5 P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9868dce521294d0b816bba7648f4ae5a in term 0.
I20260709 13:23:32.911676 25573 raft_consensus.cc:515] T b2f14aed05e24a98ad83535029b795dc P 3e07d64466934da4bb3c02262b1c2e38 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } }
I20260709 13:23:32.912197 25503 consensus_queue.cc:260] T a820954e07a94582ac0613342fcffde5 P 233e7d29aac04505bf4e65f0a39a0c4c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } }
I20260709 13:23:32.912978 25422 leader_election.cc:304] T a820954e07a94582ac0613342fcffde5 P 9868dce521294d0b816bba7648f4ae5a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3e07d64466934da4bb3c02262b1c2e38, 9868dce521294d0b816bba7648f4ae5a; no voters: 233e7d29aac04505bf4e65f0a39a0c4c
I20260709 13:23:32.914297 25503 ts_tablet_manager.cc:1434] T a820954e07a94582ac0613342fcffde5 P 233e7d29aac04505bf4e65f0a39a0c4c: Time spent starting tablet: real 0.015s	user 0.006s	sys 0.000s
I20260709 13:23:32.910990 25508 raft_consensus.cc:385] T 579da2d6aabb4ae9a2b1802e1358068b P 9868dce521294d0b816bba7648f4ae5a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:23:32.914786 25508 raft_consensus.cc:740] T 579da2d6aabb4ae9a2b1802e1358068b P 9868dce521294d0b816bba7648f4ae5a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9868dce521294d0b816bba7648f4ae5a, State: Initialized, Role: FOLLOWER
I20260709 13:23:32.914899 25573 leader_election.cc:290] T b2f14aed05e24a98ad83535029b795dc P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 1 election: Requested vote from peers 233e7d29aac04505bf4e65f0a39a0c4c (127.14.177.130:42957), 43999d813e564dc688fad116090c95ca (127.14.177.129:38155)
I20260709 13:23:32.915069 25236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2f14aed05e24a98ad83535029b795dc" candidate_uuid: "3e07d64466934da4bb3c02262b1c2e38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43999d813e564dc688fad116090c95ca"
I20260709 13:23:32.914464 25539 raft_consensus.cc:2804] T a820954e07a94582ac0613342fcffde5 P 9868dce521294d0b816bba7648f4ae5a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:23:32.915566 25508 consensus_queue.cc:260] T 579da2d6aabb4ae9a2b1802e1358068b P 9868dce521294d0b816bba7648f4ae5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } }
I20260709 13:23:32.916236 25539 raft_consensus.cc:493] T a820954e07a94582ac0613342fcffde5 P 9868dce521294d0b816bba7648f4ae5a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:23:32.916606 25539 raft_consensus.cc:3060] T a820954e07a94582ac0613342fcffde5 P 9868dce521294d0b816bba7648f4ae5a [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:23:32.916589 25350 leader_election.cc:343] T b2f14aed05e24a98ad83535029b795dc P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 43999d813e564dc688fad116090c95ca (127.14.177.129:38155): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:23:32.917845 25508 ts_tablet_manager.cc:1434] T 579da2d6aabb4ae9a2b1802e1358068b P 9868dce521294d0b816bba7648f4ae5a: Time spent starting tablet: real 0.010s	user 0.005s	sys 0.000s
I20260709 13:23:32.918380 25310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2f14aed05e24a98ad83535029b795dc" candidate_uuid: "3e07d64466934da4bb3c02262b1c2e38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "233e7d29aac04505bf4e65f0a39a0c4c"
I20260709 13:23:32.918862 25503 tablet_bootstrap.cc:492] T 3a3317f4c3514df1b2bb8e44a86be92f P 233e7d29aac04505bf4e65f0a39a0c4c: Bootstrap starting.
I20260709 13:23:32.918959 25310 raft_consensus.cc:3060] T b2f14aed05e24a98ad83535029b795dc P 233e7d29aac04505bf4e65f0a39a0c4c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:23:32.912962 25504 raft_consensus.cc:359] T e15820fa8a7d4e1fb6ad23a64ca22f90 P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } }
I20260709 13:23:32.926762 25508 tablet_bootstrap.cc:492] T 1f15ec62698c4a8cbd84f2b3d516d2ea P 9868dce521294d0b816bba7648f4ae5a: Bootstrap starting.
I20260709 13:23:32.927438 25539 raft_consensus.cc:515] T a820954e07a94582ac0613342fcffde5 P 9868dce521294d0b816bba7648f4ae5a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } }
I20260709 13:23:32.932705 25573 raft_consensus.cc:493] T 280d65245f4f4358b54bef93e18aca92 P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:23:32.933703 25508 tablet_bootstrap.cc:654] T 1f15ec62698c4a8cbd84f2b3d516d2ea P 9868dce521294d0b816bba7648f4ae5a: Neither blocks nor log segments found. Creating new log.
I20260709 13:23:32.933324 25573 raft_consensus.cc:515] T 280d65245f4f4358b54bef93e18aca92 P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } }
I20260709 13:23:32.926702 25504 raft_consensus.cc:385] T e15820fa8a7d4e1fb6ad23a64ca22f90 P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:23:32.936048 25311 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a820954e07a94582ac0613342fcffde5" candidate_uuid: "9868dce521294d0b816bba7648f4ae5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "233e7d29aac04505bf4e65f0a39a0c4c"
I20260709 13:23:32.936363 25539 leader_election.cc:290] T a820954e07a94582ac0613342fcffde5 P 9868dce521294d0b816bba7648f4ae5a [CANDIDATE]: Term 1 election: Requested vote from peers 233e7d29aac04505bf4e65f0a39a0c4c (127.14.177.130:42957), 3e07d64466934da4bb3c02262b1c2e38 (127.14.177.131:41539)
I20260709 13:23:32.936686 25458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "280d65245f4f4358b54bef93e18aca92" candidate_uuid: "3e07d64466934da4bb3c02262b1c2e38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9868dce521294d0b816bba7648f4ae5a" is_pre_election: true
I20260709 13:23:32.925755 25503 tablet_bootstrap.cc:654] T 3a3317f4c3514df1b2bb8e44a86be92f P 233e7d29aac04505bf4e65f0a39a0c4c: Neither blocks nor log segments found. Creating new log.
I20260709 13:23:32.936117 25504 raft_consensus.cc:740] T e15820fa8a7d4e1fb6ad23a64ca22f90 P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e07d64466934da4bb3c02262b1c2e38, State: Initialized, Role: FOLLOWER
W20260709 13:23:32.938360 25348 leader_election.cc:343] T 280d65245f4f4358b54bef93e18aca92 P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9868dce521294d0b816bba7648f4ae5a (127.14.177.132:33683): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:23:32.937995 25504 consensus_queue.cc:260] T e15820fa8a7d4e1fb6ad23a64ca22f90 P 3e07d64466934da4bb3c02262b1c2e38 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } }
I20260709 13:23:32.940578 25504 ts_tablet_manager.cc:1434] T e15820fa8a7d4e1fb6ad23a64ca22f90 P 3e07d64466934da4bb3c02262b1c2e38: Time spent starting tablet: real 0.030s	user 0.006s	sys 0.000s
I20260709 13:23:32.941665 25573 leader_election.cc:290] T 280d65245f4f4358b54bef93e18aca92 P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9868dce521294d0b816bba7648f4ae5a (127.14.177.132:33683), 43999d813e564dc688fad116090c95ca (127.14.177.129:38155)
I20260709 13:23:32.942768 25236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "280d65245f4f4358b54bef93e18aca92" candidate_uuid: "3e07d64466934da4bb3c02262b1c2e38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43999d813e564dc688fad116090c95ca" is_pre_election: true
I20260709 13:23:32.943315 25504 tablet_bootstrap.cc:492] T beacc815ef954fb692e90544bfa93f05 P 3e07d64466934da4bb3c02262b1c2e38: Bootstrap starting.
I20260709 13:23:32.942770 25385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a820954e07a94582ac0613342fcffde5" candidate_uuid: "9868dce521294d0b816bba7648f4ae5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e07d64466934da4bb3c02262b1c2e38"
I20260709 13:23:32.936880 25311 raft_consensus.cc:3060] T a820954e07a94582ac0613342fcffde5 P 233e7d29aac04505bf4e65f0a39a0c4c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:23:32.944152 25385 raft_consensus.cc:3060] T a820954e07a94582ac0613342fcffde5 P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:23:32.950502 25350 leader_election.cc:343] T 280d65245f4f4358b54bef93e18aca92 P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 43999d813e564dc688fad116090c95ca (127.14.177.129:38155): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:23:32.951061 25350 leader_election.cc:304] T 280d65245f4f4358b54bef93e18aca92 P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3e07d64466934da4bb3c02262b1c2e38; no voters: 43999d813e564dc688fad116090c95ca, 9868dce521294d0b816bba7648f4ae5a
I20260709 13:23:32.952553 25503 tablet_bootstrap.cc:492] T 3a3317f4c3514df1b2bb8e44a86be92f P 233e7d29aac04505bf4e65f0a39a0c4c: No bootstrap required, opened a new log
I20260709 13:23:32.953298 25503 ts_tablet_manager.cc:1403] T 3a3317f4c3514df1b2bb8e44a86be92f P 233e7d29aac04505bf4e65f0a39a0c4c: Time spent bootstrapping tablet: real 0.035s	user 0.013s	sys 0.002s
I20260709 13:23:32.956426 25385 raft_consensus.cc:2468] T a820954e07a94582ac0613342fcffde5 P 3e07d64466934da4bb3c02262b1c2e38 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9868dce521294d0b816bba7648f4ae5a in term 1.
I20260709 13:23:32.956713 25573 raft_consensus.cc:2749] T 280d65245f4f4358b54bef93e18aca92 P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:23:32.958885 25311 raft_consensus.cc:2468] T a820954e07a94582ac0613342fcffde5 P 233e7d29aac04505bf4e65f0a39a0c4c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9868dce521294d0b816bba7648f4ae5a in term 1.
I20260709 13:23:32.961997 25504 tablet_bootstrap.cc:654] T beacc815ef954fb692e90544bfa93f05 P 3e07d64466934da4bb3c02262b1c2e38: Neither blocks nor log segments found. Creating new log.
I20260709 13:23:32.962800 25425 leader_election.cc:304] T a820954e07a94582ac0613342fcffde5 P 9868dce521294d0b816bba7648f4ae5a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 233e7d29aac04505bf4e65f0a39a0c4c, 9868dce521294d0b816bba7648f4ae5a; no voters: 
I20260709 13:23:32.964272 25539 raft_consensus.cc:2804] T a820954e07a94582ac0613342fcffde5 P 9868dce521294d0b816bba7648f4ae5a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:23:32.964841 25539 raft_consensus.cc:697] T a820954e07a94582ac0613342fcffde5 P 9868dce521294d0b816bba7648f4ae5a [term 1 LEADER]: Becoming Leader. State: Replica: 9868dce521294d0b816bba7648f4ae5a, State: Running, Role: LEADER
I20260709 13:23:32.965675 25539 consensus_queue.cc:237] T a820954e07a94582ac0613342fcffde5 P 9868dce521294d0b816bba7648f4ae5a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } }
I20260709 13:23:32.970654 25310 raft_consensus.cc:2468] T b2f14aed05e24a98ad83535029b795dc P 233e7d29aac04505bf4e65f0a39a0c4c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3e07d64466934da4bb3c02262b1c2e38 in term 1.
I20260709 13:23:32.958842 25503 raft_consensus.cc:359] T 3a3317f4c3514df1b2bb8e44a86be92f P 233e7d29aac04505bf4e65f0a39a0c4c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } }
I20260709 13:23:32.971791 25503 raft_consensus.cc:385] T 3a3317f4c3514df1b2bb8e44a86be92f P 233e7d29aac04505bf4e65f0a39a0c4c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:23:32.972102 25503 raft_consensus.cc:740] T 3a3317f4c3514df1b2bb8e44a86be92f P 233e7d29aac04505bf4e65f0a39a0c4c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 233e7d29aac04505bf4e65f0a39a0c4c, State: Initialized, Role: FOLLOWER
I20260709 13:23:32.973215 25351 leader_election.cc:304] T b2f14aed05e24a98ad83535029b795dc P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 233e7d29aac04505bf4e65f0a39a0c4c, 3e07d64466934da4bb3c02262b1c2e38; no voters: 43999d813e564dc688fad116090c95ca
I20260709 13:23:32.974129 25573 raft_consensus.cc:2804] T b2f14aed05e24a98ad83535029b795dc P 3e07d64466934da4bb3c02262b1c2e38 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:23:32.974845 25573 raft_consensus.cc:697] T b2f14aed05e24a98ad83535029b795dc P 3e07d64466934da4bb3c02262b1c2e38 [term 1 LEADER]: Becoming Leader. State: Replica: 3e07d64466934da4bb3c02262b1c2e38, State: Running, Role: LEADER
I20260709 13:23:32.975476 25510 raft_consensus.cc:493] T 3a3317f4c3514df1b2bb8e44a86be92f P 43999d813e564dc688fad116090c95ca [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:23:32.977020 25503 consensus_queue.cc:260] T 3a3317f4c3514df1b2bb8e44a86be92f P 233e7d29aac04505bf4e65f0a39a0c4c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } }
I20260709 13:23:32.975970 25510 raft_consensus.cc:515] T 3a3317f4c3514df1b2bb8e44a86be92f P 43999d813e564dc688fad116090c95ca [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } }
I20260709 13:23:32.976063 25573 consensus_queue.cc:237] T b2f14aed05e24a98ad83535029b795dc P 3e07d64466934da4bb3c02262b1c2e38 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } }
I20260709 13:23:32.980706 25503 ts_tablet_manager.cc:1434] T 3a3317f4c3514df1b2bb8e44a86be92f P 233e7d29aac04505bf4e65f0a39a0c4c: Time spent starting tablet: real 0.025s	user 0.006s	sys 0.002s
I20260709 13:23:32.981897 25503 tablet_bootstrap.cc:492] T beacc815ef954fb692e90544bfa93f05 P 233e7d29aac04505bf4e65f0a39a0c4c: Bootstrap starting.
I20260709 13:23:32.988587 25510 leader_election.cc:290] T 3a3317f4c3514df1b2bb8e44a86be92f P 43999d813e564dc688fad116090c95ca [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9868dce521294d0b816bba7648f4ae5a (127.14.177.132:33683), 233e7d29aac04505bf4e65f0a39a0c4c (127.14.177.130:42957)
I20260709 13:23:32.989630 25310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a3317f4c3514df1b2bb8e44a86be92f" candidate_uuid: "43999d813e564dc688fad116090c95ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" is_pre_election: true
I20260709 13:23:32.990106 25458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a3317f4c3514df1b2bb8e44a86be92f" candidate_uuid: "43999d813e564dc688fad116090c95ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9868dce521294d0b816bba7648f4ae5a" is_pre_election: true
I20260709 13:23:32.990549 25505 tablet_bootstrap.cc:492] T e15820fa8a7d4e1fb6ad23a64ca22f90 P 43999d813e564dc688fad116090c95ca: No bootstrap required, opened a new log
I20260709 13:23:32.990621 25310 raft_consensus.cc:2468] T 3a3317f4c3514df1b2bb8e44a86be92f P 233e7d29aac04505bf4e65f0a39a0c4c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 43999d813e564dc688fad116090c95ca in term 0.
I20260709 13:23:32.991343 25505 ts_tablet_manager.cc:1403] T e15820fa8a7d4e1fb6ad23a64ca22f90 P 43999d813e564dc688fad116090c95ca: Time spent bootstrapping tablet: real 0.127s	user 0.024s	sys 0.000s
I20260709 13:23:32.992384 25202 leader_election.cc:304] T 3a3317f4c3514df1b2bb8e44a86be92f P 43999d813e564dc688fad116090c95ca [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 233e7d29aac04505bf4e65f0a39a0c4c, 43999d813e564dc688fad116090c95ca; no voters: 
I20260709 13:23:32.990891 25458 raft_consensus.cc:2468] T 3a3317f4c3514df1b2bb8e44a86be92f P 9868dce521294d0b816bba7648f4ae5a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 43999d813e564dc688fad116090c95ca in term 0.
I20260709 13:23:32.994311 25559 raft_consensus.cc:2804] T 3a3317f4c3514df1b2bb8e44a86be92f P 43999d813e564dc688fad116090c95ca [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:23:32.994805 25559 raft_consensus.cc:493] T 3a3317f4c3514df1b2bb8e44a86be92f P 43999d813e564dc688fad116090c95ca [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:23:32.993893 25505 raft_consensus.cc:359] T e15820fa8a7d4e1fb6ad23a64ca22f90 P 43999d813e564dc688fad116090c95ca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } }
I20260709 13:23:33.000401 25505 raft_consensus.cc:385] T e15820fa8a7d4e1fb6ad23a64ca22f90 P 43999d813e564dc688fad116090c95ca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:23:33.000736 25505 raft_consensus.cc:740] T e15820fa8a7d4e1fb6ad23a64ca22f90 P 43999d813e564dc688fad116090c95ca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43999d813e564dc688fad116090c95ca, State: Initialized, Role: FOLLOWER
I20260709 13:23:33.000907 25504 tablet_bootstrap.cc:492] T beacc815ef954fb692e90544bfa93f05 P 3e07d64466934da4bb3c02262b1c2e38: No bootstrap required, opened a new log
I20260709 13:23:32.996022 25559 raft_consensus.cc:3060] T 3a3317f4c3514df1b2bb8e44a86be92f P 43999d813e564dc688fad116090c95ca [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:23:33.001472 25504 ts_tablet_manager.cc:1403] T beacc815ef954fb692e90544bfa93f05 P 3e07d64466934da4bb3c02262b1c2e38: Time spent bootstrapping tablet: real 0.058s	user 0.017s	sys 0.006s
I20260709 13:23:32.999431 25539 raft_consensus.cc:493] T 467a6b42ca6548269c12dddde696f7af P 9868dce521294d0b816bba7648f4ae5a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:23:33.002194 25539 raft_consensus.cc:515] T 467a6b42ca6548269c12dddde696f7af P 9868dce521294d0b816bba7648f4ae5a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } }
I20260709 13:23:33.004485 25539 leader_election.cc:290] T 467a6b42ca6548269c12dddde696f7af P 9868dce521294d0b816bba7648f4ae5a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 233e7d29aac04505bf4e65f0a39a0c4c (127.14.177.130:42957), 3e07d64466934da4bb3c02262b1c2e38 (127.14.177.131:41539)
I20260709 13:23:33.004995 25504 raft_consensus.cc:359] T beacc815ef954fb692e90544bfa93f05 P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } }
I20260709 13:23:33.005834 25504 raft_consensus.cc:385] T beacc815ef954fb692e90544bfa93f05 P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:23:33.005625 25590 consensus_queue.cc:1048] T 0d29b499fa4c41f1b974782cd0c9c1c2 P 233e7d29aac04505bf4e65f0a39a0c4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:23:33.006181 25504 raft_consensus.cc:740] T beacc815ef954fb692e90544bfa93f05 P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e07d64466934da4bb3c02262b1c2e38, State: Initialized, Role: FOLLOWER
I20260709 13:23:33.007067 25504 consensus_queue.cc:260] T beacc815ef954fb692e90544bfa93f05 P 3e07d64466934da4bb3c02262b1c2e38 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } }
W20260709 13:23:33.008466 25182 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:23:33.009439 25385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "467a6b42ca6548269c12dddde696f7af" candidate_uuid: "9868dce521294d0b816bba7648f4ae5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e07d64466934da4bb3c02262b1c2e38" is_pre_election: true
I20260709 13:23:32.999779 25503 tablet_bootstrap.cc:654] T beacc815ef954fb692e90544bfa93f05 P 233e7d29aac04505bf4e65f0a39a0c4c: Neither blocks nor log segments found. Creating new log.
I20260709 13:23:33.010195 25385 raft_consensus.cc:2468] T 467a6b42ca6548269c12dddde696f7af P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9868dce521294d0b816bba7648f4ae5a in term 0.
I20260709 13:23:33.012967 25504 ts_tablet_manager.cc:1434] T beacc815ef954fb692e90544bfa93f05 P 3e07d64466934da4bb3c02262b1c2e38: Time spent starting tablet: real 0.011s	user 0.006s	sys 0.000s
I20260709 13:23:33.013090 25422 leader_election.cc:304] T 467a6b42ca6548269c12dddde696f7af P 9868dce521294d0b816bba7648f4ae5a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e07d64466934da4bb3c02262b1c2e38, 9868dce521294d0b816bba7648f4ae5a; no voters: 
I20260709 13:23:33.014346 25539 raft_consensus.cc:2804] T 467a6b42ca6548269c12dddde696f7af P 9868dce521294d0b816bba7648f4ae5a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:23:33.009459 25310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "467a6b42ca6548269c12dddde696f7af" candidate_uuid: "9868dce521294d0b816bba7648f4ae5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" is_pre_election: true
I20260709 13:23:32.991439 25508 tablet_bootstrap.cc:492] T 1f15ec62698c4a8cbd84f2b3d516d2ea P 9868dce521294d0b816bba7648f4ae5a: No bootstrap required, opened a new log
I20260709 13:23:33.014575 25504 tablet_bootstrap.cc:492] T 5c4ab1978a47481c8576d466bacf49d2 P 3e07d64466934da4bb3c02262b1c2e38: Bootstrap starting.
I20260709 13:23:33.014740 25539 raft_consensus.cc:493] T 467a6b42ca6548269c12dddde696f7af P 9868dce521294d0b816bba7648f4ae5a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:23:33.016669 25539 raft_consensus.cc:3060] T 467a6b42ca6548269c12dddde696f7af P 9868dce521294d0b816bba7648f4ae5a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:23:33.008558 25505 consensus_queue.cc:260] T e15820fa8a7d4e1fb6ad23a64ca22f90 P 43999d813e564dc688fad116090c95ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } }
I20260709 13:23:33.011515 25131 catalog_manager.cc:5697] T a820954e07a94582ac0613342fcffde5 P 9868dce521294d0b816bba7648f4ae5a reported cstate change: term changed from 0 to 1, leader changed from <none> to 9868dce521294d0b816bba7648f4ae5a (127.14.177.132). New cstate: current_term: 1 leader_uuid: "9868dce521294d0b816bba7648f4ae5a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } health_report { overall_health: HEALTHY } } }
I20260709 13:23:33.017853 25132 catalog_manager.cc:5697] T b2f14aed05e24a98ad83535029b795dc P 3e07d64466934da4bb3c02262b1c2e38 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3e07d64466934da4bb3c02262b1c2e38 (127.14.177.131). New cstate: current_term: 1 leader_uuid: "3e07d64466934da4bb3c02262b1c2e38" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } health_report { overall_health: UNKNOWN } } }
I20260709 13:23:33.019219 25505 ts_tablet_manager.cc:1434] T e15820fa8a7d4e1fb6ad23a64ca22f90 P 43999d813e564dc688fad116090c95ca: Time spent starting tablet: real 0.027s	user 0.005s	sys 0.000s
I20260709 13:23:33.015403 25310 raft_consensus.cc:2468] T 467a6b42ca6548269c12dddde696f7af P 233e7d29aac04505bf4e65f0a39a0c4c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9868dce521294d0b816bba7648f4ae5a in term 0.
I20260709 13:23:33.020289 25505 tablet_bootstrap.cc:492] T e9e342ecda4b46619ce12452b2cf7154 P 43999d813e564dc688fad116090c95ca: Bootstrap starting.
I20260709 13:23:33.015791 25508 ts_tablet_manager.cc:1403] T 1f15ec62698c4a8cbd84f2b3d516d2ea P 9868dce521294d0b816bba7648f4ae5a: Time spent bootstrapping tablet: real 0.089s	user 0.015s	sys 0.022s
I20260709 13:23:33.023955 25508 raft_consensus.cc:359] T 1f15ec62698c4a8cbd84f2b3d516d2ea P 9868dce521294d0b816bba7648f4ae5a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } }
I20260709 13:23:33.024680 25508 raft_consensus.cc:385] T 1f15ec62698c4a8cbd84f2b3d516d2ea P 9868dce521294d0b816bba7648f4ae5a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:23:33.024945 25508 raft_consensus.cc:740] T 1f15ec62698c4a8cbd84f2b3d516d2ea P 9868dce521294d0b816bba7648f4ae5a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9868dce521294d0b816bba7648f4ae5a, State: Initialized, Role: FOLLOWER
I20260709 13:23:33.027412 25505 tablet_bootstrap.cc:654] T e9e342ecda4b46619ce12452b2cf7154 P 43999d813e564dc688fad116090c95ca: Neither blocks nor log segments found. Creating new log.
I20260709 13:23:33.032109 25508 consensus_queue.cc:260] T 1f15ec62698c4a8cbd84f2b3d516d2ea P 9868dce521294d0b816bba7648f4ae5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } }
I20260709 13:23:33.033265 25590 consensus_queue.cc:1048] T 0d29b499fa4c41f1b974782cd0c9c1c2 P 233e7d29aac04505bf4e65f0a39a0c4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:23:33.052017 25504 tablet_bootstrap.cc:654] T 5c4ab1978a47481c8576d466bacf49d2 P 3e07d64466934da4bb3c02262b1c2e38: Neither blocks nor log segments found. Creating new log.
I20260709 13:23:33.054502 25508 ts_tablet_manager.cc:1434] T 1f15ec62698c4a8cbd84f2b3d516d2ea P 9868dce521294d0b816bba7648f4ae5a: Time spent starting tablet: real 0.033s	user 0.006s	sys 0.000s
I20260709 13:23:33.057618 25508 tablet_bootstrap.cc:492] T d053b9ae38de4173a0aded4da494bd98 P 9868dce521294d0b816bba7648f4ae5a: Bootstrap starting.
I20260709 13:23:33.057621 25539 raft_consensus.cc:515] T 467a6b42ca6548269c12dddde696f7af P 9868dce521294d0b816bba7648f4ae5a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } }
I20260709 13:23:33.072254 25310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "467a6b42ca6548269c12dddde696f7af" candidate_uuid: "9868dce521294d0b816bba7648f4ae5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "233e7d29aac04505bf4e65f0a39a0c4c"
I20260709 13:23:33.073042 25310 raft_consensus.cc:3060] T 467a6b42ca6548269c12dddde696f7af P 233e7d29aac04505bf4e65f0a39a0c4c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:23:33.078725 25508 tablet_bootstrap.cc:654] T d053b9ae38de4173a0aded4da494bd98 P 9868dce521294d0b816bba7648f4ae5a: Neither blocks nor log segments found. Creating new log.
I20260709 13:23:33.089640 25622 raft_consensus.cc:493] T e15820fa8a7d4e1fb6ad23a64ca22f90 P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:23:33.090306 25622 raft_consensus.cc:515] T e15820fa8a7d4e1fb6ad23a64ca22f90 P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } }
W20260709 13:23:33.092245 25424 consensus_peers.cc:597] T e9e342ecda4b46619ce12452b2cf7154 P 9868dce521294d0b816bba7648f4ae5a -> Peer 43999d813e564dc688fad116090c95ca (127.14.177.129:38155): Couldn't send request to peer 43999d813e564dc688fad116090c95ca. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:23:33.015921 25559 raft_consensus.cc:515] T 3a3317f4c3514df1b2bb8e44a86be92f P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } }
I20260709 13:23:33.095451 25559 leader_election.cc:290] T 3a3317f4c3514df1b2bb8e44a86be92f P 43999d813e564dc688fad116090c95ca [CANDIDATE]: Term 1 election: Requested vote from peers 9868dce521294d0b816bba7648f4ae5a (127.14.177.132:33683), 233e7d29aac04505bf4e65f0a39a0c4c (127.14.177.130:42957)
I20260709 13:23:33.096032 25539 leader_election.cc:290] T 467a6b42ca6548269c12dddde696f7af P 9868dce521294d0b816bba7648f4ae5a [CANDIDATE]: Term 1 election: Requested vote from peers 233e7d29aac04505bf4e65f0a39a0c4c (127.14.177.130:42957), 3e07d64466934da4bb3c02262b1c2e38 (127.14.177.131:41539)
I20260709 13:23:33.082949 25310 raft_consensus.cc:2468] T 467a6b42ca6548269c12dddde696f7af P 233e7d29aac04505bf4e65f0a39a0c4c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9868dce521294d0b816bba7648f4ae5a in term 1.
I20260709 13:23:33.097148 25311 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15820fa8a7d4e1fb6ad23a64ca22f90" candidate_uuid: "3e07d64466934da4bb3c02262b1c2e38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" is_pre_election: true
I20260709 13:23:33.097757 25425 leader_election.cc:304] T 467a6b42ca6548269c12dddde696f7af P 9868dce521294d0b816bba7648f4ae5a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 233e7d29aac04505bf4e65f0a39a0c4c, 9868dce521294d0b816bba7648f4ae5a; no voters: 
I20260709 13:23:33.097900 25311 raft_consensus.cc:2468] T e15820fa8a7d4e1fb6ad23a64ca22f90 P 233e7d29aac04505bf4e65f0a39a0c4c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3e07d64466934da4bb3c02262b1c2e38 in term 0.
I20260709 13:23:33.098740 25539 raft_consensus.cc:2804] T 467a6b42ca6548269c12dddde696f7af P 9868dce521294d0b816bba7648f4ae5a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:23:33.099501 25351 leader_election.cc:304] T e15820fa8a7d4e1fb6ad23a64ca22f90 P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 233e7d29aac04505bf4e65f0a39a0c4c, 3e07d64466934da4bb3c02262b1c2e38; no voters: 
I20260709 13:23:33.100189 25539 raft_consensus.cc:697] T 467a6b42ca6548269c12dddde696f7af P 9868dce521294d0b816bba7648f4ae5a [term 1 LEADER]: Becoming Leader. State: Replica: 9868dce521294d0b816bba7648f4ae5a, State: Running, Role: LEADER
I20260709 13:23:33.100656 25573 raft_consensus.cc:2804] T e15820fa8a7d4e1fb6ad23a64ca22f90 P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:23:33.101116 25573 raft_consensus.cc:493] T e15820fa8a7d4e1fb6ad23a64ca22f90 P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:23:33.101708 25573 raft_consensus.cc:3060] T e15820fa8a7d4e1fb6ad23a64ca22f90 P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:23:33.101142 25539 consensus_queue.cc:237] T 467a6b42ca6548269c12dddde696f7af P 9868dce521294d0b816bba7648f4ae5a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } }
I20260709 13:23:33.105923 25458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a3317f4c3514df1b2bb8e44a86be92f" candidate_uuid: "43999d813e564dc688fad116090c95ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9868dce521294d0b816bba7648f4ae5a"
I20260709 13:23:33.106595 25458 raft_consensus.cc:3060] T 3a3317f4c3514df1b2bb8e44a86be92f P 9868dce521294d0b816bba7648f4ae5a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:23:33.114820 25310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a3317f4c3514df1b2bb8e44a86be92f" candidate_uuid: "43999d813e564dc688fad116090c95ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "233e7d29aac04505bf4e65f0a39a0c4c"
I20260709 13:23:33.115608 25310 raft_consensus.cc:3060] T 3a3317f4c3514df1b2bb8e44a86be92f P 233e7d29aac04505bf4e65f0a39a0c4c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:23:33.119303 25622 leader_election.cc:290] T e15820fa8a7d4e1fb6ad23a64ca22f90 P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 43999d813e564dc688fad116090c95ca (127.14.177.129:38155), 233e7d29aac04505bf4e65f0a39a0c4c (127.14.177.130:42957)
I20260709 13:23:33.119558 25503 tablet_bootstrap.cc:492] T beacc815ef954fb692e90544bfa93f05 P 233e7d29aac04505bf4e65f0a39a0c4c: No bootstrap required, opened a new log
I20260709 13:23:33.120162 25503 ts_tablet_manager.cc:1403] T beacc815ef954fb692e90544bfa93f05 P 233e7d29aac04505bf4e65f0a39a0c4c: Time spent bootstrapping tablet: real 0.138s	user 0.014s	sys 0.013s
I20260709 13:23:33.121783 25504 tablet_bootstrap.cc:492] T 5c4ab1978a47481c8576d466bacf49d2 P 3e07d64466934da4bb3c02262b1c2e38: No bootstrap required, opened a new log
I20260709 13:23:33.122368 25504 ts_tablet_manager.cc:1403] T 5c4ab1978a47481c8576d466bacf49d2 P 3e07d64466934da4bb3c02262b1c2e38: Time spent bootstrapping tablet: real 0.108s	user 0.014s	sys 0.002s
I20260709 13:23:33.122893 25510 consensus_queue.cc:1048] T b7ff8dde9f5d4c49b9203d902f6b94c4 P 43999d813e564dc688fad116090c95ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:23:33.124845 25384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "467a6b42ca6548269c12dddde696f7af" candidate_uuid: "9868dce521294d0b816bba7648f4ae5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e07d64466934da4bb3c02262b1c2e38"
I20260709 13:23:33.127200 25384 raft_consensus.cc:3060] T 467a6b42ca6548269c12dddde696f7af P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:23:33.130690 25504 raft_consensus.cc:359] T 5c4ab1978a47481c8576d466bacf49d2 P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } }
I20260709 13:23:33.115800 25235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15820fa8a7d4e1fb6ad23a64ca22f90" candidate_uuid: "3e07d64466934da4bb3c02262b1c2e38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43999d813e564dc688fad116090c95ca" is_pre_election: true
I20260709 13:23:33.131261 25310 raft_consensus.cc:2468] T 3a3317f4c3514df1b2bb8e44a86be92f P 233e7d29aac04505bf4e65f0a39a0c4c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 43999d813e564dc688fad116090c95ca in term 1.
I20260709 13:23:33.131891 25458 raft_consensus.cc:2468] T 3a3317f4c3514df1b2bb8e44a86be92f P 9868dce521294d0b816bba7648f4ae5a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 43999d813e564dc688fad116090c95ca in term 1.
I20260709 13:23:33.123049 25503 raft_consensus.cc:359] T beacc815ef954fb692e90544bfa93f05 P 233e7d29aac04505bf4e65f0a39a0c4c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } }
I20260709 13:23:33.136579 25503 raft_consensus.cc:385] T beacc815ef954fb692e90544bfa93f05 P 233e7d29aac04505bf4e65f0a39a0c4c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:23:33.137089 25503 raft_consensus.cc:740] T beacc815ef954fb692e90544bfa93f05 P 233e7d29aac04505bf4e65f0a39a0c4c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 233e7d29aac04505bf4e65f0a39a0c4c, State: Initialized, Role: FOLLOWER
I20260709 13:23:33.137594 25504 raft_consensus.cc:385] T 5c4ab1978a47481c8576d466bacf49d2 P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:23:33.134694 25505 tablet_bootstrap.cc:492] T e9e342ecda4b46619ce12452b2cf7154 P 43999d813e564dc688fad116090c95ca: No bootstrap required, opened a new log
I20260709 13:23:33.136129 25235 raft_consensus.cc:2468] T e15820fa8a7d4e1fb6ad23a64ca22f90 P 43999d813e564dc688fad116090c95ca [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3e07d64466934da4bb3c02262b1c2e38 in term 0.
I20260709 13:23:33.141026 25505 ts_tablet_manager.cc:1403] T e9e342ecda4b46619ce12452b2cf7154 P 43999d813e564dc688fad116090c95ca: Time spent bootstrapping tablet: real 0.121s	user 0.017s	sys 0.054s
I20260709 13:23:33.132974 25573 raft_consensus.cc:515] T e15820fa8a7d4e1fb6ad23a64ca22f90 P 3e07d64466934da4bb3c02262b1c2e38 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } }
I20260709 13:23:33.139475 25199 leader_election.cc:304] T 3a3317f4c3514df1b2bb8e44a86be92f P 43999d813e564dc688fad116090c95ca [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 43999d813e564dc688fad116090c95ca, 9868dce521294d0b816bba7648f4ae5a; no voters: 
I20260709 13:23:33.144167 25505 raft_consensus.cc:359] T e9e342ecda4b46619ce12452b2cf7154 P 43999d813e564dc688fad116090c95ca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } }
I20260709 13:23:33.145133 25505 raft_consensus.cc:385] T e9e342ecda4b46619ce12452b2cf7154 P 43999d813e564dc688fad116090c95ca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:23:33.145531 25505 raft_consensus.cc:740] T e9e342ecda4b46619ce12452b2cf7154 P 43999d813e564dc688fad116090c95ca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43999d813e564dc688fad116090c95ca, State: Initialized, Role: FOLLOWER
I20260709 13:23:33.148396 25510 raft_consensus.cc:2804] T 3a3317f4c3514df1b2bb8e44a86be92f P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:23:33.145926 25504 raft_consensus.cc:740] T 5c4ab1978a47481c8576d466bacf49d2 P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e07d64466934da4bb3c02262b1c2e38, State: Initialized, Role: FOLLOWER
I20260709 13:23:33.149062 25510 raft_consensus.cc:697] T 3a3317f4c3514df1b2bb8e44a86be92f P 43999d813e564dc688fad116090c95ca [term 1 LEADER]: Becoming Leader. State: Replica: 43999d813e564dc688fad116090c95ca, State: Running, Role: LEADER
I20260709 13:23:33.146694 25573 leader_election.cc:290] T e15820fa8a7d4e1fb6ad23a64ca22f90 P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 1 election: Requested vote from peers 43999d813e564dc688fad116090c95ca (127.14.177.129:38155), 233e7d29aac04505bf4e65f0a39a0c4c (127.14.177.130:42957)
I20260709 13:23:33.148098 25235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15820fa8a7d4e1fb6ad23a64ca22f90" candidate_uuid: "3e07d64466934da4bb3c02262b1c2e38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43999d813e564dc688fad116090c95ca"
I20260709 13:23:33.150076 25235 raft_consensus.cc:3060] T e15820fa8a7d4e1fb6ad23a64ca22f90 P 43999d813e564dc688fad116090c95ca [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:23:33.150226 25310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15820fa8a7d4e1fb6ad23a64ca22f90" candidate_uuid: "3e07d64466934da4bb3c02262b1c2e38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "233e7d29aac04505bf4e65f0a39a0c4c"
I20260709 13:23:33.138015 25503 consensus_queue.cc:260] T beacc815ef954fb692e90544bfa93f05 P 233e7d29aac04505bf4e65f0a39a0c4c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } }
I20260709 13:23:33.151919 25508 tablet_bootstrap.cc:492] T d053b9ae38de4173a0aded4da494bd98 P 9868dce521294d0b816bba7648f4ae5a: No bootstrap required, opened a new log
I20260709 13:23:33.151536 25504 consensus_queue.cc:260] T 5c4ab1978a47481c8576d466bacf49d2 P 3e07d64466934da4bb3c02262b1c2e38 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } }
I20260709 13:23:33.152419 25508 ts_tablet_manager.cc:1403] T d053b9ae38de4173a0aded4da494bd98 P 9868dce521294d0b816bba7648f4ae5a: Time spent bootstrapping tablet: real 0.095s	user 0.012s	sys 0.001s
I20260709 13:23:33.139521 25384 raft_consensus.cc:2468] T 467a6b42ca6548269c12dddde696f7af P 3e07d64466934da4bb3c02262b1c2e38 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9868dce521294d0b816bba7648f4ae5a in term 1.
I20260709 13:23:33.150107 25510 consensus_queue.cc:237] T 3a3317f4c3514df1b2bb8e44a86be92f P 43999d813e564dc688fad116090c95ca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } }
I20260709 13:23:33.167220 25508 raft_consensus.cc:359] T d053b9ae38de4173a0aded4da494bd98 P 9868dce521294d0b816bba7648f4ae5a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } }
I20260709 13:23:33.168072 25508 raft_consensus.cc:385] T d053b9ae38de4173a0aded4da494bd98 P 9868dce521294d0b816bba7648f4ae5a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:23:33.168402 25508 raft_consensus.cc:740] T d053b9ae38de4173a0aded4da494bd98 P 9868dce521294d0b816bba7648f4ae5a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9868dce521294d0b816bba7648f4ae5a, State: Initialized, Role: FOLLOWER
I20260709 13:23:33.169075 25458 raft_consensus.cc:3060] T 1f15ec62698c4a8cbd84f2b3d516d2ea P 9868dce521294d0b816bba7648f4ae5a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:23:33.169094 25504 ts_tablet_manager.cc:1434] T 5c4ab1978a47481c8576d466bacf49d2 P 3e07d64466934da4bb3c02262b1c2e38: Time spent starting tablet: real 0.042s	user 0.006s	sys 0.001s
I20260709 13:23:33.169278 25508 consensus_queue.cc:260] T d053b9ae38de4173a0aded4da494bd98 P 9868dce521294d0b816bba7648f4ae5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } }
I20260709 13:23:33.170732 25504 tablet_bootstrap.cc:492] T aad1f7937f854cc1a726223269da5d5a P 3e07d64466934da4bb3c02262b1c2e38: Bootstrap starting.
I20260709 13:23:33.173143 25508 ts_tablet_manager.cc:1434] T d053b9ae38de4173a0aded4da494bd98 P 9868dce521294d0b816bba7648f4ae5a: Time spent starting tablet: real 0.020s	user 0.006s	sys 0.000s
I20260709 13:23:33.174247 25508 tablet_bootstrap.cc:492] T 280d65245f4f4358b54bef93e18aca92 P 9868dce521294d0b816bba7648f4ae5a: Bootstrap starting.
I20260709 13:23:33.179363 25504 tablet_bootstrap.cc:654] T aad1f7937f854cc1a726223269da5d5a P 3e07d64466934da4bb3c02262b1c2e38: Neither blocks nor log segments found. Creating new log.
I20260709 13:23:33.179555 25535 consensus_queue.cc:1048] T b7ff8dde9f5d4c49b9203d902f6b94c4 P 43999d813e564dc688fad116090c95ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:23:33.159283 25503 ts_tablet_manager.cc:1434] T beacc815ef954fb692e90544bfa93f05 P 233e7d29aac04505bf4e65f0a39a0c4c: Time spent starting tablet: real 0.039s	user 0.008s	sys 0.001s
I20260709 13:23:33.159559 25235 raft_consensus.cc:2468] T e15820fa8a7d4e1fb6ad23a64ca22f90 P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3e07d64466934da4bb3c02262b1c2e38 in term 1.
I20260709 13:23:33.160248 25505 consensus_queue.cc:260] T e9e342ecda4b46619ce12452b2cf7154 P 43999d813e564dc688fad116090c95ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } }
I20260709 13:23:33.186959 25350 leader_election.cc:304] T e15820fa8a7d4e1fb6ad23a64ca22f90 P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e07d64466934da4bb3c02262b1c2e38, 43999d813e564dc688fad116090c95ca; no voters: 
I20260709 13:23:33.188230 25505 ts_tablet_manager.cc:1434] T e9e342ecda4b46619ce12452b2cf7154 P 43999d813e564dc688fad116090c95ca: Time spent starting tablet: real 0.047s	user 0.005s	sys 0.004s
I20260709 13:23:33.188601 25503 tablet_bootstrap.cc:492] T 5c4ab1978a47481c8576d466bacf49d2 P 233e7d29aac04505bf4e65f0a39a0c4c: Bootstrap starting.
I20260709 13:23:33.187819 25132 catalog_manager.cc:5697] T 3a3317f4c3514df1b2bb8e44a86be92f P 43999d813e564dc688fad116090c95ca reported cstate change: term changed from 0 to 1, leader changed from <none> to 43999d813e564dc688fad116090c95ca (127.14.177.129). New cstate: current_term: 1 leader_uuid: "43999d813e564dc688fad116090c95ca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } health_report { overall_health: UNKNOWN } } }
I20260709 13:23:33.189244 25505 tablet_bootstrap.cc:492] T b2f14aed05e24a98ad83535029b795dc P 43999d813e564dc688fad116090c95ca: Bootstrap starting.
I20260709 13:23:33.191525 25622 raft_consensus.cc:2804] T e15820fa8a7d4e1fb6ad23a64ca22f90 P 3e07d64466934da4bb3c02262b1c2e38 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:23:33.192430 25622 raft_consensus.cc:697] T e15820fa8a7d4e1fb6ad23a64ca22f90 P 3e07d64466934da4bb3c02262b1c2e38 [term 1 LEADER]: Becoming Leader. State: Replica: 3e07d64466934da4bb3c02262b1c2e38, State: Running, Role: LEADER
I20260709 13:23:33.193837 25622 consensus_queue.cc:237] T e15820fa8a7d4e1fb6ad23a64ca22f90 P 3e07d64466934da4bb3c02262b1c2e38 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } }
I20260709 13:23:33.159890 25310 raft_consensus.cc:3060] T e15820fa8a7d4e1fb6ad23a64ca22f90 P 233e7d29aac04505bf4e65f0a39a0c4c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:23:33.154793 25568 raft_consensus.cc:493] T 5c4ab1978a47481c8576d466bacf49d2 P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:23:33.161135 25539 consensus_queue.cc:1048] T e9e342ecda4b46619ce12452b2cf7154 P 9868dce521294d0b816bba7648f4ae5a [LEADER]: Connected to new peer: Peer: permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:23:33.195986 25573 consensus_queue.cc:1048] T 1f15ec62698c4a8cbd84f2b3d516d2ea P 3e07d64466934da4bb3c02262b1c2e38 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:23:33.196319 25568 raft_consensus.cc:515] T 5c4ab1978a47481c8576d466bacf49d2 P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } }
I20260709 13:23:33.198617 25568 leader_election.cc:290] T 5c4ab1978a47481c8576d466bacf49d2 P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 233e7d29aac04505bf4e65f0a39a0c4c (127.14.177.130:42957), 9868dce521294d0b816bba7648f4ae5a (127.14.177.132:33683)
I20260709 13:23:33.147658 25131 catalog_manager.cc:5697] T 467a6b42ca6548269c12dddde696f7af P 9868dce521294d0b816bba7648f4ae5a reported cstate change: term changed from 0 to 1, leader changed from <none> to 9868dce521294d0b816bba7648f4ae5a (127.14.177.132). New cstate: current_term: 1 leader_uuid: "9868dce521294d0b816bba7648f4ae5a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } health_report { overall_health: HEALTHY } } }
I20260709 13:23:33.200743 25308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c4ab1978a47481c8576d466bacf49d2" candidate_uuid: "3e07d64466934da4bb3c02262b1c2e38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" is_pre_election: true
I20260709 13:23:33.201889 25459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c4ab1978a47481c8576d466bacf49d2" candidate_uuid: "3e07d64466934da4bb3c02262b1c2e38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9868dce521294d0b816bba7648f4ae5a" is_pre_election: true
W20260709 13:23:33.203523 25348 leader_election.cc:343] T 5c4ab1978a47481c8576d466bacf49d2 P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9868dce521294d0b816bba7648f4ae5a (127.14.177.132:33683): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:23:33.208599 25505 tablet_bootstrap.cc:654] T b2f14aed05e24a98ad83535029b795dc P 43999d813e564dc688fad116090c95ca: Neither blocks nor log segments found. Creating new log.
W20260709 13:23:33.210580 25351 leader_election.cc:343] T 5c4ab1978a47481c8576d466bacf49d2 P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 233e7d29aac04505bf4e65f0a39a0c4c (127.14.177.130:42957): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:23:33.212610 25351 leader_election.cc:304] T 5c4ab1978a47481c8576d466bacf49d2 P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3e07d64466934da4bb3c02262b1c2e38; no voters: 233e7d29aac04505bf4e65f0a39a0c4c, 9868dce521294d0b816bba7648f4ae5a
I20260709 13:23:33.213466 25568 raft_consensus.cc:2749] T 5c4ab1978a47481c8576d466bacf49d2 P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:23:33.214723 25601 raft_consensus.cc:493] T 28c0b6fc9bd64d60828d705c6a9ab1e4 P 43999d813e564dc688fad116090c95ca [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:23:33.215343 25601 raft_consensus.cc:515] T 28c0b6fc9bd64d60828d705c6a9ab1e4 P 43999d813e564dc688fad116090c95ca [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } }
I20260709 13:23:33.216645 25235 raft_consensus.cc:3060] T 28c0b6fc9bd64d60828d705c6a9ab1e4 P 43999d813e564dc688fad116090c95ca [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:23:33.218118 25601 leader_election.cc:290] T 28c0b6fc9bd64d60828d705c6a9ab1e4 P 43999d813e564dc688fad116090c95ca [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 233e7d29aac04505bf4e65f0a39a0c4c (127.14.177.130:42957), 3e07d64466934da4bb3c02262b1c2e38 (127.14.177.131:41539)
I20260709 13:23:33.220026 25308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28c0b6fc9bd64d60828d705c6a9ab1e4" candidate_uuid: "43999d813e564dc688fad116090c95ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" is_pre_election: true
I20260709 13:23:33.222280 25508 tablet_bootstrap.cc:654] T 280d65245f4f4358b54bef93e18aca92 P 9868dce521294d0b816bba7648f4ae5a: Neither blocks nor log segments found. Creating new log.
I20260709 13:23:33.234416 25458 raft_consensus.cc:3060] T 579da2d6aabb4ae9a2b1802e1358068b P 9868dce521294d0b816bba7648f4ae5a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:23:33.236014 25385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28c0b6fc9bd64d60828d705c6a9ab1e4" candidate_uuid: "43999d813e564dc688fad116090c95ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e07d64466934da4bb3c02262b1c2e38" is_pre_election: true
I20260709 13:23:33.239989 25202 leader_election.cc:304] T 28c0b6fc9bd64d60828d705c6a9ab1e4 P 43999d813e564dc688fad116090c95ca [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 43999d813e564dc688fad116090c95ca; no voters: 233e7d29aac04505bf4e65f0a39a0c4c, 3e07d64466934da4bb3c02262b1c2e38
I20260709 13:23:33.242223 25568 consensus_queue.cc:1048] T 1f15ec62698c4a8cbd84f2b3d516d2ea P 3e07d64466934da4bb3c02262b1c2e38 [LEADER]: Connected to new peer: Peer: permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:23:33.246105 25559 consensus_queue.cc:1048] T 579da2d6aabb4ae9a2b1802e1358068b P 43999d813e564dc688fad116090c95ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:23:33.248970 25503 tablet_bootstrap.cc:654] T 5c4ab1978a47481c8576d466bacf49d2 P 233e7d29aac04505bf4e65f0a39a0c4c: Neither blocks nor log segments found. Creating new log.
I20260709 13:23:33.253594 25590 consensus_queue.cc:1048] T 3fe5a944a342411ea923eb39071ac770 P 233e7d29aac04505bf4e65f0a39a0c4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.005s
I20260709 13:23:33.258553 25559 consensus_queue.cc:1048] T 579da2d6aabb4ae9a2b1802e1358068b P 43999d813e564dc688fad116090c95ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:23:33.291738 25510 raft_consensus.cc:493] T e9e342ecda4b46619ce12452b2cf7154 P 43999d813e564dc688fad116090c95ca [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:23:33.292390 25510 raft_consensus.cc:515] T e9e342ecda4b46619ce12452b2cf7154 P 43999d813e564dc688fad116090c95ca [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } }
I20260709 13:23:33.281128 25568 raft_consensus.cc:493] T ba8a000f5abe4158965449995af84b18 P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:23:33.296898 25308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9e342ecda4b46619ce12452b2cf7154" candidate_uuid: "43999d813e564dc688fad116090c95ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" is_pre_election: true
I20260709 13:23:33.297014 25568 raft_consensus.cc:515] T ba8a000f5abe4158965449995af84b18 P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } }
I20260709 13:23:33.297721 25456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9e342ecda4b46619ce12452b2cf7154" candidate_uuid: "43999d813e564dc688fad116090c95ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9868dce521294d0b816bba7648f4ae5a" is_pre_election: true
I20260709 13:23:33.298164 25510 leader_election.cc:290] T e9e342ecda4b46619ce12452b2cf7154 P 43999d813e564dc688fad116090c95ca [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9868dce521294d0b816bba7648f4ae5a (127.14.177.132:33683), 233e7d29aac04505bf4e65f0a39a0c4c (127.14.177.130:42957)
I20260709 13:23:33.268263 25310 raft_consensus.cc:2468] T e15820fa8a7d4e1fb6ad23a64ca22f90 P 233e7d29aac04505bf4e65f0a39a0c4c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3e07d64466934da4bb3c02262b1c2e38 in term 1.
I20260709 13:23:33.305394 25202 leader_election.cc:304] T e9e342ecda4b46619ce12452b2cf7154 P 43999d813e564dc688fad116090c95ca [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 43999d813e564dc688fad116090c95ca; no voters: 233e7d29aac04505bf4e65f0a39a0c4c, 9868dce521294d0b816bba7648f4ae5a
I20260709 13:23:33.306255 25559 raft_consensus.cc:2749] T e9e342ecda4b46619ce12452b2cf7154 P 43999d813e564dc688fad116090c95ca [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:23:33.325181 25591 consensus_queue.cc:1048] T 3fe5a944a342411ea923eb39071ac770 P 233e7d29aac04505bf4e65f0a39a0c4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 13:23:33.344774 25504 tablet_bootstrap.cc:492] T aad1f7937f854cc1a726223269da5d5a P 3e07d64466934da4bb3c02262b1c2e38: No bootstrap required, opened a new log
I20260709 13:23:33.345645 25504 ts_tablet_manager.cc:1403] T aad1f7937f854cc1a726223269da5d5a P 3e07d64466934da4bb3c02262b1c2e38: Time spent bootstrapping tablet: real 0.175s	user 0.015s	sys 0.018s
I20260709 13:23:33.355541 25309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba8a000f5abe4158965449995af84b18" candidate_uuid: "3e07d64466934da4bb3c02262b1c2e38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" is_pre_election: true
I20260709 13:23:33.359611 25622 raft_consensus.cc:493] T beacc815ef954fb692e90544bfa93f05 P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:23:33.360415 25622 raft_consensus.cc:515] T beacc815ef954fb692e90544bfa93f05 P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } }
W20260709 13:23:33.364631 25351 leader_election.cc:343] T ba8a000f5abe4158965449995af84b18 P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 233e7d29aac04505bf4e65f0a39a0c4c (127.14.177.130:42957): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:23:33.355844 25505 tablet_bootstrap.cc:492] T b2f14aed05e24a98ad83535029b795dc P 43999d813e564dc688fad116090c95ca: No bootstrap required, opened a new log
I20260709 13:23:33.367102 25505 ts_tablet_manager.cc:1403] T b2f14aed05e24a98ad83535029b795dc P 43999d813e564dc688fad116090c95ca: Time spent bootstrapping tablet: real 0.178s	user 0.018s	sys 0.006s
I20260709 13:23:33.369882 25505 raft_consensus.cc:359] T b2f14aed05e24a98ad83535029b795dc P 43999d813e564dc688fad116090c95ca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } }
I20260709 13:23:33.370606 25505 raft_consensus.cc:385] T b2f14aed05e24a98ad83535029b795dc P 43999d813e564dc688fad116090c95ca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:23:33.367183 25234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "beacc815ef954fb692e90544bfa93f05" candidate_uuid: "3e07d64466934da4bb3c02262b1c2e38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43999d813e564dc688fad116090c95ca" is_pre_election: true
I20260709 13:23:33.373163 25622 leader_election.cc:290] T beacc815ef954fb692e90544bfa93f05 P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 43999d813e564dc688fad116090c95ca (127.14.177.129:38155), 233e7d29aac04505bf4e65f0a39a0c4c (127.14.177.130:42957)
I20260709 13:23:33.376662 25505 raft_consensus.cc:740] T b2f14aed05e24a98ad83535029b795dc P 43999d813e564dc688fad116090c95ca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43999d813e564dc688fad116090c95ca, State: Initialized, Role: FOLLOWER
I20260709 13:23:33.377490 25505 consensus_queue.cc:260] T b2f14aed05e24a98ad83535029b795dc P 43999d813e564dc688fad116090c95ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } }
I20260709 13:23:33.378139 25504 raft_consensus.cc:359] T aad1f7937f854cc1a726223269da5d5a P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } }
W20260709 13:23:33.379055 25350 leader_election.cc:343] T beacc815ef954fb692e90544bfa93f05 P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 43999d813e564dc688fad116090c95ca (127.14.177.129:38155): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:23:33.379623 25505 ts_tablet_manager.cc:1434] T b2f14aed05e24a98ad83535029b795dc P 43999d813e564dc688fad116090c95ca: Time spent starting tablet: real 0.012s	user 0.007s	sys 0.000s
I20260709 13:23:33.380124 25309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "beacc815ef954fb692e90544bfa93f05" candidate_uuid: "3e07d64466934da4bb3c02262b1c2e38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" is_pre_election: true
I20260709 13:23:33.380638 25505 tablet_bootstrap.cc:492] T 9e5d70664720424d951f269d9d7f4346 P 43999d813e564dc688fad116090c95ca: Bootstrap starting.
I20260709 13:23:33.380826 25309 raft_consensus.cc:2468] T beacc815ef954fb692e90544bfa93f05 P 233e7d29aac04505bf4e65f0a39a0c4c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3e07d64466934da4bb3c02262b1c2e38 in term 0.
I20260709 13:23:33.383111 25351 leader_election.cc:304] T beacc815ef954fb692e90544bfa93f05 P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 233e7d29aac04505bf4e65f0a39a0c4c, 3e07d64466934da4bb3c02262b1c2e38; no voters: 43999d813e564dc688fad116090c95ca
I20260709 13:23:33.384646 25622 raft_consensus.cc:2804] T beacc815ef954fb692e90544bfa93f05 P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:23:33.385663 25622 raft_consensus.cc:493] T beacc815ef954fb692e90544bfa93f05 P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:23:33.369894 25568 leader_election.cc:290] T ba8a000f5abe4158965449995af84b18 P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 233e7d29aac04505bf4e65f0a39a0c4c (127.14.177.130:42957), 9868dce521294d0b816bba7648f4ae5a (127.14.177.132:33683)
I20260709 13:23:33.386184 25622 raft_consensus.cc:3060] T beacc815ef954fb692e90544bfa93f05 P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:23:33.387332 25505 tablet_bootstrap.cc:654] T 9e5d70664720424d951f269d9d7f4346 P 43999d813e564dc688fad116090c95ca: Neither blocks nor log segments found. Creating new log.
I20260709 13:23:33.393453 25563 raft_consensus.cc:493] T d053b9ae38de4173a0aded4da494bd98 P 9868dce521294d0b816bba7648f4ae5a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:23:33.394279 25563 raft_consensus.cc:515] T d053b9ae38de4173a0aded4da494bd98 P 9868dce521294d0b816bba7648f4ae5a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } }
I20260709 13:23:33.379096 25504 raft_consensus.cc:385] T aad1f7937f854cc1a726223269da5d5a P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:23:33.395910 25504 raft_consensus.cc:740] T aad1f7937f854cc1a726223269da5d5a P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e07d64466934da4bb3c02262b1c2e38, State: Initialized, Role: FOLLOWER
I20260709 13:23:33.396895 25459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba8a000f5abe4158965449995af84b18" candidate_uuid: "3e07d64466934da4bb3c02262b1c2e38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9868dce521294d0b816bba7648f4ae5a" is_pre_election: true
W20260709 13:23:33.398342 25348 leader_election.cc:343] T ba8a000f5abe4158965449995af84b18 P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9868dce521294d0b816bba7648f4ae5a (127.14.177.132:33683): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:23:33.398898 25348 leader_election.cc:304] T ba8a000f5abe4158965449995af84b18 P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3e07d64466934da4bb3c02262b1c2e38; no voters: 233e7d29aac04505bf4e65f0a39a0c4c, 9868dce521294d0b816bba7648f4ae5a
I20260709 13:23:33.400274 25568 raft_consensus.cc:2749] T ba8a000f5abe4158965449995af84b18 P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:23:33.402470 25234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d053b9ae38de4173a0aded4da494bd98" candidate_uuid: "9868dce521294d0b816bba7648f4ae5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43999d813e564dc688fad116090c95ca" is_pre_election: true
I20260709 13:23:33.403211 25234 raft_consensus.cc:2468] T d053b9ae38de4173a0aded4da494bd98 P 43999d813e564dc688fad116090c95ca [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9868dce521294d0b816bba7648f4ae5a in term 0.
I20260709 13:23:33.404611 25424 leader_election.cc:304] T d053b9ae38de4173a0aded4da494bd98 P 9868dce521294d0b816bba7648f4ae5a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 43999d813e564dc688fad116090c95ca, 9868dce521294d0b816bba7648f4ae5a; no voters: 
I20260709 13:23:33.405217 25503 tablet_bootstrap.cc:492] T 5c4ab1978a47481c8576d466bacf49d2 P 233e7d29aac04505bf4e65f0a39a0c4c: No bootstrap required, opened a new log
I20260709 13:23:33.405766 25503 ts_tablet_manager.cc:1403] T 5c4ab1978a47481c8576d466bacf49d2 P 233e7d29aac04505bf4e65f0a39a0c4c: Time spent bootstrapping tablet: real 0.217s	user 0.022s	sys 0.004s
I20260709 13:23:33.408681 25456 raft_consensus.cc:3060] T 0e71b6f5c69b4b35a8ac51c22c7d5f0d P 9868dce521294d0b816bba7648f4ae5a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:23:33.409612 25503 raft_consensus.cc:359] T 5c4ab1978a47481c8576d466bacf49d2 P 233e7d29aac04505bf4e65f0a39a0c4c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } }
I20260709 13:23:33.410337 25503 raft_consensus.cc:385] T 5c4ab1978a47481c8576d466bacf49d2 P 233e7d29aac04505bf4e65f0a39a0c4c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:23:33.410593 25503 raft_consensus.cc:740] T 5c4ab1978a47481c8576d466bacf49d2 P 233e7d29aac04505bf4e65f0a39a0c4c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 233e7d29aac04505bf4e65f0a39a0c4c, State: Initialized, Role: FOLLOWER
I20260709 13:23:33.411266 25601 raft_consensus.cc:2749] T 28c0b6fc9bd64d60828d705c6a9ab1e4 P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:23:33.414489 25503 consensus_queue.cc:260] T 5c4ab1978a47481c8576d466bacf49d2 P 233e7d29aac04505bf4e65f0a39a0c4c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } }
I20260709 13:23:33.416632 25503 ts_tablet_manager.cc:1434] T 5c4ab1978a47481c8576d466bacf49d2 P 233e7d29aac04505bf4e65f0a39a0c4c: Time spent starting tablet: real 0.011s	user 0.007s	sys 0.000s
I20260709 13:23:33.416253 25651 consensus_queue.cc:1048] T a820954e07a94582ac0613342fcffde5 P 9868dce521294d0b816bba7648f4ae5a [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:23:33.417621 25503 tablet_bootstrap.cc:492] T ba8a000f5abe4158965449995af84b18 P 233e7d29aac04505bf4e65f0a39a0c4c: Bootstrap starting.
I20260709 13:23:33.418147 25651 raft_consensus.cc:2804] T d053b9ae38de4173a0aded4da494bd98 P 9868dce521294d0b816bba7648f4ae5a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:23:33.418677 25651 raft_consensus.cc:493] T d053b9ae38de4173a0aded4da494bd98 P 9868dce521294d0b816bba7648f4ae5a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:23:33.419338 25563 leader_election.cc:290] T d053b9ae38de4173a0aded4da494bd98 P 9868dce521294d0b816bba7648f4ae5a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 43999d813e564dc688fad116090c95ca (127.14.177.129:38155), 233e7d29aac04505bf4e65f0a39a0c4c (127.14.177.130:42957)
I20260709 13:23:33.422417 25508 tablet_bootstrap.cc:492] T 280d65245f4f4358b54bef93e18aca92 P 9868dce521294d0b816bba7648f4ae5a: No bootstrap required, opened a new log
I20260709 13:23:33.423115 25508 ts_tablet_manager.cc:1403] T 280d65245f4f4358b54bef93e18aca92 P 9868dce521294d0b816bba7648f4ae5a: Time spent bootstrapping tablet: real 0.249s	user 0.011s	sys 0.001s
I20260709 13:23:33.425885 25504 consensus_queue.cc:260] T aad1f7937f854cc1a726223269da5d5a P 3e07d64466934da4bb3c02262b1c2e38 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } }
I20260709 13:23:33.426466 25508 raft_consensus.cc:359] T 280d65245f4f4358b54bef93e18aca92 P 9868dce521294d0b816bba7648f4ae5a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } }
I20260709 13:23:33.429682 25508 raft_consensus.cc:385] T 280d65245f4f4358b54bef93e18aca92 P 9868dce521294d0b816bba7648f4ae5a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:23:33.430053 25508 raft_consensus.cc:740] T 280d65245f4f4358b54bef93e18aca92 P 9868dce521294d0b816bba7648f4ae5a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9868dce521294d0b816bba7648f4ae5a, State: Initialized, Role: FOLLOWER
I20260709 13:23:33.430897 25508 consensus_queue.cc:260] T 280d65245f4f4358b54bef93e18aca92 P 9868dce521294d0b816bba7648f4ae5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } }
I20260709 13:23:33.419096 25651 raft_consensus.cc:3060] T d053b9ae38de4173a0aded4da494bd98 P 9868dce521294d0b816bba7648f4ae5a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:23:33.428978 25504 ts_tablet_manager.cc:1434] T aad1f7937f854cc1a726223269da5d5a P 3e07d64466934da4bb3c02262b1c2e38: Time spent starting tablet: real 0.083s	user 0.007s	sys 0.001s
I20260709 13:23:33.436043 25504 tablet_bootstrap.cc:492] T 9ffa42c3b2ff48bc87310e9ab5d218ce P 3e07d64466934da4bb3c02262b1c2e38: Bootstrap starting.
I20260709 13:23:33.437198 25133 catalog_manager.cc:5697] T e15820fa8a7d4e1fb6ad23a64ca22f90 P 3e07d64466934da4bb3c02262b1c2e38 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3e07d64466934da4bb3c02262b1c2e38 (127.14.177.131). New cstate: current_term: 1 leader_uuid: "3e07d64466934da4bb3c02262b1c2e38" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } health_report { overall_health: HEALTHY } } }
I20260709 13:23:33.440305 25508 ts_tablet_manager.cc:1434] T 280d65245f4f4358b54bef93e18aca92 P 9868dce521294d0b816bba7648f4ae5a: Time spent starting tablet: real 0.017s	user 0.007s	sys 0.000s
I20260709 13:23:33.440559 25505 tablet_bootstrap.cc:492] T 9e5d70664720424d951f269d9d7f4346 P 43999d813e564dc688fad116090c95ca: No bootstrap required, opened a new log
I20260709 13:23:33.440991 25505 ts_tablet_manager.cc:1403] T 9e5d70664720424d951f269d9d7f4346 P 43999d813e564dc688fad116090c95ca: Time spent bootstrapping tablet: real 0.061s	user 0.010s	sys 0.000s
I20260709 13:23:33.441375 25508 tablet_bootstrap.cc:492] T 51a11db644124b63bdf5f457c967473d P 9868dce521294d0b816bba7648f4ae5a: Bootstrap starting.
I20260709 13:23:33.444129 25309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d053b9ae38de4173a0aded4da494bd98" candidate_uuid: "9868dce521294d0b816bba7648f4ae5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" is_pre_election: true
W20260709 13:23:33.445629 25425 leader_election.cc:343] T d053b9ae38de4173a0aded4da494bd98 P 9868dce521294d0b816bba7648f4ae5a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 233e7d29aac04505bf4e65f0a39a0c4c (127.14.177.130:42957): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:23:33.451464 25622 raft_consensus.cc:515] T beacc815ef954fb692e90544bfa93f05 P 3e07d64466934da4bb3c02262b1c2e38 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } }
I20260709 13:23:33.453176 25508 tablet_bootstrap.cc:654] T 51a11db644124b63bdf5f457c967473d P 9868dce521294d0b816bba7648f4ae5a: Neither blocks nor log segments found. Creating new log.
I20260709 13:23:33.454491 25235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "beacc815ef954fb692e90544bfa93f05" candidate_uuid: "3e07d64466934da4bb3c02262b1c2e38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43999d813e564dc688fad116090c95ca"
I20260709 13:23:33.447860 25651 raft_consensus.cc:515] T d053b9ae38de4173a0aded4da494bd98 P 9868dce521294d0b816bba7648f4ae5a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } }
W20260709 13:23:33.457247 25350 leader_election.cc:343] T beacc815ef954fb692e90544bfa93f05 P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 43999d813e564dc688fad116090c95ca (127.14.177.129:38155): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:23:33.458756 25651 leader_election.cc:290] T d053b9ae38de4173a0aded4da494bd98 P 9868dce521294d0b816bba7648f4ae5a [CANDIDATE]: Term 1 election: Requested vote from peers 43999d813e564dc688fad116090c95ca (127.14.177.129:38155), 233e7d29aac04505bf4e65f0a39a0c4c (127.14.177.130:42957)
I20260709 13:23:33.459829 25307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d053b9ae38de4173a0aded4da494bd98" candidate_uuid: "9868dce521294d0b816bba7648f4ae5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "233e7d29aac04505bf4e65f0a39a0c4c"
W20260709 13:23:33.461268 25425 leader_election.cc:343] T d053b9ae38de4173a0aded4da494bd98 P 9868dce521294d0b816bba7648f4ae5a [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 233e7d29aac04505bf4e65f0a39a0c4c (127.14.177.130:42957): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:23:33.463435 25424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:38155, user_credentials={real_user=slave}} blocked reactor thread for 58907.7us
I20260709 13:23:33.464473 25508 tablet_bootstrap.cc:492] T 51a11db644124b63bdf5f457c967473d P 9868dce521294d0b816bba7648f4ae5a: No bootstrap required, opened a new log
I20260709 13:23:33.465003 25508 ts_tablet_manager.cc:1403] T 51a11db644124b63bdf5f457c967473d P 9868dce521294d0b816bba7648f4ae5a: Time spent bootstrapping tablet: real 0.024s	user 0.004s	sys 0.006s
I20260709 13:23:33.465034 25235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d053b9ae38de4173a0aded4da494bd98" candidate_uuid: "9868dce521294d0b816bba7648f4ae5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43999d813e564dc688fad116090c95ca"
I20260709 13:23:33.465808 25235 raft_consensus.cc:3060] T d053b9ae38de4173a0aded4da494bd98 P 43999d813e564dc688fad116090c95ca [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:23:33.473424 25235 raft_consensus.cc:2468] T d053b9ae38de4173a0aded4da494bd98 P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9868dce521294d0b816bba7648f4ae5a in term 1.
I20260709 13:23:33.473665 25651 consensus_queue.cc:1048] T a820954e07a94582ac0613342fcffde5 P 9868dce521294d0b816bba7648f4ae5a [LEADER]: Connected to new peer: Peer: permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:23:33.476015 25573 consensus_queue.cc:1048] T b2f14aed05e24a98ad83535029b795dc P 3e07d64466934da4bb3c02262b1c2e38 [LEADER]: Connected to new peer: Peer: permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:23:33.477566 25504 tablet_bootstrap.cc:654] T 9ffa42c3b2ff48bc87310e9ab5d218ce P 3e07d64466934da4bb3c02262b1c2e38: Neither blocks nor log segments found. Creating new log.
I20260709 13:23:33.481401 25622 leader_election.cc:290] T beacc815ef954fb692e90544bfa93f05 P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 1 election: Requested vote from peers 43999d813e564dc688fad116090c95ca (127.14.177.129:38155), 233e7d29aac04505bf4e65f0a39a0c4c (127.14.177.130:42957)
I20260709 13:23:33.473739 25505 raft_consensus.cc:359] T 9e5d70664720424d951f269d9d7f4346 P 43999d813e564dc688fad116090c95ca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } }
I20260709 13:23:33.484498 25505 raft_consensus.cc:385] T 9e5d70664720424d951f269d9d7f4346 P 43999d813e564dc688fad116090c95ca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:23:33.484753 25505 raft_consensus.cc:740] T 9e5d70664720424d951f269d9d7f4346 P 43999d813e564dc688fad116090c95ca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43999d813e564dc688fad116090c95ca, State: Initialized, Role: FOLLOWER
I20260709 13:23:33.488001 25424 leader_election.cc:304] T d053b9ae38de4173a0aded4da494bd98 P 9868dce521294d0b816bba7648f4ae5a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 43999d813e564dc688fad116090c95ca, 9868dce521294d0b816bba7648f4ae5a; no voters: 233e7d29aac04505bf4e65f0a39a0c4c
I20260709 13:23:33.490135 25505 consensus_queue.cc:260] T 9e5d70664720424d951f269d9d7f4346 P 43999d813e564dc688fad116090c95ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } }
I20260709 13:23:33.491919 25503 tablet_bootstrap.cc:654] T ba8a000f5abe4158965449995af84b18 P 233e7d29aac04505bf4e65f0a39a0c4c: Neither blocks nor log segments found. Creating new log.
I20260709 13:23:33.493535 25508 raft_consensus.cc:359] T 51a11db644124b63bdf5f457c967473d P 9868dce521294d0b816bba7648f4ae5a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } }
I20260709 13:23:33.494546 25508 raft_consensus.cc:385] T 51a11db644124b63bdf5f457c967473d P 9868dce521294d0b816bba7648f4ae5a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:23:33.494892 25508 raft_consensus.cc:740] T 51a11db644124b63bdf5f457c967473d P 9868dce521294d0b816bba7648f4ae5a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9868dce521294d0b816bba7648f4ae5a, State: Initialized, Role: FOLLOWER
I20260709 13:23:33.496359 25311 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "beacc815ef954fb692e90544bfa93f05" candidate_uuid: "3e07d64466934da4bb3c02262b1c2e38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "233e7d29aac04505bf4e65f0a39a0c4c"
I20260709 13:23:33.502022 25311 raft_consensus.cc:3060] T beacc815ef954fb692e90544bfa93f05 P 233e7d29aac04505bf4e65f0a39a0c4c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:23:33.512390 25650 raft_consensus.cc:2804] T d053b9ae38de4173a0aded4da494bd98 P 9868dce521294d0b816bba7648f4ae5a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:23:33.519255 25235 raft_consensus.cc:3060] T e9e342ecda4b46619ce12452b2cf7154 P 43999d813e564dc688fad116090c95ca [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:23:33.523494 25650 raft_consensus.cc:697] T d053b9ae38de4173a0aded4da494bd98 P 9868dce521294d0b816bba7648f4ae5a [term 1 LEADER]: Becoming Leader. State: Replica: 9868dce521294d0b816bba7648f4ae5a, State: Running, Role: LEADER
I20260709 13:23:33.524914 25650 consensus_queue.cc:237] T d053b9ae38de4173a0aded4da494bd98 P 9868dce521294d0b816bba7648f4ae5a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } }
I20260709 13:23:33.534564 25505 ts_tablet_manager.cc:1434] T 9e5d70664720424d951f269d9d7f4346 P 43999d813e564dc688fad116090c95ca: Time spent starting tablet: real 0.093s	user 0.006s	sys 0.002s
I20260709 13:23:33.497352 25508 consensus_queue.cc:260] T 51a11db644124b63bdf5f457c967473d P 9868dce521294d0b816bba7648f4ae5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } }
I20260709 13:23:33.548760 25505 tablet_bootstrap.cc:492] T 280d65245f4f4358b54bef93e18aca92 P 43999d813e564dc688fad116090c95ca: Bootstrap starting.
I20260709 13:23:33.549973 25508 ts_tablet_manager.cc:1434] T 51a11db644124b63bdf5f457c967473d P 9868dce521294d0b816bba7648f4ae5a: Time spent starting tablet: real 0.085s	user 0.010s	sys 0.000s
I20260709 13:23:33.550985 25508 tablet_bootstrap.cc:492] T 5c4ab1978a47481c8576d466bacf49d2 P 9868dce521294d0b816bba7648f4ae5a: Bootstrap starting.
I20260709 13:23:33.554550 25311 raft_consensus.cc:2468] T beacc815ef954fb692e90544bfa93f05 P 233e7d29aac04505bf4e65f0a39a0c4c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3e07d64466934da4bb3c02262b1c2e38 in term 1.
I20260709 13:23:33.559818 25351 leader_election.cc:304] T beacc815ef954fb692e90544bfa93f05 P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 233e7d29aac04505bf4e65f0a39a0c4c, 3e07d64466934da4bb3c02262b1c2e38; no voters: 43999d813e564dc688fad116090c95ca
I20260709 13:23:33.561087 25236 raft_consensus.cc:3060] T b2f14aed05e24a98ad83535029b795dc P 43999d813e564dc688fad116090c95ca [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:23:33.566123 25503 tablet_bootstrap.cc:492] T ba8a000f5abe4158965449995af84b18 P 233e7d29aac04505bf4e65f0a39a0c4c: No bootstrap required, opened a new log
I20260709 13:23:33.567914 25568 raft_consensus.cc:2804] T beacc815ef954fb692e90544bfa93f05 P 3e07d64466934da4bb3c02262b1c2e38 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:23:33.568249 25503 ts_tablet_manager.cc:1403] T ba8a000f5abe4158965449995af84b18 P 233e7d29aac04505bf4e65f0a39a0c4c: Time spent bootstrapping tablet: real 0.151s	user 0.016s	sys 0.000s
I20260709 13:23:33.568447 25568 raft_consensus.cc:697] T beacc815ef954fb692e90544bfa93f05 P 3e07d64466934da4bb3c02262b1c2e38 [term 1 LEADER]: Becoming Leader. State: Replica: 3e07d64466934da4bb3c02262b1c2e38, State: Running, Role: LEADER
I20260709 13:23:33.568970 25508 tablet_bootstrap.cc:654] T 5c4ab1978a47481c8576d466bacf49d2 P 9868dce521294d0b816bba7648f4ae5a: Neither blocks nor log segments found. Creating new log.
I20260709 13:23:33.567607 25504 tablet_bootstrap.cc:492] T 9ffa42c3b2ff48bc87310e9ab5d218ce P 3e07d64466934da4bb3c02262b1c2e38: No bootstrap required, opened a new log
I20260709 13:23:33.569648 25504 ts_tablet_manager.cc:1403] T 9ffa42c3b2ff48bc87310e9ab5d218ce P 3e07d64466934da4bb3c02262b1c2e38: Time spent bootstrapping tablet: real 0.134s	user 0.011s	sys 0.004s
I20260709 13:23:33.569382 25568 consensus_queue.cc:237] T beacc815ef954fb692e90544bfa93f05 P 3e07d64466934da4bb3c02262b1c2e38 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } }
I20260709 13:23:33.575456 25642 raft_consensus.cc:493] T aad1f7937f854cc1a726223269da5d5a P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:23:33.564397 25505 tablet_bootstrap.cc:654] T 280d65245f4f4358b54bef93e18aca92 P 43999d813e564dc688fad116090c95ca: Neither blocks nor log segments found. Creating new log.
I20260709 13:23:33.575977 25642 raft_consensus.cc:515] T aad1f7937f854cc1a726223269da5d5a P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } }
I20260709 13:23:33.573619 25503 raft_consensus.cc:359] T ba8a000f5abe4158965449995af84b18 P 233e7d29aac04505bf4e65f0a39a0c4c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } }
I20260709 13:23:33.575109 25504 raft_consensus.cc:359] T 9ffa42c3b2ff48bc87310e9ab5d218ce P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } }
I20260709 13:23:33.579253 25642 leader_election.cc:290] T aad1f7937f854cc1a726223269da5d5a P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 233e7d29aac04505bf4e65f0a39a0c4c (127.14.177.130:42957), 9868dce521294d0b816bba7648f4ae5a (127.14.177.132:33683)
I20260709 13:23:33.579049 25503 raft_consensus.cc:385] T ba8a000f5abe4158965449995af84b18 P 233e7d29aac04505bf4e65f0a39a0c4c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:23:33.579515 25456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aad1f7937f854cc1a726223269da5d5a" candidate_uuid: "3e07d64466934da4bb3c02262b1c2e38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9868dce521294d0b816bba7648f4ae5a" is_pre_election: true
I20260709 13:23:33.579975 25503 raft_consensus.cc:740] T ba8a000f5abe4158965449995af84b18 P 233e7d29aac04505bf4e65f0a39a0c4c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 233e7d29aac04505bf4e65f0a39a0c4c, State: Initialized, Role: FOLLOWER
I20260709 13:23:33.579330 25504 raft_consensus.cc:385] T 9ffa42c3b2ff48bc87310e9ab5d218ce P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:23:33.580749 25504 raft_consensus.cc:740] T 9ffa42c3b2ff48bc87310e9ab5d218ce P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e07d64466934da4bb3c02262b1c2e38, State: Initialized, Role: FOLLOWER
I20260709 13:23:33.581478 25504 consensus_queue.cc:260] T 9ffa42c3b2ff48bc87310e9ab5d218ce P 3e07d64466934da4bb3c02262b1c2e38 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } }
I20260709 13:23:33.582474 25503 consensus_queue.cc:260] T ba8a000f5abe4158965449995af84b18 P 233e7d29aac04505bf4e65f0a39a0c4c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } }
W20260709 13:23:33.583287 25348 leader_election.cc:343] T aad1f7937f854cc1a726223269da5d5a P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9868dce521294d0b816bba7648f4ae5a (127.14.177.132:33683): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:23:33.584484 25504 ts_tablet_manager.cc:1434] T 9ffa42c3b2ff48bc87310e9ab5d218ce P 3e07d64466934da4bb3c02262b1c2e38: Time spent starting tablet: real 0.014s	user 0.004s	sys 0.002s
I20260709 13:23:33.585516 25504 tablet_bootstrap.cc:492] T ce98a3f3b3eb4c63bcaa355ec7cd64a4 P 3e07d64466934da4bb3c02262b1c2e38: Bootstrap starting.
I20260709 13:23:33.587877 25510 consensus_queue.cc:1048] T 3a3317f4c3514df1b2bb8e44a86be92f P 43999d813e564dc688fad116090c95ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:23:33.590740 25503 ts_tablet_manager.cc:1434] T ba8a000f5abe4158965449995af84b18 P 233e7d29aac04505bf4e65f0a39a0c4c: Time spent starting tablet: real 0.022s	user 0.007s	sys 0.000s
I20260709 13:23:33.592022 25309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aad1f7937f854cc1a726223269da5d5a" candidate_uuid: "3e07d64466934da4bb3c02262b1c2e38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" is_pre_election: true
W20260709 13:23:33.595808 25351 leader_election.cc:343] T aad1f7937f854cc1a726223269da5d5a P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 233e7d29aac04505bf4e65f0a39a0c4c (127.14.177.130:42957): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:23:33.596194 25510 raft_consensus.cc:493] T 9e5d70664720424d951f269d9d7f4346 P 43999d813e564dc688fad116090c95ca [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:23:33.596483 25351 leader_election.cc:304] T aad1f7937f854cc1a726223269da5d5a P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3e07d64466934da4bb3c02262b1c2e38; no voters: 233e7d29aac04505bf4e65f0a39a0c4c, 9868dce521294d0b816bba7648f4ae5a
I20260709 13:23:33.597011 25510 raft_consensus.cc:515] T 9e5d70664720424d951f269d9d7f4346 P 43999d813e564dc688fad116090c95ca [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } }
I20260709 13:23:33.597621 25573 consensus_queue.cc:1048] T b2f14aed05e24a98ad83535029b795dc P 3e07d64466934da4bb3c02262b1c2e38 [LEADER]: Connected to new peer: Peer: permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:23:33.599414 25573 raft_consensus.cc:2749] T aad1f7937f854cc1a726223269da5d5a P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:23:33.601320 25503 tablet_bootstrap.cc:492] T d053b9ae38de4173a0aded4da494bd98 P 233e7d29aac04505bf4e65f0a39a0c4c: Bootstrap starting.
I20260709 13:23:33.607070 25505 tablet_bootstrap.cc:492] T 280d65245f4f4358b54bef93e18aca92 P 43999d813e564dc688fad116090c95ca: No bootstrap required, opened a new log
I20260709 13:23:33.609313 25650 consensus_queue.cc:1048] T 467a6b42ca6548269c12dddde696f7af P 9868dce521294d0b816bba7648f4ae5a [LEADER]: Connected to new peer: Peer: permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:23:33.613165 25505 ts_tablet_manager.cc:1403] T 280d65245f4f4358b54bef93e18aca92 P 43999d813e564dc688fad116090c95ca: Time spent bootstrapping tablet: real 0.065s	user 0.004s	sys 0.010s
I20260709 13:23:33.615682 25133 catalog_manager.cc:5697] T d053b9ae38de4173a0aded4da494bd98 P 9868dce521294d0b816bba7648f4ae5a reported cstate change: term changed from 0 to 1, leader changed from <none> to 9868dce521294d0b816bba7648f4ae5a (127.14.177.132). New cstate: current_term: 1 leader_uuid: "9868dce521294d0b816bba7648f4ae5a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } health_report { overall_health: UNKNOWN } } }
I20260709 13:23:33.623049 25503 tablet_bootstrap.cc:654] T d053b9ae38de4173a0aded4da494bd98 P 233e7d29aac04505bf4e65f0a39a0c4c: Neither blocks nor log segments found. Creating new log.
I20260709 13:23:33.622749 25456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e5d70664720424d951f269d9d7f4346" candidate_uuid: "43999d813e564dc688fad116090c95ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9868dce521294d0b816bba7648f4ae5a" is_pre_election: true
I20260709 13:23:33.626331 25505 raft_consensus.cc:359] T 280d65245f4f4358b54bef93e18aca92 P 43999d813e564dc688fad116090c95ca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } }
I20260709 13:23:33.632504 25510 leader_election.cc:290] T 9e5d70664720424d951f269d9d7f4346 P 43999d813e564dc688fad116090c95ca [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9868dce521294d0b816bba7648f4ae5a (127.14.177.132:33683), 233e7d29aac04505bf4e65f0a39a0c4c (127.14.177.130:42957)
I20260709 13:23:33.635812 25505 raft_consensus.cc:385] T 280d65245f4f4358b54bef93e18aca92 P 43999d813e564dc688fad116090c95ca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:23:33.644706 25309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e5d70664720424d951f269d9d7f4346" candidate_uuid: "43999d813e564dc688fad116090c95ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" is_pre_election: true
I20260709 13:23:33.645550 25504 tablet_bootstrap.cc:654] T ce98a3f3b3eb4c63bcaa355ec7cd64a4 P 3e07d64466934da4bb3c02262b1c2e38: Neither blocks nor log segments found. Creating new log.
I20260709 13:23:33.643453 25505 raft_consensus.cc:740] T 280d65245f4f4358b54bef93e18aca92 P 43999d813e564dc688fad116090c95ca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43999d813e564dc688fad116090c95ca, State: Initialized, Role: FOLLOWER
I20260709 13:23:33.662632 25651 consensus_queue.cc:1048] T 467a6b42ca6548269c12dddde696f7af P 9868dce521294d0b816bba7648f4ae5a [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:23:33.674752 25134 catalog_manager.cc:5697] T beacc815ef954fb692e90544bfa93f05 P 3e07d64466934da4bb3c02262b1c2e38 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3e07d64466934da4bb3c02262b1c2e38 (127.14.177.131). New cstate: current_term: 1 leader_uuid: "3e07d64466934da4bb3c02262b1c2e38" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } health_report { overall_health: HEALTHY } } }
I20260709 13:23:33.681368 25505 consensus_queue.cc:260] T 280d65245f4f4358b54bef93e18aca92 P 43999d813e564dc688fad116090c95ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } }
W20260709 13:23:33.645349 25199 leader_election.cc:343] T 9e5d70664720424d951f269d9d7f4346 P 43999d813e564dc688fad116090c95ca [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9868dce521294d0b816bba7648f4ae5a (127.14.177.132:33683): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:23:33.689339 25505 ts_tablet_manager.cc:1434] T 280d65245f4f4358b54bef93e18aca92 P 43999d813e564dc688fad116090c95ca: Time spent starting tablet: real 0.076s	user 0.006s	sys 0.004s
I20260709 13:23:33.691228 25503 tablet_bootstrap.cc:492] T d053b9ae38de4173a0aded4da494bd98 P 233e7d29aac04505bf4e65f0a39a0c4c: No bootstrap required, opened a new log
I20260709 13:23:33.691459 25505 tablet_bootstrap.cc:492] T beacc815ef954fb692e90544bfa93f05 P 43999d813e564dc688fad116090c95ca: Bootstrap starting.
I20260709 13:23:33.691761 25503 ts_tablet_manager.cc:1403] T d053b9ae38de4173a0aded4da494bd98 P 233e7d29aac04505bf4e65f0a39a0c4c: Time spent bootstrapping tablet: real 0.091s	user 0.014s	sys 0.003s
I20260709 13:23:33.692374 25508 tablet_bootstrap.cc:492] T 5c4ab1978a47481c8576d466bacf49d2 P 9868dce521294d0b816bba7648f4ae5a: No bootstrap required, opened a new log
I20260709 13:23:33.692780 25508 ts_tablet_manager.cc:1403] T 5c4ab1978a47481c8576d466bacf49d2 P 9868dce521294d0b816bba7648f4ae5a: Time spent bootstrapping tablet: real 0.142s	user 0.047s	sys 0.058s
W20260709 13:23:33.695437 25202 leader_election.cc:343] T 9e5d70664720424d951f269d9d7f4346 P 43999d813e564dc688fad116090c95ca [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 233e7d29aac04505bf4e65f0a39a0c4c (127.14.177.130:42957): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:23:33.695966 25202 leader_election.cc:304] T 9e5d70664720424d951f269d9d7f4346 P 43999d813e564dc688fad116090c95ca [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 43999d813e564dc688fad116090c95ca; no voters: 233e7d29aac04505bf4e65f0a39a0c4c, 9868dce521294d0b816bba7648f4ae5a
I20260709 13:23:33.696830 25510 raft_consensus.cc:2749] T 9e5d70664720424d951f269d9d7f4346 P 43999d813e564dc688fad116090c95ca [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:23:33.696671 25568 consensus_queue.cc:1048] T e15820fa8a7d4e1fb6ad23a64ca22f90 P 3e07d64466934da4bb3c02262b1c2e38 [LEADER]: Connected to new peer: Peer: permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:23:33.698681 25591 raft_consensus.cc:493] T ba8a000f5abe4158965449995af84b18 P 233e7d29aac04505bf4e65f0a39a0c4c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:23:33.699251 25591 raft_consensus.cc:515] T ba8a000f5abe4158965449995af84b18 P 233e7d29aac04505bf4e65f0a39a0c4c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } }
I20260709 13:23:33.699426 25508 raft_consensus.cc:359] T 5c4ab1978a47481c8576d466bacf49d2 P 9868dce521294d0b816bba7648f4ae5a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } }
I20260709 13:23:33.701412 25591 leader_election.cc:290] T ba8a000f5abe4158965449995af84b18 P 233e7d29aac04505bf4e65f0a39a0c4c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3e07d64466934da4bb3c02262b1c2e38 (127.14.177.131:41539), 9868dce521294d0b816bba7648f4ae5a (127.14.177.132:33683)
I20260709 13:23:33.701536 25508 raft_consensus.cc:385] T 5c4ab1978a47481c8576d466bacf49d2 P 9868dce521294d0b816bba7648f4ae5a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:23:33.701879 25508 raft_consensus.cc:740] T 5c4ab1978a47481c8576d466bacf49d2 P 9868dce521294d0b816bba7648f4ae5a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9868dce521294d0b816bba7648f4ae5a, State: Initialized, Role: FOLLOWER
I20260709 13:23:33.703214 25508 consensus_queue.cc:260] T 5c4ab1978a47481c8576d466bacf49d2 P 9868dce521294d0b816bba7648f4ae5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } }
I20260709 13:23:33.704478 25505 tablet_bootstrap.cc:654] T beacc815ef954fb692e90544bfa93f05 P 43999d813e564dc688fad116090c95ca: Neither blocks nor log segments found. Creating new log.
I20260709 13:23:33.705165 25385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba8a000f5abe4158965449995af84b18" candidate_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e07d64466934da4bb3c02262b1c2e38" is_pre_election: true
I20260709 13:23:33.708729 25456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba8a000f5abe4158965449995af84b18" candidate_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9868dce521294d0b816bba7648f4ae5a" is_pre_election: true
I20260709 13:23:33.713961 25508 ts_tablet_manager.cc:1434] T 5c4ab1978a47481c8576d466bacf49d2 P 9868dce521294d0b816bba7648f4ae5a: Time spent starting tablet: real 0.021s	user 0.007s	sys 0.000s
I20260709 13:23:33.725908 25508 tablet_bootstrap.cc:492] T 9e5d70664720424d951f269d9d7f4346 P 9868dce521294d0b816bba7648f4ae5a: Bootstrap starting.
W20260709 13:23:33.726253 25274 leader_election.cc:343] T ba8a000f5abe4158965449995af84b18 P 233e7d29aac04505bf4e65f0a39a0c4c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9868dce521294d0b816bba7648f4ae5a (127.14.177.132:33683): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:23:33.727136 25650 raft_consensus.cc:493] T 5c4ab1978a47481c8576d466bacf49d2 P 9868dce521294d0b816bba7648f4ae5a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:23:33.728549 25650 raft_consensus.cc:515] T 5c4ab1978a47481c8576d466bacf49d2 P 9868dce521294d0b816bba7648f4ae5a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } }
I20260709 13:23:33.735773 25650 leader_election.cc:290] T 5c4ab1978a47481c8576d466bacf49d2 P 9868dce521294d0b816bba7648f4ae5a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 233e7d29aac04505bf4e65f0a39a0c4c (127.14.177.130:42957), 3e07d64466934da4bb3c02262b1c2e38 (127.14.177.131:41539)
I20260709 13:23:33.736790 25591 raft_consensus.cc:493] T 5c4ab1978a47481c8576d466bacf49d2 P 233e7d29aac04505bf4e65f0a39a0c4c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:23:33.737581 25591 raft_consensus.cc:515] T 5c4ab1978a47481c8576d466bacf49d2 P 233e7d29aac04505bf4e65f0a39a0c4c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } }
I20260709 13:23:33.739952 25644 raft_consensus.cc:493] T 280d65245f4f4358b54bef93e18aca92 P 9868dce521294d0b816bba7648f4ae5a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:23:33.736968 25508 tablet_bootstrap.cc:654] T 9e5d70664720424d951f269d9d7f4346 P 9868dce521294d0b816bba7648f4ae5a: Neither blocks nor log segments found. Creating new log.
I20260709 13:23:33.740500 25644 raft_consensus.cc:515] T 280d65245f4f4358b54bef93e18aca92 P 9868dce521294d0b816bba7648f4ae5a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } }
I20260709 13:23:33.741406 25568 consensus_queue.cc:1048] T e15820fa8a7d4e1fb6ad23a64ca22f90 P 3e07d64466934da4bb3c02262b1c2e38 [LEADER]: Connected to new peer: Peer: permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:23:33.743265 25385 raft_consensus.cc:2468] T ba8a000f5abe4158965449995af84b18 P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 233e7d29aac04505bf4e65f0a39a0c4c in term 0.
I20260709 13:23:33.744773 25591 leader_election.cc:290] T 5c4ab1978a47481c8576d466bacf49d2 P 233e7d29aac04505bf4e65f0a39a0c4c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3e07d64466934da4bb3c02262b1c2e38 (127.14.177.131:41539), 9868dce521294d0b816bba7648f4ae5a (127.14.177.132:33683)
I20260709 13:23:33.745942 25234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "280d65245f4f4358b54bef93e18aca92" candidate_uuid: "9868dce521294d0b816bba7648f4ae5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43999d813e564dc688fad116090c95ca" is_pre_election: true
I20260709 13:23:33.746338 25456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c4ab1978a47481c8576d466bacf49d2" candidate_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9868dce521294d0b816bba7648f4ae5a" is_pre_election: true
I20260709 13:23:33.746652 25234 raft_consensus.cc:2468] T 280d65245f4f4358b54bef93e18aca92 P 43999d813e564dc688fad116090c95ca [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9868dce521294d0b816bba7648f4ae5a in term 0.
I20260709 13:23:33.744411 25644 leader_election.cc:290] T 280d65245f4f4358b54bef93e18aca92 P 9868dce521294d0b816bba7648f4ae5a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3e07d64466934da4bb3c02262b1c2e38 (127.14.177.131:41539), 43999d813e564dc688fad116090c95ca (127.14.177.129:38155)
I20260709 13:23:33.747011 25456 raft_consensus.cc:2468] T 5c4ab1978a47481c8576d466bacf49d2 P 9868dce521294d0b816bba7648f4ae5a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 233e7d29aac04505bf4e65f0a39a0c4c in term 0.
I20260709 13:23:33.750667 25601 consensus_queue.cc:1048] T 3a3317f4c3514df1b2bb8e44a86be92f P 43999d813e564dc688fad116090c95ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:23:33.751039 25383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c4ab1978a47481c8576d466bacf49d2" candidate_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e07d64466934da4bb3c02262b1c2e38" is_pre_election: true
I20260709 13:23:33.751734 25383 raft_consensus.cc:2468] T 5c4ab1978a47481c8576d466bacf49d2 P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 233e7d29aac04505bf4e65f0a39a0c4c in term 0.
I20260709 13:23:33.752638 25424 leader_election.cc:304] T 280d65245f4f4358b54bef93e18aca92 P 9868dce521294d0b816bba7648f4ae5a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 43999d813e564dc688fad116090c95ca, 9868dce521294d0b816bba7648f4ae5a; no voters: 
I20260709 13:23:33.753469 25644 raft_consensus.cc:2804] T 280d65245f4f4358b54bef93e18aca92 P 9868dce521294d0b816bba7648f4ae5a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:23:33.753587 25274 leader_election.cc:304] T ba8a000f5abe4158965449995af84b18 P 233e7d29aac04505bf4e65f0a39a0c4c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 233e7d29aac04505bf4e65f0a39a0c4c, 3e07d64466934da4bb3c02262b1c2e38; no voters: 9868dce521294d0b816bba7648f4ae5a
I20260709 13:23:33.754098 25644 raft_consensus.cc:493] T 280d65245f4f4358b54bef93e18aca92 P 9868dce521294d0b816bba7648f4ae5a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:23:33.754981 25590 raft_consensus.cc:2804] T ba8a000f5abe4158965449995af84b18 P 233e7d29aac04505bf4e65f0a39a0c4c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:23:33.747153 25384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "280d65245f4f4358b54bef93e18aca92" candidate_uuid: "9868dce521294d0b816bba7648f4ae5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e07d64466934da4bb3c02262b1c2e38" is_pre_election: true
I20260709 13:23:33.755151 25274 leader_election.cc:304] T 5c4ab1978a47481c8576d466bacf49d2 P 233e7d29aac04505bf4e65f0a39a0c4c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 233e7d29aac04505bf4e65f0a39a0c4c, 9868dce521294d0b816bba7648f4ae5a; no voters: 
I20260709 13:23:33.755705 25384 raft_consensus.cc:2468] T 280d65245f4f4358b54bef93e18aca92 P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9868dce521294d0b816bba7648f4ae5a in term 0.
I20260709 13:23:33.755450 25590 raft_consensus.cc:493] T ba8a000f5abe4158965449995af84b18 P 233e7d29aac04505bf4e65f0a39a0c4c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:23:33.759531 25503 raft_consensus.cc:359] T d053b9ae38de4173a0aded4da494bd98 P 233e7d29aac04505bf4e65f0a39a0c4c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } }
I20260709 13:23:33.766932 25503 raft_consensus.cc:385] T d053b9ae38de4173a0aded4da494bd98 P 233e7d29aac04505bf4e65f0a39a0c4c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:23:33.767716 25503 raft_consensus.cc:740] T d053b9ae38de4173a0aded4da494bd98 P 233e7d29aac04505bf4e65f0a39a0c4c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 233e7d29aac04505bf4e65f0a39a0c4c, State: Initialized, Role: FOLLOWER
I20260709 13:23:33.754773 25644 raft_consensus.cc:3060] T 280d65245f4f4358b54bef93e18aca92 P 9868dce521294d0b816bba7648f4ae5a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:23:33.770553 25503 consensus_queue.cc:260] T d053b9ae38de4173a0aded4da494bd98 P 233e7d29aac04505bf4e65f0a39a0c4c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } }
I20260709 13:23:33.760070 25591 raft_consensus.cc:2804] T 5c4ab1978a47481c8576d466bacf49d2 P 233e7d29aac04505bf4e65f0a39a0c4c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:23:33.781394 25591 raft_consensus.cc:493] T 5c4ab1978a47481c8576d466bacf49d2 P 233e7d29aac04505bf4e65f0a39a0c4c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:23:33.781862 25591 raft_consensus.cc:3060] T 5c4ab1978a47481c8576d466bacf49d2 P 233e7d29aac04505bf4e65f0a39a0c4c [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:23:33.759586 25665 log.cc:927] Time spent T e9e342ecda4b46619ce12452b2cf7154 P 43999d813e564dc688fad116090c95ca: Append to log took a long time: real 0.052s	user 0.001s	sys 0.000s
I20260709 13:23:33.757246 25590 raft_consensus.cc:3060] T ba8a000f5abe4158965449995af84b18 P 233e7d29aac04505bf4e65f0a39a0c4c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:23:33.788640 25385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c4ab1978a47481c8576d466bacf49d2" candidate_uuid: "9868dce521294d0b816bba7648f4ae5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e07d64466934da4bb3c02262b1c2e38" is_pre_election: true
I20260709 13:23:33.789736 25385 raft_consensus.cc:2468] T 5c4ab1978a47481c8576d466bacf49d2 P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9868dce521294d0b816bba7648f4ae5a in term 0.
I20260709 13:23:33.791357 25422 leader_election.cc:304] T 5c4ab1978a47481c8576d466bacf49d2 P 9868dce521294d0b816bba7648f4ae5a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e07d64466934da4bb3c02262b1c2e38, 9868dce521294d0b816bba7648f4ae5a; no voters: 
I20260709 13:23:33.792588 25650 raft_consensus.cc:2804] T 5c4ab1978a47481c8576d466bacf49d2 P 9868dce521294d0b816bba7648f4ae5a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:23:33.793087 25650 raft_consensus.cc:493] T 5c4ab1978a47481c8576d466bacf49d2 P 9868dce521294d0b816bba7648f4ae5a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:23:33.793527 25650 raft_consensus.cc:3060] T 5c4ab1978a47481c8576d466bacf49d2 P 9868dce521294d0b816bba7648f4ae5a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:23:33.758340 25311 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c4ab1978a47481c8576d466bacf49d2" candidate_uuid: "9868dce521294d0b816bba7648f4ae5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" is_pre_election: true
I20260709 13:23:33.810856 25503 ts_tablet_manager.cc:1434] T d053b9ae38de4173a0aded4da494bd98 P 233e7d29aac04505bf4e65f0a39a0c4c: Time spent starting tablet: real 0.119s	user 0.012s	sys 0.000s
I20260709 13:23:33.812247 25503 tablet_bootstrap.cc:492] T 9e5d70664720424d951f269d9d7f4346 P 233e7d29aac04505bf4e65f0a39a0c4c: Bootstrap starting.
I20260709 13:23:33.820643 25503 tablet_bootstrap.cc:654] T 9e5d70664720424d951f269d9d7f4346 P 233e7d29aac04505bf4e65f0a39a0c4c: Neither blocks nor log segments found. Creating new log.
I20260709 13:23:33.810242 25590 raft_consensus.cc:515] T ba8a000f5abe4158965449995af84b18 P 233e7d29aac04505bf4e65f0a39a0c4c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } }
I20260709 13:23:33.831357 25591 raft_consensus.cc:515] T 5c4ab1978a47481c8576d466bacf49d2 P 233e7d29aac04505bf4e65f0a39a0c4c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } }
I20260709 13:23:33.831916 25504 tablet_bootstrap.cc:492] T ce98a3f3b3eb4c63bcaa355ec7cd64a4 P 3e07d64466934da4bb3c02262b1c2e38: No bootstrap required, opened a new log
I20260709 13:23:33.832489 25508 tablet_bootstrap.cc:492] T 9e5d70664720424d951f269d9d7f4346 P 9868dce521294d0b816bba7648f4ae5a: No bootstrap required, opened a new log
I20260709 13:23:33.832691 25311 raft_consensus.cc:2393] T 5c4ab1978a47481c8576d466bacf49d2 P 233e7d29aac04505bf4e65f0a39a0c4c [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9868dce521294d0b816bba7648f4ae5a in current term 1: Already voted for candidate 233e7d29aac04505bf4e65f0a39a0c4c in this term.
I20260709 13:23:33.833031 25508 ts_tablet_manager.cc:1403] T 9e5d70664720424d951f269d9d7f4346 P 9868dce521294d0b816bba7648f4ae5a: Time spent bootstrapping tablet: real 0.107s	user 0.024s	sys 0.033s
I20260709 13:23:33.833634 25591 leader_election.cc:290] T 5c4ab1978a47481c8576d466bacf49d2 P 233e7d29aac04505bf4e65f0a39a0c4c [CANDIDATE]: Term 1 election: Requested vote from peers 3e07d64466934da4bb3c02262b1c2e38 (127.14.177.131:41539), 9868dce521294d0b816bba7648f4ae5a (127.14.177.132:33683)
I20260709 13:23:33.832499 25504 ts_tablet_manager.cc:1403] T ce98a3f3b3eb4c63bcaa355ec7cd64a4 P 3e07d64466934da4bb3c02262b1c2e38: Time spent bootstrapping tablet: real 0.247s	user 0.019s	sys 0.013s
I20260709 13:23:33.838663 25644 raft_consensus.cc:515] T 280d65245f4f4358b54bef93e18aca92 P 9868dce521294d0b816bba7648f4ae5a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } }
I20260709 13:23:33.837172 25508 raft_consensus.cc:359] T 9e5d70664720424d951f269d9d7f4346 P 9868dce521294d0b816bba7648f4ae5a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } }
I20260709 13:23:33.839709 25508 raft_consensus.cc:385] T 9e5d70664720424d951f269d9d7f4346 P 9868dce521294d0b816bba7648f4ae5a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:23:33.840008 25508 raft_consensus.cc:740] T 9e5d70664720424d951f269d9d7f4346 P 9868dce521294d0b816bba7648f4ae5a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9868dce521294d0b816bba7648f4ae5a, State: Initialized, Role: FOLLOWER
I20260709 13:23:33.841640 25508 consensus_queue.cc:260] T 9e5d70664720424d951f269d9d7f4346 P 9868dce521294d0b816bba7648f4ae5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } }
I20260709 13:23:33.843726 25504 raft_consensus.cc:359] T ce98a3f3b3eb4c63bcaa355ec7cd64a4 P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } }
I20260709 13:23:33.844756 25504 raft_consensus.cc:385] T ce98a3f3b3eb4c63bcaa355ec7cd64a4 P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:23:33.845108 25504 raft_consensus.cc:740] T ce98a3f3b3eb4c63bcaa355ec7cd64a4 P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e07d64466934da4bb3c02262b1c2e38, State: Initialized, Role: FOLLOWER
I20260709 13:23:33.857465 25508 ts_tablet_manager.cc:1434] T 9e5d70664720424d951f269d9d7f4346 P 9868dce521294d0b816bba7648f4ae5a: Time spent starting tablet: real 0.024s	user 0.009s	sys 0.000s
I20260709 13:23:33.857407 25385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "280d65245f4f4358b54bef93e18aca92" candidate_uuid: "9868dce521294d0b816bba7648f4ae5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e07d64466934da4bb3c02262b1c2e38"
I20260709 13:23:33.857818 25235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "280d65245f4f4358b54bef93e18aca92" candidate_uuid: "9868dce521294d0b816bba7648f4ae5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43999d813e564dc688fad116090c95ca"
I20260709 13:23:33.858213 25385 raft_consensus.cc:3060] T 280d65245f4f4358b54bef93e18aca92 P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:23:33.858402 25235 raft_consensus.cc:3060] T 280d65245f4f4358b54bef93e18aca92 P 43999d813e564dc688fad116090c95ca [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:23:33.858702 25508 tablet_bootstrap.cc:492] T ba8a000f5abe4158965449995af84b18 P 9868dce521294d0b816bba7648f4ae5a: Bootstrap starting.
I20260709 13:23:33.851575 25563 raft_consensus.cc:493] T 51a11db644124b63bdf5f457c967473d P 9868dce521294d0b816bba7648f4ae5a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:23:33.865303 25508 tablet_bootstrap.cc:654] T ba8a000f5abe4158965449995af84b18 P 9868dce521294d0b816bba7648f4ae5a: Neither blocks nor log segments found. Creating new log.
I20260709 13:23:33.864953 25563 raft_consensus.cc:515] T 51a11db644124b63bdf5f457c967473d P 9868dce521294d0b816bba7648f4ae5a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } }
I20260709 13:23:33.855305 25504 consensus_queue.cc:260] T ce98a3f3b3eb4c63bcaa355ec7cd64a4 P 3e07d64466934da4bb3c02262b1c2e38 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } }
I20260709 13:23:33.834592 25590 leader_election.cc:290] T ba8a000f5abe4158965449995af84b18 P 233e7d29aac04505bf4e65f0a39a0c4c [CANDIDATE]: Term 1 election: Requested vote from peers 3e07d64466934da4bb3c02262b1c2e38 (127.14.177.131:41539), 9868dce521294d0b816bba7648f4ae5a (127.14.177.132:33683)
I20260709 13:23:33.843551 25644 leader_election.cc:290] T 280d65245f4f4358b54bef93e18aca92 P 9868dce521294d0b816bba7648f4ae5a [CANDIDATE]: Term 1 election: Requested vote from peers 3e07d64466934da4bb3c02262b1c2e38 (127.14.177.131:41539), 43999d813e564dc688fad116090c95ca (127.14.177.129:38155)
I20260709 13:23:33.876413 25311 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51a11db644124b63bdf5f457c967473d" candidate_uuid: "9868dce521294d0b816bba7648f4ae5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" is_pre_election: true
W20260709 13:23:33.878155 25425 leader_election.cc:343] T 51a11db644124b63bdf5f457c967473d P 9868dce521294d0b816bba7648f4ae5a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 233e7d29aac04505bf4e65f0a39a0c4c (127.14.177.130:42957): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:23:33.867180 25563 leader_election.cc:290] T 51a11db644124b63bdf5f457c967473d P 9868dce521294d0b816bba7648f4ae5a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 43999d813e564dc688fad116090c95ca (127.14.177.129:38155), 233e7d29aac04505bf4e65f0a39a0c4c (127.14.177.130:42957)
I20260709 13:23:33.873241 25234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51a11db644124b63bdf5f457c967473d" candidate_uuid: "9868dce521294d0b816bba7648f4ae5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43999d813e564dc688fad116090c95ca" is_pre_election: true
W20260709 13:23:33.891176 25424 leader_election.cc:343] T 51a11db644124b63bdf5f457c967473d P 9868dce521294d0b816bba7648f4ae5a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 43999d813e564dc688fad116090c95ca (127.14.177.129:38155): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:23:33.891937 25424 leader_election.cc:304] T 51a11db644124b63bdf5f457c967473d P 9868dce521294d0b816bba7648f4ae5a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9868dce521294d0b816bba7648f4ae5a; no voters: 233e7d29aac04505bf4e65f0a39a0c4c, 43999d813e564dc688fad116090c95ca
I20260709 13:23:33.893396 25563 raft_consensus.cc:2749] T 51a11db644124b63bdf5f457c967473d P 9868dce521294d0b816bba7648f4ae5a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:23:33.893963 25235 raft_consensus.cc:2468] T 280d65245f4f4358b54bef93e18aca92 P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9868dce521294d0b816bba7648f4ae5a in term 1.
W20260709 13:23:33.880964 25274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:33683, user_credentials={real_user=slave}} blocked reactor thread for 125877us
I20260709 13:23:33.896661 25384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba8a000f5abe4158965449995af84b18" candidate_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e07d64466934da4bb3c02262b1c2e38"
I20260709 13:23:33.897307 25384 raft_consensus.cc:3060] T ba8a000f5abe4158965449995af84b18 P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:23:33.898347 25456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c4ab1978a47481c8576d466bacf49d2" candidate_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9868dce521294d0b816bba7648f4ae5a"
I20260709 13:23:33.838228 25650 raft_consensus.cc:515] T 5c4ab1978a47481c8576d466bacf49d2 P 9868dce521294d0b816bba7648f4ae5a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } }
I20260709 13:23:33.900849 25504 ts_tablet_manager.cc:1434] T ce98a3f3b3eb4c63bcaa355ec7cd64a4 P 3e07d64466934da4bb3c02262b1c2e38: Time spent starting tablet: real 0.065s	user 0.009s	sys 0.000s
I20260709 13:23:33.902959 25504 tablet_bootstrap.cc:492] T ad32097cd4dc48b9a69595f1651abd07 P 3e07d64466934da4bb3c02262b1c2e38: Bootstrap starting.
I20260709 13:23:33.899634 25383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c4ab1978a47481c8576d466bacf49d2" candidate_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e07d64466934da4bb3c02262b1c2e38"
I20260709 13:23:33.906718 25383 raft_consensus.cc:3060] T 5c4ab1978a47481c8576d466bacf49d2 P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:23:33.909386 25458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba8a000f5abe4158965449995af84b18" candidate_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9868dce521294d0b816bba7648f4ae5a"
I20260709 13:23:33.916239 25424 leader_election.cc:304] T 280d65245f4f4358b54bef93e18aca92 P 9868dce521294d0b816bba7648f4ae5a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 43999d813e564dc688fad116090c95ca, 9868dce521294d0b816bba7648f4ae5a; no voters: 
I20260709 13:23:33.917738 25563 raft_consensus.cc:2804] T 280d65245f4f4358b54bef93e18aca92 P 9868dce521294d0b816bba7648f4ae5a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:23:33.918257 25563 raft_consensus.cc:697] T 280d65245f4f4358b54bef93e18aca92 P 9868dce521294d0b816bba7648f4ae5a [term 1 LEADER]: Becoming Leader. State: Replica: 9868dce521294d0b816bba7648f4ae5a, State: Running, Role: LEADER
I20260709 13:23:33.919077 25563 consensus_queue.cc:237] T 280d65245f4f4358b54bef93e18aca92 P 9868dce521294d0b816bba7648f4ae5a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } }
I20260709 13:23:33.926179 25384 raft_consensus.cc:2468] T ba8a000f5abe4158965449995af84b18 P 3e07d64466934da4bb3c02262b1c2e38 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 233e7d29aac04505bf4e65f0a39a0c4c in term 1.
W20260709 13:23:33.926478 25274 leader_election.cc:343] T ba8a000f5abe4158965449995af84b18 P 233e7d29aac04505bf4e65f0a39a0c4c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 9868dce521294d0b816bba7648f4ae5a (127.14.177.132:33683): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:23:33.929527 25274 leader_election.cc:304] T ba8a000f5abe4158965449995af84b18 P 233e7d29aac04505bf4e65f0a39a0c4c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 233e7d29aac04505bf4e65f0a39a0c4c, 3e07d64466934da4bb3c02262b1c2e38; no voters: 9868dce521294d0b816bba7648f4ae5a
I20260709 13:23:33.930763 25676 raft_consensus.cc:2804] T ba8a000f5abe4158965449995af84b18 P 233e7d29aac04505bf4e65f0a39a0c4c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:23:33.931324 25676 raft_consensus.cc:697] T ba8a000f5abe4158965449995af84b18 P 233e7d29aac04505bf4e65f0a39a0c4c [term 1 LEADER]: Becoming Leader. State: Replica: 233e7d29aac04505bf4e65f0a39a0c4c, State: Running, Role: LEADER
I20260709 13:23:33.932349 25676 consensus_queue.cc:237] T ba8a000f5abe4158965449995af84b18 P 233e7d29aac04505bf4e65f0a39a0c4c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } }
I20260709 13:23:33.934266 25505 tablet_bootstrap.cc:492] T beacc815ef954fb692e90544bfa93f05 P 43999d813e564dc688fad116090c95ca: No bootstrap required, opened a new log
I20260709 13:23:33.934849 25505 ts_tablet_manager.cc:1403] T beacc815ef954fb692e90544bfa93f05 P 43999d813e564dc688fad116090c95ca: Time spent bootstrapping tablet: real 0.245s	user 0.015s	sys 0.003s
I20260709 13:23:33.935686 25456 raft_consensus.cc:2393] T 5c4ab1978a47481c8576d466bacf49d2 P 9868dce521294d0b816bba7648f4ae5a [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 233e7d29aac04505bf4e65f0a39a0c4c in current term 1: Already voted for candidate 9868dce521294d0b816bba7648f4ae5a in this term.
I20260709 13:23:33.937752 25505 raft_consensus.cc:359] T beacc815ef954fb692e90544bfa93f05 P 43999d813e564dc688fad116090c95ca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } }
I20260709 13:23:33.938499 25505 raft_consensus.cc:385] T beacc815ef954fb692e90544bfa93f05 P 43999d813e564dc688fad116090c95ca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:23:33.938856 25505 raft_consensus.cc:740] T beacc815ef954fb692e90544bfa93f05 P 43999d813e564dc688fad116090c95ca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43999d813e564dc688fad116090c95ca, State: Initialized, Role: FOLLOWER
I20260709 13:23:33.939669 25311 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c4ab1978a47481c8576d466bacf49d2" candidate_uuid: "9868dce521294d0b816bba7648f4ae5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "233e7d29aac04505bf4e65f0a39a0c4c"
I20260709 13:23:33.939568 25505 consensus_queue.cc:260] T beacc815ef954fb692e90544bfa93f05 P 43999d813e564dc688fad116090c95ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } }
I20260709 13:23:33.940479 25311 raft_consensus.cc:2393] T 5c4ab1978a47481c8576d466bacf49d2 P 233e7d29aac04505bf4e65f0a39a0c4c [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 9868dce521294d0b816bba7648f4ae5a in current term 1: Already voted for candidate 233e7d29aac04505bf4e65f0a39a0c4c in this term.
I20260709 13:23:33.941442 25644 consensus_queue.cc:1048] T d053b9ae38de4173a0aded4da494bd98 P 9868dce521294d0b816bba7648f4ae5a [LEADER]: Connected to new peer: Peer: permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:23:33.942732 25650 leader_election.cc:290] T 5c4ab1978a47481c8576d466bacf49d2 P 9868dce521294d0b816bba7648f4ae5a [CANDIDATE]: Term 1 election: Requested vote from peers 233e7d29aac04505bf4e65f0a39a0c4c (127.14.177.130:42957), 3e07d64466934da4bb3c02262b1c2e38 (127.14.177.131:41539)
I20260709 13:23:33.943429 25384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c4ab1978a47481c8576d466bacf49d2" candidate_uuid: "9868dce521294d0b816bba7648f4ae5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e07d64466934da4bb3c02262b1c2e38"
I20260709 13:23:33.944082 25385 raft_consensus.cc:2468] T 280d65245f4f4358b54bef93e18aca92 P 3e07d64466934da4bb3c02262b1c2e38 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9868dce521294d0b816bba7648f4ae5a in term 1.
I20260709 13:23:33.944600 25505 ts_tablet_manager.cc:1434] T beacc815ef954fb692e90544bfa93f05 P 43999d813e564dc688fad116090c95ca: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.002s
I20260709 13:23:33.945552 25505 tablet_bootstrap.cc:492] T 9ffa42c3b2ff48bc87310e9ab5d218ce P 43999d813e564dc688fad116090c95ca: Bootstrap starting.
I20260709 13:23:33.968546 25503 tablet_bootstrap.cc:492] T 9e5d70664720424d951f269d9d7f4346 P 233e7d29aac04505bf4e65f0a39a0c4c: No bootstrap required, opened a new log
I20260709 13:23:33.969125 25503 ts_tablet_manager.cc:1403] T 9e5d70664720424d951f269d9d7f4346 P 233e7d29aac04505bf4e65f0a39a0c4c: Time spent bootstrapping tablet: real 0.157s	user 0.043s	sys 0.011s
I20260709 13:23:33.954180 25422 leader_election.cc:304] T 5c4ab1978a47481c8576d466bacf49d2 P 9868dce521294d0b816bba7648f4ae5a [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9868dce521294d0b816bba7648f4ae5a; no voters: 233e7d29aac04505bf4e65f0a39a0c4c, 3e07d64466934da4bb3c02262b1c2e38
I20260709 13:23:33.971083 25650 raft_consensus.cc:2749] T 5c4ab1978a47481c8576d466bacf49d2 P 9868dce521294d0b816bba7648f4ae5a [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:23:33.972119 25503 raft_consensus.cc:359] T 9e5d70664720424d951f269d9d7f4346 P 233e7d29aac04505bf4e65f0a39a0c4c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } }
I20260709 13:23:33.972944 25503 raft_consensus.cc:385] T 9e5d70664720424d951f269d9d7f4346 P 233e7d29aac04505bf4e65f0a39a0c4c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:23:33.973402 25503 raft_consensus.cc:740] T 9e5d70664720424d951f269d9d7f4346 P 233e7d29aac04505bf4e65f0a39a0c4c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 233e7d29aac04505bf4e65f0a39a0c4c, State: Initialized, Role: FOLLOWER
I20260709 13:23:33.974314 25503 consensus_queue.cc:260] T 9e5d70664720424d951f269d9d7f4346 P 233e7d29aac04505bf4e65f0a39a0c4c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } }
I20260709 13:23:33.980249 25503 ts_tablet_manager.cc:1434] T 9e5d70664720424d951f269d9d7f4346 P 233e7d29aac04505bf4e65f0a39a0c4c: Time spent starting tablet: real 0.011s	user 0.006s	sys 0.000s
I20260709 13:23:33.981292 25503 tablet_bootstrap.cc:492] T aad1f7937f854cc1a726223269da5d5a P 233e7d29aac04505bf4e65f0a39a0c4c: Bootstrap starting.
I20260709 13:23:33.977931 25383 raft_consensus.cc:2468] T 5c4ab1978a47481c8576d466bacf49d2 P 3e07d64466934da4bb3c02262b1c2e38 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 233e7d29aac04505bf4e65f0a39a0c4c in term 1.
I20260709 13:23:33.962949 25504 tablet_bootstrap.cc:654] T ad32097cd4dc48b9a69595f1651abd07 P 3e07d64466934da4bb3c02262b1c2e38: Neither blocks nor log segments found. Creating new log.
I20260709 13:23:33.990211 25508 tablet_bootstrap.cc:492] T ba8a000f5abe4158965449995af84b18 P 9868dce521294d0b816bba7648f4ae5a: No bootstrap required, opened a new log
I20260709 13:23:33.990885 25508 ts_tablet_manager.cc:1403] T ba8a000f5abe4158965449995af84b18 P 9868dce521294d0b816bba7648f4ae5a: Time spent bootstrapping tablet: real 0.132s	user 0.010s	sys 0.013s
I20260709 13:23:33.994247 25274 leader_election.cc:304] T 5c4ab1978a47481c8576d466bacf49d2 P 233e7d29aac04505bf4e65f0a39a0c4c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 233e7d29aac04505bf4e65f0a39a0c4c, 3e07d64466934da4bb3c02262b1c2e38; no voters: 9868dce521294d0b816bba7648f4ae5a
I20260709 13:23:33.995986 25673 raft_consensus.cc:2804] T 5c4ab1978a47481c8576d466bacf49d2 P 233e7d29aac04505bf4e65f0a39a0c4c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:23:33.996050 25508 raft_consensus.cc:359] T ba8a000f5abe4158965449995af84b18 P 9868dce521294d0b816bba7648f4ae5a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } }
I20260709 13:23:33.997437 25508 raft_consensus.cc:385] T ba8a000f5abe4158965449995af84b18 P 9868dce521294d0b816bba7648f4ae5a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:23:33.997934 25508 raft_consensus.cc:740] T ba8a000f5abe4158965449995af84b18 P 9868dce521294d0b816bba7648f4ae5a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9868dce521294d0b816bba7648f4ae5a, State: Initialized, Role: FOLLOWER
I20260709 13:23:33.997035 25673 raft_consensus.cc:697] T 5c4ab1978a47481c8576d466bacf49d2 P 233e7d29aac04505bf4e65f0a39a0c4c [term 1 LEADER]: Becoming Leader. State: Replica: 233e7d29aac04505bf4e65f0a39a0c4c, State: Running, Role: LEADER
I20260709 13:23:33.999135 25508 consensus_queue.cc:260] T ba8a000f5abe4158965449995af84b18 P 9868dce521294d0b816bba7648f4ae5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } }
I20260709 13:23:33.999786 25673 consensus_queue.cc:237] T 5c4ab1978a47481c8576d466bacf49d2 P 233e7d29aac04505bf4e65f0a39a0c4c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } }
I20260709 13:23:34.008735 25644 raft_consensus.cc:493] T 9e5d70664720424d951f269d9d7f4346 P 9868dce521294d0b816bba7648f4ae5a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:23:34.009299 25644 raft_consensus.cc:515] T 9e5d70664720424d951f269d9d7f4346 P 9868dce521294d0b816bba7648f4ae5a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } }
I20260709 13:23:34.010056 25508 ts_tablet_manager.cc:1434] T ba8a000f5abe4158965449995af84b18 P 9868dce521294d0b816bba7648f4ae5a: Time spent starting tablet: real 0.017s	user 0.003s	sys 0.004s
I20260709 13:23:34.011142 25508 tablet_bootstrap.cc:492] T aad1f7937f854cc1a726223269da5d5a P 9868dce521294d0b816bba7648f4ae5a: Bootstrap starting.
I20260709 13:23:34.017099 25644 leader_election.cc:290] T 9e5d70664720424d951f269d9d7f4346 P 9868dce521294d0b816bba7648f4ae5a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 43999d813e564dc688fad116090c95ca (127.14.177.129:38155), 233e7d29aac04505bf4e65f0a39a0c4c (127.14.177.130:42957)
I20260709 13:23:34.016837 25133 catalog_manager.cc:5697] T ba8a000f5abe4158965449995af84b18 P 233e7d29aac04505bf4e65f0a39a0c4c reported cstate change: term changed from 0 to 1, leader changed from <none> to 233e7d29aac04505bf4e65f0a39a0c4c (127.14.177.130). New cstate: current_term: 1 leader_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } health_report { overall_health: UNKNOWN } } }
I20260709 13:23:34.018764 25311 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e5d70664720424d951f269d9d7f4346" candidate_uuid: "9868dce521294d0b816bba7648f4ae5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" is_pre_election: true
I20260709 13:23:34.019516 25311 raft_consensus.cc:2468] T 9e5d70664720424d951f269d9d7f4346 P 233e7d29aac04505bf4e65f0a39a0c4c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9868dce521294d0b816bba7648f4ae5a in term 0.
I20260709 13:23:34.020977 25425 leader_election.cc:304] T 9e5d70664720424d951f269d9d7f4346 P 9868dce521294d0b816bba7648f4ae5a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 233e7d29aac04505bf4e65f0a39a0c4c, 9868dce521294d0b816bba7648f4ae5a; no voters: 
I20260709 13:23:34.022266 25309 raft_consensus.cc:3060] T d053b9ae38de4173a0aded4da494bd98 P 233e7d29aac04505bf4e65f0a39a0c4c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:23:34.022822 25235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e5d70664720424d951f269d9d7f4346" candidate_uuid: "9868dce521294d0b816bba7648f4ae5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43999d813e564dc688fad116090c95ca" is_pre_election: true
I20260709 13:23:34.023686 25235 raft_consensus.cc:2468] T 9e5d70664720424d951f269d9d7f4346 P 43999d813e564dc688fad116090c95ca [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9868dce521294d0b816bba7648f4ae5a in term 0.
I20260709 13:23:34.026707 25539 raft_consensus.cc:2804] T 9e5d70664720424d951f269d9d7f4346 P 9868dce521294d0b816bba7648f4ae5a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:23:34.027145 25539 raft_consensus.cc:493] T 9e5d70664720424d951f269d9d7f4346 P 9868dce521294d0b816bba7648f4ae5a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:23:34.027503 25539 raft_consensus.cc:3060] T 9e5d70664720424d951f269d9d7f4346 P 9868dce521294d0b816bba7648f4ae5a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:23:34.034934 25539 raft_consensus.cc:515] T 9e5d70664720424d951f269d9d7f4346 P 9868dce521294d0b816bba7648f4ae5a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } }
I20260709 13:23:34.036100 25134 catalog_manager.cc:5697] T 280d65245f4f4358b54bef93e18aca92 P 9868dce521294d0b816bba7648f4ae5a reported cstate change: term changed from 0 to 1, leader changed from <none> to 9868dce521294d0b816bba7648f4ae5a (127.14.177.132). New cstate: current_term: 1 leader_uuid: "9868dce521294d0b816bba7648f4ae5a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } health_report { overall_health: UNKNOWN } } }
I20260709 13:23:34.038300 25539 leader_election.cc:290] T 9e5d70664720424d951f269d9d7f4346 P 9868dce521294d0b816bba7648f4ae5a [CANDIDATE]: Term 1 election: Requested vote from peers 43999d813e564dc688fad116090c95ca (127.14.177.129:38155), 233e7d29aac04505bf4e65f0a39a0c4c (127.14.177.130:42957)
I20260709 13:23:34.038973 25503 tablet_bootstrap.cc:654] T aad1f7937f854cc1a726223269da5d5a P 233e7d29aac04505bf4e65f0a39a0c4c: Neither blocks nor log segments found. Creating new log.
I20260709 13:23:34.036695 25505 tablet_bootstrap.cc:654] T 9ffa42c3b2ff48bc87310e9ab5d218ce P 43999d813e564dc688fad116090c95ca: Neither blocks nor log segments found. Creating new log.
I20260709 13:23:34.043473 25642 raft_consensus.cc:493] T 9ffa42c3b2ff48bc87310e9ab5d218ce P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:23:34.065811 25133 catalog_manager.cc:5697] T 5c4ab1978a47481c8576d466bacf49d2 P 233e7d29aac04505bf4e65f0a39a0c4c reported cstate change: term changed from 0 to 1, leader changed from <none> to 233e7d29aac04505bf4e65f0a39a0c4c (127.14.177.130). New cstate: current_term: 1 leader_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } health_report { overall_health: UNKNOWN } } }
I20260709 13:23:34.066615 25642 raft_consensus.cc:515] T 9ffa42c3b2ff48bc87310e9ab5d218ce P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } }
I20260709 13:23:34.068897 25642 leader_election.cc:290] T 9ffa42c3b2ff48bc87310e9ab5d218ce P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9868dce521294d0b816bba7648f4ae5a (127.14.177.132:33683), 43999d813e564dc688fad116090c95ca (127.14.177.129:38155)
I20260709 13:23:34.045328 25568 raft_consensus.cc:493] T ce98a3f3b3eb4c63bcaa355ec7cd64a4 P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:23:34.070055 25568 raft_consensus.cc:515] T ce98a3f3b3eb4c63bcaa355ec7cd64a4 P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } }
I20260709 13:23:34.070309 25456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ffa42c3b2ff48bc87310e9ab5d218ce" candidate_uuid: "3e07d64466934da4bb3c02262b1c2e38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9868dce521294d0b816bba7648f4ae5a" is_pre_election: true
I20260709 13:23:34.071275 25505 tablet_bootstrap.cc:492] T 9ffa42c3b2ff48bc87310e9ab5d218ce P 43999d813e564dc688fad116090c95ca: No bootstrap required, opened a new log
I20260709 13:23:34.071758 25505 ts_tablet_manager.cc:1403] T 9ffa42c3b2ff48bc87310e9ab5d218ce P 43999d813e564dc688fad116090c95ca: Time spent bootstrapping tablet: real 0.126s	user 0.012s	sys 0.000s
W20260709 13:23:34.071863 25348 leader_election.cc:343] T 9ffa42c3b2ff48bc87310e9ab5d218ce P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9868dce521294d0b816bba7648f4ae5a (127.14.177.132:33683): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:23:34.072242 25568 leader_election.cc:290] T ce98a3f3b3eb4c63bcaa355ec7cd64a4 P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 43999d813e564dc688fad116090c95ca (127.14.177.129:38155), 233e7d29aac04505bf4e65f0a39a0c4c (127.14.177.130:42957)
I20260709 13:23:34.073392 25307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce98a3f3b3eb4c63bcaa355ec7cd64a4" candidate_uuid: "3e07d64466934da4bb3c02262b1c2e38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" is_pre_election: true
W20260709 13:23:34.074946 25351 leader_election.cc:343] T ce98a3f3b3eb4c63bcaa355ec7cd64a4 P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 233e7d29aac04505bf4e65f0a39a0c4c (127.14.177.130:42957): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:23:34.047960 25182 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:23:34.075080 25505 raft_consensus.cc:359] T 9ffa42c3b2ff48bc87310e9ab5d218ce P 43999d813e564dc688fad116090c95ca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } }
I20260709 13:23:34.052639 25311 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e5d70664720424d951f269d9d7f4346" candidate_uuid: "9868dce521294d0b816bba7648f4ae5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "233e7d29aac04505bf4e65f0a39a0c4c"
I20260709 13:23:34.075829 25505 raft_consensus.cc:385] T 9ffa42c3b2ff48bc87310e9ab5d218ce P 43999d813e564dc688fad116090c95ca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:23:34.076174 25505 raft_consensus.cc:740] T 9ffa42c3b2ff48bc87310e9ab5d218ce P 43999d813e564dc688fad116090c95ca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43999d813e564dc688fad116090c95ca, State: Initialized, Role: FOLLOWER
I20260709 13:23:34.076223 25311 raft_consensus.cc:3060] T 9e5d70664720424d951f269d9d7f4346 P 233e7d29aac04505bf4e65f0a39a0c4c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:23:34.076952 25505 consensus_queue.cc:260] T 9ffa42c3b2ff48bc87310e9ab5d218ce P 43999d813e564dc688fad116090c95ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } }
I20260709 13:23:34.078289 25503 tablet_bootstrap.cc:492] T aad1f7937f854cc1a726223269da5d5a P 233e7d29aac04505bf4e65f0a39a0c4c: No bootstrap required, opened a new log
I20260709 13:23:34.078810 25503 ts_tablet_manager.cc:1403] T aad1f7937f854cc1a726223269da5d5a P 233e7d29aac04505bf4e65f0a39a0c4c: Time spent bootstrapping tablet: real 0.098s	user 0.008s	sys 0.003s
I20260709 13:23:34.079727 25233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce98a3f3b3eb4c63bcaa355ec7cd64a4" candidate_uuid: "3e07d64466934da4bb3c02262b1c2e38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43999d813e564dc688fad116090c95ca" is_pre_election: true
I20260709 13:23:34.079032 25234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ffa42c3b2ff48bc87310e9ab5d218ce" candidate_uuid: "3e07d64466934da4bb3c02262b1c2e38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43999d813e564dc688fad116090c95ca" is_pre_election: true
I20260709 13:23:34.081136 25234 raft_consensus.cc:2468] T 9ffa42c3b2ff48bc87310e9ab5d218ce P 43999d813e564dc688fad116090c95ca [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3e07d64466934da4bb3c02262b1c2e38 in term 0.
I20260709 13:23:34.081571 25503 raft_consensus.cc:359] T aad1f7937f854cc1a726223269da5d5a P 233e7d29aac04505bf4e65f0a39a0c4c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } }
I20260709 13:23:34.082281 25503 raft_consensus.cc:385] T aad1f7937f854cc1a726223269da5d5a P 233e7d29aac04505bf4e65f0a39a0c4c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:23:34.054670 25563 consensus_queue.cc:1048] T d053b9ae38de4173a0aded4da494bd98 P 9868dce521294d0b816bba7648f4ae5a [LEADER]: Connected to new peer: Peer: permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.028s
I20260709 13:23:34.056734 25235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e5d70664720424d951f269d9d7f4346" candidate_uuid: "9868dce521294d0b816bba7648f4ae5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43999d813e564dc688fad116090c95ca"
I20260709 13:23:34.083971 25235 raft_consensus.cc:3060] T 9e5d70664720424d951f269d9d7f4346 P 43999d813e564dc688fad116090c95ca [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:23:34.082998 25350 leader_election.cc:343] T ce98a3f3b3eb4c63bcaa355ec7cd64a4 P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 43999d813e564dc688fad116090c95ca (127.14.177.129:38155): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:23:34.064868 25504 tablet_bootstrap.cc:492] T ad32097cd4dc48b9a69595f1651abd07 P 3e07d64466934da4bb3c02262b1c2e38: No bootstrap required, opened a new log
I20260709 13:23:34.084740 25350 leader_election.cc:304] T ce98a3f3b3eb4c63bcaa355ec7cd64a4 P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3e07d64466934da4bb3c02262b1c2e38; no voters: 233e7d29aac04505bf4e65f0a39a0c4c, 43999d813e564dc688fad116090c95ca
I20260709 13:23:34.085083 25504 ts_tablet_manager.cc:1403] T ad32097cd4dc48b9a69595f1651abd07 P 3e07d64466934da4bb3c02262b1c2e38: Time spent bootstrapping tablet: real 0.182s	user 0.012s	sys 0.001s
I20260709 13:23:34.085544 25568 raft_consensus.cc:2749] T ce98a3f3b3eb4c63bcaa355ec7cd64a4 P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:23:34.086822 25350 leader_election.cc:304] T 9ffa42c3b2ff48bc87310e9ab5d218ce P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3e07d64466934da4bb3c02262b1c2e38, 43999d813e564dc688fad116090c95ca; no voters: 9868dce521294d0b816bba7648f4ae5a
I20260709 13:23:34.088135 25504 raft_consensus.cc:359] T ad32097cd4dc48b9a69595f1651abd07 P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } }
I20260709 13:23:34.090544 25504 raft_consensus.cc:385] T ad32097cd4dc48b9a69595f1651abd07 P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:23:34.090878 25504 raft_consensus.cc:740] T ad32097cd4dc48b9a69595f1651abd07 P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e07d64466934da4bb3c02262b1c2e38, State: Initialized, Role: FOLLOWER
I20260709 13:23:34.091578 25504 consensus_queue.cc:260] T ad32097cd4dc48b9a69595f1651abd07 P 3e07d64466934da4bb3c02262b1c2e38 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } }
I20260709 13:23:34.093636 25504 ts_tablet_manager.cc:1434] T ad32097cd4dc48b9a69595f1651abd07 P 3e07d64466934da4bb3c02262b1c2e38: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.000s
I20260709 13:23:34.093921 25573 consensus_queue.cc:1048] T beacc815ef954fb692e90544bfa93f05 P 3e07d64466934da4bb3c02262b1c2e38 [LEADER]: Connected to new peer: Peer: permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:23:34.082569 25503 raft_consensus.cc:740] T aad1f7937f854cc1a726223269da5d5a P 233e7d29aac04505bf4e65f0a39a0c4c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 233e7d29aac04505bf4e65f0a39a0c4c, State: Initialized, Role: FOLLOWER
I20260709 13:23:34.094888 25504 tablet_bootstrap.cc:492] T c902bbc7ec0848ba82a04870eab1e7f5 P 3e07d64466934da4bb3c02262b1c2e38: Bootstrap starting.
I20260709 13:23:34.095319 25503 consensus_queue.cc:260] T aad1f7937f854cc1a726223269da5d5a P 233e7d29aac04505bf4e65f0a39a0c4c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } }
I20260709 13:23:34.098268 25235 raft_consensus.cc:2468] T 9e5d70664720424d951f269d9d7f4346 P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9868dce521294d0b816bba7648f4ae5a in term 1.
I20260709 13:23:34.099687 25424 leader_election.cc:304] T 9e5d70664720424d951f269d9d7f4346 P 9868dce521294d0b816bba7648f4ae5a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 43999d813e564dc688fad116090c95ca, 9868dce521294d0b816bba7648f4ae5a; no voters: 
I20260709 13:23:34.100541 25563 raft_consensus.cc:2804] T 9e5d70664720424d951f269d9d7f4346 P 9868dce521294d0b816bba7648f4ae5a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:23:34.100831 25503 ts_tablet_manager.cc:1434] T aad1f7937f854cc1a726223269da5d5a P 233e7d29aac04505bf4e65f0a39a0c4c: Time spent starting tablet: real 0.022s	user 0.005s	sys 0.000s
I20260709 13:23:34.101022 25563 raft_consensus.cc:697] T 9e5d70664720424d951f269d9d7f4346 P 9868dce521294d0b816bba7648f4ae5a [term 1 LEADER]: Becoming Leader. State: Replica: 9868dce521294d0b816bba7648f4ae5a, State: Running, Role: LEADER
I20260709 13:23:34.079069 25508 tablet_bootstrap.cc:654] T aad1f7937f854cc1a726223269da5d5a P 9868dce521294d0b816bba7648f4ae5a: Neither blocks nor log segments found. Creating new log.
I20260709 13:23:34.094132 25311 raft_consensus.cc:2468] T 9e5d70664720424d951f269d9d7f4346 P 233e7d29aac04505bf4e65f0a39a0c4c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9868dce521294d0b816bba7648f4ae5a in term 1.
I20260709 13:23:34.088992 25568 raft_consensus.cc:2804] T 9ffa42c3b2ff48bc87310e9ab5d218ce P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:23:34.088620 25505 ts_tablet_manager.cc:1434] T 9ffa42c3b2ff48bc87310e9ab5d218ce P 43999d813e564dc688fad116090c95ca: Time spent starting tablet: real 0.017s	user 0.006s	sys 0.000s
I20260709 13:23:34.101766 25568 raft_consensus.cc:493] T 9ffa42c3b2ff48bc87310e9ab5d218ce P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:23:34.108253 25568 raft_consensus.cc:3060] T 9ffa42c3b2ff48bc87310e9ab5d218ce P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:23:34.104017 25503 tablet_bootstrap.cc:492] T ce98a3f3b3eb4c63bcaa355ec7cd64a4 P 233e7d29aac04505bf4e65f0a39a0c4c: Bootstrap starting.
I20260709 13:23:34.102106 25563 consensus_queue.cc:237] T 9e5d70664720424d951f269d9d7f4346 P 9868dce521294d0b816bba7648f4ae5a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } }
I20260709 13:23:34.118796 25505 tablet_bootstrap.cc:492] T ce98a3f3b3eb4c63bcaa355ec7cd64a4 P 43999d813e564dc688fad116090c95ca: Bootstrap starting.
I20260709 13:23:34.119886 25508 tablet_bootstrap.cc:492] T aad1f7937f854cc1a726223269da5d5a P 9868dce521294d0b816bba7648f4ae5a: No bootstrap required, opened a new log
I20260709 13:23:34.128496 25508 ts_tablet_manager.cc:1403] T aad1f7937f854cc1a726223269da5d5a P 9868dce521294d0b816bba7648f4ae5a: Time spent bootstrapping tablet: real 0.118s	user 0.011s	sys 0.026s
I20260709 13:23:34.129666 25503 tablet_bootstrap.cc:654] T ce98a3f3b3eb4c63bcaa355ec7cd64a4 P 233e7d29aac04505bf4e65f0a39a0c4c: Neither blocks nor log segments found. Creating new log.
I20260709 13:23:34.131665 25508 raft_consensus.cc:359] T aad1f7937f854cc1a726223269da5d5a P 9868dce521294d0b816bba7648f4ae5a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } }
I20260709 13:23:34.132434 25508 raft_consensus.cc:385] T aad1f7937f854cc1a726223269da5d5a P 9868dce521294d0b816bba7648f4ae5a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:23:34.132813 25508 raft_consensus.cc:740] T aad1f7937f854cc1a726223269da5d5a P 9868dce521294d0b816bba7648f4ae5a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9868dce521294d0b816bba7648f4ae5a, State: Initialized, Role: FOLLOWER
I20260709 13:23:34.133694 25508 consensus_queue.cc:260] T aad1f7937f854cc1a726223269da5d5a P 9868dce521294d0b816bba7648f4ae5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } }
I20260709 13:23:34.101435 25504 tablet_bootstrap.cc:654] T c902bbc7ec0848ba82a04870eab1e7f5 P 3e07d64466934da4bb3c02262b1c2e38: Neither blocks nor log segments found. Creating new log.
I20260709 13:23:34.135300 25505 tablet_bootstrap.cc:654] T ce98a3f3b3eb4c63bcaa355ec7cd64a4 P 43999d813e564dc688fad116090c95ca: Neither blocks nor log segments found. Creating new log.
I20260709 13:23:34.145628 25568 raft_consensus.cc:515] T 9ffa42c3b2ff48bc87310e9ab5d218ce P 3e07d64466934da4bb3c02262b1c2e38 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } }
I20260709 13:23:34.148697 25456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ffa42c3b2ff48bc87310e9ab5d218ce" candidate_uuid: "3e07d64466934da4bb3c02262b1c2e38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9868dce521294d0b816bba7648f4ae5a"
W20260709 13:23:34.150173 25348 leader_election.cc:343] T 9ffa42c3b2ff48bc87310e9ab5d218ce P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 9868dce521294d0b816bba7648f4ae5a (127.14.177.132:33683): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:23:34.153098 25235 raft_consensus.cc:3060] T beacc815ef954fb692e90544bfa93f05 P 43999d813e564dc688fad116090c95ca [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:23:34.146296 25134 catalog_manager.cc:5697] T 9e5d70664720424d951f269d9d7f4346 P 9868dce521294d0b816bba7648f4ae5a reported cstate change: term changed from 0 to 1, leader changed from <none> to 9868dce521294d0b816bba7648f4ae5a (127.14.177.132). New cstate: current_term: 1 leader_uuid: "9868dce521294d0b816bba7648f4ae5a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } health_report { overall_health: UNKNOWN } } }
I20260709 13:23:34.160123 25508 ts_tablet_manager.cc:1434] T aad1f7937f854cc1a726223269da5d5a P 9868dce521294d0b816bba7648f4ae5a: Time spent starting tablet: real 0.031s	user 0.007s	sys 0.000s
I20260709 13:23:34.161271 25508 tablet_bootstrap.cc:492] T ad32097cd4dc48b9a69595f1651abd07 P 9868dce521294d0b816bba7648f4ae5a: Bootstrap starting.
I20260709 13:23:34.162678 25622 consensus_queue.cc:1048] T beacc815ef954fb692e90544bfa93f05 P 3e07d64466934da4bb3c02262b1c2e38 [LEADER]: Connected to new peer: Peer: permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:23:34.168877 25503 tablet_bootstrap.cc:492] T ce98a3f3b3eb4c63bcaa355ec7cd64a4 P 233e7d29aac04505bf4e65f0a39a0c4c: No bootstrap required, opened a new log
I20260709 13:23:34.168882 25235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ffa42c3b2ff48bc87310e9ab5d218ce" candidate_uuid: "3e07d64466934da4bb3c02262b1c2e38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43999d813e564dc688fad116090c95ca"
I20260709 13:23:34.169935 25508 tablet_bootstrap.cc:654] T ad32097cd4dc48b9a69595f1651abd07 P 9868dce521294d0b816bba7648f4ae5a: Neither blocks nor log segments found. Creating new log.
I20260709 13:23:34.170357 25235 raft_consensus.cc:3060] T 9ffa42c3b2ff48bc87310e9ab5d218ce P 43999d813e564dc688fad116090c95ca [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:23:34.169977 25503 ts_tablet_manager.cc:1403] T ce98a3f3b3eb4c63bcaa355ec7cd64a4 P 233e7d29aac04505bf4e65f0a39a0c4c: Time spent bootstrapping tablet: real 0.066s	user 0.015s	sys 0.003s
I20260709 13:23:34.179692 25503 raft_consensus.cc:359] T ce98a3f3b3eb4c63bcaa355ec7cd64a4 P 233e7d29aac04505bf4e65f0a39a0c4c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } }
I20260709 13:23:34.180958 25503 raft_consensus.cc:385] T ce98a3f3b3eb4c63bcaa355ec7cd64a4 P 233e7d29aac04505bf4e65f0a39a0c4c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:23:34.183929 25503 raft_consensus.cc:740] T ce98a3f3b3eb4c63bcaa355ec7cd64a4 P 233e7d29aac04505bf4e65f0a39a0c4c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 233e7d29aac04505bf4e65f0a39a0c4c, State: Initialized, Role: FOLLOWER
I20260709 13:23:34.168963 25568 leader_election.cc:290] T 9ffa42c3b2ff48bc87310e9ab5d218ce P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 1 election: Requested vote from peers 9868dce521294d0b816bba7648f4ae5a (127.14.177.132:33683), 43999d813e564dc688fad116090c95ca (127.14.177.129:38155)
I20260709 13:23:34.185245 25503 consensus_queue.cc:260] T ce98a3f3b3eb4c63bcaa355ec7cd64a4 P 233e7d29aac04505bf4e65f0a39a0c4c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } }
I20260709 13:23:34.189128 25673 raft_consensus.cc:493] T ce98a3f3b3eb4c63bcaa355ec7cd64a4 P 233e7d29aac04505bf4e65f0a39a0c4c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:23:34.189950 25504 tablet_bootstrap.cc:492] T c902bbc7ec0848ba82a04870eab1e7f5 P 3e07d64466934da4bb3c02262b1c2e38: No bootstrap required, opened a new log
I20260709 13:23:34.189806 25673 raft_consensus.cc:515] T ce98a3f3b3eb4c63bcaa355ec7cd64a4 P 233e7d29aac04505bf4e65f0a39a0c4c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } }
I20260709 13:23:34.192426 25673 leader_election.cc:290] T ce98a3f3b3eb4c63bcaa355ec7cd64a4 P 233e7d29aac04505bf4e65f0a39a0c4c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 43999d813e564dc688fad116090c95ca (127.14.177.129:38155), 3e07d64466934da4bb3c02262b1c2e38 (127.14.177.131:41539)
I20260709 13:23:34.190486 25504 ts_tablet_manager.cc:1403] T c902bbc7ec0848ba82a04870eab1e7f5 P 3e07d64466934da4bb3c02262b1c2e38: Time spent bootstrapping tablet: real 0.096s	user 0.014s	sys 0.007s
I20260709 13:23:34.193169 25233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce98a3f3b3eb4c63bcaa355ec7cd64a4" candidate_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43999d813e564dc688fad116090c95ca" is_pre_election: true
W20260709 13:23:34.194738 25276 leader_election.cc:343] T ce98a3f3b3eb4c63bcaa355ec7cd64a4 P 233e7d29aac04505bf4e65f0a39a0c4c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 43999d813e564dc688fad116090c95ca (127.14.177.129:38155): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:23:34.195569 25383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce98a3f3b3eb4c63bcaa355ec7cd64a4" candidate_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e07d64466934da4bb3c02262b1c2e38" is_pre_election: true
I20260709 13:23:34.196303 25383 raft_consensus.cc:2468] T ce98a3f3b3eb4c63bcaa355ec7cd64a4 P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 233e7d29aac04505bf4e65f0a39a0c4c in term 0.
I20260709 13:23:34.196383 25235 raft_consensus.cc:2468] T 9ffa42c3b2ff48bc87310e9ab5d218ce P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3e07d64466934da4bb3c02262b1c2e38 in term 1.
I20260709 13:23:34.197463 25503 ts_tablet_manager.cc:1434] T ce98a3f3b3eb4c63bcaa355ec7cd64a4 P 233e7d29aac04505bf4e65f0a39a0c4c: Time spent starting tablet: real 0.020s	user 0.007s	sys 0.000s
I20260709 13:23:34.197669 25350 leader_election.cc:304] T 9ffa42c3b2ff48bc87310e9ab5d218ce P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3e07d64466934da4bb3c02262b1c2e38, 43999d813e564dc688fad116090c95ca; no voters: 9868dce521294d0b816bba7648f4ae5a
I20260709 13:23:34.198318 25274 leader_election.cc:304] T ce98a3f3b3eb4c63bcaa355ec7cd64a4 P 233e7d29aac04505bf4e65f0a39a0c4c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 233e7d29aac04505bf4e65f0a39a0c4c, 3e07d64466934da4bb3c02262b1c2e38; no voters: 43999d813e564dc688fad116090c95ca
I20260709 13:23:34.198875 25568 raft_consensus.cc:2804] T 9ffa42c3b2ff48bc87310e9ab5d218ce P 3e07d64466934da4bb3c02262b1c2e38 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:23:34.199671 25673 raft_consensus.cc:2804] T ce98a3f3b3eb4c63bcaa355ec7cd64a4 P 233e7d29aac04505bf4e65f0a39a0c4c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:23:34.200052 25673 raft_consensus.cc:493] T ce98a3f3b3eb4c63bcaa355ec7cd64a4 P 233e7d29aac04505bf4e65f0a39a0c4c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:23:34.200103 25568 raft_consensus.cc:697] T 9ffa42c3b2ff48bc87310e9ab5d218ce P 3e07d64466934da4bb3c02262b1c2e38 [term 1 LEADER]: Becoming Leader. State: Replica: 3e07d64466934da4bb3c02262b1c2e38, State: Running, Role: LEADER
I20260709 13:23:34.200512 25673 raft_consensus.cc:3060] T ce98a3f3b3eb4c63bcaa355ec7cd64a4 P 233e7d29aac04505bf4e65f0a39a0c4c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:23:34.200882 25503 tablet_bootstrap.cc:492] T 51a11db644124b63bdf5f457c967473d P 233e7d29aac04505bf4e65f0a39a0c4c: Bootstrap starting.
I20260709 13:23:34.201076 25568 consensus_queue.cc:237] T 9ffa42c3b2ff48bc87310e9ab5d218ce P 3e07d64466934da4bb3c02262b1c2e38 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } }
I20260709 13:23:34.195569 25504 raft_consensus.cc:359] T c902bbc7ec0848ba82a04870eab1e7f5 P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } }
I20260709 13:23:34.204252 25504 raft_consensus.cc:385] T c902bbc7ec0848ba82a04870eab1e7f5 P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:23:34.210685 25504 raft_consensus.cc:740] T c902bbc7ec0848ba82a04870eab1e7f5 P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e07d64466934da4bb3c02262b1c2e38, State: Initialized, Role: FOLLOWER
I20260709 13:23:34.211671 25504 consensus_queue.cc:260] T c902bbc7ec0848ba82a04870eab1e7f5 P 3e07d64466934da4bb3c02262b1c2e38 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } }
I20260709 13:23:34.214110 25504 ts_tablet_manager.cc:1434] T c902bbc7ec0848ba82a04870eab1e7f5 P 3e07d64466934da4bb3c02262b1c2e38: Time spent starting tablet: real 0.021s	user 0.003s	sys 0.003s
I20260709 13:23:34.206598 25505 tablet_bootstrap.cc:492] T ce98a3f3b3eb4c63bcaa355ec7cd64a4 P 43999d813e564dc688fad116090c95ca: No bootstrap required, opened a new log
I20260709 13:23:34.217087 25505 ts_tablet_manager.cc:1403] T ce98a3f3b3eb4c63bcaa355ec7cd64a4 P 43999d813e564dc688fad116090c95ca: Time spent bootstrapping tablet: real 0.099s	user 0.011s	sys 0.006s
I20260709 13:23:34.217170 25503 tablet_bootstrap.cc:654] T 51a11db644124b63bdf5f457c967473d P 233e7d29aac04505bf4e65f0a39a0c4c: Neither blocks nor log segments found. Creating new log.
I20260709 13:23:34.219995 25505 raft_consensus.cc:359] T ce98a3f3b3eb4c63bcaa355ec7cd64a4 P 43999d813e564dc688fad116090c95ca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } }
I20260709 13:23:34.219884 25131 catalog_manager.cc:5697] T 9ffa42c3b2ff48bc87310e9ab5d218ce P 3e07d64466934da4bb3c02262b1c2e38 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3e07d64466934da4bb3c02262b1c2e38 (127.14.177.131). New cstate: current_term: 1 leader_uuid: "3e07d64466934da4bb3c02262b1c2e38" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } health_report { overall_health: UNKNOWN } } }
I20260709 13:23:34.220780 25505 raft_consensus.cc:385] T ce98a3f3b3eb4c63bcaa355ec7cd64a4 P 43999d813e564dc688fad116090c95ca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:23:34.221122 25505 raft_consensus.cc:740] T ce98a3f3b3eb4c63bcaa355ec7cd64a4 P 43999d813e564dc688fad116090c95ca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43999d813e564dc688fad116090c95ca, State: Initialized, Role: FOLLOWER
I20260709 13:23:34.221771 25505 consensus_queue.cc:260] T ce98a3f3b3eb4c63bcaa355ec7cd64a4 P 43999d813e564dc688fad116090c95ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } }
I20260709 13:23:34.220043 25504 tablet_bootstrap.cc:492] T 1550ed2d2a764c08adffae679599ba41 P 3e07d64466934da4bb3c02262b1c2e38: Bootstrap starting.
I20260709 13:23:34.223893 25505 ts_tablet_manager.cc:1434] T ce98a3f3b3eb4c63bcaa355ec7cd64a4 P 43999d813e564dc688fad116090c95ca: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.000s
I20260709 13:23:34.224891 25505 tablet_bootstrap.cc:492] T 51a11db644124b63bdf5f457c967473d P 43999d813e564dc688fad116090c95ca: Bootstrap starting.
I20260709 13:23:34.216404 25508 tablet_bootstrap.cc:492] T ad32097cd4dc48b9a69595f1651abd07 P 9868dce521294d0b816bba7648f4ae5a: No bootstrap required, opened a new log
I20260709 13:23:34.227494 25508 ts_tablet_manager.cc:1403] T ad32097cd4dc48b9a69595f1651abd07 P 9868dce521294d0b816bba7648f4ae5a: Time spent bootstrapping tablet: real 0.066s	user 0.018s	sys 0.003s
I20260709 13:23:34.230540 25504 tablet_bootstrap.cc:654] T 1550ed2d2a764c08adffae679599ba41 P 3e07d64466934da4bb3c02262b1c2e38: Neither blocks nor log segments found. Creating new log.
I20260709 13:23:34.231312 25505 tablet_bootstrap.cc:654] T 51a11db644124b63bdf5f457c967473d P 43999d813e564dc688fad116090c95ca: Neither blocks nor log segments found. Creating new log.
I20260709 13:23:34.232479 25508 raft_consensus.cc:359] T ad32097cd4dc48b9a69595f1651abd07 P 9868dce521294d0b816bba7648f4ae5a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } }
I20260709 13:23:34.233968 25508 raft_consensus.cc:385] T ad32097cd4dc48b9a69595f1651abd07 P 9868dce521294d0b816bba7648f4ae5a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:23:34.234272 25508 raft_consensus.cc:740] T ad32097cd4dc48b9a69595f1651abd07 P 9868dce521294d0b816bba7648f4ae5a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9868dce521294d0b816bba7648f4ae5a, State: Initialized, Role: FOLLOWER
I20260709 13:23:34.234965 25508 consensus_queue.cc:260] T ad32097cd4dc48b9a69595f1651abd07 P 9868dce521294d0b816bba7648f4ae5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } }
I20260709 13:23:34.229849 25673 raft_consensus.cc:515] T ce98a3f3b3eb4c63bcaa355ec7cd64a4 P 233e7d29aac04505bf4e65f0a39a0c4c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } }
I20260709 13:23:34.244695 25383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce98a3f3b3eb4c63bcaa355ec7cd64a4" candidate_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e07d64466934da4bb3c02262b1c2e38"
I20260709 13:23:34.245323 25383 raft_consensus.cc:3060] T ce98a3f3b3eb4c63bcaa355ec7cd64a4 P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:23:34.248345 25234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce98a3f3b3eb4c63bcaa355ec7cd64a4" candidate_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43999d813e564dc688fad116090c95ca"
I20260709 13:23:34.249009 25234 raft_consensus.cc:3060] T ce98a3f3b3eb4c63bcaa355ec7cd64a4 P 43999d813e564dc688fad116090c95ca [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:23:34.252552 25383 raft_consensus.cc:2468] T ce98a3f3b3eb4c63bcaa355ec7cd64a4 P 3e07d64466934da4bb3c02262b1c2e38 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 233e7d29aac04505bf4e65f0a39a0c4c in term 1.
I20260709 13:23:34.254599 25274 leader_election.cc:304] T ce98a3f3b3eb4c63bcaa355ec7cd64a4 P 233e7d29aac04505bf4e65f0a39a0c4c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 233e7d29aac04505bf4e65f0a39a0c4c, 3e07d64466934da4bb3c02262b1c2e38; no voters: 
I20260709 13:23:34.255477 25676 raft_consensus.cc:2804] T ce98a3f3b3eb4c63bcaa355ec7cd64a4 P 233e7d29aac04505bf4e65f0a39a0c4c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:23:34.255970 25676 raft_consensus.cc:697] T ce98a3f3b3eb4c63bcaa355ec7cd64a4 P 233e7d29aac04505bf4e65f0a39a0c4c [term 1 LEADER]: Becoming Leader. State: Replica: 233e7d29aac04505bf4e65f0a39a0c4c, State: Running, Role: LEADER
I20260709 13:23:34.256824 25676 consensus_queue.cc:237] T ce98a3f3b3eb4c63bcaa355ec7cd64a4 P 233e7d29aac04505bf4e65f0a39a0c4c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } }
I20260709 13:23:34.262120 25673 leader_election.cc:290] T ce98a3f3b3eb4c63bcaa355ec7cd64a4 P 233e7d29aac04505bf4e65f0a39a0c4c [CANDIDATE]: Term 1 election: Requested vote from peers 43999d813e564dc688fad116090c95ca (127.14.177.129:38155), 3e07d64466934da4bb3c02262b1c2e38 (127.14.177.131:41539)
I20260709 13:23:34.265412 25508 ts_tablet_manager.cc:1434] T ad32097cd4dc48b9a69595f1651abd07 P 9868dce521294d0b816bba7648f4ae5a: Time spent starting tablet: real 0.038s	user 0.006s	sys 0.000s
I20260709 13:23:34.269714 25234 raft_consensus.cc:2468] T ce98a3f3b3eb4c63bcaa355ec7cd64a4 P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 233e7d29aac04505bf4e65f0a39a0c4c in term 1.
I20260709 13:23:34.272557 25503 tablet_bootstrap.cc:492] T 51a11db644124b63bdf5f457c967473d P 233e7d29aac04505bf4e65f0a39a0c4c: No bootstrap required, opened a new log
I20260709 13:23:34.273103 25503 ts_tablet_manager.cc:1403] T 51a11db644124b63bdf5f457c967473d P 233e7d29aac04505bf4e65f0a39a0c4c: Time spent bootstrapping tablet: real 0.072s	user 0.016s	sys 0.015s
I20260709 13:23:34.275863 25503 raft_consensus.cc:359] T 51a11db644124b63bdf5f457c967473d P 233e7d29aac04505bf4e65f0a39a0c4c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } }
I20260709 13:23:34.276533 25503 raft_consensus.cc:385] T 51a11db644124b63bdf5f457c967473d P 233e7d29aac04505bf4e65f0a39a0c4c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:23:34.276820 25503 raft_consensus.cc:740] T 51a11db644124b63bdf5f457c967473d P 233e7d29aac04505bf4e65f0a39a0c4c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 233e7d29aac04505bf4e65f0a39a0c4c, State: Initialized, Role: FOLLOWER
I20260709 13:23:34.277536 25503 consensus_queue.cc:260] T 51a11db644124b63bdf5f457c967473d P 233e7d29aac04505bf4e65f0a39a0c4c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } }
I20260709 13:23:34.279973 25503 ts_tablet_manager.cc:1434] T 51a11db644124b63bdf5f457c967473d P 233e7d29aac04505bf4e65f0a39a0c4c: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 13:23:34.281261 25503 tablet_bootstrap.cc:492] T 0cb89d70b4ce4aa8992fb68480bec8a2 P 233e7d29aac04505bf4e65f0a39a0c4c: Bootstrap starting.
I20260709 13:23:34.281370 25508 tablet_bootstrap.cc:492] T 9ffa42c3b2ff48bc87310e9ab5d218ce P 9868dce521294d0b816bba7648f4ae5a: Bootstrap starting.
I20260709 13:23:34.289624 25508 tablet_bootstrap.cc:654] T 9ffa42c3b2ff48bc87310e9ab5d218ce P 9868dce521294d0b816bba7648f4ae5a: Neither blocks nor log segments found. Creating new log.
I20260709 13:23:34.286626 25134 catalog_manager.cc:5697] T ce98a3f3b3eb4c63bcaa355ec7cd64a4 P 233e7d29aac04505bf4e65f0a39a0c4c reported cstate change: term changed from 0 to 1, leader changed from <none> to 233e7d29aac04505bf4e65f0a39a0c4c (127.14.177.130). New cstate: current_term: 1 leader_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } health_report { overall_health: UNKNOWN } } }
I20260709 13:23:34.303318 25539 consensus_queue.cc:1048] T 280d65245f4f4358b54bef93e18aca92 P 9868dce521294d0b816bba7648f4ae5a [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:23:34.304358 25503 tablet_bootstrap.cc:654] T 0cb89d70b4ce4aa8992fb68480bec8a2 P 233e7d29aac04505bf4e65f0a39a0c4c: Neither blocks nor log segments found. Creating new log.
I20260709 13:23:34.308564 25563 raft_consensus.cc:493] T aad1f7937f854cc1a726223269da5d5a P 9868dce521294d0b816bba7648f4ae5a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:23:34.309105 25563 raft_consensus.cc:515] T aad1f7937f854cc1a726223269da5d5a P 9868dce521294d0b816bba7648f4ae5a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } }
I20260709 13:23:34.314224 25563 leader_election.cc:290] T aad1f7937f854cc1a726223269da5d5a P 9868dce521294d0b816bba7648f4ae5a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 233e7d29aac04505bf4e65f0a39a0c4c (127.14.177.130:42957), 3e07d64466934da4bb3c02262b1c2e38 (127.14.177.131:41539)
I20260709 13:23:34.316238 25309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aad1f7937f854cc1a726223269da5d5a" candidate_uuid: "9868dce521294d0b816bba7648f4ae5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" is_pre_election: true
I20260709 13:23:34.316884 25504 tablet_bootstrap.cc:492] T 1550ed2d2a764c08adffae679599ba41 P 3e07d64466934da4bb3c02262b1c2e38: No bootstrap required, opened a new log
I20260709 13:23:34.316540 25563 consensus_queue.cc:1048] T 280d65245f4f4358b54bef93e18aca92 P 9868dce521294d0b816bba7648f4ae5a [LEADER]: Connected to new peer: Peer: permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:23:34.317404 25504 ts_tablet_manager.cc:1403] T 1550ed2d2a764c08adffae679599ba41 P 3e07d64466934da4bb3c02262b1c2e38: Time spent bootstrapping tablet: real 0.098s	user 0.018s	sys 0.026s
I20260709 13:23:34.317276 25385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aad1f7937f854cc1a726223269da5d5a" candidate_uuid: "9868dce521294d0b816bba7648f4ae5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e07d64466934da4bb3c02262b1c2e38" is_pre_election: true
I20260709 13:23:34.317943 25385 raft_consensus.cc:2468] T aad1f7937f854cc1a726223269da5d5a P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9868dce521294d0b816bba7648f4ae5a in term 0.
I20260709 13:23:34.316937 25309 raft_consensus.cc:2468] T aad1f7937f854cc1a726223269da5d5a P 233e7d29aac04505bf4e65f0a39a0c4c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9868dce521294d0b816bba7648f4ae5a in term 0.
I20260709 13:23:34.319852 25425 leader_election.cc:304] T aad1f7937f854cc1a726223269da5d5a P 9868dce521294d0b816bba7648f4ae5a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 233e7d29aac04505bf4e65f0a39a0c4c, 9868dce521294d0b816bba7648f4ae5a; no voters: 
I20260709 13:23:34.320302 25504 raft_consensus.cc:359] T 1550ed2d2a764c08adffae679599ba41 P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } }
I20260709 13:23:34.320971 25504 raft_consensus.cc:385] T 1550ed2d2a764c08adffae679599ba41 P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:23:34.321244 25504 raft_consensus.cc:740] T 1550ed2d2a764c08adffae679599ba41 P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e07d64466934da4bb3c02262b1c2e38, State: Initialized, Role: FOLLOWER
I20260709 13:23:34.321448 25563 raft_consensus.cc:2804] T aad1f7937f854cc1a726223269da5d5a P 9868dce521294d0b816bba7648f4ae5a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:23:34.322100 25563 raft_consensus.cc:493] T aad1f7937f854cc1a726223269da5d5a P 9868dce521294d0b816bba7648f4ae5a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:23:34.321897 25504 consensus_queue.cc:260] T 1550ed2d2a764c08adffae679599ba41 P 3e07d64466934da4bb3c02262b1c2e38 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } }
I20260709 13:23:34.322811 25563 raft_consensus.cc:3060] T aad1f7937f854cc1a726223269da5d5a P 9868dce521294d0b816bba7648f4ae5a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:23:34.324555 25504 ts_tablet_manager.cc:1434] T 1550ed2d2a764c08adffae679599ba41 P 3e07d64466934da4bb3c02262b1c2e38: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 13:23:34.325542 25504 tablet_bootstrap.cc:492] T f1ac2f4d2fe2424bb470df42199219e2 P 3e07d64466934da4bb3c02262b1c2e38: Bootstrap starting.
I20260709 13:23:34.333295 25504 tablet_bootstrap.cc:654] T f1ac2f4d2fe2424bb470df42199219e2 P 3e07d64466934da4bb3c02262b1c2e38: Neither blocks nor log segments found. Creating new log.
I20260709 13:23:34.340245 25505 tablet_bootstrap.cc:492] T 51a11db644124b63bdf5f457c967473d P 43999d813e564dc688fad116090c95ca: No bootstrap required, opened a new log
I20260709 13:23:34.342831 25505 ts_tablet_manager.cc:1403] T 51a11db644124b63bdf5f457c967473d P 43999d813e564dc688fad116090c95ca: Time spent bootstrapping tablet: real 0.118s	user 0.037s	sys 0.037s
I20260709 13:23:34.345080 25622 raft_consensus.cc:493] T ad32097cd4dc48b9a69595f1651abd07 P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:23:34.346055 25504 tablet_bootstrap.cc:492] T f1ac2f4d2fe2424bb470df42199219e2 P 3e07d64466934da4bb3c02262b1c2e38: No bootstrap required, opened a new log
I20260709 13:23:34.346813 25504 ts_tablet_manager.cc:1403] T f1ac2f4d2fe2424bb470df42199219e2 P 3e07d64466934da4bb3c02262b1c2e38: Time spent bootstrapping tablet: real 0.021s	user 0.014s	sys 0.005s
I20260709 13:23:34.345641 25622 raft_consensus.cc:515] T ad32097cd4dc48b9a69595f1651abd07 P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } }
I20260709 13:23:34.349072 25622 leader_election.cc:290] T ad32097cd4dc48b9a69595f1651abd07 P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 233e7d29aac04505bf4e65f0a39a0c4c (127.14.177.130:42957), 9868dce521294d0b816bba7648f4ae5a (127.14.177.132:33683)
I20260709 13:23:34.350912 25504 raft_consensus.cc:359] T f1ac2f4d2fe2424bb470df42199219e2 P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } }
I20260709 13:23:34.351763 25504 raft_consensus.cc:385] T f1ac2f4d2fe2424bb470df42199219e2 P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:23:34.351576 25309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad32097cd4dc48b9a69595f1651abd07" candidate_uuid: "3e07d64466934da4bb3c02262b1c2e38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" is_pre_election: true
I20260709 13:23:34.352047 25504 raft_consensus.cc:740] T f1ac2f4d2fe2424bb470df42199219e2 P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e07d64466934da4bb3c02262b1c2e38, State: Initialized, Role: FOLLOWER
I20260709 13:23:34.352777 25504 consensus_queue.cc:260] T f1ac2f4d2fe2424bb470df42199219e2 P 3e07d64466934da4bb3c02262b1c2e38 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } }
I20260709 13:23:34.353013 25458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad32097cd4dc48b9a69595f1651abd07" candidate_uuid: "3e07d64466934da4bb3c02262b1c2e38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9868dce521294d0b816bba7648f4ae5a" is_pre_election: true
W20260709 13:23:34.353277 25351 leader_election.cc:343] T ad32097cd4dc48b9a69595f1651abd07 P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 233e7d29aac04505bf4e65f0a39a0c4c (127.14.177.130:42957): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:23:34.353739 25458 raft_consensus.cc:2468] T ad32097cd4dc48b9a69595f1651abd07 P 9868dce521294d0b816bba7648f4ae5a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3e07d64466934da4bb3c02262b1c2e38 in term 0.
I20260709 13:23:34.354092 25573 raft_consensus.cc:493] T c902bbc7ec0848ba82a04870eab1e7f5 P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:23:34.354702 25573 raft_consensus.cc:515] T c902bbc7ec0848ba82a04870eab1e7f5 P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } }
I20260709 13:23:34.355067 25504 ts_tablet_manager.cc:1434] T f1ac2f4d2fe2424bb470df42199219e2 P 3e07d64466934da4bb3c02262b1c2e38: Time spent starting tablet: real 0.008s	user 0.004s	sys 0.000s
I20260709 13:23:34.346140 25505 raft_consensus.cc:359] T 51a11db644124b63bdf5f457c967473d P 43999d813e564dc688fad116090c95ca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } }
I20260709 13:23:34.356146 25505 raft_consensus.cc:385] T 51a11db644124b63bdf5f457c967473d P 43999d813e564dc688fad116090c95ca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:23:34.356305 25504 tablet_bootstrap.cc:492] T 89a26dd3ac414f6081b98fa32c2f346f P 3e07d64466934da4bb3c02262b1c2e38: Bootstrap starting.
I20260709 13:23:34.356441 25505 raft_consensus.cc:740] T 51a11db644124b63bdf5f457c967473d P 43999d813e564dc688fad116090c95ca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43999d813e564dc688fad116090c95ca, State: Initialized, Role: FOLLOWER
I20260709 13:23:34.357522 25573 leader_election.cc:290] T c902bbc7ec0848ba82a04870eab1e7f5 P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9868dce521294d0b816bba7648f4ae5a (127.14.177.132:33683), 43999d813e564dc688fad116090c95ca (127.14.177.129:38155)
I20260709 13:23:34.357352 25505 consensus_queue.cc:260] T 51a11db644124b63bdf5f457c967473d P 43999d813e564dc688fad116090c95ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } }
I20260709 13:23:34.355270 25348 leader_election.cc:304] T ad32097cd4dc48b9a69595f1651abd07 P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3e07d64466934da4bb3c02262b1c2e38, 9868dce521294d0b816bba7648f4ae5a; no voters: 233e7d29aac04505bf4e65f0a39a0c4c
I20260709 13:23:34.360075 25235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c902bbc7ec0848ba82a04870eab1e7f5" candidate_uuid: "3e07d64466934da4bb3c02262b1c2e38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43999d813e564dc688fad116090c95ca" is_pre_election: true
I20260709 13:23:34.360939 25458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c902bbc7ec0848ba82a04870eab1e7f5" candidate_uuid: "3e07d64466934da4bb3c02262b1c2e38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9868dce521294d0b816bba7648f4ae5a" is_pre_election: true
W20260709 13:23:34.361711 25350 leader_election.cc:343] T c902bbc7ec0848ba82a04870eab1e7f5 P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 43999d813e564dc688fad116090c95ca (127.14.177.129:38155): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:23:34.362414 25348 leader_election.cc:343] T c902bbc7ec0848ba82a04870eab1e7f5 P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9868dce521294d0b816bba7648f4ae5a (127.14.177.132:33683): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:23:34.363006 25504 tablet_bootstrap.cc:654] T 89a26dd3ac414f6081b98fa32c2f346f P 3e07d64466934da4bb3c02262b1c2e38: Neither blocks nor log segments found. Creating new log.
I20260709 13:23:34.362834 25348 leader_election.cc:304] T c902bbc7ec0848ba82a04870eab1e7f5 P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3e07d64466934da4bb3c02262b1c2e38; no voters: 43999d813e564dc688fad116090c95ca, 9868dce521294d0b816bba7648f4ae5a
I20260709 13:23:34.363408 25573 raft_consensus.cc:2804] T ad32097cd4dc48b9a69595f1651abd07 P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:23:34.363631 25686 raft_consensus.cc:2749] T c902bbc7ec0848ba82a04870eab1e7f5 P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:23:34.363821 25573 raft_consensus.cc:493] T ad32097cd4dc48b9a69595f1651abd07 P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:23:34.364221 25573 raft_consensus.cc:3060] T ad32097cd4dc48b9a69595f1651abd07 P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:23:34.376273 25504 tablet_bootstrap.cc:492] T 89a26dd3ac414f6081b98fa32c2f346f P 3e07d64466934da4bb3c02262b1c2e38: No bootstrap required, opened a new log
I20260709 13:23:34.382230 25504 ts_tablet_manager.cc:1403] T 89a26dd3ac414f6081b98fa32c2f346f P 3e07d64466934da4bb3c02262b1c2e38: Time spent bootstrapping tablet: real 0.026s	user 0.014s	sys 0.004s
I20260709 13:23:34.381961 25573 raft_consensus.cc:515] T ad32097cd4dc48b9a69595f1651abd07 P 3e07d64466934da4bb3c02262b1c2e38 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } }
I20260709 13:23:34.372092 25505 ts_tablet_manager.cc:1434] T 51a11db644124b63bdf5f457c967473d P 43999d813e564dc688fad116090c95ca: Time spent starting tablet: real 0.029s	user 0.007s	sys 0.000s
I20260709 13:23:34.385706 25505 tablet_bootstrap.cc:492] T 0cb89d70b4ce4aa8992fb68480bec8a2 P 43999d813e564dc688fad116090c95ca: Bootstrap starting.
I20260709 13:23:34.386018 25307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad32097cd4dc48b9a69595f1651abd07" candidate_uuid: "3e07d64466934da4bb3c02262b1c2e38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "233e7d29aac04505bf4e65f0a39a0c4c"
W20260709 13:23:34.387524 25351 leader_election.cc:343] T ad32097cd4dc48b9a69595f1651abd07 P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 233e7d29aac04505bf4e65f0a39a0c4c (127.14.177.130:42957): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:23:34.390456 25456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad32097cd4dc48b9a69595f1651abd07" candidate_uuid: "3e07d64466934da4bb3c02262b1c2e38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9868dce521294d0b816bba7648f4ae5a"
I20260709 13:23:34.391146 25456 raft_consensus.cc:3060] T ad32097cd4dc48b9a69595f1651abd07 P 9868dce521294d0b816bba7648f4ae5a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:23:34.394368 25505 tablet_bootstrap.cc:654] T 0cb89d70b4ce4aa8992fb68480bec8a2 P 43999d813e564dc688fad116090c95ca: Neither blocks nor log segments found. Creating new log.
I20260709 13:23:34.385324 25504 raft_consensus.cc:359] T 89a26dd3ac414f6081b98fa32c2f346f P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } }
I20260709 13:23:34.395363 25504 raft_consensus.cc:385] T 89a26dd3ac414f6081b98fa32c2f346f P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:23:34.395730 25504 raft_consensus.cc:740] T 89a26dd3ac414f6081b98fa32c2f346f P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e07d64466934da4bb3c02262b1c2e38, State: Initialized, Role: FOLLOWER
I20260709 13:23:34.396694 25504 consensus_queue.cc:260] T 89a26dd3ac414f6081b98fa32c2f346f P 3e07d64466934da4bb3c02262b1c2e38 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } }
I20260709 13:23:34.402956 25504 ts_tablet_manager.cc:1434] T 89a26dd3ac414f6081b98fa32c2f346f P 3e07d64466934da4bb3c02262b1c2e38: Time spent starting tablet: real 0.020s	user 0.006s	sys 0.002s
I20260709 13:23:34.403519 25539 raft_consensus.cc:493] T ba8a000f5abe4158965449995af84b18 P 9868dce521294d0b816bba7648f4ae5a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:23:34.404446 25504 tablet_bootstrap.cc:492] T 0cb89d70b4ce4aa8992fb68480bec8a2 P 3e07d64466934da4bb3c02262b1c2e38: Bootstrap starting.
I20260709 13:23:34.411895 25539 raft_consensus.cc:515] T ba8a000f5abe4158965449995af84b18 P 9868dce521294d0b816bba7648f4ae5a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } }
I20260709 13:23:34.421413 25504 tablet_bootstrap.cc:654] T 0cb89d70b4ce4aa8992fb68480bec8a2 P 3e07d64466934da4bb3c02262b1c2e38: Neither blocks nor log segments found. Creating new log.
I20260709 13:23:34.429677 25563 raft_consensus.cc:515] T aad1f7937f854cc1a726223269da5d5a P 9868dce521294d0b816bba7648f4ae5a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } }
I20260709 13:23:34.444470 25673 raft_consensus.cc:493] T 51a11db644124b63bdf5f457c967473d P 233e7d29aac04505bf4e65f0a39a0c4c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:23:34.455364 25673 raft_consensus.cc:515] T 51a11db644124b63bdf5f457c967473d P 233e7d29aac04505bf4e65f0a39a0c4c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } }
I20260709 13:23:34.455996 25457 raft_consensus.cc:3060] T ba8a000f5abe4158965449995af84b18 P 9868dce521294d0b816bba7648f4ae5a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:23:34.457667 25673 leader_election.cc:290] T 51a11db644124b63bdf5f457c967473d P 233e7d29aac04505bf4e65f0a39a0c4c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9868dce521294d0b816bba7648f4ae5a (127.14.177.132:33683), 43999d813e564dc688fad116090c95ca (127.14.177.129:38155)
I20260709 13:23:34.459105 25673 consensus_queue.cc:1048] T 5c4ab1978a47481c8576d466bacf49d2 P 233e7d29aac04505bf4e65f0a39a0c4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.007s
I20260709 13:23:34.469769 25563 leader_election.cc:290] T aad1f7937f854cc1a726223269da5d5a P 9868dce521294d0b816bba7648f4ae5a [CANDIDATE]: Term 1 election: Requested vote from peers 233e7d29aac04505bf4e65f0a39a0c4c (127.14.177.130:42957), 3e07d64466934da4bb3c02262b1c2e38 (127.14.177.131:41539)
I20260709 13:23:34.449657 25508 tablet_bootstrap.cc:492] T 9ffa42c3b2ff48bc87310e9ab5d218ce P 9868dce521294d0b816bba7648f4ae5a: No bootstrap required, opened a new log
I20260709 13:23:34.470930 25508 ts_tablet_manager.cc:1403] T 9ffa42c3b2ff48bc87310e9ab5d218ce P 9868dce521294d0b816bba7648f4ae5a: Time spent bootstrapping tablet: real 0.190s	user 0.017s	sys 0.003s
I20260709 13:23:34.471228 25307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aad1f7937f854cc1a726223269da5d5a" candidate_uuid: "9868dce521294d0b816bba7648f4ae5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "233e7d29aac04505bf4e65f0a39a0c4c"
I20260709 13:23:34.471925 25307 raft_consensus.cc:3060] T aad1f7937f854cc1a726223269da5d5a P 233e7d29aac04505bf4e65f0a39a0c4c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:23:34.473336 25503 tablet_bootstrap.cc:492] T 0cb89d70b4ce4aa8992fb68480bec8a2 P 233e7d29aac04505bf4e65f0a39a0c4c: No bootstrap required, opened a new log
I20260709 13:23:34.474362 25508 raft_consensus.cc:359] T 9ffa42c3b2ff48bc87310e9ab5d218ce P 9868dce521294d0b816bba7648f4ae5a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } }
I20260709 13:23:34.475479 25503 ts_tablet_manager.cc:1403] T 0cb89d70b4ce4aa8992fb68480bec8a2 P 233e7d29aac04505bf4e65f0a39a0c4c: Time spent bootstrapping tablet: real 0.195s	user 0.041s	sys 0.040s
I20260709 13:23:34.482337 25235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51a11db644124b63bdf5f457c967473d" candidate_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43999d813e564dc688fad116090c95ca" is_pre_election: true
I20260709 13:23:34.485551 25676 consensus_queue.cc:1048] T ba8a000f5abe4158965449995af84b18 P 233e7d29aac04505bf4e65f0a39a0c4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:23:34.505414 25235 raft_consensus.cc:2468] T 51a11db644124b63bdf5f457c967473d P 43999d813e564dc688fad116090c95ca [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 233e7d29aac04505bf4e65f0a39a0c4c in term 0.
I20260709 13:23:34.508437 25276 leader_election.cc:304] T 51a11db644124b63bdf5f457c967473d P 233e7d29aac04505bf4e65f0a39a0c4c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 233e7d29aac04505bf4e65f0a39a0c4c, 43999d813e564dc688fad116090c95ca; no voters: 
I20260709 13:23:34.511145 25679 raft_consensus.cc:2804] T 51a11db644124b63bdf5f457c967473d P 233e7d29aac04505bf4e65f0a39a0c4c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:23:34.496237 25504 tablet_bootstrap.cc:492] T 0cb89d70b4ce4aa8992fb68480bec8a2 P 3e07d64466934da4bb3c02262b1c2e38: No bootstrap required, opened a new log
I20260709 13:23:34.520222 25504 ts_tablet_manager.cc:1403] T 0cb89d70b4ce4aa8992fb68480bec8a2 P 3e07d64466934da4bb3c02262b1c2e38: Time spent bootstrapping tablet: real 0.116s	user 0.027s	sys 0.016s
I20260709 13:23:34.521349 25456 raft_consensus.cc:2468] T ad32097cd4dc48b9a69595f1651abd07 P 9868dce521294d0b816bba7648f4ae5a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3e07d64466934da4bb3c02262b1c2e38 in term 1.
I20260709 13:23:34.522861 25348 leader_election.cc:304] T ad32097cd4dc48b9a69595f1651abd07 P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3e07d64466934da4bb3c02262b1c2e38, 9868dce521294d0b816bba7648f4ae5a; no voters: 233e7d29aac04505bf4e65f0a39a0c4c
I20260709 13:23:34.523905 25568 raft_consensus.cc:2804] T ad32097cd4dc48b9a69595f1651abd07 P 3e07d64466934da4bb3c02262b1c2e38 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:23:34.524435 25568 raft_consensus.cc:697] T ad32097cd4dc48b9a69595f1651abd07 P 3e07d64466934da4bb3c02262b1c2e38 [term 1 LEADER]: Becoming Leader. State: Replica: 3e07d64466934da4bb3c02262b1c2e38, State: Running, Role: LEADER
I20260709 13:23:34.524268 25504 raft_consensus.cc:359] T 0cb89d70b4ce4aa8992fb68480bec8a2 P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } }
I20260709 13:23:34.525384 25504 raft_consensus.cc:385] T 0cb89d70b4ce4aa8992fb68480bec8a2 P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:23:34.525935 25504 raft_consensus.cc:740] T 0cb89d70b4ce4aa8992fb68480bec8a2 P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e07d64466934da4bb3c02262b1c2e38, State: Initialized, Role: FOLLOWER
I20260709 13:23:34.525480 25568 consensus_queue.cc:237] T ad32097cd4dc48b9a69595f1651abd07 P 3e07d64466934da4bb3c02262b1c2e38 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } }
I20260709 13:23:34.531361 25679 raft_consensus.cc:493] T 51a11db644124b63bdf5f457c967473d P 233e7d29aac04505bf4e65f0a39a0c4c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:23:34.390455 25573 leader_election.cc:290] T ad32097cd4dc48b9a69595f1651abd07 P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 1 election: Requested vote from peers 233e7d29aac04505bf4e65f0a39a0c4c (127.14.177.130:42957), 9868dce521294d0b816bba7648f4ae5a (127.14.177.132:33683)
I20260709 13:23:34.531927 25679 raft_consensus.cc:3060] T 51a11db644124b63bdf5f457c967473d P 233e7d29aac04505bf4e65f0a39a0c4c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:23:34.525375 25307 raft_consensus.cc:2468] T aad1f7937f854cc1a726223269da5d5a P 233e7d29aac04505bf4e65f0a39a0c4c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9868dce521294d0b816bba7648f4ae5a in term 1.
I20260709 13:23:34.535710 25676 consensus_queue.cc:1048] T 5c4ab1978a47481c8576d466bacf49d2 P 233e7d29aac04505bf4e65f0a39a0c4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:23:34.537904 25425 leader_election.cc:304] T aad1f7937f854cc1a726223269da5d5a P 9868dce521294d0b816bba7648f4ae5a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 233e7d29aac04505bf4e65f0a39a0c4c, 9868dce521294d0b816bba7648f4ae5a; no voters: 
I20260709 13:23:34.489338 25458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51a11db644124b63bdf5f457c967473d" candidate_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9868dce521294d0b816bba7648f4ae5a" is_pre_election: true
I20260709 13:23:34.504222 25508 raft_consensus.cc:385] T 9ffa42c3b2ff48bc87310e9ab5d218ce P 9868dce521294d0b816bba7648f4ae5a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:23:34.543922 25508 raft_consensus.cc:740] T 9ffa42c3b2ff48bc87310e9ab5d218ce P 9868dce521294d0b816bba7648f4ae5a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9868dce521294d0b816bba7648f4ae5a, State: Initialized, Role: FOLLOWER
I20260709 13:23:34.487213 25385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aad1f7937f854cc1a726223269da5d5a" candidate_uuid: "9868dce521294d0b816bba7648f4ae5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e07d64466934da4bb3c02262b1c2e38"
I20260709 13:23:34.546725 25385 raft_consensus.cc:3060] T aad1f7937f854cc1a726223269da5d5a P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:23:34.547652 25651 raft_consensus.cc:2804] T aad1f7937f854cc1a726223269da5d5a P 9868dce521294d0b816bba7648f4ae5a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:23:34.548333 25651 raft_consensus.cc:697] T aad1f7937f854cc1a726223269da5d5a P 9868dce521294d0b816bba7648f4ae5a [term 1 LEADER]: Becoming Leader. State: Replica: 9868dce521294d0b816bba7648f4ae5a, State: Running, Role: LEADER
I20260709 13:23:34.549316 25651 consensus_queue.cc:237] T aad1f7937f854cc1a726223269da5d5a P 9868dce521294d0b816bba7648f4ae5a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } }
I20260709 13:23:34.550657 25458 raft_consensus.cc:2468] T 51a11db644124b63bdf5f457c967473d P 9868dce521294d0b816bba7648f4ae5a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 233e7d29aac04505bf4e65f0a39a0c4c in term 0.
I20260709 13:23:34.551669 25508 consensus_queue.cc:260] T 9ffa42c3b2ff48bc87310e9ab5d218ce P 9868dce521294d0b816bba7648f4ae5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } }
I20260709 13:23:34.526953 25504 consensus_queue.cc:260] T 0cb89d70b4ce4aa8992fb68480bec8a2 P 3e07d64466934da4bb3c02262b1c2e38 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } }
I20260709 13:23:34.554076 25508 ts_tablet_manager.cc:1434] T 9ffa42c3b2ff48bc87310e9ab5d218ce P 9868dce521294d0b816bba7648f4ae5a: Time spent starting tablet: real 0.083s	user 0.003s	sys 0.002s
I20260709 13:23:34.562100 25385 raft_consensus.cc:2468] T aad1f7937f854cc1a726223269da5d5a P 3e07d64466934da4bb3c02262b1c2e38 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9868dce521294d0b816bba7648f4ae5a in term 1.
I20260709 13:23:34.563256 25508 tablet_bootstrap.cc:492] T 0c4f6b7401db47d59a4f34ed0e8b5350 P 9868dce521294d0b816bba7648f4ae5a: Bootstrap starting.
I20260709 13:23:34.567466 25504 ts_tablet_manager.cc:1434] T 0cb89d70b4ce4aa8992fb68480bec8a2 P 3e07d64466934da4bb3c02262b1c2e38: Time spent starting tablet: real 0.035s	user 0.007s	sys 0.000s
I20260709 13:23:34.568629 25504 tablet_bootstrap.cc:492] T 13bb3a23315a41c7a40e10b954b15c8e P 3e07d64466934da4bb3c02262b1c2e38: Bootstrap starting.
I20260709 13:23:34.512033 25503 raft_consensus.cc:359] T 0cb89d70b4ce4aa8992fb68480bec8a2 P 233e7d29aac04505bf4e65f0a39a0c4c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } }
I20260709 13:23:34.512902 25505 tablet_bootstrap.cc:492] T 0cb89d70b4ce4aa8992fb68480bec8a2 P 43999d813e564dc688fad116090c95ca: No bootstrap required, opened a new log
I20260709 13:23:34.514801 25309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba8a000f5abe4158965449995af84b18" candidate_uuid: "9868dce521294d0b816bba7648f4ae5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" is_pre_election: true
I20260709 13:23:34.571448 25505 ts_tablet_manager.cc:1403] T 0cb89d70b4ce4aa8992fb68480bec8a2 P 43999d813e564dc688fad116090c95ca: Time spent bootstrapping tablet: real 0.186s	user 0.010s	sys 0.047s
I20260709 13:23:34.516747 25622 raft_consensus.cc:493] T 1550ed2d2a764c08adffae679599ba41 P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:23:34.516402 25539 leader_election.cc:290] T ba8a000f5abe4158965449995af84b18 P 9868dce521294d0b816bba7648f4ae5a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 233e7d29aac04505bf4e65f0a39a0c4c (127.14.177.130:42957), 3e07d64466934da4bb3c02262b1c2e38 (127.14.177.131:41539)
I20260709 13:23:34.570655 25503 raft_consensus.cc:385] T 0cb89d70b4ce4aa8992fb68480bec8a2 P 233e7d29aac04505bf4e65f0a39a0c4c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:23:34.577664 25503 raft_consensus.cc:740] T 0cb89d70b4ce4aa8992fb68480bec8a2 P 233e7d29aac04505bf4e65f0a39a0c4c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 233e7d29aac04505bf4e65f0a39a0c4c, State: Initialized, Role: FOLLOWER
I20260709 13:23:34.578889 25503 consensus_queue.cc:260] T 0cb89d70b4ce4aa8992fb68480bec8a2 P 233e7d29aac04505bf4e65f0a39a0c4c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } }
I20260709 13:23:34.582516 25503 ts_tablet_manager.cc:1434] T 0cb89d70b4ce4aa8992fb68480bec8a2 P 233e7d29aac04505bf4e65f0a39a0c4c: Time spent starting tablet: real 0.074s	user 0.006s	sys 0.001s
I20260709 13:23:34.584479 25503 tablet_bootstrap.cc:492] T 0c4f6b7401db47d59a4f34ed0e8b5350 P 233e7d29aac04505bf4e65f0a39a0c4c: Bootstrap starting.
I20260709 13:23:34.579344 25622 raft_consensus.cc:515] T 1550ed2d2a764c08adffae679599ba41 P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } }
I20260709 13:23:34.596755 25679 raft_consensus.cc:515] T 51a11db644124b63bdf5f457c967473d P 233e7d29aac04505bf4e65f0a39a0c4c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } }
I20260709 13:23:34.600519 25622 leader_election.cc:290] T 1550ed2d2a764c08adffae679599ba41 P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9868dce521294d0b816bba7648f4ae5a (127.14.177.132:33683), 43999d813e564dc688fad116090c95ca (127.14.177.129:38155)
I20260709 13:23:34.600845 25679 leader_election.cc:290] T 51a11db644124b63bdf5f457c967473d P 233e7d29aac04505bf4e65f0a39a0c4c [CANDIDATE]: Term 1 election: Requested vote from peers 9868dce521294d0b816bba7648f4ae5a (127.14.177.132:33683), 43999d813e564dc688fad116090c95ca (127.14.177.129:38155)
I20260709 13:23:34.591548 25568 raft_consensus.cc:493] T 0cb89d70b4ce4aa8992fb68480bec8a2 P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:23:34.601679 25568 raft_consensus.cc:515] T 0cb89d70b4ce4aa8992fb68480bec8a2 P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } }
I20260709 13:23:34.602301 25456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1550ed2d2a764c08adffae679599ba41" candidate_uuid: "3e07d64466934da4bb3c02262b1c2e38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9868dce521294d0b816bba7648f4ae5a" is_pre_election: true
I20260709 13:23:34.603922 25385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba8a000f5abe4158965449995af84b18" candidate_uuid: "9868dce521294d0b816bba7648f4ae5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e07d64466934da4bb3c02262b1c2e38" is_pre_election: true
I20260709 13:23:34.604759 25385 raft_consensus.cc:2393] T ba8a000f5abe4158965449995af84b18 P 3e07d64466934da4bb3c02262b1c2e38 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9868dce521294d0b816bba7648f4ae5a in current term 1: Already voted for candidate 233e7d29aac04505bf4e65f0a39a0c4c in this term.
I20260709 13:23:34.605513 25508 tablet_bootstrap.cc:654] T 0c4f6b7401db47d59a4f34ed0e8b5350 P 9868dce521294d0b816bba7648f4ae5a: Neither blocks nor log segments found. Creating new log.
W20260709 13:23:34.605782 25348 leader_election.cc:343] T 1550ed2d2a764c08adffae679599ba41 P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9868dce521294d0b816bba7648f4ae5a (127.14.177.132:33683): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:23:34.604780 25234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1550ed2d2a764c08adffae679599ba41" candidate_uuid: "3e07d64466934da4bb3c02262b1c2e38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43999d813e564dc688fad116090c95ca" is_pre_election: true
I20260709 13:23:34.605576 25233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51a11db644124b63bdf5f457c967473d" candidate_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43999d813e564dc688fad116090c95ca"
I20260709 13:23:34.609944 25233 raft_consensus.cc:3060] T 51a11db644124b63bdf5f457c967473d P 43999d813e564dc688fad116090c95ca [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:23:34.610764 25350 leader_election.cc:343] T 1550ed2d2a764c08adffae679599ba41 P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 43999d813e564dc688fad116090c95ca (127.14.177.129:38155): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:23:34.611318 25350 leader_election.cc:304] T 1550ed2d2a764c08adffae679599ba41 P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3e07d64466934da4bb3c02262b1c2e38; no voters: 43999d813e564dc688fad116090c95ca, 9868dce521294d0b816bba7648f4ae5a
I20260709 13:23:34.611884 25505 raft_consensus.cc:359] T 0cb89d70b4ce4aa8992fb68480bec8a2 P 43999d813e564dc688fad116090c95ca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } }
I20260709 13:23:34.613871 25622 raft_consensus.cc:2749] T 1550ed2d2a764c08adffae679599ba41 P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:23:34.617775 25233 raft_consensus.cc:2468] T 51a11db644124b63bdf5f457c967473d P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 233e7d29aac04505bf4e65f0a39a0c4c in term 1.
I20260709 13:23:34.619148 25276 leader_election.cc:304] T 51a11db644124b63bdf5f457c967473d P 233e7d29aac04505bf4e65f0a39a0c4c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 233e7d29aac04505bf4e65f0a39a0c4c, 43999d813e564dc688fad116090c95ca; no voters: 
I20260709 13:23:34.619656 25505 raft_consensus.cc:385] T 0cb89d70b4ce4aa8992fb68480bec8a2 P 43999d813e564dc688fad116090c95ca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:23:34.620167 25673 raft_consensus.cc:2804] T 51a11db644124b63bdf5f457c967473d P 233e7d29aac04505bf4e65f0a39a0c4c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:23:34.620339 25505 raft_consensus.cc:740] T 0cb89d70b4ce4aa8992fb68480bec8a2 P 43999d813e564dc688fad116090c95ca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43999d813e564dc688fad116090c95ca, State: Initialized, Role: FOLLOWER
I20260709 13:23:34.620887 25673 raft_consensus.cc:697] T 51a11db644124b63bdf5f457c967473d P 233e7d29aac04505bf4e65f0a39a0c4c [term 1 LEADER]: Becoming Leader. State: Replica: 233e7d29aac04505bf4e65f0a39a0c4c, State: Running, Role: LEADER
I20260709 13:23:34.622295 25673 consensus_queue.cc:237] T 51a11db644124b63bdf5f457c967473d P 233e7d29aac04505bf4e65f0a39a0c4c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } }
I20260709 13:23:34.623313 25679 consensus_queue.cc:1048] T ba8a000f5abe4158965449995af84b18 P 233e7d29aac04505bf4e65f0a39a0c4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:23:34.625770 25456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51a11db644124b63bdf5f457c967473d" candidate_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9868dce521294d0b816bba7648f4ae5a"
I20260709 13:23:34.626435 25456 raft_consensus.cc:3060] T 51a11db644124b63bdf5f457c967473d P 9868dce521294d0b816bba7648f4ae5a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:23:34.607550 25504 tablet_bootstrap.cc:654] T 13bb3a23315a41c7a40e10b954b15c8e P 3e07d64466934da4bb3c02262b1c2e38: Neither blocks nor log segments found. Creating new log.
I20260709 13:23:34.621973 25505 consensus_queue.cc:260] T 0cb89d70b4ce4aa8992fb68480bec8a2 P 43999d813e564dc688fad116090c95ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } }
I20260709 13:23:34.630529 25568 leader_election.cc:290] T 0cb89d70b4ce4aa8992fb68480bec8a2 P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 43999d813e564dc688fad116090c95ca (127.14.177.129:38155), 233e7d29aac04505bf4e65f0a39a0c4c (127.14.177.130:42957)
I20260709 13:23:34.639541 25307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cb89d70b4ce4aa8992fb68480bec8a2" candidate_uuid: "3e07d64466934da4bb3c02262b1c2e38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" is_pre_election: true
I20260709 13:23:34.641615 25505 ts_tablet_manager.cc:1434] T 0cb89d70b4ce4aa8992fb68480bec8a2 P 43999d813e564dc688fad116090c95ca: Time spent starting tablet: real 0.070s	user 0.005s	sys 0.004s
I20260709 13:23:34.647881 25505 tablet_bootstrap.cc:492] T c902bbc7ec0848ba82a04870eab1e7f5 P 43999d813e564dc688fad116090c95ca: Bootstrap starting.
I20260709 13:23:34.650708 25307 raft_consensus.cc:2468] T 0cb89d70b4ce4aa8992fb68480bec8a2 P 233e7d29aac04505bf4e65f0a39a0c4c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3e07d64466934da4bb3c02262b1c2e38 in term 0.
I20260709 13:23:34.652079 25351 leader_election.cc:304] T 0cb89d70b4ce4aa8992fb68480bec8a2 P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 233e7d29aac04505bf4e65f0a39a0c4c, 3e07d64466934da4bb3c02262b1c2e38; no voters: 
I20260709 13:23:34.653321 25568 raft_consensus.cc:2804] T 0cb89d70b4ce4aa8992fb68480bec8a2 P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:23:34.662667 25233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cb89d70b4ce4aa8992fb68480bec8a2" candidate_uuid: "3e07d64466934da4bb3c02262b1c2e38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43999d813e564dc688fad116090c95ca" is_pre_election: true
I20260709 13:23:34.663767 25233 raft_consensus.cc:2468] T 0cb89d70b4ce4aa8992fb68480bec8a2 P 43999d813e564dc688fad116090c95ca [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3e07d64466934da4bb3c02262b1c2e38 in term 0.
I20260709 13:23:34.664991 25503 tablet_bootstrap.cc:654] T 0c4f6b7401db47d59a4f34ed0e8b5350 P 233e7d29aac04505bf4e65f0a39a0c4c: Neither blocks nor log segments found. Creating new log.
I20260709 13:23:34.654495 25539 raft_consensus.cc:493] T 9ffa42c3b2ff48bc87310e9ab5d218ce P 9868dce521294d0b816bba7648f4ae5a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:23:34.658368 25505 tablet_bootstrap.cc:654] T c902bbc7ec0848ba82a04870eab1e7f5 P 43999d813e564dc688fad116090c95ca: Neither blocks nor log segments found. Creating new log.
I20260709 13:23:34.667513 25539 raft_consensus.cc:515] T 9ffa42c3b2ff48bc87310e9ab5d218ce P 9868dce521294d0b816bba7648f4ae5a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } }
I20260709 13:23:34.659183 25456 raft_consensus.cc:2468] T 51a11db644124b63bdf5f457c967473d P 9868dce521294d0b816bba7648f4ae5a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 233e7d29aac04505bf4e65f0a39a0c4c in term 1.
I20260709 13:23:34.645955 25422 leader_election.cc:304] T ba8a000f5abe4158965449995af84b18 P 9868dce521294d0b816bba7648f4ae5a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9868dce521294d0b816bba7648f4ae5a; no voters: 233e7d29aac04505bf4e65f0a39a0c4c, 3e07d64466934da4bb3c02262b1c2e38
I20260709 13:23:34.663013 25568 raft_consensus.cc:493] T 0cb89d70b4ce4aa8992fb68480bec8a2 P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:23:34.670636 25539 leader_election.cc:290] T 9ffa42c3b2ff48bc87310e9ab5d218ce P 9868dce521294d0b816bba7648f4ae5a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3e07d64466934da4bb3c02262b1c2e38 (127.14.177.131:41539), 43999d813e564dc688fad116090c95ca (127.14.177.129:38155)
I20260709 13:23:34.673318 25568 raft_consensus.cc:3060] T 0cb89d70b4ce4aa8992fb68480bec8a2 P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:23:34.675864 25644 raft_consensus.cc:2749] T ba8a000f5abe4158965449995af84b18 P 9868dce521294d0b816bba7648f4ae5a [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:23:34.672458 25457 raft_consensus.cc:3060] T 9ffa42c3b2ff48bc87310e9ab5d218ce P 9868dce521294d0b816bba7648f4ae5a [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:23:34.684870 25422 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41539, user_credentials={real_user=slave}} blocked reactor thread for 52310.2us
I20260709 13:23:34.686101 25508 tablet_bootstrap.cc:492] T 0c4f6b7401db47d59a4f34ed0e8b5350 P 9868dce521294d0b816bba7648f4ae5a: No bootstrap required, opened a new log
I20260709 13:23:34.688364 25508 ts_tablet_manager.cc:1403] T 0c4f6b7401db47d59a4f34ed0e8b5350 P 9868dce521294d0b816bba7648f4ae5a: Time spent bootstrapping tablet: real 0.133s	user 0.018s	sys 0.013s
I20260709 13:23:34.688333 25644 consensus_queue.cc:1048] T 9e5d70664720424d951f269d9d7f4346 P 9868dce521294d0b816bba7648f4ae5a [LEADER]: Connected to new peer: Peer: permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.003s
I20260709 13:23:34.691934 25508 raft_consensus.cc:359] T 0c4f6b7401db47d59a4f34ed0e8b5350 P 9868dce521294d0b816bba7648f4ae5a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } }
I20260709 13:23:34.692631 25508 raft_consensus.cc:385] T 0c4f6b7401db47d59a4f34ed0e8b5350 P 9868dce521294d0b816bba7648f4ae5a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:23:34.692914 25508 raft_consensus.cc:740] T 0c4f6b7401db47d59a4f34ed0e8b5350 P 9868dce521294d0b816bba7648f4ae5a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9868dce521294d0b816bba7648f4ae5a, State: Initialized, Role: FOLLOWER
I20260709 13:23:34.693164 25622 consensus_queue.cc:1048] T 9ffa42c3b2ff48bc87310e9ab5d218ce P 3e07d64466934da4bb3c02262b1c2e38 [LEADER]: Connected to new peer: Peer: permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:23:34.693359 25383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ffa42c3b2ff48bc87310e9ab5d218ce" candidate_uuid: "9868dce521294d0b816bba7648f4ae5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e07d64466934da4bb3c02262b1c2e38" is_pre_election: true
I20260709 13:23:34.693593 25508 consensus_queue.cc:260] T 0c4f6b7401db47d59a4f34ed0e8b5350 P 9868dce521294d0b816bba7648f4ae5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } }
I20260709 13:23:34.693934 25573 raft_consensus.cc:493] T f1ac2f4d2fe2424bb470df42199219e2 P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:23:34.700800 25622 raft_consensus.cc:493] T 89a26dd3ac414f6081b98fa32c2f346f P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:23:34.705765 25622 raft_consensus.cc:515] T 89a26dd3ac414f6081b98fa32c2f346f P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } }
I20260709 13:23:34.708148 25622 leader_election.cc:290] T 89a26dd3ac414f6081b98fa32c2f346f P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 233e7d29aac04505bf4e65f0a39a0c4c (127.14.177.130:42957), 9868dce521294d0b816bba7648f4ae5a (127.14.177.132:33683)
I20260709 13:23:34.710026 25573 raft_consensus.cc:515] T f1ac2f4d2fe2424bb470df42199219e2 P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } }
I20260709 13:23:34.710397 25134 catalog_manager.cc:5697] T 51a11db644124b63bdf5f457c967473d P 233e7d29aac04505bf4e65f0a39a0c4c reported cstate change: term changed from 0 to 1, leader changed from <none> to 233e7d29aac04505bf4e65f0a39a0c4c (127.14.177.130). New cstate: current_term: 1 leader_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } health_report { overall_health: HEALTHY } } }
I20260709 13:23:34.712332 25459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89a26dd3ac414f6081b98fa32c2f346f" candidate_uuid: "3e07d64466934da4bb3c02262b1c2e38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9868dce521294d0b816bba7648f4ae5a" is_pre_election: true
I20260709 13:23:34.713227 25309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89a26dd3ac414f6081b98fa32c2f346f" candidate_uuid: "3e07d64466934da4bb3c02262b1c2e38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" is_pre_election: true
I20260709 13:23:34.697849 25568 raft_consensus.cc:515] T 0cb89d70b4ce4aa8992fb68480bec8a2 P 3e07d64466934da4bb3c02262b1c2e38 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } }
I20260709 13:23:34.698148 25233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ffa42c3b2ff48bc87310e9ab5d218ce" candidate_uuid: "9868dce521294d0b816bba7648f4ae5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43999d813e564dc688fad116090c95ca" is_pre_election: true
I20260709 13:23:34.714936 25235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1ac2f4d2fe2424bb470df42199219e2" candidate_uuid: "3e07d64466934da4bb3c02262b1c2e38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43999d813e564dc688fad116090c95ca" is_pre_election: true
I20260709 13:23:34.723333 25508 ts_tablet_manager.cc:1434] T 0c4f6b7401db47d59a4f34ed0e8b5350 P 9868dce521294d0b816bba7648f4ae5a: Time spent starting tablet: real 0.034s	user 0.007s	sys 0.001s
I20260709 13:23:34.724318 25508 tablet_bootstrap.cc:492] T c902bbc7ec0848ba82a04870eab1e7f5 P 9868dce521294d0b816bba7648f4ae5a: Bootstrap starting.
I20260709 13:23:34.712389 25573 leader_election.cc:290] T f1ac2f4d2fe2424bb470df42199219e2 P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9868dce521294d0b816bba7648f4ae5a (127.14.177.132:33683), 43999d813e564dc688fad116090c95ca (127.14.177.129:38155)
I20260709 13:23:34.730016 25504 tablet_bootstrap.cc:492] T 13bb3a23315a41c7a40e10b954b15c8e P 3e07d64466934da4bb3c02262b1c2e38: No bootstrap required, opened a new log
I20260709 13:23:34.730434 25504 ts_tablet_manager.cc:1403] T 13bb3a23315a41c7a40e10b954b15c8e P 3e07d64466934da4bb3c02262b1c2e38: Time spent bootstrapping tablet: real 0.162s	user 0.031s	sys 0.012s
W20260709 13:23:34.722131 25351 leader_election.cc:343] T 89a26dd3ac414f6081b98fa32c2f346f P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 233e7d29aac04505bf4e65f0a39a0c4c (127.14.177.130:42957): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:23:34.731472 25233 raft_consensus.cc:2393] T 9ffa42c3b2ff48bc87310e9ab5d218ce P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9868dce521294d0b816bba7648f4ae5a in current term 1: Already voted for candidate 3e07d64466934da4bb3c02262b1c2e38 in this term.
W20260709 13:23:34.738210 25348 leader_election.cc:343] T 89a26dd3ac414f6081b98fa32c2f346f P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9868dce521294d0b816bba7648f4ae5a (127.14.177.132:33683): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:23:34.740237 25350 leader_election.cc:343] T f1ac2f4d2fe2424bb470df42199219e2 P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 43999d813e564dc688fad116090c95ca (127.14.177.129:38155): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:23:34.740340 25424 leader_election.cc:304] T 9ffa42c3b2ff48bc87310e9ab5d218ce P 9868dce521294d0b816bba7648f4ae5a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9868dce521294d0b816bba7648f4ae5a; no voters: 3e07d64466934da4bb3c02262b1c2e38, 43999d813e564dc688fad116090c95ca
I20260709 13:23:34.742327 25651 raft_consensus.cc:2749] T 9ffa42c3b2ff48bc87310e9ab5d218ce P 9868dce521294d0b816bba7648f4ae5a [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:23:34.743033 25563 consensus_queue.cc:1048] T 9e5d70664720424d951f269d9d7f4346 P 9868dce521294d0b816bba7648f4ae5a [LEADER]: Connected to new peer: Peer: permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:23:34.695207 25130 catalog_manager.cc:5697] T aad1f7937f854cc1a726223269da5d5a P 9868dce521294d0b816bba7648f4ae5a reported cstate change: term changed from 0 to 1, leader changed from <none> to 9868dce521294d0b816bba7648f4ae5a (127.14.177.132). New cstate: current_term: 1 leader_uuid: "9868dce521294d0b816bba7648f4ae5a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } health_report { overall_health: HEALTHY } } }
I20260709 13:23:34.746464 25456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1ac2f4d2fe2424bb470df42199219e2" candidate_uuid: "3e07d64466934da4bb3c02262b1c2e38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9868dce521294d0b816bba7648f4ae5a" is_pre_election: true
I20260709 13:23:34.758723 25504 raft_consensus.cc:359] T 13bb3a23315a41c7a40e10b954b15c8e P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } }
I20260709 13:23:34.759562 25348 leader_election.cc:304] T 89a26dd3ac414f6081b98fa32c2f346f P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3e07d64466934da4bb3c02262b1c2e38; no voters: 233e7d29aac04505bf4e65f0a39a0c4c, 9868dce521294d0b816bba7648f4ae5a
I20260709 13:23:34.763613 25504 raft_consensus.cc:385] T 13bb3a23315a41c7a40e10b954b15c8e P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:23:34.764109 25503 tablet_bootstrap.cc:492] T 0c4f6b7401db47d59a4f34ed0e8b5350 P 233e7d29aac04505bf4e65f0a39a0c4c: No bootstrap required, opened a new log
I20260709 13:23:34.764508 25505 tablet_bootstrap.cc:492] T c902bbc7ec0848ba82a04870eab1e7f5 P 43999d813e564dc688fad116090c95ca: No bootstrap required, opened a new log
I20260709 13:23:34.766283 25505 ts_tablet_manager.cc:1403] T c902bbc7ec0848ba82a04870eab1e7f5 P 43999d813e564dc688fad116090c95ca: Time spent bootstrapping tablet: real 0.119s	user 0.010s	sys 0.007s
W20260709 13:23:34.766589 25348 leader_election.cc:343] T f1ac2f4d2fe2424bb470df42199219e2 P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9868dce521294d0b816bba7648f4ae5a (127.14.177.132:33683): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:23:34.767444 25236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cb89d70b4ce4aa8992fb68480bec8a2" candidate_uuid: "3e07d64466934da4bb3c02262b1c2e38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43999d813e564dc688fad116090c95ca"
I20260709 13:23:34.767622 25686 consensus_queue.cc:1048] T 9ffa42c3b2ff48bc87310e9ab5d218ce P 3e07d64466934da4bb3c02262b1c2e38 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:23:34.769158 25505 raft_consensus.cc:359] T c902bbc7ec0848ba82a04870eab1e7f5 P 43999d813e564dc688fad116090c95ca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } }
I20260709 13:23:34.767455 25622 raft_consensus.cc:2749] T 89a26dd3ac414f6081b98fa32c2f346f P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:23:34.770370 25503 ts_tablet_manager.cc:1403] T 0c4f6b7401db47d59a4f34ed0e8b5350 P 233e7d29aac04505bf4e65f0a39a0c4c: Time spent bootstrapping tablet: real 0.186s	user 0.011s	sys 0.020s
I20260709 13:23:34.768085 25236 raft_consensus.cc:3060] T 0cb89d70b4ce4aa8992fb68480bec8a2 P 43999d813e564dc688fad116090c95ca [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:23:34.765054 25504 raft_consensus.cc:740] T 13bb3a23315a41c7a40e10b954b15c8e P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e07d64466934da4bb3c02262b1c2e38, State: Initialized, Role: FOLLOWER
I20260709 13:23:34.776628 25504 consensus_queue.cc:260] T 13bb3a23315a41c7a40e10b954b15c8e P 3e07d64466934da4bb3c02262b1c2e38 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } }
I20260709 13:23:34.779350 25568 leader_election.cc:290] T 0cb89d70b4ce4aa8992fb68480bec8a2 P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 1 election: Requested vote from peers 43999d813e564dc688fad116090c95ca (127.14.177.129:38155), 233e7d29aac04505bf4e65f0a39a0c4c (127.14.177.130:42957)
I20260709 13:23:34.781510 25504 ts_tablet_manager.cc:1434] T 13bb3a23315a41c7a40e10b954b15c8e P 3e07d64466934da4bb3c02262b1c2e38: Time spent starting tablet: real 0.051s	user 0.006s	sys 0.001s
I20260709 13:23:34.781296 25309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cb89d70b4ce4aa8992fb68480bec8a2" candidate_uuid: "3e07d64466934da4bb3c02262b1c2e38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "233e7d29aac04505bf4e65f0a39a0c4c"
I20260709 13:23:34.784024 25309 raft_consensus.cc:3060] T 0cb89d70b4ce4aa8992fb68480bec8a2 P 233e7d29aac04505bf4e65f0a39a0c4c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:23:34.769881 25505 raft_consensus.cc:385] T c902bbc7ec0848ba82a04870eab1e7f5 P 43999d813e564dc688fad116090c95ca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:23:34.773145 25503 raft_consensus.cc:359] T 0c4f6b7401db47d59a4f34ed0e8b5350 P 233e7d29aac04505bf4e65f0a39a0c4c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } }
I20260709 13:23:34.778858 25348 leader_election.cc:304] T f1ac2f4d2fe2424bb470df42199219e2 P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3e07d64466934da4bb3c02262b1c2e38; no voters: 43999d813e564dc688fad116090c95ca, 9868dce521294d0b816bba7648f4ae5a
I20260709 13:23:34.794369 25568 raft_consensus.cc:2749] T f1ac2f4d2fe2424bb470df42199219e2 P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:23:34.797869 25673 consensus_queue.cc:1048] T ce98a3f3b3eb4c63bcaa355ec7cd64a4 P 233e7d29aac04505bf4e65f0a39a0c4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:23:34.792479 25505 raft_consensus.cc:740] T c902bbc7ec0848ba82a04870eab1e7f5 P 43999d813e564dc688fad116090c95ca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43999d813e564dc688fad116090c95ca, State: Initialized, Role: FOLLOWER
I20260709 13:23:34.807971 25309 raft_consensus.cc:2468] T 0cb89d70b4ce4aa8992fb68480bec8a2 P 233e7d29aac04505bf4e65f0a39a0c4c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3e07d64466934da4bb3c02262b1c2e38 in term 1.
I20260709 13:23:34.805811 25505 consensus_queue.cc:260] T c902bbc7ec0848ba82a04870eab1e7f5 P 43999d813e564dc688fad116090c95ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } }
I20260709 13:23:34.793057 25503 raft_consensus.cc:385] T 0c4f6b7401db47d59a4f34ed0e8b5350 P 233e7d29aac04505bf4e65f0a39a0c4c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:23:34.809350 25236 raft_consensus.cc:2468] T 0cb89d70b4ce4aa8992fb68480bec8a2 P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3e07d64466934da4bb3c02262b1c2e38 in term 1.
I20260709 13:23:34.810014 25504 tablet_bootstrap.cc:492] T bbab48ba177049dcba6de106fd889cfb P 3e07d64466934da4bb3c02262b1c2e38: Bootstrap starting.
I20260709 13:23:34.814730 25503 raft_consensus.cc:740] T 0c4f6b7401db47d59a4f34ed0e8b5350 P 233e7d29aac04505bf4e65f0a39a0c4c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 233e7d29aac04505bf4e65f0a39a0c4c, State: Initialized, Role: FOLLOWER
I20260709 13:23:34.823992 25351 leader_election.cc:304] T 0cb89d70b4ce4aa8992fb68480bec8a2 P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 233e7d29aac04505bf4e65f0a39a0c4c, 3e07d64466934da4bb3c02262b1c2e38; no voters: 
I20260709 13:23:34.825388 25568 raft_consensus.cc:2804] T 0cb89d70b4ce4aa8992fb68480bec8a2 P 3e07d64466934da4bb3c02262b1c2e38 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:23:34.825346 25503 consensus_queue.cc:260] T 0c4f6b7401db47d59a4f34ed0e8b5350 P 233e7d29aac04505bf4e65f0a39a0c4c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } }
I20260709 13:23:34.826265 25568 raft_consensus.cc:697] T 0cb89d70b4ce4aa8992fb68480bec8a2 P 3e07d64466934da4bb3c02262b1c2e38 [term 1 LEADER]: Becoming Leader. State: Replica: 3e07d64466934da4bb3c02262b1c2e38, State: Running, Role: LEADER
I20260709 13:23:34.830137 25503 ts_tablet_manager.cc:1434] T 0c4f6b7401db47d59a4f34ed0e8b5350 P 233e7d29aac04505bf4e65f0a39a0c4c: Time spent starting tablet: real 0.059s	user 0.006s	sys 0.000s
I20260709 13:23:34.830627 25505 ts_tablet_manager.cc:1434] T c902bbc7ec0848ba82a04870eab1e7f5 P 43999d813e564dc688fad116090c95ca: Time spent starting tablet: real 0.064s	user 0.006s	sys 0.002s
I20260709 13:23:34.831060 25503 tablet_bootstrap.cc:492] T ad32097cd4dc48b9a69595f1651abd07 P 233e7d29aac04505bf4e65f0a39a0c4c: Bootstrap starting.
I20260709 13:23:34.832034 25505 tablet_bootstrap.cc:492] T 0c4f6b7401db47d59a4f34ed0e8b5350 P 43999d813e564dc688fad116090c95ca: Bootstrap starting.
I20260709 13:23:34.827564 25568 consensus_queue.cc:237] T 0cb89d70b4ce4aa8992fb68480bec8a2 P 3e07d64466934da4bb3c02262b1c2e38 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } }
I20260709 13:23:34.847173 25673 consensus_queue.cc:1048] T ce98a3f3b3eb4c63bcaa355ec7cd64a4 P 233e7d29aac04505bf4e65f0a39a0c4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:23:34.848719 25132 catalog_manager.cc:5697] T ad32097cd4dc48b9a69595f1651abd07 P 3e07d64466934da4bb3c02262b1c2e38 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3e07d64466934da4bb3c02262b1c2e38 (127.14.177.131). New cstate: current_term: 1 leader_uuid: "3e07d64466934da4bb3c02262b1c2e38" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } health_report { overall_health: UNKNOWN } } }
I20260709 13:23:34.878463 25508 tablet_bootstrap.cc:654] T c902bbc7ec0848ba82a04870eab1e7f5 P 9868dce521294d0b816bba7648f4ae5a: Neither blocks nor log segments found. Creating new log.
I20260709 13:23:34.903607 25504 tablet_bootstrap.cc:654] T bbab48ba177049dcba6de106fd889cfb P 3e07d64466934da4bb3c02262b1c2e38: Neither blocks nor log segments found. Creating new log.
I20260709 13:23:34.914341 25601 raft_consensus.cc:493] T c902bbc7ec0848ba82a04870eab1e7f5 P 43999d813e564dc688fad116090c95ca [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:23:34.915107 25601 raft_consensus.cc:515] T c902bbc7ec0848ba82a04870eab1e7f5 P 43999d813e564dc688fad116090c95ca [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } }
I20260709 13:23:34.922739 25601 leader_election.cc:290] T c902bbc7ec0848ba82a04870eab1e7f5 P 43999d813e564dc688fad116090c95ca [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3e07d64466934da4bb3c02262b1c2e38 (127.14.177.131:41539), 9868dce521294d0b816bba7648f4ae5a (127.14.177.132:33683)
I20260709 13:23:34.920719 25503 tablet_bootstrap.cc:654] T ad32097cd4dc48b9a69595f1651abd07 P 233e7d29aac04505bf4e65f0a39a0c4c: Neither blocks nor log segments found. Creating new log.
I20260709 13:23:34.924775 25385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c902bbc7ec0848ba82a04870eab1e7f5" candidate_uuid: "43999d813e564dc688fad116090c95ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e07d64466934da4bb3c02262b1c2e38" is_pre_election: true
I20260709 13:23:34.925755 25385 raft_consensus.cc:2468] T c902bbc7ec0848ba82a04870eab1e7f5 P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 43999d813e564dc688fad116090c95ca in term 0.
I20260709 13:23:34.927563 25456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c902bbc7ec0848ba82a04870eab1e7f5" candidate_uuid: "43999d813e564dc688fad116090c95ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9868dce521294d0b816bba7648f4ae5a" is_pre_election: true
W20260709 13:23:34.929231 25199 leader_election.cc:343] T c902bbc7ec0848ba82a04870eab1e7f5 P 43999d813e564dc688fad116090c95ca [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9868dce521294d0b816bba7648f4ae5a (127.14.177.132:33683): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:23:34.929989 25505 tablet_bootstrap.cc:654] T 0c4f6b7401db47d59a4f34ed0e8b5350 P 43999d813e564dc688fad116090c95ca: Neither blocks nor log segments found. Creating new log.
I20260709 13:23:34.934234 25199 leader_election.cc:304] T c902bbc7ec0848ba82a04870eab1e7f5 P 43999d813e564dc688fad116090c95ca [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3e07d64466934da4bb3c02262b1c2e38, 43999d813e564dc688fad116090c95ca; no voters: 9868dce521294d0b816bba7648f4ae5a
I20260709 13:23:34.939829 25601 raft_consensus.cc:2804] T c902bbc7ec0848ba82a04870eab1e7f5 P 43999d813e564dc688fad116090c95ca [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:23:34.940375 25601 raft_consensus.cc:493] T c902bbc7ec0848ba82a04870eab1e7f5 P 43999d813e564dc688fad116090c95ca [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:23:34.940836 25601 raft_consensus.cc:3060] T c902bbc7ec0848ba82a04870eab1e7f5 P 43999d813e564dc688fad116090c95ca [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:23:34.946044 25622 raft_consensus.cc:493] T 13bb3a23315a41c7a40e10b954b15c8e P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:23:34.946789 25622 raft_consensus.cc:515] T 13bb3a23315a41c7a40e10b954b15c8e P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } }
I20260709 13:23:34.952087 25307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13bb3a23315a41c7a40e10b954b15c8e" candidate_uuid: "3e07d64466934da4bb3c02262b1c2e38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" is_pre_election: true
I20260709 13:23:34.952255 25622 leader_election.cc:290] T 13bb3a23315a41c7a40e10b954b15c8e P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 43999d813e564dc688fad116090c95ca (127.14.177.129:38155), 233e7d29aac04505bf4e65f0a39a0c4c (127.14.177.130:42957)
W20260709 13:23:34.953842 25351 leader_election.cc:343] T 13bb3a23315a41c7a40e10b954b15c8e P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 233e7d29aac04505bf4e65f0a39a0c4c (127.14.177.130:42957): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:23:34.963807 25235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13bb3a23315a41c7a40e10b954b15c8e" candidate_uuid: "3e07d64466934da4bb3c02262b1c2e38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43999d813e564dc688fad116090c95ca" is_pre_election: true
I20260709 13:23:34.968014 25573 consensus_queue.cc:1048] T ad32097cd4dc48b9a69595f1651abd07 P 3e07d64466934da4bb3c02262b1c2e38 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:23:34.968557 25644 raft_consensus.cc:493] T 0c4f6b7401db47d59a4f34ed0e8b5350 P 9868dce521294d0b816bba7648f4ae5a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:23:34.969388 25644 raft_consensus.cc:515] T 0c4f6b7401db47d59a4f34ed0e8b5350 P 9868dce521294d0b816bba7648f4ae5a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } }
W20260709 13:23:34.970934 25350 leader_election.cc:343] T 13bb3a23315a41c7a40e10b954b15c8e P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 43999d813e564dc688fad116090c95ca (127.14.177.129:38155): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:23:34.971669 25350 leader_election.cc:304] T 13bb3a23315a41c7a40e10b954b15c8e P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3e07d64466934da4bb3c02262b1c2e38; no voters: 233e7d29aac04505bf4e65f0a39a0c4c, 43999d813e564dc688fad116090c95ca
I20260709 13:23:34.972937 25644 leader_election.cc:290] T 0c4f6b7401db47d59a4f34ed0e8b5350 P 9868dce521294d0b816bba7648f4ae5a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 43999d813e564dc688fad116090c95ca (127.14.177.129:38155), 233e7d29aac04505bf4e65f0a39a0c4c (127.14.177.130:42957)
I20260709 13:23:34.973883 25686 raft_consensus.cc:2749] T 13bb3a23315a41c7a40e10b954b15c8e P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:23:34.975258 25235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c4f6b7401db47d59a4f34ed0e8b5350" candidate_uuid: "9868dce521294d0b816bba7648f4ae5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43999d813e564dc688fad116090c95ca" is_pre_election: true
I20260709 13:23:34.976009 25307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c4f6b7401db47d59a4f34ed0e8b5350" candidate_uuid: "9868dce521294d0b816bba7648f4ae5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" is_pre_election: true
I20260709 13:23:34.976666 25307 raft_consensus.cc:2468] T 0c4f6b7401db47d59a4f34ed0e8b5350 P 233e7d29aac04505bf4e65f0a39a0c4c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9868dce521294d0b816bba7648f4ae5a in term 0.
I20260709 13:23:34.977946 25425 leader_election.cc:304] T 0c4f6b7401db47d59a4f34ed0e8b5350 P 9868dce521294d0b816bba7648f4ae5a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 233e7d29aac04505bf4e65f0a39a0c4c, 9868dce521294d0b816bba7648f4ae5a; no voters: 
W20260709 13:23:34.978514 25424 leader_election.cc:343] T 0c4f6b7401db47d59a4f34ed0e8b5350 P 9868dce521294d0b816bba7648f4ae5a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 43999d813e564dc688fad116090c95ca (127.14.177.129:38155): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:23:34.979615 25651 raft_consensus.cc:2804] T 0c4f6b7401db47d59a4f34ed0e8b5350 P 9868dce521294d0b816bba7648f4ae5a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:23:34.980145 25651 raft_consensus.cc:493] T 0c4f6b7401db47d59a4f34ed0e8b5350 P 9868dce521294d0b816bba7648f4ae5a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:23:34.980574 25651 raft_consensus.cc:3060] T 0c4f6b7401db47d59a4f34ed0e8b5350 P 9868dce521294d0b816bba7648f4ae5a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:23:34.997742 25505 tablet_bootstrap.cc:492] T 0c4f6b7401db47d59a4f34ed0e8b5350 P 43999d813e564dc688fad116090c95ca: No bootstrap required, opened a new log
I20260709 13:23:34.998252 25505 ts_tablet_manager.cc:1403] T 0c4f6b7401db47d59a4f34ed0e8b5350 P 43999d813e564dc688fad116090c95ca: Time spent bootstrapping tablet: real 0.166s	user 0.022s	sys 0.035s
I20260709 13:23:35.000066 25508 tablet_bootstrap.cc:492] T c902bbc7ec0848ba82a04870eab1e7f5 P 9868dce521294d0b816bba7648f4ae5a: No bootstrap required, opened a new log
I20260709 13:23:35.000607 25508 ts_tablet_manager.cc:1403] T c902bbc7ec0848ba82a04870eab1e7f5 P 9868dce521294d0b816bba7648f4ae5a: Time spent bootstrapping tablet: real 0.276s	user 0.009s	sys 0.007s
I20260709 13:23:35.009644 25508 raft_consensus.cc:359] T c902bbc7ec0848ba82a04870eab1e7f5 P 9868dce521294d0b816bba7648f4ae5a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } }
I20260709 13:23:35.010377 25508 raft_consensus.cc:385] T c902bbc7ec0848ba82a04870eab1e7f5 P 9868dce521294d0b816bba7648f4ae5a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:23:35.010658 25508 raft_consensus.cc:740] T c902bbc7ec0848ba82a04870eab1e7f5 P 9868dce521294d0b816bba7648f4ae5a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9868dce521294d0b816bba7648f4ae5a, State: Initialized, Role: FOLLOWER
I20260709 13:23:35.011344 25508 consensus_queue.cc:260] T c902bbc7ec0848ba82a04870eab1e7f5 P 9868dce521294d0b816bba7648f4ae5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } }
I20260709 13:23:35.025208 25508 ts_tablet_manager.cc:1434] T c902bbc7ec0848ba82a04870eab1e7f5 P 9868dce521294d0b816bba7648f4ae5a: Time spent starting tablet: real 0.024s	user 0.008s	sys 0.000s
I20260709 13:23:35.026755 25508 tablet_bootstrap.cc:492] T 1550ed2d2a764c08adffae679599ba41 P 9868dce521294d0b816bba7648f4ae5a: Bootstrap starting.
W20260709 13:23:35.030234 25351 consensus_peers.cc:597] T ad32097cd4dc48b9a69595f1651abd07 P 3e07d64466934da4bb3c02262b1c2e38 -> Peer 233e7d29aac04505bf4e65f0a39a0c4c (127.14.177.130:42957): Couldn't send request to peer 233e7d29aac04505bf4e65f0a39a0c4c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:23:35.032320 25651 raft_consensus.cc:515] T 0c4f6b7401db47d59a4f34ed0e8b5350 P 9868dce521294d0b816bba7648f4ae5a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } }
I20260709 13:23:35.035658 25307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c4f6b7401db47d59a4f34ed0e8b5350" candidate_uuid: "9868dce521294d0b816bba7648f4ae5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "233e7d29aac04505bf4e65f0a39a0c4c"
I20260709 13:23:35.036322 25307 raft_consensus.cc:3060] T 0c4f6b7401db47d59a4f34ed0e8b5350 P 233e7d29aac04505bf4e65f0a39a0c4c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:23:35.039897 25234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c4f6b7401db47d59a4f34ed0e8b5350" candidate_uuid: "9868dce521294d0b816bba7648f4ae5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43999d813e564dc688fad116090c95ca"
I20260709 13:23:35.042919 25505 raft_consensus.cc:359] T 0c4f6b7401db47d59a4f34ed0e8b5350 P 43999d813e564dc688fad116090c95ca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } }
I20260709 13:23:35.043943 25505 raft_consensus.cc:385] T 0c4f6b7401db47d59a4f34ed0e8b5350 P 43999d813e564dc688fad116090c95ca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:23:35.044190 25505 raft_consensus.cc:740] T 0c4f6b7401db47d59a4f34ed0e8b5350 P 43999d813e564dc688fad116090c95ca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43999d813e564dc688fad116090c95ca, State: Initialized, Role: FOLLOWER
I20260709 13:23:35.049757 25504 tablet_bootstrap.cc:492] T bbab48ba177049dcba6de106fd889cfb P 3e07d64466934da4bb3c02262b1c2e38: No bootstrap required, opened a new log
I20260709 13:23:35.050295 25504 ts_tablet_manager.cc:1403] T bbab48ba177049dcba6de106fd889cfb P 3e07d64466934da4bb3c02262b1c2e38: Time spent bootstrapping tablet: real 0.241s	user 0.039s	sys 0.054s
I20260709 13:23:35.052417 25504 raft_consensus.cc:359] T bbab48ba177049dcba6de106fd889cfb P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } }
I20260709 13:23:35.053121 25504 raft_consensus.cc:385] T bbab48ba177049dcba6de106fd889cfb P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:23:35.053403 25504 raft_consensus.cc:740] T bbab48ba177049dcba6de106fd889cfb P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e07d64466934da4bb3c02262b1c2e38, State: Initialized, Role: FOLLOWER
I20260709 13:23:35.054097 25504 consensus_queue.cc:260] T bbab48ba177049dcba6de106fd889cfb P 3e07d64466934da4bb3c02262b1c2e38 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } }
I20260709 13:23:35.053503 25651 leader_election.cc:290] T 0c4f6b7401db47d59a4f34ed0e8b5350 P 9868dce521294d0b816bba7648f4ae5a [CANDIDATE]: Term 1 election: Requested vote from peers 43999d813e564dc688fad116090c95ca (127.14.177.129:38155), 233e7d29aac04505bf4e65f0a39a0c4c (127.14.177.130:42957)
I20260709 13:23:35.056056 25508 tablet_bootstrap.cc:654] T 1550ed2d2a764c08adffae679599ba41 P 9868dce521294d0b816bba7648f4ae5a: Neither blocks nor log segments found. Creating new log.
I20260709 13:23:35.056250 25504 ts_tablet_manager.cc:1434] T bbab48ba177049dcba6de106fd889cfb P 3e07d64466934da4bb3c02262b1c2e38: Time spent starting tablet: real 0.006s	user 0.007s	sys 0.000s
I20260709 13:23:35.056466 25712 consensus_queue.cc:1048] T 51a11db644124b63bdf5f457c967473d P 233e7d29aac04505bf4e65f0a39a0c4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:23:35.057576 25504 tablet_bootstrap.cc:492] T 79d4a12ef58140568c60a24ed4c12f83 P 3e07d64466934da4bb3c02262b1c2e38: Bootstrap starting.
I20260709 13:23:35.046345 25505 consensus_queue.cc:260] T 0c4f6b7401db47d59a4f34ed0e8b5350 P 43999d813e564dc688fad116090c95ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } }
I20260709 13:23:35.035719 25601 raft_consensus.cc:515] T c902bbc7ec0848ba82a04870eab1e7f5 P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } }
I20260709 13:23:35.065366 25234 raft_consensus.cc:3060] T 0c4f6b7401db47d59a4f34ed0e8b5350 P 43999d813e564dc688fad116090c95ca [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:23:35.069118 25307 raft_consensus.cc:2468] T 0c4f6b7401db47d59a4f34ed0e8b5350 P 233e7d29aac04505bf4e65f0a39a0c4c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9868dce521294d0b816bba7648f4ae5a in term 1.
I20260709 13:23:35.070464 25505 ts_tablet_manager.cc:1434] T 0c4f6b7401db47d59a4f34ed0e8b5350 P 43999d813e564dc688fad116090c95ca: Time spent starting tablet: real 0.072s	user 0.007s	sys 0.001s
I20260709 13:23:35.071403 25505 tablet_bootstrap.cc:492] T 1550ed2d2a764c08adffae679599ba41 P 43999d813e564dc688fad116090c95ca: Bootstrap starting.
I20260709 13:23:35.074522 25385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c902bbc7ec0848ba82a04870eab1e7f5" candidate_uuid: "43999d813e564dc688fad116090c95ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e07d64466934da4bb3c02262b1c2e38"
I20260709 13:23:35.075150 25385 raft_consensus.cc:3060] T c902bbc7ec0848ba82a04870eab1e7f5 P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:23:35.080241 25505 tablet_bootstrap.cc:654] T 1550ed2d2a764c08adffae679599ba41 P 43999d813e564dc688fad116090c95ca: Neither blocks nor log segments found. Creating new log.
I20260709 13:23:35.086159 25503 tablet_bootstrap.cc:492] T ad32097cd4dc48b9a69595f1651abd07 P 233e7d29aac04505bf4e65f0a39a0c4c: No bootstrap required, opened a new log
I20260709 13:23:35.063334 25130 catalog_manager.cc:5697] T 0cb89d70b4ce4aa8992fb68480bec8a2 P 3e07d64466934da4bb3c02262b1c2e38 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3e07d64466934da4bb3c02262b1c2e38 (127.14.177.131). New cstate: current_term: 1 leader_uuid: "3e07d64466934da4bb3c02262b1c2e38" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } health_report { overall_health: HEALTHY } } }
I20260709 13:23:35.086836 25503 ts_tablet_manager.cc:1403] T ad32097cd4dc48b9a69595f1651abd07 P 233e7d29aac04505bf4e65f0a39a0c4c: Time spent bootstrapping tablet: real 0.256s	user 0.015s	sys 0.014s
I20260709 13:23:35.086760 25234 raft_consensus.cc:2468] T 0c4f6b7401db47d59a4f34ed0e8b5350 P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9868dce521294d0b816bba7648f4ae5a in term 1.
I20260709 13:23:35.092439 25651 consensus_queue.cc:1048] T aad1f7937f854cc1a726223269da5d5a P 9868dce521294d0b816bba7648f4ae5a [LEADER]: Connected to new peer: Peer: permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 13:23:35.094807 25385 raft_consensus.cc:2468] T c902bbc7ec0848ba82a04870eab1e7f5 P 3e07d64466934da4bb3c02262b1c2e38 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 43999d813e564dc688fad116090c95ca in term 1.
I20260709 13:23:35.096315 25504 tablet_bootstrap.cc:654] T 79d4a12ef58140568c60a24ed4c12f83 P 3e07d64466934da4bb3c02262b1c2e38: Neither blocks nor log segments found. Creating new log.
I20260709 13:23:35.098546 25505 tablet_bootstrap.cc:492] T 1550ed2d2a764c08adffae679599ba41 P 43999d813e564dc688fad116090c95ca: No bootstrap required, opened a new log
I20260709 13:23:35.099082 25505 ts_tablet_manager.cc:1403] T 1550ed2d2a764c08adffae679599ba41 P 43999d813e564dc688fad116090c95ca: Time spent bootstrapping tablet: real 0.028s	user 0.014s	sys 0.010s
I20260709 13:23:35.098670 25651 consensus_queue.cc:1048] T aad1f7937f854cc1a726223269da5d5a P 9868dce521294d0b816bba7648f4ae5a [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:23:35.094131 25601 leader_election.cc:290] T c902bbc7ec0848ba82a04870eab1e7f5 P 43999d813e564dc688fad116090c95ca [CANDIDATE]: Term 1 election: Requested vote from peers 3e07d64466934da4bb3c02262b1c2e38 (127.14.177.131:41539), 9868dce521294d0b816bba7648f4ae5a (127.14.177.132:33683)
I20260709 13:23:35.100669 25199 leader_election.cc:304] T c902bbc7ec0848ba82a04870eab1e7f5 P 43999d813e564dc688fad116090c95ca [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e07d64466934da4bb3c02262b1c2e38, 43999d813e564dc688fad116090c95ca; no voters: 
I20260709 13:23:35.101750 25425 leader_election.cc:304] T 0c4f6b7401db47d59a4f34ed0e8b5350 P 9868dce521294d0b816bba7648f4ae5a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 233e7d29aac04505bf4e65f0a39a0c4c, 9868dce521294d0b816bba7648f4ae5a; no voters: 
I20260709 13:23:35.104202 25455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c902bbc7ec0848ba82a04870eab1e7f5" candidate_uuid: "43999d813e564dc688fad116090c95ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9868dce521294d0b816bba7648f4ae5a"
I20260709 13:23:35.104569 25651 raft_consensus.cc:2804] T 0c4f6b7401db47d59a4f34ed0e8b5350 P 9868dce521294d0b816bba7648f4ae5a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:23:35.104856 25455 raft_consensus.cc:3060] T c902bbc7ec0848ba82a04870eab1e7f5 P 9868dce521294d0b816bba7648f4ae5a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:23:35.105298 25651 raft_consensus.cc:697] T 0c4f6b7401db47d59a4f34ed0e8b5350 P 9868dce521294d0b816bba7648f4ae5a [term 1 LEADER]: Becoming Leader. State: Replica: 9868dce521294d0b816bba7648f4ae5a, State: Running, Role: LEADER
I20260709 13:23:35.105369 25503 raft_consensus.cc:359] T ad32097cd4dc48b9a69595f1651abd07 P 233e7d29aac04505bf4e65f0a39a0c4c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } }
I20260709 13:23:35.106096 25503 raft_consensus.cc:385] T ad32097cd4dc48b9a69595f1651abd07 P 233e7d29aac04505bf4e65f0a39a0c4c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:23:35.106395 25503 raft_consensus.cc:740] T ad32097cd4dc48b9a69595f1651abd07 P 233e7d29aac04505bf4e65f0a39a0c4c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 233e7d29aac04505bf4e65f0a39a0c4c, State: Initialized, Role: FOLLOWER
I20260709 13:23:35.106432 25651 consensus_queue.cc:237] T 0c4f6b7401db47d59a4f34ed0e8b5350 P 9868dce521294d0b816bba7648f4ae5a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } }
I20260709 13:23:35.103358 25601 raft_consensus.cc:2804] T c902bbc7ec0848ba82a04870eab1e7f5 P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:23:35.109735 25508 tablet_bootstrap.cc:492] T 1550ed2d2a764c08adffae679599ba41 P 9868dce521294d0b816bba7648f4ae5a: No bootstrap required, opened a new log
I20260709 13:23:35.110245 25508 ts_tablet_manager.cc:1403] T 1550ed2d2a764c08adffae679599ba41 P 9868dce521294d0b816bba7648f4ae5a: Time spent bootstrapping tablet: real 0.084s	user 0.005s	sys 0.015s
I20260709 13:23:35.111431 25601 raft_consensus.cc:697] T c902bbc7ec0848ba82a04870eab1e7f5 P 43999d813e564dc688fad116090c95ca [term 1 LEADER]: Becoming Leader. State: Replica: 43999d813e564dc688fad116090c95ca, State: Running, Role: LEADER
I20260709 13:23:35.115914 25505 raft_consensus.cc:359] T 1550ed2d2a764c08adffae679599ba41 P 43999d813e564dc688fad116090c95ca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } }
I20260709 13:23:35.116695 25505 raft_consensus.cc:385] T 1550ed2d2a764c08adffae679599ba41 P 43999d813e564dc688fad116090c95ca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:23:35.117285 25505 raft_consensus.cc:740] T 1550ed2d2a764c08adffae679599ba41 P 43999d813e564dc688fad116090c95ca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43999d813e564dc688fad116090c95ca, State: Initialized, Role: FOLLOWER
I20260709 13:23:35.118369 25505 consensus_queue.cc:260] T 1550ed2d2a764c08adffae679599ba41 P 43999d813e564dc688fad116090c95ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } }
I20260709 13:23:35.121662 25505 ts_tablet_manager.cc:1434] T 1550ed2d2a764c08adffae679599ba41 P 43999d813e564dc688fad116090c95ca: Time spent starting tablet: real 0.022s	user 0.005s	sys 0.000s
I20260709 13:23:35.121344 25503 consensus_queue.cc:260] T ad32097cd4dc48b9a69595f1651abd07 P 233e7d29aac04505bf4e65f0a39a0c4c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } }
I20260709 13:23:35.131906 25503 ts_tablet_manager.cc:1434] T ad32097cd4dc48b9a69595f1651abd07 P 233e7d29aac04505bf4e65f0a39a0c4c: Time spent starting tablet: real 0.042s	user 0.007s	sys 0.001s
I20260709 13:23:35.133234 25503 tablet_bootstrap.cc:492] T a5df750b84e0487fa530161079deb6c2 P 233e7d29aac04505bf4e65f0a39a0c4c: Bootstrap starting.
I20260709 13:23:35.135892 25712 consensus_queue.cc:1048] T 51a11db644124b63bdf5f457c967473d P 233e7d29aac04505bf4e65f0a39a0c4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:23:35.112962 25601 consensus_queue.cc:237] T c902bbc7ec0848ba82a04870eab1e7f5 P 43999d813e564dc688fad116090c95ca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } }
I20260709 13:23:35.147219 25686 raft_consensus.cc:493] T bbab48ba177049dcba6de106fd889cfb P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:23:35.144327 25455 raft_consensus.cc:2468] T c902bbc7ec0848ba82a04870eab1e7f5 P 9868dce521294d0b816bba7648f4ae5a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 43999d813e564dc688fad116090c95ca in term 1.
I20260709 13:23:35.151644 25686 raft_consensus.cc:515] T bbab48ba177049dcba6de106fd889cfb P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } }
I20260709 13:23:35.153951 25686 leader_election.cc:290] T bbab48ba177049dcba6de106fd889cfb P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 233e7d29aac04505bf4e65f0a39a0c4c (127.14.177.130:42957), 9868dce521294d0b816bba7648f4ae5a (127.14.177.132:33683)
I20260709 13:23:35.155654 25505 tablet_bootstrap.cc:492] T a5df750b84e0487fa530161079deb6c2 P 43999d813e564dc688fad116090c95ca: Bootstrap starting.
I20260709 13:23:35.157385 25508 raft_consensus.cc:359] T 1550ed2d2a764c08adffae679599ba41 P 9868dce521294d0b816bba7648f4ae5a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } }
I20260709 13:23:35.159931 25508 raft_consensus.cc:385] T 1550ed2d2a764c08adffae679599ba41 P 9868dce521294d0b816bba7648f4ae5a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:23:35.160297 25508 raft_consensus.cc:740] T 1550ed2d2a764c08adffae679599ba41 P 9868dce521294d0b816bba7648f4ae5a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9868dce521294d0b816bba7648f4ae5a, State: Initialized, Role: FOLLOWER
I20260709 13:23:35.164077 25508 consensus_queue.cc:260] T 1550ed2d2a764c08adffae679599ba41 P 9868dce521294d0b816bba7648f4ae5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } }
I20260709 13:23:35.166522 25508 ts_tablet_manager.cc:1434] T 1550ed2d2a764c08adffae679599ba41 P 9868dce521294d0b816bba7648f4ae5a: Time spent starting tablet: real 0.056s	user 0.008s	sys 0.000s
I20260709 13:23:35.168737 25309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbab48ba177049dcba6de106fd889cfb" candidate_uuid: "3e07d64466934da4bb3c02262b1c2e38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" is_pre_election: true
I20260709 13:23:35.170747 25508 tablet_bootstrap.cc:492] T 89a26dd3ac414f6081b98fa32c2f346f P 9868dce521294d0b816bba7648f4ae5a: Bootstrap starting.
W20260709 13:23:35.171802 25351 leader_election.cc:343] T bbab48ba177049dcba6de106fd889cfb P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 233e7d29aac04505bf4e65f0a39a0c4c (127.14.177.130:42957): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:23:35.174686 25182 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:23:35.175540 25455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbab48ba177049dcba6de106fd889cfb" candidate_uuid: "3e07d64466934da4bb3c02262b1c2e38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9868dce521294d0b816bba7648f4ae5a" is_pre_election: true
W20260709 13:23:35.177187 25348 leader_election.cc:343] T bbab48ba177049dcba6de106fd889cfb P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9868dce521294d0b816bba7648f4ae5a (127.14.177.132:33683): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:23:35.177641 25348 leader_election.cc:304] T bbab48ba177049dcba6de106fd889cfb P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3e07d64466934da4bb3c02262b1c2e38; no voters: 233e7d29aac04505bf4e65f0a39a0c4c, 9868dce521294d0b816bba7648f4ae5a
I20260709 13:23:35.178393 25686 raft_consensus.cc:2749] T bbab48ba177049dcba6de106fd889cfb P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:23:35.184438 25505 tablet_bootstrap.cc:654] T a5df750b84e0487fa530161079deb6c2 P 43999d813e564dc688fad116090c95ca: Neither blocks nor log segments found. Creating new log.
I20260709 13:23:35.186141 25503 tablet_bootstrap.cc:654] T a5df750b84e0487fa530161079deb6c2 P 233e7d29aac04505bf4e65f0a39a0c4c: Neither blocks nor log segments found. Creating new log.
I20260709 13:23:35.192314 25504 tablet_bootstrap.cc:492] T 79d4a12ef58140568c60a24ed4c12f83 P 3e07d64466934da4bb3c02262b1c2e38: No bootstrap required, opened a new log
I20260709 13:23:35.192766 25504 ts_tablet_manager.cc:1403] T 79d4a12ef58140568c60a24ed4c12f83 P 3e07d64466934da4bb3c02262b1c2e38: Time spent bootstrapping tablet: real 0.135s	user 0.021s	sys 0.008s
I20260709 13:23:35.189885 25132 catalog_manager.cc:5697] T 0c4f6b7401db47d59a4f34ed0e8b5350 P 9868dce521294d0b816bba7648f4ae5a reported cstate change: term changed from 0 to 1, leader changed from <none> to 9868dce521294d0b816bba7648f4ae5a (127.14.177.132). New cstate: current_term: 1 leader_uuid: "9868dce521294d0b816bba7648f4ae5a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } health_report { overall_health: UNKNOWN } } }
I20260709 13:23:35.211262 25129 catalog_manager.cc:5697] T c902bbc7ec0848ba82a04870eab1e7f5 P 43999d813e564dc688fad116090c95ca reported cstate change: term changed from 0 to 1, leader changed from <none> to 43999d813e564dc688fad116090c95ca (127.14.177.129). New cstate: current_term: 1 leader_uuid: "43999d813e564dc688fad116090c95ca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } health_report { overall_health: HEALTHY } } }
I20260709 13:23:35.220150 25504 raft_consensus.cc:359] T 79d4a12ef58140568c60a24ed4c12f83 P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } }
I20260709 13:23:35.222091 25504 raft_consensus.cc:385] T 79d4a12ef58140568c60a24ed4c12f83 P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:23:35.222478 25504 raft_consensus.cc:740] T 79d4a12ef58140568c60a24ed4c12f83 P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e07d64466934da4bb3c02262b1c2e38, State: Initialized, Role: FOLLOWER
I20260709 13:23:35.232466 25504 consensus_queue.cc:260] T 79d4a12ef58140568c60a24ed4c12f83 P 3e07d64466934da4bb3c02262b1c2e38 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } }
I20260709 13:23:35.234412 25508 tablet_bootstrap.cc:654] T 89a26dd3ac414f6081b98fa32c2f346f P 9868dce521294d0b816bba7648f4ae5a: Neither blocks nor log segments found. Creating new log.
I20260709 13:23:35.235375 25505 tablet_bootstrap.cc:492] T a5df750b84e0487fa530161079deb6c2 P 43999d813e564dc688fad116090c95ca: No bootstrap required, opened a new log
I20260709 13:23:35.235908 25505 ts_tablet_manager.cc:1403] T a5df750b84e0487fa530161079deb6c2 P 43999d813e564dc688fad116090c95ca: Time spent bootstrapping tablet: real 0.080s	user 0.014s	sys 0.008s
I20260709 13:23:35.239140 25505 raft_consensus.cc:359] T a5df750b84e0487fa530161079deb6c2 P 43999d813e564dc688fad116090c95ca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } }
I20260709 13:23:35.239863 25505 raft_consensus.cc:385] T a5df750b84e0487fa530161079deb6c2 P 43999d813e564dc688fad116090c95ca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:23:35.240136 25505 raft_consensus.cc:740] T a5df750b84e0487fa530161079deb6c2 P 43999d813e564dc688fad116090c95ca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43999d813e564dc688fad116090c95ca, State: Initialized, Role: FOLLOWER
I20260709 13:23:35.240792 25505 consensus_queue.cc:260] T a5df750b84e0487fa530161079deb6c2 P 43999d813e564dc688fad116090c95ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } }
I20260709 13:23:35.242977 25505 ts_tablet_manager.cc:1434] T a5df750b84e0487fa530161079deb6c2 P 43999d813e564dc688fad116090c95ca: Time spent starting tablet: real 0.007s	user 0.001s	sys 0.002s
I20260709 13:23:35.244001 25505 tablet_bootstrap.cc:492] T 13bb3a23315a41c7a40e10b954b15c8e P 43999d813e564dc688fad116090c95ca: Bootstrap starting.
I20260709 13:23:35.246516 25504 ts_tablet_manager.cc:1434] T 79d4a12ef58140568c60a24ed4c12f83 P 3e07d64466934da4bb3c02262b1c2e38: Time spent starting tablet: real 0.054s	user 0.006s	sys 0.002s
I20260709 13:23:35.247678 25504 tablet_bootstrap.cc:492] T 87085ac2575e4b6a86dd1fc21e6f718b P 3e07d64466934da4bb3c02262b1c2e38: Bootstrap starting.
I20260709 13:23:35.249951 25505 tablet_bootstrap.cc:654] T 13bb3a23315a41c7a40e10b954b15c8e P 43999d813e564dc688fad116090c95ca: Neither blocks nor log segments found. Creating new log.
I20260709 13:23:35.254427 25504 tablet_bootstrap.cc:654] T 87085ac2575e4b6a86dd1fc21e6f718b P 3e07d64466934da4bb3c02262b1c2e38: Neither blocks nor log segments found. Creating new log.
I20260709 13:23:35.263411 25508 tablet_bootstrap.cc:492] T 89a26dd3ac414f6081b98fa32c2f346f P 9868dce521294d0b816bba7648f4ae5a: No bootstrap required, opened a new log
I20260709 13:23:35.264034 25508 ts_tablet_manager.cc:1403] T 89a26dd3ac414f6081b98fa32c2f346f P 9868dce521294d0b816bba7648f4ae5a: Time spent bootstrapping tablet: real 0.093s	user 0.012s	sys 0.000s
I20260709 13:23:35.267768 25503 tablet_bootstrap.cc:492] T a5df750b84e0487fa530161079deb6c2 P 233e7d29aac04505bf4e65f0a39a0c4c: No bootstrap required, opened a new log
I20260709 13:23:35.267153 25508 raft_consensus.cc:359] T 89a26dd3ac414f6081b98fa32c2f346f P 9868dce521294d0b816bba7648f4ae5a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } }
I20260709 13:23:35.268288 25508 raft_consensus.cc:385] T 89a26dd3ac414f6081b98fa32c2f346f P 9868dce521294d0b816bba7648f4ae5a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:23:35.268669 25508 raft_consensus.cc:740] T 89a26dd3ac414f6081b98fa32c2f346f P 9868dce521294d0b816bba7648f4ae5a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9868dce521294d0b816bba7648f4ae5a, State: Initialized, Role: FOLLOWER
I20260709 13:23:35.268352 25503 ts_tablet_manager.cc:1403] T a5df750b84e0487fa530161079deb6c2 P 233e7d29aac04505bf4e65f0a39a0c4c: Time spent bootstrapping tablet: real 0.135s	user 0.016s	sys 0.008s
I20260709 13:23:35.277688 25508 consensus_queue.cc:260] T 89a26dd3ac414f6081b98fa32c2f346f P 9868dce521294d0b816bba7648f4ae5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } }
I20260709 13:23:35.281594 25508 ts_tablet_manager.cc:1434] T 89a26dd3ac414f6081b98fa32c2f346f P 9868dce521294d0b816bba7648f4ae5a: Time spent starting tablet: real 0.017s	user 0.007s	sys 0.000s
I20260709 13:23:35.282612 25508 tablet_bootstrap.cc:492] T a5df750b84e0487fa530161079deb6c2 P 9868dce521294d0b816bba7648f4ae5a: Bootstrap starting.
I20260709 13:23:35.289392 25508 tablet_bootstrap.cc:654] T a5df750b84e0487fa530161079deb6c2 P 9868dce521294d0b816bba7648f4ae5a: Neither blocks nor log segments found. Creating new log.
I20260709 13:23:35.292611 25505 tablet_bootstrap.cc:492] T 13bb3a23315a41c7a40e10b954b15c8e P 43999d813e564dc688fad116090c95ca: No bootstrap required, opened a new log
I20260709 13:23:35.293269 25505 ts_tablet_manager.cc:1403] T 13bb3a23315a41c7a40e10b954b15c8e P 43999d813e564dc688fad116090c95ca: Time spent bootstrapping tablet: real 0.049s	user 0.009s	sys 0.033s
I20260709 13:23:35.276429 25503 raft_consensus.cc:359] T a5df750b84e0487fa530161079deb6c2 P 233e7d29aac04505bf4e65f0a39a0c4c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } }
I20260709 13:23:35.297005 25505 raft_consensus.cc:359] T 13bb3a23315a41c7a40e10b954b15c8e P 43999d813e564dc688fad116090c95ca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } }
I20260709 13:23:35.297976 25503 raft_consensus.cc:385] T a5df750b84e0487fa530161079deb6c2 P 233e7d29aac04505bf4e65f0a39a0c4c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:23:35.298177 25505 raft_consensus.cc:385] T 13bb3a23315a41c7a40e10b954b15c8e P 43999d813e564dc688fad116090c95ca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:23:35.298377 25503 raft_consensus.cc:740] T a5df750b84e0487fa530161079deb6c2 P 233e7d29aac04505bf4e65f0a39a0c4c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 233e7d29aac04505bf4e65f0a39a0c4c, State: Initialized, Role: FOLLOWER
I20260709 13:23:35.298739 25505 raft_consensus.cc:740] T 13bb3a23315a41c7a40e10b954b15c8e P 43999d813e564dc688fad116090c95ca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43999d813e564dc688fad116090c95ca, State: Initialized, Role: FOLLOWER
I20260709 13:23:35.299573 25503 consensus_queue.cc:260] T a5df750b84e0487fa530161079deb6c2 P 233e7d29aac04505bf4e65f0a39a0c4c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } }
I20260709 13:23:35.300086 25505 consensus_queue.cc:260] T 13bb3a23315a41c7a40e10b954b15c8e P 43999d813e564dc688fad116090c95ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } }
I20260709 13:23:35.303234 25505 ts_tablet_manager.cc:1434] T 13bb3a23315a41c7a40e10b954b15c8e P 43999d813e564dc688fad116090c95ca: Time spent starting tablet: real 0.009s	user 0.009s	sys 0.000s
I20260709 13:23:35.304978 25505 tablet_bootstrap.cc:492] T f1ac2f4d2fe2424bb470df42199219e2 P 43999d813e564dc688fad116090c95ca: Bootstrap starting.
I20260709 13:23:35.313822 25505 tablet_bootstrap.cc:654] T f1ac2f4d2fe2424bb470df42199219e2 P 43999d813e564dc688fad116090c95ca: Neither blocks nor log segments found. Creating new log.
I20260709 13:23:35.314555 25504 tablet_bootstrap.cc:492] T 87085ac2575e4b6a86dd1fc21e6f718b P 3e07d64466934da4bb3c02262b1c2e38: No bootstrap required, opened a new log
I20260709 13:23:35.315130 25504 ts_tablet_manager.cc:1403] T 87085ac2575e4b6a86dd1fc21e6f718b P 3e07d64466934da4bb3c02262b1c2e38: Time spent bootstrapping tablet: real 0.068s	user 0.020s	sys 0.018s
I20260709 13:23:35.318099 25504 raft_consensus.cc:359] T 87085ac2575e4b6a86dd1fc21e6f718b P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } }
I20260709 13:23:35.318660 25504 raft_consensus.cc:385] T 87085ac2575e4b6a86dd1fc21e6f718b P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:23:35.318843 25504 raft_consensus.cc:740] T 87085ac2575e4b6a86dd1fc21e6f718b P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e07d64466934da4bb3c02262b1c2e38, State: Initialized, Role: FOLLOWER
I20260709 13:23:35.319641 25504 consensus_queue.cc:260] T 87085ac2575e4b6a86dd1fc21e6f718b P 3e07d64466934da4bb3c02262b1c2e38 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } }
I20260709 13:23:35.320923 25503 ts_tablet_manager.cc:1434] T a5df750b84e0487fa530161079deb6c2 P 233e7d29aac04505bf4e65f0a39a0c4c: Time spent starting tablet: real 0.048s	user 0.007s	sys 0.000s
I20260709 13:23:35.321339 25505 tablet_bootstrap.cc:492] T f1ac2f4d2fe2424bb470df42199219e2 P 43999d813e564dc688fad116090c95ca: No bootstrap required, opened a new log
I20260709 13:23:35.322590 25505 ts_tablet_manager.cc:1403] T f1ac2f4d2fe2424bb470df42199219e2 P 43999d813e564dc688fad116090c95ca: Time spent bootstrapping tablet: real 0.018s	user 0.007s	sys 0.005s
I20260709 13:23:35.323915 25503 tablet_bootstrap.cc:492] T 13bb3a23315a41c7a40e10b954b15c8e P 233e7d29aac04505bf4e65f0a39a0c4c: Bootstrap starting.
I20260709 13:23:35.326335 25504 ts_tablet_manager.cc:1434] T 87085ac2575e4b6a86dd1fc21e6f718b P 3e07d64466934da4bb3c02262b1c2e38: Time spent starting tablet: real 0.011s	user 0.004s	sys 0.003s
I20260709 13:23:35.327395 25504 tablet_bootstrap.cc:492] T 91ac744179de48e5bb924b6dfd9f9c9b P 3e07d64466934da4bb3c02262b1c2e38: Bootstrap starting.
I20260709 13:23:35.330938 25539 raft_consensus.cc:493] T 89a26dd3ac414f6081b98fa32c2f346f P 9868dce521294d0b816bba7648f4ae5a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:23:35.329195 25505 raft_consensus.cc:359] T f1ac2f4d2fe2424bb470df42199219e2 P 43999d813e564dc688fad116090c95ca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } }
I20260709 13:23:35.331597 25539 raft_consensus.cc:515] T 89a26dd3ac414f6081b98fa32c2f346f P 9868dce521294d0b816bba7648f4ae5a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } }
I20260709 13:23:35.332340 25505 raft_consensus.cc:385] T f1ac2f4d2fe2424bb470df42199219e2 P 43999d813e564dc688fad116090c95ca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:23:35.335335 25505 raft_consensus.cc:740] T f1ac2f4d2fe2424bb470df42199219e2 P 43999d813e564dc688fad116090c95ca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43999d813e564dc688fad116090c95ca, State: Initialized, Role: FOLLOWER
I20260709 13:23:35.332990 25504 tablet_bootstrap.cc:654] T 91ac744179de48e5bb924b6dfd9f9c9b P 3e07d64466934da4bb3c02262b1c2e38: Neither blocks nor log segments found. Creating new log.
I20260709 13:23:35.334699 25311 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89a26dd3ac414f6081b98fa32c2f346f" candidate_uuid: "9868dce521294d0b816bba7648f4ae5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" is_pre_election: true
I20260709 13:23:35.336053 25503 tablet_bootstrap.cc:654] T 13bb3a23315a41c7a40e10b954b15c8e P 233e7d29aac04505bf4e65f0a39a0c4c: Neither blocks nor log segments found. Creating new log.
W20260709 13:23:35.337423 25425 leader_election.cc:343] T 89a26dd3ac414f6081b98fa32c2f346f P 9868dce521294d0b816bba7648f4ae5a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 233e7d29aac04505bf4e65f0a39a0c4c (127.14.177.130:42957): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:23:35.336073 25508 tablet_bootstrap.cc:492] T a5df750b84e0487fa530161079deb6c2 P 9868dce521294d0b816bba7648f4ae5a: No bootstrap required, opened a new log
I20260709 13:23:35.338651 25508 ts_tablet_manager.cc:1403] T a5df750b84e0487fa530161079deb6c2 P 9868dce521294d0b816bba7648f4ae5a: Time spent bootstrapping tablet: real 0.056s	user 0.010s	sys 0.002s
I20260709 13:23:35.336055 25505 consensus_queue.cc:260] T f1ac2f4d2fe2424bb470df42199219e2 P 43999d813e564dc688fad116090c95ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } }
I20260709 13:23:35.341470 25505 ts_tablet_manager.cc:1434] T f1ac2f4d2fe2424bb470df42199219e2 P 43999d813e564dc688fad116090c95ca: Time spent starting tablet: real 0.017s	user 0.003s	sys 0.003s
I20260709 13:23:35.347275 25385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89a26dd3ac414f6081b98fa32c2f346f" candidate_uuid: "9868dce521294d0b816bba7648f4ae5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e07d64466934da4bb3c02262b1c2e38" is_pre_election: true
I20260709 13:23:35.360009 25385 raft_consensus.cc:2468] T 89a26dd3ac414f6081b98fa32c2f346f P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9868dce521294d0b816bba7648f4ae5a in term 0.
I20260709 13:23:35.360471 25505 tablet_bootstrap.cc:492] T 79d4a12ef58140568c60a24ed4c12f83 P 43999d813e564dc688fad116090c95ca: Bootstrap starting.
I20260709 13:23:35.362954 25422 leader_election.cc:304] T 89a26dd3ac414f6081b98fa32c2f346f P 9868dce521294d0b816bba7648f4ae5a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3e07d64466934da4bb3c02262b1c2e38, 9868dce521294d0b816bba7648f4ae5a; no voters: 233e7d29aac04505bf4e65f0a39a0c4c
I20260709 13:23:35.374763 25505 tablet_bootstrap.cc:654] T 79d4a12ef58140568c60a24ed4c12f83 P 43999d813e564dc688fad116090c95ca: Neither blocks nor log segments found. Creating new log.
I20260709 13:23:35.376444 25644 raft_consensus.cc:2804] T 89a26dd3ac414f6081b98fa32c2f346f P 9868dce521294d0b816bba7648f4ae5a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:23:35.377179 25644 raft_consensus.cc:493] T 89a26dd3ac414f6081b98fa32c2f346f P 9868dce521294d0b816bba7648f4ae5a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:23:35.377691 25644 raft_consensus.cc:3060] T 89a26dd3ac414f6081b98fa32c2f346f P 9868dce521294d0b816bba7648f4ae5a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:23:35.381399 25539 leader_election.cc:290] T 89a26dd3ac414f6081b98fa32c2f346f P 9868dce521294d0b816bba7648f4ae5a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 233e7d29aac04505bf4e65f0a39a0c4c (127.14.177.130:42957), 3e07d64466934da4bb3c02262b1c2e38 (127.14.177.131:41539)
I20260709 13:23:35.381646 25686 consensus_queue.cc:1048] T 0cb89d70b4ce4aa8992fb68480bec8a2 P 3e07d64466934da4bb3c02262b1c2e38 [LEADER]: Connected to new peer: Peer: permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:23:35.407830 25311 raft_consensus.cc:3060] T ad32097cd4dc48b9a69595f1651abd07 P 233e7d29aac04505bf4e65f0a39a0c4c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:23:35.350929 25508 raft_consensus.cc:359] T a5df750b84e0487fa530161079deb6c2 P 9868dce521294d0b816bba7648f4ae5a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } }
I20260709 13:23:35.436128 25508 raft_consensus.cc:385] T a5df750b84e0487fa530161079deb6c2 P 9868dce521294d0b816bba7648f4ae5a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:23:35.436622 25508 raft_consensus.cc:740] T a5df750b84e0487fa530161079deb6c2 P 9868dce521294d0b816bba7648f4ae5a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9868dce521294d0b816bba7648f4ae5a, State: Initialized, Role: FOLLOWER
I20260709 13:23:35.442816 25505 tablet_bootstrap.cc:492] T 79d4a12ef58140568c60a24ed4c12f83 P 43999d813e564dc688fad116090c95ca: No bootstrap required, opened a new log
I20260709 13:23:35.447547 25508 consensus_queue.cc:260] T a5df750b84e0487fa530161079deb6c2 P 9868dce521294d0b816bba7648f4ae5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } }
I20260709 13:23:35.465412 25505 ts_tablet_manager.cc:1403] T 79d4a12ef58140568c60a24ed4c12f83 P 43999d813e564dc688fad116090c95ca: Time spent bootstrapping tablet: real 0.105s	user 0.008s	sys 0.016s
I20260709 13:23:35.455395 25622 consensus_queue.cc:1048] T 0cb89d70b4ce4aa8992fb68480bec8a2 P 3e07d64466934da4bb3c02262b1c2e38 [LEADER]: Connected to new peer: Peer: permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:23:35.466871 25503 tablet_bootstrap.cc:492] T 13bb3a23315a41c7a40e10b954b15c8e P 233e7d29aac04505bf4e65f0a39a0c4c: No bootstrap required, opened a new log
I20260709 13:23:35.469959 25503 ts_tablet_manager.cc:1403] T 13bb3a23315a41c7a40e10b954b15c8e P 233e7d29aac04505bf4e65f0a39a0c4c: Time spent bootstrapping tablet: real 0.146s	user 0.049s	sys 0.068s
I20260709 13:23:35.466979 25508 ts_tablet_manager.cc:1434] T a5df750b84e0487fa530161079deb6c2 P 9868dce521294d0b816bba7648f4ae5a: Time spent starting tablet: real 0.128s	user 0.003s	sys 0.005s
I20260709 13:23:35.473439 25508 tablet_bootstrap.cc:492] T f1ac2f4d2fe2424bb470df42199219e2 P 9868dce521294d0b816bba7648f4ae5a: Bootstrap starting.
I20260709 13:23:35.473152 25503 raft_consensus.cc:359] T 13bb3a23315a41c7a40e10b954b15c8e P 233e7d29aac04505bf4e65f0a39a0c4c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } }
I20260709 13:23:35.473930 25503 raft_consensus.cc:385] T 13bb3a23315a41c7a40e10b954b15c8e P 233e7d29aac04505bf4e65f0a39a0c4c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:23:35.474233 25503 raft_consensus.cc:740] T 13bb3a23315a41c7a40e10b954b15c8e P 233e7d29aac04505bf4e65f0a39a0c4c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 233e7d29aac04505bf4e65f0a39a0c4c, State: Initialized, Role: FOLLOWER
I20260709 13:23:35.451329 25644 raft_consensus.cc:515] T 89a26dd3ac414f6081b98fa32c2f346f P 9868dce521294d0b816bba7648f4ae5a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } }
I20260709 13:23:35.467666 25504 tablet_bootstrap.cc:492] T 91ac744179de48e5bb924b6dfd9f9c9b P 3e07d64466934da4bb3c02262b1c2e38: No bootstrap required, opened a new log
I20260709 13:23:35.476920 25504 ts_tablet_manager.cc:1403] T 91ac744179de48e5bb924b6dfd9f9c9b P 3e07d64466934da4bb3c02262b1c2e38: Time spent bootstrapping tablet: real 0.150s	user 0.039s	sys 0.044s
I20260709 13:23:35.477600 25309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89a26dd3ac414f6081b98fa32c2f346f" candidate_uuid: "9868dce521294d0b816bba7648f4ae5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "233e7d29aac04505bf4e65f0a39a0c4c"
W20260709 13:23:35.479462 25425 leader_election.cc:343] T 89a26dd3ac414f6081b98fa32c2f346f P 9868dce521294d0b816bba7648f4ae5a [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 233e7d29aac04505bf4e65f0a39a0c4c (127.14.177.130:42957): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:23:35.479957 25504 raft_consensus.cc:359] T 91ac744179de48e5bb924b6dfd9f9c9b P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } }
I20260709 13:23:35.475088 25503 consensus_queue.cc:260] T 13bb3a23315a41c7a40e10b954b15c8e P 233e7d29aac04505bf4e65f0a39a0c4c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } }
I20260709 13:23:35.481297 25673 raft_consensus.cc:493] T ad32097cd4dc48b9a69595f1651abd07 P 233e7d29aac04505bf4e65f0a39a0c4c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3e07d64466934da4bb3c02262b1c2e38)
I20260709 13:23:35.486776 25673 raft_consensus.cc:515] T ad32097cd4dc48b9a69595f1651abd07 P 233e7d29aac04505bf4e65f0a39a0c4c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } }
I20260709 13:23:35.489213 25508 tablet_bootstrap.cc:654] T f1ac2f4d2fe2424bb470df42199219e2 P 9868dce521294d0b816bba7648f4ae5a: Neither blocks nor log segments found. Creating new log.
I20260709 13:23:35.486263 25503 ts_tablet_manager.cc:1434] T 13bb3a23315a41c7a40e10b954b15c8e P 233e7d29aac04505bf4e65f0a39a0c4c: Time spent starting tablet: real 0.016s	user 0.006s	sys 0.000s
I20260709 13:23:35.491813 25686 raft_consensus.cc:493] T 79d4a12ef58140568c60a24ed4c12f83 P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:23:35.492373 25503 tablet_bootstrap.cc:492] T 89a26dd3ac414f6081b98fa32c2f346f P 233e7d29aac04505bf4e65f0a39a0c4c: Bootstrap starting.
I20260709 13:23:35.492436 25686 raft_consensus.cc:515] T 79d4a12ef58140568c60a24ed4c12f83 P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } }
I20260709 13:23:35.492918 25455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad32097cd4dc48b9a69595f1651abd07" candidate_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9868dce521294d0b816bba7648f4ae5a" is_pre_election: true
I20260709 13:23:35.477680 25505 raft_consensus.cc:359] T 79d4a12ef58140568c60a24ed4c12f83 P 43999d813e564dc688fad116090c95ca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } }
I20260709 13:23:35.495590 25505 raft_consensus.cc:385] T 79d4a12ef58140568c60a24ed4c12f83 P 43999d813e564dc688fad116090c95ca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:23:35.496042 25505 raft_consensus.cc:740] T 79d4a12ef58140568c60a24ed4c12f83 P 43999d813e564dc688fad116090c95ca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43999d813e564dc688fad116090c95ca, State: Initialized, Role: FOLLOWER
I20260709 13:23:35.496663 25505 consensus_queue.cc:260] T 79d4a12ef58140568c60a24ed4c12f83 P 43999d813e564dc688fad116090c95ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } }
I20260709 13:23:35.480748 25504 raft_consensus.cc:385] T 91ac744179de48e5bb924b6dfd9f9c9b P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:23:35.497911 25504 raft_consensus.cc:740] T 91ac744179de48e5bb924b6dfd9f9c9b P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e07d64466934da4bb3c02262b1c2e38, State: Initialized, Role: FOLLOWER
I20260709 13:23:35.500926 25505 ts_tablet_manager.cc:1434] T 79d4a12ef58140568c60a24ed4c12f83 P 43999d813e564dc688fad116090c95ca: Time spent starting tablet: real 0.033s	user 0.005s	sys 0.001s
I20260709 13:23:35.495556 25673 leader_election.cc:290] T ad32097cd4dc48b9a69595f1651abd07 P 233e7d29aac04505bf4e65f0a39a0c4c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3e07d64466934da4bb3c02262b1c2e38 (127.14.177.131:41539), 9868dce521294d0b816bba7648f4ae5a (127.14.177.132:33683)
I20260709 13:23:35.501233 25686 leader_election.cc:290] T 79d4a12ef58140568c60a24ed4c12f83 P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 43999d813e564dc688fad116090c95ca (127.14.177.129:38155), 233e7d29aac04505bf4e65f0a39a0c4c (127.14.177.130:42957)
I20260709 13:23:35.502095 25505 tablet_bootstrap.cc:492] T 91ac744179de48e5bb924b6dfd9f9c9b P 43999d813e564dc688fad116090c95ca: Bootstrap starting.
I20260709 13:23:35.485750 25644 leader_election.cc:290] T 89a26dd3ac414f6081b98fa32c2f346f P 9868dce521294d0b816bba7648f4ae5a [CANDIDATE]: Term 1 election: Requested vote from peers 233e7d29aac04505bf4e65f0a39a0c4c (127.14.177.130:42957), 3e07d64466934da4bb3c02262b1c2e38 (127.14.177.131:41539)
I20260709 13:23:35.506065 25503 tablet_bootstrap.cc:654] T 89a26dd3ac414f6081b98fa32c2f346f P 233e7d29aac04505bf4e65f0a39a0c4c: Neither blocks nor log segments found. Creating new log.
I20260709 13:23:35.507639 25385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad32097cd4dc48b9a69595f1651abd07" candidate_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3e07d64466934da4bb3c02262b1c2e38" is_pre_election: true
I20260709 13:23:35.509147 25383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89a26dd3ac414f6081b98fa32c2f346f" candidate_uuid: "9868dce521294d0b816bba7648f4ae5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e07d64466934da4bb3c02262b1c2e38"
I20260709 13:23:35.509824 25383 raft_consensus.cc:3060] T 89a26dd3ac414f6081b98fa32c2f346f P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:23:35.510484 25505 tablet_bootstrap.cc:654] T 91ac744179de48e5bb924b6dfd9f9c9b P 43999d813e564dc688fad116090c95ca: Neither blocks nor log segments found. Creating new log.
I20260709 13:23:35.512117 25559 raft_consensus.cc:493] T 13bb3a23315a41c7a40e10b954b15c8e P 43999d813e564dc688fad116090c95ca [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:23:35.512575 25559 raft_consensus.cc:515] T 13bb3a23315a41c7a40e10b954b15c8e P 43999d813e564dc688fad116090c95ca [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } }
I20260709 13:23:35.503906 25504 consensus_queue.cc:260] T 91ac744179de48e5bb924b6dfd9f9c9b P 3e07d64466934da4bb3c02262b1c2e38 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } }
I20260709 13:23:35.514575 25559 leader_election.cc:290] T 13bb3a23315a41c7a40e10b954b15c8e P 43999d813e564dc688fad116090c95ca [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 233e7d29aac04505bf4e65f0a39a0c4c (127.14.177.130:42957), 3e07d64466934da4bb3c02262b1c2e38 (127.14.177.131:41539)
I20260709 13:23:35.515929 25504 ts_tablet_manager.cc:1434] T 91ac744179de48e5bb924b6dfd9f9c9b P 3e07d64466934da4bb3c02262b1c2e38: Time spent starting tablet: real 0.039s	user 0.007s	sys 0.000s
I20260709 13:23:35.516245 25309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13bb3a23315a41c7a40e10b954b15c8e" candidate_uuid: "43999d813e564dc688fad116090c95ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" is_pre_election: true
I20260709 13:23:35.516992 25504 tablet_bootstrap.cc:492] T 434a18a5109e4ea78349ded3968b2709 P 3e07d64466934da4bb3c02262b1c2e38: Bootstrap starting.
I20260709 13:23:35.516908 25309 raft_consensus.cc:2468] T 13bb3a23315a41c7a40e10b954b15c8e P 233e7d29aac04505bf4e65f0a39a0c4c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 43999d813e564dc688fad116090c95ca in term 0.
I20260709 13:23:35.518282 25202 leader_election.cc:304] T 13bb3a23315a41c7a40e10b954b15c8e P 43999d813e564dc688fad116090c95ca [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 233e7d29aac04505bf4e65f0a39a0c4c, 43999d813e564dc688fad116090c95ca; no voters: 
I20260709 13:23:35.519155 25559 raft_consensus.cc:2804] T 13bb3a23315a41c7a40e10b954b15c8e P 43999d813e564dc688fad116090c95ca [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:23:35.519614 25559 raft_consensus.cc:493] T 13bb3a23315a41c7a40e10b954b15c8e P 43999d813e564dc688fad116090c95ca [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:23:35.520139 25602 raft_consensus.cc:493] T 1550ed2d2a764c08adffae679599ba41 P 43999d813e564dc688fad116090c95ca [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:23:35.520627 25602 raft_consensus.cc:515] T 1550ed2d2a764c08adffae679599ba41 P 43999d813e564dc688fad116090c95ca [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } }
I20260709 13:23:35.522763 25602 leader_election.cc:290] T 1550ed2d2a764c08adffae679599ba41 P 43999d813e564dc688fad116090c95ca [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3e07d64466934da4bb3c02262b1c2e38 (127.14.177.131:41539), 9868dce521294d0b816bba7648f4ae5a (127.14.177.132:33683)
I20260709 13:23:35.523638 25234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79d4a12ef58140568c60a24ed4c12f83" candidate_uuid: "3e07d64466934da4bb3c02262b1c2e38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43999d813e564dc688fad116090c95ca" is_pre_election: true
I20260709 13:23:35.524592 25234 raft_consensus.cc:2468] T 79d4a12ef58140568c60a24ed4c12f83 P 43999d813e564dc688fad116090c95ca [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3e07d64466934da4bb3c02262b1c2e38 in term 0.
I20260709 13:23:35.525660 25559 raft_consensus.cc:3060] T 13bb3a23315a41c7a40e10b954b15c8e P 43999d813e564dc688fad116090c95ca [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:23:35.529886 25504 tablet_bootstrap.cc:654] T 434a18a5109e4ea78349ded3968b2709 P 3e07d64466934da4bb3c02262b1c2e38: Neither blocks nor log segments found. Creating new log.
I20260709 13:23:35.526666 25602 raft_consensus.cc:493] T a5df750b84e0487fa530161079deb6c2 P 43999d813e564dc688fad116090c95ca [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:23:35.530499 25274 leader_election.cc:304] T ad32097cd4dc48b9a69595f1651abd07 P 233e7d29aac04505bf4e65f0a39a0c4c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 233e7d29aac04505bf4e65f0a39a0c4c; no voters: 3e07d64466934da4bb3c02262b1c2e38, 9868dce521294d0b816bba7648f4ae5a
I20260709 13:23:35.530681 25602 raft_consensus.cc:515] T a5df750b84e0487fa530161079deb6c2 P 43999d813e564dc688fad116090c95ca [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } }
I20260709 13:23:35.534898 25310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5df750b84e0487fa530161079deb6c2" candidate_uuid: "43999d813e564dc688fad116090c95ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" is_pre_election: true
I20260709 13:23:35.535907 25310 raft_consensus.cc:2468] T a5df750b84e0487fa530161079deb6c2 P 233e7d29aac04505bf4e65f0a39a0c4c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 43999d813e564dc688fad116090c95ca in term 0.
I20260709 13:23:35.537195 25679 raft_consensus.cc:2749] T ad32097cd4dc48b9a69595f1651abd07 P 233e7d29aac04505bf4e65f0a39a0c4c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:23:35.540290 25505 tablet_bootstrap.cc:492] T 91ac744179de48e5bb924b6dfd9f9c9b P 43999d813e564dc688fad116090c95ca: No bootstrap required, opened a new log
I20260709 13:23:35.540882 25505 ts_tablet_manager.cc:1403] T 91ac744179de48e5bb924b6dfd9f9c9b P 43999d813e564dc688fad116090c95ca: Time spent bootstrapping tablet: real 0.039s	user 0.010s	sys 0.001s
I20260709 13:23:35.541039 25202 leader_election.cc:304] T a5df750b84e0487fa530161079deb6c2 P 43999d813e564dc688fad116090c95ca [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 233e7d29aac04505bf4e65f0a39a0c4c, 43999d813e564dc688fad116090c95ca; no voters: 
I20260709 13:23:35.541898 25385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13bb3a23315a41c7a40e10b954b15c8e" candidate_uuid: "43999d813e564dc688fad116090c95ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e07d64466934da4bb3c02262b1c2e38" is_pre_election: true
I20260709 13:23:35.542207 25601 raft_consensus.cc:2804] T a5df750b84e0487fa530161079deb6c2 P 43999d813e564dc688fad116090c95ca [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:23:35.542640 25385 raft_consensus.cc:2468] T 13bb3a23315a41c7a40e10b954b15c8e P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 43999d813e564dc688fad116090c95ca in term 0.
I20260709 13:23:35.542829 25601 raft_consensus.cc:493] T a5df750b84e0487fa530161079deb6c2 P 43999d813e564dc688fad116090c95ca [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:23:35.543406 25601 raft_consensus.cc:3060] T a5df750b84e0487fa530161079deb6c2 P 43999d813e564dc688fad116090c95ca [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:23:35.544183 25309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79d4a12ef58140568c60a24ed4c12f83" candidate_uuid: "3e07d64466934da4bb3c02262b1c2e38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" is_pre_election: true
W20260709 13:23:35.546968 25351 leader_election.cc:343] T 79d4a12ef58140568c60a24ed4c12f83 P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 233e7d29aac04505bf4e65f0a39a0c4c (127.14.177.130:42957): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:23:35.549571 25505 raft_consensus.cc:359] T 91ac744179de48e5bb924b6dfd9f9c9b P 43999d813e564dc688fad116090c95ca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } }
I20260709 13:23:35.550384 25505 raft_consensus.cc:385] T 91ac744179de48e5bb924b6dfd9f9c9b P 43999d813e564dc688fad116090c95ca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:23:35.550585 25456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5df750b84e0487fa530161079deb6c2" candidate_uuid: "43999d813e564dc688fad116090c95ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9868dce521294d0b816bba7648f4ae5a" is_pre_election: true
I20260709 13:23:35.550899 25505 raft_consensus.cc:740] T 91ac744179de48e5bb924b6dfd9f9c9b P 43999d813e564dc688fad116090c95ca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43999d813e564dc688fad116090c95ca, State: Initialized, Role: FOLLOWER
I20260709 13:23:35.551409 25456 raft_consensus.cc:2468] T a5df750b84e0487fa530161079deb6c2 P 9868dce521294d0b816bba7648f4ae5a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 43999d813e564dc688fad116090c95ca in term 0.
I20260709 13:23:35.551765 25505 consensus_queue.cc:260] T 91ac744179de48e5bb924b6dfd9f9c9b P 43999d813e564dc688fad116090c95ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } }
I20260709 13:23:35.552567 25455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1550ed2d2a764c08adffae679599ba41" candidate_uuid: "43999d813e564dc688fad116090c95ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9868dce521294d0b816bba7648f4ae5a" is_pre_election: true
I20260709 13:23:35.553396 25455 raft_consensus.cc:2468] T 1550ed2d2a764c08adffae679599ba41 P 9868dce521294d0b816bba7648f4ae5a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 43999d813e564dc688fad116090c95ca in term 0.
I20260709 13:23:35.560159 25350 leader_election.cc:304] T 79d4a12ef58140568c60a24ed4c12f83 P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3e07d64466934da4bb3c02262b1c2e38, 43999d813e564dc688fad116090c95ca; no voters: 233e7d29aac04505bf4e65f0a39a0c4c
I20260709 13:23:35.550431 25602 leader_election.cc:290] T a5df750b84e0487fa530161079deb6c2 P 43999d813e564dc688fad116090c95ca [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9868dce521294d0b816bba7648f4ae5a (127.14.177.132:33683), 233e7d29aac04505bf4e65f0a39a0c4c (127.14.177.130:42957)
I20260709 13:23:35.561825 25568 raft_consensus.cc:2804] T 79d4a12ef58140568c60a24ed4c12f83 P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:23:35.562281 25568 raft_consensus.cc:493] T 79d4a12ef58140568c60a24ed4c12f83 P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:23:35.562666 25568 raft_consensus.cc:3060] T 79d4a12ef58140568c60a24ed4c12f83 P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:23:35.564026 25505 ts_tablet_manager.cc:1434] T 91ac744179de48e5bb924b6dfd9f9c9b P 43999d813e564dc688fad116090c95ca: Time spent starting tablet: real 0.023s	user 0.009s	sys 0.000s
I20260709 13:23:35.567183 25601 raft_consensus.cc:515] T a5df750b84e0487fa530161079deb6c2 P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } }
I20260709 13:23:35.570695 25385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1550ed2d2a764c08adffae679599ba41" candidate_uuid: "43999d813e564dc688fad116090c95ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e07d64466934da4bb3c02262b1c2e38" is_pre_election: true
I20260709 13:23:35.571833 25385 raft_consensus.cc:2468] T 1550ed2d2a764c08adffae679599ba41 P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 43999d813e564dc688fad116090c95ca in term 0.
I20260709 13:23:35.573321 25559 raft_consensus.cc:515] T 13bb3a23315a41c7a40e10b954b15c8e P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } }
I20260709 13:23:35.576965 25311 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13bb3a23315a41c7a40e10b954b15c8e" candidate_uuid: "43999d813e564dc688fad116090c95ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "233e7d29aac04505bf4e65f0a39a0c4c"
I20260709 13:23:35.577965 25311 raft_consensus.cc:3060] T 13bb3a23315a41c7a40e10b954b15c8e P 233e7d29aac04505bf4e65f0a39a0c4c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:23:35.578699 25601 leader_election.cc:290] T a5df750b84e0487fa530161079deb6c2 P 43999d813e564dc688fad116090c95ca [CANDIDATE]: Term 1 election: Requested vote from peers 9868dce521294d0b816bba7648f4ae5a (127.14.177.132:33683), 233e7d29aac04505bf4e65f0a39a0c4c (127.14.177.130:42957)
I20260709 13:23:35.575718 25559 leader_election.cc:290] T 13bb3a23315a41c7a40e10b954b15c8e P 43999d813e564dc688fad116090c95ca [CANDIDATE]: Term 1 election: Requested vote from peers 233e7d29aac04505bf4e65f0a39a0c4c (127.14.177.130:42957), 3e07d64466934da4bb3c02262b1c2e38 (127.14.177.131:41539)
I20260709 13:23:35.585646 25309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5df750b84e0487fa530161079deb6c2" candidate_uuid: "43999d813e564dc688fad116090c95ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "233e7d29aac04505bf4e65f0a39a0c4c"
I20260709 13:23:35.591920 25309 raft_consensus.cc:3060] T a5df750b84e0487fa530161079deb6c2 P 233e7d29aac04505bf4e65f0a39a0c4c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:23:35.582630 25383 raft_consensus.cc:2468] T 89a26dd3ac414f6081b98fa32c2f346f P 3e07d64466934da4bb3c02262b1c2e38 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9868dce521294d0b816bba7648f4ae5a in term 1.
I20260709 13:23:35.597141 25573 raft_consensus.cc:493] T 91ac744179de48e5bb924b6dfd9f9c9b P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:23:35.601449 25199 leader_election.cc:304] T 1550ed2d2a764c08adffae679599ba41 P 43999d813e564dc688fad116090c95ca [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e07d64466934da4bb3c02262b1c2e38, 43999d813e564dc688fad116090c95ca; no voters: 
I20260709 13:23:35.606981 25573 raft_consensus.cc:515] T 91ac744179de48e5bb924b6dfd9f9c9b P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } }
I20260709 13:23:35.607846 25422 leader_election.cc:304] T 89a26dd3ac414f6081b98fa32c2f346f P 9868dce521294d0b816bba7648f4ae5a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3e07d64466934da4bb3c02262b1c2e38, 9868dce521294d0b816bba7648f4ae5a; no voters: 233e7d29aac04505bf4e65f0a39a0c4c
I20260709 13:23:35.608266 25559 raft_consensus.cc:2804] T 1550ed2d2a764c08adffae679599ba41 P 43999d813e564dc688fad116090c95ca [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:23:35.608727 25559 raft_consensus.cc:493] T 1550ed2d2a764c08adffae679599ba41 P 43999d813e564dc688fad116090c95ca [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:23:35.609138 25559 raft_consensus.cc:3060] T 1550ed2d2a764c08adffae679599ba41 P 43999d813e564dc688fad116090c95ca [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:23:35.609383 25573 leader_election.cc:290] T 91ac744179de48e5bb924b6dfd9f9c9b P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9868dce521294d0b816bba7648f4ae5a (127.14.177.132:33683), 43999d813e564dc688fad116090c95ca (127.14.177.129:38155)
I20260709 13:23:35.610097 25602 raft_consensus.cc:493] T f1ac2f4d2fe2424bb470df42199219e2 P 43999d813e564dc688fad116090c95ca [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:23:35.610682 25602 raft_consensus.cc:515] T f1ac2f4d2fe2424bb470df42199219e2 P 43999d813e564dc688fad116090c95ca [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } }
I20260709 13:23:35.613087 25602 leader_election.cc:290] T f1ac2f4d2fe2424bb470df42199219e2 P 43999d813e564dc688fad116090c95ca [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3e07d64466934da4bb3c02262b1c2e38 (127.14.177.131:41539), 9868dce521294d0b816bba7648f4ae5a (127.14.177.132:33683)
I20260709 13:23:35.613157 25651 raft_consensus.cc:2804] T 89a26dd3ac414f6081b98fa32c2f346f P 9868dce521294d0b816bba7648f4ae5a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:23:35.613842 25651 raft_consensus.cc:697] T 89a26dd3ac414f6081b98fa32c2f346f P 9868dce521294d0b816bba7648f4ae5a [term 1 LEADER]: Becoming Leader. State: Replica: 9868dce521294d0b816bba7648f4ae5a, State: Running, Role: LEADER
I20260709 13:23:35.614945 25311 raft_consensus.cc:2468] T 13bb3a23315a41c7a40e10b954b15c8e P 233e7d29aac04505bf4e65f0a39a0c4c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 43999d813e564dc688fad116090c95ca in term 1.
I20260709 13:23:35.614853 25651 consensus_queue.cc:237] T 89a26dd3ac414f6081b98fa32c2f346f P 9868dce521294d0b816bba7648f4ae5a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } }
I20260709 13:23:35.616317 25202 leader_election.cc:304] T 13bb3a23315a41c7a40e10b954b15c8e P 43999d813e564dc688fad116090c95ca [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 233e7d29aac04505bf4e65f0a39a0c4c, 43999d813e564dc688fad116090c95ca; no voters: 
I20260709 13:23:35.617280 25601 raft_consensus.cc:2804] T 13bb3a23315a41c7a40e10b954b15c8e P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:23:35.617810 25601 raft_consensus.cc:697] T 13bb3a23315a41c7a40e10b954b15c8e P 43999d813e564dc688fad116090c95ca [term 1 LEADER]: Becoming Leader. State: Replica: 43999d813e564dc688fad116090c95ca, State: Running, Role: LEADER
I20260709 13:23:35.618148 25234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91ac744179de48e5bb924b6dfd9f9c9b" candidate_uuid: "3e07d64466934da4bb3c02262b1c2e38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43999d813e564dc688fad116090c95ca" is_pre_election: true
I20260709 13:23:35.618848 25234 raft_consensus.cc:2468] T 91ac744179de48e5bb924b6dfd9f9c9b P 43999d813e564dc688fad116090c95ca [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3e07d64466934da4bb3c02262b1c2e38 in term 0.
I20260709 13:23:35.618726 25601 consensus_queue.cc:237] T 13bb3a23315a41c7a40e10b954b15c8e P 43999d813e564dc688fad116090c95ca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } }
I20260709 13:23:35.621392 25559 raft_consensus.cc:515] T 1550ed2d2a764c08adffae679599ba41 P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } }
I20260709 13:23:35.622182 25503 tablet_bootstrap.cc:492] T 89a26dd3ac414f6081b98fa32c2f346f P 233e7d29aac04505bf4e65f0a39a0c4c: No bootstrap required, opened a new log
I20260709 13:23:35.622714 25503 ts_tablet_manager.cc:1403] T 89a26dd3ac414f6081b98fa32c2f346f P 233e7d29aac04505bf4e65f0a39a0c4c: Time spent bootstrapping tablet: real 0.131s	user 0.017s	sys 0.012s
I20260709 13:23:35.623737 25559 leader_election.cc:290] T 1550ed2d2a764c08adffae679599ba41 P 43999d813e564dc688fad116090c95ca [CANDIDATE]: Term 1 election: Requested vote from peers 3e07d64466934da4bb3c02262b1c2e38 (127.14.177.131:41539), 9868dce521294d0b816bba7648f4ae5a (127.14.177.132:33683)
I20260709 13:23:35.588582 25505 tablet_bootstrap.cc:492] T 6242888a4d4146dcacd87d0fd9a832f4 P 43999d813e564dc688fad116090c95ca: Bootstrap starting.
I20260709 13:23:35.606060 25508 tablet_bootstrap.cc:492] T f1ac2f4d2fe2424bb470df42199219e2 P 9868dce521294d0b816bba7648f4ae5a: No bootstrap required, opened a new log
I20260709 13:23:35.625873 25503 raft_consensus.cc:359] T 89a26dd3ac414f6081b98fa32c2f346f P 233e7d29aac04505bf4e65f0a39a0c4c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } }
I20260709 13:23:35.606081 25504 tablet_bootstrap.cc:492] T 434a18a5109e4ea78349ded3968b2709 P 3e07d64466934da4bb3c02262b1c2e38: No bootstrap required, opened a new log
I20260709 13:23:35.626892 25504 ts_tablet_manager.cc:1403] T 434a18a5109e4ea78349ded3968b2709 P 3e07d64466934da4bb3c02262b1c2e38: Time spent bootstrapping tablet: real 0.110s	user 0.010s	sys 0.006s
I20260709 13:23:35.626376 25508 ts_tablet_manager.cc:1403] T f1ac2f4d2fe2424bb470df42199219e2 P 9868dce521294d0b816bba7648f4ae5a: Time spent bootstrapping tablet: real 0.153s	user 0.028s	sys 0.017s
I20260709 13:23:35.632117 25508 raft_consensus.cc:359] T f1ac2f4d2fe2424bb470df42199219e2 P 9868dce521294d0b816bba7648f4ae5a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } }
I20260709 13:23:35.633021 25508 raft_consensus.cc:385] T f1ac2f4d2fe2424bb470df42199219e2 P 9868dce521294d0b816bba7648f4ae5a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:23:35.632812 25455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5df750b84e0487fa530161079deb6c2" candidate_uuid: "43999d813e564dc688fad116090c95ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9868dce521294d0b816bba7648f4ae5a"
I20260709 13:23:35.633369 25508 raft_consensus.cc:740] T f1ac2f4d2fe2424bb470df42199219e2 P 9868dce521294d0b816bba7648f4ae5a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9868dce521294d0b816bba7648f4ae5a, State: Initialized, Role: FOLLOWER
I20260709 13:23:35.633615 25455 raft_consensus.cc:3060] T a5df750b84e0487fa530161079deb6c2 P 9868dce521294d0b816bba7648f4ae5a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:23:35.605152 25539 consensus_queue.cc:1048] T 0c4f6b7401db47d59a4f34ed0e8b5350 P 9868dce521294d0b816bba7648f4ae5a [LEADER]: Connected to new peer: Peer: permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:23:35.634298 25508 consensus_queue.cc:260] T f1ac2f4d2fe2424bb470df42199219e2 P 9868dce521294d0b816bba7648f4ae5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } }
I20260709 13:23:35.636634 25503 raft_consensus.cc:385] T 89a26dd3ac414f6081b98fa32c2f346f P 233e7d29aac04505bf4e65f0a39a0c4c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:23:35.636493 25459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1550ed2d2a764c08adffae679599ba41" candidate_uuid: "43999d813e564dc688fad116090c95ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9868dce521294d0b816bba7648f4ae5a"
I20260709 13:23:35.637101 25503 raft_consensus.cc:740] T 89a26dd3ac414f6081b98fa32c2f346f P 233e7d29aac04505bf4e65f0a39a0c4c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 233e7d29aac04505bf4e65f0a39a0c4c, State: Initialized, Role: FOLLOWER
I20260709 13:23:35.591110 25568 raft_consensus.cc:515] T 79d4a12ef58140568c60a24ed4c12f83 P 3e07d64466934da4bb3c02262b1c2e38 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } }
I20260709 13:23:35.636541 25508 ts_tablet_manager.cc:1434] T f1ac2f4d2fe2424bb470df42199219e2 P 9868dce521294d0b816bba7648f4ae5a: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 13:23:35.639194 25505 tablet_bootstrap.cc:654] T 6242888a4d4146dcacd87d0fd9a832f4 P 43999d813e564dc688fad116090c95ca: Neither blocks nor log segments found. Creating new log.
I20260709 13:23:35.638964 25459 raft_consensus.cc:3060] T 1550ed2d2a764c08adffae679599ba41 P 9868dce521294d0b816bba7648f4ae5a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:23:35.640614 25508 tablet_bootstrap.cc:492] T 6242888a4d4146dcacd87d0fd9a832f4 P 9868dce521294d0b816bba7648f4ae5a: Bootstrap starting.
I20260709 13:23:35.641938 25456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1ac2f4d2fe2424bb470df42199219e2" candidate_uuid: "43999d813e564dc688fad116090c95ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9868dce521294d0b816bba7648f4ae5a" is_pre_election: true
I20260709 13:23:35.642655 25456 raft_consensus.cc:2468] T f1ac2f4d2fe2424bb470df42199219e2 P 9868dce521294d0b816bba7648f4ae5a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 43999d813e564dc688fad116090c95ca in term 0.
I20260709 13:23:35.647660 25457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91ac744179de48e5bb924b6dfd9f9c9b" candidate_uuid: "3e07d64466934da4bb3c02262b1c2e38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9868dce521294d0b816bba7648f4ae5a" is_pre_election: true
I20260709 13:23:35.649020 25568 leader_election.cc:290] T 79d4a12ef58140568c60a24ed4c12f83 P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 1 election: Requested vote from peers 43999d813e564dc688fad116090c95ca (127.14.177.129:38155), 233e7d29aac04505bf4e65f0a39a0c4c (127.14.177.130:42957)
I20260709 13:23:35.649461 25199 leader_election.cc:304] T f1ac2f4d2fe2424bb470df42199219e2 P 43999d813e564dc688fad116090c95ca [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 43999d813e564dc688fad116090c95ca, 9868dce521294d0b816bba7648f4ae5a; no voters: 
I20260709 13:23:35.639005 25503 consensus_queue.cc:260] T 89a26dd3ac414f6081b98fa32c2f346f P 233e7d29aac04505bf4e65f0a39a0c4c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } }
I20260709 13:23:35.650786 25559 raft_consensus.cc:2804] T f1ac2f4d2fe2424bb470df42199219e2 P 43999d813e564dc688fad116090c95ca [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:23:35.658298 25559 raft_consensus.cc:493] T f1ac2f4d2fe2424bb470df42199219e2 P 43999d813e564dc688fad116090c95ca [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:23:35.658854 25559 raft_consensus.cc:3060] T f1ac2f4d2fe2424bb470df42199219e2 P 43999d813e564dc688fad116090c95ca [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:23:35.651286 25348 leader_election.cc:343] T 91ac744179de48e5bb924b6dfd9f9c9b P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9868dce521294d0b816bba7648f4ae5a (127.14.177.132:33683): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:23:35.652581 25503 ts_tablet_manager.cc:1434] T 89a26dd3ac414f6081b98fa32c2f346f P 233e7d29aac04505bf4e65f0a39a0c4c: Time spent starting tablet: real 0.030s	user 0.008s	sys 0.001s
I20260709 13:23:35.661299 25503 tablet_bootstrap.cc:492] T bbab48ba177049dcba6de106fd889cfb P 233e7d29aac04505bf4e65f0a39a0c4c: Bootstrap starting.
W20260709 13:23:35.663336 25350 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.129:38155, user_credentials={real_user=slave}} blocked reactor thread for 46677.3us
I20260709 13:23:35.666028 25350 leader_election.cc:304] T 91ac744179de48e5bb924b6dfd9f9c9b P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3e07d64466934da4bb3c02262b1c2e38, 43999d813e564dc688fad116090c95ca; no voters: 9868dce521294d0b816bba7648f4ae5a
I20260709 13:23:35.654461 25508 tablet_bootstrap.cc:654] T 6242888a4d4146dcacd87d0fd9a832f4 P 9868dce521294d0b816bba7648f4ae5a: Neither blocks nor log segments found. Creating new log.
W20260709 13:23:35.654891 25351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.130:42957, user_credentials={real_user=slave}} blocked reactor thread for 48443.8us
I20260709 13:23:35.667143 25503 tablet_bootstrap.cc:654] T bbab48ba177049dcba6de106fd889cfb P 233e7d29aac04505bf4e65f0a39a0c4c: Neither blocks nor log segments found. Creating new log.
I20260709 13:23:35.677661 25311 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79d4a12ef58140568c60a24ed4c12f83" candidate_uuid: "3e07d64466934da4bb3c02262b1c2e38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "233e7d29aac04505bf4e65f0a39a0c4c"
I20260709 13:23:35.672850 25455 raft_consensus.cc:2468] T a5df750b84e0487fa530161079deb6c2 P 9868dce521294d0b816bba7648f4ae5a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 43999d813e564dc688fad116090c95ca in term 1.
W20260709 13:23:35.695353 25351 leader_election.cc:343] T 79d4a12ef58140568c60a24ed4c12f83 P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 233e7d29aac04505bf4e65f0a39a0c4c (127.14.177.130:42957): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:23:35.681593 25384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13bb3a23315a41c7a40e10b954b15c8e" candidate_uuid: "43999d813e564dc688fad116090c95ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e07d64466934da4bb3c02262b1c2e38"
I20260709 13:23:35.696556 25384 raft_consensus.cc:3060] T 13bb3a23315a41c7a40e10b954b15c8e P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:23:35.682842 25235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79d4a12ef58140568c60a24ed4c12f83" candidate_uuid: "3e07d64466934da4bb3c02262b1c2e38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43999d813e564dc688fad116090c95ca"
I20260709 13:23:35.697633 25235 raft_consensus.cc:3060] T 79d4a12ef58140568c60a24ed4c12f83 P 43999d813e564dc688fad116090c95ca [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:23:35.683027 25383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1550ed2d2a764c08adffae679599ba41" candidate_uuid: "43999d813e564dc688fad116090c95ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e07d64466934da4bb3c02262b1c2e38"
I20260709 13:23:35.698715 25383 raft_consensus.cc:3060] T 1550ed2d2a764c08adffae679599ba41 P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:23:35.669548 25504 raft_consensus.cc:359] T 434a18a5109e4ea78349ded3968b2709 P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } }
I20260709 13:23:35.676452 25573 raft_consensus.cc:2804] T 91ac744179de48e5bb924b6dfd9f9c9b P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:23:35.704542 25573 raft_consensus.cc:493] T 91ac744179de48e5bb924b6dfd9f9c9b P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:23:35.704919 25573 raft_consensus.cc:3060] T 91ac744179de48e5bb924b6dfd9f9c9b P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:23:35.672981 25309 raft_consensus.cc:2468] T a5df750b84e0487fa530161079deb6c2 P 233e7d29aac04505bf4e65f0a39a0c4c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 43999d813e564dc688fad116090c95ca in term 1.
I20260709 13:23:35.704067 25504 raft_consensus.cc:385] T 434a18a5109e4ea78349ded3968b2709 P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:23:35.706948 25504 raft_consensus.cc:740] T 434a18a5109e4ea78349ded3968b2709 P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e07d64466934da4bb3c02262b1c2e38, State: Initialized, Role: FOLLOWER
I20260709 13:23:35.707842 25504 consensus_queue.cc:260] T 434a18a5109e4ea78349ded3968b2709 P 3e07d64466934da4bb3c02262b1c2e38 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } }
I20260709 13:23:35.706785 25644 consensus_queue.cc:1048] T 0c4f6b7401db47d59a4f34ed0e8b5350 P 9868dce521294d0b816bba7648f4ae5a [LEADER]: Connected to new peer: Peer: permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:23:35.708637 25383 raft_consensus.cc:2468] T 1550ed2d2a764c08adffae679599ba41 P 3e07d64466934da4bb3c02262b1c2e38 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 43999d813e564dc688fad116090c95ca in term 1.
I20260709 13:23:35.709450 25504 ts_tablet_manager.cc:1434] T 434a18a5109e4ea78349ded3968b2709 P 3e07d64466934da4bb3c02262b1c2e38: Time spent starting tablet: real 0.082s	user 0.006s	sys 0.000s
I20260709 13:23:35.705084 25199 leader_election.cc:304] T a5df750b84e0487fa530161079deb6c2 P 43999d813e564dc688fad116090c95ca [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 43999d813e564dc688fad116090c95ca, 9868dce521294d0b816bba7648f4ae5a; no voters: 
I20260709 13:23:35.710228 25762 consensus_queue.cc:1048] T c902bbc7ec0848ba82a04870eab1e7f5 P 43999d813e564dc688fad116090c95ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:23:35.710525 25602 raft_consensus.cc:2804] T a5df750b84e0487fa530161079deb6c2 P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:23:35.710686 25504 tablet_bootstrap.cc:492] T 0ed5edbe32044d1c83b0590dbb899801 P 3e07d64466934da4bb3c02262b1c2e38: Bootstrap starting.
I20260709 13:23:35.711122 25602 raft_consensus.cc:697] T a5df750b84e0487fa530161079deb6c2 P 43999d813e564dc688fad116090c95ca [term 1 LEADER]: Becoming Leader. State: Replica: 43999d813e564dc688fad116090c95ca, State: Running, Role: LEADER
I20260709 13:23:35.682256 25385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1ac2f4d2fe2424bb470df42199219e2" candidate_uuid: "43999d813e564dc688fad116090c95ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e07d64466934da4bb3c02262b1c2e38" is_pre_election: true
I20260709 13:23:35.712152 25385 raft_consensus.cc:2468] T f1ac2f4d2fe2424bb470df42199219e2 P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 43999d813e564dc688fad116090c95ca in term 0.
I20260709 13:23:35.712164 25602 consensus_queue.cc:237] T a5df750b84e0487fa530161079deb6c2 P 43999d813e564dc688fad116090c95ca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } }
I20260709 13:23:35.714468 25559 raft_consensus.cc:515] T f1ac2f4d2fe2424bb470df42199219e2 P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } }
I20260709 13:23:35.715335 25760 consensus_queue.cc:1048] T c902bbc7ec0848ba82a04870eab1e7f5 P 43999d813e564dc688fad116090c95ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:23:35.704906 25235 raft_consensus.cc:2468] T 79d4a12ef58140568c60a24ed4c12f83 P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3e07d64466934da4bb3c02262b1c2e38 in term 1.
I20260709 13:23:35.721006 25457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1ac2f4d2fe2424bb470df42199219e2" candidate_uuid: "43999d813e564dc688fad116090c95ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9868dce521294d0b816bba7648f4ae5a"
I20260709 13:23:35.721720 25457 raft_consensus.cc:3060] T f1ac2f4d2fe2424bb470df42199219e2 P 9868dce521294d0b816bba7648f4ae5a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:23:35.725821 25573 raft_consensus.cc:515] T 91ac744179de48e5bb924b6dfd9f9c9b P 3e07d64466934da4bb3c02262b1c2e38 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } }
I20260709 13:23:35.726985 25504 tablet_bootstrap.cc:654] T 0ed5edbe32044d1c83b0590dbb899801 P 3e07d64466934da4bb3c02262b1c2e38: Neither blocks nor log segments found. Creating new log.
I20260709 13:23:35.729429 25350 leader_election.cc:304] T 79d4a12ef58140568c60a24ed4c12f83 P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3e07d64466934da4bb3c02262b1c2e38, 43999d813e564dc688fad116090c95ca; no voters: 233e7d29aac04505bf4e65f0a39a0c4c
I20260709 13:23:35.719714 25559 leader_election.cc:290] T f1ac2f4d2fe2424bb470df42199219e2 P 43999d813e564dc688fad116090c95ca [CANDIDATE]: Term 1 election: Requested vote from peers 3e07d64466934da4bb3c02262b1c2e38 (127.14.177.131:41539), 9868dce521294d0b816bba7648f4ae5a (127.14.177.132:33683)
I20260709 13:23:35.730245 25622 raft_consensus.cc:493] T 87085ac2575e4b6a86dd1fc21e6f718b P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:23:35.730773 25686 raft_consensus.cc:2804] T 79d4a12ef58140568c60a24ed4c12f83 P 3e07d64466934da4bb3c02262b1c2e38 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:23:35.730788 25622 raft_consensus.cc:515] T 87085ac2575e4b6a86dd1fc21e6f718b P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } }
I20260709 13:23:35.729023 25199 leader_election.cc:304] T 1550ed2d2a764c08adffae679599ba41 P 43999d813e564dc688fad116090c95ca [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e07d64466934da4bb3c02262b1c2e38, 43999d813e564dc688fad116090c95ca; no voters: 
I20260709 13:23:35.734181 25622 leader_election.cc:290] T 87085ac2575e4b6a86dd1fc21e6f718b P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 43999d813e564dc688fad116090c95ca (127.14.177.129:38155), 233e7d29aac04505bf4e65f0a39a0c4c (127.14.177.130:42957)
I20260709 13:23:35.731395 25686 raft_consensus.cc:697] T 79d4a12ef58140568c60a24ed4c12f83 P 3e07d64466934da4bb3c02262b1c2e38 [term 1 LEADER]: Becoming Leader. State: Replica: 3e07d64466934da4bb3c02262b1c2e38, State: Running, Role: LEADER
I20260709 13:23:35.736027 25686 consensus_queue.cc:237] T 79d4a12ef58140568c60a24ed4c12f83 P 3e07d64466934da4bb3c02262b1c2e38 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } }
I20260709 13:23:35.739042 25456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91ac744179de48e5bb924b6dfd9f9c9b" candidate_uuid: "3e07d64466934da4bb3c02262b1c2e38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9868dce521294d0b816bba7648f4ae5a"
W20260709 13:23:35.741165 25348 leader_election.cc:343] T 91ac744179de48e5bb924b6dfd9f9c9b P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 9868dce521294d0b816bba7648f4ae5a (127.14.177.132:33683): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:23:35.742720 25573 leader_election.cc:290] T 91ac744179de48e5bb924b6dfd9f9c9b P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 1 election: Requested vote from peers 9868dce521294d0b816bba7648f4ae5a (127.14.177.132:33683), 43999d813e564dc688fad116090c95ca (127.14.177.129:38155)
I20260709 13:23:35.670593 25459 raft_consensus.cc:2468] T 1550ed2d2a764c08adffae679599ba41 P 9868dce521294d0b816bba7648f4ae5a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 43999d813e564dc688fad116090c95ca in term 1.
I20260709 13:23:35.745026 25762 raft_consensus.cc:2804] T 1550ed2d2a764c08adffae679599ba41 P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:23:35.766592 25762 raft_consensus.cc:697] T 1550ed2d2a764c08adffae679599ba41 P 43999d813e564dc688fad116090c95ca [term 1 LEADER]: Becoming Leader. State: Replica: 43999d813e564dc688fad116090c95ca, State: Running, Role: LEADER
I20260709 13:23:35.748549 25311 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87085ac2575e4b6a86dd1fc21e6f718b" candidate_uuid: "3e07d64466934da4bb3c02262b1c2e38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" is_pre_election: true
I20260709 13:23:35.767612 25762 consensus_queue.cc:237] T 1550ed2d2a764c08adffae679599ba41 P 43999d813e564dc688fad116090c95ca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } }
W20260709 13:23:35.770170 25351 leader_election.cc:343] T 87085ac2575e4b6a86dd1fc21e6f718b P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 233e7d29aac04505bf4e65f0a39a0c4c (127.14.177.130:42957): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:23:35.749305 25383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1ac2f4d2fe2424bb470df42199219e2" candidate_uuid: "43999d813e564dc688fad116090c95ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e07d64466934da4bb3c02262b1c2e38"
I20260709 13:23:35.755268 25233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91ac744179de48e5bb924b6dfd9f9c9b" candidate_uuid: "3e07d64466934da4bb3c02262b1c2e38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43999d813e564dc688fad116090c95ca"
I20260709 13:23:35.756451 25234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87085ac2575e4b6a86dd1fc21e6f718b" candidate_uuid: "3e07d64466934da4bb3c02262b1c2e38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43999d813e564dc688fad116090c95ca" is_pre_election: true
I20260709 13:23:35.771529 25233 raft_consensus.cc:3060] T 91ac744179de48e5bb924b6dfd9f9c9b P 43999d813e564dc688fad116090c95ca [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:23:35.775449 25383 raft_consensus.cc:3060] T f1ac2f4d2fe2424bb470df42199219e2 P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:23:35.776046 25350 leader_election.cc:343] T 87085ac2575e4b6a86dd1fc21e6f718b P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 43999d813e564dc688fad116090c95ca (127.14.177.129:38155): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:23:35.776526 25350 leader_election.cc:304] T 87085ac2575e4b6a86dd1fc21e6f718b P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3e07d64466934da4bb3c02262b1c2e38; no voters: 233e7d29aac04505bf4e65f0a39a0c4c, 43999d813e564dc688fad116090c95ca
I20260709 13:23:35.759402 25505 tablet_bootstrap.cc:492] T 6242888a4d4146dcacd87d0fd9a832f4 P 43999d813e564dc688fad116090c95ca: No bootstrap required, opened a new log
I20260709 13:23:35.778589 25505 ts_tablet_manager.cc:1403] T 6242888a4d4146dcacd87d0fd9a832f4 P 43999d813e564dc688fad116090c95ca: Time spent bootstrapping tablet: real 0.190s	user 0.019s	sys 0.001s
I20260709 13:23:35.781567 25622 raft_consensus.cc:2749] T 87085ac2575e4b6a86dd1fc21e6f718b P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:23:35.759136 25132 catalog_manager.cc:5697] T 89a26dd3ac414f6081b98fa32c2f346f P 9868dce521294d0b816bba7648f4ae5a reported cstate change: term changed from 0 to 1, leader changed from <none> to 9868dce521294d0b816bba7648f4ae5a (127.14.177.132). New cstate: current_term: 1 leader_uuid: "9868dce521294d0b816bba7648f4ae5a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } health_report { overall_health: HEALTHY } } }
I20260709 13:23:35.784603 25233 raft_consensus.cc:2468] T 91ac744179de48e5bb924b6dfd9f9c9b P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3e07d64466934da4bb3c02262b1c2e38 in term 1.
I20260709 13:23:35.786445 25350 leader_election.cc:304] T 91ac744179de48e5bb924b6dfd9f9c9b P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3e07d64466934da4bb3c02262b1c2e38, 43999d813e564dc688fad116090c95ca; no voters: 9868dce521294d0b816bba7648f4ae5a
I20260709 13:23:35.787338 25503 tablet_bootstrap.cc:492] T bbab48ba177049dcba6de106fd889cfb P 233e7d29aac04505bf4e65f0a39a0c4c: No bootstrap required, opened a new log
I20260709 13:23:35.788012 25503 ts_tablet_manager.cc:1403] T bbab48ba177049dcba6de106fd889cfb P 233e7d29aac04505bf4e65f0a39a0c4c: Time spent bootstrapping tablet: real 0.127s	user 0.008s	sys 0.003s
I20260709 13:23:35.788725 25568 raft_consensus.cc:2804] T 91ac744179de48e5bb924b6dfd9f9c9b P 3e07d64466934da4bb3c02262b1c2e38 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:23:35.789459 25568 raft_consensus.cc:697] T 91ac744179de48e5bb924b6dfd9f9c9b P 3e07d64466934da4bb3c02262b1c2e38 [term 1 LEADER]: Becoming Leader. State: Replica: 3e07d64466934da4bb3c02262b1c2e38, State: Running, Role: LEADER
I20260709 13:23:35.790621 25568 consensus_queue.cc:237] T 91ac744179de48e5bb924b6dfd9f9c9b P 3e07d64466934da4bb3c02262b1c2e38 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } }
I20260709 13:23:35.764199 25384 raft_consensus.cc:2468] T 13bb3a23315a41c7a40e10b954b15c8e P 3e07d64466934da4bb3c02262b1c2e38 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 43999d813e564dc688fad116090c95ca in term 1.
I20260709 13:23:35.781387 25505 raft_consensus.cc:359] T 6242888a4d4146dcacd87d0fd9a832f4 P 43999d813e564dc688fad116090c95ca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } }
I20260709 13:23:35.800486 25505 raft_consensus.cc:385] T 6242888a4d4146dcacd87d0fd9a832f4 P 43999d813e564dc688fad116090c95ca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:23:35.800801 25505 raft_consensus.cc:740] T 6242888a4d4146dcacd87d0fd9a832f4 P 43999d813e564dc688fad116090c95ca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43999d813e564dc688fad116090c95ca, State: Initialized, Role: FOLLOWER
I20260709 13:23:35.801455 25505 consensus_queue.cc:260] T 6242888a4d4146dcacd87d0fd9a832f4 P 43999d813e564dc688fad116090c95ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } }
I20260709 13:23:35.761279 25457 raft_consensus.cc:2468] T f1ac2f4d2fe2424bb470df42199219e2 P 9868dce521294d0b816bba7648f4ae5a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 43999d813e564dc688fad116090c95ca in term 1.
I20260709 13:23:35.804306 25505 ts_tablet_manager.cc:1434] T 6242888a4d4146dcacd87d0fd9a832f4 P 43999d813e564dc688fad116090c95ca: Time spent starting tablet: real 0.025s	user 0.005s	sys 0.001s
I20260709 13:23:35.805344 25505 tablet_bootstrap.cc:492] T 87085ac2575e4b6a86dd1fc21e6f718b P 43999d813e564dc688fad116090c95ca: Bootstrap starting.
I20260709 13:23:35.805043 25199 leader_election.cc:304] T f1ac2f4d2fe2424bb470df42199219e2 P 43999d813e564dc688fad116090c95ca [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 43999d813e564dc688fad116090c95ca, 9868dce521294d0b816bba7648f4ae5a; no voters: 
I20260709 13:23:35.806212 25762 raft_consensus.cc:2804] T f1ac2f4d2fe2424bb470df42199219e2 P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:23:35.806979 25762 raft_consensus.cc:697] T f1ac2f4d2fe2424bb470df42199219e2 P 43999d813e564dc688fad116090c95ca [term 1 LEADER]: Becoming Leader. State: Replica: 43999d813e564dc688fad116090c95ca, State: Running, Role: LEADER
I20260709 13:23:35.810933 25129 catalog_manager.cc:5697] T 13bb3a23315a41c7a40e10b954b15c8e P 43999d813e564dc688fad116090c95ca reported cstate change: term changed from 0 to 1, leader changed from <none> to 43999d813e564dc688fad116090c95ca (127.14.177.129). New cstate: current_term: 1 leader_uuid: "43999d813e564dc688fad116090c95ca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } health_report { overall_health: UNKNOWN } } }
I20260709 13:23:35.812654 25505 tablet_bootstrap.cc:654] T 87085ac2575e4b6a86dd1fc21e6f718b P 43999d813e564dc688fad116090c95ca: Neither blocks nor log segments found. Creating new log.
I20260709 13:23:35.820545 25762 consensus_queue.cc:237] T f1ac2f4d2fe2424bb470df42199219e2 P 43999d813e564dc688fad116090c95ca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } }
I20260709 13:23:35.812944 25130 catalog_manager.cc:5697] T 79d4a12ef58140568c60a24ed4c12f83 P 3e07d64466934da4bb3c02262b1c2e38 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3e07d64466934da4bb3c02262b1c2e38 (127.14.177.131). New cstate: current_term: 1 leader_uuid: "3e07d64466934da4bb3c02262b1c2e38" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } health_report { overall_health: HEALTHY } } }
I20260709 13:23:35.818990 25676 raft_consensus.cc:493] T 89a26dd3ac414f6081b98fa32c2f346f P 233e7d29aac04505bf4e65f0a39a0c4c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:23:35.824481 25676 raft_consensus.cc:515] T 89a26dd3ac414f6081b98fa32c2f346f P 233e7d29aac04505bf4e65f0a39a0c4c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } }
I20260709 13:23:35.826891 25676 leader_election.cc:290] T 89a26dd3ac414f6081b98fa32c2f346f P 233e7d29aac04505bf4e65f0a39a0c4c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3e07d64466934da4bb3c02262b1c2e38 (127.14.177.131:41539), 9868dce521294d0b816bba7648f4ae5a (127.14.177.132:33683)
I20260709 13:23:35.836112 25455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89a26dd3ac414f6081b98fa32c2f346f" candidate_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9868dce521294d0b816bba7648f4ae5a" is_pre_election: true
I20260709 13:23:35.790890 25503 raft_consensus.cc:359] T bbab48ba177049dcba6de106fd889cfb P 233e7d29aac04505bf4e65f0a39a0c4c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } }
I20260709 13:23:35.841815 25503 raft_consensus.cc:385] T bbab48ba177049dcba6de106fd889cfb P 233e7d29aac04505bf4e65f0a39a0c4c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:23:35.842957 25503 raft_consensus.cc:740] T bbab48ba177049dcba6de106fd889cfb P 233e7d29aac04505bf4e65f0a39a0c4c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 233e7d29aac04505bf4e65f0a39a0c4c, State: Initialized, Role: FOLLOWER
I20260709 13:23:35.843812 25503 consensus_queue.cc:260] T bbab48ba177049dcba6de106fd889cfb P 233e7d29aac04505bf4e65f0a39a0c4c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } }
I20260709 13:23:35.841250 25383 raft_consensus.cc:2468] T f1ac2f4d2fe2424bb470df42199219e2 P 3e07d64466934da4bb3c02262b1c2e38 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 43999d813e564dc688fad116090c95ca in term 1.
I20260709 13:23:35.842446 25384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89a26dd3ac414f6081b98fa32c2f346f" candidate_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e07d64466934da4bb3c02262b1c2e38" is_pre_election: true
I20260709 13:23:35.846716 25384 raft_consensus.cc:2393] T 89a26dd3ac414f6081b98fa32c2f346f P 3e07d64466934da4bb3c02262b1c2e38 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 233e7d29aac04505bf4e65f0a39a0c4c in current term 1: Already voted for candidate 9868dce521294d0b816bba7648f4ae5a in this term.
I20260709 13:23:35.849510 25274 leader_election.cc:304] T 89a26dd3ac414f6081b98fa32c2f346f P 233e7d29aac04505bf4e65f0a39a0c4c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 233e7d29aac04505bf4e65f0a39a0c4c; no voters: 3e07d64466934da4bb3c02262b1c2e38, 9868dce521294d0b816bba7648f4ae5a
I20260709 13:23:35.854430 25676 raft_consensus.cc:3060] T 89a26dd3ac414f6081b98fa32c2f346f P 233e7d29aac04505bf4e65f0a39a0c4c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:23:35.855540 25508 tablet_bootstrap.cc:492] T 6242888a4d4146dcacd87d0fd9a832f4 P 9868dce521294d0b816bba7648f4ae5a: No bootstrap required, opened a new log
I20260709 13:23:35.856061 25508 ts_tablet_manager.cc:1403] T 6242888a4d4146dcacd87d0fd9a832f4 P 9868dce521294d0b816bba7648f4ae5a: Time spent bootstrapping tablet: real 0.216s	user 0.015s	sys 0.001s
I20260709 13:23:35.858919 25508 raft_consensus.cc:359] T 6242888a4d4146dcacd87d0fd9a832f4 P 9868dce521294d0b816bba7648f4ae5a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } }
I20260709 13:23:35.859601 25508 raft_consensus.cc:385] T 6242888a4d4146dcacd87d0fd9a832f4 P 9868dce521294d0b816bba7648f4ae5a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:23:35.859881 25508 raft_consensus.cc:740] T 6242888a4d4146dcacd87d0fd9a832f4 P 9868dce521294d0b816bba7648f4ae5a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9868dce521294d0b816bba7648f4ae5a, State: Initialized, Role: FOLLOWER
I20260709 13:23:35.860723 25508 consensus_queue.cc:260] T 6242888a4d4146dcacd87d0fd9a832f4 P 9868dce521294d0b816bba7648f4ae5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } }
I20260709 13:23:35.865881 25508 ts_tablet_manager.cc:1434] T 6242888a4d4146dcacd87d0fd9a832f4 P 9868dce521294d0b816bba7648f4ae5a: Time spent starting tablet: real 0.010s	user 0.006s	sys 0.000s
I20260709 13:23:35.867132 25508 tablet_bootstrap.cc:492] T bbab48ba177049dcba6de106fd889cfb P 9868dce521294d0b816bba7648f4ae5a: Bootstrap starting.
I20260709 13:23:35.867650 25503 ts_tablet_manager.cc:1434] T bbab48ba177049dcba6de106fd889cfb P 233e7d29aac04505bf4e65f0a39a0c4c: Time spent starting tablet: real 0.079s	user 0.008s	sys 0.000s
I20260709 13:23:35.868511 25676 raft_consensus.cc:2749] T 89a26dd3ac414f6081b98fa32c2f346f P 233e7d29aac04505bf4e65f0a39a0c4c [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:23:35.869982 25503 tablet_bootstrap.cc:492] T 79d4a12ef58140568c60a24ed4c12f83 P 233e7d29aac04505bf4e65f0a39a0c4c: Bootstrap starting.
I20260709 13:23:35.875102 25508 tablet_bootstrap.cc:654] T bbab48ba177049dcba6de106fd889cfb P 9868dce521294d0b816bba7648f4ae5a: Neither blocks nor log segments found. Creating new log.
I20260709 13:23:35.881641 25503 tablet_bootstrap.cc:654] T 79d4a12ef58140568c60a24ed4c12f83 P 233e7d29aac04505bf4e65f0a39a0c4c: Neither blocks nor log segments found. Creating new log.
I20260709 13:23:35.883219 25505 tablet_bootstrap.cc:492] T 87085ac2575e4b6a86dd1fc21e6f718b P 43999d813e564dc688fad116090c95ca: No bootstrap required, opened a new log
I20260709 13:23:35.883924 25505 ts_tablet_manager.cc:1403] T 87085ac2575e4b6a86dd1fc21e6f718b P 43999d813e564dc688fad116090c95ca: Time spent bootstrapping tablet: real 0.079s	user 0.021s	sys 0.018s
I20260709 13:23:35.886878 25508 tablet_bootstrap.cc:492] T bbab48ba177049dcba6de106fd889cfb P 9868dce521294d0b816bba7648f4ae5a: No bootstrap required, opened a new log
I20260709 13:23:35.887462 25508 ts_tablet_manager.cc:1403] T bbab48ba177049dcba6de106fd889cfb P 9868dce521294d0b816bba7648f4ae5a: Time spent bootstrapping tablet: real 0.021s	user 0.011s	sys 0.006s
I20260709 13:23:35.894279 25504 tablet_bootstrap.cc:492] T 0ed5edbe32044d1c83b0590dbb899801 P 3e07d64466934da4bb3c02262b1c2e38: No bootstrap required, opened a new log
I20260709 13:23:35.894837 25504 ts_tablet_manager.cc:1403] T 0ed5edbe32044d1c83b0590dbb899801 P 3e07d64466934da4bb3c02262b1c2e38: Time spent bootstrapping tablet: real 0.184s	user 0.025s	sys 0.020s
I20260709 13:23:35.897276 25508 raft_consensus.cc:359] T bbab48ba177049dcba6de106fd889cfb P 9868dce521294d0b816bba7648f4ae5a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } }
I20260709 13:23:35.898065 25508 raft_consensus.cc:385] T bbab48ba177049dcba6de106fd889cfb P 9868dce521294d0b816bba7648f4ae5a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:23:35.898332 25508 raft_consensus.cc:740] T bbab48ba177049dcba6de106fd889cfb P 9868dce521294d0b816bba7648f4ae5a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9868dce521294d0b816bba7648f4ae5a, State: Initialized, Role: FOLLOWER
I20260709 13:23:35.900003 25504 raft_consensus.cc:359] T 0ed5edbe32044d1c83b0590dbb899801 P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } }
I20260709 13:23:35.900787 25504 raft_consensus.cc:385] T 0ed5edbe32044d1c83b0590dbb899801 P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:23:35.901067 25504 raft_consensus.cc:740] T 0ed5edbe32044d1c83b0590dbb899801 P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e07d64466934da4bb3c02262b1c2e38, State: Initialized, Role: FOLLOWER
I20260709 13:23:35.901860 25508 consensus_queue.cc:260] T bbab48ba177049dcba6de106fd889cfb P 9868dce521294d0b816bba7648f4ae5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } }
I20260709 13:23:35.903306 25504 consensus_queue.cc:260] T 0ed5edbe32044d1c83b0590dbb899801 P 3e07d64466934da4bb3c02262b1c2e38 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } }
I20260709 13:23:35.893383 25505 raft_consensus.cc:359] T 87085ac2575e4b6a86dd1fc21e6f718b P 43999d813e564dc688fad116090c95ca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } }
I20260709 13:23:35.907564 25505 raft_consensus.cc:385] T 87085ac2575e4b6a86dd1fc21e6f718b P 43999d813e564dc688fad116090c95ca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:23:35.911191 25505 raft_consensus.cc:740] T 87085ac2575e4b6a86dd1fc21e6f718b P 43999d813e564dc688fad116090c95ca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43999d813e564dc688fad116090c95ca, State: Initialized, Role: FOLLOWER
I20260709 13:23:35.908749 25508 ts_tablet_manager.cc:1434] T bbab48ba177049dcba6de106fd889cfb P 9868dce521294d0b816bba7648f4ae5a: Time spent starting tablet: real 0.021s	user 0.005s	sys 0.001s
I20260709 13:23:35.910269 25568 raft_consensus.cc:493] T 434a18a5109e4ea78349ded3968b2709 P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:23:35.911991 25505 consensus_queue.cc:260] T 87085ac2575e4b6a86dd1fc21e6f718b P 43999d813e564dc688fad116090c95ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } }
I20260709 13:23:35.912022 25568 raft_consensus.cc:515] T 434a18a5109e4ea78349ded3968b2709 P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } }
I20260709 13:23:35.914441 25505 ts_tablet_manager.cc:1434] T 87085ac2575e4b6a86dd1fc21e6f718b P 43999d813e564dc688fad116090c95ca: Time spent starting tablet: real 0.030s	user 0.006s	sys 0.000s
I20260709 13:23:35.914592 25568 leader_election.cc:290] T 434a18a5109e4ea78349ded3968b2709 P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 233e7d29aac04505bf4e65f0a39a0c4c (127.14.177.130:42957), 9868dce521294d0b816bba7648f4ae5a (127.14.177.132:33683)
I20260709 13:23:35.913823 25129 catalog_manager.cc:5697] T a5df750b84e0487fa530161079deb6c2 P 43999d813e564dc688fad116090c95ca reported cstate change: term changed from 0 to 1, leader changed from <none> to 43999d813e564dc688fad116090c95ca (127.14.177.129). New cstate: current_term: 1 leader_uuid: "43999d813e564dc688fad116090c95ca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } health_report { overall_health: UNKNOWN } } }
I20260709 13:23:35.915496 25505 tablet_bootstrap.cc:492] T ce9c60e3922c4219b6e4453fe0259b6a P 43999d813e564dc688fad116090c95ca: Bootstrap starting.
I20260709 13:23:35.916491 25309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434a18a5109e4ea78349ded3968b2709" candidate_uuid: "3e07d64466934da4bb3c02262b1c2e38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" is_pre_election: true
I20260709 13:23:35.916313 25129 catalog_manager.cc:5697] T 1550ed2d2a764c08adffae679599ba41 P 43999d813e564dc688fad116090c95ca reported cstate change: term changed from 0 to 1, leader changed from <none> to 43999d813e564dc688fad116090c95ca (127.14.177.129). New cstate: current_term: 1 leader_uuid: "43999d813e564dc688fad116090c95ca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } health_report { overall_health: HEALTHY } } }
W20260709 13:23:35.918465 25351 leader_election.cc:343] T 434a18a5109e4ea78349ded3968b2709 P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 233e7d29aac04505bf4e65f0a39a0c4c (127.14.177.130:42957): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:23:35.918187 25129 catalog_manager.cc:5697] T f1ac2f4d2fe2424bb470df42199219e2 P 43999d813e564dc688fad116090c95ca reported cstate change: term changed from 0 to 1, leader changed from <none> to 43999d813e564dc688fad116090c95ca (127.14.177.129). New cstate: current_term: 1 leader_uuid: "43999d813e564dc688fad116090c95ca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } health_report { overall_health: HEALTHY } } }
I20260709 13:23:35.922158 25505 tablet_bootstrap.cc:654] T ce9c60e3922c4219b6e4453fe0259b6a P 43999d813e564dc688fad116090c95ca: Neither blocks nor log segments found. Creating new log.
I20260709 13:23:35.922529 25503 tablet_bootstrap.cc:492] T 79d4a12ef58140568c60a24ed4c12f83 P 233e7d29aac04505bf4e65f0a39a0c4c: No bootstrap required, opened a new log
I20260709 13:23:35.923126 25503 ts_tablet_manager.cc:1403] T 79d4a12ef58140568c60a24ed4c12f83 P 233e7d29aac04505bf4e65f0a39a0c4c: Time spent bootstrapping tablet: real 0.053s	user 0.023s	sys 0.015s
I20260709 13:23:35.926666 25503 raft_consensus.cc:359] T 79d4a12ef58140568c60a24ed4c12f83 P 233e7d29aac04505bf4e65f0a39a0c4c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } }
I20260709 13:23:35.927836 25504 ts_tablet_manager.cc:1434] T 0ed5edbe32044d1c83b0590dbb899801 P 3e07d64466934da4bb3c02262b1c2e38: Time spent starting tablet: real 0.033s	user 0.008s	sys 0.000s
I20260709 13:23:35.927783 25503 raft_consensus.cc:385] T 79d4a12ef58140568c60a24ed4c12f83 P 233e7d29aac04505bf4e65f0a39a0c4c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:23:35.928469 25503 raft_consensus.cc:740] T 79d4a12ef58140568c60a24ed4c12f83 P 233e7d29aac04505bf4e65f0a39a0c4c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 233e7d29aac04505bf4e65f0a39a0c4c, State: Initialized, Role: FOLLOWER
I20260709 13:23:35.928709 25457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434a18a5109e4ea78349ded3968b2709" candidate_uuid: "3e07d64466934da4bb3c02262b1c2e38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9868dce521294d0b816bba7648f4ae5a" is_pre_election: true
I20260709 13:23:35.929530 25508 tablet_bootstrap.cc:492] T ce9c60e3922c4219b6e4453fe0259b6a P 9868dce521294d0b816bba7648f4ae5a: Bootstrap starting.
I20260709 13:23:35.929378 25503 consensus_queue.cc:260] T 79d4a12ef58140568c60a24ed4c12f83 P 233e7d29aac04505bf4e65f0a39a0c4c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } }
W20260709 13:23:35.930212 25348 leader_election.cc:343] T 434a18a5109e4ea78349ded3968b2709 P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9868dce521294d0b816bba7648f4ae5a (127.14.177.132:33683): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:23:35.930749 25348 leader_election.cc:304] T 434a18a5109e4ea78349ded3968b2709 P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3e07d64466934da4bb3c02262b1c2e38; no voters: 233e7d29aac04505bf4e65f0a39a0c4c, 9868dce521294d0b816bba7648f4ae5a
I20260709 13:23:35.931999 25503 ts_tablet_manager.cc:1434] T 79d4a12ef58140568c60a24ed4c12f83 P 233e7d29aac04505bf4e65f0a39a0c4c: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 13:23:35.932833 25568 raft_consensus.cc:2749] T 434a18a5109e4ea78349ded3968b2709 P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:23:35.936407 25508 tablet_bootstrap.cc:654] T ce9c60e3922c4219b6e4453fe0259b6a P 9868dce521294d0b816bba7648f4ae5a: Neither blocks nor log segments found. Creating new log.
I20260709 13:23:35.938203 25505 tablet_bootstrap.cc:492] T ce9c60e3922c4219b6e4453fe0259b6a P 43999d813e564dc688fad116090c95ca: No bootstrap required, opened a new log
I20260709 13:23:35.938685 25505 ts_tablet_manager.cc:1403] T ce9c60e3922c4219b6e4453fe0259b6a P 43999d813e564dc688fad116090c95ca: Time spent bootstrapping tablet: real 0.023s	user 0.010s	sys 0.006s
I20260709 13:23:35.940227 25132 catalog_manager.cc:5697] T 91ac744179de48e5bb924b6dfd9f9c9b P 3e07d64466934da4bb3c02262b1c2e38 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3e07d64466934da4bb3c02262b1c2e38 (127.14.177.131). New cstate: current_term: 1 leader_uuid: "3e07d64466934da4bb3c02262b1c2e38" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } health_report { overall_health: UNKNOWN } } }
I20260709 13:23:35.941401 25505 raft_consensus.cc:359] T ce9c60e3922c4219b6e4453fe0259b6a P 43999d813e564dc688fad116090c95ca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } }
I20260709 13:23:35.942047 25505 raft_consensus.cc:385] T ce9c60e3922c4219b6e4453fe0259b6a P 43999d813e564dc688fad116090c95ca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:23:35.942287 25505 raft_consensus.cc:740] T ce9c60e3922c4219b6e4453fe0259b6a P 43999d813e564dc688fad116090c95ca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43999d813e564dc688fad116090c95ca, State: Initialized, Role: FOLLOWER
I20260709 13:23:35.942896 25505 consensus_queue.cc:260] T ce9c60e3922c4219b6e4453fe0259b6a P 43999d813e564dc688fad116090c95ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } }
I20260709 13:23:35.944983 25505 ts_tablet_manager.cc:1434] T ce9c60e3922c4219b6e4453fe0259b6a P 43999d813e564dc688fad116090c95ca: Time spent starting tablet: real 0.006s	user 0.003s	sys 0.003s
I20260709 13:23:35.945948 25505 tablet_bootstrap.cc:492] T 0ed5edbe32044d1c83b0590dbb899801 P 43999d813e564dc688fad116090c95ca: Bootstrap starting.
I20260709 13:23:35.951196 25503 tablet_bootstrap.cc:492] T 6242888a4d4146dcacd87d0fd9a832f4 P 233e7d29aac04505bf4e65f0a39a0c4c: Bootstrap starting.
I20260709 13:23:35.957715 25503 tablet_bootstrap.cc:654] T 6242888a4d4146dcacd87d0fd9a832f4 P 233e7d29aac04505bf4e65f0a39a0c4c: Neither blocks nor log segments found. Creating new log.
I20260709 13:23:35.940747 25504 tablet_bootstrap.cc:492] T 4d837697cd0f402cacf9db89a5d92652 P 3e07d64466934da4bb3c02262b1c2e38: Bootstrap starting.
I20260709 13:23:35.968500 25504 tablet_bootstrap.cc:654] T 4d837697cd0f402cacf9db89a5d92652 P 3e07d64466934da4bb3c02262b1c2e38: Neither blocks nor log segments found. Creating new log.
I20260709 13:23:35.978070 25505 tablet_bootstrap.cc:654] T 0ed5edbe32044d1c83b0590dbb899801 P 43999d813e564dc688fad116090c95ca: Neither blocks nor log segments found. Creating new log.
I20260709 13:23:35.979092 25539 raft_consensus.cc:493] T 6242888a4d4146dcacd87d0fd9a832f4 P 9868dce521294d0b816bba7648f4ae5a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:23:35.979707 25539 raft_consensus.cc:515] T 6242888a4d4146dcacd87d0fd9a832f4 P 9868dce521294d0b816bba7648f4ae5a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } }
I20260709 13:23:35.981951 25539 leader_election.cc:290] T 6242888a4d4146dcacd87d0fd9a832f4 P 9868dce521294d0b816bba7648f4ae5a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 43999d813e564dc688fad116090c95ca (127.14.177.129:38155), 233e7d29aac04505bf4e65f0a39a0c4c (127.14.177.130:42957)
I20260709 13:23:35.983155 25503 tablet_bootstrap.cc:492] T 6242888a4d4146dcacd87d0fd9a832f4 P 233e7d29aac04505bf4e65f0a39a0c4c: No bootstrap required, opened a new log
I20260709 13:23:35.983031 25559 raft_consensus.cc:493] T 87085ac2575e4b6a86dd1fc21e6f718b P 43999d813e564dc688fad116090c95ca [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:23:35.985404 25503 ts_tablet_manager.cc:1403] T 6242888a4d4146dcacd87d0fd9a832f4 P 233e7d29aac04505bf4e65f0a39a0c4c: Time spent bootstrapping tablet: real 0.034s	user 0.012s	sys 0.019s
I20260709 13:23:35.985337 25559 raft_consensus.cc:515] T 87085ac2575e4b6a86dd1fc21e6f718b P 43999d813e564dc688fad116090c95ca [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } }
I20260709 13:23:35.986994 25309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6242888a4d4146dcacd87d0fd9a832f4" candidate_uuid: "9868dce521294d0b816bba7648f4ae5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" is_pre_election: true
I20260709 13:23:35.987529 25559 leader_election.cc:290] T 87085ac2575e4b6a86dd1fc21e6f718b P 43999d813e564dc688fad116090c95ca [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 233e7d29aac04505bf4e65f0a39a0c4c (127.14.177.130:42957), 3e07d64466934da4bb3c02262b1c2e38 (127.14.177.131:41539)
W20260709 13:23:35.988509 25425 leader_election.cc:343] T 6242888a4d4146dcacd87d0fd9a832f4 P 9868dce521294d0b816bba7648f4ae5a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 233e7d29aac04505bf4e65f0a39a0c4c (127.14.177.130:42957): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:23:35.984603 25233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6242888a4d4146dcacd87d0fd9a832f4" candidate_uuid: "9868dce521294d0b816bba7648f4ae5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43999d813e564dc688fad116090c95ca" is_pre_election: true
I20260709 13:23:35.990119 25233 raft_consensus.cc:2468] T 6242888a4d4146dcacd87d0fd9a832f4 P 43999d813e564dc688fad116090c95ca [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9868dce521294d0b816bba7648f4ae5a in term 0.
I20260709 13:23:35.990247 25384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87085ac2575e4b6a86dd1fc21e6f718b" candidate_uuid: "43999d813e564dc688fad116090c95ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e07d64466934da4bb3c02262b1c2e38" is_pre_election: true
I20260709 13:23:35.990988 25384 raft_consensus.cc:2468] T 87085ac2575e4b6a86dd1fc21e6f718b P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 43999d813e564dc688fad116090c95ca in term 0.
I20260709 13:23:35.988394 25503 raft_consensus.cc:359] T 6242888a4d4146dcacd87d0fd9a832f4 P 233e7d29aac04505bf4e65f0a39a0c4c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } }
I20260709 13:23:35.992051 25503 raft_consensus.cc:385] T 6242888a4d4146dcacd87d0fd9a832f4 P 233e7d29aac04505bf4e65f0a39a0c4c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:23:35.992388 25503 raft_consensus.cc:740] T 6242888a4d4146dcacd87d0fd9a832f4 P 233e7d29aac04505bf4e65f0a39a0c4c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 233e7d29aac04505bf4e65f0a39a0c4c, State: Initialized, Role: FOLLOWER
I20260709 13:23:35.992182 25424 leader_election.cc:304] T 6242888a4d4146dcacd87d0fd9a832f4 P 9868dce521294d0b816bba7648f4ae5a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 43999d813e564dc688fad116090c95ca, 9868dce521294d0b816bba7648f4ae5a; no voters: 233e7d29aac04505bf4e65f0a39a0c4c
I20260709 13:23:35.993480 25539 raft_consensus.cc:2804] T 6242888a4d4146dcacd87d0fd9a832f4 P 9868dce521294d0b816bba7648f4ae5a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:23:35.993865 25199 leader_election.cc:304] T 87085ac2575e4b6a86dd1fc21e6f718b P 43999d813e564dc688fad116090c95ca [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e07d64466934da4bb3c02262b1c2e38, 43999d813e564dc688fad116090c95ca; no voters: 
I20260709 13:23:35.994522 25508 tablet_bootstrap.cc:492] T ce9c60e3922c4219b6e4453fe0259b6a P 9868dce521294d0b816bba7648f4ae5a: No bootstrap required, opened a new log
I20260709 13:23:35.994915 25505 tablet_bootstrap.cc:492] T 0ed5edbe32044d1c83b0590dbb899801 P 43999d813e564dc688fad116090c95ca: No bootstrap required, opened a new log
I20260709 13:23:35.995048 25508 ts_tablet_manager.cc:1403] T ce9c60e3922c4219b6e4453fe0259b6a P 9868dce521294d0b816bba7648f4ae5a: Time spent bootstrapping tablet: real 0.066s	user 0.015s	sys 0.000s
I20260709 13:23:35.995419 25505 ts_tablet_manager.cc:1403] T 0ed5edbe32044d1c83b0590dbb899801 P 43999d813e564dc688fad116090c95ca: Time spent bootstrapping tablet: real 0.050s	user 0.014s	sys 0.000s
I20260709 13:23:35.995249 25309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87085ac2575e4b6a86dd1fc21e6f718b" candidate_uuid: "43999d813e564dc688fad116090c95ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" is_pre_election: true
W20260709 13:23:35.996773 25202 leader_election.cc:343] T 87085ac2575e4b6a86dd1fc21e6f718b P 43999d813e564dc688fad116090c95ca [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 233e7d29aac04505bf4e65f0a39a0c4c (127.14.177.130:42957): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:23:35.997630 25503 consensus_queue.cc:260] T 6242888a4d4146dcacd87d0fd9a832f4 P 233e7d29aac04505bf4e65f0a39a0c4c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } }
I20260709 13:23:35.997984 25508 raft_consensus.cc:359] T ce9c60e3922c4219b6e4453fe0259b6a P 9868dce521294d0b816bba7648f4ae5a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } }
I20260709 13:23:35.998991 25559 raft_consensus.cc:2804] T 87085ac2575e4b6a86dd1fc21e6f718b P 43999d813e564dc688fad116090c95ca [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:23:35.999464 25559 raft_consensus.cc:493] T 87085ac2575e4b6a86dd1fc21e6f718b P 43999d813e564dc688fad116090c95ca [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:23:35.999817 25559 raft_consensus.cc:3060] T 87085ac2575e4b6a86dd1fc21e6f718b P 43999d813e564dc688fad116090c95ca [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:23:36.000012 25503 ts_tablet_manager.cc:1434] T 6242888a4d4146dcacd87d0fd9a832f4 P 233e7d29aac04505bf4e65f0a39a0c4c: Time spent starting tablet: real 0.014s	user 0.007s	sys 0.000s
I20260709 13:23:36.001053 25503 tablet_bootstrap.cc:492] T 434a18a5109e4ea78349ded3968b2709 P 233e7d29aac04505bf4e65f0a39a0c4c: Bootstrap starting.
I20260709 13:23:35.993870 25539 raft_consensus.cc:493] T 6242888a4d4146dcacd87d0fd9a832f4 P 9868dce521294d0b816bba7648f4ae5a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:23:36.003672 25539 raft_consensus.cc:3060] T 6242888a4d4146dcacd87d0fd9a832f4 P 9868dce521294d0b816bba7648f4ae5a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:23:36.008682 25503 tablet_bootstrap.cc:654] T 434a18a5109e4ea78349ded3968b2709 P 233e7d29aac04505bf4e65f0a39a0c4c: Neither blocks nor log segments found. Creating new log.
I20260709 13:23:35.998618 25508 raft_consensus.cc:385] T ce9c60e3922c4219b6e4453fe0259b6a P 9868dce521294d0b816bba7648f4ae5a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:23:36.011382 25508 raft_consensus.cc:740] T ce9c60e3922c4219b6e4453fe0259b6a P 9868dce521294d0b816bba7648f4ae5a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9868dce521294d0b816bba7648f4ae5a, State: Initialized, Role: FOLLOWER
I20260709 13:23:36.012499 25504 tablet_bootstrap.cc:492] T 4d837697cd0f402cacf9db89a5d92652 P 3e07d64466934da4bb3c02262b1c2e38: No bootstrap required, opened a new log
I20260709 13:23:36.012213 25508 consensus_queue.cc:260] T ce9c60e3922c4219b6e4453fe0259b6a P 9868dce521294d0b816bba7648f4ae5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } }
I20260709 13:23:36.013058 25504 ts_tablet_manager.cc:1403] T 4d837697cd0f402cacf9db89a5d92652 P 3e07d64466934da4bb3c02262b1c2e38: Time spent bootstrapping tablet: real 0.073s	user 0.013s	sys 0.000s
I20260709 13:23:36.014394 25676 raft_consensus.cc:493] T 6242888a4d4146dcacd87d0fd9a832f4 P 233e7d29aac04505bf4e65f0a39a0c4c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:23:36.014700 25508 ts_tablet_manager.cc:1434] T ce9c60e3922c4219b6e4453fe0259b6a P 9868dce521294d0b816bba7648f4ae5a: Time spent starting tablet: real 0.019s	user 0.006s	sys 0.000s
I20260709 13:23:36.014909 25676 raft_consensus.cc:515] T 6242888a4d4146dcacd87d0fd9a832f4 P 233e7d29aac04505bf4e65f0a39a0c4c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } }
I20260709 13:23:36.008601 25505 raft_consensus.cc:359] T 0ed5edbe32044d1c83b0590dbb899801 P 43999d813e564dc688fad116090c95ca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } }
I20260709 13:23:36.017103 25505 raft_consensus.cc:385] T 0ed5edbe32044d1c83b0590dbb899801 P 43999d813e564dc688fad116090c95ca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:23:36.017441 25505 raft_consensus.cc:740] T 0ed5edbe32044d1c83b0590dbb899801 P 43999d813e564dc688fad116090c95ca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43999d813e564dc688fad116090c95ca, State: Initialized, Role: FOLLOWER
I20260709 13:23:36.018214 25505 consensus_queue.cc:260] T 0ed5edbe32044d1c83b0590dbb899801 P 43999d813e564dc688fad116090c95ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } }
I20260709 13:23:36.020258 25505 ts_tablet_manager.cc:1434] T 0ed5edbe32044d1c83b0590dbb899801 P 43999d813e564dc688fad116090c95ca: Time spent starting tablet: real 0.024s	user 0.006s	sys 0.000s
I20260709 13:23:36.016803 25508 tablet_bootstrap.cc:492] T 91ac744179de48e5bb924b6dfd9f9c9b P 9868dce521294d0b816bba7648f4ae5a: Bootstrap starting.
I20260709 13:23:36.021692 25457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6242888a4d4146dcacd87d0fd9a832f4" candidate_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9868dce521294d0b816bba7648f4ae5a" is_pre_election: true
I20260709 13:23:36.024024 25676 leader_election.cc:290] T 6242888a4d4146dcacd87d0fd9a832f4 P 233e7d29aac04505bf4e65f0a39a0c4c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9868dce521294d0b816bba7648f4ae5a (127.14.177.132:33683), 43999d813e564dc688fad116090c95ca (127.14.177.129:38155)
I20260709 13:23:36.026577 25508 tablet_bootstrap.cc:654] T 91ac744179de48e5bb924b6dfd9f9c9b P 9868dce521294d0b816bba7648f4ae5a: Neither blocks nor log segments found. Creating new log.
I20260709 13:23:36.031200 25233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6242888a4d4146dcacd87d0fd9a832f4" candidate_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43999d813e564dc688fad116090c95ca" is_pre_election: true
I20260709 13:23:36.031939 25233 raft_consensus.cc:2468] T 6242888a4d4146dcacd87d0fd9a832f4 P 43999d813e564dc688fad116090c95ca [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 233e7d29aac04505bf4e65f0a39a0c4c in term 0.
I20260709 13:23:36.032891 25504 raft_consensus.cc:359] T 4d837697cd0f402cacf9db89a5d92652 P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } }
I20260709 13:23:36.033442 25276 leader_election.cc:304] T 6242888a4d4146dcacd87d0fd9a832f4 P 233e7d29aac04505bf4e65f0a39a0c4c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 233e7d29aac04505bf4e65f0a39a0c4c, 43999d813e564dc688fad116090c95ca; no voters: 
I20260709 13:23:36.033830 25504 raft_consensus.cc:385] T 4d837697cd0f402cacf9db89a5d92652 P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:23:36.034395 25559 raft_consensus.cc:515] T 87085ac2575e4b6a86dd1fc21e6f718b P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } }
I20260709 13:23:36.034694 25676 raft_consensus.cc:2804] T 6242888a4d4146dcacd87d0fd9a832f4 P 233e7d29aac04505bf4e65f0a39a0c4c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:23:36.034729 25504 raft_consensus.cc:740] T 4d837697cd0f402cacf9db89a5d92652 P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e07d64466934da4bb3c02262b1c2e38, State: Initialized, Role: FOLLOWER
I20260709 13:23:36.035454 25676 raft_consensus.cc:493] T 6242888a4d4146dcacd87d0fd9a832f4 P 233e7d29aac04505bf4e65f0a39a0c4c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:23:36.035962 25676 raft_consensus.cc:3060] T 6242888a4d4146dcacd87d0fd9a832f4 P 233e7d29aac04505bf4e65f0a39a0c4c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:23:36.035980 25504 consensus_queue.cc:260] T 4d837697cd0f402cacf9db89a5d92652 P 3e07d64466934da4bb3c02262b1c2e38 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } }
I20260709 13:23:36.039300 25504 ts_tablet_manager.cc:1434] T 4d837697cd0f402cacf9db89a5d92652 P 3e07d64466934da4bb3c02262b1c2e38: Time spent starting tablet: real 0.026s	user 0.008s	sys 0.000s
I20260709 13:23:36.041124 25504 tablet_bootstrap.cc:492] T 434a54fcee7d4aeeab42f739855c0e8a P 3e07d64466934da4bb3c02262b1c2e38: Bootstrap starting.
I20260709 13:23:36.049384 25505 tablet_bootstrap.cc:492] T 434a54fcee7d4aeeab42f739855c0e8a P 43999d813e564dc688fad116090c95ca: Bootstrap starting.
I20260709 13:23:36.055682 25539 raft_consensus.cc:515] T 6242888a4d4146dcacd87d0fd9a832f4 P 9868dce521294d0b816bba7648f4ae5a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } }
I20260709 13:23:36.057991 25539 leader_election.cc:290] T 6242888a4d4146dcacd87d0fd9a832f4 P 9868dce521294d0b816bba7648f4ae5a [CANDIDATE]: Term 1 election: Requested vote from peers 43999d813e564dc688fad116090c95ca (127.14.177.129:38155), 233e7d29aac04505bf4e65f0a39a0c4c (127.14.177.130:42957)
I20260709 13:23:36.059938 25559 leader_election.cc:290] T 87085ac2575e4b6a86dd1fc21e6f718b P 43999d813e564dc688fad116090c95ca [CANDIDATE]: Term 1 election: Requested vote from peers 233e7d29aac04505bf4e65f0a39a0c4c (127.14.177.130:42957), 3e07d64466934da4bb3c02262b1c2e38 (127.14.177.131:41539)
I20260709 13:23:36.067296 25383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87085ac2575e4b6a86dd1fc21e6f718b" candidate_uuid: "43999d813e564dc688fad116090c95ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e07d64466934da4bb3c02262b1c2e38"
I20260709 13:23:36.067822 25233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6242888a4d4146dcacd87d0fd9a832f4" candidate_uuid: "9868dce521294d0b816bba7648f4ae5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43999d813e564dc688fad116090c95ca"
I20260709 13:23:36.068073 25311 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87085ac2575e4b6a86dd1fc21e6f718b" candidate_uuid: "43999d813e564dc688fad116090c95ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "233e7d29aac04505bf4e65f0a39a0c4c"
I20260709 13:23:36.067965 25383 raft_consensus.cc:3060] T 87085ac2575e4b6a86dd1fc21e6f718b P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:23:36.069621 25202 leader_election.cc:343] T 87085ac2575e4b6a86dd1fc21e6f718b P 43999d813e564dc688fad116090c95ca [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 233e7d29aac04505bf4e65f0a39a0c4c (127.14.177.130:42957): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:23:36.069720 25457 raft_consensus.cc:2393] T 6242888a4d4146dcacd87d0fd9a832f4 P 9868dce521294d0b816bba7648f4ae5a [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 233e7d29aac04505bf4e65f0a39a0c4c in current term 1: Already voted for candidate 9868dce521294d0b816bba7648f4ae5a in this term.
I20260709 13:23:36.070222 25679 raft_consensus.cc:493] T 79d4a12ef58140568c60a24ed4c12f83 P 233e7d29aac04505bf4e65f0a39a0c4c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:23:36.070695 25679 raft_consensus.cc:515] T 79d4a12ef58140568c60a24ed4c12f83 P 233e7d29aac04505bf4e65f0a39a0c4c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } }
I20260709 13:23:36.068480 25233 raft_consensus.cc:3060] T 6242888a4d4146dcacd87d0fd9a832f4 P 43999d813e564dc688fad116090c95ca [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:23:36.072825 25679 leader_election.cc:290] T 79d4a12ef58140568c60a24ed4c12f83 P 233e7d29aac04505bf4e65f0a39a0c4c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 43999d813e564dc688fad116090c95ca (127.14.177.129:38155), 3e07d64466934da4bb3c02262b1c2e38 (127.14.177.131:41539)
I20260709 13:23:36.074875 25235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79d4a12ef58140568c60a24ed4c12f83" candidate_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43999d813e564dc688fad116090c95ca" is_pre_election: true
I20260709 13:23:36.075697 25235 raft_consensus.cc:2393] T 79d4a12ef58140568c60a24ed4c12f83 P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 233e7d29aac04505bf4e65f0a39a0c4c in current term 1: Already voted for candidate 3e07d64466934da4bb3c02262b1c2e38 in this term.
I20260709 13:23:36.080307 25383 raft_consensus.cc:2468] T 87085ac2575e4b6a86dd1fc21e6f718b P 3e07d64466934da4bb3c02262b1c2e38 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 43999d813e564dc688fad116090c95ca in term 1.
I20260709 13:23:36.084097 25385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79d4a12ef58140568c60a24ed4c12f83" candidate_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e07d64466934da4bb3c02262b1c2e38" is_pre_election: true
I20260709 13:23:36.084260 25309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6242888a4d4146dcacd87d0fd9a832f4" candidate_uuid: "9868dce521294d0b816bba7648f4ae5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "233e7d29aac04505bf4e65f0a39a0c4c"
I20260709 13:23:36.085245 25679 raft_consensus.cc:493] T bbab48ba177049dcba6de106fd889cfb P 233e7d29aac04505bf4e65f0a39a0c4c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:23:36.085464 25762 consensus_queue.cc:1048] T 13bb3a23315a41c7a40e10b954b15c8e P 43999d813e564dc688fad116090c95ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:23:36.090927 25679 raft_consensus.cc:515] T bbab48ba177049dcba6de106fd889cfb P 233e7d29aac04505bf4e65f0a39a0c4c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } }
I20260709 13:23:36.091974 25274 leader_election.cc:304] T 79d4a12ef58140568c60a24ed4c12f83 P 233e7d29aac04505bf4e65f0a39a0c4c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 233e7d29aac04505bf4e65f0a39a0c4c; no voters: 3e07d64466934da4bb3c02262b1c2e38, 43999d813e564dc688fad116090c95ca
I20260709 13:23:36.093154 25679 leader_election.cc:290] T bbab48ba177049dcba6de106fd889cfb P 233e7d29aac04505bf4e65f0a39a0c4c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3e07d64466934da4bb3c02262b1c2e38 (127.14.177.131:41539), 9868dce521294d0b816bba7648f4ae5a (127.14.177.132:33683)
I20260709 13:23:36.094157 25679 raft_consensus.cc:3060] T 79d4a12ef58140568c60a24ed4c12f83 P 233e7d29aac04505bf4e65f0a39a0c4c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:23:36.095006 25505 tablet_bootstrap.cc:654] T 434a54fcee7d4aeeab42f739855c0e8a P 43999d813e564dc688fad116090c95ca: Neither blocks nor log segments found. Creating new log.
I20260709 13:23:36.088153 25233 raft_consensus.cc:2468] T 6242888a4d4146dcacd87d0fd9a832f4 P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9868dce521294d0b816bba7648f4ae5a in term 1.
I20260709 13:23:36.089972 25503 tablet_bootstrap.cc:492] T 434a18a5109e4ea78349ded3968b2709 P 233e7d29aac04505bf4e65f0a39a0c4c: No bootstrap required, opened a new log
I20260709 13:23:36.096573 25503 ts_tablet_manager.cc:1403] T 434a18a5109e4ea78349ded3968b2709 P 233e7d29aac04505bf4e65f0a39a0c4c: Time spent bootstrapping tablet: real 0.096s	user 0.016s	sys 0.001s
I20260709 13:23:36.100126 25424 leader_election.cc:304] T 6242888a4d4146dcacd87d0fd9a832f4 P 9868dce521294d0b816bba7648f4ae5a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 43999d813e564dc688fad116090c95ca, 9868dce521294d0b816bba7648f4ae5a; no voters: 
I20260709 13:23:36.101429 25539 raft_consensus.cc:2804] T 6242888a4d4146dcacd87d0fd9a832f4 P 9868dce521294d0b816bba7648f4ae5a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:23:36.102079 25539 raft_consensus.cc:697] T 6242888a4d4146dcacd87d0fd9a832f4 P 9868dce521294d0b816bba7648f4ae5a [term 1 LEADER]: Becoming Leader. State: Replica: 9868dce521294d0b816bba7648f4ae5a, State: Running, Role: LEADER
I20260709 13:23:36.102707 25503 raft_consensus.cc:359] T 434a18a5109e4ea78349ded3968b2709 P 233e7d29aac04505bf4e65f0a39a0c4c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } }
I20260709 13:23:36.104365 25503 raft_consensus.cc:385] T 434a18a5109e4ea78349ded3968b2709 P 233e7d29aac04505bf4e65f0a39a0c4c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:23:36.104624 25503 raft_consensus.cc:740] T 434a18a5109e4ea78349ded3968b2709 P 233e7d29aac04505bf4e65f0a39a0c4c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 233e7d29aac04505bf4e65f0a39a0c4c, State: Initialized, Role: FOLLOWER
I20260709 13:23:36.105244 25508 tablet_bootstrap.cc:492] T 91ac744179de48e5bb924b6dfd9f9c9b P 9868dce521294d0b816bba7648f4ae5a: No bootstrap required, opened a new log
I20260709 13:23:36.105719 25508 ts_tablet_manager.cc:1403] T 91ac744179de48e5bb924b6dfd9f9c9b P 9868dce521294d0b816bba7648f4ae5a: Time spent bootstrapping tablet: real 0.089s	user 0.018s	sys 0.009s
I20260709 13:23:36.105288 25503 consensus_queue.cc:260] T 434a18a5109e4ea78349ded3968b2709 P 233e7d29aac04505bf4e65f0a39a0c4c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } }
I20260709 13:23:36.105943 25539 consensus_queue.cc:237] T 6242888a4d4146dcacd87d0fd9a832f4 P 9868dce521294d0b816bba7648f4ae5a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } }
I20260709 13:23:36.112661 25503 ts_tablet_manager.cc:1434] T 434a18a5109e4ea78349ded3968b2709 P 233e7d29aac04505bf4e65f0a39a0c4c: Time spent starting tablet: real 0.016s	user 0.006s	sys 0.000s
I20260709 13:23:36.101799 25762 consensus_queue.cc:1048] T a5df750b84e0487fa530161079deb6c2 P 43999d813e564dc688fad116090c95ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.005s
I20260709 13:23:36.115587 25504 tablet_bootstrap.cc:654] T 434a54fcee7d4aeeab42f739855c0e8a P 3e07d64466934da4bb3c02262b1c2e38: Neither blocks nor log segments found. Creating new log.
I20260709 13:23:36.122962 25508 raft_consensus.cc:359] T 91ac744179de48e5bb924b6dfd9f9c9b P 9868dce521294d0b816bba7648f4ae5a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } }
I20260709 13:23:36.123759 25508 raft_consensus.cc:385] T 91ac744179de48e5bb924b6dfd9f9c9b P 9868dce521294d0b816bba7648f4ae5a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:23:36.124032 25508 raft_consensus.cc:740] T 91ac744179de48e5bb924b6dfd9f9c9b P 9868dce521294d0b816bba7648f4ae5a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9868dce521294d0b816bba7648f4ae5a, State: Initialized, Role: FOLLOWER
I20260709 13:23:36.095079 25199 leader_election.cc:304] T 87085ac2575e4b6a86dd1fc21e6f718b P 43999d813e564dc688fad116090c95ca [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3e07d64466934da4bb3c02262b1c2e38, 43999d813e564dc688fad116090c95ca; no voters: 233e7d29aac04505bf4e65f0a39a0c4c
I20260709 13:23:36.124709 25508 consensus_queue.cc:260] T 91ac744179de48e5bb924b6dfd9f9c9b P 9868dce521294d0b816bba7648f4ae5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } }
I20260709 13:23:36.115150 25676 raft_consensus.cc:515] T 6242888a4d4146dcacd87d0fd9a832f4 P 233e7d29aac04505bf4e65f0a39a0c4c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } }
I20260709 13:23:36.128010 25309 raft_consensus.cc:2393] T 6242888a4d4146dcacd87d0fd9a832f4 P 233e7d29aac04505bf4e65f0a39a0c4c [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 9868dce521294d0b816bba7648f4ae5a in current term 1: Already voted for candidate 233e7d29aac04505bf4e65f0a39a0c4c in this term.
I20260709 13:23:36.129046 25676 leader_election.cc:290] T 6242888a4d4146dcacd87d0fd9a832f4 P 233e7d29aac04505bf4e65f0a39a0c4c [CANDIDATE]: Term 1 election: Requested vote from peers 9868dce521294d0b816bba7648f4ae5a (127.14.177.132:33683), 43999d813e564dc688fad116090c95ca (127.14.177.129:38155)
I20260709 13:23:36.130300 25503 tablet_bootstrap.cc:492] T ce9c60e3922c4219b6e4453fe0259b6a P 233e7d29aac04505bf4e65f0a39a0c4c: Bootstrap starting.
I20260709 13:23:36.130452 25508 ts_tablet_manager.cc:1434] T 91ac744179de48e5bb924b6dfd9f9c9b P 9868dce521294d0b816bba7648f4ae5a: Time spent starting tablet: real 0.024s	user 0.006s	sys 0.001s
I20260709 13:23:36.131762 25508 tablet_bootstrap.cc:492] T 434a18a5109e4ea78349ded3968b2709 P 9868dce521294d0b816bba7648f4ae5a: Bootstrap starting.
I20260709 13:23:36.136436 25679 raft_consensus.cc:2749] T 79d4a12ef58140568c60a24ed4c12f83 P 233e7d29aac04505bf4e65f0a39a0c4c [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:23:36.140013 25762 raft_consensus.cc:2804] T 87085ac2575e4b6a86dd1fc21e6f718b P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:23:36.138860 25233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6242888a4d4146dcacd87d0fd9a832f4" candidate_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43999d813e564dc688fad116090c95ca"
I20260709 13:23:36.140589 25762 raft_consensus.cc:697] T 87085ac2575e4b6a86dd1fc21e6f718b P 43999d813e564dc688fad116090c95ca [term 1 LEADER]: Becoming Leader. State: Replica: 43999d813e564dc688fad116090c95ca, State: Running, Role: LEADER
I20260709 13:23:36.140892 25233 raft_consensus.cc:2393] T 6242888a4d4146dcacd87d0fd9a832f4 P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 233e7d29aac04505bf4e65f0a39a0c4c in current term 1: Already voted for candidate 9868dce521294d0b816bba7648f4ae5a in this term.
I20260709 13:23:36.141506 25762 consensus_queue.cc:237] T 87085ac2575e4b6a86dd1fc21e6f718b P 43999d813e564dc688fad116090c95ca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } }
I20260709 13:23:36.143566 25679 raft_consensus.cc:493] T 434a18a5109e4ea78349ded3968b2709 P 233e7d29aac04505bf4e65f0a39a0c4c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:23:36.143862 25602 raft_consensus.cc:493] T ce9c60e3922c4219b6e4453fe0259b6a P 43999d813e564dc688fad116090c95ca [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:23:36.144331 25679 raft_consensus.cc:515] T 434a18a5109e4ea78349ded3968b2709 P 233e7d29aac04505bf4e65f0a39a0c4c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } }
I20260709 13:23:36.144629 25602 raft_consensus.cc:515] T ce9c60e3922c4219b6e4453fe0259b6a P 43999d813e564dc688fad116090c95ca [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } }
I20260709 13:23:36.148062 25602 leader_election.cc:290] T ce9c60e3922c4219b6e4453fe0259b6a P 43999d813e564dc688fad116090c95ca [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9868dce521294d0b816bba7648f4ae5a (127.14.177.132:33683), 233e7d29aac04505bf4e65f0a39a0c4c (127.14.177.130:42957)
I20260709 13:23:36.158844 25505 tablet_bootstrap.cc:492] T 434a54fcee7d4aeeab42f739855c0e8a P 43999d813e564dc688fad116090c95ca: No bootstrap required, opened a new log
I20260709 13:23:36.159474 25505 ts_tablet_manager.cc:1403] T 434a54fcee7d4aeeab42f739855c0e8a P 43999d813e564dc688fad116090c95ca: Time spent bootstrapping tablet: real 0.110s	user 0.029s	sys 0.014s
I20260709 13:23:36.162313 25505 raft_consensus.cc:359] T 434a54fcee7d4aeeab42f739855c0e8a P 43999d813e564dc688fad116090c95ca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } }
I20260709 13:23:36.163026 25505 raft_consensus.cc:385] T 434a54fcee7d4aeeab42f739855c0e8a P 43999d813e564dc688fad116090c95ca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:23:36.163381 25505 raft_consensus.cc:740] T 434a54fcee7d4aeeab42f739855c0e8a P 43999d813e564dc688fad116090c95ca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43999d813e564dc688fad116090c95ca, State: Initialized, Role: FOLLOWER
I20260709 13:23:36.166216 25505 consensus_queue.cc:260] T 434a54fcee7d4aeeab42f739855c0e8a P 43999d813e564dc688fad116090c95ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } }
I20260709 13:23:36.164286 25679 leader_election.cc:290] T 434a18a5109e4ea78349ded3968b2709 P 233e7d29aac04505bf4e65f0a39a0c4c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3e07d64466934da4bb3c02262b1c2e38 (127.14.177.131:41539), 9868dce521294d0b816bba7648f4ae5a (127.14.177.132:33683)
I20260709 13:23:36.174937 25505 ts_tablet_manager.cc:1434] T 434a54fcee7d4aeeab42f739855c0e8a P 43999d813e564dc688fad116090c95ca: Time spent starting tablet: real 0.015s	user 0.002s	sys 0.004s
I20260709 13:23:36.175769 25457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce9c60e3922c4219b6e4453fe0259b6a" candidate_uuid: "43999d813e564dc688fad116090c95ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9868dce521294d0b816bba7648f4ae5a" is_pre_election: true
I20260709 13:23:36.175992 25505 tablet_bootstrap.cc:492] T b85a238bbc3b4621881603a534c7bd9b P 43999d813e564dc688fad116090c95ca: Bootstrap starting.
I20260709 13:23:36.176903 25503 tablet_bootstrap.cc:654] T ce9c60e3922c4219b6e4453fe0259b6a P 233e7d29aac04505bf4e65f0a39a0c4c: Neither blocks nor log segments found. Creating new log.
I20260709 13:23:36.170258 25644 raft_consensus.cc:493] T ce9c60e3922c4219b6e4453fe0259b6a P 9868dce521294d0b816bba7648f4ae5a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:23:36.177461 25644 raft_consensus.cc:515] T ce9c60e3922c4219b6e4453fe0259b6a P 9868dce521294d0b816bba7648f4ae5a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } }
I20260709 13:23:36.179709 25644 leader_election.cc:290] T ce9c60e3922c4219b6e4453fe0259b6a P 9868dce521294d0b816bba7648f4ae5a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 43999d813e564dc688fad116090c95ca (127.14.177.129:38155), 233e7d29aac04505bf4e65f0a39a0c4c (127.14.177.130:42957)
I20260709 13:23:36.181295 25233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce9c60e3922c4219b6e4453fe0259b6a" candidate_uuid: "9868dce521294d0b816bba7648f4ae5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43999d813e564dc688fad116090c95ca" is_pre_election: true
I20260709 13:23:36.181942 25233 raft_consensus.cc:2468] T ce9c60e3922c4219b6e4453fe0259b6a P 43999d813e564dc688fad116090c95ca [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9868dce521294d0b816bba7648f4ae5a in term 0.
I20260709 13:23:36.182552 25505 tablet_bootstrap.cc:654] T b85a238bbc3b4621881603a534c7bd9b P 43999d813e564dc688fad116090c95ca: Neither blocks nor log segments found. Creating new log.
I20260709 13:23:36.183393 25424 leader_election.cc:304] T ce9c60e3922c4219b6e4453fe0259b6a P 9868dce521294d0b816bba7648f4ae5a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 43999d813e564dc688fad116090c95ca, 9868dce521294d0b816bba7648f4ae5a; no voters: 
I20260709 13:23:36.186288 25457 raft_consensus.cc:2468] T ce9c60e3922c4219b6e4453fe0259b6a P 9868dce521294d0b816bba7648f4ae5a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 43999d813e564dc688fad116090c95ca in term 0.
I20260709 13:23:36.187086 25309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce9c60e3922c4219b6e4453fe0259b6a" candidate_uuid: "9868dce521294d0b816bba7648f4ae5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" is_pre_election: true
I20260709 13:23:36.188405 25650 raft_consensus.cc:2804] T ce9c60e3922c4219b6e4453fe0259b6a P 9868dce521294d0b816bba7648f4ae5a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:23:36.188468 25199 leader_election.cc:304] T ce9c60e3922c4219b6e4453fe0259b6a P 43999d813e564dc688fad116090c95ca [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 43999d813e564dc688fad116090c95ca, 9868dce521294d0b816bba7648f4ae5a; no voters: 
I20260709 13:23:36.188979 25650 raft_consensus.cc:493] T ce9c60e3922c4219b6e4453fe0259b6a P 9868dce521294d0b816bba7648f4ae5a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:23:36.189334 25650 raft_consensus.cc:3060] T ce9c60e3922c4219b6e4453fe0259b6a P 9868dce521294d0b816bba7648f4ae5a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:23:36.189473 25559 raft_consensus.cc:2804] T ce9c60e3922c4219b6e4453fe0259b6a P 43999d813e564dc688fad116090c95ca [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:23:36.189882 25559 raft_consensus.cc:493] T ce9c60e3922c4219b6e4453fe0259b6a P 43999d813e564dc688fad116090c95ca [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:23:36.190232 25559 raft_consensus.cc:3060] T ce9c60e3922c4219b6e4453fe0259b6a P 43999d813e564dc688fad116090c95ca [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:23:36.190940 25508 tablet_bootstrap.cc:654] T 434a18a5109e4ea78349ded3968b2709 P 9868dce521294d0b816bba7648f4ae5a: Neither blocks nor log segments found. Creating new log.
I20260709 13:23:36.172425 25307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce9c60e3922c4219b6e4453fe0259b6a" candidate_uuid: "43999d813e564dc688fad116090c95ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" is_pre_election: true
W20260709 13:23:36.166891 25274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41539, user_credentials={real_user=slave}} blocked reactor thread for 75020.2us
I20260709 13:23:36.195674 25601 consensus_queue.cc:1048] T 13bb3a23315a41c7a40e10b954b15c8e P 43999d813e564dc688fad116090c95ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:23:36.191640 25644 raft_consensus.cc:493] T bbab48ba177049dcba6de106fd889cfb P 9868dce521294d0b816bba7648f4ae5a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:23:36.192266 25425 leader_election.cc:343] T ce9c60e3922c4219b6e4453fe0259b6a P 9868dce521294d0b816bba7648f4ae5a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 233e7d29aac04505bf4e65f0a39a0c4c (127.14.177.130:42957): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:23:36.196923 25385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbab48ba177049dcba6de106fd889cfb" candidate_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e07d64466934da4bb3c02262b1c2e38" is_pre_election: true
I20260709 13:23:36.197556 25385 raft_consensus.cc:2468] T bbab48ba177049dcba6de106fd889cfb P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 233e7d29aac04505bf4e65f0a39a0c4c in term 0.
W20260709 13:23:36.202800 25202 leader_election.cc:343] T ce9c60e3922c4219b6e4453fe0259b6a P 43999d813e564dc688fad116090c95ca [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 233e7d29aac04505bf4e65f0a39a0c4c (127.14.177.130:42957): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:23:36.216271 25601 consensus_queue.cc:1048] T a5df750b84e0487fa530161079deb6c2 P 43999d813e564dc688fad116090c95ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:23:36.216732 25457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbab48ba177049dcba6de106fd889cfb" candidate_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9868dce521294d0b816bba7648f4ae5a" is_pre_election: true
I20260709 13:23:36.198127 25644 raft_consensus.cc:515] T bbab48ba177049dcba6de106fd889cfb P 9868dce521294d0b816bba7648f4ae5a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } }
I20260709 13:23:36.219889 25651 consensus_queue.cc:1048] T 89a26dd3ac414f6081b98fa32c2f346f P 9868dce521294d0b816bba7648f4ae5a [LEADER]: Connected to new peer: Peer: permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:23:36.221254 25644 leader_election.cc:290] T bbab48ba177049dcba6de106fd889cfb P 9868dce521294d0b816bba7648f4ae5a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 233e7d29aac04505bf4e65f0a39a0c4c (127.14.177.130:42957), 3e07d64466934da4bb3c02262b1c2e38 (127.14.177.131:41539)
I20260709 13:23:36.221944 25457 raft_consensus.cc:2468] T bbab48ba177049dcba6de106fd889cfb P 9868dce521294d0b816bba7648f4ae5a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 233e7d29aac04505bf4e65f0a39a0c4c in term 0.
I20260709 13:23:36.223598 25385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbab48ba177049dcba6de106fd889cfb" candidate_uuid: "9868dce521294d0b816bba7648f4ae5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e07d64466934da4bb3c02262b1c2e38" is_pre_election: true
I20260709 13:23:36.224395 25385 raft_consensus.cc:2468] T bbab48ba177049dcba6de106fd889cfb P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9868dce521294d0b816bba7648f4ae5a in term 0.
I20260709 13:23:36.225243 25311 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbab48ba177049dcba6de106fd889cfb" candidate_uuid: "9868dce521294d0b816bba7648f4ae5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" is_pre_election: true
I20260709 13:23:36.225951 25311 raft_consensus.cc:2468] T bbab48ba177049dcba6de106fd889cfb P 233e7d29aac04505bf4e65f0a39a0c4c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9868dce521294d0b816bba7648f4ae5a in term 0.
I20260709 13:23:36.226724 25274 leader_election.cc:304] T bbab48ba177049dcba6de106fd889cfb P 233e7d29aac04505bf4e65f0a39a0c4c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 233e7d29aac04505bf4e65f0a39a0c4c, 9868dce521294d0b816bba7648f4ae5a; no voters: 
I20260709 13:23:36.229239 25385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434a18a5109e4ea78349ded3968b2709" candidate_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e07d64466934da4bb3c02262b1c2e38" is_pre_election: true
I20260709 13:23:36.230049 25385 raft_consensus.cc:2468] T 434a18a5109e4ea78349ded3968b2709 P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 233e7d29aac04505bf4e65f0a39a0c4c in term 0.
I20260709 13:23:36.234146 25132 catalog_manager.cc:5697] T 6242888a4d4146dcacd87d0fd9a832f4 P 9868dce521294d0b816bba7648f4ae5a reported cstate change: term changed from 0 to 1, leader changed from <none> to 9868dce521294d0b816bba7648f4ae5a (127.14.177.132). New cstate: current_term: 1 leader_uuid: "9868dce521294d0b816bba7648f4ae5a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } health_report { overall_health: UNKNOWN } } }
I20260709 13:23:36.228111 25422 leader_election.cc:304] T bbab48ba177049dcba6de106fd889cfb P 9868dce521294d0b816bba7648f4ae5a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e07d64466934da4bb3c02262b1c2e38, 9868dce521294d0b816bba7648f4ae5a; no voters: 
I20260709 13:23:36.241040 25455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6242888a4d4146dcacd87d0fd9a832f4" candidate_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9868dce521294d0b816bba7648f4ae5a"
I20260709 13:23:36.244691 25503 tablet_bootstrap.cc:492] T ce9c60e3922c4219b6e4453fe0259b6a P 233e7d29aac04505bf4e65f0a39a0c4c: No bootstrap required, opened a new log
I20260709 13:23:36.245443 25503 ts_tablet_manager.cc:1403] T ce9c60e3922c4219b6e4453fe0259b6a P 233e7d29aac04505bf4e65f0a39a0c4c: Time spent bootstrapping tablet: real 0.115s	user 0.009s	sys 0.034s
I20260709 13:23:36.238616 25679 raft_consensus.cc:2804] T bbab48ba177049dcba6de106fd889cfb P 233e7d29aac04505bf4e65f0a39a0c4c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:23:36.250552 25539 raft_consensus.cc:2804] T bbab48ba177049dcba6de106fd889cfb P 9868dce521294d0b816bba7648f4ae5a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:23:36.251014 25539 raft_consensus.cc:493] T bbab48ba177049dcba6de106fd889cfb P 9868dce521294d0b816bba7648f4ae5a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:23:36.251367 25539 raft_consensus.cc:3060] T bbab48ba177049dcba6de106fd889cfb P 9868dce521294d0b816bba7648f4ae5a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:23:36.254031 25504 tablet_bootstrap.cc:492] T 434a54fcee7d4aeeab42f739855c0e8a P 3e07d64466934da4bb3c02262b1c2e38: No bootstrap required, opened a new log
I20260709 13:23:36.254561 25504 ts_tablet_manager.cc:1403] T 434a54fcee7d4aeeab42f739855c0e8a P 3e07d64466934da4bb3c02262b1c2e38: Time spent bootstrapping tablet: real 0.214s	user 0.007s	sys 0.005s
I20260709 13:23:36.244705 25457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434a18a5109e4ea78349ded3968b2709" candidate_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9868dce521294d0b816bba7648f4ae5a" is_pre_election: true
I20260709 13:23:36.253427 25559 raft_consensus.cc:515] T ce9c60e3922c4219b6e4453fe0259b6a P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } }
I20260709 13:23:36.268141 25129 catalog_manager.cc:5697] T 87085ac2575e4b6a86dd1fc21e6f718b P 43999d813e564dc688fad116090c95ca reported cstate change: term changed from 0 to 1, leader changed from <none> to 43999d813e564dc688fad116090c95ca (127.14.177.129). New cstate: current_term: 1 leader_uuid: "43999d813e564dc688fad116090c95ca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } health_report { overall_health: UNKNOWN } } }
I20260709 13:23:36.270454 25686 consensus_queue.cc:1048] T 91ac744179de48e5bb924b6dfd9f9c9b P 3e07d64466934da4bb3c02262b1c2e38 [LEADER]: Connected to new peer: Peer: permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:23:36.248605 25679 raft_consensus.cc:493] T bbab48ba177049dcba6de106fd889cfb P 233e7d29aac04505bf4e65f0a39a0c4c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:23:36.268625 25559 leader_election.cc:290] T ce9c60e3922c4219b6e4453fe0259b6a P 43999d813e564dc688fad116090c95ca [CANDIDATE]: Term 1 election: Requested vote from peers 9868dce521294d0b816bba7648f4ae5a (127.14.177.132:33683), 233e7d29aac04505bf4e65f0a39a0c4c (127.14.177.130:42957)
I20260709 13:23:36.277462 25679 raft_consensus.cc:3060] T bbab48ba177049dcba6de106fd889cfb P 233e7d29aac04505bf4e65f0a39a0c4c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:23:36.291296 25650 raft_consensus.cc:515] T ce9c60e3922c4219b6e4453fe0259b6a P 9868dce521294d0b816bba7648f4ae5a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } }
I20260709 13:23:36.294016 25650 leader_election.cc:290] T ce9c60e3922c4219b6e4453fe0259b6a P 9868dce521294d0b816bba7648f4ae5a [CANDIDATE]: Term 1 election: Requested vote from peers 43999d813e564dc688fad116090c95ca (127.14.177.129:38155), 233e7d29aac04505bf4e65f0a39a0c4c (127.14.177.130:42957)
I20260709 13:23:36.277627 25274 leader_election.cc:304] T 434a18a5109e4ea78349ded3968b2709 P 233e7d29aac04505bf4e65f0a39a0c4c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 233e7d29aac04505bf4e65f0a39a0c4c, 3e07d64466934da4bb3c02262b1c2e38; no voters: 
I20260709 13:23:36.297827 25308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce9c60e3922c4219b6e4453fe0259b6a" candidate_uuid: "9868dce521294d0b816bba7648f4ae5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "233e7d29aac04505bf4e65f0a39a0c4c"
I20260709 13:23:36.298718 25798 raft_consensus.cc:2804] T 434a18a5109e4ea78349ded3968b2709 P 233e7d29aac04505bf4e65f0a39a0c4c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:23:36.299221 25798 raft_consensus.cc:493] T 434a18a5109e4ea78349ded3968b2709 P 233e7d29aac04505bf4e65f0a39a0c4c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:23:36.299603 25798 raft_consensus.cc:3060] T 434a18a5109e4ea78349ded3968b2709 P 233e7d29aac04505bf4e65f0a39a0c4c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:23:36.303849 25504 raft_consensus.cc:359] T 434a54fcee7d4aeeab42f739855c0e8a P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } }
I20260709 13:23:36.304725 25504 raft_consensus.cc:385] T 434a54fcee7d4aeeab42f739855c0e8a P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:23:36.305083 25504 raft_consensus.cc:740] T 434a54fcee7d4aeeab42f739855c0e8a P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e07d64466934da4bb3c02262b1c2e38, State: Initialized, Role: FOLLOWER
I20260709 13:23:36.305891 25504 consensus_queue.cc:260] T 434a54fcee7d4aeeab42f739855c0e8a P 3e07d64466934da4bb3c02262b1c2e38 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } }
I20260709 13:23:36.313485 25568 raft_consensus.cc:493] T 4d837697cd0f402cacf9db89a5d92652 P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:23:36.314152 25568 raft_consensus.cc:515] T 4d837697cd0f402cacf9db89a5d92652 P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } }
I20260709 13:23:36.317032 25457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce9c60e3922c4219b6e4453fe0259b6a" candidate_uuid: "43999d813e564dc688fad116090c95ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9868dce521294d0b816bba7648f4ae5a"
I20260709 13:23:36.317981 25457 raft_consensus.cc:2393] T ce9c60e3922c4219b6e4453fe0259b6a P 9868dce521294d0b816bba7648f4ae5a [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 43999d813e564dc688fad116090c95ca in current term 1: Already voted for candidate 9868dce521294d0b816bba7648f4ae5a in this term.
I20260709 13:23:36.320858 25235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce9c60e3922c4219b6e4453fe0259b6a" candidate_uuid: "9868dce521294d0b816bba7648f4ae5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43999d813e564dc688fad116090c95ca"
I20260709 13:23:36.321784 25235 raft_consensus.cc:2393] T ce9c60e3922c4219b6e4453fe0259b6a P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 9868dce521294d0b816bba7648f4ae5a in current term 1: Already voted for candidate 43999d813e564dc688fad116090c95ca in this term.
I20260709 13:23:36.322324 25568 leader_election.cc:290] T 4d837697cd0f402cacf9db89a5d92652 P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9868dce521294d0b816bba7648f4ae5a (127.14.177.132:33683), 43999d813e564dc688fad116090c95ca (127.14.177.129:38155)
I20260709 13:23:36.320123 25644 raft_consensus.cc:493] T 91ac744179de48e5bb924b6dfd9f9c9b P 9868dce521294d0b816bba7648f4ae5a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:23:36.323261 25274 leader_election.cc:304] T 6242888a4d4146dcacd87d0fd9a832f4 P 233e7d29aac04505bf4e65f0a39a0c4c [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 233e7d29aac04505bf4e65f0a39a0c4c; no voters: 43999d813e564dc688fad116090c95ca, 9868dce521294d0b816bba7648f4ae5a
I20260709 13:23:36.323341 25644 raft_consensus.cc:515] T 91ac744179de48e5bb924b6dfd9f9c9b P 9868dce521294d0b816bba7648f4ae5a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } }
I20260709 13:23:36.321774 25504 ts_tablet_manager.cc:1434] T 434a54fcee7d4aeeab42f739855c0e8a P 3e07d64466934da4bb3c02262b1c2e38: Time spent starting tablet: real 0.067s	user 0.007s	sys 0.001s
I20260709 13:23:36.330356 25504 tablet_bootstrap.cc:492] T cfad575709c943cba40ccab278350770 P 3e07d64466934da4bb3c02262b1c2e38: Bootstrap starting.
I20260709 13:23:36.332952 25760 consensus_queue.cc:1048] T 1550ed2d2a764c08adffae679599ba41 P 43999d813e564dc688fad116090c95ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:23:36.335575 25573 consensus_queue.cc:1048] T 79d4a12ef58140568c60a24ed4c12f83 P 3e07d64466934da4bb3c02262b1c2e38 [LEADER]: Connected to new peer: Peer: permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:23:36.360916 25455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d837697cd0f402cacf9db89a5d92652" candidate_uuid: "3e07d64466934da4bb3c02262b1c2e38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9868dce521294d0b816bba7648f4ae5a" is_pre_election: true
I20260709 13:23:36.360800 25798 raft_consensus.cc:515] T 434a18a5109e4ea78349ded3968b2709 P 233e7d29aac04505bf4e65f0a39a0c4c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } }
W20260709 13:23:36.362996 25182 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:23:36.363523 25504 tablet_bootstrap.cc:654] T cfad575709c943cba40ccab278350770 P 3e07d64466934da4bb3c02262b1c2e38: Neither blocks nor log segments found. Creating new log.
I20260709 13:23:36.365942 25573 raft_consensus.cc:493] T 0ed5edbe32044d1c83b0590dbb899801 P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:23:36.358875 25503 raft_consensus.cc:359] T ce9c60e3922c4219b6e4453fe0259b6a P 233e7d29aac04505bf4e65f0a39a0c4c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } }
I20260709 13:23:36.367928 25503 raft_consensus.cc:385] T ce9c60e3922c4219b6e4453fe0259b6a P 233e7d29aac04505bf4e65f0a39a0c4c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:23:36.363381 25644 leader_election.cc:290] T 91ac744179de48e5bb924b6dfd9f9c9b P 9868dce521294d0b816bba7648f4ae5a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3e07d64466934da4bb3c02262b1c2e38 (127.14.177.131:41539), 43999d813e564dc688fad116090c95ca (127.14.177.129:38155)
I20260709 13:23:36.342404 25795 raft_consensus.cc:2749] T 6242888a4d4146dcacd87d0fd9a832f4 P 233e7d29aac04505bf4e65f0a39a0c4c [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:23:36.368232 25503 raft_consensus.cc:740] T ce9c60e3922c4219b6e4453fe0259b6a P 233e7d29aac04505bf4e65f0a39a0c4c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 233e7d29aac04505bf4e65f0a39a0c4c, State: Initialized, Role: FOLLOWER
I20260709 13:23:36.371675 25503 consensus_queue.cc:260] T ce9c60e3922c4219b6e4453fe0259b6a P 233e7d29aac04505bf4e65f0a39a0c4c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } }
I20260709 13:23:36.343796 25650 consensus_queue.cc:1048] T 89a26dd3ac414f6081b98fa32c2f346f P 9868dce521294d0b816bba7648f4ae5a [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.028s
I20260709 13:23:36.381919 25503 ts_tablet_manager.cc:1434] T ce9c60e3922c4219b6e4453fe0259b6a P 233e7d29aac04505bf4e65f0a39a0c4c: Time spent starting tablet: real 0.136s	user 0.008s	sys 0.000s
I20260709 13:23:36.383807 25805 raft_consensus.cc:493] T 434a54fcee7d4aeeab42f739855c0e8a P 43999d813e564dc688fad116090c95ca [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:23:36.384667 25805 raft_consensus.cc:515] T 434a54fcee7d4aeeab42f739855c0e8a P 43999d813e564dc688fad116090c95ca [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } }
I20260709 13:23:36.366955 25235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d837697cd0f402cacf9db89a5d92652" candidate_uuid: "3e07d64466934da4bb3c02262b1c2e38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43999d813e564dc688fad116090c95ca" is_pre_election: true
I20260709 13:23:36.366673 25573 raft_consensus.cc:515] T 0ed5edbe32044d1c83b0590dbb899801 P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } }
W20260709 13:23:36.374010 25274 leader_election.cc:343] T 434a18a5109e4ea78349ded3968b2709 P 233e7d29aac04505bf4e65f0a39a0c4c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9868dce521294d0b816bba7648f4ae5a (127.14.177.132:33683): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:23:36.403788 25348 leader_election.cc:343] T 4d837697cd0f402cacf9db89a5d92652 P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9868dce521294d0b816bba7648f4ae5a (127.14.177.132:33683): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:23:36.406586 25795 raft_consensus.cc:493] T ce9c60e3922c4219b6e4453fe0259b6a P 233e7d29aac04505bf4e65f0a39a0c4c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:23:36.407280 25795 raft_consensus.cc:515] T ce9c60e3922c4219b6e4453fe0259b6a P 233e7d29aac04505bf4e65f0a39a0c4c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } }
I20260709 13:23:36.410326 25795 leader_election.cc:290] T ce9c60e3922c4219b6e4453fe0259b6a P 233e7d29aac04505bf4e65f0a39a0c4c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9868dce521294d0b816bba7648f4ae5a (127.14.177.132:33683), 43999d813e564dc688fad116090c95ca (127.14.177.129:38155)
I20260709 13:23:36.389778 25798 leader_election.cc:290] T 434a18a5109e4ea78349ded3968b2709 P 233e7d29aac04505bf4e65f0a39a0c4c [CANDIDATE]: Term 1 election: Requested vote from peers 3e07d64466934da4bb3c02262b1c2e38 (127.14.177.131:41539), 9868dce521294d0b816bba7648f4ae5a (127.14.177.132:33683)
I20260709 13:23:36.397219 25234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91ac744179de48e5bb924b6dfd9f9c9b" candidate_uuid: "9868dce521294d0b816bba7648f4ae5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43999d813e564dc688fad116090c95ca" is_pre_election: true
W20260709 13:23:36.419803 25350 leader_election.cc:343] T 4d837697cd0f402cacf9db89a5d92652 P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 43999d813e564dc688fad116090c95ca (127.14.177.129:38155): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:23:36.420356 25350 leader_election.cc:304] T 4d837697cd0f402cacf9db89a5d92652 P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3e07d64466934da4bb3c02262b1c2e38; no voters: 43999d813e564dc688fad116090c95ca, 9868dce521294d0b816bba7648f4ae5a
I20260709 13:23:36.427028 25231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ed5edbe32044d1c83b0590dbb899801" candidate_uuid: "3e07d64466934da4bb3c02262b1c2e38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43999d813e564dc688fad116090c95ca" is_pre_election: true
I20260709 13:23:36.427834 25231 raft_consensus.cc:2468] T 0ed5edbe32044d1c83b0590dbb899801 P 43999d813e564dc688fad116090c95ca [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3e07d64466934da4bb3c02262b1c2e38 in term 0.
I20260709 13:23:36.431998 25508 tablet_bootstrap.cc:492] T 434a18a5109e4ea78349ded3968b2709 P 9868dce521294d0b816bba7648f4ae5a: No bootstrap required, opened a new log
I20260709 13:23:36.432564 25508 ts_tablet_manager.cc:1403] T 434a18a5109e4ea78349ded3968b2709 P 9868dce521294d0b816bba7648f4ae5a: Time spent bootstrapping tablet: real 0.301s	user 0.023s	sys 0.020s
I20260709 13:23:36.435829 25508 raft_consensus.cc:359] T 434a18a5109e4ea78349ded3968b2709 P 9868dce521294d0b816bba7648f4ae5a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } }
I20260709 13:23:36.436573 25508 raft_consensus.cc:385] T 434a18a5109e4ea78349ded3968b2709 P 9868dce521294d0b816bba7648f4ae5a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:23:36.436354 25679 raft_consensus.cc:515] T bbab48ba177049dcba6de106fd889cfb P 233e7d29aac04505bf4e65f0a39a0c4c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } }
I20260709 13:23:36.436911 25508 raft_consensus.cc:740] T 434a18a5109e4ea78349ded3968b2709 P 9868dce521294d0b816bba7648f4ae5a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9868dce521294d0b816bba7648f4ae5a, State: Initialized, Role: FOLLOWER
I20260709 13:23:36.438933 25503 tablet_bootstrap.cc:492] T 87085ac2575e4b6a86dd1fc21e6f718b P 233e7d29aac04505bf4e65f0a39a0c4c: Bootstrap starting.
I20260709 13:23:36.440075 25539 raft_consensus.cc:515] T bbab48ba177049dcba6de106fd889cfb P 9868dce521294d0b816bba7648f4ae5a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } }
I20260709 13:23:36.420375 25573 leader_election.cc:290] T 0ed5edbe32044d1c83b0590dbb899801 P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9868dce521294d0b816bba7648f4ae5a (127.14.177.132:33683), 43999d813e564dc688fad116090c95ca (127.14.177.129:38155)
I20260709 13:23:36.438969 25679 leader_election.cc:290] T bbab48ba177049dcba6de106fd889cfb P 233e7d29aac04505bf4e65f0a39a0c4c [CANDIDATE]: Term 1 election: Requested vote from peers 3e07d64466934da4bb3c02262b1c2e38 (127.14.177.131:41539), 9868dce521294d0b816bba7648f4ae5a (127.14.177.132:33683)
I20260709 13:23:36.442344 25311 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce9c60e3922c4219b6e4453fe0259b6a" candidate_uuid: "43999d813e564dc688fad116090c95ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "233e7d29aac04505bf4e65f0a39a0c4c"
I20260709 13:23:36.443363 25311 raft_consensus.cc:3060] T ce9c60e3922c4219b6e4453fe0259b6a P 233e7d29aac04505bf4e65f0a39a0c4c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:23:36.448411 25385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91ac744179de48e5bb924b6dfd9f9c9b" candidate_uuid: "9868dce521294d0b816bba7648f4ae5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e07d64466934da4bb3c02262b1c2e38" is_pre_election: true
I20260709 13:23:36.428293 25383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434a18a5109e4ea78349ded3968b2709" candidate_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e07d64466934da4bb3c02262b1c2e38"
I20260709 13:23:36.451725 25383 raft_consensus.cc:3060] T 434a18a5109e4ea78349ded3968b2709 P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:23:36.454731 25815 raft_consensus.cc:2749] T 4d837697cd0f402cacf9db89a5d92652 P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:23:36.455592 25686 consensus_queue.cc:1048] T 79d4a12ef58140568c60a24ed4c12f83 P 3e07d64466934da4bb3c02262b1c2e38 [LEADER]: Connected to new peer: Peer: permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:23:36.457552 25308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbab48ba177049dcba6de106fd889cfb" candidate_uuid: "9868dce521294d0b816bba7648f4ae5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "233e7d29aac04505bf4e65f0a39a0c4c"
I20260709 13:23:36.458459 25308 raft_consensus.cc:2393] T bbab48ba177049dcba6de106fd889cfb P 233e7d29aac04505bf4e65f0a39a0c4c [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 9868dce521294d0b816bba7648f4ae5a in current term 1: Already voted for candidate 233e7d29aac04505bf4e65f0a39a0c4c in this term.
I20260709 13:23:36.459131 25503 tablet_bootstrap.cc:654] T 87085ac2575e4b6a86dd1fc21e6f718b P 233e7d29aac04505bf4e65f0a39a0c4c: Neither blocks nor log segments found. Creating new log.
I20260709 13:23:36.464733 25425 leader_election.cc:304] T ce9c60e3922c4219b6e4453fe0259b6a P 9868dce521294d0b816bba7648f4ae5a [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9868dce521294d0b816bba7648f4ae5a; no voters: 233e7d29aac04505bf4e65f0a39a0c4c, 43999d813e564dc688fad116090c95ca
I20260709 13:23:36.465366 25311 raft_consensus.cc:2468] T ce9c60e3922c4219b6e4453fe0259b6a P 233e7d29aac04505bf4e65f0a39a0c4c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 43999d813e564dc688fad116090c95ca in term 1.
I20260709 13:23:36.476800 25385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbab48ba177049dcba6de106fd889cfb" candidate_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e07d64466934da4bb3c02262b1c2e38"
I20260709 13:23:36.437768 25508 consensus_queue.cc:260] T 434a18a5109e4ea78349ded3968b2709 P 9868dce521294d0b816bba7648f4ae5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } }
I20260709 13:23:36.477773 25385 raft_consensus.cc:3060] T bbab48ba177049dcba6de106fd889cfb P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:23:36.479317 25508 ts_tablet_manager.cc:1434] T 434a18a5109e4ea78349ded3968b2709 P 9868dce521294d0b816bba7648f4ae5a: Time spent starting tablet: real 0.046s	user 0.008s	sys 0.000s
I20260709 13:23:36.480484 25508 tablet_bootstrap.cc:492] T 9ab90e3e08fb4528b2a47d995e9e26d3 P 9868dce521294d0b816bba7648f4ae5a: Bootstrap starting.
I20260709 13:23:36.436944 25458 raft_consensus.cc:3060] T 91ac744179de48e5bb924b6dfd9f9c9b P 9868dce521294d0b816bba7648f4ae5a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:23:36.490394 25385 raft_consensus.cc:2468] T bbab48ba177049dcba6de106fd889cfb P 3e07d64466934da4bb3c02262b1c2e38 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 233e7d29aac04505bf4e65f0a39a0c4c in term 1.
I20260709 13:23:36.494560 25350 leader_election.cc:304] T 0ed5edbe32044d1c83b0590dbb899801 P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e07d64466934da4bb3c02262b1c2e38, 43999d813e564dc688fad116090c95ca; no voters: 
I20260709 13:23:36.500186 25539 leader_election.cc:290] T bbab48ba177049dcba6de106fd889cfb P 9868dce521294d0b816bba7648f4ae5a [CANDIDATE]: Term 1 election: Requested vote from peers 233e7d29aac04505bf4e65f0a39a0c4c (127.14.177.130:42957), 3e07d64466934da4bb3c02262b1c2e38 (127.14.177.131:41539)
I20260709 13:23:36.476152 25805 leader_election.cc:290] T 434a54fcee7d4aeeab42f739855c0e8a P 43999d813e564dc688fad116090c95ca [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 233e7d29aac04505bf4e65f0a39a0c4c (127.14.177.130:42957), 3e07d64466934da4bb3c02262b1c2e38 (127.14.177.131:41539)
I20260709 13:23:36.504316 25457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ed5edbe32044d1c83b0590dbb899801" candidate_uuid: "3e07d64466934da4bb3c02262b1c2e38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9868dce521294d0b816bba7648f4ae5a" is_pre_election: true
I20260709 13:23:36.509099 25504 tablet_bootstrap.cc:492] T cfad575709c943cba40ccab278350770 P 3e07d64466934da4bb3c02262b1c2e38: No bootstrap required, opened a new log
I20260709 13:23:36.509930 25504 ts_tablet_manager.cc:1403] T cfad575709c943cba40ccab278350770 P 3e07d64466934da4bb3c02262b1c2e38: Time spent bootstrapping tablet: real 0.180s	user 0.008s	sys 0.009s
I20260709 13:23:36.514813 25504 raft_consensus.cc:359] T cfad575709c943cba40ccab278350770 P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } }
I20260709 13:23:36.515404 25453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbab48ba177049dcba6de106fd889cfb" candidate_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9868dce521294d0b816bba7648f4ae5a"
I20260709 13:23:36.515875 25504 raft_consensus.cc:385] T cfad575709c943cba40ccab278350770 P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:23:36.516358 25453 raft_consensus.cc:2393] T bbab48ba177049dcba6de106fd889cfb P 9868dce521294d0b816bba7648f4ae5a [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 233e7d29aac04505bf4e65f0a39a0c4c in current term 1: Already voted for candidate 9868dce521294d0b816bba7648f4ae5a in this term.
I20260709 13:23:36.516454 25504 raft_consensus.cc:740] T cfad575709c943cba40ccab278350770 P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e07d64466934da4bb3c02262b1c2e38, State: Initialized, Role: FOLLOWER
I20260709 13:23:36.517889 25454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434a18a5109e4ea78349ded3968b2709" candidate_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9868dce521294d0b816bba7648f4ae5a"
I20260709 13:23:36.520294 25505 tablet_bootstrap.cc:492] T b85a238bbc3b4621881603a534c7bd9b P 43999d813e564dc688fad116090c95ca: No bootstrap required, opened a new log
I20260709 13:23:36.523317 25505 ts_tablet_manager.cc:1403] T b85a238bbc3b4621881603a534c7bd9b P 43999d813e564dc688fad116090c95ca: Time spent bootstrapping tablet: real 0.348s	user 0.016s	sys 0.012s
I20260709 13:23:36.532090 25384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434a54fcee7d4aeeab42f739855c0e8a" candidate_uuid: "43999d813e564dc688fad116090c95ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e07d64466934da4bb3c02262b1c2e38" is_pre_election: true
I20260709 13:23:36.518841 25815 raft_consensus.cc:2804] T 0ed5edbe32044d1c83b0590dbb899801 P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:23:36.525583 25454 raft_consensus.cc:3060] T 434a18a5109e4ea78349ded3968b2709 P 9868dce521294d0b816bba7648f4ae5a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:23:36.544993 25504 consensus_queue.cc:260] T cfad575709c943cba40ccab278350770 P 3e07d64466934da4bb3c02262b1c2e38 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } }
W20260709 13:23:36.530247 25348 leader_election.cc:343] T 0ed5edbe32044d1c83b0590dbb899801 P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9868dce521294d0b816bba7648f4ae5a (127.14.177.132:33683): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:23:36.543043 25815 raft_consensus.cc:493] T 0ed5edbe32044d1c83b0590dbb899801 P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:23:36.542591 25382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbab48ba177049dcba6de106fd889cfb" candidate_uuid: "9868dce521294d0b816bba7648f4ae5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e07d64466934da4bb3c02262b1c2e38"
I20260709 13:23:36.542591 25813 raft_consensus.cc:2749] T ce9c60e3922c4219b6e4453fe0259b6a P 9868dce521294d0b816bba7648f4ae5a [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:23:36.525194 25274 leader_election.cc:304] T bbab48ba177049dcba6de106fd889cfb P 233e7d29aac04505bf4e65f0a39a0c4c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 233e7d29aac04505bf4e65f0a39a0c4c, 3e07d64466934da4bb3c02262b1c2e38; no voters: 
I20260709 13:23:36.543018 25384 raft_consensus.cc:2468] T 434a54fcee7d4aeeab42f739855c0e8a P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 43999d813e564dc688fad116090c95ca in term 0.
I20260709 13:23:36.568212 25382 raft_consensus.cc:2393] T bbab48ba177049dcba6de106fd889cfb P 3e07d64466934da4bb3c02262b1c2e38 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 9868dce521294d0b816bba7648f4ae5a in current term 1: Already voted for candidate 233e7d29aac04505bf4e65f0a39a0c4c in this term.
I20260709 13:23:36.569970 25383 raft_consensus.cc:2468] T 434a18a5109e4ea78349ded3968b2709 P 3e07d64466934da4bb3c02262b1c2e38 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 233e7d29aac04505bf4e65f0a39a0c4c in term 1.
I20260709 13:23:36.576627 25508 tablet_bootstrap.cc:654] T 9ab90e3e08fb4528b2a47d995e9e26d3 P 9868dce521294d0b816bba7648f4ae5a: Neither blocks nor log segments found. Creating new log.
I20260709 13:23:36.579996 25504 ts_tablet_manager.cc:1434] T cfad575709c943cba40ccab278350770 P 3e07d64466934da4bb3c02262b1c2e38: Time spent starting tablet: real 0.070s	user 0.009s	sys 0.000s
I20260709 13:23:36.581108 25504 tablet_bootstrap.cc:492] T 9ab90e3e08fb4528b2a47d995e9e26d3 P 3e07d64466934da4bb3c02262b1c2e38: Bootstrap starting.
I20260709 13:23:36.579082 25505 raft_consensus.cc:359] T b85a238bbc3b4621881603a534c7bd9b P 43999d813e564dc688fad116090c95ca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } }
I20260709 13:23:36.588874 25505 raft_consensus.cc:385] T b85a238bbc3b4621881603a534c7bd9b P 43999d813e564dc688fad116090c95ca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:23:36.589401 25505 raft_consensus.cc:740] T b85a238bbc3b4621881603a534c7bd9b P 43999d813e564dc688fad116090c95ca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43999d813e564dc688fad116090c95ca, State: Initialized, Role: FOLLOWER
I20260709 13:23:36.590408 25799 raft_consensus.cc:2804] T bbab48ba177049dcba6de106fd889cfb P 233e7d29aac04505bf4e65f0a39a0c4c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:23:36.594471 25202 leader_election.cc:304] T ce9c60e3922c4219b6e4453fe0259b6a P 43999d813e564dc688fad116090c95ca [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 233e7d29aac04505bf4e65f0a39a0c4c, 43999d813e564dc688fad116090c95ca; no voters: 9868dce521294d0b816bba7648f4ae5a
I20260709 13:23:36.594007 25504 tablet_bootstrap.cc:654] T 9ab90e3e08fb4528b2a47d995e9e26d3 P 3e07d64466934da4bb3c02262b1c2e38: Neither blocks nor log segments found. Creating new log.
W20260709 13:23:36.584235 25274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41539, user_credentials={real_user=slave}} blocked reactor thread for 64803.2us
W20260709 13:23:36.617657 25199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.132:33683, user_credentials={real_user=slave}} blocked reactor thread for 71230.6us
I20260709 13:23:36.599778 25422 leader_election.cc:304] T 91ac744179de48e5bb924b6dfd9f9c9b P 9868dce521294d0b816bba7648f4ae5a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9868dce521294d0b816bba7648f4ae5a; no voters: 3e07d64466934da4bb3c02262b1c2e38, 43999d813e564dc688fad116090c95ca
I20260709 13:23:36.558310 25815 raft_consensus.cc:3060] T 0ed5edbe32044d1c83b0590dbb899801 P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:23:36.557904 25805 consensus_queue.cc:1048] T 1550ed2d2a764c08adffae679599ba41 P 43999d813e564dc688fad116090c95ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.060s
I20260709 13:23:36.637952 25505 consensus_queue.cc:260] T b85a238bbc3b4621881603a534c7bd9b P 43999d813e564dc688fad116090c95ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } }
I20260709 13:23:36.648301 25459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce9c60e3922c4219b6e4453fe0259b6a" candidate_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9868dce521294d0b816bba7648f4ae5a" is_pre_election: true
I20260709 13:23:36.649641 25459 raft_consensus.cc:2393] T ce9c60e3922c4219b6e4453fe0259b6a P 9868dce521294d0b816bba7648f4ae5a [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 233e7d29aac04505bf4e65f0a39a0c4c in current term 1: Already voted for candidate 9868dce521294d0b816bba7648f4ae5a in this term.
I20260709 13:23:36.655416 25601 raft_consensus.cc:2804] T ce9c60e3922c4219b6e4453fe0259b6a P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:23:36.671576 25505 ts_tablet_manager.cc:1434] T b85a238bbc3b4621881603a534c7bd9b P 43999d813e564dc688fad116090c95ca: Time spent starting tablet: real 0.148s	user 0.008s	sys 0.000s
I20260709 13:23:36.667682 25799 raft_consensus.cc:697] T bbab48ba177049dcba6de106fd889cfb P 233e7d29aac04505bf4e65f0a39a0c4c [term 1 LEADER]: Becoming Leader. State: Replica: 233e7d29aac04505bf4e65f0a39a0c4c, State: Running, Role: LEADER
I20260709 13:23:36.661567 25274 leader_election.cc:304] T 434a18a5109e4ea78349ded3968b2709 P 233e7d29aac04505bf4e65f0a39a0c4c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 233e7d29aac04505bf4e65f0a39a0c4c, 3e07d64466934da4bb3c02262b1c2e38; no voters: 
I20260709 13:23:36.675484 25601 raft_consensus.cc:697] T ce9c60e3922c4219b6e4453fe0259b6a P 43999d813e564dc688fad116090c95ca [term 1 LEADER]: Becoming Leader. State: Replica: 43999d813e564dc688fad116090c95ca, State: Running, Role: LEADER
W20260709 13:23:36.673184 25202 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:42957, user_credentials={real_user=slave}} blocked reactor thread for 78824.9us
I20260709 13:23:36.677512 25762 consensus_queue.cc:1048] T f1ac2f4d2fe2424bb470df42199219e2 P 43999d813e564dc688fad116090c95ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.010s
I20260709 13:23:36.696552 25505 tablet_bootstrap.cc:492] T 4d837697cd0f402cacf9db89a5d92652 P 43999d813e564dc688fad116090c95ca: Bootstrap starting.
I20260709 13:23:36.680629 25539 raft_consensus.cc:2749] T 91ac744179de48e5bb924b6dfd9f9c9b P 9868dce521294d0b816bba7648f4ae5a [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:23:36.673596 25422 leader_election.cc:304] T bbab48ba177049dcba6de106fd889cfb P 9868dce521294d0b816bba7648f4ae5a [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9868dce521294d0b816bba7648f4ae5a; no voters: 233e7d29aac04505bf4e65f0a39a0c4c, 3e07d64466934da4bb3c02262b1c2e38
I20260709 13:23:36.701179 25508 tablet_bootstrap.cc:492] T 9ab90e3e08fb4528b2a47d995e9e26d3 P 9868dce521294d0b816bba7648f4ae5a: No bootstrap required, opened a new log
I20260709 13:23:36.704584 25808 raft_consensus.cc:2749] T bbab48ba177049dcba6de106fd889cfb P 9868dce521294d0b816bba7648f4ae5a [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
W20260709 13:23:36.705808 25422 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41539, user_credentials={real_user=slave}} blocked reactor thread for 43726.6us
I20260709 13:23:36.706789 25504 tablet_bootstrap.cc:492] T 9ab90e3e08fb4528b2a47d995e9e26d3 P 3e07d64466934da4bb3c02262b1c2e38: No bootstrap required, opened a new log
I20260709 13:23:36.707368 25504 ts_tablet_manager.cc:1403] T 9ab90e3e08fb4528b2a47d995e9e26d3 P 3e07d64466934da4bb3c02262b1c2e38: Time spent bootstrapping tablet: real 0.126s	user 0.018s	sys 0.000s
I20260709 13:23:36.709576 25808 consensus_queue.cc:1048] T 6242888a4d4146dcacd87d0fd9a832f4 P 9868dce521294d0b816bba7648f4ae5a [LEADER]: Connected to new peer: Peer: permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:23:36.714013 25508 ts_tablet_manager.cc:1403] T 9ab90e3e08fb4528b2a47d995e9e26d3 P 9868dce521294d0b816bba7648f4ae5a: Time spent bootstrapping tablet: real 0.223s	user 0.014s	sys 0.005s
I20260709 13:23:36.717314 25799 consensus_queue.cc:237] T bbab48ba177049dcba6de106fd889cfb P 233e7d29aac04505bf4e65f0a39a0c4c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } }
I20260709 13:23:36.702105 25676 raft_consensus.cc:2804] T 434a18a5109e4ea78349ded3968b2709 P 233e7d29aac04505bf4e65f0a39a0c4c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:23:36.729053 25504 raft_consensus.cc:359] T 9ab90e3e08fb4528b2a47d995e9e26d3 P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } }
I20260709 13:23:36.730616 25504 raft_consensus.cc:385] T 9ab90e3e08fb4528b2a47d995e9e26d3 P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:23:36.722455 25505 tablet_bootstrap.cc:654] T 4d837697cd0f402cacf9db89a5d92652 P 43999d813e564dc688fad116090c95ca: Neither blocks nor log segments found. Creating new log.
W20260709 13:23:36.733013 25350 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.129:38155, user_credentials={real_user=slave}} blocked reactor thread for 55011.3us
I20260709 13:23:36.723125 25508 raft_consensus.cc:359] T 9ab90e3e08fb4528b2a47d995e9e26d3 P 9868dce521294d0b816bba7648f4ae5a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } }
I20260709 13:23:36.733810 25508 raft_consensus.cc:385] T 9ab90e3e08fb4528b2a47d995e9e26d3 P 9868dce521294d0b816bba7648f4ae5a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:23:36.734123 25199 leader_election.cc:304] T 434a54fcee7d4aeeab42f739855c0e8a P 43999d813e564dc688fad116090c95ca [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e07d64466934da4bb3c02262b1c2e38, 43999d813e564dc688fad116090c95ca; no voters: 
I20260709 13:23:36.731037 25504 raft_consensus.cc:740] T 9ab90e3e08fb4528b2a47d995e9e26d3 P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e07d64466934da4bb3c02262b1c2e38, State: Initialized, Role: FOLLOWER
I20260709 13:23:36.734112 25508 raft_consensus.cc:740] T 9ab90e3e08fb4528b2a47d995e9e26d3 P 9868dce521294d0b816bba7648f4ae5a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9868dce521294d0b816bba7648f4ae5a, State: Initialized, Role: FOLLOWER
I20260709 13:23:36.736318 25601 consensus_queue.cc:237] T ce9c60e3922c4219b6e4453fe0259b6a P 43999d813e564dc688fad116090c95ca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } }
I20260709 13:23:36.732441 25813 consensus_queue.cc:1048] T 6242888a4d4146dcacd87d0fd9a832f4 P 9868dce521294d0b816bba7648f4ae5a [LEADER]: Connected to new peer: Peer: permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.005s
I20260709 13:23:36.736879 25762 raft_consensus.cc:2804] T 434a54fcee7d4aeeab42f739855c0e8a P 43999d813e564dc688fad116090c95ca [term 0 FOLLOWER]: Leader pre-election won for term 1
W20260709 13:23:36.700711 25274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41539, user_credentials={real_user=slave}} blocked reactor thread for 48651.6us
I20260709 13:23:36.745173 25821 consensus_queue.cc:1048] T f1ac2f4d2fe2424bb470df42199219e2 P 43999d813e564dc688fad116090c95ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:23:36.745818 25814 consensus_queue.cc:1048] T 91ac744179de48e5bb924b6dfd9f9c9b P 3e07d64466934da4bb3c02262b1c2e38 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:23:36.727603 25454 raft_consensus.cc:2468] T 434a18a5109e4ea78349ded3968b2709 P 9868dce521294d0b816bba7648f4ae5a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 233e7d29aac04505bf4e65f0a39a0c4c in term 1.
I20260709 13:23:36.747856 25503 tablet_bootstrap.cc:492] T 87085ac2575e4b6a86dd1fc21e6f718b P 233e7d29aac04505bf4e65f0a39a0c4c: No bootstrap required, opened a new log
I20260709 13:23:36.751029 25503 ts_tablet_manager.cc:1403] T 87085ac2575e4b6a86dd1fc21e6f718b P 233e7d29aac04505bf4e65f0a39a0c4c: Time spent bootstrapping tablet: real 0.312s	user 0.007s	sys 0.016s
I20260709 13:23:36.744119 25504 consensus_queue.cc:260] T 9ab90e3e08fb4528b2a47d995e9e26d3 P 3e07d64466934da4bb3c02262b1c2e38 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } }
I20260709 13:23:36.734647 25676 raft_consensus.cc:697] T 434a18a5109e4ea78349ded3968b2709 P 233e7d29aac04505bf4e65f0a39a0c4c [term 1 LEADER]: Becoming Leader. State: Replica: 233e7d29aac04505bf4e65f0a39a0c4c, State: Running, Role: LEADER
I20260709 13:23:36.757928 25308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434a54fcee7d4aeeab42f739855c0e8a" candidate_uuid: "43999d813e564dc688fad116090c95ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" is_pre_election: true
I20260709 13:23:36.758272 25676 consensus_queue.cc:237] T 434a18a5109e4ea78349ded3968b2709 P 233e7d29aac04505bf4e65f0a39a0c4c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } }
I20260709 13:23:36.761392 25815 raft_consensus.cc:515] T 0ed5edbe32044d1c83b0590dbb899801 P 3e07d64466934da4bb3c02262b1c2e38 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } }
I20260709 13:23:36.766923 25232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce9c60e3922c4219b6e4453fe0259b6a" candidate_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43999d813e564dc688fad116090c95ca" is_pre_election: true
W20260709 13:23:36.767153 25202 leader_election.cc:343] T 434a54fcee7d4aeeab42f739855c0e8a P 43999d813e564dc688fad116090c95ca [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 233e7d29aac04505bf4e65f0a39a0c4c (127.14.177.130:42957): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:23:36.756377 25508 consensus_queue.cc:260] T 9ab90e3e08fb4528b2a47d995e9e26d3 P 9868dce521294d0b816bba7648f4ae5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } }
I20260709 13:23:36.746760 25762 raft_consensus.cc:493] T 434a54fcee7d4aeeab42f739855c0e8a P 43999d813e564dc688fad116090c95ca [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:23:36.800542 25825 log.cc:927] Time spent T 89a26dd3ac414f6081b98fa32c2f346f P 3e07d64466934da4bb3c02262b1c2e38: Append to log took a long time: real 0.124s	user 0.001s	sys 0.001s
I20260709 13:23:36.804239 25503 raft_consensus.cc:359] T 87085ac2575e4b6a86dd1fc21e6f718b P 233e7d29aac04505bf4e65f0a39a0c4c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } }
I20260709 13:23:36.805095 25503 raft_consensus.cc:385] T 87085ac2575e4b6a86dd1fc21e6f718b P 233e7d29aac04505bf4e65f0a39a0c4c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:23:36.780176 25539 raft_consensus.cc:493] T 9ab90e3e08fb4528b2a47d995e9e26d3 P 9868dce521294d0b816bba7648f4ae5a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:23:36.792317 25762 raft_consensus.cc:3060] T 434a54fcee7d4aeeab42f739855c0e8a P 43999d813e564dc688fad116090c95ca [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:23:36.810886 25504 ts_tablet_manager.cc:1434] T 9ab90e3e08fb4528b2a47d995e9e26d3 P 3e07d64466934da4bb3c02262b1c2e38: Time spent starting tablet: real 0.103s	user 0.009s	sys 0.000s
I20260709 13:23:36.818368 25504 tablet_bootstrap.cc:492] T b9d550d881b04735bbf89751952c09ec P 3e07d64466934da4bb3c02262b1c2e38: Bootstrap starting.
I20260709 13:23:36.805399 25503 raft_consensus.cc:740] T 87085ac2575e4b6a86dd1fc21e6f718b P 233e7d29aac04505bf4e65f0a39a0c4c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 233e7d29aac04505bf4e65f0a39a0c4c, State: Initialized, Role: FOLLOWER
I20260709 13:23:36.819559 25503 consensus_queue.cc:260] T 87085ac2575e4b6a86dd1fc21e6f718b P 233e7d29aac04505bf4e65f0a39a0c4c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } }
I20260709 13:23:36.817011 25814 raft_consensus.cc:493] T cfad575709c943cba40ccab278350770 P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:23:36.841646 25503 ts_tablet_manager.cc:1434] T 87085ac2575e4b6a86dd1fc21e6f718b P 233e7d29aac04505bf4e65f0a39a0c4c: Time spent starting tablet: real 0.090s	user 0.004s	sys 0.003s
I20260709 13:23:36.800652 25274 leader_election.cc:304] T ce9c60e3922c4219b6e4453fe0259b6a P 233e7d29aac04505bf4e65f0a39a0c4c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 233e7d29aac04505bf4e65f0a39a0c4c; no voters: 43999d813e564dc688fad116090c95ca, 9868dce521294d0b816bba7648f4ae5a
I20260709 13:23:36.822300 25539 raft_consensus.cc:515] T 9ab90e3e08fb4528b2a47d995e9e26d3 P 9868dce521294d0b816bba7648f4ae5a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } }
I20260709 13:23:36.852494 25815 leader_election.cc:290] T 0ed5edbe32044d1c83b0590dbb899801 P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 1 election: Requested vote from peers 9868dce521294d0b816bba7648f4ae5a (127.14.177.132:33683), 43999d813e564dc688fad116090c95ca (127.14.177.129:38155)
I20260709 13:23:36.901026 25503 tablet_bootstrap.cc:492] T 9ab90e3e08fb4528b2a47d995e9e26d3 P 233e7d29aac04505bf4e65f0a39a0c4c: Bootstrap starting.
I20260709 13:23:36.847875 25814 raft_consensus.cc:515] T cfad575709c943cba40ccab278350770 P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } }
W20260709 13:23:36.908890 25350 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.129:38155, user_credentials={real_user=slave}} blocked reactor thread for 109637us
I20260709 13:23:36.900817 25799 raft_consensus.cc:2749] T ce9c60e3922c4219b6e4453fe0259b6a P 233e7d29aac04505bf4e65f0a39a0c4c [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:23:36.905146 25459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ed5edbe32044d1c83b0590dbb899801" candidate_uuid: "3e07d64466934da4bb3c02262b1c2e38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9868dce521294d0b816bba7648f4ae5a"
W20260709 13:23:36.959594 25837 log.cc:927] Time spent T 434a18a5109e4ea78349ded3968b2709 P 233e7d29aac04505bf4e65f0a39a0c4c: Append to log took a long time: real 0.052s	user 0.001s	sys 0.000s
I20260709 13:23:36.961766 25504 tablet_bootstrap.cc:654] T b9d550d881b04735bbf89751952c09ec P 3e07d64466934da4bb3c02262b1c2e38: Neither blocks nor log segments found. Creating new log.
W20260709 13:23:36.910539 25274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:33683, user_credentials={real_user=slave}} blocked reactor thread for 112983us
I20260709 13:23:36.986414 25602 raft_consensus.cc:493] T b85a238bbc3b4621881603a534c7bd9b P 43999d813e564dc688fad116090c95ca [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:23:36.987062 25602 raft_consensus.cc:515] T b85a238bbc3b4621881603a534c7bd9b P 43999d813e564dc688fad116090c95ca [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } }
W20260709 13:23:36.994102 25348 leader_election.cc:343] T 0ed5edbe32044d1c83b0590dbb899801 P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 9868dce521294d0b816bba7648f4ae5a (127.14.177.132:33683): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:23:37.001080 25307 raft_consensus.cc:3060] T 87085ac2575e4b6a86dd1fc21e6f718b P 233e7d29aac04505bf4e65f0a39a0c4c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:23:37.002650 25233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ed5edbe32044d1c83b0590dbb899801" candidate_uuid: "3e07d64466934da4bb3c02262b1c2e38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43999d813e564dc688fad116090c95ca"
I20260709 13:23:37.003435 25233 raft_consensus.cc:3060] T 0ed5edbe32044d1c83b0590dbb899801 P 43999d813e564dc688fad116090c95ca [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:23:36.973672 25836 log.cc:927] Time spent T 6242888a4d4146dcacd87d0fd9a832f4 P 233e7d29aac04505bf4e65f0a39a0c4c: Append to log took a long time: real 0.070s	user 0.001s	sys 0.000s
I20260709 13:23:37.029330 25814 leader_election.cc:290] T cfad575709c943cba40ccab278350770 P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 233e7d29aac04505bf4e65f0a39a0c4c (127.14.177.130:42957), 9868dce521294d0b816bba7648f4ae5a (127.14.177.132:33683)
I20260709 13:23:37.073781 25132 catalog_manager.cc:5697] T ce9c60e3922c4219b6e4453fe0259b6a P 43999d813e564dc688fad116090c95ca reported cstate change: term changed from 0 to 1, leader changed from <none> to 43999d813e564dc688fad116090c95ca (127.14.177.129). New cstate: current_term: 1 leader_uuid: "43999d813e564dc688fad116090c95ca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } health_report { overall_health: UNKNOWN } } }
I20260709 13:23:37.038313 25602 leader_election.cc:290] T b85a238bbc3b4621881603a534c7bd9b P 43999d813e564dc688fad116090c95ca [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9868dce521294d0b816bba7648f4ae5a (127.14.177.132:33683), 233e7d29aac04505bf4e65f0a39a0c4c (127.14.177.130:42957)
I20260709 13:23:37.089813 25508 ts_tablet_manager.cc:1434] T 9ab90e3e08fb4528b2a47d995e9e26d3 P 9868dce521294d0b816bba7648f4ae5a: Time spent starting tablet: real 0.375s	user 0.008s	sys 0.005s
I20260709 13:23:37.108068 25308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b85a238bbc3b4621881603a534c7bd9b" candidate_uuid: "43999d813e564dc688fad116090c95ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" is_pre_election: true
I20260709 13:23:37.109549 25539 leader_election.cc:290] T 9ab90e3e08fb4528b2a47d995e9e26d3 P 9868dce521294d0b816bba7648f4ae5a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 233e7d29aac04505bf4e65f0a39a0c4c (127.14.177.130:42957), 3e07d64466934da4bb3c02262b1c2e38 (127.14.177.131:41539)
I20260709 13:23:37.110045 25503 tablet_bootstrap.cc:654] T 9ab90e3e08fb4528b2a47d995e9e26d3 P 233e7d29aac04505bf4e65f0a39a0c4c: Neither blocks nor log segments found. Creating new log.
W20260709 13:23:37.111812 25840 log.cc:927] Time spent T f1ac2f4d2fe2424bb470df42199219e2 P 9868dce521294d0b816bba7648f4ae5a: Append to log took a long time: real 0.085s	user 0.000s	sys 0.001s
I20260709 13:23:37.111931 25762 raft_consensus.cc:515] T 434a54fcee7d4aeeab42f739855c0e8a P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } }
I20260709 13:23:37.114004 25762 leader_election.cc:290] T 434a54fcee7d4aeeab42f739855c0e8a P 43999d813e564dc688fad116090c95ca [CANDIDATE]: Term 1 election: Requested vote from peers 233e7d29aac04505bf4e65f0a39a0c4c (127.14.177.130:42957), 3e07d64466934da4bb3c02262b1c2e38 (127.14.177.131:41539)
I20260709 13:23:37.119860 25504 tablet_bootstrap.cc:492] T b9d550d881b04735bbf89751952c09ec P 3e07d64466934da4bb3c02262b1c2e38: No bootstrap required, opened a new log
I20260709 13:23:37.120173 25233 raft_consensus.cc:2468] T 0ed5edbe32044d1c83b0590dbb899801 P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3e07d64466934da4bb3c02262b1c2e38 in term 1.
I20260709 13:23:37.120415 25504 ts_tablet_manager.cc:1403] T b9d550d881b04735bbf89751952c09ec P 3e07d64466934da4bb3c02262b1c2e38: Time spent bootstrapping tablet: real 0.302s	user 0.011s	sys 0.026s
I20260709 13:23:37.127429 25505 tablet_bootstrap.cc:492] T 4d837697cd0f402cacf9db89a5d92652 P 43999d813e564dc688fad116090c95ca: No bootstrap required, opened a new log
I20260709 13:23:37.127954 25505 ts_tablet_manager.cc:1403] T 4d837697cd0f402cacf9db89a5d92652 P 43999d813e564dc688fad116090c95ca: Time spent bootstrapping tablet: real 0.432s	user 0.009s	sys 0.010s
I20260709 13:23:37.132161 25308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfad575709c943cba40ccab278350770" candidate_uuid: "3e07d64466934da4bb3c02262b1c2e38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" is_pre_election: true
W20260709 13:23:37.133195 25835 log.cc:927] Time spent T 79d4a12ef58140568c60a24ed4c12f83 P 233e7d29aac04505bf4e65f0a39a0c4c: Append to log took a long time: real 0.252s	user 0.002s	sys 0.000s
I20260709 13:23:37.137598 25309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434a54fcee7d4aeeab42f739855c0e8a" candidate_uuid: "43999d813e564dc688fad116090c95ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "233e7d29aac04505bf4e65f0a39a0c4c"
I20260709 13:23:37.140419 25458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b85a238bbc3b4621881603a534c7bd9b" candidate_uuid: "43999d813e564dc688fad116090c95ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9868dce521294d0b816bba7648f4ae5a" is_pre_election: true
W20260709 13:23:37.144116 25202 leader_election.cc:343] T b85a238bbc3b4621881603a534c7bd9b P 43999d813e564dc688fad116090c95ca [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 233e7d29aac04505bf4e65f0a39a0c4c (127.14.177.130:42957): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:23:37.145921 25202 leader_election.cc:343] T 434a54fcee7d4aeeab42f739855c0e8a P 43999d813e564dc688fad116090c95ca [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 233e7d29aac04505bf4e65f0a39a0c4c (127.14.177.130:42957): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:23:37.148139 25351 leader_election.cc:343] T cfad575709c943cba40ccab278350770 P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 233e7d29aac04505bf4e65f0a39a0c4c (127.14.177.130:42957): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:23:37.150049 25839 log.cc:927] Time spent T ce9c60e3922c4219b6e4453fe0259b6a P 43999d813e564dc688fad116090c95ca: Append to log took a long time: real 0.237s	user 0.001s	sys 0.000s
I20260709 13:23:37.158881 25508 tablet_bootstrap.cc:492] T 0ed5edbe32044d1c83b0590dbb899801 P 9868dce521294d0b816bba7648f4ae5a: Bootstrap starting.
I20260709 13:23:37.160996 25504 raft_consensus.cc:359] T b9d550d881b04735bbf89751952c09ec P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } }
I20260709 13:23:37.162199 25504 raft_consensus.cc:385] T b9d550d881b04735bbf89751952c09ec P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260709 13:23:37.166280 25199 leader_election.cc:343] T b85a238bbc3b4621881603a534c7bd9b P 43999d813e564dc688fad116090c95ca [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9868dce521294d0b816bba7648f4ae5a (127.14.177.132:33683): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:23:37.167443 25504 raft_consensus.cc:740] T b9d550d881b04735bbf89751952c09ec P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e07d64466934da4bb3c02262b1c2e38, State: Initialized, Role: FOLLOWER
I20260709 13:23:37.167840 25199 leader_election.cc:304] T b85a238bbc3b4621881603a534c7bd9b P 43999d813e564dc688fad116090c95ca [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 43999d813e564dc688fad116090c95ca; no voters: 233e7d29aac04505bf4e65f0a39a0c4c, 9868dce521294d0b816bba7648f4ae5a
I20260709 13:23:37.172289 25383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434a54fcee7d4aeeab42f739855c0e8a" candidate_uuid: "43999d813e564dc688fad116090c95ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e07d64466934da4bb3c02262b1c2e38"
I20260709 13:23:37.173103 25383 raft_consensus.cc:3060] T 434a54fcee7d4aeeab42f739855c0e8a P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:23:37.173288 25762 raft_consensus.cc:2749] T b85a238bbc3b4621881603a534c7bd9b P 43999d813e564dc688fad116090c95ca [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:23:37.174718 25504 consensus_queue.cc:260] T b9d550d881b04735bbf89751952c09ec P 3e07d64466934da4bb3c02262b1c2e38 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } }
I20260709 13:23:37.183463 25823 consensus_queue.cc:1048] T 87085ac2575e4b6a86dd1fc21e6f718b P 43999d813e564dc688fad116090c95ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:23:37.184425 25504 ts_tablet_manager.cc:1434] T b9d550d881b04735bbf89751952c09ec P 3e07d64466934da4bb3c02262b1c2e38: Time spent starting tablet: real 0.064s	user 0.007s	sys 0.000s
I20260709 13:23:37.185971 25504 tablet_bootstrap.cc:492] T b155b866f10547fd8239579a81ecf5f9 P 3e07d64466934da4bb3c02262b1c2e38: Bootstrap starting.
I20260709 13:23:37.185695 25505 raft_consensus.cc:359] T 4d837697cd0f402cacf9db89a5d92652 P 43999d813e564dc688fad116090c95ca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } }
I20260709 13:23:37.186422 25505 raft_consensus.cc:385] T 4d837697cd0f402cacf9db89a5d92652 P 43999d813e564dc688fad116090c95ca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:23:37.186697 25505 raft_consensus.cc:740] T 4d837697cd0f402cacf9db89a5d92652 P 43999d813e564dc688fad116090c95ca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43999d813e564dc688fad116090c95ca, State: Initialized, Role: FOLLOWER
I20260709 13:23:37.176785 25459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfad575709c943cba40ccab278350770" candidate_uuid: "3e07d64466934da4bb3c02262b1c2e38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9868dce521294d0b816bba7648f4ae5a" is_pre_election: true
I20260709 13:23:37.194621 25505 consensus_queue.cc:260] T 4d837697cd0f402cacf9db89a5d92652 P 43999d813e564dc688fad116090c95ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } }
I20260709 13:23:37.197003 25822 consensus_queue.cc:1048] T 87085ac2575e4b6a86dd1fc21e6f718b P 43999d813e564dc688fad116090c95ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.008s
I20260709 13:23:37.201121 25380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ab90e3e08fb4528b2a47d995e9e26d3" candidate_uuid: "9868dce521294d0b816bba7648f4ae5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e07d64466934da4bb3c02262b1c2e38" is_pre_election: true
I20260709 13:23:37.201576 25306 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ab90e3e08fb4528b2a47d995e9e26d3" candidate_uuid: "9868dce521294d0b816bba7648f4ae5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" is_pre_election: true
I20260709 13:23:37.201870 25380 raft_consensus.cc:2468] T 9ab90e3e08fb4528b2a47d995e9e26d3 P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9868dce521294d0b816bba7648f4ae5a in term 0.
W20260709 13:23:37.211602 25425 leader_election.cc:343] T 9ab90e3e08fb4528b2a47d995e9e26d3 P 9868dce521294d0b816bba7648f4ae5a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 233e7d29aac04505bf4e65f0a39a0c4c (127.14.177.130:42957): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:23:37.219504 25348 leader_election.cc:343] T cfad575709c943cba40ccab278350770 P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9868dce521294d0b816bba7648f4ae5a (127.14.177.132:33683): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:23:37.220062 25348 leader_election.cc:304] T cfad575709c943cba40ccab278350770 P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3e07d64466934da4bb3c02262b1c2e38; no voters: 233e7d29aac04505bf4e65f0a39a0c4c, 9868dce521294d0b816bba7648f4ae5a
I20260709 13:23:37.224301 25508 tablet_bootstrap.cc:654] T 0ed5edbe32044d1c83b0590dbb899801 P 9868dce521294d0b816bba7648f4ae5a: Neither blocks nor log segments found. Creating new log.
I20260709 13:23:37.229699 25841 raft_consensus.cc:2749] T cfad575709c943cba40ccab278350770 P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:23:37.231652 25130 catalog_manager.cc:5697] T bbab48ba177049dcba6de106fd889cfb P 233e7d29aac04505bf4e65f0a39a0c4c reported cstate change: term changed from 0 to 1, leader changed from <none> to 233e7d29aac04505bf4e65f0a39a0c4c (127.14.177.130). New cstate: current_term: 1 leader_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } health_report { overall_health: UNKNOWN } } }
I20260709 13:23:37.233381 25422 leader_election.cc:304] T 9ab90e3e08fb4528b2a47d995e9e26d3 P 9868dce521294d0b816bba7648f4ae5a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3e07d64466934da4bb3c02262b1c2e38, 9868dce521294d0b816bba7648f4ae5a; no voters: 233e7d29aac04505bf4e65f0a39a0c4c
I20260709 13:23:37.236038 25813 raft_consensus.cc:2804] T 9ab90e3e08fb4528b2a47d995e9e26d3 P 9868dce521294d0b816bba7648f4ae5a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:23:37.236444 25813 raft_consensus.cc:493] T 9ab90e3e08fb4528b2a47d995e9e26d3 P 9868dce521294d0b816bba7648f4ae5a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:23:37.236786 25813 raft_consensus.cc:3060] T 9ab90e3e08fb4528b2a47d995e9e26d3 P 9868dce521294d0b816bba7648f4ae5a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:23:37.240633 25832 consensus_queue.cc:1048] T 434a18a5109e4ea78349ded3968b2709 P 233e7d29aac04505bf4e65f0a39a0c4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:23:37.244076 25795 raft_consensus.cc:493] T 87085ac2575e4b6a86dd1fc21e6f718b P 233e7d29aac04505bf4e65f0a39a0c4c [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:23:37.244788 25795 raft_consensus.cc:515] T 87085ac2575e4b6a86dd1fc21e6f718b P 233e7d29aac04505bf4e65f0a39a0c4c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } }
I20260709 13:23:37.245803 25504 tablet_bootstrap.cc:654] T b155b866f10547fd8239579a81ecf5f9 P 3e07d64466934da4bb3c02262b1c2e38: Neither blocks nor log segments found. Creating new log.
I20260709 13:23:37.252308 25795 leader_election.cc:290] T 87085ac2575e4b6a86dd1fc21e6f718b P 233e7d29aac04505bf4e65f0a39a0c4c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 43999d813e564dc688fad116090c95ca (127.14.177.129:38155), 3e07d64466934da4bb3c02262b1c2e38 (127.14.177.131:41539)
I20260709 13:23:37.257004 25383 raft_consensus.cc:2468] T 434a54fcee7d4aeeab42f739855c0e8a P 3e07d64466934da4bb3c02262b1c2e38 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 43999d813e564dc688fad116090c95ca in term 1.
I20260709 13:23:37.258030 25505 ts_tablet_manager.cc:1434] T 4d837697cd0f402cacf9db89a5d92652 P 43999d813e564dc688fad116090c95ca: Time spent starting tablet: real 0.130s	user 0.007s	sys 0.000s
I20260709 13:23:37.268986 25383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87085ac2575e4b6a86dd1fc21e6f718b" candidate_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e07d64466934da4bb3c02262b1c2e38" is_pre_election: true
I20260709 13:23:37.269917 25383 raft_consensus.cc:2468] T 87085ac2575e4b6a86dd1fc21e6f718b P 3e07d64466934da4bb3c02262b1c2e38 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 233e7d29aac04505bf4e65f0a39a0c4c in term 1.
I20260709 13:23:37.273061 25274 leader_election.cc:304] T 87085ac2575e4b6a86dd1fc21e6f718b P 233e7d29aac04505bf4e65f0a39a0c4c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 233e7d29aac04505bf4e65f0a39a0c4c, 3e07d64466934da4bb3c02262b1c2e38; no voters: 
I20260709 13:23:37.274791 25832 raft_consensus.cc:2804] T 87085ac2575e4b6a86dd1fc21e6f718b P 233e7d29aac04505bf4e65f0a39a0c4c [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:23:37.275286 25832 raft_consensus.cc:493] T 87085ac2575e4b6a86dd1fc21e6f718b P 233e7d29aac04505bf4e65f0a39a0c4c [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:23:37.275797 25832 raft_consensus.cc:3060] T 87085ac2575e4b6a86dd1fc21e6f718b P 233e7d29aac04505bf4e65f0a39a0c4c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:23:37.277907 25505 tablet_bootstrap.cc:492] T 8ef061028f38450e83324ac78cdaa14c P 43999d813e564dc688fad116090c95ca: Bootstrap starting.
W20260709 13:23:37.283774 25202 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.130:42957, user_credentials={real_user=slave}} blocked reactor thread for 93377.6us
I20260709 13:23:37.285112 25823 consensus_queue.cc:1048] T ce9c60e3922c4219b6e4453fe0259b6a P 43999d813e564dc688fad116090c95ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:23:37.292718 25350 leader_election.cc:304] T 0ed5edbe32044d1c83b0590dbb899801 P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3e07d64466934da4bb3c02262b1c2e38, 43999d813e564dc688fad116090c95ca; no voters: 9868dce521294d0b816bba7648f4ae5a
I20260709 13:23:37.299398 25622 raft_consensus.cc:2804] T 0ed5edbe32044d1c83b0590dbb899801 P 3e07d64466934da4bb3c02262b1c2e38 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:23:37.300107 25622 raft_consensus.cc:697] T 0ed5edbe32044d1c83b0590dbb899801 P 3e07d64466934da4bb3c02262b1c2e38 [term 1 LEADER]: Becoming Leader. State: Replica: 3e07d64466934da4bb3c02262b1c2e38, State: Running, Role: LEADER
I20260709 13:23:37.301213 25622 consensus_queue.cc:237] T 0ed5edbe32044d1c83b0590dbb899801 P 3e07d64466934da4bb3c02262b1c2e38 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } }
I20260709 13:23:37.323859 25503 tablet_bootstrap.cc:492] T 9ab90e3e08fb4528b2a47d995e9e26d3 P 233e7d29aac04505bf4e65f0a39a0c4c: No bootstrap required, opened a new log
I20260709 13:23:37.324432 25503 ts_tablet_manager.cc:1403] T 9ab90e3e08fb4528b2a47d995e9e26d3 P 233e7d29aac04505bf4e65f0a39a0c4c: Time spent bootstrapping tablet: real 0.479s	user 0.014s	sys 0.005s
I20260709 13:23:37.327111 25503 raft_consensus.cc:359] T 9ab90e3e08fb4528b2a47d995e9e26d3 P 233e7d29aac04505bf4e65f0a39a0c4c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } }
I20260709 13:23:37.347586 25503 raft_consensus.cc:385] T 9ab90e3e08fb4528b2a47d995e9e26d3 P 233e7d29aac04505bf4e65f0a39a0c4c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:23:37.349231 25503 raft_consensus.cc:740] T 9ab90e3e08fb4528b2a47d995e9e26d3 P 233e7d29aac04505bf4e65f0a39a0c4c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 233e7d29aac04505bf4e65f0a39a0c4c, State: Initialized, Role: FOLLOWER
I20260709 13:23:37.351617 25503 consensus_queue.cc:260] T 9ab90e3e08fb4528b2a47d995e9e26d3 P 233e7d29aac04505bf4e65f0a39a0c4c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } }
I20260709 13:23:37.353758 25813 raft_consensus.cc:515] T 9ab90e3e08fb4528b2a47d995e9e26d3 P 9868dce521294d0b816bba7648f4ae5a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } }
I20260709 13:23:37.356782 25813 leader_election.cc:290] T 9ab90e3e08fb4528b2a47d995e9e26d3 P 9868dce521294d0b816bba7648f4ae5a [CANDIDATE]: Term 1 election: Requested vote from peers 233e7d29aac04505bf4e65f0a39a0c4c (127.14.177.130:42957), 3e07d64466934da4bb3c02262b1c2e38 (127.14.177.131:41539)
I20260709 13:23:37.353829 25503 ts_tablet_manager.cc:1434] T 9ab90e3e08fb4528b2a47d995e9e26d3 P 233e7d29aac04505bf4e65f0a39a0c4c: Time spent starting tablet: real 0.029s	user 0.005s	sys 0.000s
I20260709 13:23:37.362566 25503 tablet_bootstrap.cc:492] T 434a54fcee7d4aeeab42f739855c0e8a P 233e7d29aac04505bf4e65f0a39a0c4c: Bootstrap starting.
I20260709 13:23:37.368433 25832 raft_consensus.cc:515] T 87085ac2575e4b6a86dd1fc21e6f718b P 233e7d29aac04505bf4e65f0a39a0c4c [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } }
I20260709 13:23:37.371869 25199 leader_election.cc:304] T 434a54fcee7d4aeeab42f739855c0e8a P 43999d813e564dc688fad116090c95ca [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3e07d64466934da4bb3c02262b1c2e38, 43999d813e564dc688fad116090c95ca; no voters: 233e7d29aac04505bf4e65f0a39a0c4c
I20260709 13:23:37.375272 25823 raft_consensus.cc:2804] T 434a54fcee7d4aeeab42f739855c0e8a P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:23:37.375825 25823 raft_consensus.cc:697] T 434a54fcee7d4aeeab42f739855c0e8a P 43999d813e564dc688fad116090c95ca [term 1 LEADER]: Becoming Leader. State: Replica: 43999d813e564dc688fad116090c95ca, State: Running, Role: LEADER
I20260709 13:23:37.376628 25823 consensus_queue.cc:237] T 434a54fcee7d4aeeab42f739855c0e8a P 43999d813e564dc688fad116090c95ca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } }
I20260709 13:23:37.375097 25508 tablet_bootstrap.cc:492] T 0ed5edbe32044d1c83b0590dbb899801 P 9868dce521294d0b816bba7648f4ae5a: No bootstrap required, opened a new log
I20260709 13:23:37.379119 25508 ts_tablet_manager.cc:1403] T 0ed5edbe32044d1c83b0590dbb899801 P 9868dce521294d0b816bba7648f4ae5a: Time spent bootstrapping tablet: real 0.221s	user 0.011s	sys 0.013s
I20260709 13:23:37.377030 25309 raft_consensus.cc:1240] T 87085ac2575e4b6a86dd1fc21e6f718b P 233e7d29aac04505bf4e65f0a39a0c4c [term 2 FOLLOWER]: Rejecting Update request from peer 43999d813e564dc688fad116090c95ca for earlier term 1. Current term is 2. Ops: [1.1-1.1]
I20260709 13:23:37.393110 25832 leader_election.cc:290] T 87085ac2575e4b6a86dd1fc21e6f718b P 233e7d29aac04505bf4e65f0a39a0c4c [CANDIDATE]: Term 2 election: Requested vote from peers 43999d813e564dc688fad116090c95ca (127.14.177.129:38155), 3e07d64466934da4bb3c02262b1c2e38 (127.14.177.131:41539)
I20260709 13:23:37.396556 25505 tablet_bootstrap.cc:654] T 8ef061028f38450e83324ac78cdaa14c P 43999d813e564dc688fad116090c95ca: Neither blocks nor log segments found. Creating new log.
I20260709 13:23:37.391849 25307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ab90e3e08fb4528b2a47d995e9e26d3" candidate_uuid: "9868dce521294d0b816bba7648f4ae5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "233e7d29aac04505bf4e65f0a39a0c4c"
I20260709 13:23:37.406857 25508 raft_consensus.cc:359] T 0ed5edbe32044d1c83b0590dbb899801 P 9868dce521294d0b816bba7648f4ae5a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } }
I20260709 13:23:37.407588 25508 raft_consensus.cc:385] T 0ed5edbe32044d1c83b0590dbb899801 P 9868dce521294d0b816bba7648f4ae5a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:23:37.407840 25508 raft_consensus.cc:740] T 0ed5edbe32044d1c83b0590dbb899801 P 9868dce521294d0b816bba7648f4ae5a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9868dce521294d0b816bba7648f4ae5a, State: Initialized, Role: FOLLOWER
I20260709 13:23:37.408458 25508 consensus_queue.cc:260] T 0ed5edbe32044d1c83b0590dbb899801 P 9868dce521294d0b816bba7648f4ae5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } }
I20260709 13:23:37.409812 25503 tablet_bootstrap.cc:654] T 434a54fcee7d4aeeab42f739855c0e8a P 233e7d29aac04505bf4e65f0a39a0c4c: Neither blocks nor log segments found. Creating new log.
I20260709 13:23:37.403470 25307 raft_consensus.cc:3060] T 9ab90e3e08fb4528b2a47d995e9e26d3 P 233e7d29aac04505bf4e65f0a39a0c4c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:23:37.385448 25601 consensus_queue.cc:1048] T ce9c60e3922c4219b6e4453fe0259b6a P 43999d813e564dc688fad116090c95ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.022s
I20260709 13:23:37.424134 25504 tablet_bootstrap.cc:492] T b155b866f10547fd8239579a81ecf5f9 P 3e07d64466934da4bb3c02262b1c2e38: No bootstrap required, opened a new log
I20260709 13:23:37.424978 25504 ts_tablet_manager.cc:1403] T b155b866f10547fd8239579a81ecf5f9 P 3e07d64466934da4bb3c02262b1c2e38: Time spent bootstrapping tablet: real 0.239s	user 0.008s	sys 0.015s
I20260709 13:23:37.429885 25508 ts_tablet_manager.cc:1434] T 0ed5edbe32044d1c83b0590dbb899801 P 9868dce521294d0b816bba7648f4ae5a: Time spent starting tablet: real 0.050s	user 0.007s	sys 0.001s
I20260709 13:23:37.413064 25380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ab90e3e08fb4528b2a47d995e9e26d3" candidate_uuid: "9868dce521294d0b816bba7648f4ae5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e07d64466934da4bb3c02262b1c2e38"
I20260709 13:23:37.432447 25508 tablet_bootstrap.cc:492] T 4d837697cd0f402cacf9db89a5d92652 P 9868dce521294d0b816bba7648f4ae5a: Bootstrap starting.
I20260709 13:23:37.433681 25236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87085ac2575e4b6a86dd1fc21e6f718b" candidate_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43999d813e564dc688fad116090c95ca" is_pre_election: true
I20260709 13:23:37.446542 25385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87085ac2575e4b6a86dd1fc21e6f718b" candidate_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e07d64466934da4bb3c02262b1c2e38"
I20260709 13:23:37.449312 25232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87085ac2575e4b6a86dd1fc21e6f718b" candidate_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43999d813e564dc688fad116090c95ca"
I20260709 13:23:37.463388 25380 raft_consensus.cc:3060] T 9ab90e3e08fb4528b2a47d995e9e26d3 P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:23:37.471721 25307 raft_consensus.cc:2468] T 9ab90e3e08fb4528b2a47d995e9e26d3 P 233e7d29aac04505bf4e65f0a39a0c4c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9868dce521294d0b816bba7648f4ae5a in term 1.
I20260709 13:23:37.473657 25568 raft_consensus.cc:493] T b9d550d881b04735bbf89751952c09ec P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:23:37.474412 25568 raft_consensus.cc:515] T b9d550d881b04735bbf89751952c09ec P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } }
I20260709 13:23:37.478025 25458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9d550d881b04735bbf89751952c09ec" candidate_uuid: "3e07d64466934da4bb3c02262b1c2e38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9868dce521294d0b816bba7648f4ae5a" is_pre_election: true
I20260709 13:23:37.478340 25568 leader_election.cc:290] T b9d550d881b04735bbf89751952c09ec P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9868dce521294d0b816bba7648f4ae5a (127.14.177.132:33683), 43999d813e564dc688fad116090c95ca (127.14.177.129:38155)
W20260709 13:23:37.479602 25348 leader_election.cc:343] T b9d550d881b04735bbf89751952c09ec P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9868dce521294d0b816bba7648f4ae5a (127.14.177.132:33683): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:23:37.480000 25425 leader_election.cc:304] T 9ab90e3e08fb4528b2a47d995e9e26d3 P 9868dce521294d0b816bba7648f4ae5a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 233e7d29aac04505bf4e65f0a39a0c4c, 9868dce521294d0b816bba7648f4ae5a; no voters: 
I20260709 13:23:37.482198 25232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9d550d881b04735bbf89751952c09ec" candidate_uuid: "3e07d64466934da4bb3c02262b1c2e38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43999d813e564dc688fad116090c95ca" is_pre_election: true
I20260709 13:23:37.483982 25847 raft_consensus.cc:2804] T 9ab90e3e08fb4528b2a47d995e9e26d3 P 9868dce521294d0b816bba7648f4ae5a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:23:37.484572 25847 raft_consensus.cc:697] T 9ab90e3e08fb4528b2a47d995e9e26d3 P 9868dce521294d0b816bba7648f4ae5a [term 1 LEADER]: Becoming Leader. State: Replica: 9868dce521294d0b816bba7648f4ae5a, State: Running, Role: LEADER
I20260709 13:23:37.485416 25847 consensus_queue.cc:237] T 9ab90e3e08fb4528b2a47d995e9e26d3 P 9868dce521294d0b816bba7648f4ae5a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } }
W20260709 13:23:37.486742 25350 leader_election.cc:343] T b9d550d881b04735bbf89751952c09ec P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 43999d813e564dc688fad116090c95ca (127.14.177.129:38155): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:23:37.487351 25350 leader_election.cc:304] T b9d550d881b04735bbf89751952c09ec P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3e07d64466934da4bb3c02262b1c2e38; no voters: 43999d813e564dc688fad116090c95ca, 9868dce521294d0b816bba7648f4ae5a
I20260709 13:23:37.488933 25568 raft_consensus.cc:2749] T b9d550d881b04735bbf89751952c09ec P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:23:37.493770 25504 raft_consensus.cc:359] T b155b866f10547fd8239579a81ecf5f9 P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } }
I20260709 13:23:37.495862 25504 raft_consensus.cc:385] T b155b866f10547fd8239579a81ecf5f9 P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:23:37.496253 25504 raft_consensus.cc:740] T b155b866f10547fd8239579a81ecf5f9 P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e07d64466934da4bb3c02262b1c2e38, State: Initialized, Role: FOLLOWER
I20260709 13:23:37.494158 25601 consensus_queue.cc:1059] T 87085ac2575e4b6a86dd1fc21e6f718b P 43999d813e564dc688fad116090c95ca [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 }, Status: INVALID_TERM, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.004s
I20260709 13:23:37.509054 25505 tablet_bootstrap.cc:492] T 8ef061028f38450e83324ac78cdaa14c P 43999d813e564dc688fad116090c95ca: No bootstrap required, opened a new log
I20260709 13:23:37.511873 25504 consensus_queue.cc:260] T b155b866f10547fd8239579a81ecf5f9 P 3e07d64466934da4bb3c02262b1c2e38 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } }
I20260709 13:23:37.513638 25505 ts_tablet_manager.cc:1403] T 8ef061028f38450e83324ac78cdaa14c P 43999d813e564dc688fad116090c95ca: Time spent bootstrapping tablet: real 0.254s	user 0.021s	sys 0.029s
W20260709 13:23:37.516463 25859 log.cc:927] Time spent T f1ac2f4d2fe2424bb470df42199219e2 P 43999d813e564dc688fad116090c95ca: Append to log took a long time: real 0.099s	user 0.001s	sys 0.000s
I20260709 13:23:37.528717 25504 ts_tablet_manager.cc:1434] T b155b866f10547fd8239579a81ecf5f9 P 3e07d64466934da4bb3c02262b1c2e38: Time spent starting tablet: real 0.103s	user 0.006s	sys 0.004s
I20260709 13:23:37.529182 25503 tablet_bootstrap.cc:492] T 434a54fcee7d4aeeab42f739855c0e8a P 233e7d29aac04505bf4e65f0a39a0c4c: No bootstrap required, opened a new log
I20260709 13:23:37.531529 25503 ts_tablet_manager.cc:1403] T 434a54fcee7d4aeeab42f739855c0e8a P 233e7d29aac04505bf4e65f0a39a0c4c: Time spent bootstrapping tablet: real 0.169s	user 0.021s	sys 0.018s
I20260709 13:23:37.532035 25799 consensus_queue.cc:1048] T 434a18a5109e4ea78349ded3968b2709 P 233e7d29aac04505bf4e65f0a39a0c4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:23:37.534662 25503 raft_consensus.cc:359] T 434a54fcee7d4aeeab42f739855c0e8a P 233e7d29aac04505bf4e65f0a39a0c4c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } }
I20260709 13:23:37.535844 25503 raft_consensus.cc:385] T 434a54fcee7d4aeeab42f739855c0e8a P 233e7d29aac04505bf4e65f0a39a0c4c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:23:37.536541 25503 raft_consensus.cc:740] T 434a54fcee7d4aeeab42f739855c0e8a P 233e7d29aac04505bf4e65f0a39a0c4c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 233e7d29aac04505bf4e65f0a39a0c4c, State: Initialized, Role: FOLLOWER
I20260709 13:23:37.537312 25503 consensus_queue.cc:260] T 434a54fcee7d4aeeab42f739855c0e8a P 233e7d29aac04505bf4e65f0a39a0c4c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } }
I20260709 13:23:37.537429 25504 tablet_bootstrap.cc:492] T 72b56266e8a54776b9113c7a786e1952 P 3e07d64466934da4bb3c02262b1c2e38: Bootstrap starting.
I20260709 13:23:37.539342 25503 ts_tablet_manager.cc:1434] T 434a54fcee7d4aeeab42f739855c0e8a P 233e7d29aac04505bf4e65f0a39a0c4c: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.001s
I20260709 13:23:37.540282 25503 tablet_bootstrap.cc:492] T b85a238bbc3b4621881603a534c7bd9b P 233e7d29aac04505bf4e65f0a39a0c4c: Bootstrap starting.
I20260709 13:23:37.543077 25559 raft_consensus.cc:3055] T 87085ac2575e4b6a86dd1fc21e6f718b P 43999d813e564dc688fad116090c95ca [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:23:37.543625 25559 raft_consensus.cc:740] T 87085ac2575e4b6a86dd1fc21e6f718b P 43999d813e564dc688fad116090c95ca [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 43999d813e564dc688fad116090c95ca, State: Running, Role: LEADER
I20260709 13:23:37.544549 25559 consensus_queue.cc:260] T 87085ac2575e4b6a86dd1fc21e6f718b P 43999d813e564dc688fad116090c95ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } }
I20260709 13:23:37.546247 25503 tablet_bootstrap.cc:654] T b85a238bbc3b4621881603a534c7bd9b P 233e7d29aac04505bf4e65f0a39a0c4c: Neither blocks nor log segments found. Creating new log.
I20260709 13:23:37.553567 25508 tablet_bootstrap.cc:654] T 4d837697cd0f402cacf9db89a5d92652 P 9868dce521294d0b816bba7648f4ae5a: Neither blocks nor log segments found. Creating new log.
I20260709 13:23:37.552940 25380 raft_consensus.cc:2468] T 9ab90e3e08fb4528b2a47d995e9e26d3 P 3e07d64466934da4bb3c02262b1c2e38 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9868dce521294d0b816bba7648f4ae5a in term 1.
I20260709 13:23:37.566964 25559 raft_consensus.cc:3060] T 87085ac2575e4b6a86dd1fc21e6f718b P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:23:37.573249 25795 consensus_queue.cc:1048] T bbab48ba177049dcba6de106fd889cfb P 233e7d29aac04505bf4e65f0a39a0c4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:23:37.573508 25274 leader_election.cc:304] T 87085ac2575e4b6a86dd1fc21e6f718b P 233e7d29aac04505bf4e65f0a39a0c4c [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 233e7d29aac04505bf4e65f0a39a0c4c; no voters: 3e07d64466934da4bb3c02262b1c2e38, 43999d813e564dc688fad116090c95ca
I20260709 13:23:37.575716 25798 raft_consensus.cc:2749] T 87085ac2575e4b6a86dd1fc21e6f718b P 233e7d29aac04505bf4e65f0a39a0c4c [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:23:37.579470 25676 consensus_queue.cc:1048] T bbab48ba177049dcba6de106fd889cfb P 233e7d29aac04505bf4e65f0a39a0c4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:23:37.589522 25601 raft_consensus.cc:493] T 4d837697cd0f402cacf9db89a5d92652 P 43999d813e564dc688fad116090c95ca [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:23:37.590265 25601 raft_consensus.cc:515] T 4d837697cd0f402cacf9db89a5d92652 P 43999d813e564dc688fad116090c95ca [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } }
I20260709 13:23:37.593139 25601 leader_election.cc:290] T 4d837697cd0f402cacf9db89a5d92652 P 43999d813e564dc688fad116090c95ca [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3e07d64466934da4bb3c02262b1c2e38 (127.14.177.131:41539), 9868dce521294d0b816bba7648f4ae5a (127.14.177.132:33683)
I20260709 13:23:37.595997 25504 tablet_bootstrap.cc:654] T 72b56266e8a54776b9113c7a786e1952 P 3e07d64466934da4bb3c02262b1c2e38: Neither blocks nor log segments found. Creating new log.
I20260709 13:23:37.600164 25505 raft_consensus.cc:359] T 8ef061028f38450e83324ac78cdaa14c P 43999d813e564dc688fad116090c95ca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } }
I20260709 13:23:37.601130 25505 raft_consensus.cc:385] T 8ef061028f38450e83324ac78cdaa14c P 43999d813e564dc688fad116090c95ca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:23:37.605407 25458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d837697cd0f402cacf9db89a5d92652" candidate_uuid: "43999d813e564dc688fad116090c95ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9868dce521294d0b816bba7648f4ae5a" is_pre_election: true
I20260709 13:23:37.601449 25505 raft_consensus.cc:740] T 8ef061028f38450e83324ac78cdaa14c P 43999d813e564dc688fad116090c95ca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43999d813e564dc688fad116090c95ca, State: Initialized, Role: FOLLOWER
I20260709 13:23:37.609581 25505 consensus_queue.cc:260] T 8ef061028f38450e83324ac78cdaa14c P 43999d813e564dc688fad116090c95ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } }
I20260709 13:23:37.599409 25383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d837697cd0f402cacf9db89a5d92652" candidate_uuid: "43999d813e564dc688fad116090c95ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e07d64466934da4bb3c02262b1c2e38" is_pre_election: true
W20260709 13:23:37.608342 25199 leader_election.cc:343] T 4d837697cd0f402cacf9db89a5d92652 P 43999d813e564dc688fad116090c95ca [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9868dce521294d0b816bba7648f4ae5a (127.14.177.132:33683): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:23:37.613566 25505 ts_tablet_manager.cc:1434] T 8ef061028f38450e83324ac78cdaa14c P 43999d813e564dc688fad116090c95ca: Time spent starting tablet: real 0.098s	user 0.004s	sys 0.002s
I20260709 13:23:37.617265 25505 tablet_bootstrap.cc:492] T b155b866f10547fd8239579a81ecf5f9 P 43999d813e564dc688fad116090c95ca: Bootstrap starting.
I20260709 13:23:37.620945 25383 raft_consensus.cc:2468] T 4d837697cd0f402cacf9db89a5d92652 P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 43999d813e564dc688fad116090c95ca in term 0.
I20260709 13:23:37.638154 25199 leader_election.cc:304] T 4d837697cd0f402cacf9db89a5d92652 P 43999d813e564dc688fad116090c95ca [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3e07d64466934da4bb3c02262b1c2e38, 43999d813e564dc688fad116090c95ca; no voters: 9868dce521294d0b816bba7648f4ae5a
I20260709 13:23:37.639065 25503 tablet_bootstrap.cc:492] T b85a238bbc3b4621881603a534c7bd9b P 233e7d29aac04505bf4e65f0a39a0c4c: No bootstrap required, opened a new log
I20260709 13:23:37.639603 25503 ts_tablet_manager.cc:1403] T b85a238bbc3b4621881603a534c7bd9b P 233e7d29aac04505bf4e65f0a39a0c4c: Time spent bootstrapping tablet: real 0.100s	user 0.011s	sys 0.032s
I20260709 13:23:37.640970 25823 raft_consensus.cc:2804] T 4d837697cd0f402cacf9db89a5d92652 P 43999d813e564dc688fad116090c95ca [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:23:37.641450 25823 raft_consensus.cc:493] T 4d837697cd0f402cacf9db89a5d92652 P 43999d813e564dc688fad116090c95ca [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:23:37.641804 25823 raft_consensus.cc:3060] T 4d837697cd0f402cacf9db89a5d92652 P 43999d813e564dc688fad116090c95ca [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:23:37.642447 25503 raft_consensus.cc:359] T b85a238bbc3b4621881603a534c7bd9b P 233e7d29aac04505bf4e65f0a39a0c4c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } }
I20260709 13:23:37.647148 25503 raft_consensus.cc:385] T b85a238bbc3b4621881603a534c7bd9b P 233e7d29aac04505bf4e65f0a39a0c4c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:23:37.647495 25503 raft_consensus.cc:740] T b85a238bbc3b4621881603a534c7bd9b P 233e7d29aac04505bf4e65f0a39a0c4c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 233e7d29aac04505bf4e65f0a39a0c4c, State: Initialized, Role: FOLLOWER
I20260709 13:23:37.649288 25503 consensus_queue.cc:260] T b85a238bbc3b4621881603a534c7bd9b P 233e7d29aac04505bf4e65f0a39a0c4c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } }
I20260709 13:23:37.669512 25503 ts_tablet_manager.cc:1434] T b85a238bbc3b4621881603a534c7bd9b P 233e7d29aac04505bf4e65f0a39a0c4c: Time spent starting tablet: real 0.030s	user 0.007s	sys 0.000s
I20260709 13:23:37.670456 25568 raft_consensus.cc:493] T b155b866f10547fd8239579a81ecf5f9 P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:23:37.671094 25568 raft_consensus.cc:515] T b155b866f10547fd8239579a81ecf5f9 P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } }
I20260709 13:23:37.674346 25505 tablet_bootstrap.cc:654] T b155b866f10547fd8239579a81ecf5f9 P 43999d813e564dc688fad116090c95ca: Neither blocks nor log segments found. Creating new log.
I20260709 13:23:37.674487 25307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b155b866f10547fd8239579a81ecf5f9" candidate_uuid: "3e07d64466934da4bb3c02262b1c2e38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" is_pre_election: true
I20260709 13:23:37.675782 25503 tablet_bootstrap.cc:492] T cfad575709c943cba40ccab278350770 P 233e7d29aac04505bf4e65f0a39a0c4c: Bootstrap starting.
I20260709 13:23:37.678489 25568 leader_election.cc:290] T b155b866f10547fd8239579a81ecf5f9 P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 43999d813e564dc688fad116090c95ca (127.14.177.129:38155), 233e7d29aac04505bf4e65f0a39a0c4c (127.14.177.130:42957)
I20260709 13:23:37.670729 25823 raft_consensus.cc:515] T 4d837697cd0f402cacf9db89a5d92652 P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } }
W20260709 13:23:37.686197 25351 leader_election.cc:343] T b155b866f10547fd8239579a81ecf5f9 P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 233e7d29aac04505bf4e65f0a39a0c4c (127.14.177.130:42957): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:23:37.695128 25503 tablet_bootstrap.cc:654] T cfad575709c943cba40ccab278350770 P 233e7d29aac04505bf4e65f0a39a0c4c: Neither blocks nor log segments found. Creating new log.
I20260709 13:23:37.712927 25508 tablet_bootstrap.cc:492] T 4d837697cd0f402cacf9db89a5d92652 P 9868dce521294d0b816bba7648f4ae5a: No bootstrap required, opened a new log
I20260709 13:23:37.713732 25508 ts_tablet_manager.cc:1403] T 4d837697cd0f402cacf9db89a5d92652 P 9868dce521294d0b816bba7648f4ae5a: Time spent bootstrapping tablet: real 0.282s	user 0.026s	sys 0.013s
I20260709 13:23:37.700707 25232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b155b866f10547fd8239579a81ecf5f9" candidate_uuid: "3e07d64466934da4bb3c02262b1c2e38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43999d813e564dc688fad116090c95ca" is_pre_election: true
I20260709 13:23:37.727018 25504 tablet_bootstrap.cc:492] T 72b56266e8a54776b9113c7a786e1952 P 3e07d64466934da4bb3c02262b1c2e38: No bootstrap required, opened a new log
I20260709 13:23:37.727716 25504 ts_tablet_manager.cc:1403] T 72b56266e8a54776b9113c7a786e1952 P 3e07d64466934da4bb3c02262b1c2e38: Time spent bootstrapping tablet: real 0.198s	user 0.024s	sys 0.011s
I20260709 13:23:37.731297 25504 raft_consensus.cc:359] T 72b56266e8a54776b9113c7a786e1952 P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } }
I20260709 13:23:37.732142 25504 raft_consensus.cc:385] T 72b56266e8a54776b9113c7a786e1952 P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:23:37.732478 25504 raft_consensus.cc:740] T 72b56266e8a54776b9113c7a786e1952 P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e07d64466934da4bb3c02262b1c2e38, State: Initialized, Role: FOLLOWER
W20260709 13:23:37.737028 25182 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:23:37.735543 25131 catalog_manager.cc:5697] T 434a18a5109e4ea78349ded3968b2709 P 9868dce521294d0b816bba7648f4ae5a reported cstate change: term changed from 0 to 1, leader changed from <none> to 233e7d29aac04505bf4e65f0a39a0c4c (127.14.177.130). New cstate: current_term: 1 leader_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } }
I20260709 13:23:37.741528 25131 catalog_manager.cc:5697] T 9ab90e3e08fb4528b2a47d995e9e26d3 P 9868dce521294d0b816bba7648f4ae5a reported cstate change: term changed from 0 to 1, leader changed from <none> to 9868dce521294d0b816bba7648f4ae5a (127.14.177.132). New cstate: current_term: 1 leader_uuid: "9868dce521294d0b816bba7648f4ae5a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } health_report { overall_health: HEALTHY } } }
I20260709 13:23:37.753242 25380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d837697cd0f402cacf9db89a5d92652" candidate_uuid: "43999d813e564dc688fad116090c95ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e07d64466934da4bb3c02262b1c2e38"
I20260709 13:23:37.753974 25380 raft_consensus.cc:3060] T 4d837697cd0f402cacf9db89a5d92652 P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:23:37.757169 25808 raft_consensus.cc:493] T 0ed5edbe32044d1c83b0590dbb899801 P 9868dce521294d0b816bba7648f4ae5a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:23:37.757717 25808 raft_consensus.cc:515] T 0ed5edbe32044d1c83b0590dbb899801 P 9868dce521294d0b816bba7648f4ae5a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } }
I20260709 13:23:37.759294 25508 raft_consensus.cc:359] T 4d837697cd0f402cacf9db89a5d92652 P 9868dce521294d0b816bba7648f4ae5a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } }
I20260709 13:23:37.759825 25808 leader_election.cc:290] T 0ed5edbe32044d1c83b0590dbb899801 P 9868dce521294d0b816bba7648f4ae5a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3e07d64466934da4bb3c02262b1c2e38 (127.14.177.131:41539), 43999d813e564dc688fad116090c95ca (127.14.177.129:38155)
I20260709 13:23:37.760701 25508 raft_consensus.cc:385] T 4d837697cd0f402cacf9db89a5d92652 P 9868dce521294d0b816bba7648f4ae5a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:23:37.761219 25508 raft_consensus.cc:740] T 4d837697cd0f402cacf9db89a5d92652 P 9868dce521294d0b816bba7648f4ae5a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9868dce521294d0b816bba7648f4ae5a, State: Initialized, Role: FOLLOWER
I20260709 13:23:37.762063 25799 raft_consensus.cc:493] T b85a238bbc3b4621881603a534c7bd9b P 233e7d29aac04505bf4e65f0a39a0c4c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:23:37.762580 25799 raft_consensus.cc:515] T b85a238bbc3b4621881603a534c7bd9b P 233e7d29aac04505bf4e65f0a39a0c4c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } }
I20260709 13:23:37.766657 25799 leader_election.cc:290] T b85a238bbc3b4621881603a534c7bd9b P 233e7d29aac04505bf4e65f0a39a0c4c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9868dce521294d0b816bba7648f4ae5a (127.14.177.132:33683), 43999d813e564dc688fad116090c95ca (127.14.177.129:38155)
I20260709 13:23:37.767731 25233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ed5edbe32044d1c83b0590dbb899801" candidate_uuid: "9868dce521294d0b816bba7648f4ae5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43999d813e564dc688fad116090c95ca" is_pre_election: true
I20260709 13:23:37.768901 25382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ed5edbe32044d1c83b0590dbb899801" candidate_uuid: "9868dce521294d0b816bba7648f4ae5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e07d64466934da4bb3c02262b1c2e38" is_pre_election: true
I20260709 13:23:37.769358 25233 raft_consensus.cc:2393] T 0ed5edbe32044d1c83b0590dbb899801 P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9868dce521294d0b816bba7648f4ae5a in current term 1: Already voted for candidate 3e07d64466934da4bb3c02262b1c2e38 in this term.
I20260709 13:23:37.771396 25455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b85a238bbc3b4621881603a534c7bd9b" candidate_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9868dce521294d0b816bba7648f4ae5a" is_pre_election: true
I20260709 13:23:37.772511 25503 tablet_bootstrap.cc:492] T cfad575709c943cba40ccab278350770 P 233e7d29aac04505bf4e65f0a39a0c4c: No bootstrap required, opened a new log
I20260709 13:23:37.773023 25503 ts_tablet_manager.cc:1403] T cfad575709c943cba40ccab278350770 P 233e7d29aac04505bf4e65f0a39a0c4c: Time spent bootstrapping tablet: real 0.097s	user 0.016s	sys 0.014s
I20260709 13:23:37.767913 25232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b85a238bbc3b4621881603a534c7bd9b" candidate_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43999d813e564dc688fad116090c95ca" is_pre_election: true
W20260709 13:23:37.774394 25274 leader_election.cc:343] T b85a238bbc3b4621881603a534c7bd9b P 233e7d29aac04505bf4e65f0a39a0c4c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9868dce521294d0b816bba7648f4ae5a (127.14.177.132:33683): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:23:37.774506 25232 raft_consensus.cc:2468] T b85a238bbc3b4621881603a534c7bd9b P 43999d813e564dc688fad116090c95ca [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 233e7d29aac04505bf4e65f0a39a0c4c in term 0.
I20260709 13:23:37.775321 25823 leader_election.cc:290] T 4d837697cd0f402cacf9db89a5d92652 P 43999d813e564dc688fad116090c95ca [CANDIDATE]: Term 1 election: Requested vote from peers 3e07d64466934da4bb3c02262b1c2e38 (127.14.177.131:41539), 9868dce521294d0b816bba7648f4ae5a (127.14.177.132:33683)
I20260709 13:23:37.775978 25503 raft_consensus.cc:359] T cfad575709c943cba40ccab278350770 P 233e7d29aac04505bf4e65f0a39a0c4c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } }
I20260709 13:23:37.776345 25455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d837697cd0f402cacf9db89a5d92652" candidate_uuid: "43999d813e564dc688fad116090c95ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9868dce521294d0b816bba7648f4ae5a"
I20260709 13:23:37.776691 25503 raft_consensus.cc:385] T cfad575709c943cba40ccab278350770 P 233e7d29aac04505bf4e65f0a39a0c4c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:23:37.777137 25503 raft_consensus.cc:740] T cfad575709c943cba40ccab278350770 P 233e7d29aac04505bf4e65f0a39a0c4c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 233e7d29aac04505bf4e65f0a39a0c4c, State: Initialized, Role: FOLLOWER
I20260709 13:23:37.777848 25503 consensus_queue.cc:260] T cfad575709c943cba40ccab278350770 P 233e7d29aac04505bf4e65f0a39a0c4c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } }
I20260709 13:23:37.775599 25504 consensus_queue.cc:260] T 72b56266e8a54776b9113c7a786e1952 P 3e07d64466934da4bb3c02262b1c2e38 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } }
I20260709 13:23:37.772881 25380 raft_consensus.cc:2468] T 4d837697cd0f402cacf9db89a5d92652 P 3e07d64466934da4bb3c02262b1c2e38 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 43999d813e564dc688fad116090c95ca in term 1.
I20260709 13:23:37.780051 25503 ts_tablet_manager.cc:1434] T cfad575709c943cba40ccab278350770 P 233e7d29aac04505bf4e65f0a39a0c4c: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 13:23:37.781010 25503 tablet_bootstrap.cc:492] T 8ef061028f38450e83324ac78cdaa14c P 233e7d29aac04505bf4e65f0a39a0c4c: Bootstrap starting.
I20260709 13:23:37.781312 25504 ts_tablet_manager.cc:1434] T 72b56266e8a54776b9113c7a786e1952 P 3e07d64466934da4bb3c02262b1c2e38: Time spent starting tablet: real 0.053s	user 0.005s	sys 0.002s
I20260709 13:23:37.781653 25199 leader_election.cc:304] T 4d837697cd0f402cacf9db89a5d92652 P 43999d813e564dc688fad116090c95ca [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e07d64466934da4bb3c02262b1c2e38, 43999d813e564dc688fad116090c95ca; no voters: 
I20260709 13:23:37.783092 25822 raft_consensus.cc:2804] T 4d837697cd0f402cacf9db89a5d92652 P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:23:37.783656 25822 raft_consensus.cc:697] T 4d837697cd0f402cacf9db89a5d92652 P 43999d813e564dc688fad116090c95ca [term 1 LEADER]: Becoming Leader. State: Replica: 43999d813e564dc688fad116090c95ca, State: Running, Role: LEADER
I20260709 13:23:37.784503 25822 consensus_queue.cc:237] T 4d837697cd0f402cacf9db89a5d92652 P 43999d813e564dc688fad116090c95ca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } }
I20260709 13:23:37.785422 25505 tablet_bootstrap.cc:492] T b155b866f10547fd8239579a81ecf5f9 P 43999d813e564dc688fad116090c95ca: No bootstrap required, opened a new log
I20260709 13:23:37.785583 25309 raft_consensus.cc:3060] T 434a54fcee7d4aeeab42f739855c0e8a P 233e7d29aac04505bf4e65f0a39a0c4c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:23:37.785959 25505 ts_tablet_manager.cc:1403] T b155b866f10547fd8239579a81ecf5f9 P 43999d813e564dc688fad116090c95ca: Time spent bootstrapping tablet: real 0.172s	user 0.018s	sys 0.035s
I20260709 13:23:37.788942 25505 raft_consensus.cc:359] T b155b866f10547fd8239579a81ecf5f9 P 43999d813e564dc688fad116090c95ca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } }
I20260709 13:23:37.789800 25505 raft_consensus.cc:385] T b155b866f10547fd8239579a81ecf5f9 P 43999d813e564dc688fad116090c95ca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:23:37.790156 25505 raft_consensus.cc:740] T b155b866f10547fd8239579a81ecf5f9 P 43999d813e564dc688fad116090c95ca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43999d813e564dc688fad116090c95ca, State: Initialized, Role: FOLLOWER
I20260709 13:23:37.790825 25505 consensus_queue.cc:260] T b155b866f10547fd8239579a81ecf5f9 P 43999d813e564dc688fad116090c95ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } }
I20260709 13:23:37.792847 25276 leader_election.cc:304] T b85a238bbc3b4621881603a534c7bd9b P 233e7d29aac04505bf4e65f0a39a0c4c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 233e7d29aac04505bf4e65f0a39a0c4c, 43999d813e564dc688fad116090c95ca; no voters: 9868dce521294d0b816bba7648f4ae5a
I20260709 13:23:37.793710 25798 raft_consensus.cc:2804] T b85a238bbc3b4621881603a534c7bd9b P 233e7d29aac04505bf4e65f0a39a0c4c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:23:37.794096 25798 raft_consensus.cc:493] T b85a238bbc3b4621881603a534c7bd9b P 233e7d29aac04505bf4e65f0a39a0c4c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:23:37.794456 25798 raft_consensus.cc:3060] T b85a238bbc3b4621881603a534c7bd9b P 233e7d29aac04505bf4e65f0a39a0c4c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:23:37.802177 25508 consensus_queue.cc:260] T 4d837697cd0f402cacf9db89a5d92652 P 9868dce521294d0b816bba7648f4ae5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } }
I20260709 13:23:37.803699 25505 ts_tablet_manager.cc:1434] T b155b866f10547fd8239579a81ecf5f9 P 43999d813e564dc688fad116090c95ca: Time spent starting tablet: real 0.017s	user 0.008s	sys 0.000s
I20260709 13:23:37.804592 25505 tablet_bootstrap.cc:492] T b9d550d881b04735bbf89751952c09ec P 43999d813e564dc688fad116090c95ca: Bootstrap starting.
W20260709 13:23:37.801777 25350 leader_election.cc:343] T b155b866f10547fd8239579a81ecf5f9 P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 43999d813e564dc688fad116090c95ca (127.14.177.129:38155): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:23:37.805789 25350 leader_election.cc:304] T b155b866f10547fd8239579a81ecf5f9 P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3e07d64466934da4bb3c02262b1c2e38; no voters: 233e7d29aac04505bf4e65f0a39a0c4c, 43999d813e564dc688fad116090c95ca
I20260709 13:23:37.806473 25568 raft_consensus.cc:2749] T b155b866f10547fd8239579a81ecf5f9 P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:23:37.806871 25504 tablet_bootstrap.cc:492] T e311eb90bad44c80a131b72edcd22b06 P 3e07d64466934da4bb3c02262b1c2e38: Bootstrap starting.
I20260709 13:23:37.807233 25455 raft_consensus.cc:3060] T 4d837697cd0f402cacf9db89a5d92652 P 9868dce521294d0b816bba7648f4ae5a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:23:37.813263 25503 tablet_bootstrap.cc:654] T 8ef061028f38450e83324ac78cdaa14c P 233e7d29aac04505bf4e65f0a39a0c4c: Neither blocks nor log segments found. Creating new log.
I20260709 13:23:37.814581 25798 raft_consensus.cc:515] T b85a238bbc3b4621881603a534c7bd9b P 233e7d29aac04505bf4e65f0a39a0c4c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } }
I20260709 13:23:37.815428 25455 raft_consensus.cc:2468] T 4d837697cd0f402cacf9db89a5d92652 P 9868dce521294d0b816bba7648f4ae5a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 43999d813e564dc688fad116090c95ca in term 1.
I20260709 13:23:37.817289 25798 leader_election.cc:290] T b85a238bbc3b4621881603a534c7bd9b P 233e7d29aac04505bf4e65f0a39a0c4c [CANDIDATE]: Term 1 election: Requested vote from peers 9868dce521294d0b816bba7648f4ae5a (127.14.177.132:33683), 43999d813e564dc688fad116090c95ca (127.14.177.129:38155)
I20260709 13:23:37.818415 25822 raft_consensus.cc:493] T b155b866f10547fd8239579a81ecf5f9 P 43999d813e564dc688fad116090c95ca [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:23:37.818396 25422 leader_election.cc:304] T 0ed5edbe32044d1c83b0590dbb899801 P 9868dce521294d0b816bba7648f4ae5a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9868dce521294d0b816bba7648f4ae5a; no voters: 3e07d64466934da4bb3c02262b1c2e38, 43999d813e564dc688fad116090c95ca
I20260709 13:23:37.819104 25822 raft_consensus.cc:515] T b155b866f10547fd8239579a81ecf5f9 P 43999d813e564dc688fad116090c95ca [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } }
I20260709 13:23:37.819972 25847 raft_consensus.cc:3060] T 0ed5edbe32044d1c83b0590dbb899801 P 9868dce521294d0b816bba7648f4ae5a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:23:37.820072 25508 ts_tablet_manager.cc:1434] T 4d837697cd0f402cacf9db89a5d92652 P 9868dce521294d0b816bba7648f4ae5a: Time spent starting tablet: real 0.106s	user 0.010s	sys 0.001s
I20260709 13:23:37.821452 25508 tablet_bootstrap.cc:492] T cfad575709c943cba40ccab278350770 P 9868dce521294d0b816bba7648f4ae5a: Bootstrap starting.
I20260709 13:23:37.822419 25380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b155b866f10547fd8239579a81ecf5f9" candidate_uuid: "43999d813e564dc688fad116090c95ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e07d64466934da4bb3c02262b1c2e38" is_pre_election: true
I20260709 13:23:37.823053 25380 raft_consensus.cc:2468] T b155b866f10547fd8239579a81ecf5f9 P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 43999d813e564dc688fad116090c95ca in term 0.
I20260709 13:23:37.805550 25129 catalog_manager.cc:5697] T 434a54fcee7d4aeeab42f739855c0e8a P 43999d813e564dc688fad116090c95ca reported cstate change: term changed from 0 to 1, leader changed from <none> to 43999d813e564dc688fad116090c95ca (127.14.177.129). New cstate: current_term: 1 leader_uuid: "43999d813e564dc688fad116090c95ca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } health_report { overall_health: UNKNOWN } } }
I20260709 13:23:37.824028 25199 leader_election.cc:304] T b155b866f10547fd8239579a81ecf5f9 P 43999d813e564dc688fad116090c95ca [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e07d64466934da4bb3c02262b1c2e38, 43999d813e564dc688fad116090c95ca; no voters: 
I20260709 13:23:37.828105 25823 raft_consensus.cc:2804] T b155b866f10547fd8239579a81ecf5f9 P 43999d813e564dc688fad116090c95ca [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:23:37.829234 25458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b85a238bbc3b4621881603a534c7bd9b" candidate_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9868dce521294d0b816bba7648f4ae5a"
I20260709 13:23:37.835017 25676 raft_consensus.cc:493] T cfad575709c943cba40ccab278350770 P 233e7d29aac04505bf4e65f0a39a0c4c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:23:37.835717 25274 leader_election.cc:343] T b85a238bbc3b4621881603a534c7bd9b P 233e7d29aac04505bf4e65f0a39a0c4c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 9868dce521294d0b816bba7648f4ae5a (127.14.177.132:33683): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:23:37.835575 25676 raft_consensus.cc:515] T cfad575709c943cba40ccab278350770 P 233e7d29aac04505bf4e65f0a39a0c4c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } }
I20260709 13:23:37.836779 25847 raft_consensus.cc:2749] T 0ed5edbe32044d1c83b0590dbb899801 P 9868dce521294d0b816bba7648f4ae5a [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:23:37.839123 25232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b85a238bbc3b4621881603a534c7bd9b" candidate_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43999d813e564dc688fad116090c95ca"
I20260709 13:23:37.839743 25307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b155b866f10547fd8239579a81ecf5f9" candidate_uuid: "43999d813e564dc688fad116090c95ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" is_pre_election: true
I20260709 13:23:37.839993 25232 raft_consensus.cc:3060] T b85a238bbc3b4621881603a534c7bd9b P 43999d813e564dc688fad116090c95ca [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:23:37.841686 25568 consensus_queue.cc:1048] T 0ed5edbe32044d1c83b0590dbb899801 P 3e07d64466934da4bb3c02262b1c2e38 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 13:23:37.841888 25202 leader_election.cc:343] T b155b866f10547fd8239579a81ecf5f9 P 43999d813e564dc688fad116090c95ca [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 233e7d29aac04505bf4e65f0a39a0c4c (127.14.177.130:42957): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:23:37.844339 25568 consensus_queue.cc:1048] T 0ed5edbe32044d1c83b0590dbb899801 P 3e07d64466934da4bb3c02262b1c2e38 [LEADER]: Connected to new peer: Peer: permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:23:37.837800 25676 leader_election.cc:290] T cfad575709c943cba40ccab278350770 P 233e7d29aac04505bf4e65f0a39a0c4c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3e07d64466934da4bb3c02262b1c2e38 (127.14.177.131:41539), 9868dce521294d0b816bba7648f4ae5a (127.14.177.132:33683)
I20260709 13:23:37.850586 25380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfad575709c943cba40ccab278350770" candidate_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e07d64466934da4bb3c02262b1c2e38" is_pre_election: true
I20260709 13:23:37.852638 25380 raft_consensus.cc:2468] T cfad575709c943cba40ccab278350770 P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 233e7d29aac04505bf4e65f0a39a0c4c in term 0.
I20260709 13:23:37.854014 25274 leader_election.cc:304] T cfad575709c943cba40ccab278350770 P 233e7d29aac04505bf4e65f0a39a0c4c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 233e7d29aac04505bf4e65f0a39a0c4c, 3e07d64466934da4bb3c02262b1c2e38; no voters: 
I20260709 13:23:37.855021 25676 raft_consensus.cc:2804] T cfad575709c943cba40ccab278350770 P 233e7d29aac04505bf4e65f0a39a0c4c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:23:37.855535 25676 raft_consensus.cc:493] T cfad575709c943cba40ccab278350770 P 233e7d29aac04505bf4e65f0a39a0c4c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:23:37.855931 25676 raft_consensus.cc:3060] T cfad575709c943cba40ccab278350770 P 233e7d29aac04505bf4e65f0a39a0c4c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:23:37.828610 25823 raft_consensus.cc:493] T b155b866f10547fd8239579a81ecf5f9 P 43999d813e564dc688fad116090c95ca [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:23:37.856739 25823 raft_consensus.cc:3060] T b155b866f10547fd8239579a81ecf5f9 P 43999d813e564dc688fad116090c95ca [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:23:37.827741 25822 leader_election.cc:290] T b155b866f10547fd8239579a81ecf5f9 P 43999d813e564dc688fad116090c95ca [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 233e7d29aac04505bf4e65f0a39a0c4c (127.14.177.130:42957), 3e07d64466934da4bb3c02262b1c2e38 (127.14.177.131:41539)
I20260709 13:23:37.865007 25455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfad575709c943cba40ccab278350770" candidate_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9868dce521294d0b816bba7648f4ae5a" is_pre_election: true
I20260709 13:23:37.867370 25508 tablet_bootstrap.cc:654] T cfad575709c943cba40ccab278350770 P 9868dce521294d0b816bba7648f4ae5a: Neither blocks nor log segments found. Creating new log.
I20260709 13:23:37.868712 25503 tablet_bootstrap.cc:492] T 8ef061028f38450e83324ac78cdaa14c P 233e7d29aac04505bf4e65f0a39a0c4c: No bootstrap required, opened a new log
I20260709 13:23:37.869268 25503 ts_tablet_manager.cc:1403] T 8ef061028f38450e83324ac78cdaa14c P 233e7d29aac04505bf4e65f0a39a0c4c: Time spent bootstrapping tablet: real 0.088s	user 0.024s	sys 0.009s
I20260709 13:23:37.850998 25232 raft_consensus.cc:2468] T b85a238bbc3b4621881603a534c7bd9b P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 233e7d29aac04505bf4e65f0a39a0c4c in term 1.
I20260709 13:23:37.871212 25276 leader_election.cc:304] T b85a238bbc3b4621881603a534c7bd9b P 233e7d29aac04505bf4e65f0a39a0c4c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 233e7d29aac04505bf4e65f0a39a0c4c, 43999d813e564dc688fad116090c95ca; no voters: 9868dce521294d0b816bba7648f4ae5a
I20260709 13:23:37.872370 25795 raft_consensus.cc:2804] T b85a238bbc3b4621881603a534c7bd9b P 233e7d29aac04505bf4e65f0a39a0c4c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:23:37.872062 25503 raft_consensus.cc:359] T 8ef061028f38450e83324ac78cdaa14c P 233e7d29aac04505bf4e65f0a39a0c4c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } }
I20260709 13:23:37.872900 25503 raft_consensus.cc:385] T 8ef061028f38450e83324ac78cdaa14c P 233e7d29aac04505bf4e65f0a39a0c4c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:23:37.872946 25795 raft_consensus.cc:697] T b85a238bbc3b4621881603a534c7bd9b P 233e7d29aac04505bf4e65f0a39a0c4c [term 1 LEADER]: Becoming Leader. State: Replica: 233e7d29aac04505bf4e65f0a39a0c4c, State: Running, Role: LEADER
I20260709 13:23:37.873282 25503 raft_consensus.cc:740] T 8ef061028f38450e83324ac78cdaa14c P 233e7d29aac04505bf4e65f0a39a0c4c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 233e7d29aac04505bf4e65f0a39a0c4c, State: Initialized, Role: FOLLOWER
I20260709 13:23:37.874614 25503 consensus_queue.cc:260] T 8ef061028f38450e83324ac78cdaa14c P 233e7d29aac04505bf4e65f0a39a0c4c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } }
I20260709 13:23:37.875633 25795 consensus_queue.cc:237] T b85a238bbc3b4621881603a534c7bd9b P 233e7d29aac04505bf4e65f0a39a0c4c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } }
I20260709 13:23:37.876681 25503 ts_tablet_manager.cc:1434] T 8ef061028f38450e83324ac78cdaa14c P 233e7d29aac04505bf4e65f0a39a0c4c: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 13:23:37.877297 25799 raft_consensus.cc:493] T 434a54fcee7d4aeeab42f739855c0e8a P 233e7d29aac04505bf4e65f0a39a0c4c [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:23:37.877897 25503 tablet_bootstrap.cc:492] T b155b866f10547fd8239579a81ecf5f9 P 233e7d29aac04505bf4e65f0a39a0c4c: Bootstrap starting.
I20260709 13:23:37.877820 25799 raft_consensus.cc:515] T 434a54fcee7d4aeeab42f739855c0e8a P 233e7d29aac04505bf4e65f0a39a0c4c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } }
I20260709 13:23:37.882195 25799 leader_election.cc:290] T 434a54fcee7d4aeeab42f739855c0e8a P 233e7d29aac04505bf4e65f0a39a0c4c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 43999d813e564dc688fad116090c95ca (127.14.177.129:38155), 3e07d64466934da4bb3c02262b1c2e38 (127.14.177.131:41539)
I20260709 13:23:37.882824 25233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434a54fcee7d4aeeab42f739855c0e8a" candidate_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43999d813e564dc688fad116090c95ca" is_pre_election: true
I20260709 13:23:37.885289 25822 consensus_queue.cc:1048] T 434a54fcee7d4aeeab42f739855c0e8a P 43999d813e564dc688fad116090c95ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:23:37.885972 25382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434a54fcee7d4aeeab42f739855c0e8a" candidate_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e07d64466934da4bb3c02262b1c2e38" is_pre_election: true
I20260709 13:23:37.885689 25130 catalog_manager.cc:5697] T 0ed5edbe32044d1c83b0590dbb899801 P 3e07d64466934da4bb3c02262b1c2e38 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3e07d64466934da4bb3c02262b1c2e38 (127.14.177.131). New cstate: current_term: 1 leader_uuid: "3e07d64466934da4bb3c02262b1c2e38" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } health_report { overall_health: UNKNOWN } } }
I20260709 13:23:37.886722 25382 raft_consensus.cc:2468] T 434a54fcee7d4aeeab42f739855c0e8a P 3e07d64466934da4bb3c02262b1c2e38 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 233e7d29aac04505bf4e65f0a39a0c4c in term 1.
W20260709 13:23:37.890409 25274 leader_election.cc:343] T cfad575709c943cba40ccab278350770 P 233e7d29aac04505bf4e65f0a39a0c4c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9868dce521294d0b816bba7648f4ae5a (127.14.177.132:33683): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:23:37.894546 25274 leader_election.cc:304] T 434a54fcee7d4aeeab42f739855c0e8a P 233e7d29aac04505bf4e65f0a39a0c4c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 233e7d29aac04505bf4e65f0a39a0c4c, 3e07d64466934da4bb3c02262b1c2e38; no voters: 43999d813e564dc688fad116090c95ca
I20260709 13:23:37.899849 25508 tablet_bootstrap.cc:492] T cfad575709c943cba40ccab278350770 P 9868dce521294d0b816bba7648f4ae5a: No bootstrap required, opened a new log
I20260709 13:23:37.900420 25508 ts_tablet_manager.cc:1403] T cfad575709c943cba40ccab278350770 P 9868dce521294d0b816bba7648f4ae5a: Time spent bootstrapping tablet: real 0.079s	user 0.009s	sys 0.005s
I20260709 13:23:37.901670 25823 raft_consensus.cc:515] T b155b866f10547fd8239579a81ecf5f9 P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } }
I20260709 13:23:37.904708 25504 tablet_bootstrap.cc:654] T e311eb90bad44c80a131b72edcd22b06 P 3e07d64466934da4bb3c02262b1c2e38: Neither blocks nor log segments found. Creating new log.
I20260709 13:23:37.904942 25307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b155b866f10547fd8239579a81ecf5f9" candidate_uuid: "43999d813e564dc688fad116090c95ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "233e7d29aac04505bf4e65f0a39a0c4c"
W20260709 13:23:37.914567 25202 leader_election.cc:343] T b155b866f10547fd8239579a81ecf5f9 P 43999d813e564dc688fad116090c95ca [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 233e7d29aac04505bf4e65f0a39a0c4c (127.14.177.130:42957): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:23:37.915208 25505 tablet_bootstrap.cc:654] T b9d550d881b04735bbf89751952c09ec P 43999d813e564dc688fad116090c95ca: Neither blocks nor log segments found. Creating new log.
I20260709 13:23:37.909425 25676 raft_consensus.cc:515] T cfad575709c943cba40ccab278350770 P 233e7d29aac04505bf4e65f0a39a0c4c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } }
I20260709 13:23:37.913753 25508 raft_consensus.cc:359] T cfad575709c943cba40ccab278350770 P 9868dce521294d0b816bba7648f4ae5a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } }
I20260709 13:23:37.919545 25508 raft_consensus.cc:385] T cfad575709c943cba40ccab278350770 P 9868dce521294d0b816bba7648f4ae5a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:23:37.919901 25508 raft_consensus.cc:740] T cfad575709c943cba40ccab278350770 P 9868dce521294d0b816bba7648f4ae5a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9868dce521294d0b816bba7648f4ae5a, State: Initialized, Role: FOLLOWER
I20260709 13:23:37.922020 25508 consensus_queue.cc:260] T cfad575709c943cba40ccab278350770 P 9868dce521294d0b816bba7648f4ae5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } }
I20260709 13:23:37.931274 25676 leader_election.cc:290] T cfad575709c943cba40ccab278350770 P 233e7d29aac04505bf4e65f0a39a0c4c [CANDIDATE]: Term 1 election: Requested vote from peers 3e07d64466934da4bb3c02262b1c2e38 (127.14.177.131:41539), 9868dce521294d0b816bba7648f4ae5a (127.14.177.132:33683)
I20260709 13:23:37.933959 25503 tablet_bootstrap.cc:654] T b155b866f10547fd8239579a81ecf5f9 P 233e7d29aac04505bf4e65f0a39a0c4c: Neither blocks nor log segments found. Creating new log.
I20260709 13:23:37.935981 25455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfad575709c943cba40ccab278350770" candidate_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9868dce521294d0b816bba7648f4ae5a"
I20260709 13:23:37.940126 25508 ts_tablet_manager.cc:1434] T cfad575709c943cba40ccab278350770 P 9868dce521294d0b816bba7648f4ae5a: Time spent starting tablet: real 0.039s	user 0.008s	sys 0.000s
I20260709 13:23:37.943213 25508 tablet_bootstrap.cc:492] T e311eb90bad44c80a131b72edcd22b06 P 9868dce521294d0b816bba7648f4ae5a: Bootstrap starting.
I20260709 13:23:37.954528 25382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b155b866f10547fd8239579a81ecf5f9" candidate_uuid: "43999d813e564dc688fad116090c95ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e07d64466934da4bb3c02262b1c2e38"
I20260709 13:23:37.955440 25382 raft_consensus.cc:3060] T b155b866f10547fd8239579a81ecf5f9 P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:23:37.967586 25650 consensus_queue.cc:1048] T 9ab90e3e08fb4528b2a47d995e9e26d3 P 9868dce521294d0b816bba7648f4ae5a [LEADER]: Connected to new peer: Peer: permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:23:37.972937 25380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfad575709c943cba40ccab278350770" candidate_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e07d64466934da4bb3c02262b1c2e38"
I20260709 13:23:37.973829 25380 raft_consensus.cc:3060] T cfad575709c943cba40ccab278350770 P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:23:37.980564 25795 raft_consensus.cc:2804] T 434a54fcee7d4aeeab42f739855c0e8a P 233e7d29aac04505bf4e65f0a39a0c4c [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:23:37.963903 25823 leader_election.cc:290] T b155b866f10547fd8239579a81ecf5f9 P 43999d813e564dc688fad116090c95ca [CANDIDATE]: Term 1 election: Requested vote from peers 233e7d29aac04505bf4e65f0a39a0c4c (127.14.177.130:42957), 3e07d64466934da4bb3c02262b1c2e38 (127.14.177.131:41539)
I20260709 13:23:37.950881 25455 raft_consensus.cc:3060] T cfad575709c943cba40ccab278350770 P 9868dce521294d0b816bba7648f4ae5a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:23:37.982870 25795 raft_consensus.cc:493] T 434a54fcee7d4aeeab42f739855c0e8a P 233e7d29aac04505bf4e65f0a39a0c4c [term 1 FOLLOWER]: Starting leader election (detected failure of leader 43999d813e564dc688fad116090c95ca)
I20260709 13:23:37.983451 25795 raft_consensus.cc:3060] T 434a54fcee7d4aeeab42f739855c0e8a P 233e7d29aac04505bf4e65f0a39a0c4c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:23:37.987396 25828 consensus_queue.cc:1048] T 434a54fcee7d4aeeab42f739855c0e8a P 43999d813e564dc688fad116090c95ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:23:37.997015 25503 tablet_bootstrap.cc:492] T b155b866f10547fd8239579a81ecf5f9 P 233e7d29aac04505bf4e65f0a39a0c4c: No bootstrap required, opened a new log
I20260709 13:23:37.997790 25503 ts_tablet_manager.cc:1403] T b155b866f10547fd8239579a81ecf5f9 P 233e7d29aac04505bf4e65f0a39a0c4c: Time spent bootstrapping tablet: real 0.120s	user 0.013s	sys 0.007s
I20260709 13:23:37.989023 25382 raft_consensus.cc:2468] T b155b866f10547fd8239579a81ecf5f9 P 3e07d64466934da4bb3c02262b1c2e38 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 43999d813e564dc688fad116090c95ca in term 1.
I20260709 13:23:38.015643 25130 catalog_manager.cc:5697] T b85a238bbc3b4621881603a534c7bd9b P 233e7d29aac04505bf4e65f0a39a0c4c reported cstate change: term changed from 0 to 1, leader changed from <none> to 233e7d29aac04505bf4e65f0a39a0c4c (127.14.177.130). New cstate: current_term: 1 leader_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } health_report { overall_health: HEALTHY } } }
I20260709 13:23:38.025223 25503 raft_consensus.cc:359] T b155b866f10547fd8239579a81ecf5f9 P 233e7d29aac04505bf4e65f0a39a0c4c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } }
I20260709 13:23:38.032729 25828 raft_consensus.cc:493] T 8ef061028f38450e83324ac78cdaa14c P 43999d813e564dc688fad116090c95ca [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:23:38.033466 25828 raft_consensus.cc:515] T 8ef061028f38450e83324ac78cdaa14c P 43999d813e564dc688fad116090c95ca [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } }
I20260709 13:23:38.048243 25503 raft_consensus.cc:385] T b155b866f10547fd8239579a81ecf5f9 P 233e7d29aac04505bf4e65f0a39a0c4c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:23:38.048970 25503 raft_consensus.cc:740] T b155b866f10547fd8239579a81ecf5f9 P 233e7d29aac04505bf4e65f0a39a0c4c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 233e7d29aac04505bf4e65f0a39a0c4c, State: Initialized, Role: FOLLOWER
I20260709 13:23:38.021456 25380 raft_consensus.cc:2468] T cfad575709c943cba40ccab278350770 P 3e07d64466934da4bb3c02262b1c2e38 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 233e7d29aac04505bf4e65f0a39a0c4c in term 1.
I20260709 13:23:38.051004 25503 consensus_queue.cc:260] T b155b866f10547fd8239579a81ecf5f9 P 233e7d29aac04505bf4e65f0a39a0c4c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } }
I20260709 13:23:38.038224 25199 leader_election.cc:304] T b155b866f10547fd8239579a81ecf5f9 P 43999d813e564dc688fad116090c95ca [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3e07d64466934da4bb3c02262b1c2e38, 43999d813e564dc688fad116090c95ca; no voters: 233e7d29aac04505bf4e65f0a39a0c4c
I20260709 13:23:38.053408 25822 raft_consensus.cc:2804] T b155b866f10547fd8239579a81ecf5f9 P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:23:38.053921 25822 raft_consensus.cc:697] T b155b866f10547fd8239579a81ecf5f9 P 43999d813e564dc688fad116090c95ca [term 1 LEADER]: Becoming Leader. State: Replica: 43999d813e564dc688fad116090c95ca, State: Running, Role: LEADER
I20260709 13:23:38.059111 25503 ts_tablet_manager.cc:1434] T b155b866f10547fd8239579a81ecf5f9 P 233e7d29aac04505bf4e65f0a39a0c4c: Time spent starting tablet: real 0.061s	user 0.007s	sys 0.000s
I20260709 13:23:38.056639 25132 catalog_manager.cc:5697] T 4d837697cd0f402cacf9db89a5d92652 P 43999d813e564dc688fad116090c95ca reported cstate change: term changed from 0 to 1, leader changed from <none> to 43999d813e564dc688fad116090c95ca (127.14.177.129). New cstate: current_term: 1 leader_uuid: "43999d813e564dc688fad116090c95ca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } health_report { overall_health: HEALTHY } } }
I20260709 13:23:38.062446 25822 consensus_queue.cc:237] T b155b866f10547fd8239579a81ecf5f9 P 43999d813e564dc688fad116090c95ca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } }
I20260709 13:23:38.065862 25308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ef061028f38450e83324ac78cdaa14c" candidate_uuid: "43999d813e564dc688fad116090c95ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" is_pre_election: true
I20260709 13:23:38.066872 25308 raft_consensus.cc:2468] T 8ef061028f38450e83324ac78cdaa14c P 233e7d29aac04505bf4e65f0a39a0c4c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 43999d813e564dc688fad116090c95ca in term 0.
I20260709 13:23:38.078161 25459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ef061028f38450e83324ac78cdaa14c" candidate_uuid: "43999d813e564dc688fad116090c95ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9868dce521294d0b816bba7648f4ae5a" is_pre_election: true
I20260709 13:23:38.079506 25202 leader_election.cc:304] T 8ef061028f38450e83324ac78cdaa14c P 43999d813e564dc688fad116090c95ca [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 233e7d29aac04505bf4e65f0a39a0c4c, 43999d813e564dc688fad116090c95ca; no voters: 
I20260709 13:23:38.070825 25274 leader_election.cc:304] T cfad575709c943cba40ccab278350770 P 233e7d29aac04505bf4e65f0a39a0c4c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 233e7d29aac04505bf4e65f0a39a0c4c, 3e07d64466934da4bb3c02262b1c2e38; no voters: 
I20260709 13:23:38.082535 25676 raft_consensus.cc:2804] T cfad575709c943cba40ccab278350770 P 233e7d29aac04505bf4e65f0a39a0c4c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:23:38.083040 25676 raft_consensus.cc:697] T cfad575709c943cba40ccab278350770 P 233e7d29aac04505bf4e65f0a39a0c4c [term 1 LEADER]: Becoming Leader. State: Replica: 233e7d29aac04505bf4e65f0a39a0c4c, State: Running, Role: LEADER
I20260709 13:23:38.083068 25601 raft_consensus.cc:2804] T 8ef061028f38450e83324ac78cdaa14c P 43999d813e564dc688fad116090c95ca [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:23:38.083645 25601 raft_consensus.cc:493] T 8ef061028f38450e83324ac78cdaa14c P 43999d813e564dc688fad116090c95ca [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:23:38.069103 25828 leader_election.cc:290] T 8ef061028f38450e83324ac78cdaa14c P 43999d813e564dc688fad116090c95ca [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9868dce521294d0b816bba7648f4ae5a (127.14.177.132:33683), 233e7d29aac04505bf4e65f0a39a0c4c (127.14.177.130:42957)
I20260709 13:23:38.084074 25601 raft_consensus.cc:3060] T 8ef061028f38450e83324ac78cdaa14c P 43999d813e564dc688fad116090c95ca [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:23:38.085912 25676 consensus_queue.cc:237] T cfad575709c943cba40ccab278350770 P 233e7d29aac04505bf4e65f0a39a0c4c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } }
I20260709 13:23:38.094494 25504 tablet_bootstrap.cc:492] T e311eb90bad44c80a131b72edcd22b06 P 3e07d64466934da4bb3c02262b1c2e38: No bootstrap required, opened a new log
I20260709 13:23:38.094539 25847 consensus_queue.cc:1048] T 9ab90e3e08fb4528b2a47d995e9e26d3 P 9868dce521294d0b816bba7648f4ae5a [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.004s
I20260709 13:23:38.095249 25505 tablet_bootstrap.cc:492] T b9d550d881b04735bbf89751952c09ec P 43999d813e564dc688fad116090c95ca: No bootstrap required, opened a new log
I20260709 13:23:38.095794 25505 ts_tablet_manager.cc:1403] T b9d550d881b04735bbf89751952c09ec P 43999d813e564dc688fad116090c95ca: Time spent bootstrapping tablet: real 0.291s	user 0.028s	sys 0.038s
I20260709 13:23:38.095268 25504 ts_tablet_manager.cc:1403] T e311eb90bad44c80a131b72edcd22b06 P 3e07d64466934da4bb3c02262b1c2e38: Time spent bootstrapping tablet: real 0.313s	user 0.009s	sys 0.021s
I20260709 13:23:38.098985 25504 raft_consensus.cc:359] T e311eb90bad44c80a131b72edcd22b06 P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } }
I20260709 13:23:38.099714 25504 raft_consensus.cc:385] T e311eb90bad44c80a131b72edcd22b06 P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:23:38.099996 25504 raft_consensus.cc:740] T e311eb90bad44c80a131b72edcd22b06 P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e07d64466934da4bb3c02262b1c2e38, State: Initialized, Role: FOLLOWER
I20260709 13:23:38.102690 25508 tablet_bootstrap.cc:654] T e311eb90bad44c80a131b72edcd22b06 P 9868dce521294d0b816bba7648f4ae5a: Neither blocks nor log segments found. Creating new log.
I20260709 13:23:38.105331 25455 raft_consensus.cc:2468] T cfad575709c943cba40ccab278350770 P 9868dce521294d0b816bba7648f4ae5a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 233e7d29aac04505bf4e65f0a39a0c4c in term 1.
I20260709 13:23:38.117309 25650 raft_consensus.cc:493] T cfad575709c943cba40ccab278350770 P 9868dce521294d0b816bba7648f4ae5a [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:23:38.117961 25650 raft_consensus.cc:515] T cfad575709c943cba40ccab278350770 P 9868dce521294d0b816bba7648f4ae5a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } }
I20260709 13:23:38.120762 25650 leader_election.cc:290] T cfad575709c943cba40ccab278350770 P 9868dce521294d0b816bba7648f4ae5a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 233e7d29aac04505bf4e65f0a39a0c4c (127.14.177.130:42957), 3e07d64466934da4bb3c02262b1c2e38 (127.14.177.131:41539)
I20260709 13:23:38.054737 25795 raft_consensus.cc:515] T 434a54fcee7d4aeeab42f739855c0e8a P 233e7d29aac04505bf4e65f0a39a0c4c [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } }
I20260709 13:23:38.123562 25309 raft_consensus.cc:1240] T 434a54fcee7d4aeeab42f739855c0e8a P 233e7d29aac04505bf4e65f0a39a0c4c [term 2 FOLLOWER]: Rejecting Update request from peer 43999d813e564dc688fad116090c95ca for earlier term 1. Current term is 2. Ops: []
I20260709 13:23:38.125012 25233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434a54fcee7d4aeeab42f739855c0e8a" candidate_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "43999d813e564dc688fad116090c95ca"
I20260709 13:23:38.105417 25504 consensus_queue.cc:260] T e311eb90bad44c80a131b72edcd22b06 P 3e07d64466934da4bb3c02262b1c2e38 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } }
I20260709 13:23:38.128118 25503 tablet_bootstrap.cc:492] T e311eb90bad44c80a131b72edcd22b06 P 233e7d29aac04505bf4e65f0a39a0c4c: Bootstrap starting.
I20260709 13:23:38.128325 25795 leader_election.cc:290] T 434a54fcee7d4aeeab42f739855c0e8a P 233e7d29aac04505bf4e65f0a39a0c4c [CANDIDATE]: Term 2 election: Requested vote from peers 43999d813e564dc688fad116090c95ca (127.14.177.129:38155), 3e07d64466934da4bb3c02262b1c2e38 (127.14.177.131:41539)
I20260709 13:23:38.124984 25308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfad575709c943cba40ccab278350770" candidate_uuid: "9868dce521294d0b816bba7648f4ae5a" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" is_pre_election: true
I20260709 13:23:38.129902 25504 ts_tablet_manager.cc:1434] T e311eb90bad44c80a131b72edcd22b06 P 3e07d64466934da4bb3c02262b1c2e38: Time spent starting tablet: real 0.034s	user 0.005s	sys 0.001s
W20260709 13:23:38.094471 25199 leader_election.cc:343] T 8ef061028f38450e83324ac78cdaa14c P 43999d813e564dc688fad116090c95ca [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9868dce521294d0b816bba7648f4ae5a (127.14.177.132:33683): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:23:38.128171 25822 consensus_queue.cc:1059] T 434a54fcee7d4aeeab42f739855c0e8a P 43999d813e564dc688fad116090c95ca [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 13:23:38.134402 25822 raft_consensus.cc:3055] T 434a54fcee7d4aeeab42f739855c0e8a P 43999d813e564dc688fad116090c95ca [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:23:38.134786 25822 raft_consensus.cc:740] T 434a54fcee7d4aeeab42f739855c0e8a P 43999d813e564dc688fad116090c95ca [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 43999d813e564dc688fad116090c95ca, State: Running, Role: LEADER
I20260709 13:23:38.107386 25505 raft_consensus.cc:359] T b9d550d881b04735bbf89751952c09ec P 43999d813e564dc688fad116090c95ca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } }
I20260709 13:23:38.135506 25505 raft_consensus.cc:385] T b9d550d881b04735bbf89751952c09ec P 43999d813e564dc688fad116090c95ca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:23:38.135826 25505 raft_consensus.cc:740] T b9d550d881b04735bbf89751952c09ec P 43999d813e564dc688fad116090c95ca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43999d813e564dc688fad116090c95ca, State: Initialized, Role: FOLLOWER
I20260709 13:23:38.140430 25822 consensus_queue.cc:260] T 434a54fcee7d4aeeab42f739855c0e8a P 43999d813e564dc688fad116090c95ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } }
I20260709 13:23:38.141731 25822 raft_consensus.cc:3060] T 434a54fcee7d4aeeab42f739855c0e8a P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:23:38.141952 25504 tablet_bootstrap.cc:492] T 5c7e7acbb09f4d49828fa652ca6abaac P 3e07d64466934da4bb3c02262b1c2e38: Bootstrap starting.
I20260709 13:23:38.137382 25505 consensus_queue.cc:260] T b9d550d881b04735bbf89751952c09ec P 43999d813e564dc688fad116090c95ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } }
I20260709 13:23:38.158247 25508 tablet_bootstrap.cc:492] T e311eb90bad44c80a131b72edcd22b06 P 9868dce521294d0b816bba7648f4ae5a: No bootstrap required, opened a new log
I20260709 13:23:38.158847 25508 ts_tablet_manager.cc:1403] T e311eb90bad44c80a131b72edcd22b06 P 9868dce521294d0b816bba7648f4ae5a: Time spent bootstrapping tablet: real 0.216s	user 0.011s	sys 0.002s
I20260709 13:23:38.158994 25503 tablet_bootstrap.cc:654] T e311eb90bad44c80a131b72edcd22b06 P 233e7d29aac04505bf4e65f0a39a0c4c: Neither blocks nor log segments found. Creating new log.
I20260709 13:23:38.162058 25508 raft_consensus.cc:359] T e311eb90bad44c80a131b72edcd22b06 P 9868dce521294d0b816bba7648f4ae5a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } }
I20260709 13:23:38.162786 25508 raft_consensus.cc:385] T e311eb90bad44c80a131b72edcd22b06 P 9868dce521294d0b816bba7648f4ae5a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:23:38.163058 25508 raft_consensus.cc:740] T e311eb90bad44c80a131b72edcd22b06 P 9868dce521294d0b816bba7648f4ae5a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9868dce521294d0b816bba7648f4ae5a, State: Initialized, Role: FOLLOWER
I20260709 13:23:38.165707 25508 consensus_queue.cc:260] T e311eb90bad44c80a131b72edcd22b06 P 9868dce521294d0b816bba7648f4ae5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } }
I20260709 13:23:38.145023 25601 raft_consensus.cc:515] T 8ef061028f38450e83324ac78cdaa14c P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } }
I20260709 13:23:38.175072 25601 leader_election.cc:290] T 8ef061028f38450e83324ac78cdaa14c P 43999d813e564dc688fad116090c95ca [CANDIDATE]: Term 1 election: Requested vote from peers 9868dce521294d0b816bba7648f4ae5a (127.14.177.132:33683), 233e7d29aac04505bf4e65f0a39a0c4c (127.14.177.130:42957)
I20260709 13:23:38.176437 25307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ef061028f38450e83324ac78cdaa14c" candidate_uuid: "43999d813e564dc688fad116090c95ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "233e7d29aac04505bf4e65f0a39a0c4c"
I20260709 13:23:38.177260 25307 raft_consensus.cc:3060] T 8ef061028f38450e83324ac78cdaa14c P 233e7d29aac04505bf4e65f0a39a0c4c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:23:38.178089 25458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ef061028f38450e83324ac78cdaa14c" candidate_uuid: "43999d813e564dc688fad116090c95ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9868dce521294d0b816bba7648f4ae5a"
W20260709 13:23:38.180318 25199 leader_election.cc:343] T 8ef061028f38450e83324ac78cdaa14c P 43999d813e564dc688fad116090c95ca [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 9868dce521294d0b816bba7648f4ae5a (127.14.177.132:33683): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:23:38.185801 25380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfad575709c943cba40ccab278350770" candidate_uuid: "9868dce521294d0b816bba7648f4ae5a" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e07d64466934da4bb3c02262b1c2e38" is_pre_election: true
I20260709 13:23:38.186576 25380 raft_consensus.cc:2468] T cfad575709c943cba40ccab278350770 P 3e07d64466934da4bb3c02262b1c2e38 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9868dce521294d0b816bba7648f4ae5a in term 1.
I20260709 13:23:38.187345 25504 tablet_bootstrap.cc:654] T 5c7e7acbb09f4d49828fa652ca6abaac P 3e07d64466934da4bb3c02262b1c2e38: Neither blocks nor log segments found. Creating new log.
I20260709 13:23:38.188383 25508 ts_tablet_manager.cc:1434] T e311eb90bad44c80a131b72edcd22b06 P 9868dce521294d0b816bba7648f4ae5a: Time spent starting tablet: real 0.029s	user 0.006s	sys 0.000s
I20260709 13:23:38.190618 25508 tablet_bootstrap.cc:492] T 8ef061028f38450e83324ac78cdaa14c P 9868dce521294d0b816bba7648f4ae5a: Bootstrap starting.
I20260709 13:23:38.170217 25130 catalog_manager.cc:5697] T cfad575709c943cba40ccab278350770 P 233e7d29aac04505bf4e65f0a39a0c4c reported cstate change: term changed from 0 to 1, leader changed from <none> to 233e7d29aac04505bf4e65f0a39a0c4c (127.14.177.130). New cstate: current_term: 1 leader_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } health_report { overall_health: UNKNOWN } } }
I20260709 13:23:38.197317 25503 tablet_bootstrap.cc:492] T e311eb90bad44c80a131b72edcd22b06 P 233e7d29aac04505bf4e65f0a39a0c4c: No bootstrap required, opened a new log
I20260709 13:23:38.200309 25676 raft_consensus.cc:493] T b155b866f10547fd8239579a81ecf5f9 P 233e7d29aac04505bf4e65f0a39a0c4c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:23:38.200893 25676 raft_consensus.cc:515] T b155b866f10547fd8239579a81ecf5f9 P 233e7d29aac04505bf4e65f0a39a0c4c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } }
I20260709 13:23:38.203789 25855 raft_consensus.cc:493] T 72b56266e8a54776b9113c7a786e1952 P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:23:38.204371 25855 raft_consensus.cc:515] T 72b56266e8a54776b9113c7a786e1952 P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } }
I20260709 13:23:38.206481 25855 leader_election.cc:290] T 72b56266e8a54776b9113c7a786e1952 P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 43999d813e564dc688fad116090c95ca (127.14.177.129:38155), 233e7d29aac04505bf4e65f0a39a0c4c (127.14.177.130:42957)
I20260709 13:23:38.197995 25503 ts_tablet_manager.cc:1403] T e311eb90bad44c80a131b72edcd22b06 P 233e7d29aac04505bf4e65f0a39a0c4c: Time spent bootstrapping tablet: real 0.070s	user 0.015s	sys 0.000s
I20260709 13:23:38.209980 25232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b155b866f10547fd8239579a81ecf5f9" candidate_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43999d813e564dc688fad116090c95ca" is_pre_election: true
I20260709 13:23:38.209946 25385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434a54fcee7d4aeeab42f739855c0e8a" candidate_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3e07d64466934da4bb3c02262b1c2e38"
I20260709 13:23:38.212939 25503 raft_consensus.cc:359] T e311eb90bad44c80a131b72edcd22b06 P 233e7d29aac04505bf4e65f0a39a0c4c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } }
I20260709 13:23:38.213158 25307 raft_consensus.cc:2468] T 8ef061028f38450e83324ac78cdaa14c P 233e7d29aac04505bf4e65f0a39a0c4c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 43999d813e564dc688fad116090c95ca in term 1.
I20260709 13:23:38.213649 25503 raft_consensus.cc:385] T e311eb90bad44c80a131b72edcd22b06 P 233e7d29aac04505bf4e65f0a39a0c4c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:23:38.215449 25202 leader_election.cc:304] T 8ef061028f38450e83324ac78cdaa14c P 43999d813e564dc688fad116090c95ca [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 233e7d29aac04505bf4e65f0a39a0c4c, 43999d813e564dc688fad116090c95ca; no voters: 9868dce521294d0b816bba7648f4ae5a
I20260709 13:23:38.217232 25822 consensus_queue.cc:1048] T 4d837697cd0f402cacf9db89a5d92652 P 43999d813e564dc688fad116090c95ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.004s
I20260709 13:23:38.218060 25134 catalog_manager.cc:5697] T b155b866f10547fd8239579a81ecf5f9 P 43999d813e564dc688fad116090c95ca reported cstate change: term changed from 0 to 1, leader changed from <none> to 43999d813e564dc688fad116090c95ca (127.14.177.129). New cstate: current_term: 1 leader_uuid: "43999d813e564dc688fad116090c95ca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } health_report { overall_health: UNKNOWN } } }
I20260709 13:23:38.222029 25822 raft_consensus.cc:2804] T 8ef061028f38450e83324ac78cdaa14c P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:23:38.222674 25822 raft_consensus.cc:697] T 8ef061028f38450e83324ac78cdaa14c P 43999d813e564dc688fad116090c95ca [term 1 LEADER]: Becoming Leader. State: Replica: 43999d813e564dc688fad116090c95ca, State: Running, Role: LEADER
I20260709 13:23:38.223642 25822 consensus_queue.cc:237] T 8ef061028f38450e83324ac78cdaa14c P 43999d813e564dc688fad116090c95ca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } }
I20260709 13:23:38.224794 25508 tablet_bootstrap.cc:654] T 8ef061028f38450e83324ac78cdaa14c P 9868dce521294d0b816bba7648f4ae5a: Neither blocks nor log segments found. Creating new log.
I20260709 13:23:38.215896 25676 leader_election.cc:290] T b155b866f10547fd8239579a81ecf5f9 P 233e7d29aac04505bf4e65f0a39a0c4c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 43999d813e564dc688fad116090c95ca (127.14.177.129:38155), 3e07d64466934da4bb3c02262b1c2e38 (127.14.177.131:41539)
I20260709 13:23:38.214004 25503 raft_consensus.cc:740] T e311eb90bad44c80a131b72edcd22b06 P 233e7d29aac04505bf4e65f0a39a0c4c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 233e7d29aac04505bf4e65f0a39a0c4c, State: Initialized, Role: FOLLOWER
I20260709 13:23:38.215724 25505 ts_tablet_manager.cc:1434] T b9d550d881b04735bbf89751952c09ec P 43999d813e564dc688fad116090c95ca: Time spent starting tablet: real 0.113s	user 0.019s	sys 0.018s
I20260709 13:23:38.228883 25505 tablet_bootstrap.cc:492] T 72b56266e8a54776b9113c7a786e1952 P 43999d813e564dc688fad116090c95ca: Bootstrap starting.
I20260709 13:23:38.228516 25503 consensus_queue.cc:260] T e311eb90bad44c80a131b72edcd22b06 P 233e7d29aac04505bf4e65f0a39a0c4c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } }
I20260709 13:23:38.218091 25307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72b56266e8a54776b9113c7a786e1952" candidate_uuid: "3e07d64466934da4bb3c02262b1c2e38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" is_pre_election: true
W20260709 13:23:38.232741 25351 leader_election.cc:343] T 72b56266e8a54776b9113c7a786e1952 P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 233e7d29aac04505bf4e65f0a39a0c4c (127.14.177.130:42957): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:23:38.232968 25422 leader_election.cc:304] T cfad575709c943cba40ccab278350770 P 9868dce521294d0b816bba7648f4ae5a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3e07d64466934da4bb3c02262b1c2e38, 9868dce521294d0b816bba7648f4ae5a; no voters: 233e7d29aac04505bf4e65f0a39a0c4c
I20260709 13:23:38.234282 25650 raft_consensus.cc:2804] T cfad575709c943cba40ccab278350770 P 9868dce521294d0b816bba7648f4ae5a [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:23:38.234688 25650 raft_consensus.cc:493] T cfad575709c943cba40ccab278350770 P 9868dce521294d0b816bba7648f4ae5a [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:23:38.234966 25503 ts_tablet_manager.cc:1434] T e311eb90bad44c80a131b72edcd22b06 P 233e7d29aac04505bf4e65f0a39a0c4c: Time spent starting tablet: real 0.025s	user 0.007s	sys 0.000s
I20260709 13:23:38.235019 25650 raft_consensus.cc:3060] T cfad575709c943cba40ccab278350770 P 9868dce521294d0b816bba7648f4ae5a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:23:38.236184 25503 tablet_bootstrap.cc:492] T fa0da2c845dc4059968332f4db2e0aa6 P 233e7d29aac04505bf4e65f0a39a0c4c: Bootstrap starting.
I20260709 13:23:38.241505 25232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72b56266e8a54776b9113c7a786e1952" candidate_uuid: "3e07d64466934da4bb3c02262b1c2e38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43999d813e564dc688fad116090c95ca" is_pre_election: true
I20260709 13:23:38.243328 25505 tablet_bootstrap.cc:654] T 72b56266e8a54776b9113c7a786e1952 P 43999d813e564dc688fad116090c95ca: Neither blocks nor log segments found. Creating new log.
I20260709 13:23:38.245759 25274 leader_election.cc:304] T 434a54fcee7d4aeeab42f739855c0e8a P 233e7d29aac04505bf4e65f0a39a0c4c [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 233e7d29aac04505bf4e65f0a39a0c4c; no voters: 3e07d64466934da4bb3c02262b1c2e38, 43999d813e564dc688fad116090c95ca
I20260709 13:23:38.248087 25676 raft_consensus.cc:2749] T 434a54fcee7d4aeeab42f739855c0e8a P 233e7d29aac04505bf4e65f0a39a0c4c [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:23:38.251082 25504 tablet_bootstrap.cc:492] T 5c7e7acbb09f4d49828fa652ca6abaac P 3e07d64466934da4bb3c02262b1c2e38: No bootstrap required, opened a new log
I20260709 13:23:38.276407 25504 ts_tablet_manager.cc:1403] T 5c7e7acbb09f4d49828fa652ca6abaac P 3e07d64466934da4bb3c02262b1c2e38: Time spent bootstrapping tablet: real 0.135s	user 0.015s	sys 0.005s
I20260709 13:23:38.275475 25505 tablet_bootstrap.cc:492] T 72b56266e8a54776b9113c7a786e1952 P 43999d813e564dc688fad116090c95ca: No bootstrap required, opened a new log
I20260709 13:23:38.282716 25505 ts_tablet_manager.cc:1403] T 72b56266e8a54776b9113c7a786e1952 P 43999d813e564dc688fad116090c95ca: Time spent bootstrapping tablet: real 0.054s	user 0.014s	sys 0.007s
I20260709 13:23:38.284032 25503 tablet_bootstrap.cc:654] T fa0da2c845dc4059968332f4db2e0aa6 P 233e7d29aac04505bf4e65f0a39a0c4c: Neither blocks nor log segments found. Creating new log.
I20260709 13:23:38.285825 25505 raft_consensus.cc:359] T 72b56266e8a54776b9113c7a786e1952 P 43999d813e564dc688fad116090c95ca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } }
I20260709 13:23:38.286568 25505 raft_consensus.cc:385] T 72b56266e8a54776b9113c7a786e1952 P 43999d813e564dc688fad116090c95ca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:23:38.286881 25505 raft_consensus.cc:740] T 72b56266e8a54776b9113c7a786e1952 P 43999d813e564dc688fad116090c95ca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43999d813e564dc688fad116090c95ca, State: Initialized, Role: FOLLOWER
I20260709 13:23:38.288458 25505 consensus_queue.cc:260] T 72b56266e8a54776b9113c7a786e1952 P 43999d813e564dc688fad116090c95ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } }
I20260709 13:23:38.277827 25508 tablet_bootstrap.cc:492] T 8ef061028f38450e83324ac78cdaa14c P 9868dce521294d0b816bba7648f4ae5a: No bootstrap required, opened a new log
W20260709 13:23:38.280963 25274 consensus_peers.cc:597] T b85a238bbc3b4621881603a534c7bd9b P 233e7d29aac04505bf4e65f0a39a0c4c -> Peer 9868dce521294d0b816bba7648f4ae5a (127.14.177.132:33683): Couldn't send request to peer 9868dce521294d0b816bba7648f4ae5a. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:23:38.254998 25676 raft_consensus.cc:493] T e311eb90bad44c80a131b72edcd22b06 P 233e7d29aac04505bf4e65f0a39a0c4c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:23:38.294857 25676 raft_consensus.cc:515] T e311eb90bad44c80a131b72edcd22b06 P 233e7d29aac04505bf4e65f0a39a0c4c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } }
I20260709 13:23:38.290865 25505 ts_tablet_manager.cc:1434] T 72b56266e8a54776b9113c7a786e1952 P 43999d813e564dc688fad116090c95ca: Time spent starting tablet: real 0.008s	user 0.003s	sys 0.002s
I20260709 13:23:38.295670 25508 ts_tablet_manager.cc:1403] T 8ef061028f38450e83324ac78cdaa14c P 9868dce521294d0b816bba7648f4ae5a: Time spent bootstrapping tablet: real 0.105s	user 0.023s	sys 0.001s
I20260709 13:23:38.297097 25676 leader_election.cc:290] T e311eb90bad44c80a131b72edcd22b06 P 233e7d29aac04505bf4e65f0a39a0c4c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3e07d64466934da4bb3c02262b1c2e38 (127.14.177.131:41539), 9868dce521294d0b816bba7648f4ae5a (127.14.177.132:33683)
I20260709 13:23:38.300549 25828 consensus_queue.cc:1048] T 4d837697cd0f402cacf9db89a5d92652 P 43999d813e564dc688fad116090c95ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:23:38.299216 25508 raft_consensus.cc:359] T 8ef061028f38450e83324ac78cdaa14c P 9868dce521294d0b816bba7648f4ae5a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } }
I20260709 13:23:38.301790 25508 raft_consensus.cc:385] T 8ef061028f38450e83324ac78cdaa14c P 9868dce521294d0b816bba7648f4ae5a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:23:38.300563 25458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e311eb90bad44c80a131b72edcd22b06" candidate_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9868dce521294d0b816bba7648f4ae5a" is_pre_election: true
I20260709 13:23:38.302182 25508 raft_consensus.cc:740] T 8ef061028f38450e83324ac78cdaa14c P 9868dce521294d0b816bba7648f4ae5a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9868dce521294d0b816bba7648f4ae5a, State: Initialized, Role: FOLLOWER
I20260709 13:23:38.302620 25458 raft_consensus.cc:2468] T e311eb90bad44c80a131b72edcd22b06 P 9868dce521294d0b816bba7648f4ae5a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 233e7d29aac04505bf4e65f0a39a0c4c in term 0.
I20260709 13:23:38.303076 25508 consensus_queue.cc:260] T 8ef061028f38450e83324ac78cdaa14c P 9868dce521294d0b816bba7648f4ae5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } }
I20260709 13:23:38.304716 25274 leader_election.cc:304] T e311eb90bad44c80a131b72edcd22b06 P 233e7d29aac04505bf4e65f0a39a0c4c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 233e7d29aac04505bf4e65f0a39a0c4c, 9868dce521294d0b816bba7648f4ae5a; no voters: 
I20260709 13:23:38.305795 25508 ts_tablet_manager.cc:1434] T 8ef061028f38450e83324ac78cdaa14c P 9868dce521294d0b816bba7648f4ae5a: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.000s
I20260709 13:23:38.306885 25508 tablet_bootstrap.cc:492] T b85a238bbc3b4621881603a534c7bd9b P 9868dce521294d0b816bba7648f4ae5a: Bootstrap starting.
I20260709 13:23:38.308182 25795 raft_consensus.cc:2804] T e311eb90bad44c80a131b72edcd22b06 P 233e7d29aac04505bf4e65f0a39a0c4c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:23:38.308679 25795 raft_consensus.cc:493] T e311eb90bad44c80a131b72edcd22b06 P 233e7d29aac04505bf4e65f0a39a0c4c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:23:38.309072 25795 raft_consensus.cc:3060] T e311eb90bad44c80a131b72edcd22b06 P 233e7d29aac04505bf4e65f0a39a0c4c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:23:38.312105 25385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b155b866f10547fd8239579a81ecf5f9" candidate_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e07d64466934da4bb3c02262b1c2e38" is_pre_election: true
I20260709 13:23:38.313086 25385 raft_consensus.cc:2393] T b155b866f10547fd8239579a81ecf5f9 P 3e07d64466934da4bb3c02262b1c2e38 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 233e7d29aac04505bf4e65f0a39a0c4c in current term 1: Already voted for candidate 43999d813e564dc688fad116090c95ca in this term.
I20260709 13:23:38.315150 25505 tablet_bootstrap.cc:492] T fa0da2c845dc4059968332f4db2e0aa6 P 43999d813e564dc688fad116090c95ca: Bootstrap starting.
I20260709 13:23:38.319761 25795 raft_consensus.cc:515] T e311eb90bad44c80a131b72edcd22b06 P 233e7d29aac04505bf4e65f0a39a0c4c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } }
I20260709 13:23:38.322232 25505 tablet_bootstrap.cc:654] T fa0da2c845dc4059968332f4db2e0aa6 P 43999d813e564dc688fad116090c95ca: Neither blocks nor log segments found. Creating new log.
I20260709 13:23:38.324527 25795 leader_election.cc:290] T e311eb90bad44c80a131b72edcd22b06 P 233e7d29aac04505bf4e65f0a39a0c4c [CANDIDATE]: Term 1 election: Requested vote from peers 3e07d64466934da4bb3c02262b1c2e38 (127.14.177.131:41539), 9868dce521294d0b816bba7648f4ae5a (127.14.177.132:33683)
I20260709 13:23:38.324760 25455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e311eb90bad44c80a131b72edcd22b06" candidate_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9868dce521294d0b816bba7648f4ae5a"
I20260709 13:23:38.314181 25508 tablet_bootstrap.cc:654] T b85a238bbc3b4621881603a534c7bd9b P 9868dce521294d0b816bba7648f4ae5a: Neither blocks nor log segments found. Creating new log.
I20260709 13:23:38.325989 25455 raft_consensus.cc:3060] T e311eb90bad44c80a131b72edcd22b06 P 9868dce521294d0b816bba7648f4ae5a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:23:38.326257 25504 raft_consensus.cc:359] T 5c7e7acbb09f4d49828fa652ca6abaac P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } }
I20260709 13:23:38.327097 25504 raft_consensus.cc:385] T 5c7e7acbb09f4d49828fa652ca6abaac P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:23:38.327418 25504 raft_consensus.cc:740] T 5c7e7acbb09f4d49828fa652ca6abaac P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e07d64466934da4bb3c02262b1c2e38, State: Initialized, Role: FOLLOWER
I20260709 13:23:38.324760 25382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e311eb90bad44c80a131b72edcd22b06" candidate_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e07d64466934da4bb3c02262b1c2e38" is_pre_election: true
I20260709 13:23:38.329797 25379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e311eb90bad44c80a131b72edcd22b06" candidate_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e07d64466934da4bb3c02262b1c2e38"
I20260709 13:23:38.330673 25650 raft_consensus.cc:515] T cfad575709c943cba40ccab278350770 P 9868dce521294d0b816bba7648f4ae5a [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } }
I20260709 13:23:38.329795 25382 raft_consensus.cc:2468] T e311eb90bad44c80a131b72edcd22b06 P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 233e7d29aac04505bf4e65f0a39a0c4c in term 0.
I20260709 13:23:38.340080 25308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfad575709c943cba40ccab278350770" candidate_uuid: "9868dce521294d0b816bba7648f4ae5a" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "233e7d29aac04505bf4e65f0a39a0c4c"
I20260709 13:23:38.340900 25504 consensus_queue.cc:260] T 5c7e7acbb09f4d49828fa652ca6abaac P 3e07d64466934da4bb3c02262b1c2e38 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } }
I20260709 13:23:38.340310 25650 leader_election.cc:290] T cfad575709c943cba40ccab278350770 P 9868dce521294d0b816bba7648f4ae5a [CANDIDATE]: Term 2 election: Requested vote from peers 233e7d29aac04505bf4e65f0a39a0c4c (127.14.177.130:42957), 3e07d64466934da4bb3c02262b1c2e38 (127.14.177.131:41539)
I20260709 13:23:38.343999 25504 ts_tablet_manager.cc:1434] T 5c7e7acbb09f4d49828fa652ca6abaac P 3e07d64466934da4bb3c02262b1c2e38: Time spent starting tablet: real 0.062s	user 0.006s	sys 0.001s
I20260709 13:23:38.346266 25274 leader_election.cc:304] T b155b866f10547fd8239579a81ecf5f9 P 233e7d29aac04505bf4e65f0a39a0c4c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 233e7d29aac04505bf4e65f0a39a0c4c; no voters: 3e07d64466934da4bb3c02262b1c2e38, 43999d813e564dc688fad116090c95ca
I20260709 13:23:38.356226 25676 raft_consensus.cc:3060] T b155b866f10547fd8239579a81ecf5f9 P 233e7d29aac04505bf4e65f0a39a0c4c [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:23:38.355283 25350 leader_election.cc:343] T 72b56266e8a54776b9113c7a786e1952 P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 43999d813e564dc688fad116090c95ca (127.14.177.129:38155): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:23:38.360466 25350 leader_election.cc:304] T 72b56266e8a54776b9113c7a786e1952 P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3e07d64466934da4bb3c02262b1c2e38; no voters: 233e7d29aac04505bf4e65f0a39a0c4c, 43999d813e564dc688fad116090c95ca
I20260709 13:23:38.361402 25845 raft_consensus.cc:2749] T 72b56266e8a54776b9113c7a786e1952 P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:23:38.363595 25504 tablet_bootstrap.cc:492] T 821a5866429f4173820fb64336abee02 P 3e07d64466934da4bb3c02262b1c2e38: Bootstrap starting.
I20260709 13:23:38.372854 25455 raft_consensus.cc:2468] T e311eb90bad44c80a131b72edcd22b06 P 9868dce521294d0b816bba7648f4ae5a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 233e7d29aac04505bf4e65f0a39a0c4c in term 1.
I20260709 13:23:38.374290 25274 leader_election.cc:304] T e311eb90bad44c80a131b72edcd22b06 P 233e7d29aac04505bf4e65f0a39a0c4c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 233e7d29aac04505bf4e65f0a39a0c4c, 9868dce521294d0b816bba7648f4ae5a; no voters: 3e07d64466934da4bb3c02262b1c2e38
I20260709 13:23:38.373819 25385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfad575709c943cba40ccab278350770" candidate_uuid: "9868dce521294d0b816bba7648f4ae5a" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e07d64466934da4bb3c02262b1c2e38"
I20260709 13:23:38.375943 25799 raft_consensus.cc:2804] T e311eb90bad44c80a131b72edcd22b06 P 233e7d29aac04505bf4e65f0a39a0c4c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:23:38.376498 25799 raft_consensus.cc:697] T e311eb90bad44c80a131b72edcd22b06 P 233e7d29aac04505bf4e65f0a39a0c4c [term 1 LEADER]: Becoming Leader. State: Replica: 233e7d29aac04505bf4e65f0a39a0c4c, State: Running, Role: LEADER
I20260709 13:23:38.377369 25799 consensus_queue.cc:237] T e311eb90bad44c80a131b72edcd22b06 P 233e7d29aac04505bf4e65f0a39a0c4c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } }
I20260709 13:23:38.382429 25385 raft_consensus.cc:3060] T cfad575709c943cba40ccab278350770 P 3e07d64466934da4bb3c02262b1c2e38 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:23:38.385267 25503 tablet_bootstrap.cc:492] T fa0da2c845dc4059968332f4db2e0aa6 P 233e7d29aac04505bf4e65f0a39a0c4c: No bootstrap required, opened a new log
I20260709 13:23:38.385874 25503 ts_tablet_manager.cc:1403] T fa0da2c845dc4059968332f4db2e0aa6 P 233e7d29aac04505bf4e65f0a39a0c4c: Time spent bootstrapping tablet: real 0.150s	user 0.013s	sys 0.003s
I20260709 13:23:38.364588 25132 catalog_manager.cc:5697] T 8ef061028f38450e83324ac78cdaa14c P 43999d813e564dc688fad116090c95ca reported cstate change: term changed from 0 to 1, leader changed from <none> to 43999d813e564dc688fad116090c95ca (127.14.177.129). New cstate: current_term: 1 leader_uuid: "43999d813e564dc688fad116090c95ca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } health_report { overall_health: UNKNOWN } } }
I20260709 13:23:38.389107 25503 raft_consensus.cc:359] T fa0da2c845dc4059968332f4db2e0aa6 P 233e7d29aac04505bf4e65f0a39a0c4c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } }
I20260709 13:23:38.391309 25676 raft_consensus.cc:2749] T b155b866f10547fd8239579a81ecf5f9 P 233e7d29aac04505bf4e65f0a39a0c4c [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:23:38.392445 25828 raft_consensus.cc:493] T 72b56266e8a54776b9113c7a786e1952 P 43999d813e564dc688fad116090c95ca [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:23:38.397487 25795 consensus_queue.cc:1048] T b85a238bbc3b4621881603a534c7bd9b P 233e7d29aac04505bf4e65f0a39a0c4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:23:38.404695 25828 raft_consensus.cc:515] T 72b56266e8a54776b9113c7a786e1952 P 43999d813e564dc688fad116090c95ca [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } }
I20260709 13:23:38.391794 25503 raft_consensus.cc:385] T fa0da2c845dc4059968332f4db2e0aa6 P 233e7d29aac04505bf4e65f0a39a0c4c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:23:38.406512 25503 raft_consensus.cc:740] T fa0da2c845dc4059968332f4db2e0aa6 P 233e7d29aac04505bf4e65f0a39a0c4c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 233e7d29aac04505bf4e65f0a39a0c4c, State: Initialized, Role: FOLLOWER
I20260709 13:23:38.407444 25503 consensus_queue.cc:260] T fa0da2c845dc4059968332f4db2e0aa6 P 233e7d29aac04505bf4e65f0a39a0c4c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } }
I20260709 13:23:38.413293 25503 ts_tablet_manager.cc:1434] T fa0da2c845dc4059968332f4db2e0aa6 P 233e7d29aac04505bf4e65f0a39a0c4c: Time spent starting tablet: real 0.027s	user 0.009s	sys 0.000s
I20260709 13:23:38.421334 25503 tablet_bootstrap.cc:492] T 72b56266e8a54776b9113c7a786e1952 P 233e7d29aac04505bf4e65f0a39a0c4c: Bootstrap starting.
I20260709 13:23:38.408293 25308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72b56266e8a54776b9113c7a786e1952" candidate_uuid: "43999d813e564dc688fad116090c95ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" is_pre_election: true
W20260709 13:23:38.425032 25202 leader_election.cc:343] T 72b56266e8a54776b9113c7a786e1952 P 43999d813e564dc688fad116090c95ca [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 233e7d29aac04505bf4e65f0a39a0c4c (127.14.177.130:42957): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:23:38.428318 25503 tablet_bootstrap.cc:654] T 72b56266e8a54776b9113c7a786e1952 P 233e7d29aac04505bf4e65f0a39a0c4c: Neither blocks nor log segments found. Creating new log.
I20260709 13:23:38.398346 25385 raft_consensus.cc:2468] T cfad575709c943cba40ccab278350770 P 3e07d64466934da4bb3c02262b1c2e38 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9868dce521294d0b816bba7648f4ae5a in term 2.
I20260709 13:23:38.430599 25504 tablet_bootstrap.cc:654] T 821a5866429f4173820fb64336abee02 P 3e07d64466934da4bb3c02262b1c2e38: Neither blocks nor log segments found. Creating new log.
I20260709 13:23:38.421813 25129 catalog_manager.cc:5697] T e311eb90bad44c80a131b72edcd22b06 P 233e7d29aac04505bf4e65f0a39a0c4c reported cstate change: term changed from 0 to 1, leader changed from <none> to 233e7d29aac04505bf4e65f0a39a0c4c (127.14.177.130). New cstate: current_term: 1 leader_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } health_report { overall_health: UNKNOWN } } }
I20260709 13:23:38.443115 25828 leader_election.cc:290] T 72b56266e8a54776b9113c7a786e1952 P 43999d813e564dc688fad116090c95ca [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 233e7d29aac04505bf4e65f0a39a0c4c (127.14.177.130:42957), 3e07d64466934da4bb3c02262b1c2e38 (127.14.177.131:41539)
I20260709 13:23:38.444577 25385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72b56266e8a54776b9113c7a786e1952" candidate_uuid: "43999d813e564dc688fad116090c95ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e07d64466934da4bb3c02262b1c2e38" is_pre_election: true
I20260709 13:23:38.445533 25385 raft_consensus.cc:2468] T 72b56266e8a54776b9113c7a786e1952 P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 43999d813e564dc688fad116090c95ca in term 0.
I20260709 13:23:38.449946 25199 leader_election.cc:304] T 72b56266e8a54776b9113c7a786e1952 P 43999d813e564dc688fad116090c95ca [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3e07d64466934da4bb3c02262b1c2e38, 43999d813e564dc688fad116090c95ca; no voters: 233e7d29aac04505bf4e65f0a39a0c4c
I20260709 13:23:38.463450 25505 tablet_bootstrap.cc:492] T fa0da2c845dc4059968332f4db2e0aa6 P 43999d813e564dc688fad116090c95ca: No bootstrap required, opened a new log
I20260709 13:23:38.465360 25505 ts_tablet_manager.cc:1403] T fa0da2c845dc4059968332f4db2e0aa6 P 43999d813e564dc688fad116090c95ca: Time spent bootstrapping tablet: real 0.150s	user 0.029s	sys 0.039s
I20260709 13:23:38.452804 25828 raft_consensus.cc:2804] T 72b56266e8a54776b9113c7a786e1952 P 43999d813e564dc688fad116090c95ca [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:23:38.454123 25422 leader_election.cc:304] T cfad575709c943cba40ccab278350770 P 9868dce521294d0b816bba7648f4ae5a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3e07d64466934da4bb3c02262b1c2e38, 9868dce521294d0b816bba7648f4ae5a; no voters: 233e7d29aac04505bf4e65f0a39a0c4c
I20260709 13:23:38.483623 25847 raft_consensus.cc:2804] T cfad575709c943cba40ccab278350770 P 9868dce521294d0b816bba7648f4ae5a [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:23:38.485003 25847 raft_consensus.cc:697] T cfad575709c943cba40ccab278350770 P 9868dce521294d0b816bba7648f4ae5a [term 2 LEADER]: Becoming Leader. State: Replica: 9868dce521294d0b816bba7648f4ae5a, State: Running, Role: LEADER
I20260709 13:23:38.486300 25385 raft_consensus.cc:1240] T cfad575709c943cba40ccab278350770 P 3e07d64466934da4bb3c02262b1c2e38 [term 2 FOLLOWER]: Rejecting Update request from peer 233e7d29aac04505bf4e65f0a39a0c4c for earlier term 1. Current term is 2. Ops: []
I20260709 13:23:38.488157 25676 consensus_queue.cc:1059] T cfad575709c943cba40ccab278350770 P 233e7d29aac04505bf4e65f0a39a0c4c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 }, Status: INVALID_TERM, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:23:38.489037 25676 raft_consensus.cc:3055] T cfad575709c943cba40ccab278350770 P 233e7d29aac04505bf4e65f0a39a0c4c [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:23:38.489367 25676 raft_consensus.cc:740] T cfad575709c943cba40ccab278350770 P 233e7d29aac04505bf4e65f0a39a0c4c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 233e7d29aac04505bf4e65f0a39a0c4c, State: Running, Role: LEADER
I20260709 13:23:38.490172 25676 consensus_queue.cc:260] T cfad575709c943cba40ccab278350770 P 233e7d29aac04505bf4e65f0a39a0c4c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } }
I20260709 13:23:38.470932 25828 raft_consensus.cc:493] T 72b56266e8a54776b9113c7a786e1952 P 43999d813e564dc688fad116090c95ca [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:23:38.485913 25847 consensus_queue.cc:237] T cfad575709c943cba40ccab278350770 P 9868dce521294d0b816bba7648f4ae5a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } }
I20260709 13:23:38.502502 25508 tablet_bootstrap.cc:492] T b85a238bbc3b4621881603a534c7bd9b P 9868dce521294d0b816bba7648f4ae5a: No bootstrap required, opened a new log
I20260709 13:23:38.514166 25508 ts_tablet_manager.cc:1403] T b85a238bbc3b4621881603a534c7bd9b P 9868dce521294d0b816bba7648f4ae5a: Time spent bootstrapping tablet: real 0.208s	user 0.034s	sys 0.049s
I20260709 13:23:38.491469 25828 raft_consensus.cc:3060] T 72b56266e8a54776b9113c7a786e1952 P 43999d813e564dc688fad116090c95ca [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:23:38.515255 25676 raft_consensus.cc:3060] T cfad575709c943cba40ccab278350770 P 233e7d29aac04505bf4e65f0a39a0c4c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:23:38.542151 25508 raft_consensus.cc:359] T b85a238bbc3b4621881603a534c7bd9b P 9868dce521294d0b816bba7648f4ae5a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } }
I20260709 13:23:38.544847 25503 tablet_bootstrap.cc:492] T 72b56266e8a54776b9113c7a786e1952 P 233e7d29aac04505bf4e65f0a39a0c4c: No bootstrap required, opened a new log
I20260709 13:23:38.546301 25855 raft_consensus.cc:493] T 5c7e7acbb09f4d49828fa652ca6abaac P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:23:38.555398 25508 raft_consensus.cc:385] T b85a238bbc3b4621881603a534c7bd9b P 9868dce521294d0b816bba7648f4ae5a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:23:38.531445 25505 raft_consensus.cc:359] T fa0da2c845dc4059968332f4db2e0aa6 P 43999d813e564dc688fad116090c95ca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } }
I20260709 13:23:38.555789 25508 raft_consensus.cc:740] T b85a238bbc3b4621881603a534c7bd9b P 9868dce521294d0b816bba7648f4ae5a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9868dce521294d0b816bba7648f4ae5a, State: Initialized, Role: FOLLOWER
I20260709 13:23:38.556245 25505 raft_consensus.cc:385] T fa0da2c845dc4059968332f4db2e0aa6 P 43999d813e564dc688fad116090c95ca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:23:38.555687 25855 raft_consensus.cc:515] T 5c7e7acbb09f4d49828fa652ca6abaac P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } }
I20260709 13:23:38.556596 25505 raft_consensus.cc:740] T fa0da2c845dc4059968332f4db2e0aa6 P 43999d813e564dc688fad116090c95ca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43999d813e564dc688fad116090c95ca, State: Initialized, Role: FOLLOWER
I20260709 13:23:38.555696 25503 ts_tablet_manager.cc:1403] T 72b56266e8a54776b9113c7a786e1952 P 233e7d29aac04505bf4e65f0a39a0c4c: Time spent bootstrapping tablet: real 0.135s	user 0.012s	sys 0.008s
I20260709 13:23:38.557154 25508 consensus_queue.cc:260] T b85a238bbc3b4621881603a534c7bd9b P 9868dce521294d0b816bba7648f4ae5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } }
I20260709 13:23:38.558835 25505 consensus_queue.cc:260] T fa0da2c845dc4059968332f4db2e0aa6 P 43999d813e564dc688fad116090c95ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } }
I20260709 13:23:38.563414 25828 raft_consensus.cc:515] T 72b56266e8a54776b9113c7a786e1952 P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } }
I20260709 13:23:38.561947 25504 tablet_bootstrap.cc:492] T 821a5866429f4173820fb64336abee02 P 3e07d64466934da4bb3c02262b1c2e38: No bootstrap required, opened a new log
I20260709 13:23:38.571856 25504 ts_tablet_manager.cc:1403] T 821a5866429f4173820fb64336abee02 P 3e07d64466934da4bb3c02262b1c2e38: Time spent bootstrapping tablet: real 0.208s	user 0.017s	sys 0.008s
I20260709 13:23:38.572105 25458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c7e7acbb09f4d49828fa652ca6abaac" candidate_uuid: "3e07d64466934da4bb3c02262b1c2e38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9868dce521294d0b816bba7648f4ae5a" is_pre_election: true
W20260709 13:23:38.574115 25348 leader_election.cc:343] T 5c7e7acbb09f4d49828fa652ca6abaac P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9868dce521294d0b816bba7648f4ae5a (127.14.177.132:33683): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:23:38.575793 25899 consensus_queue.cc:1048] T b155b866f10547fd8239579a81ecf5f9 P 43999d813e564dc688fad116090c95ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:23:38.562609 25232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c7e7acbb09f4d49828fa652ca6abaac" candidate_uuid: "3e07d64466934da4bb3c02262b1c2e38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43999d813e564dc688fad116090c95ca" is_pre_election: true
I20260709 13:23:38.565948 25505 ts_tablet_manager.cc:1434] T fa0da2c845dc4059968332f4db2e0aa6 P 43999d813e564dc688fad116090c95ca: Time spent starting tablet: real 0.100s	user 0.006s	sys 0.000s
I20260709 13:23:38.578680 25505 tablet_bootstrap.cc:492] T 5c7e7acbb09f4d49828fa652ca6abaac P 43999d813e564dc688fad116090c95ca: Bootstrap starting.
I20260709 13:23:38.580317 25504 raft_consensus.cc:359] T 821a5866429f4173820fb64336abee02 P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } }
I20260709 13:23:38.580827 25504 raft_consensus.cc:385] T 821a5866429f4173820fb64336abee02 P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:23:38.581001 25504 raft_consensus.cc:740] T 821a5866429f4173820fb64336abee02 P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e07d64466934da4bb3c02262b1c2e38, State: Initialized, Role: FOLLOWER
I20260709 13:23:38.581444 25504 consensus_queue.cc:260] T 821a5866429f4173820fb64336abee02 P 3e07d64466934da4bb3c02262b1c2e38 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } }
I20260709 13:23:38.583665 25504 ts_tablet_manager.cc:1434] T 821a5866429f4173820fb64336abee02 P 3e07d64466934da4bb3c02262b1c2e38: Time spent starting tablet: real 0.011s	user 0.005s	sys 0.000s
I20260709 13:23:38.585449 25504 tablet_bootstrap.cc:492] T 03e9bfbd2f774476a4a377367796a2e8 P 3e07d64466934da4bb3c02262b1c2e38: Bootstrap starting.
I20260709 13:23:38.587399 25828 leader_election.cc:290] T 72b56266e8a54776b9113c7a786e1952 P 43999d813e564dc688fad116090c95ca [CANDIDATE]: Term 1 election: Requested vote from peers 233e7d29aac04505bf4e65f0a39a0c4c (127.14.177.130:42957), 3e07d64466934da4bb3c02262b1c2e38 (127.14.177.131:41539)
I20260709 13:23:38.588086 25308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72b56266e8a54776b9113c7a786e1952" candidate_uuid: "43999d813e564dc688fad116090c95ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "233e7d29aac04505bf4e65f0a39a0c4c"
I20260709 13:23:38.559090 25855 leader_election.cc:290] T 5c7e7acbb09f4d49828fa652ca6abaac P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9868dce521294d0b816bba7648f4ae5a (127.14.177.132:33683), 43999d813e564dc688fad116090c95ca (127.14.177.129:38155)
I20260709 13:23:38.568625 25503 raft_consensus.cc:359] T 72b56266e8a54776b9113c7a786e1952 P 233e7d29aac04505bf4e65f0a39a0c4c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } }
I20260709 13:23:38.589316 25503 raft_consensus.cc:385] T 72b56266e8a54776b9113c7a786e1952 P 233e7d29aac04505bf4e65f0a39a0c4c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:23:38.589619 25503 raft_consensus.cc:740] T 72b56266e8a54776b9113c7a786e1952 P 233e7d29aac04505bf4e65f0a39a0c4c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 233e7d29aac04505bf4e65f0a39a0c4c, State: Initialized, Role: FOLLOWER
I20260709 13:23:38.590337 25503 consensus_queue.cc:260] T 72b56266e8a54776b9113c7a786e1952 P 233e7d29aac04505bf4e65f0a39a0c4c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } }
I20260709 13:23:38.591610 25380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72b56266e8a54776b9113c7a786e1952" candidate_uuid: "43999d813e564dc688fad116090c95ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e07d64466934da4bb3c02262b1c2e38"
I20260709 13:23:38.592326 25380 raft_consensus.cc:3060] T 72b56266e8a54776b9113c7a786e1952 P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:23:38.590370 25350 leader_election.cc:343] T 5c7e7acbb09f4d49828fa652ca6abaac P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 43999d813e564dc688fad116090c95ca (127.14.177.129:38155): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:23:38.594614 25350 leader_election.cc:304] T 5c7e7acbb09f4d49828fa652ca6abaac P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3e07d64466934da4bb3c02262b1c2e38; no voters: 43999d813e564dc688fad116090c95ca, 9868dce521294d0b816bba7648f4ae5a
I20260709 13:23:38.595566 25907 raft_consensus.cc:2749] T 5c7e7acbb09f4d49828fa652ca6abaac P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:23:38.597091 25129 catalog_manager.cc:5697] T cfad575709c943cba40ccab278350770 P 9868dce521294d0b816bba7648f4ae5a reported cstate change: term changed from 1 to 2, leader changed from 233e7d29aac04505bf4e65f0a39a0c4c (127.14.177.130) to 9868dce521294d0b816bba7648f4ae5a (127.14.177.132). New cstate: current_term: 2 leader_uuid: "9868dce521294d0b816bba7648f4ae5a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } health_report { overall_health: HEALTHY } } }
I20260709 13:23:38.598084 25308 raft_consensus.cc:3060] T 72b56266e8a54776b9113c7a786e1952 P 233e7d29aac04505bf4e65f0a39a0c4c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:23:38.592617 25503 ts_tablet_manager.cc:1434] T 72b56266e8a54776b9113c7a786e1952 P 233e7d29aac04505bf4e65f0a39a0c4c: Time spent starting tablet: real 0.032s	user 0.006s	sys 0.000s
I20260709 13:23:38.603019 25503 tablet_bootstrap.cc:492] T 55e8ed3aee404ce698d0626a87bd03aa P 233e7d29aac04505bf4e65f0a39a0c4c: Bootstrap starting.
I20260709 13:23:38.605623 25504 tablet_bootstrap.cc:654] T 03e9bfbd2f774476a4a377367796a2e8 P 3e07d64466934da4bb3c02262b1c2e38: Neither blocks nor log segments found. Creating new log.
I20260709 13:23:38.574996 25508 ts_tablet_manager.cc:1434] T b85a238bbc3b4621881603a534c7bd9b P 9868dce521294d0b816bba7648f4ae5a: Time spent starting tablet: real 0.059s	user 0.006s	sys 0.001s
I20260709 13:23:38.607592 25508 tablet_bootstrap.cc:492] T b9d550d881b04735bbf89751952c09ec P 9868dce521294d0b816bba7648f4ae5a: Bootstrap starting.
I20260709 13:23:38.629447 25899 raft_consensus.cc:493] T b9d550d881b04735bbf89751952c09ec P 43999d813e564dc688fad116090c95ca [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:23:38.630084 25899 raft_consensus.cc:515] T b9d550d881b04735bbf89751952c09ec P 43999d813e564dc688fad116090c95ca [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } }
I20260709 13:23:38.632356 25505 tablet_bootstrap.cc:654] T 5c7e7acbb09f4d49828fa652ca6abaac P 43999d813e564dc688fad116090c95ca: Neither blocks nor log segments found. Creating new log.
I20260709 13:23:38.633526 25382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9d550d881b04735bbf89751952c09ec" candidate_uuid: "43999d813e564dc688fad116090c95ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e07d64466934da4bb3c02262b1c2e38" is_pre_election: true
I20260709 13:23:38.634621 25382 raft_consensus.cc:2468] T b9d550d881b04735bbf89751952c09ec P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 43999d813e564dc688fad116090c95ca in term 0.
I20260709 13:23:38.626394 25503 tablet_bootstrap.cc:654] T 55e8ed3aee404ce698d0626a87bd03aa P 233e7d29aac04505bf4e65f0a39a0c4c: Neither blocks nor log segments found. Creating new log.
I20260709 13:23:38.636039 25199 leader_election.cc:304] T b9d550d881b04735bbf89751952c09ec P 43999d813e564dc688fad116090c95ca [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e07d64466934da4bb3c02262b1c2e38, 43999d813e564dc688fad116090c95ca; no voters: 
I20260709 13:23:38.637399 25828 raft_consensus.cc:2804] T b9d550d881b04735bbf89751952c09ec P 43999d813e564dc688fad116090c95ca [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:23:38.637825 25828 raft_consensus.cc:493] T b9d550d881b04735bbf89751952c09ec P 43999d813e564dc688fad116090c95ca [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:23:38.638190 25828 raft_consensus.cc:3060] T b9d550d881b04735bbf89751952c09ec P 43999d813e564dc688fad116090c95ca [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:23:38.641311 25899 leader_election.cc:290] T b9d550d881b04735bbf89751952c09ec P 43999d813e564dc688fad116090c95ca [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3e07d64466934da4bb3c02262b1c2e38 (127.14.177.131:41539), 9868dce521294d0b816bba7648f4ae5a (127.14.177.132:33683)
I20260709 13:23:38.641748 25508 tablet_bootstrap.cc:654] T b9d550d881b04735bbf89751952c09ec P 9868dce521294d0b816bba7648f4ae5a: Neither blocks nor log segments found. Creating new log.
I20260709 13:23:38.643268 25455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9d550d881b04735bbf89751952c09ec" candidate_uuid: "43999d813e564dc688fad116090c95ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9868dce521294d0b816bba7648f4ae5a" is_pre_election: true
I20260709 13:23:38.620663 25380 raft_consensus.cc:2468] T 72b56266e8a54776b9113c7a786e1952 P 3e07d64466934da4bb3c02262b1c2e38 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 43999d813e564dc688fad116090c95ca in term 1.
I20260709 13:23:38.630084 25308 raft_consensus.cc:2468] T 72b56266e8a54776b9113c7a786e1952 P 233e7d29aac04505bf4e65f0a39a0c4c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 43999d813e564dc688fad116090c95ca in term 1.
W20260709 13:23:38.668244 25199 leader_election.cc:343] T b9d550d881b04735bbf89751952c09ec P 43999d813e564dc688fad116090c95ca [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9868dce521294d0b816bba7648f4ae5a (127.14.177.132:33683): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:23:38.669819 25199 leader_election.cc:304] T 72b56266e8a54776b9113c7a786e1952 P 43999d813e564dc688fad116090c95ca [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e07d64466934da4bb3c02262b1c2e38, 43999d813e564dc688fad116090c95ca; no voters: 
I20260709 13:23:38.675904 25899 raft_consensus.cc:2804] T 72b56266e8a54776b9113c7a786e1952 P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:23:38.677364 25899 raft_consensus.cc:697] T 72b56266e8a54776b9113c7a786e1952 P 43999d813e564dc688fad116090c95ca [term 1 LEADER]: Becoming Leader. State: Replica: 43999d813e564dc688fad116090c95ca, State: Running, Role: LEADER
I20260709 13:23:38.678359 25899 consensus_queue.cc:237] T 72b56266e8a54776b9113c7a786e1952 P 43999d813e564dc688fad116090c95ca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } }
I20260709 13:23:38.686160 25601 consensus_queue.cc:1048] T b155b866f10547fd8239579a81ecf5f9 P 43999d813e564dc688fad116090c95ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:23:38.689582 25455 raft_consensus.cc:3060] T b85a238bbc3b4621881603a534c7bd9b P 9868dce521294d0b816bba7648f4ae5a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:23:38.714443 25505 tablet_bootstrap.cc:492] T 5c7e7acbb09f4d49828fa652ca6abaac P 43999d813e564dc688fad116090c95ca: No bootstrap required, opened a new log
I20260709 13:23:38.714787 25504 tablet_bootstrap.cc:492] T 03e9bfbd2f774476a4a377367796a2e8 P 3e07d64466934da4bb3c02262b1c2e38: No bootstrap required, opened a new log
I20260709 13:23:38.715090 25505 ts_tablet_manager.cc:1403] T 5c7e7acbb09f4d49828fa652ca6abaac P 43999d813e564dc688fad116090c95ca: Time spent bootstrapping tablet: real 0.137s	user 0.016s	sys 0.000s
I20260709 13:23:38.715361 25504 ts_tablet_manager.cc:1403] T 03e9bfbd2f774476a4a377367796a2e8 P 3e07d64466934da4bb3c02262b1c2e38: Time spent bootstrapping tablet: real 0.130s	user 0.025s	sys 0.024s
I20260709 13:23:38.718350 25505 raft_consensus.cc:359] T 5c7e7acbb09f4d49828fa652ca6abaac P 43999d813e564dc688fad116090c95ca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } }
I20260709 13:23:38.719201 25505 raft_consensus.cc:385] T 5c7e7acbb09f4d49828fa652ca6abaac P 43999d813e564dc688fad116090c95ca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:23:38.718577 25504 raft_consensus.cc:359] T 03e9bfbd2f774476a4a377367796a2e8 P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } }
I20260709 13:23:38.718412 25828 raft_consensus.cc:515] T b9d550d881b04735bbf89751952c09ec P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } }
I20260709 13:23:38.719554 25505 raft_consensus.cc:740] T 5c7e7acbb09f4d49828fa652ca6abaac P 43999d813e564dc688fad116090c95ca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43999d813e564dc688fad116090c95ca, State: Initialized, Role: FOLLOWER
I20260709 13:23:38.720337 25505 consensus_queue.cc:260] T 5c7e7acbb09f4d49828fa652ca6abaac P 43999d813e564dc688fad116090c95ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } }
I20260709 13:23:38.721312 25828 leader_election.cc:290] T b9d550d881b04735bbf89751952c09ec P 43999d813e564dc688fad116090c95ca [CANDIDATE]: Term 1 election: Requested vote from peers 3e07d64466934da4bb3c02262b1c2e38 (127.14.177.131:41539), 9868dce521294d0b816bba7648f4ae5a (127.14.177.132:33683)
I20260709 13:23:38.721947 25504 raft_consensus.cc:385] T 03e9bfbd2f774476a4a377367796a2e8 P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:23:38.722273 25504 raft_consensus.cc:740] T 03e9bfbd2f774476a4a377367796a2e8 P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e07d64466934da4bb3c02262b1c2e38, State: Initialized, Role: FOLLOWER
I20260709 13:23:38.722750 25505 ts_tablet_manager.cc:1434] T 5c7e7acbb09f4d49828fa652ca6abaac P 43999d813e564dc688fad116090c95ca: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.000s
I20260709 13:23:38.723069 25503 tablet_bootstrap.cc:492] T 55e8ed3aee404ce698d0626a87bd03aa P 233e7d29aac04505bf4e65f0a39a0c4c: No bootstrap required, opened a new log
I20260709 13:23:38.723009 25504 consensus_queue.cc:260] T 03e9bfbd2f774476a4a377367796a2e8 P 3e07d64466934da4bb3c02262b1c2e38 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } }
I20260709 13:23:38.723723 25503 ts_tablet_manager.cc:1403] T 55e8ed3aee404ce698d0626a87bd03aa P 233e7d29aac04505bf4e65f0a39a0c4c: Time spent bootstrapping tablet: real 0.121s	user 0.009s	sys 0.017s
I20260709 13:23:38.723800 25505 tablet_bootstrap.cc:492] T 03e9bfbd2f774476a4a377367796a2e8 P 43999d813e564dc688fad116090c95ca: Bootstrap starting.
I20260709 13:23:38.791152 25926 raft_consensus.cc:493] T fa0da2c845dc4059968332f4db2e0aa6 P 43999d813e564dc688fad116090c95ca [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:23:38.793380 25926 raft_consensus.cc:515] T fa0da2c845dc4059968332f4db2e0aa6 P 43999d813e564dc688fad116090c95ca [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } }
I20260709 13:23:38.798777 25926 leader_election.cc:290] T fa0da2c845dc4059968332f4db2e0aa6 P 43999d813e564dc688fad116090c95ca [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9868dce521294d0b816bba7648f4ae5a (127.14.177.132:33683), 233e7d29aac04505bf4e65f0a39a0c4c (127.14.177.130:42957)
I20260709 13:23:38.805579 25308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa0da2c845dc4059968332f4db2e0aa6" candidate_uuid: "43999d813e564dc688fad116090c95ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" is_pre_election: true
I20260709 13:23:38.807099 25308 raft_consensus.cc:2468] T fa0da2c845dc4059968332f4db2e0aa6 P 233e7d29aac04505bf4e65f0a39a0c4c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 43999d813e564dc688fad116090c95ca in term 0.
I20260709 13:23:38.816373 25508 tablet_bootstrap.cc:492] T b9d550d881b04735bbf89751952c09ec P 9868dce521294d0b816bba7648f4ae5a: No bootstrap required, opened a new log
I20260709 13:23:38.818194 25508 ts_tablet_manager.cc:1403] T b9d550d881b04735bbf89751952c09ec P 9868dce521294d0b816bba7648f4ae5a: Time spent bootstrapping tablet: real 0.211s	user 0.017s	sys 0.000s
I20260709 13:23:38.816370 25202 leader_election.cc:304] T fa0da2c845dc4059968332f4db2e0aa6 P 43999d813e564dc688fad116090c95ca [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 233e7d29aac04505bf4e65f0a39a0c4c, 43999d813e564dc688fad116090c95ca; no voters: 
I20260709 13:23:38.829722 25899 raft_consensus.cc:2804] T fa0da2c845dc4059968332f4db2e0aa6 P 43999d813e564dc688fad116090c95ca [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:23:38.830415 25899 raft_consensus.cc:493] T fa0da2c845dc4059968332f4db2e0aa6 P 43999d813e564dc688fad116090c95ca [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:23:38.830930 25899 raft_consensus.cc:3060] T fa0da2c845dc4059968332f4db2e0aa6 P 43999d813e564dc688fad116090c95ca [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:23:38.891489 25504 ts_tablet_manager.cc:1434] T 03e9bfbd2f774476a4a377367796a2e8 P 3e07d64466934da4bb3c02262b1c2e38: Time spent starting tablet: real 0.176s	user 0.007s	sys 0.001s
I20260709 13:23:38.905068 25508 raft_consensus.cc:359] T b9d550d881b04735bbf89751952c09ec P 9868dce521294d0b816bba7648f4ae5a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } }
I20260709 13:23:38.906443 25508 raft_consensus.cc:385] T b9d550d881b04735bbf89751952c09ec P 9868dce521294d0b816bba7648f4ae5a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:23:38.906796 25508 raft_consensus.cc:740] T b9d550d881b04735bbf89751952c09ec P 9868dce521294d0b816bba7648f4ae5a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9868dce521294d0b816bba7648f4ae5a, State: Initialized, Role: FOLLOWER
I20260709 13:23:38.907752 25508 consensus_queue.cc:260] T b9d550d881b04735bbf89751952c09ec P 9868dce521294d0b816bba7648f4ae5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } }
I20260709 13:23:38.909782 25508 ts_tablet_manager.cc:1434] T b9d550d881b04735bbf89751952c09ec P 9868dce521294d0b816bba7648f4ae5a: Time spent starting tablet: real 0.091s	user 0.009s	sys 0.000s
I20260709 13:23:38.913882 25503 raft_consensus.cc:359] T 55e8ed3aee404ce698d0626a87bd03aa P 233e7d29aac04505bf4e65f0a39a0c4c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } }
I20260709 13:23:38.915132 25503 raft_consensus.cc:385] T 55e8ed3aee404ce698d0626a87bd03aa P 233e7d29aac04505bf4e65f0a39a0c4c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:23:38.917096 25503 raft_consensus.cc:740] T 55e8ed3aee404ce698d0626a87bd03aa P 233e7d29aac04505bf4e65f0a39a0c4c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 233e7d29aac04505bf4e65f0a39a0c4c, State: Initialized, Role: FOLLOWER
I20260709 13:23:38.916301 25573 raft_consensus.cc:493] T 821a5866429f4173820fb64336abee02 P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:23:38.961565 25828 consensus_queue.cc:1048] T 8ef061028f38450e83324ac78cdaa14c P 43999d813e564dc688fad116090c95ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:23:38.964273 25508 tablet_bootstrap.cc:492] T 55e8ed3aee404ce698d0626a87bd03aa P 9868dce521294d0b816bba7648f4ae5a: Bootstrap starting.
I20260709 13:23:38.951397 25503 consensus_queue.cc:260] T 55e8ed3aee404ce698d0626a87bd03aa P 233e7d29aac04505bf4e65f0a39a0c4c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } }
I20260709 13:23:38.958979 25573 raft_consensus.cc:515] T 821a5866429f4173820fb64336abee02 P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } }
I20260709 13:23:38.977015 25845 raft_consensus.cc:493] T 87085ac2575e4b6a86dd1fc21e6f718b P 3e07d64466934da4bb3c02262b1c2e38 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 43999d813e564dc688fad116090c95ca)
I20260709 13:23:38.977964 25845 raft_consensus.cc:515] T 87085ac2575e4b6a86dd1fc21e6f718b P 3e07d64466934da4bb3c02262b1c2e38 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } }
I20260709 13:23:38.978596 25573 leader_election.cc:290] T 821a5866429f4173820fb64336abee02 P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9868dce521294d0b816bba7648f4ae5a (127.14.177.132:33683), 43999d813e564dc688fad116090c95ca (127.14.177.129:38155)
I20260709 13:23:38.988044 25845 leader_election.cc:290] T 87085ac2575e4b6a86dd1fc21e6f718b P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 43999d813e564dc688fad116090c95ca (127.14.177.129:38155), 233e7d29aac04505bf4e65f0a39a0c4c (127.14.177.130:42957)
I20260709 13:23:39.003558 25931 raft_consensus.cc:493] T 8ef061028f38450e83324ac78cdaa14c P 9868dce521294d0b816bba7648f4ae5a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:23:39.004567 25931 raft_consensus.cc:515] T 8ef061028f38450e83324ac78cdaa14c P 9868dce521294d0b816bba7648f4ae5a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } }
I20260709 13:23:39.014549 25503 ts_tablet_manager.cc:1434] T 55e8ed3aee404ce698d0626a87bd03aa P 233e7d29aac04505bf4e65f0a39a0c4c: Time spent starting tablet: real 0.290s	user 0.006s	sys 0.005s
I20260709 13:23:39.016558 25931 leader_election.cc:290] T 8ef061028f38450e83324ac78cdaa14c P 9868dce521294d0b816bba7648f4ae5a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 43999d813e564dc688fad116090c95ca (127.14.177.129:38155), 233e7d29aac04505bf4e65f0a39a0c4c (127.14.177.130:42957)
I20260709 13:23:39.016592 25503 tablet_bootstrap.cc:492] T 11949f56648c47e0a43c3140af99a304 P 233e7d29aac04505bf4e65f0a39a0c4c: Bootstrap starting.
I20260709 13:23:39.032321 25504 tablet_bootstrap.cc:492] T 55e8ed3aee404ce698d0626a87bd03aa P 3e07d64466934da4bb3c02262b1c2e38: Bootstrap starting.
I20260709 13:23:39.033511 25382 raft_consensus.cc:3060] T e311eb90bad44c80a131b72edcd22b06 P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:23:39.045244 25233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ef061028f38450e83324ac78cdaa14c" candidate_uuid: "9868dce521294d0b816bba7648f4ae5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43999d813e564dc688fad116090c95ca" is_pre_election: true
I20260709 13:23:39.061414 25508 tablet_bootstrap.cc:654] T 55e8ed3aee404ce698d0626a87bd03aa P 9868dce521294d0b816bba7648f4ae5a: Neither blocks nor log segments found. Creating new log.
I20260709 13:23:39.065579 25457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "821a5866429f4173820fb64336abee02" candidate_uuid: "3e07d64466934da4bb3c02262b1c2e38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9868dce521294d0b816bba7648f4ae5a" is_pre_election: true
I20260709 13:23:39.067265 25453 raft_consensus.cc:3060] T 8ef061028f38450e83324ac78cdaa14c P 9868dce521294d0b816bba7648f4ae5a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:23:39.072207 25927 raft_consensus.cc:493] T 5c7e7acbb09f4d49828fa652ca6abaac P 43999d813e564dc688fad116090c95ca [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:23:39.067806 25456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa0da2c845dc4059968332f4db2e0aa6" candidate_uuid: "43999d813e564dc688fad116090c95ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9868dce521294d0b816bba7648f4ae5a" is_pre_election: true
I20260709 13:23:39.030165 25899 raft_consensus.cc:515] T fa0da2c845dc4059968332f4db2e0aa6 P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } }
I20260709 13:23:39.072935 25927 raft_consensus.cc:515] T 5c7e7acbb09f4d49828fa652ca6abaac P 43999d813e564dc688fad116090c95ca [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } }
I20260709 13:23:39.085502 25129 catalog_manager.cc:5697] T 72b56266e8a54776b9113c7a786e1952 P 43999d813e564dc688fad116090c95ca reported cstate change: term changed from 0 to 1, leader changed from <none> to 43999d813e564dc688fad116090c95ca (127.14.177.129). New cstate: current_term: 1 leader_uuid: "43999d813e564dc688fad116090c95ca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } health_report { overall_health: UNKNOWN } } }
I20260709 13:23:39.088696 25459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9d550d881b04735bbf89751952c09ec" candidate_uuid: "43999d813e564dc688fad116090c95ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9868dce521294d0b816bba7648f4ae5a"
I20260709 13:23:39.089648 25459 raft_consensus.cc:3060] T b9d550d881b04735bbf89751952c09ec P 9868dce521294d0b816bba7648f4ae5a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:23:39.100033 25309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa0da2c845dc4059968332f4db2e0aa6" candidate_uuid: "43999d813e564dc688fad116090c95ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "233e7d29aac04505bf4e65f0a39a0c4c"
I20260709 13:23:39.100811 25309 raft_consensus.cc:3060] T fa0da2c845dc4059968332f4db2e0aa6 P 233e7d29aac04505bf4e65f0a39a0c4c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:23:39.105809 25305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87085ac2575e4b6a86dd1fc21e6f718b" candidate_uuid: "3e07d64466934da4bb3c02262b1c2e38" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" is_pre_election: true
I20260709 13:23:39.106408 25385 raft_consensus.cc:1275] T cfad575709c943cba40ccab278350770 P 3e07d64466934da4bb3c02262b1c2e38 [term 2 FOLLOWER]: Refusing update from remote peer 9868dce521294d0b816bba7648f4ae5a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 13:23:39.107121 25305 raft_consensus.cc:2393] T 87085ac2575e4b6a86dd1fc21e6f718b P 233e7d29aac04505bf4e65f0a39a0c4c [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3e07d64466934da4bb3c02262b1c2e38 in current term 2: Already voted for candidate 233e7d29aac04505bf4e65f0a39a0c4c in this term.
I20260709 13:23:39.119132 25927 leader_election.cc:290] T 5c7e7acbb09f4d49828fa652ca6abaac P 43999d813e564dc688fad116090c95ca [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3e07d64466934da4bb3c02262b1c2e38 (127.14.177.131:41539), 9868dce521294d0b816bba7648f4ae5a (127.14.177.132:33683)
I20260709 13:23:39.120656 25503 tablet_bootstrap.cc:654] T 11949f56648c47e0a43c3140af99a304 P 233e7d29aac04505bf4e65f0a39a0c4c: Neither blocks nor log segments found. Creating new log.
I20260709 13:23:39.129204 25380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9d550d881b04735bbf89751952c09ec" candidate_uuid: "43999d813e564dc688fad116090c95ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e07d64466934da4bb3c02262b1c2e38"
I20260709 13:23:39.130508 25380 raft_consensus.cc:3060] T b9d550d881b04735bbf89751952c09ec P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:23:39.132551 25505 tablet_bootstrap.cc:654] T 03e9bfbd2f774476a4a377367796a2e8 P 43999d813e564dc688fad116090c95ca: Neither blocks nor log segments found. Creating new log.
I20260709 13:23:39.133718 25676 consensus_queue.cc:1048] T e311eb90bad44c80a131b72edcd22b06 P 233e7d29aac04505bf4e65f0a39a0c4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:23:39.087560 25899 leader_election.cc:290] T fa0da2c845dc4059968332f4db2e0aa6 P 43999d813e564dc688fad116090c95ca [CANDIDATE]: Term 1 election: Requested vote from peers 9868dce521294d0b816bba7648f4ae5a (127.14.177.132:33683), 233e7d29aac04505bf4e65f0a39a0c4c (127.14.177.130:42957)
I20260709 13:23:39.143888 25931 consensus_queue.cc:1048] T cfad575709c943cba40ccab278350770 P 9868dce521294d0b816bba7648f4ae5a [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.016s
I20260709 13:23:39.170580 25795 raft_consensus.cc:493] T 55e8ed3aee404ce698d0626a87bd03aa P 233e7d29aac04505bf4e65f0a39a0c4c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:23:39.170542 25799 consensus_queue.cc:1048] T e311eb90bad44c80a131b72edcd22b06 P 233e7d29aac04505bf4e65f0a39a0c4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.005s
I20260709 13:23:39.170885 25233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "821a5866429f4173820fb64336abee02" candidate_uuid: "3e07d64466934da4bb3c02262b1c2e38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43999d813e564dc688fad116090c95ca" is_pre_election: true
I20260709 13:23:39.193166 25504 tablet_bootstrap.cc:654] T 55e8ed3aee404ce698d0626a87bd03aa P 3e07d64466934da4bb3c02262b1c2e38: Neither blocks nor log segments found. Creating new log.
I20260709 13:23:39.183533 25795 raft_consensus.cc:515] T 55e8ed3aee404ce698d0626a87bd03aa P 233e7d29aac04505bf4e65f0a39a0c4c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } }
I20260709 13:23:39.199747 25795 leader_election.cc:290] T 55e8ed3aee404ce698d0626a87bd03aa P 233e7d29aac04505bf4e65f0a39a0c4c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3e07d64466934da4bb3c02262b1c2e38 (127.14.177.131:41539), 9868dce521294d0b816bba7648f4ae5a (127.14.177.132:33683)
I20260709 13:23:39.211603 25233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87085ac2575e4b6a86dd1fc21e6f718b" candidate_uuid: "3e07d64466934da4bb3c02262b1c2e38" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "43999d813e564dc688fad116090c95ca" is_pre_election: true
I20260709 13:23:39.212623 25233 raft_consensus.cc:2468] T 87085ac2575e4b6a86dd1fc21e6f718b P 43999d813e564dc688fad116090c95ca [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3e07d64466934da4bb3c02262b1c2e38 in term 2.
I20260709 13:23:39.212397 25601 consensus_queue.cc:1048] T 72b56266e8a54776b9113c7a786e1952 P 43999d813e564dc688fad116090c95ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:23:39.222286 25385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55e8ed3aee404ce698d0626a87bd03aa" candidate_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e07d64466934da4bb3c02262b1c2e38" is_pre_election: true
I20260709 13:23:39.236932 25456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55e8ed3aee404ce698d0626a87bd03aa" candidate_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9868dce521294d0b816bba7648f4ae5a" is_pre_election: true
I20260709 13:23:39.243912 25378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c7e7acbb09f4d49828fa652ca6abaac" candidate_uuid: "43999d813e564dc688fad116090c95ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e07d64466934da4bb3c02262b1c2e38" is_pre_election: true
I20260709 13:23:39.244930 25378 raft_consensus.cc:2468] T 5c7e7acbb09f4d49828fa652ca6abaac P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 43999d813e564dc688fad116090c95ca in term 0.
I20260709 13:23:39.245872 25508 tablet_bootstrap.cc:492] T 55e8ed3aee404ce698d0626a87bd03aa P 9868dce521294d0b816bba7648f4ae5a: No bootstrap required, opened a new log
I20260709 13:23:39.246698 25508 ts_tablet_manager.cc:1403] T 55e8ed3aee404ce698d0626a87bd03aa P 9868dce521294d0b816bba7648f4ae5a: Time spent bootstrapping tablet: real 0.283s	user 0.008s	sys 0.012s
I20260709 13:23:39.230445 25459 raft_consensus.cc:2468] T b9d550d881b04735bbf89751952c09ec P 9868dce521294d0b816bba7648f4ae5a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 43999d813e564dc688fad116090c95ca in term 1.
W20260709 13:23:39.278262 25274 leader_election.cc:343] T 55e8ed3aee404ce698d0626a87bd03aa P 233e7d29aac04505bf4e65f0a39a0c4c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9868dce521294d0b816bba7648f4ae5a (127.14.177.132:33683): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:23:39.285904 25508 raft_consensus.cc:359] T 55e8ed3aee404ce698d0626a87bd03aa P 9868dce521294d0b816bba7648f4ae5a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } }
I20260709 13:23:39.286974 25508 raft_consensus.cc:385] T 55e8ed3aee404ce698d0626a87bd03aa P 9868dce521294d0b816bba7648f4ae5a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:23:39.287330 25508 raft_consensus.cc:740] T 55e8ed3aee404ce698d0626a87bd03aa P 9868dce521294d0b816bba7648f4ae5a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9868dce521294d0b816bba7648f4ae5a, State: Initialized, Role: FOLLOWER
I20260709 13:23:39.290607 25508 consensus_queue.cc:260] T 55e8ed3aee404ce698d0626a87bd03aa P 9868dce521294d0b816bba7648f4ae5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } }
I20260709 13:23:39.293030 25380 raft_consensus.cc:2468] T b9d550d881b04735bbf89751952c09ec P 3e07d64466934da4bb3c02262b1c2e38 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 43999d813e564dc688fad116090c95ca in term 1.
I20260709 13:23:39.293349 25508 ts_tablet_manager.cc:1434] T 55e8ed3aee404ce698d0626a87bd03aa P 9868dce521294d0b816bba7648f4ae5a: Time spent starting tablet: real 0.046s	user 0.007s	sys 0.004s
W20260709 13:23:39.283865 25199 leader_election.cc:343] T fa0da2c845dc4059968332f4db2e0aa6 P 43999d813e564dc688fad116090c95ca [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9868dce521294d0b816bba7648f4ae5a (127.14.177.132:33683): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:23:39.275964 25348 leader_election.cc:343] T 821a5866429f4173820fb64336abee02 P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9868dce521294d0b816bba7648f4ae5a (127.14.177.132:33683): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:23:39.308893 25936 raft_consensus.cc:493] T 87085ac2575e4b6a86dd1fc21e6f718b P 233e7d29aac04505bf4e65f0a39a0c4c [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:23:39.309489 25936 raft_consensus.cc:515] T 87085ac2575e4b6a86dd1fc21e6f718b P 233e7d29aac04505bf4e65f0a39a0c4c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } }
I20260709 13:23:39.314455 25508 tablet_bootstrap.cc:492] T 5c7e7acbb09f4d49828fa652ca6abaac P 9868dce521294d0b816bba7648f4ae5a: Bootstrap starting.
I20260709 13:23:39.287528 25453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa0da2c845dc4059968332f4db2e0aa6" candidate_uuid: "43999d813e564dc688fad116090c95ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9868dce521294d0b816bba7648f4ae5a"
I20260709 13:23:39.326354 25199 leader_election.cc:304] T 5c7e7acbb09f4d49828fa652ca6abaac P 43999d813e564dc688fad116090c95ca [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e07d64466934da4bb3c02262b1c2e38, 43999d813e564dc688fad116090c95ca; no voters: 
I20260709 13:23:39.350490 25503 tablet_bootstrap.cc:492] T 11949f56648c47e0a43c3140af99a304 P 233e7d29aac04505bf4e65f0a39a0c4c: No bootstrap required, opened a new log
I20260709 13:23:39.351137 25503 ts_tablet_manager.cc:1403] T 11949f56648c47e0a43c3140af99a304 P 233e7d29aac04505bf4e65f0a39a0c4c: Time spent bootstrapping tablet: real 0.335s	user 0.017s	sys 0.010s
W20260709 13:23:39.351961 25276 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.129:38155, user_credentials={real_user=slave}} blocked reactor thread for 89820.9us
I20260709 13:23:39.354182 25503 raft_consensus.cc:359] T 11949f56648c47e0a43c3140af99a304 P 233e7d29aac04505bf4e65f0a39a0c4c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } }
I20260709 13:23:39.340391 25899 raft_consensus.cc:2804] T 5c7e7acbb09f4d49828fa652ca6abaac P 43999d813e564dc688fad116090c95ca [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:23:39.355487 25899 raft_consensus.cc:493] T 5c7e7acbb09f4d49828fa652ca6abaac P 43999d813e564dc688fad116090c95ca [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:23:39.355836 25899 raft_consensus.cc:3060] T 5c7e7acbb09f4d49828fa652ca6abaac P 43999d813e564dc688fad116090c95ca [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:23:39.356052 25232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87085ac2575e4b6a86dd1fc21e6f718b" candidate_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" candidate_term: 3 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43999d813e564dc688fad116090c95ca" is_pre_election: true
I20260709 13:23:39.357007 25232 raft_consensus.cc:2410] T 87085ac2575e4b6a86dd1fc21e6f718b P 43999d813e564dc688fad116090c95ca [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 233e7d29aac04505bf4e65f0a39a0c4c for term 3 because replica has last-logged OpId of term: 1 index: 1, which is greater than that of the candidate, which has last-logged OpId of term: 0 index: 0.
I20260709 13:23:39.360882 25855 raft_consensus.cc:493] T 03e9bfbd2f774476a4a377367796a2e8 P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:23:39.361291 25847 raft_consensus.cc:493] T b85a238bbc3b4621881603a534c7bd9b P 9868dce521294d0b816bba7648f4ae5a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 233e7d29aac04505bf4e65f0a39a0c4c)
I20260709 13:23:39.361548 25855 raft_consensus.cc:515] T 03e9bfbd2f774476a4a377367796a2e8 P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } }
I20260709 13:23:39.362118 25847 raft_consensus.cc:515] T b85a238bbc3b4621881603a534c7bd9b P 9868dce521294d0b816bba7648f4ae5a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } }
I20260709 13:23:39.355036 25503 raft_consensus.cc:385] T 11949f56648c47e0a43c3140af99a304 P 233e7d29aac04505bf4e65f0a39a0c4c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:23:39.339262 25199 leader_election.cc:304] T b9d550d881b04735bbf89751952c09ec P 43999d813e564dc688fad116090c95ca [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e07d64466934da4bb3c02262b1c2e38, 43999d813e564dc688fad116090c95ca; no voters: 
I20260709 13:23:39.366282 25503 raft_consensus.cc:740] T 11949f56648c47e0a43c3140af99a304 P 233e7d29aac04505bf4e65f0a39a0c4c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 233e7d29aac04505bf4e65f0a39a0c4c, State: Initialized, Role: FOLLOWER
I20260709 13:23:39.371315 25925 raft_consensus.cc:2804] T b9d550d881b04735bbf89751952c09ec P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:23:39.371320 25855 leader_election.cc:290] T 03e9bfbd2f774476a4a377367796a2e8 P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 43999d813e564dc688fad116090c95ca (127.14.177.129:38155), 233e7d29aac04505bf4e65f0a39a0c4c (127.14.177.130:42957)
I20260709 13:23:39.372241 25925 raft_consensus.cc:697] T b9d550d881b04735bbf89751952c09ec P 43999d813e564dc688fad116090c95ca [term 1 LEADER]: Becoming Leader. State: Replica: 43999d813e564dc688fad116090c95ca, State: Running, Role: LEADER
I20260709 13:23:39.375018 25308 raft_consensus.cc:1275] T cfad575709c943cba40ccab278350770 P 233e7d29aac04505bf4e65f0a39a0c4c [term 2 FOLLOWER]: Refusing update from remote peer 9868dce521294d0b816bba7648f4ae5a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 1. (term mismatch)
I20260709 13:23:39.374823 25925 consensus_queue.cc:237] T b9d550d881b04735bbf89751952c09ec P 43999d813e564dc688fad116090c95ca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } }
I20260709 13:23:39.375453 25308 pending_rounds.cc:85] T cfad575709c943cba40ccab278350770 P 233e7d29aac04505bf4e65f0a39a0c4c: Aborting all ops after (but not including) 0
I20260709 13:23:39.375888 25308 pending_rounds.cc:107] T cfad575709c943cba40ccab278350770 P 233e7d29aac04505bf4e65f0a39a0c4c: Aborting uncommitted NO_OP operation due to leader change: 1.1
I20260709 13:23:39.376104 25505 tablet_bootstrap.cc:492] T 03e9bfbd2f774476a4a377367796a2e8 P 43999d813e564dc688fad116090c95ca: No bootstrap required, opened a new log
I20260709 13:23:39.376271 25308 raft_consensus.cc:2889] T cfad575709c943cba40ccab278350770 P 233e7d29aac04505bf4e65f0a39a0c4c [term 2 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:23:39.376794 25505 ts_tablet_manager.cc:1403] T 03e9bfbd2f774476a4a377367796a2e8 P 43999d813e564dc688fad116090c95ca: Time spent bootstrapping tablet: real 0.653s	user 0.029s	sys 0.011s
I20260709 13:23:39.382416 25503 consensus_queue.cc:260] T 11949f56648c47e0a43c3140af99a304 P 233e7d29aac04505bf4e65f0a39a0c4c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } }
I20260709 13:23:39.367643 25453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c7e7acbb09f4d49828fa652ca6abaac" candidate_uuid: "43999d813e564dc688fad116090c95ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9868dce521294d0b816bba7648f4ae5a" is_pre_election: true
I20260709 13:23:39.378947 25847 leader_election.cc:290] T b85a238bbc3b4621881603a534c7bd9b P 9868dce521294d0b816bba7648f4ae5a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 43999d813e564dc688fad116090c95ca (127.14.177.129:38155), 233e7d29aac04505bf4e65f0a39a0c4c (127.14.177.130:42957)
I20260709 13:23:39.390425 25309 raft_consensus.cc:2468] T fa0da2c845dc4059968332f4db2e0aa6 P 233e7d29aac04505bf4e65f0a39a0c4c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 43999d813e564dc688fad116090c95ca in term 1.
I20260709 13:23:39.392953 25503 ts_tablet_manager.cc:1434] T 11949f56648c47e0a43c3140af99a304 P 233e7d29aac04505bf4e65f0a39a0c4c: Time spent starting tablet: real 0.041s	user 0.006s	sys 0.001s
I20260709 13:23:39.395511 25202 leader_election.cc:304] T fa0da2c845dc4059968332f4db2e0aa6 P 43999d813e564dc688fad116090c95ca [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 233e7d29aac04505bf4e65f0a39a0c4c, 43999d813e564dc688fad116090c95ca; no voters: 
I20260709 13:23:39.396965 25503 tablet_bootstrap.cc:492] T 03e9bfbd2f774476a4a377367796a2e8 P 233e7d29aac04505bf4e65f0a39a0c4c: Bootstrap starting.
I20260709 13:23:39.382885 25927 consensus_queue.cc:1048] T 8ef061028f38450e83324ac78cdaa14c P 43999d813e564dc688fad116090c95ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.015s
W20260709 13:23:39.400043 25199 leader_election.cc:343] T fa0da2c845dc4059968332f4db2e0aa6 P 43999d813e564dc688fad116090c95ca [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 9868dce521294d0b816bba7648f4ae5a (127.14.177.132:33683): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:23:39.400322 25348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:33683, user_credentials={real_user=slave}} blocked reactor thread for 217181us
I20260709 13:23:39.403079 25601 raft_consensus.cc:2804] T fa0da2c845dc4059968332f4db2e0aa6 P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Leader election won for term 1
W20260709 13:23:39.403394 25350 leader_election.cc:343] T 821a5866429f4173820fb64336abee02 P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 43999d813e564dc688fad116090c95ca (127.14.177.129:38155): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:23:39.403882 25601 raft_consensus.cc:697] T fa0da2c845dc4059968332f4db2e0aa6 P 43999d813e564dc688fad116090c95ca [term 1 LEADER]: Becoming Leader. State: Replica: 43999d813e564dc688fad116090c95ca, State: Running, Role: LEADER
I20260709 13:23:39.404953 25601 consensus_queue.cc:237] T fa0da2c845dc4059968332f4db2e0aa6 P 43999d813e564dc688fad116090c95ca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } }
I20260709 13:23:39.406132 25350 leader_election.cc:304] T 821a5866429f4173820fb64336abee02 P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3e07d64466934da4bb3c02262b1c2e38; no voters: 43999d813e564dc688fad116090c95ca, 9868dce521294d0b816bba7648f4ae5a
I20260709 13:23:39.406973 25936 leader_election.cc:290] T 87085ac2575e4b6a86dd1fc21e6f718b P 233e7d29aac04505bf4e65f0a39a0c4c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 43999d813e564dc688fad116090c95ca (127.14.177.129:38155), 3e07d64466934da4bb3c02262b1c2e38 (127.14.177.131:41539)
I20260709 13:23:39.407768 25310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03e9bfbd2f774476a4a377367796a2e8" candidate_uuid: "3e07d64466934da4bb3c02262b1c2e38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" is_pre_election: true
I20260709 13:23:39.408468 25945 consensus_queue.cc:1048] T cfad575709c943cba40ccab278350770 P 9868dce521294d0b816bba7648f4ae5a [LEADER]: Connected to new peer: Peer: permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 13:23:39.409775 25351 leader_election.cc:343] T 03e9bfbd2f774476a4a377367796a2e8 P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 233e7d29aac04505bf4e65f0a39a0c4c (127.14.177.130:42957): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:23:39.410527 25505 raft_consensus.cc:359] T 03e9bfbd2f774476a4a377367796a2e8 P 43999d813e564dc688fad116090c95ca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } }
I20260709 13:23:39.411387 25505 raft_consensus.cc:385] T 03e9bfbd2f774476a4a377367796a2e8 P 43999d813e564dc688fad116090c95ca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:23:39.411671 25505 raft_consensus.cc:740] T 03e9bfbd2f774476a4a377367796a2e8 P 43999d813e564dc688fad116090c95ca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43999d813e564dc688fad116090c95ca, State: Initialized, Role: FOLLOWER
I20260709 13:23:39.412503 25505 consensus_queue.cc:260] T 03e9bfbd2f774476a4a377367796a2e8 P 43999d813e564dc688fad116090c95ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } }
I20260709 13:23:39.415320 25304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ef061028f38450e83324ac78cdaa14c" candidate_uuid: "9868dce521294d0b816bba7648f4ae5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" is_pre_election: true
W20260709 13:23:39.416538 25350 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:38155, user_credentials={real_user=slave}} blocked reactor thread for 139155us
I20260709 13:23:39.417860 25934 raft_consensus.cc:2749] T 821a5866429f4173820fb64336abee02 P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:23:39.422071 25508 tablet_bootstrap.cc:654] T 5c7e7acbb09f4d49828fa652ca6abaac P 9868dce521294d0b816bba7648f4ae5a: Neither blocks nor log segments found. Creating new log.
I20260709 13:23:39.423815 25899 raft_consensus.cc:515] T 5c7e7acbb09f4d49828fa652ca6abaac P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } }
I20260709 13:23:39.431193 25232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b85a238bbc3b4621881603a534c7bd9b" candidate_uuid: "9868dce521294d0b816bba7648f4ae5a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "43999d813e564dc688fad116090c95ca" is_pre_election: true
I20260709 13:23:39.437050 25350 leader_election.cc:304] T 87085ac2575e4b6a86dd1fc21e6f718b P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3e07d64466934da4bb3c02262b1c2e38, 43999d813e564dc688fad116090c95ca; no voters: 233e7d29aac04505bf4e65f0a39a0c4c
I20260709 13:23:39.437956 25855 raft_consensus.cc:2804] T 87085ac2575e4b6a86dd1fc21e6f718b P 3e07d64466934da4bb3c02262b1c2e38 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:23:39.438436 25855 raft_consensus.cc:493] T 87085ac2575e4b6a86dd1fc21e6f718b P 3e07d64466934da4bb3c02262b1c2e38 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 43999d813e564dc688fad116090c95ca)
I20260709 13:23:39.444255 25949 raft_consensus.cc:493] T 11949f56648c47e0a43c3140af99a304 P 233e7d29aac04505bf4e65f0a39a0c4c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:23:39.445595 25949 raft_consensus.cc:515] T 11949f56648c47e0a43c3140af99a304 P 233e7d29aac04505bf4e65f0a39a0c4c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } }
I20260709 13:23:39.446081 25425 leader_election.cc:304] T 8ef061028f38450e83324ac78cdaa14c P 9868dce521294d0b816bba7648f4ae5a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9868dce521294d0b816bba7648f4ae5a; no voters: 233e7d29aac04505bf4e65f0a39a0c4c, 43999d813e564dc688fad116090c95ca
I20260709 13:23:39.447439 25929 raft_consensus.cc:2749] T 8ef061028f38450e83324ac78cdaa14c P 9868dce521294d0b816bba7648f4ae5a [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:23:39.444519 25855 raft_consensus.cc:3060] T 87085ac2575e4b6a86dd1fc21e6f718b P 3e07d64466934da4bb3c02262b1c2e38 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:23:39.451700 25955 consensus_queue.cc:1048] T 72b56266e8a54776b9113c7a786e1952 P 43999d813e564dc688fad116090c95ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:23:39.455744 25505 ts_tablet_manager.cc:1434] T 03e9bfbd2f774476a4a377367796a2e8 P 43999d813e564dc688fad116090c95ca: Time spent starting tablet: real 0.079s	user 0.009s	sys 0.000s
I20260709 13:23:39.438354 25899 leader_election.cc:290] T 5c7e7acbb09f4d49828fa652ca6abaac P 43999d813e564dc688fad116090c95ca [CANDIDATE]: Term 1 election: Requested vote from peers 3e07d64466934da4bb3c02262b1c2e38 (127.14.177.131:41539), 9868dce521294d0b816bba7648f4ae5a (127.14.177.132:33683)
W20260709 13:23:39.461529 25182 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
W20260709 13:23:39.470214 25199 leader_election.cc:343] T 5c7e7acbb09f4d49828fa652ca6abaac P 43999d813e564dc688fad116090c95ca [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9868dce521294d0b816bba7648f4ae5a (127.14.177.132:33683): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:23:39.481927 25385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c7e7acbb09f4d49828fa652ca6abaac" candidate_uuid: "43999d813e564dc688fad116090c95ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e07d64466934da4bb3c02262b1c2e38"
I20260709 13:23:39.483035 25385 raft_consensus.cc:3060] T 5c7e7acbb09f4d49828fa652ca6abaac P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:23:39.479430 25274 leader_election.cc:343] T 55e8ed3aee404ce698d0626a87bd03aa P 233e7d29aac04505bf4e65f0a39a0c4c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3e07d64466934da4bb3c02262b1c2e38 (127.14.177.131:41539): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:23:39.487521 25274 leader_election.cc:304] T 55e8ed3aee404ce698d0626a87bd03aa P 233e7d29aac04505bf4e65f0a39a0c4c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 233e7d29aac04505bf4e65f0a39a0c4c; no voters: 3e07d64466934da4bb3c02262b1c2e38, 9868dce521294d0b816bba7648f4ae5a
I20260709 13:23:39.488358 25384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87085ac2575e4b6a86dd1fc21e6f718b" candidate_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" candidate_term: 3 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e07d64466934da4bb3c02262b1c2e38" is_pre_election: true
I20260709 13:23:39.491214 25308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b85a238bbc3b4621881603a534c7bd9b" candidate_uuid: "9868dce521294d0b816bba7648f4ae5a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" is_pre_election: true
I20260709 13:23:39.519490 25232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11949f56648c47e0a43c3140af99a304" candidate_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43999d813e564dc688fad116090c95ca" is_pre_election: true
I20260709 13:23:39.495553 25234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03e9bfbd2f774476a4a377367796a2e8" candidate_uuid: "3e07d64466934da4bb3c02262b1c2e38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43999d813e564dc688fad116090c95ca" is_pre_election: true
I20260709 13:23:39.567494 25504 tablet_bootstrap.cc:492] T 55e8ed3aee404ce698d0626a87bd03aa P 3e07d64466934da4bb3c02262b1c2e38: No bootstrap required, opened a new log
I20260709 13:23:39.568420 25504 ts_tablet_manager.cc:1403] T 55e8ed3aee404ce698d0626a87bd03aa P 3e07d64466934da4bb3c02262b1c2e38: Time spent bootstrapping tablet: real 0.602s	user 0.019s	sys 0.016s
I20260709 13:23:39.506009 25676 raft_consensus.cc:2749] T 55e8ed3aee404ce698d0626a87bd03aa P 233e7d29aac04505bf4e65f0a39a0c4c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:23:39.503603 25949 leader_election.cc:290] T 11949f56648c47e0a43c3140af99a304 P 233e7d29aac04505bf4e65f0a39a0c4c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9868dce521294d0b816bba7648f4ae5a (127.14.177.132:33683), 43999d813e564dc688fad116090c95ca (127.14.177.129:38155)
W20260709 13:23:39.569520 25274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41539, user_credentials={real_user=slave}} blocked reactor thread for 69180.4us
I20260709 13:23:39.579537 25935 raft_consensus.cc:493] T 434a54fcee7d4aeeab42f739855c0e8a P 3e07d64466934da4bb3c02262b1c2e38 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 43999d813e564dc688fad116090c95ca)
I20260709 13:23:39.580370 25935 raft_consensus.cc:515] T 434a54fcee7d4aeeab42f739855c0e8a P 3e07d64466934da4bb3c02262b1c2e38 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } }
I20260709 13:23:39.614194 25505 tablet_bootstrap.cc:492] T 11949f56648c47e0a43c3140af99a304 P 43999d813e564dc688fad116090c95ca: Bootstrap starting.
I20260709 13:23:39.572772 25931 raft_consensus.cc:493] T 55e8ed3aee404ce698d0626a87bd03aa P 9868dce521294d0b816bba7648f4ae5a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:23:39.586228 25234 raft_consensus.cc:2468] T 03e9bfbd2f774476a4a377367796a2e8 P 43999d813e564dc688fad116090c95ca [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3e07d64466934da4bb3c02262b1c2e38 in term 0.
I20260709 13:23:39.587121 25927 raft_consensus.cc:493] T 434a54fcee7d4aeeab42f739855c0e8a P 43999d813e564dc688fad116090c95ca [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:23:39.624765 25503 tablet_bootstrap.cc:654] T 03e9bfbd2f774476a4a377367796a2e8 P 233e7d29aac04505bf4e65f0a39a0c4c: Neither blocks nor log segments found. Creating new log.
I20260709 13:23:39.636369 25504 raft_consensus.cc:359] T 55e8ed3aee404ce698d0626a87bd03aa P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } }
I20260709 13:23:39.642306 25504 raft_consensus.cc:385] T 55e8ed3aee404ce698d0626a87bd03aa P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:23:39.642643 25504 raft_consensus.cc:740] T 55e8ed3aee404ce698d0626a87bd03aa P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e07d64466934da4bb3c02262b1c2e38, State: Initialized, Role: FOLLOWER
I20260709 13:23:39.617390 25451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11949f56648c47e0a43c3140af99a304" candidate_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9868dce521294d0b816bba7648f4ae5a" is_pre_election: true
I20260709 13:23:39.648572 25504 consensus_queue.cc:260] T 55e8ed3aee404ce698d0626a87bd03aa P 3e07d64466934da4bb3c02262b1c2e38 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } }
I20260709 13:23:39.639470 25935 leader_election.cc:290] T 434a54fcee7d4aeeab42f739855c0e8a P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 43999d813e564dc688fad116090c95ca (127.14.177.129:38155), 233e7d29aac04505bf4e65f0a39a0c4c (127.14.177.130:42957)
I20260709 13:23:39.622685 25931 raft_consensus.cc:515] T 55e8ed3aee404ce698d0626a87bd03aa P 9868dce521294d0b816bba7648f4ae5a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } }
W20260709 13:23:39.680256 25959 log.cc:927] Time spent T cfad575709c943cba40ccab278350770 P 233e7d29aac04505bf4e65f0a39a0c4c: Append to log took a long time: real 0.066s	user 0.000s	sys 0.001s
I20260709 13:23:39.683102 25931 leader_election.cc:290] T 55e8ed3aee404ce698d0626a87bd03aa P 9868dce521294d0b816bba7648f4ae5a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 233e7d29aac04505bf4e65f0a39a0c4c (127.14.177.130:42957), 3e07d64466934da4bb3c02262b1c2e38 (127.14.177.131:41539)
W20260709 13:23:39.683585 25917 log.cc:927] Time spent T b155b866f10547fd8239579a81ecf5f9 P 3e07d64466934da4bb3c02262b1c2e38: Append to log took a long time: real 0.112s	user 0.001s	sys 0.000s
W20260709 13:23:39.668648 25958 log.cc:927] Time spent T b9d550d881b04735bbf89751952c09ec P 43999d813e564dc688fad116090c95ca: Append to log took a long time: real 0.091s	user 0.002s	sys 0.000s
I20260709 13:23:39.718358 25504 ts_tablet_manager.cc:1434] T 55e8ed3aee404ce698d0626a87bd03aa P 3e07d64466934da4bb3c02262b1c2e38: Time spent starting tablet: real 0.149s	user 0.007s	sys 0.002s
W20260709 13:23:39.720394 25274 leader_election.cc:343] T 11949f56648c47e0a43c3140af99a304 P 233e7d29aac04505bf4e65f0a39a0c4c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9868dce521294d0b816bba7648f4ae5a (127.14.177.132:33683): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:23:39.634876 25927 raft_consensus.cc:515] T 434a54fcee7d4aeeab42f739855c0e8a P 43999d813e564dc688fad116090c95ca [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } }
I20260709 13:23:39.722613 25385 raft_consensus.cc:2468] T 5c7e7acbb09f4d49828fa652ca6abaac P 3e07d64466934da4bb3c02262b1c2e38 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 43999d813e564dc688fad116090c95ca in term 1.
W20260709 13:23:39.750149 25276 leader_election.cc:343] T 11949f56648c47e0a43c3140af99a304 P 233e7d29aac04505bf4e65f0a39a0c4c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 43999d813e564dc688fad116090c95ca (127.14.177.129:38155): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:23:39.753687 25276 leader_election.cc:304] T 11949f56648c47e0a43c3140af99a304 P 233e7d29aac04505bf4e65f0a39a0c4c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 233e7d29aac04505bf4e65f0a39a0c4c; no voters: 43999d813e564dc688fad116090c95ca, 9868dce521294d0b816bba7648f4ae5a
I20260709 13:23:39.764596 25795 raft_consensus.cc:2749] T 11949f56648c47e0a43c3140af99a304 P 233e7d29aac04505bf4e65f0a39a0c4c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:23:39.798542 25459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c7e7acbb09f4d49828fa652ca6abaac" candidate_uuid: "43999d813e564dc688fad116090c95ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9868dce521294d0b816bba7648f4ae5a"
I20260709 13:23:39.799058 25927 leader_election.cc:290] T 434a54fcee7d4aeeab42f739855c0e8a P 43999d813e564dc688fad116090c95ca [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 233e7d29aac04505bf4e65f0a39a0c4c (127.14.177.130:42957), 3e07d64466934da4bb3c02262b1c2e38 (127.14.177.131:41539)
I20260709 13:23:39.820287 25505 tablet_bootstrap.cc:654] T 11949f56648c47e0a43c3140af99a304 P 43999d813e564dc688fad116090c95ca: Neither blocks nor log segments found. Creating new log.
I20260709 13:23:39.820935 25855 raft_consensus.cc:515] T 87085ac2575e4b6a86dd1fc21e6f718b P 3e07d64466934da4bb3c02262b1c2e38 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } }
I20260709 13:23:39.827796 25384 raft_consensus.cc:2410] T 87085ac2575e4b6a86dd1fc21e6f718b P 3e07d64466934da4bb3c02262b1c2e38 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 233e7d29aac04505bf4e65f0a39a0c4c for term 3 because replica has last-logged OpId of term: 1 index: 1, which is greater than that of the candidate, which has last-logged OpId of term: 0 index: 0.
I20260709 13:23:39.829308 25508 tablet_bootstrap.cc:492] T 5c7e7acbb09f4d49828fa652ca6abaac P 9868dce521294d0b816bba7648f4ae5a: No bootstrap required, opened a new log
I20260709 13:23:39.829993 25508 ts_tablet_manager.cc:1403] T 5c7e7acbb09f4d49828fa652ca6abaac P 9868dce521294d0b816bba7648f4ae5a: Time spent bootstrapping tablet: real 0.518s	user 0.011s	sys 0.028s
I20260709 13:23:39.832044 25855 leader_election.cc:290] T 87085ac2575e4b6a86dd1fc21e6f718b P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 2 election: Requested vote from peers 43999d813e564dc688fad116090c95ca (127.14.177.129:38155), 233e7d29aac04505bf4e65f0a39a0c4c (127.14.177.130:42957)
I20260709 13:23:39.835311 25503 tablet_bootstrap.cc:492] T 03e9bfbd2f774476a4a377367796a2e8 P 233e7d29aac04505bf4e65f0a39a0c4c: No bootstrap required, opened a new log
I20260709 13:23:39.835924 25503 ts_tablet_manager.cc:1403] T 03e9bfbd2f774476a4a377367796a2e8 P 233e7d29aac04505bf4e65f0a39a0c4c: Time spent bootstrapping tablet: real 0.439s	user 0.011s	sys 0.016s
I20260709 13:23:39.839476 25503 raft_consensus.cc:359] T 03e9bfbd2f774476a4a377367796a2e8 P 233e7d29aac04505bf4e65f0a39a0c4c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } }
I20260709 13:23:39.861191 25503 raft_consensus.cc:385] T 03e9bfbd2f774476a4a377367796a2e8 P 233e7d29aac04505bf4e65f0a39a0c4c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:23:39.861622 25503 raft_consensus.cc:740] T 03e9bfbd2f774476a4a377367796a2e8 P 233e7d29aac04505bf4e65f0a39a0c4c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 233e7d29aac04505bf4e65f0a39a0c4c, State: Initialized, Role: FOLLOWER
W20260709 13:23:39.871966 25199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41539, user_credentials={real_user=slave}} blocked reactor thread for 260182us
W20260709 13:23:39.904557 25350 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.129:38155, user_credentials={real_user=slave}} blocked reactor thread for 292663us
I20260709 13:23:39.909210 25503 consensus_queue.cc:260] T 03e9bfbd2f774476a4a377367796a2e8 P 233e7d29aac04505bf4e65f0a39a0c4c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } }
I20260709 13:23:39.909751 25350 leader_election.cc:304] T 03e9bfbd2f774476a4a377367796a2e8 P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3e07d64466934da4bb3c02262b1c2e38, 43999d813e564dc688fad116090c95ca; no voters: 233e7d29aac04505bf4e65f0a39a0c4c
I20260709 13:23:39.912313 25966 raft_consensus.cc:2804] T 03e9bfbd2f774476a4a377367796a2e8 P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:23:39.912802 25966 raft_consensus.cc:493] T 03e9bfbd2f774476a4a377367796a2e8 P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:23:39.913180 25966 raft_consensus.cc:3060] T 03e9bfbd2f774476a4a377367796a2e8 P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:23:39.922927 25503 ts_tablet_manager.cc:1434] T 03e9bfbd2f774476a4a377367796a2e8 P 233e7d29aac04505bf4e65f0a39a0c4c: Time spent starting tablet: real 0.087s	user 0.007s	sys 0.001s
I20260709 13:23:39.925021 25508 raft_consensus.cc:359] T 5c7e7acbb09f4d49828fa652ca6abaac P 9868dce521294d0b816bba7648f4ae5a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } }
I20260709 13:23:39.925971 25508 raft_consensus.cc:385] T 5c7e7acbb09f4d49828fa652ca6abaac P 9868dce521294d0b816bba7648f4ae5a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:23:39.926358 25508 raft_consensus.cc:740] T 5c7e7acbb09f4d49828fa652ca6abaac P 9868dce521294d0b816bba7648f4ae5a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9868dce521294d0b816bba7648f4ae5a, State: Initialized, Role: FOLLOWER
I20260709 13:23:39.927290 25508 consensus_queue.cc:260] T 5c7e7acbb09f4d49828fa652ca6abaac P 9868dce521294d0b816bba7648f4ae5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } }
I20260709 13:23:39.934669 25383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55e8ed3aee404ce698d0626a87bd03aa" candidate_uuid: "9868dce521294d0b816bba7648f4ae5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e07d64466934da4bb3c02262b1c2e38" is_pre_election: true
I20260709 13:23:39.935570 25383 raft_consensus.cc:2468] T 55e8ed3aee404ce698d0626a87bd03aa P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9868dce521294d0b816bba7648f4ae5a in term 0.
W20260709 13:23:39.936839 25199 leader_election.cc:343] T 5c7e7acbb09f4d49828fa652ca6abaac P 43999d813e564dc688fad116090c95ca [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 9868dce521294d0b816bba7648f4ae5a (127.14.177.132:33683): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:23:39.937978 25966 raft_consensus.cc:515] T 03e9bfbd2f774476a4a377367796a2e8 P 3e07d64466934da4bb3c02262b1c2e38 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } }
I20260709 13:23:39.948161 25230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434a54fcee7d4aeeab42f739855c0e8a" candidate_uuid: "3e07d64466934da4bb3c02262b1c2e38" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "43999d813e564dc688fad116090c95ca" is_pre_election: true
I20260709 13:23:39.949013 25230 raft_consensus.cc:2468] T 434a54fcee7d4aeeab42f739855c0e8a P 43999d813e564dc688fad116090c95ca [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3e07d64466934da4bb3c02262b1c2e38 in term 2.
I20260709 13:23:39.951459 25234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87085ac2575e4b6a86dd1fc21e6f718b" candidate_uuid: "3e07d64466934da4bb3c02262b1c2e38" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "43999d813e564dc688fad116090c95ca"
I20260709 13:23:39.954799 25966 leader_election.cc:290] T 03e9bfbd2f774476a4a377367796a2e8 P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 1 election: Requested vote from peers 43999d813e564dc688fad116090c95ca (127.14.177.129:38155), 233e7d29aac04505bf4e65f0a39a0c4c (127.14.177.130:42957)
I20260709 13:23:39.961402 25422 leader_election.cc:304] T 55e8ed3aee404ce698d0626a87bd03aa P 9868dce521294d0b816bba7648f4ae5a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e07d64466934da4bb3c02262b1c2e38, 9868dce521294d0b816bba7648f4ae5a; no voters: 
I20260709 13:23:39.963034 25230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03e9bfbd2f774476a4a377367796a2e8" candidate_uuid: "3e07d64466934da4bb3c02262b1c2e38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43999d813e564dc688fad116090c95ca"
I20260709 13:23:39.963824 25230 raft_consensus.cc:3060] T 03e9bfbd2f774476a4a377367796a2e8 P 43999d813e564dc688fad116090c95ca [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:23:39.969882 25508 ts_tablet_manager.cc:1434] T 5c7e7acbb09f4d49828fa652ca6abaac P 9868dce521294d0b816bba7648f4ae5a: Time spent starting tablet: real 0.139s	user 0.009s	sys 0.002s
I20260709 13:23:39.970846 25350 leader_election.cc:304] T 434a54fcee7d4aeeab42f739855c0e8a P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e07d64466934da4bb3c02262b1c2e38, 43999d813e564dc688fad116090c95ca; no voters: 
I20260709 13:23:39.971643 25425 leader_election.cc:304] T b85a238bbc3b4621881603a534c7bd9b P 9868dce521294d0b816bba7648f4ae5a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9868dce521294d0b816bba7648f4ae5a; no voters: 233e7d29aac04505bf4e65f0a39a0c4c, 43999d813e564dc688fad116090c95ca
I20260709 13:23:39.972131 25845 raft_consensus.cc:2804] T 434a54fcee7d4aeeab42f739855c0e8a P 3e07d64466934da4bb3c02262b1c2e38 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:23:39.972594 25845 raft_consensus.cc:493] T 434a54fcee7d4aeeab42f739855c0e8a P 3e07d64466934da4bb3c02262b1c2e38 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 43999d813e564dc688fad116090c95ca)
I20260709 13:23:39.973012 25845 raft_consensus.cc:3060] T 434a54fcee7d4aeeab42f739855c0e8a P 3e07d64466934da4bb3c02262b1c2e38 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:23:39.974699 25929 raft_consensus.cc:2749] T b85a238bbc3b4621881603a534c7bd9b P 9868dce521294d0b816bba7648f4ae5a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:23:39.976260 25505 tablet_bootstrap.cc:492] T 11949f56648c47e0a43c3140af99a304 P 43999d813e564dc688fad116090c95ca: No bootstrap required, opened a new log
I20260709 13:23:39.976836 25505 ts_tablet_manager.cc:1403] T 11949f56648c47e0a43c3140af99a304 P 43999d813e564dc688fad116090c95ca: Time spent bootstrapping tablet: real 0.401s	user 0.017s	sys 0.014s
W20260709 13:23:39.978063 25274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41539, user_credentials={real_user=slave}} blocked reactor thread for 53057.4us
I20260709 13:23:39.981163 25274 leader_election.cc:304] T 87085ac2575e4b6a86dd1fc21e6f718b P 233e7d29aac04505bf4e65f0a39a0c4c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 233e7d29aac04505bf4e65f0a39a0c4c; no voters: 3e07d64466934da4bb3c02262b1c2e38, 43999d813e564dc688fad116090c95ca
I20260709 13:23:39.984303 25978 raft_consensus.cc:2749] T 87085ac2575e4b6a86dd1fc21e6f718b P 233e7d29aac04505bf4e65f0a39a0c4c [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:23:39.990545 25930 raft_consensus.cc:493] T 5c7e7acbb09f4d49828fa652ca6abaac P 9868dce521294d0b816bba7648f4ae5a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:23:39.991148 25930 raft_consensus.cc:515] T 5c7e7acbb09f4d49828fa652ca6abaac P 9868dce521294d0b816bba7648f4ae5a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } }
I20260709 13:23:39.998732 25508 tablet_bootstrap.cc:492] T 821a5866429f4173820fb64336abee02 P 9868dce521294d0b816bba7648f4ae5a: Bootstrap starting.
I20260709 13:23:39.980742 25929 raft_consensus.cc:2804] T 55e8ed3aee404ce698d0626a87bd03aa P 9868dce521294d0b816bba7648f4ae5a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:23:39.999748 25929 raft_consensus.cc:493] T 55e8ed3aee404ce698d0626a87bd03aa P 9868dce521294d0b816bba7648f4ae5a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:23:40.000337 25929 raft_consensus.cc:3060] T 55e8ed3aee404ce698d0626a87bd03aa P 9868dce521294d0b816bba7648f4ae5a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:23:40.001063 25930 leader_election.cc:290] T 5c7e7acbb09f4d49828fa652ca6abaac P 9868dce521294d0b816bba7648f4ae5a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3e07d64466934da4bb3c02262b1c2e38 (127.14.177.131:41539), 43999d813e564dc688fad116090c95ca (127.14.177.129:38155)
I20260709 13:23:40.010375 25230 raft_consensus.cc:2468] T 03e9bfbd2f774476a4a377367796a2e8 P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3e07d64466934da4bb3c02262b1c2e38 in term 1.
I20260709 13:23:40.014622 25381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c7e7acbb09f4d49828fa652ca6abaac" candidate_uuid: "9868dce521294d0b816bba7648f4ae5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e07d64466934da4bb3c02262b1c2e38" is_pre_election: true
I20260709 13:23:40.016032 25381 raft_consensus.cc:2393] T 5c7e7acbb09f4d49828fa652ca6abaac P 3e07d64466934da4bb3c02262b1c2e38 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9868dce521294d0b816bba7648f4ae5a in current term 1: Already voted for candidate 43999d813e564dc688fad116090c95ca in this term.
I20260709 13:23:40.017932 25505 raft_consensus.cc:359] T 11949f56648c47e0a43c3140af99a304 P 43999d813e564dc688fad116090c95ca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } }
I20260709 13:23:40.018719 25505 raft_consensus.cc:385] T 11949f56648c47e0a43c3140af99a304 P 43999d813e564dc688fad116090c95ca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:23:40.019019 25505 raft_consensus.cc:740] T 11949f56648c47e0a43c3140af99a304 P 43999d813e564dc688fad116090c95ca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43999d813e564dc688fad116090c95ca, State: Initialized, Role: FOLLOWER
I20260709 13:23:40.022887 25311 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55e8ed3aee404ce698d0626a87bd03aa" candidate_uuid: "9868dce521294d0b816bba7648f4ae5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" is_pre_election: true
I20260709 13:23:40.023653 25311 raft_consensus.cc:2468] T 55e8ed3aee404ce698d0626a87bd03aa P 233e7d29aac04505bf4e65f0a39a0c4c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9868dce521294d0b816bba7648f4ae5a in term 0.
I20260709 13:23:40.024876 25505 consensus_queue.cc:260] T 11949f56648c47e0a43c3140af99a304 P 43999d813e564dc688fad116090c95ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } }
I20260709 13:23:40.026721 25505 ts_tablet_manager.cc:1434] T 11949f56648c47e0a43c3140af99a304 P 43999d813e564dc688fad116090c95ca: Time spent starting tablet: real 0.050s	user 0.005s	sys 0.000s
I20260709 13:23:40.021183 25234 raft_consensus.cc:2468] T 87085ac2575e4b6a86dd1fc21e6f718b P 43999d813e564dc688fad116090c95ca [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3e07d64466934da4bb3c02262b1c2e38 in term 2.
I20260709 13:23:40.031502 25308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434a54fcee7d4aeeab42f739855c0e8a" candidate_uuid: "43999d813e564dc688fad116090c95ca" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" is_pre_election: true
I20260709 13:23:40.042102 25309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03e9bfbd2f774476a4a377367796a2e8" candidate_uuid: "3e07d64466934da4bb3c02262b1c2e38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "233e7d29aac04505bf4e65f0a39a0c4c"
I20260709 13:23:40.042582 25308 raft_consensus.cc:2468] T 434a54fcee7d4aeeab42f739855c0e8a P 233e7d29aac04505bf4e65f0a39a0c4c [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 43999d813e564dc688fad116090c95ca in term 2.
I20260709 13:23:40.040846 25302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87085ac2575e4b6a86dd1fc21e6f718b" candidate_uuid: "3e07d64466934da4bb3c02262b1c2e38" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "233e7d29aac04505bf4e65f0a39a0c4c"
I20260709 13:23:40.041637 25311 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434a54fcee7d4aeeab42f739855c0e8a" candidate_uuid: "3e07d64466934da4bb3c02262b1c2e38" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" is_pre_election: true
I20260709 13:23:40.051693 25302 raft_consensus.cc:2393] T 87085ac2575e4b6a86dd1fc21e6f718b P 233e7d29aac04505bf4e65f0a39a0c4c [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 3e07d64466934da4bb3c02262b1c2e38 in current term 2: Already voted for candidate 233e7d29aac04505bf4e65f0a39a0c4c in this term.
I20260709 13:23:40.052264 25311 raft_consensus.cc:2393] T 434a54fcee7d4aeeab42f739855c0e8a P 233e7d29aac04505bf4e65f0a39a0c4c [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3e07d64466934da4bb3c02262b1c2e38 in current term 2: Already voted for candidate 233e7d29aac04505bf4e65f0a39a0c4c in this term.
I20260709 13:23:40.042906 25309 raft_consensus.cc:3060] T 03e9bfbd2f774476a4a377367796a2e8 P 233e7d29aac04505bf4e65f0a39a0c4c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:23:40.055534 25385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434a54fcee7d4aeeab42f739855c0e8a" candidate_uuid: "43999d813e564dc688fad116090c95ca" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3e07d64466934da4bb3c02262b1c2e38" is_pre_election: true
I20260709 13:23:40.068387 25236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c7e7acbb09f4d49828fa652ca6abaac" candidate_uuid: "9868dce521294d0b816bba7648f4ae5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43999d813e564dc688fad116090c95ca" is_pre_election: true
I20260709 13:23:40.069942 25236 raft_consensus.cc:2393] T 5c7e7acbb09f4d49828fa652ca6abaac P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9868dce521294d0b816bba7648f4ae5a in current term 1: Already voted for candidate 43999d813e564dc688fad116090c95ca in this term.
I20260709 13:23:40.049881 25350 leader_election.cc:304] T 03e9bfbd2f774476a4a377367796a2e8 P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e07d64466934da4bb3c02262b1c2e38, 43999d813e564dc688fad116090c95ca; no voters: 
I20260709 13:23:40.072861 25971 raft_consensus.cc:2804] T 03e9bfbd2f774476a4a377367796a2e8 P 3e07d64466934da4bb3c02262b1c2e38 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:23:40.073393 25971 raft_consensus.cc:697] T 03e9bfbd2f774476a4a377367796a2e8 P 3e07d64466934da4bb3c02262b1c2e38 [term 1 LEADER]: Becoming Leader. State: Replica: 3e07d64466934da4bb3c02262b1c2e38, State: Running, Role: LEADER
I20260709 13:23:40.074357 25971 consensus_queue.cc:237] T 03e9bfbd2f774476a4a377367796a2e8 P 3e07d64466934da4bb3c02262b1c2e38 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } }
W20260709 13:23:40.079187 25199 consensus_peers.cc:597] T fa0da2c845dc4059968332f4db2e0aa6 P 43999d813e564dc688fad116090c95ca -> Peer 9868dce521294d0b816bba7648f4ae5a (127.14.177.132:33683): Couldn't send request to peer 9868dce521294d0b816bba7648f4ae5a. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:23:40.079991 25199 leader_election.cc:304] T 5c7e7acbb09f4d49828fa652ca6abaac P 43999d813e564dc688fad116090c95ca [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3e07d64466934da4bb3c02262b1c2e38, 43999d813e564dc688fad116090c95ca; no voters: 9868dce521294d0b816bba7648f4ae5a
I20260709 13:23:40.080796 25601 raft_consensus.cc:2804] T 5c7e7acbb09f4d49828fa652ca6abaac P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:23:40.081321 25601 raft_consensus.cc:697] T 5c7e7acbb09f4d49828fa652ca6abaac P 43999d813e564dc688fad116090c95ca [term 1 LEADER]: Becoming Leader. State: Replica: 43999d813e564dc688fad116090c95ca, State: Running, Role: LEADER
I20260709 13:23:40.093089 25424 leader_election.cc:304] T 5c7e7acbb09f4d49828fa652ca6abaac P 9868dce521294d0b816bba7648f4ae5a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9868dce521294d0b816bba7648f4ae5a; no voters: 3e07d64466934da4bb3c02262b1c2e38, 43999d813e564dc688fad116090c95ca
I20260709 13:23:40.094331 25930 raft_consensus.cc:3060] T 5c7e7acbb09f4d49828fa652ca6abaac P 9868dce521294d0b816bba7648f4ae5a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:23:40.098063 25845 raft_consensus.cc:515] T 434a54fcee7d4aeeab42f739855c0e8a P 3e07d64466934da4bb3c02262b1c2e38 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } }
I20260709 13:23:40.082273 25601 consensus_queue.cc:237] T 5c7e7acbb09f4d49828fa652ca6abaac P 43999d813e564dc688fad116090c95ca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } }
I20260709 13:23:40.125579 25508 tablet_bootstrap.cc:654] T 821a5866429f4173820fb64336abee02 P 9868dce521294d0b816bba7648f4ae5a: Neither blocks nor log segments found. Creating new log.
I20260709 13:23:40.138381 25309 raft_consensus.cc:2468] T 03e9bfbd2f774476a4a377367796a2e8 P 233e7d29aac04505bf4e65f0a39a0c4c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3e07d64466934da4bb3c02262b1c2e38 in term 1.
I20260709 13:23:40.105675 25828 consensus_queue.cc:1048] T b9d550d881b04735bbf89751952c09ec P 43999d813e564dc688fad116090c95ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:23:40.145462 25385 raft_consensus.cc:2468] T 434a54fcee7d4aeeab42f739855c0e8a P 3e07d64466934da4bb3c02262b1c2e38 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 43999d813e564dc688fad116090c95ca in term 2.
I20260709 13:23:40.145439 25955 consensus_queue.cc:1048] T fa0da2c845dc4059968332f4db2e0aa6 P 43999d813e564dc688fad116090c95ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:23:40.146298 25930 raft_consensus.cc:2749] T 5c7e7acbb09f4d49828fa652ca6abaac P 9868dce521294d0b816bba7648f4ae5a [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:23:40.148764 25202 leader_election.cc:304] T 434a54fcee7d4aeeab42f739855c0e8a P 43999d813e564dc688fad116090c95ca [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 233e7d29aac04505bf4e65f0a39a0c4c, 43999d813e564dc688fad116090c95ca; no voters: 
I20260709 13:23:40.150548 25845 leader_election.cc:290] T 434a54fcee7d4aeeab42f739855c0e8a P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 2 election: Requested vote from peers 43999d813e564dc688fad116090c95ca (127.14.177.129:38155), 233e7d29aac04505bf4e65f0a39a0c4c (127.14.177.130:42957)
I20260709 13:23:40.161296 25961 raft_consensus.cc:2804] T 434a54fcee7d4aeeab42f739855c0e8a P 43999d813e564dc688fad116090c95ca [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:23:40.161963 25961 raft_consensus.cc:493] T 434a54fcee7d4aeeab42f739855c0e8a P 43999d813e564dc688fad116090c95ca [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:23:40.162436 25961 raft_consensus.cc:3060] T 434a54fcee7d4aeeab42f739855c0e8a P 43999d813e564dc688fad116090c95ca [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:23:40.160555 25236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434a54fcee7d4aeeab42f739855c0e8a" candidate_uuid: "3e07d64466934da4bb3c02262b1c2e38" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "43999d813e564dc688fad116090c95ca"
I20260709 13:23:40.152220 25350 leader_election.cc:304] T 87085ac2575e4b6a86dd1fc21e6f718b P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3e07d64466934da4bb3c02262b1c2e38, 43999d813e564dc688fad116090c95ca; no voters: 233e7d29aac04505bf4e65f0a39a0c4c
I20260709 13:23:40.153337 25309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434a54fcee7d4aeeab42f739855c0e8a" candidate_uuid: "3e07d64466934da4bb3c02262b1c2e38" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "233e7d29aac04505bf4e65f0a39a0c4c"
I20260709 13:23:40.166977 25845 raft_consensus.cc:2804] T 87085ac2575e4b6a86dd1fc21e6f718b P 3e07d64466934da4bb3c02262b1c2e38 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:23:40.167675 25309 raft_consensus.cc:2393] T 434a54fcee7d4aeeab42f739855c0e8a P 233e7d29aac04505bf4e65f0a39a0c4c [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 3e07d64466934da4bb3c02262b1c2e38 in current term 2: Already voted for candidate 233e7d29aac04505bf4e65f0a39a0c4c in this term.
I20260709 13:23:40.171644 25508 tablet_bootstrap.cc:492] T 821a5866429f4173820fb64336abee02 P 9868dce521294d0b816bba7648f4ae5a: No bootstrap required, opened a new log
I20260709 13:23:40.172250 25508 ts_tablet_manager.cc:1403] T 821a5866429f4173820fb64336abee02 P 9868dce521294d0b816bba7648f4ae5a: Time spent bootstrapping tablet: real 0.185s	user 0.016s	sys 0.001s
I20260709 13:23:40.155524 25505 tablet_bootstrap.cc:492] T 821a5866429f4173820fb64336abee02 P 43999d813e564dc688fad116090c95ca: Bootstrap starting.
I20260709 13:23:40.168033 25845 raft_consensus.cc:697] T 87085ac2575e4b6a86dd1fc21e6f718b P 3e07d64466934da4bb3c02262b1c2e38 [term 2 LEADER]: Becoming Leader. State: Replica: 3e07d64466934da4bb3c02262b1c2e38, State: Running, Role: LEADER
I20260709 13:23:40.178915 25508 raft_consensus.cc:359] T 821a5866429f4173820fb64336abee02 P 9868dce521294d0b816bba7648f4ae5a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } }
I20260709 13:23:40.174028 25845 consensus_queue.cc:237] T 87085ac2575e4b6a86dd1fc21e6f718b P 3e07d64466934da4bb3c02262b1c2e38 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } }
I20260709 13:23:40.172835 25929 raft_consensus.cc:515] T 55e8ed3aee404ce698d0626a87bd03aa P 9868dce521294d0b816bba7648f4ae5a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } }
I20260709 13:23:40.181919 25961 raft_consensus.cc:515] T 434a54fcee7d4aeeab42f739855c0e8a P 43999d813e564dc688fad116090c95ca [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } }
I20260709 13:23:40.185447 25508 raft_consensus.cc:385] T 821a5866429f4173820fb64336abee02 P 9868dce521294d0b816bba7648f4ae5a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:23:40.193148 25508 raft_consensus.cc:740] T 821a5866429f4173820fb64336abee02 P 9868dce521294d0b816bba7648f4ae5a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9868dce521294d0b816bba7648f4ae5a, State: Initialized, Role: FOLLOWER
I20260709 13:23:40.193288 25304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55e8ed3aee404ce698d0626a87bd03aa" candidate_uuid: "9868dce521294d0b816bba7648f4ae5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "233e7d29aac04505bf4e65f0a39a0c4c"
I20260709 13:23:40.191655 25132 catalog_manager.cc:5697] T fa0da2c845dc4059968332f4db2e0aa6 P 43999d813e564dc688fad116090c95ca reported cstate change: term changed from 0 to 1, leader changed from <none> to 43999d813e564dc688fad116090c95ca (127.14.177.129). New cstate: current_term: 1 leader_uuid: "43999d813e564dc688fad116090c95ca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } health_report { overall_health: UNKNOWN } } }
I20260709 13:23:40.193996 25304 raft_consensus.cc:3060] T 55e8ed3aee404ce698d0626a87bd03aa P 233e7d29aac04505bf4e65f0a39a0c4c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:23:40.194100 25508 consensus_queue.cc:260] T 821a5866429f4173820fb64336abee02 P 9868dce521294d0b816bba7648f4ae5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } }
I20260709 13:23:40.190946 25236 raft_consensus.cc:2368] T 434a54fcee7d4aeeab42f739855c0e8a P 43999d813e564dc688fad116090c95ca [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 3e07d64466934da4bb3c02262b1c2e38 for earlier term 2. Current term is 3.
I20260709 13:23:40.200286 25508 ts_tablet_manager.cc:1434] T 821a5866429f4173820fb64336abee02 P 9868dce521294d0b816bba7648f4ae5a: Time spent starting tablet: real 0.028s	user 0.008s	sys 0.000s
I20260709 13:23:40.201380 25508 tablet_bootstrap.cc:492] T fa0da2c845dc4059968332f4db2e0aa6 P 9868dce521294d0b816bba7648f4ae5a: Bootstrap starting.
I20260709 13:23:40.206570 25350 leader_election.cc:400] T 434a54fcee7d4aeeab42f739855c0e8a P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 2 election: Vote denied by peer 43999d813e564dc688fad116090c95ca (127.14.177.129:38155) with higher term. Message: Invalid argument: T 434a54fcee7d4aeeab42f739855c0e8a P 43999d813e564dc688fad116090c95ca [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 3e07d64466934da4bb3c02262b1c2e38 for earlier term 2. Current term is 3.
I20260709 13:23:40.207001 25350 leader_election.cc:403] T 434a54fcee7d4aeeab42f739855c0e8a P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 2 election: Cancelling election due to peer responding with higher term
I20260709 13:23:40.211840 25132 catalog_manager.cc:5697] T b9d550d881b04735bbf89751952c09ec P 43999d813e564dc688fad116090c95ca reported cstate change: term changed from 0 to 1, leader changed from <none> to 43999d813e564dc688fad116090c95ca (127.14.177.129). New cstate: current_term: 1 leader_uuid: "43999d813e564dc688fad116090c95ca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } health_report { overall_health: HEALTHY } } }
I20260709 13:23:40.215619 25979 raft_consensus.cc:3060] T 434a54fcee7d4aeeab42f739855c0e8a P 3e07d64466934da4bb3c02262b1c2e38 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:23:40.233877 25505 tablet_bootstrap.cc:654] T 821a5866429f4173820fb64336abee02 P 43999d813e564dc688fad116090c95ca: Neither blocks nor log segments found. Creating new log.
I20260709 13:23:40.262816 25961 leader_election.cc:290] T 434a54fcee7d4aeeab42f739855c0e8a P 43999d813e564dc688fad116090c95ca [CANDIDATE]: Term 3 election: Requested vote from peers 233e7d29aac04505bf4e65f0a39a0c4c (127.14.177.130:42957), 3e07d64466934da4bb3c02262b1c2e38 (127.14.177.131:41539)
I20260709 13:23:40.264343 25379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434a54fcee7d4aeeab42f739855c0e8a" candidate_uuid: "43999d813e564dc688fad116090c95ca" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3e07d64466934da4bb3c02262b1c2e38"
I20260709 13:23:40.270624 25929 leader_election.cc:290] T 55e8ed3aee404ce698d0626a87bd03aa P 9868dce521294d0b816bba7648f4ae5a [CANDIDATE]: Term 1 election: Requested vote from peers 233e7d29aac04505bf4e65f0a39a0c4c (127.14.177.130:42957), 3e07d64466934da4bb3c02262b1c2e38 (127.14.177.131:41539)
I20260709 13:23:40.282519 25304 raft_consensus.cc:2468] T 55e8ed3aee404ce698d0626a87bd03aa P 233e7d29aac04505bf4e65f0a39a0c4c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9868dce521294d0b816bba7648f4ae5a in term 1.
I20260709 13:23:40.283468 25385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55e8ed3aee404ce698d0626a87bd03aa" candidate_uuid: "9868dce521294d0b816bba7648f4ae5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e07d64466934da4bb3c02262b1c2e38"
I20260709 13:23:40.284178 25385 raft_consensus.cc:3060] T 55e8ed3aee404ce698d0626a87bd03aa P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:23:40.287320 25508 tablet_bootstrap.cc:654] T fa0da2c845dc4059968332f4db2e0aa6 P 9868dce521294d0b816bba7648f4ae5a: Neither blocks nor log segments found. Creating new log.
I20260709 13:23:40.268642 25925 consensus_queue.cc:1048] T b9d550d881b04735bbf89751952c09ec P 43999d813e564dc688fad116090c95ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.020s
I20260709 13:23:40.285207 25425 leader_election.cc:304] T 55e8ed3aee404ce698d0626a87bd03aa P 9868dce521294d0b816bba7648f4ae5a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 233e7d29aac04505bf4e65f0a39a0c4c, 9868dce521294d0b816bba7648f4ae5a; no voters: 
I20260709 13:23:40.302079 25979 raft_consensus.cc:2749] T 434a54fcee7d4aeeab42f739855c0e8a P 3e07d64466934da4bb3c02262b1c2e38 [term 3 FOLLOWER]: Leader election lost for term 2. Reason: Vote denied by peer 43999d813e564dc688fad116090c95ca (127.14.177.129:38155) with higher term. Message: Invalid argument: T 434a54fcee7d4aeeab42f739855c0e8a P 43999d813e564dc688fad116090c95ca [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 3e07d64466934da4bb3c02262b1c2e38 for earlier term 2. Current term is 3.
I20260709 13:23:40.302498 25930 raft_consensus.cc:2804] T 55e8ed3aee404ce698d0626a87bd03aa P 9868dce521294d0b816bba7648f4ae5a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:23:40.303027 25930 raft_consensus.cc:697] T 55e8ed3aee404ce698d0626a87bd03aa P 9868dce521294d0b816bba7648f4ae5a [term 1 LEADER]: Becoming Leader. State: Replica: 9868dce521294d0b816bba7648f4ae5a, State: Running, Role: LEADER
I20260709 13:23:40.303939 25930 consensus_queue.cc:237] T 55e8ed3aee404ce698d0626a87bd03aa P 9868dce521294d0b816bba7648f4ae5a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } }
I20260709 13:23:40.306806 25302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434a54fcee7d4aeeab42f739855c0e8a" candidate_uuid: "43999d813e564dc688fad116090c95ca" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "233e7d29aac04505bf4e65f0a39a0c4c"
I20260709 13:23:40.317970 25302 raft_consensus.cc:3060] T 434a54fcee7d4aeeab42f739855c0e8a P 233e7d29aac04505bf4e65f0a39a0c4c [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:23:40.325059 25505 tablet_bootstrap.cc:492] T 821a5866429f4173820fb64336abee02 P 43999d813e564dc688fad116090c95ca: No bootstrap required, opened a new log
I20260709 13:23:40.325829 25505 ts_tablet_manager.cc:1403] T 821a5866429f4173820fb64336abee02 P 43999d813e564dc688fad116090c95ca: Time spent bootstrapping tablet: real 0.171s	user 0.012s	sys 0.012s
I20260709 13:23:40.331048 25385 raft_consensus.cc:2468] T 55e8ed3aee404ce698d0626a87bd03aa P 3e07d64466934da4bb3c02262b1c2e38 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9868dce521294d0b816bba7648f4ae5a in term 1.
I20260709 13:23:40.342777 25379 raft_consensus.cc:2468] T 434a54fcee7d4aeeab42f739855c0e8a P 3e07d64466934da4bb3c02262b1c2e38 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 43999d813e564dc688fad116090c95ca in term 3.
I20260709 13:23:40.345477 25199 leader_election.cc:304] T 434a54fcee7d4aeeab42f739855c0e8a P 43999d813e564dc688fad116090c95ca [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e07d64466934da4bb3c02262b1c2e38, 43999d813e564dc688fad116090c95ca; no voters: 
I20260709 13:23:40.348105 25977 raft_consensus.cc:2804] T 434a54fcee7d4aeeab42f739855c0e8a P 43999d813e564dc688fad116090c95ca [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:23:40.348726 25977 raft_consensus.cc:697] T 434a54fcee7d4aeeab42f739855c0e8a P 43999d813e564dc688fad116090c95ca [term 3 LEADER]: Becoming Leader. State: Replica: 43999d813e564dc688fad116090c95ca, State: Running, Role: LEADER
I20260709 13:23:40.349747 25977 consensus_queue.cc:237] T 434a54fcee7d4aeeab42f739855c0e8a P 43999d813e564dc688fad116090c95ca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } }
I20260709 13:23:40.367278 25302 raft_consensus.cc:2468] T 434a54fcee7d4aeeab42f739855c0e8a P 233e7d29aac04505bf4e65f0a39a0c4c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 43999d813e564dc688fad116090c95ca in term 3.
I20260709 13:23:40.372234 25508 tablet_bootstrap.cc:492] T fa0da2c845dc4059968332f4db2e0aa6 P 9868dce521294d0b816bba7648f4ae5a: No bootstrap required, opened a new log
I20260709 13:23:40.372761 25508 ts_tablet_manager.cc:1403] T fa0da2c845dc4059968332f4db2e0aa6 P 9868dce521294d0b816bba7648f4ae5a: Time spent bootstrapping tablet: real 0.172s	user 0.010s	sys 0.040s
I20260709 13:23:40.376488 25508 raft_consensus.cc:359] T fa0da2c845dc4059968332f4db2e0aa6 P 9868dce521294d0b816bba7648f4ae5a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } }
I20260709 13:23:40.377244 25508 raft_consensus.cc:385] T fa0da2c845dc4059968332f4db2e0aa6 P 9868dce521294d0b816bba7648f4ae5a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:23:40.378665 25508 raft_consensus.cc:740] T fa0da2c845dc4059968332f4db2e0aa6 P 9868dce521294d0b816bba7648f4ae5a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9868dce521294d0b816bba7648f4ae5a, State: Initialized, Role: FOLLOWER
I20260709 13:23:40.379804 25508 consensus_queue.cc:260] T fa0da2c845dc4059968332f4db2e0aa6 P 9868dce521294d0b816bba7648f4ae5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } }
I20260709 13:23:40.381337 25508 ts_tablet_manager.cc:1434] T fa0da2c845dc4059968332f4db2e0aa6 P 9868dce521294d0b816bba7648f4ae5a: Time spent starting tablet: real 0.008s	user 0.004s	sys 0.000s
I20260709 13:23:40.387362 25508 tablet_bootstrap.cc:492] T 11949f56648c47e0a43c3140af99a304 P 9868dce521294d0b816bba7648f4ae5a: Bootstrap starting.
I20260709 13:23:40.396597 25505 raft_consensus.cc:359] T 821a5866429f4173820fb64336abee02 P 43999d813e564dc688fad116090c95ca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } }
I20260709 13:23:40.397392 25505 raft_consensus.cc:385] T 821a5866429f4173820fb64336abee02 P 43999d813e564dc688fad116090c95ca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:23:40.397683 25505 raft_consensus.cc:740] T 821a5866429f4173820fb64336abee02 P 43999d813e564dc688fad116090c95ca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43999d813e564dc688fad116090c95ca, State: Initialized, Role: FOLLOWER
I20260709 13:23:40.401067 25508 tablet_bootstrap.cc:654] T 11949f56648c47e0a43c3140af99a304 P 9868dce521294d0b816bba7648f4ae5a: Neither blocks nor log segments found. Creating new log.
I20260709 13:23:40.412901 25505 consensus_queue.cc:260] T 821a5866429f4173820fb64336abee02 P 43999d813e564dc688fad116090c95ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } }
I20260709 13:23:40.419430 25505 ts_tablet_manager.cc:1434] T 821a5866429f4173820fb64336abee02 P 43999d813e564dc688fad116090c95ca: Time spent starting tablet: real 0.093s	user 0.004s	sys 0.006s
I20260709 13:23:40.422657 25508 tablet_bootstrap.cc:492] T 11949f56648c47e0a43c3140af99a304 P 9868dce521294d0b816bba7648f4ae5a: No bootstrap required, opened a new log
I20260709 13:23:40.423256 25508 ts_tablet_manager.cc:1403] T 11949f56648c47e0a43c3140af99a304 P 9868dce521294d0b816bba7648f4ae5a: Time spent bootstrapping tablet: real 0.041s	user 0.016s	sys 0.003s
I20260709 13:23:40.426913 25508 raft_consensus.cc:359] T 11949f56648c47e0a43c3140af99a304 P 9868dce521294d0b816bba7648f4ae5a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } }
I20260709 13:23:40.427685 25508 raft_consensus.cc:385] T 11949f56648c47e0a43c3140af99a304 P 9868dce521294d0b816bba7648f4ae5a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:23:40.427981 25508 raft_consensus.cc:740] T 11949f56648c47e0a43c3140af99a304 P 9868dce521294d0b816bba7648f4ae5a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9868dce521294d0b816bba7648f4ae5a, State: Initialized, Role: FOLLOWER
I20260709 13:23:40.428671 25508 consensus_queue.cc:260] T 11949f56648c47e0a43c3140af99a304 P 9868dce521294d0b816bba7648f4ae5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } }
I20260709 13:23:40.430609 25508 ts_tablet_manager.cc:1434] T 11949f56648c47e0a43c3140af99a304 P 9868dce521294d0b816bba7648f4ae5a: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 13:23:40.433264 25977 raft_consensus.cc:493] T 821a5866429f4173820fb64336abee02 P 43999d813e564dc688fad116090c95ca [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:23:40.433758 25977 raft_consensus.cc:515] T 821a5866429f4173820fb64336abee02 P 43999d813e564dc688fad116090c95ca [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } }
I20260709 13:23:40.437143 25385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "821a5866429f4173820fb64336abee02" candidate_uuid: "43999d813e564dc688fad116090c95ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e07d64466934da4bb3c02262b1c2e38" is_pre_election: true
I20260709 13:23:40.437394 25452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "821a5866429f4173820fb64336abee02" candidate_uuid: "43999d813e564dc688fad116090c95ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9868dce521294d0b816bba7648f4ae5a" is_pre_election: true
I20260709 13:23:40.437965 25385 raft_consensus.cc:2468] T 821a5866429f4173820fb64336abee02 P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 43999d813e564dc688fad116090c95ca in term 0.
I20260709 13:23:40.438042 25452 raft_consensus.cc:2468] T 821a5866429f4173820fb64336abee02 P 9868dce521294d0b816bba7648f4ae5a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 43999d813e564dc688fad116090c95ca in term 0.
I20260709 13:23:40.439585 25199 leader_election.cc:304] T 821a5866429f4173820fb64336abee02 P 43999d813e564dc688fad116090c95ca [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e07d64466934da4bb3c02262b1c2e38, 43999d813e564dc688fad116090c95ca; no voters: 
I20260709 13:23:40.440431 25961 raft_consensus.cc:2804] T 821a5866429f4173820fb64336abee02 P 43999d813e564dc688fad116090c95ca [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:23:40.440855 25961 raft_consensus.cc:493] T 821a5866429f4173820fb64336abee02 P 43999d813e564dc688fad116090c95ca [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:23:40.441248 25961 raft_consensus.cc:3060] T 821a5866429f4173820fb64336abee02 P 43999d813e564dc688fad116090c95ca [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:23:40.455353 25977 leader_election.cc:290] T 821a5866429f4173820fb64336abee02 P 43999d813e564dc688fad116090c95ca [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3e07d64466934da4bb3c02262b1c2e38 (127.14.177.131:41539), 9868dce521294d0b816bba7648f4ae5a (127.14.177.132:33683)
I20260709 13:23:40.459512 25961 raft_consensus.cc:515] T 821a5866429f4173820fb64336abee02 P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } }
I20260709 13:23:40.462612 25961 leader_election.cc:290] T 821a5866429f4173820fb64336abee02 P 43999d813e564dc688fad116090c95ca [CANDIDATE]: Term 1 election: Requested vote from peers 3e07d64466934da4bb3c02262b1c2e38 (127.14.177.131:41539), 9868dce521294d0b816bba7648f4ae5a (127.14.177.132:33683)
I20260709 13:23:40.465740 25452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "821a5866429f4173820fb64336abee02" candidate_uuid: "43999d813e564dc688fad116090c95ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9868dce521294d0b816bba7648f4ae5a"
I20260709 13:23:40.466416 25452 raft_consensus.cc:3060] T 821a5866429f4173820fb64336abee02 P 9868dce521294d0b816bba7648f4ae5a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:23:40.467670 25381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "821a5866429f4173820fb64336abee02" candidate_uuid: "43999d813e564dc688fad116090c95ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e07d64466934da4bb3c02262b1c2e38"
I20260709 13:23:40.468380 25381 raft_consensus.cc:3060] T 821a5866429f4173820fb64336abee02 P 3e07d64466934da4bb3c02262b1c2e38 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:23:40.475409 25452 raft_consensus.cc:2468] T 821a5866429f4173820fb64336abee02 P 9868dce521294d0b816bba7648f4ae5a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 43999d813e564dc688fad116090c95ca in term 1.
I20260709 13:23:40.478453 25130 catalog_manager.cc:5697] T 03e9bfbd2f774476a4a377367796a2e8 P 3e07d64466934da4bb3c02262b1c2e38 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3e07d64466934da4bb3c02262b1c2e38 (127.14.177.131). New cstate: current_term: 1 leader_uuid: "3e07d64466934da4bb3c02262b1c2e38" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } health_report { overall_health: HEALTHY } } }
I20260709 13:23:40.497851 25130 catalog_manager.cc:5697] T 87085ac2575e4b6a86dd1fc21e6f718b P 3e07d64466934da4bb3c02262b1c2e38 reported cstate change: term changed from 1 to 2, leader changed from 43999d813e564dc688fad116090c95ca (127.14.177.129) to 3e07d64466934da4bb3c02262b1c2e38 (127.14.177.131). New cstate: current_term: 2 leader_uuid: "3e07d64466934da4bb3c02262b1c2e38" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } health_report { overall_health: HEALTHY } } }
I20260709 13:23:40.507467 25381 raft_consensus.cc:2468] T 821a5866429f4173820fb64336abee02 P 3e07d64466934da4bb3c02262b1c2e38 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 43999d813e564dc688fad116090c95ca in term 1.
I20260709 13:23:40.516692 25199 leader_election.cc:304] T 821a5866429f4173820fb64336abee02 P 43999d813e564dc688fad116090c95ca [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e07d64466934da4bb3c02262b1c2e38, 43999d813e564dc688fad116090c95ca; no voters: 
I20260709 13:23:40.518699 25955 raft_consensus.cc:2804] T 821a5866429f4173820fb64336abee02 P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:23:40.519214 25955 raft_consensus.cc:697] T 821a5866429f4173820fb64336abee02 P 43999d813e564dc688fad116090c95ca [term 1 LEADER]: Becoming Leader. State: Replica: 43999d813e564dc688fad116090c95ca, State: Running, Role: LEADER
I20260709 13:23:40.520066 25955 consensus_queue.cc:237] T 821a5866429f4173820fb64336abee02 P 43999d813e564dc688fad116090c95ca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } }
I20260709 13:23:40.529764 25979 consensus_queue.cc:1048] T 03e9bfbd2f774476a4a377367796a2e8 P 3e07d64466934da4bb3c02262b1c2e38 [LEADER]: Connected to new peer: Peer: permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:23:40.530617 25129 catalog_manager.cc:5697] T 55e8ed3aee404ce698d0626a87bd03aa P 9868dce521294d0b816bba7648f4ae5a reported cstate change: term changed from 0 to 1, leader changed from <none> to 9868dce521294d0b816bba7648f4ae5a (127.14.177.132). New cstate: current_term: 1 leader_uuid: "9868dce521294d0b816bba7648f4ae5a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } health_report { overall_health: HEALTHY } } }
I20260709 13:23:40.545392 25930 raft_consensus.cc:493] T 11949f56648c47e0a43c3140af99a304 P 9868dce521294d0b816bba7648f4ae5a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:23:40.545918 25930 raft_consensus.cc:515] T 11949f56648c47e0a43c3140af99a304 P 9868dce521294d0b816bba7648f4ae5a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } }
I20260709 13:23:40.551451 25930 leader_election.cc:290] T 11949f56648c47e0a43c3140af99a304 P 9868dce521294d0b816bba7648f4ae5a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 43999d813e564dc688fad116090c95ca (127.14.177.129:38155), 233e7d29aac04505bf4e65f0a39a0c4c (127.14.177.130:42957)
I20260709 13:23:40.552908 25302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11949f56648c47e0a43c3140af99a304" candidate_uuid: "9868dce521294d0b816bba7648f4ae5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" is_pre_election: true
I20260709 13:23:40.554492 25302 raft_consensus.cc:2468] T 11949f56648c47e0a43c3140af99a304 P 233e7d29aac04505bf4e65f0a39a0c4c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9868dce521294d0b816bba7648f4ae5a in term 0.
I20260709 13:23:40.554947 25230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11949f56648c47e0a43c3140af99a304" candidate_uuid: "9868dce521294d0b816bba7648f4ae5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43999d813e564dc688fad116090c95ca" is_pre_election: true
I20260709 13:23:40.555712 25230 raft_consensus.cc:2468] T 11949f56648c47e0a43c3140af99a304 P 43999d813e564dc688fad116090c95ca [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9868dce521294d0b816bba7648f4ae5a in term 0.
I20260709 13:23:40.556144 25425 leader_election.cc:304] T 11949f56648c47e0a43c3140af99a304 P 9868dce521294d0b816bba7648f4ae5a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 233e7d29aac04505bf4e65f0a39a0c4c, 9868dce521294d0b816bba7648f4ae5a; no voters: 
I20260709 13:23:40.557137 25132 catalog_manager.cc:5697] T 5c7e7acbb09f4d49828fa652ca6abaac P 43999d813e564dc688fad116090c95ca reported cstate change: term changed from 0 to 1, leader changed from <none> to 43999d813e564dc688fad116090c95ca (127.14.177.129). New cstate: current_term: 1 leader_uuid: "43999d813e564dc688fad116090c95ca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } health_report { overall_health: HEALTHY } } }
I20260709 13:23:40.557930 25931 raft_consensus.cc:2804] T 11949f56648c47e0a43c3140af99a304 P 9868dce521294d0b816bba7648f4ae5a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:23:40.559422 25931 raft_consensus.cc:493] T 11949f56648c47e0a43c3140af99a304 P 9868dce521294d0b816bba7648f4ae5a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:23:40.559787 25931 raft_consensus.cc:3060] T 11949f56648c47e0a43c3140af99a304 P 9868dce521294d0b816bba7648f4ae5a [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:23:40.565923 25182 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:23:40.567453 25955 consensus_queue.cc:1048] T 5c7e7acbb09f4d49828fa652ca6abaac P 43999d813e564dc688fad116090c95ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:23:40.569197 25132 catalog_manager.cc:5697] T 434a54fcee7d4aeeab42f739855c0e8a P 43999d813e564dc688fad116090c95ca reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "43999d813e564dc688fad116090c95ca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } health_report { overall_health: UNKNOWN } } }
I20260709 13:23:40.576179 25451 raft_consensus.cc:3060] T fa0da2c845dc4059968332f4db2e0aa6 P 9868dce521294d0b816bba7648f4ae5a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:23:40.581530 25979 consensus_queue.cc:1048] T 03e9bfbd2f774476a4a377367796a2e8 P 3e07d64466934da4bb3c02262b1c2e38 [LEADER]: Connected to new peer: Peer: permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:23:40.610890 25977 consensus_queue.cc:1048] T 5c7e7acbb09f4d49828fa652ca6abaac P 43999d813e564dc688fad116090c95ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:23:40.574833 25931 raft_consensus.cc:515] T 11949f56648c47e0a43c3140af99a304 P 9868dce521294d0b816bba7648f4ae5a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } }
I20260709 13:23:40.614693 25931 leader_election.cc:290] T 11949f56648c47e0a43c3140af99a304 P 9868dce521294d0b816bba7648f4ae5a [CANDIDATE]: Term 1 election: Requested vote from peers 43999d813e564dc688fad116090c95ca (127.14.177.129:38155), 233e7d29aac04505bf4e65f0a39a0c4c (127.14.177.130:42957)
I20260709 13:23:40.616086 25236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11949f56648c47e0a43c3140af99a304" candidate_uuid: "9868dce521294d0b816bba7648f4ae5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43999d813e564dc688fad116090c95ca"
I20260709 13:23:40.617013 25236 raft_consensus.cc:3060] T 11949f56648c47e0a43c3140af99a304 P 43999d813e564dc688fad116090c95ca [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:23:40.619436 25309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11949f56648c47e0a43c3140af99a304" candidate_uuid: "9868dce521294d0b816bba7648f4ae5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "233e7d29aac04505bf4e65f0a39a0c4c"
I20260709 13:23:40.637758 25309 raft_consensus.cc:3060] T 11949f56648c47e0a43c3140af99a304 P 233e7d29aac04505bf4e65f0a39a0c4c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:23:40.641142 25236 raft_consensus.cc:2468] T 11949f56648c47e0a43c3140af99a304 P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9868dce521294d0b816bba7648f4ae5a in term 1.
I20260709 13:23:40.642441 25424 leader_election.cc:304] T 11949f56648c47e0a43c3140af99a304 P 9868dce521294d0b816bba7648f4ae5a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 43999d813e564dc688fad116090c95ca, 9868dce521294d0b816bba7648f4ae5a; no voters: 
I20260709 13:23:40.644779 25930 raft_consensus.cc:2804] T 11949f56648c47e0a43c3140af99a304 P 9868dce521294d0b816bba7648f4ae5a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:23:40.645460 25930 raft_consensus.cc:697] T 11949f56648c47e0a43c3140af99a304 P 9868dce521294d0b816bba7648f4ae5a [term 1 LEADER]: Becoming Leader. State: Replica: 9868dce521294d0b816bba7648f4ae5a, State: Running, Role: LEADER
I20260709 13:23:40.647830 25930 consensus_queue.cc:237] T 11949f56648c47e0a43c3140af99a304 P 9868dce521294d0b816bba7648f4ae5a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } }
I20260709 13:23:40.661851 25130 catalog_manager.cc:5697] T 821a5866429f4173820fb64336abee02 P 43999d813e564dc688fad116090c95ca reported cstate change: term changed from 0 to 1, leader changed from <none> to 43999d813e564dc688fad116090c95ca (127.14.177.129). New cstate: current_term: 1 leader_uuid: "43999d813e564dc688fad116090c95ca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } health_report { overall_health: HEALTHY } } }
I20260709 13:23:40.667709 25309 raft_consensus.cc:2468] T 11949f56648c47e0a43c3140af99a304 P 233e7d29aac04505bf4e65f0a39a0c4c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9868dce521294d0b816bba7648f4ae5a in term 1.
I20260709 13:23:40.687532 25132 catalog_manager.cc:5697] T 11949f56648c47e0a43c3140af99a304 P 9868dce521294d0b816bba7648f4ae5a reported cstate change: term changed from 0 to 1, leader changed from <none> to 9868dce521294d0b816bba7648f4ae5a (127.14.177.132). New cstate: current_term: 1 leader_uuid: "9868dce521294d0b816bba7648f4ae5a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } health_report { overall_health: UNKNOWN } } }
I20260709 13:23:40.730631 25309 raft_consensus.cc:1275] T 434a54fcee7d4aeeab42f739855c0e8a P 233e7d29aac04505bf4e65f0a39a0c4c [term 3 FOLLOWER]: Refusing update from remote peer 43999d813e564dc688fad116090c95ca: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:23:40.731914 25977 consensus_queue.cc:1048] T 434a54fcee7d4aeeab42f739855c0e8a P 43999d813e564dc688fad116090c95ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:23:40.735481 25931 consensus_queue.cc:1048] T 55e8ed3aee404ce698d0626a87bd03aa P 9868dce521294d0b816bba7648f4ae5a [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:23:40.739717 25302 raft_consensus.cc:1275] T 87085ac2575e4b6a86dd1fc21e6f718b P 233e7d29aac04505bf4e65f0a39a0c4c [term 2 FOLLOWER]: Refusing update from remote peer 3e07d64466934da4bb3c02262b1c2e38: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:23:40.741717 25236 raft_consensus.cc:1275] T 87085ac2575e4b6a86dd1fc21e6f718b P 43999d813e564dc688fad116090c95ca [term 2 FOLLOWER]: Refusing update from remote peer 3e07d64466934da4bb3c02262b1c2e38: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:23:40.743845 25979 consensus_queue.cc:1048] T 87085ac2575e4b6a86dd1fc21e6f718b P 3e07d64466934da4bb3c02262b1c2e38 [LEADER]: Connected to new peer: Peer: permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:23:40.747416 25971 consensus_queue.cc:1048] T 87085ac2575e4b6a86dd1fc21e6f718b P 3e07d64466934da4bb3c02262b1c2e38 [LEADER]: Connected to new peer: Peer: permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:23:40.774529 25385 raft_consensus.cc:1275] T 434a54fcee7d4aeeab42f739855c0e8a P 3e07d64466934da4bb3c02262b1c2e38 [term 3 FOLLOWER]: Refusing update from remote peer 43999d813e564dc688fad116090c95ca: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:23:40.775978 25601 consensus_queue.cc:1048] T 434a54fcee7d4aeeab42f739855c0e8a P 43999d813e564dc688fad116090c95ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:23:40.775918 25932 consensus_queue.cc:1048] T 55e8ed3aee404ce698d0626a87bd03aa P 9868dce521294d0b816bba7648f4ae5a [LEADER]: Connected to new peer: Peer: permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:23:40.904593 25961 consensus_queue.cc:1048] T 821a5866429f4173820fb64336abee02 P 43999d813e564dc688fad116090c95ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:23:40.926179 25955 consensus_queue.cc:1048] T 821a5866429f4173820fb64336abee02 P 43999d813e564dc688fad116090c95ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:23:41.081072 25932 consensus_queue.cc:1048] T 11949f56648c47e0a43c3140af99a304 P 9868dce521294d0b816bba7648f4ae5a [LEADER]: Connected to new peer: Peer: permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:23:41.092648 25847 consensus_queue.cc:1048] T 11949f56648c47e0a43c3140af99a304 P 9868dce521294d0b816bba7648f4ae5a [LEADER]: Connected to new peer: Peer: permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 13:23:42.945199 25200 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.829s	user 0.012s	sys 0.000s
W20260709 13:23:42.945593 25200 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.830s	user 0.012s	sys 0.000s
I20260709 13:23:42.956430 25977 raft_consensus.cc:493] T 1f15ec62698c4a8cbd84f2b3d516d2ea P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3e07d64466934da4bb3c02262b1c2e38)
I20260709 13:23:42.957269 25977 raft_consensus.cc:515] T 1f15ec62698c4a8cbd84f2b3d516d2ea P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } }
I20260709 13:23:42.959092 25311 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60208 (request call id 199) took 1017 ms. Trace:
I20260709 13:23:42.963050 25311 rpcz_store.cc:276] 0709 13:23:41.941229 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:23:41.941625 (+   396us) service_pool.cc:224] Handling call
0709 13:23:41.941956 (+   331us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:23:41.942166 (+   210us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:23:41.942200 (+    34us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:23:41.942246 (+    46us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:23:41.942289 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:23:41.942357 (+    68us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:23:42.959004 (+1016647us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:23:42.967015 25977 leader_election.cc:290] T 1f15ec62698c4a8cbd84f2b3d516d2ea P 43999d813e564dc688fad116090c95ca [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3e07d64466934da4bb3c02262b1c2e38 (127.14.177.131:41539), 9868dce521294d0b816bba7648f4ae5a (127.14.177.132:33683)
I20260709 13:23:42.970737 25955 raft_consensus.cc:493] T 0c4f6b7401db47d59a4f34ed0e8b5350 P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9868dce521294d0b816bba7648f4ae5a)
I20260709 13:23:42.971375 25955 raft_consensus.cc:515] T 0c4f6b7401db47d59a4f34ed0e8b5350 P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } }
I20260709 13:23:42.973564 25955 leader_election.cc:290] T 0c4f6b7401db47d59a4f34ed0e8b5350 P 43999d813e564dc688fad116090c95ca [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9868dce521294d0b816bba7648f4ae5a (127.14.177.132:33683), 233e7d29aac04505bf4e65f0a39a0c4c (127.14.177.130:42957)
I20260709 13:23:42.981937 26019 raft_consensus.cc:493] T 0d29b499fa4c41f1b974782cd0c9c1c2 P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 233e7d29aac04505bf4e65f0a39a0c4c)
I20260709 13:23:42.982582 26019 raft_consensus.cc:515] T 0d29b499fa4c41f1b974782cd0c9c1c2 P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } }
I20260709 13:23:42.985495 26019 leader_election.cc:290] T 0d29b499fa4c41f1b974782cd0c9c1c2 P 43999d813e564dc688fad116090c95ca [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 233e7d29aac04505bf4e65f0a39a0c4c (127.14.177.130:42957), 3e07d64466934da4bb3c02262b1c2e38 (127.14.177.131:41539)
I20260709 13:23:42.989358 26019 raft_consensus.cc:493] T ce98a3f3b3eb4c63bcaa355ec7cd64a4 P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 233e7d29aac04505bf4e65f0a39a0c4c)
I20260709 13:23:42.989969 26019 raft_consensus.cc:515] T ce98a3f3b3eb4c63bcaa355ec7cd64a4 P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } }
I20260709 13:23:42.992329 26019 leader_election.cc:290] T ce98a3f3b3eb4c63bcaa355ec7cd64a4 P 43999d813e564dc688fad116090c95ca [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 233e7d29aac04505bf4e65f0a39a0c4c (127.14.177.130:42957), 3e07d64466934da4bb3c02262b1c2e38 (127.14.177.131:41539)
I20260709 13:23:43.006435 26019 raft_consensus.cc:493] T 280d65245f4f4358b54bef93e18aca92 P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9868dce521294d0b816bba7648f4ae5a)
I20260709 13:23:43.007011 26019 raft_consensus.cc:515] T 280d65245f4f4358b54bef93e18aca92 P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } }
I20260709 13:23:43.007541 25955 raft_consensus.cc:493] T b85a238bbc3b4621881603a534c7bd9b P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 233e7d29aac04505bf4e65f0a39a0c4c)
I20260709 13:23:43.008307 25955 raft_consensus.cc:515] T b85a238bbc3b4621881603a534c7bd9b P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } }
I20260709 13:23:43.009506 26019 leader_election.cc:290] T 280d65245f4f4358b54bef93e18aca92 P 43999d813e564dc688fad116090c95ca [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3e07d64466934da4bb3c02262b1c2e38 (127.14.177.131:41539), 9868dce521294d0b816bba7648f4ae5a (127.14.177.132:33683)
I20260709 13:23:43.011659 25955 leader_election.cc:290] T b85a238bbc3b4621881603a534c7bd9b P 43999d813e564dc688fad116090c95ca [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9868dce521294d0b816bba7648f4ae5a (127.14.177.132:33683), 233e7d29aac04505bf4e65f0a39a0c4c (127.14.177.130:42957)
I20260709 13:23:43.039927 26020 raft_consensus.cc:493] T 1f15ec62698c4a8cbd84f2b3d516d2ea P 9868dce521294d0b816bba7648f4ae5a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3e07d64466934da4bb3c02262b1c2e38)
I20260709 13:23:43.040735 26020 raft_consensus.cc:515] T 1f15ec62698c4a8cbd84f2b3d516d2ea P 9868dce521294d0b816bba7648f4ae5a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } }
I20260709 13:23:43.044229 26020 leader_election.cc:290] T 1f15ec62698c4a8cbd84f2b3d516d2ea P 9868dce521294d0b816bba7648f4ae5a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3e07d64466934da4bb3c02262b1c2e38 (127.14.177.131:41539), 43999d813e564dc688fad116090c95ca (127.14.177.129:38155)
I20260709 13:23:43.081454 26032 raft_consensus.cc:493] T d053b9ae38de4173a0aded4da494bd98 P 233e7d29aac04505bf4e65f0a39a0c4c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9868dce521294d0b816bba7648f4ae5a)
I20260709 13:23:43.082152 26032 raft_consensus.cc:515] T d053b9ae38de4173a0aded4da494bd98 P 233e7d29aac04505bf4e65f0a39a0c4c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } }
I20260709 13:23:43.084452 26032 leader_election.cc:290] T d053b9ae38de4173a0aded4da494bd98 P 233e7d29aac04505bf4e65f0a39a0c4c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9868dce521294d0b816bba7648f4ae5a (127.14.177.132:33683), 43999d813e564dc688fad116090c95ca (127.14.177.129:38155)
I20260709 13:23:43.085078 26018 raft_consensus.cc:493] T 11949f56648c47e0a43c3140af99a304 P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9868dce521294d0b816bba7648f4ae5a)
I20260709 13:23:43.085721 26018 raft_consensus.cc:515] T 11949f56648c47e0a43c3140af99a304 P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } }
I20260709 13:23:43.081754 26017 raft_consensus.cc:493] T e9e342ecda4b46619ce12452b2cf7154 P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9868dce521294d0b816bba7648f4ae5a)
I20260709 13:23:43.087690 26017 raft_consensus.cc:515] T e9e342ecda4b46619ce12452b2cf7154 P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } }
I20260709 13:23:43.088176 25977 raft_consensus.cc:493] T 51a11db644124b63bdf5f457c967473d P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 233e7d29aac04505bf4e65f0a39a0c4c)
I20260709 13:23:43.088243 26024 raft_consensus.cc:493] T 9e5d70664720424d951f269d9d7f4346 P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9868dce521294d0b816bba7648f4ae5a)
I20260709 13:23:43.088852 25977 raft_consensus.cc:515] T 51a11db644124b63bdf5f457c967473d P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } }
I20260709 13:23:43.099838 26017 leader_election.cc:290] T e9e342ecda4b46619ce12452b2cf7154 P 43999d813e564dc688fad116090c95ca [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9868dce521294d0b816bba7648f4ae5a (127.14.177.132:33683), 233e7d29aac04505bf4e65f0a39a0c4c (127.14.177.130:42957)
I20260709 13:23:43.097846 26032 raft_consensus.cc:493] T 72b56266e8a54776b9113c7a786e1952 P 233e7d29aac04505bf4e65f0a39a0c4c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 43999d813e564dc688fad116090c95ca)
I20260709 13:23:43.102737 26032 raft_consensus.cc:515] T 72b56266e8a54776b9113c7a786e1952 P 233e7d29aac04505bf4e65f0a39a0c4c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } }
I20260709 13:23:43.089370 26024 raft_consensus.cc:515] T 9e5d70664720424d951f269d9d7f4346 P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } }
I20260709 13:23:43.122346 26036 raft_consensus.cc:493] T ad32097cd4dc48b9a69595f1651abd07 P 233e7d29aac04505bf4e65f0a39a0c4c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3e07d64466934da4bb3c02262b1c2e38)
I20260709 13:23:43.124038 26018 leader_election.cc:290] T 11949f56648c47e0a43c3140af99a304 P 43999d813e564dc688fad116090c95ca [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9868dce521294d0b816bba7648f4ae5a (127.14.177.132:33683), 233e7d29aac04505bf4e65f0a39a0c4c (127.14.177.130:42957)
I20260709 13:23:43.124832 26019 raft_consensus.cc:493] T e15820fa8a7d4e1fb6ad23a64ca22f90 P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3e07d64466934da4bb3c02262b1c2e38)
I20260709 13:23:43.122517 26035 raft_consensus.cc:493] T 0c4f6b7401db47d59a4f34ed0e8b5350 P 233e7d29aac04505bf4e65f0a39a0c4c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9868dce521294d0b816bba7648f4ae5a)
I20260709 13:23:43.134068 26032 leader_election.cc:290] T 72b56266e8a54776b9113c7a786e1952 P 233e7d29aac04505bf4e65f0a39a0c4c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 43999d813e564dc688fad116090c95ca (127.14.177.129:38155), 3e07d64466934da4bb3c02262b1c2e38 (127.14.177.131:41539)
I20260709 13:23:43.134372 26029 raft_consensus.cc:493] T ce9c60e3922c4219b6e4453fe0259b6a P 233e7d29aac04505bf4e65f0a39a0c4c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 43999d813e564dc688fad116090c95ca)
I20260709 13:23:43.135087 26029 raft_consensus.cc:515] T ce9c60e3922c4219b6e4453fe0259b6a P 233e7d29aac04505bf4e65f0a39a0c4c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } }
I20260709 13:23:43.135633 25977 leader_election.cc:290] T 51a11db644124b63bdf5f457c967473d P 43999d813e564dc688fad116090c95ca [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9868dce521294d0b816bba7648f4ae5a (127.14.177.132:33683), 233e7d29aac04505bf4e65f0a39a0c4c (127.14.177.130:42957)
I20260709 13:23:43.123363 26036 raft_consensus.cc:515] T ad32097cd4dc48b9a69595f1651abd07 P 233e7d29aac04505bf4e65f0a39a0c4c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } }
I20260709 13:23:43.115676 26021 raft_consensus.cc:493] T 5c7e7acbb09f4d49828fa652ca6abaac P 9868dce521294d0b816bba7648f4ae5a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 43999d813e564dc688fad116090c95ca)
I20260709 13:23:43.114936 26022 raft_consensus.cc:493] T c902bbc7ec0848ba82a04870eab1e7f5 P 9868dce521294d0b816bba7648f4ae5a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 43999d813e564dc688fad116090c95ca)
I20260709 13:23:43.137259 26035 raft_consensus.cc:515] T 0c4f6b7401db47d59a4f34ed0e8b5350 P 233e7d29aac04505bf4e65f0a39a0c4c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } }
I20260709 13:23:43.125880 26030 raft_consensus.cc:493] T ad32097cd4dc48b9a69595f1651abd07 P 9868dce521294d0b816bba7648f4ae5a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3e07d64466934da4bb3c02262b1c2e38)
I20260709 13:23:43.140233 26024 leader_election.cc:290] T 9e5d70664720424d951f269d9d7f4346 P 43999d813e564dc688fad116090c95ca [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9868dce521294d0b816bba7648f4ae5a (127.14.177.132:33683), 233e7d29aac04505bf4e65f0a39a0c4c (127.14.177.130:42957)
I20260709 13:23:43.140399 26029 leader_election.cc:290] T ce9c60e3922c4219b6e4453fe0259b6a P 233e7d29aac04505bf4e65f0a39a0c4c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9868dce521294d0b816bba7648f4ae5a (127.14.177.132:33683), 43999d813e564dc688fad116090c95ca (127.14.177.129:38155)
I20260709 13:23:43.141109 26022 raft_consensus.cc:515] T c902bbc7ec0848ba82a04870eab1e7f5 P 9868dce521294d0b816bba7648f4ae5a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } }
I20260709 13:23:43.125417 26019 raft_consensus.cc:515] T e15820fa8a7d4e1fb6ad23a64ca22f90 P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } }
I20260709 13:23:43.148702 26031 raft_consensus.cc:493] T 5c7e7acbb09f4d49828fa652ca6abaac P 3e07d64466934da4bb3c02262b1c2e38 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 43999d813e564dc688fad116090c95ca)
I20260709 13:23:43.136354 26028 raft_consensus.cc:493] T 0ed5edbe32044d1c83b0590dbb899801 P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3e07d64466934da4bb3c02262b1c2e38)
I20260709 13:23:43.147578 26030 raft_consensus.cc:515] T ad32097cd4dc48b9a69595f1651abd07 P 9868dce521294d0b816bba7648f4ae5a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } }
I20260709 13:23:43.160427 26022 leader_election.cc:290] T c902bbc7ec0848ba82a04870eab1e7f5 P 9868dce521294d0b816bba7648f4ae5a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3e07d64466934da4bb3c02262b1c2e38 (127.14.177.131:41539), 43999d813e564dc688fad116090c95ca (127.14.177.129:38155)
I20260709 13:23:43.149784 25977 raft_consensus.cc:493] T 03e9bfbd2f774476a4a377367796a2e8 P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3e07d64466934da4bb3c02262b1c2e38)
I20260709 13:23:43.160883 26030 leader_election.cc:290] T ad32097cd4dc48b9a69595f1651abd07 P 9868dce521294d0b816bba7648f4ae5a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 233e7d29aac04505bf4e65f0a39a0c4c (127.14.177.130:42957), 3e07d64466934da4bb3c02262b1c2e38 (127.14.177.131:41539)
I20260709 13:23:43.161302 26022 raft_consensus.cc:493] T 1550ed2d2a764c08adffae679599ba41 P 9868dce521294d0b816bba7648f4ae5a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 43999d813e564dc688fad116090c95ca)
I20260709 13:23:43.161538 26030 raft_consensus.cc:493] T a5df750b84e0487fa530161079deb6c2 P 9868dce521294d0b816bba7648f4ae5a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 43999d813e564dc688fad116090c95ca)
I20260709 13:23:43.144331 26021 raft_consensus.cc:515] T 5c7e7acbb09f4d49828fa652ca6abaac P 9868dce521294d0b816bba7648f4ae5a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } }
I20260709 13:23:43.162156 26030 raft_consensus.cc:515] T a5df750b84e0487fa530161079deb6c2 P 9868dce521294d0b816bba7648f4ae5a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } }
I20260709 13:23:43.155783 26028 raft_consensus.cc:515] T 0ed5edbe32044d1c83b0590dbb899801 P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } }
I20260709 13:23:43.162954 26025 raft_consensus.cc:493] T bbab48ba177049dcba6de106fd889cfb P 3e07d64466934da4bb3c02262b1c2e38 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 233e7d29aac04505bf4e65f0a39a0c4c)
I20260709 13:23:43.163509 26025 raft_consensus.cc:515] T bbab48ba177049dcba6de106fd889cfb P 3e07d64466934da4bb3c02262b1c2e38 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } }
I20260709 13:23:43.164238 26021 leader_election.cc:290] T 5c7e7acbb09f4d49828fa652ca6abaac P 9868dce521294d0b816bba7648f4ae5a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3e07d64466934da4bb3c02262b1c2e38 (127.14.177.131:41539), 43999d813e564dc688fad116090c95ca (127.14.177.129:38155)
I20260709 13:23:43.161306 25977 raft_consensus.cc:515] T 03e9bfbd2f774476a4a377367796a2e8 P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } }
I20260709 13:23:43.165833 26025 leader_election.cc:290] T bbab48ba177049dcba6de106fd889cfb P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 233e7d29aac04505bf4e65f0a39a0c4c (127.14.177.130:42957), 9868dce521294d0b816bba7648f4ae5a (127.14.177.132:33683)
I20260709 13:23:43.167428 26036 leader_election.cc:290] T ad32097cd4dc48b9a69595f1651abd07 P 233e7d29aac04505bf4e65f0a39a0c4c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3e07d64466934da4bb3c02262b1c2e38 (127.14.177.131:41539), 9868dce521294d0b816bba7648f4ae5a (127.14.177.132:33683)
I20260709 13:23:43.161976 26020 raft_consensus.cc:493] T 8ef061028f38450e83324ac78cdaa14c P 9868dce521294d0b816bba7648f4ae5a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 43999d813e564dc688fad116090c95ca)
I20260709 13:23:43.168222 26020 raft_consensus.cc:515] T 8ef061028f38450e83324ac78cdaa14c P 9868dce521294d0b816bba7648f4ae5a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } }
I20260709 13:23:43.161976 26022 raft_consensus.cc:515] T 1550ed2d2a764c08adffae679599ba41 P 9868dce521294d0b816bba7648f4ae5a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } }
I20260709 13:23:43.170410 26020 leader_election.cc:290] T 8ef061028f38450e83324ac78cdaa14c P 9868dce521294d0b816bba7648f4ae5a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 43999d813e564dc688fad116090c95ca (127.14.177.129:38155), 233e7d29aac04505bf4e65f0a39a0c4c (127.14.177.130:42957)
I20260709 13:23:43.172549 26035 leader_election.cc:290] T 0c4f6b7401db47d59a4f34ed0e8b5350 P 233e7d29aac04505bf4e65f0a39a0c4c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9868dce521294d0b816bba7648f4ae5a (127.14.177.132:33683), 43999d813e564dc688fad116090c95ca (127.14.177.129:38155)
I20260709 13:23:43.168753 25977 leader_election.cc:290] T 03e9bfbd2f774476a4a377367796a2e8 P 43999d813e564dc688fad116090c95ca [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 233e7d29aac04505bf4e65f0a39a0c4c (127.14.177.130:42957), 3e07d64466934da4bb3c02262b1c2e38 (127.14.177.131:41539)
I20260709 13:23:43.160450 26019 leader_election.cc:290] T e15820fa8a7d4e1fb6ad23a64ca22f90 P 43999d813e564dc688fad116090c95ca [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 233e7d29aac04505bf4e65f0a39a0c4c (127.14.177.130:42957), 3e07d64466934da4bb3c02262b1c2e38 (127.14.177.131:41539)
I20260709 13:23:43.156993 26031 raft_consensus.cc:515] T 5c7e7acbb09f4d49828fa652ca6abaac P 3e07d64466934da4bb3c02262b1c2e38 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } }
I20260709 13:23:43.172698 26022 leader_election.cc:290] T 1550ed2d2a764c08adffae679599ba41 P 9868dce521294d0b816bba7648f4ae5a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3e07d64466934da4bb3c02262b1c2e38 (127.14.177.131:41539), 43999d813e564dc688fad116090c95ca (127.14.177.129:38155)
I20260709 13:23:43.164644 26028 leader_election.cc:290] T 0ed5edbe32044d1c83b0590dbb899801 P 43999d813e564dc688fad116090c95ca [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3e07d64466934da4bb3c02262b1c2e38 (127.14.177.131:41539), 9868dce521294d0b816bba7648f4ae5a (127.14.177.132:33683)
I20260709 13:23:43.190701 26031 leader_election.cc:290] T 5c7e7acbb09f4d49828fa652ca6abaac P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9868dce521294d0b816bba7648f4ae5a (127.14.177.132:33683), 43999d813e564dc688fad116090c95ca (127.14.177.129:38155)
I20260709 13:23:43.182775 26025 raft_consensus.cc:493] T ba8a000f5abe4158965449995af84b18 P 3e07d64466934da4bb3c02262b1c2e38 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 233e7d29aac04505bf4e65f0a39a0c4c)
I20260709 13:23:43.203583 25381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f15ec62698c4a8cbd84f2b3d516d2ea" candidate_uuid: "9868dce521294d0b816bba7648f4ae5a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3e07d64466934da4bb3c02262b1c2e38" is_pre_election: true
I20260709 13:23:43.214555 25955 raft_consensus.cc:493] T b7a891bfc89e4e259f53da6cb16f4989 P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3e07d64466934da4bb3c02262b1c2e38)
I20260709 13:23:43.215446 25955 raft_consensus.cc:515] T b7a891bfc89e4e259f53da6cb16f4989 P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } }
I20260709 13:23:43.217717 26026 raft_consensus.cc:493] T 5c4ab1978a47481c8576d466bacf49d2 P 3e07d64466934da4bb3c02262b1c2e38 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 233e7d29aac04505bf4e65f0a39a0c4c)
I20260709 13:23:43.218339 26026 raft_consensus.cc:515] T 5c4ab1978a47481c8576d466bacf49d2 P 3e07d64466934da4bb3c02262b1c2e38 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } }
I20260709 13:23:43.221890 25385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad32097cd4dc48b9a69595f1651abd07" candidate_uuid: "9868dce521294d0b816bba7648f4ae5a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3e07d64466934da4bb3c02262b1c2e38" is_pre_election: true
I20260709 13:23:43.229172 25379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c902bbc7ec0848ba82a04870eab1e7f5" candidate_uuid: "9868dce521294d0b816bba7648f4ae5a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3e07d64466934da4bb3c02262b1c2e38" is_pre_election: true
I20260709 13:23:43.227742 26029 raft_consensus.cc:493] T 467a6b42ca6548269c12dddde696f7af P 233e7d29aac04505bf4e65f0a39a0c4c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9868dce521294d0b816bba7648f4ae5a)
I20260709 13:23:43.230044 25379 raft_consensus.cc:2468] T c902bbc7ec0848ba82a04870eab1e7f5 P 3e07d64466934da4bb3c02262b1c2e38 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9868dce521294d0b816bba7648f4ae5a in term 1.
I20260709 13:23:43.230403 26026 leader_election.cc:290] T 5c4ab1978a47481c8576d466bacf49d2 P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 233e7d29aac04505bf4e65f0a39a0c4c (127.14.177.130:42957), 9868dce521294d0b816bba7648f4ae5a (127.14.177.132:33683)
I20260709 13:23:43.232095 26026 raft_consensus.cc:493] T b9d550d881b04735bbf89751952c09ec P 3e07d64466934da4bb3c02262b1c2e38 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 43999d813e564dc688fad116090c95ca)
I20260709 13:23:43.232656 26026 raft_consensus.cc:515] T b9d550d881b04735bbf89751952c09ec P 3e07d64466934da4bb3c02262b1c2e38 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } }
I20260709 13:23:43.226158 26035 raft_consensus.cc:493] T b7ff8dde9f5d4c49b9203d902f6b94c4 P 233e7d29aac04505bf4e65f0a39a0c4c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 43999d813e564dc688fad116090c95ca)
I20260709 13:23:43.234225 26028 raft_consensus.cc:493] T d053b9ae38de4173a0aded4da494bd98 P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9868dce521294d0b816bba7648f4ae5a)
I20260709 13:23:43.234754 26035 raft_consensus.cc:515] T b7ff8dde9f5d4c49b9203d902f6b94c4 P 233e7d29aac04505bf4e65f0a39a0c4c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } }
I20260709 13:23:43.237200 26035 leader_election.cc:290] T b7ff8dde9f5d4c49b9203d902f6b94c4 P 233e7d29aac04505bf4e65f0a39a0c4c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 43999d813e564dc688fad116090c95ca (127.14.177.129:38155), 3e07d64466934da4bb3c02262b1c2e38 (127.14.177.131:41539)
I20260709 13:23:43.234839 26028 raft_consensus.cc:515] T d053b9ae38de4173a0aded4da494bd98 P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } }
I20260709 13:23:43.247586 26037 raft_consensus.cc:493] T 0cb89d70b4ce4aa8992fb68480bec8a2 P 233e7d29aac04505bf4e65f0a39a0c4c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3e07d64466934da4bb3c02262b1c2e38)
I20260709 13:23:43.248413 26035 raft_consensus.cc:493] T 55e8ed3aee404ce698d0626a87bd03aa P 233e7d29aac04505bf4e65f0a39a0c4c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9868dce521294d0b816bba7648f4ae5a)
I20260709 13:23:43.249040 26035 raft_consensus.cc:515] T 55e8ed3aee404ce698d0626a87bd03aa P 233e7d29aac04505bf4e65f0a39a0c4c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } }
I20260709 13:23:43.248227 26037 raft_consensus.cc:515] T 0cb89d70b4ce4aa8992fb68480bec8a2 P 233e7d29aac04505bf4e65f0a39a0c4c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } }
I20260709 13:23:43.219530 26025 raft_consensus.cc:515] T ba8a000f5abe4158965449995af84b18 P 3e07d64466934da4bb3c02262b1c2e38 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } }
I20260709 13:23:43.280055 25385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad32097cd4dc48b9a69595f1651abd07" candidate_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3e07d64466934da4bb3c02262b1c2e38" is_pre_election: true
I20260709 13:23:43.284072 25459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c4f6b7401db47d59a4f34ed0e8b5350" candidate_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9868dce521294d0b816bba7648f4ae5a" is_pre_election: true
I20260709 13:23:43.284260 25451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d053b9ae38de4173a0aded4da494bd98" candidate_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9868dce521294d0b816bba7648f4ae5a" is_pre_election: true
I20260709 13:23:43.285537 25379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1550ed2d2a764c08adffae679599ba41" candidate_uuid: "9868dce521294d0b816bba7648f4ae5a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3e07d64466934da4bb3c02262b1c2e38" is_pre_election: true
I20260709 13:23:43.286005 26030 leader_election.cc:290] T a5df750b84e0487fa530161079deb6c2 P 9868dce521294d0b816bba7648f4ae5a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 43999d813e564dc688fad116090c95ca (127.14.177.129:38155), 233e7d29aac04505bf4e65f0a39a0c4c (127.14.177.130:42957)
I20260709 13:23:43.286607 26030 raft_consensus.cc:493] T 9ffa42c3b2ff48bc87310e9ab5d218ce P 9868dce521294d0b816bba7648f4ae5a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3e07d64466934da4bb3c02262b1c2e38)
I20260709 13:23:43.286947 25383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72b56266e8a54776b9113c7a786e1952" candidate_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3e07d64466934da4bb3c02262b1c2e38" is_pre_election: true
I20260709 13:23:43.287195 26030 raft_consensus.cc:515] T 9ffa42c3b2ff48bc87310e9ab5d218ce P 9868dce521294d0b816bba7648f4ae5a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } }
I20260709 13:23:43.287856 25977 raft_consensus.cc:493] T 0cb89d70b4ce4aa8992fb68480bec8a2 P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3e07d64466934da4bb3c02262b1c2e38)
I20260709 13:23:43.255931 26020 raft_consensus.cc:493] T 579da2d6aabb4ae9a2b1802e1358068b P 9868dce521294d0b816bba7648f4ae5a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 43999d813e564dc688fad116090c95ca)
I20260709 13:23:43.230465 26029 raft_consensus.cc:515] T 467a6b42ca6548269c12dddde696f7af P 233e7d29aac04505bf4e65f0a39a0c4c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } }
I20260709 13:23:43.253332 26027 raft_consensus.cc:493] T b2f14aed05e24a98ad83535029b795dc P 233e7d29aac04505bf4e65f0a39a0c4c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3e07d64466934da4bb3c02262b1c2e38)
I20260709 13:23:43.264967 26019 raft_consensus.cc:493] T b2f14aed05e24a98ad83535029b795dc P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3e07d64466934da4bb3c02262b1c2e38)
I20260709 13:23:43.264840 25381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c7e7acbb09f4d49828fa652ca6abaac" candidate_uuid: "9868dce521294d0b816bba7648f4ae5a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3e07d64466934da4bb3c02262b1c2e38" is_pre_election: true
I20260709 13:23:43.255473 26022 raft_consensus.cc:493] T 3fe5a944a342411ea923eb39071ac770 P 9868dce521294d0b816bba7648f4ae5a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 233e7d29aac04505bf4e65f0a39a0c4c)
I20260709 13:23:43.261998 26040 raft_consensus.cc:493] T 467a6b42ca6548269c12dddde696f7af P 3e07d64466934da4bb3c02262b1c2e38 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9868dce521294d0b816bba7648f4ae5a)
I20260709 13:23:43.274441 26045 raft_consensus.cc:493] T 11949f56648c47e0a43c3140af99a304 P 233e7d29aac04505bf4e65f0a39a0c4c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9868dce521294d0b816bba7648f4ae5a)
I20260709 13:23:43.277686 26031 raft_consensus.cc:493] T e311eb90bad44c80a131b72edcd22b06 P 3e07d64466934da4bb3c02262b1c2e38 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 233e7d29aac04505bf4e65f0a39a0c4c)
W20260709 13:23:43.332659 25274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.132:33683, user_credentials={real_user=slave}} blocked reactor thread for 81032.9us
I20260709 13:23:43.331210 26017 raft_consensus.cc:493] T 28c0b6fc9bd64d60828d705c6a9ab1e4 P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 233e7d29aac04505bf4e65f0a39a0c4c)
I20260709 13:23:43.332067 26045 raft_consensus.cc:515] T 11949f56648c47e0a43c3140af99a304 P 233e7d29aac04505bf4e65f0a39a0c4c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } }
I20260709 13:23:43.333475 26017 raft_consensus.cc:515] T 28c0b6fc9bd64d60828d705c6a9ab1e4 P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } }
I20260709 13:23:43.334606 25456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad32097cd4dc48b9a69595f1651abd07" candidate_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9868dce521294d0b816bba7648f4ae5a" is_pre_election: true
I20260709 13:23:43.335393 26017 leader_election.cc:290] T 28c0b6fc9bd64d60828d705c6a9ab1e4 P 43999d813e564dc688fad116090c95ca [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 233e7d29aac04505bf4e65f0a39a0c4c (127.14.177.130:42957), 3e07d64466934da4bb3c02262b1c2e38 (127.14.177.131:41539)
I20260709 13:23:43.331265 26024 raft_consensus.cc:493] T 6242888a4d4146dcacd87d0fd9a832f4 P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9868dce521294d0b816bba7648f4ae5a)
I20260709 13:23:43.330771 25381 raft_consensus.cc:2468] T 5c7e7acbb09f4d49828fa652ca6abaac P 3e07d64466934da4bb3c02262b1c2e38 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9868dce521294d0b816bba7648f4ae5a in term 1.
I20260709 13:23:43.332238 26031 raft_consensus.cc:515] T e311eb90bad44c80a131b72edcd22b06 P 3e07d64466934da4bb3c02262b1c2e38 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } }
I20260709 13:23:43.336838 26024 raft_consensus.cc:515] T 6242888a4d4146dcacd87d0fd9a832f4 P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } }
I20260709 13:23:43.335407 25456 raft_consensus.cc:2468] T ad32097cd4dc48b9a69595f1651abd07 P 9868dce521294d0b816bba7648f4ae5a [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 233e7d29aac04505bf4e65f0a39a0c4c in term 1.
I20260709 13:23:43.330197 26022 raft_consensus.cc:515] T 3fe5a944a342411ea923eb39071ac770 P 9868dce521294d0b816bba7648f4ae5a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } }
I20260709 13:23:43.338945 26031 leader_election.cc:290] T e311eb90bad44c80a131b72edcd22b06 P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 233e7d29aac04505bf4e65f0a39a0c4c (127.14.177.130:42957), 9868dce521294d0b816bba7648f4ae5a (127.14.177.132:33683)
I20260709 13:23:43.339098 26024 leader_election.cc:290] T 6242888a4d4146dcacd87d0fd9a832f4 P 43999d813e564dc688fad116090c95ca [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9868dce521294d0b816bba7648f4ae5a (127.14.177.132:33683), 233e7d29aac04505bf4e65f0a39a0c4c (127.14.177.130:42957)
I20260709 13:23:43.340879 26022 leader_election.cc:290] T 3fe5a944a342411ea923eb39071ac770 P 9868dce521294d0b816bba7648f4ae5a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 233e7d29aac04505bf4e65f0a39a0c4c (127.14.177.130:42957), 3e07d64466934da4bb3c02262b1c2e38 (127.14.177.131:41539)
I20260709 13:23:43.335680 26045 leader_election.cc:290] T 11949f56648c47e0a43c3140af99a304 P 233e7d29aac04505bf4e65f0a39a0c4c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9868dce521294d0b816bba7648f4ae5a (127.14.177.132:33683), 43999d813e564dc688fad116090c95ca (127.14.177.129:38155)
I20260709 13:23:43.341553 26022 raft_consensus.cc:493] T fa0da2c845dc4059968332f4db2e0aa6 P 9868dce521294d0b816bba7648f4ae5a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 43999d813e564dc688fad116090c95ca)
I20260709 13:23:43.342482 26022 raft_consensus.cc:515] T fa0da2c845dc4059968332f4db2e0aa6 P 9868dce521294d0b816bba7648f4ae5a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } }
I20260709 13:23:43.343291 25453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce9c60e3922c4219b6e4453fe0259b6a" candidate_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9868dce521294d0b816bba7648f4ae5a" is_pre_election: true
I20260709 13:23:43.344604 26022 leader_election.cc:290] T fa0da2c845dc4059968332f4db2e0aa6 P 9868dce521294d0b816bba7648f4ae5a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 43999d813e564dc688fad116090c95ca (127.14.177.129:38155), 233e7d29aac04505bf4e65f0a39a0c4c (127.14.177.130:42957)
I20260709 13:23:43.344324 25453 raft_consensus.cc:2468] T ce9c60e3922c4219b6e4453fe0259b6a P 9868dce521294d0b816bba7648f4ae5a [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 233e7d29aac04505bf4e65f0a39a0c4c in term 1.
I20260709 13:23:43.331629 26019 raft_consensus.cc:515] T b2f14aed05e24a98ad83535029b795dc P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } }
I20260709 13:23:43.341607 25231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f15ec62698c4a8cbd84f2b3d516d2ea" candidate_uuid: "9868dce521294d0b816bba7648f4ae5a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "43999d813e564dc688fad116090c95ca" is_pre_election: true
I20260709 13:23:43.342461 26050 raft_consensus.cc:493] T 821a5866429f4173820fb64336abee02 P 9868dce521294d0b816bba7648f4ae5a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 43999d813e564dc688fad116090c95ca)
I20260709 13:23:43.347301 26019 leader_election.cc:290] T b2f14aed05e24a98ad83535029b795dc P 43999d813e564dc688fad116090c95ca [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 233e7d29aac04505bf4e65f0a39a0c4c (127.14.177.130:42957), 3e07d64466934da4bb3c02262b1c2e38 (127.14.177.131:41539)
I20260709 13:23:43.347518 25231 raft_consensus.cc:2468] T 1f15ec62698c4a8cbd84f2b3d516d2ea P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9868dce521294d0b816bba7648f4ae5a in term 1.
I20260709 13:23:43.347582 26050 raft_consensus.cc:515] T 821a5866429f4173820fb64336abee02 P 9868dce521294d0b816bba7648f4ae5a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } }
I20260709 13:23:43.352169 26053 raft_consensus.cc:493] T 5c4ab1978a47481c8576d466bacf49d2 P 9868dce521294d0b816bba7648f4ae5a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 233e7d29aac04505bf4e65f0a39a0c4c)
I20260709 13:23:43.353086 26053 raft_consensus.cc:515] T 5c4ab1978a47481c8576d466bacf49d2 P 9868dce521294d0b816bba7648f4ae5a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } }
I20260709 13:23:43.323534 26037 leader_election.cc:290] T 0cb89d70b4ce4aa8992fb68480bec8a2 P 233e7d29aac04505bf4e65f0a39a0c4c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 43999d813e564dc688fad116090c95ca (127.14.177.129:38155), 3e07d64466934da4bb3c02262b1c2e38 (127.14.177.131:41539)
I20260709 13:23:43.317328 26044 raft_consensus.cc:493] T ba8a000f5abe4158965449995af84b18 P 9868dce521294d0b816bba7648f4ae5a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 233e7d29aac04505bf4e65f0a39a0c4c)
I20260709 13:23:43.326406 26020 raft_consensus.cc:515] T 579da2d6aabb4ae9a2b1802e1358068b P 9868dce521294d0b816bba7648f4ae5a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } }
I20260709 13:23:43.363243 26028 leader_election.cc:290] T d053b9ae38de4173a0aded4da494bd98 P 43999d813e564dc688fad116090c95ca [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9868dce521294d0b816bba7648f4ae5a (127.14.177.132:33683), 233e7d29aac04505bf4e65f0a39a0c4c (127.14.177.130:42957)
I20260709 13:23:43.363847 15046 tablet_server.cc:179] TabletServer@127.14.177.129:0 shutting down...
I20260709 13:23:43.364816 26020 leader_election.cc:290] T 579da2d6aabb4ae9a2b1802e1358068b P 9868dce521294d0b816bba7648f4ae5a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 43999d813e564dc688fad116090c95ca (127.14.177.129:38155), 233e7d29aac04505bf4e65f0a39a0c4c (127.14.177.130:42957)
I20260709 13:23:43.364919 26031 raft_consensus.cc:493] T b155b866f10547fd8239579a81ecf5f9 P 3e07d64466934da4bb3c02262b1c2e38 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 43999d813e564dc688fad116090c95ca)
I20260709 13:23:43.365644 26031 raft_consensus.cc:515] T b155b866f10547fd8239579a81ecf5f9 P 3e07d64466934da4bb3c02262b1c2e38 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } }
I20260709 13:23:43.384188 25424 leader_election.cc:304] T 1f15ec62698c4a8cbd84f2b3d516d2ea P 9868dce521294d0b816bba7648f4ae5a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 43999d813e564dc688fad116090c95ca, 9868dce521294d0b816bba7648f4ae5a; no voters: 
I20260709 13:23:43.325168 25977 raft_consensus.cc:515] T 0cb89d70b4ce4aa8992fb68480bec8a2 P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } }
I20260709 13:23:43.329713 26027 raft_consensus.cc:515] T b2f14aed05e24a98ad83535029b795dc P 233e7d29aac04505bf4e65f0a39a0c4c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } }
I20260709 13:23:43.392711 25385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "280d65245f4f4358b54bef93e18aca92" candidate_uuid: "43999d813e564dc688fad116090c95ca" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3e07d64466934da4bb3c02262b1c2e38" is_pre_election: true
I20260709 13:23:43.366698 26049 raft_consensus.cc:493] T 79d4a12ef58140568c60a24ed4c12f83 P 233e7d29aac04505bf4e65f0a39a0c4c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3e07d64466934da4bb3c02262b1c2e38)
I20260709 13:23:43.383983 25383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7ff8dde9f5d4c49b9203d902f6b94c4" candidate_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3e07d64466934da4bb3c02262b1c2e38" is_pre_election: true
I20260709 13:23:43.392079 25379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f15ec62698c4a8cbd84f2b3d516d2ea" candidate_uuid: "43999d813e564dc688fad116090c95ca" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3e07d64466934da4bb3c02262b1c2e38" is_pre_election: true
I20260709 13:23:43.397985 26022 raft_consensus.cc:493] T 4d837697cd0f402cacf9db89a5d92652 P 9868dce521294d0b816bba7648f4ae5a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 43999d813e564dc688fad116090c95ca)
I20260709 13:23:43.389422 26051 raft_consensus.cc:493] T a820954e07a94582ac0613342fcffde5 P 233e7d29aac04505bf4e65f0a39a0c4c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9868dce521294d0b816bba7648f4ae5a)
I20260709 13:23:43.393200 26026 leader_election.cc:290] T b9d550d881b04735bbf89751952c09ec P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9868dce521294d0b816bba7648f4ae5a (127.14.177.132:33683), 43999d813e564dc688fad116090c95ca (127.14.177.129:38155)
I20260709 13:23:43.400715 26037 raft_consensus.cc:493] T 03e9bfbd2f774476a4a377367796a2e8 P 233e7d29aac04505bf4e65f0a39a0c4c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3e07d64466934da4bb3c02262b1c2e38)
I20260709 13:23:43.397126 26019 raft_consensus.cc:493] T 91ac744179de48e5bb924b6dfd9f9c9b P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3e07d64466934da4bb3c02262b1c2e38)
I20260709 13:23:43.400967 26021 raft_consensus.cc:2804] T 1f15ec62698c4a8cbd84f2b3d516d2ea P 9868dce521294d0b816bba7648f4ae5a [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:23:43.397349 26028 raft_consensus.cc:493] T beacc815ef954fb692e90544bfa93f05 P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3e07d64466934da4bb3c02262b1c2e38)
I20260709 13:23:43.400552 25380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d29b499fa4c41f1b974782cd0c9c1c2" candidate_uuid: "43999d813e564dc688fad116090c95ca" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3e07d64466934da4bb3c02262b1c2e38" is_pre_election: true
I20260709 13:23:43.329577 26040 raft_consensus.cc:515] T 467a6b42ca6548269c12dddde696f7af P 3e07d64466934da4bb3c02262b1c2e38 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } }
W20260709 13:23:43.436168 25424 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 1 similar messages]
I20260709 13:23:43.434654 26046 raft_consensus.cc:493] T 434a18a5109e4ea78349ded3968b2709 P 3e07d64466934da4bb3c02262b1c2e38 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 233e7d29aac04505bf4e65f0a39a0c4c)
I20260709 13:23:43.436888 26029 leader_election.cc:290] T 467a6b42ca6548269c12dddde696f7af P 233e7d29aac04505bf4e65f0a39a0c4c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3e07d64466934da4bb3c02262b1c2e38 (127.14.177.131:41539), 9868dce521294d0b816bba7648f4ae5a (127.14.177.132:33683)
I20260709 13:23:43.438158 25384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce98a3f3b3eb4c63bcaa355ec7cd64a4" candidate_uuid: "43999d813e564dc688fad116090c95ca" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3e07d64466934da4bb3c02262b1c2e38" is_pre_election: true
I20260709 13:23:43.372963 26044 raft_consensus.cc:515] T ba8a000f5abe4158965449995af84b18 P 9868dce521294d0b816bba7648f4ae5a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } }
I20260709 13:23:43.436525 26047 raft_consensus.cc:493] T 13bb3a23315a41c7a40e10b954b15c8e P 233e7d29aac04505bf4e65f0a39a0c4c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 43999d813e564dc688fad116090c95ca)
I20260709 13:23:43.440434 26024 raft_consensus.cc:493] T 79d4a12ef58140568c60a24ed4c12f83 P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3e07d64466934da4bb3c02262b1c2e38)
I20260709 13:23:43.447293 26036 raft_consensus.cc:493] T 441d780c3945463f92f31170f31e37a8 P 233e7d29aac04505bf4e65f0a39a0c4c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 43999d813e564dc688fad116090c95ca)
I20260709 13:23:43.441548 26043 raft_consensus.cc:493] T beacc815ef954fb692e90544bfa93f05 P 233e7d29aac04505bf4e65f0a39a0c4c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3e07d64466934da4bb3c02262b1c2e38)
I20260709 13:23:43.444384 26021 raft_consensus.cc:493] T 1f15ec62698c4a8cbd84f2b3d516d2ea P 9868dce521294d0b816bba7648f4ae5a [term 1 FOLLOWER]: Starting leader election (detected failure of leader 3e07d64466934da4bb3c02262b1c2e38)
I20260709 13:23:43.452881 26026 raft_consensus.cc:493] T 434a54fcee7d4aeeab42f739855c0e8a P 3e07d64466934da4bb3c02262b1c2e38 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 43999d813e564dc688fad116090c95ca)
I20260709 13:23:43.489988 25384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15820fa8a7d4e1fb6ad23a64ca22f90" candidate_uuid: "43999d813e564dc688fad116090c95ca" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3e07d64466934da4bb3c02262b1c2e38" is_pre_election: true
I20260709 13:23:43.491406 26027 leader_election.cc:290] T b2f14aed05e24a98ad83535029b795dc P 233e7d29aac04505bf4e65f0a39a0c4c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3e07d64466934da4bb3c02262b1c2e38 (127.14.177.131:41539), 43999d813e564dc688fad116090c95ca (127.14.177.129:38155)
I20260709 13:23:43.445802 26051 raft_consensus.cc:515] T a820954e07a94582ac0613342fcffde5 P 233e7d29aac04505bf4e65f0a39a0c4c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } }
I20260709 13:23:43.493088 25955 leader_election.cc:290] T b7a891bfc89e4e259f53da6cb16f4989 P 43999d813e564dc688fad116090c95ca [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3e07d64466934da4bb3c02262b1c2e38 (127.14.177.131:41539), 9868dce521294d0b816bba7648f4ae5a (127.14.177.132:33683)
I20260709 13:23:43.494457 26051 leader_election.cc:290] T a820954e07a94582ac0613342fcffde5 P 233e7d29aac04505bf4e65f0a39a0c4c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3e07d64466934da4bb3c02262b1c2e38 (127.14.177.131:41539), 9868dce521294d0b816bba7648f4ae5a (127.14.177.132:33683)
I20260709 13:23:43.466115 26030 leader_election.cc:290] T 9ffa42c3b2ff48bc87310e9ab5d218ce P 9868dce521294d0b816bba7648f4ae5a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3e07d64466934da4bb3c02262b1c2e38 (127.14.177.131:41539), 43999d813e564dc688fad116090c95ca (127.14.177.129:38155)
I20260709 13:23:43.444751 26019 raft_consensus.cc:515] T 91ac744179de48e5bb924b6dfd9f9c9b P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } }
I20260709 13:23:43.460431 26020 raft_consensus.cc:493] T 0e71b6f5c69b4b35a8ac51c22c7d5f0d P 9868dce521294d0b816bba7648f4ae5a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 43999d813e564dc688fad116090c95ca)
I20260709 13:23:43.460888 26042 raft_consensus.cc:493] T 72b56266e8a54776b9113c7a786e1952 P 3e07d64466934da4bb3c02262b1c2e38 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 43999d813e564dc688fad116090c95ca)
I20260709 13:23:43.441798 26022 raft_consensus.cc:515] T 4d837697cd0f402cacf9db89a5d92652 P 9868dce521294d0b816bba7648f4ae5a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } }
I20260709 13:23:43.467896 26029 raft_consensus.cc:493] T 87085ac2575e4b6a86dd1fc21e6f718b P 233e7d29aac04505bf4e65f0a39a0c4c [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3e07d64466934da4bb3c02262b1c2e38)
I20260709 13:23:43.445606 26037 raft_consensus.cc:515] T 03e9bfbd2f774476a4a377367796a2e8 P 233e7d29aac04505bf4e65f0a39a0c4c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } }
W20260709 13:23:43.511152 25274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.132:33683, user_credentials={real_user=slave}} blocked reactor thread for 177747us
W20260709 13:23:43.511723 25424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.129:38155, user_credentials={real_user=slave}} blocked reactor thread for 77093.7us
I20260709 13:23:43.446903 26028 raft_consensus.cc:515] T beacc815ef954fb692e90544bfa93f05 P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } }
I20260709 13:23:43.473321 26053 leader_election.cc:290] T 5c4ab1978a47481c8576d466bacf49d2 P 9868dce521294d0b816bba7648f4ae5a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 233e7d29aac04505bf4e65f0a39a0c4c (127.14.177.130:42957), 3e07d64466934da4bb3c02262b1c2e38 (127.14.177.131:41539)
I20260709 13:23:43.482796 26021 raft_consensus.cc:3060] T 1f15ec62698c4a8cbd84f2b3d516d2ea P 9868dce521294d0b816bba7648f4ae5a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:23:43.439625 26049 raft_consensus.cc:515] T 79d4a12ef58140568c60a24ed4c12f83 P 233e7d29aac04505bf4e65f0a39a0c4c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } }
I20260709 13:23:43.495522 26030 raft_consensus.cc:493] T 91ac744179de48e5bb924b6dfd9f9c9b P 9868dce521294d0b816bba7648f4ae5a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3e07d64466934da4bb3c02262b1c2e38)
I20260709 13:23:43.497555 26035 leader_election.cc:290] T 55e8ed3aee404ce698d0626a87bd03aa P 233e7d29aac04505bf4e65f0a39a0c4c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3e07d64466934da4bb3c02262b1c2e38 (127.14.177.131:41539), 9868dce521294d0b816bba7648f4ae5a (127.14.177.132:33683)
I20260709 13:23:43.514457 26020 raft_consensus.cc:515] T 0e71b6f5c69b4b35a8ac51c22c7d5f0d P 9868dce521294d0b816bba7648f4ae5a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } }
I20260709 13:23:43.531682 26067 raft_consensus.cc:493] T 89a26dd3ac414f6081b98fa32c2f346f P 233e7d29aac04505bf4e65f0a39a0c4c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9868dce521294d0b816bba7648f4ae5a)
I20260709 13:23:43.527769 26030 raft_consensus.cc:515] T 91ac744179de48e5bb924b6dfd9f9c9b P 9868dce521294d0b816bba7648f4ae5a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } }
I20260709 13:23:43.532317 26067 raft_consensus.cc:515] T 89a26dd3ac414f6081b98fa32c2f346f P 233e7d29aac04505bf4e65f0a39a0c4c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } }
I20260709 13:23:43.534613 26067 leader_election.cc:290] T 89a26dd3ac414f6081b98fa32c2f346f P 233e7d29aac04505bf4e65f0a39a0c4c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3e07d64466934da4bb3c02262b1c2e38 (127.14.177.131:41539), 9868dce521294d0b816bba7648f4ae5a (127.14.177.132:33683)
I20260709 13:23:43.508210 26026 raft_consensus.cc:515] T 434a54fcee7d4aeeab42f739855c0e8a P 3e07d64466934da4bb3c02262b1c2e38 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } }
I20260709 13:23:43.512948 26025 leader_election.cc:290] T ba8a000f5abe4158965449995af84b18 P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 233e7d29aac04505bf4e65f0a39a0c4c (127.14.177.130:42957), 9868dce521294d0b816bba7648f4ae5a (127.14.177.132:33683)
I20260709 13:23:43.488014 26047 raft_consensus.cc:515] T 13bb3a23315a41c7a40e10b954b15c8e P 233e7d29aac04505bf4e65f0a39a0c4c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } }
I20260709 13:23:43.516887 26029 raft_consensus.cc:515] T 87085ac2575e4b6a86dd1fc21e6f718b P 233e7d29aac04505bf4e65f0a39a0c4c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } }
I20260709 13:23:43.479055 26024 raft_consensus.cc:515] T 79d4a12ef58140568c60a24ed4c12f83 P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } }
I20260709 13:23:43.488669 26043 raft_consensus.cc:515] T beacc815ef954fb692e90544bfa93f05 P 233e7d29aac04505bf4e65f0a39a0c4c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } }
I20260709 13:23:43.482057 26036 raft_consensus.cc:515] T 441d780c3945463f92f31170f31e37a8 P 233e7d29aac04505bf4e65f0a39a0c4c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } }
I20260709 13:23:43.519135 26059 raft_consensus.cc:493] T e15820fa8a7d4e1fb6ad23a64ca22f90 P 233e7d29aac04505bf4e65f0a39a0c4c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3e07d64466934da4bb3c02262b1c2e38)
I20260709 13:23:43.466509 26046 raft_consensus.cc:515] T 434a18a5109e4ea78349ded3968b2709 P 3e07d64466934da4bb3c02262b1c2e38 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } }
I20260709 13:23:43.523797 26027 raft_consensus.cc:493] T 9e5d70664720424d951f269d9d7f4346 P 233e7d29aac04505bf4e65f0a39a0c4c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9868dce521294d0b816bba7648f4ae5a)
I20260709 13:23:43.525017 26052 raft_consensus.cc:493] T fa0da2c845dc4059968332f4db2e0aa6 P 233e7d29aac04505bf4e65f0a39a0c4c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 43999d813e564dc688fad116090c95ca)
I20260709 13:23:43.595463 26020 leader_election.cc:290] T 0e71b6f5c69b4b35a8ac51c22c7d5f0d P 9868dce521294d0b816bba7648f4ae5a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3e07d64466934da4bb3c02262b1c2e38 (127.14.177.131:41539), 43999d813e564dc688fad116090c95ca (127.14.177.129:38155)
I20260709 13:23:43.613857 26052 raft_consensus.cc:515] T fa0da2c845dc4059968332f4db2e0aa6 P 233e7d29aac04505bf4e65f0a39a0c4c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } }
I20260709 13:23:43.614624 26058 raft_consensus.cc:493] T cfad575709c943cba40ccab278350770 P 3e07d64466934da4bb3c02262b1c2e38 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 9868dce521294d0b816bba7648f4ae5a)
I20260709 13:23:43.615262 26055 raft_consensus.cc:493] T d0a0e4d32ce943fbb773bb840863647b P 9868dce521294d0b816bba7648f4ae5a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 43999d813e564dc688fad116090c95ca)
I20260709 13:23:43.615901 26055 raft_consensus.cc:515] T d0a0e4d32ce943fbb773bb840863647b P 9868dce521294d0b816bba7648f4ae5a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } }
I20260709 13:23:43.616905 26052 leader_election.cc:290] T fa0da2c845dc4059968332f4db2e0aa6 P 233e7d29aac04505bf4e65f0a39a0c4c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9868dce521294d0b816bba7648f4ae5a (127.14.177.132:33683), 43999d813e564dc688fad116090c95ca (127.14.177.129:38155)
I20260709 13:23:43.618369 26055 leader_election.cc:290] T d0a0e4d32ce943fbb773bb840863647b P 9868dce521294d0b816bba7648f4ae5a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3e07d64466934da4bb3c02262b1c2e38 (127.14.177.131:41539), 43999d813e564dc688fad116090c95ca (127.14.177.129:38155)
I20260709 13:23:43.615298 26058 raft_consensus.cc:515] T cfad575709c943cba40ccab278350770 P 3e07d64466934da4bb3c02262b1c2e38 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } }
I20260709 13:23:43.619320 25382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03e9bfbd2f774476a4a377367796a2e8" candidate_uuid: "43999d813e564dc688fad116090c95ca" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3e07d64466934da4bb3c02262b1c2e38" is_pre_election: true
I20260709 13:23:43.621847 26040 leader_election.cc:290] T 467a6b42ca6548269c12dddde696f7af P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 233e7d29aac04505bf4e65f0a39a0c4c (127.14.177.130:42957), 9868dce521294d0b816bba7648f4ae5a (127.14.177.132:33683)
I20260709 13:23:43.623929 25376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7a891bfc89e4e259f53da6cb16f4989" candidate_uuid: "43999d813e564dc688fad116090c95ca" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3e07d64466934da4bb3c02262b1c2e38" is_pre_election: true
I20260709 13:23:43.610889 26053 raft_consensus.cc:493] T 441d780c3945463f92f31170f31e37a8 P 9868dce521294d0b816bba7648f4ae5a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 43999d813e564dc688fad116090c95ca)
I20260709 13:23:43.540077 26056 raft_consensus.cc:493] T 3a3317f4c3514df1b2bb8e44a86be92f P 9868dce521294d0b816bba7648f4ae5a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 43999d813e564dc688fad116090c95ca)
I20260709 13:23:43.542055 26035 raft_consensus.cc:493] T a5df750b84e0487fa530161079deb6c2 P 233e7d29aac04505bf4e65f0a39a0c4c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 43999d813e564dc688fad116090c95ca)
I20260709 13:23:43.548250 26062 raft_consensus.cc:493] T aad1f7937f854cc1a726223269da5d5a P 233e7d29aac04505bf4e65f0a39a0c4c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9868dce521294d0b816bba7648f4ae5a)
I20260709 13:23:43.524057 26051 raft_consensus.cc:493] T 8ef061028f38450e83324ac78cdaa14c P 233e7d29aac04505bf4e65f0a39a0c4c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 43999d813e564dc688fad116090c95ca)
I20260709 13:23:43.541806 26019 leader_election.cc:290] T 91ac744179de48e5bb924b6dfd9f9c9b P 43999d813e564dc688fad116090c95ca [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3e07d64466934da4bb3c02262b1c2e38 (127.14.177.131:41539), 9868dce521294d0b816bba7648f4ae5a (127.14.177.132:33683)
I20260709 13:23:43.552903 26054 raft_consensus.cc:493] T f1ac2f4d2fe2424bb470df42199219e2 P 9868dce521294d0b816bba7648f4ae5a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 43999d813e564dc688fad116090c95ca)
W20260709 13:23:43.609923 25422 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41539, user_credentials={real_user=slave}} blocked reactor thread for 98980.5us
I20260709 13:23:43.545233 25380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ed5edbe32044d1c83b0590dbb899801" candidate_uuid: "43999d813e564dc688fad116090c95ca" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3e07d64466934da4bb3c02262b1c2e38" is_pre_election: true
I20260709 13:23:43.536291 26057 raft_consensus.cc:493] T 89a26dd3ac414f6081b98fa32c2f346f P 3e07d64466934da4bb3c02262b1c2e38 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9868dce521294d0b816bba7648f4ae5a)
I20260709 13:23:43.535766 26031 leader_election.cc:290] T b155b866f10547fd8239579a81ecf5f9 P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 43999d813e564dc688fad116090c95ca (127.14.177.129:38155), 233e7d29aac04505bf4e65f0a39a0c4c (127.14.177.130:42957)
I20260709 13:23:43.535121 26067 raft_consensus.cc:493] T 434a54fcee7d4aeeab42f739855c0e8a P 233e7d29aac04505bf4e65f0a39a0c4c [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 43999d813e564dc688fad116090c95ca)
I20260709 13:23:43.544979 26037 leader_election.cc:290] T 03e9bfbd2f774476a4a377367796a2e8 P 233e7d29aac04505bf4e65f0a39a0c4c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 43999d813e564dc688fad116090c95ca (127.14.177.129:38155), 3e07d64466934da4bb3c02262b1c2e38 (127.14.177.131:41539)
I20260709 13:23:43.539915 26022 leader_election.cc:290] T 4d837697cd0f402cacf9db89a5d92652 P 9868dce521294d0b816bba7648f4ae5a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3e07d64466934da4bb3c02262b1c2e38 (127.14.177.131:41539), 43999d813e564dc688fad116090c95ca (127.14.177.129:38155)
I20260709 13:23:43.537708 26028 leader_election.cc:290] T beacc815ef954fb692e90544bfa93f05 P 43999d813e564dc688fad116090c95ca [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 233e7d29aac04505bf4e65f0a39a0c4c (127.14.177.130:42957), 3e07d64466934da4bb3c02262b1c2e38 (127.14.177.131:41539)
I20260709 13:23:43.613443 26027 raft_consensus.cc:515] T 9e5d70664720424d951f269d9d7f4346 P 233e7d29aac04505bf4e65f0a39a0c4c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } }
I20260709 13:23:43.624312 25385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2f14aed05e24a98ad83535029b795dc" candidate_uuid: "43999d813e564dc688fad116090c95ca" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3e07d64466934da4bb3c02262b1c2e38" is_pre_election: true
I20260709 13:23:43.550662 25977 leader_election.cc:290] T 0cb89d70b4ce4aa8992fb68480bec8a2 P 43999d813e564dc688fad116090c95ca [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 233e7d29aac04505bf4e65f0a39a0c4c (127.14.177.130:42957), 3e07d64466934da4bb3c02262b1c2e38 (127.14.177.131:41539)
I20260709 13:23:43.551858 26049 leader_election.cc:290] T 79d4a12ef58140568c60a24ed4c12f83 P 233e7d29aac04505bf4e65f0a39a0c4c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 43999d813e564dc688fad116090c95ca (127.14.177.129:38155), 3e07d64466934da4bb3c02262b1c2e38 (127.14.177.131:41539)
I20260709 13:23:43.681599 25422 leader_election.cc:304] T c902bbc7ec0848ba82a04870eab1e7f5 P 9868dce521294d0b816bba7648f4ae5a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e07d64466934da4bb3c02262b1c2e38, 9868dce521294d0b816bba7648f4ae5a; no voters: 
I20260709 13:23:43.543617 26050 leader_election.cc:290] T 821a5866429f4173820fb64336abee02 P 9868dce521294d0b816bba7648f4ae5a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3e07d64466934da4bb3c02262b1c2e38 (127.14.177.131:41539), 43999d813e564dc688fad116090c95ca (127.14.177.129:38155)
I20260709 13:23:43.538475 25306 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c4ab1978a47481c8576d466bacf49d2" candidate_uuid: "3e07d64466934da4bb3c02262b1c2e38" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" is_pre_election: true
I20260709 13:23:43.538038 25303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbab48ba177049dcba6de106fd889cfb" candidate_uuid: "3e07d64466934da4bb3c02262b1c2e38" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" is_pre_election: true
I20260709 13:23:43.605803 26059 raft_consensus.cc:515] T e15820fa8a7d4e1fb6ad23a64ca22f90 P 233e7d29aac04505bf4e65f0a39a0c4c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } }
I20260709 13:23:43.518944 26042 raft_consensus.cc:515] T 72b56266e8a54776b9113c7a786e1952 P 3e07d64466934da4bb3c02262b1c2e38 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } }
W20260709 13:23:43.662631 25274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.132:33683, user_credentials={real_user=slave}} blocked reactor thread for 136773us
I20260709 13:23:43.661522 25382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28c0b6fc9bd64d60828d705c6a9ab1e4" candidate_uuid: "43999d813e564dc688fad116090c95ca" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3e07d64466934da4bb3c02262b1c2e38" is_pre_election: true
I20260709 13:23:43.668001 26030 leader_election.cc:290] T 91ac744179de48e5bb924b6dfd9f9c9b P 9868dce521294d0b816bba7648f4ae5a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3e07d64466934da4bb3c02262b1c2e38 (127.14.177.131:41539), 43999d813e564dc688fad116090c95ca (127.14.177.129:38155)
I20260709 13:23:43.667891 25456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c4f6b7401db47d59a4f34ed0e8b5350" candidate_uuid: "43999d813e564dc688fad116090c95ca" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9868dce521294d0b816bba7648f4ae5a" is_pre_election: true
I20260709 13:23:43.638600 25451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f15ec62698c4a8cbd84f2b3d516d2ea" candidate_uuid: "43999d813e564dc688fad116090c95ca" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9868dce521294d0b816bba7648f4ae5a" is_pre_election: true
I20260709 13:23:43.540572 26069 raft_consensus.cc:493] T b9d550d881b04735bbf89751952c09ec P 9868dce521294d0b816bba7648f4ae5a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 43999d813e564dc688fad116090c95ca)
I20260709 13:23:43.638602 26035 raft_consensus.cc:515] T a5df750b84e0487fa530161079deb6c2 P 233e7d29aac04505bf4e65f0a39a0c4c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } }
I20260709 13:23:43.638835 26053 raft_consensus.cc:515] T 441d780c3945463f92f31170f31e37a8 P 9868dce521294d0b816bba7648f4ae5a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } }
I20260709 13:23:43.681746 25378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "beacc815ef954fb692e90544bfa93f05" candidate_uuid: "43999d813e564dc688fad116090c95ca" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3e07d64466934da4bb3c02262b1c2e38" is_pre_election: true
I20260709 13:23:43.642916 26062 raft_consensus.cc:515] T aad1f7937f854cc1a726223269da5d5a P 233e7d29aac04505bf4e65f0a39a0c4c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } }
I20260709 13:23:43.687721 25422 leader_election.cc:304] T 5c7e7acbb09f4d49828fa652ca6abaac P 9868dce521294d0b816bba7648f4ae5a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e07d64466934da4bb3c02262b1c2e38, 9868dce521294d0b816bba7648f4ae5a; no voters: 
I20260709 13:23:43.661705 26051 raft_consensus.cc:515] T 8ef061028f38450e83324ac78cdaa14c P 233e7d29aac04505bf4e65f0a39a0c4c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } }
I20260709 13:23:43.701577 26021 raft_consensus.cc:515] T 1f15ec62698c4a8cbd84f2b3d516d2ea P 9868dce521294d0b816bba7648f4ae5a [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } }
I20260709 13:23:43.702374 26053 leader_election.cc:290] T 441d780c3945463f92f31170f31e37a8 P 9868dce521294d0b816bba7648f4ae5a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 43999d813e564dc688fad116090c95ca (127.14.177.129:38155), 233e7d29aac04505bf4e65f0a39a0c4c (127.14.177.130:42957)
I20260709 13:23:43.693838 25382 raft_consensus.cc:2468] T 28c0b6fc9bd64d60828d705c6a9ab1e4 P 3e07d64466934da4bb3c02262b1c2e38 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 43999d813e564dc688fad116090c95ca in term 1.
I20260709 13:23:43.703548 26051 leader_election.cc:290] T 8ef061028f38450e83324ac78cdaa14c P 233e7d29aac04505bf4e65f0a39a0c4c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9868dce521294d0b816bba7648f4ae5a (127.14.177.132:33683), 43999d813e564dc688fad116090c95ca (127.14.177.129:38155)
I20260709 13:23:43.704365 26022 raft_consensus.cc:2804] T 5c7e7acbb09f4d49828fa652ca6abaac P 9868dce521294d0b816bba7648f4ae5a [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:23:43.704509 26036 leader_election.cc:290] T 441d780c3945463f92f31170f31e37a8 P 233e7d29aac04505bf4e65f0a39a0c4c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9868dce521294d0b816bba7648f4ae5a (127.14.177.132:33683), 43999d813e564dc688fad116090c95ca (127.14.177.129:38155)
I20260709 13:23:43.704850 26022 raft_consensus.cc:493] T 5c7e7acbb09f4d49828fa652ca6abaac P 9868dce521294d0b816bba7648f4ae5a [term 1 FOLLOWER]: Starting leader election (detected failure of leader 43999d813e564dc688fad116090c95ca)
I20260709 13:23:43.705119 26058 leader_election.cc:290] T cfad575709c943cba40ccab278350770 P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 233e7d29aac04505bf4e65f0a39a0c4c (127.14.177.130:42957), 9868dce521294d0b816bba7648f4ae5a (127.14.177.132:33683)
I20260709 13:23:43.705400 26046 leader_election.cc:290] T 434a18a5109e4ea78349ded3968b2709 P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 233e7d29aac04505bf4e65f0a39a0c4c (127.14.177.130:42957), 9868dce521294d0b816bba7648f4ae5a (127.14.177.132:33683)
I20260709 13:23:43.674191 26057 raft_consensus.cc:515] T 89a26dd3ac414f6081b98fa32c2f346f P 3e07d64466934da4bb3c02262b1c2e38 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } }
I20260709 13:23:43.706905 26024 leader_election.cc:290] T 79d4a12ef58140568c60a24ed4c12f83 P 43999d813e564dc688fad116090c95ca [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 233e7d29aac04505bf4e65f0a39a0c4c (127.14.177.130:42957), 3e07d64466934da4bb3c02262b1c2e38 (127.14.177.131:41539)
I20260709 13:23:43.699327 25459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "280d65245f4f4358b54bef93e18aca92" candidate_uuid: "43999d813e564dc688fad116090c95ca" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9868dce521294d0b816bba7648f4ae5a" is_pre_election: true
I20260709 13:23:43.707564 26042 leader_election.cc:290] T 72b56266e8a54776b9113c7a786e1952 P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 43999d813e564dc688fad116090c95ca (127.14.177.129:38155), 233e7d29aac04505bf4e65f0a39a0c4c (127.14.177.130:42957)
I20260709 13:23:43.705334 26022 raft_consensus.cc:3060] T 5c7e7acbb09f4d49828fa652ca6abaac P 9868dce521294d0b816bba7648f4ae5a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:23:43.708251 26057 leader_election.cc:290] T 89a26dd3ac414f6081b98fa32c2f346f P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 233e7d29aac04505bf4e65f0a39a0c4c (127.14.177.130:42957), 9868dce521294d0b816bba7648f4ae5a (127.14.177.132:33683)
I20260709 13:23:43.709491 26059 leader_election.cc:290] T e15820fa8a7d4e1fb6ad23a64ca22f90 P 233e7d29aac04505bf4e65f0a39a0c4c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 43999d813e564dc688fad116090c95ca (127.14.177.129:38155), 3e07d64466934da4bb3c02262b1c2e38 (127.14.177.131:41539)
W20260709 13:23:43.698629 25276 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.129:38155, user_credentials={real_user=slave}} blocked reactor thread for 56828.8us
I20260709 13:23:43.700459 26035 leader_election.cc:290] T a5df750b84e0487fa530161079deb6c2 P 233e7d29aac04505bf4e65f0a39a0c4c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9868dce521294d0b816bba7648f4ae5a (127.14.177.132:33683), 43999d813e564dc688fad116090c95ca (127.14.177.129:38155)
I20260709 13:23:43.705451 26021 leader_election.cc:290] T 1f15ec62698c4a8cbd84f2b3d516d2ea P 9868dce521294d0b816bba7648f4ae5a [CANDIDATE]: Term 2 election: Requested vote from peers 3e07d64466934da4bb3c02262b1c2e38 (127.14.177.131:41539), 43999d813e564dc688fad116090c95ca (127.14.177.129:38155)
I20260709 13:23:43.698952 26069 raft_consensus.cc:515] T b9d550d881b04735bbf89751952c09ec P 9868dce521294d0b816bba7648f4ae5a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } }
I20260709 13:23:43.694816 26050 raft_consensus.cc:2804] T c902bbc7ec0848ba82a04870eab1e7f5 P 9868dce521294d0b816bba7648f4ae5a [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:23:43.704406 25451 raft_consensus.cc:2393] T 1f15ec62698c4a8cbd84f2b3d516d2ea P 9868dce521294d0b816bba7648f4ae5a [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 43999d813e564dc688fad116090c95ca in current term 2: Already voted for candidate 9868dce521294d0b816bba7648f4ae5a in this term.
I20260709 13:23:43.711105 26049 raft_consensus.cc:493] T cfad575709c943cba40ccab278350770 P 233e7d29aac04505bf4e65f0a39a0c4c [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 9868dce521294d0b816bba7648f4ae5a)
I20260709 13:23:43.695214 26044 leader_election.cc:290] T ba8a000f5abe4158965449995af84b18 P 9868dce521294d0b816bba7648f4ae5a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 233e7d29aac04505bf4e65f0a39a0c4c (127.14.177.130:42957), 3e07d64466934da4bb3c02262b1c2e38 (127.14.177.131:41539)
I20260709 13:23:43.711527 26026 leader_election.cc:290] T 434a54fcee7d4aeeab42f739855c0e8a P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 43999d813e564dc688fad116090c95ca (127.14.177.129:38155), 233e7d29aac04505bf4e65f0a39a0c4c (127.14.177.130:42957)
I20260709 13:23:43.680218 26067 raft_consensus.cc:515] T 434a54fcee7d4aeeab42f739855c0e8a P 233e7d29aac04505bf4e65f0a39a0c4c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } }
I20260709 13:23:43.665304 26056 raft_consensus.cc:515] T 3a3317f4c3514df1b2bb8e44a86be92f P 9868dce521294d0b816bba7648f4ae5a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } }
I20260709 13:23:43.713197 26027 leader_election.cc:290] T 9e5d70664720424d951f269d9d7f4346 P 233e7d29aac04505bf4e65f0a39a0c4c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9868dce521294d0b816bba7648f4ae5a (127.14.177.132:33683), 43999d813e564dc688fad116090c95ca (127.14.177.129:38155)
I20260709 13:23:43.714346 26047 leader_election.cc:290] T 13bb3a23315a41c7a40e10b954b15c8e P 233e7d29aac04505bf4e65f0a39a0c4c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 43999d813e564dc688fad116090c95ca (127.14.177.129:38155), 3e07d64466934da4bb3c02262b1c2e38 (127.14.177.131:41539)
I20260709 13:23:43.715662 25380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cb89d70b4ce4aa8992fb68480bec8a2" candidate_uuid: "43999d813e564dc688fad116090c95ca" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3e07d64466934da4bb3c02262b1c2e38" is_pre_election: true
W20260709 13:23:43.716490 25350 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.129:38155, user_credentials={real_user=slave}} blocked reactor thread for 266862us
I20260709 13:23:43.718394 26062 leader_election.cc:290] T aad1f7937f854cc1a726223269da5d5a P 233e7d29aac04505bf4e65f0a39a0c4c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3e07d64466934da4bb3c02262b1c2e38 (127.14.177.131:41539), 9868dce521294d0b816bba7648f4ae5a (127.14.177.132:33683)
I20260709 13:23:43.701608 26029 leader_election.cc:290] T 87085ac2575e4b6a86dd1fc21e6f718b P 233e7d29aac04505bf4e65f0a39a0c4c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 43999d813e564dc688fad116090c95ca (127.14.177.129:38155), 3e07d64466934da4bb3c02262b1c2e38 (127.14.177.131:41539)
I20260709 13:23:43.707937 26072 raft_consensus.cc:493] T b155b866f10547fd8239579a81ecf5f9 P 233e7d29aac04505bf4e65f0a39a0c4c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 43999d813e564dc688fad116090c95ca)
I20260709 13:23:43.671437 26054 raft_consensus.cc:515] T f1ac2f4d2fe2424bb470df42199219e2 P 9868dce521294d0b816bba7648f4ae5a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } }
I20260709 13:23:43.731225 25274 leader_election.cc:304] T ad32097cd4dc48b9a69595f1651abd07 P 233e7d29aac04505bf4e65f0a39a0c4c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 233e7d29aac04505bf4e65f0a39a0c4c, 9868dce521294d0b816bba7648f4ae5a; no voters: 
I20260709 13:23:43.721637 26050 raft_consensus.cc:493] T c902bbc7ec0848ba82a04870eab1e7f5 P 9868dce521294d0b816bba7648f4ae5a [term 1 FOLLOWER]: Starting leader election (detected failure of leader 43999d813e564dc688fad116090c95ca)
I20260709 13:23:43.693750 25377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91ac744179de48e5bb924b6dfd9f9c9b" candidate_uuid: "43999d813e564dc688fad116090c95ca" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3e07d64466934da4bb3c02262b1c2e38" is_pre_election: true
I20260709 13:23:43.734841 25459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b85a238bbc3b4621881603a534c7bd9b" candidate_uuid: "43999d813e564dc688fad116090c95ca" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9868dce521294d0b816bba7648f4ae5a" is_pre_election: true
I20260709 13:23:43.729842 26072 raft_consensus.cc:515] T b155b866f10547fd8239579a81ecf5f9 P 233e7d29aac04505bf4e65f0a39a0c4c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } }
I20260709 13:23:43.736284 25274 leader_election.cc:304] T ce9c60e3922c4219b6e4453fe0259b6a P 233e7d29aac04505bf4e65f0a39a0c4c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 233e7d29aac04505bf4e65f0a39a0c4c, 9868dce521294d0b816bba7648f4ae5a; no voters: 
I20260709 13:23:43.736518 26054 leader_election.cc:290] T f1ac2f4d2fe2424bb470df42199219e2 P 9868dce521294d0b816bba7648f4ae5a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3e07d64466934da4bb3c02262b1c2e38 (127.14.177.131:41539), 43999d813e564dc688fad116090c95ca (127.14.177.129:38155)
I20260709 13:23:43.736865 26062 raft_consensus.cc:2804] T ad32097cd4dc48b9a69595f1651abd07 P 233e7d29aac04505bf4e65f0a39a0c4c [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:23:43.720830 26043 leader_election.cc:290] T beacc815ef954fb692e90544bfa93f05 P 233e7d29aac04505bf4e65f0a39a0c4c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 43999d813e564dc688fad116090c95ca (127.14.177.129:38155), 3e07d64466934da4bb3c02262b1c2e38 (127.14.177.131:41539)
I20260709 13:23:43.724943 26049 raft_consensus.cc:515] T cfad575709c943cba40ccab278350770 P 233e7d29aac04505bf4e65f0a39a0c4c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } }
I20260709 13:23:43.744979 26069 leader_election.cc:290] T b9d550d881b04735bbf89751952c09ec P 9868dce521294d0b816bba7648f4ae5a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3e07d64466934da4bb3c02262b1c2e38 (127.14.177.131:41539), 43999d813e564dc688fad116090c95ca (127.14.177.129:38155)
I20260709 13:23:43.747371 26050 raft_consensus.cc:3060] T c902bbc7ec0848ba82a04870eab1e7f5 P 9868dce521294d0b816bba7648f4ae5a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:23:43.752017 26062 raft_consensus.cc:493] T ad32097cd4dc48b9a69595f1651abd07 P 233e7d29aac04505bf4e65f0a39a0c4c [term 1 FOLLOWER]: Starting leader election (detected failure of leader 3e07d64466934da4bb3c02262b1c2e38)
I20260709 13:23:43.759747 26062 raft_consensus.cc:3060] T ad32097cd4dc48b9a69595f1651abd07 P 233e7d29aac04505bf4e65f0a39a0c4c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:23:43.760097 25306 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e311eb90bad44c80a131b72edcd22b06" candidate_uuid: "3e07d64466934da4bb3c02262b1c2e38" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" is_pre_election: true
I20260709 13:23:43.759766 25453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9e342ecda4b46619ce12452b2cf7154" candidate_uuid: "43999d813e564dc688fad116090c95ca" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9868dce521294d0b816bba7648f4ae5a" is_pre_election: true
I20260709 13:23:43.763523 25456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51a11db644124b63bdf5f457c967473d" candidate_uuid: "43999d813e564dc688fad116090c95ca" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9868dce521294d0b816bba7648f4ae5a" is_pre_election: true
I20260709 13:23:43.763760 25457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ed5edbe32044d1c83b0590dbb899801" candidate_uuid: "43999d813e564dc688fad116090c95ca" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9868dce521294d0b816bba7648f4ae5a" is_pre_election: true
I20260709 13:23:43.764385 25457 raft_consensus.cc:2468] T 0ed5edbe32044d1c83b0590dbb899801 P 9868dce521294d0b816bba7648f4ae5a [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 43999d813e564dc688fad116090c95ca in term 1.
I20260709 13:23:43.764669 25458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11949f56648c47e0a43c3140af99a304" candidate_uuid: "43999d813e564dc688fad116090c95ca" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9868dce521294d0b816bba7648f4ae5a" is_pre_election: true
I20260709 13:23:43.755681 26059 raft_consensus.cc:2804] T ce9c60e3922c4219b6e4453fe0259b6a P 233e7d29aac04505bf4e65f0a39a0c4c [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:23:43.768047 25381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79d4a12ef58140568c60a24ed4c12f83" candidate_uuid: "43999d813e564dc688fad116090c95ca" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3e07d64466934da4bb3c02262b1c2e38" is_pre_election: true
I20260709 13:23:43.773342 26072 leader_election.cc:290] T b155b866f10547fd8239579a81ecf5f9 P 233e7d29aac04505bf4e65f0a39a0c4c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 43999d813e564dc688fad116090c95ca (127.14.177.129:38155), 3e07d64466934da4bb3c02262b1c2e38 (127.14.177.131:41539)
I20260709 13:23:43.773614 26056 leader_election.cc:290] T 3a3317f4c3514df1b2bb8e44a86be92f P 9868dce521294d0b816bba7648f4ae5a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 43999d813e564dc688fad116090c95ca (127.14.177.129:38155), 233e7d29aac04505bf4e65f0a39a0c4c (127.14.177.130:42957)
I20260709 13:23:43.774942 25304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba8a000f5abe4158965449995af84b18" candidate_uuid: "3e07d64466934da4bb3c02262b1c2e38" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" is_pre_election: true
I20260709 13:23:43.775241 25453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6242888a4d4146dcacd87d0fd9a832f4" candidate_uuid: "43999d813e564dc688fad116090c95ca" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9868dce521294d0b816bba7648f4ae5a" is_pre_election: true
I20260709 13:23:43.775696 25302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b155b866f10547fd8239579a81ecf5f9" candidate_uuid: "3e07d64466934da4bb3c02262b1c2e38" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" is_pre_election: true
I20260709 13:23:43.766256 25454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e5d70664720424d951f269d9d7f4346" candidate_uuid: "43999d813e564dc688fad116090c95ca" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9868dce521294d0b816bba7648f4ae5a" is_pre_election: true
I20260709 13:23:43.766280 26059 raft_consensus.cc:493] T ce9c60e3922c4219b6e4453fe0259b6a P 233e7d29aac04505bf4e65f0a39a0c4c [term 1 FOLLOWER]: Starting leader election (detected failure of leader 43999d813e564dc688fad116090c95ca)
I20260709 13:23:43.776278 25303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "467a6b42ca6548269c12dddde696f7af" candidate_uuid: "3e07d64466934da4bb3c02262b1c2e38" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" is_pre_election: true
I20260709 13:23:43.776444 25302 raft_consensus.cc:2468] T b155b866f10547fd8239579a81ecf5f9 P 233e7d29aac04505bf4e65f0a39a0c4c [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3e07d64466934da4bb3c02262b1c2e38 in term 1.
I20260709 13:23:43.776818 26059 raft_consensus.cc:3060] T ce9c60e3922c4219b6e4453fe0259b6a P 233e7d29aac04505bf4e65f0a39a0c4c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:23:43.777155 26056 raft_consensus.cc:493] T b7a891bfc89e4e259f53da6cb16f4989 P 9868dce521294d0b816bba7648f4ae5a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3e07d64466934da4bb3c02262b1c2e38)
I20260709 13:23:43.777765 25305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfad575709c943cba40ccab278350770" candidate_uuid: "3e07d64466934da4bb3c02262b1c2e38" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" is_pre_election: true
I20260709 13:23:43.777717 26056 raft_consensus.cc:515] T b7a891bfc89e4e259f53da6cb16f4989 P 9868dce521294d0b816bba7648f4ae5a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } }
I20260709 13:23:43.778475 26067 leader_election.cc:290] T 434a54fcee7d4aeeab42f739855c0e8a P 233e7d29aac04505bf4e65f0a39a0c4c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 43999d813e564dc688fad116090c95ca (127.14.177.129:38155), 3e07d64466934da4bb3c02262b1c2e38 (127.14.177.131:41539)
I20260709 13:23:43.778537 25305 raft_consensus.cc:2468] T cfad575709c943cba40ccab278350770 P 233e7d29aac04505bf4e65f0a39a0c4c [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3e07d64466934da4bb3c02262b1c2e38 in term 2.
I20260709 13:23:43.769852 25199 leader_election.cc:304] T 28c0b6fc9bd64d60828d705c6a9ab1e4 P 43999d813e564dc688fad116090c95ca [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e07d64466934da4bb3c02262b1c2e38, 43999d813e564dc688fad116090c95ca; no voters: 
I20260709 13:23:43.779784 26056 leader_election.cc:290] T b7a891bfc89e4e259f53da6cb16f4989 P 9868dce521294d0b816bba7648f4ae5a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3e07d64466934da4bb3c02262b1c2e38 (127.14.177.131:41539), 43999d813e564dc688fad116090c95ca (127.14.177.129:38155)
I20260709 13:23:43.779995 26028 raft_consensus.cc:2804] T 28c0b6fc9bd64d60828d705c6a9ab1e4 P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:23:43.780823 25309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72b56266e8a54776b9113c7a786e1952" candidate_uuid: "3e07d64466934da4bb3c02262b1c2e38" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" is_pre_election: true
I20260709 13:23:43.781481 25309 raft_consensus.cc:2468] T 72b56266e8a54776b9113c7a786e1952 P 233e7d29aac04505bf4e65f0a39a0c4c [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3e07d64466934da4bb3c02262b1c2e38 in term 1.
I20260709 13:23:43.777071 25303 raft_consensus.cc:2468] T 467a6b42ca6548269c12dddde696f7af P 233e7d29aac04505bf4e65f0a39a0c4c [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3e07d64466934da4bb3c02262b1c2e38 in term 1.
I20260709 13:23:43.784303 26028 raft_consensus.cc:493] T 28c0b6fc9bd64d60828d705c6a9ab1e4 P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Starting leader election (detected failure of leader 233e7d29aac04505bf4e65f0a39a0c4c)
I20260709 13:23:43.790690 25451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a820954e07a94582ac0613342fcffde5" candidate_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9868dce521294d0b816bba7648f4ae5a" is_pre_election: true
I20260709 13:23:43.791566 25459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7a891bfc89e4e259f53da6cb16f4989" candidate_uuid: "43999d813e564dc688fad116090c95ca" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9868dce521294d0b816bba7648f4ae5a" is_pre_election: true
I20260709 13:23:43.792349 25459 raft_consensus.cc:2468] T b7a891bfc89e4e259f53da6cb16f4989 P 9868dce521294d0b816bba7648f4ae5a [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 43999d813e564dc688fad116090c95ca in term 1.
I20260709 13:23:43.790848 26028 raft_consensus.cc:3060] T 28c0b6fc9bd64d60828d705c6a9ab1e4 P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:23:43.796909 26049 leader_election.cc:290] T cfad575709c943cba40ccab278350770 P 233e7d29aac04505bf4e65f0a39a0c4c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3e07d64466934da4bb3c02262b1c2e38 (127.14.177.131:41539), 9868dce521294d0b816bba7648f4ae5a (127.14.177.132:33683)
I20260709 13:23:43.802984 25451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91ac744179de48e5bb924b6dfd9f9c9b" candidate_uuid: "43999d813e564dc688fad116090c95ca" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9868dce521294d0b816bba7648f4ae5a" is_pre_election: true
I20260709 13:23:43.803709 25451 raft_consensus.cc:2468] T 91ac744179de48e5bb924b6dfd9f9c9b P 9868dce521294d0b816bba7648f4ae5a [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 43999d813e564dc688fad116090c95ca in term 1.
I20260709 13:23:43.805651 25307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce98a3f3b3eb4c63bcaa355ec7cd64a4" candidate_uuid: "43999d813e564dc688fad116090c95ca" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" is_pre_election: true
I20260709 13:23:43.807003 25307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b85a238bbc3b4621881603a534c7bd9b" candidate_uuid: "43999d813e564dc688fad116090c95ca" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" is_pre_election: true
I20260709 13:23:43.811558 25302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c4f6b7401db47d59a4f34ed0e8b5350" candidate_uuid: "43999d813e564dc688fad116090c95ca" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" is_pre_election: true
I20260709 13:23:43.812415 25451 raft_consensus.cc:1240] T 1f15ec62698c4a8cbd84f2b3d516d2ea P 9868dce521294d0b816bba7648f4ae5a [term 2 FOLLOWER]: Rejecting Update request from peer 3e07d64466934da4bb3c02262b1c2e38 for earlier term 1. Current term is 2. Ops: []
I20260709 13:23:43.813896 25451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbab48ba177049dcba6de106fd889cfb" candidate_uuid: "3e07d64466934da4bb3c02262b1c2e38" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9868dce521294d0b816bba7648f4ae5a" is_pre_election: true
I20260709 13:23:43.800513 26022 raft_consensus.cc:515] T 5c7e7acbb09f4d49828fa652ca6abaac P 9868dce521294d0b816bba7648f4ae5a [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } }
I20260709 13:23:43.815536 25451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c7e7acbb09f4d49828fa652ca6abaac" candidate_uuid: "3e07d64466934da4bb3c02262b1c2e38" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9868dce521294d0b816bba7648f4ae5a" is_pre_election: true
I20260709 13:23:43.820259 25310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434a54fcee7d4aeeab42f739855c0e8a" candidate_uuid: "3e07d64466934da4bb3c02262b1c2e38" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" is_pre_election: true
I20260709 13:23:43.820854 25457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c4ab1978a47481c8576d466bacf49d2" candidate_uuid: "3e07d64466934da4bb3c02262b1c2e38" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9868dce521294d0b816bba7648f4ae5a" is_pre_election: true
I20260709 13:23:43.807636 25308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d29b499fa4c41f1b974782cd0c9c1c2" candidate_uuid: "43999d813e564dc688fad116090c95ca" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" is_pre_election: true
I20260709 13:23:43.811666 26050 raft_consensus.cc:515] T c902bbc7ec0848ba82a04870eab1e7f5 P 9868dce521294d0b816bba7648f4ae5a [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } }
I20260709 13:23:43.819468 25311 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89a26dd3ac414f6081b98fa32c2f346f" candidate_uuid: "3e07d64466934da4bb3c02262b1c2e38" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" is_pre_election: true
I20260709 13:23:43.822288 25311 raft_consensus.cc:2468] T 89a26dd3ac414f6081b98fa32c2f346f P 233e7d29aac04505bf4e65f0a39a0c4c [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3e07d64466934da4bb3c02262b1c2e38 in term 1.
I20260709 13:23:43.818674 25351 leader_election.cc:304] T b155b866f10547fd8239579a81ecf5f9 P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 233e7d29aac04505bf4e65f0a39a0c4c, 3e07d64466934da4bb3c02262b1c2e38; no voters: 
I20260709 13:23:43.822914 25304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434a18a5109e4ea78349ded3968b2709" candidate_uuid: "3e07d64466934da4bb3c02262b1c2e38" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" is_pre_election: true
I20260709 13:23:43.823559 26066 raft_consensus.cc:2804] T b155b866f10547fd8239579a81ecf5f9 P 3e07d64466934da4bb3c02262b1c2e38 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:23:43.823992 26066 raft_consensus.cc:493] T b155b866f10547fd8239579a81ecf5f9 P 3e07d64466934da4bb3c02262b1c2e38 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 43999d813e564dc688fad116090c95ca)
I20260709 13:23:43.824375 25455 raft_consensus.cc:1240] T 5c7e7acbb09f4d49828fa652ca6abaac P 9868dce521294d0b816bba7648f4ae5a [term 2 FOLLOWER]: Rejecting Update request from peer 43999d813e564dc688fad116090c95ca for earlier term 1. Current term is 2. Ops: []
I20260709 13:23:43.824397 26066 raft_consensus.cc:3060] T b155b866f10547fd8239579a81ecf5f9 P 3e07d64466934da4bb3c02262b1c2e38 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:23:43.821559 25459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89a26dd3ac414f6081b98fa32c2f346f" candidate_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9868dce521294d0b816bba7648f4ae5a" is_pre_election: true
I20260709 13:23:43.829054 25453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "467a6b42ca6548269c12dddde696f7af" candidate_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9868dce521294d0b816bba7648f4ae5a" is_pre_election: true
I20260709 13:23:43.830029 25351 leader_election.cc:304] T cfad575709c943cba40ccab278350770 P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 233e7d29aac04505bf4e65f0a39a0c4c, 3e07d64466934da4bb3c02262b1c2e38; no voters: 
I20260709 13:23:43.838689 26058 raft_consensus.cc:2804] T cfad575709c943cba40ccab278350770 P 3e07d64466934da4bb3c02262b1c2e38 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:23:43.838732 25310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51a11db644124b63bdf5f457c967473d" candidate_uuid: "43999d813e564dc688fad116090c95ca" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" is_pre_election: true
I20260709 13:23:43.839335 26058 raft_consensus.cc:493] T cfad575709c943cba40ccab278350770 P 3e07d64466934da4bb3c02262b1c2e38 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 9868dce521294d0b816bba7648f4ae5a)
I20260709 13:23:43.839789 26058 raft_consensus.cc:3060] T cfad575709c943cba40ccab278350770 P 3e07d64466934da4bb3c02262b1c2e38 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:23:43.841866 25306 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11949f56648c47e0a43c3140af99a304" candidate_uuid: "43999d813e564dc688fad116090c95ca" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" is_pre_election: true
I20260709 13:23:43.842155 25302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15820fa8a7d4e1fb6ad23a64ca22f90" candidate_uuid: "43999d813e564dc688fad116090c95ca" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" is_pre_election: true
I20260709 13:23:43.786361 25458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11949f56648c47e0a43c3140af99a304" candidate_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9868dce521294d0b816bba7648f4ae5a" is_pre_election: true
I20260709 13:23:43.801627 25454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d053b9ae38de4173a0aded4da494bd98" candidate_uuid: "43999d813e564dc688fad116090c95ca" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9868dce521294d0b816bba7648f4ae5a" is_pre_election: true
I20260709 13:23:43.844175 26028 raft_consensus.cc:515] T 28c0b6fc9bd64d60828d705c6a9ab1e4 P 43999d813e564dc688fad116090c95ca [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } }
I20260709 13:23:43.844393 25351 leader_election.cc:304] T 72b56266e8a54776b9113c7a786e1952 P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 233e7d29aac04505bf4e65f0a39a0c4c, 3e07d64466934da4bb3c02262b1c2e38; no voters: 
I20260709 13:23:43.837697 25311 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9e342ecda4b46619ce12452b2cf7154" candidate_uuid: "43999d813e564dc688fad116090c95ca" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" is_pre_election: true
I20260709 13:23:43.850837 25311 raft_consensus.cc:2468] T e9e342ecda4b46619ce12452b2cf7154 P 233e7d29aac04505bf4e65f0a39a0c4c [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 43999d813e564dc688fad116090c95ca in term 1.
I20260709 13:23:43.851241 25302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28c0b6fc9bd64d60828d705c6a9ab1e4" candidate_uuid: "43999d813e564dc688fad116090c95ca" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" is_pre_election: true
I20260709 13:23:43.845930 25452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55e8ed3aee404ce698d0626a87bd03aa" candidate_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9868dce521294d0b816bba7648f4ae5a" is_pre_election: true
I20260709 13:23:43.854274 26042 raft_consensus.cc:2804] T 72b56266e8a54776b9113c7a786e1952 P 3e07d64466934da4bb3c02262b1c2e38 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:23:43.854707 26042 raft_consensus.cc:493] T 72b56266e8a54776b9113c7a786e1952 P 3e07d64466934da4bb3c02262b1c2e38 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 43999d813e564dc688fad116090c95ca)
I20260709 13:23:43.854629 25306 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6242888a4d4146dcacd87d0fd9a832f4" candidate_uuid: "43999d813e564dc688fad116090c95ca" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" is_pre_election: true
I20260709 13:23:43.855382 26042 raft_consensus.cc:3060] T 72b56266e8a54776b9113c7a786e1952 P 3e07d64466934da4bb3c02262b1c2e38 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:23:43.855680 26062 raft_consensus.cc:515] T ad32097cd4dc48b9a69595f1651abd07 P 233e7d29aac04505bf4e65f0a39a0c4c [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } }
I20260709 13:23:43.855654 25306 raft_consensus.cc:2468] T 6242888a4d4146dcacd87d0fd9a832f4 P 233e7d29aac04505bf4e65f0a39a0c4c [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 43999d813e564dc688fad116090c95ca in term 1.
I20260709 13:23:43.857303 25309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2f14aed05e24a98ad83535029b795dc" candidate_uuid: "43999d813e564dc688fad116090c95ca" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" is_pre_election: true
I20260709 13:23:43.858233 26062 leader_election.cc:290] T ad32097cd4dc48b9a69595f1651abd07 P 233e7d29aac04505bf4e65f0a39a0c4c [CANDIDATE]: Term 2 election: Requested vote from peers 3e07d64466934da4bb3c02262b1c2e38 (127.14.177.131:41539), 9868dce521294d0b816bba7648f4ae5a (127.14.177.132:33683)
W20260709 13:23:43.859062 25350 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:38155, user_credentials={real_user=slave}} blocked reactor thread for 126913us
I20260709 13:23:43.836297 26022 leader_election.cc:290] T 5c7e7acbb09f4d49828fa652ca6abaac P 9868dce521294d0b816bba7648f4ae5a [CANDIDATE]: Term 2 election: Requested vote from peers 3e07d64466934da4bb3c02262b1c2e38 (127.14.177.131:41539), 43999d813e564dc688fad116090c95ca (127.14.177.129:38155)
I20260709 13:23:43.849094 25310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e5d70664720424d951f269d9d7f4346" candidate_uuid: "43999d813e564dc688fad116090c95ca" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" is_pre_election: true
I20260709 13:23:43.849943 25305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03e9bfbd2f774476a4a377367796a2e8" candidate_uuid: "43999d813e564dc688fad116090c95ca" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" is_pre_election: true
I20260709 13:23:43.859797 25199 leader_election.cc:304] T 280d65245f4f4358b54bef93e18aca92 P 43999d813e564dc688fad116090c95ca [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 43999d813e564dc688fad116090c95ca; no voters: 3e07d64466934da4bb3c02262b1c2e38, 9868dce521294d0b816bba7648f4ae5a
I20260709 13:23:43.859330 25351 leader_election.cc:304] T 467a6b42ca6548269c12dddde696f7af P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 233e7d29aac04505bf4e65f0a39a0c4c, 3e07d64466934da4bb3c02262b1c2e38; no voters: 
I20260709 13:23:43.846832 26028 leader_election.cc:290] T 28c0b6fc9bd64d60828d705c6a9ab1e4 P 43999d813e564dc688fad116090c95ca [CANDIDATE]: Term 2 election: Requested vote from peers 233e7d29aac04505bf4e65f0a39a0c4c (127.14.177.130:42957), 3e07d64466934da4bb3c02262b1c2e38 (127.14.177.131:41539)
I20260709 13:23:43.836925 26050 leader_election.cc:290] T c902bbc7ec0848ba82a04870eab1e7f5 P 9868dce521294d0b816bba7648f4ae5a [CANDIDATE]: Term 2 election: Requested vote from peers 3e07d64466934da4bb3c02262b1c2e38 (127.14.177.131:41539), 43999d813e564dc688fad116090c95ca (127.14.177.129:38155)
I20260709 13:23:43.867528 25309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d053b9ae38de4173a0aded4da494bd98" candidate_uuid: "43999d813e564dc688fad116090c95ca" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" is_pre_election: true
I20260709 13:23:43.870689 25377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cb89d70b4ce4aa8992fb68480bec8a2" candidate_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3e07d64466934da4bb3c02262b1c2e38" is_pre_election: true
I20260709 13:23:43.871292 26026 raft_consensus.cc:2804] T 467a6b42ca6548269c12dddde696f7af P 3e07d64466934da4bb3c02262b1c2e38 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:23:43.871407 25456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "467a6b42ca6548269c12dddde696f7af" candidate_uuid: "3e07d64466934da4bb3c02262b1c2e38" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9868dce521294d0b816bba7648f4ae5a" is_pre_election: true
I20260709 13:23:43.871850 26026 raft_consensus.cc:493] T 467a6b42ca6548269c12dddde696f7af P 3e07d64466934da4bb3c02262b1c2e38 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 9868dce521294d0b816bba7648f4ae5a)
I20260709 13:23:43.872368 26026 raft_consensus.cc:3060] T 467a6b42ca6548269c12dddde696f7af P 3e07d64466934da4bb3c02262b1c2e38 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:23:43.870733 25380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "467a6b42ca6548269c12dddde696f7af" candidate_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3e07d64466934da4bb3c02262b1c2e38" is_pre_election: true
I20260709 13:23:43.874354 26024 raft_consensus.cc:2749] T 280d65245f4f4358b54bef93e18aca92 P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:23:43.864720 25351 leader_election.cc:304] T 89a26dd3ac414f6081b98fa32c2f346f P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 233e7d29aac04505bf4e65f0a39a0c4c, 3e07d64466934da4bb3c02262b1c2e38; no voters: 
I20260709 13:23:43.883062 25377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a820954e07a94582ac0613342fcffde5" candidate_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3e07d64466934da4bb3c02262b1c2e38" is_pre_election: true
I20260709 13:23:43.880690 26057 raft_consensus.cc:2804] T 89a26dd3ac414f6081b98fa32c2f346f P 3e07d64466934da4bb3c02262b1c2e38 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:23:43.883919 26057 raft_consensus.cc:493] T 89a26dd3ac414f6081b98fa32c2f346f P 3e07d64466934da4bb3c02262b1c2e38 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 9868dce521294d0b816bba7648f4ae5a)
I20260709 13:23:43.883037 25385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89a26dd3ac414f6081b98fa32c2f346f" candidate_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3e07d64466934da4bb3c02262b1c2e38" is_pre_election: true
I20260709 13:23:43.884390 26057 raft_consensus.cc:3060] T 89a26dd3ac414f6081b98fa32c2f346f P 3e07d64466934da4bb3c02262b1c2e38 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:23:43.885864 25311 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ef061028f38450e83324ac78cdaa14c" candidate_uuid: "9868dce521294d0b816bba7648f4ae5a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" is_pre_election: true
I20260709 13:23:43.886070 25303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad32097cd4dc48b9a69595f1651abd07" candidate_uuid: "9868dce521294d0b816bba7648f4ae5a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" is_pre_election: true
I20260709 13:23:43.887778 25453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e311eb90bad44c80a131b72edcd22b06" candidate_uuid: "3e07d64466934da4bb3c02262b1c2e38" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9868dce521294d0b816bba7648f4ae5a" is_pre_election: true
I20260709 13:23:43.877416 25452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa0da2c845dc4059968332f4db2e0aa6" candidate_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9868dce521294d0b816bba7648f4ae5a" is_pre_election: true
I20260709 13:23:43.889074 25453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434a18a5109e4ea78349ded3968b2709" candidate_uuid: "3e07d64466934da4bb3c02262b1c2e38" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9868dce521294d0b816bba7648f4ae5a" is_pre_election: true
I20260709 13:23:43.889789 25451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba8a000f5abe4158965449995af84b18" candidate_uuid: "3e07d64466934da4bb3c02262b1c2e38" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9868dce521294d0b816bba7648f4ae5a" is_pre_election: true
I20260709 13:23:43.874814 25376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2f14aed05e24a98ad83535029b795dc" candidate_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3e07d64466934da4bb3c02262b1c2e38" is_pre_election: true
I20260709 13:23:43.893148 25303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "beacc815ef954fb692e90544bfa93f05" candidate_uuid: "43999d813e564dc688fad116090c95ca" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" is_pre_election: true
W20260709 13:23:43.893337 25424 consensus_peers.cc:597] T d053b9ae38de4173a0aded4da494bd98 P 9868dce521294d0b816bba7648f4ae5a -> Peer 43999d813e564dc688fad116090c95ca (127.14.177.129:38155): Couldn't send request to peer 43999d813e564dc688fad116090c95ca. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:23:43.894652 25308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79d4a12ef58140568c60a24ed4c12f83" candidate_uuid: "43999d813e564dc688fad116090c95ca" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" is_pre_election: true
I20260709 13:23:43.895452 25311 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28c0b6fc9bd64d60828d705c6a9ab1e4" candidate_uuid: "43999d813e564dc688fad116090c95ca" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "233e7d29aac04505bf4e65f0a39a0c4c"
I20260709 13:23:43.895752 25302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cb89d70b4ce4aa8992fb68480bec8a2" candidate_uuid: "43999d813e564dc688fad116090c95ca" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" is_pre_election: true
I20260709 13:23:43.900116 25308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "579da2d6aabb4ae9a2b1802e1358068b" candidate_uuid: "9868dce521294d0b816bba7648f4ae5a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" is_pre_election: true
I20260709 13:23:43.901702 25305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5df750b84e0487fa530161079deb6c2" candidate_uuid: "9868dce521294d0b816bba7648f4ae5a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" is_pre_election: true
I20260709 13:23:43.903043 25305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a3317f4c3514df1b2bb8e44a86be92f" candidate_uuid: "9868dce521294d0b816bba7648f4ae5a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" is_pre_election: true
I20260709 13:23:43.904304 25305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "441d780c3945463f92f31170f31e37a8" candidate_uuid: "9868dce521294d0b816bba7648f4ae5a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" is_pre_election: true
I20260709 13:23:43.905467 25305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba8a000f5abe4158965449995af84b18" candidate_uuid: "9868dce521294d0b816bba7648f4ae5a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" is_pre_election: true
I20260709 13:23:43.906188 25311 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa0da2c845dc4059968332f4db2e0aa6" candidate_uuid: "9868dce521294d0b816bba7648f4ae5a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" is_pre_election: true
I20260709 13:23:43.797748 26059 raft_consensus.cc:515] T ce9c60e3922c4219b6e4453fe0259b6a P 233e7d29aac04505bf4e65f0a39a0c4c [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } }
I20260709 13:23:43.890666 26066 raft_consensus.cc:515] T b155b866f10547fd8239579a81ecf5f9 P 3e07d64466934da4bb3c02262b1c2e38 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } }
I20260709 13:23:43.917665 25307 raft_consensus.cc:1240] T ce9c60e3922c4219b6e4453fe0259b6a P 233e7d29aac04505bf4e65f0a39a0c4c [term 2 FOLLOWER]: Rejecting Update request from peer 43999d813e564dc688fad116090c95ca for earlier term 1. Current term is 2. Ops: []
I20260709 13:23:43.913156 26042 raft_consensus.cc:515] T 72b56266e8a54776b9113c7a786e1952 P 3e07d64466934da4bb3c02262b1c2e38 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } }
I20260709 13:23:43.914134 25378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55e8ed3aee404ce698d0626a87bd03aa" candidate_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3e07d64466934da4bb3c02262b1c2e38" is_pre_election: true
I20260709 13:23:43.919067 26066 leader_election.cc:290] T b155b866f10547fd8239579a81ecf5f9 P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 2 election: Requested vote from peers 43999d813e564dc688fad116090c95ca (127.14.177.129:38155), 233e7d29aac04505bf4e65f0a39a0c4c (127.14.177.130:42957)
I20260709 13:23:43.916277 26058 raft_consensus.cc:515] T cfad575709c943cba40ccab278350770 P 3e07d64466934da4bb3c02262b1c2e38 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } }
I20260709 13:23:43.911767 25382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03e9bfbd2f774476a4a377367796a2e8" candidate_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3e07d64466934da4bb3c02262b1c2e38" is_pre_election: true
I20260709 13:23:43.911947 25425 leader_election.cc:304] T ad32097cd4dc48b9a69595f1651abd07 P 9868dce521294d0b816bba7648f4ae5a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9868dce521294d0b816bba7648f4ae5a; no voters: 233e7d29aac04505bf4e65f0a39a0c4c, 3e07d64466934da4bb3c02262b1c2e38
I20260709 13:23:43.911863 25302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe5a944a342411ea923eb39071ac770" candidate_uuid: "9868dce521294d0b816bba7648f4ae5a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" is_pre_election: true
I20260709 13:23:43.919754 26042 leader_election.cc:290] T 72b56266e8a54776b9113c7a786e1952 P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 2 election: Requested vote from peers 43999d813e564dc688fad116090c95ca (127.14.177.129:38155), 233e7d29aac04505bf4e65f0a39a0c4c (127.14.177.130:42957)
I20260709 13:23:43.866925 25310 raft_consensus.cc:2468] T 9e5d70664720424d951f269d9d7f4346 P 233e7d29aac04505bf4e65f0a39a0c4c [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 43999d813e564dc688fad116090c95ca in term 1.
I20260709 13:23:43.928987 25378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79d4a12ef58140568c60a24ed4c12f83" candidate_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3e07d64466934da4bb3c02262b1c2e38" is_pre_election: true
I20260709 13:23:43.932643 25376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87085ac2575e4b6a86dd1fc21e6f718b" candidate_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3e07d64466934da4bb3c02262b1c2e38" is_pre_election: true
I20260709 13:23:43.925664 25451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "441d780c3945463f92f31170f31e37a8" candidate_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9868dce521294d0b816bba7648f4ae5a" is_pre_election: true
I20260709 13:23:43.925974 26069 raft_consensus.cc:2749] T ad32097cd4dc48b9a69595f1651abd07 P 9868dce521294d0b816bba7648f4ae5a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:23:43.929525 25303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c4ab1978a47481c8576d466bacf49d2" candidate_uuid: "9868dce521294d0b816bba7648f4ae5a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" is_pre_election: true
I20260709 13:23:43.936242 25455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e5d70664720424d951f269d9d7f4346" candidate_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9868dce521294d0b816bba7648f4ae5a" is_pre_election: true
I20260709 13:23:43.926808 26059 leader_election.cc:290] T ce9c60e3922c4219b6e4453fe0259b6a P 233e7d29aac04505bf4e65f0a39a0c4c [CANDIDATE]: Term 2 election: Requested vote from peers 9868dce521294d0b816bba7648f4ae5a (127.14.177.132:33683), 43999d813e564dc688fad116090c95ca (127.14.177.129:38155)
I20260709 13:23:43.938822 25451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfad575709c943cba40ccab278350770" candidate_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "9868dce521294d0b816bba7648f4ae5a" is_pre_election: true
I20260709 13:23:43.942330 25455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aad1f7937f854cc1a726223269da5d5a" candidate_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9868dce521294d0b816bba7648f4ae5a" is_pre_election: true
I20260709 13:23:43.943874 25455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad32097cd4dc48b9a69595f1651abd07" candidate_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9868dce521294d0b816bba7648f4ae5a"
W20260709 13:23:43.945247 25199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.132:33683, user_credentials={real_user=slave}} blocked reactor thread for 69846.1us
I20260709 13:23:43.946245 25377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aad1f7937f854cc1a726223269da5d5a" candidate_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3e07d64466934da4bb3c02262b1c2e38" is_pre_election: true
I20260709 13:23:43.947141 25199 leader_election.cc:304] T 1f15ec62698c4a8cbd84f2b3d516d2ea P 43999d813e564dc688fad116090c95ca [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 43999d813e564dc688fad116090c95ca; no voters: 3e07d64466934da4bb3c02262b1c2e38, 9868dce521294d0b816bba7648f4ae5a
I20260709 13:23:43.948014 25457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ef061028f38450e83324ac78cdaa14c" candidate_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9868dce521294d0b816bba7648f4ae5a" is_pre_election: true
I20260709 13:23:43.950655 25310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b155b866f10547fd8239579a81ecf5f9" candidate_uuid: "3e07d64466934da4bb3c02262b1c2e38" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "233e7d29aac04505bf4e65f0a39a0c4c"
I20260709 13:23:43.951334 26019 raft_consensus.cc:3060] T 1f15ec62698c4a8cbd84f2b3d516d2ea P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:23:43.949085 25199 leader_election.cc:304] T 0ed5edbe32044d1c83b0590dbb899801 P 43999d813e564dc688fad116090c95ca [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 43999d813e564dc688fad116090c95ca, 9868dce521294d0b816bba7648f4ae5a; no voters: 3e07d64466934da4bb3c02262b1c2e38
I20260709 13:23:43.952382 25308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72b56266e8a54776b9113c7a786e1952" candidate_uuid: "3e07d64466934da4bb3c02262b1c2e38" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "233e7d29aac04505bf4e65f0a39a0c4c"
I20260709 13:23:43.955958 25977 raft_consensus.cc:2804] T 0ed5edbe32044d1c83b0590dbb899801 P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:23:43.964771 25306 raft_consensus.cc:1240] T ad32097cd4dc48b9a69595f1651abd07 P 233e7d29aac04505bf4e65f0a39a0c4c [term 2 FOLLOWER]: Rejecting Update request from peer 3e07d64466934da4bb3c02262b1c2e38 for earlier term 1. Current term is 2. Ops: []
I20260709 13:23:43.965945 25977 raft_consensus.cc:493] T 0ed5edbe32044d1c83b0590dbb899801 P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Starting leader election (detected failure of leader 3e07d64466934da4bb3c02262b1c2e38)
I20260709 13:23:43.981688 25458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9d550d881b04735bbf89751952c09ec" candidate_uuid: "3e07d64466934da4bb3c02262b1c2e38" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9868dce521294d0b816bba7648f4ae5a" is_pre_election: true
I20260709 13:23:43.983464 25451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce9c60e3922c4219b6e4453fe0259b6a" candidate_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9868dce521294d0b816bba7648f4ae5a"
I20260709 13:23:43.984843 26058 leader_election.cc:290] T cfad575709c943cba40ccab278350770 P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 3 election: Requested vote from peers 233e7d29aac04505bf4e65f0a39a0c4c (127.14.177.130:42957), 9868dce521294d0b816bba7648f4ae5a (127.14.177.132:33683)
I20260709 13:23:43.984948 25450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfad575709c943cba40ccab278350770" candidate_uuid: "3e07d64466934da4bb3c02262b1c2e38" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "9868dce521294d0b816bba7648f4ae5a" is_pre_election: true
I20260709 13:23:43.986476 25977 raft_consensus.cc:3060] T 0ed5edbe32044d1c83b0590dbb899801 P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:23:43.986932 26057 raft_consensus.cc:515] T 89a26dd3ac414f6081b98fa32c2f346f P 3e07d64466934da4bb3c02262b1c2e38 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } }
I20260709 13:23:43.986972 25456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5df750b84e0487fa530161079deb6c2" candidate_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9868dce521294d0b816bba7648f4ae5a" is_pre_election: true
I20260709 13:23:43.989182 25348 leader_election.cc:304] T bbab48ba177049dcba6de106fd889cfb P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3e07d64466934da4bb3c02262b1c2e38; no voters: 233e7d29aac04505bf4e65f0a39a0c4c, 9868dce521294d0b816bba7648f4ae5a
I20260709 13:23:43.989964 25457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89a26dd3ac414f6081b98fa32c2f346f" candidate_uuid: "3e07d64466934da4bb3c02262b1c2e38" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9868dce521294d0b816bba7648f4ae5a" is_pre_election: true
I20260709 13:23:43.992024 25385 raft_consensus.cc:2393] T 89a26dd3ac414f6081b98fa32c2f346f P 3e07d64466934da4bb3c02262b1c2e38 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 233e7d29aac04505bf4e65f0a39a0c4c in current term 2: Already voted for candidate 3e07d64466934da4bb3c02262b1c2e38 in this term.
I20260709 13:23:43.992215 25303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfad575709c943cba40ccab278350770" candidate_uuid: "3e07d64466934da4bb3c02262b1c2e38" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "233e7d29aac04505bf4e65f0a39a0c4c"
I20260709 13:23:43.993458 25308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89a26dd3ac414f6081b98fa32c2f346f" candidate_uuid: "3e07d64466934da4bb3c02262b1c2e38" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "233e7d29aac04505bf4e65f0a39a0c4c"
I20260709 13:23:43.994464 25382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13bb3a23315a41c7a40e10b954b15c8e" candidate_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3e07d64466934da4bb3c02262b1c2e38" is_pre_election: true
I20260709 13:23:43.968792 25376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "beacc815ef954fb692e90544bfa93f05" candidate_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3e07d64466934da4bb3c02262b1c2e38" is_pre_election: true
I20260709 13:23:43.998584 25348 leader_election.cc:304] T 5c4ab1978a47481c8576d466bacf49d2 P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3e07d64466934da4bb3c02262b1c2e38; no voters: 233e7d29aac04505bf4e65f0a39a0c4c, 9868dce521294d0b816bba7648f4ae5a
W20260709 13:23:43.998777 25276 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:38155, user_credentials={real_user=slave}} blocked reactor thread for 77343.2us
I20260709 13:23:44.001777 25385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434a54fcee7d4aeeab42f739855c0e8a" candidate_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3e07d64466934da4bb3c02262b1c2e38" is_pre_election: true
I20260709 13:23:44.001950 25348 leader_election.cc:304] T e311eb90bad44c80a131b72edcd22b06 P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3e07d64466934da4bb3c02262b1c2e38; no voters: 233e7d29aac04505bf4e65f0a39a0c4c, 9868dce521294d0b816bba7648f4ae5a
I20260709 13:23:44.003366 25348 leader_election.cc:304] T 434a18a5109e4ea78349ded3968b2709 P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3e07d64466934da4bb3c02262b1c2e38; no voters: 233e7d29aac04505bf4e65f0a39a0c4c, 9868dce521294d0b816bba7648f4ae5a
I20260709 13:23:44.003995 25202 leader_election.cc:304] T ce98a3f3b3eb4c63bcaa355ec7cd64a4 P 43999d813e564dc688fad116090c95ca [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 43999d813e564dc688fad116090c95ca; no voters: 233e7d29aac04505bf4e65f0a39a0c4c, 3e07d64466934da4bb3c02262b1c2e38
I20260709 13:23:44.004683 26017 raft_consensus.cc:2749] T ce98a3f3b3eb4c63bcaa355ec7cd64a4 P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:23:44.005326 26066 raft_consensus.cc:2749] T bbab48ba177049dcba6de106fd889cfb P 3e07d64466934da4bb3c02262b1c2e38 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:23:44.005345 26046 raft_consensus.cc:2749] T 5c4ab1978a47481c8576d466bacf49d2 P 3e07d64466934da4bb3c02262b1c2e38 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:23:44.005550 25378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b155b866f10547fd8239579a81ecf5f9" candidate_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3e07d64466934da4bb3c02262b1c2e38" is_pre_election: true
I20260709 13:23:44.006685 26058 raft_consensus.cc:2749] T e311eb90bad44c80a131b72edcd22b06 P 3e07d64466934da4bb3c02262b1c2e38 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:23:44.000488 26042 consensus_queue.cc:1059] T 1f15ec62698c4a8cbd84f2b3d516d2ea P 3e07d64466934da4bb3c02262b1c2e38 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.011s
I20260709 13:23:44.008824 26042 raft_consensus.cc:3055] T 1f15ec62698c4a8cbd84f2b3d516d2ea P 3e07d64466934da4bb3c02262b1c2e38 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:23:44.009212 26042 raft_consensus.cc:740] T 1f15ec62698c4a8cbd84f2b3d516d2ea P 3e07d64466934da4bb3c02262b1c2e38 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3e07d64466934da4bb3c02262b1c2e38, State: Running, Role: LEADER
I20260709 13:23:44.001009 26026 raft_consensus.cc:515] T 467a6b42ca6548269c12dddde696f7af P 3e07d64466934da4bb3c02262b1c2e38 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } }
I20260709 13:23:44.010263 25380 raft_consensus.cc:2393] T 467a6b42ca6548269c12dddde696f7af P 3e07d64466934da4bb3c02262b1c2e38 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 233e7d29aac04505bf4e65f0a39a0c4c in current term 2: Already voted for candidate 3e07d64466934da4bb3c02262b1c2e38 in this term.
I20260709 13:23:43.991478 26057 leader_election.cc:290] T 89a26dd3ac414f6081b98fa32c2f346f P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 2 election: Requested vote from peers 233e7d29aac04505bf4e65f0a39a0c4c (127.14.177.130:42957), 9868dce521294d0b816bba7648f4ae5a (127.14.177.132:33683)
I20260709 13:23:44.014544 25376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad32097cd4dc48b9a69595f1651abd07" candidate_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3e07d64466934da4bb3c02262b1c2e38"
I20260709 13:23:44.015300 25379 raft_consensus.cc:1240] T 467a6b42ca6548269c12dddde696f7af P 3e07d64466934da4bb3c02262b1c2e38 [term 2 FOLLOWER]: Rejecting Update request from peer 9868dce521294d0b816bba7648f4ae5a for earlier term 1. Current term is 2. Ops: []
I20260709 13:23:44.015967 25382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ffa42c3b2ff48bc87310e9ab5d218ce" candidate_uuid: "9868dce521294d0b816bba7648f4ae5a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3e07d64466934da4bb3c02262b1c2e38" is_pre_election: true
I20260709 13:23:44.016060 25383 raft_consensus.cc:1240] T 89a26dd3ac414f6081b98fa32c2f346f P 3e07d64466934da4bb3c02262b1c2e38 [term 2 FOLLOWER]: Rejecting Update request from peer 9868dce521294d0b816bba7648f4ae5a for earlier term 1. Current term is 2. Ops: []
I20260709 13:23:44.016258 25384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c4ab1978a47481c8576d466bacf49d2" candidate_uuid: "9868dce521294d0b816bba7648f4ae5a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3e07d64466934da4bb3c02262b1c2e38" is_pre_election: true
I20260709 13:23:44.017030 26026 leader_election.cc:290] T 467a6b42ca6548269c12dddde696f7af P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 2 election: Requested vote from peers 233e7d29aac04505bf4e65f0a39a0c4c (127.14.177.130:42957), 9868dce521294d0b816bba7648f4ae5a (127.14.177.132:33683)
I20260709 13:23:44.017649 25385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3fe5a944a342411ea923eb39071ac770" candidate_uuid: "9868dce521294d0b816bba7648f4ae5a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3e07d64466934da4bb3c02262b1c2e38" is_pre_election: true
I20260709 13:23:44.018932 25380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfad575709c943cba40ccab278350770" candidate_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "3e07d64466934da4bb3c02262b1c2e38" is_pre_election: true
I20260709 13:23:44.036477 25310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "467a6b42ca6548269c12dddde696f7af" candidate_uuid: "3e07d64466934da4bb3c02262b1c2e38" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "233e7d29aac04505bf4e65f0a39a0c4c"
I20260709 13:23:44.012112 25377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15820fa8a7d4e1fb6ad23a64ca22f90" candidate_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3e07d64466934da4bb3c02262b1c2e38" is_pre_election: true
I20260709 13:23:44.019819 26058 raft_consensus.cc:2749] T 434a18a5109e4ea78349ded3968b2709 P 3e07d64466934da4bb3c02262b1c2e38 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:23:44.040529 26019 raft_consensus.cc:2749] T 1f15ec62698c4a8cbd84f2b3d516d2ea P 43999d813e564dc688fad116090c95ca [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:23:44.041325 25379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91ac744179de48e5bb924b6dfd9f9c9b" candidate_uuid: "9868dce521294d0b816bba7648f4ae5a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3e07d64466934da4bb3c02262b1c2e38" is_pre_election: true
I20260709 13:23:44.041939 25383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f15ec62698c4a8cbd84f2b3d516d2ea" candidate_uuid: "9868dce521294d0b816bba7648f4ae5a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3e07d64466934da4bb3c02262b1c2e38"
I20260709 13:23:44.042657 25379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c7e7acbb09f4d49828fa652ca6abaac" candidate_uuid: "9868dce521294d0b816bba7648f4ae5a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3e07d64466934da4bb3c02262b1c2e38"
I20260709 13:23:44.043771 25274 leader_election.cc:304] T a820954e07a94582ac0613342fcffde5 P 233e7d29aac04505bf4e65f0a39a0c4c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 233e7d29aac04505bf4e65f0a39a0c4c; no voters: 3e07d64466934da4bb3c02262b1c2e38, 9868dce521294d0b816bba7648f4ae5a
I20260709 13:23:44.044076 25379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "821a5866429f4173820fb64336abee02" candidate_uuid: "9868dce521294d0b816bba7648f4ae5a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3e07d64466934da4bb3c02262b1c2e38" is_pre_election: true
I20260709 13:23:44.044551 26072 raft_consensus.cc:2749] T a820954e07a94582ac0613342fcffde5 P 233e7d29aac04505bf4e65f0a39a0c4c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:23:44.046159 25384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e71b6f5c69b4b35a8ac51c22c7d5f0d" candidate_uuid: "9868dce521294d0b816bba7648f4ae5a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3e07d64466934da4bb3c02262b1c2e38" is_pre_election: true
I20260709 13:23:44.047034 25422 leader_election.cc:304] T 3fe5a944a342411ea923eb39071ac770 P 9868dce521294d0b816bba7648f4ae5a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9868dce521294d0b816bba7648f4ae5a; no voters: 233e7d29aac04505bf4e65f0a39a0c4c, 3e07d64466934da4bb3c02262b1c2e38
I20260709 13:23:44.047473 25384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d837697cd0f402cacf9db89a5d92652" candidate_uuid: "9868dce521294d0b816bba7648f4ae5a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3e07d64466934da4bb3c02262b1c2e38" is_pre_election: true
I20260709 13:23:44.047678 25274 leader_election.cc:304] T 55e8ed3aee404ce698d0626a87bd03aa P 233e7d29aac04505bf4e65f0a39a0c4c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 233e7d29aac04505bf4e65f0a39a0c4c; no voters: 3e07d64466934da4bb3c02262b1c2e38, 9868dce521294d0b816bba7648f4ae5a
I20260709 13:23:44.025420 25348 leader_election.cc:304] T ba8a000f5abe4158965449995af84b18 P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3e07d64466934da4bb3c02262b1c2e38; no voters: 233e7d29aac04505bf4e65f0a39a0c4c, 9868dce521294d0b816bba7648f4ae5a
W20260709 13:23:44.024811 25199 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.132:33683, user_credentials={real_user=slave}} blocked reactor thread for 69361.2us
I20260709 13:23:44.051362 26050 consensus_queue.cc:1059] T 467a6b42ca6548269c12dddde696f7af P 9868dce521294d0b816bba7648f4ae5a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:23:44.051986 25380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0a0e4d32ce943fbb773bb840863647b" candidate_uuid: "9868dce521294d0b816bba7648f4ae5a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3e07d64466934da4bb3c02262b1c2e38" is_pre_election: true
I20260709 13:23:44.052380 26050 raft_consensus.cc:2749] T 3fe5a944a342411ea923eb39071ac770 P 9868dce521294d0b816bba7648f4ae5a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:23:44.052589 26057 raft_consensus.cc:2749] T ba8a000f5abe4158965449995af84b18 P 3e07d64466934da4bb3c02262b1c2e38 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:23:44.024539 26042 consensus_queue.cc:260] T 1f15ec62698c4a8cbd84f2b3d516d2ea P 3e07d64466934da4bb3c02262b1c2e38 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } }
I20260709 13:23:44.027989 26046 consensus_queue.cc:1059] T ad32097cd4dc48b9a69595f1651abd07 P 3e07d64466934da4bb3c02262b1c2e38 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:23:44.052721 25199 leader_election.cc:304] T b85a238bbc3b4621881603a534c7bd9b P 43999d813e564dc688fad116090c95ca [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 43999d813e564dc688fad116090c95ca; no voters: 233e7d29aac04505bf4e65f0a39a0c4c, 9868dce521294d0b816bba7648f4ae5a
I20260709 13:23:44.021490 25378 raft_consensus.cc:1240] T cfad575709c943cba40ccab278350770 P 3e07d64466934da4bb3c02262b1c2e38 [term 3 FOLLOWER]: Rejecting Update request from peer 9868dce521294d0b816bba7648f4ae5a for earlier term 2. Current term is 3. Ops: []
W20260709 13:23:44.035056 25424 consensus_peers.cc:597] T 0c4f6b7401db47d59a4f34ed0e8b5350 P 9868dce521294d0b816bba7648f4ae5a -> Peer 43999d813e564dc688fad116090c95ca (127.14.177.129:38155): Couldn't send request to peer 43999d813e564dc688fad116090c95ca. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:23:44.058125 25199 leader_election.cc:304] T b7a891bfc89e4e259f53da6cb16f4989 P 43999d813e564dc688fad116090c95ca [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 43999d813e564dc688fad116090c95ca, 9868dce521294d0b816bba7648f4ae5a; no voters: 3e07d64466934da4bb3c02262b1c2e38
W20260709 13:23:44.058516 25424 leader_election.cc:336] T c902bbc7ec0848ba82a04870eab1e7f5 P 9868dce521294d0b816bba7648f4ae5a [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 43999d813e564dc688fad116090c95ca (127.14.177.129:38155): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:23:44.059077 26024 raft_consensus.cc:2749] T b85a238bbc3b4621881603a534c7bd9b P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:23:44.048941 25422 leader_election.cc:304] T 5c4ab1978a47481c8576d466bacf49d2 P 9868dce521294d0b816bba7648f4ae5a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9868dce521294d0b816bba7648f4ae5a; no voters: 233e7d29aac04505bf4e65f0a39a0c4c, 3e07d64466934da4bb3c02262b1c2e38
I20260709 13:23:44.059332 25377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9d550d881b04735bbf89751952c09ec" candidate_uuid: "9868dce521294d0b816bba7648f4ae5a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3e07d64466934da4bb3c02262b1c2e38" is_pre_election: true
I20260709 13:23:44.059985 26072 raft_consensus.cc:2749] T 55e8ed3aee404ce698d0626a87bd03aa P 233e7d29aac04505bf4e65f0a39a0c4c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:23:44.060182 25458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfad575709c943cba40ccab278350770" candidate_uuid: "3e07d64466934da4bb3c02262b1c2e38" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "9868dce521294d0b816bba7648f4ae5a"
I20260709 13:23:44.059532 25199 leader_election.cc:304] T 91ac744179de48e5bb924b6dfd9f9c9b P 43999d813e564dc688fad116090c95ca [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 43999d813e564dc688fad116090c95ca, 9868dce521294d0b816bba7648f4ae5a; no voters: 3e07d64466934da4bb3c02262b1c2e38
I20260709 13:23:44.062481 25274 leader_election.cc:304] T aad1f7937f854cc1a726223269da5d5a P 233e7d29aac04505bf4e65f0a39a0c4c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 233e7d29aac04505bf4e65f0a39a0c4c; no voters: 3e07d64466934da4bb3c02262b1c2e38, 9868dce521294d0b816bba7648f4ae5a
I20260709 13:23:44.062530 25385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1ac2f4d2fe2424bb470df42199219e2" candidate_uuid: "9868dce521294d0b816bba7648f4ae5a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3e07d64466934da4bb3c02262b1c2e38" is_pre_election: true
I20260709 13:23:44.063654 25274 leader_election.cc:304] T 89a26dd3ac414f6081b98fa32c2f346f P 233e7d29aac04505bf4e65f0a39a0c4c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 233e7d29aac04505bf4e65f0a39a0c4c; no voters: 3e07d64466934da4bb3c02262b1c2e38, 9868dce521294d0b816bba7648f4ae5a
W20260709 13:23:44.048125 25480 debug-util.cc:398] Leaking SignalData structure 0x7b080017a100 after lost signal to thread 15049
I20260709 13:23:44.029887 25202 leader_election.cc:304] T 0c4f6b7401db47d59a4f34ed0e8b5350 P 43999d813e564dc688fad116090c95ca [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 43999d813e564dc688fad116090c95ca; no voters: 233e7d29aac04505bf4e65f0a39a0c4c, 9868dce521294d0b816bba7648f4ae5a
I20260709 13:23:44.064780 25348 leader_election.cc:304] T cfad575709c943cba40ccab278350770 P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3e07d64466934da4bb3c02262b1c2e38; no voters: 233e7d29aac04505bf4e65f0a39a0c4c, 9868dce521294d0b816bba7648f4ae5a
I20260709 13:23:44.066994 26053 raft_consensus.cc:2749] T 5c4ab1978a47481c8576d466bacf49d2 P 9868dce521294d0b816bba7648f4ae5a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:23:44.061645 25381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba8a000f5abe4158965449995af84b18" candidate_uuid: "9868dce521294d0b816bba7648f4ae5a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3e07d64466934da4bb3c02262b1c2e38" is_pre_election: true
I20260709 13:23:44.067374 26056 consensus_queue.cc:1059] T 89a26dd3ac414f6081b98fa32c2f346f P 9868dce521294d0b816bba7648f4ae5a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:23:44.068536 26053 raft_consensus.cc:3055] T 89a26dd3ac414f6081b98fa32c2f346f P 9868dce521294d0b816bba7648f4ae5a [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:23:44.068941 26053 raft_consensus.cc:740] T 89a26dd3ac414f6081b98fa32c2f346f P 9868dce521294d0b816bba7648f4ae5a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 9868dce521294d0b816bba7648f4ae5a, State: Running, Role: LEADER
I20260709 13:23:44.061908 26042 raft_consensus.cc:3060] T 1f15ec62698c4a8cbd84f2b3d516d2ea P 3e07d64466934da4bb3c02262b1c2e38 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:23:44.069782 25480 debug-util.cc:398] Leaking SignalData structure 0x7b080010bf80 after lost signal to thread 25168
I20260709 13:23:44.070312 25458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89a26dd3ac414f6081b98fa32c2f346f" candidate_uuid: "3e07d64466934da4bb3c02262b1c2e38" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9868dce521294d0b816bba7648f4ae5a"
I20260709 13:23:44.070634 25454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "467a6b42ca6548269c12dddde696f7af" candidate_uuid: "3e07d64466934da4bb3c02262b1c2e38" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9868dce521294d0b816bba7648f4ae5a"
I20260709 13:23:44.071225 26059 raft_consensus.cc:2749] T aad1f7937f854cc1a726223269da5d5a P 233e7d29aac04505bf4e65f0a39a0c4c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:23:44.071553 25480 debug-util.cc:398] Leaking SignalData structure 0x7b080039b960 after lost signal to thread 25260
I20260709 13:23:44.067056 26057 raft_consensus.cc:2749] T cfad575709c943cba40ccab278350770 P 3e07d64466934da4bb3c02262b1c2e38 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
W20260709 13:23:44.072942 25480 debug-util.cc:398] Leaking SignalData structure 0x7b080034dd60 after lost signal to thread 25335
I20260709 13:23:44.073815 26058 raft_consensus.cc:3055] T ad32097cd4dc48b9a69595f1651abd07 P 3e07d64466934da4bb3c02262b1c2e38 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:23:44.074256 26058 raft_consensus.cc:740] T ad32097cd4dc48b9a69595f1651abd07 P 3e07d64466934da4bb3c02262b1c2e38 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3e07d64466934da4bb3c02262b1c2e38, State: Running, Role: LEADER
I20260709 13:23:44.053153 26050 raft_consensus.cc:3055] T 467a6b42ca6548269c12dddde696f7af P 9868dce521294d0b816bba7648f4ae5a [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:23:44.075111 26050 raft_consensus.cc:740] T 467a6b42ca6548269c12dddde696f7af P 9868dce521294d0b816bba7648f4ae5a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 9868dce521294d0b816bba7648f4ae5a, State: Running, Role: LEADER
W20260709 13:23:44.076907 25202 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:42957, user_credentials={real_user=slave}} blocked reactor thread for 47096.7us
I20260709 13:23:44.078644 25274 leader_election.cc:304] T 467a6b42ca6548269c12dddde696f7af P 233e7d29aac04505bf4e65f0a39a0c4c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 233e7d29aac04505bf4e65f0a39a0c4c; no voters: 3e07d64466934da4bb3c02262b1c2e38, 9868dce521294d0b816bba7648f4ae5a
I20260709 13:23:44.080971 25274 leader_election.cc:304] T cfad575709c943cba40ccab278350770 P 233e7d29aac04505bf4e65f0a39a0c4c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 233e7d29aac04505bf4e65f0a39a0c4c; no voters: 3e07d64466934da4bb3c02262b1c2e38, 9868dce521294d0b816bba7648f4ae5a
I20260709 13:23:44.083617 25422 leader_election.cc:304] T ba8a000f5abe4158965449995af84b18 P 9868dce521294d0b816bba7648f4ae5a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9868dce521294d0b816bba7648f4ae5a; no voters: 233e7d29aac04505bf4e65f0a39a0c4c, 3e07d64466934da4bb3c02262b1c2e38
W20260709 13:23:44.084259 25424 leader_election.cc:336] T 5c7e7acbb09f4d49828fa652ca6abaac P 9868dce521294d0b816bba7648f4ae5a [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 43999d813e564dc688fad116090c95ca (127.14.177.129:38155): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:23:44.085122 25424 leader_election.cc:336] T 8ef061028f38450e83324ac78cdaa14c P 9868dce521294d0b816bba7648f4ae5a [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 43999d813e564dc688fad116090c95ca (127.14.177.129:38155): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:23:44.085529 25424 leader_election.cc:304] T 8ef061028f38450e83324ac78cdaa14c P 9868dce521294d0b816bba7648f4ae5a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9868dce521294d0b816bba7648f4ae5a; no voters: 233e7d29aac04505bf4e65f0a39a0c4c, 43999d813e564dc688fad116090c95ca
I20260709 13:23:44.084808 25274 leader_election.cc:304] T ad32097cd4dc48b9a69595f1651abd07 P 233e7d29aac04505bf4e65f0a39a0c4c [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 233e7d29aac04505bf4e65f0a39a0c4c; no voters: 3e07d64466934da4bb3c02262b1c2e38, 9868dce521294d0b816bba7648f4ae5a
I20260709 13:23:44.079576 26059 raft_consensus.cc:3060] T 89a26dd3ac414f6081b98fa32c2f346f P 233e7d29aac04505bf4e65f0a39a0c4c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:23:44.086886 26043 raft_consensus.cc:2749] T cfad575709c943cba40ccab278350770 P 233e7d29aac04505bf4e65f0a39a0c4c [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:23:44.087090 25480 debug-util.cc:398] Leaking SignalData structure 0x7b08000c9260 after lost signal to thread 25409
I20260709 13:23:44.054677 25376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c902bbc7ec0848ba82a04870eab1e7f5" candidate_uuid: "9868dce521294d0b816bba7648f4ae5a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3e07d64466934da4bb3c02262b1c2e38"
I20260709 13:23:44.082829 26062 raft_consensus.cc:3060] T 467a6b42ca6548269c12dddde696f7af P 233e7d29aac04505bf4e65f0a39a0c4c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:23:44.078646 25955 raft_consensus.cc:2749] T 0c4f6b7401db47d59a4f34ed0e8b5350 P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:23:44.090942 25348 leader_election.cc:304] T 467a6b42ca6548269c12dddde696f7af P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3e07d64466934da4bb3c02262b1c2e38; no voters: 233e7d29aac04505bf4e65f0a39a0c4c, 9868dce521294d0b816bba7648f4ae5a
W20260709 13:23:44.091434 25424 leader_election.cc:336] T 1550ed2d2a764c08adffae679599ba41 P 9868dce521294d0b816bba7648f4ae5a [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 43999d813e564dc688fad116090c95ca (127.14.177.129:38155): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:23:44.088483 25480 debug-util.cc:398] Leaking SignalData structure 0x7b08002d44c0 after lost signal to thread 25483
I20260709 13:23:44.079079 26019 raft_consensus.cc:2804] T b7a891bfc89e4e259f53da6cb16f4989 P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:23:44.088088 26065 consensus_queue.cc:1059] T cfad575709c943cba40ccab278350770 P 9868dce521294d0b816bba7648f4ae5a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 }, Status: INVALID_TERM, Last received: 2.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.008s
I20260709 13:23:44.098685 25424 leader_election.cc:304] T 1550ed2d2a764c08adffae679599ba41 P 9868dce521294d0b816bba7648f4ae5a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9868dce521294d0b816bba7648f4ae5a; no voters: 3e07d64466934da4bb3c02262b1c2e38, 43999d813e564dc688fad116090c95ca
I20260709 13:23:44.053867 25382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7a891bfc89e4e259f53da6cb16f4989" candidate_uuid: "9868dce521294d0b816bba7648f4ae5a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3e07d64466934da4bb3c02262b1c2e38" is_pre_election: true
I20260709 13:23:44.088969 26054 raft_consensus.cc:2749] T ba8a000f5abe4158965449995af84b18 P 9868dce521294d0b816bba7648f4ae5a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:23:44.096386 25384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28c0b6fc9bd64d60828d705c6a9ab1e4" candidate_uuid: "43999d813e564dc688fad116090c95ca" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3e07d64466934da4bb3c02262b1c2e38"
I20260709 13:23:44.089612 26022 raft_consensus.cc:2749] T 8ef061028f38450e83324ac78cdaa14c P 9868dce521294d0b816bba7648f4ae5a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:23:44.112955 25424 leader_election.cc:336] T a5df750b84e0487fa530161079deb6c2 P 9868dce521294d0b816bba7648f4ae5a [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 43999d813e564dc688fad116090c95ca (127.14.177.129:38155): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:23:44.114074 25424 leader_election.cc:304] T a5df750b84e0487fa530161079deb6c2 P 9868dce521294d0b816bba7648f4ae5a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9868dce521294d0b816bba7648f4ae5a; no voters: 233e7d29aac04505bf4e65f0a39a0c4c, 43999d813e564dc688fad116090c95ca
I20260709 13:23:44.093338 26027 raft_consensus.cc:2749] T ad32097cd4dc48b9a69595f1651abd07 P 233e7d29aac04505bf4e65f0a39a0c4c [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:23:44.098883 26024 raft_consensus.cc:2804] T 91ac744179de48e5bb924b6dfd9f9c9b P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260709 13:23:44.118777 25424 leader_election.cc:336] T fa0da2c845dc4059968332f4db2e0aa6 P 9868dce521294d0b816bba7648f4ae5a [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 43999d813e564dc688fad116090c95ca (127.14.177.129:38155): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:23:44.127934 25424 leader_election.cc:304] T fa0da2c845dc4059968332f4db2e0aa6 P 9868dce521294d0b816bba7648f4ae5a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9868dce521294d0b816bba7648f4ae5a; no voters: 233e7d29aac04505bf4e65f0a39a0c4c, 43999d813e564dc688fad116090c95ca
I20260709 13:23:44.105816 26057 raft_consensus.cc:2749] T 467a6b42ca6548269c12dddde696f7af P 3e07d64466934da4bb3c02262b1c2e38 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:23:44.130151 25202 leader_election.cc:304] T 0d29b499fa4c41f1b974782cd0c9c1c2 P 43999d813e564dc688fad116090c95ca [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 43999d813e564dc688fad116090c95ca; no voters: 233e7d29aac04505bf4e65f0a39a0c4c, 3e07d64466934da4bb3c02262b1c2e38
I20260709 13:23:44.101719 26019 raft_consensus.cc:493] T b7a891bfc89e4e259f53da6cb16f4989 P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Starting leader election (detected failure of leader 3e07d64466934da4bb3c02262b1c2e38)
I20260709 13:23:44.120150 25977 raft_consensus.cc:515] T 0ed5edbe32044d1c83b0590dbb899801 P 43999d813e564dc688fad116090c95ca [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } }
I20260709 13:23:44.135696 26078 raft_consensus.cc:2749] T 0d29b499fa4c41f1b974782cd0c9c1c2 P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:23:44.075886 26058 consensus_queue.cc:260] T ad32097cd4dc48b9a69595f1651abd07 P 3e07d64466934da4bb3c02262b1c2e38 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } }
I20260709 13:23:44.087769 26050 consensus_queue.cc:260] T 467a6b42ca6548269c12dddde696f7af P 9868dce521294d0b816bba7648f4ae5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } }
W20260709 13:23:44.142311 25424 consensus_peers.cc:597] T 11949f56648c47e0a43c3140af99a304 P 9868dce521294d0b816bba7648f4ae5a -> Peer 43999d813e564dc688fad116090c95ca (127.14.177.129:38155): Couldn't send request to peer 43999d813e564dc688fad116090c95ca. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:23:44.143261 25424 consensus_peers.cc:597] T 280d65245f4f4358b54bef93e18aca92 P 9868dce521294d0b816bba7648f4ae5a -> Peer 43999d813e564dc688fad116090c95ca (127.14.177.129:38155): Couldn't send request to peer 43999d813e564dc688fad116090c95ca. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:23:44.136569 26019 raft_consensus.cc:3060] T b7a891bfc89e4e259f53da6cb16f4989 P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:23:44.069753 26053 consensus_queue.cc:260] T 89a26dd3ac414f6081b98fa32c2f346f P 9868dce521294d0b816bba7648f4ae5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } }
I20260709 13:23:44.098166 26028 consensus_queue.cc:1059] T 5c7e7acbb09f4d49828fa652ca6abaac P 43999d813e564dc688fad116090c95ca [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.035s
I20260709 13:23:44.116586 26030 raft_consensus.cc:2749] T 1550ed2d2a764c08adffae679599ba41 P 9868dce521294d0b816bba7648f4ae5a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:23:44.130321 26023 raft_consensus.cc:2749] T a5df750b84e0487fa530161079deb6c2 P 9868dce521294d0b816bba7648f4ae5a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:23:44.156828 25977 leader_election.cc:290] T 0ed5edbe32044d1c83b0590dbb899801 P 43999d813e564dc688fad116090c95ca [CANDIDATE]: Term 2 election: Requested vote from peers 3e07d64466934da4bb3c02262b1c2e38 (127.14.177.131:41539), 9868dce521294d0b816bba7648f4ae5a (127.14.177.132:33683)
W20260709 13:23:44.172134 25351 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:23:44.144007 25424 consensus_peers.cc:597] T 6242888a4d4146dcacd87d0fd9a832f4 P 9868dce521294d0b816bba7648f4ae5a -> Peer 43999d813e564dc688fad116090c95ca (127.14.177.129:38155): Couldn't send request to peer 43999d813e564dc688fad116090c95ca. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:23:44.127781 26024 raft_consensus.cc:493] T 91ac744179de48e5bb924b6dfd9f9c9b P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Starting leader election (detected failure of leader 3e07d64466934da4bb3c02262b1c2e38)
I20260709 13:23:44.137439 25380 raft_consensus.cc:1240] T 72b56266e8a54776b9113c7a786e1952 P 3e07d64466934da4bb3c02262b1c2e38 [term 2 FOLLOWER]: Rejecting Update request from peer 43999d813e564dc688fad116090c95ca for earlier term 1. Current term is 2. Ops: []
I20260709 13:23:44.153169 26056 raft_consensus.cc:2749] T fa0da2c845dc4059968332f4db2e0aa6 P 9868dce521294d0b816bba7648f4ae5a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:23:44.180076 25456 raft_consensus.cc:1240] T c902bbc7ec0848ba82a04870eab1e7f5 P 9868dce521294d0b816bba7648f4ae5a [term 2 FOLLOWER]: Rejecting Update request from peer 43999d813e564dc688fad116090c95ca for earlier term 1. Current term is 2. Ops: []
W20260709 13:23:44.180975 25424 leader_election.cc:336] T 579da2d6aabb4ae9a2b1802e1358068b P 9868dce521294d0b816bba7648f4ae5a [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 43999d813e564dc688fad116090c95ca (127.14.177.129:38155): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:23:44.197988 25383 raft_consensus.cc:2468] T 1f15ec62698c4a8cbd84f2b3d516d2ea P 3e07d64466934da4bb3c02262b1c2e38 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9868dce521294d0b816bba7648f4ae5a in term 2.
I20260709 13:23:44.186658 25424 leader_election.cc:304] T 579da2d6aabb4ae9a2b1802e1358068b P 9868dce521294d0b816bba7648f4ae5a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9868dce521294d0b816bba7648f4ae5a; no voters: 233e7d29aac04505bf4e65f0a39a0c4c, 43999d813e564dc688fad116090c95ca
I20260709 13:23:44.184134 26024 raft_consensus.cc:3060] T 91ac744179de48e5bb924b6dfd9f9c9b P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:23:44.205927 25202 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:42957, user_credentials={real_user=slave}} blocked reactor thread for 75880.5us
W20260709 13:23:44.204849 25424 leader_election.cc:336] T 9ffa42c3b2ff48bc87310e9ab5d218ce P 9868dce521294d0b816bba7648f4ae5a [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 43999d813e564dc688fad116090c95ca (127.14.177.129:38155): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:23:44.207768 25424 leader_election.cc:304] T 9ffa42c3b2ff48bc87310e9ab5d218ce P 9868dce521294d0b816bba7648f4ae5a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9868dce521294d0b816bba7648f4ae5a; no voters: 3e07d64466934da4bb3c02262b1c2e38, 43999d813e564dc688fad116090c95ca
W20260709 13:23:44.209908 25424 leader_election.cc:336] T 0e71b6f5c69b4b35a8ac51c22c7d5f0d P 9868dce521294d0b816bba7648f4ae5a [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 43999d813e564dc688fad116090c95ca (127.14.177.129:38155): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:23:44.203848 26058 raft_consensus.cc:3060] T ad32097cd4dc48b9a69595f1651abd07 P 3e07d64466934da4bb3c02262b1c2e38 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:23:44.205421 26030 raft_consensus.cc:2749] T 579da2d6aabb4ae9a2b1802e1358068b P 9868dce521294d0b816bba7648f4ae5a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:23:44.213521 25424 leader_election.cc:304] T 0e71b6f5c69b4b35a8ac51c22c7d5f0d P 9868dce521294d0b816bba7648f4ae5a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9868dce521294d0b816bba7648f4ae5a; no voters: 3e07d64466934da4bb3c02262b1c2e38, 43999d813e564dc688fad116090c95ca
I20260709 13:23:44.202157 26053 raft_consensus.cc:3060] T 89a26dd3ac414f6081b98fa32c2f346f P 9868dce521294d0b816bba7648f4ae5a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:23:44.211150 25202 leader_election.cc:304] T 51a11db644124b63bdf5f457c967473d P 43999d813e564dc688fad116090c95ca [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 43999d813e564dc688fad116090c95ca; no voters: 233e7d29aac04505bf4e65f0a39a0c4c, 9868dce521294d0b816bba7648f4ae5a
I20260709 13:23:44.179045 26019 raft_consensus.cc:515] T b7a891bfc89e4e259f53da6cb16f4989 P 43999d813e564dc688fad116090c95ca [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } }
I20260709 13:23:44.209605 26050 raft_consensus.cc:3060] T 467a6b42ca6548269c12dddde696f7af P 9868dce521294d0b816bba7648f4ae5a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:23:44.211230 26056 raft_consensus.cc:2749] T 9ffa42c3b2ff48bc87310e9ab5d218ce P 9868dce521294d0b816bba7648f4ae5a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:23:44.240729 26019 leader_election.cc:290] T b7a891bfc89e4e259f53da6cb16f4989 P 43999d813e564dc688fad116090c95ca [CANDIDATE]: Term 2 election: Requested vote from peers 3e07d64466934da4bb3c02262b1c2e38 (127.14.177.131:41539), 9868dce521294d0b816bba7648f4ae5a (127.14.177.132:33683)
W20260709 13:23:44.217419 25424 leader_election.cc:336] T 4d837697cd0f402cacf9db89a5d92652 P 9868dce521294d0b816bba7648f4ae5a [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 43999d813e564dc688fad116090c95ca (127.14.177.129:38155): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:23:44.242816 25424 leader_election.cc:304] T 4d837697cd0f402cacf9db89a5d92652 P 9868dce521294d0b816bba7648f4ae5a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9868dce521294d0b816bba7648f4ae5a; no voters: 3e07d64466934da4bb3c02262b1c2e38, 43999d813e564dc688fad116090c95ca
I20260709 13:23:44.246145 26030 raft_consensus.cc:2749] T 4d837697cd0f402cacf9db89a5d92652 P 9868dce521294d0b816bba7648f4ae5a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:23:44.248911 25424 leader_election.cc:336] T 821a5866429f4173820fb64336abee02 P 9868dce521294d0b816bba7648f4ae5a [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 43999d813e564dc688fad116090c95ca (127.14.177.129:38155): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:23:44.251076 25424 leader_election.cc:304] T 821a5866429f4173820fb64336abee02 P 9868dce521294d0b816bba7648f4ae5a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9868dce521294d0b816bba7648f4ae5a; no voters: 3e07d64466934da4bb3c02262b1c2e38, 43999d813e564dc688fad116090c95ca
I20260709 13:23:44.216583 26070 raft_consensus.cc:3055] T cfad575709c943cba40ccab278350770 P 9868dce521294d0b816bba7648f4ae5a [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:23:44.223443 26080 raft_consensus.cc:2749] T 51a11db644124b63bdf5f457c967473d P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:23:44.258041 26030 raft_consensus.cc:2749] T 821a5866429f4173820fb64336abee02 P 9868dce521294d0b816bba7648f4ae5a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:23:44.258411 26075 raft_consensus.cc:3055] T 5c7e7acbb09f4d49828fa652ca6abaac P 43999d813e564dc688fad116090c95ca [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:23:44.258778 26075 raft_consensus.cc:740] T 5c7e7acbb09f4d49828fa652ca6abaac P 43999d813e564dc688fad116090c95ca [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 43999d813e564dc688fad116090c95ca, State: Running, Role: LEADER
I20260709 13:23:44.258910 26070 raft_consensus.cc:740] T cfad575709c943cba40ccab278350770 P 9868dce521294d0b816bba7648f4ae5a [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 9868dce521294d0b816bba7648f4ae5a, State: Running, Role: LEADER
I20260709 13:23:44.259786 26075 consensus_queue.cc:260] T 5c7e7acbb09f4d49828fa652ca6abaac P 43999d813e564dc688fad116090c95ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } }
I20260709 13:23:44.259819 26070 consensus_queue.cc:260] T cfad575709c943cba40ccab278350770 P 9868dce521294d0b816bba7648f4ae5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } }
W20260709 13:23:44.257629 25424 consensus_peers.cc:597] T 9e5d70664720424d951f269d9d7f4346 P 9868dce521294d0b816bba7648f4ae5a -> Peer 43999d813e564dc688fad116090c95ca (127.14.177.129:38155): Couldn't send request to peer 43999d813e564dc688fad116090c95ca. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:23:44.261168 26075 raft_consensus.cc:3060] T 5c7e7acbb09f4d49828fa652ca6abaac P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:23:44.261770 25424 leader_election.cc:336] T d0a0e4d32ce943fbb773bb840863647b P 9868dce521294d0b816bba7648f4ae5a [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 43999d813e564dc688fad116090c95ca (127.14.177.129:38155): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:23:44.262221 25424 leader_election.cc:304] T d0a0e4d32ce943fbb773bb840863647b P 9868dce521294d0b816bba7648f4ae5a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9868dce521294d0b816bba7648f4ae5a; no voters: 3e07d64466934da4bb3c02262b1c2e38, 43999d813e564dc688fad116090c95ca
W20260709 13:23:44.263350 25424 leader_election.cc:336] T 91ac744179de48e5bb924b6dfd9f9c9b P 9868dce521294d0b816bba7648f4ae5a [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 43999d813e564dc688fad116090c95ca (127.14.177.129:38155): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:23:44.256728 25422 leader_election.cc:304] T 1f15ec62698c4a8cbd84f2b3d516d2ea P 9868dce521294d0b816bba7648f4ae5a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e07d64466934da4bb3c02262b1c2e38, 9868dce521294d0b816bba7648f4ae5a; no voters: 
I20260709 13:23:44.263856 25424 leader_election.cc:304] T 91ac744179de48e5bb924b6dfd9f9c9b P 9868dce521294d0b816bba7648f4ae5a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9868dce521294d0b816bba7648f4ae5a; no voters: 3e07d64466934da4bb3c02262b1c2e38, 43999d813e564dc688fad116090c95ca
W20260709 13:23:44.265152 25202 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.130:42957, user_credentials={real_user=slave}} blocked reactor thread for 49803.2us
I20260709 13:23:44.265996 25202 leader_election.cc:304] T 11949f56648c47e0a43c3140af99a304 P 43999d813e564dc688fad116090c95ca [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 43999d813e564dc688fad116090c95ca; no voters: 233e7d29aac04505bf4e65f0a39a0c4c, 9868dce521294d0b816bba7648f4ae5a
W20260709 13:23:44.266193 25424 leader_election.cc:336] T 441d780c3945463f92f31170f31e37a8 P 9868dce521294d0b816bba7648f4ae5a [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 43999d813e564dc688fad116090c95ca (127.14.177.129:38155): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:23:44.266707 25424 leader_election.cc:304] T 441d780c3945463f92f31170f31e37a8 P 9868dce521294d0b816bba7648f4ae5a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9868dce521294d0b816bba7648f4ae5a; no voters: 233e7d29aac04505bf4e65f0a39a0c4c, 43999d813e564dc688fad116090c95ca
I20260709 13:23:44.223640 26065 raft_consensus.cc:2749] T 0e71b6f5c69b4b35a8ac51c22c7d5f0d P 9868dce521294d0b816bba7648f4ae5a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:23:44.273240 25424 leader_election.cc:336] T 1f15ec62698c4a8cbd84f2b3d516d2ea P 9868dce521294d0b816bba7648f4ae5a [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 43999d813e564dc688fad116090c95ca (127.14.177.129:38155): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:23:44.272087 25202 leader_election.cc:304] T e15820fa8a7d4e1fb6ad23a64ca22f90 P 43999d813e564dc688fad116090c95ca [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 43999d813e564dc688fad116090c95ca; no voters: 233e7d29aac04505bf4e65f0a39a0c4c, 3e07d64466934da4bb3c02262b1c2e38
W20260709 13:23:44.274174 25424 leader_election.cc:336] T f1ac2f4d2fe2424bb470df42199219e2 P 9868dce521294d0b816bba7648f4ae5a [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 43999d813e564dc688fad116090c95ca (127.14.177.129:38155): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:23:44.274622 25424 leader_election.cc:304] T f1ac2f4d2fe2424bb470df42199219e2 P 9868dce521294d0b816bba7648f4ae5a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9868dce521294d0b816bba7648f4ae5a; no voters: 3e07d64466934da4bb3c02262b1c2e38, 43999d813e564dc688fad116090c95ca
I20260709 13:23:44.275673 25202 leader_election.cc:304] T e9e342ecda4b46619ce12452b2cf7154 P 43999d813e564dc688fad116090c95ca [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 233e7d29aac04505bf4e65f0a39a0c4c, 43999d813e564dc688fad116090c95ca; no voters: 9868dce521294d0b816bba7648f4ae5a
W20260709 13:23:44.275869 25424 leader_election.cc:336] T b9d550d881b04735bbf89751952c09ec P 9868dce521294d0b816bba7648f4ae5a [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 43999d813e564dc688fad116090c95ca (127.14.177.129:38155): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:23:44.276440 25424 leader_election.cc:304] T b9d550d881b04735bbf89751952c09ec P 9868dce521294d0b816bba7648f4ae5a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9868dce521294d0b816bba7648f4ae5a; no voters: 3e07d64466934da4bb3c02262b1c2e38, 43999d813e564dc688fad116090c95ca
I20260709 13:23:44.264364 26059 raft_consensus.cc:2749] T 89a26dd3ac414f6081b98fa32c2f346f P 233e7d29aac04505bf4e65f0a39a0c4c [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:23:44.273125 26060 raft_consensus.cc:2749] T 91ac744179de48e5bb924b6dfd9f9c9b P 9868dce521294d0b816bba7648f4ae5a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:23:44.277880 25955 raft_consensus.cc:2804] T e9e342ecda4b46619ce12452b2cf7154 P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:23:44.273237 26069 raft_consensus.cc:2804] T 1f15ec62698c4a8cbd84f2b3d516d2ea P 9868dce521294d0b816bba7648f4ae5a [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:23:44.279342 26022 raft_consensus.cc:2749] T f1ac2f4d2fe2424bb470df42199219e2 P 9868dce521294d0b816bba7648f4ae5a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:23:44.287770 25202 leader_election.cc:304] T 6242888a4d4146dcacd87d0fd9a832f4 P 43999d813e564dc688fad116090c95ca [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 233e7d29aac04505bf4e65f0a39a0c4c, 43999d813e564dc688fad116090c95ca; no voters: 9868dce521294d0b816bba7648f4ae5a
I20260709 13:23:44.275308 26081 raft_consensus.cc:2749] T e15820fa8a7d4e1fb6ad23a64ca22f90 P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:23:44.269273 26062 raft_consensus.cc:2749] T 467a6b42ca6548269c12dddde696f7af P 233e7d29aac04505bf4e65f0a39a0c4c [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:23:44.286324 25456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ed5edbe32044d1c83b0590dbb899801" candidate_uuid: "43999d813e564dc688fad116090c95ca" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9868dce521294d0b816bba7648f4ae5a"
I20260709 13:23:44.286593 25380 raft_consensus.cc:1240] T b155b866f10547fd8239579a81ecf5f9 P 3e07d64466934da4bb3c02262b1c2e38 [term 2 FOLLOWER]: Rejecting Update request from peer 43999d813e564dc688fad116090c95ca for earlier term 1. Current term is 2. Ops: []
I20260709 13:23:44.286741 25382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ed5edbe32044d1c83b0590dbb899801" candidate_uuid: "43999d813e564dc688fad116090c95ca" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3e07d64466934da4bb3c02262b1c2e38"
I20260709 13:23:44.273058 26023 raft_consensus.cc:2749] T 441d780c3945463f92f31170f31e37a8 P 9868dce521294d0b816bba7648f4ae5a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:23:44.267663 26056 raft_consensus.cc:2749] T d0a0e4d32ce943fbb773bb840863647b P 9868dce521294d0b816bba7648f4ae5a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:23:44.272921 26080 raft_consensus.cc:2749] T 11949f56648c47e0a43c3140af99a304 P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:23:44.289266 25202 leader_election.cc:304] T b2f14aed05e24a98ad83535029b795dc P 43999d813e564dc688fad116090c95ca [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 43999d813e564dc688fad116090c95ca; no voters: 233e7d29aac04505bf4e65f0a39a0c4c, 3e07d64466934da4bb3c02262b1c2e38
W20260709 13:23:44.295351 25350 leader_election.cc:336] T 5c7e7acbb09f4d49828fa652ca6abaac P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 43999d813e564dc688fad116090c95ca (127.14.177.129:38155): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:23:44.299378 25383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7a891bfc89e4e259f53da6cb16f4989" candidate_uuid: "43999d813e564dc688fad116090c95ca" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3e07d64466934da4bb3c02262b1c2e38"
I20260709 13:23:44.304382 25454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7a891bfc89e4e259f53da6cb16f4989" candidate_uuid: "43999d813e564dc688fad116090c95ca" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9868dce521294d0b816bba7648f4ae5a"
I20260709 13:23:44.284289 25955 raft_consensus.cc:493] T e9e342ecda4b46619ce12452b2cf7154 P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Starting leader election (detected failure of leader 9868dce521294d0b816bba7648f4ae5a)
I20260709 13:23:44.288118 26022 raft_consensus.cc:2749] T b9d550d881b04735bbf89751952c09ec P 9868dce521294d0b816bba7648f4ae5a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:23:44.306263 25350 leader_election.cc:304] T 5c7e7acbb09f4d49828fa652ca6abaac P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3e07d64466934da4bb3c02262b1c2e38; no voters: 43999d813e564dc688fad116090c95ca, 9868dce521294d0b816bba7648f4ae5a
I20260709 13:23:44.310952 25202 leader_election.cc:304] T d053b9ae38de4173a0aded4da494bd98 P 43999d813e564dc688fad116090c95ca [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 43999d813e564dc688fad116090c95ca; no voters: 233e7d29aac04505bf4e65f0a39a0c4c, 9868dce521294d0b816bba7648f4ae5a
I20260709 13:23:44.292675 25977 raft_consensus.cc:2804] T 6242888a4d4146dcacd87d0fd9a832f4 P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:23:44.313431 26077 raft_consensus.cc:2749] T b2f14aed05e24a98ad83535029b795dc P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:23:44.313861 25977 raft_consensus.cc:493] T 6242888a4d4146dcacd87d0fd9a832f4 P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Starting leader election (detected failure of leader 9868dce521294d0b816bba7648f4ae5a)
I20260709 13:23:44.314311 25977 raft_consensus.cc:3060] T 6242888a4d4146dcacd87d0fd9a832f4 P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:23:44.314812 26081 consensus_queue.cc:1059] T c902bbc7ec0848ba82a04870eab1e7f5 P 43999d813e564dc688fad116090c95ca [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:23:44.315999 26081 raft_consensus.cc:3055] T c902bbc7ec0848ba82a04870eab1e7f5 P 43999d813e564dc688fad116090c95ca [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:23:44.317703 25202 leader_election.cc:304] T beacc815ef954fb692e90544bfa93f05 P 43999d813e564dc688fad116090c95ca [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 43999d813e564dc688fad116090c95ca; no voters: 233e7d29aac04505bf4e65f0a39a0c4c, 3e07d64466934da4bb3c02262b1c2e38
I20260709 13:23:44.316376 26081 raft_consensus.cc:740] T c902bbc7ec0848ba82a04870eab1e7f5 P 43999d813e564dc688fad116090c95ca [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 43999d813e564dc688fad116090c95ca, State: Running, Role: LEADER
I20260709 13:23:44.318384 26080 raft_consensus.cc:2749] T d053b9ae38de4173a0aded4da494bd98 P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:23:44.319131 26057 raft_consensus.cc:2749] T 5c7e7acbb09f4d49828fa652ca6abaac P 3e07d64466934da4bb3c02262b1c2e38 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:23:44.319218 26081 consensus_queue.cc:260] T c902bbc7ec0848ba82a04870eab1e7f5 P 43999d813e564dc688fad116090c95ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } }
I20260709 13:23:44.309019 25955 raft_consensus.cc:3060] T e9e342ecda4b46619ce12452b2cf7154 P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:23:44.321072 26081 raft_consensus.cc:3060] T c902bbc7ec0848ba82a04870eab1e7f5 P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:23:44.325290 25977 raft_consensus.cc:515] T 6242888a4d4146dcacd87d0fd9a832f4 P 43999d813e564dc688fad116090c95ca [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } }
I20260709 13:23:44.326879 25202 leader_election.cc:304] T 79d4a12ef58140568c60a24ed4c12f83 P 43999d813e564dc688fad116090c95ca [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 43999d813e564dc688fad116090c95ca; no voters: 233e7d29aac04505bf4e65f0a39a0c4c, 3e07d64466934da4bb3c02262b1c2e38
I20260709 13:23:44.331000 26019 raft_consensus.cc:2749] T beacc815ef954fb692e90544bfa93f05 P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:23:44.326467 25424 consensus_peers.cc:597] T e9e342ecda4b46619ce12452b2cf7154 P 9868dce521294d0b816bba7648f4ae5a -> Peer 43999d813e564dc688fad116090c95ca (127.14.177.129:38155): Couldn't send request to peer 43999d813e564dc688fad116090c95ca. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:23:44.336196 25202 leader_election.cc:304] T 28c0b6fc9bd64d60828d705c6a9ab1e4 P 43999d813e564dc688fad116090c95ca [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 43999d813e564dc688fad116090c95ca; no voters: 233e7d29aac04505bf4e65f0a39a0c4c, 3e07d64466934da4bb3c02262b1c2e38
W20260709 13:23:44.336464 25424 leader_election.cc:336] T 3a3317f4c3514df1b2bb8e44a86be92f P 9868dce521294d0b816bba7648f4ae5a [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 43999d813e564dc688fad116090c95ca (127.14.177.129:38155): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:23:44.334745 26069 raft_consensus.cc:697] T 1f15ec62698c4a8cbd84f2b3d516d2ea P 9868dce521294d0b816bba7648f4ae5a [term 2 LEADER]: Becoming Leader. State: Replica: 9868dce521294d0b816bba7648f4ae5a, State: Running, Role: LEADER
I20260709 13:23:44.338425 26069 consensus_queue.cc:237] T 1f15ec62698c4a8cbd84f2b3d516d2ea P 9868dce521294d0b816bba7648f4ae5a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } }
I20260709 13:23:44.341964 25977 leader_election.cc:290] T 6242888a4d4146dcacd87d0fd9a832f4 P 43999d813e564dc688fad116090c95ca [CANDIDATE]: Term 2 election: Requested vote from peers 9868dce521294d0b816bba7648f4ae5a (127.14.177.132:33683), 233e7d29aac04505bf4e65f0a39a0c4c (127.14.177.130:42957)
I20260709 13:23:44.336314 26028 raft_consensus.cc:2749] T 79d4a12ef58140568c60a24ed4c12f83 P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:23:44.337159 25424 leader_election.cc:304] T 3a3317f4c3514df1b2bb8e44a86be92f P 9868dce521294d0b816bba7648f4ae5a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9868dce521294d0b816bba7648f4ae5a; no voters: 233e7d29aac04505bf4e65f0a39a0c4c, 43999d813e564dc688fad116090c95ca
I20260709 13:23:44.347193 25458 raft_consensus.cc:2468] T 89a26dd3ac414f6081b98fa32c2f346f P 9868dce521294d0b816bba7648f4ae5a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3e07d64466934da4bb3c02262b1c2e38 in term 2.
W20260709 13:23:44.337014 25350 consensus_peers.cc:597] T b2f14aed05e24a98ad83535029b795dc P 3e07d64466934da4bb3c02262b1c2e38 -> Peer 43999d813e564dc688fad116090c95ca (127.14.177.129:38155): Couldn't send request to peer 43999d813e564dc688fad116090c95ca. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:23:44.355075 25350 consensus_peers.cc:597] T b7a891bfc89e4e259f53da6cb16f4989 P 3e07d64466934da4bb3c02262b1c2e38 -> Peer 43999d813e564dc688fad116090c95ca (127.14.177.129:38155): Couldn't send request to peer 43999d813e564dc688fad116090c95ca. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:23:44.354959 25202 leader_election.cc:304] T 0cb89d70b4ce4aa8992fb68480bec8a2 P 43999d813e564dc688fad116090c95ca [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 43999d813e564dc688fad116090c95ca; no voters: 233e7d29aac04505bf4e65f0a39a0c4c, 3e07d64466934da4bb3c02262b1c2e38
W20260709 13:23:44.355856 25350 consensus_peers.cc:597] T 0cb89d70b4ce4aa8992fb68480bec8a2 P 3e07d64466934da4bb3c02262b1c2e38 -> Peer 43999d813e564dc688fad116090c95ca (127.14.177.129:38155): Couldn't send request to peer 43999d813e564dc688fad116090c95ca. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:23:44.356293 25350 consensus_peers.cc:597] T 03e9bfbd2f774476a4a377367796a2e8 P 3e07d64466934da4bb3c02262b1c2e38 -> Peer 43999d813e564dc688fad116090c95ca (127.14.177.129:38155): Couldn't send request to peer 43999d813e564dc688fad116090c95ca. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:23:44.348173 26079 consensus_queue.cc:1059] T 72b56266e8a54776b9113c7a786e1952 P 43999d813e564dc688fad116090c95ca [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.012s
I20260709 13:23:44.360177 26078 raft_consensus.cc:2749] T 28c0b6fc9bd64d60828d705c6a9ab1e4 P 43999d813e564dc688fad116090c95ca [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:23:44.365171 26070 raft_consensus.cc:3060] T cfad575709c943cba40ccab278350770 P 9868dce521294d0b816bba7648f4ae5a [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:23:44.368173 25199 leader_election.cc:304] T b7a891bfc89e4e259f53da6cb16f4989 P 43999d813e564dc688fad116090c95ca [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 43999d813e564dc688fad116090c95ca; no voters: 3e07d64466934da4bb3c02262b1c2e38, 9868dce521294d0b816bba7648f4ae5a
I20260709 13:23:44.369073 26079 raft_consensus.cc:3055] T 72b56266e8a54776b9113c7a786e1952 P 43999d813e564dc688fad116090c95ca [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:23:44.371338 26075 raft_consensus.cc:2749] T 0cb89d70b4ce4aa8992fb68480bec8a2 P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:23:44.370182 25348 leader_election.cc:304] T 89a26dd3ac414f6081b98fa32c2f346f P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3e07d64466934da4bb3c02262b1c2e38, 9868dce521294d0b816bba7648f4ae5a; no voters: 233e7d29aac04505bf4e65f0a39a0c4c
I20260709 13:23:44.371678 26079 raft_consensus.cc:740] T 72b56266e8a54776b9113c7a786e1952 P 43999d813e564dc688fad116090c95ca [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 43999d813e564dc688fad116090c95ca, State: Running, Role: LEADER
I20260709 13:23:44.333258 26024 raft_consensus.cc:515] T 91ac744179de48e5bb924b6dfd9f9c9b P 43999d813e564dc688fad116090c95ca [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } }
I20260709 13:23:44.373252 26031 raft_consensus.cc:2804] T 89a26dd3ac414f6081b98fa32c2f346f P 3e07d64466934da4bb3c02262b1c2e38 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:23:44.373369 25202 leader_election.cc:304] T 03e9bfbd2f774476a4a377367796a2e8 P 43999d813e564dc688fad116090c95ca [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 43999d813e564dc688fad116090c95ca; no voters: 233e7d29aac04505bf4e65f0a39a0c4c, 3e07d64466934da4bb3c02262b1c2e38
I20260709 13:23:44.373852 26031 raft_consensus.cc:697] T 89a26dd3ac414f6081b98fa32c2f346f P 3e07d64466934da4bb3c02262b1c2e38 [term 2 LEADER]: Becoming Leader. State: Replica: 3e07d64466934da4bb3c02262b1c2e38, State: Running, Role: LEADER
I20260709 13:23:44.374231 26019 raft_consensus.cc:2749] T 03e9bfbd2f774476a4a377367796a2e8 P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:23:44.374307 26024 leader_election.cc:290] T 91ac744179de48e5bb924b6dfd9f9c9b P 43999d813e564dc688fad116090c95ca [CANDIDATE]: Term 2 election: Requested vote from peers 3e07d64466934da4bb3c02262b1c2e38 (127.14.177.131:41539), 9868dce521294d0b816bba7648f4ae5a (127.14.177.132:33683)
I20260709 13:23:44.375751 26024 raft_consensus.cc:2749] T b7a891bfc89e4e259f53da6cb16f4989 P 43999d813e564dc688fad116090c95ca [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:23:44.374898 26031 consensus_queue.cc:237] T 89a26dd3ac414f6081b98fa32c2f346f P 3e07d64466934da4bb3c02262b1c2e38 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } }
I20260709 13:23:44.384306 25955 raft_consensus.cc:515] T e9e342ecda4b46619ce12452b2cf7154 P 43999d813e564dc688fad116090c95ca [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } }
I20260709 13:23:44.373873 26079 consensus_queue.cc:260] T 72b56266e8a54776b9113c7a786e1952 P 43999d813e564dc688fad116090c95ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } }
I20260709 13:23:44.390419 25199 leader_election.cc:304] T 0ed5edbe32044d1c83b0590dbb899801 P 43999d813e564dc688fad116090c95ca [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 43999d813e564dc688fad116090c95ca; no voters: 3e07d64466934da4bb3c02262b1c2e38, 9868dce521294d0b816bba7648f4ae5a
I20260709 13:23:44.390815 26079 raft_consensus.cc:3060] T 72b56266e8a54776b9113c7a786e1952 P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:23:44.391680 25424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:38155, user_credentials={real_user=slave}} blocked reactor thread for 55260.7us
W20260709 13:23:44.392367 25424 leader_election.cc:336] T b7a891bfc89e4e259f53da6cb16f4989 P 9868dce521294d0b816bba7648f4ae5a [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 43999d813e564dc688fad116090c95ca (127.14.177.129:38155): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:23:44.385000 25202 leader_election.cc:304] T 9e5d70664720424d951f269d9d7f4346 P 43999d813e564dc688fad116090c95ca [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 233e7d29aac04505bf4e65f0a39a0c4c, 43999d813e564dc688fad116090c95ca; no voters: 9868dce521294d0b816bba7648f4ae5a
I20260709 13:23:44.392771 25424 leader_election.cc:304] T b7a891bfc89e4e259f53da6cb16f4989 P 9868dce521294d0b816bba7648f4ae5a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9868dce521294d0b816bba7648f4ae5a; no voters: 3e07d64466934da4bb3c02262b1c2e38, 43999d813e564dc688fad116090c95ca
I20260709 13:23:44.393465 25458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6242888a4d4146dcacd87d0fd9a832f4" candidate_uuid: "43999d813e564dc688fad116090c95ca" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9868dce521294d0b816bba7648f4ae5a"
I20260709 13:23:44.385619 26019 consensus_queue.cc:1059] T ce9c60e3922c4219b6e4453fe0259b6a P 43999d813e564dc688fad116090c95ca [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:23:44.389498 25977 consensus_queue.cc:1059] T b155b866f10547fd8239579a81ecf5f9 P 43999d813e564dc688fad116090c95ca [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:23:44.393957 25276 consensus_peers.cc:597] T 51a11db644124b63bdf5f457c967473d P 233e7d29aac04505bf4e65f0a39a0c4c -> Peer 43999d813e564dc688fad116090c95ca (127.14.177.129:38155): Couldn't send request to peer 43999d813e564dc688fad116090c95ca. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:23:44.398627 25276 leader_election.cc:336] T d053b9ae38de4173a0aded4da494bd98 P 233e7d29aac04505bf4e65f0a39a0c4c [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 43999d813e564dc688fad116090c95ca (127.14.177.129:38155): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:23:44.399026 25276 leader_election.cc:304] T d053b9ae38de4173a0aded4da494bd98 P 233e7d29aac04505bf4e65f0a39a0c4c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 233e7d29aac04505bf4e65f0a39a0c4c; no voters: 43999d813e564dc688fad116090c95ca, 9868dce521294d0b816bba7648f4ae5a
I20260709 13:23:44.402335 25458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91ac744179de48e5bb924b6dfd9f9c9b" candidate_uuid: "43999d813e564dc688fad116090c95ca" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9868dce521294d0b816bba7648f4ae5a"
W20260709 13:23:44.401193 25276 consensus_peers.cc:597] T b85a238bbc3b4621881603a534c7bd9b P 233e7d29aac04505bf4e65f0a39a0c4c -> Peer 43999d813e564dc688fad116090c95ca (127.14.177.129:38155): Couldn't send request to peer 43999d813e564dc688fad116090c95ca. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:23:44.396200 26075 raft_consensus.cc:2749] T 0ed5edbe32044d1c83b0590dbb899801 P 43999d813e564dc688fad116090c95ca [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:23:44.395720 26054 raft_consensus.cc:2749] T 3a3317f4c3514df1b2bb8e44a86be92f P 9868dce521294d0b816bba7648f4ae5a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:23:44.397025 26028 raft_consensus.cc:2804] T 9e5d70664720424d951f269d9d7f4346 P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260709 13:23:44.412258 25276 consensus_peers.cc:597] T 28c0b6fc9bd64d60828d705c6a9ab1e4 P 233e7d29aac04505bf4e65f0a39a0c4c -> Peer 43999d813e564dc688fad116090c95ca (127.14.177.129:38155): Couldn't send request to peer 43999d813e564dc688fad116090c95ca. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:23:44.420320 25276 leader_election.cc:336] T 72b56266e8a54776b9113c7a786e1952 P 233e7d29aac04505bf4e65f0a39a0c4c [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 43999d813e564dc688fad116090c95ca (127.14.177.129:38155): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:23:44.408084 26059 raft_consensus.cc:2749] T d053b9ae38de4173a0aded4da494bd98 P 233e7d29aac04505bf4e65f0a39a0c4c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:23:44.421103 25276 leader_election.cc:304] T 72b56266e8a54776b9113c7a786e1952 P 233e7d29aac04505bf4e65f0a39a0c4c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 233e7d29aac04505bf4e65f0a39a0c4c; no voters: 3e07d64466934da4bb3c02262b1c2e38, 43999d813e564dc688fad116090c95ca
I20260709 13:23:44.410741 25303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6242888a4d4146dcacd87d0fd9a832f4" candidate_uuid: "43999d813e564dc688fad116090c95ca" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "233e7d29aac04505bf4e65f0a39a0c4c"
I20260709 13:23:44.418622 25379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91ac744179de48e5bb924b6dfd9f9c9b" candidate_uuid: "43999d813e564dc688fad116090c95ca" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3e07d64466934da4bb3c02262b1c2e38"
I20260709 13:23:44.426312 26017 raft_consensus.cc:3055] T ce9c60e3922c4219b6e4453fe0259b6a P 43999d813e564dc688fad116090c95ca [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:23:44.426765 26017 raft_consensus.cc:740] T ce9c60e3922c4219b6e4453fe0259b6a P 43999d813e564dc688fad116090c95ca [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 43999d813e564dc688fad116090c95ca, State: Running, Role: LEADER
W20260709 13:23:44.435110 25276 leader_election.cc:336] T ce9c60e3922c4219b6e4453fe0259b6a P 233e7d29aac04505bf4e65f0a39a0c4c [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 43999d813e564dc688fad116090c95ca (127.14.177.129:38155): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:23:44.436318 25276 leader_election.cc:336] T 0c4f6b7401db47d59a4f34ed0e8b5350 P 233e7d29aac04505bf4e65f0a39a0c4c [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 43999d813e564dc688fad116090c95ca (127.14.177.129:38155): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:23:44.436753 25276 leader_election.cc:304] T 0c4f6b7401db47d59a4f34ed0e8b5350 P 233e7d29aac04505bf4e65f0a39a0c4c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 233e7d29aac04505bf4e65f0a39a0c4c; no voters: 43999d813e564dc688fad116090c95ca, 9868dce521294d0b816bba7648f4ae5a
I20260709 13:23:44.419420 26028 raft_consensus.cc:493] T 9e5d70664720424d951f269d9d7f4346 P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Starting leader election (detected failure of leader 9868dce521294d0b816bba7648f4ae5a)
I20260709 13:23:44.442452 26035 raft_consensus.cc:2749] T 0c4f6b7401db47d59a4f34ed0e8b5350 P 233e7d29aac04505bf4e65f0a39a0c4c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:23:44.436231 26062 raft_consensus.cc:2749] T 72b56266e8a54776b9113c7a786e1952 P 233e7d29aac04505bf4e65f0a39a0c4c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:23:44.444676 26076 raft_consensus.cc:3055] T b155b866f10547fd8239579a81ecf5f9 P 43999d813e564dc688fad116090c95ca [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:23:44.445134 26076 raft_consensus.cc:740] T b155b866f10547fd8239579a81ecf5f9 P 43999d813e564dc688fad116090c95ca [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 43999d813e564dc688fad116090c95ca, State: Running, Role: LEADER
I20260709 13:23:44.445974 26076 consensus_queue.cc:260] T b155b866f10547fd8239579a81ecf5f9 P 43999d813e564dc688fad116090c95ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } }
I20260709 13:23:44.433614 25955 leader_election.cc:290] T e9e342ecda4b46619ce12452b2cf7154 P 43999d813e564dc688fad116090c95ca [CANDIDATE]: Term 2 election: Requested vote from peers 9868dce521294d0b816bba7648f4ae5a (127.14.177.132:33683), 233e7d29aac04505bf4e65f0a39a0c4c (127.14.177.130:42957)
I20260709 13:23:44.459187 25134 catalog_manager.cc:5697] T 89a26dd3ac414f6081b98fa32c2f346f P 3e07d64466934da4bb3c02262b1c2e38 reported cstate change: term changed from 1 to 2, leader changed from 9868dce521294d0b816bba7648f4ae5a (127.14.177.132) to 3e07d64466934da4bb3c02262b1c2e38 (127.14.177.131). New cstate: current_term: 2 leader_uuid: "3e07d64466934da4bb3c02262b1c2e38" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } health_report { overall_health: UNKNOWN } } }
W20260709 13:23:44.439677 25276 consensus_peers.cc:597] T ce98a3f3b3eb4c63bcaa355ec7cd64a4 P 233e7d29aac04505bf4e65f0a39a0c4c -> Peer 43999d813e564dc688fad116090c95ca (127.14.177.129:38155): Couldn't send request to peer 43999d813e564dc688fad116090c95ca. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:23:44.459904 25350 leader_election.cc:336] T b9d550d881b04735bbf89751952c09ec P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 43999d813e564dc688fad116090c95ca (127.14.177.129:38155): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:23:44.463537 25350 leader_election.cc:304] T b9d550d881b04735bbf89751952c09ec P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3e07d64466934da4bb3c02262b1c2e38; no voters: 43999d813e564dc688fad116090c95ca, 9868dce521294d0b816bba7648f4ae5a
I20260709 13:23:44.463960 25308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9e342ecda4b46619ce12452b2cf7154" candidate_uuid: "43999d813e564dc688fad116090c95ca" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "233e7d29aac04505bf4e65f0a39a0c4c"
I20260709 13:23:44.447113 26076 raft_consensus.cc:3060] T b155b866f10547fd8239579a81ecf5f9 P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:23:44.472713 25350 consensus_peers.cc:597] T 91ac744179de48e5bb924b6dfd9f9c9b P 3e07d64466934da4bb3c02262b1c2e38 -> Peer 43999d813e564dc688fad116090c95ca (127.14.177.129:38155): Couldn't send request to peer 43999d813e564dc688fad116090c95ca. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:23:44.473356 25350 leader_election.cc:336] T b155b866f10547fd8239579a81ecf5f9 P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 43999d813e564dc688fad116090c95ca (127.14.177.129:38155): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:23:44.474201 25350 consensus_peers.cc:597] T 79d4a12ef58140568c60a24ed4c12f83 P 3e07d64466934da4bb3c02262b1c2e38 -> Peer 43999d813e564dc688fad116090c95ca (127.14.177.129:38155): Couldn't send request to peer 43999d813e564dc688fad116090c95ca. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:23:44.474833 25350 consensus_peers.cc:597] T beacc815ef954fb692e90544bfa93f05 P 3e07d64466934da4bb3c02262b1c2e38 -> Peer 43999d813e564dc688fad116090c95ca (127.14.177.129:38155): Couldn't send request to peer 43999d813e564dc688fad116090c95ca. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:23:44.475561 26061 raft_consensus.cc:2749] T b9d550d881b04735bbf89751952c09ec P 3e07d64466934da4bb3c02262b1c2e38 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:23:44.478372 25350 leader_election.cc:336] T 72b56266e8a54776b9113c7a786e1952 P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 43999d813e564dc688fad116090c95ca (127.14.177.129:38155): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:23:44.457731 26028 raft_consensus.cc:3060] T 9e5d70664720424d951f269d9d7f4346 P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:23:44.479514 25350 leader_election.cc:336] T 434a54fcee7d4aeeab42f739855c0e8a P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 43999d813e564dc688fad116090c95ca (127.14.177.129:38155): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:23:44.479892 25350 leader_election.cc:304] T 434a54fcee7d4aeeab42f739855c0e8a P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3e07d64466934da4bb3c02262b1c2e38; no voters: 233e7d29aac04505bf4e65f0a39a0c4c, 43999d813e564dc688fad116090c95ca
W20260709 13:23:44.480952 25350 consensus_peers.cc:597] T 0ed5edbe32044d1c83b0590dbb899801 P 3e07d64466934da4bb3c02262b1c2e38 -> Peer 43999d813e564dc688fad116090c95ca (127.14.177.129:38155): Couldn't send request to peer 43999d813e564dc688fad116090c95ca. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:23:44.465739 25199 leader_election.cc:304] T 91ac744179de48e5bb924b6dfd9f9c9b P 43999d813e564dc688fad116090c95ca [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 43999d813e564dc688fad116090c95ca; no voters: 3e07d64466934da4bb3c02262b1c2e38, 9868dce521294d0b816bba7648f4ae5a
I20260709 13:23:44.466989 15046 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:23:44.483650 25454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9e342ecda4b46619ce12452b2cf7154" candidate_uuid: "43999d813e564dc688fad116090c95ca" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9868dce521294d0b816bba7648f4ae5a"
I20260709 13:23:44.484078 15046 tablet_replica.cc:333] T 72b56266e8a54776b9113c7a786e1952 P 43999d813e564dc688fad116090c95ca: stopping tablet replica
I20260709 13:23:44.484910 26058 raft_consensus.cc:2749] T 434a54fcee7d4aeeab42f739855c0e8a P 3e07d64466934da4bb3c02262b1c2e38 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:23:44.471077 26017 consensus_queue.cc:260] T ce9c60e3922c4219b6e4453fe0259b6a P 43999d813e564dc688fad116090c95ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } }
I20260709 13:23:44.487249 26017 raft_consensus.cc:3060] T ce9c60e3922c4219b6e4453fe0259b6a P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:23:44.481470 25350 consensus_peers.cc:597] T e15820fa8a7d4e1fb6ad23a64ca22f90 P 3e07d64466934da4bb3c02262b1c2e38 -> Peer 43999d813e564dc688fad116090c95ca (127.14.177.129:38155): Couldn't send request to peer 43999d813e564dc688fad116090c95ca. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:23:44.471894 26083 raft_consensus.cc:2749] T b7a891bfc89e4e259f53da6cb16f4989 P 9868dce521294d0b816bba7648f4ae5a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:23:44.492051 25350 consensus_peers.cc:597] T 9ffa42c3b2ff48bc87310e9ab5d218ce P 3e07d64466934da4bb3c02262b1c2e38 -> Peer 43999d813e564dc688fad116090c95ca (127.14.177.129:38155): Couldn't send request to peer 43999d813e564dc688fad116090c95ca. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:23:44.492657 25350 consensus_peers.cc:597] T 87085ac2575e4b6a86dd1fc21e6f718b P 3e07d64466934da4bb3c02262b1c2e38 -> Peer 43999d813e564dc688fad116090c95ca (127.14.177.129:38155): Couldn't send request to peer 43999d813e564dc688fad116090c95ca. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:23:44.493008 26080 raft_consensus.cc:2749] T 91ac744179de48e5bb924b6dfd9f9c9b P 43999d813e564dc688fad116090c95ca [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
W20260709 13:23:44.493147 25350 leader_election.cc:336] T b155b866f10547fd8239579a81ecf5f9 P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 43999d813e564dc688fad116090c95ca (127.14.177.129:38155): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:23:44.493664 25350 leader_election.cc:304] T b155b866f10547fd8239579a81ecf5f9 P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3e07d64466934da4bb3c02262b1c2e38; no voters: 233e7d29aac04505bf4e65f0a39a0c4c, 43999d813e564dc688fad116090c95ca
W20260709 13:23:44.463635 25276 leader_election.cc:336] T b7ff8dde9f5d4c49b9203d902f6b94c4 P 233e7d29aac04505bf4e65f0a39a0c4c [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 43999d813e564dc688fad116090c95ca (127.14.177.129:38155): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:23:44.494798 25350 leader_election.cc:336] T 72b56266e8a54776b9113c7a786e1952 P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 43999d813e564dc688fad116090c95ca (127.14.177.129:38155): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:23:44.494865 25276 leader_election.cc:304] T b7ff8dde9f5d4c49b9203d902f6b94c4 P 233e7d29aac04505bf4e65f0a39a0c4c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 233e7d29aac04505bf4e65f0a39a0c4c; no voters: 3e07d64466934da4bb3c02262b1c2e38, 43999d813e564dc688fad116090c95ca
I20260709 13:23:44.496737 26046 raft_consensus.cc:2749] T b155b866f10547fd8239579a81ecf5f9 P 3e07d64466934da4bb3c02262b1c2e38 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:23:44.499230 15046 raft_consensus.cc:2243] T 72b56266e8a54776b9113c7a786e1952 P 43999d813e564dc688fad116090c95ca [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:23:44.501506 25350 leader_election.cc:304] T 72b56266e8a54776b9113c7a786e1952 P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3e07d64466934da4bb3c02262b1c2e38; no voters: 233e7d29aac04505bf4e65f0a39a0c4c, 43999d813e564dc688fad116090c95ca
I20260709 13:23:44.502290 26058 raft_consensus.cc:2749] T 72b56266e8a54776b9113c7a786e1952 P 3e07d64466934da4bb3c02262b1c2e38 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:23:44.504583 26028 raft_consensus.cc:515] T 9e5d70664720424d951f269d9d7f4346 P 43999d813e564dc688fad116090c95ca [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } }
I20260709 13:23:44.506691 26028 leader_election.cc:290] T 9e5d70664720424d951f269d9d7f4346 P 43999d813e564dc688fad116090c95ca [CANDIDATE]: Term 2 election: Requested vote from peers 9868dce521294d0b816bba7648f4ae5a (127.14.177.132:33683), 233e7d29aac04505bf4e65f0a39a0c4c (127.14.177.130:42957)
I20260709 13:23:44.509666 26059 raft_consensus.cc:2749] T b7ff8dde9f5d4c49b9203d902f6b94c4 P 233e7d29aac04505bf4e65f0a39a0c4c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:23:44.510949 25424 leader_election.cc:336] T 5c7e7acbb09f4d49828fa652ca6abaac P 9868dce521294d0b816bba7648f4ae5a [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 43999d813e564dc688fad116090c95ca (127.14.177.129:38155): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:23:44.511547 25424 leader_election.cc:304] T 5c7e7acbb09f4d49828fa652ca6abaac P 9868dce521294d0b816bba7648f4ae5a [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9868dce521294d0b816bba7648f4ae5a; no voters: 3e07d64466934da4bb3c02262b1c2e38, 43999d813e564dc688fad116090c95ca
W20260709 13:23:44.513140 25276 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:38155, user_credentials={real_user=slave}} blocked reactor thread for 49547.3us
W20260709 13:23:44.513826 25424 leader_election.cc:336] T c902bbc7ec0848ba82a04870eab1e7f5 P 9868dce521294d0b816bba7648f4ae5a [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 43999d813e564dc688fad116090c95ca (127.14.177.129:38155): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:23:44.514395 25424 leader_election.cc:304] T c902bbc7ec0848ba82a04870eab1e7f5 P 9868dce521294d0b816bba7648f4ae5a [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9868dce521294d0b816bba7648f4ae5a; no voters: 3e07d64466934da4bb3c02262b1c2e38, 43999d813e564dc688fad116090c95ca
I20260709 13:23:44.517750 26022 raft_consensus.cc:2749] T 5c7e7acbb09f4d49828fa652ca6abaac P 9868dce521294d0b816bba7648f4ae5a [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
W20260709 13:23:44.513857 25276 leader_election.cc:336] T 0cb89d70b4ce4aa8992fb68480bec8a2 P 233e7d29aac04505bf4e65f0a39a0c4c [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 43999d813e564dc688fad116090c95ca (127.14.177.129:38155): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:23:44.518503 26022 raft_consensus.cc:2749] T c902bbc7ec0848ba82a04870eab1e7f5 P 9868dce521294d0b816bba7648f4ae5a [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:23:44.518841 25276 leader_election.cc:304] T 0cb89d70b4ce4aa8992fb68480bec8a2 P 233e7d29aac04505bf4e65f0a39a0c4c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 233e7d29aac04505bf4e65f0a39a0c4c; no voters: 3e07d64466934da4bb3c02262b1c2e38, 43999d813e564dc688fad116090c95ca
I20260709 13:23:44.521986 26059 raft_consensus.cc:2749] T 0cb89d70b4ce4aa8992fb68480bec8a2 P 233e7d29aac04505bf4e65f0a39a0c4c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:23:44.522400 25276 leader_election.cc:336] T 11949f56648c47e0a43c3140af99a304 P 233e7d29aac04505bf4e65f0a39a0c4c [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 43999d813e564dc688fad116090c95ca (127.14.177.129:38155): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:23:44.522886 25276 leader_election.cc:304] T 11949f56648c47e0a43c3140af99a304 P 233e7d29aac04505bf4e65f0a39a0c4c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 233e7d29aac04505bf4e65f0a39a0c4c; no voters: 43999d813e564dc688fad116090c95ca, 9868dce521294d0b816bba7648f4ae5a
I20260709 13:23:44.523510 26059 raft_consensus.cc:2749] T 11949f56648c47e0a43c3140af99a304 P 233e7d29aac04505bf4e65f0a39a0c4c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:23:44.523912 25276 leader_election.cc:336] T b2f14aed05e24a98ad83535029b795dc P 233e7d29aac04505bf4e65f0a39a0c4c [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 43999d813e564dc688fad116090c95ca (127.14.177.129:38155): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:23:44.524345 25276 leader_election.cc:304] T b2f14aed05e24a98ad83535029b795dc P 233e7d29aac04505bf4e65f0a39a0c4c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 233e7d29aac04505bf4e65f0a39a0c4c; no voters: 3e07d64466934da4bb3c02262b1c2e38, 43999d813e564dc688fad116090c95ca
I20260709 13:23:44.525002 26059 raft_consensus.cc:2749] T b2f14aed05e24a98ad83535029b795dc P 233e7d29aac04505bf4e65f0a39a0c4c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:23:44.531497 25202 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.130:42957, user_credentials={real_user=slave}} blocked reactor thread for 70500.7us
I20260709 13:23:44.532287 25202 leader_election.cc:304] T 6242888a4d4146dcacd87d0fd9a832f4 P 43999d813e564dc688fad116090c95ca [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 43999d813e564dc688fad116090c95ca; no voters: 233e7d29aac04505bf4e65f0a39a0c4c, 9868dce521294d0b816bba7648f4ae5a
I20260709 13:23:44.534583 26076 raft_consensus.cc:2749] T 6242888a4d4146dcacd87d0fd9a832f4 P 43999d813e564dc688fad116090c95ca [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:23:44.537978 25454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e5d70664720424d951f269d9d7f4346" candidate_uuid: "43999d813e564dc688fad116090c95ca" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9868dce521294d0b816bba7648f4ae5a"
I20260709 13:23:44.539214 15046 raft_consensus.cc:2272] T 72b56266e8a54776b9113c7a786e1952 P 43999d813e564dc688fad116090c95ca [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:44.541988 15046 tablet_replica.cc:333] T b9d550d881b04735bbf89751952c09ec P 43999d813e564dc688fad116090c95ca: stopping tablet replica
I20260709 13:23:44.542795 15046 raft_consensus.cc:2243] T b9d550d881b04735bbf89751952c09ec P 43999d813e564dc688fad116090c95ca [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:23:44.543748 15046 raft_consensus.cc:2272] T b9d550d881b04735bbf89751952c09ec P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:44.547648 15046 tablet_replica.cc:333] T b155b866f10547fd8239579a81ecf5f9 P 43999d813e564dc688fad116090c95ca: stopping tablet replica
I20260709 13:23:44.548555 15046 raft_consensus.cc:2243] T b155b866f10547fd8239579a81ecf5f9 P 43999d813e564dc688fad116090c95ca [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:23:44.549045 15046 raft_consensus.cc:2272] T b155b866f10547fd8239579a81ecf5f9 P 43999d813e564dc688fad116090c95ca [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:44.560288 25202 leader_election.cc:304] T e9e342ecda4b46619ce12452b2cf7154 P 43999d813e564dc688fad116090c95ca [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 43999d813e564dc688fad116090c95ca; no voters: 233e7d29aac04505bf4e65f0a39a0c4c, 9868dce521294d0b816bba7648f4ae5a
I20260709 13:23:44.562455 15046 tablet_replica.cc:333] T 4d837697cd0f402cacf9db89a5d92652 P 43999d813e564dc688fad116090c95ca: stopping tablet replica
I20260709 13:23:44.563400 15046 raft_consensus.cc:2243] T 4d837697cd0f402cacf9db89a5d92652 P 43999d813e564dc688fad116090c95ca [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:23:44.568331 25130 catalog_manager.cc:5697] T 1f15ec62698c4a8cbd84f2b3d516d2ea P 9868dce521294d0b816bba7648f4ae5a reported cstate change: term changed from 1 to 2, leader changed from 3e07d64466934da4bb3c02262b1c2e38 (127.14.177.131) to 9868dce521294d0b816bba7648f4ae5a (127.14.177.132). New cstate: current_term: 2 leader_uuid: "9868dce521294d0b816bba7648f4ae5a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } health_report { overall_health: UNKNOWN } } }
W20260709 13:23:44.571293 25276 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.129:38155, user_credentials={real_user=slave}} blocked reactor thread for 45941.4us
I20260709 13:23:44.574705 25310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e5d70664720424d951f269d9d7f4346" candidate_uuid: "43999d813e564dc688fad116090c95ca" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "233e7d29aac04505bf4e65f0a39a0c4c"
W20260709 13:23:44.573179 25276 leader_election.cc:336] T 03e9bfbd2f774476a4a377367796a2e8 P 233e7d29aac04505bf4e65f0a39a0c4c [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 43999d813e564dc688fad116090c95ca (127.14.177.129:38155): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:23:44.576720 25276 leader_election.cc:304] T 03e9bfbd2f774476a4a377367796a2e8 P 233e7d29aac04505bf4e65f0a39a0c4c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 233e7d29aac04505bf4e65f0a39a0c4c; no voters: 3e07d64466934da4bb3c02262b1c2e38, 43999d813e564dc688fad116090c95ca
I20260709 13:23:44.577736 26028 raft_consensus.cc:2749] T e9e342ecda4b46619ce12452b2cf7154 P 43999d813e564dc688fad116090c95ca [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:23:44.578783 15046 raft_consensus.cc:2272] T 4d837697cd0f402cacf9db89a5d92652 P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:44.580245 25202 leader_election.cc:304] T 9e5d70664720424d951f269d9d7f4346 P 43999d813e564dc688fad116090c95ca [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 43999d813e564dc688fad116090c95ca; no voters: 233e7d29aac04505bf4e65f0a39a0c4c, 9868dce521294d0b816bba7648f4ae5a
I20260709 13:23:44.581861 15046 tablet_replica.cc:333] T 0ed5edbe32044d1c83b0590dbb899801 P 43999d813e564dc688fad116090c95ca: stopping tablet replica
I20260709 13:23:44.581954 26076 raft_consensus.cc:2749] T 9e5d70664720424d951f269d9d7f4346 P 43999d813e564dc688fad116090c95ca [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:23:44.582906 26059 raft_consensus.cc:2749] T 03e9bfbd2f774476a4a377367796a2e8 P 233e7d29aac04505bf4e65f0a39a0c4c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:23:44.584630 25276 leader_election.cc:336] T 79d4a12ef58140568c60a24ed4c12f83 P 233e7d29aac04505bf4e65f0a39a0c4c [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 43999d813e564dc688fad116090c95ca (127.14.177.129:38155): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:23:44.582949 15046 raft_consensus.cc:2243] T 0ed5edbe32044d1c83b0590dbb899801 P 43999d813e564dc688fad116090c95ca [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:23:44.585111 25276 leader_election.cc:304] T 79d4a12ef58140568c60a24ed4c12f83 P 233e7d29aac04505bf4e65f0a39a0c4c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 233e7d29aac04505bf4e65f0a39a0c4c; no voters: 3e07d64466934da4bb3c02262b1c2e38, 43999d813e564dc688fad116090c95ca
I20260709 13:23:44.585425 15046 raft_consensus.cc:2272] T 0ed5edbe32044d1c83b0590dbb899801 P 43999d813e564dc688fad116090c95ca [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:44.586645 26059 raft_consensus.cc:2749] T 79d4a12ef58140568c60a24ed4c12f83 P 233e7d29aac04505bf4e65f0a39a0c4c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:23:44.588014 25276 leader_election.cc:336] T fa0da2c845dc4059968332f4db2e0aa6 P 233e7d29aac04505bf4e65f0a39a0c4c [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 43999d813e564dc688fad116090c95ca (127.14.177.129:38155): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:23:44.588446 25276 leader_election.cc:304] T fa0da2c845dc4059968332f4db2e0aa6 P 233e7d29aac04505bf4e65f0a39a0c4c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 233e7d29aac04505bf4e65f0a39a0c4c; no voters: 43999d813e564dc688fad116090c95ca, 9868dce521294d0b816bba7648f4ae5a
I20260709 13:23:44.589136 15046 tablet_replica.cc:333] T ce9c60e3922c4219b6e4453fe0259b6a P 43999d813e564dc688fad116090c95ca: stopping tablet replica
I20260709 13:23:44.589221 26059 raft_consensus.cc:2749] T fa0da2c845dc4059968332f4db2e0aa6 P 233e7d29aac04505bf4e65f0a39a0c4c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:23:44.590189 15046 raft_consensus.cc:2243] T ce9c60e3922c4219b6e4453fe0259b6a P 43999d813e564dc688fad116090c95ca [term 2 FOLLOWER]: Raft consensus shutting down.
W20260709 13:23:44.590440 25276 leader_election.cc:336] T a5df750b84e0487fa530161079deb6c2 P 233e7d29aac04505bf4e65f0a39a0c4c [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 43999d813e564dc688fad116090c95ca (127.14.177.129:38155): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:23:44.590745 15046 raft_consensus.cc:2272] T ce9c60e3922c4219b6e4453fe0259b6a P 43999d813e564dc688fad116090c95ca [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:44.590878 25276 leader_election.cc:304] T a5df750b84e0487fa530161079deb6c2 P 233e7d29aac04505bf4e65f0a39a0c4c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 233e7d29aac04505bf4e65f0a39a0c4c; no voters: 43999d813e564dc688fad116090c95ca, 9868dce521294d0b816bba7648f4ae5a
I20260709 13:23:44.594022 26059 raft_consensus.cc:2749] T a5df750b84e0487fa530161079deb6c2 P 233e7d29aac04505bf4e65f0a39a0c4c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:23:44.596868 25276 leader_election.cc:336] T 87085ac2575e4b6a86dd1fc21e6f718b P 233e7d29aac04505bf4e65f0a39a0c4c [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 43999d813e564dc688fad116090c95ca (127.14.177.129:38155): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:23:44.597239 15046 tablet_replica.cc:333] T 6242888a4d4146dcacd87d0fd9a832f4 P 43999d813e564dc688fad116090c95ca: stopping tablet replica
I20260709 13:23:44.597348 25276 leader_election.cc:304] T 87085ac2575e4b6a86dd1fc21e6f718b P 233e7d29aac04505bf4e65f0a39a0c4c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 233e7d29aac04505bf4e65f0a39a0c4c; no voters: 3e07d64466934da4bb3c02262b1c2e38, 43999d813e564dc688fad116090c95ca
I20260709 13:23:44.598272 15046 raft_consensus.cc:2243] T 6242888a4d4146dcacd87d0fd9a832f4 P 43999d813e564dc688fad116090c95ca [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:23:44.602212 26059 raft_consensus.cc:2749] T 87085ac2575e4b6a86dd1fc21e6f718b P 233e7d29aac04505bf4e65f0a39a0c4c [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:23:44.602856 15046 raft_consensus.cc:2272] T 6242888a4d4146dcacd87d0fd9a832f4 P 43999d813e564dc688fad116090c95ca [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:23:44.603519 25276 leader_election.cc:336] T 8ef061028f38450e83324ac78cdaa14c P 233e7d29aac04505bf4e65f0a39a0c4c [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 43999d813e564dc688fad116090c95ca (127.14.177.129:38155): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:23:44.604005 25276 leader_election.cc:304] T 8ef061028f38450e83324ac78cdaa14c P 233e7d29aac04505bf4e65f0a39a0c4c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 233e7d29aac04505bf4e65f0a39a0c4c; no voters: 43999d813e564dc688fad116090c95ca, 9868dce521294d0b816bba7648f4ae5a
I20260709 13:23:44.606521 26059 raft_consensus.cc:2749] T 8ef061028f38450e83324ac78cdaa14c P 233e7d29aac04505bf4e65f0a39a0c4c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:23:44.607817 25276 leader_election.cc:336] T 441d780c3945463f92f31170f31e37a8 P 233e7d29aac04505bf4e65f0a39a0c4c [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 43999d813e564dc688fad116090c95ca (127.14.177.129:38155): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:23:44.608299 15046 tablet_replica.cc:333] T 87085ac2575e4b6a86dd1fc21e6f718b P 43999d813e564dc688fad116090c95ca: stopping tablet replica
I20260709 13:23:44.609262 15046 raft_consensus.cc:2243] T 87085ac2575e4b6a86dd1fc21e6f718b P 43999d813e564dc688fad116090c95ca [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:23:44.609825 15046 raft_consensus.cc:2272] T 87085ac2575e4b6a86dd1fc21e6f718b P 43999d813e564dc688fad116090c95ca [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:44.608233 25276 leader_election.cc:304] T 441d780c3945463f92f31170f31e37a8 P 233e7d29aac04505bf4e65f0a39a0c4c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 233e7d29aac04505bf4e65f0a39a0c4c; no voters: 43999d813e564dc688fad116090c95ca, 9868dce521294d0b816bba7648f4ae5a
I20260709 13:23:44.613833 15046 tablet_replica.cc:333] T a5df750b84e0487fa530161079deb6c2 P 43999d813e564dc688fad116090c95ca: stopping tablet replica
I20260709 13:23:44.614934 15046 raft_consensus.cc:2243] T a5df750b84e0487fa530161079deb6c2 P 43999d813e564dc688fad116090c95ca [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:23:44.615928 15046 raft_consensus.cc:2272] T a5df750b84e0487fa530161079deb6c2 P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:44.616000 26059 raft_consensus.cc:2749] T 441d780c3945463f92f31170f31e37a8 P 233e7d29aac04505bf4e65f0a39a0c4c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:23:44.617314 25276 leader_election.cc:336] T beacc815ef954fb692e90544bfa93f05 P 233e7d29aac04505bf4e65f0a39a0c4c [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 43999d813e564dc688fad116090c95ca (127.14.177.129:38155): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:23:44.617726 25276 leader_election.cc:304] T beacc815ef954fb692e90544bfa93f05 P 233e7d29aac04505bf4e65f0a39a0c4c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 233e7d29aac04505bf4e65f0a39a0c4c; no voters: 3e07d64466934da4bb3c02262b1c2e38, 43999d813e564dc688fad116090c95ca
I20260709 13:23:44.623386 26059 raft_consensus.cc:2749] T beacc815ef954fb692e90544bfa93f05 P 233e7d29aac04505bf4e65f0a39a0c4c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:23:44.626143 25276 leader_election.cc:336] T e15820fa8a7d4e1fb6ad23a64ca22f90 P 233e7d29aac04505bf4e65f0a39a0c4c [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 43999d813e564dc688fad116090c95ca (127.14.177.129:38155): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:23:44.626605 25276 leader_election.cc:304] T e15820fa8a7d4e1fb6ad23a64ca22f90 P 233e7d29aac04505bf4e65f0a39a0c4c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 233e7d29aac04505bf4e65f0a39a0c4c; no voters: 3e07d64466934da4bb3c02262b1c2e38, 43999d813e564dc688fad116090c95ca
I20260709 13:23:44.628561 26059 raft_consensus.cc:2749] T e15820fa8a7d4e1fb6ad23a64ca22f90 P 233e7d29aac04505bf4e65f0a39a0c4c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:23:44.629835 15046 tablet_replica.cc:333] T 1550ed2d2a764c08adffae679599ba41 P 43999d813e564dc688fad116090c95ca: stopping tablet replica
I20260709 13:23:44.630909 15046 raft_consensus.cc:2243] T 1550ed2d2a764c08adffae679599ba41 P 43999d813e564dc688fad116090c95ca [term 1 LEADER]: Raft consensus shutting down.
W20260709 13:23:44.629881 25276 leader_election.cc:336] T 9e5d70664720424d951f269d9d7f4346 P 233e7d29aac04505bf4e65f0a39a0c4c [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 43999d813e564dc688fad116090c95ca (127.14.177.129:38155): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:23:44.631736 25276 leader_election.cc:304] T 9e5d70664720424d951f269d9d7f4346 P 233e7d29aac04505bf4e65f0a39a0c4c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 233e7d29aac04505bf4e65f0a39a0c4c; no voters: 43999d813e564dc688fad116090c95ca, 9868dce521294d0b816bba7648f4ae5a
I20260709 13:23:44.632119 15046 raft_consensus.cc:2272] T 1550ed2d2a764c08adffae679599ba41 P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:44.632436 26059 raft_consensus.cc:2749] T 9e5d70664720424d951f269d9d7f4346 P 233e7d29aac04505bf4e65f0a39a0c4c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:23:44.639918 15046 tablet_replica.cc:333] T c902bbc7ec0848ba82a04870eab1e7f5 P 43999d813e564dc688fad116090c95ca: stopping tablet replica
I20260709 13:23:44.640933 15046 raft_consensus.cc:2243] T c902bbc7ec0848ba82a04870eab1e7f5 P 43999d813e564dc688fad116090c95ca [term 2 FOLLOWER]: Raft consensus shutting down.
W20260709 13:23:44.636126 25276 leader_election.cc:336] T 13bb3a23315a41c7a40e10b954b15c8e P 233e7d29aac04505bf4e65f0a39a0c4c [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 43999d813e564dc688fad116090c95ca (127.14.177.129:38155): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:23:44.647555 25276 leader_election.cc:304] T 13bb3a23315a41c7a40e10b954b15c8e P 233e7d29aac04505bf4e65f0a39a0c4c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 233e7d29aac04505bf4e65f0a39a0c4c; no voters: 3e07d64466934da4bb3c02262b1c2e38, 43999d813e564dc688fad116090c95ca
I20260709 13:23:44.648303 15046 raft_consensus.cc:2272] T c902bbc7ec0848ba82a04870eab1e7f5 P 43999d813e564dc688fad116090c95ca [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:44.648342 26059 raft_consensus.cc:2749] T 13bb3a23315a41c7a40e10b954b15c8e P 233e7d29aac04505bf4e65f0a39a0c4c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:23:44.650298 25276 leader_election.cc:336] T b155b866f10547fd8239579a81ecf5f9 P 233e7d29aac04505bf4e65f0a39a0c4c [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 43999d813e564dc688fad116090c95ca (127.14.177.129:38155): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:23:44.650705 25276 leader_election.cc:304] T b155b866f10547fd8239579a81ecf5f9 P 233e7d29aac04505bf4e65f0a39a0c4c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 233e7d29aac04505bf4e65f0a39a0c4c; no voters: 3e07d64466934da4bb3c02262b1c2e38, 43999d813e564dc688fad116090c95ca
I20260709 13:23:44.651450 26059 raft_consensus.cc:2749] T b155b866f10547fd8239579a81ecf5f9 P 233e7d29aac04505bf4e65f0a39a0c4c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:23:44.651553 15046 tablet_replica.cc:333] T ce98a3f3b3eb4c63bcaa355ec7cd64a4 P 43999d813e564dc688fad116090c95ca: stopping tablet replica
I20260709 13:23:44.652637 15046 raft_consensus.cc:2243] T ce98a3f3b3eb4c63bcaa355ec7cd64a4 P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Raft consensus shutting down.
W20260709 13:23:44.653033 25276 leader_election.cc:336] T 434a54fcee7d4aeeab42f739855c0e8a P 233e7d29aac04505bf4e65f0a39a0c4c [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 43999d813e564dc688fad116090c95ca (127.14.177.129:38155): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:23:44.653375 15046 raft_consensus.cc:2272] T ce98a3f3b3eb4c63bcaa355ec7cd64a4 P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:44.653602 25276 leader_election.cc:304] T 434a54fcee7d4aeeab42f739855c0e8a P 233e7d29aac04505bf4e65f0a39a0c4c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 233e7d29aac04505bf4e65f0a39a0c4c; no voters: 3e07d64466934da4bb3c02262b1c2e38, 43999d813e564dc688fad116090c95ca
I20260709 13:23:44.654362 26059 raft_consensus.cc:2749] T 434a54fcee7d4aeeab42f739855c0e8a P 233e7d29aac04505bf4e65f0a39a0c4c [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:23:44.656658 15046 tablet_replica.cc:333] T 51a11db644124b63bdf5f457c967473d P 43999d813e564dc688fad116090c95ca: stopping tablet replica
I20260709 13:23:44.674397 15046 raft_consensus.cc:2243] T 51a11db644124b63bdf5f457c967473d P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:23:44.675050 15046 raft_consensus.cc:2272] T 51a11db644124b63bdf5f457c967473d P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:44.681303 15046 tablet_replica.cc:333] T e9e342ecda4b46619ce12452b2cf7154 P 43999d813e564dc688fad116090c95ca: stopping tablet replica
I20260709 13:23:44.689667 15046 raft_consensus.cc:2243] T e9e342ecda4b46619ce12452b2cf7154 P 43999d813e564dc688fad116090c95ca [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:23:44.690267 15046 raft_consensus.cc:2272] T e9e342ecda4b46619ce12452b2cf7154 P 43999d813e564dc688fad116090c95ca [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:44.693275 15046 tablet_replica.cc:333] T 9ffa42c3b2ff48bc87310e9ab5d218ce P 43999d813e564dc688fad116090c95ca: stopping tablet replica
I20260709 13:23:44.696257 15046 raft_consensus.cc:2243] T 9ffa42c3b2ff48bc87310e9ab5d218ce P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:23:44.696915 15046 raft_consensus.cc:2272] T 9ffa42c3b2ff48bc87310e9ab5d218ce P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:44.703065 15046 tablet_replica.cc:333] T 11949f56648c47e0a43c3140af99a304 P 43999d813e564dc688fad116090c95ca: stopping tablet replica
I20260709 13:23:44.704039 15046 raft_consensus.cc:2243] T 11949f56648c47e0a43c3140af99a304 P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:23:44.704602 15046 raft_consensus.cc:2272] T 11949f56648c47e0a43c3140af99a304 P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:44.707026 15046 tablet_replica.cc:333] T beacc815ef954fb692e90544bfa93f05 P 43999d813e564dc688fad116090c95ca: stopping tablet replica
I20260709 13:23:44.707859 15046 raft_consensus.cc:2243] T beacc815ef954fb692e90544bfa93f05 P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:23:44.708384 15046 raft_consensus.cc:2272] T beacc815ef954fb692e90544bfa93f05 P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:44.711068 15046 tablet_replica.cc:333] T f1ac2f4d2fe2424bb470df42199219e2 P 43999d813e564dc688fad116090c95ca: stopping tablet replica
I20260709 13:23:44.711928 15046 raft_consensus.cc:2243] T f1ac2f4d2fe2424bb470df42199219e2 P 43999d813e564dc688fad116090c95ca [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:23:44.713024 15046 raft_consensus.cc:2272] T f1ac2f4d2fe2424bb470df42199219e2 P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:23:44.715154 25276 consensus_peers.cc:597] T 0d29b499fa4c41f1b974782cd0c9c1c2 P 233e7d29aac04505bf4e65f0a39a0c4c -> Peer 43999d813e564dc688fad116090c95ca (127.14.177.129:38155): Couldn't send request to peer 43999d813e564dc688fad116090c95ca. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:23:44.715844 25276 leader_election.cc:336] T ce9c60e3922c4219b6e4453fe0259b6a P 233e7d29aac04505bf4e65f0a39a0c4c [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 43999d813e564dc688fad116090c95ca (127.14.177.129:38155): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:23:44.716460 25276 leader_election.cc:304] T ce9c60e3922c4219b6e4453fe0259b6a P 233e7d29aac04505bf4e65f0a39a0c4c [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 233e7d29aac04505bf4e65f0a39a0c4c; no voters: 43999d813e564dc688fad116090c95ca, 9868dce521294d0b816bba7648f4ae5a
I20260709 13:23:44.717190 26059 raft_consensus.cc:2749] T ce9c60e3922c4219b6e4453fe0259b6a P 233e7d29aac04505bf4e65f0a39a0c4c [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:23:44.727095 15046 tablet_replica.cc:333] T 9e5d70664720424d951f269d9d7f4346 P 43999d813e564dc688fad116090c95ca: stopping tablet replica
I20260709 13:23:44.728922 15046 raft_consensus.cc:2243] T 9e5d70664720424d951f269d9d7f4346 P 43999d813e564dc688fad116090c95ca [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:23:44.729488 15046 raft_consensus.cc:2272] T 9e5d70664720424d951f269d9d7f4346 P 43999d813e564dc688fad116090c95ca [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:44.732445 15046 tablet_replica.cc:333] T b2f14aed05e24a98ad83535029b795dc P 43999d813e564dc688fad116090c95ca: stopping tablet replica
I20260709 13:23:44.733254 15046 raft_consensus.cc:2243] T b2f14aed05e24a98ad83535029b795dc P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:23:44.738404 15046 raft_consensus.cc:2272] T b2f14aed05e24a98ad83535029b795dc P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:44.741660 15046 tablet_replica.cc:333] T 441d780c3945463f92f31170f31e37a8 P 43999d813e564dc688fad116090c95ca: stopping tablet replica
I20260709 13:23:44.742719 15046 raft_consensus.cc:2243] T 441d780c3945463f92f31170f31e37a8 P 43999d813e564dc688fad116090c95ca [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:23:44.743954 15046 raft_consensus.cc:2272] T 441d780c3945463f92f31170f31e37a8 P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:44.746913 15046 tablet_replica.cc:333] T d0a0e4d32ce943fbb773bb840863647b P 43999d813e564dc688fad116090c95ca: stopping tablet replica
I20260709 13:23:44.747750 15046 raft_consensus.cc:2243] T d0a0e4d32ce943fbb773bb840863647b P 43999d813e564dc688fad116090c95ca [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:23:44.748914 15046 raft_consensus.cc:2272] T d0a0e4d32ce943fbb773bb840863647b P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:44.751976 15046 tablet_replica.cc:333] T 0e71b6f5c69b4b35a8ac51c22c7d5f0d P 43999d813e564dc688fad116090c95ca: stopping tablet replica
I20260709 13:23:44.752836 15046 raft_consensus.cc:2243] T 0e71b6f5c69b4b35a8ac51c22c7d5f0d P 43999d813e564dc688fad116090c95ca [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:23:44.753947 15046 raft_consensus.cc:2272] T 0e71b6f5c69b4b35a8ac51c22c7d5f0d P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:44.756891 15046 tablet_replica.cc:333] T b7a891bfc89e4e259f53da6cb16f4989 P 43999d813e564dc688fad116090c95ca: stopping tablet replica
I20260709 13:23:44.757735 15046 raft_consensus.cc:2243] T b7a891bfc89e4e259f53da6cb16f4989 P 43999d813e564dc688fad116090c95ca [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:23:44.758349 15046 raft_consensus.cc:2272] T b7a891bfc89e4e259f53da6cb16f4989 P 43999d813e564dc688fad116090c95ca [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:44.761672 15046 tablet_replica.cc:333] T 28c0b6fc9bd64d60828d705c6a9ab1e4 P 43999d813e564dc688fad116090c95ca: stopping tablet replica
I20260709 13:23:44.762603 15046 raft_consensus.cc:2243] T 28c0b6fc9bd64d60828d705c6a9ab1e4 P 43999d813e564dc688fad116090c95ca [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:23:44.763288 15046 raft_consensus.cc:2272] T 28c0b6fc9bd64d60828d705c6a9ab1e4 P 43999d813e564dc688fad116090c95ca [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:44.766328 15046 tablet_replica.cc:333] T 13bb3a23315a41c7a40e10b954b15c8e P 43999d813e564dc688fad116090c95ca: stopping tablet replica
I20260709 13:23:44.767143 15046 raft_consensus.cc:2243] T 13bb3a23315a41c7a40e10b954b15c8e P 43999d813e564dc688fad116090c95ca [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:23:44.768242 15046 raft_consensus.cc:2272] T 13bb3a23315a41c7a40e10b954b15c8e P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:44.771135 15046 tablet_replica.cc:333] T 0cb89d70b4ce4aa8992fb68480bec8a2 P 43999d813e564dc688fad116090c95ca: stopping tablet replica
I20260709 13:23:44.772053 15046 raft_consensus.cc:2243] T 0cb89d70b4ce4aa8992fb68480bec8a2 P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:23:44.772603 15046 raft_consensus.cc:2272] T 0cb89d70b4ce4aa8992fb68480bec8a2 P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:44.775333 15046 tablet_replica.cc:333] T 8ef061028f38450e83324ac78cdaa14c P 43999d813e564dc688fad116090c95ca: stopping tablet replica
I20260709 13:23:44.776178 15046 raft_consensus.cc:2243] T 8ef061028f38450e83324ac78cdaa14c P 43999d813e564dc688fad116090c95ca [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:23:44.777163 15046 raft_consensus.cc:2272] T 8ef061028f38450e83324ac78cdaa14c P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:44.779851 15046 tablet_replica.cc:333] T b7ff8dde9f5d4c49b9203d902f6b94c4 P 43999d813e564dc688fad116090c95ca: stopping tablet replica
I20260709 13:23:44.780632 15046 raft_consensus.cc:2243] T b7ff8dde9f5d4c49b9203d902f6b94c4 P 43999d813e564dc688fad116090c95ca [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:23:44.781662 15046 raft_consensus.cc:2272] T b7ff8dde9f5d4c49b9203d902f6b94c4 P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:44.784435 15046 tablet_replica.cc:333] T 0c4f6b7401db47d59a4f34ed0e8b5350 P 43999d813e564dc688fad116090c95ca: stopping tablet replica
I20260709 13:23:44.785485 15046 raft_consensus.cc:2243] T 0c4f6b7401db47d59a4f34ed0e8b5350 P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:23:44.786008 15046 raft_consensus.cc:2272] T 0c4f6b7401db47d59a4f34ed0e8b5350 P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:44.788911 15046 tablet_replica.cc:333] T 91ac744179de48e5bb924b6dfd9f9c9b P 43999d813e564dc688fad116090c95ca: stopping tablet replica
I20260709 13:23:44.789732 15046 raft_consensus.cc:2243] T 91ac744179de48e5bb924b6dfd9f9c9b P 43999d813e564dc688fad116090c95ca [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:23:44.790323 15046 raft_consensus.cc:2272] T 91ac744179de48e5bb924b6dfd9f9c9b P 43999d813e564dc688fad116090c95ca [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:44.793247 15046 tablet_replica.cc:333] T 280d65245f4f4358b54bef93e18aca92 P 43999d813e564dc688fad116090c95ca: stopping tablet replica
I20260709 13:23:44.794013 15046 raft_consensus.cc:2243] T 280d65245f4f4358b54bef93e18aca92 P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:23:44.794579 15046 raft_consensus.cc:2272] T 280d65245f4f4358b54bef93e18aca92 P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:44.797276 15046 tablet_replica.cc:333] T 1f15ec62698c4a8cbd84f2b3d516d2ea P 43999d813e564dc688fad116090c95ca: stopping tablet replica
I20260709 13:23:44.798115 15046 raft_consensus.cc:2243] T 1f15ec62698c4a8cbd84f2b3d516d2ea P 43999d813e564dc688fad116090c95ca [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:23:44.798667 15046 raft_consensus.cc:2272] T 1f15ec62698c4a8cbd84f2b3d516d2ea P 43999d813e564dc688fad116090c95ca [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:44.801630 15046 tablet_replica.cc:333] T b85a238bbc3b4621881603a534c7bd9b P 43999d813e564dc688fad116090c95ca: stopping tablet replica
I20260709 13:23:44.802547 15046 raft_consensus.cc:2243] T b85a238bbc3b4621881603a534c7bd9b P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:23:44.803100 15046 raft_consensus.cc:2272] T b85a238bbc3b4621881603a534c7bd9b P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:44.806037 15046 tablet_replica.cc:333] T d053b9ae38de4173a0aded4da494bd98 P 43999d813e564dc688fad116090c95ca: stopping tablet replica
I20260709 13:23:44.806813 15046 raft_consensus.cc:2243] T d053b9ae38de4173a0aded4da494bd98 P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:23:44.807344 15046 raft_consensus.cc:2272] T d053b9ae38de4173a0aded4da494bd98 P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:44.810144 15046 tablet_replica.cc:333] T 434a54fcee7d4aeeab42f739855c0e8a P 43999d813e564dc688fad116090c95ca: stopping tablet replica
I20260709 13:23:44.810874 15046 raft_consensus.cc:2243] T 434a54fcee7d4aeeab42f739855c0e8a P 43999d813e564dc688fad116090c95ca [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:23:44.811872 15046 raft_consensus.cc:2272] T 434a54fcee7d4aeeab42f739855c0e8a P 43999d813e564dc688fad116090c95ca [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:44.814610 15046 tablet_replica.cc:333] T e15820fa8a7d4e1fb6ad23a64ca22f90 P 43999d813e564dc688fad116090c95ca: stopping tablet replica
I20260709 13:23:44.815376 15046 raft_consensus.cc:2243] T e15820fa8a7d4e1fb6ad23a64ca22f90 P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:23:44.815878 15046 raft_consensus.cc:2272] T e15820fa8a7d4e1fb6ad23a64ca22f90 P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:44.818614 15046 tablet_replica.cc:333] T 03e9bfbd2f774476a4a377367796a2e8 P 43999d813e564dc688fad116090c95ca: stopping tablet replica
I20260709 13:23:44.819407 15046 raft_consensus.cc:2243] T 03e9bfbd2f774476a4a377367796a2e8 P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:23:44.820087 15046 raft_consensus.cc:2272] T 03e9bfbd2f774476a4a377367796a2e8 P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:44.822788 15046 tablet_replica.cc:333] T 579da2d6aabb4ae9a2b1802e1358068b P 43999d813e564dc688fad116090c95ca: stopping tablet replica
I20260709 13:23:44.823498 15046 raft_consensus.cc:2243] T 579da2d6aabb4ae9a2b1802e1358068b P 43999d813e564dc688fad116090c95ca [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:23:44.824515 15046 raft_consensus.cc:2272] T 579da2d6aabb4ae9a2b1802e1358068b P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:44.827540 15046 tablet_replica.cc:333] T 5c7e7acbb09f4d49828fa652ca6abaac P 43999d813e564dc688fad116090c95ca: stopping tablet replica
I20260709 13:23:44.828225 15046 raft_consensus.cc:2243] T 5c7e7acbb09f4d49828fa652ca6abaac P 43999d813e564dc688fad116090c95ca [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:23:44.828737 15046 raft_consensus.cc:2272] T 5c7e7acbb09f4d49828fa652ca6abaac P 43999d813e564dc688fad116090c95ca [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:44.832348 15046 tablet_replica.cc:333] T fa0da2c845dc4059968332f4db2e0aa6 P 43999d813e564dc688fad116090c95ca: stopping tablet replica
I20260709 13:23:44.834518 15046 raft_consensus.cc:2243] T fa0da2c845dc4059968332f4db2e0aa6 P 43999d813e564dc688fad116090c95ca [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:23:44.835570 15046 raft_consensus.cc:2272] T fa0da2c845dc4059968332f4db2e0aa6 P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:44.839383 25308 raft_consensus.cc:1275] T 89a26dd3ac414f6081b98fa32c2f346f P 233e7d29aac04505bf4e65f0a39a0c4c [term 2 FOLLOWER]: Refusing update from remote peer 3e07d64466934da4bb3c02262b1c2e38: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:23:44.845259 26058 consensus_queue.cc:1048] T 89a26dd3ac414f6081b98fa32c2f346f P 3e07d64466934da4bb3c02262b1c2e38 [LEADER]: Connected to new peer: Peer: permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:23:44.851136 15046 tablet_replica.cc:333] T 3a3317f4c3514df1b2bb8e44a86be92f P 43999d813e564dc688fad116090c95ca: stopping tablet replica
I20260709 13:23:44.852512 15046 raft_consensus.cc:2243] T 3a3317f4c3514df1b2bb8e44a86be92f P 43999d813e564dc688fad116090c95ca [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:23:44.853760 15046 raft_consensus.cc:2272] T 3a3317f4c3514df1b2bb8e44a86be92f P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:44.856745 15046 tablet_replica.cc:333] T 821a5866429f4173820fb64336abee02 P 43999d813e564dc688fad116090c95ca: stopping tablet replica
I20260709 13:23:44.857661 15046 raft_consensus.cc:2243] T 821a5866429f4173820fb64336abee02 P 43999d813e564dc688fad116090c95ca [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:23:44.872686 25382 raft_consensus.cc:1275] T 1f15ec62698c4a8cbd84f2b3d516d2ea P 3e07d64466934da4bb3c02262b1c2e38 [term 2 FOLLOWER]: Refusing update from remote peer 9868dce521294d0b816bba7648f4ae5a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:23:44.888952 26069 consensus_queue.cc:1048] T 1f15ec62698c4a8cbd84f2b3d516d2ea P 9868dce521294d0b816bba7648f4ae5a [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.009s
I20260709 13:23:44.885524 15046 raft_consensus.cc:2272] T 821a5866429f4173820fb64336abee02 P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:44.909747 15046 tablet_replica.cc:333] T 79d4a12ef58140568c60a24ed4c12f83 P 43999d813e564dc688fad116090c95ca: stopping tablet replica
I20260709 13:23:44.910624 15046 raft_consensus.cc:2243] T 79d4a12ef58140568c60a24ed4c12f83 P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:23:44.912902 25450 raft_consensus.cc:1275] T 89a26dd3ac414f6081b98fa32c2f346f P 9868dce521294d0b816bba7648f4ae5a [term 2 FOLLOWER]: Refusing update from remote peer 3e07d64466934da4bb3c02262b1c2e38: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:23:44.911224 15046 raft_consensus.cc:2272] T 79d4a12ef58140568c60a24ed4c12f83 P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:44.927338 26031 consensus_queue.cc:1048] T 89a26dd3ac414f6081b98fa32c2f346f P 3e07d64466934da4bb3c02262b1c2e38 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:23:44.937252 15046 tablet_replica.cc:333] T 0d29b499fa4c41f1b974782cd0c9c1c2 P 43999d813e564dc688fad116090c95ca: stopping tablet replica
I20260709 13:23:44.938151 15046 raft_consensus.cc:2243] T 0d29b499fa4c41f1b974782cd0c9c1c2 P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:23:44.947433 15046 raft_consensus.cc:2272] T 0d29b499fa4c41f1b974782cd0c9c1c2 P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:23:44.989624 25424 consensus_peers.cc:597] T 1f15ec62698c4a8cbd84f2b3d516d2ea P 9868dce521294d0b816bba7648f4ae5a -> Peer 43999d813e564dc688fad116090c95ca (127.14.177.129:38155): Couldn't send request to peer 43999d813e564dc688fad116090c95ca. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:23:45.179662 15046 tablet_server.cc:196] TabletServer@127.14.177.129:0 shutdown complete.
I20260709 13:23:45.268607 15046 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:23:45.280200 26102 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:23:45.284358 26103 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:23:45.297711 26105 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:23:45.298597 15046 server_base.cc:1061] running on GCE node
I20260709 13:23:45.303627 15046 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:23:45.303938 15046 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:23:45.304203 15046 hybrid_clock.cc:648] HybridClock initialized: now 1783603425304181 us; error 0 us; skew 500 ppm
I20260709 13:23:45.304991 15046 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:23:45.316457 15046 webserver.cc:533] Webserver started at http://127.14.177.129:35817/ using document root <none> and password file <none>
I20260709 13:23:45.317080 15046 fs_manager.cc:362] Metadata directory not provided
I20260709 13:23:45.317339 15046 fs_manager.cc:365] Using existing metadata directory in first data directory
I20260709 13:23:45.321961 15046 fs_manager.cc:714] Time spent opening directory manager: real 0.004s	user 0.001s	sys 0.004s
I20260709 13:23:45.343492 26110 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:23:45.344583 15046 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.001s
I20260709 13:23:45.345049 15046 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskdH_Ma7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783603093801613-15046-0/minicluster-data/ts-0-root
uuid: "43999d813e564dc688fad116090c95ca"
format_stamp: "Formatted at 2026-07-09 13:23:29 on dist-test-slave-v5z2"
I20260709 13:23:45.345611 15046 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdH_Ma7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783603093801613-15046-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskdH_Ma7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783603093801613-15046-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskdH_Ma7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783603093801613-15046-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:23:45.379949 15046 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:23:45.379794 26046 raft_consensus.cc:493] T ad32097cd4dc48b9a69595f1651abd07 P 3e07d64466934da4bb3c02262b1c2e38 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:23:45.380663 26046 raft_consensus.cc:515] T ad32097cd4dc48b9a69595f1651abd07 P 3e07d64466934da4bb3c02262b1c2e38 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } }
I20260709 13:23:45.382139 15046 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:23:45.384610 25310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad32097cd4dc48b9a69595f1651abd07" candidate_uuid: "3e07d64466934da4bb3c02262b1c2e38" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" is_pre_election: true
I20260709 13:23:45.385524 25310 raft_consensus.cc:2468] T ad32097cd4dc48b9a69595f1651abd07 P 233e7d29aac04505bf4e65f0a39a0c4c [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3e07d64466934da4bb3c02262b1c2e38 in term 2.
I20260709 13:23:45.386696 26046 leader_election.cc:290] T ad32097cd4dc48b9a69595f1651abd07 P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 233e7d29aac04505bf4e65f0a39a0c4c (127.14.177.130:42957), 9868dce521294d0b816bba7648f4ae5a (127.14.177.132:33683)
I20260709 13:23:45.388513 25351 leader_election.cc:304] T ad32097cd4dc48b9a69595f1651abd07 P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 233e7d29aac04505bf4e65f0a39a0c4c, 3e07d64466934da4bb3c02262b1c2e38; no voters: 
I20260709 13:23:45.388746 25450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad32097cd4dc48b9a69595f1651abd07" candidate_uuid: "3e07d64466934da4bb3c02262b1c2e38" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9868dce521294d0b816bba7648f4ae5a" is_pre_election: true
I20260709 13:23:45.389722 25450 raft_consensus.cc:2468] T ad32097cd4dc48b9a69595f1651abd07 P 9868dce521294d0b816bba7648f4ae5a [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3e07d64466934da4bb3c02262b1c2e38 in term 1.
I20260709 13:23:45.391870 15046 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:23:45.393352 26041 raft_consensus.cc:2804] T ad32097cd4dc48b9a69595f1651abd07 P 3e07d64466934da4bb3c02262b1c2e38 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:23:45.393792 26041 raft_consensus.cc:493] T ad32097cd4dc48b9a69595f1651abd07 P 3e07d64466934da4bb3c02262b1c2e38 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:23:45.394169 26041 raft_consensus.cc:3060] T ad32097cd4dc48b9a69595f1651abd07 P 3e07d64466934da4bb3c02262b1c2e38 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:23:45.444830 26118 ts_tablet_manager.cc:548] Loading tablet metadata (0/45 complete)
I20260709 13:23:45.445243 26041 raft_consensus.cc:515] T ad32097cd4dc48b9a69595f1651abd07 P 3e07d64466934da4bb3c02262b1c2e38 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } }
I20260709 13:23:45.456590 25310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad32097cd4dc48b9a69595f1651abd07" candidate_uuid: "3e07d64466934da4bb3c02262b1c2e38" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "233e7d29aac04505bf4e65f0a39a0c4c"
I20260709 13:23:45.457304 25310 raft_consensus.cc:3060] T ad32097cd4dc48b9a69595f1651abd07 P 233e7d29aac04505bf4e65f0a39a0c4c [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:23:45.460107 26041 leader_election.cc:290] T ad32097cd4dc48b9a69595f1651abd07 P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 3 election: Requested vote from peers 233e7d29aac04505bf4e65f0a39a0c4c (127.14.177.130:42957), 9868dce521294d0b816bba7648f4ae5a (127.14.177.132:33683)
I20260709 13:23:45.461082 25450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad32097cd4dc48b9a69595f1651abd07" candidate_uuid: "3e07d64466934da4bb3c02262b1c2e38" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9868dce521294d0b816bba7648f4ae5a"
I20260709 13:23:45.461881 25450 raft_consensus.cc:3060] T ad32097cd4dc48b9a69595f1651abd07 P 9868dce521294d0b816bba7648f4ae5a [term 1 FOLLOWER]: Advancing to term 3
I20260709 13:23:45.466820 25310 raft_consensus.cc:2468] T ad32097cd4dc48b9a69595f1651abd07 P 233e7d29aac04505bf4e65f0a39a0c4c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3e07d64466934da4bb3c02262b1c2e38 in term 3.
I20260709 13:23:45.468530 25351 leader_election.cc:304] T ad32097cd4dc48b9a69595f1651abd07 P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 233e7d29aac04505bf4e65f0a39a0c4c, 3e07d64466934da4bb3c02262b1c2e38; no voters: 
I20260709 13:23:45.469986 26041 raft_consensus.cc:2804] T ad32097cd4dc48b9a69595f1651abd07 P 3e07d64466934da4bb3c02262b1c2e38 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:23:45.470237 25450 raft_consensus.cc:2468] T ad32097cd4dc48b9a69595f1651abd07 P 9868dce521294d0b816bba7648f4ae5a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3e07d64466934da4bb3c02262b1c2e38 in term 3.
I20260709 13:23:45.470934 26041 raft_consensus.cc:697] T ad32097cd4dc48b9a69595f1651abd07 P 3e07d64466934da4bb3c02262b1c2e38 [term 3 LEADER]: Becoming Leader. State: Replica: 3e07d64466934da4bb3c02262b1c2e38, State: Running, Role: LEADER
I20260709 13:23:45.472000 26041 consensus_queue.cc:237] T ad32097cd4dc48b9a69595f1651abd07 P 3e07d64466934da4bb3c02262b1c2e38 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } }
I20260709 13:23:45.525210 25134 catalog_manager.cc:5697] T ad32097cd4dc48b9a69595f1651abd07 P 3e07d64466934da4bb3c02262b1c2e38 reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "3e07d64466934da4bb3c02262b1c2e38" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } health_report { overall_health: UNKNOWN } } }
I20260709 13:23:45.669207 26022 raft_consensus.cc:493] T cfad575709c943cba40ccab278350770 P 9868dce521294d0b816bba7648f4ae5a [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:23:45.669865 26022 raft_consensus.cc:515] T cfad575709c943cba40ccab278350770 P 9868dce521294d0b816bba7648f4ae5a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } }
I20260709 13:23:45.672258 26022 leader_election.cc:290] T cfad575709c943cba40ccab278350770 P 9868dce521294d0b816bba7648f4ae5a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 233e7d29aac04505bf4e65f0a39a0c4c (127.14.177.130:42957), 3e07d64466934da4bb3c02262b1c2e38 (127.14.177.131:41539)
I20260709 13:23:45.673429 25307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfad575709c943cba40ccab278350770" candidate_uuid: "9868dce521294d0b816bba7648f4ae5a" candidate_term: 4 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" is_pre_election: true
I20260709 13:23:45.673544 25382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfad575709c943cba40ccab278350770" candidate_uuid: "9868dce521294d0b816bba7648f4ae5a" candidate_term: 4 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "3e07d64466934da4bb3c02262b1c2e38" is_pre_election: true
I20260709 13:23:45.675086 25382 raft_consensus.cc:2468] T cfad575709c943cba40ccab278350770 P 3e07d64466934da4bb3c02262b1c2e38 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9868dce521294d0b816bba7648f4ae5a in term 3.
I20260709 13:23:45.677443 25422 leader_election.cc:304] T cfad575709c943cba40ccab278350770 P 9868dce521294d0b816bba7648f4ae5a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3e07d64466934da4bb3c02262b1c2e38, 9868dce521294d0b816bba7648f4ae5a; no voters: 233e7d29aac04505bf4e65f0a39a0c4c
I20260709 13:23:45.678694 26022 raft_consensus.cc:2804] T cfad575709c943cba40ccab278350770 P 9868dce521294d0b816bba7648f4ae5a [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:23:45.679203 26022 raft_consensus.cc:493] T cfad575709c943cba40ccab278350770 P 9868dce521294d0b816bba7648f4ae5a [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:23:45.679621 26022 raft_consensus.cc:3060] T cfad575709c943cba40ccab278350770 P 9868dce521294d0b816bba7648f4ae5a [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:23:45.689337 26022 raft_consensus.cc:515] T cfad575709c943cba40ccab278350770 P 9868dce521294d0b816bba7648f4ae5a [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } }
I20260709 13:23:45.692417 25307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfad575709c943cba40ccab278350770" candidate_uuid: "9868dce521294d0b816bba7648f4ae5a" candidate_term: 4 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "233e7d29aac04505bf4e65f0a39a0c4c"
I20260709 13:23:45.693120 26022 leader_election.cc:290] T cfad575709c943cba40ccab278350770 P 9868dce521294d0b816bba7648f4ae5a [CANDIDATE]: Term 4 election: Requested vote from peers 233e7d29aac04505bf4e65f0a39a0c4c (127.14.177.130:42957), 3e07d64466934da4bb3c02262b1c2e38 (127.14.177.131:41539)
I20260709 13:23:45.694993 25382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfad575709c943cba40ccab278350770" candidate_uuid: "9868dce521294d0b816bba7648f4ae5a" candidate_term: 4 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "3e07d64466934da4bb3c02262b1c2e38"
I20260709 13:23:45.695734 25382 raft_consensus.cc:3060] T cfad575709c943cba40ccab278350770 P 3e07d64466934da4bb3c02262b1c2e38 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:23:45.704196 25382 raft_consensus.cc:2468] T cfad575709c943cba40ccab278350770 P 3e07d64466934da4bb3c02262b1c2e38 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9868dce521294d0b816bba7648f4ae5a in term 4.
I20260709 13:23:45.705703 25422 leader_election.cc:304] T cfad575709c943cba40ccab278350770 P 9868dce521294d0b816bba7648f4ae5a [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3e07d64466934da4bb3c02262b1c2e38, 9868dce521294d0b816bba7648f4ae5a; no voters: 233e7d29aac04505bf4e65f0a39a0c4c
I20260709 13:23:45.707002 26022 raft_consensus.cc:2804] T cfad575709c943cba40ccab278350770 P 9868dce521294d0b816bba7648f4ae5a [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:23:45.707674 26022 raft_consensus.cc:697] T cfad575709c943cba40ccab278350770 P 9868dce521294d0b816bba7648f4ae5a [term 4 LEADER]: Becoming Leader. State: Replica: 9868dce521294d0b816bba7648f4ae5a, State: Running, Role: LEADER
I20260709 13:23:45.708644 26022 consensus_queue.cc:237] T cfad575709c943cba40ccab278350770 P 9868dce521294d0b816bba7648f4ae5a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } }
I20260709 13:23:45.735505 26058 raft_consensus.cc:493] T 467a6b42ca6548269c12dddde696f7af P 3e07d64466934da4bb3c02262b1c2e38 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:23:45.736136 26058 raft_consensus.cc:515] T 467a6b42ca6548269c12dddde696f7af P 3e07d64466934da4bb3c02262b1c2e38 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } }
I20260709 13:23:45.739688 25454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "467a6b42ca6548269c12dddde696f7af" candidate_uuid: "3e07d64466934da4bb3c02262b1c2e38" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9868dce521294d0b816bba7648f4ae5a" is_pre_election: true
I20260709 13:23:45.740442 25454 raft_consensus.cc:2468] T 467a6b42ca6548269c12dddde696f7af P 9868dce521294d0b816bba7648f4ae5a [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3e07d64466934da4bb3c02262b1c2e38 in term 2.
I20260709 13:23:45.739933 26058 leader_election.cc:290] T 467a6b42ca6548269c12dddde696f7af P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 233e7d29aac04505bf4e65f0a39a0c4c (127.14.177.130:42957), 9868dce521294d0b816bba7648f4ae5a (127.14.177.132:33683)
I20260709 13:23:45.741328 25307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "467a6b42ca6548269c12dddde696f7af" candidate_uuid: "3e07d64466934da4bb3c02262b1c2e38" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" is_pre_election: true
I20260709 13:23:45.741834 25348 leader_election.cc:304] T 467a6b42ca6548269c12dddde696f7af P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e07d64466934da4bb3c02262b1c2e38, 9868dce521294d0b816bba7648f4ae5a; no voters: 
I20260709 13:23:45.742640 25307 raft_consensus.cc:2468] T 467a6b42ca6548269c12dddde696f7af P 233e7d29aac04505bf4e65f0a39a0c4c [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3e07d64466934da4bb3c02262b1c2e38 in term 2.
I20260709 13:23:45.745397 26058 raft_consensus.cc:2804] T 467a6b42ca6548269c12dddde696f7af P 3e07d64466934da4bb3c02262b1c2e38 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:23:45.746218 26058 raft_consensus.cc:493] T 467a6b42ca6548269c12dddde696f7af P 3e07d64466934da4bb3c02262b1c2e38 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:23:45.746569 26058 raft_consensus.cc:3060] T 467a6b42ca6548269c12dddde696f7af P 3e07d64466934da4bb3c02262b1c2e38 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:23:45.760198 25134 catalog_manager.cc:5697] T cfad575709c943cba40ccab278350770 P 9868dce521294d0b816bba7648f4ae5a reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "9868dce521294d0b816bba7648f4ae5a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } health_report { overall_health: HEALTHY } } }
I20260709 13:23:45.761641 26058 raft_consensus.cc:515] T 467a6b42ca6548269c12dddde696f7af P 3e07d64466934da4bb3c02262b1c2e38 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } }
I20260709 13:23:45.764624 26058 leader_election.cc:290] T 467a6b42ca6548269c12dddde696f7af P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 3 election: Requested vote from peers 233e7d29aac04505bf4e65f0a39a0c4c (127.14.177.130:42957), 9868dce521294d0b816bba7648f4ae5a (127.14.177.132:33683)
I20260709 13:23:45.766566 25307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "467a6b42ca6548269c12dddde696f7af" candidate_uuid: "3e07d64466934da4bb3c02262b1c2e38" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "233e7d29aac04505bf4e65f0a39a0c4c"
I20260709 13:23:45.767259 25307 raft_consensus.cc:3060] T 467a6b42ca6548269c12dddde696f7af P 233e7d29aac04505bf4e65f0a39a0c4c [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:23:45.767513 25454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "467a6b42ca6548269c12dddde696f7af" candidate_uuid: "3e07d64466934da4bb3c02262b1c2e38" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9868dce521294d0b816bba7648f4ae5a"
I20260709 13:23:45.768191 25454 raft_consensus.cc:3060] T 467a6b42ca6548269c12dddde696f7af P 9868dce521294d0b816bba7648f4ae5a [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:23:45.778105 25454 raft_consensus.cc:2468] T 467a6b42ca6548269c12dddde696f7af P 9868dce521294d0b816bba7648f4ae5a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3e07d64466934da4bb3c02262b1c2e38 in term 3.
I20260709 13:23:45.779498 25348 leader_election.cc:304] T 467a6b42ca6548269c12dddde696f7af P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e07d64466934da4bb3c02262b1c2e38, 9868dce521294d0b816bba7648f4ae5a; no voters: 
I20260709 13:23:45.778167 25307 raft_consensus.cc:2468] T 467a6b42ca6548269c12dddde696f7af P 233e7d29aac04505bf4e65f0a39a0c4c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3e07d64466934da4bb3c02262b1c2e38 in term 3.
I20260709 13:23:45.780558 26058 raft_consensus.cc:2804] T 467a6b42ca6548269c12dddde696f7af P 3e07d64466934da4bb3c02262b1c2e38 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:23:45.782213 26058 raft_consensus.cc:697] T 467a6b42ca6548269c12dddde696f7af P 3e07d64466934da4bb3c02262b1c2e38 [term 3 LEADER]: Becoming Leader. State: Replica: 3e07d64466934da4bb3c02262b1c2e38, State: Running, Role: LEADER
I20260709 13:23:45.783143 26058 consensus_queue.cc:237] T 467a6b42ca6548269c12dddde696f7af P 3e07d64466934da4bb3c02262b1c2e38 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } }
I20260709 13:23:45.784439 26041 raft_consensus.cc:493] T c902bbc7ec0848ba82a04870eab1e7f5 P 3e07d64466934da4bb3c02262b1c2e38 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 43999d813e564dc688fad116090c95ca)
I20260709 13:23:45.785032 26041 raft_consensus.cc:515] T c902bbc7ec0848ba82a04870eab1e7f5 P 3e07d64466934da4bb3c02262b1c2e38 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } }
I20260709 13:23:45.787303 26041 leader_election.cc:290] T c902bbc7ec0848ba82a04870eab1e7f5 P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9868dce521294d0b816bba7648f4ae5a (127.14.177.132:33683), 43999d813e564dc688fad116090c95ca (127.14.177.129:38155)
I20260709 13:23:45.789395 25454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c902bbc7ec0848ba82a04870eab1e7f5" candidate_uuid: "3e07d64466934da4bb3c02262b1c2e38" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9868dce521294d0b816bba7648f4ae5a" is_pre_election: true
I20260709 13:23:45.790673 25454 raft_consensus.cc:2393] T c902bbc7ec0848ba82a04870eab1e7f5 P 9868dce521294d0b816bba7648f4ae5a [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3e07d64466934da4bb3c02262b1c2e38 in current term 2: Already voted for candidate 9868dce521294d0b816bba7648f4ae5a in this term.
W20260709 13:23:45.801014 25350 leader_election.cc:336] T c902bbc7ec0848ba82a04870eab1e7f5 P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 43999d813e564dc688fad116090c95ca (127.14.177.129:38155): Network error: Client connection negotiation failed: client connection to 127.14.177.129:38155: connect: Connection refused (error 111)
I20260709 13:23:45.801532 25350 leader_election.cc:304] T c902bbc7ec0848ba82a04870eab1e7f5 P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3e07d64466934da4bb3c02262b1c2e38; no voters: 43999d813e564dc688fad116090c95ca, 9868dce521294d0b816bba7648f4ae5a
I20260709 13:23:45.802309 26046 raft_consensus.cc:3060] T c902bbc7ec0848ba82a04870eab1e7f5 P 3e07d64466934da4bb3c02262b1c2e38 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:23:45.804986 26069 raft_consensus.cc:493] T 4d837697cd0f402cacf9db89a5d92652 P 9868dce521294d0b816bba7648f4ae5a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 43999d813e564dc688fad116090c95ca)
I20260709 13:23:45.805620 26069 raft_consensus.cc:515] T 4d837697cd0f402cacf9db89a5d92652 P 9868dce521294d0b816bba7648f4ae5a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } }
I20260709 13:23:45.807869 26069 leader_election.cc:290] T 4d837697cd0f402cacf9db89a5d92652 P 9868dce521294d0b816bba7648f4ae5a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3e07d64466934da4bb3c02262b1c2e38 (127.14.177.131:41539), 43999d813e564dc688fad116090c95ca (127.14.177.129:38155)
I20260709 13:23:45.808034 26022 raft_consensus.cc:493] T 0e71b6f5c69b4b35a8ac51c22c7d5f0d P 9868dce521294d0b816bba7648f4ae5a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 43999d813e564dc688fad116090c95ca)
I20260709 13:23:45.808698 26022 raft_consensus.cc:515] T 0e71b6f5c69b4b35a8ac51c22c7d5f0d P 9868dce521294d0b816bba7648f4ae5a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } }
I20260709 13:23:45.810482 26046 raft_consensus.cc:2749] T c902bbc7ec0848ba82a04870eab1e7f5 P 3e07d64466934da4bb3c02262b1c2e38 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:23:45.812143 25379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d837697cd0f402cacf9db89a5d92652" candidate_uuid: "9868dce521294d0b816bba7648f4ae5a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3e07d64466934da4bb3c02262b1c2e38" is_pre_election: true
I20260709 13:23:45.814917 25379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e71b6f5c69b4b35a8ac51c22c7d5f0d" candidate_uuid: "9868dce521294d0b816bba7648f4ae5a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3e07d64466934da4bb3c02262b1c2e38" is_pre_election: true
I20260709 13:23:45.811048 26022 leader_election.cc:290] T 0e71b6f5c69b4b35a8ac51c22c7d5f0d P 9868dce521294d0b816bba7648f4ae5a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3e07d64466934da4bb3c02262b1c2e38 (127.14.177.131:41539), 43999d813e564dc688fad116090c95ca (127.14.177.129:38155)
W20260709 13:23:45.817235 25424 leader_election.cc:336] T 4d837697cd0f402cacf9db89a5d92652 P 9868dce521294d0b816bba7648f4ae5a [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 43999d813e564dc688fad116090c95ca (127.14.177.129:38155): Network error: Client connection negotiation failed: client connection to 127.14.177.129:38155: connect: Connection refused (error 111)
I20260709 13:23:45.817943 25424 leader_election.cc:304] T 4d837697cd0f402cacf9db89a5d92652 P 9868dce521294d0b816bba7648f4ae5a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9868dce521294d0b816bba7648f4ae5a; no voters: 3e07d64466934da4bb3c02262b1c2e38, 43999d813e564dc688fad116090c95ca
W20260709 13:23:45.818961 25424 leader_election.cc:336] T 0e71b6f5c69b4b35a8ac51c22c7d5f0d P 9868dce521294d0b816bba7648f4ae5a [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 43999d813e564dc688fad116090c95ca (127.14.177.129:38155): Network error: Client connection negotiation failed: client connection to 127.14.177.129:38155: connect: Connection refused (error 111)
I20260709 13:23:45.821899 25424 leader_election.cc:304] T 0e71b6f5c69b4b35a8ac51c22c7d5f0d P 9868dce521294d0b816bba7648f4ae5a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9868dce521294d0b816bba7648f4ae5a; no voters: 3e07d64466934da4bb3c02262b1c2e38, 43999d813e564dc688fad116090c95ca
I20260709 13:23:45.822662 26069 raft_consensus.cc:2749] T 0e71b6f5c69b4b35a8ac51c22c7d5f0d P 9868dce521294d0b816bba7648f4ae5a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:23:45.819501 26022 raft_consensus.cc:2749] T 4d837697cd0f402cacf9db89a5d92652 P 9868dce521294d0b816bba7648f4ae5a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:23:45.823823 25134 catalog_manager.cc:5697] T 467a6b42ca6548269c12dddde696f7af P 3e07d64466934da4bb3c02262b1c2e38 reported cstate change: term changed from 1 to 3, leader changed from 9868dce521294d0b816bba7648f4ae5a (127.14.177.132) to 3e07d64466934da4bb3c02262b1c2e38 (127.14.177.131). New cstate: current_term: 3 leader_uuid: "3e07d64466934da4bb3c02262b1c2e38" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } health_report { overall_health: UNKNOWN } } }
I20260709 13:23:45.842061 15046 ts_tablet_manager.cc:585] Loaded tablet metadata (45 total tablets, 45 live tablets)
I20260709 13:23:45.842550 15046 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.403s	user 0.012s	sys 0.000s
I20260709 13:23:45.843042 15046 ts_tablet_manager.cc:600] Registering tablets (0/45 complete)
I20260709 13:23:45.850409 26118 tablet_bootstrap.cc:492] T e15820fa8a7d4e1fb6ad23a64ca22f90 P 43999d813e564dc688fad116090c95ca: Bootstrap starting.
I20260709 13:23:45.867658 26029 raft_consensus.cc:493] T cfad575709c943cba40ccab278350770 P 233e7d29aac04505bf4e65f0a39a0c4c [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 9868dce521294d0b816bba7648f4ae5a)
I20260709 13:23:45.868582 26029 raft_consensus.cc:515] T cfad575709c943cba40ccab278350770 P 233e7d29aac04505bf4e65f0a39a0c4c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } }
I20260709 13:23:45.880529 25454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfad575709c943cba40ccab278350770" candidate_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "9868dce521294d0b816bba7648f4ae5a" is_pre_election: true
I20260709 13:23:45.884377 26118 tablet_bootstrap.cc:492] T e15820fa8a7d4e1fb6ad23a64ca22f90 P 43999d813e564dc688fad116090c95ca: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:23:45.885468 26118 tablet_bootstrap.cc:492] T e15820fa8a7d4e1fb6ad23a64ca22f90 P 43999d813e564dc688fad116090c95ca: Bootstrap complete.
I20260709 13:23:45.886171 26118 ts_tablet_manager.cc:1403] T e15820fa8a7d4e1fb6ad23a64ca22f90 P 43999d813e564dc688fad116090c95ca: Time spent bootstrapping tablet: real 0.036s	user 0.024s	sys 0.004s
I20260709 13:23:45.889026 26118 raft_consensus.cc:359] T e15820fa8a7d4e1fb6ad23a64ca22f90 P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } }
I20260709 13:23:45.870872 26029 leader_election.cc:290] T cfad575709c943cba40ccab278350770 P 233e7d29aac04505bf4e65f0a39a0c4c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3e07d64466934da4bb3c02262b1c2e38 (127.14.177.131:41539), 9868dce521294d0b816bba7648f4ae5a (127.14.177.132:33683)
I20260709 13:23:45.889799 26118 raft_consensus.cc:740] T e15820fa8a7d4e1fb6ad23a64ca22f90 P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43999d813e564dc688fad116090c95ca, State: Initialized, Role: FOLLOWER
I20260709 13:23:45.890687 26118 consensus_queue.cc:260] T e15820fa8a7d4e1fb6ad23a64ca22f90 P 43999d813e564dc688fad116090c95ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } }
I20260709 13:23:45.897456 25379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfad575709c943cba40ccab278350770" candidate_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "3e07d64466934da4bb3c02262b1c2e38" is_pre_election: true
I20260709 13:23:45.898850 25379 raft_consensus.cc:2368] T cfad575709c943cba40ccab278350770 P 3e07d64466934da4bb3c02262b1c2e38 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 233e7d29aac04505bf4e65f0a39a0c4c for earlier term 3. Current term is 4.
I20260709 13:23:45.901064 26022 raft_consensus.cc:493] T 1550ed2d2a764c08adffae679599ba41 P 9868dce521294d0b816bba7648f4ae5a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 43999d813e564dc688fad116090c95ca)
I20260709 13:23:45.901948 26069 raft_consensus.cc:493] T 441d780c3945463f92f31170f31e37a8 P 9868dce521294d0b816bba7648f4ae5a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 43999d813e564dc688fad116090c95ca)
I20260709 13:23:45.902578 26069 raft_consensus.cc:515] T 441d780c3945463f92f31170f31e37a8 P 9868dce521294d0b816bba7648f4ae5a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } }
I20260709 13:23:45.908600 25307 raft_consensus.cc:1275] T ad32097cd4dc48b9a69595f1651abd07 P 233e7d29aac04505bf4e65f0a39a0c4c [term 3 FOLLOWER]: Refusing update from remote peer 3e07d64466934da4bb3c02262b1c2e38: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:23:45.909678 26069 leader_election.cc:290] T 441d780c3945463f92f31170f31e37a8 P 9868dce521294d0b816bba7648f4ae5a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 43999d813e564dc688fad116090c95ca (127.14.177.129:38155), 233e7d29aac04505bf4e65f0a39a0c4c (127.14.177.130:42957)
I20260709 13:23:45.912734 26058 consensus_queue.cc:1048] T ad32097cd4dc48b9a69595f1651abd07 P 3e07d64466934da4bb3c02262b1c2e38 [LEADER]: Connected to new peer: Peer: permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:23:45.915614 25307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "441d780c3945463f92f31170f31e37a8" candidate_uuid: "9868dce521294d0b816bba7648f4ae5a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" is_pre_election: true
I20260709 13:23:45.917748 26083 raft_consensus.cc:493] T ce9c60e3922c4219b6e4453fe0259b6a P 9868dce521294d0b816bba7648f4ae5a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 43999d813e564dc688fad116090c95ca)
I20260709 13:23:45.918387 26083 raft_consensus.cc:515] T ce9c60e3922c4219b6e4453fe0259b6a P 9868dce521294d0b816bba7648f4ae5a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } }
I20260709 13:23:45.920562 26083 leader_election.cc:290] T ce9c60e3922c4219b6e4453fe0259b6a P 9868dce521294d0b816bba7648f4ae5a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 43999d813e564dc688fad116090c95ca (127.14.177.129:38155), 233e7d29aac04505bf4e65f0a39a0c4c (127.14.177.130:42957)
W20260709 13:23:45.934083 25424 leader_election.cc:336] T 441d780c3945463f92f31170f31e37a8 P 9868dce521294d0b816bba7648f4ae5a [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 43999d813e564dc688fad116090c95ca (127.14.177.129:38155): Network error: Client connection negotiation failed: client connection to 127.14.177.129:38155: connect: Connection refused (error 111)
I20260709 13:23:45.935410 25424 leader_election.cc:304] T 441d780c3945463f92f31170f31e37a8 P 9868dce521294d0b816bba7648f4ae5a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9868dce521294d0b816bba7648f4ae5a; no voters: 233e7d29aac04505bf4e65f0a39a0c4c, 43999d813e564dc688fad116090c95ca
I20260709 13:23:45.937261 25310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce9c60e3922c4219b6e4453fe0259b6a" candidate_uuid: "9868dce521294d0b816bba7648f4ae5a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" is_pre_election: true
I20260709 13:23:45.939770 25310 raft_consensus.cc:2393] T ce9c60e3922c4219b6e4453fe0259b6a P 233e7d29aac04505bf4e65f0a39a0c4c [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9868dce521294d0b816bba7648f4ae5a in current term 2: Already voted for candidate 233e7d29aac04505bf4e65f0a39a0c4c in this term.
I20260709 13:23:45.940743 26069 raft_consensus.cc:2749] T 441d780c3945463f92f31170f31e37a8 P 9868dce521294d0b816bba7648f4ae5a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:23:45.942608 26022 raft_consensus.cc:515] T 1550ed2d2a764c08adffae679599ba41 P 9868dce521294d0b816bba7648f4ae5a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } }
I20260709 13:23:45.948235 26022 leader_election.cc:290] T 1550ed2d2a764c08adffae679599ba41 P 9868dce521294d0b816bba7648f4ae5a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3e07d64466934da4bb3c02262b1c2e38 (127.14.177.131:41539), 43999d813e564dc688fad116090c95ca (127.14.177.129:38155)
W20260709 13:23:45.947480 25424 leader_election.cc:336] T ce9c60e3922c4219b6e4453fe0259b6a P 9868dce521294d0b816bba7648f4ae5a [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 43999d813e564dc688fad116090c95ca (127.14.177.129:38155): Network error: Client connection negotiation failed: client connection to 127.14.177.129:38155: connect: Connection refused (error 111)
I20260709 13:23:45.949337 25424 leader_election.cc:304] T ce9c60e3922c4219b6e4453fe0259b6a P 9868dce521294d0b816bba7648f4ae5a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9868dce521294d0b816bba7648f4ae5a; no voters: 233e7d29aac04505bf4e65f0a39a0c4c, 43999d813e564dc688fad116090c95ca
I20260709 13:23:45.951355 26022 raft_consensus.cc:3060] T ce9c60e3922c4219b6e4453fe0259b6a P 9868dce521294d0b816bba7648f4ae5a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:23:45.951797 25274 leader_election.cc:400] T cfad575709c943cba40ccab278350770 P 233e7d29aac04505bf4e65f0a39a0c4c [CANDIDATE]: Term 3 pre-election: Vote denied by peer 3e07d64466934da4bb3c02262b1c2e38 (127.14.177.131:41539) with higher term. Message: Invalid argument: T cfad575709c943cba40ccab278350770 P 3e07d64466934da4bb3c02262b1c2e38 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 233e7d29aac04505bf4e65f0a39a0c4c for earlier term 3. Current term is 4.
I20260709 13:23:46.038636 25274 leader_election.cc:403] T cfad575709c943cba40ccab278350770 P 233e7d29aac04505bf4e65f0a39a0c4c [CANDIDATE]: Term 3 pre-election: Cancelling election due to peer responding with higher term
W20260709 13:23:46.039681 25274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41539, user_credentials={real_user=slave}} blocked reactor thread for 88143.4us
I20260709 13:23:45.950750 26029 raft_consensus.cc:493] T 3a3317f4c3514df1b2bb8e44a86be92f P 233e7d29aac04505bf4e65f0a39a0c4c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 43999d813e564dc688fad116090c95ca)
I20260709 13:23:46.041487 26029 raft_consensus.cc:515] T 3a3317f4c3514df1b2bb8e44a86be92f P 233e7d29aac04505bf4e65f0a39a0c4c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } }
I20260709 13:23:46.044303 26029 leader_election.cc:290] T 3a3317f4c3514df1b2bb8e44a86be92f P 233e7d29aac04505bf4e65f0a39a0c4c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9868dce521294d0b816bba7648f4ae5a (127.14.177.132:33683), 43999d813e564dc688fad116090c95ca (127.14.177.129:38155)
I20260709 13:23:45.998260 26069 raft_consensus.cc:493] T 579da2d6aabb4ae9a2b1802e1358068b P 9868dce521294d0b816bba7648f4ae5a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 43999d813e564dc688fad116090c95ca)
I20260709 13:23:46.049162 26069 raft_consensus.cc:515] T 579da2d6aabb4ae9a2b1802e1358068b P 9868dce521294d0b816bba7648f4ae5a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } }
I20260709 13:23:45.999500 26086 raft_consensus.cc:493] T b9d550d881b04735bbf89751952c09ec P 9868dce521294d0b816bba7648f4ae5a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 43999d813e564dc688fad116090c95ca)
I20260709 13:23:46.051508 26086 raft_consensus.cc:515] T b9d550d881b04735bbf89751952c09ec P 9868dce521294d0b816bba7648f4ae5a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } }
I20260709 13:23:46.054524 26086 leader_election.cc:290] T b9d550d881b04735bbf89751952c09ec P 9868dce521294d0b816bba7648f4ae5a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3e07d64466934da4bb3c02262b1c2e38 (127.14.177.131:41539), 43999d813e564dc688fad116090c95ca (127.14.177.129:38155)
I20260709 13:23:46.056443 26069 leader_election.cc:290] T 579da2d6aabb4ae9a2b1802e1358068b P 9868dce521294d0b816bba7648f4ae5a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 43999d813e564dc688fad116090c95ca (127.14.177.129:38155), 233e7d29aac04505bf4e65f0a39a0c4c (127.14.177.130:42957)
I20260709 13:23:45.984339 26083 raft_consensus.cc:493] T 821a5866429f4173820fb64336abee02 P 9868dce521294d0b816bba7648f4ae5a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 43999d813e564dc688fad116090c95ca)
I20260709 13:23:46.031301 25382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1550ed2d2a764c08adffae679599ba41" candidate_uuid: "9868dce521294d0b816bba7648f4ae5a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3e07d64466934da4bb3c02262b1c2e38" is_pre_election: true
I20260709 13:23:46.062273 25382 raft_consensus.cc:2468] T 1550ed2d2a764c08adffae679599ba41 P 3e07d64466934da4bb3c02262b1c2e38 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9868dce521294d0b816bba7648f4ae5a in term 1.
I20260709 13:23:46.044646 26099 raft_consensus.cc:3060] T cfad575709c943cba40ccab278350770 P 233e7d29aac04505bf4e65f0a39a0c4c [term 2 FOLLOWER]: Advancing to term 4
I20260709 13:23:46.067543 26083 raft_consensus.cc:515] T 821a5866429f4173820fb64336abee02 P 9868dce521294d0b816bba7648f4ae5a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } }
I20260709 13:23:46.074537 26083 leader_election.cc:290] T 821a5866429f4173820fb64336abee02 P 9868dce521294d0b816bba7648f4ae5a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3e07d64466934da4bb3c02262b1c2e38 (127.14.177.131:41539), 43999d813e564dc688fad116090c95ca (127.14.177.129:38155)
I20260709 13:23:46.076479 26069 raft_consensus.cc:493] T f1ac2f4d2fe2424bb470df42199219e2 P 9868dce521294d0b816bba7648f4ae5a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 43999d813e564dc688fad116090c95ca)
I20260709 13:23:46.085562 26069 raft_consensus.cc:515] T f1ac2f4d2fe2424bb470df42199219e2 P 9868dce521294d0b816bba7648f4ae5a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } }
I20260709 13:23:46.088631 26069 leader_election.cc:290] T f1ac2f4d2fe2424bb470df42199219e2 P 9868dce521294d0b816bba7648f4ae5a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3e07d64466934da4bb3c02262b1c2e38 (127.14.177.131:41539), 43999d813e564dc688fad116090c95ca (127.14.177.129:38155)
I20260709 13:23:46.092149 26058 raft_consensus.cc:493] T d0a0e4d32ce943fbb773bb840863647b P 3e07d64466934da4bb3c02262b1c2e38 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 43999d813e564dc688fad116090c95ca)
I20260709 13:23:46.096691 25382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9d550d881b04735bbf89751952c09ec" candidate_uuid: "9868dce521294d0b816bba7648f4ae5a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3e07d64466934da4bb3c02262b1c2e38" is_pre_election: true
I20260709 13:23:46.096815 25384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "821a5866429f4173820fb64336abee02" candidate_uuid: "9868dce521294d0b816bba7648f4ae5a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3e07d64466934da4bb3c02262b1c2e38" is_pre_election: true
W20260709 13:23:46.097393 25424 leader_election.cc:336] T 1550ed2d2a764c08adffae679599ba41 P 9868dce521294d0b816bba7648f4ae5a [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 43999d813e564dc688fad116090c95ca (127.14.177.129:38155): Network error: Client connection negotiation failed: client connection to 127.14.177.129:38155: connect: Connection refused (error 111)
I20260709 13:23:46.117697 26058 raft_consensus.cc:515] T d0a0e4d32ce943fbb773bb840863647b P 3e07d64466934da4bb3c02262b1c2e38 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } }
W20260709 13:23:46.121374 25424 leader_election.cc:336] T b9d550d881b04735bbf89751952c09ec P 9868dce521294d0b816bba7648f4ae5a [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 43999d813e564dc688fad116090c95ca (127.14.177.129:38155): Network error: Client connection negotiation failed: client connection to 127.14.177.129:38155: connect: Connection refused (error 111)
W20260709 13:23:46.122099 25424 leader_election.cc:336] T 579da2d6aabb4ae9a2b1802e1358068b P 9868dce521294d0b816bba7648f4ae5a [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 43999d813e564dc688fad116090c95ca (127.14.177.129:38155): Network error: Client connection negotiation failed: client connection to 127.14.177.129:38155: connect: Connection refused (error 111)
I20260709 13:23:46.021282 26046 raft_consensus.cc:493] T 4d837697cd0f402cacf9db89a5d92652 P 3e07d64466934da4bb3c02262b1c2e38 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 43999d813e564dc688fad116090c95ca)
I20260709 13:23:46.123494 26046 raft_consensus.cc:515] T 4d837697cd0f402cacf9db89a5d92652 P 3e07d64466934da4bb3c02262b1c2e38 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } }
I20260709 13:23:46.111536 26029 raft_consensus.cc:493] T b7ff8dde9f5d4c49b9203d902f6b94c4 P 233e7d29aac04505bf4e65f0a39a0c4c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 43999d813e564dc688fad116090c95ca)
I20260709 13:23:46.116840 26118 ts_tablet_manager.cc:1434] T e15820fa8a7d4e1fb6ad23a64ca22f90 P 43999d813e564dc688fad116090c95ca: Time spent starting tablet: real 0.230s	user 0.076s	sys 0.151s
I20260709 13:23:46.118009 25382 raft_consensus.cc:2468] T b9d550d881b04735bbf89751952c09ec P 3e07d64466934da4bb3c02262b1c2e38 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9868dce521294d0b816bba7648f4ae5a in term 1.
I20260709 13:23:46.125581 26029 raft_consensus.cc:515] T b7ff8dde9f5d4c49b9203d902f6b94c4 P 233e7d29aac04505bf4e65f0a39a0c4c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } }
I20260709 13:23:46.126248 26118 tablet_bootstrap.cc:492] T 441d780c3945463f92f31170f31e37a8 P 43999d813e564dc688fad116090c95ca: Bootstrap starting.
I20260709 13:23:46.127882 26029 leader_election.cc:290] T b7ff8dde9f5d4c49b9203d902f6b94c4 P 233e7d29aac04505bf4e65f0a39a0c4c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 43999d813e564dc688fad116090c95ca (127.14.177.129:38155), 3e07d64466934da4bb3c02262b1c2e38 (127.14.177.131:41539)
I20260709 13:23:46.131669 26058 leader_election.cc:290] T d0a0e4d32ce943fbb773bb840863647b P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9868dce521294d0b816bba7648f4ae5a (127.14.177.132:33683), 43999d813e564dc688fad116090c95ca (127.14.177.129:38155)
I20260709 13:23:46.132035 26046 leader_election.cc:290] T 4d837697cd0f402cacf9db89a5d92652 P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9868dce521294d0b816bba7648f4ae5a (127.14.177.132:33683), 43999d813e564dc688fad116090c95ca (127.14.177.129:38155)
I20260709 13:23:46.100309 26135 raft_consensus.cc:493] T fa0da2c845dc4059968332f4db2e0aa6 P 9868dce521294d0b816bba7648f4ae5a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 43999d813e564dc688fad116090c95ca)
I20260709 13:23:46.136596 26099 raft_consensus.cc:2749] T cfad575709c943cba40ccab278350770 P 233e7d29aac04505bf4e65f0a39a0c4c [term 4 FOLLOWER]: Leader pre-election lost for term 3. Reason: Vote denied by peer 3e07d64466934da4bb3c02262b1c2e38 (127.14.177.131:41539) with higher term. Message: Invalid argument: T cfad575709c943cba40ccab278350770 P 3e07d64466934da4bb3c02262b1c2e38 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 233e7d29aac04505bf4e65f0a39a0c4c for earlier term 3. Current term is 4.
I20260709 13:23:46.136878 25422 leader_election.cc:304] T 1550ed2d2a764c08adffae679599ba41 P 9868dce521294d0b816bba7648f4ae5a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3e07d64466934da4bb3c02262b1c2e38, 9868dce521294d0b816bba7648f4ae5a; no voters: 43999d813e564dc688fad116090c95ca
I20260709 13:23:46.133729 26135 raft_consensus.cc:515] T fa0da2c845dc4059968332f4db2e0aa6 P 9868dce521294d0b816bba7648f4ae5a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } }
I20260709 13:23:46.140743 26135 leader_election.cc:290] T fa0da2c845dc4059968332f4db2e0aa6 P 9868dce521294d0b816bba7648f4ae5a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 43999d813e564dc688fad116090c95ca (127.14.177.129:38155), 233e7d29aac04505bf4e65f0a39a0c4c (127.14.177.130:42957)
W20260709 13:23:46.088385 25276 leader_election.cc:336] T 3a3317f4c3514df1b2bb8e44a86be92f P 233e7d29aac04505bf4e65f0a39a0c4c [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 43999d813e564dc688fad116090c95ca (127.14.177.129:38155): Network error: Client connection negotiation failed: client connection to 127.14.177.129:38155: connect: Connection refused (error 111)
W20260709 13:23:46.141875 25276 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:38155, user_credentials={real_user=slave}} blocked reactor thread for 53531.4us
I20260709 13:23:46.145548 26046 raft_consensus.cc:493] T 0e71b6f5c69b4b35a8ac51c22c7d5f0d P 3e07d64466934da4bb3c02262b1c2e38 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 43999d813e564dc688fad116090c95ca)
I20260709 13:23:46.145815 26099 raft_consensus.cc:493] T 579da2d6aabb4ae9a2b1802e1358068b P 233e7d29aac04505bf4e65f0a39a0c4c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 43999d813e564dc688fad116090c95ca)
I20260709 13:23:46.146176 26046 raft_consensus.cc:515] T 0e71b6f5c69b4b35a8ac51c22c7d5f0d P 3e07d64466934da4bb3c02262b1c2e38 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } }
I20260709 13:23:46.146497 26099 raft_consensus.cc:515] T 579da2d6aabb4ae9a2b1802e1358068b P 233e7d29aac04505bf4e65f0a39a0c4c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } }
I20260709 13:23:46.148370 26046 leader_election.cc:290] T 0e71b6f5c69b4b35a8ac51c22c7d5f0d P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9868dce521294d0b816bba7648f4ae5a (127.14.177.132:33683), 43999d813e564dc688fad116090c95ca (127.14.177.129:38155)
I20260709 13:23:46.148896 26099 leader_election.cc:290] T 579da2d6aabb4ae9a2b1802e1358068b P 233e7d29aac04505bf4e65f0a39a0c4c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9868dce521294d0b816bba7648f4ae5a (127.14.177.132:33683), 43999d813e564dc688fad116090c95ca (127.14.177.129:38155)
I20260709 13:23:46.152967 26069 raft_consensus.cc:2804] T 1550ed2d2a764c08adffae679599ba41 P 9868dce521294d0b816bba7648f4ae5a [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:23:46.153461 26069 raft_consensus.cc:493] T 1550ed2d2a764c08adffae679599ba41 P 9868dce521294d0b816bba7648f4ae5a [term 1 FOLLOWER]: Starting leader election (detected failure of leader 43999d813e564dc688fad116090c95ca)
I20260709 13:23:46.153880 26069 raft_consensus.cc:3060] T 1550ed2d2a764c08adffae679599ba41 P 9868dce521294d0b816bba7648f4ae5a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:23:46.155402 25454 raft_consensus.cc:1275] T ad32097cd4dc48b9a69595f1651abd07 P 9868dce521294d0b816bba7648f4ae5a [term 3 FOLLOWER]: Refusing update from remote peer 3e07d64466934da4bb3c02262b1c2e38: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
W20260709 13:23:46.156176 25424 leader_election.cc:336] T 821a5866429f4173820fb64336abee02 P 9868dce521294d0b816bba7648f4ae5a [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 43999d813e564dc688fad116090c95ca (127.14.177.129:38155): Network error: Client connection negotiation failed: client connection to 127.14.177.129:38155: connect: Connection refused (error 111)
W20260709 13:23:46.157483 25424 leader_election.cc:336] T f1ac2f4d2fe2424bb470df42199219e2 P 9868dce521294d0b816bba7648f4ae5a [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 43999d813e564dc688fad116090c95ca (127.14.177.129:38155): Network error: Client connection negotiation failed: client connection to 127.14.177.129:38155: connect: Connection refused (error 111)
W20260709 13:23:46.158836 25424 consensus_peers.cc:597] T 9e5d70664720424d951f269d9d7f4346 P 9868dce521294d0b816bba7648f4ae5a -> Peer 43999d813e564dc688fad116090c95ca (127.14.177.129:38155): Couldn't send request to peer 43999d813e564dc688fad116090c95ca. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:38155: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:23:46.160192 25422 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41539, user_credentials={real_user=slave}} blocked reactor thread for 56501.4us
I20260709 13:23:46.160985 25454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0a0e4d32ce943fbb773bb840863647b" candidate_uuid: "3e07d64466934da4bb3c02262b1c2e38" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9868dce521294d0b816bba7648f4ae5a" is_pre_election: true
W20260709 13:23:46.162886 25424 leader_election.cc:336] T fa0da2c845dc4059968332f4db2e0aa6 P 9868dce521294d0b816bba7648f4ae5a [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 43999d813e564dc688fad116090c95ca (127.14.177.129:38155): Network error: Client connection negotiation failed: client connection to 127.14.177.129:38155: connect: Connection refused (error 111)
I20260709 13:23:46.163754 25310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "579da2d6aabb4ae9a2b1802e1358068b" candidate_uuid: "9868dce521294d0b816bba7648f4ae5a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" is_pre_election: true
I20260709 13:23:46.164500 25310 raft_consensus.cc:2468] T 579da2d6aabb4ae9a2b1802e1358068b P 233e7d29aac04505bf4e65f0a39a0c4c [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9868dce521294d0b816bba7648f4ae5a in term 1.
I20260709 13:23:46.166036 25454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d837697cd0f402cacf9db89a5d92652" candidate_uuid: "3e07d64466934da4bb3c02262b1c2e38" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9868dce521294d0b816bba7648f4ae5a" is_pre_election: true
I20260709 13:23:46.166967 25454 raft_consensus.cc:2468] T 4d837697cd0f402cacf9db89a5d92652 P 9868dce521294d0b816bba7648f4ae5a [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3e07d64466934da4bb3c02262b1c2e38 in term 1.
I20260709 13:23:46.172344 26046 consensus_queue.cc:1048] T ad32097cd4dc48b9a69595f1651abd07 P 3e07d64466934da4bb3c02262b1c2e38 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:23:46.173523 25348 leader_election.cc:304] T 4d837697cd0f402cacf9db89a5d92652 P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e07d64466934da4bb3c02262b1c2e38, 9868dce521294d0b816bba7648f4ae5a; no voters: 
I20260709 13:23:46.204545 26144 raft_consensus.cc:2804] T 4d837697cd0f402cacf9db89a5d92652 P 3e07d64466934da4bb3c02262b1c2e38 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:23:46.173702 26133 raft_consensus.cc:493] T b7ff8dde9f5d4c49b9203d902f6b94c4 P 3e07d64466934da4bb3c02262b1c2e38 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 43999d813e564dc688fad116090c95ca)
I20260709 13:23:46.174993 25380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1ac2f4d2fe2424bb470df42199219e2" candidate_uuid: "9868dce521294d0b816bba7648f4ae5a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3e07d64466934da4bb3c02262b1c2e38" is_pre_election: true
I20260709 13:23:46.175510 26131 raft_consensus.cc:493] T 72b56266e8a54776b9113c7a786e1952 P 3e07d64466934da4bb3c02262b1c2e38 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:23:46.205410 26131 raft_consensus.cc:515] T 72b56266e8a54776b9113c7a786e1952 P 3e07d64466934da4bb3c02262b1c2e38 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } }
I20260709 13:23:46.205621 26133 raft_consensus.cc:515] T b7ff8dde9f5d4c49b9203d902f6b94c4 P 3e07d64466934da4bb3c02262b1c2e38 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } }
I20260709 13:23:46.205848 26069 raft_consensus.cc:515] T 1550ed2d2a764c08adffae679599ba41 P 9868dce521294d0b816bba7648f4ae5a [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } }
I20260709 13:23:46.208113 26131 leader_election.cc:290] T 72b56266e8a54776b9113c7a786e1952 P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 43999d813e564dc688fad116090c95ca (127.14.177.129:38155), 233e7d29aac04505bf4e65f0a39a0c4c (127.14.177.130:42957)
I20260709 13:23:46.208240 26069 leader_election.cc:290] T 1550ed2d2a764c08adffae679599ba41 P 9868dce521294d0b816bba7648f4ae5a [CANDIDATE]: Term 2 election: Requested vote from peers 3e07d64466934da4bb3c02262b1c2e38 (127.14.177.131:41539), 43999d813e564dc688fad116090c95ca (127.14.177.129:38155)
I20260709 13:23:46.175822 26022 raft_consensus.cc:2749] T ce9c60e3922c4219b6e4453fe0259b6a P 9868dce521294d0b816bba7648f4ae5a [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:23:46.176328 26083 raft_consensus.cc:493] T a5df750b84e0487fa530161079deb6c2 P 9868dce521294d0b816bba7648f4ae5a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 43999d813e564dc688fad116090c95ca)
I20260709 13:23:46.209087 26083 raft_consensus.cc:515] T a5df750b84e0487fa530161079deb6c2 P 9868dce521294d0b816bba7648f4ae5a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } }
I20260709 13:23:46.177099 26046 raft_consensus.cc:493] T 13bb3a23315a41c7a40e10b954b15c8e P 3e07d64466934da4bb3c02262b1c2e38 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 43999d813e564dc688fad116090c95ca)
I20260709 13:23:46.178380 26058 raft_consensus.cc:493] T 5c7e7acbb09f4d49828fa652ca6abaac P 3e07d64466934da4bb3c02262b1c2e38 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 43999d813e564dc688fad116090c95ca)
I20260709 13:23:46.178822 26086 raft_consensus.cc:493] T 3a3317f4c3514df1b2bb8e44a86be92f P 9868dce521294d0b816bba7648f4ae5a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 43999d813e564dc688fad116090c95ca)
I20260709 13:23:46.179369 26138 raft_consensus.cc:493] T d0a0e4d32ce943fbb773bb840863647b P 9868dce521294d0b816bba7648f4ae5a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 43999d813e564dc688fad116090c95ca)
I20260709 13:23:46.179987 26134 raft_consensus.cc:493] T f1ac2f4d2fe2424bb470df42199219e2 P 3e07d64466934da4bb3c02262b1c2e38 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 43999d813e564dc688fad116090c95ca)
I20260709 13:23:46.185606 25450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a3317f4c3514df1b2bb8e44a86be92f" candidate_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9868dce521294d0b816bba7648f4ae5a" is_pre_election: true
I20260709 13:23:46.185738 25454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e71b6f5c69b4b35a8ac51c22c7d5f0d" candidate_uuid: "3e07d64466934da4bb3c02262b1c2e38" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9868dce521294d0b816bba7648f4ae5a" is_pre_election: true
I20260709 13:23:46.190377 25309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa0da2c845dc4059968332f4db2e0aa6" candidate_uuid: "9868dce521294d0b816bba7648f4ae5a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" is_pre_election: true
I20260709 13:23:46.205004 26144 raft_consensus.cc:493] T 4d837697cd0f402cacf9db89a5d92652 P 3e07d64466934da4bb3c02262b1c2e38 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 43999d813e564dc688fad116090c95ca)
I20260709 13:23:46.209483 26141 raft_consensus.cc:493] T 441d780c3945463f92f31170f31e37a8 P 233e7d29aac04505bf4e65f0a39a0c4c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 43999d813e564dc688fad116090c95ca)
W20260709 13:23:46.210364 25276 leader_election.cc:336] T b7ff8dde9f5d4c49b9203d902f6b94c4 P 233e7d29aac04505bf4e65f0a39a0c4c [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 43999d813e564dc688fad116090c95ca (127.14.177.129:38155): Network error: Client connection negotiation failed: client connection to 127.14.177.129:38155: connect: Connection refused (error 111)
I20260709 13:23:46.214846 26141 raft_consensus.cc:515] T 441d780c3945463f92f31170f31e37a8 P 233e7d29aac04505bf4e65f0a39a0c4c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } }
I20260709 13:23:46.210501 26137 raft_consensus.cc:493] T fa0da2c845dc4059968332f4db2e0aa6 P 233e7d29aac04505bf4e65f0a39a0c4c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 43999d813e564dc688fad116090c95ca)
I20260709 13:23:46.215953 26137 raft_consensus.cc:515] T fa0da2c845dc4059968332f4db2e0aa6 P 233e7d29aac04505bf4e65f0a39a0c4c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } }
I20260709 13:23:46.217023 25309 raft_consensus.cc:2468] T fa0da2c845dc4059968332f4db2e0aa6 P 233e7d29aac04505bf4e65f0a39a0c4c [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9868dce521294d0b816bba7648f4ae5a in term 1.
I20260709 13:23:46.217343 26141 leader_election.cc:290] T 441d780c3945463f92f31170f31e37a8 P 233e7d29aac04505bf4e65f0a39a0c4c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9868dce521294d0b816bba7648f4ae5a (127.14.177.132:33683), 43999d813e564dc688fad116090c95ca (127.14.177.129:38155)
I20260709 13:23:46.192639 26140 raft_consensus.cc:493] T b155b866f10547fd8239579a81ecf5f9 P 233e7d29aac04505bf4e65f0a39a0c4c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 43999d813e564dc688fad116090c95ca)
I20260709 13:23:46.192670 25457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "579da2d6aabb4ae9a2b1802e1358068b" candidate_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9868dce521294d0b816bba7648f4ae5a" is_pre_election: true
W20260709 13:23:46.200744 25424 consensus_peers.cc:597] T 0c4f6b7401db47d59a4f34ed0e8b5350 P 9868dce521294d0b816bba7648f4ae5a -> Peer 43999d813e564dc688fad116090c95ca (127.14.177.129:38155): Couldn't send request to peer 43999d813e564dc688fad116090c95ca. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:38155: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:23:46.195616 26135 raft_consensus.cc:493] T 8ef061028f38450e83324ac78cdaa14c P 9868dce521294d0b816bba7648f4ae5a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 43999d813e564dc688fad116090c95ca)
I20260709 13:23:46.219014 25457 raft_consensus.cc:2468] T 579da2d6aabb4ae9a2b1802e1358068b P 9868dce521294d0b816bba7648f4ae5a [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 233e7d29aac04505bf4e65f0a39a0c4c in term 1.
I20260709 13:23:46.219110 26135 raft_consensus.cc:515] T 8ef061028f38450e83324ac78cdaa14c P 9868dce521294d0b816bba7648f4ae5a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } }
I20260709 13:23:46.219704 25308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72b56266e8a54776b9113c7a786e1952" candidate_uuid: "3e07d64466934da4bb3c02262b1c2e38" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" is_pre_election: true
I20260709 13:23:46.202848 25425 leader_election.cc:304] T 579da2d6aabb4ae9a2b1802e1358068b P 9868dce521294d0b816bba7648f4ae5a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 233e7d29aac04505bf4e65f0a39a0c4c, 9868dce521294d0b816bba7648f4ae5a; no voters: 43999d813e564dc688fad116090c95ca
I20260709 13:23:46.220116 26140 raft_consensus.cc:515] T b155b866f10547fd8239579a81ecf5f9 P 233e7d29aac04505bf4e65f0a39a0c4c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } }
I20260709 13:23:46.211410 26046 raft_consensus.cc:515] T 13bb3a23315a41c7a40e10b954b15c8e P 3e07d64466934da4bb3c02262b1c2e38 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } }
I20260709 13:23:46.211457 26058 raft_consensus.cc:515] T 5c7e7acbb09f4d49828fa652ca6abaac P 3e07d64466934da4bb3c02262b1c2e38 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } }
I20260709 13:23:46.221275 26135 leader_election.cc:290] T 8ef061028f38450e83324ac78cdaa14c P 9868dce521294d0b816bba7648f4ae5a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 43999d813e564dc688fad116090c95ca (127.14.177.129:38155), 233e7d29aac04505bf4e65f0a39a0c4c (127.14.177.130:42957)
I20260709 13:23:46.222568 26140 leader_election.cc:290] T b155b866f10547fd8239579a81ecf5f9 P 233e7d29aac04505bf4e65f0a39a0c4c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 43999d813e564dc688fad116090c95ca (127.14.177.129:38155), 3e07d64466934da4bb3c02262b1c2e38 (127.14.177.131:41539)
I20260709 13:23:46.222703 26046 leader_election.cc:290] T 13bb3a23315a41c7a40e10b954b15c8e P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 43999d813e564dc688fad116090c95ca (127.14.177.129:38155), 233e7d29aac04505bf4e65f0a39a0c4c (127.14.177.130:42957)
I20260709 13:23:46.222998 26058 leader_election.cc:290] T 5c7e7acbb09f4d49828fa652ca6abaac P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9868dce521294d0b816bba7648f4ae5a (127.14.177.132:33683), 43999d813e564dc688fad116090c95ca (127.14.177.129:38155)
I20260709 13:23:46.211139 26083 leader_election.cc:290] T a5df750b84e0487fa530161079deb6c2 P 9868dce521294d0b816bba7648f4ae5a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 43999d813e564dc688fad116090c95ca (127.14.177.129:38155), 233e7d29aac04505bf4e65f0a39a0c4c (127.14.177.130:42957)
I20260709 13:23:46.224143 26131 raft_consensus.cc:493] T 434a54fcee7d4aeeab42f739855c0e8a P 3e07d64466934da4bb3c02262b1c2e38 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 43999d813e564dc688fad116090c95ca)
I20260709 13:23:46.219995 25310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7ff8dde9f5d4c49b9203d902f6b94c4" candidate_uuid: "3e07d64466934da4bb3c02262b1c2e38" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" is_pre_election: true
I20260709 13:23:46.211669 26086 raft_consensus.cc:515] T 3a3317f4c3514df1b2bb8e44a86be92f P 9868dce521294d0b816bba7648f4ae5a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } }
I20260709 13:23:46.225333 25310 raft_consensus.cc:2468] T b7ff8dde9f5d4c49b9203d902f6b94c4 P 233e7d29aac04505bf4e65f0a39a0c4c [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3e07d64466934da4bb3c02262b1c2e38 in term 1.
I20260709 13:23:46.226423 25422 leader_election.cc:304] T 821a5866429f4173820fb64336abee02 P 9868dce521294d0b816bba7648f4ae5a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9868dce521294d0b816bba7648f4ae5a; no voters: 3e07d64466934da4bb3c02262b1c2e38, 43999d813e564dc688fad116090c95ca
I20260709 13:23:46.220429 25308 raft_consensus.cc:2468] T 72b56266e8a54776b9113c7a786e1952 P 233e7d29aac04505bf4e65f0a39a0c4c [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3e07d64466934da4bb3c02262b1c2e38 in term 1.
I20260709 13:23:46.224685 26131 raft_consensus.cc:515] T 434a54fcee7d4aeeab42f739855c0e8a P 3e07d64466934da4bb3c02262b1c2e38 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } }
I20260709 13:23:46.218245 26137 leader_election.cc:290] T fa0da2c845dc4059968332f4db2e0aa6 P 233e7d29aac04505bf4e65f0a39a0c4c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9868dce521294d0b816bba7648f4ae5a (127.14.177.132:33683), 43999d813e564dc688fad116090c95ca (127.14.177.129:38155)
I20260709 13:23:46.212111 26134 raft_consensus.cc:515] T f1ac2f4d2fe2424bb470df42199219e2 P 3e07d64466934da4bb3c02262b1c2e38 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } }
I20260709 13:23:46.211899 26138 raft_consensus.cc:515] T d0a0e4d32ce943fbb773bb840863647b P 9868dce521294d0b816bba7648f4ae5a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } }
I20260709 13:23:46.212350 25381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7ff8dde9f5d4c49b9203d902f6b94c4" candidate_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3e07d64466934da4bb3c02262b1c2e38" is_pre_election: true
I20260709 13:23:46.236826 25381 raft_consensus.cc:2468] T b7ff8dde9f5d4c49b9203d902f6b94c4 P 3e07d64466934da4bb3c02262b1c2e38 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 233e7d29aac04505bf4e65f0a39a0c4c in term 1.
I20260709 13:23:46.212734 25454 raft_consensus.cc:2468] T 0e71b6f5c69b4b35a8ac51c22c7d5f0d P 9868dce521294d0b816bba7648f4ae5a [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3e07d64466934da4bb3c02262b1c2e38 in term 1.
I20260709 13:23:46.213070 26133 leader_election.cc:290] T b7ff8dde9f5d4c49b9203d902f6b94c4 P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 43999d813e564dc688fad116090c95ca (127.14.177.129:38155), 233e7d29aac04505bf4e65f0a39a0c4c (127.14.177.130:42957)
I20260709 13:23:46.227711 26086 leader_election.cc:290] T 3a3317f4c3514df1b2bb8e44a86be92f P 9868dce521294d0b816bba7648f4ae5a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 43999d813e564dc688fad116090c95ca (127.14.177.129:38155), 233e7d29aac04505bf4e65f0a39a0c4c (127.14.177.130:42957)
I20260709 13:23:46.227854 25450 raft_consensus.cc:2468] T 3a3317f4c3514df1b2bb8e44a86be92f P 9868dce521294d0b816bba7648f4ae5a [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 233e7d29aac04505bf4e65f0a39a0c4c in term 1.
W20260709 13:23:46.230098 25424 leader_election.cc:336] T 1550ed2d2a764c08adffae679599ba41 P 9868dce521294d0b816bba7648f4ae5a [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 43999d813e564dc688fad116090c95ca (127.14.177.129:38155): Network error: Client connection negotiation failed: client connection to 127.14.177.129:38155: connect: Connection refused (error 111)
W20260709 13:23:46.241451 25276 leader_election.cc:336] T 579da2d6aabb4ae9a2b1802e1358068b P 233e7d29aac04505bf4e65f0a39a0c4c [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 43999d813e564dc688fad116090c95ca (127.14.177.129:38155): Network error: Client connection negotiation failed: client connection to 127.14.177.129:38155: connect: Connection refused (error 111)
W20260709 13:23:46.241751 25424 leader_election.cc:336] T a5df750b84e0487fa530161079deb6c2 P 9868dce521294d0b816bba7648f4ae5a [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 43999d813e564dc688fad116090c95ca (127.14.177.129:38155): Network error: Client connection negotiation failed: client connection to 127.14.177.129:38155: connect: Connection refused (error 111)
I20260709 13:23:46.241940 26141 raft_consensus.cc:493] T 434a54fcee7d4aeeab42f739855c0e8a P 233e7d29aac04505bf4e65f0a39a0c4c [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 43999d813e564dc688fad116090c95ca)
W20260709 13:23:46.242288 25424 leader_election.cc:336] T 8ef061028f38450e83324ac78cdaa14c P 9868dce521294d0b816bba7648f4ae5a [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 43999d813e564dc688fad116090c95ca (127.14.177.129:38155): Network error: Client connection negotiation failed: client connection to 127.14.177.129:38155: connect: Connection refused (error 111)
I20260709 13:23:46.242445 26141 raft_consensus.cc:515] T 434a54fcee7d4aeeab42f739855c0e8a P 233e7d29aac04505bf4e65f0a39a0c4c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } }
I20260709 13:23:46.244910 26141 leader_election.cc:290] T 434a54fcee7d4aeeab42f739855c0e8a P 233e7d29aac04505bf4e65f0a39a0c4c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 43999d813e564dc688fad116090c95ca (127.14.177.129:38155), 3e07d64466934da4bb3c02262b1c2e38 (127.14.177.131:41539)
I20260709 13:23:46.230329 26134 leader_election.cc:290] T f1ac2f4d2fe2424bb470df42199219e2 P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9868dce521294d0b816bba7648f4ae5a (127.14.177.132:33683), 43999d813e564dc688fad116090c95ca (127.14.177.129:38155)
I20260709 13:23:46.232483 26118 tablet_bootstrap.cc:492] T 441d780c3945463f92f31170f31e37a8 P 43999d813e564dc688fad116090c95ca: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:23:46.246048 26135 raft_consensus.cc:2804] T 579da2d6aabb4ae9a2b1802e1358068b P 9868dce521294d0b816bba7648f4ae5a [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:23:46.246493 26135 raft_consensus.cc:493] T 579da2d6aabb4ae9a2b1802e1358068b P 9868dce521294d0b816bba7648f4ae5a [term 1 FOLLOWER]: Starting leader election (detected failure of leader 43999d813e564dc688fad116090c95ca)
I20260709 13:23:46.233536 25309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13bb3a23315a41c7a40e10b954b15c8e" candidate_uuid: "3e07d64466934da4bb3c02262b1c2e38" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" is_pre_election: true
I20260709 13:23:46.246918 26118 tablet_bootstrap.cc:492] T 441d780c3945463f92f31170f31e37a8 P 43999d813e564dc688fad116090c95ca: Bootstrap complete.
I20260709 13:23:46.246959 26135 raft_consensus.cc:3060] T 579da2d6aabb4ae9a2b1802e1358068b P 9868dce521294d0b816bba7648f4ae5a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:23:46.247529 25309 raft_consensus.cc:2468] T 13bb3a23315a41c7a40e10b954b15c8e P 233e7d29aac04505bf4e65f0a39a0c4c [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3e07d64466934da4bb3c02262b1c2e38 in term 1.
I20260709 13:23:46.247833 26118 ts_tablet_manager.cc:1403] T 441d780c3945463f92f31170f31e37a8 P 43999d813e564dc688fad116090c95ca: Time spent bootstrapping tablet: real 0.122s	user 0.026s	sys 0.013s
I20260709 13:23:46.230484 25380 raft_consensus.cc:2468] T f1ac2f4d2fe2424bb470df42199219e2 P 3e07d64466934da4bb3c02262b1c2e38 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9868dce521294d0b816bba7648f4ae5a in term 1.
I20260709 13:23:46.247663 25425 leader_election.cc:304] T fa0da2c845dc4059968332f4db2e0aa6 P 9868dce521294d0b816bba7648f4ae5a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 233e7d29aac04505bf4e65f0a39a0c4c, 9868dce521294d0b816bba7648f4ae5a; no voters: 43999d813e564dc688fad116090c95ca
I20260709 13:23:46.249737 26145 raft_consensus.cc:2804] T fa0da2c845dc4059968332f4db2e0aa6 P 9868dce521294d0b816bba7648f4ae5a [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260709 13:23:46.246357 25424 leader_election.cc:336] T 3a3317f4c3514df1b2bb8e44a86be92f P 9868dce521294d0b816bba7648f4ae5a [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 43999d813e564dc688fad116090c95ca (127.14.177.129:38155): Network error: Client connection negotiation failed: client connection to 127.14.177.129:38155: connect: Connection refused (error 111)
I20260709 13:23:46.236066 26131 leader_election.cc:290] T 434a54fcee7d4aeeab42f739855c0e8a P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 43999d813e564dc688fad116090c95ca (127.14.177.129:38155), 233e7d29aac04505bf4e65f0a39a0c4c (127.14.177.130:42957)
I20260709 13:23:46.250139 26145 raft_consensus.cc:493] T fa0da2c845dc4059968332f4db2e0aa6 P 9868dce521294d0b816bba7648f4ae5a [term 1 FOLLOWER]: Starting leader election (detected failure of leader 43999d813e564dc688fad116090c95ca)
I20260709 13:23:46.255846 26145 raft_consensus.cc:3060] T fa0da2c845dc4059968332f4db2e0aa6 P 9868dce521294d0b816bba7648f4ae5a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:23:46.228118 26137 raft_consensus.cc:493] T ce9c60e3922c4219b6e4453fe0259b6a P 233e7d29aac04505bf4e65f0a39a0c4c [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:23:46.257970 26137 raft_consensus.cc:515] T ce9c60e3922c4219b6e4453fe0259b6a P 233e7d29aac04505bf4e65f0a39a0c4c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } }
I20260709 13:23:46.232730 26138 leader_election.cc:290] T d0a0e4d32ce943fbb773bb840863647b P 9868dce521294d0b816bba7648f4ae5a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3e07d64466934da4bb3c02262b1c2e38 (127.14.177.131:41539), 43999d813e564dc688fad116090c95ca (127.14.177.129:38155)
I20260709 13:23:46.252123 25422 leader_election.cc:304] T b9d550d881b04735bbf89751952c09ec P 9868dce521294d0b816bba7648f4ae5a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3e07d64466934da4bb3c02262b1c2e38, 9868dce521294d0b816bba7648f4ae5a; no voters: 43999d813e564dc688fad116090c95ca
I20260709 13:23:46.260517 26138 raft_consensus.cc:2804] T b9d550d881b04735bbf89751952c09ec P 9868dce521294d0b816bba7648f4ae5a [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:23:46.260867 26138 raft_consensus.cc:493] T b9d550d881b04735bbf89751952c09ec P 9868dce521294d0b816bba7648f4ae5a [term 1 FOLLOWER]: Starting leader election (detected failure of leader 43999d813e564dc688fad116090c95ca)
I20260709 13:23:46.261211 26138 raft_consensus.cc:3060] T b9d550d881b04735bbf89751952c09ec P 9868dce521294d0b816bba7648f4ae5a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:23:46.264621 26145 raft_consensus.cc:515] T fa0da2c845dc4059968332f4db2e0aa6 P 9868dce521294d0b816bba7648f4ae5a [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } }
I20260709 13:23:46.250931 26118 raft_consensus.cc:359] T 441d780c3945463f92f31170f31e37a8 P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } }
I20260709 13:23:46.268013 25457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "441d780c3945463f92f31170f31e37a8" candidate_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9868dce521294d0b816bba7648f4ae5a" is_pre_election: true
I20260709 13:23:46.251688 26143 raft_consensus.cc:2749] T 821a5866429f4173820fb64336abee02 P 9868dce521294d0b816bba7648f4ae5a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:23:46.268249 26118 raft_consensus.cc:740] T 441d780c3945463f92f31170f31e37a8 P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43999d813e564dc688fad116090c95ca, State: Initialized, Role: FOLLOWER
I20260709 13:23:46.268769 25457 raft_consensus.cc:2468] T 441d780c3945463f92f31170f31e37a8 P 9868dce521294d0b816bba7648f4ae5a [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 233e7d29aac04505bf4e65f0a39a0c4c in term 1.
W20260709 13:23:46.254832 25424 leader_election.cc:336] T d0a0e4d32ce943fbb773bb840863647b P 9868dce521294d0b816bba7648f4ae5a [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 43999d813e564dc688fad116090c95ca (127.14.177.129:38155): Network error: Client connection negotiation failed: client connection to 127.14.177.129:38155: connect: Connection refused (error 111)
I20260709 13:23:46.267381 25274 leader_election.cc:304] T 579da2d6aabb4ae9a2b1802e1358068b P 233e7d29aac04505bf4e65f0a39a0c4c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 233e7d29aac04505bf4e65f0a39a0c4c, 9868dce521294d0b816bba7648f4ae5a; no voters: 43999d813e564dc688fad116090c95ca
I20260709 13:23:46.269205 26118 consensus_queue.cc:260] T 441d780c3945463f92f31170f31e37a8 P 43999d813e564dc688fad116090c95ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } }
I20260709 13:23:46.270488 25274 leader_election.cc:304] T 3a3317f4c3514df1b2bb8e44a86be92f P 233e7d29aac04505bf4e65f0a39a0c4c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 233e7d29aac04505bf4e65f0a39a0c4c, 9868dce521294d0b816bba7648f4ae5a; no voters: 43999d813e564dc688fad116090c95ca
I20260709 13:23:46.271875 25457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c7e7acbb09f4d49828fa652ca6abaac" candidate_uuid: "3e07d64466934da4bb3c02262b1c2e38" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9868dce521294d0b816bba7648f4ae5a" is_pre_election: true
I20260709 13:23:46.272219 25454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1ac2f4d2fe2424bb470df42199219e2" candidate_uuid: "3e07d64466934da4bb3c02262b1c2e38" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9868dce521294d0b816bba7648f4ae5a" is_pre_election: true
I20260709 13:23:46.272523 26139 raft_consensus.cc:2804] T 579da2d6aabb4ae9a2b1802e1358068b P 233e7d29aac04505bf4e65f0a39a0c4c [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:23:46.272742 25457 raft_consensus.cc:2393] T 5c7e7acbb09f4d49828fa652ca6abaac P 9868dce521294d0b816bba7648f4ae5a [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3e07d64466934da4bb3c02262b1c2e38 in current term 2: Already voted for candidate 9868dce521294d0b816bba7648f4ae5a in this term.
I20260709 13:23:46.272898 25454 raft_consensus.cc:2468] T f1ac2f4d2fe2424bb470df42199219e2 P 9868dce521294d0b816bba7648f4ae5a [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3e07d64466934da4bb3c02262b1c2e38 in term 1.
I20260709 13:23:46.273260 26141 raft_consensus.cc:2804] T 3a3317f4c3514df1b2bb8e44a86be92f P 233e7d29aac04505bf4e65f0a39a0c4c [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:23:46.273666 26141 raft_consensus.cc:493] T 3a3317f4c3514df1b2bb8e44a86be92f P 233e7d29aac04505bf4e65f0a39a0c4c [term 1 FOLLOWER]: Starting leader election (detected failure of leader 43999d813e564dc688fad116090c95ca)
I20260709 13:23:46.272917 26139 raft_consensus.cc:493] T 579da2d6aabb4ae9a2b1802e1358068b P 233e7d29aac04505bf4e65f0a39a0c4c [term 1 FOLLOWER]: Starting leader election (detected failure of leader 43999d813e564dc688fad116090c95ca)
I20260709 13:23:46.274075 26141 raft_consensus.cc:3060] T 3a3317f4c3514df1b2bb8e44a86be92f P 233e7d29aac04505bf4e65f0a39a0c4c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:23:46.274204 26139 raft_consensus.cc:3060] T 579da2d6aabb4ae9a2b1802e1358068b P 233e7d29aac04505bf4e65f0a39a0c4c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:23:46.275211 26135 raft_consensus.cc:515] T 579da2d6aabb4ae9a2b1802e1358068b P 9868dce521294d0b816bba7648f4ae5a [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } }
I20260709 13:23:46.276005 26118 ts_tablet_manager.cc:1434] T 441d780c3945463f92f31170f31e37a8 P 43999d813e564dc688fad116090c95ca: Time spent starting tablet: real 0.028s	user 0.005s	sys 0.001s
I20260709 13:23:46.276887 26118 tablet_bootstrap.cc:492] T 28c0b6fc9bd64d60828d705c6a9ab1e4 P 43999d813e564dc688fad116090c95ca: Bootstrap starting.
I20260709 13:23:46.280179 25303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a3317f4c3514df1b2bb8e44a86be92f" candidate_uuid: "9868dce521294d0b816bba7648f4ae5a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" is_pre_election: true
I20260709 13:23:46.282876 25454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa0da2c845dc4059968332f4db2e0aa6" candidate_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9868dce521294d0b816bba7648f4ae5a" is_pre_election: true
I20260709 13:23:46.283839 25454 raft_consensus.cc:2393] T fa0da2c845dc4059968332f4db2e0aa6 P 9868dce521294d0b816bba7648f4ae5a [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 233e7d29aac04505bf4e65f0a39a0c4c in current term 2: Already voted for candidate 9868dce521294d0b816bba7648f4ae5a in this term.
I20260709 13:23:46.285000 25309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5df750b84e0487fa530161079deb6c2" candidate_uuid: "9868dce521294d0b816bba7648f4ae5a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" is_pre_election: true
I20260709 13:23:46.285719 25309 raft_consensus.cc:2468] T a5df750b84e0487fa530161079deb6c2 P 233e7d29aac04505bf4e65f0a39a0c4c [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9868dce521294d0b816bba7648f4ae5a in term 1.
I20260709 13:23:46.280299 25308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ef061028f38450e83324ac78cdaa14c" candidate_uuid: "9868dce521294d0b816bba7648f4ae5a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" is_pre_election: true
I20260709 13:23:46.286873 25310 raft_consensus.cc:1275] T cfad575709c943cba40ccab278350770 P 233e7d29aac04505bf4e65f0a39a0c4c [term 4 FOLLOWER]: Refusing update from remote peer 9868dce521294d0b816bba7648f4ae5a: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 4 index: 2. (index mismatch)
I20260709 13:23:46.287098 25308 raft_consensus.cc:2468] T 8ef061028f38450e83324ac78cdaa14c P 233e7d29aac04505bf4e65f0a39a0c4c [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9868dce521294d0b816bba7648f4ae5a in term 1.
I20260709 13:23:46.288316 26145 leader_election.cc:290] T fa0da2c845dc4059968332f4db2e0aa6 P 9868dce521294d0b816bba7648f4ae5a [CANDIDATE]: Term 2 election: Requested vote from peers 43999d813e564dc688fad116090c95ca (127.14.177.129:38155), 233e7d29aac04505bf4e65f0a39a0c4c (127.14.177.130:42957)
I20260709 13:23:46.287271 26135 leader_election.cc:290] T 579da2d6aabb4ae9a2b1802e1358068b P 9868dce521294d0b816bba7648f4ae5a [CANDIDATE]: Term 2 election: Requested vote from peers 43999d813e564dc688fad116090c95ca (127.14.177.129:38155), 233e7d29aac04505bf4e65f0a39a0c4c (127.14.177.130:42957)
I20260709 13:23:46.290835 25380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b155b866f10547fd8239579a81ecf5f9" candidate_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3e07d64466934da4bb3c02262b1c2e38" is_pre_election: true
I20260709 13:23:46.291734 25380 raft_consensus.cc:2393] T b155b866f10547fd8239579a81ecf5f9 P 3e07d64466934da4bb3c02262b1c2e38 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 233e7d29aac04505bf4e65f0a39a0c4c in current term 2: Already voted for candidate 3e07d64466934da4bb3c02262b1c2e38 in this term.
W20260709 13:23:46.292557 25350 leader_election.cc:336] T d0a0e4d32ce943fbb773bb840863647b P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 43999d813e564dc688fad116090c95ca (127.14.177.129:38155): Network error: Client connection negotiation failed: client connection to 127.14.177.129:38155: connect: Connection refused (error 111)
I20260709 13:23:46.292721 25381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434a54fcee7d4aeeab42f739855c0e8a" candidate_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3e07d64466934da4bb3c02262b1c2e38" is_pre_election: true
I20260709 13:23:46.293087 25350 leader_election.cc:304] T d0a0e4d32ce943fbb773bb840863647b P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3e07d64466934da4bb3c02262b1c2e38; no voters: 43999d813e564dc688fad116090c95ca, 9868dce521294d0b816bba7648f4ae5a
I20260709 13:23:46.293396 25381 raft_consensus.cc:2468] T 434a54fcee7d4aeeab42f739855c0e8a P 3e07d64466934da4bb3c02262b1c2e38 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 233e7d29aac04505bf4e65f0a39a0c4c in term 3.
I20260709 13:23:46.294389 26134 raft_consensus.cc:2749] T d0a0e4d32ce943fbb773bb840863647b P 3e07d64466934da4bb3c02262b1c2e38 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:23:46.294667 25310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "579da2d6aabb4ae9a2b1802e1358068b" candidate_uuid: "9868dce521294d0b816bba7648f4ae5a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "233e7d29aac04505bf4e65f0a39a0c4c"
I20260709 13:23:46.295377 25308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa0da2c845dc4059968332f4db2e0aa6" candidate_uuid: "9868dce521294d0b816bba7648f4ae5a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "233e7d29aac04505bf4e65f0a39a0c4c"
I20260709 13:23:46.295959 25308 raft_consensus.cc:3060] T fa0da2c845dc4059968332f4db2e0aa6 P 233e7d29aac04505bf4e65f0a39a0c4c [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:23:46.296715 25350 leader_election.cc:336] T 4d837697cd0f402cacf9db89a5d92652 P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 43999d813e564dc688fad116090c95ca (127.14.177.129:38155): Network error: Client connection negotiation failed: client connection to 127.14.177.129:38155: connect: Connection refused (error 111)
W20260709 13:23:46.298691 25350 leader_election.cc:336] T 0e71b6f5c69b4b35a8ac51c22c7d5f0d P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 43999d813e564dc688fad116090c95ca (127.14.177.129:38155): Network error: Client connection negotiation failed: client connection to 127.14.177.129:38155: connect: Connection refused (error 111)
I20260709 13:23:46.285426 26141 raft_consensus.cc:515] T 3a3317f4c3514df1b2bb8e44a86be92f P 233e7d29aac04505bf4e65f0a39a0c4c [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } }
W20260709 13:23:46.300254 25424 leader_election.cc:336] T fa0da2c845dc4059968332f4db2e0aa6 P 9868dce521294d0b816bba7648f4ae5a [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 43999d813e564dc688fad116090c95ca (127.14.177.129:38155): Network error: Client connection negotiation failed: client connection to 127.14.177.129:38155: connect: Connection refused (error 111)
I20260709 13:23:46.301381 25303 raft_consensus.cc:2393] T 3a3317f4c3514df1b2bb8e44a86be92f P 233e7d29aac04505bf4e65f0a39a0c4c [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9868dce521294d0b816bba7648f4ae5a in current term 2: Already voted for candidate 233e7d29aac04505bf4e65f0a39a0c4c in this term.
I20260709 13:23:46.301198 26139 raft_consensus.cc:515] T 579da2d6aabb4ae9a2b1802e1358068b P 233e7d29aac04505bf4e65f0a39a0c4c [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } }
I20260709 13:23:46.302371 25310 raft_consensus.cc:2393] T 579da2d6aabb4ae9a2b1802e1358068b P 233e7d29aac04505bf4e65f0a39a0c4c [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 9868dce521294d0b816bba7648f4ae5a in current term 2: Already voted for candidate 233e7d29aac04505bf4e65f0a39a0c4c in this term.
I20260709 13:23:46.303771 26141 leader_election.cc:290] T 3a3317f4c3514df1b2bb8e44a86be92f P 233e7d29aac04505bf4e65f0a39a0c4c [CANDIDATE]: Term 2 election: Requested vote from peers 9868dce521294d0b816bba7648f4ae5a (127.14.177.132:33683), 43999d813e564dc688fad116090c95ca (127.14.177.129:38155)
I20260709 13:23:46.305895 26139 leader_election.cc:290] T 579da2d6aabb4ae9a2b1802e1358068b P 233e7d29aac04505bf4e65f0a39a0c4c [CANDIDATE]: Term 2 election: Requested vote from peers 9868dce521294d0b816bba7648f4ae5a (127.14.177.132:33683), 43999d813e564dc688fad116090c95ca (127.14.177.129:38155)
I20260709 13:23:46.283757 25274 leader_election.cc:304] T 441d780c3945463f92f31170f31e37a8 P 233e7d29aac04505bf4e65f0a39a0c4c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 233e7d29aac04505bf4e65f0a39a0c4c, 9868dce521294d0b816bba7648f4ae5a; no voters: 
W20260709 13:23:46.312786 25276 leader_election.cc:336] T 441d780c3945463f92f31170f31e37a8 P 233e7d29aac04505bf4e65f0a39a0c4c [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 43999d813e564dc688fad116090c95ca (127.14.177.129:38155): Network error: Client connection negotiation failed: client connection to 127.14.177.129:38155: connect: Connection refused (error 111)
I20260709 13:23:46.213362 26144 raft_consensus.cc:3060] T 4d837697cd0f402cacf9db89a5d92652 P 3e07d64466934da4bb3c02262b1c2e38 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:23:46.306207 25422 leader_election.cc:304] T f1ac2f4d2fe2424bb470df42199219e2 P 9868dce521294d0b816bba7648f4ae5a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3e07d64466934da4bb3c02262b1c2e38, 9868dce521294d0b816bba7648f4ae5a; no voters: 43999d813e564dc688fad116090c95ca
I20260709 13:23:46.226752 25351 leader_election.cc:304] T b7ff8dde9f5d4c49b9203d902f6b94c4 P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 233e7d29aac04505bf4e65f0a39a0c4c, 3e07d64466934da4bb3c02262b1c2e38; no voters: 
W20260709 13:23:46.316800 25276 leader_election.cc:336] T fa0da2c845dc4059968332f4db2e0aa6 P 233e7d29aac04505bf4e65f0a39a0c4c [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 43999d813e564dc688fad116090c95ca (127.14.177.129:38155): Network error: Client connection negotiation failed: client connection to 127.14.177.129:38155: connect: Connection refused (error 111)
I20260709 13:23:46.317507 25276 leader_election.cc:304] T fa0da2c845dc4059968332f4db2e0aa6 P 233e7d29aac04505bf4e65f0a39a0c4c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 233e7d29aac04505bf4e65f0a39a0c4c; no voters: 43999d813e564dc688fad116090c95ca, 9868dce521294d0b816bba7648f4ae5a
I20260709 13:23:46.315044 26141 raft_consensus.cc:2804] T 441d780c3945463f92f31170f31e37a8 P 233e7d29aac04505bf4e65f0a39a0c4c [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260709 13:23:46.315877 25351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:42957, user_credentials={real_user=slave}} blocked reactor thread for 89218us
I20260709 13:23:46.311582 25380 raft_consensus.cc:1275] T cfad575709c943cba40ccab278350770 P 3e07d64466934da4bb3c02262b1c2e38 [term 4 FOLLOWER]: Refusing update from remote peer 9868dce521294d0b816bba7648f4ae5a: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 4 index: 2. (index mismatch)
I20260709 13:23:46.318595 25348 leader_election.cc:304] T 0e71b6f5c69b4b35a8ac51c22c7d5f0d P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3e07d64466934da4bb3c02262b1c2e38, 9868dce521294d0b816bba7648f4ae5a; no voters: 43999d813e564dc688fad116090c95ca
I20260709 13:23:46.318813 25454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a3317f4c3514df1b2bb8e44a86be92f" candidate_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9868dce521294d0b816bba7648f4ae5a"
W20260709 13:23:46.306185 25424 leader_election.cc:336] T 579da2d6aabb4ae9a2b1802e1358068b P 9868dce521294d0b816bba7648f4ae5a [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 43999d813e564dc688fad116090c95ca (127.14.177.129:38155): Network error: Client connection negotiation failed: client connection to 127.14.177.129:38155: connect: Connection refused (error 111)
I20260709 13:23:46.319499 25454 raft_consensus.cc:3060] T 3a3317f4c3514df1b2bb8e44a86be92f P 9868dce521294d0b816bba7648f4ae5a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:23:46.319566 26133 raft_consensus.cc:2804] T 0e71b6f5c69b4b35a8ac51c22c7d5f0d P 3e07d64466934da4bb3c02262b1c2e38 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:23:46.318220 26141 raft_consensus.cc:493] T 441d780c3945463f92f31170f31e37a8 P 233e7d29aac04505bf4e65f0a39a0c4c [term 1 FOLLOWER]: Starting leader election (detected failure of leader 43999d813e564dc688fad116090c95ca)
I20260709 13:23:46.320129 26133 raft_consensus.cc:493] T 0e71b6f5c69b4b35a8ac51c22c7d5f0d P 3e07d64466934da4bb3c02262b1c2e38 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 43999d813e564dc688fad116090c95ca)
I20260709 13:23:46.320363 26141 raft_consensus.cc:3060] T 441d780c3945463f92f31170f31e37a8 P 233e7d29aac04505bf4e65f0a39a0c4c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:23:46.320564 26133 raft_consensus.cc:3060] T 0e71b6f5c69b4b35a8ac51c22c7d5f0d P 3e07d64466934da4bb3c02262b1c2e38 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:23:46.321470 25457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "579da2d6aabb4ae9a2b1802e1358068b" candidate_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9868dce521294d0b816bba7648f4ae5a"
I20260709 13:23:46.322309 25457 raft_consensus.cc:2393] T 579da2d6aabb4ae9a2b1802e1358068b P 9868dce521294d0b816bba7648f4ae5a [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 233e7d29aac04505bf4e65f0a39a0c4c in current term 2: Already voted for candidate 9868dce521294d0b816bba7648f4ae5a in this term.
I20260709 13:23:46.315148 26134 raft_consensus.cc:493] T 821a5866429f4173820fb64336abee02 P 3e07d64466934da4bb3c02262b1c2e38 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 43999d813e564dc688fad116090c95ca)
I20260709 13:23:46.310628 26137 leader_election.cc:290] T ce9c60e3922c4219b6e4453fe0259b6a P 233e7d29aac04505bf4e65f0a39a0c4c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9868dce521294d0b816bba7648f4ae5a (127.14.177.132:33683), 43999d813e564dc688fad116090c95ca (127.14.177.129:38155)
I20260709 13:23:46.323347 26134 raft_consensus.cc:515] T 821a5866429f4173820fb64336abee02 P 3e07d64466934da4bb3c02262b1c2e38 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } }
I20260709 13:23:46.315419 26135 raft_consensus.cc:2804] T f1ac2f4d2fe2424bb470df42199219e2 P 9868dce521294d0b816bba7648f4ae5a [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:23:46.325387 25457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce9c60e3922c4219b6e4453fe0259b6a" candidate_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9868dce521294d0b816bba7648f4ae5a" is_pre_election: true
W20260709 13:23:46.325664 25424 consensus_peers.cc:597] T 6242888a4d4146dcacd87d0fd9a832f4 P 9868dce521294d0b816bba7648f4ae5a -> Peer 43999d813e564dc688fad116090c95ca (127.14.177.129:38155): Couldn't send request to peer 43999d813e564dc688fad116090c95ca. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:38155: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:23:46.326117 25457 raft_consensus.cc:2468] T ce9c60e3922c4219b6e4453fe0259b6a P 9868dce521294d0b816bba7648f4ae5a [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 233e7d29aac04505bf4e65f0a39a0c4c in term 2.
I20260709 13:23:46.325676 26135 raft_consensus.cc:493] T f1ac2f4d2fe2424bb470df42199219e2 P 9868dce521294d0b816bba7648f4ae5a [term 1 FOLLOWER]: Starting leader election (detected failure of leader 43999d813e564dc688fad116090c95ca)
I20260709 13:23:46.326743 26135 raft_consensus.cc:3060] T f1ac2f4d2fe2424bb470df42199219e2 P 9868dce521294d0b816bba7648f4ae5a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:23:46.325397 26134 leader_election.cc:290] T 821a5866429f4173820fb64336abee02 P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9868dce521294d0b816bba7648f4ae5a (127.14.177.132:33683), 43999d813e564dc688fad116090c95ca (127.14.177.129:38155)
I20260709 13:23:46.328186 25380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1550ed2d2a764c08adffae679599ba41" candidate_uuid: "9868dce521294d0b816bba7648f4ae5a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3e07d64466934da4bb3c02262b1c2e38"
I20260709 13:23:46.328830 25380 raft_consensus.cc:3060] T 1550ed2d2a764c08adffae679599ba41 P 3e07d64466934da4bb3c02262b1c2e38 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:23:46.329854 25381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0a0e4d32ce943fbb773bb840863647b" candidate_uuid: "9868dce521294d0b816bba7648f4ae5a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3e07d64466934da4bb3c02262b1c2e38" is_pre_election: true
I20260709 13:23:46.330516 25381 raft_consensus.cc:2468] T d0a0e4d32ce943fbb773bb840863647b P 3e07d64466934da4bb3c02262b1c2e38 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9868dce521294d0b816bba7648f4ae5a in term 1.
I20260709 13:23:46.330636 25454 raft_consensus.cc:2468] T 3a3317f4c3514df1b2bb8e44a86be92f P 9868dce521294d0b816bba7648f4ae5a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 233e7d29aac04505bf4e65f0a39a0c4c in term 2.
I20260709 13:23:46.329873 25274 leader_election.cc:304] T b7ff8dde9f5d4c49b9203d902f6b94c4 P 233e7d29aac04505bf4e65f0a39a0c4c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 233e7d29aac04505bf4e65f0a39a0c4c, 3e07d64466934da4bb3c02262b1c2e38; no voters: 43999d813e564dc688fad116090c95ca
I20260709 13:23:46.318465 26131 raft_consensus.cc:2804] T b7ff8dde9f5d4c49b9203d902f6b94c4 P 3e07d64466934da4bb3c02262b1c2e38 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:23:46.331535 26131 raft_consensus.cc:493] T b7ff8dde9f5d4c49b9203d902f6b94c4 P 3e07d64466934da4bb3c02262b1c2e38 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 43999d813e564dc688fad116090c95ca)
I20260709 13:23:46.331913 26131 raft_consensus.cc:3060] T b7ff8dde9f5d4c49b9203d902f6b94c4 P 3e07d64466934da4bb3c02262b1c2e38 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:23:46.333923 25276 leader_election.cc:336] T b155b866f10547fd8239579a81ecf5f9 P 233e7d29aac04505bf4e65f0a39a0c4c [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 43999d813e564dc688fad116090c95ca (127.14.177.129:38155): Network error: Client connection negotiation failed: client connection to 127.14.177.129:38155: connect: Connection refused (error 111)
I20260709 13:23:46.335047 25276 leader_election.cc:304] T b155b866f10547fd8239579a81ecf5f9 P 233e7d29aac04505bf4e65f0a39a0c4c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 233e7d29aac04505bf4e65f0a39a0c4c; no voters: 3e07d64466934da4bb3c02262b1c2e38, 43999d813e564dc688fad116090c95ca
I20260709 13:23:46.333137 26137 raft_consensus.cc:2804] T b7ff8dde9f5d4c49b9203d902f6b94c4 P 233e7d29aac04505bf4e65f0a39a0c4c [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:23:46.332376 25274 leader_election.cc:304] T 434a54fcee7d4aeeab42f739855c0e8a P 233e7d29aac04505bf4e65f0a39a0c4c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 233e7d29aac04505bf4e65f0a39a0c4c, 3e07d64466934da4bb3c02262b1c2e38; no voters: 
I20260709 13:23:46.337960 25425 leader_election.cc:304] T a5df750b84e0487fa530161079deb6c2 P 9868dce521294d0b816bba7648f4ae5a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 233e7d29aac04505bf4e65f0a39a0c4c, 9868dce521294d0b816bba7648f4ae5a; no voters: 43999d813e564dc688fad116090c95ca
I20260709 13:23:46.340040 25425 leader_election.cc:304] T 8ef061028f38450e83324ac78cdaa14c P 9868dce521294d0b816bba7648f4ae5a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 233e7d29aac04505bf4e65f0a39a0c4c, 9868dce521294d0b816bba7648f4ae5a; no voters: 43999d813e564dc688fad116090c95ca
I20260709 13:23:46.320163 25348 leader_election.cc:304] T f1ac2f4d2fe2424bb470df42199219e2 P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e07d64466934da4bb3c02262b1c2e38, 9868dce521294d0b816bba7648f4ae5a; no voters: 
I20260709 13:23:46.341548 25303 raft_consensus.cc:1275] T 467a6b42ca6548269c12dddde696f7af P 233e7d29aac04505bf4e65f0a39a0c4c [term 3 FOLLOWER]: Refusing update from remote peer 3e07d64466934da4bb3c02262b1c2e38: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:23:46.341689 26086 raft_consensus.cc:2804] T a5df750b84e0487fa530161079deb6c2 P 9868dce521294d0b816bba7648f4ae5a [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:23:46.342883 25425 leader_election.cc:304] T 3a3317f4c3514df1b2bb8e44a86be92f P 9868dce521294d0b816bba7648f4ae5a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9868dce521294d0b816bba7648f4ae5a; no voters: 233e7d29aac04505bf4e65f0a39a0c4c, 43999d813e564dc688fad116090c95ca
I20260709 13:23:46.339051 25274 leader_election.cc:304] T ce9c60e3922c4219b6e4453fe0259b6a P 233e7d29aac04505bf4e65f0a39a0c4c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 233e7d29aac04505bf4e65f0a39a0c4c, 9868dce521294d0b816bba7648f4ae5a; no voters: 
I20260709 13:23:46.342167 26086 raft_consensus.cc:493] T a5df750b84e0487fa530161079deb6c2 P 9868dce521294d0b816bba7648f4ae5a [term 1 FOLLOWER]: Starting leader election (detected failure of leader 43999d813e564dc688fad116090c95ca)
I20260709 13:23:46.344779 26086 raft_consensus.cc:3060] T a5df750b84e0487fa530161079deb6c2 P 9868dce521294d0b816bba7648f4ae5a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:23:46.344797 25310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434a54fcee7d4aeeab42f739855c0e8a" candidate_uuid: "3e07d64466934da4bb3c02262b1c2e38" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" is_pre_election: true
I20260709 13:23:46.346668 25308 raft_consensus.cc:2468] T fa0da2c845dc4059968332f4db2e0aa6 P 233e7d29aac04505bf4e65f0a39a0c4c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9868dce521294d0b816bba7648f4ae5a in term 2.
I20260709 13:23:46.347266 26138 raft_consensus.cc:515] T b9d550d881b04735bbf89751952c09ec P 9868dce521294d0b816bba7648f4ae5a [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } }
I20260709 13:23:46.348363 26022 consensus_queue.cc:1048] T cfad575709c943cba40ccab278350770 P 9868dce521294d0b816bba7648f4ae5a [LEADER]: Connected to new peer: Peer: permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:23:46.349630 26138 leader_election.cc:290] T b9d550d881b04735bbf89751952c09ec P 9868dce521294d0b816bba7648f4ae5a [CANDIDATE]: Term 2 election: Requested vote from peers 3e07d64466934da4bb3c02262b1c2e38 (127.14.177.131:41539), 43999d813e564dc688fad116090c95ca (127.14.177.129:38155)
I20260709 13:23:46.350127 26083 consensus_queue.cc:1048] T cfad575709c943cba40ccab278350770 P 9868dce521294d0b816bba7648f4ae5a [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:23:46.351563 26143 raft_consensus.cc:2804] T 8ef061028f38450e83324ac78cdaa14c P 9868dce521294d0b816bba7648f4ae5a [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:23:46.351989 26143 raft_consensus.cc:493] T 8ef061028f38450e83324ac78cdaa14c P 9868dce521294d0b816bba7648f4ae5a [term 1 FOLLOWER]: Starting leader election (detected failure of leader 43999d813e564dc688fad116090c95ca)
I20260709 13:23:46.352335 26143 raft_consensus.cc:3060] T 8ef061028f38450e83324ac78cdaa14c P 9868dce521294d0b816bba7648f4ae5a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:23:46.342612 26140 raft_consensus.cc:2804] T 434a54fcee7d4aeeab42f739855c0e8a P 233e7d29aac04505bf4e65f0a39a0c4c [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:23:46.353621 26140 raft_consensus.cc:493] T 434a54fcee7d4aeeab42f739855c0e8a P 233e7d29aac04505bf4e65f0a39a0c4c [term 3 FOLLOWER]: Starting leader election (detected failure of leader 43999d813e564dc688fad116090c95ca)
I20260709 13:23:46.354005 26140 raft_consensus.cc:3060] T 434a54fcee7d4aeeab42f739855c0e8a P 233e7d29aac04505bf4e65f0a39a0c4c [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:23:46.354391 25380 raft_consensus.cc:2468] T 1550ed2d2a764c08adffae679599ba41 P 3e07d64466934da4bb3c02262b1c2e38 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9868dce521294d0b816bba7648f4ae5a in term 2.
I20260709 13:23:46.356593 25425 leader_election.cc:304] T 579da2d6aabb4ae9a2b1802e1358068b P 9868dce521294d0b816bba7648f4ae5a [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9868dce521294d0b816bba7648f4ae5a; no voters: 233e7d29aac04505bf4e65f0a39a0c4c, 43999d813e564dc688fad116090c95ca
I20260709 13:23:46.338699 25351 leader_election.cc:304] T 72b56266e8a54776b9113c7a786e1952 P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 233e7d29aac04505bf4e65f0a39a0c4c, 3e07d64466934da4bb3c02262b1c2e38; no voters: 
I20260709 13:23:46.358224 25458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "821a5866429f4173820fb64336abee02" candidate_uuid: "3e07d64466934da4bb3c02262b1c2e38" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9868dce521294d0b816bba7648f4ae5a" is_pre_election: true
I20260709 13:23:46.359443 25458 raft_consensus.cc:2468] T 821a5866429f4173820fb64336abee02 P 9868dce521294d0b816bba7648f4ae5a [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3e07d64466934da4bb3c02262b1c2e38 in term 1.
I20260709 13:23:46.367408 26118 tablet_bootstrap.cc:492] T 28c0b6fc9bd64d60828d705c6a9ab1e4 P 43999d813e564dc688fad116090c95ca: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:23:46.359311 25351 leader_election.cc:304] T 13bb3a23315a41c7a40e10b954b15c8e P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 233e7d29aac04505bf4e65f0a39a0c4c, 3e07d64466934da4bb3c02262b1c2e38; no voters: 
I20260709 13:23:46.376211 25274 leader_election.cc:304] T 3a3317f4c3514df1b2bb8e44a86be92f P 233e7d29aac04505bf4e65f0a39a0c4c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 233e7d29aac04505bf4e65f0a39a0c4c, 9868dce521294d0b816bba7648f4ae5a; no voters: 
W20260709 13:23:46.376894 25350 leader_election.cc:336] T 72b56266e8a54776b9113c7a786e1952 P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 43999d813e564dc688fad116090c95ca (127.14.177.129:38155): Network error: Client connection negotiation failed: client connection to 127.14.177.129:38155: connect: Connection refused (error 111)
W20260709 13:23:46.381565 25350 leader_election.cc:336] T b7ff8dde9f5d4c49b9203d902f6b94c4 P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 43999d813e564dc688fad116090c95ca (127.14.177.129:38155): Network error: Client connection negotiation failed: client connection to 127.14.177.129:38155: connect: Connection refused (error 111)
W20260709 13:23:46.382349 25350 leader_election.cc:336] T 13bb3a23315a41c7a40e10b954b15c8e P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 43999d813e564dc688fad116090c95ca (127.14.177.129:38155): Network error: Client connection negotiation failed: client connection to 127.14.177.129:38155: connect: Connection refused (error 111)
W20260709 13:23:46.383064 25350 leader_election.cc:336] T 5c7e7acbb09f4d49828fa652ca6abaac P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 43999d813e564dc688fad116090c95ca (127.14.177.129:38155): Network error: Client connection negotiation failed: client connection to 127.14.177.129:38155: connect: Connection refused (error 111)
I20260709 13:23:46.383522 25350 leader_election.cc:304] T 5c7e7acbb09f4d49828fa652ca6abaac P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3e07d64466934da4bb3c02262b1c2e38; no voters: 43999d813e564dc688fad116090c95ca, 9868dce521294d0b816bba7648f4ae5a
I20260709 13:23:46.337409 26029 raft_consensus.cc:3060] T b155b866f10547fd8239579a81ecf5f9 P 233e7d29aac04505bf4e65f0a39a0c4c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:23:46.336925 26137 raft_consensus.cc:493] T b7ff8dde9f5d4c49b9203d902f6b94c4 P 233e7d29aac04505bf4e65f0a39a0c4c [term 1 FOLLOWER]: Starting leader election (detected failure of leader 43999d813e564dc688fad116090c95ca)
I20260709 13:23:46.388998 26139 raft_consensus.cc:2804] T ce9c60e3922c4219b6e4453fe0259b6a P 233e7d29aac04505bf4e65f0a39a0c4c [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:23:46.389191 26046 raft_consensus.cc:2804] T 13bb3a23315a41c7a40e10b954b15c8e P 3e07d64466934da4bb3c02262b1c2e38 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:23:46.387115 26138 raft_consensus.cc:2749] T 579da2d6aabb4ae9a2b1802e1358068b P 9868dce521294d0b816bba7648f4ae5a [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:23:46.395871 26118 tablet_bootstrap.cc:492] T 28c0b6fc9bd64d60828d705c6a9ab1e4 P 43999d813e564dc688fad116090c95ca: Bootstrap complete.
I20260709 13:23:46.409266 26118 ts_tablet_manager.cc:1403] T 28c0b6fc9bd64d60828d705c6a9ab1e4 P 43999d813e564dc688fad116090c95ca: Time spent bootstrapping tablet: real 0.133s	user 0.017s	sys 0.002s
I20260709 13:23:46.379457 26135 raft_consensus.cc:515] T f1ac2f4d2fe2424bb470df42199219e2 P 9868dce521294d0b816bba7648f4ae5a [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } }
I20260709 13:23:46.396490 26150 raft_consensus.cc:2804] T 3a3317f4c3514df1b2bb8e44a86be92f P 233e7d29aac04505bf4e65f0a39a0c4c [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:23:46.399580 25422 leader_election.cc:304] T d0a0e4d32ce943fbb773bb840863647b P 9868dce521294d0b816bba7648f4ae5a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3e07d64466934da4bb3c02262b1c2e38, 9868dce521294d0b816bba7648f4ae5a; no voters: 43999d813e564dc688fad116090c95ca
I20260709 13:23:46.375682 26134 raft_consensus.cc:2804] T f1ac2f4d2fe2424bb470df42199219e2 P 3e07d64466934da4bb3c02262b1c2e38 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:23:46.379493 26041 raft_consensus.cc:2804] T 72b56266e8a54776b9113c7a786e1952 P 3e07d64466934da4bb3c02262b1c2e38 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:23:46.375883 26099 raft_consensus.cc:2749] T fa0da2c845dc4059968332f4db2e0aa6 P 233e7d29aac04505bf4e65f0a39a0c4c [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:23:46.379662 26136 raft_consensus.cc:2749] T 3a3317f4c3514df1b2bb8e44a86be92f P 9868dce521294d0b816bba7648f4ae5a [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:23:46.413986 25425 leader_election.cc:304] T fa0da2c845dc4059968332f4db2e0aa6 P 9868dce521294d0b816bba7648f4ae5a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 233e7d29aac04505bf4e65f0a39a0c4c, 9868dce521294d0b816bba7648f4ae5a; no voters: 43999d813e564dc688fad116090c95ca
W20260709 13:23:46.427654 25274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.132:33683, user_credentials={real_user=slave}} blocked reactor thread for 46985.5us
I20260709 13:23:46.397629 26133 raft_consensus.cc:515] T 0e71b6f5c69b4b35a8ac51c22c7d5f0d P 3e07d64466934da4bb3c02262b1c2e38 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } }
I20260709 13:23:46.431295 26135 leader_election.cc:290] T f1ac2f4d2fe2424bb470df42199219e2 P 9868dce521294d0b816bba7648f4ae5a [CANDIDATE]: Term 2 election: Requested vote from peers 3e07d64466934da4bb3c02262b1c2e38 (127.14.177.131:41539), 43999d813e564dc688fad116090c95ca (127.14.177.129:38155)
I20260709 13:23:46.431628 26132 raft_consensus.cc:3060] T 5c7e7acbb09f4d49828fa652ca6abaac P 3e07d64466934da4bb3c02262b1c2e38 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:23:46.432054 26133 leader_election.cc:290] T 0e71b6f5c69b4b35a8ac51c22c7d5f0d P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 2 election: Requested vote from peers 9868dce521294d0b816bba7648f4ae5a (127.14.177.132:33683), 43999d813e564dc688fad116090c95ca (127.14.177.129:38155)
I20260709 13:23:46.401468 26139 raft_consensus.cc:493] T ce9c60e3922c4219b6e4453fe0259b6a P 233e7d29aac04505bf4e65f0a39a0c4c [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:23:46.433766 26139 raft_consensus.cc:3060] T ce9c60e3922c4219b6e4453fe0259b6a P 233e7d29aac04505bf4e65f0a39a0c4c [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:23:46.434290 26118 raft_consensus.cc:359] T 28c0b6fc9bd64d60828d705c6a9ab1e4 P 43999d813e564dc688fad116090c95ca [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } }
I20260709 13:23:46.435015 26118 raft_consensus.cc:740] T 28c0b6fc9bd64d60828d705c6a9ab1e4 P 43999d813e564dc688fad116090c95ca [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43999d813e564dc688fad116090c95ca, State: Initialized, Role: FOLLOWER
I20260709 13:23:46.421422 26041 raft_consensus.cc:493] T 72b56266e8a54776b9113c7a786e1952 P 3e07d64466934da4bb3c02262b1c2e38 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:23:46.435613 26041 raft_consensus.cc:3060] T 72b56266e8a54776b9113c7a786e1952 P 3e07d64466934da4bb3c02262b1c2e38 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:23:46.408332 26046 raft_consensus.cc:493] T 13bb3a23315a41c7a40e10b954b15c8e P 3e07d64466934da4bb3c02262b1c2e38 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 43999d813e564dc688fad116090c95ca)
I20260709 13:23:46.439747 26046 raft_consensus.cc:3060] T 13bb3a23315a41c7a40e10b954b15c8e P 3e07d64466934da4bb3c02262b1c2e38 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:23:46.439910 26136 raft_consensus.cc:2804] T d0a0e4d32ce943fbb773bb840863647b P 9868dce521294d0b816bba7648f4ae5a [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:23:46.440441 26136 raft_consensus.cc:493] T d0a0e4d32ce943fbb773bb840863647b P 9868dce521294d0b816bba7648f4ae5a [term 1 FOLLOWER]: Starting leader election (detected failure of leader 43999d813e564dc688fad116090c95ca)
I20260709 13:23:46.440855 26136 raft_consensus.cc:3060] T d0a0e4d32ce943fbb773bb840863647b P 9868dce521294d0b816bba7648f4ae5a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:23:46.440567 26141 raft_consensus.cc:515] T 441d780c3945463f92f31170f31e37a8 P 233e7d29aac04505bf4e65f0a39a0c4c [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } }
I20260709 13:23:46.442291 26118 consensus_queue.cc:260] T 28c0b6fc9bd64d60828d705c6a9ab1e4 P 43999d813e564dc688fad116090c95ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } }
I20260709 13:23:46.400614 26137 raft_consensus.cc:3060] T b7ff8dde9f5d4c49b9203d902f6b94c4 P 233e7d29aac04505bf4e65f0a39a0c4c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:23:46.444643 26118 ts_tablet_manager.cc:1434] T 28c0b6fc9bd64d60828d705c6a9ab1e4 P 43999d813e564dc688fad116090c95ca: Time spent starting tablet: real 0.035s	user 0.003s	sys 0.003s
I20260709 13:23:46.444902 26083 raft_consensus.cc:2804] T fa0da2c845dc4059968332f4db2e0aa6 P 9868dce521294d0b816bba7648f4ae5a [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:23:46.445448 26083 raft_consensus.cc:697] T fa0da2c845dc4059968332f4db2e0aa6 P 9868dce521294d0b816bba7648f4ae5a [term 2 LEADER]: Becoming Leader. State: Replica: 9868dce521294d0b816bba7648f4ae5a, State: Running, Role: LEADER
I20260709 13:23:46.445676 26118 tablet_bootstrap.cc:492] T f1ac2f4d2fe2424bb470df42199219e2 P 43999d813e564dc688fad116090c95ca: Bootstrap starting.
I20260709 13:23:46.446265 26083 consensus_queue.cc:237] T fa0da2c845dc4059968332f4db2e0aa6 P 9868dce521294d0b816bba7648f4ae5a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } }
I20260709 13:23:46.421842 26134 raft_consensus.cc:493] T f1ac2f4d2fe2424bb470df42199219e2 P 3e07d64466934da4bb3c02262b1c2e38 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 43999d813e564dc688fad116090c95ca)
I20260709 13:23:46.447593 26134 raft_consensus.cc:3060] T f1ac2f4d2fe2424bb470df42199219e2 P 3e07d64466934da4bb3c02262b1c2e38 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:23:46.447921 26144 raft_consensus.cc:515] T 4d837697cd0f402cacf9db89a5d92652 P 3e07d64466934da4bb3c02262b1c2e38 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } }
I20260709 13:23:46.450174 26144 leader_election.cc:290] T 4d837697cd0f402cacf9db89a5d92652 P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 2 election: Requested vote from peers 9868dce521294d0b816bba7648f4ae5a (127.14.177.132:33683), 43999d813e564dc688fad116090c95ca (127.14.177.129:38155)
I20260709 13:23:46.444656 26131 raft_consensus.cc:515] T b7ff8dde9f5d4c49b9203d902f6b94c4 P 3e07d64466934da4bb3c02262b1c2e38 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } }
I20260709 13:23:46.453754 26131 leader_election.cc:290] T b7ff8dde9f5d4c49b9203d902f6b94c4 P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 2 election: Requested vote from peers 43999d813e564dc688fad116090c95ca (127.14.177.129:38155), 233e7d29aac04505bf4e65f0a39a0c4c (127.14.177.130:42957)
I20260709 13:23:46.454603 25450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "441d780c3945463f92f31170f31e37a8" candidate_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9868dce521294d0b816bba7648f4ae5a"
I20260709 13:23:46.442799 26141 leader_election.cc:290] T 441d780c3945463f92f31170f31e37a8 P 233e7d29aac04505bf4e65f0a39a0c4c [CANDIDATE]: Term 2 election: Requested vote from peers 9868dce521294d0b816bba7648f4ae5a (127.14.177.132:33683), 43999d813e564dc688fad116090c95ca (127.14.177.129:38155)
I20260709 13:23:46.421355 26069 raft_consensus.cc:493] T c902bbc7ec0848ba82a04870eab1e7f5 P 9868dce521294d0b816bba7648f4ae5a [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:23:46.459537 25450 raft_consensus.cc:3060] T 441d780c3945463f92f31170f31e37a8 P 9868dce521294d0b816bba7648f4ae5a [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:23:46.472512 25350 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:38155, user_credentials={real_user=slave}} blocked reactor thread for 89465.5us
W20260709 13:23:46.473109 25350 leader_election.cc:336] T f1ac2f4d2fe2424bb470df42199219e2 P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 43999d813e564dc688fad116090c95ca (127.14.177.129:38155): Network error: Client connection negotiation failed: client connection to 127.14.177.129:38155: connect: Connection refused (error 111)
I20260709 13:23:46.473204 25348 leader_election.cc:304] T 821a5866429f4173820fb64336abee02 P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e07d64466934da4bb3c02262b1c2e38, 9868dce521294d0b816bba7648f4ae5a; no voters: 
I20260709 13:23:46.420646 26157 consensus_queue.cc:1048] T 467a6b42ca6548269c12dddde696f7af P 3e07d64466934da4bb3c02262b1c2e38 [LEADER]: Connected to new peer: Peer: permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.039s
W20260709 13:23:46.473795 25350 leader_election.cc:336] T 434a54fcee7d4aeeab42f739855c0e8a P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 43999d813e564dc688fad116090c95ca (127.14.177.129:38155): Network error: Client connection negotiation failed: client connection to 127.14.177.129:38155: connect: Connection refused (error 111)
I20260709 13:23:46.481072 25422 leader_election.cc:304] T 1550ed2d2a764c08adffae679599ba41 P 9868dce521294d0b816bba7648f4ae5a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3e07d64466934da4bb3c02262b1c2e38, 9868dce521294d0b816bba7648f4ae5a; no voters: 43999d813e564dc688fad116090c95ca
I20260709 13:23:46.486606 25379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1ac2f4d2fe2424bb470df42199219e2" candidate_uuid: "9868dce521294d0b816bba7648f4ae5a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3e07d64466934da4bb3c02262b1c2e38"
W20260709 13:23:46.491892 25424 leader_election.cc:336] T b9d550d881b04735bbf89751952c09ec P 9868dce521294d0b816bba7648f4ae5a [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 43999d813e564dc688fad116090c95ca (127.14.177.129:38155): Network error: Client connection negotiation failed: client connection to 127.14.177.129:38155: connect: Connection refused (error 111)
W20260709 13:23:46.494114 25424 leader_election.cc:336] T f1ac2f4d2fe2424bb470df42199219e2 P 9868dce521294d0b816bba7648f4ae5a [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 43999d813e564dc688fad116090c95ca (127.14.177.129:38155): Network error: Client connection negotiation failed: client connection to 127.14.177.129:38155: connect: Connection refused (error 111)
I20260709 13:23:46.474519 25457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e71b6f5c69b4b35a8ac51c22c7d5f0d" candidate_uuid: "3e07d64466934da4bb3c02262b1c2e38" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9868dce521294d0b816bba7648f4ae5a"
W20260709 13:23:46.474488 25351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.130:42957, user_credentials={real_user=slave}} blocked reactor thread for 94083.2us
I20260709 13:23:46.480623 25454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d837697cd0f402cacf9db89a5d92652" candidate_uuid: "3e07d64466934da4bb3c02262b1c2e38" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9868dce521294d0b816bba7648f4ae5a"
I20260709 13:23:46.499925 26131 raft_consensus.cc:2804] T 821a5866429f4173820fb64336abee02 P 3e07d64466934da4bb3c02262b1c2e38 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:23:46.459663 26143 raft_consensus.cc:515] T 8ef061028f38450e83324ac78cdaa14c P 9868dce521294d0b816bba7648f4ae5a [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } }
I20260709 13:23:46.501369 26135 raft_consensus.cc:2804] T 1550ed2d2a764c08adffae679599ba41 P 9868dce521294d0b816bba7648f4ae5a [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:23:46.467769 26069 raft_consensus.cc:515] T c902bbc7ec0848ba82a04870eab1e7f5 P 9868dce521294d0b816bba7648f4ae5a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } }
I20260709 13:23:46.504632 25381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9d550d881b04735bbf89751952c09ec" candidate_uuid: "9868dce521294d0b816bba7648f4ae5a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3e07d64466934da4bb3c02262b1c2e38"
I20260709 13:23:46.562041 25302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ef061028f38450e83324ac78cdaa14c" candidate_uuid: "9868dce521294d0b816bba7648f4ae5a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "233e7d29aac04505bf4e65f0a39a0c4c"
I20260709 13:23:46.562745 25381 raft_consensus.cc:3060] T b9d550d881b04735bbf89751952c09ec P 3e07d64466934da4bb3c02262b1c2e38 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:23:46.562798 25302 raft_consensus.cc:3060] T 8ef061028f38450e83324ac78cdaa14c P 233e7d29aac04505bf4e65f0a39a0c4c [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:23:46.567348 25422 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41539, user_credentials={real_user=slave}} blocked reactor thread for 55602.6us
I20260709 13:23:46.572820 25455 raft_consensus.cc:1275] T 467a6b42ca6548269c12dddde696f7af P 9868dce521294d0b816bba7648f4ae5a [term 3 FOLLOWER]: Refusing update from remote peer 3e07d64466934da4bb3c02262b1c2e38: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:23:46.515934 25303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7ff8dde9f5d4c49b9203d902f6b94c4" candidate_uuid: "3e07d64466934da4bb3c02262b1c2e38" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "233e7d29aac04505bf4e65f0a39a0c4c"
W20260709 13:23:46.511780 25350 leader_election.cc:336] T 821a5866429f4173820fb64336abee02 P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 43999d813e564dc688fad116090c95ca (127.14.177.129:38155): Network error: Client connection negotiation failed: client connection to 127.14.177.129:38155: connect: Connection refused (error 111)
W20260709 13:23:46.569065 25276 leader_election.cc:336] T 3a3317f4c3514df1b2bb8e44a86be92f P 233e7d29aac04505bf4e65f0a39a0c4c [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 43999d813e564dc688fad116090c95ca (127.14.177.129:38155): Network error: Client connection negotiation failed: client connection to 127.14.177.129:38155: connect: Connection refused (error 111)
W20260709 13:23:46.577365 25276 leader_election.cc:336] T 579da2d6aabb4ae9a2b1802e1358068b P 233e7d29aac04505bf4e65f0a39a0c4c [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 43999d813e564dc688fad116090c95ca (127.14.177.129:38155): Network error: Client connection negotiation failed: client connection to 127.14.177.129:38155: connect: Connection refused (error 111)
I20260709 13:23:46.577728 25276 leader_election.cc:304] T 579da2d6aabb4ae9a2b1802e1358068b P 233e7d29aac04505bf4e65f0a39a0c4c [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 233e7d29aac04505bf4e65f0a39a0c4c; no voters: 43999d813e564dc688fad116090c95ca, 9868dce521294d0b816bba7648f4ae5a
W20260709 13:23:46.577525 25350 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:38155, user_credentials={real_user=slave}} blocked reactor thread for 65757us
I20260709 13:23:46.502846 25457 raft_consensus.cc:3060] T 0e71b6f5c69b4b35a8ac51c22c7d5f0d P 9868dce521294d0b816bba7648f4ae5a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:23:46.509181 25454 raft_consensus.cc:3060] T 4d837697cd0f402cacf9db89a5d92652 P 9868dce521294d0b816bba7648f4ae5a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:23:46.478854 26140 raft_consensus.cc:515] T 434a54fcee7d4aeeab42f739855c0e8a P 233e7d29aac04505bf4e65f0a39a0c4c [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } }
I20260709 13:23:46.516575 26143 leader_election.cc:290] T 8ef061028f38450e83324ac78cdaa14c P 9868dce521294d0b816bba7648f4ae5a [CANDIDATE]: Term 2 election: Requested vote from peers 43999d813e564dc688fad116090c95ca (127.14.177.129:38155), 233e7d29aac04505bf4e65f0a39a0c4c (127.14.177.130:42957)
I20260709 13:23:46.502717 26150 raft_consensus.cc:697] T 3a3317f4c3514df1b2bb8e44a86be92f P 233e7d29aac04505bf4e65f0a39a0c4c [term 2 LEADER]: Becoming Leader. State: Replica: 233e7d29aac04505bf4e65f0a39a0c4c, State: Running, Role: LEADER
I20260709 13:23:46.508682 26029 raft_consensus.cc:2749] T b155b866f10547fd8239579a81ecf5f9 P 233e7d29aac04505bf4e65f0a39a0c4c [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:23:46.589488 26140 leader_election.cc:290] T 434a54fcee7d4aeeab42f739855c0e8a P 233e7d29aac04505bf4e65f0a39a0c4c [CANDIDATE]: Term 4 election: Requested vote from peers 43999d813e564dc688fad116090c95ca (127.14.177.129:38155), 3e07d64466934da4bb3c02262b1c2e38 (127.14.177.131:41539)
I20260709 13:23:46.590070 25310 raft_consensus.cc:2393] T 434a54fcee7d4aeeab42f739855c0e8a P 233e7d29aac04505bf4e65f0a39a0c4c [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3e07d64466934da4bb3c02262b1c2e38 in current term 4: Already voted for candidate 233e7d29aac04505bf4e65f0a39a0c4c in this term.
I20260709 13:23:46.513734 26131 raft_consensus.cc:493] T 821a5866429f4173820fb64336abee02 P 3e07d64466934da4bb3c02262b1c2e38 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 43999d813e564dc688fad116090c95ca)
I20260709 13:23:46.575469 26135 raft_consensus.cc:697] T 1550ed2d2a764c08adffae679599ba41 P 9868dce521294d0b816bba7648f4ae5a [term 2 LEADER]: Becoming Leader. State: Replica: 9868dce521294d0b816bba7648f4ae5a, State: Running, Role: LEADER
I20260709 13:23:46.506801 26139 raft_consensus.cc:515] T ce9c60e3922c4219b6e4453fe0259b6a P 233e7d29aac04505bf4e65f0a39a0c4c [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } }
I20260709 13:23:46.594872 25134 catalog_manager.cc:5697] T fa0da2c845dc4059968332f4db2e0aa6 P 9868dce521294d0b816bba7648f4ae5a reported cstate change: term changed from 1 to 2, leader changed from 43999d813e564dc688fad116090c95ca (127.14.177.129) to 9868dce521294d0b816bba7648f4ae5a (127.14.177.132). New cstate: current_term: 2 leader_uuid: "9868dce521294d0b816bba7648f4ae5a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } health_report { overall_health: UNKNOWN } } }
I20260709 13:23:46.507732 26046 raft_consensus.cc:515] T 13bb3a23315a41c7a40e10b954b15c8e P 3e07d64466934da4bb3c02262b1c2e38 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } }
I20260709 13:23:46.600206 26041 raft_consensus.cc:515] T 72b56266e8a54776b9113c7a786e1952 P 3e07d64466934da4bb3c02262b1c2e38 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } }
I20260709 13:23:46.604672 26150 consensus_queue.cc:237] T 3a3317f4c3514df1b2bb8e44a86be92f P 233e7d29aac04505bf4e65f0a39a0c4c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } }
I20260709 13:23:46.509814 26136 raft_consensus.cc:515] T d0a0e4d32ce943fbb773bb840863647b P 9868dce521294d0b816bba7648f4ae5a [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } }
I20260709 13:23:46.639279 26118 tablet_bootstrap.cc:492] T f1ac2f4d2fe2424bb470df42199219e2 P 43999d813e564dc688fad116090c95ca: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:23:46.576011 26086 raft_consensus.cc:515] T a5df750b84e0487fa530161079deb6c2 P 9868dce521294d0b816bba7648f4ae5a [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } }
I20260709 13:23:46.581812 26152 consensus_queue.cc:1048] T 467a6b42ca6548269c12dddde696f7af P 3e07d64466934da4bb3c02262b1c2e38 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.020s
I20260709 13:23:46.608592 26139 leader_election.cc:290] T ce9c60e3922c4219b6e4453fe0259b6a P 233e7d29aac04505bf4e65f0a39a0c4c [CANDIDATE]: Term 3 election: Requested vote from peers 9868dce521294d0b816bba7648f4ae5a (127.14.177.132:33683), 43999d813e564dc688fad116090c95ca (127.14.177.129:38155)
I20260709 13:23:46.596138 26131 raft_consensus.cc:3060] T 821a5866429f4173820fb64336abee02 P 3e07d64466934da4bb3c02262b1c2e38 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:23:46.646014 26118 tablet_bootstrap.cc:492] T f1ac2f4d2fe2424bb470df42199219e2 P 43999d813e564dc688fad116090c95ca: Bootstrap complete.
I20260709 13:23:46.646762 26118 ts_tablet_manager.cc:1403] T f1ac2f4d2fe2424bb470df42199219e2 P 43999d813e564dc688fad116090c95ca: Time spent bootstrapping tablet: real 0.201s	user 0.039s	sys 0.042s
I20260709 13:23:46.646543 26135 consensus_queue.cc:237] T 1550ed2d2a764c08adffae679599ba41 P 9868dce521294d0b816bba7648f4ae5a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } }
I20260709 13:23:46.647446 25381 raft_consensus.cc:2468] T b9d550d881b04735bbf89751952c09ec P 3e07d64466934da4bb3c02262b1c2e38 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9868dce521294d0b816bba7648f4ae5a in term 2.
I20260709 13:23:46.652709 26041 leader_election.cc:290] T 72b56266e8a54776b9113c7a786e1952 P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 3 election: Requested vote from peers 43999d813e564dc688fad116090c95ca (127.14.177.129:38155), 233e7d29aac04505bf4e65f0a39a0c4c (127.14.177.130:42957)
I20260709 13:23:46.653667 26118 raft_consensus.cc:359] T f1ac2f4d2fe2424bb470df42199219e2 P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } }
I20260709 13:23:46.654445 26118 raft_consensus.cc:740] T f1ac2f4d2fe2424bb470df42199219e2 P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43999d813e564dc688fad116090c95ca, State: Initialized, Role: FOLLOWER
I20260709 13:23:46.655382 25455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce9c60e3922c4219b6e4453fe0259b6a" candidate_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9868dce521294d0b816bba7648f4ae5a"
I20260709 13:23:46.600361 26134 raft_consensus.cc:515] T f1ac2f4d2fe2424bb470df42199219e2 P 3e07d64466934da4bb3c02262b1c2e38 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } }
I20260709 13:23:46.656044 25455 raft_consensus.cc:3060] T ce9c60e3922c4219b6e4453fe0259b6a P 9868dce521294d0b816bba7648f4ae5a [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:23:46.861019 25379 raft_consensus.cc:2393] T f1ac2f4d2fe2424bb470df42199219e2 P 3e07d64466934da4bb3c02262b1c2e38 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 9868dce521294d0b816bba7648f4ae5a in current term 2: Already voted for candidate 3e07d64466934da4bb3c02262b1c2e38 in this term.
I20260709 13:23:46.862773 26134 leader_election.cc:290] T f1ac2f4d2fe2424bb470df42199219e2 P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 2 election: Requested vote from peers 9868dce521294d0b816bba7648f4ae5a (127.14.177.132:33683), 43999d813e564dc688fad116090c95ca (127.14.177.129:38155)
I20260709 13:23:46.838846 26118 consensus_queue.cc:260] T f1ac2f4d2fe2424bb470df42199219e2 P 43999d813e564dc688fad116090c95ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } }
I20260709 13:23:46.848177 26046 leader_election.cc:290] T 13bb3a23315a41c7a40e10b954b15c8e P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 2 election: Requested vote from peers 43999d813e564dc688fad116090c95ca (127.14.177.129:38155), 233e7d29aac04505bf4e65f0a39a0c4c (127.14.177.130:42957)
I20260709 13:23:46.865986 26046 raft_consensus.cc:493] T b155b866f10547fd8239579a81ecf5f9 P 3e07d64466934da4bb3c02262b1c2e38 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:23:46.866446 26046 raft_consensus.cc:515] T b155b866f10547fd8239579a81ecf5f9 P 3e07d64466934da4bb3c02262b1c2e38 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } }
I20260709 13:23:46.868479 26046 leader_election.cc:290] T b155b866f10547fd8239579a81ecf5f9 P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 43999d813e564dc688fad116090c95ca (127.14.177.129:38155), 233e7d29aac04505bf4e65f0a39a0c4c (127.14.177.130:42957)
I20260709 13:23:46.865350 26118 ts_tablet_manager.cc:1434] T f1ac2f4d2fe2424bb470df42199219e2 P 43999d813e564dc688fad116090c95ca: Time spent starting tablet: real 0.218s	user 0.017s	sys 0.018s
I20260709 13:23:46.872162 26118 tablet_bootstrap.cc:492] T 13bb3a23315a41c7a40e10b954b15c8e P 43999d813e564dc688fad116090c95ca: Bootstrap starting.
I20260709 13:23:46.854959 26137 raft_consensus.cc:515] T b7ff8dde9f5d4c49b9203d902f6b94c4 P 233e7d29aac04505bf4e65f0a39a0c4c [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } }
I20260709 13:23:46.876092 25303 raft_consensus.cc:2393] T b7ff8dde9f5d4c49b9203d902f6b94c4 P 233e7d29aac04505bf4e65f0a39a0c4c [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 3e07d64466934da4bb3c02262b1c2e38 in current term 2: Already voted for candidate 233e7d29aac04505bf4e65f0a39a0c4c in this term.
I20260709 13:23:46.877827 25455 raft_consensus.cc:2468] T ce9c60e3922c4219b6e4453fe0259b6a P 9868dce521294d0b816bba7648f4ae5a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 233e7d29aac04505bf4e65f0a39a0c4c in term 3.
I20260709 13:23:46.843272 15046 ts_tablet_manager.cc:600] Registering tablets (28/45 complete)
I20260709 13:23:46.843621 26136 leader_election.cc:290] T d0a0e4d32ce943fbb773bb840863647b P 9868dce521294d0b816bba7648f4ae5a [CANDIDATE]: Term 2 election: Requested vote from peers 3e07d64466934da4bb3c02262b1c2e38 (127.14.177.131:41539), 43999d813e564dc688fad116090c95ca (127.14.177.129:38155)
I20260709 13:23:46.839973 26143 raft_consensus.cc:493] T 5c7e7acbb09f4d49828fa652ca6abaac P 9868dce521294d0b816bba7648f4ae5a [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:23:46.846632 26069 leader_election.cc:290] T c902bbc7ec0848ba82a04870eab1e7f5 P 9868dce521294d0b816bba7648f4ae5a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3e07d64466934da4bb3c02262b1c2e38 (127.14.177.131:41539), 43999d813e564dc688fad116090c95ca (127.14.177.129:38155)
I20260709 13:23:46.846886 25351 leader_election.cc:304] T 434a54fcee7d4aeeab42f739855c0e8a P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3e07d64466934da4bb3c02262b1c2e38; no voters: 233e7d29aac04505bf4e65f0a39a0c4c, 43999d813e564dc688fad116090c95ca
I20260709 13:23:46.848170 26086 leader_election.cc:290] T a5df750b84e0487fa530161079deb6c2 P 9868dce521294d0b816bba7648f4ae5a [CANDIDATE]: Term 2 election: Requested vote from peers 43999d813e564dc688fad116090c95ca (127.14.177.129:38155), 233e7d29aac04505bf4e65f0a39a0c4c (127.14.177.130:42957)
I20260709 13:23:46.651623 26029 raft_consensus.cc:2749] T 579da2d6aabb4ae9a2b1802e1358068b P 233e7d29aac04505bf4e65f0a39a0c4c [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:23:46.895072 26143 raft_consensus.cc:515] T 5c7e7acbb09f4d49828fa652ca6abaac P 9868dce521294d0b816bba7648f4ae5a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } }
W20260709 13:23:46.895725 25351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:42957, user_credentials={real_user=slave}} blocked reactor thread for 48917.6us
W20260709 13:23:46.850812 25350 consensus_peers.cc:597] T 79d4a12ef58140568c60a24ed4c12f83 P 3e07d64466934da4bb3c02262b1c2e38 -> Peer 43999d813e564dc688fad116090c95ca (127.14.177.129:38155): Couldn't send request to peer 43999d813e564dc688fad116090c95ca. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:38155: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:23:46.896648 25350 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.129:38155, user_credentials={real_user=slave}} blocked reactor thread for 215622us
W20260709 13:23:46.897106 25350 leader_election.cc:336] T 0e71b6f5c69b4b35a8ac51c22c7d5f0d P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 43999d813e564dc688fad116090c95ca (127.14.177.129:38155): Network error: Client connection negotiation failed: client connection to 127.14.177.129:38155: connect: Connection refused (error 111)
W20260709 13:23:46.898025 25350 leader_election.cc:336] T 4d837697cd0f402cacf9db89a5d92652 P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 43999d813e564dc688fad116090c95ca (127.14.177.129:38155): Network error: Client connection negotiation failed: client connection to 127.14.177.129:38155: connect: Connection refused (error 111)
W20260709 13:23:46.898463 25350 leader_election.cc:336] T b7ff8dde9f5d4c49b9203d902f6b94c4 P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 43999d813e564dc688fad116090c95ca (127.14.177.129:38155): Network error: Client connection negotiation failed: client connection to 127.14.177.129:38155: connect: Connection refused (error 111)
I20260709 13:23:46.853672 25450 raft_consensus.cc:2468] T 441d780c3945463f92f31170f31e37a8 P 9868dce521294d0b816bba7648f4ae5a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 233e7d29aac04505bf4e65f0a39a0c4c in term 2.
I20260709 13:23:46.880968 26131 raft_consensus.cc:515] T 821a5866429f4173820fb64336abee02 P 3e07d64466934da4bb3c02262b1c2e38 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } }
W20260709 13:23:46.886384 25276 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:38155, user_credentials={real_user=slave}} blocked reactor thread for 309013us
W20260709 13:23:46.911520 25276 leader_election.cc:336] T ce9c60e3922c4219b6e4453fe0259b6a P 233e7d29aac04505bf4e65f0a39a0c4c [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 43999d813e564dc688fad116090c95ca (127.14.177.129:38155): Network error: Client connection negotiation failed: client connection to 127.14.177.129:38155: connect: Connection refused (error 111)
I20260709 13:23:46.843792 25302 raft_consensus.cc:2468] T 8ef061028f38450e83324ac78cdaa14c P 233e7d29aac04505bf4e65f0a39a0c4c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9868dce521294d0b816bba7648f4ae5a in term 2.
I20260709 13:23:46.876830 26137 leader_election.cc:290] T b7ff8dde9f5d4c49b9203d902f6b94c4 P 233e7d29aac04505bf4e65f0a39a0c4c [CANDIDATE]: Term 2 election: Requested vote from peers 43999d813e564dc688fad116090c95ca (127.14.177.129:38155), 3e07d64466934da4bb3c02262b1c2e38 (127.14.177.131:41539)
I20260709 13:23:46.911656 25422 leader_election.cc:304] T b9d550d881b04735bbf89751952c09ec P 9868dce521294d0b816bba7648f4ae5a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3e07d64466934da4bb3c02262b1c2e38, 9868dce521294d0b816bba7648f4ae5a; no voters: 43999d813e564dc688fad116090c95ca
W20260709 13:23:46.913431 25276 consensus_peers.cc:597] T 28c0b6fc9bd64d60828d705c6a9ab1e4 P 233e7d29aac04505bf4e65f0a39a0c4c -> Peer 43999d813e564dc688fad116090c95ca (127.14.177.129:38155): Couldn't send request to peer 43999d813e564dc688fad116090c95ca. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:38155: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:23:46.913988 25276 leader_election.cc:336] T 434a54fcee7d4aeeab42f739855c0e8a P 233e7d29aac04505bf4e65f0a39a0c4c [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 43999d813e564dc688fad116090c95ca (127.14.177.129:38155): Network error: Client connection negotiation failed: client connection to 127.14.177.129:38155: connect: Connection refused (error 111)
I20260709 13:23:46.915040 25383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "434a54fcee7d4aeeab42f739855c0e8a" candidate_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3e07d64466934da4bb3c02262b1c2e38"
I20260709 13:23:46.927044 25309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5df750b84e0487fa530161079deb6c2" candidate_uuid: "9868dce521294d0b816bba7648f4ae5a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "233e7d29aac04505bf4e65f0a39a0c4c"
I20260709 13:23:46.927783 25309 raft_consensus.cc:3060] T a5df750b84e0487fa530161079deb6c2 P 233e7d29aac04505bf4e65f0a39a0c4c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:23:46.932276 26135 raft_consensus.cc:2804] T b9d550d881b04735bbf89751952c09ec P 9868dce521294d0b816bba7648f4ae5a [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:23:46.933640 25459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1ac2f4d2fe2424bb470df42199219e2" candidate_uuid: "3e07d64466934da4bb3c02262b1c2e38" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9868dce521294d0b816bba7648f4ae5a"
I20260709 13:23:46.934811 25381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c902bbc7ec0848ba82a04870eab1e7f5" candidate_uuid: "9868dce521294d0b816bba7648f4ae5a" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3e07d64466934da4bb3c02262b1c2e38" is_pre_election: true
I20260709 13:23:46.935760 25381 raft_consensus.cc:2468] T c902bbc7ec0848ba82a04870eab1e7f5 P 3e07d64466934da4bb3c02262b1c2e38 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9868dce521294d0b816bba7648f4ae5a in term 2.
I20260709 13:23:46.909848 26152 raft_consensus.cc:3060] T 434a54fcee7d4aeeab42f739855c0e8a P 3e07d64466934da4bb3c02262b1c2e38 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:23:46.942253 25379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0a0e4d32ce943fbb773bb840863647b" candidate_uuid: "9868dce521294d0b816bba7648f4ae5a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3e07d64466934da4bb3c02262b1c2e38"
I20260709 13:23:46.942982 25379 raft_consensus.cc:3060] T d0a0e4d32ce943fbb773bb840863647b P 3e07d64466934da4bb3c02262b1c2e38 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:23:46.943538 25307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72b56266e8a54776b9113c7a786e1952" candidate_uuid: "3e07d64466934da4bb3c02262b1c2e38" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "233e7d29aac04505bf4e65f0a39a0c4c"
I20260709 13:23:46.944341 25307 raft_consensus.cc:3060] T 72b56266e8a54776b9113c7a786e1952 P 233e7d29aac04505bf4e65f0a39a0c4c [term 1 FOLLOWER]: Advancing to term 3
I20260709 13:23:46.946659 25274 leader_election.cc:304] T ce9c60e3922c4219b6e4453fe0259b6a P 233e7d29aac04505bf4e65f0a39a0c4c [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 233e7d29aac04505bf4e65f0a39a0c4c, 9868dce521294d0b816bba7648f4ae5a; no voters: 
I20260709 13:23:46.838950 25457 raft_consensus.cc:2468] T 0e71b6f5c69b4b35a8ac51c22c7d5f0d P 9868dce521294d0b816bba7648f4ae5a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3e07d64466934da4bb3c02262b1c2e38 in term 2.
I20260709 13:23:46.954311 26137 raft_consensus.cc:2804] T ce9c60e3922c4219b6e4453fe0259b6a P 233e7d29aac04505bf4e65f0a39a0c4c [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:23:46.954895 26137 raft_consensus.cc:697] T ce9c60e3922c4219b6e4453fe0259b6a P 233e7d29aac04505bf4e65f0a39a0c4c [term 3 LEADER]: Becoming Leader. State: Replica: 233e7d29aac04505bf4e65f0a39a0c4c, State: Running, Role: LEADER
I20260709 13:23:46.955928 26137 consensus_queue.cc:237] T ce9c60e3922c4219b6e4453fe0259b6a P 233e7d29aac04505bf4e65f0a39a0c4c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } }
I20260709 13:23:46.957284 26143 leader_election.cc:290] T 5c7e7acbb09f4d49828fa652ca6abaac P 9868dce521294d0b816bba7648f4ae5a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3e07d64466934da4bb3c02262b1c2e38 (127.14.177.131:41539), 43999d813e564dc688fad116090c95ca (127.14.177.129:38155)
W20260709 13:23:46.958063 26132 consensus_meta.cc:287] T 5c7e7acbb09f4d49828fa652ca6abaac P 3e07d64466934da4bb3c02262b1c2e38: Time spent flushing consensus metadata: real 0.526s	user 0.019s	sys 0.000s
I20260709 13:23:46.958627 26132 raft_consensus.cc:2749] T 5c7e7acbb09f4d49828fa652ca6abaac P 3e07d64466934da4bb3c02262b1c2e38 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:23:46.955945 25454 raft_consensus.cc:2468] T 4d837697cd0f402cacf9db89a5d92652 P 9868dce521294d0b816bba7648f4ae5a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3e07d64466934da4bb3c02262b1c2e38 in term 2.
I20260709 13:23:46.962929 25302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13bb3a23315a41c7a40e10b954b15c8e" candidate_uuid: "3e07d64466934da4bb3c02262b1c2e38" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "233e7d29aac04505bf4e65f0a39a0c4c"
I20260709 13:23:46.964594 25348 leader_election.cc:304] T 0e71b6f5c69b4b35a8ac51c22c7d5f0d P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3e07d64466934da4bb3c02262b1c2e38, 9868dce521294d0b816bba7648f4ae5a; no voters: 43999d813e564dc688fad116090c95ca
I20260709 13:23:46.966355 25348 leader_election.cc:304] T 4d837697cd0f402cacf9db89a5d92652 P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3e07d64466934da4bb3c02262b1c2e38, 9868dce521294d0b816bba7648f4ae5a; no voters: 43999d813e564dc688fad116090c95ca
I20260709 13:23:46.967092 25274 leader_election.cc:304] T 441d780c3945463f92f31170f31e37a8 P 233e7d29aac04505bf4e65f0a39a0c4c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 233e7d29aac04505bf4e65f0a39a0c4c, 9868dce521294d0b816bba7648f4ae5a; no voters: 
I20260709 13:23:46.967986 25307 raft_consensus.cc:2468] T 72b56266e8a54776b9113c7a786e1952 P 233e7d29aac04505bf4e65f0a39a0c4c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3e07d64466934da4bb3c02262b1c2e38 in term 3.
W20260709 13:23:46.970427 25425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.130:42957, user_credentials={real_user=slave}} blocked reactor thread for 44736.9us
I20260709 13:23:46.974604 25351 leader_election.cc:304] T b7ff8dde9f5d4c49b9203d902f6b94c4 P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3e07d64466934da4bb3c02262b1c2e38; no voters: 233e7d29aac04505bf4e65f0a39a0c4c, 43999d813e564dc688fad116090c95ca
I20260709 13:23:46.971156 25305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b155b866f10547fd8239579a81ecf5f9" candidate_uuid: "3e07d64466934da4bb3c02262b1c2e38" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" is_pre_election: true
I20260709 13:23:46.988322 26058 raft_consensus.cc:2749] T b7ff8dde9f5d4c49b9203d902f6b94c4 P 3e07d64466934da4bb3c02262b1c2e38 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:23:46.934885 25459 raft_consensus.cc:2393] T f1ac2f4d2fe2424bb470df42199219e2 P 9868dce521294d0b816bba7648f4ae5a [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 3e07d64466934da4bb3c02262b1c2e38 in current term 2: Already voted for candidate 9868dce521294d0b816bba7648f4ae5a in this term.
W20260709 13:23:47.009335 25276 leader_election.cc:336] T 441d780c3945463f92f31170f31e37a8 P 233e7d29aac04505bf4e65f0a39a0c4c [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 43999d813e564dc688fad116090c95ca (127.14.177.129:38155): Network error: Client connection negotiation failed: client connection to 127.14.177.129:38155: connect: Connection refused (error 111)
I20260709 13:23:46.989929 25302 raft_consensus.cc:3060] T 13bb3a23315a41c7a40e10b954b15c8e P 233e7d29aac04505bf4e65f0a39a0c4c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:23:47.033773 25309 raft_consensus.cc:2468] T a5df750b84e0487fa530161079deb6c2 P 233e7d29aac04505bf4e65f0a39a0c4c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9868dce521294d0b816bba7648f4ae5a in term 2.
I20260709 13:23:47.006486 26140 raft_consensus.cc:2804] T 441d780c3945463f92f31170f31e37a8 P 233e7d29aac04505bf4e65f0a39a0c4c [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:23:47.005686 26132 raft_consensus.cc:2804] T 0e71b6f5c69b4b35a8ac51c22c7d5f0d P 3e07d64466934da4bb3c02262b1c2e38 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:23:47.041386 25425 leader_election.cc:304] T 8ef061028f38450e83324ac78cdaa14c P 9868dce521294d0b816bba7648f4ae5a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 233e7d29aac04505bf4e65f0a39a0c4c, 9868dce521294d0b816bba7648f4ae5a; no voters: 
I20260709 13:23:47.005349 25305 raft_consensus.cc:2468] T b155b866f10547fd8239579a81ecf5f9 P 233e7d29aac04505bf4e65f0a39a0c4c [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3e07d64466934da4bb3c02262b1c2e38 in term 2.
W20260709 13:23:47.049108 25424 leader_election.cc:336] T 8ef061028f38450e83324ac78cdaa14c P 9868dce521294d0b816bba7648f4ae5a [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 43999d813e564dc688fad116090c95ca (127.14.177.129:38155): Network error: Client connection negotiation failed: client connection to 127.14.177.129:38155: connect: Connection refused (error 111)
I20260709 13:23:47.039517 25384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7ff8dde9f5d4c49b9203d902f6b94c4" candidate_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3e07d64466934da4bb3c02262b1c2e38"
I20260709 13:23:47.057760 25384 raft_consensus.cc:2393] T b7ff8dde9f5d4c49b9203d902f6b94c4 P 3e07d64466934da4bb3c02262b1c2e38 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 233e7d29aac04505bf4e65f0a39a0c4c in current term 2: Already voted for candidate 3e07d64466934da4bb3c02262b1c2e38 in this term.
I20260709 13:23:47.058254 25422 leader_election.cc:304] T f1ac2f4d2fe2424bb470df42199219e2 P 9868dce521294d0b816bba7648f4ae5a [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9868dce521294d0b816bba7648f4ae5a; no voters: 3e07d64466934da4bb3c02262b1c2e38, 43999d813e564dc688fad116090c95ca
I20260709 13:23:47.048421 26140 raft_consensus.cc:697] T 441d780c3945463f92f31170f31e37a8 P 233e7d29aac04505bf4e65f0a39a0c4c [term 2 LEADER]: Becoming Leader. State: Replica: 233e7d29aac04505bf4e65f0a39a0c4c, State: Running, Role: LEADER
I20260709 13:23:47.050060 26132 raft_consensus.cc:697] T 0e71b6f5c69b4b35a8ac51c22c7d5f0d P 3e07d64466934da4bb3c02262b1c2e38 [term 2 LEADER]: Becoming Leader. State: Replica: 3e07d64466934da4bb3c02262b1c2e38, State: Running, Role: LEADER
I20260709 13:23:47.042915 26134 raft_consensus.cc:2804] T 4d837697cd0f402cacf9db89a5d92652 P 3e07d64466934da4bb3c02262b1c2e38 [term 2 FOLLOWER]: Leader election won for term 2
W20260709 13:23:47.053896 25424 consensus_peers.cc:597] T d053b9ae38de4173a0aded4da494bd98 P 9868dce521294d0b816bba7648f4ae5a -> Peer 43999d813e564dc688fad116090c95ca (127.14.177.129:38155): Couldn't send request to peer 43999d813e564dc688fad116090c95ca. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:38155: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:23:47.059526 26140 consensus_queue.cc:237] T 441d780c3945463f92f31170f31e37a8 P 233e7d29aac04505bf4e65f0a39a0c4c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } }
W20260709 13:23:47.065485 25424 consensus_peers.cc:597] T 280d65245f4f4358b54bef93e18aca92 P 9868dce521294d0b816bba7648f4ae5a -> Peer 43999d813e564dc688fad116090c95ca (127.14.177.129:38155): Couldn't send request to peer 43999d813e564dc688fad116090c95ca. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:38155: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:23:47.075760 25350 consensus_peers.cc:597] T e15820fa8a7d4e1fb6ad23a64ca22f90 P 3e07d64466934da4bb3c02262b1c2e38 -> Peer 43999d813e564dc688fad116090c95ca (127.14.177.129:38155): Couldn't send request to peer 43999d813e564dc688fad116090c95ca. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:38155: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:23:47.081274 25379 raft_consensus.cc:2468] T d0a0e4d32ce943fbb773bb840863647b P 3e07d64466934da4bb3c02262b1c2e38 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9868dce521294d0b816bba7648f4ae5a in term 2.
I20260709 13:23:47.082212 26134 raft_consensus.cc:697] T 4d837697cd0f402cacf9db89a5d92652 P 3e07d64466934da4bb3c02262b1c2e38 [term 2 LEADER]: Becoming Leader. State: Replica: 3e07d64466934da4bb3c02262b1c2e38, State: Running, Role: LEADER
I20260709 13:23:47.083230 26134 consensus_queue.cc:237] T 4d837697cd0f402cacf9db89a5d92652 P 3e07d64466934da4bb3c02262b1c2e38 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } }
I20260709 13:23:47.049403 26135 raft_consensus.cc:697] T b9d550d881b04735bbf89751952c09ec P 9868dce521294d0b816bba7648f4ae5a [term 2 LEADER]: Becoming Leader. State: Replica: 9868dce521294d0b816bba7648f4ae5a, State: Running, Role: LEADER
I20260709 13:23:47.093261 25422 leader_election.cc:304] T c902bbc7ec0848ba82a04870eab1e7f5 P 9868dce521294d0b816bba7648f4ae5a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e07d64466934da4bb3c02262b1c2e38, 9868dce521294d0b816bba7648f4ae5a; no voters: 
I20260709 13:23:47.097025 26118 tablet_bootstrap.cc:492] T 13bb3a23315a41c7a40e10b954b15c8e P 43999d813e564dc688fad116090c95ca: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:23:47.097599 26178 raft_consensus.cc:2749] T f1ac2f4d2fe2424bb470df42199219e2 P 9868dce521294d0b816bba7648f4ae5a [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
W20260709 13:23:47.098932 25351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.130:42957, user_credentials={real_user=slave}} blocked reactor thread for 88981.8us
I20260709 13:23:47.103221 25351 leader_election.cc:304] T 72b56266e8a54776b9113c7a786e1952 P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 233e7d29aac04505bf4e65f0a39a0c4c, 3e07d64466934da4bb3c02262b1c2e38; no voters: 
I20260709 13:23:47.087631 26069 raft_consensus.cc:2804] T 8ef061028f38450e83324ac78cdaa14c P 9868dce521294d0b816bba7648f4ae5a [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:23:47.105754 25351 leader_election.cc:304] T b155b866f10547fd8239579a81ecf5f9 P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 233e7d29aac04505bf4e65f0a39a0c4c, 3e07d64466934da4bb3c02262b1c2e38; no voters: 
I20260709 13:23:47.117852 26118 tablet_bootstrap.cc:492] T 13bb3a23315a41c7a40e10b954b15c8e P 43999d813e564dc688fad116090c95ca: Bootstrap complete.
I20260709 13:23:47.118849 26118 ts_tablet_manager.cc:1403] T 13bb3a23315a41c7a40e10b954b15c8e P 43999d813e564dc688fad116090c95ca: Time spent bootstrapping tablet: real 0.247s	user 0.040s	sys 0.030s
I20260709 13:23:47.120885 26179 raft_consensus.cc:2804] T c902bbc7ec0848ba82a04870eab1e7f5 P 9868dce521294d0b816bba7648f4ae5a [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:23:47.121366 26179 raft_consensus.cc:493] T c902bbc7ec0848ba82a04870eab1e7f5 P 9868dce521294d0b816bba7648f4ae5a [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:23:47.121726 26179 raft_consensus.cc:3060] T c902bbc7ec0848ba82a04870eab1e7f5 P 9868dce521294d0b816bba7648f4ae5a [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:23:47.121790 26131 leader_election.cc:290] T 821a5866429f4173820fb64336abee02 P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 2 election: Requested vote from peers 9868dce521294d0b816bba7648f4ae5a (127.14.177.132:33683), 43999d813e564dc688fad116090c95ca (127.14.177.129:38155)
I20260709 13:23:47.121861 26118 raft_consensus.cc:359] T 13bb3a23315a41c7a40e10b954b15c8e P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } }
I20260709 13:23:47.131737 26118 raft_consensus.cc:740] T 13bb3a23315a41c7a40e10b954b15c8e P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43999d813e564dc688fad116090c95ca, State: Initialized, Role: FOLLOWER
I20260709 13:23:47.132846 26118 consensus_queue.cc:260] T 13bb3a23315a41c7a40e10b954b15c8e P 43999d813e564dc688fad116090c95ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } }
I20260709 13:23:47.166709 26118 ts_tablet_manager.cc:1434] T 13bb3a23315a41c7a40e10b954b15c8e P 43999d813e564dc688fad116090c95ca: Time spent starting tablet: real 0.047s	user 0.007s	sys 0.001s
I20260709 13:23:47.112179 26133 raft_consensus.cc:2804] T 72b56266e8a54776b9113c7a786e1952 P 3e07d64466934da4bb3c02262b1c2e38 [term 3 FOLLOWER]: Leader election won for term 3
W20260709 13:23:47.181232 25425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:42957, user_credentials={real_user=slave}} blocked reactor thread for 139963us
I20260709 13:23:47.113313 26180 raft_consensus.cc:2804] T b155b866f10547fd8239579a81ecf5f9 P 3e07d64466934da4bb3c02262b1c2e38 [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 13:23:47.224575 25348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.132:33683, user_credentials={real_user=slave}} blocked reactor thread for 258270us
I20260709 13:23:47.240103 25302 raft_consensus.cc:2468] T 13bb3a23315a41c7a40e10b954b15c8e P 233e7d29aac04505bf4e65f0a39a0c4c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3e07d64466934da4bb3c02262b1c2e38 in term 2.
I20260709 13:23:47.243112 25458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "821a5866429f4173820fb64336abee02" candidate_uuid: "3e07d64466934da4bb3c02262b1c2e38" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9868dce521294d0b816bba7648f4ae5a"
I20260709 13:23:47.243749 25458 raft_consensus.cc:3060] T 821a5866429f4173820fb64336abee02 P 9868dce521294d0b816bba7648f4ae5a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:23:47.243624 25351 leader_election.cc:304] T 13bb3a23315a41c7a40e10b954b15c8e P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 233e7d29aac04505bf4e65f0a39a0c4c, 3e07d64466934da4bb3c02262b1c2e38; no voters: 
I20260709 13:23:47.247604 25425 leader_election.cc:304] T a5df750b84e0487fa530161079deb6c2 P 9868dce521294d0b816bba7648f4ae5a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 233e7d29aac04505bf4e65f0a39a0c4c, 9868dce521294d0b816bba7648f4ae5a; no voters: 
W20260709 13:23:47.257040 25422 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41539, user_credentials={real_user=slave}} blocked reactor thread for 163872us
I20260709 13:23:47.258422 25422 leader_election.cc:304] T d0a0e4d32ce943fbb773bb840863647b P 9868dce521294d0b816bba7648f4ae5a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e07d64466934da4bb3c02262b1c2e38, 9868dce521294d0b816bba7648f4ae5a; no voters: 
I20260709 13:23:47.260380 25458 raft_consensus.cc:2468] T 821a5866429f4173820fb64336abee02 P 9868dce521294d0b816bba7648f4ae5a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3e07d64466934da4bb3c02262b1c2e38 in term 2.
I20260709 13:23:47.245149 25379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c7e7acbb09f4d49828fa652ca6abaac" candidate_uuid: "9868dce521294d0b816bba7648f4ae5a" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3e07d64466934da4bb3c02262b1c2e38" is_pre_election: true
I20260709 13:23:47.238741 26152 raft_consensus.cc:2749] T 434a54fcee7d4aeeab42f739855c0e8a P 3e07d64466934da4bb3c02262b1c2e38 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:23:47.246085 26180 raft_consensus.cc:493] T b155b866f10547fd8239579a81ecf5f9 P 3e07d64466934da4bb3c02262b1c2e38 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:23:47.269649 25424 leader_election.cc:336] T 5c7e7acbb09f4d49828fa652ca6abaac P 9868dce521294d0b816bba7648f4ae5a [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 43999d813e564dc688fad116090c95ca (127.14.177.129:38155): Network error: Client connection negotiation failed: client connection to 127.14.177.129:38155: connect: Connection refused (error 111)
I20260709 13:23:47.249630 26133 raft_consensus.cc:697] T 72b56266e8a54776b9113c7a786e1952 P 3e07d64466934da4bb3c02262b1c2e38 [term 3 LEADER]: Becoming Leader. State: Replica: 3e07d64466934da4bb3c02262b1c2e38, State: Running, Role: LEADER
W20260709 13:23:47.275118 25424 consensus_peers.cc:597] T fa0da2c845dc4059968332f4db2e0aa6 P 9868dce521294d0b816bba7648f4ae5a -> Peer 43999d813e564dc688fad116090c95ca (127.14.177.129:38155): Couldn't send request to peer 43999d813e564dc688fad116090c95ca. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:38155: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:23:47.276791 25310 raft_consensus.cc:1275] T fa0da2c845dc4059968332f4db2e0aa6 P 233e7d29aac04505bf4e65f0a39a0c4c [term 2 FOLLOWER]: Refusing update from remote peer 9868dce521294d0b816bba7648f4ae5a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:23:47.279597 26143 raft_consensus.cc:2804] T d0a0e4d32ce943fbb773bb840863647b P 9868dce521294d0b816bba7648f4ae5a [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:23:47.224336 26135 consensus_queue.cc:237] T b9d550d881b04735bbf89751952c09ec P 9868dce521294d0b816bba7648f4ae5a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } }
I20260709 13:23:47.281287 26133 consensus_queue.cc:237] T 72b56266e8a54776b9113c7a786e1952 P 3e07d64466934da4bb3c02262b1c2e38 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } }
W20260709 13:23:47.282855 25350 leader_election.cc:336] T 72b56266e8a54776b9113c7a786e1952 P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 43999d813e564dc688fad116090c95ca (127.14.177.129:38155): Network error: Client connection negotiation failed: client connection to 127.14.177.129:38155: connect: Connection refused (error 111)
W20260709 13:23:47.283648 25350 consensus_peers.cc:597] T 0cb89d70b4ce4aa8992fb68480bec8a2 P 3e07d64466934da4bb3c02262b1c2e38 -> Peer 43999d813e564dc688fad116090c95ca (127.14.177.129:38155): Couldn't send request to peer 43999d813e564dc688fad116090c95ca. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:38155: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:23:47.117839 26132 consensus_queue.cc:237] T 0e71b6f5c69b4b35a8ac51c22c7d5f0d P 3e07d64466934da4bb3c02262b1c2e38 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } }
W20260709 13:23:47.284147 25350 consensus_peers.cc:597] T 0ed5edbe32044d1c83b0590dbb899801 P 3e07d64466934da4bb3c02262b1c2e38 -> Peer 43999d813e564dc688fad116090c95ca (127.14.177.129:38155): Couldn't send request to peer 43999d813e564dc688fad116090c95ca. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:38155: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:23:47.284592 25350 consensus_peers.cc:597] T 03e9bfbd2f774476a4a377367796a2e8 P 3e07d64466934da4bb3c02262b1c2e38 -> Peer 43999d813e564dc688fad116090c95ca (127.14.177.129:38155): Couldn't send request to peer 43999d813e564dc688fad116090c95ca. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:38155: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:23:47.285022 25350 leader_election.cc:336] T 13bb3a23315a41c7a40e10b954b15c8e P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 43999d813e564dc688fad116090c95ca (127.14.177.129:38155): Network error: Client connection negotiation failed: client connection to 127.14.177.129:38155: connect: Connection refused (error 111)
I20260709 13:23:47.285075 25383 raft_consensus.cc:2468] T 434a54fcee7d4aeeab42f739855c0e8a P 3e07d64466934da4bb3c02262b1c2e38 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 233e7d29aac04505bf4e65f0a39a0c4c in term 4.
W20260709 13:23:47.285614 25350 consensus_peers.cc:597] T beacc815ef954fb692e90544bfa93f05 P 3e07d64466934da4bb3c02262b1c2e38 -> Peer 43999d813e564dc688fad116090c95ca (127.14.177.129:38155): Couldn't send request to peer 43999d813e564dc688fad116090c95ca. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:38155: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:23:47.262984 26182 raft_consensus.cc:2804] T a5df750b84e0487fa530161079deb6c2 P 9868dce521294d0b816bba7648f4ae5a [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:23:47.280133 26143 raft_consensus.cc:697] T d0a0e4d32ce943fbb773bb840863647b P 9868dce521294d0b816bba7648f4ae5a [term 2 LEADER]: Becoming Leader. State: Replica: 9868dce521294d0b816bba7648f4ae5a, State: Running, Role: LEADER
W20260709 13:23:47.286118 25350 leader_election.cc:336] T f1ac2f4d2fe2424bb470df42199219e2 P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 43999d813e564dc688fad116090c95ca (127.14.177.129:38155): Network error: Client connection negotiation failed: client connection to 127.14.177.129:38155: connect: Connection refused (error 111)
W20260709 13:23:47.278244 25424 leader_election.cc:336] T d0a0e4d32ce943fbb773bb840863647b P 9868dce521294d0b816bba7648f4ae5a [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 43999d813e564dc688fad116090c95ca (127.14.177.129:38155): Network error: Client connection negotiation failed: client connection to 127.14.177.129:38155: connect: Connection refused (error 111)
W20260709 13:23:47.296264 25350 leader_election.cc:336] T b155b866f10547fd8239579a81ecf5f9 P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 43999d813e564dc688fad116090c95ca (127.14.177.129:38155): Network error: Client connection negotiation failed: client connection to 127.14.177.129:38155: connect: Connection refused (error 111)
W20260709 13:23:47.297220 25424 consensus_peers.cc:597] T 11949f56648c47e0a43c3140af99a304 P 9868dce521294d0b816bba7648f4ae5a -> Peer 43999d813e564dc688fad116090c95ca (127.14.177.129:38155): Couldn't send request to peer 43999d813e564dc688fad116090c95ca. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:38155: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:23:47.298019 25424 leader_election.cc:336] T c902bbc7ec0848ba82a04870eab1e7f5 P 9868dce521294d0b816bba7648f4ae5a [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 43999d813e564dc688fad116090c95ca (127.14.177.129:38155): Network error: Client connection negotiation failed: client connection to 127.14.177.129:38155: connect: Connection refused (error 111)
W20260709 13:23:47.299036 25424 leader_election.cc:336] T a5df750b84e0487fa530161079deb6c2 P 9868dce521294d0b816bba7648f4ae5a [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 43999d813e564dc688fad116090c95ca (127.14.177.129:38155): Network error: Client connection negotiation failed: client connection to 127.14.177.129:38155: connect: Connection refused (error 111)
I20260709 13:23:47.275507 26180 raft_consensus.cc:3060] T b155b866f10547fd8239579a81ecf5f9 P 3e07d64466934da4bb3c02262b1c2e38 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:23:47.272017 26187 raft_consensus.cc:2804] T 13bb3a23315a41c7a40e10b954b15c8e P 3e07d64466934da4bb3c02262b1c2e38 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:23:47.274802 25379 raft_consensus.cc:2468] T 5c7e7acbb09f4d49828fa652ca6abaac P 3e07d64466934da4bb3c02262b1c2e38 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9868dce521294d0b816bba7648f4ae5a in term 2.
I20260709 13:23:47.316958 26182 raft_consensus.cc:697] T a5df750b84e0487fa530161079deb6c2 P 9868dce521294d0b816bba7648f4ae5a [term 2 LEADER]: Becoming Leader. State: Replica: 9868dce521294d0b816bba7648f4ae5a, State: Running, Role: LEADER
I20260709 13:23:47.340848 26143 consensus_queue.cc:237] T d0a0e4d32ce943fbb773bb840863647b P 9868dce521294d0b816bba7648f4ae5a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } }
I20260709 13:23:47.341348 26187 raft_consensus.cc:697] T 13bb3a23315a41c7a40e10b954b15c8e P 3e07d64466934da4bb3c02262b1c2e38 [term 2 LEADER]: Becoming Leader. State: Replica: 3e07d64466934da4bb3c02262b1c2e38, State: Running, Role: LEADER
I20260709 13:23:47.383715 15046 ts_tablet_manager.cc:616] Registered 45 tablets
I20260709 13:23:47.390800 15046 ts_tablet_manager.cc:595] Time spent register tablets: real 1.548s	user 0.342s	sys 0.014s
W20260709 13:23:47.333364 25351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:42957, user_credentials={real_user=slave}} blocked reactor thread for 90190.3us
I20260709 13:23:47.402076 25348 leader_election.cc:304] T f1ac2f4d2fe2424bb470df42199219e2 P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3e07d64466934da4bb3c02262b1c2e38; no voters: 43999d813e564dc688fad116090c95ca, 9868dce521294d0b816bba7648f4ae5a
I20260709 13:23:47.298594 26069 raft_consensus.cc:697] T 8ef061028f38450e83324ac78cdaa14c P 9868dce521294d0b816bba7648f4ae5a [term 2 LEADER]: Becoming Leader. State: Replica: 9868dce521294d0b816bba7648f4ae5a, State: Running, Role: LEADER
I20260709 13:23:47.410488 26041 raft_consensus.cc:2749] T f1ac2f4d2fe2424bb470df42199219e2 P 3e07d64466934da4bb3c02262b1c2e38 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:23:47.417477 26179 raft_consensus.cc:515] T c902bbc7ec0848ba82a04870eab1e7f5 P 9868dce521294d0b816bba7648f4ae5a [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } }
I20260709 13:23:47.421057 26187 consensus_queue.cc:237] T 13bb3a23315a41c7a40e10b954b15c8e P 3e07d64466934da4bb3c02262b1c2e38 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } }
I20260709 13:23:47.401541 26182 consensus_queue.cc:237] T a5df750b84e0487fa530161079deb6c2 P 9868dce521294d0b816bba7648f4ae5a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } }
I20260709 13:23:47.455674 26127 raft_consensus.cc:493] T e15820fa8a7d4e1fb6ad23a64ca22f90 P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:23:47.456488 26127 raft_consensus.cc:515] T e15820fa8a7d4e1fb6ad23a64ca22f90 P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } }
W20260709 13:23:47.476559 25350 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.129:38155, user_credentials={real_user=slave}} blocked reactor thread for 178809us
W20260709 13:23:47.477419 25350 consensus_peers.cc:597] T b2f14aed05e24a98ad83535029b795dc P 3e07d64466934da4bb3c02262b1c2e38 -> Peer 43999d813e564dc688fad116090c95ca (127.14.177.129:38155): Couldn't send request to peer 43999d813e564dc688fad116090c95ca. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:38155: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:23:47.497457 26127 leader_election.cc:290] T e15820fa8a7d4e1fb6ad23a64ca22f90 P 43999d813e564dc688fad116090c95ca [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 233e7d29aac04505bf4e65f0a39a0c4c (127.14.177.130:42957), 3e07d64466934da4bb3c02262b1c2e38 (127.14.177.131:41539)
W20260709 13:23:47.516485 25350 consensus_peers.cc:597] T 87085ac2575e4b6a86dd1fc21e6f718b P 3e07d64466934da4bb3c02262b1c2e38 -> Peer 43999d813e564dc688fad116090c95ca (127.14.177.129:38155): Couldn't send request to peer 43999d813e564dc688fad116090c95ca. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:38155: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:23:47.518124 25350 leader_election.cc:336] T 821a5866429f4173820fb64336abee02 P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 43999d813e564dc688fad116090c95ca (127.14.177.129:38155): Network error: Client connection negotiation failed: client connection to 127.14.177.129:38155: connect: Connection refused (error 111)
W20260709 13:23:47.558465 25351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.130:42957, user_credentials={real_user=slave}} blocked reactor thread for 145096us
W20260709 13:23:47.518904 25350 consensus_peers.cc:597] T b7a891bfc89e4e259f53da6cb16f4989 P 3e07d64466934da4bb3c02262b1c2e38 -> Peer 43999d813e564dc688fad116090c95ca (127.14.177.129:38155): Couldn't send request to peer 43999d813e564dc688fad116090c95ca. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:38155: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:23:47.571775 25350 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.129:38155, user_credentials={real_user=slave}} blocked reactor thread for 52924.8us
W20260709 13:23:47.666793 25348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.132:33683, user_credentials={real_user=slave}} blocked reactor thread for 258602us
I20260709 13:23:47.688607 26127 raft_consensus.cc:493] T 28c0b6fc9bd64d60828d705c6a9ab1e4 P 43999d813e564dc688fad116090c95ca [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:23:47.689448 26127 raft_consensus.cc:515] T 28c0b6fc9bd64d60828d705c6a9ab1e4 P 43999d813e564dc688fad116090c95ca [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } }
I20260709 13:23:47.691920 26179 leader_election.cc:290] T c902bbc7ec0848ba82a04870eab1e7f5 P 9868dce521294d0b816bba7648f4ae5a [CANDIDATE]: Term 3 election: Requested vote from peers 3e07d64466934da4bb3c02262b1c2e38 (127.14.177.131:41539), 43999d813e564dc688fad116090c95ca (127.14.177.129:38155)
I20260709 13:23:47.649941 26180 raft_consensus.cc:515] T b155b866f10547fd8239579a81ecf5f9 P 3e07d64466934da4bb3c02262b1c2e38 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } }
I20260709 13:23:47.697652 26180 leader_election.cc:290] T b155b866f10547fd8239579a81ecf5f9 P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 3 election: Requested vote from peers 43999d813e564dc688fad116090c95ca (127.14.177.129:38155), 233e7d29aac04505bf4e65f0a39a0c4c (127.14.177.130:42957)
I20260709 13:23:47.675628 26069 consensus_queue.cc:237] T 8ef061028f38450e83324ac78cdaa14c P 9868dce521294d0b816bba7648f4ae5a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } }
I20260709 13:23:47.705725 26118 tablet_bootstrap.cc:492] T 0cb89d70b4ce4aa8992fb68480bec8a2 P 43999d813e564dc688fad116090c95ca: Bootstrap starting.
I20260709 13:23:47.709477 26197 raft_consensus.cc:493] T 9ab90e3e08fb4528b2a47d995e9e26d3 P 3e07d64466934da4bb3c02262b1c2e38 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9868dce521294d0b816bba7648f4ae5a)
I20260709 13:23:47.710242 26197 raft_consensus.cc:515] T 9ab90e3e08fb4528b2a47d995e9e26d3 P 3e07d64466934da4bb3c02262b1c2e38 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } }
I20260709 13:23:47.714670 25303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b155b866f10547fd8239579a81ecf5f9" candidate_uuid: "3e07d64466934da4bb3c02262b1c2e38" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "233e7d29aac04505bf4e65f0a39a0c4c"
I20260709 13:23:47.713431 26168 raft_consensus.cc:493] T 9ab90e3e08fb4528b2a47d995e9e26d3 P 233e7d29aac04505bf4e65f0a39a0c4c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9868dce521294d0b816bba7648f4ae5a)
I20260709 13:23:47.714987 26197 leader_election.cc:290] T 9ab90e3e08fb4528b2a47d995e9e26d3 P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 233e7d29aac04505bf4e65f0a39a0c4c (127.14.177.130:42957), 9868dce521294d0b816bba7648f4ae5a (127.14.177.132:33683)
I20260709 13:23:47.715406 25303 raft_consensus.cc:3060] T b155b866f10547fd8239579a81ecf5f9 P 233e7d29aac04505bf4e65f0a39a0c4c [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:23:47.715102 26127 leader_election.cc:290] T 28c0b6fc9bd64d60828d705c6a9ab1e4 P 43999d813e564dc688fad116090c95ca [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 233e7d29aac04505bf4e65f0a39a0c4c (127.14.177.130:42957), 3e07d64466934da4bb3c02262b1c2e38 (127.14.177.131:41539)
I20260709 13:23:47.718104 25305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ab90e3e08fb4528b2a47d995e9e26d3" candidate_uuid: "3e07d64466934da4bb3c02262b1c2e38" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" is_pre_election: true
I20260709 13:23:47.715709 26168 raft_consensus.cc:515] T 9ab90e3e08fb4528b2a47d995e9e26d3 P 233e7d29aac04505bf4e65f0a39a0c4c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } }
W20260709 13:23:47.737347 25348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.132:33683, user_credentials={real_user=slave}} blocked reactor thread for 69836.8us
I20260709 13:23:47.738041 25348 leader_election.cc:304] T 821a5866429f4173820fb64336abee02 P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3e07d64466934da4bb3c02262b1c2e38, 9868dce521294d0b816bba7648f4ae5a; no voters: 43999d813e564dc688fad116090c95ca
I20260709 13:23:47.731663 25305 raft_consensus.cc:2468] T 9ab90e3e08fb4528b2a47d995e9e26d3 P 233e7d29aac04505bf4e65f0a39a0c4c [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3e07d64466934da4bb3c02262b1c2e38 in term 1.
I20260709 13:23:47.768059 25459 raft_consensus.cc:1275] T 3a3317f4c3514df1b2bb8e44a86be92f P 9868dce521294d0b816bba7648f4ae5a [term 2 FOLLOWER]: Refusing update from remote peer 233e7d29aac04505bf4e65f0a39a0c4c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:23:47.749554 26212 raft_consensus.cc:2804] T 821a5866429f4173820fb64336abee02 P 3e07d64466934da4bb3c02262b1c2e38 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:23:47.774118 25384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c902bbc7ec0848ba82a04870eab1e7f5" candidate_uuid: "9868dce521294d0b816bba7648f4ae5a" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3e07d64466934da4bb3c02262b1c2e38"
I20260709 13:23:47.770155 25379 raft_consensus.cc:1275] T 1550ed2d2a764c08adffae679599ba41 P 3e07d64466934da4bb3c02262b1c2e38 [term 2 FOLLOWER]: Refusing update from remote peer 9868dce521294d0b816bba7648f4ae5a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:23:47.774961 26212 raft_consensus.cc:697] T 821a5866429f4173820fb64336abee02 P 3e07d64466934da4bb3c02262b1c2e38 [term 2 LEADER]: Becoming Leader. State: Replica: 3e07d64466934da4bb3c02262b1c2e38, State: Running, Role: LEADER
I20260709 13:23:47.774982 25384 raft_consensus.cc:3060] T c902bbc7ec0848ba82a04870eab1e7f5 P 3e07d64466934da4bb3c02262b1c2e38 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:23:47.776135 26212 consensus_queue.cc:237] T 821a5866429f4173820fb64336abee02 P 3e07d64466934da4bb3c02262b1c2e38 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } }
I20260709 13:23:47.773712 25422 leader_election.cc:304] T 5c7e7acbb09f4d49828fa652ca6abaac P 9868dce521294d0b816bba7648f4ae5a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3e07d64466934da4bb3c02262b1c2e38, 9868dce521294d0b816bba7648f4ae5a; no voters: 43999d813e564dc688fad116090c95ca
I20260709 13:23:47.778827 25454 raft_consensus.cc:1275] T 4d837697cd0f402cacf9db89a5d92652 P 9868dce521294d0b816bba7648f4ae5a [term 2 FOLLOWER]: Refusing update from remote peer 3e07d64466934da4bb3c02262b1c2e38: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:23:47.789081 26182 raft_consensus.cc:2804] T 5c7e7acbb09f4d49828fa652ca6abaac P 9868dce521294d0b816bba7648f4ae5a [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:23:47.796270 26182 raft_consensus.cc:493] T 5c7e7acbb09f4d49828fa652ca6abaac P 9868dce521294d0b816bba7648f4ae5a [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:23:47.799264 26168 leader_election.cc:290] T 9ab90e3e08fb4528b2a47d995e9e26d3 P 233e7d29aac04505bf4e65f0a39a0c4c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3e07d64466934da4bb3c02262b1c2e38 (127.14.177.131:41539), 9868dce521294d0b816bba7648f4ae5a (127.14.177.132:33683)
I20260709 13:23:47.809365 26133 consensus_queue.cc:1048] T 4d837697cd0f402cacf9db89a5d92652 P 3e07d64466934da4bb3c02262b1c2e38 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:23:47.812301 26180 raft_consensus.cc:493] T aad1f7937f854cc1a726223269da5d5a P 3e07d64466934da4bb3c02262b1c2e38 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9868dce521294d0b816bba7648f4ae5a)
I20260709 13:23:47.812973 26180 raft_consensus.cc:515] T aad1f7937f854cc1a726223269da5d5a P 3e07d64466934da4bb3c02262b1c2e38 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } }
I20260709 13:23:47.816260 26180 leader_election.cc:290] T aad1f7937f854cc1a726223269da5d5a P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 233e7d29aac04505bf4e65f0a39a0c4c (127.14.177.130:42957), 9868dce521294d0b816bba7648f4ae5a (127.14.177.132:33683)
I20260709 13:23:47.801083 26182 raft_consensus.cc:3060] T 5c7e7acbb09f4d49828fa652ca6abaac P 9868dce521294d0b816bba7648f4ae5a [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:23:47.818502 25350 consensus_peers.cc:597] T 91ac744179de48e5bb924b6dfd9f9c9b P 3e07d64466934da4bb3c02262b1c2e38 -> Peer 43999d813e564dc688fad116090c95ca (127.14.177.129:38155): Couldn't send request to peer 43999d813e564dc688fad116090c95ca. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:38155: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:23:47.819247 25350 consensus_peers.cc:597] T 4d837697cd0f402cacf9db89a5d92652 P 3e07d64466934da4bb3c02262b1c2e38 -> Peer 43999d813e564dc688fad116090c95ca (127.14.177.129:38155): Couldn't send request to peer 43999d813e564dc688fad116090c95ca. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:38155: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:23:47.819823 25350 leader_election.cc:336] T b155b866f10547fd8239579a81ecf5f9 P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 43999d813e564dc688fad116090c95ca (127.14.177.129:38155): Network error: Client connection negotiation failed: client connection to 127.14.177.129:38155: connect: Connection refused (error 111)
I20260709 13:23:47.826980 25302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aad1f7937f854cc1a726223269da5d5a" candidate_uuid: "3e07d64466934da4bb3c02262b1c2e38" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" is_pre_election: true
I20260709 13:23:47.827648 25450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aad1f7937f854cc1a726223269da5d5a" candidate_uuid: "3e07d64466934da4bb3c02262b1c2e38" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9868dce521294d0b816bba7648f4ae5a" is_pre_election: true
I20260709 13:23:47.794878 26145 consensus_queue.cc:1048] T 1550ed2d2a764c08adffae679599ba41 P 9868dce521294d0b816bba7648f4ae5a [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.034s
I20260709 13:23:47.831482 25379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28c0b6fc9bd64d60828d705c6a9ab1e4" candidate_uuid: "43999d813e564dc688fad116090c95ca" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3e07d64466934da4bb3c02262b1c2e38" is_pre_election: true
I20260709 13:23:47.832891 25348 leader_election.cc:304] T aad1f7937f854cc1a726223269da5d5a P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3e07d64466934da4bb3c02262b1c2e38; no voters: 233e7d29aac04505bf4e65f0a39a0c4c, 9868dce521294d0b816bba7648f4ae5a
I20260709 13:23:47.833220 25382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15820fa8a7d4e1fb6ad23a64ca22f90" candidate_uuid: "43999d813e564dc688fad116090c95ca" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3e07d64466934da4bb3c02262b1c2e38" is_pre_election: true
I20260709 13:23:47.834290 26190 raft_consensus.cc:2749] T aad1f7937f854cc1a726223269da5d5a P 3e07d64466934da4bb3c02262b1c2e38 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:23:47.835546 25351 leader_election.cc:304] T 9ab90e3e08fb4528b2a47d995e9e26d3 P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 233e7d29aac04505bf4e65f0a39a0c4c, 3e07d64466934da4bb3c02262b1c2e38; no voters: 
I20260709 13:23:47.836042 25451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ab90e3e08fb4528b2a47d995e9e26d3" candidate_uuid: "3e07d64466934da4bb3c02262b1c2e38" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9868dce521294d0b816bba7648f4ae5a" is_pre_election: true
I20260709 13:23:47.836449 25302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15820fa8a7d4e1fb6ad23a64ca22f90" candidate_uuid: "43999d813e564dc688fad116090c95ca" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" is_pre_election: true
I20260709 13:23:47.837280 25307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28c0b6fc9bd64d60828d705c6a9ab1e4" candidate_uuid: "43999d813e564dc688fad116090c95ca" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" is_pre_election: true
I20260709 13:23:47.836517 26199 raft_consensus.cc:2804] T 9ab90e3e08fb4528b2a47d995e9e26d3 P 3e07d64466934da4bb3c02262b1c2e38 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:23:47.839524 26199 raft_consensus.cc:493] T 9ab90e3e08fb4528b2a47d995e9e26d3 P 3e07d64466934da4bb3c02262b1c2e38 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 9868dce521294d0b816bba7648f4ae5a)
I20260709 13:23:47.839970 26199 raft_consensus.cc:3060] T 9ab90e3e08fb4528b2a47d995e9e26d3 P 3e07d64466934da4bb3c02262b1c2e38 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:23:47.858558 25385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ab90e3e08fb4528b2a47d995e9e26d3" candidate_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3e07d64466934da4bb3c02262b1c2e38" is_pre_election: true
W20260709 13:23:47.862129 25276 consensus_peers.cc:597] T ce98a3f3b3eb4c63bcaa355ec7cd64a4 P 233e7d29aac04505bf4e65f0a39a0c4c -> Peer 43999d813e564dc688fad116090c95ca (127.14.177.129:38155): Couldn't send request to peer 43999d813e564dc688fad116090c95ca. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:38155: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:23:47.862618 25452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ab90e3e08fb4528b2a47d995e9e26d3" candidate_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9868dce521294d0b816bba7648f4ae5a" is_pre_election: true
I20260709 13:23:47.863363 25456 raft_consensus.cc:1275] T ce9c60e3922c4219b6e4453fe0259b6a P 9868dce521294d0b816bba7648f4ae5a [term 3 FOLLOWER]: Refusing update from remote peer 233e7d29aac04505bf4e65f0a39a0c4c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:23:47.863948 25384 raft_consensus.cc:2468] T c902bbc7ec0848ba82a04870eab1e7f5 P 3e07d64466934da4bb3c02262b1c2e38 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9868dce521294d0b816bba7648f4ae5a in term 3.
I20260709 13:23:47.869693 25303 raft_consensus.cc:2468] T b155b866f10547fd8239579a81ecf5f9 P 233e7d29aac04505bf4e65f0a39a0c4c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3e07d64466934da4bb3c02262b1c2e38 in term 3.
W20260709 13:23:47.862653 25276 leader_election.cc:336] T 434a54fcee7d4aeeab42f739855c0e8a P 233e7d29aac04505bf4e65f0a39a0c4c [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer 43999d813e564dc688fad116090c95ca (127.14.177.129:38155): Network error: Client connection negotiation failed: client connection to 127.14.177.129:38155: connect: Connection refused (error 111)
I20260709 13:23:47.872628 25351 leader_election.cc:304] T b155b866f10547fd8239579a81ecf5f9 P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 233e7d29aac04505bf4e65f0a39a0c4c, 3e07d64466934da4bb3c02262b1c2e38; no voters: 43999d813e564dc688fad116090c95ca
W20260709 13:23:47.872121 25276 leader_election.cc:336] T ce9c60e3922c4219b6e4453fe0259b6a P 233e7d29aac04505bf4e65f0a39a0c4c [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 43999d813e564dc688fad116090c95ca (127.14.177.129:38155): Network error: Client connection negotiation failed: client connection to 127.14.177.129:38155: connect: Connection refused (error 111)
I20260709 13:23:47.882203 25422 leader_election.cc:304] T c902bbc7ec0848ba82a04870eab1e7f5 P 9868dce521294d0b816bba7648f4ae5a [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e07d64466934da4bb3c02262b1c2e38, 9868dce521294d0b816bba7648f4ae5a; no voters: 
I20260709 13:23:47.872951 25130 catalog_manager.cc:5697] T 3a3317f4c3514df1b2bb8e44a86be92f P 233e7d29aac04505bf4e65f0a39a0c4c reported cstate change: term changed from 1 to 2, leader changed from 43999d813e564dc688fad116090c95ca (127.14.177.129) to 233e7d29aac04505bf4e65f0a39a0c4c (127.14.177.130). New cstate: current_term: 2 leader_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } health_report { overall_health: HEALTHY } } }
I20260709 13:23:47.888664 26136 raft_consensus.cc:2804] T c902bbc7ec0848ba82a04870eab1e7f5 P 9868dce521294d0b816bba7648f4ae5a [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:23:47.896536 26136 raft_consensus.cc:697] T c902bbc7ec0848ba82a04870eab1e7f5 P 9868dce521294d0b816bba7648f4ae5a [term 3 LEADER]: Becoming Leader. State: Replica: 9868dce521294d0b816bba7648f4ae5a, State: Running, Role: LEADER
I20260709 13:23:47.897853 26136 consensus_queue.cc:237] T c902bbc7ec0848ba82a04870eab1e7f5 P 9868dce521294d0b816bba7648f4ae5a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } }
I20260709 13:23:47.888901 26158 raft_consensus.cc:2804] T b155b866f10547fd8239579a81ecf5f9 P 3e07d64466934da4bb3c02262b1c2e38 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:23:47.923408 26158 raft_consensus.cc:697] T b155b866f10547fd8239579a81ecf5f9 P 3e07d64466934da4bb3c02262b1c2e38 [term 3 LEADER]: Becoming Leader. State: Replica: 3e07d64466934da4bb3c02262b1c2e38, State: Running, Role: LEADER
I20260709 13:23:47.923203 26145 consensus_queue.cc:1048] T fa0da2c845dc4059968332f4db2e0aa6 P 9868dce521294d0b816bba7648f4ae5a [LEADER]: Connected to new peer: Peer: permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.004s
I20260709 13:23:47.926229 26158 consensus_queue.cc:237] T b155b866f10547fd8239579a81ecf5f9 P 3e07d64466934da4bb3c02262b1c2e38 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } }
I20260709 13:23:47.920312 25453 raft_consensus.cc:1275] T 441d780c3945463f92f31170f31e37a8 P 9868dce521294d0b816bba7648f4ae5a [term 2 FOLLOWER]: Refusing update from remote peer 233e7d29aac04505bf4e65f0a39a0c4c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 13:23:47.956859 25424 consensus_peers.cc:597] T 1f15ec62698c4a8cbd84f2b3d516d2ea P 9868dce521294d0b816bba7648f4ae5a -> Peer 43999d813e564dc688fad116090c95ca (127.14.177.129:38155): Couldn't send request to peer 43999d813e564dc688fad116090c95ca. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:38155: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:23:47.957247 25385 raft_consensus.cc:1275] T d0a0e4d32ce943fbb773bb840863647b P 3e07d64466934da4bb3c02262b1c2e38 [term 2 FOLLOWER]: Refusing update from remote peer 9868dce521294d0b816bba7648f4ae5a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 13:23:47.957508 25424 consensus_peers.cc:597] T 1550ed2d2a764c08adffae679599ba41 P 9868dce521294d0b816bba7648f4ae5a -> Peer 43999d813e564dc688fad116090c95ca (127.14.177.129:38155): Couldn't send request to peer 43999d813e564dc688fad116090c95ca. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:38155: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:23:47.958032 25424 leader_election.cc:336] T c902bbc7ec0848ba82a04870eab1e7f5 P 9868dce521294d0b816bba7648f4ae5a [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 43999d813e564dc688fad116090c95ca (127.14.177.129:38155): Network error: Client connection negotiation failed: client connection to 127.14.177.129:38155: connect: Connection refused (error 111)
I20260709 13:23:47.902523 26111 leader_election.cc:304] T 28c0b6fc9bd64d60828d705c6a9ab1e4 P 43999d813e564dc688fad116090c95ca [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 43999d813e564dc688fad116090c95ca; no voters: 233e7d29aac04505bf4e65f0a39a0c4c, 3e07d64466934da4bb3c02262b1c2e38
W20260709 13:23:47.972128 25276 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.129:38155, user_credentials={real_user=slave}} blocked reactor thread for 95836.4us
W20260709 13:23:47.973037 25276 consensus_peers.cc:597] T 51a11db644124b63bdf5f457c967473d P 233e7d29aac04505bf4e65f0a39a0c4c -> Peer 43999d813e564dc688fad116090c95ca (127.14.177.129:38155): Couldn't send request to peer 43999d813e564dc688fad116090c95ca. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:38155: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:23:47.973656 25276 leader_election.cc:336] T b7ff8dde9f5d4c49b9203d902f6b94c4 P 233e7d29aac04505bf4e65f0a39a0c4c [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 43999d813e564dc688fad116090c95ca (127.14.177.129:38155): Network error: Client connection negotiation failed: client connection to 127.14.177.129:38155: connect: Connection refused (error 111)
I20260709 13:23:47.974186 25276 leader_election.cc:304] T b7ff8dde9f5d4c49b9203d902f6b94c4 P 233e7d29aac04505bf4e65f0a39a0c4c [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 233e7d29aac04505bf4e65f0a39a0c4c; no voters: 3e07d64466934da4bb3c02262b1c2e38, 43999d813e564dc688fad116090c95ca
I20260709 13:23:47.987742 26127 raft_consensus.cc:2749] T 28c0b6fc9bd64d60828d705c6a9ab1e4 P 43999d813e564dc688fad116090c95ca [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:23:47.991941 26199 raft_consensus.cc:515] T 9ab90e3e08fb4528b2a47d995e9e26d3 P 3e07d64466934da4bb3c02262b1c2e38 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } }
I20260709 13:23:47.985122 26228 raft_consensus.cc:493] T 11949f56648c47e0a43c3140af99a304 P 233e7d29aac04505bf4e65f0a39a0c4c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9868dce521294d0b816bba7648f4ae5a)
I20260709 13:23:47.992028 26229 consensus_queue.cc:1048] T 3a3317f4c3514df1b2bb8e44a86be92f P 233e7d29aac04505bf4e65f0a39a0c4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:23:47.993161 26228 raft_consensus.cc:515] T 11949f56648c47e0a43c3140af99a304 P 233e7d29aac04505bf4e65f0a39a0c4c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } }
I20260709 13:23:47.994228 26229 raft_consensus.cc:2749] T b7ff8dde9f5d4c49b9203d902f6b94c4 P 233e7d29aac04505bf4e65f0a39a0c4c [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:23:47.994457 26199 leader_election.cc:290] T 9ab90e3e08fb4528b2a47d995e9e26d3 P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 2 election: Requested vote from peers 233e7d29aac04505bf4e65f0a39a0c4c (127.14.177.130:42957), 9868dce521294d0b816bba7648f4ae5a (127.14.177.132:33683)
I20260709 13:23:47.995002 26229 consensus_queue.cc:1048] T ce9c60e3922c4219b6e4453fe0259b6a P 233e7d29aac04505bf4e65f0a39a0c4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:23:47.995115 25308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ab90e3e08fb4528b2a47d995e9e26d3" candidate_uuid: "3e07d64466934da4bb3c02262b1c2e38" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "233e7d29aac04505bf4e65f0a39a0c4c"
I20260709 13:23:47.996049 25308 raft_consensus.cc:3060] T 9ab90e3e08fb4528b2a47d995e9e26d3 P 233e7d29aac04505bf4e65f0a39a0c4c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:23:47.977115 26138 consensus_queue.cc:1048] T d0a0e4d32ce943fbb773bb840863647b P 9868dce521294d0b816bba7648f4ae5a [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.012s
I20260709 13:23:48.003371 25450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ab90e3e08fb4528b2a47d995e9e26d3" candidate_uuid: "3e07d64466934da4bb3c02262b1c2e38" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9868dce521294d0b816bba7648f4ae5a"
W20260709 13:23:48.002061 26111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41539, user_credentials={real_user=slave}} blocked reactor thread for 114712us
I20260709 13:23:48.004940 26111 leader_election.cc:304] T e15820fa8a7d4e1fb6ad23a64ca22f90 P 43999d813e564dc688fad116090c95ca [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 43999d813e564dc688fad116090c95ca; no voters: 233e7d29aac04505bf4e65f0a39a0c4c, 3e07d64466934da4bb3c02262b1c2e38
I20260709 13:23:48.005072 26182 raft_consensus.cc:515] T 5c7e7acbb09f4d49828fa652ca6abaac P 9868dce521294d0b816bba7648f4ae5a [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } }
I20260709 13:23:48.005951 26127 raft_consensus.cc:2749] T e15820fa8a7d4e1fb6ad23a64ca22f90 P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:23:48.007032 26118 tablet_bootstrap.cc:492] T 0cb89d70b4ce4aa8992fb68480bec8a2 P 43999d813e564dc688fad116090c95ca: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:23:48.011968 26127 raft_consensus.cc:493] T 441d780c3945463f92f31170f31e37a8 P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:23:48.012650 25350 consensus_peers.cc:597] T 9ffa42c3b2ff48bc87310e9ab5d218ce P 3e07d64466934da4bb3c02262b1c2e38 -> Peer 43999d813e564dc688fad116090c95ca (127.14.177.129:38155): Couldn't send request to peer 43999d813e564dc688fad116090c95ca. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:38155: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:23:48.014051 26127 raft_consensus.cc:515] T 441d780c3945463f92f31170f31e37a8 P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } }
I20260709 13:23:48.019111 26127 leader_election.cc:290] T 441d780c3945463f92f31170f31e37a8 P 43999d813e564dc688fad116090c95ca [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9868dce521294d0b816bba7648f4ae5a (127.14.177.132:33683), 233e7d29aac04505bf4e65f0a39a0c4c (127.14.177.130:42957)
W20260709 13:23:48.019757 25424 consensus_peers.cc:597] T e9e342ecda4b46619ce12452b2cf7154 P 9868dce521294d0b816bba7648f4ae5a -> Peer 43999d813e564dc688fad116090c95ca (127.14.177.129:38155): Couldn't send request to peer 43999d813e564dc688fad116090c95ca. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:38155: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:23:48.020229 25424 consensus_peers.cc:597] T b9d550d881b04735bbf89751952c09ec P 9868dce521294d0b816bba7648f4ae5a -> Peer 43999d813e564dc688fad116090c95ca (127.14.177.129:38155): Couldn't send request to peer 43999d813e564dc688fad116090c95ca. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:38155: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:23:48.024260 26118 tablet_bootstrap.cc:492] T 0cb89d70b4ce4aa8992fb68480bec8a2 P 43999d813e564dc688fad116090c95ca: Bootstrap complete.
I20260709 13:23:48.025326 26118 ts_tablet_manager.cc:1403] T 0cb89d70b4ce4aa8992fb68480bec8a2 P 43999d813e564dc688fad116090c95ca: Time spent bootstrapping tablet: real 0.320s	user 0.021s	sys 0.012s
I20260709 13:23:48.030066 26118 raft_consensus.cc:359] T 0cb89d70b4ce4aa8992fb68480bec8a2 P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } }
I20260709 13:23:48.030994 26118 raft_consensus.cc:740] T 0cb89d70b4ce4aa8992fb68480bec8a2 P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43999d813e564dc688fad116090c95ca, State: Initialized, Role: FOLLOWER
I20260709 13:23:48.035943 26118 consensus_queue.cc:260] T 0cb89d70b4ce4aa8992fb68480bec8a2 P 43999d813e564dc688fad116090c95ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } }
I20260709 13:23:48.043496 26118 ts_tablet_manager.cc:1434] T 0cb89d70b4ce4aa8992fb68480bec8a2 P 43999d813e564dc688fad116090c95ca: Time spent starting tablet: real 0.018s	user 0.010s	sys 0.000s
I20260709 13:23:48.047240 25383 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54712 (request call id 240) took 1132 ms. Trace:
I20260709 13:23:48.047731 25383 rpcz_store.cc:276] 0709 13:23:46.914646 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:23:46.914770 (+   124us) service_pool.cc:224] Handling call
0709 13:23:48.047045 (+1132275us) spinlock_profiling.cc:243] Waited 360 ms on lock 0x7b5400042754. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac8571 00007f69b0b632fa 00007f69b5b24b17 00007f69af09f6d4 00007f69af09f662 00007f69af09f591 00007f69af09f50c 00007f69af09e7c2 00007f69aee9a48c 00007f69aee998c6 00007f69aee9926f
0709 13:23:48.047143 (+    98us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":828170112}
I20260709 13:23:48.078661 26118 tablet_bootstrap.cc:492] T b2f14aed05e24a98ad83535029b795dc P 43999d813e564dc688fad116090c95ca: Bootstrap starting.
I20260709 13:23:48.082276 25309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "441d780c3945463f92f31170f31e37a8" candidate_uuid: "43999d813e564dc688fad116090c95ca" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" is_pre_election: true
I20260709 13:23:48.091595 25385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c7e7acbb09f4d49828fa652ca6abaac" candidate_uuid: "9868dce521294d0b816bba7648f4ae5a" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3e07d64466934da4bb3c02262b1c2e38"
I20260709 13:23:48.092379 25385 raft_consensus.cc:3060] T 5c7e7acbb09f4d49828fa652ca6abaac P 3e07d64466934da4bb3c02262b1c2e38 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:23:48.078015 26182 leader_election.cc:290] T 5c7e7acbb09f4d49828fa652ca6abaac P 9868dce521294d0b816bba7648f4ae5a [CANDIDATE]: Term 3 election: Requested vote from peers 3e07d64466934da4bb3c02262b1c2e38 (127.14.177.131:41539), 43999d813e564dc688fad116090c95ca (127.14.177.129:38155)
I20260709 13:23:48.078290 26086 raft_consensus.cc:493] T ba8a000f5abe4158965449995af84b18 P 9868dce521294d0b816bba7648f4ae5a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 233e7d29aac04505bf4e65f0a39a0c4c)
I20260709 13:23:48.100553 25309 raft_consensus.cc:1275] T 13bb3a23315a41c7a40e10b954b15c8e P 233e7d29aac04505bf4e65f0a39a0c4c [term 2 FOLLOWER]: Refusing update from remote peer 3e07d64466934da4bb3c02262b1c2e38: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:23:48.100737 26086 raft_consensus.cc:515] T ba8a000f5abe4158965449995af84b18 P 9868dce521294d0b816bba7648f4ae5a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } }
I20260709 13:23:48.102119 26157 consensus_queue.cc:1048] T 13bb3a23315a41c7a40e10b954b15c8e P 3e07d64466934da4bb3c02262b1c2e38 [LEADER]: Connected to new peer: Peer: permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:23:48.106761 25424 leader_election.cc:336] T 5c7e7acbb09f4d49828fa652ca6abaac P 9868dce521294d0b816bba7648f4ae5a [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 43999d813e564dc688fad116090c95ca (127.14.177.129:38155): Network error: Client connection negotiation failed: client connection to 127.14.177.129:38155: connect: Connection refused (error 111)
I20260709 13:23:48.107585 25383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba8a000f5abe4158965449995af84b18" candidate_uuid: "9868dce521294d0b816bba7648f4ae5a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3e07d64466934da4bb3c02262b1c2e38" is_pre_election: true
I20260709 13:23:48.105942 25385 raft_consensus.cc:2468] T 5c7e7acbb09f4d49828fa652ca6abaac P 3e07d64466934da4bb3c02262b1c2e38 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9868dce521294d0b816bba7648f4ae5a in term 3.
I20260709 13:23:48.111328 25422 leader_election.cc:304] T 5c7e7acbb09f4d49828fa652ca6abaac P 9868dce521294d0b816bba7648f4ae5a [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3e07d64466934da4bb3c02262b1c2e38, 9868dce521294d0b816bba7648f4ae5a; no voters: 43999d813e564dc688fad116090c95ca
I20260709 13:23:48.103081 26086 leader_election.cc:290] T ba8a000f5abe4158965449995af84b18 P 9868dce521294d0b816bba7648f4ae5a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 233e7d29aac04505bf4e65f0a39a0c4c (127.14.177.130:42957), 3e07d64466934da4bb3c02262b1c2e38 (127.14.177.131:41539)
W20260709 13:23:48.122865 25274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.132:33683, user_credentials={real_user=slave}} blocked reactor thread for 137577us
I20260709 13:23:48.127743 25134 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:35154 (request call id 94) took 1548 ms. Trace:
W20260709 13:23:48.137739 25276 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:38155, user_credentials={real_user=slave}} blocked reactor thread for 164088us
I20260709 13:23:48.144563 25132 catalog_manager.cc:5697] T 0e71b6f5c69b4b35a8ac51c22c7d5f0d P 3e07d64466934da4bb3c02262b1c2e38 reported cstate change: term changed from 1 to 2, leader changed from 43999d813e564dc688fad116090c95ca (127.14.177.129) to 3e07d64466934da4bb3c02262b1c2e38 (127.14.177.131). New cstate: current_term: 2 leader_uuid: "3e07d64466934da4bb3c02262b1c2e38" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } health_report { overall_health: UNKNOWN } } }
I20260709 13:23:48.160368 25132 catalog_manager.cc:5697] T 4d837697cd0f402cacf9db89a5d92652 P 3e07d64466934da4bb3c02262b1c2e38 reported cstate change: term changed from 1 to 2, leader changed from 43999d813e564dc688fad116090c95ca (127.14.177.129) to 3e07d64466934da4bb3c02262b1c2e38 (127.14.177.131). New cstate: current_term: 2 leader_uuid: "3e07d64466934da4bb3c02262b1c2e38" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } health_report { overall_health: UNKNOWN } } }
I20260709 13:23:48.123951 26182 raft_consensus.cc:2804] T 5c7e7acbb09f4d49828fa652ca6abaac P 9868dce521294d0b816bba7648f4ae5a [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:23:48.125558 26228 leader_election.cc:290] T 11949f56648c47e0a43c3140af99a304 P 233e7d29aac04505bf4e65f0a39a0c4c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9868dce521294d0b816bba7648f4ae5a (127.14.177.132:33683), 43999d813e564dc688fad116090c95ca (127.14.177.129:38155)
W20260709 13:23:48.186347 26233 log.cc:927] Time spent T b155b866f10547fd8239579a81ecf5f9 P 3e07d64466934da4bb3c02262b1c2e38: Append to log took a long time: real 0.087s	user 0.001s	sys 0.000s
I20260709 13:23:48.185364 25274 leader_election.cc:304] T 9ab90e3e08fb4528b2a47d995e9e26d3 P 233e7d29aac04505bf4e65f0a39a0c4c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 233e7d29aac04505bf4e65f0a39a0c4c; no voters: 3e07d64466934da4bb3c02262b1c2e38, 9868dce521294d0b816bba7648f4ae5a
I20260709 13:23:48.183462 26182 raft_consensus.cc:697] T 5c7e7acbb09f4d49828fa652ca6abaac P 9868dce521294d0b816bba7648f4ae5a [term 3 LEADER]: Becoming Leader. State: Replica: 9868dce521294d0b816bba7648f4ae5a, State: Running, Role: LEADER
I20260709 13:23:48.162796 26224 consensus_queue.cc:1048] T 441d780c3945463f92f31170f31e37a8 P 233e7d29aac04505bf4e65f0a39a0c4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.028s
I20260709 13:23:48.200912 25308 raft_consensus.cc:2468] T 9ab90e3e08fb4528b2a47d995e9e26d3 P 233e7d29aac04505bf4e65f0a39a0c4c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3e07d64466934da4bb3c02262b1c2e38 in term 2.
I20260709 13:23:48.203269 26163 raft_consensus.cc:2749] T 9ab90e3e08fb4528b2a47d995e9e26d3 P 233e7d29aac04505bf4e65f0a39a0c4c [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:23:48.193852 25274 leader_election.cc:304] T 434a54fcee7d4aeeab42f739855c0e8a P 233e7d29aac04505bf4e65f0a39a0c4c [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 233e7d29aac04505bf4e65f0a39a0c4c, 3e07d64466934da4bb3c02262b1c2e38; no voters: 43999d813e564dc688fad116090c95ca
I20260709 13:23:48.199239 26234 raft_consensus.cc:493] T f1ac2f4d2fe2424bb470df42199219e2 P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:23:48.162065 25132 catalog_manager.cc:5697] T 72b56266e8a54776b9113c7a786e1952 P 3e07d64466934da4bb3c02262b1c2e38 reported cstate change: term changed from 1 to 3, leader changed from 43999d813e564dc688fad116090c95ca (127.14.177.129) to 3e07d64466934da4bb3c02262b1c2e38 (127.14.177.131). New cstate: current_term: 3 leader_uuid: "3e07d64466934da4bb3c02262b1c2e38" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } health_report { overall_health: HEALTHY } } }
I20260709 13:23:48.137197 25134 rpcz_store.cc:276] 0709 13:23:46.579661 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:23:46.579951 (+   290us) service_pool.cc:224] Handling call
0709 13:23:48.127655 (+1547704us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:23:46.854406 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:23:46.854974 (+   568us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:23:46.855007 (+    33us) write_op.cc:435] Acquired schema lock
0709 13:23:46.855023 (+    16us) tablet.cc:662] Decoding operations
0709 13:23:46.855352 (+   329us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:23:46.855413 (+    61us) write_op.cc:641] Partition lock acquired for write op
0709 13:23:46.855439 (+    26us) tablet.cc:685] Acquiring locks for 1 operations
0709 13:23:46.855601 (+   162us) tablet.cc:701] Row locks acquired
0709 13:23:46.855617 (+    16us) write_op.cc:260] PREPARE: finished
0709 13:23:46.855740 (+   123us) write_op.cc:270] Start()
0709 13:23:46.855843 (+   103us) write_op.cc:276] Timestamp: P: 1783603426855702 usec, L: 0
0709 13:23:46.855863 (+    20us) op_driver.cc:348] REPLICATION: starting
0709 13:23:46.856497 (+   634us) log.cc:844] Serialized 543 byte log entry
0709 13:23:47.342830 (+486333us) op_driver.cc:464] REPLICATION: finished
0709 13:23:47.511422 (+168592us) write_op.cc:301] APPLY: starting
0709 13:23:47.511535 (+   113us) tablet.cc:1367] starting BulkCheckPresence
0709 13:23:47.511666 (+   131us) tablet.cc:1370] finished BulkCheckPresence
0709 13:23:47.511829 (+   163us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:23:47.512094 (+   265us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:23:47.512325 (+   231us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0709 13:23:47.512393 (+    68us) write_op.cc:312] APPLY: finished
0709 13:23:47.512720 (+   327us) log.cc:844] Serialized 22 byte log entry
0709 13:23:47.513513 (+   793us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:23:47.513595 (+    82us) write_op.cc:454] Released schema lock
0709 13:23:47.513634 (+    39us) write_op.cc:341] FINISH: Updating metrics
0709 13:23:47.514016 (+   382us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:23:47.514043 (+    27us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":358103,"reported_tablets":2,"child_traces":[["op",{"apply.queue_time_us":168203,"apply.run_cpu_time_us":1906,"apply.run_wall_time_us":2613,"num_ops":1,"prepare.queue_time_us":248255,"prepare.run_cpu_time_us":50038,"prepare.run_wall_time_us":114294,"raft.queue_time_us":211143,"raft.run_cpu_time_us":19141,"raft.run_wall_time_us":247494,"replication_time_us":486790,"thread_start_us":553921,"threads_started":4,"wal-append.queue_time_us":189592}]]}
I20260709 13:23:48.212852 25132 catalog_manager.cc:5697] T 13bb3a23315a41c7a40e10b954b15c8e P 3e07d64466934da4bb3c02262b1c2e38 reported cstate change: term changed from 1 to 2, leader changed from 43999d813e564dc688fad116090c95ca (127.14.177.129) to 3e07d64466934da4bb3c02262b1c2e38 (127.14.177.131). New cstate: current_term: 2 leader_uuid: "3e07d64466934da4bb3c02262b1c2e38" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } health_report { overall_health: HEALTHY } } }
I20260709 13:23:48.217123 25132 catalog_manager.cc:5697] T 821a5866429f4173820fb64336abee02 P 3e07d64466934da4bb3c02262b1c2e38 reported cstate change: term changed from 1 to 2, leader changed from 43999d813e564dc688fad116090c95ca (127.14.177.129) to 3e07d64466934da4bb3c02262b1c2e38 (127.14.177.131). New cstate: current_term: 2 leader_uuid: "3e07d64466934da4bb3c02262b1c2e38" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } health_report { overall_health: UNKNOWN } } }
I20260709 13:23:48.204945 26182 consensus_queue.cc:237] T 5c7e7acbb09f4d49828fa652ca6abaac P 9868dce521294d0b816bba7648f4ae5a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } }
I20260709 13:23:48.225868 25383 raft_consensus.cc:1240] T 9ab90e3e08fb4528b2a47d995e9e26d3 P 3e07d64466934da4bb3c02262b1c2e38 [term 2 FOLLOWER]: Rejecting Update request from peer 9868dce521294d0b816bba7648f4ae5a for earlier term 1. Current term is 2. Ops: []
I20260709 13:23:48.211333 26234 raft_consensus.cc:515] T f1ac2f4d2fe2424bb470df42199219e2 P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } }
W20260709 13:23:48.235602 25424 consensus_peers.cc:597] T d0a0e4d32ce943fbb773bb840863647b P 9868dce521294d0b816bba7648f4ae5a -> Peer 43999d813e564dc688fad116090c95ca (127.14.177.129:38155): Couldn't send request to peer 43999d813e564dc688fad116090c95ca. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:38155: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:23:48.238353 26234 leader_election.cc:290] T f1ac2f4d2fe2424bb470df42199219e2 P 43999d813e564dc688fad116090c95ca [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3e07d64466934da4bb3c02262b1c2e38 (127.14.177.131:41539), 9868dce521294d0b816bba7648f4ae5a (127.14.177.132:33683)
I20260709 13:23:48.238826 25382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1ac2f4d2fe2424bb470df42199219e2" candidate_uuid: "43999d813e564dc688fad116090c95ca" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3e07d64466934da4bb3c02262b1c2e38" is_pre_election: true
I20260709 13:23:48.240113 25382 raft_consensus.cc:2393] T f1ac2f4d2fe2424bb470df42199219e2 P 3e07d64466934da4bb3c02262b1c2e38 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 43999d813e564dc688fad116090c95ca in current term 2: Already voted for candidate 3e07d64466934da4bb3c02262b1c2e38 in this term.
W20260709 13:23:48.240262 25276 consensus_peers.cc:597] T 3a3317f4c3514df1b2bb8e44a86be92f P 233e7d29aac04505bf4e65f0a39a0c4c -> Peer 43999d813e564dc688fad116090c95ca (127.14.177.129:38155): Couldn't send request to peer 43999d813e564dc688fad116090c95ca. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:38155: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:23:48.210839 26163 raft_consensus.cc:2804] T 434a54fcee7d4aeeab42f739855c0e8a P 233e7d29aac04505bf4e65f0a39a0c4c [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:23:48.242959 25132 catalog_manager.cc:5697] T 1550ed2d2a764c08adffae679599ba41 P 3e07d64466934da4bb3c02262b1c2e38 reported cstate change: term changed from 1 to 2, leader changed from 43999d813e564dc688fad116090c95ca (127.14.177.129) to 9868dce521294d0b816bba7648f4ae5a (127.14.177.132). New cstate: current_term: 2 leader_uuid: "9868dce521294d0b816bba7648f4ae5a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } }
I20260709 13:23:48.261782 25351 leader_election.cc:304] T 9ab90e3e08fb4528b2a47d995e9e26d3 P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 233e7d29aac04505bf4e65f0a39a0c4c, 3e07d64466934da4bb3c02262b1c2e38; no voters: 9868dce521294d0b816bba7648f4ae5a
I20260709 13:23:48.264842 26212 raft_consensus.cc:2804] T 9ab90e3e08fb4528b2a47d995e9e26d3 P 3e07d64466934da4bb3c02262b1c2e38 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:23:48.265522 26212 raft_consensus.cc:697] T 9ab90e3e08fb4528b2a47d995e9e26d3 P 3e07d64466934da4bb3c02262b1c2e38 [term 2 LEADER]: Becoming Leader. State: Replica: 3e07d64466934da4bb3c02262b1c2e38, State: Running, Role: LEADER
I20260709 13:23:48.268322 26163 raft_consensus.cc:697] T 434a54fcee7d4aeeab42f739855c0e8a P 233e7d29aac04505bf4e65f0a39a0c4c [term 4 LEADER]: Becoming Leader. State: Replica: 233e7d29aac04505bf4e65f0a39a0c4c, State: Running, Role: LEADER
I20260709 13:23:48.271286 26163 consensus_queue.cc:237] T 434a54fcee7d4aeeab42f739855c0e8a P 233e7d29aac04505bf4e65f0a39a0c4c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } }
I20260709 13:23:48.276656 25458 raft_consensus.cc:1275] T 821a5866429f4173820fb64336abee02 P 9868dce521294d0b816bba7648f4ae5a [term 2 FOLLOWER]: Refusing update from remote peer 3e07d64466934da4bb3c02262b1c2e38: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:23:48.278164 26212 consensus_queue.cc:237] T 9ab90e3e08fb4528b2a47d995e9e26d3 P 3e07d64466934da4bb3c02262b1c2e38 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } }
I20260709 13:23:48.293501 26118 tablet_bootstrap.cc:492] T b2f14aed05e24a98ad83535029b795dc P 43999d813e564dc688fad116090c95ca: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
W20260709 13:23:48.296753 25350 consensus_peers.cc:597] T 13bb3a23315a41c7a40e10b954b15c8e P 3e07d64466934da4bb3c02262b1c2e38 -> Peer 43999d813e564dc688fad116090c95ca (127.14.177.129:38155): Couldn't send request to peer 43999d813e564dc688fad116090c95ca. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:38155: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:23:48.299911 25311 raft_consensus.cc:1240] T 9ab90e3e08fb4528b2a47d995e9e26d3 P 233e7d29aac04505bf4e65f0a39a0c4c [term 2 FOLLOWER]: Rejecting Update request from peer 9868dce521294d0b816bba7648f4ae5a for earlier term 1. Current term is 2. Ops: []
I20260709 13:23:48.302155 25304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba8a000f5abe4158965449995af84b18" candidate_uuid: "9868dce521294d0b816bba7648f4ae5a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" is_pre_election: true
I20260709 13:23:48.302317 26152 consensus_queue.cc:1048] T 821a5866429f4173820fb64336abee02 P 3e07d64466934da4bb3c02262b1c2e38 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:23:48.302996 25308 raft_consensus.cc:1275] T 72b56266e8a54776b9113c7a786e1952 P 233e7d29aac04505bf4e65f0a39a0c4c [term 3 FOLLOWER]: Refusing update from remote peer 3e07d64466934da4bb3c02262b1c2e38: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:23:48.305879 26234 raft_consensus.cc:493] T 13bb3a23315a41c7a40e10b954b15c8e P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:23:48.306453 26234 raft_consensus.cc:515] T 13bb3a23315a41c7a40e10b954b15c8e P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } }
I20260709 13:23:48.308749 26234 leader_election.cc:290] T 13bb3a23315a41c7a40e10b954b15c8e P 43999d813e564dc688fad116090c95ca [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 233e7d29aac04505bf4e65f0a39a0c4c (127.14.177.130:42957), 3e07d64466934da4bb3c02262b1c2e38 (127.14.177.131:41539)
I20260709 13:23:48.309264 26157 consensus_queue.cc:1048] T 72b56266e8a54776b9113c7a786e1952 P 3e07d64466934da4bb3c02262b1c2e38 [LEADER]: Connected to new peer: Peer: permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:23:48.331626 25307 raft_consensus.cc:1275] T 8ef061028f38450e83324ac78cdaa14c P 233e7d29aac04505bf4e65f0a39a0c4c [term 2 FOLLOWER]: Refusing update from remote peer 9868dce521294d0b816bba7648f4ae5a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:23:48.337347 26118 tablet_bootstrap.cc:492] T b2f14aed05e24a98ad83535029b795dc P 43999d813e564dc688fad116090c95ca: Bootstrap complete.
I20260709 13:23:48.338376 26118 ts_tablet_manager.cc:1403] T b2f14aed05e24a98ad83535029b795dc P 43999d813e564dc688fad116090c95ca: Time spent bootstrapping tablet: real 0.287s	user 0.022s	sys 0.017s
I20260709 13:23:48.333050 26155 raft_consensus.cc:493] T b9d550d881b04735bbf89751952c09ec P 3e07d64466934da4bb3c02262b1c2e38 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:23:48.352028 25276 consensus_peers.cc:597] T ce9c60e3922c4219b6e4453fe0259b6a P 233e7d29aac04505bf4e65f0a39a0c4c -> Peer 43999d813e564dc688fad116090c95ca (127.14.177.129:38155): Couldn't send request to peer 43999d813e564dc688fad116090c95ca. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:38155: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:23:48.352588 25276 consensus_peers.cc:597] T 441d780c3945463f92f31170f31e37a8 P 233e7d29aac04505bf4e65f0a39a0c4c -> Peer 43999d813e564dc688fad116090c95ca (127.14.177.129:38155): Couldn't send request to peer 43999d813e564dc688fad116090c95ca. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:38155: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:23:48.353052 25276 consensus_peers.cc:597] T b85a238bbc3b4621881603a534c7bd9b P 233e7d29aac04505bf4e65f0a39a0c4c -> Peer 43999d813e564dc688fad116090c95ca (127.14.177.129:38155): Couldn't send request to peer 43999d813e564dc688fad116090c95ca. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:38155: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:23:48.361763 26194 consensus_queue.cc:1059] T 9ab90e3e08fb4528b2a47d995e9e26d3 P 9868dce521294d0b816bba7648f4ae5a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:23:48.354553 26189 raft_consensus.cc:493] T 579da2d6aabb4ae9a2b1802e1358068b P 9868dce521294d0b816bba7648f4ae5a [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:23:48.362875 26189 raft_consensus.cc:515] T 579da2d6aabb4ae9a2b1802e1358068b P 9868dce521294d0b816bba7648f4ae5a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } }
I20260709 13:23:48.363543 25425 leader_election.cc:304] T ba8a000f5abe4158965449995af84b18 P 9868dce521294d0b816bba7648f4ae5a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9868dce521294d0b816bba7648f4ae5a; no voters: 233e7d29aac04505bf4e65f0a39a0c4c, 3e07d64466934da4bb3c02262b1c2e38
I20260709 13:23:48.364904 26220 raft_consensus.cc:2749] T ba8a000f5abe4158965449995af84b18 P 9868dce521294d0b816bba7648f4ae5a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:23:48.362906 26194 raft_consensus.cc:3055] T 9ab90e3e08fb4528b2a47d995e9e26d3 P 9868dce521294d0b816bba7648f4ae5a [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:23:48.365854 26194 raft_consensus.cc:740] T 9ab90e3e08fb4528b2a47d995e9e26d3 P 9868dce521294d0b816bba7648f4ae5a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 9868dce521294d0b816bba7648f4ae5a, State: Running, Role: LEADER
I20260709 13:23:48.337152 25306 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13bb3a23315a41c7a40e10b954b15c8e" candidate_uuid: "43999d813e564dc688fad116090c95ca" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" is_pre_election: true
I20260709 13:23:48.365016 26189 leader_election.cc:290] T 579da2d6aabb4ae9a2b1802e1358068b P 9868dce521294d0b816bba7648f4ae5a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 43999d813e564dc688fad116090c95ca (127.14.177.129:38155), 233e7d29aac04505bf4e65f0a39a0c4c (127.14.177.130:42957)
I20260709 13:23:48.349181 26155 raft_consensus.cc:515] T b9d550d881b04735bbf89751952c09ec P 3e07d64466934da4bb3c02262b1c2e38 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } }
I20260709 13:23:48.394284 26155 leader_election.cc:290] T b9d550d881b04735bbf89751952c09ec P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9868dce521294d0b816bba7648f4ae5a (127.14.177.132:33683), 43999d813e564dc688fad116090c95ca (127.14.177.129:38155)
I20260709 13:23:48.369484 25382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13bb3a23315a41c7a40e10b954b15c8e" candidate_uuid: "43999d813e564dc688fad116090c95ca" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3e07d64466934da4bb3c02262b1c2e38" is_pre_election: true
W20260709 13:23:48.380251 25276 leader_election.cc:336] T 11949f56648c47e0a43c3140af99a304 P 233e7d29aac04505bf4e65f0a39a0c4c [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 43999d813e564dc688fad116090c95ca (127.14.177.129:38155): Network error: Client connection negotiation failed: client connection to 127.14.177.129:38155: connect: Connection refused (error 111)
I20260709 13:23:48.387439 25450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11949f56648c47e0a43c3140af99a304" candidate_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9868dce521294d0b816bba7648f4ae5a" is_pre_election: true
I20260709 13:23:48.388046 26220 consensus_queue.cc:1048] T 8ef061028f38450e83324ac78cdaa14c P 9868dce521294d0b816bba7648f4ae5a [LEADER]: Connected to new peer: Peer: permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:23:48.424547 25304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "579da2d6aabb4ae9a2b1802e1358068b" candidate_uuid: "9868dce521294d0b816bba7648f4ae5a" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" is_pre_election: true
I20260709 13:23:48.425379 25304 raft_consensus.cc:2468] T 579da2d6aabb4ae9a2b1802e1358068b P 233e7d29aac04505bf4e65f0a39a0c4c [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9868dce521294d0b816bba7648f4ae5a in term 2.
W20260709 13:23:48.432307 25350 consensus_peers.cc:597] T 72b56266e8a54776b9113c7a786e1952 P 3e07d64466934da4bb3c02262b1c2e38 -> Peer 43999d813e564dc688fad116090c95ca (127.14.177.129:38155): Couldn't send request to peer 43999d813e564dc688fad116090c95ca. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:38155: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:23:48.431530 25425 leader_election.cc:304] T 579da2d6aabb4ae9a2b1802e1358068b P 9868dce521294d0b816bba7648f4ae5a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 233e7d29aac04505bf4e65f0a39a0c4c, 9868dce521294d0b816bba7648f4ae5a; no voters: 
W20260709 13:23:48.433790 25350 consensus_peers.cc:597] T 0e71b6f5c69b4b35a8ac51c22c7d5f0d P 3e07d64466934da4bb3c02262b1c2e38 -> Peer 43999d813e564dc688fad116090c95ca (127.14.177.129:38155): Couldn't send request to peer 43999d813e564dc688fad116090c95ca. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:38155: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:23:48.366663 26194 consensus_queue.cc:260] T 9ab90e3e08fb4528b2a47d995e9e26d3 P 9868dce521294d0b816bba7648f4ae5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } }
I20260709 13:23:48.439378 25376 raft_consensus.cc:1275] T b9d550d881b04735bbf89751952c09ec P 3e07d64466934da4bb3c02262b1c2e38 [term 2 FOLLOWER]: Refusing update from remote peer 9868dce521294d0b816bba7648f4ae5a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:23:48.444576 26135 raft_consensus.cc:2804] T 579da2d6aabb4ae9a2b1802e1358068b P 9868dce521294d0b816bba7648f4ae5a [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 13:23:48.488727 25276 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.14.177.129:38155: connect: Connection refused (error 111) [suppressed 310 similar messages]
I20260709 13:23:48.459795 25459 raft_consensus.cc:1275] T 0e71b6f5c69b4b35a8ac51c22c7d5f0d P 9868dce521294d0b816bba7648f4ae5a [term 2 FOLLOWER]: Refusing update from remote peer 3e07d64466934da4bb3c02262b1c2e38: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:23:48.493736 26118 raft_consensus.cc:359] T b2f14aed05e24a98ad83535029b795dc P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } }
I20260709 13:23:48.496218 26118 raft_consensus.cc:740] T b2f14aed05e24a98ad83535029b795dc P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43999d813e564dc688fad116090c95ca, State: Initialized, Role: FOLLOWER
I20260709 13:23:48.501463 26118 consensus_queue.cc:260] T b2f14aed05e24a98ad83535029b795dc P 43999d813e564dc688fad116090c95ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } }
I20260709 13:23:48.467859 26135 raft_consensus.cc:493] T 579da2d6aabb4ae9a2b1802e1358068b P 9868dce521294d0b816bba7648f4ae5a [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:23:48.455408 26086 consensus_queue.cc:1059] T 9ab90e3e08fb4528b2a47d995e9e26d3 P 9868dce521294d0b816bba7648f4ae5a [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.042s
I20260709 13:23:48.467244 26111 leader_election.cc:304] T 13bb3a23315a41c7a40e10b954b15c8e P 43999d813e564dc688fad116090c95ca [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 43999d813e564dc688fad116090c95ca; no voters: 233e7d29aac04505bf4e65f0a39a0c4c, 3e07d64466934da4bb3c02262b1c2e38
W20260709 13:23:48.516069 26111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:41539, user_credentials={real_user=slave}} blocked reactor thread for 63803.4us
I20260709 13:23:48.522517 26118 ts_tablet_manager.cc:1434] T b2f14aed05e24a98ad83535029b795dc P 43999d813e564dc688fad116090c95ca: Time spent starting tablet: real 0.184s	user 0.008s	sys 0.001s
I20260709 13:23:48.525949 26118 tablet_bootstrap.cc:492] T 0ed5edbe32044d1c83b0590dbb899801 P 43999d813e564dc688fad116090c95ca: Bootstrap starting.
I20260709 13:23:48.507679 26135 raft_consensus.cc:3060] T 579da2d6aabb4ae9a2b1802e1358068b P 9868dce521294d0b816bba7648f4ae5a [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:23:48.524303 26234 raft_consensus.cc:2749] T 13bb3a23315a41c7a40e10b954b15c8e P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:23:48.524953 26058 consensus_queue.cc:1048] T 0e71b6f5c69b4b35a8ac51c22c7d5f0d P 3e07d64466934da4bb3c02262b1c2e38 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.020s
W20260709 13:23:48.560357 26258 log.cc:927] Time spent T 9ab90e3e08fb4528b2a47d995e9e26d3 P 3e07d64466934da4bb3c02262b1c2e38: Append to log took a long time: real 0.095s	user 0.000s	sys 0.000s
W20260709 13:23:48.563025 25276 consensus_peers.cc:597] T 0d29b499fa4c41f1b974782cd0c9c1c2 P 233e7d29aac04505bf4e65f0a39a0c4c -> Peer 43999d813e564dc688fad116090c95ca (127.14.177.129:38155): Couldn't send request to peer 43999d813e564dc688fad116090c95ca. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:38155: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:23:48.594379 25454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9d550d881b04735bbf89751952c09ec" candidate_uuid: "3e07d64466934da4bb3c02262b1c2e38" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9868dce521294d0b816bba7648f4ae5a" is_pre_election: true
I20260709 13:23:48.602408 26135 raft_consensus.cc:515] T 579da2d6aabb4ae9a2b1802e1358068b P 9868dce521294d0b816bba7648f4ae5a [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } }
I20260709 13:23:48.631292 25457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "441d780c3945463f92f31170f31e37a8" candidate_uuid: "43999d813e564dc688fad116090c95ca" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9868dce521294d0b816bba7648f4ae5a" is_pre_election: true
I20260709 13:23:48.632629 25456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1ac2f4d2fe2424bb470df42199219e2" candidate_uuid: "43999d813e564dc688fad116090c95ca" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9868dce521294d0b816bba7648f4ae5a" is_pre_election: true
I20260709 13:23:48.633538 25456 raft_consensus.cc:2393] T f1ac2f4d2fe2424bb470df42199219e2 P 9868dce521294d0b816bba7648f4ae5a [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 43999d813e564dc688fad116090c95ca in current term 2: Already voted for candidate 9868dce521294d0b816bba7648f4ae5a in this term.
I20260709 13:23:48.633739 26194 raft_consensus.cc:3060] T 9ab90e3e08fb4528b2a47d995e9e26d3 P 9868dce521294d0b816bba7648f4ae5a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:23:48.634922 26111 leader_election.cc:304] T 441d780c3945463f92f31170f31e37a8 P 43999d813e564dc688fad116090c95ca [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 43999d813e564dc688fad116090c95ca; no voters: 233e7d29aac04505bf4e65f0a39a0c4c, 9868dce521294d0b816bba7648f4ae5a
I20260709 13:23:48.635841 26234 raft_consensus.cc:2749] T 441d780c3945463f92f31170f31e37a8 P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:23:48.636292 26111 leader_election.cc:304] T f1ac2f4d2fe2424bb470df42199219e2 P 43999d813e564dc688fad116090c95ca [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 43999d813e564dc688fad116090c95ca; no voters: 3e07d64466934da4bb3c02262b1c2e38, 9868dce521294d0b816bba7648f4ae5a
I20260709 13:23:48.637125 26234 raft_consensus.cc:3060] T f1ac2f4d2fe2424bb470df42199219e2 P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:23:48.647578 25304 raft_consensus.cc:1275] T a5df750b84e0487fa530161079deb6c2 P 233e7d29aac04505bf4e65f0a39a0c4c [term 2 FOLLOWER]: Refusing update from remote peer 9868dce521294d0b816bba7648f4ae5a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:23:48.651842 26135 leader_election.cc:290] T 579da2d6aabb4ae9a2b1802e1358068b P 9868dce521294d0b816bba7648f4ae5a [CANDIDATE]: Term 3 election: Requested vote from peers 43999d813e564dc688fad116090c95ca (127.14.177.129:38155), 233e7d29aac04505bf4e65f0a39a0c4c (127.14.177.130:42957)
I20260709 13:23:48.652391 26234 raft_consensus.cc:2749] T f1ac2f4d2fe2424bb470df42199219e2 P 43999d813e564dc688fad116090c95ca [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:23:48.644304 25307 raft_consensus.cc:1275] T b155b866f10547fd8239579a81ecf5f9 P 233e7d29aac04505bf4e65f0a39a0c4c [term 3 FOLLOWER]: Refusing update from remote peer 3e07d64466934da4bb3c02262b1c2e38: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:23:48.665181 25304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "579da2d6aabb4ae9a2b1802e1358068b" candidate_uuid: "9868dce521294d0b816bba7648f4ae5a" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "233e7d29aac04505bf4e65f0a39a0c4c"
I20260709 13:23:48.665895 25304 raft_consensus.cc:3060] T 579da2d6aabb4ae9a2b1802e1358068b P 233e7d29aac04505bf4e65f0a39a0c4c [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:23:48.674219 25304 raft_consensus.cc:2468] T 579da2d6aabb4ae9a2b1802e1358068b P 233e7d29aac04505bf4e65f0a39a0c4c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9868dce521294d0b816bba7648f4ae5a in term 3.
I20260709 13:23:48.661679 26158 consensus_queue.cc:1048] T b155b866f10547fd8239579a81ecf5f9 P 3e07d64466934da4bb3c02262b1c2e38 [LEADER]: Connected to new peer: Peer: permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.017s
I20260709 13:23:48.675086 25274 leader_election.cc:304] T 11949f56648c47e0a43c3140af99a304 P 233e7d29aac04505bf4e65f0a39a0c4c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 233e7d29aac04505bf4e65f0a39a0c4c; no voters: 43999d813e564dc688fad116090c95ca, 9868dce521294d0b816bba7648f4ae5a
I20260709 13:23:48.684532 25425 leader_election.cc:304] T 579da2d6aabb4ae9a2b1802e1358068b P 9868dce521294d0b816bba7648f4ae5a [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 233e7d29aac04505bf4e65f0a39a0c4c, 9868dce521294d0b816bba7648f4ae5a; no voters: 
W20260709 13:23:48.677349 25424 consensus_peers.cc:597] T 8ef061028f38450e83324ac78cdaa14c P 9868dce521294d0b816bba7648f4ae5a -> Peer 43999d813e564dc688fad116090c95ca (127.14.177.129:38155): Couldn't send request to peer 43999d813e564dc688fad116090c95ca. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:38155: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:23:48.698865 25130 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:35126 (request call id 100) took 1736 ms. Trace:
W20260709 13:23:48.703620 25424 leader_election.cc:336] T 579da2d6aabb4ae9a2b1802e1358068b P 9868dce521294d0b816bba7648f4ae5a [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 43999d813e564dc688fad116090c95ca (127.14.177.129:38155): Network error: Client connection negotiation failed: client connection to 127.14.177.129:38155: connect: Connection refused (error 111)
I20260709 13:23:48.685317 26155 raft_consensus.cc:493] T b7ff8dde9f5d4c49b9203d902f6b94c4 P 3e07d64466934da4bb3c02262b1c2e38 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:23:48.661398 26135 consensus_queue.cc:1048] T a5df750b84e0487fa530161079deb6c2 P 9868dce521294d0b816bba7648f4ae5a [LEADER]: Connected to new peer: Peer: permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.044s
I20260709 13:23:48.726181 25382 raft_consensus.cc:1275] T 434a54fcee7d4aeeab42f739855c0e8a P 3e07d64466934da4bb3c02262b1c2e38 [term 4 FOLLOWER]: Refusing update from remote peer 233e7d29aac04505bf4e65f0a39a0c4c: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:23:48.702169 26220 raft_consensus.cc:2804] T 579da2d6aabb4ae9a2b1802e1358068b P 9868dce521294d0b816bba7648f4ae5a [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:23:48.726611 25379 raft_consensus.cc:1275] T c902bbc7ec0848ba82a04870eab1e7f5 P 3e07d64466934da4bb3c02262b1c2e38 [term 3 FOLLOWER]: Refusing update from remote peer 9868dce521294d0b816bba7648f4ae5a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:23:48.703027 26229 raft_consensus.cc:2749] T 11949f56648c47e0a43c3140af99a304 P 233e7d29aac04505bf4e65f0a39a0c4c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:23:48.746762 25350 leader_election.cc:336] T b9d550d881b04735bbf89751952c09ec P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 43999d813e564dc688fad116090c95ca (127.14.177.129:38155): Network error: Client connection negotiation failed: client connection to 127.14.177.129:38155: connect: Connection refused (error 111)
I20260709 13:23:48.747318 25350 leader_election.cc:304] T b9d550d881b04735bbf89751952c09ec P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3e07d64466934da4bb3c02262b1c2e38; no voters: 43999d813e564dc688fad116090c95ca, 9868dce521294d0b816bba7648f4ae5a
I20260709 13:23:48.699535 25130 rpcz_store.cc:276] 0709 13:23:46.962008 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:23:46.962167 (+   159us) service_pool.cc:224] Handling call
0709 13:23:47.026124 (+ 63957us) spinlock_profiling.cc:243] Waited 6.16 ms on lock 0x7b54007e0754. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac9090 00007f69b0b67f3a 00007f69b691e1cc 00007f69b6a1c37e 00007f69b4edd2a4 00007f69b4edd232 00007f69b4edd161 00007f69b4edd0dc 00007f69b4edc392 00007f69aee9a48c 00007f69aee998c6
0709 13:23:48.698771 (+1672647us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:23:48.251242 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:23:48.251822 (+   580us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:23:48.251867 (+    45us) write_op.cc:435] Acquired schema lock
0709 13:23:48.251879 (+    12us) tablet.cc:662] Decoding operations
0709 13:23:48.252230 (+   351us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:23:48.252294 (+    64us) write_op.cc:641] Partition lock acquired for write op
0709 13:23:48.252319 (+    25us) tablet.cc:685] Acquiring locks for 1 operations
0709 13:23:48.252508 (+   189us) tablet.cc:701] Row locks acquired
0709 13:23:48.252520 (+    12us) write_op.cc:260] PREPARE: finished
0709 13:23:48.252658 (+   138us) write_op.cc:270] Start()
0709 13:23:48.252767 (+   109us) write_op.cc:276] Timestamp: P: 1783603428252613 usec, L: 0
0709 13:23:48.252783 (+    16us) op_driver.cc:348] REPLICATION: starting
0709 13:23:48.256835 (+  4052us) log.cc:844] Serialized 543 byte log entry
0709 13:23:48.278993 (+ 22158us) op_driver.cc:464] REPLICATION: finished
0709 13:23:48.466461 (+187468us) write_op.cc:301] APPLY: starting
0709 13:23:48.469947 (+  3486us) tablet.cc:1367] starting BulkCheckPresence
0709 13:23:48.499950 (+ 30003us) tablet.cc:1370] finished BulkCheckPresence
0709 13:23:48.517291 (+ 17341us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:23:48.550469 (+ 33178us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:23:48.559454 (+  8985us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0709 13:23:48.568821 (+  9367us) write_op.cc:312] APPLY: finished
0709 13:23:48.569009 (+   188us) log.cc:844] Serialized 22 byte log entry
0709 13:23:48.569223 (+   214us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:23:48.569310 (+    87us) write_op.cc:454] Released schema lock
0709 13:23:48.569349 (+    39us) write_op.cc:341] FINISH: Updating metrics
0709 13:23:48.569701 (+   352us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:23:48.569730 (+    29us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":510413,"reported_tablets":1,"spinlock_wait_cycles":14179072,"child_traces":[["op",{"apply.queue_time_us":170360,"apply.run_cpu_time_us":2423,"apply.run_wall_time_us":114297,"num_ops":1,"prepare.queue_time_us":110059,"prepare.run_cpu_time_us":2679,"prepare.run_wall_time_us":10268,"replication_time_us":26071,"thread_start_us":155557,"threads_started":2}]]}
I20260709 13:23:48.718223 26086 consensus_queue.cc:1048] T b9d550d881b04735bbf89751952c09ec P 9868dce521294d0b816bba7648f4ae5a [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.033s
I20260709 13:23:48.749866 26220 raft_consensus.cc:697] T 579da2d6aabb4ae9a2b1802e1358068b P 9868dce521294d0b816bba7648f4ae5a [term 3 LEADER]: Becoming Leader. State: Replica: 9868dce521294d0b816bba7648f4ae5a, State: Running, Role: LEADER
I20260709 13:23:48.755227 26190 raft_consensus.cc:2749] T b9d550d881b04735bbf89751952c09ec P 3e07d64466934da4bb3c02262b1c2e38 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:23:48.778633 26182 raft_consensus.cc:493] T f1ac2f4d2fe2424bb470df42199219e2 P 9868dce521294d0b816bba7648f4ae5a [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:23:48.786039 26220 consensus_queue.cc:237] T 579da2d6aabb4ae9a2b1802e1358068b P 9868dce521294d0b816bba7648f4ae5a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } }
I20260709 13:23:48.786546 26182 raft_consensus.cc:515] T f1ac2f4d2fe2424bb470df42199219e2 P 9868dce521294d0b816bba7648f4ae5a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } }
I20260709 13:23:48.789369 26182 leader_election.cc:290] T f1ac2f4d2fe2424bb470df42199219e2 P 9868dce521294d0b816bba7648f4ae5a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3e07d64466934da4bb3c02262b1c2e38 (127.14.177.131:41539), 43999d813e564dc688fad116090c95ca (127.14.177.129:38155)
I20260709 13:23:48.740192 26155 raft_consensus.cc:515] T b7ff8dde9f5d4c49b9203d902f6b94c4 P 3e07d64466934da4bb3c02262b1c2e38 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } }
I20260709 13:23:48.807282 25134 catalog_manager.cc:5697] T c902bbc7ec0848ba82a04870eab1e7f5 P 9868dce521294d0b816bba7648f4ae5a reported cstate change: term changed from 1 to 3, leader changed from 43999d813e564dc688fad116090c95ca (127.14.177.129) to 9868dce521294d0b816bba7648f4ae5a (127.14.177.132). New cstate: current_term: 3 leader_uuid: "9868dce521294d0b816bba7648f4ae5a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } health_report { overall_health: UNKNOWN } } }
I20260709 13:23:48.844406 25134 catalog_manager.cc:5697] T d0a0e4d32ce943fbb773bb840863647b P 9868dce521294d0b816bba7648f4ae5a reported cstate change: term changed from 1 to 2, leader changed from 43999d813e564dc688fad116090c95ca (127.14.177.129) to 9868dce521294d0b816bba7648f4ae5a (127.14.177.132). New cstate: current_term: 2 leader_uuid: "9868dce521294d0b816bba7648f4ae5a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } health_report { overall_health: UNKNOWN } } }
I20260709 13:23:48.858522 25134 catalog_manager.cc:5697] T b9d550d881b04735bbf89751952c09ec P 9868dce521294d0b816bba7648f4ae5a reported cstate change: term changed from 1 to 2, leader changed from 43999d813e564dc688fad116090c95ca (127.14.177.129) to 9868dce521294d0b816bba7648f4ae5a (127.14.177.132). New cstate: current_term: 2 leader_uuid: "9868dce521294d0b816bba7648f4ae5a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } health_report { overall_health: UNKNOWN } } }
W20260709 13:23:48.864797 25424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.129:38155, user_credentials={real_user=slave}} blocked reactor thread for 103944us
W20260709 13:23:48.865723 25424 consensus_peers.cc:597] T c902bbc7ec0848ba82a04870eab1e7f5 P 9868dce521294d0b816bba7648f4ae5a -> Peer 43999d813e564dc688fad116090c95ca (127.14.177.129:38155): Couldn't send request to peer 43999d813e564dc688fad116090c95ca. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:38155: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:23:48.887713 25383 raft_consensus.cc:1275] T 5c7e7acbb09f4d49828fa652ca6abaac P 3e07d64466934da4bb3c02262b1c2e38 [term 3 FOLLOWER]: Refusing update from remote peer 9868dce521294d0b816bba7648f4ae5a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:23:48.913707 25380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1ac2f4d2fe2424bb470df42199219e2" candidate_uuid: "9868dce521294d0b816bba7648f4ae5a" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3e07d64466934da4bb3c02262b1c2e38" is_pre_election: true
I20260709 13:23:48.976989 26118 tablet_bootstrap.cc:492] T 0ed5edbe32044d1c83b0590dbb899801 P 43999d813e564dc688fad116090c95ca: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
W20260709 13:23:48.963557 25274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41539, user_credentials={real_user=slave}} blocked reactor thread for 194921us
I20260709 13:23:48.860924 25134 catalog_manager.cc:5697] T 8ef061028f38450e83324ac78cdaa14c P 9868dce521294d0b816bba7648f4ae5a reported cstate change: term changed from 1 to 2, leader changed from 43999d813e564dc688fad116090c95ca (127.14.177.129) to 9868dce521294d0b816bba7648f4ae5a (127.14.177.132). New cstate: current_term: 2 leader_uuid: "9868dce521294d0b816bba7648f4ae5a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } health_report { overall_health: UNKNOWN } } }
I20260709 13:23:48.891739 26184 consensus_queue.cc:1048] T c902bbc7ec0848ba82a04870eab1e7f5 P 9868dce521294d0b816bba7648f4ae5a [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.051s
I20260709 13:23:48.984090 25134 catalog_manager.cc:5697] T a5df750b84e0487fa530161079deb6c2 P 9868dce521294d0b816bba7648f4ae5a reported cstate change: term changed from 1 to 2, leader changed from 43999d813e564dc688fad116090c95ca (127.14.177.129) to 9868dce521294d0b816bba7648f4ae5a (127.14.177.132). New cstate: current_term: 2 leader_uuid: "9868dce521294d0b816bba7648f4ae5a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } health_report { overall_health: UNKNOWN } } }
I20260709 13:23:48.989071 25134 catalog_manager.cc:5697] T 5c7e7acbb09f4d49828fa652ca6abaac P 9868dce521294d0b816bba7648f4ae5a reported cstate change: term changed from 1 to 3, leader changed from 43999d813e564dc688fad116090c95ca (127.14.177.129) to 9868dce521294d0b816bba7648f4ae5a (127.14.177.132). New cstate: current_term: 3 leader_uuid: "9868dce521294d0b816bba7648f4ae5a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } health_report { overall_health: UNKNOWN } } }
I20260709 13:23:48.916447 26163 consensus_queue.cc:1048] T 434a54fcee7d4aeeab42f739855c0e8a P 233e7d29aac04505bf4e65f0a39a0c4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.061s
W20260709 13:23:49.050885 25274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41539, user_credentials={real_user=slave}} blocked reactor thread for 63999.6us
W20260709 13:23:48.979123 26268 log.cc:927] Time spent T 8ef061028f38450e83324ac78cdaa14c P 233e7d29aac04505bf4e65f0a39a0c4c: Append to log took a long time: real 0.065s	user 0.000s	sys 0.001s
I20260709 13:23:49.054358 26118 tablet_bootstrap.cc:492] T 0ed5edbe32044d1c83b0590dbb899801 P 43999d813e564dc688fad116090c95ca: Bootstrap complete.
I20260709 13:23:49.059535 26118 ts_tablet_manager.cc:1403] T 0ed5edbe32044d1c83b0590dbb899801 P 43999d813e564dc688fad116090c95ca: Time spent bootstrapping tablet: real 0.534s	user 0.029s	sys 0.007s
I20260709 13:23:49.063457 26118 raft_consensus.cc:359] T 0ed5edbe32044d1c83b0590dbb899801 P 43999d813e564dc688fad116090c95ca [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } }
I20260709 13:23:49.069545 26118 raft_consensus.cc:740] T 0ed5edbe32044d1c83b0590dbb899801 P 43999d813e564dc688fad116090c95ca [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43999d813e564dc688fad116090c95ca, State: Initialized, Role: FOLLOWER
I20260709 13:23:49.072813 26118 consensus_queue.cc:260] T 0ed5edbe32044d1c83b0590dbb899801 P 43999d813e564dc688fad116090c95ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } }
I20260709 13:23:48.991448 25380 raft_consensus.cc:2468] T f1ac2f4d2fe2424bb470df42199219e2 P 3e07d64466934da4bb3c02262b1c2e38 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9868dce521294d0b816bba7648f4ae5a in term 2.
I20260709 13:23:49.083106 26118 ts_tablet_manager.cc:1434] T 0ed5edbe32044d1c83b0590dbb899801 P 43999d813e564dc688fad116090c95ca: Time spent starting tablet: real 0.023s	user 0.008s	sys 0.001s
I20260709 13:23:49.095865 26155 leader_election.cc:290] T b7ff8dde9f5d4c49b9203d902f6b94c4 P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 43999d813e564dc688fad116090c95ca (127.14.177.129:38155), 233e7d29aac04505bf4e65f0a39a0c4c (127.14.177.130:42957)
I20260709 13:23:49.097631 25456 raft_consensus.cc:1275] T 9ab90e3e08fb4528b2a47d995e9e26d3 P 9868dce521294d0b816bba7648f4ae5a [term 2 FOLLOWER]: Refusing update from remote peer 3e07d64466934da4bb3c02262b1c2e38: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:23:49.156231 26118 tablet_bootstrap.cc:492] T b7a891bfc89e4e259f53da6cb16f4989 P 43999d813e564dc688fad116090c95ca: Bootstrap starting.
W20260709 13:23:49.260143 25274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41539, user_credentials={real_user=slave}} blocked reactor thread for 207840us
W20260709 13:23:49.269694 25424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.129:38155, user_credentials={real_user=slave}} blocked reactor thread for 88342.7us
W20260709 13:23:49.271809 25424 leader_election.cc:336] T 579da2d6aabb4ae9a2b1802e1358068b P 9868dce521294d0b816bba7648f4ae5a [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 43999d813e564dc688fad116090c95ca (127.14.177.129:38155): Network error: Client connection negotiation failed: client connection to 127.14.177.129:38155: connect: Connection refused (error 111)
W20260709 13:23:49.284466 25424 consensus_peers.cc:597] T a5df750b84e0487fa530161079deb6c2 P 9868dce521294d0b816bba7648f4ae5a -> Peer 43999d813e564dc688fad116090c95ca (127.14.177.129:38155): Couldn't send request to peer 43999d813e564dc688fad116090c95ca. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:38155: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:23:49.318110 25307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7ff8dde9f5d4c49b9203d902f6b94c4" candidate_uuid: "3e07d64466934da4bb3c02262b1c2e38" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" is_pre_election: true
I20260709 13:23:49.319813 25307 raft_consensus.cc:2468] T b7ff8dde9f5d4c49b9203d902f6b94c4 P 233e7d29aac04505bf4e65f0a39a0c4c [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3e07d64466934da4bb3c02262b1c2e38 in term 2.
I20260709 13:23:49.327788 26234 raft_consensus.cc:493] T 0cb89d70b4ce4aa8992fb68480bec8a2 P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:23:49.329030 26234 raft_consensus.cc:515] T 0cb89d70b4ce4aa8992fb68480bec8a2 P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } }
I20260709 13:23:49.335856 25383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cb89d70b4ce4aa8992fb68480bec8a2" candidate_uuid: "43999d813e564dc688fad116090c95ca" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3e07d64466934da4bb3c02262b1c2e38" is_pre_election: true
I20260709 13:23:49.336565 26234 leader_election.cc:290] T 0cb89d70b4ce4aa8992fb68480bec8a2 P 43999d813e564dc688fad116090c95ca [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 233e7d29aac04505bf4e65f0a39a0c4c (127.14.177.130:42957), 3e07d64466934da4bb3c02262b1c2e38 (127.14.177.131:41539)
I20260709 13:23:49.348151 25307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cb89d70b4ce4aa8992fb68480bec8a2" candidate_uuid: "43999d813e564dc688fad116090c95ca" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" is_pre_election: true
I20260709 13:23:49.351423 25305 raft_consensus.cc:1275] T 9ab90e3e08fb4528b2a47d995e9e26d3 P 233e7d29aac04505bf4e65f0a39a0c4c [term 2 FOLLOWER]: Refusing update from remote peer 3e07d64466934da4bb3c02262b1c2e38: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 13:23:49.395018 26274 log.cc:927] Time spent T b155b866f10547fd8239579a81ecf5f9 P 233e7d29aac04505bf4e65f0a39a0c4c: Append to log took a long time: real 0.438s	user 0.020s	sys 0.020s
W20260709 13:23:49.418012 25424 leader_election.cc:336] T f1ac2f4d2fe2424bb470df42199219e2 P 9868dce521294d0b816bba7648f4ae5a [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 43999d813e564dc688fad116090c95ca (127.14.177.129:38155): Network error: Client connection negotiation failed: client connection to 127.14.177.129:38155: connect: Connection refused (error 111)
I20260709 13:23:49.420239 26111 leader_election.cc:304] T 0cb89d70b4ce4aa8992fb68480bec8a2 P 43999d813e564dc688fad116090c95ca [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 43999d813e564dc688fad116090c95ca; no voters: 233e7d29aac04505bf4e65f0a39a0c4c, 3e07d64466934da4bb3c02262b1c2e38
I20260709 13:23:49.435814 26234 raft_consensus.cc:2749] T 0cb89d70b4ce4aa8992fb68480bec8a2 P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:23:49.425731 25276 consensus_peers.cc:597] T 434a54fcee7d4aeeab42f739855c0e8a P 233e7d29aac04505bf4e65f0a39a0c4c -> Peer 43999d813e564dc688fad116090c95ca (127.14.177.129:38155): Couldn't send request to peer 43999d813e564dc688fad116090c95ca. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:38155: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:23:49.426856 26295 log.cc:927] Time spent T 579da2d6aabb4ae9a2b1802e1358068b P 9868dce521294d0b816bba7648f4ae5a: Append to log took a long time: real 0.202s	user 0.001s	sys 0.000s
W20260709 13:23:49.432054 25350 consensus_peers.cc:597] T b155b866f10547fd8239579a81ecf5f9 P 3e07d64466934da4bb3c02262b1c2e38 -> Peer 43999d813e564dc688fad116090c95ca (127.14.177.129:38155): Couldn't send request to peer 43999d813e564dc688fad116090c95ca. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:38155: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:23:49.433073 26046 consensus_queue.cc:1048] T 9ab90e3e08fb4528b2a47d995e9e26d3 P 3e07d64466934da4bb3c02262b1c2e38 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 13:23:49.442461 25424 consensus_peers.cc:597] T 5c7e7acbb09f4d49828fa652ca6abaac P 9868dce521294d0b816bba7648f4ae5a -> Peer 43999d813e564dc688fad116090c95ca (127.14.177.129:38155): Couldn't send request to peer 43999d813e564dc688fad116090c95ca. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:38155: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:23:49.469278 25350 consensus_peers.cc:597] T 821a5866429f4173820fb64336abee02 P 3e07d64466934da4bb3c02262b1c2e38 -> Peer 43999d813e564dc688fad116090c95ca (127.14.177.129:38155): Couldn't send request to peer 43999d813e564dc688fad116090c95ca. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:38155: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:23:49.469734 26181 consensus_queue.cc:1048] T 5c7e7acbb09f4d49828fa652ca6abaac P 9868dce521294d0b816bba7648f4ae5a [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 13:23:49.488590 25424 consensus_peers.cc:597] T 579da2d6aabb4ae9a2b1802e1358068b P 9868dce521294d0b816bba7648f4ae5a -> Peer 43999d813e564dc688fad116090c95ca (127.14.177.129:38155): Couldn't send request to peer 43999d813e564dc688fad116090c95ca. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:38155: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:23:49.538316 26229 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.649s	user 0.009s	sys 0.017s
W20260709 13:23:49.539093 26229 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.668s	user 0.010s	sys 0.017s
I20260709 13:23:49.551733 25134 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:35154 (request call id 95) took 1210 ms. Trace:
W20260709 13:23:49.563445 25424 consensus_peers.cc:597] T 280d65245f4f4358b54bef93e18aca92 P 9868dce521294d0b816bba7648f4ae5a -> Peer 43999d813e564dc688fad116090c95ca (127.14.177.129:38155): Couldn't send request to peer 43999d813e564dc688fad116090c95ca. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:38155: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:23:49.552492 25134 rpcz_store.cc:276] 0709 13:23:48.341073 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:23:48.349446 (+  8373us) service_pool.cc:224] Handling call
0709 13:23:48.555938 (+206492us) spinlock_profiling.cc:243] Waited 79.4 ms on lock 0x7b54007e0754. stack: 00007f69ada40343 00007f69ada40240 00007f69ad3cf1d4 00007f69b710d304 00007f69b710d299 00007f69b0ac9090 00007f69b0b67f3a 00007f69b691e1cc 00007f69b6a1c37e 00007f69b4edd2a4 00007f69b4edd232 00007f69b4edd161 00007f69b4edd0dc 00007f69b4edc392 00007f69aee9a48c 00007f69aee998c6
0709 13:23:49.551418 (+995480us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:23:49.011600 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:23:49.012414 (+   814us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:23:49.012465 (+    51us) write_op.cc:435] Acquired schema lock
0709 13:23:49.012498 (+    33us) tablet.cc:662] Decoding operations
0709 13:23:49.013587 (+  1089us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:23:49.013673 (+    86us) write_op.cc:641] Partition lock acquired for write op
0709 13:23:49.013717 (+    44us) tablet.cc:685] Acquiring locks for 6 operations
0709 13:23:49.014497 (+   780us) tablet.cc:701] Row locks acquired
0709 13:23:49.014525 (+    28us) write_op.cc:260] PREPARE: finished
0709 13:23:49.014746 (+   221us) write_op.cc:270] Start()
0709 13:23:49.014904 (+   158us) write_op.cc:276] Timestamp: P: 1783603429014683 usec, L: 0
0709 13:23:49.014935 (+    31us) op_driver.cc:348] REPLICATION: starting
0709 13:23:49.015846 (+   911us) log.cc:844] Serialized 2499 byte log entry
0709 13:23:49.018899 (+  3053us) op_driver.cc:464] REPLICATION: finished
0709 13:23:49.019491 (+   592us) write_op.cc:301] APPLY: starting
0709 13:23:49.019593 (+   102us) tablet.cc:1367] starting BulkCheckPresence
0709 13:23:49.019839 (+   246us) tablet.cc:1370] finished BulkCheckPresence
0709 13:23:49.019971 (+   132us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:23:49.020790 (+   819us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:23:49.021071 (+   281us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=6
0709 13:23:49.021151 (+    80us) write_op.cc:312] APPLY: finished
0709 13:23:49.021492 (+   341us) log.cc:844] Serialized 52 byte log entry
0709 13:23:49.021706 (+   214us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:23:49.021853 (+   147us) write_op.cc:454] Released schema lock
0709 13:23:49.021920 (+    67us) write_op.cc:341] FINISH: Updating metrics
0709 13:23:49.023333 (+  1413us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:23:49.023382 (+    49us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":55,"reported_tablets":10,"spinlock_wait_cycles":182594432,"child_traces":[["op",{"apply.queue_time_us":360,"apply.run_cpu_time_us":2797,"apply.run_wall_time_us":2795,"num_ops":6,"prepare.queue_time_us":166,"prepare.run_cpu_time_us":4648,"prepare.run_wall_time_us":11715,"replication_time_us":3805}]]}
I20260709 13:23:49.575810 25351 leader_election.cc:304] T b7ff8dde9f5d4c49b9203d902f6b94c4 P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 233e7d29aac04505bf4e65f0a39a0c4c, 3e07d64466934da4bb3c02262b1c2e38; no voters: 
W20260709 13:23:49.580782 25351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:42957, user_credentials={real_user=slave}} blocked reactor thread for 52680.1us
I20260709 13:23:49.574271 25304 raft_consensus.cc:1275] T 579da2d6aabb4ae9a2b1802e1358068b P 233e7d29aac04505bf4e65f0a39a0c4c [term 3 FOLLOWER]: Refusing update from remote peer 9868dce521294d0b816bba7648f4ae5a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:23:49.584442 26180 raft_consensus.cc:2804] T b7ff8dde9f5d4c49b9203d902f6b94c4 P 3e07d64466934da4bb3c02262b1c2e38 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:23:49.585007 26180 raft_consensus.cc:493] T b7ff8dde9f5d4c49b9203d902f6b94c4 P 3e07d64466934da4bb3c02262b1c2e38 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:23:49.585386 26180 raft_consensus.cc:3060] T b7ff8dde9f5d4c49b9203d902f6b94c4 P 3e07d64466934da4bb3c02262b1c2e38 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:23:49.586850 26213 consensus_queue.cc:1048] T 9ab90e3e08fb4528b2a47d995e9e26d3 P 3e07d64466934da4bb3c02262b1c2e38 [LEADER]: Connected to new peer: Peer: permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:23:49.600143 25422 leader_election.cc:304] T f1ac2f4d2fe2424bb470df42199219e2 P 9868dce521294d0b816bba7648f4ae5a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3e07d64466934da4bb3c02262b1c2e38, 9868dce521294d0b816bba7648f4ae5a; no voters: 43999d813e564dc688fad116090c95ca
I20260709 13:23:49.605746 26220 raft_consensus.cc:2804] T f1ac2f4d2fe2424bb470df42199219e2 P 9868dce521294d0b816bba7648f4ae5a [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:23:49.606365 26220 raft_consensus.cc:493] T f1ac2f4d2fe2424bb470df42199219e2 P 9868dce521294d0b816bba7648f4ae5a [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:23:49.606846 26220 raft_consensus.cc:3060] T f1ac2f4d2fe2424bb470df42199219e2 P 9868dce521294d0b816bba7648f4ae5a [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:23:49.626741 25424 consensus_peers.cc:597] T 9e5d70664720424d951f269d9d7f4346 P 9868dce521294d0b816bba7648f4ae5a -> Peer 43999d813e564dc688fad116090c95ca (127.14.177.129:38155): Couldn't send request to peer 43999d813e564dc688fad116090c95ca. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:38155: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:23:49.645112 26208 consensus_queue.cc:1048] T 579da2d6aabb4ae9a2b1802e1358068b P 9868dce521294d0b816bba7648f4ae5a [LEADER]: Connected to new peer: Peer: permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:23:49.689045 26118 tablet_bootstrap.cc:492] T b7a891bfc89e4e259f53da6cb16f4989 P 43999d813e564dc688fad116090c95ca: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:23:49.690270 26118 tablet_bootstrap.cc:492] T b7a891bfc89e4e259f53da6cb16f4989 P 43999d813e564dc688fad116090c95ca: Bootstrap complete.
I20260709 13:23:49.692669 25130 catalog_manager.cc:5697] T b155b866f10547fd8239579a81ecf5f9 P 3e07d64466934da4bb3c02262b1c2e38 reported cstate change: term changed from 1 to 3, leader changed from 43999d813e564dc688fad116090c95ca (127.14.177.129) to 3e07d64466934da4bb3c02262b1c2e38 (127.14.177.131). New cstate: current_term: 3 leader_uuid: "3e07d64466934da4bb3c02262b1c2e38" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } health_report { overall_health: HEALTHY } } }
I20260709 13:23:49.719584 25130 catalog_manager.cc:5697] T 9ab90e3e08fb4528b2a47d995e9e26d3 P 3e07d64466934da4bb3c02262b1c2e38 reported cstate change: term changed from 1 to 2, leader changed from 9868dce521294d0b816bba7648f4ae5a (127.14.177.132) to 3e07d64466934da4bb3c02262b1c2e38 (127.14.177.131). New cstate: current_term: 2 leader_uuid: "3e07d64466934da4bb3c02262b1c2e38" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } health_report { overall_health: UNKNOWN } } }
I20260709 13:23:49.739749 26118 ts_tablet_manager.cc:1403] T b7a891bfc89e4e259f53da6cb16f4989 P 43999d813e564dc688fad116090c95ca: Time spent bootstrapping tablet: real 0.584s	user 0.019s	sys 0.046s
W20260709 13:23:49.763375 25350 leader_election.cc:336] T b7ff8dde9f5d4c49b9203d902f6b94c4 P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 43999d813e564dc688fad116090c95ca (127.14.177.129:38155): Network error: Client connection negotiation failed: client connection to 127.14.177.129:38155: connect: Connection refused (error 111)
I20260709 13:23:49.731398 26180 raft_consensus.cc:515] T b7ff8dde9f5d4c49b9203d902f6b94c4 P 3e07d64466934da4bb3c02262b1c2e38 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } }
W20260709 13:23:49.811383 25424 consensus_peers.cc:597] T 6242888a4d4146dcacd87d0fd9a832f4 P 9868dce521294d0b816bba7648f4ae5a -> Peer 43999d813e564dc688fad116090c95ca (127.14.177.129:38155): Couldn't send request to peer 43999d813e564dc688fad116090c95ca. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:38155: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:23:49.790166 26118 raft_consensus.cc:359] T b7a891bfc89e4e259f53da6cb16f4989 P 43999d813e564dc688fad116090c95ca [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } }
I20260709 13:23:49.821157 26118 raft_consensus.cc:740] T b7a891bfc89e4e259f53da6cb16f4989 P 43999d813e564dc688fad116090c95ca [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43999d813e564dc688fad116090c95ca, State: Initialized, Role: FOLLOWER
I20260709 13:23:49.819129 26180 leader_election.cc:290] T b7ff8dde9f5d4c49b9203d902f6b94c4 P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 3 election: Requested vote from peers 43999d813e564dc688fad116090c95ca (127.14.177.129:38155), 233e7d29aac04505bf4e65f0a39a0c4c (127.14.177.130:42957)
I20260709 13:23:49.822691 26234 raft_consensus.cc:493] T b2f14aed05e24a98ad83535029b795dc P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:23:49.850425 25302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7ff8dde9f5d4c49b9203d902f6b94c4" candidate_uuid: "3e07d64466934da4bb3c02262b1c2e38" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "233e7d29aac04505bf4e65f0a39a0c4c"
I20260709 13:23:49.851286 25302 raft_consensus.cc:3060] T b7ff8dde9f5d4c49b9203d902f6b94c4 P 233e7d29aac04505bf4e65f0a39a0c4c [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:23:49.848029 26118 consensus_queue.cc:260] T b7a891bfc89e4e259f53da6cb16f4989 P 43999d813e564dc688fad116090c95ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } }
I20260709 13:23:49.820926 26220 raft_consensus.cc:515] T f1ac2f4d2fe2424bb470df42199219e2 P 9868dce521294d0b816bba7648f4ae5a [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } }
I20260709 13:23:49.854085 26234 raft_consensus.cc:515] T b2f14aed05e24a98ad83535029b795dc P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } }
I20260709 13:23:49.855583 26220 leader_election.cc:290] T f1ac2f4d2fe2424bb470df42199219e2 P 9868dce521294d0b816bba7648f4ae5a [CANDIDATE]: Term 3 election: Requested vote from peers 3e07d64466934da4bb3c02262b1c2e38 (127.14.177.131:41539), 43999d813e564dc688fad116090c95ca (127.14.177.129:38155)
W20260709 13:23:49.859923 25422 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41539, user_credentials={real_user=slave}} blocked reactor thread for 70429.6us
I20260709 13:23:49.885103 26234 leader_election.cc:290] T b2f14aed05e24a98ad83535029b795dc P 43999d813e564dc688fad116090c95ca [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 233e7d29aac04505bf4e65f0a39a0c4c (127.14.177.130:42957), 3e07d64466934da4bb3c02262b1c2e38 (127.14.177.131:41539)
I20260709 13:23:49.911963 25378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2f14aed05e24a98ad83535029b795dc" candidate_uuid: "43999d813e564dc688fad116090c95ca" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3e07d64466934da4bb3c02262b1c2e38" is_pre_election: true
I20260709 13:23:49.917258 26234 raft_consensus.cc:493] T e15820fa8a7d4e1fb6ad23a64ca22f90 P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:23:49.909578 25424 leader_election.cc:336] T f1ac2f4d2fe2424bb470df42199219e2 P 9868dce521294d0b816bba7648f4ae5a [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 43999d813e564dc688fad116090c95ca (127.14.177.129:38155): Network error: Client connection negotiation failed: client connection to 127.14.177.129:38155: connect: Connection refused (error 111)
I20260709 13:23:49.922178 26234 raft_consensus.cc:515] T e15820fa8a7d4e1fb6ad23a64ca22f90 P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } }
W20260709 13:23:49.919557 25424 consensus_peers.cc:597] T 0c4f6b7401db47d59a4f34ed0e8b5350 P 9868dce521294d0b816bba7648f4ae5a -> Peer 43999d813e564dc688fad116090c95ca (127.14.177.129:38155): Couldn't send request to peer 43999d813e564dc688fad116090c95ca. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:38155: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:23:49.931713 25383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1ac2f4d2fe2424bb470df42199219e2" candidate_uuid: "9868dce521294d0b816bba7648f4ae5a" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3e07d64466934da4bb3c02262b1c2e38"
I20260709 13:23:49.932576 25383 raft_consensus.cc:3060] T f1ac2f4d2fe2424bb470df42199219e2 P 3e07d64466934da4bb3c02262b1c2e38 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:23:49.934451 25385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15820fa8a7d4e1fb6ad23a64ca22f90" candidate_uuid: "43999d813e564dc688fad116090c95ca" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3e07d64466934da4bb3c02262b1c2e38" is_pre_election: true
I20260709 13:23:49.933166 26234 leader_election.cc:290] T e15820fa8a7d4e1fb6ad23a64ca22f90 P 43999d813e564dc688fad116090c95ca [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 233e7d29aac04505bf4e65f0a39a0c4c (127.14.177.130:42957), 3e07d64466934da4bb3c02262b1c2e38 (127.14.177.131:41539)
I20260709 13:23:49.937429 25302 raft_consensus.cc:2468] T b7ff8dde9f5d4c49b9203d902f6b94c4 P 233e7d29aac04505bf4e65f0a39a0c4c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3e07d64466934da4bb3c02262b1c2e38 in term 3.
W20260709 13:23:49.941606 25350 leader_election.cc:336] T b7ff8dde9f5d4c49b9203d902f6b94c4 P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 43999d813e564dc688fad116090c95ca (127.14.177.129:38155): Network error: Client connection negotiation failed: client connection to 127.14.177.129:38155: connect: Connection refused (error 111)
W20260709 13:23:49.951473 26324 log.cc:927] Time spent T b9d550d881b04735bbf89751952c09ec P 9868dce521294d0b816bba7648f4ae5a: Append to log took a long time: real 0.056s	user 0.001s	sys 0.000s
I20260709 13:23:49.957546 25308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2f14aed05e24a98ad83535029b795dc" candidate_uuid: "43999d813e564dc688fad116090c95ca" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" is_pre_election: true
I20260709 13:23:49.959451 25351 leader_election.cc:304] T b7ff8dde9f5d4c49b9203d902f6b94c4 P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 233e7d29aac04505bf4e65f0a39a0c4c, 3e07d64466934da4bb3c02262b1c2e38; no voters: 43999d813e564dc688fad116090c95ca
I20260709 13:23:49.977070 25308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15820fa8a7d4e1fb6ad23a64ca22f90" candidate_uuid: "43999d813e564dc688fad116090c95ca" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" is_pre_election: true
I20260709 13:23:49.990116 26199 raft_consensus.cc:2804] T b7ff8dde9f5d4c49b9203d902f6b94c4 P 3e07d64466934da4bb3c02262b1c2e38 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:23:49.990772 26199 raft_consensus.cc:697] T b7ff8dde9f5d4c49b9203d902f6b94c4 P 3e07d64466934da4bb3c02262b1c2e38 [term 3 LEADER]: Becoming Leader. State: Replica: 3e07d64466934da4bb3c02262b1c2e38, State: Running, Role: LEADER
I20260709 13:23:49.998283 25383 raft_consensus.cc:2468] T f1ac2f4d2fe2424bb470df42199219e2 P 3e07d64466934da4bb3c02262b1c2e38 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9868dce521294d0b816bba7648f4ae5a in term 3.
I20260709 13:23:50.000928 26114 leader_election.cc:304] T b2f14aed05e24a98ad83535029b795dc P 43999d813e564dc688fad116090c95ca [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 43999d813e564dc688fad116090c95ca; no voters: 233e7d29aac04505bf4e65f0a39a0c4c, 3e07d64466934da4bb3c02262b1c2e38
I20260709 13:23:50.003103 26330 raft_consensus.cc:2749] T b2f14aed05e24a98ad83535029b795dc P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:23:50.004743 26199 consensus_queue.cc:237] T b7ff8dde9f5d4c49b9203d902f6b94c4 P 3e07d64466934da4bb3c02262b1c2e38 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } }
I20260709 13:23:50.023365 26114 leader_election.cc:304] T e15820fa8a7d4e1fb6ad23a64ca22f90 P 43999d813e564dc688fad116090c95ca [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 43999d813e564dc688fad116090c95ca; no voters: 233e7d29aac04505bf4e65f0a39a0c4c, 3e07d64466934da4bb3c02262b1c2e38
I20260709 13:23:50.025135 26330 raft_consensus.cc:2749] T e15820fa8a7d4e1fb6ad23a64ca22f90 P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:23:50.025308 26118 ts_tablet_manager.cc:1434] T b7a891bfc89e4e259f53da6cb16f4989 P 43999d813e564dc688fad116090c95ca: Time spent starting tablet: real 0.285s	user 0.014s	sys 0.020s
I20260709 13:23:50.026965 26118 tablet_bootstrap.cc:492] T 434a54fcee7d4aeeab42f739855c0e8a P 43999d813e564dc688fad116090c95ca: Bootstrap starting.
I20260709 13:23:50.029431 25422 leader_election.cc:304] T f1ac2f4d2fe2424bb470df42199219e2 P 9868dce521294d0b816bba7648f4ae5a [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3e07d64466934da4bb3c02262b1c2e38, 9868dce521294d0b816bba7648f4ae5a; no voters: 43999d813e564dc688fad116090c95ca
I20260709 13:23:50.039352 26184 raft_consensus.cc:2804] T f1ac2f4d2fe2424bb470df42199219e2 P 9868dce521294d0b816bba7648f4ae5a [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:23:50.039937 26184 raft_consensus.cc:697] T f1ac2f4d2fe2424bb470df42199219e2 P 9868dce521294d0b816bba7648f4ae5a [term 3 LEADER]: Becoming Leader. State: Replica: 9868dce521294d0b816bba7648f4ae5a, State: Running, Role: LEADER
I20260709 13:23:50.046144 26184 consensus_queue.cc:237] T f1ac2f4d2fe2424bb470df42199219e2 P 9868dce521294d0b816bba7648f4ae5a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } }
I20260709 13:23:50.167097 26330 raft_consensus.cc:493] T 28c0b6fc9bd64d60828d705c6a9ab1e4 P 43999d813e564dc688fad116090c95ca [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:23:50.167771 26330 raft_consensus.cc:515] T 28c0b6fc9bd64d60828d705c6a9ab1e4 P 43999d813e564dc688fad116090c95ca [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } }
I20260709 13:23:50.174543 26234 raft_consensus.cc:493] T 441d780c3945463f92f31170f31e37a8 P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:23:50.175312 26234 raft_consensus.cc:515] T 441d780c3945463f92f31170f31e37a8 P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } }
I20260709 13:23:50.174626 25311 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28c0b6fc9bd64d60828d705c6a9ab1e4" candidate_uuid: "43999d813e564dc688fad116090c95ca" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" is_pre_election: true
I20260709 13:23:50.178001 26234 leader_election.cc:290] T 441d780c3945463f92f31170f31e37a8 P 43999d813e564dc688fad116090c95ca [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9868dce521294d0b816bba7648f4ae5a (127.14.177.132:33683), 233e7d29aac04505bf4e65f0a39a0c4c (127.14.177.130:42957)
I20260709 13:23:50.180416 25456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "441d780c3945463f92f31170f31e37a8" candidate_uuid: "43999d813e564dc688fad116090c95ca" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9868dce521294d0b816bba7648f4ae5a" is_pre_election: true
I20260709 13:23:50.184736 25311 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "441d780c3945463f92f31170f31e37a8" candidate_uuid: "43999d813e564dc688fad116090c95ca" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" is_pre_election: true
W20260709 13:23:50.183929 25276 consensus_peers.cc:597] T ce98a3f3b3eb4c63bcaa355ec7cd64a4 P 233e7d29aac04505bf4e65f0a39a0c4c -> Peer 43999d813e564dc688fad116090c95ca (127.14.177.129:38155): Couldn't send request to peer 43999d813e564dc688fad116090c95ca. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:38155: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:23:50.210172 26114 leader_election.cc:304] T 441d780c3945463f92f31170f31e37a8 P 43999d813e564dc688fad116090c95ca [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 43999d813e564dc688fad116090c95ca; no voters: 233e7d29aac04505bf4e65f0a39a0c4c, 9868dce521294d0b816bba7648f4ae5a
I20260709 13:23:50.189441 26234 raft_consensus.cc:493] T f1ac2f4d2fe2424bb470df42199219e2 P 43999d813e564dc688fad116090c95ca [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:23:50.212275 26234 raft_consensus.cc:515] T f1ac2f4d2fe2424bb470df42199219e2 P 43999d813e564dc688fad116090c95ca [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } }
I20260709 13:23:50.214548 26234 leader_election.cc:290] T f1ac2f4d2fe2424bb470df42199219e2 P 43999d813e564dc688fad116090c95ca [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3e07d64466934da4bb3c02262b1c2e38 (127.14.177.131:41539), 9868dce521294d0b816bba7648f4ae5a (127.14.177.132:33683)
W20260709 13:23:50.219050 25350 consensus_peers.cc:597] T 03e9bfbd2f774476a4a377367796a2e8 P 3e07d64466934da4bb3c02262b1c2e38 -> Peer 43999d813e564dc688fad116090c95ca (127.14.177.129:38155): Couldn't send request to peer 43999d813e564dc688fad116090c95ca. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:38155: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:23:50.224056 25458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1ac2f4d2fe2424bb470df42199219e2" candidate_uuid: "43999d813e564dc688fad116090c95ca" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9868dce521294d0b816bba7648f4ae5a" is_pre_election: true
I20260709 13:23:50.232300 26330 leader_election.cc:290] T 28c0b6fc9bd64d60828d705c6a9ab1e4 P 43999d813e564dc688fad116090c95ca [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 233e7d29aac04505bf4e65f0a39a0c4c (127.14.177.130:42957), 3e07d64466934da4bb3c02262b1c2e38 (127.14.177.131:41539)
I20260709 13:23:50.244143 26118 tablet_bootstrap.cc:492] T 434a54fcee7d4aeeab42f739855c0e8a P 43999d813e564dc688fad116090c95ca: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:23:50.215051 26234 raft_consensus.cc:2749] T 441d780c3945463f92f31170f31e37a8 P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:23:50.255054 25379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1ac2f4d2fe2424bb470df42199219e2" candidate_uuid: "43999d813e564dc688fad116090c95ca" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3e07d64466934da4bb3c02262b1c2e38" is_pre_election: true
I20260709 13:23:50.256381 25379 raft_consensus.cc:2393] T f1ac2f4d2fe2424bb470df42199219e2 P 3e07d64466934da4bb3c02262b1c2e38 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 43999d813e564dc688fad116090c95ca in current term 3: Already voted for candidate 9868dce521294d0b816bba7648f4ae5a in this term.
I20260709 13:23:50.258313 26111 leader_election.cc:304] T f1ac2f4d2fe2424bb470df42199219e2 P 43999d813e564dc688fad116090c95ca [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 43999d813e564dc688fad116090c95ca; no voters: 3e07d64466934da4bb3c02262b1c2e38, 9868dce521294d0b816bba7648f4ae5a
I20260709 13:23:50.259527 26234 raft_consensus.cc:3060] T f1ac2f4d2fe2424bb470df42199219e2 P 43999d813e564dc688fad116090c95ca [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:23:50.263751 26118 tablet_bootstrap.cc:492] T 434a54fcee7d4aeeab42f739855c0e8a P 43999d813e564dc688fad116090c95ca: Bootstrap complete.
I20260709 13:23:50.264752 26118 ts_tablet_manager.cc:1403] T 434a54fcee7d4aeeab42f739855c0e8a P 43999d813e564dc688fad116090c95ca: Time spent bootstrapping tablet: real 0.238s	user 0.029s	sys 0.009s
I20260709 13:23:50.275377 25379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28c0b6fc9bd64d60828d705c6a9ab1e4" candidate_uuid: "43999d813e564dc688fad116090c95ca" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3e07d64466934da4bb3c02262b1c2e38" is_pre_election: true
I20260709 13:23:50.290310 26111 leader_election.cc:304] T 28c0b6fc9bd64d60828d705c6a9ab1e4 P 43999d813e564dc688fad116090c95ca [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 43999d813e564dc688fad116090c95ca; no voters: 233e7d29aac04505bf4e65f0a39a0c4c, 3e07d64466934da4bb3c02262b1c2e38
I20260709 13:23:50.291231 26330 raft_consensus.cc:2749] T 28c0b6fc9bd64d60828d705c6a9ab1e4 P 43999d813e564dc688fad116090c95ca [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:23:50.294589 25276 consensus_peers.cc:597] T 51a11db644124b63bdf5f457c967473d P 233e7d29aac04505bf4e65f0a39a0c4c -> Peer 43999d813e564dc688fad116090c95ca (127.14.177.129:38155): Couldn't send request to peer 43999d813e564dc688fad116090c95ca. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:38155: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:23:50.272491 26114 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:42957, user_credentials={real_user=slave}} blocked reactor thread for 71910us
I20260709 13:23:50.304292 26118 raft_consensus.cc:359] T 434a54fcee7d4aeeab42f739855c0e8a P 43999d813e564dc688fad116090c95ca [term 3 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } }
W20260709 13:23:50.305320 25424 consensus_peers.cc:597] T fa0da2c845dc4059968332f4db2e0aa6 P 9868dce521294d0b816bba7648f4ae5a -> Peer 43999d813e564dc688fad116090c95ca (127.14.177.129:38155): Couldn't send request to peer 43999d813e564dc688fad116090c95ca. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:38155: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:23:50.306319 26118 raft_consensus.cc:740] T 434a54fcee7d4aeeab42f739855c0e8a P 43999d813e564dc688fad116090c95ca [term 3 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43999d813e564dc688fad116090c95ca, State: Initialized, Role: FOLLOWER
I20260709 13:23:50.313099 26234 raft_consensus.cc:2749] T f1ac2f4d2fe2424bb470df42199219e2 P 43999d813e564dc688fad116090c95ca [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:23:50.312377 26118 consensus_queue.cc:260] T 434a54fcee7d4aeeab42f739855c0e8a P 43999d813e564dc688fad116090c95ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } }
I20260709 13:23:50.318136 26118 ts_tablet_manager.cc:1434] T 434a54fcee7d4aeeab42f739855c0e8a P 43999d813e564dc688fad116090c95ca: Time spent starting tablet: real 0.053s	user 0.006s	sys 0.000s
I20260709 13:23:50.319317 26118 tablet_bootstrap.cc:492] T 03e9bfbd2f774476a4a377367796a2e8 P 43999d813e564dc688fad116090c95ca: Bootstrap starting.
I20260709 13:23:50.428583 26234 raft_consensus.cc:493] T 13bb3a23315a41c7a40e10b954b15c8e P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:23:50.429714 26234 raft_consensus.cc:515] T 13bb3a23315a41c7a40e10b954b15c8e P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } }
I20260709 13:23:50.447283 26234 leader_election.cc:290] T 13bb3a23315a41c7a40e10b954b15c8e P 43999d813e564dc688fad116090c95ca [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 233e7d29aac04505bf4e65f0a39a0c4c (127.14.177.130:42957), 3e07d64466934da4bb3c02262b1c2e38 (127.14.177.131:41539)
I20260709 13:23:50.509370 25132 catalog_manager.cc:5697] T 441d780c3945463f92f31170f31e37a8 P 9868dce521294d0b816bba7648f4ae5a reported cstate change: term changed from 1 to 2, leader changed from 43999d813e564dc688fad116090c95ca (127.14.177.129) to 233e7d29aac04505bf4e65f0a39a0c4c (127.14.177.130). New cstate: current_term: 2 leader_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } }
I20260709 13:23:50.657753 25132 catalog_manager.cc:5697] T ce9c60e3922c4219b6e4453fe0259b6a P 9868dce521294d0b816bba7648f4ae5a reported cstate change: term changed from 1 to 3, leader changed from 43999d813e564dc688fad116090c95ca (127.14.177.129) to 233e7d29aac04505bf4e65f0a39a0c4c (127.14.177.130). New cstate: current_term: 3 leader_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } }
I20260709 13:23:50.667270 25379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13bb3a23315a41c7a40e10b954b15c8e" candidate_uuid: "43999d813e564dc688fad116090c95ca" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3e07d64466934da4bb3c02262b1c2e38" is_pre_election: true
I20260709 13:23:50.721138 25311 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13bb3a23315a41c7a40e10b954b15c8e" candidate_uuid: "43999d813e564dc688fad116090c95ca" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" is_pre_election: true
W20260709 13:23:50.746505 15046 thread.cc:641] rpc worker (service pool kudu.consensus.ConsensusService) Time spent creating pthread: real 0.606s	user 0.013s	sys 0.014s
W20260709 13:23:50.747020 15046 thread.cc:608] rpc worker (service pool kudu.consensus.ConsensusService) Time spent starting thread: real 0.607s	user 0.013s	sys 0.014s
I20260709 13:23:50.790879 26118 tablet_bootstrap.cc:492] T 03e9bfbd2f774476a4a377367796a2e8 P 43999d813e564dc688fad116090c95ca: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:23:50.792006 26118 tablet_bootstrap.cc:492] T 03e9bfbd2f774476a4a377367796a2e8 P 43999d813e564dc688fad116090c95ca: Bootstrap complete.
I20260709 13:23:50.794059 26118 ts_tablet_manager.cc:1403] T 03e9bfbd2f774476a4a377367796a2e8 P 43999d813e564dc688fad116090c95ca: Time spent bootstrapping tablet: real 0.475s	user 0.027s	sys 0.009s
W20260709 13:23:50.800625 25350 consensus_peers.cc:597] T 0ed5edbe32044d1c83b0590dbb899801 P 3e07d64466934da4bb3c02262b1c2e38 -> Peer 43999d813e564dc688fad116090c95ca (127.14.177.129:38155): Couldn't send request to peer 43999d813e564dc688fad116090c95ca. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:38155: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:23:50.801198 25350 consensus_peers.cc:597] T b2f14aed05e24a98ad83535029b795dc P 3e07d64466934da4bb3c02262b1c2e38 -> Peer 43999d813e564dc688fad116090c95ca (127.14.177.129:38155): Couldn't send request to peer 43999d813e564dc688fad116090c95ca. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:38155: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:23:50.801898 25350 consensus_peers.cc:597] T 79d4a12ef58140568c60a24ed4c12f83 P 3e07d64466934da4bb3c02262b1c2e38 -> Peer 43999d813e564dc688fad116090c95ca (127.14.177.129:38155): Couldn't send request to peer 43999d813e564dc688fad116090c95ca. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:38155: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:23:50.802286 25350 consensus_peers.cc:597] T b7ff8dde9f5d4c49b9203d902f6b94c4 P 3e07d64466934da4bb3c02262b1c2e38 -> Peer 43999d813e564dc688fad116090c95ca (127.14.177.129:38155): Couldn't send request to peer 43999d813e564dc688fad116090c95ca. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:38155: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:23:50.802659 25350 consensus_peers.cc:597] T 0cb89d70b4ce4aa8992fb68480bec8a2 P 3e07d64466934da4bb3c02262b1c2e38 -> Peer 43999d813e564dc688fad116090c95ca (127.14.177.129:38155): Couldn't send request to peer 43999d813e564dc688fad116090c95ca. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:38155: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:23:50.804270 25350 consensus_peers.cc:597] T b7a891bfc89e4e259f53da6cb16f4989 P 3e07d64466934da4bb3c02262b1c2e38 -> Peer 43999d813e564dc688fad116090c95ca (127.14.177.129:38155): Couldn't send request to peer 43999d813e564dc688fad116090c95ca. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:38155: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:23:50.860906 26118 raft_consensus.cc:359] T 03e9bfbd2f774476a4a377367796a2e8 P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } }
I20260709 13:23:50.861763 26118 raft_consensus.cc:740] T 03e9bfbd2f774476a4a377367796a2e8 P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43999d813e564dc688fad116090c95ca, State: Initialized, Role: FOLLOWER
I20260709 13:23:50.862509 26118 consensus_queue.cc:260] T 03e9bfbd2f774476a4a377367796a2e8 P 43999d813e564dc688fad116090c95ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } }
I20260709 13:23:50.863415 26114 leader_election.cc:304] T 13bb3a23315a41c7a40e10b954b15c8e P 43999d813e564dc688fad116090c95ca [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 43999d813e564dc688fad116090c95ca; no voters: 233e7d29aac04505bf4e65f0a39a0c4c, 3e07d64466934da4bb3c02262b1c2e38
I20260709 13:23:50.864624 26234 raft_consensus.cc:2749] T 13bb3a23315a41c7a40e10b954b15c8e P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:23:50.869529 26118 ts_tablet_manager.cc:1434] T 03e9bfbd2f774476a4a377367796a2e8 P 43999d813e564dc688fad116090c95ca: Time spent starting tablet: real 0.075s	user 0.006s	sys 0.000s
I20260709 13:23:50.870386 26118 tablet_bootstrap.cc:492] T a5df750b84e0487fa530161079deb6c2 P 43999d813e564dc688fad116090c95ca: Bootstrap starting.
I20260709 13:23:50.878260 25132 catalog_manager.cc:5697] T 579da2d6aabb4ae9a2b1802e1358068b P 9868dce521294d0b816bba7648f4ae5a reported cstate change: term changed from 1 to 3, leader changed from 43999d813e564dc688fad116090c95ca (127.14.177.129) to 9868dce521294d0b816bba7648f4ae5a (127.14.177.132). New cstate: current_term: 3 leader_uuid: "9868dce521294d0b816bba7648f4ae5a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } health_report { overall_health: UNKNOWN } } }
W20260709 13:23:50.897707 25350 consensus_peers.cc:597] T 4d837697cd0f402cacf9db89a5d92652 P 3e07d64466934da4bb3c02262b1c2e38 -> Peer 43999d813e564dc688fad116090c95ca (127.14.177.129:38155): Couldn't send request to peer 43999d813e564dc688fad116090c95ca. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:38155: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:23:50.898286 25350 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.129:38155, user_credentials={real_user=slave}} blocked reactor thread for 60817.8us
W20260709 13:23:50.911319 25350 consensus_peers.cc:597] T e15820fa8a7d4e1fb6ad23a64ca22f90 P 3e07d64466934da4bb3c02262b1c2e38 -> Peer 43999d813e564dc688fad116090c95ca (127.14.177.129:38155): Couldn't send request to peer 43999d813e564dc688fad116090c95ca. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:38155: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:23:50.914141 25130 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:35142 (request call id 107) took 1271 ms. Trace:
I20260709 13:23:50.925727 26118 tablet_bootstrap.cc:492] T a5df750b84e0487fa530161079deb6c2 P 43999d813e564dc688fad116090c95ca: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:23:50.920310 25130 rpcz_store.cc:276] 0709 13:23:49.642611 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:23:49.642842 (+   231us) service_pool.cc:224] Handling call
0709 13:23:50.914072 (+1271230us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:23:49.922401 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:23:49.923127 (+   726us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:23:49.926410 (+  3283us) write_op.cc:435] Acquired schema lock
0709 13:23:49.926902 (+   492us) tablet.cc:662] Decoding operations
0709 13:23:49.952897 (+ 25995us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:23:49.952996 (+    99us) write_op.cc:641] Partition lock acquired for write op
0709 13:23:49.953036 (+    40us) tablet.cc:685] Acquiring locks for 2 operations
0709 13:23:49.953534 (+   498us) tablet.cc:701] Row locks acquired
0709 13:23:49.953580 (+    46us) write_op.cc:260] PREPARE: finished
0709 13:23:49.953809 (+   229us) write_op.cc:270] Start()
0709 13:23:49.953987 (+   178us) write_op.cc:276] Timestamp: P: 1783603429953729 usec, L: 0
0709 13:23:49.954034 (+    47us) op_driver.cc:348] REPLICATION: starting
0709 13:23:49.954824 (+   790us) log.cc:844] Serialized 934 byte log entry
0709 13:23:50.078009 (+123185us) op_driver.cc:464] REPLICATION: finished
0709 13:23:50.181695 (+103686us) write_op.cc:301] APPLY: starting
0709 13:23:50.181779 (+    84us) tablet.cc:1367] starting BulkCheckPresence
0709 13:23:50.181914 (+   135us) tablet.cc:1370] finished BulkCheckPresence
0709 13:23:50.182062 (+   148us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:23:50.182630 (+   568us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:23:50.183813 (+  1183us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0709 13:23:50.189514 (+  5701us) write_op.cc:312] APPLY: finished
0709 13:23:50.211442 (+ 21928us) log.cc:844] Serialized 28 byte log entry
0709 13:23:50.211672 (+   230us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:23:50.211782 (+   110us) write_op.cc:454] Released schema lock
0709 13:23:50.211837 (+    55us) write_op.cc:341] FINISH: Updating metrics
0709 13:23:50.212340 (+   503us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:23:50.212373 (+    33us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":149051,"reported_tablets":9,"child_traces":[["op",{"apply.queue_time_us":103381,"apply.run_cpu_time_us":2646,"apply.run_wall_time_us":30621,"num_ops":2,"prepare.queue_time_us":191107,"prepare.run_cpu_time_us":3932,"prepare.run_wall_time_us":32908,"replication_time_us":123808,"thread_start_us":110941,"threads_started":2}]]}
I20260709 13:23:50.932917 26234 raft_consensus.cc:493] T 0ed5edbe32044d1c83b0590dbb899801 P 43999d813e564dc688fad116090c95ca [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:23:50.939227 25305 raft_consensus.cc:1275] T b7ff8dde9f5d4c49b9203d902f6b94c4 P 233e7d29aac04505bf4e65f0a39a0c4c [term 3 FOLLOWER]: Refusing update from remote peer 3e07d64466934da4bb3c02262b1c2e38: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:23:50.938916 26234 raft_consensus.cc:515] T 0ed5edbe32044d1c83b0590dbb899801 P 43999d813e564dc688fad116090c95ca [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } }
I20260709 13:23:50.946265 26199 consensus_queue.cc:1048] T b7ff8dde9f5d4c49b9203d902f6b94c4 P 3e07d64466934da4bb3c02262b1c2e38 [LEADER]: Connected to new peer: Peer: permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:23:50.947032 26234 leader_election.cc:290] T 0ed5edbe32044d1c83b0590dbb899801 P 43999d813e564dc688fad116090c95ca [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3e07d64466934da4bb3c02262b1c2e38 (127.14.177.131:41539), 9868dce521294d0b816bba7648f4ae5a (127.14.177.132:33683)
I20260709 13:23:50.948172 26118 tablet_bootstrap.cc:492] T a5df750b84e0487fa530161079deb6c2 P 43999d813e564dc688fad116090c95ca: Bootstrap complete.
I20260709 13:23:50.948709 25382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ed5edbe32044d1c83b0590dbb899801" candidate_uuid: "43999d813e564dc688fad116090c95ca" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3e07d64466934da4bb3c02262b1c2e38" is_pre_election: true
I20260709 13:23:50.948951 26118 ts_tablet_manager.cc:1403] T a5df750b84e0487fa530161079deb6c2 P 43999d813e564dc688fad116090c95ca: Time spent bootstrapping tablet: real 0.079s	user 0.021s	sys 0.004s
I20260709 13:23:50.959628 25377 raft_consensus.cc:1275] T f1ac2f4d2fe2424bb470df42199219e2 P 3e07d64466934da4bb3c02262b1c2e38 [term 3 FOLLOWER]: Refusing update from remote peer 9868dce521294d0b816bba7648f4ae5a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:23:50.960058 26118 raft_consensus.cc:359] T a5df750b84e0487fa530161079deb6c2 P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } }
W20260709 13:23:50.954244 25424 consensus_peers.cc:597] T 11949f56648c47e0a43c3140af99a304 P 9868dce521294d0b816bba7648f4ae5a -> Peer 43999d813e564dc688fad116090c95ca (127.14.177.129:38155): Couldn't send request to peer 43999d813e564dc688fad116090c95ca. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:38155: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:23:50.963713 26118 raft_consensus.cc:740] T a5df750b84e0487fa530161079deb6c2 P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43999d813e564dc688fad116090c95ca, State: Initialized, Role: FOLLOWER
W20260709 13:23:50.972365 25424 consensus_peers.cc:597] T b9d550d881b04735bbf89751952c09ec P 9868dce521294d0b816bba7648f4ae5a -> Peer 43999d813e564dc688fad116090c95ca (127.14.177.129:38155): Couldn't send request to peer 43999d813e564dc688fad116090c95ca. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:38155: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:23:50.973353 25424 consensus_peers.cc:597] T d053b9ae38de4173a0aded4da494bd98 P 9868dce521294d0b816bba7648f4ae5a -> Peer 43999d813e564dc688fad116090c95ca (127.14.177.129:38155): Couldn't send request to peer 43999d813e564dc688fad116090c95ca. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:38155: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:23:50.973970 25424 consensus_peers.cc:597] T e9e342ecda4b46619ce12452b2cf7154 P 9868dce521294d0b816bba7648f4ae5a -> Peer 43999d813e564dc688fad116090c95ca (127.14.177.129:38155): Couldn't send request to peer 43999d813e564dc688fad116090c95ca. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:38155: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:23:50.975401 25456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ed5edbe32044d1c83b0590dbb899801" candidate_uuid: "43999d813e564dc688fad116090c95ca" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9868dce521294d0b816bba7648f4ae5a" is_pre_election: true
I20260709 13:23:50.977772 26111 leader_election.cc:304] T 0ed5edbe32044d1c83b0590dbb899801 P 43999d813e564dc688fad116090c95ca [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 43999d813e564dc688fad116090c95ca; no voters: 3e07d64466934da4bb3c02262b1c2e38, 9868dce521294d0b816bba7648f4ae5a
I20260709 13:23:50.978171 26286 consensus_queue.cc:1048] T f1ac2f4d2fe2424bb470df42199219e2 P 9868dce521294d0b816bba7648f4ae5a [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:23:50.978965 26234 raft_consensus.cc:2749] T 0ed5edbe32044d1c83b0590dbb899801 P 43999d813e564dc688fad116090c95ca [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:23:50.994351 25424 consensus_peers.cc:597] T 1f15ec62698c4a8cbd84f2b3d516d2ea P 9868dce521294d0b816bba7648f4ae5a -> Peer 43999d813e564dc688fad116090c95ca (127.14.177.129:38155): Couldn't send request to peer 43999d813e564dc688fad116090c95ca. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:38155: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:23:51.000274 26118 consensus_queue.cc:260] T a5df750b84e0487fa530161079deb6c2 P 43999d813e564dc688fad116090c95ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } }
W20260709 13:23:51.005740 25424 consensus_peers.cc:597] T f1ac2f4d2fe2424bb470df42199219e2 P 9868dce521294d0b816bba7648f4ae5a -> Peer 43999d813e564dc688fad116090c95ca (127.14.177.129:38155): Couldn't send request to peer 43999d813e564dc688fad116090c95ca. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:38155: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:23:51.013105 25276 consensus_peers.cc:597] T 28c0b6fc9bd64d60828d705c6a9ab1e4 P 233e7d29aac04505bf4e65f0a39a0c4c -> Peer 43999d813e564dc688fad116090c95ca (127.14.177.129:38155): Couldn't send request to peer 43999d813e564dc688fad116090c95ca. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:38155: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:23:51.015415 25276 consensus_peers.cc:597] T 441d780c3945463f92f31170f31e37a8 P 233e7d29aac04505bf4e65f0a39a0c4c -> Peer 43999d813e564dc688fad116090c95ca (127.14.177.129:38155): Couldn't send request to peer 43999d813e564dc688fad116090c95ca. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:38155: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:23:51.016148 25276 consensus_peers.cc:597] T b85a238bbc3b4621881603a534c7bd9b P 233e7d29aac04505bf4e65f0a39a0c4c -> Peer 43999d813e564dc688fad116090c95ca (127.14.177.129:38155): Couldn't send request to peer 43999d813e564dc688fad116090c95ca. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:38155: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:23:51.022994 26118 ts_tablet_manager.cc:1434] T a5df750b84e0487fa530161079deb6c2 P 43999d813e564dc688fad116090c95ca: Time spent starting tablet: real 0.073s	user 0.008s	sys 0.000s
I20260709 13:23:51.025182 26118 tablet_bootstrap.cc:492] T fa0da2c845dc4059968332f4db2e0aa6 P 43999d813e564dc688fad116090c95ca: Bootstrap starting.
I20260709 13:23:51.063185 25134 catalog_manager.cc:5697] T 434a54fcee7d4aeeab42f739855c0e8a P 233e7d29aac04505bf4e65f0a39a0c4c reported cstate change: term changed from 3 to 4, leader changed from 43999d813e564dc688fad116090c95ca (127.14.177.129) to 233e7d29aac04505bf4e65f0a39a0c4c (127.14.177.130). New cstate: current_term: 4 leader_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } health_report { overall_health: UNKNOWN } } }
I20260709 13:23:51.065240 25132 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:35154 (request call id 96) took 1347 ms. Trace:
I20260709 13:23:51.069118 25132 rpcz_store.cc:276] 0709 13:23:49.717451 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:23:49.717724 (+   273us) service_pool.cc:224] Handling call
0709 13:23:51.065161 (+1347437us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:23:50.959155 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:23:50.959724 (+   569us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:23:50.959756 (+    32us) write_op.cc:435] Acquired schema lock
0709 13:23:50.959773 (+    17us) tablet.cc:662] Decoding operations
0709 13:23:50.960301 (+   528us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:23:50.960378 (+    77us) write_op.cc:641] Partition lock acquired for write op
0709 13:23:50.960401 (+    23us) tablet.cc:685] Acquiring locks for 3 operations
0709 13:23:50.960736 (+   335us) tablet.cc:701] Row locks acquired
0709 13:23:50.960752 (+    16us) write_op.cc:260] PREPARE: finished
0709 13:23:50.960889 (+   137us) write_op.cc:270] Start()
0709 13:23:50.960989 (+   100us) write_op.cc:276] Timestamp: P: 1783603430960845 usec, L: 0
0709 13:23:50.961009 (+    20us) op_driver.cc:348] REPLICATION: starting
0709 13:23:50.980847 (+ 19838us) log.cc:844] Serialized 1325 byte log entry
0709 13:23:51.018741 (+ 37894us) op_driver.cc:464] REPLICATION: finished
0709 13:23:51.037913 (+ 19172us) write_op.cc:301] APPLY: starting
0709 13:23:51.038005 (+    92us) tablet.cc:1367] starting BulkCheckPresence
0709 13:23:51.038164 (+   159us) tablet.cc:1370] finished BulkCheckPresence
0709 13:23:51.038287 (+   123us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:23:51.039517 (+  1230us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:23:51.039643 (+   126us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=3
0709 13:23:51.039671 (+    28us) write_op.cc:312] APPLY: finished
0709 13:23:51.039820 (+   149us) log.cc:844] Serialized 34 byte log entry
0709 13:23:51.040006 (+   186us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:23:51.040105 (+    99us) write_op.cc:454] Released schema lock
0709 13:23:51.040144 (+    39us) write_op.cc:341] FINISH: Updating metrics
0709 13:23:51.040491 (+   347us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:23:51.040526 (+    35us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":562219,"reported_tablets":10,"child_traces":[["op",{"apply.queue_time_us":17406,"apply.run_cpu_time_us":2564,"apply.run_wall_time_us":2564,"num_ops":3,"prepare.queue_time_us":76431,"prepare.run_cpu_time_us":3149,"prepare.run_wall_time_us":24071,"replication_time_us":57540,"thread_start_us":81764,"threads_started":2}]]}
W20260709 13:23:51.079375 25350 consensus_peers.cc:597] T 91ac744179de48e5bb924b6dfd9f9c9b P 3e07d64466934da4bb3c02262b1c2e38 -> Peer 43999d813e564dc688fad116090c95ca (127.14.177.129:38155): Couldn't send request to peer 43999d813e564dc688fad116090c95ca. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:38155: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:23:51.084789 26118 tablet_bootstrap.cc:492] T fa0da2c845dc4059968332f4db2e0aa6 P 43999d813e564dc688fad116090c95ca: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:23:51.085839 26118 tablet_bootstrap.cc:492] T fa0da2c845dc4059968332f4db2e0aa6 P 43999d813e564dc688fad116090c95ca: Bootstrap complete.
I20260709 13:23:51.086532 26118 ts_tablet_manager.cc:1403] T fa0da2c845dc4059968332f4db2e0aa6 P 43999d813e564dc688fad116090c95ca: Time spent bootstrapping tablet: real 0.062s	user 0.009s	sys 0.010s
I20260709 13:23:51.094357 25134 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:35126 (request call id 101) took 1470 ms. Trace:
I20260709 13:23:51.095582 26118 raft_consensus.cc:359] T fa0da2c845dc4059968332f4db2e0aa6 P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } }
I20260709 13:23:51.096591 26118 raft_consensus.cc:740] T fa0da2c845dc4059968332f4db2e0aa6 P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43999d813e564dc688fad116090c95ca, State: Initialized, Role: FOLLOWER
I20260709 13:23:51.094846 25134 rpcz_store.cc:276] 0709 13:23:49.623779 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:23:49.662569 (+ 38790us) service_pool.cc:224] Handling call
0709 13:23:51.094289 (+1431720us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:23:51.075895 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:23:51.080486 (+  4591us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:23:51.080522 (+    36us) write_op.cc:435] Acquired schema lock
0709 13:23:51.080541 (+    19us) tablet.cc:662] Decoding operations
0709 13:23:51.080940 (+   399us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:23:51.080997 (+    57us) write_op.cc:641] Partition lock acquired for write op
0709 13:23:51.081022 (+    25us) tablet.cc:685] Acquiring locks for 1 operations
0709 13:23:51.081219 (+   197us) tablet.cc:701] Row locks acquired
0709 13:23:51.081237 (+    18us) write_op.cc:260] PREPARE: finished
0709 13:23:51.081411 (+   174us) write_op.cc:270] Start()
0709 13:23:51.081519 (+   108us) write_op.cc:276] Timestamp: P: 1783603431081365 usec, L: 0
0709 13:23:51.081545 (+    26us) op_driver.cc:348] REPLICATION: starting
0709 13:23:51.082123 (+   578us) log.cc:844] Serialized 543 byte log entry
0709 13:23:51.084907 (+  2784us) op_driver.cc:464] REPLICATION: finished
0709 13:23:51.085424 (+   517us) write_op.cc:301] APPLY: starting
0709 13:23:51.085504 (+    80us) tablet.cc:1367] starting BulkCheckPresence
0709 13:23:51.085626 (+   122us) tablet.cc:1370] finished BulkCheckPresence
0709 13:23:51.085722 (+    96us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:23:51.085941 (+   219us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:23:51.086083 (+   142us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0709 13:23:51.086131 (+    48us) write_op.cc:312] APPLY: finished
0709 13:23:51.086317 (+   186us) log.cc:844] Serialized 22 byte log entry
0709 13:23:51.086471 (+   154us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:23:51.086553 (+    82us) write_op.cc:454] Released schema lock
0709 13:23:51.086599 (+    46us) write_op.cc:341] FINISH: Updating metrics
0709 13:23:51.087030 (+   431us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:23:51.087061 (+    31us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":10,"child_traces":[["op",{"apply.queue_time_us":291,"apply.run_cpu_time_us":1544,"apply.run_wall_time_us":1545,"num_ops":1,"prepare.queue_time_us":116,"prepare.run_cpu_time_us":2805,"prepare.run_wall_time_us":7047,"replication_time_us":3191}]]}
I20260709 13:23:51.100246 25130 catalog_manager.cc:5697] T b7ff8dde9f5d4c49b9203d902f6b94c4 P 3e07d64466934da4bb3c02262b1c2e38 reported cstate change: term changed from 1 to 3, leader changed from 43999d813e564dc688fad116090c95ca (127.14.177.129) to 3e07d64466934da4bb3c02262b1c2e38 (127.14.177.131). New cstate: current_term: 3 leader_uuid: "3e07d64466934da4bb3c02262b1c2e38" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } health_report { overall_health: HEALTHY } } }
I20260709 13:23:51.097592 26118 consensus_queue.cc:260] T fa0da2c845dc4059968332f4db2e0aa6 P 43999d813e564dc688fad116090c95ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } }
I20260709 13:23:51.105659 26118 ts_tablet_manager.cc:1434] T fa0da2c845dc4059968332f4db2e0aa6 P 43999d813e564dc688fad116090c95ca: Time spent starting tablet: real 0.019s	user 0.001s	sys 0.008s
I20260709 13:23:51.106689 26118 tablet_bootstrap.cc:492] T 821a5866429f4173820fb64336abee02 P 43999d813e564dc688fad116090c95ca: Bootstrap starting.
I20260709 13:23:51.138877 26118 tablet_bootstrap.cc:492] T 821a5866429f4173820fb64336abee02 P 43999d813e564dc688fad116090c95ca: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:23:51.139945 26118 tablet_bootstrap.cc:492] T 821a5866429f4173820fb64336abee02 P 43999d813e564dc688fad116090c95ca: Bootstrap complete.
I20260709 13:23:51.140964 26118 ts_tablet_manager.cc:1403] T 821a5866429f4173820fb64336abee02 P 43999d813e564dc688fad116090c95ca: Time spent bootstrapping tablet: real 0.035s	user 0.020s	sys 0.004s
I20260709 13:23:51.147753 26118 raft_consensus.cc:359] T 821a5866429f4173820fb64336abee02 P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } }
I20260709 13:23:51.148479 26118 raft_consensus.cc:740] T 821a5866429f4173820fb64336abee02 P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43999d813e564dc688fad116090c95ca, State: Initialized, Role: FOLLOWER
I20260709 13:23:51.149220 26118 consensus_queue.cc:260] T 821a5866429f4173820fb64336abee02 P 43999d813e564dc688fad116090c95ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } }
I20260709 13:23:51.151201 26118 ts_tablet_manager.cc:1434] T 821a5866429f4173820fb64336abee02 P 43999d813e564dc688fad116090c95ca: Time spent starting tablet: real 0.010s	user 0.003s	sys 0.000s
I20260709 13:23:51.152562 26118 tablet_bootstrap.cc:492] T d053b9ae38de4173a0aded4da494bd98 P 43999d813e564dc688fad116090c95ca: Bootstrap starting.
I20260709 13:23:51.158732 25130 catalog_manager.cc:5697] T f1ac2f4d2fe2424bb470df42199219e2 P 3e07d64466934da4bb3c02262b1c2e38 reported cstate change: term changed from 1 to 3, leader changed from 43999d813e564dc688fad116090c95ca (127.14.177.129) to 9868dce521294d0b816bba7648f4ae5a (127.14.177.132). New cstate: current_term: 3 leader_uuid: "9868dce521294d0b816bba7648f4ae5a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } }
W20260709 13:23:51.208041 25424 consensus_peers.cc:597] T d0a0e4d32ce943fbb773bb840863647b P 9868dce521294d0b816bba7648f4ae5a -> Peer 43999d813e564dc688fad116090c95ca (127.14.177.129:38155): Couldn't send request to peer 43999d813e564dc688fad116090c95ca. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:38155: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:23:51.209452 25276 consensus_peers.cc:597] T 3a3317f4c3514df1b2bb8e44a86be92f P 233e7d29aac04505bf4e65f0a39a0c4c -> Peer 43999d813e564dc688fad116090c95ca (127.14.177.129:38155): Couldn't send request to peer 43999d813e564dc688fad116090c95ca. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:38155: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:23:51.236217 26118 tablet_bootstrap.cc:492] T d053b9ae38de4173a0aded4da494bd98 P 43999d813e564dc688fad116090c95ca: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:23:51.237351 26118 tablet_bootstrap.cc:492] T d053b9ae38de4173a0aded4da494bd98 P 43999d813e564dc688fad116090c95ca: Bootstrap complete.
I20260709 13:23:51.238065 26118 ts_tablet_manager.cc:1403] T d053b9ae38de4173a0aded4da494bd98 P 43999d813e564dc688fad116090c95ca: Time spent bootstrapping tablet: real 0.086s	user 0.034s	sys 0.024s
I20260709 13:23:51.241070 26118 raft_consensus.cc:359] T d053b9ae38de4173a0aded4da494bd98 P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } }
I20260709 13:23:51.242005 26118 raft_consensus.cc:740] T d053b9ae38de4173a0aded4da494bd98 P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43999d813e564dc688fad116090c95ca, State: Initialized, Role: FOLLOWER
I20260709 13:23:51.243002 26118 consensus_queue.cc:260] T d053b9ae38de4173a0aded4da494bd98 P 43999d813e564dc688fad116090c95ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } }
I20260709 13:23:51.245265 26118 ts_tablet_manager.cc:1434] T d053b9ae38de4173a0aded4da494bd98 P 43999d813e564dc688fad116090c95ca: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 13:23:51.246282 26118 tablet_bootstrap.cc:492] T 0c4f6b7401db47d59a4f34ed0e8b5350 P 43999d813e564dc688fad116090c95ca: Bootstrap starting.
W20260709 13:23:51.297260 25350 consensus_peers.cc:597] T beacc815ef954fb692e90544bfa93f05 P 3e07d64466934da4bb3c02262b1c2e38 -> Peer 43999d813e564dc688fad116090c95ca (127.14.177.129:38155): Couldn't send request to peer 43999d813e564dc688fad116090c95ca. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:38155: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:23:51.306854 26118 tablet_bootstrap.cc:492] T 0c4f6b7401db47d59a4f34ed0e8b5350 P 43999d813e564dc688fad116090c95ca: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:23:51.308274 26118 tablet_bootstrap.cc:492] T 0c4f6b7401db47d59a4f34ed0e8b5350 P 43999d813e564dc688fad116090c95ca: Bootstrap complete.
I20260709 13:23:51.308961 26118 ts_tablet_manager.cc:1403] T 0c4f6b7401db47d59a4f34ed0e8b5350 P 43999d813e564dc688fad116090c95ca: Time spent bootstrapping tablet: real 0.063s	user 0.046s	sys 0.010s
W20260709 13:23:51.311277 25424 consensus_peers.cc:597] T 1550ed2d2a764c08adffae679599ba41 P 9868dce521294d0b816bba7648f4ae5a -> Peer 43999d813e564dc688fad116090c95ca (127.14.177.129:38155): Couldn't send request to peer 43999d813e564dc688fad116090c95ca. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:38155: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:23:51.311909 26118 raft_consensus.cc:359] T 0c4f6b7401db47d59a4f34ed0e8b5350 P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } }
I20260709 13:23:51.312753 26118 raft_consensus.cc:740] T 0c4f6b7401db47d59a4f34ed0e8b5350 P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43999d813e564dc688fad116090c95ca, State: Initialized, Role: FOLLOWER
I20260709 13:23:51.319607 26118 consensus_queue.cc:260] T 0c4f6b7401db47d59a4f34ed0e8b5350 P 43999d813e564dc688fad116090c95ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } }
W20260709 13:23:51.321938 25350 consensus_peers.cc:597] T 9ffa42c3b2ff48bc87310e9ab5d218ce P 3e07d64466934da4bb3c02262b1c2e38 -> Peer 43999d813e564dc688fad116090c95ca (127.14.177.129:38155): Couldn't send request to peer 43999d813e564dc688fad116090c95ca. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:38155: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:23:51.322026 26118 ts_tablet_manager.cc:1434] T 0c4f6b7401db47d59a4f34ed0e8b5350 P 43999d813e564dc688fad116090c95ca: Time spent starting tablet: real 0.013s	user 0.005s	sys 0.001s
I20260709 13:23:51.325438 26118 tablet_bootstrap.cc:492] T 91ac744179de48e5bb924b6dfd9f9c9b P 43999d813e564dc688fad116090c95ca: Bootstrap starting.
I20260709 13:23:51.327299 26234 raft_consensus.cc:493] T 0cb89d70b4ce4aa8992fb68480bec8a2 P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:23:51.328049 25424 consensus_peers.cc:597] T 8ef061028f38450e83324ac78cdaa14c P 9868dce521294d0b816bba7648f4ae5a -> Peer 43999d813e564dc688fad116090c95ca (127.14.177.129:38155): Couldn't send request to peer 43999d813e564dc688fad116090c95ca. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:38155: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:23:51.327989 26234 raft_consensus.cc:515] T 0cb89d70b4ce4aa8992fb68480bec8a2 P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } }
I20260709 13:23:51.331393 25305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cb89d70b4ce4aa8992fb68480bec8a2" candidate_uuid: "43999d813e564dc688fad116090c95ca" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" is_pre_election: true
I20260709 13:23:51.333606 25377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cb89d70b4ce4aa8992fb68480bec8a2" candidate_uuid: "43999d813e564dc688fad116090c95ca" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3e07d64466934da4bb3c02262b1c2e38" is_pre_election: true
I20260709 13:23:51.340502 26111 leader_election.cc:304] T 0cb89d70b4ce4aa8992fb68480bec8a2 P 43999d813e564dc688fad116090c95ca [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 43999d813e564dc688fad116090c95ca; no voters: 233e7d29aac04505bf4e65f0a39a0c4c, 3e07d64466934da4bb3c02262b1c2e38
W20260709 13:23:51.342388 25350 consensus_peers.cc:597] T 0e71b6f5c69b4b35a8ac51c22c7d5f0d P 3e07d64466934da4bb3c02262b1c2e38 -> Peer 43999d813e564dc688fad116090c95ca (127.14.177.129:38155): Couldn't send request to peer 43999d813e564dc688fad116090c95ca. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:38155: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:23:51.333684 26234 leader_election.cc:290] T 0cb89d70b4ce4aa8992fb68480bec8a2 P 43999d813e564dc688fad116090c95ca [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 233e7d29aac04505bf4e65f0a39a0c4c (127.14.177.130:42957), 3e07d64466934da4bb3c02262b1c2e38 (127.14.177.131:41539)
I20260709 13:23:51.344031 26234 raft_consensus.cc:2749] T 0cb89d70b4ce4aa8992fb68480bec8a2 P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:23:51.356835 25276 consensus_peers.cc:597] T ce9c60e3922c4219b6e4453fe0259b6a P 233e7d29aac04505bf4e65f0a39a0c4c -> Peer 43999d813e564dc688fad116090c95ca (127.14.177.129:38155): Couldn't send request to peer 43999d813e564dc688fad116090c95ca. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:38155: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:23:51.374320 25350 consensus_peers.cc:597] T 87085ac2575e4b6a86dd1fc21e6f718b P 3e07d64466934da4bb3c02262b1c2e38 -> Peer 43999d813e564dc688fad116090c95ca (127.14.177.129:38155): Couldn't send request to peer 43999d813e564dc688fad116090c95ca. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:38155: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:23:51.377930 26118 tablet_bootstrap.cc:492] T 91ac744179de48e5bb924b6dfd9f9c9b P 43999d813e564dc688fad116090c95ca: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:23:51.378810 15046 rpc_server.cc:307] RPC server started. Bound to: 127.14.177.129:38155
I20260709 13:23:51.379086 26118 tablet_bootstrap.cc:492] T 91ac744179de48e5bb924b6dfd9f9c9b P 43999d813e564dc688fad116090c95ca: Bootstrap complete.
I20260709 13:23:51.379202 26397 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.14.177.129:38155 every 8 connection(s)
I20260709 13:23:51.380159 26118 ts_tablet_manager.cc:1403] T 91ac744179de48e5bb924b6dfd9f9c9b P 43999d813e564dc688fad116090c95ca: Time spent bootstrapping tablet: real 0.055s	user 0.009s	sys 0.012s
I20260709 13:23:51.384164 26118 raft_consensus.cc:359] T 91ac744179de48e5bb924b6dfd9f9c9b P 43999d813e564dc688fad116090c95ca [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } }
I20260709 13:23:51.385241 26118 raft_consensus.cc:740] T 91ac744179de48e5bb924b6dfd9f9c9b P 43999d813e564dc688fad116090c95ca [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43999d813e564dc688fad116090c95ca, State: Initialized, Role: FOLLOWER
I20260709 13:23:51.386518 26118 consensus_queue.cc:260] T 91ac744179de48e5bb924b6dfd9f9c9b P 43999d813e564dc688fad116090c95ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } }
I20260709 13:23:51.392231 26118 ts_tablet_manager.cc:1434] T 91ac744179de48e5bb924b6dfd9f9c9b P 43999d813e564dc688fad116090c95ca: Time spent starting tablet: real 0.012s	user 0.008s	sys 0.001s
I20260709 13:23:51.393824 26118 tablet_bootstrap.cc:492] T ce9c60e3922c4219b6e4453fe0259b6a P 43999d813e564dc688fad116090c95ca: Bootstrap starting.
I20260709 13:23:51.433542 26348 raft_consensus.cc:3060] T 441d780c3945463f92f31170f31e37a8 P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:23:51.449525 26118 tablet_bootstrap.cc:492] T ce9c60e3922c4219b6e4453fe0259b6a P 43999d813e564dc688fad116090c95ca: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:23:51.450743 26118 tablet_bootstrap.cc:492] T ce9c60e3922c4219b6e4453fe0259b6a P 43999d813e564dc688fad116090c95ca: Bootstrap complete.
I20260709 13:23:51.457525 26118 ts_tablet_manager.cc:1403] T ce9c60e3922c4219b6e4453fe0259b6a P 43999d813e564dc688fad116090c95ca: Time spent bootstrapping tablet: real 0.064s	user 0.018s	sys 0.010s
I20260709 13:23:51.460003 26118 raft_consensus.cc:359] T ce9c60e3922c4219b6e4453fe0259b6a P 43999d813e564dc688fad116090c95ca [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } }
I20260709 13:23:51.460745 26118 raft_consensus.cc:740] T ce9c60e3922c4219b6e4453fe0259b6a P 43999d813e564dc688fad116090c95ca [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43999d813e564dc688fad116090c95ca, State: Initialized, Role: FOLLOWER
I20260709 13:23:51.461419 26118 consensus_queue.cc:260] T ce9c60e3922c4219b6e4453fe0259b6a P 43999d813e564dc688fad116090c95ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } }
I20260709 13:23:51.463117 26118 ts_tablet_manager.cc:1434] T ce9c60e3922c4219b6e4453fe0259b6a P 43999d813e564dc688fad116090c95ca: Time spent starting tablet: real 0.005s	user 0.002s	sys 0.000s
I20260709 13:23:51.464773 26118 tablet_bootstrap.cc:492] T 8ef061028f38450e83324ac78cdaa14c P 43999d813e564dc688fad116090c95ca: Bootstrap starting.
I20260709 13:23:51.467720 26399 heartbeater.cc:344] Connected to a master server at 127.14.177.190:45841
I20260709 13:23:51.468214 26399 heartbeater.cc:461] Registering TS with master...
I20260709 13:23:51.469255 26399 heartbeater.cc:507] Master 127.14.177.190:45841 requested a full tablet report, sending...
I20260709 13:23:51.470014 26348 raft_consensus.cc:1240] T b7a891bfc89e4e259f53da6cb16f4989 P 43999d813e564dc688fad116090c95ca [term 2 FOLLOWER]: Rejecting Update request from peer 3e07d64466934da4bb3c02262b1c2e38 for earlier term 1. Current term is 2. Ops: []
I20260709 13:23:51.472105 26190 consensus_queue.cc:1059] T b7a891bfc89e4e259f53da6cb16f4989 P 3e07d64466934da4bb3c02262b1c2e38 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:23:51.479483 26348 raft_consensus.cc:1240] T 91ac744179de48e5bb924b6dfd9f9c9b P 43999d813e564dc688fad116090c95ca [term 2 FOLLOWER]: Rejecting Update request from peer 3e07d64466934da4bb3c02262b1c2e38 for earlier term 1. Current term is 2. Ops: []
I20260709 13:23:51.481385 26213 consensus_queue.cc:1059] T 91ac744179de48e5bb924b6dfd9f9c9b P 3e07d64466934da4bb3c02262b1c2e38 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:23:51.482820 26198 raft_consensus.cc:3055] T 91ac744179de48e5bb924b6dfd9f9c9b P 3e07d64466934da4bb3c02262b1c2e38 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:23:51.483282 26198 raft_consensus.cc:740] T 91ac744179de48e5bb924b6dfd9f9c9b P 3e07d64466934da4bb3c02262b1c2e38 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3e07d64466934da4bb3c02262b1c2e38, State: Running, Role: LEADER
I20260709 13:23:51.484341 26198 consensus_queue.cc:260] T 91ac744179de48e5bb924b6dfd9f9c9b P 3e07d64466934da4bb3c02262b1c2e38 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } }
I20260709 13:23:51.490602 26198 raft_consensus.cc:3060] T 91ac744179de48e5bb924b6dfd9f9c9b P 3e07d64466934da4bb3c02262b1c2e38 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:23:51.493058 26199 raft_consensus.cc:3055] T b7a891bfc89e4e259f53da6cb16f4989 P 3e07d64466934da4bb3c02262b1c2e38 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:23:51.493603 26199 raft_consensus.cc:740] T b7a891bfc89e4e259f53da6cb16f4989 P 3e07d64466934da4bb3c02262b1c2e38 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3e07d64466934da4bb3c02262b1c2e38, State: Running, Role: LEADER
I20260709 13:23:51.494863 26199 consensus_queue.cc:260] T b7a891bfc89e4e259f53da6cb16f4989 P 3e07d64466934da4bb3c02262b1c2e38 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } }
I20260709 13:23:51.496354 26199 raft_consensus.cc:3060] T b7a891bfc89e4e259f53da6cb16f4989 P 3e07d64466934da4bb3c02262b1c2e38 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:23:51.501361 25132 ts_manager.cc:194] Re-registered known tserver with Master: 43999d813e564dc688fad116090c95ca (127.14.177.129:38155)
I20260709 13:23:51.513947 26391 raft_consensus.cc:493] T b7a891bfc89e4e259f53da6cb16f4989 P 43999d813e564dc688fad116090c95ca [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:23:51.514653 26391 raft_consensus.cc:515] T b7a891bfc89e4e259f53da6cb16f4989 P 43999d813e564dc688fad116090c95ca [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } }
I20260709 13:23:51.516723 26118 tablet_bootstrap.cc:492] T 8ef061028f38450e83324ac78cdaa14c P 43999d813e564dc688fad116090c95ca: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:23:51.516996 26391 leader_election.cc:290] T b7a891bfc89e4e259f53da6cb16f4989 P 43999d813e564dc688fad116090c95ca [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3e07d64466934da4bb3c02262b1c2e38 (127.14.177.131:41539), 9868dce521294d0b816bba7648f4ae5a (127.14.177.132:33683)
I20260709 13:23:51.517791 26118 tablet_bootstrap.cc:492] T 8ef061028f38450e83324ac78cdaa14c P 43999d813e564dc688fad116090c95ca: Bootstrap complete.
I20260709 13:23:51.518636 26118 ts_tablet_manager.cc:1403] T 8ef061028f38450e83324ac78cdaa14c P 43999d813e564dc688fad116090c95ca: Time spent bootstrapping tablet: real 0.054s	user 0.016s	sys 0.004s
I20260709 13:23:51.519295 25377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7a891bfc89e4e259f53da6cb16f4989" candidate_uuid: "43999d813e564dc688fad116090c95ca" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3e07d64466934da4bb3c02262b1c2e38" is_pre_election: true
I20260709 13:23:51.519990 25377 raft_consensus.cc:2468] T b7a891bfc89e4e259f53da6cb16f4989 P 3e07d64466934da4bb3c02262b1c2e38 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 43999d813e564dc688fad116090c95ca in term 2.
I20260709 13:23:51.521346 26118 raft_consensus.cc:359] T 8ef061028f38450e83324ac78cdaa14c P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } }
I20260709 13:23:51.521908 25456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7a891bfc89e4e259f53da6cb16f4989" candidate_uuid: "43999d813e564dc688fad116090c95ca" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9868dce521294d0b816bba7648f4ae5a" is_pre_election: true
I20260709 13:23:51.522128 26118 raft_consensus.cc:740] T 8ef061028f38450e83324ac78cdaa14c P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43999d813e564dc688fad116090c95ca, State: Initialized, Role: FOLLOWER
I20260709 13:23:51.522923 26118 consensus_queue.cc:260] T 8ef061028f38450e83324ac78cdaa14c P 43999d813e564dc688fad116090c95ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } }
I20260709 13:23:51.523661 26111 leader_election.cc:304] T b7a891bfc89e4e259f53da6cb16f4989 P 43999d813e564dc688fad116090c95ca [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e07d64466934da4bb3c02262b1c2e38, 43999d813e564dc688fad116090c95ca; no voters: 
I20260709 13:23:51.524817 26234 raft_consensus.cc:2804] T b7a891bfc89e4e259f53da6cb16f4989 P 43999d813e564dc688fad116090c95ca [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:23:51.525185 26234 raft_consensus.cc:493] T b7a891bfc89e4e259f53da6cb16f4989 P 43999d813e564dc688fad116090c95ca [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:23:51.525609 26234 raft_consensus.cc:3060] T b7a891bfc89e4e259f53da6cb16f4989 P 43999d813e564dc688fad116090c95ca [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:23:51.533320 26234 raft_consensus.cc:515] T b7a891bfc89e4e259f53da6cb16f4989 P 43999d813e564dc688fad116090c95ca [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } }
I20260709 13:23:51.536706 25377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7a891bfc89e4e259f53da6cb16f4989" candidate_uuid: "43999d813e564dc688fad116090c95ca" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3e07d64466934da4bb3c02262b1c2e38"
I20260709 13:23:51.537559 25377 raft_consensus.cc:3060] T b7a891bfc89e4e259f53da6cb16f4989 P 3e07d64466934da4bb3c02262b1c2e38 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:23:51.525251 26118 ts_tablet_manager.cc:1434] T 8ef061028f38450e83324ac78cdaa14c P 43999d813e564dc688fad116090c95ca: Time spent starting tablet: real 0.006s	user 0.003s	sys 0.005s
I20260709 13:23:51.542073 26118 tablet_bootstrap.cc:492] T 0d29b499fa4c41f1b974782cd0c9c1c2 P 43999d813e564dc688fad116090c95ca: Bootstrap starting.
I20260709 13:23:51.546300 26234 leader_election.cc:290] T b7a891bfc89e4e259f53da6cb16f4989 P 43999d813e564dc688fad116090c95ca [CANDIDATE]: Term 3 election: Requested vote from peers 3e07d64466934da4bb3c02262b1c2e38 (127.14.177.131:41539), 9868dce521294d0b816bba7648f4ae5a (127.14.177.132:33683)
I20260709 13:23:51.548341 25456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7a891bfc89e4e259f53da6cb16f4989" candidate_uuid: "43999d813e564dc688fad116090c95ca" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9868dce521294d0b816bba7648f4ae5a"
W20260709 13:23:51.549456 25165 debug-util.cc:398] Leaking SignalData structure 0x7b080028a780 after lost signal to thread 15049
W20260709 13:23:51.550613 25165 debug-util.cc:398] Leaking SignalData structure 0x7b0800329840 after lost signal to thread 25168
I20260709 13:23:51.551437 25377 raft_consensus.cc:2468] T b7a891bfc89e4e259f53da6cb16f4989 P 3e07d64466934da4bb3c02262b1c2e38 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 43999d813e564dc688fad116090c95ca in term 3.
W20260709 13:23:51.552069 25165 debug-util.cc:398] Leaking SignalData structure 0x7b080023a4a0 after lost signal to thread 25335
I20260709 13:23:51.552984 26111 leader_election.cc:304] T b7a891bfc89e4e259f53da6cb16f4989 P 43999d813e564dc688fad116090c95ca [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3e07d64466934da4bb3c02262b1c2e38, 43999d813e564dc688fad116090c95ca; no voters: 9868dce521294d0b816bba7648f4ae5a
W20260709 13:23:51.553350 25165 debug-util.cc:398] Leaking SignalData structure 0x7b08002d6840 after lost signal to thread 25409
I20260709 13:23:51.554139 26234 raft_consensus.cc:2804] T b7a891bfc89e4e259f53da6cb16f4989 P 43999d813e564dc688fad116090c95ca [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:23:51.554634 26234 raft_consensus.cc:697] T b7a891bfc89e4e259f53da6cb16f4989 P 43999d813e564dc688fad116090c95ca [term 3 LEADER]: Becoming Leader. State: Replica: 43999d813e564dc688fad116090c95ca, State: Running, Role: LEADER
I20260709 13:23:51.555454 26234 consensus_queue.cc:237] T b7a891bfc89e4e259f53da6cb16f4989 P 43999d813e564dc688fad116090c95ca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } }
W20260709 13:23:51.574725 25165 debug-util.cc:398] Leaking SignalData structure 0x7b08002503e0 after lost signal to thread 25483
I20260709 13:23:51.606226 26118 tablet_bootstrap.cc:492] T 0d29b499fa4c41f1b974782cd0c9c1c2 P 43999d813e564dc688fad116090c95ca: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:23:51.607450 26118 tablet_bootstrap.cc:492] T 0d29b499fa4c41f1b974782cd0c9c1c2 P 43999d813e564dc688fad116090c95ca: Bootstrap complete.
I20260709 13:23:51.608206 26118 ts_tablet_manager.cc:1403] T 0d29b499fa4c41f1b974782cd0c9c1c2 P 43999d813e564dc688fad116090c95ca: Time spent bootstrapping tablet: real 0.066s	user 0.018s	sys 0.000s
W20260709 13:23:51.619201 25424 consensus_peers.cc:597] T 5c7e7acbb09f4d49828fa652ca6abaac P 9868dce521294d0b816bba7648f4ae5a -> Peer 43999d813e564dc688fad116090c95ca (127.14.177.129:38155): Couldn't send request to peer 43999d813e564dc688fad116090c95ca. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:23:51.625767 26118 raft_consensus.cc:359] T 0d29b499fa4c41f1b974782cd0c9c1c2 P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } }
I20260709 13:23:51.626092 26348 raft_consensus.cc:1240] T 28c0b6fc9bd64d60828d705c6a9ab1e4 P 43999d813e564dc688fad116090c95ca [term 2 FOLLOWER]: Rejecting Update request from peer 233e7d29aac04505bf4e65f0a39a0c4c for earlier term 1. Current term is 2. Ops: []
I20260709 13:23:51.626573 26118 raft_consensus.cc:740] T 0d29b499fa4c41f1b974782cd0c9c1c2 P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43999d813e564dc688fad116090c95ca, State: Initialized, Role: FOLLOWER
I20260709 13:23:51.627390 26118 consensus_queue.cc:260] T 0d29b499fa4c41f1b974782cd0c9c1c2 P 43999d813e564dc688fad116090c95ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } }
I20260709 13:23:51.627854 26351 consensus_queue.cc:1059] T 28c0b6fc9bd64d60828d705c6a9ab1e4 P 233e7d29aac04505bf4e65f0a39a0c4c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:23:51.628971 26281 raft_consensus.cc:3055] T 28c0b6fc9bd64d60828d705c6a9ab1e4 P 233e7d29aac04505bf4e65f0a39a0c4c [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:23:51.629344 26281 raft_consensus.cc:740] T 28c0b6fc9bd64d60828d705c6a9ab1e4 P 233e7d29aac04505bf4e65f0a39a0c4c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 233e7d29aac04505bf4e65f0a39a0c4c, State: Running, Role: LEADER
I20260709 13:23:51.629565 26118 ts_tablet_manager.cc:1434] T 0d29b499fa4c41f1b974782cd0c9c1c2 P 43999d813e564dc688fad116090c95ca: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 13:23:51.630456 26118 tablet_bootstrap.cc:492] T beacc815ef954fb692e90544bfa93f05 P 43999d813e564dc688fad116090c95ca: Bootstrap starting.
I20260709 13:23:51.630118 26281 consensus_queue.cc:260] T 28c0b6fc9bd64d60828d705c6a9ab1e4 P 233e7d29aac04505bf4e65f0a39a0c4c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } }
I20260709 13:23:51.631557 26281 raft_consensus.cc:3060] T 28c0b6fc9bd64d60828d705c6a9ab1e4 P 233e7d29aac04505bf4e65f0a39a0c4c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:23:51.640086 25132 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:39414
I20260709 13:23:51.645399 26399 heartbeater.cc:499] Master 127.14.177.190:45841 was elected leader, sending a full tablet report...
I20260709 13:23:51.647518 26348 raft_consensus.cc:1240] T 0ed5edbe32044d1c83b0590dbb899801 P 43999d813e564dc688fad116090c95ca [term 2 FOLLOWER]: Rejecting Update request from peer 3e07d64466934da4bb3c02262b1c2e38 for earlier term 1. Current term is 2. Ops: []
I20260709 13:23:51.649029 26199 consensus_queue.cc:1059] T 0ed5edbe32044d1c83b0590dbb899801 P 3e07d64466934da4bb3c02262b1c2e38 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:23:51.650058 26199 raft_consensus.cc:3055] T 0ed5edbe32044d1c83b0590dbb899801 P 3e07d64466934da4bb3c02262b1c2e38 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:23:51.650386 26199 raft_consensus.cc:740] T 0ed5edbe32044d1c83b0590dbb899801 P 3e07d64466934da4bb3c02262b1c2e38 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3e07d64466934da4bb3c02262b1c2e38, State: Running, Role: LEADER
I20260709 13:23:51.651343 26199 consensus_queue.cc:260] T 0ed5edbe32044d1c83b0590dbb899801 P 3e07d64466934da4bb3c02262b1c2e38 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } }
I20260709 13:23:51.652644 26199 raft_consensus.cc:3060] T 0ed5edbe32044d1c83b0590dbb899801 P 3e07d64466934da4bb3c02262b1c2e38 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:23:51.669667 26118 tablet_bootstrap.cc:492] T beacc815ef954fb692e90544bfa93f05 P 43999d813e564dc688fad116090c95ca: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:23:51.670689 26118 tablet_bootstrap.cc:492] T beacc815ef954fb692e90544bfa93f05 P 43999d813e564dc688fad116090c95ca: Bootstrap complete.
I20260709 13:23:51.671474 26118 ts_tablet_manager.cc:1403] T beacc815ef954fb692e90544bfa93f05 P 43999d813e564dc688fad116090c95ca: Time spent bootstrapping tablet: real 0.041s	user 0.014s	sys 0.007s
I20260709 13:23:51.672004 26348 raft_consensus.cc:3060] T 434a54fcee7d4aeeab42f739855c0e8a P 43999d813e564dc688fad116090c95ca [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:23:51.674105 26118 raft_consensus.cc:359] T beacc815ef954fb692e90544bfa93f05 P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } }
I20260709 13:23:51.674858 26118 raft_consensus.cc:740] T beacc815ef954fb692e90544bfa93f05 P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43999d813e564dc688fad116090c95ca, State: Initialized, Role: FOLLOWER
I20260709 13:23:51.675932 26118 consensus_queue.cc:260] T beacc815ef954fb692e90544bfa93f05 P 43999d813e564dc688fad116090c95ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } }
I20260709 13:23:51.678323 26118 ts_tablet_manager.cc:1434] T beacc815ef954fb692e90544bfa93f05 P 43999d813e564dc688fad116090c95ca: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 13:23:51.679178 26118 tablet_bootstrap.cc:492] T ce98a3f3b3eb4c63bcaa355ec7cd64a4 P 43999d813e564dc688fad116090c95ca: Bootstrap starting.
I20260709 13:23:51.715535 26355 raft_consensus.cc:3060] T 821a5866429f4173820fb64336abee02 P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:23:51.750305 26118 tablet_bootstrap.cc:492] T ce98a3f3b3eb4c63bcaa355ec7cd64a4 P 43999d813e564dc688fad116090c95ca: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:23:51.751580 26118 tablet_bootstrap.cc:492] T ce98a3f3b3eb4c63bcaa355ec7cd64a4 P 43999d813e564dc688fad116090c95ca: Bootstrap complete.
I20260709 13:23:51.752421 26118 ts_tablet_manager.cc:1403] T ce98a3f3b3eb4c63bcaa355ec7cd64a4 P 43999d813e564dc688fad116090c95ca: Time spent bootstrapping tablet: real 0.073s	user 0.025s	sys 0.046s
I20260709 13:23:51.755420 26118 raft_consensus.cc:359] T ce98a3f3b3eb4c63bcaa355ec7cd64a4 P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } }
I20260709 13:23:51.756115 26118 raft_consensus.cc:740] T ce98a3f3b3eb4c63bcaa355ec7cd64a4 P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43999d813e564dc688fad116090c95ca, State: Initialized, Role: FOLLOWER
I20260709 13:23:51.756879 26118 consensus_queue.cc:260] T ce98a3f3b3eb4c63bcaa355ec7cd64a4 P 43999d813e564dc688fad116090c95ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } }
I20260709 13:23:51.759212 26118 ts_tablet_manager.cc:1434] T ce98a3f3b3eb4c63bcaa355ec7cd64a4 P 43999d813e564dc688fad116090c95ca: Time spent starting tablet: real 0.006s	user 0.007s	sys 0.001s
I20260709 13:23:51.759997 26118 tablet_bootstrap.cc:492] T e9e342ecda4b46619ce12452b2cf7154 P 43999d813e564dc688fad116090c95ca: Bootstrap starting.
I20260709 13:23:51.775566 26347 raft_consensus.cc:3060] T fa0da2c845dc4059968332f4db2e0aa6 P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:23:51.784093 26345 raft_consensus.cc:3060] T 8ef061028f38450e83324ac78cdaa14c P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:23:51.789508 26346 raft_consensus.cc:3060] T 13bb3a23315a41c7a40e10b954b15c8e P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:23:51.793102 26118 tablet_bootstrap.cc:492] T e9e342ecda4b46619ce12452b2cf7154 P 43999d813e564dc688fad116090c95ca: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:23:51.794138 26118 tablet_bootstrap.cc:492] T e9e342ecda4b46619ce12452b2cf7154 P 43999d813e564dc688fad116090c95ca: Bootstrap complete.
I20260709 13:23:51.794921 26118 ts_tablet_manager.cc:1403] T e9e342ecda4b46619ce12452b2cf7154 P 43999d813e564dc688fad116090c95ca: Time spent bootstrapping tablet: real 0.035s	user 0.017s	sys 0.008s
I20260709 13:23:51.797549 26118 raft_consensus.cc:359] T e9e342ecda4b46619ce12452b2cf7154 P 43999d813e564dc688fad116090c95ca [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } }
I20260709 13:23:51.798223 26118 raft_consensus.cc:740] T e9e342ecda4b46619ce12452b2cf7154 P 43999d813e564dc688fad116090c95ca [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43999d813e564dc688fad116090c95ca, State: Initialized, Role: FOLLOWER
I20260709 13:23:51.806540 26118 consensus_queue.cc:260] T e9e342ecda4b46619ce12452b2cf7154 P 43999d813e564dc688fad116090c95ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } }
W20260709 13:23:51.809278 25424 consensus_peers.cc:597] T c902bbc7ec0848ba82a04870eab1e7f5 P 9868dce521294d0b816bba7648f4ae5a -> Peer 43999d813e564dc688fad116090c95ca (127.14.177.129:38155): Couldn't send request to peer 43999d813e564dc688fad116090c95ca. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:23:51.815932 26118 ts_tablet_manager.cc:1434] T e9e342ecda4b46619ce12452b2cf7154 P 43999d813e564dc688fad116090c95ca: Time spent starting tablet: real 0.021s	user 0.005s	sys 0.000s
I20260709 13:23:51.816854 26118 tablet_bootstrap.cc:492] T 87085ac2575e4b6a86dd1fc21e6f718b P 43999d813e564dc688fad116090c95ca: Bootstrap starting.
I20260709 13:23:51.848909 26340 raft_consensus.cc:1240] T e9e342ecda4b46619ce12452b2cf7154 P 43999d813e564dc688fad116090c95ca [term 2 FOLLOWER]: Rejecting Update request from peer 9868dce521294d0b816bba7648f4ae5a for earlier term 1. Current term is 2. Ops: []
I20260709 13:23:51.852613 26086 consensus_queue.cc:1059] T e9e342ecda4b46619ce12452b2cf7154 P 9868dce521294d0b816bba7648f4ae5a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:23:51.854477 26286 raft_consensus.cc:3055] T e9e342ecda4b46619ce12452b2cf7154 P 9868dce521294d0b816bba7648f4ae5a [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:23:51.855119 26286 raft_consensus.cc:740] T e9e342ecda4b46619ce12452b2cf7154 P 9868dce521294d0b816bba7648f4ae5a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 9868dce521294d0b816bba7648f4ae5a, State: Running, Role: LEADER
I20260709 13:23:51.857353 26286 consensus_queue.cc:260] T e9e342ecda4b46619ce12452b2cf7154 P 9868dce521294d0b816bba7648f4ae5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } }
I20260709 13:23:51.858183 26118 tablet_bootstrap.cc:492] T 87085ac2575e4b6a86dd1fc21e6f718b P 43999d813e564dc688fad116090c95ca: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:23:51.861671 26118 tablet_bootstrap.cc:492] T 87085ac2575e4b6a86dd1fc21e6f718b P 43999d813e564dc688fad116090c95ca: Bootstrap complete.
I20260709 13:23:51.861796 26286 raft_consensus.cc:3060] T e9e342ecda4b46619ce12452b2cf7154 P 9868dce521294d0b816bba7648f4ae5a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:23:51.862469 26118 ts_tablet_manager.cc:1403] T 87085ac2575e4b6a86dd1fc21e6f718b P 43999d813e564dc688fad116090c95ca: Time spent bootstrapping tablet: real 0.046s	user 0.014s	sys 0.011s
I20260709 13:23:51.865577 26118 raft_consensus.cc:359] T 87085ac2575e4b6a86dd1fc21e6f718b P 43999d813e564dc688fad116090c95ca [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } }
I20260709 13:23:51.866284 26118 raft_consensus.cc:740] T 87085ac2575e4b6a86dd1fc21e6f718b P 43999d813e564dc688fad116090c95ca [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43999d813e564dc688fad116090c95ca, State: Initialized, Role: FOLLOWER
I20260709 13:23:51.867156 26118 consensus_queue.cc:260] T 87085ac2575e4b6a86dd1fc21e6f718b P 43999d813e564dc688fad116090c95ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } }
I20260709 13:23:51.870492 26118 ts_tablet_manager.cc:1434] T 87085ac2575e4b6a86dd1fc21e6f718b P 43999d813e564dc688fad116090c95ca: Time spent starting tablet: real 0.007s	user 0.003s	sys 0.004s
I20260709 13:23:51.871523 26118 tablet_bootstrap.cc:492] T 4d837697cd0f402cacf9db89a5d92652 P 43999d813e564dc688fad116090c95ca: Bootstrap starting.
I20260709 13:23:51.878851 26347 raft_consensus.cc:3060] T ce9c60e3922c4219b6e4453fe0259b6a P 43999d813e564dc688fad116090c95ca [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:23:51.881317 26345 raft_consensus.cc:3060] T a5df750b84e0487fa530161079deb6c2 P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:23:51.895560 26118 tablet_bootstrap.cc:492] T 4d837697cd0f402cacf9db89a5d92652 P 43999d813e564dc688fad116090c95ca: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:23:51.896741 26118 tablet_bootstrap.cc:492] T 4d837697cd0f402cacf9db89a5d92652 P 43999d813e564dc688fad116090c95ca: Bootstrap complete.
I20260709 13:23:51.897585 26118 ts_tablet_manager.cc:1403] T 4d837697cd0f402cacf9db89a5d92652 P 43999d813e564dc688fad116090c95ca: Time spent bootstrapping tablet: real 0.026s	user 0.014s	sys 0.008s
I20260709 13:23:51.900460 26118 raft_consensus.cc:359] T 4d837697cd0f402cacf9db89a5d92652 P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } }
I20260709 13:23:51.901292 26118 raft_consensus.cc:740] T 4d837697cd0f402cacf9db89a5d92652 P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43999d813e564dc688fad116090c95ca, State: Initialized, Role: FOLLOWER
I20260709 13:23:51.902259 26118 consensus_queue.cc:260] T 4d837697cd0f402cacf9db89a5d92652 P 43999d813e564dc688fad116090c95ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } }
I20260709 13:23:51.938009 26118 ts_tablet_manager.cc:1434] T 4d837697cd0f402cacf9db89a5d92652 P 43999d813e564dc688fad116090c95ca: Time spent starting tablet: real 0.040s	user 0.008s	sys 0.002s
I20260709 13:23:51.939684 26118 tablet_bootstrap.cc:492] T 72b56266e8a54776b9113c7a786e1952 P 43999d813e564dc688fad116090c95ca: Bootstrap starting.
I20260709 13:23:51.968977 25130 catalog_manager.cc:5697] T b7a891bfc89e4e259f53da6cb16f4989 P 43999d813e564dc688fad116090c95ca reported cstate change: term changed from 1 to 3, leader changed from 3e07d64466934da4bb3c02262b1c2e38 (127.14.177.131) to 43999d813e564dc688fad116090c95ca (127.14.177.129). New cstate: current_term: 3 leader_uuid: "43999d813e564dc688fad116090c95ca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } health_report { overall_health: HEALTHY } } }
W20260709 13:23:51.998140 25350 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.129:38155, user_credentials={real_user=slave}} blocked reactor thread for 54771.8us
W20260709 13:23:52.053097 25350 consensus_peers.cc:597] T 72b56266e8a54776b9113c7a786e1952 P 3e07d64466934da4bb3c02262b1c2e38 -> Peer 43999d813e564dc688fad116090c95ca (127.14.177.129:38155): Couldn't send request to peer 43999d813e564dc688fad116090c95ca. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:23:52.072041 25350 consensus_peers.cc:597] T b155b866f10547fd8239579a81ecf5f9 P 3e07d64466934da4bb3c02262b1c2e38 -> Peer 43999d813e564dc688fad116090c95ca (127.14.177.129:38155): Couldn't send request to peer 43999d813e564dc688fad116090c95ca. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:23:52.093125 25276 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.129:38155, user_credentials={real_user=slave}} blocked reactor thread for 52245.9us
W20260709 13:23:52.135766 25422 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41539, user_credentials={real_user=slave}} blocked reactor thread for 86831.7us
W20260709 13:23:52.146286 25424 consensus_peers.cc:597] T 9e5d70664720424d951f269d9d7f4346 P 9868dce521294d0b816bba7648f4ae5a -> Peer 43999d813e564dc688fad116090c95ca (127.14.177.129:38155): Couldn't send request to peer 43999d813e564dc688fad116090c95ca. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 16: this message will repeat every 5th retry.
I20260709 13:23:52.158524 25377 raft_consensus.cc:1275] T b7a891bfc89e4e259f53da6cb16f4989 P 3e07d64466934da4bb3c02262b1c2e38 [term 3 FOLLOWER]: Refusing update from remote peer 43999d813e564dc688fad116090c95ca: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:23:52.163934 26411 consensus_queue.cc:1048] T b7a891bfc89e4e259f53da6cb16f4989 P 43999d813e564dc688fad116090c95ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:23:52.177038 25454 raft_consensus.cc:3060] T b7a891bfc89e4e259f53da6cb16f4989 P 9868dce521294d0b816bba7648f4ae5a [term 1 FOLLOWER]: Advancing to term 3
I20260709 13:23:52.190991 26118 tablet_bootstrap.cc:492] T 72b56266e8a54776b9113c7a786e1952 P 43999d813e564dc688fad116090c95ca: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:23:52.193683 26118 tablet_bootstrap.cc:492] T 72b56266e8a54776b9113c7a786e1952 P 43999d813e564dc688fad116090c95ca: Bootstrap complete.
I20260709 13:23:52.194080 26348 raft_consensus.cc:3060] T 4d837697cd0f402cacf9db89a5d92652 P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:23:52.194468 26118 ts_tablet_manager.cc:1403] T 72b56266e8a54776b9113c7a786e1952 P 43999d813e564dc688fad116090c95ca: Time spent bootstrapping tablet: real 0.255s	user 0.012s	sys 0.025s
I20260709 13:23:52.197077 26118 raft_consensus.cc:359] T 72b56266e8a54776b9113c7a786e1952 P 43999d813e564dc688fad116090c95ca [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } }
I20260709 13:23:52.206172 26118 raft_consensus.cc:740] T 72b56266e8a54776b9113c7a786e1952 P 43999d813e564dc688fad116090c95ca [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43999d813e564dc688fad116090c95ca, State: Initialized, Role: FOLLOWER
I20260709 13:23:52.206868 26118 consensus_queue.cc:260] T 72b56266e8a54776b9113c7a786e1952 P 43999d813e564dc688fad116090c95ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } }
I20260709 13:23:52.211310 25454 raft_consensus.cc:1275] T b7a891bfc89e4e259f53da6cb16f4989 P 9868dce521294d0b816bba7648f4ae5a [term 3 FOLLOWER]: Refusing update from remote peer 43999d813e564dc688fad116090c95ca: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
W20260709 13:23:52.211364 25424 consensus_peers.cc:597] T 579da2d6aabb4ae9a2b1802e1358068b P 9868dce521294d0b816bba7648f4ae5a -> Peer 43999d813e564dc688fad116090c95ca (127.14.177.129:38155): Couldn't send request to peer 43999d813e564dc688fad116090c95ca. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:23:52.214905 26118 ts_tablet_manager.cc:1434] T 72b56266e8a54776b9113c7a786e1952 P 43999d813e564dc688fad116090c95ca: Time spent starting tablet: real 0.020s	user 0.005s	sys 0.002s
I20260709 13:23:52.216187 26118 tablet_bootstrap.cc:492] T 0e71b6f5c69b4b35a8ac51c22c7d5f0d P 43999d813e564dc688fad116090c95ca: Bootstrap starting.
I20260709 13:23:52.224841 26411 consensus_queue.cc:1048] T b7a891bfc89e4e259f53da6cb16f4989 P 43999d813e564dc688fad116090c95ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:23:52.288527 25424 consensus_peers.cc:597] T 280d65245f4f4358b54bef93e18aca92 P 9868dce521294d0b816bba7648f4ae5a -> Peer 43999d813e564dc688fad116090c95ca (127.14.177.129:38155): Couldn't send request to peer 43999d813e564dc688fad116090c95ca. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 16: this message will repeat every 5th retry.
I20260709 13:23:52.365657 26118 tablet_bootstrap.cc:492] T 0e71b6f5c69b4b35a8ac51c22c7d5f0d P 43999d813e564dc688fad116090c95ca: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:23:52.366715 26118 tablet_bootstrap.cc:492] T 0e71b6f5c69b4b35a8ac51c22c7d5f0d P 43999d813e564dc688fad116090c95ca: Bootstrap complete.
I20260709 13:23:52.367424 26118 ts_tablet_manager.cc:1403] T 0e71b6f5c69b4b35a8ac51c22c7d5f0d P 43999d813e564dc688fad116090c95ca: Time spent bootstrapping tablet: real 0.151s	user 0.028s	sys 0.058s
I20260709 13:23:52.370255 26118 raft_consensus.cc:359] T 0e71b6f5c69b4b35a8ac51c22c7d5f0d P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } }
I20260709 13:23:52.370993 26118 raft_consensus.cc:740] T 0e71b6f5c69b4b35a8ac51c22c7d5f0d P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43999d813e564dc688fad116090c95ca, State: Initialized, Role: FOLLOWER
I20260709 13:23:52.379922 26118 consensus_queue.cc:260] T 0e71b6f5c69b4b35a8ac51c22c7d5f0d P 43999d813e564dc688fad116090c95ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } }
I20260709 13:23:52.382485 26118 ts_tablet_manager.cc:1434] T 0e71b6f5c69b4b35a8ac51c22c7d5f0d P 43999d813e564dc688fad116090c95ca: Time spent starting tablet: real 0.015s	user 0.004s	sys 0.000s
I20260709 13:23:52.395848 26118 tablet_bootstrap.cc:492] T d0a0e4d32ce943fbb773bb840863647b P 43999d813e564dc688fad116090c95ca: Bootstrap starting.
I20260709 13:23:52.428486 26118 tablet_bootstrap.cc:492] T d0a0e4d32ce943fbb773bb840863647b P 43999d813e564dc688fad116090c95ca: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:23:52.429544 26118 tablet_bootstrap.cc:492] T d0a0e4d32ce943fbb773bb840863647b P 43999d813e564dc688fad116090c95ca: Bootstrap complete.
I20260709 13:23:52.430231 26118 ts_tablet_manager.cc:1403] T d0a0e4d32ce943fbb773bb840863647b P 43999d813e564dc688fad116090c95ca: Time spent bootstrapping tablet: real 0.035s	user 0.014s	sys 0.002s
W20260709 13:23:52.433406 25424 consensus_peers.cc:597] T 6242888a4d4146dcacd87d0fd9a832f4 P 9868dce521294d0b816bba7648f4ae5a -> Peer 43999d813e564dc688fad116090c95ca (127.14.177.129:38155): Couldn't send request to peer 43999d813e564dc688fad116090c95ca. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 16: this message will repeat every 5th retry.
I20260709 13:23:52.433336 26118 raft_consensus.cc:359] T d0a0e4d32ce943fbb773bb840863647b P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } }
I20260709 13:23:52.434122 26118 raft_consensus.cc:740] T d0a0e4d32ce943fbb773bb840863647b P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43999d813e564dc688fad116090c95ca, State: Initialized, Role: FOLLOWER
I20260709 13:23:52.434900 26118 consensus_queue.cc:260] T d0a0e4d32ce943fbb773bb840863647b P 43999d813e564dc688fad116090c95ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } }
I20260709 13:23:52.437220 26118 ts_tablet_manager.cc:1434] T d0a0e4d32ce943fbb773bb840863647b P 43999d813e564dc688fad116090c95ca: Time spent starting tablet: real 0.007s	user 0.009s	sys 0.000s
I20260709 13:23:52.438055 26118 tablet_bootstrap.cc:492] T 280d65245f4f4358b54bef93e18aca92 P 43999d813e564dc688fad116090c95ca: Bootstrap starting.
I20260709 13:23:52.462356 26347 raft_consensus.cc:3060] T 72b56266e8a54776b9113c7a786e1952 P 43999d813e564dc688fad116090c95ca [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:23:52.493927 26118 tablet_bootstrap.cc:492] T 280d65245f4f4358b54bef93e18aca92 P 43999d813e564dc688fad116090c95ca: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:23:52.495111 26118 tablet_bootstrap.cc:492] T 280d65245f4f4358b54bef93e18aca92 P 43999d813e564dc688fad116090c95ca: Bootstrap complete.
I20260709 13:23:52.495841 26118 ts_tablet_manager.cc:1403] T 280d65245f4f4358b54bef93e18aca92 P 43999d813e564dc688fad116090c95ca: Time spent bootstrapping tablet: real 0.058s	user 0.024s	sys 0.031s
I20260709 13:23:52.498821 26118 raft_consensus.cc:359] T 280d65245f4f4358b54bef93e18aca92 P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } }
I20260709 13:23:52.499545 26118 raft_consensus.cc:740] T 280d65245f4f4358b54bef93e18aca92 P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43999d813e564dc688fad116090c95ca, State: Initialized, Role: FOLLOWER
I20260709 13:23:52.500449 26118 consensus_queue.cc:260] T 280d65245f4f4358b54bef93e18aca92 P 43999d813e564dc688fad116090c95ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } }
I20260709 13:23:52.506875 26118 ts_tablet_manager.cc:1434] T 280d65245f4f4358b54bef93e18aca92 P 43999d813e564dc688fad116090c95ca: Time spent starting tablet: real 0.011s	user 0.006s	sys 0.000s
I20260709 13:23:52.507859 26118 tablet_bootstrap.cc:492] T 6242888a4d4146dcacd87d0fd9a832f4 P 43999d813e564dc688fad116090c95ca: Bootstrap starting.
I20260709 13:23:52.533682 26118 tablet_bootstrap.cc:492] T 6242888a4d4146dcacd87d0fd9a832f4 P 43999d813e564dc688fad116090c95ca: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:23:52.534714 26118 tablet_bootstrap.cc:492] T 6242888a4d4146dcacd87d0fd9a832f4 P 43999d813e564dc688fad116090c95ca: Bootstrap complete.
I20260709 13:23:52.535452 26118 ts_tablet_manager.cc:1403] T 6242888a4d4146dcacd87d0fd9a832f4 P 43999d813e564dc688fad116090c95ca: Time spent bootstrapping tablet: real 0.028s	user 0.020s	sys 0.004s
I20260709 13:23:52.538285 26118 raft_consensus.cc:359] T 6242888a4d4146dcacd87d0fd9a832f4 P 43999d813e564dc688fad116090c95ca [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } }
I20260709 13:23:52.539013 26118 raft_consensus.cc:740] T 6242888a4d4146dcacd87d0fd9a832f4 P 43999d813e564dc688fad116090c95ca [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43999d813e564dc688fad116090c95ca, State: Initialized, Role: FOLLOWER
I20260709 13:23:52.539866 26118 consensus_queue.cc:260] T 6242888a4d4146dcacd87d0fd9a832f4 P 43999d813e564dc688fad116090c95ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } }
I20260709 13:23:52.542208 26118 ts_tablet_manager.cc:1434] T 6242888a4d4146dcacd87d0fd9a832f4 P 43999d813e564dc688fad116090c95ca: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 13:23:52.543176 26118 tablet_bootstrap.cc:492] T 3a3317f4c3514df1b2bb8e44a86be92f P 43999d813e564dc688fad116090c95ca: Bootstrap starting.
I20260709 13:23:52.566730 26118 tablet_bootstrap.cc:492] T 3a3317f4c3514df1b2bb8e44a86be92f P 43999d813e564dc688fad116090c95ca: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:23:52.567771 26118 tablet_bootstrap.cc:492] T 3a3317f4c3514df1b2bb8e44a86be92f P 43999d813e564dc688fad116090c95ca: Bootstrap complete.
I20260709 13:23:52.568437 26118 ts_tablet_manager.cc:1403] T 3a3317f4c3514df1b2bb8e44a86be92f P 43999d813e564dc688fad116090c95ca: Time spent bootstrapping tablet: real 0.025s	user 0.021s	sys 0.002s
I20260709 13:23:52.571134 26118 raft_consensus.cc:359] T 3a3317f4c3514df1b2bb8e44a86be92f P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } }
I20260709 13:23:52.571867 26118 raft_consensus.cc:740] T 3a3317f4c3514df1b2bb8e44a86be92f P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43999d813e564dc688fad116090c95ca, State: Initialized, Role: FOLLOWER
I20260709 13:23:52.572786 26118 consensus_queue.cc:260] T 3a3317f4c3514df1b2bb8e44a86be92f P 43999d813e564dc688fad116090c95ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } }
I20260709 13:23:52.575217 26118 ts_tablet_manager.cc:1434] T 3a3317f4c3514df1b2bb8e44a86be92f P 43999d813e564dc688fad116090c95ca: Time spent starting tablet: real 0.006s	user 0.003s	sys 0.005s
I20260709 13:23:52.576015 26118 tablet_bootstrap.cc:492] T 1550ed2d2a764c08adffae679599ba41 P 43999d813e564dc688fad116090c95ca: Bootstrap starting.
I20260709 13:23:52.596233 26347 raft_consensus.cc:3060] T 3a3317f4c3514df1b2bb8e44a86be92f P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:23:52.605264 26118 tablet_bootstrap.cc:492] T 1550ed2d2a764c08adffae679599ba41 P 43999d813e564dc688fad116090c95ca: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:23:52.606318 26118 tablet_bootstrap.cc:492] T 1550ed2d2a764c08adffae679599ba41 P 43999d813e564dc688fad116090c95ca: Bootstrap complete.
I20260709 13:23:52.606986 26118 ts_tablet_manager.cc:1403] T 1550ed2d2a764c08adffae679599ba41 P 43999d813e564dc688fad116090c95ca: Time spent bootstrapping tablet: real 0.031s	user 0.016s	sys 0.008s
I20260709 13:23:52.609815 26118 raft_consensus.cc:359] T 1550ed2d2a764c08adffae679599ba41 P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } }
I20260709 13:23:52.610483 26118 raft_consensus.cc:740] T 1550ed2d2a764c08adffae679599ba41 P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43999d813e564dc688fad116090c95ca, State: Initialized, Role: FOLLOWER
I20260709 13:23:52.611213 26118 consensus_queue.cc:260] T 1550ed2d2a764c08adffae679599ba41 P 43999d813e564dc688fad116090c95ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } }
I20260709 13:23:52.612934 26118 ts_tablet_manager.cc:1434] T 1550ed2d2a764c08adffae679599ba41 P 43999d813e564dc688fad116090c95ca: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 13:23:52.613718 26118 tablet_bootstrap.cc:492] T b85a238bbc3b4621881603a534c7bd9b P 43999d813e564dc688fad116090c95ca: Bootstrap starting.
I20260709 13:23:52.649530 26118 tablet_bootstrap.cc:492] T b85a238bbc3b4621881603a534c7bd9b P 43999d813e564dc688fad116090c95ca: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:23:52.650926 26118 tablet_bootstrap.cc:492] T b85a238bbc3b4621881603a534c7bd9b P 43999d813e564dc688fad116090c95ca: Bootstrap complete.
I20260709 13:23:52.651786 26118 ts_tablet_manager.cc:1403] T b85a238bbc3b4621881603a534c7bd9b P 43999d813e564dc688fad116090c95ca: Time spent bootstrapping tablet: real 0.038s	user 0.020s	sys 0.002s
I20260709 13:23:52.654691 26118 raft_consensus.cc:359] T b85a238bbc3b4621881603a534c7bd9b P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } }
I20260709 13:23:52.655467 26118 raft_consensus.cc:740] T b85a238bbc3b4621881603a534c7bd9b P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43999d813e564dc688fad116090c95ca, State: Initialized, Role: FOLLOWER
I20260709 13:23:52.658716 26118 consensus_queue.cc:260] T b85a238bbc3b4621881603a534c7bd9b P 43999d813e564dc688fad116090c95ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } }
I20260709 13:23:52.661474 26118 ts_tablet_manager.cc:1434] T b85a238bbc3b4621881603a534c7bd9b P 43999d813e564dc688fad116090c95ca: Time spent starting tablet: real 0.009s	user 0.005s	sys 0.000s
I20260709 13:23:52.663545 26118 tablet_bootstrap.cc:492] T b9d550d881b04735bbf89751952c09ec P 43999d813e564dc688fad116090c95ca: Bootstrap starting.
I20260709 13:23:52.688652 26440 raft_consensus.cc:493] T 28c0b6fc9bd64d60828d705c6a9ab1e4 P 43999d813e564dc688fad116090c95ca [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:23:52.689288 26440 raft_consensus.cc:515] T 28c0b6fc9bd64d60828d705c6a9ab1e4 P 43999d813e564dc688fad116090c95ca [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } }
I20260709 13:23:52.692361 26440 leader_election.cc:290] T 28c0b6fc9bd64d60828d705c6a9ab1e4 P 43999d813e564dc688fad116090c95ca [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 233e7d29aac04505bf4e65f0a39a0c4c (127.14.177.130:42957), 3e07d64466934da4bb3c02262b1c2e38 (127.14.177.131:41539)
I20260709 13:23:52.693164 25307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28c0b6fc9bd64d60828d705c6a9ab1e4" candidate_uuid: "43999d813e564dc688fad116090c95ca" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" is_pre_election: true
I20260709 13:23:52.693933 25307 raft_consensus.cc:2468] T 28c0b6fc9bd64d60828d705c6a9ab1e4 P 233e7d29aac04505bf4e65f0a39a0c4c [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 43999d813e564dc688fad116090c95ca in term 2.
I20260709 13:23:52.695350 26114 leader_election.cc:304] T 28c0b6fc9bd64d60828d705c6a9ab1e4 P 43999d813e564dc688fad116090c95ca [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 233e7d29aac04505bf4e65f0a39a0c4c, 43999d813e564dc688fad116090c95ca; no voters: 
I20260709 13:23:52.698335 26440 raft_consensus.cc:2804] T 28c0b6fc9bd64d60828d705c6a9ab1e4 P 43999d813e564dc688fad116090c95ca [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:23:52.698863 26440 raft_consensus.cc:493] T 28c0b6fc9bd64d60828d705c6a9ab1e4 P 43999d813e564dc688fad116090c95ca [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:23:52.703080 26440 raft_consensus.cc:3060] T 28c0b6fc9bd64d60828d705c6a9ab1e4 P 43999d813e564dc688fad116090c95ca [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:23:52.701799 25380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28c0b6fc9bd64d60828d705c6a9ab1e4" candidate_uuid: "43999d813e564dc688fad116090c95ca" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3e07d64466934da4bb3c02262b1c2e38" is_pre_election: true
I20260709 13:23:52.718775 26440 raft_consensus.cc:515] T 28c0b6fc9bd64d60828d705c6a9ab1e4 P 43999d813e564dc688fad116090c95ca [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } }
I20260709 13:23:52.721625 25307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28c0b6fc9bd64d60828d705c6a9ab1e4" candidate_uuid: "43999d813e564dc688fad116090c95ca" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "233e7d29aac04505bf4e65f0a39a0c4c"
I20260709 13:23:52.722188 25307 raft_consensus.cc:3060] T 28c0b6fc9bd64d60828d705c6a9ab1e4 P 233e7d29aac04505bf4e65f0a39a0c4c [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:23:52.726580 26118 tablet_bootstrap.cc:492] T b9d550d881b04735bbf89751952c09ec P 43999d813e564dc688fad116090c95ca: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:23:52.727996 26118 tablet_bootstrap.cc:492] T b9d550d881b04735bbf89751952c09ec P 43999d813e564dc688fad116090c95ca: Bootstrap complete.
I20260709 13:23:52.728667 26118 ts_tablet_manager.cc:1403] T b9d550d881b04735bbf89751952c09ec P 43999d813e564dc688fad116090c95ca: Time spent bootstrapping tablet: real 0.065s	user 0.021s	sys 0.000s
I20260709 13:23:52.729979 25307 raft_consensus.cc:2468] T 28c0b6fc9bd64d60828d705c6a9ab1e4 P 233e7d29aac04505bf4e65f0a39a0c4c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 43999d813e564dc688fad116090c95ca in term 3.
I20260709 13:23:52.731339 26114 leader_election.cc:304] T 28c0b6fc9bd64d60828d705c6a9ab1e4 P 43999d813e564dc688fad116090c95ca [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 233e7d29aac04505bf4e65f0a39a0c4c, 43999d813e564dc688fad116090c95ca; no voters: 
I20260709 13:23:52.731442 26118 raft_consensus.cc:359] T b9d550d881b04735bbf89751952c09ec P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } }
I20260709 13:23:52.732141 26118 raft_consensus.cc:740] T b9d550d881b04735bbf89751952c09ec P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43999d813e564dc688fad116090c95ca, State: Initialized, Role: FOLLOWER
I20260709 13:23:52.732234 26411 raft_consensus.cc:2804] T 28c0b6fc9bd64d60828d705c6a9ab1e4 P 43999d813e564dc688fad116090c95ca [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:23:52.732748 26411 raft_consensus.cc:697] T 28c0b6fc9bd64d60828d705c6a9ab1e4 P 43999d813e564dc688fad116090c95ca [term 3 LEADER]: Becoming Leader. State: Replica: 43999d813e564dc688fad116090c95ca, State: Running, Role: LEADER
I20260709 13:23:52.732996 26118 consensus_queue.cc:260] T b9d550d881b04735bbf89751952c09ec P 43999d813e564dc688fad116090c95ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } }
I20260709 13:23:52.733577 26411 consensus_queue.cc:237] T 28c0b6fc9bd64d60828d705c6a9ab1e4 P 43999d813e564dc688fad116090c95ca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } }
I20260709 13:23:52.742218 25377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28c0b6fc9bd64d60828d705c6a9ab1e4" candidate_uuid: "43999d813e564dc688fad116090c95ca" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3e07d64466934da4bb3c02262b1c2e38"
I20260709 13:23:52.744491 26440 leader_election.cc:290] T 28c0b6fc9bd64d60828d705c6a9ab1e4 P 43999d813e564dc688fad116090c95ca [CANDIDATE]: Term 3 election: Requested vote from peers 233e7d29aac04505bf4e65f0a39a0c4c (127.14.177.130:42957), 3e07d64466934da4bb3c02262b1c2e38 (127.14.177.131:41539)
I20260709 13:23:52.755237 26436 raft_consensus.cc:493] T 91ac744179de48e5bb924b6dfd9f9c9b P 9868dce521294d0b816bba7648f4ae5a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3e07d64466934da4bb3c02262b1c2e38)
I20260709 13:23:52.755899 26436 raft_consensus.cc:515] T 91ac744179de48e5bb924b6dfd9f9c9b P 9868dce521294d0b816bba7648f4ae5a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } }
I20260709 13:23:52.759963 26347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91ac744179de48e5bb924b6dfd9f9c9b" candidate_uuid: "9868dce521294d0b816bba7648f4ae5a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "43999d813e564dc688fad116090c95ca" is_pre_election: true
I20260709 13:23:52.760908 26347 raft_consensus.cc:2393] T 91ac744179de48e5bb924b6dfd9f9c9b P 43999d813e564dc688fad116090c95ca [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9868dce521294d0b816bba7648f4ae5a in current term 2: Already voted for candidate 43999d813e564dc688fad116090c95ca in this term.
I20260709 13:23:52.762095 25380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91ac744179de48e5bb924b6dfd9f9c9b" candidate_uuid: "9868dce521294d0b816bba7648f4ae5a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3e07d64466934da4bb3c02262b1c2e38" is_pre_election: true
I20260709 13:23:52.764432 25134 catalog_manager.cc:5697] T 28c0b6fc9bd64d60828d705c6a9ab1e4 P 43999d813e564dc688fad116090c95ca reported cstate change: term changed from 1 to 3, leader changed from 233e7d29aac04505bf4e65f0a39a0c4c (127.14.177.130) to 43999d813e564dc688fad116090c95ca (127.14.177.129). New cstate: current_term: 3 leader_uuid: "43999d813e564dc688fad116090c95ca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } health_report { overall_health: UNKNOWN } } }
I20260709 13:23:52.762858 25380 raft_consensus.cc:2468] T 91ac744179de48e5bb924b6dfd9f9c9b P 3e07d64466934da4bb3c02262b1c2e38 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9868dce521294d0b816bba7648f4ae5a in term 2.
I20260709 13:23:52.771337 26436 leader_election.cc:290] T 91ac744179de48e5bb924b6dfd9f9c9b P 9868dce521294d0b816bba7648f4ae5a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3e07d64466934da4bb3c02262b1c2e38 (127.14.177.131:41539), 43999d813e564dc688fad116090c95ca (127.14.177.129:38155)
I20260709 13:23:52.777863 26118 ts_tablet_manager.cc:1434] T b9d550d881b04735bbf89751952c09ec P 43999d813e564dc688fad116090c95ca: Time spent starting tablet: real 0.049s	user 0.014s	sys 0.002s
I20260709 13:23:52.778988 26118 tablet_bootstrap.cc:492] T 9e5d70664720424d951f269d9d7f4346 P 43999d813e564dc688fad116090c95ca: Bootstrap starting.
I20260709 13:23:52.780090 26347 raft_consensus.cc:3060] T 1550ed2d2a764c08adffae679599ba41 P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:23:52.791956 25422 leader_election.cc:304] T 91ac744179de48e5bb924b6dfd9f9c9b P 9868dce521294d0b816bba7648f4ae5a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3e07d64466934da4bb3c02262b1c2e38, 9868dce521294d0b816bba7648f4ae5a; no voters: 43999d813e564dc688fad116090c95ca
I20260709 13:23:52.797849 26208 raft_consensus.cc:2804] T 91ac744179de48e5bb924b6dfd9f9c9b P 9868dce521294d0b816bba7648f4ae5a [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:23:52.798307 26208 raft_consensus.cc:493] T 91ac744179de48e5bb924b6dfd9f9c9b P 9868dce521294d0b816bba7648f4ae5a [term 1 FOLLOWER]: Starting leader election (detected failure of leader 3e07d64466934da4bb3c02262b1c2e38)
I20260709 13:23:52.798672 26208 raft_consensus.cc:3060] T 91ac744179de48e5bb924b6dfd9f9c9b P 9868dce521294d0b816bba7648f4ae5a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:23:52.803773 26348 raft_consensus.cc:3060] T 0e71b6f5c69b4b35a8ac51c22c7d5f0d P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:23:52.827093 26345 raft_consensus.cc:3060] T d0a0e4d32ce943fbb773bb840863647b P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:23:52.854604 26208 raft_consensus.cc:515] T 91ac744179de48e5bb924b6dfd9f9c9b P 9868dce521294d0b816bba7648f4ae5a [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } }
I20260709 13:23:52.876829 26411 raft_consensus.cc:493] T 0ed5edbe32044d1c83b0590dbb899801 P 43999d813e564dc688fad116090c95ca [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:23:52.877449 26411 raft_consensus.cc:515] T 0ed5edbe32044d1c83b0590dbb899801 P 43999d813e564dc688fad116090c95ca [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } }
I20260709 13:23:52.880582 25380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ed5edbe32044d1c83b0590dbb899801" candidate_uuid: "43999d813e564dc688fad116090c95ca" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3e07d64466934da4bb3c02262b1c2e38" is_pre_election: true
I20260709 13:23:52.881331 25380 raft_consensus.cc:2468] T 0ed5edbe32044d1c83b0590dbb899801 P 3e07d64466934da4bb3c02262b1c2e38 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 43999d813e564dc688fad116090c95ca in term 2.
I20260709 13:23:52.881695 26411 leader_election.cc:290] T 0ed5edbe32044d1c83b0590dbb899801 P 43999d813e564dc688fad116090c95ca [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3e07d64466934da4bb3c02262b1c2e38 (127.14.177.131:41539), 9868dce521294d0b816bba7648f4ae5a (127.14.177.132:33683)
I20260709 13:23:52.883085 26111 leader_election.cc:304] T 0ed5edbe32044d1c83b0590dbb899801 P 43999d813e564dc688fad116090c95ca [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e07d64466934da4bb3c02262b1c2e38, 43999d813e564dc688fad116090c95ca; no voters: 
I20260709 13:23:52.884120 25454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ed5edbe32044d1c83b0590dbb899801" candidate_uuid: "43999d813e564dc688fad116090c95ca" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9868dce521294d0b816bba7648f4ae5a" is_pre_election: true
I20260709 13:23:52.884987 25377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91ac744179de48e5bb924b6dfd9f9c9b" candidate_uuid: "9868dce521294d0b816bba7648f4ae5a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3e07d64466934da4bb3c02262b1c2e38"
I20260709 13:23:52.884341 26411 raft_consensus.cc:2804] T 0ed5edbe32044d1c83b0590dbb899801 P 43999d813e564dc688fad116090c95ca [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:23:52.887622 26411 raft_consensus.cc:493] T 0ed5edbe32044d1c83b0590dbb899801 P 43999d813e564dc688fad116090c95ca [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:23:52.888175 26411 raft_consensus.cc:3060] T 0ed5edbe32044d1c83b0590dbb899801 P 43999d813e564dc688fad116090c95ca [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:23:52.893528 25377 raft_consensus.cc:2468] T 91ac744179de48e5bb924b6dfd9f9c9b P 3e07d64466934da4bb3c02262b1c2e38 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9868dce521294d0b816bba7648f4ae5a in term 2.
I20260709 13:23:52.900739 26198 raft_consensus.cc:493] T 28c0b6fc9bd64d60828d705c6a9ab1e4 P 3e07d64466934da4bb3c02262b1c2e38 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 233e7d29aac04505bf4e65f0a39a0c4c)
I20260709 13:23:52.931655 26198 raft_consensus.cc:515] T 28c0b6fc9bd64d60828d705c6a9ab1e4 P 3e07d64466934da4bb3c02262b1c2e38 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } }
I20260709 13:23:52.934016 26198 leader_election.cc:290] T 28c0b6fc9bd64d60828d705c6a9ab1e4 P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 43999d813e564dc688fad116090c95ca (127.14.177.129:38155), 233e7d29aac04505bf4e65f0a39a0c4c (127.14.177.130:42957)
W20260709 13:23:52.907294 26458 log.cc:927] Time spent T 1550ed2d2a764c08adffae679599ba41 P 43999d813e564dc688fad116090c95ca: Append to log took a long time: real 0.079s	user 0.001s	sys 0.000s
I20260709 13:23:52.937847 25304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28c0b6fc9bd64d60828d705c6a9ab1e4" candidate_uuid: "3e07d64466934da4bb3c02262b1c2e38" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" is_pre_election: true
I20260709 13:23:52.940186 25304 raft_consensus.cc:2368] T 28c0b6fc9bd64d60828d705c6a9ab1e4 P 233e7d29aac04505bf4e65f0a39a0c4c [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3e07d64466934da4bb3c02262b1c2e38 for earlier term 2. Current term is 3.
I20260709 13:23:52.943291 25351 leader_election.cc:400] T 28c0b6fc9bd64d60828d705c6a9ab1e4 P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 2 pre-election: Vote denied by peer 233e7d29aac04505bf4e65f0a39a0c4c (127.14.177.130:42957) with higher term. Message: Invalid argument: T 28c0b6fc9bd64d60828d705c6a9ab1e4 P 233e7d29aac04505bf4e65f0a39a0c4c [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3e07d64466934da4bb3c02262b1c2e38 for earlier term 2. Current term is 3.
I20260709 13:23:52.943709 25351 leader_election.cc:403] T 28c0b6fc9bd64d60828d705c6a9ab1e4 P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 2 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:23:52.945128 25422 leader_election.cc:304] T 91ac744179de48e5bb924b6dfd9f9c9b P 9868dce521294d0b816bba7648f4ae5a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e07d64466934da4bb3c02262b1c2e38, 9868dce521294d0b816bba7648f4ae5a; no voters: 
I20260709 13:23:52.945508 26198 raft_consensus.cc:3060] T 28c0b6fc9bd64d60828d705c6a9ab1e4 P 3e07d64466934da4bb3c02262b1c2e38 [term 1 FOLLOWER]: Advancing to term 3
I20260709 13:23:52.946343 26355 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28c0b6fc9bd64d60828d705c6a9ab1e4" candidate_uuid: "3e07d64466934da4bb3c02262b1c2e38" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "43999d813e564dc688fad116090c95ca" is_pre_election: true
I20260709 13:23:52.949023 26433 raft_consensus.cc:2804] T 91ac744179de48e5bb924b6dfd9f9c9b P 9868dce521294d0b816bba7648f4ae5a [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:23:52.950683 26433 raft_consensus.cc:697] T 91ac744179de48e5bb924b6dfd9f9c9b P 9868dce521294d0b816bba7648f4ae5a [term 2 LEADER]: Becoming Leader. State: Replica: 9868dce521294d0b816bba7648f4ae5a, State: Running, Role: LEADER
I20260709 13:23:52.951853 26433 consensus_queue.cc:237] T 91ac744179de48e5bb924b6dfd9f9c9b P 9868dce521294d0b816bba7648f4ae5a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } }
I20260709 13:23:52.965178 26198 raft_consensus.cc:2749] T 28c0b6fc9bd64d60828d705c6a9ab1e4 P 3e07d64466934da4bb3c02262b1c2e38 [term 3 FOLLOWER]: Leader pre-election lost for term 2. Reason: Vote denied by peer 233e7d29aac04505bf4e65f0a39a0c4c (127.14.177.130:42957) with higher term. Message: Invalid argument: T 28c0b6fc9bd64d60828d705c6a9ab1e4 P 233e7d29aac04505bf4e65f0a39a0c4c [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3e07d64466934da4bb3c02262b1c2e38 for earlier term 2. Current term is 3.
I20260709 13:23:52.989552 26440 raft_consensus.cc:493] T e9e342ecda4b46619ce12452b2cf7154 P 43999d813e564dc688fad116090c95ca [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:23:52.990209 26440 raft_consensus.cc:515] T e9e342ecda4b46619ce12452b2cf7154 P 43999d813e564dc688fad116090c95ca [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } }
I20260709 13:23:52.992686 26440 leader_election.cc:290] T e9e342ecda4b46619ce12452b2cf7154 P 43999d813e564dc688fad116090c95ca [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9868dce521294d0b816bba7648f4ae5a (127.14.177.132:33683), 233e7d29aac04505bf4e65f0a39a0c4c (127.14.177.130:42957)
I20260709 13:23:52.994562 25304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9e342ecda4b46619ce12452b2cf7154" candidate_uuid: "43999d813e564dc688fad116090c95ca" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" is_pre_election: true
I20260709 13:23:53.009457 26355 raft_consensus.cc:1240] T 6242888a4d4146dcacd87d0fd9a832f4 P 43999d813e564dc688fad116090c95ca [term 2 FOLLOWER]: Rejecting Update request from peer 9868dce521294d0b816bba7648f4ae5a for earlier term 1. Current term is 2. Ops: []
I20260709 13:23:53.032749 26436 consensus_queue.cc:1059] T 6242888a4d4146dcacd87d0fd9a832f4 P 9868dce521294d0b816bba7648f4ae5a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:23:53.034157 26460 raft_consensus.cc:3055] T 6242888a4d4146dcacd87d0fd9a832f4 P 9868dce521294d0b816bba7648f4ae5a [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:23:53.034615 26460 raft_consensus.cc:740] T 6242888a4d4146dcacd87d0fd9a832f4 P 9868dce521294d0b816bba7648f4ae5a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 9868dce521294d0b816bba7648f4ae5a, State: Running, Role: LEADER
I20260709 13:23:53.035507 26460 consensus_queue.cc:260] T 6242888a4d4146dcacd87d0fd9a832f4 P 9868dce521294d0b816bba7648f4ae5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } }
I20260709 13:23:53.036870 26460 raft_consensus.cc:3060] T 6242888a4d4146dcacd87d0fd9a832f4 P 9868dce521294d0b816bba7648f4ae5a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:23:53.051468 25454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9e342ecda4b46619ce12452b2cf7154" candidate_uuid: "43999d813e564dc688fad116090c95ca" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9868dce521294d0b816bba7648f4ae5a" is_pre_election: true
I20260709 13:23:53.052392 25454 raft_consensus.cc:2468] T e9e342ecda4b46619ce12452b2cf7154 P 9868dce521294d0b816bba7648f4ae5a [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 43999d813e564dc688fad116090c95ca in term 2.
I20260709 13:23:53.060885 26286 raft_consensus.cc:493] T 0ed5edbe32044d1c83b0590dbb899801 P 9868dce521294d0b816bba7648f4ae5a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3e07d64466934da4bb3c02262b1c2e38)
I20260709 13:23:53.061542 26286 raft_consensus.cc:515] T 0ed5edbe32044d1c83b0590dbb899801 P 9868dce521294d0b816bba7648f4ae5a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } }
I20260709 13:23:53.063753 26286 leader_election.cc:290] T 0ed5edbe32044d1c83b0590dbb899801 P 9868dce521294d0b816bba7648f4ae5a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3e07d64466934da4bb3c02262b1c2e38 (127.14.177.131:41539), 43999d813e564dc688fad116090c95ca (127.14.177.129:38155)
W20260709 13:23:53.072530 25276 consensus_peers.cc:597] T 51a11db644124b63bdf5f457c967473d P 233e7d29aac04505bf4e65f0a39a0c4c -> Peer 43999d813e564dc688fad116090c95ca (127.14.177.129:38155): Couldn't send request to peer 43999d813e564dc688fad116090c95ca. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 16: this message will repeat every 5th retry.
I20260709 13:23:53.071857 25134 catalog_manager.cc:5697] T 91ac744179de48e5bb924b6dfd9f9c9b P 9868dce521294d0b816bba7648f4ae5a reported cstate change: term changed from 1 to 2, leader changed from 3e07d64466934da4bb3c02262b1c2e38 (127.14.177.131) to 9868dce521294d0b816bba7648f4ae5a (127.14.177.132). New cstate: current_term: 2 leader_uuid: "9868dce521294d0b816bba7648f4ae5a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } health_report { overall_health: UNKNOWN } } }
I20260709 13:23:53.084805 26208 leader_election.cc:290] T 91ac744179de48e5bb924b6dfd9f9c9b P 9868dce521294d0b816bba7648f4ae5a [CANDIDATE]: Term 2 election: Requested vote from peers 3e07d64466934da4bb3c02262b1c2e38 (127.14.177.131:41539), 43999d813e564dc688fad116090c95ca (127.14.177.129:38155)
I20260709 13:23:53.089067 26345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ed5edbe32044d1c83b0590dbb899801" candidate_uuid: "9868dce521294d0b816bba7648f4ae5a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "43999d813e564dc688fad116090c95ca" is_pre_election: true
I20260709 13:23:53.099119 26411 raft_consensus.cc:515] T 0ed5edbe32044d1c83b0590dbb899801 P 43999d813e564dc688fad116090c95ca [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } }
I20260709 13:23:53.105712 26411 leader_election.cc:290] T 0ed5edbe32044d1c83b0590dbb899801 P 43999d813e564dc688fad116090c95ca [CANDIDATE]: Term 3 election: Requested vote from peers 3e07d64466934da4bb3c02262b1c2e38 (127.14.177.131:41539), 9868dce521294d0b816bba7648f4ae5a (127.14.177.132:33683)
I20260709 13:23:53.106314 26345 raft_consensus.cc:2368] T 0ed5edbe32044d1c83b0590dbb899801 P 43999d813e564dc688fad116090c95ca [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9868dce521294d0b816bba7648f4ae5a for earlier term 2. Current term is 3.
I20260709 13:23:53.099936 26346 raft_consensus.cc:3060] T b9d550d881b04735bbf89751952c09ec P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:23:53.101734 26355 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91ac744179de48e5bb924b6dfd9f9c9b" candidate_uuid: "9868dce521294d0b816bba7648f4ae5a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "43999d813e564dc688fad116090c95ca"
I20260709 13:23:53.142520 26355 raft_consensus.cc:2393] T 91ac744179de48e5bb924b6dfd9f9c9b P 43999d813e564dc688fad116090c95ca [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 9868dce521294d0b816bba7648f4ae5a in current term 2: Already voted for candidate 43999d813e564dc688fad116090c95ca in this term.
I20260709 13:23:53.115048 26111 leader_election.cc:304] T e9e342ecda4b46619ce12452b2cf7154 P 43999d813e564dc688fad116090c95ca [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 43999d813e564dc688fad116090c95ca, 9868dce521294d0b816bba7648f4ae5a; no voters: 233e7d29aac04505bf4e65f0a39a0c4c
I20260709 13:23:53.144572 26455 raft_consensus.cc:2804] T e9e342ecda4b46619ce12452b2cf7154 P 43999d813e564dc688fad116090c95ca [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:23:53.144999 26455 raft_consensus.cc:493] T e9e342ecda4b46619ce12452b2cf7154 P 43999d813e564dc688fad116090c95ca [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:23:53.145362 26455 raft_consensus.cc:3060] T e9e342ecda4b46619ce12452b2cf7154 P 43999d813e564dc688fad116090c95ca [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:23:53.145941 25454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ed5edbe32044d1c83b0590dbb899801" candidate_uuid: "43999d813e564dc688fad116090c95ca" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9868dce521294d0b816bba7648f4ae5a"
I20260709 13:23:53.146688 25454 raft_consensus.cc:3060] T 0ed5edbe32044d1c83b0590dbb899801 P 9868dce521294d0b816bba7648f4ae5a [term 1 FOLLOWER]: Advancing to term 3
I20260709 13:23:53.147469 25424 leader_election.cc:400] T 0ed5edbe32044d1c83b0590dbb899801 P 9868dce521294d0b816bba7648f4ae5a [CANDIDATE]: Term 2 pre-election: Vote denied by peer 43999d813e564dc688fad116090c95ca (127.14.177.129:38155) with higher term. Message: Invalid argument: T 0ed5edbe32044d1c83b0590dbb899801 P 43999d813e564dc688fad116090c95ca [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9868dce521294d0b816bba7648f4ae5a for earlier term 2. Current term is 3.
I20260709 13:23:53.147862 25424 leader_election.cc:403] T 0ed5edbe32044d1c83b0590dbb899801 P 9868dce521294d0b816bba7648f4ae5a [CANDIDATE]: Term 2 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:23:53.131126 26118 tablet_bootstrap.cc:492] T 9e5d70664720424d951f269d9d7f4346 P 43999d813e564dc688fad116090c95ca: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:23:53.154233 25382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ed5edbe32044d1c83b0590dbb899801" candidate_uuid: "43999d813e564dc688fad116090c95ca" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3e07d64466934da4bb3c02262b1c2e38"
I20260709 13:23:53.154920 25382 raft_consensus.cc:3060] T 0ed5edbe32044d1c83b0590dbb899801 P 3e07d64466934da4bb3c02262b1c2e38 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:23:53.160518 25307 raft_consensus.cc:1275] T 28c0b6fc9bd64d60828d705c6a9ab1e4 P 233e7d29aac04505bf4e65f0a39a0c4c [term 3 FOLLOWER]: Refusing update from remote peer 43999d813e564dc688fad116090c95ca: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:23:53.162554 26411 consensus_queue.cc:1048] T 28c0b6fc9bd64d60828d705c6a9ab1e4 P 43999d813e564dc688fad116090c95ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:23:53.167578 25377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ed5edbe32044d1c83b0590dbb899801" candidate_uuid: "9868dce521294d0b816bba7648f4ae5a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3e07d64466934da4bb3c02262b1c2e38" is_pre_election: true
I20260709 13:23:53.170964 25382 raft_consensus.cc:2468] T 0ed5edbe32044d1c83b0590dbb899801 P 3e07d64466934da4bb3c02262b1c2e38 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 43999d813e564dc688fad116090c95ca in term 3.
I20260709 13:23:53.172420 26111 leader_election.cc:304] T 0ed5edbe32044d1c83b0590dbb899801 P 43999d813e564dc688fad116090c95ca [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e07d64466934da4bb3c02262b1c2e38, 43999d813e564dc688fad116090c95ca; no voters: 
I20260709 13:23:53.173628 26411 raft_consensus.cc:2804] T 0ed5edbe32044d1c83b0590dbb899801 P 43999d813e564dc688fad116090c95ca [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:23:53.179149 26455 raft_consensus.cc:515] T e9e342ecda4b46619ce12452b2cf7154 P 43999d813e564dc688fad116090c95ca [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } }
W20260709 13:23:53.180114 25350 consensus_peers.cc:597] T b7ff8dde9f5d4c49b9203d902f6b94c4 P 3e07d64466934da4bb3c02262b1c2e38 -> Peer 43999d813e564dc688fad116090c95ca (127.14.177.129:38155): Couldn't send request to peer 43999d813e564dc688fad116090c95ca. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:23:53.174458 26118 tablet_bootstrap.cc:492] T 9e5d70664720424d951f269d9d7f4346 P 43999d813e564dc688fad116090c95ca: Bootstrap complete.
I20260709 13:23:53.183092 26118 ts_tablet_manager.cc:1403] T 9e5d70664720424d951f269d9d7f4346 P 43999d813e564dc688fad116090c95ca: Time spent bootstrapping tablet: real 0.404s	user 0.017s	sys 0.003s
I20260709 13:23:53.191100 26411 raft_consensus.cc:697] T 0ed5edbe32044d1c83b0590dbb899801 P 43999d813e564dc688fad116090c95ca [term 3 LEADER]: Becoming Leader. State: Replica: 43999d813e564dc688fad116090c95ca, State: Running, Role: LEADER
I20260709 13:23:53.192740 25459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9e342ecda4b46619ce12452b2cf7154" candidate_uuid: "43999d813e564dc688fad116090c95ca" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9868dce521294d0b816bba7648f4ae5a"
I20260709 13:23:53.193023 25382 raft_consensus.cc:1275] T 28c0b6fc9bd64d60828d705c6a9ab1e4 P 3e07d64466934da4bb3c02262b1c2e38 [term 3 FOLLOWER]: Refusing update from remote peer 43999d813e564dc688fad116090c95ca: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:23:53.193554 25459 raft_consensus.cc:3060] T e9e342ecda4b46619ce12452b2cf7154 P 9868dce521294d0b816bba7648f4ae5a [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:23:53.194749 26440 consensus_queue.cc:1048] T 28c0b6fc9bd64d60828d705c6a9ab1e4 P 43999d813e564dc688fad116090c95ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:23:53.197245 26455 leader_election.cc:290] T e9e342ecda4b46619ce12452b2cf7154 P 43999d813e564dc688fad116090c95ca [CANDIDATE]: Term 3 election: Requested vote from peers 9868dce521294d0b816bba7648f4ae5a (127.14.177.132:33683), 233e7d29aac04505bf4e65f0a39a0c4c (127.14.177.130:42957)
I20260709 13:23:53.197422 25307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9e342ecda4b46619ce12452b2cf7154" candidate_uuid: "43999d813e564dc688fad116090c95ca" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "233e7d29aac04505bf4e65f0a39a0c4c"
I20260709 13:23:53.208472 25459 raft_consensus.cc:2468] T e9e342ecda4b46619ce12452b2cf7154 P 9868dce521294d0b816bba7648f4ae5a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 43999d813e564dc688fad116090c95ca in term 3.
I20260709 13:23:53.210032 26111 leader_election.cc:304] T e9e342ecda4b46619ce12452b2cf7154 P 43999d813e564dc688fad116090c95ca [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 43999d813e564dc688fad116090c95ca, 9868dce521294d0b816bba7648f4ae5a; no voters: 233e7d29aac04505bf4e65f0a39a0c4c
I20260709 13:23:53.212028 26411 consensus_queue.cc:237] T 0ed5edbe32044d1c83b0590dbb899801 P 43999d813e564dc688fad116090c95ca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } }
I20260709 13:23:53.213598 26455 raft_consensus.cc:2804] T e9e342ecda4b46619ce12452b2cf7154 P 43999d813e564dc688fad116090c95ca [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:23:53.214147 26455 raft_consensus.cc:697] T e9e342ecda4b46619ce12452b2cf7154 P 43999d813e564dc688fad116090c95ca [term 3 LEADER]: Becoming Leader. State: Replica: 43999d813e564dc688fad116090c95ca, State: Running, Role: LEADER
W20260709 13:23:53.214378 25424 consensus_peers.cc:597] T 1f15ec62698c4a8cbd84f2b3d516d2ea P 9868dce521294d0b816bba7648f4ae5a -> Peer 43999d813e564dc688fad116090c95ca (127.14.177.129:38155): Couldn't send request to peer 43999d813e564dc688fad116090c95ca. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 16: this message will repeat every 5th retry.
I20260709 13:23:53.215469 25454 raft_consensus.cc:2468] T 0ed5edbe32044d1c83b0590dbb899801 P 9868dce521294d0b816bba7648f4ae5a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 43999d813e564dc688fad116090c95ca in term 3.
I20260709 13:23:53.216560 26234 raft_consensus.cc:493] T 91ac744179de48e5bb924b6dfd9f9c9b P 43999d813e564dc688fad116090c95ca [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:23:53.215085 26455 consensus_queue.cc:237] T e9e342ecda4b46619ce12452b2cf7154 P 43999d813e564dc688fad116090c95ca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } }
I20260709 13:23:53.217161 26234 raft_consensus.cc:515] T 91ac744179de48e5bb924b6dfd9f9c9b P 43999d813e564dc688fad116090c95ca [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } }
I20260709 13:23:53.216138 26460 raft_consensus.cc:2749] T 0ed5edbe32044d1c83b0590dbb899801 P 9868dce521294d0b816bba7648f4ae5a [term 3 FOLLOWER]: Leader pre-election lost for term 2. Reason: Vote denied by peer 43999d813e564dc688fad116090c95ca (127.14.177.129:38155) with higher term. Message: Invalid argument: T 0ed5edbe32044d1c83b0590dbb899801 P 43999d813e564dc688fad116090c95ca [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9868dce521294d0b816bba7648f4ae5a for earlier term 2. Current term is 3.
I20260709 13:23:53.220748 26234 leader_election.cc:290] T 91ac744179de48e5bb924b6dfd9f9c9b P 43999d813e564dc688fad116090c95ca [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3e07d64466934da4bb3c02262b1c2e38 (127.14.177.131:41539), 9868dce521294d0b816bba7648f4ae5a (127.14.177.132:33683)
I20260709 13:23:53.221485 25380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91ac744179de48e5bb924b6dfd9f9c9b" candidate_uuid: "43999d813e564dc688fad116090c95ca" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3e07d64466934da4bb3c02262b1c2e38" is_pre_election: true
I20260709 13:23:53.222852 25380 raft_consensus.cc:2468] T 91ac744179de48e5bb924b6dfd9f9c9b P 3e07d64466934da4bb3c02262b1c2e38 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 43999d813e564dc688fad116090c95ca in term 2.
I20260709 13:23:53.226209 26111 leader_election.cc:304] T 91ac744179de48e5bb924b6dfd9f9c9b P 43999d813e564dc688fad116090c95ca [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e07d64466934da4bb3c02262b1c2e38, 43999d813e564dc688fad116090c95ca; no voters: 
I20260709 13:23:53.228191 26234 raft_consensus.cc:2804] T 91ac744179de48e5bb924b6dfd9f9c9b P 43999d813e564dc688fad116090c95ca [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:23:53.228920 26234 raft_consensus.cc:493] T 91ac744179de48e5bb924b6dfd9f9c9b P 43999d813e564dc688fad116090c95ca [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:23:53.229357 26234 raft_consensus.cc:3060] T 91ac744179de48e5bb924b6dfd9f9c9b P 43999d813e564dc688fad116090c95ca [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:23:53.231586 25459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91ac744179de48e5bb924b6dfd9f9c9b" candidate_uuid: "43999d813e564dc688fad116090c95ca" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9868dce521294d0b816bba7648f4ae5a" is_pre_election: true
I20260709 13:23:53.230979 26118 raft_consensus.cc:359] T 9e5d70664720424d951f269d9d7f4346 P 43999d813e564dc688fad116090c95ca [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } }
I20260709 13:23:53.256532 26234 raft_consensus.cc:515] T 91ac744179de48e5bb924b6dfd9f9c9b P 43999d813e564dc688fad116090c95ca [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } }
I20260709 13:23:53.269057 26118 raft_consensus.cc:740] T 9e5d70664720424d951f269d9d7f4346 P 43999d813e564dc688fad116090c95ca [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43999d813e564dc688fad116090c95ca, State: Initialized, Role: FOLLOWER
I20260709 13:23:53.258502 26350 raft_consensus.cc:493] T e9e342ecda4b46619ce12452b2cf7154 P 233e7d29aac04505bf4e65f0a39a0c4c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9868dce521294d0b816bba7648f4ae5a)
I20260709 13:23:53.270141 26350 raft_consensus.cc:515] T e9e342ecda4b46619ce12452b2cf7154 P 233e7d29aac04505bf4e65f0a39a0c4c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } }
I20260709 13:23:53.271000 25382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91ac744179de48e5bb924b6dfd9f9c9b" candidate_uuid: "43999d813e564dc688fad116090c95ca" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3e07d64466934da4bb3c02262b1c2e38"
I20260709 13:23:53.271097 26118 consensus_queue.cc:260] T 9e5d70664720424d951f269d9d7f4346 P 43999d813e564dc688fad116090c95ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } }
I20260709 13:23:53.271802 25382 raft_consensus.cc:3060] T 91ac744179de48e5bb924b6dfd9f9c9b P 3e07d64466934da4bb3c02262b1c2e38 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:23:53.275030 26346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9e342ecda4b46619ce12452b2cf7154" candidate_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "43999d813e564dc688fad116090c95ca" is_pre_election: true
I20260709 13:23:53.276948 25459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9e342ecda4b46619ce12452b2cf7154" candidate_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9868dce521294d0b816bba7648f4ae5a" is_pre_election: true
I20260709 13:23:53.277741 25459 raft_consensus.cc:2368] T e9e342ecda4b46619ce12452b2cf7154 P 9868dce521294d0b816bba7648f4ae5a [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 233e7d29aac04505bf4e65f0a39a0c4c for earlier term 2. Current term is 3.
I20260709 13:23:53.279464 25274 leader_election.cc:400] T e9e342ecda4b46619ce12452b2cf7154 P 233e7d29aac04505bf4e65f0a39a0c4c [CANDIDATE]: Term 2 pre-election: Vote denied by peer 9868dce521294d0b816bba7648f4ae5a (127.14.177.132:33683) with higher term. Message: Invalid argument: T e9e342ecda4b46619ce12452b2cf7154 P 9868dce521294d0b816bba7648f4ae5a [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 233e7d29aac04505bf4e65f0a39a0c4c for earlier term 2. Current term is 3.
I20260709 13:23:53.279830 25274 leader_election.cc:403] T e9e342ecda4b46619ce12452b2cf7154 P 233e7d29aac04505bf4e65f0a39a0c4c [CANDIDATE]: Term 2 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:23:53.281025 26429 raft_consensus.cc:3060] T e9e342ecda4b46619ce12452b2cf7154 P 233e7d29aac04505bf4e65f0a39a0c4c [term 1 FOLLOWER]: Advancing to term 3
I20260709 13:23:53.283927 26350 leader_election.cc:290] T e9e342ecda4b46619ce12452b2cf7154 P 233e7d29aac04505bf4e65f0a39a0c4c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9868dce521294d0b816bba7648f4ae5a (127.14.177.132:33683), 43999d813e564dc688fad116090c95ca (127.14.177.129:38155)
I20260709 13:23:53.289700 26234 leader_election.cc:290] T 91ac744179de48e5bb924b6dfd9f9c9b P 43999d813e564dc688fad116090c95ca [CANDIDATE]: Term 3 election: Requested vote from peers 3e07d64466934da4bb3c02262b1c2e38 (127.14.177.131:41539), 9868dce521294d0b816bba7648f4ae5a (127.14.177.132:33683)
I20260709 13:23:53.302635 25459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91ac744179de48e5bb924b6dfd9f9c9b" candidate_uuid: "43999d813e564dc688fad116090c95ca" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9868dce521294d0b816bba7648f4ae5a"
I20260709 13:23:53.307606 25382 raft_consensus.cc:2468] T 91ac744179de48e5bb924b6dfd9f9c9b P 3e07d64466934da4bb3c02262b1c2e38 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 43999d813e564dc688fad116090c95ca in term 3.
I20260709 13:23:53.301827 26429 raft_consensus.cc:2749] T e9e342ecda4b46619ce12452b2cf7154 P 233e7d29aac04505bf4e65f0a39a0c4c [term 3 FOLLOWER]: Leader pre-election lost for term 2. Reason: Vote denied by peer 9868dce521294d0b816bba7648f4ae5a (127.14.177.132:33683) with higher term. Message: Invalid argument: T e9e342ecda4b46619ce12452b2cf7154 P 9868dce521294d0b816bba7648f4ae5a [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 233e7d29aac04505bf4e65f0a39a0c4c for earlier term 2. Current term is 3.
W20260709 13:23:53.317292 25350 consensus_peers.cc:597] T 79d4a12ef58140568c60a24ed4c12f83 P 3e07d64466934da4bb3c02262b1c2e38 -> Peer 43999d813e564dc688fad116090c95ca (127.14.177.129:38155): Couldn't send request to peer 43999d813e564dc688fad116090c95ca. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 16: this message will repeat every 5th retry.
I20260709 13:23:53.312846 26111 leader_election.cc:304] T 91ac744179de48e5bb924b6dfd9f9c9b P 43999d813e564dc688fad116090c95ca [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e07d64466934da4bb3c02262b1c2e38, 43999d813e564dc688fad116090c95ca; no voters: 
I20260709 13:23:53.324193 26440 raft_consensus.cc:2804] T 91ac744179de48e5bb924b6dfd9f9c9b P 43999d813e564dc688fad116090c95ca [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:23:53.324723 26440 raft_consensus.cc:697] T 91ac744179de48e5bb924b6dfd9f9c9b P 43999d813e564dc688fad116090c95ca [term 3 LEADER]: Becoming Leader. State: Replica: 43999d813e564dc688fad116090c95ca, State: Running, Role: LEADER
I20260709 13:23:53.325579 26440 consensus_queue.cc:237] T 91ac744179de48e5bb924b6dfd9f9c9b P 43999d813e564dc688fad116090c95ca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } }
I20260709 13:23:53.335922 26118 ts_tablet_manager.cc:1434] T 9e5d70664720424d951f269d9d7f4346 P 43999d813e564dc688fad116090c95ca: Time spent starting tablet: real 0.152s	user 0.010s	sys 0.004s
I20260709 13:23:53.340338 26118 tablet_bootstrap.cc:492] T 79d4a12ef58140568c60a24ed4c12f83 P 43999d813e564dc688fad116090c95ca: Bootstrap starting.
I20260709 13:23:53.339422 25134 catalog_manager.cc:5697] T 0ed5edbe32044d1c83b0590dbb899801 P 43999d813e564dc688fad116090c95ca reported cstate change: term changed from 1 to 3, leader changed from 3e07d64466934da4bb3c02262b1c2e38 (127.14.177.131) to 43999d813e564dc688fad116090c95ca (127.14.177.129). New cstate: current_term: 3 leader_uuid: "43999d813e564dc688fad116090c95ca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } health_report { overall_health: HEALTHY } } }
I20260709 13:23:53.346235 25134 catalog_manager.cc:5697] T e9e342ecda4b46619ce12452b2cf7154 P 43999d813e564dc688fad116090c95ca reported cstate change: term changed from 1 to 3, leader changed from 9868dce521294d0b816bba7648f4ae5a (127.14.177.132) to 43999d813e564dc688fad116090c95ca (127.14.177.129). New cstate: current_term: 3 leader_uuid: "43999d813e564dc688fad116090c95ca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } health_report { overall_health: UNKNOWN } } }
I20260709 13:23:53.398340 25134 catalog_manager.cc:5697] T 91ac744179de48e5bb924b6dfd9f9c9b P 43999d813e564dc688fad116090c95ca reported cstate change: term changed from 2 to 3, leader changed from 9868dce521294d0b816bba7648f4ae5a (127.14.177.132) to 43999d813e564dc688fad116090c95ca (127.14.177.129). New cstate: current_term: 3 leader_uuid: "43999d813e564dc688fad116090c95ca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } health_report { overall_health: HEALTHY } } }
I20260709 13:23:53.405033 26118 tablet_bootstrap.cc:492] T 79d4a12ef58140568c60a24ed4c12f83 P 43999d813e564dc688fad116090c95ca: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:23:53.406301 26118 tablet_bootstrap.cc:492] T 79d4a12ef58140568c60a24ed4c12f83 P 43999d813e564dc688fad116090c95ca: Bootstrap complete.
I20260709 13:23:53.410957 26118 ts_tablet_manager.cc:1403] T 79d4a12ef58140568c60a24ed4c12f83 P 43999d813e564dc688fad116090c95ca: Time spent bootstrapping tablet: real 0.071s	user 0.015s	sys 0.004s
I20260709 13:23:53.414144 26118 raft_consensus.cc:359] T 79d4a12ef58140568c60a24ed4c12f83 P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } }
I20260709 13:23:53.414958 26118 raft_consensus.cc:740] T 79d4a12ef58140568c60a24ed4c12f83 P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43999d813e564dc688fad116090c95ca, State: Initialized, Role: FOLLOWER
I20260709 13:23:53.415787 26118 consensus_queue.cc:260] T 79d4a12ef58140568c60a24ed4c12f83 P 43999d813e564dc688fad116090c95ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } }
W20260709 13:23:53.417330 25424 consensus_peers.cc:597] T 11949f56648c47e0a43c3140af99a304 P 9868dce521294d0b816bba7648f4ae5a -> Peer 43999d813e564dc688fad116090c95ca (127.14.177.129:38155): Couldn't send request to peer 43999d813e564dc688fad116090c95ca. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 16: this message will repeat every 5th retry.
I20260709 13:23:53.417558 26355 raft_consensus.cc:1240] T 91ac744179de48e5bb924b6dfd9f9c9b P 43999d813e564dc688fad116090c95ca [term 3 LEADER]: Rejecting Update request from peer 9868dce521294d0b816bba7648f4ae5a for earlier term 2. Current term is 3. Ops: []
I20260709 13:23:53.418671 26118 ts_tablet_manager.cc:1434] T 79d4a12ef58140568c60a24ed4c12f83 P 43999d813e564dc688fad116090c95ca: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 13:23:53.419682 26118 tablet_bootstrap.cc:492] T b155b866f10547fd8239579a81ecf5f9 P 43999d813e564dc688fad116090c95ca: Bootstrap starting.
I20260709 13:23:53.419421 26460 consensus_queue.cc:1059] T 91ac744179de48e5bb924b6dfd9f9c9b P 9868dce521294d0b816bba7648f4ae5a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:23:53.420246 26460 consensus_queue.cc:1243] T 91ac744179de48e5bb924b6dfd9f9c9b P 9868dce521294d0b816bba7648f4ae5a [LEADER]: Peer 43999d813e564dc688fad116090c95ca log is divergent from this leader: its last log entry 3.2 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 1
I20260709 13:23:53.420804 26460 raft_consensus.cc:3055] T 91ac744179de48e5bb924b6dfd9f9c9b P 9868dce521294d0b816bba7648f4ae5a [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:23:53.421147 26460 raft_consensus.cc:740] T 91ac744179de48e5bb924b6dfd9f9c9b P 9868dce521294d0b816bba7648f4ae5a [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 9868dce521294d0b816bba7648f4ae5a, State: Running, Role: LEADER
I20260709 13:23:53.422070 26460 consensus_queue.cc:260] T 91ac744179de48e5bb924b6dfd9f9c9b P 9868dce521294d0b816bba7648f4ae5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } }
I20260709 13:23:53.423538 26460 raft_consensus.cc:3060] T 91ac744179de48e5bb924b6dfd9f9c9b P 9868dce521294d0b816bba7648f4ae5a [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:23:53.446300 26118 tablet_bootstrap.cc:492] T b155b866f10547fd8239579a81ecf5f9 P 43999d813e564dc688fad116090c95ca: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:23:53.447609 26118 tablet_bootstrap.cc:492] T b155b866f10547fd8239579a81ecf5f9 P 43999d813e564dc688fad116090c95ca: Bootstrap complete.
I20260709 13:23:53.451283 26118 ts_tablet_manager.cc:1403] T b155b866f10547fd8239579a81ecf5f9 P 43999d813e564dc688fad116090c95ca: Time spent bootstrapping tablet: real 0.032s	user 0.011s	sys 0.012s
I20260709 13:23:53.454649 26118 raft_consensus.cc:359] T b155b866f10547fd8239579a81ecf5f9 P 43999d813e564dc688fad116090c95ca [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } }
I20260709 13:23:53.455312 26118 raft_consensus.cc:740] T b155b866f10547fd8239579a81ecf5f9 P 43999d813e564dc688fad116090c95ca [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43999d813e564dc688fad116090c95ca, State: Initialized, Role: FOLLOWER
I20260709 13:23:53.456060 26118 consensus_queue.cc:260] T b155b866f10547fd8239579a81ecf5f9 P 43999d813e564dc688fad116090c95ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } }
I20260709 13:23:53.458240 26118 ts_tablet_manager.cc:1434] T b155b866f10547fd8239579a81ecf5f9 P 43999d813e564dc688fad116090c95ca: Time spent starting tablet: real 0.006s	user 0.003s	sys 0.000s
I20260709 13:23:53.459532 26118 tablet_bootstrap.cc:492] T c902bbc7ec0848ba82a04870eab1e7f5 P 43999d813e564dc688fad116090c95ca: Bootstrap starting.
I20260709 13:23:53.541002 26355 raft_consensus.cc:1240] T 9e5d70664720424d951f269d9d7f4346 P 43999d813e564dc688fad116090c95ca [term 2 FOLLOWER]: Rejecting Update request from peer 9868dce521294d0b816bba7648f4ae5a for earlier term 1. Current term is 2. Ops: []
I20260709 13:23:53.543663 26460 consensus_queue.cc:1059] T 9e5d70664720424d951f269d9d7f4346 P 9868dce521294d0b816bba7648f4ae5a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:23:53.556775 26286 raft_consensus.cc:3055] T 9e5d70664720424d951f269d9d7f4346 P 9868dce521294d0b816bba7648f4ae5a [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:23:53.557430 26286 raft_consensus.cc:740] T 9e5d70664720424d951f269d9d7f4346 P 9868dce521294d0b816bba7648f4ae5a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 9868dce521294d0b816bba7648f4ae5a, State: Running, Role: LEADER
I20260709 13:23:53.558781 26286 consensus_queue.cc:260] T 9e5d70664720424d951f269d9d7f4346 P 9868dce521294d0b816bba7648f4ae5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } }
I20260709 13:23:53.586907 26286 raft_consensus.cc:3060] T 9e5d70664720424d951f269d9d7f4346 P 9868dce521294d0b816bba7648f4ae5a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:23:53.600473 26118 tablet_bootstrap.cc:492] T c902bbc7ec0848ba82a04870eab1e7f5 P 43999d813e564dc688fad116090c95ca: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:23:53.601789 26118 tablet_bootstrap.cc:492] T c902bbc7ec0848ba82a04870eab1e7f5 P 43999d813e564dc688fad116090c95ca: Bootstrap complete.
I20260709 13:23:53.602669 26118 ts_tablet_manager.cc:1403] T c902bbc7ec0848ba82a04870eab1e7f5 P 43999d813e564dc688fad116090c95ca: Time spent bootstrapping tablet: real 0.143s	user 0.061s	sys 0.052s
I20260709 13:23:53.605499 26118 raft_consensus.cc:359] T c902bbc7ec0848ba82a04870eab1e7f5 P 43999d813e564dc688fad116090c95ca [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } }
I20260709 13:23:53.606271 26118 raft_consensus.cc:740] T c902bbc7ec0848ba82a04870eab1e7f5 P 43999d813e564dc688fad116090c95ca [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43999d813e564dc688fad116090c95ca, State: Initialized, Role: FOLLOWER
I20260709 13:23:53.606884 25304 raft_consensus.cc:1275] T e9e342ecda4b46619ce12452b2cf7154 P 233e7d29aac04505bf4e65f0a39a0c4c [term 3 FOLLOWER]: Refusing update from remote peer 43999d813e564dc688fad116090c95ca: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:23:53.607028 26118 consensus_queue.cc:260] T c902bbc7ec0848ba82a04870eab1e7f5 P 43999d813e564dc688fad116090c95ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } }
I20260709 13:23:53.609447 26118 ts_tablet_manager.cc:1434] T c902bbc7ec0848ba82a04870eab1e7f5 P 43999d813e564dc688fad116090c95ca: Time spent starting tablet: real 0.006s	user 0.002s	sys 0.003s
I20260709 13:23:53.610209 26118 tablet_bootstrap.cc:492] T 51a11db644124b63bdf5f457c967473d P 43999d813e564dc688fad116090c95ca: Bootstrap starting.
I20260709 13:23:53.614635 26440 consensus_queue.cc:1048] T e9e342ecda4b46619ce12452b2cf7154 P 43999d813e564dc688fad116090c95ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:23:53.639642 25459 raft_consensus.cc:1275] T e9e342ecda4b46619ce12452b2cf7154 P 9868dce521294d0b816bba7648f4ae5a [term 3 FOLLOWER]: Refusing update from remote peer 43999d813e564dc688fad116090c95ca: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:23:53.644536 26475 consensus_queue.cc:1048] T e9e342ecda4b46619ce12452b2cf7154 P 43999d813e564dc688fad116090c95ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.002s
I20260709 13:23:53.672894 25454 raft_consensus.cc:1275] T 0ed5edbe32044d1c83b0590dbb899801 P 9868dce521294d0b816bba7648f4ae5a [term 3 FOLLOWER]: Refusing update from remote peer 43999d813e564dc688fad116090c95ca: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:23:53.679657 26475 consensus_queue.cc:1048] T 0ed5edbe32044d1c83b0590dbb899801 P 43999d813e564dc688fad116090c95ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:23:53.681584 26118 tablet_bootstrap.cc:492] T 51a11db644124b63bdf5f457c967473d P 43999d813e564dc688fad116090c95ca: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:23:53.696455 26118 tablet_bootstrap.cc:492] T 51a11db644124b63bdf5f457c967473d P 43999d813e564dc688fad116090c95ca: Bootstrap complete.
I20260709 13:23:53.697335 26118 ts_tablet_manager.cc:1403] T 51a11db644124b63bdf5f457c967473d P 43999d813e564dc688fad116090c95ca: Time spent bootstrapping tablet: real 0.087s	user 0.033s	sys 0.008s
I20260709 13:23:53.731791 26118 raft_consensus.cc:359] T 51a11db644124b63bdf5f457c967473d P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } }
I20260709 13:23:53.732824 26118 raft_consensus.cc:740] T 51a11db644124b63bdf5f457c967473d P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43999d813e564dc688fad116090c95ca, State: Initialized, Role: FOLLOWER
I20260709 13:23:53.733891 26118 consensus_queue.cc:260] T 51a11db644124b63bdf5f457c967473d P 43999d813e564dc688fad116090c95ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } }
I20260709 13:23:53.736685 26118 ts_tablet_manager.cc:1434] T 51a11db644124b63bdf5f457c967473d P 43999d813e564dc688fad116090c95ca: Time spent starting tablet: real 0.039s	user 0.004s	sys 0.002s
I20260709 13:23:53.737812 26118 tablet_bootstrap.cc:492] T 579da2d6aabb4ae9a2b1802e1358068b P 43999d813e564dc688fad116090c95ca: Bootstrap starting.
I20260709 13:23:53.796665 25382 raft_consensus.cc:1275] T 0ed5edbe32044d1c83b0590dbb899801 P 3e07d64466934da4bb3c02262b1c2e38 [term 3 FOLLOWER]: Refusing update from remote peer 43999d813e564dc688fad116090c95ca: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:23:53.798597 26475 consensus_queue.cc:1048] T 0ed5edbe32044d1c83b0590dbb899801 P 43999d813e564dc688fad116090c95ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:23:53.831544 26118 tablet_bootstrap.cc:492] T 579da2d6aabb4ae9a2b1802e1358068b P 43999d813e564dc688fad116090c95ca: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:23:53.832723 26118 tablet_bootstrap.cc:492] T 579da2d6aabb4ae9a2b1802e1358068b P 43999d813e564dc688fad116090c95ca: Bootstrap complete.
I20260709 13:23:53.833622 26118 ts_tablet_manager.cc:1403] T 579da2d6aabb4ae9a2b1802e1358068b P 43999d813e564dc688fad116090c95ca: Time spent bootstrapping tablet: real 0.096s	user 0.022s	sys 0.011s
I20260709 13:23:53.836874 26118 raft_consensus.cc:359] T 579da2d6aabb4ae9a2b1802e1358068b P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } }
I20260709 13:23:53.837770 26118 raft_consensus.cc:740] T 579da2d6aabb4ae9a2b1802e1358068b P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43999d813e564dc688fad116090c95ca, State: Initialized, Role: FOLLOWER
I20260709 13:23:53.839939 26118 consensus_queue.cc:260] T 579da2d6aabb4ae9a2b1802e1358068b P 43999d813e564dc688fad116090c95ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } }
I20260709 13:23:53.842160 26118 ts_tablet_manager.cc:1434] T 579da2d6aabb4ae9a2b1802e1358068b P 43999d813e564dc688fad116090c95ca: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 13:23:53.843048 26118 tablet_bootstrap.cc:492] T b7ff8dde9f5d4c49b9203d902f6b94c4 P 43999d813e564dc688fad116090c95ca: Bootstrap starting.
I20260709 13:23:53.884388 26118 tablet_bootstrap.cc:492] T b7ff8dde9f5d4c49b9203d902f6b94c4 P 43999d813e564dc688fad116090c95ca: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:23:53.885474 26118 tablet_bootstrap.cc:492] T b7ff8dde9f5d4c49b9203d902f6b94c4 P 43999d813e564dc688fad116090c95ca: Bootstrap complete.
I20260709 13:23:53.886178 26118 ts_tablet_manager.cc:1403] T b7ff8dde9f5d4c49b9203d902f6b94c4 P 43999d813e564dc688fad116090c95ca: Time spent bootstrapping tablet: real 0.043s	user 0.015s	sys 0.011s
I20260709 13:23:53.888875 26118 raft_consensus.cc:359] T b7ff8dde9f5d4c49b9203d902f6b94c4 P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } }
I20260709 13:23:53.889580 26118 raft_consensus.cc:740] T b7ff8dde9f5d4c49b9203d902f6b94c4 P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43999d813e564dc688fad116090c95ca, State: Initialized, Role: FOLLOWER
I20260709 13:23:53.890249 26118 consensus_queue.cc:260] T b7ff8dde9f5d4c49b9203d902f6b94c4 P 43999d813e564dc688fad116090c95ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } }
I20260709 13:23:53.892704 26118 ts_tablet_manager.cc:1434] T b7ff8dde9f5d4c49b9203d902f6b94c4 P 43999d813e564dc688fad116090c95ca: Time spent starting tablet: real 0.006s	user 0.008s	sys 0.000s
I20260709 13:23:53.892805 26345 raft_consensus.cc:3060] T b155b866f10547fd8239579a81ecf5f9 P 43999d813e564dc688fad116090c95ca [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:23:53.894129 26118 tablet_bootstrap.cc:492] T 5c7e7acbb09f4d49828fa652ca6abaac P 43999d813e564dc688fad116090c95ca: Bootstrap starting.
I20260709 13:23:53.901952 25454 raft_consensus.cc:1275] T 91ac744179de48e5bb924b6dfd9f9c9b P 9868dce521294d0b816bba7648f4ae5a [term 3 FOLLOWER]: Refusing update from remote peer 43999d813e564dc688fad116090c95ca: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 2. (term mismatch)
I20260709 13:23:53.902464 25454 pending_rounds.cc:85] T 91ac744179de48e5bb924b6dfd9f9c9b P 9868dce521294d0b816bba7648f4ae5a: Aborting all ops after (but not including) 1
I20260709 13:23:53.902725 25454 pending_rounds.cc:107] T 91ac744179de48e5bb924b6dfd9f9c9b P 9868dce521294d0b816bba7648f4ae5a: Aborting uncommitted NO_OP operation due to leader change: 2.2
I20260709 13:23:53.903048 25454 raft_consensus.cc:2889] T 91ac744179de48e5bb924b6dfd9f9c9b P 9868dce521294d0b816bba7648f4ae5a [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:23:53.904995 26411 consensus_queue.cc:1048] T 91ac744179de48e5bb924b6dfd9f9c9b P 43999d813e564dc688fad116090c95ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:23:53.907941 26355 raft_consensus.cc:3060] T b7ff8dde9f5d4c49b9203d902f6b94c4 P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Advancing to term 3
I20260709 13:23:53.928586 25377 raft_consensus.cc:1275] T 91ac744179de48e5bb924b6dfd9f9c9b P 3e07d64466934da4bb3c02262b1c2e38 [term 3 FOLLOWER]: Refusing update from remote peer 43999d813e564dc688fad116090c95ca: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:23:53.935360 26118 tablet_bootstrap.cc:492] T 5c7e7acbb09f4d49828fa652ca6abaac P 43999d813e564dc688fad116090c95ca: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:23:53.942170 26118 tablet_bootstrap.cc:492] T 5c7e7acbb09f4d49828fa652ca6abaac P 43999d813e564dc688fad116090c95ca: Bootstrap complete.
I20260709 13:23:53.942965 26118 ts_tablet_manager.cc:1403] T 5c7e7acbb09f4d49828fa652ca6abaac P 43999d813e564dc688fad116090c95ca: Time spent bootstrapping tablet: real 0.049s	user 0.019s	sys 0.003s
I20260709 13:23:53.947372 26411 consensus_queue.cc:1048] T 91ac744179de48e5bb924b6dfd9f9c9b P 43999d813e564dc688fad116090c95ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:23:53.952723 26118 raft_consensus.cc:359] T 5c7e7acbb09f4d49828fa652ca6abaac P 43999d813e564dc688fad116090c95ca [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } }
I20260709 13:23:53.953686 26118 raft_consensus.cc:740] T 5c7e7acbb09f4d49828fa652ca6abaac P 43999d813e564dc688fad116090c95ca [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43999d813e564dc688fad116090c95ca, State: Initialized, Role: FOLLOWER
I20260709 13:23:53.956640 26118 consensus_queue.cc:260] T 5c7e7acbb09f4d49828fa652ca6abaac P 43999d813e564dc688fad116090c95ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } }
I20260709 13:23:53.959583 26118 ts_tablet_manager.cc:1434] T 5c7e7acbb09f4d49828fa652ca6abaac P 43999d813e564dc688fad116090c95ca: Time spent starting tablet: real 0.016s	user 0.006s	sys 0.001s
I20260709 13:23:53.965009 26118 tablet_bootstrap.cc:492] T 9ffa42c3b2ff48bc87310e9ab5d218ce P 43999d813e564dc688fad116090c95ca: Bootstrap starting.
I20260709 13:23:54.039822 26118 tablet_bootstrap.cc:492] T 9ffa42c3b2ff48bc87310e9ab5d218ce P 43999d813e564dc688fad116090c95ca: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:23:54.040890 26118 tablet_bootstrap.cc:492] T 9ffa42c3b2ff48bc87310e9ab5d218ce P 43999d813e564dc688fad116090c95ca: Bootstrap complete.
I20260709 13:23:54.041563 26118 ts_tablet_manager.cc:1403] T 9ffa42c3b2ff48bc87310e9ab5d218ce P 43999d813e564dc688fad116090c95ca: Time spent bootstrapping tablet: real 0.077s	user 0.033s	sys 0.036s
I20260709 13:23:54.044304 26118 raft_consensus.cc:359] T 9ffa42c3b2ff48bc87310e9ab5d218ce P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } }
I20260709 13:23:54.044991 26118 raft_consensus.cc:740] T 9ffa42c3b2ff48bc87310e9ab5d218ce P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43999d813e564dc688fad116090c95ca, State: Initialized, Role: FOLLOWER
I20260709 13:23:54.045678 26118 consensus_queue.cc:260] T 9ffa42c3b2ff48bc87310e9ab5d218ce P 43999d813e564dc688fad116090c95ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } }
I20260709 13:23:54.047811 26118 ts_tablet_manager.cc:1434] T 9ffa42c3b2ff48bc87310e9ab5d218ce P 43999d813e564dc688fad116090c95ca: Time spent starting tablet: real 0.006s	user 0.007s	sys 0.000s
I20260709 13:23:54.049171 26118 tablet_bootstrap.cc:492] T 11949f56648c47e0a43c3140af99a304 P 43999d813e564dc688fad116090c95ca: Bootstrap starting.
I20260709 13:23:54.053962 26345 raft_consensus.cc:3060] T c902bbc7ec0848ba82a04870eab1e7f5 P 43999d813e564dc688fad116090c95ca [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:23:54.099081 26118 tablet_bootstrap.cc:492] T 11949f56648c47e0a43c3140af99a304 P 43999d813e564dc688fad116090c95ca: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:23:54.100320 26118 tablet_bootstrap.cc:492] T 11949f56648c47e0a43c3140af99a304 P 43999d813e564dc688fad116090c95ca: Bootstrap complete.
I20260709 13:23:54.101080 26118 ts_tablet_manager.cc:1403] T 11949f56648c47e0a43c3140af99a304 P 43999d813e564dc688fad116090c95ca: Time spent bootstrapping tablet: real 0.052s	user 0.017s	sys 0.015s
I20260709 13:23:54.105888 26118 raft_consensus.cc:359] T 11949f56648c47e0a43c3140af99a304 P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } }
I20260709 13:23:54.106601 26118 raft_consensus.cc:740] T 11949f56648c47e0a43c3140af99a304 P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43999d813e564dc688fad116090c95ca, State: Initialized, Role: FOLLOWER
I20260709 13:23:54.107234 26118 consensus_queue.cc:260] T 11949f56648c47e0a43c3140af99a304 P 43999d813e564dc688fad116090c95ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } }
I20260709 13:23:54.109431 26118 ts_tablet_manager.cc:1434] T 11949f56648c47e0a43c3140af99a304 P 43999d813e564dc688fad116090c95ca: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 13:23:54.113121 26118 tablet_bootstrap.cc:492] T 1f15ec62698c4a8cbd84f2b3d516d2ea P 43999d813e564dc688fad116090c95ca: Bootstrap starting.
I20260709 13:23:54.216059 26355 raft_consensus.cc:3060] T 579da2d6aabb4ae9a2b1802e1358068b P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Advancing to term 3
W20260709 13:23:54.230006 25274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:41539, user_credentials={real_user=slave}} blocked reactor thread for 48333.2us
I20260709 13:23:54.251227 26118 tablet_bootstrap.cc:492] T 1f15ec62698c4a8cbd84f2b3d516d2ea P 43999d813e564dc688fad116090c95ca: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:23:54.252279 26118 tablet_bootstrap.cc:492] T 1f15ec62698c4a8cbd84f2b3d516d2ea P 43999d813e564dc688fad116090c95ca: Bootstrap complete.
I20260709 13:23:54.252971 26118 ts_tablet_manager.cc:1403] T 1f15ec62698c4a8cbd84f2b3d516d2ea P 43999d813e564dc688fad116090c95ca: Time spent bootstrapping tablet: real 0.140s	user 0.048s	sys 0.038s
I20260709 13:23:54.262828 26118 raft_consensus.cc:359] T 1f15ec62698c4a8cbd84f2b3d516d2ea P 43999d813e564dc688fad116090c95ca [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } }
I20260709 13:23:54.263626 26118 raft_consensus.cc:740] T 1f15ec62698c4a8cbd84f2b3d516d2ea P 43999d813e564dc688fad116090c95ca [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43999d813e564dc688fad116090c95ca, State: Initialized, Role: FOLLOWER
I20260709 13:23:54.264353 26118 consensus_queue.cc:260] T 1f15ec62698c4a8cbd84f2b3d516d2ea P 43999d813e564dc688fad116090c95ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } }
I20260709 13:23:54.266813 26118 ts_tablet_manager.cc:1434] T 1f15ec62698c4a8cbd84f2b3d516d2ea P 43999d813e564dc688fad116090c95ca: Time spent starting tablet: real 0.014s	user 0.005s	sys 0.000s
I20260709 13:23:54.282194 26345 raft_consensus.cc:3060] T 5c7e7acbb09f4d49828fa652ca6abaac P 43999d813e564dc688fad116090c95ca [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:23:54.386243 26234 raft_consensus.cc:493] T 6242888a4d4146dcacd87d0fd9a832f4 P 43999d813e564dc688fad116090c95ca [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:23:54.386816 26234 raft_consensus.cc:515] T 6242888a4d4146dcacd87d0fd9a832f4 P 43999d813e564dc688fad116090c95ca [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } }
I20260709 13:23:54.389358 26234 leader_election.cc:290] T 6242888a4d4146dcacd87d0fd9a832f4 P 43999d813e564dc688fad116090c95ca [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9868dce521294d0b816bba7648f4ae5a (127.14.177.132:33683), 233e7d29aac04505bf4e65f0a39a0c4c (127.14.177.130:42957)
I20260709 13:23:54.391572 25307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6242888a4d4146dcacd87d0fd9a832f4" candidate_uuid: "43999d813e564dc688fad116090c95ca" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" is_pre_election: true
I20260709 13:23:54.396509 25454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6242888a4d4146dcacd87d0fd9a832f4" candidate_uuid: "43999d813e564dc688fad116090c95ca" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9868dce521294d0b816bba7648f4ae5a" is_pre_election: true
I20260709 13:23:54.397233 25454 raft_consensus.cc:2468] T 6242888a4d4146dcacd87d0fd9a832f4 P 9868dce521294d0b816bba7648f4ae5a [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 43999d813e564dc688fad116090c95ca in term 2.
I20260709 13:23:54.400359 26111 leader_election.cc:304] T 6242888a4d4146dcacd87d0fd9a832f4 P 43999d813e564dc688fad116090c95ca [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 43999d813e564dc688fad116090c95ca, 9868dce521294d0b816bba7648f4ae5a; no voters: 233e7d29aac04505bf4e65f0a39a0c4c
I20260709 13:23:54.402119 26411 raft_consensus.cc:2804] T 6242888a4d4146dcacd87d0fd9a832f4 P 43999d813e564dc688fad116090c95ca [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:23:54.402558 26411 raft_consensus.cc:493] T 6242888a4d4146dcacd87d0fd9a832f4 P 43999d813e564dc688fad116090c95ca [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:23:54.402932 26411 raft_consensus.cc:3060] T 6242888a4d4146dcacd87d0fd9a832f4 P 43999d813e564dc688fad116090c95ca [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:23:54.427044 26411 raft_consensus.cc:515] T 6242888a4d4146dcacd87d0fd9a832f4 P 43999d813e564dc688fad116090c95ca [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } }
I20260709 13:23:54.434944 25459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6242888a4d4146dcacd87d0fd9a832f4" candidate_uuid: "43999d813e564dc688fad116090c95ca" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9868dce521294d0b816bba7648f4ae5a"
I20260709 13:23:54.435789 25459 raft_consensus.cc:3060] T 6242888a4d4146dcacd87d0fd9a832f4 P 9868dce521294d0b816bba7648f4ae5a [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:23:54.445083 25459 raft_consensus.cc:2468] T 6242888a4d4146dcacd87d0fd9a832f4 P 9868dce521294d0b816bba7648f4ae5a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 43999d813e564dc688fad116090c95ca in term 3.
I20260709 13:23:54.445308 26350 raft_consensus.cc:493] T 6242888a4d4146dcacd87d0fd9a832f4 P 233e7d29aac04505bf4e65f0a39a0c4c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9868dce521294d0b816bba7648f4ae5a)
I20260709 13:23:54.445892 26350 raft_consensus.cc:515] T 6242888a4d4146dcacd87d0fd9a832f4 P 233e7d29aac04505bf4e65f0a39a0c4c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } }
I20260709 13:23:54.446763 26111 leader_election.cc:304] T 6242888a4d4146dcacd87d0fd9a832f4 P 43999d813e564dc688fad116090c95ca [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 43999d813e564dc688fad116090c95ca, 9868dce521294d0b816bba7648f4ae5a; no voters: 
I20260709 13:23:54.447818 26234 raft_consensus.cc:2804] T 6242888a4d4146dcacd87d0fd9a832f4 P 43999d813e564dc688fad116090c95ca [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:23:54.448221 26350 leader_election.cc:290] T 6242888a4d4146dcacd87d0fd9a832f4 P 233e7d29aac04505bf4e65f0a39a0c4c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9868dce521294d0b816bba7648f4ae5a (127.14.177.132:33683), 43999d813e564dc688fad116090c95ca (127.14.177.129:38155)
I20260709 13:23:54.448987 25454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6242888a4d4146dcacd87d0fd9a832f4" candidate_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9868dce521294d0b816bba7648f4ae5a" is_pre_election: true
I20260709 13:23:54.449744 25454 raft_consensus.cc:2368] T 6242888a4d4146dcacd87d0fd9a832f4 P 9868dce521294d0b816bba7648f4ae5a [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 233e7d29aac04505bf4e65f0a39a0c4c for earlier term 2. Current term is 3.
I20260709 13:23:54.451154 25274 leader_election.cc:400] T 6242888a4d4146dcacd87d0fd9a832f4 P 233e7d29aac04505bf4e65f0a39a0c4c [CANDIDATE]: Term 2 pre-election: Vote denied by peer 9868dce521294d0b816bba7648f4ae5a (127.14.177.132:33683) with higher term. Message: Invalid argument: T 6242888a4d4146dcacd87d0fd9a832f4 P 9868dce521294d0b816bba7648f4ae5a [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 233e7d29aac04505bf4e65f0a39a0c4c for earlier term 2. Current term is 3.
I20260709 13:23:54.451565 25274 leader_election.cc:403] T 6242888a4d4146dcacd87d0fd9a832f4 P 233e7d29aac04505bf4e65f0a39a0c4c [CANDIDATE]: Term 2 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:23:54.453706 26350 raft_consensus.cc:3060] T 6242888a4d4146dcacd87d0fd9a832f4 P 233e7d29aac04505bf4e65f0a39a0c4c [term 1 FOLLOWER]: Advancing to term 3
I20260709 13:23:54.454221 26355 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6242888a4d4146dcacd87d0fd9a832f4" candidate_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "43999d813e564dc688fad116090c95ca" is_pre_election: true
I20260709 13:23:54.459278 26234 raft_consensus.cc:697] T 6242888a4d4146dcacd87d0fd9a832f4 P 43999d813e564dc688fad116090c95ca [term 3 LEADER]: Becoming Leader. State: Replica: 43999d813e564dc688fad116090c95ca, State: Running, Role: LEADER
I20260709 13:23:54.460518 26234 consensus_queue.cc:237] T 6242888a4d4146dcacd87d0fd9a832f4 P 43999d813e564dc688fad116090c95ca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } }
I20260709 13:23:54.459580 26350 raft_consensus.cc:2749] T 6242888a4d4146dcacd87d0fd9a832f4 P 233e7d29aac04505bf4e65f0a39a0c4c [term 3 FOLLOWER]: Leader pre-election lost for term 2. Reason: Vote denied by peer 9868dce521294d0b816bba7648f4ae5a (127.14.177.132:33683) with higher term. Message: Invalid argument: T 6242888a4d4146dcacd87d0fd9a832f4 P 9868dce521294d0b816bba7648f4ae5a [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 233e7d29aac04505bf4e65f0a39a0c4c for earlier term 2. Current term is 3.
I20260709 13:23:54.470431 26411 leader_election.cc:290] T 6242888a4d4146dcacd87d0fd9a832f4 P 43999d813e564dc688fad116090c95ca [CANDIDATE]: Term 3 election: Requested vote from peers 9868dce521294d0b816bba7648f4ae5a (127.14.177.132:33683), 233e7d29aac04505bf4e65f0a39a0c4c (127.14.177.130:42957)
I20260709 13:23:54.471663 25307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6242888a4d4146dcacd87d0fd9a832f4" candidate_uuid: "43999d813e564dc688fad116090c95ca" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "233e7d29aac04505bf4e65f0a39a0c4c"
W20260709 13:23:54.471768 25350 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.129:38155, user_credentials={real_user=slave}} blocked reactor thread for 54434us
I20260709 13:23:54.488107 25307 raft_consensus.cc:2468] T 6242888a4d4146dcacd87d0fd9a832f4 P 233e7d29aac04505bf4e65f0a39a0c4c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 43999d813e564dc688fad116090c95ca in term 3.
I20260709 13:23:54.505653 25132 catalog_manager.cc:5697] T 6242888a4d4146dcacd87d0fd9a832f4 P 43999d813e564dc688fad116090c95ca reported cstate change: term changed from 1 to 3, leader changed from 9868dce521294d0b816bba7648f4ae5a (127.14.177.132) to 43999d813e564dc688fad116090c95ca (127.14.177.129). New cstate: current_term: 3 leader_uuid: "43999d813e564dc688fad116090c95ca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } health_report { overall_health: UNKNOWN } } }
I20260709 13:23:54.741369 26475 raft_consensus.cc:493] T 9e5d70664720424d951f269d9d7f4346 P 43999d813e564dc688fad116090c95ca [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:23:54.742265 26475 raft_consensus.cc:515] T 9e5d70664720424d951f269d9d7f4346 P 43999d813e564dc688fad116090c95ca [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } }
I20260709 13:23:54.748014 25307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e5d70664720424d951f269d9d7f4346" candidate_uuid: "43999d813e564dc688fad116090c95ca" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" is_pre_election: true
I20260709 13:23:54.746482 26475 leader_election.cc:290] T 9e5d70664720424d951f269d9d7f4346 P 43999d813e564dc688fad116090c95ca [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9868dce521294d0b816bba7648f4ae5a (127.14.177.132:33683), 233e7d29aac04505bf4e65f0a39a0c4c (127.14.177.130:42957)
I20260709 13:23:54.757149 25454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e5d70664720424d951f269d9d7f4346" candidate_uuid: "43999d813e564dc688fad116090c95ca" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9868dce521294d0b816bba7648f4ae5a" is_pre_election: true
I20260709 13:23:54.757848 25454 raft_consensus.cc:2468] T 9e5d70664720424d951f269d9d7f4346 P 9868dce521294d0b816bba7648f4ae5a [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 43999d813e564dc688fad116090c95ca in term 2.
I20260709 13:23:54.775523 26111 leader_election.cc:304] T 9e5d70664720424d951f269d9d7f4346 P 43999d813e564dc688fad116090c95ca [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 43999d813e564dc688fad116090c95ca, 9868dce521294d0b816bba7648f4ae5a; no voters: 233e7d29aac04505bf4e65f0a39a0c4c
I20260709 13:23:54.783442 26234 raft_consensus.cc:2804] T 9e5d70664720424d951f269d9d7f4346 P 43999d813e564dc688fad116090c95ca [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:23:54.783901 26234 raft_consensus.cc:493] T 9e5d70664720424d951f269d9d7f4346 P 43999d813e564dc688fad116090c95ca [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:23:54.784301 26234 raft_consensus.cc:3060] T 9e5d70664720424d951f269d9d7f4346 P 43999d813e564dc688fad116090c95ca [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:23:54.832590 26234 raft_consensus.cc:515] T 9e5d70664720424d951f269d9d7f4346 P 43999d813e564dc688fad116090c95ca [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } }
I20260709 13:23:54.844154 26234 leader_election.cc:290] T 9e5d70664720424d951f269d9d7f4346 P 43999d813e564dc688fad116090c95ca [CANDIDATE]: Term 3 election: Requested vote from peers 9868dce521294d0b816bba7648f4ae5a (127.14.177.132:33683), 233e7d29aac04505bf4e65f0a39a0c4c (127.14.177.130:42957)
I20260709 13:23:54.844391 25307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e5d70664720424d951f269d9d7f4346" candidate_uuid: "43999d813e564dc688fad116090c95ca" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "233e7d29aac04505bf4e65f0a39a0c4c"
I20260709 13:23:54.852387 26350 raft_consensus.cc:493] T 9e5d70664720424d951f269d9d7f4346 P 233e7d29aac04505bf4e65f0a39a0c4c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9868dce521294d0b816bba7648f4ae5a)
I20260709 13:23:54.853125 26350 raft_consensus.cc:515] T 9e5d70664720424d951f269d9d7f4346 P 233e7d29aac04505bf4e65f0a39a0c4c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } }
I20260709 13:23:54.858718 25459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e5d70664720424d951f269d9d7f4346" candidate_uuid: "43999d813e564dc688fad116090c95ca" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9868dce521294d0b816bba7648f4ae5a"
I20260709 13:23:54.859464 25459 raft_consensus.cc:3060] T 9e5d70664720424d951f269d9d7f4346 P 9868dce521294d0b816bba7648f4ae5a [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:23:54.868212 25459 raft_consensus.cc:2468] T 9e5d70664720424d951f269d9d7f4346 P 9868dce521294d0b816bba7648f4ae5a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 43999d813e564dc688fad116090c95ca in term 3.
I20260709 13:23:54.875042 26350 leader_election.cc:290] T 9e5d70664720424d951f269d9d7f4346 P 233e7d29aac04505bf4e65f0a39a0c4c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9868dce521294d0b816bba7648f4ae5a (127.14.177.132:33683), 43999d813e564dc688fad116090c95ca (127.14.177.129:38155)
I20260709 13:23:54.876276 26340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e5d70664720424d951f269d9d7f4346" candidate_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "43999d813e564dc688fad116090c95ca" is_pre_election: true
I20260709 13:23:54.877843 26340 raft_consensus.cc:2368] T 9e5d70664720424d951f269d9d7f4346 P 43999d813e564dc688fad116090c95ca [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 233e7d29aac04505bf4e65f0a39a0c4c for earlier term 2. Current term is 3.
I20260709 13:23:54.879479 25276 leader_election.cc:400] T 9e5d70664720424d951f269d9d7f4346 P 233e7d29aac04505bf4e65f0a39a0c4c [CANDIDATE]: Term 2 pre-election: Vote denied by peer 43999d813e564dc688fad116090c95ca (127.14.177.129:38155) with higher term. Message: Invalid argument: T 9e5d70664720424d951f269d9d7f4346 P 43999d813e564dc688fad116090c95ca [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 233e7d29aac04505bf4e65f0a39a0c4c for earlier term 2. Current term is 3.
I20260709 13:23:54.879837 25276 leader_election.cc:403] T 9e5d70664720424d951f269d9d7f4346 P 233e7d29aac04505bf4e65f0a39a0c4c [CANDIDATE]: Term 2 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:23:54.880641 26350 raft_consensus.cc:3060] T 9e5d70664720424d951f269d9d7f4346 P 233e7d29aac04505bf4e65f0a39a0c4c [term 1 FOLLOWER]: Advancing to term 3
I20260709 13:23:54.889799 26350 raft_consensus.cc:2749] T 9e5d70664720424d951f269d9d7f4346 P 233e7d29aac04505bf4e65f0a39a0c4c [term 3 FOLLOWER]: Leader pre-election lost for term 2. Reason: Vote denied by peer 43999d813e564dc688fad116090c95ca (127.14.177.129:38155) with higher term. Message: Invalid argument: T 9e5d70664720424d951f269d9d7f4346 P 43999d813e564dc688fad116090c95ca [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 233e7d29aac04505bf4e65f0a39a0c4c for earlier term 2. Current term is 3.
I20260709 13:23:54.934269 26111 leader_election.cc:304] T 9e5d70664720424d951f269d9d7f4346 P 43999d813e564dc688fad116090c95ca [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 43999d813e564dc688fad116090c95ca, 9868dce521294d0b816bba7648f4ae5a; no voters: 233e7d29aac04505bf4e65f0a39a0c4c
I20260709 13:23:54.935252 26234 raft_consensus.cc:2804] T 9e5d70664720424d951f269d9d7f4346 P 43999d813e564dc688fad116090c95ca [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:23:54.942413 25459 raft_consensus.cc:1275] T 6242888a4d4146dcacd87d0fd9a832f4 P 9868dce521294d0b816bba7648f4ae5a [term 3 FOLLOWER]: Refusing update from remote peer 43999d813e564dc688fad116090c95ca: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:23:54.944067 25454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e5d70664720424d951f269d9d7f4346" candidate_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9868dce521294d0b816bba7648f4ae5a" is_pre_election: true
I20260709 13:23:54.944916 25454 raft_consensus.cc:2368] T 9e5d70664720424d951f269d9d7f4346 P 9868dce521294d0b816bba7648f4ae5a [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 233e7d29aac04505bf4e65f0a39a0c4c for earlier term 2. Current term is 3.
I20260709 13:23:54.949662 25274 leader_election.cc:400] T 9e5d70664720424d951f269d9d7f4346 P 233e7d29aac04505bf4e65f0a39a0c4c [CANDIDATE]: Term 2 pre-election: Vote denied by peer 9868dce521294d0b816bba7648f4ae5a (127.14.177.132:33683) with higher term. Message: Invalid argument: T 9e5d70664720424d951f269d9d7f4346 P 9868dce521294d0b816bba7648f4ae5a [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 233e7d29aac04505bf4e65f0a39a0c4c for earlier term 2. Current term is 3.
I20260709 13:23:54.951382 26234 raft_consensus.cc:697] T 9e5d70664720424d951f269d9d7f4346 P 43999d813e564dc688fad116090c95ca [term 3 LEADER]: Becoming Leader. State: Replica: 43999d813e564dc688fad116090c95ca, State: Running, Role: LEADER
I20260709 13:23:54.952415 26234 consensus_queue.cc:237] T 9e5d70664720424d951f269d9d7f4346 P 43999d813e564dc688fad116090c95ca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } }
I20260709 13:23:54.954453 26475 consensus_queue.cc:1048] T 6242888a4d4146dcacd87d0fd9a832f4 P 43999d813e564dc688fad116090c95ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:23:54.991611 25307 raft_consensus.cc:1275] T 6242888a4d4146dcacd87d0fd9a832f4 P 233e7d29aac04505bf4e65f0a39a0c4c [term 3 FOLLOWER]: Refusing update from remote peer 43999d813e564dc688fad116090c95ca: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:23:54.994285 26475 consensus_queue.cc:1048] T 6242888a4d4146dcacd87d0fd9a832f4 P 43999d813e564dc688fad116090c95ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:23:55.050484 25134 catalog_manager.cc:5697] T 9e5d70664720424d951f269d9d7f4346 P 43999d813e564dc688fad116090c95ca reported cstate change: term changed from 1 to 3, leader changed from 9868dce521294d0b816bba7648f4ae5a (127.14.177.132) to 43999d813e564dc688fad116090c95ca (127.14.177.129). New cstate: current_term: 3 leader_uuid: "43999d813e564dc688fad116090c95ca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } health_report { overall_health: UNKNOWN } } }
I20260709 13:23:55.548344 25454 raft_consensus.cc:1275] T 9e5d70664720424d951f269d9d7f4346 P 9868dce521294d0b816bba7648f4ae5a [term 3 FOLLOWER]: Refusing update from remote peer 43999d813e564dc688fad116090c95ca: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:23:55.554080 26516 consensus_queue.cc:1048] T 9e5d70664720424d951f269d9d7f4346 P 43999d813e564dc688fad116090c95ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:23:55.585978 25307 raft_consensus.cc:1275] T 9e5d70664720424d951f269d9d7f4346 P 233e7d29aac04505bf4e65f0a39a0c4c [term 3 FOLLOWER]: Refusing update from remote peer 43999d813e564dc688fad116090c95ca: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:23:55.588234 26516 consensus_queue.cc:1048] T 9e5d70664720424d951f269d9d7f4346 P 43999d813e564dc688fad116090c95ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 13:23:59.448760 25351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.130:42957, user_credentials={real_user=slave}} blocked reactor thread for 70283.4us
I20260709 13:24:01.497551 26543 raft_consensus.cc:493] T bbab48ba177049dcba6de106fd889cfb P 9868dce521294d0b816bba7648f4ae5a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 233e7d29aac04505bf4e65f0a39a0c4c)
I20260709 13:24:01.498312 26543 raft_consensus.cc:515] T bbab48ba177049dcba6de106fd889cfb P 9868dce521294d0b816bba7648f4ae5a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } }
I20260709 13:24:01.513511 15046 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:24:01.525557 26538 raft_consensus.cc:493] T 467a6b42ca6548269c12dddde696f7af P 233e7d29aac04505bf4e65f0a39a0c4c [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3e07d64466934da4bb3c02262b1c2e38)
W20260709 13:24:01.538223 25165 debug-util.cc:398] Leaking SignalData structure 0x7b080030fc20 after lost signal to thread 15049
W20260709 13:24:01.541946 25165 debug-util.cc:398] Leaking SignalData structure 0x7b08001033c0 after lost signal to thread 25168
I20260709 13:24:01.541180 26538 raft_consensus.cc:515] T 467a6b42ca6548269c12dddde696f7af P 233e7d29aac04505bf4e65f0a39a0c4c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } }
W20260709 13:24:01.543604 25165 debug-util.cc:398] Leaking SignalData structure 0x7b08003e7d00 after lost signal to thread 25335
W20260709 13:24:01.545212 25165 debug-util.cc:398] Leaking SignalData structure 0x7b0800320940 after lost signal to thread 25409
I20260709 13:24:01.546105 26538 leader_election.cc:290] T 467a6b42ca6548269c12dddde696f7af P 233e7d29aac04505bf4e65f0a39a0c4c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3e07d64466934da4bb3c02262b1c2e38 (127.14.177.131:41539), 9868dce521294d0b816bba7648f4ae5a (127.14.177.132:33683)
I20260709 13:24:01.582067 26527 raft_consensus.cc:493] T 280d65245f4f4358b54bef93e18aca92 P 3e07d64466934da4bb3c02262b1c2e38 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9868dce521294d0b816bba7648f4ae5a)
I20260709 13:24:01.582849 26527 raft_consensus.cc:515] T 280d65245f4f4358b54bef93e18aca92 P 3e07d64466934da4bb3c02262b1c2e38 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } }
W20260709 13:24:01.547147 25165 debug-util.cc:398] Leaking SignalData structure 0x7b0800326880 after lost signal to thread 25483
I20260709 13:24:01.586251 25456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "280d65245f4f4358b54bef93e18aca92" candidate_uuid: "3e07d64466934da4bb3c02262b1c2e38" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9868dce521294d0b816bba7648f4ae5a" is_pre_election: true
W20260709 13:24:01.586869 25165 debug-util.cc:398] Leaking SignalData structure 0x7b0800309b20 after lost signal to thread 26397
I20260709 13:24:01.590811 25458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "467a6b42ca6548269c12dddde696f7af" candidate_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9868dce521294d0b816bba7648f4ae5a" is_pre_election: true
I20260709 13:24:01.588023 26532 raft_consensus.cc:493] T 441d780c3945463f92f31170f31e37a8 P 43999d813e564dc688fad116090c95ca [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 233e7d29aac04505bf4e65f0a39a0c4c)
I20260709 13:24:01.594478 26532 raft_consensus.cc:515] T 441d780c3945463f92f31170f31e37a8 P 43999d813e564dc688fad116090c95ca [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } }
I20260709 13:24:01.593837 26527 leader_election.cc:290] T 280d65245f4f4358b54bef93e18aca92 P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9868dce521294d0b816bba7648f4ae5a (127.14.177.132:33683), 43999d813e564dc688fad116090c95ca (127.14.177.129:38155)
I20260709 13:24:01.597476 26532 leader_election.cc:290] T 441d780c3945463f92f31170f31e37a8 P 43999d813e564dc688fad116090c95ca [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9868dce521294d0b816bba7648f4ae5a (127.14.177.132:33683), 233e7d29aac04505bf4e65f0a39a0c4c (127.14.177.130:42957)
W20260709 13:24:01.558926 25424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.129:38155, user_credentials={real_user=slave}} blocked reactor thread for 49913.8us
I20260709 13:24:01.601893 26532 raft_consensus.cc:493] T 3a3317f4c3514df1b2bb8e44a86be92f P 43999d813e564dc688fad116090c95ca [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 233e7d29aac04505bf4e65f0a39a0c4c)
I20260709 13:24:01.602610 26532 raft_consensus.cc:515] T 3a3317f4c3514df1b2bb8e44a86be92f P 43999d813e564dc688fad116090c95ca [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } }
I20260709 13:24:01.573863 26547 raft_consensus.cc:493] T f1ac2f4d2fe2424bb470df42199219e2 P 3e07d64466934da4bb3c02262b1c2e38 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 9868dce521294d0b816bba7648f4ae5a)
I20260709 13:24:01.607579 26547 raft_consensus.cc:515] T f1ac2f4d2fe2424bb470df42199219e2 P 3e07d64466934da4bb3c02262b1c2e38 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } }
I20260709 13:24:01.608873 26532 leader_election.cc:290] T 3a3317f4c3514df1b2bb8e44a86be92f P 43999d813e564dc688fad116090c95ca [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9868dce521294d0b816bba7648f4ae5a (127.14.177.132:33683), 233e7d29aac04505bf4e65f0a39a0c4c (127.14.177.130:42957)
I20260709 13:24:01.609845 26547 leader_election.cc:290] T f1ac2f4d2fe2424bb470df42199219e2 P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 9868dce521294d0b816bba7648f4ae5a (127.14.177.132:33683), 43999d813e564dc688fad116090c95ca (127.14.177.129:38155)
I20260709 13:24:01.562170 26543 leader_election.cc:290] T bbab48ba177049dcba6de106fd889cfb P 9868dce521294d0b816bba7648f4ae5a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 233e7d29aac04505bf4e65f0a39a0c4c (127.14.177.130:42957), 3e07d64466934da4bb3c02262b1c2e38 (127.14.177.131:41539)
I20260709 13:24:01.581436 26565 raft_consensus.cc:493] T 0ed5edbe32044d1c83b0590dbb899801 P 9868dce521294d0b816bba7648f4ae5a [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 43999d813e564dc688fad116090c95ca)
I20260709 13:24:01.606382 25304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbab48ba177049dcba6de106fd889cfb" candidate_uuid: "9868dce521294d0b816bba7648f4ae5a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" is_pre_election: true
I20260709 13:24:01.613058 26565 raft_consensus.cc:515] T 0ed5edbe32044d1c83b0590dbb899801 P 9868dce521294d0b816bba7648f4ae5a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } }
I20260709 13:24:01.615700 26565 leader_election.cc:290] T 0ed5edbe32044d1c83b0590dbb899801 P 9868dce521294d0b816bba7648f4ae5a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3e07d64466934da4bb3c02262b1c2e38 (127.14.177.131:41539), 43999d813e564dc688fad116090c95ca (127.14.177.129:38155)
I20260709 13:24:01.616640 26547 raft_consensus.cc:493] T cfad575709c943cba40ccab278350770 P 3e07d64466934da4bb3c02262b1c2e38 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 9868dce521294d0b816bba7648f4ae5a)
I20260709 13:24:01.617580 26547 raft_consensus.cc:515] T cfad575709c943cba40ccab278350770 P 3e07d64466934da4bb3c02262b1c2e38 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } }
I20260709 13:24:01.618722 25454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1ac2f4d2fe2424bb470df42199219e2" candidate_uuid: "3e07d64466934da4bb3c02262b1c2e38" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9868dce521294d0b816bba7648f4ae5a" is_pre_election: true
I20260709 13:24:01.619311 26527 raft_consensus.cc:493] T 5c4ab1978a47481c8576d466bacf49d2 P 3e07d64466934da4bb3c02262b1c2e38 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 233e7d29aac04505bf4e65f0a39a0c4c)
I20260709 13:24:01.619946 26527 raft_consensus.cc:515] T 5c4ab1978a47481c8576d466bacf49d2 P 3e07d64466934da4bb3c02262b1c2e38 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } }
I20260709 13:24:01.577117 26561 raft_consensus.cc:493] T ba8a000f5abe4158965449995af84b18 P 3e07d64466934da4bb3c02262b1c2e38 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 233e7d29aac04505bf4e65f0a39a0c4c)
I20260709 13:24:01.621313 26561 raft_consensus.cc:515] T ba8a000f5abe4158965449995af84b18 P 3e07d64466934da4bb3c02262b1c2e38 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } }
I20260709 13:24:01.622469 26527 leader_election.cc:290] T 5c4ab1978a47481c8576d466bacf49d2 P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 233e7d29aac04505bf4e65f0a39a0c4c (127.14.177.130:42957), 9868dce521294d0b816bba7648f4ae5a (127.14.177.132:33683)
I20260709 13:24:01.624130 25454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c4ab1978a47481c8576d466bacf49d2" candidate_uuid: "3e07d64466934da4bb3c02262b1c2e38" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9868dce521294d0b816bba7648f4ae5a" is_pre_election: true
I20260709 13:24:01.625933 26547 leader_election.cc:290] T cfad575709c943cba40ccab278350770 P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 233e7d29aac04505bf4e65f0a39a0c4c (127.14.177.130:42957), 9868dce521294d0b816bba7648f4ae5a (127.14.177.132:33683)
I20260709 13:24:01.626853 25454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfad575709c943cba40ccab278350770" candidate_uuid: "3e07d64466934da4bb3c02262b1c2e38" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "9868dce521294d0b816bba7648f4ae5a" is_pre_election: true
I20260709 13:24:01.628357 25450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba8a000f5abe4158965449995af84b18" candidate_uuid: "3e07d64466934da4bb3c02262b1c2e38" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9868dce521294d0b816bba7648f4ae5a" is_pre_election: true
I20260709 13:24:01.630707 26561 leader_election.cc:290] T ba8a000f5abe4158965449995af84b18 P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 233e7d29aac04505bf4e65f0a39a0c4c (127.14.177.130:42957), 9868dce521294d0b816bba7648f4ae5a (127.14.177.132:33683)
I20260709 13:24:01.632310 26561 raft_consensus.cc:493] T 0d29b499fa4c41f1b974782cd0c9c1c2 P 3e07d64466934da4bb3c02262b1c2e38 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 233e7d29aac04505bf4e65f0a39a0c4c)
I20260709 13:24:01.632400 26355 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "280d65245f4f4358b54bef93e18aca92" candidate_uuid: "3e07d64466934da4bb3c02262b1c2e38" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "43999d813e564dc688fad116090c95ca" is_pre_election: true
I20260709 13:24:01.632467 26348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1ac2f4d2fe2424bb470df42199219e2" candidate_uuid: "3e07d64466934da4bb3c02262b1c2e38" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "43999d813e564dc688fad116090c95ca" is_pre_election: true
I20260709 13:24:01.638991 25459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "441d780c3945463f92f31170f31e37a8" candidate_uuid: "43999d813e564dc688fad116090c95ca" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9868dce521294d0b816bba7648f4ae5a" is_pre_election: true
I20260709 13:24:01.639675 25456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a3317f4c3514df1b2bb8e44a86be92f" candidate_uuid: "43999d813e564dc688fad116090c95ca" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9868dce521294d0b816bba7648f4ae5a" is_pre_election: true
I20260709 13:24:01.640439 26547 raft_consensus.cc:493] T 1550ed2d2a764c08adffae679599ba41 P 3e07d64466934da4bb3c02262b1c2e38 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 9868dce521294d0b816bba7648f4ae5a)
I20260709 13:24:01.641202 26547 raft_consensus.cc:515] T 1550ed2d2a764c08adffae679599ba41 P 3e07d64466934da4bb3c02262b1c2e38 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } }
I20260709 13:24:01.636312 26561 raft_consensus.cc:515] T 0d29b499fa4c41f1b974782cd0c9c1c2 P 3e07d64466934da4bb3c02262b1c2e38 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } }
I20260709 13:24:01.644629 25450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1550ed2d2a764c08adffae679599ba41" candidate_uuid: "3e07d64466934da4bb3c02262b1c2e38" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9868dce521294d0b816bba7648f4ae5a" is_pre_election: true
I20260709 13:24:01.644800 26547 leader_election.cc:290] T 1550ed2d2a764c08adffae679599ba41 P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9868dce521294d0b816bba7648f4ae5a (127.14.177.132:33683), 43999d813e564dc688fad116090c95ca (127.14.177.129:38155)
I20260709 13:24:01.649534 26560 raft_consensus.cc:493] T 13bb3a23315a41c7a40e10b954b15c8e P 233e7d29aac04505bf4e65f0a39a0c4c [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3e07d64466934da4bb3c02262b1c2e38)
I20260709 13:24:01.650141 26560 raft_consensus.cc:515] T 13bb3a23315a41c7a40e10b954b15c8e P 233e7d29aac04505bf4e65f0a39a0c4c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } }
I20260709 13:24:01.652169 25307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a3317f4c3514df1b2bb8e44a86be92f" candidate_uuid: "43999d813e564dc688fad116090c95ca" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" is_pre_election: true
I20260709 13:24:01.652674 26560 leader_election.cc:290] T 13bb3a23315a41c7a40e10b954b15c8e P 233e7d29aac04505bf4e65f0a39a0c4c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 43999d813e564dc688fad116090c95ca (127.14.177.129:38155), 3e07d64466934da4bb3c02262b1c2e38 (127.14.177.131:41539)
I20260709 13:24:01.652899 25304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "441d780c3945463f92f31170f31e37a8" candidate_uuid: "43999d813e564dc688fad116090c95ca" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" is_pre_election: true
I20260709 13:24:01.647697 26561 leader_election.cc:290] T 0d29b499fa4c41f1b974782cd0c9c1c2 P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 43999d813e564dc688fad116090c95ca (127.14.177.129:38155), 233e7d29aac04505bf4e65f0a39a0c4c (127.14.177.130:42957)
I20260709 13:24:01.656708 26538 raft_consensus.cc:493] T 03e9bfbd2f774476a4a377367796a2e8 P 233e7d29aac04505bf4e65f0a39a0c4c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3e07d64466934da4bb3c02262b1c2e38)
I20260709 13:24:01.657415 26569 raft_consensus.cc:493] T 9ab90e3e08fb4528b2a47d995e9e26d3 P 233e7d29aac04505bf4e65f0a39a0c4c [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3e07d64466934da4bb3c02262b1c2e38)
I20260709 13:24:01.657402 26538 raft_consensus.cc:515] T 03e9bfbd2f774476a4a377367796a2e8 P 233e7d29aac04505bf4e65f0a39a0c4c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } }
I20260709 13:24:01.658106 26569 raft_consensus.cc:515] T 9ab90e3e08fb4528b2a47d995e9e26d3 P 233e7d29aac04505bf4e65f0a39a0c4c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } }
I20260709 13:24:01.660132 26538 leader_election.cc:290] T 03e9bfbd2f774476a4a377367796a2e8 P 233e7d29aac04505bf4e65f0a39a0c4c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 43999d813e564dc688fad116090c95ca (127.14.177.129:38155), 3e07d64466934da4bb3c02262b1c2e38 (127.14.177.131:41539)
W20260709 13:24:01.660097 25424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.129:38155, user_credentials={real_user=slave}} blocked reactor thread for 58895.5us
I20260709 13:24:01.661423 26569 leader_election.cc:290] T 9ab90e3e08fb4528b2a47d995e9e26d3 P 233e7d29aac04505bf4e65f0a39a0c4c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3e07d64466934da4bb3c02262b1c2e38 (127.14.177.131:41539), 9868dce521294d0b816bba7648f4ae5a (127.14.177.132:33683)
I20260709 13:24:01.663252 25450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ab90e3e08fb4528b2a47d995e9e26d3" candidate_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9868dce521294d0b816bba7648f4ae5a" is_pre_election: true
I20260709 13:24:01.649039 26562 raft_consensus.cc:493] T 79d4a12ef58140568c60a24ed4c12f83 P 233e7d29aac04505bf4e65f0a39a0c4c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3e07d64466934da4bb3c02262b1c2e38)
I20260709 13:24:01.665237 26562 raft_consensus.cc:515] T 79d4a12ef58140568c60a24ed4c12f83 P 233e7d29aac04505bf4e65f0a39a0c4c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } }
I20260709 13:24:01.669813 26562 leader_election.cc:290] T 79d4a12ef58140568c60a24ed4c12f83 P 233e7d29aac04505bf4e65f0a39a0c4c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 43999d813e564dc688fad116090c95ca (127.14.177.129:38155), 3e07d64466934da4bb3c02262b1c2e38 (127.14.177.131:41539)
I20260709 13:24:01.672634 26340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03e9bfbd2f774476a4a377367796a2e8" candidate_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "43999d813e564dc688fad116090c95ca" is_pre_election: true
I20260709 13:24:01.672943 26345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79d4a12ef58140568c60a24ed4c12f83" candidate_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "43999d813e564dc688fad116090c95ca" is_pre_election: true
I20260709 13:24:01.673110 26348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13bb3a23315a41c7a40e10b954b15c8e" candidate_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "43999d813e564dc688fad116090c95ca" is_pre_election: true
I20260709 13:24:01.680038 26348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ed5edbe32044d1c83b0590dbb899801" candidate_uuid: "9868dce521294d0b816bba7648f4ae5a" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "43999d813e564dc688fad116090c95ca" is_pre_election: true
I20260709 13:24:01.685829 25377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbab48ba177049dcba6de106fd889cfb" candidate_uuid: "9868dce521294d0b816bba7648f4ae5a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3e07d64466934da4bb3c02262b1c2e38" is_pre_election: true
I20260709 13:24:01.690600 25308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c4ab1978a47481c8576d466bacf49d2" candidate_uuid: "3e07d64466934da4bb3c02262b1c2e38" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" is_pre_election: true
I20260709 13:24:01.691380 25380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "467a6b42ca6548269c12dddde696f7af" candidate_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3e07d64466934da4bb3c02262b1c2e38" is_pre_election: true
I20260709 13:24:01.692379 25377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ed5edbe32044d1c83b0590dbb899801" candidate_uuid: "9868dce521294d0b816bba7648f4ae5a" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3e07d64466934da4bb3c02262b1c2e38" is_pre_election: true
I20260709 13:24:01.695066 25376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13bb3a23315a41c7a40e10b954b15c8e" candidate_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3e07d64466934da4bb3c02262b1c2e38" is_pre_election: true
I20260709 13:24:01.695564 25308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfad575709c943cba40ccab278350770" candidate_uuid: "3e07d64466934da4bb3c02262b1c2e38" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" is_pre_election: true
I20260709 13:24:01.695936 25307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba8a000f5abe4158965449995af84b18" candidate_uuid: "3e07d64466934da4bb3c02262b1c2e38" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" is_pre_election: true
I20260709 13:24:01.696738 25376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03e9bfbd2f774476a4a377367796a2e8" candidate_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3e07d64466934da4bb3c02262b1c2e38" is_pre_election: true
I20260709 13:24:01.698364 25376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ab90e3e08fb4528b2a47d995e9e26d3" candidate_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3e07d64466934da4bb3c02262b1c2e38" is_pre_election: true
I20260709 13:24:01.699574 25307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d29b499fa4c41f1b974782cd0c9c1c2" candidate_uuid: "3e07d64466934da4bb3c02262b1c2e38" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" is_pre_election: true
I20260709 13:24:01.699951 25376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79d4a12ef58140568c60a24ed4c12f83" candidate_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3e07d64466934da4bb3c02262b1c2e38" is_pre_election: true
I20260709 13:24:01.707691 26114 leader_election.cc:304] T 3a3317f4c3514df1b2bb8e44a86be92f P 43999d813e564dc688fad116090c95ca [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 43999d813e564dc688fad116090c95ca; no voters: 233e7d29aac04505bf4e65f0a39a0c4c, 9868dce521294d0b816bba7648f4ae5a
I20260709 13:24:01.709259 26564 raft_consensus.cc:2749] T 3a3317f4c3514df1b2bb8e44a86be92f P 43999d813e564dc688fad116090c95ca [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:24:01.713595 26114 leader_election.cc:304] T 441d780c3945463f92f31170f31e37a8 P 43999d813e564dc688fad116090c95ca [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 43999d813e564dc688fad116090c95ca; no voters: 233e7d29aac04505bf4e65f0a39a0c4c, 9868dce521294d0b816bba7648f4ae5a
I20260709 13:24:01.714406 25350 leader_election.cc:304] T f1ac2f4d2fe2424bb470df42199219e2 P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3e07d64466934da4bb3c02262b1c2e38; no voters: 43999d813e564dc688fad116090c95ca, 9868dce521294d0b816bba7648f4ae5a
I20260709 13:24:01.715044 26564 raft_consensus.cc:2749] T 441d780c3945463f92f31170f31e37a8 P 43999d813e564dc688fad116090c95ca [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:24:01.715269 26547 raft_consensus.cc:2749] T f1ac2f4d2fe2424bb470df42199219e2 P 3e07d64466934da4bb3c02262b1c2e38 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:24:01.717797 25350 leader_election.cc:304] T 280d65245f4f4358b54bef93e18aca92 P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3e07d64466934da4bb3c02262b1c2e38; no voters: 43999d813e564dc688fad116090c95ca, 9868dce521294d0b816bba7648f4ae5a
I20260709 13:24:01.720778 26561 raft_consensus.cc:2749] T 280d65245f4f4358b54bef93e18aca92 P 3e07d64466934da4bb3c02262b1c2e38 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:24:01.725346 26348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d29b499fa4c41f1b974782cd0c9c1c2" candidate_uuid: "3e07d64466934da4bb3c02262b1c2e38" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "43999d813e564dc688fad116090c95ca" is_pre_election: true
I20260709 13:24:01.727809 26348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1550ed2d2a764c08adffae679599ba41" candidate_uuid: "3e07d64466934da4bb3c02262b1c2e38" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "43999d813e564dc688fad116090c95ca" is_pre_election: true
I20260709 13:24:01.736541 25351 leader_election.cc:304] T 5c4ab1978a47481c8576d466bacf49d2 P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3e07d64466934da4bb3c02262b1c2e38; no voters: 233e7d29aac04505bf4e65f0a39a0c4c, 9868dce521294d0b816bba7648f4ae5a
I20260709 13:24:01.738013 26561 raft_consensus.cc:2749] T 5c4ab1978a47481c8576d466bacf49d2 P 3e07d64466934da4bb3c02262b1c2e38 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:24:01.739347 25351 leader_election.cc:304] T cfad575709c943cba40ccab278350770 P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3e07d64466934da4bb3c02262b1c2e38; no voters: 233e7d29aac04505bf4e65f0a39a0c4c, 9868dce521294d0b816bba7648f4ae5a
I20260709 13:24:01.740688 26561 raft_consensus.cc:2749] T cfad575709c943cba40ccab278350770 P 3e07d64466934da4bb3c02262b1c2e38 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:24:01.742143 25351 leader_election.cc:304] T ba8a000f5abe4158965449995af84b18 P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3e07d64466934da4bb3c02262b1c2e38; no voters: 233e7d29aac04505bf4e65f0a39a0c4c, 9868dce521294d0b816bba7648f4ae5a
I20260709 13:24:01.743654 26561 raft_consensus.cc:2749] T ba8a000f5abe4158965449995af84b18 P 3e07d64466934da4bb3c02262b1c2e38 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:24:01.747488 25422 leader_election.cc:304] T bbab48ba177049dcba6de106fd889cfb P 9868dce521294d0b816bba7648f4ae5a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9868dce521294d0b816bba7648f4ae5a; no voters: 233e7d29aac04505bf4e65f0a39a0c4c, 3e07d64466934da4bb3c02262b1c2e38
I20260709 13:24:01.748558 26555 raft_consensus.cc:2749] T bbab48ba177049dcba6de106fd889cfb P 9868dce521294d0b816bba7648f4ae5a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:24:01.748718 25274 leader_election.cc:304] T 467a6b42ca6548269c12dddde696f7af P 233e7d29aac04505bf4e65f0a39a0c4c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 233e7d29aac04505bf4e65f0a39a0c4c; no voters: 3e07d64466934da4bb3c02262b1c2e38, 9868dce521294d0b816bba7648f4ae5a
I20260709 13:24:01.749723 25350 leader_election.cc:304] T 0d29b499fa4c41f1b974782cd0c9c1c2 P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3e07d64466934da4bb3c02262b1c2e38; no voters: 233e7d29aac04505bf4e65f0a39a0c4c, 43999d813e564dc688fad116090c95ca
I20260709 13:24:01.750489 26562 raft_consensus.cc:2749] T 467a6b42ca6548269c12dddde696f7af P 233e7d29aac04505bf4e65f0a39a0c4c [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:24:01.751355 25350 leader_election.cc:304] T 1550ed2d2a764c08adffae679599ba41 P 3e07d64466934da4bb3c02262b1c2e38 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3e07d64466934da4bb3c02262b1c2e38; no voters: 43999d813e564dc688fad116090c95ca, 9868dce521294d0b816bba7648f4ae5a
I20260709 13:24:01.751482 25422 leader_election.cc:304] T 0ed5edbe32044d1c83b0590dbb899801 P 9868dce521294d0b816bba7648f4ae5a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9868dce521294d0b816bba7648f4ae5a; no voters: 3e07d64466934da4bb3c02262b1c2e38, 43999d813e564dc688fad116090c95ca
I20260709 13:24:01.751937 26561 raft_consensus.cc:2749] T 0d29b499fa4c41f1b974782cd0c9c1c2 P 3e07d64466934da4bb3c02262b1c2e38 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:24:01.752243 26547 raft_consensus.cc:2749] T 1550ed2d2a764c08adffae679599ba41 P 3e07d64466934da4bb3c02262b1c2e38 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:24:01.753295 26543 raft_consensus.cc:2749] T 0ed5edbe32044d1c83b0590dbb899801 P 9868dce521294d0b816bba7648f4ae5a [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:24:01.756485 25274 leader_election.cc:304] T 13bb3a23315a41c7a40e10b954b15c8e P 233e7d29aac04505bf4e65f0a39a0c4c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 233e7d29aac04505bf4e65f0a39a0c4c; no voters: 3e07d64466934da4bb3c02262b1c2e38, 43999d813e564dc688fad116090c95ca
I20260709 13:24:01.757503 26558 raft_consensus.cc:2749] T 13bb3a23315a41c7a40e10b954b15c8e P 233e7d29aac04505bf4e65f0a39a0c4c [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:24:01.757843 25274 leader_election.cc:304] T 03e9bfbd2f774476a4a377367796a2e8 P 233e7d29aac04505bf4e65f0a39a0c4c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 233e7d29aac04505bf4e65f0a39a0c4c; no voters: 3e07d64466934da4bb3c02262b1c2e38, 43999d813e564dc688fad116090c95ca
I20260709 13:24:01.758754 26558 raft_consensus.cc:2749] T 03e9bfbd2f774476a4a377367796a2e8 P 233e7d29aac04505bf4e65f0a39a0c4c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:24:01.759313 25274 leader_election.cc:304] T 9ab90e3e08fb4528b2a47d995e9e26d3 P 233e7d29aac04505bf4e65f0a39a0c4c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 233e7d29aac04505bf4e65f0a39a0c4c; no voters: 3e07d64466934da4bb3c02262b1c2e38, 9868dce521294d0b816bba7648f4ae5a
I20260709 13:24:01.760277 26558 raft_consensus.cc:2749] T 9ab90e3e08fb4528b2a47d995e9e26d3 P 233e7d29aac04505bf4e65f0a39a0c4c [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:24:01.761772 25274 leader_election.cc:304] T 79d4a12ef58140568c60a24ed4c12f83 P 233e7d29aac04505bf4e65f0a39a0c4c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 233e7d29aac04505bf4e65f0a39a0c4c; no voters: 3e07d64466934da4bb3c02262b1c2e38, 43999d813e564dc688fad116090c95ca
I20260709 13:24:01.762558 26558 raft_consensus.cc:2749] T 79d4a12ef58140568c60a24ed4c12f83 P 233e7d29aac04505bf4e65f0a39a0c4c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:24:01.772436 25376 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "79d4a12ef58140568c60a24ed4c12f83"
dest_uuid: "3e07d64466934da4bb3c02262b1c2e38"
mode: GRACEFUL
new_leader_uuid: "43999d813e564dc688fad116090c95ca"
 from {username='slave'} at 127.0.0.1:40012
I20260709 13:24:01.773231 25376 raft_consensus.cc:606] T 79d4a12ef58140568c60a24ed4c12f83 P 3e07d64466934da4bb3c02262b1c2e38 [term 1 LEADER]: Received request to transfer leadership to TS 43999d813e564dc688fad116090c95ca
I20260709 13:24:01.776108 25376 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b155b866f10547fd8239579a81ecf5f9"
dest_uuid: "3e07d64466934da4bb3c02262b1c2e38"
mode: GRACEFUL
new_leader_uuid: "43999d813e564dc688fad116090c95ca"
 from {username='slave'} at 127.0.0.1:40012
I20260709 13:24:01.776652 25376 raft_consensus.cc:606] T b155b866f10547fd8239579a81ecf5f9 P 3e07d64466934da4bb3c02262b1c2e38 [term 3 LEADER]: Received request to transfer leadership to TS 43999d813e564dc688fad116090c95ca
I20260709 13:24:01.779026 25376 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b7ff8dde9f5d4c49b9203d902f6b94c4"
dest_uuid: "3e07d64466934da4bb3c02262b1c2e38"
mode: GRACEFUL
new_leader_uuid: "43999d813e564dc688fad116090c95ca"
 from {username='slave'} at 127.0.0.1:40012
I20260709 13:24:01.779551 25376 raft_consensus.cc:606] T b7ff8dde9f5d4c49b9203d902f6b94c4 P 3e07d64466934da4bb3c02262b1c2e38 [term 3 LEADER]: Received request to transfer leadership to TS 43999d813e564dc688fad116090c95ca
I20260709 13:24:01.791568 25450 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d053b9ae38de4173a0aded4da494bd98"
dest_uuid: "9868dce521294d0b816bba7648f4ae5a"
mode: GRACEFUL
new_leader_uuid: "43999d813e564dc688fad116090c95ca"
 from {username='slave'} at 127.0.0.1:35958
I20260709 13:24:01.792250 25450 raft_consensus.cc:606] T d053b9ae38de4173a0aded4da494bd98 P 9868dce521294d0b816bba7648f4ae5a [term 1 LEADER]: Received request to transfer leadership to TS 43999d813e564dc688fad116090c95ca
I20260709 13:24:01.796479 25376 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e15820fa8a7d4e1fb6ad23a64ca22f90"
dest_uuid: "3e07d64466934da4bb3c02262b1c2e38"
mode: GRACEFUL
new_leader_uuid: "43999d813e564dc688fad116090c95ca"
 from {username='slave'} at 127.0.0.1:40012
I20260709 13:24:01.797087 25376 raft_consensus.cc:606] T e15820fa8a7d4e1fb6ad23a64ca22f90 P 3e07d64466934da4bb3c02262b1c2e38 [term 1 LEADER]: Received request to transfer leadership to TS 43999d813e564dc688fad116090c95ca
I20260709 13:24:01.798382 15046 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:24:01.800338 15046 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:24:01.861959 26547 raft_consensus.cc:993] T b7ff8dde9f5d4c49b9203d902f6b94c4 P 3e07d64466934da4bb3c02262b1c2e38: : Instructing follower 43999d813e564dc688fad116090c95ca to start an election
I20260709 13:24:01.862612 26527 raft_consensus.cc:1081] T b7ff8dde9f5d4c49b9203d902f6b94c4 P 3e07d64466934da4bb3c02262b1c2e38 [term 3 LEADER]: Signalling peer 43999d813e564dc688fad116090c95ca to start an election
I20260709 13:24:01.862955 26547 raft_consensus.cc:993] T 79d4a12ef58140568c60a24ed4c12f83 P 3e07d64466934da4bb3c02262b1c2e38: : Instructing follower 43999d813e564dc688fad116090c95ca to start an election
I20260709 13:24:01.863554 26561 raft_consensus.cc:1081] T 79d4a12ef58140568c60a24ed4c12f83 P 3e07d64466934da4bb3c02262b1c2e38 [term 1 LEADER]: Signalling peer 43999d813e564dc688fad116090c95ca to start an election
I20260709 13:24:01.864334 26348 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b7ff8dde9f5d4c49b9203d902f6b94c4"
dest_uuid: "43999d813e564dc688fad116090c95ca"
 from {username='slave'} at 127.0.0.1:48840
I20260709 13:24:01.865075 26348 raft_consensus.cc:493] T b7ff8dde9f5d4c49b9203d902f6b94c4 P 43999d813e564dc688fad116090c95ca [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:24:01.865518 26348 raft_consensus.cc:3060] T b7ff8dde9f5d4c49b9203d902f6b94c4 P 43999d813e564dc688fad116090c95ca [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:24:01.866855 26345 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "79d4a12ef58140568c60a24ed4c12f83"
dest_uuid: "43999d813e564dc688fad116090c95ca"
 from {username='slave'} at 127.0.0.1:48840
I20260709 13:24:01.867542 26345 raft_consensus.cc:493] T 79d4a12ef58140568c60a24ed4c12f83 P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:24:01.867929 26345 raft_consensus.cc:3060] T 79d4a12ef58140568c60a24ed4c12f83 P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:24:01.875229 26348 raft_consensus.cc:515] T b7ff8dde9f5d4c49b9203d902f6b94c4 P 43999d813e564dc688fad116090c95ca [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } }
I20260709 13:24:01.878086 25307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7ff8dde9f5d4c49b9203d902f6b94c4" candidate_uuid: "43999d813e564dc688fad116090c95ca" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: true dest_uuid: "233e7d29aac04505bf4e65f0a39a0c4c"
I20260709 13:24:01.878677 25307 raft_consensus.cc:3060] T b7ff8dde9f5d4c49b9203d902f6b94c4 P 233e7d29aac04505bf4e65f0a39a0c4c [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:24:01.878525 26345 raft_consensus.cc:515] T 79d4a12ef58140568c60a24ed4c12f83 P 43999d813e564dc688fad116090c95ca [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } }
I20260709 13:24:01.881322 26345 leader_election.cc:290] T 79d4a12ef58140568c60a24ed4c12f83 P 43999d813e564dc688fad116090c95ca [CANDIDATE]: Term 2 election: Requested vote from peers 233e7d29aac04505bf4e65f0a39a0c4c (127.14.177.130:42957), 3e07d64466934da4bb3c02262b1c2e38 (127.14.177.131:41539)
I20260709 13:24:01.882048 25376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79d4a12ef58140568c60a24ed4c12f83" candidate_uuid: "43999d813e564dc688fad116090c95ca" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3e07d64466934da4bb3c02262b1c2e38"
I20260709 13:24:01.882898 25376 raft_consensus.cc:3055] T 79d4a12ef58140568c60a24ed4c12f83 P 3e07d64466934da4bb3c02262b1c2e38 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:24:01.883306 25376 raft_consensus.cc:740] T 79d4a12ef58140568c60a24ed4c12f83 P 3e07d64466934da4bb3c02262b1c2e38 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3e07d64466934da4bb3c02262b1c2e38, State: Running, Role: LEADER
I20260709 13:24:01.883986 25308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79d4a12ef58140568c60a24ed4c12f83" candidate_uuid: "43999d813e564dc688fad116090c95ca" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "233e7d29aac04505bf4e65f0a39a0c4c"
I20260709 13:24:01.884661 25308 raft_consensus.cc:3060] T 79d4a12ef58140568c60a24ed4c12f83 P 233e7d29aac04505bf4e65f0a39a0c4c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:24:01.884310 25376 consensus_queue.cc:260] T 79d4a12ef58140568c60a24ed4c12f83 P 3e07d64466934da4bb3c02262b1c2e38 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } }
I20260709 13:24:01.890697 25307 raft_consensus.cc:2468] T b7ff8dde9f5d4c49b9203d902f6b94c4 P 233e7d29aac04505bf4e65f0a39a0c4c [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 43999d813e564dc688fad116090c95ca in term 4.
I20260709 13:24:01.893003 25376 raft_consensus.cc:3060] T 79d4a12ef58140568c60a24ed4c12f83 P 3e07d64466934da4bb3c02262b1c2e38 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:24:01.894369 26114 leader_election.cc:304] T b7ff8dde9f5d4c49b9203d902f6b94c4 P 43999d813e564dc688fad116090c95ca [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 233e7d29aac04505bf4e65f0a39a0c4c, 43999d813e564dc688fad116090c95ca; no voters: 
I20260709 13:24:01.895800 26564 raft_consensus.cc:2804] T b7ff8dde9f5d4c49b9203d902f6b94c4 P 43999d813e564dc688fad116090c95ca [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:24:01.899281 26564 raft_consensus.cc:697] T b7ff8dde9f5d4c49b9203d902f6b94c4 P 43999d813e564dc688fad116090c95ca [term 4 LEADER]: Becoming Leader. State: Replica: 43999d813e564dc688fad116090c95ca, State: Running, Role: LEADER
I20260709 13:24:01.900812 26564 consensus_queue.cc:237] T b7ff8dde9f5d4c49b9203d902f6b94c4 P 43999d813e564dc688fad116090c95ca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } }
I20260709 13:24:01.910265 26552 raft_consensus.cc:993] T b155b866f10547fd8239579a81ecf5f9 P 3e07d64466934da4bb3c02262b1c2e38: : Instructing follower 43999d813e564dc688fad116090c95ca to start an election
I20260709 13:24:01.910844 26552 raft_consensus.cc:1081] T b155b866f10547fd8239579a81ecf5f9 P 3e07d64466934da4bb3c02262b1c2e38 [term 3 LEADER]: Signalling peer 43999d813e564dc688fad116090c95ca to start an election
I20260709 13:24:01.913501 26345 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b155b866f10547fd8239579a81ecf5f9"
dest_uuid: "43999d813e564dc688fad116090c95ca"
 from {username='slave'} at 127.0.0.1:48840
I20260709 13:24:01.914669 26345 raft_consensus.cc:493] T b155b866f10547fd8239579a81ecf5f9 P 43999d813e564dc688fad116090c95ca [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:24:01.919817 26345 raft_consensus.cc:3060] T b155b866f10547fd8239579a81ecf5f9 P 43999d813e564dc688fad116090c95ca [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:24:01.925877 26348 leader_election.cc:290] T b7ff8dde9f5d4c49b9203d902f6b94c4 P 43999d813e564dc688fad116090c95ca [CANDIDATE]: Term 4 election: Requested vote from peers 233e7d29aac04505bf4e65f0a39a0c4c (127.14.177.130:42957), 3e07d64466934da4bb3c02262b1c2e38 (127.14.177.131:41539)
I20260709 13:24:01.927119 25377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7ff8dde9f5d4c49b9203d902f6b94c4" candidate_uuid: "43999d813e564dc688fad116090c95ca" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: true dest_uuid: "3e07d64466934da4bb3c02262b1c2e38"
I20260709 13:24:01.927788 25377 raft_consensus.cc:3055] T b7ff8dde9f5d4c49b9203d902f6b94c4 P 3e07d64466934da4bb3c02262b1c2e38 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:24:01.928100 25377 raft_consensus.cc:740] T b7ff8dde9f5d4c49b9203d902f6b94c4 P 3e07d64466934da4bb3c02262b1c2e38 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 3e07d64466934da4bb3c02262b1c2e38, State: Running, Role: LEADER
I20260709 13:24:01.899255 25308 raft_consensus.cc:2468] T 79d4a12ef58140568c60a24ed4c12f83 P 233e7d29aac04505bf4e65f0a39a0c4c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 43999d813e564dc688fad116090c95ca in term 2.
I20260709 13:24:01.931658 25377 consensus_queue.cc:260] T b7ff8dde9f5d4c49b9203d902f6b94c4 P 3e07d64466934da4bb3c02262b1c2e38 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } }
I20260709 13:24:01.932822 25377 raft_consensus.cc:3060] T b7ff8dde9f5d4c49b9203d902f6b94c4 P 3e07d64466934da4bb3c02262b1c2e38 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:24:01.933949 26114 leader_election.cc:304] T 79d4a12ef58140568c60a24ed4c12f83 P 43999d813e564dc688fad116090c95ca [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 233e7d29aac04505bf4e65f0a39a0c4c, 43999d813e564dc688fad116090c95ca; no voters: 
I20260709 13:24:01.946713 26564 raft_consensus.cc:2804] T 79d4a12ef58140568c60a24ed4c12f83 P 43999d813e564dc688fad116090c95ca [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:24:01.947796 26564 raft_consensus.cc:697] T 79d4a12ef58140568c60a24ed4c12f83 P 43999d813e564dc688fad116090c95ca [term 2 LEADER]: Becoming Leader. State: Replica: 43999d813e564dc688fad116090c95ca, State: Running, Role: LEADER
I20260709 13:24:01.948974 26564 consensus_queue.cc:237] T 79d4a12ef58140568c60a24ed4c12f83 P 43999d813e564dc688fad116090c95ca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } }
I20260709 13:24:01.954527 26345 raft_consensus.cc:515] T b155b866f10547fd8239579a81ecf5f9 P 43999d813e564dc688fad116090c95ca [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } }
I20260709 13:24:01.963693 26345 leader_election.cc:290] T b155b866f10547fd8239579a81ecf5f9 P 43999d813e564dc688fad116090c95ca [CANDIDATE]: Term 4 election: Requested vote from peers 233e7d29aac04505bf4e65f0a39a0c4c (127.14.177.130:42957), 3e07d64466934da4bb3c02262b1c2e38 (127.14.177.131:41539)
I20260709 13:24:01.957906 25377 raft_consensus.cc:2468] T b7ff8dde9f5d4c49b9203d902f6b94c4 P 3e07d64466934da4bb3c02262b1c2e38 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 43999d813e564dc688fad116090c95ca in term 4.
I20260709 13:24:01.965323 25308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b155b866f10547fd8239579a81ecf5f9" candidate_uuid: "43999d813e564dc688fad116090c95ca" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: true dest_uuid: "233e7d29aac04505bf4e65f0a39a0c4c"
I20260709 13:24:01.972041 25308 raft_consensus.cc:3060] T b155b866f10547fd8239579a81ecf5f9 P 233e7d29aac04505bf4e65f0a39a0c4c [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:24:01.972282 26556 raft_consensus.cc:993] T d053b9ae38de4173a0aded4da494bd98 P 9868dce521294d0b816bba7648f4ae5a: : Instructing follower 43999d813e564dc688fad116090c95ca to start an election
I20260709 13:24:01.972873 26556 raft_consensus.cc:1081] T d053b9ae38de4173a0aded4da494bd98 P 9868dce521294d0b816bba7648f4ae5a [term 1 LEADER]: Signalling peer 43999d813e564dc688fad116090c95ca to start an election
I20260709 13:24:01.976163 26348 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d053b9ae38de4173a0aded4da494bd98"
dest_uuid: "43999d813e564dc688fad116090c95ca"
 from {username='slave'} at 127.0.0.1:48848
I20260709 13:24:01.976773 26348 raft_consensus.cc:493] T d053b9ae38de4173a0aded4da494bd98 P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:24:01.977160 26348 raft_consensus.cc:3060] T d053b9ae38de4173a0aded4da494bd98 P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:24:01.984782 25308 raft_consensus.cc:2468] T b155b866f10547fd8239579a81ecf5f9 P 233e7d29aac04505bf4e65f0a39a0c4c [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 43999d813e564dc688fad116090c95ca in term 4.
I20260709 13:24:01.992357 26114 leader_election.cc:304] T b155b866f10547fd8239579a81ecf5f9 P 43999d813e564dc688fad116090c95ca [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 233e7d29aac04505bf4e65f0a39a0c4c, 43999d813e564dc688fad116090c95ca; no voters: 
I20260709 13:24:01.959669 25376 raft_consensus.cc:2468] T 79d4a12ef58140568c60a24ed4c12f83 P 3e07d64466934da4bb3c02262b1c2e38 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 43999d813e564dc688fad116090c95ca in term 2.
I20260709 13:24:01.969676 25379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b155b866f10547fd8239579a81ecf5f9" candidate_uuid: "43999d813e564dc688fad116090c95ca" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: true dest_uuid: "3e07d64466934da4bb3c02262b1c2e38"
I20260709 13:24:01.993966 26557 raft_consensus.cc:2804] T b155b866f10547fd8239579a81ecf5f9 P 43999d813e564dc688fad116090c95ca [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:24:01.995576 25379 raft_consensus.cc:3055] T b155b866f10547fd8239579a81ecf5f9 P 3e07d64466934da4bb3c02262b1c2e38 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:24:01.995973 25379 raft_consensus.cc:740] T b155b866f10547fd8239579a81ecf5f9 P 3e07d64466934da4bb3c02262b1c2e38 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 3e07d64466934da4bb3c02262b1c2e38, State: Running, Role: LEADER
I20260709 13:24:01.997087 25379 consensus_queue.cc:260] T b155b866f10547fd8239579a81ecf5f9 P 3e07d64466934da4bb3c02262b1c2e38 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } }
I20260709 13:24:01.991015 26348 raft_consensus.cc:515] T d053b9ae38de4173a0aded4da494bd98 P 43999d813e564dc688fad116090c95ca [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } }
I20260709 13:24:01.998762 25379 raft_consensus.cc:3060] T b155b866f10547fd8239579a81ecf5f9 P 3e07d64466934da4bb3c02262b1c2e38 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:24:02.001458 25450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d053b9ae38de4173a0aded4da494bd98" candidate_uuid: "43999d813e564dc688fad116090c95ca" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "9868dce521294d0b816bba7648f4ae5a"
I20260709 13:24:02.001819 26348 leader_election.cc:290] T d053b9ae38de4173a0aded4da494bd98 P 43999d813e564dc688fad116090c95ca [CANDIDATE]: Term 2 election: Requested vote from peers 9868dce521294d0b816bba7648f4ae5a (127.14.177.132:33683), 233e7d29aac04505bf4e65f0a39a0c4c (127.14.177.130:42957)
I20260709 13:24:02.002233 25450 raft_consensus.cc:3055] T d053b9ae38de4173a0aded4da494bd98 P 9868dce521294d0b816bba7648f4ae5a [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:24:02.002619 25450 raft_consensus.cc:740] T d053b9ae38de4173a0aded4da494bd98 P 9868dce521294d0b816bba7648f4ae5a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 9868dce521294d0b816bba7648f4ae5a, State: Running, Role: LEADER
I20260709 13:24:02.003630 25450 consensus_queue.cc:260] T d053b9ae38de4173a0aded4da494bd98 P 9868dce521294d0b816bba7648f4ae5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } }
I20260709 13:24:01.995899 26557 raft_consensus.cc:697] T b155b866f10547fd8239579a81ecf5f9 P 43999d813e564dc688fad116090c95ca [term 4 LEADER]: Becoming Leader. State: Replica: 43999d813e564dc688fad116090c95ca, State: Running, Role: LEADER
I20260709 13:24:02.004876 25450 raft_consensus.cc:3060] T d053b9ae38de4173a0aded4da494bd98 P 9868dce521294d0b816bba7648f4ae5a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:24:02.005584 26557 consensus_queue.cc:237] T b155b866f10547fd8239579a81ecf5f9 P 43999d813e564dc688fad116090c95ca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } }
I20260709 13:24:02.009954 25308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d053b9ae38de4173a0aded4da494bd98" candidate_uuid: "43999d813e564dc688fad116090c95ca" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "233e7d29aac04505bf4e65f0a39a0c4c"
I20260709 13:24:02.011484 25308 raft_consensus.cc:3060] T d053b9ae38de4173a0aded4da494bd98 P 233e7d29aac04505bf4e65f0a39a0c4c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:24:02.020906 25450 raft_consensus.cc:2468] T d053b9ae38de4173a0aded4da494bd98 P 9868dce521294d0b816bba7648f4ae5a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 43999d813e564dc688fad116090c95ca in term 2.
I20260709 13:24:02.047284 25308 raft_consensus.cc:2468] T d053b9ae38de4173a0aded4da494bd98 P 233e7d29aac04505bf4e65f0a39a0c4c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 43999d813e564dc688fad116090c95ca in term 2.
I20260709 13:24:02.051995 26114 leader_election.cc:304] T d053b9ae38de4173a0aded4da494bd98 P 43999d813e564dc688fad116090c95ca [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 233e7d29aac04505bf4e65f0a39a0c4c, 43999d813e564dc688fad116090c95ca; no voters: 
I20260709 13:24:02.053668 26564 raft_consensus.cc:2804] T d053b9ae38de4173a0aded4da494bd98 P 43999d813e564dc688fad116090c95ca [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:24:02.054466 26564 raft_consensus.cc:697] T d053b9ae38de4173a0aded4da494bd98 P 43999d813e564dc688fad116090c95ca [term 2 LEADER]: Becoming Leader. State: Replica: 43999d813e564dc688fad116090c95ca, State: Running, Role: LEADER
I20260709 13:24:02.044886 25129 catalog_manager.cc:5697] T b7ff8dde9f5d4c49b9203d902f6b94c4 P 43999d813e564dc688fad116090c95ca reported cstate change: term changed from 3 to 4, leader changed from 3e07d64466934da4bb3c02262b1c2e38 (127.14.177.131) to 43999d813e564dc688fad116090c95ca (127.14.177.129). New cstate: current_term: 4 leader_uuid: "43999d813e564dc688fad116090c95ca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } health_report { overall_health: UNKNOWN } } }
I20260709 13:24:02.055559 26564 consensus_queue.cc:237] T d053b9ae38de4173a0aded4da494bd98 P 43999d813e564dc688fad116090c95ca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } }
I20260709 13:24:02.056794 25129 catalog_manager.cc:5697] T 79d4a12ef58140568c60a24ed4c12f83 P 43999d813e564dc688fad116090c95ca reported cstate change: term changed from 1 to 2, leader changed from 3e07d64466934da4bb3c02262b1c2e38 (127.14.177.131) to 43999d813e564dc688fad116090c95ca (127.14.177.129). New cstate: current_term: 2 leader_uuid: "43999d813e564dc688fad116090c95ca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } health_report { overall_health: UNKNOWN } } }
I20260709 13:24:02.065248 25379 raft_consensus.cc:2468] T b155b866f10547fd8239579a81ecf5f9 P 3e07d64466934da4bb3c02262b1c2e38 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 43999d813e564dc688fad116090c95ca in term 4.
I20260709 13:24:02.071213 26527 raft_consensus.cc:993] T e15820fa8a7d4e1fb6ad23a64ca22f90 P 3e07d64466934da4bb3c02262b1c2e38: : Instructing follower 43999d813e564dc688fad116090c95ca to start an election
I20260709 13:24:02.071715 26527 raft_consensus.cc:1081] T e15820fa8a7d4e1fb6ad23a64ca22f90 P 3e07d64466934da4bb3c02262b1c2e38 [term 1 LEADER]: Signalling peer 43999d813e564dc688fad116090c95ca to start an election
I20260709 13:24:02.075608 26348 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e15820fa8a7d4e1fb6ad23a64ca22f90"
dest_uuid: "43999d813e564dc688fad116090c95ca"
 from {username='slave'} at 127.0.0.1:48840
I20260709 13:24:02.078526 26348 raft_consensus.cc:493] T e15820fa8a7d4e1fb6ad23a64ca22f90 P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:24:02.078953 26348 raft_consensus.cc:3060] T e15820fa8a7d4e1fb6ad23a64ca22f90 P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:24:02.095628 26348 raft_consensus.cc:515] T e15820fa8a7d4e1fb6ad23a64ca22f90 P 43999d813e564dc688fad116090c95ca [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } }
I20260709 13:24:02.097908 26348 leader_election.cc:290] T e15820fa8a7d4e1fb6ad23a64ca22f90 P 43999d813e564dc688fad116090c95ca [CANDIDATE]: Term 2 election: Requested vote from peers 233e7d29aac04505bf4e65f0a39a0c4c (127.14.177.130:42957), 3e07d64466934da4bb3c02262b1c2e38 (127.14.177.131:41539)
I20260709 13:24:02.098502 25308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15820fa8a7d4e1fb6ad23a64ca22f90" candidate_uuid: "43999d813e564dc688fad116090c95ca" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "233e7d29aac04505bf4e65f0a39a0c4c"
I20260709 13:24:02.099273 25308 raft_consensus.cc:3060] T e15820fa8a7d4e1fb6ad23a64ca22f90 P 233e7d29aac04505bf4e65f0a39a0c4c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:24:02.100217 25379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15820fa8a7d4e1fb6ad23a64ca22f90" candidate_uuid: "43999d813e564dc688fad116090c95ca" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3e07d64466934da4bb3c02262b1c2e38"
I20260709 13:24:02.100980 25379 raft_consensus.cc:3055] T e15820fa8a7d4e1fb6ad23a64ca22f90 P 3e07d64466934da4bb3c02262b1c2e38 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:24:02.101398 25379 raft_consensus.cc:740] T e15820fa8a7d4e1fb6ad23a64ca22f90 P 3e07d64466934da4bb3c02262b1c2e38 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3e07d64466934da4bb3c02262b1c2e38, State: Running, Role: LEADER
I20260709 13:24:02.102247 25379 consensus_queue.cc:260] T e15820fa8a7d4e1fb6ad23a64ca22f90 P 3e07d64466934da4bb3c02262b1c2e38 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } }
I20260709 13:24:02.104574 25379 raft_consensus.cc:3060] T e15820fa8a7d4e1fb6ad23a64ca22f90 P 3e07d64466934da4bb3c02262b1c2e38 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:24:02.114225 25379 raft_consensus.cc:2468] T e15820fa8a7d4e1fb6ad23a64ca22f90 P 3e07d64466934da4bb3c02262b1c2e38 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 43999d813e564dc688fad116090c95ca in term 2.
I20260709 13:24:02.115464 26111 leader_election.cc:304] T e15820fa8a7d4e1fb6ad23a64ca22f90 P 43999d813e564dc688fad116090c95ca [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e07d64466934da4bb3c02262b1c2e38, 43999d813e564dc688fad116090c95ca; no voters: 
I20260709 13:24:02.116609 26564 raft_consensus.cc:2804] T e15820fa8a7d4e1fb6ad23a64ca22f90 P 43999d813e564dc688fad116090c95ca [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:24:02.117194 26564 raft_consensus.cc:697] T e15820fa8a7d4e1fb6ad23a64ca22f90 P 43999d813e564dc688fad116090c95ca [term 2 LEADER]: Becoming Leader. State: Replica: 43999d813e564dc688fad116090c95ca, State: Running, Role: LEADER
I20260709 13:24:02.117363 25308 raft_consensus.cc:2468] T e15820fa8a7d4e1fb6ad23a64ca22f90 P 233e7d29aac04505bf4e65f0a39a0c4c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 43999d813e564dc688fad116090c95ca in term 2.
I20260709 13:24:02.118393 26564 consensus_queue.cc:237] T e15820fa8a7d4e1fb6ad23a64ca22f90 P 43999d813e564dc688fad116090c95ca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } }
I20260709 13:24:02.136610 25129 catalog_manager.cc:5697] T b155b866f10547fd8239579a81ecf5f9 P 43999d813e564dc688fad116090c95ca reported cstate change: term changed from 3 to 4, leader changed from 3e07d64466934da4bb3c02262b1c2e38 (127.14.177.131) to 43999d813e564dc688fad116090c95ca (127.14.177.129). New cstate: current_term: 4 leader_uuid: "43999d813e564dc688fad116090c95ca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } health_report { overall_health: UNKNOWN } } }
I20260709 13:24:02.138234 25129 catalog_manager.cc:5697] T d053b9ae38de4173a0aded4da494bd98 P 43999d813e564dc688fad116090c95ca reported cstate change: term changed from 1 to 2, leader changed from 9868dce521294d0b816bba7648f4ae5a (127.14.177.132) to 43999d813e564dc688fad116090c95ca (127.14.177.129). New cstate: current_term: 2 leader_uuid: "43999d813e564dc688fad116090c95ca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } health_report { overall_health: UNKNOWN } } }
I20260709 13:24:02.139871 25129 catalog_manager.cc:5697] T e15820fa8a7d4e1fb6ad23a64ca22f90 P 43999d813e564dc688fad116090c95ca reported cstate change: term changed from 1 to 2, leader changed from 3e07d64466934da4bb3c02262b1c2e38 (127.14.177.131) to 43999d813e564dc688fad116090c95ca (127.14.177.129). New cstate: current_term: 2 leader_uuid: "43999d813e564dc688fad116090c95ca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } health_report { overall_health: UNKNOWN } } }
I20260709 13:24:02.400868 25308 raft_consensus.cc:1275] T 79d4a12ef58140568c60a24ed4c12f83 P 233e7d29aac04505bf4e65f0a39a0c4c [term 2 FOLLOWER]: Refusing update from remote peer 43999d813e564dc688fad116090c95ca: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:24:02.402115 26564 consensus_queue.cc:1048] T 79d4a12ef58140568c60a24ed4c12f83 P 43999d813e564dc688fad116090c95ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:24:02.418226 25379 raft_consensus.cc:1275] T 79d4a12ef58140568c60a24ed4c12f83 P 3e07d64466934da4bb3c02262b1c2e38 [term 2 FOLLOWER]: Refusing update from remote peer 43999d813e564dc688fad116090c95ca: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:24:02.419140 25308 raft_consensus.cc:1275] T b7ff8dde9f5d4c49b9203d902f6b94c4 P 233e7d29aac04505bf4e65f0a39a0c4c [term 4 FOLLOWER]: Refusing update from remote peer 43999d813e564dc688fad116090c95ca: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:24:02.420035 26532 consensus_queue.cc:1048] T 79d4a12ef58140568c60a24ed4c12f83 P 43999d813e564dc688fad116090c95ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:24:02.423774 26564 consensus_queue.cc:1048] T b7ff8dde9f5d4c49b9203d902f6b94c4 P 43999d813e564dc688fad116090c95ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:24:02.470005 25376 raft_consensus.cc:1275] T b7ff8dde9f5d4c49b9203d902f6b94c4 P 3e07d64466934da4bb3c02262b1c2e38 [term 4 FOLLOWER]: Refusing update from remote peer 43999d813e564dc688fad116090c95ca: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:24:02.474143 26557 consensus_queue.cc:1048] T b7ff8dde9f5d4c49b9203d902f6b94c4 P 43999d813e564dc688fad116090c95ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:24:02.576507 25379 raft_consensus.cc:1275] T e15820fa8a7d4e1fb6ad23a64ca22f90 P 3e07d64466934da4bb3c02262b1c2e38 [term 2 FOLLOWER]: Refusing update from remote peer 43999d813e564dc688fad116090c95ca: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:24:02.577907 26550 consensus_queue.cc:1048] T e15820fa8a7d4e1fb6ad23a64ca22f90 P 43999d813e564dc688fad116090c95ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:24:02.584365 25307 raft_consensus.cc:1275] T b155b866f10547fd8239579a81ecf5f9 P 233e7d29aac04505bf4e65f0a39a0c4c [term 4 FOLLOWER]: Refusing update from remote peer 43999d813e564dc688fad116090c95ca: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:24:02.585876 26557 consensus_queue.cc:1048] T b155b866f10547fd8239579a81ecf5f9 P 43999d813e564dc688fad116090c95ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:24:02.590750 25379 raft_consensus.cc:1275] T b155b866f10547fd8239579a81ecf5f9 P 3e07d64466934da4bb3c02262b1c2e38 [term 4 FOLLOWER]: Refusing update from remote peer 43999d813e564dc688fad116090c95ca: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:24:02.594594 25307 raft_consensus.cc:1275] T e15820fa8a7d4e1fb6ad23a64ca22f90 P 233e7d29aac04505bf4e65f0a39a0c4c [term 2 FOLLOWER]: Refusing update from remote peer 43999d813e564dc688fad116090c95ca: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:24:02.596951 26564 consensus_queue.cc:1048] T e15820fa8a7d4e1fb6ad23a64ca22f90 P 43999d813e564dc688fad116090c95ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:24:02.613874 25308 raft_consensus.cc:1275] T d053b9ae38de4173a0aded4da494bd98 P 233e7d29aac04505bf4e65f0a39a0c4c [term 2 FOLLOWER]: Refusing update from remote peer 43999d813e564dc688fad116090c95ca: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:24:02.616438 26564 consensus_queue.cc:1048] T b155b866f10547fd8239579a81ecf5f9 P 43999d813e564dc688fad116090c95ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:24:02.620381 26564 consensus_queue.cc:1048] T d053b9ae38de4173a0aded4da494bd98 P 43999d813e564dc688fad116090c95ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:24:02.646588 25459 raft_consensus.cc:1275] T d053b9ae38de4173a0aded4da494bd98 P 9868dce521294d0b816bba7648f4ae5a [term 2 FOLLOWER]: Refusing update from remote peer 43999d813e564dc688fad116090c95ca: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:24:02.649940 26590 consensus_queue.cc:1048] T d053b9ae38de4173a0aded4da494bd98 P 43999d813e564dc688fad116090c95ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:24:02.801522 15046 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:24:02.848484 15046 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:24:02.885890 25459 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5c7e7acbb09f4d49828fa652ca6abaac"
dest_uuid: "9868dce521294d0b816bba7648f4ae5a"
mode: GRACEFUL
new_leader_uuid: "43999d813e564dc688fad116090c95ca"
 from {username='slave'} at 127.0.0.1:35958
I20260709 13:24:02.886628 25459 raft_consensus.cc:606] T 5c7e7acbb09f4d49828fa652ca6abaac P 9868dce521294d0b816bba7648f4ae5a [term 3 LEADER]: Received request to transfer leadership to TS 43999d813e564dc688fad116090c95ca
I20260709 13:24:02.889397 25459 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "cfad575709c943cba40ccab278350770"
dest_uuid: "9868dce521294d0b816bba7648f4ae5a"
mode: GRACEFUL
new_leader_uuid: "233e7d29aac04505bf4e65f0a39a0c4c"
 from {username='slave'} at 127.0.0.1:35958
I20260709 13:24:02.889887 25459 raft_consensus.cc:606] T cfad575709c943cba40ccab278350770 P 9868dce521294d0b816bba7648f4ae5a [term 4 LEADER]: Received request to transfer leadership to TS 233e7d29aac04505bf4e65f0a39a0c4c
I20260709 13:24:02.892338 25459 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f1ac2f4d2fe2424bb470df42199219e2"
dest_uuid: "9868dce521294d0b816bba7648f4ae5a"
mode: GRACEFUL
new_leader_uuid: "43999d813e564dc688fad116090c95ca"
 from {username='slave'} at 127.0.0.1:35958
I20260709 13:24:02.892772 25459 raft_consensus.cc:606] T f1ac2f4d2fe2424bb470df42199219e2 P 9868dce521294d0b816bba7648f4ae5a [term 3 LEADER]: Received request to transfer leadership to TS 43999d813e564dc688fad116090c95ca
I20260709 13:24:02.893826 15046 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 13:24:02.895083 15046 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:24:03.019914 26556 raft_consensus.cc:993] T 5c7e7acbb09f4d49828fa652ca6abaac P 9868dce521294d0b816bba7648f4ae5a: : Instructing follower 43999d813e564dc688fad116090c95ca to start an election
I20260709 13:24:03.020488 26568 raft_consensus.cc:1081] T 5c7e7acbb09f4d49828fa652ca6abaac P 9868dce521294d0b816bba7648f4ae5a [term 3 LEADER]: Signalling peer 43999d813e564dc688fad116090c95ca to start an election
I20260709 13:24:03.023592 26355 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "5c7e7acbb09f4d49828fa652ca6abaac"
dest_uuid: "43999d813e564dc688fad116090c95ca"
 from {username='slave'} at 127.0.0.1:48848
I20260709 13:24:03.024184 26355 raft_consensus.cc:493] T 5c7e7acbb09f4d49828fa652ca6abaac P 43999d813e564dc688fad116090c95ca [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:24:03.024549 26355 raft_consensus.cc:3060] T 5c7e7acbb09f4d49828fa652ca6abaac P 43999d813e564dc688fad116090c95ca [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:24:03.032878 26355 raft_consensus.cc:515] T 5c7e7acbb09f4d49828fa652ca6abaac P 43999d813e564dc688fad116090c95ca [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } }
I20260709 13:24:03.035362 26355 leader_election.cc:290] T 5c7e7acbb09f4d49828fa652ca6abaac P 43999d813e564dc688fad116090c95ca [CANDIDATE]: Term 4 election: Requested vote from peers 3e07d64466934da4bb3c02262b1c2e38 (127.14.177.131:41539), 9868dce521294d0b816bba7648f4ae5a (127.14.177.132:33683)
I20260709 13:24:03.037947 25376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c7e7acbb09f4d49828fa652ca6abaac" candidate_uuid: "43999d813e564dc688fad116090c95ca" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: true dest_uuid: "3e07d64466934da4bb3c02262b1c2e38"
I20260709 13:24:03.038618 25376 raft_consensus.cc:3060] T 5c7e7acbb09f4d49828fa652ca6abaac P 3e07d64466934da4bb3c02262b1c2e38 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:24:03.042026 25459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c7e7acbb09f4d49828fa652ca6abaac" candidate_uuid: "43999d813e564dc688fad116090c95ca" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: true dest_uuid: "9868dce521294d0b816bba7648f4ae5a"
I20260709 13:24:03.042714 25459 raft_consensus.cc:3055] T 5c7e7acbb09f4d49828fa652ca6abaac P 9868dce521294d0b816bba7648f4ae5a [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:24:03.043012 25459 raft_consensus.cc:740] T 5c7e7acbb09f4d49828fa652ca6abaac P 9868dce521294d0b816bba7648f4ae5a [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 9868dce521294d0b816bba7648f4ae5a, State: Running, Role: LEADER
I20260709 13:24:03.044907 25459 consensus_queue.cc:260] T 5c7e7acbb09f4d49828fa652ca6abaac P 9868dce521294d0b816bba7648f4ae5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } }
I20260709 13:24:03.046203 25459 raft_consensus.cc:3060] T 5c7e7acbb09f4d49828fa652ca6abaac P 9868dce521294d0b816bba7648f4ae5a [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:24:03.046448 25376 raft_consensus.cc:2468] T 5c7e7acbb09f4d49828fa652ca6abaac P 3e07d64466934da4bb3c02262b1c2e38 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 43999d813e564dc688fad116090c95ca in term 4.
I20260709 13:24:03.047726 26111 leader_election.cc:304] T 5c7e7acbb09f4d49828fa652ca6abaac P 43999d813e564dc688fad116090c95ca [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e07d64466934da4bb3c02262b1c2e38, 43999d813e564dc688fad116090c95ca; no voters: 
I20260709 13:24:03.049887 26564 raft_consensus.cc:2804] T 5c7e7acbb09f4d49828fa652ca6abaac P 43999d813e564dc688fad116090c95ca [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:24:03.050747 26564 raft_consensus.cc:697] T 5c7e7acbb09f4d49828fa652ca6abaac P 43999d813e564dc688fad116090c95ca [term 4 LEADER]: Becoming Leader. State: Replica: 43999d813e564dc688fad116090c95ca, State: Running, Role: LEADER
I20260709 13:24:03.051944 26564 consensus_queue.cc:237] T 5c7e7acbb09f4d49828fa652ca6abaac P 43999d813e564dc688fad116090c95ca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } }
I20260709 13:24:03.058683 25459 raft_consensus.cc:2468] T 5c7e7acbb09f4d49828fa652ca6abaac P 9868dce521294d0b816bba7648f4ae5a [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 43999d813e564dc688fad116090c95ca in term 4.
I20260709 13:24:03.060127 26556 raft_consensus.cc:993] T f1ac2f4d2fe2424bb470df42199219e2 P 9868dce521294d0b816bba7648f4ae5a: : Instructing follower 43999d813e564dc688fad116090c95ca to start an election
I20260709 13:24:03.060746 26556 raft_consensus.cc:1081] T f1ac2f4d2fe2424bb470df42199219e2 P 9868dce521294d0b816bba7648f4ae5a [term 3 LEADER]: Signalling peer 43999d813e564dc688fad116090c95ca to start an election
I20260709 13:24:03.062631 26355 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "f1ac2f4d2fe2424bb470df42199219e2"
dest_uuid: "43999d813e564dc688fad116090c95ca"
 from {username='slave'} at 127.0.0.1:48848
I20260709 13:24:03.063261 26355 raft_consensus.cc:493] T f1ac2f4d2fe2424bb470df42199219e2 P 43999d813e564dc688fad116090c95ca [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:24:03.063612 26355 raft_consensus.cc:3060] T f1ac2f4d2fe2424bb470df42199219e2 P 43999d813e564dc688fad116090c95ca [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:24:03.065831 26556 raft_consensus.cc:993] T cfad575709c943cba40ccab278350770 P 9868dce521294d0b816bba7648f4ae5a: : Instructing follower 233e7d29aac04505bf4e65f0a39a0c4c to start an election
I20260709 13:24:03.070413 26556 raft_consensus.cc:1081] T cfad575709c943cba40ccab278350770 P 9868dce521294d0b816bba7648f4ae5a [term 4 LEADER]: Signalling peer 233e7d29aac04505bf4e65f0a39a0c4c to start an election
I20260709 13:24:03.073498 26355 raft_consensus.cc:515] T f1ac2f4d2fe2424bb470df42199219e2 P 43999d813e564dc688fad116090c95ca [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } }
I20260709 13:24:03.074048 25307 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "cfad575709c943cba40ccab278350770"
dest_uuid: "233e7d29aac04505bf4e65f0a39a0c4c"
 from {username='slave'} at 127.0.0.1:60208
I20260709 13:24:03.074683 25307 raft_consensus.cc:493] T cfad575709c943cba40ccab278350770 P 233e7d29aac04505bf4e65f0a39a0c4c [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:24:03.075063 25307 raft_consensus.cc:3060] T cfad575709c943cba40ccab278350770 P 233e7d29aac04505bf4e65f0a39a0c4c [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:24:03.075809 26355 leader_election.cc:290] T f1ac2f4d2fe2424bb470df42199219e2 P 43999d813e564dc688fad116090c95ca [CANDIDATE]: Term 4 election: Requested vote from peers 3e07d64466934da4bb3c02262b1c2e38 (127.14.177.131:41539), 9868dce521294d0b816bba7648f4ae5a (127.14.177.132:33683)
I20260709 13:24:03.078100 25459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1ac2f4d2fe2424bb470df42199219e2" candidate_uuid: "43999d813e564dc688fad116090c95ca" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: true dest_uuid: "9868dce521294d0b816bba7648f4ae5a"
I20260709 13:24:03.079918 25459 raft_consensus.cc:3055] T f1ac2f4d2fe2424bb470df42199219e2 P 9868dce521294d0b816bba7648f4ae5a [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:24:03.080256 25459 raft_consensus.cc:740] T f1ac2f4d2fe2424bb470df42199219e2 P 9868dce521294d0b816bba7648f4ae5a [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 9868dce521294d0b816bba7648f4ae5a, State: Running, Role: LEADER
I20260709 13:24:03.081933 25307 raft_consensus.cc:515] T cfad575709c943cba40ccab278350770 P 233e7d29aac04505bf4e65f0a39a0c4c [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } }
I20260709 13:24:03.082724 25459 consensus_queue.cc:260] T f1ac2f4d2fe2424bb470df42199219e2 P 9868dce521294d0b816bba7648f4ae5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } }
I20260709 13:24:03.084164 25459 raft_consensus.cc:3060] T f1ac2f4d2fe2424bb470df42199219e2 P 9868dce521294d0b816bba7648f4ae5a [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:24:03.082747 25376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1ac2f4d2fe2424bb470df42199219e2" candidate_uuid: "43999d813e564dc688fad116090c95ca" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: true dest_uuid: "3e07d64466934da4bb3c02262b1c2e38"
I20260709 13:24:03.085295 25379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfad575709c943cba40ccab278350770" candidate_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: true dest_uuid: "3e07d64466934da4bb3c02262b1c2e38"
I20260709 13:24:03.085705 25376 raft_consensus.cc:3060] T f1ac2f4d2fe2424bb470df42199219e2 P 3e07d64466934da4bb3c02262b1c2e38 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:24:03.085911 25379 raft_consensus.cc:3060] T cfad575709c943cba40ccab278350770 P 3e07d64466934da4bb3c02262b1c2e38 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:24:03.090948 25307 leader_election.cc:290] T cfad575709c943cba40ccab278350770 P 233e7d29aac04505bf4e65f0a39a0c4c [CANDIDATE]: Term 5 election: Requested vote from peers 3e07d64466934da4bb3c02262b1c2e38 (127.14.177.131:41539), 9868dce521294d0b816bba7648f4ae5a (127.14.177.132:33683)
I20260709 13:24:03.091109 25450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfad575709c943cba40ccab278350770" candidate_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: true dest_uuid: "9868dce521294d0b816bba7648f4ae5a"
I20260709 13:24:03.091732 25450 raft_consensus.cc:3055] T cfad575709c943cba40ccab278350770 P 9868dce521294d0b816bba7648f4ae5a [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:24:03.092031 25450 raft_consensus.cc:740] T cfad575709c943cba40ccab278350770 P 9868dce521294d0b816bba7648f4ae5a [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 9868dce521294d0b816bba7648f4ae5a, State: Running, Role: LEADER
I20260709 13:24:03.092839 25450 consensus_queue.cc:260] T cfad575709c943cba40ccab278350770 P 9868dce521294d0b816bba7648f4ae5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } }
I20260709 13:24:03.094082 25450 raft_consensus.cc:3060] T cfad575709c943cba40ccab278350770 P 9868dce521294d0b816bba7648f4ae5a [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:24:03.094238 25129 catalog_manager.cc:5697] T 5c7e7acbb09f4d49828fa652ca6abaac P 43999d813e564dc688fad116090c95ca reported cstate change: term changed from 3 to 4, leader changed from 9868dce521294d0b816bba7648f4ae5a (127.14.177.132) to 43999d813e564dc688fad116090c95ca (127.14.177.129). New cstate: current_term: 4 leader_uuid: "43999d813e564dc688fad116090c95ca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } health_report { overall_health: HEALTHY } } }
I20260709 13:24:03.096809 25376 raft_consensus.cc:2468] T f1ac2f4d2fe2424bb470df42199219e2 P 3e07d64466934da4bb3c02262b1c2e38 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 43999d813e564dc688fad116090c95ca in term 4.
I20260709 13:24:03.099757 26111 leader_election.cc:304] T f1ac2f4d2fe2424bb470df42199219e2 P 43999d813e564dc688fad116090c95ca [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e07d64466934da4bb3c02262b1c2e38, 43999d813e564dc688fad116090c95ca; no voters: 
I20260709 13:24:03.101040 26532 raft_consensus.cc:2804] T f1ac2f4d2fe2424bb470df42199219e2 P 43999d813e564dc688fad116090c95ca [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:24:03.101715 26532 raft_consensus.cc:697] T f1ac2f4d2fe2424bb470df42199219e2 P 43999d813e564dc688fad116090c95ca [term 4 LEADER]: Becoming Leader. State: Replica: 43999d813e564dc688fad116090c95ca, State: Running, Role: LEADER
I20260709 13:24:03.103246 25459 raft_consensus.cc:2468] T f1ac2f4d2fe2424bb470df42199219e2 P 9868dce521294d0b816bba7648f4ae5a [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 43999d813e564dc688fad116090c95ca in term 4.
I20260709 13:24:03.103004 26532 consensus_queue.cc:237] T f1ac2f4d2fe2424bb470df42199219e2 P 43999d813e564dc688fad116090c95ca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } }
I20260709 13:24:03.103809 25379 raft_consensus.cc:2468] T cfad575709c943cba40ccab278350770 P 3e07d64466934da4bb3c02262b1c2e38 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 233e7d29aac04505bf4e65f0a39a0c4c in term 5.
I20260709 13:24:03.105374 25274 leader_election.cc:304] T cfad575709c943cba40ccab278350770 P 233e7d29aac04505bf4e65f0a39a0c4c [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 233e7d29aac04505bf4e65f0a39a0c4c, 3e07d64466934da4bb3c02262b1c2e38; no voters: 
I20260709 13:24:03.106258 26558 raft_consensus.cc:2804] T cfad575709c943cba40ccab278350770 P 233e7d29aac04505bf4e65f0a39a0c4c [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:24:03.106751 26558 raft_consensus.cc:697] T cfad575709c943cba40ccab278350770 P 233e7d29aac04505bf4e65f0a39a0c4c [term 5 LEADER]: Becoming Leader. State: Replica: 233e7d29aac04505bf4e65f0a39a0c4c, State: Running, Role: LEADER
I20260709 13:24:03.107394 25450 raft_consensus.cc:2468] T cfad575709c943cba40ccab278350770 P 9868dce521294d0b816bba7648f4ae5a [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 233e7d29aac04505bf4e65f0a39a0c4c in term 5.
I20260709 13:24:03.107635 26558 consensus_queue.cc:237] T cfad575709c943cba40ccab278350770 P 233e7d29aac04505bf4e65f0a39a0c4c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } }
I20260709 13:24:03.128259 25130 catalog_manager.cc:5697] T cfad575709c943cba40ccab278350770 P 233e7d29aac04505bf4e65f0a39a0c4c reported cstate change: term changed from 4 to 5, leader changed from 9868dce521294d0b816bba7648f4ae5a (127.14.177.132) to 233e7d29aac04505bf4e65f0a39a0c4c (127.14.177.130). New cstate: current_term: 5 leader_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } health_report { overall_health: UNKNOWN } } }
I20260709 13:24:03.143409 25130 catalog_manager.cc:5697] T f1ac2f4d2fe2424bb470df42199219e2 P 43999d813e564dc688fad116090c95ca reported cstate change: term changed from 3 to 4, leader changed from 9868dce521294d0b816bba7648f4ae5a (127.14.177.132) to 43999d813e564dc688fad116090c95ca (127.14.177.129). New cstate: current_term: 4 leader_uuid: "43999d813e564dc688fad116090c95ca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } health_report { overall_health: HEALTHY } } }
I20260709 13:24:03.513700 25376 raft_consensus.cc:1275] T cfad575709c943cba40ccab278350770 P 3e07d64466934da4bb3c02262b1c2e38 [term 5 FOLLOWER]: Refusing update from remote peer 233e7d29aac04505bf4e65f0a39a0c4c: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:24:03.515245 26559 consensus_queue.cc:1048] T cfad575709c943cba40ccab278350770 P 233e7d29aac04505bf4e65f0a39a0c4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:24:03.515909 25376 raft_consensus.cc:1275] T 5c7e7acbb09f4d49828fa652ca6abaac P 3e07d64466934da4bb3c02262b1c2e38 [term 4 FOLLOWER]: Refusing update from remote peer 43999d813e564dc688fad116090c95ca: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:24:03.517678 26532 consensus_queue.cc:1048] T 5c7e7acbb09f4d49828fa652ca6abaac P 43999d813e564dc688fad116090c95ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:24:03.556311 25459 raft_consensus.cc:1275] T 5c7e7acbb09f4d49828fa652ca6abaac P 9868dce521294d0b816bba7648f4ae5a [term 4 FOLLOWER]: Refusing update from remote peer 43999d813e564dc688fad116090c95ca: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:24:03.557586 25456 raft_consensus.cc:1275] T cfad575709c943cba40ccab278350770 P 9868dce521294d0b816bba7648f4ae5a [term 5 FOLLOWER]: Refusing update from remote peer 233e7d29aac04505bf4e65f0a39a0c4c: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:24:03.559249 25377 raft_consensus.cc:1275] T f1ac2f4d2fe2424bb470df42199219e2 P 3e07d64466934da4bb3c02262b1c2e38 [term 4 FOLLOWER]: Refusing update from remote peer 43999d813e564dc688fad116090c95ca: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:24:03.564972 26558 consensus_queue.cc:1048] T cfad575709c943cba40ccab278350770 P 233e7d29aac04505bf4e65f0a39a0c4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:24:03.569289 26532 consensus_queue.cc:1048] T f1ac2f4d2fe2424bb470df42199219e2 P 43999d813e564dc688fad116090c95ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:24:03.605235 26557 consensus_queue.cc:1048] T 5c7e7acbb09f4d49828fa652ca6abaac P 43999d813e564dc688fad116090c95ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:24:03.669813 25456 raft_consensus.cc:1275] T f1ac2f4d2fe2424bb470df42199219e2 P 9868dce521294d0b816bba7648f4ae5a [term 4 FOLLOWER]: Refusing update from remote peer 43999d813e564dc688fad116090c95ca: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:24:03.674666 26550 consensus_queue.cc:1048] T f1ac2f4d2fe2424bb470df42199219e2 P 43999d813e564dc688fad116090c95ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:24:03.895941 15046 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:24:03.939762 15046 tablet_server.cc:179] TabletServer@127.14.177.129:38155 shutting down...
W20260709 13:24:03.972087 25424 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 205 similar messages]
I20260709 13:24:03.975890 15046 ts_tablet_manager.cc:1507] Shutting down tablet manager...
W20260709 13:24:03.976290 25424 consensus_peers.cc:597] T 1f15ec62698c4a8cbd84f2b3d516d2ea P 9868dce521294d0b816bba7648f4ae5a -> Peer 43999d813e564dc688fad116090c95ca (127.14.177.129:38155): Couldn't send request to peer 43999d813e564dc688fad116090c95ca. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:24:03.977110 15046 tablet_replica.cc:333] stopping tablet replica
I20260709 13:24:03.978777 15046 raft_consensus.cc:2243] T 51a11db644124b63bdf5f457c967473d P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:24:03.979462 15046 raft_consensus.cc:2272] T 51a11db644124b63bdf5f457c967473d P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:24:03.982189 15046 tablet_replica.cc:333] stopping tablet replica
I20260709 13:24:03.982851 15046 raft_consensus.cc:2243] T c902bbc7ec0848ba82a04870eab1e7f5 P 43999d813e564dc688fad116090c95ca [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:24:03.983385 15046 raft_consensus.cc:2272] T c902bbc7ec0848ba82a04870eab1e7f5 P 43999d813e564dc688fad116090c95ca [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:24:03.986053 15046 tablet_replica.cc:333] stopping tablet replica
I20260709 13:24:03.987036 15046 raft_consensus.cc:2243] T b155b866f10547fd8239579a81ecf5f9 P 43999d813e564dc688fad116090c95ca [term 4 LEADER]: Raft consensus shutting down.
I20260709 13:24:03.988233 15046 raft_consensus.cc:2272] T b155b866f10547fd8239579a81ecf5f9 P 43999d813e564dc688fad116090c95ca [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:24:03.991611 15046 tablet_replica.cc:333] stopping tablet replica
I20260709 13:24:03.992600 15046 raft_consensus.cc:2243] T b9d550d881b04735bbf89751952c09ec P 43999d813e564dc688fad116090c95ca [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:24:03.993227 15046 raft_consensus.cc:2272] T b9d550d881b04735bbf89751952c09ec P 43999d813e564dc688fad116090c95ca [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:24:03.996461 15046 tablet_replica.cc:333] stopping tablet replica
I20260709 13:24:03.997551 15046 raft_consensus.cc:2243] T 1550ed2d2a764c08adffae679599ba41 P 43999d813e564dc688fad116090c95ca [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:24:03.998149 15046 raft_consensus.cc:2272] T 1550ed2d2a764c08adffae679599ba41 P 43999d813e564dc688fad116090c95ca [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:24:03.998045 25424 consensus_peers.cc:597] T d0a0e4d32ce943fbb773bb840863647b P 9868dce521294d0b816bba7648f4ae5a -> Peer 43999d813e564dc688fad116090c95ca (127.14.177.129:38155): Couldn't send request to peer 43999d813e564dc688fad116090c95ca. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:24:04.004514 25424 consensus_peers.cc:597] T 11949f56648c47e0a43c3140af99a304 P 9868dce521294d0b816bba7648f4ae5a -> Peer 43999d813e564dc688fad116090c95ca (127.14.177.129:38155): Couldn't send request to peer 43999d813e564dc688fad116090c95ca. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:24:04.001868 15046 tablet_replica.cc:333] stopping tablet replica
I20260709 13:24:04.006187 15046 raft_consensus.cc:2243] T 6242888a4d4146dcacd87d0fd9a832f4 P 43999d813e564dc688fad116090c95ca [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:24:04.007402 15046 raft_consensus.cc:2272] T 6242888a4d4146dcacd87d0fd9a832f4 P 43999d813e564dc688fad116090c95ca [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:24:04.010365 15046 tablet_replica.cc:333] stopping tablet replica
W20260709 13:24:04.010366 25424 consensus_peers.cc:597] T 0c4f6b7401db47d59a4f34ed0e8b5350 P 9868dce521294d0b816bba7648f4ae5a -> Peer 43999d813e564dc688fad116090c95ca (127.14.177.129:38155): Couldn't send request to peer 43999d813e564dc688fad116090c95ca. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:24:04.011116 15046 raft_consensus.cc:2243] T 72b56266e8a54776b9113c7a786e1952 P 43999d813e564dc688fad116090c95ca [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:24:04.011708 15046 raft_consensus.cc:2272] T 72b56266e8a54776b9113c7a786e1952 P 43999d813e564dc688fad116090c95ca [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:24:04.015014 15046 tablet_replica.cc:333] stopping tablet replica
I20260709 13:24:04.015915 15046 raft_consensus.cc:2243] T 4d837697cd0f402cacf9db89a5d92652 P 43999d813e564dc688fad116090c95ca [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:24:04.016518 15046 raft_consensus.cc:2272] T 4d837697cd0f402cacf9db89a5d92652 P 43999d813e564dc688fad116090c95ca [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:24:04.019471 15046 tablet_replica.cc:333] stopping tablet replica
I20260709 13:24:04.020270 15046 raft_consensus.cc:2243] T 9ffa42c3b2ff48bc87310e9ab5d218ce P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:24:04.020789 15046 raft_consensus.cc:2272] T 9ffa42c3b2ff48bc87310e9ab5d218ce P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:24:04.023651 15046 tablet_replica.cc:333] stopping tablet replica
W20260709 13:24:04.023882 25350 consensus_peers.cc:597] T 03e9bfbd2f774476a4a377367796a2e8 P 3e07d64466934da4bb3c02262b1c2e38 -> Peer 43999d813e564dc688fad116090c95ca (127.14.177.129:38155): Couldn't send request to peer 43999d813e564dc688fad116090c95ca. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:24:04.024524 15046 raft_consensus.cc:2243] T e9e342ecda4b46619ce12452b2cf7154 P 43999d813e564dc688fad116090c95ca [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:24:04.025640 15046 raft_consensus.cc:2272] T e9e342ecda4b46619ce12452b2cf7154 P 43999d813e564dc688fad116090c95ca [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:24:04.028787 15046 tablet_replica.cc:333] stopping tablet replica
I20260709 13:24:04.029659 15046 raft_consensus.cc:2243] T 11949f56648c47e0a43c3140af99a304 P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:24:04.030179 15046 raft_consensus.cc:2272] T 11949f56648c47e0a43c3140af99a304 P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:24:04.030735 25350 consensus_peers.cc:597] T 72b56266e8a54776b9113c7a786e1952 P 3e07d64466934da4bb3c02262b1c2e38 -> Peer 43999d813e564dc688fad116090c95ca (127.14.177.129:38155): Couldn't send request to peer 43999d813e564dc688fad116090c95ca. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:24:04.032850 15046 tablet_replica.cc:333] stopping tablet replica
I20260709 13:24:04.033603 15046 raft_consensus.cc:2243] T beacc815ef954fb692e90544bfa93f05 P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:24:04.034040 15046 raft_consensus.cc:2272] T beacc815ef954fb692e90544bfa93f05 P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:24:04.037097 15046 tablet_replica.cc:333] stopping tablet replica
I20260709 13:24:04.037943 15046 raft_consensus.cc:2243] T ce98a3f3b3eb4c63bcaa355ec7cd64a4 P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:24:04.038439 15046 raft_consensus.cc:2272] T ce98a3f3b3eb4c63bcaa355ec7cd64a4 P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:24:04.041452 15046 tablet_replica.cc:333] stopping tablet replica
I20260709 13:24:04.042200 15046 raft_consensus.cc:2243] T b7ff8dde9f5d4c49b9203d902f6b94c4 P 43999d813e564dc688fad116090c95ca [term 4 LEADER]: Raft consensus shutting down.
I20260709 13:24:04.043001 15046 raft_consensus.cc:2272] T b7ff8dde9f5d4c49b9203d902f6b94c4 P 43999d813e564dc688fad116090c95ca [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:24:04.046020 15046 tablet_replica.cc:333] stopping tablet replica
I20260709 13:24:04.046885 15046 raft_consensus.cc:2243] T 8ef061028f38450e83324ac78cdaa14c P 43999d813e564dc688fad116090c95ca [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:24:04.047446 15046 raft_consensus.cc:2272] T 8ef061028f38450e83324ac78cdaa14c P 43999d813e564dc688fad116090c95ca [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:24:04.051105 15046 tablet_replica.cc:333] stopping tablet replica
W20260709 13:24:04.051155 25350 consensus_peers.cc:597] T b2f14aed05e24a98ad83535029b795dc P 3e07d64466934da4bb3c02262b1c2e38 -> Peer 43999d813e564dc688fad116090c95ca (127.14.177.129:38155): Couldn't send request to peer 43999d813e564dc688fad116090c95ca. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:24:04.052225 15046 raft_consensus.cc:2243] T 79d4a12ef58140568c60a24ed4c12f83 P 43999d813e564dc688fad116090c95ca [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:24:04.053553 15046 raft_consensus.cc:2272] T 79d4a12ef58140568c60a24ed4c12f83 P 43999d813e564dc688fad116090c95ca [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:24:04.055522 25276 consensus_peers.cc:597] T 434a54fcee7d4aeeab42f739855c0e8a P 233e7d29aac04505bf4e65f0a39a0c4c -> Peer 43999d813e564dc688fad116090c95ca (127.14.177.129:38155): Couldn't send request to peer 43999d813e564dc688fad116090c95ca. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:24:04.056699 15046 tablet_replica.cc:333] stopping tablet replica
I20260709 13:24:04.057657 15046 raft_consensus.cc:2243] T 821a5866429f4173820fb64336abee02 P 43999d813e564dc688fad116090c95ca [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:24:04.058236 15046 raft_consensus.cc:2272] T 821a5866429f4173820fb64336abee02 P 43999d813e564dc688fad116090c95ca [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:24:04.061268 15046 tablet_replica.cc:333] stopping tablet replica
I20260709 13:24:04.062021 15046 raft_consensus.cc:2243] T 0d29b499fa4c41f1b974782cd0c9c1c2 P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:24:04.062382 15046 raft_consensus.cc:2272] T 0d29b499fa4c41f1b974782cd0c9c1c2 P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:24:04.064697 15046 tablet_replica.cc:333] stopping tablet replica
I20260709 13:24:04.065409 15046 raft_consensus.cc:2243] T ce9c60e3922c4219b6e4453fe0259b6a P 43999d813e564dc688fad116090c95ca [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:24:04.065981 15046 raft_consensus.cc:2272] T ce9c60e3922c4219b6e4453fe0259b6a P 43999d813e564dc688fad116090c95ca [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:24:04.068666 15046 tablet_replica.cc:333] stopping tablet replica
I20260709 13:24:04.069340 15046 raft_consensus.cc:2243] T b85a238bbc3b4621881603a534c7bd9b P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:24:04.069775 15046 raft_consensus.cc:2272] T b85a238bbc3b4621881603a534c7bd9b P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:24:04.072029 15046 tablet_replica.cc:333] stopping tablet replica
I20260709 13:24:04.072676 15046 raft_consensus.cc:2243] T d053b9ae38de4173a0aded4da494bd98 P 43999d813e564dc688fad116090c95ca [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:24:04.073756 15046 raft_consensus.cc:2272] T d053b9ae38de4173a0aded4da494bd98 P 43999d813e564dc688fad116090c95ca [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:24:04.076390 15046 tablet_replica.cc:333] stopping tablet replica
I20260709 13:24:04.076906 15046 raft_consensus.cc:2243] T 5c7e7acbb09f4d49828fa652ca6abaac P 43999d813e564dc688fad116090c95ca [term 4 LEADER]: Raft consensus shutting down.
I20260709 13:24:04.077659 15046 raft_consensus.cc:2272] T 5c7e7acbb09f4d49828fa652ca6abaac P 43999d813e564dc688fad116090c95ca [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:24:04.080318 15046 tablet_replica.cc:333] stopping tablet replica
I20260709 13:24:04.081063 15046 raft_consensus.cc:2243] T 3a3317f4c3514df1b2bb8e44a86be92f P 43999d813e564dc688fad116090c95ca [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:24:04.081645 15046 raft_consensus.cc:2272] T 3a3317f4c3514df1b2bb8e44a86be92f P 43999d813e564dc688fad116090c95ca [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:24:04.084679 15046 tablet_replica.cc:333] stopping tablet replica
W20260709 13:24:04.084566 25424 consensus_peers.cc:597] T fa0da2c845dc4059968332f4db2e0aa6 P 9868dce521294d0b816bba7648f4ae5a -> Peer 43999d813e564dc688fad116090c95ca (127.14.177.129:38155): Couldn't send request to peer 43999d813e564dc688fad116090c95ca. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:24:04.085659 15046 raft_consensus.cc:2243] T fa0da2c845dc4059968332f4db2e0aa6 P 43999d813e564dc688fad116090c95ca [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:24:04.086252 15046 raft_consensus.cc:2272] T fa0da2c845dc4059968332f4db2e0aa6 P 43999d813e564dc688fad116090c95ca [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:24:04.087006 25350 consensus_peers.cc:597] T 87085ac2575e4b6a86dd1fc21e6f718b P 3e07d64466934da4bb3c02262b1c2e38 -> Peer 43999d813e564dc688fad116090c95ca (127.14.177.129:38155): Couldn't send request to peer 43999d813e564dc688fad116090c95ca. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:24:04.088752 15046 tablet_replica.cc:333] stopping tablet replica
I20260709 13:24:04.089506 15046 raft_consensus.cc:2243] T 87085ac2575e4b6a86dd1fc21e6f718b P 43999d813e564dc688fad116090c95ca [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:24:04.089954 15046 raft_consensus.cc:2272] T 87085ac2575e4b6a86dd1fc21e6f718b P 43999d813e564dc688fad116090c95ca [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:24:04.092299 15046 tablet_replica.cc:333] stopping tablet replica
I20260709 13:24:04.092909 15046 raft_consensus.cc:2243] T a5df750b84e0487fa530161079deb6c2 P 43999d813e564dc688fad116090c95ca [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:24:04.093393 15046 raft_consensus.cc:2272] T a5df750b84e0487fa530161079deb6c2 P 43999d813e564dc688fad116090c95ca [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:24:04.095744 15046 tablet_replica.cc:333] stopping tablet replica
I20260709 13:24:04.096364 15046 raft_consensus.cc:2243] T 579da2d6aabb4ae9a2b1802e1358068b P 43999d813e564dc688fad116090c95ca [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:24:04.096828 15046 raft_consensus.cc:2272] T 579da2d6aabb4ae9a2b1802e1358068b P 43999d813e564dc688fad116090c95ca [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:24:04.099493 15046 tablet_replica.cc:333] stopping tablet replica
I20260709 13:24:04.100437 15046 raft_consensus.cc:2243] T 03e9bfbd2f774476a4a377367796a2e8 P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:24:04.100965 15046 raft_consensus.cc:2272] T 03e9bfbd2f774476a4a377367796a2e8 P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:24:04.102406 25424 consensus_peers.cc:597] T c902bbc7ec0848ba82a04870eab1e7f5 P 9868dce521294d0b816bba7648f4ae5a -> Peer 43999d813e564dc688fad116090c95ca (127.14.177.129:38155): Couldn't send request to peer 43999d813e564dc688fad116090c95ca. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:24:04.104038 15046 tablet_replica.cc:333] stopping tablet replica
I20260709 13:24:04.104768 15046 raft_consensus.cc:2243] T 13bb3a23315a41c7a40e10b954b15c8e P 43999d813e564dc688fad116090c95ca [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:24:04.105286 15046 raft_consensus.cc:2272] T 13bb3a23315a41c7a40e10b954b15c8e P 43999d813e564dc688fad116090c95ca [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:24:04.108036 15046 tablet_replica.cc:333] stopping tablet replica
I20260709 13:24:04.108810 15046 raft_consensus.cc:2243] T 0cb89d70b4ce4aa8992fb68480bec8a2 P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:24:04.109295 15046 raft_consensus.cc:2272] T 0cb89d70b4ce4aa8992fb68480bec8a2 P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:24:04.110996 25276 consensus_peers.cc:597] T ce9c60e3922c4219b6e4453fe0259b6a P 233e7d29aac04505bf4e65f0a39a0c4c -> Peer 43999d813e564dc688fad116090c95ca (127.14.177.129:38155): Couldn't send request to peer 43999d813e564dc688fad116090c95ca. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:24:04.111730 15046 tablet_replica.cc:333] stopping tablet replica
I20260709 13:24:04.112434 15046 raft_consensus.cc:2243] T e15820fa8a7d4e1fb6ad23a64ca22f90 P 43999d813e564dc688fad116090c95ca [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:24:04.113238 15046 raft_consensus.cc:2272] T e15820fa8a7d4e1fb6ad23a64ca22f90 P 43999d813e564dc688fad116090c95ca [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:24:04.115511 15046 tablet_replica.cc:333] stopping tablet replica
I20260709 13:24:04.116073 15046 raft_consensus.cc:2243] T 434a54fcee7d4aeeab42f739855c0e8a P 43999d813e564dc688fad116090c95ca [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:24:04.116551 15046 raft_consensus.cc:2272] T 434a54fcee7d4aeeab42f739855c0e8a P 43999d813e564dc688fad116090c95ca [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:24:04.118710 15046 tablet_replica.cc:333] stopping tablet replica
I20260709 13:24:04.119326 15046 raft_consensus.cc:2243] T 28c0b6fc9bd64d60828d705c6a9ab1e4 P 43999d813e564dc688fad116090c95ca [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:24:04.120241 15046 raft_consensus.cc:2272] T 28c0b6fc9bd64d60828d705c6a9ab1e4 P 43999d813e564dc688fad116090c95ca [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:24:04.123890 15046 tablet_replica.cc:333] stopping tablet replica
I20260709 13:24:04.124651 15046 raft_consensus.cc:2243] T b7a891bfc89e4e259f53da6cb16f4989 P 43999d813e564dc688fad116090c95ca [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:24:04.125813 15046 raft_consensus.cc:2272] T b7a891bfc89e4e259f53da6cb16f4989 P 43999d813e564dc688fad116090c95ca [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:24:04.129369 15046 tablet_replica.cc:333] stopping tablet replica
I20260709 13:24:04.130296 15046 raft_consensus.cc:2243] T 0ed5edbe32044d1c83b0590dbb899801 P 43999d813e564dc688fad116090c95ca [term 3 LEADER]: Raft consensus shutting down.
W20260709 13:24:04.130467 25350 consensus_peers.cc:597] T 4d837697cd0f402cacf9db89a5d92652 P 3e07d64466934da4bb3c02262b1c2e38 -> Peer 43999d813e564dc688fad116090c95ca (127.14.177.129:38155): Couldn't send request to peer 43999d813e564dc688fad116090c95ca. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:24:04.131587 15046 raft_consensus.cc:2272] T 0ed5edbe32044d1c83b0590dbb899801 P 43999d813e564dc688fad116090c95ca [term 3 FOLLOWER]: Raft consensus is shut down!
W20260709 13:24:04.131960 25424 consensus_peers.cc:597] T a5df750b84e0487fa530161079deb6c2 P 9868dce521294d0b816bba7648f4ae5a -> Peer 43999d813e564dc688fad116090c95ca (127.14.177.129:38155): Couldn't send request to peer 43999d813e564dc688fad116090c95ca. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:24:04.134867 15046 tablet_replica.cc:333] stopping tablet replica
W20260709 13:24:04.135134 25276 consensus_peers.cc:597] T ce98a3f3b3eb4c63bcaa355ec7cd64a4 P 233e7d29aac04505bf4e65f0a39a0c4c -> Peer 43999d813e564dc688fad116090c95ca (127.14.177.129:38155): Couldn't send request to peer 43999d813e564dc688fad116090c95ca. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:24:04.135733 15046 raft_consensus.cc:2243] T 0c4f6b7401db47d59a4f34ed0e8b5350 P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:24:04.136307 15046 raft_consensus.cc:2272] T 0c4f6b7401db47d59a4f34ed0e8b5350 P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:24:04.139492 15046 tablet_replica.cc:333] stopping tablet replica
W20260709 13:24:04.139492 25276 consensus_peers.cc:597] T 0d29b499fa4c41f1b974782cd0c9c1c2 P 233e7d29aac04505bf4e65f0a39a0c4c -> Peer 43999d813e564dc688fad116090c95ca (127.14.177.129:38155): Couldn't send request to peer 43999d813e564dc688fad116090c95ca. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:24:04.140538 15046 raft_consensus.cc:2243] T 9e5d70664720424d951f269d9d7f4346 P 43999d813e564dc688fad116090c95ca [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:24:04.141507 15046 raft_consensus.cc:2272] T 9e5d70664720424d951f269d9d7f4346 P 43999d813e564dc688fad116090c95ca [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:24:04.144415 15046 tablet_replica.cc:333] stopping tablet replica
I20260709 13:24:04.145076 15046 raft_consensus.cc:2243] T f1ac2f4d2fe2424bb470df42199219e2 P 43999d813e564dc688fad116090c95ca [term 4 LEADER]: Raft consensus shutting down.
I20260709 13:24:04.145881 15046 raft_consensus.cc:2272] T f1ac2f4d2fe2424bb470df42199219e2 P 43999d813e564dc688fad116090c95ca [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:24:04.148933 15046 tablet_replica.cc:333] stopping tablet replica
I20260709 13:24:04.149880 15046 raft_consensus.cc:2243] T b2f14aed05e24a98ad83535029b795dc P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:24:04.150437 15046 raft_consensus.cc:2272] T b2f14aed05e24a98ad83535029b795dc P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:24:04.153352 15046 tablet_replica.cc:333] stopping tablet replica
I20260709 13:24:04.154158 15046 raft_consensus.cc:2243] T d0a0e4d32ce943fbb773bb840863647b P 43999d813e564dc688fad116090c95ca [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:24:04.154948 15046 raft_consensus.cc:2272] T d0a0e4d32ce943fbb773bb840863647b P 43999d813e564dc688fad116090c95ca [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:24:04.157089 25350 consensus_peers.cc:597] T 821a5866429f4173820fb64336abee02 P 3e07d64466934da4bb3c02262b1c2e38 -> Peer 43999d813e564dc688fad116090c95ca (127.14.177.129:38155): Couldn't send request to peer 43999d813e564dc688fad116090c95ca. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:24:04.158032 15046 tablet_replica.cc:333] stopping tablet replica
I20260709 13:24:04.158713 15046 raft_consensus.cc:2243] T 0e71b6f5c69b4b35a8ac51c22c7d5f0d P 43999d813e564dc688fad116090c95ca [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:24:04.159363 15046 raft_consensus.cc:2272] T 0e71b6f5c69b4b35a8ac51c22c7d5f0d P 43999d813e564dc688fad116090c95ca [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:24:04.161741 15046 tablet_replica.cc:333] stopping tablet replica
I20260709 13:24:04.162339 15046 raft_consensus.cc:2243] T 441d780c3945463f92f31170f31e37a8 P 43999d813e564dc688fad116090c95ca [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:24:04.162819 15046 raft_consensus.cc:2272] T 441d780c3945463f92f31170f31e37a8 P 43999d813e564dc688fad116090c95ca [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:24:04.165014 15046 tablet_replica.cc:333] stopping tablet replica
I20260709 13:24:04.165546 15046 raft_consensus.cc:2243] T 1f15ec62698c4a8cbd84f2b3d516d2ea P 43999d813e564dc688fad116090c95ca [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:24:04.165977 15046 raft_consensus.cc:2272] T 1f15ec62698c4a8cbd84f2b3d516d2ea P 43999d813e564dc688fad116090c95ca [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:24:04.168484 15046 tablet_replica.cc:333] stopping tablet replica
I20260709 13:24:04.169117 15046 raft_consensus.cc:2243] T 280d65245f4f4358b54bef93e18aca92 P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:24:04.169529 15046 raft_consensus.cc:2272] T 280d65245f4f4358b54bef93e18aca92 P 43999d813e564dc688fad116090c95ca [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:24:04.171909 15046 tablet_replica.cc:333] stopping tablet replica
I20260709 13:24:04.172415 15046 raft_consensus.cc:2243] T 91ac744179de48e5bb924b6dfd9f9c9b P 43999d813e564dc688fad116090c95ca [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:24:04.173274 15046 raft_consensus.cc:2272] T 91ac744179de48e5bb924b6dfd9f9c9b P 43999d813e564dc688fad116090c95ca [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:24:04.250936 15046 tablet_server.cc:196] TabletServer@127.14.177.129:38155 shutdown complete.
W20260709 13:24:04.261317 25424 consensus_peers.cc:597] T 280d65245f4f4358b54bef93e18aca92 P 9868dce521294d0b816bba7648f4ae5a -> Peer 43999d813e564dc688fad116090c95ca (127.14.177.129:38155): Couldn't send request to peer 43999d813e564dc688fad116090c95ca. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:38155: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:24:04.266654 25350 consensus_peers.cc:597] T 0e71b6f5c69b4b35a8ac51c22c7d5f0d P 3e07d64466934da4bb3c02262b1c2e38 -> Peer 43999d813e564dc688fad116090c95ca (127.14.177.129:38155): Couldn't send request to peer 43999d813e564dc688fad116090c95ca. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:38155: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:24:04.273684 25424 consensus_peers.cc:597] T 8ef061028f38450e83324ac78cdaa14c P 9868dce521294d0b816bba7648f4ae5a -> Peer 43999d813e564dc688fad116090c95ca (127.14.177.129:38155): Couldn't send request to peer 43999d813e564dc688fad116090c95ca. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:38155: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:24:04.275714 25350 consensus_peers.cc:597] T 13bb3a23315a41c7a40e10b954b15c8e P 3e07d64466934da4bb3c02262b1c2e38 -> Peer 43999d813e564dc688fad116090c95ca (127.14.177.129:38155): Couldn't send request to peer 43999d813e564dc688fad116090c95ca. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:38155: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:24:04.330483 25350 consensus_peers.cc:597] T 9ffa42c3b2ff48bc87310e9ab5d218ce P 3e07d64466934da4bb3c02262b1c2e38 -> Peer 43999d813e564dc688fad116090c95ca (127.14.177.129:38155): Couldn't send request to peer 43999d813e564dc688fad116090c95ca. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:38155: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:24:04.344599 25276 consensus_peers.cc:597] T 3a3317f4c3514df1b2bb8e44a86be92f P 233e7d29aac04505bf4e65f0a39a0c4c -> Peer 43999d813e564dc688fad116090c95ca (127.14.177.129:38155): Couldn't send request to peer 43999d813e564dc688fad116090c95ca. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:38155: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:24:04.344691 15046 tablet_server.cc:179] TabletServer@127.14.177.130:0 shutting down...
W20260709 13:24:04.365818 25351 consensus_peers.cc:597] T 72b56266e8a54776b9113c7a786e1952 P 3e07d64466934da4bb3c02262b1c2e38 -> Peer 233e7d29aac04505bf4e65f0a39a0c4c (127.14.177.130:42957): Couldn't send request to peer 233e7d29aac04505bf4e65f0a39a0c4c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:24:04.388761 25351 consensus_peers.cc:597] T 03e9bfbd2f774476a4a377367796a2e8 P 3e07d64466934da4bb3c02262b1c2e38 -> Peer 233e7d29aac04505bf4e65f0a39a0c4c (127.14.177.130:42957): Couldn't send request to peer 233e7d29aac04505bf4e65f0a39a0c4c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:24:04.390766 25425 consensus_peers.cc:597] T 579da2d6aabb4ae9a2b1802e1358068b P 9868dce521294d0b816bba7648f4ae5a -> Peer 233e7d29aac04505bf4e65f0a39a0c4c (127.14.177.130:42957): Couldn't send request to peer 233e7d29aac04505bf4e65f0a39a0c4c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:24:04.397996 25351 consensus_peers.cc:597] T 87085ac2575e4b6a86dd1fc21e6f718b P 3e07d64466934da4bb3c02262b1c2e38 -> Peer 233e7d29aac04505bf4e65f0a39a0c4c (127.14.177.130:42957): Couldn't send request to peer 233e7d29aac04505bf4e65f0a39a0c4c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:24:04.406461 25350 consensus_peers.cc:597] T beacc815ef954fb692e90544bfa93f05 P 3e07d64466934da4bb3c02262b1c2e38 -> Peer 43999d813e564dc688fad116090c95ca (127.14.177.129:38155): Couldn't send request to peer 43999d813e564dc688fad116090c95ca. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:38155: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:24:04.411306 25424 consensus_peers.cc:597] T b9d550d881b04735bbf89751952c09ec P 9868dce521294d0b816bba7648f4ae5a -> Peer 43999d813e564dc688fad116090c95ca (127.14.177.129:38155): Couldn't send request to peer 43999d813e564dc688fad116090c95ca. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:38155: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:24:04.412755 15046 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:24:04.413810 15046 tablet_replica.cc:333] T 72b56266e8a54776b9113c7a786e1952 P 233e7d29aac04505bf4e65f0a39a0c4c: stopping tablet replica
I20260709 13:24:04.414803 15046 raft_consensus.cc:2243] T 72b56266e8a54776b9113c7a786e1952 P 233e7d29aac04505bf4e65f0a39a0c4c [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:24:04.415340 15046 raft_consensus.cc:2272] T 72b56266e8a54776b9113c7a786e1952 P 233e7d29aac04505bf4e65f0a39a0c4c [term 3 FOLLOWER]: Raft consensus is shut down!
W20260709 13:24:04.419569 25276 consensus_peers.cc:597] T b85a238bbc3b4621881603a534c7bd9b P 233e7d29aac04505bf4e65f0a39a0c4c -> Peer 43999d813e564dc688fad116090c95ca (127.14.177.129:38155): Couldn't send request to peer 43999d813e564dc688fad116090c95ca. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:38155: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:24:04.421162 25351 consensus_peers.cc:597] T 467a6b42ca6548269c12dddde696f7af P 3e07d64466934da4bb3c02262b1c2e38 -> Peer 233e7d29aac04505bf4e65f0a39a0c4c (127.14.177.130:42957): Couldn't send request to peer 233e7d29aac04505bf4e65f0a39a0c4c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:24:04.422565 15046 tablet_replica.cc:333] T e311eb90bad44c80a131b72edcd22b06 P 233e7d29aac04505bf4e65f0a39a0c4c: stopping tablet replica
I20260709 13:24:04.423293 15046 raft_consensus.cc:2243] T e311eb90bad44c80a131b72edcd22b06 P 233e7d29aac04505bf4e65f0a39a0c4c [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:24:04.424328 15046 raft_consensus.cc:2272] T e311eb90bad44c80a131b72edcd22b06 P 233e7d29aac04505bf4e65f0a39a0c4c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:24:04.427309 15046 tablet_replica.cc:333] T b155b866f10547fd8239579a81ecf5f9 P 233e7d29aac04505bf4e65f0a39a0c4c: stopping tablet replica
I20260709 13:24:04.427971 15046 raft_consensus.cc:2243] T b155b866f10547fd8239579a81ecf5f9 P 233e7d29aac04505bf4e65f0a39a0c4c [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:24:04.428457 15046 raft_consensus.cc:2272] T b155b866f10547fd8239579a81ecf5f9 P 233e7d29aac04505bf4e65f0a39a0c4c [term 4 FOLLOWER]: Raft consensus is shut down!
W20260709 13:24:04.428809 25424 consensus_peers.cc:597] T 1550ed2d2a764c08adffae679599ba41 P 9868dce521294d0b816bba7648f4ae5a -> Peer 43999d813e564dc688fad116090c95ca (127.14.177.129:38155): Couldn't send request to peer 43999d813e564dc688fad116090c95ca. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:38155: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:24:04.430943 15046 tablet_replica.cc:333] T 9ab90e3e08fb4528b2a47d995e9e26d3 P 233e7d29aac04505bf4e65f0a39a0c4c: stopping tablet replica
W20260709 13:24:04.431324 25276 consensus_peers.cc:597] T 51a11db644124b63bdf5f457c967473d P 233e7d29aac04505bf4e65f0a39a0c4c -> Peer 43999d813e564dc688fad116090c95ca (127.14.177.129:38155): Couldn't send request to peer 43999d813e564dc688fad116090c95ca. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:38155: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:24:04.431999 15046 raft_consensus.cc:2243] T 9ab90e3e08fb4528b2a47d995e9e26d3 P 233e7d29aac04505bf4e65f0a39a0c4c [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:24:04.432546 15046 raft_consensus.cc:2272] T 9ab90e3e08fb4528b2a47d995e9e26d3 P 233e7d29aac04505bf4e65f0a39a0c4c [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:24:04.435374 15046 tablet_replica.cc:333] T ce9c60e3922c4219b6e4453fe0259b6a P 233e7d29aac04505bf4e65f0a39a0c4c: stopping tablet replica
I20260709 13:24:04.436069 15046 raft_consensus.cc:2243] T ce9c60e3922c4219b6e4453fe0259b6a P 233e7d29aac04505bf4e65f0a39a0c4c [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:24:04.437026 15046 raft_consensus.cc:2272] T ce9c60e3922c4219b6e4453fe0259b6a P 233e7d29aac04505bf4e65f0a39a0c4c [term 3 FOLLOWER]: Raft consensus is shut down!
W20260709 13:24:04.438730 25350 consensus_peers.cc:597] T 0cb89d70b4ce4aa8992fb68480bec8a2 P 3e07d64466934da4bb3c02262b1c2e38 -> Peer 43999d813e564dc688fad116090c95ca (127.14.177.129:38155): Couldn't send request to peer 43999d813e564dc688fad116090c95ca. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:38155: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:24:04.440160 15046 tablet_replica.cc:333] T 6242888a4d4146dcacd87d0fd9a832f4 P 233e7d29aac04505bf4e65f0a39a0c4c: stopping tablet replica
I20260709 13:24:04.441099 15046 raft_consensus.cc:2243] T 6242888a4d4146dcacd87d0fd9a832f4 P 233e7d29aac04505bf4e65f0a39a0c4c [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:24:04.441659 15046 raft_consensus.cc:2272] T 6242888a4d4146dcacd87d0fd9a832f4 P 233e7d29aac04505bf4e65f0a39a0c4c [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:24:04.444772 15046 tablet_replica.cc:333] T 87085ac2575e4b6a86dd1fc21e6f718b P 233e7d29aac04505bf4e65f0a39a0c4c: stopping tablet replica
I20260709 13:24:04.445624 15046 raft_consensus.cc:2243] T 87085ac2575e4b6a86dd1fc21e6f718b P 233e7d29aac04505bf4e65f0a39a0c4c [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:24:04.446219 15046 raft_consensus.cc:2272] T 87085ac2575e4b6a86dd1fc21e6f718b P 233e7d29aac04505bf4e65f0a39a0c4c [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:24:04.448948 15046 tablet_replica.cc:333] T 434a18a5109e4ea78349ded3968b2709 P 233e7d29aac04505bf4e65f0a39a0c4c: stopping tablet replica
I20260709 13:24:04.449718 15046 raft_consensus.cc:2243] T 434a18a5109e4ea78349ded3968b2709 P 233e7d29aac04505bf4e65f0a39a0c4c [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:24:04.450986 15046 raft_consensus.cc:2272] T 434a18a5109e4ea78349ded3968b2709 P 233e7d29aac04505bf4e65f0a39a0c4c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:24:04.453840 15046 tablet_replica.cc:333] T a5df750b84e0487fa530161079deb6c2 P 233e7d29aac04505bf4e65f0a39a0c4c: stopping tablet replica
I20260709 13:24:04.454704 15046 raft_consensus.cc:2243] T a5df750b84e0487fa530161079deb6c2 P 233e7d29aac04505bf4e65f0a39a0c4c [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:24:04.455304 15046 raft_consensus.cc:2272] T a5df750b84e0487fa530161079deb6c2 P 233e7d29aac04505bf4e65f0a39a0c4c [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:24:04.457942 15046 tablet_replica.cc:333] T e9e342ecda4b46619ce12452b2cf7154 P 233e7d29aac04505bf4e65f0a39a0c4c: stopping tablet replica
I20260709 13:24:04.458777 15046 raft_consensus.cc:2243] T e9e342ecda4b46619ce12452b2cf7154 P 233e7d29aac04505bf4e65f0a39a0c4c [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:24:04.459367 15046 raft_consensus.cc:2272] T e9e342ecda4b46619ce12452b2cf7154 P 233e7d29aac04505bf4e65f0a39a0c4c [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:24:04.462047 15046 tablet_replica.cc:333] T ad32097cd4dc48b9a69595f1651abd07 P 233e7d29aac04505bf4e65f0a39a0c4c: stopping tablet replica
W20260709 13:24:04.462661 25276 consensus_peers.cc:597] T 441d780c3945463f92f31170f31e37a8 P 233e7d29aac04505bf4e65f0a39a0c4c -> Peer 43999d813e564dc688fad116090c95ca (127.14.177.129:38155): Couldn't send request to peer 43999d813e564dc688fad116090c95ca. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:38155: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:24:04.462881 15046 raft_consensus.cc:2243] T ad32097cd4dc48b9a69595f1651abd07 P 233e7d29aac04505bf4e65f0a39a0c4c [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:24:04.463449 15046 raft_consensus.cc:2272] T ad32097cd4dc48b9a69595f1651abd07 P 233e7d29aac04505bf4e65f0a39a0c4c [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:24:04.465332 15046 tablet_replica.cc:333] T 0c4f6b7401db47d59a4f34ed0e8b5350 P 233e7d29aac04505bf4e65f0a39a0c4c: stopping tablet replica
I20260709 13:24:04.465858 15046 raft_consensus.cc:2243] T 0c4f6b7401db47d59a4f34ed0e8b5350 P 233e7d29aac04505bf4e65f0a39a0c4c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:24:04.466218 15046 raft_consensus.cc:2272] T 0c4f6b7401db47d59a4f34ed0e8b5350 P 233e7d29aac04505bf4e65f0a39a0c4c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:24:04.468581 15046 tablet_replica.cc:333] T 51a11db644124b63bdf5f457c967473d P 233e7d29aac04505bf4e65f0a39a0c4c: stopping tablet replica
I20260709 13:24:04.469213 15046 raft_consensus.cc:2243] T 51a11db644124b63bdf5f457c967473d P 233e7d29aac04505bf4e65f0a39a0c4c [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:24:04.470237 15046 raft_consensus.cc:2272] T 51a11db644124b63bdf5f457c967473d P 233e7d29aac04505bf4e65f0a39a0c4c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:24:04.472282 15046 tablet_replica.cc:333] T 89a26dd3ac414f6081b98fa32c2f346f P 233e7d29aac04505bf4e65f0a39a0c4c: stopping tablet replica
I20260709 13:24:04.472747 15046 raft_consensus.cc:2243] T 89a26dd3ac414f6081b98fa32c2f346f P 233e7d29aac04505bf4e65f0a39a0c4c [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:24:04.473124 15046 raft_consensus.cc:2272] T 89a26dd3ac414f6081b98fa32c2f346f P 233e7d29aac04505bf4e65f0a39a0c4c [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:24:04.475112 15046 tablet_replica.cc:333] T ce98a3f3b3eb4c63bcaa355ec7cd64a4 P 233e7d29aac04505bf4e65f0a39a0c4c: stopping tablet replica
I20260709 13:24:04.475672 15046 raft_consensus.cc:2243] T ce98a3f3b3eb4c63bcaa355ec7cd64a4 P 233e7d29aac04505bf4e65f0a39a0c4c [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:24:04.476538 15046 raft_consensus.cc:2272] T ce98a3f3b3eb4c63bcaa355ec7cd64a4 P 233e7d29aac04505bf4e65f0a39a0c4c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:24:04.478736 15046 tablet_replica.cc:333] T 9e5d70664720424d951f269d9d7f4346 P 233e7d29aac04505bf4e65f0a39a0c4c: stopping tablet replica
I20260709 13:24:04.479465 15046 raft_consensus.cc:2243] T 9e5d70664720424d951f269d9d7f4346 P 233e7d29aac04505bf4e65f0a39a0c4c [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:24:04.479938 15046 raft_consensus.cc:2272] T 9e5d70664720424d951f269d9d7f4346 P 233e7d29aac04505bf4e65f0a39a0c4c [term 3 FOLLOWER]: Raft consensus is shut down!
W20260709 13:24:04.480247 25351 consensus_peers.cc:597] T 9ab90e3e08fb4528b2a47d995e9e26d3 P 3e07d64466934da4bb3c02262b1c2e38 -> Peer 233e7d29aac04505bf4e65f0a39a0c4c (127.14.177.130:42957): Couldn't send request to peer 233e7d29aac04505bf4e65f0a39a0c4c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:24:04.482412 15046 tablet_replica.cc:333] T b85a238bbc3b4621881603a534c7bd9b P 233e7d29aac04505bf4e65f0a39a0c4c: stopping tablet replica
I20260709 13:24:04.483368 15046 raft_consensus.cc:2243] T b85a238bbc3b4621881603a534c7bd9b P 233e7d29aac04505bf4e65f0a39a0c4c [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:24:04.484366 15046 raft_consensus.cc:2272] T b85a238bbc3b4621881603a534c7bd9b P 233e7d29aac04505bf4e65f0a39a0c4c [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:24:04.485322 25425 consensus_peers.cc:597] T a5df750b84e0487fa530161079deb6c2 P 9868dce521294d0b816bba7648f4ae5a -> Peer 233e7d29aac04505bf4e65f0a39a0c4c (127.14.177.130:42957): Couldn't send request to peer 233e7d29aac04505bf4e65f0a39a0c4c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:24:04.486967 15046 tablet_replica.cc:333] T aad1f7937f854cc1a726223269da5d5a P 233e7d29aac04505bf4e65f0a39a0c4c: stopping tablet replica
I20260709 13:24:04.487643 15046 raft_consensus.cc:2243] T aad1f7937f854cc1a726223269da5d5a P 233e7d29aac04505bf4e65f0a39a0c4c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:24:04.488117 15046 raft_consensus.cc:2272] T aad1f7937f854cc1a726223269da5d5a P 233e7d29aac04505bf4e65f0a39a0c4c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:24:04.490139 15046 tablet_replica.cc:333] T d053b9ae38de4173a0aded4da494bd98 P 233e7d29aac04505bf4e65f0a39a0c4c: stopping tablet replica
I20260709 13:24:04.490818 15046 raft_consensus.cc:2243] T d053b9ae38de4173a0aded4da494bd98 P 233e7d29aac04505bf4e65f0a39a0c4c [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:24:04.491357 15046 raft_consensus.cc:2272] T d053b9ae38de4173a0aded4da494bd98 P 233e7d29aac04505bf4e65f0a39a0c4c [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:24:04.493444 15046 tablet_replica.cc:333] T 79d4a12ef58140568c60a24ed4c12f83 P 233e7d29aac04505bf4e65f0a39a0c4c: stopping tablet replica
I20260709 13:24:04.494033 15046 raft_consensus.cc:2243] T 79d4a12ef58140568c60a24ed4c12f83 P 233e7d29aac04505bf4e65f0a39a0c4c [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:24:04.494472 15046 raft_consensus.cc:2272] T 79d4a12ef58140568c60a24ed4c12f83 P 233e7d29aac04505bf4e65f0a39a0c4c [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:24:04.496488 15046 tablet_replica.cc:333] T bbab48ba177049dcba6de106fd889cfb P 233e7d29aac04505bf4e65f0a39a0c4c: stopping tablet replica
I20260709 13:24:04.497133 15046 raft_consensus.cc:2243] T bbab48ba177049dcba6de106fd889cfb P 233e7d29aac04505bf4e65f0a39a0c4c [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:24:04.497925 15046 raft_consensus.cc:2272] T bbab48ba177049dcba6de106fd889cfb P 233e7d29aac04505bf4e65f0a39a0c4c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:24:04.500121 15046 tablet_replica.cc:333] T 0d29b499fa4c41f1b974782cd0c9c1c2 P 233e7d29aac04505bf4e65f0a39a0c4c: stopping tablet replica
I20260709 13:24:04.500806 15046 raft_consensus.cc:2243] T 0d29b499fa4c41f1b974782cd0c9c1c2 P 233e7d29aac04505bf4e65f0a39a0c4c [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:24:04.501762 15046 raft_consensus.cc:2272] T 0d29b499fa4c41f1b974782cd0c9c1c2 P 233e7d29aac04505bf4e65f0a39a0c4c [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:24:04.503631 25425 consensus_peers.cc:597] T 11949f56648c47e0a43c3140af99a304 P 9868dce521294d0b816bba7648f4ae5a -> Peer 233e7d29aac04505bf4e65f0a39a0c4c (127.14.177.130:42957): Couldn't send request to peer 233e7d29aac04505bf4e65f0a39a0c4c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:24:04.504331 15046 tablet_replica.cc:333] T ba8a000f5abe4158965449995af84b18 P 233e7d29aac04505bf4e65f0a39a0c4c: stopping tablet replica
I20260709 13:24:04.504971 15046 raft_consensus.cc:2243] T ba8a000f5abe4158965449995af84b18 P 233e7d29aac04505bf4e65f0a39a0c4c [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:24:04.505755 15046 raft_consensus.cc:2272] T ba8a000f5abe4158965449995af84b18 P 233e7d29aac04505bf4e65f0a39a0c4c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:24:04.507853 15046 tablet_replica.cc:333] T 11949f56648c47e0a43c3140af99a304 P 233e7d29aac04505bf4e65f0a39a0c4c: stopping tablet replica
I20260709 13:24:04.508401 15046 raft_consensus.cc:2243] T 11949f56648c47e0a43c3140af99a304 P 233e7d29aac04505bf4e65f0a39a0c4c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:24:04.508831 15046 raft_consensus.cc:2272] T 11949f56648c47e0a43c3140af99a304 P 233e7d29aac04505bf4e65f0a39a0c4c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:24:04.510936 15046 tablet_replica.cc:333] T beacc815ef954fb692e90544bfa93f05 P 233e7d29aac04505bf4e65f0a39a0c4c: stopping tablet replica
I20260709 13:24:04.511799 15046 raft_consensus.cc:2243] T beacc815ef954fb692e90544bfa93f05 P 233e7d29aac04505bf4e65f0a39a0c4c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:24:04.512313 15046 raft_consensus.cc:2272] T beacc815ef954fb692e90544bfa93f05 P 233e7d29aac04505bf4e65f0a39a0c4c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:24:04.515064 15046 tablet_replica.cc:333] T 441d780c3945463f92f31170f31e37a8 P 233e7d29aac04505bf4e65f0a39a0c4c: stopping tablet replica
I20260709 13:24:04.515784 15046 raft_consensus.cc:2243] T 441d780c3945463f92f31170f31e37a8 P 233e7d29aac04505bf4e65f0a39a0c4c [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:24:04.516806 15046 raft_consensus.cc:2272] T 441d780c3945463f92f31170f31e37a8 P 233e7d29aac04505bf4e65f0a39a0c4c [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:24:04.520510 15046 tablet_replica.cc:333] T a820954e07a94582ac0613342fcffde5 P 233e7d29aac04505bf4e65f0a39a0c4c: stopping tablet replica
I20260709 13:24:04.521610 15046 raft_consensus.cc:2243] T a820954e07a94582ac0613342fcffde5 P 233e7d29aac04505bf4e65f0a39a0c4c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:24:04.522341 15046 raft_consensus.cc:2272] T a820954e07a94582ac0613342fcffde5 P 233e7d29aac04505bf4e65f0a39a0c4c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:24:04.525074 15046 tablet_replica.cc:333] T 03e9bfbd2f774476a4a377367796a2e8 P 233e7d29aac04505bf4e65f0a39a0c4c: stopping tablet replica
I20260709 13:24:04.525897 15046 raft_consensus.cc:2243] T 03e9bfbd2f774476a4a377367796a2e8 P 233e7d29aac04505bf4e65f0a39a0c4c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:24:04.526428 15046 raft_consensus.cc:2272] T 03e9bfbd2f774476a4a377367796a2e8 P 233e7d29aac04505bf4e65f0a39a0c4c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:24:04.528458 15046 tablet_replica.cc:333] T 579da2d6aabb4ae9a2b1802e1358068b P 233e7d29aac04505bf4e65f0a39a0c4c: stopping tablet replica
I20260709 13:24:04.529031 15046 raft_consensus.cc:2243] T 579da2d6aabb4ae9a2b1802e1358068b P 233e7d29aac04505bf4e65f0a39a0c4c [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:24:04.529458 15046 raft_consensus.cc:2272] T 579da2d6aabb4ae9a2b1802e1358068b P 233e7d29aac04505bf4e65f0a39a0c4c [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:24:04.531451 15046 tablet_replica.cc:333] T 434a54fcee7d4aeeab42f739855c0e8a P 233e7d29aac04505bf4e65f0a39a0c4c: stopping tablet replica
I20260709 13:24:04.531965 15046 raft_consensus.cc:2243] T 434a54fcee7d4aeeab42f739855c0e8a P 233e7d29aac04505bf4e65f0a39a0c4c [term 4 LEADER]: Raft consensus shutting down.
I20260709 13:24:04.532747 15046 raft_consensus.cc:2272] T 434a54fcee7d4aeeab42f739855c0e8a P 233e7d29aac04505bf4e65f0a39a0c4c [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:24:04.535210 15046 tablet_replica.cc:333] T e15820fa8a7d4e1fb6ad23a64ca22f90 P 233e7d29aac04505bf4e65f0a39a0c4c: stopping tablet replica
I20260709 13:24:04.535820 15046 raft_consensus.cc:2243] T e15820fa8a7d4e1fb6ad23a64ca22f90 P 233e7d29aac04505bf4e65f0a39a0c4c [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:24:04.536288 15046 raft_consensus.cc:2272] T e15820fa8a7d4e1fb6ad23a64ca22f90 P 233e7d29aac04505bf4e65f0a39a0c4c [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:24:04.538350 25424 consensus_peers.cc:597] T 579da2d6aabb4ae9a2b1802e1358068b P 9868dce521294d0b816bba7648f4ae5a -> Peer 43999d813e564dc688fad116090c95ca (127.14.177.129:38155): Couldn't send request to peer 43999d813e564dc688fad116090c95ca. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:38155: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:24:04.538789 15046 tablet_replica.cc:333] T 3fe5a944a342411ea923eb39071ac770 P 233e7d29aac04505bf4e65f0a39a0c4c: stopping tablet replica
I20260709 13:24:04.539492 15046 raft_consensus.cc:2243] T 3fe5a944a342411ea923eb39071ac770 P 233e7d29aac04505bf4e65f0a39a0c4c [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:24:04.540226 15046 raft_consensus.cc:2272] T 3fe5a944a342411ea923eb39071ac770 P 233e7d29aac04505bf4e65f0a39a0c4c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:24:04.542403 15046 tablet_replica.cc:333] T 28c0b6fc9bd64d60828d705c6a9ab1e4 P 233e7d29aac04505bf4e65f0a39a0c4c: stopping tablet replica
I20260709 13:24:04.542904 15046 raft_consensus.cc:2243] T 28c0b6fc9bd64d60828d705c6a9ab1e4 P 233e7d29aac04505bf4e65f0a39a0c4c [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:24:04.543421 15046 raft_consensus.cc:2272] T 28c0b6fc9bd64d60828d705c6a9ab1e4 P 233e7d29aac04505bf4e65f0a39a0c4c [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:24:04.545293 15046 tablet_replica.cc:333] T 5c4ab1978a47481c8576d466bacf49d2 P 233e7d29aac04505bf4e65f0a39a0c4c: stopping tablet replica
I20260709 13:24:04.545801 15046 raft_consensus.cc:2243] T 5c4ab1978a47481c8576d466bacf49d2 P 233e7d29aac04505bf4e65f0a39a0c4c [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:24:04.546583 15046 raft_consensus.cc:2272] T 5c4ab1978a47481c8576d466bacf49d2 P 233e7d29aac04505bf4e65f0a39a0c4c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:24:04.548444 15046 tablet_replica.cc:333] T b2f14aed05e24a98ad83535029b795dc P 233e7d29aac04505bf4e65f0a39a0c4c: stopping tablet replica
I20260709 13:24:04.548943 15046 raft_consensus.cc:2243] T b2f14aed05e24a98ad83535029b795dc P 233e7d29aac04505bf4e65f0a39a0c4c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:24:04.549496 15046 raft_consensus.cc:2272] T b2f14aed05e24a98ad83535029b795dc P 233e7d29aac04505bf4e65f0a39a0c4c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:24:04.552248 15046 tablet_replica.cc:333] T 55e8ed3aee404ce698d0626a87bd03aa P 233e7d29aac04505bf4e65f0a39a0c4c: stopping tablet replica
I20260709 13:24:04.552959 15046 raft_consensus.cc:2243] T 55e8ed3aee404ce698d0626a87bd03aa P 233e7d29aac04505bf4e65f0a39a0c4c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:24:04.553459 15046 raft_consensus.cc:2272] T 55e8ed3aee404ce698d0626a87bd03aa P 233e7d29aac04505bf4e65f0a39a0c4c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:24:04.556288 15046 tablet_replica.cc:333] T fa0da2c845dc4059968332f4db2e0aa6 P 233e7d29aac04505bf4e65f0a39a0c4c: stopping tablet replica
I20260709 13:24:04.557071 15046 raft_consensus.cc:2243] T fa0da2c845dc4059968332f4db2e0aa6 P 233e7d29aac04505bf4e65f0a39a0c4c [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:24:04.557617 15046 raft_consensus.cc:2272] T fa0da2c845dc4059968332f4db2e0aa6 P 233e7d29aac04505bf4e65f0a39a0c4c [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:24:04.560241 15046 tablet_replica.cc:333] T 3a3317f4c3514df1b2bb8e44a86be92f P 233e7d29aac04505bf4e65f0a39a0c4c: stopping tablet replica
I20260709 13:24:04.560933 15046 raft_consensus.cc:2243] T 3a3317f4c3514df1b2bb8e44a86be92f P 233e7d29aac04505bf4e65f0a39a0c4c [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:24:04.561889 15046 raft_consensus.cc:2272] T 3a3317f4c3514df1b2bb8e44a86be92f P 233e7d29aac04505bf4e65f0a39a0c4c [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:24:04.564494 15046 tablet_replica.cc:333] T cfad575709c943cba40ccab278350770 P 233e7d29aac04505bf4e65f0a39a0c4c: stopping tablet replica
I20260709 13:24:04.565153 15046 raft_consensus.cc:2243] T cfad575709c943cba40ccab278350770 P 233e7d29aac04505bf4e65f0a39a0c4c [term 5 LEADER]: Raft consensus shutting down.
I20260709 13:24:04.565846 15046 raft_consensus.cc:2272] T cfad575709c943cba40ccab278350770 P 233e7d29aac04505bf4e65f0a39a0c4c [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 13:24:04.568223 15046 tablet_replica.cc:333] T 13bb3a23315a41c7a40e10b954b15c8e P 233e7d29aac04505bf4e65f0a39a0c4c: stopping tablet replica
I20260709 13:24:04.569042 15046 raft_consensus.cc:2243] T 13bb3a23315a41c7a40e10b954b15c8e P 233e7d29aac04505bf4e65f0a39a0c4c [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:24:04.569623 15046 raft_consensus.cc:2272] T 13bb3a23315a41c7a40e10b954b15c8e P 233e7d29aac04505bf4e65f0a39a0c4c [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:24:04.571558 25425 consensus_peers.cc:597] T 0c4f6b7401db47d59a4f34ed0e8b5350 P 9868dce521294d0b816bba7648f4ae5a -> Peer 233e7d29aac04505bf4e65f0a39a0c4c (127.14.177.130:42957): Couldn't send request to peer 233e7d29aac04505bf4e65f0a39a0c4c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:24:04.572450 15046 tablet_replica.cc:333] T 0cb89d70b4ce4aa8992fb68480bec8a2 P 233e7d29aac04505bf4e65f0a39a0c4c: stopping tablet replica
I20260709 13:24:04.573063 15046 raft_consensus.cc:2243] T 0cb89d70b4ce4aa8992fb68480bec8a2 P 233e7d29aac04505bf4e65f0a39a0c4c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:24:04.573535 15046 raft_consensus.cc:2272] T 0cb89d70b4ce4aa8992fb68480bec8a2 P 233e7d29aac04505bf4e65f0a39a0c4c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:24:04.576301 15046 tablet_replica.cc:333] T 8ef061028f38450e83324ac78cdaa14c P 233e7d29aac04505bf4e65f0a39a0c4c: stopping tablet replica
I20260709 13:24:04.577011 15046 raft_consensus.cc:2243] T 8ef061028f38450e83324ac78cdaa14c P 233e7d29aac04505bf4e65f0a39a0c4c [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:24:04.577514 15046 raft_consensus.cc:2272] T 8ef061028f38450e83324ac78cdaa14c P 233e7d29aac04505bf4e65f0a39a0c4c [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:24:04.579500 25351 consensus_peers.cc:597] T ad32097cd4dc48b9a69595f1651abd07 P 3e07d64466934da4bb3c02262b1c2e38 -> Peer 233e7d29aac04505bf4e65f0a39a0c4c (127.14.177.130:42957): Couldn't send request to peer 233e7d29aac04505bf4e65f0a39a0c4c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:24:04.580432 15046 tablet_replica.cc:333] T b7ff8dde9f5d4c49b9203d902f6b94c4 P 233e7d29aac04505bf4e65f0a39a0c4c: stopping tablet replica
I20260709 13:24:04.581086 15046 raft_consensus.cc:2243] T b7ff8dde9f5d4c49b9203d902f6b94c4 P 233e7d29aac04505bf4e65f0a39a0c4c [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:24:04.581532 15046 raft_consensus.cc:2272] T b7ff8dde9f5d4c49b9203d902f6b94c4 P 233e7d29aac04505bf4e65f0a39a0c4c [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:24:04.583650 15046 tablet_replica.cc:333] T 467a6b42ca6548269c12dddde696f7af P 233e7d29aac04505bf4e65f0a39a0c4c: stopping tablet replica
I20260709 13:24:04.584158 15046 raft_consensus.cc:2243] T 467a6b42ca6548269c12dddde696f7af P 233e7d29aac04505bf4e65f0a39a0c4c [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:24:04.584558 15046 raft_consensus.cc:2272] T 467a6b42ca6548269c12dddde696f7af P 233e7d29aac04505bf4e65f0a39a0c4c [term 3 FOLLOWER]: Raft consensus is shut down!
W20260709 13:24:04.604162 25351 consensus_peers.cc:597] T 0cb89d70b4ce4aa8992fb68480bec8a2 P 3e07d64466934da4bb3c02262b1c2e38 -> Peer 233e7d29aac04505bf4e65f0a39a0c4c (127.14.177.130:42957): Couldn't send request to peer 233e7d29aac04505bf4e65f0a39a0c4c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:24:04.626894 25425 consensus_peers.cc:597] T 8ef061028f38450e83324ac78cdaa14c P 9868dce521294d0b816bba7648f4ae5a -> Peer 233e7d29aac04505bf4e65f0a39a0c4c (127.14.177.130:42957): Couldn't send request to peer 233e7d29aac04505bf4e65f0a39a0c4c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:24:04.631500 25351 consensus_peers.cc:597] T 13bb3a23315a41c7a40e10b954b15c8e P 3e07d64466934da4bb3c02262b1c2e38 -> Peer 233e7d29aac04505bf4e65f0a39a0c4c (127.14.177.130:42957): Couldn't send request to peer 233e7d29aac04505bf4e65f0a39a0c4c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:24:04.662947 15046 tablet_server.cc:196] TabletServer@127.14.177.130:0 shutdown complete.
W20260709 13:24:04.663482 25425 consensus_peers.cc:597] T a820954e07a94582ac0613342fcffde5 P 9868dce521294d0b816bba7648f4ae5a -> Peer 233e7d29aac04505bf4e65f0a39a0c4c (127.14.177.130:42957): Couldn't send request to peer 233e7d29aac04505bf4e65f0a39a0c4c. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.130:42957: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:24:04.677476 25425 consensus_peers.cc:597] T aad1f7937f854cc1a726223269da5d5a P 9868dce521294d0b816bba7648f4ae5a -> Peer 233e7d29aac04505bf4e65f0a39a0c4c (127.14.177.130:42957): Couldn't send request to peer 233e7d29aac04505bf4e65f0a39a0c4c. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.130:42957: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:24:04.687426 25351 consensus_peers.cc:597] T beacc815ef954fb692e90544bfa93f05 P 3e07d64466934da4bb3c02262b1c2e38 -> Peer 233e7d29aac04505bf4e65f0a39a0c4c (127.14.177.130:42957): Couldn't send request to peer 233e7d29aac04505bf4e65f0a39a0c4c. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.130:42957: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:24:04.692852 25425 consensus_peers.cc:597] T fa0da2c845dc4059968332f4db2e0aa6 P 9868dce521294d0b816bba7648f4ae5a -> Peer 233e7d29aac04505bf4e65f0a39a0c4c (127.14.177.130:42957): Couldn't send request to peer 233e7d29aac04505bf4e65f0a39a0c4c. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.130:42957: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:24:04.749794 15046 tablet_server.cc:179] TabletServer@127.14.177.131:0 shutting down...
W20260709 13:24:04.749943 25351 consensus_peers.cc:597] T b2f14aed05e24a98ad83535029b795dc P 3e07d64466934da4bb3c02262b1c2e38 -> Peer 233e7d29aac04505bf4e65f0a39a0c4c (127.14.177.130:42957): Couldn't send request to peer 233e7d29aac04505bf4e65f0a39a0c4c. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.130:42957: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:24:04.768929 25422 consensus_peers.cc:597] T 55e8ed3aee404ce698d0626a87bd03aa P 9868dce521294d0b816bba7648f4ae5a -> Peer 3e07d64466934da4bb3c02262b1c2e38 (127.14.177.131:41539): Couldn't send request to peer 3e07d64466934da4bb3c02262b1c2e38. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:24:04.778607 25422 consensus_peers.cc:597] T a820954e07a94582ac0613342fcffde5 P 9868dce521294d0b816bba7648f4ae5a -> Peer 3e07d64466934da4bb3c02262b1c2e38 (127.14.177.131:41539): Couldn't send request to peer 3e07d64466934da4bb3c02262b1c2e38. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:24:04.783778 25425 consensus_peers.cc:597] T 55e8ed3aee404ce698d0626a87bd03aa P 9868dce521294d0b816bba7648f4ae5a -> Peer 233e7d29aac04505bf4e65f0a39a0c4c (127.14.177.130:42957): Couldn't send request to peer 233e7d29aac04505bf4e65f0a39a0c4c. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.130:42957: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:24:04.803468 15046 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:24:04.804301 15046 tablet_replica.cc:333] T 03e9bfbd2f774476a4a377367796a2e8 P 3e07d64466934da4bb3c02262b1c2e38: stopping tablet replica
I20260709 13:24:04.804982 15046 raft_consensus.cc:2243] T 03e9bfbd2f774476a4a377367796a2e8 P 3e07d64466934da4bb3c02262b1c2e38 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:24:04.805828 15046 raft_consensus.cc:2272] T 03e9bfbd2f774476a4a377367796a2e8 P 3e07d64466934da4bb3c02262b1c2e38 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:24:04.807905 15046 tablet_replica.cc:333] T 55e8ed3aee404ce698d0626a87bd03aa P 3e07d64466934da4bb3c02262b1c2e38: stopping tablet replica
I20260709 13:24:04.808691 15046 raft_consensus.cc:2243] T 55e8ed3aee404ce698d0626a87bd03aa P 3e07d64466934da4bb3c02262b1c2e38 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:24:04.809290 15046 raft_consensus.cc:2272] T 55e8ed3aee404ce698d0626a87bd03aa P 3e07d64466934da4bb3c02262b1c2e38 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:24:04.812237 15046 tablet_replica.cc:333] T 5c7e7acbb09f4d49828fa652ca6abaac P 3e07d64466934da4bb3c02262b1c2e38: stopping tablet replica
I20260709 13:24:04.812995 15046 raft_consensus.cc:2243] T 5c7e7acbb09f4d49828fa652ca6abaac P 3e07d64466934da4bb3c02262b1c2e38 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:24:04.813570 15046 raft_consensus.cc:2272] T 5c7e7acbb09f4d49828fa652ca6abaac P 3e07d64466934da4bb3c02262b1c2e38 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:24:04.815884 15046 tablet_replica.cc:333] T e311eb90bad44c80a131b72edcd22b06 P 3e07d64466934da4bb3c02262b1c2e38: stopping tablet replica
I20260709 13:24:04.816447 15046 raft_consensus.cc:2243] T e311eb90bad44c80a131b72edcd22b06 P 3e07d64466934da4bb3c02262b1c2e38 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:24:04.816861 15046 raft_consensus.cc:2272] T e311eb90bad44c80a131b72edcd22b06 P 3e07d64466934da4bb3c02262b1c2e38 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:24:04.818774 15046 tablet_replica.cc:333] T 72b56266e8a54776b9113c7a786e1952 P 3e07d64466934da4bb3c02262b1c2e38: stopping tablet replica
I20260709 13:24:04.819346 15046 raft_consensus.cc:2243] T 72b56266e8a54776b9113c7a786e1952 P 3e07d64466934da4bb3c02262b1c2e38 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:24:04.820118 15046 raft_consensus.cc:2272] T 72b56266e8a54776b9113c7a786e1952 P 3e07d64466934da4bb3c02262b1c2e38 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:24:04.821924 15046 tablet_replica.cc:333] T b155b866f10547fd8239579a81ecf5f9 P 3e07d64466934da4bb3c02262b1c2e38: stopping tablet replica
I20260709 13:24:04.822453 15046 raft_consensus.cc:2243] T b155b866f10547fd8239579a81ecf5f9 P 3e07d64466934da4bb3c02262b1c2e38 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:24:04.822850 15046 raft_consensus.cc:2272] T b155b866f10547fd8239579a81ecf5f9 P 3e07d64466934da4bb3c02262b1c2e38 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:24:04.824702 15046 tablet_replica.cc:333] T b9d550d881b04735bbf89751952c09ec P 3e07d64466934da4bb3c02262b1c2e38: stopping tablet replica
I20260709 13:24:04.825297 15046 raft_consensus.cc:2243] T b9d550d881b04735bbf89751952c09ec P 3e07d64466934da4bb3c02262b1c2e38 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:24:04.825703 15046 raft_consensus.cc:2272] T b9d550d881b04735bbf89751952c09ec P 3e07d64466934da4bb3c02262b1c2e38 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:24:04.827657 15046 tablet_replica.cc:333] T 9ab90e3e08fb4528b2a47d995e9e26d3 P 3e07d64466934da4bb3c02262b1c2e38: stopping tablet replica
I20260709 13:24:04.828197 15046 raft_consensus.cc:2243] T 9ab90e3e08fb4528b2a47d995e9e26d3 P 3e07d64466934da4bb3c02262b1c2e38 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:24:04.829027 15046 raft_consensus.cc:2272] T 9ab90e3e08fb4528b2a47d995e9e26d3 P 3e07d64466934da4bb3c02262b1c2e38 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:24:04.830848 15046 tablet_replica.cc:333] T 4d837697cd0f402cacf9db89a5d92652 P 3e07d64466934da4bb3c02262b1c2e38: stopping tablet replica
I20260709 13:24:04.831460 15046 raft_consensus.cc:2243] T 4d837697cd0f402cacf9db89a5d92652 P 3e07d64466934da4bb3c02262b1c2e38 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:24:04.832283 15046 raft_consensus.cc:2272] T 4d837697cd0f402cacf9db89a5d92652 P 3e07d64466934da4bb3c02262b1c2e38 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:24:04.834254 15046 tablet_replica.cc:333] T 0ed5edbe32044d1c83b0590dbb899801 P 3e07d64466934da4bb3c02262b1c2e38: stopping tablet replica
I20260709 13:24:04.834793 15046 raft_consensus.cc:2243] T 0ed5edbe32044d1c83b0590dbb899801 P 3e07d64466934da4bb3c02262b1c2e38 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:24:04.835321 15046 raft_consensus.cc:2272] T 0ed5edbe32044d1c83b0590dbb899801 P 3e07d64466934da4bb3c02262b1c2e38 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:24:04.837563 15046 tablet_replica.cc:333] T 434a18a5109e4ea78349ded3968b2709 P 3e07d64466934da4bb3c02262b1c2e38: stopping tablet replica
I20260709 13:24:04.838357 15046 raft_consensus.cc:2243] T 434a18a5109e4ea78349ded3968b2709 P 3e07d64466934da4bb3c02262b1c2e38 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:24:04.838896 15046 raft_consensus.cc:2272] T 434a18a5109e4ea78349ded3968b2709 P 3e07d64466934da4bb3c02262b1c2e38 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:24:04.841866 15046 tablet_replica.cc:333] T 87085ac2575e4b6a86dd1fc21e6f718b P 3e07d64466934da4bb3c02262b1c2e38: stopping tablet replica
I20260709 13:24:04.842705 15046 raft_consensus.cc:2243] T 87085ac2575e4b6a86dd1fc21e6f718b P 3e07d64466934da4bb3c02262b1c2e38 [term 2 LEADER]: Raft consensus shutting down.
W20260709 13:24:04.843250 25351 consensus_peers.cc:597] T 89a26dd3ac414f6081b98fa32c2f346f P 3e07d64466934da4bb3c02262b1c2e38 -> Peer 233e7d29aac04505bf4e65f0a39a0c4c (127.14.177.130:42957): Couldn't send request to peer 233e7d29aac04505bf4e65f0a39a0c4c. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.130:42957: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:24:04.843730 15046 raft_consensus.cc:2272] T 87085ac2575e4b6a86dd1fc21e6f718b P 3e07d64466934da4bb3c02262b1c2e38 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:24:04.845971 15046 tablet_replica.cc:333] T cfad575709c943cba40ccab278350770 P 3e07d64466934da4bb3c02262b1c2e38: stopping tablet replica
I20260709 13:24:04.846568 15046 raft_consensus.cc:2243] T cfad575709c943cba40ccab278350770 P 3e07d64466934da4bb3c02262b1c2e38 [term 5 FOLLOWER]: Raft consensus shutting down.
I20260709 13:24:04.847066 15046 raft_consensus.cc:2272] T cfad575709c943cba40ccab278350770 P 3e07d64466934da4bb3c02262b1c2e38 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 13:24:04.849125 15046 tablet_replica.cc:333] T 13bb3a23315a41c7a40e10b954b15c8e P 3e07d64466934da4bb3c02262b1c2e38: stopping tablet replica
I20260709 13:24:04.849777 15046 raft_consensus.cc:2243] T 13bb3a23315a41c7a40e10b954b15c8e P 3e07d64466934da4bb3c02262b1c2e38 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:24:04.850605 15046 raft_consensus.cc:2272] T 13bb3a23315a41c7a40e10b954b15c8e P 3e07d64466934da4bb3c02262b1c2e38 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:24:04.853084 15046 tablet_replica.cc:333] T 0cb89d70b4ce4aa8992fb68480bec8a2 P 3e07d64466934da4bb3c02262b1c2e38: stopping tablet replica
W20260709 13:24:04.853421 25422 consensus_peers.cc:597] T aad1f7937f854cc1a726223269da5d5a P 9868dce521294d0b816bba7648f4ae5a -> Peer 3e07d64466934da4bb3c02262b1c2e38 (127.14.177.131:41539): Couldn't send request to peer 3e07d64466934da4bb3c02262b1c2e38. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:24:04.853680 15046 raft_consensus.cc:2243] T 0cb89d70b4ce4aa8992fb68480bec8a2 P 3e07d64466934da4bb3c02262b1c2e38 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:24:04.854588 15046 raft_consensus.cc:2272] T 0cb89d70b4ce4aa8992fb68480bec8a2 P 3e07d64466934da4bb3c02262b1c2e38 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:24:04.856483 15046 tablet_replica.cc:333] T c902bbc7ec0848ba82a04870eab1e7f5 P 3e07d64466934da4bb3c02262b1c2e38: stopping tablet replica
I20260709 13:24:04.857048 15046 raft_consensus.cc:2243] T c902bbc7ec0848ba82a04870eab1e7f5 P 3e07d64466934da4bb3c02262b1c2e38 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:24:04.857475 15046 raft_consensus.cc:2272] T c902bbc7ec0848ba82a04870eab1e7f5 P 3e07d64466934da4bb3c02262b1c2e38 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:24:04.859375 15046 tablet_replica.cc:333] T 9ffa42c3b2ff48bc87310e9ab5d218ce P 3e07d64466934da4bb3c02262b1c2e38: stopping tablet replica
I20260709 13:24:04.859912 15046 raft_consensus.cc:2243] T 9ffa42c3b2ff48bc87310e9ab5d218ce P 3e07d64466934da4bb3c02262b1c2e38 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:24:04.860656 15046 raft_consensus.cc:2272] T 9ffa42c3b2ff48bc87310e9ab5d218ce P 3e07d64466934da4bb3c02262b1c2e38 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:24:04.862569 15046 tablet_replica.cc:333] T ad32097cd4dc48b9a69595f1651abd07 P 3e07d64466934da4bb3c02262b1c2e38: stopping tablet replica
I20260709 13:24:04.863113 15046 raft_consensus.cc:2243] T ad32097cd4dc48b9a69595f1651abd07 P 3e07d64466934da4bb3c02262b1c2e38 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:24:04.863952 15046 raft_consensus.cc:2272] T ad32097cd4dc48b9a69595f1651abd07 P 3e07d64466934da4bb3c02262b1c2e38 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:24:04.865911 15046 tablet_replica.cc:333] T 89a26dd3ac414f6081b98fa32c2f346f P 3e07d64466934da4bb3c02262b1c2e38: stopping tablet replica
I20260709 13:24:04.866462 15046 raft_consensus.cc:2243] T 89a26dd3ac414f6081b98fa32c2f346f P 3e07d64466934da4bb3c02262b1c2e38 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:24:04.867321 15046 raft_consensus.cc:2272] T 89a26dd3ac414f6081b98fa32c2f346f P 3e07d64466934da4bb3c02262b1c2e38 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:24:04.869220 15046 tablet_replica.cc:333] T ce98a3f3b3eb4c63bcaa355ec7cd64a4 P 3e07d64466934da4bb3c02262b1c2e38: stopping tablet replica
I20260709 13:24:04.869791 15046 raft_consensus.cc:2243] T ce98a3f3b3eb4c63bcaa355ec7cd64a4 P 3e07d64466934da4bb3c02262b1c2e38 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:24:04.870240 15046 raft_consensus.cc:2272] T ce98a3f3b3eb4c63bcaa355ec7cd64a4 P 3e07d64466934da4bb3c02262b1c2e38 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:24:04.872520 15046 tablet_replica.cc:333] T beacc815ef954fb692e90544bfa93f05 P 3e07d64466934da4bb3c02262b1c2e38: stopping tablet replica
I20260709 13:24:04.873070 15046 raft_consensus.cc:2243] T beacc815ef954fb692e90544bfa93f05 P 3e07d64466934da4bb3c02262b1c2e38 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:24:04.873867 15046 raft_consensus.cc:2272] T beacc815ef954fb692e90544bfa93f05 P 3e07d64466934da4bb3c02262b1c2e38 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:24:04.876106 15046 tablet_replica.cc:333] T 434a54fcee7d4aeeab42f739855c0e8a P 3e07d64466934da4bb3c02262b1c2e38: stopping tablet replica
I20260709 13:24:04.876700 15046 raft_consensus.cc:2243] T 434a54fcee7d4aeeab42f739855c0e8a P 3e07d64466934da4bb3c02262b1c2e38 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:24:04.877146 15046 raft_consensus.cc:2272] T 434a54fcee7d4aeeab42f739855c0e8a P 3e07d64466934da4bb3c02262b1c2e38 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:24:04.879961 15046 tablet_replica.cc:333] T e15820fa8a7d4e1fb6ad23a64ca22f90 P 3e07d64466934da4bb3c02262b1c2e38: stopping tablet replica
I20260709 13:24:04.881037 15046 raft_consensus.cc:2243] T e15820fa8a7d4e1fb6ad23a64ca22f90 P 3e07d64466934da4bb3c02262b1c2e38 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:24:04.881551 15046 raft_consensus.cc:2272] T e15820fa8a7d4e1fb6ad23a64ca22f90 P 3e07d64466934da4bb3c02262b1c2e38 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:24:04.884879 15046 tablet_replica.cc:333] T 821a5866429f4173820fb64336abee02 P 3e07d64466934da4bb3c02262b1c2e38: stopping tablet replica
I20260709 13:24:04.885679 15046 raft_consensus.cc:2243] T 821a5866429f4173820fb64336abee02 P 3e07d64466934da4bb3c02262b1c2e38 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:24:04.886870 15046 raft_consensus.cc:2272] T 821a5866429f4173820fb64336abee02 P 3e07d64466934da4bb3c02262b1c2e38 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:24:04.890515 15046 tablet_replica.cc:333] T 79d4a12ef58140568c60a24ed4c12f83 P 3e07d64466934da4bb3c02262b1c2e38: stopping tablet replica
I20260709 13:24:04.891723 15046 raft_consensus.cc:2243] T 79d4a12ef58140568c60a24ed4c12f83 P 3e07d64466934da4bb3c02262b1c2e38 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:24:04.892395 15046 raft_consensus.cc:2272] T 79d4a12ef58140568c60a24ed4c12f83 P 3e07d64466934da4bb3c02262b1c2e38 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:24:04.895484 15046 tablet_replica.cc:333] T bbab48ba177049dcba6de106fd889cfb P 3e07d64466934da4bb3c02262b1c2e38: stopping tablet replica
I20260709 13:24:04.896214 15046 raft_consensus.cc:2243] T bbab48ba177049dcba6de106fd889cfb P 3e07d64466934da4bb3c02262b1c2e38 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:24:04.896750 15046 raft_consensus.cc:2272] T bbab48ba177049dcba6de106fd889cfb P 3e07d64466934da4bb3c02262b1c2e38 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:24:04.899367 15046 tablet_replica.cc:333] T 0d29b499fa4c41f1b974782cd0c9c1c2 P 3e07d64466934da4bb3c02262b1c2e38: stopping tablet replica
I20260709 13:24:04.900097 15046 raft_consensus.cc:2243] T 0d29b499fa4c41f1b974782cd0c9c1c2 P 3e07d64466934da4bb3c02262b1c2e38 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:24:04.900671 15046 raft_consensus.cc:2272] T 0d29b499fa4c41f1b974782cd0c9c1c2 P 3e07d64466934da4bb3c02262b1c2e38 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:24:04.903947 15046 tablet_replica.cc:333] T ba8a000f5abe4158965449995af84b18 P 3e07d64466934da4bb3c02262b1c2e38: stopping tablet replica
I20260709 13:24:04.904906 15046 raft_consensus.cc:2243] T ba8a000f5abe4158965449995af84b18 P 3e07d64466934da4bb3c02262b1c2e38 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:24:04.905503 15046 raft_consensus.cc:2272] T ba8a000f5abe4158965449995af84b18 P 3e07d64466934da4bb3c02262b1c2e38 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:24:04.908536 15046 tablet_replica.cc:333] T aad1f7937f854cc1a726223269da5d5a P 3e07d64466934da4bb3c02262b1c2e38: stopping tablet replica
I20260709 13:24:04.909385 15046 raft_consensus.cc:2243] T aad1f7937f854cc1a726223269da5d5a P 3e07d64466934da4bb3c02262b1c2e38 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:24:04.909922 15046 raft_consensus.cc:2272] T aad1f7937f854cc1a726223269da5d5a P 3e07d64466934da4bb3c02262b1c2e38 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:24:04.913043 15046 tablet_replica.cc:333] T 3fe5a944a342411ea923eb39071ac770 P 3e07d64466934da4bb3c02262b1c2e38: stopping tablet replica
I20260709 13:24:04.913916 15046 raft_consensus.cc:2243] T 3fe5a944a342411ea923eb39071ac770 P 3e07d64466934da4bb3c02262b1c2e38 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:24:04.914510 15046 raft_consensus.cc:2272] T 3fe5a944a342411ea923eb39071ac770 P 3e07d64466934da4bb3c02262b1c2e38 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:24:04.917312 15046 tablet_replica.cc:333] T b7ff8dde9f5d4c49b9203d902f6b94c4 P 3e07d64466934da4bb3c02262b1c2e38: stopping tablet replica
I20260709 13:24:04.918211 15046 raft_consensus.cc:2243] T b7ff8dde9f5d4c49b9203d902f6b94c4 P 3e07d64466934da4bb3c02262b1c2e38 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:24:04.918785 15046 raft_consensus.cc:2272] T b7ff8dde9f5d4c49b9203d902f6b94c4 P 3e07d64466934da4bb3c02262b1c2e38 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:24:04.921924 15046 tablet_replica.cc:333] T 91ac744179de48e5bb924b6dfd9f9c9b P 3e07d64466934da4bb3c02262b1c2e38: stopping tablet replica
I20260709 13:24:04.922865 15046 raft_consensus.cc:2243] T 91ac744179de48e5bb924b6dfd9f9c9b P 3e07d64466934da4bb3c02262b1c2e38 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:24:04.923623 15046 raft_consensus.cc:2272] T 91ac744179de48e5bb924b6dfd9f9c9b P 3e07d64466934da4bb3c02262b1c2e38 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:24:04.926729 15046 tablet_replica.cc:333] T 1f15ec62698c4a8cbd84f2b3d516d2ea P 3e07d64466934da4bb3c02262b1c2e38: stopping tablet replica
I20260709 13:24:04.927734 15046 raft_consensus.cc:2243] T 1f15ec62698c4a8cbd84f2b3d516d2ea P 3e07d64466934da4bb3c02262b1c2e38 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:24:04.928398 15046 raft_consensus.cc:2272] T 1f15ec62698c4a8cbd84f2b3d516d2ea P 3e07d64466934da4bb3c02262b1c2e38 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:24:04.931188 15046 tablet_replica.cc:333] T 280d65245f4f4358b54bef93e18aca92 P 3e07d64466934da4bb3c02262b1c2e38: stopping tablet replica
I20260709 13:24:04.931977 15046 raft_consensus.cc:2243] T 280d65245f4f4358b54bef93e18aca92 P 3e07d64466934da4bb3c02262b1c2e38 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:24:04.932474 15046 raft_consensus.cc:2272] T 280d65245f4f4358b54bef93e18aca92 P 3e07d64466934da4bb3c02262b1c2e38 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:24:04.935190 15046 tablet_replica.cc:333] T f1ac2f4d2fe2424bb470df42199219e2 P 3e07d64466934da4bb3c02262b1c2e38: stopping tablet replica
I20260709 13:24:04.935894 15046 raft_consensus.cc:2243] T f1ac2f4d2fe2424bb470df42199219e2 P 3e07d64466934da4bb3c02262b1c2e38 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:24:04.936476 15046 raft_consensus.cc:2272] T f1ac2f4d2fe2424bb470df42199219e2 P 3e07d64466934da4bb3c02262b1c2e38 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:24:04.939087 15046 tablet_replica.cc:333] T 1550ed2d2a764c08adffae679599ba41 P 3e07d64466934da4bb3c02262b1c2e38: stopping tablet replica
I20260709 13:24:04.939807 15046 raft_consensus.cc:2243] T 1550ed2d2a764c08adffae679599ba41 P 3e07d64466934da4bb3c02262b1c2e38 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:24:04.940346 15046 raft_consensus.cc:2272] T 1550ed2d2a764c08adffae679599ba41 P 3e07d64466934da4bb3c02262b1c2e38 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:24:04.943041 15046 tablet_replica.cc:333] T a820954e07a94582ac0613342fcffde5 P 3e07d64466934da4bb3c02262b1c2e38: stopping tablet replica
I20260709 13:24:04.943810 15046 raft_consensus.cc:2243] T a820954e07a94582ac0613342fcffde5 P 3e07d64466934da4bb3c02262b1c2e38 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:24:04.944343 15046 raft_consensus.cc:2272] T a820954e07a94582ac0613342fcffde5 P 3e07d64466934da4bb3c02262b1c2e38 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:24:04.946928 15046 tablet_replica.cc:333] T 0e71b6f5c69b4b35a8ac51c22c7d5f0d P 3e07d64466934da4bb3c02262b1c2e38: stopping tablet replica
I20260709 13:24:04.947649 15046 raft_consensus.cc:2243] T 0e71b6f5c69b4b35a8ac51c22c7d5f0d P 3e07d64466934da4bb3c02262b1c2e38 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:24:04.948688 15046 raft_consensus.cc:2272] T 0e71b6f5c69b4b35a8ac51c22c7d5f0d P 3e07d64466934da4bb3c02262b1c2e38 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:24:04.951035 15046 tablet_replica.cc:333] T d0a0e4d32ce943fbb773bb840863647b P 3e07d64466934da4bb3c02262b1c2e38: stopping tablet replica
I20260709 13:24:04.951596 15046 raft_consensus.cc:2243] T d0a0e4d32ce943fbb773bb840863647b P 3e07d64466934da4bb3c02262b1c2e38 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:24:04.951996 15046 raft_consensus.cc:2272] T d0a0e4d32ce943fbb773bb840863647b P 3e07d64466934da4bb3c02262b1c2e38 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:24:04.953841 15046 tablet_replica.cc:333] T 467a6b42ca6548269c12dddde696f7af P 3e07d64466934da4bb3c02262b1c2e38: stopping tablet replica
I20260709 13:24:04.954350 15046 raft_consensus.cc:2243] T 467a6b42ca6548269c12dddde696f7af P 3e07d64466934da4bb3c02262b1c2e38 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:24:04.955140 15046 raft_consensus.cc:2272] T 467a6b42ca6548269c12dddde696f7af P 3e07d64466934da4bb3c02262b1c2e38 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:24:04.957317 15046 tablet_replica.cc:333] T 5c4ab1978a47481c8576d466bacf49d2 P 3e07d64466934da4bb3c02262b1c2e38: stopping tablet replica
I20260709 13:24:04.957984 15046 raft_consensus.cc:2243] T 5c4ab1978a47481c8576d466bacf49d2 P 3e07d64466934da4bb3c02262b1c2e38 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:24:04.958511 15046 raft_consensus.cc:2272] T 5c4ab1978a47481c8576d466bacf49d2 P 3e07d64466934da4bb3c02262b1c2e38 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:24:04.961203 15046 tablet_replica.cc:333] T 28c0b6fc9bd64d60828d705c6a9ab1e4 P 3e07d64466934da4bb3c02262b1c2e38: stopping tablet replica
I20260709 13:24:04.961959 15046 raft_consensus.cc:2243] T 28c0b6fc9bd64d60828d705c6a9ab1e4 P 3e07d64466934da4bb3c02262b1c2e38 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:24:04.962541 15046 raft_consensus.cc:2272] T 28c0b6fc9bd64d60828d705c6a9ab1e4 P 3e07d64466934da4bb3c02262b1c2e38 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:24:04.965240 15046 tablet_replica.cc:333] T b7a891bfc89e4e259f53da6cb16f4989 P 3e07d64466934da4bb3c02262b1c2e38: stopping tablet replica
I20260709 13:24:04.965781 15046 raft_consensus.cc:2243] T b7a891bfc89e4e259f53da6cb16f4989 P 3e07d64466934da4bb3c02262b1c2e38 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:24:04.966215 15046 raft_consensus.cc:2272] T b7a891bfc89e4e259f53da6cb16f4989 P 3e07d64466934da4bb3c02262b1c2e38 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:24:04.968349 15046 tablet_replica.cc:333] T b2f14aed05e24a98ad83535029b795dc P 3e07d64466934da4bb3c02262b1c2e38: stopping tablet replica
I20260709 13:24:04.968830 15046 raft_consensus.cc:2243] T b2f14aed05e24a98ad83535029b795dc P 3e07d64466934da4bb3c02262b1c2e38 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:24:04.969552 15046 raft_consensus.cc:2272] T b2f14aed05e24a98ad83535029b795dc P 3e07d64466934da4bb3c02262b1c2e38 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:24:05.027601 25422 consensus_peers.cc:597] T 1f15ec62698c4a8cbd84f2b3d516d2ea P 9868dce521294d0b816bba7648f4ae5a -> Peer 3e07d64466934da4bb3c02262b1c2e38 (127.14.177.131:41539): Couldn't send request to peer 3e07d64466934da4bb3c02262b1c2e38. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:24:05.043828 15046 tablet_server.cc:196] TabletServer@127.14.177.131:0 shutdown complete.
W20260709 13:24:05.049651 25422 consensus_peers.cc:597] T b9d550d881b04735bbf89751952c09ec P 9868dce521294d0b816bba7648f4ae5a -> Peer 3e07d64466934da4bb3c02262b1c2e38 (127.14.177.131:41539): Couldn't send request to peer 3e07d64466934da4bb3c02262b1c2e38. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.131:41539: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:24:05.109864 25422 consensus_peers.cc:597] T c902bbc7ec0848ba82a04870eab1e7f5 P 9868dce521294d0b816bba7648f4ae5a -> Peer 3e07d64466934da4bb3c02262b1c2e38 (127.14.177.131:41539): Couldn't send request to peer 3e07d64466934da4bb3c02262b1c2e38. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.131:41539: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:24:05.113144 26625 raft_consensus.cc:493] T 5c7e7acbb09f4d49828fa652ca6abaac P 9868dce521294d0b816bba7648f4ae5a [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 43999d813e564dc688fad116090c95ca)
I20260709 13:24:05.113793 26625 raft_consensus.cc:515] T 5c7e7acbb09f4d49828fa652ca6abaac P 9868dce521294d0b816bba7648f4ae5a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } }
I20260709 13:24:05.116099 26625 leader_election.cc:290] T 5c7e7acbb09f4d49828fa652ca6abaac P 9868dce521294d0b816bba7648f4ae5a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3e07d64466934da4bb3c02262b1c2e38 (127.14.177.131:41539), 43999d813e564dc688fad116090c95ca (127.14.177.129:38155)
I20260709 13:24:05.119648 15046 tablet_server.cc:179] TabletServer@127.14.177.132:0 shutting down...
W20260709 13:24:05.123148 25422 leader_election.cc:336] T 5c7e7acbb09f4d49828fa652ca6abaac P 9868dce521294d0b816bba7648f4ae5a [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 3e07d64466934da4bb3c02262b1c2e38 (127.14.177.131:41539): Network error: Client connection negotiation failed: client connection to 127.14.177.131:41539: connect: Connection refused (error 111)
W20260709 13:24:05.125888 25424 leader_election.cc:336] T 5c7e7acbb09f4d49828fa652ca6abaac P 9868dce521294d0b816bba7648f4ae5a [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 43999d813e564dc688fad116090c95ca (127.14.177.129:38155): Network error: Client connection negotiation failed: client connection to 127.14.177.129:38155: connect: Connection refused (error 111)
I20260709 13:24:05.126338 25424 leader_election.cc:304] T 5c7e7acbb09f4d49828fa652ca6abaac P 9868dce521294d0b816bba7648f4ae5a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9868dce521294d0b816bba7648f4ae5a; no voters: 3e07d64466934da4bb3c02262b1c2e38, 43999d813e564dc688fad116090c95ca
I20260709 13:24:05.127326 26625 raft_consensus.cc:2749] T 5c7e7acbb09f4d49828fa652ca6abaac P 9868dce521294d0b816bba7648f4ae5a [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:24:05.135203 25422 consensus_peers.cc:597] T d0a0e4d32ce943fbb773bb840863647b P 9868dce521294d0b816bba7648f4ae5a -> Peer 3e07d64466934da4bb3c02262b1c2e38 (127.14.177.131:41539): Couldn't send request to peer 3e07d64466934da4bb3c02262b1c2e38. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.131:41539: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:24:05.139128 26543 raft_consensus.cc:493] T 6242888a4d4146dcacd87d0fd9a832f4 P 9868dce521294d0b816bba7648f4ae5a [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 43999d813e564dc688fad116090c95ca)
I20260709 13:24:05.139789 26543 raft_consensus.cc:515] T 6242888a4d4146dcacd87d0fd9a832f4 P 9868dce521294d0b816bba7648f4ae5a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } } peers { permanent_uuid: "233e7d29aac04505bf4e65f0a39a0c4c" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 42957 } }
I20260709 13:24:05.142022 26543 leader_election.cc:290] T 6242888a4d4146dcacd87d0fd9a832f4 P 9868dce521294d0b816bba7648f4ae5a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 43999d813e564dc688fad116090c95ca (127.14.177.129:38155), 233e7d29aac04505bf4e65f0a39a0c4c (127.14.177.130:42957)
I20260709 13:24:05.152072 26543 raft_consensus.cc:493] T 0ed5edbe32044d1c83b0590dbb899801 P 9868dce521294d0b816bba7648f4ae5a [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 43999d813e564dc688fad116090c95ca)
I20260709 13:24:05.152596 26543 raft_consensus.cc:515] T 0ed5edbe32044d1c83b0590dbb899801 P 9868dce521294d0b816bba7648f4ae5a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } }
W20260709 13:24:05.153924 25424 leader_election.cc:336] T 6242888a4d4146dcacd87d0fd9a832f4 P 9868dce521294d0b816bba7648f4ae5a [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 43999d813e564dc688fad116090c95ca (127.14.177.129:38155): Network error: Client connection negotiation failed: client connection to 127.14.177.129:38155: connect: Connection refused (error 111)
I20260709 13:24:05.157541 26543 leader_election.cc:290] T 0ed5edbe32044d1c83b0590dbb899801 P 9868dce521294d0b816bba7648f4ae5a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3e07d64466934da4bb3c02262b1c2e38 (127.14.177.131:41539), 43999d813e564dc688fad116090c95ca (127.14.177.129:38155)
W20260709 13:24:05.160041 25422 consensus_peers.cc:597] T 1550ed2d2a764c08adffae679599ba41 P 9868dce521294d0b816bba7648f4ae5a -> Peer 3e07d64466934da4bb3c02262b1c2e38 (127.14.177.131:41539): Couldn't send request to peer 3e07d64466934da4bb3c02262b1c2e38. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.131:41539: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:24:05.160589 25422 consensus_peers.cc:597] T 280d65245f4f4358b54bef93e18aca92 P 9868dce521294d0b816bba7648f4ae5a -> Peer 3e07d64466934da4bb3c02262b1c2e38 (127.14.177.131:41539): Couldn't send request to peer 3e07d64466934da4bb3c02262b1c2e38. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.131:41539: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:24:05.161237 25425 leader_election.cc:336] T 6242888a4d4146dcacd87d0fd9a832f4 P 9868dce521294d0b816bba7648f4ae5a [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 233e7d29aac04505bf4e65f0a39a0c4c (127.14.177.130:42957): Network error: Client connection negotiation failed: client connection to 127.14.177.130:42957: connect: Connection refused (error 111)
I20260709 13:24:05.161720 25425 leader_election.cc:304] T 6242888a4d4146dcacd87d0fd9a832f4 P 9868dce521294d0b816bba7648f4ae5a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9868dce521294d0b816bba7648f4ae5a; no voters: 233e7d29aac04505bf4e65f0a39a0c4c, 43999d813e564dc688fad116090c95ca
I20260709 13:24:05.162537 26625 raft_consensus.cc:2749] T 6242888a4d4146dcacd87d0fd9a832f4 P 9868dce521294d0b816bba7648f4ae5a [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:24:05.163285 25422 leader_election.cc:336] T 0ed5edbe32044d1c83b0590dbb899801 P 9868dce521294d0b816bba7648f4ae5a [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 3e07d64466934da4bb3c02262b1c2e38 (127.14.177.131:41539): Network error: Client connection negotiation failed: client connection to 127.14.177.131:41539: connect: Connection refused (error 111)
W20260709 13:24:05.167665 25424 leader_election.cc:336] T 0ed5edbe32044d1c83b0590dbb899801 P 9868dce521294d0b816bba7648f4ae5a [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 43999d813e564dc688fad116090c95ca (127.14.177.129:38155): Network error: Client connection negotiation failed: client connection to 127.14.177.129:38155: connect: Connection refused (error 111)
I20260709 13:24:05.168207 25424 leader_election.cc:304] T 0ed5edbe32044d1c83b0590dbb899801 P 9868dce521294d0b816bba7648f4ae5a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9868dce521294d0b816bba7648f4ae5a; no voters: 3e07d64466934da4bb3c02262b1c2e38, 43999d813e564dc688fad116090c95ca
I20260709 13:24:05.169015 26543 raft_consensus.cc:2749] T 0ed5edbe32044d1c83b0590dbb899801 P 9868dce521294d0b816bba7648f4ae5a [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:24:05.180250 26632 raft_consensus.cc:493] T f1ac2f4d2fe2424bb470df42199219e2 P 9868dce521294d0b816bba7648f4ae5a [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 43999d813e564dc688fad116090c95ca)
I20260709 13:24:05.180817 26632 raft_consensus.cc:515] T f1ac2f4d2fe2424bb470df42199219e2 P 9868dce521294d0b816bba7648f4ae5a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } }
I20260709 13:24:05.183000 26632 leader_election.cc:290] T f1ac2f4d2fe2424bb470df42199219e2 P 9868dce521294d0b816bba7648f4ae5a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3e07d64466934da4bb3c02262b1c2e38 (127.14.177.131:41539), 43999d813e564dc688fad116090c95ca (127.14.177.129:38155)
I20260709 13:24:05.186090 15046 ts_tablet_manager.cc:1507] Shutting down tablet manager...
W20260709 13:24:05.186332 25422 leader_election.cc:336] T f1ac2f4d2fe2424bb470df42199219e2 P 9868dce521294d0b816bba7648f4ae5a [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 3e07d64466934da4bb3c02262b1c2e38 (127.14.177.131:41539): Network error: Client connection negotiation failed: client connection to 127.14.177.131:41539: connect: Connection refused (error 111)
I20260709 13:24:05.187150 15046 tablet_replica.cc:333] T 11949f56648c47e0a43c3140af99a304 P 9868dce521294d0b816bba7648f4ae5a: stopping tablet replica
I20260709 13:24:05.188150 15046 raft_consensus.cc:2243] T 11949f56648c47e0a43c3140af99a304 P 9868dce521294d0b816bba7648f4ae5a [term 1 LEADER]: Raft consensus shutting down.
W20260709 13:24:05.188948 25424 leader_election.cc:336] T f1ac2f4d2fe2424bb470df42199219e2 P 9868dce521294d0b816bba7648f4ae5a [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 43999d813e564dc688fad116090c95ca (127.14.177.129:38155): Network error: Client connection negotiation failed: client connection to 127.14.177.129:38155: connect: Connection refused (error 111)
I20260709 13:24:05.189090 15046 raft_consensus.cc:2272] T 11949f56648c47e0a43c3140af99a304 P 9868dce521294d0b816bba7648f4ae5a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:24:05.189509 25424 leader_election.cc:304] T f1ac2f4d2fe2424bb470df42199219e2 P 9868dce521294d0b816bba7648f4ae5a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9868dce521294d0b816bba7648f4ae5a; no voters: 3e07d64466934da4bb3c02262b1c2e38, 43999d813e564dc688fad116090c95ca
I20260709 13:24:05.190212 26632 raft_consensus.cc:2749] T f1ac2f4d2fe2424bb470df42199219e2 P 9868dce521294d0b816bba7648f4ae5a [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:24:05.191833 15046 tablet_replica.cc:333] T b9d550d881b04735bbf89751952c09ec P 9868dce521294d0b816bba7648f4ae5a: stopping tablet replica
I20260709 13:24:05.192492 15046 raft_consensus.cc:2243] T b9d550d881b04735bbf89751952c09ec P 9868dce521294d0b816bba7648f4ae5a [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:24:05.193264 15046 raft_consensus.cc:2272] T b9d550d881b04735bbf89751952c09ec P 9868dce521294d0b816bba7648f4ae5a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:24:05.195343 15046 tablet_replica.cc:333] T 8ef061028f38450e83324ac78cdaa14c P 9868dce521294d0b816bba7648f4ae5a: stopping tablet replica
I20260709 13:24:05.195884 15046 raft_consensus.cc:2243] T 8ef061028f38450e83324ac78cdaa14c P 9868dce521294d0b816bba7648f4ae5a [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:24:05.196642 15046 raft_consensus.cc:2272] T 8ef061028f38450e83324ac78cdaa14c P 9868dce521294d0b816bba7648f4ae5a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:24:05.198614 15046 tablet_replica.cc:333] T e311eb90bad44c80a131b72edcd22b06 P 9868dce521294d0b816bba7648f4ae5a: stopping tablet replica
I20260709 13:24:05.199236 15046 raft_consensus.cc:2243] T e311eb90bad44c80a131b72edcd22b06 P 9868dce521294d0b816bba7648f4ae5a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:24:05.199689 15046 raft_consensus.cc:2272] T e311eb90bad44c80a131b72edcd22b06 P 9868dce521294d0b816bba7648f4ae5a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:24:05.201563 15046 tablet_replica.cc:333] T cfad575709c943cba40ccab278350770 P 9868dce521294d0b816bba7648f4ae5a: stopping tablet replica
I20260709 13:24:05.202075 15046 raft_consensus.cc:2243] T cfad575709c943cba40ccab278350770 P 9868dce521294d0b816bba7648f4ae5a [term 5 FOLLOWER]: Raft consensus shutting down.
I20260709 13:24:05.202415 15046 raft_consensus.cc:2272] T cfad575709c943cba40ccab278350770 P 9868dce521294d0b816bba7648f4ae5a [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 13:24:05.204361 15046 tablet_replica.cc:333] T 4d837697cd0f402cacf9db89a5d92652 P 9868dce521294d0b816bba7648f4ae5a: stopping tablet replica
I20260709 13:24:05.204844 15046 raft_consensus.cc:2243] T 4d837697cd0f402cacf9db89a5d92652 P 9868dce521294d0b816bba7648f4ae5a [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:24:05.205205 15046 raft_consensus.cc:2272] T 4d837697cd0f402cacf9db89a5d92652 P 9868dce521294d0b816bba7648f4ae5a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:24:05.207033 15046 tablet_replica.cc:333] T 0ed5edbe32044d1c83b0590dbb899801 P 9868dce521294d0b816bba7648f4ae5a: stopping tablet replica
I20260709 13:24:05.207620 15046 raft_consensus.cc:2243] T 0ed5edbe32044d1c83b0590dbb899801 P 9868dce521294d0b816bba7648f4ae5a [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:24:05.208034 15046 raft_consensus.cc:2272] T 0ed5edbe32044d1c83b0590dbb899801 P 9868dce521294d0b816bba7648f4ae5a [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:24:05.209878 15046 tablet_replica.cc:333] T 9ab90e3e08fb4528b2a47d995e9e26d3 P 9868dce521294d0b816bba7648f4ae5a: stopping tablet replica
I20260709 13:24:05.210443 15046 raft_consensus.cc:2243] T 9ab90e3e08fb4528b2a47d995e9e26d3 P 9868dce521294d0b816bba7648f4ae5a [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:24:05.210868 15046 raft_consensus.cc:2272] T 9ab90e3e08fb4528b2a47d995e9e26d3 P 9868dce521294d0b816bba7648f4ae5a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:24:05.212813 15046 tablet_replica.cc:333] T ce9c60e3922c4219b6e4453fe0259b6a P 9868dce521294d0b816bba7648f4ae5a: stopping tablet replica
I20260709 13:24:05.213378 15046 raft_consensus.cc:2243] T ce9c60e3922c4219b6e4453fe0259b6a P 9868dce521294d0b816bba7648f4ae5a [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:24:05.213805 15046 raft_consensus.cc:2272] T ce9c60e3922c4219b6e4453fe0259b6a P 9868dce521294d0b816bba7648f4ae5a [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:24:05.215904 15046 tablet_replica.cc:333] T 6242888a4d4146dcacd87d0fd9a832f4 P 9868dce521294d0b816bba7648f4ae5a: stopping tablet replica
I20260709 13:24:05.216486 15046 raft_consensus.cc:2243] T 6242888a4d4146dcacd87d0fd9a832f4 P 9868dce521294d0b816bba7648f4ae5a [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:24:05.217160 15046 raft_consensus.cc:2272] T 6242888a4d4146dcacd87d0fd9a832f4 P 9868dce521294d0b816bba7648f4ae5a [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:24:05.219867 15046 tablet_replica.cc:333] T 434a18a5109e4ea78349ded3968b2709 P 9868dce521294d0b816bba7648f4ae5a: stopping tablet replica
I20260709 13:24:05.220521 15046 raft_consensus.cc:2243] T 434a18a5109e4ea78349ded3968b2709 P 9868dce521294d0b816bba7648f4ae5a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:24:05.221050 15046 raft_consensus.cc:2272] T 434a18a5109e4ea78349ded3968b2709 P 9868dce521294d0b816bba7648f4ae5a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:24:05.223271 15046 tablet_replica.cc:333] T a5df750b84e0487fa530161079deb6c2 P 9868dce521294d0b816bba7648f4ae5a: stopping tablet replica
I20260709 13:24:05.223781 15046 raft_consensus.cc:2243] T a5df750b84e0487fa530161079deb6c2 P 9868dce521294d0b816bba7648f4ae5a [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:24:05.224510 15046 raft_consensus.cc:2272] T a5df750b84e0487fa530161079deb6c2 P 9868dce521294d0b816bba7648f4ae5a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:24:05.226673 15046 tablet_replica.cc:333] T 89a26dd3ac414f6081b98fa32c2f346f P 9868dce521294d0b816bba7648f4ae5a: stopping tablet replica
I20260709 13:24:05.226711 26632 raft_consensus.cc:493] T b7a891bfc89e4e259f53da6cb16f4989 P 9868dce521294d0b816bba7648f4ae5a [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 43999d813e564dc688fad116090c95ca)
I20260709 13:24:05.227299 15046 raft_consensus.cc:2243] T 89a26dd3ac414f6081b98fa32c2f346f P 9868dce521294d0b816bba7648f4ae5a [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:24:05.227478 26632 raft_consensus.cc:515] T b7a891bfc89e4e259f53da6cb16f4989 P 9868dce521294d0b816bba7648f4ae5a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e07d64466934da4bb3c02262b1c2e38" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 41539 } } peers { permanent_uuid: "9868dce521294d0b816bba7648f4ae5a" member_type: VOTER last_known_addr { host: "127.14.177.132" port: 33683 } } peers { permanent_uuid: "43999d813e564dc688fad116090c95ca" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38155 } }
I20260709 13:24:05.227927 15046 raft_consensus.cc:2272] T 89a26dd3ac414f6081b98fa32c2f346f P 9868dce521294d0b816bba7648f4ae5a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:24:05.229514 26632 leader_election.cc:290] T b7a891bfc89e4e259f53da6cb16f4989 P 9868dce521294d0b816bba7648f4ae5a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3e07d64466934da4bb3c02262b1c2e38 (127.14.177.131:41539), 43999d813e564dc688fad116090c95ca (127.14.177.129:38155)
I20260709 13:24:05.230610 15046 tablet_replica.cc:333] T c902bbc7ec0848ba82a04870eab1e7f5 P 9868dce521294d0b816bba7648f4ae5a: stopping tablet replica
I20260709 13:24:05.231446 15046 raft_consensus.cc:2243] T c902bbc7ec0848ba82a04870eab1e7f5 P 9868dce521294d0b816bba7648f4ae5a [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:24:05.232475 15046 raft_consensus.cc:2272] T c902bbc7ec0848ba82a04870eab1e7f5 P 9868dce521294d0b816bba7648f4ae5a [term 3 FOLLOWER]: Raft consensus is shut down!
W20260709 13:24:05.233898 25422 leader_election.cc:336] T b7a891bfc89e4e259f53da6cb16f4989 P 9868dce521294d0b816bba7648f4ae5a [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 3e07d64466934da4bb3c02262b1c2e38 (127.14.177.131:41539): Network error: Client connection negotiation failed: client connection to 127.14.177.131:41539: connect: Connection refused (error 111)
I20260709 13:24:05.235405 15046 tablet_replica.cc:333] T e9e342ecda4b46619ce12452b2cf7154 P 9868dce521294d0b816bba7648f4ae5a: stopping tablet replica
W20260709 13:24:05.235453 25424 leader_election.cc:336] T b7a891bfc89e4e259f53da6cb16f4989 P 9868dce521294d0b816bba7648f4ae5a [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 43999d813e564dc688fad116090c95ca (127.14.177.129:38155): Network error: Client connection negotiation failed: client connection to 127.14.177.129:38155: connect: Connection refused (error 111)
I20260709 13:24:05.236035 25424 leader_election.cc:304] T b7a891bfc89e4e259f53da6cb16f4989 P 9868dce521294d0b816bba7648f4ae5a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9868dce521294d0b816bba7648f4ae5a; no voters: 3e07d64466934da4bb3c02262b1c2e38, 43999d813e564dc688fad116090c95ca
I20260709 13:24:05.236342 15046 raft_consensus.cc:2243] T e9e342ecda4b46619ce12452b2cf7154 P 9868dce521294d0b816bba7648f4ae5a [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:24:05.236701 26543 raft_consensus.cc:2749] T b7a891bfc89e4e259f53da6cb16f4989 P 9868dce521294d0b816bba7648f4ae5a [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:24:05.236999 15046 raft_consensus.cc:2272] T e9e342ecda4b46619ce12452b2cf7154 P 9868dce521294d0b816bba7648f4ae5a [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:24:05.239144 15046 tablet_replica.cc:333] T 9ffa42c3b2ff48bc87310e9ab5d218ce P 9868dce521294d0b816bba7648f4ae5a: stopping tablet replica
I20260709 13:24:05.239763 15046 raft_consensus.cc:2243] T 9ffa42c3b2ff48bc87310e9ab5d218ce P 9868dce521294d0b816bba7648f4ae5a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:24:05.240193 15046 raft_consensus.cc:2272] T 9ffa42c3b2ff48bc87310e9ab5d218ce P 9868dce521294d0b816bba7648f4ae5a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:24:05.242019 15046 tablet_replica.cc:333] T ad32097cd4dc48b9a69595f1651abd07 P 9868dce521294d0b816bba7648f4ae5a: stopping tablet replica
I20260709 13:24:05.242605 15046 raft_consensus.cc:2243] T ad32097cd4dc48b9a69595f1651abd07 P 9868dce521294d0b816bba7648f4ae5a [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:24:05.242969 15046 raft_consensus.cc:2272] T ad32097cd4dc48b9a69595f1651abd07 P 9868dce521294d0b816bba7648f4ae5a [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:24:05.245581 15046 tablet_replica.cc:333] T 821a5866429f4173820fb64336abee02 P 9868dce521294d0b816bba7648f4ae5a: stopping tablet replica
I20260709 13:24:05.246380 15046 raft_consensus.cc:2243] T 821a5866429f4173820fb64336abee02 P 9868dce521294d0b816bba7648f4ae5a [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:24:05.246937 15046 raft_consensus.cc:2272] T 821a5866429f4173820fb64336abee02 P 9868dce521294d0b816bba7648f4ae5a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:24:05.249182 15046 tablet_replica.cc:333] T bbab48ba177049dcba6de106fd889cfb P 9868dce521294d0b816bba7648f4ae5a: stopping tablet replica
I20260709 13:24:05.249732 15046 raft_consensus.cc:2243] T bbab48ba177049dcba6de106fd889cfb P 9868dce521294d0b816bba7648f4ae5a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:24:05.250164 15046 raft_consensus.cc:2272] T bbab48ba177049dcba6de106fd889cfb P 9868dce521294d0b816bba7648f4ae5a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:24:05.252158 15046 tablet_replica.cc:333] T ba8a000f5abe4158965449995af84b18 P 9868dce521294d0b816bba7648f4ae5a: stopping tablet replica
I20260709 13:24:05.252731 15046 raft_consensus.cc:2243] T ba8a000f5abe4158965449995af84b18 P 9868dce521294d0b816bba7648f4ae5a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:24:05.253126 15046 raft_consensus.cc:2272] T ba8a000f5abe4158965449995af84b18 P 9868dce521294d0b816bba7648f4ae5a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:24:05.254928 15046 tablet_replica.cc:333] T f1ac2f4d2fe2424bb470df42199219e2 P 9868dce521294d0b816bba7648f4ae5a: stopping tablet replica
I20260709 13:24:05.255501 15046 raft_consensus.cc:2243] T f1ac2f4d2fe2424bb470df42199219e2 P 9868dce521294d0b816bba7648f4ae5a [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:24:05.255900 15046 raft_consensus.cc:2272] T f1ac2f4d2fe2424bb470df42199219e2 P 9868dce521294d0b816bba7648f4ae5a [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:24:05.257750 15046 tablet_replica.cc:333] T 9e5d70664720424d951f269d9d7f4346 P 9868dce521294d0b816bba7648f4ae5a: stopping tablet replica
I20260709 13:24:05.258252 15046 raft_consensus.cc:2243] T 9e5d70664720424d951f269d9d7f4346 P 9868dce521294d0b816bba7648f4ae5a [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:24:05.258653 15046 raft_consensus.cc:2272] T 9e5d70664720424d951f269d9d7f4346 P 9868dce521294d0b816bba7648f4ae5a [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:24:05.260488 15046 tablet_replica.cc:333] T 5c4ab1978a47481c8576d466bacf49d2 P 9868dce521294d0b816bba7648f4ae5a: stopping tablet replica
I20260709 13:24:05.260999 15046 raft_consensus.cc:2243] T 5c4ab1978a47481c8576d466bacf49d2 P 9868dce521294d0b816bba7648f4ae5a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:24:05.261405 15046 raft_consensus.cc:2272] T 5c4ab1978a47481c8576d466bacf49d2 P 9868dce521294d0b816bba7648f4ae5a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:24:05.263347 15046 tablet_replica.cc:333] T 51a11db644124b63bdf5f457c967473d P 9868dce521294d0b816bba7648f4ae5a: stopping tablet replica
I20260709 13:24:05.263885 15046 raft_consensus.cc:2243] T 51a11db644124b63bdf5f457c967473d P 9868dce521294d0b816bba7648f4ae5a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:24:05.264307 15046 raft_consensus.cc:2272] T 51a11db644124b63bdf5f457c967473d P 9868dce521294d0b816bba7648f4ae5a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:24:05.266157 15046 tablet_replica.cc:333] T 91ac744179de48e5bb924b6dfd9f9c9b P 9868dce521294d0b816bba7648f4ae5a: stopping tablet replica
I20260709 13:24:05.266693 15046 raft_consensus.cc:2243] T 91ac744179de48e5bb924b6dfd9f9c9b P 9868dce521294d0b816bba7648f4ae5a [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:24:05.267114 15046 raft_consensus.cc:2272] T 91ac744179de48e5bb924b6dfd9f9c9b P 9868dce521294d0b816bba7648f4ae5a [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:24:05.269006 15046 tablet_replica.cc:333] T 1f15ec62698c4a8cbd84f2b3d516d2ea P 9868dce521294d0b816bba7648f4ae5a: stopping tablet replica
I20260709 13:24:05.269542 15046 raft_consensus.cc:2243] T 1f15ec62698c4a8cbd84f2b3d516d2ea P 9868dce521294d0b816bba7648f4ae5a [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:24:05.270309 15046 raft_consensus.cc:2272] T 1f15ec62698c4a8cbd84f2b3d516d2ea P 9868dce521294d0b816bba7648f4ae5a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:24:05.272153 15046 tablet_replica.cc:333] T 280d65245f4f4358b54bef93e18aca92 P 9868dce521294d0b816bba7648f4ae5a: stopping tablet replica
I20260709 13:24:05.272747 15046 raft_consensus.cc:2243] T 280d65245f4f4358b54bef93e18aca92 P 9868dce521294d0b816bba7648f4ae5a [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:24:05.273499 15046 raft_consensus.cc:2272] T 280d65245f4f4358b54bef93e18aca92 P 9868dce521294d0b816bba7648f4ae5a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:24:05.275379 15046 tablet_replica.cc:333] T b85a238bbc3b4621881603a534c7bd9b P 9868dce521294d0b816bba7648f4ae5a: stopping tablet replica
I20260709 13:24:05.275909 15046 raft_consensus.cc:2243] T b85a238bbc3b4621881603a534c7bd9b P 9868dce521294d0b816bba7648f4ae5a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:24:05.276322 15046 raft_consensus.cc:2272] T b85a238bbc3b4621881603a534c7bd9b P 9868dce521294d0b816bba7648f4ae5a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:24:05.278483 15046 tablet_replica.cc:333] T aad1f7937f854cc1a726223269da5d5a P 9868dce521294d0b816bba7648f4ae5a: stopping tablet replica
I20260709 13:24:05.279250 15046 raft_consensus.cc:2243] T aad1f7937f854cc1a726223269da5d5a P 9868dce521294d0b816bba7648f4ae5a [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:24:05.280196 15046 raft_consensus.cc:2272] T aad1f7937f854cc1a726223269da5d5a P 9868dce521294d0b816bba7648f4ae5a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:24:05.282577 15046 tablet_replica.cc:333] T d053b9ae38de4173a0aded4da494bd98 P 9868dce521294d0b816bba7648f4ae5a: stopping tablet replica
I20260709 13:24:05.283097 15046 raft_consensus.cc:2243] T d053b9ae38de4173a0aded4da494bd98 P 9868dce521294d0b816bba7648f4ae5a [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:24:05.283567 15046 raft_consensus.cc:2272] T d053b9ae38de4173a0aded4da494bd98 P 9868dce521294d0b816bba7648f4ae5a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:24:05.285305 15046 tablet_replica.cc:333] T 1550ed2d2a764c08adffae679599ba41 P 9868dce521294d0b816bba7648f4ae5a: stopping tablet replica
I20260709 13:24:05.285801 15046 raft_consensus.cc:2243] T 1550ed2d2a764c08adffae679599ba41 P 9868dce521294d0b816bba7648f4ae5a [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:24:05.286554 15046 raft_consensus.cc:2272] T 1550ed2d2a764c08adffae679599ba41 P 9868dce521294d0b816bba7648f4ae5a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:24:05.288461 15046 tablet_replica.cc:333] T 0c4f6b7401db47d59a4f34ed0e8b5350 P 9868dce521294d0b816bba7648f4ae5a: stopping tablet replica
I20260709 13:24:05.288968 15046 raft_consensus.cc:2243] T 0c4f6b7401db47d59a4f34ed0e8b5350 P 9868dce521294d0b816bba7648f4ae5a [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:24:05.289754 15046 raft_consensus.cc:2272] T 0c4f6b7401db47d59a4f34ed0e8b5350 P 9868dce521294d0b816bba7648f4ae5a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:24:05.291549 15046 tablet_replica.cc:333] T a820954e07a94582ac0613342fcffde5 P 9868dce521294d0b816bba7648f4ae5a: stopping tablet replica
I20260709 13:24:05.292035 15046 raft_consensus.cc:2243] T a820954e07a94582ac0613342fcffde5 P 9868dce521294d0b816bba7648f4ae5a [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:24:05.292754 15046 raft_consensus.cc:2272] T a820954e07a94582ac0613342fcffde5 P 9868dce521294d0b816bba7648f4ae5a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:24:05.294672 15046 tablet_replica.cc:333] T 0e71b6f5c69b4b35a8ac51c22c7d5f0d P 9868dce521294d0b816bba7648f4ae5a: stopping tablet replica
I20260709 13:24:05.295251 15046 raft_consensus.cc:2243] T 0e71b6f5c69b4b35a8ac51c22c7d5f0d P 9868dce521294d0b816bba7648f4ae5a [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:24:05.295663 15046 raft_consensus.cc:2272] T 0e71b6f5c69b4b35a8ac51c22c7d5f0d P 9868dce521294d0b816bba7648f4ae5a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:24:05.297580 15046 tablet_replica.cc:333] T 441d780c3945463f92f31170f31e37a8 P 9868dce521294d0b816bba7648f4ae5a: stopping tablet replica
I20260709 13:24:05.298090 15046 raft_consensus.cc:2243] T 441d780c3945463f92f31170f31e37a8 P 9868dce521294d0b816bba7648f4ae5a [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:24:05.298497 15046 raft_consensus.cc:2272] T 441d780c3945463f92f31170f31e37a8 P 9868dce521294d0b816bba7648f4ae5a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:24:05.300418 15046 tablet_replica.cc:333] T d0a0e4d32ce943fbb773bb840863647b P 9868dce521294d0b816bba7648f4ae5a: stopping tablet replica
I20260709 13:24:05.300943 15046 raft_consensus.cc:2243] T d0a0e4d32ce943fbb773bb840863647b P 9868dce521294d0b816bba7648f4ae5a [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:24:05.301682 15046 raft_consensus.cc:2272] T d0a0e4d32ce943fbb773bb840863647b P 9868dce521294d0b816bba7648f4ae5a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:24:05.303994 15046 tablet_replica.cc:333] T 579da2d6aabb4ae9a2b1802e1358068b P 9868dce521294d0b816bba7648f4ae5a: stopping tablet replica
I20260709 13:24:05.304499 15046 raft_consensus.cc:2243] T 579da2d6aabb4ae9a2b1802e1358068b P 9868dce521294d0b816bba7648f4ae5a [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:24:05.305214 15046 raft_consensus.cc:2272] T 579da2d6aabb4ae9a2b1802e1358068b P 9868dce521294d0b816bba7648f4ae5a [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:24:05.306980 15046 tablet_replica.cc:333] T 467a6b42ca6548269c12dddde696f7af P 9868dce521294d0b816bba7648f4ae5a: stopping tablet replica
I20260709 13:24:05.307502 15046 raft_consensus.cc:2243] T 467a6b42ca6548269c12dddde696f7af P 9868dce521294d0b816bba7648f4ae5a [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:24:05.307979 15046 raft_consensus.cc:2272] T 467a6b42ca6548269c12dddde696f7af P 9868dce521294d0b816bba7648f4ae5a [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:24:05.309785 15046 tablet_replica.cc:333] T fa0da2c845dc4059968332f4db2e0aa6 P 9868dce521294d0b816bba7648f4ae5a: stopping tablet replica
I20260709 13:24:05.310287 15046 raft_consensus.cc:2243] T fa0da2c845dc4059968332f4db2e0aa6 P 9868dce521294d0b816bba7648f4ae5a [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:24:05.310974 15046 raft_consensus.cc:2272] T fa0da2c845dc4059968332f4db2e0aa6 P 9868dce521294d0b816bba7648f4ae5a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:24:05.312917 15046 tablet_replica.cc:333] T 5c7e7acbb09f4d49828fa652ca6abaac P 9868dce521294d0b816bba7648f4ae5a: stopping tablet replica
I20260709 13:24:05.313401 15046 raft_consensus.cc:2243] T 5c7e7acbb09f4d49828fa652ca6abaac P 9868dce521294d0b816bba7648f4ae5a [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:24:05.313786 15046 raft_consensus.cc:2272] T 5c7e7acbb09f4d49828fa652ca6abaac P 9868dce521294d0b816bba7648f4ae5a [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:24:05.315550 15046 tablet_replica.cc:333] T 55e8ed3aee404ce698d0626a87bd03aa P 9868dce521294d0b816bba7648f4ae5a: stopping tablet replica
I20260709 13:24:05.316017 15046 raft_consensus.cc:2243] T 55e8ed3aee404ce698d0626a87bd03aa P 9868dce521294d0b816bba7648f4ae5a [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:24:05.316773 15046 raft_consensus.cc:2272] T 55e8ed3aee404ce698d0626a87bd03aa P 9868dce521294d0b816bba7648f4ae5a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:24:05.318676 15046 tablet_replica.cc:333] T 3a3317f4c3514df1b2bb8e44a86be92f P 9868dce521294d0b816bba7648f4ae5a: stopping tablet replica
I20260709 13:24:05.319154 15046 raft_consensus.cc:2243] T 3a3317f4c3514df1b2bb8e44a86be92f P 9868dce521294d0b816bba7648f4ae5a [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:24:05.319577 15046 raft_consensus.cc:2272] T 3a3317f4c3514df1b2bb8e44a86be92f P 9868dce521294d0b816bba7648f4ae5a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:24:05.321396 15046 tablet_replica.cc:333] T 3fe5a944a342411ea923eb39071ac770 P 9868dce521294d0b816bba7648f4ae5a: stopping tablet replica
I20260709 13:24:05.321859 15046 raft_consensus.cc:2243] T 3fe5a944a342411ea923eb39071ac770 P 9868dce521294d0b816bba7648f4ae5a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:24:05.322252 15046 raft_consensus.cc:2272] T 3fe5a944a342411ea923eb39071ac770 P 9868dce521294d0b816bba7648f4ae5a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:24:05.324083 15046 tablet_replica.cc:333] T b7a891bfc89e4e259f53da6cb16f4989 P 9868dce521294d0b816bba7648f4ae5a: stopping tablet replica
I20260709 13:24:05.324551 15046 raft_consensus.cc:2243] T b7a891bfc89e4e259f53da6cb16f4989 P 9868dce521294d0b816bba7648f4ae5a [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:24:05.324952 15046 raft_consensus.cc:2272] T b7a891bfc89e4e259f53da6cb16f4989 P 9868dce521294d0b816bba7648f4ae5a [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:24:05.407696 15046 tablet_server.cc:196] TabletServer@127.14.177.132:0 shutdown complete.
I20260709 13:24:05.476781 15046 master.cc:562] Master@127.14.177.190:45841 shutting down...
I20260709 13:24:05.507297 15046 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 4b6484c4860b4e75a6322b756be0a819 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:24:05.507930 15046 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 4b6484c4860b4e75a6322b756be0a819 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:24:05.508314 15046 tablet_replica.cc:333] T 00000000000000000000000000000000 P 4b6484c4860b4e75a6322b756be0a819: stopping tablet replica
I20260709 13:24:05.566092 15046 master.cc:584] Master@127.14.177.190:45841 shutdown complete.
[       OK ] LeaderRebalancerTest.RestartTserver (36818 ms)
[ RUN      ] LeaderRebalancerTest.TestMaintenanceMode
I20260709 13:24:05.656419 15046 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.14.177.190:39149
I20260709 13:24:05.657526 15046 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:24:05.663921 26633 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:24:05.664314 26634 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:24:05.665035 26636 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:24:05.665263 15046 server_base.cc:1061] running on GCE node
I20260709 13:24:05.666754 15046 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:24:05.667016 15046 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:24:05.667222 15046 hybrid_clock.cc:648] HybridClock initialized: now 1783603445667202 us; error 0 us; skew 500 ppm
I20260709 13:24:05.667857 15046 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:24:05.670629 15046 webserver.cc:533] Webserver started at http://127.14.177.190:33183/ using document root <none> and password file <none>
I20260709 13:24:05.671306 15046 fs_manager.cc:362] Metadata directory not provided
I20260709 13:24:05.671562 15046 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:24:05.671883 15046 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:24:05.673195 15046 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskdH_Ma7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783603093801613-15046-0/minicluster-data/master-0-root/instance:
uuid: "3bc15f9174194ef6acd0a8614cced0ea"
format_stamp: "Formatted at 2026-07-09 13:24:05 on dist-test-slave-v5z2"
I20260709 13:24:05.678504 15046 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.006s	sys 0.000s
I20260709 13:24:05.682118 26641 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:24:05.683022 15046 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.000s
I20260709 13:24:05.683375 15046 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskdH_Ma7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783603093801613-15046-0/minicluster-data/master-0-root
uuid: "3bc15f9174194ef6acd0a8614cced0ea"
format_stamp: "Formatted at 2026-07-09 13:24:05 on dist-test-slave-v5z2"
I20260709 13:24:05.683638 15046 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdH_Ma7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783603093801613-15046-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskdH_Ma7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783603093801613-15046-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskdH_Ma7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783603093801613-15046-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:24:05.705509 15046 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:24:05.706835 15046 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:24:05.746914 15046 rpc_server.cc:307] RPC server started. Bound to: 127.14.177.190:39149
I20260709 13:24:05.747144 26692 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.14.177.190:39149 every 8 connection(s)
I20260709 13:24:05.751363 26693 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:24:05.763038 26693 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3bc15f9174194ef6acd0a8614cced0ea: Bootstrap starting.
I20260709 13:24:05.768498 26693 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3bc15f9174194ef6acd0a8614cced0ea: Neither blocks nor log segments found. Creating new log.
I20260709 13:24:05.773203 26693 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3bc15f9174194ef6acd0a8614cced0ea: No bootstrap required, opened a new log
I20260709 13:24:05.775849 26693 raft_consensus.cc:359] T 00000000000000000000000000000000 P 3bc15f9174194ef6acd0a8614cced0ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3bc15f9174194ef6acd0a8614cced0ea" member_type: VOTER }
I20260709 13:24:05.776306 26693 raft_consensus.cc:385] T 00000000000000000000000000000000 P 3bc15f9174194ef6acd0a8614cced0ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:24:05.776583 26693 raft_consensus.cc:740] T 00000000000000000000000000000000 P 3bc15f9174194ef6acd0a8614cced0ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3bc15f9174194ef6acd0a8614cced0ea, State: Initialized, Role: FOLLOWER
I20260709 13:24:05.777305 26693 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3bc15f9174194ef6acd0a8614cced0ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3bc15f9174194ef6acd0a8614cced0ea" member_type: VOTER }
I20260709 13:24:05.778105 26693 raft_consensus.cc:399] T 00000000000000000000000000000000 P 3bc15f9174194ef6acd0a8614cced0ea [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 13:24:05.778410 26693 raft_consensus.cc:493] T 00000000000000000000000000000000 P 3bc15f9174194ef6acd0a8614cced0ea [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 13:24:05.778765 26693 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 3bc15f9174194ef6acd0a8614cced0ea [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:24:05.783936 26693 raft_consensus.cc:515] T 00000000000000000000000000000000 P 3bc15f9174194ef6acd0a8614cced0ea [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3bc15f9174194ef6acd0a8614cced0ea" member_type: VOTER }
I20260709 13:24:05.784533 26693 leader_election.cc:304] T 00000000000000000000000000000000 P 3bc15f9174194ef6acd0a8614cced0ea [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3bc15f9174194ef6acd0a8614cced0ea; no voters: 
I20260709 13:24:05.786023 26693 leader_election.cc:290] T 00000000000000000000000000000000 P 3bc15f9174194ef6acd0a8614cced0ea [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 13:24:05.786451 26696 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 3bc15f9174194ef6acd0a8614cced0ea [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:24:05.788015 26696 raft_consensus.cc:697] T 00000000000000000000000000000000 P 3bc15f9174194ef6acd0a8614cced0ea [term 1 LEADER]: Becoming Leader. State: Replica: 3bc15f9174194ef6acd0a8614cced0ea, State: Running, Role: LEADER
I20260709 13:24:05.788667 26696 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3bc15f9174194ef6acd0a8614cced0ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3bc15f9174194ef6acd0a8614cced0ea" member_type: VOTER }
I20260709 13:24:05.789357 26693 sys_catalog.cc:565] T 00000000000000000000000000000000 P 3bc15f9174194ef6acd0a8614cced0ea [sys.catalog]: configured and running, proceeding with master startup.
I20260709 13:24:05.793565 26697 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3bc15f9174194ef6acd0a8614cced0ea [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3bc15f9174194ef6acd0a8614cced0ea" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3bc15f9174194ef6acd0a8614cced0ea" member_type: VOTER } }
I20260709 13:24:05.794659 26698 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3bc15f9174194ef6acd0a8614cced0ea [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3bc15f9174194ef6acd0a8614cced0ea. Latest consensus state: current_term: 1 leader_uuid: "3bc15f9174194ef6acd0a8614cced0ea" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3bc15f9174194ef6acd0a8614cced0ea" member_type: VOTER } }
I20260709 13:24:05.795274 26698 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3bc15f9174194ef6acd0a8614cced0ea [sys.catalog]: This master's current role is: LEADER
I20260709 13:24:05.795274 26697 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3bc15f9174194ef6acd0a8614cced0ea [sys.catalog]: This master's current role is: LEADER
I20260709 13:24:05.797278 26703 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 13:24:05.802069 26703 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 13:24:05.806852 15046 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 13:24:05.811743 26703 catalog_manager.cc:1383] Generated new cluster ID: fa42ac26cd4e49eab12fd23fdf970b69
I20260709 13:24:05.812021 26703 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 13:24:05.830405 26703 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 13:24:05.832051 26703 catalog_manager.cc:1540] Loading token signing keys...
I20260709 13:24:05.849277 26703 catalog_manager.cc:6070] T 00000000000000000000000000000000 P 3bc15f9174194ef6acd0a8614cced0ea: Generated new TSK 0
I20260709 13:24:05.850023 26703 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 13:24:05.876417 15046 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:24:05.882957 26714 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:24:05.883534 26715 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:24:05.887041 15046 server_base.cc:1061] running on GCE node
W20260709 13:24:05.886930 26717 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:24:05.888605 15046 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:24:05.888818 15046 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:24:05.888964 15046 hybrid_clock.cc:648] HybridClock initialized: now 1783603445888942 us; error 0 us; skew 500 ppm
I20260709 13:24:05.889519 15046 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:24:05.892182 15046 webserver.cc:533] Webserver started at http://127.14.177.129:40653/ using document root <none> and password file <none>
I20260709 13:24:05.892619 15046 fs_manager.cc:362] Metadata directory not provided
I20260709 13:24:05.892779 15046 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:24:05.892992 15046 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:24:05.894024 15046 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskdH_Ma7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783603093801613-15046-0/minicluster-data/ts-0-root/instance:
uuid: "ff90de856dd542eab446899d181dcad0"
format_stamp: "Formatted at 2026-07-09 13:24:05 on dist-test-slave-v5z2"
I20260709 13:24:05.898903 15046 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.006s	sys 0.000s
I20260709 13:24:05.902525 26722 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:24:05.903486 15046 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.000s
I20260709 13:24:05.903815 15046 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskdH_Ma7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783603093801613-15046-0/minicluster-data/ts-0-root
uuid: "ff90de856dd542eab446899d181dcad0"
format_stamp: "Formatted at 2026-07-09 13:24:05 on dist-test-slave-v5z2"
I20260709 13:24:05.904094 15046 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdH_Ma7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783603093801613-15046-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskdH_Ma7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783603093801613-15046-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskdH_Ma7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783603093801613-15046-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:24:05.930564 15046 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:24:05.931926 15046 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:24:05.933430 15046 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:24:05.935820 15046 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:24:05.936055 15046 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:24:05.936308 15046 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:24:05.936496 15046 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:24:05.977229 15046 rpc_server.cc:307] RPC server started. Bound to: 127.14.177.129:36311
I20260709 13:24:05.977351 26784 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.14.177.129:36311 every 8 connection(s)
I20260709 13:24:05.982358 15046 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:24:05.989979 26789 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:24:05.990418 26790 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:24:05.993650 15046 server_base.cc:1061] running on GCE node
W20260709 13:24:05.994632 26793 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:24:05.995704 26785 heartbeater.cc:344] Connected to a master server at 127.14.177.190:39149
I20260709 13:24:05.995766 15046 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:24:05.996197 15046 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:24:05.996215 26785 heartbeater.cc:461] Registering TS with master...
I20260709 13:24:05.996400 15046 hybrid_clock.cc:648] HybridClock initialized: now 1783603445996385 us; error 0 us; skew 500 ppm
I20260709 13:24:05.997083 15046 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:24:05.997143 26785 heartbeater.cc:507] Master 127.14.177.190:39149 requested a full tablet report, sending...
I20260709 13:24:05.999701 26658 ts_manager.cc:194] Registered new tserver with Master: ff90de856dd542eab446899d181dcad0 (127.14.177.129:36311)
I20260709 13:24:05.999866 15046 webserver.cc:533] Webserver started at http://127.14.177.130:45141/ using document root <none> and password file <none>
I20260709 13:24:06.000530 15046 fs_manager.cc:362] Metadata directory not provided
I20260709 13:24:06.000752 15046 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:24:06.001021 15046 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:24:06.001807 26658 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:38050
I20260709 13:24:06.002158 15046 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskdH_Ma7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783603093801613-15046-0/minicluster-data/ts-1-root/instance:
uuid: "326c74b0a8d84cdf86229f67ce3487f2"
format_stamp: "Formatted at 2026-07-09 13:24:05 on dist-test-slave-v5z2"
I20260709 13:24:06.006871 15046 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.003s	sys 0.004s
I20260709 13:24:06.010419 26797 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:24:06.011340 15046 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.000s	sys 0.003s
I20260709 13:24:06.011652 15046 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskdH_Ma7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783603093801613-15046-0/minicluster-data/ts-1-root
uuid: "326c74b0a8d84cdf86229f67ce3487f2"
format_stamp: "Formatted at 2026-07-09 13:24:05 on dist-test-slave-v5z2"
I20260709 13:24:06.011950 15046 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdH_Ma7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783603093801613-15046-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskdH_Ma7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783603093801613-15046-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskdH_Ma7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783603093801613-15046-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:24:06.021500 15046 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:24:06.022742 15046 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:24:06.024256 15046 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:24:06.026369 15046 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:24:06.026585 15046 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:24:06.026831 15046 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:24:06.026998 15046 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:24:06.068655 15046 rpc_server.cc:307] RPC server started. Bound to: 127.14.177.130:35517
I20260709 13:24:06.068800 26859 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.14.177.130:35517 every 8 connection(s)
I20260709 13:24:06.073475 15046 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:24:06.081174 26863 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:24:06.082458 26864 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:24:06.084956 26866 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:24:06.085311 15046 server_base.cc:1061] running on GCE node
I20260709 13:24:06.086438 15046 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:24:06.086686 15046 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:24:06.086889 15046 hybrid_clock.cc:648] HybridClock initialized: now 1783603446086873 us; error 0 us; skew 500 ppm
I20260709 13:24:06.087478 15046 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:24:06.087882 26860 heartbeater.cc:344] Connected to a master server at 127.14.177.190:39149
I20260709 13:24:06.088332 26860 heartbeater.cc:461] Registering TS with master...
I20260709 13:24:06.089268 26860 heartbeater.cc:507] Master 127.14.177.190:39149 requested a full tablet report, sending...
I20260709 13:24:06.090153 15046 webserver.cc:533] Webserver started at http://127.14.177.131:42043/ using document root <none> and password file <none>
I20260709 13:24:06.090786 15046 fs_manager.cc:362] Metadata directory not provided
I20260709 13:24:06.091017 15046 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:24:06.091351 15046 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:24:06.091542 26658 ts_manager.cc:194] Registered new tserver with Master: 326c74b0a8d84cdf86229f67ce3487f2 (127.14.177.130:35517)
I20260709 13:24:06.092682 15046 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskdH_Ma7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783603093801613-15046-0/minicluster-data/ts-2-root/instance:
uuid: "239ae18f3e44498fa52e20e9d6eec6b5"
format_stamp: "Formatted at 2026-07-09 13:24:06 on dist-test-slave-v5z2"
I20260709 13:24:06.092867 26658 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:38062
I20260709 13:24:06.097345 15046 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.005s	sys 0.000s
I20260709 13:24:06.101061 26871 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:24:06.102046 15046 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.001s	sys 0.000s
I20260709 13:24:06.102406 15046 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskdH_Ma7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783603093801613-15046-0/minicluster-data/ts-2-root
uuid: "239ae18f3e44498fa52e20e9d6eec6b5"
format_stamp: "Formatted at 2026-07-09 13:24:06 on dist-test-slave-v5z2"
I20260709 13:24:06.102722 15046 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdH_Ma7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783603093801613-15046-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskdH_Ma7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783603093801613-15046-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskdH_Ma7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783603093801613-15046-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:24:06.133232 15046 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:24:06.134589 15046 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:24:06.136111 15046 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:24:06.138299 15046 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:24:06.138520 15046 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:24:06.138767 15046 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:24:06.138938 15046 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:24:06.179322 15046 rpc_server.cc:307] RPC server started. Bound to: 127.14.177.131:39265
I20260709 13:24:06.179446 26933 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.14.177.131:39265 every 8 connection(s)
I20260709 13:24:06.193171 26934 heartbeater.cc:344] Connected to a master server at 127.14.177.190:39149
I20260709 13:24:06.193627 26934 heartbeater.cc:461] Registering TS with master...
I20260709 13:24:06.194381 26934 heartbeater.cc:507] Master 127.14.177.190:39149 requested a full tablet report, sending...
I20260709 13:24:06.196429 26658 ts_manager.cc:194] Registered new tserver with Master: 239ae18f3e44498fa52e20e9d6eec6b5 (127.14.177.131:39265)
I20260709 13:24:06.197208 15046 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.014856379s
I20260709 13:24:06.197831 26658 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:38076
I20260709 13:24:06.224443 26658 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38092:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\314\314\314\014\004\001\000\230\231\231\031\004\001\000dff&\004\001\0000333\004\001\000\374\377\377?\004\001\000\310\314\314L\004\001\000\224\231\231Y\004\001\000`fff\004\001\000,33s""\004\001\000\314\314\314\014\004\001\000\230\231\231\031\004\001\000dff&\004\001\0000333\004\001\000\374\377\377?\004\001\000\310\314\314L\004\001\000\224\231\231Y\004\001\000`fff\004\001\000,33s"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 13:24:06.227349 26658 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 13:24:06.300612 26899 tablet_service.cc:1511] Processing CreateTablet for tablet d5f6b403f0cb4c7a9180d07edb98d1bc (DEFAULT_TABLE table=test-workload [id=7ae3f365cb0a4bcb9fc05a064879672e]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 13:24:06.302361 26899 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d5f6b403f0cb4c7a9180d07edb98d1bc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:24:06.304368 26898 tablet_service.cc:1511] Processing CreateTablet for tablet 2664382aca514208a9aa49316bdcaeeb (DEFAULT_TABLE table=test-workload [id=7ae3f365cb0a4bcb9fc05a064879672e]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 13:24:06.306064 26898 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2664382aca514208a9aa49316bdcaeeb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:24:06.316083 26825 tablet_service.cc:1511] Processing CreateTablet for tablet d5f6b403f0cb4c7a9180d07edb98d1bc (DEFAULT_TABLE table=test-workload [id=7ae3f365cb0a4bcb9fc05a064879672e]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 13:24:06.317553 26825 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d5f6b403f0cb4c7a9180d07edb98d1bc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:24:06.307505 26897 tablet_service.cc:1511] Processing CreateTablet for tablet d52f26a0449247c1b928641ec8ebad34 (DEFAULT_TABLE table=test-workload [id=7ae3f365cb0a4bcb9fc05a064879672e]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 13:24:06.320138 26897 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d52f26a0449247c1b928641ec8ebad34. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:24:06.311300 26896 tablet_service.cc:1511] Processing CreateTablet for tablet 92c3adeb279b4f5f88680a40a9664750 (DEFAULT_TABLE table=test-workload [id=7ae3f365cb0a4bcb9fc05a064879672e]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 13:24:06.321928 26896 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 92c3adeb279b4f5f88680a40a9664750. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:24:06.313163 26895 tablet_service.cc:1511] Processing CreateTablet for tablet bdb51cee932741eda8c12a88f07c890e (DEFAULT_TABLE table=test-workload [id=7ae3f365cb0a4bcb9fc05a064879672e]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 13:24:06.322348 26823 tablet_service.cc:1511] Processing CreateTablet for tablet d52f26a0449247c1b928641ec8ebad34 (DEFAULT_TABLE table=test-workload [id=7ae3f365cb0a4bcb9fc05a064879672e]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 13:24:06.323689 26895 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bdb51cee932741eda8c12a88f07c890e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:24:06.323804 26823 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d52f26a0449247c1b928641ec8ebad34. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:24:06.315774 26894 tablet_service.cc:1511] Processing CreateTablet for tablet 76c716abdcd6437b9dcee43317333298 (DEFAULT_TABLE table=test-workload [id=7ae3f365cb0a4bcb9fc05a064879672e]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 13:24:06.325434 26894 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 76c716abdcd6437b9dcee43317333298. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:24:06.320406 26824 tablet_service.cc:1511] Processing CreateTablet for tablet 2664382aca514208a9aa49316bdcaeeb (DEFAULT_TABLE table=test-workload [id=7ae3f365cb0a4bcb9fc05a064879672e]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 13:24:06.327266 26824 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2664382aca514208a9aa49316bdcaeeb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:24:06.328634 26822 tablet_service.cc:1511] Processing CreateTablet for tablet 92c3adeb279b4f5f88680a40a9664750 (DEFAULT_TABLE table=test-workload [id=7ae3f365cb0a4bcb9fc05a064879672e]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 13:24:06.330124 26822 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 92c3adeb279b4f5f88680a40a9664750. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:24:06.331871 26821 tablet_service.cc:1511] Processing CreateTablet for tablet bdb51cee932741eda8c12a88f07c890e (DEFAULT_TABLE table=test-workload [id=7ae3f365cb0a4bcb9fc05a064879672e]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 13:24:06.333731 26821 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bdb51cee932741eda8c12a88f07c890e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:24:06.326527 26820 tablet_service.cc:1511] Processing CreateTablet for tablet 76c716abdcd6437b9dcee43317333298 (DEFAULT_TABLE table=test-workload [id=7ae3f365cb0a4bcb9fc05a064879672e]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 13:24:06.336320 26820 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 76c716abdcd6437b9dcee43317333298. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:24:06.359094 26819 tablet_service.cc:1511] Processing CreateTablet for tablet 7492e977a5f74ef09aa687b1d7ec89e3 (DEFAULT_TABLE table=test-workload [id=7ae3f365cb0a4bcb9fc05a064879672e]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 13:24:06.360735 26819 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7492e977a5f74ef09aa687b1d7ec89e3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:24:06.365248 26818 tablet_service.cc:1511] Processing CreateTablet for tablet 3e489a04b06247608a195856b4739a36 (DEFAULT_TABLE table=test-workload [id=7ae3f365cb0a4bcb9fc05a064879672e]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 13:24:06.366938 26818 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3e489a04b06247608a195856b4739a36. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:24:06.385046 26893 tablet_service.cc:1511] Processing CreateTablet for tablet 7492e977a5f74ef09aa687b1d7ec89e3 (DEFAULT_TABLE table=test-workload [id=7ae3f365cb0a4bcb9fc05a064879672e]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 13:24:06.386806 26893 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7492e977a5f74ef09aa687b1d7ec89e3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:24:06.388386 26892 tablet_service.cc:1511] Processing CreateTablet for tablet 3e489a04b06247608a195856b4739a36 (DEFAULT_TABLE table=test-workload [id=7ae3f365cb0a4bcb9fc05a064879672e]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 13:24:06.390928 26891 tablet_service.cc:1511] Processing CreateTablet for tablet 9ba9365ebe904f85840f6d049dbcda88 (DEFAULT_TABLE table=test-workload [id=7ae3f365cb0a4bcb9fc05a064879672e]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 13:24:06.392669 26892 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3e489a04b06247608a195856b4739a36. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:24:06.399544 26891 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9ba9365ebe904f85840f6d049dbcda88. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:24:06.402611 26817 tablet_service.cc:1511] Processing CreateTablet for tablet 9ba9365ebe904f85840f6d049dbcda88 (DEFAULT_TABLE table=test-workload [id=7ae3f365cb0a4bcb9fc05a064879672e]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 13:24:06.404268 26817 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9ba9365ebe904f85840f6d049dbcda88. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:24:06.395213 26890 tablet_service.cc:1511] Processing CreateTablet for tablet e341897686b14a2491c3a4bd24ac9e95 (DEFAULT_TABLE table=test-workload [id=7ae3f365cb0a4bcb9fc05a064879672e]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 13:24:06.405936 26816 tablet_service.cc:1511] Processing CreateTablet for tablet e341897686b14a2491c3a4bd24ac9e95 (DEFAULT_TABLE table=test-workload [id=7ae3f365cb0a4bcb9fc05a064879672e]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 13:24:06.412626 26816 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e341897686b14a2491c3a4bd24ac9e95. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:24:06.412623 26890 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e341897686b14a2491c3a4bd24ac9e95. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:24:06.450693 26750 tablet_service.cc:1511] Processing CreateTablet for tablet d5f6b403f0cb4c7a9180d07edb98d1bc (DEFAULT_TABLE table=test-workload [id=7ae3f365cb0a4bcb9fc05a064879672e]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 13:24:06.452234 26750 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d5f6b403f0cb4c7a9180d07edb98d1bc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:24:06.456483 26749 tablet_service.cc:1511] Processing CreateTablet for tablet 2664382aca514208a9aa49316bdcaeeb (DEFAULT_TABLE table=test-workload [id=7ae3f365cb0a4bcb9fc05a064879672e]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 13:24:06.458056 26749 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2664382aca514208a9aa49316bdcaeeb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:24:06.458230 26744 tablet_service.cc:1511] Processing CreateTablet for tablet 7492e977a5f74ef09aa687b1d7ec89e3 (DEFAULT_TABLE table=test-workload [id=7ae3f365cb0a4bcb9fc05a064879672e]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 13:24:06.459781 26744 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7492e977a5f74ef09aa687b1d7ec89e3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:24:06.460392 26748 tablet_service.cc:1511] Processing CreateTablet for tablet d52f26a0449247c1b928641ec8ebad34 (DEFAULT_TABLE table=test-workload [id=7ae3f365cb0a4bcb9fc05a064879672e]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 13:24:06.461892 26748 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d52f26a0449247c1b928641ec8ebad34. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:24:06.464409 26747 tablet_service.cc:1511] Processing CreateTablet for tablet 92c3adeb279b4f5f88680a40a9664750 (DEFAULT_TABLE table=test-workload [id=7ae3f365cb0a4bcb9fc05a064879672e]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 13:24:06.464854 26743 tablet_service.cc:1511] Processing CreateTablet for tablet 3e489a04b06247608a195856b4739a36 (DEFAULT_TABLE table=test-workload [id=7ae3f365cb0a4bcb9fc05a064879672e]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 13:24:06.465830 26747 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 92c3adeb279b4f5f88680a40a9664750. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:24:06.466404 26743 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3e489a04b06247608a195856b4739a36. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:24:06.471870 26746 tablet_service.cc:1511] Processing CreateTablet for tablet bdb51cee932741eda8c12a88f07c890e (DEFAULT_TABLE table=test-workload [id=7ae3f365cb0a4bcb9fc05a064879672e]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 13:24:06.473323 26746 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bdb51cee932741eda8c12a88f07c890e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:24:06.476436 26742 tablet_service.cc:1511] Processing CreateTablet for tablet 9ba9365ebe904f85840f6d049dbcda88 (DEFAULT_TABLE table=test-workload [id=7ae3f365cb0a4bcb9fc05a064879672e]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 13:24:06.477872 26742 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9ba9365ebe904f85840f6d049dbcda88. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:24:06.480408 26741 tablet_service.cc:1511] Processing CreateTablet for tablet e341897686b14a2491c3a4bd24ac9e95 (DEFAULT_TABLE table=test-workload [id=7ae3f365cb0a4bcb9fc05a064879672e]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 13:24:06.481879 26741 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e341897686b14a2491c3a4bd24ac9e95. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:24:06.456483 26745 tablet_service.cc:1511] Processing CreateTablet for tablet 76c716abdcd6437b9dcee43317333298 (DEFAULT_TABLE table=test-workload [id=7ae3f365cb0a4bcb9fc05a064879672e]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 13:24:06.501214 26745 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 76c716abdcd6437b9dcee43317333298. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:24:06.507864 26949 tablet_bootstrap.cc:492] T 2664382aca514208a9aa49316bdcaeeb P 239ae18f3e44498fa52e20e9d6eec6b5: Bootstrap starting.
I20260709 13:24:06.539626 26948 tablet_bootstrap.cc:492] T 7492e977a5f74ef09aa687b1d7ec89e3 P 326c74b0a8d84cdf86229f67ce3487f2: Bootstrap starting.
I20260709 13:24:06.545035 26949 tablet_bootstrap.cc:654] T 2664382aca514208a9aa49316bdcaeeb P 239ae18f3e44498fa52e20e9d6eec6b5: Neither blocks nor log segments found. Creating new log.
I20260709 13:24:06.566120 26950 tablet_bootstrap.cc:492] T 9ba9365ebe904f85840f6d049dbcda88 P ff90de856dd542eab446899d181dcad0: Bootstrap starting.
I20260709 13:24:06.568368 26948 tablet_bootstrap.cc:654] T 7492e977a5f74ef09aa687b1d7ec89e3 P 326c74b0a8d84cdf86229f67ce3487f2: Neither blocks nor log segments found. Creating new log.
I20260709 13:24:06.572126 26950 tablet_bootstrap.cc:654] T 9ba9365ebe904f85840f6d049dbcda88 P ff90de856dd542eab446899d181dcad0: Neither blocks nor log segments found. Creating new log.
I20260709 13:24:06.592156 26948 tablet_bootstrap.cc:492] T 7492e977a5f74ef09aa687b1d7ec89e3 P 326c74b0a8d84cdf86229f67ce3487f2: No bootstrap required, opened a new log
I20260709 13:24:06.592723 26948 ts_tablet_manager.cc:1403] T 7492e977a5f74ef09aa687b1d7ec89e3 P 326c74b0a8d84cdf86229f67ce3487f2: Time spent bootstrapping tablet: real 0.053s	user 0.010s	sys 0.012s
I20260709 13:24:06.598978 26948 raft_consensus.cc:359] T 7492e977a5f74ef09aa687b1d7ec89e3 P 326c74b0a8d84cdf86229f67ce3487f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff90de856dd542eab446899d181dcad0" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36311 } } peers { permanent_uuid: "326c74b0a8d84cdf86229f67ce3487f2" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35517 } } peers { permanent_uuid: "239ae18f3e44498fa52e20e9d6eec6b5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 39265 } }
I20260709 13:24:06.599934 26948 raft_consensus.cc:385] T 7492e977a5f74ef09aa687b1d7ec89e3 P 326c74b0a8d84cdf86229f67ce3487f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:24:06.600255 26948 raft_consensus.cc:740] T 7492e977a5f74ef09aa687b1d7ec89e3 P 326c74b0a8d84cdf86229f67ce3487f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 326c74b0a8d84cdf86229f67ce3487f2, State: Initialized, Role: FOLLOWER
I20260709 13:24:06.618098 26948 consensus_queue.cc:260] T 7492e977a5f74ef09aa687b1d7ec89e3 P 326c74b0a8d84cdf86229f67ce3487f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff90de856dd542eab446899d181dcad0" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36311 } } peers { permanent_uuid: "326c74b0a8d84cdf86229f67ce3487f2" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35517 } } peers { permanent_uuid: "239ae18f3e44498fa52e20e9d6eec6b5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 39265 } }
I20260709 13:24:06.672500 26860 heartbeater.cc:499] Master 127.14.177.190:39149 was elected leader, sending a full tablet report...
W20260709 13:24:06.674670 26645 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.CreateTablet -> {remote=127.14.177.131:39265, user_credentials={real_user=slave}} blocked reactor thread for 79535.6us
I20260709 13:24:06.675760 26949 tablet_bootstrap.cc:492] T 2664382aca514208a9aa49316bdcaeeb P 239ae18f3e44498fa52e20e9d6eec6b5: No bootstrap required, opened a new log
I20260709 13:24:06.676419 26949 ts_tablet_manager.cc:1403] T 2664382aca514208a9aa49316bdcaeeb P 239ae18f3e44498fa52e20e9d6eec6b5: Time spent bootstrapping tablet: real 0.169s	user 0.017s	sys 0.017s
I20260709 13:24:06.679948 26950 tablet_bootstrap.cc:492] T 9ba9365ebe904f85840f6d049dbcda88 P ff90de856dd542eab446899d181dcad0: No bootstrap required, opened a new log
I20260709 13:24:06.679986 26949 raft_consensus.cc:359] T 2664382aca514208a9aa49316bdcaeeb P 239ae18f3e44498fa52e20e9d6eec6b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff90de856dd542eab446899d181dcad0" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36311 } } peers { permanent_uuid: "326c74b0a8d84cdf86229f67ce3487f2" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35517 } } peers { permanent_uuid: "239ae18f3e44498fa52e20e9d6eec6b5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 39265 } }
I20260709 13:24:06.683283 26949 raft_consensus.cc:385] T 2664382aca514208a9aa49316bdcaeeb P 239ae18f3e44498fa52e20e9d6eec6b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:24:06.683525 26950 ts_tablet_manager.cc:1403] T 9ba9365ebe904f85840f6d049dbcda88 P ff90de856dd542eab446899d181dcad0: Time spent bootstrapping tablet: real 0.118s	user 0.015s	sys 0.017s
I20260709 13:24:06.683676 26949 raft_consensus.cc:740] T 2664382aca514208a9aa49316bdcaeeb P 239ae18f3e44498fa52e20e9d6eec6b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 239ae18f3e44498fa52e20e9d6eec6b5, State: Initialized, Role: FOLLOWER
I20260709 13:24:06.681933 26948 ts_tablet_manager.cc:1434] T 7492e977a5f74ef09aa687b1d7ec89e3 P 326c74b0a8d84cdf86229f67ce3487f2: Time spent starting tablet: real 0.089s	user 0.009s	sys 0.009s
W20260709 13:24:06.682559 26642 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.CreateTablet -> {remote=127.14.177.130:35517, user_credentials={real_user=slave}} blocked reactor thread for 86318.2us
I20260709 13:24:06.686493 26948 tablet_bootstrap.cc:492] T bdb51cee932741eda8c12a88f07c890e P 326c74b0a8d84cdf86229f67ce3487f2: Bootstrap starting.
I20260709 13:24:06.687532 26949 consensus_queue.cc:260] T 2664382aca514208a9aa49316bdcaeeb P 239ae18f3e44498fa52e20e9d6eec6b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff90de856dd542eab446899d181dcad0" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36311 } } peers { permanent_uuid: "326c74b0a8d84cdf86229f67ce3487f2" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35517 } } peers { permanent_uuid: "239ae18f3e44498fa52e20e9d6eec6b5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 39265 } }
I20260709 13:24:06.688175 26950 raft_consensus.cc:359] T 9ba9365ebe904f85840f6d049dbcda88 P ff90de856dd542eab446899d181dcad0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff90de856dd542eab446899d181dcad0" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36311 } } peers { permanent_uuid: "326c74b0a8d84cdf86229f67ce3487f2" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35517 } } peers { permanent_uuid: "239ae18f3e44498fa52e20e9d6eec6b5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 39265 } }
I20260709 13:24:06.689036 26950 raft_consensus.cc:385] T 9ba9365ebe904f85840f6d049dbcda88 P ff90de856dd542eab446899d181dcad0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:24:06.689397 26950 raft_consensus.cc:740] T 9ba9365ebe904f85840f6d049dbcda88 P ff90de856dd542eab446899d181dcad0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ff90de856dd542eab446899d181dcad0, State: Initialized, Role: FOLLOWER
I20260709 13:24:06.690347 26934 heartbeater.cc:499] Master 127.14.177.190:39149 was elected leader, sending a full tablet report...
I20260709 13:24:06.690299 26950 consensus_queue.cc:260] T 9ba9365ebe904f85840f6d049dbcda88 P ff90de856dd542eab446899d181dcad0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff90de856dd542eab446899d181dcad0" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36311 } } peers { permanent_uuid: "326c74b0a8d84cdf86229f67ce3487f2" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35517 } } peers { permanent_uuid: "239ae18f3e44498fa52e20e9d6eec6b5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 39265 } }
I20260709 13:24:06.692464 26949 ts_tablet_manager.cc:1434] T 2664382aca514208a9aa49316bdcaeeb P 239ae18f3e44498fa52e20e9d6eec6b5: Time spent starting tablet: real 0.016s	user 0.006s	sys 0.003s
I20260709 13:24:06.693208 26785 heartbeater.cc:499] Master 127.14.177.190:39149 was elected leader, sending a full tablet report...
I20260709 13:24:06.693859 26950 ts_tablet_manager.cc:1434] T 9ba9365ebe904f85840f6d049dbcda88 P ff90de856dd542eab446899d181dcad0: Time spent starting tablet: real 0.010s	user 0.008s	sys 0.000s
I20260709 13:24:06.695188 26950 tablet_bootstrap.cc:492] T 3e489a04b06247608a195856b4739a36 P ff90de856dd542eab446899d181dcad0: Bootstrap starting.
I20260709 13:24:06.698073 26948 tablet_bootstrap.cc:654] T bdb51cee932741eda8c12a88f07c890e P 326c74b0a8d84cdf86229f67ce3487f2: Neither blocks nor log segments found. Creating new log.
I20260709 13:24:06.703073 26950 tablet_bootstrap.cc:654] T 3e489a04b06247608a195856b4739a36 P ff90de856dd542eab446899d181dcad0: Neither blocks nor log segments found. Creating new log.
I20260709 13:24:06.708182 26949 tablet_bootstrap.cc:492] T 76c716abdcd6437b9dcee43317333298 P 239ae18f3e44498fa52e20e9d6eec6b5: Bootstrap starting.
I20260709 13:24:06.730923 26949 tablet_bootstrap.cc:654] T 76c716abdcd6437b9dcee43317333298 P 239ae18f3e44498fa52e20e9d6eec6b5: Neither blocks nor log segments found. Creating new log.
I20260709 13:24:06.731222 26948 tablet_bootstrap.cc:492] T bdb51cee932741eda8c12a88f07c890e P 326c74b0a8d84cdf86229f67ce3487f2: No bootstrap required, opened a new log
W20260709 13:24:06.731853 26786 tablet.cc:2406] T 9ba9365ebe904f85840f6d049dbcda88 P ff90de856dd542eab446899d181dcad0: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260709 13:24:06.732182 26948 ts_tablet_manager.cc:1403] T bdb51cee932741eda8c12a88f07c890e P 326c74b0a8d84cdf86229f67ce3487f2: Time spent bootstrapping tablet: real 0.046s	user 0.017s	sys 0.023s
I20260709 13:24:06.733177 26950 tablet_bootstrap.cc:492] T 3e489a04b06247608a195856b4739a36 P ff90de856dd542eab446899d181dcad0: No bootstrap required, opened a new log
I20260709 13:24:06.733754 26950 ts_tablet_manager.cc:1403] T 3e489a04b06247608a195856b4739a36 P ff90de856dd542eab446899d181dcad0: Time spent bootstrapping tablet: real 0.039s	user 0.024s	sys 0.009s
I20260709 13:24:06.735837 26948 raft_consensus.cc:359] T bdb51cee932741eda8c12a88f07c890e P 326c74b0a8d84cdf86229f67ce3487f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff90de856dd542eab446899d181dcad0" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36311 } } peers { permanent_uuid: "326c74b0a8d84cdf86229f67ce3487f2" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35517 } } peers { permanent_uuid: "239ae18f3e44498fa52e20e9d6eec6b5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 39265 } }
I20260709 13:24:06.736557 26948 raft_consensus.cc:385] T bdb51cee932741eda8c12a88f07c890e P 326c74b0a8d84cdf86229f67ce3487f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:24:06.736840 26948 raft_consensus.cc:740] T bdb51cee932741eda8c12a88f07c890e P 326c74b0a8d84cdf86229f67ce3487f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 326c74b0a8d84cdf86229f67ce3487f2, State: Initialized, Role: FOLLOWER
I20260709 13:24:06.737028 26950 raft_consensus.cc:359] T 3e489a04b06247608a195856b4739a36 P ff90de856dd542eab446899d181dcad0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff90de856dd542eab446899d181dcad0" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36311 } } peers { permanent_uuid: "326c74b0a8d84cdf86229f67ce3487f2" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35517 } } peers { permanent_uuid: "239ae18f3e44498fa52e20e9d6eec6b5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 39265 } }
I20260709 13:24:06.737859 26950 raft_consensus.cc:385] T 3e489a04b06247608a195856b4739a36 P ff90de856dd542eab446899d181dcad0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:24:06.737633 26948 consensus_queue.cc:260] T bdb51cee932741eda8c12a88f07c890e P 326c74b0a8d84cdf86229f67ce3487f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff90de856dd542eab446899d181dcad0" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36311 } } peers { permanent_uuid: "326c74b0a8d84cdf86229f67ce3487f2" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35517 } } peers { permanent_uuid: "239ae18f3e44498fa52e20e9d6eec6b5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 39265 } }
I20260709 13:24:06.738300 26950 raft_consensus.cc:740] T 3e489a04b06247608a195856b4739a36 P ff90de856dd542eab446899d181dcad0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ff90de856dd542eab446899d181dcad0, State: Initialized, Role: FOLLOWER
I20260709 13:24:06.739667 26950 consensus_queue.cc:260] T 3e489a04b06247608a195856b4739a36 P ff90de856dd542eab446899d181dcad0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff90de856dd542eab446899d181dcad0" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36311 } } peers { permanent_uuid: "326c74b0a8d84cdf86229f67ce3487f2" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35517 } } peers { permanent_uuid: "239ae18f3e44498fa52e20e9d6eec6b5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 39265 } }
I20260709 13:24:06.741115 26948 ts_tablet_manager.cc:1434] T bdb51cee932741eda8c12a88f07c890e P 326c74b0a8d84cdf86229f67ce3487f2: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.001s
I20260709 13:24:06.742086 26948 tablet_bootstrap.cc:492] T 2664382aca514208a9aa49316bdcaeeb P 326c74b0a8d84cdf86229f67ce3487f2: Bootstrap starting.
I20260709 13:24:06.742257 26950 ts_tablet_manager.cc:1434] T 3e489a04b06247608a195856b4739a36 P ff90de856dd542eab446899d181dcad0: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.001s
I20260709 13:24:06.743458 26950 tablet_bootstrap.cc:492] T 7492e977a5f74ef09aa687b1d7ec89e3 P ff90de856dd542eab446899d181dcad0: Bootstrap starting.
I20260709 13:24:06.750725 26950 tablet_bootstrap.cc:654] T 7492e977a5f74ef09aa687b1d7ec89e3 P ff90de856dd542eab446899d181dcad0: Neither blocks nor log segments found. Creating new log.
I20260709 13:24:06.756716 26948 tablet_bootstrap.cc:654] T 2664382aca514208a9aa49316bdcaeeb P 326c74b0a8d84cdf86229f67ce3487f2: Neither blocks nor log segments found. Creating new log.
I20260709 13:24:06.764570 26949 tablet_bootstrap.cc:492] T 76c716abdcd6437b9dcee43317333298 P 239ae18f3e44498fa52e20e9d6eec6b5: No bootstrap required, opened a new log
I20260709 13:24:06.765522 26949 ts_tablet_manager.cc:1403] T 76c716abdcd6437b9dcee43317333298 P 239ae18f3e44498fa52e20e9d6eec6b5: Time spent bootstrapping tablet: real 0.058s	user 0.022s	sys 0.015s
I20260709 13:24:06.770007 26948 tablet_bootstrap.cc:492] T 2664382aca514208a9aa49316bdcaeeb P 326c74b0a8d84cdf86229f67ce3487f2: No bootstrap required, opened a new log
I20260709 13:24:06.769675 26949 raft_consensus.cc:359] T 76c716abdcd6437b9dcee43317333298 P 239ae18f3e44498fa52e20e9d6eec6b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff90de856dd542eab446899d181dcad0" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36311 } } peers { permanent_uuid: "326c74b0a8d84cdf86229f67ce3487f2" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35517 } } peers { permanent_uuid: "239ae18f3e44498fa52e20e9d6eec6b5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 39265 } }
I20260709 13:24:06.770299 26950 tablet_bootstrap.cc:492] T 7492e977a5f74ef09aa687b1d7ec89e3 P ff90de856dd542eab446899d181dcad0: No bootstrap required, opened a new log
I20260709 13:24:06.770574 26948 ts_tablet_manager.cc:1403] T 2664382aca514208a9aa49316bdcaeeb P 326c74b0a8d84cdf86229f67ce3487f2: Time spent bootstrapping tablet: real 0.029s	user 0.016s	sys 0.004s
I20260709 13:24:06.770542 26949 raft_consensus.cc:385] T 76c716abdcd6437b9dcee43317333298 P 239ae18f3e44498fa52e20e9d6eec6b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:24:06.770895 26950 ts_tablet_manager.cc:1403] T 7492e977a5f74ef09aa687b1d7ec89e3 P ff90de856dd542eab446899d181dcad0: Time spent bootstrapping tablet: real 0.028s	user 0.013s	sys 0.003s
I20260709 13:24:06.771133 26949 raft_consensus.cc:740] T 76c716abdcd6437b9dcee43317333298 P 239ae18f3e44498fa52e20e9d6eec6b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 239ae18f3e44498fa52e20e9d6eec6b5, State: Initialized, Role: FOLLOWER
I20260709 13:24:06.772029 26949 consensus_queue.cc:260] T 76c716abdcd6437b9dcee43317333298 P 239ae18f3e44498fa52e20e9d6eec6b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff90de856dd542eab446899d181dcad0" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36311 } } peers { permanent_uuid: "326c74b0a8d84cdf86229f67ce3487f2" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35517 } } peers { permanent_uuid: "239ae18f3e44498fa52e20e9d6eec6b5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 39265 } }
I20260709 13:24:06.774163 26949 ts_tablet_manager.cc:1434] T 76c716abdcd6437b9dcee43317333298 P 239ae18f3e44498fa52e20e9d6eec6b5: Time spent starting tablet: real 0.008s	user 0.002s	sys 0.003s
I20260709 13:24:06.773908 26948 raft_consensus.cc:359] T 2664382aca514208a9aa49316bdcaeeb P 326c74b0a8d84cdf86229f67ce3487f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff90de856dd542eab446899d181dcad0" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36311 } } peers { permanent_uuid: "326c74b0a8d84cdf86229f67ce3487f2" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35517 } } peers { permanent_uuid: "239ae18f3e44498fa52e20e9d6eec6b5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 39265 } }
I20260709 13:24:06.774735 26948 raft_consensus.cc:385] T 2664382aca514208a9aa49316bdcaeeb P 326c74b0a8d84cdf86229f67ce3487f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:24:06.775239 26948 raft_consensus.cc:740] T 2664382aca514208a9aa49316bdcaeeb P 326c74b0a8d84cdf86229f67ce3487f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 326c74b0a8d84cdf86229f67ce3487f2, State: Initialized, Role: FOLLOWER
I20260709 13:24:06.775501 26949 tablet_bootstrap.cc:492] T e341897686b14a2491c3a4bd24ac9e95 P 239ae18f3e44498fa52e20e9d6eec6b5: Bootstrap starting.
I20260709 13:24:06.775731 26950 raft_consensus.cc:359] T 7492e977a5f74ef09aa687b1d7ec89e3 P ff90de856dd542eab446899d181dcad0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff90de856dd542eab446899d181dcad0" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36311 } } peers { permanent_uuid: "326c74b0a8d84cdf86229f67ce3487f2" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35517 } } peers { permanent_uuid: "239ae18f3e44498fa52e20e9d6eec6b5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 39265 } }
I20260709 13:24:06.776561 26950 raft_consensus.cc:385] T 7492e977a5f74ef09aa687b1d7ec89e3 P ff90de856dd542eab446899d181dcad0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:24:06.776247 26948 consensus_queue.cc:260] T 2664382aca514208a9aa49316bdcaeeb P 326c74b0a8d84cdf86229f67ce3487f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff90de856dd542eab446899d181dcad0" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36311 } } peers { permanent_uuid: "326c74b0a8d84cdf86229f67ce3487f2" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35517 } } peers { permanent_uuid: "239ae18f3e44498fa52e20e9d6eec6b5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 39265 } }
I20260709 13:24:06.776889 26950 raft_consensus.cc:740] T 7492e977a5f74ef09aa687b1d7ec89e3 P ff90de856dd542eab446899d181dcad0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ff90de856dd542eab446899d181dcad0, State: Initialized, Role: FOLLOWER
I20260709 13:24:06.777926 26950 consensus_queue.cc:260] T 7492e977a5f74ef09aa687b1d7ec89e3 P ff90de856dd542eab446899d181dcad0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff90de856dd542eab446899d181dcad0" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36311 } } peers { permanent_uuid: "326c74b0a8d84cdf86229f67ce3487f2" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35517 } } peers { permanent_uuid: "239ae18f3e44498fa52e20e9d6eec6b5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 39265 } }
I20260709 13:24:06.782655 26949 tablet_bootstrap.cc:654] T e341897686b14a2491c3a4bd24ac9e95 P 239ae18f3e44498fa52e20e9d6eec6b5: Neither blocks nor log segments found. Creating new log.
I20260709 13:24:06.785912 26948 ts_tablet_manager.cc:1434] T 2664382aca514208a9aa49316bdcaeeb P 326c74b0a8d84cdf86229f67ce3487f2: Time spent starting tablet: real 0.015s	user 0.009s	sys 0.000s
I20260709 13:24:06.788172 26950 ts_tablet_manager.cc:1434] T 7492e977a5f74ef09aa687b1d7ec89e3 P ff90de856dd542eab446899d181dcad0: Time spent starting tablet: real 0.017s	user 0.004s	sys 0.003s
I20260709 13:24:06.790838 26948 tablet_bootstrap.cc:492] T 3e489a04b06247608a195856b4739a36 P 326c74b0a8d84cdf86229f67ce3487f2: Bootstrap starting.
I20260709 13:24:06.791747 26950 tablet_bootstrap.cc:492] T d52f26a0449247c1b928641ec8ebad34 P ff90de856dd542eab446899d181dcad0: Bootstrap starting.
I20260709 13:24:06.794894 26949 tablet_bootstrap.cc:492] T e341897686b14a2491c3a4bd24ac9e95 P 239ae18f3e44498fa52e20e9d6eec6b5: No bootstrap required, opened a new log
I20260709 13:24:06.795517 26949 ts_tablet_manager.cc:1403] T e341897686b14a2491c3a4bd24ac9e95 P 239ae18f3e44498fa52e20e9d6eec6b5: Time spent bootstrapping tablet: real 0.020s	user 0.009s	sys 0.008s
I20260709 13:24:06.795607 26955 raft_consensus.cc:493] T 76c716abdcd6437b9dcee43317333298 P 239ae18f3e44498fa52e20e9d6eec6b5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:24:06.796286 26955 raft_consensus.cc:515] T 76c716abdcd6437b9dcee43317333298 P 239ae18f3e44498fa52e20e9d6eec6b5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff90de856dd542eab446899d181dcad0" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36311 } } peers { permanent_uuid: "326c74b0a8d84cdf86229f67ce3487f2" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35517 } } peers { permanent_uuid: "239ae18f3e44498fa52e20e9d6eec6b5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 39265 } }
I20260709 13:24:06.799196 26954 raft_consensus.cc:493] T bdb51cee932741eda8c12a88f07c890e P 326c74b0a8d84cdf86229f67ce3487f2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:24:06.802773 26954 raft_consensus.cc:515] T bdb51cee932741eda8c12a88f07c890e P 326c74b0a8d84cdf86229f67ce3487f2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff90de856dd542eab446899d181dcad0" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36311 } } peers { permanent_uuid: "326c74b0a8d84cdf86229f67ce3487f2" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35517 } } peers { permanent_uuid: "239ae18f3e44498fa52e20e9d6eec6b5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 39265 } }
W20260709 13:24:06.801214 26706 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:24:06.803592 26950 tablet_bootstrap.cc:654] T d52f26a0449247c1b928641ec8ebad34 P ff90de856dd542eab446899d181dcad0: Neither blocks nor log segments found. Creating new log.
I20260709 13:24:06.802809 26948 tablet_bootstrap.cc:654] T 3e489a04b06247608a195856b4739a36 P 326c74b0a8d84cdf86229f67ce3487f2: Neither blocks nor log segments found. Creating new log.
I20260709 13:24:06.800638 26949 raft_consensus.cc:359] T e341897686b14a2491c3a4bd24ac9e95 P 239ae18f3e44498fa52e20e9d6eec6b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff90de856dd542eab446899d181dcad0" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36311 } } peers { permanent_uuid: "326c74b0a8d84cdf86229f67ce3487f2" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35517 } } peers { permanent_uuid: "239ae18f3e44498fa52e20e9d6eec6b5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 39265 } }
I20260709 13:24:06.805589 26949 raft_consensus.cc:385] T e341897686b14a2491c3a4bd24ac9e95 P 239ae18f3e44498fa52e20e9d6eec6b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:24:06.808197 26949 raft_consensus.cc:740] T e341897686b14a2491c3a4bd24ac9e95 P 239ae18f3e44498fa52e20e9d6eec6b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 239ae18f3e44498fa52e20e9d6eec6b5, State: Initialized, Role: FOLLOWER
I20260709 13:24:06.809320 26949 consensus_queue.cc:260] T e341897686b14a2491c3a4bd24ac9e95 P 239ae18f3e44498fa52e20e9d6eec6b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff90de856dd542eab446899d181dcad0" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36311 } } peers { permanent_uuid: "326c74b0a8d84cdf86229f67ce3487f2" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35517 } } peers { permanent_uuid: "239ae18f3e44498fa52e20e9d6eec6b5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 39265 } }
I20260709 13:24:06.802035 26955 leader_election.cc:290] T 76c716abdcd6437b9dcee43317333298 P 239ae18f3e44498fa52e20e9d6eec6b5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ff90de856dd542eab446899d181dcad0 (127.14.177.129:36311), 326c74b0a8d84cdf86229f67ce3487f2 (127.14.177.130:35517)
I20260709 13:24:06.829777 26954 leader_election.cc:290] T bdb51cee932741eda8c12a88f07c890e P 326c74b0a8d84cdf86229f67ce3487f2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ff90de856dd542eab446899d181dcad0 (127.14.177.129:36311), 239ae18f3e44498fa52e20e9d6eec6b5 (127.14.177.131:39265)
I20260709 13:24:06.847911 26948 tablet_bootstrap.cc:492] T 3e489a04b06247608a195856b4739a36 P 326c74b0a8d84cdf86229f67ce3487f2: No bootstrap required, opened a new log
I20260709 13:24:06.848675 26948 ts_tablet_manager.cc:1403] T 3e489a04b06247608a195856b4739a36 P 326c74b0a8d84cdf86229f67ce3487f2: Time spent bootstrapping tablet: real 0.058s	user 0.017s	sys 0.010s
I20260709 13:24:06.852298 26948 raft_consensus.cc:359] T 3e489a04b06247608a195856b4739a36 P 326c74b0a8d84cdf86229f67ce3487f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff90de856dd542eab446899d181dcad0" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36311 } } peers { permanent_uuid: "326c74b0a8d84cdf86229f67ce3487f2" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35517 } } peers { permanent_uuid: "239ae18f3e44498fa52e20e9d6eec6b5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 39265 } }
I20260709 13:24:06.853135 26948 raft_consensus.cc:385] T 3e489a04b06247608a195856b4739a36 P 326c74b0a8d84cdf86229f67ce3487f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:24:06.853474 26948 raft_consensus.cc:740] T 3e489a04b06247608a195856b4739a36 P 326c74b0a8d84cdf86229f67ce3487f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 326c74b0a8d84cdf86229f67ce3487f2, State: Initialized, Role: FOLLOWER
I20260709 13:24:06.854533 26948 consensus_queue.cc:260] T 3e489a04b06247608a195856b4739a36 P 326c74b0a8d84cdf86229f67ce3487f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff90de856dd542eab446899d181dcad0" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36311 } } peers { permanent_uuid: "326c74b0a8d84cdf86229f67ce3487f2" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35517 } } peers { permanent_uuid: "239ae18f3e44498fa52e20e9d6eec6b5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 39265 } }
I20260709 13:24:06.859980 26949 ts_tablet_manager.cc:1434] T e341897686b14a2491c3a4bd24ac9e95 P 239ae18f3e44498fa52e20e9d6eec6b5: Time spent starting tablet: real 0.064s	user 0.014s	sys 0.024s
I20260709 13:24:06.864255 26948 ts_tablet_manager.cc:1434] T 3e489a04b06247608a195856b4739a36 P 326c74b0a8d84cdf86229f67ce3487f2: Time spent starting tablet: real 0.015s	user 0.006s	sys 0.003s
I20260709 13:24:06.865471 26948 tablet_bootstrap.cc:492] T d52f26a0449247c1b928641ec8ebad34 P 326c74b0a8d84cdf86229f67ce3487f2: Bootstrap starting.
I20260709 13:24:06.868438 26949 tablet_bootstrap.cc:492] T 92c3adeb279b4f5f88680a40a9664750 P 239ae18f3e44498fa52e20e9d6eec6b5: Bootstrap starting.
I20260709 13:24:06.868935 26909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdb51cee932741eda8c12a88f07c890e" candidate_uuid: "326c74b0a8d84cdf86229f67ce3487f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "239ae18f3e44498fa52e20e9d6eec6b5" is_pre_election: true
W20260709 13:24:06.872082 26801 leader_election.cc:343] T bdb51cee932741eda8c12a88f07c890e P 326c74b0a8d84cdf86229f67ce3487f2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 239ae18f3e44498fa52e20e9d6eec6b5 (127.14.177.131:39265): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:24:06.874037 26760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76c716abdcd6437b9dcee43317333298" candidate_uuid: "239ae18f3e44498fa52e20e9d6eec6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff90de856dd542eab446899d181dcad0" is_pre_election: true
I20260709 13:24:06.862095 26950 tablet_bootstrap.cc:492] T d52f26a0449247c1b928641ec8ebad34 P ff90de856dd542eab446899d181dcad0: No bootstrap required, opened a new log
I20260709 13:24:06.875836 26950 ts_tablet_manager.cc:1403] T d52f26a0449247c1b928641ec8ebad34 P ff90de856dd542eab446899d181dcad0: Time spent bootstrapping tablet: real 0.084s	user 0.031s	sys 0.035s
W20260709 13:24:06.876022 26872 leader_election.cc:343] T 76c716abdcd6437b9dcee43317333298 P 239ae18f3e44498fa52e20e9d6eec6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ff90de856dd542eab446899d181dcad0 (127.14.177.129:36311): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:24:06.876439 26835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76c716abdcd6437b9dcee43317333298" candidate_uuid: "239ae18f3e44498fa52e20e9d6eec6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "326c74b0a8d84cdf86229f67ce3487f2" is_pre_election: true
W20260709 13:24:06.878264 26872 leader_election.cc:343] T 76c716abdcd6437b9dcee43317333298 P 239ae18f3e44498fa52e20e9d6eec6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 326c74b0a8d84cdf86229f67ce3487f2 (127.14.177.130:35517): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:24:06.878777 26872 leader_election.cc:304] T 76c716abdcd6437b9dcee43317333298 P 239ae18f3e44498fa52e20e9d6eec6b5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 239ae18f3e44498fa52e20e9d6eec6b5; no voters: 326c74b0a8d84cdf86229f67ce3487f2, ff90de856dd542eab446899d181dcad0
I20260709 13:24:06.879078 26950 raft_consensus.cc:359] T d52f26a0449247c1b928641ec8ebad34 P ff90de856dd542eab446899d181dcad0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff90de856dd542eab446899d181dcad0" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36311 } } peers { permanent_uuid: "326c74b0a8d84cdf86229f67ce3487f2" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35517 } } peers { permanent_uuid: "239ae18f3e44498fa52e20e9d6eec6b5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 39265 } }
I20260709 13:24:06.879937 26950 raft_consensus.cc:385] T d52f26a0449247c1b928641ec8ebad34 P ff90de856dd542eab446899d181dcad0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:24:06.880264 26950 raft_consensus.cc:740] T d52f26a0449247c1b928641ec8ebad34 P ff90de856dd542eab446899d181dcad0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ff90de856dd542eab446899d181dcad0, State: Initialized, Role: FOLLOWER
I20260709 13:24:06.881196 26950 consensus_queue.cc:260] T d52f26a0449247c1b928641ec8ebad34 P ff90de856dd542eab446899d181dcad0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff90de856dd542eab446899d181dcad0" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36311 } } peers { permanent_uuid: "326c74b0a8d84cdf86229f67ce3487f2" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35517 } } peers { permanent_uuid: "239ae18f3e44498fa52e20e9d6eec6b5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 39265 } }
I20260709 13:24:06.881822 26760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdb51cee932741eda8c12a88f07c890e" candidate_uuid: "326c74b0a8d84cdf86229f67ce3487f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff90de856dd542eab446899d181dcad0" is_pre_election: true
I20260709 13:24:06.882962 26968 raft_consensus.cc:2749] T 76c716abdcd6437b9dcee43317333298 P 239ae18f3e44498fa52e20e9d6eec6b5 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:24:06.883594 26949 tablet_bootstrap.cc:654] T 92c3adeb279b4f5f88680a40a9664750 P 239ae18f3e44498fa52e20e9d6eec6b5: Neither blocks nor log segments found. Creating new log.
I20260709 13:24:06.884274 26950 ts_tablet_manager.cc:1434] T d52f26a0449247c1b928641ec8ebad34 P ff90de856dd542eab446899d181dcad0: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.001s
I20260709 13:24:06.885370 26950 tablet_bootstrap.cc:492] T 76c716abdcd6437b9dcee43317333298 P ff90de856dd542eab446899d181dcad0: Bootstrap starting.
W20260709 13:24:06.885321 26798 leader_election.cc:343] T bdb51cee932741eda8c12a88f07c890e P 326c74b0a8d84cdf86229f67ce3487f2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ff90de856dd542eab446899d181dcad0 (127.14.177.129:36311): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:24:06.886090 26798 leader_election.cc:304] T bdb51cee932741eda8c12a88f07c890e P 326c74b0a8d84cdf86229f67ce3487f2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 326c74b0a8d84cdf86229f67ce3487f2; no voters: 239ae18f3e44498fa52e20e9d6eec6b5, ff90de856dd542eab446899d181dcad0
I20260709 13:24:06.887506 26954 raft_consensus.cc:2749] T bdb51cee932741eda8c12a88f07c890e P 326c74b0a8d84cdf86229f67ce3487f2 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:24:06.889684 26948 tablet_bootstrap.cc:654] T d52f26a0449247c1b928641ec8ebad34 P 326c74b0a8d84cdf86229f67ce3487f2: Neither blocks nor log segments found. Creating new log.
I20260709 13:24:06.894407 26950 tablet_bootstrap.cc:654] T 76c716abdcd6437b9dcee43317333298 P ff90de856dd542eab446899d181dcad0: Neither blocks nor log segments found. Creating new log.
I20260709 13:24:06.896023 26949 tablet_bootstrap.cc:492] T 92c3adeb279b4f5f88680a40a9664750 P 239ae18f3e44498fa52e20e9d6eec6b5: No bootstrap required, opened a new log
I20260709 13:24:06.896673 26949 ts_tablet_manager.cc:1403] T 92c3adeb279b4f5f88680a40a9664750 P 239ae18f3e44498fa52e20e9d6eec6b5: Time spent bootstrapping tablet: real 0.028s	user 0.011s	sys 0.008s
I20260709 13:24:06.901173 26949 raft_consensus.cc:359] T 92c3adeb279b4f5f88680a40a9664750 P 239ae18f3e44498fa52e20e9d6eec6b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff90de856dd542eab446899d181dcad0" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36311 } } peers { permanent_uuid: "326c74b0a8d84cdf86229f67ce3487f2" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35517 } } peers { permanent_uuid: "239ae18f3e44498fa52e20e9d6eec6b5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 39265 } }
I20260709 13:24:06.902384 26949 raft_consensus.cc:385] T 92c3adeb279b4f5f88680a40a9664750 P 239ae18f3e44498fa52e20e9d6eec6b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:24:06.902832 26949 raft_consensus.cc:740] T 92c3adeb279b4f5f88680a40a9664750 P 239ae18f3e44498fa52e20e9d6eec6b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 239ae18f3e44498fa52e20e9d6eec6b5, State: Initialized, Role: FOLLOWER
I20260709 13:24:06.903960 26949 consensus_queue.cc:260] T 92c3adeb279b4f5f88680a40a9664750 P 239ae18f3e44498fa52e20e9d6eec6b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff90de856dd542eab446899d181dcad0" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36311 } } peers { permanent_uuid: "326c74b0a8d84cdf86229f67ce3487f2" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35517 } } peers { permanent_uuid: "239ae18f3e44498fa52e20e9d6eec6b5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 39265 } }
I20260709 13:24:06.907099 26950 tablet_bootstrap.cc:492] T 76c716abdcd6437b9dcee43317333298 P ff90de856dd542eab446899d181dcad0: No bootstrap required, opened a new log
I20260709 13:24:06.907671 26949 ts_tablet_manager.cc:1434] T 92c3adeb279b4f5f88680a40a9664750 P 239ae18f3e44498fa52e20e9d6eec6b5: Time spent starting tablet: real 0.011s	user 0.008s	sys 0.000s
I20260709 13:24:06.907755 26950 ts_tablet_manager.cc:1403] T 76c716abdcd6437b9dcee43317333298 P ff90de856dd542eab446899d181dcad0: Time spent bootstrapping tablet: real 0.023s	user 0.008s	sys 0.013s
I20260709 13:24:06.908505 26948 tablet_bootstrap.cc:492] T d52f26a0449247c1b928641ec8ebad34 P 326c74b0a8d84cdf86229f67ce3487f2: No bootstrap required, opened a new log
I20260709 13:24:06.908881 26949 tablet_bootstrap.cc:492] T 7492e977a5f74ef09aa687b1d7ec89e3 P 239ae18f3e44498fa52e20e9d6eec6b5: Bootstrap starting.
I20260709 13:24:06.909104 26948 ts_tablet_manager.cc:1403] T d52f26a0449247c1b928641ec8ebad34 P 326c74b0a8d84cdf86229f67ce3487f2: Time spent bootstrapping tablet: real 0.044s	user 0.022s	sys 0.003s
I20260709 13:24:06.911384 26950 raft_consensus.cc:359] T 76c716abdcd6437b9dcee43317333298 P ff90de856dd542eab446899d181dcad0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff90de856dd542eab446899d181dcad0" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36311 } } peers { permanent_uuid: "326c74b0a8d84cdf86229f67ce3487f2" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35517 } } peers { permanent_uuid: "239ae18f3e44498fa52e20e9d6eec6b5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 39265 } }
I20260709 13:24:06.912268 26950 raft_consensus.cc:385] T 76c716abdcd6437b9dcee43317333298 P ff90de856dd542eab446899d181dcad0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:24:06.912601 26950 raft_consensus.cc:740] T 76c716abdcd6437b9dcee43317333298 P ff90de856dd542eab446899d181dcad0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ff90de856dd542eab446899d181dcad0, State: Initialized, Role: FOLLOWER
I20260709 13:24:06.912549 26948 raft_consensus.cc:359] T d52f26a0449247c1b928641ec8ebad34 P 326c74b0a8d84cdf86229f67ce3487f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff90de856dd542eab446899d181dcad0" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36311 } } peers { permanent_uuid: "326c74b0a8d84cdf86229f67ce3487f2" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35517 } } peers { permanent_uuid: "239ae18f3e44498fa52e20e9d6eec6b5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 39265 } }
I20260709 13:24:06.913676 26948 raft_consensus.cc:385] T d52f26a0449247c1b928641ec8ebad34 P 326c74b0a8d84cdf86229f67ce3487f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:24:06.914201 26948 raft_consensus.cc:740] T d52f26a0449247c1b928641ec8ebad34 P 326c74b0a8d84cdf86229f67ce3487f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 326c74b0a8d84cdf86229f67ce3487f2, State: Initialized, Role: FOLLOWER
I20260709 13:24:06.914280 26950 consensus_queue.cc:260] T 76c716abdcd6437b9dcee43317333298 P ff90de856dd542eab446899d181dcad0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff90de856dd542eab446899d181dcad0" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36311 } } peers { permanent_uuid: "326c74b0a8d84cdf86229f67ce3487f2" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35517 } } peers { permanent_uuid: "239ae18f3e44498fa52e20e9d6eec6b5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 39265 } }
I20260709 13:24:06.915439 26948 consensus_queue.cc:260] T d52f26a0449247c1b928641ec8ebad34 P 326c74b0a8d84cdf86229f67ce3487f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff90de856dd542eab446899d181dcad0" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36311 } } peers { permanent_uuid: "326c74b0a8d84cdf86229f67ce3487f2" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35517 } } peers { permanent_uuid: "239ae18f3e44498fa52e20e9d6eec6b5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 39265 } }
I20260709 13:24:06.916172 26949 tablet_bootstrap.cc:654] T 7492e977a5f74ef09aa687b1d7ec89e3 P 239ae18f3e44498fa52e20e9d6eec6b5: Neither blocks nor log segments found. Creating new log.
I20260709 13:24:06.917199 26950 ts_tablet_manager.cc:1434] T 76c716abdcd6437b9dcee43317333298 P ff90de856dd542eab446899d181dcad0: Time spent starting tablet: real 0.009s	user 0.003s	sys 0.005s
I20260709 13:24:06.918402 26950 tablet_bootstrap.cc:492] T 92c3adeb279b4f5f88680a40a9664750 P ff90de856dd542eab446899d181dcad0: Bootstrap starting.
I20260709 13:24:06.923421 26948 ts_tablet_manager.cc:1434] T d52f26a0449247c1b928641ec8ebad34 P 326c74b0a8d84cdf86229f67ce3487f2: Time spent starting tablet: real 0.014s	user 0.006s	sys 0.005s
I20260709 13:24:06.924805 26948 tablet_bootstrap.cc:492] T d5f6b403f0cb4c7a9180d07edb98d1bc P 326c74b0a8d84cdf86229f67ce3487f2: Bootstrap starting.
I20260709 13:24:06.925871 26950 tablet_bootstrap.cc:654] T 92c3adeb279b4f5f88680a40a9664750 P ff90de856dd542eab446899d181dcad0: Neither blocks nor log segments found. Creating new log.
I20260709 13:24:06.927259 26956 raft_consensus.cc:493] T d52f26a0449247c1b928641ec8ebad34 P ff90de856dd542eab446899d181dcad0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:24:06.928117 26949 tablet_bootstrap.cc:492] T 7492e977a5f74ef09aa687b1d7ec89e3 P 239ae18f3e44498fa52e20e9d6eec6b5: No bootstrap required, opened a new log
I20260709 13:24:06.927822 26956 raft_consensus.cc:515] T d52f26a0449247c1b928641ec8ebad34 P ff90de856dd542eab446899d181dcad0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff90de856dd542eab446899d181dcad0" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36311 } } peers { permanent_uuid: "326c74b0a8d84cdf86229f67ce3487f2" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35517 } } peers { permanent_uuid: "239ae18f3e44498fa52e20e9d6eec6b5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 39265 } }
I20260709 13:24:06.928645 26949 ts_tablet_manager.cc:1403] T 7492e977a5f74ef09aa687b1d7ec89e3 P 239ae18f3e44498fa52e20e9d6eec6b5: Time spent bootstrapping tablet: real 0.020s	user 0.011s	sys 0.007s
I20260709 13:24:06.932065 26949 raft_consensus.cc:359] T 7492e977a5f74ef09aa687b1d7ec89e3 P 239ae18f3e44498fa52e20e9d6eec6b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff90de856dd542eab446899d181dcad0" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36311 } } peers { permanent_uuid: "326c74b0a8d84cdf86229f67ce3487f2" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35517 } } peers { permanent_uuid: "239ae18f3e44498fa52e20e9d6eec6b5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 39265 } }
I20260709 13:24:06.932849 26949 raft_consensus.cc:385] T 7492e977a5f74ef09aa687b1d7ec89e3 P 239ae18f3e44498fa52e20e9d6eec6b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:24:06.933178 26949 raft_consensus.cc:740] T 7492e977a5f74ef09aa687b1d7ec89e3 P 239ae18f3e44498fa52e20e9d6eec6b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 239ae18f3e44498fa52e20e9d6eec6b5, State: Initialized, Role: FOLLOWER
I20260709 13:24:06.933359 26948 tablet_bootstrap.cc:654] T d5f6b403f0cb4c7a9180d07edb98d1bc P 326c74b0a8d84cdf86229f67ce3487f2: Neither blocks nor log segments found. Creating new log.
I20260709 13:24:06.934227 26949 consensus_queue.cc:260] T 7492e977a5f74ef09aa687b1d7ec89e3 P 239ae18f3e44498fa52e20e9d6eec6b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff90de856dd542eab446899d181dcad0" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36311 } } peers { permanent_uuid: "326c74b0a8d84cdf86229f67ce3487f2" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35517 } } peers { permanent_uuid: "239ae18f3e44498fa52e20e9d6eec6b5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 39265 } }
I20260709 13:24:06.936599 26949 ts_tablet_manager.cc:1434] T 7492e977a5f74ef09aa687b1d7ec89e3 P 239ae18f3e44498fa52e20e9d6eec6b5: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 13:24:06.937700 26949 tablet_bootstrap.cc:492] T d52f26a0449247c1b928641ec8ebad34 P 239ae18f3e44498fa52e20e9d6eec6b5: Bootstrap starting.
I20260709 13:24:06.942519 26956 leader_election.cc:290] T d52f26a0449247c1b928641ec8ebad34 P ff90de856dd542eab446899d181dcad0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 326c74b0a8d84cdf86229f67ce3487f2 (127.14.177.130:35517), 239ae18f3e44498fa52e20e9d6eec6b5 (127.14.177.131:39265)
I20260709 13:24:06.945760 26949 tablet_bootstrap.cc:654] T d52f26a0449247c1b928641ec8ebad34 P 239ae18f3e44498fa52e20e9d6eec6b5: Neither blocks nor log segments found. Creating new log.
I20260709 13:24:06.958262 26949 tablet_bootstrap.cc:492] T d52f26a0449247c1b928641ec8ebad34 P 239ae18f3e44498fa52e20e9d6eec6b5: No bootstrap required, opened a new log
I20260709 13:24:06.958928 26949 ts_tablet_manager.cc:1403] T d52f26a0449247c1b928641ec8ebad34 P 239ae18f3e44498fa52e20e9d6eec6b5: Time spent bootstrapping tablet: real 0.021s	user 0.007s	sys 0.010s
I20260709 13:24:06.963264 26949 raft_consensus.cc:359] T d52f26a0449247c1b928641ec8ebad34 P 239ae18f3e44498fa52e20e9d6eec6b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff90de856dd542eab446899d181dcad0" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36311 } } peers { permanent_uuid: "326c74b0a8d84cdf86229f67ce3487f2" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35517 } } peers { permanent_uuid: "239ae18f3e44498fa52e20e9d6eec6b5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 39265 } }
I20260709 13:24:06.964190 26950 tablet_bootstrap.cc:492] T 92c3adeb279b4f5f88680a40a9664750 P ff90de856dd542eab446899d181dcad0: No bootstrap required, opened a new log
I20260709 13:24:06.964354 26949 raft_consensus.cc:385] T d52f26a0449247c1b928641ec8ebad34 P 239ae18f3e44498fa52e20e9d6eec6b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:24:06.965286 26949 raft_consensus.cc:740] T d52f26a0449247c1b928641ec8ebad34 P 239ae18f3e44498fa52e20e9d6eec6b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 239ae18f3e44498fa52e20e9d6eec6b5, State: Initialized, Role: FOLLOWER
I20260709 13:24:06.965591 26950 ts_tablet_manager.cc:1403] T 92c3adeb279b4f5f88680a40a9664750 P ff90de856dd542eab446899d181dcad0: Time spent bootstrapping tablet: real 0.047s	user 0.018s	sys 0.011s
I20260709 13:24:06.966238 26949 consensus_queue.cc:260] T d52f26a0449247c1b928641ec8ebad34 P 239ae18f3e44498fa52e20e9d6eec6b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff90de856dd542eab446899d181dcad0" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36311 } } peers { permanent_uuid: "326c74b0a8d84cdf86229f67ce3487f2" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35517 } } peers { permanent_uuid: "239ae18f3e44498fa52e20e9d6eec6b5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 39265 } }
I20260709 13:24:06.969625 26949 ts_tablet_manager.cc:1434] T d52f26a0449247c1b928641ec8ebad34 P 239ae18f3e44498fa52e20e9d6eec6b5: Time spent starting tablet: real 0.010s	user 0.003s	sys 0.005s
I20260709 13:24:06.970814 26949 tablet_bootstrap.cc:492] T 9ba9365ebe904f85840f6d049dbcda88 P 239ae18f3e44498fa52e20e9d6eec6b5: Bootstrap starting.
I20260709 13:24:06.971819 26950 raft_consensus.cc:359] T 92c3adeb279b4f5f88680a40a9664750 P ff90de856dd542eab446899d181dcad0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff90de856dd542eab446899d181dcad0" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36311 } } peers { permanent_uuid: "326c74b0a8d84cdf86229f67ce3487f2" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35517 } } peers { permanent_uuid: "239ae18f3e44498fa52e20e9d6eec6b5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 39265 } }
I20260709 13:24:06.973147 26950 raft_consensus.cc:385] T 92c3adeb279b4f5f88680a40a9664750 P ff90de856dd542eab446899d181dcad0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:24:06.973678 26950 raft_consensus.cc:740] T 92c3adeb279b4f5f88680a40a9664750 P ff90de856dd542eab446899d181dcad0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ff90de856dd542eab446899d181dcad0, State: Initialized, Role: FOLLOWER
I20260709 13:24:06.974830 26950 consensus_queue.cc:260] T 92c3adeb279b4f5f88680a40a9664750 P ff90de856dd542eab446899d181dcad0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff90de856dd542eab446899d181dcad0" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36311 } } peers { permanent_uuid: "326c74b0a8d84cdf86229f67ce3487f2" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35517 } } peers { permanent_uuid: "239ae18f3e44498fa52e20e9d6eec6b5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 39265 } }
I20260709 13:24:06.978120 26949 tablet_bootstrap.cc:654] T 9ba9365ebe904f85840f6d049dbcda88 P 239ae18f3e44498fa52e20e9d6eec6b5: Neither blocks nor log segments found. Creating new log.
I20260709 13:24:06.981801 26950 ts_tablet_manager.cc:1434] T 92c3adeb279b4f5f88680a40a9664750 P ff90de856dd542eab446899d181dcad0: Time spent starting tablet: real 0.015s	user 0.008s	sys 0.000s
I20260709 13:24:06.983527 26950 tablet_bootstrap.cc:492] T 2664382aca514208a9aa49316bdcaeeb P ff90de856dd542eab446899d181dcad0: Bootstrap starting.
I20260709 13:24:06.993743 26950 tablet_bootstrap.cc:654] T 2664382aca514208a9aa49316bdcaeeb P ff90de856dd542eab446899d181dcad0: Neither blocks nor log segments found. Creating new log.
I20260709 13:24:06.999418 26968 raft_consensus.cc:493] T 92c3adeb279b4f5f88680a40a9664750 P 239ae18f3e44498fa52e20e9d6eec6b5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:24:07.000227 26968 raft_consensus.cc:515] T 92c3adeb279b4f5f88680a40a9664750 P 239ae18f3e44498fa52e20e9d6eec6b5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff90de856dd542eab446899d181dcad0" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36311 } } peers { permanent_uuid: "326c74b0a8d84cdf86229f67ce3487f2" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35517 } } peers { permanent_uuid: "239ae18f3e44498fa52e20e9d6eec6b5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 39265 } }
I20260709 13:24:07.003635 26955 raft_consensus.cc:493] T 2664382aca514208a9aa49316bdcaeeb P 239ae18f3e44498fa52e20e9d6eec6b5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:24:07.004415 26955 raft_consensus.cc:515] T 2664382aca514208a9aa49316bdcaeeb P 239ae18f3e44498fa52e20e9d6eec6b5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff90de856dd542eab446899d181dcad0" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36311 } } peers { permanent_uuid: "326c74b0a8d84cdf86229f67ce3487f2" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35517 } } peers { permanent_uuid: "239ae18f3e44498fa52e20e9d6eec6b5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 39265 } }
I20260709 13:24:07.007390 26835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92c3adeb279b4f5f88680a40a9664750" candidate_uuid: "239ae18f3e44498fa52e20e9d6eec6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "326c74b0a8d84cdf86229f67ce3487f2" is_pre_election: true
I20260709 13:24:07.007776 26955 leader_election.cc:290] T 2664382aca514208a9aa49316bdcaeeb P 239ae18f3e44498fa52e20e9d6eec6b5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ff90de856dd542eab446899d181dcad0 (127.14.177.129:36311), 326c74b0a8d84cdf86229f67ce3487f2 (127.14.177.130:35517)
W20260709 13:24:07.010361 26872 leader_election.cc:343] T 92c3adeb279b4f5f88680a40a9664750 P 239ae18f3e44498fa52e20e9d6eec6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 326c74b0a8d84cdf86229f67ce3487f2 (127.14.177.130:35517): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:24:07.011855 26968 leader_election.cc:290] T 92c3adeb279b4f5f88680a40a9664750 P 239ae18f3e44498fa52e20e9d6eec6b5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ff90de856dd542eab446899d181dcad0 (127.14.177.129:36311), 326c74b0a8d84cdf86229f67ce3487f2 (127.14.177.130:35517)
I20260709 13:24:07.012487 26950 tablet_bootstrap.cc:492] T 2664382aca514208a9aa49316bdcaeeb P ff90de856dd542eab446899d181dcad0: No bootstrap required, opened a new log
I20260709 13:24:07.013047 26835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2664382aca514208a9aa49316bdcaeeb" candidate_uuid: "239ae18f3e44498fa52e20e9d6eec6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "326c74b0a8d84cdf86229f67ce3487f2" is_pre_election: true
I20260709 13:24:07.013356 26950 ts_tablet_manager.cc:1403] T 2664382aca514208a9aa49316bdcaeeb P ff90de856dd542eab446899d181dcad0: Time spent bootstrapping tablet: real 0.030s	user 0.018s	sys 0.005s
I20260709 13:24:07.014098 26835 raft_consensus.cc:2468] T 2664382aca514208a9aa49316bdcaeeb P 326c74b0a8d84cdf86229f67ce3487f2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 239ae18f3e44498fa52e20e9d6eec6b5 in term 0.
I20260709 13:24:07.016252 26872 leader_election.cc:304] T 2664382aca514208a9aa49316bdcaeeb P 239ae18f3e44498fa52e20e9d6eec6b5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 239ae18f3e44498fa52e20e9d6eec6b5, 326c74b0a8d84cdf86229f67ce3487f2; no voters: 
I20260709 13:24:07.017522 26968 raft_consensus.cc:2804] T 2664382aca514208a9aa49316bdcaeeb P 239ae18f3e44498fa52e20e9d6eec6b5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:24:07.018028 26968 raft_consensus.cc:493] T 2664382aca514208a9aa49316bdcaeeb P 239ae18f3e44498fa52e20e9d6eec6b5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:24:07.018460 26968 raft_consensus.cc:3060] T 2664382aca514208a9aa49316bdcaeeb P 239ae18f3e44498fa52e20e9d6eec6b5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:24:07.018567 26950 raft_consensus.cc:359] T 2664382aca514208a9aa49316bdcaeeb P ff90de856dd542eab446899d181dcad0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff90de856dd542eab446899d181dcad0" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36311 } } peers { permanent_uuid: "326c74b0a8d84cdf86229f67ce3487f2" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35517 } } peers { permanent_uuid: "239ae18f3e44498fa52e20e9d6eec6b5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 39265 } }
I20260709 13:24:07.019940 26950 raft_consensus.cc:385] T 2664382aca514208a9aa49316bdcaeeb P ff90de856dd542eab446899d181dcad0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:24:07.020401 26950 raft_consensus.cc:740] T 2664382aca514208a9aa49316bdcaeeb P ff90de856dd542eab446899d181dcad0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ff90de856dd542eab446899d181dcad0, State: Initialized, Role: FOLLOWER
I20260709 13:24:07.021657 26950 consensus_queue.cc:260] T 2664382aca514208a9aa49316bdcaeeb P ff90de856dd542eab446899d181dcad0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff90de856dd542eab446899d181dcad0" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36311 } } peers { permanent_uuid: "326c74b0a8d84cdf86229f67ce3487f2" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35517 } } peers { permanent_uuid: "239ae18f3e44498fa52e20e9d6eec6b5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 39265 } }
I20260709 13:24:07.025178 26950 ts_tablet_manager.cc:1434] T 2664382aca514208a9aa49316bdcaeeb P ff90de856dd542eab446899d181dcad0: Time spent starting tablet: real 0.011s	user 0.008s	sys 0.000s
I20260709 13:24:07.026697 26950 tablet_bootstrap.cc:492] T d5f6b403f0cb4c7a9180d07edb98d1bc P ff90de856dd542eab446899d181dcad0: Bootstrap starting.
I20260709 13:24:07.039974 26950 tablet_bootstrap.cc:654] T d5f6b403f0cb4c7a9180d07edb98d1bc P ff90de856dd542eab446899d181dcad0: Neither blocks nor log segments found. Creating new log.
I20260709 13:24:07.049224 26760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92c3adeb279b4f5f88680a40a9664750" candidate_uuid: "239ae18f3e44498fa52e20e9d6eec6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff90de856dd542eab446899d181dcad0" is_pre_election: true
I20260709 13:24:07.050304 26760 raft_consensus.cc:2468] T 92c3adeb279b4f5f88680a40a9664750 P ff90de856dd542eab446899d181dcad0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 239ae18f3e44498fa52e20e9d6eec6b5 in term 0.
I20260709 13:24:07.052383 26760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2664382aca514208a9aa49316bdcaeeb" candidate_uuid: "239ae18f3e44498fa52e20e9d6eec6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff90de856dd542eab446899d181dcad0" is_pre_election: true
I20260709 13:24:07.053278 26760 raft_consensus.cc:2468] T 2664382aca514208a9aa49316bdcaeeb P ff90de856dd542eab446899d181dcad0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 239ae18f3e44498fa52e20e9d6eec6b5 in term 0.
I20260709 13:24:07.053926 26950 tablet_bootstrap.cc:492] T d5f6b403f0cb4c7a9180d07edb98d1bc P ff90de856dd542eab446899d181dcad0: No bootstrap required, opened a new log
I20260709 13:24:07.054775 26950 ts_tablet_manager.cc:1403] T d5f6b403f0cb4c7a9180d07edb98d1bc P ff90de856dd542eab446899d181dcad0: Time spent bootstrapping tablet: real 0.028s	user 0.011s	sys 0.010s
I20260709 13:24:07.054826 26872 leader_election.cc:304] T 92c3adeb279b4f5f88680a40a9664750 P 239ae18f3e44498fa52e20e9d6eec6b5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 239ae18f3e44498fa52e20e9d6eec6b5, ff90de856dd542eab446899d181dcad0; no voters: 326c74b0a8d84cdf86229f67ce3487f2
I20260709 13:24:07.056681 26955 raft_consensus.cc:2804] T 92c3adeb279b4f5f88680a40a9664750 P 239ae18f3e44498fa52e20e9d6eec6b5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:24:07.057186 26955 raft_consensus.cc:493] T 92c3adeb279b4f5f88680a40a9664750 P 239ae18f3e44498fa52e20e9d6eec6b5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:24:07.057639 26955 raft_consensus.cc:3060] T 92c3adeb279b4f5f88680a40a9664750 P 239ae18f3e44498fa52e20e9d6eec6b5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:24:07.059186 26950 raft_consensus.cc:359] T d5f6b403f0cb4c7a9180d07edb98d1bc P ff90de856dd542eab446899d181dcad0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "239ae18f3e44498fa52e20e9d6eec6b5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 39265 } } peers { permanent_uuid: "326c74b0a8d84cdf86229f67ce3487f2" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35517 } } peers { permanent_uuid: "ff90de856dd542eab446899d181dcad0" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36311 } }
I20260709 13:24:07.060298 26950 raft_consensus.cc:385] T d5f6b403f0cb4c7a9180d07edb98d1bc P ff90de856dd542eab446899d181dcad0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:24:07.060734 26950 raft_consensus.cc:740] T d5f6b403f0cb4c7a9180d07edb98d1bc P ff90de856dd542eab446899d181dcad0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ff90de856dd542eab446899d181dcad0, State: Initialized, Role: FOLLOWER
I20260709 13:24:07.061930 26950 consensus_queue.cc:260] T d5f6b403f0cb4c7a9180d07edb98d1bc P ff90de856dd542eab446899d181dcad0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "239ae18f3e44498fa52e20e9d6eec6b5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 39265 } } peers { permanent_uuid: "326c74b0a8d84cdf86229f67ce3487f2" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35517 } } peers { permanent_uuid: "ff90de856dd542eab446899d181dcad0" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36311 } }
I20260709 13:24:07.065179 26950 ts_tablet_manager.cc:1434] T d5f6b403f0cb4c7a9180d07edb98d1bc P ff90de856dd542eab446899d181dcad0: Time spent starting tablet: real 0.010s	user 0.003s	sys 0.004s
I20260709 13:24:07.065821 26954 raft_consensus.cc:493] T 7492e977a5f74ef09aa687b1d7ec89e3 P 326c74b0a8d84cdf86229f67ce3487f2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:24:07.066538 26954 raft_consensus.cc:515] T 7492e977a5f74ef09aa687b1d7ec89e3 P 326c74b0a8d84cdf86229f67ce3487f2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff90de856dd542eab446899d181dcad0" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36311 } } peers { permanent_uuid: "326c74b0a8d84cdf86229f67ce3487f2" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35517 } } peers { permanent_uuid: "239ae18f3e44498fa52e20e9d6eec6b5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 39265 } }
I20260709 13:24:07.070654 26954 leader_election.cc:290] T 7492e977a5f74ef09aa687b1d7ec89e3 P 326c74b0a8d84cdf86229f67ce3487f2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ff90de856dd542eab446899d181dcad0 (127.14.177.129:36311), 239ae18f3e44498fa52e20e9d6eec6b5 (127.14.177.131:39265)
I20260709 13:24:07.066565 26950 tablet_bootstrap.cc:492] T e341897686b14a2491c3a4bd24ac9e95 P ff90de856dd542eab446899d181dcad0: Bootstrap starting.
I20260709 13:24:07.071681 26909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7492e977a5f74ef09aa687b1d7ec89e3" candidate_uuid: "326c74b0a8d84cdf86229f67ce3487f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "239ae18f3e44498fa52e20e9d6eec6b5" is_pre_election: true
I20260709 13:24:07.072633 26909 raft_consensus.cc:2468] T 7492e977a5f74ef09aa687b1d7ec89e3 P 239ae18f3e44498fa52e20e9d6eec6b5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 326c74b0a8d84cdf86229f67ce3487f2 in term 0.
I20260709 13:24:07.074463 26801 leader_election.cc:304] T 7492e977a5f74ef09aa687b1d7ec89e3 P 326c74b0a8d84cdf86229f67ce3487f2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 239ae18f3e44498fa52e20e9d6eec6b5, 326c74b0a8d84cdf86229f67ce3487f2; no voters: 
I20260709 13:24:07.075860 26954 raft_consensus.cc:2804] T 7492e977a5f74ef09aa687b1d7ec89e3 P 326c74b0a8d84cdf86229f67ce3487f2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:24:07.076332 26954 raft_consensus.cc:493] T 7492e977a5f74ef09aa687b1d7ec89e3 P 326c74b0a8d84cdf86229f67ce3487f2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:24:07.076762 26954 raft_consensus.cc:3060] T 7492e977a5f74ef09aa687b1d7ec89e3 P 326c74b0a8d84cdf86229f67ce3487f2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:24:07.078889 26950 tablet_bootstrap.cc:654] T e341897686b14a2491c3a4bd24ac9e95 P ff90de856dd542eab446899d181dcad0: Neither blocks nor log segments found. Creating new log.
I20260709 13:24:07.080081 26956 raft_consensus.cc:493] T 7492e977a5f74ef09aa687b1d7ec89e3 P ff90de856dd542eab446899d181dcad0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:24:07.080808 26956 raft_consensus.cc:515] T 7492e977a5f74ef09aa687b1d7ec89e3 P ff90de856dd542eab446899d181dcad0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff90de856dd542eab446899d181dcad0" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36311 } } peers { permanent_uuid: "326c74b0a8d84cdf86229f67ce3487f2" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35517 } } peers { permanent_uuid: "239ae18f3e44498fa52e20e9d6eec6b5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 39265 } }
I20260709 13:24:07.084860 26956 leader_election.cc:290] T 7492e977a5f74ef09aa687b1d7ec89e3 P ff90de856dd542eab446899d181dcad0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 326c74b0a8d84cdf86229f67ce3487f2 (127.14.177.130:35517), 239ae18f3e44498fa52e20e9d6eec6b5 (127.14.177.131:39265)
I20260709 13:24:07.090968 26760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7492e977a5f74ef09aa687b1d7ec89e3" candidate_uuid: "326c74b0a8d84cdf86229f67ce3487f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff90de856dd542eab446899d181dcad0" is_pre_election: true
I20260709 13:24:07.091746 26950 tablet_bootstrap.cc:492] T e341897686b14a2491c3a4bd24ac9e95 P ff90de856dd542eab446899d181dcad0: No bootstrap required, opened a new log
I20260709 13:24:07.092208 26760 raft_consensus.cc:2468] T 7492e977a5f74ef09aa687b1d7ec89e3 P ff90de856dd542eab446899d181dcad0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 326c74b0a8d84cdf86229f67ce3487f2 in term 0.
I20260709 13:24:07.092494 26950 ts_tablet_manager.cc:1403] T e341897686b14a2491c3a4bd24ac9e95 P ff90de856dd542eab446899d181dcad0: Time spent bootstrapping tablet: real 0.026s	user 0.011s	sys 0.009s
I20260709 13:24:07.094156 26956 raft_consensus.cc:493] T 3e489a04b06247608a195856b4739a36 P ff90de856dd542eab446899d181dcad0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:24:07.094856 26956 raft_consensus.cc:515] T 3e489a04b06247608a195856b4739a36 P ff90de856dd542eab446899d181dcad0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff90de856dd542eab446899d181dcad0" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36311 } } peers { permanent_uuid: "326c74b0a8d84cdf86229f67ce3487f2" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35517 } } peers { permanent_uuid: "239ae18f3e44498fa52e20e9d6eec6b5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 39265 } }
I20260709 13:24:07.097620 26950 raft_consensus.cc:359] T e341897686b14a2491c3a4bd24ac9e95 P ff90de856dd542eab446899d181dcad0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff90de856dd542eab446899d181dcad0" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36311 } } peers { permanent_uuid: "326c74b0a8d84cdf86229f67ce3487f2" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35517 } } peers { permanent_uuid: "239ae18f3e44498fa52e20e9d6eec6b5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 39265 } }
I20260709 13:24:07.098583 26950 raft_consensus.cc:385] T e341897686b14a2491c3a4bd24ac9e95 P ff90de856dd542eab446899d181dcad0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:24:07.098948 26950 raft_consensus.cc:740] T e341897686b14a2491c3a4bd24ac9e95 P ff90de856dd542eab446899d181dcad0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ff90de856dd542eab446899d181dcad0, State: Initialized, Role: FOLLOWER
I20260709 13:24:07.100845 26986 raft_consensus.cc:493] T 9ba9365ebe904f85840f6d049dbcda88 P ff90de856dd542eab446899d181dcad0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:24:07.101533 26986 raft_consensus.cc:515] T 9ba9365ebe904f85840f6d049dbcda88 P ff90de856dd542eab446899d181dcad0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff90de856dd542eab446899d181dcad0" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36311 } } peers { permanent_uuid: "326c74b0a8d84cdf86229f67ce3487f2" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35517 } } peers { permanent_uuid: "239ae18f3e44498fa52e20e9d6eec6b5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 39265 } }
I20260709 13:24:07.106884 26950 consensus_queue.cc:260] T e341897686b14a2491c3a4bd24ac9e95 P ff90de856dd542eab446899d181dcad0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff90de856dd542eab446899d181dcad0" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36311 } } peers { permanent_uuid: "326c74b0a8d84cdf86229f67ce3487f2" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35517 } } peers { permanent_uuid: "239ae18f3e44498fa52e20e9d6eec6b5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 39265 } }
I20260709 13:24:07.109440 26986 leader_election.cc:290] T 9ba9365ebe904f85840f6d049dbcda88 P ff90de856dd542eab446899d181dcad0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 326c74b0a8d84cdf86229f67ce3487f2 (127.14.177.130:35517), 239ae18f3e44498fa52e20e9d6eec6b5 (127.14.177.131:39265)
I20260709 13:24:07.116910 26950 ts_tablet_manager.cc:1434] T e341897686b14a2491c3a4bd24ac9e95 P ff90de856dd542eab446899d181dcad0: Time spent starting tablet: real 0.024s	user 0.010s	sys 0.002s
I20260709 13:24:07.117583 26956 leader_election.cc:290] T 3e489a04b06247608a195856b4739a36 P ff90de856dd542eab446899d181dcad0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 326c74b0a8d84cdf86229f67ce3487f2 (127.14.177.130:35517), 239ae18f3e44498fa52e20e9d6eec6b5 (127.14.177.131:39265)
I20260709 13:24:07.125731 26955 raft_consensus.cc:515] T 92c3adeb279b4f5f88680a40a9664750 P 239ae18f3e44498fa52e20e9d6eec6b5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff90de856dd542eab446899d181dcad0" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36311 } } peers { permanent_uuid: "326c74b0a8d84cdf86229f67ce3487f2" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35517 } } peers { permanent_uuid: "239ae18f3e44498fa52e20e9d6eec6b5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 39265 } }
I20260709 13:24:07.118330 26950 tablet_bootstrap.cc:492] T bdb51cee932741eda8c12a88f07c890e P ff90de856dd542eab446899d181dcad0: Bootstrap starting.
I20260709 13:24:07.152671 26988 raft_consensus.cc:493] T e341897686b14a2491c3a4bd24ac9e95 P 239ae18f3e44498fa52e20e9d6eec6b5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:24:07.153504 26988 raft_consensus.cc:515] T e341897686b14a2491c3a4bd24ac9e95 P 239ae18f3e44498fa52e20e9d6eec6b5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff90de856dd542eab446899d181dcad0" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36311 } } peers { permanent_uuid: "326c74b0a8d84cdf86229f67ce3487f2" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35517 } } peers { permanent_uuid: "239ae18f3e44498fa52e20e9d6eec6b5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 39265 } }
I20260709 13:24:07.160416 26760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e341897686b14a2491c3a4bd24ac9e95" candidate_uuid: "239ae18f3e44498fa52e20e9d6eec6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff90de856dd542eab446899d181dcad0" is_pre_election: true
I20260709 13:24:07.161142 26760 raft_consensus.cc:2468] T e341897686b14a2491c3a4bd24ac9e95 P ff90de856dd542eab446899d181dcad0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 239ae18f3e44498fa52e20e9d6eec6b5 in term 0.
I20260709 13:24:07.162247 26948 tablet_bootstrap.cc:492] T d5f6b403f0cb4c7a9180d07edb98d1bc P 326c74b0a8d84cdf86229f67ce3487f2: No bootstrap required, opened a new log
I20260709 13:24:07.163056 26948 ts_tablet_manager.cc:1403] T d5f6b403f0cb4c7a9180d07edb98d1bc P 326c74b0a8d84cdf86229f67ce3487f2: Time spent bootstrapping tablet: real 0.238s	user 0.092s	sys 0.137s
I20260709 13:24:07.163000 26872 leader_election.cc:304] T e341897686b14a2491c3a4bd24ac9e95 P 239ae18f3e44498fa52e20e9d6eec6b5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 239ae18f3e44498fa52e20e9d6eec6b5, ff90de856dd542eab446899d181dcad0; no voters: 
I20260709 13:24:07.162537 26759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92c3adeb279b4f5f88680a40a9664750" candidate_uuid: "239ae18f3e44498fa52e20e9d6eec6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff90de856dd542eab446899d181dcad0"
I20260709 13:24:07.164805 26759 raft_consensus.cc:3060] T 92c3adeb279b4f5f88680a40a9664750 P ff90de856dd542eab446899d181dcad0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:24:07.166905 26950 tablet_bootstrap.cc:654] T bdb51cee932741eda8c12a88f07c890e P ff90de856dd542eab446899d181dcad0: Neither blocks nor log segments found. Creating new log.
I20260709 13:24:07.167512 26948 raft_consensus.cc:359] T d5f6b403f0cb4c7a9180d07edb98d1bc P 326c74b0a8d84cdf86229f67ce3487f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "239ae18f3e44498fa52e20e9d6eec6b5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 39265 } } peers { permanent_uuid: "326c74b0a8d84cdf86229f67ce3487f2" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35517 } } peers { permanent_uuid: "ff90de856dd542eab446899d181dcad0" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36311 } }
I20260709 13:24:07.168805 26948 raft_consensus.cc:385] T d5f6b403f0cb4c7a9180d07edb98d1bc P 326c74b0a8d84cdf86229f67ce3487f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:24:07.169214 26948 raft_consensus.cc:740] T d5f6b403f0cb4c7a9180d07edb98d1bc P 326c74b0a8d84cdf86229f67ce3487f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 326c74b0a8d84cdf86229f67ce3487f2, State: Initialized, Role: FOLLOWER
I20260709 13:24:07.170197 26948 consensus_queue.cc:260] T d5f6b403f0cb4c7a9180d07edb98d1bc P 326c74b0a8d84cdf86229f67ce3487f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "239ae18f3e44498fa52e20e9d6eec6b5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 39265 } } peers { permanent_uuid: "326c74b0a8d84cdf86229f67ce3487f2" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35517 } } peers { permanent_uuid: "ff90de856dd542eab446899d181dcad0" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36311 } }
I20260709 13:24:07.150779 26954 raft_consensus.cc:515] T 7492e977a5f74ef09aa687b1d7ec89e3 P 326c74b0a8d84cdf86229f67ce3487f2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff90de856dd542eab446899d181dcad0" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36311 } } peers { permanent_uuid: "326c74b0a8d84cdf86229f67ce3487f2" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35517 } } peers { permanent_uuid: "239ae18f3e44498fa52e20e9d6eec6b5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 39265 } }
I20260709 13:24:07.159530 26968 raft_consensus.cc:515] T 2664382aca514208a9aa49316bdcaeeb P 239ae18f3e44498fa52e20e9d6eec6b5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff90de856dd542eab446899d181dcad0" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36311 } } peers { permanent_uuid: "326c74b0a8d84cdf86229f67ce3487f2" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35517 } } peers { permanent_uuid: "239ae18f3e44498fa52e20e9d6eec6b5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 39265 } }
I20260709 13:24:07.175249 26759 raft_consensus.cc:2468] T 92c3adeb279b4f5f88680a40a9664750 P ff90de856dd542eab446899d181dcad0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 239ae18f3e44498fa52e20e9d6eec6b5 in term 1.
I20260709 13:24:07.175626 26954 leader_election.cc:290] T 7492e977a5f74ef09aa687b1d7ec89e3 P 326c74b0a8d84cdf86229f67ce3487f2 [CANDIDATE]: Term 1 election: Requested vote from peers ff90de856dd542eab446899d181dcad0 (127.14.177.129:36311), 239ae18f3e44498fa52e20e9d6eec6b5 (127.14.177.131:39265)
I20260709 13:24:07.176447 26954 raft_consensus.cc:493] T 3e489a04b06247608a195856b4739a36 P 326c74b0a8d84cdf86229f67ce3487f2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:24:07.177009 26954 raft_consensus.cc:515] T 3e489a04b06247608a195856b4739a36 P 326c74b0a8d84cdf86229f67ce3487f2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff90de856dd542eab446899d181dcad0" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36311 } } peers { permanent_uuid: "326c74b0a8d84cdf86229f67ce3487f2" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35517 } } peers { permanent_uuid: "239ae18f3e44498fa52e20e9d6eec6b5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 39265 } }
I20260709 13:24:07.177702 26760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7492e977a5f74ef09aa687b1d7ec89e3" candidate_uuid: "326c74b0a8d84cdf86229f67ce3487f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff90de856dd542eab446899d181dcad0"
I20260709 13:24:07.178510 26760 raft_consensus.cc:3060] T 7492e977a5f74ef09aa687b1d7ec89e3 P ff90de856dd542eab446899d181dcad0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:24:07.190076 26956 raft_consensus.cc:493] T 76c716abdcd6437b9dcee43317333298 P ff90de856dd542eab446899d181dcad0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:24:07.198601 26759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e489a04b06247608a195856b4739a36" candidate_uuid: "326c74b0a8d84cdf86229f67ce3487f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff90de856dd542eab446899d181dcad0" is_pre_election: true
I20260709 13:24:07.199359 26759 raft_consensus.cc:2468] T 3e489a04b06247608a195856b4739a36 P ff90de856dd542eab446899d181dcad0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 326c74b0a8d84cdf86229f67ce3487f2 in term 0.
I20260709 13:24:07.198662 26909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7492e977a5f74ef09aa687b1d7ec89e3" candidate_uuid: "326c74b0a8d84cdf86229f67ce3487f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "239ae18f3e44498fa52e20e9d6eec6b5"
I20260709 13:24:07.200181 26834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7492e977a5f74ef09aa687b1d7ec89e3" candidate_uuid: "ff90de856dd542eab446899d181dcad0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "326c74b0a8d84cdf86229f67ce3487f2" is_pre_election: true
I20260709 13:24:07.200858 26909 raft_consensus.cc:3060] T 7492e977a5f74ef09aa687b1d7ec89e3 P 239ae18f3e44498fa52e20e9d6eec6b5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:24:07.201257 26834 raft_consensus.cc:2393] T 7492e977a5f74ef09aa687b1d7ec89e3 P 326c74b0a8d84cdf86229f67ce3487f2 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff90de856dd542eab446899d181dcad0 in current term 1: Already voted for candidate 326c74b0a8d84cdf86229f67ce3487f2 in this term.
I20260709 13:24:07.200990 26833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e489a04b06247608a195856b4739a36" candidate_uuid: "ff90de856dd542eab446899d181dcad0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "326c74b0a8d84cdf86229f67ce3487f2" is_pre_election: true
I20260709 13:24:07.201781 26833 raft_consensus.cc:2468] T 3e489a04b06247608a195856b4739a36 P 326c74b0a8d84cdf86229f67ce3487f2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff90de856dd542eab446899d181dcad0 in term 0.
I20260709 13:24:07.202760 26872 leader_election.cc:304] T 92c3adeb279b4f5f88680a40a9664750 P 239ae18f3e44498fa52e20e9d6eec6b5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 239ae18f3e44498fa52e20e9d6eec6b5, ff90de856dd542eab446899d181dcad0; no voters: 
I20260709 13:24:07.202948 26832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ba9365ebe904f85840f6d049dbcda88" candidate_uuid: "ff90de856dd542eab446899d181dcad0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "326c74b0a8d84cdf86229f67ce3487f2" is_pre_election: true
I20260709 13:24:07.205029 26908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e489a04b06247608a195856b4739a36" candidate_uuid: "326c74b0a8d84cdf86229f67ce3487f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "239ae18f3e44498fa52e20e9d6eec6b5" is_pre_election: true
I20260709 13:24:07.206372 26835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d52f26a0449247c1b928641ec8ebad34" candidate_uuid: "ff90de856dd542eab446899d181dcad0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "326c74b0a8d84cdf86229f67ce3487f2" is_pre_election: true
I20260709 13:24:07.207764 26907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d52f26a0449247c1b928641ec8ebad34" candidate_uuid: "ff90de856dd542eab446899d181dcad0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "239ae18f3e44498fa52e20e9d6eec6b5" is_pre_election: true
I20260709 13:24:07.209501 26907 raft_consensus.cc:2468] T d52f26a0449247c1b928641ec8ebad34 P 239ae18f3e44498fa52e20e9d6eec6b5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff90de856dd542eab446899d181dcad0 in term 0.
I20260709 13:24:07.210413 26949 tablet_bootstrap.cc:492] T 9ba9365ebe904f85840f6d049dbcda88 P 239ae18f3e44498fa52e20e9d6eec6b5: No bootstrap required, opened a new log
I20260709 13:24:07.210978 26949 ts_tablet_manager.cc:1403] T 9ba9365ebe904f85840f6d049dbcda88 P 239ae18f3e44498fa52e20e9d6eec6b5: Time spent bootstrapping tablet: real 0.240s	user 0.096s	sys 0.112s
I20260709 13:24:07.211448 26723 leader_election.cc:304] T 3e489a04b06247608a195856b4739a36 P ff90de856dd542eab446899d181dcad0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 326c74b0a8d84cdf86229f67ce3487f2, ff90de856dd542eab446899d181dcad0; no voters: 
I20260709 13:24:07.211925 26905 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e489a04b06247608a195856b4739a36" candidate_uuid: "ff90de856dd542eab446899d181dcad0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "239ae18f3e44498fa52e20e9d6eec6b5" is_pre_election: true
W20260709 13:24:07.212532 26723 leader_election.cc:343] T 9ba9365ebe904f85840f6d049dbcda88 P ff90de856dd542eab446899d181dcad0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 326c74b0a8d84cdf86229f67ce3487f2 (127.14.177.130:35517): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:24:07.207022 26835 raft_consensus.cc:2468] T d52f26a0449247c1b928641ec8ebad34 P 326c74b0a8d84cdf86229f67ce3487f2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff90de856dd542eab446899d181dcad0 in term 0.
I20260709 13:24:07.213222 26904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ba9365ebe904f85840f6d049dbcda88" candidate_uuid: "ff90de856dd542eab446899d181dcad0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "239ae18f3e44498fa52e20e9d6eec6b5" is_pre_election: true
I20260709 13:24:07.214277 26949 raft_consensus.cc:359] T 9ba9365ebe904f85840f6d049dbcda88 P 239ae18f3e44498fa52e20e9d6eec6b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff90de856dd542eab446899d181dcad0" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36311 } } peers { permanent_uuid: "326c74b0a8d84cdf86229f67ce3487f2" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35517 } } peers { permanent_uuid: "239ae18f3e44498fa52e20e9d6eec6b5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 39265 } }
I20260709 13:24:07.194810 26954 leader_election.cc:290] T 3e489a04b06247608a195856b4739a36 P 326c74b0a8d84cdf86229f67ce3487f2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ff90de856dd542eab446899d181dcad0 (127.14.177.129:36311), 239ae18f3e44498fa52e20e9d6eec6b5 (127.14.177.131:39265)
I20260709 13:24:07.215061 26726 leader_election.cc:304] T d52f26a0449247c1b928641ec8ebad34 P ff90de856dd542eab446899d181dcad0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 239ae18f3e44498fa52e20e9d6eec6b5, ff90de856dd542eab446899d181dcad0; no voters: 
I20260709 13:24:07.196534 26956 raft_consensus.cc:515] T 76c716abdcd6437b9dcee43317333298 P ff90de856dd542eab446899d181dcad0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff90de856dd542eab446899d181dcad0" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36311 } } peers { permanent_uuid: "326c74b0a8d84cdf86229f67ce3487f2" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35517 } } peers { permanent_uuid: "239ae18f3e44498fa52e20e9d6eec6b5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 39265 } }
I20260709 13:24:07.214972 26949 raft_consensus.cc:385] T 9ba9365ebe904f85840f6d049dbcda88 P 239ae18f3e44498fa52e20e9d6eec6b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:24:07.220180 26949 raft_consensus.cc:740] T 9ba9365ebe904f85840f6d049dbcda88 P 239ae18f3e44498fa52e20e9d6eec6b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 239ae18f3e44498fa52e20e9d6eec6b5, State: Initialized, Role: FOLLOWER
I20260709 13:24:07.220908 26986 raft_consensus.cc:2804] T d52f26a0449247c1b928641ec8ebad34 P ff90de856dd542eab446899d181dcad0 [term 0 FOLLOWER]: Leader pre-election won for term 1
W20260709 13:24:07.221204 26726 leader_election.cc:343] T 3e489a04b06247608a195856b4739a36 P ff90de856dd542eab446899d181dcad0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 239ae18f3e44498fa52e20e9d6eec6b5 (127.14.177.131:39265): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:24:07.221355 26986 raft_consensus.cc:493] T d52f26a0449247c1b928641ec8ebad34 P ff90de856dd542eab446899d181dcad0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:24:07.221443 26909 raft_consensus.cc:2468] T 7492e977a5f74ef09aa687b1d7ec89e3 P 239ae18f3e44498fa52e20e9d6eec6b5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 326c74b0a8d84cdf86229f67ce3487f2 in term 1.
I20260709 13:24:07.222071 26986 raft_consensus.cc:3060] T d52f26a0449247c1b928641ec8ebad34 P ff90de856dd542eab446899d181dcad0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:24:07.197153 26955 leader_election.cc:290] T 92c3adeb279b4f5f88680a40a9664750 P 239ae18f3e44498fa52e20e9d6eec6b5 [CANDIDATE]: Term 1 election: Requested vote from peers ff90de856dd542eab446899d181dcad0 (127.14.177.129:36311), 326c74b0a8d84cdf86229f67ce3487f2 (127.14.177.130:35517)
W20260709 13:24:07.222724 26726 leader_election.cc:343] T 9ba9365ebe904f85840f6d049dbcda88 P ff90de856dd542eab446899d181dcad0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 239ae18f3e44498fa52e20e9d6eec6b5 (127.14.177.131:39265): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:24:07.232441 26955 raft_consensus.cc:2804] T 92c3adeb279b4f5f88680a40a9664750 P 239ae18f3e44498fa52e20e9d6eec6b5 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:24:07.233033 26955 raft_consensus.cc:697] T 92c3adeb279b4f5f88680a40a9664750 P 239ae18f3e44498fa52e20e9d6eec6b5 [term 1 LEADER]: Becoming Leader. State: Replica: 239ae18f3e44498fa52e20e9d6eec6b5, State: Running, Role: LEADER
I20260709 13:24:07.233999 26955 consensus_queue.cc:237] T 92c3adeb279b4f5f88680a40a9664750 P 239ae18f3e44498fa52e20e9d6eec6b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff90de856dd542eab446899d181dcad0" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36311 } } peers { permanent_uuid: "326c74b0a8d84cdf86229f67ce3487f2" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35517 } } peers { permanent_uuid: "239ae18f3e44498fa52e20e9d6eec6b5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 39265 } }
I20260709 13:24:07.233057 26989 raft_consensus.cc:2804] T 3e489a04b06247608a195856b4739a36 P ff90de856dd542eab446899d181dcad0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:24:07.235672 26968 leader_election.cc:290] T 2664382aca514208a9aa49316bdcaeeb P 239ae18f3e44498fa52e20e9d6eec6b5 [CANDIDATE]: Term 1 election: Requested vote from peers ff90de856dd542eab446899d181dcad0 (127.14.177.129:36311), 326c74b0a8d84cdf86229f67ce3487f2 (127.14.177.130:35517)
I20260709 13:24:07.235893 26989 raft_consensus.cc:493] T 3e489a04b06247608a195856b4739a36 P ff90de856dd542eab446899d181dcad0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:24:07.236312 26989 raft_consensus.cc:3060] T 3e489a04b06247608a195856b4739a36 P ff90de856dd542eab446899d181dcad0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:24:07.216653 26798 leader_election.cc:304] T 3e489a04b06247608a195856b4739a36 P 326c74b0a8d84cdf86229f67ce3487f2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 326c74b0a8d84cdf86229f67ce3487f2, ff90de856dd542eab446899d181dcad0; no voters: 
W20260709 13:24:07.242007 26801 leader_election.cc:343] T 3e489a04b06247608a195856b4739a36 P 326c74b0a8d84cdf86229f67ce3487f2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 239ae18f3e44498fa52e20e9d6eec6b5 (127.14.177.131:39265): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:24:07.242103 26954 raft_consensus.cc:2804] T 3e489a04b06247608a195856b4739a36 P 326c74b0a8d84cdf86229f67ce3487f2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:24:07.242924 26954 raft_consensus.cc:493] T 3e489a04b06247608a195856b4739a36 P 326c74b0a8d84cdf86229f67ce3487f2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:24:07.243418 26954 raft_consensus.cc:3060] T 3e489a04b06247608a195856b4739a36 P 326c74b0a8d84cdf86229f67ce3487f2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:24:07.244342 26956 leader_election.cc:290] T 76c716abdcd6437b9dcee43317333298 P ff90de856dd542eab446899d181dcad0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 326c74b0a8d84cdf86229f67ce3487f2 (127.14.177.130:35517), 239ae18f3e44498fa52e20e9d6eec6b5 (127.14.177.131:39265)
I20260709 13:24:07.211480 26906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7492e977a5f74ef09aa687b1d7ec89e3" candidate_uuid: "ff90de856dd542eab446899d181dcad0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "239ae18f3e44498fa52e20e9d6eec6b5" is_pre_election: true
I20260709 13:24:07.245857 26906 raft_consensus.cc:2393] T 7492e977a5f74ef09aa687b1d7ec89e3 P 239ae18f3e44498fa52e20e9d6eec6b5 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ff90de856dd542eab446899d181dcad0 in current term 1: Already voted for candidate 326c74b0a8d84cdf86229f67ce3487f2 in this term.
I20260709 13:24:07.248739 26835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76c716abdcd6437b9dcee43317333298" candidate_uuid: "ff90de856dd542eab446899d181dcad0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "326c74b0a8d84cdf86229f67ce3487f2" is_pre_election: true
I20260709 13:24:07.250489 26760 raft_consensus.cc:2468] T 7492e977a5f74ef09aa687b1d7ec89e3 P ff90de856dd542eab446899d181dcad0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 326c74b0a8d84cdf86229f67ce3487f2 in term 1.
I20260709 13:24:07.252362 26798 leader_election.cc:304] T 7492e977a5f74ef09aa687b1d7ec89e3 P 326c74b0a8d84cdf86229f67ce3487f2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 326c74b0a8d84cdf86229f67ce3487f2, ff90de856dd542eab446899d181dcad0; no voters: 
I20260709 13:24:07.253419 26987 raft_consensus.cc:2804] T 7492e977a5f74ef09aa687b1d7ec89e3 P 326c74b0a8d84cdf86229f67ce3487f2 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:24:07.253895 26987 raft_consensus.cc:697] T 7492e977a5f74ef09aa687b1d7ec89e3 P 326c74b0a8d84cdf86229f67ce3487f2 [term 1 LEADER]: Becoming Leader. State: Replica: 326c74b0a8d84cdf86229f67ce3487f2, State: Running, Role: LEADER
I20260709 13:24:07.254204 26954 raft_consensus.cc:515] T 3e489a04b06247608a195856b4739a36 P 326c74b0a8d84cdf86229f67ce3487f2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff90de856dd542eab446899d181dcad0" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36311 } } peers { permanent_uuid: "326c74b0a8d84cdf86229f67ce3487f2" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35517 } } peers { permanent_uuid: "239ae18f3e44498fa52e20e9d6eec6b5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 39265 } }
I20260709 13:24:07.254787 26987 consensus_queue.cc:237] T 7492e977a5f74ef09aa687b1d7ec89e3 P 326c74b0a8d84cdf86229f67ce3487f2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff90de856dd542eab446899d181dcad0" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36311 } } peers { permanent_uuid: "326c74b0a8d84cdf86229f67ce3487f2" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35517 } } peers { permanent_uuid: "239ae18f3e44498fa52e20e9d6eec6b5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 39265 } }
I20260709 13:24:07.256269 26988 leader_election.cc:290] T e341897686b14a2491c3a4bd24ac9e95 P 239ae18f3e44498fa52e20e9d6eec6b5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ff90de856dd542eab446899d181dcad0 (127.14.177.129:36311), 326c74b0a8d84cdf86229f67ce3487f2 (127.14.177.130:35517)
I20260709 13:24:07.216454 26990 raft_consensus.cc:2804] T e341897686b14a2491c3a4bd24ac9e95 P 239ae18f3e44498fa52e20e9d6eec6b5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:24:07.258165 26726 leader_election.cc:304] T 9ba9365ebe904f85840f6d049dbcda88 P ff90de856dd542eab446899d181dcad0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff90de856dd542eab446899d181dcad0; no voters: 239ae18f3e44498fa52e20e9d6eec6b5, 326c74b0a8d84cdf86229f67ce3487f2
I20260709 13:24:07.269351 26760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e489a04b06247608a195856b4739a36" candidate_uuid: "326c74b0a8d84cdf86229f67ce3487f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff90de856dd542eab446899d181dcad0"
I20260709 13:24:07.271358 26906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e489a04b06247608a195856b4739a36" candidate_uuid: "326c74b0a8d84cdf86229f67ce3487f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "239ae18f3e44498fa52e20e9d6eec6b5"
I20260709 13:24:07.272300 26948 ts_tablet_manager.cc:1434] T d5f6b403f0cb4c7a9180d07edb98d1bc P 326c74b0a8d84cdf86229f67ce3487f2: Time spent starting tablet: real 0.109s	user 0.019s	sys 0.001s
I20260709 13:24:07.272128 26956 raft_consensus.cc:2749] T 9ba9365ebe904f85840f6d049dbcda88 P ff90de856dd542eab446899d181dcad0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:24:07.273051 26801 leader_election.cc:343] T 3e489a04b06247608a195856b4739a36 P 326c74b0a8d84cdf86229f67ce3487f2 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 239ae18f3e44498fa52e20e9d6eec6b5 (127.14.177.131:39265): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:24:07.269646 26954 leader_election.cc:290] T 3e489a04b06247608a195856b4739a36 P 326c74b0a8d84cdf86229f67ce3487f2 [CANDIDATE]: Term 1 election: Requested vote from peers ff90de856dd542eab446899d181dcad0 (127.14.177.129:36311), 239ae18f3e44498fa52e20e9d6eec6b5 (127.14.177.131:39265)
I20260709 13:24:07.273758 26948 tablet_bootstrap.cc:492] T 92c3adeb279b4f5f88680a40a9664750 P 326c74b0a8d84cdf86229f67ce3487f2: Bootstrap starting.
W20260709 13:24:07.274210 26726 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:39265, user_credentials={real_user=slave}} blocked reactor thread for 51527.9us
I20260709 13:24:07.279619 26906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76c716abdcd6437b9dcee43317333298" candidate_uuid: "ff90de856dd542eab446899d181dcad0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "239ae18f3e44498fa52e20e9d6eec6b5" is_pre_election: true
I20260709 13:24:07.280601 26906 raft_consensus.cc:2468] T 76c716abdcd6437b9dcee43317333298 P 239ae18f3e44498fa52e20e9d6eec6b5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff90de856dd542eab446899d181dcad0 in term 0.
W20260709 13:24:07.258513 26723 leader_election.cc:343] T 76c716abdcd6437b9dcee43317333298 P ff90de856dd542eab446899d181dcad0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 326c74b0a8d84cdf86229f67ce3487f2 (127.14.177.130:35517): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:24:07.261416 26949 consensus_queue.cc:260] T 9ba9365ebe904f85840f6d049dbcda88 P 239ae18f3e44498fa52e20e9d6eec6b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff90de856dd542eab446899d181dcad0" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36311 } } peers { permanent_uuid: "326c74b0a8d84cdf86229f67ce3487f2" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35517 } } peers { permanent_uuid: "239ae18f3e44498fa52e20e9d6eec6b5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 39265 } }
I20260709 13:24:07.259357 26990 raft_consensus.cc:493] T e341897686b14a2491c3a4bd24ac9e95 P 239ae18f3e44498fa52e20e9d6eec6b5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:24:07.283535 26990 raft_consensus.cc:3060] T e341897686b14a2491c3a4bd24ac9e95 P 239ae18f3e44498fa52e20e9d6eec6b5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:24:07.276695 26986 raft_consensus.cc:515] T d52f26a0449247c1b928641ec8ebad34 P ff90de856dd542eab446899d181dcad0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff90de856dd542eab446899d181dcad0" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36311 } } peers { permanent_uuid: "326c74b0a8d84cdf86229f67ce3487f2" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35517 } } peers { permanent_uuid: "239ae18f3e44498fa52e20e9d6eec6b5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 39265 } }
W20260709 13:24:07.261955 26872 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:36311, user_credentials={real_user=slave}} blocked reactor thread for 63713.1us
I20260709 13:24:07.258152 26989 raft_consensus.cc:515] T 3e489a04b06247608a195856b4739a36 P ff90de856dd542eab446899d181dcad0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff90de856dd542eab446899d181dcad0" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36311 } } peers { permanent_uuid: "326c74b0a8d84cdf86229f67ce3487f2" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35517 } } peers { permanent_uuid: "239ae18f3e44498fa52e20e9d6eec6b5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 39265 } }
I20260709 13:24:07.290974 26726 leader_election.cc:304] T 7492e977a5f74ef09aa687b1d7ec89e3 P ff90de856dd542eab446899d181dcad0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff90de856dd542eab446899d181dcad0; no voters: 239ae18f3e44498fa52e20e9d6eec6b5, 326c74b0a8d84cdf86229f67ce3487f2
I20260709 13:24:07.296350 26986 leader_election.cc:290] T d52f26a0449247c1b928641ec8ebad34 P ff90de856dd542eab446899d181dcad0 [CANDIDATE]: Term 1 election: Requested vote from peers 326c74b0a8d84cdf86229f67ce3487f2 (127.14.177.130:35517), 239ae18f3e44498fa52e20e9d6eec6b5 (127.14.177.131:39265)
I20260709 13:24:07.297289 26835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d52f26a0449247c1b928641ec8ebad34" candidate_uuid: "ff90de856dd542eab446899d181dcad0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "326c74b0a8d84cdf86229f67ce3487f2"
I20260709 13:24:07.300149 26760 raft_consensus.cc:2393] T 3e489a04b06247608a195856b4739a36 P ff90de856dd542eab446899d181dcad0 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 326c74b0a8d84cdf86229f67ce3487f2 in current term 1: Already voted for candidate ff90de856dd542eab446899d181dcad0 in this term.
I20260709 13:24:07.300181 26832 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92c3adeb279b4f5f88680a40a9664750" candidate_uuid: "239ae18f3e44498fa52e20e9d6eec6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "326c74b0a8d84cdf86229f67ce3487f2"
I20260709 13:24:07.293958 26956 raft_consensus.cc:2749] T 7492e977a5f74ef09aa687b1d7ec89e3 P ff90de856dd542eab446899d181dcad0 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:24:07.301460 26833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e341897686b14a2491c3a4bd24ac9e95" candidate_uuid: "239ae18f3e44498fa52e20e9d6eec6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "326c74b0a8d84cdf86229f67ce3487f2" is_pre_election: true
I20260709 13:24:07.302227 26949 ts_tablet_manager.cc:1434] T 9ba9365ebe904f85840f6d049dbcda88 P 239ae18f3e44498fa52e20e9d6eec6b5: Time spent starting tablet: real 0.090s	user 0.010s	sys 0.000s
I20260709 13:24:07.303364 26949 tablet_bootstrap.cc:492] T 3e489a04b06247608a195856b4739a36 P 239ae18f3e44498fa52e20e9d6eec6b5: Bootstrap starting.
I20260709 13:24:07.303390 26834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2664382aca514208a9aa49316bdcaeeb" candidate_uuid: "239ae18f3e44498fa52e20e9d6eec6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "326c74b0a8d84cdf86229f67ce3487f2"
I20260709 13:24:07.304152 26834 raft_consensus.cc:3060] T 2664382aca514208a9aa49316bdcaeeb P 326c74b0a8d84cdf86229f67ce3487f2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:24:07.298935 26835 raft_consensus.cc:3060] T d52f26a0449247c1b928641ec8ebad34 P 326c74b0a8d84cdf86229f67ce3487f2 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:24:07.307559 26872 leader_election.cc:343] T 92c3adeb279b4f5f88680a40a9664750 P 239ae18f3e44498fa52e20e9d6eec6b5 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 326c74b0a8d84cdf86229f67ce3487f2 (127.14.177.130:35517): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:24:07.308590 26872 leader_election.cc:343] T e341897686b14a2491c3a4bd24ac9e95 P 239ae18f3e44498fa52e20e9d6eec6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 326c74b0a8d84cdf86229f67ce3487f2 (127.14.177.130:35517): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:24:07.308727 26798 leader_election.cc:304] T 3e489a04b06247608a195856b4739a36 P 326c74b0a8d84cdf86229f67ce3487f2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 326c74b0a8d84cdf86229f67ce3487f2; no voters: 239ae18f3e44498fa52e20e9d6eec6b5, ff90de856dd542eab446899d181dcad0
I20260709 13:24:07.313781 26655 catalog_manager.cc:5697] T 92c3adeb279b4f5f88680a40a9664750 P 239ae18f3e44498fa52e20e9d6eec6b5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 239ae18f3e44498fa52e20e9d6eec6b5 (127.14.177.131). New cstate: current_term: 1 leader_uuid: "239ae18f3e44498fa52e20e9d6eec6b5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff90de856dd542eab446899d181dcad0" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36311 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "326c74b0a8d84cdf86229f67ce3487f2" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35517 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "239ae18f3e44498fa52e20e9d6eec6b5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 39265 } health_report { overall_health: HEALTHY } } }
I20260709 13:24:07.315979 26834 raft_consensus.cc:2468] T 2664382aca514208a9aa49316bdcaeeb P 326c74b0a8d84cdf86229f67ce3487f2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 239ae18f3e44498fa52e20e9d6eec6b5 in term 1.
I20260709 13:24:07.316125 26949 tablet_bootstrap.cc:654] T 3e489a04b06247608a195856b4739a36 P 239ae18f3e44498fa52e20e9d6eec6b5: Neither blocks nor log segments found. Creating new log.
I20260709 13:24:07.317322 26872 leader_election.cc:304] T 2664382aca514208a9aa49316bdcaeeb P 239ae18f3e44498fa52e20e9d6eec6b5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 239ae18f3e44498fa52e20e9d6eec6b5, 326c74b0a8d84cdf86229f67ce3487f2; no voters: 
I20260709 13:24:07.312361 26657 catalog_manager.cc:5697] T 7492e977a5f74ef09aa687b1d7ec89e3 P 326c74b0a8d84cdf86229f67ce3487f2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 326c74b0a8d84cdf86229f67ce3487f2 (127.14.177.130). New cstate: current_term: 1 leader_uuid: "326c74b0a8d84cdf86229f67ce3487f2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff90de856dd542eab446899d181dcad0" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36311 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "326c74b0a8d84cdf86229f67ce3487f2" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35517 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "239ae18f3e44498fa52e20e9d6eec6b5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 39265 } health_report { overall_health: UNKNOWN } } }
I20260709 13:24:07.319713 26992 raft_consensus.cc:2804] T 2664382aca514208a9aa49316bdcaeeb P 239ae18f3e44498fa52e20e9d6eec6b5 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:24:07.320201 26987 raft_consensus.cc:2749] T 3e489a04b06247608a195856b4739a36 P 326c74b0a8d84cdf86229f67ce3487f2 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:24:07.319491 26833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e489a04b06247608a195856b4739a36" candidate_uuid: "ff90de856dd542eab446899d181dcad0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "326c74b0a8d84cdf86229f67ce3487f2"
I20260709 13:24:07.317816 26989 leader_election.cc:290] T 3e489a04b06247608a195856b4739a36 P ff90de856dd542eab446899d181dcad0 [CANDIDATE]: Term 1 election: Requested vote from peers 326c74b0a8d84cdf86229f67ce3487f2 (127.14.177.130:35517), 239ae18f3e44498fa52e20e9d6eec6b5 (127.14.177.131:39265)
I20260709 13:24:07.325484 26992 raft_consensus.cc:697] T 2664382aca514208a9aa49316bdcaeeb P 239ae18f3e44498fa52e20e9d6eec6b5 [term 1 LEADER]: Becoming Leader. State: Replica: 239ae18f3e44498fa52e20e9d6eec6b5, State: Running, Role: LEADER
I20260709 13:24:07.326403 26833 raft_consensus.cc:2393] T 3e489a04b06247608a195856b4739a36 P 326c74b0a8d84cdf86229f67ce3487f2 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate ff90de856dd542eab446899d181dcad0 in current term 1: Already voted for candidate 326c74b0a8d84cdf86229f67ce3487f2 in this term.
I20260709 13:24:07.326743 26992 consensus_queue.cc:237] T 2664382aca514208a9aa49316bdcaeeb P 239ae18f3e44498fa52e20e9d6eec6b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff90de856dd542eab446899d181dcad0" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36311 } } peers { permanent_uuid: "326c74b0a8d84cdf86229f67ce3487f2" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35517 } } peers { permanent_uuid: "239ae18f3e44498fa52e20e9d6eec6b5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 39265 } }
I20260709 13:24:07.331274 26948 tablet_bootstrap.cc:654] T 92c3adeb279b4f5f88680a40a9664750 P 326c74b0a8d84cdf86229f67ce3487f2: Neither blocks nor log segments found. Creating new log.
I20260709 13:24:07.332717 26990 raft_consensus.cc:515] T e341897686b14a2491c3a4bd24ac9e95 P 239ae18f3e44498fa52e20e9d6eec6b5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff90de856dd542eab446899d181dcad0" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36311 } } peers { permanent_uuid: "326c74b0a8d84cdf86229f67ce3487f2" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35517 } } peers { permanent_uuid: "239ae18f3e44498fa52e20e9d6eec6b5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 39265 } }
I20260709 13:24:07.326841 26835 raft_consensus.cc:2468] T d52f26a0449247c1b928641ec8ebad34 P 326c74b0a8d84cdf86229f67ce3487f2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff90de856dd542eab446899d181dcad0 in term 1.
I20260709 13:24:07.341212 26723 leader_election.cc:304] T d52f26a0449247c1b928641ec8ebad34 P ff90de856dd542eab446899d181dcad0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 326c74b0a8d84cdf86229f67ce3487f2, ff90de856dd542eab446899d181dcad0; no voters: 
I20260709 13:24:07.323580 26726 leader_election.cc:304] T 76c716abdcd6437b9dcee43317333298 P ff90de856dd542eab446899d181dcad0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 239ae18f3e44498fa52e20e9d6eec6b5, ff90de856dd542eab446899d181dcad0; no voters: 326c74b0a8d84cdf86229f67ce3487f2
I20260709 13:24:07.342952 26950 tablet_bootstrap.cc:492] T bdb51cee932741eda8c12a88f07c890e P ff90de856dd542eab446899d181dcad0: No bootstrap required, opened a new log
I20260709 13:24:07.343804 26956 raft_consensus.cc:2804] T 76c716abdcd6437b9dcee43317333298 P ff90de856dd542eab446899d181dcad0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:24:07.343935 26950 ts_tablet_manager.cc:1403] T bdb51cee932741eda8c12a88f07c890e P ff90de856dd542eab446899d181dcad0: Time spent bootstrapping tablet: real 0.226s	user 0.023s	sys 0.033s
I20260709 13:24:07.344363 26956 raft_consensus.cc:493] T 76c716abdcd6437b9dcee43317333298 P ff90de856dd542eab446899d181dcad0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:24:07.344844 26956 raft_consensus.cc:3060] T 76c716abdcd6437b9dcee43317333298 P ff90de856dd542eab446899d181dcad0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:24:07.343402 26989 raft_consensus.cc:2804] T d52f26a0449247c1b928641ec8ebad34 P ff90de856dd542eab446899d181dcad0 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:24:07.347999 26989 raft_consensus.cc:697] T d52f26a0449247c1b928641ec8ebad34 P ff90de856dd542eab446899d181dcad0 [term 1 LEADER]: Becoming Leader. State: Replica: ff90de856dd542eab446899d181dcad0, State: Running, Role: LEADER
I20260709 13:24:07.348855 26949 tablet_bootstrap.cc:492] T 3e489a04b06247608a195856b4739a36 P 239ae18f3e44498fa52e20e9d6eec6b5: No bootstrap required, opened a new log
I20260709 13:24:07.349380 26949 ts_tablet_manager.cc:1403] T 3e489a04b06247608a195856b4739a36 P 239ae18f3e44498fa52e20e9d6eec6b5: Time spent bootstrapping tablet: real 0.046s	user 0.014s	sys 0.018s
I20260709 13:24:07.348866 26989 consensus_queue.cc:237] T d52f26a0449247c1b928641ec8ebad34 P ff90de856dd542eab446899d181dcad0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff90de856dd542eab446899d181dcad0" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36311 } } peers { permanent_uuid: "326c74b0a8d84cdf86229f67ce3487f2" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35517 } } peers { permanent_uuid: "239ae18f3e44498fa52e20e9d6eec6b5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 39265 } }
I20260709 13:24:07.350109 26950 raft_consensus.cc:359] T bdb51cee932741eda8c12a88f07c890e P ff90de856dd542eab446899d181dcad0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff90de856dd542eab446899d181dcad0" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36311 } } peers { permanent_uuid: "326c74b0a8d84cdf86229f67ce3487f2" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35517 } } peers { permanent_uuid: "239ae18f3e44498fa52e20e9d6eec6b5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 39265 } }
I20260709 13:24:07.350792 26950 raft_consensus.cc:385] T bdb51cee932741eda8c12a88f07c890e P ff90de856dd542eab446899d181dcad0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:24:07.351046 26950 raft_consensus.cc:740] T bdb51cee932741eda8c12a88f07c890e P ff90de856dd542eab446899d181dcad0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ff90de856dd542eab446899d181dcad0, State: Initialized, Role: FOLLOWER
I20260709 13:24:07.351840 26950 consensus_queue.cc:260] T bdb51cee932741eda8c12a88f07c890e P ff90de856dd542eab446899d181dcad0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff90de856dd542eab446899d181dcad0" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36311 } } peers { permanent_uuid: "326c74b0a8d84cdf86229f67ce3487f2" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35517 } } peers { permanent_uuid: "239ae18f3e44498fa52e20e9d6eec6b5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 39265 } }
I20260709 13:24:07.352089 26949 raft_consensus.cc:359] T 3e489a04b06247608a195856b4739a36 P 239ae18f3e44498fa52e20e9d6eec6b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff90de856dd542eab446899d181dcad0" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36311 } } peers { permanent_uuid: "326c74b0a8d84cdf86229f67ce3487f2" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35517 } } peers { permanent_uuid: "239ae18f3e44498fa52e20e9d6eec6b5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 39265 } }
I20260709 13:24:07.352340 26990 leader_election.cc:290] T e341897686b14a2491c3a4bd24ac9e95 P 239ae18f3e44498fa52e20e9d6eec6b5 [CANDIDATE]: Term 1 election: Requested vote from peers ff90de856dd542eab446899d181dcad0 (127.14.177.129:36311), 326c74b0a8d84cdf86229f67ce3487f2 (127.14.177.130:35517)
I20260709 13:24:07.352607 26835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e341897686b14a2491c3a4bd24ac9e95" candidate_uuid: "239ae18f3e44498fa52e20e9d6eec6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "326c74b0a8d84cdf86229f67ce3487f2"
I20260709 13:24:07.352754 26949 raft_consensus.cc:385] T 3e489a04b06247608a195856b4739a36 P 239ae18f3e44498fa52e20e9d6eec6b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260709 13:24:07.354362 26872 leader_election.cc:343] T e341897686b14a2491c3a4bd24ac9e95 P 239ae18f3e44498fa52e20e9d6eec6b5 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 326c74b0a8d84cdf86229f67ce3487f2 (127.14.177.130:35517): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:24:07.354631 26950 ts_tablet_manager.cc:1434] T bdb51cee932741eda8c12a88f07c890e P ff90de856dd542eab446899d181dcad0: Time spent starting tablet: real 0.010s	user 0.005s	sys 0.000s
I20260709 13:24:07.354823 26949 raft_consensus.cc:740] T 3e489a04b06247608a195856b4739a36 P 239ae18f3e44498fa52e20e9d6eec6b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 239ae18f3e44498fa52e20e9d6eec6b5, State: Initialized, Role: FOLLOWER
I20260709 13:24:07.356324 26949 consensus_queue.cc:260] T 3e489a04b06247608a195856b4739a36 P 239ae18f3e44498fa52e20e9d6eec6b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff90de856dd542eab446899d181dcad0" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36311 } } peers { permanent_uuid: "326c74b0a8d84cdf86229f67ce3487f2" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35517 } } peers { permanent_uuid: "239ae18f3e44498fa52e20e9d6eec6b5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 39265 } }
I20260709 13:24:07.358608 26949 ts_tablet_manager.cc:1434] T 3e489a04b06247608a195856b4739a36 P 239ae18f3e44498fa52e20e9d6eec6b5: Time spent starting tablet: real 0.009s	user 0.004s	sys 0.004s
I20260709 13:24:07.359654 26949 tablet_bootstrap.cc:492] T bdb51cee932741eda8c12a88f07c890e P 239ae18f3e44498fa52e20e9d6eec6b5: Bootstrap starting.
I20260709 13:24:07.361277 26906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d52f26a0449247c1b928641ec8ebad34" candidate_uuid: "ff90de856dd542eab446899d181dcad0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "239ae18f3e44498fa52e20e9d6eec6b5"
I20260709 13:24:07.362278 26906 raft_consensus.cc:3060] T d52f26a0449247c1b928641ec8ebad34 P 239ae18f3e44498fa52e20e9d6eec6b5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:24:07.365584 26949 tablet_bootstrap.cc:654] T bdb51cee932741eda8c12a88f07c890e P 239ae18f3e44498fa52e20e9d6eec6b5: Neither blocks nor log segments found. Creating new log.
I20260709 13:24:07.366451 26909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e489a04b06247608a195856b4739a36" candidate_uuid: "ff90de856dd542eab446899d181dcad0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "239ae18f3e44498fa52e20e9d6eec6b5"
I20260709 13:24:07.367349 26909 raft_consensus.cc:3060] T 3e489a04b06247608a195856b4739a36 P 239ae18f3e44498fa52e20e9d6eec6b5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:24:07.370378 26986 raft_consensus.cc:493] T d5f6b403f0cb4c7a9180d07edb98d1bc P ff90de856dd542eab446899d181dcad0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:24:07.371595 26992 raft_consensus.cc:493] T 9ba9365ebe904f85840f6d049dbcda88 P 239ae18f3e44498fa52e20e9d6eec6b5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:24:07.371274 26986 raft_consensus.cc:515] T d5f6b403f0cb4c7a9180d07edb98d1bc P ff90de856dd542eab446899d181dcad0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "239ae18f3e44498fa52e20e9d6eec6b5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 39265 } } peers { permanent_uuid: "326c74b0a8d84cdf86229f67ce3487f2" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35517 } } peers { permanent_uuid: "ff90de856dd542eab446899d181dcad0" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36311 } }
I20260709 13:24:07.372165 26992 raft_consensus.cc:515] T 9ba9365ebe904f85840f6d049dbcda88 P 239ae18f3e44498fa52e20e9d6eec6b5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff90de856dd542eab446899d181dcad0" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36311 } } peers { permanent_uuid: "326c74b0a8d84cdf86229f67ce3487f2" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35517 } } peers { permanent_uuid: "239ae18f3e44498fa52e20e9d6eec6b5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 39265 } }
I20260709 13:24:07.361480 26760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2664382aca514208a9aa49316bdcaeeb" candidate_uuid: "239ae18f3e44498fa52e20e9d6eec6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff90de856dd542eab446899d181dcad0"
I20260709 13:24:07.376399 26986 leader_election.cc:290] T d5f6b403f0cb4c7a9180d07edb98d1bc P ff90de856dd542eab446899d181dcad0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 239ae18f3e44498fa52e20e9d6eec6b5 (127.14.177.131:39265), 326c74b0a8d84cdf86229f67ce3487f2 (127.14.177.130:35517)
I20260709 13:24:07.379545 26835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5f6b403f0cb4c7a9180d07edb98d1bc" candidate_uuid: "ff90de856dd542eab446899d181dcad0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "326c74b0a8d84cdf86229f67ce3487f2" is_pre_election: true
I20260709 13:24:07.380031 26833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ba9365ebe904f85840f6d049dbcda88" candidate_uuid: "239ae18f3e44498fa52e20e9d6eec6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "326c74b0a8d84cdf86229f67ce3487f2" is_pre_election: true
I20260709 13:24:07.381758 26992 leader_election.cc:290] T 9ba9365ebe904f85840f6d049dbcda88 P 239ae18f3e44498fa52e20e9d6eec6b5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ff90de856dd542eab446899d181dcad0 (127.14.177.129:36311), 326c74b0a8d84cdf86229f67ce3487f2 (127.14.177.130:35517)
I20260709 13:24:07.381139 26760 raft_consensus.cc:3060] T 2664382aca514208a9aa49316bdcaeeb P ff90de856dd542eab446899d181dcad0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:24:07.382005 26759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e341897686b14a2491c3a4bd24ac9e95" candidate_uuid: "239ae18f3e44498fa52e20e9d6eec6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff90de856dd542eab446899d181dcad0"
I20260709 13:24:07.384116 26759 raft_consensus.cc:3060] T e341897686b14a2491c3a4bd24ac9e95 P ff90de856dd542eab446899d181dcad0 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:24:07.386569 26872 leader_election.cc:343] T 9ba9365ebe904f85840f6d049dbcda88 P 239ae18f3e44498fa52e20e9d6eec6b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 326c74b0a8d84cdf86229f67ce3487f2 (127.14.177.130:35517): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:24:07.380633 26835 raft_consensus.cc:2468] T d5f6b403f0cb4c7a9180d07edb98d1bc P 326c74b0a8d84cdf86229f67ce3487f2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff90de856dd542eab446899d181dcad0 in term 0.
I20260709 13:24:07.388625 26723 leader_election.cc:304] T d5f6b403f0cb4c7a9180d07edb98d1bc P ff90de856dd542eab446899d181dcad0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 326c74b0a8d84cdf86229f67ce3487f2, ff90de856dd542eab446899d181dcad0; no voters: 
I20260709 13:24:07.389622 26986 raft_consensus.cc:2804] T d5f6b403f0cb4c7a9180d07edb98d1bc P ff90de856dd542eab446899d181dcad0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:24:07.390065 26986 raft_consensus.cc:493] T d5f6b403f0cb4c7a9180d07edb98d1bc P ff90de856dd542eab446899d181dcad0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:24:07.390447 26986 raft_consensus.cc:3060] T d5f6b403f0cb4c7a9180d07edb98d1bc P ff90de856dd542eab446899d181dcad0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:24:07.396770 26760 raft_consensus.cc:2468] T 2664382aca514208a9aa49316bdcaeeb P ff90de856dd542eab446899d181dcad0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 239ae18f3e44498fa52e20e9d6eec6b5 in term 1.
I20260709 13:24:07.398818 26986 raft_consensus.cc:515] T d5f6b403f0cb4c7a9180d07edb98d1bc P ff90de856dd542eab446899d181dcad0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "239ae18f3e44498fa52e20e9d6eec6b5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 39265 } } peers { permanent_uuid: "326c74b0a8d84cdf86229f67ce3487f2" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35517 } } peers { permanent_uuid: "ff90de856dd542eab446899d181dcad0" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36311 } }
I20260709 13:24:07.399199 26906 raft_consensus.cc:2468] T d52f26a0449247c1b928641ec8ebad34 P 239ae18f3e44498fa52e20e9d6eec6b5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff90de856dd542eab446899d181dcad0 in term 1.
I20260709 13:24:07.400776 26904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5f6b403f0cb4c7a9180d07edb98d1bc" candidate_uuid: "ff90de856dd542eab446899d181dcad0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "239ae18f3e44498fa52e20e9d6eec6b5" is_pre_election: true
I20260709 13:24:07.402724 26835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5f6b403f0cb4c7a9180d07edb98d1bc" candidate_uuid: "ff90de856dd542eab446899d181dcad0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "326c74b0a8d84cdf86229f67ce3487f2"
I20260709 13:24:07.403439 26835 raft_consensus.cc:3060] T d5f6b403f0cb4c7a9180d07edb98d1bc P 326c74b0a8d84cdf86229f67ce3487f2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:24:07.404366 26904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5f6b403f0cb4c7a9180d07edb98d1bc" candidate_uuid: "ff90de856dd542eab446899d181dcad0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "239ae18f3e44498fa52e20e9d6eec6b5"
I20260709 13:24:07.403215 26760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ba9365ebe904f85840f6d049dbcda88" candidate_uuid: "239ae18f3e44498fa52e20e9d6eec6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff90de856dd542eab446899d181dcad0" is_pre_election: true
W20260709 13:24:07.404604 26726 leader_election.cc:343] T d5f6b403f0cb4c7a9180d07edb98d1bc P ff90de856dd542eab446899d181dcad0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 239ae18f3e44498fa52e20e9d6eec6b5 (127.14.177.131:39265): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:24:07.407425 26726 leader_election.cc:343] T d5f6b403f0cb4c7a9180d07edb98d1bc P ff90de856dd542eab446899d181dcad0 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 239ae18f3e44498fa52e20e9d6eec6b5 (127.14.177.131:39265): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:24:07.413065 26986 leader_election.cc:290] T d5f6b403f0cb4c7a9180d07edb98d1bc P ff90de856dd542eab446899d181dcad0 [CANDIDATE]: Term 1 election: Requested vote from peers 239ae18f3e44498fa52e20e9d6eec6b5 (127.14.177.131:39265), 326c74b0a8d84cdf86229f67ce3487f2 (127.14.177.130:35517)
I20260709 13:24:07.415432 26948 tablet_bootstrap.cc:492] T 92c3adeb279b4f5f88680a40a9664750 P 326c74b0a8d84cdf86229f67ce3487f2: No bootstrap required, opened a new log
I20260709 13:24:07.414490 26835 raft_consensus.cc:2468] T d5f6b403f0cb4c7a9180d07edb98d1bc P 326c74b0a8d84cdf86229f67ce3487f2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff90de856dd542eab446899d181dcad0 in term 1.
I20260709 13:24:07.416195 26948 ts_tablet_manager.cc:1403] T 92c3adeb279b4f5f88680a40a9664750 P 326c74b0a8d84cdf86229f67ce3487f2: Time spent bootstrapping tablet: real 0.143s	user 0.018s	sys 0.002s
I20260709 13:24:07.416910 26723 leader_election.cc:304] T d5f6b403f0cb4c7a9180d07edb98d1bc P ff90de856dd542eab446899d181dcad0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 326c74b0a8d84cdf86229f67ce3487f2, ff90de856dd542eab446899d181dcad0; no voters: 239ae18f3e44498fa52e20e9d6eec6b5
I20260709 13:24:07.418720 26986 raft_consensus.cc:2804] T d5f6b403f0cb4c7a9180d07edb98d1bc P ff90de856dd542eab446899d181dcad0 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:24:07.419951 26949 tablet_bootstrap.cc:492] T bdb51cee932741eda8c12a88f07c890e P 239ae18f3e44498fa52e20e9d6eec6b5: No bootstrap required, opened a new log
I20260709 13:24:07.419927 26956 raft_consensus.cc:515] T 76c716abdcd6437b9dcee43317333298 P ff90de856dd542eab446899d181dcad0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff90de856dd542eab446899d181dcad0" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36311 } } peers { permanent_uuid: "326c74b0a8d84cdf86229f67ce3487f2" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35517 } } peers { permanent_uuid: "239ae18f3e44498fa52e20e9d6eec6b5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 39265 } }
I20260709 13:24:07.420620 26949 ts_tablet_manager.cc:1403] T bdb51cee932741eda8c12a88f07c890e P 239ae18f3e44498fa52e20e9d6eec6b5: Time spent bootstrapping tablet: real 0.061s	user 0.024s	sys 0.024s
I20260709 13:24:07.422318 26956 leader_election.cc:290] T 76c716abdcd6437b9dcee43317333298 P ff90de856dd542eab446899d181dcad0 [CANDIDATE]: Term 1 election: Requested vote from peers 326c74b0a8d84cdf86229f67ce3487f2 (127.14.177.130:35517), 239ae18f3e44498fa52e20e9d6eec6b5 (127.14.177.131:39265)
I20260709 13:24:07.423615 26835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76c716abdcd6437b9dcee43317333298" candidate_uuid: "ff90de856dd542eab446899d181dcad0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "326c74b0a8d84cdf86229f67ce3487f2"
I20260709 13:24:07.423568 26949 raft_consensus.cc:359] T bdb51cee932741eda8c12a88f07c890e P 239ae18f3e44498fa52e20e9d6eec6b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff90de856dd542eab446899d181dcad0" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36311 } } peers { permanent_uuid: "326c74b0a8d84cdf86229f67ce3487f2" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35517 } } peers { permanent_uuid: "239ae18f3e44498fa52e20e9d6eec6b5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 39265 } }
I20260709 13:24:07.424273 26904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76c716abdcd6437b9dcee43317333298" candidate_uuid: "ff90de856dd542eab446899d181dcad0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "239ae18f3e44498fa52e20e9d6eec6b5"
I20260709 13:24:07.424619 26949 raft_consensus.cc:385] T bdb51cee932741eda8c12a88f07c890e P 239ae18f3e44498fa52e20e9d6eec6b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:24:07.424913 26904 raft_consensus.cc:3060] T 76c716abdcd6437b9dcee43317333298 P 239ae18f3e44498fa52e20e9d6eec6b5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:24:07.424940 26949 raft_consensus.cc:740] T bdb51cee932741eda8c12a88f07c890e P 239ae18f3e44498fa52e20e9d6eec6b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 239ae18f3e44498fa52e20e9d6eec6b5, State: Initialized, Role: FOLLOWER
W20260709 13:24:07.425318 26723 leader_election.cc:343] T 76c716abdcd6437b9dcee43317333298 P ff90de856dd542eab446899d181dcad0 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 326c74b0a8d84cdf86229f67ce3487f2 (127.14.177.130:35517): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:24:07.426007 26949 consensus_queue.cc:260] T bdb51cee932741eda8c12a88f07c890e P 239ae18f3e44498fa52e20e9d6eec6b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff90de856dd542eab446899d181dcad0" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36311 } } peers { permanent_uuid: "326c74b0a8d84cdf86229f67ce3487f2" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35517 } } peers { permanent_uuid: "239ae18f3e44498fa52e20e9d6eec6b5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 39265 } }
I20260709 13:24:07.426785 26759 raft_consensus.cc:2468] T e341897686b14a2491c3a4bd24ac9e95 P ff90de856dd542eab446899d181dcad0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 239ae18f3e44498fa52e20e9d6eec6b5 in term 1.
I20260709 13:24:07.428337 26949 ts_tablet_manager.cc:1434] T bdb51cee932741eda8c12a88f07c890e P 239ae18f3e44498fa52e20e9d6eec6b5: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 13:24:07.432991 26949 tablet_bootstrap.cc:492] T d5f6b403f0cb4c7a9180d07edb98d1bc P 239ae18f3e44498fa52e20e9d6eec6b5: Bootstrap starting.
I20260709 13:24:07.406131 26760 raft_consensus.cc:2468] T 9ba9365ebe904f85840f6d049dbcda88 P ff90de856dd542eab446899d181dcad0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 239ae18f3e44498fa52e20e9d6eec6b5 in term 0.
I20260709 13:24:07.435735 26872 leader_election.cc:304] T e341897686b14a2491c3a4bd24ac9e95 P 239ae18f3e44498fa52e20e9d6eec6b5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 239ae18f3e44498fa52e20e9d6eec6b5, ff90de856dd542eab446899d181dcad0; no voters: 326c74b0a8d84cdf86229f67ce3487f2
I20260709 13:24:07.437772 26872 leader_election.cc:304] T 9ba9365ebe904f85840f6d049dbcda88 P 239ae18f3e44498fa52e20e9d6eec6b5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 239ae18f3e44498fa52e20e9d6eec6b5, ff90de856dd542eab446899d181dcad0; no voters: 326c74b0a8d84cdf86229f67ce3487f2
I20260709 13:24:07.438470 26949 tablet_bootstrap.cc:654] T d5f6b403f0cb4c7a9180d07edb98d1bc P 239ae18f3e44498fa52e20e9d6eec6b5: Neither blocks nor log segments found. Creating new log.
I20260709 13:24:07.440894 26990 raft_consensus.cc:2804] T e341897686b14a2491c3a4bd24ac9e95 P 239ae18f3e44498fa52e20e9d6eec6b5 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:24:07.442699 26990 raft_consensus.cc:697] T e341897686b14a2491c3a4bd24ac9e95 P 239ae18f3e44498fa52e20e9d6eec6b5 [term 1 LEADER]: Becoming Leader. State: Replica: 239ae18f3e44498fa52e20e9d6eec6b5, State: Running, Role: LEADER
I20260709 13:24:07.443677 26990 consensus_queue.cc:237] T e341897686b14a2491c3a4bd24ac9e95 P 239ae18f3e44498fa52e20e9d6eec6b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff90de856dd542eab446899d181dcad0" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36311 } } peers { permanent_uuid: "326c74b0a8d84cdf86229f67ce3487f2" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35517 } } peers { permanent_uuid: "239ae18f3e44498fa52e20e9d6eec6b5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 39265 } }
I20260709 13:24:07.441560 26992 raft_consensus.cc:2804] T 9ba9365ebe904f85840f6d049dbcda88 P 239ae18f3e44498fa52e20e9d6eec6b5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:24:07.432471 26909 raft_consensus.cc:2468] T 3e489a04b06247608a195856b4739a36 P 239ae18f3e44498fa52e20e9d6eec6b5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff90de856dd542eab446899d181dcad0 in term 1.
I20260709 13:24:07.450429 26992 raft_consensus.cc:493] T 9ba9365ebe904f85840f6d049dbcda88 P 239ae18f3e44498fa52e20e9d6eec6b5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:24:07.449982 26986 raft_consensus.cc:697] T d5f6b403f0cb4c7a9180d07edb98d1bc P ff90de856dd542eab446899d181dcad0 [term 1 LEADER]: Becoming Leader. State: Replica: ff90de856dd542eab446899d181dcad0, State: Running, Role: LEADER
I20260709 13:24:07.453179 26986 consensus_queue.cc:237] T d5f6b403f0cb4c7a9180d07edb98d1bc P ff90de856dd542eab446899d181dcad0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "239ae18f3e44498fa52e20e9d6eec6b5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 39265 } } peers { permanent_uuid: "326c74b0a8d84cdf86229f67ce3487f2" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35517 } } peers { permanent_uuid: "ff90de856dd542eab446899d181dcad0" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36311 } }
I20260709 13:24:07.451261 26992 raft_consensus.cc:3060] T 9ba9365ebe904f85840f6d049dbcda88 P 239ae18f3e44498fa52e20e9d6eec6b5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:24:07.455713 26726 leader_election.cc:304] T 3e489a04b06247608a195856b4739a36 P ff90de856dd542eab446899d181dcad0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 239ae18f3e44498fa52e20e9d6eec6b5, ff90de856dd542eab446899d181dcad0; no voters: 326c74b0a8d84cdf86229f67ce3487f2
I20260709 13:24:07.457508 27005 raft_consensus.cc:2804] T 3e489a04b06247608a195856b4739a36 P ff90de856dd542eab446899d181dcad0 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:24:07.458047 27005 raft_consensus.cc:697] T 3e489a04b06247608a195856b4739a36 P ff90de856dd542eab446899d181dcad0 [term 1 LEADER]: Becoming Leader. State: Replica: ff90de856dd542eab446899d181dcad0, State: Running, Role: LEADER
I20260709 13:24:07.458894 27005 consensus_queue.cc:237] T 3e489a04b06247608a195856b4739a36 P ff90de856dd542eab446899d181dcad0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff90de856dd542eab446899d181dcad0" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36311 } } peers { permanent_uuid: "326c74b0a8d84cdf86229f67ce3487f2" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35517 } } peers { permanent_uuid: "239ae18f3e44498fa52e20e9d6eec6b5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 39265 } }
I20260709 13:24:07.459584 26948 raft_consensus.cc:359] T 92c3adeb279b4f5f88680a40a9664750 P 326c74b0a8d84cdf86229f67ce3487f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff90de856dd542eab446899d181dcad0" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36311 } } peers { permanent_uuid: "326c74b0a8d84cdf86229f67ce3487f2" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35517 } } peers { permanent_uuid: "239ae18f3e44498fa52e20e9d6eec6b5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 39265 } }
I20260709 13:24:07.460408 26948 raft_consensus.cc:385] T 92c3adeb279b4f5f88680a40a9664750 P 326c74b0a8d84cdf86229f67ce3487f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:24:07.460705 26948 raft_consensus.cc:740] T 92c3adeb279b4f5f88680a40a9664750 P 326c74b0a8d84cdf86229f67ce3487f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 326c74b0a8d84cdf86229f67ce3487f2, State: Initialized, Role: FOLLOWER
I20260709 13:24:07.461483 26948 consensus_queue.cc:260] T 92c3adeb279b4f5f88680a40a9664750 P 326c74b0a8d84cdf86229f67ce3487f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff90de856dd542eab446899d181dcad0" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36311 } } peers { permanent_uuid: "326c74b0a8d84cdf86229f67ce3487f2" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35517 } } peers { permanent_uuid: "239ae18f3e44498fa52e20e9d6eec6b5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 39265 } }
I20260709 13:24:07.451460 26988 raft_consensus.cc:493] T bdb51cee932741eda8c12a88f07c890e P 239ae18f3e44498fa52e20e9d6eec6b5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:24:07.463555 26988 raft_consensus.cc:515] T bdb51cee932741eda8c12a88f07c890e P 239ae18f3e44498fa52e20e9d6eec6b5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff90de856dd542eab446899d181dcad0" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36311 } } peers { permanent_uuid: "326c74b0a8d84cdf86229f67ce3487f2" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35517 } } peers { permanent_uuid: "239ae18f3e44498fa52e20e9d6eec6b5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 39265 } }
I20260709 13:24:07.465953 26988 leader_election.cc:290] T bdb51cee932741eda8c12a88f07c890e P 239ae18f3e44498fa52e20e9d6eec6b5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ff90de856dd542eab446899d181dcad0 (127.14.177.129:36311), 326c74b0a8d84cdf86229f67ce3487f2 (127.14.177.130:35517)
I20260709 13:24:07.466603 26760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdb51cee932741eda8c12a88f07c890e" candidate_uuid: "239ae18f3e44498fa52e20e9d6eec6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff90de856dd542eab446899d181dcad0" is_pre_election: true
I20260709 13:24:07.467327 26760 raft_consensus.cc:2468] T bdb51cee932741eda8c12a88f07c890e P ff90de856dd542eab446899d181dcad0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 239ae18f3e44498fa52e20e9d6eec6b5 in term 0.
I20260709 13:24:07.467967 26948 ts_tablet_manager.cc:1434] T 92c3adeb279b4f5f88680a40a9664750 P 326c74b0a8d84cdf86229f67ce3487f2: Time spent starting tablet: real 0.051s	user 0.008s	sys 0.000s
I20260709 13:24:07.469739 26872 leader_election.cc:304] T bdb51cee932741eda8c12a88f07c890e P 239ae18f3e44498fa52e20e9d6eec6b5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 239ae18f3e44498fa52e20e9d6eec6b5, ff90de856dd542eab446899d181dcad0; no voters: 
I20260709 13:24:07.465029 26904 raft_consensus.cc:2468] T 76c716abdcd6437b9dcee43317333298 P 239ae18f3e44498fa52e20e9d6eec6b5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff90de856dd542eab446899d181dcad0 in term 1.
I20260709 13:24:07.470759 26988 raft_consensus.cc:2804] T bdb51cee932741eda8c12a88f07c890e P 239ae18f3e44498fa52e20e9d6eec6b5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:24:07.471206 26988 raft_consensus.cc:493] T bdb51cee932741eda8c12a88f07c890e P 239ae18f3e44498fa52e20e9d6eec6b5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:24:07.471577 26988 raft_consensus.cc:3060] T bdb51cee932741eda8c12a88f07c890e P 239ae18f3e44498fa52e20e9d6eec6b5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:24:07.472308 26992 raft_consensus.cc:515] T 9ba9365ebe904f85840f6d049dbcda88 P 239ae18f3e44498fa52e20e9d6eec6b5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff90de856dd542eab446899d181dcad0" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36311 } } peers { permanent_uuid: "326c74b0a8d84cdf86229f67ce3487f2" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35517 } } peers { permanent_uuid: "239ae18f3e44498fa52e20e9d6eec6b5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 39265 } }
I20260709 13:24:07.472538 26655 catalog_manager.cc:5697] T d52f26a0449247c1b928641ec8ebad34 P ff90de856dd542eab446899d181dcad0 reported cstate change: term changed from 0 to 1, leader changed from <none> to ff90de856dd542eab446899d181dcad0 (127.14.177.129). New cstate: current_term: 1 leader_uuid: "ff90de856dd542eab446899d181dcad0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff90de856dd542eab446899d181dcad0" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36311 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "326c74b0a8d84cdf86229f67ce3487f2" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35517 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "239ae18f3e44498fa52e20e9d6eec6b5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 39265 } health_report { overall_health: UNKNOWN } } }
I20260709 13:24:07.481564 26949 tablet_bootstrap.cc:492] T d5f6b403f0cb4c7a9180d07edb98d1bc P 239ae18f3e44498fa52e20e9d6eec6b5: No bootstrap required, opened a new log
I20260709 13:24:07.481110 26988 raft_consensus.cc:515] T bdb51cee932741eda8c12a88f07c890e P 239ae18f3e44498fa52e20e9d6eec6b5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff90de856dd542eab446899d181dcad0" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36311 } } peers { permanent_uuid: "326c74b0a8d84cdf86229f67ce3487f2" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35517 } } peers { permanent_uuid: "239ae18f3e44498fa52e20e9d6eec6b5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 39265 } }
I20260709 13:24:07.482007 26992 leader_election.cc:290] T 9ba9365ebe904f85840f6d049dbcda88 P 239ae18f3e44498fa52e20e9d6eec6b5 [CANDIDATE]: Term 1 election: Requested vote from peers ff90de856dd542eab446899d181dcad0 (127.14.177.129:36311), 326c74b0a8d84cdf86229f67ce3487f2 (127.14.177.130:35517)
I20260709 13:24:07.482312 26949 ts_tablet_manager.cc:1403] T d5f6b403f0cb4c7a9180d07edb98d1bc P 239ae18f3e44498fa52e20e9d6eec6b5: Time spent bootstrapping tablet: real 0.050s	user 0.017s	sys 0.023s
I20260709 13:24:07.484274 26988 leader_election.cc:290] T bdb51cee932741eda8c12a88f07c890e P 239ae18f3e44498fa52e20e9d6eec6b5 [CANDIDATE]: Term 1 election: Requested vote from peers ff90de856dd542eab446899d181dcad0 (127.14.177.129:36311), 326c74b0a8d84cdf86229f67ce3487f2 (127.14.177.130:35517)
I20260709 13:24:07.485337 26949 raft_consensus.cc:359] T d5f6b403f0cb4c7a9180d07edb98d1bc P 239ae18f3e44498fa52e20e9d6eec6b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "239ae18f3e44498fa52e20e9d6eec6b5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 39265 } } peers { permanent_uuid: "326c74b0a8d84cdf86229f67ce3487f2" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35517 } } peers { permanent_uuid: "ff90de856dd542eab446899d181dcad0" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36311 } }
I20260709 13:24:07.486356 26759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdb51cee932741eda8c12a88f07c890e" candidate_uuid: "239ae18f3e44498fa52e20e9d6eec6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff90de856dd542eab446899d181dcad0"
I20260709 13:24:07.487092 26759 raft_consensus.cc:3060] T bdb51cee932741eda8c12a88f07c890e P ff90de856dd542eab446899d181dcad0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:24:07.486302 26726 leader_election.cc:304] T 76c716abdcd6437b9dcee43317333298 P ff90de856dd542eab446899d181dcad0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 239ae18f3e44498fa52e20e9d6eec6b5, ff90de856dd542eab446899d181dcad0; no voters: 326c74b0a8d84cdf86229f67ce3487f2
I20260709 13:24:07.486112 26760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ba9365ebe904f85840f6d049dbcda88" candidate_uuid: "239ae18f3e44498fa52e20e9d6eec6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff90de856dd542eab446899d181dcad0"
I20260709 13:24:07.491425 26760 raft_consensus.cc:3060] T 9ba9365ebe904f85840f6d049dbcda88 P ff90de856dd542eab446899d181dcad0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:24:07.494974 26759 raft_consensus.cc:2468] T bdb51cee932741eda8c12a88f07c890e P ff90de856dd542eab446899d181dcad0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 239ae18f3e44498fa52e20e9d6eec6b5 in term 1.
I20260709 13:24:07.495901 26949 raft_consensus.cc:385] T d5f6b403f0cb4c7a9180d07edb98d1bc P 239ae18f3e44498fa52e20e9d6eec6b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:24:07.496340 26949 raft_consensus.cc:740] T d5f6b403f0cb4c7a9180d07edb98d1bc P 239ae18f3e44498fa52e20e9d6eec6b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 239ae18f3e44498fa52e20e9d6eec6b5, State: Initialized, Role: FOLLOWER
I20260709 13:24:07.496294 26872 leader_election.cc:304] T bdb51cee932741eda8c12a88f07c890e P 239ae18f3e44498fa52e20e9d6eec6b5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 239ae18f3e44498fa52e20e9d6eec6b5, ff90de856dd542eab446899d181dcad0; no voters: 
I20260709 13:24:07.497458 26949 consensus_queue.cc:260] T d5f6b403f0cb4c7a9180d07edb98d1bc P 239ae18f3e44498fa52e20e9d6eec6b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "239ae18f3e44498fa52e20e9d6eec6b5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 39265 } } peers { permanent_uuid: "326c74b0a8d84cdf86229f67ce3487f2" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35517 } } peers { permanent_uuid: "ff90de856dd542eab446899d181dcad0" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36311 } }
I20260709 13:24:07.498070 26989 raft_consensus.cc:2804] T 76c716abdcd6437b9dcee43317333298 P ff90de856dd542eab446899d181dcad0 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:24:07.498353 26988 raft_consensus.cc:2804] T bdb51cee932741eda8c12a88f07c890e P 239ae18f3e44498fa52e20e9d6eec6b5 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:24:07.498631 26989 raft_consensus.cc:697] T 76c716abdcd6437b9dcee43317333298 P ff90de856dd542eab446899d181dcad0 [term 1 LEADER]: Becoming Leader. State: Replica: ff90de856dd542eab446899d181dcad0, State: Running, Role: LEADER
I20260709 13:24:07.498976 26988 raft_consensus.cc:697] T bdb51cee932741eda8c12a88f07c890e P 239ae18f3e44498fa52e20e9d6eec6b5 [term 1 LEADER]: Becoming Leader. State: Replica: 239ae18f3e44498fa52e20e9d6eec6b5, State: Running, Role: LEADER
I20260709 13:24:07.499591 26949 ts_tablet_manager.cc:1434] T d5f6b403f0cb4c7a9180d07edb98d1bc P 239ae18f3e44498fa52e20e9d6eec6b5: Time spent starting tablet: real 0.017s	user 0.006s	sys 0.000s
I20260709 13:24:07.499886 26988 consensus_queue.cc:237] T bdb51cee932741eda8c12a88f07c890e P 239ae18f3e44498fa52e20e9d6eec6b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff90de856dd542eab446899d181dcad0" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36311 } } peers { permanent_uuid: "326c74b0a8d84cdf86229f67ce3487f2" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35517 } } peers { permanent_uuid: "239ae18f3e44498fa52e20e9d6eec6b5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 39265 } }
I20260709 13:24:07.500360 26835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdb51cee932741eda8c12a88f07c890e" candidate_uuid: "239ae18f3e44498fa52e20e9d6eec6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "326c74b0a8d84cdf86229f67ce3487f2" is_pre_election: true
I20260709 13:24:07.500989 26835 raft_consensus.cc:2468] T bdb51cee932741eda8c12a88f07c890e P 326c74b0a8d84cdf86229f67ce3487f2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 239ae18f3e44498fa52e20e9d6eec6b5 in term 0.
I20260709 13:24:07.501799 26833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ba9365ebe904f85840f6d049dbcda88" candidate_uuid: "239ae18f3e44498fa52e20e9d6eec6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "326c74b0a8d84cdf86229f67ce3487f2"
I20260709 13:24:07.499574 26989 consensus_queue.cc:237] T 76c716abdcd6437b9dcee43317333298 P ff90de856dd542eab446899d181dcad0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff90de856dd542eab446899d181dcad0" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36311 } } peers { permanent_uuid: "326c74b0a8d84cdf86229f67ce3487f2" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35517 } } peers { permanent_uuid: "239ae18f3e44498fa52e20e9d6eec6b5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 39265 } }
I20260709 13:24:07.502744 26834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdb51cee932741eda8c12a88f07c890e" candidate_uuid: "239ae18f3e44498fa52e20e9d6eec6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "326c74b0a8d84cdf86229f67ce3487f2"
I20260709 13:24:07.503404 26834 raft_consensus.cc:3060] T bdb51cee932741eda8c12a88f07c890e P 326c74b0a8d84cdf86229f67ce3487f2 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:24:07.509017 26872 leader_election.cc:343] T 9ba9365ebe904f85840f6d049dbcda88 P 239ae18f3e44498fa52e20e9d6eec6b5 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 326c74b0a8d84cdf86229f67ce3487f2 (127.14.177.130:35517): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:24:07.508178 26658 catalog_manager.cc:5697] T 2664382aca514208a9aa49316bdcaeeb P 239ae18f3e44498fa52e20e9d6eec6b5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 239ae18f3e44498fa52e20e9d6eec6b5 (127.14.177.131). New cstate: current_term: 1 leader_uuid: "239ae18f3e44498fa52e20e9d6eec6b5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff90de856dd542eab446899d181dcad0" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36311 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "326c74b0a8d84cdf86229f67ce3487f2" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35517 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "239ae18f3e44498fa52e20e9d6eec6b5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 39265 } health_report { overall_health: HEALTHY } } }
I20260709 13:24:07.515020 26948 tablet_bootstrap.cc:492] T 9ba9365ebe904f85840f6d049dbcda88 P 326c74b0a8d84cdf86229f67ce3487f2: Bootstrap starting.
I20260709 13:24:07.518362 26760 raft_consensus.cc:2468] T 9ba9365ebe904f85840f6d049dbcda88 P ff90de856dd542eab446899d181dcad0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 239ae18f3e44498fa52e20e9d6eec6b5 in term 1.
I20260709 13:24:07.524451 26872 leader_election.cc:304] T 9ba9365ebe904f85840f6d049dbcda88 P 239ae18f3e44498fa52e20e9d6eec6b5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 239ae18f3e44498fa52e20e9d6eec6b5, ff90de856dd542eab446899d181dcad0; no voters: 326c74b0a8d84cdf86229f67ce3487f2
I20260709 13:24:07.525224 26834 raft_consensus.cc:2468] T bdb51cee932741eda8c12a88f07c890e P 326c74b0a8d84cdf86229f67ce3487f2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 239ae18f3e44498fa52e20e9d6eec6b5 in term 1.
I20260709 13:24:07.525604 26948 tablet_bootstrap.cc:654] T 9ba9365ebe904f85840f6d049dbcda88 P 326c74b0a8d84cdf86229f67ce3487f2: Neither blocks nor log segments found. Creating new log.
I20260709 13:24:07.525763 26990 raft_consensus.cc:2804] T 9ba9365ebe904f85840f6d049dbcda88 P 239ae18f3e44498fa52e20e9d6eec6b5 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:24:07.526675 26990 raft_consensus.cc:697] T 9ba9365ebe904f85840f6d049dbcda88 P 239ae18f3e44498fa52e20e9d6eec6b5 [term 1 LEADER]: Becoming Leader. State: Replica: 239ae18f3e44498fa52e20e9d6eec6b5, State: Running, Role: LEADER
I20260709 13:24:07.527720 26990 consensus_queue.cc:237] T 9ba9365ebe904f85840f6d049dbcda88 P 239ae18f3e44498fa52e20e9d6eec6b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff90de856dd542eab446899d181dcad0" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36311 } } peers { permanent_uuid: "326c74b0a8d84cdf86229f67ce3487f2" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35517 } } peers { permanent_uuid: "239ae18f3e44498fa52e20e9d6eec6b5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 39265 } }
I20260709 13:24:07.532136 26948 tablet_bootstrap.cc:492] T 9ba9365ebe904f85840f6d049dbcda88 P 326c74b0a8d84cdf86229f67ce3487f2: No bootstrap required, opened a new log
I20260709 13:24:07.532738 26948 ts_tablet_manager.cc:1403] T 9ba9365ebe904f85840f6d049dbcda88 P 326c74b0a8d84cdf86229f67ce3487f2: Time spent bootstrapping tablet: real 0.018s	user 0.004s	sys 0.008s
I20260709 13:24:07.532562 26655 catalog_manager.cc:5697] T 3e489a04b06247608a195856b4739a36 P ff90de856dd542eab446899d181dcad0 reported cstate change: term changed from 0 to 1, leader changed from <none> to ff90de856dd542eab446899d181dcad0 (127.14.177.129). New cstate: current_term: 1 leader_uuid: "ff90de856dd542eab446899d181dcad0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff90de856dd542eab446899d181dcad0" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36311 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "326c74b0a8d84cdf86229f67ce3487f2" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35517 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "239ae18f3e44498fa52e20e9d6eec6b5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 39265 } health_report { overall_health: UNKNOWN } } }
I20260709 13:24:07.534207 26655 catalog_manager.cc:5697] T 76c716abdcd6437b9dcee43317333298 P ff90de856dd542eab446899d181dcad0 reported cstate change: term changed from 0 to 1, leader changed from <none> to ff90de856dd542eab446899d181dcad0 (127.14.177.129). New cstate: current_term: 1 leader_uuid: "ff90de856dd542eab446899d181dcad0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff90de856dd542eab446899d181dcad0" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36311 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "326c74b0a8d84cdf86229f67ce3487f2" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35517 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "239ae18f3e44498fa52e20e9d6eec6b5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 39265 } health_report { overall_health: UNKNOWN } } }
I20260709 13:24:07.535883 26655 catalog_manager.cc:5697] T d5f6b403f0cb4c7a9180d07edb98d1bc P ff90de856dd542eab446899d181dcad0 reported cstate change: term changed from 0 to 1, leader changed from <none> to ff90de856dd542eab446899d181dcad0 (127.14.177.129). New cstate: current_term: 1 leader_uuid: "ff90de856dd542eab446899d181dcad0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "239ae18f3e44498fa52e20e9d6eec6b5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 39265 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "326c74b0a8d84cdf86229f67ce3487f2" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35517 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ff90de856dd542eab446899d181dcad0" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36311 } health_report { overall_health: HEALTHY } } }
I20260709 13:24:07.538427 26948 raft_consensus.cc:359] T 9ba9365ebe904f85840f6d049dbcda88 P 326c74b0a8d84cdf86229f67ce3487f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff90de856dd542eab446899d181dcad0" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36311 } } peers { permanent_uuid: "326c74b0a8d84cdf86229f67ce3487f2" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35517 } } peers { permanent_uuid: "239ae18f3e44498fa52e20e9d6eec6b5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 39265 } }
I20260709 13:24:07.539108 26948 raft_consensus.cc:385] T 9ba9365ebe904f85840f6d049dbcda88 P 326c74b0a8d84cdf86229f67ce3487f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:24:07.539393 26948 raft_consensus.cc:740] T 9ba9365ebe904f85840f6d049dbcda88 P 326c74b0a8d84cdf86229f67ce3487f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 326c74b0a8d84cdf86229f67ce3487f2, State: Initialized, Role: FOLLOWER
I20260709 13:24:07.540081 26948 consensus_queue.cc:260] T 9ba9365ebe904f85840f6d049dbcda88 P 326c74b0a8d84cdf86229f67ce3487f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff90de856dd542eab446899d181dcad0" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36311 } } peers { permanent_uuid: "326c74b0a8d84cdf86229f67ce3487f2" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35517 } } peers { permanent_uuid: "239ae18f3e44498fa52e20e9d6eec6b5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 39265 } }
I20260709 13:24:07.542075 26948 ts_tablet_manager.cc:1434] T 9ba9365ebe904f85840f6d049dbcda88 P 326c74b0a8d84cdf86229f67ce3487f2: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.000s
I20260709 13:24:07.542915 26948 tablet_bootstrap.cc:492] T 76c716abdcd6437b9dcee43317333298 P 326c74b0a8d84cdf86229f67ce3487f2: Bootstrap starting.
I20260709 13:24:07.548264 26658 catalog_manager.cc:5697] T bdb51cee932741eda8c12a88f07c890e P 239ae18f3e44498fa52e20e9d6eec6b5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 239ae18f3e44498fa52e20e9d6eec6b5 (127.14.177.131). New cstate: current_term: 1 leader_uuid: "239ae18f3e44498fa52e20e9d6eec6b5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff90de856dd542eab446899d181dcad0" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36311 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "326c74b0a8d84cdf86229f67ce3487f2" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35517 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "239ae18f3e44498fa52e20e9d6eec6b5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 39265 } health_report { overall_health: HEALTHY } } }
I20260709 13:24:07.550154 26948 tablet_bootstrap.cc:654] T 76c716abdcd6437b9dcee43317333298 P 326c74b0a8d84cdf86229f67ce3487f2: Neither blocks nor log segments found. Creating new log.
I20260709 13:24:07.550215 26658 catalog_manager.cc:5697] T 9ba9365ebe904f85840f6d049dbcda88 P 239ae18f3e44498fa52e20e9d6eec6b5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 239ae18f3e44498fa52e20e9d6eec6b5 (127.14.177.131). New cstate: current_term: 1 leader_uuid: "239ae18f3e44498fa52e20e9d6eec6b5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff90de856dd542eab446899d181dcad0" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36311 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "326c74b0a8d84cdf86229f67ce3487f2" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35517 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "239ae18f3e44498fa52e20e9d6eec6b5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 39265 } health_report { overall_health: HEALTHY } } }
I20260709 13:24:07.551715 26658 catalog_manager.cc:5697] T e341897686b14a2491c3a4bd24ac9e95 P 239ae18f3e44498fa52e20e9d6eec6b5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 239ae18f3e44498fa52e20e9d6eec6b5 (127.14.177.131). New cstate: current_term: 1 leader_uuid: "239ae18f3e44498fa52e20e9d6eec6b5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff90de856dd542eab446899d181dcad0" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36311 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "326c74b0a8d84cdf86229f67ce3487f2" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35517 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "239ae18f3e44498fa52e20e9d6eec6b5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 39265 } health_report { overall_health: HEALTHY } } }
I20260709 13:24:07.555025 26948 tablet_bootstrap.cc:492] T 76c716abdcd6437b9dcee43317333298 P 326c74b0a8d84cdf86229f67ce3487f2: No bootstrap required, opened a new log
I20260709 13:24:07.555611 26948 ts_tablet_manager.cc:1403] T 76c716abdcd6437b9dcee43317333298 P 326c74b0a8d84cdf86229f67ce3487f2: Time spent bootstrapping tablet: real 0.013s	user 0.008s	sys 0.003s
I20260709 13:24:07.558130 26948 raft_consensus.cc:359] T 76c716abdcd6437b9dcee43317333298 P 326c74b0a8d84cdf86229f67ce3487f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff90de856dd542eab446899d181dcad0" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36311 } } peers { permanent_uuid: "326c74b0a8d84cdf86229f67ce3487f2" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35517 } } peers { permanent_uuid: "239ae18f3e44498fa52e20e9d6eec6b5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 39265 } }
I20260709 13:24:07.558677 26948 raft_consensus.cc:385] T 76c716abdcd6437b9dcee43317333298 P 326c74b0a8d84cdf86229f67ce3487f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:24:07.558941 26948 raft_consensus.cc:740] T 76c716abdcd6437b9dcee43317333298 P 326c74b0a8d84cdf86229f67ce3487f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 326c74b0a8d84cdf86229f67ce3487f2, State: Initialized, Role: FOLLOWER
I20260709 13:24:07.559577 26948 consensus_queue.cc:260] T 76c716abdcd6437b9dcee43317333298 P 326c74b0a8d84cdf86229f67ce3487f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff90de856dd542eab446899d181dcad0" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36311 } } peers { permanent_uuid: "326c74b0a8d84cdf86229f67ce3487f2" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35517 } } peers { permanent_uuid: "239ae18f3e44498fa52e20e9d6eec6b5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 39265 } }
I20260709 13:24:07.561364 26948 ts_tablet_manager.cc:1434] T 76c716abdcd6437b9dcee43317333298 P 326c74b0a8d84cdf86229f67ce3487f2: Time spent starting tablet: real 0.005s	user 0.001s	sys 0.003s
I20260709 13:24:07.562263 26948 tablet_bootstrap.cc:492] T e341897686b14a2491c3a4bd24ac9e95 P 326c74b0a8d84cdf86229f67ce3487f2: Bootstrap starting.
I20260709 13:24:07.566586 26948 tablet_bootstrap.cc:654] T e341897686b14a2491c3a4bd24ac9e95 P 326c74b0a8d84cdf86229f67ce3487f2: Neither blocks nor log segments found. Creating new log.
I20260709 13:24:07.570922 26948 tablet_bootstrap.cc:492] T e341897686b14a2491c3a4bd24ac9e95 P 326c74b0a8d84cdf86229f67ce3487f2: No bootstrap required, opened a new log
I20260709 13:24:07.571393 26948 ts_tablet_manager.cc:1403] T e341897686b14a2491c3a4bd24ac9e95 P 326c74b0a8d84cdf86229f67ce3487f2: Time spent bootstrapping tablet: real 0.009s	user 0.008s	sys 0.000s
I20260709 13:24:07.573520 26948 raft_consensus.cc:359] T e341897686b14a2491c3a4bd24ac9e95 P 326c74b0a8d84cdf86229f67ce3487f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff90de856dd542eab446899d181dcad0" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36311 } } peers { permanent_uuid: "326c74b0a8d84cdf86229f67ce3487f2" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35517 } } peers { permanent_uuid: "239ae18f3e44498fa52e20e9d6eec6b5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 39265 } }
I20260709 13:24:07.574057 26948 raft_consensus.cc:385] T e341897686b14a2491c3a4bd24ac9e95 P 326c74b0a8d84cdf86229f67ce3487f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:24:07.574306 26948 raft_consensus.cc:740] T e341897686b14a2491c3a4bd24ac9e95 P 326c74b0a8d84cdf86229f67ce3487f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 326c74b0a8d84cdf86229f67ce3487f2, State: Initialized, Role: FOLLOWER
I20260709 13:24:07.574832 26948 consensus_queue.cc:260] T e341897686b14a2491c3a4bd24ac9e95 P 326c74b0a8d84cdf86229f67ce3487f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff90de856dd542eab446899d181dcad0" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36311 } } peers { permanent_uuid: "326c74b0a8d84cdf86229f67ce3487f2" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35517 } } peers { permanent_uuid: "239ae18f3e44498fa52e20e9d6eec6b5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 39265 } }
I20260709 13:24:07.576368 26948 ts_tablet_manager.cc:1434] T e341897686b14a2491c3a4bd24ac9e95 P 326c74b0a8d84cdf86229f67ce3487f2: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.000s
I20260709 13:24:07.670732 26990 raft_consensus.cc:493] T d5f6b403f0cb4c7a9180d07edb98d1bc P 239ae18f3e44498fa52e20e9d6eec6b5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:24:07.671597 26990 raft_consensus.cc:515] T d5f6b403f0cb4c7a9180d07edb98d1bc P 239ae18f3e44498fa52e20e9d6eec6b5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "239ae18f3e44498fa52e20e9d6eec6b5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 39265 } } peers { permanent_uuid: "326c74b0a8d84cdf86229f67ce3487f2" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35517 } } peers { permanent_uuid: "ff90de856dd542eab446899d181dcad0" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36311 } }
I20260709 13:24:07.673682 26990 leader_election.cc:290] T d5f6b403f0cb4c7a9180d07edb98d1bc P 239ae18f3e44498fa52e20e9d6eec6b5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 326c74b0a8d84cdf86229f67ce3487f2 (127.14.177.130:35517), ff90de856dd542eab446899d181dcad0 (127.14.177.129:36311)
I20260709 13:24:07.674631 26834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5f6b403f0cb4c7a9180d07edb98d1bc" candidate_uuid: "239ae18f3e44498fa52e20e9d6eec6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "326c74b0a8d84cdf86229f67ce3487f2" is_pre_election: true
I20260709 13:24:07.674906 26760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5f6b403f0cb4c7a9180d07edb98d1bc" candidate_uuid: "239ae18f3e44498fa52e20e9d6eec6b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff90de856dd542eab446899d181dcad0" is_pre_election: true
I20260709 13:24:07.675523 26834 raft_consensus.cc:2393] T d5f6b403f0cb4c7a9180d07edb98d1bc P 326c74b0a8d84cdf86229f67ce3487f2 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 239ae18f3e44498fa52e20e9d6eec6b5 in current term 1: Already voted for candidate ff90de856dd542eab446899d181dcad0 in this term.
I20260709 13:24:07.676709 26872 leader_election.cc:304] T d5f6b403f0cb4c7a9180d07edb98d1bc P 239ae18f3e44498fa52e20e9d6eec6b5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 239ae18f3e44498fa52e20e9d6eec6b5; no voters: 326c74b0a8d84cdf86229f67ce3487f2, ff90de856dd542eab446899d181dcad0
I20260709 13:24:07.677333 26990 raft_consensus.cc:3060] T d5f6b403f0cb4c7a9180d07edb98d1bc P 239ae18f3e44498fa52e20e9d6eec6b5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:24:07.683321 26990 raft_consensus.cc:2749] T d5f6b403f0cb4c7a9180d07edb98d1bc P 239ae18f3e44498fa52e20e9d6eec6b5 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:24:07.694222 26987 raft_consensus.cc:493] T 76c716abdcd6437b9dcee43317333298 P 326c74b0a8d84cdf86229f67ce3487f2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:24:07.694846 26987 raft_consensus.cc:515] T 76c716abdcd6437b9dcee43317333298 P 326c74b0a8d84cdf86229f67ce3487f2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff90de856dd542eab446899d181dcad0" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36311 } } peers { permanent_uuid: "326c74b0a8d84cdf86229f67ce3487f2" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35517 } } peers { permanent_uuid: "239ae18f3e44498fa52e20e9d6eec6b5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 39265 } }
I20260709 13:24:07.696671 26987 leader_election.cc:290] T 76c716abdcd6437b9dcee43317333298 P 326c74b0a8d84cdf86229f67ce3487f2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ff90de856dd542eab446899d181dcad0 (127.14.177.129:36311), 239ae18f3e44498fa52e20e9d6eec6b5 (127.14.177.131:39265)
I20260709 13:24:07.697535 26760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76c716abdcd6437b9dcee43317333298" candidate_uuid: "326c74b0a8d84cdf86229f67ce3487f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff90de856dd542eab446899d181dcad0" is_pre_election: true
I20260709 13:24:07.697799 26904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76c716abdcd6437b9dcee43317333298" candidate_uuid: "326c74b0a8d84cdf86229f67ce3487f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "239ae18f3e44498fa52e20e9d6eec6b5" is_pre_election: true
I20260709 13:24:07.698653 26904 raft_consensus.cc:2393] T 76c716abdcd6437b9dcee43317333298 P 239ae18f3e44498fa52e20e9d6eec6b5 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 326c74b0a8d84cdf86229f67ce3487f2 in current term 1: Already voted for candidate ff90de856dd542eab446899d181dcad0 in this term.
I20260709 13:24:07.699522 26801 leader_election.cc:304] T 76c716abdcd6437b9dcee43317333298 P 326c74b0a8d84cdf86229f67ce3487f2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 326c74b0a8d84cdf86229f67ce3487f2; no voters: 239ae18f3e44498fa52e20e9d6eec6b5, ff90de856dd542eab446899d181dcad0
I20260709 13:24:07.700161 26987 raft_consensus.cc:3060] T 76c716abdcd6437b9dcee43317333298 P 326c74b0a8d84cdf86229f67ce3487f2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:24:07.706022 26987 raft_consensus.cc:2749] T 76c716abdcd6437b9dcee43317333298 P 326c74b0a8d84cdf86229f67ce3487f2 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:24:07.732165 26987 raft_consensus.cc:493] T 92c3adeb279b4f5f88680a40a9664750 P 326c74b0a8d84cdf86229f67ce3487f2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:24:07.732595 26987 raft_consensus.cc:515] T 92c3adeb279b4f5f88680a40a9664750 P 326c74b0a8d84cdf86229f67ce3487f2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff90de856dd542eab446899d181dcad0" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36311 } } peers { permanent_uuid: "326c74b0a8d84cdf86229f67ce3487f2" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35517 } } peers { permanent_uuid: "239ae18f3e44498fa52e20e9d6eec6b5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 39265 } }
I20260709 13:24:07.734969 26987 leader_election.cc:290] T 92c3adeb279b4f5f88680a40a9664750 P 326c74b0a8d84cdf86229f67ce3487f2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ff90de856dd542eab446899d181dcad0 (127.14.177.129:36311), 239ae18f3e44498fa52e20e9d6eec6b5 (127.14.177.131:39265)
I20260709 13:24:07.735445 26760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92c3adeb279b4f5f88680a40a9664750" candidate_uuid: "326c74b0a8d84cdf86229f67ce3487f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff90de856dd542eab446899d181dcad0" is_pre_election: true
I20260709 13:24:07.735584 26904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92c3adeb279b4f5f88680a40a9664750" candidate_uuid: "326c74b0a8d84cdf86229f67ce3487f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "239ae18f3e44498fa52e20e9d6eec6b5" is_pre_election: true
I20260709 13:24:07.736230 26760 raft_consensus.cc:2393] T 92c3adeb279b4f5f88680a40a9664750 P ff90de856dd542eab446899d181dcad0 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 326c74b0a8d84cdf86229f67ce3487f2 in current term 1: Already voted for candidate 239ae18f3e44498fa52e20e9d6eec6b5 in this term.
I20260709 13:24:07.737440 26798 leader_election.cc:304] T 92c3adeb279b4f5f88680a40a9664750 P 326c74b0a8d84cdf86229f67ce3487f2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 326c74b0a8d84cdf86229f67ce3487f2; no voters: 239ae18f3e44498fa52e20e9d6eec6b5, ff90de856dd542eab446899d181dcad0
I20260709 13:24:07.738224 26987 raft_consensus.cc:3060] T 92c3adeb279b4f5f88680a40a9664750 P 326c74b0a8d84cdf86229f67ce3487f2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:24:07.743188 26987 raft_consensus.cc:2749] T 92c3adeb279b4f5f88680a40a9664750 P 326c74b0a8d84cdf86229f67ce3487f2 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:24:07.773573 26990 consensus_queue.cc:1048] T 92c3adeb279b4f5f88680a40a9664750 P 239ae18f3e44498fa52e20e9d6eec6b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff90de856dd542eab446899d181dcad0" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36311 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:24:07.785159 26988 consensus_queue.cc:1048] T 92c3adeb279b4f5f88680a40a9664750 P 239ae18f3e44498fa52e20e9d6eec6b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "326c74b0a8d84cdf86229f67ce3487f2" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35517 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:24:07.798348 15046 ts_manager.cc:295] Set tserver state for ff90de856dd542eab446899d181dcad0 to MAINTENANCE_MODE
I20260709 13:24:07.799317 15046 tablet_server.cc:179] TabletServer@127.14.177.129:0 shutting down...
W20260709 13:24:07.804437 26706 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
W20260709 13:24:07.817109 26798 consensus_peers.cc:597] T 7492e977a5f74ef09aa687b1d7ec89e3 P 326c74b0a8d84cdf86229f67ce3487f2 -> Peer ff90de856dd542eab446899d181dcad0 (127.14.177.129:36311): Couldn't send request to peer ff90de856dd542eab446899d181dcad0. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:24:07.823884 15046 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:24:07.824949 15046 tablet_replica.cc:333] T bdb51cee932741eda8c12a88f07c890e P ff90de856dd542eab446899d181dcad0: stopping tablet replica
I20260709 13:24:07.825868 15046 raft_consensus.cc:2243] T bdb51cee932741eda8c12a88f07c890e P ff90de856dd542eab446899d181dcad0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:24:07.826238 15046 raft_consensus.cc:2272] T bdb51cee932741eda8c12a88f07c890e P ff90de856dd542eab446899d181dcad0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:24:07.828490 15046 tablet_replica.cc:333] T 2664382aca514208a9aa49316bdcaeeb P ff90de856dd542eab446899d181dcad0: stopping tablet replica
I20260709 13:24:07.829025 15046 raft_consensus.cc:2243] T 2664382aca514208a9aa49316bdcaeeb P ff90de856dd542eab446899d181dcad0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:24:07.829340 15046 raft_consensus.cc:2272] T 2664382aca514208a9aa49316bdcaeeb P ff90de856dd542eab446899d181dcad0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:24:07.831013 15046 tablet_replica.cc:333] T 92c3adeb279b4f5f88680a40a9664750 P ff90de856dd542eab446899d181dcad0: stopping tablet replica
I20260709 13:24:07.831583 15046 raft_consensus.cc:2243] T 92c3adeb279b4f5f88680a40a9664750 P ff90de856dd542eab446899d181dcad0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:24:07.831935 15046 raft_consensus.cc:2272] T 92c3adeb279b4f5f88680a40a9664750 P ff90de856dd542eab446899d181dcad0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:24:07.834036 15046 tablet_replica.cc:333] T 9ba9365ebe904f85840f6d049dbcda88 P ff90de856dd542eab446899d181dcad0: stopping tablet replica
I20260709 13:24:07.834528 15046 raft_consensus.cc:2243] T 9ba9365ebe904f85840f6d049dbcda88 P ff90de856dd542eab446899d181dcad0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:24:07.834951 15046 raft_consensus.cc:2272] T 9ba9365ebe904f85840f6d049dbcda88 P ff90de856dd542eab446899d181dcad0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:24:07.837183 15046 tablet_replica.cc:333] T 76c716abdcd6437b9dcee43317333298 P ff90de856dd542eab446899d181dcad0: stopping tablet replica
I20260709 13:24:07.837945 15046 raft_consensus.cc:2243] T 76c716abdcd6437b9dcee43317333298 P ff90de856dd542eab446899d181dcad0 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:24:07.838707 15046 pending_rounds.cc:70] T 76c716abdcd6437b9dcee43317333298 P ff90de856dd542eab446899d181dcad0: Trying to abort 1 pending ops.
I20260709 13:24:07.838863 15046 pending_rounds.cc:77] T 76c716abdcd6437b9dcee43317333298 P ff90de856dd542eab446899d181dcad0: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7305639720976703488 op_type: NO_OP noop_request { }
I20260709 13:24:07.839109 15046 raft_consensus.cc:2889] T 76c716abdcd6437b9dcee43317333298 P ff90de856dd542eab446899d181dcad0 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:24:07.839411 15046 raft_consensus.cc:2272] T 76c716abdcd6437b9dcee43317333298 P ff90de856dd542eab446899d181dcad0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:24:07.841428 15046 tablet_replica.cc:333] T d52f26a0449247c1b928641ec8ebad34 P ff90de856dd542eab446899d181dcad0: stopping tablet replica
I20260709 13:24:07.841928 15046 raft_consensus.cc:2243] T d52f26a0449247c1b928641ec8ebad34 P ff90de856dd542eab446899d181dcad0 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:24:07.842453 15046 pending_rounds.cc:70] T d52f26a0449247c1b928641ec8ebad34 P ff90de856dd542eab446899d181dcad0: Trying to abort 1 pending ops.
I20260709 13:24:07.842617 15046 pending_rounds.cc:77] T d52f26a0449247c1b928641ec8ebad34 P ff90de856dd542eab446899d181dcad0: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7305639720353730560 op_type: NO_OP noop_request { }
I20260709 13:24:07.842837 15046 raft_consensus.cc:2889] T d52f26a0449247c1b928641ec8ebad34 P ff90de856dd542eab446899d181dcad0 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:24:07.843058 15046 raft_consensus.cc:2272] T d52f26a0449247c1b928641ec8ebad34 P ff90de856dd542eab446899d181dcad0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:24:07.845475 15046 tablet_replica.cc:333] T e341897686b14a2491c3a4bd24ac9e95 P ff90de856dd542eab446899d181dcad0: stopping tablet replica
I20260709 13:24:07.846016 15046 raft_consensus.cc:2243] T e341897686b14a2491c3a4bd24ac9e95 P ff90de856dd542eab446899d181dcad0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:24:07.846374 15046 raft_consensus.cc:2272] T e341897686b14a2491c3a4bd24ac9e95 P ff90de856dd542eab446899d181dcad0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:24:07.848237 15046 tablet_replica.cc:333] T 7492e977a5f74ef09aa687b1d7ec89e3 P ff90de856dd542eab446899d181dcad0: stopping tablet replica
I20260709 13:24:07.848722 15046 raft_consensus.cc:2243] T 7492e977a5f74ef09aa687b1d7ec89e3 P ff90de856dd542eab446899d181dcad0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:24:07.849198 15046 raft_consensus.cc:2272] T 7492e977a5f74ef09aa687b1d7ec89e3 P ff90de856dd542eab446899d181dcad0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:24:07.851336 15046 tablet_replica.cc:333] T d5f6b403f0cb4c7a9180d07edb98d1bc P ff90de856dd542eab446899d181dcad0: stopping tablet replica
I20260709 13:24:07.852048 15046 raft_consensus.cc:2243] T d5f6b403f0cb4c7a9180d07edb98d1bc P ff90de856dd542eab446899d181dcad0 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:24:07.852190 26989 consensus_queue.cc:1048] T 3e489a04b06247608a195856b4739a36 P ff90de856dd542eab446899d181dcad0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "326c74b0a8d84cdf86229f67ce3487f2" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35517 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:24:07.852735 15046 pending_rounds.cc:70] T d5f6b403f0cb4c7a9180d07edb98d1bc P ff90de856dd542eab446899d181dcad0: Trying to abort 1 pending ops.
I20260709 13:24:07.853067 15046 pending_rounds.cc:77] T d5f6b403f0cb4c7a9180d07edb98d1bc P ff90de856dd542eab446899d181dcad0: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7305639720837844992 op_type: NO_OP noop_request { }
I20260709 13:24:07.853410 15046 raft_consensus.cc:2889] T d5f6b403f0cb4c7a9180d07edb98d1bc P ff90de856dd542eab446899d181dcad0 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:24:07.853797 15046 raft_consensus.cc:2272] T d5f6b403f0cb4c7a9180d07edb98d1bc P ff90de856dd542eab446899d181dcad0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:24:07.856846 15046 tablet_replica.cc:333] T 3e489a04b06247608a195856b4739a36 P ff90de856dd542eab446899d181dcad0: stopping tablet replica
I20260709 13:24:07.857597 15046 raft_consensus.cc:2243] T 3e489a04b06247608a195856b4739a36 P ff90de856dd542eab446899d181dcad0 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:24:07.858320 15046 pending_rounds.cc:70] T 3e489a04b06247608a195856b4739a36 P ff90de856dd542eab446899d181dcad0: Trying to abort 1 pending ops.
I20260709 13:24:07.858587 15046 pending_rounds.cc:77] T 3e489a04b06247608a195856b4739a36 P ff90de856dd542eab446899d181dcad0: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7305639720912060416 op_type: NO_OP noop_request { }
I20260709 13:24:07.858940 15046 raft_consensus.cc:2889] T 3e489a04b06247608a195856b4739a36 P ff90de856dd542eab446899d181dcad0 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:24:07.859323 15046 raft_consensus.cc:2272] T 3e489a04b06247608a195856b4739a36 P ff90de856dd542eab446899d181dcad0 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:24:07.859431 26872 consensus_peers.cc:597] T e341897686b14a2491c3a4bd24ac9e95 P 239ae18f3e44498fa52e20e9d6eec6b5 -> Peer ff90de856dd542eab446899d181dcad0 (127.14.177.129:36311): Couldn't send request to peer ff90de856dd542eab446899d181dcad0. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:24:07.872231 26987 consensus_queue.cc:1048] T 7492e977a5f74ef09aa687b1d7ec89e3 P 326c74b0a8d84cdf86229f67ce3487f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "239ae18f3e44498fa52e20e9d6eec6b5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 39265 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:24:07.883644 26988 consensus_queue.cc:1048] T bdb51cee932741eda8c12a88f07c890e P 239ae18f3e44498fa52e20e9d6eec6b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "326c74b0a8d84cdf86229f67ce3487f2" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35517 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:24:07.898051 26834 raft_consensus.cc:3060] T e341897686b14a2491c3a4bd24ac9e95 P 326c74b0a8d84cdf86229f67ce3487f2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:24:07.907696 15046 tablet_server.cc:196] TabletServer@127.14.177.129:0 shutdown complete.
I20260709 13:24:07.910900 26987 raft_consensus.cc:493] T e341897686b14a2491c3a4bd24ac9e95 P 326c74b0a8d84cdf86229f67ce3487f2 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:24:07.911527 26987 raft_consensus.cc:515] T e341897686b14a2491c3a4bd24ac9e95 P 326c74b0a8d84cdf86229f67ce3487f2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff90de856dd542eab446899d181dcad0" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36311 } } peers { permanent_uuid: "326c74b0a8d84cdf86229f67ce3487f2" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35517 } } peers { permanent_uuid: "239ae18f3e44498fa52e20e9d6eec6b5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 39265 } }
I20260709 13:24:07.917919 26904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e341897686b14a2491c3a4bd24ac9e95" candidate_uuid: "326c74b0a8d84cdf86229f67ce3487f2" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "239ae18f3e44498fa52e20e9d6eec6b5" is_pre_election: true
I20260709 13:24:07.919915 26987 leader_election.cc:290] T e341897686b14a2491c3a4bd24ac9e95 P 326c74b0a8d84cdf86229f67ce3487f2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ff90de856dd542eab446899d181dcad0 (127.14.177.129:36311), 239ae18f3e44498fa52e20e9d6eec6b5 (127.14.177.131:39265)
W20260709 13:24:07.929731 26798 leader_election.cc:336] T e341897686b14a2491c3a4bd24ac9e95 P 326c74b0a8d84cdf86229f67ce3487f2 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer ff90de856dd542eab446899d181dcad0 (127.14.177.129:36311): Network error: Client connection negotiation failed: client connection to 127.14.177.129:36311: connect: Connection refused (error 111)
I20260709 13:24:07.929754 26990 consensus_queue.cc:1048] T e341897686b14a2491c3a4bd24ac9e95 P 239ae18f3e44498fa52e20e9d6eec6b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "326c74b0a8d84cdf86229f67ce3487f2" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35517 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:24:07.930217 26798 leader_election.cc:304] T e341897686b14a2491c3a4bd24ac9e95 P 326c74b0a8d84cdf86229f67ce3487f2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 326c74b0a8d84cdf86229f67ce3487f2; no voters: 239ae18f3e44498fa52e20e9d6eec6b5, ff90de856dd542eab446899d181dcad0
I20260709 13:24:07.931021 26987 raft_consensus.cc:2749] T e341897686b14a2491c3a4bd24ac9e95 P 326c74b0a8d84cdf86229f67ce3487f2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:24:07.931685 26990 consensus_queue.cc:1048] T 2664382aca514208a9aa49316bdcaeeb P 239ae18f3e44498fa52e20e9d6eec6b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "326c74b0a8d84cdf86229f67ce3487f2" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35517 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 13:24:07.935377 26872 consensus_peers.cc:597] T bdb51cee932741eda8c12a88f07c890e P 239ae18f3e44498fa52e20e9d6eec6b5 -> Peer ff90de856dd542eab446899d181dcad0 (127.14.177.129:36311): Couldn't send request to peer ff90de856dd542eab446899d181dcad0. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36311: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:24:07.935881 26872 consensus_peers.cc:597] T 2664382aca514208a9aa49316bdcaeeb P 239ae18f3e44498fa52e20e9d6eec6b5 -> Peer ff90de856dd542eab446899d181dcad0 (127.14.177.129:36311): Couldn't send request to peer ff90de856dd542eab446899d181dcad0. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36311: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:24:07.938460 26872 consensus_peers.cc:597] T 9ba9365ebe904f85840f6d049dbcda88 P 239ae18f3e44498fa52e20e9d6eec6b5 -> Peer ff90de856dd542eab446899d181dcad0 (127.14.177.129:36311): Couldn't send request to peer ff90de856dd542eab446899d181dcad0. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36311: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:24:08.016578 26987 raft_consensus.cc:493] T 9ba9365ebe904f85840f6d049dbcda88 P 326c74b0a8d84cdf86229f67ce3487f2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:24:08.017233 26987 raft_consensus.cc:515] T 9ba9365ebe904f85840f6d049dbcda88 P 326c74b0a8d84cdf86229f67ce3487f2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff90de856dd542eab446899d181dcad0" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36311 } } peers { permanent_uuid: "326c74b0a8d84cdf86229f67ce3487f2" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35517 } } peers { permanent_uuid: "239ae18f3e44498fa52e20e9d6eec6b5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 39265 } }
I20260709 13:24:08.019925 26987 leader_election.cc:290] T 9ba9365ebe904f85840f6d049dbcda88 P 326c74b0a8d84cdf86229f67ce3487f2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ff90de856dd542eab446899d181dcad0 (127.14.177.129:36311), 239ae18f3e44498fa52e20e9d6eec6b5 (127.14.177.131:39265)
I20260709 13:24:08.020977 26904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ba9365ebe904f85840f6d049dbcda88" candidate_uuid: "326c74b0a8d84cdf86229f67ce3487f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "239ae18f3e44498fa52e20e9d6eec6b5" is_pre_election: true
W20260709 13:24:08.024154 26798 leader_election.cc:336] T 9ba9365ebe904f85840f6d049dbcda88 P 326c74b0a8d84cdf86229f67ce3487f2 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer ff90de856dd542eab446899d181dcad0 (127.14.177.129:36311): Network error: Client connection negotiation failed: client connection to 127.14.177.129:36311: connect: Connection refused (error 111)
I20260709 13:24:08.024531 26798 leader_election.cc:304] T 9ba9365ebe904f85840f6d049dbcda88 P 326c74b0a8d84cdf86229f67ce3487f2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 326c74b0a8d84cdf86229f67ce3487f2; no voters: 239ae18f3e44498fa52e20e9d6eec6b5, ff90de856dd542eab446899d181dcad0
I20260709 13:24:08.025188 26987 raft_consensus.cc:2749] T 9ba9365ebe904f85840f6d049dbcda88 P 326c74b0a8d84cdf86229f67ce3487f2 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:24:08.094363 26833 raft_consensus.cc:3060] T 9ba9365ebe904f85840f6d049dbcda88 P 326c74b0a8d84cdf86229f67ce3487f2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:24:08.101588 26990 consensus_queue.cc:1048] T 9ba9365ebe904f85840f6d049dbcda88 P 239ae18f3e44498fa52e20e9d6eec6b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "326c74b0a8d84cdf86229f67ce3487f2" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35517 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 13:24:08.273679 26872 consensus_peers.cc:597] T 92c3adeb279b4f5f88680a40a9664750 P 239ae18f3e44498fa52e20e9d6eec6b5 -> Peer ff90de856dd542eab446899d181dcad0 (127.14.177.129:36311): Couldn't send request to peer ff90de856dd542eab446899d181dcad0. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36311: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:24:08.805611 26706 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 13:24:08.911697 26987 raft_consensus.cc:493] T d52f26a0449247c1b928641ec8ebad34 P 326c74b0a8d84cdf86229f67ce3487f2 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:24:08.912328 26987 raft_consensus.cc:515] T d52f26a0449247c1b928641ec8ebad34 P 326c74b0a8d84cdf86229f67ce3487f2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff90de856dd542eab446899d181dcad0" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36311 } } peers { permanent_uuid: "326c74b0a8d84cdf86229f67ce3487f2" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35517 } } peers { permanent_uuid: "239ae18f3e44498fa52e20e9d6eec6b5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 39265 } }
I20260709 13:24:08.914595 26987 leader_election.cc:290] T d52f26a0449247c1b928641ec8ebad34 P 326c74b0a8d84cdf86229f67ce3487f2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ff90de856dd542eab446899d181dcad0 (127.14.177.129:36311), 239ae18f3e44498fa52e20e9d6eec6b5 (127.14.177.131:39265)
I20260709 13:24:08.915717 26904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d52f26a0449247c1b928641ec8ebad34" candidate_uuid: "326c74b0a8d84cdf86229f67ce3487f2" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "239ae18f3e44498fa52e20e9d6eec6b5" is_pre_election: true
I20260709 13:24:08.916410 26904 raft_consensus.cc:2468] T d52f26a0449247c1b928641ec8ebad34 P 239ae18f3e44498fa52e20e9d6eec6b5 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 326c74b0a8d84cdf86229f67ce3487f2 in term 1.
I20260709 13:24:08.917517 26801 leader_election.cc:304] T d52f26a0449247c1b928641ec8ebad34 P 326c74b0a8d84cdf86229f67ce3487f2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 239ae18f3e44498fa52e20e9d6eec6b5, 326c74b0a8d84cdf86229f67ce3487f2; no voters: 
I20260709 13:24:08.918236 26987 raft_consensus.cc:2804] T d52f26a0449247c1b928641ec8ebad34 P 326c74b0a8d84cdf86229f67ce3487f2 [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260709 13:24:08.918380 26798 leader_election.cc:336] T d52f26a0449247c1b928641ec8ebad34 P 326c74b0a8d84cdf86229f67ce3487f2 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer ff90de856dd542eab446899d181dcad0 (127.14.177.129:36311): Network error: Client connection negotiation failed: client connection to 127.14.177.129:36311: connect: Connection refused (error 111)
I20260709 13:24:08.918612 26987 raft_consensus.cc:493] T d52f26a0449247c1b928641ec8ebad34 P 326c74b0a8d84cdf86229f67ce3487f2 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:24:08.919037 26987 raft_consensus.cc:3060] T d52f26a0449247c1b928641ec8ebad34 P 326c74b0a8d84cdf86229f67ce3487f2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:24:08.924180 26987 raft_consensus.cc:515] T d52f26a0449247c1b928641ec8ebad34 P 326c74b0a8d84cdf86229f67ce3487f2 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff90de856dd542eab446899d181dcad0" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36311 } } peers { permanent_uuid: "326c74b0a8d84cdf86229f67ce3487f2" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35517 } } peers { permanent_uuid: "239ae18f3e44498fa52e20e9d6eec6b5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 39265 } }
I20260709 13:24:08.925874 26987 leader_election.cc:290] T d52f26a0449247c1b928641ec8ebad34 P 326c74b0a8d84cdf86229f67ce3487f2 [CANDIDATE]: Term 2 election: Requested vote from peers ff90de856dd542eab446899d181dcad0 (127.14.177.129:36311), 239ae18f3e44498fa52e20e9d6eec6b5 (127.14.177.131:39265)
I20260709 13:24:08.926941 26904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d52f26a0449247c1b928641ec8ebad34" candidate_uuid: "326c74b0a8d84cdf86229f67ce3487f2" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "239ae18f3e44498fa52e20e9d6eec6b5"
I20260709 13:24:08.927601 26904 raft_consensus.cc:3060] T d52f26a0449247c1b928641ec8ebad34 P 239ae18f3e44498fa52e20e9d6eec6b5 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:24:08.929715 26798 leader_election.cc:336] T d52f26a0449247c1b928641ec8ebad34 P 326c74b0a8d84cdf86229f67ce3487f2 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer ff90de856dd542eab446899d181dcad0 (127.14.177.129:36311): Network error: Client connection negotiation failed: client connection to 127.14.177.129:36311: connect: Connection refused (error 111)
I20260709 13:24:08.932905 26904 raft_consensus.cc:2468] T d52f26a0449247c1b928641ec8ebad34 P 239ae18f3e44498fa52e20e9d6eec6b5 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 326c74b0a8d84cdf86229f67ce3487f2 in term 2.
I20260709 13:24:08.934248 26801 leader_election.cc:304] T d52f26a0449247c1b928641ec8ebad34 P 326c74b0a8d84cdf86229f67ce3487f2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 239ae18f3e44498fa52e20e9d6eec6b5, 326c74b0a8d84cdf86229f67ce3487f2; no voters: ff90de856dd542eab446899d181dcad0
I20260709 13:24:08.935141 26987 raft_consensus.cc:2804] T d52f26a0449247c1b928641ec8ebad34 P 326c74b0a8d84cdf86229f67ce3487f2 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:24:08.935742 26987 raft_consensus.cc:697] T d52f26a0449247c1b928641ec8ebad34 P 326c74b0a8d84cdf86229f67ce3487f2 [term 2 LEADER]: Becoming Leader. State: Replica: 326c74b0a8d84cdf86229f67ce3487f2, State: Running, Role: LEADER
I20260709 13:24:08.936756 26987 consensus_queue.cc:237] T d52f26a0449247c1b928641ec8ebad34 P 326c74b0a8d84cdf86229f67ce3487f2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff90de856dd542eab446899d181dcad0" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36311 } } peers { permanent_uuid: "326c74b0a8d84cdf86229f67ce3487f2" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35517 } } peers { permanent_uuid: "239ae18f3e44498fa52e20e9d6eec6b5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 39265 } }
I20260709 13:24:08.945144 26656 catalog_manager.cc:5697] T d52f26a0449247c1b928641ec8ebad34 P 326c74b0a8d84cdf86229f67ce3487f2 reported cstate change: term changed from 1 to 2, leader changed from ff90de856dd542eab446899d181dcad0 (127.14.177.129) to 326c74b0a8d84cdf86229f67ce3487f2 (127.14.177.130). New cstate: current_term: 2 leader_uuid: "326c74b0a8d84cdf86229f67ce3487f2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff90de856dd542eab446899d181dcad0" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36311 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "326c74b0a8d84cdf86229f67ce3487f2" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35517 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "239ae18f3e44498fa52e20e9d6eec6b5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 39265 } health_report { overall_health: UNKNOWN } } }
W20260709 13:24:08.976150 26872 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36311: connect: Connection refused (error 111) [suppressed 129 similar messages]
W20260709 13:24:09.343462 26798 consensus_peers.cc:597] T d52f26a0449247c1b928641ec8ebad34 P 326c74b0a8d84cdf86229f67ce3487f2 -> Peer ff90de856dd542eab446899d181dcad0 (127.14.177.129:36311): Couldn't send request to peer ff90de856dd542eab446899d181dcad0. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36311: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:24:09.356375 26987 raft_consensus.cc:493] T 3e489a04b06247608a195856b4739a36 P 326c74b0a8d84cdf86229f67ce3487f2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ff90de856dd542eab446899d181dcad0)
I20260709 13:24:09.357095 26987 raft_consensus.cc:515] T 3e489a04b06247608a195856b4739a36 P 326c74b0a8d84cdf86229f67ce3487f2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff90de856dd542eab446899d181dcad0" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36311 } } peers { permanent_uuid: "326c74b0a8d84cdf86229f67ce3487f2" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35517 } } peers { permanent_uuid: "239ae18f3e44498fa52e20e9d6eec6b5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 39265 } }
I20260709 13:24:09.359391 26987 leader_election.cc:290] T 3e489a04b06247608a195856b4739a36 P 326c74b0a8d84cdf86229f67ce3487f2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ff90de856dd542eab446899d181dcad0 (127.14.177.129:36311), 239ae18f3e44498fa52e20e9d6eec6b5 (127.14.177.131:39265)
I20260709 13:24:09.360689 26904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e489a04b06247608a195856b4739a36" candidate_uuid: "326c74b0a8d84cdf86229f67ce3487f2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "239ae18f3e44498fa52e20e9d6eec6b5" is_pre_election: true
I20260709 13:24:09.361467 26904 raft_consensus.cc:2468] T 3e489a04b06247608a195856b4739a36 P 239ae18f3e44498fa52e20e9d6eec6b5 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 326c74b0a8d84cdf86229f67ce3487f2 in term 1.
I20260709 13:24:09.362540 26801 leader_election.cc:304] T 3e489a04b06247608a195856b4739a36 P 326c74b0a8d84cdf86229f67ce3487f2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 239ae18f3e44498fa52e20e9d6eec6b5, 326c74b0a8d84cdf86229f67ce3487f2; no voters: 
I20260709 13:24:09.363377 26987 raft_consensus.cc:2804] T 3e489a04b06247608a195856b4739a36 P 326c74b0a8d84cdf86229f67ce3487f2 [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260709 13:24:09.363498 26798 leader_election.cc:336] T 3e489a04b06247608a195856b4739a36 P 326c74b0a8d84cdf86229f67ce3487f2 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer ff90de856dd542eab446899d181dcad0 (127.14.177.129:36311): Network error: Client connection negotiation failed: client connection to 127.14.177.129:36311: connect: Connection refused (error 111)
I20260709 13:24:09.363742 26987 raft_consensus.cc:493] T 3e489a04b06247608a195856b4739a36 P 326c74b0a8d84cdf86229f67ce3487f2 [term 1 FOLLOWER]: Starting leader election (detected failure of leader ff90de856dd542eab446899d181dcad0)
I20260709 13:24:09.364178 26987 raft_consensus.cc:3060] T 3e489a04b06247608a195856b4739a36 P 326c74b0a8d84cdf86229f67ce3487f2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:24:09.369704 26987 raft_consensus.cc:515] T 3e489a04b06247608a195856b4739a36 P 326c74b0a8d84cdf86229f67ce3487f2 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff90de856dd542eab446899d181dcad0" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36311 } } peers { permanent_uuid: "326c74b0a8d84cdf86229f67ce3487f2" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35517 } } peers { permanent_uuid: "239ae18f3e44498fa52e20e9d6eec6b5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 39265 } }
I20260709 13:24:09.371665 26987 leader_election.cc:290] T 3e489a04b06247608a195856b4739a36 P 326c74b0a8d84cdf86229f67ce3487f2 [CANDIDATE]: Term 2 election: Requested vote from peers ff90de856dd542eab446899d181dcad0 (127.14.177.129:36311), 239ae18f3e44498fa52e20e9d6eec6b5 (127.14.177.131:39265)
I20260709 13:24:09.372820 26904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e489a04b06247608a195856b4739a36" candidate_uuid: "326c74b0a8d84cdf86229f67ce3487f2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "239ae18f3e44498fa52e20e9d6eec6b5"
I20260709 13:24:09.373464 26904 raft_consensus.cc:3060] T 3e489a04b06247608a195856b4739a36 P 239ae18f3e44498fa52e20e9d6eec6b5 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:24:09.375500 26798 leader_election.cc:336] T 3e489a04b06247608a195856b4739a36 P 326c74b0a8d84cdf86229f67ce3487f2 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer ff90de856dd542eab446899d181dcad0 (127.14.177.129:36311): Network error: Client connection negotiation failed: client connection to 127.14.177.129:36311: connect: Connection refused (error 111)
I20260709 13:24:09.379004 26904 raft_consensus.cc:2468] T 3e489a04b06247608a195856b4739a36 P 239ae18f3e44498fa52e20e9d6eec6b5 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 326c74b0a8d84cdf86229f67ce3487f2 in term 2.
I20260709 13:24:09.380296 26801 leader_election.cc:304] T 3e489a04b06247608a195856b4739a36 P 326c74b0a8d84cdf86229f67ce3487f2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 239ae18f3e44498fa52e20e9d6eec6b5, 326c74b0a8d84cdf86229f67ce3487f2; no voters: ff90de856dd542eab446899d181dcad0
I20260709 13:24:09.381318 26987 raft_consensus.cc:2804] T 3e489a04b06247608a195856b4739a36 P 326c74b0a8d84cdf86229f67ce3487f2 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:24:09.382004 26987 raft_consensus.cc:697] T 3e489a04b06247608a195856b4739a36 P 326c74b0a8d84cdf86229f67ce3487f2 [term 2 LEADER]: Becoming Leader. State: Replica: 326c74b0a8d84cdf86229f67ce3487f2, State: Running, Role: LEADER
I20260709 13:24:09.382906 26987 consensus_queue.cc:237] T 3e489a04b06247608a195856b4739a36 P 326c74b0a8d84cdf86229f67ce3487f2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff90de856dd542eab446899d181dcad0" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36311 } } peers { permanent_uuid: "326c74b0a8d84cdf86229f67ce3487f2" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35517 } } peers { permanent_uuid: "239ae18f3e44498fa52e20e9d6eec6b5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 39265 } }
I20260709 13:24:09.386112 26988 raft_consensus.cc:493] T d5f6b403f0cb4c7a9180d07edb98d1bc P 239ae18f3e44498fa52e20e9d6eec6b5 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:24:09.386770 26988 raft_consensus.cc:515] T d5f6b403f0cb4c7a9180d07edb98d1bc P 239ae18f3e44498fa52e20e9d6eec6b5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "239ae18f3e44498fa52e20e9d6eec6b5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 39265 } } peers { permanent_uuid: "326c74b0a8d84cdf86229f67ce3487f2" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35517 } } peers { permanent_uuid: "ff90de856dd542eab446899d181dcad0" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36311 } }
I20260709 13:24:09.389882 26988 leader_election.cc:290] T d5f6b403f0cb4c7a9180d07edb98d1bc P 239ae18f3e44498fa52e20e9d6eec6b5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 326c74b0a8d84cdf86229f67ce3487f2 (127.14.177.130:35517), ff90de856dd542eab446899d181dcad0 (127.14.177.129:36311)
I20260709 13:24:09.390654 26833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5f6b403f0cb4c7a9180d07edb98d1bc" candidate_uuid: "239ae18f3e44498fa52e20e9d6eec6b5" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "326c74b0a8d84cdf86229f67ce3487f2" is_pre_election: true
I20260709 13:24:09.391361 26833 raft_consensus.cc:2468] T d5f6b403f0cb4c7a9180d07edb98d1bc P 326c74b0a8d84cdf86229f67ce3487f2 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 239ae18f3e44498fa52e20e9d6eec6b5 in term 1.
I20260709 13:24:09.392870 26872 leader_election.cc:304] T d5f6b403f0cb4c7a9180d07edb98d1bc P 239ae18f3e44498fa52e20e9d6eec6b5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 239ae18f3e44498fa52e20e9d6eec6b5, 326c74b0a8d84cdf86229f67ce3487f2; no voters: 
I20260709 13:24:09.394140 26988 raft_consensus.cc:2804] T d5f6b403f0cb4c7a9180d07edb98d1bc P 239ae18f3e44498fa52e20e9d6eec6b5 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:24:09.392840 26656 catalog_manager.cc:5697] T 3e489a04b06247608a195856b4739a36 P 326c74b0a8d84cdf86229f67ce3487f2 reported cstate change: term changed from 1 to 2, leader changed from ff90de856dd542eab446899d181dcad0 (127.14.177.129) to 326c74b0a8d84cdf86229f67ce3487f2 (127.14.177.130). New cstate: current_term: 2 leader_uuid: "326c74b0a8d84cdf86229f67ce3487f2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff90de856dd542eab446899d181dcad0" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36311 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "326c74b0a8d84cdf86229f67ce3487f2" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35517 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "239ae18f3e44498fa52e20e9d6eec6b5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 39265 } health_report { overall_health: UNKNOWN } } }
I20260709 13:24:09.394625 26988 raft_consensus.cc:493] T d5f6b403f0cb4c7a9180d07edb98d1bc P 239ae18f3e44498fa52e20e9d6eec6b5 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:24:09.395028 26988 raft_consensus.cc:3060] T d5f6b403f0cb4c7a9180d07edb98d1bc P 239ae18f3e44498fa52e20e9d6eec6b5 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:24:09.395913 26872 leader_election.cc:336] T d5f6b403f0cb4c7a9180d07edb98d1bc P 239ae18f3e44498fa52e20e9d6eec6b5 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer ff90de856dd542eab446899d181dcad0 (127.14.177.129:36311): Network error: Client connection negotiation failed: client connection to 127.14.177.129:36311: connect: Connection refused (error 111)
I20260709 13:24:09.401603 26988 raft_consensus.cc:515] T d5f6b403f0cb4c7a9180d07edb98d1bc P 239ae18f3e44498fa52e20e9d6eec6b5 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "239ae18f3e44498fa52e20e9d6eec6b5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 39265 } } peers { permanent_uuid: "326c74b0a8d84cdf86229f67ce3487f2" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35517 } } peers { permanent_uuid: "ff90de856dd542eab446899d181dcad0" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36311 } }
I20260709 13:24:09.403606 26988 leader_election.cc:290] T d5f6b403f0cb4c7a9180d07edb98d1bc P 239ae18f3e44498fa52e20e9d6eec6b5 [CANDIDATE]: Term 2 election: Requested vote from peers 326c74b0a8d84cdf86229f67ce3487f2 (127.14.177.130:35517), ff90de856dd542eab446899d181dcad0 (127.14.177.129:36311)
I20260709 13:24:09.404340 26833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5f6b403f0cb4c7a9180d07edb98d1bc" candidate_uuid: "239ae18f3e44498fa52e20e9d6eec6b5" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "326c74b0a8d84cdf86229f67ce3487f2"
I20260709 13:24:09.405043 26833 raft_consensus.cc:3060] T d5f6b403f0cb4c7a9180d07edb98d1bc P 326c74b0a8d84cdf86229f67ce3487f2 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:24:09.408718 26872 leader_election.cc:336] T d5f6b403f0cb4c7a9180d07edb98d1bc P 239ae18f3e44498fa52e20e9d6eec6b5 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer ff90de856dd542eab446899d181dcad0 (127.14.177.129:36311): Network error: Client connection negotiation failed: client connection to 127.14.177.129:36311: connect: Connection refused (error 111)
I20260709 13:24:09.412014 26833 raft_consensus.cc:2468] T d5f6b403f0cb4c7a9180d07edb98d1bc P 326c74b0a8d84cdf86229f67ce3487f2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 239ae18f3e44498fa52e20e9d6eec6b5 in term 2.
I20260709 13:24:09.413224 26872 leader_election.cc:304] T d5f6b403f0cb4c7a9180d07edb98d1bc P 239ae18f3e44498fa52e20e9d6eec6b5 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 239ae18f3e44498fa52e20e9d6eec6b5, 326c74b0a8d84cdf86229f67ce3487f2; no voters: ff90de856dd542eab446899d181dcad0
I20260709 13:24:09.414011 26988 raft_consensus.cc:2804] T d5f6b403f0cb4c7a9180d07edb98d1bc P 239ae18f3e44498fa52e20e9d6eec6b5 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:24:09.414484 26988 raft_consensus.cc:697] T d5f6b403f0cb4c7a9180d07edb98d1bc P 239ae18f3e44498fa52e20e9d6eec6b5 [term 2 LEADER]: Becoming Leader. State: Replica: 239ae18f3e44498fa52e20e9d6eec6b5, State: Running, Role: LEADER
I20260709 13:24:09.415347 26988 consensus_queue.cc:237] T d5f6b403f0cb4c7a9180d07edb98d1bc P 239ae18f3e44498fa52e20e9d6eec6b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "239ae18f3e44498fa52e20e9d6eec6b5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 39265 } } peers { permanent_uuid: "326c74b0a8d84cdf86229f67ce3487f2" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35517 } } peers { permanent_uuid: "ff90de856dd542eab446899d181dcad0" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36311 } }
I20260709 13:24:09.422509 26656 catalog_manager.cc:5697] T d5f6b403f0cb4c7a9180d07edb98d1bc P 239ae18f3e44498fa52e20e9d6eec6b5 reported cstate change: term changed from 1 to 2, leader changed from ff90de856dd542eab446899d181dcad0 (127.14.177.129) to 239ae18f3e44498fa52e20e9d6eec6b5 (127.14.177.131). New cstate: current_term: 2 leader_uuid: "239ae18f3e44498fa52e20e9d6eec6b5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "239ae18f3e44498fa52e20e9d6eec6b5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 39265 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "326c74b0a8d84cdf86229f67ce3487f2" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35517 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ff90de856dd542eab446899d181dcad0" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36311 } health_report { overall_health: UNKNOWN } } }
I20260709 13:24:09.495939 26904 raft_consensus.cc:1275] T d52f26a0449247c1b928641ec8ebad34 P 239ae18f3e44498fa52e20e9d6eec6b5 [term 2 FOLLOWER]: Refusing update from remote peer 326c74b0a8d84cdf86229f67ce3487f2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 13:24:09.498301 26987 consensus_queue.cc:1048] T d52f26a0449247c1b928641ec8ebad34 P 326c74b0a8d84cdf86229f67ce3487f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "239ae18f3e44498fa52e20e9d6eec6b5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 39265 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:24:09.545825 27035 raft_consensus.cc:493] T 76c716abdcd6437b9dcee43317333298 P 326c74b0a8d84cdf86229f67ce3487f2 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:24:09.546658 27035 raft_consensus.cc:515] T 76c716abdcd6437b9dcee43317333298 P 326c74b0a8d84cdf86229f67ce3487f2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff90de856dd542eab446899d181dcad0" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36311 } } peers { permanent_uuid: "326c74b0a8d84cdf86229f67ce3487f2" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35517 } } peers { permanent_uuid: "239ae18f3e44498fa52e20e9d6eec6b5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 39265 } }
I20260709 13:24:09.549078 27035 leader_election.cc:290] T 76c716abdcd6437b9dcee43317333298 P 326c74b0a8d84cdf86229f67ce3487f2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ff90de856dd542eab446899d181dcad0 (127.14.177.129:36311), 239ae18f3e44498fa52e20e9d6eec6b5 (127.14.177.131:39265)
I20260709 13:24:09.550511 26904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76c716abdcd6437b9dcee43317333298" candidate_uuid: "326c74b0a8d84cdf86229f67ce3487f2" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "239ae18f3e44498fa52e20e9d6eec6b5" is_pre_election: true
I20260709 13:24:09.551566 26904 raft_consensus.cc:2468] T 76c716abdcd6437b9dcee43317333298 P 239ae18f3e44498fa52e20e9d6eec6b5 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 326c74b0a8d84cdf86229f67ce3487f2 in term 1.
I20260709 13:24:09.552738 26801 leader_election.cc:304] T 76c716abdcd6437b9dcee43317333298 P 326c74b0a8d84cdf86229f67ce3487f2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 239ae18f3e44498fa52e20e9d6eec6b5, 326c74b0a8d84cdf86229f67ce3487f2; no voters: 
I20260709 13:24:09.553572 27035 raft_consensus.cc:2804] T 76c716abdcd6437b9dcee43317333298 P 326c74b0a8d84cdf86229f67ce3487f2 [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260709 13:24:09.553870 26798 leader_election.cc:336] T 76c716abdcd6437b9dcee43317333298 P 326c74b0a8d84cdf86229f67ce3487f2 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer ff90de856dd542eab446899d181dcad0 (127.14.177.129:36311): Network error: Client connection negotiation failed: client connection to 127.14.177.129:36311: connect: Connection refused (error 111)
I20260709 13:24:09.553898 27035 raft_consensus.cc:493] T 76c716abdcd6437b9dcee43317333298 P 326c74b0a8d84cdf86229f67ce3487f2 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:24:09.554504 27035 raft_consensus.cc:3060] T 76c716abdcd6437b9dcee43317333298 P 326c74b0a8d84cdf86229f67ce3487f2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:24:09.562704 27035 raft_consensus.cc:515] T 76c716abdcd6437b9dcee43317333298 P 326c74b0a8d84cdf86229f67ce3487f2 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff90de856dd542eab446899d181dcad0" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36311 } } peers { permanent_uuid: "326c74b0a8d84cdf86229f67ce3487f2" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35517 } } peers { permanent_uuid: "239ae18f3e44498fa52e20e9d6eec6b5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 39265 } }
I20260709 13:24:09.564502 27035 leader_election.cc:290] T 76c716abdcd6437b9dcee43317333298 P 326c74b0a8d84cdf86229f67ce3487f2 [CANDIDATE]: Term 2 election: Requested vote from peers ff90de856dd542eab446899d181dcad0 (127.14.177.129:36311), 239ae18f3e44498fa52e20e9d6eec6b5 (127.14.177.131:39265)
I20260709 13:24:09.565773 26904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76c716abdcd6437b9dcee43317333298" candidate_uuid: "326c74b0a8d84cdf86229f67ce3487f2" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "239ae18f3e44498fa52e20e9d6eec6b5"
I20260709 13:24:09.566399 26904 raft_consensus.cc:3060] T 76c716abdcd6437b9dcee43317333298 P 239ae18f3e44498fa52e20e9d6eec6b5 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:24:09.568615 26798 leader_election.cc:336] T 76c716abdcd6437b9dcee43317333298 P 326c74b0a8d84cdf86229f67ce3487f2 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer ff90de856dd542eab446899d181dcad0 (127.14.177.129:36311): Network error: Client connection negotiation failed: client connection to 127.14.177.129:36311: connect: Connection refused (error 111)
I20260709 13:24:09.575026 26904 raft_consensus.cc:2468] T 76c716abdcd6437b9dcee43317333298 P 239ae18f3e44498fa52e20e9d6eec6b5 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 326c74b0a8d84cdf86229f67ce3487f2 in term 2.
I20260709 13:24:09.576489 26801 leader_election.cc:304] T 76c716abdcd6437b9dcee43317333298 P 326c74b0a8d84cdf86229f67ce3487f2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 239ae18f3e44498fa52e20e9d6eec6b5, 326c74b0a8d84cdf86229f67ce3487f2; no voters: ff90de856dd542eab446899d181dcad0
I20260709 13:24:09.577155 27035 raft_consensus.cc:2804] T 76c716abdcd6437b9dcee43317333298 P 326c74b0a8d84cdf86229f67ce3487f2 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:24:09.577615 27035 raft_consensus.cc:697] T 76c716abdcd6437b9dcee43317333298 P 326c74b0a8d84cdf86229f67ce3487f2 [term 2 LEADER]: Becoming Leader. State: Replica: 326c74b0a8d84cdf86229f67ce3487f2, State: Running, Role: LEADER
I20260709 13:24:09.578750 27035 consensus_queue.cc:237] T 76c716abdcd6437b9dcee43317333298 P 326c74b0a8d84cdf86229f67ce3487f2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff90de856dd542eab446899d181dcad0" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36311 } } peers { permanent_uuid: "326c74b0a8d84cdf86229f67ce3487f2" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35517 } } peers { permanent_uuid: "239ae18f3e44498fa52e20e9d6eec6b5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 39265 } }
I20260709 13:24:09.586990 26656 catalog_manager.cc:5697] T 76c716abdcd6437b9dcee43317333298 P 326c74b0a8d84cdf86229f67ce3487f2 reported cstate change: term changed from 1 to 2, leader changed from ff90de856dd542eab446899d181dcad0 (127.14.177.129) to 326c74b0a8d84cdf86229f67ce3487f2 (127.14.177.130). New cstate: current_term: 2 leader_uuid: "326c74b0a8d84cdf86229f67ce3487f2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff90de856dd542eab446899d181dcad0" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36311 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "326c74b0a8d84cdf86229f67ce3487f2" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35517 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "239ae18f3e44498fa52e20e9d6eec6b5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 39265 } health_report { overall_health: UNKNOWN } } }
W20260709 13:24:09.806818 26706 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
W20260709 13:24:09.821782 26798 consensus_peers.cc:597] T 3e489a04b06247608a195856b4739a36 P 326c74b0a8d84cdf86229f67ce3487f2 -> Peer ff90de856dd542eab446899d181dcad0 (127.14.177.129:36311): Couldn't send request to peer ff90de856dd542eab446899d181dcad0. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36311: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:24:09.830101 26904 raft_consensus.cc:1275] T 3e489a04b06247608a195856b4739a36 P 239ae18f3e44498fa52e20e9d6eec6b5 [term 2 FOLLOWER]: Refusing update from remote peer 326c74b0a8d84cdf86229f67ce3487f2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:24:09.831568 27035 consensus_queue.cc:1048] T 3e489a04b06247608a195856b4739a36 P 326c74b0a8d84cdf86229f67ce3487f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "239ae18f3e44498fa52e20e9d6eec6b5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 39265 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 13:24:09.851454 26872 consensus_peers.cc:597] T d5f6b403f0cb4c7a9180d07edb98d1bc P 239ae18f3e44498fa52e20e9d6eec6b5 -> Peer ff90de856dd542eab446899d181dcad0 (127.14.177.129:36311): Couldn't send request to peer ff90de856dd542eab446899d181dcad0. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36311: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:24:10.017618 26833 raft_consensus.cc:1275] T d5f6b403f0cb4c7a9180d07edb98d1bc P 326c74b0a8d84cdf86229f67ce3487f2 [term 2 FOLLOWER]: Refusing update from remote peer 239ae18f3e44498fa52e20e9d6eec6b5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 13:24:10.019106 27051 consensus_queue.cc:1048] T d5f6b403f0cb4c7a9180d07edb98d1bc P 239ae18f3e44498fa52e20e9d6eec6b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "326c74b0a8d84cdf86229f67ce3487f2" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35517 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:24:10.079672 26904 raft_consensus.cc:1275] T 76c716abdcd6437b9dcee43317333298 P 239ae18f3e44498fa52e20e9d6eec6b5 [term 2 FOLLOWER]: Refusing update from remote peer 326c74b0a8d84cdf86229f67ce3487f2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 13:24:10.081332 27035 consensus_queue.cc:1048] T 76c716abdcd6437b9dcee43317333298 P 326c74b0a8d84cdf86229f67ce3487f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "239ae18f3e44498fa52e20e9d6eec6b5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 39265 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 13:24:10.096544 26798 consensus_peers.cc:597] T 76c716abdcd6437b9dcee43317333298 P 326c74b0a8d84cdf86229f67ce3487f2 -> Peer ff90de856dd542eab446899d181dcad0 (127.14.177.129:36311): Couldn't send request to peer ff90de856dd542eab446899d181dcad0. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36311: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:24:10.176470 26798 consensus_peers.cc:597] T 7492e977a5f74ef09aa687b1d7ec89e3 P 326c74b0a8d84cdf86229f67ce3487f2 -> Peer ff90de856dd542eab446899d181dcad0 (127.14.177.129:36311): Couldn't send request to peer ff90de856dd542eab446899d181dcad0. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36311: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:24:10.340952 26872 consensus_peers.cc:597] T bdb51cee932741eda8c12a88f07c890e P 239ae18f3e44498fa52e20e9d6eec6b5 -> Peer ff90de856dd542eab446899d181dcad0 (127.14.177.129:36311): Couldn't send request to peer ff90de856dd542eab446899d181dcad0. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36311: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:24:10.577817 26872 consensus_peers.cc:597] T e341897686b14a2491c3a4bd24ac9e95 P 239ae18f3e44498fa52e20e9d6eec6b5 -> Peer ff90de856dd542eab446899d181dcad0 (127.14.177.129:36311): Couldn't send request to peer ff90de856dd542eab446899d181dcad0. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36311: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:24:10.662770 26872 consensus_peers.cc:597] T 9ba9365ebe904f85840f6d049dbcda88 P 239ae18f3e44498fa52e20e9d6eec6b5 -> Peer ff90de856dd542eab446899d181dcad0 (127.14.177.129:36311): Couldn't send request to peer ff90de856dd542eab446899d181dcad0. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36311: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:24:10.719338 26872 consensus_peers.cc:597] T 2664382aca514208a9aa49316bdcaeeb P 239ae18f3e44498fa52e20e9d6eec6b5 -> Peer ff90de856dd542eab446899d181dcad0 (127.14.177.129:36311): Couldn't send request to peer ff90de856dd542eab446899d181dcad0. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36311: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:24:10.779515 26872 consensus_peers.cc:597] T 92c3adeb279b4f5f88680a40a9664750 P 239ae18f3e44498fa52e20e9d6eec6b5 -> Peer ff90de856dd542eab446899d181dcad0 (127.14.177.129:36311): Couldn't send request to peer ff90de856dd542eab446899d181dcad0. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36311: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:24:10.807823 26706 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
W20260709 13:24:11.779862 26798 consensus_peers.cc:597] T d52f26a0449247c1b928641ec8ebad34 P 326c74b0a8d84cdf86229f67ce3487f2 -> Peer ff90de856dd542eab446899d181dcad0 (127.14.177.129:36311): Couldn't send request to peer ff90de856dd542eab446899d181dcad0. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36311: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:24:11.808951 26706 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
W20260709 13:24:12.223133 26798 consensus_peers.cc:597] T 3e489a04b06247608a195856b4739a36 P 326c74b0a8d84cdf86229f67ce3487f2 -> Peer ff90de856dd542eab446899d181dcad0 (127.14.177.129:36311): Couldn't send request to peer ff90de856dd542eab446899d181dcad0. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36311: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:24:12.309722 26872 consensus_peers.cc:597] T d5f6b403f0cb4c7a9180d07edb98d1bc P 239ae18f3e44498fa52e20e9d6eec6b5 -> Peer ff90de856dd542eab446899d181dcad0 (127.14.177.129:36311): Couldn't send request to peer ff90de856dd542eab446899d181dcad0. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36311: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:24:12.357409 26872 consensus_peers.cc:597] T bdb51cee932741eda8c12a88f07c890e P 239ae18f3e44498fa52e20e9d6eec6b5 -> Peer ff90de856dd542eab446899d181dcad0 (127.14.177.129:36311): Couldn't send request to peer ff90de856dd542eab446899d181dcad0. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36311: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:24:12.810240 26706 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
W20260709 13:24:12.835393 26798 consensus_peers.cc:597] T 7492e977a5f74ef09aa687b1d7ec89e3 P 326c74b0a8d84cdf86229f67ce3487f2 -> Peer ff90de856dd542eab446899d181dcad0 (127.14.177.129:36311): Couldn't send request to peer ff90de856dd542eab446899d181dcad0. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36311: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:24:12.854399 26798 consensus_peers.cc:597] T 76c716abdcd6437b9dcee43317333298 P 326c74b0a8d84cdf86229f67ce3487f2 -> Peer ff90de856dd542eab446899d181dcad0 (127.14.177.129:36311): Couldn't send request to peer ff90de856dd542eab446899d181dcad0. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36311: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:24:12.955080 15046 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:24:12.963016 27056 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:24:12.963696 27057 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:24:12.966024 27059 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:24:12.967625 15046 server_base.cc:1061] running on GCE node
W20260709 13:24:12.971138 26872 consensus_peers.cc:597] T 9ba9365ebe904f85840f6d049dbcda88 P 239ae18f3e44498fa52e20e9d6eec6b5 -> Peer ff90de856dd542eab446899d181dcad0 (127.14.177.129:36311): Couldn't send request to peer ff90de856dd542eab446899d181dcad0. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36311: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:24:12.971484 15046 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:24:12.971843 15046 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:24:12.972050 15046 hybrid_clock.cc:648] HybridClock initialized: now 1783603452972034 us; error 0 us; skew 500 ppm
I20260709 13:24:12.972730 15046 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:24:12.975942 15046 webserver.cc:533] Webserver started at http://127.14.177.129:40653/ using document root <none> and password file <none>
I20260709 13:24:12.976584 15046 fs_manager.cc:362] Metadata directory not provided
I20260709 13:24:12.976823 15046 fs_manager.cc:365] Using existing metadata directory in first data directory
I20260709 13:24:12.981634 15046 fs_manager.cc:714] Time spent opening directory manager: real 0.004s	user 0.005s	sys 0.000s
I20260709 13:24:12.987069 27064 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:24:12.988005 15046 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.000s
I20260709 13:24:12.988364 15046 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskdH_Ma7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783603093801613-15046-0/minicluster-data/ts-0-root
uuid: "ff90de856dd542eab446899d181dcad0"
format_stamp: "Formatted at 2026-07-09 13:24:05 on dist-test-slave-v5z2"
I20260709 13:24:12.988786 15046 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdH_Ma7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783603093801613-15046-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskdH_Ma7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783603093801613-15046-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskdH_Ma7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783603093801613-15046-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:24:12.999548 15046 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:24:13.001050 15046 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:24:13.002998 15046 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:24:13.007050 27071 ts_tablet_manager.cc:548] Loading tablet metadata (0/10 complete)
W20260709 13:24:13.021741 26872 consensus_peers.cc:597] T e341897686b14a2491c3a4bd24ac9e95 P 239ae18f3e44498fa52e20e9d6eec6b5 -> Peer ff90de856dd542eab446899d181dcad0 (127.14.177.129:36311): Couldn't send request to peer ff90de856dd542eab446899d181dcad0. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36311: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:24:13.063958 26872 consensus_peers.cc:597] T 2664382aca514208a9aa49316bdcaeeb P 239ae18f3e44498fa52e20e9d6eec6b5 -> Peer ff90de856dd542eab446899d181dcad0 (127.14.177.129:36311): Couldn't send request to peer ff90de856dd542eab446899d181dcad0. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36311: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:24:13.071117 15046 ts_tablet_manager.cc:585] Loaded tablet metadata (10 total tablets, 10 live tablets)
I20260709 13:24:13.071664 15046 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.066s	user 0.001s	sys 0.002s
I20260709 13:24:13.071997 15046 ts_tablet_manager.cc:600] Registering tablets (0/10 complete)
I20260709 13:24:13.077468 27071 tablet_bootstrap.cc:492] T 3e489a04b06247608a195856b4739a36 P ff90de856dd542eab446899d181dcad0: Bootstrap starting.
I20260709 13:24:13.091004 27071 tablet_bootstrap.cc:492] T 3e489a04b06247608a195856b4739a36 P ff90de856dd542eab446899d181dcad0: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 1 replicates
I20260709 13:24:13.091877 27071 tablet_bootstrap.cc:492] T 3e489a04b06247608a195856b4739a36 P ff90de856dd542eab446899d181dcad0: Bootstrap complete.
I20260709 13:24:13.092478 27071 ts_tablet_manager.cc:1403] T 3e489a04b06247608a195856b4739a36 P ff90de856dd542eab446899d181dcad0: Time spent bootstrapping tablet: real 0.015s	user 0.007s	sys 0.007s
I20260709 13:24:13.094671 27071 raft_consensus.cc:359] T 3e489a04b06247608a195856b4739a36 P ff90de856dd542eab446899d181dcad0 [term 1 FOLLOWER]: Replica starting. Triggering 1 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff90de856dd542eab446899d181dcad0" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36311 } } peers { permanent_uuid: "326c74b0a8d84cdf86229f67ce3487f2" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35517 } } peers { permanent_uuid: "239ae18f3e44498fa52e20e9d6eec6b5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 39265 } }
I20260709 13:24:13.095389 27071 raft_consensus.cc:740] T 3e489a04b06247608a195856b4739a36 P ff90de856dd542eab446899d181dcad0 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: ff90de856dd542eab446899d181dcad0, State: Initialized, Role: FOLLOWER
I20260709 13:24:13.096087 27071 consensus_queue.cc:260] T 3e489a04b06247608a195856b4739a36 P ff90de856dd542eab446899d181dcad0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff90de856dd542eab446899d181dcad0" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36311 } } peers { permanent_uuid: "326c74b0a8d84cdf86229f67ce3487f2" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35517 } } peers { permanent_uuid: "239ae18f3e44498fa52e20e9d6eec6b5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 39265 } }
I20260709 13:24:13.098493 27071 ts_tablet_manager.cc:1434] T 3e489a04b06247608a195856b4739a36 P ff90de856dd542eab446899d181dcad0: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.001s
I20260709 13:24:13.099282 27071 tablet_bootstrap.cc:492] T 76c716abdcd6437b9dcee43317333298 P ff90de856dd542eab446899d181dcad0: Bootstrap starting.
I20260709 13:24:13.113258 27071 tablet_bootstrap.cc:492] T 76c716abdcd6437b9dcee43317333298 P ff90de856dd542eab446899d181dcad0: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 1 replicates
I20260709 13:24:13.114382 27071 tablet_bootstrap.cc:492] T 76c716abdcd6437b9dcee43317333298 P ff90de856dd542eab446899d181dcad0: Bootstrap complete.
I20260709 13:24:13.115017 15046 ts_tablet_manager.cc:616] Registered 10 tablets
I20260709 13:24:13.115072 27071 ts_tablet_manager.cc:1403] T 76c716abdcd6437b9dcee43317333298 P ff90de856dd542eab446899d181dcad0: Time spent bootstrapping tablet: real 0.016s	user 0.010s	sys 0.004s
I20260709 13:24:13.115674 15046 ts_tablet_manager.cc:595] Time spent register tablets: real 0.044s	user 0.039s	sys 0.005s
I20260709 13:24:13.118674 27071 raft_consensus.cc:359] T 76c716abdcd6437b9dcee43317333298 P ff90de856dd542eab446899d181dcad0 [term 1 FOLLOWER]: Replica starting. Triggering 1 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff90de856dd542eab446899d181dcad0" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36311 } } peers { permanent_uuid: "326c74b0a8d84cdf86229f67ce3487f2" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35517 } } peers { permanent_uuid: "239ae18f3e44498fa52e20e9d6eec6b5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 39265 } }
I20260709 13:24:13.119758 27071 raft_consensus.cc:740] T 76c716abdcd6437b9dcee43317333298 P ff90de856dd542eab446899d181dcad0 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: ff90de856dd542eab446899d181dcad0, State: Initialized, Role: FOLLOWER
I20260709 13:24:13.120587 27071 consensus_queue.cc:260] T 76c716abdcd6437b9dcee43317333298 P ff90de856dd542eab446899d181dcad0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff90de856dd542eab446899d181dcad0" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36311 } } peers { permanent_uuid: "326c74b0a8d84cdf86229f67ce3487f2" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35517 } } peers { permanent_uuid: "239ae18f3e44498fa52e20e9d6eec6b5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 39265 } }
I20260709 13:24:13.122915 27071 ts_tablet_manager.cc:1434] T 76c716abdcd6437b9dcee43317333298 P ff90de856dd542eab446899d181dcad0: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.000s
I20260709 13:24:13.123848 27071 tablet_bootstrap.cc:492] T 7492e977a5f74ef09aa687b1d7ec89e3 P ff90de856dd542eab446899d181dcad0: Bootstrap starting.
I20260709 13:24:13.141741 27071 tablet_bootstrap.cc:492] T 7492e977a5f74ef09aa687b1d7ec89e3 P ff90de856dd542eab446899d181dcad0: Bootstrap replayed 1/1 log segments. Stats: ops{read=0 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:24:13.142517 27071 tablet_bootstrap.cc:492] T 7492e977a5f74ef09aa687b1d7ec89e3 P ff90de856dd542eab446899d181dcad0: Bootstrap complete.
I20260709 13:24:13.143213 27071 ts_tablet_manager.cc:1403] T 7492e977a5f74ef09aa687b1d7ec89e3 P ff90de856dd542eab446899d181dcad0: Time spent bootstrapping tablet: real 0.020s	user 0.012s	sys 0.004s
I20260709 13:24:13.145485 27071 raft_consensus.cc:359] T 7492e977a5f74ef09aa687b1d7ec89e3 P ff90de856dd542eab446899d181dcad0 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff90de856dd542eab446899d181dcad0" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36311 } } peers { permanent_uuid: "326c74b0a8d84cdf86229f67ce3487f2" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35517 } } peers { permanent_uuid: "239ae18f3e44498fa52e20e9d6eec6b5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 39265 } }
I20260709 13:24:13.146077 27071 raft_consensus.cc:740] T 7492e977a5f74ef09aa687b1d7ec89e3 P ff90de856dd542eab446899d181dcad0 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: ff90de856dd542eab446899d181dcad0, State: Initialized, Role: FOLLOWER
I20260709 13:24:13.146796 27071 consensus_queue.cc:260] T 7492e977a5f74ef09aa687b1d7ec89e3 P ff90de856dd542eab446899d181dcad0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff90de856dd542eab446899d181dcad0" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36311 } } peers { permanent_uuid: "326c74b0a8d84cdf86229f67ce3487f2" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35517 } } peers { permanent_uuid: "239ae18f3e44498fa52e20e9d6eec6b5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 39265 } }
I20260709 13:24:13.148550 27071 ts_tablet_manager.cc:1434] T 7492e977a5f74ef09aa687b1d7ec89e3 P ff90de856dd542eab446899d181dcad0: Time spent starting tablet: real 0.005s	user 0.000s	sys 0.004s
I20260709 13:24:13.149251 27071 tablet_bootstrap.cc:492] T d52f26a0449247c1b928641ec8ebad34 P ff90de856dd542eab446899d181dcad0: Bootstrap starting.
I20260709 13:24:13.162475 27071 tablet_bootstrap.cc:492] T d52f26a0449247c1b928641ec8ebad34 P ff90de856dd542eab446899d181dcad0: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 1 replicates
I20260709 13:24:13.163372 27071 tablet_bootstrap.cc:492] T d52f26a0449247c1b928641ec8ebad34 P ff90de856dd542eab446899d181dcad0: Bootstrap complete.
I20260709 13:24:13.163920 27071 ts_tablet_manager.cc:1403] T d52f26a0449247c1b928641ec8ebad34 P ff90de856dd542eab446899d181dcad0: Time spent bootstrapping tablet: real 0.015s	user 0.007s	sys 0.007s
I20260709 13:24:13.166085 27071 raft_consensus.cc:359] T d52f26a0449247c1b928641ec8ebad34 P ff90de856dd542eab446899d181dcad0 [term 1 FOLLOWER]: Replica starting. Triggering 1 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff90de856dd542eab446899d181dcad0" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36311 } } peers { permanent_uuid: "326c74b0a8d84cdf86229f67ce3487f2" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35517 } } peers { permanent_uuid: "239ae18f3e44498fa52e20e9d6eec6b5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 39265 } }
I20260709 13:24:13.166738 27071 raft_consensus.cc:740] T d52f26a0449247c1b928641ec8ebad34 P ff90de856dd542eab446899d181dcad0 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: ff90de856dd542eab446899d181dcad0, State: Initialized, Role: FOLLOWER
I20260709 13:24:13.167426 27071 consensus_queue.cc:260] T d52f26a0449247c1b928641ec8ebad34 P ff90de856dd542eab446899d181dcad0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff90de856dd542eab446899d181dcad0" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36311 } } peers { permanent_uuid: "326c74b0a8d84cdf86229f67ce3487f2" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35517 } } peers { permanent_uuid: "239ae18f3e44498fa52e20e9d6eec6b5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 39265 } }
I20260709 13:24:13.168810 27071 ts_tablet_manager.cc:1434] T d52f26a0449247c1b928641ec8ebad34 P ff90de856dd542eab446899d181dcad0: Time spent starting tablet: real 0.005s	user 0.003s	sys 0.001s
I20260709 13:24:13.169523 27071 tablet_bootstrap.cc:492] T e341897686b14a2491c3a4bd24ac9e95 P ff90de856dd542eab446899d181dcad0: Bootstrap starting.
I20260709 13:24:13.181679 27071 tablet_bootstrap.cc:492] T e341897686b14a2491c3a4bd24ac9e95 P ff90de856dd542eab446899d181dcad0: Bootstrap replayed 1/1 log segments. Stats: ops{read=0 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:24:13.182420 27071 tablet_bootstrap.cc:492] T e341897686b14a2491c3a4bd24ac9e95 P ff90de856dd542eab446899d181dcad0: Bootstrap complete.
I20260709 13:24:13.182713 15046 rpc_server.cc:307] RPC server started. Bound to: 127.14.177.129:36311
I20260709 13:24:13.182816 27133 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.14.177.129:36311 every 8 connection(s)
I20260709 13:24:13.182976 27071 ts_tablet_manager.cc:1403] T e341897686b14a2491c3a4bd24ac9e95 P ff90de856dd542eab446899d181dcad0: Time spent bootstrapping tablet: real 0.014s	user 0.007s	sys 0.004s
I20260709 13:24:13.186002 27071 raft_consensus.cc:359] T e341897686b14a2491c3a4bd24ac9e95 P ff90de856dd542eab446899d181dcad0 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff90de856dd542eab446899d181dcad0" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36311 } } peers { permanent_uuid: "326c74b0a8d84cdf86229f67ce3487f2" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35517 } } peers { permanent_uuid: "239ae18f3e44498fa52e20e9d6eec6b5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 39265 } }
I20260709 13:24:13.186981 27071 raft_consensus.cc:740] T e341897686b14a2491c3a4bd24ac9e95 P ff90de856dd542eab446899d181dcad0 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: ff90de856dd542eab446899d181dcad0, State: Initialized, Role: FOLLOWER
I20260709 13:24:13.188107 27071 consensus_queue.cc:260] T e341897686b14a2491c3a4bd24ac9e95 P ff90de856dd542eab446899d181dcad0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff90de856dd542eab446899d181dcad0" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36311 } } peers { permanent_uuid: "326c74b0a8d84cdf86229f67ce3487f2" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35517 } } peers { permanent_uuid: "239ae18f3e44498fa52e20e9d6eec6b5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 39265 } }
I20260709 13:24:13.190883 27071 ts_tablet_manager.cc:1434] T e341897686b14a2491c3a4bd24ac9e95 P ff90de856dd542eab446899d181dcad0: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.000s
I20260709 13:24:13.192198 27071 tablet_bootstrap.cc:492] T 92c3adeb279b4f5f88680a40a9664750 P ff90de856dd542eab446899d181dcad0: Bootstrap starting.
I20260709 13:24:13.198985 15046 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:24:13.213935 27071 tablet_bootstrap.cc:492] T 92c3adeb279b4f5f88680a40a9664750 P ff90de856dd542eab446899d181dcad0: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:24:13.215222 27071 tablet_bootstrap.cc:492] T 92c3adeb279b4f5f88680a40a9664750 P ff90de856dd542eab446899d181dcad0: Bootstrap complete.
I20260709 13:24:13.216104 27071 ts_tablet_manager.cc:1403] T 92c3adeb279b4f5f88680a40a9664750 P ff90de856dd542eab446899d181dcad0: Time spent bootstrapping tablet: real 0.025s	user 0.018s	sys 0.004s
I20260709 13:24:13.219048 27071 raft_consensus.cc:359] T 92c3adeb279b4f5f88680a40a9664750 P ff90de856dd542eab446899d181dcad0 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff90de856dd542eab446899d181dcad0" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36311 } } peers { permanent_uuid: "326c74b0a8d84cdf86229f67ce3487f2" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35517 } } peers { permanent_uuid: "239ae18f3e44498fa52e20e9d6eec6b5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 39265 } }
I20260709 13:24:13.219749 27071 raft_consensus.cc:740] T 92c3adeb279b4f5f88680a40a9664750 P ff90de856dd542eab446899d181dcad0 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: ff90de856dd542eab446899d181dcad0, State: Initialized, Role: FOLLOWER
I20260709 13:24:13.220309 27071 consensus_queue.cc:260] T 92c3adeb279b4f5f88680a40a9664750 P ff90de856dd542eab446899d181dcad0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff90de856dd542eab446899d181dcad0" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36311 } } peers { permanent_uuid: "326c74b0a8d84cdf86229f67ce3487f2" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35517 } } peers { permanent_uuid: "239ae18f3e44498fa52e20e9d6eec6b5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 39265 } }
I20260709 13:24:13.222124 27071 ts_tablet_manager.cc:1434] T 92c3adeb279b4f5f88680a40a9664750 P ff90de856dd542eab446899d181dcad0: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.000s
I20260709 13:24:13.222942 27071 tablet_bootstrap.cc:492] T 2664382aca514208a9aa49316bdcaeeb P ff90de856dd542eab446899d181dcad0: Bootstrap starting.
I20260709 13:24:13.226997 26833 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7492e977a5f74ef09aa687b1d7ec89e3"
dest_uuid: "326c74b0a8d84cdf86229f67ce3487f2"
mode: GRACEFUL
new_leader_uuid: "239ae18f3e44498fa52e20e9d6eec6b5"
 from {username='slave'} at 127.0.0.1:47924
I20260709 13:24:13.227857 26833 raft_consensus.cc:606] T 7492e977a5f74ef09aa687b1d7ec89e3 P 326c74b0a8d84cdf86229f67ce3487f2 [term 1 LEADER]: Received request to transfer leadership to TS 239ae18f3e44498fa52e20e9d6eec6b5
I20260709 13:24:13.228825 27134 heartbeater.cc:344] Connected to a master server at 127.14.177.190:39149
I20260709 13:24:13.229293 27134 heartbeater.cc:461] Registering TS with master...
I20260709 13:24:13.230077 27134 heartbeater.cc:507] Master 127.14.177.190:39149 requested a full tablet report, sending...
I20260709 13:24:13.237807 27071 tablet_bootstrap.cc:492] T 2664382aca514208a9aa49316bdcaeeb P ff90de856dd542eab446899d181dcad0: Bootstrap replayed 1/1 log segments. Stats: ops{read=0 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:24:13.238005 26658 ts_manager.cc:194] Re-registered known tserver with Master: ff90de856dd542eab446899d181dcad0 (127.14.177.129:36311)
I20260709 13:24:13.238722 27071 tablet_bootstrap.cc:492] T 2664382aca514208a9aa49316bdcaeeb P ff90de856dd542eab446899d181dcad0: Bootstrap complete.
I20260709 13:24:13.239562 27071 ts_tablet_manager.cc:1403] T 2664382aca514208a9aa49316bdcaeeb P ff90de856dd542eab446899d181dcad0: Time spent bootstrapping tablet: real 0.017s	user 0.016s	sys 0.000s
I20260709 13:24:13.242637 27071 raft_consensus.cc:359] T 2664382aca514208a9aa49316bdcaeeb P ff90de856dd542eab446899d181dcad0 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff90de856dd542eab446899d181dcad0" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36311 } } peers { permanent_uuid: "326c74b0a8d84cdf86229f67ce3487f2" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35517 } } peers { permanent_uuid: "239ae18f3e44498fa52e20e9d6eec6b5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 39265 } }
I20260709 13:24:13.243386 27071 raft_consensus.cc:740] T 2664382aca514208a9aa49316bdcaeeb P ff90de856dd542eab446899d181dcad0 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: ff90de856dd542eab446899d181dcad0, State: Initialized, Role: FOLLOWER
I20260709 13:24:13.243953 26904 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "92c3adeb279b4f5f88680a40a9664750"
dest_uuid: "239ae18f3e44498fa52e20e9d6eec6b5"
mode: GRACEFUL
new_leader_uuid: "326c74b0a8d84cdf86229f67ce3487f2"
 from {username='slave'} at 127.0.0.1:38598
I20260709 13:24:13.244010 27071 consensus_queue.cc:260] T 2664382aca514208a9aa49316bdcaeeb P ff90de856dd542eab446899d181dcad0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff90de856dd542eab446899d181dcad0" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36311 } } peers { permanent_uuid: "326c74b0a8d84cdf86229f67ce3487f2" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35517 } } peers { permanent_uuid: "239ae18f3e44498fa52e20e9d6eec6b5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 39265 } }
I20260709 13:24:13.244729 26904 raft_consensus.cc:606] T 92c3adeb279b4f5f88680a40a9664750 P 239ae18f3e44498fa52e20e9d6eec6b5 [term 1 LEADER]: Received request to transfer leadership to TS 326c74b0a8d84cdf86229f67ce3487f2
I20260709 13:24:13.245991 27071 ts_tablet_manager.cc:1434] T 2664382aca514208a9aa49316bdcaeeb P ff90de856dd542eab446899d181dcad0: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 13:24:13.246929 27071 tablet_bootstrap.cc:492] T 9ba9365ebe904f85840f6d049dbcda88 P ff90de856dd542eab446899d181dcad0: Bootstrap starting.
I20260709 13:24:13.248186 26904 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9ba9365ebe904f85840f6d049dbcda88"
dest_uuid: "239ae18f3e44498fa52e20e9d6eec6b5"
mode: GRACEFUL
new_leader_uuid: "326c74b0a8d84cdf86229f67ce3487f2"
 from {username='slave'} at 127.0.0.1:38598
I20260709 13:24:13.248842 26904 raft_consensus.cc:606] T 9ba9365ebe904f85840f6d049dbcda88 P 239ae18f3e44498fa52e20e9d6eec6b5 [term 1 LEADER]: Received request to transfer leadership to TS 326c74b0a8d84cdf86229f67ce3487f2
I20260709 13:24:13.249492 26658 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:49340
I20260709 13:24:13.250177 15046 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 13:24:13.250949 15046 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:24:13.251778 15046 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:24:13.253870 27134 heartbeater.cc:499] Master 127.14.177.190:39149 was elected leader, sending a full tablet report...
I20260709 13:24:13.265637 27071 tablet_bootstrap.cc:492] T 9ba9365ebe904f85840f6d049dbcda88 P ff90de856dd542eab446899d181dcad0: Bootstrap replayed 1/1 log segments. Stats: ops{read=0 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:24:13.266556 27071 tablet_bootstrap.cc:492] T 9ba9365ebe904f85840f6d049dbcda88 P ff90de856dd542eab446899d181dcad0: Bootstrap complete.
I20260709 13:24:13.267355 27071 ts_tablet_manager.cc:1403] T 9ba9365ebe904f85840f6d049dbcda88 P ff90de856dd542eab446899d181dcad0: Time spent bootstrapping tablet: real 0.021s	user 0.020s	sys 0.000s
I20260709 13:24:13.269833 27071 raft_consensus.cc:359] T 9ba9365ebe904f85840f6d049dbcda88 P ff90de856dd542eab446899d181dcad0 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff90de856dd542eab446899d181dcad0" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36311 } } peers { permanent_uuid: "326c74b0a8d84cdf86229f67ce3487f2" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35517 } } peers { permanent_uuid: "239ae18f3e44498fa52e20e9d6eec6b5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 39265 } }
I20260709 13:24:13.270413 27071 raft_consensus.cc:740] T 9ba9365ebe904f85840f6d049dbcda88 P ff90de856dd542eab446899d181dcad0 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: ff90de856dd542eab446899d181dcad0, State: Initialized, Role: FOLLOWER
I20260709 13:24:13.271092 27071 consensus_queue.cc:260] T 9ba9365ebe904f85840f6d049dbcda88 P ff90de856dd542eab446899d181dcad0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff90de856dd542eab446899d181dcad0" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36311 } } peers { permanent_uuid: "326c74b0a8d84cdf86229f67ce3487f2" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35517 } } peers { permanent_uuid: "239ae18f3e44498fa52e20e9d6eec6b5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 39265 } }
I20260709 13:24:13.273221 27071 ts_tablet_manager.cc:1434] T 9ba9365ebe904f85840f6d049dbcda88 P ff90de856dd542eab446899d181dcad0: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 13:24:13.274046 27071 tablet_bootstrap.cc:492] T d5f6b403f0cb4c7a9180d07edb98d1bc P ff90de856dd542eab446899d181dcad0: Bootstrap starting.
I20260709 13:24:13.275825 27107 raft_consensus.cc:3060] T 3e489a04b06247608a195856b4739a36 P ff90de856dd542eab446899d181dcad0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:24:13.289592 27071 tablet_bootstrap.cc:492] T d5f6b403f0cb4c7a9180d07edb98d1bc P ff90de856dd542eab446899d181dcad0: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 1 replicates
I20260709 13:24:13.290618 27071 tablet_bootstrap.cc:492] T d5f6b403f0cb4c7a9180d07edb98d1bc P ff90de856dd542eab446899d181dcad0: Bootstrap complete.
I20260709 13:24:13.291332 27071 ts_tablet_manager.cc:1403] T d5f6b403f0cb4c7a9180d07edb98d1bc P ff90de856dd542eab446899d181dcad0: Time spent bootstrapping tablet: real 0.018s	user 0.017s	sys 0.000s
I20260709 13:24:13.293905 27071 raft_consensus.cc:359] T d5f6b403f0cb4c7a9180d07edb98d1bc P ff90de856dd542eab446899d181dcad0 [term 1 FOLLOWER]: Replica starting. Triggering 1 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "239ae18f3e44498fa52e20e9d6eec6b5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 39265 } } peers { permanent_uuid: "326c74b0a8d84cdf86229f67ce3487f2" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35517 } } peers { permanent_uuid: "ff90de856dd542eab446899d181dcad0" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36311 } }
I20260709 13:24:13.294771 27071 raft_consensus.cc:740] T d5f6b403f0cb4c7a9180d07edb98d1bc P ff90de856dd542eab446899d181dcad0 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: ff90de856dd542eab446899d181dcad0, State: Initialized, Role: FOLLOWER
I20260709 13:24:13.295508 27071 consensus_queue.cc:260] T d5f6b403f0cb4c7a9180d07edb98d1bc P ff90de856dd542eab446899d181dcad0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "239ae18f3e44498fa52e20e9d6eec6b5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 39265 } } peers { permanent_uuid: "326c74b0a8d84cdf86229f67ce3487f2" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35517 } } peers { permanent_uuid: "ff90de856dd542eab446899d181dcad0" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36311 } }
I20260709 13:24:13.297668 27071 ts_tablet_manager.cc:1434] T d5f6b403f0cb4c7a9180d07edb98d1bc P ff90de856dd542eab446899d181dcad0: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 13:24:13.298475 27071 tablet_bootstrap.cc:492] T bdb51cee932741eda8c12a88f07c890e P ff90de856dd542eab446899d181dcad0: Bootstrap starting.
I20260709 13:24:13.311094 27071 tablet_bootstrap.cc:492] T bdb51cee932741eda8c12a88f07c890e P ff90de856dd542eab446899d181dcad0: Bootstrap replayed 1/1 log segments. Stats: ops{read=0 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:24:13.311964 27071 tablet_bootstrap.cc:492] T bdb51cee932741eda8c12a88f07c890e P ff90de856dd542eab446899d181dcad0: Bootstrap complete.
I20260709 13:24:13.312531 27071 ts_tablet_manager.cc:1403] T bdb51cee932741eda8c12a88f07c890e P ff90de856dd542eab446899d181dcad0: Time spent bootstrapping tablet: real 0.014s	user 0.009s	sys 0.004s
I20260709 13:24:13.314602 27071 raft_consensus.cc:359] T bdb51cee932741eda8c12a88f07c890e P ff90de856dd542eab446899d181dcad0 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff90de856dd542eab446899d181dcad0" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36311 } } peers { permanent_uuid: "326c74b0a8d84cdf86229f67ce3487f2" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35517 } } peers { permanent_uuid: "239ae18f3e44498fa52e20e9d6eec6b5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 39265 } }
I20260709 13:24:13.315271 27071 raft_consensus.cc:740] T bdb51cee932741eda8c12a88f07c890e P ff90de856dd542eab446899d181dcad0 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: ff90de856dd542eab446899d181dcad0, State: Initialized, Role: FOLLOWER
I20260709 13:24:13.315901 27071 consensus_queue.cc:260] T bdb51cee932741eda8c12a88f07c890e P ff90de856dd542eab446899d181dcad0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff90de856dd542eab446899d181dcad0" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36311 } } peers { permanent_uuid: "326c74b0a8d84cdf86229f67ce3487f2" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35517 } } peers { permanent_uuid: "239ae18f3e44498fa52e20e9d6eec6b5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 39265 } }
I20260709 13:24:13.317368 27071 ts_tablet_manager.cc:1434] T bdb51cee932741eda8c12a88f07c890e P ff90de856dd542eab446899d181dcad0: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.000s
I20260709 13:24:13.329711 27055 raft_consensus.cc:993] T 7492e977a5f74ef09aa687b1d7ec89e3 P 326c74b0a8d84cdf86229f67ce3487f2: : Instructing follower 239ae18f3e44498fa52e20e9d6eec6b5 to start an election
I20260709 13:24:13.330260 27150 raft_consensus.cc:1081] T 7492e977a5f74ef09aa687b1d7ec89e3 P 326c74b0a8d84cdf86229f67ce3487f2 [term 1 LEADER]: Signalling peer 239ae18f3e44498fa52e20e9d6eec6b5 to start an election
I20260709 13:24:13.332098 26904 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "7492e977a5f74ef09aa687b1d7ec89e3"
dest_uuid: "239ae18f3e44498fa52e20e9d6eec6b5"
 from {username='slave'} at 127.0.0.1:49792
I20260709 13:24:13.332674 26904 raft_consensus.cc:493] T 7492e977a5f74ef09aa687b1d7ec89e3 P 239ae18f3e44498fa52e20e9d6eec6b5 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:24:13.332962 26904 raft_consensus.cc:3060] T 7492e977a5f74ef09aa687b1d7ec89e3 P 239ae18f3e44498fa52e20e9d6eec6b5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:24:13.340480 26904 raft_consensus.cc:515] T 7492e977a5f74ef09aa687b1d7ec89e3 P 239ae18f3e44498fa52e20e9d6eec6b5 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff90de856dd542eab446899d181dcad0" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36311 } } peers { permanent_uuid: "326c74b0a8d84cdf86229f67ce3487f2" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35517 } } peers { permanent_uuid: "239ae18f3e44498fa52e20e9d6eec6b5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 39265 } }
I20260709 13:24:13.342360 26904 leader_election.cc:290] T 7492e977a5f74ef09aa687b1d7ec89e3 P 239ae18f3e44498fa52e20e9d6eec6b5 [CANDIDATE]: Term 2 election: Requested vote from peers ff90de856dd542eab446899d181dcad0 (127.14.177.129:36311), 326c74b0a8d84cdf86229f67ce3487f2 (127.14.177.130:35517)
I20260709 13:24:13.343549 26833 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7492e977a5f74ef09aa687b1d7ec89e3" candidate_uuid: "239ae18f3e44498fa52e20e9d6eec6b5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "326c74b0a8d84cdf86229f67ce3487f2"
I20260709 13:24:13.344183 26833 raft_consensus.cc:3055] T 7492e977a5f74ef09aa687b1d7ec89e3 P 326c74b0a8d84cdf86229f67ce3487f2 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:24:13.344525 26833 raft_consensus.cc:740] T 7492e977a5f74ef09aa687b1d7ec89e3 P 326c74b0a8d84cdf86229f67ce3487f2 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 326c74b0a8d84cdf86229f67ce3487f2, State: Running, Role: LEADER
I20260709 13:24:13.345371 27106 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7492e977a5f74ef09aa687b1d7ec89e3" candidate_uuid: "239ae18f3e44498fa52e20e9d6eec6b5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "ff90de856dd542eab446899d181dcad0"
I20260709 13:24:13.345449 26833 consensus_queue.cc:260] T 7492e977a5f74ef09aa687b1d7ec89e3 P 326c74b0a8d84cdf86229f67ce3487f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff90de856dd542eab446899d181dcad0" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36311 } } peers { permanent_uuid: "326c74b0a8d84cdf86229f67ce3487f2" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35517 } } peers { permanent_uuid: "239ae18f3e44498fa52e20e9d6eec6b5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 39265 } }
I20260709 13:24:13.346185 27106 raft_consensus.cc:3060] T 7492e977a5f74ef09aa687b1d7ec89e3 P ff90de856dd542eab446899d181dcad0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:24:13.347091 26833 raft_consensus.cc:3060] T 7492e977a5f74ef09aa687b1d7ec89e3 P 326c74b0a8d84cdf86229f67ce3487f2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:24:13.355046 26833 raft_consensus.cc:2468] T 7492e977a5f74ef09aa687b1d7ec89e3 P 326c74b0a8d84cdf86229f67ce3487f2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 239ae18f3e44498fa52e20e9d6eec6b5 in term 2.
I20260709 13:24:13.355372 27106 raft_consensus.cc:2468] T 7492e977a5f74ef09aa687b1d7ec89e3 P ff90de856dd542eab446899d181dcad0 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 239ae18f3e44498fa52e20e9d6eec6b5 in term 2.
I20260709 13:24:13.356379 26872 leader_election.cc:304] T 7492e977a5f74ef09aa687b1d7ec89e3 P 239ae18f3e44498fa52e20e9d6eec6b5 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 239ae18f3e44498fa52e20e9d6eec6b5, 326c74b0a8d84cdf86229f67ce3487f2; no voters: 
I20260709 13:24:13.357271 26988 raft_consensus.cc:2804] T 7492e977a5f74ef09aa687b1d7ec89e3 P 239ae18f3e44498fa52e20e9d6eec6b5 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:24:13.358453 26988 raft_consensus.cc:697] T 7492e977a5f74ef09aa687b1d7ec89e3 P 239ae18f3e44498fa52e20e9d6eec6b5 [term 2 LEADER]: Becoming Leader. State: Replica: 239ae18f3e44498fa52e20e9d6eec6b5, State: Running, Role: LEADER
I20260709 13:24:13.359433 26988 consensus_queue.cc:237] T 7492e977a5f74ef09aa687b1d7ec89e3 P 239ae18f3e44498fa52e20e9d6eec6b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff90de856dd542eab446899d181dcad0" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36311 } } peers { permanent_uuid: "326c74b0a8d84cdf86229f67ce3487f2" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35517 } } peers { permanent_uuid: "239ae18f3e44498fa52e20e9d6eec6b5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 39265 } }
I20260709 13:24:13.367570 26658 catalog_manager.cc:5697] T 7492e977a5f74ef09aa687b1d7ec89e3 P 239ae18f3e44498fa52e20e9d6eec6b5 reported cstate change: term changed from 1 to 2, leader changed from 326c74b0a8d84cdf86229f67ce3487f2 (127.14.177.130) to 239ae18f3e44498fa52e20e9d6eec6b5 (127.14.177.131). New cstate: current_term: 2 leader_uuid: "239ae18f3e44498fa52e20e9d6eec6b5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff90de856dd542eab446899d181dcad0" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36311 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "326c74b0a8d84cdf86229f67ce3487f2" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35517 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "239ae18f3e44498fa52e20e9d6eec6b5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 39265 } health_report { overall_health: HEALTHY } } }
I20260709 13:24:13.387363 27152 raft_consensus.cc:993] T 92c3adeb279b4f5f88680a40a9664750 P 239ae18f3e44498fa52e20e9d6eec6b5: : Instructing follower 326c74b0a8d84cdf86229f67ce3487f2 to start an election
I20260709 13:24:13.387873 26988 raft_consensus.cc:1081] T 92c3adeb279b4f5f88680a40a9664750 P 239ae18f3e44498fa52e20e9d6eec6b5 [term 1 LEADER]: Signalling peer 326c74b0a8d84cdf86229f67ce3487f2 to start an election
I20260709 13:24:13.389513 26833 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "92c3adeb279b4f5f88680a40a9664750"
dest_uuid: "326c74b0a8d84cdf86229f67ce3487f2"
 from {username='slave'} at 127.0.0.1:40060
I20260709 13:24:13.390022 26833 raft_consensus.cc:493] T 92c3adeb279b4f5f88680a40a9664750 P 326c74b0a8d84cdf86229f67ce3487f2 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:24:13.390336 26833 raft_consensus.cc:3060] T 92c3adeb279b4f5f88680a40a9664750 P 326c74b0a8d84cdf86229f67ce3487f2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:24:13.396818 26833 raft_consensus.cc:515] T 92c3adeb279b4f5f88680a40a9664750 P 326c74b0a8d84cdf86229f67ce3487f2 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff90de856dd542eab446899d181dcad0" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36311 } } peers { permanent_uuid: "326c74b0a8d84cdf86229f67ce3487f2" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35517 } } peers { permanent_uuid: "239ae18f3e44498fa52e20e9d6eec6b5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 39265 } }
I20260709 13:24:13.399224 26833 leader_election.cc:290] T 92c3adeb279b4f5f88680a40a9664750 P 326c74b0a8d84cdf86229f67ce3487f2 [CANDIDATE]: Term 2 election: Requested vote from peers ff90de856dd542eab446899d181dcad0 (127.14.177.129:36311), 239ae18f3e44498fa52e20e9d6eec6b5 (127.14.177.131:39265)
I20260709 13:24:13.399638 27106 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92c3adeb279b4f5f88680a40a9664750" candidate_uuid: "326c74b0a8d84cdf86229f67ce3487f2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "ff90de856dd542eab446899d181dcad0"
I20260709 13:24:13.400336 27106 raft_consensus.cc:3060] T 92c3adeb279b4f5f88680a40a9664750 P ff90de856dd542eab446899d181dcad0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:24:13.401099 26904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92c3adeb279b4f5f88680a40a9664750" candidate_uuid: "326c74b0a8d84cdf86229f67ce3487f2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "239ae18f3e44498fa52e20e9d6eec6b5"
I20260709 13:24:13.401813 26904 raft_consensus.cc:3055] T 92c3adeb279b4f5f88680a40a9664750 P 239ae18f3e44498fa52e20e9d6eec6b5 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:24:13.402176 26904 raft_consensus.cc:740] T 92c3adeb279b4f5f88680a40a9664750 P 239ae18f3e44498fa52e20e9d6eec6b5 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 239ae18f3e44498fa52e20e9d6eec6b5, State: Running, Role: LEADER
I20260709 13:24:13.402987 26904 consensus_queue.cc:260] T 92c3adeb279b4f5f88680a40a9664750 P 239ae18f3e44498fa52e20e9d6eec6b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff90de856dd542eab446899d181dcad0" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36311 } } peers { permanent_uuid: "326c74b0a8d84cdf86229f67ce3487f2" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35517 } } peers { permanent_uuid: "239ae18f3e44498fa52e20e9d6eec6b5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 39265 } }
I20260709 13:24:13.404199 26904 raft_consensus.cc:3060] T 92c3adeb279b4f5f88680a40a9664750 P 239ae18f3e44498fa52e20e9d6eec6b5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:24:13.406105 27106 raft_consensus.cc:2468] T 92c3adeb279b4f5f88680a40a9664750 P ff90de856dd542eab446899d181dcad0 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 326c74b0a8d84cdf86229f67ce3487f2 in term 2.
I20260709 13:24:13.407228 26798 leader_election.cc:304] T 92c3adeb279b4f5f88680a40a9664750 P 326c74b0a8d84cdf86229f67ce3487f2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 326c74b0a8d84cdf86229f67ce3487f2, ff90de856dd542eab446899d181dcad0; no voters: 
I20260709 13:24:13.407943 27150 raft_consensus.cc:2804] T 92c3adeb279b4f5f88680a40a9664750 P 326c74b0a8d84cdf86229f67ce3487f2 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:24:13.408406 27150 raft_consensus.cc:697] T 92c3adeb279b4f5f88680a40a9664750 P 326c74b0a8d84cdf86229f67ce3487f2 [term 2 LEADER]: Becoming Leader. State: Replica: 326c74b0a8d84cdf86229f67ce3487f2, State: Running, Role: LEADER
I20260709 13:24:13.409332 27150 consensus_queue.cc:237] T 92c3adeb279b4f5f88680a40a9664750 P 326c74b0a8d84cdf86229f67ce3487f2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff90de856dd542eab446899d181dcad0" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36311 } } peers { permanent_uuid: "326c74b0a8d84cdf86229f67ce3487f2" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35517 } } peers { permanent_uuid: "239ae18f3e44498fa52e20e9d6eec6b5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 39265 } }
I20260709 13:24:13.410104 26904 raft_consensus.cc:2468] T 92c3adeb279b4f5f88680a40a9664750 P 239ae18f3e44498fa52e20e9d6eec6b5 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 326c74b0a8d84cdf86229f67ce3487f2 in term 2.
I20260709 13:24:13.418143 26658 catalog_manager.cc:5697] T 92c3adeb279b4f5f88680a40a9664750 P 326c74b0a8d84cdf86229f67ce3487f2 reported cstate change: term changed from 1 to 2, leader changed from 239ae18f3e44498fa52e20e9d6eec6b5 (127.14.177.131) to 326c74b0a8d84cdf86229f67ce3487f2 (127.14.177.130). New cstate: current_term: 2 leader_uuid: "326c74b0a8d84cdf86229f67ce3487f2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff90de856dd542eab446899d181dcad0" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36311 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "326c74b0a8d84cdf86229f67ce3487f2" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35517 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "239ae18f3e44498fa52e20e9d6eec6b5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 39265 } health_report { overall_health: UNKNOWN } } }
I20260709 13:24:13.435102 27152 raft_consensus.cc:993] T 9ba9365ebe904f85840f6d049dbcda88 P 239ae18f3e44498fa52e20e9d6eec6b5: : Instructing follower 326c74b0a8d84cdf86229f67ce3487f2 to start an election
I20260709 13:24:13.435642 26988 raft_consensus.cc:1081] T 9ba9365ebe904f85840f6d049dbcda88 P 239ae18f3e44498fa52e20e9d6eec6b5 [term 1 LEADER]: Signalling peer 326c74b0a8d84cdf86229f67ce3487f2 to start an election
I20260709 13:24:13.437312 26834 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "9ba9365ebe904f85840f6d049dbcda88"
dest_uuid: "326c74b0a8d84cdf86229f67ce3487f2"
 from {username='slave'} at 127.0.0.1:40060
I20260709 13:24:13.437871 26834 raft_consensus.cc:493] T 9ba9365ebe904f85840f6d049dbcda88 P 326c74b0a8d84cdf86229f67ce3487f2 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:24:13.438195 26834 raft_consensus.cc:3060] T 9ba9365ebe904f85840f6d049dbcda88 P 326c74b0a8d84cdf86229f67ce3487f2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:24:13.444895 26834 raft_consensus.cc:515] T 9ba9365ebe904f85840f6d049dbcda88 P 326c74b0a8d84cdf86229f67ce3487f2 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff90de856dd542eab446899d181dcad0" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36311 } } peers { permanent_uuid: "326c74b0a8d84cdf86229f67ce3487f2" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35517 } } peers { permanent_uuid: "239ae18f3e44498fa52e20e9d6eec6b5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 39265 } }
I20260709 13:24:13.446554 26834 leader_election.cc:290] T 9ba9365ebe904f85840f6d049dbcda88 P 326c74b0a8d84cdf86229f67ce3487f2 [CANDIDATE]: Term 2 election: Requested vote from peers ff90de856dd542eab446899d181dcad0 (127.14.177.129:36311), 239ae18f3e44498fa52e20e9d6eec6b5 (127.14.177.131:39265)
I20260709 13:24:13.447525 27106 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ba9365ebe904f85840f6d049dbcda88" candidate_uuid: "326c74b0a8d84cdf86229f67ce3487f2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "ff90de856dd542eab446899d181dcad0"
I20260709 13:24:13.447772 26904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ba9365ebe904f85840f6d049dbcda88" candidate_uuid: "326c74b0a8d84cdf86229f67ce3487f2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "239ae18f3e44498fa52e20e9d6eec6b5"
I20260709 13:24:13.448225 27106 raft_consensus.cc:3060] T 9ba9365ebe904f85840f6d049dbcda88 P ff90de856dd542eab446899d181dcad0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:24:13.448413 26904 raft_consensus.cc:3055] T 9ba9365ebe904f85840f6d049dbcda88 P 239ae18f3e44498fa52e20e9d6eec6b5 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:24:13.448801 26904 raft_consensus.cc:740] T 9ba9365ebe904f85840f6d049dbcda88 P 239ae18f3e44498fa52e20e9d6eec6b5 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 239ae18f3e44498fa52e20e9d6eec6b5, State: Running, Role: LEADER
I20260709 13:24:13.449522 26904 consensus_queue.cc:260] T 9ba9365ebe904f85840f6d049dbcda88 P 239ae18f3e44498fa52e20e9d6eec6b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff90de856dd542eab446899d181dcad0" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36311 } } peers { permanent_uuid: "326c74b0a8d84cdf86229f67ce3487f2" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35517 } } peers { permanent_uuid: "239ae18f3e44498fa52e20e9d6eec6b5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 39265 } }
I20260709 13:24:13.450520 26904 raft_consensus.cc:3060] T 9ba9365ebe904f85840f6d049dbcda88 P 239ae18f3e44498fa52e20e9d6eec6b5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:24:13.453943 27106 raft_consensus.cc:2468] T 9ba9365ebe904f85840f6d049dbcda88 P ff90de856dd542eab446899d181dcad0 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 326c74b0a8d84cdf86229f67ce3487f2 in term 2.
I20260709 13:24:13.455286 26798 leader_election.cc:304] T 9ba9365ebe904f85840f6d049dbcda88 P 326c74b0a8d84cdf86229f67ce3487f2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 326c74b0a8d84cdf86229f67ce3487f2, ff90de856dd542eab446899d181dcad0; no voters: 
I20260709 13:24:13.455997 26904 raft_consensus.cc:2468] T 9ba9365ebe904f85840f6d049dbcda88 P 239ae18f3e44498fa52e20e9d6eec6b5 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 326c74b0a8d84cdf86229f67ce3487f2 in term 2.
I20260709 13:24:13.456171 27150 raft_consensus.cc:2804] T 9ba9365ebe904f85840f6d049dbcda88 P 326c74b0a8d84cdf86229f67ce3487f2 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:24:13.456773 27150 raft_consensus.cc:697] T 9ba9365ebe904f85840f6d049dbcda88 P 326c74b0a8d84cdf86229f67ce3487f2 [term 2 LEADER]: Becoming Leader. State: Replica: 326c74b0a8d84cdf86229f67ce3487f2, State: Running, Role: LEADER
I20260709 13:24:13.457625 27150 consensus_queue.cc:237] T 9ba9365ebe904f85840f6d049dbcda88 P 326c74b0a8d84cdf86229f67ce3487f2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff90de856dd542eab446899d181dcad0" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36311 } } peers { permanent_uuid: "326c74b0a8d84cdf86229f67ce3487f2" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35517 } } peers { permanent_uuid: "239ae18f3e44498fa52e20e9d6eec6b5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 39265 } }
I20260709 13:24:13.465608 26658 catalog_manager.cc:5697] T 9ba9365ebe904f85840f6d049dbcda88 P 326c74b0a8d84cdf86229f67ce3487f2 reported cstate change: term changed from 1 to 2, leader changed from 239ae18f3e44498fa52e20e9d6eec6b5 (127.14.177.131) to 326c74b0a8d84cdf86229f67ce3487f2 (127.14.177.130). New cstate: current_term: 2 leader_uuid: "326c74b0a8d84cdf86229f67ce3487f2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff90de856dd542eab446899d181dcad0" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36311 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "326c74b0a8d84cdf86229f67ce3487f2" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35517 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "239ae18f3e44498fa52e20e9d6eec6b5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 39265 } health_report { overall_health: UNKNOWN } } }
I20260709 13:24:13.467685 27106 raft_consensus.cc:3060] T 76c716abdcd6437b9dcee43317333298 P ff90de856dd542eab446899d181dcad0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:24:13.474372 27106 pending_rounds.cc:85] T 76c716abdcd6437b9dcee43317333298 P ff90de856dd542eab446899d181dcad0: Aborting all ops after (but not including) 0
I20260709 13:24:13.474673 27106 pending_rounds.cc:107] T 76c716abdcd6437b9dcee43317333298 P ff90de856dd542eab446899d181dcad0: Aborting uncommitted NO_OP operation due to leader change: 1.1
I20260709 13:24:13.474949 27106 raft_consensus.cc:2889] T 76c716abdcd6437b9dcee43317333298 P ff90de856dd542eab446899d181dcad0 [term 2 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:24:13.531080 27106 raft_consensus.cc:3060] T d52f26a0449247c1b928641ec8ebad34 P ff90de856dd542eab446899d181dcad0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:24:13.536506 27106 pending_rounds.cc:85] T d52f26a0449247c1b928641ec8ebad34 P ff90de856dd542eab446899d181dcad0: Aborting all ops after (but not including) 0
I20260709 13:24:13.536742 27106 pending_rounds.cc:107] T d52f26a0449247c1b928641ec8ebad34 P ff90de856dd542eab446899d181dcad0: Aborting uncommitted NO_OP operation due to leader change: 1.1
I20260709 13:24:13.536916 27106 raft_consensus.cc:2889] T d52f26a0449247c1b928641ec8ebad34 P ff90de856dd542eab446899d181dcad0 [term 2 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:24:13.548290 27106 raft_consensus.cc:3060] T d5f6b403f0cb4c7a9180d07edb98d1bc P ff90de856dd542eab446899d181dcad0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:24:13.553375 27106 pending_rounds.cc:85] T d5f6b403f0cb4c7a9180d07edb98d1bc P ff90de856dd542eab446899d181dcad0: Aborting all ops after (but not including) 0
I20260709 13:24:13.553591 27106 pending_rounds.cc:107] T d5f6b403f0cb4c7a9180d07edb98d1bc P ff90de856dd542eab446899d181dcad0: Aborting uncommitted NO_OP operation due to leader change: 1.1
I20260709 13:24:13.553857 27106 raft_consensus.cc:2889] T d5f6b403f0cb4c7a9180d07edb98d1bc P ff90de856dd542eab446899d181dcad0 [term 2 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:24:13.811515 26706 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 13:24:13.889298 27107 raft_consensus.cc:1275] T 7492e977a5f74ef09aa687b1d7ec89e3 P ff90de856dd542eab446899d181dcad0 [term 2 FOLLOWER]: Refusing update from remote peer 239ae18f3e44498fa52e20e9d6eec6b5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:24:13.890885 27152 consensus_queue.cc:1048] T 7492e977a5f74ef09aa687b1d7ec89e3 P 239ae18f3e44498fa52e20e9d6eec6b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff90de856dd542eab446899d181dcad0" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36311 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:24:13.900309 26834 raft_consensus.cc:1275] T 7492e977a5f74ef09aa687b1d7ec89e3 P 326c74b0a8d84cdf86229f67ce3487f2 [term 2 FOLLOWER]: Refusing update from remote peer 239ae18f3e44498fa52e20e9d6eec6b5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:24:13.902136 26988 consensus_queue.cc:1048] T 7492e977a5f74ef09aa687b1d7ec89e3 P 239ae18f3e44498fa52e20e9d6eec6b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "326c74b0a8d84cdf86229f67ce3487f2" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35517 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:24:13.909243 26904 raft_consensus.cc:1275] T 9ba9365ebe904f85840f6d049dbcda88 P 239ae18f3e44498fa52e20e9d6eec6b5 [term 2 FOLLOWER]: Refusing update from remote peer 326c74b0a8d84cdf86229f67ce3487f2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:24:13.911317 27107 raft_consensus.cc:1275] T 9ba9365ebe904f85840f6d049dbcda88 P ff90de856dd542eab446899d181dcad0 [term 2 FOLLOWER]: Refusing update from remote peer 326c74b0a8d84cdf86229f67ce3487f2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:24:13.911353 27055 consensus_queue.cc:1048] T 9ba9365ebe904f85840f6d049dbcda88 P 326c74b0a8d84cdf86229f67ce3487f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "239ae18f3e44498fa52e20e9d6eec6b5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 39265 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:24:13.913512 27167 consensus_queue.cc:1048] T 9ba9365ebe904f85840f6d049dbcda88 P 326c74b0a8d84cdf86229f67ce3487f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff90de856dd542eab446899d181dcad0" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36311 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:24:13.927947 26909 raft_consensus.cc:1275] T 92c3adeb279b4f5f88680a40a9664750 P 239ae18f3e44498fa52e20e9d6eec6b5 [term 2 FOLLOWER]: Refusing update from remote peer 326c74b0a8d84cdf86229f67ce3487f2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:24:13.929967 27167 consensus_queue.cc:1048] T 92c3adeb279b4f5f88680a40a9664750 P 326c74b0a8d84cdf86229f67ce3487f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "239ae18f3e44498fa52e20e9d6eec6b5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 39265 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:24:13.964126 27107 raft_consensus.cc:1275] T 92c3adeb279b4f5f88680a40a9664750 P ff90de856dd542eab446899d181dcad0 [term 2 FOLLOWER]: Refusing update from remote peer 326c74b0a8d84cdf86229f67ce3487f2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:24:13.966004 27173 consensus_queue.cc:1048] T 92c3adeb279b4f5f88680a40a9664750 P 326c74b0a8d84cdf86229f67ce3487f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff90de856dd542eab446899d181dcad0" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36311 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:24:14.262859 15046 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:24:14.270304 26834 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "92c3adeb279b4f5f88680a40a9664750"
dest_uuid: "326c74b0a8d84cdf86229f67ce3487f2"
mode: GRACEFUL
new_leader_uuid: "239ae18f3e44498fa52e20e9d6eec6b5"
 from {username='slave'} at 127.0.0.1:47924
I20260709 13:24:14.270825 26834 raft_consensus.cc:606] T 92c3adeb279b4f5f88680a40a9664750 P 326c74b0a8d84cdf86229f67ce3487f2 [term 2 LEADER]: Received request to transfer leadership to TS 239ae18f3e44498fa52e20e9d6eec6b5
I20260709 13:24:14.273413 26834 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d52f26a0449247c1b928641ec8ebad34"
dest_uuid: "326c74b0a8d84cdf86229f67ce3487f2"
mode: GRACEFUL
new_leader_uuid: "239ae18f3e44498fa52e20e9d6eec6b5"
 from {username='slave'} at 127.0.0.1:47924
I20260709 13:24:14.273921 26834 raft_consensus.cc:606] T d52f26a0449247c1b928641ec8ebad34 P 326c74b0a8d84cdf86229f67ce3487f2 [term 2 LEADER]: Received request to transfer leadership to TS 239ae18f3e44498fa52e20e9d6eec6b5
I20260709 13:24:14.276499 26904 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d5f6b403f0cb4c7a9180d07edb98d1bc"
dest_uuid: "239ae18f3e44498fa52e20e9d6eec6b5"
mode: GRACEFUL
new_leader_uuid: "326c74b0a8d84cdf86229f67ce3487f2"
 from {username='slave'} at 127.0.0.1:38598
I20260709 13:24:14.277024 26904 raft_consensus.cc:606] T d5f6b403f0cb4c7a9180d07edb98d1bc P 239ae18f3e44498fa52e20e9d6eec6b5 [term 2 LEADER]: Received request to transfer leadership to TS 326c74b0a8d84cdf86229f67ce3487f2
I20260709 13:24:14.278522 15046 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 13:24:14.279039 15046 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:24:14.279630 15046 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:24:14.283914 27167 raft_consensus.cc:993] T d52f26a0449247c1b928641ec8ebad34 P 326c74b0a8d84cdf86229f67ce3487f2: : Instructing follower 239ae18f3e44498fa52e20e9d6eec6b5 to start an election
I20260709 13:24:14.284569 27173 raft_consensus.cc:1081] T d52f26a0449247c1b928641ec8ebad34 P 326c74b0a8d84cdf86229f67ce3487f2 [term 2 LEADER]: Signalling peer 239ae18f3e44498fa52e20e9d6eec6b5 to start an election
I20260709 13:24:14.286361 26904 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d52f26a0449247c1b928641ec8ebad34"
dest_uuid: "239ae18f3e44498fa52e20e9d6eec6b5"
 from {username='slave'} at 127.0.0.1:49792
I20260709 13:24:14.286906 26904 raft_consensus.cc:493] T d52f26a0449247c1b928641ec8ebad34 P 239ae18f3e44498fa52e20e9d6eec6b5 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:24:14.287266 26904 raft_consensus.cc:3060] T d52f26a0449247c1b928641ec8ebad34 P 239ae18f3e44498fa52e20e9d6eec6b5 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:24:14.293272 26904 raft_consensus.cc:515] T d52f26a0449247c1b928641ec8ebad34 P 239ae18f3e44498fa52e20e9d6eec6b5 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff90de856dd542eab446899d181dcad0" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36311 } } peers { permanent_uuid: "326c74b0a8d84cdf86229f67ce3487f2" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35517 } } peers { permanent_uuid: "239ae18f3e44498fa52e20e9d6eec6b5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 39265 } }
I20260709 13:24:14.294924 26904 leader_election.cc:290] T d52f26a0449247c1b928641ec8ebad34 P 239ae18f3e44498fa52e20e9d6eec6b5 [CANDIDATE]: Term 3 election: Requested vote from peers ff90de856dd542eab446899d181dcad0 (127.14.177.129:36311), 326c74b0a8d84cdf86229f67ce3487f2 (127.14.177.130:35517)
I20260709 13:24:14.295758 27107 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d52f26a0449247c1b928641ec8ebad34" candidate_uuid: "239ae18f3e44498fa52e20e9d6eec6b5" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: true dest_uuid: "ff90de856dd542eab446899d181dcad0"
I20260709 13:24:14.296130 26834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d52f26a0449247c1b928641ec8ebad34" candidate_uuid: "239ae18f3e44498fa52e20e9d6eec6b5" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: true dest_uuid: "326c74b0a8d84cdf86229f67ce3487f2"
I20260709 13:24:14.296435 27107 raft_consensus.cc:3060] T d52f26a0449247c1b928641ec8ebad34 P ff90de856dd542eab446899d181dcad0 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:24:14.296757 26834 raft_consensus.cc:3055] T d52f26a0449247c1b928641ec8ebad34 P 326c74b0a8d84cdf86229f67ce3487f2 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:24:14.297080 26834 raft_consensus.cc:740] T d52f26a0449247c1b928641ec8ebad34 P 326c74b0a8d84cdf86229f67ce3487f2 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 326c74b0a8d84cdf86229f67ce3487f2, State: Running, Role: LEADER
I20260709 13:24:14.297930 26834 consensus_queue.cc:260] T d52f26a0449247c1b928641ec8ebad34 P 326c74b0a8d84cdf86229f67ce3487f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff90de856dd542eab446899d181dcad0" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36311 } } peers { permanent_uuid: "326c74b0a8d84cdf86229f67ce3487f2" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35517 } } peers { permanent_uuid: "239ae18f3e44498fa52e20e9d6eec6b5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 39265 } }
I20260709 13:24:14.299286 26834 raft_consensus.cc:3060] T d52f26a0449247c1b928641ec8ebad34 P 326c74b0a8d84cdf86229f67ce3487f2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:24:14.304232 27107 raft_consensus.cc:2468] T d52f26a0449247c1b928641ec8ebad34 P ff90de856dd542eab446899d181dcad0 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 239ae18f3e44498fa52e20e9d6eec6b5 in term 3.
I20260709 13:24:14.305747 26872 leader_election.cc:304] T d52f26a0449247c1b928641ec8ebad34 P 239ae18f3e44498fa52e20e9d6eec6b5 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 239ae18f3e44498fa52e20e9d6eec6b5, ff90de856dd542eab446899d181dcad0; no voters: 
I20260709 13:24:14.306594 27166 raft_consensus.cc:2804] T d52f26a0449247c1b928641ec8ebad34 P 239ae18f3e44498fa52e20e9d6eec6b5 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:24:14.307183 27166 raft_consensus.cc:697] T d52f26a0449247c1b928641ec8ebad34 P 239ae18f3e44498fa52e20e9d6eec6b5 [term 3 LEADER]: Becoming Leader. State: Replica: 239ae18f3e44498fa52e20e9d6eec6b5, State: Running, Role: LEADER
I20260709 13:24:14.308002 27166 consensus_queue.cc:237] T d52f26a0449247c1b928641ec8ebad34 P 239ae18f3e44498fa52e20e9d6eec6b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff90de856dd542eab446899d181dcad0" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36311 } } peers { permanent_uuid: "326c74b0a8d84cdf86229f67ce3487f2" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35517 } } peers { permanent_uuid: "239ae18f3e44498fa52e20e9d6eec6b5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 39265 } }
I20260709 13:24:14.309988 26834 raft_consensus.cc:2468] T d52f26a0449247c1b928641ec8ebad34 P 326c74b0a8d84cdf86229f67ce3487f2 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 239ae18f3e44498fa52e20e9d6eec6b5 in term 3.
I20260709 13:24:14.315551 26655 catalog_manager.cc:5697] T d52f26a0449247c1b928641ec8ebad34 P 239ae18f3e44498fa52e20e9d6eec6b5 reported cstate change: term changed from 2 to 3, leader changed from 326c74b0a8d84cdf86229f67ce3487f2 (127.14.177.130) to 239ae18f3e44498fa52e20e9d6eec6b5 (127.14.177.131). New cstate: current_term: 3 leader_uuid: "239ae18f3e44498fa52e20e9d6eec6b5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff90de856dd542eab446899d181dcad0" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36311 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "326c74b0a8d84cdf86229f67ce3487f2" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35517 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "239ae18f3e44498fa52e20e9d6eec6b5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 39265 } health_report { overall_health: HEALTHY } } }
I20260709 13:24:14.337100 27167 raft_consensus.cc:993] T 92c3adeb279b4f5f88680a40a9664750 P 326c74b0a8d84cdf86229f67ce3487f2: : Instructing follower 239ae18f3e44498fa52e20e9d6eec6b5 to start an election
I20260709 13:24:14.337601 27167 raft_consensus.cc:1081] T 92c3adeb279b4f5f88680a40a9664750 P 326c74b0a8d84cdf86229f67ce3487f2 [term 2 LEADER]: Signalling peer 239ae18f3e44498fa52e20e9d6eec6b5 to start an election
I20260709 13:24:14.339121 26904 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "92c3adeb279b4f5f88680a40a9664750"
dest_uuid: "239ae18f3e44498fa52e20e9d6eec6b5"
 from {username='slave'} at 127.0.0.1:49792
I20260709 13:24:14.339632 26904 raft_consensus.cc:493] T 92c3adeb279b4f5f88680a40a9664750 P 239ae18f3e44498fa52e20e9d6eec6b5 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:24:14.339871 26904 raft_consensus.cc:3060] T 92c3adeb279b4f5f88680a40a9664750 P 239ae18f3e44498fa52e20e9d6eec6b5 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:24:14.344807 26904 raft_consensus.cc:515] T 92c3adeb279b4f5f88680a40a9664750 P 239ae18f3e44498fa52e20e9d6eec6b5 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff90de856dd542eab446899d181dcad0" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36311 } } peers { permanent_uuid: "326c74b0a8d84cdf86229f67ce3487f2" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35517 } } peers { permanent_uuid: "239ae18f3e44498fa52e20e9d6eec6b5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 39265 } }
I20260709 13:24:14.346494 26904 leader_election.cc:290] T 92c3adeb279b4f5f88680a40a9664750 P 239ae18f3e44498fa52e20e9d6eec6b5 [CANDIDATE]: Term 3 election: Requested vote from peers ff90de856dd542eab446899d181dcad0 (127.14.177.129:36311), 326c74b0a8d84cdf86229f67ce3487f2 (127.14.177.130:35517)
I20260709 13:24:14.347517 27107 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92c3adeb279b4f5f88680a40a9664750" candidate_uuid: "239ae18f3e44498fa52e20e9d6eec6b5" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "ff90de856dd542eab446899d181dcad0"
I20260709 13:24:14.347818 26834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92c3adeb279b4f5f88680a40a9664750" candidate_uuid: "239ae18f3e44498fa52e20e9d6eec6b5" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "326c74b0a8d84cdf86229f67ce3487f2"
I20260709 13:24:14.348102 27107 raft_consensus.cc:3060] T 92c3adeb279b4f5f88680a40a9664750 P ff90de856dd542eab446899d181dcad0 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:24:14.348495 26834 raft_consensus.cc:3055] T 92c3adeb279b4f5f88680a40a9664750 P 326c74b0a8d84cdf86229f67ce3487f2 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:24:14.348812 26834 raft_consensus.cc:740] T 92c3adeb279b4f5f88680a40a9664750 P 326c74b0a8d84cdf86229f67ce3487f2 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 326c74b0a8d84cdf86229f67ce3487f2, State: Running, Role: LEADER
I20260709 13:24:14.349584 26834 consensus_queue.cc:260] T 92c3adeb279b4f5f88680a40a9664750 P 326c74b0a8d84cdf86229f67ce3487f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff90de856dd542eab446899d181dcad0" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36311 } } peers { permanent_uuid: "326c74b0a8d84cdf86229f67ce3487f2" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35517 } } peers { permanent_uuid: "239ae18f3e44498fa52e20e9d6eec6b5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 39265 } }
I20260709 13:24:14.350835 26834 raft_consensus.cc:3060] T 92c3adeb279b4f5f88680a40a9664750 P 326c74b0a8d84cdf86229f67ce3487f2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:24:14.353979 27107 raft_consensus.cc:2468] T 92c3adeb279b4f5f88680a40a9664750 P ff90de856dd542eab446899d181dcad0 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 239ae18f3e44498fa52e20e9d6eec6b5 in term 3.
I20260709 13:24:14.355253 26872 leader_election.cc:304] T 92c3adeb279b4f5f88680a40a9664750 P 239ae18f3e44498fa52e20e9d6eec6b5 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 239ae18f3e44498fa52e20e9d6eec6b5, ff90de856dd542eab446899d181dcad0; no voters: 
I20260709 13:24:14.356112 27166 raft_consensus.cc:2804] T 92c3adeb279b4f5f88680a40a9664750 P 239ae18f3e44498fa52e20e9d6eec6b5 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:24:14.356673 27166 raft_consensus.cc:697] T 92c3adeb279b4f5f88680a40a9664750 P 239ae18f3e44498fa52e20e9d6eec6b5 [term 3 LEADER]: Becoming Leader. State: Replica: 239ae18f3e44498fa52e20e9d6eec6b5, State: Running, Role: LEADER
I20260709 13:24:14.357570 26834 raft_consensus.cc:2468] T 92c3adeb279b4f5f88680a40a9664750 P 326c74b0a8d84cdf86229f67ce3487f2 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 239ae18f3e44498fa52e20e9d6eec6b5 in term 3.
I20260709 13:24:14.357489 27166 consensus_queue.cc:237] T 92c3adeb279b4f5f88680a40a9664750 P 239ae18f3e44498fa52e20e9d6eec6b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff90de856dd542eab446899d181dcad0" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36311 } } peers { permanent_uuid: "326c74b0a8d84cdf86229f67ce3487f2" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35517 } } peers { permanent_uuid: "239ae18f3e44498fa52e20e9d6eec6b5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 39265 } }
I20260709 13:24:14.364151 26655 catalog_manager.cc:5697] T 92c3adeb279b4f5f88680a40a9664750 P 239ae18f3e44498fa52e20e9d6eec6b5 reported cstate change: term changed from 2 to 3, leader changed from 326c74b0a8d84cdf86229f67ce3487f2 (127.14.177.130) to 239ae18f3e44498fa52e20e9d6eec6b5 (127.14.177.131). New cstate: current_term: 3 leader_uuid: "239ae18f3e44498fa52e20e9d6eec6b5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff90de856dd542eab446899d181dcad0" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36311 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "326c74b0a8d84cdf86229f67ce3487f2" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35517 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "239ae18f3e44498fa52e20e9d6eec6b5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 39265 } health_report { overall_health: HEALTHY } } }
I20260709 13:24:14.413749 27152 raft_consensus.cc:993] T d5f6b403f0cb4c7a9180d07edb98d1bc P 239ae18f3e44498fa52e20e9d6eec6b5: : Instructing follower 326c74b0a8d84cdf86229f67ce3487f2 to start an election
I20260709 13:24:14.414312 27166 raft_consensus.cc:1081] T d5f6b403f0cb4c7a9180d07edb98d1bc P 239ae18f3e44498fa52e20e9d6eec6b5 [term 2 LEADER]: Signalling peer 326c74b0a8d84cdf86229f67ce3487f2 to start an election
I20260709 13:24:14.415861 26834 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d5f6b403f0cb4c7a9180d07edb98d1bc"
dest_uuid: "326c74b0a8d84cdf86229f67ce3487f2"
 from {username='slave'} at 127.0.0.1:40060
I20260709 13:24:14.416414 26834 raft_consensus.cc:493] T d5f6b403f0cb4c7a9180d07edb98d1bc P 326c74b0a8d84cdf86229f67ce3487f2 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:24:14.416725 26834 raft_consensus.cc:3060] T d5f6b403f0cb4c7a9180d07edb98d1bc P 326c74b0a8d84cdf86229f67ce3487f2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:24:14.422710 26834 raft_consensus.cc:515] T d5f6b403f0cb4c7a9180d07edb98d1bc P 326c74b0a8d84cdf86229f67ce3487f2 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "239ae18f3e44498fa52e20e9d6eec6b5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 39265 } } peers { permanent_uuid: "326c74b0a8d84cdf86229f67ce3487f2" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35517 } } peers { permanent_uuid: "ff90de856dd542eab446899d181dcad0" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36311 } }
I20260709 13:24:14.424980 26834 leader_election.cc:290] T d5f6b403f0cb4c7a9180d07edb98d1bc P 326c74b0a8d84cdf86229f67ce3487f2 [CANDIDATE]: Term 3 election: Requested vote from peers 239ae18f3e44498fa52e20e9d6eec6b5 (127.14.177.131:39265), ff90de856dd542eab446899d181dcad0 (127.14.177.129:36311)
I20260709 13:24:14.425839 26904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5f6b403f0cb4c7a9180d07edb98d1bc" candidate_uuid: "326c74b0a8d84cdf86229f67ce3487f2" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: true dest_uuid: "239ae18f3e44498fa52e20e9d6eec6b5"
I20260709 13:24:14.426179 27107 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5f6b403f0cb4c7a9180d07edb98d1bc" candidate_uuid: "326c74b0a8d84cdf86229f67ce3487f2" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: true dest_uuid: "ff90de856dd542eab446899d181dcad0"
I20260709 13:24:14.426524 26904 raft_consensus.cc:3055] T d5f6b403f0cb4c7a9180d07edb98d1bc P 239ae18f3e44498fa52e20e9d6eec6b5 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:24:14.426887 26904 raft_consensus.cc:740] T d5f6b403f0cb4c7a9180d07edb98d1bc P 239ae18f3e44498fa52e20e9d6eec6b5 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 239ae18f3e44498fa52e20e9d6eec6b5, State: Running, Role: LEADER
I20260709 13:24:14.426911 27107 raft_consensus.cc:3060] T d5f6b403f0cb4c7a9180d07edb98d1bc P ff90de856dd542eab446899d181dcad0 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:24:14.427702 26904 consensus_queue.cc:260] T d5f6b403f0cb4c7a9180d07edb98d1bc P 239ae18f3e44498fa52e20e9d6eec6b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "239ae18f3e44498fa52e20e9d6eec6b5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 39265 } } peers { permanent_uuid: "326c74b0a8d84cdf86229f67ce3487f2" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35517 } } peers { permanent_uuid: "ff90de856dd542eab446899d181dcad0" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36311 } }
I20260709 13:24:14.428638 26904 raft_consensus.cc:3060] T d5f6b403f0cb4c7a9180d07edb98d1bc P 239ae18f3e44498fa52e20e9d6eec6b5 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:24:14.432245 27107 raft_consensus.cc:2468] T d5f6b403f0cb4c7a9180d07edb98d1bc P ff90de856dd542eab446899d181dcad0 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 326c74b0a8d84cdf86229f67ce3487f2 in term 3.
I20260709 13:24:14.433068 26904 raft_consensus.cc:2468] T d5f6b403f0cb4c7a9180d07edb98d1bc P 239ae18f3e44498fa52e20e9d6eec6b5 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 326c74b0a8d84cdf86229f67ce3487f2 in term 3.
I20260709 13:24:14.433547 26798 leader_election.cc:304] T d5f6b403f0cb4c7a9180d07edb98d1bc P 326c74b0a8d84cdf86229f67ce3487f2 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 326c74b0a8d84cdf86229f67ce3487f2, ff90de856dd542eab446899d181dcad0; no voters: 
I20260709 13:24:14.434618 27167 raft_consensus.cc:2804] T d5f6b403f0cb4c7a9180d07edb98d1bc P 326c74b0a8d84cdf86229f67ce3487f2 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:24:14.435107 27167 raft_consensus.cc:697] T d5f6b403f0cb4c7a9180d07edb98d1bc P 326c74b0a8d84cdf86229f67ce3487f2 [term 3 LEADER]: Becoming Leader. State: Replica: 326c74b0a8d84cdf86229f67ce3487f2, State: Running, Role: LEADER
I20260709 13:24:14.435891 27167 consensus_queue.cc:237] T d5f6b403f0cb4c7a9180d07edb98d1bc P 326c74b0a8d84cdf86229f67ce3487f2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "239ae18f3e44498fa52e20e9d6eec6b5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 39265 } } peers { permanent_uuid: "326c74b0a8d84cdf86229f67ce3487f2" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35517 } } peers { permanent_uuid: "ff90de856dd542eab446899d181dcad0" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36311 } }
I20260709 13:24:14.442878 26655 catalog_manager.cc:5697] T d5f6b403f0cb4c7a9180d07edb98d1bc P 326c74b0a8d84cdf86229f67ce3487f2 reported cstate change: term changed from 2 to 3, leader changed from 239ae18f3e44498fa52e20e9d6eec6b5 (127.14.177.131) to 326c74b0a8d84cdf86229f67ce3487f2 (127.14.177.130). New cstate: current_term: 3 leader_uuid: "326c74b0a8d84cdf86229f67ce3487f2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "239ae18f3e44498fa52e20e9d6eec6b5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 39265 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "326c74b0a8d84cdf86229f67ce3487f2" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35517 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ff90de856dd542eab446899d181dcad0" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36311 } health_report { overall_health: UNKNOWN } } }
I20260709 13:24:14.737142 26834 raft_consensus.cc:1275] T 92c3adeb279b4f5f88680a40a9664750 P 326c74b0a8d84cdf86229f67ce3487f2 [term 3 FOLLOWER]: Refusing update from remote peer 239ae18f3e44498fa52e20e9d6eec6b5: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:24:14.738468 27166 consensus_queue.cc:1048] T 92c3adeb279b4f5f88680a40a9664750 P 239ae18f3e44498fa52e20e9d6eec6b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "326c74b0a8d84cdf86229f67ce3487f2" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35517 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:24:14.747156 27107 raft_consensus.cc:1275] T 92c3adeb279b4f5f88680a40a9664750 P ff90de856dd542eab446899d181dcad0 [term 3 FOLLOWER]: Refusing update from remote peer 239ae18f3e44498fa52e20e9d6eec6b5: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:24:14.748569 27152 consensus_queue.cc:1048] T 92c3adeb279b4f5f88680a40a9664750 P 239ae18f3e44498fa52e20e9d6eec6b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff90de856dd542eab446899d181dcad0" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36311 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:24:14.795284 27107 raft_consensus.cc:1275] T d52f26a0449247c1b928641ec8ebad34 P ff90de856dd542eab446899d181dcad0 [term 3 FOLLOWER]: Refusing update from remote peer 239ae18f3e44498fa52e20e9d6eec6b5: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:24:14.796784 27152 consensus_queue.cc:1048] T d52f26a0449247c1b928641ec8ebad34 P 239ae18f3e44498fa52e20e9d6eec6b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff90de856dd542eab446899d181dcad0" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36311 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:24:14.806507 26834 raft_consensus.cc:1275] T d52f26a0449247c1b928641ec8ebad34 P 326c74b0a8d84cdf86229f67ce3487f2 [term 3 FOLLOWER]: Refusing update from remote peer 239ae18f3e44498fa52e20e9d6eec6b5: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:24:14.808573 27152 consensus_queue.cc:1048] T d52f26a0449247c1b928641ec8ebad34 P 239ae18f3e44498fa52e20e9d6eec6b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "326c74b0a8d84cdf86229f67ce3487f2" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35517 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:24:14.812848 26706 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 13:24:14.821851 27107 raft_consensus.cc:1275] T d5f6b403f0cb4c7a9180d07edb98d1bc P ff90de856dd542eab446899d181dcad0 [term 3 FOLLOWER]: Refusing update from remote peer 326c74b0a8d84cdf86229f67ce3487f2: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:24:14.823625 27167 consensus_queue.cc:1048] T d5f6b403f0cb4c7a9180d07edb98d1bc P 326c74b0a8d84cdf86229f67ce3487f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff90de856dd542eab446899d181dcad0" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36311 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:24:14.830237 26904 raft_consensus.cc:1275] T d5f6b403f0cb4c7a9180d07edb98d1bc P 239ae18f3e44498fa52e20e9d6eec6b5 [term 3 FOLLOWER]: Refusing update from remote peer 326c74b0a8d84cdf86229f67ce3487f2: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:24:14.832544 27167 consensus_queue.cc:1048] T d5f6b403f0cb4c7a9180d07edb98d1bc P 326c74b0a8d84cdf86229f67ce3487f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "239ae18f3e44498fa52e20e9d6eec6b5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 39265 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:24:15.289083 15046 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:24:15.296659 26904 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2664382aca514208a9aa49316bdcaeeb"
dest_uuid: "239ae18f3e44498fa52e20e9d6eec6b5"
mode: GRACEFUL
new_leader_uuid: "326c74b0a8d84cdf86229f67ce3487f2"
 from {username='slave'} at 127.0.0.1:38598
I20260709 13:24:15.297209 26904 raft_consensus.cc:606] T 2664382aca514208a9aa49316bdcaeeb P 239ae18f3e44498fa52e20e9d6eec6b5 [term 1 LEADER]: Received request to transfer leadership to TS 326c74b0a8d84cdf86229f67ce3487f2
I20260709 13:24:15.299860 26834 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "76c716abdcd6437b9dcee43317333298"
dest_uuid: "326c74b0a8d84cdf86229f67ce3487f2"
mode: GRACEFUL
new_leader_uuid: "239ae18f3e44498fa52e20e9d6eec6b5"
 from {username='slave'} at 127.0.0.1:47924
I20260709 13:24:15.300450 26834 raft_consensus.cc:606] T 76c716abdcd6437b9dcee43317333298 P 326c74b0a8d84cdf86229f67ce3487f2 [term 2 LEADER]: Received request to transfer leadership to TS 239ae18f3e44498fa52e20e9d6eec6b5
I20260709 13:24:15.303118 26904 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "bdb51cee932741eda8c12a88f07c890e"
dest_uuid: "239ae18f3e44498fa52e20e9d6eec6b5"
mode: GRACEFUL
new_leader_uuid: "326c74b0a8d84cdf86229f67ce3487f2"
 from {username='slave'} at 127.0.0.1:38598
I20260709 13:24:15.303532 26904 raft_consensus.cc:606] T bdb51cee932741eda8c12a88f07c890e P 239ae18f3e44498fa52e20e9d6eec6b5 [term 1 LEADER]: Received request to transfer leadership to TS 326c74b0a8d84cdf86229f67ce3487f2
I20260709 13:24:15.304654 15046 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 13:24:15.305102 15046 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:24:15.305598 15046 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:24:15.318919 26988 raft_consensus.cc:993] T 2664382aca514208a9aa49316bdcaeeb P 239ae18f3e44498fa52e20e9d6eec6b5: : Instructing follower 326c74b0a8d84cdf86229f67ce3487f2 to start an election
I20260709 13:24:15.319500 27152 raft_consensus.cc:1081] T 2664382aca514208a9aa49316bdcaeeb P 239ae18f3e44498fa52e20e9d6eec6b5 [term 1 LEADER]: Signalling peer 326c74b0a8d84cdf86229f67ce3487f2 to start an election
I20260709 13:24:15.320822 26834 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "2664382aca514208a9aa49316bdcaeeb"
dest_uuid: "326c74b0a8d84cdf86229f67ce3487f2"
 from {username='slave'} at 127.0.0.1:40060
I20260709 13:24:15.321305 26834 raft_consensus.cc:493] T 2664382aca514208a9aa49316bdcaeeb P 326c74b0a8d84cdf86229f67ce3487f2 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:24:15.321626 26834 raft_consensus.cc:3060] T 2664382aca514208a9aa49316bdcaeeb P 326c74b0a8d84cdf86229f67ce3487f2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:24:15.327656 26834 raft_consensus.cc:515] T 2664382aca514208a9aa49316bdcaeeb P 326c74b0a8d84cdf86229f67ce3487f2 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff90de856dd542eab446899d181dcad0" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36311 } } peers { permanent_uuid: "326c74b0a8d84cdf86229f67ce3487f2" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35517 } } peers { permanent_uuid: "239ae18f3e44498fa52e20e9d6eec6b5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 39265 } }
I20260709 13:24:15.329602 26834 leader_election.cc:290] T 2664382aca514208a9aa49316bdcaeeb P 326c74b0a8d84cdf86229f67ce3487f2 [CANDIDATE]: Term 2 election: Requested vote from peers ff90de856dd542eab446899d181dcad0 (127.14.177.129:36311), 239ae18f3e44498fa52e20e9d6eec6b5 (127.14.177.131:39265)
I20260709 13:24:15.330659 27106 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2664382aca514208a9aa49316bdcaeeb" candidate_uuid: "326c74b0a8d84cdf86229f67ce3487f2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "ff90de856dd542eab446899d181dcad0"
I20260709 13:24:15.330826 26904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2664382aca514208a9aa49316bdcaeeb" candidate_uuid: "326c74b0a8d84cdf86229f67ce3487f2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "239ae18f3e44498fa52e20e9d6eec6b5"
I20260709 13:24:15.331377 27106 raft_consensus.cc:3060] T 2664382aca514208a9aa49316bdcaeeb P ff90de856dd542eab446899d181dcad0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:24:15.331468 26904 raft_consensus.cc:3055] T 2664382aca514208a9aa49316bdcaeeb P 239ae18f3e44498fa52e20e9d6eec6b5 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:24:15.331893 26904 raft_consensus.cc:740] T 2664382aca514208a9aa49316bdcaeeb P 239ae18f3e44498fa52e20e9d6eec6b5 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 239ae18f3e44498fa52e20e9d6eec6b5, State: Running, Role: LEADER
I20260709 13:24:15.332711 26904 consensus_queue.cc:260] T 2664382aca514208a9aa49316bdcaeeb P 239ae18f3e44498fa52e20e9d6eec6b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff90de856dd542eab446899d181dcad0" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36311 } } peers { permanent_uuid: "326c74b0a8d84cdf86229f67ce3487f2" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35517 } } peers { permanent_uuid: "239ae18f3e44498fa52e20e9d6eec6b5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 39265 } }
I20260709 13:24:15.333926 26904 raft_consensus.cc:3060] T 2664382aca514208a9aa49316bdcaeeb P 239ae18f3e44498fa52e20e9d6eec6b5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:24:15.338999 26904 raft_consensus.cc:2468] T 2664382aca514208a9aa49316bdcaeeb P 239ae18f3e44498fa52e20e9d6eec6b5 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 326c74b0a8d84cdf86229f67ce3487f2 in term 2.
I20260709 13:24:15.338999 27106 raft_consensus.cc:2468] T 2664382aca514208a9aa49316bdcaeeb P ff90de856dd542eab446899d181dcad0 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 326c74b0a8d84cdf86229f67ce3487f2 in term 2.
I20260709 13:24:15.340111 26801 leader_election.cc:304] T 2664382aca514208a9aa49316bdcaeeb P 326c74b0a8d84cdf86229f67ce3487f2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 239ae18f3e44498fa52e20e9d6eec6b5, 326c74b0a8d84cdf86229f67ce3487f2; no voters: 
I20260709 13:24:15.340926 27167 raft_consensus.cc:2804] T 2664382aca514208a9aa49316bdcaeeb P 326c74b0a8d84cdf86229f67ce3487f2 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:24:15.341363 27167 raft_consensus.cc:697] T 2664382aca514208a9aa49316bdcaeeb P 326c74b0a8d84cdf86229f67ce3487f2 [term 2 LEADER]: Becoming Leader. State: Replica: 326c74b0a8d84cdf86229f67ce3487f2, State: Running, Role: LEADER
I20260709 13:24:15.342231 27167 consensus_queue.cc:237] T 2664382aca514208a9aa49316bdcaeeb P 326c74b0a8d84cdf86229f67ce3487f2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff90de856dd542eab446899d181dcad0" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36311 } } peers { permanent_uuid: "326c74b0a8d84cdf86229f67ce3487f2" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35517 } } peers { permanent_uuid: "239ae18f3e44498fa52e20e9d6eec6b5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 39265 } }
I20260709 13:24:15.349923 26656 catalog_manager.cc:5697] T 2664382aca514208a9aa49316bdcaeeb P 326c74b0a8d84cdf86229f67ce3487f2 reported cstate change: term changed from 1 to 2, leader changed from 239ae18f3e44498fa52e20e9d6eec6b5 (127.14.177.131) to 326c74b0a8d84cdf86229f67ce3487f2 (127.14.177.130). New cstate: current_term: 2 leader_uuid: "326c74b0a8d84cdf86229f67ce3487f2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff90de856dd542eab446899d181dcad0" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36311 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "326c74b0a8d84cdf86229f67ce3487f2" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35517 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "239ae18f3e44498fa52e20e9d6eec6b5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 39265 } health_report { overall_health: UNKNOWN } } }
I20260709 13:24:15.412288 26988 raft_consensus.cc:993] T bdb51cee932741eda8c12a88f07c890e P 239ae18f3e44498fa52e20e9d6eec6b5: : Instructing follower 326c74b0a8d84cdf86229f67ce3487f2 to start an election
I20260709 13:24:15.412896 27152 raft_consensus.cc:1081] T bdb51cee932741eda8c12a88f07c890e P 239ae18f3e44498fa52e20e9d6eec6b5 [term 1 LEADER]: Signalling peer 326c74b0a8d84cdf86229f67ce3487f2 to start an election
I20260709 13:24:15.414538 26834 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "bdb51cee932741eda8c12a88f07c890e"
dest_uuid: "326c74b0a8d84cdf86229f67ce3487f2"
 from {username='slave'} at 127.0.0.1:40060
I20260709 13:24:15.415036 26834 raft_consensus.cc:493] T bdb51cee932741eda8c12a88f07c890e P 326c74b0a8d84cdf86229f67ce3487f2 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:24:15.415378 26834 raft_consensus.cc:3060] T bdb51cee932741eda8c12a88f07c890e P 326c74b0a8d84cdf86229f67ce3487f2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:24:15.420362 26834 raft_consensus.cc:515] T bdb51cee932741eda8c12a88f07c890e P 326c74b0a8d84cdf86229f67ce3487f2 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff90de856dd542eab446899d181dcad0" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36311 } } peers { permanent_uuid: "326c74b0a8d84cdf86229f67ce3487f2" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35517 } } peers { permanent_uuid: "239ae18f3e44498fa52e20e9d6eec6b5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 39265 } }
I20260709 13:24:15.422024 26834 leader_election.cc:290] T bdb51cee932741eda8c12a88f07c890e P 326c74b0a8d84cdf86229f67ce3487f2 [CANDIDATE]: Term 2 election: Requested vote from peers ff90de856dd542eab446899d181dcad0 (127.14.177.129:36311), 239ae18f3e44498fa52e20e9d6eec6b5 (127.14.177.131:39265)
I20260709 13:24:15.422799 27106 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdb51cee932741eda8c12a88f07c890e" candidate_uuid: "326c74b0a8d84cdf86229f67ce3487f2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "ff90de856dd542eab446899d181dcad0"
I20260709 13:24:15.422973 26904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdb51cee932741eda8c12a88f07c890e" candidate_uuid: "326c74b0a8d84cdf86229f67ce3487f2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "239ae18f3e44498fa52e20e9d6eec6b5"
I20260709 13:24:15.423544 26904 raft_consensus.cc:3055] T bdb51cee932741eda8c12a88f07c890e P 239ae18f3e44498fa52e20e9d6eec6b5 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:24:15.423544 27106 raft_consensus.cc:3060] T bdb51cee932741eda8c12a88f07c890e P ff90de856dd542eab446899d181dcad0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:24:15.423918 26904 raft_consensus.cc:740] T bdb51cee932741eda8c12a88f07c890e P 239ae18f3e44498fa52e20e9d6eec6b5 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 239ae18f3e44498fa52e20e9d6eec6b5, State: Running, Role: LEADER
I20260709 13:24:15.424832 26904 consensus_queue.cc:260] T bdb51cee932741eda8c12a88f07c890e P 239ae18f3e44498fa52e20e9d6eec6b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff90de856dd542eab446899d181dcad0" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36311 } } peers { permanent_uuid: "326c74b0a8d84cdf86229f67ce3487f2" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35517 } } peers { permanent_uuid: "239ae18f3e44498fa52e20e9d6eec6b5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 39265 } }
I20260709 13:24:15.426077 26904 raft_consensus.cc:3060] T bdb51cee932741eda8c12a88f07c890e P 239ae18f3e44498fa52e20e9d6eec6b5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:24:15.430986 27106 raft_consensus.cc:2468] T bdb51cee932741eda8c12a88f07c890e P ff90de856dd542eab446899d181dcad0 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 326c74b0a8d84cdf86229f67ce3487f2 in term 2.
I20260709 13:24:15.432268 26798 leader_election.cc:304] T bdb51cee932741eda8c12a88f07c890e P 326c74b0a8d84cdf86229f67ce3487f2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 326c74b0a8d84cdf86229f67ce3487f2, ff90de856dd542eab446899d181dcad0; no voters: 
I20260709 13:24:15.432737 26904 raft_consensus.cc:2468] T bdb51cee932741eda8c12a88f07c890e P 239ae18f3e44498fa52e20e9d6eec6b5 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 326c74b0a8d84cdf86229f67ce3487f2 in term 2.
I20260709 13:24:15.432991 27167 raft_consensus.cc:2804] T bdb51cee932741eda8c12a88f07c890e P 326c74b0a8d84cdf86229f67ce3487f2 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:24:15.433542 27167 raft_consensus.cc:697] T bdb51cee932741eda8c12a88f07c890e P 326c74b0a8d84cdf86229f67ce3487f2 [term 2 LEADER]: Becoming Leader. State: Replica: 326c74b0a8d84cdf86229f67ce3487f2, State: Running, Role: LEADER
I20260709 13:24:15.434453 27167 consensus_queue.cc:237] T bdb51cee932741eda8c12a88f07c890e P 326c74b0a8d84cdf86229f67ce3487f2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff90de856dd542eab446899d181dcad0" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36311 } } peers { permanent_uuid: "326c74b0a8d84cdf86229f67ce3487f2" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35517 } } peers { permanent_uuid: "239ae18f3e44498fa52e20e9d6eec6b5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 39265 } }
I20260709 13:24:15.442397 26656 catalog_manager.cc:5697] T bdb51cee932741eda8c12a88f07c890e P 326c74b0a8d84cdf86229f67ce3487f2 reported cstate change: term changed from 1 to 2, leader changed from 239ae18f3e44498fa52e20e9d6eec6b5 (127.14.177.131) to 326c74b0a8d84cdf86229f67ce3487f2 (127.14.177.130). New cstate: current_term: 2 leader_uuid: "326c74b0a8d84cdf86229f67ce3487f2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff90de856dd542eab446899d181dcad0" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36311 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "326c74b0a8d84cdf86229f67ce3487f2" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35517 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "239ae18f3e44498fa52e20e9d6eec6b5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 39265 } health_report { overall_health: UNKNOWN } } }
I20260709 13:24:15.457885 27173 raft_consensus.cc:993] T 76c716abdcd6437b9dcee43317333298 P 326c74b0a8d84cdf86229f67ce3487f2: : Instructing follower 239ae18f3e44498fa52e20e9d6eec6b5 to start an election
I20260709 13:24:15.458336 27173 raft_consensus.cc:1081] T 76c716abdcd6437b9dcee43317333298 P 326c74b0a8d84cdf86229f67ce3487f2 [term 2 LEADER]: Signalling peer 239ae18f3e44498fa52e20e9d6eec6b5 to start an election
I20260709 13:24:15.460011 26904 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "76c716abdcd6437b9dcee43317333298"
dest_uuid: "239ae18f3e44498fa52e20e9d6eec6b5"
 from {username='slave'} at 127.0.0.1:49792
I20260709 13:24:15.460464 26904 raft_consensus.cc:493] T 76c716abdcd6437b9dcee43317333298 P 239ae18f3e44498fa52e20e9d6eec6b5 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:24:15.460714 26904 raft_consensus.cc:3060] T 76c716abdcd6437b9dcee43317333298 P 239ae18f3e44498fa52e20e9d6eec6b5 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:24:15.465706 26904 raft_consensus.cc:515] T 76c716abdcd6437b9dcee43317333298 P 239ae18f3e44498fa52e20e9d6eec6b5 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff90de856dd542eab446899d181dcad0" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36311 } } peers { permanent_uuid: "326c74b0a8d84cdf86229f67ce3487f2" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35517 } } peers { permanent_uuid: "239ae18f3e44498fa52e20e9d6eec6b5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 39265 } }
I20260709 13:24:15.467300 26904 leader_election.cc:290] T 76c716abdcd6437b9dcee43317333298 P 239ae18f3e44498fa52e20e9d6eec6b5 [CANDIDATE]: Term 3 election: Requested vote from peers ff90de856dd542eab446899d181dcad0 (127.14.177.129:36311), 326c74b0a8d84cdf86229f67ce3487f2 (127.14.177.130:35517)
I20260709 13:24:15.468329 27106 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76c716abdcd6437b9dcee43317333298" candidate_uuid: "239ae18f3e44498fa52e20e9d6eec6b5" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: true dest_uuid: "ff90de856dd542eab446899d181dcad0"
I20260709 13:24:15.468643 26834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76c716abdcd6437b9dcee43317333298" candidate_uuid: "239ae18f3e44498fa52e20e9d6eec6b5" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: true dest_uuid: "326c74b0a8d84cdf86229f67ce3487f2"
I20260709 13:24:15.469125 27106 raft_consensus.cc:3060] T 76c716abdcd6437b9dcee43317333298 P ff90de856dd542eab446899d181dcad0 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:24:15.469310 26834 raft_consensus.cc:3055] T 76c716abdcd6437b9dcee43317333298 P 326c74b0a8d84cdf86229f67ce3487f2 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:24:15.469750 26834 raft_consensus.cc:740] T 76c716abdcd6437b9dcee43317333298 P 326c74b0a8d84cdf86229f67ce3487f2 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 326c74b0a8d84cdf86229f67ce3487f2, State: Running, Role: LEADER
I20260709 13:24:15.470697 26834 consensus_queue.cc:260] T 76c716abdcd6437b9dcee43317333298 P 326c74b0a8d84cdf86229f67ce3487f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff90de856dd542eab446899d181dcad0" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36311 } } peers { permanent_uuid: "326c74b0a8d84cdf86229f67ce3487f2" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35517 } } peers { permanent_uuid: "239ae18f3e44498fa52e20e9d6eec6b5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 39265 } }
I20260709 13:24:15.472182 26834 raft_consensus.cc:3060] T 76c716abdcd6437b9dcee43317333298 P 326c74b0a8d84cdf86229f67ce3487f2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:24:15.477747 27106 raft_consensus.cc:2468] T 76c716abdcd6437b9dcee43317333298 P ff90de856dd542eab446899d181dcad0 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 239ae18f3e44498fa52e20e9d6eec6b5 in term 3.
I20260709 13:24:15.479344 26872 leader_election.cc:304] T 76c716abdcd6437b9dcee43317333298 P 239ae18f3e44498fa52e20e9d6eec6b5 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 239ae18f3e44498fa52e20e9d6eec6b5, ff90de856dd542eab446899d181dcad0; no voters: 
I20260709 13:24:15.480006 26834 raft_consensus.cc:2468] T 76c716abdcd6437b9dcee43317333298 P 326c74b0a8d84cdf86229f67ce3487f2 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 239ae18f3e44498fa52e20e9d6eec6b5 in term 3.
I20260709 13:24:15.480116 27152 raft_consensus.cc:2804] T 76c716abdcd6437b9dcee43317333298 P 239ae18f3e44498fa52e20e9d6eec6b5 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:24:15.480727 27152 raft_consensus.cc:697] T 76c716abdcd6437b9dcee43317333298 P 239ae18f3e44498fa52e20e9d6eec6b5 [term 3 LEADER]: Becoming Leader. State: Replica: 239ae18f3e44498fa52e20e9d6eec6b5, State: Running, Role: LEADER
I20260709 13:24:15.481614 27152 consensus_queue.cc:237] T 76c716abdcd6437b9dcee43317333298 P 239ae18f3e44498fa52e20e9d6eec6b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff90de856dd542eab446899d181dcad0" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36311 } } peers { permanent_uuid: "326c74b0a8d84cdf86229f67ce3487f2" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35517 } } peers { permanent_uuid: "239ae18f3e44498fa52e20e9d6eec6b5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 39265 } }
I20260709 13:24:15.489940 26656 catalog_manager.cc:5697] T 76c716abdcd6437b9dcee43317333298 P 239ae18f3e44498fa52e20e9d6eec6b5 reported cstate change: term changed from 2 to 3, leader changed from 326c74b0a8d84cdf86229f67ce3487f2 (127.14.177.130) to 239ae18f3e44498fa52e20e9d6eec6b5 (127.14.177.131). New cstate: current_term: 3 leader_uuid: "239ae18f3e44498fa52e20e9d6eec6b5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff90de856dd542eab446899d181dcad0" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36311 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "326c74b0a8d84cdf86229f67ce3487f2" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35517 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "239ae18f3e44498fa52e20e9d6eec6b5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 39265 } health_report { overall_health: HEALTHY } } }
W20260709 13:24:15.814131 26706 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 13:24:15.822822 26904 raft_consensus.cc:1275] T bdb51cee932741eda8c12a88f07c890e P 239ae18f3e44498fa52e20e9d6eec6b5 [term 2 FOLLOWER]: Refusing update from remote peer 326c74b0a8d84cdf86229f67ce3487f2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:24:15.824422 27173 consensus_queue.cc:1048] T bdb51cee932741eda8c12a88f07c890e P 326c74b0a8d84cdf86229f67ce3487f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "239ae18f3e44498fa52e20e9d6eec6b5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 39265 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:24:15.834928 27106 raft_consensus.cc:1275] T bdb51cee932741eda8c12a88f07c890e P ff90de856dd542eab446899d181dcad0 [term 2 FOLLOWER]: Refusing update from remote peer 326c74b0a8d84cdf86229f67ce3487f2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:24:15.837157 27173 consensus_queue.cc:1048] T bdb51cee932741eda8c12a88f07c890e P 326c74b0a8d84cdf86229f67ce3487f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff90de856dd542eab446899d181dcad0" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36311 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:24:15.882794 26834 raft_consensus.cc:1275] T 76c716abdcd6437b9dcee43317333298 P 326c74b0a8d84cdf86229f67ce3487f2 [term 3 FOLLOWER]: Refusing update from remote peer 239ae18f3e44498fa52e20e9d6eec6b5: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:24:15.884234 27152 consensus_queue.cc:1048] T 76c716abdcd6437b9dcee43317333298 P 239ae18f3e44498fa52e20e9d6eec6b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "326c74b0a8d84cdf86229f67ce3487f2" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35517 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:24:15.887022 26904 raft_consensus.cc:1275] T 2664382aca514208a9aa49316bdcaeeb P 239ae18f3e44498fa52e20e9d6eec6b5 [term 2 FOLLOWER]: Refusing update from remote peer 326c74b0a8d84cdf86229f67ce3487f2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:24:15.887317 27106 raft_consensus.cc:1275] T 2664382aca514208a9aa49316bdcaeeb P ff90de856dd542eab446899d181dcad0 [term 2 FOLLOWER]: Refusing update from remote peer 326c74b0a8d84cdf86229f67ce3487f2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:24:15.888447 27167 consensus_queue.cc:1048] T 2664382aca514208a9aa49316bdcaeeb P 326c74b0a8d84cdf86229f67ce3487f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "239ae18f3e44498fa52e20e9d6eec6b5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 39265 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:24:15.889256 27173 consensus_queue.cc:1048] T 2664382aca514208a9aa49316bdcaeeb P 326c74b0a8d84cdf86229f67ce3487f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff90de856dd542eab446899d181dcad0" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36311 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:24:15.899859 27107 raft_consensus.cc:1275] T 76c716abdcd6437b9dcee43317333298 P ff90de856dd542eab446899d181dcad0 [term 3 FOLLOWER]: Refusing update from remote peer 239ae18f3e44498fa52e20e9d6eec6b5: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:24:15.901836 27166 consensus_queue.cc:1048] T 76c716abdcd6437b9dcee43317333298 P 239ae18f3e44498fa52e20e9d6eec6b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff90de856dd542eab446899d181dcad0" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36311 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:24:16.311785 15046 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:24:16.320118 26834 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2664382aca514208a9aa49316bdcaeeb"
dest_uuid: "326c74b0a8d84cdf86229f67ce3487f2"
mode: GRACEFUL
new_leader_uuid: "239ae18f3e44498fa52e20e9d6eec6b5"
 from {username='slave'} at 127.0.0.1:47924
I20260709 13:24:16.320690 26834 raft_consensus.cc:606] T 2664382aca514208a9aa49316bdcaeeb P 326c74b0a8d84cdf86229f67ce3487f2 [term 2 LEADER]: Received request to transfer leadership to TS 239ae18f3e44498fa52e20e9d6eec6b5
I20260709 13:24:16.323650 26904 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7492e977a5f74ef09aa687b1d7ec89e3"
dest_uuid: "239ae18f3e44498fa52e20e9d6eec6b5"
mode: GRACEFUL
new_leader_uuid: "326c74b0a8d84cdf86229f67ce3487f2"
 from {username='slave'} at 127.0.0.1:38598
I20260709 13:24:16.324193 26904 raft_consensus.cc:606] T 7492e977a5f74ef09aa687b1d7ec89e3 P 239ae18f3e44498fa52e20e9d6eec6b5 [term 2 LEADER]: Received request to transfer leadership to TS 326c74b0a8d84cdf86229f67ce3487f2
I20260709 13:24:16.326766 26834 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "bdb51cee932741eda8c12a88f07c890e"
dest_uuid: "326c74b0a8d84cdf86229f67ce3487f2"
mode: GRACEFUL
new_leader_uuid: "239ae18f3e44498fa52e20e9d6eec6b5"
 from {username='slave'} at 127.0.0.1:47924
I20260709 13:24:16.327334 26834 raft_consensus.cc:606] T bdb51cee932741eda8c12a88f07c890e P 326c74b0a8d84cdf86229f67ce3487f2 [term 2 LEADER]: Received request to transfer leadership to TS 239ae18f3e44498fa52e20e9d6eec6b5
I20260709 13:24:16.328562 15046 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 13:24:16.329018 15046 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:24:16.329531 15046 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:24:16.477315 27205 raft_consensus.cc:993] T 2664382aca514208a9aa49316bdcaeeb P 326c74b0a8d84cdf86229f67ce3487f2: : Instructing follower 239ae18f3e44498fa52e20e9d6eec6b5 to start an election
I20260709 13:24:16.477882 27205 raft_consensus.cc:1081] T 2664382aca514208a9aa49316bdcaeeb P 326c74b0a8d84cdf86229f67ce3487f2 [term 2 LEADER]: Signalling peer 239ae18f3e44498fa52e20e9d6eec6b5 to start an election
I20260709 13:24:16.479334 26904 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "2664382aca514208a9aa49316bdcaeeb"
dest_uuid: "239ae18f3e44498fa52e20e9d6eec6b5"
 from {username='slave'} at 127.0.0.1:49792
I20260709 13:24:16.479825 26904 raft_consensus.cc:493] T 2664382aca514208a9aa49316bdcaeeb P 239ae18f3e44498fa52e20e9d6eec6b5 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:24:16.480134 26904 raft_consensus.cc:3060] T 2664382aca514208a9aa49316bdcaeeb P 239ae18f3e44498fa52e20e9d6eec6b5 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:24:16.484826 26904 raft_consensus.cc:515] T 2664382aca514208a9aa49316bdcaeeb P 239ae18f3e44498fa52e20e9d6eec6b5 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff90de856dd542eab446899d181dcad0" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36311 } } peers { permanent_uuid: "326c74b0a8d84cdf86229f67ce3487f2" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35517 } } peers { permanent_uuid: "239ae18f3e44498fa52e20e9d6eec6b5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 39265 } }
I20260709 13:24:16.486588 26904 leader_election.cc:290] T 2664382aca514208a9aa49316bdcaeeb P 239ae18f3e44498fa52e20e9d6eec6b5 [CANDIDATE]: Term 3 election: Requested vote from peers ff90de856dd542eab446899d181dcad0 (127.14.177.129:36311), 326c74b0a8d84cdf86229f67ce3487f2 (127.14.177.130:35517)
I20260709 13:24:16.487386 27106 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2664382aca514208a9aa49316bdcaeeb" candidate_uuid: "239ae18f3e44498fa52e20e9d6eec6b5" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "ff90de856dd542eab446899d181dcad0"
I20260709 13:24:16.487618 26834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2664382aca514208a9aa49316bdcaeeb" candidate_uuid: "239ae18f3e44498fa52e20e9d6eec6b5" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "326c74b0a8d84cdf86229f67ce3487f2"
I20260709 13:24:16.487965 27106 raft_consensus.cc:3060] T 2664382aca514208a9aa49316bdcaeeb P ff90de856dd542eab446899d181dcad0 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:24:16.488221 26834 raft_consensus.cc:3055] T 2664382aca514208a9aa49316bdcaeeb P 326c74b0a8d84cdf86229f67ce3487f2 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:24:16.488588 26834 raft_consensus.cc:740] T 2664382aca514208a9aa49316bdcaeeb P 326c74b0a8d84cdf86229f67ce3487f2 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 326c74b0a8d84cdf86229f67ce3487f2, State: Running, Role: LEADER
I20260709 13:24:16.489197 26834 consensus_queue.cc:260] T 2664382aca514208a9aa49316bdcaeeb P 326c74b0a8d84cdf86229f67ce3487f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff90de856dd542eab446899d181dcad0" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36311 } } peers { permanent_uuid: "326c74b0a8d84cdf86229f67ce3487f2" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35517 } } peers { permanent_uuid: "239ae18f3e44498fa52e20e9d6eec6b5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 39265 } }
I20260709 13:24:16.490083 26834 raft_consensus.cc:3060] T 2664382aca514208a9aa49316bdcaeeb P 326c74b0a8d84cdf86229f67ce3487f2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:24:16.493713 27106 raft_consensus.cc:2468] T 2664382aca514208a9aa49316bdcaeeb P ff90de856dd542eab446899d181dcad0 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 239ae18f3e44498fa52e20e9d6eec6b5 in term 3.
I20260709 13:24:16.494982 26872 leader_election.cc:304] T 2664382aca514208a9aa49316bdcaeeb P 239ae18f3e44498fa52e20e9d6eec6b5 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 239ae18f3e44498fa52e20e9d6eec6b5, ff90de856dd542eab446899d181dcad0; no voters: 
I20260709 13:24:16.495299 26834 raft_consensus.cc:2468] T 2664382aca514208a9aa49316bdcaeeb P 326c74b0a8d84cdf86229f67ce3487f2 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 239ae18f3e44498fa52e20e9d6eec6b5 in term 3.
I20260709 13:24:16.495962 27152 raft_consensus.cc:2804] T 2664382aca514208a9aa49316bdcaeeb P 239ae18f3e44498fa52e20e9d6eec6b5 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:24:16.497205 27152 raft_consensus.cc:697] T 2664382aca514208a9aa49316bdcaeeb P 239ae18f3e44498fa52e20e9d6eec6b5 [term 3 LEADER]: Becoming Leader. State: Replica: 239ae18f3e44498fa52e20e9d6eec6b5, State: Running, Role: LEADER
I20260709 13:24:16.498010 27152 consensus_queue.cc:237] T 2664382aca514208a9aa49316bdcaeeb P 239ae18f3e44498fa52e20e9d6eec6b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff90de856dd542eab446899d181dcad0" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36311 } } peers { permanent_uuid: "326c74b0a8d84cdf86229f67ce3487f2" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35517 } } peers { permanent_uuid: "239ae18f3e44498fa52e20e9d6eec6b5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 39265 } }
I20260709 13:24:16.505213 26655 catalog_manager.cc:5697] T 2664382aca514208a9aa49316bdcaeeb P 239ae18f3e44498fa52e20e9d6eec6b5 reported cstate change: term changed from 2 to 3, leader changed from 326c74b0a8d84cdf86229f67ce3487f2 (127.14.177.130) to 239ae18f3e44498fa52e20e9d6eec6b5 (127.14.177.131). New cstate: current_term: 3 leader_uuid: "239ae18f3e44498fa52e20e9d6eec6b5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff90de856dd542eab446899d181dcad0" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36311 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "326c74b0a8d84cdf86229f67ce3487f2" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35517 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "239ae18f3e44498fa52e20e9d6eec6b5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 39265 } health_report { overall_health: HEALTHY } } }
I20260709 13:24:16.696275 27173 raft_consensus.cc:993] T bdb51cee932741eda8c12a88f07c890e P 326c74b0a8d84cdf86229f67ce3487f2: : Instructing follower 239ae18f3e44498fa52e20e9d6eec6b5 to start an election
I20260709 13:24:16.696781 27205 raft_consensus.cc:1081] T bdb51cee932741eda8c12a88f07c890e P 326c74b0a8d84cdf86229f67ce3487f2 [term 2 LEADER]: Signalling peer 239ae18f3e44498fa52e20e9d6eec6b5 to start an election
I20260709 13:24:16.698274 26904 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "bdb51cee932741eda8c12a88f07c890e"
dest_uuid: "239ae18f3e44498fa52e20e9d6eec6b5"
 from {username='slave'} at 127.0.0.1:49792
I20260709 13:24:16.698884 26904 raft_consensus.cc:493] T bdb51cee932741eda8c12a88f07c890e P 239ae18f3e44498fa52e20e9d6eec6b5 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:24:16.699209 26904 raft_consensus.cc:3060] T bdb51cee932741eda8c12a88f07c890e P 239ae18f3e44498fa52e20e9d6eec6b5 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:24:16.705389 26904 raft_consensus.cc:515] T bdb51cee932741eda8c12a88f07c890e P 239ae18f3e44498fa52e20e9d6eec6b5 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff90de856dd542eab446899d181dcad0" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36311 } } peers { permanent_uuid: "326c74b0a8d84cdf86229f67ce3487f2" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35517 } } peers { permanent_uuid: "239ae18f3e44498fa52e20e9d6eec6b5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 39265 } }
I20260709 13:24:16.707212 26904 leader_election.cc:290] T bdb51cee932741eda8c12a88f07c890e P 239ae18f3e44498fa52e20e9d6eec6b5 [CANDIDATE]: Term 3 election: Requested vote from peers ff90de856dd542eab446899d181dcad0 (127.14.177.129:36311), 326c74b0a8d84cdf86229f67ce3487f2 (127.14.177.130:35517)
I20260709 13:24:16.708182 27107 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdb51cee932741eda8c12a88f07c890e" candidate_uuid: "239ae18f3e44498fa52e20e9d6eec6b5" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "ff90de856dd542eab446899d181dcad0"
I20260709 13:24:16.708288 26834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdb51cee932741eda8c12a88f07c890e" candidate_uuid: "239ae18f3e44498fa52e20e9d6eec6b5" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "326c74b0a8d84cdf86229f67ce3487f2"
I20260709 13:24:16.708910 27107 raft_consensus.cc:3060] T bdb51cee932741eda8c12a88f07c890e P ff90de856dd542eab446899d181dcad0 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:24:16.708936 26834 raft_consensus.cc:3055] T bdb51cee932741eda8c12a88f07c890e P 326c74b0a8d84cdf86229f67ce3487f2 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:24:16.709547 26834 raft_consensus.cc:740] T bdb51cee932741eda8c12a88f07c890e P 326c74b0a8d84cdf86229f67ce3487f2 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 326c74b0a8d84cdf86229f67ce3487f2, State: Running, Role: LEADER
I20260709 13:24:16.710361 26834 consensus_queue.cc:260] T bdb51cee932741eda8c12a88f07c890e P 326c74b0a8d84cdf86229f67ce3487f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff90de856dd542eab446899d181dcad0" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36311 } } peers { permanent_uuid: "326c74b0a8d84cdf86229f67ce3487f2" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35517 } } peers { permanent_uuid: "239ae18f3e44498fa52e20e9d6eec6b5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 39265 } }
I20260709 13:24:16.711674 26834 raft_consensus.cc:3060] T bdb51cee932741eda8c12a88f07c890e P 326c74b0a8d84cdf86229f67ce3487f2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:24:16.716696 27107 raft_consensus.cc:2468] T bdb51cee932741eda8c12a88f07c890e P ff90de856dd542eab446899d181dcad0 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 239ae18f3e44498fa52e20e9d6eec6b5 in term 3.
I20260709 13:24:16.717883 26872 leader_election.cc:304] T bdb51cee932741eda8c12a88f07c890e P 239ae18f3e44498fa52e20e9d6eec6b5 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 239ae18f3e44498fa52e20e9d6eec6b5, ff90de856dd542eab446899d181dcad0; no voters: 
I20260709 13:24:16.718252 27152 raft_consensus.cc:993] T 7492e977a5f74ef09aa687b1d7ec89e3 P 239ae18f3e44498fa52e20e9d6eec6b5: : Instructing follower 326c74b0a8d84cdf86229f67ce3487f2 to start an election
I20260709 13:24:16.718760 27212 raft_consensus.cc:2804] T bdb51cee932741eda8c12a88f07c890e P 239ae18f3e44498fa52e20e9d6eec6b5 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:24:16.718891 27207 raft_consensus.cc:1081] T 7492e977a5f74ef09aa687b1d7ec89e3 P 239ae18f3e44498fa52e20e9d6eec6b5 [term 2 LEADER]: Signalling peer 326c74b0a8d84cdf86229f67ce3487f2 to start an election
I20260709 13:24:16.718779 26834 raft_consensus.cc:2468] T bdb51cee932741eda8c12a88f07c890e P 326c74b0a8d84cdf86229f67ce3487f2 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 239ae18f3e44498fa52e20e9d6eec6b5 in term 3.
I20260709 13:24:16.719522 27212 raft_consensus.cc:697] T bdb51cee932741eda8c12a88f07c890e P 239ae18f3e44498fa52e20e9d6eec6b5 [term 3 LEADER]: Becoming Leader. State: Replica: 239ae18f3e44498fa52e20e9d6eec6b5, State: Running, Role: LEADER
I20260709 13:24:16.720860 26834 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "7492e977a5f74ef09aa687b1d7ec89e3"
dest_uuid: "326c74b0a8d84cdf86229f67ce3487f2"
 from {username='slave'} at 127.0.0.1:40060
I20260709 13:24:16.721331 26834 raft_consensus.cc:493] T 7492e977a5f74ef09aa687b1d7ec89e3 P 326c74b0a8d84cdf86229f67ce3487f2 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:24:16.721298 27212 consensus_queue.cc:237] T bdb51cee932741eda8c12a88f07c890e P 239ae18f3e44498fa52e20e9d6eec6b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff90de856dd542eab446899d181dcad0" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36311 } } peers { permanent_uuid: "326c74b0a8d84cdf86229f67ce3487f2" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35517 } } peers { permanent_uuid: "239ae18f3e44498fa52e20e9d6eec6b5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 39265 } }
I20260709 13:24:16.721868 26834 raft_consensus.cc:3060] T 7492e977a5f74ef09aa687b1d7ec89e3 P 326c74b0a8d84cdf86229f67ce3487f2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:24:16.727962 26834 raft_consensus.cc:515] T 7492e977a5f74ef09aa687b1d7ec89e3 P 326c74b0a8d84cdf86229f67ce3487f2 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff90de856dd542eab446899d181dcad0" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36311 } } peers { permanent_uuid: "326c74b0a8d84cdf86229f67ce3487f2" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35517 } } peers { permanent_uuid: "239ae18f3e44498fa52e20e9d6eec6b5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 39265 } }
I20260709 13:24:16.728125 26655 catalog_manager.cc:5697] T bdb51cee932741eda8c12a88f07c890e P 239ae18f3e44498fa52e20e9d6eec6b5 reported cstate change: term changed from 2 to 3, leader changed from 326c74b0a8d84cdf86229f67ce3487f2 (127.14.177.130) to 239ae18f3e44498fa52e20e9d6eec6b5 (127.14.177.131). New cstate: current_term: 3 leader_uuid: "239ae18f3e44498fa52e20e9d6eec6b5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff90de856dd542eab446899d181dcad0" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36311 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "326c74b0a8d84cdf86229f67ce3487f2" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35517 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "239ae18f3e44498fa52e20e9d6eec6b5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 39265 } health_report { overall_health: HEALTHY } } }
I20260709 13:24:16.729516 26834 leader_election.cc:290] T 7492e977a5f74ef09aa687b1d7ec89e3 P 326c74b0a8d84cdf86229f67ce3487f2 [CANDIDATE]: Term 3 election: Requested vote from peers ff90de856dd542eab446899d181dcad0 (127.14.177.129:36311), 239ae18f3e44498fa52e20e9d6eec6b5 (127.14.177.131:39265)
I20260709 13:24:16.730430 27107 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7492e977a5f74ef09aa687b1d7ec89e3" candidate_uuid: "326c74b0a8d84cdf86229f67ce3487f2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "ff90de856dd542eab446899d181dcad0"
I20260709 13:24:16.730636 26904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7492e977a5f74ef09aa687b1d7ec89e3" candidate_uuid: "326c74b0a8d84cdf86229f67ce3487f2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "239ae18f3e44498fa52e20e9d6eec6b5"
I20260709 13:24:16.731084 27107 raft_consensus.cc:3060] T 7492e977a5f74ef09aa687b1d7ec89e3 P ff90de856dd542eab446899d181dcad0 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:24:16.731226 26904 raft_consensus.cc:3055] T 7492e977a5f74ef09aa687b1d7ec89e3 P 239ae18f3e44498fa52e20e9d6eec6b5 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:24:16.731668 26904 raft_consensus.cc:740] T 7492e977a5f74ef09aa687b1d7ec89e3 P 239ae18f3e44498fa52e20e9d6eec6b5 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 239ae18f3e44498fa52e20e9d6eec6b5, State: Running, Role: LEADER
I20260709 13:24:16.732256 26904 consensus_queue.cc:260] T 7492e977a5f74ef09aa687b1d7ec89e3 P 239ae18f3e44498fa52e20e9d6eec6b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff90de856dd542eab446899d181dcad0" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36311 } } peers { permanent_uuid: "326c74b0a8d84cdf86229f67ce3487f2" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35517 } } peers { permanent_uuid: "239ae18f3e44498fa52e20e9d6eec6b5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 39265 } }
I20260709 13:24:16.733045 26904 raft_consensus.cc:3060] T 7492e977a5f74ef09aa687b1d7ec89e3 P 239ae18f3e44498fa52e20e9d6eec6b5 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:24:16.738310 27107 raft_consensus.cc:2468] T 7492e977a5f74ef09aa687b1d7ec89e3 P ff90de856dd542eab446899d181dcad0 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 326c74b0a8d84cdf86229f67ce3487f2 in term 3.
I20260709 13:24:16.739347 26904 raft_consensus.cc:2468] T 7492e977a5f74ef09aa687b1d7ec89e3 P 239ae18f3e44498fa52e20e9d6eec6b5 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 326c74b0a8d84cdf86229f67ce3487f2 in term 3.
I20260709 13:24:16.739601 26798 leader_election.cc:304] T 7492e977a5f74ef09aa687b1d7ec89e3 P 326c74b0a8d84cdf86229f67ce3487f2 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 326c74b0a8d84cdf86229f67ce3487f2, ff90de856dd542eab446899d181dcad0; no voters: 
I20260709 13:24:16.740473 27205 raft_consensus.cc:2804] T 7492e977a5f74ef09aa687b1d7ec89e3 P 326c74b0a8d84cdf86229f67ce3487f2 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:24:16.740923 27205 raft_consensus.cc:697] T 7492e977a5f74ef09aa687b1d7ec89e3 P 326c74b0a8d84cdf86229f67ce3487f2 [term 3 LEADER]: Becoming Leader. State: Replica: 326c74b0a8d84cdf86229f67ce3487f2, State: Running, Role: LEADER
I20260709 13:24:16.741662 27205 consensus_queue.cc:237] T 7492e977a5f74ef09aa687b1d7ec89e3 P 326c74b0a8d84cdf86229f67ce3487f2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff90de856dd542eab446899d181dcad0" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36311 } } peers { permanent_uuid: "326c74b0a8d84cdf86229f67ce3487f2" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35517 } } peers { permanent_uuid: "239ae18f3e44498fa52e20e9d6eec6b5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 39265 } }
I20260709 13:24:16.748556 26655 catalog_manager.cc:5697] T 7492e977a5f74ef09aa687b1d7ec89e3 P 326c74b0a8d84cdf86229f67ce3487f2 reported cstate change: term changed from 2 to 3, leader changed from 239ae18f3e44498fa52e20e9d6eec6b5 (127.14.177.131) to 326c74b0a8d84cdf86229f67ce3487f2 (127.14.177.130). New cstate: current_term: 3 leader_uuid: "326c74b0a8d84cdf86229f67ce3487f2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff90de856dd542eab446899d181dcad0" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36311 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "326c74b0a8d84cdf86229f67ce3487f2" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35517 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "239ae18f3e44498fa52e20e9d6eec6b5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 39265 } health_report { overall_health: UNKNOWN } } }
W20260709 13:24:16.815575 26706 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 13:24:16.928463 27107 raft_consensus.cc:1275] T 2664382aca514208a9aa49316bdcaeeb P ff90de856dd542eab446899d181dcad0 [term 3 FOLLOWER]: Refusing update from remote peer 239ae18f3e44498fa52e20e9d6eec6b5: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:24:16.930091 27212 consensus_queue.cc:1048] T 2664382aca514208a9aa49316bdcaeeb P 239ae18f3e44498fa52e20e9d6eec6b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff90de856dd542eab446899d181dcad0" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36311 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:24:16.945274 26834 raft_consensus.cc:1275] T 2664382aca514208a9aa49316bdcaeeb P 326c74b0a8d84cdf86229f67ce3487f2 [term 3 FOLLOWER]: Refusing update from remote peer 239ae18f3e44498fa52e20e9d6eec6b5: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:24:16.946617 27212 consensus_queue.cc:1048] T 2664382aca514208a9aa49316bdcaeeb P 239ae18f3e44498fa52e20e9d6eec6b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "326c74b0a8d84cdf86229f67ce3487f2" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35517 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:24:17.274175 27106 raft_consensus.cc:1275] T bdb51cee932741eda8c12a88f07c890e P ff90de856dd542eab446899d181dcad0 [term 3 FOLLOWER]: Refusing update from remote peer 239ae18f3e44498fa52e20e9d6eec6b5: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:24:17.276067 27217 consensus_queue.cc:1048] T bdb51cee932741eda8c12a88f07c890e P 239ae18f3e44498fa52e20e9d6eec6b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff90de856dd542eab446899d181dcad0" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36311 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:24:17.292243 26834 raft_consensus.cc:1275] T bdb51cee932741eda8c12a88f07c890e P 326c74b0a8d84cdf86229f67ce3487f2 [term 3 FOLLOWER]: Refusing update from remote peer 239ae18f3e44498fa52e20e9d6eec6b5: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:24:17.292243 26904 raft_consensus.cc:1275] T 7492e977a5f74ef09aa687b1d7ec89e3 P 239ae18f3e44498fa52e20e9d6eec6b5 [term 3 FOLLOWER]: Refusing update from remote peer 326c74b0a8d84cdf86229f67ce3487f2: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:24:17.294916 27212 consensus_queue.cc:1048] T bdb51cee932741eda8c12a88f07c890e P 239ae18f3e44498fa52e20e9d6eec6b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "326c74b0a8d84cdf86229f67ce3487f2" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35517 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:24:17.294001 27205 consensus_queue.cc:1048] T 7492e977a5f74ef09aa687b1d7ec89e3 P 326c74b0a8d84cdf86229f67ce3487f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "239ae18f3e44498fa52e20e9d6eec6b5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 39265 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:24:17.314277 27107 raft_consensus.cc:1275] T 7492e977a5f74ef09aa687b1d7ec89e3 P ff90de856dd542eab446899d181dcad0 [term 3 FOLLOWER]: Refusing update from remote peer 326c74b0a8d84cdf86229f67ce3487f2: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:24:17.316300 27227 consensus_queue.cc:1048] T 7492e977a5f74ef09aa687b1d7ec89e3 P 326c74b0a8d84cdf86229f67ce3487f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff90de856dd542eab446899d181dcad0" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36311 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:24:17.355859 15046 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:24:17.363299 26904 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2664382aca514208a9aa49316bdcaeeb"
dest_uuid: "239ae18f3e44498fa52e20e9d6eec6b5"
mode: GRACEFUL
new_leader_uuid: "326c74b0a8d84cdf86229f67ce3487f2"
 from {username='slave'} at 127.0.0.1:38598
I20260709 13:24:17.363767 26904 raft_consensus.cc:606] T 2664382aca514208a9aa49316bdcaeeb P 239ae18f3e44498fa52e20e9d6eec6b5 [term 3 LEADER]: Received request to transfer leadership to TS 326c74b0a8d84cdf86229f67ce3487f2
I20260709 13:24:17.366355 26834 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9ba9365ebe904f85840f6d049dbcda88"
dest_uuid: "326c74b0a8d84cdf86229f67ce3487f2"
mode: GRACEFUL
new_leader_uuid: "239ae18f3e44498fa52e20e9d6eec6b5"
 from {username='slave'} at 127.0.0.1:47924
I20260709 13:24:17.366828 26834 raft_consensus.cc:606] T 9ba9365ebe904f85840f6d049dbcda88 P 326c74b0a8d84cdf86229f67ce3487f2 [term 2 LEADER]: Received request to transfer leadership to TS 239ae18f3e44498fa52e20e9d6eec6b5
I20260709 13:24:17.369405 26904 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d52f26a0449247c1b928641ec8ebad34"
dest_uuid: "239ae18f3e44498fa52e20e9d6eec6b5"
mode: GRACEFUL
new_leader_uuid: "326c74b0a8d84cdf86229f67ce3487f2"
 from {username='slave'} at 127.0.0.1:38598
I20260709 13:24:17.369875 26904 raft_consensus.cc:606] T d52f26a0449247c1b928641ec8ebad34 P 239ae18f3e44498fa52e20e9d6eec6b5 [term 3 LEADER]: Received request to transfer leadership to TS 326c74b0a8d84cdf86229f67ce3487f2
I20260709 13:24:17.370968 15046 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 13:24:17.371426 15046 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:24:17.371964 15046 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:24:17.450789 27152 raft_consensus.cc:993] T d52f26a0449247c1b928641ec8ebad34 P 239ae18f3e44498fa52e20e9d6eec6b5: : Instructing follower 326c74b0a8d84cdf86229f67ce3487f2 to start an election
I20260709 13:24:17.451380 27152 raft_consensus.cc:1081] T d52f26a0449247c1b928641ec8ebad34 P 239ae18f3e44498fa52e20e9d6eec6b5 [term 3 LEADER]: Signalling peer 326c74b0a8d84cdf86229f67ce3487f2 to start an election
I20260709 13:24:17.452754 26834 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d52f26a0449247c1b928641ec8ebad34"
dest_uuid: "326c74b0a8d84cdf86229f67ce3487f2"
 from {username='slave'} at 127.0.0.1:40060
I20260709 13:24:17.453218 26834 raft_consensus.cc:493] T d52f26a0449247c1b928641ec8ebad34 P 326c74b0a8d84cdf86229f67ce3487f2 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:24:17.453459 26834 raft_consensus.cc:3060] T d52f26a0449247c1b928641ec8ebad34 P 326c74b0a8d84cdf86229f67ce3487f2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:24:17.458247 26834 raft_consensus.cc:515] T d52f26a0449247c1b928641ec8ebad34 P 326c74b0a8d84cdf86229f67ce3487f2 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff90de856dd542eab446899d181dcad0" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36311 } } peers { permanent_uuid: "326c74b0a8d84cdf86229f67ce3487f2" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35517 } } peers { permanent_uuid: "239ae18f3e44498fa52e20e9d6eec6b5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 39265 } }
I20260709 13:24:17.459857 26834 leader_election.cc:290] T d52f26a0449247c1b928641ec8ebad34 P 326c74b0a8d84cdf86229f67ce3487f2 [CANDIDATE]: Term 4 election: Requested vote from peers ff90de856dd542eab446899d181dcad0 (127.14.177.129:36311), 239ae18f3e44498fa52e20e9d6eec6b5 (127.14.177.131:39265)
I20260709 13:24:17.461045 27107 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d52f26a0449247c1b928641ec8ebad34" candidate_uuid: "326c74b0a8d84cdf86229f67ce3487f2" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: true dest_uuid: "ff90de856dd542eab446899d181dcad0"
I20260709 13:24:17.461046 26904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d52f26a0449247c1b928641ec8ebad34" candidate_uuid: "326c74b0a8d84cdf86229f67ce3487f2" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: true dest_uuid: "239ae18f3e44498fa52e20e9d6eec6b5"
I20260709 13:24:17.462134 27107 raft_consensus.cc:3060] T d52f26a0449247c1b928641ec8ebad34 P ff90de856dd542eab446899d181dcad0 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:24:17.462182 26904 raft_consensus.cc:3055] T d52f26a0449247c1b928641ec8ebad34 P 239ae18f3e44498fa52e20e9d6eec6b5 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:24:17.462708 26904 raft_consensus.cc:740] T d52f26a0449247c1b928641ec8ebad34 P 239ae18f3e44498fa52e20e9d6eec6b5 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 239ae18f3e44498fa52e20e9d6eec6b5, State: Running, Role: LEADER
I20260709 13:24:17.463593 26904 consensus_queue.cc:260] T d52f26a0449247c1b928641ec8ebad34 P 239ae18f3e44498fa52e20e9d6eec6b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff90de856dd542eab446899d181dcad0" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36311 } } peers { permanent_uuid: "326c74b0a8d84cdf86229f67ce3487f2" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35517 } } peers { permanent_uuid: "239ae18f3e44498fa52e20e9d6eec6b5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 39265 } }
I20260709 13:24:17.464875 26904 raft_consensus.cc:3060] T d52f26a0449247c1b928641ec8ebad34 P 239ae18f3e44498fa52e20e9d6eec6b5 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:24:17.470582 27107 raft_consensus.cc:2468] T d52f26a0449247c1b928641ec8ebad34 P ff90de856dd542eab446899d181dcad0 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 326c74b0a8d84cdf86229f67ce3487f2 in term 4.
I20260709 13:24:17.472132 26798 leader_election.cc:304] T d52f26a0449247c1b928641ec8ebad34 P 326c74b0a8d84cdf86229f67ce3487f2 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 326c74b0a8d84cdf86229f67ce3487f2, ff90de856dd542eab446899d181dcad0; no voters: 
I20260709 13:24:17.472311 26904 raft_consensus.cc:2468] T d52f26a0449247c1b928641ec8ebad34 P 239ae18f3e44498fa52e20e9d6eec6b5 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 326c74b0a8d84cdf86229f67ce3487f2 in term 4.
I20260709 13:24:17.472918 27222 raft_consensus.cc:2804] T d52f26a0449247c1b928641ec8ebad34 P 326c74b0a8d84cdf86229f67ce3487f2 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:24:17.473448 27222 raft_consensus.cc:697] T d52f26a0449247c1b928641ec8ebad34 P 326c74b0a8d84cdf86229f67ce3487f2 [term 4 LEADER]: Becoming Leader. State: Replica: 326c74b0a8d84cdf86229f67ce3487f2, State: Running, Role: LEADER
I20260709 13:24:17.474362 27222 consensus_queue.cc:237] T d52f26a0449247c1b928641ec8ebad34 P 326c74b0a8d84cdf86229f67ce3487f2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff90de856dd542eab446899d181dcad0" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36311 } } peers { permanent_uuid: "326c74b0a8d84cdf86229f67ce3487f2" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35517 } } peers { permanent_uuid: "239ae18f3e44498fa52e20e9d6eec6b5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 39265 } }
I20260709 13:24:17.482254 26656 catalog_manager.cc:5697] T d52f26a0449247c1b928641ec8ebad34 P 326c74b0a8d84cdf86229f67ce3487f2 reported cstate change: term changed from 3 to 4, leader changed from 239ae18f3e44498fa52e20e9d6eec6b5 (127.14.177.131) to 326c74b0a8d84cdf86229f67ce3487f2 (127.14.177.130). New cstate: current_term: 4 leader_uuid: "326c74b0a8d84cdf86229f67ce3487f2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff90de856dd542eab446899d181dcad0" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36311 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "326c74b0a8d84cdf86229f67ce3487f2" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35517 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "239ae18f3e44498fa52e20e9d6eec6b5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 39265 } health_report { overall_health: UNKNOWN } } }
I20260709 13:24:17.547580 27212 raft_consensus.cc:993] T 2664382aca514208a9aa49316bdcaeeb P 239ae18f3e44498fa52e20e9d6eec6b5: : Instructing follower 326c74b0a8d84cdf86229f67ce3487f2 to start an election
I20260709 13:24:17.548117 27217 raft_consensus.cc:1081] T 2664382aca514208a9aa49316bdcaeeb P 239ae18f3e44498fa52e20e9d6eec6b5 [term 3 LEADER]: Signalling peer 326c74b0a8d84cdf86229f67ce3487f2 to start an election
I20260709 13:24:17.549718 26834 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "2664382aca514208a9aa49316bdcaeeb"
dest_uuid: "326c74b0a8d84cdf86229f67ce3487f2"
 from {username='slave'} at 127.0.0.1:40060
I20260709 13:24:17.550287 26834 raft_consensus.cc:493] T 2664382aca514208a9aa49316bdcaeeb P 326c74b0a8d84cdf86229f67ce3487f2 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:24:17.550529 26834 raft_consensus.cc:3060] T 2664382aca514208a9aa49316bdcaeeb P 326c74b0a8d84cdf86229f67ce3487f2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:24:17.555399 26834 raft_consensus.cc:515] T 2664382aca514208a9aa49316bdcaeeb P 326c74b0a8d84cdf86229f67ce3487f2 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff90de856dd542eab446899d181dcad0" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36311 } } peers { permanent_uuid: "326c74b0a8d84cdf86229f67ce3487f2" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35517 } } peers { permanent_uuid: "239ae18f3e44498fa52e20e9d6eec6b5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 39265 } }
I20260709 13:24:17.556771 26834 leader_election.cc:290] T 2664382aca514208a9aa49316bdcaeeb P 326c74b0a8d84cdf86229f67ce3487f2 [CANDIDATE]: Term 4 election: Requested vote from peers ff90de856dd542eab446899d181dcad0 (127.14.177.129:36311), 239ae18f3e44498fa52e20e9d6eec6b5 (127.14.177.131:39265)
I20260709 13:24:17.557817 27107 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2664382aca514208a9aa49316bdcaeeb" candidate_uuid: "326c74b0a8d84cdf86229f67ce3487f2" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "ff90de856dd542eab446899d181dcad0"
I20260709 13:24:17.557901 26904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2664382aca514208a9aa49316bdcaeeb" candidate_uuid: "326c74b0a8d84cdf86229f67ce3487f2" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "239ae18f3e44498fa52e20e9d6eec6b5"
I20260709 13:24:17.558468 27107 raft_consensus.cc:3060] T 2664382aca514208a9aa49316bdcaeeb P ff90de856dd542eab446899d181dcad0 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:24:17.558573 26904 raft_consensus.cc:3055] T 2664382aca514208a9aa49316bdcaeeb P 239ae18f3e44498fa52e20e9d6eec6b5 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:24:17.559058 26904 raft_consensus.cc:740] T 2664382aca514208a9aa49316bdcaeeb P 239ae18f3e44498fa52e20e9d6eec6b5 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 239ae18f3e44498fa52e20e9d6eec6b5, State: Running, Role: LEADER
I20260709 13:24:17.560022 26904 consensus_queue.cc:260] T 2664382aca514208a9aa49316bdcaeeb P 239ae18f3e44498fa52e20e9d6eec6b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff90de856dd542eab446899d181dcad0" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36311 } } peers { permanent_uuid: "326c74b0a8d84cdf86229f67ce3487f2" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35517 } } peers { permanent_uuid: "239ae18f3e44498fa52e20e9d6eec6b5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 39265 } }
I20260709 13:24:17.561218 26904 raft_consensus.cc:3060] T 2664382aca514208a9aa49316bdcaeeb P 239ae18f3e44498fa52e20e9d6eec6b5 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:24:17.564272 27107 raft_consensus.cc:2468] T 2664382aca514208a9aa49316bdcaeeb P ff90de856dd542eab446899d181dcad0 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 326c74b0a8d84cdf86229f67ce3487f2 in term 4.
I20260709 13:24:17.565882 26798 leader_election.cc:304] T 2664382aca514208a9aa49316bdcaeeb P 326c74b0a8d84cdf86229f67ce3487f2 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 326c74b0a8d84cdf86229f67ce3487f2, ff90de856dd542eab446899d181dcad0; no voters: 
I20260709 13:24:17.566732 27222 raft_consensus.cc:2804] T 2664382aca514208a9aa49316bdcaeeb P 326c74b0a8d84cdf86229f67ce3487f2 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:24:17.566869 26904 raft_consensus.cc:2468] T 2664382aca514208a9aa49316bdcaeeb P 239ae18f3e44498fa52e20e9d6eec6b5 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 326c74b0a8d84cdf86229f67ce3487f2 in term 4.
I20260709 13:24:17.567377 27222 raft_consensus.cc:697] T 2664382aca514208a9aa49316bdcaeeb P 326c74b0a8d84cdf86229f67ce3487f2 [term 4 LEADER]: Becoming Leader. State: Replica: 326c74b0a8d84cdf86229f67ce3487f2, State: Running, Role: LEADER
I20260709 13:24:17.568308 27222 consensus_queue.cc:237] T 2664382aca514208a9aa49316bdcaeeb P 326c74b0a8d84cdf86229f67ce3487f2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff90de856dd542eab446899d181dcad0" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36311 } } peers { permanent_uuid: "326c74b0a8d84cdf86229f67ce3487f2" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35517 } } peers { permanent_uuid: "239ae18f3e44498fa52e20e9d6eec6b5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 39265 } }
I20260709 13:24:17.574726 26656 catalog_manager.cc:5697] T 2664382aca514208a9aa49316bdcaeeb P 326c74b0a8d84cdf86229f67ce3487f2 reported cstate change: term changed from 3 to 4, leader changed from 239ae18f3e44498fa52e20e9d6eec6b5 (127.14.177.131) to 326c74b0a8d84cdf86229f67ce3487f2 (127.14.177.130). New cstate: current_term: 4 leader_uuid: "326c74b0a8d84cdf86229f67ce3487f2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff90de856dd542eab446899d181dcad0" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36311 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "326c74b0a8d84cdf86229f67ce3487f2" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35517 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "239ae18f3e44498fa52e20e9d6eec6b5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 39265 } health_report { overall_health: UNKNOWN } } }
W20260709 13:24:17.816586 26706 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 13:24:17.818722 27225 raft_consensus.cc:993] T 9ba9365ebe904f85840f6d049dbcda88 P 326c74b0a8d84cdf86229f67ce3487f2: : Instructing follower 239ae18f3e44498fa52e20e9d6eec6b5 to start an election
I20260709 13:24:17.819311 27222 raft_consensus.cc:1081] T 9ba9365ebe904f85840f6d049dbcda88 P 326c74b0a8d84cdf86229f67ce3487f2 [term 2 LEADER]: Signalling peer 239ae18f3e44498fa52e20e9d6eec6b5 to start an election
I20260709 13:24:17.820951 26904 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "9ba9365ebe904f85840f6d049dbcda88"
dest_uuid: "239ae18f3e44498fa52e20e9d6eec6b5"
 from {username='slave'} at 127.0.0.1:49792
I20260709 13:24:17.821470 26904 raft_consensus.cc:493] T 9ba9365ebe904f85840f6d049dbcda88 P 239ae18f3e44498fa52e20e9d6eec6b5 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:24:17.821770 26904 raft_consensus.cc:3060] T 9ba9365ebe904f85840f6d049dbcda88 P 239ae18f3e44498fa52e20e9d6eec6b5 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:24:17.826625 26904 raft_consensus.cc:515] T 9ba9365ebe904f85840f6d049dbcda88 P 239ae18f3e44498fa52e20e9d6eec6b5 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff90de856dd542eab446899d181dcad0" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36311 } } peers { permanent_uuid: "326c74b0a8d84cdf86229f67ce3487f2" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35517 } } peers { permanent_uuid: "239ae18f3e44498fa52e20e9d6eec6b5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 39265 } }
I20260709 13:24:17.828181 26904 leader_election.cc:290] T 9ba9365ebe904f85840f6d049dbcda88 P 239ae18f3e44498fa52e20e9d6eec6b5 [CANDIDATE]: Term 3 election: Requested vote from peers ff90de856dd542eab446899d181dcad0 (127.14.177.129:36311), 326c74b0a8d84cdf86229f67ce3487f2 (127.14.177.130:35517)
I20260709 13:24:17.829159 27107 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ba9365ebe904f85840f6d049dbcda88" candidate_uuid: "239ae18f3e44498fa52e20e9d6eec6b5" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "ff90de856dd542eab446899d181dcad0"
I20260709 13:24:17.829340 26834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ba9365ebe904f85840f6d049dbcda88" candidate_uuid: "239ae18f3e44498fa52e20e9d6eec6b5" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "326c74b0a8d84cdf86229f67ce3487f2"
I20260709 13:24:17.829797 27107 raft_consensus.cc:3060] T 9ba9365ebe904f85840f6d049dbcda88 P ff90de856dd542eab446899d181dcad0 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:24:17.829964 26834 raft_consensus.cc:3055] T 9ba9365ebe904f85840f6d049dbcda88 P 326c74b0a8d84cdf86229f67ce3487f2 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:24:17.830360 26834 raft_consensus.cc:740] T 9ba9365ebe904f85840f6d049dbcda88 P 326c74b0a8d84cdf86229f67ce3487f2 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 326c74b0a8d84cdf86229f67ce3487f2, State: Running, Role: LEADER
I20260709 13:24:17.831049 26834 consensus_queue.cc:260] T 9ba9365ebe904f85840f6d049dbcda88 P 326c74b0a8d84cdf86229f67ce3487f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff90de856dd542eab446899d181dcad0" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36311 } } peers { permanent_uuid: "326c74b0a8d84cdf86229f67ce3487f2" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35517 } } peers { permanent_uuid: "239ae18f3e44498fa52e20e9d6eec6b5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 39265 } }
I20260709 13:24:17.832082 26834 raft_consensus.cc:3060] T 9ba9365ebe904f85840f6d049dbcda88 P 326c74b0a8d84cdf86229f67ce3487f2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:24:17.835032 27107 raft_consensus.cc:2468] T 9ba9365ebe904f85840f6d049dbcda88 P ff90de856dd542eab446899d181dcad0 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 239ae18f3e44498fa52e20e9d6eec6b5 in term 3.
I20260709 13:24:17.836404 26872 leader_election.cc:304] T 9ba9365ebe904f85840f6d049dbcda88 P 239ae18f3e44498fa52e20e9d6eec6b5 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 239ae18f3e44498fa52e20e9d6eec6b5, ff90de856dd542eab446899d181dcad0; no voters: 
I20260709 13:24:17.837215 27217 raft_consensus.cc:2804] T 9ba9365ebe904f85840f6d049dbcda88 P 239ae18f3e44498fa52e20e9d6eec6b5 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:24:17.837766 27217 raft_consensus.cc:697] T 9ba9365ebe904f85840f6d049dbcda88 P 239ae18f3e44498fa52e20e9d6eec6b5 [term 3 LEADER]: Becoming Leader. State: Replica: 239ae18f3e44498fa52e20e9d6eec6b5, State: Running, Role: LEADER
I20260709 13:24:17.838341 26834 raft_consensus.cc:2468] T 9ba9365ebe904f85840f6d049dbcda88 P 326c74b0a8d84cdf86229f67ce3487f2 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 239ae18f3e44498fa52e20e9d6eec6b5 in term 3.
I20260709 13:24:17.838572 27217 consensus_queue.cc:237] T 9ba9365ebe904f85840f6d049dbcda88 P 239ae18f3e44498fa52e20e9d6eec6b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff90de856dd542eab446899d181dcad0" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36311 } } peers { permanent_uuid: "326c74b0a8d84cdf86229f67ce3487f2" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35517 } } peers { permanent_uuid: "239ae18f3e44498fa52e20e9d6eec6b5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 39265 } }
I20260709 13:24:17.846283 26656 catalog_manager.cc:5697] T 9ba9365ebe904f85840f6d049dbcda88 P 239ae18f3e44498fa52e20e9d6eec6b5 reported cstate change: term changed from 2 to 3, leader changed from 326c74b0a8d84cdf86229f67ce3487f2 (127.14.177.130) to 239ae18f3e44498fa52e20e9d6eec6b5 (127.14.177.131). New cstate: current_term: 3 leader_uuid: "239ae18f3e44498fa52e20e9d6eec6b5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff90de856dd542eab446899d181dcad0" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36311 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "326c74b0a8d84cdf86229f67ce3487f2" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35517 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "239ae18f3e44498fa52e20e9d6eec6b5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 39265 } health_report { overall_health: HEALTHY } } }
I20260709 13:24:17.955797 26904 raft_consensus.cc:1275] T d52f26a0449247c1b928641ec8ebad34 P 239ae18f3e44498fa52e20e9d6eec6b5 [term 4 FOLLOWER]: Refusing update from remote peer 326c74b0a8d84cdf86229f67ce3487f2: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:24:17.957048 27222 consensus_queue.cc:1048] T d52f26a0449247c1b928641ec8ebad34 P 326c74b0a8d84cdf86229f67ce3487f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "239ae18f3e44498fa52e20e9d6eec6b5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 39265 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:24:17.969697 26904 raft_consensus.cc:1275] T 2664382aca514208a9aa49316bdcaeeb P 239ae18f3e44498fa52e20e9d6eec6b5 [term 4 FOLLOWER]: Refusing update from remote peer 326c74b0a8d84cdf86229f67ce3487f2: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:24:17.970431 27107 raft_consensus.cc:1275] T d52f26a0449247c1b928641ec8ebad34 P ff90de856dd542eab446899d181dcad0 [term 4 FOLLOWER]: Refusing update from remote peer 326c74b0a8d84cdf86229f67ce3487f2: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:24:17.971200 27222 consensus_queue.cc:1048] T 2664382aca514208a9aa49316bdcaeeb P 326c74b0a8d84cdf86229f67ce3487f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "239ae18f3e44498fa52e20e9d6eec6b5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 39265 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:24:17.972870 27222 consensus_queue.cc:1048] T d52f26a0449247c1b928641ec8ebad34 P 326c74b0a8d84cdf86229f67ce3487f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff90de856dd542eab446899d181dcad0" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36311 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:24:17.988049 27107 raft_consensus.cc:1275] T 2664382aca514208a9aa49316bdcaeeb P ff90de856dd542eab446899d181dcad0 [term 4 FOLLOWER]: Refusing update from remote peer 326c74b0a8d84cdf86229f67ce3487f2: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:24:17.989409 27205 consensus_queue.cc:1048] T 2664382aca514208a9aa49316bdcaeeb P 326c74b0a8d84cdf86229f67ce3487f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff90de856dd542eab446899d181dcad0" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36311 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
W20260709 13:24:18.145799 26689 debug-util.cc:398] Leaking SignalData structure 0x7b08000ca8e0 after lost signal to thread 15049
W20260709 13:24:18.146886 26689 debug-util.cc:398] Leaking SignalData structure 0x7b080008a040 after lost signal to thread 26692
W20260709 13:24:18.148068 26689 debug-util.cc:398] Leaking SignalData structure 0x7b08002233e0 after lost signal to thread 26859
W20260709 13:24:18.149052 26689 debug-util.cc:398] Leaking SignalData structure 0x7b0800043a40 after lost signal to thread 26933
W20260709 13:24:18.150039 26689 debug-util.cc:398] Leaking SignalData structure 0x7b080027f460 after lost signal to thread 27133
I20260709 13:24:18.360172 27105 raft_consensus.cc:1275] T 9ba9365ebe904f85840f6d049dbcda88 P ff90de856dd542eab446899d181dcad0 [term 3 FOLLOWER]: Refusing update from remote peer 239ae18f3e44498fa52e20e9d6eec6b5: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:24:18.361913 27152 consensus_queue.cc:1048] T 9ba9365ebe904f85840f6d049dbcda88 P 239ae18f3e44498fa52e20e9d6eec6b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff90de856dd542eab446899d181dcad0" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36311 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:24:18.375272 26833 raft_consensus.cc:1275] T 9ba9365ebe904f85840f6d049dbcda88 P 326c74b0a8d84cdf86229f67ce3487f2 [term 3 FOLLOWER]: Refusing update from remote peer 239ae18f3e44498fa52e20e9d6eec6b5: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:24:18.376864 27217 consensus_queue.cc:1048] T 9ba9365ebe904f85840f6d049dbcda88 P 239ae18f3e44498fa52e20e9d6eec6b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "326c74b0a8d84cdf86229f67ce3487f2" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 35517 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:24:18.377869 15046 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:24:18.421937 15046 tablet_server.cc:179] TabletServer@127.14.177.129:36311 shutting down...
I20260709 13:24:18.453166 15046 ts_tablet_manager.cc:1507] Shutting down tablet manager...
W20260709 13:24:18.453949 26872 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 85 similar messages]
I20260709 13:24:18.455107 15046 tablet_replica.cc:333] stopping tablet replica
I20260709 13:24:18.456220 15046 raft_consensus.cc:2243] T bdb51cee932741eda8c12a88f07c890e P ff90de856dd542eab446899d181dcad0 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:24:18.456873 15046 raft_consensus.cc:2272] T bdb51cee932741eda8c12a88f07c890e P ff90de856dd542eab446899d181dcad0 [term 3 FOLLOWER]: Raft consensus is shut down!
W20260709 13:24:18.457795 26872 consensus_peers.cc:597] T 76c716abdcd6437b9dcee43317333298 P 239ae18f3e44498fa52e20e9d6eec6b5 -> Peer ff90de856dd542eab446899d181dcad0 (127.14.177.129:36311): Couldn't send request to peer ff90de856dd542eab446899d181dcad0. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:24:18.459311 15046 tablet_replica.cc:333] stopping tablet replica
I20260709 13:24:18.460009 15046 raft_consensus.cc:2243] T 2664382aca514208a9aa49316bdcaeeb P ff90de856dd542eab446899d181dcad0 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:24:18.460494 15046 raft_consensus.cc:2272] T 2664382aca514208a9aa49316bdcaeeb P ff90de856dd542eab446899d181dcad0 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:24:18.463442 15046 tablet_replica.cc:333] stopping tablet replica
I20260709 13:24:18.464051 15046 raft_consensus.cc:2243] T 92c3adeb279b4f5f88680a40a9664750 P ff90de856dd542eab446899d181dcad0 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:24:18.464502 15046 raft_consensus.cc:2272] T 92c3adeb279b4f5f88680a40a9664750 P ff90de856dd542eab446899d181dcad0 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:24:18.466650 15046 tablet_replica.cc:333] stopping tablet replica
I20260709 13:24:18.467518 15046 raft_consensus.cc:2243] T 7492e977a5f74ef09aa687b1d7ec89e3 P ff90de856dd542eab446899d181dcad0 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:24:18.468089 15046 raft_consensus.cc:2272] T 7492e977a5f74ef09aa687b1d7ec89e3 P ff90de856dd542eab446899d181dcad0 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:24:18.469973 15046 tablet_replica.cc:333] stopping tablet replica
I20260709 13:24:18.470551 15046 raft_consensus.cc:2243] T e341897686b14a2491c3a4bd24ac9e95 P ff90de856dd542eab446899d181dcad0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:24:18.470973 15046 raft_consensus.cc:2272] T e341897686b14a2491c3a4bd24ac9e95 P ff90de856dd542eab446899d181dcad0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:24:18.472873 15046 tablet_replica.cc:333] stopping tablet replica
I20260709 13:24:18.473438 15046 raft_consensus.cc:2243] T d52f26a0449247c1b928641ec8ebad34 P ff90de856dd542eab446899d181dcad0 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:24:18.473904 15046 raft_consensus.cc:2272] T d52f26a0449247c1b928641ec8ebad34 P ff90de856dd542eab446899d181dcad0 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:24:18.476480 15046 tablet_replica.cc:333] stopping tablet replica
I20260709 13:24:18.477088 15046 raft_consensus.cc:2243] T 9ba9365ebe904f85840f6d049dbcda88 P ff90de856dd542eab446899d181dcad0 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:24:18.477555 15046 raft_consensus.cc:2272] T 9ba9365ebe904f85840f6d049dbcda88 P ff90de856dd542eab446899d181dcad0 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:24:18.480182 15046 tablet_replica.cc:333] stopping tablet replica
I20260709 13:24:18.480782 15046 raft_consensus.cc:2243] T 76c716abdcd6437b9dcee43317333298 P ff90de856dd542eab446899d181dcad0 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:24:18.481271 15046 raft_consensus.cc:2272] T 76c716abdcd6437b9dcee43317333298 P ff90de856dd542eab446899d181dcad0 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:24:18.483347 15046 tablet_replica.cc:333] stopping tablet replica
I20260709 13:24:18.483891 15046 raft_consensus.cc:2243] T d5f6b403f0cb4c7a9180d07edb98d1bc P ff90de856dd542eab446899d181dcad0 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:24:18.484376 15046 raft_consensus.cc:2272] T d5f6b403f0cb4c7a9180d07edb98d1bc P ff90de856dd542eab446899d181dcad0 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:24:18.486347 15046 tablet_replica.cc:333] stopping tablet replica
I20260709 13:24:18.486923 15046 raft_consensus.cc:2243] T 3e489a04b06247608a195856b4739a36 P ff90de856dd542eab446899d181dcad0 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:24:18.487396 15046 raft_consensus.cc:2272] T 3e489a04b06247608a195856b4739a36 P ff90de856dd542eab446899d181dcad0 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:24:18.509593 15046 tablet_server.cc:196] TabletServer@127.14.177.129:36311 shutdown complete.
W20260709 13:24:18.539740 26798 consensus_peers.cc:597] T 2664382aca514208a9aa49316bdcaeeb P 326c74b0a8d84cdf86229f67ce3487f2 -> Peer ff90de856dd542eab446899d181dcad0 (127.14.177.129:36311): Couldn't send request to peer ff90de856dd542eab446899d181dcad0. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36311: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:24:18.543833 15046 tablet_server.cc:179] TabletServer@127.14.177.130:0 shutting down...
I20260709 13:24:18.584429 15046 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:24:18.585157 15046 tablet_replica.cc:333] T 76c716abdcd6437b9dcee43317333298 P 326c74b0a8d84cdf86229f67ce3487f2: stopping tablet replica
I20260709 13:24:18.585995 15046 raft_consensus.cc:2243] T 76c716abdcd6437b9dcee43317333298 P 326c74b0a8d84cdf86229f67ce3487f2 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:24:18.586514 15046 raft_consensus.cc:2272] T 76c716abdcd6437b9dcee43317333298 P 326c74b0a8d84cdf86229f67ce3487f2 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:24:18.588999 15046 tablet_replica.cc:333] T 9ba9365ebe904f85840f6d049dbcda88 P 326c74b0a8d84cdf86229f67ce3487f2: stopping tablet replica
I20260709 13:24:18.589826 15046 raft_consensus.cc:2243] T 9ba9365ebe904f85840f6d049dbcda88 P 326c74b0a8d84cdf86229f67ce3487f2 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:24:18.590301 15046 raft_consensus.cc:2272] T 9ba9365ebe904f85840f6d049dbcda88 P 326c74b0a8d84cdf86229f67ce3487f2 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:24:18.593291 15046 tablet_replica.cc:333] T 92c3adeb279b4f5f88680a40a9664750 P 326c74b0a8d84cdf86229f67ce3487f2: stopping tablet replica
I20260709 13:24:18.593947 15046 raft_consensus.cc:2243] T 92c3adeb279b4f5f88680a40a9664750 P 326c74b0a8d84cdf86229f67ce3487f2 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:24:18.594456 15046 raft_consensus.cc:2272] T 92c3adeb279b4f5f88680a40a9664750 P 326c74b0a8d84cdf86229f67ce3487f2 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:24:18.596944 15046 tablet_replica.cc:333] T 2664382aca514208a9aa49316bdcaeeb P 326c74b0a8d84cdf86229f67ce3487f2: stopping tablet replica
I20260709 13:24:18.597870 15046 raft_consensus.cc:2243] T 2664382aca514208a9aa49316bdcaeeb P 326c74b0a8d84cdf86229f67ce3487f2 [term 4 LEADER]: Raft consensus shutting down.
I20260709 13:24:18.598970 15046 raft_consensus.cc:2272] T 2664382aca514208a9aa49316bdcaeeb P 326c74b0a8d84cdf86229f67ce3487f2 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:24:18.602516 15046 tablet_replica.cc:333] T bdb51cee932741eda8c12a88f07c890e P 326c74b0a8d84cdf86229f67ce3487f2: stopping tablet replica
I20260709 13:24:18.603363 15046 raft_consensus.cc:2243] T bdb51cee932741eda8c12a88f07c890e P 326c74b0a8d84cdf86229f67ce3487f2 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:24:18.603870 15046 raft_consensus.cc:2272] T bdb51cee932741eda8c12a88f07c890e P 326c74b0a8d84cdf86229f67ce3487f2 [term 3 FOLLOWER]: Raft consensus is shut down!
W20260709 13:24:18.604517 26798 consensus_peers.cc:597] T d52f26a0449247c1b928641ec8ebad34 P 326c74b0a8d84cdf86229f67ce3487f2 -> Peer ff90de856dd542eab446899d181dcad0 (127.14.177.129:36311): Couldn't send request to peer ff90de856dd542eab446899d181dcad0. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36311: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:24:18.606391 15046 tablet_replica.cc:333] T d52f26a0449247c1b928641ec8ebad34 P 326c74b0a8d84cdf86229f67ce3487f2: stopping tablet replica
I20260709 13:24:18.607056 15046 raft_consensus.cc:2243] T d52f26a0449247c1b928641ec8ebad34 P 326c74b0a8d84cdf86229f67ce3487f2 [term 4 LEADER]: Raft consensus shutting down.
I20260709 13:24:18.607952 15046 raft_consensus.cc:2272] T d52f26a0449247c1b928641ec8ebad34 P 326c74b0a8d84cdf86229f67ce3487f2 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:24:18.610642 15046 tablet_replica.cc:333] T d5f6b403f0cb4c7a9180d07edb98d1bc P 326c74b0a8d84cdf86229f67ce3487f2: stopping tablet replica
I20260709 13:24:18.611260 15046 raft_consensus.cc:2243] T d5f6b403f0cb4c7a9180d07edb98d1bc P 326c74b0a8d84cdf86229f67ce3487f2 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:24:18.612033 15046 raft_consensus.cc:2272] T d5f6b403f0cb4c7a9180d07edb98d1bc P 326c74b0a8d84cdf86229f67ce3487f2 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:24:18.614297 15046 tablet_replica.cc:333] T 3e489a04b06247608a195856b4739a36 P 326c74b0a8d84cdf86229f67ce3487f2: stopping tablet replica
I20260709 13:24:18.614826 15046 raft_consensus.cc:2243] T 3e489a04b06247608a195856b4739a36 P 326c74b0a8d84cdf86229f67ce3487f2 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:24:18.615664 15046 raft_consensus.cc:2272] T 3e489a04b06247608a195856b4739a36 P 326c74b0a8d84cdf86229f67ce3487f2 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:24:18.617918 15046 tablet_replica.cc:333] T e341897686b14a2491c3a4bd24ac9e95 P 326c74b0a8d84cdf86229f67ce3487f2: stopping tablet replica
I20260709 13:24:18.618477 15046 raft_consensus.cc:2243] T e341897686b14a2491c3a4bd24ac9e95 P 326c74b0a8d84cdf86229f67ce3487f2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:24:18.618973 15046 raft_consensus.cc:2272] T e341897686b14a2491c3a4bd24ac9e95 P 326c74b0a8d84cdf86229f67ce3487f2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:24:18.621492 15046 tablet_replica.cc:333] T 7492e977a5f74ef09aa687b1d7ec89e3 P 326c74b0a8d84cdf86229f67ce3487f2: stopping tablet replica
I20260709 13:24:18.622031 15046 raft_consensus.cc:2243] T 7492e977a5f74ef09aa687b1d7ec89e3 P 326c74b0a8d84cdf86229f67ce3487f2 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:24:18.622846 15046 raft_consensus.cc:2272] T 7492e977a5f74ef09aa687b1d7ec89e3 P 326c74b0a8d84cdf86229f67ce3487f2 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:24:18.661023 15046 tablet_server.cc:196] TabletServer@127.14.177.130:0 shutdown complete.
W20260709 13:24:18.687371 26872 consensus_peers.cc:597] T 92c3adeb279b4f5f88680a40a9664750 P 239ae18f3e44498fa52e20e9d6eec6b5 -> Peer ff90de856dd542eab446899d181dcad0 (127.14.177.129:36311): Couldn't send request to peer ff90de856dd542eab446899d181dcad0. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36311: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:24:18.688195 26872 consensus_peers.cc:597] T 92c3adeb279b4f5f88680a40a9664750 P 239ae18f3e44498fa52e20e9d6eec6b5 -> Peer 326c74b0a8d84cdf86229f67ce3487f2 (127.14.177.130:35517): Couldn't send request to peer 326c74b0a8d84cdf86229f67ce3487f2. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.130:35517: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:24:18.694198 15046 tablet_server.cc:179] TabletServer@127.14.177.131:0 shutting down...
I20260709 13:24:18.742240 15046 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:24:18.743249 15046 tablet_replica.cc:333] T bdb51cee932741eda8c12a88f07c890e P 239ae18f3e44498fa52e20e9d6eec6b5: stopping tablet replica
I20260709 13:24:18.744674 15046 raft_consensus.cc:2243] T bdb51cee932741eda8c12a88f07c890e P 239ae18f3e44498fa52e20e9d6eec6b5 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:24:18.746043 15046 raft_consensus.cc:2272] T bdb51cee932741eda8c12a88f07c890e P 239ae18f3e44498fa52e20e9d6eec6b5 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:24:18.749825 15046 tablet_replica.cc:333] T d5f6b403f0cb4c7a9180d07edb98d1bc P 239ae18f3e44498fa52e20e9d6eec6b5: stopping tablet replica
I20260709 13:24:18.750777 15046 raft_consensus.cc:2243] T d5f6b403f0cb4c7a9180d07edb98d1bc P 239ae18f3e44498fa52e20e9d6eec6b5 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:24:18.751423 15046 raft_consensus.cc:2272] T d5f6b403f0cb4c7a9180d07edb98d1bc P 239ae18f3e44498fa52e20e9d6eec6b5 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:24:18.754756 15046 tablet_replica.cc:333] T 3e489a04b06247608a195856b4739a36 P 239ae18f3e44498fa52e20e9d6eec6b5: stopping tablet replica
I20260709 13:24:18.755837 15046 raft_consensus.cc:2243] T 3e489a04b06247608a195856b4739a36 P 239ae18f3e44498fa52e20e9d6eec6b5 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:24:18.756526 15046 raft_consensus.cc:2272] T 3e489a04b06247608a195856b4739a36 P 239ae18f3e44498fa52e20e9d6eec6b5 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:24:18.758198 26872 consensus_peers.cc:597] T e341897686b14a2491c3a4bd24ac9e95 P 239ae18f3e44498fa52e20e9d6eec6b5 -> Peer ff90de856dd542eab446899d181dcad0 (127.14.177.129:36311): Couldn't send request to peer ff90de856dd542eab446899d181dcad0. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36311: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:24:18.759840 15046 tablet_replica.cc:333] T d52f26a0449247c1b928641ec8ebad34 P 239ae18f3e44498fa52e20e9d6eec6b5: stopping tablet replica
I20260709 13:24:18.760982 15046 raft_consensus.cc:2243] T d52f26a0449247c1b928641ec8ebad34 P 239ae18f3e44498fa52e20e9d6eec6b5 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:24:18.761583 15046 raft_consensus.cc:2272] T d52f26a0449247c1b928641ec8ebad34 P 239ae18f3e44498fa52e20e9d6eec6b5 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:24:18.765209 15046 tablet_replica.cc:333] T e341897686b14a2491c3a4bd24ac9e95 P 239ae18f3e44498fa52e20e9d6eec6b5: stopping tablet replica
I20260709 13:24:18.766027 15046 raft_consensus.cc:2243] T e341897686b14a2491c3a4bd24ac9e95 P 239ae18f3e44498fa52e20e9d6eec6b5 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:24:18.767148 15046 raft_consensus.cc:2272] T e341897686b14a2491c3a4bd24ac9e95 P 239ae18f3e44498fa52e20e9d6eec6b5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:24:18.770540 15046 tablet_replica.cc:333] T 7492e977a5f74ef09aa687b1d7ec89e3 P 239ae18f3e44498fa52e20e9d6eec6b5: stopping tablet replica
I20260709 13:24:18.771431 15046 raft_consensus.cc:2243] T 7492e977a5f74ef09aa687b1d7ec89e3 P 239ae18f3e44498fa52e20e9d6eec6b5 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:24:18.772004 15046 raft_consensus.cc:2272] T 7492e977a5f74ef09aa687b1d7ec89e3 P 239ae18f3e44498fa52e20e9d6eec6b5 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:24:18.775274 15046 tablet_replica.cc:333] T 9ba9365ebe904f85840f6d049dbcda88 P 239ae18f3e44498fa52e20e9d6eec6b5: stopping tablet replica
I20260709 13:24:18.776156 15046 raft_consensus.cc:2243] T 9ba9365ebe904f85840f6d049dbcda88 P 239ae18f3e44498fa52e20e9d6eec6b5 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:24:18.777302 15046 raft_consensus.cc:2272] T 9ba9365ebe904f85840f6d049dbcda88 P 239ae18f3e44498fa52e20e9d6eec6b5 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:24:18.780786 15046 tablet_replica.cc:333] T 76c716abdcd6437b9dcee43317333298 P 239ae18f3e44498fa52e20e9d6eec6b5: stopping tablet replica
I20260709 13:24:18.781733 15046 raft_consensus.cc:2243] T 76c716abdcd6437b9dcee43317333298 P 239ae18f3e44498fa52e20e9d6eec6b5 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:24:18.782917 15046 raft_consensus.cc:2272] T 76c716abdcd6437b9dcee43317333298 P 239ae18f3e44498fa52e20e9d6eec6b5 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:24:18.786338 15046 tablet_replica.cc:333] T 92c3adeb279b4f5f88680a40a9664750 P 239ae18f3e44498fa52e20e9d6eec6b5: stopping tablet replica
I20260709 13:24:18.787132 15046 raft_consensus.cc:2243] T 92c3adeb279b4f5f88680a40a9664750 P 239ae18f3e44498fa52e20e9d6eec6b5 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:24:18.788209 15046 raft_consensus.cc:2272] T 92c3adeb279b4f5f88680a40a9664750 P 239ae18f3e44498fa52e20e9d6eec6b5 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:24:18.791119 15046 tablet_replica.cc:333] T 2664382aca514208a9aa49316bdcaeeb P 239ae18f3e44498fa52e20e9d6eec6b5: stopping tablet replica
I20260709 13:24:18.791884 15046 raft_consensus.cc:2243] T 2664382aca514208a9aa49316bdcaeeb P 239ae18f3e44498fa52e20e9d6eec6b5 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:24:18.792428 15046 raft_consensus.cc:2272] T 2664382aca514208a9aa49316bdcaeeb P 239ae18f3e44498fa52e20e9d6eec6b5 [term 4 FOLLOWER]: Raft consensus is shut down!
W20260709 13:24:18.817893 26706 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 13:24:18.827728 15046 tablet_server.cc:196] TabletServer@127.14.177.131:0 shutdown complete.
I20260709 13:24:18.873750 15046 master.cc:562] Master@127.14.177.190:39149 shutting down...
I20260709 13:24:18.914641 15046 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 3bc15f9174194ef6acd0a8614cced0ea [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:24:18.915447 15046 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 3bc15f9174194ef6acd0a8614cced0ea [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:24:18.915915 15046 tablet_replica.cc:333] T 00000000000000000000000000000000 P 3bc15f9174194ef6acd0a8614cced0ea: stopping tablet replica
W20260709 13:24:19.936367 15046 thread.cc:527] Waited for 1000ms trying to join with diag-logger (tid 26689)
W20260709 13:24:20.937121 15046 thread.cc:527] Waited for 2000ms trying to join with diag-logger (tid 26689)
I20260709 13:24:21.017767 15046 master.cc:584] Master@127.14.177.190:39149 shutdown complete.
[       OK ] LeaderRebalancerTest.TestMaintenanceMode (15424 ms)
[ RUN      ] LeaderRebalancerTest.MultiTableLeaderBalance
I20260709 13:24:21.080473 15046 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.14.177.190:38139
I20260709 13:24:21.081601 15046 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:24:21.087710 27254 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:24:21.088701 27255 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:24:21.089520 27257 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:24:21.089560 15046 server_base.cc:1061] running on GCE node
I20260709 13:24:21.090711 15046 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:24:21.090893 15046 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:24:21.091105 15046 hybrid_clock.cc:648] HybridClock initialized: now 1783603461091086 us; error 0 us; skew 500 ppm
I20260709 13:24:21.091722 15046 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:24:21.094300 15046 webserver.cc:533] Webserver started at http://127.14.177.190:37289/ using document root <none> and password file <none>
I20260709 13:24:21.094831 15046 fs_manager.cc:362] Metadata directory not provided
I20260709 13:24:21.095016 15046 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:24:21.095350 15046 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:24:21.096742 15046 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskdH_Ma7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783603093801613-15046-0/minicluster-data/master-0-root/instance:
uuid: "6728a69b22e545c9b04d1fa45a60a637"
format_stamp: "Formatted at 2026-07-09 13:24:21 on dist-test-slave-v5z2"
I20260709 13:24:21.101394 15046 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.005s	sys 0.000s
I20260709 13:24:21.105139 27262 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:24:21.105896 15046 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260709 13:24:21.106223 15046 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskdH_Ma7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783603093801613-15046-0/minicluster-data/master-0-root
uuid: "6728a69b22e545c9b04d1fa45a60a637"
format_stamp: "Formatted at 2026-07-09 13:24:21 on dist-test-slave-v5z2"
I20260709 13:24:21.106493 15046 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdH_Ma7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783603093801613-15046-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskdH_Ma7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783603093801613-15046-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskdH_Ma7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783603093801613-15046-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:24:21.126968 15046 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:24:21.128376 15046 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:24:21.172610 15046 rpc_server.cc:307] RPC server started. Bound to: 127.14.177.190:38139
I20260709 13:24:21.172744 27313 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.14.177.190:38139 every 8 connection(s)
I20260709 13:24:21.177265 27314 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:24:21.192220 27314 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6728a69b22e545c9b04d1fa45a60a637: Bootstrap starting.
I20260709 13:24:21.197584 27314 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6728a69b22e545c9b04d1fa45a60a637: Neither blocks nor log segments found. Creating new log.
I20260709 13:24:21.202500 27314 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6728a69b22e545c9b04d1fa45a60a637: No bootstrap required, opened a new log
I20260709 13:24:21.205403 27314 raft_consensus.cc:359] T 00000000000000000000000000000000 P 6728a69b22e545c9b04d1fa45a60a637 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6728a69b22e545c9b04d1fa45a60a637" member_type: VOTER }
I20260709 13:24:21.205999 27314 raft_consensus.cc:385] T 00000000000000000000000000000000 P 6728a69b22e545c9b04d1fa45a60a637 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:24:21.206313 27314 raft_consensus.cc:740] T 00000000000000000000000000000000 P 6728a69b22e545c9b04d1fa45a60a637 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6728a69b22e545c9b04d1fa45a60a637, State: Initialized, Role: FOLLOWER
I20260709 13:24:21.207115 27314 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6728a69b22e545c9b04d1fa45a60a637 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6728a69b22e545c9b04d1fa45a60a637" member_type: VOTER }
I20260709 13:24:21.207715 27314 raft_consensus.cc:399] T 00000000000000000000000000000000 P 6728a69b22e545c9b04d1fa45a60a637 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 13:24:21.207967 27314 raft_consensus.cc:493] T 00000000000000000000000000000000 P 6728a69b22e545c9b04d1fa45a60a637 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 13:24:21.208402 27314 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 6728a69b22e545c9b04d1fa45a60a637 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:24:21.214565 27314 raft_consensus.cc:515] T 00000000000000000000000000000000 P 6728a69b22e545c9b04d1fa45a60a637 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6728a69b22e545c9b04d1fa45a60a637" member_type: VOTER }
I20260709 13:24:21.215260 27314 leader_election.cc:304] T 00000000000000000000000000000000 P 6728a69b22e545c9b04d1fa45a60a637 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6728a69b22e545c9b04d1fa45a60a637; no voters: 
I20260709 13:24:21.216553 27314 leader_election.cc:290] T 00000000000000000000000000000000 P 6728a69b22e545c9b04d1fa45a60a637 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 13:24:21.217043 27317 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 6728a69b22e545c9b04d1fa45a60a637 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:24:21.218742 27317 raft_consensus.cc:697] T 00000000000000000000000000000000 P 6728a69b22e545c9b04d1fa45a60a637 [term 1 LEADER]: Becoming Leader. State: Replica: 6728a69b22e545c9b04d1fa45a60a637, State: Running, Role: LEADER
I20260709 13:24:21.219672 27314 sys_catalog.cc:565] T 00000000000000000000000000000000 P 6728a69b22e545c9b04d1fa45a60a637 [sys.catalog]: configured and running, proceeding with master startup.
I20260709 13:24:21.219597 27317 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6728a69b22e545c9b04d1fa45a60a637 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6728a69b22e545c9b04d1fa45a60a637" member_type: VOTER }
I20260709 13:24:21.228050 27318 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6728a69b22e545c9b04d1fa45a60a637 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6728a69b22e545c9b04d1fa45a60a637" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6728a69b22e545c9b04d1fa45a60a637" member_type: VOTER } }
I20260709 13:24:21.228827 27318 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6728a69b22e545c9b04d1fa45a60a637 [sys.catalog]: This master's current role is: LEADER
I20260709 13:24:21.228916 27319 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6728a69b22e545c9b04d1fa45a60a637 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6728a69b22e545c9b04d1fa45a60a637. Latest consensus state: current_term: 1 leader_uuid: "6728a69b22e545c9b04d1fa45a60a637" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6728a69b22e545c9b04d1fa45a60a637" member_type: VOTER } }
I20260709 13:24:21.229614 27319 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6728a69b22e545c9b04d1fa45a60a637 [sys.catalog]: This master's current role is: LEADER
I20260709 13:24:21.231129 27327 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 13:24:21.239430 27327 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 13:24:21.240609 15046 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 13:24:21.249974 27327 catalog_manager.cc:1383] Generated new cluster ID: ade5a0caa33c42c9b5d9e2745f2ec554
I20260709 13:24:21.250283 27327 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 13:24:21.269291 27327 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 13:24:21.270773 27327 catalog_manager.cc:1540] Loading token signing keys...
I20260709 13:24:21.283269 27327 catalog_manager.cc:6070] T 00000000000000000000000000000000 P 6728a69b22e545c9b04d1fa45a60a637: Generated new TSK 0
I20260709 13:24:21.283911 27327 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 13:24:21.307893 15046 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:24:21.314601 27335 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:24:21.315766 27336 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:24:21.317710 27338 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:24:21.318346 15046 server_base.cc:1061] running on GCE node
I20260709 13:24:21.319422 15046 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:24:21.319656 15046 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:24:21.319825 15046 hybrid_clock.cc:648] HybridClock initialized: now 1783603461319805 us; error 0 us; skew 500 ppm
I20260709 13:24:21.320405 15046 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:24:21.322849 15046 webserver.cc:533] Webserver started at http://127.14.177.129:41209/ using document root <none> and password file <none>
I20260709 13:24:21.323410 15046 fs_manager.cc:362] Metadata directory not provided
I20260709 13:24:21.323623 15046 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:24:21.323889 15046 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:24:21.325273 15046 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskdH_Ma7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783603093801613-15046-0/minicluster-data/ts-0-root/instance:
uuid: "8dc4c6d0eae14641b9c5e44d31fcd335"
format_stamp: "Formatted at 2026-07-09 13:24:21 on dist-test-slave-v5z2"
I20260709 13:24:21.330329 15046 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.006s	sys 0.000s
I20260709 13:24:21.334312 27343 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:24:21.335309 15046 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.000s
I20260709 13:24:21.335755 15046 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskdH_Ma7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783603093801613-15046-0/minicluster-data/ts-0-root
uuid: "8dc4c6d0eae14641b9c5e44d31fcd335"
format_stamp: "Formatted at 2026-07-09 13:24:21 on dist-test-slave-v5z2"
I20260709 13:24:21.336084 15046 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdH_Ma7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783603093801613-15046-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskdH_Ma7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783603093801613-15046-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskdH_Ma7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783603093801613-15046-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:24:21.351686 15046 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:24:21.353186 15046 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:24:21.354898 15046 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:24:21.357617 15046 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:24:21.357892 15046 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:24:21.358209 15046 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:24:21.358460 15046 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:24:21.406061 15046 rpc_server.cc:307] RPC server started. Bound to: 127.14.177.129:36523
I20260709 13:24:21.406190 27405 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.14.177.129:36523 every 8 connection(s)
I20260709 13:24:21.412514 15046 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:24:21.421592 27410 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:24:21.421775 27411 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:24:21.425781 27413 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:24:21.427213 15046 server_base.cc:1061] running on GCE node
I20260709 13:24:21.427405 27406 heartbeater.cc:344] Connected to a master server at 127.14.177.190:38139
I20260709 13:24:21.428197 27406 heartbeater.cc:461] Registering TS with master...
I20260709 13:24:21.429065 15046 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
I20260709 13:24:21.429230 27406 heartbeater.cc:507] Master 127.14.177.190:38139 requested a full tablet report, sending...
W20260709 13:24:21.429507 15046 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:24:21.429852 15046 hybrid_clock.cc:648] HybridClock initialized: now 1783603461429828 us; error 0 us; skew 500 ppm
I20260709 13:24:21.430691 15046 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:24:21.432193 27279 ts_manager.cc:194] Registered new tserver with Master: 8dc4c6d0eae14641b9c5e44d31fcd335 (127.14.177.129:36523)
I20260709 13:24:21.434900 15046 webserver.cc:533] Webserver started at http://127.14.177.130:45053/ using document root <none> and password file <none>
I20260709 13:24:21.435056 27279 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:34098
I20260709 13:24:21.435703 15046 fs_manager.cc:362] Metadata directory not provided
I20260709 13:24:21.435977 15046 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:24:21.436290 15046 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:24:21.437803 15046 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskdH_Ma7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783603093801613-15046-0/minicluster-data/ts-1-root/instance:
uuid: "29bb2c7b65094fafa8a7c5e5ecee48de"
format_stamp: "Formatted at 2026-07-09 13:24:21 on dist-test-slave-v5z2"
I20260709 13:24:21.443342 15046 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.006s	sys 0.000s
I20260709 13:24:21.447536 27418 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:24:21.448398 15046 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.004s	sys 0.000s
I20260709 13:24:21.448822 15046 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskdH_Ma7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783603093801613-15046-0/minicluster-data/ts-1-root
uuid: "29bb2c7b65094fafa8a7c5e5ecee48de"
format_stamp: "Formatted at 2026-07-09 13:24:21 on dist-test-slave-v5z2"
I20260709 13:24:21.449162 15046 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdH_Ma7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783603093801613-15046-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskdH_Ma7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783603093801613-15046-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskdH_Ma7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783603093801613-15046-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:24:21.461105 15046 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:24:21.462447 15046 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:24:21.464215 15046 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:24:21.466801 15046 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:24:21.467060 15046 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:24:21.467391 15046 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:24:21.467597 15046 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:24:21.517616 15046 rpc_server.cc:307] RPC server started. Bound to: 127.14.177.130:43785
I20260709 13:24:21.517728 27480 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.14.177.130:43785 every 8 connection(s)
I20260709 13:24:21.523484 15046 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:24:21.532757 27484 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:24:21.536604 27485 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:24:21.536896 27487 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:24:21.537761 15046 server_base.cc:1061] running on GCE node
I20260709 13:24:21.538976 15046 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:24:21.539311 15046 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:24:21.539534 15046 hybrid_clock.cc:648] HybridClock initialized: now 1783603461539507 us; error 0 us; skew 500 ppm
I20260709 13:24:21.539752 27481 heartbeater.cc:344] Connected to a master server at 127.14.177.190:38139
I20260709 13:24:21.540266 27481 heartbeater.cc:461] Registering TS with master...
I20260709 13:24:21.540396 15046 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:24:21.541339 27481 heartbeater.cc:507] Master 127.14.177.190:38139 requested a full tablet report, sending...
I20260709 13:24:21.543805 15046 webserver.cc:533] Webserver started at http://127.14.177.131:33647/ using document root <none> and password file <none>
I20260709 13:24:21.544116 27279 ts_manager.cc:194] Registered new tserver with Master: 29bb2c7b65094fafa8a7c5e5ecee48de (127.14.177.130:43785)
I20260709 13:24:21.544514 15046 fs_manager.cc:362] Metadata directory not provided
I20260709 13:24:21.544831 15046 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:24:21.545125 15046 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:24:21.545996 27279 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:34106
I20260709 13:24:21.546676 15046 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskdH_Ma7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783603093801613-15046-0/minicluster-data/ts-2-root/instance:
uuid: "6c8db9dae5644d6290b55e3cae161043"
format_stamp: "Formatted at 2026-07-09 13:24:21 on dist-test-slave-v5z2"
I20260709 13:24:21.552608 15046 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.003s	sys 0.002s
I20260709 13:24:21.556716 27492 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:24:21.557696 15046 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.001s
I20260709 13:24:21.558069 15046 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskdH_Ma7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783603093801613-15046-0/minicluster-data/ts-2-root
uuid: "6c8db9dae5644d6290b55e3cae161043"
format_stamp: "Formatted at 2026-07-09 13:24:21 on dist-test-slave-v5z2"
I20260709 13:24:21.558411 15046 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdH_Ma7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783603093801613-15046-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskdH_Ma7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783603093801613-15046-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskdH_Ma7/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783603093801613-15046-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:24:21.581616 15046 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:24:21.582976 15046 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:24:21.584698 15046 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:24:21.587283 15046 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:24:21.587565 15046 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:24:21.587872 15046 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:24:21.588061 15046 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:24:21.637955 15046 rpc_server.cc:307] RPC server started. Bound to: 127.14.177.131:39577
I20260709 13:24:21.638049 27554 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.14.177.131:39577 every 8 connection(s)
I20260709 13:24:21.653486 27555 heartbeater.cc:344] Connected to a master server at 127.14.177.190:38139
I20260709 13:24:21.653854 27555 heartbeater.cc:461] Registering TS with master...
I20260709 13:24:21.654670 27555 heartbeater.cc:507] Master 127.14.177.190:38139 requested a full tablet report, sending...
I20260709 13:24:21.656613 27279 ts_manager.cc:194] Registered new tserver with Master: 6c8db9dae5644d6290b55e3cae161043 (127.14.177.131:39577)
I20260709 13:24:21.656800 15046 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.014974363s
I20260709 13:24:21.658144 27279 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:34114
I20260709 13:24:21.685794 27279 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34124:
name: "multi_table_leader_balance_table1"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\377\377\377?""\004\001\000\377\377\377?"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 13:24:21.688203 27279 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table multi_table_leader_balance_table1 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 13:24:21.738343 27520 tablet_service.cc:1511] Processing CreateTablet for tablet 341e749fb378438e8fa98ebcb9066d34 (DEFAULT_TABLE table=multi_table_leader_balance_table1 [id=1367f4fd2a954368970e195e5fbe82be]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20260709 13:24:21.740023 27520 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 341e749fb378438e8fa98ebcb9066d34. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:24:21.739662 27519 tablet_service.cc:1511] Processing CreateTablet for tablet 2aaf9a0870eb46249ccf60a2123fd678 (DEFAULT_TABLE table=multi_table_leader_balance_table1 [id=1367f4fd2a954368970e195e5fbe82be]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20260709 13:24:21.741302 27519 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2aaf9a0870eb46249ccf60a2123fd678. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:24:21.749415 27371 tablet_service.cc:1511] Processing CreateTablet for tablet 341e749fb378438e8fa98ebcb9066d34 (DEFAULT_TABLE table=multi_table_leader_balance_table1 [id=1367f4fd2a954368970e195e5fbe82be]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20260709 13:24:21.751020 27371 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 341e749fb378438e8fa98ebcb9066d34. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:24:21.751190 27446 tablet_service.cc:1511] Processing CreateTablet for tablet 341e749fb378438e8fa98ebcb9066d34 (DEFAULT_TABLE table=multi_table_leader_balance_table1 [id=1367f4fd2a954368970e195e5fbe82be]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20260709 13:24:21.751732 27445 tablet_service.cc:1511] Processing CreateTablet for tablet 2aaf9a0870eb46249ccf60a2123fd678 (DEFAULT_TABLE table=multi_table_leader_balance_table1 [id=1367f4fd2a954368970e195e5fbe82be]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20260709 13:24:21.751981 27370 tablet_service.cc:1511] Processing CreateTablet for tablet 2aaf9a0870eb46249ccf60a2123fd678 (DEFAULT_TABLE table=multi_table_leader_balance_table1 [id=1367f4fd2a954368970e195e5fbe82be]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20260709 13:24:21.752804 27446 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 341e749fb378438e8fa98ebcb9066d34. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:24:21.753343 27370 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2aaf9a0870eb46249ccf60a2123fd678. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:24:21.753715 27445 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2aaf9a0870eb46249ccf60a2123fd678. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:24:21.781055 27570 tablet_bootstrap.cc:492] T 341e749fb378438e8fa98ebcb9066d34 P 8dc4c6d0eae14641b9c5e44d31fcd335: Bootstrap starting.
I20260709 13:24:21.787293 27571 tablet_bootstrap.cc:492] T 341e749fb378438e8fa98ebcb9066d34 P 29bb2c7b65094fafa8a7c5e5ecee48de: Bootstrap starting.
I20260709 13:24:21.790318 27570 tablet_bootstrap.cc:654] T 341e749fb378438e8fa98ebcb9066d34 P 8dc4c6d0eae14641b9c5e44d31fcd335: Neither blocks nor log segments found. Creating new log.
I20260709 13:24:21.791203 27572 tablet_bootstrap.cc:492] T 341e749fb378438e8fa98ebcb9066d34 P 6c8db9dae5644d6290b55e3cae161043: Bootstrap starting.
I20260709 13:24:21.794193 27571 tablet_bootstrap.cc:654] T 341e749fb378438e8fa98ebcb9066d34 P 29bb2c7b65094fafa8a7c5e5ecee48de: Neither blocks nor log segments found. Creating new log.
I20260709 13:24:21.797452 27570 tablet_bootstrap.cc:492] T 341e749fb378438e8fa98ebcb9066d34 P 8dc4c6d0eae14641b9c5e44d31fcd335: No bootstrap required, opened a new log
I20260709 13:24:21.797976 27570 ts_tablet_manager.cc:1403] T 341e749fb378438e8fa98ebcb9066d34 P 8dc4c6d0eae14641b9c5e44d31fcd335: Time spent bootstrapping tablet: real 0.017s	user 0.012s	sys 0.000s
I20260709 13:24:21.799472 27572 tablet_bootstrap.cc:654] T 341e749fb378438e8fa98ebcb9066d34 P 6c8db9dae5644d6290b55e3cae161043: Neither blocks nor log segments found. Creating new log.
I20260709 13:24:21.799724 27571 tablet_bootstrap.cc:492] T 341e749fb378438e8fa98ebcb9066d34 P 29bb2c7b65094fafa8a7c5e5ecee48de: No bootstrap required, opened a new log
I20260709 13:24:21.800252 27571 ts_tablet_manager.cc:1403] T 341e749fb378438e8fa98ebcb9066d34 P 29bb2c7b65094fafa8a7c5e5ecee48de: Time spent bootstrapping tablet: real 0.013s	user 0.008s	sys 0.004s
I20260709 13:24:21.801414 27570 raft_consensus.cc:359] T 341e749fb378438e8fa98ebcb9066d34 P 8dc4c6d0eae14641b9c5e44d31fcd335 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c8db9dae5644d6290b55e3cae161043" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 39577 } } peers { permanent_uuid: "29bb2c7b65094fafa8a7c5e5ecee48de" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 43785 } } peers { permanent_uuid: "8dc4c6d0eae14641b9c5e44d31fcd335" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36523 } }
I20260709 13:24:21.802232 27570 raft_consensus.cc:385] T 341e749fb378438e8fa98ebcb9066d34 P 8dc4c6d0eae14641b9c5e44d31fcd335 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:24:21.802574 27570 raft_consensus.cc:740] T 341e749fb378438e8fa98ebcb9066d34 P 8dc4c6d0eae14641b9c5e44d31fcd335 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8dc4c6d0eae14641b9c5e44d31fcd335, State: Initialized, Role: FOLLOWER
I20260709 13:24:21.803125 27571 raft_consensus.cc:359] T 341e749fb378438e8fa98ebcb9066d34 P 29bb2c7b65094fafa8a7c5e5ecee48de [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c8db9dae5644d6290b55e3cae161043" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 39577 } } peers { permanent_uuid: "29bb2c7b65094fafa8a7c5e5ecee48de" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 43785 } } peers { permanent_uuid: "8dc4c6d0eae14641b9c5e44d31fcd335" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36523 } }
I20260709 13:24:21.804004 27571 raft_consensus.cc:385] T 341e749fb378438e8fa98ebcb9066d34 P 29bb2c7b65094fafa8a7c5e5ecee48de [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:24:21.803565 27570 consensus_queue.cc:260] T 341e749fb378438e8fa98ebcb9066d34 P 8dc4c6d0eae14641b9c5e44d31fcd335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c8db9dae5644d6290b55e3cae161043" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 39577 } } peers { permanent_uuid: "29bb2c7b65094fafa8a7c5e5ecee48de" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 43785 } } peers { permanent_uuid: "8dc4c6d0eae14641b9c5e44d31fcd335" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36523 } }
I20260709 13:24:21.804280 27571 raft_consensus.cc:740] T 341e749fb378438e8fa98ebcb9066d34 P 29bb2c7b65094fafa8a7c5e5ecee48de [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 29bb2c7b65094fafa8a7c5e5ecee48de, State: Initialized, Role: FOLLOWER
I20260709 13:24:21.805153 27571 consensus_queue.cc:260] T 341e749fb378438e8fa98ebcb9066d34 P 29bb2c7b65094fafa8a7c5e5ecee48de [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c8db9dae5644d6290b55e3cae161043" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 39577 } } peers { permanent_uuid: "29bb2c7b65094fafa8a7c5e5ecee48de" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 43785 } } peers { permanent_uuid: "8dc4c6d0eae14641b9c5e44d31fcd335" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36523 } }
I20260709 13:24:21.805397 27572 tablet_bootstrap.cc:492] T 341e749fb378438e8fa98ebcb9066d34 P 6c8db9dae5644d6290b55e3cae161043: No bootstrap required, opened a new log
I20260709 13:24:21.806111 27572 ts_tablet_manager.cc:1403] T 341e749fb378438e8fa98ebcb9066d34 P 6c8db9dae5644d6290b55e3cae161043: Time spent bootstrapping tablet: real 0.015s	user 0.003s	sys 0.011s
I20260709 13:24:21.807500 27481 heartbeater.cc:499] Master 127.14.177.190:38139 was elected leader, sending a full tablet report...
I20260709 13:24:21.808007 27571 ts_tablet_manager.cc:1434] T 341e749fb378438e8fa98ebcb9066d34 P 29bb2c7b65094fafa8a7c5e5ecee48de: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 13:24:21.808990 27571 tablet_bootstrap.cc:492] T 2aaf9a0870eb46249ccf60a2123fd678 P 29bb2c7b65094fafa8a7c5e5ecee48de: Bootstrap starting.
I20260709 13:24:21.811039 27572 raft_consensus.cc:359] T 341e749fb378438e8fa98ebcb9066d34 P 6c8db9dae5644d6290b55e3cae161043 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c8db9dae5644d6290b55e3cae161043" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 39577 } } peers { permanent_uuid: "29bb2c7b65094fafa8a7c5e5ecee48de" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 43785 } } peers { permanent_uuid: "8dc4c6d0eae14641b9c5e44d31fcd335" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36523 } }
I20260709 13:24:21.812141 27572 raft_consensus.cc:385] T 341e749fb378438e8fa98ebcb9066d34 P 6c8db9dae5644d6290b55e3cae161043 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:24:21.812577 27572 raft_consensus.cc:740] T 341e749fb378438e8fa98ebcb9066d34 P 6c8db9dae5644d6290b55e3cae161043 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c8db9dae5644d6290b55e3cae161043, State: Initialized, Role: FOLLOWER
I20260709 13:24:21.813586 27572 consensus_queue.cc:260] T 341e749fb378438e8fa98ebcb9066d34 P 6c8db9dae5644d6290b55e3cae161043 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c8db9dae5644d6290b55e3cae161043" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 39577 } } peers { permanent_uuid: "29bb2c7b65094fafa8a7c5e5ecee48de" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 43785 } } peers { permanent_uuid: "8dc4c6d0eae14641b9c5e44d31fcd335" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36523 } }
I20260709 13:24:21.816646 27406 heartbeater.cc:499] Master 127.14.177.190:38139 was elected leader, sending a full tablet report...
I20260709 13:24:21.817001 27570 ts_tablet_manager.cc:1434] T 341e749fb378438e8fa98ebcb9066d34 P 8dc4c6d0eae14641b9c5e44d31fcd335: Time spent starting tablet: real 0.019s	user 0.016s	sys 0.004s
I20260709 13:24:21.818011 27570 tablet_bootstrap.cc:492] T 2aaf9a0870eb46249ccf60a2123fd678 P 8dc4c6d0eae14641b9c5e44d31fcd335: Bootstrap starting.
I20260709 13:24:21.816689 27571 tablet_bootstrap.cc:654] T 2aaf9a0870eb46249ccf60a2123fd678 P 29bb2c7b65094fafa8a7c5e5ecee48de: Neither blocks nor log segments found. Creating new log.
I20260709 13:24:21.825525 27555 heartbeater.cc:499] Master 127.14.177.190:38139 was elected leader, sending a full tablet report...
I20260709 13:24:21.825968 27572 ts_tablet_manager.cc:1434] T 341e749fb378438e8fa98ebcb9066d34 P 6c8db9dae5644d6290b55e3cae161043: Time spent starting tablet: real 0.019s	user 0.011s	sys 0.001s
I20260709 13:24:21.826897 27572 tablet_bootstrap.cc:492] T 2aaf9a0870eb46249ccf60a2123fd678 P 6c8db9dae5644d6290b55e3cae161043: Bootstrap starting.
I20260709 13:24:21.827803 27571 tablet_bootstrap.cc:492] T 2aaf9a0870eb46249ccf60a2123fd678 P 29bb2c7b65094fafa8a7c5e5ecee48de: No bootstrap required, opened a new log
I20260709 13:24:21.828365 27571 ts_tablet_manager.cc:1403] T 2aaf9a0870eb46249ccf60a2123fd678 P 29bb2c7b65094fafa8a7c5e5ecee48de: Time spent bootstrapping tablet: real 0.020s	user 0.013s	sys 0.000s
I20260709 13:24:21.831645 27571 raft_consensus.cc:359] T 2aaf9a0870eb46249ccf60a2123fd678 P 29bb2c7b65094fafa8a7c5e5ecee48de [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dc4c6d0eae14641b9c5e44d31fcd335" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36523 } } peers { permanent_uuid: "29bb2c7b65094fafa8a7c5e5ecee48de" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 43785 } } peers { permanent_uuid: "6c8db9dae5644d6290b55e3cae161043" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 39577 } }
I20260709 13:24:21.832443 27571 raft_consensus.cc:385] T 2aaf9a0870eb46249ccf60a2123fd678 P 29bb2c7b65094fafa8a7c5e5ecee48de [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:24:21.832772 27571 raft_consensus.cc:740] T 2aaf9a0870eb46249ccf60a2123fd678 P 29bb2c7b65094fafa8a7c5e5ecee48de [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 29bb2c7b65094fafa8a7c5e5ecee48de, State: Initialized, Role: FOLLOWER
I20260709 13:24:21.832908 27570 tablet_bootstrap.cc:654] T 2aaf9a0870eb46249ccf60a2123fd678 P 8dc4c6d0eae14641b9c5e44d31fcd335: Neither blocks nor log segments found. Creating new log.
I20260709 13:24:21.833772 27571 consensus_queue.cc:260] T 2aaf9a0870eb46249ccf60a2123fd678 P 29bb2c7b65094fafa8a7c5e5ecee48de [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dc4c6d0eae14641b9c5e44d31fcd335" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36523 } } peers { permanent_uuid: "29bb2c7b65094fafa8a7c5e5ecee48de" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 43785 } } peers { permanent_uuid: "6c8db9dae5644d6290b55e3cae161043" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 39577 } }
I20260709 13:24:21.836540 27571 ts_tablet_manager.cc:1434] T 2aaf9a0870eb46249ccf60a2123fd678 P 29bb2c7b65094fafa8a7c5e5ecee48de: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 13:24:21.836755 27572 tablet_bootstrap.cc:654] T 2aaf9a0870eb46249ccf60a2123fd678 P 6c8db9dae5644d6290b55e3cae161043: Neither blocks nor log segments found. Creating new log.
I20260709 13:24:21.853222 27570 tablet_bootstrap.cc:492] T 2aaf9a0870eb46249ccf60a2123fd678 P 8dc4c6d0eae14641b9c5e44d31fcd335: No bootstrap required, opened a new log
I20260709 13:24:21.853790 27570 ts_tablet_manager.cc:1403] T 2aaf9a0870eb46249ccf60a2123fd678 P 8dc4c6d0eae14641b9c5e44d31fcd335: Time spent bootstrapping tablet: real 0.036s	user 0.017s	sys 0.009s
I20260709 13:24:21.854135 27572 tablet_bootstrap.cc:492] T 2aaf9a0870eb46249ccf60a2123fd678 P 6c8db9dae5644d6290b55e3cae161043: No bootstrap required, opened a new log
I20260709 13:24:21.854719 27572 ts_tablet_manager.cc:1403] T 2aaf9a0870eb46249ccf60a2123fd678 P 6c8db9dae5644d6290b55e3cae161043: Time spent bootstrapping tablet: real 0.028s	user 0.021s	sys 0.002s
I20260709 13:24:21.856843 27570 raft_consensus.cc:359] T 2aaf9a0870eb46249ccf60a2123fd678 P 8dc4c6d0eae14641b9c5e44d31fcd335 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dc4c6d0eae14641b9c5e44d31fcd335" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36523 } } peers { permanent_uuid: "29bb2c7b65094fafa8a7c5e5ecee48de" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 43785 } } peers { permanent_uuid: "6c8db9dae5644d6290b55e3cae161043" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 39577 } }
I20260709 13:24:21.857673 27570 raft_consensus.cc:385] T 2aaf9a0870eb46249ccf60a2123fd678 P 8dc4c6d0eae14641b9c5e44d31fcd335 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:24:21.858011 27570 raft_consensus.cc:740] T 2aaf9a0870eb46249ccf60a2123fd678 P 8dc4c6d0eae14641b9c5e44d31fcd335 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8dc4c6d0eae14641b9c5e44d31fcd335, State: Initialized, Role: FOLLOWER
I20260709 13:24:21.857743 27572 raft_consensus.cc:359] T 2aaf9a0870eb46249ccf60a2123fd678 P 6c8db9dae5644d6290b55e3cae161043 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dc4c6d0eae14641b9c5e44d31fcd335" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36523 } } peers { permanent_uuid: "29bb2c7b65094fafa8a7c5e5ecee48de" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 43785 } } peers { permanent_uuid: "6c8db9dae5644d6290b55e3cae161043" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 39577 } }
I20260709 13:24:21.858539 27572 raft_consensus.cc:385] T 2aaf9a0870eb46249ccf60a2123fd678 P 6c8db9dae5644d6290b55e3cae161043 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:24:21.858847 27572 raft_consensus.cc:740] T 2aaf9a0870eb46249ccf60a2123fd678 P 6c8db9dae5644d6290b55e3cae161043 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c8db9dae5644d6290b55e3cae161043, State: Initialized, Role: FOLLOWER
I20260709 13:24:21.858847 27570 consensus_queue.cc:260] T 2aaf9a0870eb46249ccf60a2123fd678 P 8dc4c6d0eae14641b9c5e44d31fcd335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dc4c6d0eae14641b9c5e44d31fcd335" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36523 } } peers { permanent_uuid: "29bb2c7b65094fafa8a7c5e5ecee48de" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 43785 } } peers { permanent_uuid: "6c8db9dae5644d6290b55e3cae161043" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 39577 } }
I20260709 13:24:21.859767 27572 consensus_queue.cc:260] T 2aaf9a0870eb46249ccf60a2123fd678 P 6c8db9dae5644d6290b55e3cae161043 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dc4c6d0eae14641b9c5e44d31fcd335" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36523 } } peers { permanent_uuid: "29bb2c7b65094fafa8a7c5e5ecee48de" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 43785 } } peers { permanent_uuid: "6c8db9dae5644d6290b55e3cae161043" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 39577 } }
I20260709 13:24:21.861269 27570 ts_tablet_manager.cc:1434] T 2aaf9a0870eb46249ccf60a2123fd678 P 8dc4c6d0eae14641b9c5e44d31fcd335: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 13:24:21.861879 27572 ts_tablet_manager.cc:1434] T 2aaf9a0870eb46249ccf60a2123fd678 P 6c8db9dae5644d6290b55e3cae161043: Time spent starting tablet: real 0.007s	user 0.002s	sys 0.003s
I20260709 13:24:21.890239 27577 raft_consensus.cc:493] T 2aaf9a0870eb46249ccf60a2123fd678 P 29bb2c7b65094fafa8a7c5e5ecee48de [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:24:21.890836 27577 raft_consensus.cc:515] T 2aaf9a0870eb46249ccf60a2123fd678 P 29bb2c7b65094fafa8a7c5e5ecee48de [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dc4c6d0eae14641b9c5e44d31fcd335" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36523 } } peers { permanent_uuid: "29bb2c7b65094fafa8a7c5e5ecee48de" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 43785 } } peers { permanent_uuid: "6c8db9dae5644d6290b55e3cae161043" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 39577 } }
I20260709 13:24:21.893292 27577 leader_election.cc:290] T 2aaf9a0870eb46249ccf60a2123fd678 P 29bb2c7b65094fafa8a7c5e5ecee48de [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8dc4c6d0eae14641b9c5e44d31fcd335 (127.14.177.129:36523), 6c8db9dae5644d6290b55e3cae161043 (127.14.177.131:39577)
I20260709 13:24:21.903569 27381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2aaf9a0870eb46249ccf60a2123fd678" candidate_uuid: "29bb2c7b65094fafa8a7c5e5ecee48de" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8dc4c6d0eae14641b9c5e44d31fcd335" is_pre_election: true
I20260709 13:24:21.904403 27381 raft_consensus.cc:2468] T 2aaf9a0870eb46249ccf60a2123fd678 P 8dc4c6d0eae14641b9c5e44d31fcd335 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29bb2c7b65094fafa8a7c5e5ecee48de in term 0.
I20260709 13:24:21.905179 27530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2aaf9a0870eb46249ccf60a2123fd678" candidate_uuid: "29bb2c7b65094fafa8a7c5e5ecee48de" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c8db9dae5644d6290b55e3cae161043" is_pre_election: true
I20260709 13:24:21.905853 27530 raft_consensus.cc:2468] T 2aaf9a0870eb46249ccf60a2123fd678 P 6c8db9dae5644d6290b55e3cae161043 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29bb2c7b65094fafa8a7c5e5ecee48de in term 0.
I20260709 13:24:21.906935 27422 leader_election.cc:304] T 2aaf9a0870eb46249ccf60a2123fd678 P 29bb2c7b65094fafa8a7c5e5ecee48de [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29bb2c7b65094fafa8a7c5e5ecee48de, 6c8db9dae5644d6290b55e3cae161043; no voters: 
I20260709 13:24:21.907634 27577 raft_consensus.cc:2804] T 2aaf9a0870eb46249ccf60a2123fd678 P 29bb2c7b65094fafa8a7c5e5ecee48de [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:24:21.907950 27577 raft_consensus.cc:493] T 2aaf9a0870eb46249ccf60a2123fd678 P 29bb2c7b65094fafa8a7c5e5ecee48de [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:24:21.908283 27577 raft_consensus.cc:3060] T 2aaf9a0870eb46249ccf60a2123fd678 P 29bb2c7b65094fafa8a7c5e5ecee48de [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:24:21.915194 27577 raft_consensus.cc:515] T 2aaf9a0870eb46249ccf60a2123fd678 P 29bb2c7b65094fafa8a7c5e5ecee48de [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dc4c6d0eae14641b9c5e44d31fcd335" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36523 } } peers { permanent_uuid: "29bb2c7b65094fafa8a7c5e5ecee48de" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 43785 } } peers { permanent_uuid: "6c8db9dae5644d6290b55e3cae161043" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 39577 } }
I20260709 13:24:21.916924 27577 leader_election.cc:290] T 2aaf9a0870eb46249ccf60a2123fd678 P 29bb2c7b65094fafa8a7c5e5ecee48de [CANDIDATE]: Term 1 election: Requested vote from peers 8dc4c6d0eae14641b9c5e44d31fcd335 (127.14.177.129:36523), 6c8db9dae5644d6290b55e3cae161043 (127.14.177.131:39577)
I20260709 13:24:21.917734 27381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2aaf9a0870eb46249ccf60a2123fd678" candidate_uuid: "29bb2c7b65094fafa8a7c5e5ecee48de" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8dc4c6d0eae14641b9c5e44d31fcd335"
I20260709 13:24:21.917987 27530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2aaf9a0870eb46249ccf60a2123fd678" candidate_uuid: "29bb2c7b65094fafa8a7c5e5ecee48de" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c8db9dae5644d6290b55e3cae161043"
I20260709 13:24:21.918421 27381 raft_consensus.cc:3060] T 2aaf9a0870eb46249ccf60a2123fd678 P 8dc4c6d0eae14641b9c5e44d31fcd335 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:24:21.918572 27530 raft_consensus.cc:3060] T 2aaf9a0870eb46249ccf60a2123fd678 P 6c8db9dae5644d6290b55e3cae161043 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:24:21.927407 27530 raft_consensus.cc:2468] T 2aaf9a0870eb46249ccf60a2123fd678 P 6c8db9dae5644d6290b55e3cae161043 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29bb2c7b65094fafa8a7c5e5ecee48de in term 1.
I20260709 13:24:21.927407 27381 raft_consensus.cc:2468] T 2aaf9a0870eb46249ccf60a2123fd678 P 8dc4c6d0eae14641b9c5e44d31fcd335 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29bb2c7b65094fafa8a7c5e5ecee48de in term 1.
I20260709 13:24:21.929108 27421 leader_election.cc:304] T 2aaf9a0870eb46249ccf60a2123fd678 P 29bb2c7b65094fafa8a7c5e5ecee48de [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29bb2c7b65094fafa8a7c5e5ecee48de, 8dc4c6d0eae14641b9c5e44d31fcd335; no voters: 
I20260709 13:24:21.929934 27577 raft_consensus.cc:2804] T 2aaf9a0870eb46249ccf60a2123fd678 P 29bb2c7b65094fafa8a7c5e5ecee48de [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:24:21.930891 27577 raft_consensus.cc:697] T 2aaf9a0870eb46249ccf60a2123fd678 P 29bb2c7b65094fafa8a7c5e5ecee48de [term 1 LEADER]: Becoming Leader. State: Replica: 29bb2c7b65094fafa8a7c5e5ecee48de, State: Running, Role: LEADER
I20260709 13:24:21.931607 27577 consensus_queue.cc:237] T 2aaf9a0870eb46249ccf60a2123fd678 P 29bb2c7b65094fafa8a7c5e5ecee48de [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dc4c6d0eae14641b9c5e44d31fcd335" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36523 } } peers { permanent_uuid: "29bb2c7b65094fafa8a7c5e5ecee48de" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 43785 } } peers { permanent_uuid: "6c8db9dae5644d6290b55e3cae161043" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 39577 } }
I20260709 13:24:21.939409 27279 catalog_manager.cc:5697] T 2aaf9a0870eb46249ccf60a2123fd678 P 29bb2c7b65094fafa8a7c5e5ecee48de reported cstate change: term changed from 0 to 1, leader changed from <none> to 29bb2c7b65094fafa8a7c5e5ecee48de (127.14.177.130). New cstate: current_term: 1 leader_uuid: "29bb2c7b65094fafa8a7c5e5ecee48de" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dc4c6d0eae14641b9c5e44d31fcd335" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36523 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "29bb2c7b65094fafa8a7c5e5ecee48de" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 43785 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6c8db9dae5644d6290b55e3cae161043" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 39577 } health_report { overall_health: UNKNOWN } } }
I20260709 13:24:22.105463 27578 raft_consensus.cc:493] T 341e749fb378438e8fa98ebcb9066d34 P 6c8db9dae5644d6290b55e3cae161043 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:24:22.106093 27578 raft_consensus.cc:515] T 341e749fb378438e8fa98ebcb9066d34 P 6c8db9dae5644d6290b55e3cae161043 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c8db9dae5644d6290b55e3cae161043" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 39577 } } peers { permanent_uuid: "29bb2c7b65094fafa8a7c5e5ecee48de" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 43785 } } peers { permanent_uuid: "8dc4c6d0eae14641b9c5e44d31fcd335" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36523 } }
I20260709 13:24:22.108394 27578 leader_election.cc:290] T 341e749fb378438e8fa98ebcb9066d34 P 6c8db9dae5644d6290b55e3cae161043 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 29bb2c7b65094fafa8a7c5e5ecee48de (127.14.177.130:43785), 8dc4c6d0eae14641b9c5e44d31fcd335 (127.14.177.129:36523)
I20260709 13:24:22.116958 27456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "341e749fb378438e8fa98ebcb9066d34" candidate_uuid: "6c8db9dae5644d6290b55e3cae161043" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "29bb2c7b65094fafa8a7c5e5ecee48de" is_pre_election: true
I20260709 13:24:22.117807 27456 raft_consensus.cc:2468] T 341e749fb378438e8fa98ebcb9066d34 P 29bb2c7b65094fafa8a7c5e5ecee48de [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c8db9dae5644d6290b55e3cae161043 in term 0.
I20260709 13:24:22.118803 27381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "341e749fb378438e8fa98ebcb9066d34" candidate_uuid: "6c8db9dae5644d6290b55e3cae161043" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8dc4c6d0eae14641b9c5e44d31fcd335" is_pre_election: true
I20260709 13:24:22.119086 27494 leader_election.cc:304] T 341e749fb378438e8fa98ebcb9066d34 P 6c8db9dae5644d6290b55e3cae161043 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29bb2c7b65094fafa8a7c5e5ecee48de, 6c8db9dae5644d6290b55e3cae161043; no voters: 
I20260709 13:24:22.119540 27381 raft_consensus.cc:2468] T 341e749fb378438e8fa98ebcb9066d34 P 8dc4c6d0eae14641b9c5e44d31fcd335 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c8db9dae5644d6290b55e3cae161043 in term 0.
I20260709 13:24:22.119993 27578 raft_consensus.cc:2804] T 341e749fb378438e8fa98ebcb9066d34 P 6c8db9dae5644d6290b55e3cae161043 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:24:22.120337 27578 raft_consensus.cc:493] T 341e749fb378438e8fa98ebcb9066d34 P 6c8db9dae5644d6290b55e3cae161043 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:24:22.120697 27578 raft_consensus.cc:3060] T 341e749fb378438e8fa98ebcb9066d34 P 6c8db9dae5644d6290b55e3cae161043 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:24:22.126338 27578 raft_consensus.cc:515] T 341e749fb378438e8fa98ebcb9066d34 P 6c8db9dae5644d6290b55e3cae161043 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c8db9dae5644d6290b55e3cae161043" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 39577 } } peers { permanent_uuid: "29bb2c7b65094fafa8a7c5e5ecee48de" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 43785 } } peers { permanent_uuid: "8dc4c6d0eae14641b9c5e44d31fcd335" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36523 } }
I20260709 13:24:22.128301 27578 leader_election.cc:290] T 341e749fb378438e8fa98ebcb9066d34 P 6c8db9dae5644d6290b55e3cae161043 [CANDIDATE]: Term 1 election: Requested vote from peers 29bb2c7b65094fafa8a7c5e5ecee48de (127.14.177.130:43785), 8dc4c6d0eae14641b9c5e44d31fcd335 (127.14.177.129:36523)
I20260709 13:24:22.128953 27456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "341e749fb378438e8fa98ebcb9066d34" candidate_uuid: "6c8db9dae5644d6290b55e3cae161043" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "29bb2c7b65094fafa8a7c5e5ecee48de"
I20260709 13:24:22.129302 27381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "341e749fb378438e8fa98ebcb9066d34" candidate_uuid: "6c8db9dae5644d6290b55e3cae161043" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8dc4c6d0eae14641b9c5e44d31fcd335"
I20260709 13:24:22.129664 27456 raft_consensus.cc:3060] T 341e749fb378438e8fa98ebcb9066d34 P 29bb2c7b65094fafa8a7c5e5ecee48de [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:24:22.129937 27381 raft_consensus.cc:3060] T 341e749fb378438e8fa98ebcb9066d34 P 8dc4c6d0eae14641b9c5e44d31fcd335 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:24:22.135252 27381 raft_consensus.cc:2468] T 341e749fb378438e8fa98ebcb9066d34 P 8dc4c6d0eae14641b9c5e44d31fcd335 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c8db9dae5644d6290b55e3cae161043 in term 1.
I20260709 13:24:22.135838 27456 raft_consensus.cc:2468] T 341e749fb378438e8fa98ebcb9066d34 P 29bb2c7b65094fafa8a7c5e5ecee48de [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c8db9dae5644d6290b55e3cae161043 in term 1.
I20260709 13:24:22.136509 27495 leader_election.cc:304] T 341e749fb378438e8fa98ebcb9066d34 P 6c8db9dae5644d6290b55e3cae161043 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c8db9dae5644d6290b55e3cae161043, 8dc4c6d0eae14641b9c5e44d31fcd335; no voters: 
I20260709 13:24:22.137370 27578 raft_consensus.cc:2804] T 341e749fb378438e8fa98ebcb9066d34 P 6c8db9dae5644d6290b55e3cae161043 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:24:22.138265 27578 raft_consensus.cc:697] T 341e749fb378438e8fa98ebcb9066d34 P 6c8db9dae5644d6290b55e3cae161043 [term 1 LEADER]: Becoming Leader. State: Replica: 6c8db9dae5644d6290b55e3cae161043, State: Running, Role: LEADER
I20260709 13:24:22.139117 27578 consensus_queue.cc:237] T 341e749fb378438e8fa98ebcb9066d34 P 6c8db9dae5644d6290b55e3cae161043 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c8db9dae5644d6290b55e3cae161043" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 39577 } } peers { permanent_uuid: "29bb2c7b65094fafa8a7c5e5ecee48de" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 43785 } } peers { permanent_uuid: "8dc4c6d0eae14641b9c5e44d31fcd335" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36523 } }
I20260709 13:24:22.146265 27279 catalog_manager.cc:5697] T 341e749fb378438e8fa98ebcb9066d34 P 6c8db9dae5644d6290b55e3cae161043 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6c8db9dae5644d6290b55e3cae161043 (127.14.177.131). New cstate: current_term: 1 leader_uuid: "6c8db9dae5644d6290b55e3cae161043" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c8db9dae5644d6290b55e3cae161043" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 39577 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "29bb2c7b65094fafa8a7c5e5ecee48de" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 43785 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8dc4c6d0eae14641b9c5e44d31fcd335" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36523 } health_report { overall_health: UNKNOWN } } }
I20260709 13:24:22.246039 27279 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34128:
name: "multi_table_leader_balance_table2"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\377\377\377?""\004\001\000\377\377\377?"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 13:24:22.248163 27279 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table multi_table_leader_balance_table2 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 13:24:22.275395 27370 tablet_service.cc:1511] Processing CreateTablet for tablet dec6410d8b0546bab2e4f3d352b54681 (DEFAULT_TABLE table=multi_table_leader_balance_table2 [id=a61fa39e0a6f4d9899611d854d6b16bd]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20260709 13:24:22.276887 27370 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dec6410d8b0546bab2e4f3d352b54681. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:24:22.276736 27446 tablet_service.cc:1511] Processing CreateTablet for tablet dec6410d8b0546bab2e4f3d352b54681 (DEFAULT_TABLE table=multi_table_leader_balance_table2 [id=a61fa39e0a6f4d9899611d854d6b16bd]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20260709 13:24:22.278368 27446 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dec6410d8b0546bab2e4f3d352b54681. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:24:22.278769 27520 tablet_service.cc:1511] Processing CreateTablet for tablet dec6410d8b0546bab2e4f3d352b54681 (DEFAULT_TABLE table=multi_table_leader_balance_table2 [id=a61fa39e0a6f4d9899611d854d6b16bd]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20260709 13:24:22.279851 27371 tablet_service.cc:1511] Processing CreateTablet for tablet 4f9d369fbd364a7c958c7698fa8dd33f (DEFAULT_TABLE table=multi_table_leader_balance_table2 [id=a61fa39e0a6f4d9899611d854d6b16bd]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20260709 13:24:22.280258 27520 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dec6410d8b0546bab2e4f3d352b54681. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:24:22.281275 27371 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4f9d369fbd364a7c958c7698fa8dd33f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:24:22.294200 27445 tablet_service.cc:1511] Processing CreateTablet for tablet 4f9d369fbd364a7c958c7698fa8dd33f (DEFAULT_TABLE table=multi_table_leader_balance_table2 [id=a61fa39e0a6f4d9899611d854d6b16bd]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20260709 13:24:22.295799 27445 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4f9d369fbd364a7c958c7698fa8dd33f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:24:22.295544 27519 tablet_service.cc:1511] Processing CreateTablet for tablet 4f9d369fbd364a7c958c7698fa8dd33f (DEFAULT_TABLE table=multi_table_leader_balance_table2 [id=a61fa39e0a6f4d9899611d854d6b16bd]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20260709 13:24:22.297048 27519 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4f9d369fbd364a7c958c7698fa8dd33f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:24:22.302846 27572 tablet_bootstrap.cc:492] T dec6410d8b0546bab2e4f3d352b54681 P 6c8db9dae5644d6290b55e3cae161043: Bootstrap starting.
I20260709 13:24:22.304466 27570 tablet_bootstrap.cc:492] T dec6410d8b0546bab2e4f3d352b54681 P 8dc4c6d0eae14641b9c5e44d31fcd335: Bootstrap starting.
I20260709 13:24:22.311180 27572 tablet_bootstrap.cc:654] T dec6410d8b0546bab2e4f3d352b54681 P 6c8db9dae5644d6290b55e3cae161043: Neither blocks nor log segments found. Creating new log.
I20260709 13:24:22.314488 27571 tablet_bootstrap.cc:492] T dec6410d8b0546bab2e4f3d352b54681 P 29bb2c7b65094fafa8a7c5e5ecee48de: Bootstrap starting.
I20260709 13:24:22.319517 27570 tablet_bootstrap.cc:654] T dec6410d8b0546bab2e4f3d352b54681 P 8dc4c6d0eae14641b9c5e44d31fcd335: Neither blocks nor log segments found. Creating new log.
I20260709 13:24:22.319296 27577 consensus_queue.cc:1048] T 2aaf9a0870eb46249ccf60a2123fd678 P 29bb2c7b65094fafa8a7c5e5ecee48de [LEADER]: Connected to new peer: Peer: permanent_uuid: "8dc4c6d0eae14641b9c5e44d31fcd335" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36523 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:24:22.322283 27571 tablet_bootstrap.cc:654] T dec6410d8b0546bab2e4f3d352b54681 P 29bb2c7b65094fafa8a7c5e5ecee48de: Neither blocks nor log segments found. Creating new log.
I20260709 13:24:22.330979 27577 consensus_queue.cc:1048] T 2aaf9a0870eb46249ccf60a2123fd678 P 29bb2c7b65094fafa8a7c5e5ecee48de [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c8db9dae5644d6290b55e3cae161043" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 39577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:24:22.342552 27572 tablet_bootstrap.cc:492] T dec6410d8b0546bab2e4f3d352b54681 P 6c8db9dae5644d6290b55e3cae161043: No bootstrap required, opened a new log
I20260709 13:24:22.350916 27572 ts_tablet_manager.cc:1403] T dec6410d8b0546bab2e4f3d352b54681 P 6c8db9dae5644d6290b55e3cae161043: Time spent bootstrapping tablet: real 0.048s	user 0.014s	sys 0.000s
I20260709 13:24:22.356031 27571 tablet_bootstrap.cc:492] T dec6410d8b0546bab2e4f3d352b54681 P 29bb2c7b65094fafa8a7c5e5ecee48de: No bootstrap required, opened a new log
I20260709 13:24:22.356719 27571 ts_tablet_manager.cc:1403] T dec6410d8b0546bab2e4f3d352b54681 P 29bb2c7b65094fafa8a7c5e5ecee48de: Time spent bootstrapping tablet: real 0.043s	user 0.019s	sys 0.007s
I20260709 13:24:22.360759 27572 raft_consensus.cc:359] T dec6410d8b0546bab2e4f3d352b54681 P 6c8db9dae5644d6290b55e3cae161043 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dc4c6d0eae14641b9c5e44d31fcd335" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36523 } } peers { permanent_uuid: "29bb2c7b65094fafa8a7c5e5ecee48de" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 43785 } } peers { permanent_uuid: "6c8db9dae5644d6290b55e3cae161043" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 39577 } }
I20260709 13:24:22.362064 27572 raft_consensus.cc:385] T dec6410d8b0546bab2e4f3d352b54681 P 6c8db9dae5644d6290b55e3cae161043 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:24:22.366043 27572 raft_consensus.cc:740] T dec6410d8b0546bab2e4f3d352b54681 P 6c8db9dae5644d6290b55e3cae161043 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c8db9dae5644d6290b55e3cae161043, State: Initialized, Role: FOLLOWER
I20260709 13:24:22.367381 27572 consensus_queue.cc:260] T dec6410d8b0546bab2e4f3d352b54681 P 6c8db9dae5644d6290b55e3cae161043 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dc4c6d0eae14641b9c5e44d31fcd335" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36523 } } peers { permanent_uuid: "29bb2c7b65094fafa8a7c5e5ecee48de" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 43785 } } peers { permanent_uuid: "6c8db9dae5644d6290b55e3cae161043" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 39577 } }
I20260709 13:24:22.363803 27571 raft_consensus.cc:359] T dec6410d8b0546bab2e4f3d352b54681 P 29bb2c7b65094fafa8a7c5e5ecee48de [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dc4c6d0eae14641b9c5e44d31fcd335" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36523 } } peers { permanent_uuid: "29bb2c7b65094fafa8a7c5e5ecee48de" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 43785 } } peers { permanent_uuid: "6c8db9dae5644d6290b55e3cae161043" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 39577 } }
I20260709 13:24:22.368705 27571 raft_consensus.cc:385] T dec6410d8b0546bab2e4f3d352b54681 P 29bb2c7b65094fafa8a7c5e5ecee48de [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:24:22.373659 27571 raft_consensus.cc:740] T dec6410d8b0546bab2e4f3d352b54681 P 29bb2c7b65094fafa8a7c5e5ecee48de [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 29bb2c7b65094fafa8a7c5e5ecee48de, State: Initialized, Role: FOLLOWER
I20260709 13:24:22.374692 27571 consensus_queue.cc:260] T dec6410d8b0546bab2e4f3d352b54681 P 29bb2c7b65094fafa8a7c5e5ecee48de [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dc4c6d0eae14641b9c5e44d31fcd335" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36523 } } peers { permanent_uuid: "29bb2c7b65094fafa8a7c5e5ecee48de" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 43785 } } peers { permanent_uuid: "6c8db9dae5644d6290b55e3cae161043" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 39577 } }
I20260709 13:24:22.373203 27572 ts_tablet_manager.cc:1434] T dec6410d8b0546bab2e4f3d352b54681 P 6c8db9dae5644d6290b55e3cae161043: Time spent starting tablet: real 0.022s	user 0.004s	sys 0.003s
I20260709 13:24:22.377797 27572 tablet_bootstrap.cc:492] T 4f9d369fbd364a7c958c7698fa8dd33f P 6c8db9dae5644d6290b55e3cae161043: Bootstrap starting.
I20260709 13:24:22.381677 27570 tablet_bootstrap.cc:492] T dec6410d8b0546bab2e4f3d352b54681 P 8dc4c6d0eae14641b9c5e44d31fcd335: No bootstrap required, opened a new log
I20260709 13:24:22.382682 27570 ts_tablet_manager.cc:1403] T dec6410d8b0546bab2e4f3d352b54681 P 8dc4c6d0eae14641b9c5e44d31fcd335: Time spent bootstrapping tablet: real 0.078s	user 0.024s	sys 0.007s
I20260709 13:24:22.388027 27570 raft_consensus.cc:359] T dec6410d8b0546bab2e4f3d352b54681 P 8dc4c6d0eae14641b9c5e44d31fcd335 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dc4c6d0eae14641b9c5e44d31fcd335" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36523 } } peers { permanent_uuid: "29bb2c7b65094fafa8a7c5e5ecee48de" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 43785 } } peers { permanent_uuid: "6c8db9dae5644d6290b55e3cae161043" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 39577 } }
I20260709 13:24:22.388968 27570 raft_consensus.cc:385] T dec6410d8b0546bab2e4f3d352b54681 P 8dc4c6d0eae14641b9c5e44d31fcd335 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:24:22.389375 27570 raft_consensus.cc:740] T dec6410d8b0546bab2e4f3d352b54681 P 8dc4c6d0eae14641b9c5e44d31fcd335 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8dc4c6d0eae14641b9c5e44d31fcd335, State: Initialized, Role: FOLLOWER
I20260709 13:24:22.390478 27570 consensus_queue.cc:260] T dec6410d8b0546bab2e4f3d352b54681 P 8dc4c6d0eae14641b9c5e44d31fcd335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dc4c6d0eae14641b9c5e44d31fcd335" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36523 } } peers { permanent_uuid: "29bb2c7b65094fafa8a7c5e5ecee48de" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 43785 } } peers { permanent_uuid: "6c8db9dae5644d6290b55e3cae161043" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 39577 } }
I20260709 13:24:22.401305 27571 ts_tablet_manager.cc:1434] T dec6410d8b0546bab2e4f3d352b54681 P 29bb2c7b65094fafa8a7c5e5ecee48de: Time spent starting tablet: real 0.044s	user 0.007s	sys 0.006s
I20260709 13:24:22.402887 27571 tablet_bootstrap.cc:492] T 4f9d369fbd364a7c958c7698fa8dd33f P 29bb2c7b65094fafa8a7c5e5ecee48de: Bootstrap starting.
I20260709 13:24:22.404183 27572 tablet_bootstrap.cc:654] T 4f9d369fbd364a7c958c7698fa8dd33f P 6c8db9dae5644d6290b55e3cae161043: Neither blocks nor log segments found. Creating new log.
I20260709 13:24:22.408404 27570 ts_tablet_manager.cc:1434] T dec6410d8b0546bab2e4f3d352b54681 P 8dc4c6d0eae14641b9c5e44d31fcd335: Time spent starting tablet: real 0.024s	user 0.007s	sys 0.000s
I20260709 13:24:22.410562 27570 tablet_bootstrap.cc:492] T 4f9d369fbd364a7c958c7698fa8dd33f P 8dc4c6d0eae14641b9c5e44d31fcd335: Bootstrap starting.
I20260709 13:24:22.412787 27571 tablet_bootstrap.cc:654] T 4f9d369fbd364a7c958c7698fa8dd33f P 29bb2c7b65094fafa8a7c5e5ecee48de: Neither blocks nor log segments found. Creating new log.
I20260709 13:24:22.416179 27572 tablet_bootstrap.cc:492] T 4f9d369fbd364a7c958c7698fa8dd33f P 6c8db9dae5644d6290b55e3cae161043: No bootstrap required, opened a new log
I20260709 13:24:22.416714 27572 ts_tablet_manager.cc:1403] T 4f9d369fbd364a7c958c7698fa8dd33f P 6c8db9dae5644d6290b55e3cae161043: Time spent bootstrapping tablet: real 0.039s	user 0.018s	sys 0.000s
I20260709 13:24:22.419927 27570 tablet_bootstrap.cc:654] T 4f9d369fbd364a7c958c7698fa8dd33f P 8dc4c6d0eae14641b9c5e44d31fcd335: Neither blocks nor log segments found. Creating new log.
I20260709 13:24:22.419994 27572 raft_consensus.cc:359] T 4f9d369fbd364a7c958c7698fa8dd33f P 6c8db9dae5644d6290b55e3cae161043 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dc4c6d0eae14641b9c5e44d31fcd335" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36523 } } peers { permanent_uuid: "29bb2c7b65094fafa8a7c5e5ecee48de" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 43785 } } peers { permanent_uuid: "6c8db9dae5644d6290b55e3cae161043" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 39577 } }
I20260709 13:24:22.420847 27572 raft_consensus.cc:385] T 4f9d369fbd364a7c958c7698fa8dd33f P 6c8db9dae5644d6290b55e3cae161043 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:24:22.421161 27572 raft_consensus.cc:740] T 4f9d369fbd364a7c958c7698fa8dd33f P 6c8db9dae5644d6290b55e3cae161043 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c8db9dae5644d6290b55e3cae161043, State: Initialized, Role: FOLLOWER
I20260709 13:24:22.421913 27572 consensus_queue.cc:260] T 4f9d369fbd364a7c958c7698fa8dd33f P 6c8db9dae5644d6290b55e3cae161043 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dc4c6d0eae14641b9c5e44d31fcd335" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36523 } } peers { permanent_uuid: "29bb2c7b65094fafa8a7c5e5ecee48de" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 43785 } } peers { permanent_uuid: "6c8db9dae5644d6290b55e3cae161043" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 39577 } }
I20260709 13:24:22.422322 27571 tablet_bootstrap.cc:492] T 4f9d369fbd364a7c958c7698fa8dd33f P 29bb2c7b65094fafa8a7c5e5ecee48de: No bootstrap required, opened a new log
I20260709 13:24:22.422982 27571 ts_tablet_manager.cc:1403] T 4f9d369fbd364a7c958c7698fa8dd33f P 29bb2c7b65094fafa8a7c5e5ecee48de: Time spent bootstrapping tablet: real 0.020s	user 0.007s	sys 0.009s
I20260709 13:24:22.424189 27572 ts_tablet_manager.cc:1434] T 4f9d369fbd364a7c958c7698fa8dd33f P 6c8db9dae5644d6290b55e3cae161043: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 13:24:22.426992 27571 raft_consensus.cc:359] T 4f9d369fbd364a7c958c7698fa8dd33f P 29bb2c7b65094fafa8a7c5e5ecee48de [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dc4c6d0eae14641b9c5e44d31fcd335" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36523 } } peers { permanent_uuid: "29bb2c7b65094fafa8a7c5e5ecee48de" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 43785 } } peers { permanent_uuid: "6c8db9dae5644d6290b55e3cae161043" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 39577 } }
I20260709 13:24:22.427937 27571 raft_consensus.cc:385] T 4f9d369fbd364a7c958c7698fa8dd33f P 29bb2c7b65094fafa8a7c5e5ecee48de [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:24:22.428115 27570 tablet_bootstrap.cc:492] T 4f9d369fbd364a7c958c7698fa8dd33f P 8dc4c6d0eae14641b9c5e44d31fcd335: No bootstrap required, opened a new log
I20260709 13:24:22.428288 27571 raft_consensus.cc:740] T 4f9d369fbd364a7c958c7698fa8dd33f P 29bb2c7b65094fafa8a7c5e5ecee48de [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 29bb2c7b65094fafa8a7c5e5ecee48de, State: Initialized, Role: FOLLOWER
I20260709 13:24:22.428867 27570 ts_tablet_manager.cc:1403] T 4f9d369fbd364a7c958c7698fa8dd33f P 8dc4c6d0eae14641b9c5e44d31fcd335: Time spent bootstrapping tablet: real 0.019s	user 0.007s	sys 0.010s
I20260709 13:24:22.429437 27571 consensus_queue.cc:260] T 4f9d369fbd364a7c958c7698fa8dd33f P 29bb2c7b65094fafa8a7c5e5ecee48de [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dc4c6d0eae14641b9c5e44d31fcd335" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36523 } } peers { permanent_uuid: "29bb2c7b65094fafa8a7c5e5ecee48de" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 43785 } } peers { permanent_uuid: "6c8db9dae5644d6290b55e3cae161043" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 39577 } }
I20260709 13:24:22.432248 27571 ts_tablet_manager.cc:1434] T 4f9d369fbd364a7c958c7698fa8dd33f P 29bb2c7b65094fafa8a7c5e5ecee48de: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.001s
I20260709 13:24:22.432523 27570 raft_consensus.cc:359] T 4f9d369fbd364a7c958c7698fa8dd33f P 8dc4c6d0eae14641b9c5e44d31fcd335 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dc4c6d0eae14641b9c5e44d31fcd335" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36523 } } peers { permanent_uuid: "29bb2c7b65094fafa8a7c5e5ecee48de" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 43785 } } peers { permanent_uuid: "6c8db9dae5644d6290b55e3cae161043" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 39577 } }
I20260709 13:24:22.433455 27570 raft_consensus.cc:385] T 4f9d369fbd364a7c958c7698fa8dd33f P 8dc4c6d0eae14641b9c5e44d31fcd335 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:24:22.433820 27570 raft_consensus.cc:740] T 4f9d369fbd364a7c958c7698fa8dd33f P 8dc4c6d0eae14641b9c5e44d31fcd335 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8dc4c6d0eae14641b9c5e44d31fcd335, State: Initialized, Role: FOLLOWER
I20260709 13:24:22.434664 27570 consensus_queue.cc:260] T 4f9d369fbd364a7c958c7698fa8dd33f P 8dc4c6d0eae14641b9c5e44d31fcd335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dc4c6d0eae14641b9c5e44d31fcd335" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36523 } } peers { permanent_uuid: "29bb2c7b65094fafa8a7c5e5ecee48de" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 43785 } } peers { permanent_uuid: "6c8db9dae5644d6290b55e3cae161043" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 39577 } }
I20260709 13:24:22.436800 27570 ts_tablet_manager.cc:1434] T 4f9d369fbd364a7c958c7698fa8dd33f P 8dc4c6d0eae14641b9c5e44d31fcd335: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.001s
I20260709 13:24:22.440116 27578 raft_consensus.cc:493] T 4f9d369fbd364a7c958c7698fa8dd33f P 6c8db9dae5644d6290b55e3cae161043 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:24:22.440719 27578 raft_consensus.cc:515] T 4f9d369fbd364a7c958c7698fa8dd33f P 6c8db9dae5644d6290b55e3cae161043 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dc4c6d0eae14641b9c5e44d31fcd335" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36523 } } peers { permanent_uuid: "29bb2c7b65094fafa8a7c5e5ecee48de" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 43785 } } peers { permanent_uuid: "6c8db9dae5644d6290b55e3cae161043" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 39577 } }
I20260709 13:24:22.444814 27456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f9d369fbd364a7c958c7698fa8dd33f" candidate_uuid: "6c8db9dae5644d6290b55e3cae161043" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "29bb2c7b65094fafa8a7c5e5ecee48de" is_pre_election: true
I20260709 13:24:22.444829 27578 leader_election.cc:290] T 4f9d369fbd364a7c958c7698fa8dd33f P 6c8db9dae5644d6290b55e3cae161043 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8dc4c6d0eae14641b9c5e44d31fcd335 (127.14.177.129:36523), 29bb2c7b65094fafa8a7c5e5ecee48de (127.14.177.130:43785)
I20260709 13:24:22.445557 27456 raft_consensus.cc:2468] T 4f9d369fbd364a7c958c7698fa8dd33f P 29bb2c7b65094fafa8a7c5e5ecee48de [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c8db9dae5644d6290b55e3cae161043 in term 0.
I20260709 13:24:22.446635 27381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f9d369fbd364a7c958c7698fa8dd33f" candidate_uuid: "6c8db9dae5644d6290b55e3cae161043" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8dc4c6d0eae14641b9c5e44d31fcd335" is_pre_election: true
I20260709 13:24:22.446717 27494 leader_election.cc:304] T 4f9d369fbd364a7c958c7698fa8dd33f P 6c8db9dae5644d6290b55e3cae161043 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29bb2c7b65094fafa8a7c5e5ecee48de, 6c8db9dae5644d6290b55e3cae161043; no voters: 
I20260709 13:24:22.447481 27381 raft_consensus.cc:2468] T 4f9d369fbd364a7c958c7698fa8dd33f P 8dc4c6d0eae14641b9c5e44d31fcd335 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c8db9dae5644d6290b55e3cae161043 in term 0.
I20260709 13:24:22.447769 27578 raft_consensus.cc:2804] T 4f9d369fbd364a7c958c7698fa8dd33f P 6c8db9dae5644d6290b55e3cae161043 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:24:22.448235 27578 raft_consensus.cc:493] T 4f9d369fbd364a7c958c7698fa8dd33f P 6c8db9dae5644d6290b55e3cae161043 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:24:22.448565 27578 raft_consensus.cc:3060] T 4f9d369fbd364a7c958c7698fa8dd33f P 6c8db9dae5644d6290b55e3cae161043 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:24:22.454001 27578 raft_consensus.cc:515] T 4f9d369fbd364a7c958c7698fa8dd33f P 6c8db9dae5644d6290b55e3cae161043 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dc4c6d0eae14641b9c5e44d31fcd335" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36523 } } peers { permanent_uuid: "29bb2c7b65094fafa8a7c5e5ecee48de" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 43785 } } peers { permanent_uuid: "6c8db9dae5644d6290b55e3cae161043" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 39577 } }
I20260709 13:24:22.455868 27578 leader_election.cc:290] T 4f9d369fbd364a7c958c7698fa8dd33f P 6c8db9dae5644d6290b55e3cae161043 [CANDIDATE]: Term 1 election: Requested vote from peers 8dc4c6d0eae14641b9c5e44d31fcd335 (127.14.177.129:36523), 29bb2c7b65094fafa8a7c5e5ecee48de (127.14.177.130:43785)
I20260709 13:24:22.456732 27381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f9d369fbd364a7c958c7698fa8dd33f" candidate_uuid: "6c8db9dae5644d6290b55e3cae161043" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8dc4c6d0eae14641b9c5e44d31fcd335"
I20260709 13:24:22.456938 27456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f9d369fbd364a7c958c7698fa8dd33f" candidate_uuid: "6c8db9dae5644d6290b55e3cae161043" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "29bb2c7b65094fafa8a7c5e5ecee48de"
I20260709 13:24:22.457315 27381 raft_consensus.cc:3060] T 4f9d369fbd364a7c958c7698fa8dd33f P 8dc4c6d0eae14641b9c5e44d31fcd335 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:24:22.457670 27456 raft_consensus.cc:3060] T 4f9d369fbd364a7c958c7698fa8dd33f P 29bb2c7b65094fafa8a7c5e5ecee48de [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:24:22.462824 27381 raft_consensus.cc:2468] T 4f9d369fbd364a7c958c7698fa8dd33f P 8dc4c6d0eae14641b9c5e44d31fcd335 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c8db9dae5644d6290b55e3cae161043 in term 1.
I20260709 13:24:22.463694 27456 raft_consensus.cc:2468] T 4f9d369fbd364a7c958c7698fa8dd33f P 29bb2c7b65094fafa8a7c5e5ecee48de [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c8db9dae5644d6290b55e3cae161043 in term 1.
I20260709 13:24:22.464169 27495 leader_election.cc:304] T 4f9d369fbd364a7c958c7698fa8dd33f P 6c8db9dae5644d6290b55e3cae161043 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c8db9dae5644d6290b55e3cae161043, 8dc4c6d0eae14641b9c5e44d31fcd335; no voters: 
I20260709 13:24:22.464323 27603 raft_consensus.cc:493] T dec6410d8b0546bab2e4f3d352b54681 P 29bb2c7b65094fafa8a7c5e5ecee48de [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:24:22.465124 27578 raft_consensus.cc:2804] T 4f9d369fbd364a7c958c7698fa8dd33f P 6c8db9dae5644d6290b55e3cae161043 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:24:22.464922 27603 raft_consensus.cc:515] T dec6410d8b0546bab2e4f3d352b54681 P 29bb2c7b65094fafa8a7c5e5ecee48de [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dc4c6d0eae14641b9c5e44d31fcd335" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36523 } } peers { permanent_uuid: "29bb2c7b65094fafa8a7c5e5ecee48de" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 43785 } } peers { permanent_uuid: "6c8db9dae5644d6290b55e3cae161043" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 39577 } }
I20260709 13:24:22.465917 27578 raft_consensus.cc:697] T 4f9d369fbd364a7c958c7698fa8dd33f P 6c8db9dae5644d6290b55e3cae161043 [term 1 LEADER]: Becoming Leader. State: Replica: 6c8db9dae5644d6290b55e3cae161043, State: Running, Role: LEADER
I20260709 13:24:22.466794 27578 consensus_queue.cc:237] T 4f9d369fbd364a7c958c7698fa8dd33f P 6c8db9dae5644d6290b55e3cae161043 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dc4c6d0eae14641b9c5e44d31fcd335" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36523 } } peers { permanent_uuid: "29bb2c7b65094fafa8a7c5e5ecee48de" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 43785 } } peers { permanent_uuid: "6c8db9dae5644d6290b55e3cae161043" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 39577 } }
I20260709 13:24:22.467420 27603 leader_election.cc:290] T dec6410d8b0546bab2e4f3d352b54681 P 29bb2c7b65094fafa8a7c5e5ecee48de [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8dc4c6d0eae14641b9c5e44d31fcd335 (127.14.177.129:36523), 6c8db9dae5644d6290b55e3cae161043 (127.14.177.131:39577)
I20260709 13:24:22.468308 27381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dec6410d8b0546bab2e4f3d352b54681" candidate_uuid: "29bb2c7b65094fafa8a7c5e5ecee48de" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8dc4c6d0eae14641b9c5e44d31fcd335" is_pre_election: true
I20260709 13:24:22.468659 27530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dec6410d8b0546bab2e4f3d352b54681" candidate_uuid: "29bb2c7b65094fafa8a7c5e5ecee48de" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c8db9dae5644d6290b55e3cae161043" is_pre_election: true
I20260709 13:24:22.469014 27381 raft_consensus.cc:2468] T dec6410d8b0546bab2e4f3d352b54681 P 8dc4c6d0eae14641b9c5e44d31fcd335 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29bb2c7b65094fafa8a7c5e5ecee48de in term 0.
I20260709 13:24:22.469347 27530 raft_consensus.cc:2468] T dec6410d8b0546bab2e4f3d352b54681 P 6c8db9dae5644d6290b55e3cae161043 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29bb2c7b65094fafa8a7c5e5ecee48de in term 0.
I20260709 13:24:22.470147 27421 leader_election.cc:304] T dec6410d8b0546bab2e4f3d352b54681 P 29bb2c7b65094fafa8a7c5e5ecee48de [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29bb2c7b65094fafa8a7c5e5ecee48de, 8dc4c6d0eae14641b9c5e44d31fcd335; no voters: 
I20260709 13:24:22.471287 27603 raft_consensus.cc:2804] T dec6410d8b0546bab2e4f3d352b54681 P 29bb2c7b65094fafa8a7c5e5ecee48de [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:24:22.471721 27603 raft_consensus.cc:493] T dec6410d8b0546bab2e4f3d352b54681 P 29bb2c7b65094fafa8a7c5e5ecee48de [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:24:22.472033 27603 raft_consensus.cc:3060] T dec6410d8b0546bab2e4f3d352b54681 P 29bb2c7b65094fafa8a7c5e5ecee48de [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:24:22.476457 27278 catalog_manager.cc:5697] T 4f9d369fbd364a7c958c7698fa8dd33f P 6c8db9dae5644d6290b55e3cae161043 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6c8db9dae5644d6290b55e3cae161043 (127.14.177.131). New cstate: current_term: 1 leader_uuid: "6c8db9dae5644d6290b55e3cae161043" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dc4c6d0eae14641b9c5e44d31fcd335" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36523 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "29bb2c7b65094fafa8a7c5e5ecee48de" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 43785 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c8db9dae5644d6290b55e3cae161043" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 39577 } health_report { overall_health: HEALTHY } } }
I20260709 13:24:22.480494 27603 raft_consensus.cc:515] T dec6410d8b0546bab2e4f3d352b54681 P 29bb2c7b65094fafa8a7c5e5ecee48de [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dc4c6d0eae14641b9c5e44d31fcd335" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36523 } } peers { permanent_uuid: "29bb2c7b65094fafa8a7c5e5ecee48de" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 43785 } } peers { permanent_uuid: "6c8db9dae5644d6290b55e3cae161043" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 39577 } }
I20260709 13:24:22.482662 27603 leader_election.cc:290] T dec6410d8b0546bab2e4f3d352b54681 P 29bb2c7b65094fafa8a7c5e5ecee48de [CANDIDATE]: Term 1 election: Requested vote from peers 8dc4c6d0eae14641b9c5e44d31fcd335 (127.14.177.129:36523), 6c8db9dae5644d6290b55e3cae161043 (127.14.177.131:39577)
I20260709 13:24:22.483753 27381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dec6410d8b0546bab2e4f3d352b54681" candidate_uuid: "29bb2c7b65094fafa8a7c5e5ecee48de" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8dc4c6d0eae14641b9c5e44d31fcd335"
I20260709 13:24:22.483832 27530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dec6410d8b0546bab2e4f3d352b54681" candidate_uuid: "29bb2c7b65094fafa8a7c5e5ecee48de" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c8db9dae5644d6290b55e3cae161043"
I20260709 13:24:22.484604 27381 raft_consensus.cc:3060] T dec6410d8b0546bab2e4f3d352b54681 P 8dc4c6d0eae14641b9c5e44d31fcd335 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:24:22.484627 27530 raft_consensus.cc:3060] T dec6410d8b0546bab2e4f3d352b54681 P 6c8db9dae5644d6290b55e3cae161043 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:24:22.492233 27530 raft_consensus.cc:2468] T dec6410d8b0546bab2e4f3d352b54681 P 6c8db9dae5644d6290b55e3cae161043 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29bb2c7b65094fafa8a7c5e5ecee48de in term 1.
I20260709 13:24:22.492233 27381 raft_consensus.cc:2468] T dec6410d8b0546bab2e4f3d352b54681 P 8dc4c6d0eae14641b9c5e44d31fcd335 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29bb2c7b65094fafa8a7c5e5ecee48de in term 1.
I20260709 13:24:22.493598 27421 leader_election.cc:304] T dec6410d8b0546bab2e4f3d352b54681 P 29bb2c7b65094fafa8a7c5e5ecee48de [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29bb2c7b65094fafa8a7c5e5ecee48de, 8dc4c6d0eae14641b9c5e44d31fcd335; no voters: 
I20260709 13:24:22.494328 27603 raft_consensus.cc:2804] T dec6410d8b0546bab2e4f3d352b54681 P 29bb2c7b65094fafa8a7c5e5ecee48de [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:24:22.494771 27603 raft_consensus.cc:697] T dec6410d8b0546bab2e4f3d352b54681 P 29bb2c7b65094fafa8a7c5e5ecee48de [term 1 LEADER]: Becoming Leader. State: Replica: 29bb2c7b65094fafa8a7c5e5ecee48de, State: Running, Role: LEADER
I20260709 13:24:22.495550 27603 consensus_queue.cc:237] T dec6410d8b0546bab2e4f3d352b54681 P 29bb2c7b65094fafa8a7c5e5ecee48de [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dc4c6d0eae14641b9c5e44d31fcd335" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36523 } } peers { permanent_uuid: "29bb2c7b65094fafa8a7c5e5ecee48de" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 43785 } } peers { permanent_uuid: "6c8db9dae5644d6290b55e3cae161043" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 39577 } }
I20260709 13:24:22.503074 27278 catalog_manager.cc:5697] T dec6410d8b0546bab2e4f3d352b54681 P 29bb2c7b65094fafa8a7c5e5ecee48de reported cstate change: term changed from 0 to 1, leader changed from <none> to 29bb2c7b65094fafa8a7c5e5ecee48de (127.14.177.130). New cstate: current_term: 1 leader_uuid: "29bb2c7b65094fafa8a7c5e5ecee48de" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dc4c6d0eae14641b9c5e44d31fcd335" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36523 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "29bb2c7b65094fafa8a7c5e5ecee48de" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 43785 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6c8db9dae5644d6290b55e3cae161043" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 39577 } health_report { overall_health: UNKNOWN } } }
I20260709 13:24:22.534649 27530 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "341e749fb378438e8fa98ebcb9066d34"
dest_uuid: "6c8db9dae5644d6290b55e3cae161043"
mode: GRACEFUL
new_leader_uuid: "8dc4c6d0eae14641b9c5e44d31fcd335"
 from {username='slave'} at 127.0.0.1:50142
I20260709 13:24:22.535331 27530 raft_consensus.cc:606] T 341e749fb378438e8fa98ebcb9066d34 P 6c8db9dae5644d6290b55e3cae161043 [term 1 LEADER]: Received request to transfer leadership to TS 8dc4c6d0eae14641b9c5e44d31fcd335
I20260709 13:24:22.541797 27586 consensus_queue.cc:1048] T 341e749fb378438e8fa98ebcb9066d34 P 6c8db9dae5644d6290b55e3cae161043 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29bb2c7b65094fafa8a7c5e5ecee48de" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 43785 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:24:22.544340 27586 consensus_queue.cc:1048] T 341e749fb378438e8fa98ebcb9066d34 P 6c8db9dae5644d6290b55e3cae161043 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8dc4c6d0eae14641b9c5e44d31fcd335" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36523 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:24:22.564592 27455 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "dec6410d8b0546bab2e4f3d352b54681"
dest_uuid: "29bb2c7b65094fafa8a7c5e5ecee48de"
mode: GRACEFUL
new_leader_uuid: "8dc4c6d0eae14641b9c5e44d31fcd335"
 from {username='slave'} at 127.0.0.1:34892
I20260709 13:24:22.565541 27455 raft_consensus.cc:606] T dec6410d8b0546bab2e4f3d352b54681 P 29bb2c7b65094fafa8a7c5e5ecee48de [term 1 LEADER]: Received request to transfer leadership to TS 8dc4c6d0eae14641b9c5e44d31fcd335
I20260709 13:24:22.571780 27614 raft_consensus.cc:993] T 341e749fb378438e8fa98ebcb9066d34 P 6c8db9dae5644d6290b55e3cae161043: : Instructing follower 8dc4c6d0eae14641b9c5e44d31fcd335 to start an election
I20260709 13:24:22.572315 27586 raft_consensus.cc:1081] T 341e749fb378438e8fa98ebcb9066d34 P 6c8db9dae5644d6290b55e3cae161043 [term 1 LEADER]: Signalling peer 8dc4c6d0eae14641b9c5e44d31fcd335 to start an election
I20260709 13:24:22.573326 27530 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4f9d369fbd364a7c958c7698fa8dd33f"
dest_uuid: "6c8db9dae5644d6290b55e3cae161043"
mode: GRACEFUL
new_leader_uuid: "8dc4c6d0eae14641b9c5e44d31fcd335"
 from {username='slave'} at 127.0.0.1:50142
I20260709 13:24:22.574080 27530 raft_consensus.cc:606] T 4f9d369fbd364a7c958c7698fa8dd33f P 6c8db9dae5644d6290b55e3cae161043 [term 1 LEADER]: Received request to transfer leadership to TS 8dc4c6d0eae14641b9c5e44d31fcd335
I20260709 13:24:22.576323 27381 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "341e749fb378438e8fa98ebcb9066d34"
dest_uuid: "8dc4c6d0eae14641b9c5e44d31fcd335"
 from {username='slave'} at 127.0.0.1:47584
I20260709 13:24:22.577106 27381 raft_consensus.cc:493] T 341e749fb378438e8fa98ebcb9066d34 P 8dc4c6d0eae14641b9c5e44d31fcd335 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:24:22.577548 27381 raft_consensus.cc:3060] T 341e749fb378438e8fa98ebcb9066d34 P 8dc4c6d0eae14641b9c5e44d31fcd335 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:24:22.585512 27381 raft_consensus.cc:515] T 341e749fb378438e8fa98ebcb9066d34 P 8dc4c6d0eae14641b9c5e44d31fcd335 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c8db9dae5644d6290b55e3cae161043" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 39577 } } peers { permanent_uuid: "29bb2c7b65094fafa8a7c5e5ecee48de" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 43785 } } peers { permanent_uuid: "8dc4c6d0eae14641b9c5e44d31fcd335" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36523 } }
I20260709 13:24:22.586742 27380 raft_consensus.cc:1240] T 341e749fb378438e8fa98ebcb9066d34 P 8dc4c6d0eae14641b9c5e44d31fcd335 [term 2 FOLLOWER]: Rejecting Update request from peer 6c8db9dae5644d6290b55e3cae161043 for earlier term 1. Current term is 2. Ops: []
I20260709 13:24:22.588322 27381 leader_election.cc:290] T 341e749fb378438e8fa98ebcb9066d34 P 8dc4c6d0eae14641b9c5e44d31fcd335 [CANDIDATE]: Term 2 election: Requested vote from peers 6c8db9dae5644d6290b55e3cae161043 (127.14.177.131:39577), 29bb2c7b65094fafa8a7c5e5ecee48de (127.14.177.130:43785)
I20260709 13:24:22.592691 27578 consensus_queue.cc:1059] T 341e749fb378438e8fa98ebcb9066d34 P 6c8db9dae5644d6290b55e3cae161043 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8dc4c6d0eae14641b9c5e44d31fcd335" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36523 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:24:22.593828 27578 raft_consensus.cc:3055] T 341e749fb378438e8fa98ebcb9066d34 P 6c8db9dae5644d6290b55e3cae161043 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:24:22.594290 27578 raft_consensus.cc:740] T 341e749fb378438e8fa98ebcb9066d34 P 6c8db9dae5644d6290b55e3cae161043 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 6c8db9dae5644d6290b55e3cae161043, State: Running, Role: LEADER
I20260709 13:24:22.595155 27578 consensus_queue.cc:260] T 341e749fb378438e8fa98ebcb9066d34 P 6c8db9dae5644d6290b55e3cae161043 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c8db9dae5644d6290b55e3cae161043" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 39577 } } peers { permanent_uuid: "29bb2c7b65094fafa8a7c5e5ecee48de" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 43785 } } peers { permanent_uuid: "8dc4c6d0eae14641b9c5e44d31fcd335" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36523 } }
I20260709 13:24:22.596555 27578 raft_consensus.cc:3060] T 341e749fb378438e8fa98ebcb9066d34 P 6c8db9dae5644d6290b55e3cae161043 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:24:22.605655 27530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "341e749fb378438e8fa98ebcb9066d34" candidate_uuid: "8dc4c6d0eae14641b9c5e44d31fcd335" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "6c8db9dae5644d6290b55e3cae161043"
I20260709 13:24:22.610381 27455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "341e749fb378438e8fa98ebcb9066d34" candidate_uuid: "8dc4c6d0eae14641b9c5e44d31fcd335" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "29bb2c7b65094fafa8a7c5e5ecee48de"
I20260709 13:24:22.611084 27455 raft_consensus.cc:3060] T 341e749fb378438e8fa98ebcb9066d34 P 29bb2c7b65094fafa8a7c5e5ecee48de [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:24:22.615288 27530 raft_consensus.cc:2468] T 341e749fb378438e8fa98ebcb9066d34 P 6c8db9dae5644d6290b55e3cae161043 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8dc4c6d0eae14641b9c5e44d31fcd335 in term 2.
I20260709 13:24:22.618724 27347 leader_election.cc:304] T 341e749fb378438e8fa98ebcb9066d34 P 8dc4c6d0eae14641b9c5e44d31fcd335 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c8db9dae5644d6290b55e3cae161043, 8dc4c6d0eae14641b9c5e44d31fcd335; no voters: 
I20260709 13:24:22.618884 27455 raft_consensus.cc:2468] T 341e749fb378438e8fa98ebcb9066d34 P 29bb2c7b65094fafa8a7c5e5ecee48de [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8dc4c6d0eae14641b9c5e44d31fcd335 in term 2.
I20260709 13:24:22.619768 27576 raft_consensus.cc:2804] T 341e749fb378438e8fa98ebcb9066d34 P 8dc4c6d0eae14641b9c5e44d31fcd335 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:24:22.621219 27576 raft_consensus.cc:697] T 341e749fb378438e8fa98ebcb9066d34 P 8dc4c6d0eae14641b9c5e44d31fcd335 [term 2 LEADER]: Becoming Leader. State: Replica: 8dc4c6d0eae14641b9c5e44d31fcd335, State: Running, Role: LEADER
I20260709 13:24:22.622201 27576 consensus_queue.cc:237] T 341e749fb378438e8fa98ebcb9066d34 P 8dc4c6d0eae14641b9c5e44d31fcd335 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c8db9dae5644d6290b55e3cae161043" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 39577 } } peers { permanent_uuid: "29bb2c7b65094fafa8a7c5e5ecee48de" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 43785 } } peers { permanent_uuid: "8dc4c6d0eae14641b9c5e44d31fcd335" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36523 } }
I20260709 13:24:22.630432 27279 catalog_manager.cc:5697] T 341e749fb378438e8fa98ebcb9066d34 P 8dc4c6d0eae14641b9c5e44d31fcd335 reported cstate change: term changed from 1 to 2, leader changed from 6c8db9dae5644d6290b55e3cae161043 (127.14.177.131) to 8dc4c6d0eae14641b9c5e44d31fcd335 (127.14.177.129). New cstate: current_term: 2 leader_uuid: "8dc4c6d0eae14641b9c5e44d31fcd335" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c8db9dae5644d6290b55e3cae161043" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 39577 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "29bb2c7b65094fafa8a7c5e5ecee48de" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 43785 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8dc4c6d0eae14641b9c5e44d31fcd335" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36523 } health_report { overall_health: HEALTHY } } }
I20260709 13:24:22.929415 27578 consensus_queue.cc:1048] T 4f9d369fbd364a7c958c7698fa8dd33f P 6c8db9dae5644d6290b55e3cae161043 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8dc4c6d0eae14641b9c5e44d31fcd335" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36523 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:24:22.937192 27615 raft_consensus.cc:993] T 4f9d369fbd364a7c958c7698fa8dd33f P 6c8db9dae5644d6290b55e3cae161043: : Instructing follower 8dc4c6d0eae14641b9c5e44d31fcd335 to start an election
I20260709 13:24:22.937741 27615 raft_consensus.cc:1081] T 4f9d369fbd364a7c958c7698fa8dd33f P 6c8db9dae5644d6290b55e3cae161043 [term 1 LEADER]: Signalling peer 8dc4c6d0eae14641b9c5e44d31fcd335 to start an election
I20260709 13:24:22.939743 27380 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "4f9d369fbd364a7c958c7698fa8dd33f"
dest_uuid: "8dc4c6d0eae14641b9c5e44d31fcd335"
 from {username='slave'} at 127.0.0.1:47584
I20260709 13:24:22.941774 27380 raft_consensus.cc:493] T 4f9d369fbd364a7c958c7698fa8dd33f P 8dc4c6d0eae14641b9c5e44d31fcd335 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:24:22.942189 27380 raft_consensus.cc:3060] T 4f9d369fbd364a7c958c7698fa8dd33f P 8dc4c6d0eae14641b9c5e44d31fcd335 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:24:22.943539 27614 consensus_queue.cc:1048] T 4f9d369fbd364a7c958c7698fa8dd33f P 6c8db9dae5644d6290b55e3cae161043 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29bb2c7b65094fafa8a7c5e5ecee48de" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 43785 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:24:22.951459 27380 raft_consensus.cc:515] T 4f9d369fbd364a7c958c7698fa8dd33f P 8dc4c6d0eae14641b9c5e44d31fcd335 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dc4c6d0eae14641b9c5e44d31fcd335" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36523 } } peers { permanent_uuid: "29bb2c7b65094fafa8a7c5e5ecee48de" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 43785 } } peers { permanent_uuid: "6c8db9dae5644d6290b55e3cae161043" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 39577 } }
I20260709 13:24:22.958379 27530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f9d369fbd364a7c958c7698fa8dd33f" candidate_uuid: "8dc4c6d0eae14641b9c5e44d31fcd335" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "6c8db9dae5644d6290b55e3cae161043"
I20260709 13:24:22.958979 27530 raft_consensus.cc:3055] T 4f9d369fbd364a7c958c7698fa8dd33f P 6c8db9dae5644d6290b55e3cae161043 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:24:22.959349 27530 raft_consensus.cc:740] T 4f9d369fbd364a7c958c7698fa8dd33f P 6c8db9dae5644d6290b55e3cae161043 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 6c8db9dae5644d6290b55e3cae161043, State: Running, Role: LEADER
I20260709 13:24:22.960016 27455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f9d369fbd364a7c958c7698fa8dd33f" candidate_uuid: "8dc4c6d0eae14641b9c5e44d31fcd335" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "29bb2c7b65094fafa8a7c5e5ecee48de"
I20260709 13:24:22.960248 27530 consensus_queue.cc:260] T 4f9d369fbd364a7c958c7698fa8dd33f P 6c8db9dae5644d6290b55e3cae161043 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dc4c6d0eae14641b9c5e44d31fcd335" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36523 } } peers { permanent_uuid: "29bb2c7b65094fafa8a7c5e5ecee48de" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 43785 } } peers { permanent_uuid: "6c8db9dae5644d6290b55e3cae161043" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 39577 } }
I20260709 13:24:22.960707 27455 raft_consensus.cc:3060] T 4f9d369fbd364a7c958c7698fa8dd33f P 29bb2c7b65094fafa8a7c5e5ecee48de [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:24:22.961320 27530 raft_consensus.cc:3060] T 4f9d369fbd364a7c958c7698fa8dd33f P 6c8db9dae5644d6290b55e3cae161043 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:24:22.966528 27455 raft_consensus.cc:2468] T 4f9d369fbd364a7c958c7698fa8dd33f P 29bb2c7b65094fafa8a7c5e5ecee48de [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8dc4c6d0eae14641b9c5e44d31fcd335 in term 2.
I20260709 13:24:22.967056 27530 raft_consensus.cc:2468] T 4f9d369fbd364a7c958c7698fa8dd33f P 6c8db9dae5644d6290b55e3cae161043 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8dc4c6d0eae14641b9c5e44d31fcd335 in term 2.
I20260709 13:24:22.967233 27380 leader_election.cc:290] T 4f9d369fbd364a7c958c7698fa8dd33f P 8dc4c6d0eae14641b9c5e44d31fcd335 [CANDIDATE]: Term 2 election: Requested vote from peers 29bb2c7b65094fafa8a7c5e5ecee48de (127.14.177.130:43785), 6c8db9dae5644d6290b55e3cae161043 (127.14.177.131:39577)
I20260709 13:24:22.967950 27345 leader_election.cc:304] T 4f9d369fbd364a7c958c7698fa8dd33f P 8dc4c6d0eae14641b9c5e44d31fcd335 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29bb2c7b65094fafa8a7c5e5ecee48de, 8dc4c6d0eae14641b9c5e44d31fcd335; no voters: 
I20260709 13:24:22.968886 27576 raft_consensus.cc:2804] T 4f9d369fbd364a7c958c7698fa8dd33f P 8dc4c6d0eae14641b9c5e44d31fcd335 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:24:22.969297 27576 raft_consensus.cc:697] T 4f9d369fbd364a7c958c7698fa8dd33f P 8dc4c6d0eae14641b9c5e44d31fcd335 [term 2 LEADER]: Becoming Leader. State: Replica: 8dc4c6d0eae14641b9c5e44d31fcd335, State: Running, Role: LEADER
I20260709 13:24:22.970013 27576 consensus_queue.cc:237] T 4f9d369fbd364a7c958c7698fa8dd33f P 8dc4c6d0eae14641b9c5e44d31fcd335 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dc4c6d0eae14641b9c5e44d31fcd335" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36523 } } peers { permanent_uuid: "29bb2c7b65094fafa8a7c5e5ecee48de" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 43785 } } peers { permanent_uuid: "6c8db9dae5644d6290b55e3cae161043" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 39577 } }
I20260709 13:24:22.976711 27278 catalog_manager.cc:5697] T 4f9d369fbd364a7c958c7698fa8dd33f P 8dc4c6d0eae14641b9c5e44d31fcd335 reported cstate change: term changed from 1 to 2, leader changed from 6c8db9dae5644d6290b55e3cae161043 (127.14.177.131) to 8dc4c6d0eae14641b9c5e44d31fcd335 (127.14.177.129). New cstate: current_term: 2 leader_uuid: "8dc4c6d0eae14641b9c5e44d31fcd335" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dc4c6d0eae14641b9c5e44d31fcd335" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36523 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "29bb2c7b65094fafa8a7c5e5ecee48de" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 43785 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c8db9dae5644d6290b55e3cae161043" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 39577 } health_report { overall_health: UNKNOWN } } }
I20260709 13:24:23.007874 27603 consensus_queue.cc:1048] T dec6410d8b0546bab2e4f3d352b54681 P 29bb2c7b65094fafa8a7c5e5ecee48de [LEADER]: Connected to new peer: Peer: permanent_uuid: "8dc4c6d0eae14641b9c5e44d31fcd335" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36523 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:24:23.015566 27623 raft_consensus.cc:993] T dec6410d8b0546bab2e4f3d352b54681 P 29bb2c7b65094fafa8a7c5e5ecee48de: : Instructing follower 8dc4c6d0eae14641b9c5e44d31fcd335 to start an election
I20260709 13:24:23.016403 27603 raft_consensus.cc:1081] T dec6410d8b0546bab2e4f3d352b54681 P 29bb2c7b65094fafa8a7c5e5ecee48de [term 1 LEADER]: Signalling peer 8dc4c6d0eae14641b9c5e44d31fcd335 to start an election
I20260709 13:24:23.018646 27381 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "dec6410d8b0546bab2e4f3d352b54681"
dest_uuid: "8dc4c6d0eae14641b9c5e44d31fcd335"
 from {username='slave'} at 127.0.0.1:47570
I20260709 13:24:23.019239 27381 raft_consensus.cc:493] T dec6410d8b0546bab2e4f3d352b54681 P 8dc4c6d0eae14641b9c5e44d31fcd335 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:24:23.019642 27381 raft_consensus.cc:3060] T dec6410d8b0546bab2e4f3d352b54681 P 8dc4c6d0eae14641b9c5e44d31fcd335 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:24:23.023698 27623 consensus_queue.cc:1048] T dec6410d8b0546bab2e4f3d352b54681 P 29bb2c7b65094fafa8a7c5e5ecee48de [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c8db9dae5644d6290b55e3cae161043" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 39577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:24:23.027761 27381 raft_consensus.cc:515] T dec6410d8b0546bab2e4f3d352b54681 P 8dc4c6d0eae14641b9c5e44d31fcd335 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dc4c6d0eae14641b9c5e44d31fcd335" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36523 } } peers { permanent_uuid: "29bb2c7b65094fafa8a7c5e5ecee48de" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 43785 } } peers { permanent_uuid: "6c8db9dae5644d6290b55e3cae161043" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 39577 } }
I20260709 13:24:23.032373 27381 leader_election.cc:290] T dec6410d8b0546bab2e4f3d352b54681 P 8dc4c6d0eae14641b9c5e44d31fcd335 [CANDIDATE]: Term 2 election: Requested vote from peers 29bb2c7b65094fafa8a7c5e5ecee48de (127.14.177.130:43785), 6c8db9dae5644d6290b55e3cae161043 (127.14.177.131:39577)
I20260709 13:24:23.033473 27529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dec6410d8b0546bab2e4f3d352b54681" candidate_uuid: "8dc4c6d0eae14641b9c5e44d31fcd335" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "6c8db9dae5644d6290b55e3cae161043"
I20260709 13:24:23.033646 27455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dec6410d8b0546bab2e4f3d352b54681" candidate_uuid: "8dc4c6d0eae14641b9c5e44d31fcd335" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "29bb2c7b65094fafa8a7c5e5ecee48de"
I20260709 13:24:23.034430 27455 raft_consensus.cc:3055] T dec6410d8b0546bab2e4f3d352b54681 P 29bb2c7b65094fafa8a7c5e5ecee48de [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:24:23.034874 27455 raft_consensus.cc:740] T dec6410d8b0546bab2e4f3d352b54681 P 29bb2c7b65094fafa8a7c5e5ecee48de [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 29bb2c7b65094fafa8a7c5e5ecee48de, State: Running, Role: LEADER
I20260709 13:24:23.037781 27455 consensus_queue.cc:260] T dec6410d8b0546bab2e4f3d352b54681 P 29bb2c7b65094fafa8a7c5e5ecee48de [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dc4c6d0eae14641b9c5e44d31fcd335" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36523 } } peers { permanent_uuid: "29bb2c7b65094fafa8a7c5e5ecee48de" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 43785 } } peers { permanent_uuid: "6c8db9dae5644d6290b55e3cae161043" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 39577 } }
I20260709 13:24:23.039498 27455 raft_consensus.cc:3060] T dec6410d8b0546bab2e4f3d352b54681 P 29bb2c7b65094fafa8a7c5e5ecee48de [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:24:23.048251 27455 raft_consensus.cc:2468] T dec6410d8b0546bab2e4f3d352b54681 P 29bb2c7b65094fafa8a7c5e5ecee48de [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8dc4c6d0eae14641b9c5e44d31fcd335 in term 2.
I20260709 13:24:23.050338 27345 leader_election.cc:304] T dec6410d8b0546bab2e4f3d352b54681 P 8dc4c6d0eae14641b9c5e44d31fcd335 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29bb2c7b65094fafa8a7c5e5ecee48de, 8dc4c6d0eae14641b9c5e44d31fcd335; no voters: 6c8db9dae5644d6290b55e3cae161043
I20260709 13:24:23.051544 27576 raft_consensus.cc:2804] T dec6410d8b0546bab2e4f3d352b54681 P 8dc4c6d0eae14641b9c5e44d31fcd335 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:24:23.052330 27576 raft_consensus.cc:697] T dec6410d8b0546bab2e4f3d352b54681 P 8dc4c6d0eae14641b9c5e44d31fcd335 [term 2 LEADER]: Becoming Leader. State: Replica: 8dc4c6d0eae14641b9c5e44d31fcd335, State: Running, Role: LEADER
I20260709 13:24:23.053227 27576 consensus_queue.cc:237] T dec6410d8b0546bab2e4f3d352b54681 P 8dc4c6d0eae14641b9c5e44d31fcd335 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dc4c6d0eae14641b9c5e44d31fcd335" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36523 } } peers { permanent_uuid: "29bb2c7b65094fafa8a7c5e5ecee48de" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 43785 } } peers { permanent_uuid: "6c8db9dae5644d6290b55e3cae161043" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 39577 } }
I20260709 13:24:23.060681 27279 catalog_manager.cc:5697] T dec6410d8b0546bab2e4f3d352b54681 P 8dc4c6d0eae14641b9c5e44d31fcd335 reported cstate change: term changed from 1 to 2, leader changed from 29bb2c7b65094fafa8a7c5e5ecee48de (127.14.177.130) to 8dc4c6d0eae14641b9c5e44d31fcd335 (127.14.177.129). New cstate: current_term: 2 leader_uuid: "8dc4c6d0eae14641b9c5e44d31fcd335" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dc4c6d0eae14641b9c5e44d31fcd335" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36523 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "29bb2c7b65094fafa8a7c5e5ecee48de" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 43785 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c8db9dae5644d6290b55e3cae161043" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 39577 } health_report { overall_health: UNKNOWN } } }
I20260709 13:24:23.141395 27455 raft_consensus.cc:1275] T 341e749fb378438e8fa98ebcb9066d34 P 29bb2c7b65094fafa8a7c5e5ecee48de [term 2 FOLLOWER]: Refusing update from remote peer 8dc4c6d0eae14641b9c5e44d31fcd335: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:24:23.142737 27576 consensus_queue.cc:1048] T 341e749fb378438e8fa98ebcb9066d34 P 8dc4c6d0eae14641b9c5e44d31fcd335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29bb2c7b65094fafa8a7c5e5ecee48de" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 43785 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:24:23.152524 27530 raft_consensus.cc:1275] T 341e749fb378438e8fa98ebcb9066d34 P 6c8db9dae5644d6290b55e3cae161043 [term 2 FOLLOWER]: Refusing update from remote peer 8dc4c6d0eae14641b9c5e44d31fcd335: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:24:23.154075 27626 consensus_queue.cc:1048] T 341e749fb378438e8fa98ebcb9066d34 P 8dc4c6d0eae14641b9c5e44d31fcd335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c8db9dae5644d6290b55e3cae161043" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 39577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:24:23.335124 15046 auto_leader_rebalancer.cc:141] leader rebalance for table multi_table_leader_balance_table1
I20260709 13:24:23.336741 15046 auto_leader_rebalancer.cc:452] table: multi_table_leader_balance_table1, leader rebalance finish, leader transfer count: 0
I20260709 13:24:23.337153 15046 auto_leader_rebalancer.cc:141] leader rebalance for table multi_table_leader_balance_table2
I20260709 13:24:23.349061 27381 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "dec6410d8b0546bab2e4f3d352b54681"
dest_uuid: "8dc4c6d0eae14641b9c5e44d31fcd335"
mode: GRACEFUL
new_leader_uuid: "6c8db9dae5644d6290b55e3cae161043"
 from {username='slave'} at 127.0.0.1:47590
I20260709 13:24:23.349550 27381 raft_consensus.cc:606] T dec6410d8b0546bab2e4f3d352b54681 P 8dc4c6d0eae14641b9c5e44d31fcd335 [term 2 LEADER]: Received request to transfer leadership to TS 6c8db9dae5644d6290b55e3cae161043
I20260709 13:24:23.350706 15046 auto_leader_rebalancer.cc:452] table: multi_table_leader_balance_table2, leader rebalance finish, leader transfer count: 1
I20260709 13:24:23.403311 27530 raft_consensus.cc:1275] T 4f9d369fbd364a7c958c7698fa8dd33f P 6c8db9dae5644d6290b55e3cae161043 [term 2 FOLLOWER]: Refusing update from remote peer 8dc4c6d0eae14641b9c5e44d31fcd335: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:24:23.404663 27626 consensus_queue.cc:1048] T 4f9d369fbd364a7c958c7698fa8dd33f P 8dc4c6d0eae14641b9c5e44d31fcd335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c8db9dae5644d6290b55e3cae161043" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 39577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:24:23.413582 27455 raft_consensus.cc:1275] T 4f9d369fbd364a7c958c7698fa8dd33f P 29bb2c7b65094fafa8a7c5e5ecee48de [term 2 FOLLOWER]: Refusing update from remote peer 8dc4c6d0eae14641b9c5e44d31fcd335: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:24:23.415613 27576 consensus_queue.cc:1048] T 4f9d369fbd364a7c958c7698fa8dd33f P 8dc4c6d0eae14641b9c5e44d31fcd335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29bb2c7b65094fafa8a7c5e5ecee48de" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 43785 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:24:23.517982 27530 raft_consensus.cc:3060] T dec6410d8b0546bab2e4f3d352b54681 P 6c8db9dae5644d6290b55e3cae161043 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:24:23.523981 27530 raft_consensus.cc:1275] T dec6410d8b0546bab2e4f3d352b54681 P 6c8db9dae5644d6290b55e3cae161043 [term 2 FOLLOWER]: Refusing update from remote peer 8dc4c6d0eae14641b9c5e44d31fcd335: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:24:23.525533 27576 consensus_queue.cc:1048] T dec6410d8b0546bab2e4f3d352b54681 P 8dc4c6d0eae14641b9c5e44d31fcd335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c8db9dae5644d6290b55e3cae161043" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 39577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:24:23.531446 27626 raft_consensus.cc:993] T dec6410d8b0546bab2e4f3d352b54681 P 8dc4c6d0eae14641b9c5e44d31fcd335: : Instructing follower 6c8db9dae5644d6290b55e3cae161043 to start an election
I20260709 13:24:23.532037 27618 raft_consensus.cc:1081] T dec6410d8b0546bab2e4f3d352b54681 P 8dc4c6d0eae14641b9c5e44d31fcd335 [term 2 LEADER]: Signalling peer 6c8db9dae5644d6290b55e3cae161043 to start an election
I20260709 13:24:23.537108 27530 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "dec6410d8b0546bab2e4f3d352b54681"
dest_uuid: "6c8db9dae5644d6290b55e3cae161043"
 from {username='slave'} at 127.0.0.1:50156
I20260709 13:24:23.537930 27530 raft_consensus.cc:493] T dec6410d8b0546bab2e4f3d352b54681 P 6c8db9dae5644d6290b55e3cae161043 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:24:23.538166 27455 raft_consensus.cc:1275] T dec6410d8b0546bab2e4f3d352b54681 P 29bb2c7b65094fafa8a7c5e5ecee48de [term 2 FOLLOWER]: Refusing update from remote peer 8dc4c6d0eae14641b9c5e44d31fcd335: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:24:23.538439 27530 raft_consensus.cc:3060] T dec6410d8b0546bab2e4f3d352b54681 P 6c8db9dae5644d6290b55e3cae161043 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:24:23.541041 27618 consensus_queue.cc:1048] T dec6410d8b0546bab2e4f3d352b54681 P 8dc4c6d0eae14641b9c5e44d31fcd335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29bb2c7b65094fafa8a7c5e5ecee48de" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 43785 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:24:23.547127 27530 raft_consensus.cc:515] T dec6410d8b0546bab2e4f3d352b54681 P 6c8db9dae5644d6290b55e3cae161043 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dc4c6d0eae14641b9c5e44d31fcd335" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36523 } } peers { permanent_uuid: "29bb2c7b65094fafa8a7c5e5ecee48de" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 43785 } } peers { permanent_uuid: "6c8db9dae5644d6290b55e3cae161043" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 39577 } }
I20260709 13:24:23.550037 27530 leader_election.cc:290] T dec6410d8b0546bab2e4f3d352b54681 P 6c8db9dae5644d6290b55e3cae161043 [CANDIDATE]: Term 3 election: Requested vote from peers 8dc4c6d0eae14641b9c5e44d31fcd335 (127.14.177.129:36523), 29bb2c7b65094fafa8a7c5e5ecee48de (127.14.177.130:43785)
I20260709 13:24:23.550683 27381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dec6410d8b0546bab2e4f3d352b54681" candidate_uuid: "6c8db9dae5644d6290b55e3cae161043" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "8dc4c6d0eae14641b9c5e44d31fcd335"
I20260709 13:24:23.551230 27456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dec6410d8b0546bab2e4f3d352b54681" candidate_uuid: "6c8db9dae5644d6290b55e3cae161043" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "29bb2c7b65094fafa8a7c5e5ecee48de"
I20260709 13:24:23.551429 27381 raft_consensus.cc:3055] T dec6410d8b0546bab2e4f3d352b54681 P 8dc4c6d0eae14641b9c5e44d31fcd335 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:24:23.551869 27381 raft_consensus.cc:740] T dec6410d8b0546bab2e4f3d352b54681 P 8dc4c6d0eae14641b9c5e44d31fcd335 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 8dc4c6d0eae14641b9c5e44d31fcd335, State: Running, Role: LEADER
I20260709 13:24:23.551872 27456 raft_consensus.cc:3060] T dec6410d8b0546bab2e4f3d352b54681 P 29bb2c7b65094fafa8a7c5e5ecee48de [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:24:23.552866 27381 consensus_queue.cc:260] T dec6410d8b0546bab2e4f3d352b54681 P 8dc4c6d0eae14641b9c5e44d31fcd335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dc4c6d0eae14641b9c5e44d31fcd335" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36523 } } peers { permanent_uuid: "29bb2c7b65094fafa8a7c5e5ecee48de" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 43785 } } peers { permanent_uuid: "6c8db9dae5644d6290b55e3cae161043" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 39577 } }
I20260709 13:24:23.554153 27381 raft_consensus.cc:3060] T dec6410d8b0546bab2e4f3d352b54681 P 8dc4c6d0eae14641b9c5e44d31fcd335 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:24:23.557657 27456 raft_consensus.cc:2468] T dec6410d8b0546bab2e4f3d352b54681 P 29bb2c7b65094fafa8a7c5e5ecee48de [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c8db9dae5644d6290b55e3cae161043 in term 3.
I20260709 13:24:23.560019 27381 raft_consensus.cc:2468] T dec6410d8b0546bab2e4f3d352b54681 P 8dc4c6d0eae14641b9c5e44d31fcd335 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c8db9dae5644d6290b55e3cae161043 in term 3.
I20260709 13:24:23.561332 27495 leader_election.cc:304] T dec6410d8b0546bab2e4f3d352b54681 P 6c8db9dae5644d6290b55e3cae161043 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c8db9dae5644d6290b55e3cae161043, 8dc4c6d0eae14641b9c5e44d31fcd335; no voters: 
I20260709 13:24:23.562193 27614 raft_consensus.cc:2804] T dec6410d8b0546bab2e4f3d352b54681 P 6c8db9dae5644d6290b55e3cae161043 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:24:23.563357 27614 raft_consensus.cc:697] T dec6410d8b0546bab2e4f3d352b54681 P 6c8db9dae5644d6290b55e3cae161043 [term 3 LEADER]: Becoming Leader. State: Replica: 6c8db9dae5644d6290b55e3cae161043, State: Running, Role: LEADER
I20260709 13:24:23.564193 27614 consensus_queue.cc:237] T dec6410d8b0546bab2e4f3d352b54681 P 6c8db9dae5644d6290b55e3cae161043 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dc4c6d0eae14641b9c5e44d31fcd335" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36523 } } peers { permanent_uuid: "29bb2c7b65094fafa8a7c5e5ecee48de" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 43785 } } peers { permanent_uuid: "6c8db9dae5644d6290b55e3cae161043" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 39577 } }
I20260709 13:24:23.572935 27279 catalog_manager.cc:5697] T dec6410d8b0546bab2e4f3d352b54681 P 6c8db9dae5644d6290b55e3cae161043 reported cstate change: term changed from 2 to 3, leader changed from 8dc4c6d0eae14641b9c5e44d31fcd335 (127.14.177.129) to 6c8db9dae5644d6290b55e3cae161043 (127.14.177.131). New cstate: current_term: 3 leader_uuid: "6c8db9dae5644d6290b55e3cae161043" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dc4c6d0eae14641b9c5e44d31fcd335" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 36523 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "29bb2c7b65094fafa8a7c5e5ecee48de" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 43785 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c8db9dae5644d6290b55e3cae161043" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 39577 } health_report { overall_health: HEALTHY } } }
I20260709 13:24:23.718379 15046 tablet_server.cc:179] TabletServer@127.14.177.129:0 shutting down...
I20260709 13:24:23.738735 15046 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:24:23.739516 15046 tablet_replica.cc:333] T 4f9d369fbd364a7c958c7698fa8dd33f P 8dc4c6d0eae14641b9c5e44d31fcd335: stopping tablet replica
I20260709 13:24:23.740136 15046 raft_consensus.cc:2243] T 4f9d369fbd364a7c958c7698fa8dd33f P 8dc4c6d0eae14641b9c5e44d31fcd335 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:24:23.741000 15046 raft_consensus.cc:2272] T 4f9d369fbd364a7c958c7698fa8dd33f P 8dc4c6d0eae14641b9c5e44d31fcd335 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:24:23.743703 15046 tablet_replica.cc:333] T dec6410d8b0546bab2e4f3d352b54681 P 8dc4c6d0eae14641b9c5e44d31fcd335: stopping tablet replica
I20260709 13:24:23.744257 15046 raft_consensus.cc:2243] T dec6410d8b0546bab2e4f3d352b54681 P 8dc4c6d0eae14641b9c5e44d31fcd335 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:24:23.744693 15046 raft_consensus.cc:2272] T dec6410d8b0546bab2e4f3d352b54681 P 8dc4c6d0eae14641b9c5e44d31fcd335 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:24:23.747490 15046 tablet_replica.cc:333] T 2aaf9a0870eb46249ccf60a2123fd678 P 8dc4c6d0eae14641b9c5e44d31fcd335: stopping tablet replica
I20260709 13:24:23.748011 15046 raft_consensus.cc:2243] T 2aaf9a0870eb46249ccf60a2123fd678 P 8dc4c6d0eae14641b9c5e44d31fcd335 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:24:23.748425 15046 raft_consensus.cc:2272] T 2aaf9a0870eb46249ccf60a2123fd678 P 8dc4c6d0eae14641b9c5e44d31fcd335 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:24:23.750236 15046 tablet_replica.cc:333] T 341e749fb378438e8fa98ebcb9066d34 P 8dc4c6d0eae14641b9c5e44d31fcd335: stopping tablet replica
I20260709 13:24:23.750701 15046 raft_consensus.cc:2243] T 341e749fb378438e8fa98ebcb9066d34 P 8dc4c6d0eae14641b9c5e44d31fcd335 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:24:23.751628 15046 raft_consensus.cc:2272] T 341e749fb378438e8fa98ebcb9066d34 P 8dc4c6d0eae14641b9c5e44d31fcd335 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:24:23.768962 27421 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36523: connect: Connection refused (error 111) [suppressed 5 similar messages]
W20260709 13:24:23.772148 27421 consensus_peers.cc:597] T 2aaf9a0870eb46249ccf60a2123fd678 P 29bb2c7b65094fafa8a7c5e5ecee48de -> Peer 8dc4c6d0eae14641b9c5e44d31fcd335 (127.14.177.129:36523): Couldn't send request to peer 8dc4c6d0eae14641b9c5e44d31fcd335. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:36523: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:24:23.776556 15046 tablet_server.cc:196] TabletServer@127.14.177.129:0 shutdown complete.
I20260709 13:24:23.790408 15046 tablet_server.cc:179] TabletServer@127.14.177.130:0 shutting down...
I20260709 13:24:23.810175 15046 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:24:23.810765 15046 tablet_replica.cc:333] T 4f9d369fbd364a7c958c7698fa8dd33f P 29bb2c7b65094fafa8a7c5e5ecee48de: stopping tablet replica
I20260709 13:24:23.811372 15046 raft_consensus.cc:2243] T 4f9d369fbd364a7c958c7698fa8dd33f P 29bb2c7b65094fafa8a7c5e5ecee48de [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:24:23.811793 15046 raft_consensus.cc:2272] T 4f9d369fbd364a7c958c7698fa8dd33f P 29bb2c7b65094fafa8a7c5e5ecee48de [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:24:23.813936 15046 tablet_replica.cc:333] T dec6410d8b0546bab2e4f3d352b54681 P 29bb2c7b65094fafa8a7c5e5ecee48de: stopping tablet replica
I20260709 13:24:23.814417 15046 raft_consensus.cc:2243] T dec6410d8b0546bab2e4f3d352b54681 P 29bb2c7b65094fafa8a7c5e5ecee48de [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:24:23.814803 15046 raft_consensus.cc:2272] T dec6410d8b0546bab2e4f3d352b54681 P 29bb2c7b65094fafa8a7c5e5ecee48de [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:24:23.816905 15046 tablet_replica.cc:333] T 2aaf9a0870eb46249ccf60a2123fd678 P 29bb2c7b65094fafa8a7c5e5ecee48de: stopping tablet replica
I20260709 13:24:23.817373 15046 raft_consensus.cc:2243] T 2aaf9a0870eb46249ccf60a2123fd678 P 29bb2c7b65094fafa8a7c5e5ecee48de [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:24:23.818102 15046 raft_consensus.cc:2272] T 2aaf9a0870eb46249ccf60a2123fd678 P 29bb2c7b65094fafa8a7c5e5ecee48de [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:24:23.820035 15046 tablet_replica.cc:333] T 341e749fb378438e8fa98ebcb9066d34 P 29bb2c7b65094fafa8a7c5e5ecee48de: stopping tablet replica
I20260709 13:24:23.820472 15046 raft_consensus.cc:2243] T 341e749fb378438e8fa98ebcb9066d34 P 29bb2c7b65094fafa8a7c5e5ecee48de [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:24:23.820832 15046 raft_consensus.cc:2272] T 341e749fb378438e8fa98ebcb9066d34 P 29bb2c7b65094fafa8a7c5e5ecee48de [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:24:23.844928 15046 tablet_server.cc:196] TabletServer@127.14.177.130:0 shutdown complete.
I20260709 13:24:23.859584 15046 tablet_server.cc:179] TabletServer@127.14.177.131:0 shutting down...
I20260709 13:24:23.877206 15046 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:24:23.877873 15046 tablet_replica.cc:333] T 4f9d369fbd364a7c958c7698fa8dd33f P 6c8db9dae5644d6290b55e3cae161043: stopping tablet replica
I20260709 13:24:23.878453 15046 raft_consensus.cc:2243] T 4f9d369fbd364a7c958c7698fa8dd33f P 6c8db9dae5644d6290b55e3cae161043 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:24:23.878928 15046 raft_consensus.cc:2272] T 4f9d369fbd364a7c958c7698fa8dd33f P 6c8db9dae5644d6290b55e3cae161043 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:24:23.881297 15046 tablet_replica.cc:333] T dec6410d8b0546bab2e4f3d352b54681 P 6c8db9dae5644d6290b55e3cae161043: stopping tablet replica
I20260709 13:24:23.881834 15046 raft_consensus.cc:2243] T dec6410d8b0546bab2e4f3d352b54681 P 6c8db9dae5644d6290b55e3cae161043 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:24:23.882465 15046 pending_rounds.cc:70] T dec6410d8b0546bab2e4f3d352b54681 P 6c8db9dae5644d6290b55e3cae161043: Trying to abort 1 pending ops.
I20260709 13:24:23.882616 15046 pending_rounds.cc:77] T dec6410d8b0546bab2e4f3d352b54681 P 6c8db9dae5644d6290b55e3cae161043: Aborting op as it isn't in flight: id { term: 3 index: 3 } timestamp: 7305639786769760256 op_type: NO_OP noop_request { }
I20260709 13:24:23.882928 15046 raft_consensus.cc:2889] T dec6410d8b0546bab2e4f3d352b54681 P 6c8db9dae5644d6290b55e3cae161043 [term 3 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:24:23.883275 15046 raft_consensus.cc:2272] T dec6410d8b0546bab2e4f3d352b54681 P 6c8db9dae5644d6290b55e3cae161043 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:24:23.885488 15046 tablet_replica.cc:333] T 2aaf9a0870eb46249ccf60a2123fd678 P 6c8db9dae5644d6290b55e3cae161043: stopping tablet replica
I20260709 13:24:23.886018 15046 raft_consensus.cc:2243] T 2aaf9a0870eb46249ccf60a2123fd678 P 6c8db9dae5644d6290b55e3cae161043 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:24:23.886441 15046 raft_consensus.cc:2272] T 2aaf9a0870eb46249ccf60a2123fd678 P 6c8db9dae5644d6290b55e3cae161043 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:24:23.888458 15046 tablet_replica.cc:333] T 341e749fb378438e8fa98ebcb9066d34 P 6c8db9dae5644d6290b55e3cae161043: stopping tablet replica
I20260709 13:24:23.888926 15046 raft_consensus.cc:2243] T 341e749fb378438e8fa98ebcb9066d34 P 6c8db9dae5644d6290b55e3cae161043 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:24:23.889369 15046 raft_consensus.cc:2272] T 341e749fb378438e8fa98ebcb9066d34 P 6c8db9dae5644d6290b55e3cae161043 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:24:23.912281 15046 tablet_server.cc:196] TabletServer@127.14.177.131:0 shutdown complete.
I20260709 13:24:23.925961 15046 master.cc:562] Master@127.14.177.190:38139 shutting down...
I20260709 13:24:23.940779 15046 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 6728a69b22e545c9b04d1fa45a60a637 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:24:23.941319 15046 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 6728a69b22e545c9b04d1fa45a60a637 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:24:23.941598 15046 tablet_replica.cc:333] T 00000000000000000000000000000000 P 6728a69b22e545c9b04d1fa45a60a637: stopping tablet replica
I20260709 13:24:23.960793 15046 master.cc:584] Master@127.14.177.190:38139 shutdown complete.
[       OK ] LeaderRebalancerTest.MultiTableLeaderBalance (2910 ms)
[----------] 6 tests from LeaderRebalancerTest (370021 ms total)

[----------] 2 tests from FilterSoftDeletedTableTest
[ RUN      ] FilterSoftDeletedTableTest.TestFilterSofteDeletedTable/0
I20260709 13:24:23.991143 15046 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.14.177.190:41127
I20260709 13:24:23.992442 15046 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:24:23.997884 27635 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:24:23.998960 27636 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:24:24.000443 15046 server_base.cc:1061] running on GCE node
W20260709 13:24:24.000890 27638 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:24:24.001791 15046 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:24:24.001996 15046 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:24:24.002174 15046 hybrid_clock.cc:648] HybridClock initialized: now 1783603464002154 us; error 0 us; skew 500 ppm
I20260709 13:24:24.002714 15046 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:24:24.005300 15046 webserver.cc:533] Webserver started at http://127.14.177.190:42397/ using document root <none> and password file <none>
I20260709 13:24:24.005790 15046 fs_manager.cc:362] Metadata directory not provided
I20260709 13:24:24.005980 15046 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:24:24.006256 15046 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:24:24.007381 15046 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskdH_Ma7/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783603093801613-15046-0/minicluster-data/master-0-root/instance:
uuid: "c3a531df1ce7417cbf6095f23465e39b"
format_stamp: "Formatted at 2026-07-09 13:24:23 on dist-test-slave-v5z2"
I20260709 13:24:24.011958 15046 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.004s	sys 0.000s
I20260709 13:24:24.015623 27643 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:24:24.016646 15046 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.001s
I20260709 13:24:24.017045 15046 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskdH_Ma7/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783603093801613-15046-0/minicluster-data/master-0-root
uuid: "c3a531df1ce7417cbf6095f23465e39b"
format_stamp: "Formatted at 2026-07-09 13:24:23 on dist-test-slave-v5z2"
I20260709 13:24:24.017392 15046 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdH_Ma7/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783603093801613-15046-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskdH_Ma7/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783603093801613-15046-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskdH_Ma7/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783603093801613-15046-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:24:24.030004 15046 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:24:24.031272 15046 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:24:24.070178 15046 rpc_server.cc:307] RPC server started. Bound to: 127.14.177.190:41127
I20260709 13:24:24.070266 27694 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.14.177.190:41127 every 8 connection(s)
I20260709 13:24:24.074539 27695 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:24:24.086153 27695 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c3a531df1ce7417cbf6095f23465e39b: Bootstrap starting.
I20260709 13:24:24.090866 27695 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c3a531df1ce7417cbf6095f23465e39b: Neither blocks nor log segments found. Creating new log.
I20260709 13:24:24.095312 27695 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c3a531df1ce7417cbf6095f23465e39b: No bootstrap required, opened a new log
I20260709 13:24:24.097496 27695 raft_consensus.cc:359] T 00000000000000000000000000000000 P c3a531df1ce7417cbf6095f23465e39b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c3a531df1ce7417cbf6095f23465e39b" member_type: VOTER }
I20260709 13:24:24.097901 27695 raft_consensus.cc:385] T 00000000000000000000000000000000 P c3a531df1ce7417cbf6095f23465e39b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:24:24.098188 27695 raft_consensus.cc:740] T 00000000000000000000000000000000 P c3a531df1ce7417cbf6095f23465e39b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c3a531df1ce7417cbf6095f23465e39b, State: Initialized, Role: FOLLOWER
I20260709 13:24:24.098868 27695 consensus_queue.cc:260] T 00000000000000000000000000000000 P c3a531df1ce7417cbf6095f23465e39b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c3a531df1ce7417cbf6095f23465e39b" member_type: VOTER }
I20260709 13:24:24.099381 27695 raft_consensus.cc:399] T 00000000000000000000000000000000 P c3a531df1ce7417cbf6095f23465e39b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 13:24:24.099613 27695 raft_consensus.cc:493] T 00000000000000000000000000000000 P c3a531df1ce7417cbf6095f23465e39b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 13:24:24.099880 27695 raft_consensus.cc:3060] T 00000000000000000000000000000000 P c3a531df1ce7417cbf6095f23465e39b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:24:24.105028 27695 raft_consensus.cc:515] T 00000000000000000000000000000000 P c3a531df1ce7417cbf6095f23465e39b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c3a531df1ce7417cbf6095f23465e39b" member_type: VOTER }
I20260709 13:24:24.105585 27695 leader_election.cc:304] T 00000000000000000000000000000000 P c3a531df1ce7417cbf6095f23465e39b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c3a531df1ce7417cbf6095f23465e39b; no voters: 
I20260709 13:24:24.106732 27695 leader_election.cc:290] T 00000000000000000000000000000000 P c3a531df1ce7417cbf6095f23465e39b [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 13:24:24.107103 27698 raft_consensus.cc:2804] T 00000000000000000000000000000000 P c3a531df1ce7417cbf6095f23465e39b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:24:24.108625 27698 raft_consensus.cc:697] T 00000000000000000000000000000000 P c3a531df1ce7417cbf6095f23465e39b [term 1 LEADER]: Becoming Leader. State: Replica: c3a531df1ce7417cbf6095f23465e39b, State: Running, Role: LEADER
I20260709 13:24:24.109308 27698 consensus_queue.cc:237] T 00000000000000000000000000000000 P c3a531df1ce7417cbf6095f23465e39b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c3a531df1ce7417cbf6095f23465e39b" member_type: VOTER }
I20260709 13:24:24.109694 27695 sys_catalog.cc:565] T 00000000000000000000000000000000 P c3a531df1ce7417cbf6095f23465e39b [sys.catalog]: configured and running, proceeding with master startup.
I20260709 13:24:24.112174 27700 sys_catalog.cc:455] T 00000000000000000000000000000000 P c3a531df1ce7417cbf6095f23465e39b [sys.catalog]: SysCatalogTable state changed. Reason: New leader c3a531df1ce7417cbf6095f23465e39b. Latest consensus state: current_term: 1 leader_uuid: "c3a531df1ce7417cbf6095f23465e39b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c3a531df1ce7417cbf6095f23465e39b" member_type: VOTER } }
I20260709 13:24:24.112140 27699 sys_catalog.cc:455] T 00000000000000000000000000000000 P c3a531df1ce7417cbf6095f23465e39b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c3a531df1ce7417cbf6095f23465e39b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c3a531df1ce7417cbf6095f23465e39b" member_type: VOTER } }
I20260709 13:24:24.113036 27700 sys_catalog.cc:458] T 00000000000000000000000000000000 P c3a531df1ce7417cbf6095f23465e39b [sys.catalog]: This master's current role is: LEADER
I20260709 13:24:24.113116 27699 sys_catalog.cc:458] T 00000000000000000000000000000000 P c3a531df1ce7417cbf6095f23465e39b [sys.catalog]: This master's current role is: LEADER
I20260709 13:24:24.115732 27703 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 13:24:24.121026 27703 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 13:24:24.129575 15046 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 13:24:24.132942 27703 catalog_manager.cc:1383] Generated new cluster ID: 5b5315303bc546a886d4499839702f8d
I20260709 13:24:24.133381 27703 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 13:24:24.146837 27703 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 13:24:24.148258 27703 catalog_manager.cc:1540] Loading token signing keys...
I20260709 13:24:24.163313 27703 catalog_manager.cc:6070] T 00000000000000000000000000000000 P c3a531df1ce7417cbf6095f23465e39b: Generated new TSK 0
I20260709 13:24:24.164008 27703 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 13:24:24.199574 15046 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:24:24.205945 27716 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:24:24.207046 27717 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:24:24.208280 27719 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:24:24.209004 15046 server_base.cc:1061] running on GCE node
I20260709 13:24:24.209908 15046 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:24:24.210125 15046 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:24:24.210352 15046 hybrid_clock.cc:648] HybridClock initialized: now 1783603464210333 us; error 0 us; skew 500 ppm
I20260709 13:24:24.210994 15046 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:24:24.213496 15046 webserver.cc:533] Webserver started at http://127.14.177.129:34255/ using document root <none> and password file <none>
I20260709 13:24:24.214015 15046 fs_manager.cc:362] Metadata directory not provided
I20260709 13:24:24.214231 15046 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:24:24.214514 15046 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:24:24.215720 15046 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskdH_Ma7/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783603093801613-15046-0/minicluster-data/ts-0-root/instance:
uuid: "9cb4300ea183421ab40709a05de5a877"
format_stamp: "Formatted at 2026-07-09 13:24:24 on dist-test-slave-v5z2"
I20260709 13:24:24.220432 15046 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.001s	sys 0.004s
I20260709 13:24:24.223902 27724 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:24:24.224709 15046 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.001s
I20260709 13:24:24.225015 15046 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskdH_Ma7/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783603093801613-15046-0/minicluster-data/ts-0-root
uuid: "9cb4300ea183421ab40709a05de5a877"
format_stamp: "Formatted at 2026-07-09 13:24:24 on dist-test-slave-v5z2"
I20260709 13:24:24.225291 15046 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdH_Ma7/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783603093801613-15046-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskdH_Ma7/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783603093801613-15046-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskdH_Ma7/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783603093801613-15046-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:24:24.244303 15046 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:24:24.245498 15046 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:24:24.247001 15046 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:24:24.249361 15046 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:24:24.249538 15046 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:24:24.249837 15046 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:24:24.249989 15046 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:24:24.291678 15046 rpc_server.cc:307] RPC server started. Bound to: 127.14.177.129:33597
I20260709 13:24:24.291867 27786 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.14.177.129:33597 every 8 connection(s)
I20260709 13:24:24.296869 15046 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:24:24.304514 27791 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:24:24.305472 27792 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:24:24.307466 27794 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:24:24.309742 15046 server_base.cc:1061] running on GCE node
I20260709 13:24:24.311645 15046 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:24:24.311925 15046 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:24:24.312143 15046 hybrid_clock.cc:648] HybridClock initialized: now 1783603464312125 us; error 0 us; skew 500 ppm
I20260709 13:24:24.312772 15046 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:24:24.315049 27787 heartbeater.cc:344] Connected to a master server at 127.14.177.190:41127
I20260709 13:24:24.315610 27787 heartbeater.cc:461] Registering TS with master...
I20260709 13:24:24.315909 15046 webserver.cc:533] Webserver started at http://127.14.177.130:41063/ using document root <none> and password file <none>
I20260709 13:24:24.316632 15046 fs_manager.cc:362] Metadata directory not provided
I20260709 13:24:24.316537 27787 heartbeater.cc:507] Master 127.14.177.190:41127 requested a full tablet report, sending...
I20260709 13:24:24.317080 15046 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:24:24.317438 15046 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:24:24.318714 15046 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskdH_Ma7/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783603093801613-15046-0/minicluster-data/ts-1-root/instance:
uuid: "0788dbdabe574355bb323dcdc3be0058"
format_stamp: "Formatted at 2026-07-09 13:24:24 on dist-test-slave-v5z2"
I20260709 13:24:24.319258 27660 ts_manager.cc:194] Registered new tserver with Master: 9cb4300ea183421ab40709a05de5a877 (127.14.177.129:33597)
I20260709 13:24:24.321115 27660 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:60468
I20260709 13:24:24.324091 15046 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.000s	sys 0.004s
I20260709 13:24:24.328001 27799 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:24:24.328872 15046 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.002s	sys 0.002s
I20260709 13:24:24.329151 15046 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskdH_Ma7/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783603093801613-15046-0/minicluster-data/ts-1-root
uuid: "0788dbdabe574355bb323dcdc3be0058"
format_stamp: "Formatted at 2026-07-09 13:24:24 on dist-test-slave-v5z2"
I20260709 13:24:24.329406 15046 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdH_Ma7/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783603093801613-15046-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskdH_Ma7/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783603093801613-15046-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskdH_Ma7/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783603093801613-15046-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:24:24.352752 15046 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:24:24.353792 15046 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:24:24.355233 15046 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:24:24.357600 15046 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:24:24.357790 15046 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:24:24.358105 15046 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:24:24.358265 15046 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:24:24.397771 15046 rpc_server.cc:307] RPC server started. Bound to: 127.14.177.130:41335
I20260709 13:24:24.397888 27861 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.14.177.130:41335 every 8 connection(s)
I20260709 13:24:24.402937 15046 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:24:24.411055 27865 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:24:24.411919 27866 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:24:24.414430 15046 server_base.cc:1061] running on GCE node
W20260709 13:24:24.414364 27868 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:24:24.416085 15046 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
I20260709 13:24:24.416220 27862 heartbeater.cc:344] Connected to a master server at 127.14.177.190:41127
W20260709 13:24:24.416380 15046 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:24:24.416812 15046 hybrid_clock.cc:648] HybridClock initialized: now 1783603464416785 us; error 0 us; skew 500 ppm
I20260709 13:24:24.416842 27862 heartbeater.cc:461] Registering TS with master...
I20260709 13:24:24.417470 15046 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:24:24.417847 27862 heartbeater.cc:507] Master 127.14.177.190:41127 requested a full tablet report, sending...
I20260709 13:24:24.420451 27660 ts_manager.cc:194] Registered new tserver with Master: 0788dbdabe574355bb323dcdc3be0058 (127.14.177.130:41335)
I20260709 13:24:24.420527 15046 webserver.cc:533] Webserver started at http://127.14.177.131:37777/ using document root <none> and password file <none>
I20260709 13:24:24.421200 15046 fs_manager.cc:362] Metadata directory not provided
I20260709 13:24:24.421389 15046 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:24:24.421676 15046 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:24:24.422201 27660 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:60476
I20260709 13:24:24.422816 15046 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskdH_Ma7/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783603093801613-15046-0/minicluster-data/ts-2-root/instance:
uuid: "880fc4c7dff4417187b75cc1d9e9dcf5"
format_stamp: "Formatted at 2026-07-09 13:24:24 on dist-test-slave-v5z2"
I20260709 13:24:24.427903 15046 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.004s	sys 0.000s
I20260709 13:24:24.431456 27873 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:24:24.432209 15046 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.001s
I20260709 13:24:24.432533 15046 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskdH_Ma7/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783603093801613-15046-0/minicluster-data/ts-2-root
uuid: "880fc4c7dff4417187b75cc1d9e9dcf5"
format_stamp: "Formatted at 2026-07-09 13:24:24 on dist-test-slave-v5z2"
I20260709 13:24:24.432833 15046 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdH_Ma7/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783603093801613-15046-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskdH_Ma7/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783603093801613-15046-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskdH_Ma7/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783603093801613-15046-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:24:24.469777 15046 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:24:24.470887 15046 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:24:24.472477 15046 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:24:24.475023 15046 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:24:24.475313 15046 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.001s	sys 0.000s
I20260709 13:24:24.475576 15046 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:24:24.475761 15046 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:24:24.516793 15046 rpc_server.cc:307] RPC server started. Bound to: 127.14.177.131:34235
I20260709 13:24:24.516937 27935 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.14.177.131:34235 every 8 connection(s)
I20260709 13:24:24.531507 27936 heartbeater.cc:344] Connected to a master server at 127.14.177.190:41127
I20260709 13:24:24.532035 27936 heartbeater.cc:461] Registering TS with master...
I20260709 13:24:24.532917 27936 heartbeater.cc:507] Master 127.14.177.190:41127 requested a full tablet report, sending...
I20260709 13:24:24.535288 27660 ts_manager.cc:194] Registered new tserver with Master: 880fc4c7dff4417187b75cc1d9e9dcf5 (127.14.177.131:34235)
I20260709 13:24:24.536145 15046 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.01590176s
I20260709 13:24:24.536732 27660 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:60492
I20260709 13:24:24.564186 27660 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60498:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q""\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 13:24:24.566890 27660 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 13:24:24.637286 27752 tablet_service.cc:1511] Processing CreateTablet for tablet 20ef9c03fcd94115be0b592d85f89421 (DEFAULT_TABLE table=test-workload [id=db6f52ac21cb44e38a4318c2fc93ad42]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:24:24.638298 27750 tablet_service.cc:1511] Processing CreateTablet for tablet db487682b03b4a738f664fa1459714e5 (DEFAULT_TABLE table=test-workload [id=db6f52ac21cb44e38a4318c2fc93ad42]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:24:24.640192 27751 tablet_service.cc:1511] Processing CreateTablet for tablet d16bbb66a57e4079897aaa2c4e6ba144 (DEFAULT_TABLE table=test-workload [id=db6f52ac21cb44e38a4318c2fc93ad42]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:24:24.642519 27749 tablet_service.cc:1511] Processing CreateTablet for tablet f3e47f5448ff47a382c61886f1a8cc7b (DEFAULT_TABLE table=test-workload [id=db6f52ac21cb44e38a4318c2fc93ad42]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:24:24.638764 27752 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 20ef9c03fcd94115be0b592d85f89421. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:24:24.647640 27749 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f3e47f5448ff47a382c61886f1a8cc7b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:24:24.648290 27751 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d16bbb66a57e4079897aaa2c4e6ba144. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:24:24.655037 27827 tablet_service.cc:1511] Processing CreateTablet for tablet 20ef9c03fcd94115be0b592d85f89421 (DEFAULT_TABLE table=test-workload [id=db6f52ac21cb44e38a4318c2fc93ad42]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:24:24.657239 27827 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 20ef9c03fcd94115be0b592d85f89421. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:24:24.659531 27750 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet db487682b03b4a738f664fa1459714e5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:24:24.665218 27826 tablet_service.cc:1511] Processing CreateTablet for tablet d16bbb66a57e4079897aaa2c4e6ba144 (DEFAULT_TABLE table=test-workload [id=db6f52ac21cb44e38a4318c2fc93ad42]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:24:24.666769 27826 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d16bbb66a57e4079897aaa2c4e6ba144. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:24:24.669600 27825 tablet_service.cc:1511] Processing CreateTablet for tablet db487682b03b4a738f664fa1459714e5 (DEFAULT_TABLE table=test-workload [id=db6f52ac21cb44e38a4318c2fc93ad42]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:24:24.672642 27824 tablet_service.cc:1511] Processing CreateTablet for tablet f3e47f5448ff47a382c61886f1a8cc7b (DEFAULT_TABLE table=test-workload [id=db6f52ac21cb44e38a4318c2fc93ad42]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:24:24.674932 27825 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet db487682b03b4a738f664fa1459714e5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:24:24.681309 27824 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f3e47f5448ff47a382c61886f1a8cc7b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:24:24.695897 27950 tablet_bootstrap.cc:492] T d16bbb66a57e4079897aaa2c4e6ba144 P 9cb4300ea183421ab40709a05de5a877: Bootstrap starting.
I20260709 13:24:24.707383 27750 tablet_service.cc:1511] Processing CreateTablet for tablet e7366c36a1154d4d94281a90fde291f8 (DEFAULT_TABLE table=test-workload [id=db6f52ac21cb44e38a4318c2fc93ad42]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:24:24.708089 27951 tablet_bootstrap.cc:492] T 20ef9c03fcd94115be0b592d85f89421 P 0788dbdabe574355bb323dcdc3be0058: Bootstrap starting.
I20260709 13:24:24.709023 27750 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e7366c36a1154d4d94281a90fde291f8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:24:24.726087 27826 tablet_service.cc:1511] Processing CreateTablet for tablet e7366c36a1154d4d94281a90fde291f8 (DEFAULT_TABLE table=test-workload [id=db6f52ac21cb44e38a4318c2fc93ad42]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:24:24.728070 27826 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e7366c36a1154d4d94281a90fde291f8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:24:24.733484 27827 tablet_service.cc:1511] Processing CreateTablet for tablet 4554587582094286b577c9c9a4d3a7fc (DEFAULT_TABLE table=test-workload [id=db6f52ac21cb44e38a4318c2fc93ad42]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:24:24.735044 27827 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4554587582094286b577c9c9a4d3a7fc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:24:24.735502 27749 tablet_service.cc:1511] Processing CreateTablet for tablet 4554587582094286b577c9c9a4d3a7fc (DEFAULT_TABLE table=test-workload [id=db6f52ac21cb44e38a4318c2fc93ad42]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:24:24.737138 27901 tablet_service.cc:1511] Processing CreateTablet for tablet 20ef9c03fcd94115be0b592d85f89421 (DEFAULT_TABLE table=test-workload [id=db6f52ac21cb44e38a4318c2fc93ad42]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:24:24.738606 27901 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 20ef9c03fcd94115be0b592d85f89421. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:24:24.740406 27900 tablet_service.cc:1511] Processing CreateTablet for tablet d16bbb66a57e4079897aaa2c4e6ba144 (DEFAULT_TABLE table=test-workload [id=db6f52ac21cb44e38a4318c2fc93ad42]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:24:24.741128 27951 tablet_bootstrap.cc:654] T 20ef9c03fcd94115be0b592d85f89421 P 0788dbdabe574355bb323dcdc3be0058: Neither blocks nor log segments found. Creating new log.
I20260709 13:24:24.742019 27900 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d16bbb66a57e4079897aaa2c4e6ba144. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:24:24.742167 27750 tablet_service.cc:1511] Processing CreateTablet for tablet cd1140068fc8455280934d7a30c4f559 (DEFAULT_TABLE table=test-workload [id=db6f52ac21cb44e38a4318c2fc93ad42]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:24:24.743604 27823 tablet_service.cc:1511] Processing CreateTablet for tablet cd1140068fc8455280934d7a30c4f559 (DEFAULT_TABLE table=test-workload [id=db6f52ac21cb44e38a4318c2fc93ad42]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:24:24.744362 27899 tablet_service.cc:1511] Processing CreateTablet for tablet db487682b03b4a738f664fa1459714e5 (DEFAULT_TABLE table=test-workload [id=db6f52ac21cb44e38a4318c2fc93ad42]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:24:24.745002 27823 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cd1140068fc8455280934d7a30c4f559. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:24:24.745384 27950 tablet_bootstrap.cc:654] T d16bbb66a57e4079897aaa2c4e6ba144 P 9cb4300ea183421ab40709a05de5a877: Neither blocks nor log segments found. Creating new log.
I20260709 13:24:24.745923 27899 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet db487682b03b4a738f664fa1459714e5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:24:24.748384 27898 tablet_service.cc:1511] Processing CreateTablet for tablet f3e47f5448ff47a382c61886f1a8cc7b (DEFAULT_TABLE table=test-workload [id=db6f52ac21cb44e38a4318c2fc93ad42]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:24:24.749876 27898 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f3e47f5448ff47a382c61886f1a8cc7b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:24:24.752338 27897 tablet_service.cc:1511] Processing CreateTablet for tablet e7366c36a1154d4d94281a90fde291f8 (DEFAULT_TABLE table=test-workload [id=db6f52ac21cb44e38a4318c2fc93ad42]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:24:24.753851 27897 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e7366c36a1154d4d94281a90fde291f8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:24:24.756430 27896 tablet_service.cc:1511] Processing CreateTablet for tablet 4554587582094286b577c9c9a4d3a7fc (DEFAULT_TABLE table=test-workload [id=db6f52ac21cb44e38a4318c2fc93ad42]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:24:24.759420 27896 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4554587582094286b577c9c9a4d3a7fc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:24:24.757076 27749 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4554587582094286b577c9c9a4d3a7fc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:24:24.764163 27750 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cd1140068fc8455280934d7a30c4f559. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:24:24.771450 27895 tablet_service.cc:1511] Processing CreateTablet for tablet cd1140068fc8455280934d7a30c4f559 (DEFAULT_TABLE table=test-workload [id=db6f52ac21cb44e38a4318c2fc93ad42]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:24:24.773734 27895 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cd1140068fc8455280934d7a30c4f559. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:24:24.805393 27752 tablet_service.cc:1511] Processing CreateTablet for tablet 2f38443d4b8d4e418c8f0520893a31e4 (DEFAULT_TABLE table=test-workload [id=db6f52ac21cb44e38a4318c2fc93ad42]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:24:24.806864 27752 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2f38443d4b8d4e418c8f0520893a31e4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:24:24.817991 27894 tablet_service.cc:1511] Processing CreateTablet for tablet 2f38443d4b8d4e418c8f0520893a31e4 (DEFAULT_TABLE table=test-workload [id=db6f52ac21cb44e38a4318c2fc93ad42]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:24:24.819927 27825 tablet_service.cc:1511] Processing CreateTablet for tablet 2f38443d4b8d4e418c8f0520893a31e4 (DEFAULT_TABLE table=test-workload [id=db6f52ac21cb44e38a4318c2fc93ad42]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:24:24.820360 27894 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2f38443d4b8d4e418c8f0520893a31e4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:24:24.821282 27825 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2f38443d4b8d4e418c8f0520893a31e4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:24:24.805500 27951 tablet_bootstrap.cc:492] T 20ef9c03fcd94115be0b592d85f89421 P 0788dbdabe574355bb323dcdc3be0058: No bootstrap required, opened a new log
I20260709 13:24:24.823495 27951 ts_tablet_manager.cc:1403] T 20ef9c03fcd94115be0b592d85f89421 P 0788dbdabe574355bb323dcdc3be0058: Time spent bootstrapping tablet: real 0.116s	user 0.023s	sys 0.011s
I20260709 13:24:24.826627 27951 raft_consensus.cc:359] T 20ef9c03fcd94115be0b592d85f89421 P 0788dbdabe574355bb323dcdc3be0058 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb4300ea183421ab40709a05de5a877" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 33597 } } peers { permanent_uuid: "0788dbdabe574355bb323dcdc3be0058" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 41335 } } peers { permanent_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34235 } }
I20260709 13:24:24.827544 27951 raft_consensus.cc:385] T 20ef9c03fcd94115be0b592d85f89421 P 0788dbdabe574355bb323dcdc3be0058 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:24:24.827838 27951 raft_consensus.cc:740] T 20ef9c03fcd94115be0b592d85f89421 P 0788dbdabe574355bb323dcdc3be0058 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0788dbdabe574355bb323dcdc3be0058, State: Initialized, Role: FOLLOWER
I20260709 13:24:24.828644 27951 consensus_queue.cc:260] T 20ef9c03fcd94115be0b592d85f89421 P 0788dbdabe574355bb323dcdc3be0058 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb4300ea183421ab40709a05de5a877" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 33597 } } peers { permanent_uuid: "0788dbdabe574355bb323dcdc3be0058" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 41335 } } peers { permanent_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34235 } }
I20260709 13:24:24.869457 27749 tablet_service.cc:1511] Processing CreateTablet for tablet 028750ef0efa43e4b11ac10f3814902c (DEFAULT_TABLE table=test-workload [id=db6f52ac21cb44e38a4318c2fc93ad42]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:24:24.870880 27749 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 028750ef0efa43e4b11ac10f3814902c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:24:24.871608 27954 tablet_bootstrap.cc:492] T d16bbb66a57e4079897aaa2c4e6ba144 P 880fc4c7dff4417187b75cc1d9e9dcf5: Bootstrap starting.
I20260709 13:24:24.877597 27954 tablet_bootstrap.cc:654] T d16bbb66a57e4079897aaa2c4e6ba144 P 880fc4c7dff4417187b75cc1d9e9dcf5: Neither blocks nor log segments found. Creating new log.
I20260709 13:24:24.877728 27950 tablet_bootstrap.cc:492] T d16bbb66a57e4079897aaa2c4e6ba144 P 9cb4300ea183421ab40709a05de5a877: No bootstrap required, opened a new log
I20260709 13:24:24.878350 27950 ts_tablet_manager.cc:1403] T d16bbb66a57e4079897aaa2c4e6ba144 P 9cb4300ea183421ab40709a05de5a877: Time spent bootstrapping tablet: real 0.183s	user 0.017s	sys 0.023s
I20260709 13:24:24.881450 27950 raft_consensus.cc:359] T d16bbb66a57e4079897aaa2c4e6ba144 P 9cb4300ea183421ab40709a05de5a877 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb4300ea183421ab40709a05de5a877" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 33597 } } peers { permanent_uuid: "0788dbdabe574355bb323dcdc3be0058" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 41335 } } peers { permanent_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34235 } }
I20260709 13:24:24.882182 27950 raft_consensus.cc:385] T d16bbb66a57e4079897aaa2c4e6ba144 P 9cb4300ea183421ab40709a05de5a877 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:24:24.881453 27826 tablet_service.cc:1511] Processing CreateTablet for tablet 028750ef0efa43e4b11ac10f3814902c (DEFAULT_TABLE table=test-workload [id=db6f52ac21cb44e38a4318c2fc93ad42]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:24:24.886543 27826 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 028750ef0efa43e4b11ac10f3814902c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:24:24.882475 27950 raft_consensus.cc:740] T d16bbb66a57e4079897aaa2c4e6ba144 P 9cb4300ea183421ab40709a05de5a877 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9cb4300ea183421ab40709a05de5a877, State: Initialized, Role: FOLLOWER
I20260709 13:24:24.889212 27862 heartbeater.cc:499] Master 127.14.177.190:41127 was elected leader, sending a full tablet report...
I20260709 13:24:24.892094 27895 tablet_service.cc:1511] Processing CreateTablet for tablet 028750ef0efa43e4b11ac10f3814902c (DEFAULT_TABLE table=test-workload [id=db6f52ac21cb44e38a4318c2fc93ad42]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:24:24.893736 27895 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 028750ef0efa43e4b11ac10f3814902c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:24:24.890085 27951 ts_tablet_manager.cc:1434] T 20ef9c03fcd94115be0b592d85f89421 P 0788dbdabe574355bb323dcdc3be0058: Time spent starting tablet: real 0.066s	user 0.015s	sys 0.001s
I20260709 13:24:24.896939 27951 tablet_bootstrap.cc:492] T d16bbb66a57e4079897aaa2c4e6ba144 P 0788dbdabe574355bb323dcdc3be0058: Bootstrap starting.
I20260709 13:24:24.898669 27954 tablet_bootstrap.cc:492] T d16bbb66a57e4079897aaa2c4e6ba144 P 880fc4c7dff4417187b75cc1d9e9dcf5: No bootstrap required, opened a new log
I20260709 13:24:24.899247 27954 ts_tablet_manager.cc:1403] T d16bbb66a57e4079897aaa2c4e6ba144 P 880fc4c7dff4417187b75cc1d9e9dcf5: Time spent bootstrapping tablet: real 0.028s	user 0.015s	sys 0.007s
I20260709 13:24:24.902275 27954 raft_consensus.cc:359] T d16bbb66a57e4079897aaa2c4e6ba144 P 880fc4c7dff4417187b75cc1d9e9dcf5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb4300ea183421ab40709a05de5a877" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 33597 } } peers { permanent_uuid: "0788dbdabe574355bb323dcdc3be0058" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 41335 } } peers { permanent_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34235 } }
I20260709 13:24:24.903314 27955 raft_consensus.cc:493] T 20ef9c03fcd94115be0b592d85f89421 P 0788dbdabe574355bb323dcdc3be0058 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:24:24.905723 27951 tablet_bootstrap.cc:654] T d16bbb66a57e4079897aaa2c4e6ba144 P 0788dbdabe574355bb323dcdc3be0058: Neither blocks nor log segments found. Creating new log.
I20260709 13:24:24.911283 27955 raft_consensus.cc:515] T 20ef9c03fcd94115be0b592d85f89421 P 0788dbdabe574355bb323dcdc3be0058 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb4300ea183421ab40709a05de5a877" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 33597 } } peers { permanent_uuid: "0788dbdabe574355bb323dcdc3be0058" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 41335 } } peers { permanent_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34235 } }
I20260709 13:24:24.911913 27954 raft_consensus.cc:385] T d16bbb66a57e4079897aaa2c4e6ba144 P 880fc4c7dff4417187b75cc1d9e9dcf5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:24:24.912269 27954 raft_consensus.cc:740] T d16bbb66a57e4079897aaa2c4e6ba144 P 880fc4c7dff4417187b75cc1d9e9dcf5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 880fc4c7dff4417187b75cc1d9e9dcf5, State: Initialized, Role: FOLLOWER
I20260709 13:24:24.913084 27954 consensus_queue.cc:260] T d16bbb66a57e4079897aaa2c4e6ba144 P 880fc4c7dff4417187b75cc1d9e9dcf5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb4300ea183421ab40709a05de5a877" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 33597 } } peers { permanent_uuid: "0788dbdabe574355bb323dcdc3be0058" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 41335 } } peers { permanent_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34235 } }
I20260709 13:24:24.907827 27950 consensus_queue.cc:260] T d16bbb66a57e4079897aaa2c4e6ba144 P 9cb4300ea183421ab40709a05de5a877 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb4300ea183421ab40709a05de5a877" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 33597 } } peers { permanent_uuid: "0788dbdabe574355bb323dcdc3be0058" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 41335 } } peers { permanent_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34235 } }
I20260709 13:24:24.918234 27787 heartbeater.cc:499] Master 127.14.177.190:41127 was elected leader, sending a full tablet report...
I20260709 13:24:24.924538 27950 ts_tablet_manager.cc:1434] T d16bbb66a57e4079897aaa2c4e6ba144 P 9cb4300ea183421ab40709a05de5a877: Time spent starting tablet: real 0.046s	user 0.006s	sys 0.004s
I20260709 13:24:24.925752 27950 tablet_bootstrap.cc:492] T db487682b03b4a738f664fa1459714e5 P 9cb4300ea183421ab40709a05de5a877: Bootstrap starting.
I20260709 13:24:24.931048 27955 leader_election.cc:290] T 20ef9c03fcd94115be0b592d85f89421 P 0788dbdabe574355bb323dcdc3be0058 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9cb4300ea183421ab40709a05de5a877 (127.14.177.129:33597), 880fc4c7dff4417187b75cc1d9e9dcf5 (127.14.177.131:34235)
I20260709 13:24:24.933458 27950 tablet_bootstrap.cc:654] T db487682b03b4a738f664fa1459714e5 P 9cb4300ea183421ab40709a05de5a877: Neither blocks nor log segments found. Creating new log.
I20260709 13:24:24.938887 27911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20ef9c03fcd94115be0b592d85f89421" candidate_uuid: "0788dbdabe574355bb323dcdc3be0058" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" is_pre_election: true
I20260709 13:24:24.939884 27954 ts_tablet_manager.cc:1434] T d16bbb66a57e4079897aaa2c4e6ba144 P 880fc4c7dff4417187b75cc1d9e9dcf5: Time spent starting tablet: real 0.040s	user 0.021s	sys 0.002s
I20260709 13:24:24.939905 27951 tablet_bootstrap.cc:492] T d16bbb66a57e4079897aaa2c4e6ba144 P 0788dbdabe574355bb323dcdc3be0058: No bootstrap required, opened a new log
I20260709 13:24:24.940703 27951 ts_tablet_manager.cc:1403] T d16bbb66a57e4079897aaa2c4e6ba144 P 0788dbdabe574355bb323dcdc3be0058: Time spent bootstrapping tablet: real 0.044s	user 0.011s	sys 0.003s
W20260709 13:24:24.940811 27802 leader_election.cc:343] T 20ef9c03fcd94115be0b592d85f89421 P 0788dbdabe574355bb323dcdc3be0058 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 880fc4c7dff4417187b75cc1d9e9dcf5 (127.14.177.131:34235): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:24:24.941318 27936 heartbeater.cc:499] Master 127.14.177.190:41127 was elected leader, sending a full tablet report...
I20260709 13:24:24.944329 27954 tablet_bootstrap.cc:492] T db487682b03b4a738f664fa1459714e5 P 880fc4c7dff4417187b75cc1d9e9dcf5: Bootstrap starting.
I20260709 13:24:24.944216 27951 raft_consensus.cc:359] T d16bbb66a57e4079897aaa2c4e6ba144 P 0788dbdabe574355bb323dcdc3be0058 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb4300ea183421ab40709a05de5a877" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 33597 } } peers { permanent_uuid: "0788dbdabe574355bb323dcdc3be0058" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 41335 } } peers { permanent_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34235 } }
I20260709 13:24:24.946712 27951 raft_consensus.cc:385] T d16bbb66a57e4079897aaa2c4e6ba144 P 0788dbdabe574355bb323dcdc3be0058 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:24:24.947085 27951 raft_consensus.cc:740] T d16bbb66a57e4079897aaa2c4e6ba144 P 0788dbdabe574355bb323dcdc3be0058 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0788dbdabe574355bb323dcdc3be0058, State: Initialized, Role: FOLLOWER
I20260709 13:24:24.948050 27951 consensus_queue.cc:260] T d16bbb66a57e4079897aaa2c4e6ba144 P 0788dbdabe574355bb323dcdc3be0058 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb4300ea183421ab40709a05de5a877" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 33597 } } peers { permanent_uuid: "0788dbdabe574355bb323dcdc3be0058" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 41335 } } peers { permanent_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34235 } }
I20260709 13:24:24.950573 27951 ts_tablet_manager.cc:1434] T d16bbb66a57e4079897aaa2c4e6ba144 P 0788dbdabe574355bb323dcdc3be0058: Time spent starting tablet: real 0.009s	user 0.005s	sys 0.001s
I20260709 13:24:24.951812 27951 tablet_bootstrap.cc:492] T f3e47f5448ff47a382c61886f1a8cc7b P 0788dbdabe574355bb323dcdc3be0058: Bootstrap starting.
I20260709 13:24:24.953629 27954 tablet_bootstrap.cc:654] T db487682b03b4a738f664fa1459714e5 P 880fc4c7dff4417187b75cc1d9e9dcf5: Neither blocks nor log segments found. Creating new log.
I20260709 13:24:24.960640 27954 tablet_bootstrap.cc:492] T db487682b03b4a738f664fa1459714e5 P 880fc4c7dff4417187b75cc1d9e9dcf5: No bootstrap required, opened a new log
I20260709 13:24:24.961210 27951 tablet_bootstrap.cc:654] T f3e47f5448ff47a382c61886f1a8cc7b P 0788dbdabe574355bb323dcdc3be0058: Neither blocks nor log segments found. Creating new log.
I20260709 13:24:24.961444 27954 ts_tablet_manager.cc:1403] T db487682b03b4a738f664fa1459714e5 P 880fc4c7dff4417187b75cc1d9e9dcf5: Time spent bootstrapping tablet: real 0.017s	user 0.013s	sys 0.000s
I20260709 13:24:24.965281 27954 raft_consensus.cc:359] T db487682b03b4a738f664fa1459714e5 P 880fc4c7dff4417187b75cc1d9e9dcf5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb4300ea183421ab40709a05de5a877" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 33597 } } peers { permanent_uuid: "0788dbdabe574355bb323dcdc3be0058" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 41335 } } peers { permanent_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34235 } }
I20260709 13:24:24.966219 27954 raft_consensus.cc:385] T db487682b03b4a738f664fa1459714e5 P 880fc4c7dff4417187b75cc1d9e9dcf5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:24:24.966615 27954 raft_consensus.cc:740] T db487682b03b4a738f664fa1459714e5 P 880fc4c7dff4417187b75cc1d9e9dcf5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 880fc4c7dff4417187b75cc1d9e9dcf5, State: Initialized, Role: FOLLOWER
I20260709 13:24:24.967288 27950 tablet_bootstrap.cc:492] T db487682b03b4a738f664fa1459714e5 P 9cb4300ea183421ab40709a05de5a877: No bootstrap required, opened a new log
I20260709 13:24:24.967908 27950 ts_tablet_manager.cc:1403] T db487682b03b4a738f664fa1459714e5 P 9cb4300ea183421ab40709a05de5a877: Time spent bootstrapping tablet: real 0.042s	user 0.012s	sys 0.006s
I20260709 13:24:24.967499 27954 consensus_queue.cc:260] T db487682b03b4a738f664fa1459714e5 P 880fc4c7dff4417187b75cc1d9e9dcf5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb4300ea183421ab40709a05de5a877" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 33597 } } peers { permanent_uuid: "0788dbdabe574355bb323dcdc3be0058" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 41335 } } peers { permanent_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34235 } }
I20260709 13:24:24.970245 27954 ts_tablet_manager.cc:1434] T db487682b03b4a738f664fa1459714e5 P 880fc4c7dff4417187b75cc1d9e9dcf5: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 13:24:24.970067 27762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20ef9c03fcd94115be0b592d85f89421" candidate_uuid: "0788dbdabe574355bb323dcdc3be0058" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9cb4300ea183421ab40709a05de5a877" is_pre_election: true
I20260709 13:24:24.971465 27954 tablet_bootstrap.cc:492] T 4554587582094286b577c9c9a4d3a7fc P 880fc4c7dff4417187b75cc1d9e9dcf5: Bootstrap starting.
W20260709 13:24:24.971896 27800 leader_election.cc:343] T 20ef9c03fcd94115be0b592d85f89421 P 0788dbdabe574355bb323dcdc3be0058 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9cb4300ea183421ab40709a05de5a877 (127.14.177.129:33597): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:24:24.972504 27800 leader_election.cc:304] T 20ef9c03fcd94115be0b592d85f89421 P 0788dbdabe574355bb323dcdc3be0058 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0788dbdabe574355bb323dcdc3be0058; no voters: 880fc4c7dff4417187b75cc1d9e9dcf5, 9cb4300ea183421ab40709a05de5a877
I20260709 13:24:24.973887 27955 raft_consensus.cc:2749] T 20ef9c03fcd94115be0b592d85f89421 P 0788dbdabe574355bb323dcdc3be0058 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:24:24.976562 27950 raft_consensus.cc:359] T db487682b03b4a738f664fa1459714e5 P 9cb4300ea183421ab40709a05de5a877 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb4300ea183421ab40709a05de5a877" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 33597 } } peers { permanent_uuid: "0788dbdabe574355bb323dcdc3be0058" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 41335 } } peers { permanent_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34235 } }
I20260709 13:24:24.978662 27950 raft_consensus.cc:385] T db487682b03b4a738f664fa1459714e5 P 9cb4300ea183421ab40709a05de5a877 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:24:24.979000 27950 raft_consensus.cc:740] T db487682b03b4a738f664fa1459714e5 P 9cb4300ea183421ab40709a05de5a877 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9cb4300ea183421ab40709a05de5a877, State: Initialized, Role: FOLLOWER
I20260709 13:24:24.979797 27954 tablet_bootstrap.cc:654] T 4554587582094286b577c9c9a4d3a7fc P 880fc4c7dff4417187b75cc1d9e9dcf5: Neither blocks nor log segments found. Creating new log.
I20260709 13:24:24.980120 27950 consensus_queue.cc:260] T db487682b03b4a738f664fa1459714e5 P 9cb4300ea183421ab40709a05de5a877 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb4300ea183421ab40709a05de5a877" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 33597 } } peers { permanent_uuid: "0788dbdabe574355bb323dcdc3be0058" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 41335 } } peers { permanent_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34235 } }
I20260709 13:24:24.985036 27950 ts_tablet_manager.cc:1434] T db487682b03b4a738f664fa1459714e5 P 9cb4300ea183421ab40709a05de5a877: Time spent starting tablet: real 0.017s	user 0.008s	sys 0.000s
I20260709 13:24:24.985975 27950 tablet_bootstrap.cc:492] T 20ef9c03fcd94115be0b592d85f89421 P 9cb4300ea183421ab40709a05de5a877: Bootstrap starting.
I20260709 13:24:24.989624 27954 tablet_bootstrap.cc:492] T 4554587582094286b577c9c9a4d3a7fc P 880fc4c7dff4417187b75cc1d9e9dcf5: No bootstrap required, opened a new log
I20260709 13:24:24.990206 27954 ts_tablet_manager.cc:1403] T 4554587582094286b577c9c9a4d3a7fc P 880fc4c7dff4417187b75cc1d9e9dcf5: Time spent bootstrapping tablet: real 0.019s	user 0.009s	sys 0.003s
I20260709 13:24:24.992648 27950 tablet_bootstrap.cc:654] T 20ef9c03fcd94115be0b592d85f89421 P 9cb4300ea183421ab40709a05de5a877: Neither blocks nor log segments found. Creating new log.
I20260709 13:24:24.993199 27954 raft_consensus.cc:359] T 4554587582094286b577c9c9a4d3a7fc P 880fc4c7dff4417187b75cc1d9e9dcf5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb4300ea183421ab40709a05de5a877" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 33597 } } peers { permanent_uuid: "0788dbdabe574355bb323dcdc3be0058" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 41335 } } peers { permanent_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34235 } }
I20260709 13:24:24.993806 27951 tablet_bootstrap.cc:492] T f3e47f5448ff47a382c61886f1a8cc7b P 0788dbdabe574355bb323dcdc3be0058: No bootstrap required, opened a new log
I20260709 13:24:24.994061 27954 raft_consensus.cc:385] T 4554587582094286b577c9c9a4d3a7fc P 880fc4c7dff4417187b75cc1d9e9dcf5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:24:24.994496 27954 raft_consensus.cc:740] T 4554587582094286b577c9c9a4d3a7fc P 880fc4c7dff4417187b75cc1d9e9dcf5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 880fc4c7dff4417187b75cc1d9e9dcf5, State: Initialized, Role: FOLLOWER
I20260709 13:24:24.994697 27951 ts_tablet_manager.cc:1403] T f3e47f5448ff47a382c61886f1a8cc7b P 0788dbdabe574355bb323dcdc3be0058: Time spent bootstrapping tablet: real 0.043s	user 0.014s	sys 0.015s
I20260709 13:24:24.995348 27954 consensus_queue.cc:260] T 4554587582094286b577c9c9a4d3a7fc P 880fc4c7dff4417187b75cc1d9e9dcf5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb4300ea183421ab40709a05de5a877" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 33597 } } peers { permanent_uuid: "0788dbdabe574355bb323dcdc3be0058" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 41335 } } peers { permanent_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34235 } }
I20260709 13:24:24.997628 27954 ts_tablet_manager.cc:1434] T 4554587582094286b577c9c9a4d3a7fc P 880fc4c7dff4417187b75cc1d9e9dcf5: Time spent starting tablet: real 0.007s	user 0.003s	sys 0.004s
I20260709 13:24:24.998687 27954 tablet_bootstrap.cc:492] T e7366c36a1154d4d94281a90fde291f8 P 880fc4c7dff4417187b75cc1d9e9dcf5: Bootstrap starting.
I20260709 13:24:25.001503 27951 raft_consensus.cc:359] T f3e47f5448ff47a382c61886f1a8cc7b P 0788dbdabe574355bb323dcdc3be0058 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb4300ea183421ab40709a05de5a877" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 33597 } } peers { permanent_uuid: "0788dbdabe574355bb323dcdc3be0058" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 41335 } } peers { permanent_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34235 } }
I20260709 13:24:25.005398 27951 raft_consensus.cc:385] T f3e47f5448ff47a382c61886f1a8cc7b P 0788dbdabe574355bb323dcdc3be0058 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:24:25.006479 27951 raft_consensus.cc:740] T f3e47f5448ff47a382c61886f1a8cc7b P 0788dbdabe574355bb323dcdc3be0058 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0788dbdabe574355bb323dcdc3be0058, State: Initialized, Role: FOLLOWER
I20260709 13:24:25.007586 27951 consensus_queue.cc:260] T f3e47f5448ff47a382c61886f1a8cc7b P 0788dbdabe574355bb323dcdc3be0058 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb4300ea183421ab40709a05de5a877" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 33597 } } peers { permanent_uuid: "0788dbdabe574355bb323dcdc3be0058" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 41335 } } peers { permanent_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34235 } }
I20260709 13:24:25.007915 27954 tablet_bootstrap.cc:654] T e7366c36a1154d4d94281a90fde291f8 P 880fc4c7dff4417187b75cc1d9e9dcf5: Neither blocks nor log segments found. Creating new log.
I20260709 13:24:25.010886 27951 ts_tablet_manager.cc:1434] T f3e47f5448ff47a382c61886f1a8cc7b P 0788dbdabe574355bb323dcdc3be0058: Time spent starting tablet: real 0.016s	user 0.006s	sys 0.001s
I20260709 13:24:25.012226 27951 tablet_bootstrap.cc:492] T 4554587582094286b577c9c9a4d3a7fc P 0788dbdabe574355bb323dcdc3be0058: Bootstrap starting.
I20260709 13:24:25.014716 27954 tablet_bootstrap.cc:492] T e7366c36a1154d4d94281a90fde291f8 P 880fc4c7dff4417187b75cc1d9e9dcf5: No bootstrap required, opened a new log
I20260709 13:24:25.015465 27954 ts_tablet_manager.cc:1403] T e7366c36a1154d4d94281a90fde291f8 P 880fc4c7dff4417187b75cc1d9e9dcf5: Time spent bootstrapping tablet: real 0.017s	user 0.015s	sys 0.000s
I20260709 13:24:25.018615 27954 raft_consensus.cc:359] T e7366c36a1154d4d94281a90fde291f8 P 880fc4c7dff4417187b75cc1d9e9dcf5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb4300ea183421ab40709a05de5a877" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 33597 } } peers { permanent_uuid: "0788dbdabe574355bb323dcdc3be0058" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 41335 } } peers { permanent_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34235 } }
I20260709 13:24:25.019457 27954 raft_consensus.cc:385] T e7366c36a1154d4d94281a90fde291f8 P 880fc4c7dff4417187b75cc1d9e9dcf5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:24:25.019815 27954 raft_consensus.cc:740] T e7366c36a1154d4d94281a90fde291f8 P 880fc4c7dff4417187b75cc1d9e9dcf5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 880fc4c7dff4417187b75cc1d9e9dcf5, State: Initialized, Role: FOLLOWER
I20260709 13:24:25.020476 27951 tablet_bootstrap.cc:654] T 4554587582094286b577c9c9a4d3a7fc P 0788dbdabe574355bb323dcdc3be0058: Neither blocks nor log segments found. Creating new log.
I20260709 13:24:25.020591 27954 consensus_queue.cc:260] T e7366c36a1154d4d94281a90fde291f8 P 880fc4c7dff4417187b75cc1d9e9dcf5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb4300ea183421ab40709a05de5a877" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 33597 } } peers { permanent_uuid: "0788dbdabe574355bb323dcdc3be0058" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 41335 } } peers { permanent_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34235 } }
I20260709 13:24:25.023046 27954 ts_tablet_manager.cc:1434] T e7366c36a1154d4d94281a90fde291f8 P 880fc4c7dff4417187b75cc1d9e9dcf5: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:24:25.024168 27954 tablet_bootstrap.cc:492] T cd1140068fc8455280934d7a30c4f559 P 880fc4c7dff4417187b75cc1d9e9dcf5: Bootstrap starting.
I20260709 13:24:25.027360 27951 tablet_bootstrap.cc:492] T 4554587582094286b577c9c9a4d3a7fc P 0788dbdabe574355bb323dcdc3be0058: No bootstrap required, opened a new log
I20260709 13:24:25.028019 27951 ts_tablet_manager.cc:1403] T 4554587582094286b577c9c9a4d3a7fc P 0788dbdabe574355bb323dcdc3be0058: Time spent bootstrapping tablet: real 0.016s	user 0.013s	sys 0.001s
I20260709 13:24:25.031339 27954 tablet_bootstrap.cc:654] T cd1140068fc8455280934d7a30c4f559 P 880fc4c7dff4417187b75cc1d9e9dcf5: Neither blocks nor log segments found. Creating new log.
I20260709 13:24:25.031631 27951 raft_consensus.cc:359] T 4554587582094286b577c9c9a4d3a7fc P 0788dbdabe574355bb323dcdc3be0058 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb4300ea183421ab40709a05de5a877" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 33597 } } peers { permanent_uuid: "0788dbdabe574355bb323dcdc3be0058" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 41335 } } peers { permanent_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34235 } }
I20260709 13:24:25.032639 27951 raft_consensus.cc:385] T 4554587582094286b577c9c9a4d3a7fc P 0788dbdabe574355bb323dcdc3be0058 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:24:25.033020 27951 raft_consensus.cc:740] T 4554587582094286b577c9c9a4d3a7fc P 0788dbdabe574355bb323dcdc3be0058 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0788dbdabe574355bb323dcdc3be0058, State: Initialized, Role: FOLLOWER
I20260709 13:24:25.033968 27951 consensus_queue.cc:260] T 4554587582094286b577c9c9a4d3a7fc P 0788dbdabe574355bb323dcdc3be0058 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb4300ea183421ab40709a05de5a877" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 33597 } } peers { permanent_uuid: "0788dbdabe574355bb323dcdc3be0058" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 41335 } } peers { permanent_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34235 } }
I20260709 13:24:25.036586 27951 ts_tablet_manager.cc:1434] T 4554587582094286b577c9c9a4d3a7fc P 0788dbdabe574355bb323dcdc3be0058: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 13:24:25.037102 27950 tablet_bootstrap.cc:492] T 20ef9c03fcd94115be0b592d85f89421 P 9cb4300ea183421ab40709a05de5a877: No bootstrap required, opened a new log
I20260709 13:24:25.037698 27950 ts_tablet_manager.cc:1403] T 20ef9c03fcd94115be0b592d85f89421 P 9cb4300ea183421ab40709a05de5a877: Time spent bootstrapping tablet: real 0.052s	user 0.009s	sys 0.008s
I20260709 13:24:25.038509 27951 tablet_bootstrap.cc:492] T db487682b03b4a738f664fa1459714e5 P 0788dbdabe574355bb323dcdc3be0058: Bootstrap starting.
I20260709 13:24:25.039815 27959 raft_consensus.cc:493] T db487682b03b4a738f664fa1459714e5 P 9cb4300ea183421ab40709a05de5a877 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:24:25.040400 27959 raft_consensus.cc:515] T db487682b03b4a738f664fa1459714e5 P 9cb4300ea183421ab40709a05de5a877 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb4300ea183421ab40709a05de5a877" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 33597 } } peers { permanent_uuid: "0788dbdabe574355bb323dcdc3be0058" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 41335 } } peers { permanent_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34235 } }
I20260709 13:24:25.046141 27959 leader_election.cc:290] T db487682b03b4a738f664fa1459714e5 P 9cb4300ea183421ab40709a05de5a877 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0788dbdabe574355bb323dcdc3be0058 (127.14.177.130:41335), 880fc4c7dff4417187b75cc1d9e9dcf5 (127.14.177.131:34235)
I20260709 13:24:25.050112 27950 raft_consensus.cc:359] T 20ef9c03fcd94115be0b592d85f89421 P 9cb4300ea183421ab40709a05de5a877 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb4300ea183421ab40709a05de5a877" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 33597 } } peers { permanent_uuid: "0788dbdabe574355bb323dcdc3be0058" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 41335 } } peers { permanent_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34235 } }
I20260709 13:24:25.050944 27950 raft_consensus.cc:385] T 20ef9c03fcd94115be0b592d85f89421 P 9cb4300ea183421ab40709a05de5a877 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:24:25.051286 27950 raft_consensus.cc:740] T 20ef9c03fcd94115be0b592d85f89421 P 9cb4300ea183421ab40709a05de5a877 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9cb4300ea183421ab40709a05de5a877, State: Initialized, Role: FOLLOWER
I20260709 13:24:25.051787 27951 tablet_bootstrap.cc:654] T db487682b03b4a738f664fa1459714e5 P 0788dbdabe574355bb323dcdc3be0058: Neither blocks nor log segments found. Creating new log.
I20260709 13:24:25.052181 27950 consensus_queue.cc:260] T 20ef9c03fcd94115be0b592d85f89421 P 9cb4300ea183421ab40709a05de5a877 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb4300ea183421ab40709a05de5a877" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 33597 } } peers { permanent_uuid: "0788dbdabe574355bb323dcdc3be0058" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 41335 } } peers { permanent_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34235 } }
I20260709 13:24:25.054839 27950 ts_tablet_manager.cc:1434] T 20ef9c03fcd94115be0b592d85f89421 P 9cb4300ea183421ab40709a05de5a877: Time spent starting tablet: real 0.017s	user 0.001s	sys 0.006s
I20260709 13:24:25.073387 27954 tablet_bootstrap.cc:492] T cd1140068fc8455280934d7a30c4f559 P 880fc4c7dff4417187b75cc1d9e9dcf5: No bootstrap required, opened a new log
I20260709 13:24:25.074014 27954 ts_tablet_manager.cc:1403] T cd1140068fc8455280934d7a30c4f559 P 880fc4c7dff4417187b75cc1d9e9dcf5: Time spent bootstrapping tablet: real 0.050s	user 0.019s	sys 0.018s
I20260709 13:24:25.076865 27951 tablet_bootstrap.cc:492] T db487682b03b4a738f664fa1459714e5 P 0788dbdabe574355bb323dcdc3be0058: No bootstrap required, opened a new log
I20260709 13:24:25.077557 27951 ts_tablet_manager.cc:1403] T db487682b03b4a738f664fa1459714e5 P 0788dbdabe574355bb323dcdc3be0058: Time spent bootstrapping tablet: real 0.039s	user 0.007s	sys 0.011s
I20260709 13:24:25.077608 27954 raft_consensus.cc:359] T cd1140068fc8455280934d7a30c4f559 P 880fc4c7dff4417187b75cc1d9e9dcf5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb4300ea183421ab40709a05de5a877" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 33597 } } peers { permanent_uuid: "0788dbdabe574355bb323dcdc3be0058" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 41335 } } peers { permanent_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34235 } }
I20260709 13:24:25.078732 27954 raft_consensus.cc:385] T cd1140068fc8455280934d7a30c4f559 P 880fc4c7dff4417187b75cc1d9e9dcf5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:24:25.079043 27954 raft_consensus.cc:740] T cd1140068fc8455280934d7a30c4f559 P 880fc4c7dff4417187b75cc1d9e9dcf5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 880fc4c7dff4417187b75cc1d9e9dcf5, State: Initialized, Role: FOLLOWER
I20260709 13:24:25.080065 27954 consensus_queue.cc:260] T cd1140068fc8455280934d7a30c4f559 P 880fc4c7dff4417187b75cc1d9e9dcf5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb4300ea183421ab40709a05de5a877" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 33597 } } peers { permanent_uuid: "0788dbdabe574355bb323dcdc3be0058" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 41335 } } peers { permanent_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34235 } }
I20260709 13:24:25.081077 27951 raft_consensus.cc:359] T db487682b03b4a738f664fa1459714e5 P 0788dbdabe574355bb323dcdc3be0058 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb4300ea183421ab40709a05de5a877" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 33597 } } peers { permanent_uuid: "0788dbdabe574355bb323dcdc3be0058" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 41335 } } peers { permanent_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34235 } }
I20260709 13:24:25.081939 27951 raft_consensus.cc:385] T db487682b03b4a738f664fa1459714e5 P 0788dbdabe574355bb323dcdc3be0058 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:24:25.082257 27951 raft_consensus.cc:740] T db487682b03b4a738f664fa1459714e5 P 0788dbdabe574355bb323dcdc3be0058 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0788dbdabe574355bb323dcdc3be0058, State: Initialized, Role: FOLLOWER
I20260709 13:24:25.083827 27954 ts_tablet_manager.cc:1434] T cd1140068fc8455280934d7a30c4f559 P 880fc4c7dff4417187b75cc1d9e9dcf5: Time spent starting tablet: real 0.009s	user 0.009s	sys 0.001s
I20260709 13:24:25.083717 27951 consensus_queue.cc:260] T db487682b03b4a738f664fa1459714e5 P 0788dbdabe574355bb323dcdc3be0058 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb4300ea183421ab40709a05de5a877" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 33597 } } peers { permanent_uuid: "0788dbdabe574355bb323dcdc3be0058" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 41335 } } peers { permanent_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34235 } }
I20260709 13:24:25.085059 27954 tablet_bootstrap.cc:492] T f3e47f5448ff47a382c61886f1a8cc7b P 880fc4c7dff4417187b75cc1d9e9dcf5: Bootstrap starting.
I20260709 13:24:25.084148 27837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db487682b03b4a738f664fa1459714e5" candidate_uuid: "9cb4300ea183421ab40709a05de5a877" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0788dbdabe574355bb323dcdc3be0058" is_pre_election: true
I20260709 13:24:25.085897 27837 raft_consensus.cc:2468] T db487682b03b4a738f664fa1459714e5 P 0788dbdabe574355bb323dcdc3be0058 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9cb4300ea183421ab40709a05de5a877 in term 0.
I20260709 13:24:25.086928 27951 ts_tablet_manager.cc:1434] T db487682b03b4a738f664fa1459714e5 P 0788dbdabe574355bb323dcdc3be0058: Time spent starting tablet: real 0.009s	user 0.009s	sys 0.001s
I20260709 13:24:25.087433 27727 leader_election.cc:304] T db487682b03b4a738f664fa1459714e5 P 9cb4300ea183421ab40709a05de5a877 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0788dbdabe574355bb323dcdc3be0058, 9cb4300ea183421ab40709a05de5a877; no voters: 
I20260709 13:24:25.088061 27951 tablet_bootstrap.cc:492] T cd1140068fc8455280934d7a30c4f559 P 0788dbdabe574355bb323dcdc3be0058: Bootstrap starting.
I20260709 13:24:25.088280 27950 tablet_bootstrap.cc:492] T f3e47f5448ff47a382c61886f1a8cc7b P 9cb4300ea183421ab40709a05de5a877: Bootstrap starting.
I20260709 13:24:25.088400 27959 raft_consensus.cc:2804] T db487682b03b4a738f664fa1459714e5 P 9cb4300ea183421ab40709a05de5a877 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:24:25.090939 27959 raft_consensus.cc:493] T db487682b03b4a738f664fa1459714e5 P 9cb4300ea183421ab40709a05de5a877 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:24:25.091315 27959 raft_consensus.cc:3060] T db487682b03b4a738f664fa1459714e5 P 9cb4300ea183421ab40709a05de5a877 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:24:25.095999 27951 tablet_bootstrap.cc:654] T cd1140068fc8455280934d7a30c4f559 P 0788dbdabe574355bb323dcdc3be0058: Neither blocks nor log segments found. Creating new log.
I20260709 13:24:25.096028 27950 tablet_bootstrap.cc:654] T f3e47f5448ff47a382c61886f1a8cc7b P 9cb4300ea183421ab40709a05de5a877: Neither blocks nor log segments found. Creating new log.
I20260709 13:24:25.099467 27911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db487682b03b4a738f664fa1459714e5" candidate_uuid: "9cb4300ea183421ab40709a05de5a877" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" is_pre_election: true
I20260709 13:24:25.107288 27954 tablet_bootstrap.cc:654] T f3e47f5448ff47a382c61886f1a8cc7b P 880fc4c7dff4417187b75cc1d9e9dcf5: Neither blocks nor log segments found. Creating new log.
I20260709 13:24:25.107460 27911 raft_consensus.cc:2468] T db487682b03b4a738f664fa1459714e5 P 880fc4c7dff4417187b75cc1d9e9dcf5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9cb4300ea183421ab40709a05de5a877 in term 0.
I20260709 13:24:25.106962 27959 raft_consensus.cc:515] T db487682b03b4a738f664fa1459714e5 P 9cb4300ea183421ab40709a05de5a877 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb4300ea183421ab40709a05de5a877" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 33597 } } peers { permanent_uuid: "0788dbdabe574355bb323dcdc3be0058" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 41335 } } peers { permanent_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34235 } }
I20260709 13:24:25.112471 27837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db487682b03b4a738f664fa1459714e5" candidate_uuid: "9cb4300ea183421ab40709a05de5a877" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0788dbdabe574355bb323dcdc3be0058"
I20260709 13:24:25.113458 27837 raft_consensus.cc:3060] T db487682b03b4a738f664fa1459714e5 P 0788dbdabe574355bb323dcdc3be0058 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:24:25.125284 27708 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:24:25.127674 27951 tablet_bootstrap.cc:492] T cd1140068fc8455280934d7a30c4f559 P 0788dbdabe574355bb323dcdc3be0058: No bootstrap required, opened a new log
I20260709 13:24:25.128459 27951 ts_tablet_manager.cc:1403] T cd1140068fc8455280934d7a30c4f559 P 0788dbdabe574355bb323dcdc3be0058: Time spent bootstrapping tablet: real 0.041s	user 0.010s	sys 0.014s
I20260709 13:24:25.129009 27911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db487682b03b4a738f664fa1459714e5" candidate_uuid: "9cb4300ea183421ab40709a05de5a877" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5"
I20260709 13:24:25.129976 27911 raft_consensus.cc:3060] T db487682b03b4a738f664fa1459714e5 P 880fc4c7dff4417187b75cc1d9e9dcf5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:24:25.132723 27951 raft_consensus.cc:359] T cd1140068fc8455280934d7a30c4f559 P 0788dbdabe574355bb323dcdc3be0058 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb4300ea183421ab40709a05de5a877" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 33597 } } peers { permanent_uuid: "0788dbdabe574355bb323dcdc3be0058" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 41335 } } peers { permanent_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34235 } }
I20260709 13:24:25.133846 27951 raft_consensus.cc:385] T cd1140068fc8455280934d7a30c4f559 P 0788dbdabe574355bb323dcdc3be0058 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:24:25.134384 27951 raft_consensus.cc:740] T cd1140068fc8455280934d7a30c4f559 P 0788dbdabe574355bb323dcdc3be0058 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0788dbdabe574355bb323dcdc3be0058, State: Initialized, Role: FOLLOWER
I20260709 13:24:25.135613 27958 raft_consensus.cc:493] T 4554587582094286b577c9c9a4d3a7fc P 880fc4c7dff4417187b75cc1d9e9dcf5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:24:25.135651 27951 consensus_queue.cc:260] T cd1140068fc8455280934d7a30c4f559 P 0788dbdabe574355bb323dcdc3be0058 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb4300ea183421ab40709a05de5a877" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 33597 } } peers { permanent_uuid: "0788dbdabe574355bb323dcdc3be0058" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 41335 } } peers { permanent_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34235 } }
I20260709 13:24:25.136341 27958 raft_consensus.cc:515] T 4554587582094286b577c9c9a4d3a7fc P 880fc4c7dff4417187b75cc1d9e9dcf5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb4300ea183421ab40709a05de5a877" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 33597 } } peers { permanent_uuid: "0788dbdabe574355bb323dcdc3be0058" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 41335 } } peers { permanent_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34235 } }
I20260709 13:24:25.138345 27951 ts_tablet_manager.cc:1434] T cd1140068fc8455280934d7a30c4f559 P 0788dbdabe574355bb323dcdc3be0058: Time spent starting tablet: real 0.009s	user 0.008s	sys 0.000s
I20260709 13:24:25.139467 27951 tablet_bootstrap.cc:492] T 2f38443d4b8d4e418c8f0520893a31e4 P 0788dbdabe574355bb323dcdc3be0058: Bootstrap starting.
I20260709 13:24:25.143306 27958 leader_election.cc:290] T 4554587582094286b577c9c9a4d3a7fc P 880fc4c7dff4417187b75cc1d9e9dcf5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9cb4300ea183421ab40709a05de5a877 (127.14.177.129:33597), 0788dbdabe574355bb323dcdc3be0058 (127.14.177.130:41335)
I20260709 13:24:25.147285 27959 leader_election.cc:290] T db487682b03b4a738f664fa1459714e5 P 9cb4300ea183421ab40709a05de5a877 [CANDIDATE]: Term 1 election: Requested vote from peers 0788dbdabe574355bb323dcdc3be0058 (127.14.177.130:41335), 880fc4c7dff4417187b75cc1d9e9dcf5 (127.14.177.131:34235)
I20260709 13:24:25.148104 27951 tablet_bootstrap.cc:654] T 2f38443d4b8d4e418c8f0520893a31e4 P 0788dbdabe574355bb323dcdc3be0058: Neither blocks nor log segments found. Creating new log.
I20260709 13:24:25.157797 27911 raft_consensus.cc:2468] T db487682b03b4a738f664fa1459714e5 P 880fc4c7dff4417187b75cc1d9e9dcf5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9cb4300ea183421ab40709a05de5a877 in term 1.
I20260709 13:24:25.162631 27727 leader_election.cc:304] T db487682b03b4a738f664fa1459714e5 P 9cb4300ea183421ab40709a05de5a877 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 880fc4c7dff4417187b75cc1d9e9dcf5, 9cb4300ea183421ab40709a05de5a877; no voters: 
I20260709 13:24:25.163378 27950 tablet_bootstrap.cc:492] T f3e47f5448ff47a382c61886f1a8cc7b P 9cb4300ea183421ab40709a05de5a877: No bootstrap required, opened a new log
I20260709 13:24:25.163785 27959 raft_consensus.cc:2804] T db487682b03b4a738f664fa1459714e5 P 9cb4300ea183421ab40709a05de5a877 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:24:25.161129 27837 raft_consensus.cc:2468] T db487682b03b4a738f664fa1459714e5 P 0788dbdabe574355bb323dcdc3be0058 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9cb4300ea183421ab40709a05de5a877 in term 1.
I20260709 13:24:25.164013 27950 ts_tablet_manager.cc:1403] T f3e47f5448ff47a382c61886f1a8cc7b P 9cb4300ea183421ab40709a05de5a877: Time spent bootstrapping tablet: real 0.076s	user 0.024s	sys 0.015s
I20260709 13:24:25.172986 27954 tablet_bootstrap.cc:492] T f3e47f5448ff47a382c61886f1a8cc7b P 880fc4c7dff4417187b75cc1d9e9dcf5: No bootstrap required, opened a new log
I20260709 13:24:25.173803 27954 ts_tablet_manager.cc:1403] T f3e47f5448ff47a382c61886f1a8cc7b P 880fc4c7dff4417187b75cc1d9e9dcf5: Time spent bootstrapping tablet: real 0.089s	user 0.041s	sys 0.028s
I20260709 13:24:25.176992 27951 tablet_bootstrap.cc:492] T 2f38443d4b8d4e418c8f0520893a31e4 P 0788dbdabe574355bb323dcdc3be0058: No bootstrap required, opened a new log
I20260709 13:24:25.177632 27951 ts_tablet_manager.cc:1403] T 2f38443d4b8d4e418c8f0520893a31e4 P 0788dbdabe574355bb323dcdc3be0058: Time spent bootstrapping tablet: real 0.038s	user 0.014s	sys 0.020s
I20260709 13:24:25.177403 27954 raft_consensus.cc:359] T f3e47f5448ff47a382c61886f1a8cc7b P 880fc4c7dff4417187b75cc1d9e9dcf5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb4300ea183421ab40709a05de5a877" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 33597 } } peers { permanent_uuid: "0788dbdabe574355bb323dcdc3be0058" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 41335 } } peers { permanent_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34235 } }
I20260709 13:24:25.178594 27954 raft_consensus.cc:385] T f3e47f5448ff47a382c61886f1a8cc7b P 880fc4c7dff4417187b75cc1d9e9dcf5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:24:25.178910 27954 raft_consensus.cc:740] T f3e47f5448ff47a382c61886f1a8cc7b P 880fc4c7dff4417187b75cc1d9e9dcf5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 880fc4c7dff4417187b75cc1d9e9dcf5, State: Initialized, Role: FOLLOWER
I20260709 13:24:25.179359 27762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4554587582094286b577c9c9a4d3a7fc" candidate_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9cb4300ea183421ab40709a05de5a877" is_pre_election: true
I20260709 13:24:25.180063 27959 raft_consensus.cc:697] T db487682b03b4a738f664fa1459714e5 P 9cb4300ea183421ab40709a05de5a877 [term 1 LEADER]: Becoming Leader. State: Replica: 9cb4300ea183421ab40709a05de5a877, State: Running, Role: LEADER
I20260709 13:24:25.179795 27954 consensus_queue.cc:260] T f3e47f5448ff47a382c61886f1a8cc7b P 880fc4c7dff4417187b75cc1d9e9dcf5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb4300ea183421ab40709a05de5a877" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 33597 } } peers { permanent_uuid: "0788dbdabe574355bb323dcdc3be0058" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 41335 } } peers { permanent_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34235 } }
I20260709 13:24:25.170934 27950 raft_consensus.cc:359] T f3e47f5448ff47a382c61886f1a8cc7b P 9cb4300ea183421ab40709a05de5a877 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb4300ea183421ab40709a05de5a877" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 33597 } } peers { permanent_uuid: "0788dbdabe574355bb323dcdc3be0058" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 41335 } } peers { permanent_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34235 } }
I20260709 13:24:25.181856 27950 raft_consensus.cc:385] T f3e47f5448ff47a382c61886f1a8cc7b P 9cb4300ea183421ab40709a05de5a877 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260709 13:24:25.181938 27874 leader_election.cc:343] T 4554587582094286b577c9c9a4d3a7fc P 880fc4c7dff4417187b75cc1d9e9dcf5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9cb4300ea183421ab40709a05de5a877 (127.14.177.129:33597): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:24:25.182282 27950 raft_consensus.cc:740] T f3e47f5448ff47a382c61886f1a8cc7b P 9cb4300ea183421ab40709a05de5a877 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9cb4300ea183421ab40709a05de5a877, State: Initialized, Role: FOLLOWER
I20260709 13:24:25.183288 27951 raft_consensus.cc:359] T 2f38443d4b8d4e418c8f0520893a31e4 P 0788dbdabe574355bb323dcdc3be0058 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb4300ea183421ab40709a05de5a877" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 33597 } } peers { permanent_uuid: "0788dbdabe574355bb323dcdc3be0058" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 41335 } } peers { permanent_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34235 } }
I20260709 13:24:25.184084 27951 raft_consensus.cc:385] T 2f38443d4b8d4e418c8f0520893a31e4 P 0788dbdabe574355bb323dcdc3be0058 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:24:25.184369 27951 raft_consensus.cc:740] T 2f38443d4b8d4e418c8f0520893a31e4 P 0788dbdabe574355bb323dcdc3be0058 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0788dbdabe574355bb323dcdc3be0058, State: Initialized, Role: FOLLOWER
I20260709 13:24:25.184918 27954 ts_tablet_manager.cc:1434] T f3e47f5448ff47a382c61886f1a8cc7b P 880fc4c7dff4417187b75cc1d9e9dcf5: Time spent starting tablet: real 0.011s	user 0.009s	sys 0.000s
I20260709 13:24:25.185146 27951 consensus_queue.cc:260] T 2f38443d4b8d4e418c8f0520893a31e4 P 0788dbdabe574355bb323dcdc3be0058 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb4300ea183421ab40709a05de5a877" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 33597 } } peers { permanent_uuid: "0788dbdabe574355bb323dcdc3be0058" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 41335 } } peers { permanent_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34235 } }
I20260709 13:24:25.186039 27954 tablet_bootstrap.cc:492] T 2f38443d4b8d4e418c8f0520893a31e4 P 880fc4c7dff4417187b75cc1d9e9dcf5: Bootstrap starting.
I20260709 13:24:25.185940 27837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4554587582094286b577c9c9a4d3a7fc" candidate_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0788dbdabe574355bb323dcdc3be0058" is_pre_election: true
I20260709 13:24:25.181011 27959 consensus_queue.cc:237] T db487682b03b4a738f664fa1459714e5 P 9cb4300ea183421ab40709a05de5a877 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb4300ea183421ab40709a05de5a877" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 33597 } } peers { permanent_uuid: "0788dbdabe574355bb323dcdc3be0058" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 41335 } } peers { permanent_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34235 } }
I20260709 13:24:25.186868 27837 raft_consensus.cc:2468] T 4554587582094286b577c9c9a4d3a7fc P 0788dbdabe574355bb323dcdc3be0058 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 880fc4c7dff4417187b75cc1d9e9dcf5 in term 0.
I20260709 13:24:25.190461 27876 leader_election.cc:304] T 4554587582094286b577c9c9a4d3a7fc P 880fc4c7dff4417187b75cc1d9e9dcf5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0788dbdabe574355bb323dcdc3be0058, 880fc4c7dff4417187b75cc1d9e9dcf5; no voters: 9cb4300ea183421ab40709a05de5a877
I20260709 13:24:25.183326 27950 consensus_queue.cc:260] T f3e47f5448ff47a382c61886f1a8cc7b P 9cb4300ea183421ab40709a05de5a877 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb4300ea183421ab40709a05de5a877" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 33597 } } peers { permanent_uuid: "0788dbdabe574355bb323dcdc3be0058" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 41335 } } peers { permanent_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34235 } }
I20260709 13:24:25.191991 27958 raft_consensus.cc:2804] T 4554587582094286b577c9c9a4d3a7fc P 880fc4c7dff4417187b75cc1d9e9dcf5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:24:25.192412 27958 raft_consensus.cc:493] T 4554587582094286b577c9c9a4d3a7fc P 880fc4c7dff4417187b75cc1d9e9dcf5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:24:25.192803 27958 raft_consensus.cc:3060] T 4554587582094286b577c9c9a4d3a7fc P 880fc4c7dff4417187b75cc1d9e9dcf5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:24:25.192895 27950 ts_tablet_manager.cc:1434] T f3e47f5448ff47a382c61886f1a8cc7b P 9cb4300ea183421ab40709a05de5a877: Time spent starting tablet: real 0.026s	user 0.007s	sys 0.000s
I20260709 13:24:25.194314 27950 tablet_bootstrap.cc:492] T e7366c36a1154d4d94281a90fde291f8 P 9cb4300ea183421ab40709a05de5a877: Bootstrap starting.
I20260709 13:24:25.196692 27954 tablet_bootstrap.cc:654] T 2f38443d4b8d4e418c8f0520893a31e4 P 880fc4c7dff4417187b75cc1d9e9dcf5: Neither blocks nor log segments found. Creating new log.
I20260709 13:24:25.192433 27955 raft_consensus.cc:493] T f3e47f5448ff47a382c61886f1a8cc7b P 0788dbdabe574355bb323dcdc3be0058 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:24:25.200542 27951 ts_tablet_manager.cc:1434] T 2f38443d4b8d4e418c8f0520893a31e4 P 0788dbdabe574355bb323dcdc3be0058: Time spent starting tablet: real 0.023s	user 0.010s	sys 0.000s
I20260709 13:24:25.200191 27955 raft_consensus.cc:515] T f3e47f5448ff47a382c61886f1a8cc7b P 0788dbdabe574355bb323dcdc3be0058 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb4300ea183421ab40709a05de5a877" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 33597 } } peers { permanent_uuid: "0788dbdabe574355bb323dcdc3be0058" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 41335 } } peers { permanent_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34235 } }
I20260709 13:24:25.201397 27950 tablet_bootstrap.cc:654] T e7366c36a1154d4d94281a90fde291f8 P 9cb4300ea183421ab40709a05de5a877: Neither blocks nor log segments found. Creating new log.
I20260709 13:24:25.201829 27951 tablet_bootstrap.cc:492] T e7366c36a1154d4d94281a90fde291f8 P 0788dbdabe574355bb323dcdc3be0058: Bootstrap starting.
I20260709 13:24:25.204674 27911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3e47f5448ff47a382c61886f1a8cc7b" candidate_uuid: "0788dbdabe574355bb323dcdc3be0058" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" is_pre_election: true
I20260709 13:24:25.205775 27911 raft_consensus.cc:2468] T f3e47f5448ff47a382c61886f1a8cc7b P 880fc4c7dff4417187b75cc1d9e9dcf5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0788dbdabe574355bb323dcdc3be0058 in term 0.
I20260709 13:24:25.207891 27802 leader_election.cc:304] T f3e47f5448ff47a382c61886f1a8cc7b P 0788dbdabe574355bb323dcdc3be0058 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0788dbdabe574355bb323dcdc3be0058, 880fc4c7dff4417187b75cc1d9e9dcf5; no voters: 
I20260709 13:24:25.210656 27951 tablet_bootstrap.cc:654] T e7366c36a1154d4d94281a90fde291f8 P 0788dbdabe574355bb323dcdc3be0058: Neither blocks nor log segments found. Creating new log.
I20260709 13:24:25.212352 27762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3e47f5448ff47a382c61886f1a8cc7b" candidate_uuid: "0788dbdabe574355bb323dcdc3be0058" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9cb4300ea183421ab40709a05de5a877" is_pre_election: true
I20260709 13:24:25.213178 27762 raft_consensus.cc:2468] T f3e47f5448ff47a382c61886f1a8cc7b P 9cb4300ea183421ab40709a05de5a877 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0788dbdabe574355bb323dcdc3be0058 in term 0.
I20260709 13:24:25.214179 27955 leader_election.cc:290] T f3e47f5448ff47a382c61886f1a8cc7b P 0788dbdabe574355bb323dcdc3be0058 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9cb4300ea183421ab40709a05de5a877 (127.14.177.129:33597), 880fc4c7dff4417187b75cc1d9e9dcf5 (127.14.177.131:34235)
I20260709 13:24:25.214915 27955 raft_consensus.cc:2804] T f3e47f5448ff47a382c61886f1a8cc7b P 0788dbdabe574355bb323dcdc3be0058 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:24:25.215459 27955 raft_consensus.cc:493] T f3e47f5448ff47a382c61886f1a8cc7b P 0788dbdabe574355bb323dcdc3be0058 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:24:25.215921 27955 raft_consensus.cc:3060] T f3e47f5448ff47a382c61886f1a8cc7b P 0788dbdabe574355bb323dcdc3be0058 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:24:25.217571 27959 raft_consensus.cc:493] T d16bbb66a57e4079897aaa2c4e6ba144 P 9cb4300ea183421ab40709a05de5a877 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:24:25.218291 27959 raft_consensus.cc:515] T d16bbb66a57e4079897aaa2c4e6ba144 P 9cb4300ea183421ab40709a05de5a877 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb4300ea183421ab40709a05de5a877" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 33597 } } peers { permanent_uuid: "0788dbdabe574355bb323dcdc3be0058" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 41335 } } peers { permanent_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34235 } }
I20260709 13:24:25.217163 27658 catalog_manager.cc:5697] T db487682b03b4a738f664fa1459714e5 P 9cb4300ea183421ab40709a05de5a877 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9cb4300ea183421ab40709a05de5a877 (127.14.177.129). New cstate: current_term: 1 leader_uuid: "9cb4300ea183421ab40709a05de5a877" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb4300ea183421ab40709a05de5a877" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 33597 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0788dbdabe574355bb323dcdc3be0058" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 41335 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34235 } health_report { overall_health: UNKNOWN } } }
I20260709 13:24:25.222436 27837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d16bbb66a57e4079897aaa2c4e6ba144" candidate_uuid: "9cb4300ea183421ab40709a05de5a877" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0788dbdabe574355bb323dcdc3be0058" is_pre_election: true
I20260709 13:24:25.223362 27837 raft_consensus.cc:2468] T d16bbb66a57e4079897aaa2c4e6ba144 P 0788dbdabe574355bb323dcdc3be0058 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9cb4300ea183421ab40709a05de5a877 in term 0.
I20260709 13:24:25.225039 27727 leader_election.cc:304] T d16bbb66a57e4079897aaa2c4e6ba144 P 9cb4300ea183421ab40709a05de5a877 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0788dbdabe574355bb323dcdc3be0058, 9cb4300ea183421ab40709a05de5a877; no voters: 
I20260709 13:24:25.226693 27978 raft_consensus.cc:2804] T d16bbb66a57e4079897aaa2c4e6ba144 P 9cb4300ea183421ab40709a05de5a877 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:24:25.227308 27978 raft_consensus.cc:493] T d16bbb66a57e4079897aaa2c4e6ba144 P 9cb4300ea183421ab40709a05de5a877 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:24:25.227762 27978 raft_consensus.cc:3060] T d16bbb66a57e4079897aaa2c4e6ba144 P 9cb4300ea183421ab40709a05de5a877 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:24:25.233062 27911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d16bbb66a57e4079897aaa2c4e6ba144" candidate_uuid: "9cb4300ea183421ab40709a05de5a877" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" is_pre_election: true
I20260709 13:24:25.234161 27911 raft_consensus.cc:2468] T d16bbb66a57e4079897aaa2c4e6ba144 P 880fc4c7dff4417187b75cc1d9e9dcf5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9cb4300ea183421ab40709a05de5a877 in term 0.
I20260709 13:24:25.233136 27958 raft_consensus.cc:515] T 4554587582094286b577c9c9a4d3a7fc P 880fc4c7dff4417187b75cc1d9e9dcf5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb4300ea183421ab40709a05de5a877" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 33597 } } peers { permanent_uuid: "0788dbdabe574355bb323dcdc3be0058" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 41335 } } peers { permanent_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34235 } }
I20260709 13:24:25.238111 27955 raft_consensus.cc:515] T f3e47f5448ff47a382c61886f1a8cc7b P 0788dbdabe574355bb323dcdc3be0058 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb4300ea183421ab40709a05de5a877" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 33597 } } peers { permanent_uuid: "0788dbdabe574355bb323dcdc3be0058" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 41335 } } peers { permanent_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34235 } }
I20260709 13:24:25.243960 27762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4554587582094286b577c9c9a4d3a7fc" candidate_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9cb4300ea183421ab40709a05de5a877"
I20260709 13:24:25.244735 27959 leader_election.cc:290] T d16bbb66a57e4079897aaa2c4e6ba144 P 9cb4300ea183421ab40709a05de5a877 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0788dbdabe574355bb323dcdc3be0058 (127.14.177.130:41335), 880fc4c7dff4417187b75cc1d9e9dcf5 (127.14.177.131:34235)
I20260709 13:24:25.248082 27837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4554587582094286b577c9c9a4d3a7fc" candidate_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0788dbdabe574355bb323dcdc3be0058"
I20260709 13:24:25.248764 27837 raft_consensus.cc:3060] T 4554587582094286b577c9c9a4d3a7fc P 0788dbdabe574355bb323dcdc3be0058 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:24:25.248778 27874 leader_election.cc:343] T 4554587582094286b577c9c9a4d3a7fc P 880fc4c7dff4417187b75cc1d9e9dcf5 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 9cb4300ea183421ab40709a05de5a877 (127.14.177.129:33597): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:24:25.249138 27762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3e47f5448ff47a382c61886f1a8cc7b" candidate_uuid: "0788dbdabe574355bb323dcdc3be0058" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9cb4300ea183421ab40709a05de5a877"
I20260709 13:24:25.249598 27954 tablet_bootstrap.cc:492] T 2f38443d4b8d4e418c8f0520893a31e4 P 880fc4c7dff4417187b75cc1d9e9dcf5: No bootstrap required, opened a new log
I20260709 13:24:25.249976 27762 raft_consensus.cc:3060] T f3e47f5448ff47a382c61886f1a8cc7b P 9cb4300ea183421ab40709a05de5a877 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:24:25.250182 27954 ts_tablet_manager.cc:1403] T 2f38443d4b8d4e418c8f0520893a31e4 P 880fc4c7dff4417187b75cc1d9e9dcf5: Time spent bootstrapping tablet: real 0.064s	user 0.025s	sys 0.012s
I20260709 13:24:25.251436 27958 leader_election.cc:290] T 4554587582094286b577c9c9a4d3a7fc P 880fc4c7dff4417187b75cc1d9e9dcf5 [CANDIDATE]: Term 1 election: Requested vote from peers 9cb4300ea183421ab40709a05de5a877 (127.14.177.129:33597), 0788dbdabe574355bb323dcdc3be0058 (127.14.177.130:41335)
I20260709 13:24:25.253530 27954 raft_consensus.cc:359] T 2f38443d4b8d4e418c8f0520893a31e4 P 880fc4c7dff4417187b75cc1d9e9dcf5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb4300ea183421ab40709a05de5a877" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 33597 } } peers { permanent_uuid: "0788dbdabe574355bb323dcdc3be0058" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 41335 } } peers { permanent_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34235 } }
I20260709 13:24:25.254091 27951 tablet_bootstrap.cc:492] T e7366c36a1154d4d94281a90fde291f8 P 0788dbdabe574355bb323dcdc3be0058: No bootstrap required, opened a new log
I20260709 13:24:25.254344 27954 raft_consensus.cc:385] T 2f38443d4b8d4e418c8f0520893a31e4 P 880fc4c7dff4417187b75cc1d9e9dcf5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:24:25.254739 27954 raft_consensus.cc:740] T 2f38443d4b8d4e418c8f0520893a31e4 P 880fc4c7dff4417187b75cc1d9e9dcf5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 880fc4c7dff4417187b75cc1d9e9dcf5, State: Initialized, Role: FOLLOWER
I20260709 13:24:25.254895 27951 ts_tablet_manager.cc:1403] T e7366c36a1154d4d94281a90fde291f8 P 0788dbdabe574355bb323dcdc3be0058: Time spent bootstrapping tablet: real 0.053s	user 0.011s	sys 0.003s
I20260709 13:24:25.258358 27762 raft_consensus.cc:2468] T f3e47f5448ff47a382c61886f1a8cc7b P 9cb4300ea183421ab40709a05de5a877 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0788dbdabe574355bb323dcdc3be0058 in term 1.
I20260709 13:24:25.259420 27955 leader_election.cc:290] T f3e47f5448ff47a382c61886f1a8cc7b P 0788dbdabe574355bb323dcdc3be0058 [CANDIDATE]: Term 1 election: Requested vote from peers 9cb4300ea183421ab40709a05de5a877 (127.14.177.129:33597), 880fc4c7dff4417187b75cc1d9e9dcf5 (127.14.177.131:34235)
I20260709 13:24:25.259429 27837 raft_consensus.cc:2468] T 4554587582094286b577c9c9a4d3a7fc P 0788dbdabe574355bb323dcdc3be0058 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 880fc4c7dff4417187b75cc1d9e9dcf5 in term 1.
I20260709 13:24:25.258127 27951 raft_consensus.cc:359] T e7366c36a1154d4d94281a90fde291f8 P 0788dbdabe574355bb323dcdc3be0058 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb4300ea183421ab40709a05de5a877" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 33597 } } peers { permanent_uuid: "0788dbdabe574355bb323dcdc3be0058" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 41335 } } peers { permanent_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34235 } }
I20260709 13:24:25.260692 27951 raft_consensus.cc:385] T e7366c36a1154d4d94281a90fde291f8 P 0788dbdabe574355bb323dcdc3be0058 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:24:25.260804 27911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3e47f5448ff47a382c61886f1a8cc7b" candidate_uuid: "0788dbdabe574355bb323dcdc3be0058" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5"
I20260709 13:24:25.261091 27951 raft_consensus.cc:740] T e7366c36a1154d4d94281a90fde291f8 P 0788dbdabe574355bb323dcdc3be0058 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0788dbdabe574355bb323dcdc3be0058, State: Initialized, Role: FOLLOWER
I20260709 13:24:25.261531 27911 raft_consensus.cc:3060] T f3e47f5448ff47a382c61886f1a8cc7b P 880fc4c7dff4417187b75cc1d9e9dcf5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:24:25.262427 27951 consensus_queue.cc:260] T e7366c36a1154d4d94281a90fde291f8 P 0788dbdabe574355bb323dcdc3be0058 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb4300ea183421ab40709a05de5a877" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 33597 } } peers { permanent_uuid: "0788dbdabe574355bb323dcdc3be0058" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 41335 } } peers { permanent_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34235 } }
I20260709 13:24:25.265950 27978 raft_consensus.cc:515] T d16bbb66a57e4079897aaa2c4e6ba144 P 9cb4300ea183421ab40709a05de5a877 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb4300ea183421ab40709a05de5a877" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 33597 } } peers { permanent_uuid: "0788dbdabe574355bb323dcdc3be0058" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 41335 } } peers { permanent_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34235 } }
I20260709 13:24:25.261339 27876 leader_election.cc:304] T 4554587582094286b577c9c9a4d3a7fc P 880fc4c7dff4417187b75cc1d9e9dcf5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0788dbdabe574355bb323dcdc3be0058, 880fc4c7dff4417187b75cc1d9e9dcf5; no voters: 9cb4300ea183421ab40709a05de5a877
I20260709 13:24:25.267141 27950 tablet_bootstrap.cc:492] T e7366c36a1154d4d94281a90fde291f8 P 9cb4300ea183421ab40709a05de5a877: No bootstrap required, opened a new log
I20260709 13:24:25.267783 27950 ts_tablet_manager.cc:1403] T e7366c36a1154d4d94281a90fde291f8 P 9cb4300ea183421ab40709a05de5a877: Time spent bootstrapping tablet: real 0.074s	user 0.010s	sys 0.004s
I20260709 13:24:25.267660 27954 consensus_queue.cc:260] T 2f38443d4b8d4e418c8f0520893a31e4 P 880fc4c7dff4417187b75cc1d9e9dcf5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb4300ea183421ab40709a05de5a877" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 33597 } } peers { permanent_uuid: "0788dbdabe574355bb323dcdc3be0058" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 41335 } } peers { permanent_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34235 } }
I20260709 13:24:25.268163 27958 raft_consensus.cc:2804] T 4554587582094286b577c9c9a4d3a7fc P 880fc4c7dff4417187b75cc1d9e9dcf5 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:24:25.268961 27951 ts_tablet_manager.cc:1434] T e7366c36a1154d4d94281a90fde291f8 P 0788dbdabe574355bb323dcdc3be0058: Time spent starting tablet: real 0.014s	user 0.006s	sys 0.001s
I20260709 13:24:25.269238 27800 leader_election.cc:304] T f3e47f5448ff47a382c61886f1a8cc7b P 0788dbdabe574355bb323dcdc3be0058 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0788dbdabe574355bb323dcdc3be0058, 9cb4300ea183421ab40709a05de5a877; no voters: 
I20260709 13:24:25.270176 27951 tablet_bootstrap.cc:492] T 028750ef0efa43e4b11ac10f3814902c P 0788dbdabe574355bb323dcdc3be0058: Bootstrap starting.
I20260709 13:24:25.271624 27950 raft_consensus.cc:359] T e7366c36a1154d4d94281a90fde291f8 P 9cb4300ea183421ab40709a05de5a877 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb4300ea183421ab40709a05de5a877" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 33597 } } peers { permanent_uuid: "0788dbdabe574355bb323dcdc3be0058" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 41335 } } peers { permanent_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34235 } }
I20260709 13:24:25.272651 27950 raft_consensus.cc:385] T e7366c36a1154d4d94281a90fde291f8 P 9cb4300ea183421ab40709a05de5a877 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:24:25.273031 27950 raft_consensus.cc:740] T e7366c36a1154d4d94281a90fde291f8 P 9cb4300ea183421ab40709a05de5a877 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9cb4300ea183421ab40709a05de5a877, State: Initialized, Role: FOLLOWER
I20260709 13:24:25.274565 27955 raft_consensus.cc:2804] T f3e47f5448ff47a382c61886f1a8cc7b P 0788dbdabe574355bb323dcdc3be0058 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:24:25.274132 27950 consensus_queue.cc:260] T e7366c36a1154d4d94281a90fde291f8 P 9cb4300ea183421ab40709a05de5a877 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb4300ea183421ab40709a05de5a877" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 33597 } } peers { permanent_uuid: "0788dbdabe574355bb323dcdc3be0058" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 41335 } } peers { permanent_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34235 } }
I20260709 13:24:25.277491 27950 ts_tablet_manager.cc:1434] T e7366c36a1154d4d94281a90fde291f8 P 9cb4300ea183421ab40709a05de5a877: Time spent starting tablet: real 0.009s	user 0.008s	sys 0.000s
I20260709 13:24:25.278764 27950 tablet_bootstrap.cc:492] T cd1140068fc8455280934d7a30c4f559 P 9cb4300ea183421ab40709a05de5a877: Bootstrap starting.
I20260709 13:24:25.280807 27951 tablet_bootstrap.cc:654] T 028750ef0efa43e4b11ac10f3814902c P 0788dbdabe574355bb323dcdc3be0058: Neither blocks nor log segments found. Creating new log.
I20260709 13:24:25.280865 27958 raft_consensus.cc:697] T 4554587582094286b577c9c9a4d3a7fc P 880fc4c7dff4417187b75cc1d9e9dcf5 [term 1 LEADER]: Becoming Leader. State: Replica: 880fc4c7dff4417187b75cc1d9e9dcf5, State: Running, Role: LEADER
I20260709 13:24:25.282856 27958 consensus_queue.cc:237] T 4554587582094286b577c9c9a4d3a7fc P 880fc4c7dff4417187b75cc1d9e9dcf5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb4300ea183421ab40709a05de5a877" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 33597 } } peers { permanent_uuid: "0788dbdabe574355bb323dcdc3be0058" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 41335 } } peers { permanent_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34235 } }
I20260709 13:24:25.282958 27955 raft_consensus.cc:697] T f3e47f5448ff47a382c61886f1a8cc7b P 0788dbdabe574355bb323dcdc3be0058 [term 1 LEADER]: Becoming Leader. State: Replica: 0788dbdabe574355bb323dcdc3be0058, State: Running, Role: LEADER
I20260709 13:24:25.285856 27955 consensus_queue.cc:237] T f3e47f5448ff47a382c61886f1a8cc7b P 0788dbdabe574355bb323dcdc3be0058 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb4300ea183421ab40709a05de5a877" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 33597 } } peers { permanent_uuid: "0788dbdabe574355bb323dcdc3be0058" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 41335 } } peers { permanent_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34235 } }
I20260709 13:24:25.288298 27954 ts_tablet_manager.cc:1434] T 2f38443d4b8d4e418c8f0520893a31e4 P 880fc4c7dff4417187b75cc1d9e9dcf5: Time spent starting tablet: real 0.038s	user 0.008s	sys 0.000s
I20260709 13:24:25.289390 27954 tablet_bootstrap.cc:492] T 20ef9c03fcd94115be0b592d85f89421 P 880fc4c7dff4417187b75cc1d9e9dcf5: Bootstrap starting.
I20260709 13:24:25.290227 27911 raft_consensus.cc:2468] T f3e47f5448ff47a382c61886f1a8cc7b P 880fc4c7dff4417187b75cc1d9e9dcf5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0788dbdabe574355bb323dcdc3be0058 in term 1.
I20260709 13:24:25.290818 27987 raft_consensus.cc:493] T e7366c36a1154d4d94281a90fde291f8 P 880fc4c7dff4417187b75cc1d9e9dcf5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:24:25.291306 27980 raft_consensus.cc:493] T 20ef9c03fcd94115be0b592d85f89421 P 9cb4300ea183421ab40709a05de5a877 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:24:25.291409 27987 raft_consensus.cc:515] T e7366c36a1154d4d94281a90fde291f8 P 880fc4c7dff4417187b75cc1d9e9dcf5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb4300ea183421ab40709a05de5a877" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 33597 } } peers { permanent_uuid: "0788dbdabe574355bb323dcdc3be0058" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 41335 } } peers { permanent_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34235 } }
I20260709 13:24:25.291860 27980 raft_consensus.cc:515] T 20ef9c03fcd94115be0b592d85f89421 P 9cb4300ea183421ab40709a05de5a877 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb4300ea183421ab40709a05de5a877" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 33597 } } peers { permanent_uuid: "0788dbdabe574355bb323dcdc3be0058" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 41335 } } peers { permanent_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34235 } }
I20260709 13:24:25.294000 27980 leader_election.cc:290] T 20ef9c03fcd94115be0b592d85f89421 P 9cb4300ea183421ab40709a05de5a877 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0788dbdabe574355bb323dcdc3be0058 (127.14.177.130:41335), 880fc4c7dff4417187b75cc1d9e9dcf5 (127.14.177.131:34235)
I20260709 13:24:25.296455 27837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20ef9c03fcd94115be0b592d85f89421" candidate_uuid: "9cb4300ea183421ab40709a05de5a877" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0788dbdabe574355bb323dcdc3be0058" is_pre_election: true
I20260709 13:24:25.297276 27837 raft_consensus.cc:2468] T 20ef9c03fcd94115be0b592d85f89421 P 0788dbdabe574355bb323dcdc3be0058 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9cb4300ea183421ab40709a05de5a877 in term 0.
I20260709 13:24:25.285823 27950 tablet_bootstrap.cc:654] T cd1140068fc8455280934d7a30c4f559 P 9cb4300ea183421ab40709a05de5a877: Neither blocks nor log segments found. Creating new log.
I20260709 13:24:25.298594 27727 leader_election.cc:304] T 20ef9c03fcd94115be0b592d85f89421 P 9cb4300ea183421ab40709a05de5a877 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0788dbdabe574355bb323dcdc3be0058, 9cb4300ea183421ab40709a05de5a877; no voters: 
I20260709 13:24:25.299186 27978 leader_election.cc:290] T d16bbb66a57e4079897aaa2c4e6ba144 P 9cb4300ea183421ab40709a05de5a877 [CANDIDATE]: Term 1 election: Requested vote from peers 0788dbdabe574355bb323dcdc3be0058 (127.14.177.130:41335), 880fc4c7dff4417187b75cc1d9e9dcf5 (127.14.177.131:34235)
I20260709 13:24:25.299786 27978 raft_consensus.cc:2804] T 20ef9c03fcd94115be0b592d85f89421 P 9cb4300ea183421ab40709a05de5a877 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:24:25.300714 27837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d16bbb66a57e4079897aaa2c4e6ba144" candidate_uuid: "9cb4300ea183421ab40709a05de5a877" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0788dbdabe574355bb323dcdc3be0058"
I20260709 13:24:25.301355 27837 raft_consensus.cc:3060] T d16bbb66a57e4079897aaa2c4e6ba144 P 0788dbdabe574355bb323dcdc3be0058 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:24:25.305857 27987 leader_election.cc:290] T e7366c36a1154d4d94281a90fde291f8 P 880fc4c7dff4417187b75cc1d9e9dcf5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9cb4300ea183421ab40709a05de5a877 (127.14.177.129:33597), 0788dbdabe574355bb323dcdc3be0058 (127.14.177.130:41335)
I20260709 13:24:25.307603 27762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7366c36a1154d4d94281a90fde291f8" candidate_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9cb4300ea183421ab40709a05de5a877" is_pre_election: true
I20260709 13:24:25.307749 27836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7366c36a1154d4d94281a90fde291f8" candidate_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0788dbdabe574355bb323dcdc3be0058" is_pre_election: true
I20260709 13:24:25.300150 27978 raft_consensus.cc:493] T 20ef9c03fcd94115be0b592d85f89421 P 9cb4300ea183421ab40709a05de5a877 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:24:25.308357 27762 raft_consensus.cc:2468] T e7366c36a1154d4d94281a90fde291f8 P 9cb4300ea183421ab40709a05de5a877 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 880fc4c7dff4417187b75cc1d9e9dcf5 in term 0.
I20260709 13:24:25.308770 27978 raft_consensus.cc:3060] T 20ef9c03fcd94115be0b592d85f89421 P 9cb4300ea183421ab40709a05de5a877 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:24:25.308821 27836 raft_consensus.cc:2468] T e7366c36a1154d4d94281a90fde291f8 P 0788dbdabe574355bb323dcdc3be0058 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 880fc4c7dff4417187b75cc1d9e9dcf5 in term 0.
I20260709 13:24:25.309700 27874 leader_election.cc:304] T e7366c36a1154d4d94281a90fde291f8 P 880fc4c7dff4417187b75cc1d9e9dcf5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 880fc4c7dff4417187b75cc1d9e9dcf5, 9cb4300ea183421ab40709a05de5a877; no voters: 
I20260709 13:24:25.311348 27987 raft_consensus.cc:2804] T e7366c36a1154d4d94281a90fde291f8 P 880fc4c7dff4417187b75cc1d9e9dcf5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:24:25.311723 27987 raft_consensus.cc:493] T e7366c36a1154d4d94281a90fde291f8 P 880fc4c7dff4417187b75cc1d9e9dcf5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:24:25.312067 27987 raft_consensus.cc:3060] T e7366c36a1154d4d94281a90fde291f8 P 880fc4c7dff4417187b75cc1d9e9dcf5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:24:25.312789 27910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d16bbb66a57e4079897aaa2c4e6ba144" candidate_uuid: "9cb4300ea183421ab40709a05de5a877" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5"
I20260709 13:24:25.313907 27910 raft_consensus.cc:3060] T d16bbb66a57e4079897aaa2c4e6ba144 P 880fc4c7dff4417187b75cc1d9e9dcf5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:24:25.317112 27911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20ef9c03fcd94115be0b592d85f89421" candidate_uuid: "9cb4300ea183421ab40709a05de5a877" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" is_pre_election: true
I20260709 13:24:25.312753 27837 raft_consensus.cc:2468] T d16bbb66a57e4079897aaa2c4e6ba144 P 0788dbdabe574355bb323dcdc3be0058 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9cb4300ea183421ab40709a05de5a877 in term 1.
W20260709 13:24:25.320813 27727 leader_election.cc:343] T 20ef9c03fcd94115be0b592d85f89421 P 9cb4300ea183421ab40709a05de5a877 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 880fc4c7dff4417187b75cc1d9e9dcf5 (127.14.177.131:34235): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:24:25.326267 27987 raft_consensus.cc:515] T e7366c36a1154d4d94281a90fde291f8 P 880fc4c7dff4417187b75cc1d9e9dcf5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb4300ea183421ab40709a05de5a877" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 33597 } } peers { permanent_uuid: "0788dbdabe574355bb323dcdc3be0058" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 41335 } } peers { permanent_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34235 } }
I20260709 13:24:25.326445 27978 raft_consensus.cc:515] T 20ef9c03fcd94115be0b592d85f89421 P 9cb4300ea183421ab40709a05de5a877 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb4300ea183421ab40709a05de5a877" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 33597 } } peers { permanent_uuid: "0788dbdabe574355bb323dcdc3be0058" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 41335 } } peers { permanent_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34235 } }
I20260709 13:24:25.329119 27910 raft_consensus.cc:2468] T d16bbb66a57e4079897aaa2c4e6ba144 P 880fc4c7dff4417187b75cc1d9e9dcf5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9cb4300ea183421ab40709a05de5a877 in term 1.
I20260709 13:24:25.329253 27978 leader_election.cc:290] T 20ef9c03fcd94115be0b592d85f89421 P 9cb4300ea183421ab40709a05de5a877 [CANDIDATE]: Term 1 election: Requested vote from peers 0788dbdabe574355bb323dcdc3be0058 (127.14.177.130:41335), 880fc4c7dff4417187b75cc1d9e9dcf5 (127.14.177.131:34235)
I20260709 13:24:25.330004 27727 leader_election.cc:304] T d16bbb66a57e4079897aaa2c4e6ba144 P 9cb4300ea183421ab40709a05de5a877 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0788dbdabe574355bb323dcdc3be0058, 9cb4300ea183421ab40709a05de5a877; no voters: 
I20260709 13:24:25.330879 27978 raft_consensus.cc:2804] T d16bbb66a57e4079897aaa2c4e6ba144 P 9cb4300ea183421ab40709a05de5a877 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:24:25.330925 27762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7366c36a1154d4d94281a90fde291f8" candidate_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9cb4300ea183421ab40709a05de5a877"
I20260709 13:24:25.331642 27978 raft_consensus.cc:697] T d16bbb66a57e4079897aaa2c4e6ba144 P 9cb4300ea183421ab40709a05de5a877 [term 1 LEADER]: Becoming Leader. State: Replica: 9cb4300ea183421ab40709a05de5a877, State: Running, Role: LEADER
I20260709 13:24:25.331790 27762 raft_consensus.cc:3060] T e7366c36a1154d4d94281a90fde291f8 P 9cb4300ea183421ab40709a05de5a877 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:24:25.332717 27978 consensus_queue.cc:237] T d16bbb66a57e4079897aaa2c4e6ba144 P 9cb4300ea183421ab40709a05de5a877 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb4300ea183421ab40709a05de5a877" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 33597 } } peers { permanent_uuid: "0788dbdabe574355bb323dcdc3be0058" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 41335 } } peers { permanent_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34235 } }
I20260709 13:24:25.336704 27951 tablet_bootstrap.cc:492] T 028750ef0efa43e4b11ac10f3814902c P 0788dbdabe574355bb323dcdc3be0058: No bootstrap required, opened a new log
I20260709 13:24:25.333412 27660 catalog_manager.cc:5697] T f3e47f5448ff47a382c61886f1a8cc7b P 0788dbdabe574355bb323dcdc3be0058 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0788dbdabe574355bb323dcdc3be0058 (127.14.177.130). New cstate: current_term: 1 leader_uuid: "0788dbdabe574355bb323dcdc3be0058" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb4300ea183421ab40709a05de5a877" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 33597 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0788dbdabe574355bb323dcdc3be0058" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 41335 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34235 } health_report { overall_health: UNKNOWN } } }
I20260709 13:24:25.337049 27837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20ef9c03fcd94115be0b592d85f89421" candidate_uuid: "9cb4300ea183421ab40709a05de5a877" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0788dbdabe574355bb323dcdc3be0058"
I20260709 13:24:25.340026 27951 ts_tablet_manager.cc:1403] T 028750ef0efa43e4b11ac10f3814902c P 0788dbdabe574355bb323dcdc3be0058: Time spent bootstrapping tablet: real 0.070s	user 0.028s	sys 0.016s
I20260709 13:24:25.340353 27837 raft_consensus.cc:3060] T 20ef9c03fcd94115be0b592d85f89421 P 0788dbdabe574355bb323dcdc3be0058 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:24:25.343044 27951 raft_consensus.cc:359] T 028750ef0efa43e4b11ac10f3814902c P 0788dbdabe574355bb323dcdc3be0058 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb4300ea183421ab40709a05de5a877" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 33597 } } peers { permanent_uuid: "0788dbdabe574355bb323dcdc3be0058" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 41335 } } peers { permanent_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34235 } }
I20260709 13:24:25.326609 27659 catalog_manager.cc:5697] T 4554587582094286b577c9c9a4d3a7fc P 880fc4c7dff4417187b75cc1d9e9dcf5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 880fc4c7dff4417187b75cc1d9e9dcf5 (127.14.177.131). New cstate: current_term: 1 leader_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb4300ea183421ab40709a05de5a877" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 33597 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0788dbdabe574355bb323dcdc3be0058" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 41335 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34235 } health_report { overall_health: HEALTHY } } }
I20260709 13:24:25.344903 27954 tablet_bootstrap.cc:654] T 20ef9c03fcd94115be0b592d85f89421 P 880fc4c7dff4417187b75cc1d9e9dcf5: Neither blocks nor log segments found. Creating new log.
I20260709 13:24:25.345522 27836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7366c36a1154d4d94281a90fde291f8" candidate_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0788dbdabe574355bb323dcdc3be0058"
I20260709 13:24:25.346139 27836 raft_consensus.cc:3060] T e7366c36a1154d4d94281a90fde291f8 P 0788dbdabe574355bb323dcdc3be0058 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:24:25.344049 27951 raft_consensus.cc:385] T 028750ef0efa43e4b11ac10f3814902c P 0788dbdabe574355bb323dcdc3be0058 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:24:25.347759 27951 raft_consensus.cc:740] T 028750ef0efa43e4b11ac10f3814902c P 0788dbdabe574355bb323dcdc3be0058 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0788dbdabe574355bb323dcdc3be0058, State: Initialized, Role: FOLLOWER
I20260709 13:24:25.353089 27951 consensus_queue.cc:260] T 028750ef0efa43e4b11ac10f3814902c P 0788dbdabe574355bb323dcdc3be0058 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb4300ea183421ab40709a05de5a877" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 33597 } } peers { permanent_uuid: "0788dbdabe574355bb323dcdc3be0058" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 41335 } } peers { permanent_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34235 } }
I20260709 13:24:25.344089 27987 leader_election.cc:290] T e7366c36a1154d4d94281a90fde291f8 P 880fc4c7dff4417187b75cc1d9e9dcf5 [CANDIDATE]: Term 1 election: Requested vote from peers 9cb4300ea183421ab40709a05de5a877 (127.14.177.129:33597), 0788dbdabe574355bb323dcdc3be0058 (127.14.177.130:41335)
I20260709 13:24:25.353263 27762 raft_consensus.cc:2468] T e7366c36a1154d4d94281a90fde291f8 P 9cb4300ea183421ab40709a05de5a877 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 880fc4c7dff4417187b75cc1d9e9dcf5 in term 1.
I20260709 13:24:25.355156 27910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20ef9c03fcd94115be0b592d85f89421" candidate_uuid: "9cb4300ea183421ab40709a05de5a877" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5"
I20260709 13:24:25.356515 27951 ts_tablet_manager.cc:1434] T 028750ef0efa43e4b11ac10f3814902c P 0788dbdabe574355bb323dcdc3be0058: Time spent starting tablet: real 0.016s	user 0.002s	sys 0.005s
W20260709 13:24:25.357043 27727 leader_election.cc:343] T 20ef9c03fcd94115be0b592d85f89421 P 9cb4300ea183421ab40709a05de5a877 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 880fc4c7dff4417187b75cc1d9e9dcf5 (127.14.177.131:34235): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:24:25.352790 27837 raft_consensus.cc:2468] T 20ef9c03fcd94115be0b592d85f89421 P 0788dbdabe574355bb323dcdc3be0058 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9cb4300ea183421ab40709a05de5a877 in term 1.
I20260709 13:24:25.359833 27874 leader_election.cc:304] T e7366c36a1154d4d94281a90fde291f8 P 880fc4c7dff4417187b75cc1d9e9dcf5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 880fc4c7dff4417187b75cc1d9e9dcf5, 9cb4300ea183421ab40709a05de5a877; no voters: 
I20260709 13:24:25.358456 27836 raft_consensus.cc:2468] T e7366c36a1154d4d94281a90fde291f8 P 0788dbdabe574355bb323dcdc3be0058 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 880fc4c7dff4417187b75cc1d9e9dcf5 in term 1.
I20260709 13:24:25.361271 27727 leader_election.cc:304] T 20ef9c03fcd94115be0b592d85f89421 P 9cb4300ea183421ab40709a05de5a877 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0788dbdabe574355bb323dcdc3be0058, 9cb4300ea183421ab40709a05de5a877; no voters: 880fc4c7dff4417187b75cc1d9e9dcf5
I20260709 13:24:25.362083 27987 raft_consensus.cc:2804] T e7366c36a1154d4d94281a90fde291f8 P 880fc4c7dff4417187b75cc1d9e9dcf5 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:24:25.362599 27980 raft_consensus.cc:2804] T 20ef9c03fcd94115be0b592d85f89421 P 9cb4300ea183421ab40709a05de5a877 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:24:25.362677 27987 raft_consensus.cc:697] T e7366c36a1154d4d94281a90fde291f8 P 880fc4c7dff4417187b75cc1d9e9dcf5 [term 1 LEADER]: Becoming Leader. State: Replica: 880fc4c7dff4417187b75cc1d9e9dcf5, State: Running, Role: LEADER
I20260709 13:24:25.363062 27950 tablet_bootstrap.cc:492] T cd1140068fc8455280934d7a30c4f559 P 9cb4300ea183421ab40709a05de5a877: No bootstrap required, opened a new log
I20260709 13:24:25.363226 27980 raft_consensus.cc:697] T 20ef9c03fcd94115be0b592d85f89421 P 9cb4300ea183421ab40709a05de5a877 [term 1 LEADER]: Becoming Leader. State: Replica: 9cb4300ea183421ab40709a05de5a877, State: Running, Role: LEADER
I20260709 13:24:25.363668 27950 ts_tablet_manager.cc:1403] T cd1140068fc8455280934d7a30c4f559 P 9cb4300ea183421ab40709a05de5a877: Time spent bootstrapping tablet: real 0.085s	user 0.004s	sys 0.015s
I20260709 13:24:25.363935 27987 consensus_queue.cc:237] T e7366c36a1154d4d94281a90fde291f8 P 880fc4c7dff4417187b75cc1d9e9dcf5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb4300ea183421ab40709a05de5a877" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 33597 } } peers { permanent_uuid: "0788dbdabe574355bb323dcdc3be0058" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 41335 } } peers { permanent_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34235 } }
I20260709 13:24:25.364320 27980 consensus_queue.cc:237] T 20ef9c03fcd94115be0b592d85f89421 P 9cb4300ea183421ab40709a05de5a877 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb4300ea183421ab40709a05de5a877" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 33597 } } peers { permanent_uuid: "0788dbdabe574355bb323dcdc3be0058" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 41335 } } peers { permanent_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34235 } }
I20260709 13:24:25.366948 27950 raft_consensus.cc:359] T cd1140068fc8455280934d7a30c4f559 P 9cb4300ea183421ab40709a05de5a877 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb4300ea183421ab40709a05de5a877" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 33597 } } peers { permanent_uuid: "0788dbdabe574355bb323dcdc3be0058" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 41335 } } peers { permanent_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34235 } }
I20260709 13:24:25.368147 27950 raft_consensus.cc:385] T cd1140068fc8455280934d7a30c4f559 P 9cb4300ea183421ab40709a05de5a877 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:24:25.372262 27950 raft_consensus.cc:740] T cd1140068fc8455280934d7a30c4f559 P 9cb4300ea183421ab40709a05de5a877 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9cb4300ea183421ab40709a05de5a877, State: Initialized, Role: FOLLOWER
I20260709 13:24:25.373229 27950 consensus_queue.cc:260] T cd1140068fc8455280934d7a30c4f559 P 9cb4300ea183421ab40709a05de5a877 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb4300ea183421ab40709a05de5a877" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 33597 } } peers { permanent_uuid: "0788dbdabe574355bb323dcdc3be0058" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 41335 } } peers { permanent_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34235 } }
I20260709 13:24:25.376382 27950 ts_tablet_manager.cc:1434] T cd1140068fc8455280934d7a30c4f559 P 9cb4300ea183421ab40709a05de5a877: Time spent starting tablet: real 0.012s	user 0.006s	sys 0.002s
I20260709 13:24:25.377715 27950 tablet_bootstrap.cc:492] T 4554587582094286b577c9c9a4d3a7fc P 9cb4300ea183421ab40709a05de5a877: Bootstrap starting.
I20260709 13:24:25.379201 27990 raft_consensus.cc:493] T cd1140068fc8455280934d7a30c4f559 P 880fc4c7dff4417187b75cc1d9e9dcf5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:24:25.379809 27990 raft_consensus.cc:515] T cd1140068fc8455280934d7a30c4f559 P 880fc4c7dff4417187b75cc1d9e9dcf5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb4300ea183421ab40709a05de5a877" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 33597 } } peers { permanent_uuid: "0788dbdabe574355bb323dcdc3be0058" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 41335 } } peers { permanent_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34235 } }
I20260709 13:24:25.381955 27658 catalog_manager.cc:5697] T d16bbb66a57e4079897aaa2c4e6ba144 P 9cb4300ea183421ab40709a05de5a877 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9cb4300ea183421ab40709a05de5a877 (127.14.177.129). New cstate: current_term: 1 leader_uuid: "9cb4300ea183421ab40709a05de5a877" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb4300ea183421ab40709a05de5a877" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 33597 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0788dbdabe574355bb323dcdc3be0058" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 41335 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34235 } health_report { overall_health: UNKNOWN } } }
I20260709 13:24:25.385555 27762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd1140068fc8455280934d7a30c4f559" candidate_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9cb4300ea183421ab40709a05de5a877" is_pre_election: true
I20260709 13:24:25.386304 27762 raft_consensus.cc:2468] T cd1140068fc8455280934d7a30c4f559 P 9cb4300ea183421ab40709a05de5a877 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 880fc4c7dff4417187b75cc1d9e9dcf5 in term 0.
I20260709 13:24:25.390866 27874 leader_election.cc:304] T cd1140068fc8455280934d7a30c4f559 P 880fc4c7dff4417187b75cc1d9e9dcf5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 880fc4c7dff4417187b75cc1d9e9dcf5, 9cb4300ea183421ab40709a05de5a877; no voters: 
I20260709 13:24:25.391866 27958 raft_consensus.cc:2804] T cd1140068fc8455280934d7a30c4f559 P 880fc4c7dff4417187b75cc1d9e9dcf5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:24:25.392298 27958 raft_consensus.cc:493] T cd1140068fc8455280934d7a30c4f559 P 880fc4c7dff4417187b75cc1d9e9dcf5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:24:25.392735 27954 tablet_bootstrap.cc:492] T 20ef9c03fcd94115be0b592d85f89421 P 880fc4c7dff4417187b75cc1d9e9dcf5: No bootstrap required, opened a new log
I20260709 13:24:25.392688 27958 raft_consensus.cc:3060] T cd1140068fc8455280934d7a30c4f559 P 880fc4c7dff4417187b75cc1d9e9dcf5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:24:25.384684 27836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd1140068fc8455280934d7a30c4f559" candidate_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0788dbdabe574355bb323dcdc3be0058" is_pre_election: true
I20260709 13:24:25.393565 27836 raft_consensus.cc:2468] T cd1140068fc8455280934d7a30c4f559 P 0788dbdabe574355bb323dcdc3be0058 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 880fc4c7dff4417187b75cc1d9e9dcf5 in term 0.
I20260709 13:24:25.396613 27990 leader_election.cc:290] T cd1140068fc8455280934d7a30c4f559 P 880fc4c7dff4417187b75cc1d9e9dcf5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9cb4300ea183421ab40709a05de5a877 (127.14.177.129:33597), 0788dbdabe574355bb323dcdc3be0058 (127.14.177.130:41335)
I20260709 13:24:25.403590 27954 ts_tablet_manager.cc:1403] T 20ef9c03fcd94115be0b592d85f89421 P 880fc4c7dff4417187b75cc1d9e9dcf5: Time spent bootstrapping tablet: real 0.114s	user 0.013s	sys 0.002s
I20260709 13:24:25.405014 27950 tablet_bootstrap.cc:654] T 4554587582094286b577c9c9a4d3a7fc P 9cb4300ea183421ab40709a05de5a877: Neither blocks nor log segments found. Creating new log.
I20260709 13:24:25.402753 27958 raft_consensus.cc:515] T cd1140068fc8455280934d7a30c4f559 P 880fc4c7dff4417187b75cc1d9e9dcf5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb4300ea183421ab40709a05de5a877" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 33597 } } peers { permanent_uuid: "0788dbdabe574355bb323dcdc3be0058" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 41335 } } peers { permanent_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34235 } }
I20260709 13:24:25.418453 27950 tablet_bootstrap.cc:492] T 4554587582094286b577c9c9a4d3a7fc P 9cb4300ea183421ab40709a05de5a877: No bootstrap required, opened a new log
I20260709 13:24:25.418510 27762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd1140068fc8455280934d7a30c4f559" candidate_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9cb4300ea183421ab40709a05de5a877"
I20260709 13:24:25.419803 27950 ts_tablet_manager.cc:1403] T 4554587582094286b577c9c9a4d3a7fc P 9cb4300ea183421ab40709a05de5a877: Time spent bootstrapping tablet: real 0.042s	user 0.021s	sys 0.001s
I20260709 13:24:25.419804 27762 raft_consensus.cc:3060] T cd1140068fc8455280934d7a30c4f559 P 9cb4300ea183421ab40709a05de5a877 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:24:25.424407 27950 raft_consensus.cc:359] T 4554587582094286b577c9c9a4d3a7fc P 9cb4300ea183421ab40709a05de5a877 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb4300ea183421ab40709a05de5a877" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 33597 } } peers { permanent_uuid: "0788dbdabe574355bb323dcdc3be0058" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 41335 } } peers { permanent_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34235 } }
I20260709 13:24:25.425484 27950 raft_consensus.cc:385] T 4554587582094286b577c9c9a4d3a7fc P 9cb4300ea183421ab40709a05de5a877 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:24:25.425879 27950 raft_consensus.cc:740] T 4554587582094286b577c9c9a4d3a7fc P 9cb4300ea183421ab40709a05de5a877 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9cb4300ea183421ab40709a05de5a877, State: Initialized, Role: FOLLOWER
I20260709 13:24:25.426966 27950 consensus_queue.cc:260] T 4554587582094286b577c9c9a4d3a7fc P 9cb4300ea183421ab40709a05de5a877 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb4300ea183421ab40709a05de5a877" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 33597 } } peers { permanent_uuid: "0788dbdabe574355bb323dcdc3be0058" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 41335 } } peers { permanent_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34235 } }
I20260709 13:24:25.430485 27950 ts_tablet_manager.cc:1434] T 4554587582094286b577c9c9a4d3a7fc P 9cb4300ea183421ab40709a05de5a877: Time spent starting tablet: real 0.010s	user 0.008s	sys 0.000s
I20260709 13:24:25.431921 27950 tablet_bootstrap.cc:492] T 2f38443d4b8d4e418c8f0520893a31e4 P 9cb4300ea183421ab40709a05de5a877: Bootstrap starting.
I20260709 13:24:25.414031 27954 raft_consensus.cc:359] T 20ef9c03fcd94115be0b592d85f89421 P 880fc4c7dff4417187b75cc1d9e9dcf5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb4300ea183421ab40709a05de5a877" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 33597 } } peers { permanent_uuid: "0788dbdabe574355bb323dcdc3be0058" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 41335 } } peers { permanent_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34235 } }
I20260709 13:24:25.434669 27954 raft_consensus.cc:385] T 20ef9c03fcd94115be0b592d85f89421 P 880fc4c7dff4417187b75cc1d9e9dcf5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:24:25.435202 27658 catalog_manager.cc:5697] T 20ef9c03fcd94115be0b592d85f89421 P 9cb4300ea183421ab40709a05de5a877 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9cb4300ea183421ab40709a05de5a877 (127.14.177.129). New cstate: current_term: 1 leader_uuid: "9cb4300ea183421ab40709a05de5a877" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb4300ea183421ab40709a05de5a877" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 33597 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0788dbdabe574355bb323dcdc3be0058" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 41335 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34235 } health_report { overall_health: UNKNOWN } } }
I20260709 13:24:25.437666 27954 raft_consensus.cc:740] T 20ef9c03fcd94115be0b592d85f89421 P 880fc4c7dff4417187b75cc1d9e9dcf5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 880fc4c7dff4417187b75cc1d9e9dcf5, State: Initialized, Role: FOLLOWER
I20260709 13:24:25.438678 27954 consensus_queue.cc:260] T 20ef9c03fcd94115be0b592d85f89421 P 880fc4c7dff4417187b75cc1d9e9dcf5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb4300ea183421ab40709a05de5a877" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 33597 } } peers { permanent_uuid: "0788dbdabe574355bb323dcdc3be0058" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 41335 } } peers { permanent_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34235 } }
I20260709 13:24:25.441017 27950 tablet_bootstrap.cc:654] T 2f38443d4b8d4e418c8f0520893a31e4 P 9cb4300ea183421ab40709a05de5a877: Neither blocks nor log segments found. Creating new log.
I20260709 13:24:25.436573 27958 leader_election.cc:290] T cd1140068fc8455280934d7a30c4f559 P 880fc4c7dff4417187b75cc1d9e9dcf5 [CANDIDATE]: Term 1 election: Requested vote from peers 9cb4300ea183421ab40709a05de5a877 (127.14.177.129:33597), 0788dbdabe574355bb323dcdc3be0058 (127.14.177.130:41335)
I20260709 13:24:25.436849 27836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd1140068fc8455280934d7a30c4f559" candidate_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0788dbdabe574355bb323dcdc3be0058"
I20260709 13:24:25.436144 27762 raft_consensus.cc:2468] T cd1140068fc8455280934d7a30c4f559 P 9cb4300ea183421ab40709a05de5a877 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 880fc4c7dff4417187b75cc1d9e9dcf5 in term 1.
I20260709 13:24:25.447642 27836 raft_consensus.cc:3060] T cd1140068fc8455280934d7a30c4f559 P 0788dbdabe574355bb323dcdc3be0058 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:24:25.446425 27660 catalog_manager.cc:5697] T e7366c36a1154d4d94281a90fde291f8 P 880fc4c7dff4417187b75cc1d9e9dcf5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 880fc4c7dff4417187b75cc1d9e9dcf5 (127.14.177.131). New cstate: current_term: 1 leader_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb4300ea183421ab40709a05de5a877" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 33597 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0788dbdabe574355bb323dcdc3be0058" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 41335 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34235 } health_report { overall_health: HEALTHY } } }
I20260709 13:24:25.448925 27954 ts_tablet_manager.cc:1434] T 20ef9c03fcd94115be0b592d85f89421 P 880fc4c7dff4417187b75cc1d9e9dcf5: Time spent starting tablet: real 0.044s	user 0.007s	sys 0.000s
I20260709 13:24:25.452490 27874 leader_election.cc:304] T cd1140068fc8455280934d7a30c4f559 P 880fc4c7dff4417187b75cc1d9e9dcf5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 880fc4c7dff4417187b75cc1d9e9dcf5, 9cb4300ea183421ab40709a05de5a877; no voters: 
I20260709 13:24:25.453442 27958 raft_consensus.cc:2804] T cd1140068fc8455280934d7a30c4f559 P 880fc4c7dff4417187b75cc1d9e9dcf5 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:24:25.454124 27958 raft_consensus.cc:697] T cd1140068fc8455280934d7a30c4f559 P 880fc4c7dff4417187b75cc1d9e9dcf5 [term 1 LEADER]: Becoming Leader. State: Replica: 880fc4c7dff4417187b75cc1d9e9dcf5, State: Running, Role: LEADER
I20260709 13:24:25.454986 27958 consensus_queue.cc:237] T cd1140068fc8455280934d7a30c4f559 P 880fc4c7dff4417187b75cc1d9e9dcf5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb4300ea183421ab40709a05de5a877" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 33597 } } peers { permanent_uuid: "0788dbdabe574355bb323dcdc3be0058" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 41335 } } peers { permanent_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34235 } }
I20260709 13:24:25.458570 27954 tablet_bootstrap.cc:492] T 028750ef0efa43e4b11ac10f3814902c P 880fc4c7dff4417187b75cc1d9e9dcf5: Bootstrap starting.
I20260709 13:24:25.460857 27836 raft_consensus.cc:2468] T cd1140068fc8455280934d7a30c4f559 P 0788dbdabe574355bb323dcdc3be0058 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 880fc4c7dff4417187b75cc1d9e9dcf5 in term 1.
I20260709 13:24:25.465622 27954 tablet_bootstrap.cc:654] T 028750ef0efa43e4b11ac10f3814902c P 880fc4c7dff4417187b75cc1d9e9dcf5: Neither blocks nor log segments found. Creating new log.
I20260709 13:24:25.470461 27950 tablet_bootstrap.cc:492] T 2f38443d4b8d4e418c8f0520893a31e4 P 9cb4300ea183421ab40709a05de5a877: No bootstrap required, opened a new log
I20260709 13:24:25.470959 27950 ts_tablet_manager.cc:1403] T 2f38443d4b8d4e418c8f0520893a31e4 P 9cb4300ea183421ab40709a05de5a877: Time spent bootstrapping tablet: real 0.039s	user 0.015s	sys 0.015s
I20260709 13:24:25.473917 27954 tablet_bootstrap.cc:492] T 028750ef0efa43e4b11ac10f3814902c P 880fc4c7dff4417187b75cc1d9e9dcf5: No bootstrap required, opened a new log
I20260709 13:24:25.473556 27950 raft_consensus.cc:359] T 2f38443d4b8d4e418c8f0520893a31e4 P 9cb4300ea183421ab40709a05de5a877 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb4300ea183421ab40709a05de5a877" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 33597 } } peers { permanent_uuid: "0788dbdabe574355bb323dcdc3be0058" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 41335 } } peers { permanent_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34235 } }
I20260709 13:24:25.474355 27950 raft_consensus.cc:385] T 2f38443d4b8d4e418c8f0520893a31e4 P 9cb4300ea183421ab40709a05de5a877 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:24:25.474503 27954 ts_tablet_manager.cc:1403] T 028750ef0efa43e4b11ac10f3814902c P 880fc4c7dff4417187b75cc1d9e9dcf5: Time spent bootstrapping tablet: real 0.025s	user 0.014s	sys 0.000s
I20260709 13:24:25.474668 27950 raft_consensus.cc:740] T 2f38443d4b8d4e418c8f0520893a31e4 P 9cb4300ea183421ab40709a05de5a877 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9cb4300ea183421ab40709a05de5a877, State: Initialized, Role: FOLLOWER
I20260709 13:24:25.474895 27660 catalog_manager.cc:5697] T cd1140068fc8455280934d7a30c4f559 P 880fc4c7dff4417187b75cc1d9e9dcf5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 880fc4c7dff4417187b75cc1d9e9dcf5 (127.14.177.131). New cstate: current_term: 1 leader_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb4300ea183421ab40709a05de5a877" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 33597 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0788dbdabe574355bb323dcdc3be0058" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 41335 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34235 } health_report { overall_health: HEALTHY } } }
I20260709 13:24:25.475533 27950 consensus_queue.cc:260] T 2f38443d4b8d4e418c8f0520893a31e4 P 9cb4300ea183421ab40709a05de5a877 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb4300ea183421ab40709a05de5a877" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 33597 } } peers { permanent_uuid: "0788dbdabe574355bb323dcdc3be0058" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 41335 } } peers { permanent_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34235 } }
I20260709 13:24:25.477685 27950 ts_tablet_manager.cc:1434] T 2f38443d4b8d4e418c8f0520893a31e4 P 9cb4300ea183421ab40709a05de5a877: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 13:24:25.477800 27954 raft_consensus.cc:359] T 028750ef0efa43e4b11ac10f3814902c P 880fc4c7dff4417187b75cc1d9e9dcf5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb4300ea183421ab40709a05de5a877" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 33597 } } peers { permanent_uuid: "0788dbdabe574355bb323dcdc3be0058" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 41335 } } peers { permanent_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34235 } }
I20260709 13:24:25.478524 27954 raft_consensus.cc:385] T 028750ef0efa43e4b11ac10f3814902c P 880fc4c7dff4417187b75cc1d9e9dcf5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:24:25.478694 27950 tablet_bootstrap.cc:492] T 028750ef0efa43e4b11ac10f3814902c P 9cb4300ea183421ab40709a05de5a877: Bootstrap starting.
I20260709 13:24:25.478794 27954 raft_consensus.cc:740] T 028750ef0efa43e4b11ac10f3814902c P 880fc4c7dff4417187b75cc1d9e9dcf5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 880fc4c7dff4417187b75cc1d9e9dcf5, State: Initialized, Role: FOLLOWER
I20260709 13:24:25.479765 27954 consensus_queue.cc:260] T 028750ef0efa43e4b11ac10f3814902c P 880fc4c7dff4417187b75cc1d9e9dcf5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb4300ea183421ab40709a05de5a877" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 33597 } } peers { permanent_uuid: "0788dbdabe574355bb323dcdc3be0058" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 41335 } } peers { permanent_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34235 } }
I20260709 13:24:25.482074 27954 ts_tablet_manager.cc:1434] T 028750ef0efa43e4b11ac10f3814902c P 880fc4c7dff4417187b75cc1d9e9dcf5: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.003s
I20260709 13:24:25.485155 27950 tablet_bootstrap.cc:654] T 028750ef0efa43e4b11ac10f3814902c P 9cb4300ea183421ab40709a05de5a877: Neither blocks nor log segments found. Creating new log.
I20260709 13:24:25.491816 27950 tablet_bootstrap.cc:492] T 028750ef0efa43e4b11ac10f3814902c P 9cb4300ea183421ab40709a05de5a877: No bootstrap required, opened a new log
I20260709 13:24:25.492332 27950 ts_tablet_manager.cc:1403] T 028750ef0efa43e4b11ac10f3814902c P 9cb4300ea183421ab40709a05de5a877: Time spent bootstrapping tablet: real 0.014s	user 0.012s	sys 0.000s
I20260709 13:24:25.494549 27950 raft_consensus.cc:359] T 028750ef0efa43e4b11ac10f3814902c P 9cb4300ea183421ab40709a05de5a877 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb4300ea183421ab40709a05de5a877" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 33597 } } peers { permanent_uuid: "0788dbdabe574355bb323dcdc3be0058" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 41335 } } peers { permanent_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34235 } }
I20260709 13:24:25.495044 27950 raft_consensus.cc:385] T 028750ef0efa43e4b11ac10f3814902c P 9cb4300ea183421ab40709a05de5a877 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:24:25.495293 27950 raft_consensus.cc:740] T 028750ef0efa43e4b11ac10f3814902c P 9cb4300ea183421ab40709a05de5a877 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9cb4300ea183421ab40709a05de5a877, State: Initialized, Role: FOLLOWER
I20260709 13:24:25.495743 27950 consensus_queue.cc:260] T 028750ef0efa43e4b11ac10f3814902c P 9cb4300ea183421ab40709a05de5a877 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb4300ea183421ab40709a05de5a877" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 33597 } } peers { permanent_uuid: "0788dbdabe574355bb323dcdc3be0058" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 41335 } } peers { permanent_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34235 } }
I20260709 13:24:25.497004 27950 ts_tablet_manager.cc:1434] T 028750ef0efa43e4b11ac10f3814902c P 9cb4300ea183421ab40709a05de5a877: Time spent starting tablet: real 0.004s	user 0.004s	sys 0.000s
I20260709 13:24:25.602656 27955 raft_consensus.cc:493] T 2f38443d4b8d4e418c8f0520893a31e4 P 0788dbdabe574355bb323dcdc3be0058 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:24:25.603314 27955 raft_consensus.cc:515] T 2f38443d4b8d4e418c8f0520893a31e4 P 0788dbdabe574355bb323dcdc3be0058 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb4300ea183421ab40709a05de5a877" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 33597 } } peers { permanent_uuid: "0788dbdabe574355bb323dcdc3be0058" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 41335 } } peers { permanent_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34235 } }
I20260709 13:24:25.605183 27958 raft_consensus.cc:493] T 028750ef0efa43e4b11ac10f3814902c P 880fc4c7dff4417187b75cc1d9e9dcf5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:24:25.605248 27955 leader_election.cc:290] T 2f38443d4b8d4e418c8f0520893a31e4 P 0788dbdabe574355bb323dcdc3be0058 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9cb4300ea183421ab40709a05de5a877 (127.14.177.129:33597), 880fc4c7dff4417187b75cc1d9e9dcf5 (127.14.177.131:34235)
I20260709 13:24:25.606133 27762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f38443d4b8d4e418c8f0520893a31e4" candidate_uuid: "0788dbdabe574355bb323dcdc3be0058" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9cb4300ea183421ab40709a05de5a877" is_pre_election: true
I20260709 13:24:25.605826 27958 raft_consensus.cc:515] T 028750ef0efa43e4b11ac10f3814902c P 880fc4c7dff4417187b75cc1d9e9dcf5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb4300ea183421ab40709a05de5a877" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 33597 } } peers { permanent_uuid: "0788dbdabe574355bb323dcdc3be0058" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 41335 } } peers { permanent_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34235 } }
I20260709 13:24:25.606462 27910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f38443d4b8d4e418c8f0520893a31e4" candidate_uuid: "0788dbdabe574355bb323dcdc3be0058" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" is_pre_election: true
I20260709 13:24:25.606837 27762 raft_consensus.cc:2468] T 2f38443d4b8d4e418c8f0520893a31e4 P 9cb4300ea183421ab40709a05de5a877 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0788dbdabe574355bb323dcdc3be0058 in term 0.
I20260709 13:24:25.607219 27910 raft_consensus.cc:2468] T 2f38443d4b8d4e418c8f0520893a31e4 P 880fc4c7dff4417187b75cc1d9e9dcf5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0788dbdabe574355bb323dcdc3be0058 in term 0.
I20260709 13:24:25.608642 27958 leader_election.cc:290] T 028750ef0efa43e4b11ac10f3814902c P 880fc4c7dff4417187b75cc1d9e9dcf5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9cb4300ea183421ab40709a05de5a877 (127.14.177.129:33597), 0788dbdabe574355bb323dcdc3be0058 (127.14.177.130:41335)
I20260709 13:24:25.608841 27800 leader_election.cc:304] T 2f38443d4b8d4e418c8f0520893a31e4 P 0788dbdabe574355bb323dcdc3be0058 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0788dbdabe574355bb323dcdc3be0058, 9cb4300ea183421ab40709a05de5a877; no voters: 
I20260709 13:24:25.609411 27762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "028750ef0efa43e4b11ac10f3814902c" candidate_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9cb4300ea183421ab40709a05de5a877" is_pre_election: true
I20260709 13:24:25.609841 27955 raft_consensus.cc:2804] T 2f38443d4b8d4e418c8f0520893a31e4 P 0788dbdabe574355bb323dcdc3be0058 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:24:25.610100 27762 raft_consensus.cc:2468] T 028750ef0efa43e4b11ac10f3814902c P 9cb4300ea183421ab40709a05de5a877 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 880fc4c7dff4417187b75cc1d9e9dcf5 in term 0.
I20260709 13:24:25.610340 27955 raft_consensus.cc:493] T 2f38443d4b8d4e418c8f0520893a31e4 P 0788dbdabe574355bb323dcdc3be0058 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:24:25.610878 27955 raft_consensus.cc:3060] T 2f38443d4b8d4e418c8f0520893a31e4 P 0788dbdabe574355bb323dcdc3be0058 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:24:25.611361 27874 leader_election.cc:304] T 028750ef0efa43e4b11ac10f3814902c P 880fc4c7dff4417187b75cc1d9e9dcf5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 880fc4c7dff4417187b75cc1d9e9dcf5, 9cb4300ea183421ab40709a05de5a877; no voters: 
I20260709 13:24:25.611953 27836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "028750ef0efa43e4b11ac10f3814902c" candidate_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0788dbdabe574355bb323dcdc3be0058" is_pre_election: true
I20260709 13:24:25.612270 27958 raft_consensus.cc:2804] T 028750ef0efa43e4b11ac10f3814902c P 880fc4c7dff4417187b75cc1d9e9dcf5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:24:25.612603 27836 raft_consensus.cc:2468] T 028750ef0efa43e4b11ac10f3814902c P 0788dbdabe574355bb323dcdc3be0058 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 880fc4c7dff4417187b75cc1d9e9dcf5 in term 0.
I20260709 13:24:25.612830 27958 raft_consensus.cc:493] T 028750ef0efa43e4b11ac10f3814902c P 880fc4c7dff4417187b75cc1d9e9dcf5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:24:25.613284 27958 raft_consensus.cc:3060] T 028750ef0efa43e4b11ac10f3814902c P 880fc4c7dff4417187b75cc1d9e9dcf5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:24:25.619757 27955 raft_consensus.cc:515] T 2f38443d4b8d4e418c8f0520893a31e4 P 0788dbdabe574355bb323dcdc3be0058 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb4300ea183421ab40709a05de5a877" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 33597 } } peers { permanent_uuid: "0788dbdabe574355bb323dcdc3be0058" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 41335 } } peers { permanent_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34235 } }
I20260709 13:24:25.621255 27958 raft_consensus.cc:515] T 028750ef0efa43e4b11ac10f3814902c P 880fc4c7dff4417187b75cc1d9e9dcf5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb4300ea183421ab40709a05de5a877" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 33597 } } peers { permanent_uuid: "0788dbdabe574355bb323dcdc3be0058" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 41335 } } peers { permanent_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34235 } }
I20260709 13:24:25.622798 27955 leader_election.cc:290] T 2f38443d4b8d4e418c8f0520893a31e4 P 0788dbdabe574355bb323dcdc3be0058 [CANDIDATE]: Term 1 election: Requested vote from peers 9cb4300ea183421ab40709a05de5a877 (127.14.177.129:33597), 880fc4c7dff4417187b75cc1d9e9dcf5 (127.14.177.131:34235)
I20260709 13:24:25.623085 27762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f38443d4b8d4e418c8f0520893a31e4" candidate_uuid: "0788dbdabe574355bb323dcdc3be0058" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9cb4300ea183421ab40709a05de5a877"
I20260709 13:24:25.623418 27910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f38443d4b8d4e418c8f0520893a31e4" candidate_uuid: "0788dbdabe574355bb323dcdc3be0058" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5"
I20260709 13:24:25.623802 27958 leader_election.cc:290] T 028750ef0efa43e4b11ac10f3814902c P 880fc4c7dff4417187b75cc1d9e9dcf5 [CANDIDATE]: Term 1 election: Requested vote from peers 9cb4300ea183421ab40709a05de5a877 (127.14.177.129:33597), 0788dbdabe574355bb323dcdc3be0058 (127.14.177.130:41335)
I20260709 13:24:25.623967 27910 raft_consensus.cc:3060] T 2f38443d4b8d4e418c8f0520893a31e4 P 880fc4c7dff4417187b75cc1d9e9dcf5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:24:25.623848 27762 raft_consensus.cc:3060] T 2f38443d4b8d4e418c8f0520893a31e4 P 9cb4300ea183421ab40709a05de5a877 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:24:25.624821 27761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "028750ef0efa43e4b11ac10f3814902c" candidate_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9cb4300ea183421ab40709a05de5a877"
I20260709 13:24:25.625250 27836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "028750ef0efa43e4b11ac10f3814902c" candidate_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0788dbdabe574355bb323dcdc3be0058"
I20260709 13:24:25.625557 27761 raft_consensus.cc:3060] T 028750ef0efa43e4b11ac10f3814902c P 9cb4300ea183421ab40709a05de5a877 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:24:25.625993 27836 raft_consensus.cc:3060] T 028750ef0efa43e4b11ac10f3814902c P 0788dbdabe574355bb323dcdc3be0058 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:24:25.630906 27980 raft_consensus.cc:493] T 4554587582094286b577c9c9a4d3a7fc P 9cb4300ea183421ab40709a05de5a877 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:24:25.631569 27980 raft_consensus.cc:515] T 4554587582094286b577c9c9a4d3a7fc P 9cb4300ea183421ab40709a05de5a877 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb4300ea183421ab40709a05de5a877" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 33597 } } peers { permanent_uuid: "0788dbdabe574355bb323dcdc3be0058" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 41335 } } peers { permanent_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34235 } }
I20260709 13:24:25.633720 27761 raft_consensus.cc:2468] T 028750ef0efa43e4b11ac10f3814902c P 9cb4300ea183421ab40709a05de5a877 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 880fc4c7dff4417187b75cc1d9e9dcf5 in term 1.
I20260709 13:24:25.633940 27762 raft_consensus.cc:2468] T 2f38443d4b8d4e418c8f0520893a31e4 P 9cb4300ea183421ab40709a05de5a877 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0788dbdabe574355bb323dcdc3be0058 in term 1.
I20260709 13:24:25.634721 27836 raft_consensus.cc:2468] T 028750ef0efa43e4b11ac10f3814902c P 0788dbdabe574355bb323dcdc3be0058 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 880fc4c7dff4417187b75cc1d9e9dcf5 in term 1.
I20260709 13:24:25.635378 27874 leader_election.cc:304] T 028750ef0efa43e4b11ac10f3814902c P 880fc4c7dff4417187b75cc1d9e9dcf5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 880fc4c7dff4417187b75cc1d9e9dcf5, 9cb4300ea183421ab40709a05de5a877; no voters: 
I20260709 13:24:25.636135 27911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4554587582094286b577c9c9a4d3a7fc" candidate_uuid: "9cb4300ea183421ab40709a05de5a877" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" is_pre_election: true
I20260709 13:24:25.636204 27836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4554587582094286b577c9c9a4d3a7fc" candidate_uuid: "9cb4300ea183421ab40709a05de5a877" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0788dbdabe574355bb323dcdc3be0058" is_pre_election: true
I20260709 13:24:25.633877 27980 leader_election.cc:290] T 4554587582094286b577c9c9a4d3a7fc P 9cb4300ea183421ab40709a05de5a877 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0788dbdabe574355bb323dcdc3be0058 (127.14.177.130:41335), 880fc4c7dff4417187b75cc1d9e9dcf5 (127.14.177.131:34235)
I20260709 13:24:25.637245 27836 raft_consensus.cc:2393] T 4554587582094286b577c9c9a4d3a7fc P 0788dbdabe574355bb323dcdc3be0058 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9cb4300ea183421ab40709a05de5a877 in current term 1: Already voted for candidate 880fc4c7dff4417187b75cc1d9e9dcf5 in this term.
I20260709 13:24:25.636997 27800 leader_election.cc:304] T 2f38443d4b8d4e418c8f0520893a31e4 P 0788dbdabe574355bb323dcdc3be0058 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0788dbdabe574355bb323dcdc3be0058, 9cb4300ea183421ab40709a05de5a877; no voters: 
I20260709 13:24:25.637719 27958 raft_consensus.cc:2804] T 028750ef0efa43e4b11ac10f3814902c P 880fc4c7dff4417187b75cc1d9e9dcf5 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:24:25.638058 27955 raft_consensus.cc:2804] T 2f38443d4b8d4e418c8f0520893a31e4 P 0788dbdabe574355bb323dcdc3be0058 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:24:25.638860 27958 raft_consensus.cc:697] T 028750ef0efa43e4b11ac10f3814902c P 880fc4c7dff4417187b75cc1d9e9dcf5 [term 1 LEADER]: Becoming Leader. State: Replica: 880fc4c7dff4417187b75cc1d9e9dcf5, State: Running, Role: LEADER
I20260709 13:24:25.639501 27955 raft_consensus.cc:697] T 2f38443d4b8d4e418c8f0520893a31e4 P 0788dbdabe574355bb323dcdc3be0058 [term 1 LEADER]: Becoming Leader. State: Replica: 0788dbdabe574355bb323dcdc3be0058, State: Running, Role: LEADER
I20260709 13:24:25.639909 27727 leader_election.cc:304] T 4554587582094286b577c9c9a4d3a7fc P 9cb4300ea183421ab40709a05de5a877 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9cb4300ea183421ab40709a05de5a877; no voters: 0788dbdabe574355bb323dcdc3be0058, 880fc4c7dff4417187b75cc1d9e9dcf5
I20260709 13:24:25.640000 27958 consensus_queue.cc:237] T 028750ef0efa43e4b11ac10f3814902c P 880fc4c7dff4417187b75cc1d9e9dcf5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb4300ea183421ab40709a05de5a877" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 33597 } } peers { permanent_uuid: "0788dbdabe574355bb323dcdc3be0058" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 41335 } } peers { permanent_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34235 } }
I20260709 13:24:25.640427 27910 raft_consensus.cc:2468] T 2f38443d4b8d4e418c8f0520893a31e4 P 880fc4c7dff4417187b75cc1d9e9dcf5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0788dbdabe574355bb323dcdc3be0058 in term 1.
I20260709 13:24:25.640673 27955 consensus_queue.cc:237] T 2f38443d4b8d4e418c8f0520893a31e4 P 0788dbdabe574355bb323dcdc3be0058 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb4300ea183421ab40709a05de5a877" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 33597 } } peers { permanent_uuid: "0788dbdabe574355bb323dcdc3be0058" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 41335 } } peers { permanent_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34235 } }
I20260709 13:24:25.641481 27980 raft_consensus.cc:3060] T 4554587582094286b577c9c9a4d3a7fc P 9cb4300ea183421ab40709a05de5a877 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:24:25.650463 27658 catalog_manager.cc:5697] T 028750ef0efa43e4b11ac10f3814902c P 880fc4c7dff4417187b75cc1d9e9dcf5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 880fc4c7dff4417187b75cc1d9e9dcf5 (127.14.177.131). New cstate: current_term: 1 leader_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb4300ea183421ab40709a05de5a877" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 33597 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0788dbdabe574355bb323dcdc3be0058" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 41335 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34235 } health_report { overall_health: HEALTHY } } }
I20260709 13:24:25.652853 27980 raft_consensus.cc:2749] T 4554587582094286b577c9c9a4d3a7fc P 9cb4300ea183421ab40709a05de5a877 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:24:25.653441 27660 catalog_manager.cc:5697] T 2f38443d4b8d4e418c8f0520893a31e4 P 0788dbdabe574355bb323dcdc3be0058 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0788dbdabe574355bb323dcdc3be0058 (127.14.177.130). New cstate: current_term: 1 leader_uuid: "0788dbdabe574355bb323dcdc3be0058" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb4300ea183421ab40709a05de5a877" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 33597 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0788dbdabe574355bb323dcdc3be0058" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 41335 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34235 } health_report { overall_health: UNKNOWN } } }
I20260709 13:24:25.700116 27958 consensus_queue.cc:1048] T 4554587582094286b577c9c9a4d3a7fc P 880fc4c7dff4417187b75cc1d9e9dcf5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0788dbdabe574355bb323dcdc3be0058" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 41335 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:24:25.708271 27990 raft_consensus.cc:493] T 20ef9c03fcd94115be0b592d85f89421 P 880fc4c7dff4417187b75cc1d9e9dcf5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:24:25.708736 27990 raft_consensus.cc:515] T 20ef9c03fcd94115be0b592d85f89421 P 880fc4c7dff4417187b75cc1d9e9dcf5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb4300ea183421ab40709a05de5a877" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 33597 } } peers { permanent_uuid: "0788dbdabe574355bb323dcdc3be0058" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 41335 } } peers { permanent_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34235 } }
I20260709 13:24:25.710511 27990 leader_election.cc:290] T 20ef9c03fcd94115be0b592d85f89421 P 880fc4c7dff4417187b75cc1d9e9dcf5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9cb4300ea183421ab40709a05de5a877 (127.14.177.129:33597), 0788dbdabe574355bb323dcdc3be0058 (127.14.177.130:41335)
I20260709 13:24:25.711356 27761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20ef9c03fcd94115be0b592d85f89421" candidate_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9cb4300ea183421ab40709a05de5a877" is_pre_election: true
I20260709 13:24:25.713478 27836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20ef9c03fcd94115be0b592d85f89421" candidate_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0788dbdabe574355bb323dcdc3be0058" is_pre_election: true
I20260709 13:24:25.715158 27836 raft_consensus.cc:2393] T 20ef9c03fcd94115be0b592d85f89421 P 0788dbdabe574355bb323dcdc3be0058 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 880fc4c7dff4417187b75cc1d9e9dcf5 in current term 1: Already voted for candidate 9cb4300ea183421ab40709a05de5a877 in this term.
I20260709 13:24:25.716252 27876 leader_election.cc:304] T 20ef9c03fcd94115be0b592d85f89421 P 880fc4c7dff4417187b75cc1d9e9dcf5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 880fc4c7dff4417187b75cc1d9e9dcf5; no voters: 0788dbdabe574355bb323dcdc3be0058, 9cb4300ea183421ab40709a05de5a877
I20260709 13:24:25.717450 27987 raft_consensus.cc:3060] T 20ef9c03fcd94115be0b592d85f89421 P 880fc4c7dff4417187b75cc1d9e9dcf5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:24:25.720356 27990 consensus_queue.cc:1048] T 4554587582094286b577c9c9a4d3a7fc P 880fc4c7dff4417187b75cc1d9e9dcf5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9cb4300ea183421ab40709a05de5a877" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 33597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:24:25.725144 27987 raft_consensus.cc:2749] T 20ef9c03fcd94115be0b592d85f89421 P 880fc4c7dff4417187b75cc1d9e9dcf5 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:24:25.730482 27978 consensus_queue.cc:1048] T db487682b03b4a738f664fa1459714e5 P 9cb4300ea183421ab40709a05de5a877 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0788dbdabe574355bb323dcdc3be0058" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 41335 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:24:25.741957 27959 consensus_queue.cc:1048] T db487682b03b4a738f664fa1459714e5 P 9cb4300ea183421ab40709a05de5a877 [LEADER]: Connected to new peer: Peer: permanent_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34235 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:24:25.776559 27955 consensus_queue.cc:1048] T f3e47f5448ff47a382c61886f1a8cc7b P 0788dbdabe574355bb323dcdc3be0058 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9cb4300ea183421ab40709a05de5a877" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 33597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:24:25.788301 28014 consensus_queue.cc:1048] T f3e47f5448ff47a382c61886f1a8cc7b P 0788dbdabe574355bb323dcdc3be0058 [LEADER]: Connected to new peer: Peer: permanent_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34235 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:24:25.821161 27837 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f3e47f5448ff47a382c61886f1a8cc7b"
dest_uuid: "0788dbdabe574355bb323dcdc3be0058"
mode: GRACEFUL
new_leader_uuid: "9cb4300ea183421ab40709a05de5a877"
 from {username='slave'} at 127.0.0.1:48802
I20260709 13:24:25.821779 27837 raft_consensus.cc:606] T f3e47f5448ff47a382c61886f1a8cc7b P 0788dbdabe574355bb323dcdc3be0058 [term 1 LEADER]: Received request to transfer leadership to TS 9cb4300ea183421ab40709a05de5a877
I20260709 13:24:25.836259 27910 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e7366c36a1154d4d94281a90fde291f8"
dest_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5"
mode: GRACEFUL
new_leader_uuid: "0788dbdabe574355bb323dcdc3be0058"
 from {username='slave'} at 127.0.0.1:58342
I20260709 13:24:25.836836 27910 raft_consensus.cc:606] T e7366c36a1154d4d94281a90fde291f8 P 880fc4c7dff4417187b75cc1d9e9dcf5 [term 1 LEADER]: Received request to transfer leadership to TS 0788dbdabe574355bb323dcdc3be0058
I20260709 13:24:25.842391 27910 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4554587582094286b577c9c9a4d3a7fc"
dest_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5"
mode: GRACEFUL
new_leader_uuid: "0788dbdabe574355bb323dcdc3be0058"
 from {username='slave'} at 127.0.0.1:58342
I20260709 13:24:25.842914 27910 raft_consensus.cc:606] T 4554587582094286b577c9c9a4d3a7fc P 880fc4c7dff4417187b75cc1d9e9dcf5 [term 1 LEADER]: Received request to transfer leadership to TS 0788dbdabe574355bb323dcdc3be0058
I20260709 13:24:25.848215 27980 consensus_queue.cc:1048] T 20ef9c03fcd94115be0b592d85f89421 P 9cb4300ea183421ab40709a05de5a877 [LEADER]: Connected to new peer: Peer: permanent_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34235 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:24:25.848311 27910 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "cd1140068fc8455280934d7a30c4f559"
dest_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5"
mode: GRACEFUL
new_leader_uuid: "0788dbdabe574355bb323dcdc3be0058"
 from {username='slave'} at 127.0.0.1:58342
I20260709 13:24:25.848975 27910 raft_consensus.cc:606] T cd1140068fc8455280934d7a30c4f559 P 880fc4c7dff4417187b75cc1d9e9dcf5 [term 1 LEADER]: Received request to transfer leadership to TS 0788dbdabe574355bb323dcdc3be0058
I20260709 13:24:25.862445 27978 consensus_queue.cc:1048] T 20ef9c03fcd94115be0b592d85f89421 P 9cb4300ea183421ab40709a05de5a877 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0788dbdabe574355bb323dcdc3be0058" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 41335 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:24:25.905247 27978 consensus_queue.cc:1048] T d16bbb66a57e4079897aaa2c4e6ba144 P 9cb4300ea183421ab40709a05de5a877 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0788dbdabe574355bb323dcdc3be0058" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 41335 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:24:25.919752 27978 consensus_queue.cc:1048] T d16bbb66a57e4079897aaa2c4e6ba144 P 9cb4300ea183421ab40709a05de5a877 [LEADER]: Connected to new peer: Peer: permanent_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34235 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:24:25.937922 27987 consensus_queue.cc:1048] T e7366c36a1154d4d94281a90fde291f8 P 880fc4c7dff4417187b75cc1d9e9dcf5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0788dbdabe574355bb323dcdc3be0058" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 41335 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:24:25.945398 27990 raft_consensus.cc:993] T e7366c36a1154d4d94281a90fde291f8 P 880fc4c7dff4417187b75cc1d9e9dcf5: : Instructing follower 0788dbdabe574355bb323dcdc3be0058 to start an election
I20260709 13:24:25.946038 27958 raft_consensus.cc:1081] T e7366c36a1154d4d94281a90fde291f8 P 880fc4c7dff4417187b75cc1d9e9dcf5 [term 1 LEADER]: Signalling peer 0788dbdabe574355bb323dcdc3be0058 to start an election
I20260709 13:24:25.949153 27836 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e7366c36a1154d4d94281a90fde291f8"
dest_uuid: "0788dbdabe574355bb323dcdc3be0058"
 from {username='slave'} at 127.0.0.1:48800
I20260709 13:24:25.950347 27836 raft_consensus.cc:493] T e7366c36a1154d4d94281a90fde291f8 P 0788dbdabe574355bb323dcdc3be0058 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:24:25.950793 27836 raft_consensus.cc:3060] T e7366c36a1154d4d94281a90fde291f8 P 0788dbdabe574355bb323dcdc3be0058 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:24:25.952920 27987 consensus_queue.cc:1048] T e7366c36a1154d4d94281a90fde291f8 P 880fc4c7dff4417187b75cc1d9e9dcf5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9cb4300ea183421ab40709a05de5a877" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 33597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:24:25.957918 27836 raft_consensus.cc:515] T e7366c36a1154d4d94281a90fde291f8 P 0788dbdabe574355bb323dcdc3be0058 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb4300ea183421ab40709a05de5a877" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 33597 } } peers { permanent_uuid: "0788dbdabe574355bb323dcdc3be0058" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 41335 } } peers { permanent_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34235 } }
I20260709 13:24:25.964111 27761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7366c36a1154d4d94281a90fde291f8" candidate_uuid: "0788dbdabe574355bb323dcdc3be0058" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "9cb4300ea183421ab40709a05de5a877"
I20260709 13:24:25.964804 27761 raft_consensus.cc:3060] T e7366c36a1154d4d94281a90fde291f8 P 9cb4300ea183421ab40709a05de5a877 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:24:25.968532 27990 consensus_queue.cc:1048] T cd1140068fc8455280934d7a30c4f559 P 880fc4c7dff4417187b75cc1d9e9dcf5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9cb4300ea183421ab40709a05de5a877" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 33597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:24:25.970099 27836 leader_election.cc:290] T e7366c36a1154d4d94281a90fde291f8 P 0788dbdabe574355bb323dcdc3be0058 [CANDIDATE]: Term 2 election: Requested vote from peers 9cb4300ea183421ab40709a05de5a877 (127.14.177.129:33597), 880fc4c7dff4417187b75cc1d9e9dcf5 (127.14.177.131:34235)
I20260709 13:24:25.971496 27910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7366c36a1154d4d94281a90fde291f8" candidate_uuid: "0788dbdabe574355bb323dcdc3be0058" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5"
I20260709 13:24:25.972276 27910 raft_consensus.cc:3055] T e7366c36a1154d4d94281a90fde291f8 P 880fc4c7dff4417187b75cc1d9e9dcf5 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:24:25.972640 27910 raft_consensus.cc:740] T e7366c36a1154d4d94281a90fde291f8 P 880fc4c7dff4417187b75cc1d9e9dcf5 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 880fc4c7dff4417187b75cc1d9e9dcf5, State: Running, Role: LEADER
I20260709 13:24:25.972834 27987 consensus_queue.cc:1048] T cd1140068fc8455280934d7a30c4f559 P 880fc4c7dff4417187b75cc1d9e9dcf5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0788dbdabe574355bb323dcdc3be0058" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 41335 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:24:25.973469 27761 raft_consensus.cc:2468] T e7366c36a1154d4d94281a90fde291f8 P 9cb4300ea183421ab40709a05de5a877 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0788dbdabe574355bb323dcdc3be0058 in term 2.
I20260709 13:24:25.974817 27800 leader_election.cc:304] T e7366c36a1154d4d94281a90fde291f8 P 0788dbdabe574355bb323dcdc3be0058 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0788dbdabe574355bb323dcdc3be0058, 9cb4300ea183421ab40709a05de5a877; no voters: 
I20260709 13:24:25.973717 27910 consensus_queue.cc:260] T e7366c36a1154d4d94281a90fde291f8 P 880fc4c7dff4417187b75cc1d9e9dcf5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb4300ea183421ab40709a05de5a877" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 33597 } } peers { permanent_uuid: "0788dbdabe574355bb323dcdc3be0058" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 41335 } } peers { permanent_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34235 } }
I20260709 13:24:25.980098 28014 raft_consensus.cc:2804] T e7366c36a1154d4d94281a90fde291f8 P 0788dbdabe574355bb323dcdc3be0058 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:24:25.980845 28014 raft_consensus.cc:697] T e7366c36a1154d4d94281a90fde291f8 P 0788dbdabe574355bb323dcdc3be0058 [term 2 LEADER]: Becoming Leader. State: Replica: 0788dbdabe574355bb323dcdc3be0058, State: Running, Role: LEADER
I20260709 13:24:25.981365 27910 raft_consensus.cc:3060] T e7366c36a1154d4d94281a90fde291f8 P 880fc4c7dff4417187b75cc1d9e9dcf5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:24:25.982012 28014 consensus_queue.cc:237] T e7366c36a1154d4d94281a90fde291f8 P 0788dbdabe574355bb323dcdc3be0058 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb4300ea183421ab40709a05de5a877" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 33597 } } peers { permanent_uuid: "0788dbdabe574355bb323dcdc3be0058" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 41335 } } peers { permanent_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34235 } }
I20260709 13:24:25.995394 27990 raft_consensus.cc:993] T cd1140068fc8455280934d7a30c4f559 P 880fc4c7dff4417187b75cc1d9e9dcf5: : Instructing follower 0788dbdabe574355bb323dcdc3be0058 to start an election
I20260709 13:24:25.995904 27990 raft_consensus.cc:1081] T cd1140068fc8455280934d7a30c4f559 P 880fc4c7dff4417187b75cc1d9e9dcf5 [term 1 LEADER]: Signalling peer 0788dbdabe574355bb323dcdc3be0058 to start an election
I20260709 13:24:25.998418 27835 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "cd1140068fc8455280934d7a30c4f559"
dest_uuid: "0788dbdabe574355bb323dcdc3be0058"
 from {username='slave'} at 127.0.0.1:48800
I20260709 13:24:25.999259 27835 raft_consensus.cc:493] T cd1140068fc8455280934d7a30c4f559 P 0788dbdabe574355bb323dcdc3be0058 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:24:25.999677 27835 raft_consensus.cc:3060] T cd1140068fc8455280934d7a30c4f559 P 0788dbdabe574355bb323dcdc3be0058 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:24:25.995810 27910 raft_consensus.cc:2468] T e7366c36a1154d4d94281a90fde291f8 P 880fc4c7dff4417187b75cc1d9e9dcf5 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0788dbdabe574355bb323dcdc3be0058 in term 2.
I20260709 13:24:26.008878 27835 raft_consensus.cc:515] T cd1140068fc8455280934d7a30c4f559 P 0788dbdabe574355bb323dcdc3be0058 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb4300ea183421ab40709a05de5a877" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 33597 } } peers { permanent_uuid: "0788dbdabe574355bb323dcdc3be0058" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 41335 } } peers { permanent_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34235 } }
I20260709 13:24:26.008297 27657 catalog_manager.cc:5697] T e7366c36a1154d4d94281a90fde291f8 P 0788dbdabe574355bb323dcdc3be0058 reported cstate change: term changed from 1 to 2, leader changed from 880fc4c7dff4417187b75cc1d9e9dcf5 (127.14.177.131) to 0788dbdabe574355bb323dcdc3be0058 (127.14.177.130). New cstate: current_term: 2 leader_uuid: "0788dbdabe574355bb323dcdc3be0058" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb4300ea183421ab40709a05de5a877" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 33597 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0788dbdabe574355bb323dcdc3be0058" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 41335 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34235 } health_report { overall_health: UNKNOWN } } }
I20260709 13:24:26.011991 27762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd1140068fc8455280934d7a30c4f559" candidate_uuid: "0788dbdabe574355bb323dcdc3be0058" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "9cb4300ea183421ab40709a05de5a877"
I20260709 13:24:26.012550 27910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd1140068fc8455280934d7a30c4f559" candidate_uuid: "0788dbdabe574355bb323dcdc3be0058" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5"
I20260709 13:24:26.012915 27762 raft_consensus.cc:3060] T cd1140068fc8455280934d7a30c4f559 P 9cb4300ea183421ab40709a05de5a877 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:24:26.013275 27910 raft_consensus.cc:3055] T cd1140068fc8455280934d7a30c4f559 P 880fc4c7dff4417187b75cc1d9e9dcf5 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:24:26.013722 27910 raft_consensus.cc:740] T cd1140068fc8455280934d7a30c4f559 P 880fc4c7dff4417187b75cc1d9e9dcf5 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 880fc4c7dff4417187b75cc1d9e9dcf5, State: Running, Role: LEADER
I20260709 13:24:26.012164 27835 leader_election.cc:290] T cd1140068fc8455280934d7a30c4f559 P 0788dbdabe574355bb323dcdc3be0058 [CANDIDATE]: Term 2 election: Requested vote from peers 9cb4300ea183421ab40709a05de5a877 (127.14.177.129:33597), 880fc4c7dff4417187b75cc1d9e9dcf5 (127.14.177.131:34235)
I20260709 13:24:26.017504 27910 consensus_queue.cc:260] T cd1140068fc8455280934d7a30c4f559 P 880fc4c7dff4417187b75cc1d9e9dcf5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb4300ea183421ab40709a05de5a877" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 33597 } } peers { permanent_uuid: "0788dbdabe574355bb323dcdc3be0058" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 41335 } } peers { permanent_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34235 } }
I20260709 13:24:26.018942 27910 raft_consensus.cc:3060] T cd1140068fc8455280934d7a30c4f559 P 880fc4c7dff4417187b75cc1d9e9dcf5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:24:26.022197 27762 raft_consensus.cc:2468] T cd1140068fc8455280934d7a30c4f559 P 9cb4300ea183421ab40709a05de5a877 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0788dbdabe574355bb323dcdc3be0058 in term 2.
I20260709 13:24:26.026207 27800 leader_election.cc:304] T cd1140068fc8455280934d7a30c4f559 P 0788dbdabe574355bb323dcdc3be0058 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0788dbdabe574355bb323dcdc3be0058, 9cb4300ea183421ab40709a05de5a877; no voters: 
I20260709 13:24:26.026676 27910 raft_consensus.cc:2468] T cd1140068fc8455280934d7a30c4f559 P 880fc4c7dff4417187b75cc1d9e9dcf5 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0788dbdabe574355bb323dcdc3be0058 in term 2.
I20260709 13:24:26.026777 27987 consensus_queue.cc:1048] T 028750ef0efa43e4b11ac10f3814902c P 880fc4c7dff4417187b75cc1d9e9dcf5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9cb4300ea183421ab40709a05de5a877" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 33597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:24:26.027251 28014 raft_consensus.cc:2804] T cd1140068fc8455280934d7a30c4f559 P 0788dbdabe574355bb323dcdc3be0058 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:24:26.027940 28014 raft_consensus.cc:697] T cd1140068fc8455280934d7a30c4f559 P 0788dbdabe574355bb323dcdc3be0058 [term 2 LEADER]: Becoming Leader. State: Replica: 0788dbdabe574355bb323dcdc3be0058, State: Running, Role: LEADER
I20260709 13:24:26.028890 28014 consensus_queue.cc:237] T cd1140068fc8455280934d7a30c4f559 P 0788dbdabe574355bb323dcdc3be0058 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb4300ea183421ab40709a05de5a877" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 33597 } } peers { permanent_uuid: "0788dbdabe574355bb323dcdc3be0058" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 41335 } } peers { permanent_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34235 } }
I20260709 13:24:26.038306 27657 catalog_manager.cc:5697] T cd1140068fc8455280934d7a30c4f559 P 0788dbdabe574355bb323dcdc3be0058 reported cstate change: term changed from 1 to 2, leader changed from 880fc4c7dff4417187b75cc1d9e9dcf5 (127.14.177.131) to 0788dbdabe574355bb323dcdc3be0058 (127.14.177.130). New cstate: current_term: 2 leader_uuid: "0788dbdabe574355bb323dcdc3be0058" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb4300ea183421ab40709a05de5a877" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 33597 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0788dbdabe574355bb323dcdc3be0058" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 41335 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34235 } health_report { overall_health: UNKNOWN } } }
I20260709 13:24:26.044486 27990 consensus_queue.cc:1048] T 028750ef0efa43e4b11ac10f3814902c P 880fc4c7dff4417187b75cc1d9e9dcf5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0788dbdabe574355bb323dcdc3be0058" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 41335 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:24:26.094659 28014 consensus_queue.cc:1048] T 2f38443d4b8d4e418c8f0520893a31e4 P 0788dbdabe574355bb323dcdc3be0058 [LEADER]: Connected to new peer: Peer: permanent_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34235 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:24:26.106002 27982 consensus_queue.cc:1048] T 2f38443d4b8d4e418c8f0520893a31e4 P 0788dbdabe574355bb323dcdc3be0058 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9cb4300ea183421ab40709a05de5a877" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 33597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:24:26.291561 27990 raft_consensus.cc:993] T 4554587582094286b577c9c9a4d3a7fc P 880fc4c7dff4417187b75cc1d9e9dcf5: : Instructing follower 0788dbdabe574355bb323dcdc3be0058 to start an election
I20260709 13:24:26.292023 27987 raft_consensus.cc:1081] T 4554587582094286b577c9c9a4d3a7fc P 880fc4c7dff4417187b75cc1d9e9dcf5 [term 1 LEADER]: Signalling peer 0788dbdabe574355bb323dcdc3be0058 to start an election
I20260709 13:24:26.293812 27835 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "4554587582094286b577c9c9a4d3a7fc"
dest_uuid: "0788dbdabe574355bb323dcdc3be0058"
 from {username='slave'} at 127.0.0.1:48800
I20260709 13:24:26.294279 27835 raft_consensus.cc:493] T 4554587582094286b577c9c9a4d3a7fc P 0788dbdabe574355bb323dcdc3be0058 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:24:26.294545 27835 raft_consensus.cc:3060] T 4554587582094286b577c9c9a4d3a7fc P 0788dbdabe574355bb323dcdc3be0058 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:24:26.300380 27835 raft_consensus.cc:515] T 4554587582094286b577c9c9a4d3a7fc P 0788dbdabe574355bb323dcdc3be0058 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb4300ea183421ab40709a05de5a877" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 33597 } } peers { permanent_uuid: "0788dbdabe574355bb323dcdc3be0058" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 41335 } } peers { permanent_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34235 } }
I20260709 13:24:26.302240 27835 leader_election.cc:290] T 4554587582094286b577c9c9a4d3a7fc P 0788dbdabe574355bb323dcdc3be0058 [CANDIDATE]: Term 2 election: Requested vote from peers 9cb4300ea183421ab40709a05de5a877 (127.14.177.129:33597), 880fc4c7dff4417187b75cc1d9e9dcf5 (127.14.177.131:34235)
I20260709 13:24:26.303370 27910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4554587582094286b577c9c9a4d3a7fc" candidate_uuid: "0788dbdabe574355bb323dcdc3be0058" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5"
I20260709 13:24:26.303311 27760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4554587582094286b577c9c9a4d3a7fc" candidate_uuid: "0788dbdabe574355bb323dcdc3be0058" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "9cb4300ea183421ab40709a05de5a877"
I20260709 13:24:26.304157 27910 raft_consensus.cc:3055] T 4554587582094286b577c9c9a4d3a7fc P 880fc4c7dff4417187b75cc1d9e9dcf5 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:24:26.304234 27760 raft_consensus.cc:3060] T 4554587582094286b577c9c9a4d3a7fc P 9cb4300ea183421ab40709a05de5a877 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:24:26.304582 27910 raft_consensus.cc:740] T 4554587582094286b577c9c9a4d3a7fc P 880fc4c7dff4417187b75cc1d9e9dcf5 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 880fc4c7dff4417187b75cc1d9e9dcf5, State: Running, Role: LEADER
I20260709 13:24:26.305437 27910 consensus_queue.cc:260] T 4554587582094286b577c9c9a4d3a7fc P 880fc4c7dff4417187b75cc1d9e9dcf5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb4300ea183421ab40709a05de5a877" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 33597 } } peers { permanent_uuid: "0788dbdabe574355bb323dcdc3be0058" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 41335 } } peers { permanent_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34235 } }
I20260709 13:24:26.306782 27910 raft_consensus.cc:3060] T 4554587582094286b577c9c9a4d3a7fc P 880fc4c7dff4417187b75cc1d9e9dcf5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:24:26.312513 27760 raft_consensus.cc:2468] T 4554587582094286b577c9c9a4d3a7fc P 9cb4300ea183421ab40709a05de5a877 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0788dbdabe574355bb323dcdc3be0058 in term 2.
I20260709 13:24:26.314071 27800 leader_election.cc:304] T 4554587582094286b577c9c9a4d3a7fc P 0788dbdabe574355bb323dcdc3be0058 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0788dbdabe574355bb323dcdc3be0058, 9cb4300ea183421ab40709a05de5a877; no voters: 
I20260709 13:24:26.314314 27910 raft_consensus.cc:2468] T 4554587582094286b577c9c9a4d3a7fc P 880fc4c7dff4417187b75cc1d9e9dcf5 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0788dbdabe574355bb323dcdc3be0058 in term 2.
I20260709 13:24:26.315003 27982 raft_consensus.cc:2804] T 4554587582094286b577c9c9a4d3a7fc P 0788dbdabe574355bb323dcdc3be0058 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:24:26.315696 27982 raft_consensus.cc:697] T 4554587582094286b577c9c9a4d3a7fc P 0788dbdabe574355bb323dcdc3be0058 [term 2 LEADER]: Becoming Leader. State: Replica: 0788dbdabe574355bb323dcdc3be0058, State: Running, Role: LEADER
I20260709 13:24:26.316629 27982 consensus_queue.cc:237] T 4554587582094286b577c9c9a4d3a7fc P 0788dbdabe574355bb323dcdc3be0058 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb4300ea183421ab40709a05de5a877" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 33597 } } peers { permanent_uuid: "0788dbdabe574355bb323dcdc3be0058" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 41335 } } peers { permanent_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34235 } }
I20260709 13:24:26.323509 27659 catalog_manager.cc:5697] T 4554587582094286b577c9c9a4d3a7fc P 0788dbdabe574355bb323dcdc3be0058 reported cstate change: term changed from 1 to 2, leader changed from 880fc4c7dff4417187b75cc1d9e9dcf5 (127.14.177.131) to 0788dbdabe574355bb323dcdc3be0058 (127.14.177.130). New cstate: current_term: 2 leader_uuid: "0788dbdabe574355bb323dcdc3be0058" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb4300ea183421ab40709a05de5a877" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 33597 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0788dbdabe574355bb323dcdc3be0058" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 41335 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34235 } health_report { overall_health: UNKNOWN } } }
I20260709 13:24:26.343670 27955 raft_consensus.cc:993] T f3e47f5448ff47a382c61886f1a8cc7b P 0788dbdabe574355bb323dcdc3be0058: : Instructing follower 9cb4300ea183421ab40709a05de5a877 to start an election
I20260709 13:24:26.344172 27982 raft_consensus.cc:1081] T f3e47f5448ff47a382c61886f1a8cc7b P 0788dbdabe574355bb323dcdc3be0058 [term 1 LEADER]: Signalling peer 9cb4300ea183421ab40709a05de5a877 to start an election
I20260709 13:24:26.345784 27760 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "f3e47f5448ff47a382c61886f1a8cc7b"
dest_uuid: "9cb4300ea183421ab40709a05de5a877"
 from {username='slave'} at 127.0.0.1:59038
I20260709 13:24:26.346241 27760 raft_consensus.cc:493] T f3e47f5448ff47a382c61886f1a8cc7b P 9cb4300ea183421ab40709a05de5a877 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:24:26.346494 27760 raft_consensus.cc:3060] T f3e47f5448ff47a382c61886f1a8cc7b P 9cb4300ea183421ab40709a05de5a877 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:24:26.351810 27760 raft_consensus.cc:515] T f3e47f5448ff47a382c61886f1a8cc7b P 9cb4300ea183421ab40709a05de5a877 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb4300ea183421ab40709a05de5a877" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 33597 } } peers { permanent_uuid: "0788dbdabe574355bb323dcdc3be0058" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 41335 } } peers { permanent_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34235 } }
I20260709 13:24:26.354512 27760 leader_election.cc:290] T f3e47f5448ff47a382c61886f1a8cc7b P 9cb4300ea183421ab40709a05de5a877 [CANDIDATE]: Term 2 election: Requested vote from peers 0788dbdabe574355bb323dcdc3be0058 (127.14.177.130:41335), 880fc4c7dff4417187b75cc1d9e9dcf5 (127.14.177.131:34235)
I20260709 13:24:26.354971 27835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3e47f5448ff47a382c61886f1a8cc7b" candidate_uuid: "9cb4300ea183421ab40709a05de5a877" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "0788dbdabe574355bb323dcdc3be0058"
I20260709 13:24:26.355690 27835 raft_consensus.cc:3055] T f3e47f5448ff47a382c61886f1a8cc7b P 0788dbdabe574355bb323dcdc3be0058 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:24:26.355687 27910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3e47f5448ff47a382c61886f1a8cc7b" candidate_uuid: "9cb4300ea183421ab40709a05de5a877" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5"
I20260709 13:24:26.356356 27835 raft_consensus.cc:740] T f3e47f5448ff47a382c61886f1a8cc7b P 0788dbdabe574355bb323dcdc3be0058 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 0788dbdabe574355bb323dcdc3be0058, State: Running, Role: LEADER
I20260709 13:24:26.356603 27910 raft_consensus.cc:3060] T f3e47f5448ff47a382c61886f1a8cc7b P 880fc4c7dff4417187b75cc1d9e9dcf5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:24:26.357326 27835 consensus_queue.cc:260] T f3e47f5448ff47a382c61886f1a8cc7b P 0788dbdabe574355bb323dcdc3be0058 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb4300ea183421ab40709a05de5a877" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 33597 } } peers { permanent_uuid: "0788dbdabe574355bb323dcdc3be0058" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 41335 } } peers { permanent_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34235 } }
I20260709 13:24:26.358804 27835 raft_consensus.cc:3060] T f3e47f5448ff47a382c61886f1a8cc7b P 0788dbdabe574355bb323dcdc3be0058 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:24:26.363803 27910 raft_consensus.cc:2468] T f3e47f5448ff47a382c61886f1a8cc7b P 880fc4c7dff4417187b75cc1d9e9dcf5 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9cb4300ea183421ab40709a05de5a877 in term 2.
I20260709 13:24:26.365113 27835 raft_consensus.cc:2468] T f3e47f5448ff47a382c61886f1a8cc7b P 0788dbdabe574355bb323dcdc3be0058 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9cb4300ea183421ab40709a05de5a877 in term 2.
I20260709 13:24:26.365316 27727 leader_election.cc:304] T f3e47f5448ff47a382c61886f1a8cc7b P 9cb4300ea183421ab40709a05de5a877 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 880fc4c7dff4417187b75cc1d9e9dcf5, 9cb4300ea183421ab40709a05de5a877; no voters: 
I20260709 13:24:26.366302 27978 raft_consensus.cc:2804] T f3e47f5448ff47a382c61886f1a8cc7b P 9cb4300ea183421ab40709a05de5a877 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:24:26.366811 27978 raft_consensus.cc:697] T f3e47f5448ff47a382c61886f1a8cc7b P 9cb4300ea183421ab40709a05de5a877 [term 2 LEADER]: Becoming Leader. State: Replica: 9cb4300ea183421ab40709a05de5a877, State: Running, Role: LEADER
I20260709 13:24:26.367599 27978 consensus_queue.cc:237] T f3e47f5448ff47a382c61886f1a8cc7b P 9cb4300ea183421ab40709a05de5a877 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb4300ea183421ab40709a05de5a877" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 33597 } } peers { permanent_uuid: "0788dbdabe574355bb323dcdc3be0058" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 41335 } } peers { permanent_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34235 } }
I20260709 13:24:26.374795 27659 catalog_manager.cc:5697] T f3e47f5448ff47a382c61886f1a8cc7b P 9cb4300ea183421ab40709a05de5a877 reported cstate change: term changed from 1 to 2, leader changed from 0788dbdabe574355bb323dcdc3be0058 (127.14.177.130) to 9cb4300ea183421ab40709a05de5a877 (127.14.177.129). New cstate: current_term: 2 leader_uuid: "9cb4300ea183421ab40709a05de5a877" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb4300ea183421ab40709a05de5a877" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 33597 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0788dbdabe574355bb323dcdc3be0058" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 41335 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34235 } health_report { overall_health: UNKNOWN } } }
I20260709 13:24:26.435380 27910 raft_consensus.cc:1275] T e7366c36a1154d4d94281a90fde291f8 P 880fc4c7dff4417187b75cc1d9e9dcf5 [term 2 FOLLOWER]: Refusing update from remote peer 0788dbdabe574355bb323dcdc3be0058: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:24:26.436800 27982 consensus_queue.cc:1048] T e7366c36a1154d4d94281a90fde291f8 P 0788dbdabe574355bb323dcdc3be0058 [LEADER]: Connected to new peer: Peer: permanent_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34235 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:24:26.447256 27760 raft_consensus.cc:1275] T e7366c36a1154d4d94281a90fde291f8 P 9cb4300ea183421ab40709a05de5a877 [term 2 FOLLOWER]: Refusing update from remote peer 0788dbdabe574355bb323dcdc3be0058: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:24:26.449373 28014 consensus_queue.cc:1048] T e7366c36a1154d4d94281a90fde291f8 P 0788dbdabe574355bb323dcdc3be0058 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9cb4300ea183421ab40709a05de5a877" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 33597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:24:26.457588 27760 raft_consensus.cc:1275] T cd1140068fc8455280934d7a30c4f559 P 9cb4300ea183421ab40709a05de5a877 [term 2 FOLLOWER]: Refusing update from remote peer 0788dbdabe574355bb323dcdc3be0058: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:24:26.459151 28014 consensus_queue.cc:1048] T cd1140068fc8455280934d7a30c4f559 P 0788dbdabe574355bb323dcdc3be0058 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9cb4300ea183421ab40709a05de5a877" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 33597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:24:26.468180 27910 raft_consensus.cc:1275] T cd1140068fc8455280934d7a30c4f559 P 880fc4c7dff4417187b75cc1d9e9dcf5 [term 2 FOLLOWER]: Refusing update from remote peer 0788dbdabe574355bb323dcdc3be0058: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:24:26.469667 28032 consensus_queue.cc:1048] T cd1140068fc8455280934d7a30c4f559 P 0788dbdabe574355bb323dcdc3be0058 [LEADER]: Connected to new peer: Peer: permanent_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34235 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:24:26.732366 27910 raft_consensus.cc:1275] T 4554587582094286b577c9c9a4d3a7fc P 880fc4c7dff4417187b75cc1d9e9dcf5 [term 2 FOLLOWER]: Refusing update from remote peer 0788dbdabe574355bb323dcdc3be0058: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:24:26.733724 28032 consensus_queue.cc:1048] T 4554587582094286b577c9c9a4d3a7fc P 0788dbdabe574355bb323dcdc3be0058 [LEADER]: Connected to new peer: Peer: permanent_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34235 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:24:26.743116 27760 raft_consensus.cc:1275] T 4554587582094286b577c9c9a4d3a7fc P 9cb4300ea183421ab40709a05de5a877 [term 2 FOLLOWER]: Refusing update from remote peer 0788dbdabe574355bb323dcdc3be0058: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:24:26.745599 28014 consensus_queue.cc:1048] T 4554587582094286b577c9c9a4d3a7fc P 0788dbdabe574355bb323dcdc3be0058 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9cb4300ea183421ab40709a05de5a877" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 33597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:24:26.869083 27835 raft_consensus.cc:1275] T f3e47f5448ff47a382c61886f1a8cc7b P 0788dbdabe574355bb323dcdc3be0058 [term 2 FOLLOWER]: Refusing update from remote peer 9cb4300ea183421ab40709a05de5a877: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:24:26.870587 27978 consensus_queue.cc:1048] T f3e47f5448ff47a382c61886f1a8cc7b P 9cb4300ea183421ab40709a05de5a877 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0788dbdabe574355bb323dcdc3be0058" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 41335 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:24:26.884871 27910 raft_consensus.cc:1275] T f3e47f5448ff47a382c61886f1a8cc7b P 880fc4c7dff4417187b75cc1d9e9dcf5 [term 2 FOLLOWER]: Refusing update from remote peer 9cb4300ea183421ab40709a05de5a877: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:24:26.888038 27978 consensus_queue.cc:1048] T f3e47f5448ff47a382c61886f1a8cc7b P 9cb4300ea183421ab40709a05de5a877 [LEADER]: Connected to new peer: Peer: permanent_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34235 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:24:28.890372 27657 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60506:
name: "soft_deleted_table"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q""\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 13:24:28.893139 27657 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table soft_deleted_table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 13:24:28.948087 27749 tablet_service.cc:1511] Processing CreateTablet for tablet 27629b85cc3e4cb6b837dd26e3ebff47 (DEFAULT_TABLE table=soft_deleted_table [id=32a3558cca9046bb846711e7e3f090fc]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:24:28.949445 27749 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 27629b85cc3e4cb6b837dd26e3ebff47. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:24:28.949581 27826 tablet_service.cc:1511] Processing CreateTablet for tablet 27629b85cc3e4cb6b837dd26e3ebff47 (DEFAULT_TABLE table=soft_deleted_table [id=32a3558cca9046bb846711e7e3f090fc]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:24:28.951115 27826 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 27629b85cc3e4cb6b837dd26e3ebff47. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:24:28.951304 27895 tablet_service.cc:1511] Processing CreateTablet for tablet 27629b85cc3e4cb6b837dd26e3ebff47 (DEFAULT_TABLE table=soft_deleted_table [id=32a3558cca9046bb846711e7e3f090fc]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:24:28.952716 27895 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 27629b85cc3e4cb6b837dd26e3ebff47. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:24:28.953116 27752 tablet_service.cc:1511] Processing CreateTablet for tablet 8e6b55496c134353a770e2642fd7490d (DEFAULT_TABLE table=soft_deleted_table [id=32a3558cca9046bb846711e7e3f090fc]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:24:28.954648 27901 tablet_service.cc:1511] Processing CreateTablet for tablet 8e6b55496c134353a770e2642fd7490d (DEFAULT_TABLE table=soft_deleted_table [id=32a3558cca9046bb846711e7e3f090fc]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:24:28.954648 27752 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8e6b55496c134353a770e2642fd7490d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:24:28.956135 27901 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8e6b55496c134353a770e2642fd7490d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:24:28.956774 27750 tablet_service.cc:1511] Processing CreateTablet for tablet d7fd85f92c0a481aa54401371638252b (DEFAULT_TABLE table=soft_deleted_table [id=32a3558cca9046bb846711e7e3f090fc]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:24:28.958118 27750 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d7fd85f92c0a481aa54401371638252b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:24:28.960377 27894 tablet_service.cc:1511] Processing CreateTablet for tablet d7fd85f92c0a481aa54401371638252b (DEFAULT_TABLE table=soft_deleted_table [id=32a3558cca9046bb846711e7e3f090fc]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:24:28.962020 27894 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d7fd85f92c0a481aa54401371638252b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:24:28.960377 27823 tablet_service.cc:1511] Processing CreateTablet for tablet d7fd85f92c0a481aa54401371638252b (DEFAULT_TABLE table=soft_deleted_table [id=32a3558cca9046bb846711e7e3f090fc]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:24:28.962564 27751 tablet_service.cc:1511] Processing CreateTablet for tablet 04c481ece2f94a1b8cb6337900da4862 (DEFAULT_TABLE table=soft_deleted_table [id=32a3558cca9046bb846711e7e3f090fc]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:24:28.963874 27751 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 04c481ece2f94a1b8cb6337900da4862. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:24:28.965487 27898 tablet_service.cc:1511] Processing CreateTablet for tablet 04c481ece2f94a1b8cb6337900da4862 (DEFAULT_TABLE table=soft_deleted_table [id=32a3558cca9046bb846711e7e3f090fc]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:24:28.967139 27823 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d7fd85f92c0a481aa54401371638252b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:24:28.967694 27827 tablet_service.cc:1511] Processing CreateTablet for tablet 04c481ece2f94a1b8cb6337900da4862 (DEFAULT_TABLE table=soft_deleted_table [id=32a3558cca9046bb846711e7e3f090fc]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:24:28.969199 27827 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 04c481ece2f94a1b8cb6337900da4862. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:24:28.971482 27748 tablet_service.cc:1511] Processing CreateTablet for tablet d747d5acdf4f41b69c132cde690e49ff (DEFAULT_TABLE table=soft_deleted_table [id=32a3558cca9046bb846711e7e3f090fc]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:24:28.972900 27748 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d747d5acdf4f41b69c132cde690e49ff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:24:28.972535 27825 tablet_service.cc:1511] Processing CreateTablet for tablet 8e6b55496c134353a770e2642fd7490d (DEFAULT_TABLE table=soft_deleted_table [id=32a3558cca9046bb846711e7e3f090fc]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:24:28.974131 27825 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8e6b55496c134353a770e2642fd7490d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:24:28.976102 27898 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 04c481ece2f94a1b8cb6337900da4862. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:24:28.976856 27824 tablet_service.cc:1511] Processing CreateTablet for tablet d747d5acdf4f41b69c132cde690e49ff (DEFAULT_TABLE table=soft_deleted_table [id=32a3558cca9046bb846711e7e3f090fc]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:24:28.978365 27824 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d747d5acdf4f41b69c132cde690e49ff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:24:28.985021 27897 tablet_service.cc:1511] Processing CreateTablet for tablet d747d5acdf4f41b69c132cde690e49ff (DEFAULT_TABLE table=soft_deleted_table [id=32a3558cca9046bb846711e7e3f090fc]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:24:28.987023 27897 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d747d5acdf4f41b69c132cde690e49ff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:24:28.992424 27747 tablet_service.cc:1511] Processing CreateTablet for tablet 660aa4903c494ba1bce988191700e129 (DEFAULT_TABLE table=soft_deleted_table [id=32a3558cca9046bb846711e7e3f090fc]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:24:28.993981 27747 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 660aa4903c494ba1bce988191700e129. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:24:29.025071 27822 tablet_service.cc:1511] Processing CreateTablet for tablet 660aa4903c494ba1bce988191700e129 (DEFAULT_TABLE table=soft_deleted_table [id=32a3558cca9046bb846711e7e3f090fc]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:24:29.028368 27822 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 660aa4903c494ba1bce988191700e129. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:24:29.067770 27895 tablet_service.cc:1511] Processing CreateTablet for tablet 660aa4903c494ba1bce988191700e129 (DEFAULT_TABLE table=soft_deleted_table [id=32a3558cca9046bb846711e7e3f090fc]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:24:29.069198 27895 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 660aa4903c494ba1bce988191700e129. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:24:29.075345 28050 tablet_bootstrap.cc:492] T d747d5acdf4f41b69c132cde690e49ff P 880fc4c7dff4417187b75cc1d9e9dcf5: Bootstrap starting.
I20260709 13:24:29.076967 27747 tablet_service.cc:1511] Processing CreateTablet for tablet 032f598cb30343c2a1ca3bab728e5b39 (DEFAULT_TABLE table=soft_deleted_table [id=32a3558cca9046bb846711e7e3f090fc]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:24:29.078413 27747 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 032f598cb30343c2a1ca3bab728e5b39. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:24:29.085460 28051 tablet_bootstrap.cc:492] T 27629b85cc3e4cb6b837dd26e3ebff47 P 0788dbdabe574355bb323dcdc3be0058: Bootstrap starting.
I20260709 13:24:29.097193 27825 tablet_service.cc:1511] Processing CreateTablet for tablet 032f598cb30343c2a1ca3bab728e5b39 (DEFAULT_TABLE table=soft_deleted_table [id=32a3558cca9046bb846711e7e3f090fc]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:24:29.100415 27825 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 032f598cb30343c2a1ca3bab728e5b39. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:24:29.115697 27751 tablet_service.cc:1511] Processing CreateTablet for tablet 96f830fa400c473881739b3a953f0178 (DEFAULT_TABLE table=soft_deleted_table [id=32a3558cca9046bb846711e7e3f090fc]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:24:29.117342 27751 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 96f830fa400c473881739b3a953f0178. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:24:29.106887 27901 tablet_service.cc:1511] Processing CreateTablet for tablet 032f598cb30343c2a1ca3bab728e5b39 (DEFAULT_TABLE table=soft_deleted_table [id=32a3558cca9046bb846711e7e3f090fc]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:24:29.120271 27901 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 032f598cb30343c2a1ca3bab728e5b39. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:24:29.117321 28051 tablet_bootstrap.cc:654] T 27629b85cc3e4cb6b837dd26e3ebff47 P 0788dbdabe574355bb323dcdc3be0058: Neither blocks nor log segments found. Creating new log.
I20260709 13:24:29.129086 27822 tablet_service.cc:1511] Processing CreateTablet for tablet 96f830fa400c473881739b3a953f0178 (DEFAULT_TABLE table=soft_deleted_table [id=32a3558cca9046bb846711e7e3f090fc]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:24:29.129573 27898 tablet_service.cc:1511] Processing CreateTablet for tablet 96f830fa400c473881739b3a953f0178 (DEFAULT_TABLE table=soft_deleted_table [id=32a3558cca9046bb846711e7e3f090fc]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:24:29.130726 27822 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 96f830fa400c473881739b3a953f0178. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:24:29.131428 27898 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 96f830fa400c473881739b3a953f0178. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:24:29.131040 27824 tablet_service.cc:1511] Processing CreateTablet for tablet 1af49a625b20491586dcea4961e46734 (DEFAULT_TABLE table=soft_deleted_table [id=32a3558cca9046bb846711e7e3f090fc]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:24:29.132618 27824 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1af49a625b20491586dcea4961e46734. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:24:29.133723 27897 tablet_service.cc:1511] Processing CreateTablet for tablet 1af49a625b20491586dcea4961e46734 (DEFAULT_TABLE table=soft_deleted_table [id=32a3558cca9046bb846711e7e3f090fc]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:24:29.135337 27897 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1af49a625b20491586dcea4961e46734. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:24:29.137401 27748 tablet_service.cc:1511] Processing CreateTablet for tablet 1af49a625b20491586dcea4961e46734 (DEFAULT_TABLE table=soft_deleted_table [id=32a3558cca9046bb846711e7e3f090fc]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:24:29.138825 27748 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1af49a625b20491586dcea4961e46734. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:24:29.145823 28050 tablet_bootstrap.cc:654] T d747d5acdf4f41b69c132cde690e49ff P 880fc4c7dff4417187b75cc1d9e9dcf5: Neither blocks nor log segments found. Creating new log.
I20260709 13:24:29.133669 28052 tablet_bootstrap.cc:492] T 8e6b55496c134353a770e2642fd7490d P 9cb4300ea183421ab40709a05de5a877: Bootstrap starting.
W20260709 13:24:29.169159 27708 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:24:29.183350 28051 tablet_bootstrap.cc:492] T 27629b85cc3e4cb6b837dd26e3ebff47 P 0788dbdabe574355bb323dcdc3be0058: No bootstrap required, opened a new log
I20260709 13:24:29.184219 28051 ts_tablet_manager.cc:1403] T 27629b85cc3e4cb6b837dd26e3ebff47 P 0788dbdabe574355bb323dcdc3be0058: Time spent bootstrapping tablet: real 0.099s	user 0.017s	sys 0.015s
I20260709 13:24:29.184823 28052 tablet_bootstrap.cc:654] T 8e6b55496c134353a770e2642fd7490d P 9cb4300ea183421ab40709a05de5a877: Neither blocks nor log segments found. Creating new log.
I20260709 13:24:29.188380 28051 raft_consensus.cc:359] T 27629b85cc3e4cb6b837dd26e3ebff47 P 0788dbdabe574355bb323dcdc3be0058 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb4300ea183421ab40709a05de5a877" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 33597 } } peers { permanent_uuid: "0788dbdabe574355bb323dcdc3be0058" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 41335 } } peers { permanent_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34235 } }
I20260709 13:24:29.191823 28051 raft_consensus.cc:385] T 27629b85cc3e4cb6b837dd26e3ebff47 P 0788dbdabe574355bb323dcdc3be0058 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:24:29.192175 28051 raft_consensus.cc:740] T 27629b85cc3e4cb6b837dd26e3ebff47 P 0788dbdabe574355bb323dcdc3be0058 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0788dbdabe574355bb323dcdc3be0058, State: Initialized, Role: FOLLOWER
I20260709 13:24:29.192951 28051 consensus_queue.cc:260] T 27629b85cc3e4cb6b837dd26e3ebff47 P 0788dbdabe574355bb323dcdc3be0058 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb4300ea183421ab40709a05de5a877" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 33597 } } peers { permanent_uuid: "0788dbdabe574355bb323dcdc3be0058" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 41335 } } peers { permanent_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34235 } }
I20260709 13:24:29.195334 28051 ts_tablet_manager.cc:1434] T 27629b85cc3e4cb6b837dd26e3ebff47 P 0788dbdabe574355bb323dcdc3be0058: Time spent starting tablet: real 0.011s	user 0.008s	sys 0.000s
I20260709 13:24:29.201435 28051 tablet_bootstrap.cc:492] T 8e6b55496c134353a770e2642fd7490d P 0788dbdabe574355bb323dcdc3be0058: Bootstrap starting.
I20260709 13:24:29.205943 28050 tablet_bootstrap.cc:492] T d747d5acdf4f41b69c132cde690e49ff P 880fc4c7dff4417187b75cc1d9e9dcf5: No bootstrap required, opened a new log
I20260709 13:24:29.207829 28050 ts_tablet_manager.cc:1403] T d747d5acdf4f41b69c132cde690e49ff P 880fc4c7dff4417187b75cc1d9e9dcf5: Time spent bootstrapping tablet: real 0.133s	user 0.015s	sys 0.008s
I20260709 13:24:29.213121 28052 tablet_bootstrap.cc:492] T 8e6b55496c134353a770e2642fd7490d P 9cb4300ea183421ab40709a05de5a877: No bootstrap required, opened a new log
I20260709 13:24:29.213807 28052 ts_tablet_manager.cc:1403] T 8e6b55496c134353a770e2642fd7490d P 9cb4300ea183421ab40709a05de5a877: Time spent bootstrapping tablet: real 0.080s	user 0.012s	sys 0.007s
I20260709 13:24:29.215790 28051 tablet_bootstrap.cc:654] T 8e6b55496c134353a770e2642fd7490d P 0788dbdabe574355bb323dcdc3be0058: Neither blocks nor log segments found. Creating new log.
I20260709 13:24:29.217285 28052 raft_consensus.cc:359] T 8e6b55496c134353a770e2642fd7490d P 9cb4300ea183421ab40709a05de5a877 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb4300ea183421ab40709a05de5a877" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 33597 } } peers { permanent_uuid: "0788dbdabe574355bb323dcdc3be0058" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 41335 } } peers { permanent_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34235 } }
I20260709 13:24:29.218215 28052 raft_consensus.cc:385] T 8e6b55496c134353a770e2642fd7490d P 9cb4300ea183421ab40709a05de5a877 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:24:29.218521 28052 raft_consensus.cc:740] T 8e6b55496c134353a770e2642fd7490d P 9cb4300ea183421ab40709a05de5a877 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9cb4300ea183421ab40709a05de5a877, State: Initialized, Role: FOLLOWER
I20260709 13:24:29.219414 28052 consensus_queue.cc:260] T 8e6b55496c134353a770e2642fd7490d P 9cb4300ea183421ab40709a05de5a877 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb4300ea183421ab40709a05de5a877" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 33597 } } peers { permanent_uuid: "0788dbdabe574355bb323dcdc3be0058" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 41335 } } peers { permanent_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34235 } }
I20260709 13:24:29.220842 28050 raft_consensus.cc:359] T d747d5acdf4f41b69c132cde690e49ff P 880fc4c7dff4417187b75cc1d9e9dcf5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb4300ea183421ab40709a05de5a877" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 33597 } } peers { permanent_uuid: "0788dbdabe574355bb323dcdc3be0058" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 41335 } } peers { permanent_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34235 } }
I20260709 13:24:29.221728 28050 raft_consensus.cc:385] T d747d5acdf4f41b69c132cde690e49ff P 880fc4c7dff4417187b75cc1d9e9dcf5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:24:29.222043 28050 raft_consensus.cc:740] T d747d5acdf4f41b69c132cde690e49ff P 880fc4c7dff4417187b75cc1d9e9dcf5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 880fc4c7dff4417187b75cc1d9e9dcf5, State: Initialized, Role: FOLLOWER
I20260709 13:24:29.222834 28051 tablet_bootstrap.cc:492] T 8e6b55496c134353a770e2642fd7490d P 0788dbdabe574355bb323dcdc3be0058: No bootstrap required, opened a new log
I20260709 13:24:29.223618 28051 ts_tablet_manager.cc:1403] T 8e6b55496c134353a770e2642fd7490d P 0788dbdabe574355bb323dcdc3be0058: Time spent bootstrapping tablet: real 0.023s	user 0.013s	sys 0.002s
I20260709 13:24:29.222872 28050 consensus_queue.cc:260] T d747d5acdf4f41b69c132cde690e49ff P 880fc4c7dff4417187b75cc1d9e9dcf5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb4300ea183421ab40709a05de5a877" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 33597 } } peers { permanent_uuid: "0788dbdabe574355bb323dcdc3be0058" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 41335 } } peers { permanent_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34235 } }
I20260709 13:24:29.224771 28052 ts_tablet_manager.cc:1434] T 8e6b55496c134353a770e2642fd7490d P 9cb4300ea183421ab40709a05de5a877: Time spent starting tablet: real 0.011s	user 0.004s	sys 0.006s
I20260709 13:24:29.225880 28050 ts_tablet_manager.cc:1434] T d747d5acdf4f41b69c132cde690e49ff P 880fc4c7dff4417187b75cc1d9e9dcf5: Time spent starting tablet: real 0.018s	user 0.006s	sys 0.001s
I20260709 13:24:29.227617 28050 tablet_bootstrap.cc:492] T 27629b85cc3e4cb6b837dd26e3ebff47 P 880fc4c7dff4417187b75cc1d9e9dcf5: Bootstrap starting.
I20260709 13:24:29.227589 28051 raft_consensus.cc:359] T 8e6b55496c134353a770e2642fd7490d P 0788dbdabe574355bb323dcdc3be0058 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb4300ea183421ab40709a05de5a877" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 33597 } } peers { permanent_uuid: "0788dbdabe574355bb323dcdc3be0058" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 41335 } } peers { permanent_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34235 } }
I20260709 13:24:29.228545 28051 raft_consensus.cc:385] T 8e6b55496c134353a770e2642fd7490d P 0788dbdabe574355bb323dcdc3be0058 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:24:29.228888 28051 raft_consensus.cc:740] T 8e6b55496c134353a770e2642fd7490d P 0788dbdabe574355bb323dcdc3be0058 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0788dbdabe574355bb323dcdc3be0058, State: Initialized, Role: FOLLOWER
I20260709 13:24:29.229804 28051 consensus_queue.cc:260] T 8e6b55496c134353a770e2642fd7490d P 0788dbdabe574355bb323dcdc3be0058 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb4300ea183421ab40709a05de5a877" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 33597 } } peers { permanent_uuid: "0788dbdabe574355bb323dcdc3be0058" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 41335 } } peers { permanent_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34235 } }
I20260709 13:24:29.232494 28051 ts_tablet_manager.cc:1434] T 8e6b55496c134353a770e2642fd7490d P 0788dbdabe574355bb323dcdc3be0058: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 13:24:29.236515 28050 tablet_bootstrap.cc:654] T 27629b85cc3e4cb6b837dd26e3ebff47 P 880fc4c7dff4417187b75cc1d9e9dcf5: Neither blocks nor log segments found. Creating new log.
I20260709 13:24:29.239840 28052 tablet_bootstrap.cc:492] T 660aa4903c494ba1bce988191700e129 P 9cb4300ea183421ab40709a05de5a877: Bootstrap starting.
I20260709 13:24:29.245090 28051 tablet_bootstrap.cc:492] T 04c481ece2f94a1b8cb6337900da4862 P 0788dbdabe574355bb323dcdc3be0058: Bootstrap starting.
I20260709 13:24:29.248992 28052 tablet_bootstrap.cc:654] T 660aa4903c494ba1bce988191700e129 P 9cb4300ea183421ab40709a05de5a877: Neither blocks nor log segments found. Creating new log.
I20260709 13:24:29.261936 28050 tablet_bootstrap.cc:492] T 27629b85cc3e4cb6b837dd26e3ebff47 P 880fc4c7dff4417187b75cc1d9e9dcf5: No bootstrap required, opened a new log
I20260709 13:24:29.262645 28050 ts_tablet_manager.cc:1403] T 27629b85cc3e4cb6b837dd26e3ebff47 P 880fc4c7dff4417187b75cc1d9e9dcf5: Time spent bootstrapping tablet: real 0.036s	user 0.011s	sys 0.021s
I20260709 13:24:29.263871 28051 tablet_bootstrap.cc:654] T 04c481ece2f94a1b8cb6337900da4862 P 0788dbdabe574355bb323dcdc3be0058: Neither blocks nor log segments found. Creating new log.
I20260709 13:24:29.265657 28050 raft_consensus.cc:359] T 27629b85cc3e4cb6b837dd26e3ebff47 P 880fc4c7dff4417187b75cc1d9e9dcf5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb4300ea183421ab40709a05de5a877" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 33597 } } peers { permanent_uuid: "0788dbdabe574355bb323dcdc3be0058" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 41335 } } peers { permanent_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34235 } }
I20260709 13:24:29.266364 28050 raft_consensus.cc:385] T 27629b85cc3e4cb6b837dd26e3ebff47 P 880fc4c7dff4417187b75cc1d9e9dcf5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:24:29.266407 28052 tablet_bootstrap.cc:492] T 660aa4903c494ba1bce988191700e129 P 9cb4300ea183421ab40709a05de5a877: No bootstrap required, opened a new log
I20260709 13:24:29.266690 28050 raft_consensus.cc:740] T 27629b85cc3e4cb6b837dd26e3ebff47 P 880fc4c7dff4417187b75cc1d9e9dcf5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 880fc4c7dff4417187b75cc1d9e9dcf5, State: Initialized, Role: FOLLOWER
I20260709 13:24:29.267150 28052 ts_tablet_manager.cc:1403] T 660aa4903c494ba1bce988191700e129 P 9cb4300ea183421ab40709a05de5a877: Time spent bootstrapping tablet: real 0.028s	user 0.009s	sys 0.011s
I20260709 13:24:29.267637 28050 consensus_queue.cc:260] T 27629b85cc3e4cb6b837dd26e3ebff47 P 880fc4c7dff4417187b75cc1d9e9dcf5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb4300ea183421ab40709a05de5a877" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 33597 } } peers { permanent_uuid: "0788dbdabe574355bb323dcdc3be0058" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 41335 } } peers { permanent_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34235 } }
I20260709 13:24:29.270268 28050 ts_tablet_manager.cc:1434] T 27629b85cc3e4cb6b837dd26e3ebff47 P 880fc4c7dff4417187b75cc1d9e9dcf5: Time spent starting tablet: real 0.007s	user 0.000s	sys 0.004s
I20260709 13:24:29.271528 28050 tablet_bootstrap.cc:492] T 8e6b55496c134353a770e2642fd7490d P 880fc4c7dff4417187b75cc1d9e9dcf5: Bootstrap starting.
I20260709 13:24:29.272128 28052 raft_consensus.cc:359] T 660aa4903c494ba1bce988191700e129 P 9cb4300ea183421ab40709a05de5a877 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb4300ea183421ab40709a05de5a877" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 33597 } } peers { permanent_uuid: "0788dbdabe574355bb323dcdc3be0058" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 41335 } } peers { permanent_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34235 } }
I20260709 13:24:29.275890 28052 raft_consensus.cc:385] T 660aa4903c494ba1bce988191700e129 P 9cb4300ea183421ab40709a05de5a877 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:24:29.276309 28052 raft_consensus.cc:740] T 660aa4903c494ba1bce988191700e129 P 9cb4300ea183421ab40709a05de5a877 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9cb4300ea183421ab40709a05de5a877, State: Initialized, Role: FOLLOWER
I20260709 13:24:29.276944 28051 tablet_bootstrap.cc:492] T 04c481ece2f94a1b8cb6337900da4862 P 0788dbdabe574355bb323dcdc3be0058: No bootstrap required, opened a new log
I20260709 13:24:29.277915 28051 ts_tablet_manager.cc:1403] T 04c481ece2f94a1b8cb6337900da4862 P 0788dbdabe574355bb323dcdc3be0058: Time spent bootstrapping tablet: real 0.045s	user 0.016s	sys 0.006s
I20260709 13:24:29.277228 28052 consensus_queue.cc:260] T 660aa4903c494ba1bce988191700e129 P 9cb4300ea183421ab40709a05de5a877 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb4300ea183421ab40709a05de5a877" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 33597 } } peers { permanent_uuid: "0788dbdabe574355bb323dcdc3be0058" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 41335 } } peers { permanent_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34235 } }
I20260709 13:24:29.279810 28050 tablet_bootstrap.cc:654] T 8e6b55496c134353a770e2642fd7490d P 880fc4c7dff4417187b75cc1d9e9dcf5: Neither blocks nor log segments found. Creating new log.
I20260709 13:24:29.280547 28052 ts_tablet_manager.cc:1434] T 660aa4903c494ba1bce988191700e129 P 9cb4300ea183421ab40709a05de5a877: Time spent starting tablet: real 0.013s	user 0.007s	sys 0.001s
I20260709 13:24:29.281764 28052 tablet_bootstrap.cc:492] T d747d5acdf4f41b69c132cde690e49ff P 9cb4300ea183421ab40709a05de5a877: Bootstrap starting.
I20260709 13:24:29.286383 28051 raft_consensus.cc:359] T 04c481ece2f94a1b8cb6337900da4862 P 0788dbdabe574355bb323dcdc3be0058 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb4300ea183421ab40709a05de5a877" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 33597 } } peers { permanent_uuid: "0788dbdabe574355bb323dcdc3be0058" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 41335 } } peers { permanent_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34235 } }
I20260709 13:24:29.287652 28051 raft_consensus.cc:385] T 04c481ece2f94a1b8cb6337900da4862 P 0788dbdabe574355bb323dcdc3be0058 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:24:29.288125 28051 raft_consensus.cc:740] T 04c481ece2f94a1b8cb6337900da4862 P 0788dbdabe574355bb323dcdc3be0058 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0788dbdabe574355bb323dcdc3be0058, State: Initialized, Role: FOLLOWER
I20260709 13:24:29.289608 28052 tablet_bootstrap.cc:654] T d747d5acdf4f41b69c132cde690e49ff P 9cb4300ea183421ab40709a05de5a877: Neither blocks nor log segments found. Creating new log.
I20260709 13:24:29.289443 28051 consensus_queue.cc:260] T 04c481ece2f94a1b8cb6337900da4862 P 0788dbdabe574355bb323dcdc3be0058 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb4300ea183421ab40709a05de5a877" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 33597 } } peers { permanent_uuid: "0788dbdabe574355bb323dcdc3be0058" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 41335 } } peers { permanent_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34235 } }
I20260709 13:24:29.297006 28051 ts_tablet_manager.cc:1434] T 04c481ece2f94a1b8cb6337900da4862 P 0788dbdabe574355bb323dcdc3be0058: Time spent starting tablet: real 0.019s	user 0.009s	sys 0.001s
I20260709 13:24:29.298699 28051 tablet_bootstrap.cc:492] T d747d5acdf4f41b69c132cde690e49ff P 0788dbdabe574355bb323dcdc3be0058: Bootstrap starting.
I20260709 13:24:29.301484 28050 tablet_bootstrap.cc:492] T 8e6b55496c134353a770e2642fd7490d P 880fc4c7dff4417187b75cc1d9e9dcf5: No bootstrap required, opened a new log
I20260709 13:24:29.302229 28050 ts_tablet_manager.cc:1403] T 8e6b55496c134353a770e2642fd7490d P 880fc4c7dff4417187b75cc1d9e9dcf5: Time spent bootstrapping tablet: real 0.031s	user 0.010s	sys 0.019s
I20260709 13:24:29.306066 28051 tablet_bootstrap.cc:654] T d747d5acdf4f41b69c132cde690e49ff P 0788dbdabe574355bb323dcdc3be0058: Neither blocks nor log segments found. Creating new log.
I20260709 13:24:29.305776 28050 raft_consensus.cc:359] T 8e6b55496c134353a770e2642fd7490d P 880fc4c7dff4417187b75cc1d9e9dcf5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb4300ea183421ab40709a05de5a877" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 33597 } } peers { permanent_uuid: "0788dbdabe574355bb323dcdc3be0058" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 41335 } } peers { permanent_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34235 } }
I20260709 13:24:29.306692 28050 raft_consensus.cc:385] T 8e6b55496c134353a770e2642fd7490d P 880fc4c7dff4417187b75cc1d9e9dcf5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:24:29.307123 28050 raft_consensus.cc:740] T 8e6b55496c134353a770e2642fd7490d P 880fc4c7dff4417187b75cc1d9e9dcf5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 880fc4c7dff4417187b75cc1d9e9dcf5, State: Initialized, Role: FOLLOWER
I20260709 13:24:29.308179 28050 consensus_queue.cc:260] T 8e6b55496c134353a770e2642fd7490d P 880fc4c7dff4417187b75cc1d9e9dcf5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb4300ea183421ab40709a05de5a877" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 33597 } } peers { permanent_uuid: "0788dbdabe574355bb323dcdc3be0058" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 41335 } } peers { permanent_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34235 } }
I20260709 13:24:29.309808 28052 tablet_bootstrap.cc:492] T d747d5acdf4f41b69c132cde690e49ff P 9cb4300ea183421ab40709a05de5a877: No bootstrap required, opened a new log
I20260709 13:24:29.310583 28052 ts_tablet_manager.cc:1403] T d747d5acdf4f41b69c132cde690e49ff P 9cb4300ea183421ab40709a05de5a877: Time spent bootstrapping tablet: real 0.029s	user 0.015s	sys 0.003s
I20260709 13:24:29.311408 28050 ts_tablet_manager.cc:1434] T 8e6b55496c134353a770e2642fd7490d P 880fc4c7dff4417187b75cc1d9e9dcf5: Time spent starting tablet: real 0.009s	user 0.005s	sys 0.003s
I20260709 13:24:29.313115 28050 tablet_bootstrap.cc:492] T 04c481ece2f94a1b8cb6337900da4862 P 880fc4c7dff4417187b75cc1d9e9dcf5: Bootstrap starting.
I20260709 13:24:29.316136 28052 raft_consensus.cc:359] T d747d5acdf4f41b69c132cde690e49ff P 9cb4300ea183421ab40709a05de5a877 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb4300ea183421ab40709a05de5a877" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 33597 } } peers { permanent_uuid: "0788dbdabe574355bb323dcdc3be0058" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 41335 } } peers { permanent_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34235 } }
I20260709 13:24:29.317673 28052 raft_consensus.cc:385] T d747d5acdf4f41b69c132cde690e49ff P 9cb4300ea183421ab40709a05de5a877 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:24:29.318063 28052 raft_consensus.cc:740] T d747d5acdf4f41b69c132cde690e49ff P 9cb4300ea183421ab40709a05de5a877 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9cb4300ea183421ab40709a05de5a877, State: Initialized, Role: FOLLOWER
I20260709 13:24:29.319031 28052 consensus_queue.cc:260] T d747d5acdf4f41b69c132cde690e49ff P 9cb4300ea183421ab40709a05de5a877 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb4300ea183421ab40709a05de5a877" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 33597 } } peers { permanent_uuid: "0788dbdabe574355bb323dcdc3be0058" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 41335 } } peers { permanent_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34235 } }
I20260709 13:24:29.321635 28052 ts_tablet_manager.cc:1434] T d747d5acdf4f41b69c132cde690e49ff P 9cb4300ea183421ab40709a05de5a877: Time spent starting tablet: real 0.011s	user 0.006s	sys 0.000s
I20260709 13:24:29.321954 28050 tablet_bootstrap.cc:654] T 04c481ece2f94a1b8cb6337900da4862 P 880fc4c7dff4417187b75cc1d9e9dcf5: Neither blocks nor log segments found. Creating new log.
I20260709 13:24:29.322767 28052 tablet_bootstrap.cc:492] T d7fd85f92c0a481aa54401371638252b P 9cb4300ea183421ab40709a05de5a877: Bootstrap starting.
I20260709 13:24:29.330564 28052 tablet_bootstrap.cc:654] T d7fd85f92c0a481aa54401371638252b P 9cb4300ea183421ab40709a05de5a877: Neither blocks nor log segments found. Creating new log.
I20260709 13:24:29.331800 28051 tablet_bootstrap.cc:492] T d747d5acdf4f41b69c132cde690e49ff P 0788dbdabe574355bb323dcdc3be0058: No bootstrap required, opened a new log
I20260709 13:24:29.332589 28051 ts_tablet_manager.cc:1403] T d747d5acdf4f41b69c132cde690e49ff P 0788dbdabe574355bb323dcdc3be0058: Time spent bootstrapping tablet: real 0.034s	user 0.006s	sys 0.012s
I20260709 13:24:29.335474 28050 tablet_bootstrap.cc:492] T 04c481ece2f94a1b8cb6337900da4862 P 880fc4c7dff4417187b75cc1d9e9dcf5: No bootstrap required, opened a new log
I20260709 13:24:29.336228 28050 ts_tablet_manager.cc:1403] T 04c481ece2f94a1b8cb6337900da4862 P 880fc4c7dff4417187b75cc1d9e9dcf5: Time spent bootstrapping tablet: real 0.023s	user 0.016s	sys 0.005s
I20260709 13:24:29.337162 28051 raft_consensus.cc:359] T d747d5acdf4f41b69c132cde690e49ff P 0788dbdabe574355bb323dcdc3be0058 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb4300ea183421ab40709a05de5a877" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 33597 } } peers { permanent_uuid: "0788dbdabe574355bb323dcdc3be0058" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 41335 } } peers { permanent_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34235 } }
I20260709 13:24:29.338495 28051 raft_consensus.cc:385] T d747d5acdf4f41b69c132cde690e49ff P 0788dbdabe574355bb323dcdc3be0058 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:24:29.338989 28051 raft_consensus.cc:740] T d747d5acdf4f41b69c132cde690e49ff P 0788dbdabe574355bb323dcdc3be0058 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0788dbdabe574355bb323dcdc3be0058, State: Initialized, Role: FOLLOWER
I20260709 13:24:29.340087 28050 raft_consensus.cc:359] T 04c481ece2f94a1b8cb6337900da4862 P 880fc4c7dff4417187b75cc1d9e9dcf5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb4300ea183421ab40709a05de5a877" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 33597 } } peers { permanent_uuid: "0788dbdabe574355bb323dcdc3be0058" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 41335 } } peers { permanent_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34235 } }
I20260709 13:24:29.341116 28050 raft_consensus.cc:385] T 04c481ece2f94a1b8cb6337900da4862 P 880fc4c7dff4417187b75cc1d9e9dcf5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:24:29.340178 28051 consensus_queue.cc:260] T d747d5acdf4f41b69c132cde690e49ff P 0788dbdabe574355bb323dcdc3be0058 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb4300ea183421ab40709a05de5a877" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 33597 } } peers { permanent_uuid: "0788dbdabe574355bb323dcdc3be0058" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 41335 } } peers { permanent_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34235 } }
I20260709 13:24:29.341533 28050 raft_consensus.cc:740] T 04c481ece2f94a1b8cb6337900da4862 P 880fc4c7dff4417187b75cc1d9e9dcf5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 880fc4c7dff4417187b75cc1d9e9dcf5, State: Initialized, Role: FOLLOWER
I20260709 13:24:29.342530 28050 consensus_queue.cc:260] T 04c481ece2f94a1b8cb6337900da4862 P 880fc4c7dff4417187b75cc1d9e9dcf5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb4300ea183421ab40709a05de5a877" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 33597 } } peers { permanent_uuid: "0788dbdabe574355bb323dcdc3be0058" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 41335 } } peers { permanent_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34235 } }
I20260709 13:24:29.352890 28050 ts_tablet_manager.cc:1434] T 04c481ece2f94a1b8cb6337900da4862 P 880fc4c7dff4417187b75cc1d9e9dcf5: Time spent starting tablet: real 0.016s	user 0.012s	sys 0.001s
I20260709 13:24:29.354316 28050 tablet_bootstrap.cc:492] T d7fd85f92c0a481aa54401371638252b P 880fc4c7dff4417187b75cc1d9e9dcf5: Bootstrap starting.
I20260709 13:24:29.355706 28051 ts_tablet_manager.cc:1434] T d747d5acdf4f41b69c132cde690e49ff P 0788dbdabe574355bb323dcdc3be0058: Time spent starting tablet: real 0.023s	user 0.011s	sys 0.000s
I20260709 13:24:29.357678 28051 tablet_bootstrap.cc:492] T 660aa4903c494ba1bce988191700e129 P 0788dbdabe574355bb323dcdc3be0058: Bootstrap starting.
I20260709 13:24:29.363024 28052 tablet_bootstrap.cc:492] T d7fd85f92c0a481aa54401371638252b P 9cb4300ea183421ab40709a05de5a877: No bootstrap required, opened a new log
I20260709 13:24:29.363891 28052 ts_tablet_manager.cc:1403] T d7fd85f92c0a481aa54401371638252b P 9cb4300ea183421ab40709a05de5a877: Time spent bootstrapping tablet: real 0.041s	user 0.025s	sys 0.015s
I20260709 13:24:29.367228 28051 tablet_bootstrap.cc:654] T 660aa4903c494ba1bce988191700e129 P 0788dbdabe574355bb323dcdc3be0058: Neither blocks nor log segments found. Creating new log.
I20260709 13:24:29.368085 28050 tablet_bootstrap.cc:654] T d7fd85f92c0a481aa54401371638252b P 880fc4c7dff4417187b75cc1d9e9dcf5: Neither blocks nor log segments found. Creating new log.
I20260709 13:24:29.367708 28052 raft_consensus.cc:359] T d7fd85f92c0a481aa54401371638252b P 9cb4300ea183421ab40709a05de5a877 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb4300ea183421ab40709a05de5a877" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 33597 } } peers { permanent_uuid: "0788dbdabe574355bb323dcdc3be0058" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 41335 } } peers { permanent_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34235 } }
I20260709 13:24:29.368752 28052 raft_consensus.cc:385] T d7fd85f92c0a481aa54401371638252b P 9cb4300ea183421ab40709a05de5a877 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:24:29.369204 28052 raft_consensus.cc:740] T d7fd85f92c0a481aa54401371638252b P 9cb4300ea183421ab40709a05de5a877 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9cb4300ea183421ab40709a05de5a877, State: Initialized, Role: FOLLOWER
I20260709 13:24:29.370152 28052 consensus_queue.cc:260] T d7fd85f92c0a481aa54401371638252b P 9cb4300ea183421ab40709a05de5a877 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb4300ea183421ab40709a05de5a877" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 33597 } } peers { permanent_uuid: "0788dbdabe574355bb323dcdc3be0058" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 41335 } } peers { permanent_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34235 } }
I20260709 13:24:29.373873 28052 ts_tablet_manager.cc:1434] T d7fd85f92c0a481aa54401371638252b P 9cb4300ea183421ab40709a05de5a877: Time spent starting tablet: real 0.010s	user 0.004s	sys 0.003s
I20260709 13:24:29.375463 28052 tablet_bootstrap.cc:492] T 04c481ece2f94a1b8cb6337900da4862 P 9cb4300ea183421ab40709a05de5a877: Bootstrap starting.
I20260709 13:24:29.380051 28051 tablet_bootstrap.cc:492] T 660aa4903c494ba1bce988191700e129 P 0788dbdabe574355bb323dcdc3be0058: No bootstrap required, opened a new log
I20260709 13:24:29.380767 28051 ts_tablet_manager.cc:1403] T 660aa4903c494ba1bce988191700e129 P 0788dbdabe574355bb323dcdc3be0058: Time spent bootstrapping tablet: real 0.024s	user 0.012s	sys 0.009s
I20260709 13:24:29.384747 28051 raft_consensus.cc:359] T 660aa4903c494ba1bce988191700e129 P 0788dbdabe574355bb323dcdc3be0058 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb4300ea183421ab40709a05de5a877" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 33597 } } peers { permanent_uuid: "0788dbdabe574355bb323dcdc3be0058" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 41335 } } peers { permanent_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34235 } }
I20260709 13:24:29.385699 28051 raft_consensus.cc:385] T 660aa4903c494ba1bce988191700e129 P 0788dbdabe574355bb323dcdc3be0058 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:24:29.386085 28051 raft_consensus.cc:740] T 660aa4903c494ba1bce988191700e129 P 0788dbdabe574355bb323dcdc3be0058 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0788dbdabe574355bb323dcdc3be0058, State: Initialized, Role: FOLLOWER
I20260709 13:24:29.387127 28052 tablet_bootstrap.cc:654] T 04c481ece2f94a1b8cb6337900da4862 P 9cb4300ea183421ab40709a05de5a877: Neither blocks nor log segments found. Creating new log.
I20260709 13:24:29.386953 28051 consensus_queue.cc:260] T 660aa4903c494ba1bce988191700e129 P 0788dbdabe574355bb323dcdc3be0058 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb4300ea183421ab40709a05de5a877" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 33597 } } peers { permanent_uuid: "0788dbdabe574355bb323dcdc3be0058" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 41335 } } peers { permanent_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34235 } }
I20260709 13:24:29.389613 28051 ts_tablet_manager.cc:1434] T 660aa4903c494ba1bce988191700e129 P 0788dbdabe574355bb323dcdc3be0058: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 13:24:29.391204 28051 tablet_bootstrap.cc:492] T d7fd85f92c0a481aa54401371638252b P 0788dbdabe574355bb323dcdc3be0058: Bootstrap starting.
I20260709 13:24:29.391515 28050 tablet_bootstrap.cc:492] T d7fd85f92c0a481aa54401371638252b P 880fc4c7dff4417187b75cc1d9e9dcf5: No bootstrap required, opened a new log
I20260709 13:24:29.392109 28050 ts_tablet_manager.cc:1403] T d7fd85f92c0a481aa54401371638252b P 880fc4c7dff4417187b75cc1d9e9dcf5: Time spent bootstrapping tablet: real 0.038s	user 0.025s	sys 0.002s
I20260709 13:24:29.395619 28050 raft_consensus.cc:359] T d7fd85f92c0a481aa54401371638252b P 880fc4c7dff4417187b75cc1d9e9dcf5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb4300ea183421ab40709a05de5a877" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 33597 } } peers { permanent_uuid: "0788dbdabe574355bb323dcdc3be0058" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 41335 } } peers { permanent_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34235 } }
I20260709 13:24:29.396492 28050 raft_consensus.cc:385] T d7fd85f92c0a481aa54401371638252b P 880fc4c7dff4417187b75cc1d9e9dcf5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:24:29.396859 28050 raft_consensus.cc:740] T d7fd85f92c0a481aa54401371638252b P 880fc4c7dff4417187b75cc1d9e9dcf5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 880fc4c7dff4417187b75cc1d9e9dcf5, State: Initialized, Role: FOLLOWER
I20260709 13:24:29.397887 28050 consensus_queue.cc:260] T d7fd85f92c0a481aa54401371638252b P 880fc4c7dff4417187b75cc1d9e9dcf5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb4300ea183421ab40709a05de5a877" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 33597 } } peers { permanent_uuid: "0788dbdabe574355bb323dcdc3be0058" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 41335 } } peers { permanent_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34235 } }
I20260709 13:24:29.401072 28050 ts_tablet_manager.cc:1434] T d7fd85f92c0a481aa54401371638252b P 880fc4c7dff4417187b75cc1d9e9dcf5: Time spent starting tablet: real 0.009s	user 0.003s	sys 0.004s
I20260709 13:24:29.402313 28050 tablet_bootstrap.cc:492] T 660aa4903c494ba1bce988191700e129 P 880fc4c7dff4417187b75cc1d9e9dcf5: Bootstrap starting.
I20260709 13:24:29.402966 28051 tablet_bootstrap.cc:654] T d7fd85f92c0a481aa54401371638252b P 0788dbdabe574355bb323dcdc3be0058: Neither blocks nor log segments found. Creating new log.
I20260709 13:24:29.406350 28052 tablet_bootstrap.cc:492] T 04c481ece2f94a1b8cb6337900da4862 P 9cb4300ea183421ab40709a05de5a877: No bootstrap required, opened a new log
I20260709 13:24:29.407076 28052 ts_tablet_manager.cc:1403] T 04c481ece2f94a1b8cb6337900da4862 P 9cb4300ea183421ab40709a05de5a877: Time spent bootstrapping tablet: real 0.032s	user 0.014s	sys 0.011s
I20260709 13:24:29.403620 27955 raft_consensus.cc:493] T 27629b85cc3e4cb6b837dd26e3ebff47 P 0788dbdabe574355bb323dcdc3be0058 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:24:29.408090 27955 raft_consensus.cc:515] T 27629b85cc3e4cb6b837dd26e3ebff47 P 0788dbdabe574355bb323dcdc3be0058 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb4300ea183421ab40709a05de5a877" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 33597 } } peers { permanent_uuid: "0788dbdabe574355bb323dcdc3be0058" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 41335 } } peers { permanent_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34235 } }
I20260709 13:24:29.409323 28050 tablet_bootstrap.cc:654] T 660aa4903c494ba1bce988191700e129 P 880fc4c7dff4417187b75cc1d9e9dcf5: Neither blocks nor log segments found. Creating new log.
I20260709 13:24:29.410542 27955 leader_election.cc:290] T 27629b85cc3e4cb6b837dd26e3ebff47 P 0788dbdabe574355bb323dcdc3be0058 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9cb4300ea183421ab40709a05de5a877 (127.14.177.129:33597), 880fc4c7dff4417187b75cc1d9e9dcf5 (127.14.177.131:34235)
I20260709 13:24:29.411154 28052 raft_consensus.cc:359] T 04c481ece2f94a1b8cb6337900da4862 P 9cb4300ea183421ab40709a05de5a877 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb4300ea183421ab40709a05de5a877" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 33597 } } peers { permanent_uuid: "0788dbdabe574355bb323dcdc3be0058" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 41335 } } peers { permanent_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34235 } }
I20260709 13:24:29.411943 27911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27629b85cc3e4cb6b837dd26e3ebff47" candidate_uuid: "0788dbdabe574355bb323dcdc3be0058" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" is_pre_election: true
I20260709 13:24:29.412647 28052 raft_consensus.cc:385] T 04c481ece2f94a1b8cb6337900da4862 P 9cb4300ea183421ab40709a05de5a877 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:24:29.412657 27760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27629b85cc3e4cb6b837dd26e3ebff47" candidate_uuid: "0788dbdabe574355bb323dcdc3be0058" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9cb4300ea183421ab40709a05de5a877" is_pre_election: true
I20260709 13:24:29.412868 27911 raft_consensus.cc:2468] T 27629b85cc3e4cb6b837dd26e3ebff47 P 880fc4c7dff4417187b75cc1d9e9dcf5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0788dbdabe574355bb323dcdc3be0058 in term 0.
I20260709 13:24:29.413522 28052 raft_consensus.cc:740] T 04c481ece2f94a1b8cb6337900da4862 P 9cb4300ea183421ab40709a05de5a877 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9cb4300ea183421ab40709a05de5a877, State: Initialized, Role: FOLLOWER
I20260709 13:24:29.414759 27802 leader_election.cc:304] T 27629b85cc3e4cb6b837dd26e3ebff47 P 0788dbdabe574355bb323dcdc3be0058 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0788dbdabe574355bb323dcdc3be0058, 880fc4c7dff4417187b75cc1d9e9dcf5; no voters: 
I20260709 13:24:29.414696 28052 consensus_queue.cc:260] T 04c481ece2f94a1b8cb6337900da4862 P 9cb4300ea183421ab40709a05de5a877 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb4300ea183421ab40709a05de5a877" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 33597 } } peers { permanent_uuid: "0788dbdabe574355bb323dcdc3be0058" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 41335 } } peers { permanent_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34235 } }
W20260709 13:24:29.415884 27800 leader_election.cc:343] T 27629b85cc3e4cb6b837dd26e3ebff47 P 0788dbdabe574355bb323dcdc3be0058 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9cb4300ea183421ab40709a05de5a877 (127.14.177.129:33597): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:24:29.418517 28052 ts_tablet_manager.cc:1434] T 04c481ece2f94a1b8cb6337900da4862 P 9cb4300ea183421ab40709a05de5a877: Time spent starting tablet: real 0.011s	user 0.008s	sys 0.000s
I20260709 13:24:29.419576 27955 raft_consensus.cc:2804] T 27629b85cc3e4cb6b837dd26e3ebff47 P 0788dbdabe574355bb323dcdc3be0058 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:24:29.420136 27955 raft_consensus.cc:493] T 27629b85cc3e4cb6b837dd26e3ebff47 P 0788dbdabe574355bb323dcdc3be0058 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:24:29.420401 28052 tablet_bootstrap.cc:492] T 032f598cb30343c2a1ca3bab728e5b39 P 9cb4300ea183421ab40709a05de5a877: Bootstrap starting.
I20260709 13:24:29.420573 27955 raft_consensus.cc:3060] T 27629b85cc3e4cb6b837dd26e3ebff47 P 0788dbdabe574355bb323dcdc3be0058 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:24:29.421891 28051 tablet_bootstrap.cc:492] T d7fd85f92c0a481aa54401371638252b P 0788dbdabe574355bb323dcdc3be0058: No bootstrap required, opened a new log
I20260709 13:24:29.422526 28051 ts_tablet_manager.cc:1403] T d7fd85f92c0a481aa54401371638252b P 0788dbdabe574355bb323dcdc3be0058: Time spent bootstrapping tablet: real 0.032s	user 0.023s	sys 0.004s
I20260709 13:24:29.426183 28051 raft_consensus.cc:359] T d7fd85f92c0a481aa54401371638252b P 0788dbdabe574355bb323dcdc3be0058 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb4300ea183421ab40709a05de5a877" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 33597 } } peers { permanent_uuid: "0788dbdabe574355bb323dcdc3be0058" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 41335 } } peers { permanent_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34235 } }
I20260709 13:24:29.427055 28051 raft_consensus.cc:385] T d7fd85f92c0a481aa54401371638252b P 0788dbdabe574355bb323dcdc3be0058 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:24:29.427436 28051 raft_consensus.cc:740] T d7fd85f92c0a481aa54401371638252b P 0788dbdabe574355bb323dcdc3be0058 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0788dbdabe574355bb323dcdc3be0058, State: Initialized, Role: FOLLOWER
I20260709 13:24:29.428267 28051 consensus_queue.cc:260] T d7fd85f92c0a481aa54401371638252b P 0788dbdabe574355bb323dcdc3be0058 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb4300ea183421ab40709a05de5a877" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 33597 } } peers { permanent_uuid: "0788dbdabe574355bb323dcdc3be0058" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 41335 } } peers { permanent_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34235 } }
I20260709 13:24:29.428776 28052 tablet_bootstrap.cc:654] T 032f598cb30343c2a1ca3bab728e5b39 P 9cb4300ea183421ab40709a05de5a877: Neither blocks nor log segments found. Creating new log.
I20260709 13:24:29.431417 27955 raft_consensus.cc:515] T 27629b85cc3e4cb6b837dd26e3ebff47 P 0788dbdabe574355bb323dcdc3be0058 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb4300ea183421ab40709a05de5a877" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 33597 } } peers { permanent_uuid: "0788dbdabe574355bb323dcdc3be0058" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 41335 } } peers { permanent_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34235 } }
I20260709 13:24:29.435230 27760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27629b85cc3e4cb6b837dd26e3ebff47" candidate_uuid: "0788dbdabe574355bb323dcdc3be0058" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9cb4300ea183421ab40709a05de5a877"
W20260709 13:24:29.437345 27800 leader_election.cc:343] T 27629b85cc3e4cb6b837dd26e3ebff47 P 0788dbdabe574355bb323dcdc3be0058 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 9cb4300ea183421ab40709a05de5a877 (127.14.177.129:33597): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:24:29.439139 28051 ts_tablet_manager.cc:1434] T d7fd85f92c0a481aa54401371638252b P 0788dbdabe574355bb323dcdc3be0058: Time spent starting tablet: real 0.016s	user 0.009s	sys 0.000s
I20260709 13:24:29.438913 27911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27629b85cc3e4cb6b837dd26e3ebff47" candidate_uuid: "0788dbdabe574355bb323dcdc3be0058" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5"
I20260709 13:24:29.439824 27911 raft_consensus.cc:3060] T 27629b85cc3e4cb6b837dd26e3ebff47 P 880fc4c7dff4417187b75cc1d9e9dcf5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:24:29.439904 28050 tablet_bootstrap.cc:492] T 660aa4903c494ba1bce988191700e129 P 880fc4c7dff4417187b75cc1d9e9dcf5: No bootstrap required, opened a new log
I20260709 13:24:29.440595 28050 ts_tablet_manager.cc:1403] T 660aa4903c494ba1bce988191700e129 P 880fc4c7dff4417187b75cc1d9e9dcf5: Time spent bootstrapping tablet: real 0.039s	user 0.009s	sys 0.010s
I20260709 13:24:29.440351 28051 tablet_bootstrap.cc:492] T 1af49a625b20491586dcea4961e46734 P 0788dbdabe574355bb323dcdc3be0058: Bootstrap starting.
I20260709 13:24:29.444357 28050 raft_consensus.cc:359] T 660aa4903c494ba1bce988191700e129 P 880fc4c7dff4417187b75cc1d9e9dcf5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb4300ea183421ab40709a05de5a877" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 33597 } } peers { permanent_uuid: "0788dbdabe574355bb323dcdc3be0058" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 41335 } } peers { permanent_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34235 } }
I20260709 13:24:29.445329 28050 raft_consensus.cc:385] T 660aa4903c494ba1bce988191700e129 P 880fc4c7dff4417187b75cc1d9e9dcf5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:24:29.445771 28050 raft_consensus.cc:740] T 660aa4903c494ba1bce988191700e129 P 880fc4c7dff4417187b75cc1d9e9dcf5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 880fc4c7dff4417187b75cc1d9e9dcf5, State: Initialized, Role: FOLLOWER
I20260709 13:24:29.446762 28050 consensus_queue.cc:260] T 660aa4903c494ba1bce988191700e129 P 880fc4c7dff4417187b75cc1d9e9dcf5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb4300ea183421ab40709a05de5a877" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 33597 } } peers { permanent_uuid: "0788dbdabe574355bb323dcdc3be0058" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 41335 } } peers { permanent_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34235 } }
I20260709 13:24:29.449753 28051 tablet_bootstrap.cc:654] T 1af49a625b20491586dcea4961e46734 P 0788dbdabe574355bb323dcdc3be0058: Neither blocks nor log segments found. Creating new log.
I20260709 13:24:29.450515 28050 ts_tablet_manager.cc:1434] T 660aa4903c494ba1bce988191700e129 P 880fc4c7dff4417187b75cc1d9e9dcf5: Time spent starting tablet: real 0.010s	user 0.006s	sys 0.004s
I20260709 13:24:29.452319 28050 tablet_bootstrap.cc:492] T 1af49a625b20491586dcea4961e46734 P 880fc4c7dff4417187b75cc1d9e9dcf5: Bootstrap starting.
I20260709 13:24:29.455623 27911 raft_consensus.cc:2468] T 27629b85cc3e4cb6b837dd26e3ebff47 P 880fc4c7dff4417187b75cc1d9e9dcf5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0788dbdabe574355bb323dcdc3be0058 in term 1.
I20260709 13:24:29.457830 27802 leader_election.cc:304] T 27629b85cc3e4cb6b837dd26e3ebff47 P 0788dbdabe574355bb323dcdc3be0058 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0788dbdabe574355bb323dcdc3be0058, 880fc4c7dff4417187b75cc1d9e9dcf5; no voters: 9cb4300ea183421ab40709a05de5a877
I20260709 13:24:29.456296 27955 leader_election.cc:290] T 27629b85cc3e4cb6b837dd26e3ebff47 P 0788dbdabe574355bb323dcdc3be0058 [CANDIDATE]: Term 1 election: Requested vote from peers 9cb4300ea183421ab40709a05de5a877 (127.14.177.129:33597), 880fc4c7dff4417187b75cc1d9e9dcf5 (127.14.177.131:34235)
I20260709 13:24:29.462100 28050 tablet_bootstrap.cc:654] T 1af49a625b20491586dcea4961e46734 P 880fc4c7dff4417187b75cc1d9e9dcf5: Neither blocks nor log segments found. Creating new log.
I20260709 13:24:29.463225 28065 raft_consensus.cc:493] T 8e6b55496c134353a770e2642fd7490d P 880fc4c7dff4417187b75cc1d9e9dcf5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:24:29.463863 28065 raft_consensus.cc:515] T 8e6b55496c134353a770e2642fd7490d P 880fc4c7dff4417187b75cc1d9e9dcf5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb4300ea183421ab40709a05de5a877" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 33597 } } peers { permanent_uuid: "0788dbdabe574355bb323dcdc3be0058" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 41335 } } peers { permanent_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34235 } }
I20260709 13:24:29.466401 28065 leader_election.cc:290] T 8e6b55496c134353a770e2642fd7490d P 880fc4c7dff4417187b75cc1d9e9dcf5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9cb4300ea183421ab40709a05de5a877 (127.14.177.129:33597), 0788dbdabe574355bb323dcdc3be0058 (127.14.177.130:41335)
I20260709 13:24:29.462596 27955 raft_consensus.cc:2804] T 27629b85cc3e4cb6b837dd26e3ebff47 P 0788dbdabe574355bb323dcdc3be0058 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:24:29.468106 27760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e6b55496c134353a770e2642fd7490d" candidate_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9cb4300ea183421ab40709a05de5a877" is_pre_election: true
I20260709 13:24:29.469228 27760 raft_consensus.cc:2468] T 8e6b55496c134353a770e2642fd7490d P 9cb4300ea183421ab40709a05de5a877 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 880fc4c7dff4417187b75cc1d9e9dcf5 in term 0.
I20260709 13:24:29.471107 27837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e6b55496c134353a770e2642fd7490d" candidate_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0788dbdabe574355bb323dcdc3be0058" is_pre_election: true
I20260709 13:24:29.472007 27837 raft_consensus.cc:2468] T 8e6b55496c134353a770e2642fd7490d P 0788dbdabe574355bb323dcdc3be0058 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 880fc4c7dff4417187b75cc1d9e9dcf5 in term 0.
I20260709 13:24:29.472179 27874 leader_election.cc:304] T 8e6b55496c134353a770e2642fd7490d P 880fc4c7dff4417187b75cc1d9e9dcf5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 880fc4c7dff4417187b75cc1d9e9dcf5, 9cb4300ea183421ab40709a05de5a877; no voters: 
I20260709 13:24:29.478052 28065 raft_consensus.cc:2804] T 8e6b55496c134353a770e2642fd7490d P 880fc4c7dff4417187b75cc1d9e9dcf5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:24:29.478637 28065 raft_consensus.cc:493] T 8e6b55496c134353a770e2642fd7490d P 880fc4c7dff4417187b75cc1d9e9dcf5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:24:29.479081 28065 raft_consensus.cc:3060] T 8e6b55496c134353a770e2642fd7490d P 880fc4c7dff4417187b75cc1d9e9dcf5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:24:29.468093 27955 raft_consensus.cc:697] T 27629b85cc3e4cb6b837dd26e3ebff47 P 0788dbdabe574355bb323dcdc3be0058 [term 1 LEADER]: Becoming Leader. State: Replica: 0788dbdabe574355bb323dcdc3be0058, State: Running, Role: LEADER
I20260709 13:24:29.483014 27978 raft_consensus.cc:493] T 04c481ece2f94a1b8cb6337900da4862 P 9cb4300ea183421ab40709a05de5a877 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:24:29.485396 27978 raft_consensus.cc:515] T 04c481ece2f94a1b8cb6337900da4862 P 9cb4300ea183421ab40709a05de5a877 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb4300ea183421ab40709a05de5a877" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 33597 } } peers { permanent_uuid: "0788dbdabe574355bb323dcdc3be0058" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 41335 } } peers { permanent_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34235 } }
I20260709 13:24:29.487548 27990 raft_consensus.cc:493] T 660aa4903c494ba1bce988191700e129 P 880fc4c7dff4417187b75cc1d9e9dcf5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:24:29.488196 27990 raft_consensus.cc:515] T 660aa4903c494ba1bce988191700e129 P 880fc4c7dff4417187b75cc1d9e9dcf5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb4300ea183421ab40709a05de5a877" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 33597 } } peers { permanent_uuid: "0788dbdabe574355bb323dcdc3be0058" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 41335 } } peers { permanent_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34235 } }
I20260709 13:24:29.489699 27837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04c481ece2f94a1b8cb6337900da4862" candidate_uuid: "9cb4300ea183421ab40709a05de5a877" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0788dbdabe574355bb323dcdc3be0058" is_pre_election: true
I20260709 13:24:29.490653 27837 raft_consensus.cc:2468] T 04c481ece2f94a1b8cb6337900da4862 P 0788dbdabe574355bb323dcdc3be0058 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9cb4300ea183421ab40709a05de5a877 in term 0.
I20260709 13:24:29.490653 27990 leader_election.cc:290] T 660aa4903c494ba1bce988191700e129 P 880fc4c7dff4417187b75cc1d9e9dcf5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9cb4300ea183421ab40709a05de5a877 (127.14.177.129:33597), 0788dbdabe574355bb323dcdc3be0058 (127.14.177.130:41335)
I20260709 13:24:29.492126 27835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "660aa4903c494ba1bce988191700e129" candidate_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0788dbdabe574355bb323dcdc3be0058" is_pre_election: true
I20260709 13:24:29.493046 27835 raft_consensus.cc:2468] T 660aa4903c494ba1bce988191700e129 P 0788dbdabe574355bb323dcdc3be0058 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 880fc4c7dff4417187b75cc1d9e9dcf5 in term 0.
I20260709 13:24:29.494778 27876 leader_election.cc:304] T 660aa4903c494ba1bce988191700e129 P 880fc4c7dff4417187b75cc1d9e9dcf5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0788dbdabe574355bb323dcdc3be0058, 880fc4c7dff4417187b75cc1d9e9dcf5; no voters: 
I20260709 13:24:29.497530 27760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "660aa4903c494ba1bce988191700e129" candidate_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9cb4300ea183421ab40709a05de5a877" is_pre_election: true
I20260709 13:24:29.498478 27760 raft_consensus.cc:2468] T 660aa4903c494ba1bce988191700e129 P 9cb4300ea183421ab40709a05de5a877 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 880fc4c7dff4417187b75cc1d9e9dcf5 in term 0.
I20260709 13:24:29.499471 28052 tablet_bootstrap.cc:492] T 032f598cb30343c2a1ca3bab728e5b39 P 9cb4300ea183421ab40709a05de5a877: No bootstrap required, opened a new log
I20260709 13:24:29.499804 28071 raft_consensus.cc:493] T d7fd85f92c0a481aa54401371638252b P 0788dbdabe574355bb323dcdc3be0058 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:24:29.500447 28052 ts_tablet_manager.cc:1403] T 032f598cb30343c2a1ca3bab728e5b39 P 9cb4300ea183421ab40709a05de5a877: Time spent bootstrapping tablet: real 0.080s	user 0.029s	sys 0.014s
I20260709 13:24:29.500568 28071 raft_consensus.cc:515] T d7fd85f92c0a481aa54401371638252b P 0788dbdabe574355bb323dcdc3be0058 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb4300ea183421ab40709a05de5a877" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 33597 } } peers { permanent_uuid: "0788dbdabe574355bb323dcdc3be0058" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 41335 } } peers { permanent_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34235 } }
I20260709 13:24:29.501334 27990 raft_consensus.cc:2804] T 660aa4903c494ba1bce988191700e129 P 880fc4c7dff4417187b75cc1d9e9dcf5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:24:29.502417 27990 raft_consensus.cc:493] T 660aa4903c494ba1bce988191700e129 P 880fc4c7dff4417187b75cc1d9e9dcf5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:24:29.502846 27990 raft_consensus.cc:3060] T 660aa4903c494ba1bce988191700e129 P 880fc4c7dff4417187b75cc1d9e9dcf5 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:24:29.504492 27802 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:34235, user_credentials={real_user=slave}} blocked reactor thread for 46756.8us
I20260709 13:24:29.504806 28052 raft_consensus.cc:359] T 032f598cb30343c2a1ca3bab728e5b39 P 9cb4300ea183421ab40709a05de5a877 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb4300ea183421ab40709a05de5a877" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 33597 } } peers { permanent_uuid: "0788dbdabe574355bb323dcdc3be0058" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 41335 } } peers { permanent_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34235 } }
I20260709 13:24:29.506083 28052 raft_consensus.cc:385] T 032f598cb30343c2a1ca3bab728e5b39 P 9cb4300ea183421ab40709a05de5a877 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:24:29.506946 28052 raft_consensus.cc:740] T 032f598cb30343c2a1ca3bab728e5b39 P 9cb4300ea183421ab40709a05de5a877 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9cb4300ea183421ab40709a05de5a877, State: Initialized, Role: FOLLOWER
I20260709 13:24:29.508549 28052 consensus_queue.cc:260] T 032f598cb30343c2a1ca3bab728e5b39 P 9cb4300ea183421ab40709a05de5a877 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb4300ea183421ab40709a05de5a877" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 33597 } } peers { permanent_uuid: "0788dbdabe574355bb323dcdc3be0058" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 41335 } } peers { permanent_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34235 } }
I20260709 13:24:29.492157 27978 leader_election.cc:290] T 04c481ece2f94a1b8cb6337900da4862 P 9cb4300ea183421ab40709a05de5a877 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0788dbdabe574355bb323dcdc3be0058 (127.14.177.130:41335), 880fc4c7dff4417187b75cc1d9e9dcf5 (127.14.177.131:34235)
I20260709 13:24:29.513295 27978 raft_consensus.cc:493] T d7fd85f92c0a481aa54401371638252b P 9cb4300ea183421ab40709a05de5a877 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:24:29.513864 27760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7fd85f92c0a481aa54401371638252b" candidate_uuid: "0788dbdabe574355bb323dcdc3be0058" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9cb4300ea183421ab40709a05de5a877" is_pre_election: true
I20260709 13:24:29.515967 27978 raft_consensus.cc:515] T d7fd85f92c0a481aa54401371638252b P 9cb4300ea183421ab40709a05de5a877 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb4300ea183421ab40709a05de5a877" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 33597 } } peers { permanent_uuid: "0788dbdabe574355bb323dcdc3be0058" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 41335 } } peers { permanent_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34235 } }
I20260709 13:24:29.519912 27760 raft_consensus.cc:2468] T d7fd85f92c0a481aa54401371638252b P 9cb4300ea183421ab40709a05de5a877 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0788dbdabe574355bb323dcdc3be0058 in term 0.
I20260709 13:24:29.520263 27911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04c481ece2f94a1b8cb6337900da4862" candidate_uuid: "9cb4300ea183421ab40709a05de5a877" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" is_pre_election: true
I20260709 13:24:29.522883 27911 raft_consensus.cc:2468] T 04c481ece2f94a1b8cb6337900da4862 P 880fc4c7dff4417187b75cc1d9e9dcf5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9cb4300ea183421ab40709a05de5a877 in term 0.
I20260709 13:24:29.521759 27727 leader_election.cc:304] T 04c481ece2f94a1b8cb6337900da4862 P 9cb4300ea183421ab40709a05de5a877 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0788dbdabe574355bb323dcdc3be0058, 9cb4300ea183421ab40709a05de5a877; no voters: 
I20260709 13:24:29.524024 27800 leader_election.cc:304] T d7fd85f92c0a481aa54401371638252b P 0788dbdabe574355bb323dcdc3be0058 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0788dbdabe574355bb323dcdc3be0058, 9cb4300ea183421ab40709a05de5a877; no voters: 
I20260709 13:24:29.486501 27955 consensus_queue.cc:237] T 27629b85cc3e4cb6b837dd26e3ebff47 P 0788dbdabe574355bb323dcdc3be0058 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb4300ea183421ab40709a05de5a877" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 33597 } } peers { permanent_uuid: "0788dbdabe574355bb323dcdc3be0058" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 41335 } } peers { permanent_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34235 } }
I20260709 13:24:29.526732 28079 raft_consensus.cc:2804] T 04c481ece2f94a1b8cb6337900da4862 P 9cb4300ea183421ab40709a05de5a877 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:24:29.527361 28079 raft_consensus.cc:493] T 04c481ece2f94a1b8cb6337900da4862 P 9cb4300ea183421ab40709a05de5a877 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:24:29.527841 28079 raft_consensus.cc:3060] T 04c481ece2f94a1b8cb6337900da4862 P 9cb4300ea183421ab40709a05de5a877 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:24:29.528072 28052 ts_tablet_manager.cc:1434] T 032f598cb30343c2a1ca3bab728e5b39 P 9cb4300ea183421ab40709a05de5a877: Time spent starting tablet: real 0.027s	user 0.008s	sys 0.008s
I20260709 13:24:29.531595 28052 tablet_bootstrap.cc:492] T 27629b85cc3e4cb6b837dd26e3ebff47 P 9cb4300ea183421ab40709a05de5a877: Bootstrap starting.
I20260709 13:24:29.541774 28065 raft_consensus.cc:515] T 8e6b55496c134353a770e2642fd7490d P 880fc4c7dff4417187b75cc1d9e9dcf5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb4300ea183421ab40709a05de5a877" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 33597 } } peers { permanent_uuid: "0788dbdabe574355bb323dcdc3be0058" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 41335 } } peers { permanent_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34235 } }
I20260709 13:24:29.543497 27835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7fd85f92c0a481aa54401371638252b" candidate_uuid: "9cb4300ea183421ab40709a05de5a877" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0788dbdabe574355bb323dcdc3be0058" is_pre_election: true
I20260709 13:24:29.544359 27835 raft_consensus.cc:2468] T d7fd85f92c0a481aa54401371638252b P 0788dbdabe574355bb323dcdc3be0058 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9cb4300ea183421ab40709a05de5a877 in term 0.
I20260709 13:24:29.546195 27727 leader_election.cc:304] T d7fd85f92c0a481aa54401371638252b P 9cb4300ea183421ab40709a05de5a877 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0788dbdabe574355bb323dcdc3be0058, 9cb4300ea183421ab40709a05de5a877; no voters: 
I20260709 13:24:29.547400 28076 raft_consensus.cc:2804] T d7fd85f92c0a481aa54401371638252b P 9cb4300ea183421ab40709a05de5a877 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:24:29.547768 27978 leader_election.cc:290] T d7fd85f92c0a481aa54401371638252b P 9cb4300ea183421ab40709a05de5a877 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0788dbdabe574355bb323dcdc3be0058 (127.14.177.130:41335), 880fc4c7dff4417187b75cc1d9e9dcf5 (127.14.177.131:34235)
I20260709 13:24:29.547967 28076 raft_consensus.cc:493] T d7fd85f92c0a481aa54401371638252b P 9cb4300ea183421ab40709a05de5a877 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:24:29.548543 28076 raft_consensus.cc:3060] T d7fd85f92c0a481aa54401371638252b P 9cb4300ea183421ab40709a05de5a877 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:24:29.564389 28065 leader_election.cc:290] T 8e6b55496c134353a770e2642fd7490d P 880fc4c7dff4417187b75cc1d9e9dcf5 [CANDIDATE]: Term 1 election: Requested vote from peers 9cb4300ea183421ab40709a05de5a877 (127.14.177.129:33597), 0788dbdabe574355bb323dcdc3be0058 (127.14.177.130:41335)
I20260709 13:24:29.549624 27911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7fd85f92c0a481aa54401371638252b" candidate_uuid: "9cb4300ea183421ab40709a05de5a877" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" is_pre_election: true
I20260709 13:24:29.565313 27911 raft_consensus.cc:2468] T d7fd85f92c0a481aa54401371638252b P 880fc4c7dff4417187b75cc1d9e9dcf5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9cb4300ea183421ab40709a05de5a877 in term 0.
I20260709 13:24:29.565775 27760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e6b55496c134353a770e2642fd7490d" candidate_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9cb4300ea183421ab40709a05de5a877"
I20260709 13:24:29.542910 28079 raft_consensus.cc:515] T 04c481ece2f94a1b8cb6337900da4862 P 9cb4300ea183421ab40709a05de5a877 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb4300ea183421ab40709a05de5a877" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 33597 } } peers { permanent_uuid: "0788dbdabe574355bb323dcdc3be0058" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 41335 } } peers { permanent_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34235 } }
I20260709 13:24:29.551245 27978 raft_consensus.cc:493] T d747d5acdf4f41b69c132cde690e49ff P 9cb4300ea183421ab40709a05de5a877 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:24:29.514950 28071 leader_election.cc:290] T d7fd85f92c0a481aa54401371638252b P 0788dbdabe574355bb323dcdc3be0058 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9cb4300ea183421ab40709a05de5a877 (127.14.177.129:33597), 880fc4c7dff4417187b75cc1d9e9dcf5 (127.14.177.131:34235)
I20260709 13:24:29.553707 27910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7fd85f92c0a481aa54401371638252b" candidate_uuid: "0788dbdabe574355bb323dcdc3be0058" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" is_pre_election: true
I20260709 13:24:29.556217 28051 tablet_bootstrap.cc:492] T 1af49a625b20491586dcea4961e46734 P 0788dbdabe574355bb323dcdc3be0058: No bootstrap required, opened a new log
I20260709 13:24:29.569438 27978 raft_consensus.cc:515] T d747d5acdf4f41b69c132cde690e49ff P 9cb4300ea183421ab40709a05de5a877 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb4300ea183421ab40709a05de5a877" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 33597 } } peers { permanent_uuid: "0788dbdabe574355bb323dcdc3be0058" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 41335 } } peers { permanent_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34235 } }
I20260709 13:24:29.569923 27910 raft_consensus.cc:2468] T d7fd85f92c0a481aa54401371638252b P 880fc4c7dff4417187b75cc1d9e9dcf5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0788dbdabe574355bb323dcdc3be0058 in term 0.
I20260709 13:24:29.570120 28051 ts_tablet_manager.cc:1403] T 1af49a625b20491586dcea4961e46734 P 0788dbdabe574355bb323dcdc3be0058: Time spent bootstrapping tablet: real 0.130s	user 0.032s	sys 0.024s
I20260709 13:24:29.570765 28079 leader_election.cc:290] T 04c481ece2f94a1b8cb6337900da4862 P 9cb4300ea183421ab40709a05de5a877 [CANDIDATE]: Term 1 election: Requested vote from peers 0788dbdabe574355bb323dcdc3be0058 (127.14.177.130:41335), 880fc4c7dff4417187b75cc1d9e9dcf5 (127.14.177.131:34235)
I20260709 13:24:29.549470 28052 tablet_bootstrap.cc:654] T 27629b85cc3e4cb6b837dd26e3ebff47 P 9cb4300ea183421ab40709a05de5a877: Neither blocks nor log segments found. Creating new log.
I20260709 13:24:29.550195 28078 raft_consensus.cc:2804] T d7fd85f92c0a481aa54401371638252b P 0788dbdabe574355bb323dcdc3be0058 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:24:29.571936 27978 leader_election.cc:290] T d747d5acdf4f41b69c132cde690e49ff P 9cb4300ea183421ab40709a05de5a877 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0788dbdabe574355bb323dcdc3be0058 (127.14.177.130:41335), 880fc4c7dff4417187b75cc1d9e9dcf5 (127.14.177.131:34235)
I20260709 13:24:29.572256 28078 raft_consensus.cc:493] T d7fd85f92c0a481aa54401371638252b P 0788dbdabe574355bb323dcdc3be0058 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:24:29.572713 28078 raft_consensus.cc:3060] T d7fd85f92c0a481aa54401371638252b P 0788dbdabe574355bb323dcdc3be0058 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:24:29.574120 27837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04c481ece2f94a1b8cb6337900da4862" candidate_uuid: "9cb4300ea183421ab40709a05de5a877" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0788dbdabe574355bb323dcdc3be0058"
I20260709 13:24:29.574116 28051 raft_consensus.cc:359] T 1af49a625b20491586dcea4961e46734 P 0788dbdabe574355bb323dcdc3be0058 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb4300ea183421ab40709a05de5a877" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 33597 } } peers { permanent_uuid: "0788dbdabe574355bb323dcdc3be0058" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 41335 } } peers { permanent_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34235 } }
I20260709 13:24:29.575080 27837 raft_consensus.cc:3060] T 04c481ece2f94a1b8cb6337900da4862 P 0788dbdabe574355bb323dcdc3be0058 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:24:29.575196 28051 raft_consensus.cc:385] T 1af49a625b20491586dcea4961e46734 P 0788dbdabe574355bb323dcdc3be0058 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:24:29.575812 28051 raft_consensus.cc:740] T 1af49a625b20491586dcea4961e46734 P 0788dbdabe574355bb323dcdc3be0058 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0788dbdabe574355bb323dcdc3be0058, State: Initialized, Role: FOLLOWER
I20260709 13:24:29.576726 28051 consensus_queue.cc:260] T 1af49a625b20491586dcea4961e46734 P 0788dbdabe574355bb323dcdc3be0058 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb4300ea183421ab40709a05de5a877" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 33597 } } peers { permanent_uuid: "0788dbdabe574355bb323dcdc3be0058" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 41335 } } peers { permanent_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34235 } }
I20260709 13:24:29.568442 27835 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e6b55496c134353a770e2642fd7490d" candidate_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0788dbdabe574355bb323dcdc3be0058"
I20260709 13:24:29.579558 27835 raft_consensus.cc:3060] T 8e6b55496c134353a770e2642fd7490d P 0788dbdabe574355bb323dcdc3be0058 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:24:29.582368 28078 raft_consensus.cc:515] T d7fd85f92c0a481aa54401371638252b P 0788dbdabe574355bb323dcdc3be0058 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb4300ea183421ab40709a05de5a877" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 33597 } } peers { permanent_uuid: "0788dbdabe574355bb323dcdc3be0058" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 41335 } } peers { permanent_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34235 } }
I20260709 13:24:29.584641 27910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04c481ece2f94a1b8cb6337900da4862" candidate_uuid: "9cb4300ea183421ab40709a05de5a877" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5"
I20260709 13:24:29.585481 27910 raft_consensus.cc:3060] T 04c481ece2f94a1b8cb6337900da4862 P 880fc4c7dff4417187b75cc1d9e9dcf5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:24:29.585577 27990 raft_consensus.cc:515] T 660aa4903c494ba1bce988191700e129 P 880fc4c7dff4417187b75cc1d9e9dcf5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb4300ea183421ab40709a05de5a877" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 33597 } } peers { permanent_uuid: "0788dbdabe574355bb323dcdc3be0058" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 41335 } } peers { permanent_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34235 } }
I20260709 13:24:29.586212 27911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7fd85f92c0a481aa54401371638252b" candidate_uuid: "0788dbdabe574355bb323dcdc3be0058" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5"
I20260709 13:24:29.587190 27911 raft_consensus.cc:3060] T d7fd85f92c0a481aa54401371638252b P 880fc4c7dff4417187b75cc1d9e9dcf5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:24:29.588591 28051 ts_tablet_manager.cc:1434] T 1af49a625b20491586dcea4961e46734 P 0788dbdabe574355bb323dcdc3be0058: Time spent starting tablet: real 0.018s	user 0.009s	sys 0.000s
I20260709 13:24:29.589941 28051 tablet_bootstrap.cc:492] T 032f598cb30343c2a1ca3bab728e5b39 P 0788dbdabe574355bb323dcdc3be0058: Bootstrap starting.
I20260709 13:24:29.593189 27762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7fd85f92c0a481aa54401371638252b" candidate_uuid: "0788dbdabe574355bb323dcdc3be0058" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9cb4300ea183421ab40709a05de5a877"
I20260709 13:24:29.594403 27836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "660aa4903c494ba1bce988191700e129" candidate_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0788dbdabe574355bb323dcdc3be0058"
I20260709 13:24:29.594981 27990 leader_election.cc:290] T 660aa4903c494ba1bce988191700e129 P 880fc4c7dff4417187b75cc1d9e9dcf5 [CANDIDATE]: Term 1 election: Requested vote from peers 9cb4300ea183421ab40709a05de5a877 (127.14.177.129:33597), 0788dbdabe574355bb323dcdc3be0058 (127.14.177.130:41335)
I20260709 13:24:29.595268 27836 raft_consensus.cc:3060] T 660aa4903c494ba1bce988191700e129 P 0788dbdabe574355bb323dcdc3be0058 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:24:29.597633 28050 tablet_bootstrap.cc:492] T 1af49a625b20491586dcea4961e46734 P 880fc4c7dff4417187b75cc1d9e9dcf5: No bootstrap required, opened a new log
I20260709 13:24:29.598354 28050 ts_tablet_manager.cc:1403] T 1af49a625b20491586dcea4961e46734 P 880fc4c7dff4417187b75cc1d9e9dcf5: Time spent bootstrapping tablet: real 0.146s	user 0.044s	sys 0.032s
I20260709 13:24:29.602520 28051 tablet_bootstrap.cc:654] T 032f598cb30343c2a1ca3bab728e5b39 P 0788dbdabe574355bb323dcdc3be0058: Neither blocks nor log segments found. Creating new log.
I20260709 13:24:29.602208 28050 raft_consensus.cc:359] T 1af49a625b20491586dcea4961e46734 P 880fc4c7dff4417187b75cc1d9e9dcf5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb4300ea183421ab40709a05de5a877" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 33597 } } peers { permanent_uuid: "0788dbdabe574355bb323dcdc3be0058" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 41335 } } peers { permanent_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34235 } }
I20260709 13:24:29.603327 28050 raft_consensus.cc:385] T 1af49a625b20491586dcea4961e46734 P 880fc4c7dff4417187b75cc1d9e9dcf5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:24:29.603749 28050 raft_consensus.cc:740] T 1af49a625b20491586dcea4961e46734 P 880fc4c7dff4417187b75cc1d9e9dcf5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 880fc4c7dff4417187b75cc1d9e9dcf5, State: Initialized, Role: FOLLOWER
I20260709 13:24:29.604646 28050 consensus_queue.cc:260] T 1af49a625b20491586dcea4961e46734 P 880fc4c7dff4417187b75cc1d9e9dcf5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb4300ea183421ab40709a05de5a877" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 33597 } } peers { permanent_uuid: "0788dbdabe574355bb323dcdc3be0058" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 41335 } } peers { permanent_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34235 } }
I20260709 13:24:29.606205 27990 raft_consensus.cc:493] T d747d5acdf4f41b69c132cde690e49ff P 880fc4c7dff4417187b75cc1d9e9dcf5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:24:29.606491 27911 raft_consensus.cc:2468] T d7fd85f92c0a481aa54401371638252b P 880fc4c7dff4417187b75cc1d9e9dcf5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0788dbdabe574355bb323dcdc3be0058 in term 1.
I20260709 13:24:29.606804 27990 raft_consensus.cc:515] T d747d5acdf4f41b69c132cde690e49ff P 880fc4c7dff4417187b75cc1d9e9dcf5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb4300ea183421ab40709a05de5a877" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 33597 } } peers { permanent_uuid: "0788dbdabe574355bb323dcdc3be0058" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 41335 } } peers { permanent_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34235 } }
I20260709 13:24:29.607442 28050 ts_tablet_manager.cc:1434] T 1af49a625b20491586dcea4961e46734 P 880fc4c7dff4417187b75cc1d9e9dcf5: Time spent starting tablet: real 0.009s	user 0.005s	sys 0.003s
I20260709 13:24:29.616127 27990 leader_election.cc:290] T d747d5acdf4f41b69c132cde690e49ff P 880fc4c7dff4417187b75cc1d9e9dcf5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9cb4300ea183421ab40709a05de5a877 (127.14.177.129:33597), 0788dbdabe574355bb323dcdc3be0058 (127.14.177.130:41335)
I20260709 13:24:29.616376 28050 tablet_bootstrap.cc:492] T 032f598cb30343c2a1ca3bab728e5b39 P 880fc4c7dff4417187b75cc1d9e9dcf5: Bootstrap starting.
I20260709 13:24:29.609964 27835 raft_consensus.cc:2468] T 8e6b55496c134353a770e2642fd7490d P 0788dbdabe574355bb323dcdc3be0058 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 880fc4c7dff4417187b75cc1d9e9dcf5 in term 1.
I20260709 13:24:29.611123 27836 raft_consensus.cc:2468] T 660aa4903c494ba1bce988191700e129 P 0788dbdabe574355bb323dcdc3be0058 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 880fc4c7dff4417187b75cc1d9e9dcf5 in term 1.
I20260709 13:24:29.579506 28076 raft_consensus.cc:515] T d7fd85f92c0a481aa54401371638252b P 9cb4300ea183421ab40709a05de5a877 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb4300ea183421ab40709a05de5a877" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 33597 } } peers { permanent_uuid: "0788dbdabe574355bb323dcdc3be0058" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 41335 } } peers { permanent_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34235 } }
I20260709 13:24:29.618855 27834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d747d5acdf4f41b69c132cde690e49ff" candidate_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0788dbdabe574355bb323dcdc3be0058" is_pre_election: true
I20260709 13:24:29.620042 27876 leader_election.cc:304] T 8e6b55496c134353a770e2642fd7490d P 880fc4c7dff4417187b75cc1d9e9dcf5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0788dbdabe574355bb323dcdc3be0058, 880fc4c7dff4417187b75cc1d9e9dcf5; no voters: 
I20260709 13:24:29.620471 27834 raft_consensus.cc:2468] T d747d5acdf4f41b69c132cde690e49ff P 0788dbdabe574355bb323dcdc3be0058 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 880fc4c7dff4417187b75cc1d9e9dcf5 in term 0.
I20260709 13:24:29.621027 27876 leader_election.cc:304] T 660aa4903c494ba1bce988191700e129 P 880fc4c7dff4417187b75cc1d9e9dcf5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0788dbdabe574355bb323dcdc3be0058, 880fc4c7dff4417187b75cc1d9e9dcf5; no voters: 
I20260709 13:24:29.622051 28065 raft_consensus.cc:2804] T 8e6b55496c134353a770e2642fd7490d P 880fc4c7dff4417187b75cc1d9e9dcf5 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:24:29.622134 27990 raft_consensus.cc:2804] T 660aa4903c494ba1bce988191700e129 P 880fc4c7dff4417187b75cc1d9e9dcf5 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:24:29.622611 28065 raft_consensus.cc:697] T 8e6b55496c134353a770e2642fd7490d P 880fc4c7dff4417187b75cc1d9e9dcf5 [term 1 LEADER]: Becoming Leader. State: Replica: 880fc4c7dff4417187b75cc1d9e9dcf5, State: Running, Role: LEADER
I20260709 13:24:29.618793 27802 leader_election.cc:304] T d7fd85f92c0a481aa54401371638252b P 0788dbdabe574355bb323dcdc3be0058 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0788dbdabe574355bb323dcdc3be0058, 880fc4c7dff4417187b75cc1d9e9dcf5; no voters: 
I20260709 13:24:29.623131 27762 raft_consensus.cc:2393] T d7fd85f92c0a481aa54401371638252b P 9cb4300ea183421ab40709a05de5a877 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 0788dbdabe574355bb323dcdc3be0058 in current term 1: Already voted for candidate 9cb4300ea183421ab40709a05de5a877 in this term.
I20260709 13:24:29.623574 27876 leader_election.cc:304] T d747d5acdf4f41b69c132cde690e49ff P 880fc4c7dff4417187b75cc1d9e9dcf5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0788dbdabe574355bb323dcdc3be0058, 880fc4c7dff4417187b75cc1d9e9dcf5; no voters: 
I20260709 13:24:29.623991 27955 raft_consensus.cc:2804] T d7fd85f92c0a481aa54401371638252b P 0788dbdabe574355bb323dcdc3be0058 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:24:29.624622 27955 raft_consensus.cc:697] T d7fd85f92c0a481aa54401371638252b P 0788dbdabe574355bb323dcdc3be0058 [term 1 LEADER]: Becoming Leader. State: Replica: 0788dbdabe574355bb323dcdc3be0058, State: Running, Role: LEADER
I20260709 13:24:29.623620 28065 consensus_queue.cc:237] T 8e6b55496c134353a770e2642fd7490d P 880fc4c7dff4417187b75cc1d9e9dcf5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb4300ea183421ab40709a05de5a877" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 33597 } } peers { permanent_uuid: "0788dbdabe574355bb323dcdc3be0058" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 41335 } } peers { permanent_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34235 } }
I20260709 13:24:29.625552 27955 consensus_queue.cc:237] T d7fd85f92c0a481aa54401371638252b P 0788dbdabe574355bb323dcdc3be0058 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb4300ea183421ab40709a05de5a877" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 33597 } } peers { permanent_uuid: "0788dbdabe574355bb323dcdc3be0058" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 41335 } } peers { permanent_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34235 } }
I20260709 13:24:29.630100 27837 raft_consensus.cc:2468] T 04c481ece2f94a1b8cb6337900da4862 P 0788dbdabe574355bb323dcdc3be0058 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9cb4300ea183421ab40709a05de5a877 in term 1.
I20260709 13:24:29.634352 28082 raft_consensus.cc:2804] T d747d5acdf4f41b69c132cde690e49ff P 880fc4c7dff4417187b75cc1d9e9dcf5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:24:29.635136 28082 raft_consensus.cc:493] T d747d5acdf4f41b69c132cde690e49ff P 880fc4c7dff4417187b75cc1d9e9dcf5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:24:29.635720 28082 raft_consensus.cc:3060] T d747d5acdf4f41b69c132cde690e49ff P 880fc4c7dff4417187b75cc1d9e9dcf5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:24:29.621239 28076 leader_election.cc:290] T d7fd85f92c0a481aa54401371638252b P 9cb4300ea183421ab40709a05de5a877 [CANDIDATE]: Term 1 election: Requested vote from peers 0788dbdabe574355bb323dcdc3be0058 (127.14.177.130:41335), 880fc4c7dff4417187b75cc1d9e9dcf5 (127.14.177.131:34235)
I20260709 13:24:29.644742 27910 raft_consensus.cc:2468] T 04c481ece2f94a1b8cb6337900da4862 P 880fc4c7dff4417187b75cc1d9e9dcf5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9cb4300ea183421ab40709a05de5a877 in term 1.
I20260709 13:24:29.584686 28078 leader_election.cc:290] T d7fd85f92c0a481aa54401371638252b P 0788dbdabe574355bb323dcdc3be0058 [CANDIDATE]: Term 1 election: Requested vote from peers 9cb4300ea183421ab40709a05de5a877 (127.14.177.129:33597), 880fc4c7dff4417187b75cc1d9e9dcf5 (127.14.177.131:34235)
I20260709 13:24:29.650597 28052 tablet_bootstrap.cc:492] T 27629b85cc3e4cb6b837dd26e3ebff47 P 9cb4300ea183421ab40709a05de5a877: No bootstrap required, opened a new log
I20260709 13:24:29.651255 27990 raft_consensus.cc:697] T 660aa4903c494ba1bce988191700e129 P 880fc4c7dff4417187b75cc1d9e9dcf5 [term 1 LEADER]: Becoming Leader. State: Replica: 880fc4c7dff4417187b75cc1d9e9dcf5, State: Running, Role: LEADER
I20260709 13:24:29.651450 28052 ts_tablet_manager.cc:1403] T 27629b85cc3e4cb6b837dd26e3ebff47 P 9cb4300ea183421ab40709a05de5a877: Time spent bootstrapping tablet: real 0.120s	user 0.011s	sys 0.013s
I20260709 13:24:29.652366 27990 consensus_queue.cc:237] T 660aa4903c494ba1bce988191700e129 P 880fc4c7dff4417187b75cc1d9e9dcf5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb4300ea183421ab40709a05de5a877" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 33597 } } peers { permanent_uuid: "0788dbdabe574355bb323dcdc3be0058" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 41335 } } peers { permanent_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34235 } }
I20260709 13:24:29.569415 27760 raft_consensus.cc:3060] T 8e6b55496c134353a770e2642fd7490d P 9cb4300ea183421ab40709a05de5a877 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:24:29.696666 27761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d747d5acdf4f41b69c132cde690e49ff" candidate_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9cb4300ea183421ab40709a05de5a877" is_pre_election: true
I20260709 13:24:29.700207 27762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "660aa4903c494ba1bce988191700e129" candidate_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9cb4300ea183421ab40709a05de5a877"
I20260709 13:24:29.700524 28052 raft_consensus.cc:359] T 27629b85cc3e4cb6b837dd26e3ebff47 P 9cb4300ea183421ab40709a05de5a877 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb4300ea183421ab40709a05de5a877" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 33597 } } peers { permanent_uuid: "0788dbdabe574355bb323dcdc3be0058" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 41335 } } peers { permanent_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34235 } }
I20260709 13:24:29.701669 27762 raft_consensus.cc:3060] T 660aa4903c494ba1bce988191700e129 P 9cb4300ea183421ab40709a05de5a877 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:24:29.702131 28052 raft_consensus.cc:385] T 27629b85cc3e4cb6b837dd26e3ebff47 P 9cb4300ea183421ab40709a05de5a877 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:24:29.702618 28052 raft_consensus.cc:740] T 27629b85cc3e4cb6b837dd26e3ebff47 P 9cb4300ea183421ab40709a05de5a877 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9cb4300ea183421ab40709a05de5a877, State: Initialized, Role: FOLLOWER
I20260709 13:24:29.703975 28052 consensus_queue.cc:260] T 27629b85cc3e4cb6b837dd26e3ebff47 P 9cb4300ea183421ab40709a05de5a877 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb4300ea183421ab40709a05de5a877" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 33597 } } peers { permanent_uuid: "0788dbdabe574355bb323dcdc3be0058" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 41335 } } peers { permanent_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34235 } }
I20260709 13:24:29.698956 27761 raft_consensus.cc:2468] T d747d5acdf4f41b69c132cde690e49ff P 9cb4300ea183421ab40709a05de5a877 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 880fc4c7dff4417187b75cc1d9e9dcf5 in term 0.
I20260709 13:24:29.709308 27909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7fd85f92c0a481aa54401371638252b" candidate_uuid: "9cb4300ea183421ab40709a05de5a877" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5"
I20260709 13:24:29.711364 27909 raft_consensus.cc:2393] T d7fd85f92c0a481aa54401371638252b P 880fc4c7dff4417187b75cc1d9e9dcf5 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 9cb4300ea183421ab40709a05de5a877 in current term 1: Already voted for candidate 0788dbdabe574355bb323dcdc3be0058 in this term.
I20260709 13:24:29.715116 27910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d747d5acdf4f41b69c132cde690e49ff" candidate_uuid: "9cb4300ea183421ab40709a05de5a877" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" is_pre_election: true
I20260709 13:24:29.715914 28051 tablet_bootstrap.cc:492] T 032f598cb30343c2a1ca3bab728e5b39 P 0788dbdabe574355bb323dcdc3be0058: No bootstrap required, opened a new log
I20260709 13:24:29.717840 28051 ts_tablet_manager.cc:1403] T 032f598cb30343c2a1ca3bab728e5b39 P 0788dbdabe574355bb323dcdc3be0058: Time spent bootstrapping tablet: real 0.128s	user 0.024s	sys 0.023s
I20260709 13:24:29.719954 27837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d747d5acdf4f41b69c132cde690e49ff" candidate_uuid: "9cb4300ea183421ab40709a05de5a877" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0788dbdabe574355bb323dcdc3be0058" is_pre_election: true
I20260709 13:24:29.721042 27836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7fd85f92c0a481aa54401371638252b" candidate_uuid: "9cb4300ea183421ab40709a05de5a877" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0788dbdabe574355bb323dcdc3be0058"
I20260709 13:24:29.721330 27837 raft_consensus.cc:2468] T d747d5acdf4f41b69c132cde690e49ff P 0788dbdabe574355bb323dcdc3be0058 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9cb4300ea183421ab40709a05de5a877 in term 0.
I20260709 13:24:29.704021 27727 leader_election.cc:304] T 04c481ece2f94a1b8cb6337900da4862 P 9cb4300ea183421ab40709a05de5a877 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0788dbdabe574355bb323dcdc3be0058, 9cb4300ea183421ab40709a05de5a877; no voters: 
I20260709 13:24:29.732169 27660 catalog_manager.cc:5697] T 27629b85cc3e4cb6b837dd26e3ebff47 P 0788dbdabe574355bb323dcdc3be0058 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0788dbdabe574355bb323dcdc3be0058 (127.14.177.130). New cstate: current_term: 1 leader_uuid: "0788dbdabe574355bb323dcdc3be0058" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb4300ea183421ab40709a05de5a877" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 33597 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0788dbdabe574355bb323dcdc3be0058" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 41335 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34235 } health_report { overall_health: UNKNOWN } } }
I20260709 13:24:29.737044 28050 tablet_bootstrap.cc:654] T 032f598cb30343c2a1ca3bab728e5b39 P 880fc4c7dff4417187b75cc1d9e9dcf5: Neither blocks nor log segments found. Creating new log.
I20260709 13:24:29.744493 28076 raft_consensus.cc:2804] T 04c481ece2f94a1b8cb6337900da4862 P 9cb4300ea183421ab40709a05de5a877 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:24:29.745896 28076 raft_consensus.cc:697] T 04c481ece2f94a1b8cb6337900da4862 P 9cb4300ea183421ab40709a05de5a877 [term 1 LEADER]: Becoming Leader. State: Replica: 9cb4300ea183421ab40709a05de5a877, State: Running, Role: LEADER
W20260709 13:24:29.742271 27727 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:41335, user_credentials={real_user=slave}} blocked reactor thread for 51627.8us
I20260709 13:24:29.747654 28051 raft_consensus.cc:359] T 032f598cb30343c2a1ca3bab728e5b39 P 0788dbdabe574355bb323dcdc3be0058 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb4300ea183421ab40709a05de5a877" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 33597 } } peers { permanent_uuid: "0788dbdabe574355bb323dcdc3be0058" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 41335 } } peers { permanent_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34235 } }
I20260709 13:24:29.748826 28051 raft_consensus.cc:385] T 032f598cb30343c2a1ca3bab728e5b39 P 0788dbdabe574355bb323dcdc3be0058 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:24:29.749127 28051 raft_consensus.cc:740] T 032f598cb30343c2a1ca3bab728e5b39 P 0788dbdabe574355bb323dcdc3be0058 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0788dbdabe574355bb323dcdc3be0058, State: Initialized, Role: FOLLOWER
I20260709 13:24:29.749260 28082 raft_consensus.cc:515] T d747d5acdf4f41b69c132cde690e49ff P 880fc4c7dff4417187b75cc1d9e9dcf5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb4300ea183421ab40709a05de5a877" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 33597 } } peers { permanent_uuid: "0788dbdabe574355bb323dcdc3be0058" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 41335 } } peers { permanent_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34235 } }
I20260709 13:24:29.750043 28051 consensus_queue.cc:260] T 032f598cb30343c2a1ca3bab728e5b39 P 0788dbdabe574355bb323dcdc3be0058 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb4300ea183421ab40709a05de5a877" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 33597 } } peers { permanent_uuid: "0788dbdabe574355bb323dcdc3be0058" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 41335 } } peers { permanent_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34235 } }
I20260709 13:24:29.751372 27910 raft_consensus.cc:2393] T d747d5acdf4f41b69c132cde690e49ff P 880fc4c7dff4417187b75cc1d9e9dcf5 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9cb4300ea183421ab40709a05de5a877 in current term 1: Already voted for candidate 880fc4c7dff4417187b75cc1d9e9dcf5 in this term.
I20260709 13:24:29.754739 28051 ts_tablet_manager.cc:1434] T 032f598cb30343c2a1ca3bab728e5b39 P 0788dbdabe574355bb323dcdc3be0058: Time spent starting tablet: real 0.036s	user 0.009s	sys 0.000s
I20260709 13:24:29.755681 28082 leader_election.cc:290] T d747d5acdf4f41b69c132cde690e49ff P 880fc4c7dff4417187b75cc1d9e9dcf5 [CANDIDATE]: Term 1 election: Requested vote from peers 9cb4300ea183421ab40709a05de5a877 (127.14.177.129:33597), 0788dbdabe574355bb323dcdc3be0058 (127.14.177.130:41335)
I20260709 13:24:29.756045 28076 consensus_queue.cc:237] T 04c481ece2f94a1b8cb6337900da4862 P 9cb4300ea183421ab40709a05de5a877 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb4300ea183421ab40709a05de5a877" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 33597 } } peers { permanent_uuid: "0788dbdabe574355bb323dcdc3be0058" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 41335 } } peers { permanent_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34235 } }
I20260709 13:24:29.744660 27760 raft_consensus.cc:2468] T 8e6b55496c134353a770e2642fd7490d P 9cb4300ea183421ab40709a05de5a877 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 880fc4c7dff4417187b75cc1d9e9dcf5 in term 1.
I20260709 13:24:29.757436 27727 leader_election.cc:304] T d7fd85f92c0a481aa54401371638252b P 9cb4300ea183421ab40709a05de5a877 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9cb4300ea183421ab40709a05de5a877; no voters: 0788dbdabe574355bb323dcdc3be0058, 880fc4c7dff4417187b75cc1d9e9dcf5
I20260709 13:24:29.758841 27837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d747d5acdf4f41b69c132cde690e49ff" candidate_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0788dbdabe574355bb323dcdc3be0058"
I20260709 13:24:29.759867 27837 raft_consensus.cc:3060] T d747d5acdf4f41b69c132cde690e49ff P 0788dbdabe574355bb323dcdc3be0058 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:24:29.760228 28079 raft_consensus.cc:2749] T d7fd85f92c0a481aa54401371638252b P 9cb4300ea183421ab40709a05de5a877 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:24:29.757499 27759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d747d5acdf4f41b69c132cde690e49ff" candidate_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9cb4300ea183421ab40709a05de5a877"
I20260709 13:24:29.756997 28051 tablet_bootstrap.cc:492] T 96f830fa400c473881739b3a953f0178 P 0788dbdabe574355bb323dcdc3be0058: Bootstrap starting.
I20260709 13:24:29.767874 27759 raft_consensus.cc:3060] T d747d5acdf4f41b69c132cde690e49ff P 9cb4300ea183421ab40709a05de5a877 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:24:29.774210 27727 leader_election.cc:304] T d747d5acdf4f41b69c132cde690e49ff P 9cb4300ea183421ab40709a05de5a877 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0788dbdabe574355bb323dcdc3be0058, 9cb4300ea183421ab40709a05de5a877; no voters: 
I20260709 13:24:29.745915 28052 ts_tablet_manager.cc:1434] T 27629b85cc3e4cb6b837dd26e3ebff47 P 9cb4300ea183421ab40709a05de5a877: Time spent starting tablet: real 0.094s	user 0.014s	sys 0.000s
I20260709 13:24:29.780104 27762 raft_consensus.cc:2468] T 660aa4903c494ba1bce988191700e129 P 9cb4300ea183421ab40709a05de5a877 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 880fc4c7dff4417187b75cc1d9e9dcf5 in term 1.
I20260709 13:24:29.788568 27978 raft_consensus.cc:493] T 032f598cb30343c2a1ca3bab728e5b39 P 9cb4300ea183421ab40709a05de5a877 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:24:29.797248 27978 raft_consensus.cc:515] T 032f598cb30343c2a1ca3bab728e5b39 P 9cb4300ea183421ab40709a05de5a877 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb4300ea183421ab40709a05de5a877" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 33597 } } peers { permanent_uuid: "0788dbdabe574355bb323dcdc3be0058" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 41335 } } peers { permanent_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34235 } }
I20260709 13:24:29.792737 27658 catalog_manager.cc:5697] T 8e6b55496c134353a770e2642fd7490d P 880fc4c7dff4417187b75cc1d9e9dcf5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 880fc4c7dff4417187b75cc1d9e9dcf5 (127.14.177.131). New cstate: current_term: 1 leader_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb4300ea183421ab40709a05de5a877" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 33597 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0788dbdabe574355bb323dcdc3be0058" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 41335 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34235 } health_report { overall_health: HEALTHY } } }
I20260709 13:24:29.801138 27990 raft_consensus.cc:493] T 1af49a625b20491586dcea4961e46734 P 880fc4c7dff4417187b75cc1d9e9dcf5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:24:29.805197 27990 raft_consensus.cc:515] T 1af49a625b20491586dcea4961e46734 P 880fc4c7dff4417187b75cc1d9e9dcf5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb4300ea183421ab40709a05de5a877" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 33597 } } peers { permanent_uuid: "0788dbdabe574355bb323dcdc3be0058" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 41335 } } peers { permanent_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34235 } }
I20260709 13:24:29.815689 28050 tablet_bootstrap.cc:492] T 032f598cb30343c2a1ca3bab728e5b39 P 880fc4c7dff4417187b75cc1d9e9dcf5: No bootstrap required, opened a new log
I20260709 13:24:29.817804 28052 tablet_bootstrap.cc:492] T 96f830fa400c473881739b3a953f0178 P 9cb4300ea183421ab40709a05de5a877: Bootstrap starting.
I20260709 13:24:29.818650 27978 leader_election.cc:290] T 032f598cb30343c2a1ca3bab728e5b39 P 9cb4300ea183421ab40709a05de5a877 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0788dbdabe574355bb323dcdc3be0058 (127.14.177.130:41335), 880fc4c7dff4417187b75cc1d9e9dcf5 (127.14.177.131:34235)
I20260709 13:24:29.819341 28050 ts_tablet_manager.cc:1403] T 032f598cb30343c2a1ca3bab728e5b39 P 880fc4c7dff4417187b75cc1d9e9dcf5: Time spent bootstrapping tablet: real 0.203s	user 0.025s	sys 0.020s
I20260709 13:24:29.820326 27990 leader_election.cc:290] T 1af49a625b20491586dcea4961e46734 P 880fc4c7dff4417187b75cc1d9e9dcf5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9cb4300ea183421ab40709a05de5a877 (127.14.177.129:33597), 0788dbdabe574355bb323dcdc3be0058 (127.14.177.130:41335)
I20260709 13:24:29.822575 27834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1af49a625b20491586dcea4961e46734" candidate_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0788dbdabe574355bb323dcdc3be0058" is_pre_election: true
I20260709 13:24:29.827260 27834 raft_consensus.cc:2468] T 1af49a625b20491586dcea4961e46734 P 0788dbdabe574355bb323dcdc3be0058 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 880fc4c7dff4417187b75cc1d9e9dcf5 in term 0.
I20260709 13:24:29.829319 27836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "032f598cb30343c2a1ca3bab728e5b39" candidate_uuid: "9cb4300ea183421ab40709a05de5a877" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0788dbdabe574355bb323dcdc3be0058" is_pre_election: true
I20260709 13:24:29.830194 27836 raft_consensus.cc:2468] T 032f598cb30343c2a1ca3bab728e5b39 P 0788dbdabe574355bb323dcdc3be0058 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9cb4300ea183421ab40709a05de5a877 in term 0.
I20260709 13:24:29.827651 27658 catalog_manager.cc:5697] T 660aa4903c494ba1bce988191700e129 P 880fc4c7dff4417187b75cc1d9e9dcf5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 880fc4c7dff4417187b75cc1d9e9dcf5 (127.14.177.131). New cstate: current_term: 1 leader_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb4300ea183421ab40709a05de5a877" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 33597 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0788dbdabe574355bb323dcdc3be0058" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 41335 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34235 } health_report { overall_health: HEALTHY } } }
I20260709 13:24:29.835238 27876 leader_election.cc:304] T 1af49a625b20491586dcea4961e46734 P 880fc4c7dff4417187b75cc1d9e9dcf5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0788dbdabe574355bb323dcdc3be0058, 880fc4c7dff4417187b75cc1d9e9dcf5; no voters: 
I20260709 13:24:29.836356 27990 raft_consensus.cc:2804] T 1af49a625b20491586dcea4961e46734 P 880fc4c7dff4417187b75cc1d9e9dcf5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:24:29.839504 27990 raft_consensus.cc:493] T 1af49a625b20491586dcea4961e46734 P 880fc4c7dff4417187b75cc1d9e9dcf5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:24:29.840476 27990 raft_consensus.cc:3060] T 1af49a625b20491586dcea4961e46734 P 880fc4c7dff4417187b75cc1d9e9dcf5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:24:29.841065 28050 raft_consensus.cc:359] T 032f598cb30343c2a1ca3bab728e5b39 P 880fc4c7dff4417187b75cc1d9e9dcf5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb4300ea183421ab40709a05de5a877" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 33597 } } peers { permanent_uuid: "0788dbdabe574355bb323dcdc3be0058" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 41335 } } peers { permanent_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34235 } }
I20260709 13:24:29.842293 28050 raft_consensus.cc:385] T 032f598cb30343c2a1ca3bab728e5b39 P 880fc4c7dff4417187b75cc1d9e9dcf5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:24:29.842715 28050 raft_consensus.cc:740] T 032f598cb30343c2a1ca3bab728e5b39 P 880fc4c7dff4417187b75cc1d9e9dcf5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 880fc4c7dff4417187b75cc1d9e9dcf5, State: Initialized, Role: FOLLOWER
I20260709 13:24:29.843762 28050 consensus_queue.cc:260] T 032f598cb30343c2a1ca3bab728e5b39 P 880fc4c7dff4417187b75cc1d9e9dcf5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb4300ea183421ab40709a05de5a877" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 33597 } } peers { permanent_uuid: "0788dbdabe574355bb323dcdc3be0058" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 41335 } } peers { permanent_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34235 } }
I20260709 13:24:29.847021 28052 tablet_bootstrap.cc:654] T 96f830fa400c473881739b3a953f0178 P 9cb4300ea183421ab40709a05de5a877: Neither blocks nor log segments found. Creating new log.
I20260709 13:24:29.848069 28050 ts_tablet_manager.cc:1434] T 032f598cb30343c2a1ca3bab728e5b39 P 880fc4c7dff4417187b75cc1d9e9dcf5: Time spent starting tablet: real 0.024s	user 0.007s	sys 0.005s
I20260709 13:24:29.849099 27762 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1af49a625b20491586dcea4961e46734" candidate_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9cb4300ea183421ab40709a05de5a877" is_pre_election: true
I20260709 13:24:29.850354 28050 tablet_bootstrap.cc:492] T 96f830fa400c473881739b3a953f0178 P 880fc4c7dff4417187b75cc1d9e9dcf5: Bootstrap starting.
W20260709 13:24:29.851511 27874 leader_election.cc:343] T 1af49a625b20491586dcea4961e46734 P 880fc4c7dff4417187b75cc1d9e9dcf5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9cb4300ea183421ab40709a05de5a877 (127.14.177.129:33597): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:24:29.853564 27909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "032f598cb30343c2a1ca3bab728e5b39" candidate_uuid: "9cb4300ea183421ab40709a05de5a877" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" is_pre_election: true
I20260709 13:24:29.854952 28051 tablet_bootstrap.cc:654] T 96f830fa400c473881739b3a953f0178 P 0788dbdabe574355bb323dcdc3be0058: Neither blocks nor log segments found. Creating new log.
I20260709 13:24:29.854470 27909 raft_consensus.cc:2468] T 032f598cb30343c2a1ca3bab728e5b39 P 880fc4c7dff4417187b75cc1d9e9dcf5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9cb4300ea183421ab40709a05de5a877 in term 0.
I20260709 13:24:29.836578 27759 raft_consensus.cc:2468] T d747d5acdf4f41b69c132cde690e49ff P 9cb4300ea183421ab40709a05de5a877 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 880fc4c7dff4417187b75cc1d9e9dcf5 in term 1.
I20260709 13:24:29.859997 28079 raft_consensus.cc:2764] T d747d5acdf4f41b69c132cde690e49ff P 9cb4300ea183421ab40709a05de5a877 [term 1 FOLLOWER]: Leader pre-election decision vote started in defunct term 0: won
I20260709 13:24:29.860960 27874 leader_election.cc:304] T d747d5acdf4f41b69c132cde690e49ff P 880fc4c7dff4417187b75cc1d9e9dcf5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 880fc4c7dff4417187b75cc1d9e9dcf5, 9cb4300ea183421ab40709a05de5a877; no voters: 
I20260709 13:24:29.863945 28082 raft_consensus.cc:2804] T d747d5acdf4f41b69c132cde690e49ff P 880fc4c7dff4417187b75cc1d9e9dcf5 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:24:29.864681 28082 raft_consensus.cc:697] T d747d5acdf4f41b69c132cde690e49ff P 880fc4c7dff4417187b75cc1d9e9dcf5 [term 1 LEADER]: Becoming Leader. State: Replica: 880fc4c7dff4417187b75cc1d9e9dcf5, State: Running, Role: LEADER
I20260709 13:24:29.865980 28082 consensus_queue.cc:237] T d747d5acdf4f41b69c132cde690e49ff P 880fc4c7dff4417187b75cc1d9e9dcf5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb4300ea183421ab40709a05de5a877" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 33597 } } peers { permanent_uuid: "0788dbdabe574355bb323dcdc3be0058" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 41335 } } peers { permanent_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34235 } }
I20260709 13:24:29.862046 27837 raft_consensus.cc:2468] T d747d5acdf4f41b69c132cde690e49ff P 0788dbdabe574355bb323dcdc3be0058 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 880fc4c7dff4417187b75cc1d9e9dcf5 in term 1.
I20260709 13:24:29.866847 27657 catalog_manager.cc:5697] T 04c481ece2f94a1b8cb6337900da4862 P 9cb4300ea183421ab40709a05de5a877 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9cb4300ea183421ab40709a05de5a877 (127.14.177.129). New cstate: current_term: 1 leader_uuid: "9cb4300ea183421ab40709a05de5a877" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb4300ea183421ab40709a05de5a877" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 33597 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0788dbdabe574355bb323dcdc3be0058" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 41335 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34235 } health_report { overall_health: UNKNOWN } } }
I20260709 13:24:29.859817 27727 leader_election.cc:304] T 032f598cb30343c2a1ca3bab728e5b39 P 9cb4300ea183421ab40709a05de5a877 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0788dbdabe574355bb323dcdc3be0058, 9cb4300ea183421ab40709a05de5a877; no voters: 
I20260709 13:24:29.896736 28052 tablet_bootstrap.cc:492] T 96f830fa400c473881739b3a953f0178 P 9cb4300ea183421ab40709a05de5a877: No bootstrap required, opened a new log
I20260709 13:24:29.880028 28079 raft_consensus.cc:2804] T 032f598cb30343c2a1ca3bab728e5b39 P 9cb4300ea183421ab40709a05de5a877 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:24:29.904691 28052 ts_tablet_manager.cc:1403] T 96f830fa400c473881739b3a953f0178 P 9cb4300ea183421ab40709a05de5a877: Time spent bootstrapping tablet: real 0.087s	user 0.013s	sys 0.007s
I20260709 13:24:29.905030 28079 raft_consensus.cc:493] T 032f598cb30343c2a1ca3bab728e5b39 P 9cb4300ea183421ab40709a05de5a877 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:24:29.905539 28079 raft_consensus.cc:3060] T 032f598cb30343c2a1ca3bab728e5b39 P 9cb4300ea183421ab40709a05de5a877 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:24:29.886960 27660 catalog_manager.cc:5697] T d7fd85f92c0a481aa54401371638252b P 0788dbdabe574355bb323dcdc3be0058 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0788dbdabe574355bb323dcdc3be0058 (127.14.177.130). New cstate: current_term: 1 leader_uuid: "0788dbdabe574355bb323dcdc3be0058" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb4300ea183421ab40709a05de5a877" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 33597 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0788dbdabe574355bb323dcdc3be0058" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 41335 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34235 } health_report { overall_health: UNKNOWN } } }
I20260709 13:24:29.911895 28051 tablet_bootstrap.cc:492] T 96f830fa400c473881739b3a953f0178 P 0788dbdabe574355bb323dcdc3be0058: No bootstrap required, opened a new log
I20260709 13:24:29.878822 27990 raft_consensus.cc:515] T 1af49a625b20491586dcea4961e46734 P 880fc4c7dff4417187b75cc1d9e9dcf5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb4300ea183421ab40709a05de5a877" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 33597 } } peers { permanent_uuid: "0788dbdabe574355bb323dcdc3be0058" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 41335 } } peers { permanent_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34235 } }
I20260709 13:24:29.917224 28051 ts_tablet_manager.cc:1403] T 96f830fa400c473881739b3a953f0178 P 0788dbdabe574355bb323dcdc3be0058: Time spent bootstrapping tablet: real 0.161s	user 0.013s	sys 0.011s
I20260709 13:24:29.879047 28050 tablet_bootstrap.cc:654] T 96f830fa400c473881739b3a953f0178 P 880fc4c7dff4417187b75cc1d9e9dcf5: Neither blocks nor log segments found. Creating new log.
I20260709 13:24:29.920562 27990 leader_election.cc:290] T 1af49a625b20491586dcea4961e46734 P 880fc4c7dff4417187b75cc1d9e9dcf5 [CANDIDATE]: Term 1 election: Requested vote from peers 9cb4300ea183421ab40709a05de5a877 (127.14.177.129:33597), 0788dbdabe574355bb323dcdc3be0058 (127.14.177.130:41335)
I20260709 13:24:29.915125 28052 raft_consensus.cc:359] T 96f830fa400c473881739b3a953f0178 P 9cb4300ea183421ab40709a05de5a877 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb4300ea183421ab40709a05de5a877" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 33597 } } peers { permanent_uuid: "0788dbdabe574355bb323dcdc3be0058" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 41335 } } peers { permanent_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34235 } }
I20260709 13:24:29.920600 27759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1af49a625b20491586dcea4961e46734" candidate_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9cb4300ea183421ab40709a05de5a877"
I20260709 13:24:29.921777 28052 raft_consensus.cc:385] T 96f830fa400c473881739b3a953f0178 P 9cb4300ea183421ab40709a05de5a877 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:24:29.922142 28052 raft_consensus.cc:740] T 96f830fa400c473881739b3a953f0178 P 9cb4300ea183421ab40709a05de5a877 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9cb4300ea183421ab40709a05de5a877, State: Initialized, Role: FOLLOWER
I20260709 13:24:29.922461 27837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1af49a625b20491586dcea4961e46734" candidate_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0788dbdabe574355bb323dcdc3be0058"
I20260709 13:24:29.923205 27837 raft_consensus.cc:3060] T 1af49a625b20491586dcea4961e46734 P 0788dbdabe574355bb323dcdc3be0058 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:24:29.923017 28052 consensus_queue.cc:260] T 96f830fa400c473881739b3a953f0178 P 9cb4300ea183421ab40709a05de5a877 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb4300ea183421ab40709a05de5a877" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 33597 } } peers { permanent_uuid: "0788dbdabe574355bb323dcdc3be0058" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 41335 } } peers { permanent_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34235 } }
I20260709 13:24:29.924995 27657 catalog_manager.cc:5697] T d747d5acdf4f41b69c132cde690e49ff P 880fc4c7dff4417187b75cc1d9e9dcf5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 880fc4c7dff4417187b75cc1d9e9dcf5 (127.14.177.131). New cstate: current_term: 1 leader_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb4300ea183421ab40709a05de5a877" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 33597 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0788dbdabe574355bb323dcdc3be0058" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 41335 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34235 } health_report { overall_health: HEALTHY } } }
I20260709 13:24:29.927249 28052 ts_tablet_manager.cc:1434] T 96f830fa400c473881739b3a953f0178 P 9cb4300ea183421ab40709a05de5a877: Time spent starting tablet: real 0.022s	user 0.010s	sys 0.000s
W20260709 13:24:29.928392 27874 leader_election.cc:343] T 1af49a625b20491586dcea4961e46734 P 880fc4c7dff4417187b75cc1d9e9dcf5 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 9cb4300ea183421ab40709a05de5a877 (127.14.177.129:33597): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:24:29.928867 28052 tablet_bootstrap.cc:492] T 1af49a625b20491586dcea4961e46734 P 9cb4300ea183421ab40709a05de5a877: Bootstrap starting.
I20260709 13:24:29.937062 28052 tablet_bootstrap.cc:654] T 1af49a625b20491586dcea4961e46734 P 9cb4300ea183421ab40709a05de5a877: Neither blocks nor log segments found. Creating new log.
I20260709 13:24:29.941524 27837 raft_consensus.cc:2468] T 1af49a625b20491586dcea4961e46734 P 0788dbdabe574355bb323dcdc3be0058 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 880fc4c7dff4417187b75cc1d9e9dcf5 in term 1.
I20260709 13:24:29.943092 27876 leader_election.cc:304] T 1af49a625b20491586dcea4961e46734 P 880fc4c7dff4417187b75cc1d9e9dcf5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0788dbdabe574355bb323dcdc3be0058, 880fc4c7dff4417187b75cc1d9e9dcf5; no voters: 9cb4300ea183421ab40709a05de5a877
I20260709 13:24:29.944468 27990 raft_consensus.cc:2804] T 1af49a625b20491586dcea4961e46734 P 880fc4c7dff4417187b75cc1d9e9dcf5 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:24:29.944376 28051 raft_consensus.cc:359] T 96f830fa400c473881739b3a953f0178 P 0788dbdabe574355bb323dcdc3be0058 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb4300ea183421ab40709a05de5a877" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 33597 } } peers { permanent_uuid: "0788dbdabe574355bb323dcdc3be0058" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 41335 } } peers { permanent_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34235 } }
I20260709 13:24:29.945328 28051 raft_consensus.cc:385] T 96f830fa400c473881739b3a953f0178 P 0788dbdabe574355bb323dcdc3be0058 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:24:29.945623 28051 raft_consensus.cc:740] T 96f830fa400c473881739b3a953f0178 P 0788dbdabe574355bb323dcdc3be0058 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0788dbdabe574355bb323dcdc3be0058, State: Initialized, Role: FOLLOWER
I20260709 13:24:29.946197 27990 raft_consensus.cc:697] T 1af49a625b20491586dcea4961e46734 P 880fc4c7dff4417187b75cc1d9e9dcf5 [term 1 LEADER]: Becoming Leader. State: Replica: 880fc4c7dff4417187b75cc1d9e9dcf5, State: Running, Role: LEADER
I20260709 13:24:29.946285 28051 consensus_queue.cc:260] T 96f830fa400c473881739b3a953f0178 P 0788dbdabe574355bb323dcdc3be0058 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb4300ea183421ab40709a05de5a877" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 33597 } } peers { permanent_uuid: "0788dbdabe574355bb323dcdc3be0058" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 41335 } } peers { permanent_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34235 } }
I20260709 13:24:29.949028 28051 ts_tablet_manager.cc:1434] T 96f830fa400c473881739b3a953f0178 P 0788dbdabe574355bb323dcdc3be0058: Time spent starting tablet: real 0.031s	user 0.003s	sys 0.003s
I20260709 13:24:29.949728 28052 tablet_bootstrap.cc:492] T 1af49a625b20491586dcea4961e46734 P 9cb4300ea183421ab40709a05de5a877: No bootstrap required, opened a new log
I20260709 13:24:29.950366 28052 ts_tablet_manager.cc:1403] T 1af49a625b20491586dcea4961e46734 P 9cb4300ea183421ab40709a05de5a877: Time spent bootstrapping tablet: real 0.022s	user 0.018s	sys 0.000s
I20260709 13:24:29.952803 27759 raft_consensus.cc:3060] T 27629b85cc3e4cb6b837dd26e3ebff47 P 9cb4300ea183421ab40709a05de5a877 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:24:29.953115 28050 tablet_bootstrap.cc:492] T 96f830fa400c473881739b3a953f0178 P 880fc4c7dff4417187b75cc1d9e9dcf5: No bootstrap required, opened a new log
I20260709 13:24:29.953718 28052 raft_consensus.cc:359] T 1af49a625b20491586dcea4961e46734 P 9cb4300ea183421ab40709a05de5a877 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb4300ea183421ab40709a05de5a877" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 33597 } } peers { permanent_uuid: "0788dbdabe574355bb323dcdc3be0058" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 41335 } } peers { permanent_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34235 } }
I20260709 13:24:29.954550 28052 raft_consensus.cc:385] T 1af49a625b20491586dcea4961e46734 P 9cb4300ea183421ab40709a05de5a877 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:24:29.954830 28052 raft_consensus.cc:740] T 1af49a625b20491586dcea4961e46734 P 9cb4300ea183421ab40709a05de5a877 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9cb4300ea183421ab40709a05de5a877, State: Initialized, Role: FOLLOWER
I20260709 13:24:29.955667 28052 consensus_queue.cc:260] T 1af49a625b20491586dcea4961e46734 P 9cb4300ea183421ab40709a05de5a877 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb4300ea183421ab40709a05de5a877" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 33597 } } peers { permanent_uuid: "0788dbdabe574355bb323dcdc3be0058" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 41335 } } peers { permanent_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34235 } }
I20260709 13:24:29.955860 28079 raft_consensus.cc:515] T 032f598cb30343c2a1ca3bab728e5b39 P 9cb4300ea183421ab40709a05de5a877 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb4300ea183421ab40709a05de5a877" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 33597 } } peers { permanent_uuid: "0788dbdabe574355bb323dcdc3be0058" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 41335 } } peers { permanent_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34235 } }
I20260709 13:24:29.957116 27990 consensus_queue.cc:237] T 1af49a625b20491586dcea4961e46734 P 880fc4c7dff4417187b75cc1d9e9dcf5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb4300ea183421ab40709a05de5a877" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 33597 } } peers { permanent_uuid: "0788dbdabe574355bb323dcdc3be0058" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 41335 } } peers { permanent_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34235 } }
I20260709 13:24:29.953719 28050 ts_tablet_manager.cc:1403] T 96f830fa400c473881739b3a953f0178 P 880fc4c7dff4417187b75cc1d9e9dcf5: Time spent bootstrapping tablet: real 0.104s	user 0.019s	sys 0.014s
I20260709 13:24:29.965903 28050 raft_consensus.cc:359] T 96f830fa400c473881739b3a953f0178 P 880fc4c7dff4417187b75cc1d9e9dcf5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb4300ea183421ab40709a05de5a877" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 33597 } } peers { permanent_uuid: "0788dbdabe574355bb323dcdc3be0058" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 41335 } } peers { permanent_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34235 } }
I20260709 13:24:29.967114 28050 raft_consensus.cc:385] T 96f830fa400c473881739b3a953f0178 P 880fc4c7dff4417187b75cc1d9e9dcf5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:24:29.967586 28050 raft_consensus.cc:740] T 96f830fa400c473881739b3a953f0178 P 880fc4c7dff4417187b75cc1d9e9dcf5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 880fc4c7dff4417187b75cc1d9e9dcf5, State: Initialized, Role: FOLLOWER
I20260709 13:24:29.968544 28050 consensus_queue.cc:260] T 96f830fa400c473881739b3a953f0178 P 880fc4c7dff4417187b75cc1d9e9dcf5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb4300ea183421ab40709a05de5a877" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 33597 } } peers { permanent_uuid: "0788dbdabe574355bb323dcdc3be0058" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 41335 } } peers { permanent_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34235 } }
I20260709 13:24:29.971663 28050 ts_tablet_manager.cc:1434] T 96f830fa400c473881739b3a953f0178 P 880fc4c7dff4417187b75cc1d9e9dcf5: Time spent starting tablet: real 0.009s	user 0.005s	sys 0.003s
I20260709 13:24:29.972709 27837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "032f598cb30343c2a1ca3bab728e5b39" candidate_uuid: "9cb4300ea183421ab40709a05de5a877" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0788dbdabe574355bb323dcdc3be0058"
I20260709 13:24:29.973006 28079 leader_election.cc:290] T 032f598cb30343c2a1ca3bab728e5b39 P 9cb4300ea183421ab40709a05de5a877 [CANDIDATE]: Term 1 election: Requested vote from peers 0788dbdabe574355bb323dcdc3be0058 (127.14.177.130:41335), 880fc4c7dff4417187b75cc1d9e9dcf5 (127.14.177.131:34235)
I20260709 13:24:29.973459 27837 raft_consensus.cc:3060] T 032f598cb30343c2a1ca3bab728e5b39 P 0788dbdabe574355bb323dcdc3be0058 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:24:29.974423 27909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "032f598cb30343c2a1ca3bab728e5b39" candidate_uuid: "9cb4300ea183421ab40709a05de5a877" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5"
I20260709 13:24:29.974973 28052 ts_tablet_manager.cc:1434] T 1af49a625b20491586dcea4961e46734 P 9cb4300ea183421ab40709a05de5a877: Time spent starting tablet: real 0.024s	user 0.008s	sys 0.000s
I20260709 13:24:29.975196 27909 raft_consensus.cc:3060] T 032f598cb30343c2a1ca3bab728e5b39 P 880fc4c7dff4417187b75cc1d9e9dcf5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:24:29.982764 27837 raft_consensus.cc:2468] T 032f598cb30343c2a1ca3bab728e5b39 P 0788dbdabe574355bb323dcdc3be0058 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9cb4300ea183421ab40709a05de5a877 in term 1.
I20260709 13:24:29.983464 28095 raft_consensus.cc:493] T 27629b85cc3e4cb6b837dd26e3ebff47 P 9cb4300ea183421ab40709a05de5a877 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:24:29.984145 28095 raft_consensus.cc:515] T 27629b85cc3e4cb6b837dd26e3ebff47 P 9cb4300ea183421ab40709a05de5a877 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb4300ea183421ab40709a05de5a877" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 33597 } } peers { permanent_uuid: "0788dbdabe574355bb323dcdc3be0058" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 41335 } } peers { permanent_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34235 } }
I20260709 13:24:29.986681 28078 consensus_queue.cc:1048] T 27629b85cc3e4cb6b837dd26e3ebff47 P 0788dbdabe574355bb323dcdc3be0058 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9cb4300ea183421ab40709a05de5a877" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 33597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:24:29.987603 27727 leader_election.cc:304] T 032f598cb30343c2a1ca3bab728e5b39 P 9cb4300ea183421ab40709a05de5a877 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0788dbdabe574355bb323dcdc3be0058, 9cb4300ea183421ab40709a05de5a877; no voters: 
I20260709 13:24:29.988966 28079 raft_consensus.cc:2804] T 032f598cb30343c2a1ca3bab728e5b39 P 9cb4300ea183421ab40709a05de5a877 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:24:29.989957 28079 raft_consensus.cc:697] T 032f598cb30343c2a1ca3bab728e5b39 P 9cb4300ea183421ab40709a05de5a877 [term 1 LEADER]: Becoming Leader. State: Replica: 9cb4300ea183421ab40709a05de5a877, State: Running, Role: LEADER
I20260709 13:24:29.991958 28079 consensus_queue.cc:237] T 032f598cb30343c2a1ca3bab728e5b39 P 9cb4300ea183421ab40709a05de5a877 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb4300ea183421ab40709a05de5a877" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 33597 } } peers { permanent_uuid: "0788dbdabe574355bb323dcdc3be0058" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 41335 } } peers { permanent_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34235 } }
I20260709 13:24:29.994966 27910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27629b85cc3e4cb6b837dd26e3ebff47" candidate_uuid: "9cb4300ea183421ab40709a05de5a877" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" is_pre_election: true
I20260709 13:24:29.996068 27910 raft_consensus.cc:2468] T 27629b85cc3e4cb6b837dd26e3ebff47 P 880fc4c7dff4417187b75cc1d9e9dcf5 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9cb4300ea183421ab40709a05de5a877 in term 1.
I20260709 13:24:29.989957 28095 leader_election.cc:290] T 27629b85cc3e4cb6b837dd26e3ebff47 P 9cb4300ea183421ab40709a05de5a877 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0788dbdabe574355bb323dcdc3be0058 (127.14.177.130:41335), 880fc4c7dff4417187b75cc1d9e9dcf5 (127.14.177.131:34235)
I20260709 13:24:29.992983 27837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27629b85cc3e4cb6b837dd26e3ebff47" candidate_uuid: "9cb4300ea183421ab40709a05de5a877" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0788dbdabe574355bb323dcdc3be0058" is_pre_election: true
I20260709 13:24:29.997184 27657 catalog_manager.cc:5697] T 1af49a625b20491586dcea4961e46734 P 880fc4c7dff4417187b75cc1d9e9dcf5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 880fc4c7dff4417187b75cc1d9e9dcf5 (127.14.177.131). New cstate: current_term: 1 leader_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb4300ea183421ab40709a05de5a877" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 33597 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0788dbdabe574355bb323dcdc3be0058" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 41335 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34235 } health_report { overall_health: HEALTHY } } }
I20260709 13:24:29.997879 27727 leader_election.cc:304] T 27629b85cc3e4cb6b837dd26e3ebff47 P 9cb4300ea183421ab40709a05de5a877 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 880fc4c7dff4417187b75cc1d9e9dcf5, 9cb4300ea183421ab40709a05de5a877; no voters: 
I20260709 13:24:29.990554 27909 raft_consensus.cc:2468] T 032f598cb30343c2a1ca3bab728e5b39 P 880fc4c7dff4417187b75cc1d9e9dcf5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9cb4300ea183421ab40709a05de5a877 in term 1.
I20260709 13:24:30.002657 28095 raft_consensus.cc:2804] T 27629b85cc3e4cb6b837dd26e3ebff47 P 9cb4300ea183421ab40709a05de5a877 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:24:30.007012 28095 raft_consensus.cc:493] T 27629b85cc3e4cb6b837dd26e3ebff47 P 9cb4300ea183421ab40709a05de5a877 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 0788dbdabe574355bb323dcdc3be0058)
I20260709 13:24:30.007637 28095 raft_consensus.cc:3060] T 27629b85cc3e4cb6b837dd26e3ebff47 P 9cb4300ea183421ab40709a05de5a877 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:24:30.009598 27955 consensus_queue.cc:1048] T 27629b85cc3e4cb6b837dd26e3ebff47 P 0788dbdabe574355bb323dcdc3be0058 [LEADER]: Connected to new peer: Peer: permanent_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34235 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:24:30.016355 27660 catalog_manager.cc:5697] T 032f598cb30343c2a1ca3bab728e5b39 P 9cb4300ea183421ab40709a05de5a877 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9cb4300ea183421ab40709a05de5a877 (127.14.177.129). New cstate: current_term: 1 leader_uuid: "9cb4300ea183421ab40709a05de5a877" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb4300ea183421ab40709a05de5a877" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 33597 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0788dbdabe574355bb323dcdc3be0058" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 41335 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34235 } health_report { overall_health: UNKNOWN } } }
I20260709 13:24:30.018189 28095 raft_consensus.cc:515] T 27629b85cc3e4cb6b837dd26e3ebff47 P 9cb4300ea183421ab40709a05de5a877 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb4300ea183421ab40709a05de5a877" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 33597 } } peers { permanent_uuid: "0788dbdabe574355bb323dcdc3be0058" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 41335 } } peers { permanent_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34235 } }
I20260709 13:24:30.024241 27762 raft_consensus.cc:1240] T 27629b85cc3e4cb6b837dd26e3ebff47 P 9cb4300ea183421ab40709a05de5a877 [term 2 FOLLOWER]: Rejecting Update request from peer 0788dbdabe574355bb323dcdc3be0058 for earlier term 1. Current term is 2. Ops: []
I20260709 13:24:30.027621 28095 leader_election.cc:290] T 27629b85cc3e4cb6b837dd26e3ebff47 P 9cb4300ea183421ab40709a05de5a877 [CANDIDATE]: Term 2 election: Requested vote from peers 0788dbdabe574355bb323dcdc3be0058 (127.14.177.130:41335), 880fc4c7dff4417187b75cc1d9e9dcf5 (127.14.177.131:34235)
I20260709 13:24:30.028887 27837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27629b85cc3e4cb6b837dd26e3ebff47" candidate_uuid: "9cb4300ea183421ab40709a05de5a877" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0788dbdabe574355bb323dcdc3be0058"
I20260709 13:24:30.033843 27955 consensus_queue.cc:1059] T 27629b85cc3e4cb6b837dd26e3ebff47 P 0788dbdabe574355bb323dcdc3be0058 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9cb4300ea183421ab40709a05de5a877" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 33597 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:24:30.034750 27910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27629b85cc3e4cb6b837dd26e3ebff47" candidate_uuid: "9cb4300ea183421ab40709a05de5a877" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5"
I20260709 13:24:30.035461 28071 consensus_queue.cc:1048] T d7fd85f92c0a481aa54401371638252b P 0788dbdabe574355bb323dcdc3be0058 [LEADER]: Connected to new peer: Peer: permanent_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34235 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:24:30.034786 27955 raft_consensus.cc:3055] T 27629b85cc3e4cb6b837dd26e3ebff47 P 0788dbdabe574355bb323dcdc3be0058 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:24:30.036749 27955 raft_consensus.cc:740] T 27629b85cc3e4cb6b837dd26e3ebff47 P 0788dbdabe574355bb323dcdc3be0058 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 0788dbdabe574355bb323dcdc3be0058, State: Running, Role: LEADER
I20260709 13:24:30.036901 27727 leader_election.cc:304] T 27629b85cc3e4cb6b837dd26e3ebff47 P 9cb4300ea183421ab40709a05de5a877 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9cb4300ea183421ab40709a05de5a877; no voters: 0788dbdabe574355bb323dcdc3be0058, 880fc4c7dff4417187b75cc1d9e9dcf5
I20260709 13:24:30.037802 27955 consensus_queue.cc:260] T 27629b85cc3e4cb6b837dd26e3ebff47 P 0788dbdabe574355bb323dcdc3be0058 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb4300ea183421ab40709a05de5a877" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 33597 } } peers { permanent_uuid: "0788dbdabe574355bb323dcdc3be0058" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 41335 } } peers { permanent_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34235 } }
I20260709 13:24:30.038781 28095 raft_consensus.cc:2749] T 27629b85cc3e4cb6b837dd26e3ebff47 P 9cb4300ea183421ab40709a05de5a877 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:24:30.039361 27955 raft_consensus.cc:3060] T 27629b85cc3e4cb6b837dd26e3ebff47 P 0788dbdabe574355bb323dcdc3be0058 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:24:30.057050 27955 consensus_queue.cc:1048] T d7fd85f92c0a481aa54401371638252b P 0788dbdabe574355bb323dcdc3be0058 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9cb4300ea183421ab40709a05de5a877" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 33597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:24:30.156222 27990 consensus_queue.cc:1048] T 660aa4903c494ba1bce988191700e129 P 880fc4c7dff4417187b75cc1d9e9dcf5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9cb4300ea183421ab40709a05de5a877" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 33597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:24:30.157521 28082 consensus_queue.cc:1048] T 8e6b55496c134353a770e2642fd7490d P 880fc4c7dff4417187b75cc1d9e9dcf5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0788dbdabe574355bb323dcdc3be0058" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 41335 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:24:30.175887 28095 raft_consensus.cc:493] T 1af49a625b20491586dcea4961e46734 P 9cb4300ea183421ab40709a05de5a877 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:24:30.176611 28095 raft_consensus.cc:515] T 1af49a625b20491586dcea4961e46734 P 9cb4300ea183421ab40709a05de5a877 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb4300ea183421ab40709a05de5a877" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 33597 } } peers { permanent_uuid: "0788dbdabe574355bb323dcdc3be0058" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 41335 } } peers { permanent_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34235 } }
I20260709 13:24:30.181210 28115 consensus_queue.cc:1048] T 660aa4903c494ba1bce988191700e129 P 880fc4c7dff4417187b75cc1d9e9dcf5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0788dbdabe574355bb323dcdc3be0058" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 41335 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:24:30.181903 27834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1af49a625b20491586dcea4961e46734" candidate_uuid: "9cb4300ea183421ab40709a05de5a877" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0788dbdabe574355bb323dcdc3be0058" is_pre_election: true
I20260709 13:24:30.182791 27834 raft_consensus.cc:2393] T 1af49a625b20491586dcea4961e46734 P 0788dbdabe574355bb323dcdc3be0058 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9cb4300ea183421ab40709a05de5a877 in current term 1: Already voted for candidate 880fc4c7dff4417187b75cc1d9e9dcf5 in this term.
I20260709 13:24:30.184484 27990 consensus_queue.cc:1048] T 8e6b55496c134353a770e2642fd7490d P 880fc4c7dff4417187b75cc1d9e9dcf5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9cb4300ea183421ab40709a05de5a877" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 33597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 13:24:30.184981 27708 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:24:30.189486 28079 raft_consensus.cc:493] T 96f830fa400c473881739b3a953f0178 P 9cb4300ea183421ab40709a05de5a877 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:24:30.190063 28079 raft_consensus.cc:515] T 96f830fa400c473881739b3a953f0178 P 9cb4300ea183421ab40709a05de5a877 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb4300ea183421ab40709a05de5a877" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 33597 } } peers { permanent_uuid: "0788dbdabe574355bb323dcdc3be0058" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 41335 } } peers { permanent_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34235 } }
I20260709 13:24:30.190891 28095 leader_election.cc:290] T 1af49a625b20491586dcea4961e46734 P 9cb4300ea183421ab40709a05de5a877 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0788dbdabe574355bb323dcdc3be0058 (127.14.177.130:41335), 880fc4c7dff4417187b75cc1d9e9dcf5 (127.14.177.131:34235)
I20260709 13:24:30.192328 27910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1af49a625b20491586dcea4961e46734" candidate_uuid: "9cb4300ea183421ab40709a05de5a877" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" is_pre_election: true
I20260709 13:24:30.194661 27727 leader_election.cc:304] T 1af49a625b20491586dcea4961e46734 P 9cb4300ea183421ab40709a05de5a877 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9cb4300ea183421ab40709a05de5a877; no voters: 0788dbdabe574355bb323dcdc3be0058, 880fc4c7dff4417187b75cc1d9e9dcf5
I20260709 13:24:30.195845 28079 leader_election.cc:290] T 96f830fa400c473881739b3a953f0178 P 9cb4300ea183421ab40709a05de5a877 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0788dbdabe574355bb323dcdc3be0058 (127.14.177.130:41335), 880fc4c7dff4417187b75cc1d9e9dcf5 (127.14.177.131:34235)
I20260709 13:24:30.198051 27910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96f830fa400c473881739b3a953f0178" candidate_uuid: "9cb4300ea183421ab40709a05de5a877" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" is_pre_election: true
I20260709 13:24:30.198773 27910 raft_consensus.cc:2468] T 96f830fa400c473881739b3a953f0178 P 880fc4c7dff4417187b75cc1d9e9dcf5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9cb4300ea183421ab40709a05de5a877 in term 0.
I20260709 13:24:30.203634 27727 leader_election.cc:304] T 96f830fa400c473881739b3a953f0178 P 9cb4300ea183421ab40709a05de5a877 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 880fc4c7dff4417187b75cc1d9e9dcf5, 9cb4300ea183421ab40709a05de5a877; no voters: 
I20260709 13:24:30.207406 28096 raft_consensus.cc:2804] T 96f830fa400c473881739b3a953f0178 P 9cb4300ea183421ab40709a05de5a877 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:24:30.207829 28096 raft_consensus.cc:493] T 96f830fa400c473881739b3a953f0178 P 9cb4300ea183421ab40709a05de5a877 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:24:30.208202 28096 raft_consensus.cc:3060] T 96f830fa400c473881739b3a953f0178 P 9cb4300ea183421ab40709a05de5a877 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:24:30.215476 27837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96f830fa400c473881739b3a953f0178" candidate_uuid: "9cb4300ea183421ab40709a05de5a877" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0788dbdabe574355bb323dcdc3be0058" is_pre_election: true
I20260709 13:24:30.195925 28095 raft_consensus.cc:3060] T 1af49a625b20491586dcea4961e46734 P 9cb4300ea183421ab40709a05de5a877 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:24:30.216348 27837 raft_consensus.cc:2468] T 96f830fa400c473881739b3a953f0178 P 0788dbdabe574355bb323dcdc3be0058 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9cb4300ea183421ab40709a05de5a877 in term 0.
I20260709 13:24:30.221565 28076 consensus_queue.cc:1048] T 04c481ece2f94a1b8cb6337900da4862 P 9cb4300ea183421ab40709a05de5a877 [LEADER]: Connected to new peer: Peer: permanent_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34235 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:24:30.223407 28079 consensus_queue.cc:1048] T 04c481ece2f94a1b8cb6337900da4862 P 9cb4300ea183421ab40709a05de5a877 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0788dbdabe574355bb323dcdc3be0058" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 41335 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:24:30.228293 28095 raft_consensus.cc:2749] T 1af49a625b20491586dcea4961e46734 P 9cb4300ea183421ab40709a05de5a877 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:24:30.220105 28096 raft_consensus.cc:515] T 96f830fa400c473881739b3a953f0178 P 9cb4300ea183421ab40709a05de5a877 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb4300ea183421ab40709a05de5a877" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 33597 } } peers { permanent_uuid: "0788dbdabe574355bb323dcdc3be0058" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 41335 } } peers { permanent_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34235 } }
I20260709 13:24:30.238453 27836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96f830fa400c473881739b3a953f0178" candidate_uuid: "9cb4300ea183421ab40709a05de5a877" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0788dbdabe574355bb323dcdc3be0058"
I20260709 13:24:30.239147 27836 raft_consensus.cc:3060] T 96f830fa400c473881739b3a953f0178 P 0788dbdabe574355bb323dcdc3be0058 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:24:30.250284 27836 raft_consensus.cc:2468] T 96f830fa400c473881739b3a953f0178 P 0788dbdabe574355bb323dcdc3be0058 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9cb4300ea183421ab40709a05de5a877 in term 1.
I20260709 13:24:30.251600 27727 leader_election.cc:304] T 96f830fa400c473881739b3a953f0178 P 9cb4300ea183421ab40709a05de5a877 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0788dbdabe574355bb323dcdc3be0058, 9cb4300ea183421ab40709a05de5a877; no voters: 
I20260709 13:24:30.252621 28079 raft_consensus.cc:2804] T 96f830fa400c473881739b3a953f0178 P 9cb4300ea183421ab40709a05de5a877 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:24:30.253139 28079 raft_consensus.cc:697] T 96f830fa400c473881739b3a953f0178 P 9cb4300ea183421ab40709a05de5a877 [term 1 LEADER]: Becoming Leader. State: Replica: 9cb4300ea183421ab40709a05de5a877, State: Running, Role: LEADER
I20260709 13:24:30.253175 28096 leader_election.cc:290] T 96f830fa400c473881739b3a953f0178 P 9cb4300ea183421ab40709a05de5a877 [CANDIDATE]: Term 1 election: Requested vote from peers 0788dbdabe574355bb323dcdc3be0058 (127.14.177.130:41335), 880fc4c7dff4417187b75cc1d9e9dcf5 (127.14.177.131:34235)
I20260709 13:24:30.254052 28079 consensus_queue.cc:237] T 96f830fa400c473881739b3a953f0178 P 9cb4300ea183421ab40709a05de5a877 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb4300ea183421ab40709a05de5a877" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 33597 } } peers { permanent_uuid: "0788dbdabe574355bb323dcdc3be0058" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 41335 } } peers { permanent_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34235 } }
I20260709 13:24:30.254987 27910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96f830fa400c473881739b3a953f0178" candidate_uuid: "9cb4300ea183421ab40709a05de5a877" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5"
I20260709 13:24:30.255692 27910 raft_consensus.cc:3060] T 96f830fa400c473881739b3a953f0178 P 880fc4c7dff4417187b75cc1d9e9dcf5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:24:30.264881 27910 raft_consensus.cc:2468] T 96f830fa400c473881739b3a953f0178 P 880fc4c7dff4417187b75cc1d9e9dcf5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9cb4300ea183421ab40709a05de5a877 in term 1.
I20260709 13:24:30.270188 27657 catalog_manager.cc:5697] T 96f830fa400c473881739b3a953f0178 P 9cb4300ea183421ab40709a05de5a877 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9cb4300ea183421ab40709a05de5a877 (127.14.177.129). New cstate: current_term: 1 leader_uuid: "9cb4300ea183421ab40709a05de5a877" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb4300ea183421ab40709a05de5a877" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 33597 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0788dbdabe574355bb323dcdc3be0058" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 41335 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34235 } health_report { overall_health: UNKNOWN } } }
I20260709 13:24:30.320266 28115 consensus_queue.cc:1048] T d747d5acdf4f41b69c132cde690e49ff P 880fc4c7dff4417187b75cc1d9e9dcf5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0788dbdabe574355bb323dcdc3be0058" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 41335 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:24:30.333101 28115 consensus_queue.cc:1048] T d747d5acdf4f41b69c132cde690e49ff P 880fc4c7dff4417187b75cc1d9e9dcf5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9cb4300ea183421ab40709a05de5a877" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 33597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:24:30.350564 28115 consensus_queue.cc:1048] T 1af49a625b20491586dcea4961e46734 P 880fc4c7dff4417187b75cc1d9e9dcf5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9cb4300ea183421ab40709a05de5a877" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 33597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:24:30.362583 28115 consensus_queue.cc:1048] T 1af49a625b20491586dcea4961e46734 P 880fc4c7dff4417187b75cc1d9e9dcf5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0788dbdabe574355bb323dcdc3be0058" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 41335 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:24:30.472847 28079 consensus_queue.cc:1048] T 032f598cb30343c2a1ca3bab728e5b39 P 9cb4300ea183421ab40709a05de5a877 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0788dbdabe574355bb323dcdc3be0058" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 41335 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:24:30.485255 28096 consensus_queue.cc:1048] T 032f598cb30343c2a1ca3bab728e5b39 P 9cb4300ea183421ab40709a05de5a877 [LEADER]: Connected to new peer: Peer: permanent_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34235 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:24:30.533751 27836 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "27629b85cc3e4cb6b837dd26e3ebff47"
dest_uuid: "0788dbdabe574355bb323dcdc3be0058"
mode: GRACEFUL
new_leader_uuid: "9cb4300ea183421ab40709a05de5a877"
 from {username='slave'} at 127.0.0.1:48802
I20260709 13:24:30.534462 27836 raft_consensus.cc:606] T 27629b85cc3e4cb6b837dd26e3ebff47 P 0788dbdabe574355bb323dcdc3be0058 [term 2 FOLLOWER]: Received request to transfer leadership to TS 9cb4300ea183421ab40709a05de5a877
I20260709 13:24:30.534763 27836 raft_consensus.cc:614] T 27629b85cc3e4cb6b837dd26e3ebff47 P 0788dbdabe574355bb323dcdc3be0058 [term 2 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:24:30.540459 27910 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8e6b55496c134353a770e2642fd7490d"
dest_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5"
mode: GRACEFUL
new_leader_uuid: "9cb4300ea183421ab40709a05de5a877"
 from {username='slave'} at 127.0.0.1:58342
I20260709 13:24:30.541065 27910 raft_consensus.cc:606] T 8e6b55496c134353a770e2642fd7490d P 880fc4c7dff4417187b75cc1d9e9dcf5 [term 1 LEADER]: Received request to transfer leadership to TS 9cb4300ea183421ab40709a05de5a877
I20260709 13:24:30.546500 27836 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d7fd85f92c0a481aa54401371638252b"
dest_uuid: "0788dbdabe574355bb323dcdc3be0058"
mode: GRACEFUL
new_leader_uuid: "9cb4300ea183421ab40709a05de5a877"
 from {username='slave'} at 127.0.0.1:48802
I20260709 13:24:30.547360 27836 raft_consensus.cc:606] T d7fd85f92c0a481aa54401371638252b P 0788dbdabe574355bb323dcdc3be0058 [term 1 LEADER]: Received request to transfer leadership to TS 9cb4300ea183421ab40709a05de5a877
I20260709 13:24:30.556314 27910 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d747d5acdf4f41b69c132cde690e49ff"
dest_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5"
mode: GRACEFUL
new_leader_uuid: "0788dbdabe574355bb323dcdc3be0058"
 from {username='slave'} at 127.0.0.1:58342
I20260709 13:24:30.556921 27910 raft_consensus.cc:606] T d747d5acdf4f41b69c132cde690e49ff P 880fc4c7dff4417187b75cc1d9e9dcf5 [term 1 LEADER]: Received request to transfer leadership to TS 0788dbdabe574355bb323dcdc3be0058
I20260709 13:24:30.561862 27910 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "660aa4903c494ba1bce988191700e129"
dest_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5"
mode: GRACEFUL
new_leader_uuid: "0788dbdabe574355bb323dcdc3be0058"
 from {username='slave'} at 127.0.0.1:58342
I20260709 13:24:30.562381 27910 raft_consensus.cc:606] T 660aa4903c494ba1bce988191700e129 P 880fc4c7dff4417187b75cc1d9e9dcf5 [term 1 LEADER]: Received request to transfer leadership to TS 0788dbdabe574355bb323dcdc3be0058
I20260709 13:24:30.585700 27762 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "032f598cb30343c2a1ca3bab728e5b39"
dest_uuid: "9cb4300ea183421ab40709a05de5a877"
mode: GRACEFUL
new_leader_uuid: "0788dbdabe574355bb323dcdc3be0058"
 from {username='slave'} at 127.0.0.1:43836
I20260709 13:24:30.586400 27762 raft_consensus.cc:606] T 032f598cb30343c2a1ca3bab728e5b39 P 9cb4300ea183421ab40709a05de5a877 [term 1 LEADER]: Received request to transfer leadership to TS 0788dbdabe574355bb323dcdc3be0058
I20260709 13:24:30.592563 27762 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "96f830fa400c473881739b3a953f0178"
dest_uuid: "9cb4300ea183421ab40709a05de5a877"
mode: GRACEFUL
new_leader_uuid: "0788dbdabe574355bb323dcdc3be0058"
 from {username='slave'} at 127.0.0.1:43836
I20260709 13:24:30.593171 27762 raft_consensus.cc:606] T 96f830fa400c473881739b3a953f0178 P 9cb4300ea183421ab40709a05de5a877 [term 1 LEADER]: Received request to transfer leadership to TS 0788dbdabe574355bb323dcdc3be0058
I20260709 13:24:30.707772 28115 raft_consensus.cc:993] T d747d5acdf4f41b69c132cde690e49ff P 880fc4c7dff4417187b75cc1d9e9dcf5: : Instructing follower 0788dbdabe574355bb323dcdc3be0058 to start an election
I20260709 13:24:30.708191 28115 raft_consensus.cc:1081] T d747d5acdf4f41b69c132cde690e49ff P 880fc4c7dff4417187b75cc1d9e9dcf5 [term 1 LEADER]: Signalling peer 0788dbdabe574355bb323dcdc3be0058 to start an election
I20260709 13:24:30.709651 27836 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d747d5acdf4f41b69c132cde690e49ff"
dest_uuid: "0788dbdabe574355bb323dcdc3be0058"
 from {username='slave'} at 127.0.0.1:48800
I20260709 13:24:30.710213 27836 raft_consensus.cc:493] T d747d5acdf4f41b69c132cde690e49ff P 0788dbdabe574355bb323dcdc3be0058 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:24:30.710528 27836 raft_consensus.cc:3060] T d747d5acdf4f41b69c132cde690e49ff P 0788dbdabe574355bb323dcdc3be0058 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:24:30.715975 27836 raft_consensus.cc:515] T d747d5acdf4f41b69c132cde690e49ff P 0788dbdabe574355bb323dcdc3be0058 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb4300ea183421ab40709a05de5a877" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 33597 } } peers { permanent_uuid: "0788dbdabe574355bb323dcdc3be0058" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 41335 } } peers { permanent_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34235 } }
I20260709 13:24:30.718662 27836 leader_election.cc:290] T d747d5acdf4f41b69c132cde690e49ff P 0788dbdabe574355bb323dcdc3be0058 [CANDIDATE]: Term 2 election: Requested vote from peers 9cb4300ea183421ab40709a05de5a877 (127.14.177.129:33597), 880fc4c7dff4417187b75cc1d9e9dcf5 (127.14.177.131:34235)
I20260709 13:24:30.719461 27759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d747d5acdf4f41b69c132cde690e49ff" candidate_uuid: "0788dbdabe574355bb323dcdc3be0058" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "9cb4300ea183421ab40709a05de5a877"
I20260709 13:24:30.720157 27759 raft_consensus.cc:3060] T d747d5acdf4f41b69c132cde690e49ff P 9cb4300ea183421ab40709a05de5a877 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:24:30.720504 27911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d747d5acdf4f41b69c132cde690e49ff" candidate_uuid: "0788dbdabe574355bb323dcdc3be0058" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5"
I20260709 13:24:30.721196 27911 raft_consensus.cc:3055] T d747d5acdf4f41b69c132cde690e49ff P 880fc4c7dff4417187b75cc1d9e9dcf5 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:24:30.721403 28115 raft_consensus.cc:993] T 8e6b55496c134353a770e2642fd7490d P 880fc4c7dff4417187b75cc1d9e9dcf5: : Instructing follower 9cb4300ea183421ab40709a05de5a877 to start an election
I20260709 13:24:30.721493 27911 raft_consensus.cc:740] T d747d5acdf4f41b69c132cde690e49ff P 880fc4c7dff4417187b75cc1d9e9dcf5 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 880fc4c7dff4417187b75cc1d9e9dcf5, State: Running, Role: LEADER
I20260709 13:24:30.722144 28082 raft_consensus.cc:1081] T 8e6b55496c134353a770e2642fd7490d P 880fc4c7dff4417187b75cc1d9e9dcf5 [term 1 LEADER]: Signalling peer 9cb4300ea183421ab40709a05de5a877 to start an election
I20260709 13:24:30.722587 27911 consensus_queue.cc:260] T d747d5acdf4f41b69c132cde690e49ff P 880fc4c7dff4417187b75cc1d9e9dcf5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb4300ea183421ab40709a05de5a877" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 33597 } } peers { permanent_uuid: "0788dbdabe574355bb323dcdc3be0058" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 41335 } } peers { permanent_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34235 } }
I20260709 13:24:30.723933 27911 raft_consensus.cc:3060] T d747d5acdf4f41b69c132cde690e49ff P 880fc4c7dff4417187b75cc1d9e9dcf5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:24:30.724711 27760 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "8e6b55496c134353a770e2642fd7490d"
dest_uuid: "9cb4300ea183421ab40709a05de5a877"
 from {username='slave'} at 127.0.0.1:59050
I20260709 13:24:30.725353 27760 raft_consensus.cc:493] T 8e6b55496c134353a770e2642fd7490d P 9cb4300ea183421ab40709a05de5a877 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:24:30.725945 27760 raft_consensus.cc:3060] T 8e6b55496c134353a770e2642fd7490d P 9cb4300ea183421ab40709a05de5a877 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:24:30.729305 27759 raft_consensus.cc:2468] T d747d5acdf4f41b69c132cde690e49ff P 9cb4300ea183421ab40709a05de5a877 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0788dbdabe574355bb323dcdc3be0058 in term 2.
I20260709 13:24:30.731055 27800 leader_election.cc:304] T d747d5acdf4f41b69c132cde690e49ff P 0788dbdabe574355bb323dcdc3be0058 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0788dbdabe574355bb323dcdc3be0058, 9cb4300ea183421ab40709a05de5a877; no voters: 
I20260709 13:24:30.731938 28071 raft_consensus.cc:2804] T d747d5acdf4f41b69c132cde690e49ff P 0788dbdabe574355bb323dcdc3be0058 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:24:30.732460 28071 raft_consensus.cc:697] T d747d5acdf4f41b69c132cde690e49ff P 0788dbdabe574355bb323dcdc3be0058 [term 2 LEADER]: Becoming Leader. State: Replica: 0788dbdabe574355bb323dcdc3be0058, State: Running, Role: LEADER
I20260709 13:24:30.733415 28071 consensus_queue.cc:237] T d747d5acdf4f41b69c132cde690e49ff P 0788dbdabe574355bb323dcdc3be0058 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb4300ea183421ab40709a05de5a877" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 33597 } } peers { permanent_uuid: "0788dbdabe574355bb323dcdc3be0058" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 41335 } } peers { permanent_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34235 } }
I20260709 13:24:30.733860 27911 raft_consensus.cc:2468] T d747d5acdf4f41b69c132cde690e49ff P 880fc4c7dff4417187b75cc1d9e9dcf5 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0788dbdabe574355bb323dcdc3be0058 in term 2.
I20260709 13:24:30.734368 27760 raft_consensus.cc:515] T 8e6b55496c134353a770e2642fd7490d P 9cb4300ea183421ab40709a05de5a877 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb4300ea183421ab40709a05de5a877" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 33597 } } peers { permanent_uuid: "0788dbdabe574355bb323dcdc3be0058" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 41335 } } peers { permanent_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34235 } }
I20260709 13:24:30.736651 27760 leader_election.cc:290] T 8e6b55496c134353a770e2642fd7490d P 9cb4300ea183421ab40709a05de5a877 [CANDIDATE]: Term 2 election: Requested vote from peers 0788dbdabe574355bb323dcdc3be0058 (127.14.177.130:41335), 880fc4c7dff4417187b75cc1d9e9dcf5 (127.14.177.131:34235)
I20260709 13:24:30.738466 27911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e6b55496c134353a770e2642fd7490d" candidate_uuid: "9cb4300ea183421ab40709a05de5a877" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5"
I20260709 13:24:30.739006 27911 raft_consensus.cc:3055] T 8e6b55496c134353a770e2642fd7490d P 880fc4c7dff4417187b75cc1d9e9dcf5 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:24:30.739374 27911 raft_consensus.cc:740] T 8e6b55496c134353a770e2642fd7490d P 880fc4c7dff4417187b75cc1d9e9dcf5 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 880fc4c7dff4417187b75cc1d9e9dcf5, State: Running, Role: LEADER
I20260709 13:24:30.740319 27911 consensus_queue.cc:260] T 8e6b55496c134353a770e2642fd7490d P 880fc4c7dff4417187b75cc1d9e9dcf5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb4300ea183421ab40709a05de5a877" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 33597 } } peers { permanent_uuid: "0788dbdabe574355bb323dcdc3be0058" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 41335 } } peers { permanent_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34235 } }
I20260709 13:24:30.740763 27836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e6b55496c134353a770e2642fd7490d" candidate_uuid: "9cb4300ea183421ab40709a05de5a877" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "0788dbdabe574355bb323dcdc3be0058"
I20260709 13:24:30.741447 27836 raft_consensus.cc:3060] T 8e6b55496c134353a770e2642fd7490d P 0788dbdabe574355bb323dcdc3be0058 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:24:30.741806 27911 raft_consensus.cc:3060] T 8e6b55496c134353a770e2642fd7490d P 880fc4c7dff4417187b75cc1d9e9dcf5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:24:30.744036 27660 catalog_manager.cc:5697] T d747d5acdf4f41b69c132cde690e49ff P 0788dbdabe574355bb323dcdc3be0058 reported cstate change: term changed from 1 to 2, leader changed from 880fc4c7dff4417187b75cc1d9e9dcf5 (127.14.177.131) to 0788dbdabe574355bb323dcdc3be0058 (127.14.177.130). New cstate: current_term: 2 leader_uuid: "0788dbdabe574355bb323dcdc3be0058" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb4300ea183421ab40709a05de5a877" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 33597 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0788dbdabe574355bb323dcdc3be0058" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 41335 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34235 } health_report { overall_health: UNKNOWN } } }
I20260709 13:24:30.748322 27836 raft_consensus.cc:2468] T 8e6b55496c134353a770e2642fd7490d P 0788dbdabe574355bb323dcdc3be0058 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9cb4300ea183421ab40709a05de5a877 in term 2.
I20260709 13:24:30.749428 27911 raft_consensus.cc:2468] T 8e6b55496c134353a770e2642fd7490d P 880fc4c7dff4417187b75cc1d9e9dcf5 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9cb4300ea183421ab40709a05de5a877 in term 2.
I20260709 13:24:30.749984 27727 leader_election.cc:304] T 8e6b55496c134353a770e2642fd7490d P 9cb4300ea183421ab40709a05de5a877 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0788dbdabe574355bb323dcdc3be0058, 9cb4300ea183421ab40709a05de5a877; no voters: 
I20260709 13:24:30.750851 28096 raft_consensus.cc:2804] T 8e6b55496c134353a770e2642fd7490d P 9cb4300ea183421ab40709a05de5a877 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:24:30.751770 28096 raft_consensus.cc:697] T 8e6b55496c134353a770e2642fd7490d P 9cb4300ea183421ab40709a05de5a877 [term 2 LEADER]: Becoming Leader. State: Replica: 9cb4300ea183421ab40709a05de5a877, State: Running, Role: LEADER
I20260709 13:24:30.752768 28096 consensus_queue.cc:237] T 8e6b55496c134353a770e2642fd7490d P 9cb4300ea183421ab40709a05de5a877 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb4300ea183421ab40709a05de5a877" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 33597 } } peers { permanent_uuid: "0788dbdabe574355bb323dcdc3be0058" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 41335 } } peers { permanent_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34235 } }
I20260709 13:24:30.755955 28082 raft_consensus.cc:993] T 660aa4903c494ba1bce988191700e129 P 880fc4c7dff4417187b75cc1d9e9dcf5: : Instructing follower 0788dbdabe574355bb323dcdc3be0058 to start an election
I20260709 13:24:30.756575 28082 raft_consensus.cc:1081] T 660aa4903c494ba1bce988191700e129 P 880fc4c7dff4417187b75cc1d9e9dcf5 [term 1 LEADER]: Signalling peer 0788dbdabe574355bb323dcdc3be0058 to start an election
I20260709 13:24:30.760589 27836 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "660aa4903c494ba1bce988191700e129"
dest_uuid: "0788dbdabe574355bb323dcdc3be0058"
 from {username='slave'} at 127.0.0.1:48800
I20260709 13:24:30.761217 27836 raft_consensus.cc:493] T 660aa4903c494ba1bce988191700e129 P 0788dbdabe574355bb323dcdc3be0058 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:24:30.761618 27836 raft_consensus.cc:3060] T 660aa4903c494ba1bce988191700e129 P 0788dbdabe574355bb323dcdc3be0058 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:24:30.762821 27660 catalog_manager.cc:5697] T 8e6b55496c134353a770e2642fd7490d P 9cb4300ea183421ab40709a05de5a877 reported cstate change: term changed from 1 to 2, leader changed from 880fc4c7dff4417187b75cc1d9e9dcf5 (127.14.177.131) to 9cb4300ea183421ab40709a05de5a877 (127.14.177.129). New cstate: current_term: 2 leader_uuid: "9cb4300ea183421ab40709a05de5a877" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb4300ea183421ab40709a05de5a877" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 33597 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0788dbdabe574355bb323dcdc3be0058" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 41335 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34235 } health_report { overall_health: UNKNOWN } } }
I20260709 13:24:30.768096 27836 raft_consensus.cc:515] T 660aa4903c494ba1bce988191700e129 P 0788dbdabe574355bb323dcdc3be0058 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb4300ea183421ab40709a05de5a877" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 33597 } } peers { permanent_uuid: "0788dbdabe574355bb323dcdc3be0058" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 41335 } } peers { permanent_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34235 } }
I20260709 13:24:30.770355 27836 leader_election.cc:290] T 660aa4903c494ba1bce988191700e129 P 0788dbdabe574355bb323dcdc3be0058 [CANDIDATE]: Term 2 election: Requested vote from peers 9cb4300ea183421ab40709a05de5a877 (127.14.177.129:33597), 880fc4c7dff4417187b75cc1d9e9dcf5 (127.14.177.131:34235)
I20260709 13:24:30.771880 27760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "660aa4903c494ba1bce988191700e129" candidate_uuid: "0788dbdabe574355bb323dcdc3be0058" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "9cb4300ea183421ab40709a05de5a877"
I20260709 13:24:30.772229 27911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "660aa4903c494ba1bce988191700e129" candidate_uuid: "0788dbdabe574355bb323dcdc3be0058" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5"
I20260709 13:24:30.772541 27760 raft_consensus.cc:3060] T 660aa4903c494ba1bce988191700e129 P 9cb4300ea183421ab40709a05de5a877 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:24:30.772997 27911 raft_consensus.cc:3055] T 660aa4903c494ba1bce988191700e129 P 880fc4c7dff4417187b75cc1d9e9dcf5 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:24:30.773342 27911 raft_consensus.cc:740] T 660aa4903c494ba1bce988191700e129 P 880fc4c7dff4417187b75cc1d9e9dcf5 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 880fc4c7dff4417187b75cc1d9e9dcf5, State: Running, Role: LEADER
I20260709 13:24:30.774214 27911 consensus_queue.cc:260] T 660aa4903c494ba1bce988191700e129 P 880fc4c7dff4417187b75cc1d9e9dcf5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb4300ea183421ab40709a05de5a877" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 33597 } } peers { permanent_uuid: "0788dbdabe574355bb323dcdc3be0058" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 41335 } } peers { permanent_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34235 } }
I20260709 13:24:30.775418 27911 raft_consensus.cc:3060] T 660aa4903c494ba1bce988191700e129 P 880fc4c7dff4417187b75cc1d9e9dcf5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:24:30.778141 27760 raft_consensus.cc:2468] T 660aa4903c494ba1bce988191700e129 P 9cb4300ea183421ab40709a05de5a877 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0788dbdabe574355bb323dcdc3be0058 in term 2.
I20260709 13:24:30.779510 27800 leader_election.cc:304] T 660aa4903c494ba1bce988191700e129 P 0788dbdabe574355bb323dcdc3be0058 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0788dbdabe574355bb323dcdc3be0058, 9cb4300ea183421ab40709a05de5a877; no voters: 
I20260709 13:24:30.780274 28071 raft_consensus.cc:2804] T 660aa4903c494ba1bce988191700e129 P 0788dbdabe574355bb323dcdc3be0058 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:24:30.780862 28071 raft_consensus.cc:697] T 660aa4903c494ba1bce988191700e129 P 0788dbdabe574355bb323dcdc3be0058 [term 2 LEADER]: Becoming Leader. State: Replica: 0788dbdabe574355bb323dcdc3be0058, State: Running, Role: LEADER
I20260709 13:24:30.782110 27911 raft_consensus.cc:2468] T 660aa4903c494ba1bce988191700e129 P 880fc4c7dff4417187b75cc1d9e9dcf5 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0788dbdabe574355bb323dcdc3be0058 in term 2.
I20260709 13:24:30.782020 28071 consensus_queue.cc:237] T 660aa4903c494ba1bce988191700e129 P 0788dbdabe574355bb323dcdc3be0058 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb4300ea183421ab40709a05de5a877" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 33597 } } peers { permanent_uuid: "0788dbdabe574355bb323dcdc3be0058" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 41335 } } peers { permanent_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34235 } }
I20260709 13:24:30.785986 28096 consensus_queue.cc:1048] T 96f830fa400c473881739b3a953f0178 P 9cb4300ea183421ab40709a05de5a877 [LEADER]: Connected to new peer: Peer: permanent_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34235 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:24:30.790453 27660 catalog_manager.cc:5697] T 660aa4903c494ba1bce988191700e129 P 0788dbdabe574355bb323dcdc3be0058 reported cstate change: term changed from 1 to 2, leader changed from 880fc4c7dff4417187b75cc1d9e9dcf5 (127.14.177.131) to 0788dbdabe574355bb323dcdc3be0058 (127.14.177.130). New cstate: current_term: 2 leader_uuid: "0788dbdabe574355bb323dcdc3be0058" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb4300ea183421ab40709a05de5a877" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 33597 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0788dbdabe574355bb323dcdc3be0058" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 41335 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34235 } health_report { overall_health: UNKNOWN } } }
I20260709 13:24:30.799209 28095 consensus_queue.cc:1048] T 96f830fa400c473881739b3a953f0178 P 9cb4300ea183421ab40709a05de5a877 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0788dbdabe574355bb323dcdc3be0058" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 41335 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:24:30.813834 28095 raft_consensus.cc:993] T 96f830fa400c473881739b3a953f0178 P 9cb4300ea183421ab40709a05de5a877: : Instructing follower 0788dbdabe574355bb323dcdc3be0058 to start an election
I20260709 13:24:30.814360 28095 raft_consensus.cc:1081] T 96f830fa400c473881739b3a953f0178 P 9cb4300ea183421ab40709a05de5a877 [term 1 LEADER]: Signalling peer 0788dbdabe574355bb323dcdc3be0058 to start an election
I20260709 13:24:30.816111 27836 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "96f830fa400c473881739b3a953f0178"
dest_uuid: "0788dbdabe574355bb323dcdc3be0058"
 from {username='slave'} at 127.0.0.1:48788
I20260709 13:24:30.816676 27836 raft_consensus.cc:493] T 96f830fa400c473881739b3a953f0178 P 0788dbdabe574355bb323dcdc3be0058 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:24:30.817055 27836 raft_consensus.cc:3060] T 96f830fa400c473881739b3a953f0178 P 0788dbdabe574355bb323dcdc3be0058 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:24:30.824690 27836 raft_consensus.cc:515] T 96f830fa400c473881739b3a953f0178 P 0788dbdabe574355bb323dcdc3be0058 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb4300ea183421ab40709a05de5a877" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 33597 } } peers { permanent_uuid: "0788dbdabe574355bb323dcdc3be0058" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 41335 } } peers { permanent_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34235 } }
I20260709 13:24:30.827730 27760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96f830fa400c473881739b3a953f0178" candidate_uuid: "0788dbdabe574355bb323dcdc3be0058" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "9cb4300ea183421ab40709a05de5a877"
I20260709 13:24:30.828222 27911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96f830fa400c473881739b3a953f0178" candidate_uuid: "0788dbdabe574355bb323dcdc3be0058" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5"
I20260709 13:24:30.827121 27836 leader_election.cc:290] T 96f830fa400c473881739b3a953f0178 P 0788dbdabe574355bb323dcdc3be0058 [CANDIDATE]: Term 2 election: Requested vote from peers 9cb4300ea183421ab40709a05de5a877 (127.14.177.129:33597), 880fc4c7dff4417187b75cc1d9e9dcf5 (127.14.177.131:34235)
I20260709 13:24:30.828404 27760 raft_consensus.cc:3055] T 96f830fa400c473881739b3a953f0178 P 9cb4300ea183421ab40709a05de5a877 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:24:30.828877 27760 raft_consensus.cc:740] T 96f830fa400c473881739b3a953f0178 P 9cb4300ea183421ab40709a05de5a877 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 9cb4300ea183421ab40709a05de5a877, State: Running, Role: LEADER
I20260709 13:24:30.828884 27911 raft_consensus.cc:3060] T 96f830fa400c473881739b3a953f0178 P 880fc4c7dff4417187b75cc1d9e9dcf5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:24:30.830133 27760 consensus_queue.cc:260] T 96f830fa400c473881739b3a953f0178 P 9cb4300ea183421ab40709a05de5a877 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb4300ea183421ab40709a05de5a877" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 33597 } } peers { permanent_uuid: "0788dbdabe574355bb323dcdc3be0058" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 41335 } } peers { permanent_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34235 } }
I20260709 13:24:30.831363 27760 raft_consensus.cc:3060] T 96f830fa400c473881739b3a953f0178 P 9cb4300ea183421ab40709a05de5a877 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:24:30.836417 27911 raft_consensus.cc:2468] T 96f830fa400c473881739b3a953f0178 P 880fc4c7dff4417187b75cc1d9e9dcf5 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0788dbdabe574355bb323dcdc3be0058 in term 2.
I20260709 13:24:30.836932 27760 raft_consensus.cc:2468] T 96f830fa400c473881739b3a953f0178 P 9cb4300ea183421ab40709a05de5a877 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0788dbdabe574355bb323dcdc3be0058 in term 2.
I20260709 13:24:30.837957 27802 leader_election.cc:304] T 96f830fa400c473881739b3a953f0178 P 0788dbdabe574355bb323dcdc3be0058 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0788dbdabe574355bb323dcdc3be0058, 880fc4c7dff4417187b75cc1d9e9dcf5; no voters: 
I20260709 13:24:30.838855 28071 raft_consensus.cc:2804] T 96f830fa400c473881739b3a953f0178 P 0788dbdabe574355bb323dcdc3be0058 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:24:30.839417 28071 raft_consensus.cc:697] T 96f830fa400c473881739b3a953f0178 P 0788dbdabe574355bb323dcdc3be0058 [term 2 LEADER]: Becoming Leader. State: Replica: 0788dbdabe574355bb323dcdc3be0058, State: Running, Role: LEADER
I20260709 13:24:30.840087 28071 consensus_queue.cc:237] T 96f830fa400c473881739b3a953f0178 P 0788dbdabe574355bb323dcdc3be0058 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb4300ea183421ab40709a05de5a877" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 33597 } } peers { permanent_uuid: "0788dbdabe574355bb323dcdc3be0058" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 41335 } } peers { permanent_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34235 } }
I20260709 13:24:30.846799 27657 catalog_manager.cc:5697] T 96f830fa400c473881739b3a953f0178 P 0788dbdabe574355bb323dcdc3be0058 reported cstate change: term changed from 1 to 2, leader changed from 9cb4300ea183421ab40709a05de5a877 (127.14.177.129) to 0788dbdabe574355bb323dcdc3be0058 (127.14.177.130). New cstate: current_term: 2 leader_uuid: "0788dbdabe574355bb323dcdc3be0058" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb4300ea183421ab40709a05de5a877" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 33597 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0788dbdabe574355bb323dcdc3be0058" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 41335 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34235 } health_report { overall_health: UNKNOWN } } }
I20260709 13:24:30.932901 27955 raft_consensus.cc:993] T d7fd85f92c0a481aa54401371638252b P 0788dbdabe574355bb323dcdc3be0058: : Instructing follower 9cb4300ea183421ab40709a05de5a877 to start an election
I20260709 13:24:30.933597 27955 raft_consensus.cc:1081] T d7fd85f92c0a481aa54401371638252b P 0788dbdabe574355bb323dcdc3be0058 [term 1 LEADER]: Signalling peer 9cb4300ea183421ab40709a05de5a877 to start an election
I20260709 13:24:30.935475 27760 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d7fd85f92c0a481aa54401371638252b"
dest_uuid: "9cb4300ea183421ab40709a05de5a877"
 from {username='slave'} at 127.0.0.1:59038
I20260709 13:24:30.936022 27760 raft_consensus.cc:493] T d7fd85f92c0a481aa54401371638252b P 9cb4300ea183421ab40709a05de5a877 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:24:30.936342 27760 raft_consensus.cc:3060] T d7fd85f92c0a481aa54401371638252b P 9cb4300ea183421ab40709a05de5a877 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:24:30.941807 27760 raft_consensus.cc:515] T d7fd85f92c0a481aa54401371638252b P 9cb4300ea183421ab40709a05de5a877 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb4300ea183421ab40709a05de5a877" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 33597 } } peers { permanent_uuid: "0788dbdabe574355bb323dcdc3be0058" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 41335 } } peers { permanent_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34235 } }
I20260709 13:24:30.943689 27760 leader_election.cc:290] T d7fd85f92c0a481aa54401371638252b P 9cb4300ea183421ab40709a05de5a877 [CANDIDATE]: Term 2 election: Requested vote from peers 0788dbdabe574355bb323dcdc3be0058 (127.14.177.130:41335), 880fc4c7dff4417187b75cc1d9e9dcf5 (127.14.177.131:34235)
I20260709 13:24:30.944773 27836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7fd85f92c0a481aa54401371638252b" candidate_uuid: "9cb4300ea183421ab40709a05de5a877" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "0788dbdabe574355bb323dcdc3be0058"
I20260709 13:24:30.945021 27911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7fd85f92c0a481aa54401371638252b" candidate_uuid: "9cb4300ea183421ab40709a05de5a877" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5"
I20260709 13:24:30.945749 27836 raft_consensus.cc:3055] T d7fd85f92c0a481aa54401371638252b P 0788dbdabe574355bb323dcdc3be0058 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:24:30.946096 27911 raft_consensus.cc:3060] T d7fd85f92c0a481aa54401371638252b P 880fc4c7dff4417187b75cc1d9e9dcf5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:24:30.946233 27836 raft_consensus.cc:740] T d7fd85f92c0a481aa54401371638252b P 0788dbdabe574355bb323dcdc3be0058 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 0788dbdabe574355bb323dcdc3be0058, State: Running, Role: LEADER
I20260709 13:24:30.947325 27836 consensus_queue.cc:260] T d7fd85f92c0a481aa54401371638252b P 0788dbdabe574355bb323dcdc3be0058 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb4300ea183421ab40709a05de5a877" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 33597 } } peers { permanent_uuid: "0788dbdabe574355bb323dcdc3be0058" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 41335 } } peers { permanent_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34235 } }
I20260709 13:24:30.948904 27836 raft_consensus.cc:3060] T d7fd85f92c0a481aa54401371638252b P 0788dbdabe574355bb323dcdc3be0058 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:24:30.955054 27911 raft_consensus.cc:2468] T d7fd85f92c0a481aa54401371638252b P 880fc4c7dff4417187b75cc1d9e9dcf5 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9cb4300ea183421ab40709a05de5a877 in term 2.
I20260709 13:24:30.956586 27727 leader_election.cc:304] T d7fd85f92c0a481aa54401371638252b P 9cb4300ea183421ab40709a05de5a877 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 880fc4c7dff4417187b75cc1d9e9dcf5, 9cb4300ea183421ab40709a05de5a877; no voters: 
I20260709 13:24:30.956810 27836 raft_consensus.cc:2468] T d7fd85f92c0a481aa54401371638252b P 0788dbdabe574355bb323dcdc3be0058 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9cb4300ea183421ab40709a05de5a877 in term 2.
I20260709 13:24:30.957427 28079 raft_consensus.cc:2804] T d7fd85f92c0a481aa54401371638252b P 9cb4300ea183421ab40709a05de5a877 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:24:30.957963 28079 raft_consensus.cc:697] T d7fd85f92c0a481aa54401371638252b P 9cb4300ea183421ab40709a05de5a877 [term 2 LEADER]: Becoming Leader. State: Replica: 9cb4300ea183421ab40709a05de5a877, State: Running, Role: LEADER
I20260709 13:24:30.958925 28079 consensus_queue.cc:237] T d7fd85f92c0a481aa54401371638252b P 9cb4300ea183421ab40709a05de5a877 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb4300ea183421ab40709a05de5a877" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 33597 } } peers { permanent_uuid: "0788dbdabe574355bb323dcdc3be0058" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 41335 } } peers { permanent_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34235 } }
I20260709 13:24:30.966588 27657 catalog_manager.cc:5697] T d7fd85f92c0a481aa54401371638252b P 9cb4300ea183421ab40709a05de5a877 reported cstate change: term changed from 1 to 2, leader changed from 0788dbdabe574355bb323dcdc3be0058 (127.14.177.130) to 9cb4300ea183421ab40709a05de5a877 (127.14.177.129). New cstate: current_term: 2 leader_uuid: "9cb4300ea183421ab40709a05de5a877" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb4300ea183421ab40709a05de5a877" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 33597 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0788dbdabe574355bb323dcdc3be0058" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 41335 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34235 } health_report { overall_health: UNKNOWN } } }
I20260709 13:24:31.083030 28076 raft_consensus.cc:993] T 032f598cb30343c2a1ca3bab728e5b39 P 9cb4300ea183421ab40709a05de5a877: : Instructing follower 0788dbdabe574355bb323dcdc3be0058 to start an election
I20260709 13:24:31.083520 28076 raft_consensus.cc:1081] T 032f598cb30343c2a1ca3bab728e5b39 P 9cb4300ea183421ab40709a05de5a877 [term 1 LEADER]: Signalling peer 0788dbdabe574355bb323dcdc3be0058 to start an election
I20260709 13:24:31.085235 27836 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "032f598cb30343c2a1ca3bab728e5b39"
dest_uuid: "0788dbdabe574355bb323dcdc3be0058"
 from {username='slave'} at 127.0.0.1:48788
I20260709 13:24:31.085932 27836 raft_consensus.cc:493] T 032f598cb30343c2a1ca3bab728e5b39 P 0788dbdabe574355bb323dcdc3be0058 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:24:31.086305 27836 raft_consensus.cc:3060] T 032f598cb30343c2a1ca3bab728e5b39 P 0788dbdabe574355bb323dcdc3be0058 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:24:31.091543 27836 raft_consensus.cc:515] T 032f598cb30343c2a1ca3bab728e5b39 P 0788dbdabe574355bb323dcdc3be0058 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb4300ea183421ab40709a05de5a877" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 33597 } } peers { permanent_uuid: "0788dbdabe574355bb323dcdc3be0058" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 41335 } } peers { permanent_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34235 } }
I20260709 13:24:31.093310 27836 leader_election.cc:290] T 032f598cb30343c2a1ca3bab728e5b39 P 0788dbdabe574355bb323dcdc3be0058 [CANDIDATE]: Term 2 election: Requested vote from peers 9cb4300ea183421ab40709a05de5a877 (127.14.177.129:33597), 880fc4c7dff4417187b75cc1d9e9dcf5 (127.14.177.131:34235)
I20260709 13:24:31.094163 27760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "032f598cb30343c2a1ca3bab728e5b39" candidate_uuid: "0788dbdabe574355bb323dcdc3be0058" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "9cb4300ea183421ab40709a05de5a877"
I20260709 13:24:31.094552 27911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "032f598cb30343c2a1ca3bab728e5b39" candidate_uuid: "0788dbdabe574355bb323dcdc3be0058" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5"
I20260709 13:24:31.094888 27760 raft_consensus.cc:3055] T 032f598cb30343c2a1ca3bab728e5b39 P 9cb4300ea183421ab40709a05de5a877 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:24:31.095243 27911 raft_consensus.cc:3060] T 032f598cb30343c2a1ca3bab728e5b39 P 880fc4c7dff4417187b75cc1d9e9dcf5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:24:31.095326 27760 raft_consensus.cc:740] T 032f598cb30343c2a1ca3bab728e5b39 P 9cb4300ea183421ab40709a05de5a877 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 9cb4300ea183421ab40709a05de5a877, State: Running, Role: LEADER
I20260709 13:24:31.096271 27760 consensus_queue.cc:260] T 032f598cb30343c2a1ca3bab728e5b39 P 9cb4300ea183421ab40709a05de5a877 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb4300ea183421ab40709a05de5a877" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 33597 } } peers { permanent_uuid: "0788dbdabe574355bb323dcdc3be0058" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 41335 } } peers { permanent_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34235 } }
I20260709 13:24:31.097635 27760 raft_consensus.cc:3060] T 032f598cb30343c2a1ca3bab728e5b39 P 9cb4300ea183421ab40709a05de5a877 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:24:31.103343 27911 raft_consensus.cc:2468] T 032f598cb30343c2a1ca3bab728e5b39 P 880fc4c7dff4417187b75cc1d9e9dcf5 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0788dbdabe574355bb323dcdc3be0058 in term 2.
I20260709 13:24:31.104692 27802 leader_election.cc:304] T 032f598cb30343c2a1ca3bab728e5b39 P 0788dbdabe574355bb323dcdc3be0058 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0788dbdabe574355bb323dcdc3be0058, 880fc4c7dff4417187b75cc1d9e9dcf5; no voters: 
I20260709 13:24:31.105556 27955 raft_consensus.cc:2804] T 032f598cb30343c2a1ca3bab728e5b39 P 0788dbdabe574355bb323dcdc3be0058 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:24:31.106055 27955 raft_consensus.cc:697] T 032f598cb30343c2a1ca3bab728e5b39 P 0788dbdabe574355bb323dcdc3be0058 [term 2 LEADER]: Becoming Leader. State: Replica: 0788dbdabe574355bb323dcdc3be0058, State: Running, Role: LEADER
I20260709 13:24:31.106799 27760 raft_consensus.cc:2468] T 032f598cb30343c2a1ca3bab728e5b39 P 9cb4300ea183421ab40709a05de5a877 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0788dbdabe574355bb323dcdc3be0058 in term 2.
I20260709 13:24:31.106915 27955 consensus_queue.cc:237] T 032f598cb30343c2a1ca3bab728e5b39 P 0788dbdabe574355bb323dcdc3be0058 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb4300ea183421ab40709a05de5a877" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 33597 } } peers { permanent_uuid: "0788dbdabe574355bb323dcdc3be0058" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 41335 } } peers { permanent_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34235 } }
I20260709 13:24:31.115252 27657 catalog_manager.cc:5697] T 032f598cb30343c2a1ca3bab728e5b39 P 0788dbdabe574355bb323dcdc3be0058 reported cstate change: term changed from 1 to 2, leader changed from 9cb4300ea183421ab40709a05de5a877 (127.14.177.129) to 0788dbdabe574355bb323dcdc3be0058 (127.14.177.130). New cstate: current_term: 2 leader_uuid: "0788dbdabe574355bb323dcdc3be0058" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb4300ea183421ab40709a05de5a877" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 33597 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0788dbdabe574355bb323dcdc3be0058" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 41335 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34235 } health_report { overall_health: UNKNOWN } } }
I20260709 13:24:31.194931 27836 raft_consensus.cc:1275] T 8e6b55496c134353a770e2642fd7490d P 0788dbdabe574355bb323dcdc3be0058 [term 2 FOLLOWER]: Refusing update from remote peer 9cb4300ea183421ab40709a05de5a877: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:24:31.196475 28076 consensus_queue.cc:1048] T 8e6b55496c134353a770e2642fd7490d P 9cb4300ea183421ab40709a05de5a877 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0788dbdabe574355bb323dcdc3be0058" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 41335 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:24:31.200902 27911 raft_consensus.cc:1275] T 8e6b55496c134353a770e2642fd7490d P 880fc4c7dff4417187b75cc1d9e9dcf5 [term 2 FOLLOWER]: Refusing update from remote peer 9cb4300ea183421ab40709a05de5a877: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:24:31.203207 28076 consensus_queue.cc:1048] T 8e6b55496c134353a770e2642fd7490d P 9cb4300ea183421ab40709a05de5a877 [LEADER]: Connected to new peer: Peer: permanent_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34235 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:24:31.207486 27910 raft_consensus.cc:1275] T d747d5acdf4f41b69c132cde690e49ff P 880fc4c7dff4417187b75cc1d9e9dcf5 [term 2 FOLLOWER]: Refusing update from remote peer 0788dbdabe574355bb323dcdc3be0058: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:24:31.211058 27955 consensus_queue.cc:1048] T d747d5acdf4f41b69c132cde690e49ff P 0788dbdabe574355bb323dcdc3be0058 [LEADER]: Connected to new peer: Peer: permanent_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34235 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:24:31.231588 27911 raft_consensus.cc:1275] T 660aa4903c494ba1bce988191700e129 P 880fc4c7dff4417187b75cc1d9e9dcf5 [term 2 FOLLOWER]: Refusing update from remote peer 0788dbdabe574355bb323dcdc3be0058: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:24:31.234485 28133 consensus_queue.cc:1048] T 660aa4903c494ba1bce988191700e129 P 0788dbdabe574355bb323dcdc3be0058 [LEADER]: Connected to new peer: Peer: permanent_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34235 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:24:31.239765 27760 raft_consensus.cc:1275] T d747d5acdf4f41b69c132cde690e49ff P 9cb4300ea183421ab40709a05de5a877 [term 2 FOLLOWER]: Refusing update from remote peer 0788dbdabe574355bb323dcdc3be0058: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:24:31.243013 28133 consensus_queue.cc:1048] T d747d5acdf4f41b69c132cde690e49ff P 0788dbdabe574355bb323dcdc3be0058 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9cb4300ea183421ab40709a05de5a877" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 33597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:24:31.246655 27760 raft_consensus.cc:1275] T 660aa4903c494ba1bce988191700e129 P 9cb4300ea183421ab40709a05de5a877 [term 2 FOLLOWER]: Refusing update from remote peer 0788dbdabe574355bb323dcdc3be0058: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:24:31.248605 28133 consensus_queue.cc:1048] T 660aa4903c494ba1bce988191700e129 P 0788dbdabe574355bb323dcdc3be0058 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9cb4300ea183421ab40709a05de5a877" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 33597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:24:31.359951 27759 raft_consensus.cc:1275] T 96f830fa400c473881739b3a953f0178 P 9cb4300ea183421ab40709a05de5a877 [term 2 FOLLOWER]: Refusing update from remote peer 0788dbdabe574355bb323dcdc3be0058: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:24:31.361729 28133 consensus_queue.cc:1048] T 96f830fa400c473881739b3a953f0178 P 0788dbdabe574355bb323dcdc3be0058 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9cb4300ea183421ab40709a05de5a877" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 33597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:24:31.371829 27910 raft_consensus.cc:1275] T 96f830fa400c473881739b3a953f0178 P 880fc4c7dff4417187b75cc1d9e9dcf5 [term 2 FOLLOWER]: Refusing update from remote peer 0788dbdabe574355bb323dcdc3be0058: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:24:31.374732 27955 consensus_queue.cc:1048] T 96f830fa400c473881739b3a953f0178 P 0788dbdabe574355bb323dcdc3be0058 [LEADER]: Connected to new peer: Peer: permanent_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34235 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:24:31.433132 27836 raft_consensus.cc:1275] T d7fd85f92c0a481aa54401371638252b P 0788dbdabe574355bb323dcdc3be0058 [term 2 FOLLOWER]: Refusing update from remote peer 9cb4300ea183421ab40709a05de5a877: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:24:31.434580 28076 consensus_queue.cc:1048] T d7fd85f92c0a481aa54401371638252b P 9cb4300ea183421ab40709a05de5a877 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0788dbdabe574355bb323dcdc3be0058" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 41335 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:24:31.442917 28133 raft_consensus.cc:493] T 27629b85cc3e4cb6b837dd26e3ebff47 P 0788dbdabe574355bb323dcdc3be0058 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:24:31.443550 28133 raft_consensus.cc:515] T 27629b85cc3e4cb6b837dd26e3ebff47 P 0788dbdabe574355bb323dcdc3be0058 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb4300ea183421ab40709a05de5a877" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 33597 } } peers { permanent_uuid: "0788dbdabe574355bb323dcdc3be0058" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 41335 } } peers { permanent_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34235 } }
I20260709 13:24:31.447378 27911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27629b85cc3e4cb6b837dd26e3ebff47" candidate_uuid: "0788dbdabe574355bb323dcdc3be0058" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" is_pre_election: true
I20260709 13:24:31.449025 27909 raft_consensus.cc:1275] T d7fd85f92c0a481aa54401371638252b P 880fc4c7dff4417187b75cc1d9e9dcf5 [term 2 FOLLOWER]: Refusing update from remote peer 9cb4300ea183421ab40709a05de5a877: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:24:31.449714 28133 leader_election.cc:290] T 27629b85cc3e4cb6b837dd26e3ebff47 P 0788dbdabe574355bb323dcdc3be0058 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9cb4300ea183421ab40709a05de5a877 (127.14.177.129:33597), 880fc4c7dff4417187b75cc1d9e9dcf5 (127.14.177.131:34235)
I20260709 13:24:31.451269 28095 consensus_queue.cc:1048] T d7fd85f92c0a481aa54401371638252b P 9cb4300ea183421ab40709a05de5a877 [LEADER]: Connected to new peer: Peer: permanent_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34235 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:24:31.451587 27759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27629b85cc3e4cb6b837dd26e3ebff47" candidate_uuid: "0788dbdabe574355bb323dcdc3be0058" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9cb4300ea183421ab40709a05de5a877" is_pre_election: true
I20260709 13:24:31.453972 27800 leader_election.cc:304] T 27629b85cc3e4cb6b837dd26e3ebff47 P 0788dbdabe574355bb323dcdc3be0058 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0788dbdabe574355bb323dcdc3be0058; no voters: 880fc4c7dff4417187b75cc1d9e9dcf5, 9cb4300ea183421ab40709a05de5a877
I20260709 13:24:31.454730 28133 raft_consensus.cc:2749] T 27629b85cc3e4cb6b837dd26e3ebff47 P 0788dbdabe574355bb323dcdc3be0058 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:24:31.516323 28082 raft_consensus.cc:493] T 27629b85cc3e4cb6b837dd26e3ebff47 P 880fc4c7dff4417187b75cc1d9e9dcf5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0788dbdabe574355bb323dcdc3be0058)
I20260709 13:24:31.517033 28082 raft_consensus.cc:515] T 27629b85cc3e4cb6b837dd26e3ebff47 P 880fc4c7dff4417187b75cc1d9e9dcf5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb4300ea183421ab40709a05de5a877" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 33597 } } peers { permanent_uuid: "0788dbdabe574355bb323dcdc3be0058" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 41335 } } peers { permanent_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34235 } }
I20260709 13:24:31.518814 28082 leader_election.cc:290] T 27629b85cc3e4cb6b837dd26e3ebff47 P 880fc4c7dff4417187b75cc1d9e9dcf5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9cb4300ea183421ab40709a05de5a877 (127.14.177.129:33597), 0788dbdabe574355bb323dcdc3be0058 (127.14.177.130:41335)
I20260709 13:24:31.519840 27759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27629b85cc3e4cb6b837dd26e3ebff47" candidate_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9cb4300ea183421ab40709a05de5a877" is_pre_election: true
I20260709 13:24:31.520066 27836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27629b85cc3e4cb6b837dd26e3ebff47" candidate_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0788dbdabe574355bb323dcdc3be0058" is_pre_election: true
I20260709 13:24:31.520732 27759 raft_consensus.cc:2393] T 27629b85cc3e4cb6b837dd26e3ebff47 P 9cb4300ea183421ab40709a05de5a877 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 880fc4c7dff4417187b75cc1d9e9dcf5 in current term 2: Already voted for candidate 9cb4300ea183421ab40709a05de5a877 in this term.
I20260709 13:24:31.520702 27836 raft_consensus.cc:2468] T 27629b85cc3e4cb6b837dd26e3ebff47 P 0788dbdabe574355bb323dcdc3be0058 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 880fc4c7dff4417187b75cc1d9e9dcf5 in term 2.
I20260709 13:24:31.521796 27876 leader_election.cc:304] T 27629b85cc3e4cb6b837dd26e3ebff47 P 880fc4c7dff4417187b75cc1d9e9dcf5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0788dbdabe574355bb323dcdc3be0058, 880fc4c7dff4417187b75cc1d9e9dcf5; no voters: 
I20260709 13:24:31.522550 28082 raft_consensus.cc:2804] T 27629b85cc3e4cb6b837dd26e3ebff47 P 880fc4c7dff4417187b75cc1d9e9dcf5 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:24:31.522961 28082 raft_consensus.cc:493] T 27629b85cc3e4cb6b837dd26e3ebff47 P 880fc4c7dff4417187b75cc1d9e9dcf5 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 0788dbdabe574355bb323dcdc3be0058)
I20260709 13:24:31.523313 28082 raft_consensus.cc:3060] T 27629b85cc3e4cb6b837dd26e3ebff47 P 880fc4c7dff4417187b75cc1d9e9dcf5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:24:31.531730 28082 raft_consensus.cc:515] T 27629b85cc3e4cb6b837dd26e3ebff47 P 880fc4c7dff4417187b75cc1d9e9dcf5 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb4300ea183421ab40709a05de5a877" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 33597 } } peers { permanent_uuid: "0788dbdabe574355bb323dcdc3be0058" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 41335 } } peers { permanent_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34235 } }
I20260709 13:24:31.533519 28082 leader_election.cc:290] T 27629b85cc3e4cb6b837dd26e3ebff47 P 880fc4c7dff4417187b75cc1d9e9dcf5 [CANDIDATE]: Term 2 election: Requested vote from peers 9cb4300ea183421ab40709a05de5a877 (127.14.177.129:33597), 0788dbdabe574355bb323dcdc3be0058 (127.14.177.130:41335)
I20260709 13:24:31.534366 27759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27629b85cc3e4cb6b837dd26e3ebff47" candidate_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9cb4300ea183421ab40709a05de5a877"
I20260709 13:24:31.534543 27836 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27629b85cc3e4cb6b837dd26e3ebff47" candidate_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0788dbdabe574355bb323dcdc3be0058"
I20260709 13:24:31.535315 27759 raft_consensus.cc:2393] T 27629b85cc3e4cb6b837dd26e3ebff47 P 9cb4300ea183421ab40709a05de5a877 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 880fc4c7dff4417187b75cc1d9e9dcf5 in current term 2: Already voted for candidate 9cb4300ea183421ab40709a05de5a877 in this term.
I20260709 13:24:31.540110 27836 raft_consensus.cc:2468] T 27629b85cc3e4cb6b837dd26e3ebff47 P 0788dbdabe574355bb323dcdc3be0058 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 880fc4c7dff4417187b75cc1d9e9dcf5 in term 2.
I20260709 13:24:31.541321 27876 leader_election.cc:304] T 27629b85cc3e4cb6b837dd26e3ebff47 P 880fc4c7dff4417187b75cc1d9e9dcf5 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0788dbdabe574355bb323dcdc3be0058, 880fc4c7dff4417187b75cc1d9e9dcf5; no voters: 9cb4300ea183421ab40709a05de5a877
I20260709 13:24:31.542241 28082 raft_consensus.cc:2804] T 27629b85cc3e4cb6b837dd26e3ebff47 P 880fc4c7dff4417187b75cc1d9e9dcf5 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:24:31.542721 28082 raft_consensus.cc:697] T 27629b85cc3e4cb6b837dd26e3ebff47 P 880fc4c7dff4417187b75cc1d9e9dcf5 [term 2 LEADER]: Becoming Leader. State: Replica: 880fc4c7dff4417187b75cc1d9e9dcf5, State: Running, Role: LEADER
I20260709 13:24:31.543639 28082 consensus_queue.cc:237] T 27629b85cc3e4cb6b837dd26e3ebff47 P 880fc4c7dff4417187b75cc1d9e9dcf5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb4300ea183421ab40709a05de5a877" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 33597 } } peers { permanent_uuid: "0788dbdabe574355bb323dcdc3be0058" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 41335 } } peers { permanent_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34235 } }
I20260709 13:24:31.551928 27660 catalog_manager.cc:5697] T 27629b85cc3e4cb6b837dd26e3ebff47 P 880fc4c7dff4417187b75cc1d9e9dcf5 reported cstate change: term changed from 1 to 2, leader changed from 0788dbdabe574355bb323dcdc3be0058 (127.14.177.130) to 880fc4c7dff4417187b75cc1d9e9dcf5 (127.14.177.131). New cstate: current_term: 2 leader_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb4300ea183421ab40709a05de5a877" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 33597 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0788dbdabe574355bb323dcdc3be0058" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 41335 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34235 } health_report { overall_health: HEALTHY } } }
I20260709 13:24:31.694512 27909 raft_consensus.cc:1275] T 032f598cb30343c2a1ca3bab728e5b39 P 880fc4c7dff4417187b75cc1d9e9dcf5 [term 2 FOLLOWER]: Refusing update from remote peer 0788dbdabe574355bb323dcdc3be0058: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:24:31.694571 27759 raft_consensus.cc:1275] T 032f598cb30343c2a1ca3bab728e5b39 P 9cb4300ea183421ab40709a05de5a877 [term 2 FOLLOWER]: Refusing update from remote peer 0788dbdabe574355bb323dcdc3be0058: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:24:31.696087 28133 consensus_queue.cc:1048] T 032f598cb30343c2a1ca3bab728e5b39 P 0788dbdabe574355bb323dcdc3be0058 [LEADER]: Connected to new peer: Peer: permanent_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34235 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:24:31.696732 27955 consensus_queue.cc:1048] T 032f598cb30343c2a1ca3bab728e5b39 P 0788dbdabe574355bb323dcdc3be0058 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9cb4300ea183421ab40709a05de5a877" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 33597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:24:32.092624 27836 raft_consensus.cc:1275] T 27629b85cc3e4cb6b837dd26e3ebff47 P 0788dbdabe574355bb323dcdc3be0058 [term 2 FOLLOWER]: Refusing update from remote peer 880fc4c7dff4417187b75cc1d9e9dcf5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:24:32.093791 28082 consensus_queue.cc:1048] T 27629b85cc3e4cb6b837dd26e3ebff47 P 880fc4c7dff4417187b75cc1d9e9dcf5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0788dbdabe574355bb323dcdc3be0058" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 41335 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:24:32.095717 28076 raft_consensus.cc:493] T 27629b85cc3e4cb6b837dd26e3ebff47 P 9cb4300ea183421ab40709a05de5a877 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:24:32.096292 28076 raft_consensus.cc:515] T 27629b85cc3e4cb6b837dd26e3ebff47 P 9cb4300ea183421ab40709a05de5a877 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb4300ea183421ab40709a05de5a877" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 33597 } } peers { permanent_uuid: "0788dbdabe574355bb323dcdc3be0058" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 41335 } } peers { permanent_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34235 } }
I20260709 13:24:32.098574 28076 leader_election.cc:290] T 27629b85cc3e4cb6b837dd26e3ebff47 P 9cb4300ea183421ab40709a05de5a877 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0788dbdabe574355bb323dcdc3be0058 (127.14.177.130:41335), 880fc4c7dff4417187b75cc1d9e9dcf5 (127.14.177.131:34235)
I20260709 13:24:32.099691 27837 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27629b85cc3e4cb6b837dd26e3ebff47" candidate_uuid: "9cb4300ea183421ab40709a05de5a877" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0788dbdabe574355bb323dcdc3be0058" is_pre_election: true
I20260709 13:24:32.100050 27909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27629b85cc3e4cb6b837dd26e3ebff47" candidate_uuid: "9cb4300ea183421ab40709a05de5a877" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" is_pre_election: true
I20260709 13:24:32.102540 27727 leader_election.cc:304] T 27629b85cc3e4cb6b837dd26e3ebff47 P 9cb4300ea183421ab40709a05de5a877 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9cb4300ea183421ab40709a05de5a877; no voters: 0788dbdabe574355bb323dcdc3be0058, 880fc4c7dff4417187b75cc1d9e9dcf5
I20260709 13:24:32.105010 27759 raft_consensus.cc:1275] T 27629b85cc3e4cb6b837dd26e3ebff47 P 9cb4300ea183421ab40709a05de5a877 [term 2 FOLLOWER]: Refusing update from remote peer 880fc4c7dff4417187b75cc1d9e9dcf5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:24:32.106528 28143 raft_consensus.cc:2749] T 27629b85cc3e4cb6b837dd26e3ebff47 P 9cb4300ea183421ab40709a05de5a877 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:24:32.106738 28082 consensus_queue.cc:1048] T 27629b85cc3e4cb6b837dd26e3ebff47 P 880fc4c7dff4417187b75cc1d9e9dcf5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9cb4300ea183421ab40709a05de5a877" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 33597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:24:33.599607 27660 catalog_manager.cc:2533] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60506:
table { table_name: "soft_deleted_table" } modify_external_catalogs: true reserve_seconds: 3600
I20260709 13:24:33.620931 27660 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60506:
table { table_name: "soft_deleted_table" } new_extra_configs { key: "kudu.table.disable_compaction" value: "true" }
I20260709 13:24:33.648460 15046 auto_leader_rebalancer.cc:141] leader rebalance for table soft_deleted_table
W20260709 13:24:33.852350 28167 log.cc:927] Time spent T 96f830fa400c473881739b3a953f0178 P 0788dbdabe574355bb323dcdc3be0058: Append to log took a long time: real 0.064s	user 0.000s	sys 0.001s
W20260709 13:24:33.873492 27802 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:34235, user_credentials={real_user=slave}} blocked reactor thread for 82810.3us
I20260709 13:24:33.915202 28191 tablet.cc:1724] T 8e6b55496c134353a770e2642fd7490d P 9cb4300ea183421ab40709a05de5a877: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:24:33.983551 28194 tablet.cc:1724] T 8e6b55496c134353a770e2642fd7490d P 0788dbdabe574355bb323dcdc3be0058: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 13:24:34.011826 28189 log.cc:927] Time spent T 1af49a625b20491586dcea4961e46734 P 9cb4300ea183421ab40709a05de5a877: Append to log took a long time: real 0.081s	user 0.004s	sys 0.000s
W20260709 13:24:34.017115 27727 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.130:41335, user_credentials={real_user=slave}} blocked reactor thread for 47295.7us
W20260709 13:24:34.082834 27788 tablet_mm_ops.cc:139] Rowset compaction is disabled (check --enable_rowset_compaction and disable_compaction in extra_config for tablet:8e6b55496c134353a770e2642fd7490d)
W20260709 13:24:34.100021 28206 log.cc:927] Time spent T 96f830fa400c473881739b3a953f0178 P 9cb4300ea183421ab40709a05de5a877: Append to log took a long time: real 0.071s	user 0.000s	sys 0.001s
I20260709 13:24:34.094123 27834 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "032f598cb30343c2a1ca3bab728e5b39"
dest_uuid: "0788dbdabe574355bb323dcdc3be0058"
mode: GRACEFUL
new_leader_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5"
 from {username='slave'} at 127.0.0.1:39584
I20260709 13:24:34.139374 28213 tablet.cc:1724] T 27629b85cc3e4cb6b837dd26e3ebff47 P 9cb4300ea183421ab40709a05de5a877: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:24:34.141206 27834 raft_consensus.cc:606] T 032f598cb30343c2a1ca3bab728e5b39 P 0788dbdabe574355bb323dcdc3be0058 [term 2 LEADER]: Received request to transfer leadership to TS 880fc4c7dff4417187b75cc1d9e9dcf5
I20260709 13:24:34.093560 28211 tablet.cc:1724] T 660aa4903c494ba1bce988191700e129 P 0788dbdabe574355bb323dcdc3be0058: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:24:34.143245 28217 tablet.cc:1724] T 1af49a625b20491586dcea4961e46734 P 880fc4c7dff4417187b75cc1d9e9dcf5: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:24:34.031519 28204 tablet.cc:1724] T 8e6b55496c134353a770e2642fd7490d P 880fc4c7dff4417187b75cc1d9e9dcf5: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:24:34.111543 28209 tablet.cc:1724] T 27629b85cc3e4cb6b837dd26e3ebff47 P 880fc4c7dff4417187b75cc1d9e9dcf5: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:24:34.160503 28214 tablet.cc:1724] T d747d5acdf4f41b69c132cde690e49ff P 0788dbdabe574355bb323dcdc3be0058: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:24:34.153842 28210 tablet.cc:1724] T d7fd85f92c0a481aa54401371638252b P 9cb4300ea183421ab40709a05de5a877: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:24:34.167388 28224 tablet.cc:1724] T d747d5acdf4f41b69c132cde690e49ff P 9cb4300ea183421ab40709a05de5a877: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 13:24:34.248266 27788 tablet_mm_ops.cc:220] Minor delta compaction is disabled (check --enable_minor_delta_compaction and disable_compaction in extra_config for tablet:27629b85cc3e4cb6b837dd26e3ebff47)
W20260709 13:24:34.250214 27863 tablet_mm_ops.cc:304] Major delta compaction is disabled (check --enable_major_delta_compaction and disable_compaction in extra_config for tablet:660aa4903c494ba1bce988191700e129)
W20260709 13:24:34.248734 27802 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.131:34235, user_credentials={real_user=slave}} blocked reactor thread for 54010.6us
I20260709 13:24:34.257334 28225 tablet.cc:1724] T 660aa4903c494ba1bce988191700e129 P 880fc4c7dff4417187b75cc1d9e9dcf5: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:24:34.257160 28211 tablet.cc:1724] T 04c481ece2f94a1b8cb6337900da4862 P 0788dbdabe574355bb323dcdc3be0058: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:24:34.269302 28175 raft_consensus.cc:993] T 032f598cb30343c2a1ca3bab728e5b39 P 0788dbdabe574355bb323dcdc3be0058: : Instructing follower 880fc4c7dff4417187b75cc1d9e9dcf5 to start an election
I20260709 13:24:34.271386 28177 raft_consensus.cc:1081] T 032f598cb30343c2a1ca3bab728e5b39 P 0788dbdabe574355bb323dcdc3be0058 [term 2 LEADER]: Signalling peer 880fc4c7dff4417187b75cc1d9e9dcf5 to start an election
I20260709 13:24:34.272969 15046 auto_leader_rebalancer.cc:452] table: soft_deleted_table, leader rebalance finish, leader transfer count: 1
I20260709 13:24:34.273627 15046 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:24:34.271530 28213 tablet.cc:1724] T 660aa4903c494ba1bce988191700e129 P 9cb4300ea183421ab40709a05de5a877: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:24:34.291676 28220 tablet.cc:1724] T d7fd85f92c0a481aa54401371638252b P 0788dbdabe574355bb323dcdc3be0058: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:24:34.294392 28214 tablet.cc:1724] T 27629b85cc3e4cb6b837dd26e3ebff47 P 0788dbdabe574355bb323dcdc3be0058: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:24:34.301976 28210 tablet.cc:1724] T 96f830fa400c473881739b3a953f0178 P 9cb4300ea183421ab40709a05de5a877: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:24:34.311134 27911 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "032f598cb30343c2a1ca3bab728e5b39"
dest_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5"
 from {username='slave'} at 127.0.0.1:58328
I20260709 13:24:34.311789 27911 raft_consensus.cc:493] T 032f598cb30343c2a1ca3bab728e5b39 P 880fc4c7dff4417187b75cc1d9e9dcf5 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:24:34.312160 27911 raft_consensus.cc:3060] T 032f598cb30343c2a1ca3bab728e5b39 P 880fc4c7dff4417187b75cc1d9e9dcf5 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:24:34.290589 28217 tablet.cc:1724] T d747d5acdf4f41b69c132cde690e49ff P 880fc4c7dff4417187b75cc1d9e9dcf5: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:24:34.321056 27834 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e7366c36a1154d4d94281a90fde291f8"
dest_uuid: "0788dbdabe574355bb323dcdc3be0058"
mode: GRACEFUL
new_leader_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5"
 from {username='slave'} at 127.0.0.1:39584
I20260709 13:24:34.321771 27834 raft_consensus.cc:606] T e7366c36a1154d4d94281a90fde291f8 P 0788dbdabe574355bb323dcdc3be0058 [term 2 LEADER]: Received request to transfer leadership to TS 880fc4c7dff4417187b75cc1d9e9dcf5
I20260709 13:24:34.321458 28204 tablet.cc:1724] T 96f830fa400c473881739b3a953f0178 P 880fc4c7dff4417187b75cc1d9e9dcf5: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:24:34.324143 27911 raft_consensus.cc:515] T 032f598cb30343c2a1ca3bab728e5b39 P 880fc4c7dff4417187b75cc1d9e9dcf5 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb4300ea183421ab40709a05de5a877" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 33597 } } peers { permanent_uuid: "0788dbdabe574355bb323dcdc3be0058" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 41335 } } peers { permanent_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34235 } }
I20260709 13:24:34.325865 27905 raft_consensus.cc:1240] T 032f598cb30343c2a1ca3bab728e5b39 P 880fc4c7dff4417187b75cc1d9e9dcf5 [term 3 FOLLOWER]: Rejecting Update request from peer 0788dbdabe574355bb323dcdc3be0058 for earlier term 2. Current term is 3. Ops: []
I20260709 13:24:34.265031 28191 tablet.cc:1724] T 04c481ece2f94a1b8cb6337900da4862 P 9cb4300ea183421ab40709a05de5a877: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:24:34.318625 28211 tablet.cc:1724] T 1af49a625b20491586dcea4961e46734 P 0788dbdabe574355bb323dcdc3be0058: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:24:34.330080 28193 consensus_queue.cc:1059] T 032f598cb30343c2a1ca3bab728e5b39 P 0788dbdabe574355bb323dcdc3be0058 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34235 }, Status: INVALID_TERM, Last received: 2.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:24:34.331368 28185 raft_consensus.cc:3055] T 032f598cb30343c2a1ca3bab728e5b39 P 0788dbdabe574355bb323dcdc3be0058 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:24:34.294394 28224 tablet.cc:1724] T 1af49a625b20491586dcea4961e46734 P 9cb4300ea183421ab40709a05de5a877: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:24:34.331758 28185 raft_consensus.cc:740] T 032f598cb30343c2a1ca3bab728e5b39 P 0788dbdabe574355bb323dcdc3be0058 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 0788dbdabe574355bb323dcdc3be0058, State: Running, Role: LEADER
I20260709 13:24:34.305480 28194 tablet.cc:1724] T 96f830fa400c473881739b3a953f0178 P 0788dbdabe574355bb323dcdc3be0058: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:24:34.335737 28185 consensus_queue.cc:260] T 032f598cb30343c2a1ca3bab728e5b39 P 0788dbdabe574355bb323dcdc3be0058 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb4300ea183421ab40709a05de5a877" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 33597 } } peers { permanent_uuid: "0788dbdabe574355bb323dcdc3be0058" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 41335 } } peers { permanent_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34235 } }
I20260709 13:24:34.350119 28209 tablet.cc:1724] T 04c481ece2f94a1b8cb6337900da4862 P 880fc4c7dff4417187b75cc1d9e9dcf5: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:24:34.352505 27911 leader_election.cc:290] T 032f598cb30343c2a1ca3bab728e5b39 P 880fc4c7dff4417187b75cc1d9e9dcf5 [CANDIDATE]: Term 3 election: Requested vote from peers 9cb4300ea183421ab40709a05de5a877 (127.14.177.129:33597), 0788dbdabe574355bb323dcdc3be0058 (127.14.177.130:41335)
I20260709 13:24:34.352905 28185 raft_consensus.cc:3060] T 032f598cb30343c2a1ca3bab728e5b39 P 0788dbdabe574355bb323dcdc3be0058 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:24:34.353744 27834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "032f598cb30343c2a1ca3bab728e5b39" candidate_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: true dest_uuid: "0788dbdabe574355bb323dcdc3be0058"
I20260709 13:24:34.365307 28220 tablet.cc:1724] T 032f598cb30343c2a1ca3bab728e5b39 P 0788dbdabe574355bb323dcdc3be0058: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:24:34.378341 28213 tablet.cc:1724] T 032f598cb30343c2a1ca3bab728e5b39 P 9cb4300ea183421ab40709a05de5a877: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:24:34.400364 28217 tablet.cc:1724] T d7fd85f92c0a481aa54401371638252b P 880fc4c7dff4417187b75cc1d9e9dcf5: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:24:34.405925 27758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "032f598cb30343c2a1ca3bab728e5b39" candidate_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: true dest_uuid: "9cb4300ea183421ab40709a05de5a877"
I20260709 13:24:34.406663 27758 raft_consensus.cc:3060] T 032f598cb30343c2a1ca3bab728e5b39 P 9cb4300ea183421ab40709a05de5a877 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:24:34.422752 27834 raft_consensus.cc:2468] T 032f598cb30343c2a1ca3bab728e5b39 P 0788dbdabe574355bb323dcdc3be0058 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 880fc4c7dff4417187b75cc1d9e9dcf5 in term 3.
I20260709 13:24:34.427930 27876 leader_election.cc:304] T 032f598cb30343c2a1ca3bab728e5b39 P 880fc4c7dff4417187b75cc1d9e9dcf5 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0788dbdabe574355bb323dcdc3be0058, 880fc4c7dff4417187b75cc1d9e9dcf5; no voters: 
I20260709 13:24:34.428871 28082 raft_consensus.cc:2804] T 032f598cb30343c2a1ca3bab728e5b39 P 880fc4c7dff4417187b75cc1d9e9dcf5 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:24:34.429402 28082 raft_consensus.cc:697] T 032f598cb30343c2a1ca3bab728e5b39 P 880fc4c7dff4417187b75cc1d9e9dcf5 [term 3 LEADER]: Becoming Leader. State: Replica: 880fc4c7dff4417187b75cc1d9e9dcf5, State: Running, Role: LEADER
I20260709 13:24:34.430094 27761 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f3e47f5448ff47a382c61886f1a8cc7b"
dest_uuid: "9cb4300ea183421ab40709a05de5a877"
mode: GRACEFUL
new_leader_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5"
 from {username='slave'} at 127.0.0.1:43838
I20260709 13:24:34.430317 28082 consensus_queue.cc:237] T 032f598cb30343c2a1ca3bab728e5b39 P 880fc4c7dff4417187b75cc1d9e9dcf5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.3, Last appended by leader: 3, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb4300ea183421ab40709a05de5a877" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 33597 } } peers { permanent_uuid: "0788dbdabe574355bb323dcdc3be0058" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 41335 } } peers { permanent_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34235 } }
I20260709 13:24:34.430910 27761 raft_consensus.cc:606] T f3e47f5448ff47a382c61886f1a8cc7b P 9cb4300ea183421ab40709a05de5a877 [term 2 LEADER]: Received request to transfer leadership to TS 880fc4c7dff4417187b75cc1d9e9dcf5
I20260709 13:24:34.433152 15046 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 13:24:34.433780 15046 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:24:34.439967 27758 raft_consensus.cc:2468] T 032f598cb30343c2a1ca3bab728e5b39 P 9cb4300ea183421ab40709a05de5a877 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 880fc4c7dff4417187b75cc1d9e9dcf5 in term 3.
I20260709 13:24:34.460119 27658 catalog_manager.cc:5697] T 032f598cb30343c2a1ca3bab728e5b39 P 880fc4c7dff4417187b75cc1d9e9dcf5 reported cstate change: term changed from 2 to 3, leader changed from 0788dbdabe574355bb323dcdc3be0058 (127.14.177.130) to 880fc4c7dff4417187b75cc1d9e9dcf5 (127.14.177.131). New cstate: current_term: 3 leader_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb4300ea183421ab40709a05de5a877" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 33597 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0788dbdabe574355bb323dcdc3be0058" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 41335 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34235 } health_report { overall_health: HEALTHY } } }
I20260709 13:24:34.461357 27658 catalog_manager.cc:5789] TS 880fc4c7dff4417187b75cc1d9e9dcf5 (127.14.177.131:34235) does not have the latest schema for tablet 032f598cb30343c2a1ca3bab728e5b39 (table soft_deleted_table [id=32a3558cca9046bb846711e7e3f090fc]). Expected version 1 got 0
I20260709 13:24:34.465273 27657 catalog_manager.cc:6242] T 00000000000000000000000000000000 P c3a531df1ce7417cbf6095f23465e39b: soft_deleted_table [id=32a3558cca9046bb846711e7e3f090fc] alter complete (version 1)
I20260709 13:24:34.767459 28193 raft_consensus.cc:993] T e7366c36a1154d4d94281a90fde291f8 P 0788dbdabe574355bb323dcdc3be0058: : Instructing follower 880fc4c7dff4417187b75cc1d9e9dcf5 to start an election
I20260709 13:24:34.767987 28185 raft_consensus.cc:1081] T e7366c36a1154d4d94281a90fde291f8 P 0788dbdabe574355bb323dcdc3be0058 [term 2 LEADER]: Signalling peer 880fc4c7dff4417187b75cc1d9e9dcf5 to start an election
I20260709 13:24:34.769775 27911 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e7366c36a1154d4d94281a90fde291f8"
dest_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5"
 from {username='slave'} at 127.0.0.1:58328
I20260709 13:24:34.770367 27911 raft_consensus.cc:493] T e7366c36a1154d4d94281a90fde291f8 P 880fc4c7dff4417187b75cc1d9e9dcf5 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:24:34.770743 27911 raft_consensus.cc:3060] T e7366c36a1154d4d94281a90fde291f8 P 880fc4c7dff4417187b75cc1d9e9dcf5 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:24:34.778579 27911 raft_consensus.cc:515] T e7366c36a1154d4d94281a90fde291f8 P 880fc4c7dff4417187b75cc1d9e9dcf5 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb4300ea183421ab40709a05de5a877" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 33597 } } peers { permanent_uuid: "0788dbdabe574355bb323dcdc3be0058" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 41335 } } peers { permanent_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34235 } }
I20260709 13:24:34.780606 27911 leader_election.cc:290] T e7366c36a1154d4d94281a90fde291f8 P 880fc4c7dff4417187b75cc1d9e9dcf5 [CANDIDATE]: Term 3 election: Requested vote from peers 9cb4300ea183421ab40709a05de5a877 (127.14.177.129:33597), 0788dbdabe574355bb323dcdc3be0058 (127.14.177.130:41335)
I20260709 13:24:34.781306 27761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7366c36a1154d4d94281a90fde291f8" candidate_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "9cb4300ea183421ab40709a05de5a877"
I20260709 13:24:34.781733 27834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7366c36a1154d4d94281a90fde291f8" candidate_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "0788dbdabe574355bb323dcdc3be0058"
I20260709 13:24:34.781982 27761 raft_consensus.cc:3060] T e7366c36a1154d4d94281a90fde291f8 P 9cb4300ea183421ab40709a05de5a877 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:24:34.782461 27834 raft_consensus.cc:3055] T e7366c36a1154d4d94281a90fde291f8 P 0788dbdabe574355bb323dcdc3be0058 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:24:34.782789 27834 raft_consensus.cc:740] T e7366c36a1154d4d94281a90fde291f8 P 0788dbdabe574355bb323dcdc3be0058 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 0788dbdabe574355bb323dcdc3be0058, State: Running, Role: LEADER
I20260709 13:24:34.783668 27834 consensus_queue.cc:260] T e7366c36a1154d4d94281a90fde291f8 P 0788dbdabe574355bb323dcdc3be0058 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb4300ea183421ab40709a05de5a877" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 33597 } } peers { permanent_uuid: "0788dbdabe574355bb323dcdc3be0058" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 41335 } } peers { permanent_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34235 } }
I20260709 13:24:34.785298 27834 raft_consensus.cc:3060] T e7366c36a1154d4d94281a90fde291f8 P 0788dbdabe574355bb323dcdc3be0058 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:24:34.789441 27761 raft_consensus.cc:2468] T e7366c36a1154d4d94281a90fde291f8 P 9cb4300ea183421ab40709a05de5a877 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 880fc4c7dff4417187b75cc1d9e9dcf5 in term 3.
I20260709 13:24:34.790867 27874 leader_election.cc:304] T e7366c36a1154d4d94281a90fde291f8 P 880fc4c7dff4417187b75cc1d9e9dcf5 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 880fc4c7dff4417187b75cc1d9e9dcf5, 9cb4300ea183421ab40709a05de5a877; no voters: 
I20260709 13:24:34.791759 28082 raft_consensus.cc:2804] T e7366c36a1154d4d94281a90fde291f8 P 880fc4c7dff4417187b75cc1d9e9dcf5 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:24:34.792223 28082 raft_consensus.cc:697] T e7366c36a1154d4d94281a90fde291f8 P 880fc4c7dff4417187b75cc1d9e9dcf5 [term 3 LEADER]: Becoming Leader. State: Replica: 880fc4c7dff4417187b75cc1d9e9dcf5, State: Running, Role: LEADER
I20260709 13:24:34.793009 28082 consensus_queue.cc:237] T e7366c36a1154d4d94281a90fde291f8 P 880fc4c7dff4417187b75cc1d9e9dcf5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb4300ea183421ab40709a05de5a877" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 33597 } } peers { permanent_uuid: "0788dbdabe574355bb323dcdc3be0058" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 41335 } } peers { permanent_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34235 } }
I20260709 13:24:34.793329 27834 raft_consensus.cc:2468] T e7366c36a1154d4d94281a90fde291f8 P 0788dbdabe574355bb323dcdc3be0058 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 880fc4c7dff4417187b75cc1d9e9dcf5 in term 3.
I20260709 13:24:34.807391 27657 catalog_manager.cc:5697] T e7366c36a1154d4d94281a90fde291f8 P 880fc4c7dff4417187b75cc1d9e9dcf5 reported cstate change: term changed from 2 to 3, leader changed from 0788dbdabe574355bb323dcdc3be0058 (127.14.177.130) to 880fc4c7dff4417187b75cc1d9e9dcf5 (127.14.177.131). New cstate: current_term: 3 leader_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb4300ea183421ab40709a05de5a877" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 33597 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0788dbdabe574355bb323dcdc3be0058" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 41335 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34235 } health_report { overall_health: HEALTHY } } }
I20260709 13:24:34.809549 27657 catalog_manager.cc:5789] TS 880fc4c7dff4417187b75cc1d9e9dcf5 (127.14.177.131:34235) does not have the latest schema for tablet 032f598cb30343c2a1ca3bab728e5b39 (table soft_deleted_table [id=32a3558cca9046bb846711e7e3f090fc]). Expected version 1 got 0
I20260709 13:24:34.846711 27834 raft_consensus.cc:1275] T 032f598cb30343c2a1ca3bab728e5b39 P 0788dbdabe574355bb323dcdc3be0058 [term 3 FOLLOWER]: Refusing update from remote peer 880fc4c7dff4417187b75cc1d9e9dcf5: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 13:24:34.848109 28082 consensus_queue.cc:1048] T 032f598cb30343c2a1ca3bab728e5b39 P 880fc4c7dff4417187b75cc1d9e9dcf5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0788dbdabe574355bb323dcdc3be0058" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 41335 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:24:34.854387 28217 tablet.cc:1724] T 032f598cb30343c2a1ca3bab728e5b39 P 880fc4c7dff4417187b75cc1d9e9dcf5: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:24:34.858045 27761 raft_consensus.cc:1275] T 032f598cb30343c2a1ca3bab728e5b39 P 9cb4300ea183421ab40709a05de5a877 [term 3 FOLLOWER]: Refusing update from remote peer 880fc4c7dff4417187b75cc1d9e9dcf5: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 13:24:34.865660 28231 consensus_queue.cc:1048] T 032f598cb30343c2a1ca3bab728e5b39 P 880fc4c7dff4417187b75cc1d9e9dcf5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9cb4300ea183421ab40709a05de5a877" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 33597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:24:34.887362 28217 tablet.cc:1719] T 032f598cb30343c2a1ca3bab728e5b39 P 880fc4c7dff4417187b75cc1d9e9dcf5: Skipping requested alter to schema version 1, tablet already version 1
I20260709 13:24:34.895403 28211 tablet.cc:1719] T 032f598cb30343c2a1ca3bab728e5b39 P 0788dbdabe574355bb323dcdc3be0058: Skipping requested alter to schema version 1, tablet already version 1
I20260709 13:24:34.907769 28213 tablet.cc:1719] T 032f598cb30343c2a1ca3bab728e5b39 P 9cb4300ea183421ab40709a05de5a877: Skipping requested alter to schema version 1, tablet already version 1
I20260709 13:24:34.935037 28095 raft_consensus.cc:993] T f3e47f5448ff47a382c61886f1a8cc7b P 9cb4300ea183421ab40709a05de5a877: : Instructing follower 880fc4c7dff4417187b75cc1d9e9dcf5 to start an election
I20260709 13:24:34.935525 28095 raft_consensus.cc:1081] T f3e47f5448ff47a382c61886f1a8cc7b P 9cb4300ea183421ab40709a05de5a877 [term 2 LEADER]: Signalling peer 880fc4c7dff4417187b75cc1d9e9dcf5 to start an election
I20260709 13:24:34.937188 27911 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "f3e47f5448ff47a382c61886f1a8cc7b"
dest_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5"
 from {username='slave'} at 127.0.0.1:58340
I20260709 13:24:34.937619 27911 raft_consensus.cc:493] T f3e47f5448ff47a382c61886f1a8cc7b P 880fc4c7dff4417187b75cc1d9e9dcf5 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:24:34.937850 27911 raft_consensus.cc:3060] T f3e47f5448ff47a382c61886f1a8cc7b P 880fc4c7dff4417187b75cc1d9e9dcf5 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:24:34.943364 27911 raft_consensus.cc:515] T f3e47f5448ff47a382c61886f1a8cc7b P 880fc4c7dff4417187b75cc1d9e9dcf5 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb4300ea183421ab40709a05de5a877" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 33597 } } peers { permanent_uuid: "0788dbdabe574355bb323dcdc3be0058" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 41335 } } peers { permanent_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34235 } }
I20260709 13:24:34.945065 27911 leader_election.cc:290] T f3e47f5448ff47a382c61886f1a8cc7b P 880fc4c7dff4417187b75cc1d9e9dcf5 [CANDIDATE]: Term 3 election: Requested vote from peers 9cb4300ea183421ab40709a05de5a877 (127.14.177.129:33597), 0788dbdabe574355bb323dcdc3be0058 (127.14.177.130:41335)
I20260709 13:24:34.945996 27761 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3e47f5448ff47a382c61886f1a8cc7b" candidate_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "9cb4300ea183421ab40709a05de5a877"
I20260709 13:24:34.946206 27834 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3e47f5448ff47a382c61886f1a8cc7b" candidate_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "0788dbdabe574355bb323dcdc3be0058"
I20260709 13:24:34.946633 27761 raft_consensus.cc:3055] T f3e47f5448ff47a382c61886f1a8cc7b P 9cb4300ea183421ab40709a05de5a877 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:24:34.946816 27834 raft_consensus.cc:3060] T f3e47f5448ff47a382c61886f1a8cc7b P 0788dbdabe574355bb323dcdc3be0058 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:24:34.946955 27761 raft_consensus.cc:740] T f3e47f5448ff47a382c61886f1a8cc7b P 9cb4300ea183421ab40709a05de5a877 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 9cb4300ea183421ab40709a05de5a877, State: Running, Role: LEADER
I20260709 13:24:34.947940 27761 consensus_queue.cc:260] T f3e47f5448ff47a382c61886f1a8cc7b P 9cb4300ea183421ab40709a05de5a877 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb4300ea183421ab40709a05de5a877" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 33597 } } peers { permanent_uuid: "0788dbdabe574355bb323dcdc3be0058" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 41335 } } peers { permanent_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34235 } }
I20260709 13:24:34.949039 27761 raft_consensus.cc:3060] T f3e47f5448ff47a382c61886f1a8cc7b P 9cb4300ea183421ab40709a05de5a877 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:24:34.952977 27834 raft_consensus.cc:2468] T f3e47f5448ff47a382c61886f1a8cc7b P 0788dbdabe574355bb323dcdc3be0058 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 880fc4c7dff4417187b75cc1d9e9dcf5 in term 3.
I20260709 13:24:34.954309 27876 leader_election.cc:304] T f3e47f5448ff47a382c61886f1a8cc7b P 880fc4c7dff4417187b75cc1d9e9dcf5 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0788dbdabe574355bb323dcdc3be0058, 880fc4c7dff4417187b75cc1d9e9dcf5; no voters: 
I20260709 13:24:34.955114 28231 raft_consensus.cc:2804] T f3e47f5448ff47a382c61886f1a8cc7b P 880fc4c7dff4417187b75cc1d9e9dcf5 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:24:34.955147 27761 raft_consensus.cc:2468] T f3e47f5448ff47a382c61886f1a8cc7b P 9cb4300ea183421ab40709a05de5a877 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 880fc4c7dff4417187b75cc1d9e9dcf5 in term 3.
I20260709 13:24:34.955680 28231 raft_consensus.cc:697] T f3e47f5448ff47a382c61886f1a8cc7b P 880fc4c7dff4417187b75cc1d9e9dcf5 [term 3 LEADER]: Becoming Leader. State: Replica: 880fc4c7dff4417187b75cc1d9e9dcf5, State: Running, Role: LEADER
I20260709 13:24:34.956638 28231 consensus_queue.cc:237] T f3e47f5448ff47a382c61886f1a8cc7b P 880fc4c7dff4417187b75cc1d9e9dcf5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb4300ea183421ab40709a05de5a877" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 33597 } } peers { permanent_uuid: "0788dbdabe574355bb323dcdc3be0058" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 41335 } } peers { permanent_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34235 } }
I20260709 13:24:34.966171 27658 catalog_manager.cc:5697] T f3e47f5448ff47a382c61886f1a8cc7b P 880fc4c7dff4417187b75cc1d9e9dcf5 reported cstate change: term changed from 2 to 3, leader changed from 9cb4300ea183421ab40709a05de5a877 (127.14.177.129) to 880fc4c7dff4417187b75cc1d9e9dcf5 (127.14.177.131). New cstate: current_term: 3 leader_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb4300ea183421ab40709a05de5a877" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 33597 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0788dbdabe574355bb323dcdc3be0058" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 41335 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "880fc4c7dff4417187b75cc1d9e9dcf5" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 34235 } health_report { overall_health: HEALTHY } } }
I20260709 13:24:35.309227 27834 raft_consensus.cc:1275] T e7366c36a1154d4d94281a90fde291f8 P 0788dbdabe574355bb323dcdc3be0058 [term 3 FOLLOWER]: Refusing update from remote peer 880fc4c7dff4417187b75cc1d9e9dcf5: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:24:35.310722 28231 consensus_queue.cc:1048] T e7366c36a1154d4d94281a90fde291f8 P 880fc4c7dff4417187b75cc1d9e9dcf5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0788dbdabe574355bb323dcdc3be0058" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 41335 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:24:35.319344 27761 raft_consensus.cc:1275] T e7366c36a1154d4d94281a90fde291f8 P 9cb4300ea183421ab40709a05de5a877 [term 3 FOLLOWER]: Refusing update from remote peer 880fc4c7dff4417187b75cc1d9e9dcf5: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:24:35.321077 28235 consensus_queue.cc:1048] T e7366c36a1154d4d94281a90fde291f8 P 880fc4c7dff4417187b75cc1d9e9dcf5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9cb4300ea183421ab40709a05de5a877" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 33597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:24:35.503144 15046 auto_leader_rebalancer.cc:141] leader rebalance for table soft_deleted_table
I20260709 13:24:35.509583 15046 auto_leader_rebalancer.cc:452] table: soft_deleted_table, leader rebalance finish, leader transfer count: 0
I20260709 13:24:35.510094 15046 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:24:35.516467 15046 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 13:24:35.516991 15046 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:24:35.523540 27834 raft_consensus.cc:1275] T f3e47f5448ff47a382c61886f1a8cc7b P 0788dbdabe574355bb323dcdc3be0058 [term 3 FOLLOWER]: Refusing update from remote peer 880fc4c7dff4417187b75cc1d9e9dcf5: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:24:35.525247 28235 consensus_queue.cc:1048] T f3e47f5448ff47a382c61886f1a8cc7b P 880fc4c7dff4417187b75cc1d9e9dcf5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0788dbdabe574355bb323dcdc3be0058" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 41335 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:24:35.540822 27761 raft_consensus.cc:1275] T f3e47f5448ff47a382c61886f1a8cc7b P 9cb4300ea183421ab40709a05de5a877 [term 3 FOLLOWER]: Refusing update from remote peer 880fc4c7dff4417187b75cc1d9e9dcf5: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:24:35.542982 28235 consensus_queue.cc:1048] T f3e47f5448ff47a382c61886f1a8cc7b P 880fc4c7dff4417187b75cc1d9e9dcf5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9cb4300ea183421ab40709a05de5a877" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 33597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:24:36.552589 15046 tablet_server.cc:179] TabletServer@127.14.177.129:0 shutting down...
I20260709 13:24:36.600988 15046 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:24:36.601902 15046 tablet_replica.cc:333] T 1af49a625b20491586dcea4961e46734 P 9cb4300ea183421ab40709a05de5a877: stopping tablet replica
I20260709 13:24:36.602934 15046 raft_consensus.cc:2243] T 1af49a625b20491586dcea4961e46734 P 9cb4300ea183421ab40709a05de5a877 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:24:36.603623 15046 raft_consensus.cc:2272] T 1af49a625b20491586dcea4961e46734 P 9cb4300ea183421ab40709a05de5a877 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:24:36.605784 15046 tablet_replica.cc:333] T 96f830fa400c473881739b3a953f0178 P 9cb4300ea183421ab40709a05de5a877: stopping tablet replica
I20260709 13:24:36.606559 15046 raft_consensus.cc:2243] T 96f830fa400c473881739b3a953f0178 P 9cb4300ea183421ab40709a05de5a877 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:24:36.607072 15046 raft_consensus.cc:2272] T 96f830fa400c473881739b3a953f0178 P 9cb4300ea183421ab40709a05de5a877 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:24:36.608990 15046 tablet_replica.cc:333] T 27629b85cc3e4cb6b837dd26e3ebff47 P 9cb4300ea183421ab40709a05de5a877: stopping tablet replica
I20260709 13:24:36.609634 15046 raft_consensus.cc:2243] T 27629b85cc3e4cb6b837dd26e3ebff47 P 9cb4300ea183421ab40709a05de5a877 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:24:36.610078 15046 raft_consensus.cc:2272] T 27629b85cc3e4cb6b837dd26e3ebff47 P 9cb4300ea183421ab40709a05de5a877 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:24:36.612356 15046 tablet_replica.cc:333] T 2f38443d4b8d4e418c8f0520893a31e4 P 9cb4300ea183421ab40709a05de5a877: stopping tablet replica
I20260709 13:24:36.612950 15046 raft_consensus.cc:2243] T 2f38443d4b8d4e418c8f0520893a31e4 P 9cb4300ea183421ab40709a05de5a877 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:24:36.613457 15046 raft_consensus.cc:2272] T 2f38443d4b8d4e418c8f0520893a31e4 P 9cb4300ea183421ab40709a05de5a877 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:24:36.616577 15046 tablet_replica.cc:333] T 032f598cb30343c2a1ca3bab728e5b39 P 9cb4300ea183421ab40709a05de5a877: stopping tablet replica
I20260709 13:24:36.617128 15046 raft_consensus.cc:2243] T 032f598cb30343c2a1ca3bab728e5b39 P 9cb4300ea183421ab40709a05de5a877 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:24:36.617576 15046 raft_consensus.cc:2272] T 032f598cb30343c2a1ca3bab728e5b39 P 9cb4300ea183421ab40709a05de5a877 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:24:36.619810 15046 tablet_replica.cc:333] T d7fd85f92c0a481aa54401371638252b P 9cb4300ea183421ab40709a05de5a877: stopping tablet replica
I20260709 13:24:36.620415 15046 raft_consensus.cc:2243] T d7fd85f92c0a481aa54401371638252b P 9cb4300ea183421ab40709a05de5a877 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:24:36.621198 15046 raft_consensus.cc:2272] T d7fd85f92c0a481aa54401371638252b P 9cb4300ea183421ab40709a05de5a877 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:24:36.623337 15046 tablet_replica.cc:333] T 4554587582094286b577c9c9a4d3a7fc P 9cb4300ea183421ab40709a05de5a877: stopping tablet replica
I20260709 13:24:36.623996 15046 raft_consensus.cc:2243] T 4554587582094286b577c9c9a4d3a7fc P 9cb4300ea183421ab40709a05de5a877 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:24:36.624498 15046 raft_consensus.cc:2272] T 4554587582094286b577c9c9a4d3a7fc P 9cb4300ea183421ab40709a05de5a877 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:24:36.626801 15046 tablet_replica.cc:333] T d747d5acdf4f41b69c132cde690e49ff P 9cb4300ea183421ab40709a05de5a877: stopping tablet replica
I20260709 13:24:36.627601 15046 raft_consensus.cc:2243] T d747d5acdf4f41b69c132cde690e49ff P 9cb4300ea183421ab40709a05de5a877 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:24:36.628145 15046 raft_consensus.cc:2272] T d747d5acdf4f41b69c132cde690e49ff P 9cb4300ea183421ab40709a05de5a877 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:24:36.630564 15046 tablet_replica.cc:333] T d16bbb66a57e4079897aaa2c4e6ba144 P 9cb4300ea183421ab40709a05de5a877: stopping tablet replica
I20260709 13:24:36.631260 15046 raft_consensus.cc:2243] T d16bbb66a57e4079897aaa2c4e6ba144 P 9cb4300ea183421ab40709a05de5a877 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:24:36.632107 15046 raft_consensus.cc:2272] T d16bbb66a57e4079897aaa2c4e6ba144 P 9cb4300ea183421ab40709a05de5a877 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:24:36.634478 15046 tablet_replica.cc:333] T 660aa4903c494ba1bce988191700e129 P 9cb4300ea183421ab40709a05de5a877: stopping tablet replica
I20260709 13:24:36.635272 15046 raft_consensus.cc:2243] T 660aa4903c494ba1bce988191700e129 P 9cb4300ea183421ab40709a05de5a877 [term 2 FOLLOWER]: Raft consensus shutting down.
W20260709 13:24:36.635808 27800 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:24:36.635828 15046 raft_consensus.cc:2272] T 660aa4903c494ba1bce988191700e129 P 9cb4300ea183421ab40709a05de5a877 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:24:36.638700 15046 tablet_replica.cc:333] T 028750ef0efa43e4b11ac10f3814902c P 9cb4300ea183421ab40709a05de5a877: stopping tablet replica
W20260709 13:24:36.639281 27800 consensus_peers.cc:597] T 96f830fa400c473881739b3a953f0178 P 0788dbdabe574355bb323dcdc3be0058 -> Peer 9cb4300ea183421ab40709a05de5a877 (127.14.177.129:33597): Couldn't send request to peer 9cb4300ea183421ab40709a05de5a877. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:24:36.639472 15046 raft_consensus.cc:2243] T 028750ef0efa43e4b11ac10f3814902c P 9cb4300ea183421ab40709a05de5a877 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:24:36.640117 15046 raft_consensus.cc:2272] T 028750ef0efa43e4b11ac10f3814902c P 9cb4300ea183421ab40709a05de5a877 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:24:36.642913 15046 tablet_replica.cc:333] T cd1140068fc8455280934d7a30c4f559 P 9cb4300ea183421ab40709a05de5a877: stopping tablet replica
I20260709 13:24:36.643560 15046 raft_consensus.cc:2243] T cd1140068fc8455280934d7a30c4f559 P 9cb4300ea183421ab40709a05de5a877 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:24:36.644155 15046 raft_consensus.cc:2272] T cd1140068fc8455280934d7a30c4f559 P 9cb4300ea183421ab40709a05de5a877 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:24:36.644778 27800 consensus_peers.cc:597] T cd1140068fc8455280934d7a30c4f559 P 0788dbdabe574355bb323dcdc3be0058 -> Peer 9cb4300ea183421ab40709a05de5a877 (127.14.177.129:33597): Couldn't send request to peer 9cb4300ea183421ab40709a05de5a877. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:24:36.646683 15046 tablet_replica.cc:333] T 04c481ece2f94a1b8cb6337900da4862 P 9cb4300ea183421ab40709a05de5a877: stopping tablet replica
I20260709 13:24:36.647396 15046 raft_consensus.cc:2243] T 04c481ece2f94a1b8cb6337900da4862 P 9cb4300ea183421ab40709a05de5a877 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:24:36.648150 15046 raft_consensus.cc:2272] T 04c481ece2f94a1b8cb6337900da4862 P 9cb4300ea183421ab40709a05de5a877 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:24:36.650250 15046 tablet_replica.cc:333] T 20ef9c03fcd94115be0b592d85f89421 P 9cb4300ea183421ab40709a05de5a877: stopping tablet replica
I20260709 13:24:36.650892 15046 raft_consensus.cc:2243] T 20ef9c03fcd94115be0b592d85f89421 P 9cb4300ea183421ab40709a05de5a877 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:24:36.651772 15046 raft_consensus.cc:2272] T 20ef9c03fcd94115be0b592d85f89421 P 9cb4300ea183421ab40709a05de5a877 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:24:36.654472 15046 tablet_replica.cc:333] T e7366c36a1154d4d94281a90fde291f8 P 9cb4300ea183421ab40709a05de5a877: stopping tablet replica
I20260709 13:24:36.655256 15046 raft_consensus.cc:2243] T e7366c36a1154d4d94281a90fde291f8 P 9cb4300ea183421ab40709a05de5a877 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:24:36.655897 15046 raft_consensus.cc:2272] T e7366c36a1154d4d94281a90fde291f8 P 9cb4300ea183421ab40709a05de5a877 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:24:36.658562 15046 tablet_replica.cc:333] T 8e6b55496c134353a770e2642fd7490d P 9cb4300ea183421ab40709a05de5a877: stopping tablet replica
I20260709 13:24:36.659209 15046 raft_consensus.cc:2243] T 8e6b55496c134353a770e2642fd7490d P 9cb4300ea183421ab40709a05de5a877 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:24:36.660148 15046 raft_consensus.cc:2272] T 8e6b55496c134353a770e2642fd7490d P 9cb4300ea183421ab40709a05de5a877 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:24:36.660683 27800 consensus_peers.cc:597] T 660aa4903c494ba1bce988191700e129 P 0788dbdabe574355bb323dcdc3be0058 -> Peer 9cb4300ea183421ab40709a05de5a877 (127.14.177.129:33597): Couldn't send request to peer 9cb4300ea183421ab40709a05de5a877. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:24:36.662504 15046 tablet_replica.cc:333] T f3e47f5448ff47a382c61886f1a8cc7b P 9cb4300ea183421ab40709a05de5a877: stopping tablet replica
I20260709 13:24:36.663435 15046 raft_consensus.cc:2243] T f3e47f5448ff47a382c61886f1a8cc7b P 9cb4300ea183421ab40709a05de5a877 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:24:36.664048 15046 raft_consensus.cc:2272] T f3e47f5448ff47a382c61886f1a8cc7b P 9cb4300ea183421ab40709a05de5a877 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:24:36.666280 15046 tablet_replica.cc:333] T db487682b03b4a738f664fa1459714e5 P 9cb4300ea183421ab40709a05de5a877: stopping tablet replica
I20260709 13:24:36.666795 15046 raft_consensus.cc:2243] T db487682b03b4a738f664fa1459714e5 P 9cb4300ea183421ab40709a05de5a877 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:24:36.667668 15046 raft_consensus.cc:2272] T db487682b03b4a738f664fa1459714e5 P 9cb4300ea183421ab40709a05de5a877 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:24:36.698872 15046 tablet_server.cc:196] TabletServer@127.14.177.129:0 shutdown complete.
W20260709 13:24:36.702055 27874 consensus_peers.cc:597] T 1af49a625b20491586dcea4961e46734 P 880fc4c7dff4417187b75cc1d9e9dcf5 -> Peer 9cb4300ea183421ab40709a05de5a877 (127.14.177.129:33597): Couldn't send request to peer 9cb4300ea183421ab40709a05de5a877. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:33597: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:24:36.720532 27874 consensus_peers.cc:597] T f3e47f5448ff47a382c61886f1a8cc7b P 880fc4c7dff4417187b75cc1d9e9dcf5 -> Peer 9cb4300ea183421ab40709a05de5a877 (127.14.177.129:33597): Couldn't send request to peer 9cb4300ea183421ab40709a05de5a877. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:33597: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:24:36.721091 27874 consensus_peers.cc:597] T 032f598cb30343c2a1ca3bab728e5b39 P 880fc4c7dff4417187b75cc1d9e9dcf5 -> Peer 9cb4300ea183421ab40709a05de5a877 (127.14.177.129:33597): Couldn't send request to peer 9cb4300ea183421ab40709a05de5a877. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:33597: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:24:36.739464 15046 tablet_server.cc:179] TabletServer@127.14.177.130:0 shutting down...
W20260709 13:24:36.773783 27876 consensus_peers.cc:597] T 032f598cb30343c2a1ca3bab728e5b39 P 880fc4c7dff4417187b75cc1d9e9dcf5 -> Peer 0788dbdabe574355bb323dcdc3be0058 (127.14.177.130:41335): Couldn't send request to peer 0788dbdabe574355bb323dcdc3be0058. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:24:36.776845 27800 consensus_peers.cc:597] T 2f38443d4b8d4e418c8f0520893a31e4 P 0788dbdabe574355bb323dcdc3be0058 -> Peer 9cb4300ea183421ab40709a05de5a877 (127.14.177.129:33597): Couldn't send request to peer 9cb4300ea183421ab40709a05de5a877. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:33597: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:24:36.777242 15046 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:24:36.778084 15046 tablet_replica.cc:333] T 96f830fa400c473881739b3a953f0178 P 0788dbdabe574355bb323dcdc3be0058: stopping tablet replica
I20260709 13:24:36.779067 15046 raft_consensus.cc:2243] T 96f830fa400c473881739b3a953f0178 P 0788dbdabe574355bb323dcdc3be0058 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:24:36.779934 15046 raft_consensus.cc:2272] T 96f830fa400c473881739b3a953f0178 P 0788dbdabe574355bb323dcdc3be0058 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:24:36.781812 15046 tablet_replica.cc:333] T 1af49a625b20491586dcea4961e46734 P 0788dbdabe574355bb323dcdc3be0058: stopping tablet replica
I20260709 13:24:36.782374 15046 raft_consensus.cc:2243] T 1af49a625b20491586dcea4961e46734 P 0788dbdabe574355bb323dcdc3be0058 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:24:36.782778 15046 raft_consensus.cc:2272] T 1af49a625b20491586dcea4961e46734 P 0788dbdabe574355bb323dcdc3be0058 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:24:36.784730 15046 tablet_replica.cc:333] T 27629b85cc3e4cb6b837dd26e3ebff47 P 0788dbdabe574355bb323dcdc3be0058: stopping tablet replica
I20260709 13:24:36.785220 15046 raft_consensus.cc:2243] T 27629b85cc3e4cb6b837dd26e3ebff47 P 0788dbdabe574355bb323dcdc3be0058 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:24:36.785588 15046 raft_consensus.cc:2272] T 27629b85cc3e4cb6b837dd26e3ebff47 P 0788dbdabe574355bb323dcdc3be0058 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:24:36.787510 15046 tablet_replica.cc:333] T 2f38443d4b8d4e418c8f0520893a31e4 P 0788dbdabe574355bb323dcdc3be0058: stopping tablet replica
I20260709 13:24:36.787997 15046 raft_consensus.cc:2243] T 2f38443d4b8d4e418c8f0520893a31e4 P 0788dbdabe574355bb323dcdc3be0058 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:24:36.788725 15046 raft_consensus.cc:2272] T 2f38443d4b8d4e418c8f0520893a31e4 P 0788dbdabe574355bb323dcdc3be0058 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:24:36.791087 15046 tablet_replica.cc:333] T 660aa4903c494ba1bce988191700e129 P 0788dbdabe574355bb323dcdc3be0058: stopping tablet replica
I20260709 13:24:36.791674 15046 raft_consensus.cc:2243] T 660aa4903c494ba1bce988191700e129 P 0788dbdabe574355bb323dcdc3be0058 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:24:36.792495 15046 raft_consensus.cc:2272] T 660aa4903c494ba1bce988191700e129 P 0788dbdabe574355bb323dcdc3be0058 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:24:36.794479 15046 tablet_replica.cc:333] T d747d5acdf4f41b69c132cde690e49ff P 0788dbdabe574355bb323dcdc3be0058: stopping tablet replica
I20260709 13:24:36.795075 15046 raft_consensus.cc:2243] T d747d5acdf4f41b69c132cde690e49ff P 0788dbdabe574355bb323dcdc3be0058 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:24:36.795881 15046 raft_consensus.cc:2272] T d747d5acdf4f41b69c132cde690e49ff P 0788dbdabe574355bb323dcdc3be0058 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:24:36.797853 15046 tablet_replica.cc:333] T d16bbb66a57e4079897aaa2c4e6ba144 P 0788dbdabe574355bb323dcdc3be0058: stopping tablet replica
I20260709 13:24:36.798393 15046 raft_consensus.cc:2243] T d16bbb66a57e4079897aaa2c4e6ba144 P 0788dbdabe574355bb323dcdc3be0058 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:24:36.798835 15046 raft_consensus.cc:2272] T d16bbb66a57e4079897aaa2c4e6ba144 P 0788dbdabe574355bb323dcdc3be0058 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:24:36.800876 15046 tablet_replica.cc:333] T 028750ef0efa43e4b11ac10f3814902c P 0788dbdabe574355bb323dcdc3be0058: stopping tablet replica
I20260709 13:24:36.801425 15046 raft_consensus.cc:2243] T 028750ef0efa43e4b11ac10f3814902c P 0788dbdabe574355bb323dcdc3be0058 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:24:36.801859 15046 raft_consensus.cc:2272] T 028750ef0efa43e4b11ac10f3814902c P 0788dbdabe574355bb323dcdc3be0058 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:24:36.803887 15046 tablet_replica.cc:333] T cd1140068fc8455280934d7a30c4f559 P 0788dbdabe574355bb323dcdc3be0058: stopping tablet replica
I20260709 13:24:36.804466 15046 raft_consensus.cc:2243] T cd1140068fc8455280934d7a30c4f559 P 0788dbdabe574355bb323dcdc3be0058 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:24:36.805327 15046 raft_consensus.cc:2272] T cd1140068fc8455280934d7a30c4f559 P 0788dbdabe574355bb323dcdc3be0058 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:24:36.807746 15046 tablet_replica.cc:333] T 032f598cb30343c2a1ca3bab728e5b39 P 0788dbdabe574355bb323dcdc3be0058: stopping tablet replica
I20260709 13:24:36.808568 15046 raft_consensus.cc:2243] T 032f598cb30343c2a1ca3bab728e5b39 P 0788dbdabe574355bb323dcdc3be0058 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:24:36.809283 15046 raft_consensus.cc:2272] T 032f598cb30343c2a1ca3bab728e5b39 P 0788dbdabe574355bb323dcdc3be0058 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:24:36.812479 15046 tablet_replica.cc:333] T d7fd85f92c0a481aa54401371638252b P 0788dbdabe574355bb323dcdc3be0058: stopping tablet replica
I20260709 13:24:36.813308 15046 raft_consensus.cc:2243] T d7fd85f92c0a481aa54401371638252b P 0788dbdabe574355bb323dcdc3be0058 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:24:36.813913 15046 raft_consensus.cc:2272] T d7fd85f92c0a481aa54401371638252b P 0788dbdabe574355bb323dcdc3be0058 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:24:36.816686 15046 tablet_replica.cc:333] T 4554587582094286b577c9c9a4d3a7fc P 0788dbdabe574355bb323dcdc3be0058: stopping tablet replica
I20260709 13:24:36.817454 15046 raft_consensus.cc:2243] T 4554587582094286b577c9c9a4d3a7fc P 0788dbdabe574355bb323dcdc3be0058 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:24:36.818481 15046 raft_consensus.cc:2272] T 4554587582094286b577c9c9a4d3a7fc P 0788dbdabe574355bb323dcdc3be0058 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:24:36.820971 15046 tablet_replica.cc:333] T 8e6b55496c134353a770e2642fd7490d P 0788dbdabe574355bb323dcdc3be0058: stopping tablet replica
I20260709 13:24:36.821525 15046 raft_consensus.cc:2243] T 8e6b55496c134353a770e2642fd7490d P 0788dbdabe574355bb323dcdc3be0058 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:24:36.821977 15046 raft_consensus.cc:2272] T 8e6b55496c134353a770e2642fd7490d P 0788dbdabe574355bb323dcdc3be0058 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:24:36.824240 15046 tablet_replica.cc:333] T f3e47f5448ff47a382c61886f1a8cc7b P 0788dbdabe574355bb323dcdc3be0058: stopping tablet replica
I20260709 13:24:36.824812 15046 raft_consensus.cc:2243] T f3e47f5448ff47a382c61886f1a8cc7b P 0788dbdabe574355bb323dcdc3be0058 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:24:36.825281 15046 raft_consensus.cc:2272] T f3e47f5448ff47a382c61886f1a8cc7b P 0788dbdabe574355bb323dcdc3be0058 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:24:36.827593 15046 tablet_replica.cc:333] T db487682b03b4a738f664fa1459714e5 P 0788dbdabe574355bb323dcdc3be0058: stopping tablet replica
I20260709 13:24:36.828135 15046 raft_consensus.cc:2243] T db487682b03b4a738f664fa1459714e5 P 0788dbdabe574355bb323dcdc3be0058 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:24:36.828584 15046 raft_consensus.cc:2272] T db487682b03b4a738f664fa1459714e5 P 0788dbdabe574355bb323dcdc3be0058 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:24:36.831010 15046 tablet_replica.cc:333] T e7366c36a1154d4d94281a90fde291f8 P 0788dbdabe574355bb323dcdc3be0058: stopping tablet replica
I20260709 13:24:36.831564 15046 raft_consensus.cc:2243] T e7366c36a1154d4d94281a90fde291f8 P 0788dbdabe574355bb323dcdc3be0058 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:24:36.831984 15046 raft_consensus.cc:2272] T e7366c36a1154d4d94281a90fde291f8 P 0788dbdabe574355bb323dcdc3be0058 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:24:36.834087 15046 tablet_replica.cc:333] T 04c481ece2f94a1b8cb6337900da4862 P 0788dbdabe574355bb323dcdc3be0058: stopping tablet replica
I20260709 13:24:36.834602 15046 raft_consensus.cc:2243] T 04c481ece2f94a1b8cb6337900da4862 P 0788dbdabe574355bb323dcdc3be0058 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:24:36.835085 15046 raft_consensus.cc:2272] T 04c481ece2f94a1b8cb6337900da4862 P 0788dbdabe574355bb323dcdc3be0058 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:24:36.837605 15046 tablet_replica.cc:333] T 20ef9c03fcd94115be0b592d85f89421 P 0788dbdabe574355bb323dcdc3be0058: stopping tablet replica
I20260709 13:24:36.838102 15046 raft_consensus.cc:2243] T 20ef9c03fcd94115be0b592d85f89421 P 0788dbdabe574355bb323dcdc3be0058 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:24:36.838527 15046 raft_consensus.cc:2272] T 20ef9c03fcd94115be0b592d85f89421 P 0788dbdabe574355bb323dcdc3be0058 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:24:36.880874 15046 tablet_server.cc:196] TabletServer@127.14.177.130:0 shutdown complete.
W20260709 13:24:36.884164 27874 consensus_peers.cc:597] T 27629b85cc3e4cb6b837dd26e3ebff47 P 880fc4c7dff4417187b75cc1d9e9dcf5 -> Peer 9cb4300ea183421ab40709a05de5a877 (127.14.177.129:33597): Couldn't send request to peer 9cb4300ea183421ab40709a05de5a877. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:33597: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:24:36.903029 27874 consensus_peers.cc:597] T 028750ef0efa43e4b11ac10f3814902c P 880fc4c7dff4417187b75cc1d9e9dcf5 -> Peer 9cb4300ea183421ab40709a05de5a877 (127.14.177.129:33597): Couldn't send request to peer 9cb4300ea183421ab40709a05de5a877. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:33597: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:24:36.919118 15046 tablet_server.cc:179] TabletServer@127.14.177.131:0 shutting down...
I20260709 13:24:36.953411 15046 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:24:36.954159 15046 tablet_replica.cc:333] T 96f830fa400c473881739b3a953f0178 P 880fc4c7dff4417187b75cc1d9e9dcf5: stopping tablet replica
I20260709 13:24:36.954725 15046 raft_consensus.cc:2243] T 96f830fa400c473881739b3a953f0178 P 880fc4c7dff4417187b75cc1d9e9dcf5 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:24:36.955271 15046 raft_consensus.cc:2272] T 96f830fa400c473881739b3a953f0178 P 880fc4c7dff4417187b75cc1d9e9dcf5 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:24:36.957187 15046 tablet_replica.cc:333] T 1af49a625b20491586dcea4961e46734 P 880fc4c7dff4417187b75cc1d9e9dcf5: stopping tablet replica
I20260709 13:24:36.957756 15046 raft_consensus.cc:2243] T 1af49a625b20491586dcea4961e46734 P 880fc4c7dff4417187b75cc1d9e9dcf5 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:24:36.958530 15046 raft_consensus.cc:2272] T 1af49a625b20491586dcea4961e46734 P 880fc4c7dff4417187b75cc1d9e9dcf5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:24:36.960602 15046 tablet_replica.cc:333] T 8e6b55496c134353a770e2642fd7490d P 880fc4c7dff4417187b75cc1d9e9dcf5: stopping tablet replica
I20260709 13:24:36.961189 15046 raft_consensus.cc:2243] T 8e6b55496c134353a770e2642fd7490d P 880fc4c7dff4417187b75cc1d9e9dcf5 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:24:36.961645 15046 raft_consensus.cc:2272] T 8e6b55496c134353a770e2642fd7490d P 880fc4c7dff4417187b75cc1d9e9dcf5 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:24:36.963675 15046 tablet_replica.cc:333] T 27629b85cc3e4cb6b837dd26e3ebff47 P 880fc4c7dff4417187b75cc1d9e9dcf5: stopping tablet replica
I20260709 13:24:36.964233 15046 raft_consensus.cc:2243] T 27629b85cc3e4cb6b837dd26e3ebff47 P 880fc4c7dff4417187b75cc1d9e9dcf5 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:24:36.965045 15046 raft_consensus.cc:2272] T 27629b85cc3e4cb6b837dd26e3ebff47 P 880fc4c7dff4417187b75cc1d9e9dcf5 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:24:36.967149 15046 tablet_replica.cc:333] T 2f38443d4b8d4e418c8f0520893a31e4 P 880fc4c7dff4417187b75cc1d9e9dcf5: stopping tablet replica
I20260709 13:24:36.967767 15046 raft_consensus.cc:2243] T 2f38443d4b8d4e418c8f0520893a31e4 P 880fc4c7dff4417187b75cc1d9e9dcf5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:24:36.968262 15046 raft_consensus.cc:2272] T 2f38443d4b8d4e418c8f0520893a31e4 P 880fc4c7dff4417187b75cc1d9e9dcf5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:24:36.970338 15046 tablet_replica.cc:333] T f3e47f5448ff47a382c61886f1a8cc7b P 880fc4c7dff4417187b75cc1d9e9dcf5: stopping tablet replica
I20260709 13:24:36.970934 15046 raft_consensus.cc:2243] T f3e47f5448ff47a382c61886f1a8cc7b P 880fc4c7dff4417187b75cc1d9e9dcf5 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:24:36.971813 15046 raft_consensus.cc:2272] T f3e47f5448ff47a382c61886f1a8cc7b P 880fc4c7dff4417187b75cc1d9e9dcf5 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:24:36.973790 15046 tablet_replica.cc:333] T 04c481ece2f94a1b8cb6337900da4862 P 880fc4c7dff4417187b75cc1d9e9dcf5: stopping tablet replica
I20260709 13:24:36.974377 15046 raft_consensus.cc:2243] T 04c481ece2f94a1b8cb6337900da4862 P 880fc4c7dff4417187b75cc1d9e9dcf5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:24:36.974822 15046 raft_consensus.cc:2272] T 04c481ece2f94a1b8cb6337900da4862 P 880fc4c7dff4417187b75cc1d9e9dcf5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:24:36.976708 15046 tablet_replica.cc:333] T 20ef9c03fcd94115be0b592d85f89421 P 880fc4c7dff4417187b75cc1d9e9dcf5: stopping tablet replica
I20260709 13:24:36.977237 15046 raft_consensus.cc:2243] T 20ef9c03fcd94115be0b592d85f89421 P 880fc4c7dff4417187b75cc1d9e9dcf5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:24:36.977667 15046 raft_consensus.cc:2272] T 20ef9c03fcd94115be0b592d85f89421 P 880fc4c7dff4417187b75cc1d9e9dcf5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:24:36.979815 15046 tablet_replica.cc:333] T e7366c36a1154d4d94281a90fde291f8 P 880fc4c7dff4417187b75cc1d9e9dcf5: stopping tablet replica
I20260709 13:24:36.980376 15046 raft_consensus.cc:2243] T e7366c36a1154d4d94281a90fde291f8 P 880fc4c7dff4417187b75cc1d9e9dcf5 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:24:36.981138 15046 raft_consensus.cc:2272] T e7366c36a1154d4d94281a90fde291f8 P 880fc4c7dff4417187b75cc1d9e9dcf5 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:24:36.983258 15046 tablet_replica.cc:333] T 032f598cb30343c2a1ca3bab728e5b39 P 880fc4c7dff4417187b75cc1d9e9dcf5: stopping tablet replica
I20260709 13:24:36.983824 15046 raft_consensus.cc:2243] T 032f598cb30343c2a1ca3bab728e5b39 P 880fc4c7dff4417187b75cc1d9e9dcf5 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:24:36.984762 15046 raft_consensus.cc:2272] T 032f598cb30343c2a1ca3bab728e5b39 P 880fc4c7dff4417187b75cc1d9e9dcf5 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:24:36.986600 15046 tablet_replica.cc:333] T d7fd85f92c0a481aa54401371638252b P 880fc4c7dff4417187b75cc1d9e9dcf5: stopping tablet replica
I20260709 13:24:36.987138 15046 raft_consensus.cc:2243] T d7fd85f92c0a481aa54401371638252b P 880fc4c7dff4417187b75cc1d9e9dcf5 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:24:36.987666 15046 raft_consensus.cc:2272] T d7fd85f92c0a481aa54401371638252b P 880fc4c7dff4417187b75cc1d9e9dcf5 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:24:36.989518 15046 tablet_replica.cc:333] T 4554587582094286b577c9c9a4d3a7fc P 880fc4c7dff4417187b75cc1d9e9dcf5: stopping tablet replica
I20260709 13:24:36.990069 15046 raft_consensus.cc:2243] T 4554587582094286b577c9c9a4d3a7fc P 880fc4c7dff4417187b75cc1d9e9dcf5 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:24:36.990504 15046 raft_consensus.cc:2272] T 4554587582094286b577c9c9a4d3a7fc P 880fc4c7dff4417187b75cc1d9e9dcf5 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:24:36.992596 15046 tablet_replica.cc:333] T 660aa4903c494ba1bce988191700e129 P 880fc4c7dff4417187b75cc1d9e9dcf5: stopping tablet replica
I20260709 13:24:36.993127 15046 raft_consensus.cc:2243] T 660aa4903c494ba1bce988191700e129 P 880fc4c7dff4417187b75cc1d9e9dcf5 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:24:36.993573 15046 raft_consensus.cc:2272] T 660aa4903c494ba1bce988191700e129 P 880fc4c7dff4417187b75cc1d9e9dcf5 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:24:36.995507 15046 tablet_replica.cc:333] T d16bbb66a57e4079897aaa2c4e6ba144 P 880fc4c7dff4417187b75cc1d9e9dcf5: stopping tablet replica
I20260709 13:24:36.996047 15046 raft_consensus.cc:2243] T d16bbb66a57e4079897aaa2c4e6ba144 P 880fc4c7dff4417187b75cc1d9e9dcf5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:24:36.996474 15046 raft_consensus.cc:2272] T d16bbb66a57e4079897aaa2c4e6ba144 P 880fc4c7dff4417187b75cc1d9e9dcf5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:24:36.998441 15046 tablet_replica.cc:333] T d747d5acdf4f41b69c132cde690e49ff P 880fc4c7dff4417187b75cc1d9e9dcf5: stopping tablet replica
I20260709 13:24:36.998983 15046 raft_consensus.cc:2243] T d747d5acdf4f41b69c132cde690e49ff P 880fc4c7dff4417187b75cc1d9e9dcf5 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:24:36.999447 15046 raft_consensus.cc:2272] T d747d5acdf4f41b69c132cde690e49ff P 880fc4c7dff4417187b75cc1d9e9dcf5 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:24:37.001336 15046 tablet_replica.cc:333] T 028750ef0efa43e4b11ac10f3814902c P 880fc4c7dff4417187b75cc1d9e9dcf5: stopping tablet replica
I20260709 13:24:37.001844 15046 raft_consensus.cc:2243] T 028750ef0efa43e4b11ac10f3814902c P 880fc4c7dff4417187b75cc1d9e9dcf5 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:24:37.002707 15046 raft_consensus.cc:2272] T 028750ef0efa43e4b11ac10f3814902c P 880fc4c7dff4417187b75cc1d9e9dcf5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:24:37.004791 15046 tablet_replica.cc:333] T cd1140068fc8455280934d7a30c4f559 P 880fc4c7dff4417187b75cc1d9e9dcf5: stopping tablet replica
I20260709 13:24:37.005321 15046 raft_consensus.cc:2243] T cd1140068fc8455280934d7a30c4f559 P 880fc4c7dff4417187b75cc1d9e9dcf5 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:24:37.005769 15046 raft_consensus.cc:2272] T cd1140068fc8455280934d7a30c4f559 P 880fc4c7dff4417187b75cc1d9e9dcf5 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:24:37.007822 15046 tablet_replica.cc:333] T db487682b03b4a738f664fa1459714e5 P 880fc4c7dff4417187b75cc1d9e9dcf5: stopping tablet replica
I20260709 13:24:37.008349 15046 raft_consensus.cc:2243] T db487682b03b4a738f664fa1459714e5 P 880fc4c7dff4417187b75cc1d9e9dcf5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:24:37.008788 15046 raft_consensus.cc:2272] T db487682b03b4a738f664fa1459714e5 P 880fc4c7dff4417187b75cc1d9e9dcf5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:24:37.048763 15046 tablet_server.cc:196] TabletServer@127.14.177.131:0 shutdown complete.
I20260709 13:24:37.086055 15046 master.cc:562] Master@127.14.177.190:41127 shutting down...
I20260709 13:24:37.111387 15046 raft_consensus.cc:2243] T 00000000000000000000000000000000 P c3a531df1ce7417cbf6095f23465e39b [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:24:37.112000 15046 raft_consensus.cc:2272] T 00000000000000000000000000000000 P c3a531df1ce7417cbf6095f23465e39b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:24:37.112349 15046 tablet_replica.cc:333] T 00000000000000000000000000000000 P c3a531df1ce7417cbf6095f23465e39b: stopping tablet replica
I20260709 13:24:37.133703 15046 master.cc:584] Master@127.14.177.190:41127 shutdown complete.
[       OK ] FilterSoftDeletedTableTest.TestFilterSofteDeletedTable/0 (13196 ms)
[ RUN      ] FilterSoftDeletedTableTest.TestFilterSofteDeletedTable/1
I20260709 13:24:37.187484 15046 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.14.177.190:43663
I20260709 13:24:37.188616 15046 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:24:37.194638 28247 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:24:37.196561 28250 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:24:37.196576 15046 server_base.cc:1061] running on GCE node
W20260709 13:24:37.196648 28248 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:24:37.198325 15046 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:24:37.198623 15046 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:24:37.198801 15046 hybrid_clock.cc:648] HybridClock initialized: now 1783603477198786 us; error 0 us; skew 500 ppm
I20260709 13:24:37.199378 15046 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:24:37.201884 15046 webserver.cc:533] Webserver started at http://127.14.177.190:33799/ using document root <none> and password file <none>
I20260709 13:24:37.202390 15046 fs_manager.cc:362] Metadata directory not provided
I20260709 13:24:37.202620 15046 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:24:37.202901 15046 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:24:37.204262 15046 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskdH_Ma7/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783603093801613-15046-0/minicluster-data/master-0-root/instance:
uuid: "e26120d9496c4a7a9362997579e10fe2"
format_stamp: "Formatted at 2026-07-09 13:24:37 on dist-test-slave-v5z2"
I20260709 13:24:37.208964 15046 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.005s	sys 0.000s
I20260709 13:24:37.212498 28255 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:24:37.213519 15046 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.001s
I20260709 13:24:37.213792 15046 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskdH_Ma7/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783603093801613-15046-0/minicluster-data/master-0-root
uuid: "e26120d9496c4a7a9362997579e10fe2"
format_stamp: "Formatted at 2026-07-09 13:24:37 on dist-test-slave-v5z2"
I20260709 13:24:37.214068 15046 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdH_Ma7/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783603093801613-15046-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskdH_Ma7/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783603093801613-15046-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskdH_Ma7/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783603093801613-15046-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:24:37.235932 15046 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:24:37.237155 15046 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:24:37.276090 15046 rpc_server.cc:307] RPC server started. Bound to: 127.14.177.190:43663
I20260709 13:24:37.276202 28306 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.14.177.190:43663 every 8 connection(s)
I20260709 13:24:37.280231 28307 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:24:37.291237 28307 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e26120d9496c4a7a9362997579e10fe2: Bootstrap starting.
I20260709 13:24:37.296110 28307 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e26120d9496c4a7a9362997579e10fe2: Neither blocks nor log segments found. Creating new log.
I20260709 13:24:37.300408 28307 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e26120d9496c4a7a9362997579e10fe2: No bootstrap required, opened a new log
I20260709 13:24:37.302465 28307 raft_consensus.cc:359] T 00000000000000000000000000000000 P e26120d9496c4a7a9362997579e10fe2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e26120d9496c4a7a9362997579e10fe2" member_type: VOTER }
I20260709 13:24:37.302821 28307 raft_consensus.cc:385] T 00000000000000000000000000000000 P e26120d9496c4a7a9362997579e10fe2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:24:37.303009 28307 raft_consensus.cc:740] T 00000000000000000000000000000000 P e26120d9496c4a7a9362997579e10fe2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e26120d9496c4a7a9362997579e10fe2, State: Initialized, Role: FOLLOWER
I20260709 13:24:37.303620 28307 consensus_queue.cc:260] T 00000000000000000000000000000000 P e26120d9496c4a7a9362997579e10fe2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e26120d9496c4a7a9362997579e10fe2" member_type: VOTER }
I20260709 13:24:37.304028 28307 raft_consensus.cc:399] T 00000000000000000000000000000000 P e26120d9496c4a7a9362997579e10fe2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 13:24:37.304214 28307 raft_consensus.cc:493] T 00000000000000000000000000000000 P e26120d9496c4a7a9362997579e10fe2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 13:24:37.304422 28307 raft_consensus.cc:3060] T 00000000000000000000000000000000 P e26120d9496c4a7a9362997579e10fe2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:24:37.309089 28307 raft_consensus.cc:515] T 00000000000000000000000000000000 P e26120d9496c4a7a9362997579e10fe2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e26120d9496c4a7a9362997579e10fe2" member_type: VOTER }
I20260709 13:24:37.309551 28307 leader_election.cc:304] T 00000000000000000000000000000000 P e26120d9496c4a7a9362997579e10fe2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e26120d9496c4a7a9362997579e10fe2; no voters: 
I20260709 13:24:37.310708 28307 leader_election.cc:290] T 00000000000000000000000000000000 P e26120d9496c4a7a9362997579e10fe2 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 13:24:37.311123 28310 raft_consensus.cc:2804] T 00000000000000000000000000000000 P e26120d9496c4a7a9362997579e10fe2 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:24:37.312554 28310 raft_consensus.cc:697] T 00000000000000000000000000000000 P e26120d9496c4a7a9362997579e10fe2 [term 1 LEADER]: Becoming Leader. State: Replica: e26120d9496c4a7a9362997579e10fe2, State: Running, Role: LEADER
I20260709 13:24:37.313234 28310 consensus_queue.cc:237] T 00000000000000000000000000000000 P e26120d9496c4a7a9362997579e10fe2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e26120d9496c4a7a9362997579e10fe2" member_type: VOTER }
I20260709 13:24:37.313851 28307 sys_catalog.cc:565] T 00000000000000000000000000000000 P e26120d9496c4a7a9362997579e10fe2 [sys.catalog]: configured and running, proceeding with master startup.
I20260709 13:24:37.317385 28311 sys_catalog.cc:455] T 00000000000000000000000000000000 P e26120d9496c4a7a9362997579e10fe2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e26120d9496c4a7a9362997579e10fe2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e26120d9496c4a7a9362997579e10fe2" member_type: VOTER } }
I20260709 13:24:37.317592 28312 sys_catalog.cc:455] T 00000000000000000000000000000000 P e26120d9496c4a7a9362997579e10fe2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e26120d9496c4a7a9362997579e10fe2. Latest consensus state: current_term: 1 leader_uuid: "e26120d9496c4a7a9362997579e10fe2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e26120d9496c4a7a9362997579e10fe2" member_type: VOTER } }
I20260709 13:24:37.318068 28311 sys_catalog.cc:458] T 00000000000000000000000000000000 P e26120d9496c4a7a9362997579e10fe2 [sys.catalog]: This master's current role is: LEADER
I20260709 13:24:37.318156 28312 sys_catalog.cc:458] T 00000000000000000000000000000000 P e26120d9496c4a7a9362997579e10fe2 [sys.catalog]: This master's current role is: LEADER
I20260709 13:24:37.329524 28316 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 13:24:37.334414 28316 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 13:24:37.335582 15046 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 13:24:37.343329 28316 catalog_manager.cc:1383] Generated new cluster ID: 6337ee71a06c4ec097f68d88a922c20e
I20260709 13:24:37.343708 28316 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 13:24:37.360513 28316 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 13:24:37.361814 28316 catalog_manager.cc:1540] Loading token signing keys...
I20260709 13:24:37.382839 28316 catalog_manager.cc:6070] T 00000000000000000000000000000000 P e26120d9496c4a7a9362997579e10fe2: Generated new TSK 0
I20260709 13:24:37.383596 28316 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 13:24:37.402516 15046 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:24:37.408762 28328 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:24:37.410226 28329 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:24:37.411650 28331 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:24:37.414590 15046 server_base.cc:1061] running on GCE node
I20260709 13:24:37.415655 15046 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:24:37.415871 15046 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:24:37.416039 15046 hybrid_clock.cc:648] HybridClock initialized: now 1783603477416007 us; error 0 us; skew 500 ppm
I20260709 13:24:37.416591 15046 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:24:37.418825 15046 webserver.cc:533] Webserver started at http://127.14.177.129:40985/ using document root <none> and password file <none>
I20260709 13:24:37.419327 15046 fs_manager.cc:362] Metadata directory not provided
I20260709 13:24:37.419489 15046 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:24:37.419692 15046 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:24:37.420696 15046 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskdH_Ma7/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783603093801613-15046-0/minicluster-data/ts-0-root/instance:
uuid: "542ea9e2fcff4028b6766459e90bc5fc"
format_stamp: "Formatted at 2026-07-09 13:24:37 on dist-test-slave-v5z2"
I20260709 13:24:37.425431 15046 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.004s	sys 0.002s
I20260709 13:24:37.429196 28336 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:24:37.429984 15046 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.001s	sys 0.000s
I20260709 13:24:37.430297 15046 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskdH_Ma7/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783603093801613-15046-0/minicluster-data/ts-0-root
uuid: "542ea9e2fcff4028b6766459e90bc5fc"
format_stamp: "Formatted at 2026-07-09 13:24:37 on dist-test-slave-v5z2"
I20260709 13:24:37.430567 15046 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdH_Ma7/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783603093801613-15046-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskdH_Ma7/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783603093801613-15046-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskdH_Ma7/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783603093801613-15046-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:24:37.457494 15046 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:24:37.458798 15046 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:24:37.460340 15046 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:24:37.462667 15046 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:24:37.462890 15046 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:24:37.463137 15046 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:24:37.463344 15046 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.001s	sys 0.000s
I20260709 13:24:37.504859 15046 rpc_server.cc:307] RPC server started. Bound to: 127.14.177.129:38409
I20260709 13:24:37.504982 28398 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.14.177.129:38409 every 8 connection(s)
I20260709 13:24:37.510143 15046 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:24:37.517242 28403 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:24:37.518527 28404 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:24:37.522172 15046 server_base.cc:1061] running on GCE node
W20260709 13:24:37.523072 28406 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:24:37.524156 15046 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:24:37.524379 15046 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:24:37.524567 15046 hybrid_clock.cc:648] HybridClock initialized: now 1783603477524536 us; error 0 us; skew 500 ppm
I20260709 13:24:37.525084 15046 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:24:37.528087 15046 webserver.cc:533] Webserver started at http://127.14.177.130:39707/ using document root <none> and password file <none>
I20260709 13:24:37.528585 15046 fs_manager.cc:362] Metadata directory not provided
I20260709 13:24:37.528797 15046 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:24:37.529071 15046 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:24:37.529273 28399 heartbeater.cc:344] Connected to a master server at 127.14.177.190:43663
I20260709 13:24:37.529717 28399 heartbeater.cc:461] Registering TS with master...
I20260709 13:24:37.530242 15046 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskdH_Ma7/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783603093801613-15046-0/minicluster-data/ts-1-root/instance:
uuid: "96cba770f3334f79af3162475b54fa0f"
format_stamp: "Formatted at 2026-07-09 13:24:37 on dist-test-slave-v5z2"
I20260709 13:24:37.530411 28399 heartbeater.cc:507] Master 127.14.177.190:43663 requested a full tablet report, sending...
I20260709 13:24:37.532820 28272 ts_manager.cc:194] Registered new tserver with Master: 542ea9e2fcff4028b6766459e90bc5fc (127.14.177.129:38409)
I20260709 13:24:37.534925 28272 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:39864
I20260709 13:24:37.535375 15046 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.007s	sys 0.000s
I20260709 13:24:37.539045 28411 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:24:37.539896 15046 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.000s
I20260709 13:24:37.540215 15046 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskdH_Ma7/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783603093801613-15046-0/minicluster-data/ts-1-root
uuid: "96cba770f3334f79af3162475b54fa0f"
format_stamp: "Formatted at 2026-07-09 13:24:37 on dist-test-slave-v5z2"
I20260709 13:24:37.540500 15046 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdH_Ma7/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783603093801613-15046-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskdH_Ma7/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783603093801613-15046-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskdH_Ma7/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783603093801613-15046-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:24:37.551654 15046 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:24:37.552850 15046 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:24:37.554301 15046 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:24:37.556429 15046 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:24:37.556644 15046 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:24:37.556878 15046 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:24:37.557049 15046 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:24:37.596637 15046 rpc_server.cc:307] RPC server started. Bound to: 127.14.177.130:33585
I20260709 13:24:37.596772 28473 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.14.177.130:33585 every 8 connection(s)
I20260709 13:24:37.601611 15046 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:24:37.609078 28477 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:24:37.610929 28478 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:24:37.613657 15046 server_base.cc:1061] running on GCE node
W20260709 13:24:37.613735 28480 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:24:37.614686 28474 heartbeater.cc:344] Connected to a master server at 127.14.177.190:43663
I20260709 13:24:37.615191 28474 heartbeater.cc:461] Registering TS with master...
I20260709 13:24:37.615204 15046 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:24:37.615644 15046 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:24:37.615912 15046 hybrid_clock.cc:648] HybridClock initialized: now 1783603477615893 us; error 0 us; skew 500 ppm
I20260709 13:24:37.616286 28474 heartbeater.cc:507] Master 127.14.177.190:43663 requested a full tablet report, sending...
I20260709 13:24:37.616746 15046 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:24:37.618810 28272 ts_manager.cc:194] Registered new tserver with Master: 96cba770f3334f79af3162475b54fa0f (127.14.177.130:33585)
I20260709 13:24:37.619606 15046 webserver.cc:533] Webserver started at http://127.14.177.131:39061/ using document root <none> and password file <none>
I20260709 13:24:37.620177 15046 fs_manager.cc:362] Metadata directory not provided
I20260709 13:24:37.620366 15046 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:24:37.620407 28272 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:39866
I20260709 13:24:37.620638 15046 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:24:37.622359 15046 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskdH_Ma7/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783603093801613-15046-0/minicluster-data/ts-2-root/instance:
uuid: "2173e3fd92f34ad6ad8d56d2e6b05299"
format_stamp: "Formatted at 2026-07-09 13:24:37 on dist-test-slave-v5z2"
I20260709 13:24:37.627393 15046 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.004s	sys 0.003s
I20260709 13:24:37.630702 28485 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:24:37.631567 15046 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.000s	sys 0.003s
I20260709 13:24:37.631897 15046 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskdH_Ma7/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783603093801613-15046-0/minicluster-data/ts-2-root
uuid: "2173e3fd92f34ad6ad8d56d2e6b05299"
format_stamp: "Formatted at 2026-07-09 13:24:37 on dist-test-slave-v5z2"
I20260709 13:24:37.632150 15046 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskdH_Ma7/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783603093801613-15046-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskdH_Ma7/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783603093801613-15046-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskdH_Ma7/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783603093801613-15046-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:24:37.652829 15046 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:24:37.653944 15046 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:24:37.655457 15046 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:24:37.658167 15046 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:24:37.658449 15046 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:24:37.658756 15046 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:24:37.658980 15046 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:24:37.700014 15046 rpc_server.cc:307] RPC server started. Bound to: 127.14.177.131:37597
I20260709 13:24:37.700124 28547 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.14.177.131:37597 every 8 connection(s)
I20260709 13:24:37.714877 28548 heartbeater.cc:344] Connected to a master server at 127.14.177.190:43663
I20260709 13:24:37.715524 28548 heartbeater.cc:461] Registering TS with master...
I20260709 13:24:37.716324 28548 heartbeater.cc:507] Master 127.14.177.190:43663 requested a full tablet report, sending...
I20260709 13:24:37.718379 28272 ts_manager.cc:194] Registered new tserver with Master: 2173e3fd92f34ad6ad8d56d2e6b05299 (127.14.177.131:37597)
I20260709 13:24:37.719441 15046 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.016028728s
I20260709 13:24:37.720018 28272 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:39882
I20260709 13:24:37.746208 28272 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39886:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q""\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 13:24:37.748884 28272 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 13:24:37.817766 28364 tablet_service.cc:1511] Processing CreateTablet for tablet 68541149575b487b8c4e2360c2c93921 (DEFAULT_TABLE table=test-workload [id=959bc522cc6a45fb9cff3854343da602]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:24:37.819701 28364 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 68541149575b487b8c4e2360c2c93921. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:24:37.824342 28363 tablet_service.cc:1511] Processing CreateTablet for tablet 41515f53db8540a8b4f52d9ac91dab50 (DEFAULT_TABLE table=test-workload [id=959bc522cc6a45fb9cff3854343da602]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:24:37.824608 28362 tablet_service.cc:1511] Processing CreateTablet for tablet 73a4db7e3a924c6f99b416e6dda95c51 (DEFAULT_TABLE table=test-workload [id=959bc522cc6a45fb9cff3854343da602]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:24:37.826021 28363 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 41515f53db8540a8b4f52d9ac91dab50. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:24:37.827804 28362 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 73a4db7e3a924c6f99b416e6dda95c51. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:24:37.836663 28361 tablet_service.cc:1511] Processing CreateTablet for tablet 037bfc9d92244fcb872b2ca382c21794 (DEFAULT_TABLE table=test-workload [id=959bc522cc6a45fb9cff3854343da602]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:24:37.838757 28361 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 037bfc9d92244fcb872b2ca382c21794. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:24:37.847714 28360 tablet_service.cc:1511] Processing CreateTablet for tablet bbaf4eecd3c241e3aabff37f6ba0256a (DEFAULT_TABLE table=test-workload [id=959bc522cc6a45fb9cff3854343da602]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:24:37.849663 28360 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bbaf4eecd3c241e3aabff37f6ba0256a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:24:37.861383 28513 tablet_service.cc:1511] Processing CreateTablet for tablet 68541149575b487b8c4e2360c2c93921 (DEFAULT_TABLE table=test-workload [id=959bc522cc6a45fb9cff3854343da602]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:24:37.862840 28513 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 68541149575b487b8c4e2360c2c93921. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:24:37.864192 28436 tablet_service.cc:1511] Processing CreateTablet for tablet 037bfc9d92244fcb872b2ca382c21794 (DEFAULT_TABLE table=test-workload [id=959bc522cc6a45fb9cff3854343da602]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:24:37.865620 28436 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 037bfc9d92244fcb872b2ca382c21794. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:24:37.866267 28435 tablet_service.cc:1511] Processing CreateTablet for tablet bbaf4eecd3c241e3aabff37f6ba0256a (DEFAULT_TABLE table=test-workload [id=959bc522cc6a45fb9cff3854343da602]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:24:37.867668 28435 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bbaf4eecd3c241e3aabff37f6ba0256a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:24:37.868312 28437 tablet_service.cc:1511] Processing CreateTablet for tablet 73a4db7e3a924c6f99b416e6dda95c51 (DEFAULT_TABLE table=test-workload [id=959bc522cc6a45fb9cff3854343da602]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:24:37.869683 28437 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 73a4db7e3a924c6f99b416e6dda95c51. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:24:37.872349 28438 tablet_service.cc:1511] Processing CreateTablet for tablet 41515f53db8540a8b4f52d9ac91dab50 (DEFAULT_TABLE table=test-workload [id=959bc522cc6a45fb9cff3854343da602]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:24:37.873673 28438 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 41515f53db8540a8b4f52d9ac91dab50. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:24:37.876570 28510 tablet_service.cc:1511] Processing CreateTablet for tablet 037bfc9d92244fcb872b2ca382c21794 (DEFAULT_TABLE table=test-workload [id=959bc522cc6a45fb9cff3854343da602]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:24:37.878047 28510 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 037bfc9d92244fcb872b2ca382c21794. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:24:37.877604 28512 tablet_service.cc:1511] Processing CreateTablet for tablet 41515f53db8540a8b4f52d9ac91dab50 (DEFAULT_TABLE table=test-workload [id=959bc522cc6a45fb9cff3854343da602]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:24:37.878976 28439 tablet_service.cc:1511] Processing CreateTablet for tablet 68541149575b487b8c4e2360c2c93921 (DEFAULT_TABLE table=test-workload [id=959bc522cc6a45fb9cff3854343da602]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:24:37.877751 28511 tablet_service.cc:1511] Processing CreateTablet for tablet 73a4db7e3a924c6f99b416e6dda95c51 (DEFAULT_TABLE table=test-workload [id=959bc522cc6a45fb9cff3854343da602]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:24:37.879110 28434 tablet_service.cc:1511] Processing CreateTablet for tablet a6cf33518cd34784a1b85843d90badb5 (DEFAULT_TABLE table=test-workload [id=959bc522cc6a45fb9cff3854343da602]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:24:37.879417 28512 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 41515f53db8540a8b4f52d9ac91dab50. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:24:37.877697 28509 tablet_service.cc:1511] Processing CreateTablet for tablet bbaf4eecd3c241e3aabff37f6ba0256a (DEFAULT_TABLE table=test-workload [id=959bc522cc6a45fb9cff3854343da602]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:24:37.880455 28439 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 68541149575b487b8c4e2360c2c93921. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:24:37.891110 28434 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a6cf33518cd34784a1b85843d90badb5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:24:37.881657 28508 tablet_service.cc:1511] Processing CreateTablet for tablet a6cf33518cd34784a1b85843d90badb5 (DEFAULT_TABLE table=test-workload [id=959bc522cc6a45fb9cff3854343da602]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:24:37.882712 28433 tablet_service.cc:1511] Processing CreateTablet for tablet 0701509f07184014b94c627c93b3bd59 (DEFAULT_TABLE table=test-workload [id=959bc522cc6a45fb9cff3854343da602]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:24:37.894373 28433 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0701509f07184014b94c627c93b3bd59. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:24:37.884058 28507 tablet_service.cc:1511] Processing CreateTablet for tablet 0701509f07184014b94c627c93b3bd59 (DEFAULT_TABLE table=test-workload [id=959bc522cc6a45fb9cff3854343da602]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:24:37.886240 28359 tablet_service.cc:1511] Processing CreateTablet for tablet a6cf33518cd34784a1b85843d90badb5 (DEFAULT_TABLE table=test-workload [id=959bc522cc6a45fb9cff3854343da602]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:24:37.898269 28359 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a6cf33518cd34784a1b85843d90badb5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:24:37.888921 28357 tablet_service.cc:1511] Processing CreateTablet for tablet 255f88c066ac422d9d8a582d9b24a958 (DEFAULT_TABLE table=test-workload [id=959bc522cc6a45fb9cff3854343da602]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:24:37.900607 28357 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 255f88c066ac422d9d8a582d9b24a958. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:24:37.889132 28358 tablet_service.cc:1511] Processing CreateTablet for tablet 0701509f07184014b94c627c93b3bd59 (DEFAULT_TABLE table=test-workload [id=959bc522cc6a45fb9cff3854343da602]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:24:37.904690 28358 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0701509f07184014b94c627c93b3bd59. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:24:37.906881 28506 tablet_service.cc:1511] Processing CreateTablet for tablet 255f88c066ac422d9d8a582d9b24a958 (DEFAULT_TABLE table=test-workload [id=959bc522cc6a45fb9cff3854343da602]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:24:37.890283 28509 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bbaf4eecd3c241e3aabff37f6ba0256a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:24:37.908414 28506 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 255f88c066ac422d9d8a582d9b24a958. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:24:37.908459 28356 tablet_service.cc:1511] Processing CreateTablet for tablet b75c7d0f42e6499f936d08ff343fca5b (DEFAULT_TABLE table=test-workload [id=959bc522cc6a45fb9cff3854343da602]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:24:37.885107 28432 tablet_service.cc:1511] Processing CreateTablet for tablet 255f88c066ac422d9d8a582d9b24a958 (DEFAULT_TABLE table=test-workload [id=959bc522cc6a45fb9cff3854343da602]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:24:37.909943 28356 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b75c7d0f42e6499f936d08ff343fca5b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:24:37.909955 28432 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 255f88c066ac422d9d8a582d9b24a958. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:24:37.911577 28508 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a6cf33518cd34784a1b85843d90badb5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:24:37.914682 28511 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 73a4db7e3a924c6f99b416e6dda95c51. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:24:37.916028 28431 tablet_service.cc:1511] Processing CreateTablet for tablet b75c7d0f42e6499f936d08ff343fca5b (DEFAULT_TABLE table=test-workload [id=959bc522cc6a45fb9cff3854343da602]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:24:37.917465 28431 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b75c7d0f42e6499f936d08ff343fca5b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:24:37.919564 28507 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0701509f07184014b94c627c93b3bd59. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:24:37.928190 28505 tablet_service.cc:1511] Processing CreateTablet for tablet b75c7d0f42e6499f936d08ff343fca5b (DEFAULT_TABLE table=test-workload [id=959bc522cc6a45fb9cff3854343da602]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:24:37.955314 28563 tablet_bootstrap.cc:492] T 68541149575b487b8c4e2360c2c93921 P 542ea9e2fcff4028b6766459e90bc5fc: Bootstrap starting.
I20260709 13:24:37.967326 28505 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b75c7d0f42e6499f936d08ff343fca5b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:24:37.973834 28563 tablet_bootstrap.cc:654] T 68541149575b487b8c4e2360c2c93921 P 542ea9e2fcff4028b6766459e90bc5fc: Neither blocks nor log segments found. Creating new log.
I20260709 13:24:38.026212 28565 tablet_bootstrap.cc:492] T 68541149575b487b8c4e2360c2c93921 P 2173e3fd92f34ad6ad8d56d2e6b05299: Bootstrap starting.
I20260709 13:24:38.035454 28563 tablet_bootstrap.cc:492] T 68541149575b487b8c4e2360c2c93921 P 542ea9e2fcff4028b6766459e90bc5fc: No bootstrap required, opened a new log
I20260709 13:24:38.036130 28563 ts_tablet_manager.cc:1403] T 68541149575b487b8c4e2360c2c93921 P 542ea9e2fcff4028b6766459e90bc5fc: Time spent bootstrapping tablet: real 0.081s	user 0.018s	sys 0.037s
I20260709 13:24:38.041347 28563 raft_consensus.cc:359] T 68541149575b487b8c4e2360c2c93921 P 542ea9e2fcff4028b6766459e90bc5fc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 37597 } } peers { permanent_uuid: "542ea9e2fcff4028b6766459e90bc5fc" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38409 } } peers { permanent_uuid: "96cba770f3334f79af3162475b54fa0f" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 33585 } }
I20260709 13:24:38.042578 28563 raft_consensus.cc:385] T 68541149575b487b8c4e2360c2c93921 P 542ea9e2fcff4028b6766459e90bc5fc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:24:38.042915 28563 raft_consensus.cc:740] T 68541149575b487b8c4e2360c2c93921 P 542ea9e2fcff4028b6766459e90bc5fc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 542ea9e2fcff4028b6766459e90bc5fc, State: Initialized, Role: FOLLOWER
I20260709 13:24:38.043938 28563 consensus_queue.cc:260] T 68541149575b487b8c4e2360c2c93921 P 542ea9e2fcff4028b6766459e90bc5fc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 37597 } } peers { permanent_uuid: "542ea9e2fcff4028b6766459e90bc5fc" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38409 } } peers { permanent_uuid: "96cba770f3334f79af3162475b54fa0f" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 33585 } }
I20260709 13:24:38.052170 28564 tablet_bootstrap.cc:492] T 73a4db7e3a924c6f99b416e6dda95c51 P 96cba770f3334f79af3162475b54fa0f: Bootstrap starting.
I20260709 13:24:38.079954 28565 tablet_bootstrap.cc:654] T 68541149575b487b8c4e2360c2c93921 P 2173e3fd92f34ad6ad8d56d2e6b05299: Neither blocks nor log segments found. Creating new log.
I20260709 13:24:38.083240 28564 tablet_bootstrap.cc:654] T 73a4db7e3a924c6f99b416e6dda95c51 P 96cba770f3334f79af3162475b54fa0f: Neither blocks nor log segments found. Creating new log.
I20260709 13:24:38.084529 28399 heartbeater.cc:499] Master 127.14.177.190:43663 was elected leader, sending a full tablet report...
I20260709 13:24:38.085119 28563 ts_tablet_manager.cc:1434] T 68541149575b487b8c4e2360c2c93921 P 542ea9e2fcff4028b6766459e90bc5fc: Time spent starting tablet: real 0.049s	user 0.017s	sys 0.024s
I20260709 13:24:38.086371 28563 tablet_bootstrap.cc:492] T 73a4db7e3a924c6f99b416e6dda95c51 P 542ea9e2fcff4028b6766459e90bc5fc: Bootstrap starting.
I20260709 13:24:38.090687 28565 tablet_bootstrap.cc:492] T 68541149575b487b8c4e2360c2c93921 P 2173e3fd92f34ad6ad8d56d2e6b05299: No bootstrap required, opened a new log
I20260709 13:24:38.091323 28564 tablet_bootstrap.cc:492] T 73a4db7e3a924c6f99b416e6dda95c51 P 96cba770f3334f79af3162475b54fa0f: No bootstrap required, opened a new log
I20260709 13:24:38.091436 28565 ts_tablet_manager.cc:1403] T 68541149575b487b8c4e2360c2c93921 P 2173e3fd92f34ad6ad8d56d2e6b05299: Time spent bootstrapping tablet: real 0.065s	user 0.016s	sys 0.000s
I20260709 13:24:38.092061 28564 ts_tablet_manager.cc:1403] T 73a4db7e3a924c6f99b416e6dda95c51 P 96cba770f3334f79af3162475b54fa0f: Time spent bootstrapping tablet: real 0.049s	user 0.005s	sys 0.008s
I20260709 13:24:38.093897 28563 tablet_bootstrap.cc:654] T 73a4db7e3a924c6f99b416e6dda95c51 P 542ea9e2fcff4028b6766459e90bc5fc: Neither blocks nor log segments found. Creating new log.
I20260709 13:24:38.095463 28565 raft_consensus.cc:359] T 68541149575b487b8c4e2360c2c93921 P 2173e3fd92f34ad6ad8d56d2e6b05299 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 37597 } } peers { permanent_uuid: "542ea9e2fcff4028b6766459e90bc5fc" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38409 } } peers { permanent_uuid: "96cba770f3334f79af3162475b54fa0f" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 33585 } }
I20260709 13:24:38.095896 28564 raft_consensus.cc:359] T 73a4db7e3a924c6f99b416e6dda95c51 P 96cba770f3334f79af3162475b54fa0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "542ea9e2fcff4028b6766459e90bc5fc" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38409 } } peers { permanent_uuid: "96cba770f3334f79af3162475b54fa0f" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 33585 } } peers { permanent_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 37597 } }
I20260709 13:24:38.096647 28565 raft_consensus.cc:385] T 68541149575b487b8c4e2360c2c93921 P 2173e3fd92f34ad6ad8d56d2e6b05299 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:24:38.096910 28564 raft_consensus.cc:385] T 73a4db7e3a924c6f99b416e6dda95c51 P 96cba770f3334f79af3162475b54fa0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:24:38.097237 28565 raft_consensus.cc:740] T 68541149575b487b8c4e2360c2c93921 P 2173e3fd92f34ad6ad8d56d2e6b05299 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2173e3fd92f34ad6ad8d56d2e6b05299, State: Initialized, Role: FOLLOWER
I20260709 13:24:38.097496 28564 raft_consensus.cc:740] T 73a4db7e3a924c6f99b416e6dda95c51 P 96cba770f3334f79af3162475b54fa0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 96cba770f3334f79af3162475b54fa0f, State: Initialized, Role: FOLLOWER
I20260709 13:24:38.098460 28565 consensus_queue.cc:260] T 68541149575b487b8c4e2360c2c93921 P 2173e3fd92f34ad6ad8d56d2e6b05299 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 37597 } } peers { permanent_uuid: "542ea9e2fcff4028b6766459e90bc5fc" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38409 } } peers { permanent_uuid: "96cba770f3334f79af3162475b54fa0f" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 33585 } }
I20260709 13:24:38.099124 28564 consensus_queue.cc:260] T 73a4db7e3a924c6f99b416e6dda95c51 P 96cba770f3334f79af3162475b54fa0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "542ea9e2fcff4028b6766459e90bc5fc" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38409 } } peers { permanent_uuid: "96cba770f3334f79af3162475b54fa0f" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 33585 } } peers { permanent_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 37597 } }
I20260709 13:24:38.107877 28474 heartbeater.cc:499] Master 127.14.177.190:43663 was elected leader, sending a full tablet report...
I20260709 13:24:38.108491 28564 ts_tablet_manager.cc:1434] T 73a4db7e3a924c6f99b416e6dda95c51 P 96cba770f3334f79af3162475b54fa0f: Time spent starting tablet: real 0.016s	user 0.008s	sys 0.007s
I20260709 13:24:38.109252 28565 ts_tablet_manager.cc:1434] T 68541149575b487b8c4e2360c2c93921 P 2173e3fd92f34ad6ad8d56d2e6b05299: Time spent starting tablet: real 0.017s	user 0.005s	sys 0.008s
I20260709 13:24:38.109671 28564 tablet_bootstrap.cc:492] T a6cf33518cd34784a1b85843d90badb5 P 96cba770f3334f79af3162475b54fa0f: Bootstrap starting.
I20260709 13:24:38.110087 28563 tablet_bootstrap.cc:492] T 73a4db7e3a924c6f99b416e6dda95c51 P 542ea9e2fcff4028b6766459e90bc5fc: No bootstrap required, opened a new log
I20260709 13:24:38.110610 28565 tablet_bootstrap.cc:492] T 255f88c066ac422d9d8a582d9b24a958 P 2173e3fd92f34ad6ad8d56d2e6b05299: Bootstrap starting.
I20260709 13:24:38.110692 28563 ts_tablet_manager.cc:1403] T 73a4db7e3a924c6f99b416e6dda95c51 P 542ea9e2fcff4028b6766459e90bc5fc: Time spent bootstrapping tablet: real 0.025s	user 0.017s	sys 0.004s
I20260709 13:24:38.112465 28548 heartbeater.cc:499] Master 127.14.177.190:43663 was elected leader, sending a full tablet report...
I20260709 13:24:38.114425 28563 raft_consensus.cc:359] T 73a4db7e3a924c6f99b416e6dda95c51 P 542ea9e2fcff4028b6766459e90bc5fc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "542ea9e2fcff4028b6766459e90bc5fc" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38409 } } peers { permanent_uuid: "96cba770f3334f79af3162475b54fa0f" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 33585 } } peers { permanent_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 37597 } }
I20260709 13:24:38.117671 28563 raft_consensus.cc:385] T 73a4db7e3a924c6f99b416e6dda95c51 P 542ea9e2fcff4028b6766459e90bc5fc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:24:38.118149 28563 raft_consensus.cc:740] T 73a4db7e3a924c6f99b416e6dda95c51 P 542ea9e2fcff4028b6766459e90bc5fc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 542ea9e2fcff4028b6766459e90bc5fc, State: Initialized, Role: FOLLOWER
I20260709 13:24:38.119345 28565 tablet_bootstrap.cc:654] T 255f88c066ac422d9d8a582d9b24a958 P 2173e3fd92f34ad6ad8d56d2e6b05299: Neither blocks nor log segments found. Creating new log.
I20260709 13:24:38.119024 28563 consensus_queue.cc:260] T 73a4db7e3a924c6f99b416e6dda95c51 P 542ea9e2fcff4028b6766459e90bc5fc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "542ea9e2fcff4028b6766459e90bc5fc" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38409 } } peers { permanent_uuid: "96cba770f3334f79af3162475b54fa0f" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 33585 } } peers { permanent_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 37597 } }
I20260709 13:24:38.117472 28571 raft_consensus.cc:493] T 68541149575b487b8c4e2360c2c93921 P 2173e3fd92f34ad6ad8d56d2e6b05299 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:24:38.121943 28563 ts_tablet_manager.cc:1434] T 73a4db7e3a924c6f99b416e6dda95c51 P 542ea9e2fcff4028b6766459e90bc5fc: Time spent starting tablet: real 0.011s	user 0.007s	sys 0.000s
I20260709 13:24:38.121903 28571 raft_consensus.cc:515] T 68541149575b487b8c4e2360c2c93921 P 2173e3fd92f34ad6ad8d56d2e6b05299 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 37597 } } peers { permanent_uuid: "542ea9e2fcff4028b6766459e90bc5fc" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38409 } } peers { permanent_uuid: "96cba770f3334f79af3162475b54fa0f" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 33585 } }
I20260709 13:24:38.123495 28563 tablet_bootstrap.cc:492] T 037bfc9d92244fcb872b2ca382c21794 P 542ea9e2fcff4028b6766459e90bc5fc: Bootstrap starting.
I20260709 13:24:38.128264 28564 tablet_bootstrap.cc:654] T a6cf33518cd34784a1b85843d90badb5 P 96cba770f3334f79af3162475b54fa0f: Neither blocks nor log segments found. Creating new log.
I20260709 13:24:38.135020 28563 tablet_bootstrap.cc:654] T 037bfc9d92244fcb872b2ca382c21794 P 542ea9e2fcff4028b6766459e90bc5fc: Neither blocks nor log segments found. Creating new log.
I20260709 13:24:38.136679 28571 leader_election.cc:290] T 68541149575b487b8c4e2360c2c93921 P 2173e3fd92f34ad6ad8d56d2e6b05299 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 542ea9e2fcff4028b6766459e90bc5fc (127.14.177.129:38409), 96cba770f3334f79af3162475b54fa0f (127.14.177.130:33585)
I20260709 13:24:38.161003 28449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68541149575b487b8c4e2360c2c93921" candidate_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96cba770f3334f79af3162475b54fa0f" is_pre_election: true
W20260709 13:24:38.163678 28489 leader_election.cc:343] T 68541149575b487b8c4e2360c2c93921 P 2173e3fd92f34ad6ad8d56d2e6b05299 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 96cba770f3334f79af3162475b54fa0f (127.14.177.130:33585): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:24:38.163654 28564 tablet_bootstrap.cc:492] T a6cf33518cd34784a1b85843d90badb5 P 96cba770f3334f79af3162475b54fa0f: No bootstrap required, opened a new log
I20260709 13:24:38.164921 28564 ts_tablet_manager.cc:1403] T a6cf33518cd34784a1b85843d90badb5 P 96cba770f3334f79af3162475b54fa0f: Time spent bootstrapping tablet: real 0.056s	user 0.018s	sys 0.001s
I20260709 13:24:38.168066 28374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68541149575b487b8c4e2360c2c93921" candidate_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "542ea9e2fcff4028b6766459e90bc5fc" is_pre_election: true
I20260709 13:24:38.168388 28564 raft_consensus.cc:359] T a6cf33518cd34784a1b85843d90badb5 P 96cba770f3334f79af3162475b54fa0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "542ea9e2fcff4028b6766459e90bc5fc" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38409 } } peers { permanent_uuid: "96cba770f3334f79af3162475b54fa0f" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 33585 } } peers { permanent_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 37597 } }
I20260709 13:24:38.168849 28374 raft_consensus.cc:2468] T 68541149575b487b8c4e2360c2c93921 P 542ea9e2fcff4028b6766459e90bc5fc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2173e3fd92f34ad6ad8d56d2e6b05299 in term 0.
I20260709 13:24:38.169178 28564 raft_consensus.cc:385] T a6cf33518cd34784a1b85843d90badb5 P 96cba770f3334f79af3162475b54fa0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:24:38.169526 28564 raft_consensus.cc:740] T a6cf33518cd34784a1b85843d90badb5 P 96cba770f3334f79af3162475b54fa0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 96cba770f3334f79af3162475b54fa0f, State: Initialized, Role: FOLLOWER
I20260709 13:24:38.170326 28487 leader_election.cc:304] T 68541149575b487b8c4e2360c2c93921 P 2173e3fd92f34ad6ad8d56d2e6b05299 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2173e3fd92f34ad6ad8d56d2e6b05299, 542ea9e2fcff4028b6766459e90bc5fc; no voters: 96cba770f3334f79af3162475b54fa0f
I20260709 13:24:38.171340 28571 raft_consensus.cc:2804] T 68541149575b487b8c4e2360c2c93921 P 2173e3fd92f34ad6ad8d56d2e6b05299 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:24:38.171706 28571 raft_consensus.cc:493] T 68541149575b487b8c4e2360c2c93921 P 2173e3fd92f34ad6ad8d56d2e6b05299 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:24:38.161510 28563 tablet_bootstrap.cc:492] T 037bfc9d92244fcb872b2ca382c21794 P 542ea9e2fcff4028b6766459e90bc5fc: No bootstrap required, opened a new log
I20260709 13:24:38.172101 28571 raft_consensus.cc:3060] T 68541149575b487b8c4e2360c2c93921 P 2173e3fd92f34ad6ad8d56d2e6b05299 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:24:38.172868 28563 ts_tablet_manager.cc:1403] T 037bfc9d92244fcb872b2ca382c21794 P 542ea9e2fcff4028b6766459e90bc5fc: Time spent bootstrapping tablet: real 0.050s	user 0.029s	sys 0.009s
I20260709 13:24:38.174304 28565 tablet_bootstrap.cc:492] T 255f88c066ac422d9d8a582d9b24a958 P 2173e3fd92f34ad6ad8d56d2e6b05299: No bootstrap required, opened a new log
I20260709 13:24:38.174978 28565 ts_tablet_manager.cc:1403] T 255f88c066ac422d9d8a582d9b24a958 P 2173e3fd92f34ad6ad8d56d2e6b05299: Time spent bootstrapping tablet: real 0.065s	user 0.017s	sys 0.041s
I20260709 13:24:38.178297 28565 raft_consensus.cc:359] T 255f88c066ac422d9d8a582d9b24a958 P 2173e3fd92f34ad6ad8d56d2e6b05299 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "542ea9e2fcff4028b6766459e90bc5fc" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38409 } } peers { permanent_uuid: "96cba770f3334f79af3162475b54fa0f" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 33585 } } peers { permanent_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 37597 } }
I20260709 13:24:38.178406 28563 raft_consensus.cc:359] T 037bfc9d92244fcb872b2ca382c21794 P 542ea9e2fcff4028b6766459e90bc5fc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "542ea9e2fcff4028b6766459e90bc5fc" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38409 } } peers { permanent_uuid: "96cba770f3334f79af3162475b54fa0f" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 33585 } } peers { permanent_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 37597 } }
I20260709 13:24:38.179229 28565 raft_consensus.cc:385] T 255f88c066ac422d9d8a582d9b24a958 P 2173e3fd92f34ad6ad8d56d2e6b05299 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:24:38.179440 28563 raft_consensus.cc:385] T 037bfc9d92244fcb872b2ca382c21794 P 542ea9e2fcff4028b6766459e90bc5fc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:24:38.179653 28565 raft_consensus.cc:740] T 255f88c066ac422d9d8a582d9b24a958 P 2173e3fd92f34ad6ad8d56d2e6b05299 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2173e3fd92f34ad6ad8d56d2e6b05299, State: Initialized, Role: FOLLOWER
I20260709 13:24:38.180084 28563 raft_consensus.cc:740] T 037bfc9d92244fcb872b2ca382c21794 P 542ea9e2fcff4028b6766459e90bc5fc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 542ea9e2fcff4028b6766459e90bc5fc, State: Initialized, Role: FOLLOWER
I20260709 13:24:38.180855 28571 raft_consensus.cc:515] T 68541149575b487b8c4e2360c2c93921 P 2173e3fd92f34ad6ad8d56d2e6b05299 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 37597 } } peers { permanent_uuid: "542ea9e2fcff4028b6766459e90bc5fc" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38409 } } peers { permanent_uuid: "96cba770f3334f79af3162475b54fa0f" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 33585 } }
I20260709 13:24:38.180900 28565 consensus_queue.cc:260] T 255f88c066ac422d9d8a582d9b24a958 P 2173e3fd92f34ad6ad8d56d2e6b05299 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "542ea9e2fcff4028b6766459e90bc5fc" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38409 } } peers { permanent_uuid: "96cba770f3334f79af3162475b54fa0f" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 33585 } } peers { permanent_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 37597 } }
I20260709 13:24:38.181018 28563 consensus_queue.cc:260] T 037bfc9d92244fcb872b2ca382c21794 P 542ea9e2fcff4028b6766459e90bc5fc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "542ea9e2fcff4028b6766459e90bc5fc" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38409 } } peers { permanent_uuid: "96cba770f3334f79af3162475b54fa0f" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 33585 } } peers { permanent_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 37597 } }
I20260709 13:24:38.183970 28374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68541149575b487b8c4e2360c2c93921" candidate_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "542ea9e2fcff4028b6766459e90bc5fc"
I20260709 13:24:38.184598 28374 raft_consensus.cc:3060] T 68541149575b487b8c4e2360c2c93921 P 542ea9e2fcff4028b6766459e90bc5fc [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:24:38.185002 28563 ts_tablet_manager.cc:1434] T 037bfc9d92244fcb872b2ca382c21794 P 542ea9e2fcff4028b6766459e90bc5fc: Time spent starting tablet: real 0.011s	user 0.009s	sys 0.000s
I20260709 13:24:38.186797 28563 tablet_bootstrap.cc:492] T bbaf4eecd3c241e3aabff37f6ba0256a P 542ea9e2fcff4028b6766459e90bc5fc: Bootstrap starting.
I20260709 13:24:38.170444 28564 consensus_queue.cc:260] T a6cf33518cd34784a1b85843d90badb5 P 96cba770f3334f79af3162475b54fa0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "542ea9e2fcff4028b6766459e90bc5fc" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38409 } } peers { permanent_uuid: "96cba770f3334f79af3162475b54fa0f" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 33585 } } peers { permanent_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 37597 } }
I20260709 13:24:38.194777 28564 ts_tablet_manager.cc:1434] T a6cf33518cd34784a1b85843d90badb5 P 96cba770f3334f79af3162475b54fa0f: Time spent starting tablet: real 0.029s	user 0.006s	sys 0.001s
I20260709 13:24:38.195925 28564 tablet_bootstrap.cc:492] T b75c7d0f42e6499f936d08ff343fca5b P 96cba770f3334f79af3162475b54fa0f: Bootstrap starting.
I20260709 13:24:38.198030 28563 tablet_bootstrap.cc:654] T bbaf4eecd3c241e3aabff37f6ba0256a P 542ea9e2fcff4028b6766459e90bc5fc: Neither blocks nor log segments found. Creating new log.
I20260709 13:24:38.202778 28564 tablet_bootstrap.cc:654] T b75c7d0f42e6499f936d08ff343fca5b P 96cba770f3334f79af3162475b54fa0f: Neither blocks nor log segments found. Creating new log.
W20260709 13:24:38.204629 28549 tablet.cc:2406] T 68541149575b487b8c4e2360c2c93921 P 2173e3fd92f34ad6ad8d56d2e6b05299: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260709 13:24:38.206388 28374 raft_consensus.cc:2468] T 68541149575b487b8c4e2360c2c93921 P 542ea9e2fcff4028b6766459e90bc5fc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2173e3fd92f34ad6ad8d56d2e6b05299 in term 1.
I20260709 13:24:38.207798 28487 leader_election.cc:304] T 68541149575b487b8c4e2360c2c93921 P 2173e3fd92f34ad6ad8d56d2e6b05299 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2173e3fd92f34ad6ad8d56d2e6b05299, 542ea9e2fcff4028b6766459e90bc5fc; no voters: 
I20260709 13:24:38.212355 28565 ts_tablet_manager.cc:1434] T 255f88c066ac422d9d8a582d9b24a958 P 2173e3fd92f34ad6ad8d56d2e6b05299: Time spent starting tablet: real 0.037s	user 0.015s	sys 0.021s
I20260709 13:24:38.212827 28563 tablet_bootstrap.cc:492] T bbaf4eecd3c241e3aabff37f6ba0256a P 542ea9e2fcff4028b6766459e90bc5fc: No bootstrap required, opened a new log
I20260709 13:24:38.213541 28563 ts_tablet_manager.cc:1403] T bbaf4eecd3c241e3aabff37f6ba0256a P 542ea9e2fcff4028b6766459e90bc5fc: Time spent bootstrapping tablet: real 0.027s	user 0.019s	sys 0.005s
I20260709 13:24:38.213704 28565 tablet_bootstrap.cc:492] T a6cf33518cd34784a1b85843d90badb5 P 2173e3fd92f34ad6ad8d56d2e6b05299: Bootstrap starting.
I20260709 13:24:38.214915 28449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68541149575b487b8c4e2360c2c93921" candidate_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96cba770f3334f79af3162475b54fa0f"
W20260709 13:24:38.216514 28489 leader_election.cc:343] T 68541149575b487b8c4e2360c2c93921 P 2173e3fd92f34ad6ad8d56d2e6b05299 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 96cba770f3334f79af3162475b54fa0f (127.14.177.130:33585): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:24:38.217701 28563 raft_consensus.cc:359] T bbaf4eecd3c241e3aabff37f6ba0256a P 542ea9e2fcff4028b6766459e90bc5fc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "542ea9e2fcff4028b6766459e90bc5fc" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38409 } } peers { permanent_uuid: "96cba770f3334f79af3162475b54fa0f" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 33585 } } peers { permanent_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 37597 } }
I20260709 13:24:38.218874 28563 raft_consensus.cc:385] T bbaf4eecd3c241e3aabff37f6ba0256a P 542ea9e2fcff4028b6766459e90bc5fc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:24:38.219364 28563 raft_consensus.cc:740] T bbaf4eecd3c241e3aabff37f6ba0256a P 542ea9e2fcff4028b6766459e90bc5fc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 542ea9e2fcff4028b6766459e90bc5fc, State: Initialized, Role: FOLLOWER
I20260709 13:24:38.220433 28563 consensus_queue.cc:260] T bbaf4eecd3c241e3aabff37f6ba0256a P 542ea9e2fcff4028b6766459e90bc5fc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "542ea9e2fcff4028b6766459e90bc5fc" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38409 } } peers { permanent_uuid: "96cba770f3334f79af3162475b54fa0f" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 33585 } } peers { permanent_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 37597 } }
I20260709 13:24:38.221650 28579 raft_consensus.cc:2804] T 68541149575b487b8c4e2360c2c93921 P 2173e3fd92f34ad6ad8d56d2e6b05299 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:24:38.222057 28565 tablet_bootstrap.cc:654] T a6cf33518cd34784a1b85843d90badb5 P 2173e3fd92f34ad6ad8d56d2e6b05299: Neither blocks nor log segments found. Creating new log.
I20260709 13:24:38.217675 28571 leader_election.cc:290] T 68541149575b487b8c4e2360c2c93921 P 2173e3fd92f34ad6ad8d56d2e6b05299 [CANDIDATE]: Term 1 election: Requested vote from peers 542ea9e2fcff4028b6766459e90bc5fc (127.14.177.129:38409), 96cba770f3334f79af3162475b54fa0f (127.14.177.130:33585)
I20260709 13:24:38.224896 28563 ts_tablet_manager.cc:1434] T bbaf4eecd3c241e3aabff37f6ba0256a P 542ea9e2fcff4028b6766459e90bc5fc: Time spent starting tablet: real 0.011s	user 0.007s	sys 0.000s
I20260709 13:24:38.226142 28579 raft_consensus.cc:697] T 68541149575b487b8c4e2360c2c93921 P 2173e3fd92f34ad6ad8d56d2e6b05299 [term 1 LEADER]: Becoming Leader. State: Replica: 2173e3fd92f34ad6ad8d56d2e6b05299, State: Running, Role: LEADER
I20260709 13:24:38.226696 28563 tablet_bootstrap.cc:492] T 41515f53db8540a8b4f52d9ac91dab50 P 542ea9e2fcff4028b6766459e90bc5fc: Bootstrap starting.
I20260709 13:24:38.227097 28579 consensus_queue.cc:237] T 68541149575b487b8c4e2360c2c93921 P 2173e3fd92f34ad6ad8d56d2e6b05299 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 37597 } } peers { permanent_uuid: "542ea9e2fcff4028b6766459e90bc5fc" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38409 } } peers { permanent_uuid: "96cba770f3334f79af3162475b54fa0f" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 33585 } }
I20260709 13:24:38.234740 28564 tablet_bootstrap.cc:492] T b75c7d0f42e6499f936d08ff343fca5b P 96cba770f3334f79af3162475b54fa0f: No bootstrap required, opened a new log
I20260709 13:24:38.235440 28564 ts_tablet_manager.cc:1403] T b75c7d0f42e6499f936d08ff343fca5b P 96cba770f3334f79af3162475b54fa0f: Time spent bootstrapping tablet: real 0.040s	user 0.008s	sys 0.004s
I20260709 13:24:38.238577 28563 tablet_bootstrap.cc:654] T 41515f53db8540a8b4f52d9ac91dab50 P 542ea9e2fcff4028b6766459e90bc5fc: Neither blocks nor log segments found. Creating new log.
I20260709 13:24:38.240798 28567 raft_consensus.cc:493] T 73a4db7e3a924c6f99b416e6dda95c51 P 542ea9e2fcff4028b6766459e90bc5fc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:24:38.241484 28567 raft_consensus.cc:515] T 73a4db7e3a924c6f99b416e6dda95c51 P 542ea9e2fcff4028b6766459e90bc5fc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "542ea9e2fcff4028b6766459e90bc5fc" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38409 } } peers { permanent_uuid: "96cba770f3334f79af3162475b54fa0f" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 33585 } } peers { permanent_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 37597 } }
I20260709 13:24:38.245263 28567 leader_election.cc:290] T 73a4db7e3a924c6f99b416e6dda95c51 P 542ea9e2fcff4028b6766459e90bc5fc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 96cba770f3334f79af3162475b54fa0f (127.14.177.130:33585), 2173e3fd92f34ad6ad8d56d2e6b05299 (127.14.177.131:37597)
I20260709 13:24:38.242146 28564 raft_consensus.cc:359] T b75c7d0f42e6499f936d08ff343fca5b P 96cba770f3334f79af3162475b54fa0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "542ea9e2fcff4028b6766459e90bc5fc" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38409 } } peers { permanent_uuid: "96cba770f3334f79af3162475b54fa0f" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 33585 } } peers { permanent_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 37597 } }
I20260709 13:24:38.247859 28564 raft_consensus.cc:385] T b75c7d0f42e6499f936d08ff343fca5b P 96cba770f3334f79af3162475b54fa0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:24:38.248226 28564 raft_consensus.cc:740] T b75c7d0f42e6499f936d08ff343fca5b P 96cba770f3334f79af3162475b54fa0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 96cba770f3334f79af3162475b54fa0f, State: Initialized, Role: FOLLOWER
I20260709 13:24:38.248741 28565 tablet_bootstrap.cc:492] T a6cf33518cd34784a1b85843d90badb5 P 2173e3fd92f34ad6ad8d56d2e6b05299: No bootstrap required, opened a new log
I20260709 13:24:38.249361 28565 ts_tablet_manager.cc:1403] T a6cf33518cd34784a1b85843d90badb5 P 2173e3fd92f34ad6ad8d56d2e6b05299: Time spent bootstrapping tablet: real 0.036s	user 0.020s	sys 0.007s
I20260709 13:24:38.249346 28564 consensus_queue.cc:260] T b75c7d0f42e6499f936d08ff343fca5b P 96cba770f3334f79af3162475b54fa0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "542ea9e2fcff4028b6766459e90bc5fc" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38409 } } peers { permanent_uuid: "96cba770f3334f79af3162475b54fa0f" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 33585 } } peers { permanent_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 37597 } }
I20260709 13:24:38.251600 28564 ts_tablet_manager.cc:1434] T b75c7d0f42e6499f936d08ff343fca5b P 96cba770f3334f79af3162475b54fa0f: Time spent starting tablet: real 0.016s	user 0.007s	sys 0.001s
I20260709 13:24:38.252557 28564 tablet_bootstrap.cc:492] T bbaf4eecd3c241e3aabff37f6ba0256a P 96cba770f3334f79af3162475b54fa0f: Bootstrap starting.
I20260709 13:24:38.252856 28565 raft_consensus.cc:359] T a6cf33518cd34784a1b85843d90badb5 P 2173e3fd92f34ad6ad8d56d2e6b05299 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "542ea9e2fcff4028b6766459e90bc5fc" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38409 } } peers { permanent_uuid: "96cba770f3334f79af3162475b54fa0f" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 33585 } } peers { permanent_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 37597 } }
I20260709 13:24:38.253849 28565 raft_consensus.cc:385] T a6cf33518cd34784a1b85843d90badb5 P 2173e3fd92f34ad6ad8d56d2e6b05299 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:24:38.254251 28565 raft_consensus.cc:740] T a6cf33518cd34784a1b85843d90badb5 P 2173e3fd92f34ad6ad8d56d2e6b05299 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2173e3fd92f34ad6ad8d56d2e6b05299, State: Initialized, Role: FOLLOWER
I20260709 13:24:38.255486 28565 consensus_queue.cc:260] T a6cf33518cd34784a1b85843d90badb5 P 2173e3fd92f34ad6ad8d56d2e6b05299 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "542ea9e2fcff4028b6766459e90bc5fc" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38409 } } peers { permanent_uuid: "96cba770f3334f79af3162475b54fa0f" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 33585 } } peers { permanent_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 37597 } }
I20260709 13:24:38.258072 28565 ts_tablet_manager.cc:1434] T a6cf33518cd34784a1b85843d90badb5 P 2173e3fd92f34ad6ad8d56d2e6b05299: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 13:24:38.261262 28565 tablet_bootstrap.cc:492] T 41515f53db8540a8b4f52d9ac91dab50 P 2173e3fd92f34ad6ad8d56d2e6b05299: Bootstrap starting.
I20260709 13:24:38.262693 28564 tablet_bootstrap.cc:654] T bbaf4eecd3c241e3aabff37f6ba0256a P 96cba770f3334f79af3162475b54fa0f: Neither blocks nor log segments found. Creating new log.
I20260709 13:24:38.258347 28272 catalog_manager.cc:5697] T 68541149575b487b8c4e2360c2c93921 P 2173e3fd92f34ad6ad8d56d2e6b05299 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2173e3fd92f34ad6ad8d56d2e6b05299 (127.14.177.131). New cstate: current_term: 1 leader_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 37597 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "542ea9e2fcff4028b6766459e90bc5fc" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38409 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "96cba770f3334f79af3162475b54fa0f" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 33585 } health_report { overall_health: UNKNOWN } } }
I20260709 13:24:38.269619 28565 tablet_bootstrap.cc:654] T 41515f53db8540a8b4f52d9ac91dab50 P 2173e3fd92f34ad6ad8d56d2e6b05299: Neither blocks nor log segments found. Creating new log.
I20260709 13:24:38.273484 28567 raft_consensus.cc:493] T 037bfc9d92244fcb872b2ca382c21794 P 542ea9e2fcff4028b6766459e90bc5fc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:24:38.291720 28589 raft_consensus.cc:493] T bbaf4eecd3c241e3aabff37f6ba0256a P 542ea9e2fcff4028b6766459e90bc5fc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:24:38.293597 28563 tablet_bootstrap.cc:492] T 41515f53db8540a8b4f52d9ac91dab50 P 542ea9e2fcff4028b6766459e90bc5fc: No bootstrap required, opened a new log
I20260709 13:24:38.296492 28589 raft_consensus.cc:515] T bbaf4eecd3c241e3aabff37f6ba0256a P 542ea9e2fcff4028b6766459e90bc5fc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "542ea9e2fcff4028b6766459e90bc5fc" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38409 } } peers { permanent_uuid: "96cba770f3334f79af3162475b54fa0f" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 33585 } } peers { permanent_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 37597 } }
I20260709 13:24:38.296970 28563 ts_tablet_manager.cc:1403] T 41515f53db8540a8b4f52d9ac91dab50 P 542ea9e2fcff4028b6766459e90bc5fc: Time spent bootstrapping tablet: real 0.071s	user 0.018s	sys 0.003s
I20260709 13:24:38.294828 28565 tablet_bootstrap.cc:492] T 41515f53db8540a8b4f52d9ac91dab50 P 2173e3fd92f34ad6ad8d56d2e6b05299: No bootstrap required, opened a new log
I20260709 13:24:38.297137 28449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73a4db7e3a924c6f99b416e6dda95c51" candidate_uuid: "542ea9e2fcff4028b6766459e90bc5fc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96cba770f3334f79af3162475b54fa0f" is_pre_election: true
I20260709 13:24:38.299031 28565 ts_tablet_manager.cc:1403] T 41515f53db8540a8b4f52d9ac91dab50 P 2173e3fd92f34ad6ad8d56d2e6b05299: Time spent bootstrapping tablet: real 0.040s	user 0.019s	sys 0.011s
I20260709 13:24:38.294988 28564 tablet_bootstrap.cc:492] T bbaf4eecd3c241e3aabff37f6ba0256a P 96cba770f3334f79af3162475b54fa0f: No bootstrap required, opened a new log
I20260709 13:24:38.298971 28449 raft_consensus.cc:2468] T 73a4db7e3a924c6f99b416e6dda95c51 P 96cba770f3334f79af3162475b54fa0f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 542ea9e2fcff4028b6766459e90bc5fc in term 0.
I20260709 13:24:38.292063 28567 raft_consensus.cc:515] T 037bfc9d92244fcb872b2ca382c21794 P 542ea9e2fcff4028b6766459e90bc5fc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "542ea9e2fcff4028b6766459e90bc5fc" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38409 } } peers { permanent_uuid: "96cba770f3334f79af3162475b54fa0f" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 33585 } } peers { permanent_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 37597 } }
I20260709 13:24:38.299989 28563 raft_consensus.cc:359] T 41515f53db8540a8b4f52d9ac91dab50 P 542ea9e2fcff4028b6766459e90bc5fc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "542ea9e2fcff4028b6766459e90bc5fc" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38409 } } peers { permanent_uuid: "96cba770f3334f79af3162475b54fa0f" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 33585 } } peers { permanent_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 37597 } }
I20260709 13:24:38.300794 28563 raft_consensus.cc:385] T 41515f53db8540a8b4f52d9ac91dab50 P 542ea9e2fcff4028b6766459e90bc5fc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:24:38.301121 28563 raft_consensus.cc:740] T 41515f53db8540a8b4f52d9ac91dab50 P 542ea9e2fcff4028b6766459e90bc5fc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 542ea9e2fcff4028b6766459e90bc5fc, State: Initialized, Role: FOLLOWER
I20260709 13:24:38.302059 28567 leader_election.cc:290] T 037bfc9d92244fcb872b2ca382c21794 P 542ea9e2fcff4028b6766459e90bc5fc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 96cba770f3334f79af3162475b54fa0f (127.14.177.130:33585), 2173e3fd92f34ad6ad8d56d2e6b05299 (127.14.177.131:37597)
I20260709 13:24:38.301900 28563 consensus_queue.cc:260] T 41515f53db8540a8b4f52d9ac91dab50 P 542ea9e2fcff4028b6766459e90bc5fc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "542ea9e2fcff4028b6766459e90bc5fc" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38409 } } peers { permanent_uuid: "96cba770f3334f79af3162475b54fa0f" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 33585 } } peers { permanent_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 37597 } }
I20260709 13:24:38.302839 28564 ts_tablet_manager.cc:1403] T bbaf4eecd3c241e3aabff37f6ba0256a P 96cba770f3334f79af3162475b54fa0f: Time spent bootstrapping tablet: real 0.050s	user 0.014s	sys 0.017s
I20260709 13:24:38.303053 28565 raft_consensus.cc:359] T 41515f53db8540a8b4f52d9ac91dab50 P 2173e3fd92f34ad6ad8d56d2e6b05299 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "542ea9e2fcff4028b6766459e90bc5fc" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38409 } } peers { permanent_uuid: "96cba770f3334f79af3162475b54fa0f" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 33585 } } peers { permanent_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 37597 } }
I20260709 13:24:38.303936 28565 raft_consensus.cc:385] T 41515f53db8540a8b4f52d9ac91dab50 P 2173e3fd92f34ad6ad8d56d2e6b05299 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:24:38.304277 28565 raft_consensus.cc:740] T 41515f53db8540a8b4f52d9ac91dab50 P 2173e3fd92f34ad6ad8d56d2e6b05299 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2173e3fd92f34ad6ad8d56d2e6b05299, State: Initialized, Role: FOLLOWER
I20260709 13:24:38.305116 28565 consensus_queue.cc:260] T 41515f53db8540a8b4f52d9ac91dab50 P 2173e3fd92f34ad6ad8d56d2e6b05299 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "542ea9e2fcff4028b6766459e90bc5fc" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38409 } } peers { permanent_uuid: "96cba770f3334f79af3162475b54fa0f" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 33585 } } peers { permanent_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 37597 } }
I20260709 13:24:38.305971 28564 raft_consensus.cc:359] T bbaf4eecd3c241e3aabff37f6ba0256a P 96cba770f3334f79af3162475b54fa0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "542ea9e2fcff4028b6766459e90bc5fc" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38409 } } peers { permanent_uuid: "96cba770f3334f79af3162475b54fa0f" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 33585 } } peers { permanent_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 37597 } }
I20260709 13:24:38.306792 28564 raft_consensus.cc:385] T bbaf4eecd3c241e3aabff37f6ba0256a P 96cba770f3334f79af3162475b54fa0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:24:38.307106 28564 raft_consensus.cc:740] T bbaf4eecd3c241e3aabff37f6ba0256a P 96cba770f3334f79af3162475b54fa0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 96cba770f3334f79af3162475b54fa0f, State: Initialized, Role: FOLLOWER
I20260709 13:24:38.307699 28565 ts_tablet_manager.cc:1434] T 41515f53db8540a8b4f52d9ac91dab50 P 2173e3fd92f34ad6ad8d56d2e6b05299: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.002s
I20260709 13:24:38.308182 28564 consensus_queue.cc:260] T bbaf4eecd3c241e3aabff37f6ba0256a P 96cba770f3334f79af3162475b54fa0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "542ea9e2fcff4028b6766459e90bc5fc" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38409 } } peers { permanent_uuid: "96cba770f3334f79af3162475b54fa0f" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 33585 } } peers { permanent_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 37597 } }
I20260709 13:24:38.308928 28565 tablet_bootstrap.cc:492] T 0701509f07184014b94c627c93b3bd59 P 2173e3fd92f34ad6ad8d56d2e6b05299: Bootstrap starting.
I20260709 13:24:38.310382 28589 leader_election.cc:290] T bbaf4eecd3c241e3aabff37f6ba0256a P 542ea9e2fcff4028b6766459e90bc5fc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 96cba770f3334f79af3162475b54fa0f (127.14.177.130:33585), 2173e3fd92f34ad6ad8d56d2e6b05299 (127.14.177.131:37597)
I20260709 13:24:38.312639 28564 ts_tablet_manager.cc:1434] T bbaf4eecd3c241e3aabff37f6ba0256a P 96cba770f3334f79af3162475b54fa0f: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.000s
I20260709 13:24:38.313774 28564 tablet_bootstrap.cc:492] T 255f88c066ac422d9d8a582d9b24a958 P 96cba770f3334f79af3162475b54fa0f: Bootstrap starting.
I20260709 13:24:38.304272 28563 ts_tablet_manager.cc:1434] T 41515f53db8540a8b4f52d9ac91dab50 P 542ea9e2fcff4028b6766459e90bc5fc: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:24:38.316493 28563 tablet_bootstrap.cc:492] T b75c7d0f42e6499f936d08ff343fca5b P 542ea9e2fcff4028b6766459e90bc5fc: Bootstrap starting.
I20260709 13:24:38.320747 28564 tablet_bootstrap.cc:654] T 255f88c066ac422d9d8a582d9b24a958 P 96cba770f3334f79af3162475b54fa0f: Neither blocks nor log segments found. Creating new log.
I20260709 13:24:38.322474 28449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "037bfc9d92244fcb872b2ca382c21794" candidate_uuid: "542ea9e2fcff4028b6766459e90bc5fc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96cba770f3334f79af3162475b54fa0f" is_pre_election: true
I20260709 13:24:38.323980 28448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbaf4eecd3c241e3aabff37f6ba0256a" candidate_uuid: "542ea9e2fcff4028b6766459e90bc5fc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96cba770f3334f79af3162475b54fa0f" is_pre_election: true
I20260709 13:24:38.324329 28565 tablet_bootstrap.cc:654] T 0701509f07184014b94c627c93b3bd59 P 2173e3fd92f34ad6ad8d56d2e6b05299: Neither blocks nor log segments found. Creating new log.
I20260709 13:24:38.324764 28448 raft_consensus.cc:2468] T bbaf4eecd3c241e3aabff37f6ba0256a P 96cba770f3334f79af3162475b54fa0f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 542ea9e2fcff4028b6766459e90bc5fc in term 0.
I20260709 13:24:38.325309 28340 leader_election.cc:304] T 73a4db7e3a924c6f99b416e6dda95c51 P 542ea9e2fcff4028b6766459e90bc5fc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 542ea9e2fcff4028b6766459e90bc5fc, 96cba770f3334f79af3162475b54fa0f; no voters: 
W20260709 13:24:38.326936 28320 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:24:38.328154 28563 tablet_bootstrap.cc:654] T b75c7d0f42e6499f936d08ff343fca5b P 542ea9e2fcff4028b6766459e90bc5fc: Neither blocks nor log segments found. Creating new log.
I20260709 13:24:38.337505 28564 tablet_bootstrap.cc:492] T 255f88c066ac422d9d8a582d9b24a958 P 96cba770f3334f79af3162475b54fa0f: No bootstrap required, opened a new log
I20260709 13:24:38.338097 28564 ts_tablet_manager.cc:1403] T 255f88c066ac422d9d8a582d9b24a958 P 96cba770f3334f79af3162475b54fa0f: Time spent bootstrapping tablet: real 0.025s	user 0.010s	sys 0.005s
I20260709 13:24:38.339488 28567 raft_consensus.cc:2804] T 73a4db7e3a924c6f99b416e6dda95c51 P 542ea9e2fcff4028b6766459e90bc5fc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:24:38.340163 28567 raft_consensus.cc:493] T 73a4db7e3a924c6f99b416e6dda95c51 P 542ea9e2fcff4028b6766459e90bc5fc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:24:38.340543 28523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73a4db7e3a924c6f99b416e6dda95c51" candidate_uuid: "542ea9e2fcff4028b6766459e90bc5fc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" is_pre_election: true
I20260709 13:24:38.340659 28567 raft_consensus.cc:3060] T 73a4db7e3a924c6f99b416e6dda95c51 P 542ea9e2fcff4028b6766459e90bc5fc [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:24:38.341465 28564 raft_consensus.cc:359] T 255f88c066ac422d9d8a582d9b24a958 P 96cba770f3334f79af3162475b54fa0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "542ea9e2fcff4028b6766459e90bc5fc" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38409 } } peers { permanent_uuid: "96cba770f3334f79af3162475b54fa0f" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 33585 } } peers { permanent_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 37597 } }
I20260709 13:24:38.342424 28564 raft_consensus.cc:385] T 255f88c066ac422d9d8a582d9b24a958 P 96cba770f3334f79af3162475b54fa0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:24:38.342702 28564 raft_consensus.cc:740] T 255f88c066ac422d9d8a582d9b24a958 P 96cba770f3334f79af3162475b54fa0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 96cba770f3334f79af3162475b54fa0f, State: Initialized, Role: FOLLOWER
I20260709 13:24:38.342933 28523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "037bfc9d92244fcb872b2ca382c21794" candidate_uuid: "542ea9e2fcff4028b6766459e90bc5fc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" is_pre_election: true
I20260709 13:24:38.343567 28564 consensus_queue.cc:260] T 255f88c066ac422d9d8a582d9b24a958 P 96cba770f3334f79af3162475b54fa0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "542ea9e2fcff4028b6766459e90bc5fc" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38409 } } peers { permanent_uuid: "96cba770f3334f79af3162475b54fa0f" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 33585 } } peers { permanent_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 37597 } }
I20260709 13:24:38.345932 28579 raft_consensus.cc:493] T 255f88c066ac422d9d8a582d9b24a958 P 2173e3fd92f34ad6ad8d56d2e6b05299 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:24:38.346175 28564 ts_tablet_manager.cc:1434] T 255f88c066ac422d9d8a582d9b24a958 P 96cba770f3334f79af3162475b54fa0f: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.005s
I20260709 13:24:38.346787 28579 raft_consensus.cc:515] T 255f88c066ac422d9d8a582d9b24a958 P 2173e3fd92f34ad6ad8d56d2e6b05299 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "542ea9e2fcff4028b6766459e90bc5fc" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38409 } } peers { permanent_uuid: "96cba770f3334f79af3162475b54fa0f" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 33585 } } peers { permanent_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 37597 } }
I20260709 13:24:38.347524 28564 tablet_bootstrap.cc:492] T 41515f53db8540a8b4f52d9ac91dab50 P 96cba770f3334f79af3162475b54fa0f: Bootstrap starting.
I20260709 13:24:38.348063 28523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbaf4eecd3c241e3aabff37f6ba0256a" candidate_uuid: "542ea9e2fcff4028b6766459e90bc5fc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" is_pre_election: true
I20260709 13:24:38.349633 28340 leader_election.cc:304] T bbaf4eecd3c241e3aabff37f6ba0256a P 542ea9e2fcff4028b6766459e90bc5fc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 542ea9e2fcff4028b6766459e90bc5fc, 96cba770f3334f79af3162475b54fa0f; no voters: 
I20260709 13:24:38.350411 28589 raft_consensus.cc:2804] T bbaf4eecd3c241e3aabff37f6ba0256a P 542ea9e2fcff4028b6766459e90bc5fc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:24:38.350764 28589 raft_consensus.cc:493] T bbaf4eecd3c241e3aabff37f6ba0256a P 542ea9e2fcff4028b6766459e90bc5fc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:24:38.351176 28589 raft_consensus.cc:3060] T bbaf4eecd3c241e3aabff37f6ba0256a P 542ea9e2fcff4028b6766459e90bc5fc [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:24:38.352968 28577 raft_consensus.cc:493] T 41515f53db8540a8b4f52d9ac91dab50 P 2173e3fd92f34ad6ad8d56d2e6b05299 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:24:38.353564 28577 raft_consensus.cc:515] T 41515f53db8540a8b4f52d9ac91dab50 P 2173e3fd92f34ad6ad8d56d2e6b05299 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "542ea9e2fcff4028b6766459e90bc5fc" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38409 } } peers { permanent_uuid: "96cba770f3334f79af3162475b54fa0f" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 33585 } } peers { permanent_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 37597 } }
I20260709 13:24:38.355271 28564 tablet_bootstrap.cc:654] T 41515f53db8540a8b4f52d9ac91dab50 P 96cba770f3334f79af3162475b54fa0f: Neither blocks nor log segments found. Creating new log.
I20260709 13:24:38.356192 28577 leader_election.cc:290] T 41515f53db8540a8b4f52d9ac91dab50 P 2173e3fd92f34ad6ad8d56d2e6b05299 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 542ea9e2fcff4028b6766459e90bc5fc (127.14.177.129:38409), 96cba770f3334f79af3162475b54fa0f (127.14.177.130:33585)
I20260709 13:24:38.356382 28449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255f88c066ac422d9d8a582d9b24a958" candidate_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96cba770f3334f79af3162475b54fa0f" is_pre_election: true
I20260709 13:24:38.357270 28449 raft_consensus.cc:2468] T 255f88c066ac422d9d8a582d9b24a958 P 96cba770f3334f79af3162475b54fa0f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2173e3fd92f34ad6ad8d56d2e6b05299 in term 0.
I20260709 13:24:38.359422 28579 leader_election.cc:290] T 255f88c066ac422d9d8a582d9b24a958 P 2173e3fd92f34ad6ad8d56d2e6b05299 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 542ea9e2fcff4028b6766459e90bc5fc (127.14.177.129:38409), 96cba770f3334f79af3162475b54fa0f (127.14.177.130:33585)
I20260709 13:24:38.369701 28563 tablet_bootstrap.cc:492] T b75c7d0f42e6499f936d08ff343fca5b P 542ea9e2fcff4028b6766459e90bc5fc: No bootstrap required, opened a new log
I20260709 13:24:38.370718 28563 ts_tablet_manager.cc:1403] T b75c7d0f42e6499f936d08ff343fca5b P 542ea9e2fcff4028b6766459e90bc5fc: Time spent bootstrapping tablet: real 0.054s	user 0.014s	sys 0.000s
I20260709 13:24:38.371444 28489 leader_election.cc:304] T 255f88c066ac422d9d8a582d9b24a958 P 2173e3fd92f34ad6ad8d56d2e6b05299 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2173e3fd92f34ad6ad8d56d2e6b05299, 96cba770f3334f79af3162475b54fa0f; no voters: 
I20260709 13:24:38.373032 28579 raft_consensus.cc:2804] T 255f88c066ac422d9d8a582d9b24a958 P 2173e3fd92f34ad6ad8d56d2e6b05299 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:24:38.373594 28579 raft_consensus.cc:493] T 255f88c066ac422d9d8a582d9b24a958 P 2173e3fd92f34ad6ad8d56d2e6b05299 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:24:38.374068 28579 raft_consensus.cc:3060] T 255f88c066ac422d9d8a582d9b24a958 P 2173e3fd92f34ad6ad8d56d2e6b05299 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:24:38.374900 28563 raft_consensus.cc:359] T b75c7d0f42e6499f936d08ff343fca5b P 542ea9e2fcff4028b6766459e90bc5fc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "542ea9e2fcff4028b6766459e90bc5fc" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38409 } } peers { permanent_uuid: "96cba770f3334f79af3162475b54fa0f" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 33585 } } peers { permanent_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 37597 } }
I20260709 13:24:38.376008 28563 raft_consensus.cc:385] T b75c7d0f42e6499f936d08ff343fca5b P 542ea9e2fcff4028b6766459e90bc5fc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:24:38.376396 28563 raft_consensus.cc:740] T b75c7d0f42e6499f936d08ff343fca5b P 542ea9e2fcff4028b6766459e90bc5fc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 542ea9e2fcff4028b6766459e90bc5fc, State: Initialized, Role: FOLLOWER
I20260709 13:24:38.377377 28563 consensus_queue.cc:260] T b75c7d0f42e6499f936d08ff343fca5b P 542ea9e2fcff4028b6766459e90bc5fc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "542ea9e2fcff4028b6766459e90bc5fc" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38409 } } peers { permanent_uuid: "96cba770f3334f79af3162475b54fa0f" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 33585 } } peers { permanent_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 37597 } }
I20260709 13:24:38.383242 28577 raft_consensus.cc:493] T a6cf33518cd34784a1b85843d90badb5 P 2173e3fd92f34ad6ad8d56d2e6b05299 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:24:38.383872 28577 raft_consensus.cc:515] T a6cf33518cd34784a1b85843d90badb5 P 2173e3fd92f34ad6ad8d56d2e6b05299 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "542ea9e2fcff4028b6766459e90bc5fc" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38409 } } peers { permanent_uuid: "96cba770f3334f79af3162475b54fa0f" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 33585 } } peers { permanent_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 37597 } }
I20260709 13:24:38.385361 28374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255f88c066ac422d9d8a582d9b24a958" candidate_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "542ea9e2fcff4028b6766459e90bc5fc" is_pre_election: true
I20260709 13:24:38.386842 28577 leader_election.cc:290] T a6cf33518cd34784a1b85843d90badb5 P 2173e3fd92f34ad6ad8d56d2e6b05299 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 542ea9e2fcff4028b6766459e90bc5fc (127.14.177.129:38409), 96cba770f3334f79af3162475b54fa0f (127.14.177.130:33585)
I20260709 13:24:38.386982 28373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41515f53db8540a8b4f52d9ac91dab50" candidate_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "542ea9e2fcff4028b6766459e90bc5fc" is_pre_election: true
I20260709 13:24:38.387939 28373 raft_consensus.cc:2468] T 41515f53db8540a8b4f52d9ac91dab50 P 542ea9e2fcff4028b6766459e90bc5fc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2173e3fd92f34ad6ad8d56d2e6b05299 in term 0.
W20260709 13:24:38.383304 28340 leader_election.cc:343] T 037bfc9d92244fcb872b2ca382c21794 P 542ea9e2fcff4028b6766459e90bc5fc [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 96cba770f3334f79af3162475b54fa0f (127.14.177.130:33585): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:24:38.395506 28449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41515f53db8540a8b4f52d9ac91dab50" candidate_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96cba770f3334f79af3162475b54fa0f" is_pre_election: true
I20260709 13:24:38.396912 28373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6cf33518cd34784a1b85843d90badb5" candidate_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "542ea9e2fcff4028b6766459e90bc5fc" is_pre_election: true
I20260709 13:24:38.397037 28448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6cf33518cd34784a1b85843d90badb5" candidate_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96cba770f3334f79af3162475b54fa0f" is_pre_election: true
I20260709 13:24:38.397881 28448 raft_consensus.cc:2468] T a6cf33518cd34784a1b85843d90badb5 P 96cba770f3334f79af3162475b54fa0f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2173e3fd92f34ad6ad8d56d2e6b05299 in term 0.
I20260709 13:24:38.398186 28563 ts_tablet_manager.cc:1434] T b75c7d0f42e6499f936d08ff343fca5b P 542ea9e2fcff4028b6766459e90bc5fc: Time spent starting tablet: real 0.027s	user 0.006s	sys 0.011s
I20260709 13:24:38.399400 28563 tablet_bootstrap.cc:492] T 255f88c066ac422d9d8a582d9b24a958 P 542ea9e2fcff4028b6766459e90bc5fc: Bootstrap starting.
W20260709 13:24:38.402113 28340 leader_election.cc:343] T 73a4db7e3a924c6f99b416e6dda95c51 P 542ea9e2fcff4028b6766459e90bc5fc [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2173e3fd92f34ad6ad8d56d2e6b05299 (127.14.177.131:37597): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:24:38.403502 28340 leader_election.cc:343] T 037bfc9d92244fcb872b2ca382c21794 P 542ea9e2fcff4028b6766459e90bc5fc [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2173e3fd92f34ad6ad8d56d2e6b05299 (127.14.177.131:37597): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:24:38.404023 28340 leader_election.cc:304] T 037bfc9d92244fcb872b2ca382c21794 P 542ea9e2fcff4028b6766459e90bc5fc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 542ea9e2fcff4028b6766459e90bc5fc; no voters: 2173e3fd92f34ad6ad8d56d2e6b05299, 96cba770f3334f79af3162475b54fa0f
I20260709 13:24:38.382812 28567 raft_consensus.cc:515] T 73a4db7e3a924c6f99b416e6dda95c51 P 542ea9e2fcff4028b6766459e90bc5fc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "542ea9e2fcff4028b6766459e90bc5fc" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38409 } } peers { permanent_uuid: "96cba770f3334f79af3162475b54fa0f" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 33585 } } peers { permanent_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 37597 } }
I20260709 13:24:38.405686 28589 raft_consensus.cc:515] T bbaf4eecd3c241e3aabff37f6ba0256a P 542ea9e2fcff4028b6766459e90bc5fc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "542ea9e2fcff4028b6766459e90bc5fc" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38409 } } peers { permanent_uuid: "96cba770f3334f79af3162475b54fa0f" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 33585 } } peers { permanent_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 37597 } }
W20260709 13:24:38.406613 28487 leader_election.cc:343] T 255f88c066ac422d9d8a582d9b24a958 P 2173e3fd92f34ad6ad8d56d2e6b05299 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 542ea9e2fcff4028b6766459e90bc5fc (127.14.177.129:38409): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:24:38.408154 28589 leader_election.cc:290] T bbaf4eecd3c241e3aabff37f6ba0256a P 542ea9e2fcff4028b6766459e90bc5fc [CANDIDATE]: Term 1 election: Requested vote from peers 96cba770f3334f79af3162475b54fa0f (127.14.177.130:33585), 2173e3fd92f34ad6ad8d56d2e6b05299 (127.14.177.131:37597)
I20260709 13:24:38.407279 28489 leader_election.cc:304] T a6cf33518cd34784a1b85843d90badb5 P 2173e3fd92f34ad6ad8d56d2e6b05299 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2173e3fd92f34ad6ad8d56d2e6b05299, 96cba770f3334f79af3162475b54fa0f; no voters: 
I20260709 13:24:38.409368 28589 raft_consensus.cc:2749] T 037bfc9d92244fcb872b2ca382c21794 P 542ea9e2fcff4028b6766459e90bc5fc [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:24:38.409555 28577 raft_consensus.cc:2804] T a6cf33518cd34784a1b85843d90badb5 P 2173e3fd92f34ad6ad8d56d2e6b05299 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:24:38.410022 28577 raft_consensus.cc:493] T a6cf33518cd34784a1b85843d90badb5 P 2173e3fd92f34ad6ad8d56d2e6b05299 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:24:38.410348 28577 raft_consensus.cc:3060] T a6cf33518cd34784a1b85843d90badb5 P 2173e3fd92f34ad6ad8d56d2e6b05299 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:24:38.412951 28563 tablet_bootstrap.cc:654] T 255f88c066ac422d9d8a582d9b24a958 P 542ea9e2fcff4028b6766459e90bc5fc: Neither blocks nor log segments found. Creating new log.
I20260709 13:24:38.414072 28487 leader_election.cc:304] T 41515f53db8540a8b4f52d9ac91dab50 P 2173e3fd92f34ad6ad8d56d2e6b05299 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2173e3fd92f34ad6ad8d56d2e6b05299, 542ea9e2fcff4028b6766459e90bc5fc; no voters: 
I20260709 13:24:38.415609 28580 raft_consensus.cc:2804] T 41515f53db8540a8b4f52d9ac91dab50 P 2173e3fd92f34ad6ad8d56d2e6b05299 [term 0 FOLLOWER]: Leader pre-election won for term 1
W20260709 13:24:38.415836 28487 leader_election.cc:343] T a6cf33518cd34784a1b85843d90badb5 P 2173e3fd92f34ad6ad8d56d2e6b05299 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 542ea9e2fcff4028b6766459e90bc5fc (127.14.177.129:38409): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:24:38.416008 28580 raft_consensus.cc:493] T 41515f53db8540a8b4f52d9ac91dab50 P 2173e3fd92f34ad6ad8d56d2e6b05299 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:24:38.416625 28580 raft_consensus.cc:3060] T 41515f53db8540a8b4f52d9ac91dab50 P 2173e3fd92f34ad6ad8d56d2e6b05299 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:24:38.417487 28567 leader_election.cc:290] T 73a4db7e3a924c6f99b416e6dda95c51 P 542ea9e2fcff4028b6766459e90bc5fc [CANDIDATE]: Term 1 election: Requested vote from peers 96cba770f3334f79af3162475b54fa0f (127.14.177.130:33585), 2173e3fd92f34ad6ad8d56d2e6b05299 (127.14.177.131:37597)
I20260709 13:24:38.420713 28565 tablet_bootstrap.cc:492] T 0701509f07184014b94c627c93b3bd59 P 2173e3fd92f34ad6ad8d56d2e6b05299: No bootstrap required, opened a new log
W20260709 13:24:38.421090 28489 leader_election.cc:343] T 41515f53db8540a8b4f52d9ac91dab50 P 2173e3fd92f34ad6ad8d56d2e6b05299 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 96cba770f3334f79af3162475b54fa0f (127.14.177.130:33585): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:24:38.421423 28565 ts_tablet_manager.cc:1403] T 0701509f07184014b94c627c93b3bd59 P 2173e3fd92f34ad6ad8d56d2e6b05299: Time spent bootstrapping tablet: real 0.113s	user 0.026s	sys 0.043s
I20260709 13:24:38.418043 28579 raft_consensus.cc:515] T 255f88c066ac422d9d8a582d9b24a958 P 2173e3fd92f34ad6ad8d56d2e6b05299 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "542ea9e2fcff4028b6766459e90bc5fc" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38409 } } peers { permanent_uuid: "96cba770f3334f79af3162475b54fa0f" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 33585 } } peers { permanent_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 37597 } }
I20260709 13:24:38.425067 28565 raft_consensus.cc:359] T 0701509f07184014b94c627c93b3bd59 P 2173e3fd92f34ad6ad8d56d2e6b05299 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "542ea9e2fcff4028b6766459e90bc5fc" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38409 } } peers { permanent_uuid: "96cba770f3334f79af3162475b54fa0f" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 33585 } } peers { permanent_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 37597 } }
I20260709 13:24:38.425997 28565 raft_consensus.cc:385] T 0701509f07184014b94c627c93b3bd59 P 2173e3fd92f34ad6ad8d56d2e6b05299 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:24:38.426378 28565 raft_consensus.cc:740] T 0701509f07184014b94c627c93b3bd59 P 2173e3fd92f34ad6ad8d56d2e6b05299 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2173e3fd92f34ad6ad8d56d2e6b05299, State: Initialized, Role: FOLLOWER
I20260709 13:24:38.426604 28580 raft_consensus.cc:515] T 41515f53db8540a8b4f52d9ac91dab50 P 2173e3fd92f34ad6ad8d56d2e6b05299 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "542ea9e2fcff4028b6766459e90bc5fc" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38409 } } peers { permanent_uuid: "96cba770f3334f79af3162475b54fa0f" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 33585 } } peers { permanent_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 37597 } }
I20260709 13:24:38.427691 28373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255f88c066ac422d9d8a582d9b24a958" candidate_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "542ea9e2fcff4028b6766459e90bc5fc"
I20260709 13:24:38.427806 28565 consensus_queue.cc:260] T 0701509f07184014b94c627c93b3bd59 P 2173e3fd92f34ad6ad8d56d2e6b05299 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "542ea9e2fcff4028b6766459e90bc5fc" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38409 } } peers { permanent_uuid: "96cba770f3334f79af3162475b54fa0f" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 33585 } } peers { permanent_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 37597 } }
W20260709 13:24:38.430722 28340 leader_election.cc:343] T bbaf4eecd3c241e3aabff37f6ba0256a P 542ea9e2fcff4028b6766459e90bc5fc [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2173e3fd92f34ad6ad8d56d2e6b05299 (127.14.177.131:37597): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:24:38.431274 28487 leader_election.cc:343] T 255f88c066ac422d9d8a582d9b24a958 P 2173e3fd92f34ad6ad8d56d2e6b05299 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 542ea9e2fcff4028b6766459e90bc5fc (127.14.177.129:38409): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:24:38.431759 28564 tablet_bootstrap.cc:492] T 41515f53db8540a8b4f52d9ac91dab50 P 96cba770f3334f79af3162475b54fa0f: No bootstrap required, opened a new log
I20260709 13:24:38.432359 28564 ts_tablet_manager.cc:1403] T 41515f53db8540a8b4f52d9ac91dab50 P 96cba770f3334f79af3162475b54fa0f: Time spent bootstrapping tablet: real 0.085s	user 0.040s	sys 0.024s
I20260709 13:24:38.429978 28577 raft_consensus.cc:515] T a6cf33518cd34784a1b85843d90badb5 P 2173e3fd92f34ad6ad8d56d2e6b05299 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "542ea9e2fcff4028b6766459e90bc5fc" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38409 } } peers { permanent_uuid: "96cba770f3334f79af3162475b54fa0f" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 33585 } } peers { permanent_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 37597 } }
I20260709 13:24:38.433940 28373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41515f53db8540a8b4f52d9ac91dab50" candidate_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "542ea9e2fcff4028b6766459e90bc5fc"
I20260709 13:24:38.434880 28447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41515f53db8540a8b4f52d9ac91dab50" candidate_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96cba770f3334f79af3162475b54fa0f"
I20260709 13:24:38.435452 28373 raft_consensus.cc:3060] T 41515f53db8540a8b4f52d9ac91dab50 P 542ea9e2fcff4028b6766459e90bc5fc [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:24:38.435710 28449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbaf4eecd3c241e3aabff37f6ba0256a" candidate_uuid: "542ea9e2fcff4028b6766459e90bc5fc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96cba770f3334f79af3162475b54fa0f"
I20260709 13:24:38.436388 28449 raft_consensus.cc:3060] T bbaf4eecd3c241e3aabff37f6ba0256a P 96cba770f3334f79af3162475b54fa0f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:24:38.436548 28565 ts_tablet_manager.cc:1434] T 0701509f07184014b94c627c93b3bd59 P 2173e3fd92f34ad6ad8d56d2e6b05299: Time spent starting tablet: real 0.015s	user 0.007s	sys 0.000s
I20260709 13:24:38.437716 28565 tablet_bootstrap.cc:492] T 73a4db7e3a924c6f99b416e6dda95c51 P 2173e3fd92f34ad6ad8d56d2e6b05299: Bootstrap starting.
I20260709 13:24:38.437911 28564 raft_consensus.cc:359] T 41515f53db8540a8b4f52d9ac91dab50 P 96cba770f3334f79af3162475b54fa0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "542ea9e2fcff4028b6766459e90bc5fc" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38409 } } peers { permanent_uuid: "96cba770f3334f79af3162475b54fa0f" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 33585 } } peers { permanent_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 37597 } }
I20260709 13:24:38.438694 28564 raft_consensus.cc:385] T 41515f53db8540a8b4f52d9ac91dab50 P 96cba770f3334f79af3162475b54fa0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:24:38.438982 28564 raft_consensus.cc:740] T 41515f53db8540a8b4f52d9ac91dab50 P 96cba770f3334f79af3162475b54fa0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 96cba770f3334f79af3162475b54fa0f, State: Initialized, Role: FOLLOWER
I20260709 13:24:38.439754 28564 consensus_queue.cc:260] T 41515f53db8540a8b4f52d9ac91dab50 P 96cba770f3334f79af3162475b54fa0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "542ea9e2fcff4028b6766459e90bc5fc" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38409 } } peers { permanent_uuid: "96cba770f3334f79af3162475b54fa0f" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 33585 } } peers { permanent_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 37597 } }
I20260709 13:24:38.434670 28580 leader_election.cc:290] T 41515f53db8540a8b4f52d9ac91dab50 P 2173e3fd92f34ad6ad8d56d2e6b05299 [CANDIDATE]: Term 1 election: Requested vote from peers 542ea9e2fcff4028b6766459e90bc5fc (127.14.177.129:38409), 96cba770f3334f79af3162475b54fa0f (127.14.177.130:33585)
I20260709 13:24:38.434207 28448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73a4db7e3a924c6f99b416e6dda95c51" candidate_uuid: "542ea9e2fcff4028b6766459e90bc5fc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96cba770f3334f79af3162475b54fa0f"
I20260709 13:24:38.441313 28448 raft_consensus.cc:3060] T 73a4db7e3a924c6f99b416e6dda95c51 P 96cba770f3334f79af3162475b54fa0f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:24:38.442657 28523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbaf4eecd3c241e3aabff37f6ba0256a" candidate_uuid: "542ea9e2fcff4028b6766459e90bc5fc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299"
I20260709 13:24:38.443526 28564 ts_tablet_manager.cc:1434] T 41515f53db8540a8b4f52d9ac91dab50 P 96cba770f3334f79af3162475b54fa0f: Time spent starting tablet: real 0.011s	user 0.007s	sys 0.000s
I20260709 13:24:38.444628 28523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73a4db7e3a924c6f99b416e6dda95c51" candidate_uuid: "542ea9e2fcff4028b6766459e90bc5fc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299"
I20260709 13:24:38.444800 28564 tablet_bootstrap.cc:492] T 68541149575b487b8c4e2360c2c93921 P 96cba770f3334f79af3162475b54fa0f: Bootstrap starting.
W20260709 13:24:38.446696 28340 leader_election.cc:343] T bbaf4eecd3c241e3aabff37f6ba0256a P 542ea9e2fcff4028b6766459e90bc5fc [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 2173e3fd92f34ad6ad8d56d2e6b05299 (127.14.177.131:37597): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:24:38.447474 28340 leader_election.cc:343] T 73a4db7e3a924c6f99b416e6dda95c51 P 542ea9e2fcff4028b6766459e90bc5fc [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 2173e3fd92f34ad6ad8d56d2e6b05299 (127.14.177.131:37597): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:24:38.448388 28579 leader_election.cc:290] T 255f88c066ac422d9d8a582d9b24a958 P 2173e3fd92f34ad6ad8d56d2e6b05299 [CANDIDATE]: Term 1 election: Requested vote from peers 542ea9e2fcff4028b6766459e90bc5fc (127.14.177.129:38409), 96cba770f3334f79af3162475b54fa0f (127.14.177.130:33585)
W20260709 13:24:38.436474 28489 leader_election.cc:343] T 41515f53db8540a8b4f52d9ac91dab50 P 2173e3fd92f34ad6ad8d56d2e6b05299 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 96cba770f3334f79af3162475b54fa0f (127.14.177.130:33585): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:24:38.450299 28447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255f88c066ac422d9d8a582d9b24a958" candidate_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96cba770f3334f79af3162475b54fa0f"
I20260709 13:24:38.450984 28447 raft_consensus.cc:3060] T 255f88c066ac422d9d8a582d9b24a958 P 96cba770f3334f79af3162475b54fa0f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:24:38.451943 28565 tablet_bootstrap.cc:654] T 73a4db7e3a924c6f99b416e6dda95c51 P 2173e3fd92f34ad6ad8d56d2e6b05299: Neither blocks nor log segments found. Creating new log.
I20260709 13:24:38.452293 28374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6cf33518cd34784a1b85843d90badb5" candidate_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "542ea9e2fcff4028b6766459e90bc5fc"
I20260709 13:24:38.453570 28564 tablet_bootstrap.cc:654] T 68541149575b487b8c4e2360c2c93921 P 96cba770f3334f79af3162475b54fa0f: Neither blocks nor log segments found. Creating new log.
I20260709 13:24:38.454203 28577 leader_election.cc:290] T a6cf33518cd34784a1b85843d90badb5 P 2173e3fd92f34ad6ad8d56d2e6b05299 [CANDIDATE]: Term 1 election: Requested vote from peers 542ea9e2fcff4028b6766459e90bc5fc (127.14.177.129:38409), 96cba770f3334f79af3162475b54fa0f (127.14.177.130:33585)
I20260709 13:24:38.458514 28446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6cf33518cd34784a1b85843d90badb5" candidate_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96cba770f3334f79af3162475b54fa0f"
I20260709 13:24:38.459336 28446 raft_consensus.cc:3060] T a6cf33518cd34784a1b85843d90badb5 P 96cba770f3334f79af3162475b54fa0f [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:24:38.459519 28487 leader_election.cc:343] T a6cf33518cd34784a1b85843d90badb5 P 2173e3fd92f34ad6ad8d56d2e6b05299 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 542ea9e2fcff4028b6766459e90bc5fc (127.14.177.129:38409): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:24:38.467411 28446 raft_consensus.cc:2468] T a6cf33518cd34784a1b85843d90badb5 P 96cba770f3334f79af3162475b54fa0f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2173e3fd92f34ad6ad8d56d2e6b05299 in term 1.
I20260709 13:24:38.468873 28489 leader_election.cc:304] T a6cf33518cd34784a1b85843d90badb5 P 2173e3fd92f34ad6ad8d56d2e6b05299 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2173e3fd92f34ad6ad8d56d2e6b05299, 96cba770f3334f79af3162475b54fa0f; no voters: 542ea9e2fcff4028b6766459e90bc5fc
I20260709 13:24:38.468916 28373 raft_consensus.cc:2468] T 41515f53db8540a8b4f52d9ac91dab50 P 542ea9e2fcff4028b6766459e90bc5fc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2173e3fd92f34ad6ad8d56d2e6b05299 in term 1.
I20260709 13:24:38.470180 28577 raft_consensus.cc:2804] T a6cf33518cd34784a1b85843d90badb5 P 2173e3fd92f34ad6ad8d56d2e6b05299 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:24:38.471086 28487 leader_election.cc:304] T 41515f53db8540a8b4f52d9ac91dab50 P 2173e3fd92f34ad6ad8d56d2e6b05299 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2173e3fd92f34ad6ad8d56d2e6b05299, 542ea9e2fcff4028b6766459e90bc5fc; no voters: 96cba770f3334f79af3162475b54fa0f
I20260709 13:24:38.473392 28447 raft_consensus.cc:2468] T 255f88c066ac422d9d8a582d9b24a958 P 96cba770f3334f79af3162475b54fa0f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2173e3fd92f34ad6ad8d56d2e6b05299 in term 1.
I20260709 13:24:38.474682 28577 raft_consensus.cc:697] T a6cf33518cd34784a1b85843d90badb5 P 2173e3fd92f34ad6ad8d56d2e6b05299 [term 1 LEADER]: Becoming Leader. State: Replica: 2173e3fd92f34ad6ad8d56d2e6b05299, State: Running, Role: LEADER
I20260709 13:24:38.474958 28449 raft_consensus.cc:2468] T bbaf4eecd3c241e3aabff37f6ba0256a P 96cba770f3334f79af3162475b54fa0f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 542ea9e2fcff4028b6766459e90bc5fc in term 1.
I20260709 13:24:38.475844 28577 consensus_queue.cc:237] T a6cf33518cd34784a1b85843d90badb5 P 2173e3fd92f34ad6ad8d56d2e6b05299 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "542ea9e2fcff4028b6766459e90bc5fc" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38409 } } peers { permanent_uuid: "96cba770f3334f79af3162475b54fa0f" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 33585 } } peers { permanent_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 37597 } }
I20260709 13:24:38.476948 28340 leader_election.cc:304] T bbaf4eecd3c241e3aabff37f6ba0256a P 542ea9e2fcff4028b6766459e90bc5fc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 542ea9e2fcff4028b6766459e90bc5fc, 96cba770f3334f79af3162475b54fa0f; no voters: 2173e3fd92f34ad6ad8d56d2e6b05299
I20260709 13:24:38.478011 28595 raft_consensus.cc:2804] T bbaf4eecd3c241e3aabff37f6ba0256a P 542ea9e2fcff4028b6766459e90bc5fc [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:24:38.478895 28595 raft_consensus.cc:697] T bbaf4eecd3c241e3aabff37f6ba0256a P 542ea9e2fcff4028b6766459e90bc5fc [term 1 LEADER]: Becoming Leader. State: Replica: 542ea9e2fcff4028b6766459e90bc5fc, State: Running, Role: LEADER
I20260709 13:24:38.479949 28579 raft_consensus.cc:2804] T 41515f53db8540a8b4f52d9ac91dab50 P 2173e3fd92f34ad6ad8d56d2e6b05299 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:24:38.480537 28579 raft_consensus.cc:697] T 41515f53db8540a8b4f52d9ac91dab50 P 2173e3fd92f34ad6ad8d56d2e6b05299 [term 1 LEADER]: Becoming Leader. State: Replica: 2173e3fd92f34ad6ad8d56d2e6b05299, State: Running, Role: LEADER
I20260709 13:24:38.481205 28448 raft_consensus.cc:2468] T 73a4db7e3a924c6f99b416e6dda95c51 P 96cba770f3334f79af3162475b54fa0f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 542ea9e2fcff4028b6766459e90bc5fc in term 1.
I20260709 13:24:38.480326 28595 consensus_queue.cc:237] T bbaf4eecd3c241e3aabff37f6ba0256a P 542ea9e2fcff4028b6766459e90bc5fc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "542ea9e2fcff4028b6766459e90bc5fc" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38409 } } peers { permanent_uuid: "96cba770f3334f79af3162475b54fa0f" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 33585 } } peers { permanent_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 37597 } }
I20260709 13:24:38.481638 28579 consensus_queue.cc:237] T 41515f53db8540a8b4f52d9ac91dab50 P 2173e3fd92f34ad6ad8d56d2e6b05299 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "542ea9e2fcff4028b6766459e90bc5fc" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38409 } } peers { permanent_uuid: "96cba770f3334f79af3162475b54fa0f" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 33585 } } peers { permanent_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 37597 } }
I20260709 13:24:38.482600 28340 leader_election.cc:304] T 73a4db7e3a924c6f99b416e6dda95c51 P 542ea9e2fcff4028b6766459e90bc5fc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 542ea9e2fcff4028b6766459e90bc5fc, 96cba770f3334f79af3162475b54fa0f; no voters: 2173e3fd92f34ad6ad8d56d2e6b05299
I20260709 13:24:38.483853 28594 raft_consensus.cc:2804] T 73a4db7e3a924c6f99b416e6dda95c51 P 542ea9e2fcff4028b6766459e90bc5fc [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:24:38.484356 28594 raft_consensus.cc:697] T 73a4db7e3a924c6f99b416e6dda95c51 P 542ea9e2fcff4028b6766459e90bc5fc [term 1 LEADER]: Becoming Leader. State: Replica: 542ea9e2fcff4028b6766459e90bc5fc, State: Running, Role: LEADER
I20260709 13:24:38.485172 28594 consensus_queue.cc:237] T 73a4db7e3a924c6f99b416e6dda95c51 P 542ea9e2fcff4028b6766459e90bc5fc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "542ea9e2fcff4028b6766459e90bc5fc" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38409 } } peers { permanent_uuid: "96cba770f3334f79af3162475b54fa0f" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 33585 } } peers { permanent_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 37597 } }
I20260709 13:24:38.486536 28489 leader_election.cc:304] T 255f88c066ac422d9d8a582d9b24a958 P 2173e3fd92f34ad6ad8d56d2e6b05299 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2173e3fd92f34ad6ad8d56d2e6b05299, 96cba770f3334f79af3162475b54fa0f; no voters: 542ea9e2fcff4028b6766459e90bc5fc
I20260709 13:24:38.488098 28580 raft_consensus.cc:2804] T 255f88c066ac422d9d8a582d9b24a958 P 2173e3fd92f34ad6ad8d56d2e6b05299 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:24:38.488582 28580 raft_consensus.cc:697] T 255f88c066ac422d9d8a582d9b24a958 P 2173e3fd92f34ad6ad8d56d2e6b05299 [term 1 LEADER]: Becoming Leader. State: Replica: 2173e3fd92f34ad6ad8d56d2e6b05299, State: Running, Role: LEADER
I20260709 13:24:38.489498 28580 consensus_queue.cc:237] T 255f88c066ac422d9d8a582d9b24a958 P 2173e3fd92f34ad6ad8d56d2e6b05299 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "542ea9e2fcff4028b6766459e90bc5fc" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38409 } } peers { permanent_uuid: "96cba770f3334f79af3162475b54fa0f" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 33585 } } peers { permanent_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 37597 } }
I20260709 13:24:38.515659 28564 tablet_bootstrap.cc:492] T 68541149575b487b8c4e2360c2c93921 P 96cba770f3334f79af3162475b54fa0f: No bootstrap required, opened a new log
I20260709 13:24:38.516394 28564 ts_tablet_manager.cc:1403] T 68541149575b487b8c4e2360c2c93921 P 96cba770f3334f79af3162475b54fa0f: Time spent bootstrapping tablet: real 0.072s	user 0.018s	sys 0.028s
I20260709 13:24:38.514483 28272 catalog_manager.cc:5697] T bbaf4eecd3c241e3aabff37f6ba0256a P 542ea9e2fcff4028b6766459e90bc5fc reported cstate change: term changed from 0 to 1, leader changed from <none> to 542ea9e2fcff4028b6766459e90bc5fc (127.14.177.129). New cstate: current_term: 1 leader_uuid: "542ea9e2fcff4028b6766459e90bc5fc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "542ea9e2fcff4028b6766459e90bc5fc" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38409 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "96cba770f3334f79af3162475b54fa0f" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 33585 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 37597 } health_report { overall_health: UNKNOWN } } }
I20260709 13:24:38.520162 28564 raft_consensus.cc:359] T 68541149575b487b8c4e2360c2c93921 P 96cba770f3334f79af3162475b54fa0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 37597 } } peers { permanent_uuid: "542ea9e2fcff4028b6766459e90bc5fc" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38409 } } peers { permanent_uuid: "96cba770f3334f79af3162475b54fa0f" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 33585 } }
I20260709 13:24:38.521009 28564 raft_consensus.cc:385] T 68541149575b487b8c4e2360c2c93921 P 96cba770f3334f79af3162475b54fa0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:24:38.521373 28564 raft_consensus.cc:740] T 68541149575b487b8c4e2360c2c93921 P 96cba770f3334f79af3162475b54fa0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 96cba770f3334f79af3162475b54fa0f, State: Initialized, Role: FOLLOWER
I20260709 13:24:38.525182 28565 tablet_bootstrap.cc:492] T 73a4db7e3a924c6f99b416e6dda95c51 P 2173e3fd92f34ad6ad8d56d2e6b05299: No bootstrap required, opened a new log
W20260709 13:24:38.570772 28599 log.cc:927] Time spent T 41515f53db8540a8b4f52d9ac91dab50 P 2173e3fd92f34ad6ad8d56d2e6b05299: Append to log took a long time: real 0.063s	user 0.001s	sys 0.000s
I20260709 13:24:38.572256 28565 ts_tablet_manager.cc:1403] T 73a4db7e3a924c6f99b416e6dda95c51 P 2173e3fd92f34ad6ad8d56d2e6b05299: Time spent bootstrapping tablet: real 0.135s	user 0.015s	sys 0.024s
I20260709 13:24:38.573215 28564 consensus_queue.cc:260] T 68541149575b487b8c4e2360c2c93921 P 96cba770f3334f79af3162475b54fa0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 37597 } } peers { permanent_uuid: "542ea9e2fcff4028b6766459e90bc5fc" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38409 } } peers { permanent_uuid: "96cba770f3334f79af3162475b54fa0f" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 33585 } }
I20260709 13:24:38.574999 28563 tablet_bootstrap.cc:492] T 255f88c066ac422d9d8a582d9b24a958 P 542ea9e2fcff4028b6766459e90bc5fc: No bootstrap required, opened a new log
I20260709 13:24:38.575634 28563 ts_tablet_manager.cc:1403] T 255f88c066ac422d9d8a582d9b24a958 P 542ea9e2fcff4028b6766459e90bc5fc: Time spent bootstrapping tablet: real 0.177s	user 0.012s	sys 0.008s
I20260709 13:24:38.576633 28565 raft_consensus.cc:359] T 73a4db7e3a924c6f99b416e6dda95c51 P 2173e3fd92f34ad6ad8d56d2e6b05299 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "542ea9e2fcff4028b6766459e90bc5fc" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38409 } } peers { permanent_uuid: "96cba770f3334f79af3162475b54fa0f" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 33585 } } peers { permanent_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 37597 } }
I20260709 13:24:38.577160 28564 ts_tablet_manager.cc:1434] T 68541149575b487b8c4e2360c2c93921 P 96cba770f3334f79af3162475b54fa0f: Time spent starting tablet: real 0.060s	user 0.008s	sys 0.001s
I20260709 13:24:38.577484 28565 raft_consensus.cc:385] T 73a4db7e3a924c6f99b416e6dda95c51 P 2173e3fd92f34ad6ad8d56d2e6b05299 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:24:38.577870 28565 raft_consensus.cc:740] T 73a4db7e3a924c6f99b416e6dda95c51 P 2173e3fd92f34ad6ad8d56d2e6b05299 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2173e3fd92f34ad6ad8d56d2e6b05299, State: Initialized, Role: FOLLOWER
I20260709 13:24:38.578789 28564 tablet_bootstrap.cc:492] T 0701509f07184014b94c627c93b3bd59 P 96cba770f3334f79af3162475b54fa0f: Bootstrap starting.
I20260709 13:24:38.578768 28565 consensus_queue.cc:260] T 73a4db7e3a924c6f99b416e6dda95c51 P 2173e3fd92f34ad6ad8d56d2e6b05299 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "542ea9e2fcff4028b6766459e90bc5fc" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38409 } } peers { permanent_uuid: "96cba770f3334f79af3162475b54fa0f" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 33585 } } peers { permanent_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 37597 } }
I20260709 13:24:38.581473 28565 ts_tablet_manager.cc:1434] T 73a4db7e3a924c6f99b416e6dda95c51 P 2173e3fd92f34ad6ad8d56d2e6b05299: Time spent starting tablet: real 0.008s	user 0.003s	sys 0.003s
I20260709 13:24:38.581528 28580 raft_consensus.cc:493] T 0701509f07184014b94c627c93b3bd59 P 2173e3fd92f34ad6ad8d56d2e6b05299 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:24:38.582099 28580 raft_consensus.cc:515] T 0701509f07184014b94c627c93b3bd59 P 2173e3fd92f34ad6ad8d56d2e6b05299 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "542ea9e2fcff4028b6766459e90bc5fc" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38409 } } peers { permanent_uuid: "96cba770f3334f79af3162475b54fa0f" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 33585 } } peers { permanent_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 37597 } }
I20260709 13:24:38.582561 28565 tablet_bootstrap.cc:492] T bbaf4eecd3c241e3aabff37f6ba0256a P 2173e3fd92f34ad6ad8d56d2e6b05299: Bootstrap starting.
I20260709 13:24:38.585391 28580 leader_election.cc:290] T 0701509f07184014b94c627c93b3bd59 P 2173e3fd92f34ad6ad8d56d2e6b05299 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 542ea9e2fcff4028b6766459e90bc5fc (127.14.177.129:38409), 96cba770f3334f79af3162475b54fa0f (127.14.177.130:33585)
I20260709 13:24:38.578740 28563 raft_consensus.cc:359] T 255f88c066ac422d9d8a582d9b24a958 P 542ea9e2fcff4028b6766459e90bc5fc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "542ea9e2fcff4028b6766459e90bc5fc" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38409 } } peers { permanent_uuid: "96cba770f3334f79af3162475b54fa0f" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 33585 } } peers { permanent_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 37597 } }
I20260709 13:24:38.586943 28563 raft_consensus.cc:385] T 255f88c066ac422d9d8a582d9b24a958 P 542ea9e2fcff4028b6766459e90bc5fc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:24:38.587277 28563 raft_consensus.cc:740] T 255f88c066ac422d9d8a582d9b24a958 P 542ea9e2fcff4028b6766459e90bc5fc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 542ea9e2fcff4028b6766459e90bc5fc, State: Initialized, Role: FOLLOWER
I20260709 13:24:38.588104 28563 consensus_queue.cc:260] T 255f88c066ac422d9d8a582d9b24a958 P 542ea9e2fcff4028b6766459e90bc5fc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "542ea9e2fcff4028b6766459e90bc5fc" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38409 } } peers { permanent_uuid: "96cba770f3334f79af3162475b54fa0f" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 33585 } } peers { permanent_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 37597 } }
I20260709 13:24:38.589198 28564 tablet_bootstrap.cc:654] T 0701509f07184014b94c627c93b3bd59 P 96cba770f3334f79af3162475b54fa0f: Neither blocks nor log segments found. Creating new log.
I20260709 13:24:38.589946 28565 tablet_bootstrap.cc:654] T bbaf4eecd3c241e3aabff37f6ba0256a P 2173e3fd92f34ad6ad8d56d2e6b05299: Neither blocks nor log segments found. Creating new log.
I20260709 13:24:38.590758 28563 ts_tablet_manager.cc:1434] T 255f88c066ac422d9d8a582d9b24a958 P 542ea9e2fcff4028b6766459e90bc5fc: Time spent starting tablet: real 0.015s	user 0.005s	sys 0.000s
I20260709 13:24:38.591946 28563 tablet_bootstrap.cc:492] T a6cf33518cd34784a1b85843d90badb5 P 542ea9e2fcff4028b6766459e90bc5fc: Bootstrap starting.
I20260709 13:24:38.756508 28572 raft_consensus.cc:493] T 41515f53db8540a8b4f52d9ac91dab50 P 96cba770f3334f79af3162475b54fa0f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:24:38.757463 28572 raft_consensus.cc:515] T 41515f53db8540a8b4f52d9ac91dab50 P 96cba770f3334f79af3162475b54fa0f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "542ea9e2fcff4028b6766459e90bc5fc" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38409 } } peers { permanent_uuid: "96cba770f3334f79af3162475b54fa0f" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 33585 } } peers { permanent_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 37597 } }
I20260709 13:24:38.761210 28572 leader_election.cc:290] T 41515f53db8540a8b4f52d9ac91dab50 P 96cba770f3334f79af3162475b54fa0f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 542ea9e2fcff4028b6766459e90bc5fc (127.14.177.129:38409), 2173e3fd92f34ad6ad8d56d2e6b05299 (127.14.177.131:37597)
I20260709 13:24:38.783656 28564 tablet_bootstrap.cc:492] T 0701509f07184014b94c627c93b3bd59 P 96cba770f3334f79af3162475b54fa0f: No bootstrap required, opened a new log
I20260709 13:24:38.784317 28564 ts_tablet_manager.cc:1403] T 0701509f07184014b94c627c93b3bd59 P 96cba770f3334f79af3162475b54fa0f: Time spent bootstrapping tablet: real 0.206s	user 0.026s	sys 0.026s
I20260709 13:24:38.788527 28564 raft_consensus.cc:359] T 0701509f07184014b94c627c93b3bd59 P 96cba770f3334f79af3162475b54fa0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "542ea9e2fcff4028b6766459e90bc5fc" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38409 } } peers { permanent_uuid: "96cba770f3334f79af3162475b54fa0f" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 33585 } } peers { permanent_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 37597 } }
I20260709 13:24:38.789912 28564 raft_consensus.cc:385] T 0701509f07184014b94c627c93b3bd59 P 96cba770f3334f79af3162475b54fa0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:24:38.790337 28564 raft_consensus.cc:740] T 0701509f07184014b94c627c93b3bd59 P 96cba770f3334f79af3162475b54fa0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 96cba770f3334f79af3162475b54fa0f, State: Initialized, Role: FOLLOWER
I20260709 13:24:38.800942 28564 consensus_queue.cc:260] T 0701509f07184014b94c627c93b3bd59 P 96cba770f3334f79af3162475b54fa0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "542ea9e2fcff4028b6766459e90bc5fc" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38409 } } peers { permanent_uuid: "96cba770f3334f79af3162475b54fa0f" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 33585 } } peers { permanent_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 37597 } }
I20260709 13:24:38.804370 28564 ts_tablet_manager.cc:1434] T 0701509f07184014b94c627c93b3bd59 P 96cba770f3334f79af3162475b54fa0f: Time spent starting tablet: real 0.020s	user 0.010s	sys 0.000s
I20260709 13:24:38.819470 28564 tablet_bootstrap.cc:492] T 037bfc9d92244fcb872b2ca382c21794 P 96cba770f3334f79af3162475b54fa0f: Bootstrap starting.
I20260709 13:24:38.823022 28595 raft_consensus.cc:493] T b75c7d0f42e6499f936d08ff343fca5b P 542ea9e2fcff4028b6766459e90bc5fc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:24:38.823760 28595 raft_consensus.cc:515] T b75c7d0f42e6499f936d08ff343fca5b P 542ea9e2fcff4028b6766459e90bc5fc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "542ea9e2fcff4028b6766459e90bc5fc" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38409 } } peers { permanent_uuid: "96cba770f3334f79af3162475b54fa0f" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 33585 } } peers { permanent_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 37597 } }
I20260709 13:24:38.826252 28595 leader_election.cc:290] T b75c7d0f42e6499f936d08ff343fca5b P 542ea9e2fcff4028b6766459e90bc5fc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 96cba770f3334f79af3162475b54fa0f (127.14.177.130:33585), 2173e3fd92f34ad6ad8d56d2e6b05299 (127.14.177.131:37597)
I20260709 13:24:38.826720 28563 tablet_bootstrap.cc:654] T a6cf33518cd34784a1b85843d90badb5 P 542ea9e2fcff4028b6766459e90bc5fc: Neither blocks nor log segments found. Creating new log.
I20260709 13:24:38.827862 28565 tablet_bootstrap.cc:492] T bbaf4eecd3c241e3aabff37f6ba0256a P 2173e3fd92f34ad6ad8d56d2e6b05299: No bootstrap required, opened a new log
I20260709 13:24:38.829144 28565 ts_tablet_manager.cc:1403] T bbaf4eecd3c241e3aabff37f6ba0256a P 2173e3fd92f34ad6ad8d56d2e6b05299: Time spent bootstrapping tablet: real 0.247s	user 0.029s	sys 0.027s
I20260709 13:24:38.830801 28564 tablet_bootstrap.cc:654] T 037bfc9d92244fcb872b2ca382c21794 P 96cba770f3334f79af3162475b54fa0f: Neither blocks nor log segments found. Creating new log.
I20260709 13:24:38.832823 28565 raft_consensus.cc:359] T bbaf4eecd3c241e3aabff37f6ba0256a P 2173e3fd92f34ad6ad8d56d2e6b05299 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "542ea9e2fcff4028b6766459e90bc5fc" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38409 } } peers { permanent_uuid: "96cba770f3334f79af3162475b54fa0f" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 33585 } } peers { permanent_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 37597 } }
I20260709 13:24:38.833771 28565 raft_consensus.cc:385] T bbaf4eecd3c241e3aabff37f6ba0256a P 2173e3fd92f34ad6ad8d56d2e6b05299 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:24:38.834136 28565 raft_consensus.cc:740] T bbaf4eecd3c241e3aabff37f6ba0256a P 2173e3fd92f34ad6ad8d56d2e6b05299 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2173e3fd92f34ad6ad8d56d2e6b05299, State: Initialized, Role: FOLLOWER
I20260709 13:24:38.835783 28565 consensus_queue.cc:260] T bbaf4eecd3c241e3aabff37f6ba0256a P 2173e3fd92f34ad6ad8d56d2e6b05299 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "542ea9e2fcff4028b6766459e90bc5fc" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38409 } } peers { permanent_uuid: "96cba770f3334f79af3162475b54fa0f" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 33585 } } peers { permanent_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 37597 } }
I20260709 13:24:38.840160 28565 ts_tablet_manager.cc:1434] T bbaf4eecd3c241e3aabff37f6ba0256a P 2173e3fd92f34ad6ad8d56d2e6b05299: Time spent starting tablet: real 0.011s	user 0.008s	sys 0.001s
I20260709 13:24:38.840706 28447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75c7d0f42e6499f936d08ff343fca5b" candidate_uuid: "542ea9e2fcff4028b6766459e90bc5fc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96cba770f3334f79af3162475b54fa0f" is_pre_election: true
I20260709 13:24:38.841590 28565 tablet_bootstrap.cc:492] T 037bfc9d92244fcb872b2ca382c21794 P 2173e3fd92f34ad6ad8d56d2e6b05299: Bootstrap starting.
I20260709 13:24:38.841568 28447 raft_consensus.cc:2468] T b75c7d0f42e6499f936d08ff343fca5b P 96cba770f3334f79af3162475b54fa0f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 542ea9e2fcff4028b6766459e90bc5fc in term 0.
I20260709 13:24:38.843552 28564 tablet_bootstrap.cc:492] T 037bfc9d92244fcb872b2ca382c21794 P 96cba770f3334f79af3162475b54fa0f: No bootstrap required, opened a new log
I20260709 13:24:38.843395 28340 leader_election.cc:304] T b75c7d0f42e6499f936d08ff343fca5b P 542ea9e2fcff4028b6766459e90bc5fc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 542ea9e2fcff4028b6766459e90bc5fc, 96cba770f3334f79af3162475b54fa0f; no voters: 
I20260709 13:24:38.844307 28564 ts_tablet_manager.cc:1403] T 037bfc9d92244fcb872b2ca382c21794 P 96cba770f3334f79af3162475b54fa0f: Time spent bootstrapping tablet: real 0.025s	user 0.016s	sys 0.003s
I20260709 13:24:38.844630 28595 raft_consensus.cc:2804] T b75c7d0f42e6499f936d08ff343fca5b P 542ea9e2fcff4028b6766459e90bc5fc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:24:38.845417 28595 raft_consensus.cc:493] T b75c7d0f42e6499f936d08ff343fca5b P 542ea9e2fcff4028b6766459e90bc5fc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:24:38.845893 28595 raft_consensus.cc:3060] T b75c7d0f42e6499f936d08ff343fca5b P 542ea9e2fcff4028b6766459e90bc5fc [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:24:38.847721 28564 raft_consensus.cc:359] T 037bfc9d92244fcb872b2ca382c21794 P 96cba770f3334f79af3162475b54fa0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "542ea9e2fcff4028b6766459e90bc5fc" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38409 } } peers { permanent_uuid: "96cba770f3334f79af3162475b54fa0f" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 33585 } } peers { permanent_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 37597 } }
I20260709 13:24:38.848640 28564 raft_consensus.cc:385] T 037bfc9d92244fcb872b2ca382c21794 P 96cba770f3334f79af3162475b54fa0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:24:38.849010 28564 raft_consensus.cc:740] T 037bfc9d92244fcb872b2ca382c21794 P 96cba770f3334f79af3162475b54fa0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 96cba770f3334f79af3162475b54fa0f, State: Initialized, Role: FOLLOWER
I20260709 13:24:38.849852 28564 consensus_queue.cc:260] T 037bfc9d92244fcb872b2ca382c21794 P 96cba770f3334f79af3162475b54fa0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "542ea9e2fcff4028b6766459e90bc5fc" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38409 } } peers { permanent_uuid: "96cba770f3334f79af3162475b54fa0f" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 33585 } } peers { permanent_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 37597 } }
I20260709 13:24:38.850775 28565 tablet_bootstrap.cc:654] T 037bfc9d92244fcb872b2ca382c21794 P 2173e3fd92f34ad6ad8d56d2e6b05299: Neither blocks nor log segments found. Creating new log.
I20260709 13:24:38.841575 28563 tablet_bootstrap.cc:492] T a6cf33518cd34784a1b85843d90badb5 P 542ea9e2fcff4028b6766459e90bc5fc: No bootstrap required, opened a new log
I20260709 13:24:38.851655 28563 ts_tablet_manager.cc:1403] T a6cf33518cd34784a1b85843d90badb5 P 542ea9e2fcff4028b6766459e90bc5fc: Time spent bootstrapping tablet: real 0.260s	user 0.012s	sys 0.007s
I20260709 13:24:38.852888 28564 ts_tablet_manager.cc:1434] T 037bfc9d92244fcb872b2ca382c21794 P 96cba770f3334f79af3162475b54fa0f: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 13:24:38.855012 28563 raft_consensus.cc:359] T a6cf33518cd34784a1b85843d90badb5 P 542ea9e2fcff4028b6766459e90bc5fc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "542ea9e2fcff4028b6766459e90bc5fc" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38409 } } peers { permanent_uuid: "96cba770f3334f79af3162475b54fa0f" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 33585 } } peers { permanent_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 37597 } }
I20260709 13:24:38.855894 28563 raft_consensus.cc:385] T a6cf33518cd34784a1b85843d90badb5 P 542ea9e2fcff4028b6766459e90bc5fc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:24:38.856221 28563 raft_consensus.cc:740] T a6cf33518cd34784a1b85843d90badb5 P 542ea9e2fcff4028b6766459e90bc5fc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 542ea9e2fcff4028b6766459e90bc5fc, State: Initialized, Role: FOLLOWER
I20260709 13:24:38.856588 28595 raft_consensus.cc:515] T b75c7d0f42e6499f936d08ff343fca5b P 542ea9e2fcff4028b6766459e90bc5fc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "542ea9e2fcff4028b6766459e90bc5fc" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38409 } } peers { permanent_uuid: "96cba770f3334f79af3162475b54fa0f" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 33585 } } peers { permanent_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 37597 } }
I20260709 13:24:38.857074 28563 consensus_queue.cc:260] T a6cf33518cd34784a1b85843d90badb5 P 542ea9e2fcff4028b6766459e90bc5fc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "542ea9e2fcff4028b6766459e90bc5fc" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38409 } } peers { permanent_uuid: "96cba770f3334f79af3162475b54fa0f" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 33585 } } peers { permanent_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 37597 } }
I20260709 13:24:38.859467 28563 ts_tablet_manager.cc:1434] T a6cf33518cd34784a1b85843d90badb5 P 542ea9e2fcff4028b6766459e90bc5fc: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 13:24:38.860067 28447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75c7d0f42e6499f936d08ff343fca5b" candidate_uuid: "542ea9e2fcff4028b6766459e90bc5fc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96cba770f3334f79af3162475b54fa0f"
I20260709 13:24:38.860515 28563 tablet_bootstrap.cc:492] T 0701509f07184014b94c627c93b3bd59 P 542ea9e2fcff4028b6766459e90bc5fc: Bootstrap starting.
I20260709 13:24:38.860759 28447 raft_consensus.cc:3060] T b75c7d0f42e6499f936d08ff343fca5b P 96cba770f3334f79af3162475b54fa0f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:24:38.871277 28563 tablet_bootstrap.cc:654] T 0701509f07184014b94c627c93b3bd59 P 542ea9e2fcff4028b6766459e90bc5fc: Neither blocks nor log segments found. Creating new log.
I20260709 13:24:38.996731 28594 raft_consensus.cc:493] T a6cf33518cd34784a1b85843d90badb5 P 542ea9e2fcff4028b6766459e90bc5fc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:24:38.997596 28594 raft_consensus.cc:515] T a6cf33518cd34784a1b85843d90badb5 P 542ea9e2fcff4028b6766459e90bc5fc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "542ea9e2fcff4028b6766459e90bc5fc" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38409 } } peers { permanent_uuid: "96cba770f3334f79af3162475b54fa0f" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 33585 } } peers { permanent_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 37597 } }
I20260709 13:24:39.033917 28446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6cf33518cd34784a1b85843d90badb5" candidate_uuid: "542ea9e2fcff4028b6766459e90bc5fc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96cba770f3334f79af3162475b54fa0f" is_pre_election: true
I20260709 13:24:39.034214 28594 leader_election.cc:290] T a6cf33518cd34784a1b85843d90badb5 P 542ea9e2fcff4028b6766459e90bc5fc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 96cba770f3334f79af3162475b54fa0f (127.14.177.130:33585), 2173e3fd92f34ad6ad8d56d2e6b05299 (127.14.177.131:37597)
I20260709 13:24:39.035209 28446 raft_consensus.cc:2393] T a6cf33518cd34784a1b85843d90badb5 P 96cba770f3334f79af3162475b54fa0f [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 542ea9e2fcff4028b6766459e90bc5fc in current term 1: Already voted for candidate 2173e3fd92f34ad6ad8d56d2e6b05299 in this term.
I20260709 13:24:39.157254 28594 raft_consensus.cc:493] T 255f88c066ac422d9d8a582d9b24a958 P 542ea9e2fcff4028b6766459e90bc5fc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:24:39.157840 28594 raft_consensus.cc:515] T 255f88c066ac422d9d8a582d9b24a958 P 542ea9e2fcff4028b6766459e90bc5fc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "542ea9e2fcff4028b6766459e90bc5fc" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38409 } } peers { permanent_uuid: "96cba770f3334f79af3162475b54fa0f" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 33585 } } peers { permanent_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 37597 } }
I20260709 13:24:39.164377 28594 leader_election.cc:290] T 255f88c066ac422d9d8a582d9b24a958 P 542ea9e2fcff4028b6766459e90bc5fc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 96cba770f3334f79af3162475b54fa0f (127.14.177.130:33585), 2173e3fd92f34ad6ad8d56d2e6b05299 (127.14.177.131:37597)
I20260709 13:24:39.171152 28563 tablet_bootstrap.cc:492] T 0701509f07184014b94c627c93b3bd59 P 542ea9e2fcff4028b6766459e90bc5fc: No bootstrap required, opened a new log
I20260709 13:24:39.170817 28595 leader_election.cc:290] T b75c7d0f42e6499f936d08ff343fca5b P 542ea9e2fcff4028b6766459e90bc5fc [CANDIDATE]: Term 1 election: Requested vote from peers 96cba770f3334f79af3162475b54fa0f (127.14.177.130:33585), 2173e3fd92f34ad6ad8d56d2e6b05299 (127.14.177.131:37597)
I20260709 13:24:39.171954 28563 ts_tablet_manager.cc:1403] T 0701509f07184014b94c627c93b3bd59 P 542ea9e2fcff4028b6766459e90bc5fc: Time spent bootstrapping tablet: real 0.312s	user 0.051s	sys 0.103s
I20260709 13:24:39.176222 28446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255f88c066ac422d9d8a582d9b24a958" candidate_uuid: "542ea9e2fcff4028b6766459e90bc5fc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96cba770f3334f79af3162475b54fa0f" is_pre_election: true
I20260709 13:24:39.177152 28446 raft_consensus.cc:2393] T 255f88c066ac422d9d8a582d9b24a958 P 96cba770f3334f79af3162475b54fa0f [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 542ea9e2fcff4028b6766459e90bc5fc in current term 1: Already voted for candidate 2173e3fd92f34ad6ad8d56d2e6b05299 in this term.
I20260709 13:24:39.177364 28563 raft_consensus.cc:359] T 0701509f07184014b94c627c93b3bd59 P 542ea9e2fcff4028b6766459e90bc5fc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "542ea9e2fcff4028b6766459e90bc5fc" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38409 } } peers { permanent_uuid: "96cba770f3334f79af3162475b54fa0f" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 33585 } } peers { permanent_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 37597 } }
I20260709 13:24:39.178180 28563 raft_consensus.cc:385] T 0701509f07184014b94c627c93b3bd59 P 542ea9e2fcff4028b6766459e90bc5fc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:24:39.178499 28563 raft_consensus.cc:740] T 0701509f07184014b94c627c93b3bd59 P 542ea9e2fcff4028b6766459e90bc5fc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 542ea9e2fcff4028b6766459e90bc5fc, State: Initialized, Role: FOLLOWER
I20260709 13:24:39.165987 28447 raft_consensus.cc:2468] T b75c7d0f42e6499f936d08ff343fca5b P 96cba770f3334f79af3162475b54fa0f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 542ea9e2fcff4028b6766459e90bc5fc in term 1.
I20260709 13:24:39.183382 28340 leader_election.cc:304] T b75c7d0f42e6499f936d08ff343fca5b P 542ea9e2fcff4028b6766459e90bc5fc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 542ea9e2fcff4028b6766459e90bc5fc, 96cba770f3334f79af3162475b54fa0f; no voters: 
I20260709 13:24:39.184419 28611 raft_consensus.cc:2804] T b75c7d0f42e6499f936d08ff343fca5b P 542ea9e2fcff4028b6766459e90bc5fc [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:24:39.184085 28563 consensus_queue.cc:260] T 0701509f07184014b94c627c93b3bd59 P 542ea9e2fcff4028b6766459e90bc5fc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "542ea9e2fcff4028b6766459e90bc5fc" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38409 } } peers { permanent_uuid: "96cba770f3334f79af3162475b54fa0f" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 33585 } } peers { permanent_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 37597 } }
I20260709 13:24:39.185038 28611 raft_consensus.cc:697] T b75c7d0f42e6499f936d08ff343fca5b P 542ea9e2fcff4028b6766459e90bc5fc [term 1 LEADER]: Becoming Leader. State: Replica: 542ea9e2fcff4028b6766459e90bc5fc, State: Running, Role: LEADER
I20260709 13:24:39.186126 28577 raft_consensus.cc:493] T 73a4db7e3a924c6f99b416e6dda95c51 P 2173e3fd92f34ad6ad8d56d2e6b05299 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:24:39.190793 28577 raft_consensus.cc:515] T 73a4db7e3a924c6f99b416e6dda95c51 P 2173e3fd92f34ad6ad8d56d2e6b05299 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "542ea9e2fcff4028b6766459e90bc5fc" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38409 } } peers { permanent_uuid: "96cba770f3334f79af3162475b54fa0f" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 33585 } } peers { permanent_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 37597 } }
I20260709 13:24:39.191145 28611 consensus_queue.cc:237] T b75c7d0f42e6499f936d08ff343fca5b P 542ea9e2fcff4028b6766459e90bc5fc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "542ea9e2fcff4028b6766459e90bc5fc" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38409 } } peers { permanent_uuid: "96cba770f3334f79af3162475b54fa0f" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 33585 } } peers { permanent_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 37597 } }
I20260709 13:24:39.192498 28563 ts_tablet_manager.cc:1434] T 0701509f07184014b94c627c93b3bd59 P 542ea9e2fcff4028b6766459e90bc5fc: Time spent starting tablet: real 0.018s	user 0.006s	sys 0.000s
I20260709 13:24:39.193019 28577 leader_election.cc:290] T 73a4db7e3a924c6f99b416e6dda95c51 P 2173e3fd92f34ad6ad8d56d2e6b05299 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 542ea9e2fcff4028b6766459e90bc5fc (127.14.177.129:38409), 96cba770f3334f79af3162475b54fa0f (127.14.177.130:33585)
I20260709 13:24:39.190099 28565 tablet_bootstrap.cc:492] T 037bfc9d92244fcb872b2ca382c21794 P 2173e3fd92f34ad6ad8d56d2e6b05299: No bootstrap required, opened a new log
I20260709 13:24:39.193956 28565 ts_tablet_manager.cc:1403] T 037bfc9d92244fcb872b2ca382c21794 P 2173e3fd92f34ad6ad8d56d2e6b05299: Time spent bootstrapping tablet: real 0.353s	user 0.066s	sys 0.100s
I20260709 13:24:39.215349 28522 raft_consensus.cc:3060] T bbaf4eecd3c241e3aabff37f6ba0256a P 2173e3fd92f34ad6ad8d56d2e6b05299 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:24:39.221046 28521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6cf33518cd34784a1b85843d90badb5" candidate_uuid: "542ea9e2fcff4028b6766459e90bc5fc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" is_pre_election: true
I20260709 13:24:39.223569 28519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255f88c066ac422d9d8a582d9b24a958" candidate_uuid: "542ea9e2fcff4028b6766459e90bc5fc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" is_pre_election: true
I20260709 13:24:39.225569 28519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75c7d0f42e6499f936d08ff343fca5b" candidate_uuid: "542ea9e2fcff4028b6766459e90bc5fc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299"
I20260709 13:24:39.225756 28340 leader_election.cc:304] T a6cf33518cd34784a1b85843d90badb5 P 542ea9e2fcff4028b6766459e90bc5fc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 542ea9e2fcff4028b6766459e90bc5fc; no voters: 2173e3fd92f34ad6ad8d56d2e6b05299, 96cba770f3334f79af3162475b54fa0f
I20260709 13:24:39.226768 28611 raft_consensus.cc:3060] T a6cf33518cd34784a1b85843d90badb5 P 542ea9e2fcff4028b6766459e90bc5fc [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:24:39.227596 28340 leader_election.cc:304] T 255f88c066ac422d9d8a582d9b24a958 P 542ea9e2fcff4028b6766459e90bc5fc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 542ea9e2fcff4028b6766459e90bc5fc; no voters: 2173e3fd92f34ad6ad8d56d2e6b05299, 96cba770f3334f79af3162475b54fa0f
I20260709 13:24:39.228091 28572 raft_consensus.cc:493] T 0701509f07184014b94c627c93b3bd59 P 96cba770f3334f79af3162475b54fa0f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:24:39.228641 28572 raft_consensus.cc:515] T 0701509f07184014b94c627c93b3bd59 P 96cba770f3334f79af3162475b54fa0f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "542ea9e2fcff4028b6766459e90bc5fc" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38409 } } peers { permanent_uuid: "96cba770f3334f79af3162475b54fa0f" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 33585 } } peers { permanent_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 37597 } }
W20260709 13:24:39.231820 28340 leader_election.cc:343] T b75c7d0f42e6499f936d08ff343fca5b P 542ea9e2fcff4028b6766459e90bc5fc [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 2173e3fd92f34ad6ad8d56d2e6b05299 (127.14.177.131:37597): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:24:39.230865 28572 leader_election.cc:290] T 0701509f07184014b94c627c93b3bd59 P 96cba770f3334f79af3162475b54fa0f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 542ea9e2fcff4028b6766459e90bc5fc (127.14.177.129:38409), 2173e3fd92f34ad6ad8d56d2e6b05299 (127.14.177.131:37597)
I20260709 13:24:39.233793 28565 raft_consensus.cc:359] T 037bfc9d92244fcb872b2ca382c21794 P 2173e3fd92f34ad6ad8d56d2e6b05299 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "542ea9e2fcff4028b6766459e90bc5fc" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38409 } } peers { permanent_uuid: "96cba770f3334f79af3162475b54fa0f" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 33585 } } peers { permanent_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 37597 } }
I20260709 13:24:39.235882 28565 raft_consensus.cc:385] T 037bfc9d92244fcb872b2ca382c21794 P 2173e3fd92f34ad6ad8d56d2e6b05299 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:24:39.234053 28521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0701509f07184014b94c627c93b3bd59" candidate_uuid: "96cba770f3334f79af3162475b54fa0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" is_pre_election: true
I20260709 13:24:39.236363 28565 raft_consensus.cc:740] T 037bfc9d92244fcb872b2ca382c21794 P 2173e3fd92f34ad6ad8d56d2e6b05299 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2173e3fd92f34ad6ad8d56d2e6b05299, State: Initialized, Role: FOLLOWER
I20260709 13:24:39.236603 28521 raft_consensus.cc:2468] T 0701509f07184014b94c627c93b3bd59 P 2173e3fd92f34ad6ad8d56d2e6b05299 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96cba770f3334f79af3162475b54fa0f in term 0.
I20260709 13:24:39.234946 28519 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41515f53db8540a8b4f52d9ac91dab50" candidate_uuid: "96cba770f3334f79af3162475b54fa0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" is_pre_election: true
I20260709 13:24:39.237371 28565 consensus_queue.cc:260] T 037bfc9d92244fcb872b2ca382c21794 P 2173e3fd92f34ad6ad8d56d2e6b05299 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "542ea9e2fcff4028b6766459e90bc5fc" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38409 } } peers { permanent_uuid: "96cba770f3334f79af3162475b54fa0f" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 33585 } } peers { permanent_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 37597 } }
I20260709 13:24:39.236107 28523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75c7d0f42e6499f936d08ff343fca5b" candidate_uuid: "542ea9e2fcff4028b6766459e90bc5fc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" is_pre_election: true
I20260709 13:24:39.238699 28373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41515f53db8540a8b4f52d9ac91dab50" candidate_uuid: "96cba770f3334f79af3162475b54fa0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "542ea9e2fcff4028b6766459e90bc5fc" is_pre_election: true
I20260709 13:24:39.239529 28373 raft_consensus.cc:2393] T 41515f53db8540a8b4f52d9ac91dab50 P 542ea9e2fcff4028b6766459e90bc5fc [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 96cba770f3334f79af3162475b54fa0f in current term 1: Already voted for candidate 2173e3fd92f34ad6ad8d56d2e6b05299 in this term.
I20260709 13:24:39.235365 28520 raft_consensus.cc:3060] T 73a4db7e3a924c6f99b416e6dda95c51 P 2173e3fd92f34ad6ad8d56d2e6b05299 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:24:39.240223 28565 ts_tablet_manager.cc:1434] T 037bfc9d92244fcb872b2ca382c21794 P 2173e3fd92f34ad6ad8d56d2e6b05299: Time spent starting tablet: real 0.046s	user 0.007s	sys 0.004s
I20260709 13:24:39.240592 28374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0701509f07184014b94c627c93b3bd59" candidate_uuid: "96cba770f3334f79af3162475b54fa0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "542ea9e2fcff4028b6766459e90bc5fc" is_pre_election: true
I20260709 13:24:39.241297 28374 raft_consensus.cc:2468] T 0701509f07184014b94c627c93b3bd59 P 542ea9e2fcff4028b6766459e90bc5fc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96cba770f3334f79af3162475b54fa0f in term 0.
I20260709 13:24:39.241456 28565 tablet_bootstrap.cc:492] T b75c7d0f42e6499f936d08ff343fca5b P 2173e3fd92f34ad6ad8d56d2e6b05299: Bootstrap starting.
I20260709 13:24:39.242671 28447 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0701509f07184014b94c627c93b3bd59" candidate_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96cba770f3334f79af3162475b54fa0f" is_pre_election: true
I20260709 13:24:39.243440 28447 raft_consensus.cc:2468] T 0701509f07184014b94c627c93b3bd59 P 96cba770f3334f79af3162475b54fa0f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2173e3fd92f34ad6ad8d56d2e6b05299 in term 0.
I20260709 13:24:39.243680 28415 leader_election.cc:304] T 0701509f07184014b94c627c93b3bd59 P 96cba770f3334f79af3162475b54fa0f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2173e3fd92f34ad6ad8d56d2e6b05299, 96cba770f3334f79af3162475b54fa0f; no voters: 
I20260709 13:24:39.244786 28446 raft_consensus.cc:3060] T 68541149575b487b8c4e2360c2c93921 P 96cba770f3334f79af3162475b54fa0f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:24:39.246454 28449 raft_consensus.cc:3060] T 41515f53db8540a8b4f52d9ac91dab50 P 96cba770f3334f79af3162475b54fa0f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:24:39.242861 28611 raft_consensus.cc:2749] T a6cf33518cd34784a1b85843d90badb5 P 542ea9e2fcff4028b6766459e90bc5fc [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:24:39.261739 28565 tablet_bootstrap.cc:654] T b75c7d0f42e6499f936d08ff343fca5b P 2173e3fd92f34ad6ad8d56d2e6b05299: Neither blocks nor log segments found. Creating new log.
W20260709 13:24:39.260406 28340 leader_election.cc:343] T b75c7d0f42e6499f936d08ff343fca5b P 542ea9e2fcff4028b6766459e90bc5fc [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2173e3fd92f34ad6ad8d56d2e6b05299 (127.14.177.131:37597): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:24:39.252192 28444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73a4db7e3a924c6f99b416e6dda95c51" candidate_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96cba770f3334f79af3162475b54fa0f" is_pre_election: true
I20260709 13:24:39.265777 28444 raft_consensus.cc:2393] T 73a4db7e3a924c6f99b416e6dda95c51 P 96cba770f3334f79af3162475b54fa0f [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2173e3fd92f34ad6ad8d56d2e6b05299 in current term 1: Already voted for candidate 542ea9e2fcff4028b6766459e90bc5fc in this term.
I20260709 13:24:39.272408 28594 consensus_queue.cc:1048] T bbaf4eecd3c241e3aabff37f6ba0256a P 542ea9e2fcff4028b6766459e90bc5fc [LEADER]: Connected to new peer: Peer: permanent_uuid: "96cba770f3334f79af3162475b54fa0f" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 33585 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:24:39.279663 28595 consensus_queue.cc:1048] T bbaf4eecd3c241e3aabff37f6ba0256a P 542ea9e2fcff4028b6766459e90bc5fc [LEADER]: Connected to new peer: Peer: permanent_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 37597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:24:39.280800 28270 catalog_manager.cc:5697] T 255f88c066ac422d9d8a582d9b24a958 P 2173e3fd92f34ad6ad8d56d2e6b05299 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2173e3fd92f34ad6ad8d56d2e6b05299 (127.14.177.131). New cstate: current_term: 1 leader_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "542ea9e2fcff4028b6766459e90bc5fc" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38409 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "96cba770f3334f79af3162475b54fa0f" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 33585 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 37597 } health_report { overall_health: HEALTHY } } }
I20260709 13:24:39.253821 28611 raft_consensus.cc:3060] T 255f88c066ac422d9d8a582d9b24a958 P 542ea9e2fcff4028b6766459e90bc5fc [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:24:39.275516 28594 consensus_queue.cc:1048] T 73a4db7e3a924c6f99b416e6dda95c51 P 542ea9e2fcff4028b6766459e90bc5fc [LEADER]: Connected to new peer: Peer: permanent_uuid: "96cba770f3334f79af3162475b54fa0f" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 33585 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.013s
I20260709 13:24:39.255952 28489 leader_election.cc:304] T 0701509f07184014b94c627c93b3bd59 P 2173e3fd92f34ad6ad8d56d2e6b05299 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2173e3fd92f34ad6ad8d56d2e6b05299, 96cba770f3334f79af3162475b54fa0f; no voters: 
W20260709 13:24:39.300069 28415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:37597, user_credentials={real_user=slave}} blocked reactor thread for 56497.5us
I20260709 13:24:39.309376 28374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0701509f07184014b94c627c93b3bd59" candidate_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "542ea9e2fcff4028b6766459e90bc5fc" is_pre_election: true
I20260709 13:24:39.310410 28374 raft_consensus.cc:2468] T 0701509f07184014b94c627c93b3bd59 P 542ea9e2fcff4028b6766459e90bc5fc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2173e3fd92f34ad6ad8d56d2e6b05299 in term 0.
W20260709 13:24:39.301766 28489 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.130:33585, user_credentials={real_user=slave}} blocked reactor thread for 50371us
I20260709 13:24:39.313333 28613 consensus_queue.cc:1048] T a6cf33518cd34784a1b85843d90badb5 P 2173e3fd92f34ad6ad8d56d2e6b05299 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96cba770f3334f79af3162475b54fa0f" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 33585 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:24:39.315477 28614 consensus_queue.cc:1048] T 255f88c066ac422d9d8a582d9b24a958 P 2173e3fd92f34ad6ad8d56d2e6b05299 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96cba770f3334f79af3162475b54fa0f" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 33585 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:24:39.302352 28577 raft_consensus.cc:2804] T 0701509f07184014b94c627c93b3bd59 P 2173e3fd92f34ad6ad8d56d2e6b05299 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:24:39.319634 28577 raft_consensus.cc:493] T 0701509f07184014b94c627c93b3bd59 P 2173e3fd92f34ad6ad8d56d2e6b05299 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:24:39.320122 28577 raft_consensus.cc:3060] T 0701509f07184014b94c627c93b3bd59 P 2173e3fd92f34ad6ad8d56d2e6b05299 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:24:39.303325 28270 catalog_manager.cc:5697] T a6cf33518cd34784a1b85843d90badb5 P 2173e3fd92f34ad6ad8d56d2e6b05299 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2173e3fd92f34ad6ad8d56d2e6b05299 (127.14.177.131). New cstate: current_term: 1 leader_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "542ea9e2fcff4028b6766459e90bc5fc" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38409 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "96cba770f3334f79af3162475b54fa0f" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 33585 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 37597 } health_report { overall_health: HEALTHY } } }
I20260709 13:24:39.333822 28565 tablet_bootstrap.cc:492] T b75c7d0f42e6499f936d08ff343fca5b P 2173e3fd92f34ad6ad8d56d2e6b05299: No bootstrap required, opened a new log
I20260709 13:24:39.334347 28565 ts_tablet_manager.cc:1403] T b75c7d0f42e6499f936d08ff343fca5b P 2173e3fd92f34ad6ad8d56d2e6b05299: Time spent bootstrapping tablet: real 0.093s	user 0.011s	sys 0.015s
I20260709 13:24:39.334952 28618 raft_consensus.cc:2804] T 0701509f07184014b94c627c93b3bd59 P 96cba770f3334f79af3162475b54fa0f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:24:39.325620 28270 catalog_manager.cc:5697] T 41515f53db8540a8b4f52d9ac91dab50 P 2173e3fd92f34ad6ad8d56d2e6b05299 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2173e3fd92f34ad6ad8d56d2e6b05299 (127.14.177.131). New cstate: current_term: 1 leader_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "542ea9e2fcff4028b6766459e90bc5fc" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38409 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "96cba770f3334f79af3162475b54fa0f" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 33585 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 37597 } health_report { overall_health: HEALTHY } } }
I20260709 13:24:39.338059 28618 raft_consensus.cc:493] T 0701509f07184014b94c627c93b3bd59 P 96cba770f3334f79af3162475b54fa0f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:24:39.349893 28374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73a4db7e3a924c6f99b416e6dda95c51" candidate_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "542ea9e2fcff4028b6766459e90bc5fc" is_pre_election: true
I20260709 13:24:39.347561 28618 raft_consensus.cc:3060] T 0701509f07184014b94c627c93b3bd59 P 96cba770f3334f79af3162475b54fa0f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:24:39.359951 28613 consensus_queue.cc:1048] T 41515f53db8540a8b4f52d9ac91dab50 P 2173e3fd92f34ad6ad8d56d2e6b05299 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96cba770f3334f79af3162475b54fa0f" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 33585 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:24:39.370218 28613 consensus_queue.cc:1048] T 68541149575b487b8c4e2360c2c93921 P 2173e3fd92f34ad6ad8d56d2e6b05299 [LEADER]: Connected to new peer: Peer: permanent_uuid: "542ea9e2fcff4028b6766459e90bc5fc" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38409 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 13:24:39.377127 28320 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:24:39.378970 28487 leader_election.cc:304] T 73a4db7e3a924c6f99b416e6dda95c51 P 2173e3fd92f34ad6ad8d56d2e6b05299 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2173e3fd92f34ad6ad8d56d2e6b05299; no voters: 542ea9e2fcff4028b6766459e90bc5fc, 96cba770f3334f79af3162475b54fa0f
I20260709 13:24:39.374475 28618 raft_consensus.cc:515] T 0701509f07184014b94c627c93b3bd59 P 96cba770f3334f79af3162475b54fa0f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "542ea9e2fcff4028b6766459e90bc5fc" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38409 } } peers { permanent_uuid: "96cba770f3334f79af3162475b54fa0f" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 33585 } } peers { permanent_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 37597 } }
I20260709 13:24:39.337692 28415 leader_election.cc:304] T 41515f53db8540a8b4f52d9ac91dab50 P 96cba770f3334f79af3162475b54fa0f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96cba770f3334f79af3162475b54fa0f; no voters: 2173e3fd92f34ad6ad8d56d2e6b05299, 542ea9e2fcff4028b6766459e90bc5fc
I20260709 13:24:39.385730 28614 consensus_queue.cc:1048] T 41515f53db8540a8b4f52d9ac91dab50 P 2173e3fd92f34ad6ad8d56d2e6b05299 [LEADER]: Connected to new peer: Peer: permanent_uuid: "542ea9e2fcff4028b6766459e90bc5fc" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38409 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.007s
I20260709 13:24:39.386829 28613 consensus_queue.cc:1048] T a6cf33518cd34784a1b85843d90badb5 P 2173e3fd92f34ad6ad8d56d2e6b05299 [LEADER]: Connected to new peer: Peer: permanent_uuid: "542ea9e2fcff4028b6766459e90bc5fc" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38409 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.007s
I20260709 13:24:39.395886 28621 raft_consensus.cc:2749] T 41515f53db8540a8b4f52d9ac91dab50 P 96cba770f3334f79af3162475b54fa0f [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:24:39.393168 28572 raft_consensus.cc:493] T 68541149575b487b8c4e2360c2c93921 P 96cba770f3334f79af3162475b54fa0f [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:24:39.391657 28577 raft_consensus.cc:515] T 0701509f07184014b94c627c93b3bd59 P 2173e3fd92f34ad6ad8d56d2e6b05299 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "542ea9e2fcff4028b6766459e90bc5fc" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38409 } } peers { permanent_uuid: "96cba770f3334f79af3162475b54fa0f" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 33585 } } peers { permanent_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 37597 } }
I20260709 13:24:39.410782 28613 consensus_queue.cc:1048] T 68541149575b487b8c4e2360c2c93921 P 2173e3fd92f34ad6ad8d56d2e6b05299 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96cba770f3334f79af3162475b54fa0f" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 33585 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:24:39.406576 28572 raft_consensus.cc:515] T 68541149575b487b8c4e2360c2c93921 P 96cba770f3334f79af3162475b54fa0f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 37597 } } peers { permanent_uuid: "542ea9e2fcff4028b6766459e90bc5fc" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38409 } } peers { permanent_uuid: "96cba770f3334f79af3162475b54fa0f" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 33585 } }
W20260709 13:24:39.396945 28415 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.131:37597, user_credentials={real_user=slave}} blocked reactor thread for 70142.2us
I20260709 13:24:39.422636 28565 raft_consensus.cc:359] T b75c7d0f42e6499f936d08ff343fca5b P 2173e3fd92f34ad6ad8d56d2e6b05299 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "542ea9e2fcff4028b6766459e90bc5fc" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38409 } } peers { permanent_uuid: "96cba770f3334f79af3162475b54fa0f" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 33585 } } peers { permanent_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 37597 } }
I20260709 13:24:39.425294 28371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0701509f07184014b94c627c93b3bd59" candidate_uuid: "96cba770f3334f79af3162475b54fa0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "542ea9e2fcff4028b6766459e90bc5fc"
I20260709 13:24:39.425788 28614 raft_consensus.cc:2749] T 73a4db7e3a924c6f99b416e6dda95c51 P 2173e3fd92f34ad6ad8d56d2e6b05299 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:24:39.425396 28572 leader_election.cc:290] T 68541149575b487b8c4e2360c2c93921 P 96cba770f3334f79af3162475b54fa0f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2173e3fd92f34ad6ad8d56d2e6b05299 (127.14.177.131:37597), 542ea9e2fcff4028b6766459e90bc5fc (127.14.177.129:38409)
I20260709 13:24:39.428027 28373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68541149575b487b8c4e2360c2c93921" candidate_uuid: "96cba770f3334f79af3162475b54fa0f" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "542ea9e2fcff4028b6766459e90bc5fc" is_pre_election: true
I20260709 13:24:39.436856 28371 raft_consensus.cc:3060] T 0701509f07184014b94c627c93b3bd59 P 542ea9e2fcff4028b6766459e90bc5fc [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:24:39.427574 28618 leader_election.cc:290] T 0701509f07184014b94c627c93b3bd59 P 96cba770f3334f79af3162475b54fa0f [CANDIDATE]: Term 1 election: Requested vote from peers 542ea9e2fcff4028b6766459e90bc5fc (127.14.177.129:38409), 2173e3fd92f34ad6ad8d56d2e6b05299 (127.14.177.131:37597)
I20260709 13:24:39.431020 28565 raft_consensus.cc:385] T b75c7d0f42e6499f936d08ff343fca5b P 2173e3fd92f34ad6ad8d56d2e6b05299 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:24:39.439579 28565 raft_consensus.cc:740] T b75c7d0f42e6499f936d08ff343fca5b P 2173e3fd92f34ad6ad8d56d2e6b05299 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2173e3fd92f34ad6ad8d56d2e6b05299, State: Initialized, Role: FOLLOWER
I20260709 13:24:39.446836 28565 consensus_queue.cc:260] T b75c7d0f42e6499f936d08ff343fca5b P 2173e3fd92f34ad6ad8d56d2e6b05299 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "542ea9e2fcff4028b6766459e90bc5fc" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38409 } } peers { permanent_uuid: "96cba770f3334f79af3162475b54fa0f" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 33585 } } peers { permanent_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 37597 } }
I20260709 13:24:39.448696 28565 ts_tablet_manager.cc:1434] T b75c7d0f42e6499f936d08ff343fca5b P 2173e3fd92f34ad6ad8d56d2e6b05299: Time spent starting tablet: real 0.114s	user 0.008s	sys 0.000s
W20260709 13:24:39.435277 28487 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:38409, user_credentials={real_user=slave}} blocked reactor thread for 44824.5us
I20260709 13:24:39.459563 28520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68541149575b487b8c4e2360c2c93921" candidate_uuid: "96cba770f3334f79af3162475b54fa0f" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" is_pre_election: true
I20260709 13:24:39.461151 28577 leader_election.cc:290] T 0701509f07184014b94c627c93b3bd59 P 2173e3fd92f34ad6ad8d56d2e6b05299 [CANDIDATE]: Term 1 election: Requested vote from peers 542ea9e2fcff4028b6766459e90bc5fc (127.14.177.129:38409), 96cba770f3334f79af3162475b54fa0f (127.14.177.130:33585)
I20260709 13:24:39.445986 28611 raft_consensus.cc:2749] T 255f88c066ac422d9d8a582d9b24a958 P 542ea9e2fcff4028b6766459e90bc5fc [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:24:39.469576 28595 consensus_queue.cc:1048] T 73a4db7e3a924c6f99b416e6dda95c51 P 542ea9e2fcff4028b6766459e90bc5fc [LEADER]: Connected to new peer: Peer: permanent_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 37597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:24:39.477177 28369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0701509f07184014b94c627c93b3bd59" candidate_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "542ea9e2fcff4028b6766459e90bc5fc"
I20260709 13:24:39.479617 28415 leader_election.cc:304] T 68541149575b487b8c4e2360c2c93921 P 96cba770f3334f79af3162475b54fa0f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96cba770f3334f79af3162475b54fa0f; no voters: 2173e3fd92f34ad6ad8d56d2e6b05299, 542ea9e2fcff4028b6766459e90bc5fc
I20260709 13:24:39.480049 28522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0701509f07184014b94c627c93b3bd59" candidate_uuid: "96cba770f3334f79af3162475b54fa0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299"
I20260709 13:24:39.480907 28522 raft_consensus.cc:2393] T 0701509f07184014b94c627c93b3bd59 P 2173e3fd92f34ad6ad8d56d2e6b05299 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 96cba770f3334f79af3162475b54fa0f in current term 1: Already voted for candidate 2173e3fd92f34ad6ad8d56d2e6b05299 in this term.
I20260709 13:24:39.464037 28443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0701509f07184014b94c627c93b3bd59" candidate_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96cba770f3334f79af3162475b54fa0f"
I20260709 13:24:39.486863 28618 raft_consensus.cc:2749] T 68541149575b487b8c4e2360c2c93921 P 96cba770f3334f79af3162475b54fa0f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:24:39.507297 28443 raft_consensus.cc:2393] T 0701509f07184014b94c627c93b3bd59 P 96cba770f3334f79af3162475b54fa0f [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 2173e3fd92f34ad6ad8d56d2e6b05299 in current term 1: Already voted for candidate 96cba770f3334f79af3162475b54fa0f in this term.
I20260709 13:24:39.499985 28371 raft_consensus.cc:2468] T 0701509f07184014b94c627c93b3bd59 P 542ea9e2fcff4028b6766459e90bc5fc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96cba770f3334f79af3162475b54fa0f in term 1.
I20260709 13:24:39.517148 28618 raft_consensus.cc:493] T 037bfc9d92244fcb872b2ca382c21794 P 96cba770f3334f79af3162475b54fa0f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:24:39.525357 28413 leader_election.cc:304] T 0701509f07184014b94c627c93b3bd59 P 96cba770f3334f79af3162475b54fa0f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 542ea9e2fcff4028b6766459e90bc5fc, 96cba770f3334f79af3162475b54fa0f; no voters: 2173e3fd92f34ad6ad8d56d2e6b05299
I20260709 13:24:39.526640 28572 raft_consensus.cc:2804] T 0701509f07184014b94c627c93b3bd59 P 96cba770f3334f79af3162475b54fa0f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:24:39.527115 28572 raft_consensus.cc:697] T 0701509f07184014b94c627c93b3bd59 P 96cba770f3334f79af3162475b54fa0f [term 1 LEADER]: Becoming Leader. State: Replica: 96cba770f3334f79af3162475b54fa0f, State: Running, Role: LEADER
I20260709 13:24:39.519145 28618 raft_consensus.cc:515] T 037bfc9d92244fcb872b2ca382c21794 P 96cba770f3334f79af3162475b54fa0f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "542ea9e2fcff4028b6766459e90bc5fc" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38409 } } peers { permanent_uuid: "96cba770f3334f79af3162475b54fa0f" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 33585 } } peers { permanent_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 37597 } }
I20260709 13:24:39.527992 28572 consensus_queue.cc:237] T 0701509f07184014b94c627c93b3bd59 P 96cba770f3334f79af3162475b54fa0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "542ea9e2fcff4028b6766459e90bc5fc" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38409 } } peers { permanent_uuid: "96cba770f3334f79af3162475b54fa0f" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 33585 } } peers { permanent_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 37597 } }
I20260709 13:24:39.529455 28618 leader_election.cc:290] T 037bfc9d92244fcb872b2ca382c21794 P 96cba770f3334f79af3162475b54fa0f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 542ea9e2fcff4028b6766459e90bc5fc (127.14.177.129:38409), 2173e3fd92f34ad6ad8d56d2e6b05299 (127.14.177.131:37597)
W20260709 13:24:39.535614 28489 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.130:33585, user_credentials={real_user=slave}} blocked reactor thread for 47432.9us
W20260709 13:24:39.545229 28487 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.129:38409, user_credentials={real_user=slave}} blocked reactor thread for 45396.8us
I20260709 13:24:39.546814 28522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "037bfc9d92244fcb872b2ca382c21794" candidate_uuid: "96cba770f3334f79af3162475b54fa0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" is_pre_election: true
I20260709 13:24:39.547556 28522 raft_consensus.cc:2468] T 037bfc9d92244fcb872b2ca382c21794 P 2173e3fd92f34ad6ad8d56d2e6b05299 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96cba770f3334f79af3162475b54fa0f in term 0.
I20260709 13:24:39.551081 28415 leader_election.cc:304] T 037bfc9d92244fcb872b2ca382c21794 P 96cba770f3334f79af3162475b54fa0f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2173e3fd92f34ad6ad8d56d2e6b05299, 96cba770f3334f79af3162475b54fa0f; no voters: 
I20260709 13:24:39.552038 28618 raft_consensus.cc:2804] T 037bfc9d92244fcb872b2ca382c21794 P 96cba770f3334f79af3162475b54fa0f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:24:39.552428 28618 raft_consensus.cc:493] T 037bfc9d92244fcb872b2ca382c21794 P 96cba770f3334f79af3162475b54fa0f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:24:39.552778 28618 raft_consensus.cc:3060] T 037bfc9d92244fcb872b2ca382c21794 P 96cba770f3334f79af3162475b54fa0f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:24:39.548532 28269 catalog_manager.cc:5697] T b75c7d0f42e6499f936d08ff343fca5b P 542ea9e2fcff4028b6766459e90bc5fc reported cstate change: term changed from 0 to 1, leader changed from <none> to 542ea9e2fcff4028b6766459e90bc5fc (127.14.177.129). New cstate: current_term: 1 leader_uuid: "542ea9e2fcff4028b6766459e90bc5fc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "542ea9e2fcff4028b6766459e90bc5fc" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38409 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "96cba770f3334f79af3162475b54fa0f" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 33585 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 37597 } health_report { overall_health: UNKNOWN } } }
I20260709 13:24:39.554252 28269 catalog_manager.cc:5697] T 73a4db7e3a924c6f99b416e6dda95c51 P 542ea9e2fcff4028b6766459e90bc5fc reported cstate change: term changed from 0 to 1, leader changed from <none> to 542ea9e2fcff4028b6766459e90bc5fc (127.14.177.129). New cstate: current_term: 1 leader_uuid: "542ea9e2fcff4028b6766459e90bc5fc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "542ea9e2fcff4028b6766459e90bc5fc" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38409 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "96cba770f3334f79af3162475b54fa0f" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 33585 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 37597 } health_report { overall_health: UNKNOWN } } }
I20260709 13:24:39.557955 28370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "037bfc9d92244fcb872b2ca382c21794" candidate_uuid: "96cba770f3334f79af3162475b54fa0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "542ea9e2fcff4028b6766459e90bc5fc" is_pre_election: true
I20260709 13:24:39.558796 28370 raft_consensus.cc:2468] T 037bfc9d92244fcb872b2ca382c21794 P 542ea9e2fcff4028b6766459e90bc5fc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96cba770f3334f79af3162475b54fa0f in term 0.
I20260709 13:24:39.561661 28577 consensus_queue.cc:1048] T 255f88c066ac422d9d8a582d9b24a958 P 2173e3fd92f34ad6ad8d56d2e6b05299 [LEADER]: Connected to new peer: Peer: permanent_uuid: "542ea9e2fcff4028b6766459e90bc5fc" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38409 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:24:39.584553 28618 raft_consensus.cc:515] T 037bfc9d92244fcb872b2ca382c21794 P 96cba770f3334f79af3162475b54fa0f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "542ea9e2fcff4028b6766459e90bc5fc" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38409 } } peers { permanent_uuid: "96cba770f3334f79af3162475b54fa0f" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 33585 } } peers { permanent_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 37597 } }
I20260709 13:24:39.587641 28373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "037bfc9d92244fcb872b2ca382c21794" candidate_uuid: "96cba770f3334f79af3162475b54fa0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "542ea9e2fcff4028b6766459e90bc5fc"
I20260709 13:24:39.588274 28373 raft_consensus.cc:3060] T 037bfc9d92244fcb872b2ca382c21794 P 542ea9e2fcff4028b6766459e90bc5fc [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:24:39.607446 28522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "037bfc9d92244fcb872b2ca382c21794" candidate_uuid: "96cba770f3334f79af3162475b54fa0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299"
I20260709 13:24:39.608201 28522 raft_consensus.cc:3060] T 037bfc9d92244fcb872b2ca382c21794 P 2173e3fd92f34ad6ad8d56d2e6b05299 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:24:39.605195 28373 raft_consensus.cc:2468] T 037bfc9d92244fcb872b2ca382c21794 P 542ea9e2fcff4028b6766459e90bc5fc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96cba770f3334f79af3162475b54fa0f in term 1.
I20260709 13:24:39.609543 28413 leader_election.cc:304] T 037bfc9d92244fcb872b2ca382c21794 P 96cba770f3334f79af3162475b54fa0f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 542ea9e2fcff4028b6766459e90bc5fc, 96cba770f3334f79af3162475b54fa0f; no voters: 
I20260709 13:24:39.610528 28572 raft_consensus.cc:2804] T 037bfc9d92244fcb872b2ca382c21794 P 96cba770f3334f79af3162475b54fa0f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:24:39.611258 28572 raft_consensus.cc:697] T 037bfc9d92244fcb872b2ca382c21794 P 96cba770f3334f79af3162475b54fa0f [term 1 LEADER]: Becoming Leader. State: Replica: 96cba770f3334f79af3162475b54fa0f, State: Running, Role: LEADER
I20260709 13:24:39.612267 28572 consensus_queue.cc:237] T 037bfc9d92244fcb872b2ca382c21794 P 96cba770f3334f79af3162475b54fa0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "542ea9e2fcff4028b6766459e90bc5fc" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38409 } } peers { permanent_uuid: "96cba770f3334f79af3162475b54fa0f" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 33585 } } peers { permanent_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 37597 } }
W20260709 13:24:39.604786 28489 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.130:33585, user_credentials={real_user=slave}} blocked reactor thread for 67972.7us
I20260709 13:24:39.620893 28618 leader_election.cc:290] T 037bfc9d92244fcb872b2ca382c21794 P 96cba770f3334f79af3162475b54fa0f [CANDIDATE]: Term 1 election: Requested vote from peers 542ea9e2fcff4028b6766459e90bc5fc (127.14.177.129:38409), 2173e3fd92f34ad6ad8d56d2e6b05299 (127.14.177.131:37597)
I20260709 13:24:39.620407 28489 leader_election.cc:304] T 0701509f07184014b94c627c93b3bd59 P 2173e3fd92f34ad6ad8d56d2e6b05299 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2173e3fd92f34ad6ad8d56d2e6b05299; no voters: 542ea9e2fcff4028b6766459e90bc5fc, 96cba770f3334f79af3162475b54fa0f
I20260709 13:24:39.625655 28647 raft_consensus.cc:2749] T 0701509f07184014b94c627c93b3bd59 P 2173e3fd92f34ad6ad8d56d2e6b05299 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:24:39.630739 28522 raft_consensus.cc:2468] T 037bfc9d92244fcb872b2ca382c21794 P 2173e3fd92f34ad6ad8d56d2e6b05299 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96cba770f3334f79af3162475b54fa0f in term 1.
I20260709 13:24:39.643826 28611 consensus_queue.cc:1048] T b75c7d0f42e6499f936d08ff343fca5b P 542ea9e2fcff4028b6766459e90bc5fc [LEADER]: Connected to new peer: Peer: permanent_uuid: "96cba770f3334f79af3162475b54fa0f" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 33585 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:24:39.657166 28270 catalog_manager.cc:5697] T 037bfc9d92244fcb872b2ca382c21794 P 96cba770f3334f79af3162475b54fa0f reported cstate change: term changed from 0 to 1, leader changed from <none> to 96cba770f3334f79af3162475b54fa0f (127.14.177.130). New cstate: current_term: 1 leader_uuid: "96cba770f3334f79af3162475b54fa0f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "542ea9e2fcff4028b6766459e90bc5fc" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38409 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "96cba770f3334f79af3162475b54fa0f" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 33585 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 37597 } health_report { overall_health: UNKNOWN } } }
I20260709 13:24:39.659813 28270 catalog_manager.cc:5697] T 0701509f07184014b94c627c93b3bd59 P 96cba770f3334f79af3162475b54fa0f reported cstate change: term changed from 0 to 1, leader changed from <none> to 96cba770f3334f79af3162475b54fa0f (127.14.177.130). New cstate: current_term: 1 leader_uuid: "96cba770f3334f79af3162475b54fa0f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "542ea9e2fcff4028b6766459e90bc5fc" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38409 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "96cba770f3334f79af3162475b54fa0f" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 33585 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 37597 } health_report { overall_health: UNKNOWN } } }
I20260709 13:24:39.671612 28522 raft_consensus.cc:3060] T b75c7d0f42e6499f936d08ff343fca5b P 2173e3fd92f34ad6ad8d56d2e6b05299 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:24:39.694429 28619 consensus_queue.cc:1048] T b75c7d0f42e6499f936d08ff343fca5b P 542ea9e2fcff4028b6766459e90bc5fc [LEADER]: Connected to new peer: Peer: permanent_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 37597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:24:39.915431 28520 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "68541149575b487b8c4e2360c2c93921"
dest_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299"
mode: GRACEFUL
new_leader_uuid: "542ea9e2fcff4028b6766459e90bc5fc"
 from {username='slave'} at 127.0.0.1:53728
I20260709 13:24:39.916329 28520 raft_consensus.cc:606] T 68541149575b487b8c4e2360c2c93921 P 2173e3fd92f34ad6ad8d56d2e6b05299 [term 1 LEADER]: Received request to transfer leadership to TS 542ea9e2fcff4028b6766459e90bc5fc
I20260709 13:24:39.923192 28520 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "41515f53db8540a8b4f52d9ac91dab50"
dest_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299"
mode: GRACEFUL
new_leader_uuid: "542ea9e2fcff4028b6766459e90bc5fc"
 from {username='slave'} at 127.0.0.1:53728
I20260709 13:24:39.923800 28520 raft_consensus.cc:606] T 41515f53db8540a8b4f52d9ac91dab50 P 2173e3fd92f34ad6ad8d56d2e6b05299 [term 1 LEADER]: Received request to transfer leadership to TS 542ea9e2fcff4028b6766459e90bc5fc
I20260709 13:24:39.942214 28444 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "037bfc9d92244fcb872b2ca382c21794"
dest_uuid: "96cba770f3334f79af3162475b54fa0f"
mode: GRACEFUL
new_leader_uuid: "542ea9e2fcff4028b6766459e90bc5fc"
 from {username='slave'} at 127.0.0.1:34650
I20260709 13:24:39.942816 28444 raft_consensus.cc:606] T 037bfc9d92244fcb872b2ca382c21794 P 96cba770f3334f79af3162475b54fa0f [term 1 LEADER]: Received request to transfer leadership to TS 542ea9e2fcff4028b6766459e90bc5fc
I20260709 13:24:39.962028 28373 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "bbaf4eecd3c241e3aabff37f6ba0256a"
dest_uuid: "542ea9e2fcff4028b6766459e90bc5fc"
mode: GRACEFUL
new_leader_uuid: "96cba770f3334f79af3162475b54fa0f"
 from {username='slave'} at 127.0.0.1:59726
I20260709 13:24:39.962863 28373 raft_consensus.cc:606] T bbaf4eecd3c241e3aabff37f6ba0256a P 542ea9e2fcff4028b6766459e90bc5fc [term 1 LEADER]: Received request to transfer leadership to TS 96cba770f3334f79af3162475b54fa0f
I20260709 13:24:39.970162 28520 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a6cf33518cd34784a1b85843d90badb5"
dest_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299"
mode: GRACEFUL
new_leader_uuid: "96cba770f3334f79af3162475b54fa0f"
 from {username='slave'} at 127.0.0.1:53728
I20260709 13:24:39.971185 28520 raft_consensus.cc:606] T a6cf33518cd34784a1b85843d90badb5 P 2173e3fd92f34ad6ad8d56d2e6b05299 [term 1 LEADER]: Received request to transfer leadership to TS 96cba770f3334f79af3162475b54fa0f
I20260709 13:24:39.982822 28520 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "255f88c066ac422d9d8a582d9b24a958"
dest_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299"
mode: GRACEFUL
new_leader_uuid: "96cba770f3334f79af3162475b54fa0f"
 from {username='slave'} at 127.0.0.1:53728
I20260709 13:24:39.983438 28520 raft_consensus.cc:606] T 255f88c066ac422d9d8a582d9b24a958 P 2173e3fd92f34ad6ad8d56d2e6b05299 [term 1 LEADER]: Received request to transfer leadership to TS 96cba770f3334f79af3162475b54fa0f
I20260709 13:24:39.991607 28373 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b75c7d0f42e6499f936d08ff343fca5b"
dest_uuid: "542ea9e2fcff4028b6766459e90bc5fc"
mode: GRACEFUL
new_leader_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299"
 from {username='slave'} at 127.0.0.1:59726
I20260709 13:24:39.992264 28373 raft_consensus.cc:606] T b75c7d0f42e6499f936d08ff343fca5b P 542ea9e2fcff4028b6766459e90bc5fc [term 1 LEADER]: Received request to transfer leadership to TS 2173e3fd92f34ad6ad8d56d2e6b05299
I20260709 13:24:40.038635 28628 raft_consensus.cc:993] T 255f88c066ac422d9d8a582d9b24a958 P 2173e3fd92f34ad6ad8d56d2e6b05299: : Instructing follower 96cba770f3334f79af3162475b54fa0f to start an election
I20260709 13:24:40.039278 28647 raft_consensus.cc:1081] T 255f88c066ac422d9d8a582d9b24a958 P 2173e3fd92f34ad6ad8d56d2e6b05299 [term 1 LEADER]: Signalling peer 96cba770f3334f79af3162475b54fa0f to start an election
I20260709 13:24:40.040953 28444 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "255f88c066ac422d9d8a582d9b24a958"
dest_uuid: "96cba770f3334f79af3162475b54fa0f"
 from {username='slave'} at 127.0.0.1:34622
I20260709 13:24:40.041445 28444 raft_consensus.cc:493] T 255f88c066ac422d9d8a582d9b24a958 P 96cba770f3334f79af3162475b54fa0f [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:24:40.041740 28444 raft_consensus.cc:3060] T 255f88c066ac422d9d8a582d9b24a958 P 96cba770f3334f79af3162475b54fa0f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:24:40.051144 28444 raft_consensus.cc:515] T 255f88c066ac422d9d8a582d9b24a958 P 96cba770f3334f79af3162475b54fa0f [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "542ea9e2fcff4028b6766459e90bc5fc" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38409 } } peers { permanent_uuid: "96cba770f3334f79af3162475b54fa0f" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 33585 } } peers { permanent_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 37597 } }
I20260709 13:24:40.055104 28444 leader_election.cc:290] T 255f88c066ac422d9d8a582d9b24a958 P 96cba770f3334f79af3162475b54fa0f [CANDIDATE]: Term 2 election: Requested vote from peers 542ea9e2fcff4028b6766459e90bc5fc (127.14.177.129:38409), 2173e3fd92f34ad6ad8d56d2e6b05299 (127.14.177.131:37597)
I20260709 13:24:40.055156 28373 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255f88c066ac422d9d8a582d9b24a958" candidate_uuid: "96cba770f3334f79af3162475b54fa0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "542ea9e2fcff4028b6766459e90bc5fc"
I20260709 13:24:40.056402 28373 raft_consensus.cc:3060] T 255f88c066ac422d9d8a582d9b24a958 P 542ea9e2fcff4028b6766459e90bc5fc [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:24:40.056854 28522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255f88c066ac422d9d8a582d9b24a958" candidate_uuid: "96cba770f3334f79af3162475b54fa0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299"
I20260709 13:24:40.057703 28522 raft_consensus.cc:3055] T 255f88c066ac422d9d8a582d9b24a958 P 2173e3fd92f34ad6ad8d56d2e6b05299 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:24:40.058034 28522 raft_consensus.cc:740] T 255f88c066ac422d9d8a582d9b24a958 P 2173e3fd92f34ad6ad8d56d2e6b05299 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 2173e3fd92f34ad6ad8d56d2e6b05299, State: Running, Role: LEADER
I20260709 13:24:40.059363 28522 consensus_queue.cc:260] T 255f88c066ac422d9d8a582d9b24a958 P 2173e3fd92f34ad6ad8d56d2e6b05299 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "542ea9e2fcff4028b6766459e90bc5fc" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38409 } } peers { permanent_uuid: "96cba770f3334f79af3162475b54fa0f" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 33585 } } peers { permanent_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 37597 } }
I20260709 13:24:40.061614 28522 raft_consensus.cc:3060] T 255f88c066ac422d9d8a582d9b24a958 P 2173e3fd92f34ad6ad8d56d2e6b05299 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:24:40.065325 28373 raft_consensus.cc:2468] T 255f88c066ac422d9d8a582d9b24a958 P 542ea9e2fcff4028b6766459e90bc5fc [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96cba770f3334f79af3162475b54fa0f in term 2.
I20260709 13:24:40.066565 28413 leader_election.cc:304] T 255f88c066ac422d9d8a582d9b24a958 P 96cba770f3334f79af3162475b54fa0f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 542ea9e2fcff4028b6766459e90bc5fc, 96cba770f3334f79af3162475b54fa0f; no voters: 
I20260709 13:24:40.067602 28572 raft_consensus.cc:2804] T 255f88c066ac422d9d8a582d9b24a958 P 96cba770f3334f79af3162475b54fa0f [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:24:40.068118 28572 raft_consensus.cc:697] T 255f88c066ac422d9d8a582d9b24a958 P 96cba770f3334f79af3162475b54fa0f [term 2 LEADER]: Becoming Leader. State: Replica: 96cba770f3334f79af3162475b54fa0f, State: Running, Role: LEADER
I20260709 13:24:40.069379 28572 consensus_queue.cc:237] T 255f88c066ac422d9d8a582d9b24a958 P 96cba770f3334f79af3162475b54fa0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "542ea9e2fcff4028b6766459e90bc5fc" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38409 } } peers { permanent_uuid: "96cba770f3334f79af3162475b54fa0f" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 33585 } } peers { permanent_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 37597 } }
I20260709 13:24:40.070536 28522 raft_consensus.cc:2468] T 255f88c066ac422d9d8a582d9b24a958 P 2173e3fd92f34ad6ad8d56d2e6b05299 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96cba770f3334f79af3162475b54fa0f in term 2.
I20260709 13:24:40.077466 28269 catalog_manager.cc:5697] T 255f88c066ac422d9d8a582d9b24a958 P 96cba770f3334f79af3162475b54fa0f reported cstate change: term changed from 1 to 2, leader changed from 2173e3fd92f34ad6ad8d56d2e6b05299 (127.14.177.131) to 96cba770f3334f79af3162475b54fa0f (127.14.177.130). New cstate: current_term: 2 leader_uuid: "96cba770f3334f79af3162475b54fa0f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "542ea9e2fcff4028b6766459e90bc5fc" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38409 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "96cba770f3334f79af3162475b54fa0f" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 33585 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 37597 } health_report { overall_health: UNKNOWN } } }
I20260709 13:24:40.103381 28628 raft_consensus.cc:993] T 68541149575b487b8c4e2360c2c93921 P 2173e3fd92f34ad6ad8d56d2e6b05299: : Instructing follower 542ea9e2fcff4028b6766459e90bc5fc to start an election
I20260709 13:24:40.104095 28614 raft_consensus.cc:1081] T 68541149575b487b8c4e2360c2c93921 P 2173e3fd92f34ad6ad8d56d2e6b05299 [term 1 LEADER]: Signalling peer 542ea9e2fcff4028b6766459e90bc5fc to start an election
I20260709 13:24:40.106099 28372 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "68541149575b487b8c4e2360c2c93921"
dest_uuid: "542ea9e2fcff4028b6766459e90bc5fc"
 from {username='slave'} at 127.0.0.1:59710
I20260709 13:24:40.106696 28372 raft_consensus.cc:493] T 68541149575b487b8c4e2360c2c93921 P 542ea9e2fcff4028b6766459e90bc5fc [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:24:40.107156 28372 raft_consensus.cc:3060] T 68541149575b487b8c4e2360c2c93921 P 542ea9e2fcff4028b6766459e90bc5fc [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:24:40.109771 28619 raft_consensus.cc:993] T b75c7d0f42e6499f936d08ff343fca5b P 542ea9e2fcff4028b6766459e90bc5fc: : Instructing follower 2173e3fd92f34ad6ad8d56d2e6b05299 to start an election
I20260709 13:24:40.110533 28595 raft_consensus.cc:1081] T b75c7d0f42e6499f936d08ff343fca5b P 542ea9e2fcff4028b6766459e90bc5fc [term 1 LEADER]: Signalling peer 2173e3fd92f34ad6ad8d56d2e6b05299 to start an election
I20260709 13:24:40.112730 28520 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b75c7d0f42e6499f936d08ff343fca5b"
dest_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299"
 from {username='slave'} at 127.0.0.1:53704
I20260709 13:24:40.113325 28520 raft_consensus.cc:493] T b75c7d0f42e6499f936d08ff343fca5b P 2173e3fd92f34ad6ad8d56d2e6b05299 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:24:40.113713 28520 raft_consensus.cc:3060] T b75c7d0f42e6499f936d08ff343fca5b P 2173e3fd92f34ad6ad8d56d2e6b05299 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:24:40.115909 28572 consensus_queue.cc:1048] T 0701509f07184014b94c627c93b3bd59 P 96cba770f3334f79af3162475b54fa0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "542ea9e2fcff4028b6766459e90bc5fc" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38409 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:24:40.119994 28572 consensus_queue.cc:1048] T 0701509f07184014b94c627c93b3bd59 P 96cba770f3334f79af3162475b54fa0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 37597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:24:40.125808 28647 raft_consensus.cc:993] T 41515f53db8540a8b4f52d9ac91dab50 P 2173e3fd92f34ad6ad8d56d2e6b05299: : Instructing follower 542ea9e2fcff4028b6766459e90bc5fc to start an election
I20260709 13:24:40.126461 28628 raft_consensus.cc:1081] T 41515f53db8540a8b4f52d9ac91dab50 P 2173e3fd92f34ad6ad8d56d2e6b05299 [term 1 LEADER]: Signalling peer 542ea9e2fcff4028b6766459e90bc5fc to start an election
I20260709 13:24:40.130662 28370 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "41515f53db8540a8b4f52d9ac91dab50"
dest_uuid: "542ea9e2fcff4028b6766459e90bc5fc"
 from {username='slave'} at 127.0.0.1:59710
I20260709 13:24:40.131381 28370 raft_consensus.cc:493] T 41515f53db8540a8b4f52d9ac91dab50 P 542ea9e2fcff4028b6766459e90bc5fc [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:24:40.132833 28370 raft_consensus.cc:3060] T 41515f53db8540a8b4f52d9ac91dab50 P 542ea9e2fcff4028b6766459e90bc5fc [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:24:40.137259 28520 raft_consensus.cc:515] T b75c7d0f42e6499f936d08ff343fca5b P 2173e3fd92f34ad6ad8d56d2e6b05299 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "542ea9e2fcff4028b6766459e90bc5fc" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38409 } } peers { permanent_uuid: "96cba770f3334f79af3162475b54fa0f" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 33585 } } peers { permanent_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 37597 } }
I20260709 13:24:40.142129 28372 raft_consensus.cc:515] T 68541149575b487b8c4e2360c2c93921 P 542ea9e2fcff4028b6766459e90bc5fc [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 37597 } } peers { permanent_uuid: "542ea9e2fcff4028b6766459e90bc5fc" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38409 } } peers { permanent_uuid: "96cba770f3334f79af3162475b54fa0f" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 33585 } }
I20260709 13:24:40.143232 28520 leader_election.cc:290] T b75c7d0f42e6499f936d08ff343fca5b P 2173e3fd92f34ad6ad8d56d2e6b05299 [CANDIDATE]: Term 2 election: Requested vote from peers 542ea9e2fcff4028b6766459e90bc5fc (127.14.177.129:38409), 96cba770f3334f79af3162475b54fa0f (127.14.177.130:33585)
I20260709 13:24:40.144937 28444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75c7d0f42e6499f936d08ff343fca5b" candidate_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "96cba770f3334f79af3162475b54fa0f"
I20260709 13:24:40.145768 28444 raft_consensus.cc:3060] T b75c7d0f42e6499f936d08ff343fca5b P 96cba770f3334f79af3162475b54fa0f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:24:40.145889 28371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b75c7d0f42e6499f936d08ff343fca5b" candidate_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "542ea9e2fcff4028b6766459e90bc5fc"
I20260709 13:24:40.146788 28371 raft_consensus.cc:3055] T b75c7d0f42e6499f936d08ff343fca5b P 542ea9e2fcff4028b6766459e90bc5fc [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:24:40.147137 28371 raft_consensus.cc:740] T b75c7d0f42e6499f936d08ff343fca5b P 542ea9e2fcff4028b6766459e90bc5fc [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 542ea9e2fcff4028b6766459e90bc5fc, State: Running, Role: LEADER
I20260709 13:24:40.148208 28371 consensus_queue.cc:260] T b75c7d0f42e6499f936d08ff343fca5b P 542ea9e2fcff4028b6766459e90bc5fc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "542ea9e2fcff4028b6766459e90bc5fc" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38409 } } peers { permanent_uuid: "96cba770f3334f79af3162475b54fa0f" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 33585 } } peers { permanent_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 37597 } }
I20260709 13:24:40.148918 28370 raft_consensus.cc:515] T 41515f53db8540a8b4f52d9ac91dab50 P 542ea9e2fcff4028b6766459e90bc5fc [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "542ea9e2fcff4028b6766459e90bc5fc" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38409 } } peers { permanent_uuid: "96cba770f3334f79af3162475b54fa0f" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 33585 } } peers { permanent_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 37597 } }
I20260709 13:24:40.149886 28371 raft_consensus.cc:3060] T b75c7d0f42e6499f936d08ff343fca5b P 542ea9e2fcff4028b6766459e90bc5fc [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:24:40.153007 28370 leader_election.cc:290] T 41515f53db8540a8b4f52d9ac91dab50 P 542ea9e2fcff4028b6766459e90bc5fc [CANDIDATE]: Term 2 election: Requested vote from peers 96cba770f3334f79af3162475b54fa0f (127.14.177.130:33585), 2173e3fd92f34ad6ad8d56d2e6b05299 (127.14.177.131:37597)
I20260709 13:24:40.153507 28523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41515f53db8540a8b4f52d9ac91dab50" candidate_uuid: "542ea9e2fcff4028b6766459e90bc5fc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299"
I20260709 13:24:40.154232 28523 raft_consensus.cc:3055] T 41515f53db8540a8b4f52d9ac91dab50 P 2173e3fd92f34ad6ad8d56d2e6b05299 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:24:40.154579 28523 raft_consensus.cc:740] T 41515f53db8540a8b4f52d9ac91dab50 P 2173e3fd92f34ad6ad8d56d2e6b05299 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 2173e3fd92f34ad6ad8d56d2e6b05299, State: Running, Role: LEADER
I20260709 13:24:40.154703 28448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41515f53db8540a8b4f52d9ac91dab50" candidate_uuid: "542ea9e2fcff4028b6766459e90bc5fc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "96cba770f3334f79af3162475b54fa0f"
I20260709 13:24:40.155504 28448 raft_consensus.cc:3060] T 41515f53db8540a8b4f52d9ac91dab50 P 96cba770f3334f79af3162475b54fa0f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:24:40.156491 28372 leader_election.cc:290] T 68541149575b487b8c4e2360c2c93921 P 542ea9e2fcff4028b6766459e90bc5fc [CANDIDATE]: Term 2 election: Requested vote from peers 2173e3fd92f34ad6ad8d56d2e6b05299 (127.14.177.131:37597), 96cba770f3334f79af3162475b54fa0f (127.14.177.130:33585)
I20260709 13:24:40.155798 28523 consensus_queue.cc:260] T 41515f53db8540a8b4f52d9ac91dab50 P 2173e3fd92f34ad6ad8d56d2e6b05299 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "542ea9e2fcff4028b6766459e90bc5fc" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38409 } } peers { permanent_uuid: "96cba770f3334f79af3162475b54fa0f" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 33585 } } peers { permanent_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 37597 } }
I20260709 13:24:40.155001 28444 raft_consensus.cc:2468] T b75c7d0f42e6499f936d08ff343fca5b P 96cba770f3334f79af3162475b54fa0f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2173e3fd92f34ad6ad8d56d2e6b05299 in term 2.
I20260709 13:24:40.157720 28523 raft_consensus.cc:3060] T 41515f53db8540a8b4f52d9ac91dab50 P 2173e3fd92f34ad6ad8d56d2e6b05299 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:24:40.145782 28520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68541149575b487b8c4e2360c2c93921" candidate_uuid: "542ea9e2fcff4028b6766459e90bc5fc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299"
I20260709 13:24:40.158574 28520 raft_consensus.cc:3055] T 68541149575b487b8c4e2360c2c93921 P 2173e3fd92f34ad6ad8d56d2e6b05299 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:24:40.158921 28520 raft_consensus.cc:740] T 68541149575b487b8c4e2360c2c93921 P 2173e3fd92f34ad6ad8d56d2e6b05299 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 2173e3fd92f34ad6ad8d56d2e6b05299, State: Running, Role: LEADER
I20260709 13:24:40.160169 28520 consensus_queue.cc:260] T 68541149575b487b8c4e2360c2c93921 P 2173e3fd92f34ad6ad8d56d2e6b05299 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 37597 } } peers { permanent_uuid: "542ea9e2fcff4028b6766459e90bc5fc" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38409 } } peers { permanent_uuid: "96cba770f3334f79af3162475b54fa0f" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 33585 } }
I20260709 13:24:40.161675 28520 raft_consensus.cc:3060] T 68541149575b487b8c4e2360c2c93921 P 2173e3fd92f34ad6ad8d56d2e6b05299 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:24:40.161679 28489 leader_election.cc:304] T b75c7d0f42e6499f936d08ff343fca5b P 2173e3fd92f34ad6ad8d56d2e6b05299 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2173e3fd92f34ad6ad8d56d2e6b05299, 96cba770f3334f79af3162475b54fa0f; no voters: 
I20260709 13:24:40.170135 28614 raft_consensus.cc:2804] T b75c7d0f42e6499f936d08ff343fca5b P 2173e3fd92f34ad6ad8d56d2e6b05299 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:24:40.170687 28614 raft_consensus.cc:697] T b75c7d0f42e6499f936d08ff343fca5b P 2173e3fd92f34ad6ad8d56d2e6b05299 [term 2 LEADER]: Becoming Leader. State: Replica: 2173e3fd92f34ad6ad8d56d2e6b05299, State: Running, Role: LEADER
I20260709 13:24:40.172144 28523 raft_consensus.cc:2468] T 41515f53db8540a8b4f52d9ac91dab50 P 2173e3fd92f34ad6ad8d56d2e6b05299 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 542ea9e2fcff4028b6766459e90bc5fc in term 2.
I20260709 13:24:40.171819 28614 consensus_queue.cc:237] T b75c7d0f42e6499f936d08ff343fca5b P 2173e3fd92f34ad6ad8d56d2e6b05299 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "542ea9e2fcff4028b6766459e90bc5fc" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38409 } } peers { permanent_uuid: "96cba770f3334f79af3162475b54fa0f" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 33585 } } peers { permanent_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 37597 } }
I20260709 13:24:40.174845 28520 raft_consensus.cc:2468] T 68541149575b487b8c4e2360c2c93921 P 2173e3fd92f34ad6ad8d56d2e6b05299 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 542ea9e2fcff4028b6766459e90bc5fc in term 2.
I20260709 13:24:40.179584 28340 leader_election.cc:304] T 41515f53db8540a8b4f52d9ac91dab50 P 542ea9e2fcff4028b6766459e90bc5fc [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2173e3fd92f34ad6ad8d56d2e6b05299, 542ea9e2fcff4028b6766459e90bc5fc; no voters: 
I20260709 13:24:40.180851 28619 raft_consensus.cc:2804] T 41515f53db8540a8b4f52d9ac91dab50 P 542ea9e2fcff4028b6766459e90bc5fc [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:24:40.181354 28619 raft_consensus.cc:697] T 41515f53db8540a8b4f52d9ac91dab50 P 542ea9e2fcff4028b6766459e90bc5fc [term 2 LEADER]: Becoming Leader. State: Replica: 542ea9e2fcff4028b6766459e90bc5fc, State: Running, Role: LEADER
I20260709 13:24:40.182416 28619 consensus_queue.cc:237] T 41515f53db8540a8b4f52d9ac91dab50 P 542ea9e2fcff4028b6766459e90bc5fc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "542ea9e2fcff4028b6766459e90bc5fc" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38409 } } peers { permanent_uuid: "96cba770f3334f79af3162475b54fa0f" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 33585 } } peers { permanent_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 37597 } }
I20260709 13:24:40.183737 28444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68541149575b487b8c4e2360c2c93921" candidate_uuid: "542ea9e2fcff4028b6766459e90bc5fc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "96cba770f3334f79af3162475b54fa0f"
I20260709 13:24:40.183058 28269 catalog_manager.cc:5697] T b75c7d0f42e6499f936d08ff343fca5b P 2173e3fd92f34ad6ad8d56d2e6b05299 reported cstate change: term changed from 1 to 2, leader changed from 542ea9e2fcff4028b6766459e90bc5fc (127.14.177.129) to 2173e3fd92f34ad6ad8d56d2e6b05299 (127.14.177.131). New cstate: current_term: 2 leader_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "542ea9e2fcff4028b6766459e90bc5fc" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38409 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "96cba770f3334f79af3162475b54fa0f" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 33585 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 37597 } health_report { overall_health: HEALTHY } } }
I20260709 13:24:40.184408 28444 raft_consensus.cc:3060] T 68541149575b487b8c4e2360c2c93921 P 96cba770f3334f79af3162475b54fa0f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:24:40.193141 28448 raft_consensus.cc:2468] T 41515f53db8540a8b4f52d9ac91dab50 P 96cba770f3334f79af3162475b54fa0f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 542ea9e2fcff4028b6766459e90bc5fc in term 2.
W20260709 13:24:40.197386 28544 debug-util.cc:398] Leaking SignalData structure 0x7b08001cfd20 after lost signal to thread 15049
W20260709 13:24:40.199191 28544 debug-util.cc:398] Leaking SignalData structure 0x7b08002b50c0 after lost signal to thread 28306
I20260709 13:24:40.199182 28572 consensus_queue.cc:1048] T 037bfc9d92244fcb872b2ca382c21794 P 96cba770f3334f79af3162475b54fa0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "542ea9e2fcff4028b6766459e90bc5fc" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38409 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.003s
I20260709 13:24:40.204042 28595 raft_consensus.cc:993] T bbaf4eecd3c241e3aabff37f6ba0256a P 542ea9e2fcff4028b6766459e90bc5fc: : Instructing follower 96cba770f3334f79af3162475b54fa0f to start an election
I20260709 13:24:40.204921 28595 raft_consensus.cc:1081] T bbaf4eecd3c241e3aabff37f6ba0256a P 542ea9e2fcff4028b6766459e90bc5fc [term 1 LEADER]: Signalling peer 96cba770f3334f79af3162475b54fa0f to start an election
I20260709 13:24:40.207242 28340 leader_election.cc:304] T 68541149575b487b8c4e2360c2c93921 P 542ea9e2fcff4028b6766459e90bc5fc [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2173e3fd92f34ad6ad8d56d2e6b05299, 542ea9e2fcff4028b6766459e90bc5fc; no voters: 
I20260709 13:24:40.208029 28443 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "bbaf4eecd3c241e3aabff37f6ba0256a"
dest_uuid: "96cba770f3334f79af3162475b54fa0f"
 from {username='slave'} at 127.0.0.1:34636
I20260709 13:24:40.208302 28595 raft_consensus.cc:2804] T 68541149575b487b8c4e2360c2c93921 P 542ea9e2fcff4028b6766459e90bc5fc [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:24:40.209013 28443 raft_consensus.cc:493] T bbaf4eecd3c241e3aabff37f6ba0256a P 96cba770f3334f79af3162475b54fa0f [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:24:40.209141 28595 raft_consensus.cc:697] T 68541149575b487b8c4e2360c2c93921 P 542ea9e2fcff4028b6766459e90bc5fc [term 2 LEADER]: Becoming Leader. State: Replica: 542ea9e2fcff4028b6766459e90bc5fc, State: Running, Role: LEADER
I20260709 13:24:40.209646 28443 raft_consensus.cc:3060] T bbaf4eecd3c241e3aabff37f6ba0256a P 96cba770f3334f79af3162475b54fa0f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:24:40.210593 28595 consensus_queue.cc:237] T 68541149575b487b8c4e2360c2c93921 P 542ea9e2fcff4028b6766459e90bc5fc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 37597 } } peers { permanent_uuid: "542ea9e2fcff4028b6766459e90bc5fc" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38409 } } peers { permanent_uuid: "96cba770f3334f79af3162475b54fa0f" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 33585 } }
I20260709 13:24:40.213040 28270 catalog_manager.cc:5697] T 41515f53db8540a8b4f52d9ac91dab50 P 542ea9e2fcff4028b6766459e90bc5fc reported cstate change: term changed from 1 to 2, leader changed from 2173e3fd92f34ad6ad8d56d2e6b05299 (127.14.177.131) to 542ea9e2fcff4028b6766459e90bc5fc (127.14.177.129). New cstate: current_term: 2 leader_uuid: "542ea9e2fcff4028b6766459e90bc5fc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "542ea9e2fcff4028b6766459e90bc5fc" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38409 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "96cba770f3334f79af3162475b54fa0f" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 33585 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 37597 } health_report { overall_health: UNKNOWN } } }
I20260709 13:24:40.227200 28371 raft_consensus.cc:2468] T b75c7d0f42e6499f936d08ff343fca5b P 542ea9e2fcff4028b6766459e90bc5fc [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2173e3fd92f34ad6ad8d56d2e6b05299 in term 2.
I20260709 13:24:40.239079 28444 raft_consensus.cc:2468] T 68541149575b487b8c4e2360c2c93921 P 96cba770f3334f79af3162475b54fa0f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 542ea9e2fcff4028b6766459e90bc5fc in term 2.
I20260709 13:24:40.243754 28572 raft_consensus.cc:993] T 037bfc9d92244fcb872b2ca382c21794 P 96cba770f3334f79af3162475b54fa0f: : Instructing follower 542ea9e2fcff4028b6766459e90bc5fc to start an election
I20260709 13:24:40.244388 28572 raft_consensus.cc:1081] T 037bfc9d92244fcb872b2ca382c21794 P 96cba770f3334f79af3162475b54fa0f [term 1 LEADER]: Signalling peer 542ea9e2fcff4028b6766459e90bc5fc to start an election
I20260709 13:24:40.247327 28372 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "037bfc9d92244fcb872b2ca382c21794"
dest_uuid: "542ea9e2fcff4028b6766459e90bc5fc"
 from {username='slave'} at 127.0.0.1:59714
I20260709 13:24:40.246580 28270 catalog_manager.cc:5697] T 68541149575b487b8c4e2360c2c93921 P 542ea9e2fcff4028b6766459e90bc5fc reported cstate change: term changed from 1 to 2, leader changed from 2173e3fd92f34ad6ad8d56d2e6b05299 (127.14.177.131) to 542ea9e2fcff4028b6766459e90bc5fc (127.14.177.129). New cstate: current_term: 2 leader_uuid: "542ea9e2fcff4028b6766459e90bc5fc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 37597 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "542ea9e2fcff4028b6766459e90bc5fc" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38409 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "96cba770f3334f79af3162475b54fa0f" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 33585 } health_report { overall_health: UNKNOWN } } }
I20260709 13:24:40.249516 28666 consensus_queue.cc:1048] T 037bfc9d92244fcb872b2ca382c21794 P 96cba770f3334f79af3162475b54fa0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 37597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:24:40.250026 28372 raft_consensus.cc:493] T 037bfc9d92244fcb872b2ca382c21794 P 542ea9e2fcff4028b6766459e90bc5fc [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:24:40.250563 28372 raft_consensus.cc:3060] T 037bfc9d92244fcb872b2ca382c21794 P 542ea9e2fcff4028b6766459e90bc5fc [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:24:40.243850 28443 raft_consensus.cc:515] T bbaf4eecd3c241e3aabff37f6ba0256a P 96cba770f3334f79af3162475b54fa0f [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "542ea9e2fcff4028b6766459e90bc5fc" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38409 } } peers { permanent_uuid: "96cba770f3334f79af3162475b54fa0f" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 33585 } } peers { permanent_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 37597 } }
I20260709 13:24:40.257113 28443 leader_election.cc:290] T bbaf4eecd3c241e3aabff37f6ba0256a P 96cba770f3334f79af3162475b54fa0f [CANDIDATE]: Term 2 election: Requested vote from peers 542ea9e2fcff4028b6766459e90bc5fc (127.14.177.129:38409), 2173e3fd92f34ad6ad8d56d2e6b05299 (127.14.177.131:37597)
I20260709 13:24:40.258682 28522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbaf4eecd3c241e3aabff37f6ba0256a" candidate_uuid: "96cba770f3334f79af3162475b54fa0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299"
I20260709 13:24:40.259431 28522 raft_consensus.cc:3060] T bbaf4eecd3c241e3aabff37f6ba0256a P 2173e3fd92f34ad6ad8d56d2e6b05299 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:24:40.260799 28371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbaf4eecd3c241e3aabff37f6ba0256a" candidate_uuid: "96cba770f3334f79af3162475b54fa0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "542ea9e2fcff4028b6766459e90bc5fc"
I20260709 13:24:40.261476 28371 raft_consensus.cc:3055] T bbaf4eecd3c241e3aabff37f6ba0256a P 542ea9e2fcff4028b6766459e90bc5fc [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:24:40.261798 28371 raft_consensus.cc:740] T bbaf4eecd3c241e3aabff37f6ba0256a P 542ea9e2fcff4028b6766459e90bc5fc [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 542ea9e2fcff4028b6766459e90bc5fc, State: Running, Role: LEADER
I20260709 13:24:40.262879 28371 consensus_queue.cc:260] T bbaf4eecd3c241e3aabff37f6ba0256a P 542ea9e2fcff4028b6766459e90bc5fc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "542ea9e2fcff4028b6766459e90bc5fc" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38409 } } peers { permanent_uuid: "96cba770f3334f79af3162475b54fa0f" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 33585 } } peers { permanent_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 37597 } }
I20260709 13:24:40.267068 28371 raft_consensus.cc:3060] T bbaf4eecd3c241e3aabff37f6ba0256a P 542ea9e2fcff4028b6766459e90bc5fc [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:24:40.268791 28522 raft_consensus.cc:2468] T bbaf4eecd3c241e3aabff37f6ba0256a P 2173e3fd92f34ad6ad8d56d2e6b05299 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96cba770f3334f79af3162475b54fa0f in term 2.
I20260709 13:24:40.270484 28415 leader_election.cc:304] T bbaf4eecd3c241e3aabff37f6ba0256a P 96cba770f3334f79af3162475b54fa0f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2173e3fd92f34ad6ad8d56d2e6b05299, 96cba770f3334f79af3162475b54fa0f; no voters: 
I20260709 13:24:40.265779 28372 raft_consensus.cc:515] T 037bfc9d92244fcb872b2ca382c21794 P 542ea9e2fcff4028b6766459e90bc5fc [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "542ea9e2fcff4028b6766459e90bc5fc" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38409 } } peers { permanent_uuid: "96cba770f3334f79af3162475b54fa0f" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 33585 } } peers { permanent_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 37597 } }
I20260709 13:24:40.272112 28665 raft_consensus.cc:2804] T bbaf4eecd3c241e3aabff37f6ba0256a P 96cba770f3334f79af3162475b54fa0f [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:24:40.272811 28665 raft_consensus.cc:697] T bbaf4eecd3c241e3aabff37f6ba0256a P 96cba770f3334f79af3162475b54fa0f [term 2 LEADER]: Becoming Leader. State: Replica: 96cba770f3334f79af3162475b54fa0f, State: Running, Role: LEADER
I20260709 13:24:40.274206 28665 consensus_queue.cc:237] T bbaf4eecd3c241e3aabff37f6ba0256a P 96cba770f3334f79af3162475b54fa0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "542ea9e2fcff4028b6766459e90bc5fc" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38409 } } peers { permanent_uuid: "96cba770f3334f79af3162475b54fa0f" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 33585 } } peers { permanent_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 37597 } }
I20260709 13:24:40.290133 28443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "037bfc9d92244fcb872b2ca382c21794" candidate_uuid: "542ea9e2fcff4028b6766459e90bc5fc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "96cba770f3334f79af3162475b54fa0f"
I20260709 13:24:40.289549 28270 catalog_manager.cc:5697] T bbaf4eecd3c241e3aabff37f6ba0256a P 96cba770f3334f79af3162475b54fa0f reported cstate change: term changed from 1 to 2, leader changed from 542ea9e2fcff4028b6766459e90bc5fc (127.14.177.129) to 96cba770f3334f79af3162475b54fa0f (127.14.177.130). New cstate: current_term: 2 leader_uuid: "96cba770f3334f79af3162475b54fa0f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "542ea9e2fcff4028b6766459e90bc5fc" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38409 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "96cba770f3334f79af3162475b54fa0f" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 33585 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 37597 } health_report { overall_health: UNKNOWN } } }
I20260709 13:24:40.290884 28443 raft_consensus.cc:3055] T 037bfc9d92244fcb872b2ca382c21794 P 96cba770f3334f79af3162475b54fa0f [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:24:40.291141 28522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "037bfc9d92244fcb872b2ca382c21794" candidate_uuid: "542ea9e2fcff4028b6766459e90bc5fc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299"
I20260709 13:24:40.291345 28443 raft_consensus.cc:740] T 037bfc9d92244fcb872b2ca382c21794 P 96cba770f3334f79af3162475b54fa0f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 96cba770f3334f79af3162475b54fa0f, State: Running, Role: LEADER
I20260709 13:24:40.292665 28443 consensus_queue.cc:260] T 037bfc9d92244fcb872b2ca382c21794 P 96cba770f3334f79af3162475b54fa0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "542ea9e2fcff4028b6766459e90bc5fc" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38409 } } peers { permanent_uuid: "96cba770f3334f79af3162475b54fa0f" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 33585 } } peers { permanent_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 37597 } }
I20260709 13:24:40.293116 28372 leader_election.cc:290] T 037bfc9d92244fcb872b2ca382c21794 P 542ea9e2fcff4028b6766459e90bc5fc [CANDIDATE]: Term 2 election: Requested vote from peers 96cba770f3334f79af3162475b54fa0f (127.14.177.130:33585), 2173e3fd92f34ad6ad8d56d2e6b05299 (127.14.177.131:37597)
I20260709 13:24:40.293761 28443 raft_consensus.cc:3060] T 037bfc9d92244fcb872b2ca382c21794 P 96cba770f3334f79af3162475b54fa0f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:24:40.296911 28371 raft_consensus.cc:2468] T bbaf4eecd3c241e3aabff37f6ba0256a P 542ea9e2fcff4028b6766459e90bc5fc [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96cba770f3334f79af3162475b54fa0f in term 2.
I20260709 13:24:40.303998 28443 raft_consensus.cc:2468] T 037bfc9d92244fcb872b2ca382c21794 P 96cba770f3334f79af3162475b54fa0f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 542ea9e2fcff4028b6766459e90bc5fc in term 2.
I20260709 13:24:40.305019 28340 leader_election.cc:304] T 037bfc9d92244fcb872b2ca382c21794 P 542ea9e2fcff4028b6766459e90bc5fc [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 542ea9e2fcff4028b6766459e90bc5fc, 96cba770f3334f79af3162475b54fa0f; no voters: 2173e3fd92f34ad6ad8d56d2e6b05299
I20260709 13:24:40.305783 28595 raft_consensus.cc:2804] T 037bfc9d92244fcb872b2ca382c21794 P 542ea9e2fcff4028b6766459e90bc5fc [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:24:40.306356 28595 raft_consensus.cc:697] T 037bfc9d92244fcb872b2ca382c21794 P 542ea9e2fcff4028b6766459e90bc5fc [term 2 LEADER]: Becoming Leader. State: Replica: 542ea9e2fcff4028b6766459e90bc5fc, State: Running, Role: LEADER
I20260709 13:24:40.307536 28595 consensus_queue.cc:237] T 037bfc9d92244fcb872b2ca382c21794 P 542ea9e2fcff4028b6766459e90bc5fc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "542ea9e2fcff4028b6766459e90bc5fc" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38409 } } peers { permanent_uuid: "96cba770f3334f79af3162475b54fa0f" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 33585 } } peers { permanent_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 37597 } }
I20260709 13:24:40.316363 28269 catalog_manager.cc:5697] T 037bfc9d92244fcb872b2ca382c21794 P 542ea9e2fcff4028b6766459e90bc5fc reported cstate change: term changed from 1 to 2, leader changed from 96cba770f3334f79af3162475b54fa0f (127.14.177.130) to 542ea9e2fcff4028b6766459e90bc5fc (127.14.177.129). New cstate: current_term: 2 leader_uuid: "542ea9e2fcff4028b6766459e90bc5fc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "542ea9e2fcff4028b6766459e90bc5fc" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38409 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "96cba770f3334f79af3162475b54fa0f" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 33585 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 37597 } health_report { overall_health: UNKNOWN } } }
I20260709 13:24:40.386737 28647 raft_consensus.cc:993] T a6cf33518cd34784a1b85843d90badb5 P 2173e3fd92f34ad6ad8d56d2e6b05299: : Instructing follower 96cba770f3334f79af3162475b54fa0f to start an election
I20260709 13:24:40.387324 28647 raft_consensus.cc:1081] T a6cf33518cd34784a1b85843d90badb5 P 2173e3fd92f34ad6ad8d56d2e6b05299 [term 1 LEADER]: Signalling peer 96cba770f3334f79af3162475b54fa0f to start an election
I20260709 13:24:40.389595 28443 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a6cf33518cd34784a1b85843d90badb5"
dest_uuid: "96cba770f3334f79af3162475b54fa0f"
 from {username='slave'} at 127.0.0.1:34622
I20260709 13:24:40.390255 28443 raft_consensus.cc:493] T a6cf33518cd34784a1b85843d90badb5 P 96cba770f3334f79af3162475b54fa0f [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:24:40.390610 28443 raft_consensus.cc:3060] T a6cf33518cd34784a1b85843d90badb5 P 96cba770f3334f79af3162475b54fa0f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:24:40.396323 28443 raft_consensus.cc:515] T a6cf33518cd34784a1b85843d90badb5 P 96cba770f3334f79af3162475b54fa0f [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "542ea9e2fcff4028b6766459e90bc5fc" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38409 } } peers { permanent_uuid: "96cba770f3334f79af3162475b54fa0f" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 33585 } } peers { permanent_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 37597 } }
I20260709 13:24:40.399235 28443 leader_election.cc:290] T a6cf33518cd34784a1b85843d90badb5 P 96cba770f3334f79af3162475b54fa0f [CANDIDATE]: Term 2 election: Requested vote from peers 542ea9e2fcff4028b6766459e90bc5fc (127.14.177.129:38409), 2173e3fd92f34ad6ad8d56d2e6b05299 (127.14.177.131:37597)
I20260709 13:24:40.399268 28371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6cf33518cd34784a1b85843d90badb5" candidate_uuid: "96cba770f3334f79af3162475b54fa0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "542ea9e2fcff4028b6766459e90bc5fc"
I20260709 13:24:40.399554 28520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6cf33518cd34784a1b85843d90badb5" candidate_uuid: "96cba770f3334f79af3162475b54fa0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299"
I20260709 13:24:40.400120 28371 raft_consensus.cc:3060] T a6cf33518cd34784a1b85843d90badb5 P 542ea9e2fcff4028b6766459e90bc5fc [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:24:40.400408 28520 raft_consensus.cc:3055] T a6cf33518cd34784a1b85843d90badb5 P 2173e3fd92f34ad6ad8d56d2e6b05299 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:24:40.400872 28520 raft_consensus.cc:740] T a6cf33518cd34784a1b85843d90badb5 P 2173e3fd92f34ad6ad8d56d2e6b05299 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 2173e3fd92f34ad6ad8d56d2e6b05299, State: Running, Role: LEADER
I20260709 13:24:40.402217 28520 consensus_queue.cc:260] T a6cf33518cd34784a1b85843d90badb5 P 2173e3fd92f34ad6ad8d56d2e6b05299 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "542ea9e2fcff4028b6766459e90bc5fc" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38409 } } peers { permanent_uuid: "96cba770f3334f79af3162475b54fa0f" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 33585 } } peers { permanent_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 37597 } }
I20260709 13:24:40.404017 28520 raft_consensus.cc:3060] T a6cf33518cd34784a1b85843d90badb5 P 2173e3fd92f34ad6ad8d56d2e6b05299 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:24:40.406524 28371 raft_consensus.cc:2468] T a6cf33518cd34784a1b85843d90badb5 P 542ea9e2fcff4028b6766459e90bc5fc [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96cba770f3334f79af3162475b54fa0f in term 2.
I20260709 13:24:40.407997 28413 leader_election.cc:304] T a6cf33518cd34784a1b85843d90badb5 P 96cba770f3334f79af3162475b54fa0f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 542ea9e2fcff4028b6766459e90bc5fc, 96cba770f3334f79af3162475b54fa0f; no voters: 
I20260709 13:24:40.408910 28665 raft_consensus.cc:2804] T a6cf33518cd34784a1b85843d90badb5 P 96cba770f3334f79af3162475b54fa0f [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:24:40.409413 28665 raft_consensus.cc:697] T a6cf33518cd34784a1b85843d90badb5 P 96cba770f3334f79af3162475b54fa0f [term 2 LEADER]: Becoming Leader. State: Replica: 96cba770f3334f79af3162475b54fa0f, State: Running, Role: LEADER
I20260709 13:24:40.410614 28665 consensus_queue.cc:237] T a6cf33518cd34784a1b85843d90badb5 P 96cba770f3334f79af3162475b54fa0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "542ea9e2fcff4028b6766459e90bc5fc" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38409 } } peers { permanent_uuid: "96cba770f3334f79af3162475b54fa0f" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 33585 } } peers { permanent_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 37597 } }
I20260709 13:24:40.414770 28520 raft_consensus.cc:2468] T a6cf33518cd34784a1b85843d90badb5 P 2173e3fd92f34ad6ad8d56d2e6b05299 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96cba770f3334f79af3162475b54fa0f in term 2.
I20260709 13:24:40.419117 28269 catalog_manager.cc:5697] T a6cf33518cd34784a1b85843d90badb5 P 96cba770f3334f79af3162475b54fa0f reported cstate change: term changed from 1 to 2, leader changed from 2173e3fd92f34ad6ad8d56d2e6b05299 (127.14.177.131) to 96cba770f3334f79af3162475b54fa0f (127.14.177.130). New cstate: current_term: 2 leader_uuid: "96cba770f3334f79af3162475b54fa0f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "542ea9e2fcff4028b6766459e90bc5fc" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38409 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "96cba770f3334f79af3162475b54fa0f" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 33585 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 37597 } health_report { overall_health: UNKNOWN } } }
I20260709 13:24:40.464831 28520 raft_consensus.cc:1275] T 255f88c066ac422d9d8a582d9b24a958 P 2173e3fd92f34ad6ad8d56d2e6b05299 [term 2 FOLLOWER]: Refusing update from remote peer 96cba770f3334f79af3162475b54fa0f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:24:40.466629 28665 consensus_queue.cc:1048] T 255f88c066ac422d9d8a582d9b24a958 P 96cba770f3334f79af3162475b54fa0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 37597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:24:40.492733 28371 raft_consensus.cc:1275] T 255f88c066ac422d9d8a582d9b24a958 P 542ea9e2fcff4028b6766459e90bc5fc [term 2 FOLLOWER]: Refusing update from remote peer 96cba770f3334f79af3162475b54fa0f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:24:40.494235 28666 consensus_queue.cc:1048] T 255f88c066ac422d9d8a582d9b24a958 P 96cba770f3334f79af3162475b54fa0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "542ea9e2fcff4028b6766459e90bc5fc" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38409 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:24:40.556529 28443 raft_consensus.cc:1275] T b75c7d0f42e6499f936d08ff343fca5b P 96cba770f3334f79af3162475b54fa0f [term 2 FOLLOWER]: Refusing update from remote peer 2173e3fd92f34ad6ad8d56d2e6b05299: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:24:40.557963 28614 consensus_queue.cc:1048] T b75c7d0f42e6499f936d08ff343fca5b P 2173e3fd92f34ad6ad8d56d2e6b05299 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96cba770f3334f79af3162475b54fa0f" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 33585 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:24:40.566972 28371 raft_consensus.cc:1275] T b75c7d0f42e6499f936d08ff343fca5b P 542ea9e2fcff4028b6766459e90bc5fc [term 2 FOLLOWER]: Refusing update from remote peer 2173e3fd92f34ad6ad8d56d2e6b05299: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:24:40.569097 28614 consensus_queue.cc:1048] T b75c7d0f42e6499f936d08ff343fca5b P 2173e3fd92f34ad6ad8d56d2e6b05299 [LEADER]: Connected to new peer: Peer: permanent_uuid: "542ea9e2fcff4028b6766459e90bc5fc" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38409 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:24:40.592809 28443 raft_consensus.cc:1275] T 68541149575b487b8c4e2360c2c93921 P 96cba770f3334f79af3162475b54fa0f [term 2 FOLLOWER]: Refusing update from remote peer 542ea9e2fcff4028b6766459e90bc5fc: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:24:40.594487 28595 consensus_queue.cc:1048] T 68541149575b487b8c4e2360c2c93921 P 542ea9e2fcff4028b6766459e90bc5fc [LEADER]: Connected to new peer: Peer: permanent_uuid: "96cba770f3334f79af3162475b54fa0f" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 33585 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:24:40.609673 28520 raft_consensus.cc:1275] T 68541149575b487b8c4e2360c2c93921 P 2173e3fd92f34ad6ad8d56d2e6b05299 [term 2 FOLLOWER]: Refusing update from remote peer 542ea9e2fcff4028b6766459e90bc5fc: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:24:40.611675 28662 consensus_queue.cc:1048] T 68541149575b487b8c4e2360c2c93921 P 542ea9e2fcff4028b6766459e90bc5fc [LEADER]: Connected to new peer: Peer: permanent_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 37597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:24:40.698185 28520 raft_consensus.cc:1275] T bbaf4eecd3c241e3aabff37f6ba0256a P 2173e3fd92f34ad6ad8d56d2e6b05299 [term 2 FOLLOWER]: Refusing update from remote peer 96cba770f3334f79af3162475b54fa0f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:24:40.699811 28666 consensus_queue.cc:1048] T bbaf4eecd3c241e3aabff37f6ba0256a P 96cba770f3334f79af3162475b54fa0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 37597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:24:40.712607 28371 raft_consensus.cc:1275] T bbaf4eecd3c241e3aabff37f6ba0256a P 542ea9e2fcff4028b6766459e90bc5fc [term 2 FOLLOWER]: Refusing update from remote peer 96cba770f3334f79af3162475b54fa0f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:24:40.714150 28618 consensus_queue.cc:1048] T bbaf4eecd3c241e3aabff37f6ba0256a P 96cba770f3334f79af3162475b54fa0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "542ea9e2fcff4028b6766459e90bc5fc" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38409 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:24:40.788926 28520 raft_consensus.cc:1275] T 41515f53db8540a8b4f52d9ac91dab50 P 2173e3fd92f34ad6ad8d56d2e6b05299 [term 2 FOLLOWER]: Refusing update from remote peer 542ea9e2fcff4028b6766459e90bc5fc: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:24:40.790629 28662 consensus_queue.cc:1048] T 41515f53db8540a8b4f52d9ac91dab50 P 542ea9e2fcff4028b6766459e90bc5fc [LEADER]: Connected to new peer: Peer: permanent_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 37597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:24:40.802433 28443 raft_consensus.cc:1275] T 41515f53db8540a8b4f52d9ac91dab50 P 96cba770f3334f79af3162475b54fa0f [term 2 FOLLOWER]: Refusing update from remote peer 542ea9e2fcff4028b6766459e90bc5fc: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:24:40.804731 28619 consensus_queue.cc:1048] T 41515f53db8540a8b4f52d9ac91dab50 P 542ea9e2fcff4028b6766459e90bc5fc [LEADER]: Connected to new peer: Peer: permanent_uuid: "96cba770f3334f79af3162475b54fa0f" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 33585 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:24:40.820307 28444 raft_consensus.cc:1275] T 037bfc9d92244fcb872b2ca382c21794 P 96cba770f3334f79af3162475b54fa0f [term 2 FOLLOWER]: Refusing update from remote peer 542ea9e2fcff4028b6766459e90bc5fc: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:24:40.822608 28619 consensus_queue.cc:1048] T 037bfc9d92244fcb872b2ca382c21794 P 542ea9e2fcff4028b6766459e90bc5fc [LEADER]: Connected to new peer: Peer: permanent_uuid: "96cba770f3334f79af3162475b54fa0f" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 33585 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:24:40.833458 28520 raft_consensus.cc:3060] T 037bfc9d92244fcb872b2ca382c21794 P 2173e3fd92f34ad6ad8d56d2e6b05299 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:24:40.844475 28520 raft_consensus.cc:1275] T 037bfc9d92244fcb872b2ca382c21794 P 2173e3fd92f34ad6ad8d56d2e6b05299 [term 2 FOLLOWER]: Refusing update from remote peer 542ea9e2fcff4028b6766459e90bc5fc: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:24:40.846328 28619 consensus_queue.cc:1048] T 037bfc9d92244fcb872b2ca382c21794 P 542ea9e2fcff4028b6766459e90bc5fc [LEADER]: Connected to new peer: Peer: permanent_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 37597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:24:40.862352 28371 raft_consensus.cc:1275] T a6cf33518cd34784a1b85843d90badb5 P 542ea9e2fcff4028b6766459e90bc5fc [term 2 FOLLOWER]: Refusing update from remote peer 96cba770f3334f79af3162475b54fa0f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:24:40.863677 28618 consensus_queue.cc:1048] T a6cf33518cd34784a1b85843d90badb5 P 96cba770f3334f79af3162475b54fa0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "542ea9e2fcff4028b6766459e90bc5fc" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38409 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:24:40.875710 28520 raft_consensus.cc:1275] T a6cf33518cd34784a1b85843d90badb5 P 2173e3fd92f34ad6ad8d56d2e6b05299 [term 2 FOLLOWER]: Refusing update from remote peer 96cba770f3334f79af3162475b54fa0f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:24:40.877393 28668 consensus_queue.cc:1048] T a6cf33518cd34784a1b85843d90badb5 P 96cba770f3334f79af3162475b54fa0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 37597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:24:43.032681 28270 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39122:
name: "soft_deleted_table"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q""\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 13:24:43.036362 28270 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table soft_deleted_table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 13:24:43.102411 28358 tablet_service.cc:1511] Processing CreateTablet for tablet 94c1c4dff8e8404dbb00c79a7b7c69c0 (DEFAULT_TABLE table=soft_deleted_table [id=67a3bcc0b4a0443b913cf8eda79d4082]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:24:43.103772 28436 tablet_service.cc:1511] Processing CreateTablet for tablet 94c1c4dff8e8404dbb00c79a7b7c69c0 (DEFAULT_TABLE table=soft_deleted_table [id=67a3bcc0b4a0443b913cf8eda79d4082]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:24:43.105369 28436 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 94c1c4dff8e8404dbb00c79a7b7c69c0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:24:43.104231 28358 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 94c1c4dff8e8404dbb00c79a7b7c69c0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:24:43.120632 28505 tablet_service.cc:1511] Processing CreateTablet for tablet 94c1c4dff8e8404dbb00c79a7b7c69c0 (DEFAULT_TABLE table=soft_deleted_table [id=67a3bcc0b4a0443b913cf8eda79d4082]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:24:43.122197 28505 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 94c1c4dff8e8404dbb00c79a7b7c69c0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:24:43.124061 28359 tablet_service.cc:1511] Processing CreateTablet for tablet e4c7e579c94a4f5fa195def741ee4180 (DEFAULT_TABLE table=soft_deleted_table [id=67a3bcc0b4a0443b913cf8eda79d4082]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:24:43.125506 28359 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e4c7e579c94a4f5fa195def741ee4180. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:24:43.128921 28433 tablet_service.cc:1511] Processing CreateTablet for tablet e4c7e579c94a4f5fa195def741ee4180 (DEFAULT_TABLE table=soft_deleted_table [id=67a3bcc0b4a0443b913cf8eda79d4082]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:24:43.130422 28433 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e4c7e579c94a4f5fa195def741ee4180. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:24:43.130913 28510 tablet_service.cc:1511] Processing CreateTablet for tablet e4c7e579c94a4f5fa195def741ee4180 (DEFAULT_TABLE table=soft_deleted_table [id=67a3bcc0b4a0443b913cf8eda79d4082]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:24:43.132424 28510 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e4c7e579c94a4f5fa195def741ee4180. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:24:43.133316 28357 tablet_service.cc:1511] Processing CreateTablet for tablet fbb3d76b1491485287e0dc48b24dbb46 (DEFAULT_TABLE table=soft_deleted_table [id=67a3bcc0b4a0443b913cf8eda79d4082]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:24:43.134673 28357 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fbb3d76b1491485287e0dc48b24dbb46. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:24:43.137799 28432 tablet_service.cc:1511] Processing CreateTablet for tablet fbb3d76b1491485287e0dc48b24dbb46 (DEFAULT_TABLE table=soft_deleted_table [id=67a3bcc0b4a0443b913cf8eda79d4082]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:24:43.139310 28432 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fbb3d76b1491485287e0dc48b24dbb46. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:24:43.137878 28509 tablet_service.cc:1511] Processing CreateTablet for tablet fbb3d76b1491485287e0dc48b24dbb46 (DEFAULT_TABLE table=soft_deleted_table [id=67a3bcc0b4a0443b913cf8eda79d4082]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:24:43.146617 28509 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fbb3d76b1491485287e0dc48b24dbb46. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:24:43.150189 28356 tablet_service.cc:1511] Processing CreateTablet for tablet d489ac98a4654a7cb46771880f89b89b (DEFAULT_TABLE table=soft_deleted_table [id=67a3bcc0b4a0443b913cf8eda79d4082]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:24:43.151700 28356 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d489ac98a4654a7cb46771880f89b89b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:24:43.161273 28439 tablet_service.cc:1511] Processing CreateTablet for tablet d489ac98a4654a7cb46771880f89b89b (DEFAULT_TABLE table=soft_deleted_table [id=67a3bcc0b4a0443b913cf8eda79d4082]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:24:43.162773 28439 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d489ac98a4654a7cb46771880f89b89b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:24:43.164404 28511 tablet_service.cc:1511] Processing CreateTablet for tablet d489ac98a4654a7cb46771880f89b89b (DEFAULT_TABLE table=soft_deleted_table [id=67a3bcc0b4a0443b913cf8eda79d4082]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:24:43.166121 28511 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d489ac98a4654a7cb46771880f89b89b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:24:43.182647 28363 tablet_service.cc:1511] Processing CreateTablet for tablet 2b05204beeff49309ed4e0dd43b4be7f (DEFAULT_TABLE table=soft_deleted_table [id=67a3bcc0b4a0443b913cf8eda79d4082]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:24:43.184149 28363 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2b05204beeff49309ed4e0dd43b4be7f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:24:43.194346 28507 tablet_service.cc:1511] Processing CreateTablet for tablet 2b05204beeff49309ed4e0dd43b4be7f (DEFAULT_TABLE table=soft_deleted_table [id=67a3bcc0b4a0443b913cf8eda79d4082]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:24:43.195914 28507 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2b05204beeff49309ed4e0dd43b4be7f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:24:43.198336 28438 tablet_service.cc:1511] Processing CreateTablet for tablet 6efeed092ec840f2a7748c4a613dd6a7 (DEFAULT_TABLE table=soft_deleted_table [id=67a3bcc0b4a0443b913cf8eda79d4082]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:24:43.199805 28438 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6efeed092ec840f2a7748c4a613dd6a7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:24:43.199072 28360 tablet_service.cc:1511] Processing CreateTablet for tablet 6efeed092ec840f2a7748c4a613dd6a7 (DEFAULT_TABLE table=soft_deleted_table [id=67a3bcc0b4a0443b913cf8eda79d4082]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:24:43.205029 28360 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6efeed092ec840f2a7748c4a613dd6a7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:24:43.215691 28512 tablet_service.cc:1511] Processing CreateTablet for tablet 6efeed092ec840f2a7748c4a613dd6a7 (DEFAULT_TABLE table=soft_deleted_table [id=67a3bcc0b4a0443b913cf8eda79d4082]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:24:43.217193 28512 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6efeed092ec840f2a7748c4a613dd6a7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:24:43.219103 28364 tablet_service.cc:1511] Processing CreateTablet for tablet d5de0c782ce844ab8dbec004c32abafa (DEFAULT_TABLE table=soft_deleted_table [id=67a3bcc0b4a0443b913cf8eda79d4082]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:24:43.220587 28364 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d5de0c782ce844ab8dbec004c32abafa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:24:43.220343 28437 tablet_service.cc:1511] Processing CreateTablet for tablet d5de0c782ce844ab8dbec004c32abafa (DEFAULT_TABLE table=soft_deleted_table [id=67a3bcc0b4a0443b913cf8eda79d4082]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:24:43.221760 28437 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d5de0c782ce844ab8dbec004c32abafa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:24:43.233796 28508 tablet_service.cc:1511] Processing CreateTablet for tablet d5de0c782ce844ab8dbec004c32abafa (DEFAULT_TABLE table=soft_deleted_table [id=67a3bcc0b4a0443b913cf8eda79d4082]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:24:43.235327 28508 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d5de0c782ce844ab8dbec004c32abafa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:24:43.240759 28362 tablet_service.cc:1511] Processing CreateTablet for tablet 07bb4101b0c24ee6b460e066cf78eb53 (DEFAULT_TABLE table=soft_deleted_table [id=67a3bcc0b4a0443b913cf8eda79d4082]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:24:43.242268 28362 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 07bb4101b0c24ee6b460e066cf78eb53. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:24:43.244065 28435 tablet_service.cc:1511] Processing CreateTablet for tablet 07bb4101b0c24ee6b460e066cf78eb53 (DEFAULT_TABLE table=soft_deleted_table [id=67a3bcc0b4a0443b913cf8eda79d4082]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:24:43.245608 28435 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 07bb4101b0c24ee6b460e066cf78eb53. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:24:43.252619 28361 tablet_service.cc:1511] Processing CreateTablet for tablet cbb4f686d0884c178ea6b01616da20e8 (DEFAULT_TABLE table=soft_deleted_table [id=67a3bcc0b4a0443b913cf8eda79d4082]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:24:43.254117 28361 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cbb4f686d0884c178ea6b01616da20e8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:24:43.255872 28436 tablet_service.cc:1511] Processing CreateTablet for tablet cbb4f686d0884c178ea6b01616da20e8 (DEFAULT_TABLE table=soft_deleted_table [id=67a3bcc0b4a0443b913cf8eda79d4082]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:24:43.257414 28436 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cbb4f686d0884c178ea6b01616da20e8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:24:43.259506 28432 tablet_service.cc:1511] Processing CreateTablet for tablet 2b05204beeff49309ed4e0dd43b4be7f (DEFAULT_TABLE table=soft_deleted_table [id=67a3bcc0b4a0443b913cf8eda79d4082]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:24:43.260998 28432 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2b05204beeff49309ed4e0dd43b4be7f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:24:43.271903 28688 tablet_bootstrap.cc:492] T 94c1c4dff8e8404dbb00c79a7b7c69c0 P 96cba770f3334f79af3162475b54fa0f: Bootstrap starting.
I20260709 13:24:43.282033 28688 tablet_bootstrap.cc:654] T 94c1c4dff8e8404dbb00c79a7b7c69c0 P 96cba770f3334f79af3162475b54fa0f: Neither blocks nor log segments found. Creating new log.
I20260709 13:24:43.274133 28511 tablet_service.cc:1511] Processing CreateTablet for tablet cbb4f686d0884c178ea6b01616da20e8 (DEFAULT_TABLE table=soft_deleted_table [id=67a3bcc0b4a0443b913cf8eda79d4082]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:24:43.290616 28511 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cbb4f686d0884c178ea6b01616da20e8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:24:43.322819 28689 tablet_bootstrap.cc:492] T fbb3d76b1491485287e0dc48b24dbb46 P 2173e3fd92f34ad6ad8d56d2e6b05299: Bootstrap starting.
I20260709 13:24:43.328187 28690 tablet_bootstrap.cc:492] T 94c1c4dff8e8404dbb00c79a7b7c69c0 P 542ea9e2fcff4028b6766459e90bc5fc: Bootstrap starting.
I20260709 13:24:43.343462 28689 tablet_bootstrap.cc:654] T fbb3d76b1491485287e0dc48b24dbb46 P 2173e3fd92f34ad6ad8d56d2e6b05299: Neither blocks nor log segments found. Creating new log.
I20260709 13:24:43.296236 28510 tablet_service.cc:1511] Processing CreateTablet for tablet 07bb4101b0c24ee6b460e066cf78eb53 (DEFAULT_TABLE table=soft_deleted_table [id=67a3bcc0b4a0443b913cf8eda79d4082]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:24:43.352325 28510 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 07bb4101b0c24ee6b460e066cf78eb53. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:24:43.355722 28690 tablet_bootstrap.cc:654] T 94c1c4dff8e8404dbb00c79a7b7c69c0 P 542ea9e2fcff4028b6766459e90bc5fc: Neither blocks nor log segments found. Creating new log.
I20260709 13:24:43.366286 28690 tablet_bootstrap.cc:492] T 94c1c4dff8e8404dbb00c79a7b7c69c0 P 542ea9e2fcff4028b6766459e90bc5fc: No bootstrap required, opened a new log
I20260709 13:24:43.366968 28690 ts_tablet_manager.cc:1403] T 94c1c4dff8e8404dbb00c79a7b7c69c0 P 542ea9e2fcff4028b6766459e90bc5fc: Time spent bootstrapping tablet: real 0.039s	user 0.014s	sys 0.000s
I20260709 13:24:43.367296 28689 tablet_bootstrap.cc:492] T fbb3d76b1491485287e0dc48b24dbb46 P 2173e3fd92f34ad6ad8d56d2e6b05299: No bootstrap required, opened a new log
I20260709 13:24:43.368054 28689 ts_tablet_manager.cc:1403] T fbb3d76b1491485287e0dc48b24dbb46 P 2173e3fd92f34ad6ad8d56d2e6b05299: Time spent bootstrapping tablet: real 0.046s	user 0.005s	sys 0.009s
I20260709 13:24:43.370647 28690 raft_consensus.cc:359] T 94c1c4dff8e8404dbb00c79a7b7c69c0 P 542ea9e2fcff4028b6766459e90bc5fc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "542ea9e2fcff4028b6766459e90bc5fc" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38409 } } peers { permanent_uuid: "96cba770f3334f79af3162475b54fa0f" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 33585 } } peers { permanent_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 37597 } }
I20260709 13:24:43.371753 28690 raft_consensus.cc:385] T 94c1c4dff8e8404dbb00c79a7b7c69c0 P 542ea9e2fcff4028b6766459e90bc5fc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:24:43.372262 28690 raft_consensus.cc:740] T 94c1c4dff8e8404dbb00c79a7b7c69c0 P 542ea9e2fcff4028b6766459e90bc5fc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 542ea9e2fcff4028b6766459e90bc5fc, State: Initialized, Role: FOLLOWER
I20260709 13:24:43.371922 28689 raft_consensus.cc:359] T fbb3d76b1491485287e0dc48b24dbb46 P 2173e3fd92f34ad6ad8d56d2e6b05299 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "542ea9e2fcff4028b6766459e90bc5fc" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38409 } } peers { permanent_uuid: "96cba770f3334f79af3162475b54fa0f" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 33585 } } peers { permanent_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 37597 } }
I20260709 13:24:43.373072 28689 raft_consensus.cc:385] T fbb3d76b1491485287e0dc48b24dbb46 P 2173e3fd92f34ad6ad8d56d2e6b05299 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:24:43.373515 28689 raft_consensus.cc:740] T fbb3d76b1491485287e0dc48b24dbb46 P 2173e3fd92f34ad6ad8d56d2e6b05299 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2173e3fd92f34ad6ad8d56d2e6b05299, State: Initialized, Role: FOLLOWER
I20260709 13:24:43.373296 28690 consensus_queue.cc:260] T 94c1c4dff8e8404dbb00c79a7b7c69c0 P 542ea9e2fcff4028b6766459e90bc5fc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "542ea9e2fcff4028b6766459e90bc5fc" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38409 } } peers { permanent_uuid: "96cba770f3334f79af3162475b54fa0f" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 33585 } } peers { permanent_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 37597 } }
I20260709 13:24:43.374850 28689 consensus_queue.cc:260] T fbb3d76b1491485287e0dc48b24dbb46 P 2173e3fd92f34ad6ad8d56d2e6b05299 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "542ea9e2fcff4028b6766459e90bc5fc" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38409 } } peers { permanent_uuid: "96cba770f3334f79af3162475b54fa0f" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 33585 } } peers { permanent_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 37597 } }
I20260709 13:24:43.379436 28690 ts_tablet_manager.cc:1434] T 94c1c4dff8e8404dbb00c79a7b7c69c0 P 542ea9e2fcff4028b6766459e90bc5fc: Time spent starting tablet: real 0.012s	user 0.009s	sys 0.000s
I20260709 13:24:43.380873 28690 tablet_bootstrap.cc:492] T fbb3d76b1491485287e0dc48b24dbb46 P 542ea9e2fcff4028b6766459e90bc5fc: Bootstrap starting.
I20260709 13:24:43.382385 28688 tablet_bootstrap.cc:492] T 94c1c4dff8e8404dbb00c79a7b7c69c0 P 96cba770f3334f79af3162475b54fa0f: No bootstrap required, opened a new log
I20260709 13:24:43.383133 28688 ts_tablet_manager.cc:1403] T 94c1c4dff8e8404dbb00c79a7b7c69c0 P 96cba770f3334f79af3162475b54fa0f: Time spent bootstrapping tablet: real 0.112s	user 0.011s	sys 0.003s
I20260709 13:24:43.387961 28689 ts_tablet_manager.cc:1434] T fbb3d76b1491485287e0dc48b24dbb46 P 2173e3fd92f34ad6ad8d56d2e6b05299: Time spent starting tablet: real 0.019s	user 0.008s	sys 0.000s
I20260709 13:24:43.394107 28690 tablet_bootstrap.cc:654] T fbb3d76b1491485287e0dc48b24dbb46 P 542ea9e2fcff4028b6766459e90bc5fc: Neither blocks nor log segments found. Creating new log.
I20260709 13:24:43.402611 28689 tablet_bootstrap.cc:492] T e4c7e579c94a4f5fa195def741ee4180 P 2173e3fd92f34ad6ad8d56d2e6b05299: Bootstrap starting.
I20260709 13:24:43.407203 28688 raft_consensus.cc:359] T 94c1c4dff8e8404dbb00c79a7b7c69c0 P 96cba770f3334f79af3162475b54fa0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "542ea9e2fcff4028b6766459e90bc5fc" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38409 } } peers { permanent_uuid: "96cba770f3334f79af3162475b54fa0f" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 33585 } } peers { permanent_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 37597 } }
I20260709 13:24:43.408087 28688 raft_consensus.cc:385] T 94c1c4dff8e8404dbb00c79a7b7c69c0 P 96cba770f3334f79af3162475b54fa0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:24:43.408402 28688 raft_consensus.cc:740] T 94c1c4dff8e8404dbb00c79a7b7c69c0 P 96cba770f3334f79af3162475b54fa0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 96cba770f3334f79af3162475b54fa0f, State: Initialized, Role: FOLLOWER
I20260709 13:24:43.412947 28688 consensus_queue.cc:260] T 94c1c4dff8e8404dbb00c79a7b7c69c0 P 96cba770f3334f79af3162475b54fa0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "542ea9e2fcff4028b6766459e90bc5fc" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38409 } } peers { permanent_uuid: "96cba770f3334f79af3162475b54fa0f" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 33585 } } peers { permanent_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 37597 } }
I20260709 13:24:43.415364 28689 tablet_bootstrap.cc:654] T e4c7e579c94a4f5fa195def741ee4180 P 2173e3fd92f34ad6ad8d56d2e6b05299: Neither blocks nor log segments found. Creating new log.
I20260709 13:24:43.420405 28690 tablet_bootstrap.cc:492] T fbb3d76b1491485287e0dc48b24dbb46 P 542ea9e2fcff4028b6766459e90bc5fc: No bootstrap required, opened a new log
I20260709 13:24:43.420832 28688 ts_tablet_manager.cc:1434] T 94c1c4dff8e8404dbb00c79a7b7c69c0 P 96cba770f3334f79af3162475b54fa0f: Time spent starting tablet: real 0.037s	user 0.003s	sys 0.005s
I20260709 13:24:43.421094 28690 ts_tablet_manager.cc:1403] T fbb3d76b1491485287e0dc48b24dbb46 P 542ea9e2fcff4028b6766459e90bc5fc: Time spent bootstrapping tablet: real 0.040s	user 0.020s	sys 0.000s
I20260709 13:24:43.421979 28688 tablet_bootstrap.cc:492] T fbb3d76b1491485287e0dc48b24dbb46 P 96cba770f3334f79af3162475b54fa0f: Bootstrap starting.
W20260709 13:24:43.429288 28320 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:24:43.432201 28690 raft_consensus.cc:359] T fbb3d76b1491485287e0dc48b24dbb46 P 542ea9e2fcff4028b6766459e90bc5fc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "542ea9e2fcff4028b6766459e90bc5fc" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38409 } } peers { permanent_uuid: "96cba770f3334f79af3162475b54fa0f" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 33585 } } peers { permanent_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 37597 } }
I20260709 13:24:43.433916 28690 raft_consensus.cc:385] T fbb3d76b1491485287e0dc48b24dbb46 P 542ea9e2fcff4028b6766459e90bc5fc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:24:43.434302 28690 raft_consensus.cc:740] T fbb3d76b1491485287e0dc48b24dbb46 P 542ea9e2fcff4028b6766459e90bc5fc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 542ea9e2fcff4028b6766459e90bc5fc, State: Initialized, Role: FOLLOWER
I20260709 13:24:43.435575 28688 tablet_bootstrap.cc:654] T fbb3d76b1491485287e0dc48b24dbb46 P 96cba770f3334f79af3162475b54fa0f: Neither blocks nor log segments found. Creating new log.
I20260709 13:24:43.435156 28690 consensus_queue.cc:260] T fbb3d76b1491485287e0dc48b24dbb46 P 542ea9e2fcff4028b6766459e90bc5fc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "542ea9e2fcff4028b6766459e90bc5fc" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38409 } } peers { permanent_uuid: "96cba770f3334f79af3162475b54fa0f" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 33585 } } peers { permanent_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 37597 } }
I20260709 13:24:43.437280 28690 ts_tablet_manager.cc:1434] T fbb3d76b1491485287e0dc48b24dbb46 P 542ea9e2fcff4028b6766459e90bc5fc: Time spent starting tablet: real 0.016s	user 0.006s	sys 0.000s
I20260709 13:24:43.438443 28690 tablet_bootstrap.cc:492] T e4c7e579c94a4f5fa195def741ee4180 P 542ea9e2fcff4028b6766459e90bc5fc: Bootstrap starting.
I20260709 13:24:43.449152 28690 tablet_bootstrap.cc:654] T e4c7e579c94a4f5fa195def741ee4180 P 542ea9e2fcff4028b6766459e90bc5fc: Neither blocks nor log segments found. Creating new log.
I20260709 13:24:43.452422 28688 tablet_bootstrap.cc:492] T fbb3d76b1491485287e0dc48b24dbb46 P 96cba770f3334f79af3162475b54fa0f: No bootstrap required, opened a new log
I20260709 13:24:43.452953 28688 ts_tablet_manager.cc:1403] T fbb3d76b1491485287e0dc48b24dbb46 P 96cba770f3334f79af3162475b54fa0f: Time spent bootstrapping tablet: real 0.031s	user 0.016s	sys 0.002s
I20260709 13:24:43.453609 28689 tablet_bootstrap.cc:492] T e4c7e579c94a4f5fa195def741ee4180 P 2173e3fd92f34ad6ad8d56d2e6b05299: No bootstrap required, opened a new log
I20260709 13:24:43.454267 28689 ts_tablet_manager.cc:1403] T e4c7e579c94a4f5fa195def741ee4180 P 2173e3fd92f34ad6ad8d56d2e6b05299: Time spent bootstrapping tablet: real 0.052s	user 0.011s	sys 0.005s
I20260709 13:24:43.456210 28688 raft_consensus.cc:359] T fbb3d76b1491485287e0dc48b24dbb46 P 96cba770f3334f79af3162475b54fa0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "542ea9e2fcff4028b6766459e90bc5fc" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38409 } } peers { permanent_uuid: "96cba770f3334f79af3162475b54fa0f" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 33585 } } peers { permanent_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 37597 } }
I20260709 13:24:43.457389 28688 raft_consensus.cc:385] T fbb3d76b1491485287e0dc48b24dbb46 P 96cba770f3334f79af3162475b54fa0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:24:43.457772 28688 raft_consensus.cc:740] T fbb3d76b1491485287e0dc48b24dbb46 P 96cba770f3334f79af3162475b54fa0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 96cba770f3334f79af3162475b54fa0f, State: Initialized, Role: FOLLOWER
I20260709 13:24:43.457460 28689 raft_consensus.cc:359] T e4c7e579c94a4f5fa195def741ee4180 P 2173e3fd92f34ad6ad8d56d2e6b05299 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "542ea9e2fcff4028b6766459e90bc5fc" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38409 } } peers { permanent_uuid: "96cba770f3334f79af3162475b54fa0f" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 33585 } } peers { permanent_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 37597 } }
I20260709 13:24:43.458349 28689 raft_consensus.cc:385] T e4c7e579c94a4f5fa195def741ee4180 P 2173e3fd92f34ad6ad8d56d2e6b05299 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:24:43.458743 28689 raft_consensus.cc:740] T e4c7e579c94a4f5fa195def741ee4180 P 2173e3fd92f34ad6ad8d56d2e6b05299 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2173e3fd92f34ad6ad8d56d2e6b05299, State: Initialized, Role: FOLLOWER
I20260709 13:24:43.458698 28688 consensus_queue.cc:260] T fbb3d76b1491485287e0dc48b24dbb46 P 96cba770f3334f79af3162475b54fa0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "542ea9e2fcff4028b6766459e90bc5fc" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38409 } } peers { permanent_uuid: "96cba770f3334f79af3162475b54fa0f" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 33585 } } peers { permanent_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 37597 } }
I20260709 13:24:43.459957 28689 consensus_queue.cc:260] T e4c7e579c94a4f5fa195def741ee4180 P 2173e3fd92f34ad6ad8d56d2e6b05299 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "542ea9e2fcff4028b6766459e90bc5fc" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38409 } } peers { permanent_uuid: "96cba770f3334f79af3162475b54fa0f" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 33585 } } peers { permanent_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 37597 } }
I20260709 13:24:43.464672 28688 ts_tablet_manager.cc:1434] T fbb3d76b1491485287e0dc48b24dbb46 P 96cba770f3334f79af3162475b54fa0f: Time spent starting tablet: real 0.011s	user 0.007s	sys 0.000s
I20260709 13:24:43.465916 28688 tablet_bootstrap.cc:492] T d489ac98a4654a7cb46771880f89b89b P 96cba770f3334f79af3162475b54fa0f: Bootstrap starting.
I20260709 13:24:43.472909 28688 tablet_bootstrap.cc:654] T d489ac98a4654a7cb46771880f89b89b P 96cba770f3334f79af3162475b54fa0f: Neither blocks nor log segments found. Creating new log.
I20260709 13:24:43.476536 28690 tablet_bootstrap.cc:492] T e4c7e579c94a4f5fa195def741ee4180 P 542ea9e2fcff4028b6766459e90bc5fc: No bootstrap required, opened a new log
I20260709 13:24:43.477353 28690 ts_tablet_manager.cc:1403] T e4c7e579c94a4f5fa195def741ee4180 P 542ea9e2fcff4028b6766459e90bc5fc: Time spent bootstrapping tablet: real 0.039s	user 0.013s	sys 0.023s
I20260709 13:24:43.483009 28690 raft_consensus.cc:359] T e4c7e579c94a4f5fa195def741ee4180 P 542ea9e2fcff4028b6766459e90bc5fc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "542ea9e2fcff4028b6766459e90bc5fc" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38409 } } peers { permanent_uuid: "96cba770f3334f79af3162475b54fa0f" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 33585 } } peers { permanent_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 37597 } }
I20260709 13:24:43.484066 28690 raft_consensus.cc:385] T e4c7e579c94a4f5fa195def741ee4180 P 542ea9e2fcff4028b6766459e90bc5fc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:24:43.484501 28690 raft_consensus.cc:740] T e4c7e579c94a4f5fa195def741ee4180 P 542ea9e2fcff4028b6766459e90bc5fc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 542ea9e2fcff4028b6766459e90bc5fc, State: Initialized, Role: FOLLOWER
I20260709 13:24:43.485607 28690 consensus_queue.cc:260] T e4c7e579c94a4f5fa195def741ee4180 P 542ea9e2fcff4028b6766459e90bc5fc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "542ea9e2fcff4028b6766459e90bc5fc" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38409 } } peers { permanent_uuid: "96cba770f3334f79af3162475b54fa0f" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 33585 } } peers { permanent_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 37597 } }
I20260709 13:24:43.488729 28690 ts_tablet_manager.cc:1434] T e4c7e579c94a4f5fa195def741ee4180 P 542ea9e2fcff4028b6766459e90bc5fc: Time spent starting tablet: real 0.011s	user 0.004s	sys 0.004s
I20260709 13:24:43.491824 28690 tablet_bootstrap.cc:492] T 2b05204beeff49309ed4e0dd43b4be7f P 542ea9e2fcff4028b6766459e90bc5fc: Bootstrap starting.
I20260709 13:24:43.495044 28689 ts_tablet_manager.cc:1434] T e4c7e579c94a4f5fa195def741ee4180 P 2173e3fd92f34ad6ad8d56d2e6b05299: Time spent starting tablet: real 0.040s	user 0.006s	sys 0.000s
I20260709 13:24:43.498978 28688 tablet_bootstrap.cc:492] T d489ac98a4654a7cb46771880f89b89b P 96cba770f3334f79af3162475b54fa0f: No bootstrap required, opened a new log
I20260709 13:24:43.499745 28688 ts_tablet_manager.cc:1403] T d489ac98a4654a7cb46771880f89b89b P 96cba770f3334f79af3162475b54fa0f: Time spent bootstrapping tablet: real 0.034s	user 0.007s	sys 0.018s
I20260709 13:24:43.503212 28688 raft_consensus.cc:359] T d489ac98a4654a7cb46771880f89b89b P 96cba770f3334f79af3162475b54fa0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "542ea9e2fcff4028b6766459e90bc5fc" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38409 } } peers { permanent_uuid: "96cba770f3334f79af3162475b54fa0f" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 33585 } } peers { permanent_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 37597 } }
I20260709 13:24:43.504050 28688 raft_consensus.cc:385] T d489ac98a4654a7cb46771880f89b89b P 96cba770f3334f79af3162475b54fa0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:24:43.504413 28688 raft_consensus.cc:740] T d489ac98a4654a7cb46771880f89b89b P 96cba770f3334f79af3162475b54fa0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 96cba770f3334f79af3162475b54fa0f, State: Initialized, Role: FOLLOWER
I20260709 13:24:43.506235 28668 raft_consensus.cc:493] T fbb3d76b1491485287e0dc48b24dbb46 P 96cba770f3334f79af3162475b54fa0f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:24:43.506834 28668 raft_consensus.cc:515] T fbb3d76b1491485287e0dc48b24dbb46 P 96cba770f3334f79af3162475b54fa0f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "542ea9e2fcff4028b6766459e90bc5fc" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38409 } } peers { permanent_uuid: "96cba770f3334f79af3162475b54fa0f" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 33585 } } peers { permanent_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 37597 } }
I20260709 13:24:43.507041 28690 tablet_bootstrap.cc:654] T 2b05204beeff49309ed4e0dd43b4be7f P 542ea9e2fcff4028b6766459e90bc5fc: Neither blocks nor log segments found. Creating new log.
I20260709 13:24:43.507055 28689 tablet_bootstrap.cc:492] T d489ac98a4654a7cb46771880f89b89b P 2173e3fd92f34ad6ad8d56d2e6b05299: Bootstrap starting.
I20260709 13:24:43.511555 28372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb3d76b1491485287e0dc48b24dbb46" candidate_uuid: "96cba770f3334f79af3162475b54fa0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "542ea9e2fcff4028b6766459e90bc5fc" is_pre_election: true
I20260709 13:24:43.512384 28372 raft_consensus.cc:2468] T fbb3d76b1491485287e0dc48b24dbb46 P 542ea9e2fcff4028b6766459e90bc5fc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96cba770f3334f79af3162475b54fa0f in term 0.
I20260709 13:24:43.513710 28520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb3d76b1491485287e0dc48b24dbb46" candidate_uuid: "96cba770f3334f79af3162475b54fa0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" is_pre_election: true
I20260709 13:24:43.514443 28520 raft_consensus.cc:2468] T fbb3d76b1491485287e0dc48b24dbb46 P 2173e3fd92f34ad6ad8d56d2e6b05299 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96cba770f3334f79af3162475b54fa0f in term 0.
I20260709 13:24:43.515451 28689 tablet_bootstrap.cc:654] T d489ac98a4654a7cb46771880f89b89b P 2173e3fd92f34ad6ad8d56d2e6b05299: Neither blocks nor log segments found. Creating new log.
I20260709 13:24:43.515969 28413 leader_election.cc:304] T fbb3d76b1491485287e0dc48b24dbb46 P 96cba770f3334f79af3162475b54fa0f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 542ea9e2fcff4028b6766459e90bc5fc, 96cba770f3334f79af3162475b54fa0f; no voters: 
I20260709 13:24:43.518147 28700 raft_consensus.cc:2804] T fbb3d76b1491485287e0dc48b24dbb46 P 96cba770f3334f79af3162475b54fa0f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:24:43.518319 28668 leader_election.cc:290] T fbb3d76b1491485287e0dc48b24dbb46 P 96cba770f3334f79af3162475b54fa0f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 542ea9e2fcff4028b6766459e90bc5fc (127.14.177.129:38409), 2173e3fd92f34ad6ad8d56d2e6b05299 (127.14.177.131:37597)
I20260709 13:24:43.505415 28688 consensus_queue.cc:260] T d489ac98a4654a7cb46771880f89b89b P 96cba770f3334f79af3162475b54fa0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "542ea9e2fcff4028b6766459e90bc5fc" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38409 } } peers { permanent_uuid: "96cba770f3334f79af3162475b54fa0f" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 33585 } } peers { permanent_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 37597 } }
I20260709 13:24:43.523056 28688 ts_tablet_manager.cc:1434] T d489ac98a4654a7cb46771880f89b89b P 96cba770f3334f79af3162475b54fa0f: Time spent starting tablet: real 0.023s	user 0.007s	sys 0.000s
I20260709 13:24:43.520298 28700 raft_consensus.cc:493] T fbb3d76b1491485287e0dc48b24dbb46 P 96cba770f3334f79af3162475b54fa0f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:24:43.524828 28700 raft_consensus.cc:3060] T fbb3d76b1491485287e0dc48b24dbb46 P 96cba770f3334f79af3162475b54fa0f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:24:43.526082 28688 tablet_bootstrap.cc:492] T 6efeed092ec840f2a7748c4a613dd6a7 P 96cba770f3334f79af3162475b54fa0f: Bootstrap starting.
I20260709 13:24:43.529040 28690 tablet_bootstrap.cc:492] T 2b05204beeff49309ed4e0dd43b4be7f P 542ea9e2fcff4028b6766459e90bc5fc: No bootstrap required, opened a new log
I20260709 13:24:43.534042 28690 ts_tablet_manager.cc:1403] T 2b05204beeff49309ed4e0dd43b4be7f P 542ea9e2fcff4028b6766459e90bc5fc: Time spent bootstrapping tablet: real 0.042s	user 0.011s	sys 0.010s
I20260709 13:24:43.534508 28700 raft_consensus.cc:515] T fbb3d76b1491485287e0dc48b24dbb46 P 96cba770f3334f79af3162475b54fa0f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "542ea9e2fcff4028b6766459e90bc5fc" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38409 } } peers { permanent_uuid: "96cba770f3334f79af3162475b54fa0f" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 33585 } } peers { permanent_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 37597 } }
I20260709 13:24:43.537952 28690 raft_consensus.cc:359] T 2b05204beeff49309ed4e0dd43b4be7f P 542ea9e2fcff4028b6766459e90bc5fc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "542ea9e2fcff4028b6766459e90bc5fc" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38409 } } peers { permanent_uuid: "96cba770f3334f79af3162475b54fa0f" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 33585 } } peers { permanent_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 37597 } }
I20260709 13:24:43.538771 28690 raft_consensus.cc:385] T 2b05204beeff49309ed4e0dd43b4be7f P 542ea9e2fcff4028b6766459e90bc5fc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:24:43.539086 28690 raft_consensus.cc:740] T 2b05204beeff49309ed4e0dd43b4be7f P 542ea9e2fcff4028b6766459e90bc5fc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 542ea9e2fcff4028b6766459e90bc5fc, State: Initialized, Role: FOLLOWER
I20260709 13:24:43.539117 28372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb3d76b1491485287e0dc48b24dbb46" candidate_uuid: "96cba770f3334f79af3162475b54fa0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "542ea9e2fcff4028b6766459e90bc5fc"
I20260709 13:24:43.539991 28372 raft_consensus.cc:3060] T fbb3d76b1491485287e0dc48b24dbb46 P 542ea9e2fcff4028b6766459e90bc5fc [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:24:43.539937 28690 consensus_queue.cc:260] T 2b05204beeff49309ed4e0dd43b4be7f P 542ea9e2fcff4028b6766459e90bc5fc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "542ea9e2fcff4028b6766459e90bc5fc" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38409 } } peers { permanent_uuid: "96cba770f3334f79af3162475b54fa0f" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 33585 } } peers { permanent_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 37597 } }
I20260709 13:24:43.543087 28690 ts_tablet_manager.cc:1434] T 2b05204beeff49309ed4e0dd43b4be7f P 542ea9e2fcff4028b6766459e90bc5fc: Time spent starting tablet: real 0.009s	user 0.004s	sys 0.000s
I20260709 13:24:43.544518 28690 tablet_bootstrap.cc:492] T d489ac98a4654a7cb46771880f89b89b P 542ea9e2fcff4028b6766459e90bc5fc: Bootstrap starting.
I20260709 13:24:43.549252 28619 raft_consensus.cc:493] T 2b05204beeff49309ed4e0dd43b4be7f P 542ea9e2fcff4028b6766459e90bc5fc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:24:43.550174 28619 raft_consensus.cc:515] T 2b05204beeff49309ed4e0dd43b4be7f P 542ea9e2fcff4028b6766459e90bc5fc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "542ea9e2fcff4028b6766459e90bc5fc" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38409 } } peers { permanent_uuid: "96cba770f3334f79af3162475b54fa0f" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 33585 } } peers { permanent_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 37597 } }
I20260709 13:24:43.555521 28690 tablet_bootstrap.cc:654] T d489ac98a4654a7cb46771880f89b89b P 542ea9e2fcff4028b6766459e90bc5fc: Neither blocks nor log segments found. Creating new log.
I20260709 13:24:43.559142 28688 tablet_bootstrap.cc:654] T 6efeed092ec840f2a7748c4a613dd6a7 P 96cba770f3334f79af3162475b54fa0f: Neither blocks nor log segments found. Creating new log.
I20260709 13:24:43.559127 28619 leader_election.cc:290] T 2b05204beeff49309ed4e0dd43b4be7f P 542ea9e2fcff4028b6766459e90bc5fc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 96cba770f3334f79af3162475b54fa0f (127.14.177.130:33585), 2173e3fd92f34ad6ad8d56d2e6b05299 (127.14.177.131:37597)
I20260709 13:24:43.562060 28444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b05204beeff49309ed4e0dd43b4be7f" candidate_uuid: "542ea9e2fcff4028b6766459e90bc5fc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96cba770f3334f79af3162475b54fa0f" is_pre_election: true
I20260709 13:24:43.565064 28522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b05204beeff49309ed4e0dd43b4be7f" candidate_uuid: "542ea9e2fcff4028b6766459e90bc5fc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" is_pre_election: true
W20260709 13:24:43.566519 28340 leader_election.cc:343] T 2b05204beeff49309ed4e0dd43b4be7f P 542ea9e2fcff4028b6766459e90bc5fc [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 96cba770f3334f79af3162475b54fa0f (127.14.177.130:33585): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:24:43.569293 28340 leader_election.cc:343] T 2b05204beeff49309ed4e0dd43b4be7f P 542ea9e2fcff4028b6766459e90bc5fc [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2173e3fd92f34ad6ad8d56d2e6b05299 (127.14.177.131:37597): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:24:43.569788 28340 leader_election.cc:304] T 2b05204beeff49309ed4e0dd43b4be7f P 542ea9e2fcff4028b6766459e90bc5fc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 542ea9e2fcff4028b6766459e90bc5fc; no voters: 2173e3fd92f34ad6ad8d56d2e6b05299, 96cba770f3334f79af3162475b54fa0f
I20260709 13:24:43.574034 28688 tablet_bootstrap.cc:492] T 6efeed092ec840f2a7748c4a613dd6a7 P 96cba770f3334f79af3162475b54fa0f: No bootstrap required, opened a new log
I20260709 13:24:43.574679 28688 ts_tablet_manager.cc:1403] T 6efeed092ec840f2a7748c4a613dd6a7 P 96cba770f3334f79af3162475b54fa0f: Time spent bootstrapping tablet: real 0.049s	user 0.016s	sys 0.003s
I20260709 13:24:43.576510 28619 raft_consensus.cc:2749] T 2b05204beeff49309ed4e0dd43b4be7f P 542ea9e2fcff4028b6766459e90bc5fc [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:24:43.585306 28522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb3d76b1491485287e0dc48b24dbb46" candidate_uuid: "96cba770f3334f79af3162475b54fa0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299"
I20260709 13:24:43.586251 28522 raft_consensus.cc:3060] T fbb3d76b1491485287e0dc48b24dbb46 P 2173e3fd92f34ad6ad8d56d2e6b05299 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:24:43.587541 28619 raft_consensus.cc:493] T 94c1c4dff8e8404dbb00c79a7b7c69c0 P 542ea9e2fcff4028b6766459e90bc5fc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:24:43.578274 28688 raft_consensus.cc:359] T 6efeed092ec840f2a7748c4a613dd6a7 P 96cba770f3334f79af3162475b54fa0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "542ea9e2fcff4028b6766459e90bc5fc" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38409 } } peers { permanent_uuid: "96cba770f3334f79af3162475b54fa0f" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 33585 } } peers { permanent_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 37597 } }
I20260709 13:24:43.588078 28619 raft_consensus.cc:515] T 94c1c4dff8e8404dbb00c79a7b7c69c0 P 542ea9e2fcff4028b6766459e90bc5fc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "542ea9e2fcff4028b6766459e90bc5fc" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38409 } } peers { permanent_uuid: "96cba770f3334f79af3162475b54fa0f" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 33585 } } peers { permanent_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 37597 } }
I20260709 13:24:43.588440 28688 raft_consensus.cc:385] T 6efeed092ec840f2a7748c4a613dd6a7 P 96cba770f3334f79af3162475b54fa0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:24:43.588845 28688 raft_consensus.cc:740] T 6efeed092ec840f2a7748c4a613dd6a7 P 96cba770f3334f79af3162475b54fa0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 96cba770f3334f79af3162475b54fa0f, State: Initialized, Role: FOLLOWER
I20260709 13:24:43.589709 28688 consensus_queue.cc:260] T 6efeed092ec840f2a7748c4a613dd6a7 P 96cba770f3334f79af3162475b54fa0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "542ea9e2fcff4028b6766459e90bc5fc" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38409 } } peers { permanent_uuid: "96cba770f3334f79af3162475b54fa0f" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 33585 } } peers { permanent_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 37597 } }
I20260709 13:24:43.590217 28690 tablet_bootstrap.cc:492] T d489ac98a4654a7cb46771880f89b89b P 542ea9e2fcff4028b6766459e90bc5fc: No bootstrap required, opened a new log
I20260709 13:24:43.590310 28619 leader_election.cc:290] T 94c1c4dff8e8404dbb00c79a7b7c69c0 P 542ea9e2fcff4028b6766459e90bc5fc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 96cba770f3334f79af3162475b54fa0f (127.14.177.130:33585), 2173e3fd92f34ad6ad8d56d2e6b05299 (127.14.177.131:37597)
I20260709 13:24:43.591084 28690 ts_tablet_manager.cc:1403] T d489ac98a4654a7cb46771880f89b89b P 542ea9e2fcff4028b6766459e90bc5fc: Time spent bootstrapping tablet: real 0.047s	user 0.016s	sys 0.016s
I20260709 13:24:43.592196 28688 ts_tablet_manager.cc:1434] T 6efeed092ec840f2a7748c4a613dd6a7 P 96cba770f3334f79af3162475b54fa0f: Time spent starting tablet: real 0.017s	user 0.009s	sys 0.001s
I20260709 13:24:43.592679 28689 tablet_bootstrap.cc:492] T d489ac98a4654a7cb46771880f89b89b P 2173e3fd92f34ad6ad8d56d2e6b05299: No bootstrap required, opened a new log
I20260709 13:24:43.593242 28689 ts_tablet_manager.cc:1403] T d489ac98a4654a7cb46771880f89b89b P 2173e3fd92f34ad6ad8d56d2e6b05299: Time spent bootstrapping tablet: real 0.086s	user 0.010s	sys 0.008s
I20260709 13:24:43.593405 28688 tablet_bootstrap.cc:492] T e4c7e579c94a4f5fa195def741ee4180 P 96cba770f3334f79af3162475b54fa0f: Bootstrap starting.
I20260709 13:24:43.598944 28520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94c1c4dff8e8404dbb00c79a7b7c69c0" candidate_uuid: "542ea9e2fcff4028b6766459e90bc5fc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" is_pre_election: true
W20260709 13:24:43.601152 28340 leader_election.cc:343] T 94c1c4dff8e8404dbb00c79a7b7c69c0 P 542ea9e2fcff4028b6766459e90bc5fc [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2173e3fd92f34ad6ad8d56d2e6b05299 (127.14.177.131:37597): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:24:43.601863 28700 leader_election.cc:290] T fbb3d76b1491485287e0dc48b24dbb46 P 96cba770f3334f79af3162475b54fa0f [CANDIDATE]: Term 1 election: Requested vote from peers 542ea9e2fcff4028b6766459e90bc5fc (127.14.177.129:38409), 2173e3fd92f34ad6ad8d56d2e6b05299 (127.14.177.131:37597)
I20260709 13:24:43.602854 28372 raft_consensus.cc:2468] T fbb3d76b1491485287e0dc48b24dbb46 P 542ea9e2fcff4028b6766459e90bc5fc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96cba770f3334f79af3162475b54fa0f in term 1.
I20260709 13:24:43.606832 28688 tablet_bootstrap.cc:654] T e4c7e579c94a4f5fa195def741ee4180 P 96cba770f3334f79af3162475b54fa0f: Neither blocks nor log segments found. Creating new log.
I20260709 13:24:43.597250 28444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94c1c4dff8e8404dbb00c79a7b7c69c0" candidate_uuid: "542ea9e2fcff4028b6766459e90bc5fc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96cba770f3334f79af3162475b54fa0f" is_pre_election: true
I20260709 13:24:43.594946 28690 raft_consensus.cc:359] T d489ac98a4654a7cb46771880f89b89b P 542ea9e2fcff4028b6766459e90bc5fc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "542ea9e2fcff4028b6766459e90bc5fc" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38409 } } peers { permanent_uuid: "96cba770f3334f79af3162475b54fa0f" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 33585 } } peers { permanent_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 37597 } }
I20260709 13:24:43.607579 28690 raft_consensus.cc:385] T d489ac98a4654a7cb46771880f89b89b P 542ea9e2fcff4028b6766459e90bc5fc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:24:43.607669 28444 raft_consensus.cc:2468] T 94c1c4dff8e8404dbb00c79a7b7c69c0 P 96cba770f3334f79af3162475b54fa0f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 542ea9e2fcff4028b6766459e90bc5fc in term 0.
I20260709 13:24:43.608055 28690 raft_consensus.cc:740] T d489ac98a4654a7cb46771880f89b89b P 542ea9e2fcff4028b6766459e90bc5fc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 542ea9e2fcff4028b6766459e90bc5fc, State: Initialized, Role: FOLLOWER
I20260709 13:24:43.609426 28413 leader_election.cc:304] T fbb3d76b1491485287e0dc48b24dbb46 P 96cba770f3334f79af3162475b54fa0f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 542ea9e2fcff4028b6766459e90bc5fc, 96cba770f3334f79af3162475b54fa0f; no voters: 
I20260709 13:24:43.609333 28690 consensus_queue.cc:260] T d489ac98a4654a7cb46771880f89b89b P 542ea9e2fcff4028b6766459e90bc5fc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "542ea9e2fcff4028b6766459e90bc5fc" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38409 } } peers { permanent_uuid: "96cba770f3334f79af3162475b54fa0f" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 33585 } } peers { permanent_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 37597 } }
I20260709 13:24:43.610287 28689 raft_consensus.cc:359] T d489ac98a4654a7cb46771880f89b89b P 2173e3fd92f34ad6ad8d56d2e6b05299 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "542ea9e2fcff4028b6766459e90bc5fc" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38409 } } peers { permanent_uuid: "96cba770f3334f79af3162475b54fa0f" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 33585 } } peers { permanent_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 37597 } }
I20260709 13:24:43.613214 28689 raft_consensus.cc:385] T d489ac98a4654a7cb46771880f89b89b P 2173e3fd92f34ad6ad8d56d2e6b05299 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:24:43.614917 28340 leader_election.cc:304] T 94c1c4dff8e8404dbb00c79a7b7c69c0 P 542ea9e2fcff4028b6766459e90bc5fc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 542ea9e2fcff4028b6766459e90bc5fc, 96cba770f3334f79af3162475b54fa0f; no voters: 2173e3fd92f34ad6ad8d56d2e6b05299
I20260709 13:24:43.612464 28690 ts_tablet_manager.cc:1434] T d489ac98a4654a7cb46771880f89b89b P 542ea9e2fcff4028b6766459e90bc5fc: Time spent starting tablet: real 0.021s	user 0.008s	sys 0.000s
I20260709 13:24:43.621356 28689 raft_consensus.cc:740] T d489ac98a4654a7cb46771880f89b89b P 2173e3fd92f34ad6ad8d56d2e6b05299 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2173e3fd92f34ad6ad8d56d2e6b05299, State: Initialized, Role: FOLLOWER
I20260709 13:24:43.622342 28704 raft_consensus.cc:2804] T 94c1c4dff8e8404dbb00c79a7b7c69c0 P 542ea9e2fcff4028b6766459e90bc5fc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:24:43.622540 28690 tablet_bootstrap.cc:492] T 6efeed092ec840f2a7748c4a613dd6a7 P 542ea9e2fcff4028b6766459e90bc5fc: Bootstrap starting.
I20260709 13:24:43.622840 28704 raft_consensus.cc:493] T 94c1c4dff8e8404dbb00c79a7b7c69c0 P 542ea9e2fcff4028b6766459e90bc5fc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:24:43.623302 28704 raft_consensus.cc:3060] T 94c1c4dff8e8404dbb00c79a7b7c69c0 P 542ea9e2fcff4028b6766459e90bc5fc [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:24:43.629583 28690 tablet_bootstrap.cc:654] T 6efeed092ec840f2a7748c4a613dd6a7 P 542ea9e2fcff4028b6766459e90bc5fc: Neither blocks nor log segments found. Creating new log.
I20260709 13:24:43.615736 28700 raft_consensus.cc:2804] T fbb3d76b1491485287e0dc48b24dbb46 P 96cba770f3334f79af3162475b54fa0f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:24:43.631724 28700 raft_consensus.cc:697] T fbb3d76b1491485287e0dc48b24dbb46 P 96cba770f3334f79af3162475b54fa0f [term 1 LEADER]: Becoming Leader. State: Replica: 96cba770f3334f79af3162475b54fa0f, State: Running, Role: LEADER
I20260709 13:24:43.633000 28689 consensus_queue.cc:260] T d489ac98a4654a7cb46771880f89b89b P 2173e3fd92f34ad6ad8d56d2e6b05299 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "542ea9e2fcff4028b6766459e90bc5fc" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38409 } } peers { permanent_uuid: "96cba770f3334f79af3162475b54fa0f" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 33585 } } peers { permanent_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 37597 } }
I20260709 13:24:43.637590 28689 ts_tablet_manager.cc:1434] T d489ac98a4654a7cb46771880f89b89b P 2173e3fd92f34ad6ad8d56d2e6b05299: Time spent starting tablet: real 0.039s	user 0.007s	sys 0.001s
I20260709 13:24:43.639339 28700 consensus_queue.cc:237] T fbb3d76b1491485287e0dc48b24dbb46 P 96cba770f3334f79af3162475b54fa0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "542ea9e2fcff4028b6766459e90bc5fc" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38409 } } peers { permanent_uuid: "96cba770f3334f79af3162475b54fa0f" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 33585 } } peers { permanent_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 37597 } }
I20260709 13:24:43.617816 28522 raft_consensus.cc:2468] T fbb3d76b1491485287e0dc48b24dbb46 P 2173e3fd92f34ad6ad8d56d2e6b05299 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96cba770f3334f79af3162475b54fa0f in term 1.
I20260709 13:24:43.642407 28690 tablet_bootstrap.cc:492] T 6efeed092ec840f2a7748c4a613dd6a7 P 542ea9e2fcff4028b6766459e90bc5fc: No bootstrap required, opened a new log
I20260709 13:24:43.643051 28690 ts_tablet_manager.cc:1403] T 6efeed092ec840f2a7748c4a613dd6a7 P 542ea9e2fcff4028b6766459e90bc5fc: Time spent bootstrapping tablet: real 0.021s	user 0.018s	sys 0.000s
I20260709 13:24:43.645270 28704 raft_consensus.cc:515] T 94c1c4dff8e8404dbb00c79a7b7c69c0 P 542ea9e2fcff4028b6766459e90bc5fc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "542ea9e2fcff4028b6766459e90bc5fc" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38409 } } peers { permanent_uuid: "96cba770f3334f79af3162475b54fa0f" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 33585 } } peers { permanent_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 37597 } }
I20260709 13:24:43.646658 28690 raft_consensus.cc:359] T 6efeed092ec840f2a7748c4a613dd6a7 P 542ea9e2fcff4028b6766459e90bc5fc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "542ea9e2fcff4028b6766459e90bc5fc" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38409 } } peers { permanent_uuid: "96cba770f3334f79af3162475b54fa0f" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 33585 } } peers { permanent_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 37597 } }
I20260709 13:24:43.647616 28690 raft_consensus.cc:385] T 6efeed092ec840f2a7748c4a613dd6a7 P 542ea9e2fcff4028b6766459e90bc5fc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:24:43.648008 28690 raft_consensus.cc:740] T 6efeed092ec840f2a7748c4a613dd6a7 P 542ea9e2fcff4028b6766459e90bc5fc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 542ea9e2fcff4028b6766459e90bc5fc, State: Initialized, Role: FOLLOWER
I20260709 13:24:43.648988 28688 tablet_bootstrap.cc:492] T e4c7e579c94a4f5fa195def741ee4180 P 96cba770f3334f79af3162475b54fa0f: No bootstrap required, opened a new log
I20260709 13:24:43.649803 28688 ts_tablet_manager.cc:1403] T e4c7e579c94a4f5fa195def741ee4180 P 96cba770f3334f79af3162475b54fa0f: Time spent bootstrapping tablet: real 0.057s	user 0.014s	sys 0.005s
I20260709 13:24:43.649026 28690 consensus_queue.cc:260] T 6efeed092ec840f2a7748c4a613dd6a7 P 542ea9e2fcff4028b6766459e90bc5fc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "542ea9e2fcff4028b6766459e90bc5fc" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38409 } } peers { permanent_uuid: "96cba770f3334f79af3162475b54fa0f" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 33585 } } peers { permanent_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 37597 } }
I20260709 13:24:43.652092 28690 ts_tablet_manager.cc:1434] T 6efeed092ec840f2a7748c4a613dd6a7 P 542ea9e2fcff4028b6766459e90bc5fc: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.000s
I20260709 13:24:43.653273 28690 tablet_bootstrap.cc:492] T 07bb4101b0c24ee6b460e066cf78eb53 P 542ea9e2fcff4028b6766459e90bc5fc: Bootstrap starting.
I20260709 13:24:43.653267 28688 raft_consensus.cc:359] T e4c7e579c94a4f5fa195def741ee4180 P 96cba770f3334f79af3162475b54fa0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "542ea9e2fcff4028b6766459e90bc5fc" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38409 } } peers { permanent_uuid: "96cba770f3334f79af3162475b54fa0f" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 33585 } } peers { permanent_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 37597 } }
I20260709 13:24:43.654340 28688 raft_consensus.cc:385] T e4c7e579c94a4f5fa195def741ee4180 P 96cba770f3334f79af3162475b54fa0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:24:43.654690 28688 raft_consensus.cc:740] T e4c7e579c94a4f5fa195def741ee4180 P 96cba770f3334f79af3162475b54fa0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 96cba770f3334f79af3162475b54fa0f, State: Initialized, Role: FOLLOWER
I20260709 13:24:43.639079 28689 tablet_bootstrap.cc:492] T 6efeed092ec840f2a7748c4a613dd6a7 P 2173e3fd92f34ad6ad8d56d2e6b05299: Bootstrap starting.
I20260709 13:24:43.659291 28688 consensus_queue.cc:260] T e4c7e579c94a4f5fa195def741ee4180 P 96cba770f3334f79af3162475b54fa0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "542ea9e2fcff4028b6766459e90bc5fc" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38409 } } peers { permanent_uuid: "96cba770f3334f79af3162475b54fa0f" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 33585 } } peers { permanent_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 37597 } }
I20260709 13:24:43.662778 28688 ts_tablet_manager.cc:1434] T e4c7e579c94a4f5fa195def741ee4180 P 96cba770f3334f79af3162475b54fa0f: Time spent starting tablet: real 0.013s	user 0.008s	sys 0.000s
I20260709 13:24:43.663466 28690 tablet_bootstrap.cc:654] T 07bb4101b0c24ee6b460e066cf78eb53 P 542ea9e2fcff4028b6766459e90bc5fc: Neither blocks nor log segments found. Creating new log.
I20260709 13:24:43.663887 28688 tablet_bootstrap.cc:492] T d5de0c782ce844ab8dbec004c32abafa P 96cba770f3334f79af3162475b54fa0f: Bootstrap starting.
I20260709 13:24:43.667979 28444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94c1c4dff8e8404dbb00c79a7b7c69c0" candidate_uuid: "542ea9e2fcff4028b6766459e90bc5fc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96cba770f3334f79af3162475b54fa0f"
I20260709 13:24:43.668697 28444 raft_consensus.cc:3060] T 94c1c4dff8e8404dbb00c79a7b7c69c0 P 96cba770f3334f79af3162475b54fa0f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:24:43.680604 28704 leader_election.cc:290] T 94c1c4dff8e8404dbb00c79a7b7c69c0 P 542ea9e2fcff4028b6766459e90bc5fc [CANDIDATE]: Term 1 election: Requested vote from peers 96cba770f3334f79af3162475b54fa0f (127.14.177.130:33585), 2173e3fd92f34ad6ad8d56d2e6b05299 (127.14.177.131:37597)
I20260709 13:24:43.683418 28522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94c1c4dff8e8404dbb00c79a7b7c69c0" candidate_uuid: "542ea9e2fcff4028b6766459e90bc5fc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299"
I20260709 13:24:43.691884 28444 raft_consensus.cc:2468] T 94c1c4dff8e8404dbb00c79a7b7c69c0 P 96cba770f3334f79af3162475b54fa0f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 542ea9e2fcff4028b6766459e90bc5fc in term 1.
W20260709 13:24:43.692960 28340 leader_election.cc:343] T 94c1c4dff8e8404dbb00c79a7b7c69c0 P 542ea9e2fcff4028b6766459e90bc5fc [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 2173e3fd92f34ad6ad8d56d2e6b05299 (127.14.177.131:37597): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:24:43.695632 28340 leader_election.cc:304] T 94c1c4dff8e8404dbb00c79a7b7c69c0 P 542ea9e2fcff4028b6766459e90bc5fc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 542ea9e2fcff4028b6766459e90bc5fc, 96cba770f3334f79af3162475b54fa0f; no voters: 2173e3fd92f34ad6ad8d56d2e6b05299
I20260709 13:24:43.697319 28707 raft_consensus.cc:2804] T 94c1c4dff8e8404dbb00c79a7b7c69c0 P 542ea9e2fcff4028b6766459e90bc5fc [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:24:43.697889 28707 raft_consensus.cc:697] T 94c1c4dff8e8404dbb00c79a7b7c69c0 P 542ea9e2fcff4028b6766459e90bc5fc [term 1 LEADER]: Becoming Leader. State: Replica: 542ea9e2fcff4028b6766459e90bc5fc, State: Running, Role: LEADER
I20260709 13:24:43.698253 28689 tablet_bootstrap.cc:654] T 6efeed092ec840f2a7748c4a613dd6a7 P 2173e3fd92f34ad6ad8d56d2e6b05299: Neither blocks nor log segments found. Creating new log.
I20260709 13:24:43.698833 28707 consensus_queue.cc:237] T 94c1c4dff8e8404dbb00c79a7b7c69c0 P 542ea9e2fcff4028b6766459e90bc5fc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "542ea9e2fcff4028b6766459e90bc5fc" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38409 } } peers { permanent_uuid: "96cba770f3334f79af3162475b54fa0f" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 33585 } } peers { permanent_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 37597 } }
I20260709 13:24:43.702811 28619 raft_consensus.cc:493] T 6efeed092ec840f2a7748c4a613dd6a7 P 542ea9e2fcff4028b6766459e90bc5fc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:24:43.703521 28619 raft_consensus.cc:515] T 6efeed092ec840f2a7748c4a613dd6a7 P 542ea9e2fcff4028b6766459e90bc5fc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "542ea9e2fcff4028b6766459e90bc5fc" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38409 } } peers { permanent_uuid: "96cba770f3334f79af3162475b54fa0f" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 33585 } } peers { permanent_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 37597 } }
I20260709 13:24:43.704262 28688 tablet_bootstrap.cc:654] T d5de0c782ce844ab8dbec004c32abafa P 96cba770f3334f79af3162475b54fa0f: Neither blocks nor log segments found. Creating new log.
I20260709 13:24:43.706035 28619 leader_election.cc:290] T 6efeed092ec840f2a7748c4a613dd6a7 P 542ea9e2fcff4028b6766459e90bc5fc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 96cba770f3334f79af3162475b54fa0f (127.14.177.130:33585), 2173e3fd92f34ad6ad8d56d2e6b05299 (127.14.177.131:37597)
I20260709 13:24:43.708817 28522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6efeed092ec840f2a7748c4a613dd6a7" candidate_uuid: "542ea9e2fcff4028b6766459e90bc5fc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" is_pre_election: true
I20260709 13:24:43.711848 28444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6efeed092ec840f2a7748c4a613dd6a7" candidate_uuid: "542ea9e2fcff4028b6766459e90bc5fc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96cba770f3334f79af3162475b54fa0f" is_pre_election: true
I20260709 13:24:43.712770 28444 raft_consensus.cc:2468] T 6efeed092ec840f2a7748c4a613dd6a7 P 96cba770f3334f79af3162475b54fa0f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 542ea9e2fcff4028b6766459e90bc5fc in term 0.
W20260709 13:24:43.714198 28340 leader_election.cc:343] T 6efeed092ec840f2a7748c4a613dd6a7 P 542ea9e2fcff4028b6766459e90bc5fc [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2173e3fd92f34ad6ad8d56d2e6b05299 (127.14.177.131:37597): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:24:43.715595 28689 tablet_bootstrap.cc:492] T 6efeed092ec840f2a7748c4a613dd6a7 P 2173e3fd92f34ad6ad8d56d2e6b05299: No bootstrap required, opened a new log
I20260709 13:24:43.716527 28689 ts_tablet_manager.cc:1403] T 6efeed092ec840f2a7748c4a613dd6a7 P 2173e3fd92f34ad6ad8d56d2e6b05299: Time spent bootstrapping tablet: real 0.078s	user 0.017s	sys 0.001s
I20260709 13:24:43.715418 28340 leader_election.cc:304] T 6efeed092ec840f2a7748c4a613dd6a7 P 542ea9e2fcff4028b6766459e90bc5fc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 542ea9e2fcff4028b6766459e90bc5fc, 96cba770f3334f79af3162475b54fa0f; no voters: 2173e3fd92f34ad6ad8d56d2e6b05299
I20260709 13:24:43.724637 28690 tablet_bootstrap.cc:492] T 07bb4101b0c24ee6b460e066cf78eb53 P 542ea9e2fcff4028b6766459e90bc5fc: No bootstrap required, opened a new log
I20260709 13:24:43.724397 28689 raft_consensus.cc:359] T 6efeed092ec840f2a7748c4a613dd6a7 P 2173e3fd92f34ad6ad8d56d2e6b05299 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "542ea9e2fcff4028b6766459e90bc5fc" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38409 } } peers { permanent_uuid: "96cba770f3334f79af3162475b54fa0f" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 33585 } } peers { permanent_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 37597 } }
I20260709 13:24:43.725237 28689 raft_consensus.cc:385] T 6efeed092ec840f2a7748c4a613dd6a7 P 2173e3fd92f34ad6ad8d56d2e6b05299 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:24:43.724184 28271 catalog_manager.cc:5697] T fbb3d76b1491485287e0dc48b24dbb46 P 96cba770f3334f79af3162475b54fa0f reported cstate change: term changed from 0 to 1, leader changed from <none> to 96cba770f3334f79af3162475b54fa0f (127.14.177.130). New cstate: current_term: 1 leader_uuid: "96cba770f3334f79af3162475b54fa0f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "542ea9e2fcff4028b6766459e90bc5fc" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38409 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "96cba770f3334f79af3162475b54fa0f" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 33585 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 37597 } health_report { overall_health: UNKNOWN } } }
I20260709 13:24:43.725351 28690 ts_tablet_manager.cc:1403] T 07bb4101b0c24ee6b460e066cf78eb53 P 542ea9e2fcff4028b6766459e90bc5fc: Time spent bootstrapping tablet: real 0.072s	user 0.035s	sys 0.026s
I20260709 13:24:43.725610 28689 raft_consensus.cc:740] T 6efeed092ec840f2a7748c4a613dd6a7 P 2173e3fd92f34ad6ad8d56d2e6b05299 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2173e3fd92f34ad6ad8d56d2e6b05299, State: Initialized, Role: FOLLOWER
I20260709 13:24:43.726660 28689 consensus_queue.cc:260] T 6efeed092ec840f2a7748c4a613dd6a7 P 2173e3fd92f34ad6ad8d56d2e6b05299 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "542ea9e2fcff4028b6766459e90bc5fc" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38409 } } peers { permanent_uuid: "96cba770f3334f79af3162475b54fa0f" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 33585 } } peers { permanent_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 37597 } }
I20260709 13:24:43.729178 28704 raft_consensus.cc:2804] T 6efeed092ec840f2a7748c4a613dd6a7 P 542ea9e2fcff4028b6766459e90bc5fc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:24:43.731590 28704 raft_consensus.cc:493] T 6efeed092ec840f2a7748c4a613dd6a7 P 542ea9e2fcff4028b6766459e90bc5fc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:24:43.732127 28704 raft_consensus.cc:3060] T 6efeed092ec840f2a7748c4a613dd6a7 P 542ea9e2fcff4028b6766459e90bc5fc [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:24:43.729291 28690 raft_consensus.cc:359] T 07bb4101b0c24ee6b460e066cf78eb53 P 542ea9e2fcff4028b6766459e90bc5fc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "542ea9e2fcff4028b6766459e90bc5fc" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38409 } } peers { permanent_uuid: "96cba770f3334f79af3162475b54fa0f" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 33585 } } peers { permanent_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 37597 } }
I20260709 13:24:43.730074 28689 ts_tablet_manager.cc:1434] T 6efeed092ec840f2a7748c4a613dd6a7 P 2173e3fd92f34ad6ad8d56d2e6b05299: Time spent starting tablet: real 0.013s	user 0.005s	sys 0.001s
I20260709 13:24:43.733647 28690 raft_consensus.cc:385] T 07bb4101b0c24ee6b460e066cf78eb53 P 542ea9e2fcff4028b6766459e90bc5fc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:24:43.733992 28690 raft_consensus.cc:740] T 07bb4101b0c24ee6b460e066cf78eb53 P 542ea9e2fcff4028b6766459e90bc5fc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 542ea9e2fcff4028b6766459e90bc5fc, State: Initialized, Role: FOLLOWER
I20260709 13:24:43.734628 28689 tablet_bootstrap.cc:492] T 2b05204beeff49309ed4e0dd43b4be7f P 2173e3fd92f34ad6ad8d56d2e6b05299: Bootstrap starting.
I20260709 13:24:43.734747 28690 consensus_queue.cc:260] T 07bb4101b0c24ee6b460e066cf78eb53 P 542ea9e2fcff4028b6766459e90bc5fc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "542ea9e2fcff4028b6766459e90bc5fc" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38409 } } peers { permanent_uuid: "96cba770f3334f79af3162475b54fa0f" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 33585 } } peers { permanent_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 37597 } }
I20260709 13:24:43.736373 28269 catalog_manager.cc:5697] T 94c1c4dff8e8404dbb00c79a7b7c69c0 P 542ea9e2fcff4028b6766459e90bc5fc reported cstate change: term changed from 0 to 1, leader changed from <none> to 542ea9e2fcff4028b6766459e90bc5fc (127.14.177.129). New cstate: current_term: 1 leader_uuid: "542ea9e2fcff4028b6766459e90bc5fc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "542ea9e2fcff4028b6766459e90bc5fc" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38409 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "96cba770f3334f79af3162475b54fa0f" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 33585 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 37597 } health_report { overall_health: UNKNOWN } } }
I20260709 13:24:43.745182 28690 ts_tablet_manager.cc:1434] T 07bb4101b0c24ee6b460e066cf78eb53 P 542ea9e2fcff4028b6766459e90bc5fc: Time spent starting tablet: real 0.019s	user 0.006s	sys 0.001s
I20260709 13:24:43.746183 28688 tablet_bootstrap.cc:492] T d5de0c782ce844ab8dbec004c32abafa P 96cba770f3334f79af3162475b54fa0f: No bootstrap required, opened a new log
I20260709 13:24:43.747393 28690 tablet_bootstrap.cc:492] T d5de0c782ce844ab8dbec004c32abafa P 542ea9e2fcff4028b6766459e90bc5fc: Bootstrap starting.
I20260709 13:24:43.747768 28688 ts_tablet_manager.cc:1403] T d5de0c782ce844ab8dbec004c32abafa P 96cba770f3334f79af3162475b54fa0f: Time spent bootstrapping tablet: real 0.084s	user 0.013s	sys 0.011s
I20260709 13:24:43.750874 28702 raft_consensus.cc:493] T e4c7e579c94a4f5fa195def741ee4180 P 2173e3fd92f34ad6ad8d56d2e6b05299 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:24:43.751502 28702 raft_consensus.cc:515] T e4c7e579c94a4f5fa195def741ee4180 P 2173e3fd92f34ad6ad8d56d2e6b05299 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "542ea9e2fcff4028b6766459e90bc5fc" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38409 } } peers { permanent_uuid: "96cba770f3334f79af3162475b54fa0f" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 33585 } } peers { permanent_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 37597 } }
I20260709 13:24:43.757772 28688 raft_consensus.cc:359] T d5de0c782ce844ab8dbec004c32abafa P 96cba770f3334f79af3162475b54fa0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "542ea9e2fcff4028b6766459e90bc5fc" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38409 } } peers { permanent_uuid: "96cba770f3334f79af3162475b54fa0f" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 33585 } } peers { permanent_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 37597 } }
I20260709 13:24:43.758504 28702 leader_election.cc:290] T e4c7e579c94a4f5fa195def741ee4180 P 2173e3fd92f34ad6ad8d56d2e6b05299 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 542ea9e2fcff4028b6766459e90bc5fc (127.14.177.129:38409), 96cba770f3334f79af3162475b54fa0f (127.14.177.130:33585)
I20260709 13:24:43.758411 28444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4c7e579c94a4f5fa195def741ee4180" candidate_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96cba770f3334f79af3162475b54fa0f" is_pre_election: true
I20260709 13:24:43.750866 28704 raft_consensus.cc:515] T 6efeed092ec840f2a7748c4a613dd6a7 P 542ea9e2fcff4028b6766459e90bc5fc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "542ea9e2fcff4028b6766459e90bc5fc" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38409 } } peers { permanent_uuid: "96cba770f3334f79af3162475b54fa0f" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 33585 } } peers { permanent_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 37597 } }
I20260709 13:24:43.763585 28444 raft_consensus.cc:2468] T e4c7e579c94a4f5fa195def741ee4180 P 96cba770f3334f79af3162475b54fa0f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2173e3fd92f34ad6ad8d56d2e6b05299 in term 0.
I20260709 13:24:43.759580 28372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4c7e579c94a4f5fa195def741ee4180" candidate_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "542ea9e2fcff4028b6766459e90bc5fc" is_pre_election: true
I20260709 13:24:43.759637 28689 tablet_bootstrap.cc:654] T 2b05204beeff49309ed4e0dd43b4be7f P 2173e3fd92f34ad6ad8d56d2e6b05299: Neither blocks nor log segments found. Creating new log.
I20260709 13:24:43.764487 28372 raft_consensus.cc:2468] T e4c7e579c94a4f5fa195def741ee4180 P 542ea9e2fcff4028b6766459e90bc5fc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2173e3fd92f34ad6ad8d56d2e6b05299 in term 0.
I20260709 13:24:43.765070 28704 leader_election.cc:290] T 6efeed092ec840f2a7748c4a613dd6a7 P 542ea9e2fcff4028b6766459e90bc5fc [CANDIDATE]: Term 1 election: Requested vote from peers 96cba770f3334f79af3162475b54fa0f (127.14.177.130:33585), 2173e3fd92f34ad6ad8d56d2e6b05299 (127.14.177.131:37597)
I20260709 13:24:43.762439 28690 tablet_bootstrap.cc:654] T d5de0c782ce844ab8dbec004c32abafa P 542ea9e2fcff4028b6766459e90bc5fc: Neither blocks nor log segments found. Creating new log.
I20260709 13:24:43.766703 28522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6efeed092ec840f2a7748c4a613dd6a7" candidate_uuid: "542ea9e2fcff4028b6766459e90bc5fc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299"
I20260709 13:24:43.762840 28688 raft_consensus.cc:385] T d5de0c782ce844ab8dbec004c32abafa P 96cba770f3334f79af3162475b54fa0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:24:43.768891 28688 raft_consensus.cc:740] T d5de0c782ce844ab8dbec004c32abafa P 96cba770f3334f79af3162475b54fa0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 96cba770f3334f79af3162475b54fa0f, State: Initialized, Role: FOLLOWER
I20260709 13:24:43.768954 28522 raft_consensus.cc:3060] T 6efeed092ec840f2a7748c4a613dd6a7 P 2173e3fd92f34ad6ad8d56d2e6b05299 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:24:43.769814 28688 consensus_queue.cc:260] T d5de0c782ce844ab8dbec004c32abafa P 96cba770f3334f79af3162475b54fa0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "542ea9e2fcff4028b6766459e90bc5fc" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38409 } } peers { permanent_uuid: "96cba770f3334f79af3162475b54fa0f" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 33585 } } peers { permanent_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 37597 } }
I20260709 13:24:43.768059 28489 leader_election.cc:304] T e4c7e579c94a4f5fa195def741ee4180 P 2173e3fd92f34ad6ad8d56d2e6b05299 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2173e3fd92f34ad6ad8d56d2e6b05299, 96cba770f3334f79af3162475b54fa0f; no voters: 
I20260709 13:24:43.772128 28702 raft_consensus.cc:2804] T e4c7e579c94a4f5fa195def741ee4180 P 2173e3fd92f34ad6ad8d56d2e6b05299 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:24:43.772579 28688 ts_tablet_manager.cc:1434] T d5de0c782ce844ab8dbec004c32abafa P 96cba770f3334f79af3162475b54fa0f: Time spent starting tablet: real 0.024s	user 0.006s	sys 0.001s
I20260709 13:24:43.773865 28688 tablet_bootstrap.cc:492] T 07bb4101b0c24ee6b460e066cf78eb53 P 96cba770f3334f79af3162475b54fa0f: Bootstrap starting.
I20260709 13:24:43.775642 28444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6efeed092ec840f2a7748c4a613dd6a7" candidate_uuid: "542ea9e2fcff4028b6766459e90bc5fc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96cba770f3334f79af3162475b54fa0f"
I20260709 13:24:43.776425 28444 raft_consensus.cc:3060] T 6efeed092ec840f2a7748c4a613dd6a7 P 96cba770f3334f79af3162475b54fa0f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:24:43.772591 28702 raft_consensus.cc:493] T e4c7e579c94a4f5fa195def741ee4180 P 2173e3fd92f34ad6ad8d56d2e6b05299 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:24:43.779831 28702 raft_consensus.cc:3060] T e4c7e579c94a4f5fa195def741ee4180 P 2173e3fd92f34ad6ad8d56d2e6b05299 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:24:43.781185 28690 tablet_bootstrap.cc:492] T d5de0c782ce844ab8dbec004c32abafa P 542ea9e2fcff4028b6766459e90bc5fc: No bootstrap required, opened a new log
I20260709 13:24:43.781692 28690 ts_tablet_manager.cc:1403] T d5de0c782ce844ab8dbec004c32abafa P 542ea9e2fcff4028b6766459e90bc5fc: Time spent bootstrapping tablet: real 0.035s	user 0.010s	sys 0.003s
I20260709 13:24:43.784461 28690 raft_consensus.cc:359] T d5de0c782ce844ab8dbec004c32abafa P 542ea9e2fcff4028b6766459e90bc5fc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "542ea9e2fcff4028b6766459e90bc5fc" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38409 } } peers { permanent_uuid: "96cba770f3334f79af3162475b54fa0f" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 33585 } } peers { permanent_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 37597 } }
I20260709 13:24:43.785558 28690 raft_consensus.cc:385] T d5de0c782ce844ab8dbec004c32abafa P 542ea9e2fcff4028b6766459e90bc5fc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:24:43.786029 28690 raft_consensus.cc:740] T d5de0c782ce844ab8dbec004c32abafa P 542ea9e2fcff4028b6766459e90bc5fc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 542ea9e2fcff4028b6766459e90bc5fc, State: Initialized, Role: FOLLOWER
I20260709 13:24:43.812994 28688 tablet_bootstrap.cc:654] T 07bb4101b0c24ee6b460e066cf78eb53 P 96cba770f3334f79af3162475b54fa0f: Neither blocks nor log segments found. Creating new log.
I20260709 13:24:43.813700 28690 consensus_queue.cc:260] T d5de0c782ce844ab8dbec004c32abafa P 542ea9e2fcff4028b6766459e90bc5fc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "542ea9e2fcff4028b6766459e90bc5fc" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38409 } } peers { permanent_uuid: "96cba770f3334f79af3162475b54fa0f" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 33585 } } peers { permanent_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 37597 } }
I20260709 13:24:43.819203 28690 ts_tablet_manager.cc:1434] T d5de0c782ce844ab8dbec004c32abafa P 542ea9e2fcff4028b6766459e90bc5fc: Time spent starting tablet: real 0.037s	user 0.012s	sys 0.013s
I20260709 13:24:43.820467 28690 tablet_bootstrap.cc:492] T cbb4f686d0884c178ea6b01616da20e8 P 542ea9e2fcff4028b6766459e90bc5fc: Bootstrap starting.
W20260709 13:24:43.822633 28487 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.14.177.129:38409, user_credentials={real_user=slave}} blocked reactor thread for 50853.1us
I20260709 13:24:43.827622 28690 tablet_bootstrap.cc:654] T cbb4f686d0884c178ea6b01616da20e8 P 542ea9e2fcff4028b6766459e90bc5fc: Neither blocks nor log segments found. Creating new log.
I20260709 13:24:43.833766 28704 raft_consensus.cc:493] T d489ac98a4654a7cb46771880f89b89b P 542ea9e2fcff4028b6766459e90bc5fc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:24:43.834517 28704 raft_consensus.cc:515] T d489ac98a4654a7cb46771880f89b89b P 542ea9e2fcff4028b6766459e90bc5fc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "542ea9e2fcff4028b6766459e90bc5fc" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38409 } } peers { permanent_uuid: "96cba770f3334f79af3162475b54fa0f" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 33585 } } peers { permanent_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 37597 } }
I20260709 13:24:43.836876 28704 leader_election.cc:290] T d489ac98a4654a7cb46771880f89b89b P 542ea9e2fcff4028b6766459e90bc5fc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 96cba770f3334f79af3162475b54fa0f (127.14.177.130:33585), 2173e3fd92f34ad6ad8d56d2e6b05299 (127.14.177.131:37597)
I20260709 13:24:43.837841 28647 raft_consensus.cc:493] T d489ac98a4654a7cb46771880f89b89b P 2173e3fd92f34ad6ad8d56d2e6b05299 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:24:43.838584 28647 raft_consensus.cc:515] T d489ac98a4654a7cb46771880f89b89b P 2173e3fd92f34ad6ad8d56d2e6b05299 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "542ea9e2fcff4028b6766459e90bc5fc" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38409 } } peers { permanent_uuid: "96cba770f3334f79af3162475b54fa0f" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 33585 } } peers { permanent_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 37597 } }
I20260709 13:24:43.839023 28443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d489ac98a4654a7cb46771880f89b89b" candidate_uuid: "542ea9e2fcff4028b6766459e90bc5fc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96cba770f3334f79af3162475b54fa0f" is_pre_election: true
I20260709 13:24:43.839848 28443 raft_consensus.cc:2468] T d489ac98a4654a7cb46771880f89b89b P 96cba770f3334f79af3162475b54fa0f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 542ea9e2fcff4028b6766459e90bc5fc in term 0.
I20260709 13:24:43.841377 28340 leader_election.cc:304] T d489ac98a4654a7cb46771880f89b89b P 542ea9e2fcff4028b6766459e90bc5fc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 542ea9e2fcff4028b6766459e90bc5fc, 96cba770f3334f79af3162475b54fa0f; no voters: 
I20260709 13:24:43.842923 28702 raft_consensus.cc:515] T e4c7e579c94a4f5fa195def741ee4180 P 2173e3fd92f34ad6ad8d56d2e6b05299 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "542ea9e2fcff4028b6766459e90bc5fc" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38409 } } peers { permanent_uuid: "96cba770f3334f79af3162475b54fa0f" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 33585 } } peers { permanent_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 37597 } }
I20260709 13:24:43.843150 28443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d489ac98a4654a7cb46771880f89b89b" candidate_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96cba770f3334f79af3162475b54fa0f" is_pre_election: true
I20260709 13:24:43.843482 28520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d489ac98a4654a7cb46771880f89b89b" candidate_uuid: "542ea9e2fcff4028b6766459e90bc5fc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" is_pre_election: true
I20260709 13:24:43.843922 28647 leader_election.cc:290] T d489ac98a4654a7cb46771880f89b89b P 2173e3fd92f34ad6ad8d56d2e6b05299 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 542ea9e2fcff4028b6766459e90bc5fc (127.14.177.129:38409), 96cba770f3334f79af3162475b54fa0f (127.14.177.130:33585)
I20260709 13:24:43.844187 28520 raft_consensus.cc:2468] T d489ac98a4654a7cb46771880f89b89b P 2173e3fd92f34ad6ad8d56d2e6b05299 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 542ea9e2fcff4028b6766459e90bc5fc in term 0.
I20260709 13:24:43.845782 28372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d489ac98a4654a7cb46771880f89b89b" candidate_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "542ea9e2fcff4028b6766459e90bc5fc" is_pre_election: true
I20260709 13:24:43.846479 28372 raft_consensus.cc:2468] T d489ac98a4654a7cb46771880f89b89b P 542ea9e2fcff4028b6766459e90bc5fc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2173e3fd92f34ad6ad8d56d2e6b05299 in term 0.
I20260709 13:24:43.848055 28487 leader_election.cc:304] T d489ac98a4654a7cb46771880f89b89b P 2173e3fd92f34ad6ad8d56d2e6b05299 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2173e3fd92f34ad6ad8d56d2e6b05299, 542ea9e2fcff4028b6766459e90bc5fc; no voters: 
I20260709 13:24:43.849133 28647 raft_consensus.cc:2804] T d489ac98a4654a7cb46771880f89b89b P 2173e3fd92f34ad6ad8d56d2e6b05299 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:24:43.849609 28647 raft_consensus.cc:493] T d489ac98a4654a7cb46771880f89b89b P 2173e3fd92f34ad6ad8d56d2e6b05299 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:24:43.850080 28647 raft_consensus.cc:3060] T d489ac98a4654a7cb46771880f89b89b P 2173e3fd92f34ad6ad8d56d2e6b05299 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:24:43.851295 28704 raft_consensus.cc:2804] T d489ac98a4654a7cb46771880f89b89b P 542ea9e2fcff4028b6766459e90bc5fc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:24:43.855027 28704 raft_consensus.cc:493] T d489ac98a4654a7cb46771880f89b89b P 542ea9e2fcff4028b6766459e90bc5fc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:24:43.855588 28704 raft_consensus.cc:3060] T d489ac98a4654a7cb46771880f89b89b P 542ea9e2fcff4028b6766459e90bc5fc [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:24:43.844120 28443 raft_consensus.cc:2468] T d489ac98a4654a7cb46771880f89b89b P 96cba770f3334f79af3162475b54fa0f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2173e3fd92f34ad6ad8d56d2e6b05299 in term 0.
I20260709 13:24:43.855723 28372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4c7e579c94a4f5fa195def741ee4180" candidate_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "542ea9e2fcff4028b6766459e90bc5fc"
I20260709 13:24:43.857973 28372 raft_consensus.cc:3060] T e4c7e579c94a4f5fa195def741ee4180 P 542ea9e2fcff4028b6766459e90bc5fc [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:24:43.859902 28444 raft_consensus.cc:2468] T 6efeed092ec840f2a7748c4a613dd6a7 P 96cba770f3334f79af3162475b54fa0f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 542ea9e2fcff4028b6766459e90bc5fc in term 1.
I20260709 13:24:43.861505 28340 leader_election.cc:304] T 6efeed092ec840f2a7748c4a613dd6a7 P 542ea9e2fcff4028b6766459e90bc5fc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 542ea9e2fcff4028b6766459e90bc5fc, 96cba770f3334f79af3162475b54fa0f; no voters: 
I20260709 13:24:43.862555 28707 raft_consensus.cc:2804] T 6efeed092ec840f2a7748c4a613dd6a7 P 542ea9e2fcff4028b6766459e90bc5fc [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:24:43.863025 28707 raft_consensus.cc:697] T 6efeed092ec840f2a7748c4a613dd6a7 P 542ea9e2fcff4028b6766459e90bc5fc [term 1 LEADER]: Becoming Leader. State: Replica: 542ea9e2fcff4028b6766459e90bc5fc, State: Running, Role: LEADER
I20260709 13:24:43.864116 28707 consensus_queue.cc:237] T 6efeed092ec840f2a7748c4a613dd6a7 P 542ea9e2fcff4028b6766459e90bc5fc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "542ea9e2fcff4028b6766459e90bc5fc" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38409 } } peers { permanent_uuid: "96cba770f3334f79af3162475b54fa0f" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 33585 } } peers { permanent_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 37597 } }
I20260709 13:24:43.865357 28443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4c7e579c94a4f5fa195def741ee4180" candidate_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96cba770f3334f79af3162475b54fa0f"
I20260709 13:24:43.866048 28443 raft_consensus.cc:3060] T e4c7e579c94a4f5fa195def741ee4180 P 96cba770f3334f79af3162475b54fa0f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:24:43.866917 28689 tablet_bootstrap.cc:492] T 2b05204beeff49309ed4e0dd43b4be7f P 2173e3fd92f34ad6ad8d56d2e6b05299: No bootstrap required, opened a new log
I20260709 13:24:43.867451 28689 ts_tablet_manager.cc:1403] T 2b05204beeff49309ed4e0dd43b4be7f P 2173e3fd92f34ad6ad8d56d2e6b05299: Time spent bootstrapping tablet: real 0.133s	user 0.053s	sys 0.050s
I20260709 13:24:43.869040 28522 raft_consensus.cc:2468] T 6efeed092ec840f2a7748c4a613dd6a7 P 2173e3fd92f34ad6ad8d56d2e6b05299 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 542ea9e2fcff4028b6766459e90bc5fc in term 1.
I20260709 13:24:43.870611 28689 raft_consensus.cc:359] T 2b05204beeff49309ed4e0dd43b4be7f P 2173e3fd92f34ad6ad8d56d2e6b05299 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "542ea9e2fcff4028b6766459e90bc5fc" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38409 } } peers { permanent_uuid: "96cba770f3334f79af3162475b54fa0f" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 33585 } } peers { permanent_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 37597 } }
I20260709 13:24:43.870848 28702 leader_election.cc:290] T e4c7e579c94a4f5fa195def741ee4180 P 2173e3fd92f34ad6ad8d56d2e6b05299 [CANDIDATE]: Term 1 election: Requested vote from peers 542ea9e2fcff4028b6766459e90bc5fc (127.14.177.129:38409), 96cba770f3334f79af3162475b54fa0f (127.14.177.130:33585)
I20260709 13:24:43.871269 28689 raft_consensus.cc:385] T 2b05204beeff49309ed4e0dd43b4be7f P 2173e3fd92f34ad6ad8d56d2e6b05299 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:24:43.871677 28689 raft_consensus.cc:740] T 2b05204beeff49309ed4e0dd43b4be7f P 2173e3fd92f34ad6ad8d56d2e6b05299 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2173e3fd92f34ad6ad8d56d2e6b05299, State: Initialized, Role: FOLLOWER
I20260709 13:24:43.872555 28689 consensus_queue.cc:260] T 2b05204beeff49309ed4e0dd43b4be7f P 2173e3fd92f34ad6ad8d56d2e6b05299 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "542ea9e2fcff4028b6766459e90bc5fc" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38409 } } peers { permanent_uuid: "96cba770f3334f79af3162475b54fa0f" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 33585 } } peers { permanent_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 37597 } }
I20260709 13:24:43.873644 28688 tablet_bootstrap.cc:492] T 07bb4101b0c24ee6b460e066cf78eb53 P 96cba770f3334f79af3162475b54fa0f: No bootstrap required, opened a new log
I20260709 13:24:43.874262 28688 ts_tablet_manager.cc:1403] T 07bb4101b0c24ee6b460e066cf78eb53 P 96cba770f3334f79af3162475b54fa0f: Time spent bootstrapping tablet: real 0.101s	user 0.020s	sys 0.020s
I20260709 13:24:43.875101 28689 ts_tablet_manager.cc:1434] T 2b05204beeff49309ed4e0dd43b4be7f P 2173e3fd92f34ad6ad8d56d2e6b05299: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.001s
I20260709 13:24:43.876915 28689 tablet_bootstrap.cc:492] T d5de0c782ce844ab8dbec004c32abafa P 2173e3fd92f34ad6ad8d56d2e6b05299: Bootstrap starting.
I20260709 13:24:43.877625 28688 raft_consensus.cc:359] T 07bb4101b0c24ee6b460e066cf78eb53 P 96cba770f3334f79af3162475b54fa0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "542ea9e2fcff4028b6766459e90bc5fc" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38409 } } peers { permanent_uuid: "96cba770f3334f79af3162475b54fa0f" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 33585 } } peers { permanent_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 37597 } }
I20260709 13:24:43.883459 28372 raft_consensus.cc:2468] T e4c7e579c94a4f5fa195def741ee4180 P 542ea9e2fcff4028b6766459e90bc5fc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2173e3fd92f34ad6ad8d56d2e6b05299 in term 1.
I20260709 13:24:43.882350 28647 raft_consensus.cc:515] T d489ac98a4654a7cb46771880f89b89b P 2173e3fd92f34ad6ad8d56d2e6b05299 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "542ea9e2fcff4028b6766459e90bc5fc" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38409 } } peers { permanent_uuid: "96cba770f3334f79af3162475b54fa0f" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 33585 } } peers { permanent_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 37597 } }
I20260709 13:24:43.885404 28689 tablet_bootstrap.cc:654] T d5de0c782ce844ab8dbec004c32abafa P 2173e3fd92f34ad6ad8d56d2e6b05299: Neither blocks nor log segments found. Creating new log.
I20260709 13:24:43.878831 28700 raft_consensus.cc:493] T d5de0c782ce844ab8dbec004c32abafa P 96cba770f3334f79af3162475b54fa0f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:24:43.886468 28700 raft_consensus.cc:515] T d5de0c782ce844ab8dbec004c32abafa P 96cba770f3334f79af3162475b54fa0f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "542ea9e2fcff4028b6766459e90bc5fc" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38409 } } peers { permanent_uuid: "96cba770f3334f79af3162475b54fa0f" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 33585 } } peers { permanent_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 37597 } }
I20260709 13:24:43.887122 28647 leader_election.cc:290] T d489ac98a4654a7cb46771880f89b89b P 2173e3fd92f34ad6ad8d56d2e6b05299 [CANDIDATE]: Term 1 election: Requested vote from peers 542ea9e2fcff4028b6766459e90bc5fc (127.14.177.129:38409), 96cba770f3334f79af3162475b54fa0f (127.14.177.130:33585)
I20260709 13:24:43.883865 28688 raft_consensus.cc:385] T 07bb4101b0c24ee6b460e066cf78eb53 P 96cba770f3334f79af3162475b54fa0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:24:43.888497 28688 raft_consensus.cc:740] T 07bb4101b0c24ee6b460e066cf78eb53 P 96cba770f3334f79af3162475b54fa0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 96cba770f3334f79af3162475b54fa0f, State: Initialized, Role: FOLLOWER
I20260709 13:24:43.889082 28487 leader_election.cc:304] T e4c7e579c94a4f5fa195def741ee4180 P 2173e3fd92f34ad6ad8d56d2e6b05299 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2173e3fd92f34ad6ad8d56d2e6b05299, 542ea9e2fcff4028b6766459e90bc5fc; no voters: 
I20260709 13:24:43.889278 28688 consensus_queue.cc:260] T 07bb4101b0c24ee6b460e066cf78eb53 P 96cba770f3334f79af3162475b54fa0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "542ea9e2fcff4028b6766459e90bc5fc" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38409 } } peers { permanent_uuid: "96cba770f3334f79af3162475b54fa0f" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 33585 } } peers { permanent_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 37597 } }
I20260709 13:24:43.891232 28444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d489ac98a4654a7cb46771880f89b89b" candidate_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96cba770f3334f79af3162475b54fa0f"
I20260709 13:24:43.891752 28688 ts_tablet_manager.cc:1434] T 07bb4101b0c24ee6b460e066cf78eb53 P 96cba770f3334f79af3162475b54fa0f: Time spent starting tablet: real 0.017s	user 0.004s	sys 0.003s
I20260709 13:24:43.903568 28444 raft_consensus.cc:3060] T d489ac98a4654a7cb46771880f89b89b P 96cba770f3334f79af3162475b54fa0f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:24:43.904356 28688 tablet_bootstrap.cc:492] T 2b05204beeff49309ed4e0dd43b4be7f P 96cba770f3334f79af3162475b54fa0f: Bootstrap starting.
I20260709 13:24:43.892233 28372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d489ac98a4654a7cb46771880f89b89b" candidate_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "542ea9e2fcff4028b6766459e90bc5fc"
I20260709 13:24:43.893648 28700 leader_election.cc:290] T d5de0c782ce844ab8dbec004c32abafa P 96cba770f3334f79af3162475b54fa0f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 542ea9e2fcff4028b6766459e90bc5fc (127.14.177.129:38409), 2173e3fd92f34ad6ad8d56d2e6b05299 (127.14.177.131:37597)
I20260709 13:24:43.896306 28371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5de0c782ce844ab8dbec004c32abafa" candidate_uuid: "96cba770f3334f79af3162475b54fa0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "542ea9e2fcff4028b6766459e90bc5fc" is_pre_election: true
I20260709 13:24:43.911824 28371 raft_consensus.cc:2468] T d5de0c782ce844ab8dbec004c32abafa P 542ea9e2fcff4028b6766459e90bc5fc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96cba770f3334f79af3162475b54fa0f in term 0.
I20260709 13:24:43.883369 28443 raft_consensus.cc:2468] T e4c7e579c94a4f5fa195def741ee4180 P 96cba770f3334f79af3162475b54fa0f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2173e3fd92f34ad6ad8d56d2e6b05299 in term 1.
I20260709 13:24:43.912624 28688 tablet_bootstrap.cc:654] T 2b05204beeff49309ed4e0dd43b4be7f P 96cba770f3334f79af3162475b54fa0f: Neither blocks nor log segments found. Creating new log.
I20260709 13:24:43.897238 28647 raft_consensus.cc:2804] T e4c7e579c94a4f5fa195def741ee4180 P 2173e3fd92f34ad6ad8d56d2e6b05299 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:24:43.913295 28647 raft_consensus.cc:697] T e4c7e579c94a4f5fa195def741ee4180 P 2173e3fd92f34ad6ad8d56d2e6b05299 [term 1 LEADER]: Becoming Leader. State: Replica: 2173e3fd92f34ad6ad8d56d2e6b05299, State: Running, Role: LEADER
I20260709 13:24:43.914167 28647 consensus_queue.cc:237] T e4c7e579c94a4f5fa195def741ee4180 P 2173e3fd92f34ad6ad8d56d2e6b05299 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "542ea9e2fcff4028b6766459e90bc5fc" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38409 } } peers { permanent_uuid: "96cba770f3334f79af3162475b54fa0f" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 33585 } } peers { permanent_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 37597 } }
I20260709 13:24:43.914841 28271 catalog_manager.cc:5697] T 6efeed092ec840f2a7748c4a613dd6a7 P 542ea9e2fcff4028b6766459e90bc5fc reported cstate change: term changed from 0 to 1, leader changed from <none> to 542ea9e2fcff4028b6766459e90bc5fc (127.14.177.129). New cstate: current_term: 1 leader_uuid: "542ea9e2fcff4028b6766459e90bc5fc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "542ea9e2fcff4028b6766459e90bc5fc" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38409 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "96cba770f3334f79af3162475b54fa0f" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 33585 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 37597 } health_report { overall_health: UNKNOWN } } }
I20260709 13:24:43.899641 28690 tablet_bootstrap.cc:492] T cbb4f686d0884c178ea6b01616da20e8 P 542ea9e2fcff4028b6766459e90bc5fc: No bootstrap required, opened a new log
I20260709 13:24:43.918527 28413 leader_election.cc:304] T d5de0c782ce844ab8dbec004c32abafa P 96cba770f3334f79af3162475b54fa0f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 542ea9e2fcff4028b6766459e90bc5fc, 96cba770f3334f79af3162475b54fa0f; no voters: 
I20260709 13:24:43.918962 28690 ts_tablet_manager.cc:1403] T cbb4f686d0884c178ea6b01616da20e8 P 542ea9e2fcff4028b6766459e90bc5fc: Time spent bootstrapping tablet: real 0.099s	user 0.011s	sys 0.005s
I20260709 13:24:43.919567 28700 raft_consensus.cc:2804] T d5de0c782ce844ab8dbec004c32abafa P 96cba770f3334f79af3162475b54fa0f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:24:43.919947 28700 raft_consensus.cc:493] T d5de0c782ce844ab8dbec004c32abafa P 96cba770f3334f79af3162475b54fa0f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:24:43.920269 28700 raft_consensus.cc:3060] T d5de0c782ce844ab8dbec004c32abafa P 96cba770f3334f79af3162475b54fa0f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:24:43.929271 28690 raft_consensus.cc:359] T cbb4f686d0884c178ea6b01616da20e8 P 542ea9e2fcff4028b6766459e90bc5fc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "542ea9e2fcff4028b6766459e90bc5fc" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38409 } } peers { permanent_uuid: "96cba770f3334f79af3162475b54fa0f" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 33585 } } peers { permanent_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 37597 } }
I20260709 13:24:43.934607 28690 raft_consensus.cc:385] T cbb4f686d0884c178ea6b01616da20e8 P 542ea9e2fcff4028b6766459e90bc5fc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:24:43.935060 28690 raft_consensus.cc:740] T cbb4f686d0884c178ea6b01616da20e8 P 542ea9e2fcff4028b6766459e90bc5fc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 542ea9e2fcff4028b6766459e90bc5fc, State: Initialized, Role: FOLLOWER
I20260709 13:24:43.936234 28690 consensus_queue.cc:260] T cbb4f686d0884c178ea6b01616da20e8 P 542ea9e2fcff4028b6766459e90bc5fc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "542ea9e2fcff4028b6766459e90bc5fc" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38409 } } peers { permanent_uuid: "96cba770f3334f79af3162475b54fa0f" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 33585 } } peers { permanent_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 37597 } }
I20260709 13:24:43.902282 28704 raft_consensus.cc:515] T d489ac98a4654a7cb46771880f89b89b P 542ea9e2fcff4028b6766459e90bc5fc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "542ea9e2fcff4028b6766459e90bc5fc" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38409 } } peers { permanent_uuid: "96cba770f3334f79af3162475b54fa0f" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 33585 } } peers { permanent_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 37597 } }
I20260709 13:24:43.936589 28271 catalog_manager.cc:5697] T e4c7e579c94a4f5fa195def741ee4180 P 2173e3fd92f34ad6ad8d56d2e6b05299 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2173e3fd92f34ad6ad8d56d2e6b05299 (127.14.177.131). New cstate: current_term: 1 leader_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "542ea9e2fcff4028b6766459e90bc5fc" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38409 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "96cba770f3334f79af3162475b54fa0f" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 33585 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 37597 } health_report { overall_health: HEALTHY } } }
I20260709 13:24:43.937748 28372 raft_consensus.cc:2393] T d489ac98a4654a7cb46771880f89b89b P 542ea9e2fcff4028b6766459e90bc5fc [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 2173e3fd92f34ad6ad8d56d2e6b05299 in current term 1: Already voted for candidate 542ea9e2fcff4028b6766459e90bc5fc in this term.
I20260709 13:24:43.940706 28522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5de0c782ce844ab8dbec004c32abafa" candidate_uuid: "96cba770f3334f79af3162475b54fa0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" is_pre_election: true
I20260709 13:24:43.929425 28700 raft_consensus.cc:515] T d5de0c782ce844ab8dbec004c32abafa P 96cba770f3334f79af3162475b54fa0f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "542ea9e2fcff4028b6766459e90bc5fc" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38409 } } peers { permanent_uuid: "96cba770f3334f79af3162475b54fa0f" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 33585 } } peers { permanent_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 37597 } }
W20260709 13:24:43.942260 28415 leader_election.cc:343] T d5de0c782ce844ab8dbec004c32abafa P 96cba770f3334f79af3162475b54fa0f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2173e3fd92f34ad6ad8d56d2e6b05299 (127.14.177.131:37597): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:24:43.945395 28448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d489ac98a4654a7cb46771880f89b89b" candidate_uuid: "542ea9e2fcff4028b6766459e90bc5fc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96cba770f3334f79af3162475b54fa0f"
I20260709 13:24:43.947672 28372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5de0c782ce844ab8dbec004c32abafa" candidate_uuid: "96cba770f3334f79af3162475b54fa0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "542ea9e2fcff4028b6766459e90bc5fc"
I20260709 13:24:43.952656 28372 raft_consensus.cc:3060] T d5de0c782ce844ab8dbec004c32abafa P 542ea9e2fcff4028b6766459e90bc5fc [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:24:43.951891 28520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5de0c782ce844ab8dbec004c32abafa" candidate_uuid: "96cba770f3334f79af3162475b54fa0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299"
I20260709 13:24:43.954764 28523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d489ac98a4654a7cb46771880f89b89b" candidate_uuid: "542ea9e2fcff4028b6766459e90bc5fc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299"
I20260709 13:24:43.955662 28523 raft_consensus.cc:2393] T d489ac98a4654a7cb46771880f89b89b P 2173e3fd92f34ad6ad8d56d2e6b05299 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 542ea9e2fcff4028b6766459e90bc5fc in current term 1: Already voted for candidate 2173e3fd92f34ad6ad8d56d2e6b05299 in this term.
I20260709 13:24:43.957001 28340 leader_election.cc:304] T d489ac98a4654a7cb46771880f89b89b P 542ea9e2fcff4028b6766459e90bc5fc [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 542ea9e2fcff4028b6766459e90bc5fc; no voters: 2173e3fd92f34ad6ad8d56d2e6b05299, 96cba770f3334f79af3162475b54fa0f
I20260709 13:24:43.958024 28707 raft_consensus.cc:2749] T d489ac98a4654a7cb46771880f89b89b P 542ea9e2fcff4028b6766459e90bc5fc [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
W20260709 13:24:43.959072 28415 leader_election.cc:343] T d5de0c782ce844ab8dbec004c32abafa P 96cba770f3334f79af3162475b54fa0f [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 2173e3fd92f34ad6ad8d56d2e6b05299 (127.14.177.131:37597): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:24:43.950650 28700 leader_election.cc:290] T d5de0c782ce844ab8dbec004c32abafa P 96cba770f3334f79af3162475b54fa0f [CANDIDATE]: Term 1 election: Requested vote from peers 542ea9e2fcff4028b6766459e90bc5fc (127.14.177.129:38409), 2173e3fd92f34ad6ad8d56d2e6b05299 (127.14.177.131:37597)
I20260709 13:24:43.954555 28704 leader_election.cc:290] T d489ac98a4654a7cb46771880f89b89b P 542ea9e2fcff4028b6766459e90bc5fc [CANDIDATE]: Term 1 election: Requested vote from peers 96cba770f3334f79af3162475b54fa0f (127.14.177.130:33585), 2173e3fd92f34ad6ad8d56d2e6b05299 (127.14.177.131:37597)
I20260709 13:24:43.963263 28689 tablet_bootstrap.cc:492] T d5de0c782ce844ab8dbec004c32abafa P 2173e3fd92f34ad6ad8d56d2e6b05299: No bootstrap required, opened a new log
I20260709 13:24:43.963925 28689 ts_tablet_manager.cc:1403] T d5de0c782ce844ab8dbec004c32abafa P 2173e3fd92f34ad6ad8d56d2e6b05299: Time spent bootstrapping tablet: real 0.087s	user 0.039s	sys 0.028s
I20260709 13:24:43.965605 28688 tablet_bootstrap.cc:492] T 2b05204beeff49309ed4e0dd43b4be7f P 96cba770f3334f79af3162475b54fa0f: No bootstrap required, opened a new log
I20260709 13:24:43.966183 28688 ts_tablet_manager.cc:1403] T 2b05204beeff49309ed4e0dd43b4be7f P 96cba770f3334f79af3162475b54fa0f: Time spent bootstrapping tablet: real 0.062s	user 0.014s	sys 0.005s
I20260709 13:24:43.933943 28444 raft_consensus.cc:2468] T d489ac98a4654a7cb46771880f89b89b P 96cba770f3334f79af3162475b54fa0f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2173e3fd92f34ad6ad8d56d2e6b05299 in term 1.
I20260709 13:24:43.967417 28689 raft_consensus.cc:359] T d5de0c782ce844ab8dbec004c32abafa P 2173e3fd92f34ad6ad8d56d2e6b05299 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "542ea9e2fcff4028b6766459e90bc5fc" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38409 } } peers { permanent_uuid: "96cba770f3334f79af3162475b54fa0f" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 33585 } } peers { permanent_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 37597 } }
I20260709 13:24:43.968329 28689 raft_consensus.cc:385] T d5de0c782ce844ab8dbec004c32abafa P 2173e3fd92f34ad6ad8d56d2e6b05299 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:24:43.968708 28689 raft_consensus.cc:740] T d5de0c782ce844ab8dbec004c32abafa P 2173e3fd92f34ad6ad8d56d2e6b05299 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2173e3fd92f34ad6ad8d56d2e6b05299, State: Initialized, Role: FOLLOWER
I20260709 13:24:43.969247 28688 raft_consensus.cc:359] T 2b05204beeff49309ed4e0dd43b4be7f P 96cba770f3334f79af3162475b54fa0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "542ea9e2fcff4028b6766459e90bc5fc" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38409 } } peers { permanent_uuid: "96cba770f3334f79af3162475b54fa0f" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 33585 } } peers { permanent_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 37597 } }
I20260709 13:24:43.969982 28688 raft_consensus.cc:385] T 2b05204beeff49309ed4e0dd43b4be7f P 96cba770f3334f79af3162475b54fa0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:24:43.969554 28689 consensus_queue.cc:260] T d5de0c782ce844ab8dbec004c32abafa P 2173e3fd92f34ad6ad8d56d2e6b05299 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "542ea9e2fcff4028b6766459e90bc5fc" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38409 } } peers { permanent_uuid: "96cba770f3334f79af3162475b54fa0f" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 33585 } } peers { permanent_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 37597 } }
I20260709 13:24:43.970299 28688 raft_consensus.cc:740] T 2b05204beeff49309ed4e0dd43b4be7f P 96cba770f3334f79af3162475b54fa0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 96cba770f3334f79af3162475b54fa0f, State: Initialized, Role: FOLLOWER
I20260709 13:24:43.970624 28690 ts_tablet_manager.cc:1434] T cbb4f686d0884c178ea6b01616da20e8 P 542ea9e2fcff4028b6766459e90bc5fc: Time spent starting tablet: real 0.051s	user 0.006s	sys 0.000s
I20260709 13:24:43.971247 28688 consensus_queue.cc:260] T 2b05204beeff49309ed4e0dd43b4be7f P 96cba770f3334f79af3162475b54fa0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "542ea9e2fcff4028b6766459e90bc5fc" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38409 } } peers { permanent_uuid: "96cba770f3334f79af3162475b54fa0f" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 33585 } } peers { permanent_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 37597 } }
I20260709 13:24:43.978289 28688 ts_tablet_manager.cc:1434] T 2b05204beeff49309ed4e0dd43b4be7f P 96cba770f3334f79af3162475b54fa0f: Time spent starting tablet: real 0.012s	user 0.006s	sys 0.000s
I20260709 13:24:43.979912 28689 ts_tablet_manager.cc:1434] T d5de0c782ce844ab8dbec004c32abafa P 2173e3fd92f34ad6ad8d56d2e6b05299: Time spent starting tablet: real 0.016s	user 0.007s	sys 0.001s
I20260709 13:24:43.980376 28688 tablet_bootstrap.cc:492] T cbb4f686d0884c178ea6b01616da20e8 P 96cba770f3334f79af3162475b54fa0f: Bootstrap starting.
I20260709 13:24:43.980888 28704 raft_consensus.cc:493] T 07bb4101b0c24ee6b460e066cf78eb53 P 542ea9e2fcff4028b6766459e90bc5fc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:24:43.981204 28689 tablet_bootstrap.cc:492] T 94c1c4dff8e8404dbb00c79a7b7c69c0 P 2173e3fd92f34ad6ad8d56d2e6b05299: Bootstrap starting.
I20260709 13:24:43.981467 28704 raft_consensus.cc:515] T 07bb4101b0c24ee6b460e066cf78eb53 P 542ea9e2fcff4028b6766459e90bc5fc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "542ea9e2fcff4028b6766459e90bc5fc" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38409 } } peers { permanent_uuid: "96cba770f3334f79af3162475b54fa0f" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 33585 } } peers { permanent_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 37597 } }
I20260709 13:24:43.985608 28522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07bb4101b0c24ee6b460e066cf78eb53" candidate_uuid: "542ea9e2fcff4028b6766459e90bc5fc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" is_pre_election: true
I20260709 13:24:43.987102 28688 tablet_bootstrap.cc:654] T cbb4f686d0884c178ea6b01616da20e8 P 96cba770f3334f79af3162475b54fa0f: Neither blocks nor log segments found. Creating new log.
I20260709 13:24:43.987831 28700 raft_consensus.cc:493] T 07bb4101b0c24ee6b460e066cf78eb53 P 96cba770f3334f79af3162475b54fa0f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:24:43.969152 28489 leader_election.cc:304] T d489ac98a4654a7cb46771880f89b89b P 2173e3fd92f34ad6ad8d56d2e6b05299 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2173e3fd92f34ad6ad8d56d2e6b05299, 96cba770f3334f79af3162475b54fa0f; no voters: 542ea9e2fcff4028b6766459e90bc5fc
I20260709 13:24:43.988353 28700 raft_consensus.cc:515] T 07bb4101b0c24ee6b460e066cf78eb53 P 96cba770f3334f79af3162475b54fa0f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "542ea9e2fcff4028b6766459e90bc5fc" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38409 } } peers { permanent_uuid: "96cba770f3334f79af3162475b54fa0f" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 33585 } } peers { permanent_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 37597 } }
W20260709 13:24:43.988842 28340 leader_election.cc:343] T 07bb4101b0c24ee6b460e066cf78eb53 P 542ea9e2fcff4028b6766459e90bc5fc [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2173e3fd92f34ad6ad8d56d2e6b05299 (127.14.177.131:37597): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:24:43.989679 28702 raft_consensus.cc:2804] T d489ac98a4654a7cb46771880f89b89b P 2173e3fd92f34ad6ad8d56d2e6b05299 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:24:43.989900 28444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07bb4101b0c24ee6b460e066cf78eb53" candidate_uuid: "542ea9e2fcff4028b6766459e90bc5fc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96cba770f3334f79af3162475b54fa0f" is_pre_election: true
I20260709 13:24:43.990226 28702 raft_consensus.cc:697] T d489ac98a4654a7cb46771880f89b89b P 2173e3fd92f34ad6ad8d56d2e6b05299 [term 1 LEADER]: Becoming Leader. State: Replica: 2173e3fd92f34ad6ad8d56d2e6b05299, State: Running, Role: LEADER
I20260709 13:24:43.990566 28700 leader_election.cc:290] T 07bb4101b0c24ee6b460e066cf78eb53 P 96cba770f3334f79af3162475b54fa0f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 542ea9e2fcff4028b6766459e90bc5fc (127.14.177.129:38409), 2173e3fd92f34ad6ad8d56d2e6b05299 (127.14.177.131:37597)
I20260709 13:24:43.990666 28444 raft_consensus.cc:2468] T 07bb4101b0c24ee6b460e066cf78eb53 P 96cba770f3334f79af3162475b54fa0f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 542ea9e2fcff4028b6766459e90bc5fc in term 0.
I20260709 13:24:43.991132 28702 consensus_queue.cc:237] T d489ac98a4654a7cb46771880f89b89b P 2173e3fd92f34ad6ad8d56d2e6b05299 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "542ea9e2fcff4028b6766459e90bc5fc" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38409 } } peers { permanent_uuid: "96cba770f3334f79af3162475b54fa0f" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 33585 } } peers { permanent_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 37597 } }
I20260709 13:24:43.992273 28523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07bb4101b0c24ee6b460e066cf78eb53" candidate_uuid: "96cba770f3334f79af3162475b54fa0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" is_pre_election: true
I20260709 13:24:43.993614 28340 leader_election.cc:304] T 07bb4101b0c24ee6b460e066cf78eb53 P 542ea9e2fcff4028b6766459e90bc5fc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 542ea9e2fcff4028b6766459e90bc5fc, 96cba770f3334f79af3162475b54fa0f; no voters: 2173e3fd92f34ad6ad8d56d2e6b05299
W20260709 13:24:43.996111 28415 leader_election.cc:343] T 07bb4101b0c24ee6b460e066cf78eb53 P 96cba770f3334f79af3162475b54fa0f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2173e3fd92f34ad6ad8d56d2e6b05299 (127.14.177.131:37597): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:24:43.996440 28647 raft_consensus.cc:493] T 2b05204beeff49309ed4e0dd43b4be7f P 2173e3fd92f34ad6ad8d56d2e6b05299 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:24:43.996887 28707 raft_consensus.cc:2804] T 07bb4101b0c24ee6b460e066cf78eb53 P 542ea9e2fcff4028b6766459e90bc5fc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:24:43.997438 28707 raft_consensus.cc:493] T 07bb4101b0c24ee6b460e066cf78eb53 P 542ea9e2fcff4028b6766459e90bc5fc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:24:43.997048 28647 raft_consensus.cc:515] T 2b05204beeff49309ed4e0dd43b4be7f P 2173e3fd92f34ad6ad8d56d2e6b05299 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "542ea9e2fcff4028b6766459e90bc5fc" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38409 } } peers { permanent_uuid: "96cba770f3334f79af3162475b54fa0f" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 33585 } } peers { permanent_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 37597 } }
I20260709 13:24:43.997905 28707 raft_consensus.cc:3060] T 07bb4101b0c24ee6b460e066cf78eb53 P 542ea9e2fcff4028b6766459e90bc5fc [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:24:44.000048 28647 leader_election.cc:290] T 2b05204beeff49309ed4e0dd43b4be7f P 2173e3fd92f34ad6ad8d56d2e6b05299 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 542ea9e2fcff4028b6766459e90bc5fc (127.14.177.129:38409), 96cba770f3334f79af3162475b54fa0f (127.14.177.130:33585)
I20260709 13:24:44.004724 28370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b05204beeff49309ed4e0dd43b4be7f" candidate_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "542ea9e2fcff4028b6766459e90bc5fc" is_pre_election: true
I20260709 13:24:44.005651 28370 raft_consensus.cc:2468] T 2b05204beeff49309ed4e0dd43b4be7f P 542ea9e2fcff4028b6766459e90bc5fc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2173e3fd92f34ad6ad8d56d2e6b05299 in term 0.
I20260709 13:24:43.994861 28689 tablet_bootstrap.cc:654] T 94c1c4dff8e8404dbb00c79a7b7c69c0 P 2173e3fd92f34ad6ad8d56d2e6b05299: Neither blocks nor log segments found. Creating new log.
I20260709 13:24:44.007221 28487 leader_election.cc:304] T 2b05204beeff49309ed4e0dd43b4be7f P 2173e3fd92f34ad6ad8d56d2e6b05299 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2173e3fd92f34ad6ad8d56d2e6b05299, 542ea9e2fcff4028b6766459e90bc5fc; no voters: 
I20260709 13:24:44.008315 28647 raft_consensus.cc:2804] T 2b05204beeff49309ed4e0dd43b4be7f P 2173e3fd92f34ad6ad8d56d2e6b05299 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:24:44.008834 28647 raft_consensus.cc:493] T 2b05204beeff49309ed4e0dd43b4be7f P 2173e3fd92f34ad6ad8d56d2e6b05299 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:24:44.009289 28647 raft_consensus.cc:3060] T 2b05204beeff49309ed4e0dd43b4be7f P 2173e3fd92f34ad6ad8d56d2e6b05299 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:24:43.993198 28704 leader_election.cc:290] T 07bb4101b0c24ee6b460e066cf78eb53 P 542ea9e2fcff4028b6766459e90bc5fc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 96cba770f3334f79af3162475b54fa0f (127.14.177.130:33585), 2173e3fd92f34ad6ad8d56d2e6b05299 (127.14.177.131:37597)
I20260709 13:24:44.013819 28688 tablet_bootstrap.cc:492] T cbb4f686d0884c178ea6b01616da20e8 P 96cba770f3334f79af3162475b54fa0f: No bootstrap required, opened a new log
I20260709 13:24:44.014433 28688 ts_tablet_manager.cc:1403] T cbb4f686d0884c178ea6b01616da20e8 P 96cba770f3334f79af3162475b54fa0f: Time spent bootstrapping tablet: real 0.034s	user 0.014s	sys 0.004s
I20260709 13:24:43.994364 28371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07bb4101b0c24ee6b460e066cf78eb53" candidate_uuid: "96cba770f3334f79af3162475b54fa0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "542ea9e2fcff4028b6766459e90bc5fc" is_pre_election: true
I20260709 13:24:44.017241 28688 raft_consensus.cc:359] T cbb4f686d0884c178ea6b01616da20e8 P 96cba770f3334f79af3162475b54fa0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "542ea9e2fcff4028b6766459e90bc5fc" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38409 } } peers { permanent_uuid: "96cba770f3334f79af3162475b54fa0f" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 33585 } } peers { permanent_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 37597 } }
I20260709 13:24:44.018021 28688 raft_consensus.cc:385] T cbb4f686d0884c178ea6b01616da20e8 P 96cba770f3334f79af3162475b54fa0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:24:44.018314 28688 raft_consensus.cc:740] T cbb4f686d0884c178ea6b01616da20e8 P 96cba770f3334f79af3162475b54fa0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 96cba770f3334f79af3162475b54fa0f, State: Initialized, Role: FOLLOWER
I20260709 13:24:44.010001 28444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b05204beeff49309ed4e0dd43b4be7f" candidate_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96cba770f3334f79af3162475b54fa0f" is_pre_election: true
I20260709 13:24:44.019021 28688 consensus_queue.cc:260] T cbb4f686d0884c178ea6b01616da20e8 P 96cba770f3334f79af3162475b54fa0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "542ea9e2fcff4028b6766459e90bc5fc" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38409 } } peers { permanent_uuid: "96cba770f3334f79af3162475b54fa0f" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 33585 } } peers { permanent_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 37597 } }
I20260709 13:24:44.019686 28444 raft_consensus.cc:2468] T 2b05204beeff49309ed4e0dd43b4be7f P 96cba770f3334f79af3162475b54fa0f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2173e3fd92f34ad6ad8d56d2e6b05299 in term 0.
I20260709 13:24:44.021466 28688 ts_tablet_manager.cc:1434] T cbb4f686d0884c178ea6b01616da20e8 P 96cba770f3334f79af3162475b54fa0f: Time spent starting tablet: real 0.007s	user 0.002s	sys 0.003s
I20260709 13:24:44.026307 28707 raft_consensus.cc:515] T 07bb4101b0c24ee6b460e066cf78eb53 P 542ea9e2fcff4028b6766459e90bc5fc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "542ea9e2fcff4028b6766459e90bc5fc" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38409 } } peers { permanent_uuid: "96cba770f3334f79af3162475b54fa0f" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 33585 } } peers { permanent_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 37597 } }
I20260709 13:24:44.028357 28372 raft_consensus.cc:2468] T d5de0c782ce844ab8dbec004c32abafa P 542ea9e2fcff4028b6766459e90bc5fc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96cba770f3334f79af3162475b54fa0f in term 1.
I20260709 13:24:44.029619 28371 raft_consensus.cc:2393] T 07bb4101b0c24ee6b460e066cf78eb53 P 542ea9e2fcff4028b6766459e90bc5fc [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 96cba770f3334f79af3162475b54fa0f in current term 1: Already voted for candidate 542ea9e2fcff4028b6766459e90bc5fc in this term.
I20260709 13:24:44.030197 28413 leader_election.cc:304] T d5de0c782ce844ab8dbec004c32abafa P 96cba770f3334f79af3162475b54fa0f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 542ea9e2fcff4028b6766459e90bc5fc, 96cba770f3334f79af3162475b54fa0f; no voters: 2173e3fd92f34ad6ad8d56d2e6b05299
I20260709 13:24:44.032557 28523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07bb4101b0c24ee6b460e066cf78eb53" candidate_uuid: "542ea9e2fcff4028b6766459e90bc5fc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299"
I20260709 13:24:44.033470 28689 tablet_bootstrap.cc:492] T 94c1c4dff8e8404dbb00c79a7b7c69c0 P 2173e3fd92f34ad6ad8d56d2e6b05299: No bootstrap required, opened a new log
I20260709 13:24:44.033144 28647 raft_consensus.cc:515] T 2b05204beeff49309ed4e0dd43b4be7f P 2173e3fd92f34ad6ad8d56d2e6b05299 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "542ea9e2fcff4028b6766459e90bc5fc" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38409 } } peers { permanent_uuid: "96cba770f3334f79af3162475b54fa0f" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 33585 } } peers { permanent_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 37597 } }
I20260709 13:24:44.033710 28703 raft_consensus.cc:2804] T d5de0c782ce844ab8dbec004c32abafa P 96cba770f3334f79af3162475b54fa0f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:24:44.034113 28689 ts_tablet_manager.cc:1403] T 94c1c4dff8e8404dbb00c79a7b7c69c0 P 2173e3fd92f34ad6ad8d56d2e6b05299: Time spent bootstrapping tablet: real 0.053s	user 0.005s	sys 0.015s
I20260709 13:24:44.034293 28703 raft_consensus.cc:697] T d5de0c782ce844ab8dbec004c32abafa P 96cba770f3334f79af3162475b54fa0f [term 1 LEADER]: Becoming Leader. State: Replica: 96cba770f3334f79af3162475b54fa0f, State: Running, Role: LEADER
W20260709 13:24:44.034722 28340 leader_election.cc:343] T 07bb4101b0c24ee6b460e066cf78eb53 P 542ea9e2fcff4028b6766459e90bc5fc [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 2173e3fd92f34ad6ad8d56d2e6b05299 (127.14.177.131:37597): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:24:44.035624 28647 leader_election.cc:290] T 2b05204beeff49309ed4e0dd43b4be7f P 2173e3fd92f34ad6ad8d56d2e6b05299 [CANDIDATE]: Term 1 election: Requested vote from peers 542ea9e2fcff4028b6766459e90bc5fc (127.14.177.129:38409), 96cba770f3334f79af3162475b54fa0f (127.14.177.130:33585)
I20260709 13:24:44.035272 28703 consensus_queue.cc:237] T d5de0c782ce844ab8dbec004c32abafa P 96cba770f3334f79af3162475b54fa0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "542ea9e2fcff4028b6766459e90bc5fc" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38409 } } peers { permanent_uuid: "96cba770f3334f79af3162475b54fa0f" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 33585 } } peers { permanent_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 37597 } }
I20260709 13:24:44.030638 28707 leader_election.cc:290] T 07bb4101b0c24ee6b460e066cf78eb53 P 542ea9e2fcff4028b6766459e90bc5fc [CANDIDATE]: Term 1 election: Requested vote from peers 96cba770f3334f79af3162475b54fa0f (127.14.177.130:33585), 2173e3fd92f34ad6ad8d56d2e6b05299 (127.14.177.131:37597)
I20260709 13:24:44.037513 28448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b05204beeff49309ed4e0dd43b4be7f" candidate_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96cba770f3334f79af3162475b54fa0f"
I20260709 13:24:44.038196 28448 raft_consensus.cc:3060] T 2b05204beeff49309ed4e0dd43b4be7f P 96cba770f3334f79af3162475b54fa0f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:24:44.037690 28689 raft_consensus.cc:359] T 94c1c4dff8e8404dbb00c79a7b7c69c0 P 2173e3fd92f34ad6ad8d56d2e6b05299 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "542ea9e2fcff4028b6766459e90bc5fc" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38409 } } peers { permanent_uuid: "96cba770f3334f79af3162475b54fa0f" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 33585 } } peers { permanent_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 37597 } }
I20260709 13:24:44.038659 28689 raft_consensus.cc:385] T 94c1c4dff8e8404dbb00c79a7b7c69c0 P 2173e3fd92f34ad6ad8d56d2e6b05299 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:24:44.039027 28689 raft_consensus.cc:740] T 94c1c4dff8e8404dbb00c79a7b7c69c0 P 2173e3fd92f34ad6ad8d56d2e6b05299 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2173e3fd92f34ad6ad8d56d2e6b05299, State: Initialized, Role: FOLLOWER
I20260709 13:24:44.038782 28444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07bb4101b0c24ee6b460e066cf78eb53" candidate_uuid: "542ea9e2fcff4028b6766459e90bc5fc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96cba770f3334f79af3162475b54fa0f"
I20260709 13:24:44.039952 28444 raft_consensus.cc:3060] T 07bb4101b0c24ee6b460e066cf78eb53 P 96cba770f3334f79af3162475b54fa0f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:24:44.040082 28689 consensus_queue.cc:260] T 94c1c4dff8e8404dbb00c79a7b7c69c0 P 2173e3fd92f34ad6ad8d56d2e6b05299 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "542ea9e2fcff4028b6766459e90bc5fc" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38409 } } peers { permanent_uuid: "96cba770f3334f79af3162475b54fa0f" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 33585 } } peers { permanent_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 37597 } }
I20260709 13:24:44.040146 28413 leader_election.cc:304] T 07bb4101b0c24ee6b460e066cf78eb53 P 96cba770f3334f79af3162475b54fa0f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 96cba770f3334f79af3162475b54fa0f; no voters: 2173e3fd92f34ad6ad8d56d2e6b05299, 542ea9e2fcff4028b6766459e90bc5fc
I20260709 13:24:44.043401 28689 ts_tablet_manager.cc:1434] T 94c1c4dff8e8404dbb00c79a7b7c69c0 P 2173e3fd92f34ad6ad8d56d2e6b05299: Time spent starting tablet: real 0.009s	user 0.005s	sys 0.006s
I20260709 13:24:44.043768 28371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b05204beeff49309ed4e0dd43b4be7f" candidate_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "542ea9e2fcff4028b6766459e90bc5fc"
I20260709 13:24:44.044404 28371 raft_consensus.cc:3060] T 2b05204beeff49309ed4e0dd43b4be7f P 542ea9e2fcff4028b6766459e90bc5fc [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:24:44.044600 28689 tablet_bootstrap.cc:492] T cbb4f686d0884c178ea6b01616da20e8 P 2173e3fd92f34ad6ad8d56d2e6b05299: Bootstrap starting.
I20260709 13:24:44.051213 28448 raft_consensus.cc:2468] T 2b05204beeff49309ed4e0dd43b4be7f P 96cba770f3334f79af3162475b54fa0f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2173e3fd92f34ad6ad8d56d2e6b05299 in term 1.
I20260709 13:24:44.052197 28371 raft_consensus.cc:2468] T 2b05204beeff49309ed4e0dd43b4be7f P 542ea9e2fcff4028b6766459e90bc5fc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2173e3fd92f34ad6ad8d56d2e6b05299 in term 1.
I20260709 13:24:44.053815 28487 leader_election.cc:304] T 2b05204beeff49309ed4e0dd43b4be7f P 2173e3fd92f34ad6ad8d56d2e6b05299 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2173e3fd92f34ad6ad8d56d2e6b05299, 542ea9e2fcff4028b6766459e90bc5fc; no voters: 
I20260709 13:24:44.055670 28271 catalog_manager.cc:5697] T d489ac98a4654a7cb46771880f89b89b P 2173e3fd92f34ad6ad8d56d2e6b05299 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2173e3fd92f34ad6ad8d56d2e6b05299 (127.14.177.131). New cstate: current_term: 1 leader_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "542ea9e2fcff4028b6766459e90bc5fc" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38409 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "96cba770f3334f79af3162475b54fa0f" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 33585 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 37597 } health_report { overall_health: HEALTHY } } }
I20260709 13:24:44.056838 28647 raft_consensus.cc:2804] T 2b05204beeff49309ed4e0dd43b4be7f P 2173e3fd92f34ad6ad8d56d2e6b05299 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:24:44.057868 28647 raft_consensus.cc:697] T 2b05204beeff49309ed4e0dd43b4be7f P 2173e3fd92f34ad6ad8d56d2e6b05299 [term 1 LEADER]: Becoming Leader. State: Replica: 2173e3fd92f34ad6ad8d56d2e6b05299, State: Running, Role: LEADER
I20260709 13:24:44.058984 28647 consensus_queue.cc:237] T 2b05204beeff49309ed4e0dd43b4be7f P 2173e3fd92f34ad6ad8d56d2e6b05299 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "542ea9e2fcff4028b6766459e90bc5fc" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38409 } } peers { permanent_uuid: "96cba770f3334f79af3162475b54fa0f" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 33585 } } peers { permanent_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 37597 } }
I20260709 13:24:44.061091 28444 raft_consensus.cc:2468] T 07bb4101b0c24ee6b460e066cf78eb53 P 96cba770f3334f79af3162475b54fa0f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 542ea9e2fcff4028b6766459e90bc5fc in term 1.
I20260709 13:24:44.062330 28700 raft_consensus.cc:2749] T 07bb4101b0c24ee6b460e066cf78eb53 P 96cba770f3334f79af3162475b54fa0f [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:24:44.062263 28340 leader_election.cc:304] T 07bb4101b0c24ee6b460e066cf78eb53 P 542ea9e2fcff4028b6766459e90bc5fc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 542ea9e2fcff4028b6766459e90bc5fc, 96cba770f3334f79af3162475b54fa0f; no voters: 2173e3fd92f34ad6ad8d56d2e6b05299
I20260709 13:24:44.062743 28689 tablet_bootstrap.cc:654] T cbb4f686d0884c178ea6b01616da20e8 P 2173e3fd92f34ad6ad8d56d2e6b05299: Neither blocks nor log segments found. Creating new log.
I20260709 13:24:44.063593 28704 raft_consensus.cc:2804] T 07bb4101b0c24ee6b460e066cf78eb53 P 542ea9e2fcff4028b6766459e90bc5fc [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:24:44.064109 28704 raft_consensus.cc:697] T 07bb4101b0c24ee6b460e066cf78eb53 P 542ea9e2fcff4028b6766459e90bc5fc [term 1 LEADER]: Becoming Leader. State: Replica: 542ea9e2fcff4028b6766459e90bc5fc, State: Running, Role: LEADER
I20260709 13:24:44.064922 28704 consensus_queue.cc:237] T 07bb4101b0c24ee6b460e066cf78eb53 P 542ea9e2fcff4028b6766459e90bc5fc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "542ea9e2fcff4028b6766459e90bc5fc" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38409 } } peers { permanent_uuid: "96cba770f3334f79af3162475b54fa0f" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 33585 } } peers { permanent_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 37597 } }
I20260709 13:24:44.066864 28269 catalog_manager.cc:5697] T d5de0c782ce844ab8dbec004c32abafa P 96cba770f3334f79af3162475b54fa0f reported cstate change: term changed from 0 to 1, leader changed from <none> to 96cba770f3334f79af3162475b54fa0f (127.14.177.130). New cstate: current_term: 1 leader_uuid: "96cba770f3334f79af3162475b54fa0f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "542ea9e2fcff4028b6766459e90bc5fc" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38409 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "96cba770f3334f79af3162475b54fa0f" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 33585 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 37597 } health_report { overall_health: UNKNOWN } } }
I20260709 13:24:44.075488 28700 consensus_queue.cc:1048] T fbb3d76b1491485287e0dc48b24dbb46 P 96cba770f3334f79af3162475b54fa0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 37597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:24:44.094240 28707 consensus_queue.cc:1048] T 94c1c4dff8e8404dbb00c79a7b7c69c0 P 542ea9e2fcff4028b6766459e90bc5fc [LEADER]: Connected to new peer: Peer: permanent_uuid: "96cba770f3334f79af3162475b54fa0f" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 33585 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:24:44.098671 28700 raft_consensus.cc:493] T cbb4f686d0884c178ea6b01616da20e8 P 96cba770f3334f79af3162475b54fa0f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:24:44.099480 28700 raft_consensus.cc:515] T cbb4f686d0884c178ea6b01616da20e8 P 96cba770f3334f79af3162475b54fa0f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "542ea9e2fcff4028b6766459e90bc5fc" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38409 } } peers { permanent_uuid: "96cba770f3334f79af3162475b54fa0f" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 33585 } } peers { permanent_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 37597 } }
I20260709 13:24:44.102329 28700 leader_election.cc:290] T cbb4f686d0884c178ea6b01616da20e8 P 96cba770f3334f79af3162475b54fa0f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 542ea9e2fcff4028b6766459e90bc5fc (127.14.177.129:38409), 2173e3fd92f34ad6ad8d56d2e6b05299 (127.14.177.131:37597)
I20260709 13:24:44.106545 28371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbb4f686d0884c178ea6b01616da20e8" candidate_uuid: "96cba770f3334f79af3162475b54fa0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "542ea9e2fcff4028b6766459e90bc5fc" is_pre_election: true
I20260709 13:24:44.106856 28523 raft_consensus.cc:3060] T 94c1c4dff8e8404dbb00c79a7b7c69c0 P 2173e3fd92f34ad6ad8d56d2e6b05299 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:24:44.106676 28520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbb4f686d0884c178ea6b01616da20e8" candidate_uuid: "96cba770f3334f79af3162475b54fa0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" is_pre_election: true
I20260709 13:24:44.107360 28371 raft_consensus.cc:2468] T cbb4f686d0884c178ea6b01616da20e8 P 542ea9e2fcff4028b6766459e90bc5fc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96cba770f3334f79af3162475b54fa0f in term 0.
W20260709 13:24:44.109407 28415 leader_election.cc:343] T cbb4f686d0884c178ea6b01616da20e8 P 96cba770f3334f79af3162475b54fa0f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2173e3fd92f34ad6ad8d56d2e6b05299 (127.14.177.131:37597): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:24:44.112200 28689 tablet_bootstrap.cc:492] T cbb4f686d0884c178ea6b01616da20e8 P 2173e3fd92f34ad6ad8d56d2e6b05299: No bootstrap required, opened a new log
I20260709 13:24:44.112768 28689 ts_tablet_manager.cc:1403] T cbb4f686d0884c178ea6b01616da20e8 P 2173e3fd92f34ad6ad8d56d2e6b05299: Time spent bootstrapping tablet: real 0.068s	user 0.018s	sys 0.009s
I20260709 13:24:44.118745 28269 catalog_manager.cc:5697] T 07bb4101b0c24ee6b460e066cf78eb53 P 542ea9e2fcff4028b6766459e90bc5fc reported cstate change: term changed from 0 to 1, leader changed from <none> to 542ea9e2fcff4028b6766459e90bc5fc (127.14.177.129). New cstate: current_term: 1 leader_uuid: "542ea9e2fcff4028b6766459e90bc5fc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "542ea9e2fcff4028b6766459e90bc5fc" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38409 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "96cba770f3334f79af3162475b54fa0f" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 33585 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 37597 } health_report { overall_health: UNKNOWN } } }
I20260709 13:24:44.116662 28413 leader_election.cc:304] T cbb4f686d0884c178ea6b01616da20e8 P 96cba770f3334f79af3162475b54fa0f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 542ea9e2fcff4028b6766459e90bc5fc, 96cba770f3334f79af3162475b54fa0f; no voters: 2173e3fd92f34ad6ad8d56d2e6b05299
I20260709 13:24:44.123852 28700 raft_consensus.cc:2804] T cbb4f686d0884c178ea6b01616da20e8 P 96cba770f3334f79af3162475b54fa0f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:24:44.124528 28700 raft_consensus.cc:493] T cbb4f686d0884c178ea6b01616da20e8 P 96cba770f3334f79af3162475b54fa0f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:24:44.125058 28700 raft_consensus.cc:3060] T cbb4f686d0884c178ea6b01616da20e8 P 96cba770f3334f79af3162475b54fa0f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:24:44.123230 28271 catalog_manager.cc:5697] T 2b05204beeff49309ed4e0dd43b4be7f P 2173e3fd92f34ad6ad8d56d2e6b05299 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2173e3fd92f34ad6ad8d56d2e6b05299 (127.14.177.131). New cstate: current_term: 1 leader_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "542ea9e2fcff4028b6766459e90bc5fc" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38409 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "96cba770f3334f79af3162475b54fa0f" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 33585 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 37597 } health_report { overall_health: HEALTHY } } }
I20260709 13:24:44.131786 28647 raft_consensus.cc:493] T 94c1c4dff8e8404dbb00c79a7b7c69c0 P 2173e3fd92f34ad6ad8d56d2e6b05299 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:24:44.132364 28647 raft_consensus.cc:515] T 94c1c4dff8e8404dbb00c79a7b7c69c0 P 2173e3fd92f34ad6ad8d56d2e6b05299 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "542ea9e2fcff4028b6766459e90bc5fc" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38409 } } peers { permanent_uuid: "96cba770f3334f79af3162475b54fa0f" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 33585 } } peers { permanent_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 37597 } }
I20260709 13:24:44.133402 28735 consensus_queue.cc:1048] T 94c1c4dff8e8404dbb00c79a7b7c69c0 P 542ea9e2fcff4028b6766459e90bc5fc [LEADER]: Connected to new peer: Peer: permanent_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 37597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:24:44.136508 28647 leader_election.cc:290] T 94c1c4dff8e8404dbb00c79a7b7c69c0 P 2173e3fd92f34ad6ad8d56d2e6b05299 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 542ea9e2fcff4028b6766459e90bc5fc (127.14.177.129:38409), 96cba770f3334f79af3162475b54fa0f (127.14.177.130:33585)
I20260709 13:24:44.137712 28448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94c1c4dff8e8404dbb00c79a7b7c69c0" candidate_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96cba770f3334f79af3162475b54fa0f" is_pre_election: true
I20260709 13:24:44.140991 28372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94c1c4dff8e8404dbb00c79a7b7c69c0" candidate_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "542ea9e2fcff4028b6766459e90bc5fc" is_pre_election: true
I20260709 13:24:44.142395 28487 leader_election.cc:304] T 94c1c4dff8e8404dbb00c79a7b7c69c0 P 2173e3fd92f34ad6ad8d56d2e6b05299 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2173e3fd92f34ad6ad8d56d2e6b05299; no voters: 542ea9e2fcff4028b6766459e90bc5fc, 96cba770f3334f79af3162475b54fa0f
I20260709 13:24:44.142627 28689 raft_consensus.cc:359] T cbb4f686d0884c178ea6b01616da20e8 P 2173e3fd92f34ad6ad8d56d2e6b05299 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "542ea9e2fcff4028b6766459e90bc5fc" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38409 } } peers { permanent_uuid: "96cba770f3334f79af3162475b54fa0f" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 33585 } } peers { permanent_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 37597 } }
I20260709 13:24:44.143409 28689 raft_consensus.cc:385] T cbb4f686d0884c178ea6b01616da20e8 P 2173e3fd92f34ad6ad8d56d2e6b05299 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:24:44.143728 28689 raft_consensus.cc:740] T cbb4f686d0884c178ea6b01616da20e8 P 2173e3fd92f34ad6ad8d56d2e6b05299 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2173e3fd92f34ad6ad8d56d2e6b05299, State: Initialized, Role: FOLLOWER
I20260709 13:24:44.144078 28647 raft_consensus.cc:2749] T 94c1c4dff8e8404dbb00c79a7b7c69c0 P 2173e3fd92f34ad6ad8d56d2e6b05299 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:24:44.144459 28689 consensus_queue.cc:260] T cbb4f686d0884c178ea6b01616da20e8 P 2173e3fd92f34ad6ad8d56d2e6b05299 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "542ea9e2fcff4028b6766459e90bc5fc" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38409 } } peers { permanent_uuid: "96cba770f3334f79af3162475b54fa0f" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 33585 } } peers { permanent_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 37597 } }
I20260709 13:24:44.138978 28700 raft_consensus.cc:515] T cbb4f686d0884c178ea6b01616da20e8 P 96cba770f3334f79af3162475b54fa0f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "542ea9e2fcff4028b6766459e90bc5fc" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38409 } } peers { permanent_uuid: "96cba770f3334f79af3162475b54fa0f" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 33585 } } peers { permanent_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 37597 } }
I20260709 13:24:44.148947 28520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbb4f686d0884c178ea6b01616da20e8" candidate_uuid: "96cba770f3334f79af3162475b54fa0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299"
I20260709 13:24:44.149351 28371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbb4f686d0884c178ea6b01616da20e8" candidate_uuid: "96cba770f3334f79af3162475b54fa0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "542ea9e2fcff4028b6766459e90bc5fc"
I20260709 13:24:44.149638 28520 raft_consensus.cc:3060] T cbb4f686d0884c178ea6b01616da20e8 P 2173e3fd92f34ad6ad8d56d2e6b05299 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:24:44.150096 28371 raft_consensus.cc:3060] T cbb4f686d0884c178ea6b01616da20e8 P 542ea9e2fcff4028b6766459e90bc5fc [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:24:44.156034 28689 ts_tablet_manager.cc:1434] T cbb4f686d0884c178ea6b01616da20e8 P 2173e3fd92f34ad6ad8d56d2e6b05299: Time spent starting tablet: real 0.043s	user 0.006s	sys 0.002s
I20260709 13:24:44.148730 28700 leader_election.cc:290] T cbb4f686d0884c178ea6b01616da20e8 P 96cba770f3334f79af3162475b54fa0f [CANDIDATE]: Term 1 election: Requested vote from peers 542ea9e2fcff4028b6766459e90bc5fc (127.14.177.129:38409), 2173e3fd92f34ad6ad8d56d2e6b05299 (127.14.177.131:37597)
I20260709 13:24:44.157261 28689 tablet_bootstrap.cc:492] T 07bb4101b0c24ee6b460e066cf78eb53 P 2173e3fd92f34ad6ad8d56d2e6b05299: Bootstrap starting.
I20260709 13:24:44.160917 28371 raft_consensus.cc:2468] T cbb4f686d0884c178ea6b01616da20e8 P 542ea9e2fcff4028b6766459e90bc5fc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96cba770f3334f79af3162475b54fa0f in term 1.
I20260709 13:24:44.161646 28520 raft_consensus.cc:2468] T cbb4f686d0884c178ea6b01616da20e8 P 2173e3fd92f34ad6ad8d56d2e6b05299 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96cba770f3334f79af3162475b54fa0f in term 1.
I20260709 13:24:44.162377 28413 leader_election.cc:304] T cbb4f686d0884c178ea6b01616da20e8 P 96cba770f3334f79af3162475b54fa0f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 542ea9e2fcff4028b6766459e90bc5fc, 96cba770f3334f79af3162475b54fa0f; no voters: 
I20260709 13:24:44.166298 28668 raft_consensus.cc:2804] T cbb4f686d0884c178ea6b01616da20e8 P 96cba770f3334f79af3162475b54fa0f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:24:44.164826 28689 tablet_bootstrap.cc:654] T 07bb4101b0c24ee6b460e066cf78eb53 P 2173e3fd92f34ad6ad8d56d2e6b05299: Neither blocks nor log segments found. Creating new log.
I20260709 13:24:44.203733 28668 raft_consensus.cc:697] T cbb4f686d0884c178ea6b01616da20e8 P 96cba770f3334f79af3162475b54fa0f [term 1 LEADER]: Becoming Leader. State: Replica: 96cba770f3334f79af3162475b54fa0f, State: Running, Role: LEADER
I20260709 13:24:44.205145 28668 consensus_queue.cc:237] T cbb4f686d0884c178ea6b01616da20e8 P 96cba770f3334f79af3162475b54fa0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "542ea9e2fcff4028b6766459e90bc5fc" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38409 } } peers { permanent_uuid: "96cba770f3334f79af3162475b54fa0f" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 33585 } } peers { permanent_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 37597 } }
I20260709 13:24:44.212646 28689 tablet_bootstrap.cc:492] T 07bb4101b0c24ee6b460e066cf78eb53 P 2173e3fd92f34ad6ad8d56d2e6b05299: No bootstrap required, opened a new log
I20260709 13:24:44.213548 28689 ts_tablet_manager.cc:1403] T 07bb4101b0c24ee6b460e066cf78eb53 P 2173e3fd92f34ad6ad8d56d2e6b05299: Time spent bootstrapping tablet: real 0.057s	user 0.012s	sys 0.013s
I20260709 13:24:44.216508 28689 raft_consensus.cc:359] T 07bb4101b0c24ee6b460e066cf78eb53 P 2173e3fd92f34ad6ad8d56d2e6b05299 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "542ea9e2fcff4028b6766459e90bc5fc" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38409 } } peers { permanent_uuid: "96cba770f3334f79af3162475b54fa0f" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 33585 } } peers { permanent_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 37597 } }
I20260709 13:24:44.220782 28689 raft_consensus.cc:385] T 07bb4101b0c24ee6b460e066cf78eb53 P 2173e3fd92f34ad6ad8d56d2e6b05299 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:24:44.221211 28689 raft_consensus.cc:740] T 07bb4101b0c24ee6b460e066cf78eb53 P 2173e3fd92f34ad6ad8d56d2e6b05299 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2173e3fd92f34ad6ad8d56d2e6b05299, State: Initialized, Role: FOLLOWER
I20260709 13:24:44.222018 28689 consensus_queue.cc:260] T 07bb4101b0c24ee6b460e066cf78eb53 P 2173e3fd92f34ad6ad8d56d2e6b05299 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "542ea9e2fcff4028b6766459e90bc5fc" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38409 } } peers { permanent_uuid: "96cba770f3334f79af3162475b54fa0f" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 33585 } } peers { permanent_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 37597 } }
I20260709 13:24:44.224241 28689 ts_tablet_manager.cc:1434] T 07bb4101b0c24ee6b460e066cf78eb53 P 2173e3fd92f34ad6ad8d56d2e6b05299: Time spent starting tablet: real 0.010s	user 0.005s	sys 0.001s
I20260709 13:24:44.229331 28272 catalog_manager.cc:5697] T cbb4f686d0884c178ea6b01616da20e8 P 96cba770f3334f79af3162475b54fa0f reported cstate change: term changed from 0 to 1, leader changed from <none> to 96cba770f3334f79af3162475b54fa0f (127.14.177.130). New cstate: current_term: 1 leader_uuid: "96cba770f3334f79af3162475b54fa0f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "542ea9e2fcff4028b6766459e90bc5fc" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38409 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "96cba770f3334f79af3162475b54fa0f" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 33585 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 37597 } health_report { overall_health: UNKNOWN } } }
I20260709 13:24:44.246965 28703 consensus_queue.cc:1048] T fbb3d76b1491485287e0dc48b24dbb46 P 96cba770f3334f79af3162475b54fa0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "542ea9e2fcff4028b6766459e90bc5fc" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38409 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:24:44.324983 28647 raft_consensus.cc:493] T 07bb4101b0c24ee6b460e066cf78eb53 P 2173e3fd92f34ad6ad8d56d2e6b05299 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:24:44.325568 28647 raft_consensus.cc:515] T 07bb4101b0c24ee6b460e066cf78eb53 P 2173e3fd92f34ad6ad8d56d2e6b05299 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "542ea9e2fcff4028b6766459e90bc5fc" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38409 } } peers { permanent_uuid: "96cba770f3334f79af3162475b54fa0f" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 33585 } } peers { permanent_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 37597 } }
I20260709 13:24:44.327405 28647 leader_election.cc:290] T 07bb4101b0c24ee6b460e066cf78eb53 P 2173e3fd92f34ad6ad8d56d2e6b05299 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 542ea9e2fcff4028b6766459e90bc5fc (127.14.177.129:38409), 96cba770f3334f79af3162475b54fa0f (127.14.177.130:33585)
I20260709 13:24:44.328279 28371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07bb4101b0c24ee6b460e066cf78eb53" candidate_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "542ea9e2fcff4028b6766459e90bc5fc" is_pre_election: true
I20260709 13:24:44.328637 28444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07bb4101b0c24ee6b460e066cf78eb53" candidate_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96cba770f3334f79af3162475b54fa0f" is_pre_election: true
I20260709 13:24:44.329406 28444 raft_consensus.cc:2393] T 07bb4101b0c24ee6b460e066cf78eb53 P 96cba770f3334f79af3162475b54fa0f [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2173e3fd92f34ad6ad8d56d2e6b05299 in current term 1: Already voted for candidate 542ea9e2fcff4028b6766459e90bc5fc in this term.
I20260709 13:24:44.330389 28489 leader_election.cc:304] T 07bb4101b0c24ee6b460e066cf78eb53 P 2173e3fd92f34ad6ad8d56d2e6b05299 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2173e3fd92f34ad6ad8d56d2e6b05299; no voters: 542ea9e2fcff4028b6766459e90bc5fc, 96cba770f3334f79af3162475b54fa0f
I20260709 13:24:44.331204 28647 raft_consensus.cc:3060] T 07bb4101b0c24ee6b460e066cf78eb53 P 2173e3fd92f34ad6ad8d56d2e6b05299 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:24:44.337273 28647 raft_consensus.cc:2749] T 07bb4101b0c24ee6b460e066cf78eb53 P 2173e3fd92f34ad6ad8d56d2e6b05299 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:24:44.340919 28647 consensus_queue.cc:1048] T e4c7e579c94a4f5fa195def741ee4180 P 2173e3fd92f34ad6ad8d56d2e6b05299 [LEADER]: Connected to new peer: Peer: permanent_uuid: "542ea9e2fcff4028b6766459e90bc5fc" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38409 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:24:44.356805 28702 consensus_queue.cc:1048] T e4c7e579c94a4f5fa195def741ee4180 P 2173e3fd92f34ad6ad8d56d2e6b05299 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96cba770f3334f79af3162475b54fa0f" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 33585 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:24:44.394843 28735 consensus_queue.cc:1048] T 6efeed092ec840f2a7748c4a613dd6a7 P 542ea9e2fcff4028b6766459e90bc5fc [LEADER]: Connected to new peer: Peer: permanent_uuid: "96cba770f3334f79af3162475b54fa0f" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 33585 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:24:44.405678 28619 consensus_queue.cc:1048] T 6efeed092ec840f2a7748c4a613dd6a7 P 542ea9e2fcff4028b6766459e90bc5fc [LEADER]: Connected to new peer: Peer: permanent_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 37597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:24:44.414052 28702 raft_consensus.cc:493] T d5de0c782ce844ab8dbec004c32abafa P 2173e3fd92f34ad6ad8d56d2e6b05299 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:24:44.414575 28702 raft_consensus.cc:515] T d5de0c782ce844ab8dbec004c32abafa P 2173e3fd92f34ad6ad8d56d2e6b05299 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "542ea9e2fcff4028b6766459e90bc5fc" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38409 } } peers { permanent_uuid: "96cba770f3334f79af3162475b54fa0f" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 33585 } } peers { permanent_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 37597 } }
I20260709 13:24:44.416740 28702 leader_election.cc:290] T d5de0c782ce844ab8dbec004c32abafa P 2173e3fd92f34ad6ad8d56d2e6b05299 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 542ea9e2fcff4028b6766459e90bc5fc (127.14.177.129:38409), 96cba770f3334f79af3162475b54fa0f (127.14.177.130:33585)
I20260709 13:24:44.418711 28371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5de0c782ce844ab8dbec004c32abafa" candidate_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "542ea9e2fcff4028b6766459e90bc5fc" is_pre_election: true
I20260709 13:24:44.419723 28371 raft_consensus.cc:2393] T d5de0c782ce844ab8dbec004c32abafa P 542ea9e2fcff4028b6766459e90bc5fc [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2173e3fd92f34ad6ad8d56d2e6b05299 in current term 1: Already voted for candidate 96cba770f3334f79af3162475b54fa0f in this term.
I20260709 13:24:44.418711 28448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5de0c782ce844ab8dbec004c32abafa" candidate_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96cba770f3334f79af3162475b54fa0f" is_pre_election: true
I20260709 13:24:44.423516 28489 leader_election.cc:304] T d5de0c782ce844ab8dbec004c32abafa P 2173e3fd92f34ad6ad8d56d2e6b05299 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2173e3fd92f34ad6ad8d56d2e6b05299; no voters: 542ea9e2fcff4028b6766459e90bc5fc, 96cba770f3334f79af3162475b54fa0f
I20260709 13:24:44.424719 28702 raft_consensus.cc:3060] T d5de0c782ce844ab8dbec004c32abafa P 2173e3fd92f34ad6ad8d56d2e6b05299 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:24:44.430719 28702 raft_consensus.cc:2749] T d5de0c782ce844ab8dbec004c32abafa P 2173e3fd92f34ad6ad8d56d2e6b05299 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:24:44.436128 28703 consensus_queue.cc:1048] T d5de0c782ce844ab8dbec004c32abafa P 96cba770f3334f79af3162475b54fa0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 37597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:24:44.444726 28702 consensus_queue.cc:1048] T d489ac98a4654a7cb46771880f89b89b P 2173e3fd92f34ad6ad8d56d2e6b05299 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96cba770f3334f79af3162475b54fa0f" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 33585 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:24:44.466735 28703 consensus_queue.cc:1048] T d5de0c782ce844ab8dbec004c32abafa P 96cba770f3334f79af3162475b54fa0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "542ea9e2fcff4028b6766459e90bc5fc" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38409 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:24:44.482728 28619 consensus_queue.cc:1048] T 07bb4101b0c24ee6b460e066cf78eb53 P 542ea9e2fcff4028b6766459e90bc5fc [LEADER]: Connected to new peer: Peer: permanent_uuid: "96cba770f3334f79af3162475b54fa0f" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 33585 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:24:44.485011 28702 consensus_queue.cc:1048] T d489ac98a4654a7cb46771880f89b89b P 2173e3fd92f34ad6ad8d56d2e6b05299 [LEADER]: Connected to new peer: Peer: permanent_uuid: "542ea9e2fcff4028b6766459e90bc5fc" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38409 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:24:44.517220 28707 consensus_queue.cc:1048] T 07bb4101b0c24ee6b460e066cf78eb53 P 542ea9e2fcff4028b6766459e90bc5fc [LEADER]: Connected to new peer: Peer: permanent_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 37597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:24:44.523974 28647 consensus_queue.cc:1048] T 2b05204beeff49309ed4e0dd43b4be7f P 2173e3fd92f34ad6ad8d56d2e6b05299 [LEADER]: Connected to new peer: Peer: permanent_uuid: "542ea9e2fcff4028b6766459e90bc5fc" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38409 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:24:44.531597 28520 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e4c7e579c94a4f5fa195def741ee4180"
dest_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299"
mode: GRACEFUL
new_leader_uuid: "542ea9e2fcff4028b6766459e90bc5fc"
 from {username='slave'} at 127.0.0.1:53728
I20260709 13:24:44.532311 28520 raft_consensus.cc:606] T e4c7e579c94a4f5fa195def741ee4180 P 2173e3fd92f34ad6ad8d56d2e6b05299 [term 1 LEADER]: Received request to transfer leadership to TS 542ea9e2fcff4028b6766459e90bc5fc
I20260709 13:24:44.539456 28444 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "fbb3d76b1491485287e0dc48b24dbb46"
dest_uuid: "96cba770f3334f79af3162475b54fa0f"
mode: GRACEFUL
new_leader_uuid: "542ea9e2fcff4028b6766459e90bc5fc"
 from {username='slave'} at 127.0.0.1:34650
I20260709 13:24:44.540351 28444 raft_consensus.cc:606] T fbb3d76b1491485287e0dc48b24dbb46 P 96cba770f3334f79af3162475b54fa0f [term 1 LEADER]: Received request to transfer leadership to TS 542ea9e2fcff4028b6766459e90bc5fc
I20260709 13:24:44.547231 28520 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d489ac98a4654a7cb46771880f89b89b"
dest_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299"
mode: GRACEFUL
new_leader_uuid: "542ea9e2fcff4028b6766459e90bc5fc"
 from {username='slave'} at 127.0.0.1:53728
I20260709 13:24:44.547818 28520 raft_consensus.cc:606] T d489ac98a4654a7cb46771880f89b89b P 2173e3fd92f34ad6ad8d56d2e6b05299 [term 1 LEADER]: Received request to transfer leadership to TS 542ea9e2fcff4028b6766459e90bc5fc
I20260709 13:24:44.564734 28754 consensus_queue.cc:1048] T 2b05204beeff49309ed4e0dd43b4be7f P 2173e3fd92f34ad6ad8d56d2e6b05299 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96cba770f3334f79af3162475b54fa0f" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 33585 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:24:44.565716 28520 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2b05204beeff49309ed4e0dd43b4be7f"
dest_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299"
mode: GRACEFUL
new_leader_uuid: "96cba770f3334f79af3162475b54fa0f"
 from {username='slave'} at 127.0.0.1:53728
I20260709 13:24:44.577737 28520 raft_consensus.cc:606] T 2b05204beeff49309ed4e0dd43b4be7f P 2173e3fd92f34ad6ad8d56d2e6b05299 [term 1 LEADER]: Received request to transfer leadership to TS 96cba770f3334f79af3162475b54fa0f
I20260709 13:24:44.586186 28371 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6efeed092ec840f2a7748c4a613dd6a7"
dest_uuid: "542ea9e2fcff4028b6766459e90bc5fc"
mode: GRACEFUL
new_leader_uuid: "96cba770f3334f79af3162475b54fa0f"
 from {username='slave'} at 127.0.0.1:59726
I20260709 13:24:44.586856 28371 raft_consensus.cc:606] T 6efeed092ec840f2a7748c4a613dd6a7 P 542ea9e2fcff4028b6766459e90bc5fc [term 1 LEADER]: Received request to transfer leadership to TS 96cba770f3334f79af3162475b54fa0f
I20260709 13:24:44.587353 28754 raft_consensus.cc:993] T 2b05204beeff49309ed4e0dd43b4be7f P 2173e3fd92f34ad6ad8d56d2e6b05299: : Instructing follower 96cba770f3334f79af3162475b54fa0f to start an election
I20260709 13:24:44.587925 28647 raft_consensus.cc:1081] T 2b05204beeff49309ed4e0dd43b4be7f P 2173e3fd92f34ad6ad8d56d2e6b05299 [term 1 LEADER]: Signalling peer 96cba770f3334f79af3162475b54fa0f to start an election
I20260709 13:24:44.590324 28444 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "2b05204beeff49309ed4e0dd43b4be7f"
dest_uuid: "96cba770f3334f79af3162475b54fa0f"
 from {username='slave'} at 127.0.0.1:34622
I20260709 13:24:44.590900 28444 raft_consensus.cc:493] T 2b05204beeff49309ed4e0dd43b4be7f P 96cba770f3334f79af3162475b54fa0f [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:24:44.591300 28444 raft_consensus.cc:3060] T 2b05204beeff49309ed4e0dd43b4be7f P 96cba770f3334f79af3162475b54fa0f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:24:44.599546 28739 consensus_queue.cc:1048] T cbb4f686d0884c178ea6b01616da20e8 P 96cba770f3334f79af3162475b54fa0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 37597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:24:44.599769 28444 raft_consensus.cc:515] T 2b05204beeff49309ed4e0dd43b4be7f P 96cba770f3334f79af3162475b54fa0f [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "542ea9e2fcff4028b6766459e90bc5fc" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38409 } } peers { permanent_uuid: "96cba770f3334f79af3162475b54fa0f" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 33585 } } peers { permanent_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 37597 } }
I20260709 13:24:44.602569 28371 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "07bb4101b0c24ee6b460e066cf78eb53"
dest_uuid: "542ea9e2fcff4028b6766459e90bc5fc"
mode: GRACEFUL
new_leader_uuid: "96cba770f3334f79af3162475b54fa0f"
 from {username='slave'} at 127.0.0.1:59726
I20260709 13:24:44.603305 28371 raft_consensus.cc:606] T 07bb4101b0c24ee6b460e066cf78eb53 P 542ea9e2fcff4028b6766459e90bc5fc [term 1 LEADER]: Received request to transfer leadership to TS 96cba770f3334f79af3162475b54fa0f
I20260709 13:24:44.602623 28372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b05204beeff49309ed4e0dd43b4be7f" candidate_uuid: "96cba770f3334f79af3162475b54fa0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "542ea9e2fcff4028b6766459e90bc5fc"
I20260709 13:24:44.604547 28520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b05204beeff49309ed4e0dd43b4be7f" candidate_uuid: "96cba770f3334f79af3162475b54fa0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299"
I20260709 13:24:44.604732 28372 raft_consensus.cc:3060] T 2b05204beeff49309ed4e0dd43b4be7f P 542ea9e2fcff4028b6766459e90bc5fc [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:24:44.605139 28520 raft_consensus.cc:3055] T 2b05204beeff49309ed4e0dd43b4be7f P 2173e3fd92f34ad6ad8d56d2e6b05299 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:24:44.605432 28520 raft_consensus.cc:740] T 2b05204beeff49309ed4e0dd43b4be7f P 2173e3fd92f34ad6ad8d56d2e6b05299 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 2173e3fd92f34ad6ad8d56d2e6b05299, State: Running, Role: LEADER
I20260709 13:24:44.606216 28520 consensus_queue.cc:260] T 2b05204beeff49309ed4e0dd43b4be7f P 2173e3fd92f34ad6ad8d56d2e6b05299 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "542ea9e2fcff4028b6766459e90bc5fc" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38409 } } peers { permanent_uuid: "96cba770f3334f79af3162475b54fa0f" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 33585 } } peers { permanent_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 37597 } }
I20260709 13:24:44.607249 28520 raft_consensus.cc:3060] T 2b05204beeff49309ed4e0dd43b4be7f P 2173e3fd92f34ad6ad8d56d2e6b05299 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:24:44.614912 28520 raft_consensus.cc:2468] T 2b05204beeff49309ed4e0dd43b4be7f P 2173e3fd92f34ad6ad8d56d2e6b05299 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96cba770f3334f79af3162475b54fa0f in term 2.
I20260709 13:24:44.616542 28415 leader_election.cc:304] T 2b05204beeff49309ed4e0dd43b4be7f P 96cba770f3334f79af3162475b54fa0f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2173e3fd92f34ad6ad8d56d2e6b05299, 96cba770f3334f79af3162475b54fa0f; no voters: 
I20260709 13:24:44.617486 28739 raft_consensus.cc:2804] T 2b05204beeff49309ed4e0dd43b4be7f P 96cba770f3334f79af3162475b54fa0f [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:24:44.618046 28739 raft_consensus.cc:697] T 2b05204beeff49309ed4e0dd43b4be7f P 96cba770f3334f79af3162475b54fa0f [term 2 LEADER]: Becoming Leader. State: Replica: 96cba770f3334f79af3162475b54fa0f, State: Running, Role: LEADER
I20260709 13:24:44.618148 28448 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "cbb4f686d0884c178ea6b01616da20e8"
dest_uuid: "96cba770f3334f79af3162475b54fa0f"
mode: GRACEFUL
new_leader_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299"
 from {username='slave'} at 127.0.0.1:34650
I20260709 13:24:44.618894 28448 raft_consensus.cc:606] T cbb4f686d0884c178ea6b01616da20e8 P 96cba770f3334f79af3162475b54fa0f [term 1 LEADER]: Received request to transfer leadership to TS 2173e3fd92f34ad6ad8d56d2e6b05299
I20260709 13:24:44.619521 28739 consensus_queue.cc:237] T 2b05204beeff49309ed4e0dd43b4be7f P 96cba770f3334f79af3162475b54fa0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "542ea9e2fcff4028b6766459e90bc5fc" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38409 } } peers { permanent_uuid: "96cba770f3334f79af3162475b54fa0f" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 33585 } } peers { permanent_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 37597 } }
I20260709 13:24:44.615399 28444 leader_election.cc:290] T 2b05204beeff49309ed4e0dd43b4be7f P 96cba770f3334f79af3162475b54fa0f [CANDIDATE]: Term 2 election: Requested vote from peers 542ea9e2fcff4028b6766459e90bc5fc (127.14.177.129:38409), 2173e3fd92f34ad6ad8d56d2e6b05299 (127.14.177.131:37597)
I20260709 13:24:44.626982 28372 raft_consensus.cc:2468] T 2b05204beeff49309ed4e0dd43b4be7f P 542ea9e2fcff4028b6766459e90bc5fc [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96cba770f3334f79af3162475b54fa0f in term 2.
I20260709 13:24:44.632297 28739 consensus_queue.cc:1048] T cbb4f686d0884c178ea6b01616da20e8 P 96cba770f3334f79af3162475b54fa0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "542ea9e2fcff4028b6766459e90bc5fc" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38409 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:24:44.638065 28272 catalog_manager.cc:5697] T 2b05204beeff49309ed4e0dd43b4be7f P 96cba770f3334f79af3162475b54fa0f reported cstate change: term changed from 1 to 2, leader changed from 2173e3fd92f34ad6ad8d56d2e6b05299 (127.14.177.131) to 96cba770f3334f79af3162475b54fa0f (127.14.177.130). New cstate: current_term: 2 leader_uuid: "96cba770f3334f79af3162475b54fa0f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "542ea9e2fcff4028b6766459e90bc5fc" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38409 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "96cba770f3334f79af3162475b54fa0f" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 33585 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 37597 } health_report { overall_health: UNKNOWN } } }
I20260709 13:24:44.648288 28703 raft_consensus.cc:993] T cbb4f686d0884c178ea6b01616da20e8 P 96cba770f3334f79af3162475b54fa0f: : Instructing follower 2173e3fd92f34ad6ad8d56d2e6b05299 to start an election
I20260709 13:24:44.648959 28703 raft_consensus.cc:1081] T cbb4f686d0884c178ea6b01616da20e8 P 96cba770f3334f79af3162475b54fa0f [term 1 LEADER]: Signalling peer 2173e3fd92f34ad6ad8d56d2e6b05299 to start an election
I20260709 13:24:44.652644 28523 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "cbb4f686d0884c178ea6b01616da20e8"
dest_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299"
 from {username='slave'} at 127.0.0.1:53714
I20260709 13:24:44.653259 28523 raft_consensus.cc:493] T cbb4f686d0884c178ea6b01616da20e8 P 2173e3fd92f34ad6ad8d56d2e6b05299 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:24:44.653631 28523 raft_consensus.cc:3060] T cbb4f686d0884c178ea6b01616da20e8 P 2173e3fd92f34ad6ad8d56d2e6b05299 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:24:44.661434 28523 raft_consensus.cc:515] T cbb4f686d0884c178ea6b01616da20e8 P 2173e3fd92f34ad6ad8d56d2e6b05299 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "542ea9e2fcff4028b6766459e90bc5fc" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38409 } } peers { permanent_uuid: "96cba770f3334f79af3162475b54fa0f" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 33585 } } peers { permanent_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 37597 } }
I20260709 13:24:44.662518 28520 raft_consensus.cc:1240] T cbb4f686d0884c178ea6b01616da20e8 P 2173e3fd92f34ad6ad8d56d2e6b05299 [term 2 FOLLOWER]: Rejecting Update request from peer 96cba770f3334f79af3162475b54fa0f for earlier term 1. Current term is 2. Ops: []
I20260709 13:24:44.666080 28703 consensus_queue.cc:1059] T cbb4f686d0884c178ea6b01616da20e8 P 96cba770f3334f79af3162475b54fa0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 37597 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:24:44.667057 28739 raft_consensus.cc:3055] T cbb4f686d0884c178ea6b01616da20e8 P 96cba770f3334f79af3162475b54fa0f [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:24:44.667444 28739 raft_consensus.cc:740] T cbb4f686d0884c178ea6b01616da20e8 P 96cba770f3334f79af3162475b54fa0f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 96cba770f3334f79af3162475b54fa0f, State: Running, Role: LEADER
I20260709 13:24:44.668259 28739 consensus_queue.cc:260] T cbb4f686d0884c178ea6b01616da20e8 P 96cba770f3334f79af3162475b54fa0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "542ea9e2fcff4028b6766459e90bc5fc" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38409 } } peers { permanent_uuid: "96cba770f3334f79af3162475b54fa0f" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 33585 } } peers { permanent_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 37597 } }
I20260709 13:24:44.669055 28523 leader_election.cc:290] T cbb4f686d0884c178ea6b01616da20e8 P 2173e3fd92f34ad6ad8d56d2e6b05299 [CANDIDATE]: Term 2 election: Requested vote from peers 542ea9e2fcff4028b6766459e90bc5fc (127.14.177.129:38409), 96cba770f3334f79af3162475b54fa0f (127.14.177.130:33585)
I20260709 13:24:44.669428 28739 raft_consensus.cc:3060] T cbb4f686d0884c178ea6b01616da20e8 P 96cba770f3334f79af3162475b54fa0f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:24:44.670243 28444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbb4f686d0884c178ea6b01616da20e8" candidate_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "96cba770f3334f79af3162475b54fa0f"
I20260709 13:24:44.671530 28371 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbb4f686d0884c178ea6b01616da20e8" candidate_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "542ea9e2fcff4028b6766459e90bc5fc"
I20260709 13:24:44.672137 28371 raft_consensus.cc:3060] T cbb4f686d0884c178ea6b01616da20e8 P 542ea9e2fcff4028b6766459e90bc5fc [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:24:44.679250 28371 raft_consensus.cc:2468] T cbb4f686d0884c178ea6b01616da20e8 P 542ea9e2fcff4028b6766459e90bc5fc [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2173e3fd92f34ad6ad8d56d2e6b05299 in term 2.
I20260709 13:24:44.680639 28487 leader_election.cc:304] T cbb4f686d0884c178ea6b01616da20e8 P 2173e3fd92f34ad6ad8d56d2e6b05299 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2173e3fd92f34ad6ad8d56d2e6b05299, 542ea9e2fcff4028b6766459e90bc5fc; no voters: 
I20260709 13:24:44.681668 28647 raft_consensus.cc:2804] T cbb4f686d0884c178ea6b01616da20e8 P 2173e3fd92f34ad6ad8d56d2e6b05299 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:24:44.682190 28647 raft_consensus.cc:697] T cbb4f686d0884c178ea6b01616da20e8 P 2173e3fd92f34ad6ad8d56d2e6b05299 [term 2 LEADER]: Becoming Leader. State: Replica: 2173e3fd92f34ad6ad8d56d2e6b05299, State: Running, Role: LEADER
I20260709 13:24:44.683079 28647 consensus_queue.cc:237] T cbb4f686d0884c178ea6b01616da20e8 P 2173e3fd92f34ad6ad8d56d2e6b05299 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "542ea9e2fcff4028b6766459e90bc5fc" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38409 } } peers { permanent_uuid: "96cba770f3334f79af3162475b54fa0f" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 33585 } } peers { permanent_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 37597 } }
I20260709 13:24:44.685739 28444 raft_consensus.cc:2468] T cbb4f686d0884c178ea6b01616da20e8 P 96cba770f3334f79af3162475b54fa0f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2173e3fd92f34ad6ad8d56d2e6b05299 in term 2.
I20260709 13:24:44.694202 28271 catalog_manager.cc:5697] T cbb4f686d0884c178ea6b01616da20e8 P 2173e3fd92f34ad6ad8d56d2e6b05299 reported cstate change: term changed from 1 to 2, leader changed from 96cba770f3334f79af3162475b54fa0f (127.14.177.130) to 2173e3fd92f34ad6ad8d56d2e6b05299 (127.14.177.131). New cstate: current_term: 2 leader_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "542ea9e2fcff4028b6766459e90bc5fc" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38409 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "96cba770f3334f79af3162475b54fa0f" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 33585 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 37597 } health_report { overall_health: HEALTHY } } }
I20260709 13:24:44.787371 28759 raft_consensus.cc:993] T e4c7e579c94a4f5fa195def741ee4180 P 2173e3fd92f34ad6ad8d56d2e6b05299: : Instructing follower 542ea9e2fcff4028b6766459e90bc5fc to start an election
I20260709 13:24:44.787936 28647 raft_consensus.cc:1081] T e4c7e579c94a4f5fa195def741ee4180 P 2173e3fd92f34ad6ad8d56d2e6b05299 [term 1 LEADER]: Signalling peer 542ea9e2fcff4028b6766459e90bc5fc to start an election
I20260709 13:24:44.789722 28371 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e4c7e579c94a4f5fa195def741ee4180"
dest_uuid: "542ea9e2fcff4028b6766459e90bc5fc"
 from {username='slave'} at 127.0.0.1:59710
I20260709 13:24:44.790374 28371 raft_consensus.cc:493] T e4c7e579c94a4f5fa195def741ee4180 P 542ea9e2fcff4028b6766459e90bc5fc [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:24:44.790782 28371 raft_consensus.cc:3060] T e4c7e579c94a4f5fa195def741ee4180 P 542ea9e2fcff4028b6766459e90bc5fc [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:24:44.799608 28371 raft_consensus.cc:515] T e4c7e579c94a4f5fa195def741ee4180 P 542ea9e2fcff4028b6766459e90bc5fc [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "542ea9e2fcff4028b6766459e90bc5fc" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38409 } } peers { permanent_uuid: "96cba770f3334f79af3162475b54fa0f" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 33585 } } peers { permanent_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 37597 } }
I20260709 13:24:44.802044 28371 leader_election.cc:290] T e4c7e579c94a4f5fa195def741ee4180 P 542ea9e2fcff4028b6766459e90bc5fc [CANDIDATE]: Term 2 election: Requested vote from peers 96cba770f3334f79af3162475b54fa0f (127.14.177.130:33585), 2173e3fd92f34ad6ad8d56d2e6b05299 (127.14.177.131:37597)
I20260709 13:24:44.802878 28444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4c7e579c94a4f5fa195def741ee4180" candidate_uuid: "542ea9e2fcff4028b6766459e90bc5fc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "96cba770f3334f79af3162475b54fa0f"
I20260709 13:24:44.803300 28523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4c7e579c94a4f5fa195def741ee4180" candidate_uuid: "542ea9e2fcff4028b6766459e90bc5fc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299"
I20260709 13:24:44.803694 28444 raft_consensus.cc:3060] T e4c7e579c94a4f5fa195def741ee4180 P 96cba770f3334f79af3162475b54fa0f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:24:44.804025 28523 raft_consensus.cc:3055] T e4c7e579c94a4f5fa195def741ee4180 P 2173e3fd92f34ad6ad8d56d2e6b05299 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:24:44.804382 28523 raft_consensus.cc:740] T e4c7e579c94a4f5fa195def741ee4180 P 2173e3fd92f34ad6ad8d56d2e6b05299 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 2173e3fd92f34ad6ad8d56d2e6b05299, State: Running, Role: LEADER
I20260709 13:24:44.805312 28523 consensus_queue.cc:260] T e4c7e579c94a4f5fa195def741ee4180 P 2173e3fd92f34ad6ad8d56d2e6b05299 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "542ea9e2fcff4028b6766459e90bc5fc" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38409 } } peers { permanent_uuid: "96cba770f3334f79af3162475b54fa0f" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 33585 } } peers { permanent_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 37597 } }
I20260709 13:24:44.806788 28523 raft_consensus.cc:3060] T e4c7e579c94a4f5fa195def741ee4180 P 2173e3fd92f34ad6ad8d56d2e6b05299 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:24:44.823318 28523 raft_consensus.cc:2468] T e4c7e579c94a4f5fa195def741ee4180 P 2173e3fd92f34ad6ad8d56d2e6b05299 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 542ea9e2fcff4028b6766459e90bc5fc in term 2.
I20260709 13:24:44.824991 28340 leader_election.cc:304] T e4c7e579c94a4f5fa195def741ee4180 P 542ea9e2fcff4028b6766459e90bc5fc [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2173e3fd92f34ad6ad8d56d2e6b05299, 542ea9e2fcff4028b6766459e90bc5fc; no voters: 
I20260709 13:24:44.826122 28704 raft_consensus.cc:2804] T e4c7e579c94a4f5fa195def741ee4180 P 542ea9e2fcff4028b6766459e90bc5fc [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:24:44.826665 28704 raft_consensus.cc:697] T e4c7e579c94a4f5fa195def741ee4180 P 542ea9e2fcff4028b6766459e90bc5fc [term 2 LEADER]: Becoming Leader. State: Replica: 542ea9e2fcff4028b6766459e90bc5fc, State: Running, Role: LEADER
I20260709 13:24:44.826753 28668 raft_consensus.cc:993] T fbb3d76b1491485287e0dc48b24dbb46 P 96cba770f3334f79af3162475b54fa0f: : Instructing follower 542ea9e2fcff4028b6766459e90bc5fc to start an election
I20260709 13:24:44.827708 28668 raft_consensus.cc:1081] T fbb3d76b1491485287e0dc48b24dbb46 P 96cba770f3334f79af3162475b54fa0f [term 1 LEADER]: Signalling peer 542ea9e2fcff4028b6766459e90bc5fc to start an election
I20260709 13:24:44.827674 28704 consensus_queue.cc:237] T e4c7e579c94a4f5fa195def741ee4180 P 542ea9e2fcff4028b6766459e90bc5fc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "542ea9e2fcff4028b6766459e90bc5fc" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38409 } } peers { permanent_uuid: "96cba770f3334f79af3162475b54fa0f" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 33585 } } peers { permanent_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 37597 } }
I20260709 13:24:44.828861 28444 raft_consensus.cc:2468] T e4c7e579c94a4f5fa195def741ee4180 P 96cba770f3334f79af3162475b54fa0f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 542ea9e2fcff4028b6766459e90bc5fc in term 2.
I20260709 13:24:44.831228 28371 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "fbb3d76b1491485287e0dc48b24dbb46"
dest_uuid: "542ea9e2fcff4028b6766459e90bc5fc"
 from {username='slave'} at 127.0.0.1:59714
I20260709 13:24:44.831837 28371 raft_consensus.cc:493] T fbb3d76b1491485287e0dc48b24dbb46 P 542ea9e2fcff4028b6766459e90bc5fc [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:24:44.832342 28371 raft_consensus.cc:3060] T fbb3d76b1491485287e0dc48b24dbb46 P 542ea9e2fcff4028b6766459e90bc5fc [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:24:44.838223 28271 catalog_manager.cc:5697] T e4c7e579c94a4f5fa195def741ee4180 P 542ea9e2fcff4028b6766459e90bc5fc reported cstate change: term changed from 1 to 2, leader changed from 2173e3fd92f34ad6ad8d56d2e6b05299 (127.14.177.131) to 542ea9e2fcff4028b6766459e90bc5fc (127.14.177.129). New cstate: current_term: 2 leader_uuid: "542ea9e2fcff4028b6766459e90bc5fc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "542ea9e2fcff4028b6766459e90bc5fc" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38409 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "96cba770f3334f79af3162475b54fa0f" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 33585 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 37597 } health_report { overall_health: UNKNOWN } } }
I20260709 13:24:44.840077 28371 raft_consensus.cc:515] T fbb3d76b1491485287e0dc48b24dbb46 P 542ea9e2fcff4028b6766459e90bc5fc [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "542ea9e2fcff4028b6766459e90bc5fc" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38409 } } peers { permanent_uuid: "96cba770f3334f79af3162475b54fa0f" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 33585 } } peers { permanent_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 37597 } }
I20260709 13:24:44.842262 28371 leader_election.cc:290] T fbb3d76b1491485287e0dc48b24dbb46 P 542ea9e2fcff4028b6766459e90bc5fc [CANDIDATE]: Term 2 election: Requested vote from peers 96cba770f3334f79af3162475b54fa0f (127.14.177.130:33585), 2173e3fd92f34ad6ad8d56d2e6b05299 (127.14.177.131:37597)
I20260709 13:24:44.844111 28523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb3d76b1491485287e0dc48b24dbb46" candidate_uuid: "542ea9e2fcff4028b6766459e90bc5fc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299"
I20260709 13:24:44.844648 28444 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb3d76b1491485287e0dc48b24dbb46" candidate_uuid: "542ea9e2fcff4028b6766459e90bc5fc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "96cba770f3334f79af3162475b54fa0f"
I20260709 13:24:44.844787 28523 raft_consensus.cc:3060] T fbb3d76b1491485287e0dc48b24dbb46 P 2173e3fd92f34ad6ad8d56d2e6b05299 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:24:44.845506 28444 raft_consensus.cc:3055] T fbb3d76b1491485287e0dc48b24dbb46 P 96cba770f3334f79af3162475b54fa0f [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:24:44.845952 28444 raft_consensus.cc:740] T fbb3d76b1491485287e0dc48b24dbb46 P 96cba770f3334f79af3162475b54fa0f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 96cba770f3334f79af3162475b54fa0f, State: Running, Role: LEADER
I20260709 13:24:44.847028 28444 consensus_queue.cc:260] T fbb3d76b1491485287e0dc48b24dbb46 P 96cba770f3334f79af3162475b54fa0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "542ea9e2fcff4028b6766459e90bc5fc" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38409 } } peers { permanent_uuid: "96cba770f3334f79af3162475b54fa0f" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 33585 } } peers { permanent_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 37597 } }
I20260709 13:24:44.848556 28444 raft_consensus.cc:3060] T fbb3d76b1491485287e0dc48b24dbb46 P 96cba770f3334f79af3162475b54fa0f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:24:44.853152 28523 raft_consensus.cc:2468] T fbb3d76b1491485287e0dc48b24dbb46 P 2173e3fd92f34ad6ad8d56d2e6b05299 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 542ea9e2fcff4028b6766459e90bc5fc in term 2.
I20260709 13:24:44.854559 28340 leader_election.cc:304] T fbb3d76b1491485287e0dc48b24dbb46 P 542ea9e2fcff4028b6766459e90bc5fc [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2173e3fd92f34ad6ad8d56d2e6b05299, 542ea9e2fcff4028b6766459e90bc5fc; no voters: 
I20260709 13:24:44.855530 28704 raft_consensus.cc:2804] T fbb3d76b1491485287e0dc48b24dbb46 P 542ea9e2fcff4028b6766459e90bc5fc [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:24:44.856042 28704 raft_consensus.cc:697] T fbb3d76b1491485287e0dc48b24dbb46 P 542ea9e2fcff4028b6766459e90bc5fc [term 2 LEADER]: Becoming Leader. State: Replica: 542ea9e2fcff4028b6766459e90bc5fc, State: Running, Role: LEADER
I20260709 13:24:44.856866 28704 consensus_queue.cc:237] T fbb3d76b1491485287e0dc48b24dbb46 P 542ea9e2fcff4028b6766459e90bc5fc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "542ea9e2fcff4028b6766459e90bc5fc" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38409 } } peers { permanent_uuid: "96cba770f3334f79af3162475b54fa0f" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 33585 } } peers { permanent_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 37597 } }
I20260709 13:24:44.857542 28444 raft_consensus.cc:2468] T fbb3d76b1491485287e0dc48b24dbb46 P 96cba770f3334f79af3162475b54fa0f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 542ea9e2fcff4028b6766459e90bc5fc in term 2.
I20260709 13:24:44.864886 28271 catalog_manager.cc:5697] T fbb3d76b1491485287e0dc48b24dbb46 P 542ea9e2fcff4028b6766459e90bc5fc reported cstate change: term changed from 1 to 2, leader changed from 96cba770f3334f79af3162475b54fa0f (127.14.177.130) to 542ea9e2fcff4028b6766459e90bc5fc (127.14.177.129). New cstate: current_term: 2 leader_uuid: "542ea9e2fcff4028b6766459e90bc5fc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "542ea9e2fcff4028b6766459e90bc5fc" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38409 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "96cba770f3334f79af3162475b54fa0f" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 33585 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 37597 } health_report { overall_health: UNKNOWN } } }
I20260709 13:24:44.962733 28704 raft_consensus.cc:993] T 6efeed092ec840f2a7748c4a613dd6a7 P 542ea9e2fcff4028b6766459e90bc5fc: : Instructing follower 96cba770f3334f79af3162475b54fa0f to start an election
I20260709 13:24:44.963343 28707 raft_consensus.cc:1081] T 6efeed092ec840f2a7748c4a613dd6a7 P 542ea9e2fcff4028b6766459e90bc5fc [term 1 LEADER]: Signalling peer 96cba770f3334f79af3162475b54fa0f to start an election
I20260709 13:24:44.965500 28444 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "6efeed092ec840f2a7748c4a613dd6a7"
dest_uuid: "96cba770f3334f79af3162475b54fa0f"
 from {username='slave'} at 127.0.0.1:34636
I20260709 13:24:44.966144 28647 raft_consensus.cc:993] T d489ac98a4654a7cb46771880f89b89b P 2173e3fd92f34ad6ad8d56d2e6b05299: : Instructing follower 542ea9e2fcff4028b6766459e90bc5fc to start an election
I20260709 13:24:44.966747 28754 raft_consensus.cc:1081] T d489ac98a4654a7cb46771880f89b89b P 2173e3fd92f34ad6ad8d56d2e6b05299 [term 1 LEADER]: Signalling peer 542ea9e2fcff4028b6766459e90bc5fc to start an election
I20260709 13:24:44.966147 28444 raft_consensus.cc:493] T 6efeed092ec840f2a7748c4a613dd6a7 P 96cba770f3334f79af3162475b54fa0f [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:24:44.967543 28444 raft_consensus.cc:3060] T 6efeed092ec840f2a7748c4a613dd6a7 P 96cba770f3334f79af3162475b54fa0f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:24:44.968940 28371 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d489ac98a4654a7cb46771880f89b89b"
dest_uuid: "542ea9e2fcff4028b6766459e90bc5fc"
 from {username='slave'} at 127.0.0.1:59710
I20260709 13:24:44.969594 28371 raft_consensus.cc:493] T d489ac98a4654a7cb46771880f89b89b P 542ea9e2fcff4028b6766459e90bc5fc [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:24:44.969947 28371 raft_consensus.cc:3060] T d489ac98a4654a7cb46771880f89b89b P 542ea9e2fcff4028b6766459e90bc5fc [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:24:44.976406 28444 raft_consensus.cc:515] T 6efeed092ec840f2a7748c4a613dd6a7 P 96cba770f3334f79af3162475b54fa0f [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "542ea9e2fcff4028b6766459e90bc5fc" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38409 } } peers { permanent_uuid: "96cba770f3334f79af3162475b54fa0f" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 33585 } } peers { permanent_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 37597 } }
I20260709 13:24:44.979707 28372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6efeed092ec840f2a7748c4a613dd6a7" candidate_uuid: "96cba770f3334f79af3162475b54fa0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "542ea9e2fcff4028b6766459e90bc5fc"
I20260709 13:24:44.978091 28371 raft_consensus.cc:515] T d489ac98a4654a7cb46771880f89b89b P 542ea9e2fcff4028b6766459e90bc5fc [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "542ea9e2fcff4028b6766459e90bc5fc" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38409 } } peers { permanent_uuid: "96cba770f3334f79af3162475b54fa0f" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 33585 } } peers { permanent_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 37597 } }
I20260709 13:24:44.979962 28520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6efeed092ec840f2a7748c4a613dd6a7" candidate_uuid: "96cba770f3334f79af3162475b54fa0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299"
I20260709 13:24:44.980436 28372 raft_consensus.cc:3055] T 6efeed092ec840f2a7748c4a613dd6a7 P 542ea9e2fcff4028b6766459e90bc5fc [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:24:44.980659 28520 raft_consensus.cc:3060] T 6efeed092ec840f2a7748c4a613dd6a7 P 2173e3fd92f34ad6ad8d56d2e6b05299 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:24:44.980896 28372 raft_consensus.cc:740] T 6efeed092ec840f2a7748c4a613dd6a7 P 542ea9e2fcff4028b6766459e90bc5fc [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 542ea9e2fcff4028b6766459e90bc5fc, State: Running, Role: LEADER
I20260709 13:24:44.981892 28372 consensus_queue.cc:260] T 6efeed092ec840f2a7748c4a613dd6a7 P 542ea9e2fcff4028b6766459e90bc5fc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "542ea9e2fcff4028b6766459e90bc5fc" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38409 } } peers { permanent_uuid: "96cba770f3334f79af3162475b54fa0f" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 33585 } } peers { permanent_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 37597 } }
I20260709 13:24:44.983137 28372 raft_consensus.cc:3060] T 6efeed092ec840f2a7748c4a613dd6a7 P 542ea9e2fcff4028b6766459e90bc5fc [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:24:44.984503 28448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d489ac98a4654a7cb46771880f89b89b" candidate_uuid: "542ea9e2fcff4028b6766459e90bc5fc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "96cba770f3334f79af3162475b54fa0f"
I20260709 13:24:44.985152 28448 raft_consensus.cc:3060] T d489ac98a4654a7cb46771880f89b89b P 96cba770f3334f79af3162475b54fa0f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:24:44.989012 28520 raft_consensus.cc:2468] T 6efeed092ec840f2a7748c4a613dd6a7 P 2173e3fd92f34ad6ad8d56d2e6b05299 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96cba770f3334f79af3162475b54fa0f in term 2.
I20260709 13:24:44.990433 28415 leader_election.cc:304] T 6efeed092ec840f2a7748c4a613dd6a7 P 96cba770f3334f79af3162475b54fa0f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2173e3fd92f34ad6ad8d56d2e6b05299, 96cba770f3334f79af3162475b54fa0f; no voters: 
I20260709 13:24:44.991503 28739 raft_consensus.cc:2804] T 6efeed092ec840f2a7748c4a613dd6a7 P 96cba770f3334f79af3162475b54fa0f [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:24:44.979398 28444 leader_election.cc:290] T 6efeed092ec840f2a7748c4a613dd6a7 P 96cba770f3334f79af3162475b54fa0f [CANDIDATE]: Term 2 election: Requested vote from peers 542ea9e2fcff4028b6766459e90bc5fc (127.14.177.129:38409), 2173e3fd92f34ad6ad8d56d2e6b05299 (127.14.177.131:37597)
I20260709 13:24:44.983597 28523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d489ac98a4654a7cb46771880f89b89b" candidate_uuid: "542ea9e2fcff4028b6766459e90bc5fc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299"
I20260709 13:24:44.984126 28371 leader_election.cc:290] T d489ac98a4654a7cb46771880f89b89b P 542ea9e2fcff4028b6766459e90bc5fc [CANDIDATE]: Term 2 election: Requested vote from peers 96cba770f3334f79af3162475b54fa0f (127.14.177.130:33585), 2173e3fd92f34ad6ad8d56d2e6b05299 (127.14.177.131:37597)
I20260709 13:24:44.992761 28523 raft_consensus.cc:3055] T d489ac98a4654a7cb46771880f89b89b P 2173e3fd92f34ad6ad8d56d2e6b05299 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:24:44.992152 28372 raft_consensus.cc:2468] T 6efeed092ec840f2a7748c4a613dd6a7 P 542ea9e2fcff4028b6766459e90bc5fc [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96cba770f3334f79af3162475b54fa0f in term 2.
I20260709 13:24:44.993182 28523 raft_consensus.cc:740] T d489ac98a4654a7cb46771880f89b89b P 2173e3fd92f34ad6ad8d56d2e6b05299 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 2173e3fd92f34ad6ad8d56d2e6b05299, State: Running, Role: LEADER
I20260709 13:24:44.994220 28523 consensus_queue.cc:260] T d489ac98a4654a7cb46771880f89b89b P 2173e3fd92f34ad6ad8d56d2e6b05299 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "542ea9e2fcff4028b6766459e90bc5fc" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38409 } } peers { permanent_uuid: "96cba770f3334f79af3162475b54fa0f" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 33585 } } peers { permanent_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 37597 } }
I20260709 13:24:44.995301 28739 raft_consensus.cc:697] T 6efeed092ec840f2a7748c4a613dd6a7 P 96cba770f3334f79af3162475b54fa0f [term 2 LEADER]: Becoming Leader. State: Replica: 96cba770f3334f79af3162475b54fa0f, State: Running, Role: LEADER
I20260709 13:24:44.995606 28523 raft_consensus.cc:3060] T d489ac98a4654a7cb46771880f89b89b P 2173e3fd92f34ad6ad8d56d2e6b05299 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:24:44.996520 28739 consensus_queue.cc:237] T 6efeed092ec840f2a7748c4a613dd6a7 P 96cba770f3334f79af3162475b54fa0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "542ea9e2fcff4028b6766459e90bc5fc" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38409 } } peers { permanent_uuid: "96cba770f3334f79af3162475b54fa0f" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 33585 } } peers { permanent_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 37597 } }
I20260709 13:24:45.009699 28523 raft_consensus.cc:2468] T d489ac98a4654a7cb46771880f89b89b P 2173e3fd92f34ad6ad8d56d2e6b05299 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 542ea9e2fcff4028b6766459e90bc5fc in term 2.
I20260709 13:24:45.011086 28448 raft_consensus.cc:2468] T d489ac98a4654a7cb46771880f89b89b P 96cba770f3334f79af3162475b54fa0f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 542ea9e2fcff4028b6766459e90bc5fc in term 2.
I20260709 13:24:45.011264 28340 leader_election.cc:304] T d489ac98a4654a7cb46771880f89b89b P 542ea9e2fcff4028b6766459e90bc5fc [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2173e3fd92f34ad6ad8d56d2e6b05299, 542ea9e2fcff4028b6766459e90bc5fc; no voters: 
I20260709 13:24:45.012342 28704 raft_consensus.cc:2804] T d489ac98a4654a7cb46771880f89b89b P 542ea9e2fcff4028b6766459e90bc5fc [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:24:45.013031 28704 raft_consensus.cc:697] T d489ac98a4654a7cb46771880f89b89b P 542ea9e2fcff4028b6766459e90bc5fc [term 2 LEADER]: Becoming Leader. State: Replica: 542ea9e2fcff4028b6766459e90bc5fc, State: Running, Role: LEADER
I20260709 13:24:45.013989 28704 consensus_queue.cc:237] T d489ac98a4654a7cb46771880f89b89b P 542ea9e2fcff4028b6766459e90bc5fc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "542ea9e2fcff4028b6766459e90bc5fc" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38409 } } peers { permanent_uuid: "96cba770f3334f79af3162475b54fa0f" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 33585 } } peers { permanent_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 37597 } }
I20260709 13:24:45.015098 28271 catalog_manager.cc:5697] T 6efeed092ec840f2a7748c4a613dd6a7 P 96cba770f3334f79af3162475b54fa0f reported cstate change: term changed from 1 to 2, leader changed from 542ea9e2fcff4028b6766459e90bc5fc (127.14.177.129) to 96cba770f3334f79af3162475b54fa0f (127.14.177.130). New cstate: current_term: 2 leader_uuid: "96cba770f3334f79af3162475b54fa0f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "542ea9e2fcff4028b6766459e90bc5fc" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38409 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "96cba770f3334f79af3162475b54fa0f" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 33585 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 37597 } health_report { overall_health: UNKNOWN } } }
I20260709 13:24:45.025220 28269 catalog_manager.cc:5697] T d489ac98a4654a7cb46771880f89b89b P 542ea9e2fcff4028b6766459e90bc5fc reported cstate change: term changed from 1 to 2, leader changed from 2173e3fd92f34ad6ad8d56d2e6b05299 (127.14.177.131) to 542ea9e2fcff4028b6766459e90bc5fc (127.14.177.129). New cstate: current_term: 2 leader_uuid: "542ea9e2fcff4028b6766459e90bc5fc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "542ea9e2fcff4028b6766459e90bc5fc" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38409 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "96cba770f3334f79af3162475b54fa0f" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 33585 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 37597 } health_report { overall_health: UNKNOWN } } }
I20260709 13:24:45.036870 28372 raft_consensus.cc:1275] T 2b05204beeff49309ed4e0dd43b4be7f P 542ea9e2fcff4028b6766459e90bc5fc [term 2 FOLLOWER]: Refusing update from remote peer 96cba770f3334f79af3162475b54fa0f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:24:45.038354 28739 consensus_queue.cc:1048] T 2b05204beeff49309ed4e0dd43b4be7f P 96cba770f3334f79af3162475b54fa0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "542ea9e2fcff4028b6766459e90bc5fc" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38409 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:24:45.048723 28523 raft_consensus.cc:1275] T 2b05204beeff49309ed4e0dd43b4be7f P 2173e3fd92f34ad6ad8d56d2e6b05299 [term 2 FOLLOWER]: Refusing update from remote peer 96cba770f3334f79af3162475b54fa0f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:24:45.050411 28700 consensus_queue.cc:1048] T 2b05204beeff49309ed4e0dd43b4be7f P 96cba770f3334f79af3162475b54fa0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 37597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:24:45.096458 28707 raft_consensus.cc:993] T 07bb4101b0c24ee6b460e066cf78eb53 P 542ea9e2fcff4028b6766459e90bc5fc: : Instructing follower 96cba770f3334f79af3162475b54fa0f to start an election
I20260709 13:24:45.097031 28704 raft_consensus.cc:1081] T 07bb4101b0c24ee6b460e066cf78eb53 P 542ea9e2fcff4028b6766459e90bc5fc [term 1 LEADER]: Signalling peer 96cba770f3334f79af3162475b54fa0f to start an election
I20260709 13:24:45.098789 28448 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "07bb4101b0c24ee6b460e066cf78eb53"
dest_uuid: "96cba770f3334f79af3162475b54fa0f"
 from {username='slave'} at 127.0.0.1:34636
I20260709 13:24:45.099354 28448 raft_consensus.cc:493] T 07bb4101b0c24ee6b460e066cf78eb53 P 96cba770f3334f79af3162475b54fa0f [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:24:45.099731 28448 raft_consensus.cc:3060] T 07bb4101b0c24ee6b460e066cf78eb53 P 96cba770f3334f79af3162475b54fa0f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:24:45.106832 28448 raft_consensus.cc:515] T 07bb4101b0c24ee6b460e066cf78eb53 P 96cba770f3334f79af3162475b54fa0f [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "542ea9e2fcff4028b6766459e90bc5fc" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38409 } } peers { permanent_uuid: "96cba770f3334f79af3162475b54fa0f" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 33585 } } peers { permanent_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 37597 } }
I20260709 13:24:45.110069 28372 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07bb4101b0c24ee6b460e066cf78eb53" candidate_uuid: "96cba770f3334f79af3162475b54fa0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "542ea9e2fcff4028b6766459e90bc5fc"
I20260709 13:24:45.110121 28520 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07bb4101b0c24ee6b460e066cf78eb53" candidate_uuid: "96cba770f3334f79af3162475b54fa0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299"
I20260709 13:24:45.110740 28448 leader_election.cc:290] T 07bb4101b0c24ee6b460e066cf78eb53 P 96cba770f3334f79af3162475b54fa0f [CANDIDATE]: Term 2 election: Requested vote from peers 542ea9e2fcff4028b6766459e90bc5fc (127.14.177.129:38409), 2173e3fd92f34ad6ad8d56d2e6b05299 (127.14.177.131:37597)
I20260709 13:24:45.111147 28520 raft_consensus.cc:3060] T 07bb4101b0c24ee6b460e066cf78eb53 P 2173e3fd92f34ad6ad8d56d2e6b05299 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:24:45.111801 28372 raft_consensus.cc:3055] T 07bb4101b0c24ee6b460e066cf78eb53 P 542ea9e2fcff4028b6766459e90bc5fc [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:24:45.112244 28372 raft_consensus.cc:740] T 07bb4101b0c24ee6b460e066cf78eb53 P 542ea9e2fcff4028b6766459e90bc5fc [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 542ea9e2fcff4028b6766459e90bc5fc, State: Running, Role: LEADER
I20260709 13:24:45.113085 28372 consensus_queue.cc:260] T 07bb4101b0c24ee6b460e066cf78eb53 P 542ea9e2fcff4028b6766459e90bc5fc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "542ea9e2fcff4028b6766459e90bc5fc" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38409 } } peers { permanent_uuid: "96cba770f3334f79af3162475b54fa0f" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 33585 } } peers { permanent_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 37597 } }
I20260709 13:24:45.114416 28372 raft_consensus.cc:3060] T 07bb4101b0c24ee6b460e066cf78eb53 P 542ea9e2fcff4028b6766459e90bc5fc [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:24:45.120961 28372 raft_consensus.cc:2468] T 07bb4101b0c24ee6b460e066cf78eb53 P 542ea9e2fcff4028b6766459e90bc5fc [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96cba770f3334f79af3162475b54fa0f in term 2.
I20260709 13:24:45.122008 28413 leader_election.cc:304] T 07bb4101b0c24ee6b460e066cf78eb53 P 96cba770f3334f79af3162475b54fa0f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 542ea9e2fcff4028b6766459e90bc5fc, 96cba770f3334f79af3162475b54fa0f; no voters: 
I20260709 13:24:45.122826 28700 raft_consensus.cc:2804] T 07bb4101b0c24ee6b460e066cf78eb53 P 96cba770f3334f79af3162475b54fa0f [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:24:45.123351 28700 raft_consensus.cc:697] T 07bb4101b0c24ee6b460e066cf78eb53 P 96cba770f3334f79af3162475b54fa0f [term 2 LEADER]: Becoming Leader. State: Replica: 96cba770f3334f79af3162475b54fa0f, State: Running, Role: LEADER
I20260709 13:24:45.124171 28700 consensus_queue.cc:237] T 07bb4101b0c24ee6b460e066cf78eb53 P 96cba770f3334f79af3162475b54fa0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "542ea9e2fcff4028b6766459e90bc5fc" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38409 } } peers { permanent_uuid: "96cba770f3334f79af3162475b54fa0f" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 33585 } } peers { permanent_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 37597 } }
I20260709 13:24:45.131078 28520 raft_consensus.cc:2468] T 07bb4101b0c24ee6b460e066cf78eb53 P 2173e3fd92f34ad6ad8d56d2e6b05299 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96cba770f3334f79af3162475b54fa0f in term 2.
I20260709 13:24:45.133049 28269 catalog_manager.cc:5697] T 07bb4101b0c24ee6b460e066cf78eb53 P 96cba770f3334f79af3162475b54fa0f reported cstate change: term changed from 1 to 2, leader changed from 542ea9e2fcff4028b6766459e90bc5fc (127.14.177.129) to 96cba770f3334f79af3162475b54fa0f (127.14.177.130). New cstate: current_term: 2 leader_uuid: "96cba770f3334f79af3162475b54fa0f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "542ea9e2fcff4028b6766459e90bc5fc" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38409 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "96cba770f3334f79af3162475b54fa0f" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 33585 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 37597 } health_report { overall_health: UNKNOWN } } }
I20260709 13:24:45.215070 28520 raft_consensus.cc:1275] T e4c7e579c94a4f5fa195def741ee4180 P 2173e3fd92f34ad6ad8d56d2e6b05299 [term 2 FOLLOWER]: Refusing update from remote peer 542ea9e2fcff4028b6766459e90bc5fc: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:24:45.216696 28619 consensus_queue.cc:1048] T e4c7e579c94a4f5fa195def741ee4180 P 542ea9e2fcff4028b6766459e90bc5fc [LEADER]: Connected to new peer: Peer: permanent_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 37597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:24:45.225678 28448 raft_consensus.cc:1275] T e4c7e579c94a4f5fa195def741ee4180 P 96cba770f3334f79af3162475b54fa0f [term 2 FOLLOWER]: Refusing update from remote peer 542ea9e2fcff4028b6766459e90bc5fc: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:24:45.228312 28619 consensus_queue.cc:1048] T e4c7e579c94a4f5fa195def741ee4180 P 542ea9e2fcff4028b6766459e90bc5fc [LEADER]: Connected to new peer: Peer: permanent_uuid: "96cba770f3334f79af3162475b54fa0f" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 33585 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:24:45.272991 28372 raft_consensus.cc:1275] T cbb4f686d0884c178ea6b01616da20e8 P 542ea9e2fcff4028b6766459e90bc5fc [term 2 FOLLOWER]: Refusing update from remote peer 2173e3fd92f34ad6ad8d56d2e6b05299: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:24:45.274921 28647 consensus_queue.cc:1048] T cbb4f686d0884c178ea6b01616da20e8 P 2173e3fd92f34ad6ad8d56d2e6b05299 [LEADER]: Connected to new peer: Peer: permanent_uuid: "542ea9e2fcff4028b6766459e90bc5fc" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38409 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:24:45.284869 28448 raft_consensus.cc:1275] T cbb4f686d0884c178ea6b01616da20e8 P 96cba770f3334f79af3162475b54fa0f [term 2 FOLLOWER]: Refusing update from remote peer 2173e3fd92f34ad6ad8d56d2e6b05299: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:24:45.286553 28759 consensus_queue.cc:1048] T cbb4f686d0884c178ea6b01616da20e8 P 2173e3fd92f34ad6ad8d56d2e6b05299 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96cba770f3334f79af3162475b54fa0f" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 33585 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:24:45.389192 28444 raft_consensus.cc:1275] T fbb3d76b1491485287e0dc48b24dbb46 P 96cba770f3334f79af3162475b54fa0f [term 2 FOLLOWER]: Refusing update from remote peer 542ea9e2fcff4028b6766459e90bc5fc: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:24:45.390441 28619 consensus_queue.cc:1048] T fbb3d76b1491485287e0dc48b24dbb46 P 542ea9e2fcff4028b6766459e90bc5fc [LEADER]: Connected to new peer: Peer: permanent_uuid: "96cba770f3334f79af3162475b54fa0f" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 33585 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:24:45.399353 28520 raft_consensus.cc:1275] T fbb3d76b1491485287e0dc48b24dbb46 P 2173e3fd92f34ad6ad8d56d2e6b05299 [term 2 FOLLOWER]: Refusing update from remote peer 542ea9e2fcff4028b6766459e90bc5fc: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:24:45.401492 28704 consensus_queue.cc:1048] T fbb3d76b1491485287e0dc48b24dbb46 P 542ea9e2fcff4028b6766459e90bc5fc [LEADER]: Connected to new peer: Peer: permanent_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 37597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:24:45.430122 28520 raft_consensus.cc:1275] T 6efeed092ec840f2a7748c4a613dd6a7 P 2173e3fd92f34ad6ad8d56d2e6b05299 [term 2 FOLLOWER]: Refusing update from remote peer 96cba770f3334f79af3162475b54fa0f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:24:45.431298 28700 consensus_queue.cc:1048] T 6efeed092ec840f2a7748c4a613dd6a7 P 96cba770f3334f79af3162475b54fa0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 37597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:24:45.434741 28444 raft_consensus.cc:1275] T d489ac98a4654a7cb46771880f89b89b P 96cba770f3334f79af3162475b54fa0f [term 2 FOLLOWER]: Refusing update from remote peer 542ea9e2fcff4028b6766459e90bc5fc: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:24:45.436278 28704 consensus_queue.cc:1048] T d489ac98a4654a7cb46771880f89b89b P 542ea9e2fcff4028b6766459e90bc5fc [LEADER]: Connected to new peer: Peer: permanent_uuid: "96cba770f3334f79af3162475b54fa0f" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 33585 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:24:45.440546 28372 raft_consensus.cc:1275] T 6efeed092ec840f2a7748c4a613dd6a7 P 542ea9e2fcff4028b6766459e90bc5fc [term 2 FOLLOWER]: Refusing update from remote peer 96cba770f3334f79af3162475b54fa0f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:24:45.442284 28700 consensus_queue.cc:1048] T 6efeed092ec840f2a7748c4a613dd6a7 P 96cba770f3334f79af3162475b54fa0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "542ea9e2fcff4028b6766459e90bc5fc" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38409 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:24:45.448140 28520 raft_consensus.cc:1275] T d489ac98a4654a7cb46771880f89b89b P 2173e3fd92f34ad6ad8d56d2e6b05299 [term 2 FOLLOWER]: Refusing update from remote peer 542ea9e2fcff4028b6766459e90bc5fc: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:24:45.449393 28707 consensus_queue.cc:1048] T d489ac98a4654a7cb46771880f89b89b P 542ea9e2fcff4028b6766459e90bc5fc [LEADER]: Connected to new peer: Peer: permanent_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 37597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:24:45.553093 28372 raft_consensus.cc:1275] T 07bb4101b0c24ee6b460e066cf78eb53 P 542ea9e2fcff4028b6766459e90bc5fc [term 2 FOLLOWER]: Refusing update from remote peer 96cba770f3334f79af3162475b54fa0f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:24:45.554333 28700 consensus_queue.cc:1048] T 07bb4101b0c24ee6b460e066cf78eb53 P 96cba770f3334f79af3162475b54fa0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "542ea9e2fcff4028b6766459e90bc5fc" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38409 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:24:45.563369 28520 raft_consensus.cc:1275] T 07bb4101b0c24ee6b460e066cf78eb53 P 2173e3fd92f34ad6ad8d56d2e6b05299 [term 2 FOLLOWER]: Refusing update from remote peer 96cba770f3334f79af3162475b54fa0f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:24:45.564983 28739 consensus_queue.cc:1048] T 07bb4101b0c24ee6b460e066cf78eb53 P 96cba770f3334f79af3162475b54fa0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 37597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:24:47.064071 28772 raft_consensus.cc:493] T 07bb4101b0c24ee6b460e066cf78eb53 P 542ea9e2fcff4028b6766459e90bc5fc [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 96cba770f3334f79af3162475b54fa0f)
I20260709 13:24:47.064802 28772 raft_consensus.cc:515] T 07bb4101b0c24ee6b460e066cf78eb53 P 542ea9e2fcff4028b6766459e90bc5fc [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "542ea9e2fcff4028b6766459e90bc5fc" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38409 } } peers { permanent_uuid: "96cba770f3334f79af3162475b54fa0f" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 33585 } } peers { permanent_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 37597 } }
I20260709 13:24:47.072793 28772 leader_election.cc:290] T 07bb4101b0c24ee6b460e066cf78eb53 P 542ea9e2fcff4028b6766459e90bc5fc [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 96cba770f3334f79af3162475b54fa0f (127.14.177.130:33585), 2173e3fd92f34ad6ad8d56d2e6b05299 (127.14.177.131:37597)
I20260709 13:24:47.074239 28445 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07bb4101b0c24ee6b460e066cf78eb53" candidate_uuid: "542ea9e2fcff4028b6766459e90bc5fc" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "96cba770f3334f79af3162475b54fa0f" is_pre_election: true
I20260709 13:24:47.127936 28521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07bb4101b0c24ee6b460e066cf78eb53" candidate_uuid: "542ea9e2fcff4028b6766459e90bc5fc" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" is_pre_election: true
I20260709 13:24:47.138047 28340 leader_election.cc:304] T 07bb4101b0c24ee6b460e066cf78eb53 P 542ea9e2fcff4028b6766459e90bc5fc [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 542ea9e2fcff4028b6766459e90bc5fc; no voters: 2173e3fd92f34ad6ad8d56d2e6b05299, 96cba770f3334f79af3162475b54fa0f
I20260709 13:24:47.139743 28619 raft_consensus.cc:2749] T 07bb4101b0c24ee6b460e066cf78eb53 P 542ea9e2fcff4028b6766459e90bc5fc [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:24:47.572757 28303 debug-util.cc:398] Leaking SignalData structure 0x7b0800031140 after lost signal to thread 15049
W20260709 13:24:47.574030 28303 debug-util.cc:398] Leaking SignalData structure 0x7b0800191e80 after lost signal to thread 28306
W20260709 13:24:47.575451 28303 debug-util.cc:398] Leaking SignalData structure 0x7b080034c3a0 after lost signal to thread 28398
W20260709 13:24:47.576555 28303 debug-util.cc:398] Leaking SignalData structure 0x7b08001bac80 after lost signal to thread 28473
W20260709 13:24:47.577930 28303 debug-util.cc:398] Leaking SignalData structure 0x7b080033dde0 after lost signal to thread 28547
W20260709 13:24:47.578464 28303 debug-util.cc:398] Leaking SignalData structure 0x7b08000b4400 after lost signal to thread 28739
I20260709 13:24:47.822926 28272 catalog_manager.cc:2533] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:39122:
table { table_name: "soft_deleted_table" } modify_external_catalogs: true reserve_seconds: 3600
I20260709 13:24:47.849959 28272 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:39122:
table { table_name: "soft_deleted_table" } new_extra_configs { key: "kudu.table.disable_compaction" value: "true" }
W20260709 13:24:48.068712 28340 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.14.177.130:33585, user_credentials={real_user=slave}} blocked reactor thread for 64011.9us
I20260709 13:24:48.062610 28822 tablet.cc:1724] T 2b05204beeff49309ed4e0dd43b4be7f P 542ea9e2fcff4028b6766459e90bc5fc: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:24:48.126540 28825 tablet.cc:1724] T cbb4f686d0884c178ea6b01616da20e8 P 542ea9e2fcff4028b6766459e90bc5fc: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:24:48.232896 15046 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:24:48.068751 28820 tablet.cc:1724] T cbb4f686d0884c178ea6b01616da20e8 P 2173e3fd92f34ad6ad8d56d2e6b05299: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:24:48.296849 28815 tablet.cc:1724] T fbb3d76b1491485287e0dc48b24dbb46 P 542ea9e2fcff4028b6766459e90bc5fc: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:24:48.248459 28830 tablet.cc:1724] T 2b05204beeff49309ed4e0dd43b4be7f P 96cba770f3334f79af3162475b54fa0f: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:24:48.304171 28834 tablet.cc:1724] T d489ac98a4654a7cb46771880f89b89b P 2173e3fd92f34ad6ad8d56d2e6b05299: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:24:48.307483 28838 tablet.cc:1724] T fbb3d76b1491485287e0dc48b24dbb46 P 96cba770f3334f79af3162475b54fa0f: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:24:48.307957 28812 tablet.cc:1724] T d5de0c782ce844ab8dbec004c32abafa P 96cba770f3334f79af3162475b54fa0f: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:24:48.341560 28822 tablet.cc:1724] T d489ac98a4654a7cb46771880f89b89b P 542ea9e2fcff4028b6766459e90bc5fc: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 13:24:48.387424 28831 log.cc:927] Time spent T 94c1c4dff8e8404dbb00c79a7b7c69c0 P 96cba770f3334f79af3162475b54fa0f: Append to log took a long time: real 0.077s	user 0.002s	sys 0.000s
W20260709 13:24:48.408083 28823 log.cc:927] Time spent T fbb3d76b1491485287e0dc48b24dbb46 P 2173e3fd92f34ad6ad8d56d2e6b05299: Append to log took a long time: real 0.104s	user 0.003s	sys 0.000s
I20260709 13:24:48.425581 28272 catalog_manager.cc:5789] TS 542ea9e2fcff4028b6766459e90bc5fc (127.14.177.129:38409) does not have the latest schema for tablet 94c1c4dff8e8404dbb00c79a7b7c69c0 (table soft_deleted_table [id=67a3bcc0b4a0443b913cf8eda79d4082]). Expected version 1 got 0
W20260709 13:24:48.428716 28837 log.cc:927] Time spent T 2b05204beeff49309ed4e0dd43b4be7f P 2173e3fd92f34ad6ad8d56d2e6b05299: Append to log took a long time: real 0.093s	user 0.002s	sys 0.000s
I20260709 13:24:48.434412 28272 catalog_manager.cc:5789] TS 542ea9e2fcff4028b6766459e90bc5fc (127.14.177.129:38409) does not have the latest schema for tablet e4c7e579c94a4f5fa195def741ee4180 (table soft_deleted_table [id=67a3bcc0b4a0443b913cf8eda79d4082]). Expected version 1 got 0
I20260709 13:24:48.446612 28820 tablet.cc:1724] T d5de0c782ce844ab8dbec004c32abafa P 2173e3fd92f34ad6ad8d56d2e6b05299: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:24:48.405325 28839 tablet.cc:1724] T d5de0c782ce844ab8dbec004c32abafa P 542ea9e2fcff4028b6766459e90bc5fc: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 13:24:48.467213 28829 log.cc:927] Time spent T d489ac98a4654a7cb46771880f89b89b P 96cba770f3334f79af3162475b54fa0f: Append to log took a long time: real 0.160s	user 0.003s	sys 0.000s
I20260709 13:24:48.469086 28850 tablet.cc:1724] T 6efeed092ec840f2a7748c4a613dd6a7 P 96cba770f3334f79af3162475b54fa0f: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:24:48.472034 28847 tablet.cc:1724] T e4c7e579c94a4f5fa195def741ee4180 P 2173e3fd92f34ad6ad8d56d2e6b05299: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:24:48.476388 28838 tablet.cc:1724] T 94c1c4dff8e8404dbb00c79a7b7c69c0 P 96cba770f3334f79af3162475b54fa0f: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 13:24:48.483402 28844 log.cc:927] Time spent T 07bb4101b0c24ee6b460e066cf78eb53 P 542ea9e2fcff4028b6766459e90bc5fc: Append to log took a long time: real 0.056s	user 0.000s	sys 0.001s
W20260709 13:24:48.484836 28845 log.cc:927] Time spent T 94c1c4dff8e8404dbb00c79a7b7c69c0 P 2173e3fd92f34ad6ad8d56d2e6b05299: Append to log took a long time: real 0.073s	user 0.001s	sys 0.000s
I20260709 13:24:48.493007 28839 tablet.cc:1724] T 6efeed092ec840f2a7748c4a613dd6a7 P 542ea9e2fcff4028b6766459e90bc5fc: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:24:48.504098 28820 tablet.cc:1724] T fbb3d76b1491485287e0dc48b24dbb46 P 2173e3fd92f34ad6ad8d56d2e6b05299: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:24:48.506140 28815 tablet.cc:1724] T e4c7e579c94a4f5fa195def741ee4180 P 542ea9e2fcff4028b6766459e90bc5fc: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:24:48.532868 28825 tablet.cc:1724] T 07bb4101b0c24ee6b460e066cf78eb53 P 542ea9e2fcff4028b6766459e90bc5fc: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:24:48.540668 28830 tablet.cc:1724] T 07bb4101b0c24ee6b460e066cf78eb53 P 96cba770f3334f79af3162475b54fa0f: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:24:48.513243 28838 tablet.cc:1724] T cbb4f686d0884c178ea6b01616da20e8 P 96cba770f3334f79af3162475b54fa0f: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:24:48.513800 28839 tablet.cc:1724] T 94c1c4dff8e8404dbb00c79a7b7c69c0 P 542ea9e2fcff4028b6766459e90bc5fc: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:24:48.521317 28374 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "68541149575b487b8c4e2360c2c93921"
dest_uuid: "542ea9e2fcff4028b6766459e90bc5fc"
mode: GRACEFUL
new_leader_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299"
 from {username='slave'} at 127.0.0.1:38314
W20260709 13:24:48.523125 28842 log.cc:927] Time spent T 6efeed092ec840f2a7748c4a613dd6a7 P 2173e3fd92f34ad6ad8d56d2e6b05299: Append to log took a long time: real 0.125s	user 0.002s	sys 0.000s
I20260709 13:24:48.524653 28850 tablet.cc:1724] T d489ac98a4654a7cb46771880f89b89b P 96cba770f3334f79af3162475b54fa0f: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:24:48.548856 28374 raft_consensus.cc:606] T 68541149575b487b8c4e2360c2c93921 P 542ea9e2fcff4028b6766459e90bc5fc [term 2 LEADER]: Received request to transfer leadership to TS 2173e3fd92f34ad6ad8d56d2e6b05299
I20260709 13:24:48.525923 28847 tablet.cc:1724] T 07bb4101b0c24ee6b460e066cf78eb53 P 2173e3fd92f34ad6ad8d56d2e6b05299: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:24:48.528704 28834 tablet.cc:1724] T 94c1c4dff8e8404dbb00c79a7b7c69c0 P 2173e3fd92f34ad6ad8d56d2e6b05299: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:24:48.530576 28269 catalog_manager.cc:5789] TS 96cba770f3334f79af3162475b54fa0f (127.14.177.130:33585) does not have the latest schema for tablet 07bb4101b0c24ee6b460e066cf78eb53 (table soft_deleted_table [id=67a3bcc0b4a0443b913cf8eda79d4082]). Expected version 1 got 0
I20260709 13:24:48.551964 28269 catalog_manager.cc:5789] TS 96cba770f3334f79af3162475b54fa0f (127.14.177.130:33585) does not have the latest schema for tablet 6efeed092ec840f2a7748c4a613dd6a7 (table soft_deleted_table [id=67a3bcc0b4a0443b913cf8eda79d4082]). Expected version 1 got 0
I20260709 13:24:48.530572 28812 tablet.cc:1724] T e4c7e579c94a4f5fa195def741ee4180 P 96cba770f3334f79af3162475b54fa0f: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:24:48.531839 28849 tablet.cc:1724] T 2b05204beeff49309ed4e0dd43b4be7f P 2173e3fd92f34ad6ad8d56d2e6b05299: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:24:48.569027 28820 tablet.cc:1724] T 6efeed092ec840f2a7748c4a613dd6a7 P 2173e3fd92f34ad6ad8d56d2e6b05299: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:24:48.612540 28443 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a6cf33518cd34784a1b85843d90badb5"
dest_uuid: "96cba770f3334f79af3162475b54fa0f"
mode: GRACEFUL
new_leader_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299"
 from {username='slave'} at 127.0.0.1:49000
I20260709 13:24:48.613109 28443 raft_consensus.cc:606] T a6cf33518cd34784a1b85843d90badb5 P 96cba770f3334f79af3162475b54fa0f [term 2 LEADER]: Received request to transfer leadership to TS 2173e3fd92f34ad6ad8d56d2e6b05299
I20260709 13:24:48.614461 15046 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 13:24:48.614979 15046 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:24:48.619817 28824 raft_consensus.cc:993] T 68541149575b487b8c4e2360c2c93921 P 542ea9e2fcff4028b6766459e90bc5fc: : Instructing follower 2173e3fd92f34ad6ad8d56d2e6b05299 to start an election
I20260709 13:24:48.620349 28824 raft_consensus.cc:1081] T 68541149575b487b8c4e2360c2c93921 P 542ea9e2fcff4028b6766459e90bc5fc [term 2 LEADER]: Signalling peer 2173e3fd92f34ad6ad8d56d2e6b05299 to start an election
I20260709 13:24:48.625411 28514 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "68541149575b487b8c4e2360c2c93921"
dest_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299"
 from {username='slave'} at 127.0.0.1:53704
I20260709 13:24:48.626102 28514 raft_consensus.cc:493] T 68541149575b487b8c4e2360c2c93921 P 2173e3fd92f34ad6ad8d56d2e6b05299 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:24:48.626505 28514 raft_consensus.cc:3060] T 68541149575b487b8c4e2360c2c93921 P 2173e3fd92f34ad6ad8d56d2e6b05299 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:24:48.637679 28514 raft_consensus.cc:515] T 68541149575b487b8c4e2360c2c93921 P 2173e3fd92f34ad6ad8d56d2e6b05299 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 37597 } } peers { permanent_uuid: "542ea9e2fcff4028b6766459e90bc5fc" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38409 } } peers { permanent_uuid: "96cba770f3334f79af3162475b54fa0f" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 33585 } }
I20260709 13:24:48.640949 28370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68541149575b487b8c4e2360c2c93921" candidate_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "542ea9e2fcff4028b6766459e90bc5fc"
I20260709 13:24:48.641654 28370 raft_consensus.cc:3055] T 68541149575b487b8c4e2360c2c93921 P 542ea9e2fcff4028b6766459e90bc5fc [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:24:48.641078 28514 leader_election.cc:290] T 68541149575b487b8c4e2360c2c93921 P 2173e3fd92f34ad6ad8d56d2e6b05299 [CANDIDATE]: Term 3 election: Requested vote from peers 542ea9e2fcff4028b6766459e90bc5fc (127.14.177.129:38409), 96cba770f3334f79af3162475b54fa0f (127.14.177.130:33585)
I20260709 13:24:48.642073 28370 raft_consensus.cc:740] T 68541149575b487b8c4e2360c2c93921 P 542ea9e2fcff4028b6766459e90bc5fc [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 542ea9e2fcff4028b6766459e90bc5fc, State: Running, Role: LEADER
I20260709 13:24:48.642140 28443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68541149575b487b8c4e2360c2c93921" candidate_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "96cba770f3334f79af3162475b54fa0f"
I20260709 13:24:48.642850 28443 raft_consensus.cc:3060] T 68541149575b487b8c4e2360c2c93921 P 96cba770f3334f79af3162475b54fa0f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:24:48.643106 28370 consensus_queue.cc:260] T 68541149575b487b8c4e2360c2c93921 P 542ea9e2fcff4028b6766459e90bc5fc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 37597 } } peers { permanent_uuid: "542ea9e2fcff4028b6766459e90bc5fc" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38409 } } peers { permanent_uuid: "96cba770f3334f79af3162475b54fa0f" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 33585 } }
I20260709 13:24:48.645545 28370 raft_consensus.cc:3060] T 68541149575b487b8c4e2360c2c93921 P 542ea9e2fcff4028b6766459e90bc5fc [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:24:48.650547 28443 raft_consensus.cc:2468] T 68541149575b487b8c4e2360c2c93921 P 96cba770f3334f79af3162475b54fa0f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2173e3fd92f34ad6ad8d56d2e6b05299 in term 3.
I20260709 13:24:48.651949 28489 leader_election.cc:304] T 68541149575b487b8c4e2360c2c93921 P 2173e3fd92f34ad6ad8d56d2e6b05299 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2173e3fd92f34ad6ad8d56d2e6b05299, 96cba770f3334f79af3162475b54fa0f; no voters: 
I20260709 13:24:48.652982 28819 raft_consensus.cc:2804] T 68541149575b487b8c4e2360c2c93921 P 2173e3fd92f34ad6ad8d56d2e6b05299 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:24:48.649010 28270 catalog_manager.cc:6242] T 00000000000000000000000000000000 P e26120d9496c4a7a9362997579e10fe2: soft_deleted_table [id=67a3bcc0b4a0443b913cf8eda79d4082] alter complete (version 1)
I20260709 13:24:48.662015 28370 raft_consensus.cc:2468] T 68541149575b487b8c4e2360c2c93921 P 542ea9e2fcff4028b6766459e90bc5fc [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2173e3fd92f34ad6ad8d56d2e6b05299 in term 3.
I20260709 13:24:48.670056 28777 raft_consensus.cc:993] T a6cf33518cd34784a1b85843d90badb5 P 96cba770f3334f79af3162475b54fa0f: : Instructing follower 2173e3fd92f34ad6ad8d56d2e6b05299 to start an election
I20260709 13:24:48.670666 28777 raft_consensus.cc:1081] T a6cf33518cd34784a1b85843d90badb5 P 96cba770f3334f79af3162475b54fa0f [term 2 LEADER]: Signalling peer 2173e3fd92f34ad6ad8d56d2e6b05299 to start an election
I20260709 13:24:48.676344 28514 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a6cf33518cd34784a1b85843d90badb5"
dest_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299"
 from {username='slave'} at 127.0.0.1:53714
I20260709 13:24:48.677054 28514 raft_consensus.cc:493] T a6cf33518cd34784a1b85843d90badb5 P 2173e3fd92f34ad6ad8d56d2e6b05299 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:24:48.677506 28514 raft_consensus.cc:3060] T a6cf33518cd34784a1b85843d90badb5 P 2173e3fd92f34ad6ad8d56d2e6b05299 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:24:48.686107 28514 raft_consensus.cc:515] T a6cf33518cd34784a1b85843d90badb5 P 2173e3fd92f34ad6ad8d56d2e6b05299 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "542ea9e2fcff4028b6766459e90bc5fc" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38409 } } peers { permanent_uuid: "96cba770f3334f79af3162475b54fa0f" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 33585 } } peers { permanent_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 37597 } }
I20260709 13:24:48.693430 28370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6cf33518cd34784a1b85843d90badb5" candidate_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "542ea9e2fcff4028b6766459e90bc5fc"
I20260709 13:24:48.693593 28514 leader_election.cc:290] T a6cf33518cd34784a1b85843d90badb5 P 2173e3fd92f34ad6ad8d56d2e6b05299 [CANDIDATE]: Term 3 election: Requested vote from peers 542ea9e2fcff4028b6766459e90bc5fc (127.14.177.129:38409), 96cba770f3334f79af3162475b54fa0f (127.14.177.130:33585)
I20260709 13:24:48.693713 28443 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6cf33518cd34784a1b85843d90badb5" candidate_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "96cba770f3334f79af3162475b54fa0f"
I20260709 13:24:48.694092 28370 raft_consensus.cc:3060] T a6cf33518cd34784a1b85843d90badb5 P 542ea9e2fcff4028b6766459e90bc5fc [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:24:48.694499 28443 raft_consensus.cc:3055] T a6cf33518cd34784a1b85843d90badb5 P 96cba770f3334f79af3162475b54fa0f [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:24:48.694943 28443 raft_consensus.cc:740] T a6cf33518cd34784a1b85843d90badb5 P 96cba770f3334f79af3162475b54fa0f [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 96cba770f3334f79af3162475b54fa0f, State: Running, Role: LEADER
I20260709 13:24:48.696063 28443 consensus_queue.cc:260] T a6cf33518cd34784a1b85843d90badb5 P 96cba770f3334f79af3162475b54fa0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "542ea9e2fcff4028b6766459e90bc5fc" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38409 } } peers { permanent_uuid: "96cba770f3334f79af3162475b54fa0f" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 33585 } } peers { permanent_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 37597 } }
I20260709 13:24:48.697744 28443 raft_consensus.cc:3060] T a6cf33518cd34784a1b85843d90badb5 P 96cba770f3334f79af3162475b54fa0f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:24:48.697593 28819 raft_consensus.cc:697] T 68541149575b487b8c4e2360c2c93921 P 2173e3fd92f34ad6ad8d56d2e6b05299 [term 3 LEADER]: Becoming Leader. State: Replica: 2173e3fd92f34ad6ad8d56d2e6b05299, State: Running, Role: LEADER
I20260709 13:24:48.702266 28819 consensus_queue.cc:237] T 68541149575b487b8c4e2360c2c93921 P 2173e3fd92f34ad6ad8d56d2e6b05299 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 37597 } } peers { permanent_uuid: "542ea9e2fcff4028b6766459e90bc5fc" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38409 } } peers { permanent_uuid: "96cba770f3334f79af3162475b54fa0f" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 33585 } }
I20260709 13:24:48.705785 28443 raft_consensus.cc:2468] T a6cf33518cd34784a1b85843d90badb5 P 96cba770f3334f79af3162475b54fa0f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2173e3fd92f34ad6ad8d56d2e6b05299 in term 3.
I20260709 13:24:48.716989 28489 leader_election.cc:304] T a6cf33518cd34784a1b85843d90badb5 P 2173e3fd92f34ad6ad8d56d2e6b05299 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2173e3fd92f34ad6ad8d56d2e6b05299, 96cba770f3334f79af3162475b54fa0f; no voters: 
I20260709 13:24:48.718601 28856 raft_consensus.cc:2804] T a6cf33518cd34784a1b85843d90badb5 P 2173e3fd92f34ad6ad8d56d2e6b05299 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:24:48.719276 28856 raft_consensus.cc:697] T a6cf33518cd34784a1b85843d90badb5 P 2173e3fd92f34ad6ad8d56d2e6b05299 [term 3 LEADER]: Becoming Leader. State: Replica: 2173e3fd92f34ad6ad8d56d2e6b05299, State: Running, Role: LEADER
I20260709 13:24:48.719558 28370 raft_consensus.cc:2468] T a6cf33518cd34784a1b85843d90badb5 P 542ea9e2fcff4028b6766459e90bc5fc [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2173e3fd92f34ad6ad8d56d2e6b05299 in term 3.
I20260709 13:24:48.720378 28856 consensus_queue.cc:237] T a6cf33518cd34784a1b85843d90badb5 P 2173e3fd92f34ad6ad8d56d2e6b05299 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "542ea9e2fcff4028b6766459e90bc5fc" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38409 } } peers { permanent_uuid: "96cba770f3334f79af3162475b54fa0f" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 33585 } } peers { permanent_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 37597 } }
I20260709 13:24:48.744771 28270 catalog_manager.cc:5697] T 68541149575b487b8c4e2360c2c93921 P 2173e3fd92f34ad6ad8d56d2e6b05299 reported cstate change: term changed from 2 to 3, leader changed from 542ea9e2fcff4028b6766459e90bc5fc (127.14.177.129) to 2173e3fd92f34ad6ad8d56d2e6b05299 (127.14.177.131). New cstate: current_term: 3 leader_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 37597 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "542ea9e2fcff4028b6766459e90bc5fc" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38409 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "96cba770f3334f79af3162475b54fa0f" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 33585 } health_report { overall_health: UNKNOWN } } }
I20260709 13:24:48.761337 28270 catalog_manager.cc:5697] T a6cf33518cd34784a1b85843d90badb5 P 2173e3fd92f34ad6ad8d56d2e6b05299 reported cstate change: term changed from 2 to 3, leader changed from 96cba770f3334f79af3162475b54fa0f (127.14.177.130) to 2173e3fd92f34ad6ad8d56d2e6b05299 (127.14.177.131). New cstate: current_term: 3 leader_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "542ea9e2fcff4028b6766459e90bc5fc" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38409 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "96cba770f3334f79af3162475b54fa0f" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 33585 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2173e3fd92f34ad6ad8d56d2e6b05299" member_type: VOTER last_known_addr { host: "127.14.177.131" port: 37597 } health_report { overall_health: HEALTHY } } }
I20260709 13:24:49.128753 28443 raft_consensus.cc:1275] T 68541149575b487b8c4e2360c2c93921 P 96cba770f3334f79af3162475b54fa0f [term 3 FOLLOWER]: Refusing update from remote peer 2173e3fd92f34ad6ad8d56d2e6b05299: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:24:49.130125 28856 consensus_queue.cc:1048] T 68541149575b487b8c4e2360c2c93921 P 2173e3fd92f34ad6ad8d56d2e6b05299 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96cba770f3334f79af3162475b54fa0f" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 33585 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:24:49.143514 28370 raft_consensus.cc:1275] T 68541149575b487b8c4e2360c2c93921 P 542ea9e2fcff4028b6766459e90bc5fc [term 3 FOLLOWER]: Refusing update from remote peer 2173e3fd92f34ad6ad8d56d2e6b05299: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:24:49.145128 28856 consensus_queue.cc:1048] T 68541149575b487b8c4e2360c2c93921 P 2173e3fd92f34ad6ad8d56d2e6b05299 [LEADER]: Connected to new peer: Peer: permanent_uuid: "542ea9e2fcff4028b6766459e90bc5fc" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38409 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:24:49.162745 28449 raft_consensus.cc:1275] T a6cf33518cd34784a1b85843d90badb5 P 96cba770f3334f79af3162475b54fa0f [term 3 FOLLOWER]: Refusing update from remote peer 2173e3fd92f34ad6ad8d56d2e6b05299: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:24:49.164372 28855 consensus_queue.cc:1048] T a6cf33518cd34784a1b85843d90badb5 P 2173e3fd92f34ad6ad8d56d2e6b05299 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96cba770f3334f79af3162475b54fa0f" member_type: VOTER last_known_addr { host: "127.14.177.130" port: 33585 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:24:49.180132 28374 raft_consensus.cc:1275] T a6cf33518cd34784a1b85843d90badb5 P 542ea9e2fcff4028b6766459e90bc5fc [term 3 FOLLOWER]: Refusing update from remote peer 2173e3fd92f34ad6ad8d56d2e6b05299: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:24:49.182127 28856 consensus_queue.cc:1048] T a6cf33518cd34784a1b85843d90badb5 P 2173e3fd92f34ad6ad8d56d2e6b05299 [LEADER]: Connected to new peer: Peer: permanent_uuid: "542ea9e2fcff4028b6766459e90bc5fc" member_type: VOTER last_known_addr { host: "127.14.177.129" port: 38409 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:24:49.694181 15046 tablet_server.cc:179] TabletServer@127.14.177.129:0 shutting down...
W20260709 13:24:49.717437 28487 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 9 similar messages]
W20260709 13:24:49.721379 28487 consensus_peers.cc:597] T 68541149575b487b8c4e2360c2c93921 P 2173e3fd92f34ad6ad8d56d2e6b05299 -> Peer 542ea9e2fcff4028b6766459e90bc5fc (127.14.177.129:38409): Couldn't send request to peer 542ea9e2fcff4028b6766459e90bc5fc. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:24:49.731356 28413 consensus_peers.cc:597] T 255f88c066ac422d9d8a582d9b24a958 P 96cba770f3334f79af3162475b54fa0f -> Peer 542ea9e2fcff4028b6766459e90bc5fc (127.14.177.129:38409): Couldn't send request to peer 542ea9e2fcff4028b6766459e90bc5fc. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:24:49.763444 15046 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:24:49.764384 15046 tablet_replica.cc:333] T 6efeed092ec840f2a7748c4a613dd6a7 P 542ea9e2fcff4028b6766459e90bc5fc: stopping tablet replica
I20260709 13:24:49.765105 15046 raft_consensus.cc:2243] T 6efeed092ec840f2a7748c4a613dd6a7 P 542ea9e2fcff4028b6766459e90bc5fc [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:24:49.765676 15046 raft_consensus.cc:2272] T 6efeed092ec840f2a7748c4a613dd6a7 P 542ea9e2fcff4028b6766459e90bc5fc [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:24:49.767980 15046 tablet_replica.cc:333] T d489ac98a4654a7cb46771880f89b89b P 542ea9e2fcff4028b6766459e90bc5fc: stopping tablet replica
I20260709 13:24:49.768662 15046 raft_consensus.cc:2243] T d489ac98a4654a7cb46771880f89b89b P 542ea9e2fcff4028b6766459e90bc5fc [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:24:49.769625 15046 raft_consensus.cc:2272] T d489ac98a4654a7cb46771880f89b89b P 542ea9e2fcff4028b6766459e90bc5fc [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:24:49.772018 15046 tablet_replica.cc:333] T cbb4f686d0884c178ea6b01616da20e8 P 542ea9e2fcff4028b6766459e90bc5fc: stopping tablet replica
I20260709 13:24:49.772655 15046 raft_consensus.cc:2243] T cbb4f686d0884c178ea6b01616da20e8 P 542ea9e2fcff4028b6766459e90bc5fc [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:24:49.773141 15046 raft_consensus.cc:2272] T cbb4f686d0884c178ea6b01616da20e8 P 542ea9e2fcff4028b6766459e90bc5fc [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:24:49.775157 15046 tablet_replica.cc:333] T 2b05204beeff49309ed4e0dd43b4be7f P 542ea9e2fcff4028b6766459e90bc5fc: stopping tablet replica
I20260709 13:24:49.775832 15046 raft_consensus.cc:2243] T 2b05204beeff49309ed4e0dd43b4be7f P 542ea9e2fcff4028b6766459e90bc5fc [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:24:49.776301 15046 raft_consensus.cc:2272] T 2b05204beeff49309ed4e0dd43b4be7f P 542ea9e2fcff4028b6766459e90bc5fc [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:24:49.778618 15046 tablet_replica.cc:333] T 07bb4101b0c24ee6b460e066cf78eb53 P 542ea9e2fcff4028b6766459e90bc5fc: stopping tablet replica
I20260709 13:24:49.779477 15046 raft_consensus.cc:2243] T 07bb4101b0c24ee6b460e066cf78eb53 P 542ea9e2fcff4028b6766459e90bc5fc [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:24:49.780022 15046 raft_consensus.cc:2272] T 07bb4101b0c24ee6b460e066cf78eb53 P 542ea9e2fcff4028b6766459e90bc5fc [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:24:49.782825 15046 tablet_replica.cc:333] T e4c7e579c94a4f5fa195def741ee4180 P 542ea9e2fcff4028b6766459e90bc5fc: stopping tablet replica
W20260709 13:24:49.783154 28413 consensus_peers.cc:597] T 6efeed092ec840f2a7748c4a613dd6a7 P 96cba770f3334f79af3162475b54fa0f -> Peer 542ea9e2fcff4028b6766459e90bc5fc (127.14.177.129:38409): Couldn't send request to peer 542ea9e2fcff4028b6766459e90bc5fc. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:24:49.783804 15046 raft_consensus.cc:2243] T e4c7e579c94a4f5fa195def741ee4180 P 542ea9e2fcff4028b6766459e90bc5fc [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:24:49.784998 15046 raft_consensus.cc:2272] T e4c7e579c94a4f5fa195def741ee4180 P 542ea9e2fcff4028b6766459e90bc5fc [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:24:49.787118 15046 tablet_replica.cc:333] T 94c1c4dff8e8404dbb00c79a7b7c69c0 P 542ea9e2fcff4028b6766459e90bc5fc: stopping tablet replica
I20260709 13:24:49.787799 15046 raft_consensus.cc:2243] T 94c1c4dff8e8404dbb00c79a7b7c69c0 P 542ea9e2fcff4028b6766459e90bc5fc [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:24:49.788612 15046 raft_consensus.cc:2272] T 94c1c4dff8e8404dbb00c79a7b7c69c0 P 542ea9e2fcff4028b6766459e90bc5fc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:24:49.790853 15046 tablet_replica.cc:333] T a6cf33518cd34784a1b85843d90badb5 P 542ea9e2fcff4028b6766459e90bc5fc: stopping tablet replica
I20260709 13:24:49.791790 15046 raft_consensus.cc:2243] T a6cf33518cd34784a1b85843d90badb5 P 542ea9e2fcff4028b6766459e90bc5fc [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:24:49.792389 15046 raft_consensus.cc:2272] T a6cf33518cd34784a1b85843d90badb5 P 542ea9e2fcff4028b6766459e90bc5fc [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:24:49.831594 15046 tablet_replica.cc:333] T fbb3d76b1491485287e0dc48b24dbb46 P 542ea9e2fcff4028b6766459e90bc5fc: stopping tablet replica
I20260709 13:24:49.835378 15046 raft_consensus.cc:2243] T fbb3d76b1491485287e0dc48b24dbb46 P 542ea9e2fcff4028b6766459e90bc5fc [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:24:49.838629 15046 raft_consensus.cc:2272] T fbb3d76b1491485287e0dc48b24dbb46 P 542ea9e2fcff4028b6766459e90bc5fc [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:24:49.843886 15046 tablet_replica.cc:333] T 41515f53db8540a8b4f52d9ac91dab50 P 542ea9e2fcff4028b6766459e90bc5fc: stopping tablet replica
I20260709 13:24:49.845880 15046 raft_consensus.cc:2243] T 41515f53db8540a8b4f52d9ac91dab50 P 542ea9e2fcff4028b6766459e90bc5fc [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:24:49.848215 15046 raft_consensus.cc:2272] T 41515f53db8540a8b4f52d9ac91dab50 P 542ea9e2fcff4028b6766459e90bc5fc [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:24:49.854662 15046 tablet_replica.cc:333] T 255f88c066ac422d9d8a582d9b24a958 P 542ea9e2fcff4028b6766459e90bc5fc: stopping tablet replica
I20260709 13:24:49.855917 15046 raft_consensus.cc:2243] T 255f88c066ac422d9d8a582d9b24a958 P 542ea9e2fcff4028b6766459e90bc5fc [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:24:49.856981 15046 raft_consensus.cc:2272] T 255f88c066ac422d9d8a582d9b24a958 P 542ea9e2fcff4028b6766459e90bc5fc [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:24:49.861210 15046 tablet_replica.cc:333] T 0701509f07184014b94c627c93b3bd59 P 542ea9e2fcff4028b6766459e90bc5fc: stopping tablet replica
I20260709 13:24:49.861922 15046 raft_consensus.cc:2243] T 0701509f07184014b94c627c93b3bd59 P 542ea9e2fcff4028b6766459e90bc5fc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:24:49.862704 15046 raft_consensus.cc:2272] T 0701509f07184014b94c627c93b3bd59 P 542ea9e2fcff4028b6766459e90bc5fc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:24:49.866430 15046 tablet_replica.cc:333] T d5de0c782ce844ab8dbec004c32abafa P 542ea9e2fcff4028b6766459e90bc5fc: stopping tablet replica
I20260709 13:24:49.867125 15046 raft_consensus.cc:2243] T d5de0c782ce844ab8dbec004c32abafa P 542ea9e2fcff4028b6766459e90bc5fc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:24:49.868116 15046 raft_consensus.cc:2272] T d5de0c782ce844ab8dbec004c32abafa P 542ea9e2fcff4028b6766459e90bc5fc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:24:49.872325 15046 tablet_replica.cc:333] T b75c7d0f42e6499f936d08ff343fca5b P 542ea9e2fcff4028b6766459e90bc5fc: stopping tablet replica
I20260709 13:24:49.873241 15046 raft_consensus.cc:2243] T b75c7d0f42e6499f936d08ff343fca5b P 542ea9e2fcff4028b6766459e90bc5fc [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:24:49.874032 15046 raft_consensus.cc:2272] T b75c7d0f42e6499f936d08ff343fca5b P 542ea9e2fcff4028b6766459e90bc5fc [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:24:49.878343 15046 tablet_replica.cc:333] T bbaf4eecd3c241e3aabff37f6ba0256a P 542ea9e2fcff4028b6766459e90bc5fc: stopping tablet replica
I20260709 13:24:49.879455 15046 raft_consensus.cc:2243] T bbaf4eecd3c241e3aabff37f6ba0256a P 542ea9e2fcff4028b6766459e90bc5fc [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:24:49.880595 15046 raft_consensus.cc:2272] T bbaf4eecd3c241e3aabff37f6ba0256a P 542ea9e2fcff4028b6766459e90bc5fc [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:24:49.884624 15046 tablet_replica.cc:333] T 037bfc9d92244fcb872b2ca382c21794 P 542ea9e2fcff4028b6766459e90bc5fc: stopping tablet replica
I20260709 13:24:49.885231 15046 raft_consensus.cc:2243] T 037bfc9d92244fcb872b2ca382c21794 P 542ea9e2fcff4028b6766459e90bc5fc [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:24:49.887416 15046 raft_consensus.cc:2272] T 037bfc9d92244fcb872b2ca382c21794 P 542ea9e2fcff4028b6766459e90bc5fc [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:24:49.891076 15046 tablet_replica.cc:333] T 73a4db7e3a924c6f99b416e6dda95c51 P 542ea9e2fcff4028b6766459e90bc5fc: stopping tablet replica
I20260709 13:24:49.891772 15046 raft_consensus.cc:2243] T 73a4db7e3a924c6f99b416e6dda95c51 P 542ea9e2fcff4028b6766459e90bc5fc [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:24:49.893793 15046 raft_consensus.cc:2272] T 73a4db7e3a924c6f99b416e6dda95c51 P 542ea9e2fcff4028b6766459e90bc5fc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:24:49.897423 15046 tablet_replica.cc:333] T 68541149575b487b8c4e2360c2c93921 P 542ea9e2fcff4028b6766459e90bc5fc: stopping tablet replica
I20260709 13:24:49.898005 15046 raft_consensus.cc:2243] T 68541149575b487b8c4e2360c2c93921 P 542ea9e2fcff4028b6766459e90bc5fc [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:24:49.898855 15046 raft_consensus.cc:2272] T 68541149575b487b8c4e2360c2c93921 P 542ea9e2fcff4028b6766459e90bc5fc [term 3 FOLLOWER]: Raft consensus is shut down!
W20260709 13:24:49.943817 28413 consensus_peers.cc:597] T 2b05204beeff49309ed4e0dd43b4be7f P 96cba770f3334f79af3162475b54fa0f -> Peer 542ea9e2fcff4028b6766459e90bc5fc (127.14.177.129:38409): Couldn't send request to peer 542ea9e2fcff4028b6766459e90bc5fc. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:24:49.959952 28487 consensus_peers.cc:597] T b75c7d0f42e6499f936d08ff343fca5b P 2173e3fd92f34ad6ad8d56d2e6b05299 -> Peer 542ea9e2fcff4028b6766459e90bc5fc (127.14.177.129:38409): Couldn't send request to peer 542ea9e2fcff4028b6766459e90bc5fc. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:38409: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:24:49.970155 15046 tablet_server.cc:196] TabletServer@127.14.177.129:0 shutdown complete.
W20260709 13:24:50.030323 28413 consensus_peers.cc:597] T 07bb4101b0c24ee6b460e066cf78eb53 P 96cba770f3334f79af3162475b54fa0f -> Peer 542ea9e2fcff4028b6766459e90bc5fc (127.14.177.129:38409): Couldn't send request to peer 542ea9e2fcff4028b6766459e90bc5fc. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:38409: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:24:50.031090 28413 consensus_peers.cc:597] T 0701509f07184014b94c627c93b3bd59 P 96cba770f3334f79af3162475b54fa0f -> Peer 542ea9e2fcff4028b6766459e90bc5fc (127.14.177.129:38409): Couldn't send request to peer 542ea9e2fcff4028b6766459e90bc5fc. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:38409: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:24:50.048906 28413 consensus_peers.cc:597] T bbaf4eecd3c241e3aabff37f6ba0256a P 96cba770f3334f79af3162475b54fa0f -> Peer 542ea9e2fcff4028b6766459e90bc5fc (127.14.177.129:38409): Couldn't send request to peer 542ea9e2fcff4028b6766459e90bc5fc. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:38409: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:24:50.059595 15046 tablet_server.cc:179] TabletServer@127.14.177.130:0 shutting down...
W20260709 13:24:50.092860 28489 consensus_peers.cc:597] T 68541149575b487b8c4e2360c2c93921 P 2173e3fd92f34ad6ad8d56d2e6b05299 -> Peer 96cba770f3334f79af3162475b54fa0f (127.14.177.130:33585): Couldn't send request to peer 96cba770f3334f79af3162475b54fa0f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:24:50.112023 15046 ts_tablet_manager.cc:1507] Shutting down tablet manager...
W20260709 13:24:50.112283 28487 consensus_peers.cc:597] T cbb4f686d0884c178ea6b01616da20e8 P 2173e3fd92f34ad6ad8d56d2e6b05299 -> Peer 542ea9e2fcff4028b6766459e90bc5fc (127.14.177.129:38409): Couldn't send request to peer 542ea9e2fcff4028b6766459e90bc5fc. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:38409: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:24:50.113410 15046 tablet_replica.cc:333] T cbb4f686d0884c178ea6b01616da20e8 P 96cba770f3334f79af3162475b54fa0f: stopping tablet replica
I20260709 13:24:50.114826 15046 raft_consensus.cc:2243] T cbb4f686d0884c178ea6b01616da20e8 P 96cba770f3334f79af3162475b54fa0f [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:24:50.115924 15046 raft_consensus.cc:2272] T cbb4f686d0884c178ea6b01616da20e8 P 96cba770f3334f79af3162475b54fa0f [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:24:50.116521 28413 consensus_peers.cc:597] T d5de0c782ce844ab8dbec004c32abafa P 96cba770f3334f79af3162475b54fa0f -> Peer 542ea9e2fcff4028b6766459e90bc5fc (127.14.177.129:38409): Couldn't send request to peer 542ea9e2fcff4028b6766459e90bc5fc. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:38409: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:24:50.119302 15046 tablet_replica.cc:333] T 2b05204beeff49309ed4e0dd43b4be7f P 96cba770f3334f79af3162475b54fa0f: stopping tablet replica
I20260709 13:24:50.120129 15046 raft_consensus.cc:2243] T 2b05204beeff49309ed4e0dd43b4be7f P 96cba770f3334f79af3162475b54fa0f [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:24:50.121622 15046 raft_consensus.cc:2272] T 2b05204beeff49309ed4e0dd43b4be7f P 96cba770f3334f79af3162475b54fa0f [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:24:50.125169 15046 tablet_replica.cc:333] T 07bb4101b0c24ee6b460e066cf78eb53 P 96cba770f3334f79af3162475b54fa0f: stopping tablet replica
W20260709 13:24:50.125783 28489 consensus_peers.cc:597] T a6cf33518cd34784a1b85843d90badb5 P 2173e3fd92f34ad6ad8d56d2e6b05299 -> Peer 96cba770f3334f79af3162475b54fa0f (127.14.177.130:33585): Couldn't send request to peer 96cba770f3334f79af3162475b54fa0f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:24:50.126233 15046 raft_consensus.cc:2243] T 07bb4101b0c24ee6b460e066cf78eb53 P 96cba770f3334f79af3162475b54fa0f [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:24:50.127547 15046 raft_consensus.cc:2272] T 07bb4101b0c24ee6b460e066cf78eb53 P 96cba770f3334f79af3162475b54fa0f [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:24:50.130507 15046 tablet_replica.cc:333] T e4c7e579c94a4f5fa195def741ee4180 P 96cba770f3334f79af3162475b54fa0f: stopping tablet replica
I20260709 13:24:50.131294 15046 raft_consensus.cc:2243] T e4c7e579c94a4f5fa195def741ee4180 P 96cba770f3334f79af3162475b54fa0f [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:24:50.132100 15046 raft_consensus.cc:2272] T e4c7e579c94a4f5fa195def741ee4180 P 96cba770f3334f79af3162475b54fa0f [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:24:50.134903 15046 tablet_replica.cc:333] T 6efeed092ec840f2a7748c4a613dd6a7 P 96cba770f3334f79af3162475b54fa0f: stopping tablet replica
I20260709 13:24:50.135807 15046 raft_consensus.cc:2243] T 6efeed092ec840f2a7748c4a613dd6a7 P 96cba770f3334f79af3162475b54fa0f [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:24:50.137848 15046 raft_consensus.cc:2272] T 6efeed092ec840f2a7748c4a613dd6a7 P 96cba770f3334f79af3162475b54fa0f [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:24:50.142297 15046 tablet_replica.cc:333] T d489ac98a4654a7cb46771880f89b89b P 96cba770f3334f79af3162475b54fa0f: stopping tablet replica
I20260709 13:24:50.143337 15046 raft_consensus.cc:2243] T d489ac98a4654a7cb46771880f89b89b P 96cba770f3334f79af3162475b54fa0f [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:24:50.144308 15046 raft_consensus.cc:2272] T d489ac98a4654a7cb46771880f89b89b P 96cba770f3334f79af3162475b54fa0f [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:24:50.145759 28487 consensus_peers.cc:597] T a6cf33518cd34784a1b85843d90badb5 P 2173e3fd92f34ad6ad8d56d2e6b05299 -> Peer 542ea9e2fcff4028b6766459e90bc5fc (127.14.177.129:38409): Couldn't send request to peer 542ea9e2fcff4028b6766459e90bc5fc. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.129:38409: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:24:50.147584 15046 tablet_replica.cc:333] T 94c1c4dff8e8404dbb00c79a7b7c69c0 P 96cba770f3334f79af3162475b54fa0f: stopping tablet replica
I20260709 13:24:50.148370 15046 raft_consensus.cc:2243] T 94c1c4dff8e8404dbb00c79a7b7c69c0 P 96cba770f3334f79af3162475b54fa0f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:24:50.149071 15046 raft_consensus.cc:2272] T 94c1c4dff8e8404dbb00c79a7b7c69c0 P 96cba770f3334f79af3162475b54fa0f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:24:50.151960 15046 tablet_replica.cc:333] T 68541149575b487b8c4e2360c2c93921 P 96cba770f3334f79af3162475b54fa0f: stopping tablet replica
I20260709 13:24:50.152575 15046 raft_consensus.cc:2243] T 68541149575b487b8c4e2360c2c93921 P 96cba770f3334f79af3162475b54fa0f [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:24:50.153258 15046 raft_consensus.cc:2272] T 68541149575b487b8c4e2360c2c93921 P 96cba770f3334f79af3162475b54fa0f [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:24:50.156847 15046 tablet_replica.cc:333] T 0701509f07184014b94c627c93b3bd59 P 96cba770f3334f79af3162475b54fa0f: stopping tablet replica
I20260709 13:24:50.157488 15046 raft_consensus.cc:2243] T 0701509f07184014b94c627c93b3bd59 P 96cba770f3334f79af3162475b54fa0f [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:24:50.158603 15046 raft_consensus.cc:2272] T 0701509f07184014b94c627c93b3bd59 P 96cba770f3334f79af3162475b54fa0f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:24:50.162133 15046 tablet_replica.cc:333] T 255f88c066ac422d9d8a582d9b24a958 P 96cba770f3334f79af3162475b54fa0f: stopping tablet replica
I20260709 13:24:50.162942 15046 raft_consensus.cc:2243] T 255f88c066ac422d9d8a582d9b24a958 P 96cba770f3334f79af3162475b54fa0f [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:24:50.164718 15046 raft_consensus.cc:2272] T 255f88c066ac422d9d8a582d9b24a958 P 96cba770f3334f79af3162475b54fa0f [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:24:50.168514 15046 tablet_replica.cc:333] T 41515f53db8540a8b4f52d9ac91dab50 P 96cba770f3334f79af3162475b54fa0f: stopping tablet replica
I20260709 13:24:50.169478 15046 raft_consensus.cc:2243] T 41515f53db8540a8b4f52d9ac91dab50 P 96cba770f3334f79af3162475b54fa0f [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:24:50.170187 15046 raft_consensus.cc:2272] T 41515f53db8540a8b4f52d9ac91dab50 P 96cba770f3334f79af3162475b54fa0f [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:24:50.173674 15046 tablet_replica.cc:333] T 73a4db7e3a924c6f99b416e6dda95c51 P 96cba770f3334f79af3162475b54fa0f: stopping tablet replica
I20260709 13:24:50.174300 15046 raft_consensus.cc:2243] T 73a4db7e3a924c6f99b416e6dda95c51 P 96cba770f3334f79af3162475b54fa0f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:24:50.175005 15046 raft_consensus.cc:2272] T 73a4db7e3a924c6f99b416e6dda95c51 P 96cba770f3334f79af3162475b54fa0f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:24:50.178040 15046 tablet_replica.cc:333] T 037bfc9d92244fcb872b2ca382c21794 P 96cba770f3334f79af3162475b54fa0f: stopping tablet replica
I20260709 13:24:50.178807 15046 raft_consensus.cc:2243] T 037bfc9d92244fcb872b2ca382c21794 P 96cba770f3334f79af3162475b54fa0f [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:24:50.179566 15046 raft_consensus.cc:2272] T 037bfc9d92244fcb872b2ca382c21794 P 96cba770f3334f79af3162475b54fa0f [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:24:50.182210 15046 tablet_replica.cc:333] T bbaf4eecd3c241e3aabff37f6ba0256a P 96cba770f3334f79af3162475b54fa0f: stopping tablet replica
I20260709 13:24:50.183143 15046 raft_consensus.cc:2243] T bbaf4eecd3c241e3aabff37f6ba0256a P 96cba770f3334f79af3162475b54fa0f [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:24:50.184494 15046 raft_consensus.cc:2272] T bbaf4eecd3c241e3aabff37f6ba0256a P 96cba770f3334f79af3162475b54fa0f [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:24:50.186932 15046 tablet_replica.cc:333] T d5de0c782ce844ab8dbec004c32abafa P 96cba770f3334f79af3162475b54fa0f: stopping tablet replica
I20260709 13:24:50.187532 15046 raft_consensus.cc:2243] T d5de0c782ce844ab8dbec004c32abafa P 96cba770f3334f79af3162475b54fa0f [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:24:50.188635 15046 raft_consensus.cc:2272] T d5de0c782ce844ab8dbec004c32abafa P 96cba770f3334f79af3162475b54fa0f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:24:50.191332 15046 tablet_replica.cc:333] T b75c7d0f42e6499f936d08ff343fca5b P 96cba770f3334f79af3162475b54fa0f: stopping tablet replica
I20260709 13:24:50.191917 15046 raft_consensus.cc:2243] T b75c7d0f42e6499f936d08ff343fca5b P 96cba770f3334f79af3162475b54fa0f [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:24:50.192507 15046 raft_consensus.cc:2272] T b75c7d0f42e6499f936d08ff343fca5b P 96cba770f3334f79af3162475b54fa0f [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:24:50.195286 15046 tablet_replica.cc:333] T a6cf33518cd34784a1b85843d90badb5 P 96cba770f3334f79af3162475b54fa0f: stopping tablet replica
I20260709 13:24:50.195811 15046 raft_consensus.cc:2243] T a6cf33518cd34784a1b85843d90badb5 P 96cba770f3334f79af3162475b54fa0f [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:24:50.196434 15046 raft_consensus.cc:2272] T a6cf33518cd34784a1b85843d90badb5 P 96cba770f3334f79af3162475b54fa0f [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:24:50.199077 15046 tablet_replica.cc:333] T fbb3d76b1491485287e0dc48b24dbb46 P 96cba770f3334f79af3162475b54fa0f: stopping tablet replica
I20260709 13:24:50.199625 15046 raft_consensus.cc:2243] T fbb3d76b1491485287e0dc48b24dbb46 P 96cba770f3334f79af3162475b54fa0f [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:24:50.200284 15046 raft_consensus.cc:2272] T fbb3d76b1491485287e0dc48b24dbb46 P 96cba770f3334f79af3162475b54fa0f [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:24:50.228322 28489 consensus_peers.cc:597] T b75c7d0f42e6499f936d08ff343fca5b P 2173e3fd92f34ad6ad8d56d2e6b05299 -> Peer 96cba770f3334f79af3162475b54fa0f (127.14.177.130:33585): Couldn't send request to peer 96cba770f3334f79af3162475b54fa0f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:24:50.253142 15046 tablet_server.cc:196] TabletServer@127.14.177.130:0 shutdown complete.
W20260709 13:24:50.297256 28489 consensus_peers.cc:597] T cbb4f686d0884c178ea6b01616da20e8 P 2173e3fd92f34ad6ad8d56d2e6b05299 -> Peer 96cba770f3334f79af3162475b54fa0f (127.14.177.130:33585): Couldn't send request to peer 96cba770f3334f79af3162475b54fa0f. Status: Network error: Client connection negotiation failed: client connection to 127.14.177.130:33585: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:24:50.306341 15046 tablet_server.cc:179] TabletServer@127.14.177.131:0 shutting down...
I20260709 13:24:50.351099 15046 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:24:50.352198 15046 tablet_replica.cc:333] T 94c1c4dff8e8404dbb00c79a7b7c69c0 P 2173e3fd92f34ad6ad8d56d2e6b05299: stopping tablet replica
I20260709 13:24:50.353080 15046 raft_consensus.cc:2243] T 94c1c4dff8e8404dbb00c79a7b7c69c0 P 2173e3fd92f34ad6ad8d56d2e6b05299 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:24:50.353765 15046 raft_consensus.cc:2272] T 94c1c4dff8e8404dbb00c79a7b7c69c0 P 2173e3fd92f34ad6ad8d56d2e6b05299 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:24:50.356573 15046 tablet_replica.cc:333] T cbb4f686d0884c178ea6b01616da20e8 P 2173e3fd92f34ad6ad8d56d2e6b05299: stopping tablet replica
I20260709 13:24:50.357393 15046 raft_consensus.cc:2243] T cbb4f686d0884c178ea6b01616da20e8 P 2173e3fd92f34ad6ad8d56d2e6b05299 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:24:50.358413 15046 raft_consensus.cc:2272] T cbb4f686d0884c178ea6b01616da20e8 P 2173e3fd92f34ad6ad8d56d2e6b05299 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:24:50.361155 15046 tablet_replica.cc:333] T 2b05204beeff49309ed4e0dd43b4be7f P 2173e3fd92f34ad6ad8d56d2e6b05299: stopping tablet replica
I20260709 13:24:50.361864 15046 raft_consensus.cc:2243] T 2b05204beeff49309ed4e0dd43b4be7f P 2173e3fd92f34ad6ad8d56d2e6b05299 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:24:50.362465 15046 raft_consensus.cc:2272] T 2b05204beeff49309ed4e0dd43b4be7f P 2173e3fd92f34ad6ad8d56d2e6b05299 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:24:50.365170 15046 tablet_replica.cc:333] T 6efeed092ec840f2a7748c4a613dd6a7 P 2173e3fd92f34ad6ad8d56d2e6b05299: stopping tablet replica
I20260709 13:24:50.365903 15046 raft_consensus.cc:2243] T 6efeed092ec840f2a7748c4a613dd6a7 P 2173e3fd92f34ad6ad8d56d2e6b05299 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:24:50.366518 15046 raft_consensus.cc:2272] T 6efeed092ec840f2a7748c4a613dd6a7 P 2173e3fd92f34ad6ad8d56d2e6b05299 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:24:50.369251 15046 tablet_replica.cc:333] T d489ac98a4654a7cb46771880f89b89b P 2173e3fd92f34ad6ad8d56d2e6b05299: stopping tablet replica
I20260709 13:24:50.369887 15046 raft_consensus.cc:2243] T d489ac98a4654a7cb46771880f89b89b P 2173e3fd92f34ad6ad8d56d2e6b05299 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:24:50.370543 15046 raft_consensus.cc:2272] T d489ac98a4654a7cb46771880f89b89b P 2173e3fd92f34ad6ad8d56d2e6b05299 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:24:50.373536 15046 tablet_replica.cc:333] T 07bb4101b0c24ee6b460e066cf78eb53 P 2173e3fd92f34ad6ad8d56d2e6b05299: stopping tablet replica
I20260709 13:24:50.374536 15046 raft_consensus.cc:2243] T 07bb4101b0c24ee6b460e066cf78eb53 P 2173e3fd92f34ad6ad8d56d2e6b05299 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:24:50.375381 15046 raft_consensus.cc:2272] T 07bb4101b0c24ee6b460e066cf78eb53 P 2173e3fd92f34ad6ad8d56d2e6b05299 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:24:50.378429 15046 tablet_replica.cc:333] T e4c7e579c94a4f5fa195def741ee4180 P 2173e3fd92f34ad6ad8d56d2e6b05299: stopping tablet replica
I20260709 13:24:50.379292 15046 raft_consensus.cc:2243] T e4c7e579c94a4f5fa195def741ee4180 P 2173e3fd92f34ad6ad8d56d2e6b05299 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:24:50.379916 15046 raft_consensus.cc:2272] T e4c7e579c94a4f5fa195def741ee4180 P 2173e3fd92f34ad6ad8d56d2e6b05299 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:24:50.382484 15046 tablet_replica.cc:333] T bbaf4eecd3c241e3aabff37f6ba0256a P 2173e3fd92f34ad6ad8d56d2e6b05299: stopping tablet replica
I20260709 13:24:50.383059 15046 raft_consensus.cc:2243] T bbaf4eecd3c241e3aabff37f6ba0256a P 2173e3fd92f34ad6ad8d56d2e6b05299 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:24:50.383694 15046 raft_consensus.cc:2272] T bbaf4eecd3c241e3aabff37f6ba0256a P 2173e3fd92f34ad6ad8d56d2e6b05299 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:24:50.386349 15046 tablet_replica.cc:333] T 037bfc9d92244fcb872b2ca382c21794 P 2173e3fd92f34ad6ad8d56d2e6b05299: stopping tablet replica
I20260709 13:24:50.386977 15046 raft_consensus.cc:2243] T 037bfc9d92244fcb872b2ca382c21794 P 2173e3fd92f34ad6ad8d56d2e6b05299 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:24:50.387665 15046 raft_consensus.cc:2272] T 037bfc9d92244fcb872b2ca382c21794 P 2173e3fd92f34ad6ad8d56d2e6b05299 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:24:50.390308 15046 tablet_replica.cc:333] T 73a4db7e3a924c6f99b416e6dda95c51 P 2173e3fd92f34ad6ad8d56d2e6b05299: stopping tablet replica
I20260709 13:24:50.390916 15046 raft_consensus.cc:2243] T 73a4db7e3a924c6f99b416e6dda95c51 P 2173e3fd92f34ad6ad8d56d2e6b05299 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:24:50.391511 15046 raft_consensus.cc:2272] T 73a4db7e3a924c6f99b416e6dda95c51 P 2173e3fd92f34ad6ad8d56d2e6b05299 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:24:50.394038 15046 tablet_replica.cc:333] T d5de0c782ce844ab8dbec004c32abafa P 2173e3fd92f34ad6ad8d56d2e6b05299: stopping tablet replica
I20260709 13:24:50.394622 15046 raft_consensus.cc:2243] T d5de0c782ce844ab8dbec004c32abafa P 2173e3fd92f34ad6ad8d56d2e6b05299 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:24:50.395328 15046 raft_consensus.cc:2272] T d5de0c782ce844ab8dbec004c32abafa P 2173e3fd92f34ad6ad8d56d2e6b05299 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:24:50.397877 15046 tablet_replica.cc:333] T b75c7d0f42e6499f936d08ff343fca5b P 2173e3fd92f34ad6ad8d56d2e6b05299: stopping tablet replica
I20260709 13:24:50.398427 15046 raft_consensus.cc:2243] T b75c7d0f42e6499f936d08ff343fca5b P 2173e3fd92f34ad6ad8d56d2e6b05299 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:24:50.399533 15046 raft_consensus.cc:2272] T b75c7d0f42e6499f936d08ff343fca5b P 2173e3fd92f34ad6ad8d56d2e6b05299 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:24:50.402025 15046 tablet_replica.cc:333] T 0701509f07184014b94c627c93b3bd59 P 2173e3fd92f34ad6ad8d56d2e6b05299: stopping tablet replica
I20260709 13:24:50.402667 15046 raft_consensus.cc:2243] T 0701509f07184014b94c627c93b3bd59 P 2173e3fd92f34ad6ad8d56d2e6b05299 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:24:50.403252 15046 raft_consensus.cc:2272] T 0701509f07184014b94c627c93b3bd59 P 2173e3fd92f34ad6ad8d56d2e6b05299 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:24:50.405893 15046 tablet_replica.cc:333] T 255f88c066ac422d9d8a582d9b24a958 P 2173e3fd92f34ad6ad8d56d2e6b05299: stopping tablet replica
I20260709 13:24:50.406515 15046 raft_consensus.cc:2243] T 255f88c066ac422d9d8a582d9b24a958 P 2173e3fd92f34ad6ad8d56d2e6b05299 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:24:50.407083 15046 raft_consensus.cc:2272] T 255f88c066ac422d9d8a582d9b24a958 P 2173e3fd92f34ad6ad8d56d2e6b05299 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:24:50.409582 15046 tablet_replica.cc:333] T 41515f53db8540a8b4f52d9ac91dab50 P 2173e3fd92f34ad6ad8d56d2e6b05299: stopping tablet replica
I20260709 13:24:50.410172 15046 raft_consensus.cc:2243] T 41515f53db8540a8b4f52d9ac91dab50 P 2173e3fd92f34ad6ad8d56d2e6b05299 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:24:50.410701 15046 raft_consensus.cc:2272] T 41515f53db8540a8b4f52d9ac91dab50 P 2173e3fd92f34ad6ad8d56d2e6b05299 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:24:50.413244 15046 tablet_replica.cc:333] T a6cf33518cd34784a1b85843d90badb5 P 2173e3fd92f34ad6ad8d56d2e6b05299: stopping tablet replica
I20260709 13:24:50.413851 15046 raft_consensus.cc:2243] T a6cf33518cd34784a1b85843d90badb5 P 2173e3fd92f34ad6ad8d56d2e6b05299 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:24:50.414822 15046 raft_consensus.cc:2272] T a6cf33518cd34784a1b85843d90badb5 P 2173e3fd92f34ad6ad8d56d2e6b05299 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:24:50.417449 15046 tablet_replica.cc:333] T fbb3d76b1491485287e0dc48b24dbb46 P 2173e3fd92f34ad6ad8d56d2e6b05299: stopping tablet replica
I20260709 13:24:50.418062 15046 raft_consensus.cc:2243] T fbb3d76b1491485287e0dc48b24dbb46 P 2173e3fd92f34ad6ad8d56d2e6b05299 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:24:50.418776 15046 raft_consensus.cc:2272] T fbb3d76b1491485287e0dc48b24dbb46 P 2173e3fd92f34ad6ad8d56d2e6b05299 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:24:50.421521 15046 tablet_replica.cc:333] T 68541149575b487b8c4e2360c2c93921 P 2173e3fd92f34ad6ad8d56d2e6b05299: stopping tablet replica
I20260709 13:24:50.422142 15046 raft_consensus.cc:2243] T 68541149575b487b8c4e2360c2c93921 P 2173e3fd92f34ad6ad8d56d2e6b05299 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:24:50.423245 15046 raft_consensus.cc:2272] T 68541149575b487b8c4e2360c2c93921 P 2173e3fd92f34ad6ad8d56d2e6b05299 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:24:50.495993 15046 tablet_server.cc:196] TabletServer@127.14.177.131:0 shutdown complete.
I20260709 13:24:50.544135 15046 master.cc:562] Master@127.14.177.190:43663 shutting down...
I20260709 13:24:50.575484 15046 raft_consensus.cc:2243] T 00000000000000000000000000000000 P e26120d9496c4a7a9362997579e10fe2 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:24:50.576190 15046 raft_consensus.cc:2272] T 00000000000000000000000000000000 P e26120d9496c4a7a9362997579e10fe2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:24:50.576574 15046 tablet_replica.cc:333] T 00000000000000000000000000000000 P e26120d9496c4a7a9362997579e10fe2: stopping tablet replica
I20260709 13:24:51.437613 15046 master.cc:584] Master@127.14.177.190:43663 shutdown complete.
[       OK ] FilterSoftDeletedTableTest.TestFilterSofteDeletedTable/1 (14314 ms)
[----------] 2 tests from FilterSoftDeletedTableTest (27511 ms total)

[----------] Global test environment tear-down
[==========] 8 tests from 2 test suites ran. (397532 ms total)
[  PASSED  ] 7 tests.
[  FAILED  ] 1 test, listed below:
[  FAILED  ] LeaderRebalancerTest.AddTserver

 1 FAILED TEST
I20260709 13:24:51.506305 15046 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tablet/tablet_mm_ops.cc:220: suppressed but not reported on 433 messages since previous log ~17 seconds ago
I20260709 13:24:51.506613 15046 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tablet/tablet_mm_ops.cc:304: suppressed but not reported on 432 messages since previous log ~17 seconds ago
I20260709 13:24:51.506839 15046 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tablet/tablet_mm_ops.cc:139: suppressed but not reported on 428 messages since previous log ~17 seconds ago
I20260709 13:24:51.507079 15046 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tserver/ts_tablet_manager.cc:600: suppressed but not reported on 9 messages since previous log ~38 seconds ago
I20260709 13:24:51.507555 15046 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tserver/ts_tablet_manager.cc:548: suppressed but not reported on 9 messages since previous log ~38 seconds ago
I20260709 13:24:51.507798 15046 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/master/catalog_manager.cc:5227: suppressed but not reported on 1 messages since previous log ~137 seconds ago
I20260709 13:24:51.508023 15046 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/consensus/raft_consensus.cc:670: suppressed but not reported on 2 messages since previous log ~159 seconds ago
I20260709 13:24:51.508268 15046 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tserver/heartbeater.cc:646: suppressed but not reported on 7 messages since previous log ~176 seconds ago
I20260709 13:24:51.508504 15046 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/rpc/proxy.cc:239: suppressed but not reported on 16 messages since previous log ~1 seconds ago
I20260709 13:24:51.508773 15046 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tablet/tablet.cc:2406: suppressed but not reported on 53 messages since previous log ~13 seconds ago
